2024-05-22T20:57:11.353 INFO:root:teuthology version: 0.0.1.dev287+gb9e3da8 2024-05-22T20:57:11.353 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} --archive /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718018 --verbose --name leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi --owner scheduled_leonidus@teuthology -- /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718018/orig.config.yaml 2024-05-22T20:57:11.397 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-22T20:57:11.483 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718018 branch: reef description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} email: null first_in_suite: false job_id: '7718018' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd - disabled conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 client.0: debug client: 20 debug objectcacher: 20 debug objecter: 20 client.1: debug client: 20 debug objectcacher: 20 debug objecter: 20 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon osd backfillfull ratio: 0.6 mon osd full ratio: 0.7 mon osd nearfull ratio: 0.6 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 memstore device bytes: 200000000 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 1.0 osd mon report interval: 5 osd objectstore: memstore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL - PG_RECOVERY_FULL - PG_DEGRADED sha1: b9b067bc6d30ec00e391f0fa43f50affe981ba2f ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: b9b067bc6d30ec00e391f0fa43f50affe981ba2f kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: 630900aeb5fd525af2a375cc2059645127fe426f owner: scheduled_leonidus@teuthology priority: 120 repo: https://github.com/ceph/ceph roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7924 sha1: b9b067bc6d30ec00e391f0fa43f50affe981ba2f sleep_before_teardown: 0 subset: 244/512 suite: fs suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_630900aeb5fd525af2a375cc2059645127fe426f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 630900aeb5fd525af2a375cc2059645127fe426f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLvjX/VGhFxeFt7GOEjv8tshM58p/iYZCunKdpy5b8LMU/Z/G9BR2HMNkA5Ref3VGU6RYfCQN2YGBgdeX1Mwh0s= smithi179.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBuQB+z1kAxwr4ct4kbsucuTKIMV1zjW8eZV/r1mYFjaAZq1c3sjL6gkyUUV8YXodK7j/6o8l/16kyRJg9862fs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_full teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a timestamp: 2024-05-21_10:46:42 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-22T20:57:11.483 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_630900aeb5fd525af2a375cc2059645127fe426f/qa; will attempt to use it 2024-05-22T20:57:11.484 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_630900aeb5fd525af2a375cc2059645127fe426f/qa/tasks 2024-05-22T20:57:11.484 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-22T20:57:11.485 INFO:teuthology.task.internal:Checking packages... 2024-05-22T20:57:11.506 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'b9b067bc6d30ec00e391f0fa43f50affe981ba2f' 2024-05-22T20:57:11.506 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-22T20:57:11.506 INFO:teuthology.packaging:ref: None 2024-05-22T20:57:11.506 INFO:teuthology.packaging:tag: None 2024-05-22T20:57:11.506 INFO:teuthology.packaging:branch: reef 2024-05-22T20:57:11.507 INFO:teuthology.packaging:sha1: b9b067bc6d30ec00e391f0fa43f50affe981ba2f 2024-05-22T20:57:11.507 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=reef 2024-05-22T20:57:11.840 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1353-g70bbb060-1jammy 2024-05-22T20:57:11.842 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-22T20:57:11.849 INFO:teuthology.task.internal:no buildpackages task found 2024-05-22T20:57:11.849 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-22T20:57:11.857 INFO:teuthology.task.internal:Saving configuration 2024-05-22T20:57:11.873 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-22T20:57:11.880 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-22T20:57:11.905 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718018', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-22 20:51:13.102582', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOyMZlzdKZvVxqh4f3jTvDmi+6Ppk0Bfll0AwcXSd0xB'} 2024-05-22T20:57:11.925 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718018', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-22 20:51:13.101171', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKBa4Tlz6QTOJNPATFU108hJw4dI8Klj09PD+pOD4YU6'} 2024-05-22T20:57:11.926 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-22T20:57:11.932 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-22T20:57:11.932 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-22T20:57:11.932 INFO:teuthology.run_tasks:Running task console_log... 2024-05-22T20:57:11.994 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc1170655a0>, signals=[15]) 2024-05-22T20:57:11.994 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-22T20:57:12.001 INFO:teuthology.task.internal:Opening connections... 2024-05-22T20:57:12.001 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T20:57:12.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T20:57:12.080 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T20:57:12.081 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T20:57:12.156 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-22T20:57:12.162 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-05-22T20:57:12.167 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-05-22T20:57:12.167 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-05-22T20:57:12.214 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-05-22T20:57:12.214 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Ubuntu" 2024-05-22T20:57:12.214 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="22.04" 2024-05-22T20:57:12.214 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-05-22T20:57:12.214 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_CODENAME=jammy 2024-05-22T20:57:12.214 INFO:teuthology.orchestra.run.smithi161.stdout:ID=ubuntu 2024-05-22T20:57:12.214 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE=debian 2024-05-22T20:57:12.214 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-22T20:57:12.214 INFO:teuthology.orchestra.run.smithi161.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-22T20:57:12.215 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-22T20:57:12.215 INFO:teuthology.orchestra.run.smithi161.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-22T20:57:12.215 INFO:teuthology.orchestra.run.smithi161.stdout:UBUNTU_CODENAME=jammy 2024-05-22T20:57:12.215 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-05-22T20:57:12.231 DEBUG:teuthology.orchestra.run.smithi179:> uname -m 2024-05-22T20:57:12.236 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2024-05-22T20:57:12.236 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release 2024-05-22T20:57:12.283 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-05-22T20:57:12.283 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="Ubuntu" 2024-05-22T20:57:12.283 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="22.04" 2024-05-22T20:57:12.283 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-05-22T20:57:12.283 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_CODENAME=jammy 2024-05-22T20:57:12.283 INFO:teuthology.orchestra.run.smithi179.stdout:ID=ubuntu 2024-05-22T20:57:12.283 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE=debian 2024-05-22T20:57:12.283 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-22T20:57:12.283 INFO:teuthology.orchestra.run.smithi179.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-22T20:57:12.283 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-22T20:57:12.283 INFO:teuthology.orchestra.run.smithi179.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-22T20:57:12.283 INFO:teuthology.orchestra.run.smithi179.stdout:UBUNTU_CODENAME=jammy 2024-05-22T20:57:12.284 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2024-05-22T20:57:12.306 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-22T20:57:12.315 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-22T20:57:12.321 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-22T20:57:12.321 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-05-22T20:57:12.324 DEBUG:teuthology.orchestra.run.smithi179:> test '!' -e /home/ubuntu/cephtest 2024-05-22T20:57:12.331 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-22T20:57:12.337 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-22T20:57:12.337 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-05-22T20:57:12.372 DEBUG:teuthology.orchestra.run.smithi179:> test -z $(ls -A /var/lib/ceph) 2024-05-22T20:57:12.443 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-22T20:57:12.497 INFO:teuthology.run_tasks:Running task kernel... 2024-05-22T20:57:12.512 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-05-22T20:57:12.512 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-05-22T20:57:12.512 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-05-22T20:57:12.512 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-22T20:57:12.513 DEBUG:teuthology.orchestra.run.smithi179:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-22T20:57:12.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T20:57:12.517 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-05-22T20:57:12.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T20:57:12.521 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-05-22T20:57:12.797 DEBUG:teuthology.task.kernel:sha1 for client.1 is 4fed890e8ce3966dbe380e9eb4b0a796eddb841c 2024-05-22T20:57:12.798 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "4fed890e8ce3966dbe380e9eb4b0a796eddb841c"... 2024-05-22T20:57:12.798 DEBUG:teuthology.orchestra.run.smithi179:> uname -r 2024-05-22T20:57:12.805 INFO:teuthology.orchestra.run.smithi179.stdout:5.15.0-56-generic 2024-05-22T20:57:12.805 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 4fed890e8ce3966dbe380e9eb4b0a796eddb841c 2024-05-22T20:57:12.806 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-22T20:57:12.806 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-05-22T20:57:12.806 DEBUG:teuthology.orchestra.run.smithi179:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-05-22T20:57:12.826 DEBUG:teuthology.task.kernel:sha1 for client.0 is 4fed890e8ce3966dbe380e9eb4b0a796eddb841c 2024-05-22T20:57:12.826 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4fed890e8ce3966dbe380e9eb4b0a796eddb841c"... 2024-05-22T20:57:12.826 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-05-22T20:57:12.832 INFO:teuthology.orchestra.run.smithi161.stdout:5.15.0-56-generic 2024-05-22T20:57:12.832 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 4fed890e8ce3966dbe380e9eb4b0a796eddb841c 2024-05-22T20:57:12.832 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-22T20:57:12.832 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-05-22T20:57:12.832 DEBUG:teuthology.orchestra.run.smithi161:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-05-22T20:57:12.859 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-05-22T20:57:12.892 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-05-22T20:57:13.011 INFO:teuthology.orchestra.run.smithi179.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-05-22T20:57:13.011 INFO:teuthology.orchestra.run.smithi179.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-05-22T20:57:13.011 INFO:teuthology.orchestra.run.smithi179.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-05-22T20:57:13.011 INFO:teuthology.orchestra.run.smithi179.stderr:hint: 2024-05-22T20:57:13.011 INFO:teuthology.orchestra.run.smithi179.stderr:hint: git config --global init.defaultBranch 2024-05-22T20:57:13.011 INFO:teuthology.orchestra.run.smithi179.stderr:hint: 2024-05-22T20:57:13.012 INFO:teuthology.orchestra.run.smithi179.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-05-22T20:57:13.012 INFO:teuthology.orchestra.run.smithi179.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-05-22T20:57:13.012 INFO:teuthology.orchestra.run.smithi179.stderr:hint: 2024-05-22T20:57:13.012 INFO:teuthology.orchestra.run.smithi179.stderr:hint: git branch -m 2024-05-22T20:57:13.012 INFO:teuthology.orchestra.run.smithi179.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-05-22T20:57:13.014 DEBUG:teuthology.orchestra.run.smithi179:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-05-22T20:57:13.035 INFO:teuthology.orchestra.run.smithi161.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-05-22T20:57:13.035 INFO:teuthology.orchestra.run.smithi161.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-05-22T20:57:13.035 INFO:teuthology.orchestra.run.smithi161.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-05-22T20:57:13.035 INFO:teuthology.orchestra.run.smithi161.stderr:hint: 2024-05-22T20:57:13.035 INFO:teuthology.orchestra.run.smithi161.stderr:hint: git config --global init.defaultBranch 2024-05-22T20:57:13.036 INFO:teuthology.orchestra.run.smithi161.stderr:hint: 2024-05-22T20:57:13.036 INFO:teuthology.orchestra.run.smithi161.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-05-22T20:57:13.036 INFO:teuthology.orchestra.run.smithi161.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-05-22T20:57:13.036 INFO:teuthology.orchestra.run.smithi161.stderr:hint: 2024-05-22T20:57:13.036 INFO:teuthology.orchestra.run.smithi161.stderr:hint: git branch -m 2024-05-22T20:57:13.037 INFO:teuthology.orchestra.run.smithi161.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-05-22T20:57:13.039 DEBUG:teuthology.orchestra.run.smithi161:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-05-22T20:57:13.085 DEBUG:teuthology.orchestra.run.smithi179:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-05-22T20:57:13.106 DEBUG:teuthology.orchestra.run.smithi161:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-05-22T20:57:13.143 DEBUG:teuthology.orchestra.run.smithi179:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-05-22T20:57:13.164 DEBUG:teuthology.orchestra.run.smithi161:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-05-22T20:59:00.855 INFO:teuthology.orchestra.run.smithi179.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-05-22T20:59:00.856 INFO:teuthology.orchestra.run.smithi179.stderr: * [new branch] main -> origin/main 2024-05-22T20:59:00.889 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20190312 -> 20190312 2024-05-22T20:59:00.889 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20190416 -> 20190416 2024-05-22T20:59:00.890 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20190514 -> 20190514 2024-05-22T20:59:00.890 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20190618 -> 20190618 2024-05-22T20:59:00.890 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20190717 -> 20190717 2024-05-22T20:59:00.891 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20190815 -> 20190815 2024-05-22T20:59:00.891 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20190923 -> 20190923 2024-05-22T20:59:00.891 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20191022 -> 20191022 2024-05-22T20:59:00.891 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20191215 -> 20191215 2024-05-22T20:59:00.891 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20200122 -> 20200122 2024-05-22T20:59:00.891 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20200316 -> 20200316 2024-05-22T20:59:00.892 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20200421 -> 20200421 2024-05-22T20:59:00.892 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20200519 -> 20200519 2024-05-22T20:59:00.892 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20200619 -> 20200619 2024-05-22T20:59:00.893 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20200721 -> 20200721 2024-05-22T20:59:00.893 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20200817 -> 20200817 2024-05-22T20:59:00.893 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20200918 -> 20200918 2024-05-22T20:59:00.894 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20201022 -> 20201022 2024-05-22T20:59:00.894 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20201118 -> 20201118 2024-05-22T20:59:00.894 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20201218 -> 20201218 2024-05-22T20:59:00.894 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20210208 -> 20210208 2024-05-22T20:59:00.894 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20210315 -> 20210315 2024-05-22T20:59:00.894 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20210511 -> 20210511 2024-05-22T20:59:00.894 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20210716 -> 20210716 2024-05-22T20:59:00.894 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20210818 -> 20210818 2024-05-22T20:59:00.894 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20210919 -> 20210919 2024-05-22T20:59:00.896 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20211027 -> 20211027 2024-05-22T20:59:00.896 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20211216 -> 20211216 2024-05-22T20:59:00.896 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20220209 -> 20220209 2024-05-22T20:59:00.896 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20220310 -> 20220310 2024-05-22T20:59:00.896 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20220411 -> 20220411 2024-05-22T20:59:00.896 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20220509 -> 20220509 2024-05-22T20:59:00.897 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20220610 -> 20220610 2024-05-22T20:59:00.897 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20220708 -> 20220708 2024-05-22T20:59:00.897 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20220815 -> 20220815 2024-05-22T20:59:00.897 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20220913 -> 20220913 2024-05-22T20:59:00.897 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20221012 -> 20221012 2024-05-22T20:59:00.897 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20221109 -> 20221109 2024-05-22T20:59:00.897 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20221214 -> 20221214 2024-05-22T20:59:00.897 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20230117 -> 20230117 2024-05-22T20:59:00.899 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20230210 -> 20230210 2024-05-22T20:59:00.899 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20230310 -> 20230310 2024-05-22T20:59:00.899 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20230404 -> 20230404 2024-05-22T20:59:00.899 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20230515 -> 20230515 2024-05-22T20:59:00.899 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20230625 -> 20230625 2024-05-22T20:59:00.899 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20230804 -> 20230804 2024-05-22T20:59:00.899 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20230919 -> 20230919 2024-05-22T20:59:00.900 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20231030 -> 20231030 2024-05-22T20:59:00.900 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20231111 -> 20231111 2024-05-22T20:59:00.900 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20231211 -> 20231211 2024-05-22T20:59:00.900 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20240115 -> 20240115 2024-05-22T20:59:00.900 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20240220 -> 20240220 2024-05-22T20:59:00.900 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20240312 -> 20240312 2024-05-22T20:59:00.900 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20240410 -> 20240410 2024-05-22T20:59:00.900 INFO:teuthology.orchestra.run.smithi179.stderr: * [new tag] 20240513 -> 20240513 2024-05-22T20:59:03.355 INFO:teuthology.orchestra.run.smithi161.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-05-22T20:59:03.355 INFO:teuthology.orchestra.run.smithi161.stderr: * [new branch] main -> origin/main 2024-05-22T20:59:03.381 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20190312 -> 20190312 2024-05-22T20:59:03.381 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20190416 -> 20190416 2024-05-22T20:59:03.381 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20190514 -> 20190514 2024-05-22T20:59:03.382 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20190618 -> 20190618 2024-05-22T20:59:03.382 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20190717 -> 20190717 2024-05-22T20:59:03.382 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20190815 -> 20190815 2024-05-22T20:59:03.382 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20190923 -> 20190923 2024-05-22T20:59:03.384 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20191022 -> 20191022 2024-05-22T20:59:03.384 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20191215 -> 20191215 2024-05-22T20:59:03.384 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20200122 -> 20200122 2024-05-22T20:59:03.384 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20200316 -> 20200316 2024-05-22T20:59:03.384 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20200421 -> 20200421 2024-05-22T20:59:03.384 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20200519 -> 20200519 2024-05-22T20:59:03.385 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20200619 -> 20200619 2024-05-22T20:59:03.385 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20200721 -> 20200721 2024-05-22T20:59:03.385 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20200817 -> 20200817 2024-05-22T20:59:03.385 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20200918 -> 20200918 2024-05-22T20:59:03.385 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20201022 -> 20201022 2024-05-22T20:59:03.385 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20201118 -> 20201118 2024-05-22T20:59:03.385 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20201218 -> 20201218 2024-05-22T20:59:03.385 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20210208 -> 20210208 2024-05-22T20:59:03.385 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20210315 -> 20210315 2024-05-22T20:59:03.385 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20210511 -> 20210511 2024-05-22T20:59:03.385 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20210716 -> 20210716 2024-05-22T20:59:03.385 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20210818 -> 20210818 2024-05-22T20:59:03.385 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20210919 -> 20210919 2024-05-22T20:59:03.385 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20211027 -> 20211027 2024-05-22T20:59:03.386 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20211216 -> 20211216 2024-05-22T20:59:03.388 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20220209 -> 20220209 2024-05-22T20:59:03.388 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20220310 -> 20220310 2024-05-22T20:59:03.388 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20220411 -> 20220411 2024-05-22T20:59:03.388 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20220509 -> 20220509 2024-05-22T20:59:03.389 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20220610 -> 20220610 2024-05-22T20:59:03.389 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20220708 -> 20220708 2024-05-22T20:59:03.389 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20220815 -> 20220815 2024-05-22T20:59:03.389 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20220913 -> 20220913 2024-05-22T20:59:03.389 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20221012 -> 20221012 2024-05-22T20:59:03.389 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20221109 -> 20221109 2024-05-22T20:59:03.389 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20221214 -> 20221214 2024-05-22T20:59:03.389 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20230117 -> 20230117 2024-05-22T20:59:03.389 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20230210 -> 20230210 2024-05-22T20:59:03.389 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20230310 -> 20230310 2024-05-22T20:59:03.389 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20230404 -> 20230404 2024-05-22T20:59:03.389 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20230515 -> 20230515 2024-05-22T20:59:03.389 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20230625 -> 20230625 2024-05-22T20:59:03.389 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20230804 -> 20230804 2024-05-22T20:59:03.389 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20230919 -> 20230919 2024-05-22T20:59:03.390 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20231030 -> 20231030 2024-05-22T20:59:03.390 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20231111 -> 20231111 2024-05-22T20:59:03.390 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20231211 -> 20231211 2024-05-22T20:59:03.390 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20240115 -> 20240115 2024-05-22T20:59:03.390 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20240220 -> 20240220 2024-05-22T20:59:03.390 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20240312 -> 20240312 2024-05-22T20:59:03.390 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20240410 -> 20240410 2024-05-22T20:59:03.390 INFO:teuthology.orchestra.run.smithi161.stderr: * [new tag] 20240513 -> 20240513 2024-05-22T20:59:05.233 INFO:teuthology.orchestra.run.smithi179.stderr:Updating files: 44% (1537/3421) Updating files: 45% (1540/3421) Updating files: 46% (1574/3421) Updating files: 47% (1608/3421) Updating files: 48% (1643/3421) Updating files: 49% (1677/3421) Updating files: 50% (1711/3421) Updating files: 51% (1745/3421) Updating files: 52% (1779/3421) Updating files: 53% (1814/3421) Updating files: 54% (1848/3421) Updating files: 55% (1882/3421) Updating files: 56% (1916/3421) Updating files: 57% (1950/3421) Updating files: 58% (1985/3421) Updating files: 59% (2019/3421) Updating files: 59% (2027/3421) Updating files: 60% (2053/3421) Updating files: 61% (2087/3421) Updating files: 62% (2122/3421) Updating files: 63% (2156/3421) Updating files: 64% (2190/3421) Updating files: 65% (2224/3421) Updating files: 66% (2258/3421) Updating files: 67% (2293/3421) Updating files: 68% (2327/3421) Updating files: 69% (2361/3421) Updating files: 70% (2395/3421) Updating files: 70% (2402/3421) Updating files: 71% (2429/3421) Updating files: 72% (2464/3421) Updating files: 73% (2498/3421) Updating files: 74% (2532/3421) Updating files: 75% (2566/3421) Updating files: 76% (2600/3421) Updating files: 77% (2635/3421) Updating files: 78% (2669/3421) Updating files: 79% (2703/3421) Updating files: 80% (2737/3421) Updating files: 81% (2772/3421) Updating files: 82% (2806/3421) Updating files: 83% (2840/3421) Updating files: 84% (2874/3421) Updating files: 85% (2908/3421) Updating files: 86% (2943/3421) Updating files: 87% (2977/3421) Updating files: 88% (3011/3421) Updating files: 89% (3045/3421) Updating files: 90% (3079/3421) Updating files: 91% (3114/3421) Updating files: 92% (3148/3421) Updating files: 93% (3182/3421) Updating files: 94% (3216/3421) Updating files: 95% (3250/3421) Updating files: 96% (3285/3421) Updating files: 97% (3319/3421) Updating files: 98% (3353/3421) Updating files: 99% (3387/3421) Updating files: 100% (3421/3421) Updating files: 100% (3421/3421), done. 2024-05-22T20:59:05.238 INFO:teuthology.orchestra.run.smithi179.stdout:HEAD is now at ec8627ef Merge branch 'amd-isp-staging' into 'main' 2024-05-22T20:59:05.260 INFO:teuthology.task.kernel:Downloading kernel 4fed890e8ce3966dbe380e9eb4b0a796eddb841c on client.1... 2024-05-22T20:59:05.260 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4fed890e8ce3966dbe380e9eb4b0a796eddb841c 2024-05-22T20:59:05.480 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/4fed890e8ce3966dbe380e9eb4b0a796eddb841c/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g4fed890e8ce3/ 2024-05-22T20:59:05.480 DEBUG:teuthology.orchestra.run.smithi179:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g4fed890e8ce3_6.9.0-rc3-g4fed890e8ce3-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://1.chacra.ceph.com/r/kernel/testing/4fed890e8ce3966dbe380e9eb4b0a796eddb841c/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g4fed890e8ce3/ --input-file=- 2024-05-22T20:59:05.483 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi179.front.sepia.ceph.com to complete... 2024-05-22T20:59:07.348 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22 20:59:07 URL:https://1.chacra.ceph.com/r/kernel/testing/4fed890e8ce3966dbe380e9eb4b0a796eddb841c/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g4fed890e8ce3/linux-image-6.9.0-rc3-g4fed890e8ce3_6.9.0-rc3-g4fed890e8ce3-1_amd64.deb [44803860/44803860] -> "/tmp/linux-image.deb" [1] 2024-05-22T20:59:07.348 INFO:teuthology.orchestra.run.smithi179.stderr:FINISHED --2024-05-22 20:59:07-- 2024-05-22T20:59:07.348 INFO:teuthology.orchestra.run.smithi179.stderr:Total wall clock time: 1.8s 2024-05-22T20:59:07.348 INFO:teuthology.orchestra.run.smithi179.stderr:Downloaded: 1 files, 43M in 1.6s (26.0 MB/s) 2024-05-22T20:59:07.352 INFO:teuthology.task.kernel:Installing kernel 4fed890e8ce3966dbe380e9eb4b0a796eddb841c on client.1... 2024-05-22T20:59:07.353 DEBUG:teuthology.orchestra.run.smithi179:> sudo dpkg -i /tmp/linux-image.deb 2024-05-22T20:59:07.631 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g4fed890e8ce3. 2024-05-22T20:59:07.690 INFO:teuthology.orchestra.run.smithi161.stderr:Updating files: 45% (1552/3421) Updating files: 46% (1574/3421) Updating files: 47% (1608/3421) Updating files: 48% (1643/3421) Updating files: 49% (1677/3421) Updating files: 50% (1711/3421) Updating files: 51% (1745/3421) Updating files: 52% (1779/3421) Updating files: 53% (1814/3421) Updating files: 54% (1848/3421) Updating files: 55% (1882/3421) Updating files: 56% (1916/3421) Updating files: 57% (1950/3421) Updating files: 58% (1985/3421) Updating files: 59% (2019/3421) Updating files: 59% (2043/3421) Updating files: 60% (2053/3421) Updating files: 61% (2087/3421) Updating files: 62% (2122/3421) Updating files: 63% (2156/3421) Updating files: 64% (2190/3421) Updating files: 65% (2224/3421) Updating files: 66% (2258/3421) Updating files: 67% (2293/3421) Updating files: 68% (2327/3421) Updating files: 69% (2361/3421) Updating files: 70% (2395/3421) Updating files: 70% (2402/3421) Updating files: 71% (2429/3421) Updating files: 72% (2464/3421) Updating files: 73% (2498/3421) Updating files: 74% (2532/3421) Updating files: 75% (2566/3421) Updating files: 76% (2600/3421) Updating files: 77% (2635/3421) Updating files: 78% (2669/3421) Updating files: 79% (2703/3421) Updating files: 80% (2737/3421) Updating files: 81% (2772/3421) Updating files: 82% (2806/3421) Updating files: 83% (2840/3421) Updating files: 83% (2847/3421) Updating files: 84% (2874/3421) Updating files: 85% (2908/3421) Updating files: 86% (2943/3421) Updating files: 87% (2977/3421) Updating files: 88% (3011/3421) Updating files: 89% (3045/3421) Updating files: 90% (3079/3421) Updating files: 91% (3114/3421) Updating files: 92% (3148/3421) Updating files: 93% (3182/3421) Updating files: 94% (3216/3421) Updating files: 95% (3250/3421) Updating files: 96% (3285/3421) Updating files: 97% (3319/3421) Updating files: 98% (3353/3421) Updating files: 99% (3387/3421) Updating files: 100% (3421/3421) Updating files: 100% (3421/3421), done. 2024-05-22T20:59:07.695 INFO:teuthology.orchestra.run.smithi161.stdout:HEAD is now at ec8627ef Merge branch 'amd-isp-staging' into 'main' 2024-05-22T20:59:07.717 INFO:teuthology.task.kernel:Downloading kernel 4fed890e8ce3966dbe380e9eb4b0a796eddb841c on client.0... 2024-05-22T20:59:07.717 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4fed890e8ce3966dbe380e9eb4b0a796eddb841c 2024-05-22T20:59:07.928 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/4fed890e8ce3966dbe380e9eb4b0a796eddb841c/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g4fed890e8ce3/ 2024-05-22T20:59:07.928 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g4fed890e8ce3_6.9.0-rc3-g4fed890e8ce3-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://1.chacra.ceph.com/r/kernel/testing/4fed890e8ce3966dbe380e9eb4b0a796eddb841c/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g4fed890e8ce3/ --input-file=- 2024-05-22T20:59:07.930 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi161.front.sepia.ceph.com to complete... 2024-05-22T20:59:09.711 INFO:teuthology.orchestra.run.smithi179.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-05-22T20:59:09.715 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-05-22T20:59:09.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22 20:59:09 URL:https://1.chacra.ceph.com/r/kernel/testing/4fed890e8ce3966dbe380e9eb4b0a796eddb841c/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g4fed890e8ce3/linux-image-6.9.0-rc3-g4fed890e8ce3_6.9.0-rc3-g4fed890e8ce3-1_amd64.deb [44803860/44803860] -> "/tmp/linux-image.deb" [1] 2024-05-22T20:59:09.890 INFO:teuthology.orchestra.run.smithi161.stderr:FINISHED --2024-05-22 20:59:09-- 2024-05-22T20:59:09.890 INFO:teuthology.orchestra.run.smithi161.stderr:Total wall clock time: 1.9s 2024-05-22T20:59:09.890 INFO:teuthology.orchestra.run.smithi161.stderr:Downloaded: 1 files, 43M in 1.7s (25.8 MB/s) 2024-05-22T20:59:09.894 INFO:teuthology.task.kernel:Installing kernel 4fed890e8ce3966dbe380e9eb4b0a796eddb841c on client.0... 2024-05-22T20:59:09.894 DEBUG:teuthology.orchestra.run.smithi161:> sudo dpkg -i /tmp/linux-image.deb 2024-05-22T20:59:09.925 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking linux-image-6.9.0-rc3-g4fed890e8ce3 (6.9.0-rc3-g4fed890e8ce3-1) ... 2024-05-22T20:59:10.163 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g4fed890e8ce3. 2024-05-22T20:59:12.158 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-05-22T20:59:12.162 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-05-22T20:59:12.258 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-image-6.9.0-rc3-g4fed890e8ce3 (6.9.0-rc3-g4fed890e8ce3-1) ... 2024-05-22T20:59:13.748 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up linux-image-6.9.0-rc3-g4fed890e8ce3 (6.9.0-rc3-g4fed890e8ce3-1) ... 2024-05-22T20:59:13.853 INFO:teuthology.orchestra.run.smithi179.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:16.220 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-image-6.9.0-rc3-g4fed890e8ce3 (6.9.0-rc3-g4fed890e8ce3-1) ... 2024-05-22T20:59:16.319 INFO:teuthology.orchestra.run.smithi161.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:36.726 INFO:teuthology.orchestra.run.smithi179.stderr:Sourcing file `/etc/default/grub' 2024-05-22T20:59:36.741 INFO:teuthology.orchestra.run.smithi179.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-22T20:59:36.768 INFO:teuthology.orchestra.run.smithi179.stderr:Generating grub configuration file ... 2024-05-22T20:59:37.070 INFO:teuthology.orchestra.run.smithi179.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:37.086 INFO:teuthology.orchestra.run.smithi179.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:37.401 INFO:teuthology.orchestra.run.smithi179.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-22T20:59:37.408 INFO:teuthology.orchestra.run.smithi179.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-22T20:59:37.471 INFO:teuthology.orchestra.run.smithi179.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-22T20:59:37.476 INFO:teuthology.orchestra.run.smithi179.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-22T20:59:37.630 INFO:teuthology.orchestra.run.smithi179.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-22T20:59:37.631 INFO:teuthology.orchestra.run.smithi179.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-22T20:59:37.631 INFO:teuthology.orchestra.run.smithi179.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-22T20:59:37.685 INFO:teuthology.orchestra.run.smithi179.stderr:done 2024-05-22T20:59:37.838 DEBUG:teuthology.orchestra.run.smithi179:> dpkg-deb -c /tmp/linux-image.deb 2024-05-22T20:59:37.846 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./ 2024-05-22T20:59:37.846 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./boot/ 2024-05-22T20:59:37.846 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 7002012 2024-05-21 01:11 ./boot/System.map-6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:37.862 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 172073 2024-05-21 01:11 ./boot/config-6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:37.862 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13120512 2024-05-21 01:11 ./boot/vmlinuz-6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:37.882 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./etc/ 2024-05-22T20:59:37.882 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./etc/kernel/ 2024-05-22T20:59:37.882 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./etc/kernel/postinst.d/ 2024-05-22T20:59:37.883 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./etc/kernel/postrm.d/ 2024-05-22T20:59:37.883 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./etc/kernel/preinst.d/ 2024-05-22T20:59:37.883 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./etc/kernel/prerm.d/ 2024-05-22T20:59:37.883 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/ 2024-05-22T20:59:37.883 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/ 2024-05-22T20:59:37.883 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/ 2024-05-22T20:59:37.883 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/ 2024-05-22T20:59:37.883 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/ 2024-05-22T20:59:37.883 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/ 2024-05-22T20:59:37.883 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/ 2024-05-22T20:59:37.883 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 507569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/aesni-intel.ko 2024-05-22T20:59:37.886 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-22T20:59:37.886 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-22T20:59:37.887 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-22T20:59:37.887 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 97953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-22T20:59:37.887 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 109905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-22T20:59:37.887 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 155249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-22T20:59:37.887 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-22T20:59:37.887 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-22T20:59:37.887 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 87009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-22T20:59:37.887 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-22T20:59:37.887 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 50305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-22T20:59:37.887 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 50377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-22T20:59:37.887 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 54513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-22T20:59:37.888 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-22T20:59:37.888 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 48049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-22T20:59:37.888 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 62873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-22T20:59:37.888 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 53569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-22T20:59:37.888 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-22T20:59:37.888 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-22T20:59:37.888 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kernel/ 2024-05-22T20:59:37.888 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kernel/cpu/ 2024-05-22T20:59:37.888 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kernel/cpu/mce/ 2024-05-22T20:59:37.888 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 49081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-22T20:59:37.888 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kernel/cpuid.ko 2024-05-22T20:59:37.888 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kernel/msr.ko 2024-05-22T20:59:37.888 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kvm/ 2024-05-22T20:59:37.889 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 444425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-22T20:59:37.889 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1035881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-22T20:59:37.889 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 2849609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kvm/kvm.ko 2024-05-22T20:59:37.901 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/ 2024-05-22T20:59:37.901 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 65025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/af_alg.ko 2024-05-22T20:59:37.901 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/algif_hash.ko 2024-05-22T20:59:37.901 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/algif_skcipher.ko 2024-05-22T20:59:37.901 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/ansi_cprng.ko 2024-05-22T20:59:37.901 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/anubis.ko 2024-05-22T20:59:37.901 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10929 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/arc4.ko 2024-05-22T20:59:37.901 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/asymmetric_keys/ 2024-05-22T20:59:37.901 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-05-22T20:59:37.901 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/ 2024-05-22T20:59:37.902 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/async_memcpy.ko 2024-05-22T20:59:37.902 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/async_pq.ko 2024-05-22T20:59:37.902 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-22T20:59:37.902 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/async_tx.ko 2024-05-22T20:59:37.902 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/async_xor.ko 2024-05-22T20:59:37.902 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/raid6test.ko 2024-05-22T20:59:37.902 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/authenc.ko 2024-05-22T20:59:37.902 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/authencesn.ko 2024-05-22T20:59:37.902 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/blake2b_generic.ko 2024-05-22T20:59:37.902 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/blowfish_common.ko 2024-05-22T20:59:37.902 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/blowfish_generic.ko 2024-05-22T20:59:37.902 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/camellia_generic.ko 2024-05-22T20:59:37.903 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cast5_generic.ko 2024-05-22T20:59:37.903 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cast6_generic.ko 2024-05-22T20:59:37.903 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cast_common.ko 2024-05-22T20:59:37.903 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/ccm.ko 2024-05-22T20:59:37.903 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cmac.ko 2024-05-22T20:59:37.903 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/crc32_generic.ko 2024-05-22T20:59:37.903 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 59129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cryptd.ko 2024-05-22T20:59:37.903 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 48113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/crypto_engine.ko 2024-05-22T20:59:37.903 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/crypto_simd.ko 2024-05-22T20:59:37.903 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/crypto_user.ko 2024-05-22T20:59:37.903 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cts.ko 2024-05-22T20:59:37.903 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/des_generic.ko 2024-05-22T20:59:37.903 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/dh_generic.ko 2024-05-22T20:59:37.904 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/echainiv.ko 2024-05-22T20:59:37.904 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/essiv.ko 2024-05-22T20:59:37.904 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/fcrypt.ko 2024-05-22T20:59:37.904 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/khazad.ko 2024-05-22T20:59:37.904 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/lrw.ko 2024-05-22T20:59:37.904 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/lz4.ko 2024-05-22T20:59:37.904 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/lz4hc.ko 2024-05-22T20:59:37.904 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/md4.ko 2024-05-22T20:59:37.904 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/michael_mic.ko 2024-05-22T20:59:37.904 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/pcbc.ko 2024-05-22T20:59:37.904 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/pcrypt.ko 2024-05-22T20:59:37.904 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/rmd160.ko 2024-05-22T20:59:37.905 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/seed.ko 2024-05-22T20:59:37.905 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/serpent_generic.ko 2024-05-22T20:59:37.905 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 169065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/tcrypt.ko 2024-05-22T20:59:37.905 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/tea.ko 2024-05-22T20:59:37.905 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/twofish_common.ko 2024-05-22T20:59:37.905 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/twofish_generic.ko 2024-05-22T20:59:37.905 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/vmac.ko 2024-05-22T20:59:37.905 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 38209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/wp512.ko 2024-05-22T20:59:37.905 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/xcbc.ko 2024-05-22T20:59:37.905 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/xor.ko 2024-05-22T20:59:37.905 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/xts.ko 2024-05-22T20:59:37.905 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/xxhash_generic.ko 2024-05-22T20:59:37.905 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ 2024-05-22T20:59:37.905 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/ 2024-05-22T20:59:37.906 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/acpi_extlog.ko 2024-05-22T20:59:37.906 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-22T20:59:37.906 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/acpi_pad.ko 2024-05-22T20:59:37.906 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/apei/ 2024-05-22T20:59:37.906 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/apei/einj.ko 2024-05-22T20:59:37.906 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/ec_sys.ko 2024-05-22T20:59:37.906 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/platform_profile.ko 2024-05-22T20:59:37.906 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/sbs.ko 2024-05-22T20:59:37.906 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/sbshc.ko 2024-05-22T20:59:37.906 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 119033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/video.ko 2024-05-22T20:59:37.906 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/ 2024-05-22T20:59:37.910 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/acard-ahci.ko 2024-05-22T20:59:37.911 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 96825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/ahci.ko 2024-05-22T20:59:37.911 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/ahci_platform.ko 2024-05-22T20:59:37.911 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 107737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/libahci.ko 2024-05-22T20:59:37.911 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 42001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/libahci_platform.ko 2024-05-22T20:59:37.911 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_acpi.ko 2024-05-22T20:59:37.911 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_ali.ko 2024-05-22T20:59:37.911 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 38329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_amd.ko 2024-05-22T20:59:37.911 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_artop.ko 2024-05-22T20:59:37.911 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_atiixp.ko 2024-05-22T20:59:37.911 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_atp867x.ko 2024-05-22T20:59:37.911 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_cmd640.ko 2024-05-22T20:59:37.911 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28929 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_cmd64x.ko 2024-05-22T20:59:37.912 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_cypress.ko 2024-05-22T20:59:37.912 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_efar.ko 2024-05-22T20:59:37.912 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_hpt366.ko 2024-05-22T20:59:37.912 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 38913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_hpt37x.ko 2024-05-22T20:59:37.912 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-22T20:59:37.912 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-22T20:59:37.912 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_it8213.ko 2024-05-22T20:59:37.912 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 41289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_it821x.ko 2024-05-22T20:59:37.912 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_jmicron.ko 2024-05-22T20:59:37.912 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 60257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_legacy.ko 2024-05-22T20:59:37.912 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_marvell.ko 2024-05-22T20:59:37.912 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_mpiix.ko 2024-05-22T20:59:37.913 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_netcell.ko 2024-05-22T20:59:37.913 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_ninja32.ko 2024-05-22T20:59:37.913 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_ns87410.ko 2024-05-22T20:59:37.913 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_ns87415.ko 2024-05-22T20:59:37.913 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_oldpiix.ko 2024-05-22T20:59:37.913 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_opti.ko 2024-05-22T20:59:37.913 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_optidma.ko 2024-05-22T20:59:37.913 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_pcmcia.ko 2024-05-22T20:59:37.913 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-22T20:59:37.913 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-22T20:59:37.913 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_piccolo.ko 2024-05-22T20:59:37.913 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_radisys.ko 2024-05-22T20:59:37.913 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_rdc.ko 2024-05-22T20:59:37.914 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_rz1000.ko 2024-05-22T20:59:37.914 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_sch.ko 2024-05-22T20:59:37.914 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_serverworks.ko 2024-05-22T20:59:37.914 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_sil680.ko 2024-05-22T20:59:37.914 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_sl82c105.ko 2024-05-22T20:59:37.914 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_triflex.ko 2024-05-22T20:59:37.914 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_via.ko 2024-05-22T20:59:37.914 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pdc_adma.ko 2024-05-22T20:59:37.914 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_inic162x.ko 2024-05-22T20:59:37.914 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 102121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_mv.ko 2024-05-22T20:59:37.914 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 85409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_nv.ko 2024-05-22T20:59:37.914 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_promise.ko 2024-05-22T20:59:37.915 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_qstor.ko 2024-05-22T20:59:37.915 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_sil.ko 2024-05-22T20:59:37.915 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 45905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_sil24.ko 2024-05-22T20:59:37.915 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_sis.ko 2024-05-22T20:59:37.915 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_svw.ko 2024-05-22T20:59:37.915 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 53353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_sx4.ko 2024-05-22T20:59:37.915 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_uli.ko 2024-05-22T20:59:37.915 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 41049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_via.ko 2024-05-22T20:59:37.915 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_vsc.ko 2024-05-22T20:59:37.915 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/auxdisplay/ 2024-05-22T20:59:37.915 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/auxdisplay/ks0108.ko 2024-05-22T20:59:37.915 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/bcma/ 2024-05-22T20:59:37.915 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 163969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/bcma/bcma.ko 2024-05-22T20:59:37.915 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/ 2024-05-22T20:59:37.916 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/aoe/ 2024-05-22T20:59:37.916 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 140865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/aoe/aoe.ko 2024-05-22T20:59:37.916 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/drbd/ 2024-05-22T20:59:37.916 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1241657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/drbd/drbd.ko 2024-05-22T20:59:37.972 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 203289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/floppy.ko 2024-05-22T20:59:37.973 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/mtip32xx/ 2024-05-22T20:59:37.973 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 115265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-22T20:59:37.973 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 143433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/nbd.ko 2024-05-22T20:59:37.973 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/null_blk/ 2024-05-22T20:59:37.973 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 126617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/null_blk/null_blk.ko 2024-05-22T20:59:37.973 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 119873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/pktcdvd.ko 2024-05-22T20:59:37.973 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 313217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/rbd.ko 2024-05-22T20:59:37.973 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/xen-blkback/ 2024-05-22T20:59:37.973 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 117233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-22T20:59:37.973 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/zram/ 2024-05-22T20:59:37.973 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 70385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/zram/zram.ko 2024-05-22T20:59:37.974 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ 2024-05-22T20:59:37.974 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/applicom.ko 2024-05-22T20:59:37.974 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hangcheck-timer.ko 2024-05-22T20:59:37.974 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/ 2024-05-22T20:59:37.974 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-22T20:59:37.974 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-22T20:59:37.974 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-22T20:59:37.974 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/via-rng.ko 2024-05-22T20:59:37.974 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-22T20:59:37.974 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ 2024-05-22T20:59:37.974 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 37257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-22T20:59:37.974 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 186921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-22T20:59:37.975 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-22T20:59:37.975 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 179561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-22T20:59:37.975 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 85441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-22T20:59:37.975 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 65313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-22T20:59:37.975 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/lp.ko 2024-05-22T20:59:37.975 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/mwave/ 2024-05-22T20:59:37.975 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 103409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/mwave/mwave.ko 2024-05-22T20:59:37.975 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/nvram.ko 2024-05-22T20:59:37.975 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 38977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ppdev.ko 2024-05-22T20:59:37.975 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 59241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tlclk.ko 2024-05-22T20:59:37.975 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/ 2024-05-22T20:59:37.975 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-22T20:59:37.976 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-05-22T20:59:37.976 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-05-22T20:59:37.976 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 39193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-05-22T20:59:37.976 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-22T20:59:37.976 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-22T20:59:37.976 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-05-22T20:59:37.976 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/xillybus/ 2024-05-22T20:59:37.976 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/xillybus/xillybus_class.ko 2024-05-22T20:59:37.976 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 62545 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/xillybus/xillybus_core.ko 2024-05-22T20:59:37.976 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13929 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-05-22T20:59:37.976 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/clk/ 2024-05-22T20:59:37.976 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/clk/clk-palmas.ko 2024-05-22T20:59:37.976 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 57289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/clk/clk-si5351.ko 2024-05-22T20:59:37.977 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/clk/clk-twl6040.ko 2024-05-22T20:59:37.977 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/clk/clk-wm831x.ko 2024-05-22T20:59:37.977 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/cpufreq/ 2024-05-22T20:59:37.977 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-22T20:59:37.977 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-22T20:59:37.977 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-22T20:59:37.977 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/ 2024-05-22T20:59:37.977 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/ccp/ 2024-05-22T20:59:37.977 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 89257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-22T20:59:37.977 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 333657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/ccp/ccp.ko 2024-05-22T20:59:37.977 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/ 2024-05-22T20:59:37.977 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/qat/ 2024-05-22T20:59:37.977 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/qat/qat_common/ 2024-05-22T20:59:37.978 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 751713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-22T20:59:37.978 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-05-22T20:59:37.978 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-22T20:59:37.978 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/padlock-aes.ko 2024-05-22T20:59:37.978 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/padlock-sha.ko 2024-05-22T20:59:37.978 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/virtio/ 2024-05-22T20:59:37.978 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 90441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-22T20:59:37.978 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dca/ 2024-05-22T20:59:37.978 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dca/dca.ko 2024-05-22T20:59:37.978 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dma/ 2024-05-22T20:59:37.978 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dma/dw/ 2024-05-22T20:59:37.978 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-22T20:59:37.979 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dma/ioat/ 2024-05-22T20:59:37.979 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 146769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-22T20:59:37.979 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/ 2024-05-22T20:59:37.979 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 96145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/amd64_edac.ko 2024-05-22T20:59:37.979 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 41993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/e752x_edac.ko 2024-05-22T20:59:37.979 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 51105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/edac_mce_amd.ko 2024-05-22T20:59:37.979 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i3000_edac.ko 2024-05-22T20:59:37.979 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i3200_edac.ko 2024-05-22T20:59:37.979 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i5100_edac.ko 2024-05-22T20:59:37.979 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i5400_edac.ko 2024-05-22T20:59:37.979 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i7300_edac.ko 2024-05-22T20:59:37.979 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 64161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i7core_edac.ko 2024-05-22T20:59:37.980 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i82975x_edac.ko 2024-05-22T20:59:37.980 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/ie31200_edac.ko 2024-05-22T20:59:37.980 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 68697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/sb_edac.ko 2024-05-22T20:59:37.980 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/x38_edac.ko 2024-05-22T20:59:37.980 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/extcon/ 2024-05-22T20:59:37.980 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/extcon/extcon-gpio.ko 2024-05-22T20:59:37.980 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/extcon/extcon-max14577.ko 2024-05-22T20:59:37.980 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/extcon/extcon-max77693.ko 2024-05-22T20:59:37.980 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/extcon/extcon-max8997.ko 2024-05-22T20:59:37.980 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/extcon/extcon-palmas.ko 2024-05-22T20:59:37.980 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/extcon/extcon-rt8973a.ko 2024-05-22T20:59:37.980 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/extcon/extcon-sm5502.ko 2024-05-22T20:59:37.980 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/firmware/ 2024-05-22T20:59:37.981 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 42953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/firmware/dmi-sysfs.ko 2024-05-22T20:59:37.981 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/firmware/efi/ 2024-05-22T20:59:37.981 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/firmware/efi/efi-pstore.ko 2024-05-22T20:59:37.981 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-22T20:59:37.981 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/ 2024-05-22T20:59:37.981 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-adp5520.ko 2024-05-22T20:59:37.981 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-amd8111.ko 2024-05-22T20:59:37.981 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-arizona.ko 2024-05-22T20:59:37.981 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-da9052.ko 2024-05-22T20:59:37.981 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-da9055.ko 2024-05-22T20:59:37.981 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-dln2.ko 2024-05-22T20:59:37.981 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 81073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-f7188x.ko 2024-05-22T20:59:37.982 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-generic.ko 2024-05-22T20:59:37.982 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-ich.ko 2024-05-22T20:59:37.982 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-05-22T20:59:37.982 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-kempld.ko 2024-05-22T20:59:37.982 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-lp3943.ko 2024-05-22T20:59:37.982 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-max7300.ko 2024-05-22T20:59:37.982 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-max730x.ko 2024-05-22T20:59:37.982 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-max732x.ko 2024-05-22T20:59:37.982 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-05-22T20:59:37.982 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 55505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-pca953x.ko 2024-05-22T20:59:37.982 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-pcf857x.ko 2024-05-22T20:59:37.982 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-rdc321x.ko 2024-05-22T20:59:37.983 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-sch.ko 2024-05-22T20:59:37.983 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-sch311x.ko 2024-05-22T20:59:37.983 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-tps65912.ko 2024-05-22T20:59:37.983 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-twl4030.ko 2024-05-22T20:59:37.983 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-twl6040.ko 2024-05-22T20:59:37.983 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-22T20:59:37.983 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17897 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-vx855.ko 2024-05-22T20:59:37.983 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-wm831x.ko 2024-05-22T20:59:37.983 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-wm8350.ko 2024-05-22T20:59:37.983 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-wm8994.ko 2024-05-22T20:59:37.983 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/ 2024-05-22T20:59:37.983 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-a4tech.ko 2024-05-22T20:59:37.983 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-apple.ko 2024-05-22T20:59:37.983 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-appleir.ko 2024-05-22T20:59:37.984 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-aureal.ko 2024-05-22T20:59:37.984 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-axff.ko 2024-05-22T20:59:37.984 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-belkin.ko 2024-05-22T20:59:37.984 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-cherry.ko 2024-05-22T20:59:37.984 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-chicony.ko 2024-05-22T20:59:37.984 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 74393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-cp2112.ko 2024-05-22T20:59:37.984 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-cypress.ko 2024-05-22T20:59:37.984 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-dr.ko 2024-05-22T20:59:37.984 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-elecom.ko 2024-05-22T20:59:37.984 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-elo.ko 2024-05-22T20:59:37.984 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-emsff.ko 2024-05-22T20:59:37.984 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-ezkey.ko 2024-05-22T20:59:37.984 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-gaff.ko 2024-05-22T20:59:37.984 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-generic.ko 2024-05-22T20:59:37.985 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-gt683r.ko 2024-05-22T20:59:37.985 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-gyration.ko 2024-05-22T20:59:37.985 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-22T20:59:37.985 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-22T20:59:37.985 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-holtekff.ko 2024-05-22T20:59:37.985 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-hyperv.ko 2024-05-22T20:59:37.985 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-icade.ko 2024-05-22T20:59:37.985 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-kensington.ko 2024-05-22T20:59:37.985 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-keytouch.ko 2024-05-22T20:59:37.985 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-kye.ko 2024-05-22T20:59:37.985 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-lcpower.ko 2024-05-22T20:59:37.985 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-led.ko 2024-05-22T20:59:37.986 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 54217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-lenovo.ko 2024-05-22T20:59:37.986 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 37553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-lg-g15.ko 2024-05-22T20:59:37.986 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 68081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-22T20:59:37.986 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 121369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-22T20:59:37.986 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 97953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-logitech.ko 2024-05-22T20:59:37.986 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-magicmouse.ko 2024-05-22T20:59:37.986 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-microsoft.ko 2024-05-22T20:59:37.986 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-monterey.ko 2024-05-22T20:59:37.986 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 59001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-multitouch.ko 2024-05-22T20:59:37.986 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 39697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-ntrig.ko 2024-05-22T20:59:37.986 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-ortek.ko 2024-05-22T20:59:37.986 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-penmount.ko 2024-05-22T20:59:37.987 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-petalynx.ko 2024-05-22T20:59:37.987 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 91241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-picolcd.ko 2024-05-22T20:59:37.987 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-pl.ko 2024-05-22T20:59:37.987 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-plantronics.ko 2024-05-22T20:59:37.987 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-primax.ko 2024-05-22T20:59:37.987 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 42889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-rmi.ko 2024-05-22T20:59:37.987 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-22T20:59:37.987 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-common.ko 2024-05-22T20:59:37.987 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 38865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-22T20:59:37.987 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-22T20:59:37.987 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 38073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-22T20:59:37.987 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-22T20:59:37.987 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-22T20:59:37.988 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-22T20:59:37.988 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-22T20:59:37.988 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-22T20:59:37.988 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-22T20:59:37.988 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat.ko 2024-05-22T20:59:37.988 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-saitek.ko 2024-05-22T20:59:37.988 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 42305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-samsung.ko 2024-05-22T20:59:37.988 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-22T20:59:37.988 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-sjoy.ko 2024-05-22T20:59:37.988 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 62129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-sony.ko 2024-05-22T20:59:37.988 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-speedlink.ko 2024-05-22T20:59:37.988 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-steelseries.ko 2024-05-22T20:59:37.988 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-sunplus.ko 2024-05-22T20:59:37.989 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-22T20:59:37.989 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-tivo.ko 2024-05-22T20:59:37.989 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-tmff.ko 2024-05-22T20:59:37.989 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-topseed.ko 2024-05-22T20:59:37.989 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-twinhan.ko 2024-05-22T20:59:37.989 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 81113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-uclogic.ko 2024-05-22T20:59:37.989 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-waltop.ko 2024-05-22T20:59:37.989 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 184745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-wiimote.ko 2024-05-22T20:59:37.989 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-xinmo.ko 2024-05-22T20:59:37.989 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-zpff.ko 2024-05-22T20:59:37.989 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-zydacron.ko 2024-05-22T20:59:37.989 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 319921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid.ko 2024-05-22T20:59:37.989 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 39529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/uhid.ko 2024-05-22T20:59:37.990 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/usbhid/ 2024-05-22T20:59:37.990 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 153633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/usbhid/usbhid.ko 2024-05-22T20:59:37.990 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/usbhid/usbkbd.ko 2024-05-22T20:59:37.990 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/usbhid/usbmouse.ko 2024-05-22T20:59:37.990 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 261945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/wacom.ko 2024-05-22T20:59:37.990 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hv/ 2024-05-22T20:59:37.990 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 75065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hv/hv_balloon.ko 2024-05-22T20:59:37.990 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 110681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hv/hv_utils.ko 2024-05-22T20:59:37.990 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 382033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hv/hv_vmbus.ko 2024-05-22T20:59:37.990 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ 2024-05-22T20:59:37.990 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 77153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/abituguru.ko 2024-05-22T20:59:37.990 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 67009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/abituguru3.ko 2024-05-22T20:59:37.991 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 42249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-22T20:59:37.991 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ad7414.ko 2024-05-22T20:59:37.991 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ad7418.ko 2024-05-22T20:59:37.991 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adc128d818.ko 2024-05-22T20:59:37.991 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 38097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adm1025.ko 2024-05-22T20:59:37.991 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 115057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adm1026.ko 2024-05-22T20:59:37.991 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adm1029.ko 2024-05-22T20:59:37.991 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 55225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adm1031.ko 2024-05-22T20:59:37.991 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adm9240.ko 2024-05-22T20:59:37.991 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ads7828.ko 2024-05-22T20:59:37.991 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7410.ko 2024-05-22T20:59:37.991 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7411.ko 2024-05-22T20:59:37.991 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 89025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7462.ko 2024-05-22T20:59:37.992 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 49153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7470.ko 2024-05-22T20:59:37.992 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 81473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7475.ko 2024-05-22T20:59:37.992 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7x10.ko 2024-05-22T20:59:37.992 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 48177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/amc6821.ko 2024-05-22T20:59:37.992 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 68865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/applesmc.ko 2024-05-22T20:59:37.992 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 60193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/asb100.ko 2024-05-22T20:59:37.992 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 69009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/asc7621.ko 2024-05-22T20:59:37.992 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 47977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-22T20:59:37.992 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/atxp1.ko 2024-05-22T20:59:37.992 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/coretemp.ko 2024-05-22T20:59:37.992 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/da9052-hwmon.ko 2024-05-22T20:59:37.992 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/da9055-hwmon.ko 2024-05-22T20:59:37.992 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 55713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-22T20:59:37.993 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 113201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/dme1737.ko 2024-05-22T20:59:37.993 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ds1621.ko 2024-05-22T20:59:37.993 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ds620.ko 2024-05-22T20:59:37.993 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/emc1403.ko 2024-05-22T20:59:37.993 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/emc2103.ko 2024-05-22T20:59:37.993 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/emc6w201.ko 2024-05-22T20:59:37.993 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 78921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/f71805f.ko 2024-05-22T20:59:37.993 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 108961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/f71882fg.ko 2024-05-22T20:59:37.993 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 50697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/f75375s.ko 2024-05-22T20:59:37.993 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/fam15h_power.ko 2024-05-22T20:59:37.993 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 65769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/fschmd.ko 2024-05-22T20:59:37.993 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/g760a.ko 2024-05-22T20:59:37.993 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/g762.ko 2024-05-22T20:59:37.993 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 59761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/gl518sm.ko 2024-05-22T20:59:37.994 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 56313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/gl520sm.ko 2024-05-22T20:59:37.994 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/hih6130.ko 2024-05-22T20:59:37.994 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-22T20:59:37.994 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/i5500_temp.ko 2024-05-22T20:59:37.994 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/i5k_amb.ko 2024-05-22T20:59:37.994 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 42585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ibmaem.ko 2024-05-22T20:59:37.994 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ibmpex.ko 2024-05-22T20:59:37.994 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ina209.ko 2024-05-22T20:59:37.994 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ina2xx.ko 2024-05-22T20:59:37.994 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 156465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/it87.ko 2024-05-22T20:59:37.994 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/jc42.ko 2024-05-22T20:59:37.994 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/k10temp.ko 2024-05-22T20:59:37.994 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/k8temp.ko 2024-05-22T20:59:37.995 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lineage-pem.ko 2024-05-22T20:59:37.995 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 57769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm63.ko 2024-05-22T20:59:37.995 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm73.ko 2024-05-22T20:59:37.995 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm75.ko 2024-05-22T20:59:37.995 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm77.ko 2024-05-22T20:59:37.995 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 41473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm78.ko 2024-05-22T20:59:37.995 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 37049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm80.ko 2024-05-22T20:59:37.995 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20545 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm83.ko 2024-05-22T20:59:37.995 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 86497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm85.ko 2024-05-22T20:59:37.995 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 55641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm87.ko 2024-05-22T20:59:37.995 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 42249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm90.ko 2024-05-22T20:59:37.995 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm92.ko 2024-05-22T20:59:37.995 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 131505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm93.ko 2024-05-22T20:59:37.996 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm95234.ko 2024-05-22T20:59:37.996 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm95241.ko 2024-05-22T20:59:37.996 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm95245.ko 2024-05-22T20:59:37.996 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc2945.ko 2024-05-22T20:59:37.996 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4151.ko 2024-05-22T20:59:37.996 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4215.ko 2024-05-22T20:59:37.996 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4222.ko 2024-05-22T20:59:37.996 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4245.ko 2024-05-22T20:59:37.996 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4260.ko 2024-05-22T20:59:37.996 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4261.ko 2024-05-22T20:59:37.996 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max16065.ko 2024-05-22T20:59:37.996 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max1619.ko 2024-05-22T20:59:37.997 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max1668.ko 2024-05-22T20:59:37.997 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max197.ko 2024-05-22T20:59:37.997 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max6639.ko 2024-05-22T20:59:37.997 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max6650.ko 2024-05-22T20:59:37.997 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 39729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max6697.ko 2024-05-22T20:59:37.997 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/mc13783-adc.ko 2024-05-22T20:59:37.997 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/mcp3021.ko 2024-05-22T20:59:37.997 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-05-22T20:59:37.997 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 56505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/nct6683.ko 2024-05-22T20:59:37.997 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 141353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/nct6775-core.ko 2024-05-22T20:59:37.997 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 72737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/nct6775.ko 2024-05-22T20:59:37.997 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 67841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/nct7802.ko 2024-05-22T20:59:37.997 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 97777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pc87360.ko 2024-05-22T20:59:37.998 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 61921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pc87427.ko 2024-05-22T20:59:37.998 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pcf8591.ko 2024-05-22T20:59:37.998 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/ 2024-05-22T20:59:37.998 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-22T20:59:37.998 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-22T20:59:37.998 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-22T20:59:37.998 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-22T20:59:37.998 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-22T20:59:37.998 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-22T20:59:37.998 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-22T20:59:37.998 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 101233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-22T20:59:37.998 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-22T20:59:37.998 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-22T20:59:37.999 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-22T20:59:37.999 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-22T20:59:37.999 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/powr1220.ko 2024-05-22T20:59:37.999 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sch5627.ko 2024-05-22T20:59:37.999 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sch5636.ko 2024-05-22T20:59:37.999 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-22T20:59:37.999 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sht15.ko 2024-05-22T20:59:37.999 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sht21.ko 2024-05-22T20:59:37.999 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/shtc1.ko 2024-05-22T20:59:37.999 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 48713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sis5595.ko 2024-05-22T20:59:37.999 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/smsc47b397.ko 2024-05-22T20:59:37.999 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 46473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/smsc47m1.ko 2024-05-22T20:59:37.999 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/smsc47m192.ko 2024-05-22T20:59:38.000 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/thmc50.ko 2024-05-22T20:59:38.000 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/tmp102.ko 2024-05-22T20:59:38.000 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/tmp103.ko 2024-05-22T20:59:38.000 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/tmp401.ko 2024-05-22T20:59:38.000 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/tmp421.ko 2024-05-22T20:59:38.000 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/via-cputemp.ko 2024-05-22T20:59:38.000 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 50217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/via686a.ko 2024-05-22T20:59:38.000 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 59665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/vt1211.ko 2024-05-22T20:59:38.000 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 62737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/vt8231.ko 2024-05-22T20:59:38.000 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 71169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83627ehf.ko 2024-05-22T20:59:38.000 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 96577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83627hf.ko 2024-05-22T20:59:38.000 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 85913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83781d.ko 2024-05-22T20:59:38.001 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 75129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83791d.ko 2024-05-22T20:59:38.001 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 83425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83792d.ko 2024-05-22T20:59:38.001 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 116777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83793.ko 2024-05-22T20:59:38.001 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 134441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83795.ko 2024-05-22T20:59:38.001 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83l785ts.ko 2024-05-22T20:59:38.001 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83l786ng.ko 2024-05-22T20:59:38.001 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-05-22T20:59:38.001 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-05-22T20:59:38.001 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/ 2024-05-22T20:59:38.001 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/algos/ 2024-05-22T20:59:38.001 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-22T20:59:38.001 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-22T20:59:38.001 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/ 2024-05-22T20:59:38.002 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-05-22T20:59:38.002 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-05-22T20:59:38.002 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-05-22T20:59:38.002 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-22T20:59:38.002 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-22T20:59:38.002 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-22T20:59:38.002 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-05-22T20:59:38.002 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 7729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-22T20:59:38.002 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-05-22T20:59:38.002 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 64233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-22T20:59:38.002 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-22T20:59:38.002 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-22T20:59:38.003 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-22T20:59:38.003 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-05-22T20:59:38.003 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-05-22T20:59:38.003 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 76969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-22T20:59:38.003 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-22T20:59:38.003 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 52937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-22T20:59:38.003 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-05-22T20:59:38.003 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-22T20:59:38.003 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-22T20:59:38.003 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-05-22T20:59:38.003 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-22T20:59:38.003 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-22T20:59:38.003 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 57217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-22T20:59:38.003 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-05-22T20:59:38.004 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-22T20:59:38.004 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-22T20:59:38.004 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-05-22T20:59:38.004 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-05-22T20:59:38.004 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-22T20:59:38.004 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-05-22T20:59:38.004 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-22T20:59:38.004 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-22T20:59:38.004 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-22T20:59:38.004 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-22T20:59:38.004 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 59193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-05-22T20:59:38.004 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/i2c-mux.ko 2024-05-22T20:59:38.005 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/i2c-smbus.ko 2024-05-22T20:59:38.005 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/i2c-stub.ko 2024-05-22T20:59:38.005 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/muxes/ 2024-05-22T20:59:38.005 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-05-22T20:59:38.005 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-05-22T20:59:38.005 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-05-22T20:59:38.005 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ 2024-05-22T20:59:38.005 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/ 2024-05-22T20:59:38.005 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 329697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-22T20:59:38.005 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1119913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/ib_core.ko 2024-05-22T20:59:38.005 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 83905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-22T20:59:38.005 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 408433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-22T20:59:38.006 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 120921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-22T20:59:38.006 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 347217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-22T20:59:38.006 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 84945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-22T20:59:38.006 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/ 2024-05-22T20:59:38.006 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/cxgb4/ 2024-05-22T20:59:38.006 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 590929 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-22T20:59:38.006 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mlx4/ 2024-05-22T20:59:38.006 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 514641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-22T20:59:38.006 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mlx5/ 2024-05-22T20:59:38.006 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 911665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-22T20:59:38.006 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mthca/ 2024-05-22T20:59:38.006 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 375545 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-22T20:59:38.007 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/ocrdma/ 2024-05-22T20:59:38.007 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 197121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-05-22T20:59:38.007 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/usnic/ 2024-05-22T20:59:38.007 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 212553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-05-22T20:59:38.007 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/ 2024-05-22T20:59:38.007 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/ipoib/ 2024-05-22T20:59:38.007 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 334401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-22T20:59:38.007 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/iser/ 2024-05-22T20:59:38.007 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 120673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-22T20:59:38.007 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/isert/ 2024-05-22T20:59:38.007 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 132585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-22T20:59:38.007 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/srp/ 2024-05-22T20:59:38.007 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 157937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-22T20:59:38.007 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/srpt/ 2024-05-22T20:59:38.008 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 156673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-22T20:59:38.008 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/ 2024-05-22T20:59:38.008 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/evbug.ko 2024-05-22T20:59:38.008 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/ff-memless.ko 2024-05-22T20:59:38.008 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/gameport/ 2024-05-22T20:59:38.008 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-22T20:59:38.008 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-22T20:59:38.008 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 50713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/gameport/gameport.ko 2024-05-22T20:59:38.008 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 49441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/joydev.ko 2024-05-22T20:59:38.008 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/ 2024-05-22T20:59:38.008 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-05-22T20:59:38.008 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-05-22T20:59:38.008 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-05-22T20:59:38.009 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-05-22T20:59:38.009 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 45473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-22T20:59:38.009 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-05-22T20:59:38.009 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/lkkbd.ko 2024-05-22T20:59:38.009 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/lm8323.ko 2024-05-22T20:59:38.009 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/lm8333.ko 2024-05-22T20:59:38.009 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-05-22T20:59:38.009 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-05-22T20:59:38.009 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-05-22T20:59:38.009 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-05-22T20:59:38.009 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/newtonkbd.ko 2024-05-22T20:59:38.009 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-05-22T20:59:38.009 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/qt1070.ko 2024-05-22T20:59:38.010 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/qt2160.ko 2024-05-22T20:59:38.010 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-05-22T20:59:38.010 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/stowaway.ko 2024-05-22T20:59:38.010 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/sunkbd.ko 2024-05-22T20:59:38.010 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-05-22T20:59:38.010 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-05-22T20:59:38.010 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-05-22T20:59:38.010 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/xtkbd.ko 2024-05-22T20:59:38.010 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/matrix-keymap.ko 2024-05-22T20:59:38.010 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/ 2024-05-22T20:59:38.010 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/appletouch.ko 2024-05-22T20:59:38.011 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 39553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/bcm5974.ko 2024-05-22T20:59:38.011 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 160017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/cyapatp.ko 2024-05-22T20:59:38.011 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 114273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-22T20:59:38.011 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/gpio_mouse.ko 2024-05-22T20:59:38.011 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 428185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/psmouse.ko 2024-05-22T20:59:38.011 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/sermouse.ko 2024-05-22T20:59:38.011 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-22T20:59:38.011 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-22T20:59:38.011 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-22T20:59:38.011 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/rmi4/ 2024-05-22T20:59:38.011 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 140969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-22T20:59:38.011 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/ 2024-05-22T20:59:38.011 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/altera_ps2.ko 2024-05-22T20:59:38.012 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/arc_ps2.ko 2024-05-22T20:59:38.012 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/ct82c710.ko 2024-05-22T20:59:38.012 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-22T20:59:38.012 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/parkbd.ko 2024-05-22T20:59:38.012 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/pcips2.ko 2024-05-22T20:59:38.012 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/ps2mult.ko 2024-05-22T20:59:38.012 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/serio_raw.ko 2024-05-22T20:59:38.012 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/serport.ko 2024-05-22T20:59:38.012 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/sparse-keymap.ko 2024-05-22T20:59:38.012 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/ 2024-05-22T20:59:38.012 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-88pm860x.ko 2024-05-22T20:59:38.012 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-adp5520.ko 2024-05-22T20:59:38.012 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 77265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-bd2802.ko 2024-05-22T20:59:38.013 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-blinkm.ko 2024-05-22T20:59:38.013 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11897 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-da903x.ko 2024-05-22T20:59:38.013 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-da9052.ko 2024-05-22T20:59:38.013 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-gpio.ko 2024-05-22T20:59:38.013 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-lm3530.ko 2024-05-22T20:59:38.013 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-lm3533.ko 2024-05-22T20:59:38.013 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-lm355x.ko 2024-05-22T20:59:38.013 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-lm3642.ko 2024-05-22T20:59:38.013 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-lp3944.ko 2024-05-22T20:59:38.013 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-lp8788.ko 2024-05-22T20:59:38.013 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-max8997.ko 2024-05-22T20:59:38.013 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-mc13783.ko 2024-05-22T20:59:38.013 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-menf21bmc.ko 2024-05-22T20:59:38.014 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-pca9532.ko 2024-05-22T20:59:38.014 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-pca955x.ko 2024-05-22T20:59:38.014 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-pca963x.ko 2024-05-22T20:59:38.014 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 38113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-ss4200.ko 2024-05-22T20:59:38.014 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-tca6507.ko 2024-05-22T20:59:38.014 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-wm831x-status.ko 2024-05-22T20:59:38.014 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-wm8350.ko 2024-05-22T20:59:38.014 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ 2024-05-22T20:59:38.014 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-22T20:59:38.014 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-22T20:59:38.014 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-22T20:59:38.014 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-22T20:59:38.015 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-22T20:59:38.015 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-22T20:59:38.015 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-22T20:59:38.015 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-22T20:59:38.015 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-22T20:59:38.015 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/ 2024-05-22T20:59:38.015 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/bcache/ 2024-05-22T20:59:38.015 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 790753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/bcache/bcache.ko 2024-05-22T20:59:38.015 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 49081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-bio-prison.ko 2024-05-22T20:59:38.015 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 116825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-bufio.ko 2024-05-22T20:59:38.015 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 58281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-cache-smq.ko 2024-05-22T20:59:38.015 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 199041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-cache.ko 2024-05-22T20:59:38.015 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 134945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-crypt.ko 2024-05-22T20:59:38.016 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28897 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-delay.ko 2024-05-22T20:59:38.016 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 67497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-era.ko 2024-05-22T20:59:38.016 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-flakey.ko 2024-05-22T20:59:38.016 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 50457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-log-userspace.ko 2024-05-22T20:59:38.016 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 41145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-log.ko 2024-05-22T20:59:38.016 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 60977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-mirror.ko 2024-05-22T20:59:38.016 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 97209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-multipath.ko 2024-05-22T20:59:38.016 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-queue-length.ko 2024-05-22T20:59:38.016 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 85689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-raid.ko 2024-05-22T20:59:38.016 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-region-hash.ko 2024-05-22T20:59:38.016 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-round-robin.ko 2024-05-22T20:59:38.016 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-service-time.ko 2024-05-22T20:59:38.016 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 139817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-snapshot.ko 2024-05-22T20:59:38.017 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-switch.ko 2024-05-22T20:59:38.017 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 234457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-thin-pool.ko 2024-05-22T20:59:38.017 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 66425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-verity.ko 2024-05-22T20:59:38.017 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-zero.ko 2024-05-22T20:59:38.017 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/persistent-data/ 2024-05-22T20:59:38.017 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 264329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-22T20:59:38.017 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/raid0.ko 2024-05-22T20:59:38.017 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 125009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/raid1.ko 2024-05-22T20:59:38.017 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 157681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/raid10.ko 2024-05-22T20:59:38.017 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 444665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/raid456.ko 2024-05-22T20:59:38.017 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/ 2024-05-22T20:59:38.017 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/ 2024-05-22T20:59:38.017 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 214841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptbase.ko 2024-05-22T20:59:38.018 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 77985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptctl.ko 2024-05-22T20:59:38.018 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 54785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptfc.ko 2024-05-22T20:59:38.018 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptlan.ko 2024-05-22T20:59:38.018 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 150761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptsas.ko 2024-05-22T20:59:38.018 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 97281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptscsih.ko 2024-05-22T20:59:38.018 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 63873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptspi.ko 2024-05-22T20:59:38.018 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/ 2024-05-22T20:59:38.018 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/88pm800.ko 2024-05-22T20:59:38.018 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/88pm805.ko 2024-05-22T20:59:38.018 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/88pm80x.ko 2024-05-22T20:59:38.018 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/arizona-i2c.ko 2024-05-22T20:59:38.018 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 148633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/arizona.ko 2024-05-22T20:59:38.018 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/bcm590xx.ko 2024-05-22T20:59:38.019 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/cros_ec_dev.ko 2024-05-22T20:59:38.019 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 41009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/dln2.ko 2024-05-22T20:59:38.019 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/janz-cmodio.ko 2024-05-22T20:59:38.019 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 59089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/kempld-core.ko 2024-05-22T20:59:38.019 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31897 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/lm3533-core.ko 2024-05-22T20:59:38.019 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-05-22T20:59:38.019 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/lp3943.ko 2024-05-22T20:59:38.019 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 60497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/lpc_ich.ko 2024-05-22T20:59:38.019 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/lpc_sch.ko 2024-05-22T20:59:38.019 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/max8907.ko 2024-05-22T20:59:38.019 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/mc13xxx-core.ko 2024-05-22T20:59:38.019 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-05-22T20:59:38.019 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/menf21bmc.ko 2024-05-22T20:59:38.019 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/pcf50633-adc.ko 2024-05-22T20:59:38.020 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/pcf50633-gpio.ko 2024-05-22T20:59:38.020 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 42297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/pcf50633.ko 2024-05-22T20:59:38.020 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-05-22T20:59:38.020 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/retu-mfd.ko 2024-05-22T20:59:38.020 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 99425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/si476x-core.ko 2024-05-22T20:59:38.020 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 58057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/sm501.ko 2024-05-22T20:59:38.020 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 60681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/tps65010.ko 2024-05-22T20:59:38.020 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/tps6507x.ko 2024-05-22T20:59:38.020 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/viperboard.ko 2024-05-22T20:59:38.020 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/vx855.ko 2024-05-22T20:59:38.020 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/wl1273-core.ko 2024-05-22T20:59:38.020 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 52361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/wm8994.ko 2024-05-22T20:59:38.020 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ 2024-05-22T20:59:38.021 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-05-22T20:59:38.021 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 64265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ad525x_dpot.ko 2024-05-22T20:59:38.021 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/altera-stapl/ 2024-05-22T20:59:38.021 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 57521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-22T20:59:38.021 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/apds9802als.ko 2024-05-22T20:59:38.021 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 47689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/apds990x.ko 2024-05-22T20:59:38.021 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 58633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/bh1770glc.ko 2024-05-22T20:59:38.021 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/c2port/ 2024-05-22T20:59:38.021 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-05-22T20:59:38.021 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 47449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/c2port/core.ko 2024-05-22T20:59:38.021 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/cb710/ 2024-05-22T20:59:38.021 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/cb710/cb710.ko 2024-05-22T20:59:38.021 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ds1682.ko 2024-05-22T20:59:38.022 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/dummy-irq.ko 2024-05-22T20:59:38.022 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/echo/ 2024-05-22T20:59:38.022 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/echo/echo.ko 2024-05-22T20:59:38.022 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/eeprom/ 2024-05-22T20:59:38.022 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/eeprom/at24.ko 2024-05-22T20:59:38.022 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-22T20:59:38.022 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/eeprom/max6875.ko 2024-05-22T20:59:38.022 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 41185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/enclosure.ko 2024-05-22T20:59:38.022 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/genwqe/ 2024-05-22T20:59:38.022 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 191649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-05-22T20:59:38.022 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/hmc6352.ko 2024-05-22T20:59:38.022 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 37761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/hpilo.ko 2024-05-22T20:59:38.022 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ibmasm/ 2024-05-22T20:59:38.023 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 92345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-05-22T20:59:38.023 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ics932s401.ko 2024-05-22T20:59:38.023 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/isl29003.ko 2024-05-22T20:59:38.023 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/isl29020.ko 2024-05-22T20:59:38.023 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/lis3lv02d/ 2024-05-22T20:59:38.023 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 54865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-22T20:59:38.023 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-22T20:59:38.023 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/mei/ 2024-05-22T20:59:38.023 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 154169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/mei/mei-me.ko 2024-05-22T20:59:38.023 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 69809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/mei/mei-txe.ko 2024-05-22T20:59:38.023 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 389033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/mei/mei.ko 2024-05-22T20:59:38.023 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 37961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/phantom.ko 2024-05-22T20:59:38.024 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ti-st/ 2024-05-22T20:59:38.024 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 91145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ti-st/st_drv.ko 2024-05-22T20:59:38.024 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/tifm_7xx1.ko 2024-05-22T20:59:38.024 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/tifm_core.ko 2024-05-22T20:59:38.024 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/tsl2550.ko 2024-05-22T20:59:38.024 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 55793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/vmw_balloon.ko 2024-05-22T20:59:38.024 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/vmw_vmci/ 2024-05-22T20:59:38.024 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 234905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-22T20:59:38.024 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ 2024-05-22T20:59:38.024 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/bonding/ 2024-05-22T20:59:38.024 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 620961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/bonding/bonding.ko 2024-05-22T20:59:38.024 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/ 2024-05-22T20:59:38.024 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/b53/ 2024-05-22T20:59:38.025 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 139817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-22T20:59:38.025 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 49545 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-22T20:59:38.025 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-22T20:59:38.025 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/mv88e6xxx/ 2024-05-22T20:59:38.025 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 587065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-22T20:59:38.025 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dummy.ko 2024-05-22T20:59:38.025 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/eql.ko 2024-05-22T20:59:38.025 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/ 2024-05-22T20:59:38.025 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/3com/ 2024-05-22T20:59:38.025 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 53753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-22T20:59:38.025 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 49201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-22T20:59:38.025 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 158649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-22T20:59:38.025 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 85953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-22T20:59:38.025 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/8390/ 2024-05-22T20:59:38.026 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-22T20:59:38.026 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 73865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-22T20:59:38.026 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-22T20:59:38.026 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 109521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-22T20:59:38.026 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/adaptec/ 2024-05-22T20:59:38.026 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 64705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-22T20:59:38.026 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/agere/ 2024-05-22T20:59:38.026 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 84329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-22T20:59:38.026 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/alteon/ 2024-05-22T20:59:38.026 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 70737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-22T20:59:38.026 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/altera/ 2024-05-22T20:59:38.026 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 52705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-22T20:59:38.026 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/amd/ 2024-05-22T20:59:38.027 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 65857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-22T20:59:38.027 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 56057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-22T20:59:38.027 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 115881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-22T20:59:38.027 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/ 2024-05-22T20:59:38.027 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/alx/ 2024-05-22T20:59:38.027 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 124497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-22T20:59:38.027 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-05-22T20:59:38.027 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 128409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-22T20:59:38.027 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-05-22T20:59:38.027 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 110345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-22T20:59:38.027 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atlx/ 2024-05-22T20:59:38.027 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 101857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-22T20:59:38.027 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 88953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-22T20:59:38.028 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/ 2024-05-22T20:59:38.028 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 105633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-22T20:59:38.028 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 228985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-22T20:59:38.028 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-05-22T20:59:38.028 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1733353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-22T20:59:38.028 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 167185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-22T20:59:38.028 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/genet/ 2024-05-22T20:59:38.028 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 161417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-22T20:59:38.028 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 440625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-22T20:59:38.028 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/brocade/ 2024-05-22T20:59:38.028 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/brocade/bna/ 2024-05-22T20:59:38.028 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 432201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-22T20:59:38.060 INFO:teuthology.orchestra.run.smithi161.stderr:Sourcing file `/etc/default/grub' 2024-05-22T20:59:38.067 INFO:teuthology.orchestra.run.smithi161.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-22T20:59:38.093 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/ 2024-05-22T20:59:38.093 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-05-22T20:59:38.093 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 257201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-22T20:59:38.094 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-05-22T20:59:38.094 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 446177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-22T20:59:38.094 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-05-22T20:59:38.094 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1311433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-22T20:59:38.094 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-05-22T20:59:38.094 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 206689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-22T20:59:38.094 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-05-22T20:59:38.094 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-22T20:59:38.094 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/cisco/ 2024-05-22T20:59:38.094 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/cisco/enic/ 2024-05-22T20:59:38.094 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 269313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-22T20:59:38.094 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/ 2024-05-22T20:59:38.094 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/ 2024-05-22T20:59:38.095 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 85385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-22T20:59:38.095 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 77457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-22T20:59:38.095 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 169089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-22T20:59:38.095 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 65913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-22T20:59:38.095 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 76209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-22T20:59:38.095 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 38065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-22T20:59:38.095 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dlink/ 2024-05-22T20:59:38.095 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 75945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-22T20:59:38.095 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 78705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-22T20:59:38.095 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 52313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dnet.ko 2024-05-22T20:59:38.095 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/ec_bhf.ko 2024-05-22T20:59:38.095 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/emulex/ 2024-05-22T20:59:38.096 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/emulex/benet/ 2024-05-22T20:59:38.096 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 356377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-22T20:59:38.096 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 61737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/ethoc.ko 2024-05-22T20:59:38.096 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 63185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/fealnx.ko 2024-05-22T20:59:38.096 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/fujitsu/ 2024-05-22T20:59:38.096 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 51513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-05-22T20:59:38.096 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/ 2024-05-22T20:59:38.096 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 115569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-22T20:59:38.096 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/e1000/ 2024-05-22T20:59:38.096 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 366217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-22T20:59:38.096 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/e1000e/ 2024-05-22T20:59:38.096 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 755793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-22T20:59:38.096 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/fm10k/ 2024-05-22T20:59:38.097 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 324577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-22T20:59:38.097 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/i40e/ 2024-05-22T20:59:38.097 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1196297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-22T20:59:38.097 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/iavf/ 2024-05-22T20:59:38.097 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 362593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-22T20:59:38.097 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/igb/ 2024-05-22T20:59:38.097 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 671801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-22T20:59:38.097 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/igbvf/ 2024-05-22T20:59:38.097 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 121537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-22T20:59:38.097 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-05-22T20:59:38.097 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 966705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-22T20:59:38.097 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-05-22T20:59:38.097 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 193585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-22T20:59:38.098 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 108633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/jme.ko 2024-05-22T20:59:38.098 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/marvell/ 2024-05-22T20:59:38.098 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-22T20:59:38.098 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 126881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-22T20:59:38.098 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 150777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-22T20:59:38.098 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/ 2024-05-22T20:59:38.098 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-05-22T20:59:38.098 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 905033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-22T20:59:38.098 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 344145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-22T20:59:38.098 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-05-22T20:59:38.098 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-05-22T20:59:38.098 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1412417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-22T20:59:38.098 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/micrel/ 2024-05-22T20:59:38.099 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 65913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-05-22T20:59:38.099 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 54497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-05-22T20:59:38.099 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-05-22T20:59:38.099 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 117353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-22T20:59:38.099 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/myricom/ 2024-05-22T20:59:38.099 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-05-22T20:59:38.099 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 131425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-22T20:59:38.099 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/natsemi/ 2024-05-22T20:59:38.099 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 91465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-22T20:59:38.099 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 56689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-22T20:59:38.099 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/neterion/ 2024-05-22T20:59:38.099 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 181081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-22T20:59:38.099 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/nvidia/ 2024-05-22T20:59:38.100 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 171417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-22T20:59:38.100 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/packetengines/ 2024-05-22T20:59:38.100 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 58249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-22T20:59:38.100 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 50121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-22T20:59:38.100 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/ 2024-05-22T20:59:38.100 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-05-22T20:59:38.100 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 300817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-22T20:59:38.100 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 105689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-22T20:59:38.100 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-05-22T20:59:38.100 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 885129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-22T20:59:38.100 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/rdc/ 2024-05-22T20:59:38.100 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 52305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-22T20:59:38.100 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/realtek/ 2024-05-22T20:59:38.101 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 81641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-22T20:59:38.101 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 100689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-22T20:59:38.101 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-22T20:59:38.101 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 264161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-22T20:59:38.101 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/samsung/ 2024-05-22T20:59:38.101 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-05-22T20:59:38.101 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 142449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-05-22T20:59:38.101 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sfc/ 2024-05-22T20:59:38.101 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1350385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-22T20:59:38.101 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/silan/ 2024-05-22T20:59:38.101 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 57953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-22T20:59:38.101 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sis/ 2024-05-22T20:59:38.101 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 66505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-22T20:59:38.102 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 80065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-22T20:59:38.102 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/smsc/ 2024-05-22T20:59:38.102 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 74289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-22T20:59:38.102 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 79457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-22T20:59:38.102 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 100785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-22T20:59:38.102 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 83441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-22T20:59:38.102 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/stmicro/ 2024-05-22T20:59:38.102 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-05-22T20:59:38.102 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-05-22T20:59:38.102 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 46985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-22T20:59:38.102 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-05-22T20:59:38.102 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 644537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-22T20:59:38.102 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sun/ 2024-05-22T20:59:38.103 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 128497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-22T20:59:38.103 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 207993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-22T20:59:38.103 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 82633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-22T20:59:38.103 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 91353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-22T20:59:38.103 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/tehuti/ 2024-05-22T20:59:38.103 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 59865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-22T20:59:38.103 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/ti/ 2024-05-22T20:59:38.103 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 110729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-22T20:59:38.103 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/via/ 2024-05-22T20:59:38.103 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 98689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-22T20:59:38.103 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 102513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-22T20:59:38.103 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/wiznet/ 2024-05-22T20:59:38.103 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 61497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-22T20:59:38.104 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-22T20:59:38.104 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/xircom/ 2024-05-22T20:59:38.104 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 69913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-22T20:59:38.104 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/fddi/ 2024-05-22T20:59:38.104 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 49449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/fddi/defxx.ko 2024-05-22T20:59:38.104 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/fddi/skfp/ 2024-05-22T20:59:38.104 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 275801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/fddi/skfp/skfp.ko 2024-05-22T20:59:38.104 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 91105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/geneve.ko 2024-05-22T20:59:38.104 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/hyperv/ 2024-05-22T20:59:38.104 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 290441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-22T20:59:38.104 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ieee802154/ 2024-05-22T20:59:38.104 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-22T20:59:38.104 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ifb.ko 2024-05-22T20:59:38.105 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ipvlan/ 2024-05-22T20:59:38.105 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 108329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-22T20:59:38.105 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 78065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/macvlan.ko 2024-05-22T20:59:38.105 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/macvtap.ko 2024-05-22T20:59:38.105 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio/ 2024-05-22T20:59:38.105 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-22T20:59:38.105 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-22T20:59:38.105 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio/mdio-gpio.ko 2024-05-22T20:59:38.105 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-22T20:59:38.105 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio.ko 2024-05-22T20:59:38.105 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mii.ko 2024-05-22T20:59:38.105 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 62681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/netconsole.ko 2024-05-22T20:59:38.105 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/nlmon.ko 2024-05-22T20:59:38.106 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/pcs/ 2024-05-22T20:59:38.106 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-22T20:59:38.106 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 48313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-22T20:59:38.106 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/ 2024-05-22T20:59:38.106 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/amd.ko 2024-05-22T20:59:38.106 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 61985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-22T20:59:38.106 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 52865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-22T20:59:38.106 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/bcm87xx.ko 2024-05-22T20:59:38.106 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 53409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/broadcom.ko 2024-05-22T20:59:38.106 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/cicada.ko 2024-05-22T20:59:38.106 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/davicom.ko 2024-05-22T20:59:38.106 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/et1011c.ko 2024-05-22T20:59:38.107 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/icplus.ko 2024-05-22T20:59:38.107 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/lxt.ko 2024-05-22T20:59:38.107 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 103377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/marvell.ko 2024-05-22T20:59:38.107 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 96025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/micrel.ko 2024-05-22T20:59:38.107 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/national.ko 2024-05-22T20:59:38.107 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 121105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/phylink.ko 2024-05-22T20:59:38.107 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/qsemi.ko 2024-05-22T20:59:38.107 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 62113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/realtek.ko 2024-05-22T20:59:38.107 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/smsc.ko 2024-05-22T20:59:38.107 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/ste10Xp.ko 2024-05-22T20:59:38.107 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/vitesse.ko 2024-05-22T20:59:38.107 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/plip/ 2024-05-22T20:59:38.107 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 55353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/plip/plip.ko 2024-05-22T20:59:38.108 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/ 2024-05-22T20:59:38.108 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-22T20:59:38.108 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/ppp_async.ko 2024-05-22T20:59:38.108 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-22T20:59:38.108 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-22T20:59:38.108 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-22T20:59:38.108 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 53233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/pppoe.ko 2024-05-22T20:59:38.108 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/pppox.ko 2024-05-22T20:59:38.108 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/pptp.ko 2024-05-22T20:59:38.108 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/sb1000.ko 2024-05-22T20:59:38.108 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/slip/ 2024-05-22T20:59:38.108 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 52073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/slip/slip.ko 2024-05-22T20:59:38.108 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 42409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/sungem_phy.ko 2024-05-22T20:59:38.108 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 80161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/tap.ko 2024-05-22T20:59:38.109 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/ 2024-05-22T20:59:38.109 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 137321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team.ko 2024-05-22T20:59:38.109 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-22T20:59:38.109 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-22T20:59:38.109 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-22T20:59:38.109 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team_mode_random.ko 2024-05-22T20:59:38.109 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-22T20:59:38.109 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 108089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/veth.ko 2024-05-22T20:59:38.109 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/vmxnet3/ 2024-05-22T20:59:38.109 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 176497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-22T20:59:38.109 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/vxlan/ 2024-05-22T20:59:38.109 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 383593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/vxlan/vxlan.ko 2024-05-22T20:59:38.109 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/xen-netback/ 2024-05-22T20:59:38.110 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 175537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-22T20:59:38.110 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ntb/ 2024-05-22T20:59:38.110 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ntb/ntb.ko 2024-05-22T20:59:38.110 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/nvme/ 2024-05-22T20:59:38.110 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/nvme/host/ 2024-05-22T20:59:38.110 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 335857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/nvme/host/nvme-core.ko 2024-05-22T20:59:38.110 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 126553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/nvme/host/nvme.ko 2024-05-22T20:59:38.110 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/parport/ 2024-05-22T20:59:38.110 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 168145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/parport/parport.ko 2024-05-22T20:59:38.110 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/parport/parport_cs.ko 2024-05-22T20:59:38.110 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 103889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/parport/parport_pc.ko 2024-05-22T20:59:38.110 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/parport/parport_serial.ko 2024-05-22T20:59:38.110 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/ 2024-05-22T20:59:38.111 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/hotplug/ 2024-05-22T20:59:38.111 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-22T20:59:38.111 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-05-22T20:59:38.111 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-05-22T20:59:38.111 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/pci-stub.ko 2024-05-22T20:59:38.111 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 55329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/xen-pcifront.ko 2024-05-22T20:59:38.111 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/ 2024-05-22T20:59:38.111 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/i82092.ko 2024-05-22T20:59:38.111 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 179961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/pcmcia.ko 2024-05-22T20:59:38.111 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 78161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/pcmcia_core.ko 2024-05-22T20:59:38.111 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 41945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-05-22T20:59:38.111 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/pd6729.ko 2024-05-22T20:59:38.111 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 133753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-22T20:59:38.111 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/phy/ 2024-05-22T20:59:38.112 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/phy/broadcom/ 2024-05-22T20:59:38.112 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-05-22T20:59:38.112 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/phy/samsung/ 2024-05-22T20:59:38.112 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-05-22T20:59:38.112 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pinctrl/ 2024-05-22T20:59:38.112 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pinctrl/intel/ 2024-05-22T20:59:38.112 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 78193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-22T20:59:38.112 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/ 2024-05-22T20:59:38.112 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/chrome/ 2024-05-22T20:59:38.112 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 61041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-05-22T20:59:38.112 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/chrome/cros_ec.ko 2024-05-22T20:59:38.112 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-05-22T20:59:38.112 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-05-22T20:59:38.112 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-05-22T20:59:38.113 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-05-22T20:59:38.113 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-05-22T20:59:38.113 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/ 2024-05-22T20:59:38.113 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 87481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-22T20:59:38.113 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/acerhdf.ko 2024-05-22T20:59:38.113 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 48257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-22T20:59:38.113 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 73865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-22T20:59:38.113 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-22T20:59:38.113 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 164769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-22T20:59:38.113 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 46185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-22T20:59:38.113 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 62073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-22T20:59:38.113 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-22T20:59:38.113 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 51881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-22T20:59:38.114 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-22T20:59:38.114 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 42761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/hdaps.ko 2024-05-22T20:59:38.114 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/ibm_rtl.ko 2024-05-22T20:59:38.114 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/intel/ 2024-05-22T20:59:38.114 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-22T20:59:38.114 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-05-22T20:59:38.114 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 54001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/intel_ips.ko 2024-05-22T20:59:38.114 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-22T20:59:38.114 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-22T20:59:38.114 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-22T20:59:38.114 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 51897 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-22T20:59:38.114 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-22T20:59:38.114 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-22T20:59:38.115 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-22T20:59:38.115 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-22T20:59:38.115 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-22T20:59:38.115 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 61417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/wmi.ko 2024-05-22T20:59:38.115 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/ 2024-05-22T20:59:38.115 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/ 2024-05-22T20:59:38.115 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/88pm860x_battery.ko 2024-05-22T20:59:38.115 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/88pm860x_charger.ko 2024-05-22T20:59:38.115 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 68833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/bq2415x_charger.ko 2024-05-22T20:59:38.115 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 73073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/bq24190_charger.ko 2024-05-22T20:59:38.115 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/bq24735-charger.ko 2024-05-22T20:59:38.115 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-05-22T20:59:38.115 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/da9030_battery.ko 2024-05-22T20:59:38.116 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/da9052-battery.ko 2024-05-22T20:59:38.116 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/ds2760_battery.ko 2024-05-22T20:59:38.116 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/ds2780_battery.ko 2024-05-22T20:59:38.116 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/ds2781_battery.ko 2024-05-22T20:59:38.116 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/ds2782_battery.ko 2024-05-22T20:59:38.116 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/gpio-charger.ko 2024-05-22T20:59:38.116 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/isp1704_charger.ko 2024-05-22T20:59:38.116 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/lp8727_charger.ko 2024-05-22T20:59:38.116 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/max14577_charger.ko 2024-05-22T20:59:38.116 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 37833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/max17042_battery.ko 2024-05-22T20:59:38.116 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/max8903_charger.ko 2024-05-22T20:59:38.116 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/max8925_power.ko 2024-05-22T20:59:38.116 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/pcf50633-charger.ko 2024-05-22T20:59:38.117 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/sbs-battery.ko 2024-05-22T20:59:38.117 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/test_power.ko 2024-05-22T20:59:38.117 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/tps65090-charger.ko 2024-05-22T20:59:38.117 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/wm831x_backup.ko 2024-05-22T20:59:38.117 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/wm831x_power.ko 2024-05-22T20:59:38.117 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/wm8350_power.ko 2024-05-22T20:59:38.117 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/powercap/ 2024-05-22T20:59:38.117 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 73297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-22T20:59:38.117 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-22T20:59:38.117 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/ 2024-05-22T20:59:38.117 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/clients/ 2024-05-22T20:59:38.117 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-22T20:59:38.117 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-22T20:59:38.117 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/clients/pps_parport.ko 2024-05-22T20:59:38.118 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 49817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/pps_core.ko 2024-05-22T20:59:38.118 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ptp/ 2024-05-22T20:59:38.118 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 91393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ptp/ptp.ko 2024-05-22T20:59:38.118 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ptp/ptp_kvm.ko 2024-05-22T20:59:38.118 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ras/ 2024-05-22T20:59:38.118 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ras/amd/ 2024-05-22T20:59:38.118 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ras/amd/atl/ 2024-05-22T20:59:38.118 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 67513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-22T20:59:38.118 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 47137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ras/amd/fmpm.ko 2024-05-22T20:59:38.118 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/ 2024-05-22T20:59:38.118 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-88pm80x.ko 2024-05-22T20:59:38.118 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-88pm860x.ko 2024-05-22T20:59:38.118 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-22T20:59:38.119 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-da9052.ko 2024-05-22T20:59:38.119 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-da9055.ko 2024-05-22T20:59:38.119 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-da9063.ko 2024-05-22T20:59:38.119 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-22T20:59:38.119 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 82225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-22T20:59:38.119 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-22T20:59:38.119 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-22T20:59:38.119 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-22T20:59:38.119 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-22T20:59:38.119 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-22T20:59:38.119 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-22T20:59:38.119 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-22T20:59:38.119 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-em3027.ko 2024-05-22T20:59:38.119 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-22T20:59:38.120 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-22T20:59:38.120 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-22T20:59:38.120 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-lp8788.ko 2024-05-22T20:59:38.120 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 46913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-22T20:59:38.120 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-22T20:59:38.120 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-22T20:59:38.120 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-m48t86.ko 2024-05-22T20:59:38.120 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-max6900.ko 2024-05-22T20:59:38.120 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-max8907.ko 2024-05-22T20:59:38.120 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-max8925.ko 2024-05-22T20:59:38.120 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-max8997.ko 2024-05-22T20:59:38.120 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-max8998.ko 2024-05-22T20:59:38.120 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-05-22T20:59:38.120 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-22T20:59:38.121 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-palmas.ko 2024-05-22T20:59:38.121 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 56777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-22T20:59:38.121 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf50633.ko 2024-05-22T20:59:38.121 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-22T20:59:38.121 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-22T20:59:38.121 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-22T20:59:38.121 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-22T20:59:38.121 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rc5t583.ko 2024-05-22T20:59:38.121 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-22T20:59:38.121 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-22T20:59:38.121 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-22T20:59:38.121 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-22T20:59:38.122 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-22T20:59:38.122 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-s35390a.ko 2024-05-22T20:59:38.122 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-22T20:59:38.122 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-tps6586x.ko 2024-05-22T20:59:38.122 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-tps65910.ko 2024-05-22T20:59:38.122 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-wm831x.ko 2024-05-22T20:59:38.122 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-wm8350.ko 2024-05-22T20:59:38.122 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-x1205.ko 2024-05-22T20:59:38.122 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/ 2024-05-22T20:59:38.122 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 90377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/3w-9xxx.ko 2024-05-22T20:59:38.122 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 69593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/3w-sas.ko 2024-05-22T20:59:38.122 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 65945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/3w-xxxx.ko 2024-05-22T20:59:38.122 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 164569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/BusLogic.ko 2024-05-22T20:59:38.123 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/a100u2w.ko 2024-05-22T20:59:38.123 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aacraid/ 2024-05-22T20:59:38.123 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 340721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-22T20:59:38.123 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 120337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/advansys.ko 2024-05-22T20:59:38.123 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aic7xxx/ 2024-05-22T20:59:38.123 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 468081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-22T20:59:38.123 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 380673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-22T20:59:38.123 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aic94xx/ 2024-05-22T20:59:38.123 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 243705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-05-22T20:59:38.123 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/am53c974.ko 2024-05-22T20:59:38.123 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/arcmsr/ 2024-05-22T20:59:38.123 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 135617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-22T20:59:38.123 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/atp870u.ko 2024-05-22T20:59:38.124 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/be2iscsi/ 2024-05-22T20:59:38.124 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 304993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-22T20:59:38.124 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bfa/ 2024-05-22T20:59:38.124 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1527113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bfa/bfa.ko 2024-05-22T20:59:38.124 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bnx2fc/ 2024-05-22T20:59:38.124 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 297801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-22T20:59:38.124 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bnx2i/ 2024-05-22T20:59:38.124 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 144401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-22T20:59:38.124 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 47457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/ch.ko 2024-05-22T20:59:38.124 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/csiostor/ 2024-05-22T20:59:38.124 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 390713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-22T20:59:38.124 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/ 2024-05-22T20:59:38.124 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-05-22T20:59:38.125 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 88881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-22T20:59:38.125 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-05-22T20:59:38.125 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 117481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-22T20:59:38.125 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 173017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-22T20:59:38.125 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 91569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/dc395x.ko 2024-05-22T20:59:38.125 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 45401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/dmx3191d.ko 2024-05-22T20:59:38.125 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/esas2r/ 2024-05-22T20:59:38.125 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 267577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-22T20:59:38.125 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 78969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/esp_scsi.ko 2024-05-22T20:59:38.125 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fcoe/ 2024-05-22T20:59:38.125 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 95329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-22T20:59:38.125 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 194017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-22T20:59:38.125 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fdomain.ko 2024-05-22T20:59:38.126 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fnic/ 2024-05-22T20:59:38.126 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 366529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fnic/fnic.ko 2024-05-22T20:59:38.126 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 251505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/hpsa.ko 2024-05-22T20:59:38.126 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 65761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/hptiop.ko 2024-05-22T20:59:38.126 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 57097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/hv_storvsc.ko 2024-05-22T20:59:38.126 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/imm.ko 2024-05-22T20:59:38.126 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 55153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/initio.ko 2024-05-22T20:59:38.126 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 327841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/ipr.ko 2024-05-22T20:59:38.126 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 129785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/ips.ko 2024-05-22T20:59:38.126 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/isci/ 2024-05-22T20:59:38.126 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 397785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/isci/isci.ko 2024-05-22T20:59:38.126 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-22T20:59:38.127 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 63121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-22T20:59:38.127 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libfc/ 2024-05-22T20:59:38.127 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 386849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libfc/libfc.ko 2024-05-22T20:59:38.127 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 178177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libiscsi.ko 2024-05-22T20:59:38.127 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 64809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-22T20:59:38.127 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libsas/ 2024-05-22T20:59:38.127 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 274009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libsas/libsas.ko 2024-05-22T20:59:38.127 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/lpfc/ 2024-05-22T20:59:38.127 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 2736001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-22T20:59:38.127 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/megaraid/ 2024-05-22T20:59:38.127 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 88529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-22T20:59:38.127 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-22T20:59:38.127 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 389177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-22T20:59:38.128 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 104769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/megaraid.ko 2024-05-22T20:59:38.128 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/mpt3sas/ 2024-05-22T20:59:38.128 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 836945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-22T20:59:38.128 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/mvsas/ 2024-05-22T20:59:38.128 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 158729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-22T20:59:38.128 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 96537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/mvumi.ko 2024-05-22T20:59:38.128 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pcmcia/ 2024-05-22T20:59:38.128 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 101625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-05-22T20:59:38.128 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-05-22T20:59:38.128 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-05-22T20:59:38.128 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-05-22T20:59:38.128 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pm8001/ 2024-05-22T20:59:38.128 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 443745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-22T20:59:38.128 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 152033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pmcraid.ko 2024-05-22T20:59:38.129 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 41625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/ppa.ko 2024-05-22T20:59:38.129 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 66977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla1280.ko 2024-05-22T20:59:38.129 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla2xxx/ 2024-05-22T20:59:38.129 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 2324809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-22T20:59:38.129 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 103601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-22T20:59:38.129 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla4xxx/ 2024-05-22T20:59:38.129 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 697337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-22T20:59:38.129 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qlogicfas408.ko 2024-05-22T20:59:38.129 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/raid_class.ko 2024-05-22T20:59:38.129 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 322865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_debug.ko 2024-05-22T20:59:38.129 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 219857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-22T20:59:38.129 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 357153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-22T20:59:38.129 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 121185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-22T20:59:38.130 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 93481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-22T20:59:38.130 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 55401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-22T20:59:38.130 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/ses.ko 2024-05-22T20:59:38.130 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 138121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/st.ko 2024-05-22T20:59:38.130 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 57401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/stex.ko 2024-05-22T20:59:38.130 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/sym53c8xx_2/ 2024-05-22T20:59:38.130 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 205025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-22T20:59:38.130 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 55425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/virtio_scsi.ko 2024-05-22T20:59:38.130 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 59185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-22T20:59:38.130 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 48777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/wd719x.ko 2024-05-22T20:59:38.130 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 59417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/xen-scsifront.ko 2024-05-22T20:59:38.130 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ssb/ 2024-05-22T20:59:38.130 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 174913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ssb/ssb.ko 2024-05-22T20:59:38.131 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/ 2024-05-22T20:59:38.131 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/iscsi/ 2024-05-22T20:59:38.131 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 920169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-22T20:59:38.131 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/loopback/ 2024-05-22T20:59:38.131 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 60321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-22T20:59:38.131 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 48129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/target_core_file.ko 2024-05-22T20:59:38.131 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 54233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/target_core_iblock.ko 2024-05-22T20:59:38.131 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1170033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/target_core_mod.ko 2024-05-22T20:59:38.131 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 51897 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/target_core_pscsi.ko 2024-05-22T20:59:38.131 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 149721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/target_core_user.ko 2024-05-22T20:59:38.131 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/tcm_fc/ 2024-05-22T20:59:38.131 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 90057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-22T20:59:38.131 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/ 2024-05-22T20:59:38.132 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/ 2024-05-22T20:59:38.132 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/ 2024-05-22T20:59:38.132 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-22T20:59:38.132 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-22T20:59:38.132 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-22T20:59:38.132 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-22T20:59:38.132 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-22T20:59:38.132 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-22T20:59:38.132 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-22T20:59:38.132 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-22T20:59:38.132 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-22T20:59:38.132 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12929 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-22T20:59:38.132 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-22T20:59:38.132 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-22T20:59:38.133 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 46065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-22T20:59:38.133 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-22T20:59:38.133 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-22T20:59:38.133 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-05-22T20:59:38.133 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-22T20:59:38.133 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-22T20:59:38.133 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-22T20:59:38.133 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/ 2024-05-22T20:59:38.133 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/ipwireless/ 2024-05-22T20:59:38.133 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 99313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-22T20:59:38.133 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 69729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/moxa.ko 2024-05-22T20:59:38.133 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 61649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/mxser.ko 2024-05-22T20:59:38.133 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 134401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/n_gsm.ko 2024-05-22T20:59:38.134 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 37905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/n_hdlc.ko 2024-05-22T20:59:38.134 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 78969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/nozomi.ko 2024-05-22T20:59:38.134 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/ 2024-05-22T20:59:38.134 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/8250/ 2024-05-22T20:59:38.134 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-05-22T20:59:38.134 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 65121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-22T20:59:38.134 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-05-22T20:59:38.134 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/altera_uart.ko 2024-05-22T20:59:38.134 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/arc_uart.ko 2024-05-22T20:59:38.134 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 91665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-05-22T20:59:38.134 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/jsm/ 2024-05-22T20:59:38.134 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 120577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-22T20:59:38.134 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/rp2.ko 2024-05-22T20:59:38.135 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 65641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/sc16is7xx.ko 2024-05-22T20:59:38.135 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs/ 2024-05-22T20:59:38.135 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs/core/ 2024-05-22T20:59:38.135 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 693081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-22T20:59:38.135 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs/host/ 2024-05-22T20:59:38.135 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-22T20:59:38.135 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-05-22T20:59:38.135 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/ 2024-05-22T20:59:38.135 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 55617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio.ko 2024-05-22T20:59:38.135 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_aec.ko 2024-05-22T20:59:38.135 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_cif.ko 2024-05-22T20:59:38.135 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_dmem_genirq.ko 2024-05-22T20:59:38.135 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_mf624.ko 2024-05-22T20:59:38.136 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_netx.ko 2024-05-22T20:59:38.136 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_pci_generic.ko 2024-05-22T20:59:38.136 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-05-22T20:59:38.136 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_sercos3.ko 2024-05-22T20:59:38.136 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/ 2024-05-22T20:59:38.136 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/c67x00/ 2024-05-22T20:59:38.136 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 90321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/c67x00/c67x00.ko 2024-05-22T20:59:38.136 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/chipidea/ 2024-05-22T20:59:38.136 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 189209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-05-22T20:59:38.136 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-05-22T20:59:38.136 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-05-22T20:59:38.136 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-05-22T20:59:38.136 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-05-22T20:59:38.137 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/class/ 2024-05-22T20:59:38.137 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 101545 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/class/cdc-acm.ko 2024-05-22T20:59:38.137 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 63073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-22T20:59:38.137 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 56721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/class/usblp.ko 2024-05-22T20:59:38.137 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 107417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/class/usbtmc.ko 2024-05-22T20:59:38.137 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/common/ 2024-05-22T20:59:38.137 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/common/ulpi.ko 2024-05-22T20:59:38.137 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/dwc2/ 2024-05-22T20:59:38.137 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-05-22T20:59:38.137 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/dwc3/ 2024-05-22T20:59:38.137 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-05-22T20:59:38.137 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-05-22T20:59:38.137 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 574593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/dwc3/dwc3.ko 2024-05-22T20:59:38.138 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/ 2024-05-22T20:59:38.138 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/ 2024-05-22T20:59:38.138 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 72249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/u_ether.ko 2024-05-22T20:59:38.138 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 61953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/u_serial.ko 2024-05-22T20:59:38.138 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-05-22T20:59:38.138 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 49737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-05-22T20:59:38.138 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-05-22T20:59:38.138 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-05-22T20:59:38.138 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 131345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-05-22T20:59:38.138 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 62321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-05-22T20:59:38.138 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 146617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-05-22T20:59:38.138 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 70689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-05-22T20:59:38.138 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-05-22T20:59:38.139 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 37649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-05-22T20:59:38.139 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 70017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-05-22T20:59:38.139 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 101249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-05-22T20:59:38.139 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-05-22T20:59:38.139 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 71257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-05-22T20:59:38.139 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 84937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-05-22T20:59:38.139 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/ 2024-05-22T20:59:38.139 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-05-22T20:59:38.139 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-05-22T20:59:38.139 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-05-22T20:59:38.139 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-05-22T20:59:38.139 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-05-22T20:59:38.139 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-05-22T20:59:38.140 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-05-22T20:59:38.140 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-05-22T20:59:38.140 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-05-22T20:59:38.140 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-05-22T20:59:38.140 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-05-22T20:59:38.140 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-05-22T20:59:38.140 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 78073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-05-22T20:59:38.140 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-05-22T20:59:38.140 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 204049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/libcomposite.ko 2024-05-22T20:59:38.140 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/ 2024-05-22T20:59:38.140 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-05-22T20:59:38.140 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/bdc/ 2024-05-22T20:59:38.140 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 182561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-05-22T20:59:38.140 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 45545 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-05-22T20:59:38.141 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 66513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-05-22T20:59:38.141 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 85217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-05-22T20:59:38.141 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 69313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-05-22T20:59:38.141 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 90313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/net2272.ko 2024-05-22T20:59:38.141 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 90361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/net2280.ko 2024-05-22T20:59:38.141 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 94841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-05-22T20:59:38.141 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 80521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-05-22T20:59:38.141 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 87465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-05-22T20:59:38.141 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 95441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-05-22T20:59:38.141 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 195969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-05-22T20:59:38.141 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/ 2024-05-22T20:59:38.142 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/bcma-hcd.ko 2024-05-22T20:59:38.142 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 108321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/isp116x-hcd.ko 2024-05-22T20:59:38.142 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 104377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-05-22T20:59:38.142 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 102273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-05-22T20:59:38.142 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 60105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-22T20:59:38.142 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/sl811_cs.ko 2024-05-22T20:59:38.142 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/ssb-hcd.ko 2024-05-22T20:59:38.142 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-05-22T20:59:38.142 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/image/ 2024-05-22T20:59:38.142 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/image/mdc800.ko 2024-05-22T20:59:38.142 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/image/microtek.ko 2024-05-22T20:59:38.142 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/ 2024-05-22T20:59:38.142 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 47353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/adutux.ko 2024-05-22T20:59:38.142 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/appledisplay.ko 2024-05-22T20:59:38.143 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-05-22T20:59:38.143 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/cytherm.ko 2024-05-22T20:59:38.143 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/ehset.ko 2024-05-22T20:59:38.143 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/emi26.ko 2024-05-22T20:59:38.143 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/emi62.ko 2024-05-22T20:59:38.143 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/ezusb.ko 2024-05-22T20:59:38.143 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/idmouse.ko 2024-05-22T20:59:38.143 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 42729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/iowarrior.ko 2024-05-22T20:59:38.143 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-22T20:59:38.143 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 42241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/ldusb.ko 2024-05-22T20:59:38.143 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 42233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/legousbtower.ko 2024-05-22T20:59:38.143 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/lvstest.ko 2024-05-22T20:59:38.143 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/sisusbvga/ 2024-05-22T20:59:38.144 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 85313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-22T20:59:38.144 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-22T20:59:38.144 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/usb3503.ko 2024-05-22T20:59:38.144 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/usblcd.ko 2024-05-22T20:59:38.144 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-22T20:59:38.144 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 84265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/usbtest.ko 2024-05-22T20:59:38.144 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 52065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/uss720.ko 2024-05-22T20:59:38.144 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/yurex.ko 2024-05-22T20:59:38.144 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/mon/ 2024-05-22T20:59:38.144 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 91529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/mon/usbmon.ko 2024-05-22T20:59:38.144 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/musb/ 2024-05-22T20:59:38.144 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 396209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/musb/musb_hdrc.ko 2024-05-22T20:59:38.144 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/phy/ 2024-05-22T20:59:38.145 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/phy/phy-generic.ko 2024-05-22T20:59:38.145 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-05-22T20:59:38.145 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/phy/phy-isp1301.ko 2024-05-22T20:59:38.145 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/phy/phy-tahvo.ko 2024-05-22T20:59:38.145 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ 2024-05-22T20:59:38.145 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/aircable.ko 2024-05-22T20:59:38.145 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 38209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ark3116.ko 2024-05-22T20:59:38.145 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-22T20:59:38.145 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ch341.ko 2024-05-22T20:59:38.145 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 79681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/cp210x.ko 2024-05-22T20:59:38.145 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/cyberjack.ko 2024-05-22T20:59:38.145 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 66873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-22T20:59:38.145 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 58329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-22T20:59:38.146 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/empeg.ko 2024-05-22T20:59:38.146 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 53009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/f81232.ko 2024-05-22T20:59:38.146 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 188609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-22T20:59:38.146 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 61489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-22T20:59:38.146 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 152305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-22T20:59:38.146 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 142833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/io_ti.ko 2024-05-22T20:59:38.146 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 69217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ipaq.ko 2024-05-22T20:59:38.146 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ipw.ko 2024-05-22T20:59:38.146 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ir-usb.ko 2024-05-22T20:59:38.146 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 66361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-22T20:59:38.146 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 122329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/keyspan.ko 2024-05-22T20:59:38.146 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 41297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-22T20:59:38.146 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-22T20:59:38.147 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 38049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-22T20:59:38.147 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 38041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/mct_u232.ko 2024-05-22T20:59:38.147 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/metro-usb.ko 2024-05-22T20:59:38.147 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 79681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/mos7720.ko 2024-05-22T20:59:38.147 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 89969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/mos7840.ko 2024-05-22T20:59:38.147 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 65073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/mxuport.ko 2024-05-22T20:59:38.147 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/navman.ko 2024-05-22T20:59:38.147 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/omninet.ko 2024-05-22T20:59:38.147 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/opticon.ko 2024-05-22T20:59:38.147 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 193857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/option.ko 2024-05-22T20:59:38.147 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 45337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/oti6858.ko 2024-05-22T20:59:38.147 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 61761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/pl2303.ko 2024-05-22T20:59:38.147 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/qcaux.ko 2024-05-22T20:59:38.147 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 39985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/qcserial.ko 2024-05-22T20:59:38.148 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/quatech2.ko 2024-05-22T20:59:38.148 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/safe_serial.ko 2024-05-22T20:59:38.148 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 59345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/sierra.ko 2024-05-22T20:59:38.148 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-22T20:59:38.148 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ssu100.ko 2024-05-22T20:59:38.148 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/symbolserial.ko 2024-05-22T20:59:38.148 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 70497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-22T20:59:38.148 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-22T20:59:38.148 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/usb_debug.ko 2024-05-22T20:59:38.148 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 39321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-22T20:59:38.148 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 151137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/usbserial.ko 2024-05-22T20:59:38.148 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/visor.ko 2024-05-22T20:59:38.148 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/whiteheat.ko 2024-05-22T20:59:38.148 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/wishbone-serial.ko 2024-05-22T20:59:38.149 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-22T20:59:38.149 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ 2024-05-22T20:59:38.149 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 64049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/uas.ko 2024-05-22T20:59:38.149 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-22T20:59:38.149 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-22T20:59:38.149 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-22T20:59:38.149 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 41073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-22T20:59:38.149 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-22T20:59:38.149 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-22T20:59:38.149 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-22T20:59:38.149 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-karma.ko 2024-05-22T20:59:38.149 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-22T20:59:38.149 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-22T20:59:38.150 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-22T20:59:38.150 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-22T20:59:38.150 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 41785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-22T20:59:38.150 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 201193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/usb-storage.ko 2024-05-22T20:59:38.150 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/usbip/ 2024-05-22T20:59:38.150 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 76041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-22T20:59:38.150 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 85257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-22T20:59:38.150 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 122321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-22T20:59:38.150 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/ 2024-05-22T20:59:38.150 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/pci/ 2024-05-22T20:59:38.150 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 206921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-22T20:59:38.150 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-22T20:59:38.150 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 134265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/vfio.ko 2024-05-22T20:59:38.151 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 93673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-22T20:59:38.151 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vhost/ 2024-05-22T20:59:38.151 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 138721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vhost/vhost.ko 2024-05-22T20:59:38.151 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-22T20:59:38.151 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 71281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vhost/vhost_net.ko 2024-05-22T20:59:38.151 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 107905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vhost/vhost_scsi.ko 2024-05-22T20:59:38.151 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/ 2024-05-22T20:59:38.151 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/ 2024-05-22T20:59:38.151 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-05-22T20:59:38.151 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/aat2870_bl.ko 2024-05-22T20:59:38.151 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/adp5520_bl.ko 2024-05-22T20:59:38.151 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 41169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/adp8860_bl.ko 2024-05-22T20:59:38.151 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 51745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/adp8870_bl.ko 2024-05-22T20:59:38.152 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/apple_bl.ko 2024-05-22T20:59:38.152 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/as3711_bl.ko 2024-05-22T20:59:38.152 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/bd6107.ko 2024-05-22T20:59:38.152 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/da903x_bl.ko 2024-05-22T20:59:38.152 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/da9052_bl.ko 2024-05-22T20:59:38.152 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/gpio_backlight.ko 2024-05-22T20:59:38.152 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/kb3886_bl.ko 2024-05-22T20:59:38.152 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/lcd.ko 2024-05-22T20:59:38.152 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/lm3533_bl.ko 2024-05-22T20:59:38.152 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/lm3639_bl.ko 2024-05-22T20:59:38.152 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/lv5207lp.ko 2024-05-22T20:59:38.152 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/max8925_bl.ko 2024-05-22T20:59:38.152 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/pandora_bl.ko 2024-05-22T20:59:38.153 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-05-22T20:59:38.153 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-22T20:59:38.153 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/wm831x_bl.ko 2024-05-22T20:59:38.153 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/ 2024-05-22T20:59:38.153 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/masters/ 2024-05-22T20:59:38.153 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/masters/ds2482.ko 2024-05-22T20:59:38.153 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 46801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/masters/ds2490.ko 2024-05-22T20:59:38.153 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/masters/matrox_w1.ko 2024-05-22T20:59:38.153 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/masters/w1-gpio.ko 2024-05-22T20:59:38.153 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/ 2024-05-22T20:59:38.153 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-22T20:59:38.153 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-22T20:59:38.154 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-22T20:59:38.154 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-22T20:59:38.154 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-22T20:59:38.154 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-22T20:59:38.154 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-22T20:59:38.154 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-22T20:59:38.154 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-22T20:59:38.154 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-22T20:59:38.154 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 70369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-22T20:59:38.154 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 129433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/wire.ko 2024-05-22T20:59:38.154 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/ 2024-05-22T20:59:38.155 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/acquirewdt.ko 2024-05-22T20:59:38.155 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/advantechwdt.ko 2024-05-22T20:59:38.155 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-22T20:59:38.155 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-22T20:59:38.155 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/cpu5wdt.ko 2024-05-22T20:59:38.155 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/da9052_wdt.ko 2024-05-22T20:59:38.155 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/da9055_wdt.ko 2024-05-22T20:59:38.155 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/da9063_wdt.ko 2024-05-22T20:59:38.155 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/dw_wdt.ko 2024-05-22T20:59:38.155 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/eurotechwdt.ko 2024-05-22T20:59:38.155 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-22T20:59:38.155 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/hpwdt.ko 2024-05-22T20:59:38.155 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/i6300esb.ko 2024-05-22T20:59:38.155 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-22T20:59:38.156 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-22T20:59:38.156 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/ib700wdt.ko 2024-05-22T20:59:38.156 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/ibmasr.ko 2024-05-22T20:59:38.156 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-22T20:59:38.156 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-22T20:59:38.156 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/it87_wdt.ko 2024-05-22T20:59:38.156 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/kempld_wdt.ko 2024-05-22T20:59:38.156 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/machzwd.ko 2024-05-22T20:59:38.156 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/mena21_wdt.ko 2024-05-22T20:59:38.156 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-05-22T20:59:38.156 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/nv_tco.ko 2024-05-22T20:59:38.156 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-05-22T20:59:38.156 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/pc87413_wdt.ko 2024-05-22T20:59:38.157 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 46953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-22T20:59:38.157 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 37193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-22T20:59:38.157 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/retu_wdt.ko 2024-05-22T20:59:38.157 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-05-22T20:59:38.157 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-05-22T20:59:38.157 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-22T20:59:38.157 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/sc1200wdt.ko 2024-05-22T20:59:38.157 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-22T20:59:38.157 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-05-22T20:59:38.157 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/softdog.ko 2024-05-22T20:59:38.157 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-22T20:59:38.157 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/twl4030_wdt.ko 2024-05-22T20:59:38.158 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/via_wdt.ko 2024-05-22T20:59:38.158 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-22T20:59:38.158 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-22T20:59:38.158 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-22T20:59:38.158 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/wafer5823wdt.ko 2024-05-22T20:59:38.158 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/wdt_pci.ko 2024-05-22T20:59:38.158 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/wm831x_wdt.ko 2024-05-22T20:59:38.158 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/wm8350_wdt.ko 2024-05-22T20:59:38.158 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/xen_wdt.ko 2024-05-22T20:59:38.158 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/ 2024-05-22T20:59:38.158 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-evtchn.ko 2024-05-22T20:59:38.158 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-gntalloc.ko 2024-05-22T20:59:38.158 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 55713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-gntdev.ko 2024-05-22T20:59:38.159 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-pciback/ 2024-05-22T20:59:38.159 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 213913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-22T20:59:38.159 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 47689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-privcmd.ko 2024-05-22T20:59:38.159 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 85713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-scsiback.ko 2024-05-22T20:59:38.159 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xenfs/ 2024-05-22T20:59:38.159 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-22T20:59:38.159 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ 2024-05-22T20:59:38.159 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/9p/ 2024-05-22T20:59:38.159 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 192617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/9p/9p.ko 2024-05-22T20:59:38.159 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/adfs/ 2024-05-22T20:59:38.159 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 78153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/adfs/adfs.ko 2024-05-22T20:59:38.159 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/affs/ 2024-05-22T20:59:38.160 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 197849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/affs/affs.ko 2024-05-22T20:59:38.160 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/afs/ 2024-05-22T20:59:38.160 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1261113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/afs/kafs.ko 2024-05-22T20:59:38.160 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/autofs/ 2024-05-22T20:59:38.160 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 127377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/autofs/autofs4.ko 2024-05-22T20:59:38.160 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/befs/ 2024-05-22T20:59:38.160 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 80521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/befs/befs.ko 2024-05-22T20:59:38.160 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/bfs/ 2024-05-22T20:59:38.160 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 50473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/bfs/bfs.ko 2024-05-22T20:59:38.160 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 45561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/binfmt_misc.ko 2024-05-22T20:59:38.160 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/btrfs/ 2024-05-22T20:59:38.160 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 4489593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/btrfs/btrfs.ko 2024-05-22T20:59:38.161 INFO:teuthology.orchestra.run.smithi161.stderr:Generating grub configuration file ... 2024-05-22T20:59:38.204 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/cachefiles/ 2024-05-22T20:59:38.204 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 376857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/cachefiles/cachefiles.ko 2024-05-22T20:59:38.204 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ceph/ 2024-05-22T20:59:38.205 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1587233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ceph/ceph.ko 2024-05-22T20:59:38.205 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/coda/ 2024-05-22T20:59:38.205 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 130073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/coda/coda.ko 2024-05-22T20:59:38.205 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/configfs/ 2024-05-22T20:59:38.205 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 139081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/configfs/configfs.ko 2024-05-22T20:59:38.205 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/cramfs/ 2024-05-22T20:59:38.205 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/cramfs/cramfs.ko 2024-05-22T20:59:38.205 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/dlm/ 2024-05-22T20:59:38.205 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 813225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/dlm/dlm.ko 2024-05-22T20:59:38.205 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/efs/ 2024-05-22T20:59:38.205 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 46409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/efs/efs.ko 2024-05-22T20:59:38.205 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/f2fs/ 2024-05-22T20:59:38.205 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1906817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/f2fs/f2fs.ko 2024-05-22T20:59:38.205 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/fat/ 2024-05-22T20:59:38.206 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/fat/msdos.ko 2024-05-22T20:59:38.206 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/freevxfs/ 2024-05-22T20:59:38.206 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 52313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/freevxfs/freevxfs.ko 2024-05-22T20:59:38.206 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/fuse/ 2024-05-22T20:59:38.206 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/fuse/cuse.ko 2024-05-22T20:59:38.206 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/gfs2/ 2024-05-22T20:59:38.206 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1486521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/gfs2/gfs2.ko 2024-05-22T20:59:38.206 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/hfs/ 2024-05-22T20:59:38.206 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 183305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/hfs/hfs.ko 2024-05-22T20:59:38.206 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/hfsplus/ 2024-05-22T20:59:38.206 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 282097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/hfsplus/hfsplus.ko 2024-05-22T20:59:38.207 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/hpfs/ 2024-05-22T20:59:38.207 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 249009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/hpfs/hpfs.ko 2024-05-22T20:59:38.207 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/isofs/ 2024-05-22T20:59:38.207 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 93033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/isofs/isofs.ko 2024-05-22T20:59:38.207 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/jfs/ 2024-05-22T20:59:38.207 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 533073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/jfs/jfs.ko 2024-05-22T20:59:38.207 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/lockd/ 2024-05-22T20:59:38.207 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 361369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/lockd/lockd.ko 2024-05-22T20:59:38.207 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/minix/ 2024-05-22T20:59:38.207 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 112609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/minix/minix.ko 2024-05-22T20:59:38.207 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/netfs/ 2024-05-22T20:59:38.207 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 591585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/netfs/netfs.ko 2024-05-22T20:59:38.207 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/ 2024-05-22T20:59:38.208 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/blocklayout/ 2024-05-22T20:59:38.208 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 102561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-22T20:59:38.208 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/filelayout/ 2024-05-22T20:59:38.208 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 74353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-22T20:59:38.208 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/flexfilelayout/ 2024-05-22T20:59:38.208 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 147689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-22T20:59:38.208 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1320249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/nfs.ko 2024-05-22T20:59:38.208 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 82105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/nfsv2.ko 2024-05-22T20:59:38.208 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 151641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/nfsv3.ko 2024-05-22T20:59:38.208 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 2518505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/nfsv4.ko 2024-05-22T20:59:38.208 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs_common/ 2024-05-22T20:59:38.208 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs_common/grace.ko 2024-05-22T20:59:38.208 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs_common/nfs_acl.ko 2024-05-22T20:59:38.208 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfsd/ 2024-05-22T20:59:38.209 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 2024049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfsd/nfsd.ko 2024-05-22T20:59:38.209 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nilfs2/ 2024-05-22T20:59:38.209 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 734025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nilfs2/nilfs2.ko 2024-05-22T20:59:38.209 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/ 2024-05-22T20:59:38.209 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-celtic.ko 2024-05-22T20:59:38.209 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-centeuro.ko 2024-05-22T20:59:38.209 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-croatian.ko 2024-05-22T20:59:38.209 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-cyrillic.ko 2024-05-22T20:59:38.209 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-gaelic.ko 2024-05-22T20:59:38.209 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-greek.ko 2024-05-22T20:59:38.209 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-iceland.ko 2024-05-22T20:59:38.209 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-inuit.ko 2024-05-22T20:59:38.209 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-roman.ko 2024-05-22T20:59:38.210 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-romanian.ko 2024-05-22T20:59:38.210 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-turkish.ko 2024-05-22T20:59:38.210 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_ascii.ko 2024-05-22T20:59:38.210 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp1250.ko 2024-05-22T20:59:38.210 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp1251.ko 2024-05-22T20:59:38.210 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp1255.ko 2024-05-22T20:59:38.210 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp737.ko 2024-05-22T20:59:38.210 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp775.ko 2024-05-22T20:59:38.210 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp850.ko 2024-05-22T20:59:38.210 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp852.ko 2024-05-22T20:59:38.210 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp855.ko 2024-05-22T20:59:38.210 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp857.ko 2024-05-22T20:59:38.210 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp860.ko 2024-05-22T20:59:38.211 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp861.ko 2024-05-22T20:59:38.211 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp862.ko 2024-05-22T20:59:38.211 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp863.ko 2024-05-22T20:59:38.211 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp864.ko 2024-05-22T20:59:38.211 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp865.ko 2024-05-22T20:59:38.211 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp866.ko 2024-05-22T20:59:38.211 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp869.ko 2024-05-22T20:59:38.211 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp874.ko 2024-05-22T20:59:38.211 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 95625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp932.ko 2024-05-22T20:59:38.211 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 144209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp936.ko 2024-05-22T20:59:38.211 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 167137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp949.ko 2024-05-22T20:59:38.211 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 119513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp950.ko 2024-05-22T20:59:38.211 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_euc-jp.ko 2024-05-22T20:59:38.212 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-1.ko 2024-05-22T20:59:38.212 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-13.ko 2024-05-22T20:59:38.212 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-14.ko 2024-05-22T20:59:38.212 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-15.ko 2024-05-22T20:59:38.212 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-2.ko 2024-05-22T20:59:38.212 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-3.ko 2024-05-22T20:59:38.212 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-4.ko 2024-05-22T20:59:38.212 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-5.ko 2024-05-22T20:59:38.212 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-6.ko 2024-05-22T20:59:38.212 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-7.ko 2024-05-22T20:59:38.212 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-9.ko 2024-05-22T20:59:38.212 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_koi8-r.ko 2024-05-22T20:59:38.212 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_koi8-ru.ko 2024-05-22T20:59:38.213 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_koi8-u.ko 2024-05-22T20:59:38.213 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 7521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-22T20:59:38.213 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_utf8.ko 2024-05-22T20:59:38.213 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/ 2024-05-22T20:59:38.213 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/cluster/ 2024-05-22T20:59:38.213 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 302073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-22T20:59:38.213 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/dlm/ 2024-05-22T20:59:38.213 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 548169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-22T20:59:38.213 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/dlmfs/ 2024-05-22T20:59:38.213 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 66361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-22T20:59:38.213 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 3245617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/ocfs2.ko 2024-05-22T20:59:38.213 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 22113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-22T20:59:38.213 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 39313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-22T20:59:38.213 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-22T20:59:38.214 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/omfs/ 2024-05-22T20:59:38.214 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 67489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/omfs/omfs.ko 2024-05-22T20:59:38.214 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/overlayfs/ 2024-05-22T20:59:38.214 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 462873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/overlayfs/overlay.ko 2024-05-22T20:59:38.214 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/pstore/ 2024-05-22T20:59:38.214 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 67977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/pstore/ramoops.ko 2024-05-22T20:59:38.214 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/qnx4/ 2024-05-22T20:59:38.214 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/qnx4/qnx4.ko 2024-05-22T20:59:38.214 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/qnx6/ 2024-05-22T20:59:38.214 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 56097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/qnx6/qnx6.ko 2024-05-22T20:59:38.214 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/quota/ 2024-05-22T20:59:38.214 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/quota/quota_tree.ko 2024-05-22T20:59:38.215 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/quota/quota_v1.ko 2024-05-22T20:59:38.215 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/quota/quota_v2.ko 2024-05-22T20:59:38.215 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/reiserfs/ 2024-05-22T20:59:38.215 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 684841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/reiserfs/reiserfs.ko 2024-05-22T20:59:38.215 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/romfs/ 2024-05-22T20:59:38.215 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/romfs/romfs.ko 2024-05-22T20:59:38.215 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/ 2024-05-22T20:59:38.215 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/client/ 2024-05-22T20:59:38.215 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 3319041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/client/cifs.ko 2024-05-22T20:59:38.215 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/common/ 2024-05-22T20:59:38.215 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/common/cifs_arc4.ko 2024-05-22T20:59:38.215 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/common/cifs_md4.ko 2024-05-22T20:59:38.215 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/squashfs/ 2024-05-22T20:59:38.216 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 164185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/squashfs/squashfs.ko 2024-05-22T20:59:38.216 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/sysv/ 2024-05-22T20:59:38.216 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 115977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/sysv/sysv.ko 2024-05-22T20:59:38.216 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/udf/ 2024-05-22T20:59:38.216 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 310017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/udf/udf.ko 2024-05-22T20:59:38.216 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ufs/ 2024-05-22T20:59:38.216 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 198233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ufs/ufs.ko 2024-05-22T20:59:38.216 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/xfs/ 2024-05-22T20:59:38.216 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 5192705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/xfs/xfs.ko 2024-05-22T20:59:38.216 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/ 2024-05-22T20:59:38.216 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/locking/ 2024-05-22T20:59:38.216 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 97625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/locking/locktorture.ko 2024-05-22T20:59:38.216 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/time/ 2024-05-22T20:59:38.216 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/time/test_udelay.ko 2024-05-22T20:59:38.217 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 72665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/torture.ko 2024-05-22T20:59:38.217 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/ 2024-05-22T20:59:38.217 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 7313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crc-itu-t.ko 2024-05-22T20:59:38.217 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 7081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crc7.ko 2024-05-22T20:59:38.217 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 7953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crc8.ko 2024-05-22T20:59:38.217 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crypto/ 2024-05-22T20:59:38.217 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crypto/libarc4.ko 2024-05-22T20:59:38.217 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 38985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crypto/libdes.ko 2024-05-22T20:59:38.217 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/interval_tree_test.ko 2024-05-22T20:59:38.217 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/libcrc32c.ko 2024-05-22T20:59:38.217 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/lru_cache.ko 2024-05-22T20:59:38.217 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/lz4/ 2024-05-22T20:59:38.217 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/lz4/lz4_compress.ko 2024-05-22T20:59:38.218 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/lz4/lz4hc_compress.ko 2024-05-22T20:59:38.218 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/math/ 2024-05-22T20:59:38.218 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 7281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/math/cordic.ko 2024-05-22T20:59:38.218 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/memory-notifier-error-inject.ko 2024-05-22T20:59:38.218 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/notifier-error-inject.ko 2024-05-22T20:59:38.218 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/percpu_test.ko 2024-05-22T20:59:38.218 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/pm-notifier-error-inject.ko 2024-05-22T20:59:38.218 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/raid6/ 2024-05-22T20:59:38.218 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 152025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/raid6/raid6_pq.ko 2024-05-22T20:59:38.218 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/rbtree_test.ko 2024-05-22T20:59:38.218 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/reed_solomon/ 2024-05-22T20:59:38.218 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-22T20:59:38.219 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/test-kstrtox.ko 2024-05-22T20:59:38.219 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 4912441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/test_bpf.ko 2024-05-22T20:59:38.219 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 61169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/test_firmware.ko 2024-05-22T20:59:38.219 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 7121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/test_module.ko 2024-05-22T20:59:38.219 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/test_user_copy.ko 2024-05-22T20:59:38.219 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/ts_bm.ko 2024-05-22T20:59:38.219 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/ts_fsm.ko 2024-05-22T20:59:38.219 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/ts_kmp.ko 2024-05-22T20:59:38.219 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/xz/ 2024-05-22T20:59:38.219 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/xz/xz_dec_test.ko 2024-05-22T20:59:38.219 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/mm/ 2024-05-22T20:59:38.219 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/mm/hwpoison-inject.ko 2024-05-22T20:59:38.219 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ 2024-05-22T20:59:38.220 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/ 2024-05-22T20:59:38.220 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 66969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/6lowpan.ko 2024-05-22T20:59:38.220 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 7577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_dest.ko 2024-05-22T20:59:38.220 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 7529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_fragment.ko 2024-05-22T20:59:38.220 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 7513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_hop.ko 2024-05-22T20:59:38.220 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 7513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-22T20:59:38.220 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 7529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_mobility.ko 2024-05-22T20:59:38.220 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 7521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_routing.ko 2024-05-22T20:59:38.220 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_udp.ko 2024-05-22T20:59:38.220 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/ 2024-05-22T20:59:38.220 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/garp.ko 2024-05-22T20:59:38.220 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 42297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/mrp.ko 2024-05-22T20:59:38.220 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/p8022.ko 2024-05-22T20:59:38.221 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/psnap.ko 2024-05-22T20:59:38.221 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/stp.ko 2024-05-22T20:59:38.221 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/8021q/ 2024-05-22T20:59:38.221 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 106793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/8021q/8021q.ko 2024-05-22T20:59:38.221 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/9p/ 2024-05-22T20:59:38.221 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 236793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/9p/9pnet.ko 2024-05-22T20:59:38.221 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/9p/9pnet_fd.ko 2024-05-22T20:59:38.221 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/9p/9pnet_rdma.ko 2024-05-22T20:59:38.221 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 39513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/9p/9pnet_virtio.ko 2024-05-22T20:59:38.221 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/batman-adv/ 2024-05-22T20:59:38.221 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 879697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/batman-adv/batman-adv.ko 2024-05-22T20:59:38.221 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/ 2024-05-22T20:59:38.221 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 84761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/br_netfilter.ko 2024-05-22T20:59:38.222 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1030873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/bridge.ko 2024-05-22T20:59:38.222 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ 2024-05-22T20:59:38.222 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-22T20:59:38.222 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-22T20:59:38.222 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-22T20:59:38.222 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-22T20:59:38.222 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-22T20:59:38.222 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-22T20:59:38.222 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-22T20:59:38.222 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-22T20:59:38.222 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-22T20:59:38.222 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-22T20:59:38.222 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-22T20:59:38.223 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-22T20:59:38.223 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-22T20:59:38.223 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-22T20:59:38.223 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-22T20:59:38.223 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-22T20:59:38.223 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-22T20:59:38.223 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-22T20:59:38.223 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-22T20:59:38.223 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-22T20:59:38.223 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 70529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebtables.ko 2024-05-22T20:59:38.223 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-22T20:59:38.223 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-22T20:59:38.224 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ceph/ 2024-05-22T20:59:38.224 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1243505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ceph/libceph.ko 2024-05-22T20:59:38.224 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/core/ 2024-05-22T20:59:38.224 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 153801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/core/pktgen.ko 2024-05-22T20:59:38.224 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dccp/ 2024-05-22T20:59:38.224 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 244889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dccp/dccp.ko 2024-05-22T20:59:38.224 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dccp/dccp_diag.ko 2024-05-22T20:59:38.224 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 71033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dccp/dccp_ipv4.ko 2024-05-22T20:59:38.224 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 74993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dccp/dccp_ipv6.ko 2024-05-22T20:59:38.224 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/ 2024-05-22T20:59:38.224 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 507705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/dsa_core.ko 2024-05-22T20:59:38.224 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/tag_brcm.ko 2024-05-22T20:59:38.224 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/tag_dsa.ko 2024-05-22T20:59:38.225 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/tag_none.ko 2024-05-22T20:59:38.225 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/tag_trailer.ko 2024-05-22T20:59:38.225 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/hsr/ 2024-05-22T20:59:38.225 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 128873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/hsr/hsr.ko 2024-05-22T20:59:38.225 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ieee802154/ 2024-05-22T20:59:38.225 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ieee802154/6lowpan/ 2024-05-22T20:59:38.225 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 55505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-22T20:59:38.225 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 357425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ieee802154/ieee802154.ko 2024-05-22T20:59:38.225 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 59393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-22T20:59:38.225 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ 2024-05-22T20:59:38.225 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ah4.ko 2024-05-22T20:59:38.225 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 46417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/esp4.ko 2024-05-22T20:59:38.225 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 60233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/fou.ko 2024-05-22T20:59:38.226 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/gre.ko 2024-05-22T20:59:38.226 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 65665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/inet_diag.ko 2024-05-22T20:59:38.226 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 75489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ip_gre.ko 2024-05-22T20:59:38.226 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 77721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ip_tunnel.ko 2024-05-22T20:59:38.226 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ip_vti.ko 2024-05-22T20:59:38.226 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ipcomp.ko 2024-05-22T20:59:38.226 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ipip.ko 2024-05-22T20:59:38.226 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ 2024-05-22T20:59:38.226 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 45521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-22T20:59:38.226 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-22T20:59:38.226 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-22T20:59:38.226 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 50169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ip_tables.ko 2024-05-22T20:59:38.226 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-22T20:59:38.227 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-05-22T20:59:38.227 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-22T20:59:38.227 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-22T20:59:38.227 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-22T20:59:38.227 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-05-22T20:59:38.227 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-22T20:59:38.227 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-22T20:59:38.227 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-22T20:59:38.227 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-22T20:59:38.227 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-22T20:59:38.227 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-22T20:59:38.227 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-22T20:59:38.227 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-22T20:59:38.227 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-22T20:59:38.228 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-05-22T20:59:38.228 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-22T20:59:38.228 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-22T20:59:38.228 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-22T20:59:38.228 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-22T20:59:38.228 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-22T20:59:38.228 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_bic.ko 2024-05-22T20:59:38.228 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_dctcp.ko 2024-05-22T20:59:38.228 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_diag.ko 2024-05-22T20:59:38.228 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_highspeed.ko 2024-05-22T20:59:38.228 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_htcp.ko 2024-05-22T20:59:38.228 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_hybla.ko 2024-05-22T20:59:38.229 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_illinois.ko 2024-05-22T20:59:38.229 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_lp.ko 2024-05-22T20:59:38.229 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_scalable.ko 2024-05-22T20:59:38.229 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_vegas.ko 2024-05-22T20:59:38.229 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_veno.ko 2024-05-22T20:59:38.229 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_westwood.ko 2024-05-22T20:59:38.229 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_yeah.ko 2024-05-22T20:59:38.229 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tunnel4.ko 2024-05-22T20:59:38.229 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/udp_diag.ko 2024-05-22T20:59:38.229 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 54441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/udp_tunnel.ko 2024-05-22T20:59:38.229 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-22T20:59:38.229 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ 2024-05-22T20:59:38.229 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ah6.ko 2024-05-22T20:59:38.229 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 47433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/esp6.ko 2024-05-22T20:59:38.230 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/fou6.ko 2024-05-22T20:59:38.230 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 103177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ip6_gre.ko 2024-05-22T20:59:38.230 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 118881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ip6_tunnel.ko 2024-05-22T20:59:38.230 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-22T20:59:38.230 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 71089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ip6_vti.ko 2024-05-22T20:59:38.230 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ipcomp6.ko 2024-05-22T20:59:38.230 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/mip6.ko 2024-05-22T20:59:38.230 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ 2024-05-22T20:59:38.230 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 50673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-22T20:59:38.230 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-05-22T20:59:38.230 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-22T20:59:38.230 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-22T20:59:38.231 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15929 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-22T20:59:38.231 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-22T20:59:38.231 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-22T20:59:38.231 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-22T20:59:38.231 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-22T20:59:38.231 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-22T20:59:38.231 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-22T20:59:38.231 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-22T20:59:38.231 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-22T20:59:38.231 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-22T20:59:38.231 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-22T20:59:38.231 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-22T20:59:38.231 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-22T20:59:38.232 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 46313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-22T20:59:38.232 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-22T20:59:38.232 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 29817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-22T20:59:38.232 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-22T20:59:38.232 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-22T20:59:38.232 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-22T20:59:38.232 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-22T20:59:38.232 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-22T20:59:38.232 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 94089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/sit.ko 2024-05-22T20:59:38.232 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 30297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/tunnel6.ko 2024-05-22T20:59:38.232 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-22T20:59:38.232 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/key/ 2024-05-22T20:59:38.232 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 106369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/key/af_key.ko 2024-05-22T20:59:38.233 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/ 2024-05-22T20:59:38.233 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 153217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_core.ko 2024-05-22T20:59:38.233 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-22T20:59:38.233 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_eth.ko 2024-05-22T20:59:38.233 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_ip.ko 2024-05-22T20:59:38.233 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_ip6.ko 2024-05-22T20:59:38.233 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 42049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_netlink.ko 2024-05-22T20:59:38.233 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 66241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_ppp.ko 2024-05-22T20:59:38.233 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/lapb/ 2024-05-22T20:59:38.233 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 63345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/lapb/lapb.ko 2024-05-22T20:59:38.233 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/llc/ 2024-05-22T20:59:38.233 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/llc/llc.ko 2024-05-22T20:59:38.233 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 332889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/llc/llc2.ko 2024-05-22T20:59:38.234 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/mac802154/ 2024-05-22T20:59:38.234 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 388873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/mac802154/mac802154.ko 2024-05-22T20:59:38.234 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/mpls/ 2024-05-22T20:59:38.234 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/mpls/mpls_gso.ko 2024-05-22T20:59:38.234 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ 2024-05-22T20:59:38.234 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ 2024-05-22T20:59:38.234 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 147745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set.ko 2024-05-22T20:59:38.234 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-22T20:59:38.234 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-22T20:59:38.234 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-22T20:59:38.234 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 124041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-22T20:59:38.234 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 124345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-22T20:59:38.234 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 126681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-22T20:59:38.235 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 126473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-22T20:59:38.235 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 137833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-22T20:59:38.235 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 72353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-22T20:59:38.235 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 133681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-22T20:59:38.235 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 139073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-22T20:59:38.235 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 138265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-22T20:59:38.235 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 136177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-22T20:59:38.235 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 139897 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-22T20:59:38.235 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 37129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-22T20:59:38.235 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ 2024-05-22T20:59:38.235 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 534113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-22T20:59:38.235 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-22T20:59:38.235 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-22T20:59:38.236 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-22T20:59:38.236 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-22T20:59:38.236 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-22T20:59:38.236 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-22T20:59:38.236 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-22T20:59:38.236 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-22T20:59:38.236 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-22T20:59:38.236 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-22T20:59:38.236 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-22T20:59:38.236 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-22T20:59:38.236 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-22T20:59:38.236 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 55217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conncount.ko 2024-05-22T20:59:38.236 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 496817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack.ko 2024-05-22T20:59:38.237 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-22T20:59:38.237 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-22T20:59:38.237 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 37865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-22T20:59:38.237 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 134377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-22T20:59:38.237 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-22T20:59:38.237 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-22T20:59:38.237 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 140801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-22T20:59:38.237 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 38305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-22T20:59:38.237 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-22T20:59:38.237 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 79873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-22T20:59:38.237 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-22T20:59:38.237 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-22T20:59:38.237 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-22T20:59:38.238 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 117201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_flow_table.ko 2024-05-22T20:59:38.238 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-22T20:59:38.238 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 46105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_log_syslog.ko 2024-05-22T20:59:38.238 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 118369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat.ko 2024-05-22T20:59:38.238 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-22T20:59:38.238 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-22T20:59:38.238 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat_irc.ko 2024-05-22T20:59:38.238 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat_sip.ko 2024-05-22T20:59:38.238 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-22T20:59:38.238 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 53393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-22T20:59:38.238 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 807985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_tables.ko 2024-05-22T20:59:38.238 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 53049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink.ko 2024-05-22T20:59:38.238 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-22T20:59:38.239 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 37209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-05-22T20:59:38.239 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 49297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink_log.ko 2024-05-22T20:59:38.239 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-22T20:59:38.239 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 65145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-22T20:59:38.239 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_chain_nat.ko 2024-05-22T20:59:38.239 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 41769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_compat.ko 2024-05-22T20:59:38.239 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 50657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_ct.ko 2024-05-22T20:59:38.239 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-22T20:59:38.239 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_fib.ko 2024-05-22T20:59:38.239 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16545 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-22T20:59:38.239 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_hash.ko 2024-05-22T20:59:38.239 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_limit.ko 2024-05-22T20:59:38.239 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_log.ko 2024-05-22T20:59:38.240 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_masq.ko 2024-05-22T20:59:38.240 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_nat.ko 2024-05-22T20:59:38.240 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_numgen.ko 2024-05-22T20:59:38.240 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_osf.ko 2024-05-22T20:59:38.240 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_queue.ko 2024-05-22T20:59:38.240 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_redir.ko 2024-05-22T20:59:38.240 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_reject.ko 2024-05-22T20:59:38.240 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_reject_inet.ko 2024-05-22T20:59:38.240 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_socket.ko 2024-05-22T20:59:38.240 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 24993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_synproxy.ko 2024-05-22T20:59:38.240 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_tproxy.ko 2024-05-22T20:59:38.240 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 108393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/x_tables.ko 2024-05-22T20:59:38.240 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_AUDIT.ko 2024-05-22T20:59:38.240 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-22T20:59:38.240 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-22T20:59:38.241 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-22T20:59:38.241 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 21865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_CT.ko 2024-05-22T20:59:38.241 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_DSCP.ko 2024-05-22T20:59:38.241 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_HL.ko 2024-05-22T20:59:38.241 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_HMARK.ko 2024-05-22T20:59:38.241 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 45217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-22T20:59:38.241 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_LED.ko 2024-05-22T20:59:38.241 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_LOG.ko 2024-05-22T20:59:38.241 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-22T20:59:38.241 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_NETMAP.ko 2024-05-22T20:59:38.241 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_NFLOG.ko 2024-05-22T20:59:38.241 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-22T20:59:38.241 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_RATEEST.ko 2024-05-22T20:59:38.242 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-22T20:59:38.242 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_SECMARK.ko 2024-05-22T20:59:38.242 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-22T20:59:38.242 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-22T20:59:38.242 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_TEE.ko 2024-05-22T20:59:38.242 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_TPROXY.ko 2024-05-22T20:59:38.242 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_TRACE.ko 2024-05-22T20:59:38.242 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_addrtype.ko 2024-05-22T20:59:38.242 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_bpf.ko 2024-05-22T20:59:38.242 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_cgroup.ko 2024-05-22T20:59:38.242 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_cluster.ko 2024-05-22T20:59:38.242 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_comment.ko 2024-05-22T20:59:38.243 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_connbytes.ko 2024-05-22T20:59:38.243 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_connlabel.ko 2024-05-22T20:59:38.243 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_connlimit.ko 2024-05-22T20:59:38.243 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_connmark.ko 2024-05-22T20:59:38.243 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_conntrack.ko 2024-05-22T20:59:38.243 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_cpu.ko 2024-05-22T20:59:38.243 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_dccp.ko 2024-05-22T20:59:38.243 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_devgroup.ko 2024-05-22T20:59:38.243 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_dscp.ko 2024-05-22T20:59:38.243 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_ecn.ko 2024-05-22T20:59:38.243 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_esp.ko 2024-05-22T20:59:38.243 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 52817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_hashlimit.ko 2024-05-22T20:59:38.243 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_helper.ko 2024-05-22T20:59:38.243 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_hl.ko 2024-05-22T20:59:38.244 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_ipcomp.ko 2024-05-22T20:59:38.244 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_iprange.ko 2024-05-22T20:59:38.244 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_ipvs.ko 2024-05-22T20:59:38.244 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_l2tp.ko 2024-05-22T20:59:38.244 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_length.ko 2024-05-22T20:59:38.244 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_limit.ko 2024-05-22T20:59:38.244 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_mac.ko 2024-05-22T20:59:38.244 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_mark.ko 2024-05-22T20:59:38.244 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_multiport.ko 2024-05-22T20:59:38.244 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 16161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_nat.ko 2024-05-22T20:59:38.244 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_nfacct.ko 2024-05-22T20:59:38.244 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_osf.ko 2024-05-22T20:59:38.244 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_owner.ko 2024-05-22T20:59:38.245 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_physdev.ko 2024-05-22T20:59:38.245 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_pkttype.ko 2024-05-22T20:59:38.245 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_policy.ko 2024-05-22T20:59:38.245 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_quota.ko 2024-05-22T20:59:38.245 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 11881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_rateest.ko 2024-05-22T20:59:38.245 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_realm.ko 2024-05-22T20:59:38.245 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_recent.ko 2024-05-22T20:59:38.245 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_sctp.ko 2024-05-22T20:59:38.245 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_set.ko 2024-05-22T20:59:38.245 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 19777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_socket.ko 2024-05-22T20:59:38.245 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_state.ko 2024-05-22T20:59:38.245 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_statistic.ko 2024-05-22T20:59:38.246 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_string.ko 2024-05-22T20:59:38.246 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_tcpmss.ko 2024-05-22T20:59:38.246 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_tcpudp.ko 2024-05-22T20:59:38.246 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_time.ko 2024-05-22T20:59:38.246 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_u32.ko 2024-05-22T20:59:38.246 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netlink/ 2024-05-22T20:59:38.246 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 17233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netlink/netlink_diag.ko 2024-05-22T20:59:38.246 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/nsh/ 2024-05-22T20:59:38.246 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 14073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/nsh/nsh.ko 2024-05-22T20:59:38.246 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/openvswitch/ 2024-05-22T20:59:38.246 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 488825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/openvswitch/openvswitch.ko 2024-05-22T20:59:38.246 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/openvswitch/vport-geneve.ko 2024-05-22T20:59:38.246 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 10873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/openvswitch/vport-gre.ko 2024-05-22T20:59:38.247 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/openvswitch/vport-vxlan.ko 2024-05-22T20:59:38.247 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/packet/ 2024-05-22T20:59:38.247 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15929 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/packet/af_packet_diag.ko 2024-05-22T20:59:38.247 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/phonet/ 2024-05-22T20:59:38.247 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 142025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/phonet/phonet.ko 2024-05-22T20:59:38.247 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 68817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/phonet/pn_pep.ko 2024-05-22T20:59:38.247 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/rds/ 2024-05-22T20:59:38.247 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 318617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/rds/rds.ko 2024-05-22T20:59:38.247 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 202305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/rds/rds_rdma.ko 2024-05-22T20:59:38.247 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 85105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/rds/rds_tcp.ko 2024-05-22T20:59:38.247 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/rxrpc/ 2024-05-22T20:59:38.247 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1027897 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/rxrpc/rxrpc.ko 2024-05-22T20:59:38.247 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/ 2024-05-22T20:59:38.248 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 39001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_csum.ko 2024-05-22T20:59:38.248 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 28721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_gact.ko 2024-05-22T20:59:38.248 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 45409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_mirred.ko 2024-05-22T20:59:38.248 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 27593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_nat.ko 2024-05-22T20:59:38.248 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 53465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_pedit.ko 2024-05-22T20:59:38.248 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 38705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_police.ko 2024-05-22T20:59:38.248 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_simple.ko 2024-05-22T20:59:38.248 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_skbedit.ko 2024-05-22T20:59:38.248 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 45673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_vlan.ko 2024-05-22T20:59:38.248 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_basic.ko 2024-05-22T20:59:38.248 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 41457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_bpf.ko 2024-05-22T20:59:38.248 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_cgroup.ko 2024-05-22T20:59:38.249 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 37305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_flow.ko 2024-05-22T20:59:38.249 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 39601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_fw.ko 2024-05-22T20:59:38.249 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 49321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_route.ko 2024-05-22T20:59:38.249 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 79145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_u32.ko 2024-05-22T20:59:38.249 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_cmp.ko 2024-05-22T20:59:38.249 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 15385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_ipset.ko 2024-05-22T20:59:38.249 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 48785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_meta.ko 2024-05-22T20:59:38.249 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_nbyte.ko 2024-05-22T20:59:38.249 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_text.ko 2024-05-22T20:59:38.249 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 8377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_u32.ko 2024-05-22T20:59:38.249 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_choke.ko 2024-05-22T20:59:38.249 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 25097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_codel.ko 2024-05-22T20:59:38.249 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_drr.ko 2024-05-22T20:59:38.250 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 43153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_fq.ko 2024-05-22T20:59:38.250 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 39177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_fq_codel.ko 2024-05-22T20:59:38.250 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 42889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_gred.ko 2024-05-22T20:59:38.250 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 62817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_hfsc.ko 2024-05-22T20:59:38.250 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_hhf.ko 2024-05-22T20:59:38.250 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 81833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_htb.ko 2024-05-22T20:59:38.250 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33545 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_ingress.ko 2024-05-22T20:59:38.250 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 36625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_mqprio.ko 2024-05-22T20:59:38.250 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_mqprio_lib.ko 2024-05-22T20:59:38.250 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_multiq.ko 2024-05-22T20:59:38.250 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 44793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_netem.ko 2024-05-22T20:59:38.250 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 32833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_pie.ko 2024-05-22T20:59:38.250 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 13505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_plug.ko 2024-05-22T20:59:38.251 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 31801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_prio.ko 2024-05-22T20:59:38.251 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 60249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_qfq.ko 2024-05-22T20:59:38.251 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 40889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_red.ko 2024-05-22T20:59:38.251 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 33433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_sfb.ko 2024-05-22T20:59:38.251 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 38329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_sfq.ko 2024-05-22T20:59:38.251 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 35777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_tbf.ko 2024-05-22T20:59:38.251 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 34161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_teql.ko 2024-05-22T20:59:38.251 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sctp/ 2024-05-22T20:59:38.251 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1187313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sctp/sctp.ko 2024-05-22T20:59:38.251 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sctp/sctp_diag.ko 2024-05-22T20:59:38.251 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/ 2024-05-22T20:59:38.251 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/auth_gss/ 2024-05-22T20:59:38.251 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 432409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-22T20:59:38.251 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 70113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-22T20:59:38.252 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1940889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/sunrpc.ko 2024-05-22T20:59:38.252 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/xprtrdma/ 2024-05-22T20:59:38.252 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 890729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-22T20:59:38.252 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/tipc/ 2024-05-22T20:59:38.252 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 1122337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/tipc/tipc.ko 2024-05-22T20:59:38.252 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/tipc/tipc_diag.ko 2024-05-22T20:59:38.252 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/unix/ 2024-05-22T20:59:38.252 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 18801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/unix/unix_diag.ko 2024-05-22T20:59:38.252 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/ 2024-05-22T20:59:38.252 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 105609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-22T20:59:38.252 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 90265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-22T20:59:38.252 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 123769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/vsock.ko 2024-05-22T20:59:38.252 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 12905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-22T20:59:38.253 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 20401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-22T20:59:38.253 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/x25/ 2024-05-22T20:59:38.253 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 165841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/x25/x25.ko 2024-05-22T20:59:38.253 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/xfrm/ 2024-05-22T20:59:38.253 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 26729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/xfrm/xfrm_algo.ko 2024-05-22T20:59:38.253 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-22T20:59:38.253 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 131865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/xfrm/xfrm_user.ko 2024-05-22T20:59:38.253 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 644769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.alias 2024-05-22T20:59:38.253 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 628542 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.alias.bin 2024-05-22T20:59:38.255 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 7706 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.builtin 2024-05-22T20:59:38.255 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 23609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.builtin.alias.bin 2024-05-22T20:59:38.255 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 9941 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.builtin.bin 2024-05-22T20:59:38.255 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 70990 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.builtin.modinfo 2024-05-22T20:59:38.255 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 146959 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.dep 2024-05-22T20:59:38.255 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 232835 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.dep.bin 2024-05-22T20:59:38.255 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 210 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.devname 2024-05-22T20:59:38.256 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 72602 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.order 2024-05-22T20:59:38.256 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 879 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.softdep 2024-05-22T20:59:38.256 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 258450 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.symbols 2024-05-22T20:59:38.256 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 313019 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.symbols.bin 2024-05-22T20:59:38.256 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./usr/ 2024-05-22T20:59:38.256 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./usr/share/ 2024-05-22T20:59:38.256 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./usr/share/doc/ 2024-05-22T20:59:38.256 INFO:teuthology.orchestra.run.smithi179.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./usr/share/doc/linux-image-6.9.0-rc3-g4fed890e8ce3/ 2024-05-22T20:59:38.256 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 174 2024-05-21 01:11 ./usr/share/doc/linux-image-6.9.0-rc3-g4fed890e8ce3/changelog.Debian.gz 2024-05-22T20:59:38.256 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r-- root/root 691 2024-05-21 01:11 ./usr/share/doc/linux-image-6.9.0-rc3-g4fed890e8ce3/copyright 2024-05-22T20:59:38.261 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:38.261 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:38.261 DEBUG:teuthology.orchestra.run.smithi179:> egrep '(submenu|menuentry.*6.9.0-rc3-g4fed890e8ce3).*{' /boot/grub/grub.cfg 2024-05-22T20:59:38.303 INFO:teuthology.orchestra.run.smithi179.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-22T20:59:38.303 INFO:teuthology.orchestra.run.smithi179.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g4fed890e8ce3' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g4fed890e8ce3-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-22T20:59:38.303 INFO:teuthology.orchestra.run.smithi179.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g4fed890e8ce3 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g4fed890e8ce3-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-22T20:59:38.304 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-05-22T20:59:38.304 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:38.304 DEBUG:teuthology.orchestra.run.smithi179:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-05-22T20:59:38.347 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi179.front.sepia.ceph.com to complete... 2024-05-22T20:59:38.431 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:38.449 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:38.730 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-22T20:59:38.739 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-22T20:59:38.766 INFO:teuthology.orchestra.run.smithi179.stderr:Sourcing file `/etc/default/grub' 2024-05-22T20:59:38.767 INFO:teuthology.orchestra.run.smithi179.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-22T20:59:38.789 INFO:teuthology.orchestra.run.smithi179.stderr:Generating grub configuration file ... 2024-05-22T20:59:38.806 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-22T20:59:38.811 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-22T20:59:38.968 INFO:teuthology.orchestra.run.smithi161.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-22T20:59:38.968 INFO:teuthology.orchestra.run.smithi161.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-22T20:59:38.969 INFO:teuthology.orchestra.run.smithi161.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-22T20:59:38.975 INFO:teuthology.orchestra.run.smithi179.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:38.981 INFO:teuthology.orchestra.run.smithi179.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:39.017 INFO:teuthology.orchestra.run.smithi161.stderr:done 2024-05-22T20:59:39.167 DEBUG:teuthology.orchestra.run.smithi161:> dpkg-deb -c /tmp/linux-image.deb 2024-05-22T20:59:39.175 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./ 2024-05-22T20:59:39.175 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./boot/ 2024-05-22T20:59:39.175 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 7002012 2024-05-21 01:11 ./boot/System.map-6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:39.193 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 172073 2024-05-21 01:11 ./boot/config-6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:39.194 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13120512 2024-05-21 01:11 ./boot/vmlinuz-6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:39.213 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./etc/ 2024-05-22T20:59:39.213 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./etc/kernel/ 2024-05-22T20:59:39.213 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./etc/kernel/postinst.d/ 2024-05-22T20:59:39.213 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./etc/kernel/postrm.d/ 2024-05-22T20:59:39.213 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./etc/kernel/preinst.d/ 2024-05-22T20:59:39.214 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./etc/kernel/prerm.d/ 2024-05-22T20:59:39.214 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/ 2024-05-22T20:59:39.214 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/ 2024-05-22T20:59:39.214 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/ 2024-05-22T20:59:39.214 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/ 2024-05-22T20:59:39.214 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/ 2024-05-22T20:59:39.214 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/ 2024-05-22T20:59:39.214 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/ 2024-05-22T20:59:39.214 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 507569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/aesni-intel.ko 2024-05-22T20:59:39.217 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-22T20:59:39.217 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-22T20:59:39.217 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-22T20:59:39.217 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 97953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-22T20:59:39.217 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 109905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-22T20:59:39.217 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 155249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-22T20:59:39.217 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-22T20:59:39.218 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-22T20:59:39.218 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 87009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-22T20:59:39.218 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-22T20:59:39.218 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 50305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-22T20:59:39.218 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 50377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-22T20:59:39.218 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 54513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-22T20:59:39.218 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-22T20:59:39.218 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 48049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-22T20:59:39.218 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 62873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-22T20:59:39.218 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 53569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-22T20:59:39.218 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-22T20:59:39.218 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-22T20:59:39.218 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kernel/ 2024-05-22T20:59:39.219 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kernel/cpu/ 2024-05-22T20:59:39.219 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kernel/cpu/mce/ 2024-05-22T20:59:39.219 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 49081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-22T20:59:39.219 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kernel/cpuid.ko 2024-05-22T20:59:39.219 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kernel/msr.ko 2024-05-22T20:59:39.219 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kvm/ 2024-05-22T20:59:39.219 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 444425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-22T20:59:39.219 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1035881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-22T20:59:39.219 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 2849609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kvm/kvm.ko 2024-05-22T20:59:39.266 INFO:teuthology.orchestra.run.smithi179.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-22T20:59:39.272 INFO:teuthology.orchestra.run.smithi179.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-22T20:59:39.295 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/ 2024-05-22T20:59:39.295 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 65025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/af_alg.ko 2024-05-22T20:59:39.295 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/algif_hash.ko 2024-05-22T20:59:39.295 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/algif_skcipher.ko 2024-05-22T20:59:39.295 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/ansi_cprng.ko 2024-05-22T20:59:39.295 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/anubis.ko 2024-05-22T20:59:39.295 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10929 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/arc4.ko 2024-05-22T20:59:39.295 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/asymmetric_keys/ 2024-05-22T20:59:39.295 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-05-22T20:59:39.295 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/ 2024-05-22T20:59:39.296 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/async_memcpy.ko 2024-05-22T20:59:39.296 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/async_pq.ko 2024-05-22T20:59:39.296 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-22T20:59:39.296 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/async_tx.ko 2024-05-22T20:59:39.296 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/async_xor.ko 2024-05-22T20:59:39.296 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/raid6test.ko 2024-05-22T20:59:39.296 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/authenc.ko 2024-05-22T20:59:39.296 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/authencesn.ko 2024-05-22T20:59:39.296 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/blake2b_generic.ko 2024-05-22T20:59:39.296 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/blowfish_common.ko 2024-05-22T20:59:39.296 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/blowfish_generic.ko 2024-05-22T20:59:39.296 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/camellia_generic.ko 2024-05-22T20:59:39.296 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cast5_generic.ko 2024-05-22T20:59:39.297 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cast6_generic.ko 2024-05-22T20:59:39.297 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cast_common.ko 2024-05-22T20:59:39.297 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/ccm.ko 2024-05-22T20:59:39.297 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cmac.ko 2024-05-22T20:59:39.297 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/crc32_generic.ko 2024-05-22T20:59:39.297 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 59129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cryptd.ko 2024-05-22T20:59:39.297 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 48113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/crypto_engine.ko 2024-05-22T20:59:39.297 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/crypto_simd.ko 2024-05-22T20:59:39.297 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/crypto_user.ko 2024-05-22T20:59:39.297 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cts.ko 2024-05-22T20:59:39.297 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/des_generic.ko 2024-05-22T20:59:39.297 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/dh_generic.ko 2024-05-22T20:59:39.298 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/echainiv.ko 2024-05-22T20:59:39.298 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/essiv.ko 2024-05-22T20:59:39.298 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/fcrypt.ko 2024-05-22T20:59:39.298 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/khazad.ko 2024-05-22T20:59:39.298 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/lrw.ko 2024-05-22T20:59:39.298 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/lz4.ko 2024-05-22T20:59:39.298 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/lz4hc.ko 2024-05-22T20:59:39.298 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/md4.ko 2024-05-22T20:59:39.298 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/michael_mic.ko 2024-05-22T20:59:39.298 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/pcbc.ko 2024-05-22T20:59:39.298 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/pcrypt.ko 2024-05-22T20:59:39.298 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/rmd160.ko 2024-05-22T20:59:39.299 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/seed.ko 2024-05-22T20:59:39.299 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/serpent_generic.ko 2024-05-22T20:59:39.299 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 169065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/tcrypt.ko 2024-05-22T20:59:39.299 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/tea.ko 2024-05-22T20:59:39.299 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/twofish_common.ko 2024-05-22T20:59:39.299 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/twofish_generic.ko 2024-05-22T20:59:39.299 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/vmac.ko 2024-05-22T20:59:39.299 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 38209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/wp512.ko 2024-05-22T20:59:39.299 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/xcbc.ko 2024-05-22T20:59:39.299 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/xor.ko 2024-05-22T20:59:39.299 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/xts.ko 2024-05-22T20:59:39.299 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/xxhash_generic.ko 2024-05-22T20:59:39.299 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ 2024-05-22T20:59:39.300 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/ 2024-05-22T20:59:39.300 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/acpi_extlog.ko 2024-05-22T20:59:39.300 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-22T20:59:39.300 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/acpi_pad.ko 2024-05-22T20:59:39.300 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/apei/ 2024-05-22T20:59:39.300 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/apei/einj.ko 2024-05-22T20:59:39.300 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/ec_sys.ko 2024-05-22T20:59:39.300 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/platform_profile.ko 2024-05-22T20:59:39.300 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/sbs.ko 2024-05-22T20:59:39.300 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/sbshc.ko 2024-05-22T20:59:39.300 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 119033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/video.ko 2024-05-22T20:59:39.300 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/ 2024-05-22T20:59:39.300 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/acard-ahci.ko 2024-05-22T20:59:39.301 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 96825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/ahci.ko 2024-05-22T20:59:39.301 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/ahci_platform.ko 2024-05-22T20:59:39.301 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 107737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/libahci.ko 2024-05-22T20:59:39.301 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 42001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/libahci_platform.ko 2024-05-22T20:59:39.301 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_acpi.ko 2024-05-22T20:59:39.301 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_ali.ko 2024-05-22T20:59:39.301 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 38329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_amd.ko 2024-05-22T20:59:39.301 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_artop.ko 2024-05-22T20:59:39.301 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_atiixp.ko 2024-05-22T20:59:39.301 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_atp867x.ko 2024-05-22T20:59:39.301 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_cmd640.ko 2024-05-22T20:59:39.301 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28929 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_cmd64x.ko 2024-05-22T20:59:39.302 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_cypress.ko 2024-05-22T20:59:39.302 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_efar.ko 2024-05-22T20:59:39.302 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_hpt366.ko 2024-05-22T20:59:39.302 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 38913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_hpt37x.ko 2024-05-22T20:59:39.302 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-22T20:59:39.302 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-22T20:59:39.302 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_it8213.ko 2024-05-22T20:59:39.302 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 41289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_it821x.ko 2024-05-22T20:59:39.302 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_jmicron.ko 2024-05-22T20:59:39.302 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 60257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_legacy.ko 2024-05-22T20:59:39.302 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_marvell.ko 2024-05-22T20:59:39.302 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_mpiix.ko 2024-05-22T20:59:39.302 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_netcell.ko 2024-05-22T20:59:39.303 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_ninja32.ko 2024-05-22T20:59:39.303 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_ns87410.ko 2024-05-22T20:59:39.303 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_ns87415.ko 2024-05-22T20:59:39.303 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_oldpiix.ko 2024-05-22T20:59:39.303 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_opti.ko 2024-05-22T20:59:39.303 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_optidma.ko 2024-05-22T20:59:39.303 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_pcmcia.ko 2024-05-22T20:59:39.303 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-22T20:59:39.303 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-22T20:59:39.303 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_piccolo.ko 2024-05-22T20:59:39.303 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_radisys.ko 2024-05-22T20:59:39.303 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_rdc.ko 2024-05-22T20:59:39.303 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_rz1000.ko 2024-05-22T20:59:39.304 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_sch.ko 2024-05-22T20:59:39.304 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_serverworks.ko 2024-05-22T20:59:39.304 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_sil680.ko 2024-05-22T20:59:39.304 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_sl82c105.ko 2024-05-22T20:59:39.304 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_triflex.ko 2024-05-22T20:59:39.304 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_via.ko 2024-05-22T20:59:39.304 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pdc_adma.ko 2024-05-22T20:59:39.304 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_inic162x.ko 2024-05-22T20:59:39.304 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 102121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_mv.ko 2024-05-22T20:59:39.304 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 85409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_nv.ko 2024-05-22T20:59:39.304 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_promise.ko 2024-05-22T20:59:39.304 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_qstor.ko 2024-05-22T20:59:39.304 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_sil.ko 2024-05-22T20:59:39.305 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 45905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_sil24.ko 2024-05-22T20:59:39.305 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_sis.ko 2024-05-22T20:59:39.305 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_svw.ko 2024-05-22T20:59:39.305 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 53353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_sx4.ko 2024-05-22T20:59:39.305 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_uli.ko 2024-05-22T20:59:39.305 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 41049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_via.ko 2024-05-22T20:59:39.305 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_vsc.ko 2024-05-22T20:59:39.305 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/auxdisplay/ 2024-05-22T20:59:39.305 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/auxdisplay/ks0108.ko 2024-05-22T20:59:39.305 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/bcma/ 2024-05-22T20:59:39.305 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 163969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/bcma/bcma.ko 2024-05-22T20:59:39.305 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/ 2024-05-22T20:59:39.306 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/aoe/ 2024-05-22T20:59:39.306 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 140865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/aoe/aoe.ko 2024-05-22T20:59:39.306 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/drbd/ 2024-05-22T20:59:39.306 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1241657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/drbd/drbd.ko 2024-05-22T20:59:39.306 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 203289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/floppy.ko 2024-05-22T20:59:39.306 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/mtip32xx/ 2024-05-22T20:59:39.306 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 115265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-22T20:59:39.306 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 143433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/nbd.ko 2024-05-22T20:59:39.306 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/null_blk/ 2024-05-22T20:59:39.306 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 126617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/null_blk/null_blk.ko 2024-05-22T20:59:39.306 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 119873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/pktcdvd.ko 2024-05-22T20:59:39.306 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 313217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/rbd.ko 2024-05-22T20:59:39.307 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/xen-blkback/ 2024-05-22T20:59:39.307 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 117233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-22T20:59:39.307 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/zram/ 2024-05-22T20:59:39.307 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 70385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/zram/zram.ko 2024-05-22T20:59:39.307 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ 2024-05-22T20:59:39.307 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/applicom.ko 2024-05-22T20:59:39.307 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hangcheck-timer.ko 2024-05-22T20:59:39.307 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/ 2024-05-22T20:59:39.307 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-22T20:59:39.307 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-22T20:59:39.307 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-22T20:59:39.307 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/via-rng.ko 2024-05-22T20:59:39.307 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-22T20:59:39.307 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ 2024-05-22T20:59:39.308 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 37257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-22T20:59:39.308 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 186921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-22T20:59:39.308 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-22T20:59:39.308 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 179561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-22T20:59:39.308 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 85441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-22T20:59:39.308 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 65313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-22T20:59:39.308 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/lp.ko 2024-05-22T20:59:39.308 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/mwave/ 2024-05-22T20:59:39.308 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 103409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/mwave/mwave.ko 2024-05-22T20:59:39.308 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/nvram.ko 2024-05-22T20:59:39.308 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 38977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ppdev.ko 2024-05-22T20:59:39.308 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 59241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tlclk.ko 2024-05-22T20:59:39.308 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/ 2024-05-22T20:59:39.308 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-22T20:59:39.309 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-05-22T20:59:39.309 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-05-22T20:59:39.309 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 39193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-05-22T20:59:39.309 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-22T20:59:39.309 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-22T20:59:39.309 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-05-22T20:59:39.309 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/xillybus/ 2024-05-22T20:59:39.309 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/xillybus/xillybus_class.ko 2024-05-22T20:59:39.309 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 62545 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/xillybus/xillybus_core.ko 2024-05-22T20:59:39.309 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13929 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-05-22T20:59:39.309 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/clk/ 2024-05-22T20:59:39.309 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/clk/clk-palmas.ko 2024-05-22T20:59:39.309 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 57289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/clk/clk-si5351.ko 2024-05-22T20:59:39.310 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/clk/clk-twl6040.ko 2024-05-22T20:59:39.310 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/clk/clk-wm831x.ko 2024-05-22T20:59:39.310 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/cpufreq/ 2024-05-22T20:59:39.310 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-22T20:59:39.310 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-22T20:59:39.310 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-22T20:59:39.310 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/ 2024-05-22T20:59:39.310 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/ccp/ 2024-05-22T20:59:39.310 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 89257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-22T20:59:39.310 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 333657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/ccp/ccp.ko 2024-05-22T20:59:39.310 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/ 2024-05-22T20:59:39.310 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/qat/ 2024-05-22T20:59:39.311 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/qat/qat_common/ 2024-05-22T20:59:39.311 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 751713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-22T20:59:39.311 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-05-22T20:59:39.311 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-22T20:59:39.311 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/padlock-aes.ko 2024-05-22T20:59:39.311 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/padlock-sha.ko 2024-05-22T20:59:39.311 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/virtio/ 2024-05-22T20:59:39.311 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 90441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-22T20:59:39.311 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dca/ 2024-05-22T20:59:39.311 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dca/dca.ko 2024-05-22T20:59:39.311 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dma/ 2024-05-22T20:59:39.311 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dma/dw/ 2024-05-22T20:59:39.311 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-22T20:59:39.312 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dma/ioat/ 2024-05-22T20:59:39.312 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 146769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-22T20:59:39.312 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/ 2024-05-22T20:59:39.312 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 96145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/amd64_edac.ko 2024-05-22T20:59:39.312 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 41993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/e752x_edac.ko 2024-05-22T20:59:39.312 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 51105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/edac_mce_amd.ko 2024-05-22T20:59:39.312 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i3000_edac.ko 2024-05-22T20:59:39.312 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i3200_edac.ko 2024-05-22T20:59:39.312 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i5100_edac.ko 2024-05-22T20:59:39.312 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i5400_edac.ko 2024-05-22T20:59:39.312 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i7300_edac.ko 2024-05-22T20:59:39.312 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 64161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i7core_edac.ko 2024-05-22T20:59:39.313 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i82975x_edac.ko 2024-05-22T20:59:39.313 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/ie31200_edac.ko 2024-05-22T20:59:39.313 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 68697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/sb_edac.ko 2024-05-22T20:59:39.313 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/x38_edac.ko 2024-05-22T20:59:39.313 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/extcon/ 2024-05-22T20:59:39.313 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/extcon/extcon-gpio.ko 2024-05-22T20:59:39.313 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/extcon/extcon-max14577.ko 2024-05-22T20:59:39.313 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/extcon/extcon-max77693.ko 2024-05-22T20:59:39.313 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/extcon/extcon-max8997.ko 2024-05-22T20:59:39.313 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/extcon/extcon-palmas.ko 2024-05-22T20:59:39.313 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/extcon/extcon-rt8973a.ko 2024-05-22T20:59:39.313 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/extcon/extcon-sm5502.ko 2024-05-22T20:59:39.313 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/firmware/ 2024-05-22T20:59:39.314 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 42953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/firmware/dmi-sysfs.ko 2024-05-22T20:59:39.314 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/firmware/efi/ 2024-05-22T20:59:39.314 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/firmware/efi/efi-pstore.ko 2024-05-22T20:59:39.314 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-22T20:59:39.314 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/ 2024-05-22T20:59:39.314 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-adp5520.ko 2024-05-22T20:59:39.314 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-amd8111.ko 2024-05-22T20:59:39.314 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-arizona.ko 2024-05-22T20:59:39.314 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-da9052.ko 2024-05-22T20:59:39.314 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-da9055.ko 2024-05-22T20:59:39.314 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-dln2.ko 2024-05-22T20:59:39.314 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 81073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-f7188x.ko 2024-05-22T20:59:39.315 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-generic.ko 2024-05-22T20:59:39.315 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-ich.ko 2024-05-22T20:59:39.315 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-05-22T20:59:39.315 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-kempld.ko 2024-05-22T20:59:39.315 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-lp3943.ko 2024-05-22T20:59:39.315 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-max7300.ko 2024-05-22T20:59:39.315 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-max730x.ko 2024-05-22T20:59:39.315 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-max732x.ko 2024-05-22T20:59:39.315 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-05-22T20:59:39.315 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 55505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-pca953x.ko 2024-05-22T20:59:39.315 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-pcf857x.ko 2024-05-22T20:59:39.315 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-rdc321x.ko 2024-05-22T20:59:39.315 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-sch.ko 2024-05-22T20:59:39.316 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-sch311x.ko 2024-05-22T20:59:39.316 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-tps65912.ko 2024-05-22T20:59:39.316 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-twl4030.ko 2024-05-22T20:59:39.316 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-twl6040.ko 2024-05-22T20:59:39.316 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-22T20:59:39.316 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17897 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-vx855.ko 2024-05-22T20:59:39.316 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-wm831x.ko 2024-05-22T20:59:39.316 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-wm8350.ko 2024-05-22T20:59:39.316 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-wm8994.ko 2024-05-22T20:59:39.316 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/ 2024-05-22T20:59:39.316 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-a4tech.ko 2024-05-22T20:59:39.316 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-apple.ko 2024-05-22T20:59:39.316 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-appleir.ko 2024-05-22T20:59:39.316 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-aureal.ko 2024-05-22T20:59:39.317 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-axff.ko 2024-05-22T20:59:39.317 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-belkin.ko 2024-05-22T20:59:39.317 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-cherry.ko 2024-05-22T20:59:39.317 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-chicony.ko 2024-05-22T20:59:39.317 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 74393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-cp2112.ko 2024-05-22T20:59:39.317 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-cypress.ko 2024-05-22T20:59:39.317 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-dr.ko 2024-05-22T20:59:39.317 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-elecom.ko 2024-05-22T20:59:39.317 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-elo.ko 2024-05-22T20:59:39.317 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-emsff.ko 2024-05-22T20:59:39.317 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-ezkey.ko 2024-05-22T20:59:39.317 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-gaff.ko 2024-05-22T20:59:39.318 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-generic.ko 2024-05-22T20:59:39.318 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-gt683r.ko 2024-05-22T20:59:39.318 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-gyration.ko 2024-05-22T20:59:39.318 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-22T20:59:39.318 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-22T20:59:39.318 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-holtekff.ko 2024-05-22T20:59:39.318 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-hyperv.ko 2024-05-22T20:59:39.318 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-icade.ko 2024-05-22T20:59:39.318 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-kensington.ko 2024-05-22T20:59:39.318 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-keytouch.ko 2024-05-22T20:59:39.318 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-kye.ko 2024-05-22T20:59:39.318 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-lcpower.ko 2024-05-22T20:59:39.319 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-led.ko 2024-05-22T20:59:39.319 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 54217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-lenovo.ko 2024-05-22T20:59:39.319 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 37553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-lg-g15.ko 2024-05-22T20:59:39.319 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 68081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-22T20:59:39.319 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 121369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-22T20:59:39.319 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 97953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-logitech.ko 2024-05-22T20:59:39.319 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-magicmouse.ko 2024-05-22T20:59:39.319 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-microsoft.ko 2024-05-22T20:59:39.319 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-monterey.ko 2024-05-22T20:59:39.319 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 59001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-multitouch.ko 2024-05-22T20:59:39.319 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 39697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-ntrig.ko 2024-05-22T20:59:39.319 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-ortek.ko 2024-05-22T20:59:39.319 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-penmount.ko 2024-05-22T20:59:39.320 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-petalynx.ko 2024-05-22T20:59:39.320 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 91241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-picolcd.ko 2024-05-22T20:59:39.320 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-pl.ko 2024-05-22T20:59:39.320 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-plantronics.ko 2024-05-22T20:59:39.320 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-primax.ko 2024-05-22T20:59:39.320 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 42889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-rmi.ko 2024-05-22T20:59:39.320 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-22T20:59:39.320 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-common.ko 2024-05-22T20:59:39.320 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 38865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-22T20:59:39.320 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-22T20:59:39.320 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 38073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-22T20:59:39.320 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-22T20:59:39.320 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-22T20:59:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-22T20:59:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-22T20:59:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-22T20:59:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-22T20:59:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat.ko 2024-05-22T20:59:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-saitek.ko 2024-05-22T20:59:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 42305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-samsung.ko 2024-05-22T20:59:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-22T20:59:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-sjoy.ko 2024-05-22T20:59:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 62129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-sony.ko 2024-05-22T20:59:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-speedlink.ko 2024-05-22T20:59:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-steelseries.ko 2024-05-22T20:59:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-sunplus.ko 2024-05-22T20:59:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-22T20:59:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-tivo.ko 2024-05-22T20:59:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-tmff.ko 2024-05-22T20:59:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-topseed.ko 2024-05-22T20:59:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-twinhan.ko 2024-05-22T20:59:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 81113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-uclogic.ko 2024-05-22T20:59:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-waltop.ko 2024-05-22T20:59:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 184745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-wiimote.ko 2024-05-22T20:59:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-xinmo.ko 2024-05-22T20:59:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-zpff.ko 2024-05-22T20:59:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-zydacron.ko 2024-05-22T20:59:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 319921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid.ko 2024-05-22T20:59:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 39529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/uhid.ko 2024-05-22T20:59:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/usbhid/ 2024-05-22T20:59:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 153633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/usbhid/usbhid.ko 2024-05-22T20:59:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/usbhid/usbkbd.ko 2024-05-22T20:59:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/usbhid/usbmouse.ko 2024-05-22T20:59:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 261945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/wacom.ko 2024-05-22T20:59:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hv/ 2024-05-22T20:59:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 75065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hv/hv_balloon.ko 2024-05-22T20:59:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 110681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hv/hv_utils.ko 2024-05-22T20:59:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 382033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hv/hv_vmbus.ko 2024-05-22T20:59:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ 2024-05-22T20:59:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 77153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/abituguru.ko 2024-05-22T20:59:39.323 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 67009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/abituguru3.ko 2024-05-22T20:59:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 42249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-22T20:59:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ad7414.ko 2024-05-22T20:59:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ad7418.ko 2024-05-22T20:59:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adc128d818.ko 2024-05-22T20:59:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 38097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adm1025.ko 2024-05-22T20:59:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 115057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adm1026.ko 2024-05-22T20:59:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adm1029.ko 2024-05-22T20:59:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 55225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adm1031.ko 2024-05-22T20:59:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adm9240.ko 2024-05-22T20:59:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ads7828.ko 2024-05-22T20:59:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7410.ko 2024-05-22T20:59:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7411.ko 2024-05-22T20:59:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 89025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7462.ko 2024-05-22T20:59:39.324 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 49153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7470.ko 2024-05-22T20:59:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 81473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7475.ko 2024-05-22T20:59:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7x10.ko 2024-05-22T20:59:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 48177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/amc6821.ko 2024-05-22T20:59:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 68865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/applesmc.ko 2024-05-22T20:59:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 60193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/asb100.ko 2024-05-22T20:59:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 69009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/asc7621.ko 2024-05-22T20:59:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 47977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-22T20:59:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/atxp1.ko 2024-05-22T20:59:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/coretemp.ko 2024-05-22T20:59:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/da9052-hwmon.ko 2024-05-22T20:59:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/da9055-hwmon.ko 2024-05-22T20:59:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 55713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-22T20:59:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 113201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/dme1737.ko 2024-05-22T20:59:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ds1621.ko 2024-05-22T20:59:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ds620.ko 2024-05-22T20:59:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/emc1403.ko 2024-05-22T20:59:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/emc2103.ko 2024-05-22T20:59:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/emc6w201.ko 2024-05-22T20:59:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 78921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/f71805f.ko 2024-05-22T20:59:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 108961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/f71882fg.ko 2024-05-22T20:59:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 50697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/f75375s.ko 2024-05-22T20:59:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/fam15h_power.ko 2024-05-22T20:59:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 65769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/fschmd.ko 2024-05-22T20:59:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/g760a.ko 2024-05-22T20:59:39.326 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/g762.ko 2024-05-22T20:59:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 59761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/gl518sm.ko 2024-05-22T20:59:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 56313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/gl520sm.ko 2024-05-22T20:59:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/hih6130.ko 2024-05-22T20:59:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-22T20:59:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/i5500_temp.ko 2024-05-22T20:59:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/i5k_amb.ko 2024-05-22T20:59:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 42585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ibmaem.ko 2024-05-22T20:59:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ibmpex.ko 2024-05-22T20:59:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ina209.ko 2024-05-22T20:59:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ina2xx.ko 2024-05-22T20:59:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 156465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/it87.ko 2024-05-22T20:59:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/jc42.ko 2024-05-22T20:59:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/k10temp.ko 2024-05-22T20:59:39.327 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/k8temp.ko 2024-05-22T20:59:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lineage-pem.ko 2024-05-22T20:59:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 57769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm63.ko 2024-05-22T20:59:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm73.ko 2024-05-22T20:59:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm75.ko 2024-05-22T20:59:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm77.ko 2024-05-22T20:59:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 41473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm78.ko 2024-05-22T20:59:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 37049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm80.ko 2024-05-22T20:59:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20545 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm83.ko 2024-05-22T20:59:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 86497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm85.ko 2024-05-22T20:59:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 55641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm87.ko 2024-05-22T20:59:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 42249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm90.ko 2024-05-22T20:59:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm92.ko 2024-05-22T20:59:39.328 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 131505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm93.ko 2024-05-22T20:59:39.329 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm95234.ko 2024-05-22T20:59:39.329 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm95241.ko 2024-05-22T20:59:39.329 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm95245.ko 2024-05-22T20:59:39.329 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc2945.ko 2024-05-22T20:59:39.329 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4151.ko 2024-05-22T20:59:39.329 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4215.ko 2024-05-22T20:59:39.329 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4222.ko 2024-05-22T20:59:39.329 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4245.ko 2024-05-22T20:59:39.329 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4260.ko 2024-05-22T20:59:39.329 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4261.ko 2024-05-22T20:59:39.329 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max16065.ko 2024-05-22T20:59:39.329 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max1619.ko 2024-05-22T20:59:39.329 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max1668.ko 2024-05-22T20:59:39.330 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max197.ko 2024-05-22T20:59:39.330 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max6639.ko 2024-05-22T20:59:39.330 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max6650.ko 2024-05-22T20:59:39.330 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 39729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max6697.ko 2024-05-22T20:59:39.330 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/mc13783-adc.ko 2024-05-22T20:59:39.330 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/mcp3021.ko 2024-05-22T20:59:39.330 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-05-22T20:59:39.330 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 56505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/nct6683.ko 2024-05-22T20:59:39.330 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 141353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/nct6775-core.ko 2024-05-22T20:59:39.330 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 72737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/nct6775.ko 2024-05-22T20:59:39.330 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 67841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/nct7802.ko 2024-05-22T20:59:39.330 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 97777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pc87360.ko 2024-05-22T20:59:39.330 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 61921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pc87427.ko 2024-05-22T20:59:39.331 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pcf8591.ko 2024-05-22T20:59:39.331 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/ 2024-05-22T20:59:39.331 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-22T20:59:39.331 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-22T20:59:39.331 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-22T20:59:39.331 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-22T20:59:39.331 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-22T20:59:39.331 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-22T20:59:39.331 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-22T20:59:39.331 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 101233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-22T20:59:39.331 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-22T20:59:39.331 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-22T20:59:39.331 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-22T20:59:39.332 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-22T20:59:39.332 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/powr1220.ko 2024-05-22T20:59:39.332 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sch5627.ko 2024-05-22T20:59:39.332 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sch5636.ko 2024-05-22T20:59:39.332 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-22T20:59:39.332 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sht15.ko 2024-05-22T20:59:39.332 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sht21.ko 2024-05-22T20:59:39.332 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/shtc1.ko 2024-05-22T20:59:39.332 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 48713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sis5595.ko 2024-05-22T20:59:39.332 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/smsc47b397.ko 2024-05-22T20:59:39.332 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 46473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/smsc47m1.ko 2024-05-22T20:59:39.332 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/smsc47m192.ko 2024-05-22T20:59:39.333 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/thmc50.ko 2024-05-22T20:59:39.333 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/tmp102.ko 2024-05-22T20:59:39.333 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/tmp103.ko 2024-05-22T20:59:39.333 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/tmp401.ko 2024-05-22T20:59:39.333 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/tmp421.ko 2024-05-22T20:59:39.333 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/via-cputemp.ko 2024-05-22T20:59:39.333 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 50217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/via686a.ko 2024-05-22T20:59:39.333 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 59665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/vt1211.ko 2024-05-22T20:59:39.333 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 62737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/vt8231.ko 2024-05-22T20:59:39.333 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 71169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83627ehf.ko 2024-05-22T20:59:39.333 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 96577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83627hf.ko 2024-05-22T20:59:39.333 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 85913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83781d.ko 2024-05-22T20:59:39.333 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 75129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83791d.ko 2024-05-22T20:59:39.333 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 83425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83792d.ko 2024-05-22T20:59:39.334 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 116777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83793.ko 2024-05-22T20:59:39.334 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 134441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83795.ko 2024-05-22T20:59:39.334 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83l785ts.ko 2024-05-22T20:59:39.334 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83l786ng.ko 2024-05-22T20:59:39.334 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-05-22T20:59:39.334 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-05-22T20:59:39.334 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/ 2024-05-22T20:59:39.334 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/algos/ 2024-05-22T20:59:39.334 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-22T20:59:39.334 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-22T20:59:39.334 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/ 2024-05-22T20:59:39.334 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-05-22T20:59:39.335 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-05-22T20:59:39.339 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-05-22T20:59:39.339 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-22T20:59:39.339 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-22T20:59:39.339 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-22T20:59:39.339 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-05-22T20:59:39.339 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 7729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-22T20:59:39.339 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-05-22T20:59:39.339 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 64233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-22T20:59:39.339 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-22T20:59:39.339 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-22T20:59:39.339 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-22T20:59:39.340 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-05-22T20:59:39.340 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-05-22T20:59:39.340 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 76969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-22T20:59:39.340 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-22T20:59:39.340 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 52937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-22T20:59:39.340 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-05-22T20:59:39.340 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-22T20:59:39.340 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-22T20:59:39.340 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-05-22T20:59:39.340 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-22T20:59:39.340 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-22T20:59:39.340 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 57217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-22T20:59:39.340 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-05-22T20:59:39.341 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-22T20:59:39.341 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-22T20:59:39.341 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-05-22T20:59:39.341 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-05-22T20:59:39.341 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-22T20:59:39.341 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-05-22T20:59:39.341 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-22T20:59:39.341 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-22T20:59:39.341 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-22T20:59:39.341 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-22T20:59:39.341 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 59193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-05-22T20:59:39.341 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/i2c-mux.ko 2024-05-22T20:59:39.341 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/i2c-smbus.ko 2024-05-22T20:59:39.342 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/i2c-stub.ko 2024-05-22T20:59:39.342 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/muxes/ 2024-05-22T20:59:39.342 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-05-22T20:59:39.342 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-05-22T20:59:39.342 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-05-22T20:59:39.342 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ 2024-05-22T20:59:39.342 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/ 2024-05-22T20:59:39.342 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 329697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-22T20:59:39.342 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1119913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/ib_core.ko 2024-05-22T20:59:39.342 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 83905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-22T20:59:39.342 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 408433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-22T20:59:39.342 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 120921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-22T20:59:39.342 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 347217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-22T20:59:39.343 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 84945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-22T20:59:39.343 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/ 2024-05-22T20:59:39.343 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/cxgb4/ 2024-05-22T20:59:39.343 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 590929 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-22T20:59:39.343 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mlx4/ 2024-05-22T20:59:39.343 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 514641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-22T20:59:39.343 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mlx5/ 2024-05-22T20:59:39.343 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 911665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-22T20:59:39.343 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mthca/ 2024-05-22T20:59:39.343 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 375545 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-22T20:59:39.343 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/ocrdma/ 2024-05-22T20:59:39.343 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 197121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-05-22T20:59:39.343 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/usnic/ 2024-05-22T20:59:39.344 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 212553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-05-22T20:59:39.344 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/ 2024-05-22T20:59:39.344 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/ipoib/ 2024-05-22T20:59:39.344 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 334401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-22T20:59:39.344 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/iser/ 2024-05-22T20:59:39.344 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 120673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-22T20:59:39.344 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/isert/ 2024-05-22T20:59:39.344 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 132585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-22T20:59:39.344 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/srp/ 2024-05-22T20:59:39.344 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 157937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-22T20:59:39.344 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/srpt/ 2024-05-22T20:59:39.344 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 156673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-22T20:59:39.345 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/ 2024-05-22T20:59:39.345 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/evbug.ko 2024-05-22T20:59:39.345 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/ff-memless.ko 2024-05-22T20:59:39.345 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/gameport/ 2024-05-22T20:59:39.345 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-22T20:59:39.345 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-22T20:59:39.345 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 50713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/gameport/gameport.ko 2024-05-22T20:59:39.345 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 49441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/joydev.ko 2024-05-22T20:59:39.345 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/ 2024-05-22T20:59:39.345 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-05-22T20:59:39.345 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-05-22T20:59:39.345 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-05-22T20:59:39.345 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-05-22T20:59:39.346 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 45473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-22T20:59:39.346 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-05-22T20:59:39.346 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/lkkbd.ko 2024-05-22T20:59:39.346 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/lm8323.ko 2024-05-22T20:59:39.346 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/lm8333.ko 2024-05-22T20:59:39.346 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-05-22T20:59:39.346 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-05-22T20:59:39.346 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-05-22T20:59:39.346 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-05-22T20:59:39.346 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/newtonkbd.ko 2024-05-22T20:59:39.346 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-05-22T20:59:39.346 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/qt1070.ko 2024-05-22T20:59:39.347 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/qt2160.ko 2024-05-22T20:59:39.347 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-05-22T20:59:39.347 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/stowaway.ko 2024-05-22T20:59:39.347 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/sunkbd.ko 2024-05-22T20:59:39.347 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-05-22T20:59:39.347 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-05-22T20:59:39.347 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-05-22T20:59:39.347 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/xtkbd.ko 2024-05-22T20:59:39.347 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/matrix-keymap.ko 2024-05-22T20:59:39.347 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/ 2024-05-22T20:59:39.347 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/appletouch.ko 2024-05-22T20:59:39.347 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 39553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/bcm5974.ko 2024-05-22T20:59:39.347 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 160017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/cyapatp.ko 2024-05-22T20:59:39.347 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 114273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-22T20:59:39.348 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/gpio_mouse.ko 2024-05-22T20:59:39.348 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 428185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/psmouse.ko 2024-05-22T20:59:39.348 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/sermouse.ko 2024-05-22T20:59:39.348 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-22T20:59:39.348 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-22T20:59:39.348 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-22T20:59:39.348 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/rmi4/ 2024-05-22T20:59:39.348 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 140969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-22T20:59:39.348 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/ 2024-05-22T20:59:39.348 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/altera_ps2.ko 2024-05-22T20:59:39.348 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/arc_ps2.ko 2024-05-22T20:59:39.348 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/ct82c710.ko 2024-05-22T20:59:39.349 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-22T20:59:39.349 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/parkbd.ko 2024-05-22T20:59:39.349 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/pcips2.ko 2024-05-22T20:59:39.349 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/ps2mult.ko 2024-05-22T20:59:39.349 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/serio_raw.ko 2024-05-22T20:59:39.349 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/serport.ko 2024-05-22T20:59:39.349 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/sparse-keymap.ko 2024-05-22T20:59:39.349 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/ 2024-05-22T20:59:39.349 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-88pm860x.ko 2024-05-22T20:59:39.349 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-adp5520.ko 2024-05-22T20:59:39.349 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 77265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-bd2802.ko 2024-05-22T20:59:39.349 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-blinkm.ko 2024-05-22T20:59:39.349 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11897 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-da903x.ko 2024-05-22T20:59:39.350 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-da9052.ko 2024-05-22T20:59:39.350 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-gpio.ko 2024-05-22T20:59:39.350 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-lm3530.ko 2024-05-22T20:59:39.350 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-lm3533.ko 2024-05-22T20:59:39.350 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-lm355x.ko 2024-05-22T20:59:39.350 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-lm3642.ko 2024-05-22T20:59:39.350 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-lp3944.ko 2024-05-22T20:59:39.350 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-lp8788.ko 2024-05-22T20:59:39.350 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-max8997.ko 2024-05-22T20:59:39.350 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-mc13783.ko 2024-05-22T20:59:39.350 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-menf21bmc.ko 2024-05-22T20:59:39.350 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-pca9532.ko 2024-05-22T20:59:39.350 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-pca955x.ko 2024-05-22T20:59:39.351 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-pca963x.ko 2024-05-22T20:59:39.351 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 38113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-ss4200.ko 2024-05-22T20:59:39.351 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-tca6507.ko 2024-05-22T20:59:39.351 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-wm831x-status.ko 2024-05-22T20:59:39.351 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-wm8350.ko 2024-05-22T20:59:39.351 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ 2024-05-22T20:59:39.351 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-22T20:59:39.351 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-22T20:59:39.351 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-22T20:59:39.351 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-22T20:59:39.351 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-22T20:59:39.351 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-22T20:59:39.351 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-22T20:59:39.352 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-22T20:59:39.352 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-22T20:59:39.352 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/ 2024-05-22T20:59:39.352 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/bcache/ 2024-05-22T20:59:39.352 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 790753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/bcache/bcache.ko 2024-05-22T20:59:39.352 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 49081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-bio-prison.ko 2024-05-22T20:59:39.352 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 116825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-bufio.ko 2024-05-22T20:59:39.352 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 58281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-cache-smq.ko 2024-05-22T20:59:39.352 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 199041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-cache.ko 2024-05-22T20:59:39.352 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 134945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-crypt.ko 2024-05-22T20:59:39.352 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28897 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-delay.ko 2024-05-22T20:59:39.352 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 67497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-era.ko 2024-05-22T20:59:39.352 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-flakey.ko 2024-05-22T20:59:39.353 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 50457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-log-userspace.ko 2024-05-22T20:59:39.353 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 41145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-log.ko 2024-05-22T20:59:39.353 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 60977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-mirror.ko 2024-05-22T20:59:39.353 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 97209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-multipath.ko 2024-05-22T20:59:39.353 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-queue-length.ko 2024-05-22T20:59:39.353 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 85689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-raid.ko 2024-05-22T20:59:39.353 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-region-hash.ko 2024-05-22T20:59:39.353 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-round-robin.ko 2024-05-22T20:59:39.353 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-service-time.ko 2024-05-22T20:59:39.353 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 139817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-snapshot.ko 2024-05-22T20:59:39.353 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-switch.ko 2024-05-22T20:59:39.353 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 234457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-thin-pool.ko 2024-05-22T20:59:39.353 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 66425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-verity.ko 2024-05-22T20:59:39.354 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-zero.ko 2024-05-22T20:59:39.354 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/persistent-data/ 2024-05-22T20:59:39.354 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 264329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-22T20:59:39.354 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/raid0.ko 2024-05-22T20:59:39.354 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 125009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/raid1.ko 2024-05-22T20:59:39.354 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 157681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/raid10.ko 2024-05-22T20:59:39.354 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 444665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/raid456.ko 2024-05-22T20:59:39.354 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/ 2024-05-22T20:59:39.354 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/ 2024-05-22T20:59:39.354 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 214841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptbase.ko 2024-05-22T20:59:39.354 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 77985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptctl.ko 2024-05-22T20:59:39.354 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 54785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptfc.ko 2024-05-22T20:59:39.355 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptlan.ko 2024-05-22T20:59:39.355 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 150761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptsas.ko 2024-05-22T20:59:39.355 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 97281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptscsih.ko 2024-05-22T20:59:39.355 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 63873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptspi.ko 2024-05-22T20:59:39.355 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/ 2024-05-22T20:59:39.355 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/88pm800.ko 2024-05-22T20:59:39.355 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/88pm805.ko 2024-05-22T20:59:39.355 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/88pm80x.ko 2024-05-22T20:59:39.355 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/arizona-i2c.ko 2024-05-22T20:59:39.355 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 148633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/arizona.ko 2024-05-22T20:59:39.355 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/bcm590xx.ko 2024-05-22T20:59:39.355 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/cros_ec_dev.ko 2024-05-22T20:59:39.355 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 41009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/dln2.ko 2024-05-22T20:59:39.356 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/janz-cmodio.ko 2024-05-22T20:59:39.356 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 59089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/kempld-core.ko 2024-05-22T20:59:39.356 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31897 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/lm3533-core.ko 2024-05-22T20:59:39.356 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-05-22T20:59:39.356 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/lp3943.ko 2024-05-22T20:59:39.356 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 60497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/lpc_ich.ko 2024-05-22T20:59:39.356 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/lpc_sch.ko 2024-05-22T20:59:39.356 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/max8907.ko 2024-05-22T20:59:39.356 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/mc13xxx-core.ko 2024-05-22T20:59:39.356 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-05-22T20:59:39.356 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/menf21bmc.ko 2024-05-22T20:59:39.356 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/pcf50633-adc.ko 2024-05-22T20:59:39.356 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/pcf50633-gpio.ko 2024-05-22T20:59:39.357 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 42297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/pcf50633.ko 2024-05-22T20:59:39.357 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-05-22T20:59:39.357 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/retu-mfd.ko 2024-05-22T20:59:39.357 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 99425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/si476x-core.ko 2024-05-22T20:59:39.357 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 58057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/sm501.ko 2024-05-22T20:59:39.357 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 60681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/tps65010.ko 2024-05-22T20:59:39.357 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/tps6507x.ko 2024-05-22T20:59:39.357 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/viperboard.ko 2024-05-22T20:59:39.357 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/vx855.ko 2024-05-22T20:59:39.357 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/wl1273-core.ko 2024-05-22T20:59:39.357 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 52361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/wm8994.ko 2024-05-22T20:59:39.357 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ 2024-05-22T20:59:39.357 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-05-22T20:59:39.358 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 64265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ad525x_dpot.ko 2024-05-22T20:59:39.358 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/altera-stapl/ 2024-05-22T20:59:39.358 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 57521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-22T20:59:39.358 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/apds9802als.ko 2024-05-22T20:59:39.358 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 47689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/apds990x.ko 2024-05-22T20:59:39.358 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 58633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/bh1770glc.ko 2024-05-22T20:59:39.358 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/c2port/ 2024-05-22T20:59:39.358 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-05-22T20:59:39.358 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 47449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/c2port/core.ko 2024-05-22T20:59:39.358 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/cb710/ 2024-05-22T20:59:39.358 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/cb710/cb710.ko 2024-05-22T20:59:39.358 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ds1682.ko 2024-05-22T20:59:39.359 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/dummy-irq.ko 2024-05-22T20:59:39.359 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/echo/ 2024-05-22T20:59:39.359 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/echo/echo.ko 2024-05-22T20:59:39.359 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/eeprom/ 2024-05-22T20:59:39.359 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/eeprom/at24.ko 2024-05-22T20:59:39.359 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-22T20:59:39.359 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/eeprom/max6875.ko 2024-05-22T20:59:39.359 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 41185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/enclosure.ko 2024-05-22T20:59:39.359 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/genwqe/ 2024-05-22T20:59:39.359 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 191649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-05-22T20:59:39.359 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/hmc6352.ko 2024-05-22T20:59:39.359 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 37761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/hpilo.ko 2024-05-22T20:59:39.360 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ibmasm/ 2024-05-22T20:59:39.360 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 92345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-05-22T20:59:39.360 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ics932s401.ko 2024-05-22T20:59:39.360 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/isl29003.ko 2024-05-22T20:59:39.360 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/isl29020.ko 2024-05-22T20:59:39.360 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/lis3lv02d/ 2024-05-22T20:59:39.360 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 54865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-22T20:59:39.360 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-22T20:59:39.360 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/mei/ 2024-05-22T20:59:39.360 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 154169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/mei/mei-me.ko 2024-05-22T20:59:39.360 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 69809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/mei/mei-txe.ko 2024-05-22T20:59:39.360 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 389033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/mei/mei.ko 2024-05-22T20:59:39.360 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 37961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/phantom.ko 2024-05-22T20:59:39.361 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ti-st/ 2024-05-22T20:59:39.361 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 91145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/ti-st/st_drv.ko 2024-05-22T20:59:39.361 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/tifm_7xx1.ko 2024-05-22T20:59:39.361 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/tifm_core.ko 2024-05-22T20:59:39.361 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/tsl2550.ko 2024-05-22T20:59:39.361 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 55793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/vmw_balloon.ko 2024-05-22T20:59:39.361 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/vmw_vmci/ 2024-05-22T20:59:39.361 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 234905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-22T20:59:39.361 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ 2024-05-22T20:59:39.361 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/bonding/ 2024-05-22T20:59:39.361 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 620961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/bonding/bonding.ko 2024-05-22T20:59:39.361 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/ 2024-05-22T20:59:39.361 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/b53/ 2024-05-22T20:59:39.362 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 139817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-22T20:59:39.362 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 49545 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-22T20:59:39.362 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-22T20:59:39.362 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/mv88e6xxx/ 2024-05-22T20:59:39.362 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 587065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-22T20:59:39.362 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dummy.ko 2024-05-22T20:59:39.362 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/eql.ko 2024-05-22T20:59:39.362 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/ 2024-05-22T20:59:39.362 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/3com/ 2024-05-22T20:59:39.362 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 53753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-22T20:59:39.362 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 49201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-22T20:59:39.362 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 158649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-22T20:59:39.362 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 85953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-22T20:59:39.363 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/8390/ 2024-05-22T20:59:39.363 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-22T20:59:39.363 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 73865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-22T20:59:39.363 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-22T20:59:39.363 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 109521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-22T20:59:39.363 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/adaptec/ 2024-05-22T20:59:39.363 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 64705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-22T20:59:39.363 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/agere/ 2024-05-22T20:59:39.363 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 84329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-22T20:59:39.363 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/alteon/ 2024-05-22T20:59:39.363 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 70737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-22T20:59:39.363 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/altera/ 2024-05-22T20:59:39.363 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 52705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-22T20:59:39.363 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/amd/ 2024-05-22T20:59:39.364 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 65857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-22T20:59:39.364 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 56057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-22T20:59:39.364 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 115881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-22T20:59:39.364 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/ 2024-05-22T20:59:39.364 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/alx/ 2024-05-22T20:59:39.364 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 124497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-22T20:59:39.364 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-05-22T20:59:39.364 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 128409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-22T20:59:39.364 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-05-22T20:59:39.364 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 110345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-22T20:59:39.364 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atlx/ 2024-05-22T20:59:39.364 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 101857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-22T20:59:39.364 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 88953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-22T20:59:39.365 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/ 2024-05-22T20:59:39.365 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 105633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-22T20:59:39.365 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 228985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-22T20:59:39.365 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-05-22T20:59:39.365 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1733353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-22T20:59:39.366 INFO:teuthology.orchestra.run.smithi179.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-22T20:59:39.366 INFO:teuthology.orchestra.run.smithi179.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-22T20:59:39.432 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 167185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-22T20:59:39.432 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/genet/ 2024-05-22T20:59:39.432 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 161417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-22T20:59:39.433 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 440625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-22T20:59:39.433 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/brocade/ 2024-05-22T20:59:39.433 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/brocade/bna/ 2024-05-22T20:59:39.433 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 432201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-22T20:59:39.433 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/ 2024-05-22T20:59:39.433 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-05-22T20:59:39.433 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 257201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-22T20:59:39.433 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-05-22T20:59:39.433 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 446177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-22T20:59:39.433 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-05-22T20:59:39.433 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1311433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-22T20:59:39.433 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-05-22T20:59:39.433 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 206689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-22T20:59:39.434 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-05-22T20:59:39.434 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-22T20:59:39.434 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/cisco/ 2024-05-22T20:59:39.434 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/cisco/enic/ 2024-05-22T20:59:39.434 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 269313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-22T20:59:39.434 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/ 2024-05-22T20:59:39.434 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/ 2024-05-22T20:59:39.434 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 85385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-22T20:59:39.434 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 77457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-22T20:59:39.434 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 169089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-22T20:59:39.434 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 65913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-22T20:59:39.434 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 76209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-22T20:59:39.435 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 38065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-22T20:59:39.435 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dlink/ 2024-05-22T20:59:39.435 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 75945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-22T20:59:39.435 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 78705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-22T20:59:39.435 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 52313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dnet.ko 2024-05-22T20:59:39.435 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/ec_bhf.ko 2024-05-22T20:59:39.435 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/emulex/ 2024-05-22T20:59:39.435 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/emulex/benet/ 2024-05-22T20:59:39.435 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 356377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-22T20:59:39.435 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 61737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/ethoc.ko 2024-05-22T20:59:39.435 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 63185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/fealnx.ko 2024-05-22T20:59:39.435 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/fujitsu/ 2024-05-22T20:59:39.435 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 51513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-05-22T20:59:39.436 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/ 2024-05-22T20:59:39.436 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 115569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-22T20:59:39.436 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/e1000/ 2024-05-22T20:59:39.436 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 366217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-22T20:59:39.436 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/e1000e/ 2024-05-22T20:59:39.436 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 755793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-22T20:59:39.436 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/fm10k/ 2024-05-22T20:59:39.436 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 324577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-22T20:59:39.436 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/i40e/ 2024-05-22T20:59:39.436 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1196297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-22T20:59:39.436 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/iavf/ 2024-05-22T20:59:39.436 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 362593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-22T20:59:39.436 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/igb/ 2024-05-22T20:59:39.436 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 671801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-22T20:59:39.437 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/igbvf/ 2024-05-22T20:59:39.437 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 121537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-22T20:59:39.437 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-05-22T20:59:39.437 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 966705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-22T20:59:39.437 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-05-22T20:59:39.437 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 193585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-22T20:59:39.437 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 108633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/jme.ko 2024-05-22T20:59:39.437 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/marvell/ 2024-05-22T20:59:39.437 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-22T20:59:39.437 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 126881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-22T20:59:39.437 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 150777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-22T20:59:39.437 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/ 2024-05-22T20:59:39.437 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-05-22T20:59:39.437 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 905033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-22T20:59:39.438 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 344145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-22T20:59:39.438 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-05-22T20:59:39.438 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-05-22T20:59:39.438 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1412417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-22T20:59:39.438 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/micrel/ 2024-05-22T20:59:39.438 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 65913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-05-22T20:59:39.438 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 54497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-05-22T20:59:39.438 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-05-22T20:59:39.438 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 117353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-22T20:59:39.438 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/myricom/ 2024-05-22T20:59:39.438 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-05-22T20:59:39.438 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 131425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-22T20:59:39.439 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/natsemi/ 2024-05-22T20:59:39.439 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 91465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-22T20:59:39.439 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 56689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-22T20:59:39.439 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/neterion/ 2024-05-22T20:59:39.439 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 181081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-22T20:59:39.439 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/nvidia/ 2024-05-22T20:59:39.439 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 171417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-22T20:59:39.439 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/packetengines/ 2024-05-22T20:59:39.439 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 58249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-22T20:59:39.439 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 50121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-22T20:59:39.439 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/ 2024-05-22T20:59:39.439 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-05-22T20:59:39.439 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 300817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-22T20:59:39.440 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 105689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-22T20:59:39.440 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-05-22T20:59:39.440 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 885129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-22T20:59:39.440 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/rdc/ 2024-05-22T20:59:39.440 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 52305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-22T20:59:39.440 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/realtek/ 2024-05-22T20:59:39.440 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 81641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-22T20:59:39.440 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 100689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-22T20:59:39.440 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-22T20:59:39.440 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 264161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-22T20:59:39.440 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/samsung/ 2024-05-22T20:59:39.440 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-05-22T20:59:39.440 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 142449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-05-22T20:59:39.441 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sfc/ 2024-05-22T20:59:39.441 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1350385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-22T20:59:39.441 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/silan/ 2024-05-22T20:59:39.441 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 57953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-22T20:59:39.441 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sis/ 2024-05-22T20:59:39.441 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 66505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-22T20:59:39.441 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 80065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-22T20:59:39.441 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/smsc/ 2024-05-22T20:59:39.441 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 74289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-22T20:59:39.441 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 79457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-22T20:59:39.441 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 100785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-22T20:59:39.441 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 83441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-22T20:59:39.441 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/stmicro/ 2024-05-22T20:59:39.442 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-05-22T20:59:39.442 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-05-22T20:59:39.442 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 46985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-22T20:59:39.442 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-05-22T20:59:39.442 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 644537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-22T20:59:39.442 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sun/ 2024-05-22T20:59:39.442 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 128497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-22T20:59:39.442 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 207993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-22T20:59:39.442 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 82633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-22T20:59:39.442 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 91353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-22T20:59:39.442 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/tehuti/ 2024-05-22T20:59:39.442 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 59865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-22T20:59:39.443 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/ti/ 2024-05-22T20:59:39.443 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 110729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-22T20:59:39.443 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/via/ 2024-05-22T20:59:39.443 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 98689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-22T20:59:39.443 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 102513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-22T20:59:39.443 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/wiznet/ 2024-05-22T20:59:39.443 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 61497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-22T20:59:39.443 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-22T20:59:39.443 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/xircom/ 2024-05-22T20:59:39.443 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 69913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-22T20:59:39.443 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/fddi/ 2024-05-22T20:59:39.443 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 49449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/fddi/defxx.ko 2024-05-22T20:59:39.444 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/fddi/skfp/ 2024-05-22T20:59:39.444 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 275801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/fddi/skfp/skfp.ko 2024-05-22T20:59:39.444 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 91105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/geneve.ko 2024-05-22T20:59:39.444 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/hyperv/ 2024-05-22T20:59:39.444 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 290441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-22T20:59:39.444 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ieee802154/ 2024-05-22T20:59:39.444 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-22T20:59:39.444 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ifb.ko 2024-05-22T20:59:39.444 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ipvlan/ 2024-05-22T20:59:39.444 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 108329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-22T20:59:39.444 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 78065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/macvlan.ko 2024-05-22T20:59:39.444 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/macvtap.ko 2024-05-22T20:59:39.444 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio/ 2024-05-22T20:59:39.445 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-22T20:59:39.445 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-22T20:59:39.445 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio/mdio-gpio.ko 2024-05-22T20:59:39.445 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-22T20:59:39.445 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio.ko 2024-05-22T20:59:39.445 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mii.ko 2024-05-22T20:59:39.445 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 62681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/netconsole.ko 2024-05-22T20:59:39.445 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/nlmon.ko 2024-05-22T20:59:39.445 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/pcs/ 2024-05-22T20:59:39.445 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-22T20:59:39.445 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 48313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-22T20:59:39.445 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/ 2024-05-22T20:59:39.445 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/amd.ko 2024-05-22T20:59:39.445 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 61985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-22T20:59:39.446 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 52865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-22T20:59:39.446 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/bcm87xx.ko 2024-05-22T20:59:39.446 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 53409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/broadcom.ko 2024-05-22T20:59:39.446 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/cicada.ko 2024-05-22T20:59:39.446 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/davicom.ko 2024-05-22T20:59:39.446 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/et1011c.ko 2024-05-22T20:59:39.446 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/icplus.ko 2024-05-22T20:59:39.446 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/lxt.ko 2024-05-22T20:59:39.446 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 103377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/marvell.ko 2024-05-22T20:59:39.446 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 96025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/micrel.ko 2024-05-22T20:59:39.446 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/national.ko 2024-05-22T20:59:39.446 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 121105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/phylink.ko 2024-05-22T20:59:39.446 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/qsemi.ko 2024-05-22T20:59:39.447 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 62113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/realtek.ko 2024-05-22T20:59:39.447 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/smsc.ko 2024-05-22T20:59:39.447 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/ste10Xp.ko 2024-05-22T20:59:39.447 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/vitesse.ko 2024-05-22T20:59:39.447 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/plip/ 2024-05-22T20:59:39.447 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 55353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/plip/plip.ko 2024-05-22T20:59:39.447 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/ 2024-05-22T20:59:39.447 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-22T20:59:39.447 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/ppp_async.ko 2024-05-22T20:59:39.447 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-22T20:59:39.447 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-22T20:59:39.447 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-22T20:59:39.447 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 53233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/pppoe.ko 2024-05-22T20:59:39.448 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/pppox.ko 2024-05-22T20:59:39.448 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/pptp.ko 2024-05-22T20:59:39.448 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/sb1000.ko 2024-05-22T20:59:39.448 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/slip/ 2024-05-22T20:59:39.448 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 52073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/slip/slip.ko 2024-05-22T20:59:39.448 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 42409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/sungem_phy.ko 2024-05-22T20:59:39.448 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 80161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/tap.ko 2024-05-22T20:59:39.448 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/ 2024-05-22T20:59:39.448 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 137321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team.ko 2024-05-22T20:59:39.448 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-22T20:59:39.448 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-22T20:59:39.448 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-22T20:59:39.448 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team_mode_random.ko 2024-05-22T20:59:39.449 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-22T20:59:39.449 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 108089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/veth.ko 2024-05-22T20:59:39.449 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/vmxnet3/ 2024-05-22T20:59:39.449 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 176497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-22T20:59:39.449 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/vxlan/ 2024-05-22T20:59:39.449 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 383593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/vxlan/vxlan.ko 2024-05-22T20:59:39.449 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/xen-netback/ 2024-05-22T20:59:39.449 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 175537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-22T20:59:39.449 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ntb/ 2024-05-22T20:59:39.449 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ntb/ntb.ko 2024-05-22T20:59:39.449 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/nvme/ 2024-05-22T20:59:39.449 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/nvme/host/ 2024-05-22T20:59:39.450 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 335857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/nvme/host/nvme-core.ko 2024-05-22T20:59:39.450 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 126553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/nvme/host/nvme.ko 2024-05-22T20:59:39.450 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/parport/ 2024-05-22T20:59:39.450 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 168145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/parport/parport.ko 2024-05-22T20:59:39.450 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/parport/parport_cs.ko 2024-05-22T20:59:39.450 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 103889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/parport/parport_pc.ko 2024-05-22T20:59:39.450 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/parport/parport_serial.ko 2024-05-22T20:59:39.450 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/ 2024-05-22T20:59:39.450 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/hotplug/ 2024-05-22T20:59:39.450 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-22T20:59:39.450 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-05-22T20:59:39.450 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-05-22T20:59:39.450 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/pci-stub.ko 2024-05-22T20:59:39.451 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 55329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/xen-pcifront.ko 2024-05-22T20:59:39.451 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/ 2024-05-22T20:59:39.451 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/i82092.ko 2024-05-22T20:59:39.451 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 179961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/pcmcia.ko 2024-05-22T20:59:39.451 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 78161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/pcmcia_core.ko 2024-05-22T20:59:39.451 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 41945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-05-22T20:59:39.451 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/pd6729.ko 2024-05-22T20:59:39.451 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 133753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-22T20:59:39.451 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/phy/ 2024-05-22T20:59:39.451 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/phy/broadcom/ 2024-05-22T20:59:39.451 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-05-22T20:59:39.451 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/phy/samsung/ 2024-05-22T20:59:39.452 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-05-22T20:59:39.452 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pinctrl/ 2024-05-22T20:59:39.452 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pinctrl/intel/ 2024-05-22T20:59:39.452 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 78193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-22T20:59:39.452 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/ 2024-05-22T20:59:39.452 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/chrome/ 2024-05-22T20:59:39.452 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 61041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-05-22T20:59:39.452 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/chrome/cros_ec.ko 2024-05-22T20:59:39.452 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-05-22T20:59:39.452 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-05-22T20:59:39.452 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-05-22T20:59:39.452 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-05-22T20:59:39.452 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-05-22T20:59:39.453 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/ 2024-05-22T20:59:39.453 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 87481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-22T20:59:39.453 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/acerhdf.ko 2024-05-22T20:59:39.453 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 48257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-22T20:59:39.453 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 73865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-22T20:59:39.453 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-22T20:59:39.453 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 164769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-22T20:59:39.453 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 46185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-22T20:59:39.453 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 62073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-22T20:59:39.453 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-22T20:59:39.453 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 51881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-22T20:59:39.453 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-22T20:59:39.453 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 42761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/hdaps.ko 2024-05-22T20:59:39.454 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/ibm_rtl.ko 2024-05-22T20:59:39.454 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/intel/ 2024-05-22T20:59:39.454 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-22T20:59:39.454 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-05-22T20:59:39.454 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 54001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/intel_ips.ko 2024-05-22T20:59:39.454 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-22T20:59:39.454 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-22T20:59:39.454 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-22T20:59:39.454 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 51897 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-22T20:59:39.454 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-22T20:59:39.454 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-22T20:59:39.454 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-22T20:59:39.455 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-22T20:59:39.455 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-22T20:59:39.455 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 61417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/wmi.ko 2024-05-22T20:59:39.455 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/ 2024-05-22T20:59:39.455 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/ 2024-05-22T20:59:39.455 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/88pm860x_battery.ko 2024-05-22T20:59:39.455 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/88pm860x_charger.ko 2024-05-22T20:59:39.455 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 68833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/bq2415x_charger.ko 2024-05-22T20:59:39.455 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 73073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/bq24190_charger.ko 2024-05-22T20:59:39.455 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/bq24735-charger.ko 2024-05-22T20:59:39.455 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-05-22T20:59:39.455 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/da9030_battery.ko 2024-05-22T20:59:39.455 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/da9052-battery.ko 2024-05-22T20:59:39.455 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/ds2760_battery.ko 2024-05-22T20:59:39.456 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/ds2780_battery.ko 2024-05-22T20:59:39.456 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/ds2781_battery.ko 2024-05-22T20:59:39.456 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/ds2782_battery.ko 2024-05-22T20:59:39.456 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/gpio-charger.ko 2024-05-22T20:59:39.456 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/isp1704_charger.ko 2024-05-22T20:59:39.456 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/lp8727_charger.ko 2024-05-22T20:59:39.456 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/max14577_charger.ko 2024-05-22T20:59:39.456 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 37833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/max17042_battery.ko 2024-05-22T20:59:39.456 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/max8903_charger.ko 2024-05-22T20:59:39.456 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/max8925_power.ko 2024-05-22T20:59:39.456 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/pcf50633-charger.ko 2024-05-22T20:59:39.456 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/sbs-battery.ko 2024-05-22T20:59:39.456 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/test_power.ko 2024-05-22T20:59:39.457 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/tps65090-charger.ko 2024-05-22T20:59:39.457 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/wm831x_backup.ko 2024-05-22T20:59:39.457 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/wm831x_power.ko 2024-05-22T20:59:39.457 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/power/supply/wm8350_power.ko 2024-05-22T20:59:39.457 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/powercap/ 2024-05-22T20:59:39.457 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 73297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-22T20:59:39.457 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-22T20:59:39.457 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/ 2024-05-22T20:59:39.457 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/clients/ 2024-05-22T20:59:39.457 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-22T20:59:39.457 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-22T20:59:39.457 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/clients/pps_parport.ko 2024-05-22T20:59:39.457 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 49817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/pps_core.ko 2024-05-22T20:59:39.458 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ptp/ 2024-05-22T20:59:39.458 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 91393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ptp/ptp.ko 2024-05-22T20:59:39.458 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ptp/ptp_kvm.ko 2024-05-22T20:59:39.458 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ras/ 2024-05-22T20:59:39.458 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ras/amd/ 2024-05-22T20:59:39.458 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ras/amd/atl/ 2024-05-22T20:59:39.458 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 67513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-22T20:59:39.458 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 47137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ras/amd/fmpm.ko 2024-05-22T20:59:39.458 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/ 2024-05-22T20:59:39.458 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-88pm80x.ko 2024-05-22T20:59:39.458 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-88pm860x.ko 2024-05-22T20:59:39.458 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-22T20:59:39.458 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-da9052.ko 2024-05-22T20:59:39.459 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-da9055.ko 2024-05-22T20:59:39.459 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-da9063.ko 2024-05-22T20:59:39.459 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-22T20:59:39.459 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 82225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-22T20:59:39.459 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-22T20:59:39.459 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-22T20:59:39.459 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-22T20:59:39.459 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-22T20:59:39.459 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-22T20:59:39.459 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-22T20:59:39.459 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-22T20:59:39.459 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-em3027.ko 2024-05-22T20:59:39.459 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-22T20:59:39.460 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-22T20:59:39.460 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-22T20:59:39.460 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-lp8788.ko 2024-05-22T20:59:39.460 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 46913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-22T20:59:39.460 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-22T20:59:39.460 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-22T20:59:39.460 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-m48t86.ko 2024-05-22T20:59:39.460 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-max6900.ko 2024-05-22T20:59:39.460 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-max8907.ko 2024-05-22T20:59:39.460 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-max8925.ko 2024-05-22T20:59:39.460 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-max8997.ko 2024-05-22T20:59:39.460 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-max8998.ko 2024-05-22T20:59:39.460 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-05-22T20:59:39.461 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-22T20:59:39.461 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-palmas.ko 2024-05-22T20:59:39.461 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 56777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-22T20:59:39.461 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf50633.ko 2024-05-22T20:59:39.461 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-22T20:59:39.461 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-22T20:59:39.461 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-22T20:59:39.461 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-22T20:59:39.461 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rc5t583.ko 2024-05-22T20:59:39.461 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-22T20:59:39.461 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-22T20:59:39.461 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-22T20:59:39.461 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-22T20:59:39.461 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-22T20:59:39.462 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-s35390a.ko 2024-05-22T20:59:39.462 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-22T20:59:39.462 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-tps6586x.ko 2024-05-22T20:59:39.462 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-tps65910.ko 2024-05-22T20:59:39.462 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-wm831x.ko 2024-05-22T20:59:39.462 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-wm8350.ko 2024-05-22T20:59:39.462 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-x1205.ko 2024-05-22T20:59:39.462 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/ 2024-05-22T20:59:39.462 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 90377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/3w-9xxx.ko 2024-05-22T20:59:39.462 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 69593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/3w-sas.ko 2024-05-22T20:59:39.462 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 65945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/3w-xxxx.ko 2024-05-22T20:59:39.462 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 164569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/BusLogic.ko 2024-05-22T20:59:39.463 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/a100u2w.ko 2024-05-22T20:59:39.463 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aacraid/ 2024-05-22T20:59:39.463 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 340721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-22T20:59:39.463 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 120337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/advansys.ko 2024-05-22T20:59:39.463 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aic7xxx/ 2024-05-22T20:59:39.463 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 468081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-22T20:59:39.463 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 380673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-22T20:59:39.463 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aic94xx/ 2024-05-22T20:59:39.463 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 243705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-05-22T20:59:39.463 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/am53c974.ko 2024-05-22T20:59:39.463 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/arcmsr/ 2024-05-22T20:59:39.463 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 135617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-22T20:59:39.464 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/atp870u.ko 2024-05-22T20:59:39.464 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/be2iscsi/ 2024-05-22T20:59:39.464 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 304993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-22T20:59:39.464 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bfa/ 2024-05-22T20:59:39.464 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1527113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bfa/bfa.ko 2024-05-22T20:59:39.464 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bnx2fc/ 2024-05-22T20:59:39.464 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 297801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-22T20:59:39.464 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bnx2i/ 2024-05-22T20:59:39.464 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 144401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-22T20:59:39.464 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 47457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/ch.ko 2024-05-22T20:59:39.464 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/csiostor/ 2024-05-22T20:59:39.464 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 390713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-22T20:59:39.464 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/ 2024-05-22T20:59:39.464 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-05-22T20:59:39.465 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 88881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-22T20:59:39.465 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-05-22T20:59:39.465 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 117481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-22T20:59:39.465 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 173017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-22T20:59:39.465 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 91569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/dc395x.ko 2024-05-22T20:59:39.465 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 45401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/dmx3191d.ko 2024-05-22T20:59:39.465 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/esas2r/ 2024-05-22T20:59:39.465 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 267577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-22T20:59:39.465 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 78969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/esp_scsi.ko 2024-05-22T20:59:39.465 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fcoe/ 2024-05-22T20:59:39.465 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 95329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-22T20:59:39.465 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 194017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-22T20:59:39.466 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fdomain.ko 2024-05-22T20:59:39.466 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fnic/ 2024-05-22T20:59:39.466 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 366529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fnic/fnic.ko 2024-05-22T20:59:39.466 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 251505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/hpsa.ko 2024-05-22T20:59:39.466 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 65761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/hptiop.ko 2024-05-22T20:59:39.466 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 57097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/hv_storvsc.ko 2024-05-22T20:59:39.466 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/imm.ko 2024-05-22T20:59:39.466 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 55153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/initio.ko 2024-05-22T20:59:39.466 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 327841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/ipr.ko 2024-05-22T20:59:39.466 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 129785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/ips.ko 2024-05-22T20:59:39.466 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/isci/ 2024-05-22T20:59:39.466 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 397785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/isci/isci.ko 2024-05-22T20:59:39.467 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-22T20:59:39.467 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 63121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-22T20:59:39.467 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libfc/ 2024-05-22T20:59:39.467 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 386849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libfc/libfc.ko 2024-05-22T20:59:39.467 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 178177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libiscsi.ko 2024-05-22T20:59:39.467 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 64809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-22T20:59:39.467 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libsas/ 2024-05-22T20:59:39.467 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 274009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libsas/libsas.ko 2024-05-22T20:59:39.467 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/lpfc/ 2024-05-22T20:59:39.467 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 2736001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-22T20:59:39.467 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/megaraid/ 2024-05-22T20:59:39.467 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 88529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-22T20:59:39.467 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-22T20:59:39.467 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 389177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-22T20:59:39.468 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 104769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/megaraid.ko 2024-05-22T20:59:39.468 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/mpt3sas/ 2024-05-22T20:59:39.468 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 836945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-22T20:59:39.468 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/mvsas/ 2024-05-22T20:59:39.468 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 158729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-22T20:59:39.468 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 96537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/mvumi.ko 2024-05-22T20:59:39.468 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pcmcia/ 2024-05-22T20:59:39.468 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 101625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-05-22T20:59:39.468 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-05-22T20:59:39.468 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-05-22T20:59:39.468 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-05-22T20:59:39.468 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pm8001/ 2024-05-22T20:59:39.468 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 443745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-22T20:59:39.469 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 152033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pmcraid.ko 2024-05-22T20:59:39.469 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 41625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/ppa.ko 2024-05-22T20:59:39.469 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 66977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla1280.ko 2024-05-22T20:59:39.469 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla2xxx/ 2024-05-22T20:59:39.469 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 2324809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-22T20:59:39.469 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 103601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-22T20:59:39.469 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla4xxx/ 2024-05-22T20:59:39.469 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 697337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-22T20:59:39.469 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qlogicfas408.ko 2024-05-22T20:59:39.469 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/raid_class.ko 2024-05-22T20:59:39.469 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 322865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_debug.ko 2024-05-22T20:59:39.469 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 219857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-22T20:59:39.469 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 357153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-22T20:59:39.470 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 121185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-22T20:59:39.470 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 93481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-22T20:59:39.470 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 55401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-22T20:59:39.470 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/ses.ko 2024-05-22T20:59:39.470 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 138121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/st.ko 2024-05-22T20:59:39.470 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 57401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/stex.ko 2024-05-22T20:59:39.470 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/sym53c8xx_2/ 2024-05-22T20:59:39.470 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 205025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-22T20:59:39.470 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 55425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/virtio_scsi.ko 2024-05-22T20:59:39.470 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 59185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-22T20:59:39.470 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 48777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/wd719x.ko 2024-05-22T20:59:39.470 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 59417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/xen-scsifront.ko 2024-05-22T20:59:39.471 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ssb/ 2024-05-22T20:59:39.471 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 174913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ssb/ssb.ko 2024-05-22T20:59:39.471 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/ 2024-05-22T20:59:39.471 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/iscsi/ 2024-05-22T20:59:39.471 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 920169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-22T20:59:39.471 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/loopback/ 2024-05-22T20:59:39.471 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 60321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-22T20:59:39.471 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 48129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/target_core_file.ko 2024-05-22T20:59:39.471 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 54233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/target_core_iblock.ko 2024-05-22T20:59:39.471 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1170033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/target_core_mod.ko 2024-05-22T20:59:39.471 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 51897 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/target_core_pscsi.ko 2024-05-22T20:59:39.471 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 149721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/target_core_user.ko 2024-05-22T20:59:39.471 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/tcm_fc/ 2024-05-22T20:59:39.471 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 90057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-22T20:59:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/ 2024-05-22T20:59:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/ 2024-05-22T20:59:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/ 2024-05-22T20:59:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-22T20:59:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-22T20:59:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-22T20:59:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-22T20:59:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-22T20:59:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-22T20:59:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-22T20:59:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-22T20:59:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-22T20:59:39.472 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12929 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-22T20:59:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-22T20:59:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-22T20:59:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 46065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-22T20:59:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-22T20:59:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-22T20:59:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-05-22T20:59:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-22T20:59:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-22T20:59:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-22T20:59:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/ 2024-05-22T20:59:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/ipwireless/ 2024-05-22T20:59:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 99313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-22T20:59:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 69729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/moxa.ko 2024-05-22T20:59:39.474 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 61649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/mxser.ko 2024-05-22T20:59:39.474 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 134401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/n_gsm.ko 2024-05-22T20:59:39.474 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 37905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/n_hdlc.ko 2024-05-22T20:59:39.474 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 78969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/nozomi.ko 2024-05-22T20:59:39.474 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/ 2024-05-22T20:59:39.474 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/8250/ 2024-05-22T20:59:39.474 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-05-22T20:59:39.474 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 65121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-22T20:59:39.474 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-05-22T20:59:39.474 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/altera_uart.ko 2024-05-22T20:59:39.474 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/arc_uart.ko 2024-05-22T20:59:39.475 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 91665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-05-22T20:59:39.475 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/jsm/ 2024-05-22T20:59:39.475 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 120577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-22T20:59:39.475 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/rp2.ko 2024-05-22T20:59:39.475 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 65641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/sc16is7xx.ko 2024-05-22T20:59:39.475 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs/ 2024-05-22T20:59:39.475 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs/core/ 2024-05-22T20:59:39.475 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 693081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-22T20:59:39.475 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs/host/ 2024-05-22T20:59:39.475 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-22T20:59:39.475 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-05-22T20:59:39.476 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/ 2024-05-22T20:59:39.476 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 55617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio.ko 2024-05-22T20:59:39.476 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_aec.ko 2024-05-22T20:59:39.476 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_cif.ko 2024-05-22T20:59:39.476 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_dmem_genirq.ko 2024-05-22T20:59:39.476 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_mf624.ko 2024-05-22T20:59:39.476 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_netx.ko 2024-05-22T20:59:39.476 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_pci_generic.ko 2024-05-22T20:59:39.476 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-05-22T20:59:39.476 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_sercos3.ko 2024-05-22T20:59:39.476 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/ 2024-05-22T20:59:39.477 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/c67x00/ 2024-05-22T20:59:39.477 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 90321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/c67x00/c67x00.ko 2024-05-22T20:59:39.477 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/chipidea/ 2024-05-22T20:59:39.477 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 189209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-05-22T20:59:39.477 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-05-22T20:59:39.477 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-05-22T20:59:39.477 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-05-22T20:59:39.477 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-05-22T20:59:39.477 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/class/ 2024-05-22T20:59:39.477 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 101545 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/class/cdc-acm.ko 2024-05-22T20:59:39.477 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 63073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-22T20:59:39.477 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 56721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/class/usblp.ko 2024-05-22T20:59:39.477 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 107417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/class/usbtmc.ko 2024-05-22T20:59:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/common/ 2024-05-22T20:59:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/common/ulpi.ko 2024-05-22T20:59:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/dwc2/ 2024-05-22T20:59:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-05-22T20:59:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/dwc3/ 2024-05-22T20:59:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-05-22T20:59:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-05-22T20:59:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 574593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/dwc3/dwc3.ko 2024-05-22T20:59:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/ 2024-05-22T20:59:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/ 2024-05-22T20:59:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 72249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/u_ether.ko 2024-05-22T20:59:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 61953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/u_serial.ko 2024-05-22T20:59:39.478 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-05-22T20:59:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 49737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-05-22T20:59:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-05-22T20:59:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-05-22T20:59:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 131345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-05-22T20:59:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 62321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-05-22T20:59:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 146617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-05-22T20:59:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 70689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-05-22T20:59:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-05-22T20:59:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 37649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-05-22T20:59:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 70017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-05-22T20:59:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 101249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-05-22T20:59:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-05-22T20:59:39.479 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 71257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-05-22T20:59:39.480 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 84937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-05-22T20:59:39.480 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/ 2024-05-22T20:59:39.480 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-05-22T20:59:39.480 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-05-22T20:59:39.480 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-05-22T20:59:39.480 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-05-22T20:59:39.480 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-05-22T20:59:39.480 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-05-22T20:59:39.480 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-05-22T20:59:39.480 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-05-22T20:59:39.480 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-05-22T20:59:39.480 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-05-22T20:59:39.480 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-05-22T20:59:39.480 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-05-22T20:59:39.481 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 78073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-05-22T20:59:39.481 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-05-22T20:59:39.481 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 204049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/libcomposite.ko 2024-05-22T20:59:39.481 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/ 2024-05-22T20:59:39.481 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-05-22T20:59:39.481 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/bdc/ 2024-05-22T20:59:39.481 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 182561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-05-22T20:59:39.481 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 45545 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-05-22T20:59:39.481 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 66513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-05-22T20:59:39.481 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 85217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-05-22T20:59:39.481 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 69313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-05-22T20:59:39.481 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 90313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/net2272.ko 2024-05-22T20:59:39.481 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 90361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/net2280.ko 2024-05-22T20:59:39.482 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 94841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-05-22T20:59:39.482 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 80521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-05-22T20:59:39.482 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 87465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-05-22T20:59:39.482 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 95441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-05-22T20:59:39.482 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 195969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-05-22T20:59:39.482 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/ 2024-05-22T20:59:39.482 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/bcma-hcd.ko 2024-05-22T20:59:39.482 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 108321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/isp116x-hcd.ko 2024-05-22T20:59:39.482 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 104377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-05-22T20:59:39.482 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 102273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-05-22T20:59:39.482 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 60105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-22T20:59:39.482 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/sl811_cs.ko 2024-05-22T20:59:39.482 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/ssb-hcd.ko 2024-05-22T20:59:39.483 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-05-22T20:59:39.483 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/image/ 2024-05-22T20:59:39.483 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/image/mdc800.ko 2024-05-22T20:59:39.483 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/image/microtek.ko 2024-05-22T20:59:39.483 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/ 2024-05-22T20:59:39.483 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 47353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/adutux.ko 2024-05-22T20:59:39.483 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/appledisplay.ko 2024-05-22T20:59:39.483 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-05-22T20:59:39.483 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/cytherm.ko 2024-05-22T20:59:39.483 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/ehset.ko 2024-05-22T20:59:39.483 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/emi26.ko 2024-05-22T20:59:39.483 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/emi62.ko 2024-05-22T20:59:39.483 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/ezusb.ko 2024-05-22T20:59:39.484 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/idmouse.ko 2024-05-22T20:59:39.484 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 42729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/iowarrior.ko 2024-05-22T20:59:39.484 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-22T20:59:39.484 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 42241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/ldusb.ko 2024-05-22T20:59:39.484 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 42233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/legousbtower.ko 2024-05-22T20:59:39.484 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/lvstest.ko 2024-05-22T20:59:39.484 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/sisusbvga/ 2024-05-22T20:59:39.484 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 85313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-22T20:59:39.484 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-22T20:59:39.484 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/usb3503.ko 2024-05-22T20:59:39.484 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/usblcd.ko 2024-05-22T20:59:39.484 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-22T20:59:39.484 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 84265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/usbtest.ko 2024-05-22T20:59:39.485 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 52065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/uss720.ko 2024-05-22T20:59:39.485 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/yurex.ko 2024-05-22T20:59:39.485 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/mon/ 2024-05-22T20:59:39.485 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 91529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/mon/usbmon.ko 2024-05-22T20:59:39.485 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/musb/ 2024-05-22T20:59:39.485 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 396209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/musb/musb_hdrc.ko 2024-05-22T20:59:39.485 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/phy/ 2024-05-22T20:59:39.485 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/phy/phy-generic.ko 2024-05-22T20:59:39.485 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-05-22T20:59:39.485 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/phy/phy-isp1301.ko 2024-05-22T20:59:39.485 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/phy/phy-tahvo.ko 2024-05-22T20:59:39.485 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ 2024-05-22T20:59:39.485 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/aircable.ko 2024-05-22T20:59:39.486 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 38209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ark3116.ko 2024-05-22T20:59:39.486 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-22T20:59:39.486 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ch341.ko 2024-05-22T20:59:39.486 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 79681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/cp210x.ko 2024-05-22T20:59:39.486 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/cyberjack.ko 2024-05-22T20:59:39.486 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 66873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-22T20:59:39.486 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 58329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-22T20:59:39.486 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/empeg.ko 2024-05-22T20:59:39.486 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 53009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/f81232.ko 2024-05-22T20:59:39.486 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 188609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-22T20:59:39.486 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 61489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-22T20:59:39.486 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 152305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-22T20:59:39.486 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 142833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/io_ti.ko 2024-05-22T20:59:39.487 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 69217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ipaq.ko 2024-05-22T20:59:39.487 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ipw.ko 2024-05-22T20:59:39.487 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ir-usb.ko 2024-05-22T20:59:39.487 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 66361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-22T20:59:39.487 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 122329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/keyspan.ko 2024-05-22T20:59:39.487 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 41297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-22T20:59:39.487 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-22T20:59:39.487 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 38049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-22T20:59:39.487 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 38041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/mct_u232.ko 2024-05-22T20:59:39.487 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/metro-usb.ko 2024-05-22T20:59:39.487 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 79681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/mos7720.ko 2024-05-22T20:59:39.487 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 89969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/mos7840.ko 2024-05-22T20:59:39.487 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 65073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/mxuport.ko 2024-05-22T20:59:39.488 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/navman.ko 2024-05-22T20:59:39.488 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/omninet.ko 2024-05-22T20:59:39.488 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/opticon.ko 2024-05-22T20:59:39.488 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 193857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/option.ko 2024-05-22T20:59:39.488 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 45337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/oti6858.ko 2024-05-22T20:59:39.488 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 61761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/pl2303.ko 2024-05-22T20:59:39.488 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/qcaux.ko 2024-05-22T20:59:39.488 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 39985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/qcserial.ko 2024-05-22T20:59:39.488 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/quatech2.ko 2024-05-22T20:59:39.488 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/safe_serial.ko 2024-05-22T20:59:39.488 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 59345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/sierra.ko 2024-05-22T20:59:39.488 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-22T20:59:39.488 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ssu100.ko 2024-05-22T20:59:39.488 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/symbolserial.ko 2024-05-22T20:59:39.489 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 70497 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-22T20:59:39.489 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-22T20:59:39.489 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/usb_debug.ko 2024-05-22T20:59:39.489 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 39321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-22T20:59:39.489 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 151137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/usbserial.ko 2024-05-22T20:59:39.489 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/visor.ko 2024-05-22T20:59:39.489 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/whiteheat.ko 2024-05-22T20:59:39.489 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/wishbone-serial.ko 2024-05-22T20:59:39.489 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-22T20:59:39.489 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ 2024-05-22T20:59:39.489 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 64049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/uas.ko 2024-05-22T20:59:39.489 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-22T20:59:39.489 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-22T20:59:39.490 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-22T20:59:39.490 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 41073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-22T20:59:39.490 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-22T20:59:39.490 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-22T20:59:39.490 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-22T20:59:39.490 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-karma.ko 2024-05-22T20:59:39.490 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-22T20:59:39.490 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-22T20:59:39.490 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-22T20:59:39.490 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-22T20:59:39.490 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 41785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-22T20:59:39.490 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 201193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/usb-storage.ko 2024-05-22T20:59:39.490 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/usbip/ 2024-05-22T20:59:39.491 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 76041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-22T20:59:39.491 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 85257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-22T20:59:39.491 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 122321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-22T20:59:39.491 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/ 2024-05-22T20:59:39.491 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/pci/ 2024-05-22T20:59:39.491 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 206921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-22T20:59:39.491 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-22T20:59:39.491 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 134265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/vfio.ko 2024-05-22T20:59:39.491 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 93673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-22T20:59:39.491 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vhost/ 2024-05-22T20:59:39.491 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 138721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vhost/vhost.ko 2024-05-22T20:59:39.491 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-22T20:59:39.491 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 71281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vhost/vhost_net.ko 2024-05-22T20:59:39.492 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 107905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vhost/vhost_scsi.ko 2024-05-22T20:59:39.492 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/ 2024-05-22T20:59:39.492 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/ 2024-05-22T20:59:39.492 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-05-22T20:59:39.492 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/aat2870_bl.ko 2024-05-22T20:59:39.492 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/adp5520_bl.ko 2024-05-22T20:59:39.492 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 41169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/adp8860_bl.ko 2024-05-22T20:59:39.492 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 51745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/adp8870_bl.ko 2024-05-22T20:59:39.492 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/apple_bl.ko 2024-05-22T20:59:39.492 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/as3711_bl.ko 2024-05-22T20:59:39.492 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/bd6107.ko 2024-05-22T20:59:39.492 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/da903x_bl.ko 2024-05-22T20:59:39.492 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/da9052_bl.ko 2024-05-22T20:59:39.493 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/gpio_backlight.ko 2024-05-22T20:59:39.493 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/kb3886_bl.ko 2024-05-22T20:59:39.493 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/lcd.ko 2024-05-22T20:59:39.493 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/lm3533_bl.ko 2024-05-22T20:59:39.493 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/lm3639_bl.ko 2024-05-22T20:59:39.493 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/lv5207lp.ko 2024-05-22T20:59:39.493 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/max8925_bl.ko 2024-05-22T20:59:39.493 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/pandora_bl.ko 2024-05-22T20:59:39.493 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-05-22T20:59:39.493 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-22T20:59:39.493 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/wm831x_bl.ko 2024-05-22T20:59:39.493 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/ 2024-05-22T20:59:39.493 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/masters/ 2024-05-22T20:59:39.493 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/masters/ds2482.ko 2024-05-22T20:59:39.494 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 46801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/masters/ds2490.ko 2024-05-22T20:59:39.494 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/masters/matrox_w1.ko 2024-05-22T20:59:39.494 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/masters/w1-gpio.ko 2024-05-22T20:59:39.494 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/ 2024-05-22T20:59:39.494 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-22T20:59:39.494 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-22T20:59:39.494 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-22T20:59:39.494 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-22T20:59:39.494 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-22T20:59:39.494 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-22T20:59:39.495 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-22T20:59:39.495 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-22T20:59:39.495 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-22T20:59:39.495 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-22T20:59:39.495 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 70369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-22T20:59:39.495 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 129433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/wire.ko 2024-05-22T20:59:39.495 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/ 2024-05-22T20:59:39.495 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/acquirewdt.ko 2024-05-22T20:59:39.495 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/advantechwdt.ko 2024-05-22T20:59:39.495 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-22T20:59:39.495 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-22T20:59:39.495 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/cpu5wdt.ko 2024-05-22T20:59:39.495 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/da9052_wdt.ko 2024-05-22T20:59:39.496 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/da9055_wdt.ko 2024-05-22T20:59:39.496 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/da9063_wdt.ko 2024-05-22T20:59:39.496 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/dw_wdt.ko 2024-05-22T20:59:39.496 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/eurotechwdt.ko 2024-05-22T20:59:39.496 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-22T20:59:39.496 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/hpwdt.ko 2024-05-22T20:59:39.496 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/i6300esb.ko 2024-05-22T20:59:39.496 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-22T20:59:39.496 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-22T20:59:39.496 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/ib700wdt.ko 2024-05-22T20:59:39.496 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/ibmasr.ko 2024-05-22T20:59:39.496 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-22T20:59:39.496 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-22T20:59:39.497 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/it87_wdt.ko 2024-05-22T20:59:39.497 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/kempld_wdt.ko 2024-05-22T20:59:39.497 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/machzwd.ko 2024-05-22T20:59:39.497 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/mena21_wdt.ko 2024-05-22T20:59:39.497 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-05-22T20:59:39.497 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/nv_tco.ko 2024-05-22T20:59:39.497 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-05-22T20:59:39.497 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/pc87413_wdt.ko 2024-05-22T20:59:39.497 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 46953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-22T20:59:39.497 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 37193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-22T20:59:39.497 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/retu_wdt.ko 2024-05-22T20:59:39.497 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-05-22T20:59:39.497 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-05-22T20:59:39.498 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-22T20:59:39.498 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/sc1200wdt.ko 2024-05-22T20:59:39.498 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-22T20:59:39.498 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-05-22T20:59:39.498 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/softdog.ko 2024-05-22T20:59:39.498 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-22T20:59:39.498 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/twl4030_wdt.ko 2024-05-22T20:59:39.498 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/via_wdt.ko 2024-05-22T20:59:39.498 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-22T20:59:39.498 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-22T20:59:39.498 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-22T20:59:39.498 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/wafer5823wdt.ko 2024-05-22T20:59:39.499 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/wdt_pci.ko 2024-05-22T20:59:39.499 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/wm831x_wdt.ko 2024-05-22T20:59:39.499 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/wm8350_wdt.ko 2024-05-22T20:59:39.499 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/xen_wdt.ko 2024-05-22T20:59:39.499 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/ 2024-05-22T20:59:39.499 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-evtchn.ko 2024-05-22T20:59:39.499 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-gntalloc.ko 2024-05-22T20:59:39.499 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 55713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-gntdev.ko 2024-05-22T20:59:39.499 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-pciback/ 2024-05-22T20:59:39.499 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 213913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-22T20:59:39.499 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 47689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-privcmd.ko 2024-05-22T20:59:39.499 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 85713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-scsiback.ko 2024-05-22T20:59:39.499 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xenfs/ 2024-05-22T20:59:39.500 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-22T20:59:39.500 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ 2024-05-22T20:59:39.500 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/9p/ 2024-05-22T20:59:39.500 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 192617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/9p/9p.ko 2024-05-22T20:59:39.500 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/adfs/ 2024-05-22T20:59:39.500 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 78153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/adfs/adfs.ko 2024-05-22T20:59:39.500 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/affs/ 2024-05-22T20:59:39.500 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 197849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/affs/affs.ko 2024-05-22T20:59:39.500 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/afs/ 2024-05-22T20:59:39.500 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1261113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/afs/kafs.ko 2024-05-22T20:59:39.500 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/autofs/ 2024-05-22T20:59:39.500 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 127377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/autofs/autofs4.ko 2024-05-22T20:59:39.500 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/befs/ 2024-05-22T20:59:39.501 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 80521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/befs/befs.ko 2024-05-22T20:59:39.501 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/bfs/ 2024-05-22T20:59:39.501 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 50473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/bfs/bfs.ko 2024-05-22T20:59:39.501 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 45561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/binfmt_misc.ko 2024-05-22T20:59:39.501 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/btrfs/ 2024-05-22T20:59:39.501 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 4489593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/btrfs/btrfs.ko 2024-05-22T20:59:39.501 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/cachefiles/ 2024-05-22T20:59:39.501 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 376857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/cachefiles/cachefiles.ko 2024-05-22T20:59:39.501 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ceph/ 2024-05-22T20:59:39.501 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1587233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ceph/ceph.ko 2024-05-22T20:59:39.501 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/coda/ 2024-05-22T20:59:39.501 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 130073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/coda/coda.ko 2024-05-22T20:59:39.501 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/configfs/ 2024-05-22T20:59:39.502 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 139081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/configfs/configfs.ko 2024-05-22T20:59:39.502 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/cramfs/ 2024-05-22T20:59:39.502 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/cramfs/cramfs.ko 2024-05-22T20:59:39.502 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/dlm/ 2024-05-22T20:59:39.502 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 813225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/dlm/dlm.ko 2024-05-22T20:59:39.502 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/efs/ 2024-05-22T20:59:39.502 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 46409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/efs/efs.ko 2024-05-22T20:59:39.502 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/f2fs/ 2024-05-22T20:59:39.502 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1906817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/f2fs/f2fs.ko 2024-05-22T20:59:39.502 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/fat/ 2024-05-22T20:59:39.502 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/fat/msdos.ko 2024-05-22T20:59:39.502 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/freevxfs/ 2024-05-22T20:59:39.502 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 52313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/freevxfs/freevxfs.ko 2024-05-22T20:59:39.503 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/fuse/ 2024-05-22T20:59:39.503 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/fuse/cuse.ko 2024-05-22T20:59:39.503 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/gfs2/ 2024-05-22T20:59:39.503 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1486521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/gfs2/gfs2.ko 2024-05-22T20:59:39.503 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/hfs/ 2024-05-22T20:59:39.503 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 183305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/hfs/hfs.ko 2024-05-22T20:59:39.504 INFO:teuthology.orchestra.run.smithi179.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-22T20:59:39.504 INFO:teuthology.orchestra.run.smithi179.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-22T20:59:39.504 INFO:teuthology.orchestra.run.smithi179.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-22T20:59:39.504 INFO:teuthology.orchestra.run.smithi179.stderr:done 2024-05-22T20:59:39.550 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/hfsplus/ 2024-05-22T20:59:39.550 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 282097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/hfsplus/hfsplus.ko 2024-05-22T20:59:39.550 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/hpfs/ 2024-05-22T20:59:39.550 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 249009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/hpfs/hpfs.ko 2024-05-22T20:59:39.550 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/isofs/ 2024-05-22T20:59:39.550 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 93033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/isofs/isofs.ko 2024-05-22T20:59:39.551 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/jfs/ 2024-05-22T20:59:39.551 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 533073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/jfs/jfs.ko 2024-05-22T20:59:39.551 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/lockd/ 2024-05-22T20:59:39.551 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 361369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/lockd/lockd.ko 2024-05-22T20:59:39.551 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/minix/ 2024-05-22T20:59:39.551 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 112609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/minix/minix.ko 2024-05-22T20:59:39.551 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/netfs/ 2024-05-22T20:59:39.551 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 591585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/netfs/netfs.ko 2024-05-22T20:59:39.551 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/ 2024-05-22T20:59:39.551 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/blocklayout/ 2024-05-22T20:59:39.551 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 102561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-22T20:59:39.551 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/filelayout/ 2024-05-22T20:59:39.551 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 74353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-22T20:59:39.552 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/flexfilelayout/ 2024-05-22T20:59:39.552 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 147689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-22T20:59:39.552 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1320249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/nfs.ko 2024-05-22T20:59:39.552 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 82105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/nfsv2.ko 2024-05-22T20:59:39.552 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 151641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/nfsv3.ko 2024-05-22T20:59:39.552 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 2518505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/nfsv4.ko 2024-05-22T20:59:39.552 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs_common/ 2024-05-22T20:59:39.552 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs_common/grace.ko 2024-05-22T20:59:39.552 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs_common/nfs_acl.ko 2024-05-22T20:59:39.552 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfsd/ 2024-05-22T20:59:39.552 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 2024049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfsd/nfsd.ko 2024-05-22T20:59:39.552 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nilfs2/ 2024-05-22T20:59:39.552 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 734025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nilfs2/nilfs2.ko 2024-05-22T20:59:39.552 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/ 2024-05-22T20:59:39.553 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-celtic.ko 2024-05-22T20:59:39.553 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-centeuro.ko 2024-05-22T20:59:39.553 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-croatian.ko 2024-05-22T20:59:39.553 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-cyrillic.ko 2024-05-22T20:59:39.553 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-gaelic.ko 2024-05-22T20:59:39.553 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-greek.ko 2024-05-22T20:59:39.553 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-iceland.ko 2024-05-22T20:59:39.553 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-inuit.ko 2024-05-22T20:59:39.553 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-roman.ko 2024-05-22T20:59:39.553 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-romanian.ko 2024-05-22T20:59:39.553 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-turkish.ko 2024-05-22T20:59:39.553 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_ascii.ko 2024-05-22T20:59:39.553 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp1250.ko 2024-05-22T20:59:39.554 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp1251.ko 2024-05-22T20:59:39.554 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp1255.ko 2024-05-22T20:59:39.554 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp737.ko 2024-05-22T20:59:39.554 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp775.ko 2024-05-22T20:59:39.554 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp850.ko 2024-05-22T20:59:39.554 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp852.ko 2024-05-22T20:59:39.554 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp855.ko 2024-05-22T20:59:39.554 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp857.ko 2024-05-22T20:59:39.554 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp860.ko 2024-05-22T20:59:39.554 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp861.ko 2024-05-22T20:59:39.554 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp862.ko 2024-05-22T20:59:39.554 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp863.ko 2024-05-22T20:59:39.554 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp864.ko 2024-05-22T20:59:39.555 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp865.ko 2024-05-22T20:59:39.555 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp866.ko 2024-05-22T20:59:39.555 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp869.ko 2024-05-22T20:59:39.555 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp874.ko 2024-05-22T20:59:39.555 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 95625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp932.ko 2024-05-22T20:59:39.555 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 144209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp936.ko 2024-05-22T20:59:39.555 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 167137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp949.ko 2024-05-22T20:59:39.555 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 119513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp950.ko 2024-05-22T20:59:39.555 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_euc-jp.ko 2024-05-22T20:59:39.555 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-1.ko 2024-05-22T20:59:39.555 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-13.ko 2024-05-22T20:59:39.555 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-14.ko 2024-05-22T20:59:39.555 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-15.ko 2024-05-22T20:59:39.556 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-2.ko 2024-05-22T20:59:39.556 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-3.ko 2024-05-22T20:59:39.556 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-4.ko 2024-05-22T20:59:39.556 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-5.ko 2024-05-22T20:59:39.556 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-6.ko 2024-05-22T20:59:39.556 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-7.ko 2024-05-22T20:59:39.556 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-9.ko 2024-05-22T20:59:39.556 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_koi8-r.ko 2024-05-22T20:59:39.556 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_koi8-ru.ko 2024-05-22T20:59:39.556 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_koi8-u.ko 2024-05-22T20:59:39.556 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 7521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-22T20:59:39.556 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_utf8.ko 2024-05-22T20:59:39.556 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/ 2024-05-22T20:59:39.557 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/cluster/ 2024-05-22T20:59:39.557 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 302073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-22T20:59:39.557 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/dlm/ 2024-05-22T20:59:39.557 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 548169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-22T20:59:39.557 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/dlmfs/ 2024-05-22T20:59:39.557 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 66361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-22T20:59:39.557 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 3245617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/ocfs2.ko 2024-05-22T20:59:39.557 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 22113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-22T20:59:39.557 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 39313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-22T20:59:39.557 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-22T20:59:39.557 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/omfs/ 2024-05-22T20:59:39.557 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 67489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/omfs/omfs.ko 2024-05-22T20:59:39.557 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/overlayfs/ 2024-05-22T20:59:39.558 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 462873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/overlayfs/overlay.ko 2024-05-22T20:59:39.558 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/pstore/ 2024-05-22T20:59:39.558 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 67977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/pstore/ramoops.ko 2024-05-22T20:59:39.558 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/qnx4/ 2024-05-22T20:59:39.558 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/qnx4/qnx4.ko 2024-05-22T20:59:39.558 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/qnx6/ 2024-05-22T20:59:39.558 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 56097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/qnx6/qnx6.ko 2024-05-22T20:59:39.558 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/quota/ 2024-05-22T20:59:39.558 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/quota/quota_tree.ko 2024-05-22T20:59:39.558 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/quota/quota_v1.ko 2024-05-22T20:59:39.558 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/quota/quota_v2.ko 2024-05-22T20:59:39.558 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/reiserfs/ 2024-05-22T20:59:39.558 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 684841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/reiserfs/reiserfs.ko 2024-05-22T20:59:39.559 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/romfs/ 2024-05-22T20:59:39.559 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/romfs/romfs.ko 2024-05-22T20:59:39.559 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/ 2024-05-22T20:59:39.559 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/client/ 2024-05-22T20:59:39.559 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 3319041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/client/cifs.ko 2024-05-22T20:59:39.559 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/common/ 2024-05-22T20:59:39.559 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/common/cifs_arc4.ko 2024-05-22T20:59:39.559 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/common/cifs_md4.ko 2024-05-22T20:59:39.559 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/squashfs/ 2024-05-22T20:59:39.559 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 164185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/squashfs/squashfs.ko 2024-05-22T20:59:39.559 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/sysv/ 2024-05-22T20:59:39.559 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 115977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/sysv/sysv.ko 2024-05-22T20:59:39.559 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/udf/ 2024-05-22T20:59:39.560 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 310017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/udf/udf.ko 2024-05-22T20:59:39.560 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ufs/ 2024-05-22T20:59:39.560 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 198233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ufs/ufs.ko 2024-05-22T20:59:39.560 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/xfs/ 2024-05-22T20:59:39.560 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 5192705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/xfs/xfs.ko 2024-05-22T20:59:39.560 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/ 2024-05-22T20:59:39.560 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/locking/ 2024-05-22T20:59:39.560 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 97625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/locking/locktorture.ko 2024-05-22T20:59:39.560 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/time/ 2024-05-22T20:59:39.560 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/time/test_udelay.ko 2024-05-22T20:59:39.560 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 72665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/torture.ko 2024-05-22T20:59:39.560 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/ 2024-05-22T20:59:39.561 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 7313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crc-itu-t.ko 2024-05-22T20:59:39.561 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 7081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crc7.ko 2024-05-22T20:59:39.561 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 7953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crc8.ko 2024-05-22T20:59:39.561 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crypto/ 2024-05-22T20:59:39.561 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crypto/libarc4.ko 2024-05-22T20:59:39.561 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 38985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crypto/libdes.ko 2024-05-22T20:59:39.561 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/interval_tree_test.ko 2024-05-22T20:59:39.561 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/libcrc32c.ko 2024-05-22T20:59:39.561 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/lru_cache.ko 2024-05-22T20:59:39.561 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/lz4/ 2024-05-22T20:59:39.561 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/lz4/lz4_compress.ko 2024-05-22T20:59:39.561 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/lz4/lz4hc_compress.ko 2024-05-22T20:59:39.561 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/math/ 2024-05-22T20:59:39.562 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 7281 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/math/cordic.ko 2024-05-22T20:59:39.562 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/memory-notifier-error-inject.ko 2024-05-22T20:59:39.562 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/notifier-error-inject.ko 2024-05-22T20:59:39.562 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/percpu_test.ko 2024-05-22T20:59:39.562 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/pm-notifier-error-inject.ko 2024-05-22T20:59:39.562 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/raid6/ 2024-05-22T20:59:39.562 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 152025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/raid6/raid6_pq.ko 2024-05-22T20:59:39.562 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/rbtree_test.ko 2024-05-22T20:59:39.562 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/reed_solomon/ 2024-05-22T20:59:39.562 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-22T20:59:39.562 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/test-kstrtox.ko 2024-05-22T20:59:39.562 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 4912441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/test_bpf.ko 2024-05-22T20:59:39.563 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 61169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/test_firmware.ko 2024-05-22T20:59:39.563 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 7121 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/test_module.ko 2024-05-22T20:59:39.563 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/test_user_copy.ko 2024-05-22T20:59:39.563 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/ts_bm.ko 2024-05-22T20:59:39.563 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/ts_fsm.ko 2024-05-22T20:59:39.563 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/ts_kmp.ko 2024-05-22T20:59:39.563 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/xz/ 2024-05-22T20:59:39.563 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/xz/xz_dec_test.ko 2024-05-22T20:59:39.563 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/mm/ 2024-05-22T20:59:39.563 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/mm/hwpoison-inject.ko 2024-05-22T20:59:39.563 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ 2024-05-22T20:59:39.563 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/ 2024-05-22T20:59:39.563 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 66969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/6lowpan.ko 2024-05-22T20:59:39.564 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 7577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_dest.ko 2024-05-22T20:59:39.564 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 7529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_fragment.ko 2024-05-22T20:59:39.564 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 7513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_hop.ko 2024-05-22T20:59:39.564 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 7513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-22T20:59:39.564 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 7529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_mobility.ko 2024-05-22T20:59:39.564 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 7521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_routing.ko 2024-05-22T20:59:39.564 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_udp.ko 2024-05-22T20:59:39.564 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/ 2024-05-22T20:59:39.564 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/garp.ko 2024-05-22T20:59:39.564 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 42297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/mrp.ko 2024-05-22T20:59:39.564 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/p8022.ko 2024-05-22T20:59:39.564 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/psnap.ko 2024-05-22T20:59:39.564 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13537 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/stp.ko 2024-05-22T20:59:39.565 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/8021q/ 2024-05-22T20:59:39.565 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 106793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/8021q/8021q.ko 2024-05-22T20:59:39.565 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/9p/ 2024-05-22T20:59:39.565 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 236793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/9p/9pnet.ko 2024-05-22T20:59:39.565 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/9p/9pnet_fd.ko 2024-05-22T20:59:39.565 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/9p/9pnet_rdma.ko 2024-05-22T20:59:39.565 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 39513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/9p/9pnet_virtio.ko 2024-05-22T20:59:39.565 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/batman-adv/ 2024-05-22T20:59:39.565 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 879697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/batman-adv/batman-adv.ko 2024-05-22T20:59:39.565 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/ 2024-05-22T20:59:39.565 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 84761 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/br_netfilter.ko 2024-05-22T20:59:39.565 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1030873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/bridge.ko 2024-05-22T20:59:39.565 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ 2024-05-22T20:59:39.566 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-22T20:59:39.566 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-22T20:59:39.566 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-22T20:59:39.566 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-22T20:59:39.566 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-22T20:59:39.566 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-22T20:59:39.566 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-22T20:59:39.566 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-22T20:59:39.566 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-22T20:59:39.566 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-22T20:59:39.566 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-22T20:59:39.566 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9569 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-22T20:59:39.566 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-22T20:59:39.567 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-22T20:59:39.567 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-22T20:59:39.567 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-22T20:59:39.567 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-22T20:59:39.567 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-22T20:59:39.567 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-22T20:59:39.567 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-22T20:59:39.567 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 70529 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebtables.ko 2024-05-22T20:59:39.567 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-22T20:59:39.567 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-22T20:59:39.567 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ceph/ 2024-05-22T20:59:39.567 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1243505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ceph/libceph.ko 2024-05-22T20:59:39.567 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/core/ 2024-05-22T20:59:39.568 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 153801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/core/pktgen.ko 2024-05-22T20:59:39.568 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dccp/ 2024-05-22T20:59:39.568 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 244889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dccp/dccp.ko 2024-05-22T20:59:39.568 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dccp/dccp_diag.ko 2024-05-22T20:59:39.568 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 71033 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dccp/dccp_ipv4.ko 2024-05-22T20:59:39.568 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 74993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dccp/dccp_ipv6.ko 2024-05-22T20:59:39.568 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/ 2024-05-22T20:59:39.568 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 507705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/dsa_core.ko 2024-05-22T20:59:39.568 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/tag_brcm.ko 2024-05-22T20:59:39.568 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/tag_dsa.ko 2024-05-22T20:59:39.568 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/tag_none.ko 2024-05-22T20:59:39.568 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/tag_trailer.ko 2024-05-22T20:59:39.569 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/hsr/ 2024-05-22T20:59:39.569 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 128873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/hsr/hsr.ko 2024-05-22T20:59:39.569 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ieee802154/ 2024-05-22T20:59:39.569 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ieee802154/6lowpan/ 2024-05-22T20:59:39.569 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 55505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-22T20:59:39.569 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 357425 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ieee802154/ieee802154.ko 2024-05-22T20:59:39.569 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 59393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-22T20:59:39.569 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ 2024-05-22T20:59:39.569 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ah4.ko 2024-05-22T20:59:39.569 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 46417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/esp4.ko 2024-05-22T20:59:39.569 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 60233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/fou.ko 2024-05-22T20:59:39.569 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/gre.ko 2024-05-22T20:59:39.569 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 65665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/inet_diag.ko 2024-05-22T20:59:39.569 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 75489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ip_gre.ko 2024-05-22T20:59:39.570 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 77721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ip_tunnel.ko 2024-05-22T20:59:39.570 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ip_vti.ko 2024-05-22T20:59:39.570 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ipcomp.ko 2024-05-22T20:59:39.570 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ipip.ko 2024-05-22T20:59:39.570 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ 2024-05-22T20:59:39.570 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 45521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-22T20:59:39.570 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-22T20:59:39.570 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-22T20:59:39.570 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 50169 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ip_tables.ko 2024-05-22T20:59:39.570 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-22T20:59:39.570 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-05-22T20:59:39.570 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-22T20:59:39.571 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-22T20:59:39.571 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-22T20:59:39.571 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-05-22T20:59:39.571 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-22T20:59:39.571 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-22T20:59:39.571 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-22T20:59:39.571 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-22T20:59:39.571 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-22T20:59:39.571 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-22T20:59:39.571 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-22T20:59:39.571 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-22T20:59:39.571 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-22T20:59:39.571 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-05-22T20:59:39.572 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-22T20:59:39.572 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-22T20:59:39.572 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-22T20:59:39.572 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-22T20:59:39.572 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-22T20:59:39.572 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_bic.ko 2024-05-22T20:59:39.572 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_dctcp.ko 2024-05-22T20:59:39.572 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_diag.ko 2024-05-22T20:59:39.572 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_highspeed.ko 2024-05-22T20:59:39.572 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_htcp.ko 2024-05-22T20:59:39.572 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11665 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_hybla.ko 2024-05-22T20:59:39.572 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_illinois.ko 2024-05-22T20:59:39.572 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11737 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_lp.ko 2024-05-22T20:59:39.572 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_scalable.ko 2024-05-22T20:59:39.573 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_vegas.ko 2024-05-22T20:59:39.573 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_veno.ko 2024-05-22T20:59:39.573 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_westwood.ko 2024-05-22T20:59:39.573 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_yeah.ko 2024-05-22T20:59:39.573 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tunnel4.ko 2024-05-22T20:59:39.573 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/udp_diag.ko 2024-05-22T20:59:39.573 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 54441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/udp_tunnel.ko 2024-05-22T20:59:39.573 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-22T20:59:39.573 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ 2024-05-22T20:59:39.573 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ah6.ko 2024-05-22T20:59:39.573 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 47433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/esp6.ko 2024-05-22T20:59:39.573 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/fou6.ko 2024-05-22T20:59:39.573 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 103177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ip6_gre.ko 2024-05-22T20:59:39.573 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 118881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ip6_tunnel.ko 2024-05-22T20:59:39.574 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-22T20:59:39.574 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 71089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ip6_vti.ko 2024-05-22T20:59:39.574 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ipcomp6.ko 2024-05-22T20:59:39.574 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/mip6.ko 2024-05-22T20:59:39.574 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ 2024-05-22T20:59:39.574 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 50673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-22T20:59:39.574 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-05-22T20:59:39.574 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-22T20:59:39.574 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-22T20:59:39.574 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15929 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-22T20:59:39.574 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-22T20:59:39.574 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-22T20:59:39.575 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-22T20:59:39.575 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-22T20:59:39.575 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-22T20:59:39.575 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-22T20:59:39.575 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-22T20:59:39.575 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-22T20:59:39.575 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-22T20:59:39.575 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-22T20:59:39.575 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-22T20:59:39.575 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-22T20:59:39.575 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 46313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-22T20:59:39.576 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-22T20:59:39.576 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 29817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-22T20:59:39.576 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-22T20:59:39.576 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-22T20:59:39.576 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-22T20:59:39.576 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-22T20:59:39.576 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-22T20:59:39.576 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 94089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/sit.ko 2024-05-22T20:59:39.576 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 30297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/tunnel6.ko 2024-05-22T20:59:39.576 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-22T20:59:39.576 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/key/ 2024-05-22T20:59:39.577 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 106369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/key/af_key.ko 2024-05-22T20:59:39.577 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/ 2024-05-22T20:59:39.577 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 153217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_core.ko 2024-05-22T20:59:39.577 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-22T20:59:39.577 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_eth.ko 2024-05-22T20:59:39.577 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_ip.ko 2024-05-22T20:59:39.577 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_ip6.ko 2024-05-22T20:59:39.577 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 42049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_netlink.ko 2024-05-22T20:59:39.577 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 66241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_ppp.ko 2024-05-22T20:59:39.577 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/lapb/ 2024-05-22T20:59:39.577 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 63345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/lapb/lapb.ko 2024-05-22T20:59:39.577 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/llc/ 2024-05-22T20:59:39.577 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/llc/llc.ko 2024-05-22T20:59:39.578 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 332889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/llc/llc2.ko 2024-05-22T20:59:39.578 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/mac802154/ 2024-05-22T20:59:39.578 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 388873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/mac802154/mac802154.ko 2024-05-22T20:59:39.578 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/mpls/ 2024-05-22T20:59:39.578 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/mpls/mpls_gso.ko 2024-05-22T20:59:39.578 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ 2024-05-22T20:59:39.578 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ 2024-05-22T20:59:39.578 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 147745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set.ko 2024-05-22T20:59:39.578 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-22T20:59:39.578 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-22T20:59:39.578 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-22T20:59:39.578 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 124041 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-22T20:59:39.579 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 124345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-22T20:59:39.579 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 126681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-22T20:59:39.579 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 126473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-22T20:59:39.579 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 137833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-22T20:59:39.579 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 72353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-22T20:59:39.579 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 133681 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-22T20:59:39.579 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 139073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-22T20:59:39.579 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 138265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-22T20:59:39.579 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 136177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-22T20:59:39.579 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 139897 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-22T20:59:39.579 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 37129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-22T20:59:39.579 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ 2024-05-22T20:59:39.579 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 534113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-22T20:59:39.580 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-22T20:59:39.580 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-22T20:59:39.580 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-22T20:59:39.580 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20953 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-22T20:59:39.580 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24577 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-22T20:59:39.580 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-22T20:59:39.580 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-22T20:59:39.580 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12649 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-22T20:59:39.580 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-22T20:59:39.580 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-22T20:59:39.580 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15977 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-22T20:59:39.580 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-22T20:59:39.580 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12809 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-22T20:59:39.580 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 55217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conncount.ko 2024-05-22T20:59:39.581 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 496817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack.ko 2024-05-22T20:59:39.581 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-22T20:59:39.581 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-22T20:59:39.581 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 37865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-22T20:59:39.581 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 134377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-22T20:59:39.581 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24513 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-22T20:59:39.581 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-22T20:59:39.581 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 140801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-22T20:59:39.581 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 38305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-22T20:59:39.581 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-22T20:59:39.581 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 79873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-22T20:59:39.581 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-22T20:59:39.581 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-22T20:59:39.582 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-22T20:59:39.582 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 117201 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_flow_table.ko 2024-05-22T20:59:39.582 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-22T20:59:39.582 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 46105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_log_syslog.ko 2024-05-22T20:59:39.582 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 118369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat.ko 2024-05-22T20:59:39.582 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-22T20:59:39.582 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14185 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-22T20:59:39.582 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13345 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat_irc.ko 2024-05-22T20:59:39.582 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat_sip.ko 2024-05-22T20:59:39.582 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-22T20:59:39.582 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 53393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-22T20:59:39.582 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 807985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_tables.ko 2024-05-22T20:59:39.582 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 53049 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink.ko 2024-05-22T20:59:39.583 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-22T20:59:39.583 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 37209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-05-22T20:59:39.583 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 49297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink_log.ko 2024-05-22T20:59:39.583 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-22T20:59:39.583 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 65145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-22T20:59:39.583 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_chain_nat.ko 2024-05-22T20:59:39.583 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 41769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_compat.ko 2024-05-22T20:59:39.583 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 50657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_ct.ko 2024-05-22T20:59:39.583 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11081 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-22T20:59:39.583 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_fib.ko 2024-05-22T20:59:39.583 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16545 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-22T20:59:39.583 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_hash.ko 2024-05-22T20:59:39.583 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_limit.ko 2024-05-22T20:59:39.583 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17921 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_log.ko 2024-05-22T20:59:39.584 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_masq.ko 2024-05-22T20:59:39.584 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_nat.ko 2024-05-22T20:59:39.584 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16473 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_numgen.ko 2024-05-22T20:59:39.584 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13945 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_osf.ko 2024-05-22T20:59:39.584 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_queue.ko 2024-05-22T20:59:39.584 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_redir.ko 2024-05-22T20:59:39.584 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_reject.ko 2024-05-22T20:59:39.584 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_reject_inet.ko 2024-05-22T20:59:39.584 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15057 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_socket.ko 2024-05-22T20:59:39.584 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 24993 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_synproxy.ko 2024-05-22T20:59:39.584 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17641 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_tproxy.ko 2024-05-22T20:59:39.584 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 108393 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/x_tables.ko 2024-05-22T20:59:39.584 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_AUDIT.ko 2024-05-22T20:59:39.585 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-22T20:59:39.585 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-22T20:59:39.585 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-22T20:59:39.585 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 21865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_CT.ko 2024-05-22T20:59:39.585 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_DSCP.ko 2024-05-22T20:59:39.585 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_HL.ko 2024-05-22T20:59:39.585 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15745 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_HMARK.ko 2024-05-22T20:59:39.585 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 45217 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-22T20:59:39.585 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15937 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_LED.ko 2024-05-22T20:59:39.585 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13017 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_LOG.ko 2024-05-22T20:59:39.585 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-22T20:59:39.585 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_NETMAP.ko 2024-05-22T20:59:39.585 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10857 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_NFLOG.ko 2024-05-22T20:59:39.585 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15361 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-22T20:59:39.586 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23089 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_RATEEST.ko 2024-05-22T20:59:39.586 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-22T20:59:39.586 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16689 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_SECMARK.ko 2024-05-22T20:59:39.586 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18985 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-22T20:59:39.586 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-22T20:59:39.586 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20697 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_TEE.ko 2024-05-22T20:59:39.586 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_TPROXY.ko 2024-05-22T20:59:39.586 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_TRACE.ko 2024-05-22T20:59:39.586 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15289 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_addrtype.ko 2024-05-22T20:59:39.586 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16633 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_bpf.ko 2024-05-22T20:59:39.586 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_cgroup.ko 2024-05-22T20:59:39.586 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_cluster.ko 2024-05-22T20:59:39.587 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_comment.ko 2024-05-22T20:59:39.587 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11961 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_connbytes.ko 2024-05-22T20:59:39.587 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_connlabel.ko 2024-05-22T20:59:39.587 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_connlimit.ko 2024-05-22T20:59:39.587 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_connmark.ko 2024-05-22T20:59:39.587 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14209 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_conntrack.ko 2024-05-22T20:59:39.587 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_cpu.ko 2024-05-22T20:59:39.587 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_dccp.ko 2024-05-22T20:59:39.587 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_devgroup.ko 2024-05-22T20:59:39.587 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10257 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_dscp.ko 2024-05-22T20:59:39.587 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12657 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_ecn.ko 2024-05-22T20:59:39.587 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_esp.ko 2024-05-22T20:59:39.587 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 52817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_hashlimit.ko 2024-05-22T20:59:39.587 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_helper.ko 2024-05-22T20:59:39.588 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_hl.ko 2024-05-22T20:59:39.588 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_ipcomp.ko 2024-05-22T20:59:39.588 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_iprange.ko 2024-05-22T20:59:39.588 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_ipvs.ko 2024-05-22T20:59:39.588 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19193 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_l2tp.ko 2024-05-22T20:59:39.588 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9137 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_length.ko 2024-05-22T20:59:39.588 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11521 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_limit.ko 2024-05-22T20:59:39.588 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_mac.ko 2024-05-22T20:59:39.588 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_mark.ko 2024-05-22T20:59:39.588 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_multiport.ko 2024-05-22T20:59:39.588 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 16161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_nat.ko 2024-05-22T20:59:39.588 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_nfacct.ko 2024-05-22T20:59:39.588 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_osf.ko 2024-05-22T20:59:39.589 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11561 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_owner.ko 2024-05-22T20:59:39.589 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_physdev.ko 2024-05-22T20:59:39.589 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_pkttype.ko 2024-05-22T20:59:39.589 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13553 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_policy.ko 2024-05-22T20:59:39.589 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_quota.ko 2024-05-22T20:59:39.589 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 11881 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_rateest.ko 2024-05-22T20:59:39.589 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9273 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_realm.ko 2024-05-22T20:59:39.589 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_recent.ko 2024-05-22T20:59:39.589 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_sctp.ko 2024-05-22T20:59:39.589 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36489 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_set.ko 2024-05-22T20:59:39.589 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 19777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_socket.ko 2024-05-22T20:59:39.589 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10753 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_state.ko 2024-05-22T20:59:39.589 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10849 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_statistic.ko 2024-05-22T20:59:39.590 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_string.ko 2024-05-22T20:59:39.590 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_tcpmss.ko 2024-05-22T20:59:39.590 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18449 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_tcpudp.ko 2024-05-22T20:59:39.590 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12441 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_time.ko 2024-05-22T20:59:39.590 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_u32.ko 2024-05-22T20:59:39.590 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netlink/ 2024-05-22T20:59:39.590 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 17233 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netlink/netlink_diag.ko 2024-05-22T20:59:39.590 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/nsh/ 2024-05-22T20:59:39.590 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 14073 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/nsh/nsh.ko 2024-05-22T20:59:39.590 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/openvswitch/ 2024-05-22T20:59:39.590 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 488825 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/openvswitch/openvswitch.ko 2024-05-22T20:59:39.590 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12009 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/openvswitch/vport-geneve.ko 2024-05-22T20:59:39.590 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 10873 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/openvswitch/vport-gre.ko 2024-05-22T20:59:39.591 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/openvswitch/vport-vxlan.ko 2024-05-22T20:59:39.591 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/packet/ 2024-05-22T20:59:39.591 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15929 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/packet/af_packet_diag.ko 2024-05-22T20:59:39.591 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/phonet/ 2024-05-22T20:59:39.591 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 142025 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/phonet/phonet.ko 2024-05-22T20:59:39.591 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 68817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/phonet/pn_pep.ko 2024-05-22T20:59:39.591 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/rds/ 2024-05-22T20:59:39.591 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 318617 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/rds/rds.ko 2024-05-22T20:59:39.591 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 202305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/rds/rds_rdma.ko 2024-05-22T20:59:39.591 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 85105 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/rds/rds_tcp.ko 2024-05-22T20:59:39.591 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/rxrpc/ 2024-05-22T20:59:39.591 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1027897 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/rxrpc/rxrpc.ko 2024-05-22T20:59:39.591 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/ 2024-05-22T20:59:39.592 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 39001 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_csum.ko 2024-05-22T20:59:39.592 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 28721 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_gact.ko 2024-05-22T20:59:39.592 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 45409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_mirred.ko 2024-05-22T20:59:39.592 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 27593 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_nat.ko 2024-05-22T20:59:39.592 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 53465 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_pedit.ko 2024-05-22T20:59:39.592 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 38705 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_police.ko 2024-05-22T20:59:39.592 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23969 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_simple.ko 2024-05-22T20:59:39.592 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43417 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_skbedit.ko 2024-05-22T20:59:39.592 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 45673 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_vlan.ko 2024-05-22T20:59:39.592 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32129 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_basic.ko 2024-05-22T20:59:39.592 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 41457 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_bpf.ko 2024-05-22T20:59:39.592 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23225 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_cgroup.ko 2024-05-22T20:59:39.592 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 37305 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_flow.ko 2024-05-22T20:59:39.593 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 39601 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_fw.ko 2024-05-22T20:59:39.593 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 49321 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_route.ko 2024-05-22T20:59:39.593 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 79145 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_u32.ko 2024-05-22T20:59:39.593 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8713 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_cmp.ko 2024-05-22T20:59:39.593 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 15385 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_ipset.ko 2024-05-22T20:59:39.593 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 48785 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_meta.ko 2024-05-22T20:59:39.593 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9241 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_nbyte.ko 2024-05-22T20:59:39.593 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13369 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_text.ko 2024-05-22T20:59:39.593 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 8377 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_u32.ko 2024-05-22T20:59:39.593 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_choke.ko 2024-05-22T20:59:39.593 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 25097 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_codel.ko 2024-05-22T20:59:39.593 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43481 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_drr.ko 2024-05-22T20:59:39.594 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 43153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_fq.ko 2024-05-22T20:59:39.594 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 39177 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_fq_codel.ko 2024-05-22T20:59:39.594 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 42889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_gred.ko 2024-05-22T20:59:39.594 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 62817 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_hfsc.ko 2024-05-22T20:59:39.594 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26585 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_hhf.ko 2024-05-22T20:59:39.594 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 81833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_htb.ko 2024-05-22T20:59:39.594 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33545 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_ingress.ko 2024-05-22T20:59:39.594 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 36625 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_mqprio.ko 2024-05-22T20:59:39.594 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12297 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_mqprio_lib.ko 2024-05-22T20:59:39.594 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33065 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_multiq.ko 2024-05-22T20:59:39.594 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 44793 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_netem.ko 2024-05-22T20:59:39.594 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 32833 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_pie.ko 2024-05-22T20:59:39.595 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 13505 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_plug.ko 2024-05-22T20:59:39.595 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 31801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_prio.ko 2024-05-22T20:59:39.595 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 60249 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_qfq.ko 2024-05-22T20:59:39.595 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 40889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_red.ko 2024-05-22T20:59:39.595 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 33433 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_sfb.ko 2024-05-22T20:59:39.595 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 38329 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_sfq.ko 2024-05-22T20:59:39.595 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 35777 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_tbf.ko 2024-05-22T20:59:39.595 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 34161 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_teql.ko 2024-05-22T20:59:39.595 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sctp/ 2024-05-22T20:59:39.595 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1187313 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sctp/sctp.ko 2024-05-22T20:59:39.595 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23913 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sctp/sctp_diag.ko 2024-05-22T20:59:39.595 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/ 2024-05-22T20:59:39.596 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/auth_gss/ 2024-05-22T20:59:39.596 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 432409 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-22T20:59:39.596 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 70113 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-22T20:59:39.596 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1940889 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/sunrpc.ko 2024-05-22T20:59:39.596 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/xprtrdma/ 2024-05-22T20:59:39.596 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 890729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-22T20:59:39.596 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/tipc/ 2024-05-22T20:59:39.596 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 1122337 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/tipc/tipc.ko 2024-05-22T20:59:39.596 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12153 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/tipc/tipc_diag.ko 2024-05-22T20:59:39.596 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/unix/ 2024-05-22T20:59:39.596 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 18801 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/unix/unix_diag.ko 2024-05-22T20:59:39.596 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/ 2024-05-22T20:59:39.596 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 105609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-22T20:59:39.597 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 90265 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-22T20:59:39.597 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 123769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/vsock.ko 2024-05-22T20:59:39.597 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 12905 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-22T20:59:39.597 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 20401 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-22T20:59:39.597 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/x25/ 2024-05-22T20:59:39.597 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 165841 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/x25/x25.ko 2024-05-22T20:59:39.597 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/xfrm/ 2024-05-22T20:59:39.597 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 26729 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/xfrm/xfrm_algo.ko 2024-05-22T20:59:39.597 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23353 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-22T20:59:39.597 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 131865 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/xfrm/xfrm_user.ko 2024-05-22T20:59:39.597 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 644769 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.alias 2024-05-22T20:59:39.597 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 628542 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.alias.bin 2024-05-22T20:59:39.597 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 7706 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.builtin 2024-05-22T20:59:39.598 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 23609 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.builtin.alias.bin 2024-05-22T20:59:39.598 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 9941 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.builtin.bin 2024-05-22T20:59:39.598 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 70990 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.builtin.modinfo 2024-05-22T20:59:39.598 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 146959 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.dep 2024-05-22T20:59:39.598 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 232835 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.dep.bin 2024-05-22T20:59:39.598 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 210 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.devname 2024-05-22T20:59:39.598 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 72602 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.order 2024-05-22T20:59:39.598 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 879 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.softdep 2024-05-22T20:59:39.598 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 258450 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.symbols 2024-05-22T20:59:39.598 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 313019 2024-05-21 01:11 ./lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.symbols.bin 2024-05-22T20:59:39.598 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./usr/ 2024-05-22T20:59:39.598 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./usr/share/ 2024-05-22T20:59:39.598 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./usr/share/doc/ 2024-05-22T20:59:39.599 INFO:teuthology.orchestra.run.smithi161.stdout:drwxr-xr-x root/root 0 2024-05-21 01:11 ./usr/share/doc/linux-image-6.9.0-rc3-g4fed890e8ce3/ 2024-05-22T20:59:39.599 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 174 2024-05-21 01:11 ./usr/share/doc/linux-image-6.9.0-rc3-g4fed890e8ce3/changelog.Debian.gz 2024-05-22T20:59:39.599 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r-- root/root 691 2024-05-21 01:11 ./usr/share/doc/linux-image-6.9.0-rc3-g4fed890e8ce3/copyright 2024-05-22T20:59:39.603 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:39.603 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:39.603 DEBUG:teuthology.orchestra.run.smithi161:> egrep '(submenu|menuentry.*6.9.0-rc3-g4fed890e8ce3).*{' /boot/grub/grub.cfg 2024-05-22T20:59:39.609 INFO:teuthology.orchestra.run.smithi161.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-22T20:59:39.609 INFO:teuthology.orchestra.run.smithi161.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g4fed890e8ce3' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g4fed890e8ce3-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-22T20:59:39.609 INFO:teuthology.orchestra.run.smithi161.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g4fed890e8ce3 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g4fed890e8ce3-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-05-22T20:59:39.610 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-05-22T20:59:39.610 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:39.610 DEBUG:teuthology.orchestra.run.smithi161:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-05-22T20:59:39.654 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi161.front.sepia.ceph.com to complete... 2024-05-22T20:59:40.048 INFO:teuthology.orchestra.run.smithi161.stderr:Sourcing file `/etc/default/grub' 2024-05-22T20:59:40.049 INFO:teuthology.orchestra.run.smithi161.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-22T20:59:40.059 INFO:teuthology.orchestra.run.smithi161.stderr:Generating grub configuration file ... 2024-05-22T20:59:40.262 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:40.268 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g4fed890e8ce3 2024-05-22T20:59:40.506 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-22T20:59:40.511 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-22T20:59:40.549 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-22T20:59:40.553 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-22T20:59:40.665 INFO:teuthology.orchestra.run.smithi161.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-22T20:59:40.665 INFO:teuthology.orchestra.run.smithi161.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-22T20:59:40.665 INFO:teuthology.orchestra.run.smithi161.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-22T20:59:40.690 INFO:teuthology.orchestra.run.smithi161.stderr:done 2024-05-22T21:00:10.598 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-05-22T21:00:10.598 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-22T21:00:10.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T21:00:11.820 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-22T21:00:11.820 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-22T21:00:11.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T21:00:29.087 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.179 2024-05-22T21:00:38.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-22T21:00:38.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T21:00:41.150 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.179 2024-05-22T21:00:46.494 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-05-22T21:00:53.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-22T21:00:53.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T21:00:55.498 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-22T21:00:55.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T21:01:11.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.179 2024-05-22T21:01:26.590 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-22T21:01:26.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T21:01:27.107 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-05-22T21:01:28.448 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-22T21:01:28.448 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g4fed890e8ce3"... 2024-05-22T21:01:28.448 DEBUG:teuthology.orchestra.run.smithi179:> uname -r 2024-05-22T21:01:28.501 INFO:teuthology.orchestra.run.smithi179.stdout:6.9.0-rc3-g4fed890e8ce3 2024-05-22T21:01:28.501 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g4fed890e8ce3 vs 6.9.0-rc3-g4fed890e8ce3 2024-05-22T21:01:28.502 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-22T21:01:28.502 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-22T21:01:29.503 DEBUG:teuthology.parallel:result is None 2024-05-22T21:01:55.501 DEBUG:teuthology.orchestra.remote:timed out 2024-05-22T21:02:07.502 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-22T21:02:07.503 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T21:02:07.711 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-05-22T21:02:08.337 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-22T21:02:08.337 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g4fed890e8ce3"... 2024-05-22T21:02:08.337 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-05-22T21:02:08.386 INFO:teuthology.orchestra.run.smithi161.stdout:6.9.0-rc3-g4fed890e8ce3 2024-05-22T21:02:08.386 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g4fed890e8ce3 vs 6.9.0-rc3-g4fed890e8ce3 2024-05-22T21:02:08.386 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-22T21:02:08.386 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-22T21:02:09.389 DEBUG:teuthology.parallel:result is None 2024-05-22T21:02:09.389 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-22T21:02:09.397 INFO:teuthology.task.internal:Creating test directory... 2024-05-22T21:02:09.397 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-22T21:02:09.400 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-22T21:02:09.410 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-22T21:02:09.421 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-22T21:02:09.434 INFO:teuthology.task.internal:Creating archive directory... 2024-05-22T21:02:09.435 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-22T21:02:09.449 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-22T21:02:09.568 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-22T21:02:09.576 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-22T21:02:09.577 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-22T21:02:09.580 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-22T21:02:09.659 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-22T21:02:09.681 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-22T21:02:09.711 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-22T21:02:09.733 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-22T21:02:09.735 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-22T21:02:09.744 INFO:teuthology.task.internal:Configuring sudo... 2024-05-22T21:02:09.744 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-22T21:02:09.748 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-22T21:02:09.769 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-22T21:02:09.782 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-22T21:02:09.782 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-22T21:02:09.810 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-22T21:02:09.822 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-22T21:02:09.866 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-22T21:02:09.919 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:02:09.919 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-22T21:02:10.010 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-22T21:02:10.019 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-22T21:02:10.069 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:02:10.069 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-22T21:02:10.138 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-05-22T21:02:10.142 DEBUG:teuthology.orchestra.run.smithi179:> sudo service rsyslog restart 2024-05-22T21:02:10.254 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-22T21:02:10.263 INFO:teuthology.task.internal:Starting timer... 2024-05-22T21:02:10.263 INFO:teuthology.run_tasks:Running task pcp... 2024-05-22T21:02:10.282 INFO:teuthology.run_tasks:Running task selinux... 2024-05-22T21:02:10.296 DEBUG:teuthology.task.selinux:Excluding smithi161: OS 'ubuntu' does not support SELinux 2024-05-22T21:02:10.296 DEBUG:teuthology.task.selinux:Excluding smithi179: OS 'ubuntu' does not support SELinux 2024-05-22T21:02:10.296 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-22T21:02:10.296 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-05-22T21:02:10.296 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-22T21:02:10.296 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-22T21:02:10.311 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-22T21:02:10.323 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-22T21:02:10.324 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi161.front.sepia.ceph.com,smithi179.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-22T21:09:47.888 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi161.front.sepia.ceph.com'), Remote(name='ubuntu@smithi179.front.sepia.ceph.com')] 2024-05-22T21:09:47.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-22T21:09:47.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T21:09:47.973 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-05-22T21:09:48.093 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-22T21:09:48.093 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-22T21:09:48.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-22T21:09:48.172 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-05-22T21:09:48.313 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-22T21:09:48.313 INFO:teuthology.run_tasks:Running task clock... 2024-05-22T21:09:48.324 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-22T21:09:48.324 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-22T21:09:48.325 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-22T21:09:48.328 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-22T21:09:48.328 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-22T21:09:48.363 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-05-22T21:09:48.363 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: Command line: ntpd -gq 2024-05-22T21:09:48.363 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: ---------------------------------------------------- 2024-05-22T21:09:48.363 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: ntp-4 is maintained by Network Time Foundation, 2024-05-22T21:09:48.363 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-05-22T21:09:48.364 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: corporation. Support and training for ntp-4 are 2024-05-22T21:09:48.364 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: available at https://www.nwtime.org/support 2024-05-22T21:09:48.364 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: ---------------------------------------------------- 2024-05-22T21:09:48.364 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: proto: precision = 0.042 usec (-24) 2024-05-22T21:09:48.365 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: basedate set to 2022-02-04 2024-05-22T21:09:48.365 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: gps base set to 2022-02-06 (week 2196) 2024-05-22T21:09:48.365 INFO:teuthology.orchestra.run.smithi161.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-22T21:09:48.365 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-22T21:09:48.366 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: restrict ::: KOD does nothing without LIMITED. 2024-05-22T21:09:48.366 INFO:teuthology.orchestra.run.smithi161.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-22T21:09:48.367 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: Listen and drop on 0 v6wildcard [::]:123 2024-05-22T21:09:48.367 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-22T21:09:48.367 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: Listen normally on 2 lo 127.0.0.1:123 2024-05-22T21:09:48.367 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: Listen normally on 3 enp3s0f1 172.21.15.161:123 2024-05-22T21:09:48.368 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: Listen normally on 4 lo [::1]:123 2024-05-22T21:09:48.368 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7251%5]:123 2024-05-22T21:09:48.368 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:48 ntpd[17624]: Listening on routing socket on fd #22 for interface updates 2024-05-22T21:09:48.394 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-05-22T21:09:48.394 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: Command line: ntpd -gq 2024-05-22T21:09:48.394 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: ---------------------------------------------------- 2024-05-22T21:09:48.394 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: ntp-4 is maintained by Network Time Foundation, 2024-05-22T21:09:48.394 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-05-22T21:09:48.394 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: corporation. Support and training for ntp-4 are 2024-05-22T21:09:48.394 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: available at https://www.nwtime.org/support 2024-05-22T21:09:48.394 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: ---------------------------------------------------- 2024-05-22T21:09:48.395 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: proto: precision = 0.042 usec (-24) 2024-05-22T21:09:48.395 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: basedate set to 2022-02-04 2024-05-22T21:09:48.395 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: gps base set to 2022-02-06 (week 2196) 2024-05-22T21:09:48.396 INFO:teuthology.orchestra.run.smithi179.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-22T21:09:48.396 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-22T21:09:48.396 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: restrict ::: KOD does nothing without LIMITED. 2024-05-22T21:09:48.396 INFO:teuthology.orchestra.run.smithi179.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-22T21:09:48.397 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: Listen and drop on 0 v6wildcard [::]:123 2024-05-22T21:09:48.397 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-22T21:09:48.397 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: Listen normally on 2 lo 127.0.0.1:123 2024-05-22T21:09:48.397 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: Listen normally on 3 enp3s0f1 172.21.15.179:123 2024-05-22T21:09:48.397 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: Listen normally on 4 lo [::1]:123 2024-05-22T21:09:48.398 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:725b%5]:123 2024-05-22T21:09:48.398 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:48 ntpd[17562]: Listening on routing socket on fd #22 for interface updates 2024-05-22T21:09:49.364 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:49 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:49.364 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:49 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:49.364 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:49 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:49.364 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:49 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:49.364 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:49 ntpd[17624]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-22T21:09:49.365 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:49 ntpd[17624]: can't open /var/log/ntpstats/peerstats.20240522: Permission denied 2024-05-22T21:09:49.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:49 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:49.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:49 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:49.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:49 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:49.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:49 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:49.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:49 ntpd[17562]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-22T21:09:49.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:49 ntpd[17562]: can't open /var/log/ntpstats/peerstats.20240522: Permission denied 2024-05-22T21:09:51.364 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:51 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:51.364 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:51 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:51.364 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:51 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:51.364 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:51 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:51.364 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:51 ntpd[17624]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-22T21:09:51.365 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:51 ntpd[17624]: can't open /var/log/ntpstats/peerstats.20240522: Permission denied 2024-05-22T21:09:51.394 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:51 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:51.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:51 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:51.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:51 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:51.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:51 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:51.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:51 ntpd[17562]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-22T21:09:51.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:51 ntpd[17562]: can't open /var/log/ntpstats/peerstats.20240522: Permission denied 2024-05-22T21:09:52.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:52 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:52.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:52 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:52.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:52 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:52.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:52 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:52.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:52 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:52.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:52 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:52.396 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:52 ntpd[17562]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-22T21:09:52.396 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:52 ntpd[17562]: can't open /var/log/ntpstats/peerstats.20240522: Permission denied 2024-05-22T21:09:53.367 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:53.368 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:53.368 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:53.368 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:53.368 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:53.368 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:53.368 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:53.369 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:53.369 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:53.369 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:53.369 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-22T21:09:53.369 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: can't open /var/log/ntpstats/peerstats.20240522: Permission denied 2024-05-22T21:09:53.369 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:53.370 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:53.370 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-22T21:09:53.370 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: can't open /var/log/ntpstats/peerstats.20240522: Permission denied 2024-05-22T21:09:53.370 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:53.370 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:53.371 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-22T21:09:53.371 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:53 ntpd[17624]: can't open /var/log/ntpstats/peerstats.20240522: Permission denied 2024-05-22T21:09:53.396 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:53 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:53.397 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:53 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:53.397 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:53 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:53.397 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:53 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:53.397 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:53 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:53.397 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:53 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:53.397 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:53 ntpd[17562]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-22T21:09:53.397 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:53 ntpd[17562]: can't open /var/log/ntpstats/peerstats.20240522: Permission denied 2024-05-22T21:09:53.397 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:53 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:53.397 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:53 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:53.397 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:53 ntpd[17562]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-22T21:09:53.397 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:53 ntpd[17562]: can't open /var/log/ntpstats/peerstats.20240522: Permission denied 2024-05-22T21:09:54.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:54 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:54.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:54 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:54.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:54 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:54.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:54 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:54.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:54 ntpd[17562]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-22T21:09:54.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:54 ntpd[17562]: can't open /var/log/ntpstats/peerstats.20240522: Permission denied 2024-05-22T21:09:55.365 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:55.365 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:55.365 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:55.365 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:55.366 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:55.366 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:55.366 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:55.366 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:55.366 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-22T21:09:55.366 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: can't open /var/log/ntpstats/peerstats.20240522: Permission denied 2024-05-22T21:09:55.366 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:55.366 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:55.366 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-22T21:09:55.366 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: can't open /var/log/ntpstats/peerstats.20240522: Permission denied 2024-05-22T21:09:55.366 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:55.366 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:55.367 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-22T21:09:55.367 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: can't open /var/log/ntpstats/peerstats.20240522: Permission denied 2024-05-22T21:09:55.367 INFO:teuthology.orchestra.run.smithi161.stdout:22 May 21:09:55 ntpd[17624]: ntpd: time slew -0.000721 s 2024-05-22T21:09:55.367 INFO:teuthology.orchestra.run.smithi161.stdout:ntpd: time slew -0.000721s 2024-05-22T21:09:55.368 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-22T21:09:55.368 INFO:teuthology.orchestra.run.smithi161.stderr:22 May 21:09:55 ntpd[17624]: can't open /var/log/ntpstats/loopstats.20240522: Permission denied 2024-05-22T21:09:55.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:55 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:55.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:55 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:55.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:55 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:55.395 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:55 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:55.396 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:55 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:55.396 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:55 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:55.396 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:55 ntpd[17562]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-22T21:09:55.396 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:55 ntpd[17562]: can't open /var/log/ntpstats/peerstats.20240522: Permission denied 2024-05-22T21:09:55.396 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:55 ntpd[17562]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-22T21:09:55.396 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:55 ntpd[17562]: can't open /var/log/ntpstats/rawstats.20240522: Permission denied 2024-05-22T21:09:55.396 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:55 ntpd[17562]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-22T21:09:55.396 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:55 ntpd[17562]: can't open /var/log/ntpstats/peerstats.20240522: Permission denied 2024-05-22T21:09:55.396 INFO:teuthology.orchestra.run.smithi179.stdout:22 May 21:09:55 ntpd[17562]: ntpd: time slew -0.002042 s 2024-05-22T21:09:55.396 INFO:teuthology.orchestra.run.smithi179.stdout:ntpd: time slew -0.002042s 2024-05-22T21:09:55.397 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:55 ntpd[17562]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-22T21:09:55.397 INFO:teuthology.orchestra.run.smithi179.stderr:22 May 21:09:55 ntpd[17562]: can't open /var/log/ntpstats/loopstats.20240522: Permission denied 2024-05-22T21:09:55.475 INFO:teuthology.orchestra.run.smithi161.stdout: remote refid st t when poll reach delay offset jitter 2024-05-22T21:09:55.475 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================== 2024-05-22T21:09:55.476 INFO:teuthology.orchestra.run.smithi161.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-22T21:09:55.476 INFO:teuthology.orchestra.run.smithi161.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-22T21:09:55.476 INFO:teuthology.orchestra.run.smithi161.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-22T21:09:55.476 INFO:teuthology.orchestra.run.smithi161.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-22T21:09:55.509 INFO:teuthology.orchestra.run.smithi179.stdout: remote refid st t when poll reach delay offset jitter 2024-05-22T21:09:55.509 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================== 2024-05-22T21:09:55.510 INFO:teuthology.orchestra.run.smithi179.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-22T21:09:55.510 INFO:teuthology.orchestra.run.smithi179.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-22T21:09:55.510 INFO:teuthology.orchestra.run.smithi179.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-22T21:09:55.510 INFO:teuthology.orchestra.run.smithi179.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-22T21:09:55.511 INFO:teuthology.run_tasks:Running task install... 2024-05-22T21:09:55.521 DEBUG:teuthology.task.install:project ceph 2024-05-22T21:09:55.521 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b9b067bc6d30ec00e391f0fa43f50affe981ba2f'}} 2024-05-22T21:09:55.521 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b9b067bc6d30ec00e391f0fa43f50affe981ba2f'} 2024-05-22T21:09:55.521 INFO:teuthology.task.install:Using flavor: default 2024-05-22T21:09:55.527 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-22T21:09:55.527 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-22T21:09:55.527 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-key list | grep Ceph 2024-05-22T21:09:55.528 DEBUG:teuthology.orchestra.run.smithi179:> sudo apt-key list | grep Ceph 2024-05-22T21:09:55.628 INFO:teuthology.orchestra.run.smithi161.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-05-22T21:09:55.658 INFO:teuthology.orchestra.run.smithi179.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-05-22T21:09:55.678 INFO:teuthology.orchestra.run.smithi161.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-22T21:09:55.678 INFO:teuthology.orchestra.run.smithi161.stdout:uid [ unknown] Ceph.com (release key) 2024-05-22T21:09:55.679 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-05-22T21:09:55.679 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-05-22T21:09:55.679 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b9b067bc6d30ec00e391f0fa43f50affe981ba2f 2024-05-22T21:09:55.712 INFO:teuthology.orchestra.run.smithi179.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-22T21:09:55.713 INFO:teuthology.orchestra.run.smithi179.stdout:uid [ unknown] Ceph.com (release key) 2024-05-22T21:09:55.713 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-05-22T21:09:55.713 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-05-22T21:09:55.713 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b9b067bc6d30ec00e391f0fa43f50affe981ba2f 2024-05-22T21:09:55.833 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default/ 2024-05-22T21:09:55.833 INFO:teuthology.task.install.deb:Package version is 18.2.2-1349-gb9b067bc-1jammy 2024-05-22T21:09:55.854 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default/ 2024-05-22T21:09:55.854 INFO:teuthology.task.install.deb:Package version is 18.2.2-1349-gb9b067bc-1jammy 2024-05-22T21:09:56.171 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:09:56.172 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-22T21:09:56.191 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-get update 2024-05-22T21:09:56.214 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:09:56.214 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-22T21:09:56.233 DEBUG:teuthology.orchestra.run.smithi179:> sudo apt-get update 2024-05-22T21:09:56.435 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-22T21:09:56.435 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-22T21:09:56.452 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-22T21:09:56.491 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-22T21:09:56.501 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-22T21:09:56.501 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-22T21:09:56.573 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-22T21:09:56.654 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-22T21:09:56.777 INFO:teuthology.orchestra.run.smithi161.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy InRelease 2024-05-22T21:09:56.790 INFO:teuthology.orchestra.run.smithi179.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy InRelease 2024-05-22T21:09:56.873 INFO:teuthology.orchestra.run.smithi161.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-05-22T21:09:56.885 INFO:teuthology.orchestra.run.smithi179.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-05-22T21:09:56.968 INFO:teuthology.orchestra.run.smithi161.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy Release.gpg 2024-05-22T21:09:56.980 INFO:teuthology.orchestra.run.smithi179.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy Release.gpg 2024-05-22T21:09:57.067 INFO:teuthology.orchestra.run.smithi161.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.7 kB] 2024-05-22T21:09:57.070 INFO:teuthology.orchestra.run.smithi179.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.7 kB] 2024-05-22T21:09:57.153 INFO:teuthology.orchestra.run.smithi179.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,415 B] 2024-05-22T21:09:57.218 INFO:teuthology.orchestra.run.smithi161.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,415 B] 2024-05-22T21:09:57.262 INFO:teuthology.orchestra.run.smithi179.stdout:Fetched 28.8 kB in 1s (35.7 kB/s) 2024-05-22T21:09:57.327 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 28.8 kB in 1s (31.5 kB/s) 2024-05-22T21:09:58.256 INFO:teuthology.orchestra.run.smithi179.stdout:Reading package lists... 2024-05-22T21:09:58.271 DEBUG:teuthology.orchestra.run.smithi179:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.2-1349-gb9b067bc-1jammy cephadm=18.2.2-1349-gb9b067bc-1jammy ceph-mds=18.2.2-1349-gb9b067bc-1jammy ceph-mgr=18.2.2-1349-gb9b067bc-1jammy ceph-common=18.2.2-1349-gb9b067bc-1jammy ceph-fuse=18.2.2-1349-gb9b067bc-1jammy ceph-test=18.2.2-1349-gb9b067bc-1jammy ceph-volume=18.2.2-1349-gb9b067bc-1jammy radosgw=18.2.2-1349-gb9b067bc-1jammy python3-rados=18.2.2-1349-gb9b067bc-1jammy python3-rgw=18.2.2-1349-gb9b067bc-1jammy python3-cephfs=18.2.2-1349-gb9b067bc-1jammy python3-rbd=18.2.2-1349-gb9b067bc-1jammy libcephfs2=18.2.2-1349-gb9b067bc-1jammy libcephfs-dev=18.2.2-1349-gb9b067bc-1jammy librados2=18.2.2-1349-gb9b067bc-1jammy librbd1=18.2.2-1349-gb9b067bc-1jammy rbd-fuse=18.2.2-1349-gb9b067bc-1jammy python3-cephfs=18.2.2-1349-gb9b067bc-1jammy cephfs-shell=18.2.2-1349-gb9b067bc-1jammy cephfs-top=18.2.2-1349-gb9b067bc-1jammy cephfs-mirror=18.2.2-1349-gb9b067bc-1jammy 2024-05-22T21:09:58.323 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2024-05-22T21:09:58.338 DEBUG:teuthology.orchestra.run.smithi161:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.2-1349-gb9b067bc-1jammy cephadm=18.2.2-1349-gb9b067bc-1jammy ceph-mds=18.2.2-1349-gb9b067bc-1jammy ceph-mgr=18.2.2-1349-gb9b067bc-1jammy ceph-common=18.2.2-1349-gb9b067bc-1jammy ceph-fuse=18.2.2-1349-gb9b067bc-1jammy ceph-test=18.2.2-1349-gb9b067bc-1jammy ceph-volume=18.2.2-1349-gb9b067bc-1jammy radosgw=18.2.2-1349-gb9b067bc-1jammy python3-rados=18.2.2-1349-gb9b067bc-1jammy python3-rgw=18.2.2-1349-gb9b067bc-1jammy python3-cephfs=18.2.2-1349-gb9b067bc-1jammy python3-rbd=18.2.2-1349-gb9b067bc-1jammy libcephfs2=18.2.2-1349-gb9b067bc-1jammy libcephfs-dev=18.2.2-1349-gb9b067bc-1jammy librados2=18.2.2-1349-gb9b067bc-1jammy librbd1=18.2.2-1349-gb9b067bc-1jammy rbd-fuse=18.2.2-1349-gb9b067bc-1jammy python3-cephfs=18.2.2-1349-gb9b067bc-1jammy cephfs-shell=18.2.2-1349-gb9b067bc-1jammy cephfs-top=18.2.2-1349-gb9b067bc-1jammy cephfs-mirror=18.2.2-1349-gb9b067bc-1jammy 2024-05-22T21:09:58.344 INFO:teuthology.orchestra.run.smithi179.stdout:Reading package lists... 2024-05-22T21:09:58.412 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2024-05-22T21:09:58.519 INFO:teuthology.orchestra.run.smithi179.stdout:Building dependency tree... 2024-05-22T21:09:58.520 INFO:teuthology.orchestra.run.smithi179.stdout:Reading state information... 2024-05-22T21:09:58.588 INFO:teuthology.orchestra.run.smithi161.stdout:Building dependency tree... 2024-05-22T21:09:58.589 INFO:teuthology.orchestra.run.smithi161.stdout:Reading state information... 2024-05-22T21:09:58.693 INFO:teuthology.orchestra.run.smithi179.stdout:The following packages were automatically installed and are no longer required: 2024-05-22T21:09:58.693 INFO:teuthology.orchestra.run.smithi179.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-22T21:09:58.693 INFO:teuthology.orchestra.run.smithi179.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-22T21:09:58.693 INFO:teuthology.orchestra.run.smithi179.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-22T21:09:58.694 INFO:teuthology.orchestra.run.smithi179.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-22T21:09:58.694 INFO:teuthology.orchestra.run.smithi179.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-22T21:09:58.694 INFO:teuthology.orchestra.run.smithi179.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-22T21:09:58.694 INFO:teuthology.orchestra.run.smithi179.stdout:The following additional packages will be installed: 2024-05-22T21:09:58.695 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-22T21:09:58.696 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-05-22T21:09:58.696 INFO:teuthology.orchestra.run.smithi179.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-05-22T21:09:58.696 INFO:teuthology.orchestra.run.smithi179.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-05-22T21:09:58.696 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-05-22T21:09:58.696 INFO:teuthology.orchestra.run.smithi179.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-05-22T21:09:58.696 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-05-22T21:09:58.696 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-05-22T21:09:58.696 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-05-22T21:09:58.697 INFO:teuthology.orchestra.run.smithi179.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-05-22T21:09:58.697 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-05-22T21:09:58.697 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-05-22T21:09:58.697 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-05-22T21:09:58.697 INFO:teuthology.orchestra.run.smithi179.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-05-22T21:09:58.697 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-05-22T21:09:58.697 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-05-22T21:09:58.697 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-05-22T21:09:58.697 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-05-22T21:09:58.697 INFO:teuthology.orchestra.run.smithi179.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-05-22T21:09:58.698 INFO:teuthology.orchestra.run.smithi179.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-05-22T21:09:58.698 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-05-22T21:09:58.698 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-05-22T21:09:58.698 INFO:teuthology.orchestra.run.smithi179.stdout: qttranslations5-l10n socat xmlstarlet 2024-05-22T21:09:58.698 INFO:teuthology.orchestra.run.smithi179.stdout:Suggested packages: 2024-05-22T21:09:58.698 INFO:teuthology.orchestra.run.smithi179.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-05-22T21:09:58.698 INFO:teuthology.orchestra.run.smithi179.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-05-22T21:09:58.699 INFO:teuthology.orchestra.run.smithi179.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-05-22T21:09:58.699 INFO:teuthology.orchestra.run.smithi179.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-05-22T21:09:58.699 INFO:teuthology.orchestra.run.smithi179.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-05-22T21:09:58.699 INFO:teuthology.orchestra.run.smithi179.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-05-22T21:09:58.699 INFO:teuthology.orchestra.run.smithi179.stdout: python3-watchdog 2024-05-22T21:09:58.699 INFO:teuthology.orchestra.run.smithi179.stdout:Recommended packages: 2024-05-22T21:09:58.699 INFO:teuthology.orchestra.run.smithi179.stdout: btrfs-tools 2024-05-22T21:09:58.763 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages were automatically installed and are no longer required: 2024-05-22T21:09:58.763 INFO:teuthology.orchestra.run.smithi161.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-22T21:09:58.763 INFO:teuthology.orchestra.run.smithi161.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-22T21:09:58.763 INFO:teuthology.orchestra.run.smithi161.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-22T21:09:58.763 INFO:teuthology.orchestra.run.smithi161.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-22T21:09:58.764 INFO:teuthology.orchestra.run.smithi161.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-22T21:09:58.764 INFO:teuthology.orchestra.run.smithi161.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-22T21:09:58.764 INFO:teuthology.orchestra.run.smithi161.stdout:The following additional packages will be installed: 2024-05-22T21:09:58.764 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-22T21:09:58.765 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-05-22T21:09:58.765 INFO:teuthology.orchestra.run.smithi161.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-05-22T21:09:58.765 INFO:teuthology.orchestra.run.smithi161.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-05-22T21:09:58.766 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-05-22T21:09:58.766 INFO:teuthology.orchestra.run.smithi161.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-05-22T21:09:58.766 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-05-22T21:09:58.766 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-05-22T21:09:58.766 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-05-22T21:09:58.767 INFO:teuthology.orchestra.run.smithi161.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-05-22T21:09:58.767 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-05-22T21:09:58.767 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-05-22T21:09:58.767 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-05-22T21:09:58.767 INFO:teuthology.orchestra.run.smithi161.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-05-22T21:09:58.767 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-05-22T21:09:58.767 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-05-22T21:09:58.767 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-05-22T21:09:58.767 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-05-22T21:09:58.767 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-05-22T21:09:58.767 INFO:teuthology.orchestra.run.smithi161.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-05-22T21:09:58.768 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-05-22T21:09:58.768 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-05-22T21:09:58.768 INFO:teuthology.orchestra.run.smithi161.stdout: qttranslations5-l10n socat xmlstarlet 2024-05-22T21:09:58.768 INFO:teuthology.orchestra.run.smithi161.stdout:Suggested packages: 2024-05-22T21:09:58.768 INFO:teuthology.orchestra.run.smithi161.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-05-22T21:09:58.768 INFO:teuthology.orchestra.run.smithi161.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-05-22T21:09:58.769 INFO:teuthology.orchestra.run.smithi161.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-05-22T21:09:58.769 INFO:teuthology.orchestra.run.smithi161.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-05-22T21:09:58.769 INFO:teuthology.orchestra.run.smithi161.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-05-22T21:09:58.769 INFO:teuthology.orchestra.run.smithi161.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-05-22T21:09:58.769 INFO:teuthology.orchestra.run.smithi161.stdout: python3-watchdog 2024-05-22T21:09:58.769 INFO:teuthology.orchestra.run.smithi161.stdout:Recommended packages: 2024-05-22T21:09:58.769 INFO:teuthology.orchestra.run.smithi161.stdout: btrfs-tools 2024-05-22T21:09:58.850 INFO:teuthology.orchestra.run.smithi179.stdout:The following NEW packages will be installed: 2024-05-22T21:09:58.851 INFO:teuthology.orchestra.run.smithi179.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-22T21:09:58.851 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-22T21:09:58.851 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-05-22T21:09:58.851 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-05-22T21:09:58.851 INFO:teuthology.orchestra.run.smithi179.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-05-22T21:09:58.851 INFO:teuthology.orchestra.run.smithi179.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-05-22T21:09:58.851 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-05-22T21:09:58.851 INFO:teuthology.orchestra.run.smithi179.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-05-22T21:09:58.851 INFO:teuthology.orchestra.run.smithi179.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-05-22T21:09:58.851 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-05-22T21:09:58.851 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-05-22T21:09:58.851 INFO:teuthology.orchestra.run.smithi179.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-05-22T21:09:58.851 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-22T21:09:58.851 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-22T21:09:58.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-22T21:09:58.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-22T21:09:58.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-22T21:09:58.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-22T21:09:58.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-05-22T21:09:58.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-05-22T21:09:58.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-05-22T21:09:58.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-05-22T21:09:58.852 INFO:teuthology.orchestra.run.smithi179.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-05-22T21:09:58.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-05-22T21:09:58.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-05-22T21:09:58.853 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-05-22T21:09:58.853 INFO:teuthology.orchestra.run.smithi179.stdout: socat xmlstarlet 2024-05-22T21:09:58.917 INFO:teuthology.orchestra.run.smithi161.stdout:The following NEW packages will be installed: 2024-05-22T21:09:58.917 INFO:teuthology.orchestra.run.smithi161.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-22T21:09:58.917 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-22T21:09:58.917 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-05-22T21:09:58.919 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-05-22T21:09:58.919 INFO:teuthology.orchestra.run.smithi161.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-05-22T21:09:58.919 INFO:teuthology.orchestra.run.smithi161.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-05-22T21:09:58.919 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-05-22T21:09:58.919 INFO:teuthology.orchestra.run.smithi161.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-05-22T21:09:58.919 INFO:teuthology.orchestra.run.smithi161.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-05-22T21:09:58.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-05-22T21:09:58.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-05-22T21:09:58.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-05-22T21:09:58.919 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-22T21:09:58.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-22T21:09:58.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-22T21:09:58.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-22T21:09:58.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-22T21:09:58.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-22T21:09:58.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-05-22T21:09:58.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-05-22T21:09:58.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-05-22T21:09:58.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-05-22T21:09:58.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-05-22T21:09:58.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-05-22T21:09:58.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-05-22T21:09:58.920 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-05-22T21:09:58.920 INFO:teuthology.orchestra.run.smithi161.stdout: socat xmlstarlet 2024-05-22T21:09:58.931 INFO:teuthology.orchestra.run.smithi179.stdout:0 upgraded, 115 newly installed, 0 to remove and 310 not upgraded. 2024-05-22T21:09:58.932 INFO:teuthology.orchestra.run.smithi179.stdout:Need to get 163 MB of archives. 2024-05-22T21:09:58.932 INFO:teuthology.orchestra.run.smithi179.stdout:After this operation, 745 MB of additional disk space will be used. 2024-05-22T21:09:58.932 INFO:teuthology.orchestra.run.smithi179.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-05-22T21:09:58.955 INFO:teuthology.orchestra.run.smithi179.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-05-22T21:09:58.967 INFO:teuthology.orchestra.run.smithi179.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-05-22T21:09:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:0 upgraded, 115 newly installed, 0 to remove and 310 not upgraded. 2024-05-22T21:09:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:Need to get 163 MB of archives. 2024-05-22T21:09:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:After this operation, 745 MB of additional disk space will be used. 2024-05-22T21:09:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-05-22T21:09:59.015 INFO:teuthology.orchestra.run.smithi161.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-05-22T21:09:59.028 INFO:teuthology.orchestra.run.smithi179.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-05-22T21:09:59.029 INFO:teuthology.orchestra.run.smithi161.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-05-22T21:09:59.038 INFO:teuthology.orchestra.run.smithi179.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-05-22T21:09:59.060 INFO:teuthology.orchestra.run.smithi179.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-05-22T21:09:59.063 INFO:teuthology.orchestra.run.smithi179.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-05-22T21:09:59.076 INFO:teuthology.orchestra.run.smithi179.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-05-22T21:09:59.079 INFO:teuthology.orchestra.run.smithi161.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-05-22T21:09:59.090 INFO:teuthology.orchestra.run.smithi161.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-05-22T21:09:59.110 INFO:teuthology.orchestra.run.smithi161.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-05-22T21:09:59.113 INFO:teuthology.orchestra.run.smithi161.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-05-22T21:09:59.125 INFO:teuthology.orchestra.run.smithi161.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-05-22T21:09:59.125 INFO:teuthology.orchestra.run.smithi179.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-05-22T21:09:59.129 INFO:teuthology.orchestra.run.smithi179.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-05-22T21:09:59.137 INFO:teuthology.orchestra.run.smithi179.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-05-22T21:09:59.141 INFO:teuthology.orchestra.run.smithi179.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-05-22T21:09:59.141 INFO:teuthology.orchestra.run.smithi179.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-05-22T21:09:59.149 INFO:teuthology.orchestra.run.smithi179.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-05-22T21:09:59.150 INFO:teuthology.orchestra.run.smithi179.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-05-22T21:09:59.150 INFO:teuthology.orchestra.run.smithi179.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-05-22T21:09:59.151 INFO:teuthology.orchestra.run.smithi179.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-05-22T21:09:59.158 INFO:teuthology.orchestra.run.smithi179.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-05-22T21:09:59.175 INFO:teuthology.orchestra.run.smithi179.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-05-22T21:09:59.175 INFO:teuthology.orchestra.run.smithi179.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-05-22T21:09:59.176 INFO:teuthology.orchestra.run.smithi179.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-05-22T21:09:59.176 INFO:teuthology.orchestra.run.smithi179.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-05-22T21:09:59.177 INFO:teuthology.orchestra.run.smithi179.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-05-22T21:09:59.179 INFO:teuthology.orchestra.run.smithi179.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-05-22T21:09:59.180 INFO:teuthology.orchestra.run.smithi179.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-05-22T21:09:59.180 INFO:teuthology.orchestra.run.smithi179.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-05-22T21:09:59.181 INFO:teuthology.orchestra.run.smithi179.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-05-22T21:09:59.183 INFO:teuthology.orchestra.run.smithi161.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-05-22T21:09:59.186 INFO:teuthology.orchestra.run.smithi161.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-05-22T21:09:59.194 INFO:teuthology.orchestra.run.smithi161.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-05-22T21:09:59.197 INFO:teuthology.orchestra.run.smithi161.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-05-22T21:09:59.198 INFO:teuthology.orchestra.run.smithi161.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-05-22T21:09:59.205 INFO:teuthology.orchestra.run.smithi161.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-05-22T21:09:59.206 INFO:teuthology.orchestra.run.smithi161.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-05-22T21:09:59.206 INFO:teuthology.orchestra.run.smithi161.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-05-22T21:09:59.207 INFO:teuthology.orchestra.run.smithi179.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-22T21:09:59.207 INFO:teuthology.orchestra.run.smithi161.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-05-22T21:09:59.208 INFO:teuthology.orchestra.run.smithi179.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-05-22T21:09:59.208 INFO:teuthology.orchestra.run.smithi179.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-05-22T21:09:59.209 INFO:teuthology.orchestra.run.smithi179.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-05-22T21:09:59.210 INFO:teuthology.orchestra.run.smithi179.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-05-22T21:09:59.211 INFO:teuthology.orchestra.run.smithi179.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-05-22T21:09:59.214 INFO:teuthology.orchestra.run.smithi161.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-05-22T21:09:59.216 INFO:teuthology.orchestra.run.smithi179.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-05-22T21:09:59.216 INFO:teuthology.orchestra.run.smithi179.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-05-22T21:09:59.216 INFO:teuthology.orchestra.run.smithi179.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-05-22T21:09:59.229 INFO:teuthology.orchestra.run.smithi161.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-05-22T21:09:59.229 INFO:teuthology.orchestra.run.smithi161.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-05-22T21:09:59.230 INFO:teuthology.orchestra.run.smithi161.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-05-22T21:09:59.230 INFO:teuthology.orchestra.run.smithi161.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-05-22T21:09:59.230 INFO:teuthology.orchestra.run.smithi161.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-05-22T21:09:59.231 INFO:teuthology.orchestra.run.smithi179.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-05-22T21:09:59.233 INFO:teuthology.orchestra.run.smithi161.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-05-22T21:09:59.233 INFO:teuthology.orchestra.run.smithi161.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-05-22T21:09:59.234 INFO:teuthology.orchestra.run.smithi161.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-05-22T21:09:59.238 INFO:teuthology.orchestra.run.smithi161.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-05-22T21:09:59.241 INFO:teuthology.orchestra.run.smithi179.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-05-22T21:09:59.243 INFO:teuthology.orchestra.run.smithi179.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-05-22T21:09:59.244 INFO:teuthology.orchestra.run.smithi179.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-05-22T21:09:59.245 INFO:teuthology.orchestra.run.smithi179.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-05-22T21:09:59.246 INFO:teuthology.orchestra.run.smithi179.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-05-22T21:09:59.249 INFO:teuthology.orchestra.run.smithi179.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-05-22T21:09:59.249 INFO:teuthology.orchestra.run.smithi179.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-05-22T21:09:59.257 INFO:teuthology.orchestra.run.smithi179.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-05-22T21:09:59.260 INFO:teuthology.orchestra.run.smithi179.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-05-22T21:09:59.268 INFO:teuthology.orchestra.run.smithi161.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-22T21:09:59.268 INFO:teuthology.orchestra.run.smithi161.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-05-22T21:09:59.268 INFO:teuthology.orchestra.run.smithi161.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-05-22T21:09:59.270 INFO:teuthology.orchestra.run.smithi161.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-05-22T21:09:59.270 INFO:teuthology.orchestra.run.smithi161.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-05-22T21:09:59.271 INFO:teuthology.orchestra.run.smithi161.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-05-22T21:09:59.280 INFO:teuthology.orchestra.run.smithi161.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-05-22T21:09:59.280 INFO:teuthology.orchestra.run.smithi161.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-05-22T21:09:59.281 INFO:teuthology.orchestra.run.smithi161.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-05-22T21:09:59.341 INFO:teuthology.orchestra.run.smithi161.stdout:Get:37 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.2-1349-gb9b067bc-1jammy [3,447 kB] 2024-05-22T21:09:59.343 INFO:teuthology.orchestra.run.smithi179.stdout:Get:47 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.2-1349-gb9b067bc-1jammy [3,447 kB] 2024-05-22T21:09:59.344 INFO:teuthology.orchestra.run.smithi161.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-05-22T21:09:59.413 INFO:teuthology.orchestra.run.smithi161.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-05-22T21:09:59.415 INFO:teuthology.orchestra.run.smithi161.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-05-22T21:09:59.416 INFO:teuthology.orchestra.run.smithi161.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-05-22T21:09:59.417 INFO:teuthology.orchestra.run.smithi161.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-05-22T21:09:59.417 INFO:teuthology.orchestra.run.smithi161.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-05-22T21:09:59.419 INFO:teuthology.orchestra.run.smithi161.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-05-22T21:09:59.420 INFO:teuthology.orchestra.run.smithi161.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-05-22T21:09:59.421 INFO:teuthology.orchestra.run.smithi161.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-05-22T21:09:59.423 INFO:teuthology.orchestra.run.smithi161.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-05-22T21:09:59.435 INFO:teuthology.orchestra.run.smithi179.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-05-22T21:09:59.436 INFO:teuthology.orchestra.run.smithi179.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-05-22T21:09:59.437 INFO:teuthology.orchestra.run.smithi179.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-05-22T21:09:59.438 INFO:teuthology.orchestra.run.smithi179.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-05-22T21:09:59.439 INFO:teuthology.orchestra.run.smithi179.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-05-22T21:09:59.439 INFO:teuthology.orchestra.run.smithi179.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-05-22T21:09:59.440 INFO:teuthology.orchestra.run.smithi179.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-05-22T21:09:59.442 INFO:teuthology.orchestra.run.smithi179.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-22T21:09:59.442 INFO:teuthology.orchestra.run.smithi179.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-05-22T21:09:59.472 INFO:teuthology.orchestra.run.smithi161.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-05-22T21:09:59.472 INFO:teuthology.orchestra.run.smithi179.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-05-22T21:09:59.483 INFO:teuthology.orchestra.run.smithi161.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-05-22T21:09:59.485 INFO:teuthology.orchestra.run.smithi161.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-05-22T21:09:59.486 INFO:teuthology.orchestra.run.smithi161.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-05-22T21:09:59.487 INFO:teuthology.orchestra.run.smithi161.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-05-22T21:09:59.487 INFO:teuthology.orchestra.run.smithi161.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-05-22T21:09:59.488 INFO:teuthology.orchestra.run.smithi161.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-05-22T21:09:59.491 INFO:teuthology.orchestra.run.smithi161.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-22T21:09:59.492 INFO:teuthology.orchestra.run.smithi161.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-05-22T21:09:59.501 INFO:teuthology.orchestra.run.smithi179.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-05-22T21:09:59.504 INFO:teuthology.orchestra.run.smithi179.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-05-22T21:09:59.504 INFO:teuthology.orchestra.run.smithi179.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-05-22T21:09:59.531 INFO:teuthology.orchestra.run.smithi179.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-05-22T21:09:59.532 INFO:teuthology.orchestra.run.smithi179.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-05-22T21:09:59.532 INFO:teuthology.orchestra.run.smithi179.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-05-22T21:09:59.535 INFO:teuthology.orchestra.run.smithi161.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-05-22T21:09:59.557 INFO:teuthology.orchestra.run.smithi179.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-05-22T21:09:59.620 INFO:teuthology.orchestra.run.smithi161.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-05-22T21:09:59.622 INFO:teuthology.orchestra.run.smithi161.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-05-22T21:09:59.623 INFO:teuthology.orchestra.run.smithi161.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-05-22T21:09:59.629 INFO:teuthology.orchestra.run.smithi179.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-05-22T21:09:59.630 INFO:teuthology.orchestra.run.smithi179.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-05-22T21:09:59.661 INFO:teuthology.orchestra.run.smithi179.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-05-22T21:09:59.661 INFO:teuthology.orchestra.run.smithi179.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-05-22T21:09:59.665 INFO:teuthology.orchestra.run.smithi179.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-05-22T21:09:59.669 INFO:teuthology.orchestra.run.smithi161.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-05-22T21:09:59.670 INFO:teuthology.orchestra.run.smithi161.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-05-22T21:09:59.670 INFO:teuthology.orchestra.run.smithi161.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-05-22T21:09:59.671 INFO:teuthology.orchestra.run.smithi161.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-05-22T21:09:59.671 INFO:teuthology.orchestra.run.smithi161.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-05-22T21:09:59.672 INFO:teuthology.orchestra.run.smithi161.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-05-22T21:09:59.673 INFO:teuthology.orchestra.run.smithi161.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-05-22T21:09:59.678 INFO:teuthology.orchestra.run.smithi179.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-05-22T21:09:59.680 INFO:teuthology.orchestra.run.smithi179.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-05-22T21:09:59.680 INFO:teuthology.orchestra.run.smithi179.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-05-22T21:09:59.694 INFO:teuthology.orchestra.run.smithi179.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-05-22T21:09:59.712 INFO:teuthology.orchestra.run.smithi179.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-05-22T21:09:59.712 INFO:teuthology.orchestra.run.smithi179.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-05-22T21:09:59.728 INFO:teuthology.orchestra.run.smithi161.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-05-22T21:09:59.733 INFO:teuthology.orchestra.run.smithi161.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-05-22T21:09:59.735 INFO:teuthology.orchestra.run.smithi161.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-05-22T21:09:59.736 INFO:teuthology.orchestra.run.smithi161.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-05-22T21:09:59.737 INFO:teuthology.orchestra.run.smithi161.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-05-22T21:09:59.741 INFO:teuthology.orchestra.run.smithi161.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-05-22T21:09:59.752 INFO:teuthology.orchestra.run.smithi179.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-05-22T21:09:59.804 INFO:teuthology.orchestra.run.smithi161.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-05-22T21:09:59.804 INFO:teuthology.orchestra.run.smithi161.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-05-22T21:09:59.809 INFO:teuthology.orchestra.run.smithi161.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-05-22T21:09:59.812 INFO:teuthology.orchestra.run.smithi161.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-22T21:09:59.812 INFO:teuthology.orchestra.run.smithi161.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-05-22T21:09:59.813 INFO:teuthology.orchestra.run.smithi179.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-22T21:09:59.813 INFO:teuthology.orchestra.run.smithi179.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-05-22T21:09:59.814 INFO:teuthology.orchestra.run.smithi161.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-05-22T21:09:59.815 INFO:teuthology.orchestra.run.smithi179.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-05-22T21:09:59.821 INFO:teuthology.orchestra.run.smithi179.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-05-22T21:09:59.822 INFO:teuthology.orchestra.run.smithi179.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-05-22T21:09:59.823 INFO:teuthology.orchestra.run.smithi179.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-05-22T21:09:59.832 INFO:teuthology.orchestra.run.smithi161.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-05-22T21:09:59.832 INFO:teuthology.orchestra.run.smithi161.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-05-22T21:09:59.833 INFO:teuthology.orchestra.run.smithi161.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-05-22T21:09:59.861 INFO:teuthology.orchestra.run.smithi161.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.2-1349-gb9b067bc-1jammy [3,106 kB] 2024-05-22T21:09:59.977 INFO:teuthology.orchestra.run.smithi161.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.2-1349-gb9b067bc-1jammy [680 kB] 2024-05-22T21:10:00.216 INFO:teuthology.orchestra.run.smithi161.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.2-1349-gb9b067bc-1jammy [357 kB] 2024-05-22T21:10:00.235 INFO:teuthology.orchestra.run.smithi161.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.2-1349-gb9b067bc-1jammy [32.8 kB] 2024-05-22T21:10:00.255 INFO:teuthology.orchestra.run.smithi161.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.2-1349-gb9b067bc-1jammy [180 kB] 2024-05-22T21:10:00.268 INFO:teuthology.orchestra.run.smithi161.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.2-1349-gb9b067bc-1jammy [63.7 kB] 2024-05-22T21:10:00.270 INFO:teuthology.orchestra.run.smithi161.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.2-1349-gb9b067bc-1jammy [330 kB] 2024-05-22T21:10:00.278 INFO:teuthology.orchestra.run.smithi161.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.2-1349-gb9b067bc-1jammy [5,891 kB] 2024-05-22T21:10:00.557 INFO:teuthology.orchestra.run.smithi161.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.2-1349-gb9b067bc-1jammy [112 kB] 2024-05-22T21:10:00.558 INFO:teuthology.orchestra.run.smithi161.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.2-1349-gb9b067bc-1jammy [442 kB] 2024-05-22T21:10:00.572 INFO:teuthology.orchestra.run.smithi161.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.2-1349-gb9b067bc-1jammy [21.9 MB] 2024-05-22T21:10:01.463 INFO:teuthology.orchestra.run.smithi161.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.2-1349-gb9b067bc-1jammy [5,002 kB] 2024-05-22T21:10:01.619 INFO:teuthology.orchestra.run.smithi161.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.2-1349-gb9b067bc-1jammy [241 kB] 2024-05-22T21:10:01.650 INFO:teuthology.orchestra.run.smithi161.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.2-1349-gb9b067bc-1jammy [130 kB] 2024-05-22T21:10:01.676 INFO:teuthology.orchestra.run.smithi161.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.2-1349-gb9b067bc-1jammy [1,490 kB] 2024-05-22T21:10:01.721 INFO:teuthology.orchestra.run.smithi161.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.2-1349-gb9b067bc-1jammy [6,080 kB] 2024-05-22T21:10:01.923 INFO:teuthology.orchestra.run.smithi161.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.2-1349-gb9b067bc-1jammy [22.3 MB] 2024-05-22T21:10:02.360 INFO:teuthology.orchestra.run.smithi179.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.2-1349-gb9b067bc-1jammy [3,106 kB] 2024-05-22T21:10:02.693 INFO:teuthology.orchestra.run.smithi161.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.2-1349-gb9b067bc-1jammy [14.1 kB] 2024-05-22T21:10:02.694 INFO:teuthology.orchestra.run.smithi161.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.2-1349-gb9b067bc-1jammy [831 kB] 2024-05-22T21:10:02.718 INFO:teuthology.orchestra.run.smithi161.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.2-1349-gb9b067bc-1jammy [2,155 kB] 2024-05-22T21:10:02.793 INFO:teuthology.orchestra.run.smithi161.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.2-1349-gb9b067bc-1jammy [238 kB] 2024-05-22T21:10:02.814 INFO:teuthology.orchestra.run.smithi161.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.2-1349-gb9b067bc-1jammy [145 kB] 2024-05-22T21:10:02.823 INFO:teuthology.orchestra.run.smithi161.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.2-1349-gb9b067bc-1jammy [2,237 kB] 2024-05-22T21:10:02.969 INFO:teuthology.orchestra.run.smithi161.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.2-1349-gb9b067bc-1jammy [8,625 kB] 2024-05-22T21:10:03.295 INFO:teuthology.orchestra.run.smithi161.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.2-1349-gb9b067bc-1jammy [14.3 kB] 2024-05-22T21:10:03.296 INFO:teuthology.orchestra.run.smithi161.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.2-1349-gb9b067bc-1jammy [42.6 MB] 2024-05-22T21:10:04.354 INFO:teuthology.orchestra.run.smithi179.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.2-1349-gb9b067bc-1jammy [680 kB] 2024-05-22T21:10:04.734 INFO:teuthology.orchestra.run.smithi179.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.2-1349-gb9b067bc-1jammy [357 kB] 2024-05-22T21:10:04.796 INFO:teuthology.orchestra.run.smithi161.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.2-1349-gb9b067bc-1jammy [121 kB] 2024-05-22T21:10:04.797 INFO:teuthology.orchestra.run.smithi161.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.2-1349-gb9b067bc-1jammy [230 kB] 2024-05-22T21:10:04.812 INFO:teuthology.orchestra.run.smithi161.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.2-1349-gb9b067bc-1jammy [34.1 kB] 2024-05-22T21:10:04.814 INFO:teuthology.orchestra.run.smithi161.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.2-1349-gb9b067bc-1jammy [23.5 kB] 2024-05-22T21:10:04.840 INFO:teuthology.orchestra.run.smithi179.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.2-1349-gb9b067bc-1jammy [32.8 kB] 2024-05-22T21:10:04.870 INFO:teuthology.orchestra.run.smithi179.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.2-1349-gb9b067bc-1jammy [180 kB] 2024-05-22T21:10:04.907 INFO:teuthology.orchestra.run.smithi179.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.2-1349-gb9b067bc-1jammy [63.7 kB] 2024-05-22T21:10:04.908 INFO:teuthology.orchestra.run.smithi179.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.2-1349-gb9b067bc-1jammy [330 kB] 2024-05-22T21:10:05.232 INFO:teuthology.orchestra.run.smithi161.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.2-1349-gb9b067bc-1jammy [38.3 kB] 2024-05-22T21:10:05.268 INFO:teuthology.orchestra.run.smithi161.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.2-1349-gb9b067bc-1jammy [11.6 MB] 2024-05-22T21:10:05.435 INFO:teuthology.orchestra.run.smithi179.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.2-1349-gb9b067bc-1jammy [5,891 kB] 2024-05-22T21:10:05.682 INFO:teuthology.orchestra.run.smithi161.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.2-1349-gb9b067bc-1jammy [93.1 kB] 2024-05-22T21:10:06.255 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 163 MB in 7s (24.1 MB/s) 2024-05-22T21:10:06.399 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pyperclip. 2024-05-22T21:10:06.467 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-05-22T21:10:06.470 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-05-22T21:10:06.515 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-05-22T21:10:06.817 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-22T21:10:06.831 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-05-22T21:10:06.875 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-22T21:10:07.170 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cmd2. 2024-05-22T21:10:07.183 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-05-22T21:10:07.228 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-05-22T21:10:07.556 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libfmt8:amd64. 2024-05-22T21:10:07.570 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-05-22T21:10:07.648 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-05-22T21:10:07.917 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-05-22T21:10:07.931 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-05-22T21:10:08.001 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-22T21:10:08.264 INFO:teuthology.orchestra.run.smithi179.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.2-1349-gb9b067bc-1jammy [112 kB] 2024-05-22T21:10:08.321 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-22T21:10:08.333 INFO:teuthology.orchestra.run.smithi179.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.2-1349-gb9b067bc-1jammy [442 kB] 2024-05-22T21:10:08.335 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-05-22T21:10:08.395 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-22T21:10:08.546 INFO:teuthology.orchestra.run.smithi179.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.2-1349-gb9b067bc-1jammy [21.9 MB] 2024-05-22T21:10:08.732 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-22T21:10:08.747 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-05-22T21:10:08.790 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-22T21:10:09.202 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-22T21:10:09.217 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-22T21:10:09.259 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-22T21:10:09.671 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-22T21:10:09.686 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-22T21:10:09.729 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-22T21:10:10.099 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-22T21:10:10.115 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-22T21:10:10.157 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-22T21:10:10.485 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-05-22T21:10:10.500 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-05-22T21:10:10.535 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-22T21:10:10.838 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package librados2. 2024-05-22T21:10:10.853 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../011-librados2_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:10.896 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librados2 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:11.557 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package librbd1. 2024-05-22T21:10:11.572 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../012-librbd1_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:11.615 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librbd1 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:12.034 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libcephfs2. 2024-05-22T21:10:12.049 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../013-libcephfs2_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:12.093 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libcephfs2 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:12.387 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-rados. 2024-05-22T21:10:12.402 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../014-python3-rados_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:12.436 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-rados (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:12.756 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-22T21:10:12.771 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:10:12.814 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-ceph-argparse (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:13.067 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cephfs. 2024-05-22T21:10:13.082 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../016-python3-cephfs_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:13.125 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cephfs (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:13.444 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-22T21:10:13.459 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:10:13.527 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-ceph-common (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:13.847 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-prettytable. 2024-05-22T21:10:13.862 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-05-22T21:10:13.905 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-05-22T21:10:14.158 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-rbd. 2024-05-22T21:10:14.173 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../019-python3-rbd_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:14.208 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-rbd (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:14.544 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-22T21:10:14.559 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-05-22T21:10:14.602 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-22T21:10:14.946 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package librgw2. 2024-05-22T21:10:14.961 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../021-librgw2_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:14.996 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librgw2 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:15.440 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-rgw. 2024-05-22T21:10:15.455 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../022-python3-rgw_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:15.499 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-rgw (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:15.843 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package liboath0:amd64. 2024-05-22T21:10:15.858 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2024-05-22T21:10:15.901 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-05-22T21:10:16.270 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libradosstriper1. 2024-05-22T21:10:16.286 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../024-libradosstriper1_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:16.329 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libradosstriper1 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:16.485 INFO:teuthology.orchestra.run.smithi179.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.2-1349-gb9b067bc-1jammy [5,002 kB] 2024-05-22T21:10:16.758 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-common. 2024-05-22T21:10:16.773 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../025-ceph-common_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:16.816 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-common (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:17.785 INFO:teuthology.orchestra.run.smithi179.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.2-1349-gb9b067bc-1jammy [241 kB] 2024-05-22T21:10:17.864 INFO:teuthology.orchestra.run.smithi179.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.2-1349-gb9b067bc-1jammy [130 kB] 2024-05-22T21:10:17.878 INFO:teuthology.orchestra.run.smithi179.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.2-1349-gb9b067bc-1jammy [1,490 kB] 2024-05-22T21:10:17.921 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-base. 2024-05-22T21:10:17.936 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../026-ceph-base_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:17.988 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-base (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:18.293 INFO:teuthology.orchestra.run.smithi179.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.2-1349-gb9b067bc-1jammy [6,080 kB] 2024-05-22T21:10:18.533 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-05-22T21:10:18.547 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-05-22T21:10:18.583 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-05-22T21:10:18.844 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cheroot. 2024-05-22T21:10:18.858 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-05-22T21:10:18.903 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-22T21:10:19.264 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-05-22T21:10:19.278 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-05-22T21:10:19.322 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-05-22T21:10:19.625 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-jaraco.text. 2024-05-22T21:10:19.640 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2024-05-22T21:10:19.683 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-05-22T21:10:19.961 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-05-22T21:10:19.976 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-05-22T21:10:20.010 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-05-22T21:10:20.045 INFO:teuthology.orchestra.run.smithi179.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.2-1349-gb9b067bc-1jammy [22.3 MB] 2024-05-22T21:10:20.297 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-tempora. 2024-05-22T21:10:20.311 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2024-05-22T21:10:20.392 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-05-22T21:10:20.716 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-portend. 2024-05-22T21:10:20.731 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2024-05-22T21:10:20.774 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-05-22T21:10:21.077 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-05-22T21:10:21.092 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2024-05-22T21:10:21.135 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-05-22T21:10:21.580 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-22T21:10:21.595 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2024-05-22T21:10:21.629 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-05-22T21:10:22.017 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-natsort. 2024-05-22T21:10:22.031 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2024-05-22T21:10:22.075 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-05-22T21:10:22.344 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-logutils. 2024-05-22T21:10:22.359 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2024-05-22T21:10:22.394 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-05-22T21:10:22.672 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-22T21:10:22.687 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-05-22T21:10:22.739 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-05-22T21:10:23.041 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-mako. 2024-05-22T21:10:23.056 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-05-22T21:10:23.099 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-22T21:10:23.461 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-22T21:10:23.476 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-22T21:10:23.519 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-22T21:10:23.780 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-22T21:10:23.796 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-05-22T21:10:23.839 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-05-22T21:10:24.141 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-webob. 2024-05-22T21:10:24.156 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-05-22T21:10:24.199 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-05-22T21:10:24.561 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-waitress. 2024-05-22T21:10:24.576 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-05-22T21:10:24.622 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-22T21:10:24.930 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-tempita. 2024-05-22T21:10:24.945 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-05-22T21:10:24.988 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-05-22T21:10:25.250 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-paste. 2024-05-22T21:10:25.265 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-05-22T21:10:25.299 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-05-22T21:10:25.485 INFO:teuthology.orchestra.run.smithi179.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.2-1349-gb9b067bc-1jammy [14.1 kB] 2024-05-22T21:10:25.485 INFO:teuthology.orchestra.run.smithi179.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.2-1349-gb9b067bc-1jammy [831 kB] 2024-05-22T21:10:25.627 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-22T21:10:25.637 INFO:teuthology.orchestra.run.smithi179.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.2-1349-gb9b067bc-1jammy [2,155 kB] 2024-05-22T21:10:25.642 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-05-22T21:10:25.685 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-05-22T21:10:25.946 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-22T21:10:25.961 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2024-05-22T21:10:25.990 INFO:teuthology.orchestra.run.smithi179.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.2-1349-gb9b067bc-1jammy [238 kB] 2024-05-22T21:10:25.996 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-05-22T21:10:26.032 INFO:teuthology.orchestra.run.smithi179.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.2-1349-gb9b067bc-1jammy [145 kB] 2024-05-22T21:10:26.039 INFO:teuthology.orchestra.run.smithi179.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.2-1349-gb9b067bc-1jammy [2,237 kB] 2024-05-22T21:10:26.274 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-webtest. 2024-05-22T21:10:26.289 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2024-05-22T21:10:26.332 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-05-22T21:10:26.413 INFO:teuthology.orchestra.run.smithi179.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.2-1349-gb9b067bc-1jammy [8,625 kB] 2024-05-22T21:10:26.627 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pecan. 2024-05-22T21:10:26.642 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-05-22T21:10:26.685 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-05-22T21:10:27.025 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-certifi. 2024-05-22T21:10:27.040 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2024-05-22T21:10:27.083 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-05-22T21:10:27.420 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-idna. 2024-05-22T21:10:27.435 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../051-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-05-22T21:10:27.469 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-05-22T21:10:27.602 INFO:teuthology.orchestra.run.smithi179.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.2-1349-gb9b067bc-1jammy [14.3 kB] 2024-05-22T21:10:27.602 INFO:teuthology.orchestra.run.smithi179.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.2-1349-gb9b067bc-1jammy [42.6 MB] 2024-05-22T21:10:27.772 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-urllib3. 2024-05-22T21:10:27.787 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-05-22T21:10:27.830 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-22T21:10:28.126 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-requests. 2024-05-22T21:10:28.140 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-05-22T21:10:28.175 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-22T21:10:28.461 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-22T21:10:28.476 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-05-22T21:10:28.519 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-22T21:10:28.831 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-22T21:10:28.846 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:10:28.881 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-modules-core (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:29.269 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-22T21:10:29.283 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:29.327 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libsqlite3-mod-ceph (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:29.622 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-22T21:10:29.636 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../057-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-05-22T21:10:29.671 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-05-22T21:10:29.941 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-openssl. 2024-05-22T21:10:29.956 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../058-python3-openssl_21.0.0-1_all.deb ... 2024-05-22T21:10:30.026 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-05-22T21:10:30.319 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr. 2024-05-22T21:10:30.334 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../059-ceph-mgr_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:30.367 INFO:teuthology.orchestra.run.smithi179.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.2-1349-gb9b067bc-1jammy [121 kB] 2024-05-22T21:10:30.368 INFO:teuthology.orchestra.run.smithi179.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.2-1349-gb9b067bc-1jammy [230 kB] 2024-05-22T21:10:30.368 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:30.370 INFO:teuthology.orchestra.run.smithi179.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.2-1349-gb9b067bc-1jammy [34.1 kB] 2024-05-22T21:10:30.371 INFO:teuthology.orchestra.run.smithi179.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.2-1349-gb9b067bc-1jammy [23.5 kB] 2024-05-22T21:10:30.371 INFO:teuthology.orchestra.run.smithi179.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.2-1349-gb9b067bc-1jammy [38.3 kB] 2024-05-22T21:10:30.377 INFO:teuthology.orchestra.run.smithi179.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.2-1349-gb9b067bc-1jammy [11.6 MB] 2024-05-22T21:10:30.762 INFO:teuthology.orchestra.run.smithi179.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/reef/b9b067bc6d30ec00e391f0fa43f50affe981ba2f/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.2-1349-gb9b067bc-1jammy [93.1 kB] 2024-05-22T21:10:30.764 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mon. 2024-05-22T21:10:30.779 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../060-ceph-mon_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:30.822 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mon (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:31.327 INFO:teuthology.orchestra.run.smithi179.stdout:Fetched 163 MB in 32s (5,097 kB/s) 2024-05-22T21:10:32.226 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pyperclip. 2024-05-22T21:10:32.268 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-osd. 2024-05-22T21:10:32.283 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../061-ceph-osd_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:32.298 INFO:teuthology.orchestra.run.smithi179.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-05-22T21:10:32.302 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-05-22T21:10:32.326 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-osd (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:32.350 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-05-22T21:10:32.819 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-22T21:10:32.833 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-05-22T21:10:32.877 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-22T21:10:33.112 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph. 2024-05-22T21:10:33.127 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../062-ceph_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:33.163 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-cmd2. 2024-05-22T21:10:33.170 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:33.180 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-05-22T21:10:33.213 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-05-22T21:10:33.483 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libfmt8:amd64. 2024-05-22T21:10:33.490 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-fuse. 2024-05-22T21:10:33.498 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-05-22T21:10:33.505 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../063-ceph-fuse_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:33.540 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-fuse (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:33.575 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-05-22T21:10:33.902 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mds. 2024-05-22T21:10:33.917 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../064-ceph-mds_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:33.944 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-05-22T21:10:33.959 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-05-22T21:10:33.960 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mds (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:34.002 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-22T21:10:34.338 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package cephadm. 2024-05-22T21:10:34.353 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../065-cephadm_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:34.364 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-22T21:10:34.380 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-05-22T21:10:34.396 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking cephadm (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:34.422 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-22T21:10:34.724 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-22T21:10:34.725 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-22T21:10:34.739 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../066-python3-asyncssh_2.5.0-1_all.deb ... 2024-05-22T21:10:34.741 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-05-22T21:10:34.774 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-05-22T21:10:34.775 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-22T21:10:35.102 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python-babel-localedata. 2024-05-22T21:10:35.117 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../067-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-05-22T21:10:35.160 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-22T21:10:35.229 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-22T21:10:35.244 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-22T21:10:35.295 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-22T21:10:35.699 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-22T21:10:35.714 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-22T21:10:35.757 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-22T21:10:36.077 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-22T21:10:36.091 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-22T21:10:36.127 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-22T21:10:36.190 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-babel. 2024-05-22T21:10:36.204 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../068-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-05-22T21:10:36.248 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-05-22T21:10:36.463 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-05-22T21:10:36.478 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-05-22T21:10:36.522 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-22T21:10:36.560 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-jinja2. 2024-05-22T21:10:36.574 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../069-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-05-22T21:10:36.618 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-22T21:10:36.866 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package librados2. 2024-05-22T21:10:36.881 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../011-librados2_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:36.924 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking librados2 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:36.971 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-22T21:10:36.985 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../070-ceph-mgr-cephadm_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:10:37.029 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-cephadm (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:37.335 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package librbd1. 2024-05-22T21:10:37.350 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../012-librbd1_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:37.385 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking librbd1 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:37.417 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pluggy. 2024-05-22T21:10:37.431 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2024-05-22T21:10:37.475 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-05-22T21:10:37.779 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libcephfs2. 2024-05-22T21:10:37.786 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-py. 2024-05-22T21:10:37.794 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../013-libcephfs2_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:37.801 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2024-05-22T21:10:37.836 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-py (1.10.0-1) ... 2024-05-22T21:10:37.838 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libcephfs2 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:38.134 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-iniconfig. 2024-05-22T21:10:38.140 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-rados. 2024-05-22T21:10:38.145 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2024-05-22T21:10:38.155 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../014-python3-rados_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:38.181 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-05-22T21:10:38.199 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-rados (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:38.467 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-toml. 2024-05-22T21:10:38.482 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2024-05-22T21:10:38.518 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-22T21:10:38.525 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-05-22T21:10:38.533 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:10:38.576 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-ceph-argparse (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:38.812 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-cephfs. 2024-05-22T21:10:38.827 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../016-python3-cephfs_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:38.853 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pytest. 2024-05-22T21:10:38.862 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-cephfs (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:38.868 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-05-22T21:10:38.911 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-05-22T21:10:39.173 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-22T21:10:39.187 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:10:39.231 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-ceph-common (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:39.248 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-22T21:10:39.263 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-05-22T21:10:39.307 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-22T21:10:39.559 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-prettytable. 2024-05-22T21:10:39.574 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-05-22T21:10:39.610 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-routes. 2024-05-22T21:10:39.625 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-05-22T21:10:39.642 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-05-22T21:10:39.668 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-05-22T21:10:39.904 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-rbd. 2024-05-22T21:10:39.918 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../019-python3-rbd_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:39.962 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-rbd (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:40.013 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-22T21:10:40.029 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:10:40.107 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-dashboard (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:40.298 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-22T21:10:40.313 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-05-22T21:10:40.348 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-22T21:10:40.651 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package librgw2. 2024-05-22T21:10:40.666 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../021-librgw2_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:40.709 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking librgw2 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:41.211 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-rgw. 2024-05-22T21:10:41.226 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../022-python3-rgw_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:41.269 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-rgw (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:41.326 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-05-22T21:10:41.340 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-05-22T21:10:41.385 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-22T21:10:41.614 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package liboath0:amd64. 2024-05-22T21:10:41.629 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2024-05-22T21:10:41.664 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-05-22T21:10:41.888 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-joblib. 2024-05-22T21:10:41.901 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-05-22T21:10:41.929 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-05-22T21:10:41.958 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libradosstriper1. 2024-05-22T21:10:41.972 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../024-libradosstriper1_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:42.018 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libradosstriper1 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:42.387 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-common. 2024-05-22T21:10:42.401 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../025-ceph-common_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:42.436 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-common (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:42.500 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-05-22T21:10:42.513 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-05-22T21:10:42.558 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-05-22T21:10:42.894 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-sklearn. 2024-05-22T21:10:42.907 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-05-22T21:10:42.952 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-22T21:10:43.510 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-base. 2024-05-22T21:10:43.517 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../026-ceph-base_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:43.568 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-base (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:43.646 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-22T21:10:43.662 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:10:43.704 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:44.139 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-05-22T21:10:44.147 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-05-22T21:10:44.189 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-05-22T21:10:44.491 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-cheroot. 2024-05-22T21:10:44.499 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-05-22T21:10:44.541 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-22T21:10:44.724 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cachetools. 2024-05-22T21:10:44.740 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-05-22T21:10:44.782 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-05-22T21:10:44.852 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-05-22T21:10:44.860 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-05-22T21:10:44.902 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-05-22T21:10:45.052 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-22T21:10:45.068 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-05-22T21:10:45.102 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-05-22T21:10:45.197 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-jaraco.text. 2024-05-22T21:10:45.205 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2024-05-22T21:10:45.247 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-05-22T21:10:45.389 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-22T21:10:45.404 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-05-22T21:10:45.447 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-05-22T21:10:45.542 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-05-22T21:10:45.549 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-05-22T21:10:45.625 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-05-22T21:10:45.750 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-rsa. 2024-05-22T21:10:45.765 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-05-22T21:10:45.808 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-rsa (4.8-1) ... 2024-05-22T21:10:45.927 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-tempora. 2024-05-22T21:10:45.936 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2024-05-22T21:10:45.977 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-05-22T21:10:46.129 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-google-auth. 2024-05-22T21:10:46.144 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-05-22T21:10:46.195 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-05-22T21:10:46.272 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-portend. 2024-05-22T21:10:46.284 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2024-05-22T21:10:46.314 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-05-22T21:10:46.582 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-22T21:10:46.597 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-05-22T21:10:46.600 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-05-22T21:10:46.609 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2024-05-22T21:10:46.648 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-22T21:10:46.650 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-05-22T21:10:46.953 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-22T21:10:46.962 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2024-05-22T21:10:47.002 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-05-22T21:10:47.051 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-websocket. 2024-05-22T21:10:47.067 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-05-22T21:10:47.117 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-05-22T21:10:47.356 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-natsort. 2024-05-22T21:10:47.366 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2024-05-22T21:10:47.406 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-05-22T21:10:47.421 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-22T21:10:47.436 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-05-22T21:10:47.471 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-22T21:10:47.701 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-logutils. 2024-05-22T21:10:47.711 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2024-05-22T21:10:47.751 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-05-22T21:10:48.037 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-22T21:10:48.046 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-05-22T21:10:48.087 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-05-22T21:10:48.143 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-22T21:10:48.157 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:10:48.201 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-k8sevents (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:48.487 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-22T21:10:48.502 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-05-22T21:10:48.537 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-22T21:10:48.590 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-mako. 2024-05-22T21:10:48.600 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-05-22T21:10:48.631 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-22T21:10:48.865 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-22T21:10:48.879 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-05-22T21:10:48.918 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-22T21:10:48.928 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-22T21:10:48.940 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-22T21:10:48.959 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-22T21:10:49.218 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package jq. 2024-05-22T21:10:49.233 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-05-22T21:10:49.262 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-22T21:10:49.272 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-05-22T21:10:49.276 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-05-22T21:10:49.312 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-05-22T21:10:49.520 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package socat. 2024-05-22T21:10:49.535 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-05-22T21:10:49.570 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-05-22T21:10:49.615 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-webob. 2024-05-22T21:10:49.625 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-05-22T21:10:49.665 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-05-22T21:10:49.924 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package xmlstarlet. 2024-05-22T21:10:49.938 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-05-22T21:10:49.982 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-05-22T21:10:50.060 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-waitress. 2024-05-22T21:10:50.069 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-05-22T21:10:50.112 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-22T21:10:50.336 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-test. 2024-05-22T21:10:50.350 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../098-ceph-test_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:50.394 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-test (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:50.404 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-tempita. 2024-05-22T21:10:50.414 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-05-22T21:10:50.454 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-05-22T21:10:50.732 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-paste. 2024-05-22T21:10:50.742 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-05-22T21:10:50.782 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-05-22T21:10:51.068 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-22T21:10:51.077 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-05-22T21:10:51.118 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-05-22T21:10:51.421 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-22T21:10:51.430 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2024-05-22T21:10:51.470 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-05-22T21:10:51.815 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-webtest. 2024-05-22T21:10:51.825 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2024-05-22T21:10:51.866 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-05-22T21:10:52.035 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-volume. 2024-05-22T21:10:52.045 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../099-ceph-volume_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:10:52.081 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-volume (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:52.168 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pecan. 2024-05-22T21:10:52.178 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-05-22T21:10:52.210 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-05-22T21:10:52.540 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-certifi. 2024-05-22T21:10:52.549 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2024-05-22T21:10:52.581 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-05-22T21:10:52.728 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package cephfs-mirror. 2024-05-22T21:10:52.741 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:52.786 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking cephfs-mirror (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:52.892 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-idna. 2024-05-22T21:10:52.901 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../051-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-05-22T21:10:52.942 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-05-22T21:10:53.105 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-colorama. 2024-05-22T21:10:53.117 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2024-05-22T21:10:53.155 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-05-22T21:10:53.270 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-urllib3. 2024-05-22T21:10:53.279 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-05-22T21:10:53.321 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-22T21:10:53.416 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package cephfs-shell. 2024-05-22T21:10:53.430 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../102-cephfs-shell_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:10:53.474 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking cephfs-shell (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:53.633 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-requests. 2024-05-22T21:10:53.642 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-05-22T21:10:53.728 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package cephfs-top. 2024-05-22T21:10:53.739 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../103-cephfs-top_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:10:53.777 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking cephfs-top (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:53.809 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-22T21:10:54.022 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-22T21:10:54.034 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:54.063 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libcephfs-dev (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:54.078 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-22T21:10:54.087 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-05-22T21:10:54.128 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-22T21:10:54.398 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-22T21:10:54.407 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:10:54.425 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package nvme-cli. 2024-05-22T21:10:54.437 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-05-22T21:10:54.465 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr-modules-core (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:54.474 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-05-22T21:10:54.813 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-22T21:10:54.825 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-05-22T21:10:54.854 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-22T21:10:54.862 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:54.862 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-05-22T21:10:54.903 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libsqlite3-mod-ceph (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:55.206 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-22T21:10:55.215 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../057-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-05-22T21:10:55.256 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-05-22T21:10:55.324 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pastescript. 2024-05-22T21:10:55.339 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-05-22T21:10:55.382 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-22T21:10:55.535 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-openssl. 2024-05-22T21:10:55.543 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../058-python3-openssl_21.0.0-1_all.deb ... 2024-05-22T21:10:55.585 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-05-22T21:10:55.661 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-psutil. 2024-05-22T21:10:55.674 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-05-22T21:10:55.710 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-05-22T21:10:55.938 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr. 2024-05-22T21:10:55.947 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../059-ceph-mgr_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:55.988 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:56.038 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pygments. 2024-05-22T21:10:56.052 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-05-22T21:10:56.088 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-05-22T21:10:56.342 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mon. 2024-05-22T21:10:56.350 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../060-ceph-mon_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:56.383 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mon (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:56.525 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-22T21:10:56.537 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-05-22T21:10:56.575 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-05-22T21:10:56.886 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-simplejson. 2024-05-22T21:10:56.899 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-05-22T21:10:56.937 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-osd. 2024-05-22T21:10:56.944 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-05-22T21:10:56.945 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../061-ceph-osd_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:56.987 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-osd (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:57.214 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-22T21:10:57.227 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-05-22T21:10:57.272 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-05-22T21:10:57.858 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph. 2024-05-22T21:10:57.867 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../062-ceph_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:57.958 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:57.959 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package radosgw. 2024-05-22T21:10:57.971 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../113-radosgw_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:58.017 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking radosgw (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:58.219 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-fuse. 2024-05-22T21:10:58.228 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../063-ceph-fuse_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:58.261 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-fuse (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:58.596 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package rbd-fuse. 2024-05-22T21:10:58.606 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mds. 2024-05-22T21:10:58.609 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../114-rbd-fuse_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:58.615 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../064-ceph-mds_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:58.645 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking rbd-fuse (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:58.656 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mds (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-05-22T21:10:59.060 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package cephadm. 2024-05-22T21:10:59.068 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../065-cephadm_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:10:59.109 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking cephadm (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:10:59.225 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-22T21:10:59.360 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-05-22T21:10:59.371 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-22T21:10:59.380 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../066-python3-asyncssh_2.5.0-1_all.deb ... 2024-05-22T21:10:59.421 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-05-22T21:10:59.648 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-05-22T21:10:59.707 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python-babel-localedata. 2024-05-22T21:10:59.715 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../067-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-05-22T21:10:59.757 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-22T21:11:00.062 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-05-22T21:11:00.686 INFO:teuthology.orchestra.run.smithi161.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-05-22T21:11:00.730 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-babel. 2024-05-22T21:11:00.730 INFO:teuthology.orchestra.run.smithi161.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-05-22T21:11:00.737 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../068-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-05-22T21:11:00.771 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-05-22T21:11:00.808 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up cephadm (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:01.091 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-jinja2. 2024-05-22T21:11:01.101 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../069-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-05-22T21:11:01.141 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-22T21:11:01.330 INFO:teuthology.orchestra.run.smithi161.stdout:Adding system user cephadm....done 2024-05-22T21:11:01.462 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-22T21:11:01.470 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../070-ceph-mgr-cephadm_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:11:01.483 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-22T21:11:01.512 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr-cephadm (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:01.768 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-05-22T21:11:01.849 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pluggy. 2024-05-22T21:11:01.858 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2024-05-22T21:11:01.907 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-05-22T21:11:02.028 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-05-22T21:11:02.137 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-05-22T21:11:02.211 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-py. 2024-05-22T21:11:02.220 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2024-05-22T21:11:02.261 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-py (1.10.0-1) ... 2024-05-22T21:11:02.422 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-22T21:11:02.598 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-iniconfig. 2024-05-22T21:11:02.608 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2024-05-22T21:11:02.648 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-05-22T21:11:02.831 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-05-22T21:11:02.951 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-toml. 2024-05-22T21:11:02.957 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-py (1.10.0-1) ... 2024-05-22T21:11:02.961 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2024-05-22T21:11:03.001 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-05-22T21:11:03.263 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pytest. 2024-05-22T21:11:03.272 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-05-22T21:11:03.276 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-05-22T21:11:03.312 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-05-22T21:11:03.653 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-05-22T21:11:03.658 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-22T21:11:03.668 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-05-22T21:11:03.708 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-22T21:11:03.904 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-05-22T21:11:04.145 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-routes. 2024-05-22T21:11:04.146 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-05-22T21:11:04.155 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-05-22T21:11:04.195 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-05-22T21:11:04.423 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-05-22T21:11:04.516 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-22T21:11:04.525 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:11:04.565 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr-dashboard (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:04.682 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-05-22T21:11:04.958 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-05-22T21:11:05.234 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-ceph-argparse (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:05.503 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-22T21:11:05.629 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-05-22T21:11:05.821 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-05-22T21:11:05.830 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-05-22T21:11:05.863 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-22T21:11:05.863 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-05-22T21:11:06.122 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-22T21:11:06.248 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-05-22T21:11:06.392 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-joblib. 2024-05-22T21:11:06.401 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-05-22T21:11:06.450 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-05-22T21:11:06.633 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-22T21:11:06.758 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-05-22T21:11:06.806 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-05-22T21:11:06.812 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-05-22T21:11:06.854 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-05-22T21:11:07.035 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-05-22T21:11:07.124 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-sklearn. 2024-05-22T21:11:07.133 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-05-22T21:11:07.173 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-22T21:11:07.294 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-05-22T21:11:07.595 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-05-22T21:11:07.935 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-22T21:11:07.944 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:11:07.985 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:08.306 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-05-22T21:11:08.573 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-05-22T21:11:08.699 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-05-22T21:11:08.808 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-22T21:11:08.947 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-cachetools. 2024-05-22T21:11:08.957 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-05-22T21:11:08.997 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-05-22T21:11:09.126 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-05-22T21:11:09.300 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-22T21:11:09.310 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-05-22T21:11:09.350 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-05-22T21:11:09.519 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-05-22T21:11:09.713 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-22T21:11:09.726 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-05-22T21:11:09.770 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-05-22T21:11:09.771 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-05-22T21:11:10.083 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-rsa. 2024-05-22T21:11:10.094 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-05-22T21:11:10.132 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-rsa (4.8-1) ... 2024-05-22T21:11:10.147 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-22T21:11:10.395 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-google-auth. 2024-05-22T21:11:10.406 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-05-22T21:11:10.445 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-05-22T21:11:10.456 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-05-22T21:11:10.740 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-22T21:11:10.751 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-05-22T21:11:10.782 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-22T21:11:10.790 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-22T21:11:11.127 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-05-22T21:11:11.135 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-websocket. 2024-05-22T21:11:11.146 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-05-22T21:11:11.193 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-05-22T21:11:11.370 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-05-22T21:11:11.496 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-ceph-common (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:11.513 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-22T21:11:11.524 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-05-22T21:11:11.563 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-22T21:11:11.822 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-05-22T21:11:12.173 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-22T21:11:12.211 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-22T21:11:12.221 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:11:12.261 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr-k8sevents (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:12.614 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-22T21:11:12.624 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-05-22T21:11:12.664 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-22T21:11:13.001 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-22T21:11:13.011 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-05-22T21:11:13.050 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-22T21:11:13.287 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package jq. 2024-05-22T21:11:13.297 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-05-22T21:11:13.337 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-05-22T21:11:13.396 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-22T21:11:13.530 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-toml (0.10.2-1) ... 2024-05-22T21:11:13.590 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package socat. 2024-05-22T21:11:13.600 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-05-22T21:11:13.640 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-05-22T21:11:13.807 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-22T21:11:13.932 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-05-22T21:11:13.977 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package xmlstarlet. 2024-05-22T21:11:13.987 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-05-22T21:11:14.027 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-05-22T21:11:14.033 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-22T21:11:14.318 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-05-22T21:11:14.382 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-test. 2024-05-22T21:11:14.392 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../098-ceph-test_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:11:14.431 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-test (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:14.594 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-05-22T21:11:14.862 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-22T21:11:14.988 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-05-22T21:11:15.130 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-05-22T21:11:15.440 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-05-22T21:11:15.700 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-05-22T21:11:15.967 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-05-22T21:11:16.145 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-volume. 2024-05-22T21:11:16.156 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../099-ceph-volume_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:11:16.203 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-volume (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:16.226 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-22T21:11:16.495 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-05-22T21:11:16.650 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package cephfs-mirror. 2024-05-22T21:11:16.661 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:11:16.707 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking cephfs-mirror (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:16.754 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-22T21:11:16.880 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-05-22T21:11:17.037 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-colorama. 2024-05-22T21:11:17.048 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2024-05-22T21:11:17.104 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-05-22T21:11:17.164 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-22T21:11:17.310 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-22T21:11:17.389 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package cephfs-shell. 2024-05-22T21:11:17.401 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../102-cephfs-shell_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:11:17.448 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking cephfs-shell (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:17.608 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-05-22T21:11:17.759 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package cephfs-top. 2024-05-22T21:11:17.770 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../103-cephfs-top_18.2.2-1349-gb9b067bc-1jammy_all.deb ... 2024-05-22T21:11:17.817 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking cephfs-top (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:17.901 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-05-22T21:11:18.104 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-22T21:11:18.115 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:11:18.154 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libcephfs-dev (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:18.187 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-05-22T21:11:18.599 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package nvme-cli. 2024-05-22T21:11:18.610 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-05-22T21:11:18.657 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-05-22T21:11:18.749 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-05-22T21:11:19.013 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-22T21:11:19.037 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-22T21:11:19.048 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-05-22T21:11:19.097 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-05-22T21:11:19.139 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-05-22T21:11:19.540 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-05-22T21:11:19.792 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-portend (3.0.0-1) ... 2024-05-22T21:11:19.799 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pastescript. 2024-05-22T21:11:19.811 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-05-22T21:11:19.849 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-22T21:11:20.064 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-22T21:11:20.185 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-22T21:11:20.211 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-psutil. 2024-05-22T21:11:20.222 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-05-22T21:11:20.261 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-05-22T21:11:20.512 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-05-22T21:11:20.589 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pygments. 2024-05-22T21:11:20.601 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-05-22T21:11:20.639 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-05-22T21:11:20.821 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-22T21:11:21.097 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-05-22T21:11:21.110 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-22T21:11:21.120 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-05-22T21:11:21.160 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-05-22T21:11:21.223 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-rsa (4.8-1) ... 2024-05-22T21:11:21.463 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-simplejson. 2024-05-22T21:11:21.474 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-05-22T21:11:21.499 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-05-22T21:11:21.513 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-05-22T21:11:21.759 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-22T21:11:21.816 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-22T21:11:21.827 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-05-22T21:11:21.866 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-05-22T21:11:22.018 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-05-22T21:11:22.403 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-22T21:11:22.571 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package radosgw. 2024-05-22T21:11:22.581 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../113-radosgw_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:11:22.612 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking radosgw (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:22.696 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-05-22T21:11:23.056 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-22T21:11:23.182 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librados2 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:23.183 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package rbd-fuse. 2024-05-22T21:11:23.195 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../114-rbd-fuse_18.2.2-1349-gb9b067bc-1jammy_amd64.deb ... 2024-05-22T21:11:23.232 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking rbd-fuse (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:23.284 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librgw2 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:23.495 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libsqlite3-mod-ceph (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:23.506 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-05-22T21:11:23.621 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libcephfs2 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:23.747 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libradosstriper1 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:23.787 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-22T21:11:23.873 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-05-22T21:11:23.913 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-05-22T21:11:24.166 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-05-22T21:11:24.203 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librbd1 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:24.334 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-modules-core (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:24.435 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-fuse (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:24.560 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-05-22T21:11:24.629 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-22T21:11:24.629 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-22T21:11:25.188 INFO:teuthology.orchestra.run.smithi179.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-05-22T21:11:25.201 INFO:teuthology.orchestra.run.smithi179.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-05-22T21:11:25.258 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up cephadm (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:25.312 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libcephfs-dev (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:25.438 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-rados (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:25.581 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up cephfs-top (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:25.724 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-22T21:11:25.927 INFO:teuthology.orchestra.run.smithi179.stdout:Adding system user cephadm....done 2024-05-22T21:11:26.090 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-22T21:11:26.366 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-05-22T21:11:26.626 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-05-22T21:11:26.752 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-05-22T21:11:27.009 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-rbd (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:27.012 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-22T21:11:27.118 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up rbd-fuse (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:27.228 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-rgw (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:27.280 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-05-22T21:11:27.354 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cephfs (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:27.406 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-py (1.10.0-1) ... 2024-05-22T21:11:27.480 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up cephfs-shell (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:27.623 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-common (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:27.716 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-05-22T21:11:28.085 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-05-22T21:11:28.117 INFO:teuthology.orchestra.run.smithi161.stdout:Adding group ceph....done 2024-05-22T21:11:28.354 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-05-22T21:11:28.527 INFO:teuthology.orchestra.run.smithi161.stdout:Adding system user ceph....done 2024-05-22T21:11:28.613 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-05-22T21:11:28.721 INFO:teuthology.orchestra.run.smithi161.stdout:Setting system user ceph properties....done 2024-05-22T21:11:28.738 INFO:teuthology.orchestra.run.smithi161.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-22T21:11:28.880 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-22T21:11:28.881 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-05-22T21:11:29.124 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-05-22T21:11:29.290 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-22T21:11:29.392 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-05-22T21:11:29.660 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-ceph-argparse (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:29.928 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-22T21:11:30.037 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-05-22T21:11:30.083 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up cephfs-mirror (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:30.297 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-05-22T21:11:30.556 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-22T21:11:30.589 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-22T21:11:30.590 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-22T21:11:30.674 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-05-22T21:11:31.042 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-22T21:11:31.168 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-05-22T21:11:31.235 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-test (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:31.335 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up radosgw (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:31.445 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-05-22T21:11:31.721 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-05-22T21:11:31.876 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-22T21:11:31.876 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-22T21:11:32.022 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-05-22T21:11:32.487 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-base (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:32.699 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-05-22T21:11:32.844 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-22T21:11:32.976 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-05-22T21:11:33.102 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-05-22T21:11:33.221 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-22T21:11:33.531 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-05-22T21:11:33.733 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mds (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:33.916 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-05-22T21:11:33.997 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-22T21:11:33.997 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-22T21:11:34.175 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-05-22T21:11:34.560 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-22T21:11:34.652 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:34.865 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-22T21:11:34.866 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-22T21:11:34.870 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-05-22T21:11:35.197 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-22T21:11:35.540 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-05-22T21:11:35.569 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-osd (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:35.799 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-05-22T21:11:35.884 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-22T21:11:35.884 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-22T21:11:35.926 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-ceph-common (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:36.227 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-05-22T21:11:36.521 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-k8sevents (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:36.578 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-22T21:11:36.681 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-diskprediction-local (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:36.823 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mon (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:37.036 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-22T21:11:37.036 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-22T21:11:37.691 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-cephadm (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:37.801 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-22T21:11:37.825 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:37.944 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-dashboard (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:37.961 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-toml (0.10.2-1) ... 2024-05-22T21:11:38.069 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-volume (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:38.220 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-22T21:11:38.339 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-05-22T21:11:38.430 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-22T21:11:38.464 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-22T21:11:38.766 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-05-22T21:11:39.051 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-05-22T21:11:39.310 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-22T21:11:39.426 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-22T21:11:39.461 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-05-22T21:11:39.579 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-05-22T21:11:39.873 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-05-22T21:11:39.928 INFO:teuthology.orchestra.run.smithi161.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-22T21:11:39.933 DEBUG:teuthology.orchestra.run.smithi161:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-05-22T21:11:40.008 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2024-05-22T21:11:40.124 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-05-22T21:11:40.196 INFO:teuthology.orchestra.run.smithi161.stdout:Building dependency tree... 2024-05-22T21:11:40.197 INFO:teuthology.orchestra.run.smithi161.stdout:Reading state information... 2024-05-22T21:11:40.381 INFO:teuthology.orchestra.run.smithi161.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-05-22T21:11:40.381 INFO:teuthology.orchestra.run.smithi161.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-05-22T21:11:40.381 INFO:teuthology.orchestra.run.smithi161.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-05-22T21:11:40.381 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages were automatically installed and are no longer required: 2024-05-22T21:11:40.381 INFO:teuthology.orchestra.run.smithi161.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-22T21:11:40.381 INFO:teuthology.orchestra.run.smithi161.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-22T21:11:40.381 INFO:teuthology.orchestra.run.smithi161.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-22T21:11:40.382 INFO:teuthology.orchestra.run.smithi161.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-22T21:11:40.382 INFO:teuthology.orchestra.run.smithi161.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-22T21:11:40.382 INFO:teuthology.orchestra.run.smithi161.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-22T21:11:40.383 INFO:teuthology.orchestra.run.smithi161.stdout:The following additional packages will be installed: 2024-05-22T21:11:40.383 INFO:teuthology.orchestra.run.smithi161.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-05-22T21:11:40.383 INFO:teuthology.orchestra.run.smithi161.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-05-22T21:11:40.384 INFO:teuthology.orchestra.run.smithi161.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-05-22T21:11:40.384 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-05-22T21:11:40.384 INFO:teuthology.orchestra.run.smithi161.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-22T21:11:40.385 INFO:teuthology.orchestra.run.smithi161.stdout:Suggested packages: 2024-05-22T21:11:40.385 INFO:teuthology.orchestra.run.smithi161.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-05-22T21:11:40.385 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-05-22T21:11:40.385 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-05-22T21:11:40.400 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-05-22T21:11:40.439 INFO:teuthology.orchestra.run.smithi161.stdout:The following NEW packages will be installed: 2024-05-22T21:11:40.440 INFO:teuthology.orchestra.run.smithi161.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-05-22T21:11:40.440 INFO:teuthology.orchestra.run.smithi161.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-05-22T21:11:40.440 INFO:teuthology.orchestra.run.smithi161.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-05-22T21:11:40.440 INFO:teuthology.orchestra.run.smithi161.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-05-22T21:11:40.441 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-05-22T21:11:40.441 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-05-22T21:11:40.441 INFO:teuthology.orchestra.run.smithi161.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-22T21:11:40.442 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages will be upgraded: 2024-05-22T21:11:40.442 INFO:teuthology.orchestra.run.smithi161.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-05-22T21:11:40.508 INFO:teuthology.orchestra.run.smithi161.stdout:4 upgraded, 39 newly installed, 0 to remove and 306 not upgraded. 2024-05-22T21:11:40.508 INFO:teuthology.orchestra.run.smithi161.stdout:Need to get 55.7 MB of archives. 2024-05-22T21:11:40.509 INFO:teuthology.orchestra.run.smithi161.stdout:After this operation, 202 MB of additional disk space will be used. 2024-05-22T21:11:40.509 INFO:teuthology.orchestra.run.smithi161.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-05-22T21:11:40.668 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-22T21:11:40.714 INFO:teuthology.orchestra.run.smithi161.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-05-22T21:11:40.719 INFO:teuthology.orchestra.run.smithi161.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-05-22T21:11:40.770 INFO:teuthology.orchestra.run.smithi161.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-05-22T21:11:40.771 INFO:teuthology.orchestra.run.smithi161.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-05-22T21:11:40.772 INFO:teuthology.orchestra.run.smithi161.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-05-22T21:11:40.790 INFO:teuthology.orchestra.run.smithi161.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-05-22T21:11:40.791 INFO:teuthology.orchestra.run.smithi161.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-05-22T21:11:40.791 INFO:teuthology.orchestra.run.smithi161.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-05-22T21:11:40.791 INFO:teuthology.orchestra.run.smithi161.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-05-22T21:11:40.791 INFO:teuthology.orchestra.run.smithi161.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-05-22T21:11:40.792 INFO:teuthology.orchestra.run.smithi161.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-05-22T21:11:40.811 INFO:teuthology.orchestra.run.smithi161.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-05-22T21:11:40.811 INFO:teuthology.orchestra.run.smithi161.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-05-22T21:11:40.812 INFO:teuthology.orchestra.run.smithi161.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-05-22T21:11:40.839 INFO:teuthology.orchestra.run.smithi161.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-05-22T21:11:40.936 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-05-22T21:11:41.206 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-22T21:11:41.288 INFO:teuthology.orchestra.run.smithi161.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-05-22T21:11:41.293 INFO:teuthology.orchestra.run.smithi161.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-05-22T21:11:41.293 INFO:teuthology.orchestra.run.smithi161.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-05-22T21:11:41.296 INFO:teuthology.orchestra.run.smithi161.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-05-22T21:11:41.297 INFO:teuthology.orchestra.run.smithi161.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-05-22T21:11:41.302 INFO:teuthology.orchestra.run.smithi161.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-05-22T21:11:41.303 INFO:teuthology.orchestra.run.smithi161.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-05-22T21:11:41.320 INFO:teuthology.orchestra.run.smithi161.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-05-22T21:11:41.321 INFO:teuthology.orchestra.run.smithi161.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-05-22T21:11:41.338 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-05-22T21:11:41.358 INFO:teuthology.orchestra.run.smithi161.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-05-22T21:11:41.365 INFO:teuthology.orchestra.run.smithi161.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-05-22T21:11:41.381 INFO:teuthology.orchestra.run.smithi161.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-05-22T21:11:41.412 INFO:teuthology.orchestra.run.smithi161.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-05-22T21:11:41.425 INFO:teuthology.orchestra.run.smithi161.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-05-22T21:11:41.426 INFO:teuthology.orchestra.run.smithi161.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-05-22T21:11:41.427 INFO:teuthology.orchestra.run.smithi161.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-05-22T21:11:41.427 INFO:teuthology.orchestra.run.smithi161.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-05-22T21:11:41.449 INFO:teuthology.orchestra.run.smithi161.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-05-22T21:11:41.451 INFO:teuthology.orchestra.run.smithi161.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-05-22T21:11:41.615 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-22T21:11:41.730 INFO:teuthology.orchestra.run.smithi161.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-05-22T21:11:41.730 INFO:teuthology.orchestra.run.smithi161.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-05-22T21:11:41.730 INFO:teuthology.orchestra.run.smithi161.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-05-22T21:11:41.730 INFO:teuthology.orchestra.run.smithi161.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-05-22T21:11:41.731 INFO:teuthology.orchestra.run.smithi161.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-05-22T21:11:41.731 INFO:teuthology.orchestra.run.smithi161.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-05-22T21:11:41.732 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-22T21:11:41.745 INFO:teuthology.orchestra.run.smithi161.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-05-22T21:11:41.746 INFO:teuthology.orchestra.run.smithi161.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-05-22T21:11:42.034 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-05-22T21:11:42.122 INFO:teuthology.orchestra.run.smithi161.stdout:Preconfiguring packages ... 2024-05-22T21:11:42.283 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 55.7 MB in 1s (43.0 MB/s) 2024-05-22T21:11:42.377 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-05-22T21:11:42.569 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140163 files and directories currently installed.) 2024-05-22T21:11:42.573 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-22T21:11:42.644 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-05-22T21:11:42.724 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-22T21:11:43.067 INFO:teuthology.orchestra.run.smithi179.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-05-22T21:11:43.315 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-22T21:11:43.338 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-05-22T21:11:43.441 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-05-22T21:11:43.859 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-05-22T21:11:44.127 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-portend (3.0.0-1) ... 2024-05-22T21:11:44.395 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-22T21:11:44.478 INFO:teuthology.orchestra.run.smithi161.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-05-22T21:11:44.504 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-22T21:11:44.712 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package flex. 2024-05-22T21:11:44.782 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140163 files and directories currently installed.) 2024-05-22T21:11:44.786 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-05-22T21:11:44.822 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-05-22T21:11:44.852 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking flex (2.6.4-8build2) ... 2024-05-22T21:11:45.157 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-22T21:11:45.388 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-22T21:11:45.442 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-05-22T21:11:45.566 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-05-22T21:11:45.568 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-rsa (4.8-1) ... 2024-05-22T21:11:45.844 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-05-22T21:11:45.996 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-22T21:11:46.129 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-22T21:11:46.388 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-05-22T21:11:46.545 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140236 files and directories currently installed.) 2024-05-22T21:11:46.549 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-22T21:11:46.657 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-22T21:11:46.801 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-22T21:11:47.061 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-22T21:11:47.100 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-05-22T21:11:47.197 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-22T21:11:47.452 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-22T21:11:47.509 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-22T21:11:47.578 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up librados2 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:47.696 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up librgw2 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:47.897 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package bison. 2024-05-22T21:11:47.908 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libsqlite3-mod-ceph (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:47.970 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140236 files and directories currently installed.) 2024-05-22T21:11:47.974 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-05-22T21:11:48.016 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-05-22T21:11:48.034 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libcephfs2 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:48.160 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libradosstriper1 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:48.303 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-05-22T21:11:48.433 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-05-22T21:11:48.448 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-05-22T21:11:48.492 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-05-22T21:11:48.581 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up librbd1 (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:48.706 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr-modules-core (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:48.828 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package dump. 2024-05-22T21:11:48.832 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-fuse (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:48.843 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-05-22T21:11:48.886 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking dump (0.4b47-3) ... 2024-05-22T21:11:49.042 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-22T21:11:49.042 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-22T21:11:49.215 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package indent. 2024-05-22T21:11:49.230 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-05-22T21:11:49.273 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-22T21:11:49.518 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-05-22T21:11:49.533 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-05-22T21:11:49.567 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-22T21:11:49.718 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libcephfs-dev (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:49.905 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-05-22T21:11:49.920 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-05-22T21:11:49.927 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-rados (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:49.963 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-22T21:11:50.079 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up cephfs-top (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:50.222 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-22T21:11:50.224 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-05-22T21:11:50.239 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-05-22T21:11:50.283 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-05-22T21:11:50.636 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libfl2:amd64. 2024-05-22T21:11:50.651 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-05-22T21:11:50.695 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-05-22T21:11:50.889 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-05-22T21:11:50.903 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-05-22T21:11:50.947 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-22T21:11:51.200 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libjson-perl. 2024-05-22T21:11:51.215 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-05-22T21:11:51.258 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-05-22T21:11:51.528 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libllvm14:amd64. 2024-05-22T21:11:51.543 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-05-22T21:11:51.553 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-rbd (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:51.577 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-22T21:11:51.662 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up rbd-fuse (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:51.789 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-rgw (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:51.915 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-cephfs (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:52.041 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up cephfs-shell (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:52.168 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-common (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:52.655 INFO:teuthology.orchestra.run.smithi179.stdout:Adding group ceph....done 2024-05-22T21:11:52.964 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-05-22T21:11:52.980 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-22T21:11:53.014 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-22T21:11:53.073 INFO:teuthology.orchestra.run.smithi179.stdout:Adding system user ceph....done 2024-05-22T21:11:53.242 INFO:teuthology.orchestra.run.smithi179.stdout:Setting system user ceph properties....done 2024-05-22T21:11:53.257 INFO:teuthology.orchestra.run.smithi179.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-22T21:11:53.292 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libmbim-proxy. 2024-05-22T21:11:53.307 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-22T21:11:53.349 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-22T21:11:53.402 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-22T21:11:53.678 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-05-22T21:11:53.693 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-22T21:11:53.736 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-22T21:11:53.808 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-22T21:11:54.039 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libndp0:amd64. 2024-05-22T21:11:54.055 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-05-22T21:11:54.089 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-22T21:11:54.425 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libnm0:amd64. 2024-05-22T21:11:54.441 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-05-22T21:11:54.495 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up cephfs-mirror (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:54.516 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-22T21:11:54.694 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-22T21:11:54.694 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-22T21:11:54.886 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libpq5:amd64. 2024-05-22T21:11:54.902 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-22T21:11:54.944 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-22T21:11:55.264 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-05-22T21:11:55.279 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-22T21:11:55.339 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-22T21:11:55.617 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libqmi-proxy. 2024-05-22T21:11:55.632 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-22T21:11:55.660 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-test (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:55.675 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-22T21:11:55.803 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up radosgw (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:55.936 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-05-22T21:11:55.951 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-22T21:11:55.994 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-22T21:11:56.310 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-22T21:11:56.310 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-22T21:11:56.515 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-05-22T21:11:56.530 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-05-22T21:11:56.573 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-05-22T21:11:56.926 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package modemmanager. 2024-05-22T21:11:56.941 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-22T21:11:56.947 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-base (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:56.976 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-22T21:11:57.308 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-22T21:11:57.641 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package network-manager. 2024-05-22T21:11:57.656 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-05-22T21:11:57.708 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-05-22T21:11:57.983 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mds (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:58.206 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-22T21:11:58.206 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-22T21:11:58.265 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ppp. 2024-05-22T21:11:58.280 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-05-22T21:11:58.611 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-05-22T21:11:58.876 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:58.986 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package pptp-linux. 2024-05-22T21:11:59.000 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-05-22T21:11:59.053 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-05-22T21:11:59.080 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-22T21:11:59.080 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-22T21:11:59.324 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package network-manager-pptp. 2024-05-22T21:11:59.339 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-05-22T21:11:59.391 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-05-22T21:11:59.719 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-osd (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:11:59.744 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package postgresql-client-common. 2024-05-22T21:11:59.758 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-05-22T21:11:59.810 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking postgresql-client-common (238) ... 2024-05-22T21:12:00.013 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-22T21:12:00.013 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-22T21:12:00.206 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package postgresql-client-14. 2024-05-22T21:12:00.222 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-22T21:12:00.272 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-22T21:12:00.655 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr-k8sevents (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:12:00.781 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr-diskprediction-local (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:12:00.852 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package postgresql-common. 2024-05-22T21:12:00.867 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-05-22T21:12:00.919 INFO:teuthology.orchestra.run.smithi161.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-05-22T21:12:00.923 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mon (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:12:00.950 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking postgresql-common (238) ... 2024-05-22T21:12:01.133 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-22T21:12:01.133 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-22T21:12:01.374 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package postgresql-14. 2024-05-22T21:12:01.388 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-22T21:12:01.436 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-22T21:12:01.800 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr-cephadm (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:12:01.943 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:12:02.044 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr-dashboard (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:12:02.162 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-volume (18.2.2-1349-gb9b067bc-1jammy) ... 2024-05-22T21:12:02.515 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-22T21:12:02.569 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package postgresql. 2024-05-22T21:12:02.584 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-05-22T21:12:02.628 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking postgresql (14+238) ... 2024-05-22T21:12:02.896 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package postgresql-client. 2024-05-22T21:12:02.911 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-05-22T21:12:02.955 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking postgresql-client (14+238) ... 2024-05-22T21:12:03.216 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package postgresql-contrib. 2024-05-22T21:12:03.230 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-05-22T21:12:03.274 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking postgresql-contrib (14+238) ... 2024-05-22T21:12:03.534 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package tcl8.6. 2024-05-22T21:12:03.550 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-05-22T21:12:03.584 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-22T21:12:03.792 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-22T21:12:03.870 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package tcl. 2024-05-22T21:12:03.886 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-05-22T21:12:03.933 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-05-22T21:12:04.288 INFO:teuthology.orchestra.run.smithi179.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-22T21:12:04.293 DEBUG:teuthology.orchestra.run.smithi179:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-05-22T21:12:04.306 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package wpasupplicant. 2024-05-22T21:12:04.322 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-05-22T21:12:04.367 INFO:teuthology.orchestra.run.smithi179.stdout:Reading package lists... 2024-05-22T21:12:04.373 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-22T21:12:04.545 INFO:teuthology.orchestra.run.smithi179.stdout:Building dependency tree... 2024-05-22T21:12:04.545 INFO:teuthology.orchestra.run.smithi179.stdout:Reading state information... 2024-05-22T21:12:04.722 INFO:teuthology.orchestra.run.smithi179.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-05-22T21:12:04.722 INFO:teuthology.orchestra.run.smithi179.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-05-22T21:12:04.722 INFO:teuthology.orchestra.run.smithi179.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-05-22T21:12:04.722 INFO:teuthology.orchestra.run.smithi179.stdout:The following packages were automatically installed and are no longer required: 2024-05-22T21:12:04.722 INFO:teuthology.orchestra.run.smithi179.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-22T21:12:04.722 INFO:teuthology.orchestra.run.smithi179.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-22T21:12:04.723 INFO:teuthology.orchestra.run.smithi179.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-22T21:12:04.723 INFO:teuthology.orchestra.run.smithi179.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-22T21:12:04.723 INFO:teuthology.orchestra.run.smithi179.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-22T21:12:04.723 INFO:teuthology.orchestra.run.smithi179.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-22T21:12:04.724 INFO:teuthology.orchestra.run.smithi179.stdout:The following additional packages will be installed: 2024-05-22T21:12:04.724 INFO:teuthology.orchestra.run.smithi179.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-05-22T21:12:04.724 INFO:teuthology.orchestra.run.smithi179.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-05-22T21:12:04.724 INFO:teuthology.orchestra.run.smithi179.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-05-22T21:12:04.724 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-05-22T21:12:04.724 INFO:teuthology.orchestra.run.smithi179.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-22T21:12:04.725 INFO:teuthology.orchestra.run.smithi179.stdout:Suggested packages: 2024-05-22T21:12:04.725 INFO:teuthology.orchestra.run.smithi179.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-05-22T21:12:04.725 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-05-22T21:12:04.725 INFO:teuthology.orchestra.run.smithi179.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-05-22T21:12:04.727 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-05-22T21:12:04.743 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-05-22T21:12:04.779 INFO:teuthology.orchestra.run.smithi179.stdout:The following NEW packages will be installed: 2024-05-22T21:12:04.780 INFO:teuthology.orchestra.run.smithi179.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-05-22T21:12:04.780 INFO:teuthology.orchestra.run.smithi179.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-05-22T21:12:04.780 INFO:teuthology.orchestra.run.smithi179.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-05-22T21:12:04.780 INFO:teuthology.orchestra.run.smithi179.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-05-22T21:12:04.780 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-05-22T21:12:04.781 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-05-22T21:12:04.781 INFO:teuthology.orchestra.run.smithi179.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-22T21:12:04.781 INFO:teuthology.orchestra.run.smithi179.stdout:The following packages will be upgraded: 2024-05-22T21:12:04.782 INFO:teuthology.orchestra.run.smithi179.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-05-22T21:12:04.784 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-05-22T21:12:04.844 INFO:teuthology.orchestra.run.smithi179.stdout:4 upgraded, 39 newly installed, 0 to remove and 306 not upgraded. 2024-05-22T21:12:04.845 INFO:teuthology.orchestra.run.smithi179.stdout:Need to get 55.7 MB of archives. 2024-05-22T21:12:04.845 INFO:teuthology.orchestra.run.smithi179.stdout:After this operation, 202 MB of additional disk space will be used. 2024-05-22T21:12:04.845 INFO:teuthology.orchestra.run.smithi179.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-05-22T21:12:05.080 INFO:teuthology.orchestra.run.smithi179.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-05-22T21:12:05.092 INFO:teuthology.orchestra.run.smithi179.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-05-22T21:12:05.194 INFO:teuthology.orchestra.run.smithi179.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-05-22T21:12:05.194 INFO:teuthology.orchestra.run.smithi179.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-05-22T21:12:05.195 INFO:teuthology.orchestra.run.smithi179.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-05-22T21:12:05.206 INFO:teuthology.orchestra.run.smithi179.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-05-22T21:12:05.208 INFO:teuthology.orchestra.run.smithi179.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-05-22T21:12:05.210 INFO:teuthology.orchestra.run.smithi179.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-05-22T21:12:05.211 INFO:teuthology.orchestra.run.smithi179.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-05-22T21:12:05.211 INFO:teuthology.orchestra.run.smithi179.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-05-22T21:12:05.217 INFO:teuthology.orchestra.run.smithi179.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-05-22T21:12:05.219 INFO:teuthology.orchestra.run.smithi179.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-05-22T21:12:05.241 INFO:teuthology.orchestra.run.smithi179.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-05-22T21:12:05.241 INFO:teuthology.orchestra.run.smithi179.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-05-22T21:12:05.243 INFO:teuthology.orchestra.run.smithi179.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-05-22T21:12:05.513 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package usb-modeswitch. 2024-05-22T21:12:05.521 INFO:teuthology.orchestra.run.smithi179.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-05-22T21:12:05.523 INFO:teuthology.orchestra.run.smithi179.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-05-22T21:12:05.524 INFO:teuthology.orchestra.run.smithi179.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-05-22T21:12:05.527 INFO:teuthology.orchestra.run.smithi179.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-05-22T21:12:05.527 INFO:teuthology.orchestra.run.smithi179.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-05-22T21:12:05.528 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-05-22T21:12:05.532 INFO:teuthology.orchestra.run.smithi179.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-05-22T21:12:05.533 INFO:teuthology.orchestra.run.smithi179.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-05-22T21:12:05.541 INFO:teuthology.orchestra.run.smithi179.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-05-22T21:12:05.541 INFO:teuthology.orchestra.run.smithi179.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-05-22T21:12:05.563 INFO:teuthology.orchestra.run.smithi179.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-05-22T21:12:05.564 INFO:teuthology.orchestra.run.smithi179.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-05-22T21:12:05.574 INFO:teuthology.orchestra.run.smithi179.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-05-22T21:12:05.579 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-22T21:12:05.601 INFO:teuthology.orchestra.run.smithi179.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-05-22T21:12:05.605 INFO:teuthology.orchestra.run.smithi179.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-05-22T21:12:05.605 INFO:teuthology.orchestra.run.smithi179.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-05-22T21:12:05.606 INFO:teuthology.orchestra.run.smithi179.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-05-22T21:12:05.607 INFO:teuthology.orchestra.run.smithi179.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-05-22T21:12:05.622 INFO:teuthology.orchestra.run.smithi179.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-05-22T21:12:05.624 INFO:teuthology.orchestra.run.smithi179.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-05-22T21:12:05.821 INFO:teuthology.orchestra.run.smithi179.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-05-22T21:12:05.821 INFO:teuthology.orchestra.run.smithi179.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-05-22T21:12:05.821 INFO:teuthology.orchestra.run.smithi179.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-05-22T21:12:05.821 INFO:teuthology.orchestra.run.smithi179.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-05-22T21:12:05.822 INFO:teuthology.orchestra.run.smithi179.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-05-22T21:12:05.822 INFO:teuthology.orchestra.run.smithi179.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-05-22T21:12:05.835 INFO:teuthology.orchestra.run.smithi179.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-05-22T21:12:05.836 INFO:teuthology.orchestra.run.smithi179.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-05-22T21:12:05.881 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up postgresql-client-common (238) ... 2024-05-22T21:12:06.093 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up flex (2.6.4-8build2) ... 2024-05-22T21:12:06.214 INFO:teuthology.orchestra.run.smithi179.stdout:Preconfiguring packages ... 2024-05-22T21:12:06.219 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-22T21:12:06.345 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-22T21:12:06.358 INFO:teuthology.orchestra.run.smithi179.stdout:Fetched 55.7 MB in 1s (53.3 MB/s) 2024-05-22T21:12:06.471 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-05-22T21:12:06.596 INFO:teuthology.orchestra.run.smithi179.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140163 files and directories currently installed.) 2024-05-22T21:12:06.597 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-22T21:12:06.599 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-22T21:12:06.723 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-22T21:12:06.741 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-22T21:12:07.368 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-05-22T21:12:07.676 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-05-22T21:12:07.787 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-22T21:12:08.164 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-22T21:12:08.164 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-22T21:12:08.523 INFO:teuthology.orchestra.run.smithi179.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-05-22T21:12:08.774 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package flex. 2024-05-22T21:12:08.845 INFO:teuthology.orchestra.run.smithi179.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140163 files and directories currently installed.) 2024-05-22T21:12:08.849 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-05-22T21:12:08.890 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking flex (2.6.4-8build2) ... 2024-05-22T21:12:08.898 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-22T21:12:09.016 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-05-22T21:12:09.117 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-22T21:12:09.217 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-05-22T21:12:09.335 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-05-22T21:12:09.435 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-22T21:12:09.461 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-05-22T21:12:09.587 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-22T21:12:09.588 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-05-22T21:12:09.738 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-05-22T21:12:09.986 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-22T21:12:10.335 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-22T21:12:10.509 INFO:teuthology.orchestra.run.smithi179.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140236 files and directories currently installed.) 2024-05-22T21:12:10.512 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-22T21:12:10.655 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-22T21:12:10.828 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up dump (0.4b47-3) ... 2024-05-22T21:12:10.923 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-05-22T21:12:11.004 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-22T21:12:11.085 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-22T21:12:11.130 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-22T21:12:11.221 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-22T21:12:11.265 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up tcl (8.6.11+1build2) ... 2024-05-22T21:12:11.391 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libjson-perl (4.04000-1) ... 2024-05-22T21:12:11.508 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-05-22T21:12:11.536 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-22T21:12:11.589 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-05-22T21:12:11.651 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-22T21:12:11.769 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-22T21:12:11.906 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package bison. 2024-05-22T21:12:11.945 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-22T21:12:11.982 INFO:teuthology.orchestra.run.smithi179.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140236 files and directories currently installed.) 2024-05-22T21:12:11.986 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-05-22T21:12:12.033 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-05-22T21:12:12.071 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-22T21:12:12.197 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-05-22T21:12:12.390 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up postgresql-client (14+238) ... 2024-05-22T21:12:12.468 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-05-22T21:12:12.482 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-05-22T21:12:12.526 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-05-22T21:12:12.533 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-05-22T21:12:12.855 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package dump. 2024-05-22T21:12:12.869 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-05-22T21:12:12.913 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking dump (0.4b47-3) ... 2024-05-22T21:12:13.104 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:13.104 INFO:teuthology.orchestra.run.smithi161.stdout:The following network interfaces were found in /etc/network/interfaces 2024-05-22T21:12:13.104 INFO:teuthology.orchestra.run.smithi161.stdout:which means they are currently configured by ifupdown: 2024-05-22T21:12:13.104 INFO:teuthology.orchestra.run.smithi161.stdout:- enp3s0f1 2024-05-22T21:12:13.104 INFO:teuthology.orchestra.run.smithi161.stdout:If you want to manage those interfaces with NetworkManager instead 2024-05-22T21:12:13.104 INFO:teuthology.orchestra.run.smithi161.stdout:remove their configuration from /etc/network/interfaces. 2024-05-22T21:12:13.104 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:13.193 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package indent. 2024-05-22T21:12:13.208 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-05-22T21:12:13.235 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-05-22T21:12:13.251 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-22T21:12:13.504 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-05-22T21:12:13.521 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-05-22T21:12:13.562 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-22T21:12:13.622 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-05-22T21:12:13.859 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-05-22T21:12:13.874 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-05-22T21:12:13.908 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-22T21:12:14.054 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-05-22T21:12:14.145 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-05-22T21:12:14.160 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-05-22T21:12:14.203 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-05-22T21:12:14.524 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libfl2:amd64. 2024-05-22T21:12:14.542 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-05-22T21:12:14.582 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-05-22T21:12:14.743 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-22T21:12:14.810 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-05-22T21:12:14.822 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-05-22T21:12:14.851 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-22T21:12:14.894 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-22T21:12:15.020 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-05-22T21:12:15.121 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libjson-perl. 2024-05-22T21:12:15.139 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-05-22T21:12:15.171 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up postgresql-common (238) ... 2024-05-22T21:12:15.180 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-05-22T21:12:15.475 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libllvm14:amd64. 2024-05-22T21:12:15.489 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-05-22T21:12:15.533 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-22T21:12:16.067 INFO:teuthology.orchestra.run.smithi161.stdout:Adding user postgres to group ssl-cert 2024-05-22T21:12:16.380 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:16.380 INFO:teuthology.orchestra.run.smithi161.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-05-22T21:12:16.675 INFO:teuthology.orchestra.run.smithi161.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-05-22T21:12:16.676 INFO:teuthology.orchestra.run.smithi161.stdout:Removing obsolete dictionary files: 2024-05-22T21:12:16.895 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-05-22T21:12:16.908 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-22T21:12:16.953 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-22T21:12:17.214 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libmbim-proxy. 2024-05-22T21:12:17.228 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-22T21:12:17.273 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-22T21:12:17.410 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-05-22T21:12:17.609 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-05-22T21:12:17.623 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-22T21:12:17.668 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-22T21:12:17.891 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-22T21:12:17.987 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libndp0:amd64. 2024-05-22T21:12:18.001 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-05-22T21:12:18.046 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-22T21:12:18.225 INFO:teuthology.orchestra.run.smithi161.stdout:Creating new PostgreSQL cluster 14/main ... 2024-05-22T21:12:18.250 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-05-22T21:12:18.308 INFO:teuthology.orchestra.run.smithi161.stdout:The files belonging to this database system will be owned by user "postgres". 2024-05-22T21:12:18.308 INFO:teuthology.orchestra.run.smithi161.stdout:This user must also own the server process. 2024-05-22T21:12:18.308 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:18.308 INFO:teuthology.orchestra.run.smithi161.stdout:The database cluster will be initialized with locale "en_US". 2024-05-22T21:12:18.308 INFO:teuthology.orchestra.run.smithi161.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-05-22T21:12:18.308 INFO:teuthology.orchestra.run.smithi161.stdout:The default text search configuration will be set to "english". 2024-05-22T21:12:18.309 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:18.309 INFO:teuthology.orchestra.run.smithi161.stdout:Data page checksums are disabled. 2024-05-22T21:12:18.309 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:18.309 INFO:teuthology.orchestra.run.smithi161.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-05-22T21:12:18.309 INFO:teuthology.orchestra.run.smithi161.stdout:creating subdirectories ... ok 2024-05-22T21:12:18.309 INFO:teuthology.orchestra.run.smithi161.stdout:selecting dynamic shared memory implementation ... posix 2024-05-22T21:12:18.333 INFO:teuthology.orchestra.run.smithi161.stdout:selecting default max_connections ... 100 2024-05-22T21:12:18.366 INFO:teuthology.orchestra.run.smithi161.stdout:selecting default shared_buffers ... 128MB 2024-05-22T21:12:18.380 INFO:teuthology.orchestra.run.smithi161.stdout:selecting default time zone ... Etc/UTC 2024-05-22T21:12:18.381 INFO:teuthology.orchestra.run.smithi161.stdout:creating configuration files ... ok 2024-05-22T21:12:18.391 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libnm0:amd64. 2024-05-22T21:12:18.405 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-05-22T21:12:18.449 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-22T21:12:18.546 INFO:teuthology.orchestra.run.smithi161.stdout:running bootstrap script ... ok 2024-05-22T21:12:18.794 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libpq5:amd64. 2024-05-22T21:12:18.808 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-22T21:12:18.844 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-22T21:12:19.137 INFO:teuthology.orchestra.run.smithi161.stdout:performing post-bootstrap initialization ... ok 2024-05-22T21:12:19.164 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-05-22T21:12:19.180 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-22T21:12:19.222 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-22T21:12:19.526 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libqmi-proxy. 2024-05-22T21:12:19.543 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-22T21:12:19.609 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-22T21:12:19.704 INFO:teuthology.orchestra.run.smithi161.stdout:syncing data to disk ... ok 2024-05-22T21:12:19.845 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-05-22T21:12:19.860 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-22T21:12:19.895 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-22T21:12:20.207 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-05-22T21:12:20.374 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-05-22T21:12:20.389 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-05-22T21:12:20.432 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-05-22T21:12:20.769 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package modemmanager. 2024-05-22T21:12:20.785 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-22T21:12:20.828 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-22T21:12:21.461 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package network-manager. 2024-05-22T21:12:21.477 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-05-22T21:12:21.528 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-05-22T21:12:22.127 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ppp. 2024-05-22T21:12:22.142 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-05-22T21:12:22.494 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-05-22T21:12:22.882 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package pptp-linux. 2024-05-22T21:12:22.896 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-05-22T21:12:22.948 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-05-22T21:12:23.243 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package network-manager-pptp. 2024-05-22T21:12:23.257 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-05-22T21:12:23.280 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-22T21:12:23.309 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-05-22T21:12:23.414 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up postgresql-contrib (14+238) ... 2024-05-22T21:12:23.523 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up postgresql (14+238) ... 2024-05-22T21:12:23.624 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-22T21:12:23.679 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package postgresql-client-common. 2024-05-22T21:12:23.694 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-05-22T21:12:23.737 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking postgresql-client-common (238) ... 2024-05-22T21:12:23.750 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-22T21:12:24.010 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-05-22T21:12:24.011 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-05-22T21:12:24.091 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package postgresql-client-14. 2024-05-22T21:12:24.106 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-22T21:12:24.158 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-22T21:12:24.654 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package postgresql-common. 2024-05-22T21:12:24.669 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-05-22T21:12:24.722 INFO:teuthology.orchestra.run.smithi179.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-05-22T21:12:24.761 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking postgresql-common (238) ... 2024-05-22T21:12:24.786 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-22T21:12:24.995 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-22T21:12:25.220 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package postgresql-14. 2024-05-22T21:12:25.229 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-22T21:12:25.272 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-22T21:12:25.330 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-22T21:12:25.474 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-22T21:12:26.340 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package postgresql. 2024-05-22T21:12:26.352 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-05-22T21:12:26.389 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking postgresql (14+238) ... 2024-05-22T21:12:26.626 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package postgresql-client. 2024-05-22T21:12:26.638 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-05-22T21:12:26.667 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking postgresql-client (14+238) ... 2024-05-22T21:12:26.876 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-05-22T21:12:26.894 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package postgresql-contrib. 2024-05-22T21:12:26.905 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-05-22T21:12:26.944 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking postgresql-contrib (14+238) ... 2024-05-22T21:12:26.968 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-22T21:12:27.288 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package tcl8.6. 2024-05-22T21:12:27.303 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-05-22T21:12:27.346 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-22T21:12:27.640 INFO:teuthology.orchestra.run.smithi161.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-22T21:12:27.650 DEBUG:teuthology.parallel:result is None 2024-05-22T21:12:27.741 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package tcl. 2024-05-22T21:12:27.754 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-05-22T21:12:27.803 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-05-22T21:12:28.152 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package wpasupplicant. 2024-05-22T21:12:28.165 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-05-22T21:12:28.210 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-22T21:12:28.573 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-05-22T21:12:28.586 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-05-22T21:12:28.631 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-05-22T21:12:29.009 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package usb-modeswitch. 2024-05-22T21:12:29.022 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-05-22T21:12:29.067 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-22T21:12:29.350 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up postgresql-client-common (238) ... 2024-05-22T21:12:29.556 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up flex (2.6.4-8build2) ... 2024-05-22T21:12:29.682 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-22T21:12:29.800 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-22T21:12:29.919 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-05-22T21:12:30.077 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-22T21:12:30.187 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-22T21:12:31.121 INFO:teuthology.orchestra.run.smithi179.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-05-22T21:12:31.662 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-22T21:12:32.054 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-22T21:12:32.054 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-22T21:12:32.770 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-22T21:12:32.879 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-05-22T21:12:33.005 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-22T21:12:33.156 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-05-22T21:12:33.282 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-05-22T21:12:33.408 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-05-22T21:12:33.534 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-22T21:12:33.661 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-05-22T21:12:34.290 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-22T21:12:34.417 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up dump (0.4b47-3) ... 2024-05-22T21:12:34.511 INFO:teuthology.orchestra.run.smithi179.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-05-22T21:12:34.610 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-22T21:12:34.736 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-22T21:12:34.862 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up tcl (8.6.11+1build2) ... 2024-05-22T21:12:34.963 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libjson-perl (4.04000-1) ... 2024-05-22T21:12:35.080 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-05-22T21:12:35.170 INFO:teuthology.orchestra.run.smithi179.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-05-22T21:12:35.240 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-22T21:12:35.367 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-22T21:12:35.543 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-22T21:12:35.677 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-22T21:12:35.803 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-05-22T21:12:35.963 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up postgresql-client (14+238) ... 2024-05-22T21:12:36.089 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-05-22T21:12:36.707 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:36.707 INFO:teuthology.orchestra.run.smithi179.stdout:The following network interfaces were found in /etc/network/interfaces 2024-05-22T21:12:36.707 INFO:teuthology.orchestra.run.smithi179.stdout:which means they are currently configured by ifupdown: 2024-05-22T21:12:36.707 INFO:teuthology.orchestra.run.smithi179.stdout:- enp3s0f1 2024-05-22T21:12:36.707 INFO:teuthology.orchestra.run.smithi179.stdout:If you want to manage those interfaces with NetworkManager instead 2024-05-22T21:12:36.707 INFO:teuthology.orchestra.run.smithi179.stdout:remove their configuration from /etc/network/interfaces. 2024-05-22T21:12:36.707 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:36.840 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-05-22T21:12:37.234 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-05-22T21:12:37.647 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-05-22T21:12:38.350 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-22T21:12:38.501 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-22T21:12:38.635 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-05-22T21:12:38.761 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up postgresql-common (238) ... 2024-05-22T21:12:39.635 INFO:teuthology.orchestra.run.smithi179.stdout:Adding user postgres to group ssl-cert 2024-05-22T21:12:39.885 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:39.885 INFO:teuthology.orchestra.run.smithi179.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-05-22T21:12:40.137 INFO:teuthology.orchestra.run.smithi179.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-05-22T21:12:40.137 INFO:teuthology.orchestra.run.smithi179.stdout:Removing obsolete dictionary files: 2024-05-22T21:12:40.837 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-05-22T21:12:41.321 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-22T21:12:41.643 INFO:teuthology.orchestra.run.smithi179.stdout:Creating new PostgreSQL cluster 14/main ... 2024-05-22T21:12:41.663 INFO:teuthology.orchestra.run.smithi179.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-05-22T21:12:41.711 INFO:teuthology.orchestra.run.smithi179.stdout:The files belonging to this database system will be owned by user "postgres". 2024-05-22T21:12:41.711 INFO:teuthology.orchestra.run.smithi179.stdout:This user must also own the server process. 2024-05-22T21:12:41.711 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:41.711 INFO:teuthology.orchestra.run.smithi179.stdout:The database cluster will be initialized with locale "en_US". 2024-05-22T21:12:41.711 INFO:teuthology.orchestra.run.smithi179.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-05-22T21:12:41.711 INFO:teuthology.orchestra.run.smithi179.stdout:The default text search configuration will be set to "english". 2024-05-22T21:12:41.711 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:41.711 INFO:teuthology.orchestra.run.smithi179.stdout:Data page checksums are disabled. 2024-05-22T21:12:41.711 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:41.712 INFO:teuthology.orchestra.run.smithi179.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-05-22T21:12:41.712 INFO:teuthology.orchestra.run.smithi179.stdout:creating subdirectories ... ok 2024-05-22T21:12:41.712 INFO:teuthology.orchestra.run.smithi179.stdout:selecting dynamic shared memory implementation ... posix 2024-05-22T21:12:41.735 INFO:teuthology.orchestra.run.smithi179.stdout:selecting default max_connections ... 100 2024-05-22T21:12:41.769 INFO:teuthology.orchestra.run.smithi179.stdout:selecting default shared_buffers ... 128MB 2024-05-22T21:12:41.783 INFO:teuthology.orchestra.run.smithi179.stdout:selecting default time zone ... Etc/UTC 2024-05-22T21:12:41.784 INFO:teuthology.orchestra.run.smithi179.stdout:creating configuration files ... ok 2024-05-22T21:12:41.948 INFO:teuthology.orchestra.run.smithi179.stdout:running bootstrap script ... ok 2024-05-22T21:12:42.584 INFO:teuthology.orchestra.run.smithi179.stdout:performing post-bootstrap initialization ... ok 2024-05-22T21:12:43.286 INFO:teuthology.orchestra.run.smithi179.stdout:syncing data to disk ... ok 2024-05-22T21:12:43.816 INFO:teuthology.orchestra.run.smithi179.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-05-22T21:12:46.887 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-22T21:12:46.987 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up postgresql-contrib (14+238) ... 2024-05-22T21:12:47.114 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up postgresql (14+238) ... 2024-05-22T21:12:47.223 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-22T21:12:47.340 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-22T21:12:47.602 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-05-22T21:12:47.602 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-05-22T21:12:48.385 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-22T21:12:48.611 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-22T21:12:48.963 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-22T21:12:49.106 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-22T21:12:50.509 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-05-22T21:12:50.609 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-22T21:12:51.337 INFO:teuthology.orchestra.run.smithi179.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-22T21:12:51.344 DEBUG:teuthology.parallel:result is None 2024-05-22T21:12:51.345 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b9b067bc6d30ec00e391f0fa43f50affe981ba2f 2024-05-22T21:12:51.492 DEBUG:teuthology.orchestra.run.smithi161:> dpkg-query -W -f '${Version}' ceph 2024-05-22T21:12:51.516 INFO:teuthology.orchestra.run.smithi161.stdout:18.2.2-1349-gb9b067bc-1jammy 2024-05-22T21:12:51.517 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1349-gb9b067bc-1jammy 2024-05-22T21:12:51.517 INFO:teuthology.task.install:The correct ceph version 18.2.2-1349-gb9b067bc-1jammy is installed. 2024-05-22T21:12:51.519 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b9b067bc6d30ec00e391f0fa43f50affe981ba2f 2024-05-22T21:12:51.789 DEBUG:teuthology.orchestra.run.smithi179:> dpkg-query -W -f '${Version}' ceph 2024-05-22T21:12:51.812 INFO:teuthology.orchestra.run.smithi179.stdout:18.2.2-1349-gb9b067bc-1jammy 2024-05-22T21:12:51.813 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1349-gb9b067bc-1jammy 2024-05-22T21:12:51.813 INFO:teuthology.task.install:The correct ceph version 18.2.2-1349-gb9b067bc-1jammy is installed. 2024-05-22T21:12:51.815 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-22T21:12:51.815 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:12:51.815 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-22T21:12:51.838 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:12:51.838 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-22T21:12:51.875 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-22T21:12:51.875 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:12:51.876 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-05-22T21:12:51.902 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-22T21:12:51.966 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:12:51.967 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/daemon-helper 2024-05-22T21:12:51.986 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-22T21:12:52.046 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-22T21:12:52.046 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:12:52.046 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-22T21:12:52.067 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-22T21:12:52.134 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:12:52.134 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-22T21:12:52.154 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-22T21:12:52.214 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-22T21:12:52.214 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:12:52.214 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-05-22T21:12:52.235 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-22T21:12:52.302 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:12:52.302 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/stdin-killer 2024-05-22T21:12:52.321 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-22T21:12:52.382 INFO:teuthology.run_tasks:Running task ceph... 2024-05-22T21:12:52.451 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-22T21:12:52.451 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /var/log/ceph 2024-05-22T21:12:52.454 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 777 /var/log/ceph 2024-05-22T21:12:52.473 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-22T21:12:52.473 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-22T21:12:52.521 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-22T21:12:52.540 INFO:tasks.ceph:Creating extra log directories... 2024-05-22T21:12:52.540 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-22T21:12:52.589 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-22T21:12:52.615 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-22T21:12:52.615 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-22T21:12:52.615 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-22T21:12:52.616 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:12:52.616 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-22T21:12:52.663 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-22T21:12:52.755 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:12:52.755 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-22T21:12:52.763 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-22T21:12:52.853 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-22T21:12:52.853 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-22T21:12:52.854 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718018', 'branch': 'reef', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full}', 'email': None, 'first_in_suite': False, 'job_id': '7718018', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': 'b9b067bc6d30ec00e391f0fa43f50affe981ba2f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b9b067bc6d30ec00e391f0fa43f50affe981ba2f'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '630900aeb5fd525af2a375cc2059645127fe426f'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 120, 'repo': 'https://github.com/ceph/ceph', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7924, 'sha1': 'b9b067bc6d30ec00e391f0fa43f50affe981ba2f', 'sleep_before_teardown': 0, 'subset': '244/512', 'suite': 'fs', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_630900aeb5fd525af2a375cc2059645127fe426f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '630900aeb5fd525af2a375cc2059645127fe426f', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi161.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLvjX/VGhFxeFt7GOEjv8tshM58p/iYZCunKdpy5b8LMU/Z/G9BR2HMNkA5Ref3VGU6RYfCQN2YGBgdeX1Mwh0s=', 'smithi179.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBuQB+z1kAxwr4ct4kbsucuTKIMV1zjW8eZV/r1mYFjaAZq1c3sjL6gkyUUV8YXodK7j/6o8l/16kyRJg9862fs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b9b067bc6d30ec00e391f0fa43f50affe981ba2f'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': 'b9b067bc6d30ec00e391f0fa43f50affe981ba2f', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_full']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a', 'timestamp': '2024-05-21_10:46:42', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841'} 2024-05-22T21:12:52.854 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-22T21:12:52.858 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-22T21:12:52.899 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m0777 -- /var/run/ceph 2024-05-22T21:12:52.913 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m0777 -- /var/run/ceph 2024-05-22T21:12:52.962 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:12:52.962 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-05-22T21:12:52.982 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-22T21:12:52.983 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-05-22T21:12:53.035 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-22T21:12:53.035 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T21:12:53.035 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 891 Links: 1 2024-05-22T21:12:53.035 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T21:12:53.035 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-05-22 21:06:19.667377645 +0000 2024-05-22T21:12:53.036 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-05-22 21:06:19.403380004 +0000 2024-05-22T21:12:53.036 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-05-22 21:06:19.403380004 +0000 2024-05-22T21:12:53.036 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-05-22 21:06:19.403380004 +0000 2024-05-22T21:12:53.036 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-22T21:12:53.097 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-05-22T21:12:53.097 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-05-22T21:12:53.097 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000403475 s, 1.3 MB/s 2024-05-22T21:12:53.099 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-22T21:12:53.153 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-05-22T21:12:53.203 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-22T21:12:53.203 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T21:12:53.203 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 908 Links: 1 2024-05-22T21:12:53.203 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T21:12:53.203 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-05-22 21:06:19.971374930 +0000 2024-05-22T21:12:53.203 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-05-22 21:06:19.967374966 +0000 2024-05-22T21:12:53.203 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-05-22 21:06:19.967374966 +0000 2024-05-22T21:12:53.203 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-05-22 21:06:19.967374966 +0000 2024-05-22T21:12:53.204 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-22T21:12:53.265 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-05-22T21:12:53.265 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-05-22T21:12:53.265 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000385118 s, 1.3 MB/s 2024-05-22T21:12:53.267 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-22T21:12:53.320 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-05-22T21:12:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-22T21:12:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T21:12:53.371 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 928 Links: 1 2024-05-22T21:12:53.371 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T21:12:53.371 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-05-22 21:06:20.839367177 +0000 2024-05-22T21:12:53.372 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-05-22 21:06:20.587369428 +0000 2024-05-22T21:12:53.372 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-05-22 21:06:20.587369428 +0000 2024-05-22T21:12:53.372 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-05-22 21:06:20.587369428 +0000 2024-05-22T21:12:53.372 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-22T21:12:53.433 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-05-22T21:12:53.433 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-05-22T21:12:53.433 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000640449 s, 799 kB/s 2024-05-22T21:12:53.435 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-22T21:12:53.489 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-05-22T21:12:53.539 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-22T21:12:53.539 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T21:12:53.539 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 942 Links: 1 2024-05-22T21:12:53.539 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T21:12:53.539 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-05-22 21:06:21.423361961 +0000 2024-05-22T21:12:53.539 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-05-22 21:06:21.139364498 +0000 2024-05-22T21:12:53.539 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-05-22 21:06:21.139364498 +0000 2024-05-22T21:12:53.539 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-05-22 21:06:21.139364498 +0000 2024-05-22T21:12:53.539 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-22T21:12:53.601 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-05-22T21:12:53.601 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-05-22T21:12:53.601 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000392696 s, 1.3 MB/s 2024-05-22T21:12:53.603 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-22T21:12:53.657 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-22T21:12:53.657 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:12:53.657 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/scratch_devs of=/dev/stdout 2024-05-22T21:12:53.666 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-22T21:12:53.666 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_1 2024-05-22T21:12:53.716 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-22T21:12:53.716 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T21:12:53.716 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 881 Links: 1 2024-05-22T21:12:53.717 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T21:12:53.717 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-05-22 21:06:20.167842695 +0000 2024-05-22T21:12:53.717 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-05-22 21:06:19.891845161 +0000 2024-05-22T21:12:53.717 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-05-22 21:06:19.891845161 +0000 2024-05-22T21:12:53.717 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-05-22 21:06:19.891845161 +0000 2024-05-22T21:12:53.717 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-22T21:12:53.778 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-05-22T21:12:53.779 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-05-22T21:12:53.779 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000411014 s, 1.2 MB/s 2024-05-22T21:12:53.780 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-22T21:12:53.834 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_2 2024-05-22T21:12:53.884 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-22T21:12:53.885 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T21:12:53.885 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 898 Links: 1 2024-05-22T21:12:53.885 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T21:12:53.885 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-05-22 21:06:20.475839945 +0000 2024-05-22T21:12:53.885 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-05-22 21:06:20.471839980 +0000 2024-05-22T21:12:53.885 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-05-22 21:06:20.471839980 +0000 2024-05-22T21:12:53.885 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-05-22 21:06:20.471839980 +0000 2024-05-22T21:12:53.885 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-22T21:12:53.946 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-05-22T21:12:53.946 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-05-22T21:12:53.946 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000669934 s, 764 kB/s 2024-05-22T21:12:53.948 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-22T21:12:54.003 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_3 2024-05-22T21:12:54.056 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-22T21:12:54.056 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T21:12:54.056 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 915 Links: 1 2024-05-22T21:12:54.056 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T21:12:54.056 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-05-22 21:06:21.243833084 +0000 2024-05-22T21:12:54.057 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-05-22 21:06:20.999835265 +0000 2024-05-22T21:12:54.057 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-05-22 21:06:20.999835265 +0000 2024-05-22T21:12:54.057 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-05-22 21:06:20.999835265 +0000 2024-05-22T21:12:54.057 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-22T21:12:54.118 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-05-22T21:12:54.119 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-05-22T21:12:54.119 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.00037795 s, 1.4 MB/s 2024-05-22T21:12:54.120 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-22T21:12:54.174 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_4 2024-05-22T21:12:54.224 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-22T21:12:54.224 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-22T21:12:54.224 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 930 Links: 1 2024-05-22T21:12:54.225 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-22T21:12:54.225 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-05-22 21:06:21.787828226 +0000 2024-05-22T21:12:54.225 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-05-22 21:06:21.523830584 +0000 2024-05-22T21:12:54.225 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-05-22 21:06:21.523830584 +0000 2024-05-22T21:12:54.225 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-05-22 21:06:21.523830584 +0000 2024-05-22T21:12:54.225 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-22T21:12:54.290 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-05-22T21:12:54.290 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-05-22T21:12:54.291 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000393808 s, 1.3 MB/s 2024-05-22T21:12:54.292 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-22T21:12:54.346 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-22T21:12:54.346 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi161.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi179.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-22T21:12:54.346 INFO:tasks.ceph:Generating config... 2024-05-22T21:12:54.348 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-22T21:12:54.348 INFO:tasks.ceph:[client] debug client = 20 2024-05-22T21:12:54.348 INFO:tasks.ceph:[client] debug ms = 1 2024-05-22T21:12:54.348 INFO:tasks.ceph:[client] pid file = 2024-05-22T21:12:54.348 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-22T21:12:54.348 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-22T21:12:54.348 INFO:tasks.ceph:[client.0] debug client = 20 2024-05-22T21:12:54.348 INFO:tasks.ceph:[client.0] debug objectcacher = 20 2024-05-22T21:12:54.348 INFO:tasks.ceph:[client.0] debug objecter = 20 2024-05-22T21:12:54.349 INFO:tasks.ceph:[client.1] debug client = 20 2024-05-22T21:12:54.349 INFO:tasks.ceph:[client.1] debug objectcacher = 20 2024-05-22T21:12:54.349 INFO:tasks.ceph:[client.1] debug objecter = 20 2024-05-22T21:12:54.349 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-22T21:12:54.349 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-22T21:12:54.349 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-22T21:12:54.350 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-22T21:12:54.350 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-22T21:12:54.350 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-22T21:12:54.350 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-22T21:12:54.350 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-22T21:12:54.350 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-22T21:12:54.350 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-22T21:12:54.350 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-22T21:12:54.350 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-22T21:12:54.350 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-22T21:12:54.350 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-22T21:12:54.350 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-22T21:12:54.350 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-22T21:12:54.351 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-22T21:12:54.351 INFO:tasks.ceph:[mon] mon osd backfillfull ratio = 0.6 2024-05-22T21:12:54.351 INFO:tasks.ceph:[mon] mon osd full ratio = 0.7 2024-05-22T21:12:54.351 INFO:tasks.ceph:[mon] mon osd nearfull ratio = 0.6 2024-05-22T21:12:54.351 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-22T21:12:54.351 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-22T21:12:54.351 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-22T21:12:54.351 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-22T21:12:54.351 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-22T21:12:54.351 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-22T21:12:54.351 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-22T21:12:54.351 INFO:tasks.ceph:[osd] memstore device bytes = 200000000 2024-05-22T21:12:54.352 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-22T21:12:54.352 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-22T21:12:54.352 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-22T21:12:54.352 INFO:tasks.ceph:[osd] osd failsafe full ratio = 1.0 2024-05-22T21:12:54.352 INFO:tasks.ceph:[osd] osd mon report interval = 5 2024-05-22T21:12:54.352 INFO:tasks.ceph:[osd] osd objectstore = memstore 2024-05-22T21:12:54.352 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-22T21:12:54.352 INFO:tasks.ceph:Setting up mon.a... 2024-05-22T21:12:54.352 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-22T21:12:54.396 INFO:teuthology.orchestra.run.smithi161.stdout:creating /etc/ceph/ceph.keyring 2024-05-22T21:12:54.409 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-22T21:12:54.499 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-22T21:12:54.518 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.161'), ('mon.b', '172.21.15.179'), ('mon.c', '[v2:172.21.15.179:3301,v1:172.21.15.179:6790]')] 2024-05-22T21:12:54.519 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.161,172.21.15.179,[v2:172.21.15.179:3301,v1:172.21.15.179:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-22T21:12:54.520 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:12:54.520 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-22T21:12:54.574 DEBUG:teuthology.orchestra.run.smithi161:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.161 --add b 172.21.15.179 --addv c '[v2:172.21.15.179:3301,v1:172.21.15.179:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-22T21:12:54.663 INFO:teuthology.orchestra.run.smithi161.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-22T21:12:54.664 INFO:teuthology.orchestra.run.smithi161.stdout:monmaptool: generated fsid d6fa8528-b1bf-41b6-9f76-c0d715394087 2024-05-22T21:12:54.664 INFO:teuthology.orchestra.run.smithi161.stdout:setting min_mon_release = pacific 2024-05-22T21:12:54.664 INFO:teuthology.orchestra.run.smithi161.stdout:epoch 0 2024-05-22T21:12:54.664 INFO:teuthology.orchestra.run.smithi161.stdout:fsid d6fa8528-b1bf-41b6-9f76-c0d715394087 2024-05-22T21:12:54.664 INFO:teuthology.orchestra.run.smithi161.stdout:last_changed 2024-05-22T21:12:54.666201+0000 2024-05-22T21:12:54.664 INFO:teuthology.orchestra.run.smithi161.stdout:created 2024-05-22T21:12:54.666201+0000 2024-05-22T21:12:54.664 INFO:teuthology.orchestra.run.smithi161.stdout:min_mon_release 16 (pacific) 2024-05-22T21:12:54.665 INFO:teuthology.orchestra.run.smithi161.stdout:election_strategy: 1 2024-05-22T21:12:54.665 INFO:teuthology.orchestra.run.smithi161.stdout:0: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.a 2024-05-22T21:12:54.665 INFO:teuthology.orchestra.run.smithi161.stdout:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.b 2024-05-22T21:12:54.665 INFO:teuthology.orchestra.run.smithi161.stdout:2: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.c 2024-05-22T21:12:54.665 INFO:teuthology.orchestra.run.smithi161.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-22T21:12:54.667 DEBUG:teuthology.orchestra.run.smithi161:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-22T21:12:54.675 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d6fa8528-b1bf-41b6-9f76-c0d715394087... 2024-05-22T21:12:54.676 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-22T21:12:54.721 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-22T21:12:54.760 INFO:teuthology.orchestra.run.smithi161.stdout:[global] 2024-05-22T21:12:54.760 INFO:teuthology.orchestra.run.smithi161.stdout: chdir = "" 2024-05-22T21:12:54.760 INFO:teuthology.orchestra.run.smithi161.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-22T21:12:54.761 INFO:teuthology.orchestra.run.smithi161.stdout: auth supported = cephx 2024-05-22T21:12:54.761 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.761 INFO:teuthology.orchestra.run.smithi161.stdout: filestore xattr use omap = true 2024-05-22T21:12:54.761 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.761 INFO:teuthology.orchestra.run.smithi161.stdout: mon clock drift allowed = 1.000 2024-05-22T21:12:54.761 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.761 INFO:teuthology.orchestra.run.smithi161.stdout: osd crush chooseleaf type = 0 2024-05-22T21:12:54.761 INFO:teuthology.orchestra.run.smithi161.stdout: auth debug = true 2024-05-22T21:12:54.761 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.761 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on old message = true 2024-05-22T21:12:54.761 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on bug = true 2024-05-22T21:12:54.761 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.762 INFO:teuthology.orchestra.run.smithi161.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-22T21:12:54.762 INFO:teuthology.orchestra.run.smithi161.stdout: mon pg warn max object skew = 0 2024-05-22T21:12:54.762 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.762 INFO:teuthology.orchestra.run.smithi161.stdout: # disable pg_autoscaler by default for new pools 2024-05-22T21:12:54.762 INFO:teuthology.orchestra.run.smithi161.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-22T21:12:54.762 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.762 INFO:teuthology.orchestra.run.smithi161.stdout: osd pool default size = 2 2024-05-22T21:12:54.762 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.762 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd allow primary affinity = true 2024-05-22T21:12:54.762 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd allow pg remap = true 2024-05-22T21:12:54.762 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on legacy crush tunables = false 2024-05-22T21:12:54.762 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on crush straw calc version zero = false 2024-05-22T21:12:54.763 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on no sortbitwise = false 2024-05-22T21:12:54.763 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on osd down out interval zero = false 2024-05-22T21:12:54.763 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on too few osds = false 2024-05-22T21:12:54.763 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-22T21:12:54.763 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-22T21:12:54.763 INFO:teuthology.orchestra.run.smithi161.stdout: mon_allow_pool_size_one = true 2024-05-22T21:12:54.763 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.763 INFO:teuthology.orchestra.run.smithi161.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-22T21:12:54.763 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.763 INFO:teuthology.orchestra.run.smithi161.stdout: osd default data pool replay window = 5 2024-05-22T21:12:54.763 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.763 INFO:teuthology.orchestra.run.smithi161.stdout: mon allow pool delete = true 2024-05-22T21:12:54.763 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.764 INFO:teuthology.orchestra.run.smithi161.stdout: mon cluster log file level = debug 2024-05-22T21:12:54.764 INFO:teuthology.orchestra.run.smithi161.stdout: debug asserts on shutdown = true 2024-05-22T21:12:54.764 INFO:teuthology.orchestra.run.smithi161.stdout: mon health detail to clog = false 2024-05-22T21:12:54.764 INFO:teuthology.orchestra.run.smithi161.stdout: mon host = "172.21.15.161,172.21.15.179,[v2:172.21.15.179:3301,v1:172.21.15.179:6790]" 2024-05-22T21:12:54.764 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on skipped message = False 2024-05-22T21:12:54.764 INFO:teuthology.orchestra.run.smithi161.stdout: fsid = d6fa8528-b1bf-41b6-9f76-c0d715394087 2024-05-22T21:12:54.764 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.764 INFO:teuthology.orchestra.run.smithi161.stdout:[osd] 2024-05-22T21:12:54.764 INFO:teuthology.orchestra.run.smithi161.stdout: osd journal size = 100 2024-05-22T21:12:54.764 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.764 INFO:teuthology.orchestra.run.smithi161.stdout: osd scrub load threshold = 5.0 2024-05-22T21:12:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: osd scrub max interval = 600 2024-05-22T21:12:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: osd mclock profile = high_recovery_ops 2024-05-22T21:12:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: osd recover clone overlap = true 2024-05-22T21:12:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: osd recovery max chunk = 1048576 2024-05-22T21:12:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug shutdown = true 2024-05-22T21:12:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug op order = true 2024-05-22T21:12:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug verify stray on activate = true 2024-05-22T21:12:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: osd open classes on start = true 2024-05-22T21:12:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug pg log writeout = true 2024-05-22T21:12:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: osd deep scrub update digest min age = 30 2024-05-22T21:12:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: osd map max advance = 10 2024-05-22T21:12:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: journal zero on create = true 2024-05-22T21:12:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: filestore ondisk finisher threads = 3 2024-05-22T21:12:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: filestore apply finisher threads = 3 2024-05-22T21:12:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: bdev debug aio = true 2024-05-22T21:12:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug misdirected ops = true 2024-05-22T21:12:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore block size = 96636764160 2024-05-22T21:12:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore fsck on mount = True 2024-05-22T21:12:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: debug bluefs = 20 2024-05-22T21:12:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: debug bluestore = 20 2024-05-22T21:12:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-05-22T21:12:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: debug osd = 20 2024-05-22T21:12:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: debug rocksdb = 10 2024-05-22T21:12:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: memstore device bytes = 200000000 2024-05-22T21:12:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd backfillfull_ratio = 0.85 2024-05-22T21:12:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd full ratio = 0.9 2024-05-22T21:12:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd nearfull ratio = 0.8 2024-05-22T21:12:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: osd failsafe full ratio = 1.0 2024-05-22T21:12:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: osd mon report interval = 5 2024-05-22T21:12:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: osd objectstore = memstore 2024-05-22T21:12:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: osd op complaint time = 180 2024-05-22T21:12:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.768 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr] 2024-05-22T21:12:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-05-22T21:12:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: debug mgr = 20 2024-05-22T21:12:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: debug mon = 20 2024-05-22T21:12:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: debug auth = 20 2024-05-22T21:12:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min pgs per osd = 4 2024-05-22T21:12:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min bytes per osd = 10 2024-05-22T21:12:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: mgr/telemetry/nag = false 2024-05-22T21:12:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.769 INFO:teuthology.orchestra.run.smithi161.stdout:[mon] 2024-05-22T21:12:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-05-22T21:12:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: debug mon = 20 2024-05-22T21:12:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: debug paxos = 20 2024-05-22T21:12:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: debug auth = 20 2024-05-22T21:12:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: mon data avail warn = 5 2024-05-22T21:12:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: mon mgr mkfs grace = 240 2024-05-22T21:12:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min pgs per osd = 4 2024-05-22T21:12:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd reporter subtree level = osd 2024-05-22T21:12:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd prime pg temp = true 2024-05-22T21:12:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min bytes per osd = 10 2024-05-22T21:12:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-22T21:12:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: auth mon ticket ttl = 660 # 11m 2024-05-22T21:12:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: auth service ticket ttl = 240 # 4m 2024-05-22T21:12:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: # don't complain about insecure global_id in the test suite 2024-05-22T21:12:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-22T21:12:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-22T21:12:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: # 1m isn't quite enough 2024-05-22T21:12:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: mon_down_mkfs_grace = 2m 2024-05-22T21:12:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_filestore_osds = false 2024-05-22T21:12:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: mon op complaint time = 120 2024-05-22T21:12:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd backfillfull ratio = 0.6 2024-05-22T21:12:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd full ratio = 0.7 2024-05-22T21:12:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd nearfull ratio = 0.6 2024-05-22T21:12:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:12:54.772 INFO:teuthology.orchestra.run.smithi161.stdout:[client] 2024-05-22T21:12:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: rgw cache enabled = true 2024-05-22T21:12:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: rgw enable ops log = true 2024-05-22T21:12:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: rgw enable usage log = true 2024-05-22T21:12:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-22T21:12:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-22T21:12:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: client mount timeout = 600 2024-05-22T21:12:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: debug client = 20 2024-05-22T21:12:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-05-22T21:12:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: pid file = "" 2024-05-22T21:12:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: rados mon op timeout = 900 2024-05-22T21:12:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: rados osd op timeout = 900 2024-05-22T21:12:54.773 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.a] 2024-05-22T21:12:54.773 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.b] 2024-05-22T21:12:54.773 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.c] 2024-05-22T21:12:54.773 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.a] 2024-05-22T21:12:54.773 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.c] 2024-05-22T21:12:54.773 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.b] 2024-05-22T21:12:54.773 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.d] 2024-05-22T21:12:54.773 INFO:teuthology.orchestra.run.smithi161.stdout:[client.0] 2024-05-22T21:12:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: debug client = 20 2024-05-22T21:12:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: debug objectcacher = 20 2024-05-22T21:12:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: debug objecter = 20 2024-05-22T21:12:54.774 INFO:teuthology.orchestra.run.smithi161.stdout:[client.1] 2024-05-22T21:12:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: debug client = 20 2024-05-22T21:12:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: debug objectcacher = 20 2024-05-22T21:12:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: debug objecter = 20 2024-05-22T21:12:54.774 INFO:teuthology.orchestra.run.smithi161.stdout:[mds] 2024-05-22T21:12:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: debug mds = 20 2024-05-22T21:12:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: debug mds balancer = 20 2024-05-22T21:12:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-05-22T21:12:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: mds debug frag = True 2024-05-22T21:12:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: mds debug scatterstat = True 2024-05-22T21:12:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: mds op complaint time = 180 2024-05-22T21:12:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: mds verify scatter = True 2024-05-22T21:12:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: osd op complaint time = 180 2024-05-22T21:12:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: rados mon op timeout = 900 2024-05-22T21:12:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: rados osd op timeout = 900 2024-05-22T21:12:54.776 INFO:teuthology.orchestra.run.smithi179.stdout:[global] 2024-05-22T21:12:54.776 INFO:teuthology.orchestra.run.smithi179.stdout: chdir = "" 2024-05-22T21:12:54.776 INFO:teuthology.orchestra.run.smithi179.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-22T21:12:54.776 INFO:teuthology.orchestra.run.smithi179.stdout: auth supported = cephx 2024-05-22T21:12:54.777 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.777 INFO:teuthology.orchestra.run.smithi179.stdout: filestore xattr use omap = true 2024-05-22T21:12:54.777 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.777 INFO:teuthology.orchestra.run.smithi179.stdout: mon clock drift allowed = 1.000 2024-05-22T21:12:54.777 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.777 INFO:teuthology.orchestra.run.smithi179.stdout: osd crush chooseleaf type = 0 2024-05-22T21:12:54.777 INFO:teuthology.orchestra.run.smithi179.stdout: auth debug = true 2024-05-22T21:12:54.777 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.777 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on old message = true 2024-05-22T21:12:54.777 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on bug = true 2024-05-22T21:12:54.777 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.777 INFO:teuthology.orchestra.run.smithi179.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-22T21:12:54.778 INFO:teuthology.orchestra.run.smithi179.stdout: mon pg warn max object skew = 0 2024-05-22T21:12:54.778 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.778 INFO:teuthology.orchestra.run.smithi179.stdout: # disable pg_autoscaler by default for new pools 2024-05-22T21:12:54.778 INFO:teuthology.orchestra.run.smithi179.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-22T21:12:54.778 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.778 INFO:teuthology.orchestra.run.smithi179.stdout: osd pool default size = 2 2024-05-22T21:12:54.778 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.778 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd allow primary affinity = true 2024-05-22T21:12:54.778 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd allow pg remap = true 2024-05-22T21:12:54.778 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on legacy crush tunables = false 2024-05-22T21:12:54.779 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on crush straw calc version zero = false 2024-05-22T21:12:54.779 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on no sortbitwise = false 2024-05-22T21:12:54.779 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on osd down out interval zero = false 2024-05-22T21:12:54.779 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on too few osds = false 2024-05-22T21:12:54.779 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-22T21:12:54.779 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-22T21:12:54.779 INFO:teuthology.orchestra.run.smithi179.stdout: mon_allow_pool_size_one = true 2024-05-22T21:12:54.779 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.779 INFO:teuthology.orchestra.run.smithi179.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-22T21:12:54.779 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.779 INFO:teuthology.orchestra.run.smithi179.stdout: osd default data pool replay window = 5 2024-05-22T21:12:54.779 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.780 INFO:teuthology.orchestra.run.smithi179.stdout: mon allow pool delete = true 2024-05-22T21:12:54.780 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.780 INFO:teuthology.orchestra.run.smithi179.stdout: mon cluster log file level = debug 2024-05-22T21:12:54.780 INFO:teuthology.orchestra.run.smithi179.stdout: debug asserts on shutdown = true 2024-05-22T21:12:54.780 INFO:teuthology.orchestra.run.smithi179.stdout: mon health detail to clog = false 2024-05-22T21:12:54.780 INFO:teuthology.orchestra.run.smithi179.stdout: mon host = "172.21.15.161,172.21.15.179,[v2:172.21.15.179:3301,v1:172.21.15.179:6790]" 2024-05-22T21:12:54.780 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on skipped message = False 2024-05-22T21:12:54.780 INFO:teuthology.orchestra.run.smithi179.stdout: fsid = d6fa8528-b1bf-41b6-9f76-c0d715394087 2024-05-22T21:12:54.780 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.780 INFO:teuthology.orchestra.run.smithi179.stdout:[osd] 2024-05-22T21:12:54.780 INFO:teuthology.orchestra.run.smithi179.stdout: osd journal size = 100 2024-05-22T21:12:54.780 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.780 INFO:teuthology.orchestra.run.smithi179.stdout: osd scrub load threshold = 5.0 2024-05-22T21:12:54.781 INFO:teuthology.orchestra.run.smithi179.stdout: osd scrub max interval = 600 2024-05-22T21:12:54.781 INFO:teuthology.orchestra.run.smithi179.stdout: osd mclock profile = high_recovery_ops 2024-05-22T21:12:54.781 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.781 INFO:teuthology.orchestra.run.smithi179.stdout: osd recover clone overlap = true 2024-05-22T21:12:54.781 INFO:teuthology.orchestra.run.smithi179.stdout: osd recovery max chunk = 1048576 2024-05-22T21:12:54.781 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.781 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug shutdown = true 2024-05-22T21:12:54.781 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug op order = true 2024-05-22T21:12:54.781 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug verify stray on activate = true 2024-05-22T21:12:54.781 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.781 INFO:teuthology.orchestra.run.smithi179.stdout: osd open classes on start = true 2024-05-22T21:12:54.781 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug pg log writeout = true 2024-05-22T21:12:54.782 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.782 INFO:teuthology.orchestra.run.smithi179.stdout: osd deep scrub update digest min age = 30 2024-05-22T21:12:54.782 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.782 INFO:teuthology.orchestra.run.smithi179.stdout: osd map max advance = 10 2024-05-22T21:12:54.782 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.782 INFO:teuthology.orchestra.run.smithi179.stdout: journal zero on create = true 2024-05-22T21:12:54.782 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.782 INFO:teuthology.orchestra.run.smithi179.stdout: filestore ondisk finisher threads = 3 2024-05-22T21:12:54.782 INFO:teuthology.orchestra.run.smithi179.stdout: filestore apply finisher threads = 3 2024-05-22T21:12:54.782 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.782 INFO:teuthology.orchestra.run.smithi179.stdout: bdev debug aio = true 2024-05-22T21:12:54.782 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug misdirected ops = true 2024-05-22T21:12:54.783 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore block size = 96636764160 2024-05-22T21:12:54.783 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore fsck on mount = True 2024-05-22T21:12:54.783 INFO:teuthology.orchestra.run.smithi179.stdout: debug bluefs = 20 2024-05-22T21:12:54.783 INFO:teuthology.orchestra.run.smithi179.stdout: debug bluestore = 20 2024-05-22T21:12:54.783 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-22T21:12:54.783 INFO:teuthology.orchestra.run.smithi179.stdout: debug osd = 20 2024-05-22T21:12:54.783 INFO:teuthology.orchestra.run.smithi179.stdout: debug rocksdb = 10 2024-05-22T21:12:54.783 INFO:teuthology.orchestra.run.smithi179.stdout: memstore device bytes = 200000000 2024-05-22T21:12:54.783 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd backfillfull_ratio = 0.85 2024-05-22T21:12:54.783 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd full ratio = 0.9 2024-05-22T21:12:54.783 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd nearfull ratio = 0.8 2024-05-22T21:12:54.783 INFO:teuthology.orchestra.run.smithi179.stdout: osd failsafe full ratio = 1.0 2024-05-22T21:12:54.784 INFO:teuthology.orchestra.run.smithi179.stdout: osd mon report interval = 5 2024-05-22T21:12:54.784 INFO:teuthology.orchestra.run.smithi179.stdout: osd objectstore = memstore 2024-05-22T21:12:54.784 INFO:teuthology.orchestra.run.smithi179.stdout: osd op complaint time = 180 2024-05-22T21:12:54.784 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.784 INFO:teuthology.orchestra.run.smithi179.stdout:[mgr] 2024-05-22T21:12:54.784 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-22T21:12:54.784 INFO:teuthology.orchestra.run.smithi179.stdout: debug mgr = 20 2024-05-22T21:12:54.784 INFO:teuthology.orchestra.run.smithi179.stdout: debug mon = 20 2024-05-22T21:12:54.784 INFO:teuthology.orchestra.run.smithi179.stdout: debug auth = 20 2024-05-22T21:12:54.784 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min pgs per osd = 4 2024-05-22T21:12:54.784 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min bytes per osd = 10 2024-05-22T21:12:54.784 INFO:teuthology.orchestra.run.smithi179.stdout: mgr/telemetry/nag = false 2024-05-22T21:12:54.784 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.785 INFO:teuthology.orchestra.run.smithi179.stdout:[mon] 2024-05-22T21:12:54.785 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-22T21:12:54.785 INFO:teuthology.orchestra.run.smithi179.stdout: debug mon = 20 2024-05-22T21:12:54.785 INFO:teuthology.orchestra.run.smithi179.stdout: debug paxos = 20 2024-05-22T21:12:54.785 INFO:teuthology.orchestra.run.smithi179.stdout: debug auth = 20 2024-05-22T21:12:54.785 INFO:teuthology.orchestra.run.smithi179.stdout: mon data avail warn = 5 2024-05-22T21:12:54.785 INFO:teuthology.orchestra.run.smithi179.stdout: mon mgr mkfs grace = 240 2024-05-22T21:12:54.785 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min pgs per osd = 4 2024-05-22T21:12:54.785 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd reporter subtree level = osd 2024-05-22T21:12:54.785 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd prime pg temp = true 2024-05-22T21:12:54.785 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min bytes per osd = 10 2024-05-22T21:12:54.785 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.785 INFO:teuthology.orchestra.run.smithi179.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-22T21:12:54.786 INFO:teuthology.orchestra.run.smithi179.stdout: auth mon ticket ttl = 660 # 11m 2024-05-22T21:12:54.786 INFO:teuthology.orchestra.run.smithi179.stdout: auth service ticket ttl = 240 # 4m 2024-05-22T21:12:54.786 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.786 INFO:teuthology.orchestra.run.smithi179.stdout: # don't complain about insecure global_id in the test suite 2024-05-22T21:12:54.786 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-22T21:12:54.786 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-22T21:12:54.786 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.786 INFO:teuthology.orchestra.run.smithi179.stdout: # 1m isn't quite enough 2024-05-22T21:12:54.786 INFO:teuthology.orchestra.run.smithi179.stdout: mon_down_mkfs_grace = 2m 2024-05-22T21:12:54.787 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.787 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_filestore_osds = false 2024-05-22T21:12:54.787 INFO:teuthology.orchestra.run.smithi179.stdout: mon op complaint time = 120 2024-05-22T21:12:54.787 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd backfillfull ratio = 0.6 2024-05-22T21:12:54.787 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd full ratio = 0.7 2024-05-22T21:12:54.787 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd nearfull ratio = 0.6 2024-05-22T21:12:54.787 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-22T21:12:54.787 INFO:teuthology.orchestra.run.smithi179.stdout:[client] 2024-05-22T21:12:54.787 INFO:teuthology.orchestra.run.smithi179.stdout: rgw cache enabled = true 2024-05-22T21:12:54.787 INFO:teuthology.orchestra.run.smithi179.stdout: rgw enable ops log = true 2024-05-22T21:12:54.787 INFO:teuthology.orchestra.run.smithi179.stdout: rgw enable usage log = true 2024-05-22T21:12:54.788 INFO:teuthology.orchestra.run.smithi179.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-22T21:12:54.788 INFO:teuthology.orchestra.run.smithi179.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-22T21:12:54.788 INFO:teuthology.orchestra.run.smithi179.stdout: client mount timeout = 600 2024-05-22T21:12:54.788 INFO:teuthology.orchestra.run.smithi179.stdout: debug client = 20 2024-05-22T21:12:54.788 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-22T21:12:54.788 INFO:teuthology.orchestra.run.smithi179.stdout: pid file = "" 2024-05-22T21:12:54.788 INFO:teuthology.orchestra.run.smithi179.stdout: rados mon op timeout = 900 2024-05-22T21:12:54.788 INFO:teuthology.orchestra.run.smithi179.stdout: rados osd op timeout = 900 2024-05-22T21:12:54.788 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.a] 2024-05-22T21:12:54.788 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.b] 2024-05-22T21:12:54.789 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.c] 2024-05-22T21:12:54.789 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.a] 2024-05-22T21:12:54.789 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.c] 2024-05-22T21:12:54.789 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.b] 2024-05-22T21:12:54.789 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.d] 2024-05-22T21:12:54.789 INFO:teuthology.orchestra.run.smithi179.stdout:[client.0] 2024-05-22T21:12:54.789 INFO:teuthology.orchestra.run.smithi179.stdout: debug client = 20 2024-05-22T21:12:54.789 INFO:teuthology.orchestra.run.smithi179.stdout: debug objectcacher = 20 2024-05-22T21:12:54.789 INFO:teuthology.orchestra.run.smithi179.stdout: debug objecter = 20 2024-05-22T21:12:54.789 INFO:teuthology.orchestra.run.smithi179.stdout:[client.1] 2024-05-22T21:12:54.789 INFO:teuthology.orchestra.run.smithi179.stdout: debug client = 20 2024-05-22T21:12:54.789 INFO:teuthology.orchestra.run.smithi179.stdout: debug objectcacher = 20 2024-05-22T21:12:54.789 INFO:teuthology.orchestra.run.smithi179.stdout: debug objecter = 20 2024-05-22T21:12:54.789 INFO:teuthology.orchestra.run.smithi179.stdout:[mds] 2024-05-22T21:12:54.790 INFO:teuthology.orchestra.run.smithi179.stdout: debug mds = 20 2024-05-22T21:12:54.790 INFO:teuthology.orchestra.run.smithi179.stdout: debug mds balancer = 20 2024-05-22T21:12:54.790 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-05-22T21:12:54.790 INFO:teuthology.orchestra.run.smithi179.stdout: mds debug frag = True 2024-05-22T21:12:54.790 INFO:teuthology.orchestra.run.smithi179.stdout: mds debug scatterstat = True 2024-05-22T21:12:54.790 INFO:teuthology.orchestra.run.smithi179.stdout: mds op complaint time = 180 2024-05-22T21:12:54.790 INFO:teuthology.orchestra.run.smithi179.stdout: mds verify scatter = True 2024-05-22T21:12:54.790 INFO:teuthology.orchestra.run.smithi179.stdout: osd op complaint time = 180 2024-05-22T21:12:54.790 INFO:teuthology.orchestra.run.smithi179.stdout: rados mon op timeout = 900 2024-05-22T21:12:54.790 INFO:teuthology.orchestra.run.smithi179.stdout: rados osd op timeout = 900 2024-05-22T21:12:54.792 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-22T21:12:54.792 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-22T21:12:54.894 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-22T21:12:54.895 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:12:54.895 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-22T21:12:54.903 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:12:54.903 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-22T21:12:54.959 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:12:54.959 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:12:54.959 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-22T21:12:54.959 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-22T21:12:55.033 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:12:55.033 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-22T21:12:55.082 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:12:55.082 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:12:55.082 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-22T21:12:55.082 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-22T21:12:55.117 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:12:55.117 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-22T21:12:55.168 INFO:tasks.ceph:Setting up mon nodes... 2024-05-22T21:12:55.168 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-22T21:12:55.168 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-22T21:12:55.222 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-22T21:12:55.227 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-22T21:12:55.285 INFO:teuthology.orchestra.run.smithi179.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-22T21:12:55.297 INFO:tasks.ceph:Setting up mds nodes... 2024-05-22T21:12:55.298 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-22T21:12:55.351 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-22T21:12:55.355 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-22T21:12:55.375 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-22T21:12:55.471 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-22T21:12:55.475 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-22T21:12:55.494 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-22T21:12:55.555 INFO:teuthology.orchestra.run.smithi179.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-22T21:12:55.560 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-22T21:12:55.580 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-22T21:12:55.678 INFO:teuthology.orchestra.run.smithi179.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-22T21:12:55.683 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-22T21:12:55.702 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-22T21:12:55.703 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-22T21:12:55.743 INFO:teuthology.orchestra.run.smithi161.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-22T21:12:55.760 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-22T21:12:55.843 INFO:teuthology.orchestra.run.smithi161.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-22T21:12:55.859 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-22T21:12:55.902 INFO:teuthology.orchestra.run.smithi179.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-22T21:12:55.919 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-22T21:12:56.000 INFO:teuthology.orchestra.run.smithi179.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-22T21:12:56.017 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-22T21:12:56.017 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi161.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi179.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-22T21:12:56.017 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-22T21:12:56.036 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-22T21:12:56.036 INFO:tasks.ceph:role: osd.0 2024-05-22T21:12:56.036 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:12:56.036 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-22T21:12:56.104 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-22T21:12:56.104 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-22T21:12:56.104 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-22T21:12:56.104 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-22T21:12:56.104 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-22T21:12:56.104 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2024-05-22T21:12:56.104 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-22T21:12:56.104 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-22T21:12:56.104 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-22T21:12:56.105 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-22T21:12:56.487 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2024-05-22T21:12:56.490 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2024-05-22T21:12:56.490 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-22T21:12:56.570 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-22T21:12:56.626 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: /sbin/restorecon: command not found 2024-05-22T21:12:56.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:12:56.627 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-22T21:12:56.689 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-22T21:12:56.689 INFO:tasks.ceph:role: osd.1 2024-05-22T21:12:56.689 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:12:56.689 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-22T21:12:56.761 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-22T21:12:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-22T21:12:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-22T21:12:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-22T21:12:56.762 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-22T21:12:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2024-05-22T21:12:56.762 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-22T21:12:56.762 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-22T21:12:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-22T21:12:56.762 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-22T21:12:57.168 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2024-05-22T21:12:57.171 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2024-05-22T21:12:57.171 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-22T21:12:57.297 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-22T21:12:57.312 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: /sbin/restorecon: command not found 2024-05-22T21:12:57.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:12:57.314 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-22T21:12:57.377 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-22T21:12:57.377 INFO:tasks.ceph:role: osd.2 2024-05-22T21:12:57.377 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:12:57.378 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-22T21:12:57.445 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-22T21:12:57.445 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-22T21:12:57.445 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-22T21:12:57.445 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-22T21:12:57.445 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-22T21:12:57.446 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2024-05-22T21:12:57.446 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-22T21:12:57.446 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-22T21:12:57.446 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-22T21:12:57.446 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-22T21:12:57.851 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2024-05-22T21:12:57.854 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2024-05-22T21:12:57.854 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-22T21:12:57.990 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-22T21:12:58.008 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: /sbin/restorecon: command not found 2024-05-22T21:12:58.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:12:58.009 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-22T21:12:58.072 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-22T21:12:58.072 INFO:tasks.ceph:role: osd.3 2024-05-22T21:12:58.072 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:12:58.072 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-22T21:12:58.143 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-22T21:12:58.143 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-22T21:12:58.145 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-22T21:12:58.145 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-22T21:12:58.145 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-22T21:12:58.145 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2024-05-22T21:12:58.145 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-22T21:12:58.145 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-22T21:12:58.146 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-22T21:12:58.146 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-22T21:12:58.498 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2024-05-22T21:12:58.501 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2024-05-22T21:12:58.501 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-22T21:12:58.577 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-22T21:12:58.634 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: /sbin/restorecon: command not found 2024-05-22T21:12:58.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:12:58.635 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-22T21:12:58.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:12:58.772+0000 7f7b28c2b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-22T21:12:58.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:12:58.772+0000 7f7b28c2b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-22T21:12:58.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:12:58.780+0000 7f7b28c2b7c0 -1 memstore(/var/lib/ceph/osd/ceph-0) /var/lib/ceph/osd/ceph-0 2024-05-22T21:12:58.800 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-22T21:12:58.820 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-22T21:12:58.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:12:58.912+0000 7fb62cf207c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-22T21:12:58.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:12:58.912+0000 7fb62cf207c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-22T21:12:58.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:12:58.916+0000 7fb62cf207c0 -1 memstore(/var/lib/ceph/osd/ceph-1) /var/lib/ceph/osd/ceph-1 2024-05-22T21:12:58.938 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-22T21:12:58.958 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-22T21:12:59.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:12:59.052+0000 7f2bb45077c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-22T21:12:59.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:12:59.052+0000 7f2bb45077c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-22T21:12:59.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:12:59.056+0000 7f2bb45077c0 -1 memstore(/var/lib/ceph/osd/ceph-2) /var/lib/ceph/osd/ceph-2 2024-05-22T21:12:59.078 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-22T21:12:59.098 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-22T21:12:59.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:12:59.188+0000 7f67a1a2f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-22T21:12:59.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:12:59.192+0000 7f67a1a2f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-22T21:12:59.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:12:59.204+0000 7f67a1a2f7c0 -1 memstore(/var/lib/ceph/osd/ceph-3) /var/lib/ceph/osd/ceph-3 2024-05-22T21:12:59.232 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-22T21:12:59.252 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-22T21:12:59.273 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-22T21:12:59.273 INFO:tasks.ceph:role: osd.4 2024-05-22T21:12:59.273 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:12:59.273 DEBUG:teuthology.orchestra.run.smithi179:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-22T21:12:59.344 INFO:teuthology.orchestra.run.smithi179.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-22T21:12:59.344 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-22T21:12:59.344 INFO:teuthology.orchestra.run.smithi179.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-22T21:12:59.344 INFO:teuthology.orchestra.run.smithi179.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-22T21:12:59.344 INFO:teuthology.orchestra.run.smithi179.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-22T21:12:59.345 INFO:teuthology.orchestra.run.smithi179.stdout: = sunit=0 swidth=0 blks 2024-05-22T21:12:59.345 INFO:teuthology.orchestra.run.smithi179.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-22T21:12:59.345 INFO:teuthology.orchestra.run.smithi179.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-22T21:12:59.345 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-22T21:12:59.345 INFO:teuthology.orchestra.run.smithi179.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-22T21:12:59.737 INFO:teuthology.orchestra.run.smithi179.stdout:Discarding blocks...Done. 2024-05-22T21:12:59.741 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi179.front.sepia.ceph.com -o noatime 2024-05-22T21:12:59.741 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-22T21:12:59.840 DEBUG:teuthology.orchestra.run.smithi179:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-22T21:12:59.856 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: /sbin/restorecon: command not found 2024-05-22T21:12:59.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:12:59.858 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-22T21:12:59.919 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-22T21:12:59.920 INFO:tasks.ceph:role: osd.5 2024-05-22T21:12:59.920 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:12:59.920 DEBUG:teuthology.orchestra.run.smithi179:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-22T21:12:59.991 INFO:teuthology.orchestra.run.smithi179.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-22T21:12:59.991 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-22T21:12:59.992 INFO:teuthology.orchestra.run.smithi179.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-22T21:12:59.992 INFO:teuthology.orchestra.run.smithi179.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-22T21:12:59.992 INFO:teuthology.orchestra.run.smithi179.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-22T21:12:59.992 INFO:teuthology.orchestra.run.smithi179.stdout: = sunit=0 swidth=0 blks 2024-05-22T21:12:59.992 INFO:teuthology.orchestra.run.smithi179.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-22T21:12:59.992 INFO:teuthology.orchestra.run.smithi179.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-22T21:12:59.992 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-22T21:12:59.992 INFO:teuthology.orchestra.run.smithi179.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-22T21:13:00.397 INFO:teuthology.orchestra.run.smithi179.stdout:Discarding blocks...Done. 2024-05-22T21:13:00.400 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi179.front.sepia.ceph.com -o noatime 2024-05-22T21:13:00.400 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-22T21:13:00.528 DEBUG:teuthology.orchestra.run.smithi179:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-22T21:13:00.544 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: /sbin/restorecon: command not found 2024-05-22T21:13:00.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:13:00.545 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-22T21:13:00.611 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-22T21:13:00.611 INFO:tasks.ceph:role: osd.6 2024-05-22T21:13:00.611 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:13:00.611 DEBUG:teuthology.orchestra.run.smithi179:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-22T21:13:00.683 INFO:teuthology.orchestra.run.smithi179.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-22T21:13:00.683 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-22T21:13:00.683 INFO:teuthology.orchestra.run.smithi179.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-22T21:13:00.683 INFO:teuthology.orchestra.run.smithi179.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-22T21:13:00.684 INFO:teuthology.orchestra.run.smithi179.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-22T21:13:00.684 INFO:teuthology.orchestra.run.smithi179.stdout: = sunit=0 swidth=0 blks 2024-05-22T21:13:00.684 INFO:teuthology.orchestra.run.smithi179.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-22T21:13:00.684 INFO:teuthology.orchestra.run.smithi179.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-22T21:13:00.684 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-22T21:13:00.684 INFO:teuthology.orchestra.run.smithi179.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-22T21:13:01.099 INFO:teuthology.orchestra.run.smithi179.stdout:Discarding blocks...Done. 2024-05-22T21:13:01.102 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi179.front.sepia.ceph.com -o noatime 2024-05-22T21:13:01.102 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-22T21:13:01.221 DEBUG:teuthology.orchestra.run.smithi179:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-22T21:13:01.238 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: /sbin/restorecon: command not found 2024-05-22T21:13:01.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:13:01.240 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-22T21:13:01.308 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-22T21:13:01.309 INFO:tasks.ceph:role: osd.7 2024-05-22T21:13:01.309 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:13:01.309 DEBUG:teuthology.orchestra.run.smithi179:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-22T21:13:01.379 INFO:teuthology.orchestra.run.smithi179.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-22T21:13:01.379 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-22T21:13:01.379 INFO:teuthology.orchestra.run.smithi179.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-22T21:13:01.379 INFO:teuthology.orchestra.run.smithi179.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-22T21:13:01.379 INFO:teuthology.orchestra.run.smithi179.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-22T21:13:01.379 INFO:teuthology.orchestra.run.smithi179.stdout: = sunit=0 swidth=0 blks 2024-05-22T21:13:01.379 INFO:teuthology.orchestra.run.smithi179.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-22T21:13:01.380 INFO:teuthology.orchestra.run.smithi179.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-22T21:13:01.380 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-22T21:13:01.380 INFO:teuthology.orchestra.run.smithi179.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-22T21:13:01.743 INFO:teuthology.orchestra.run.smithi179.stdout:Discarding blocks...Done. 2024-05-22T21:13:01.746 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi179.front.sepia.ceph.com -o noatime 2024-05-22T21:13:01.746 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-22T21:13:01.821 DEBUG:teuthology.orchestra.run.smithi179:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-22T21:13:01.880 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: /sbin/restorecon: command not found 2024-05-22T21:13:01.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:13:01.882 DEBUG:teuthology.orchestra.run.smithi179:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-22T21:13:02.010 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:02.008+0000 7fe2acc397c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-22T21:13:02.013 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:02.012+0000 7fe2acc397c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-22T21:13:02.021 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:02.020+0000 7fe2acc397c0 -1 memstore(/var/lib/ceph/osd/ceph-4) /var/lib/ceph/osd/ceph-4 2024-05-22T21:13:02.041 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-22T21:13:02.062 DEBUG:teuthology.orchestra.run.smithi179:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-22T21:13:02.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:02.152+0000 7f20c15a17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-22T21:13:02.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:02.152+0000 7f20c15a17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-22T21:13:02.164 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:02.164+0000 7f20c15a17c0 -1 memstore(/var/lib/ceph/osd/ceph-5) /var/lib/ceph/osd/ceph-5 2024-05-22T21:13:02.184 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-22T21:13:02.203 DEBUG:teuthology.orchestra.run.smithi179:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-22T21:13:02.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:02.292+0000 7fcdfe5967c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-22T21:13:02.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:02.296+0000 7fcdfe5967c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-22T21:13:02.308 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:02.304+0000 7fcdfe5967c0 -1 memstore(/var/lib/ceph/osd/ceph-6) /var/lib/ceph/osd/ceph-6 2024-05-22T21:13:02.327 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-22T21:13:02.346 DEBUG:teuthology.orchestra.run.smithi179:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-22T21:13:02.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:02.432+0000 7fbce30a47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-22T21:13:02.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:02.440+0000 7fbce30a47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-22T21:13:02.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:02.448+0000 7fbce30a47c0 -1 memstore(/var/lib/ceph/osd/ceph-7) /var/lib/ceph/osd/ceph-7 2024-05-22T21:13:02.477 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-22T21:13:02.496 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-22T21:13:02.496 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:02.497 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-22T21:13:02.519 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:02.519 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-22T21:13:02.583 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:02.583 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-22T21:13:02.651 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:02.651 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-22T21:13:02.715 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:02.715 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-22T21:13:02.778 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:02.778 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-22T21:13:02.842 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:02.842 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-22T21:13:02.902 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:02.902 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-22T21:13:02.922 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:02.922 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-22T21:13:02.982 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:02.982 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-22T21:13:03.042 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:03.043 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-22T21:13:03.102 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:03.103 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-22T21:13:03.162 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:03.162 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-22T21:13:03.222 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:03.222 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-22T21:13:03.282 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:03.282 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-22T21:13:03.291 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:03.291 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-22T21:13:03.343 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:03.343 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-22T21:13:03.351 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:03.351 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-22T21:13:03.399 INFO:tasks.ceph:Adding keys to all mons... 2024-05-22T21:13:03.400 DEBUG:teuthology.orchestra.run.smithi161:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-22T21:13:03.402 DEBUG:teuthology.orchestra.run.smithi179:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-22T21:13:03.447 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr.y] 2024-05-22T21:13:03.448 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDXX05mZRdiDRAA0vUwD+rPSfyhjgA+9XPp5w== 2024-05-22T21:13:03.448 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.a] 2024-05-22T21:13:03.448 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDXX05m00QTFRAAPSRrpBRY75FlcWf3QxOOfg== 2024-05-22T21:13:03.448 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.c] 2024-05-22T21:13:03.448 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDXX05mWlM2HBAAEmJJWMDlTG9E7Qzld2gd5w== 2024-05-22T21:13:03.448 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.0] 2024-05-22T21:13:03.448 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDaX05mJZokLhAAh2vbQnRX25iYUIbvVmURuQ== 2024-05-22T21:13:03.448 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.1] 2024-05-22T21:13:03.449 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDaX05m8kWNNhAA6aaVT2xshFkqnk4G4gOttA== 2024-05-22T21:13:03.449 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.2] 2024-05-22T21:13:03.449 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDbX05ml1k9AxAAtL3XaJJI6cilpmvurh18Pw== 2024-05-22T21:13:03.449 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.3] 2024-05-22T21:13:03.449 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDbX05mbNBgCxAAas1L+QR4Aez+H1G0EVltfg== 2024-05-22T21:13:03.449 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr.x] 2024-05-22T21:13:03.449 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDXX05mH2kuERAAgigMUWK3M3WaMXlqGUcNrw== 2024-05-22T21:13:03.449 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.b] 2024-05-22T21:13:03.449 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDXX05m/ipAIRAA4gj5jGqTpNCr1K2OU/bsAQ== 2024-05-22T21:13:03.450 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.d] 2024-05-22T21:13:03.450 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDXX05mmsuaKBAAqpCYRZZUGK5ZuxflUBcPgg== 2024-05-22T21:13:03.450 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.4] 2024-05-22T21:13:03.450 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDeX05mOPLPABAAZBGIMF/FasNB27n/7di0AA== 2024-05-22T21:13:03.450 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.5] 2024-05-22T21:13:03.450 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDeX05mT1poCRAASXm4tdP01pA1VOviqeoLVw== 2024-05-22T21:13:03.450 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.6] 2024-05-22T21:13:03.451 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDeX05m1ZPRERAA2S7WeTwysLCZm1OB/qCrAA== 2024-05-22T21:13:03.451 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.7] 2024-05-22T21:13:03.451 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDeX05mCrUhGhAAvXoFWV7X3303rfXaxmfWqA== 2024-05-22T21:13:03.451 INFO:teuthology.orchestra.run.smithi161.stdout:[client.0] 2024-05-22T21:13:03.451 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDXX05mh9B7LBAAevhvj5KpwP9pEIpJcxUvFA== 2024-05-22T21:13:03.451 INFO:teuthology.orchestra.run.smithi161.stdout:[client.2] 2024-05-22T21:13:03.451 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDXX05mOzdkMhAAbTiNdvyxqIyrluMiZW0cOw== 2024-05-22T21:13:03.451 INFO:teuthology.orchestra.run.smithi161.stdout:[client.1] 2024-05-22T21:13:03.451 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDXX05mMnHtNRAAg/a/Mb1oRqx73d9efIHCfA== 2024-05-22T21:13:03.451 INFO:teuthology.orchestra.run.smithi161.stdout:[client.3] 2024-05-22T21:13:03.452 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDYX05mr4oxABAA1+bQQbxhoeU+kb0OCa+Wyw== 2024-05-22T21:13:03.463 INFO:teuthology.orchestra.run.smithi179.stdout:[mgr.y] 2024-05-22T21:13:03.463 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDXX05mZRdiDRAA0vUwD+rPSfyhjgA+9XPp5w== 2024-05-22T21:13:03.463 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.a] 2024-05-22T21:13:03.464 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDXX05m00QTFRAAPSRrpBRY75FlcWf3QxOOfg== 2024-05-22T21:13:03.464 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.c] 2024-05-22T21:13:03.464 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDXX05mWlM2HBAAEmJJWMDlTG9E7Qzld2gd5w== 2024-05-22T21:13:03.464 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.0] 2024-05-22T21:13:03.464 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDaX05mJZokLhAAh2vbQnRX25iYUIbvVmURuQ== 2024-05-22T21:13:03.464 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.1] 2024-05-22T21:13:03.464 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDaX05m8kWNNhAA6aaVT2xshFkqnk4G4gOttA== 2024-05-22T21:13:03.464 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.2] 2024-05-22T21:13:03.464 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDbX05ml1k9AxAAtL3XaJJI6cilpmvurh18Pw== 2024-05-22T21:13:03.464 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.3] 2024-05-22T21:13:03.464 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDbX05mbNBgCxAAas1L+QR4Aez+H1G0EVltfg== 2024-05-22T21:13:03.464 INFO:teuthology.orchestra.run.smithi179.stdout:[mgr.x] 2024-05-22T21:13:03.464 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDXX05mH2kuERAAgigMUWK3M3WaMXlqGUcNrw== 2024-05-22T21:13:03.464 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.b] 2024-05-22T21:13:03.465 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDXX05m/ipAIRAA4gj5jGqTpNCr1K2OU/bsAQ== 2024-05-22T21:13:03.465 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.d] 2024-05-22T21:13:03.465 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDXX05mmsuaKBAAqpCYRZZUGK5ZuxflUBcPgg== 2024-05-22T21:13:03.465 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.4] 2024-05-22T21:13:03.465 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDeX05mOPLPABAAZBGIMF/FasNB27n/7di0AA== 2024-05-22T21:13:03.465 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.5] 2024-05-22T21:13:03.465 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDeX05mT1poCRAASXm4tdP01pA1VOviqeoLVw== 2024-05-22T21:13:03.465 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.6] 2024-05-22T21:13:03.465 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDeX05m1ZPRERAA2S7WeTwysLCZm1OB/qCrAA== 2024-05-22T21:13:03.465 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.7] 2024-05-22T21:13:03.465 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDeX05mCrUhGhAAvXoFWV7X3303rfXaxmfWqA== 2024-05-22T21:13:03.465 INFO:teuthology.orchestra.run.smithi179.stdout:[client.0] 2024-05-22T21:13:03.465 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDXX05mh9B7LBAAevhvj5KpwP9pEIpJcxUvFA== 2024-05-22T21:13:03.466 INFO:teuthology.orchestra.run.smithi179.stdout:[client.2] 2024-05-22T21:13:03.466 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDXX05mOzdkMhAAbTiNdvyxqIyrluMiZW0cOw== 2024-05-22T21:13:03.466 INFO:teuthology.orchestra.run.smithi179.stdout:[client.1] 2024-05-22T21:13:03.466 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDXX05mMnHtNRAAg/a/Mb1oRqx73d9efIHCfA== 2024-05-22T21:13:03.466 INFO:teuthology.orchestra.run.smithi179.stdout:[client.3] 2024-05-22T21:13:03.466 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQDYX05mr4oxABAA1+bQQbxhoeU+kb0OCa+Wyw== 2024-05-22T21:13:03.466 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-22T21:13:03.501 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-22T21:13:03.557 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-22T21:13:03.561 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-22T21:13:03.617 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-22T21:13:03.620 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-22T21:13:03.675 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-22T21:13:03.679 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-22T21:13:03.733 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-22T21:13:03.736 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-22T21:13:03.788 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-22T21:13:03.792 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-22T21:13:03.844 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-22T21:13:03.847 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-22T21:13:03.900 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-22T21:13:03.903 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-22T21:13:03.955 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-22T21:13:03.959 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-22T21:13:04.016 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-22T21:13:04.019 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-22T21:13:04.076 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-22T21:13:04.079 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-22T21:13:04.131 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-22T21:13:04.133 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-22T21:13:04.186 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-22T21:13:04.189 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-22T21:13:04.243 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-22T21:13:04.246 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-22T21:13:04.299 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-22T21:13:04.301 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-22T21:13:04.357 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-22T21:13:04.360 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-22T21:13:04.416 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-22T21:13:04.420 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-22T21:13:04.475 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-22T21:13:04.478 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-22T21:13:04.533 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-22T21:13:04.533 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-22T21:13:04.553 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-22T21:13:04.665 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-22T21:13:04.684 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-22T21:13:04.703 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-22T21:13:04.815 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-22T21:13:04.834 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-22T21:13:04.898 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-22T21:13:05.005 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-22T21:13:05.025 DEBUG:teuthology.orchestra.run.smithi161:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-22T21:13:05.028 DEBUG:teuthology.orchestra.run.smithi179:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-22T21:13:05.087 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-22T21:13:05.087 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-22T21:13:05.087 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-22T21:13:05.090 INFO:tasks.ceph.mon.a:Started 2024-05-22T21:13:05.090 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-22T21:13:05.090 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-22T21:13:05.130 INFO:tasks.ceph.mon.b:Started 2024-05-22T21:13:05.131 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-22T21:13:05.131 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-22T21:13:05.134 INFO:tasks.ceph.mon.c:Started 2024-05-22T21:13:05.134 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-22T21:13:05.134 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-22T21:13:05.134 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-22T21:13:05.137 INFO:tasks.ceph.mgr.y:Started 2024-05-22T21:13:05.137 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-22T21:13:05.137 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-22T21:13:05.140 INFO:tasks.ceph.mgr.x:Started 2024-05-22T21:13:05.140 INFO:tasks.ceph:Setting crush tunables to default 2024-05-22T21:13:05.140 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd crush tunables default 2024-05-22T21:13:05.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3d6400640 1 Processor -- start 2024-05-22T21:13:05.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3d6400640 1 -- start start 2024-05-22T21:13:05.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3d6400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe3d0160ad0 0x7fe3d0099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:05.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3d6400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00d0bb0 0x7fe3d0099f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:05.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3d6400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d006e4a0 con 0x7fe3d00b37b0 2024-05-22T21:13:05.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3d6400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3d006c690 con 0x7fe3d0160ad0 2024-05-22T21:13:05.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3d6400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d006c0f0 con 0x7fe3d00d0bb0 2024-05-22T21:13:05.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3cf400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe3d0160ad0 0x7fe3d0099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:05.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3cf400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe3d0160ad0 0x7fe3d0099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38950/0 (socket says 172.21.15.161:38950) 2024-05-22T21:13:05.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3cf400640 1 -- 172.21.15.161:0/2090007969 learned_addr learned my addr 172.21.15.161:0/2090007969 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:05.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3cf400640 1 -- 172.21.15.161:0/2090007969 >> v2:172.21.15.161:3300/0 conn(0x7fe3d0160ad0 msgr2=0x7fe3d0099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-22T21:13:05.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3cf400640 1 -- 172.21.15.161:0/2090007969 >> v2:172.21.15.161:3300/0 conn(0x7fe3d0160ad0 msgr2=0x7fe3d0099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-22T21:13:05.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3cf400640 1 --2- 172.21.15.161:0/2090007969 >> v2:172.21.15.161:3300/0 conn(0x7fe3d0160ad0 0x7fe3d0099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-22T21:13:05.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3cf400640 1 --2- 172.21.15.161:0/2090007969 >> v2:172.21.15.161:3300/0 conn(0x7fe3d0160ad0 0x7fe3d0099a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T21:13:05.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3cea00640 1 --2- 172.21.15.161:0/2090007969 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00d0bb0 0x7fe3d0099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:05.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3cea00640 1 -- 172.21.15.161:0/2090007969 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00d0bb0 msgr2=0x7fe3d0099f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-22T21:13:05.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3cea00640 1 -- 172.21.15.161:0/2090007969 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00d0bb0 msgr2=0x7fe3d0099f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-22T21:13:05.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3cea00640 1 --2- 172.21.15.161:0/2090007969 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00d0bb0 0x7fe3d0099f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-22T21:13:05.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.280+0000 7fe3cea00640 1 --2- 172.21.15.161:0/2090007969 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00d0bb0 0x7fe3d0099f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-22T21:13:05.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.428+0000 7fe3ce000640 1 -- 172.21.15.161:0/2090007969 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743439234 0 0) 0x7fe3d006e4a0 con 0x7fe3d00b37b0 2024-05-22T21:13:05.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.428+0000 7fe3ce000640 1 -- 172.21.15.161:0/2090007969 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3b4003530 con 0x7fe3d00b37b0 2024-05-22T21:13:05.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.428+0000 7fe3ce000640 1 -- 172.21.15.161:0/2090007969 <== mon.1 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 316625781 0 0) 0x7fe3b4003530 con 0x7fe3d00b37b0 2024-05-22T21:13:05.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.428+0000 7fe3ce000640 1 -- 172.21.15.161:0/2090007969 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00d0bb0 msgr2=0x7fe3d0099f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-22T21:13:05.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.428+0000 7fe3ce000640 1 --2- 172.21.15.161:0/2090007969 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00d0bb0 0x7fe3d0099f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:05.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.428+0000 7fe3ce000640 1 -- 172.21.15.161:0/2090007969 >> v2:172.21.15.161:3300/0 conn(0x7fe3d0160ad0 msgr2=0x7fe3d0099a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-22T21:13:05.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.428+0000 7fe3ce000640 1 --2- 172.21.15.161:0/2090007969 >> v2:172.21.15.161:3300/0 conn(0x7fe3d0160ad0 0x7fe3d0099a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:05.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.428+0000 7fe3ce000640 1 -- 172.21.15.161:0/2090007969 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d006cf50 con 0x7fe3d00b37b0 2024-05-22T21:13:05.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.428+0000 7fe3ce000640 1 -- 172.21.15.161:0/2090007969 <== mon.1 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fe3c8001680 con 0x7fe3d00b37b0 2024-05-22T21:13:05.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.428+0000 7fe3ce000640 1 -- 172.21.15.161:0/2090007969 >> v1:172.21.15.179:6789/0 conn(0x7fe3d00b37b0 legacy=0x7fe3d0160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:05.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.428+0000 7fe3ce000640 1 --2- 172.21.15.161:0/2090007969 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00d0bb0 0x7fe3b4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:05.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.428+0000 7fe3ce000640 1 --2- 172.21.15.161:0/2090007969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b4003fb0 0x7fe3b40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:05.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.432+0000 7fe3ce000640 1 --2- 172.21.15.161:0/2090007969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3b4004d10 0x7fe3b4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:05.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.432+0000 7fe3ce000640 1 -- 172.21.15.161:0/2090007969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d006c690 con 0x7fe3b4003fb0 2024-05-22T21:13:05.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.432+0000 7fe3ce000640 1 -- 172.21.15.161:0/2090007969 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d006c0f0 con 0x7fe3b4004d10 2024-05-22T21:13:05.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.432+0000 7fe3ce000640 1 -- 172.21.15.161:0/2090007969 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b400ba90 con 0x7fe3d00d0bb0 2024-05-22T21:13:05.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.432+0000 7fe3cea00640 1 --2- 172.21.15.161:0/2090007969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b4003fb0 0x7fe3b40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:05.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.432+0000 7fe3cf400640 1 --2- 172.21.15.161:0/2090007969 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00d0bb0 0x7fe3b4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:05.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.432+0000 7fe3cfe00640 1 --2- 172.21.15.161:0/2090007969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3b4004d10 0x7fe3b4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:05.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.432+0000 7fe3cea00640 1 -- 172.21.15.161:0/2090007969 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00d0bb0 msgr2=0x7fe3b4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:05.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.432+0000 7fe3cea00640 1 --2- 172.21.15.161:0/2090007969 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00d0bb0 0x7fe3b4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:05.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.432+0000 7fe3cea00640 1 -- 172.21.15.161:0/2090007969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3b4004d10 msgr2=0x7fe3b4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:05.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.432+0000 7fe3cea00640 1 --2- 172.21.15.161:0/2090007969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3b4004d10 0x7fe3b4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:05.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.432+0000 7fe3cea00640 1 -- 172.21.15.161:0/2090007969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b4003530 con 0x7fe3b4003fb0 2024-05-22T21:13:05.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.432+0000 7fe3cf400640 1 --2- 172.21.15.161:0/2090007969 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00d0bb0 0x7fe3b4003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:05.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.436+0000 7fe3cea00640 1 --2- 172.21.15.161:0/2090007969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b4003fb0 0x7fe3b40047d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00b9e0 tx=0x7fe3bc01c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1402db23bef71a96 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:05.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.436+0000 7fe3ce000640 1 -- 172.21.15.161:0/2090007969 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc0ab650 con 0x7fe3b4003fb0 2024-05-22T21:13:05.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.436+0000 7fe3ce000640 1 -- 172.21.15.161:0/2090007969 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3bc0ab7d0 con 0x7fe3b4003fb0 2024-05-22T21:13:05.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.436+0000 7fe3ce000640 1 -- 172.21.15.161:0/2090007969 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc0abb80 con 0x7fe3b4003fb0 2024-05-22T21:13:05.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.436+0000 7fe3d6400640 1 -- 172.21.15.161:0/2090007969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b4003fb0 msgr2=0x7fe3b40047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:05.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.436+0000 7fe3d6400640 1 --2- 172.21.15.161:0/2090007969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b4003fb0 0x7fe3b40047d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00b9e0 tx=0x7fe3bc01c640 comp rx=0 tx=0).stop 2024-05-22T21:13:05.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.436+0000 7fe3cfe00640 1 -- 172.21.15.161:0/2090007969 reap_dead start 2024-05-22T21:13:05.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.436+0000 7fe3d6400640 1 -- 172.21.15.161:0/2090007969 shutdown_connections 2024-05-22T21:13:05.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.436+0000 7fe3d6400640 1 -- 172.21.15.161:0/2090007969 >> 172.21.15.161:0/2090007969 conn(0x7fe3d00b2a80 msgr2=0x7fe3d00b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:05.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.436+0000 7fe3d6400640 1 -- 172.21.15.161:0/2090007969 shutdown_connections 2024-05-22T21:13:05.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.436+0000 7fe3d6400640 1 -- 172.21.15.161:0/2090007969 wait complete. 2024-05-22T21:13:05.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3d6400640 1 Processor -- start 2024-05-22T21:13:05.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3d6400640 1 -- start start 2024-05-22T21:13:05.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3d6400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b4003fb0 0x7fe3d01491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:05.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3d6400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d00af310 0x7fe3d00af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:05.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3d6400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00c63e0 0x7fe3d00c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:05.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3d6400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d00c7940 con 0x7fe3b4003fb0 2024-05-22T21:13:05.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3d6400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d00afcd0 con 0x7fe3d00af310 2024-05-22T21:13:05.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3d6400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d00c6dd0 con 0x7fe3d00c63e0 2024-05-22T21:13:05.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3cf400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b4003fb0 0x7fe3d01491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:05.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3cf400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b4003fb0 0x7fe3d01491e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38980/0 (socket says 172.21.15.161:38980) 2024-05-22T21:13:05.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3cf400640 1 -- 172.21.15.161:0/3210001604 learned_addr learned my addr 172.21.15.161:0/3210001604 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3cea00640 1 --2- 172.21.15.161:0/3210001604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d00af310 0x7fe3d00af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3cfe00640 1 --2- 172.21.15.161:0/3210001604 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00c63e0 0x7fe3d00c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3cf400640 1 -- 172.21.15.161:0/3210001604 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00c63e0 msgr2=0x7fe3d00c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3cf400640 1 --2- 172.21.15.161:0/3210001604 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00c63e0 0x7fe3d00c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3cf400640 1 -- 172.21.15.161:0/3210001604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d00af310 msgr2=0x7fe3d00af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3cf400640 1 --2- 172.21.15.161:0/3210001604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d00af310 0x7fe3d00af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3cf400640 1 -- 172.21.15.161:0/3210001604 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d00c70a0 con 0x7fe3b4003fb0 2024-05-22T21:13:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3cf400640 1 --2- 172.21.15.161:0/3210001604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b4003fb0 0x7fe3d01491e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0007e00 tx=0x7fe3c000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3c7400640 1 -- 172.21.15.161:0/3210001604 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0019070 con 0x7fe3b4003fb0 2024-05-22T21:13:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3c7400640 1 -- 172.21.15.161:0/3210001604 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c000fe40 con 0x7fe3b4003fb0 2024-05-22T21:13:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3c7400640 1 -- 172.21.15.161:0/3210001604 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0003bc0 con 0x7fe3b4003fb0 2024-05-22T21:13:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3d6400640 1 -- 172.21.15.161:0/3210001604 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d00c7380 con 0x7fe3b4003fb0 2024-05-22T21:13:05.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3d6400640 1 -- 172.21.15.161:0/3210001604 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d00b3d50 con 0x7fe3b4003fb0 2024-05-22T21:13:05.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3c7400640 1 -- 172.21.15.161:0/3210001604 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fe3c0003d20 con 0x7fe3b4003fb0 2024-05-22T21:13:05.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.440+0000 7fe3c7400640 1 -- 172.21.15.161:0/3210001604 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe3c0015470 con 0x7fe3b4003fb0 2024-05-22T21:13:05.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.444+0000 7fe3d6400640 1 -- 172.21.15.161:0/3210001604 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d00b3bb0 con 0x7fe3b4003fb0 2024-05-22T21:13:05.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.448+0000 7fe3c7400640 1 -- 172.21.15.161:0/3210001604 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fe3d00b3bb0 con 0x7fe3b4003fb0 2024-05-22T21:13:05.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.516+0000 7fe3d6400640 1 -- 172.21.15.161:0/3210001604 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fe3d0073ca0 con 0x7fe3b4003fb0 2024-05-22T21:13:05.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.520+0000 7fe3c7400640 1 -- 172.21.15.161:0/3210001604 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe3c0004840 con 0x7fe3b4003fb0 2024-05-22T21:13:05.521 INFO:teuthology.orchestra.run.smithi161.stderr:adjusted tunables profile to default 2024-05-22T21:13:05.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.520+0000 7fe3d6400640 1 -- 172.21.15.161:0/3210001604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b4003fb0 msgr2=0x7fe3d01491e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:05.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.520+0000 7fe3d6400640 1 --2- 172.21.15.161:0/3210001604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b4003fb0 0x7fe3d01491e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0007e00 tx=0x7fe3c000f520 comp rx=0 tx=0).stop 2024-05-22T21:13:05.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.520+0000 7fe3d6400640 1 -- 172.21.15.161:0/3210001604 shutdown_connections 2024-05-22T21:13:05.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.520+0000 7fe3d6400640 1 --2- 172.21.15.161:0/3210001604 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3d00c63e0 0x7fe3d00c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:05.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.520+0000 7fe3d6400640 1 --2- 172.21.15.161:0/3210001604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3d00af310 0x7fe3d00af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:05.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.520+0000 7fe3d6400640 1 --2- 172.21.15.161:0/3210001604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b4003fb0 0x7fe3d01491e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:05.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.520+0000 7fe3d6400640 1 -- 172.21.15.161:0/3210001604 >> 172.21.15.161:0/3210001604 conn(0x7fe3d00b2a80 msgr2=0x7fe3d00cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:05.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.520+0000 7fe3d6400640 1 -- 172.21.15.161:0/3210001604 shutdown_connections 2024-05-22T21:13:05.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:05.520+0000 7fe3d6400640 1 -- 172.21.15.161:0/3210001604 wait complete. 2024-05-22T21:13:05.540 INFO:tasks.ceph:check_enable_crimson: False 2024-05-22T21:13:05.541 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-22T21:13:05.541 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:05.541 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-22T21:13:05.560 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:05.560 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-22T21:13:05.625 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:05.626 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-22T21:13:05.666 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:05.664+0000 7f60af32e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-22T21:13:05.672 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:05.672+0000 7fb75831b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-22T21:13:05.685 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:05.685 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-22T21:13:05.745 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:05.746 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-22T21:13:05.766 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:05.766 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-22T21:13:05.827 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:05.827 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-22T21:13:05.890 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:05.891 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-22T21:13:05.951 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 0 2024-05-22T21:13:06.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.124+0000 7f7dde600640 1 Processor -- start 2024-05-22T21:13:06.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.124+0000 7f7dde600640 1 -- start start 2024-05-22T21:13:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.124+0000 7f7dde600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7dd8160ad0 0x7f7dd8099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.124+0000 7f7dde600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7dd80b37b0 0x7f7dd8099f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.124+0000 7f7dde600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd806e4a0 con 0x7f7dd80d0bb0 2024-05-22T21:13:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.124+0000 7f7dde600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd806c690 con 0x7f7dd8160ad0 2024-05-22T21:13:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.124+0000 7f7dde600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd806c0f0 con 0x7f7dd80b37b0 2024-05-22T21:13:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.124+0000 7f7dd7e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7dd80b37b0 0x7f7dd8099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.126 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.124+0000 7f7dd7e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7dd80b37b0 0x7f7dd8099f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33844/0 (socket says 172.21.15.179:33844) 2024-05-22T21:13:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.124+0000 7f7dd7e00640 1 -- 172.21.15.179:0/3333298927 learned_addr learned my addr 172.21.15.179:0/3333298927 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-22T21:13:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.124+0000 7f7dd7400640 1 --2- 172.21.15.179:0/3333298927 >> v2:172.21.15.161:3300/0 conn(0x7f7dd8160ad0 0x7f7dd8099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.124+0000 7f7dd7e00640 1 -- 172.21.15.179:0/3333298927 >> v2:172.21.15.161:3300/0 conn(0x7f7dd8160ad0 msgr2=0x7f7dd8099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.124+0000 7f7dd7e00640 1 --2- 172.21.15.179:0/3333298927 >> v2:172.21.15.161:3300/0 conn(0x7f7dd8160ad0 0x7f7dd8099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.124+0000 7f7dd7e00640 1 -- 172.21.15.179:0/3333298927 >> v1:172.21.15.161:6789/0 conn(0x7f7dd80d0bb0 legacy=0x7f7dd8160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.128 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.124+0000 7f7dd7e00640 1 -- 172.21.15.179:0/3333298927 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd806cf50 con 0x7f7dd80b37b0 2024-05-22T21:13:06.129 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.128+0000 7f7dd7e00640 1 --2- 172.21.15.179:0/3333298927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7dd80b37b0 0x7f7dd8099f80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7dd806d7f0 tx=0x7f7dd001c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b54c4b12f27b33a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:06.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.128+0000 7f7dd6000640 1 -- 172.21.15.179:0/3333298927 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd00a58b0 con 0x7f7dd80b37b0 2024-05-22T21:13:06.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.128+0000 7f7dd6000640 1 -- 172.21.15.179:0/3333298927 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd00a5a10 con 0x7f7dd80b37b0 2024-05-22T21:13:06.130 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.128+0000 7f7dd6000640 1 -- 172.21.15.179:0/3333298927 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd00a5da0 con 0x7f7dd80b37b0 2024-05-22T21:13:06.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.128+0000 7f7dde600640 1 -- 172.21.15.179:0/3333298927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7dd80b37b0 msgr2=0x7f7dd8099f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.128+0000 7f7dde600640 1 --2- 172.21.15.179:0/3333298927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7dd80b37b0 0x7f7dd8099f80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7dd806d7f0 tx=0x7f7dd001c9f0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.128+0000 7f7dde600640 1 -- 172.21.15.179:0/3333298927 shutdown_connections 2024-05-22T21:13:06.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.128+0000 7f7dde600640 1 --2- 172.21.15.179:0/3333298927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7dd80b37b0 0x7f7dd8099f80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.128+0000 7f7dde600640 1 --2- 172.21.15.179:0/3333298927 >> v2:172.21.15.161:3300/0 conn(0x7f7dd8160ad0 0x7f7dd8099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.128+0000 7f7dde600640 1 -- 172.21.15.179:0/3333298927 >> 172.21.15.179:0/3333298927 conn(0x7f7dd80b2a80 msgr2=0x7f7dd80b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:06.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.128+0000 7f7dde600640 1 -- 172.21.15.179:0/3333298927 shutdown_connections 2024-05-22T21:13:06.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.128+0000 7f7dde600640 1 -- 172.21.15.179:0/3333298927 wait complete. 2024-05-22T21:13:06.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dde600640 1 Processor -- start 2024-05-22T21:13:06.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dde600640 1 -- start start 2024-05-22T21:13:06.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dde600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7dd80b37b0 0x7f7dd8155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dde600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7dd80d0bb0 0x7f7dd8158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dde600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd8160ad0 0x7f7dd80c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dde600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd815a220 con 0x7f7dd8160ad0 2024-05-22T21:13:06.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dde600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd8159980 con 0x7f7dd80d0bb0 2024-05-22T21:13:06.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dde600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd8159c60 con 0x7f7dd80b37b0 2024-05-22T21:13:06.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dd6a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7dd80d0bb0 0x7f7dd8158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dd7400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7dd80b37b0 0x7f7dd8155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dd6a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7dd80d0bb0 0x7f7dd8158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33856/0 (socket says 172.21.15.179:33856) 2024-05-22T21:13:06.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dd6a00640 1 -- 172.21.15.179:0/3047185426 learned_addr learned my addr 172.21.15.179:0/3047185426 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-22T21:13:06.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dd7e00640 1 --2- 172.21.15.179:0/3047185426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd8160ad0 0x7f7dd80c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dd6a00640 1 -- 172.21.15.179:0/3047185426 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7dd80b37b0 msgr2=0x7f7dd8155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dd6a00640 1 --2- 172.21.15.179:0/3047185426 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7dd80b37b0 0x7f7dd8155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dd6a00640 1 -- 172.21.15.179:0/3047185426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd8160ad0 msgr2=0x7f7dd80c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dd6a00640 1 --2- 172.21.15.179:0/3047185426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd8160ad0 0x7f7dd80c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dd6a00640 1 -- 172.21.15.179:0/3047185426 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd0009b30 con 0x7f7dd80d0bb0 2024-05-22T21:13:06.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dd7e00640 1 --2- 172.21.15.179:0/3047185426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd8160ad0 0x7f7dd80c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:06.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dd6a00640 1 --2- 172.21.15.179:0/3047185426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7dd80d0bb0 0x7f7dd8158ec0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400c400 tx=0x7f7dc400c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:06.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dcf400640 1 -- 172.21.15.179:0/3047185426 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc40052d0 con 0x7f7dd80d0bb0 2024-05-22T21:13:06.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dcf400640 1 -- 172.21.15.179:0/3047185426 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc4005430 con 0x7f7dd80d0bb0 2024-05-22T21:13:06.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dcf400640 1 -- 172.21.15.179:0/3047185426 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4003840 con 0x7f7dd80d0bb0 2024-05-22T21:13:06.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dde600640 1 -- 172.21.15.179:0/3047185426 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd8159f40 con 0x7f7dd80d0bb0 2024-05-22T21:13:06.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dde600640 1 -- 172.21.15.179:0/3047185426 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd80ca990 con 0x7f7dd80d0bb0 2024-05-22T21:13:06.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dcf400640 1 -- 172.21.15.179:0/3047185426 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f7dc4003ca0 con 0x7f7dd80d0bb0 2024-05-22T21:13:06.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dcf400640 1 -- 172.21.15.179:0/3047185426 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7dc401c780 con 0x7f7dd80d0bb0 2024-05-22T21:13:06.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.132+0000 7f7dde600640 1 -- 172.21.15.179:0/3047185426 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd8073ca0 con 0x7f7dd80d0bb0 2024-05-22T21:13:06.138 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.136+0000 7f7dcf400640 1 -- 172.21.15.179:0/3047185426 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f7dc4017070 con 0x7f7dd80d0bb0 2024-05-22T21:13:06.154 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:06.152+0000 7f60af32e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-22T21:13:06.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.204+0000 7f7dde600640 1 -- 172.21.15.179:0/3047185426 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0} v 0) v1 -- 0x7f7dd8067220 con 0x7f7dd80d0bb0 2024-05-22T21:13:06.209 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.208+0000 7f7dcf400640 1 -- 172.21.15.179:0/3047185426 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7dc401c940 con 0x7f7dd80d0bb0 2024-05-22T21:13:06.209 INFO:teuthology.orchestra.run.smithi179.stdout:0 2024-05-22T21:13:06.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.208+0000 7f7dde600640 1 -- 172.21.15.179:0/3047185426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7dd80d0bb0 msgr2=0x7f7dd8158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.208+0000 7f7dde600640 1 --2- 172.21.15.179:0/3047185426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7dd80d0bb0 0x7f7dd8158ec0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400c400 tx=0x7f7dc400c8c0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.208+0000 7f7dde600640 1 -- 172.21.15.179:0/3047185426 shutdown_connections 2024-05-22T21:13:06.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.208+0000 7f7dde600640 1 --2- 172.21.15.179:0/3047185426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd8160ad0 0x7f7dd80c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.208+0000 7f7dde600640 1 --2- 172.21.15.179:0/3047185426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7dd80d0bb0 0x7f7dd8158ec0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.208+0000 7f7dde600640 1 --2- 172.21.15.179:0/3047185426 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7dd80b37b0 0x7f7dd8155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.208+0000 7f7dde600640 1 -- 172.21.15.179:0/3047185426 >> 172.21.15.179:0/3047185426 conn(0x7f7dd80b2a80 msgr2=0x7f7dd80ae370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:06.211 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.208+0000 7f7dde600640 1 -- 172.21.15.179:0/3047185426 shutdown_connections 2024-05-22T21:13:06.212 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.208+0000 7f7dde600640 1 -- 172.21.15.179:0/3047185426 wait complete. 2024-05-22T21:13:06.225 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:06.224+0000 7fb75831b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-22T21:13:06.229 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 1 2024-05-22T21:13:06.345 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:06.344+0000 7f60af32e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-22T21:13:06.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.360+0000 7fb211c00640 1 Processor -- start 2024-05-22T21:13:06.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.360+0000 7fb211c00640 1 -- start start 2024-05-22T21:13:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.364+0000 7fb211c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb20c0cf710 0x7fb20c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.364+0000 7fb211c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20c06f030 con 0x7fb20c06fad0 2024-05-22T21:13:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.364+0000 7fb211c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20c15efc0 con 0x7fb20c15e5f0 2024-05-22T21:13:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.364+0000 7fb211c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20c06f840 con 0x7fb20c0cf710 2024-05-22T21:13:06.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.364+0000 7fb20aa00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb20c15e5f0 0x7fb20c15f450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:59952/0 (socket says 172.21.15.179:59952) 2024-05-22T21:13:06.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.364+0000 7fb20aa00640 1 -- 172.21.15.179:0/389633396 learned_addr learned my addr 172.21.15.179:0/389633396 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-22T21:13:06.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.364+0000 7fb20b400640 1 --2- 172.21.15.179:0/389633396 >> v2:172.21.15.161:3300/0 conn(0x7fb20c0cf710 0x7fb20c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.364+0000 7fb209600640 1 -- 172.21.15.179:0/389633396 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172312374 0 0) 0x7fb20c15efc0 con 0x7fb20c15e5f0 2024-05-22T21:13:06.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.364+0000 7fb209600640 1 -- 172.21.15.179:0/389633396 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1f0003530 con 0x7fb20c15e5f0 2024-05-22T21:13:06.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.364+0000 7fb20b400640 1 -- 172.21.15.179:0/389633396 >> v1:172.21.15.179:6789/0 conn(0x7fb20c15e5f0 legacy=0x7fb20c15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.364+0000 7fb20b400640 1 -- 172.21.15.179:0/389633396 >> v1:172.21.15.161:6789/0 conn(0x7fb20c06fad0 legacy=0x7fb20c0cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.364+0000 7fb209600640 1 -- 172.21.15.179:0/389633396 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035070715 0 0) 0x7fb20c06f030 con 0x7fb20c06fad0 2024-05-22T21:13:06.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.364+0000 7fb20b400640 1 -- 172.21.15.179:0/389633396 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c0cfcc0 con 0x7fb20c0cf710 2024-05-22T21:13:06.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.364+0000 7fb20b400640 1 --2- 172.21.15.179:0/389633396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb20c0cf710 0x7fb20c06e9a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb204009d90 tx=0x7fb20401c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fccc0241782400e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:06.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb209600640 1 -- 172.21.15.179:0/389633396 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2040a58e0 con 0x7fb20c0cf710 2024-05-22T21:13:06.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb209600640 1 -- 172.21.15.179:0/389633396 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2040a5a40 con 0x7fb20c0cf710 2024-05-22T21:13:06.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb209600640 1 -- 172.21.15.179:0/389633396 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2040a5dd0 con 0x7fb20c0cf710 2024-05-22T21:13:06.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb211c00640 1 -- 172.21.15.179:0/389633396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb20c0cf710 msgr2=0x7fb20c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb211c00640 1 --2- 172.21.15.179:0/389633396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb20c0cf710 0x7fb20c06e9a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb204009d90 tx=0x7fb20401c820 comp rx=0 tx=0).stop 2024-05-22T21:13:06.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb211c00640 1 -- 172.21.15.179:0/389633396 shutdown_connections 2024-05-22T21:13:06.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb211c00640 1 --2- 172.21.15.179:0/389633396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb20c0cf710 0x7fb20c06e9a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb211c00640 1 -- 172.21.15.179:0/389633396 >> 172.21.15.179:0/389633396 conn(0x7fb20c0b2a80 msgr2=0x7fb20c0b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:06.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb211c00640 1 -- 172.21.15.179:0/389633396 shutdown_connections 2024-05-22T21:13:06.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb211c00640 1 -- 172.21.15.179:0/389633396 wait complete. 2024-05-22T21:13:06.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb211c00640 1 Processor -- start 2024-05-22T21:13:06.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb211c00640 1 -- start start 2024-05-22T21:13:06.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb211c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb20c06fad0 0x7fb20c155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb211c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb20c0cf710 0x7fb20c0c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.372 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb211c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb20c15e5f0 0x7fb20c0af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb211c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c155550 con 0x7fb20c0cf710 2024-05-22T21:13:06.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb211c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0c8ec0 con 0x7fb20c15e5f0 2024-05-22T21:13:06.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb211c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0c9700 con 0x7fb20c06fad0 2024-05-22T21:13:06.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb20b400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb20c15e5f0 0x7fb20c0af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb20aa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb20c06fad0 0x7fb20c155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb20b400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb20c15e5f0 0x7fb20c0af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39938/0 (socket says 172.21.15.179:39938) 2024-05-22T21:13:06.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb20b400640 1 -- 172.21.15.179:0/1704754818 learned_addr learned my addr 172.21.15.179:0/1704754818 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-22T21:13:06.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.368+0000 7fb20a000640 1 --2- 172.21.15.179:0/1704754818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb20c0cf710 0x7fb20c0c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.372+0000 7fb20b400640 1 -- 172.21.15.179:0/1704754818 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb20c06fad0 msgr2=0x7fb20c155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.372+0000 7fb20b400640 1 --2- 172.21.15.179:0/1704754818 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb20c06fad0 0x7fb20c155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.372+0000 7fb20b400640 1 -- 172.21.15.179:0/1704754818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb20c0cf710 msgr2=0x7fb20c0c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.372+0000 7fb20b400640 1 --2- 172.21.15.179:0/1704754818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb20c0cf710 0x7fb20c0c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.372+0000 7fb20b400640 1 -- 172.21.15.179:0/1704754818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb204009a70 con 0x7fb20c15e5f0 2024-05-22T21:13:06.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.372+0000 7fb20a000640 1 --2- 172.21.15.179:0/1704754818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb20c0cf710 0x7fb20c0c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:06.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.372+0000 7fb20b400640 1 --2- 172.21.15.179:0/1704754818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb20c15e5f0 0x7fb20c0af150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb20401cd00 tx=0x7fb2040195c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:06.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.372+0000 7fb202a00640 1 -- 172.21.15.179:0/1704754818 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204003ba0 con 0x7fb20c15e5f0 2024-05-22T21:13:06.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.372+0000 7fb202a00640 1 -- 172.21.15.179:0/1704754818 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb204003d00 con 0x7fb20c15e5f0 2024-05-22T21:13:06.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.372+0000 7fb211c00640 1 -- 172.21.15.179:0/1704754818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb20c0c90f0 con 0x7fb20c15e5f0 2024-05-22T21:13:06.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.372+0000 7fb202a00640 1 -- 172.21.15.179:0/1704754818 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2040ad890 con 0x7fb20c15e5f0 2024-05-22T21:13:06.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.372+0000 7fb211c00640 1 -- 172.21.15.179:0/1704754818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20c06be30 con 0x7fb20c15e5f0 2024-05-22T21:13:06.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.372+0000 7fb202a00640 1 -- 172.21.15.179:0/1704754818 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fb2040b4050 con 0x7fb20c15e5f0 2024-05-22T21:13:06.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.372+0000 7fb202a00640 1 -- 172.21.15.179:0/1704754818 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fb2040b7410 con 0x7fb20c15e5f0 2024-05-22T21:13:06.375 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.372+0000 7fb211c00640 1 -- 172.21.15.179:0/1704754818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20c073ca0 con 0x7fb20c15e5f0 2024-05-22T21:13:06.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.376+0000 7fb202a00640 1 -- 172.21.15.179:0/1704754818 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fb2040c6060 con 0x7fb20c15e5f0 2024-05-22T21:13:06.443 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.440+0000 7fb211c00640 1 -- 172.21.15.179:0/1704754818 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1} v 0) v1 -- 0x7fb20c0c9870 con 0x7fb20c15e5f0 2024-05-22T21:13:06.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.448+0000 7fb202a00640 1 -- 172.21.15.179:0/1704754818 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb2040b2070 con 0x7fb20c15e5f0 2024-05-22T21:13:06.448 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-22T21:13:06.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.448+0000 7fb211c00640 1 -- 172.21.15.179:0/1704754818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb20c15e5f0 msgr2=0x7fb20c0af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.448+0000 7fb211c00640 1 --2- 172.21.15.179:0/1704754818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb20c15e5f0 0x7fb20c0af150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb20401cd00 tx=0x7fb2040195c0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.449 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.448+0000 7fb211c00640 1 -- 172.21.15.179:0/1704754818 shutdown_connections 2024-05-22T21:13:06.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.448+0000 7fb211c00640 1 --2- 172.21.15.179:0/1704754818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb20c15e5f0 0x7fb20c0af150 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.448+0000 7fb211c00640 1 --2- 172.21.15.179:0/1704754818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb20c0cf710 0x7fb20c0c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.448+0000 7fb211c00640 1 --2- 172.21.15.179:0/1704754818 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb20c06fad0 0x7fb20c155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.448+0000 7fb211c00640 1 -- 172.21.15.179:0/1704754818 >> 172.21.15.179:0/1704754818 conn(0x7fb20c0b2a80 msgr2=0x7fb20c0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:06.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.448+0000 7fb211c00640 1 -- 172.21.15.179:0/1704754818 shutdown_connections 2024-05-22T21:13:06.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.448+0000 7fb211c00640 1 -- 172.21.15.179:0/1704754818 wait complete. 2024-05-22T21:13:06.466 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:06.464+0000 7fb75831b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-22T21:13:06.467 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 2 2024-05-22T21:13:06.605 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.600+0000 7f0e34400640 1 Processor -- start 2024-05-22T21:13:06.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.600+0000 7f0e34400640 1 -- start start 2024-05-22T21:13:06.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.600+0000 7f0e34400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0e2c160ad0 0x7f0e2c099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.600+0000 7f0e34400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e2c0d0bb0 0x7f0e2c099f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.600+0000 7f0e34400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2c06e4a0 con 0x7f0e2c0b37b0 2024-05-22T21:13:06.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.600+0000 7f0e34400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2c06c690 con 0x7f0e2c160ad0 2024-05-22T21:13:06.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.600+0000 7f0e34400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c06c0f0 con 0x7f0e2c0d0bb0 2024-05-22T21:13:06.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.600+0000 7f0e31000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e2c0d0bb0 0x7f0e2c099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.600+0000 7f0e31a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0e2c160ad0 0x7f0e2c099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.600+0000 7f0e31000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e2c0d0bb0 0x7f0e2c099f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:36854/0 (socket says 172.21.15.179:36854) 2024-05-22T21:13:06.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.600+0000 7f0e31000640 1 -- 172.21.15.179:0/321147771 learned_addr learned my addr 172.21.15.179:0/321147771 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-22T21:13:06.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.600+0000 7f0e31a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0e2c160ad0 0x7f0e2c099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39952/0 (socket says 172.21.15.179:39952) 2024-05-22T21:13:06.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.600+0000 7f0e31000640 1 -- 172.21.15.179:0/321147771 >> v2:172.21.15.179:3300/0 conn(0x7f0e2c160ad0 msgr2=0x7f0e2c099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:06.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.600+0000 7f0e31000640 1 --2- 172.21.15.179:0/321147771 >> v2:172.21.15.179:3300/0 conn(0x7f0e2c160ad0 0x7f0e2c099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.600+0000 7f0e31000640 1 -- 172.21.15.179:0/321147771 >> v1:172.21.15.161:6789/0 conn(0x7f0e2c0b37b0 legacy=0x7f0e2c160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.600+0000 7f0e31000640 1 -- 172.21.15.179:0/321147771 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c06cf50 con 0x7f0e2c0d0bb0 2024-05-22T21:13:06.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.604+0000 7f0e31000640 1 --2- 172.21.15.179:0/321147771 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e2c0d0bb0 0x7f0e2c099f80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009870 tx=0x7f0e1801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=298bcdb48cad8197 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:06.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.604+0000 7f0e2be00640 1 -- 172.21.15.179:0/321147771 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e180a9620 con 0x7f0e2c0d0bb0 2024-05-22T21:13:06.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.604+0000 7f0e2be00640 1 -- 172.21.15.179:0/321147771 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e180a9780 con 0x7f0e2c0d0bb0 2024-05-22T21:13:06.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.604+0000 7f0e2be00640 1 -- 172.21.15.179:0/321147771 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e180a9b10 con 0x7f0e2c0d0bb0 2024-05-22T21:13:06.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.604+0000 7f0e34400640 1 -- 172.21.15.179:0/321147771 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e2c0d0bb0 msgr2=0x7f0e2c099f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.604+0000 7f0e34400640 1 --2- 172.21.15.179:0/321147771 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e2c0d0bb0 0x7f0e2c099f80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009870 tx=0x7f0e1801c560 comp rx=0 tx=0).stop 2024-05-22T21:13:06.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.604+0000 7f0e34400640 1 -- 172.21.15.179:0/321147771 shutdown_connections 2024-05-22T21:13:06.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.604+0000 7f0e34400640 1 --2- 172.21.15.179:0/321147771 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e2c0d0bb0 0x7f0e2c099f80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.604+0000 7f0e34400640 1 --2- 172.21.15.179:0/321147771 >> v2:172.21.15.179:3300/0 conn(0x7f0e2c160ad0 0x7f0e2c099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.604+0000 7f0e34400640 1 -- 172.21.15.179:0/321147771 >> 172.21.15.179:0/321147771 conn(0x7f0e2c0b2a80 msgr2=0x7f0e2c0b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:06.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.604+0000 7f0e34400640 1 -- 172.21.15.179:0/321147771 shutdown_connections 2024-05-22T21:13:06.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.604+0000 7f0e34400640 1 -- 172.21.15.179:0/321147771 wait complete. 2024-05-22T21:13:06.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e34400640 1 Processor -- start 2024-05-22T21:13:06.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e34400640 1 -- start start 2024-05-22T21:13:06.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e34400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e2c0b37b0 0x7f0e2c15e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e34400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0d0bb0 0x7f0e2c0c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e34400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e2c160ad0 0x7f0e2c0c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e34400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0c7830 con 0x7f0e2c160ad0 2024-05-22T21:13:06.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e34400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c15a520 con 0x7f0e2c0d0bb0 2024-05-22T21:13:06.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e34400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0c7550 con 0x7f0e2c0b37b0 2024-05-22T21:13:06.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e31a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e2c0b37b0 0x7f0e2c15e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e31000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0d0bb0 0x7f0e2c0c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e31000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0d0bb0 0x7f0e2c0c69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39956/0 (socket says 172.21.15.179:39956) 2024-05-22T21:13:06.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e31000640 1 -- 172.21.15.179:0/3151348734 learned_addr learned my addr 172.21.15.179:0/3151348734 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-22T21:13:06.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e31a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e2c0b37b0 0x7f0e2c15e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:36862/0 (socket says 172.21.15.179:36862) 2024-05-22T21:13:06.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e32400640 1 --2- 172.21.15.179:0/3151348734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e2c160ad0 0x7f0e2c0c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e31a00640 1 -- 172.21.15.179:0/3151348734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0d0bb0 msgr2=0x7f0e2c0c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e31a00640 1 --2- 172.21.15.179:0/3151348734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0d0bb0 0x7f0e2c0c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e31a00640 1 -- 172.21.15.179:0/3151348734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e2c160ad0 msgr2=0x7f0e2c0c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e31a00640 1 --2- 172.21.15.179:0/3151348734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e2c160ad0 0x7f0e2c0c6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e31a00640 1 -- 172.21.15.179:0/3151348734 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180094d0 con 0x7f0e2c0b37b0 2024-05-22T21:13:06.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e31000640 1 --2- 172.21.15.179:0/3151348734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0d0bb0 0x7f0e2c0c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:06.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e31a00640 1 --2- 172.21.15.179:0/3151348734 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e2c0b37b0 0x7f0e2c15e630 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000e870 tx=0x7f0e2000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:06.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e29600640 1 -- 172.21.15.179:0/3151348734 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2000ee60 con 0x7f0e2c0b37b0 2024-05-22T21:13:06.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e29600640 1 -- 172.21.15.179:0/3151348734 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2000c940 con 0x7f0e2c0b37b0 2024-05-22T21:13:06.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e29600640 1 -- 172.21.15.179:0/3151348734 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2000ccd0 con 0x7f0e2c0b37b0 2024-05-22T21:13:06.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e34400640 1 -- 172.21.15.179:0/3151348734 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2c0b3ce0 con 0x7f0e2c0b37b0 2024-05-22T21:13:06.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.608+0000 7f0e34400640 1 -- 172.21.15.179:0/3151348734 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c0b42a0 con 0x7f0e2c0b37b0 2024-05-22T21:13:06.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.612+0000 7f0e29600640 1 -- 172.21.15.179:0/3151348734 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0e20005ca0 con 0x7f0e2c0b37b0 2024-05-22T21:13:06.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.612+0000 7f0e29600640 1 -- 172.21.15.179:0/3151348734 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f0e2000ce30 con 0x7f0e2c0b37b0 2024-05-22T21:13:06.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.612+0000 7f0e34400640 1 -- 172.21.15.179:0/3151348734 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2c0b4430 con 0x7f0e2c0b37b0 2024-05-22T21:13:06.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.612+0000 7f0e29600640 1 -- 172.21.15.179:0/3151348734 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f0e20021060 con 0x7f0e2c0b37b0 2024-05-22T21:13:06.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.680+0000 7f0e34400640 1 -- 172.21.15.179:0/3151348734 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2} v 0) v1 -- 0x7f0e2c073ca0 con 0x7f0e2c0b37b0 2024-05-22T21:13:06.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.692+0000 7f0e29600640 1 -- 172.21.15.179:0/3151348734 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0e20029020 con 0x7f0e2c0b37b0 2024-05-22T21:13:06.692 INFO:teuthology.orchestra.run.smithi179.stdout:2 2024-05-22T21:13:06.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.692+0000 7f0e34400640 1 -- 172.21.15.179:0/3151348734 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e2c0b37b0 msgr2=0x7f0e2c15e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.692+0000 7f0e34400640 1 --2- 172.21.15.179:0/3151348734 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e2c0b37b0 0x7f0e2c15e630 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000e870 tx=0x7f0e2000ed30 comp rx=0 tx=0).stop 2024-05-22T21:13:06.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.692+0000 7f0e34400640 1 -- 172.21.15.179:0/3151348734 shutdown_connections 2024-05-22T21:13:06.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.692+0000 7f0e34400640 1 --2- 172.21.15.179:0/3151348734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e2c160ad0 0x7f0e2c0c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.692+0000 7f0e34400640 1 --2- 172.21.15.179:0/3151348734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e2c0d0bb0 0x7f0e2c0c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.692+0000 7f0e34400640 1 --2- 172.21.15.179:0/3151348734 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e2c0b37b0 0x7f0e2c15e630 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.692+0000 7f0e34400640 1 -- 172.21.15.179:0/3151348734 >> 172.21.15.179:0/3151348734 conn(0x7f0e2c0b2a80 msgr2=0x7f0e2c0b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:06.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.692+0000 7f0e34400640 1 -- 172.21.15.179:0/3151348734 shutdown_connections 2024-05-22T21:13:06.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.692+0000 7f0e34400640 1 -- 172.21.15.179:0/3151348734 wait complete. 2024-05-22T21:13:06.712 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 3 2024-05-22T21:13:06.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.844+0000 7f0c0be00640 1 Processor -- start 2024-05-22T21:13:06.846 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.844+0000 7f0c0be00640 1 -- start start 2024-05-22T21:13:06.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.844+0000 7f0c0be00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0c0c06cbe0 0x7f0c0c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.844+0000 7f0c0be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c0c06d040 0x7f0c0c06a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.844+0000 7f0c0be00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c0c0ce8f0 con 0x7f0c0c15aeb0 2024-05-22T21:13:06.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.844+0000 7f0c0be00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c0c0cc5d0 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.844+0000 7f0c0be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c15b290 con 0x7f0c0c06d040 2024-05-22T21:13:06.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.844+0000 7f0c0a000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0c0c06cbe0 0x7f0c0c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.844+0000 7f0c0aa00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f0c0c15aeb0 0x7f0c0c0d05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:59964/0 (socket says 172.21.15.179:59964) 2024-05-22T21:13:06.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.844+0000 7f0c0a000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0c0c06cbe0 0x7f0c0c069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39968/0 (socket says 172.21.15.179:39968) 2024-05-22T21:13:06.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.844+0000 7f0c0b400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c0c06d040 0x7f0c0c06a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.848 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.844+0000 7f0c0aa00640 1 -- 172.21.15.179:0/2747135601 learned_addr learned my addr 172.21.15.179:0/2747135601 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-05-22T21:13:06.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.848+0000 7f0c0a000640 1 -- 172.21.15.179:0/2747135601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c0c06d040 msgr2=0x7f0c0c06a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:06.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.848+0000 7f0c0a000640 1 --2- 172.21.15.179:0/2747135601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c0c06d040 0x7f0c0c06a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.848+0000 7f0c0a000640 1 -- 172.21.15.179:0/2747135601 >> v1:172.21.15.179:6789/0 conn(0x7f0c0c15aeb0 legacy=0x7f0c0c0d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.848+0000 7f0c0a000640 1 -- 172.21.15.179:0/2747135601 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c0cc870 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.848+0000 7f0c09600640 1 -- 172.21.15.179:0/2747135601 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286418250 0 0) 0x7f0c0c0ce8f0 con 0x7f0c0c15aeb0 2024-05-22T21:13:06.849 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.848+0000 7f0c0b400640 1 --2- 172.21.15.179:0/2747135601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c0c06d040 0x7f0c0c06a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:06.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.848+0000 7f0c0a000640 1 --2- 172.21.15.179:0/2747135601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c0c06cbe0 0x7f0c0c069ed0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80097c0 tx=0x7f0bf801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13e89842c726a271 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:06.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.848+0000 7f0c09600640 1 -- 172.21.15.179:0/2747135601 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80ac640 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.848+0000 7f0c09600640 1 -- 172.21.15.179:0/2747135601 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf80ac7a0 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.851 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.848+0000 7f0c09600640 1 -- 172.21.15.179:0/2747135601 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80acb30 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0be00640 1 -- 172.21.15.179:0/2747135601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c0c06cbe0 msgr2=0x7f0c0c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0be00640 1 --2- 172.21.15.179:0/2747135601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c0c06cbe0 0x7f0c0c069ed0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80097c0 tx=0x7f0bf801c3e0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0be00640 1 -- 172.21.15.179:0/2747135601 shutdown_connections 2024-05-22T21:13:06.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0be00640 1 --2- 172.21.15.179:0/2747135601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c0c06d040 0x7f0c0c06a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0be00640 1 --2- 172.21.15.179:0/2747135601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c0c06cbe0 0x7f0c0c069ed0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0be00640 1 -- 172.21.15.179:0/2747135601 >> 172.21.15.179:0/2747135601 conn(0x7f0c0c06afe0 msgr2=0x7f0c0c06b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:06.852 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0be00640 1 -- 172.21.15.179:0/2747135601 shutdown_connections 2024-05-22T21:13:06.853 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0be00640 1 -- 172.21.15.179:0/2747135601 wait complete. 2024-05-22T21:13:06.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0be00640 1 Processor -- start 2024-05-22T21:13:06.854 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0be00640 1 -- start start 2024-05-22T21:13:06.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c0c06cbe0 0x7f0c0c06d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.856 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c0c06d040 0x7f0c0c158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c0c15aeb0 0x7f0c0c13da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0be00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c163250 con 0x7f0c0c06d040 2024-05-22T21:13:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0be00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c159990 con 0x7f0c0c15aeb0 2024-05-22T21:13:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c1593d0 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0aa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c0c06cbe0 0x7f0c0c06d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0aa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c0c06cbe0 0x7f0c0c06d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:36882/0 (socket says 172.21.15.179:36882) 2024-05-22T21:13:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0aa00640 1 -- 172.21.15.179:0/3377930386 learned_addr learned my addr 172.21.15.179:0/3377930386 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-22T21:13:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0b400640 1 --2- 172.21.15.179:0/3377930386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c0c15aeb0 0x7f0c0c13da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0a000640 1 --2- 172.21.15.179:0/3377930386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c0c06d040 0x7f0c0c158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0aa00640 1 -- 172.21.15.179:0/3377930386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c0c15aeb0 msgr2=0x7f0c0c13da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0aa00640 1 --2- 172.21.15.179:0/3377930386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c0c15aeb0 0x7f0c0c13da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0aa00640 1 -- 172.21.15.179:0/3377930386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c0c06d040 msgr2=0x7f0c0c158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.857 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0aa00640 1 --2- 172.21.15.179:0/3377930386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c0c06d040 0x7f0c0c158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0aa00640 1 -- 172.21.15.179:0/3377930386 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c04009a70 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.852+0000 7f0c0a000640 1 --2- 172.21.15.179:0/3377930386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c0c06d040 0x7f0c0c158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:06.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.856+0000 7f0c0aa00640 1 --2- 172.21.15.179:0/3377930386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c0c06cbe0 0x7f0c0c06d6d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00c480 tx=0x7f0bfc00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:06.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.856+0000 7f0c02a00640 1 -- 172.21.15.179:0/3377930386 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc0074e0 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.856+0000 7f0c02a00640 1 -- 172.21.15.179:0/3377930386 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bfc007660 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.856+0000 7f0c02a00640 1 -- 172.21.15.179:0/3377930386 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc004a30 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.856+0000 7f0c0be00640 1 -- 172.21.15.179:0/3377930386 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf8009420 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.856+0000 7f0c0be00640 1 -- 172.21.15.179:0/3377930386 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0c162ee0 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.856+0000 7f0c02a00640 1 -- 172.21.15.179:0/3377930386 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0bfc01d3f0 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.858 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.856+0000 7f0c02a00640 1 -- 172.21.15.179:0/3377930386 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f0bfc01db30 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.856+0000 7f0c0be00640 1 -- 172.21.15.179:0/3377930386 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0c162d40 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.859 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.856+0000 7f0c02a00640 1 -- 172.21.15.179:0/3377930386 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f0c0c162d40 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.928+0000 7f0c0be00640 1 -- 172.21.15.179:0/3377930386 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3} v 0) v1 -- 0x7f0c0c067070 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.936+0000 7f0c02a00640 1 -- 172.21.15.179:0/3377930386 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0bfc01d770 con 0x7f0c0c06cbe0 2024-05-22T21:13:06.938 INFO:teuthology.orchestra.run.smithi179.stdout:3 2024-05-22T21:13:06.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.936+0000 7f0c0be00640 1 -- 172.21.15.179:0/3377930386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c0c06cbe0 msgr2=0x7f0c0c06d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:06.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.936+0000 7f0c0be00640 1 --2- 172.21.15.179:0/3377930386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c0c06cbe0 0x7f0c0c06d6d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00c480 tx=0x7f0bfc00c880 comp rx=0 tx=0).stop 2024-05-22T21:13:06.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.936+0000 7f0c0be00640 1 -- 172.21.15.179:0/3377930386 shutdown_connections 2024-05-22T21:13:06.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.936+0000 7f0c0be00640 1 --2- 172.21.15.179:0/3377930386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c0c15aeb0 0x7f0c0c13da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.936+0000 7f0c0be00640 1 --2- 172.21.15.179:0/3377930386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c0c06d040 0x7f0c0c158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.936+0000 7f0c0be00640 1 --2- 172.21.15.179:0/3377930386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c0c06cbe0 0x7f0c0c06d6d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:06.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.936+0000 7f0c0be00640 1 -- 172.21.15.179:0/3377930386 >> 172.21.15.179:0/3377930386 conn(0x7f0c0c06afe0 msgr2=0x7f0c0c0cbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:06.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.936+0000 7f0c0be00640 1 -- 172.21.15.179:0/3377930386 shutdown_connections 2024-05-22T21:13:06.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:06.936+0000 7f0c0be00640 1 -- 172.21.15.179:0/3377930386 wait complete. 2024-05-22T21:13:06.956 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 4 2024-05-22T21:13:06.963 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:06.964+0000 7f60af32e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-22T21:13:07.024 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:07.024+0000 7f60af32e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-22T21:13:07.084 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:07.084+0000 7f60af32e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-22T21:13:07.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.088+0000 7f838f800640 1 Processor -- start 2024-05-22T21:13:07.091 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.088+0000 7f838f800640 1 -- start start 2024-05-22T21:13:07.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.092+0000 7f838f800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83880d1e60 0x7f83880d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.092+0000 7f838f800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83880cf060 con 0x7f83880d38a0 2024-05-22T21:13:07.092 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.092+0000 7f838f800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83880d0f00 con 0x7f838815d500 2024-05-22T21:13:07.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.092+0000 7f838f800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83880d11e0 con 0x7f83880d1e60 2024-05-22T21:13:07.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.092+0000 7f8387e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83880d1e60 0x7f83880d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.092+0000 7f8387e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83880d1e60 0x7f83880d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:36884/0 (socket says 172.21.15.179:36884) 2024-05-22T21:13:07.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.092+0000 7f838ce00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f838815d500 0x7f83880d1750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:59978/0 (socket says 172.21.15.179:59978) 2024-05-22T21:13:07.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.092+0000 7f8387e00640 1 -- 172.21.15.179:0/1913410789 learned_addr learned my addr 172.21.15.179:0/1913410789 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-22T21:13:07.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.092+0000 7f8387e00640 1 -- 172.21.15.179:0/1913410789 >> v1:172.21.15.179:6789/0 conn(0x7f838815d500 legacy=0x7f83880d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.092+0000 7f8387e00640 1 -- 172.21.15.179:0/1913410789 >> v1:172.21.15.161:6789/0 conn(0x7f83880d38a0 legacy=0x7f83880c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.093 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.092+0000 7f8387e00640 1 -- 172.21.15.179:0/1913410789 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83880d14c0 con 0x7f83880d1e60 2024-05-22T21:13:07.095 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.092+0000 7f8387e00640 1 --2- 172.21.15.179:0/1913410789 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83880d1e60 0x7f83880d03e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f83740098a0 tx=0x7f837401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b00cf1f903931ec0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:07.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f8387400640 1 -- 172.21.15.179:0/1913410789 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83740aa620 con 0x7f83880d1e60 2024-05-22T21:13:07.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f8387400640 1 -- 172.21.15.179:0/1913410789 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83740aa780 con 0x7f83880d1e60 2024-05-22T21:13:07.096 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f8387400640 1 -- 172.21.15.179:0/1913410789 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83740aab10 con 0x7f83880d1e60 2024-05-22T21:13:07.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838f800640 1 -- 172.21.15.179:0/1913410789 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83880d1e60 msgr2=0x7f83880d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838f800640 1 --2- 172.21.15.179:0/1913410789 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83880d1e60 0x7f83880d03e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f83740098a0 tx=0x7f837401c4c0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838f800640 1 -- 172.21.15.179:0/1913410789 shutdown_connections 2024-05-22T21:13:07.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838f800640 1 --2- 172.21.15.179:0/1913410789 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83880d1e60 0x7f83880d03e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838f800640 1 -- 172.21.15.179:0/1913410789 >> 172.21.15.179:0/1913410789 conn(0x7f838809c470 msgr2=0x7f83880d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:07.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838f800640 1 -- 172.21.15.179:0/1913410789 shutdown_connections 2024-05-22T21:13:07.097 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838f800640 1 -- 172.21.15.179:0/1913410789 wait complete. 2024-05-22T21:13:07.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838f800640 1 Processor -- start 2024-05-22T21:13:07.098 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838f800640 1 -- start start 2024-05-22T21:13:07.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838f800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83880d1e60 0x7f83880ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838f800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83880d38a0 0x7f83880cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838f800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f838815bee0 0x7f83880b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838f800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838815ac10 con 0x7f83880d38a0 2024-05-22T21:13:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838f800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838815ae60 con 0x7f838815bee0 2024-05-22T21:13:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838f800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838815b140 con 0x7f83880d1e60 2024-05-22T21:13:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838ce00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83880d1e60 0x7f83880ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838ce00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83880d1e60 0x7f83880ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:36888/0 (socket says 172.21.15.179:36888) 2024-05-22T21:13:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838d800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f838815bee0 0x7f83880b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.096+0000 7f838ce00640 1 -- 172.21.15.179:0/2582536136 learned_addr learned my addr 172.21.15.179:0/2582536136 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-22T21:13:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.100+0000 7f8387e00640 1 --2- 172.21.15.179:0/2582536136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83880d38a0 0x7f83880cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.100+0000 7f838d800640 1 -- 172.21.15.179:0/2582536136 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83880d1e60 msgr2=0x7f83880ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.100+0000 7f838d800640 1 --2- 172.21.15.179:0/2582536136 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83880d1e60 0x7f83880ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.100+0000 7f838d800640 1 -- 172.21.15.179:0/2582536136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83880d38a0 msgr2=0x7f83880cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.100+0000 7f838d800640 1 --2- 172.21.15.179:0/2582536136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83880d38a0 0x7f83880cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.100+0000 7f838d800640 1 -- 172.21.15.179:0/2582536136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83740094d0 con 0x7f838815bee0 2024-05-22T21:13:07.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.100+0000 7f838ce00640 1 --2- 172.21.15.179:0/2582536136 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83880d1e60 0x7f83880ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:07.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.100+0000 7f838d800640 1 --2- 172.21.15.179:0/2582536136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f838815bee0 0x7f83880b73a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f83880d3d50 tx=0x7f838000cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:07.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.100+0000 7f8384c00640 1 -- 172.21.15.179:0/2582536136 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380003b80 con 0x7f838815bee0 2024-05-22T21:13:07.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.100+0000 7f8384c00640 1 -- 172.21.15.179:0/2582536136 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8380003ce0 con 0x7f838815bee0 2024-05-22T21:13:07.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.100+0000 7f838f800640 1 -- 172.21.15.179:0/2582536136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838815b420 con 0x7f838815bee0 2024-05-22T21:13:07.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.100+0000 7f8384c00640 1 -- 172.21.15.179:0/2582536136 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380005310 con 0x7f838815bee0 2024-05-22T21:13:07.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.100+0000 7f838f800640 1 -- 172.21.15.179:0/2582536136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838815b9e0 con 0x7f838815bee0 2024-05-22T21:13:07.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.100+0000 7f8384c00640 1 -- 172.21.15.179:0/2582536136 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f838000a640 con 0x7f838815bee0 2024-05-22T21:13:07.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.100+0000 7f8384c00640 1 -- 172.21.15.179:0/2582536136 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f838000ae00 con 0x7f838815bee0 2024-05-22T21:13:07.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.100+0000 7f838f800640 1 -- 172.21.15.179:0/2582536136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8388067220 con 0x7f838815bee0 2024-05-22T21:13:07.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.104+0000 7f8384c00640 1 -- 172.21.15.179:0/2582536136 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f838000a8f0 con 0x7f838815bee0 2024-05-22T21:13:07.106 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:07.104+0000 7fb75831b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-22T21:13:07.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.168+0000 7f838f800640 1 -- 172.21.15.179:0/2582536136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4} v 0) v1 -- 0x7f838815a9f0 con 0x7f838815bee0 2024-05-22T21:13:07.172 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:07.172+0000 7f60af32e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-22T21:13:07.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.172+0000 7f8384c00640 1 -- 172.21.15.179:0/2582536136 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f838000aa90 con 0x7f838815bee0 2024-05-22T21:13:07.175 INFO:teuthology.orchestra.run.smithi179.stdout:4 2024-05-22T21:13:07.175 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:07.172+0000 7fb75831b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-22T21:13:07.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.172+0000 7f838f800640 1 -- 172.21.15.179:0/2582536136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f838815bee0 msgr2=0x7f83880b73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.172+0000 7f838f800640 1 --2- 172.21.15.179:0/2582536136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f838815bee0 0x7f83880b73a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f83880d3d50 tx=0x7f838000cb90 comp rx=0 tx=0).stop 2024-05-22T21:13:07.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.176+0000 7f838f800640 1 -- 172.21.15.179:0/2582536136 shutdown_connections 2024-05-22T21:13:07.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.176+0000 7f838f800640 1 --2- 172.21.15.179:0/2582536136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f838815bee0 0x7f83880b73a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.176+0000 7f838f800640 1 --2- 172.21.15.179:0/2582536136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83880d38a0 0x7f83880cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.176+0000 7f838f800640 1 --2- 172.21.15.179:0/2582536136 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83880d1e60 0x7f83880ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.176+0000 7f838f800640 1 -- 172.21.15.179:0/2582536136 >> 172.21.15.179:0/2582536136 conn(0x7f838809c470 msgr2=0x7f838806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:07.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.176+0000 7f838f800640 1 -- 172.21.15.179:0/2582536136 shutdown_connections 2024-05-22T21:13:07.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.176+0000 7f838f800640 1 -- 172.21.15.179:0/2582536136 wait complete. 2024-05-22T21:13:07.193 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 5 2024-05-22T21:13:07.232 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:07.232+0000 7f60af32e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-22T21:13:07.239 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:07.236+0000 7fb75831b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-22T21:13:07.319 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:07.320+0000 7f60af32e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-22T21:13:07.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.328+0000 7f3da1000640 1 Processor -- start 2024-05-22T21:13:07.329 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.328+0000 7f3da1000640 1 -- start start 2024-05-22T21:13:07.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.328+0000 7f3da1000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9c161840 0x7f3d9c0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.331 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.328+0000 7f3da1000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3d9c0b3630 0x7f3d9c0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.328+0000 7f3da1000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d9c161f70 con 0x7f3d9c0c3f60 2024-05-22T21:13:07.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.328+0000 7f3da1000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d9c0b0870 con 0x7f3d9c0b3630 2024-05-22T21:13:07.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.328+0000 7f3da1000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c162250 con 0x7f3d9c161840 2024-05-22T21:13:07.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.328+0000 7f3d99e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9c161840 0x7f3d9c0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.328+0000 7f3d99400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3d9c0b3630 0x7f3d9c0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.332 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.328+0000 7f3d99400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3d9c0b3630 0x7f3d9c0b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39984/0 (socket says 172.21.15.179:39984) 2024-05-22T21:13:07.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.328+0000 7f3d99400640 1 -- 172.21.15.179:0/1584631969 learned_addr learned my addr 172.21.15.179:0/1584631969 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-22T21:13:07.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.328+0000 7f3d99e00640 1 --2- 172.21.15.179:0/1584631969 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9c161840 0x7f3d9c0cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:36894/0 (socket says 172.21.15.179:36894) 2024-05-22T21:13:07.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.328+0000 7f3d99400640 1 -- 172.21.15.179:0/1584631969 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9c161840 msgr2=0x7f3d9c0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:07.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.328+0000 7f3d99400640 1 --2- 172.21.15.179:0/1584631969 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9c161840 0x7f3d9c0cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.328+0000 7f3d99400640 1 -- 172.21.15.179:0/1584631969 >> v1:172.21.15.161:6789/0 conn(0x7f3d9c0c3f60 legacy=0x7f3d9c099d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.328+0000 7f3d99400640 1 -- 172.21.15.179:0/1584631969 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c162530 con 0x7f3d9c0b3630 2024-05-22T21:13:07.333 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:07.328+0000 7fb75831b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-22T21:13:07.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.332+0000 7f3d99400640 1 --2- 172.21.15.179:0/1584631969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d9c0b3630 0x7f3d9c0b0330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009870 tx=0x7f3d8801c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=685306872458ab47 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:07.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.332+0000 7f3d98a00640 1 -- 172.21.15.179:0/1584631969 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d880aa650 con 0x7f3d9c0b3630 2024-05-22T21:13:07.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.332+0000 7f3d98a00640 1 -- 172.21.15.179:0/1584631969 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d880aa7b0 con 0x7f3d9c0b3630 2024-05-22T21:13:07.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.332+0000 7f3d98a00640 1 -- 172.21.15.179:0/1584631969 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d880aab40 con 0x7f3d9c0b3630 2024-05-22T21:13:07.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.332+0000 7f3da1000640 1 -- 172.21.15.179:0/1584631969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d9c0b3630 msgr2=0x7f3d9c0b0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.332+0000 7f3da1000640 1 --2- 172.21.15.179:0/1584631969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d9c0b3630 0x7f3d9c0b0330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009870 tx=0x7f3d8801c4d0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.332+0000 7f3da1000640 1 -- 172.21.15.179:0/1584631969 shutdown_connections 2024-05-22T21:13:07.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.332+0000 7f3da1000640 1 --2- 172.21.15.179:0/1584631969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d9c0b3630 0x7f3d9c0b0330 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.332+0000 7f3da1000640 1 --2- 172.21.15.179:0/1584631969 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9c161840 0x7f3d9c0cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.332+0000 7f3da1000640 1 -- 172.21.15.179:0/1584631969 >> 172.21.15.179:0/1584631969 conn(0x7f3d9c0b2a80 msgr2=0x7f3d9c0b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:07.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.332+0000 7f3da1000640 1 -- 172.21.15.179:0/1584631969 shutdown_connections 2024-05-22T21:13:07.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.332+0000 7f3da1000640 1 -- 172.21.15.179:0/1584631969 wait complete. 2024-05-22T21:13:07.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3da1000640 1 Processor -- start 2024-05-22T21:13:07.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3da1000640 1 -- start start 2024-05-22T21:13:07.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3da1000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9c0b3630 0x7f3d9c15b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3da1000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d9c0c3f60 0x7f3d9c160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3da1000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d9c161840 0x7f3d9c157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.339 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3da1000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c161020 con 0x7f3d9c0c3f60 2024-05-22T21:13:07.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3da1000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c1576d0 con 0x7f3d9c161840 2024-05-22T21:13:07.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3da1000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c15ef70 con 0x7f3d9c0b3630 2024-05-22T21:13:07.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3d9a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d9c161840 0x7f3d9c157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3d99e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9c0b3630 0x7f3d9c15b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3d99400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d9c0c3f60 0x7f3d9c160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3d99e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9c0b3630 0x7f3d9c15b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:36902/0 (socket says 172.21.15.179:36902) 2024-05-22T21:13:07.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3d99e00640 1 -- 172.21.15.179:0/397577318 learned_addr learned my addr 172.21.15.179:0/397577318 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-22T21:13:07.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3d9a800640 1 -- 172.21.15.179:0/397577318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9c0b3630 msgr2=0x7f3d9c15b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3d9a800640 1 --2- 172.21.15.179:0/397577318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9c0b3630 0x7f3d9c15b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3d9a800640 1 -- 172.21.15.179:0/397577318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d9c0c3f60 msgr2=0x7f3d9c160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.340 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3d9a800640 1 --2- 172.21.15.179:0/397577318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d9c0c3f60 0x7f3d9c160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3d9a800640 1 -- 172.21.15.179:0/397577318 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d880094d0 con 0x7f3d9c161840 2024-05-22T21:13:07.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3d9a800640 1 --2- 172.21.15.179:0/397577318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d9c161840 0x7f3d9c157190 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400c700 tx=0x7f3d9400cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:07.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3d92000640 1 -- 172.21.15.179:0/397577318 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d94003b80 con 0x7f3d9c161840 2024-05-22T21:13:07.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3d92000640 1 -- 172.21.15.179:0/397577318 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d94003ce0 con 0x7f3d9c161840 2024-05-22T21:13:07.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3d92000640 1 -- 172.21.15.179:0/397577318 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9400aa30 con 0x7f3d9c161840 2024-05-22T21:13:07.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3da1000640 1 -- 172.21.15.179:0/397577318 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9c1581a0 con 0x7f3d9c161840 2024-05-22T21:13:07.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3da1000640 1 -- 172.21.15.179:0/397577318 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9c157be0 con 0x7f3d9c161840 2024-05-22T21:13:07.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3d92000640 1 -- 172.21.15.179:0/397577318 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f3d940050d0 con 0x7f3d9c161840 2024-05-22T21:13:07.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.336+0000 7f3d92000640 1 -- 172.21.15.179:0/397577318 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f3d94012420 con 0x7f3d9c161840 2024-05-22T21:13:07.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.340+0000 7f3da1000640 1 -- 172.21.15.179:0/397577318 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d9c073ca0 con 0x7f3d9c161840 2024-05-22T21:13:07.342 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.340+0000 7f3d92000640 1 -- 172.21.15.179:0/397577318 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f3d94005380 con 0x7f3d9c161840 2024-05-22T21:13:07.379 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:07.376+0000 7f60af32e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-22T21:13:07.394 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:07.392+0000 7fb75831b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-22T21:13:07.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.408+0000 7f3da1000640 1 -- 172.21.15.179:0/397577318 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5} v 0) v1 -- 0x7f3d9c15f0e0 con 0x7f3d9c161840 2024-05-22T21:13:07.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.412+0000 7f3d92000640 1 -- 172.21.15.179:0/397577318 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3d94005520 con 0x7f3d9c161840 2024-05-22T21:13:07.414 INFO:teuthology.orchestra.run.smithi179.stdout:5 2024-05-22T21:13:07.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.412+0000 7f3da1000640 1 -- 172.21.15.179:0/397577318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d9c161840 msgr2=0x7f3d9c157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.412+0000 7f3da1000640 1 --2- 172.21.15.179:0/397577318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d9c161840 0x7f3d9c157190 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400c700 tx=0x7f3d9400cbc0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.412+0000 7f3da1000640 1 -- 172.21.15.179:0/397577318 shutdown_connections 2024-05-22T21:13:07.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.412+0000 7f3da1000640 1 --2- 172.21.15.179:0/397577318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d9c161840 0x7f3d9c157190 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.412+0000 7f3da1000640 1 --2- 172.21.15.179:0/397577318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d9c0c3f60 0x7f3d9c160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.412+0000 7f3da1000640 1 --2- 172.21.15.179:0/397577318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9c0b3630 0x7f3d9c15b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.412+0000 7f3da1000640 1 -- 172.21.15.179:0/397577318 >> 172.21.15.179:0/397577318 conn(0x7f3d9c0b2a80 msgr2=0x7f3d9c0cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:07.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.412+0000 7f3da1000640 1 -- 172.21.15.179:0/397577318 shutdown_connections 2024-05-22T21:13:07.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.412+0000 7f3da1000640 1 -- 172.21.15.179:0/397577318 wait complete. 2024-05-22T21:13:07.435 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 6 2024-05-22T21:13:07.488 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:07.488+0000 7fb75831b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-22T21:13:07.548 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:07.548+0000 7f60af32e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-22T21:13:07.550 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:07.548+0000 7fb75831b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-22T21:13:07.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.608+0000 7f4ad0a00640 1 Processor -- start 2024-05-22T21:13:07.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.608+0000 7f4ad0a00640 1 -- start start 2024-05-22T21:13:07.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.608+0000 7f4ad0a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4acc0d0bb0 0x7f4acc06b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.608+0000 7f4ad0a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4acc0b37b0 0x7f4acc06b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.608+0000 7f4ad0a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4acc06e4a0 con 0x7f4acc160ad0 2024-05-22T21:13:07.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.608+0000 7f4ad0a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4acc06c690 con 0x7f4acc0d0bb0 2024-05-22T21:13:07.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.608+0000 7f4ad0a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4acc06cf50 con 0x7f4acc0b37b0 2024-05-22T21:13:07.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.608+0000 7f4aca800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4acc0b37b0 0x7f4acc06b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.608+0000 7f4aca800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4acc0b37b0 0x7f4acc06b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40008/0 (socket says 172.21.15.179:40008) 2024-05-22T21:13:07.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.608+0000 7f4aca800640 1 -- 172.21.15.179:0/3006599531 learned_addr learned my addr 172.21.15.179:0/3006599531 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-22T21:13:07.612 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.612+0000 7f4ac3e00640 1 --2- 172.21.15.179:0/3006599531 >> v2:172.21.15.161:3300/0 conn(0x7f4acc0d0bb0 0x7f4acc06b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.612+0000 7f4aca800640 1 -- 172.21.15.179:0/3006599531 >> v2:172.21.15.161:3300/0 conn(0x7f4acc0d0bb0 msgr2=0x7f4acc06b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:07.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.612+0000 7f4aca800640 1 --2- 172.21.15.179:0/3006599531 >> v2:172.21.15.161:3300/0 conn(0x7f4acc0d0bb0 0x7f4acc06b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.612+0000 7f4aca800640 1 -- 172.21.15.179:0/3006599531 >> v1:172.21.15.161:6789/0 conn(0x7f4acc160ad0 legacy=0x7f4acc099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.612+0000 7f4aca800640 1 -- 172.21.15.179:0/3006599531 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4acc06c3b0 con 0x7f4acc0b37b0 2024-05-22T21:13:07.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.612+0000 7f4ac3e00640 1 --2- 172.21.15.179:0/3006599531 >> v2:172.21.15.161:3300/0 conn(0x7f4acc0d0bb0 0x7f4acc06b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:07.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.612+0000 7f4aca800640 1 --2- 172.21.15.179:0/3006599531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4acc0b37b0 0x7f4acc06b620 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4acc06d7f0 tx=0x7f4ac401c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cba57c388c8c4e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:07.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.612+0000 7f4ac9400640 1 -- 172.21.15.179:0/3006599531 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac40a58e0 con 0x7f4acc0b37b0 2024-05-22T21:13:07.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.612+0000 7f4ac9400640 1 -- 172.21.15.179:0/3006599531 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac40a5a40 con 0x7f4acc0b37b0 2024-05-22T21:13:07.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.612+0000 7f4ac9400640 1 -- 172.21.15.179:0/3006599531 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac40a5dd0 con 0x7f4acc0b37b0 2024-05-22T21:13:07.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ad0a00640 1 -- 172.21.15.179:0/3006599531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4acc0b37b0 msgr2=0x7f4acc06b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ad0a00640 1 --2- 172.21.15.179:0/3006599531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4acc0b37b0 0x7f4acc06b620 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4acc06d7f0 tx=0x7f4ac401c9f0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.616 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:07.616+0000 7f60af32e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-22T21:13:07.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ad0a00640 1 -- 172.21.15.179:0/3006599531 shutdown_connections 2024-05-22T21:13:07.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ad0a00640 1 --2- 172.21.15.179:0/3006599531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4acc0b37b0 0x7f4acc06b620 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ad0a00640 1 --2- 172.21.15.179:0/3006599531 >> v2:172.21.15.161:3300/0 conn(0x7f4acc0d0bb0 0x7f4acc06b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ad0a00640 1 -- 172.21.15.179:0/3006599531 >> 172.21.15.179:0/3006599531 conn(0x7f4acc0b2a80 msgr2=0x7f4acc0b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:07.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ad0a00640 1 -- 172.21.15.179:0/3006599531 shutdown_connections 2024-05-22T21:13:07.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ad0a00640 1 -- 172.21.15.179:0/3006599531 wait complete. 2024-05-22T21:13:07.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ad0a00640 1 Processor -- start 2024-05-22T21:13:07.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ad0a00640 1 -- start start 2024-05-22T21:13:07.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ad0a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4acc0b37b0 0x7f4acc157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ad0a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4acc0d0bb0 0x7f4acc0b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ad0a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4acc160ad0 0x7f4acc1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ad0a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc0b6ba0 con 0x7f4acc0b37b0 2024-05-22T21:13:07.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ad0a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc158d10 con 0x7f4acc0d0bb0 2024-05-22T21:13:07.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ad0a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc0c5ce0 con 0x7f4acc160ad0 2024-05-22T21:13:07.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ac3e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4acc0d0bb0 0x7f4acc0b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ac3e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4acc0d0bb0 0x7f4acc0b6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40014/0 (socket says 172.21.15.179:40014) 2024-05-22T21:13:07.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4aca800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4acc160ad0 0x7f4acc1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ac3e00640 1 -- 172.21.15.179:0/2456903996 learned_addr learned my addr 172.21.15.179:0/2456903996 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-22T21:13:07.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ac9e00640 1 --2- 172.21.15.179:0/2456903996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4acc0b37b0 0x7f4acc157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ac3e00640 1 -- 172.21.15.179:0/2456903996 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4acc160ad0 msgr2=0x7f4acc1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.620 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ac3e00640 1 --2- 172.21.15.179:0/2456903996 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4acc160ad0 0x7f4acc1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ac3e00640 1 -- 172.21.15.179:0/2456903996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4acc0b37b0 msgr2=0x7f4acc157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ac3e00640 1 --2- 172.21.15.179:0/2456903996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4acc0b37b0 0x7f4acc157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4ac3e00640 1 -- 172.21.15.179:0/2456903996 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac4009b30 con 0x7f4acc0d0bb0 2024-05-22T21:13:07.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.616+0000 7f4aca800640 1 --2- 172.21.15.179:0/2456903996 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4acc160ad0 0x7f4acc1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:07.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.620+0000 7f4ac3e00640 1 --2- 172.21.15.179:0/2456903996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4acc0d0bb0 0x7f4acc0b6660 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400e810 tx=0x7f4ab400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:07.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.620+0000 7f4ac2000640 1 -- 172.21.15.179:0/2456903996 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab400c8d0 con 0x7f4acc0d0bb0 2024-05-22T21:13:07.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.620+0000 7f4ac2000640 1 -- 172.21.15.179:0/2456903996 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab400ca30 con 0x7f4acc0d0bb0 2024-05-22T21:13:07.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.620+0000 7f4ac2000640 1 -- 172.21.15.179:0/2456903996 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab400cde0 con 0x7f4acc0d0bb0 2024-05-22T21:13:07.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.620+0000 7f4ad0a00640 1 -- 172.21.15.179:0/2456903996 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4acc1597a0 con 0x7f4acc0d0bb0 2024-05-22T21:13:07.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.620+0000 7f4ad0a00640 1 -- 172.21.15.179:0/2456903996 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4acc1591e0 con 0x7f4acc0d0bb0 2024-05-22T21:13:07.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.620+0000 7f4ac2000640 1 -- 172.21.15.179:0/2456903996 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f4ab401c750 con 0x7f4acc0d0bb0 2024-05-22T21:13:07.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.620+0000 7f4ac2000640 1 -- 172.21.15.179:0/2456903996 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f4ab4013b40 con 0x7f4acc0d0bb0 2024-05-22T21:13:07.622 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.620+0000 7f4ad0a00640 1 -- 172.21.15.179:0/2456903996 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4acc073ca0 con 0x7f4acc0d0bb0 2024-05-22T21:13:07.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.620+0000 7f4ac2000640 1 -- 172.21.15.179:0/2456903996 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f4ab403bdc0 con 0x7f4acc0d0bb0 2024-05-22T21:13:07.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.688+0000 7f4ad0a00640 1 -- 172.21.15.179:0/2456903996 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6} v 0) v1 -- 0x7f4acc067220 con 0x7f4acc0d0bb0 2024-05-22T21:13:07.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.692+0000 7f4ac2000640 1 -- 172.21.15.179:0/2456903996 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4ab402a020 con 0x7f4acc0d0bb0 2024-05-22T21:13:07.695 INFO:teuthology.orchestra.run.smithi179.stdout:6 2024-05-22T21:13:07.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.692+0000 7f4ad0a00640 1 -- 172.21.15.179:0/2456903996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4acc0d0bb0 msgr2=0x7f4acc0b6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.692+0000 7f4ad0a00640 1 --2- 172.21.15.179:0/2456903996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4acc0d0bb0 0x7f4acc0b6660 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400e810 tx=0x7f4ab400ecd0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.692+0000 7f4ad0a00640 1 -- 172.21.15.179:0/2456903996 shutdown_connections 2024-05-22T21:13:07.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.692+0000 7f4ad0a00640 1 --2- 172.21.15.179:0/2456903996 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4acc160ad0 0x7f4acc1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.692+0000 7f4ad0a00640 1 --2- 172.21.15.179:0/2456903996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4acc0d0bb0 0x7f4acc0b6660 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.692+0000 7f4ad0a00640 1 --2- 172.21.15.179:0/2456903996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4acc0b37b0 0x7f4acc157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.692+0000 7f4ad0a00640 1 -- 172.21.15.179:0/2456903996 >> 172.21.15.179:0/2456903996 conn(0x7f4acc0b2a80 msgr2=0x7f4acc0af560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:07.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.696+0000 7f4ad0a00640 1 -- 172.21.15.179:0/2456903996 shutdown_connections 2024-05-22T21:13:07.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.696+0000 7f4ad0a00640 1 -- 172.21.15.179:0/2456903996 wait complete. 2024-05-22T21:13:07.713 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 7 2024-05-22T21:13:07.725 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:07.724+0000 7fb75831b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-22T21:13:07.796 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:07.792+0000 7fb75831b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-22T21:13:07.861 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9afb600640 1 Processor -- start 2024-05-22T21:13:07.862 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9afb600640 1 -- start start 2024-05-22T21:13:07.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9afb600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9af40d0bb0 0x7f9af406b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.863 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9afb600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9af40b37b0 0x7f9af406b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9afb600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af406e4a0 con 0x7f9af4160ad0 2024-05-22T21:13:07.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9afb600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af406c690 con 0x7f9af40b37b0 2024-05-22T21:13:07.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9afb600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af406cf50 con 0x7f9af40d0bb0 2024-05-22T21:13:07.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9aebe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9af40d0bb0 0x7f9af406b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9aebe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9af40d0bb0 0x7f9af406b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:36918/0 (socket says 172.21.15.179:36918) 2024-05-22T21:13:07.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9aebe00640 1 -- 172.21.15.179:0/583862909 learned_addr learned my addr 172.21.15.179:0/583862909 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-22T21:13:07.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9af9600640 1 --2- 172.21.15.179:0/583862909 >> v2:172.21.15.161:3300/0 conn(0x7f9af40b37b0 0x7f9af406b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9aebe00640 1 -- 172.21.15.179:0/583862909 >> v2:172.21.15.161:3300/0 conn(0x7f9af40b37b0 msgr2=0x7f9af406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:07.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9aebe00640 1 --2- 172.21.15.179:0/583862909 >> v2:172.21.15.161:3300/0 conn(0x7f9af40b37b0 0x7f9af406b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9aebe00640 1 -- 172.21.15.179:0/583862909 >> v1:172.21.15.179:6789/0 conn(0x7f9af4160ad0 legacy=0x7f9af4099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.864 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9aebe00640 1 -- 172.21.15.179:0/583862909 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af406c3b0 con 0x7f9af40d0bb0 2024-05-22T21:13:07.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9af3e00640 1 -- 172.21.15.179:0/583862909 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552377674 0 0) 0x7f9af406e4a0 con 0x7f9af4160ad0 2024-05-22T21:13:07.865 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.860+0000 7f9af9600640 1 --2- 172.21.15.179:0/583862909 >> v2:172.21.15.161:3300/0 conn(0x7f9af40b37b0 0x7f9af406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.864+0000 7f9aebe00640 1 --2- 172.21.15.179:0/583862909 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9af40d0bb0 0x7f9af406b0e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009870 tx=0x7f9adc01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcf0dcc61e9cd800 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.864+0000 7f9af3e00640 1 -- 172.21.15.179:0/583862909 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0a9640 con 0x7f9af40d0bb0 2024-05-22T21:13:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.864+0000 7f9af3e00640 1 -- 172.21.15.179:0/583862909 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9adc0a97a0 con 0x7f9af40d0bb0 2024-05-22T21:13:07.867 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.864+0000 7f9af3e00640 1 -- 172.21.15.179:0/583862909 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0a9b30 con 0x7f9af40d0bb0 2024-05-22T21:13:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.864+0000 7f9afb600640 1 -- 172.21.15.179:0/583862909 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9af40d0bb0 msgr2=0x7f9af406b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.864+0000 7f9afb600640 1 --2- 172.21.15.179:0/583862909 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9af40d0bb0 0x7f9af406b0e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009870 tx=0x7f9adc01c560 comp rx=0 tx=0).stop 2024-05-22T21:13:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.864+0000 7f9afb600640 1 -- 172.21.15.179:0/583862909 shutdown_connections 2024-05-22T21:13:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.864+0000 7f9afb600640 1 --2- 172.21.15.179:0/583862909 >> v2:172.21.15.161:3300/0 conn(0x7f9af40b37b0 0x7f9af406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.864+0000 7f9afb600640 1 --2- 172.21.15.179:0/583862909 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9af40d0bb0 0x7f9af406b0e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.864+0000 7f9afb600640 1 -- 172.21.15.179:0/583862909 >> 172.21.15.179:0/583862909 conn(0x7f9af40b2a80 msgr2=0x7f9af40b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.864+0000 7f9afb600640 1 -- 172.21.15.179:0/583862909 shutdown_connections 2024-05-22T21:13:07.868 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.864+0000 7f9afb600640 1 -- 172.21.15.179:0/583862909 wait complete. 2024-05-22T21:13:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9afb600640 1 Processor -- start 2024-05-22T21:13:07.869 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9afb600640 1 -- start start 2024-05-22T21:13:07.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9afb600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9af40b37b0 0x7f9af41550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9afb600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9af40d0bb0 0x7f9af40ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9afb600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9af4160ad0 0x7f9af40b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:07.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9afb600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af40c8d90 con 0x7f9af4160ad0 2024-05-22T21:13:07.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9afb600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af40c8060 con 0x7f9af40d0bb0 2024-05-22T21:13:07.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9afb600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af4153690 con 0x7f9af40b37b0 2024-05-22T21:13:07.871 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9aebe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9af40d0bb0 0x7f9af40ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9af8c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9af40b37b0 0x7f9af41550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9aebe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9af40d0bb0 0x7f9af40ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40022/0 (socket says 172.21.15.179:40022) 2024-05-22T21:13:07.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9aebe00640 1 -- 172.21.15.179:0/4164061922 learned_addr learned my addr 172.21.15.179:0/4164061922 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-22T21:13:07.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9af9600640 1 --2- 172.21.15.179:0/4164061922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9af4160ad0 0x7f9af40b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:07.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9aebe00640 1 -- 172.21.15.179:0/4164061922 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9af40b37b0 msgr2=0x7f9af41550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9aebe00640 1 --2- 172.21.15.179:0/4164061922 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9af40b37b0 0x7f9af41550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9aebe00640 1 -- 172.21.15.179:0/4164061922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9af4160ad0 msgr2=0x7f9af40b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9aebe00640 1 --2- 172.21.15.179:0/4164061922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9af4160ad0 0x7f9af40b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9aebe00640 1 -- 172.21.15.179:0/4164061922 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc0094d0 con 0x7f9af40d0bb0 2024-05-22T21:13:07.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9af8c00640 1 --2- 172.21.15.179:0/4164061922 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9af40b37b0 0x7f9af41550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:07.872 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9aebe00640 1 --2- 172.21.15.179:0/4164061922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9af40d0bb0 0x7f9af40ca280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9adc01ac10 tx=0x7f9adc01ada0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:07.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9af1600640 1 -- 172.21.15.179:0/4164061922 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0b1800 con 0x7f9af40d0bb0 2024-05-22T21:13:07.873 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9af1600640 1 -- 172.21.15.179:0/4164061922 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9adc0b1980 con 0x7f9af40d0bb0 2024-05-22T21:13:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9af1600640 1 -- 172.21.15.179:0/4164061922 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0b1d30 con 0x7f9af40d0bb0 2024-05-22T21:13:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9afb600640 1 -- 172.21.15.179:0/4164061922 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af40cacf0 con 0x7f9af40d0bb0 2024-05-22T21:13:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.868+0000 7f9afb600640 1 -- 172.21.15.179:0/4164061922 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af40c8590 con 0x7f9af40d0bb0 2024-05-22T21:13:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.872+0000 7f9af1600640 1 -- 172.21.15.179:0/4164061922 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f9adc0a7070 con 0x7f9af40d0bb0 2024-05-22T21:13:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.872+0000 7f9af1600640 1 -- 172.21.15.179:0/4164061922 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f9adc0bcdb0 con 0x7f9af40d0bb0 2024-05-22T21:13:07.874 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.872+0000 7f9afb600640 1 -- 172.21.15.179:0/4164061922 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af4073ca0 con 0x7f9af40d0bb0 2024-05-22T21:13:07.875 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.872+0000 7f9af1600640 1 -- 172.21.15.179:0/4164061922 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f9adc0bc750 con 0x7f9af40d0bb0 2024-05-22T21:13:07.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.940+0000 7f9afb600640 1 -- 172.21.15.179:0/4164061922 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7} v 0) v1 -- 0x7f9af4067220 con 0x7f9af40d0bb0 2024-05-22T21:13:07.946 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.944+0000 7f9af1600640 1 -- 172.21.15.179:0/4164061922 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9adc0bca80 con 0x7f9af40d0bb0 2024-05-22T21:13:07.946 INFO:teuthology.orchestra.run.smithi179.stdout:7 2024-05-22T21:13:07.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.944+0000 7f9afb600640 1 -- 172.21.15.179:0/4164061922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9af40d0bb0 msgr2=0x7f9af40ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:07.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.944+0000 7f9afb600640 1 --2- 172.21.15.179:0/4164061922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9af40d0bb0 0x7f9af40ca280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9adc01ac10 tx=0x7f9adc01ada0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.947 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.944+0000 7f9afb600640 1 -- 172.21.15.179:0/4164061922 shutdown_connections 2024-05-22T21:13:07.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.944+0000 7f9afb600640 1 --2- 172.21.15.179:0/4164061922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9af4160ad0 0x7f9af40b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.944+0000 7f9afb600640 1 --2- 172.21.15.179:0/4164061922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9af40d0bb0 0x7f9af40ca280 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.944+0000 7f9afb600640 1 --2- 172.21.15.179:0/4164061922 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9af40b37b0 0x7f9af41550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:07.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.944+0000 7f9afb600640 1 -- 172.21.15.179:0/4164061922 >> 172.21.15.179:0/4164061922 conn(0x7f9af40b2a80 msgr2=0x7f9af40cbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:07.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.944+0000 7f9afb600640 1 -- 172.21.15.179:0/4164061922 shutdown_connections 2024-05-22T21:13:07.948 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-22T21:13:07.944+0000 7f9afb600640 1 -- 172.21.15.179:0/4164061922 wait complete. 2024-05-22T21:13:07.965 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-22T21:13:07.965 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-22T21:13:07.969 INFO:tasks.ceph.osd.0:Started 2024-05-22T21:13:07.969 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-22T21:13:07.970 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-22T21:13:07.978 INFO:tasks.ceph.osd.1:Started 2024-05-22T21:13:07.978 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-22T21:13:07.978 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-22T21:13:07.990 INFO:tasks.ceph.osd.2:Started 2024-05-22T21:13:07.990 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-22T21:13:07.990 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-22T21:13:07.994 INFO:tasks.ceph.osd.3:Started 2024-05-22T21:13:07.994 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-22T21:13:07.994 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-22T21:13:07.997 INFO:tasks.ceph.osd.4:Started 2024-05-22T21:13:07.998 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-22T21:13:07.998 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-22T21:13:08.000 INFO:tasks.ceph.osd.5:Started 2024-05-22T21:13:08.000 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-22T21:13:08.001 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-22T21:13:08.003 INFO:tasks.ceph.osd.6:Started 2024-05-22T21:13:08.003 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-22T21:13:08.004 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-22T21:13:08.006 INFO:tasks.ceph.osd.7:Started 2024-05-22T21:13:08.007 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:08.088 INFO:tasks.ceph.osd.4.smithi179.stderr:2024-05-22T21:13:08.084+0000 7f10c62a97c0 -1 Falling back to public interface 2024-05-22T21:13:08.089 INFO:tasks.ceph.osd.5.smithi179.stderr:2024-05-22T21:13:08.088+0000 7febea0127c0 -1 Falling back to public interface 2024-05-22T21:13:08.100 INFO:tasks.ceph.osd.7.smithi179.stderr:2024-05-22T21:13:08.100+0000 7f53aa5e87c0 -1 Falling back to public interface 2024-05-22T21:13:08.102 INFO:tasks.ceph.osd.6.smithi179.stderr:2024-05-22T21:13:08.100+0000 7fe976c467c0 -1 Falling back to public interface 2024-05-22T21:13:08.105 INFO:tasks.ceph.osd.4.smithi179.stderr:2024-05-22T21:13:08.104+0000 7f10c62a97c0 -1 osd.4 0 log_to_monitors true 2024-05-22T21:13:08.108 INFO:tasks.ceph.osd.1.smithi161.stderr:2024-05-22T21:13:08.108+0000 7fa9d3e2b7c0 -1 Falling back to public interface 2024-05-22T21:13:08.110 INFO:tasks.ceph.osd.5.smithi179.stderr:2024-05-22T21:13:08.108+0000 7febea0127c0 -1 osd.5 0 log_to_monitors true 2024-05-22T21:13:08.121 INFO:tasks.ceph.osd.7.smithi179.stderr:2024-05-22T21:13:08.120+0000 7f53aa5e87c0 -1 osd.7 0 log_to_monitors true 2024-05-22T21:13:08.125 INFO:tasks.ceph.osd.1.smithi161.stderr:2024-05-22T21:13:08.124+0000 7fa9d3e2b7c0 -1 osd.1 0 log_to_monitors true 2024-05-22T21:13:08.126 INFO:tasks.ceph.osd.0.smithi161.stderr:2024-05-22T21:13:08.124+0000 7fd98686e7c0 -1 Falling back to public interface 2024-05-22T21:13:08.134 INFO:tasks.ceph.osd.6.smithi179.stderr:2024-05-22T21:13:08.132+0000 7fe976c467c0 -1 osd.6 0 log_to_monitors true 2024-05-22T21:13:08.135 INFO:tasks.ceph.osd.3.smithi161.stderr:2024-05-22T21:13:08.136+0000 7f957ac2b7c0 -1 Falling back to public interface 2024-05-22T21:13:08.143 INFO:tasks.ceph.osd.2.smithi161.stderr:2024-05-22T21:13:08.144+0000 7f16888937c0 -1 Falling back to public interface 2024-05-22T21:13:08.146 INFO:tasks.ceph.osd.0.smithi161.stderr:2024-05-22T21:13:08.144+0000 7fd98686e7c0 -1 osd.0 0 log_to_monitors true 2024-05-22T21:13:08.155 INFO:tasks.ceph.osd.3.smithi161.stderr:2024-05-22T21:13:08.156+0000 7f957ac2b7c0 -1 osd.3 0 log_to_monitors true 2024-05-22T21:13:08.177 INFO:tasks.ceph.osd.2.smithi161.stderr:2024-05-22T21:13:08.176+0000 7f16888937c0 -1 osd.2 0 log_to_monitors true 2024-05-22T21:13:08.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.200+0000 7efd52600640 1 Processor -- start 2024-05-22T21:13:08.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.200+0000 7efd52600640 1 -- start start 2024-05-22T21:13:08.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.200+0000 7efd52600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efd4c0715a0 0x7efd4c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:08.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.200+0000 7efd52600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efd4c15ec90 0x7efd4c15f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:08.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.200+0000 7efd52600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd4c1442a0 con 0x7efd4c143ec0 2024-05-22T21:13:08.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.200+0000 7efd52600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd4c15f640 con 0x7efd4c0715a0 2024-05-22T21:13:08.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.200+0000 7efd52600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd4c070500 con 0x7efd4c15ec90 2024-05-22T21:13:08.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.200+0000 7efd43e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efd4c0715a0 0x7efd4c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:08.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.200+0000 7efd43e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efd4c0715a0 0x7efd4c15f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46424/0 (socket says 172.21.15.161:46424) 2024-05-22T21:13:08.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.200+0000 7efd43e00640 1 -- 172.21.15.161:0/3645576850 learned_addr learned my addr 172.21.15.161:0/3645576850 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:08.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.200+0000 7efd4be00640 1 --2- 172.21.15.161:0/3645576850 >> v2:172.21.15.179:3300/0 conn(0x7efd4c15ec90 0x7efd4c15f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:08.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.200+0000 7efd43e00640 1 -- 172.21.15.161:0/3645576850 >> v2:172.21.15.179:3300/0 conn(0x7efd4c15ec90 msgr2=0x7efd4c15f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:08.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.200+0000 7efd43e00640 1 --2- 172.21.15.161:0/3645576850 >> v2:172.21.15.179:3300/0 conn(0x7efd4c15ec90 0x7efd4c15f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:08.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.200+0000 7efd43e00640 1 -- 172.21.15.161:0/3645576850 >> v1:172.21.15.179:6789/0 conn(0x7efd4c143ec0 legacy=0x7efd4c06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:08.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.200+0000 7efd43e00640 1 -- 172.21.15.161:0/3645576850 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c0707e0 con 0x7efd4c0715a0 2024-05-22T21:13:08.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.204+0000 7efd43e00640 1 --2- 172.21.15.161:0/3645576850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c0715a0 0x7efd4c15f830 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0cedc0 tx=0x7efd3401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f7c3890eadf6bc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:08.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.204+0000 7efd4aa00640 1 -- 172.21.15.161:0/3645576850 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340ab5f0 con 0x7efd4c0715a0 2024-05-22T21:13:08.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.204+0000 7efd4aa00640 1 -- 172.21.15.161:0/3645576850 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd340ab750 con 0x7efd4c0715a0 2024-05-22T21:13:08.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.204+0000 7efd4aa00640 1 -- 172.21.15.161:0/3645576850 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340abae0 con 0x7efd4c0715a0 2024-05-22T21:13:08.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.204+0000 7efd52600640 1 -- 172.21.15.161:0/3645576850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c0715a0 msgr2=0x7efd4c15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:08.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.204+0000 7efd52600640 1 --2- 172.21.15.161:0/3645576850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c0715a0 0x7efd4c15f830 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0cedc0 tx=0x7efd3401c4f0 comp rx=0 tx=0).stop 2024-05-22T21:13:08.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.204+0000 7efd52600640 1 -- 172.21.15.161:0/3645576850 shutdown_connections 2024-05-22T21:13:08.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.204+0000 7efd52600640 1 --2- 172.21.15.161:0/3645576850 >> v2:172.21.15.179:3300/0 conn(0x7efd4c15ec90 0x7efd4c15f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:08.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.204+0000 7efd52600640 1 --2- 172.21.15.161:0/3645576850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c0715a0 0x7efd4c15f830 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:08.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.204+0000 7efd52600640 1 -- 172.21.15.161:0/3645576850 >> 172.21.15.161:0/3645576850 conn(0x7efd4c0b0bc0 msgr2=0x7efd4c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:08.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.204+0000 7efd52600640 1 -- 172.21.15.161:0/3645576850 shutdown_connections 2024-05-22T21:13:08.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.204+0000 7efd52600640 1 -- 172.21.15.161:0/3645576850 wait complete. 2024-05-22T21:13:08.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd52600640 1 Processor -- start 2024-05-22T21:13:08.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd52600640 1 -- start start 2024-05-22T21:13:08.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd52600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c143ec0 0x7efd4c15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:08.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd52600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd4c15ec90 0x7efd4c1f8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:08.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd52600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd4c15e800 0x7efd4c1f9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:08.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd52600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c1f5930 con 0x7efd4c143ec0 2024-05-22T21:13:08.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd52600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c162db0 con 0x7efd4c15e800 2024-05-22T21:13:08.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd52600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c15aaf0 con 0x7efd4c15ec90 2024-05-22T21:13:08.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd4b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c143ec0 0x7efd4c15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:08.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd4b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c143ec0 0x7efd4c15a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46432/0 (socket says 172.21.15.161:46432) 2024-05-22T21:13:08.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd4b400640 1 -- 172.21.15.161:0/3811678464 learned_addr learned my addr 172.21.15.161:0/3811678464 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:08.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd43e00640 1 --2- 172.21.15.161:0/3811678464 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd4c15ec90 0x7efd4c1f8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:08.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd4be00640 1 --2- 172.21.15.161:0/3811678464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd4c15e800 0x7efd4c1f9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:08.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd4b400640 1 -- 172.21.15.161:0/3811678464 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd4c15ec90 msgr2=0x7efd4c1f8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:08.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd4b400640 1 --2- 172.21.15.161:0/3811678464 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd4c15ec90 0x7efd4c1f8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:08.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd4b400640 1 -- 172.21.15.161:0/3811678464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd4c15e800 msgr2=0x7efd4c1f9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:08.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd4b400640 1 --2- 172.21.15.161:0/3811678464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd4c15e800 0x7efd4c1f9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:08.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd4b400640 1 -- 172.21.15.161:0/3811678464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd340094d0 con 0x7efd4c143ec0 2024-05-22T21:13:08.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd4be00640 1 --2- 172.21.15.161:0/3811678464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd4c15e800 0x7efd4c1f9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:08.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd4b400640 1 --2- 172.21.15.161:0/3811678464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c143ec0 0x7efd4c15a5b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00c530 tx=0x7efd3c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:08.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd43400640 1 -- 172.21.15.161:0/3811678464 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c005220 con 0x7efd4c143ec0 2024-05-22T21:13:08.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd43400640 1 -- 172.21.15.161:0/3811678464 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c005380 con 0x7efd4c143ec0 2024-05-22T21:13:08.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd43400640 1 -- 172.21.15.161:0/3811678464 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c007270 con 0x7efd4c143ec0 2024-05-22T21:13:08.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd52600640 1 -- 172.21.15.161:0/3811678464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4c15d070 con 0x7efd4c143ec0 2024-05-22T21:13:08.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd52600640 1 -- 172.21.15.161:0/3811678464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4c15fbb0 con 0x7efd4c143ec0 2024-05-22T21:13:08.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd43400640 1 -- 172.21.15.161:0/3811678464 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7efd3c0067e0 con 0x7efd4c143ec0 2024-05-22T21:13:08.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.208+0000 7efd43400640 1 -- 172.21.15.161:0/3811678464 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7efd3c01bad0 con 0x7efd4c143ec0 2024-05-22T21:13:08.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.212+0000 7efd52600640 1 -- 172.21.15.161:0/3811678464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4c067220 con 0x7efd4c143ec0 2024-05-22T21:13:08.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.212+0000 7efd43400640 1 -- 172.21.15.161:0/3811678464 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7efd3c016020 con 0x7efd4c143ec0 2024-05-22T21:13:08.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.268+0000 7efd52600640 1 -- 172.21.15.161:0/3811678464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd4c06dcc0 con 0x7efd4c143ec0 2024-05-22T21:13:08.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.268+0000 7efd43400640 1 -- 172.21.15.161:0/3811678464 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7efd3c02a090 con 0x7efd4c143ec0 2024-05-22T21:13:08.270 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:08.270 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":10,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:07.945035+0000","last_up_change":"0.000000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:08.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.272+0000 7efd52600640 1 -- 172.21.15.161:0/3811678464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c143ec0 msgr2=0x7efd4c15a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:08.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.272+0000 7efd52600640 1 --2- 172.21.15.161:0/3811678464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c143ec0 0x7efd4c15a5b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00c530 tx=0x7efd3c00c9f0 comp rx=0 tx=0).stop 2024-05-22T21:13:08.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.272+0000 7efd52600640 1 -- 172.21.15.161:0/3811678464 shutdown_connections 2024-05-22T21:13:08.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.272+0000 7efd52600640 1 --2- 172.21.15.161:0/3811678464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd4c15e800 0x7efd4c1f9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:08.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.272+0000 7efd52600640 1 --2- 172.21.15.161:0/3811678464 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd4c15ec90 0x7efd4c1f8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:08.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.272+0000 7efd52600640 1 --2- 172.21.15.161:0/3811678464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c143ec0 0x7efd4c15a5b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:08.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.272+0000 7efd52600640 1 -- 172.21.15.161:0/3811678464 >> 172.21.15.161:0/3811678464 conn(0x7efd4c0b0bc0 msgr2=0x7efd4c142c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:08.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.272+0000 7efd52600640 1 -- 172.21.15.161:0/3811678464 shutdown_connections 2024-05-22T21:13:08.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:08.272+0000 7efd52600640 1 -- 172.21.15.161:0/3811678464 wait complete. 2024-05-22T21:13:08.289 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-22T21:13:08.290 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-22T21:13:08.292 INFO:tasks.ceph.mgr.y.smithi161.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-05-22T21:13:08.292 INFO:tasks.ceph.mgr.y.smithi161.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-05-22T21:13:08.292 INFO:tasks.ceph.mgr.y.smithi161.stderr: from numpy import show_config as show_numpy_config 2024-05-22T21:13:08.339 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:08.340+0000 7f60af32e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-22T21:13:08.461 INFO:tasks.ceph.mgr.x.smithi179.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-05-22T21:13:08.461 INFO:tasks.ceph.mgr.x.smithi179.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-05-22T21:13:08.461 INFO:tasks.ceph.mgr.x.smithi179.stderr: from numpy import show_config as show_numpy_config 2024-05-22T21:13:08.487 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:08.488+0000 7f60af32e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-22T21:13:08.511 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:08.508+0000 7fb75831b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-22T21:13:08.657 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:08.656+0000 7fb75831b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-22T21:13:08.668 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:08.668+0000 7f60af32e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-22T21:13:08.740 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:08.740+0000 7f60af32e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-22T21:13:08.832 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:08.828+0000 7fb75831b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-22T21:13:08.887 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:08.888+0000 7f60af32e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-22T21:13:08.903 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:08.900+0000 7fb75831b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-22T21:13:09.054 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:09.052+0000 7fb75831b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-22T21:13:09.094 DEBUG:teuthology.orchestra.run.smithi161:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:09.220 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdce2c00640 1 Processor -- start 2024-05-22T21:13:09.220 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdce2c00640 1 -- start start 2024-05-22T21:13:09.221 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdce2c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcdc06a5e0 0x7fdcdc168160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:09.221 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdce2c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcdc0b4a80 con 0x7fdcdc15af80 2024-05-22T21:13:09.222 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdce2c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcdc15db30 con 0x7fdcdc15aba0 2024-05-22T21:13:09.222 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdce2c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc15de10 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.222 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdcd3e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fdcdc15af80 0x7fdcdc069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54308/0 (socket says 172.21.15.161:54308) 2024-05-22T21:13:09.222 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdcd3e00640 1 -- 172.21.15.161:0/4096231535 learned_addr learned my addr 172.21.15.161:0/4096231535 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:09.222 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdce0c00640 1 --2- 172.21.15.161:0/4096231535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcdc06a5e0 0x7fdcdc168160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:09.223 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdcdb400640 1 -- 172.21.15.161:0/4096231535 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449514923 0 0) 0x7fdcdc0b4a80 con 0x7fdcdc15af80 2024-05-22T21:13:09.223 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdcdb400640 1 -- 172.21.15.161:0/4096231535 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc0003530 con 0x7fdcdc15af80 2024-05-22T21:13:09.223 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdcdb400640 1 -- 172.21.15.161:0/4096231535 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880741425 0 0) 0x7fdcdc15db30 con 0x7fdcdc15aba0 2024-05-22T21:13:09.223 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdcdb400640 1 -- 172.21.15.161:0/4096231535 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcdc0b4a80 con 0x7fdcdc15aba0 2024-05-22T21:13:09.223 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdcdb400640 1 -- 172.21.15.161:0/4096231535 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1849438431 0 0) 0x7fdcc0003530 con 0x7fdcdc15af80 2024-05-22T21:13:09.223 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdce0c00640 1 -- 172.21.15.161:0/4096231535 >> v1:172.21.15.179:6789/0 conn(0x7fdcdc15aba0 legacy=0x7fdcdc0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:09.223 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdce0c00640 1 -- 172.21.15.161:0/4096231535 >> v1:172.21.15.161:6789/0 conn(0x7fdcdc15af80 legacy=0x7fdcdc069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:09.223 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.220+0000 7fdce0c00640 1 -- 172.21.15.161:0/4096231535 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc15e0f0 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.227 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.228+0000 7fdce0c00640 1 --2- 172.21.15.161:0/4096231535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcdc06a5e0 0x7fdcdc168160 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009dc0 tx=0x7fdcd401c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deb8fd3e8b13e629 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:09.227 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.228+0000 7fdcdb400640 1 -- 172.21.15.161:0/4096231535 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4001680 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.227 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.228+0000 7fdcdb400640 1 -- 172.21.15.161:0/4096231535 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd40a5710 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.228 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.228+0000 7fdcdb400640 1 -- 172.21.15.161:0/4096231535 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd40a5a80 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.228 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.228+0000 7fdce2c00640 1 -- 172.21.15.161:0/4096231535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcdc06a5e0 msgr2=0x7fdcdc168160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:09.228 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.228+0000 7fdce2c00640 1 --2- 172.21.15.161:0/4096231535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcdc06a5e0 0x7fdcdc168160 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009dc0 tx=0x7fdcd401c870 comp rx=0 tx=0).stop 2024-05-22T21:13:09.228 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.228+0000 7fdce2c00640 1 -- 172.21.15.161:0/4096231535 shutdown_connections 2024-05-22T21:13:09.228 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.228+0000 7fdce2c00640 1 --2- 172.21.15.161:0/4096231535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcdc06a5e0 0x7fdcdc168160 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:09.228 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.228+0000 7fdce2c00640 1 -- 172.21.15.161:0/4096231535 >> 172.21.15.161:0/4096231535 conn(0x7fdcdc06afe0 msgr2=0x7fdcdc06b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:09.229 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.228+0000 7fdce2c00640 1 -- 172.21.15.161:0/4096231535 shutdown_connections 2024-05-22T21:13:09.229 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.228+0000 7fdce2c00640 1 -- 172.21.15.161:0/4096231535 wait complete. 2024-05-22T21:13:09.230 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.228+0000 7fdce2c00640 1 Processor -- start 2024-05-22T21:13:09.230 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.228+0000 7fdce2c00640 1 -- start start 2024-05-22T21:13:09.232 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdce2c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcdc06a5e0 0x7fdcdc15fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:09.232 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdce2c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcdc15aba0 0x7fdcdc147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:09.232 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdce2c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcdc15af80 0x7fdcdc1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:09.232 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdce2c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc15f970 con 0x7fdcdc15aba0 2024-05-22T21:13:09.232 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdce2c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc0af050 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.232 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdce2c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc147a40 con 0x7fdcdc15af80 2024-05-22T21:13:09.232 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdcdbe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcdc06a5e0 0x7fdcdc15fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:09.232 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdce0c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcdc15af80 0x7fdcdc1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:09.232 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdcdbe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcdc06a5e0 0x7fdcdc15fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:50618/0 (socket says 172.21.15.161:50618) 2024-05-22T21:13:09.232 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdcdbe00640 1 -- 172.21.15.161:0/3124697580 learned_addr learned my addr 172.21.15.161:0/3124697580 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:09.233 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdcd3e00640 1 --2- 172.21.15.161:0/3124697580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcdc15aba0 0x7fdcdc147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:09.233 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdcdbe00640 1 -- 172.21.15.161:0/3124697580 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcdc15af80 msgr2=0x7fdcdc1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:09.233 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdcdbe00640 1 --2- 172.21.15.161:0/3124697580 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcdc15af80 0x7fdcdc1501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:09.233 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdcdbe00640 1 -- 172.21.15.161:0/3124697580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcdc15aba0 msgr2=0x7fdcdc147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:09.233 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdcdbe00640 1 --2- 172.21.15.161:0/3124697580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcdc15aba0 0x7fdcdc147420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:09.233 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdcdbe00640 1 -- 172.21.15.161:0/3124697580 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd4009a70 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.233 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdce0c00640 1 --2- 172.21.15.161:0/3124697580 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcdc15af80 0x7fdcdc1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:09.233 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdcd3e00640 1 --2- 172.21.15.161:0/3124697580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcdc15aba0 0x7fdcdc147420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:09.233 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdcdbe00640 1 --2- 172.21.15.161:0/3124697580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcdc06a5e0 0x7fdcdc15fe50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00c490 tx=0x7fdccc00c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:09.234 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdcd8c00640 1 -- 172.21.15.161:0/3124697580 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc0052d0 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.234 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdcd8c00640 1 -- 172.21.15.161:0/3124697580 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdccc005430 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.234 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdce2c00640 1 -- 172.21.15.161:0/3124697580 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcdc1593d0 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.234 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdcd8c00640 1 -- 172.21.15.161:0/3124697580 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc012870 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.234 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdce2c00640 1 -- 172.21.15.161:0/3124697580 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcdc159ca0 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.234 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdcd8c00640 1 -- 172.21.15.161:0/3124697580 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fdccc012cd0 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.235 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdcd8c00640 1 -- 172.21.15.161:0/3124697580 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1883+0+0 (secure 0 0 0) 0x7fdccc011380 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.235 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.232+0000 7fdce2c00640 1 -- 172.21.15.161:0/3124697580 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc0671b0 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.236 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.236+0000 7fdcd8c00640 1 -- 172.21.15.161:0/3124697580 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fdccc003600 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.306 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.304+0000 7fdce2c00640 1 -- 172.21.15.161:0/3124697580 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcdc06d650 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.306 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.304+0000 7fdcd8c00640 1 -- 172.21.15.161:0/3124697580 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fdccc0037a0 con 0x7fdcdc06a5e0 2024-05-22T21:13:09.307 INFO:teuthology.misc.health.smithi161.stdout: 2024-05-22T21:13:09.307 INFO:teuthology.misc.health.smithi161.stdout:{"epoch":11,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:08.946571+0000","last_up_change":"0.000000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:09.308 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.308+0000 7fdce2c00640 1 -- 172.21.15.161:0/3124697580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcdc06a5e0 msgr2=0x7fdcdc15fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:09.308 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.308+0000 7fdce2c00640 1 --2- 172.21.15.161:0/3124697580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcdc06a5e0 0x7fdcdc15fe50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00c490 tx=0x7fdccc00c950 comp rx=0 tx=0).stop 2024-05-22T21:13:09.308 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.308+0000 7fdce2c00640 1 -- 172.21.15.161:0/3124697580 shutdown_connections 2024-05-22T21:13:09.309 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.308+0000 7fdce2c00640 1 --2- 172.21.15.161:0/3124697580 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcdc15af80 0x7fdcdc1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:09.309 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.308+0000 7fdce2c00640 1 --2- 172.21.15.161:0/3124697580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcdc15aba0 0x7fdcdc147420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:09.309 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.308+0000 7fdce2c00640 1 --2- 172.21.15.161:0/3124697580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcdc06a5e0 0x7fdcdc15fe50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:09.309 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.308+0000 7fdce2c00640 1 -- 172.21.15.161:0/3124697580 >> 172.21.15.161:0/3124697580 conn(0x7fdcdc06afe0 msgr2=0x7fdcdc069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:09.309 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.308+0000 7fdce2c00640 1 -- 172.21.15.161:0/3124697580 shutdown_connections 2024-05-22T21:13:09.309 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:09.308+0000 7fdce2c00640 1 -- 172.21.15.161:0/3124697580 wait complete. 2024-05-22T21:13:09.326 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-22T21:13:09.591 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:09.592+0000 7f60af32e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-22T21:13:09.677 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:09.676+0000 7f60af32e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-22T21:13:09.729 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:09.728+0000 7fb75831b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-22T21:13:09.819 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:09.812+0000 7fb75831b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-22T21:13:09.875 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:09.876+0000 7f60af32e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-22T21:13:09.966 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:09.964+0000 7f60af32e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-22T21:13:09.978 INFO:tasks.ceph.osd.2.smithi161.stderr:2024-05-22T21:13:09.976+0000 7f1683800640 -1 osd.2 0 waiting for initial osdmap 2024-05-22T21:13:09.979 INFO:tasks.ceph.osd.6.smithi179.stderr:2024-05-22T21:13:09.976+0000 7fe971c00640 -1 osd.6 0 waiting for initial osdmap 2024-05-22T21:13:09.979 INFO:tasks.ceph.osd.2.smithi161.stderr:2024-05-22T21:13:09.980+0000 7f167b600640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T21:13:09.981 INFO:tasks.ceph.osd.6.smithi179.stderr:2024-05-22T21:13:09.980+0000 7fe969a00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T21:13:09.996 INFO:tasks.ceph.osd.1.smithi161.stderr:2024-05-22T21:13:09.996+0000 7fa9cec00640 -1 osd.1 0 waiting for initial osdmap 2024-05-22T21:13:09.996 INFO:tasks.ceph.osd.4.smithi179.stderr:2024-05-22T21:13:09.996+0000 7f10c1200640 -1 osd.4 0 waiting for initial osdmap 2024-05-22T21:13:09.996 INFO:tasks.ceph.osd.3.smithi161.stderr:2024-05-22T21:13:09.996+0000 7f9575a00640 -1 osd.3 0 waiting for initial osdmap 2024-05-22T21:13:09.997 INFO:tasks.ceph.osd.7.smithi179.stderr:2024-05-22T21:13:09.996+0000 7f53a5400640 -1 osd.7 0 waiting for initial osdmap 2024-05-22T21:13:09.997 INFO:tasks.ceph.osd.5.smithi179.stderr:2024-05-22T21:13:09.996+0000 7febe4e00640 -1 osd.5 0 waiting for initial osdmap 2024-05-22T21:13:09.998 INFO:tasks.ceph.osd.0.smithi161.stderr:2024-05-22T21:13:09.996+0000 7fd981800640 -1 osd.0 0 waiting for initial osdmap 2024-05-22T21:13:09.998 INFO:tasks.ceph.osd.1.smithi161.stderr:2024-05-22T21:13:09.996+0000 7fa9c6000640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T21:13:09.998 INFO:tasks.ceph.osd.3.smithi161.stderr:2024-05-22T21:13:09.996+0000 7f956ce00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T21:13:09.999 INFO:tasks.ceph.osd.4.smithi179.stderr:2024-05-22T21:13:09.996+0000 7f10b9000640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T21:13:10.000 INFO:tasks.ceph.osd.7.smithi179.stderr:2024-05-22T21:13:09.996+0000 7f539d200640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T21:13:10.000 INFO:tasks.ceph.osd.5.smithi179.stderr:2024-05-22T21:13:10.000+0000 7febdcc00640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T21:13:10.000 INFO:tasks.ceph.osd.0.smithi161.stderr:2024-05-22T21:13:10.000+0000 7fd979600640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-22T21:13:10.041 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:10.040+0000 7fb75831b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-22T21:13:10.124 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:10.124+0000 7fb75831b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-22T21:13:10.306 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:10.304+0000 7f60af32e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-22T21:13:10.375 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:10.376+0000 7f60af32e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-22T21:13:10.458 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:10.456+0000 7fb75831b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-22T21:13:10.528 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-05-22T21:13:10.528+0000 7fb75831b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-22T21:13:11.399 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:11.400+0000 7f6055400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-22T21:13:11.399 INFO:tasks.ceph.mgr.y.smithi161.stderr:2024-05-22T21:13:11.400+0000 7f6055400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-22T21:13:16.134 DEBUG:teuthology.orchestra.run.smithi161:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:16.270 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.267+0000 7ff91ce00640 1 Processor -- start 2024-05-22T21:13:16.270 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.267+0000 7ff91ce00640 1 -- start start 2024-05-22T21:13:16.271 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.267+0000 7ff91ce00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff918144260 0x7ff9180ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.271 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.267+0000 7ff91ce00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff91806fcc0 0x7ff918070110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.271 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.267+0000 7ff91ce00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff91815edb0 con 0x7ff91815b590 2024-05-22T21:13:16.271 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.267+0000 7ff91ce00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff918144bb0 con 0x7ff918144260 2024-05-22T21:13:16.271 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.267+0000 7ff91ce00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff918144640 con 0x7ff91806fcc0 2024-05-22T21:13:16.271 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.271+0000 7ff915e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff918144260 0x7ff9180ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.271 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.271+0000 7ff915e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff918144260 0x7ff9180ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41108/0 (socket says 172.21.15.161:41108) 2024-05-22T21:13:16.272 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.271+0000 7ff915e00640 1 -- 172.21.15.161:0/1685281868 learned_addr learned my addr 172.21.15.161:0/1685281868 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:16.272 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.271+0000 7ff916800640 1 --2- 172.21.15.161:0/1685281868 >> v2:172.21.15.179:3300/0 conn(0x7ff91806fcc0 0x7ff918070110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.272 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.271+0000 7ff915e00640 1 -- 172.21.15.161:0/1685281868 >> v2:172.21.15.179:3300/0 conn(0x7ff91806fcc0 msgr2=0x7ff918070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:16.272 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.271+0000 7ff915e00640 1 --2- 172.21.15.161:0/1685281868 >> v2:172.21.15.179:3300/0 conn(0x7ff91806fcc0 0x7ff918070110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.272 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.271+0000 7ff915e00640 1 -- 172.21.15.161:0/1685281868 >> v1:172.21.15.179:6789/0 conn(0x7ff91815b590 legacy=0x7ff91815f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:16.272 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.271+0000 7ff915e00640 1 -- 172.21.15.161:0/1685281868 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91815bdb0 con 0x7ff918144260 2024-05-22T21:13:16.274 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.271+0000 7ff915e00640 1 --2- 172.21.15.161:0/1685281868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff918144260 0x7ff9180ce760 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff91815eae0 tx=0x7ff90801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b02c7777ae08edf server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:16.275 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.271+0000 7ff915400640 1 -- 172.21.15.161:0/1685281868 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9080a5840 con 0x7ff918144260 2024-05-22T21:13:16.275 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff915400640 1 -- 172.21.15.161:0/1685281868 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9080a59a0 con 0x7ff918144260 2024-05-22T21:13:16.275 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff915400640 1 -- 172.21.15.161:0/1685281868 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9080a5d30 con 0x7ff918144260 2024-05-22T21:13:16.275 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff91ce00640 1 -- 172.21.15.161:0/1685281868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff918144260 msgr2=0x7ff9180ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.276 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff91ce00640 1 --2- 172.21.15.161:0/1685281868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff918144260 0x7ff9180ce760 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff91815eae0 tx=0x7ff90801c980 comp rx=0 tx=0).stop 2024-05-22T21:13:16.276 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff91ce00640 1 -- 172.21.15.161:0/1685281868 shutdown_connections 2024-05-22T21:13:16.276 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff91ce00640 1 --2- 172.21.15.161:0/1685281868 >> v2:172.21.15.179:3300/0 conn(0x7ff91806fcc0 0x7ff918070110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.276 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff91ce00640 1 --2- 172.21.15.161:0/1685281868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff918144260 0x7ff9180ce760 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.276 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff91ce00640 1 -- 172.21.15.161:0/1685281868 >> 172.21.15.161:0/1685281868 conn(0x7ff91806a2d0 msgr2=0x7ff91806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:16.276 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff91ce00640 1 -- 172.21.15.161:0/1685281868 shutdown_connections 2024-05-22T21:13:16.276 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff91ce00640 1 -- 172.21.15.161:0/1685281868 wait complete. 2024-05-22T21:13:16.277 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff91ce00640 1 Processor -- start 2024-05-22T21:13:16.277 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff91ce00640 1 -- start start 2024-05-22T21:13:16.279 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff91ce00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91806fcc0 0x7ff9180aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.279 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff91ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff918144260 0x7ff918136790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.279 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff91ce00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff91815b590 0x7ff9181589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.279 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff91ce00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91815ae40 con 0x7ff918144260 2024-05-22T21:13:16.279 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff91ce00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff918159f40 con 0x7ff91806fcc0 2024-05-22T21:13:16.279 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff91ce00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91815a190 con 0x7ff91815b590 2024-05-22T21:13:16.279 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff90d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff918144260 0x7ff918136790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.279 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff915e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91806fcc0 0x7ff9180aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.280 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff915e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91806fcc0 0x7ff9180aeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:49760/0 (socket says 172.21.15.161:49760) 2024-05-22T21:13:16.280 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff915e00640 1 -- 172.21.15.161:0/1886728895 learned_addr learned my addr 172.21.15.161:0/1886728895 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:16.280 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff90d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff918144260 0x7ff918136790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41118/0 (socket says 172.21.15.161:41118) 2024-05-22T21:13:16.280 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.275+0000 7ff916800640 1 --2- 172.21.15.161:0/1886728895 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff91815b590 0x7ff9181589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.280 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff90d400640 1 -- 172.21.15.161:0/1886728895 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff91815b590 msgr2=0x7ff9181589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.280 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff90d400640 1 --2- 172.21.15.161:0/1886728895 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff91815b590 0x7ff9181589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.280 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff90d400640 1 -- 172.21.15.161:0/1886728895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91806fcc0 msgr2=0x7ff9180aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.280 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff90d400640 1 --2- 172.21.15.161:0/1886728895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91806fcc0 0x7ff9180aeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.280 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff90d400640 1 -- 172.21.15.161:0/1886728895 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff908009b30 con 0x7ff918144260 2024-05-22T21:13:16.280 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff915e00640 1 --2- 172.21.15.161:0/1886728895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91806fcc0 0x7ff9180aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:16.280 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff90d400640 1 --2- 172.21.15.161:0/1886728895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff918144260 0x7ff918136790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff90000b410 tx=0x7ff90000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:16.281 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff90ea00640 1 -- 172.21.15.161:0/1886728895 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff900004020 con 0x7ff918144260 2024-05-22T21:13:16.281 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff90ea00640 1 -- 172.21.15.161:0/1886728895 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff900004180 con 0x7ff918144260 2024-05-22T21:13:16.281 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff91ce00640 1 -- 172.21.15.161:0/1886728895 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91815a470 con 0x7ff918144260 2024-05-22T21:13:16.281 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff90ea00640 1 -- 172.21.15.161:0/1886728895 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff900011630 con 0x7ff918144260 2024-05-22T21:13:16.282 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff91ce00640 1 -- 172.21.15.161:0/1886728895 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9181320b0 con 0x7ff918144260 2024-05-22T21:13:16.282 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff91ce00640 1 -- 172.21.15.161:0/1886728895 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9181322d0 con 0x7ff918144260 2024-05-22T21:13:16.282 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff90ea00640 1 -- 172.21.15.161:0/1886728895 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff900011790 con 0x7ff918144260 2024-05-22T21:13:16.282 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff90ea00640 1 --2- 172.21.15.161:0/1886728895 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8f4066690 0x7ff8f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.282 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff90ea00640 1 -- 172.21.15.161:0/1886728895 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff900083a70 con 0x7ff918144260 2024-05-22T21:13:16.282 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.279+0000 7ff915e00640 1 --2- 172.21.15.161:0/1886728895 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8f4066690 0x7ff8f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.283 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.283+0000 7ff915e00640 1 --2- 172.21.15.161:0/1886728895 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8f4066690 0x7ff8f4068b50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff908000f90 tx=0x7ff9080a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:16.287 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.287+0000 7ff90ea00640 1 -- 172.21.15.161:0/1886728895 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff900051140 con 0x7ff918144260 2024-05-22T21:13:16.447 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.443+0000 7ff91ce00640 1 -- 172.21.15.161:0/1886728895 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9180671b0 con 0x7ff918144260 2024-05-22T21:13:16.447 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.447+0000 7ff90ea00640 1 -- 172.21.15.161:0/1886728895 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12430 (secure 0 0 0) 0x7ff900015090 con 0x7ff918144260 2024-05-22T21:13:16.448 INFO:teuthology.misc.health.smithi161.stdout: 2024-05-22T21:13:16.448 INFO:teuthology.misc.health.smithi161.stdout:{"epoch":17,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:14.993684+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:16.450 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.447+0000 7ff91ce00640 1 -- 172.21.15.161:0/1886728895 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8f4066690 msgr2=0x7ff8f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.450 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.447+0000 7ff91ce00640 1 --2- 172.21.15.161:0/1886728895 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8f4066690 0x7ff8f4068b50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff908000f90 tx=0x7ff9080a7070 comp rx=0 tx=0).stop 2024-05-22T21:13:16.450 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.447+0000 7ff91ce00640 1 -- 172.21.15.161:0/1886728895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff918144260 msgr2=0x7ff918136790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.450 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.447+0000 7ff91ce00640 1 --2- 172.21.15.161:0/1886728895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff918144260 0x7ff918136790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff90000b410 tx=0x7ff90000b8d0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.450 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.447+0000 7ff91ce00640 1 -- 172.21.15.161:0/1886728895 shutdown_connections 2024-05-22T21:13:16.450 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.447+0000 7ff91ce00640 1 --2- 172.21.15.161:0/1886728895 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8f4066690 0x7ff8f4068b50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.450 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.447+0000 7ff91ce00640 1 --2- 172.21.15.161:0/1886728895 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff91815b590 0x7ff9181589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.450 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.447+0000 7ff91ce00640 1 --2- 172.21.15.161:0/1886728895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff918144260 0x7ff918136790 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.450 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.447+0000 7ff91ce00640 1 --2- 172.21.15.161:0/1886728895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91806fcc0 0x7ff9180aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.451 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.447+0000 7ff91ce00640 1 -- 172.21.15.161:0/1886728895 >> 172.21.15.161:0/1886728895 conn(0x7ff91806a2d0 msgr2=0x7ff918071570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:16.451 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.447+0000 7ff91ce00640 1 -- 172.21.15.161:0/1886728895 shutdown_connections 2024-05-22T21:13:16.451 INFO:teuthology.misc.health.smithi161.stderr:2024-05-22T21:13:16.447+0000 7ff91ce00640 1 -- 172.21.15.161:0/1886728895 wait complete. 2024-05-22T21:13:16.467 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-22T21:13:16.467 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-22T21:13:16.468 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-22T21:13:16.468 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-22T21:13:16.471 INFO:tasks.ceph.mds.a:Started 2024-05-22T21:13:16.471 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-22T21:13:16.471 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-22T21:13:16.474 INFO:tasks.ceph.mds.c:Started 2024-05-22T21:13:16.474 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-22T21:13:16.474 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-22T21:13:16.478 INFO:tasks.ceph.mds.b:Started 2024-05-22T21:13:16.478 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-22T21:13:16.478 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-22T21:13:16.482 INFO:tasks.ceph.mds.d:Started 2024-05-22T21:13:16.482 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-22T21:13:16.483 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:16.577 INFO:tasks.ceph.mds.c.smithi161.stdout:starting mds.c at 2024-05-22T21:13:16.580 INFO:tasks.ceph.mds.a.smithi161.stdout:starting mds.a at 2024-05-22T21:13:16.585 INFO:tasks.ceph.mds.d.smithi179.stdout:starting mds.d at 2024-05-22T21:13:16.585 INFO:tasks.ceph.mds.b.smithi179.stdout:starting mds.b at 2024-05-22T21:13:16.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc41200640 1 Processor -- start 2024-05-22T21:13:16.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc41200640 1 -- start start 2024-05-22T21:13:16.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc41200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efc3c070db0 0x7efc3c0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc41200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3c159bd0 con 0x7efc3c0b2ea0 2024-05-22T21:13:16.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc41200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3c0cbdd0 con 0x7efc3c1597f0 2024-05-22T21:13:16.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc41200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3c0cc050 con 0x7efc3c070db0 2024-05-22T21:13:16.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc3a800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7efc3c0b2ea0 0x7efc3c0cc550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51140/0 (socket says 172.21.15.161:51140) 2024-05-22T21:13:16.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc3a800640 1 -- 172.21.15.161:0/3201092665 learned_addr learned my addr 172.21.15.161:0/3201092665 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:16.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc33e00640 1 --2- 172.21.15.161:0/3201092665 >> v2:172.21.15.179:3300/0 conn(0x7efc3c070db0 0x7efc3c0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc39400640 1 -- 172.21.15.161:0/3201092665 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161716658 0 0) 0x7efc3c159bd0 con 0x7efc3c0b2ea0 2024-05-22T21:13:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc39400640 1 -- 172.21.15.161:0/3201092665 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc20003530 con 0x7efc3c0b2ea0 2024-05-22T21:13:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc39400640 1 -- 172.21.15.161:0/3201092665 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2583114160 0 0) 0x7efc20003530 con 0x7efc3c0b2ea0 2024-05-22T21:13:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc39400640 1 -- 172.21.15.161:0/3201092665 >> v2:172.21.15.179:3300/0 conn(0x7efc3c070db0 msgr2=0x7efc3c0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc39400640 1 --2- 172.21.15.161:0/3201092665 >> v2:172.21.15.179:3300/0 conn(0x7efc3c070db0 0x7efc3c0b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc39400640 1 -- 172.21.15.161:0/3201092665 >> v1:172.21.15.179:6789/0 conn(0x7efc3c1597f0 legacy=0x7efc3c0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc39400640 1 -- 172.21.15.161:0/3201092665 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c0b3720 con 0x7efc3c0b2ea0 2024-05-22T21:13:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc39400640 1 -- 172.21.15.161:0/3201092665 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7efc34001a20 con 0x7efc3c0b2ea0 2024-05-22T21:13:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc39400640 1 -- 172.21.15.161:0/3201092665 >> v1:172.21.15.161:6789/0 conn(0x7efc3c0b2ea0 legacy=0x7efc3c0cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc39400640 1 --2- 172.21.15.161:0/3201092665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc20003980 0x7efc20003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc39400640 1 --2- 172.21.15.161:0/3201092665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc200047d0 0x7efc20004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc39400640 1 --2- 172.21.15.161:0/3201092665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc20005190 0x7efc2000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc39400640 1 -- 172.21.15.161:0/3201092665 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c0cc050 con 0x7efc20005190 2024-05-22T21:13:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc39400640 1 -- 172.21.15.161:0/3201092665 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2000c030 con 0x7efc200047d0 2024-05-22T21:13:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc39400640 1 -- 172.21.15.161:0/3201092665 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2000c1f0 con 0x7efc20003980 2024-05-22T21:13:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc3a800640 1 --2- 172.21.15.161:0/3201092665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc20005190 0x7efc2000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.627+0000 7efc33e00640 1 --2- 172.21.15.161:0/3201092665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc200047d0 0x7efc20004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.631+0000 7efc39e00640 1 --2- 172.21.15.161:0/3201092665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc20003980 0x7efc20003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.631+0000 7efc3a800640 1 -- 172.21.15.161:0/3201092665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc20003980 msgr2=0x7efc20003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.631+0000 7efc3a800640 1 --2- 172.21.15.161:0/3201092665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc20003980 0x7efc20003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.631+0000 7efc3a800640 1 -- 172.21.15.161:0/3201092665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc200047d0 msgr2=0x7efc20004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.631+0000 7efc3a800640 1 --2- 172.21.15.161:0/3201092665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc200047d0 0x7efc20004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.631+0000 7efc3a800640 1 -- 172.21.15.161:0/3201092665 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc20003530 con 0x7efc20005190 2024-05-22T21:13:16.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.631+0000 7efc33e00640 1 --2- 172.21.15.161:0/3201092665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc200047d0 0x7efc20004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:16.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.631+0000 7efc3a800640 1 -- 172.21.15.161:0/3201092665 reap_dead start 2024-05-22T21:13:16.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.631+0000 7efc3a800640 1 --2- 172.21.15.161:0/3201092665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc20005190 0x7efc2000b990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efc3c159d60 tx=0x7efc3401d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52287d3877c976b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:16.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.631+0000 7efc39400640 1 -- 172.21.15.161:0/3201092665 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc340ad630 con 0x7efc20005190 2024-05-22T21:13:16.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.631+0000 7efc39400640 1 -- 172.21.15.161:0/3201092665 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc340a8070 con 0x7efc20005190 2024-05-22T21:13:16.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.631+0000 7efc39400640 1 -- 172.21.15.161:0/3201092665 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc340ad990 con 0x7efc20005190 2024-05-22T21:13:16.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.631+0000 7efc41200640 1 -- 172.21.15.161:0/3201092665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc20005190 msgr2=0x7efc2000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.631+0000 7efc41200640 1 --2- 172.21.15.161:0/3201092665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc20005190 0x7efc2000b990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efc3c159d60 tx=0x7efc3401d320 comp rx=0 tx=0).stop 2024-05-22T21:13:16.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc41200640 1 -- 172.21.15.161:0/3201092665 shutdown_connections 2024-05-22T21:13:16.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc41200640 1 --2- 172.21.15.161:0/3201092665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc20005190 0x7efc2000b990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc41200640 1 -- 172.21.15.161:0/3201092665 >> 172.21.15.161:0/3201092665 conn(0x7efc3c0b0bc0 msgr2=0x7efc3c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:16.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc41200640 1 -- 172.21.15.161:0/3201092665 shutdown_connections 2024-05-22T21:13:16.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc41200640 1 -- 172.21.15.161:0/3201092665 wait complete. 2024-05-22T21:13:16.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc41200640 1 Processor -- start 2024-05-22T21:13:16.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc41200640 1 -- start start 2024-05-22T21:13:16.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc41200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc20005190 0x7efc3c14d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc41200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc3c15c650 0x7efc3c15b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc41200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc3c0a84a0 0x7efc3c0a8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc41200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c15b690 con 0x7efc3c0a84a0 2024-05-22T21:13:16.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc41200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c0a8e60 con 0x7efc20005190 2024-05-22T21:13:16.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc41200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c1f2d00 con 0x7efc3c15c650 2024-05-22T21:13:16.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc3a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc3c0a84a0 0x7efc3c0a8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc3a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc3c0a84a0 0x7efc3c0a8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41186/0 (socket says 172.21.15.161:41186) 2024-05-22T21:13:16.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc3a800640 1 -- 172.21.15.161:0/910098016 learned_addr learned my addr 172.21.15.161:0/910098016 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:16.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc33e00640 1 --2- 172.21.15.161:0/910098016 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc3c15c650 0x7efc3c15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc39e00640 1 --2- 172.21.15.161:0/910098016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc20005190 0x7efc3c14d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc3a800640 1 -- 172.21.15.161:0/910098016 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc3c15c650 msgr2=0x7efc3c15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc3a800640 1 --2- 172.21.15.161:0/910098016 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc3c15c650 0x7efc3c15b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc3a800640 1 -- 172.21.15.161:0/910098016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc20005190 msgr2=0x7efc3c14d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc3a800640 1 --2- 172.21.15.161:0/910098016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc20005190 0x7efc3c14d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc3a800640 1 -- 172.21.15.161:0/910098016 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c0a9280 con 0x7efc3c0a84a0 2024-05-22T21:13:16.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc3a800640 1 --2- 172.21.15.161:0/910098016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc3c0a84a0 0x7efc3c0a8920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efc3401d150 tx=0x7efc20005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:16.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc32000640 1 -- 172.21.15.161:0/910098016 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc34004d70 con 0x7efc3c0a84a0 2024-05-22T21:13:16.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc32000640 1 -- 172.21.15.161:0/910098016 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc340019f0 con 0x7efc3c0a84a0 2024-05-22T21:13:16.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc41200640 1 -- 172.21.15.161:0/910098016 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3c15b8a0 con 0x7efc3c0a84a0 2024-05-22T21:13:16.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.635+0000 7efc32000640 1 -- 172.21.15.161:0/910098016 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc340ba680 con 0x7efc3c0a84a0 2024-05-22T21:13:16.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.639+0000 7efc41200640 1 -- 172.21.15.161:0/910098016 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3c14e830 con 0x7efc3c0a84a0 2024-05-22T21:13:16.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.639+0000 7efc41200640 1 -- 172.21.15.161:0/910098016 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3c067220 con 0x7efc3c0a84a0 2024-05-22T21:13:16.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.639+0000 7efc32000640 1 -- 172.21.15.161:0/910098016 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efc340b6070 con 0x7efc3c0a84a0 2024-05-22T21:13:16.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.639+0000 7efc32000640 1 --2- 172.21.15.161:0/910098016 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc18066690 0x7efc18068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.639+0000 7efc32000640 1 -- 172.21.15.161:0/910098016 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efc3411e540 con 0x7efc3c0a84a0 2024-05-22T21:13:16.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.639+0000 7efc39e00640 1 --2- 172.21.15.161:0/910098016 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc18066690 0x7efc18068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.639+0000 7efc39e00640 1 --2- 172.21.15.161:0/910098016 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc18066690 0x7efc18068b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efc2c000c10 tx=0x7efc2c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:16.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.643+0000 7efc32000640 1 -- 172.21.15.161:0/910098016 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efc340ebb70 con 0x7efc3c0a84a0 2024-05-22T21:13:16.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.799+0000 7efc41200640 1 -- 172.21.15.161:0/910098016 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc3c0afe50 con 0x7efc3c0a84a0 2024-05-22T21:13:16.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.799+0000 7efc32000640 1 -- 172.21.15.161:0/910098016 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12430 (secure 0 0 0) 0x7efc340ef190 con 0x7efc3c0a84a0 2024-05-22T21:13:16.802 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:16.802 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":17,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:14.993684+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:16.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.803+0000 7efc41200640 1 -- 172.21.15.161:0/910098016 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc18066690 msgr2=0x7efc18068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.803+0000 7efc41200640 1 --2- 172.21.15.161:0/910098016 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc18066690 0x7efc18068b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efc2c000c10 tx=0x7efc2c009040 comp rx=0 tx=0).stop 2024-05-22T21:13:16.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.803+0000 7efc41200640 1 -- 172.21.15.161:0/910098016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc3c0a84a0 msgr2=0x7efc3c0a8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.803+0000 7efc41200640 1 --2- 172.21.15.161:0/910098016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc3c0a84a0 0x7efc3c0a8920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efc3401d150 tx=0x7efc20005800 comp rx=0 tx=0).stop 2024-05-22T21:13:16.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.803+0000 7efc41200640 1 -- 172.21.15.161:0/910098016 shutdown_connections 2024-05-22T21:13:16.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.803+0000 7efc41200640 1 --2- 172.21.15.161:0/910098016 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc18066690 0x7efc18068b50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.803+0000 7efc41200640 1 --2- 172.21.15.161:0/910098016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc3c0a84a0 0x7efc3c0a8920 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.803+0000 7efc41200640 1 --2- 172.21.15.161:0/910098016 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc3c15c650 0x7efc3c15b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.803+0000 7efc41200640 1 --2- 172.21.15.161:0/910098016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc20005190 0x7efc3c14d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.803+0000 7efc41200640 1 -- 172.21.15.161:0/910098016 >> 172.21.15.161:0/910098016 conn(0x7efc3c0b0bc0 msgr2=0x7efc3c0aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:16.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.803+0000 7efc41200640 1 -- 172.21.15.161:0/910098016 shutdown_connections 2024-05-22T21:13:16.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.803+0000 7efc41200640 1 -- 172.21.15.161:0/910098016 wait complete. 2024-05-22T21:13:16.822 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:13:16.822 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:13:16.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc32ac00640 1 Processor -- start 2024-05-22T21:13:16.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc32ac00640 1 -- start start 2024-05-22T21:13:16.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc32ac00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc3240d1e60 0x7fc3240d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc32ac00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3240cf060 con 0x7fc3240d38a0 2024-05-22T21:13:16.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc32ac00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3240d0f00 con 0x7fc32415d500 2024-05-22T21:13:16.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc32ac00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3240d11e0 con 0x7fc3240d1e60 2024-05-22T21:13:16.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc328c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc3240d38a0 0x7fc3240c7580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51152/0 (socket says 172.21.15.161:51152) 2024-05-22T21:13:16.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc328c00640 1 -- 172.21.15.161:0/1200826381 learned_addr learned my addr 172.21.15.161:0/1200826381 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:16.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc323400640 1 --2- 172.21.15.161:0/1200826381 >> v2:172.21.15.179:3300/0 conn(0x7fc3240d1e60 0x7fc3240d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc322a00640 1 -- 172.21.15.161:0/1200826381 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101818970 0 0) 0x7fc3240cf060 con 0x7fc3240d38a0 2024-05-22T21:13:16.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc322a00640 1 -- 172.21.15.161:0/1200826381 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc308003530 con 0x7fc3240d38a0 2024-05-22T21:13:16.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc322a00640 1 -- 172.21.15.161:0/1200826381 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3237754566 0 0) 0x7fc308003530 con 0x7fc3240d38a0 2024-05-22T21:13:16.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc322a00640 1 -- 172.21.15.161:0/1200826381 >> v2:172.21.15.179:3300/0 conn(0x7fc3240d1e60 msgr2=0x7fc3240d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:16.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc322a00640 1 --2- 172.21.15.161:0/1200826381 >> v2:172.21.15.179:3300/0 conn(0x7fc3240d1e60 0x7fc3240d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc322a00640 1 -- 172.21.15.161:0/1200826381 >> v1:172.21.15.179:6789/0 conn(0x7fc32415d500 legacy=0x7fc3240d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc322a00640 1 -- 172.21.15.161:0/1200826381 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3240d14c0 con 0x7fc3240d38a0 2024-05-22T21:13:16.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc322a00640 1 -- 172.21.15.161:0/1200826381 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fc31c001a20 con 0x7fc3240d38a0 2024-05-22T21:13:16.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc323400640 1 --2- 172.21.15.161:0/1200826381 >> v2:172.21.15.179:3300/0 conn(0x7fc3240d1e60 0x7fc3240d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:16.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc322a00640 1 -- 172.21.15.161:0/1200826381 >> v1:172.21.15.161:6789/0 conn(0x7fc3240d38a0 legacy=0x7fc3240c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc322a00640 1 --2- 172.21.15.161:0/1200826381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc308003980 0x7fc308003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc322a00640 1 --2- 172.21.15.161:0/1200826381 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc3080047d0 0x7fc308004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc322a00640 1 --2- 172.21.15.161:0/1200826381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3080052b0 0x7fc30800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc322a00640 1 -- 172.21.15.161:0/1200826381 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3240d11e0 con 0x7fc308003980 2024-05-22T21:13:16.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc322a00640 1 -- 172.21.15.161:0/1200826381 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30800c180 con 0x7fc3080052b0 2024-05-22T21:13:16.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc322a00640 1 -- 172.21.15.161:0/1200826381 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30800c340 con 0x7fc3080047d0 2024-05-22T21:13:16.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc323e00640 1 --2- 172.21.15.161:0/1200826381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc308003980 0x7fc308003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc328c00640 1 --2- 172.21.15.161:0/1200826381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3080052b0 0x7fc30800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc323400640 1 --2- 172.21.15.161:0/1200826381 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc3080047d0 0x7fc308004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc323e00640 1 -- 172.21.15.161:0/1200826381 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc3080047d0 msgr2=0x7fc308004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:16.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc323e00640 1 --2- 172.21.15.161:0/1200826381 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc3080047d0 0x7fc308004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc323e00640 1 -- 172.21.15.161:0/1200826381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3080052b0 msgr2=0x7fc30800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:16.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc323e00640 1 --2- 172.21.15.161:0/1200826381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3080052b0 0x7fc30800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc323e00640 1 -- 172.21.15.161:0/1200826381 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc308003530 con 0x7fc308003980 2024-05-22T21:13:16.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.959+0000 7fc328c00640 1 -- 172.21.15.161:0/1200826381 reap_dead start 2024-05-22T21:13:16.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.963+0000 7fc323e00640 1 --2- 172.21.15.161:0/1200826381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc308003980 0x7fc308003da0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc314001750 tx=0x7fc31401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d86af3594cf1c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:16.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.963+0000 7fc322a00640 1 -- 172.21.15.161:0/1200826381 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3140052d0 con 0x7fc308003980 2024-05-22T21:13:16.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.963+0000 7fc322a00640 1 -- 172.21.15.161:0/1200826381 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3140a7050 con 0x7fc308003980 2024-05-22T21:13:16.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.963+0000 7fc322a00640 1 -- 172.21.15.161:0/1200826381 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314003600 con 0x7fc308003980 2024-05-22T21:13:16.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.963+0000 7fc32ac00640 1 -- 172.21.15.161:0/1200826381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc308003980 msgr2=0x7fc308003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.963+0000 7fc32ac00640 1 --2- 172.21.15.161:0/1200826381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc308003980 0x7fc308003da0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc314001750 tx=0x7fc31401c390 comp rx=0 tx=0).stop 2024-05-22T21:13:16.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.963+0000 7fc32ac00640 1 -- 172.21.15.161:0/1200826381 shutdown_connections 2024-05-22T21:13:16.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.963+0000 7fc32ac00640 1 --2- 172.21.15.161:0/1200826381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc308003980 0x7fc308003da0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.963+0000 7fc32ac00640 1 -- 172.21.15.161:0/1200826381 >> 172.21.15.161:0/1200826381 conn(0x7fc32409c470 msgr2=0x7fc3240d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:16.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.963+0000 7fc32ac00640 1 -- 172.21.15.161:0/1200826381 shutdown_connections 2024-05-22T21:13:16.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.963+0000 7fc32ac00640 1 -- 172.21.15.161:0/1200826381 wait complete. 2024-05-22T21:13:16.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc32ac00640 1 Processor -- start 2024-05-22T21:13:16.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc32ac00640 1 -- start start 2024-05-22T21:13:16.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc32ac00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc308003980 0x7fc3241f7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc32ac00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc324160510 0x7fc3241fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc32ac00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc3240b7ca0 0x7fc324158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc32ac00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3241f7940 con 0x7fc308003980 2024-05-22T21:13:16.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc32ac00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32415f820 con 0x7fc324160510 2024-05-22T21:13:16.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc32ac00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32415fb00 con 0x7fc3240b7ca0 2024-05-22T21:13:16.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc323e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc308003980 0x7fc3241f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc323e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc308003980 0x7fc3241f7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41216/0 (socket says 172.21.15.161:41216) 2024-05-22T21:13:16.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc323e00640 1 -- 172.21.15.161:0/4160376370 learned_addr learned my addr 172.21.15.161:0/4160376370 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:16.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc328c00640 1 --2- 172.21.15.161:0/4160376370 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc3240b7ca0 0x7fc324158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc323400640 1 --2- 172.21.15.161:0/4160376370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc324160510 0x7fc3241fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc323e00640 1 -- 172.21.15.161:0/4160376370 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc3240b7ca0 msgr2=0x7fc324158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc323e00640 1 --2- 172.21.15.161:0/4160376370 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc3240b7ca0 0x7fc324158fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc323e00640 1 -- 172.21.15.161:0/4160376370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc324160510 msgr2=0x7fc3241fa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:16.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc323e00640 1 --2- 172.21.15.161:0/4160376370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc324160510 0x7fc3241fa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:16.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc323e00640 1 -- 172.21.15.161:0/4160376370 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32415fde0 con 0x7fc308003980 2024-05-22T21:13:16.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc323400640 1 --2- 172.21.15.161:0/4160376370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc324160510 0x7fc3241fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:16.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc323e00640 1 --2- 172.21.15.161:0/4160376370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc308003980 0x7fc3241f7400 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc314005040 tx=0x7fc308005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:16.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc31be00640 1 -- 172.21.15.161:0/4160376370 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314007180 con 0x7fc308003980 2024-05-22T21:13:16.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc31be00640 1 -- 172.21.15.161:0/4160376370 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3140072e0 con 0x7fc308003980 2024-05-22T21:13:16.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc32ac00640 1 -- 172.21.15.161:0/4160376370 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3241f9fd0 con 0x7fc308003980 2024-05-22T21:13:16.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc31be00640 1 -- 172.21.15.161:0/4160376370 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314007650 con 0x7fc308003980 2024-05-22T21:13:16.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc32ac00640 1 -- 172.21.15.161:0/4160376370 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3241facf0 con 0x7fc308003980 2024-05-22T21:13:16.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.967+0000 7fc32ac00640 1 -- 172.21.15.161:0/4160376370 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc324067220 con 0x7fc308003980 2024-05-22T21:13:16.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.971+0000 7fc31be00640 1 -- 172.21.15.161:0/4160376370 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc3140067e0 con 0x7fc308003980 2024-05-22T21:13:16.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.971+0000 7fc31be00640 1 --2- 172.21.15.161:0/4160376370 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc304066690 0x7fc304068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:16.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.971+0000 7fc31be00640 1 -- 172.21.15.161:0/4160376370 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc31411e1c0 con 0x7fc308003980 2024-05-22T21:13:16.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.971+0000 7fc323400640 1 --2- 172.21.15.161:0/4160376370 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc304066690 0x7fc304068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:16.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.971+0000 7fc323400640 1 --2- 172.21.15.161:0/4160376370 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc304066690 0x7fc304068b50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc3240cff00 tx=0x7fc310008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:16.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:16.975+0000 7fc31be00640 1 -- 172.21.15.161:0/4160376370 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc3140ef270 con 0x7fc308003980 2024-05-22T21:13:17.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.131+0000 7fc32ac00640 1 -- 172.21.15.161:0/4160376370 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3240cefb0 con 0x7fc308003980 2024-05-22T21:13:17.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.131+0000 7fc31be00640 1 -- 172.21.15.161:0/4160376370 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc3140eee00 con 0x7fc308003980 2024-05-22T21:13:17.135 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:13:17.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.135+0000 7fc32ac00640 1 -- 172.21.15.161:0/4160376370 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc304066690 msgr2=0x7fc304068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:17.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.135+0000 7fc32ac00640 1 --2- 172.21.15.161:0/4160376370 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc304066690 0x7fc304068b50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc3240cff00 tx=0x7fc310008040 comp rx=0 tx=0).stop 2024-05-22T21:13:17.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.135+0000 7fc32ac00640 1 -- 172.21.15.161:0/4160376370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc308003980 msgr2=0x7fc3241f7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:17.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.135+0000 7fc32ac00640 1 --2- 172.21.15.161:0/4160376370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc308003980 0x7fc3241f7400 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc314005040 tx=0x7fc308005800 comp rx=0 tx=0).stop 2024-05-22T21:13:17.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.135+0000 7fc32ac00640 1 -- 172.21.15.161:0/4160376370 shutdown_connections 2024-05-22T21:13:17.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.135+0000 7fc32ac00640 1 --2- 172.21.15.161:0/4160376370 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc304066690 0x7fc304068b50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:17.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.135+0000 7fc32ac00640 1 --2- 172.21.15.161:0/4160376370 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc3240b7ca0 0x7fc324158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:17.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.135+0000 7fc32ac00640 1 --2- 172.21.15.161:0/4160376370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc324160510 0x7fc3241fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:17.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.135+0000 7fc32ac00640 1 --2- 172.21.15.161:0/4160376370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc308003980 0x7fc3241f7400 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:17.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.135+0000 7fc32ac00640 1 -- 172.21.15.161:0/4160376370 >> 172.21.15.161:0/4160376370 conn(0x7fc32409c470 msgr2=0x7fc3240c7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:17.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.135+0000 7fc32ac00640 1 -- 172.21.15.161:0/4160376370 shutdown_connections 2024-05-22T21:13:17.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.135+0000 7fc32ac00640 1 -- 172.21.15.161:0/4160376370 wait complete. 2024-05-22T21:13:17.155 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:13:17.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.287+0000 7f0041000640 1 Processor -- start 2024-05-22T21:13:17.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.287+0000 7f0041000640 1 -- start start 2024-05-22T21:13:17.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.287+0000 7f0041000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f003c15c6d0 0x7f003c164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:17.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.287+0000 7f0041000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003c071670 con 0x7f003c15e130 2024-05-22T21:13:17.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.287+0000 7f0041000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003c070a60 con 0x7f003c15dd50 2024-05-22T21:13:17.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.287+0000 7f0041000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f003c070d40 con 0x7f003c15c6d0 2024-05-22T21:13:17.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.287+0000 7f0031400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f003c15e130 0x7f003c15ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51166/0 (socket says 172.21.15.161:51166) 2024-05-22T21:13:17.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.287+0000 7f0031400640 1 -- 172.21.15.161:0/1076164406 learned_addr learned my addr 172.21.15.161:0/1076164406 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:17.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.291+0000 7f003a800640 1 --2- 172.21.15.161:0/1076164406 >> v2:172.21.15.179:3300/0 conn(0x7f003c15c6d0 0x7f003c164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:17.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.291+0000 7f0039400640 1 -- 172.21.15.161:0/1076164406 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3042498243 0 0) 0x7f003c071670 con 0x7f003c15e130 2024-05-22T21:13:17.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.291+0000 7f0039400640 1 -- 172.21.15.161:0/1076164406 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0020003530 con 0x7f003c15e130 2024-05-22T21:13:17.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.291+0000 7f0039400640 1 -- 172.21.15.161:0/1076164406 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1183138909 0 0) 0x7f003c070a60 con 0x7f003c15dd50 2024-05-22T21:13:17.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.291+0000 7f0039400640 1 -- 172.21.15.161:0/1076164406 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f003c071670 con 0x7f003c15dd50 2024-05-22T21:13:17.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.291+0000 7f0039400640 1 -- 172.21.15.161:0/1076164406 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 939124426 0 0) 0x7f0020003530 con 0x7f003c15e130 2024-05-22T21:13:17.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.291+0000 7f003a800640 1 -- 172.21.15.161:0/1076164406 >> v1:172.21.15.179:6789/0 conn(0x7f003c15dd50 legacy=0x7f003c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:17.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.291+0000 7f003a800640 1 -- 172.21.15.161:0/1076164406 >> v1:172.21.15.161:6789/0 conn(0x7f003c15e130 legacy=0x7f003c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:17.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.291+0000 7f003a800640 1 -- 172.21.15.161:0/1076164406 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c071020 con 0x7f003c15c6d0 2024-05-22T21:13:17.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.291+0000 7f003a800640 1 --2- 172.21.15.161:0/1076164406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c15c6d0 0x7f003c164e80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f00340097c0 tx=0x7f003401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f3acd573e149a7c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:17.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.291+0000 7f0039400640 1 -- 172.21.15.161:0/1076164406 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024001680 con 0x7f003c15c6d0 2024-05-22T21:13:17.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.291+0000 7f0039400640 1 -- 172.21.15.161:0/1076164406 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00340a7050 con 0x7f003c15c6d0 2024-05-22T21:13:17.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.291+0000 7f0039400640 1 -- 172.21.15.161:0/1076164406 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00340ab620 con 0x7f003c15c6d0 2024-05-22T21:13:17.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0041000640 1 -- 172.21.15.161:0/1076164406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c15c6d0 msgr2=0x7f003c164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:17.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0041000640 1 --2- 172.21.15.161:0/1076164406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c15c6d0 0x7f003c164e80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f00340097c0 tx=0x7f003401c390 comp rx=0 tx=0).stop 2024-05-22T21:13:17.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0041000640 1 -- 172.21.15.161:0/1076164406 shutdown_connections 2024-05-22T21:13:17.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0041000640 1 --2- 172.21.15.161:0/1076164406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c15c6d0 0x7f003c164e80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:17.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0041000640 1 -- 172.21.15.161:0/1076164406 >> 172.21.15.161:0/1076164406 conn(0x7f003c0b0bc0 msgr2=0x7f003c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:17.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0041000640 1 -- 172.21.15.161:0/1076164406 shutdown_connections 2024-05-22T21:13:17.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0041000640 1 -- 172.21.15.161:0/1076164406 wait complete. 2024-05-22T21:13:17.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0041000640 1 Processor -- start 2024-05-22T21:13:17.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0041000640 1 -- start start 2024-05-22T21:13:17.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0041000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003c15c6d0 0x7f003c135460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:17.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0041000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003c15dd50 0x7f003c138710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:17.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0041000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c15e130 0x7f003c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:17.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0041000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c153290 con 0x7f003c15dd50 2024-05-22T21:13:17.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0041000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c1359a0 con 0x7f003c15e130 2024-05-22T21:13:17.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0041000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003c138c50 con 0x7f003c15c6d0 2024-05-22T21:13:17.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0031400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003c15dd50 0x7f003c138710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:17.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0031400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003c15dd50 0x7f003c138710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41228/0 (socket says 172.21.15.161:41228) 2024-05-22T21:13:17.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0031400640 1 -- 172.21.15.161:0/326378948 learned_addr learned my addr 172.21.15.161:0/326378948 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:17.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0031400640 1 -- 172.21.15.161:0/326378948 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003c15c6d0 msgr2=0x7f003c135460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:17.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0031400640 1 --2- 172.21.15.161:0/326378948 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003c15c6d0 0x7f003c135460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:17.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f003a800640 1 --2- 172.21.15.161:0/326378948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c15e130 0x7f003c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:17.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0031400640 1 -- 172.21.15.161:0/326378948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c15e130 msgr2=0x7f003c06d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:17.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.295+0000 7f0031400640 1 --2- 172.21.15.161:0/326378948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c15e130 0x7f003c06d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:17.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.299+0000 7f0031400640 1 -- 172.21.15.161:0/326378948 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0034009420 con 0x7f003c15dd50 2024-05-22T21:13:17.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.299+0000 7f003a800640 1 --2- 172.21.15.161:0/326378948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c15e130 0x7f003c06d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:17.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.299+0000 7f0031400640 1 --2- 172.21.15.161:0/326378948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003c15dd50 0x7f003c138710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f002400c730 tx=0x7f002400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:17.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.299+0000 7f0032a00640 1 -- 172.21.15.161:0/326378948 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024005ae0 con 0x7f003c15dd50 2024-05-22T21:13:17.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.299+0000 7f0032a00640 1 -- 172.21.15.161:0/326378948 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0024005c60 con 0x7f003c15dd50 2024-05-22T21:13:17.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.299+0000 7f0032a00640 1 -- 172.21.15.161:0/326378948 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024007070 con 0x7f003c15dd50 2024-05-22T21:13:17.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.299+0000 7f0041000640 1 -- 172.21.15.161:0/326378948 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003c152610 con 0x7f003c15dd50 2024-05-22T21:13:17.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.299+0000 7f0041000640 1 -- 172.21.15.161:0/326378948 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003c152bd0 con 0x7f003c15dd50 2024-05-22T21:13:17.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.299+0000 7f0041000640 1 -- 172.21.15.161:0/326378948 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003c067220 con 0x7f003c15dd50 2024-05-22T21:13:17.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.299+0000 7f0032a00640 1 -- 172.21.15.161:0/326378948 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f00240071d0 con 0x7f003c15dd50 2024-05-22T21:13:17.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.299+0000 7f0032a00640 1 --2- 172.21.15.161:0/326378948 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0018066690 0x7f0018068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:17.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.299+0000 7f0032a00640 1 -- 172.21.15.161:0/326378948 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0024083730 con 0x7f003c15dd50 2024-05-22T21:13:17.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.299+0000 7f0039e00640 1 --2- 172.21.15.161:0/326378948 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0018066690 0x7f0018068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:17.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.303+0000 7f0039e00640 1 --2- 172.21.15.161:0/326378948 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0018066690 0x7f0018068b50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f002c007fd0 tx=0x7f002c007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:17.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.303+0000 7f0032a00640 1 -- 172.21.15.161:0/326378948 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0024050d50 con 0x7f003c15dd50 2024-05-22T21:13:17.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.491+0000 7f0041000640 1 -- 172.21.15.161:0/326378948 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f003c06e000 con 0x7f003c15dd50 2024-05-22T21:13:17.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.491+0000 7f0032a00640 1 -- 172.21.15.161:0/326378948 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1259 (secure 0 0 0) 0x7f0024017090 con 0x7f003c15dd50 2024-05-22T21:13:17.495 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:17.495 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2196055309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2196055309},{"type":"v1","addr":"172.21.15.161:6835","nonce":2196055309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-22T21:13:17.495 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 2 2024-05-22T21:13:17.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.495+0000 7f0041000640 1 -- 172.21.15.161:0/326378948 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0018066690 msgr2=0x7f0018068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:17.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.495+0000 7f0041000640 1 --2- 172.21.15.161:0/326378948 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0018066690 0x7f0018068b50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f002c007fd0 tx=0x7f002c007ce0 comp rx=0 tx=0).stop 2024-05-22T21:13:17.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.495+0000 7f0041000640 1 -- 172.21.15.161:0/326378948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003c15dd50 msgr2=0x7f003c138710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:17.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.495+0000 7f0041000640 1 --2- 172.21.15.161:0/326378948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003c15dd50 0x7f003c138710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f002400c730 tx=0x7f002400cbf0 comp rx=0 tx=0).stop 2024-05-22T21:13:17.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.495+0000 7f0041000640 1 -- 172.21.15.161:0/326378948 shutdown_connections 2024-05-22T21:13:17.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.495+0000 7f0041000640 1 --2- 172.21.15.161:0/326378948 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0018066690 0x7f0018068b50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:17.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.495+0000 7f0041000640 1 --2- 172.21.15.161:0/326378948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c15e130 0x7f003c06d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:17.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.495+0000 7f0041000640 1 --2- 172.21.15.161:0/326378948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003c15dd50 0x7f003c138710 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:17.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.495+0000 7f0041000640 1 --2- 172.21.15.161:0/326378948 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003c15c6d0 0x7f003c135460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:17.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.495+0000 7f0041000640 1 -- 172.21.15.161:0/326378948 >> 172.21.15.161:0/326378948 conn(0x7f003c0b0bc0 msgr2=0x7f003c0ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:17.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.495+0000 7f0041000640 1 -- 172.21.15.161:0/326378948 shutdown_connections 2024-05-22T21:13:17.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:17.495+0000 7f0041000640 1 -- 172.21.15.161:0/326378948 wait complete. 2024-05-22T21:13:19.515 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:13:19.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.659+0000 7f9cc9400640 1 Processor -- start 2024-05-22T21:13:19.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.659+0000 7f9cc9400640 1 -- start start 2024-05-22T21:13:19.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.659+0000 7f9cc9400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9cc415dd50 0x7f9cc406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:19.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.659+0000 7f9cc9400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cc415e130 0x7f9cc415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:19.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.659+0000 7f9cc9400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9cc415d410 0x7f9cc415a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:19.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.659+0000 7f9cc9400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc4071670 con 0x7f9cc415dd50 2024-05-22T21:13:19.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.659+0000 7f9cc9400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc415d8a0 con 0x7f9cc415d410 2024-05-22T21:13:19.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.659+0000 7f9cc9400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc406de40 con 0x7f9cc415e130 2024-05-22T21:13:19.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.659+0000 7f9cc2800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9cc415dd50 0x7f9cc406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:19.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.659+0000 7f9cc2800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9cc415dd50 0x7f9cc406d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41242/0 (socket says 172.21.15.161:41242) 2024-05-22T21:13:19.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.659+0000 7f9cc2800640 1 -- 172.21.15.161:0/4177003818 learned_addr learned my addr 172.21.15.161:0/4177003818 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:19.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.659+0000 7f9cc3200640 1 --2- 172.21.15.161:0/4177003818 >> v2:172.21.15.179:3300/0 conn(0x7f9cc415d410 0x7f9cc415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:19.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.659+0000 7f9cb9e00640 1 --2- 172.21.15.161:0/4177003818 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cc415e130 0x7f9cc415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:19.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.663+0000 7f9cc2800640 1 -- 172.21.15.161:0/4177003818 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cc415e130 msgr2=0x7f9cc415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:19.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.663+0000 7f9cc2800640 1 --2- 172.21.15.161:0/4177003818 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cc415e130 0x7f9cc415ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:19.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.663+0000 7f9cc2800640 1 -- 172.21.15.161:0/4177003818 >> v2:172.21.15.179:3300/0 conn(0x7f9cc415d410 msgr2=0x7f9cc415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:19.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.663+0000 7f9cc2800640 1 --2- 172.21.15.161:0/4177003818 >> v2:172.21.15.179:3300/0 conn(0x7f9cc415d410 0x7f9cc415a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:19.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.663+0000 7f9cc2800640 1 -- 172.21.15.161:0/4177003818 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc4070a60 con 0x7f9cc415dd50 2024-05-22T21:13:19.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.663+0000 7f9cb9e00640 1 --2- 172.21.15.161:0/4177003818 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cc415e130 0x7f9cc415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:19.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.663+0000 7f9cc2800640 1 --2- 172.21.15.161:0/4177003818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cc415dd50 0x7f9cc406d900 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40721d0 tx=0x7f9cb401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b5e3a4fa12ed45 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:19.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.663+0000 7f9cc1e00640 1 -- 172.21.15.161:0/4177003818 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb40a57b0 con 0x7f9cc415dd50 2024-05-22T21:13:19.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.663+0000 7f9cc1e00640 1 -- 172.21.15.161:0/4177003818 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb40a5910 con 0x7f9cc415dd50 2024-05-22T21:13:19.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.663+0000 7f9cc1e00640 1 -- 172.21.15.161:0/4177003818 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb40a5ca0 con 0x7f9cc415dd50 2024-05-22T21:13:19.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.663+0000 7f9cc9400640 1 -- 172.21.15.161:0/4177003818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cc415dd50 msgr2=0x7f9cc406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:19.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.663+0000 7f9cc9400640 1 --2- 172.21.15.161:0/4177003818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cc415dd50 0x7f9cc406d900 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40721d0 tx=0x7f9cb401c930 comp rx=0 tx=0).stop 2024-05-22T21:13:19.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc9400640 1 -- 172.21.15.161:0/4177003818 shutdown_connections 2024-05-22T21:13:19.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc9400640 1 --2- 172.21.15.161:0/4177003818 >> v2:172.21.15.179:3300/0 conn(0x7f9cc415d410 0x7f9cc415a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:19.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc9400640 1 --2- 172.21.15.161:0/4177003818 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cc415e130 0x7f9cc415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:19.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc9400640 1 --2- 172.21.15.161:0/4177003818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cc415dd50 0x7f9cc406d900 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:19.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc9400640 1 -- 172.21.15.161:0/4177003818 >> 172.21.15.161:0/4177003818 conn(0x7f9cc40b0bc0 msgr2=0x7f9cc40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:19.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc9400640 1 -- 172.21.15.161:0/4177003818 shutdown_connections 2024-05-22T21:13:19.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc9400640 1 -- 172.21.15.161:0/4177003818 wait complete. 2024-05-22T21:13:19.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc9400640 1 Processor -- start 2024-05-22T21:13:19.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc9400640 1 -- start start 2024-05-22T21:13:19.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc9400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cc415d410 0x7f9cc414dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:19.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc9400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cc415dd50 0x7f9cc40b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:19.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cc415e130 0x7f9cc4157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:19.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc9400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc40b14e0 con 0x7f9cc415e130 2024-05-22T21:13:19.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc9400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc4155bb0 con 0x7f9cc415dd50 2024-05-22T21:13:19.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc9400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc41f5e50 con 0x7f9cc415d410 2024-05-22T21:13:19.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc3200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cc415e130 0x7f9cc4157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:19.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc3200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cc415e130 0x7f9cc4157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41258/0 (socket says 172.21.15.161:41258) 2024-05-22T21:13:19.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc3200640 1 -- 172.21.15.161:0/3098728633 learned_addr learned my addr 172.21.15.161:0/3098728633 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:19.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cb9e00640 1 --2- 172.21.15.161:0/3098728633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cc415dd50 0x7f9cc40b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:19.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc2800640 1 --2- 172.21.15.161:0/3098728633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cc415d410 0x7f9cc414dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:19.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc3200640 1 -- 172.21.15.161:0/3098728633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cc415d410 msgr2=0x7f9cc414dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:19.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc3200640 1 --2- 172.21.15.161:0/3098728633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cc415d410 0x7f9cc414dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:19.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc3200640 1 -- 172.21.15.161:0/3098728633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cc415dd50 msgr2=0x7f9cc40b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:19.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc3200640 1 --2- 172.21.15.161:0/3098728633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cc415dd50 0x7f9cc40b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:19.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc3200640 1 -- 172.21.15.161:0/3098728633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb4009b30 con 0x7f9cc415e130 2024-05-22T21:13:19.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc2800640 1 --2- 172.21.15.161:0/3098728633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cc415d410 0x7f9cc414dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:19.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cc3200640 1 --2- 172.21.15.161:0/3098728633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cc415e130 0x7f9cc4157500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00e930 tx=0x7f9cbc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:19.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.667+0000 7f9cbb400640 1 -- 172.21.15.161:0/3098728633 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc0096b0 con 0x7f9cc415e130 2024-05-22T21:13:19.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.671+0000 7f9cbb400640 1 -- 172.21.15.161:0/3098728633 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cbc00c9a0 con 0x7f9cc415e130 2024-05-22T21:13:19.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.671+0000 7f9cbb400640 1 -- 172.21.15.161:0/3098728633 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc00cd30 con 0x7f9cc415e130 2024-05-22T21:13:19.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.671+0000 7f9cc9400640 1 -- 172.21.15.161:0/3098728633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc4159770 con 0x7f9cc415e130 2024-05-22T21:13:19.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.671+0000 7f9cc9400640 1 -- 172.21.15.161:0/3098728633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc40b3d70 con 0x7f9cc415e130 2024-05-22T21:13:19.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.671+0000 7f9cc9400640 1 -- 172.21.15.161:0/3098728633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc4155950 con 0x7f9cc415e130 2024-05-22T21:13:19.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.671+0000 7f9cbb400640 1 -- 172.21.15.161:0/3098728633 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9cbc013470 con 0x7f9cc415e130 2024-05-22T21:13:19.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.671+0000 7f9cbb400640 1 --2- 172.21.15.161:0/3098728633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ca0066690 0x7f9ca0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:19.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.671+0000 7f9cbb400640 1 -- 172.21.15.161:0/3098728633 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9cbc0840d0 con 0x7f9cc415e130 2024-05-22T21:13:19.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.671+0000 7f9cc2800640 1 --2- 172.21.15.161:0/3098728633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ca0066690 0x7f9ca0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:19.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.675+0000 7f9cc2800640 1 --2- 172.21.15.161:0/3098728633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ca0066690 0x7f9ca0068b50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40ae000 tx=0x7f9cb40a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:19.678 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.675+0000 7f9cbb400640 1 -- 172.21.15.161:0/3098728633 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9cbc0517b0 con 0x7f9cc415e130 2024-05-22T21:13:19.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.867+0000 7f9cc9400640 1 -- 172.21.15.161:0/3098728633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cc40b3f90 con 0x7f9cc415e130 2024-05-22T21:13:19.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.867+0000 7f9cbb400640 1 -- 172.21.15.161:0/3098728633 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3536 (secure 0 0 0) 0x7f9cbc017de0 con 0x7f9cc415e130 2024-05-22T21:13:19.869 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:19.869 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3179612470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3179612470},{"type":"v1","addr":"172.21.15.179:6834","nonce":3179612470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2196055309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2196055309},{"type":"v1","addr":"172.21.15.161:6835","nonce":2196055309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1793681759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1793681759},{"type":"v1","addr":"172.21.15.161:6837","nonce":1793681759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3095599688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3095599688},{"type":"v1","addr":"172.21.15.179:6835","nonce":3095599688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-05-22T21:13:19.869 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 3 2024-05-22T21:13:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.867+0000 7f9cc9400640 1 -- 172.21.15.161:0/3098728633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ca0066690 msgr2=0x7f9ca0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.867+0000 7f9cc9400640 1 --2- 172.21.15.161:0/3098728633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ca0066690 0x7f9ca0068b50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40ae000 tx=0x7f9cb40a7070 comp rx=0 tx=0).stop 2024-05-22T21:13:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.867+0000 7f9cc9400640 1 -- 172.21.15.161:0/3098728633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cc415e130 msgr2=0x7f9cc4157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.867+0000 7f9cc9400640 1 --2- 172.21.15.161:0/3098728633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cc415e130 0x7f9cc4157500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00e930 tx=0x7f9cbc00edf0 comp rx=0 tx=0).stop 2024-05-22T21:13:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.867+0000 7f9cc9400640 1 -- 172.21.15.161:0/3098728633 shutdown_connections 2024-05-22T21:13:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.867+0000 7f9cc9400640 1 --2- 172.21.15.161:0/3098728633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ca0066690 0x7f9ca0068b50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.867+0000 7f9cc9400640 1 --2- 172.21.15.161:0/3098728633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cc415e130 0x7f9cc4157500 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:19.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.867+0000 7f9cc9400640 1 --2- 172.21.15.161:0/3098728633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cc415dd50 0x7f9cc40b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:19.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.867+0000 7f9cc9400640 1 --2- 172.21.15.161:0/3098728633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cc415d410 0x7f9cc414dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:19.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.867+0000 7f9cc9400640 1 -- 172.21.15.161:0/3098728633 >> 172.21.15.161:0/3098728633 conn(0x7f9cc40b0bc0 msgr2=0x7f9cc40aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:19.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.867+0000 7f9cc9400640 1 -- 172.21.15.161:0/3098728633 shutdown_connections 2024-05-22T21:13:19.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:19.867+0000 7f9cc9400640 1 -- 172.21.15.161:0/3098728633 wait complete. 2024-05-22T21:13:19.890 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:20.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.023+0000 7f2b98a00640 1 Processor -- start 2024-05-22T21:13:20.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.023+0000 7f2b98a00640 1 -- start start 2024-05-22T21:13:20.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.023+0000 7f2b98a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2b9415e130 0x7f2b9415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.023+0000 7f2b98a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b94071670 con 0x7f2b9415dd50 2024-05-22T21:13:20.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.023+0000 7f2b98a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9415d870 con 0x7f2b9415d410 2024-05-22T21:13:20.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.023+0000 7f2b98a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b94070a60 con 0x7f2b9415e130 2024-05-22T21:13:20.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.023+0000 7f2b91e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f2b9415dd50 0x7f2b9406d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51170/0 (socket says 172.21.15.161:51170) 2024-05-22T21:13:20.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.023+0000 7f2b91e00640 1 -- 172.21.15.161:0/2008254203 learned_addr learned my addr 172.21.15.161:0/2008254203 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:20.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.023+0000 7f2b91400640 1 --2- 172.21.15.161:0/2008254203 >> v2:172.21.15.179:3300/0 conn(0x7f2b9415e130 0x7f2b9415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.023+0000 7f2b90a00640 1 -- 172.21.15.161:0/2008254203 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597991167 0 0) 0x7f2b94071670 con 0x7f2b9415dd50 2024-05-22T21:13:20.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.023+0000 7f2b90a00640 1 -- 172.21.15.161:0/2008254203 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b78003530 con 0x7f2b9415dd50 2024-05-22T21:13:20.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b90a00640 1 -- 172.21.15.161:0/2008254203 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 394901205 0 0) 0x7f2b78003530 con 0x7f2b9415dd50 2024-05-22T21:13:20.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b90a00640 1 -- 172.21.15.161:0/2008254203 >> v2:172.21.15.179:3300/0 conn(0x7f2b9415e130 msgr2=0x7f2b9415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:20.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b90a00640 1 --2- 172.21.15.161:0/2008254203 >> v2:172.21.15.179:3300/0 conn(0x7f2b9415e130 0x7f2b9415ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b90a00640 1 -- 172.21.15.161:0/2008254203 >> v1:172.21.15.179:6789/0 conn(0x7f2b9415d410 legacy=0x7f2b9415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b90a00640 1 -- 172.21.15.161:0/2008254203 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b94070d40 con 0x7f2b9415dd50 2024-05-22T21:13:20.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b90a00640 1 -- 172.21.15.161:0/2008254203 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f2b84001680 con 0x7f2b9415dd50 2024-05-22T21:13:20.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b90a00640 1 -- 172.21.15.161:0/2008254203 >> v1:172.21.15.161:6789/0 conn(0x7f2b9415dd50 legacy=0x7f2b9406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b90a00640 1 --2- 172.21.15.161:0/2008254203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b78003980 0x7f2b78003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b90a00640 1 --2- 172.21.15.161:0/2008254203 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b780047d0 0x7f2b78004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b90a00640 1 --2- 172.21.15.161:0/2008254203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b78005190 0x7f2b7800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b90a00640 1 -- 172.21.15.161:0/2008254203 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b94070a60 con 0x7f2b78005190 2024-05-22T21:13:20.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b90a00640 1 -- 172.21.15.161:0/2008254203 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7800e240 con 0x7f2b78003980 2024-05-22T21:13:20.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b90a00640 1 -- 172.21.15.161:0/2008254203 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7800e400 con 0x7f2b780047d0 2024-05-22T21:13:20.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b91400640 1 --2- 172.21.15.161:0/2008254203 >> v2:172.21.15.179:3300/0 conn(0x7f2b9415e130 0x7f2b9415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:20.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b92800640 1 --2- 172.21.15.161:0/2008254203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b78005190 0x7f2b7800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b91e00640 1 --2- 172.21.15.161:0/2008254203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b78003980 0x7f2b78003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b91400640 1 --2- 172.21.15.161:0/2008254203 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b780047d0 0x7f2b78004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b92800640 1 -- 172.21.15.161:0/2008254203 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b780047d0 msgr2=0x7f2b78004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:20.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b92800640 1 --2- 172.21.15.161:0/2008254203 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b780047d0 0x7f2b78004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b92800640 1 -- 172.21.15.161:0/2008254203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b78003980 msgr2=0x7f2b78003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:20.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b92800640 1 --2- 172.21.15.161:0/2008254203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b78003980 0x7f2b78003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b92800640 1 -- 172.21.15.161:0/2008254203 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b78003530 con 0x7f2b78005190 2024-05-22T21:13:20.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b91400640 1 --2- 172.21.15.161:0/2008254203 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b780047d0 0x7f2b78004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:20.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b92800640 1 -- 172.21.15.161:0/2008254203 reap_dead start 2024-05-22T21:13:20.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b92800640 1 --2- 172.21.15.161:0/2008254203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b78005190 0x7f2b7800dba0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c001e70 tx=0x7f2b8c01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f294fe225da956 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:20.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b90a00640 1 -- 172.21.15.161:0/2008254203 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c003b80 con 0x7f2b78005190 2024-05-22T21:13:20.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b90a00640 1 -- 172.21.15.161:0/2008254203 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8c003d20 con 0x7f2b78005190 2024-05-22T21:13:20.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.027+0000 7f2b90a00640 1 -- 172.21.15.161:0/2008254203 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c005310 con 0x7f2b78005190 2024-05-22T21:13:20.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b98a00640 1 -- 172.21.15.161:0/2008254203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b78005190 msgr2=0x7f2b7800dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b98a00640 1 --2- 172.21.15.161:0/2008254203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b78005190 0x7f2b7800dba0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c001e70 tx=0x7f2b8c01c810 comp rx=0 tx=0).stop 2024-05-22T21:13:20.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b98a00640 1 -- 172.21.15.161:0/2008254203 shutdown_connections 2024-05-22T21:13:20.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b98a00640 1 --2- 172.21.15.161:0/2008254203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b78005190 0x7f2b7800dba0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b98a00640 1 -- 172.21.15.161:0/2008254203 >> 172.21.15.161:0/2008254203 conn(0x7f2b940b0bc0 msgr2=0x7f2b940b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:20.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b98a00640 1 -- 172.21.15.161:0/2008254203 shutdown_connections 2024-05-22T21:13:20.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b98a00640 1 -- 172.21.15.161:0/2008254203 wait complete. 2024-05-22T21:13:20.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b98a00640 1 Processor -- start 2024-05-22T21:13:20.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b98a00640 1 -- start start 2024-05-22T21:13:20.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b98a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b78005190 0x7f2b94151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b98a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b941f3a10 0x7f2b940cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b98a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b940cd6b0 0x7f2b941ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b98a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b940cdb30 con 0x7f2b940cd6b0 2024-05-22T21:13:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b98a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b94152100 con 0x7f2b78005190 2024-05-22T21:13:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b98a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b941f04a0 con 0x7f2b941f3a10 2024-05-22T21:13:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b92800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b940cd6b0 0x7f2b941ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b92800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b940cd6b0 0x7f2b941ef8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41284/0 (socket says 172.21.15.161:41284) 2024-05-22T21:13:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b92800640 1 -- 172.21.15.161:0/3283223656 learned_addr learned my addr 172.21.15.161:0/3283223656 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b91400640 1 --2- 172.21.15.161:0/3283223656 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b941f3a10 0x7f2b940cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.031+0000 7f2b91e00640 1 --2- 172.21.15.161:0/3283223656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b78005190 0x7f2b94151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.035+0000 7f2b92800640 1 -- 172.21.15.161:0/3283223656 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b941f3a10 msgr2=0x7f2b940cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.035+0000 7f2b92800640 1 --2- 172.21.15.161:0/3283223656 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b941f3a10 0x7f2b940cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.035+0000 7f2b92800640 1 -- 172.21.15.161:0/3283223656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b78005190 msgr2=0x7f2b94151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.035+0000 7f2b92800640 1 --2- 172.21.15.161:0/3283223656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b78005190 0x7f2b94151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.035+0000 7f2b92800640 1 -- 172.21.15.161:0/3283223656 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b941f2f30 con 0x7f2b940cd6b0 2024-05-22T21:13:20.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.035+0000 7f2b92800640 1 --2- 172.21.15.161:0/3283223656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b940cd6b0 0x7f2b941ef8c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c01c640 tx=0x7f2b78005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:20.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.035+0000 7f2b8a000640 1 -- 172.21.15.161:0/3283223656 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c003b80 con 0x7f2b940cd6b0 2024-05-22T21:13:20.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.035+0000 7f2b8a000640 1 -- 172.21.15.161:0/3283223656 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8c003e90 con 0x7f2b940cd6b0 2024-05-22T21:13:20.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.035+0000 7f2b8a000640 1 -- 172.21.15.161:0/3283223656 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c006d80 con 0x7f2b940cd6b0 2024-05-22T21:13:20.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.035+0000 7f2b98a00640 1 -- 172.21.15.161:0/3283223656 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b941f3210 con 0x7f2b940cd6b0 2024-05-22T21:13:20.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.035+0000 7f2b98a00640 1 -- 172.21.15.161:0/3283223656 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b941575f0 con 0x7f2b940cd6b0 2024-05-22T21:13:20.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.035+0000 7f2b98a00640 1 -- 172.21.15.161:0/3283223656 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b94067220 con 0x7f2b940cd6b0 2024-05-22T21:13:20.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.035+0000 7f2b8a000640 1 -- 172.21.15.161:0/3283223656 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2b8c0a64f0 con 0x7f2b940cd6b0 2024-05-22T21:13:20.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.035+0000 7f2b8a000640 1 --2- 172.21.15.161:0/3283223656 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2b74066690 0x7f2b74068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.035+0000 7f2b8a000640 1 -- 172.21.15.161:0/3283223656 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2b8c0aa070 con 0x7f2b940cd6b0 2024-05-22T21:13:20.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.035+0000 7f2b91e00640 1 --2- 172.21.15.161:0/3283223656 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2b74066690 0x7f2b74068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.039+0000 7f2b91e00640 1 --2- 172.21.15.161:0/3283223656 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2b74066690 0x7f2b74068b50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2b84001060 tx=0x7f2b84001c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:20.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.039+0000 7f2b8a000640 1 -- 172.21.15.161:0/3283223656 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2b8c0eb040 con 0x7f2b940cd6b0 2024-05-22T21:13:20.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.195+0000 7f2b98a00640 1 -- 172.21.15.161:0/3283223656 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b94067a40 con 0x7f2b940cd6b0 2024-05-22T21:13:20.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.195+0000 7f2b8a000640 1 -- 172.21.15.161:0/3283223656 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12430 (secure 0 0 0) 0x7f2b8c0ee660 con 0x7f2b940cd6b0 2024-05-22T21:13:20.197 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:20.197 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":17,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:14.993684+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:20.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.195+0000 7f2b98a00640 1 -- 172.21.15.161:0/3283223656 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2b74066690 msgr2=0x7f2b74068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.195+0000 7f2b98a00640 1 --2- 172.21.15.161:0/3283223656 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2b74066690 0x7f2b74068b50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2b84001060 tx=0x7f2b84001c10 comp rx=0 tx=0).stop 2024-05-22T21:13:20.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.195+0000 7f2b98a00640 1 -- 172.21.15.161:0/3283223656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b940cd6b0 msgr2=0x7f2b941ef8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.195+0000 7f2b98a00640 1 --2- 172.21.15.161:0/3283223656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b940cd6b0 0x7f2b941ef8c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c01c640 tx=0x7f2b78005800 comp rx=0 tx=0).stop 2024-05-22T21:13:20.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.195+0000 7f2b98a00640 1 -- 172.21.15.161:0/3283223656 shutdown_connections 2024-05-22T21:13:20.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.195+0000 7f2b98a00640 1 --2- 172.21.15.161:0/3283223656 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2b74066690 0x7f2b74068b50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.195+0000 7f2b98a00640 1 --2- 172.21.15.161:0/3283223656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b940cd6b0 0x7f2b941ef8c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.195+0000 7f2b98a00640 1 --2- 172.21.15.161:0/3283223656 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b941f3a10 0x7f2b940cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.195+0000 7f2b98a00640 1 --2- 172.21.15.161:0/3283223656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b78005190 0x7f2b94151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.195+0000 7f2b98a00640 1 -- 172.21.15.161:0/3283223656 >> 172.21.15.161:0/3283223656 conn(0x7f2b940b0bc0 msgr2=0x7f2b9415a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:20.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.199+0000 7f2b98a00640 1 -- 172.21.15.161:0/3283223656 shutdown_connections 2024-05-22T21:13:20.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.199+0000 7f2b98a00640 1 -- 172.21.15.161:0/3283223656 wait complete. 2024-05-22T21:13:20.218 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:13:20.219 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:13:20.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.355+0000 7ffaddc00640 1 Processor -- start 2024-05-22T21:13:20.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.355+0000 7ffaddc00640 1 -- start start 2024-05-22T21:13:20.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.355+0000 7ffaddc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffad81402e0 0x7ffad813f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.355+0000 7ffaddc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ffad813fc80 0x7ffad806fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.355+0000 7ffaddc00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad81400d0 con 0x7ffad813cfc0 2024-05-22T21:13:20.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.355+0000 7ffaddc00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad813d3a0 con 0x7ffad813fc80 2024-05-22T21:13:20.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.355+0000 7ffaddc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80cd920 con 0x7ffad81402e0 2024-05-22T21:13:20.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.355+0000 7ffad7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ffad813fc80 0x7ffad806fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.355+0000 7ffad7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ffad813fc80 0x7ffad806fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41292/0 (socket says 172.21.15.161:41292) 2024-05-22T21:13:20.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.355+0000 7ffad7400640 1 -- 172.21.15.161:0/2774463785 learned_addr learned my addr 172.21.15.161:0/2774463785 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:20.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.355+0000 7ffad7400640 1 -- 172.21.15.161:0/2774463785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffad81402e0 msgr2=0x7ffad813f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:20.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.355+0000 7ffad7400640 1 --2- 172.21.15.161:0/2774463785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffad81402e0 0x7ffad813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.355+0000 7ffad7400640 1 -- 172.21.15.161:0/2774463785 >> v1:172.21.15.179:6789/0 conn(0x7ffad813cfc0 legacy=0x7ffad815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:20.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.355+0000 7ffad7400640 1 -- 172.21.15.161:0/2774463785 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad80cdc00 con 0x7ffad813fc80 2024-05-22T21:13:20.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.359+0000 7ffad7400640 1 --2- 172.21.15.161:0/2774463785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffad813fc80 0x7ffad806fce0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009870 tx=0x7ffad001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38f43977705456 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:20.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.359+0000 7ffad5600640 1 -- 172.21.15.161:0/2774463785 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad00ab600 con 0x7ffad813fc80 2024-05-22T21:13:20.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.359+0000 7ffad5600640 1 -- 172.21.15.161:0/2774463785 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad00a7050 con 0x7ffad813fc80 2024-05-22T21:13:20.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.359+0000 7ffad5600640 1 -- 172.21.15.161:0/2774463785 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad00ab980 con 0x7ffad813fc80 2024-05-22T21:13:20.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.359+0000 7ffaddc00640 1 -- 172.21.15.161:0/2774463785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffad813fc80 msgr2=0x7ffad806fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.359+0000 7ffaddc00640 1 --2- 172.21.15.161:0/2774463785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffad813fc80 0x7ffad806fce0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009870 tx=0x7ffad001c460 comp rx=0 tx=0).stop 2024-05-22T21:13:20.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.359+0000 7ffaddc00640 1 -- 172.21.15.161:0/2774463785 shutdown_connections 2024-05-22T21:13:20.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.359+0000 7ffaddc00640 1 --2- 172.21.15.161:0/2774463785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffad813fc80 0x7ffad806fce0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.359+0000 7ffaddc00640 1 --2- 172.21.15.161:0/2774463785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffad81402e0 0x7ffad813f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.359+0000 7ffaddc00640 1 -- 172.21.15.161:0/2774463785 >> 172.21.15.161:0/2774463785 conn(0x7ffad806a2d0 msgr2=0x7ffad806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:20.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.359+0000 7ffaddc00640 1 -- 172.21.15.161:0/2774463785 shutdown_connections 2024-05-22T21:13:20.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.359+0000 7ffaddc00640 1 -- 172.21.15.161:0/2774463785 wait complete. 2024-05-22T21:13:20.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffaddc00640 1 Processor -- start 2024-05-22T21:13:20.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffaddc00640 1 -- start start 2024-05-22T21:13:20.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffaddc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffad813cfc0 0x7ffad813be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffaddc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffad813fc80 0x7ffad80adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffaddc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffad81402e0 0x7ffad8141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffaddc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad8159f90 con 0x7ffad813fc80 2024-05-22T21:13:20.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffaddc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80ae2e0 con 0x7ffad81402e0 2024-05-22T21:13:20.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffaddc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad815cfe0 con 0x7ffad813cfc0 2024-05-22T21:13:20.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffad6000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffad813fc80 0x7ffad80adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffad6000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffad813fc80 0x7ffad80adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41298/0 (socket says 172.21.15.161:41298) 2024-05-22T21:13:20.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffad6000640 1 -- 172.21.15.161:0/1968298540 learned_addr learned my addr 172.21.15.161:0/1968298540 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:20.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffad7400640 1 --2- 172.21.15.161:0/1968298540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffad81402e0 0x7ffad8141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffad6000640 1 -- 172.21.15.161:0/1968298540 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffad813cfc0 msgr2=0x7ffad813be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffad6a00640 1 --2- 172.21.15.161:0/1968298540 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffad813cfc0 0x7ffad813be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffad6000640 1 --2- 172.21.15.161:0/1968298540 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffad813cfc0 0x7ffad813be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffad6000640 1 -- 172.21.15.161:0/1968298540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffad81402e0 msgr2=0x7ffad8141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffad6000640 1 --2- 172.21.15.161:0/1968298540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffad81402e0 0x7ffad8141370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffad6000640 1 -- 172.21.15.161:0/1968298540 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad00094d0 con 0x7ffad813fc80 2024-05-22T21:13:20.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffad7400640 1 --2- 172.21.15.161:0/1968298540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffad81402e0 0x7ffad8141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:20.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffad6a00640 1 --2- 172.21.15.161:0/1968298540 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffad813cfc0 0x7ffad813be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:20.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffad6000640 1 --2- 172.21.15.161:0/1968298540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffad813fc80 0x7ffad80adda0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffac400b410 tx=0x7ffac400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:20.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffacea00640 1 -- 172.21.15.161:0/1968298540 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac4011830 con 0x7ffad813fc80 2024-05-22T21:13:20.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffacea00640 1 -- 172.21.15.161:0/1968298540 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac4011990 con 0x7ffad813fc80 2024-05-22T21:13:20.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffaddc00640 1 -- 172.21.15.161:0/1968298540 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad815d2c0 con 0x7ffad813fc80 2024-05-22T21:13:20.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffacea00640 1 -- 172.21.15.161:0/1968298540 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac4011d40 con 0x7ffad813fc80 2024-05-22T21:13:20.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffaddc00640 1 -- 172.21.15.161:0/1968298540 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad80ad5a0 con 0x7ffad813fc80 2024-05-22T21:13:20.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.363+0000 7ffaddc00640 1 -- 172.21.15.161:0/1968298540 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad8067220 con 0x7ffad813fc80 2024-05-22T21:13:20.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.367+0000 7ffacea00640 1 -- 172.21.15.161:0/1968298540 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffac401a400 con 0x7ffad813fc80 2024-05-22T21:13:20.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.367+0000 7ffacea00640 1 --2- 172.21.15.161:0/1968298540 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffab8066690 0x7ffab8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.367+0000 7ffacea00640 1 -- 172.21.15.161:0/1968298540 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ffac40840a0 con 0x7ffad813fc80 2024-05-22T21:13:20.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.367+0000 7ffad6a00640 1 --2- 172.21.15.161:0/1968298540 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffab8066690 0x7ffab8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.367+0000 7ffad6a00640 1 --2- 172.21.15.161:0/1968298540 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffab8066690 0x7ffab8068b50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffac8007fd0 tx=0x7ffac8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:20.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.371+0000 7ffacea00640 1 -- 172.21.15.161:0/1968298540 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffac40516c0 con 0x7ffad813fc80 2024-05-22T21:13:20.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.527+0000 7ffaddc00640 1 -- 172.21.15.161:0/1968298540 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffad80cd040 con 0x7ffad813fc80 2024-05-22T21:13:20.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.531+0000 7ffacea00640 1 -- 172.21.15.161:0/1968298540 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffac40150a0 con 0x7ffad813fc80 2024-05-22T21:13:20.532 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:13:20.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.531+0000 7ffaddc00640 1 -- 172.21.15.161:0/1968298540 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffab8066690 msgr2=0x7ffab8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.531+0000 7ffaddc00640 1 --2- 172.21.15.161:0/1968298540 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffab8066690 0x7ffab8068b50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffac8007fd0 tx=0x7ffac8009040 comp rx=0 tx=0).stop 2024-05-22T21:13:20.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.531+0000 7ffaddc00640 1 -- 172.21.15.161:0/1968298540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffad813fc80 msgr2=0x7ffad80adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.531+0000 7ffaddc00640 1 --2- 172.21.15.161:0/1968298540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffad813fc80 0x7ffad80adda0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffac400b410 tx=0x7ffac400b8d0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.531+0000 7ffaddc00640 1 -- 172.21.15.161:0/1968298540 shutdown_connections 2024-05-22T21:13:20.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.531+0000 7ffaddc00640 1 --2- 172.21.15.161:0/1968298540 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffab8066690 0x7ffab8068b50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.531+0000 7ffaddc00640 1 --2- 172.21.15.161:0/1968298540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffad81402e0 0x7ffad8141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.531+0000 7ffaddc00640 1 --2- 172.21.15.161:0/1968298540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffad813fc80 0x7ffad80adda0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.531+0000 7ffaddc00640 1 --2- 172.21.15.161:0/1968298540 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffad813cfc0 0x7ffad813be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.531+0000 7ffaddc00640 1 -- 172.21.15.161:0/1968298540 >> 172.21.15.161:0/1968298540 conn(0x7ffad806a2d0 msgr2=0x7ffad80afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:20.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.531+0000 7ffaddc00640 1 -- 172.21.15.161:0/1968298540 shutdown_connections 2024-05-22T21:13:20.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.531+0000 7ffaddc00640 1 -- 172.21.15.161:0/1968298540 wait complete. 2024-05-22T21:13:20.552 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-22T21:13:20.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.683+0000 7f61cd600640 1 Processor -- start 2024-05-22T21:13:20.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.683+0000 7f61cd600640 1 -- start start 2024-05-22T21:13:20.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.687+0000 7f61cd600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61c815e130 0x7f61c815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.687+0000 7f61cd600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f61c815d410 0x7f61c815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.687+0000 7f61cd600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c8071670 con 0x7f61c815dd50 2024-05-22T21:13:20.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.687+0000 7f61cd600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c815d870 con 0x7f61c815d410 2024-05-22T21:13:20.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.687+0000 7f61cd600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c8070a60 con 0x7f61c815e130 2024-05-22T21:13:20.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.687+0000 7f61c7400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f61c815d410 0x7f61c815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.687+0000 7f61c6000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61c815e130 0x7f61c815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.687+0000 7f61c7400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f61c815d410 0x7f61c815b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:49910/0 (socket says 172.21.15.161:49910) 2024-05-22T21:13:20.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.687+0000 7f61c7400640 1 -- 172.21.15.161:0/746932682 learned_addr learned my addr 172.21.15.161:0/746932682 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:20.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.687+0000 7f61c7400640 1 -- 172.21.15.161:0/746932682 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61c815e130 msgr2=0x7f61c815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:20.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.687+0000 7f61c7400640 1 --2- 172.21.15.161:0/746932682 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61c815e130 0x7f61c815ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.687+0000 7f61c7400640 1 -- 172.21.15.161:0/746932682 >> v1:172.21.15.179:6789/0 conn(0x7f61c815dd50 legacy=0x7f61c806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.687+0000 7f61c7400640 1 -- 172.21.15.161:0/746932682 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c8070d40 con 0x7f61c815d410 2024-05-22T21:13:20.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.687+0000 7f61c6000640 1 --2- 172.21.15.161:0/746932682 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61c815e130 0x7f61c815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:20.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.691+0000 7f61c7400640 1 --2- 172.21.15.161:0/746932682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61c815d410 0x7f61c815b6c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f61c0009e50 tx=0x7f61c001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8d3e142198751fa server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:20.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.691+0000 7f61c5600640 1 -- 172.21.15.161:0/746932682 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00a5830 con 0x7f61c815d410 2024-05-22T21:13:20.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.691+0000 7f61c5600640 1 -- 172.21.15.161:0/746932682 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c00a5990 con 0x7f61c815d410 2024-05-22T21:13:20.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.691+0000 7f61c5600640 1 -- 172.21.15.161:0/746932682 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00a5d20 con 0x7f61c815d410 2024-05-22T21:13:20.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.691+0000 7f61cd600640 1 -- 172.21.15.161:0/746932682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61c815d410 msgr2=0x7f61c815b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.691+0000 7f61cd600640 1 --2- 172.21.15.161:0/746932682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61c815d410 0x7f61c815b6c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f61c0009e50 tx=0x7f61c001c970 comp rx=0 tx=0).stop 2024-05-22T21:13:20.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.691+0000 7f61cd600640 1 -- 172.21.15.161:0/746932682 shutdown_connections 2024-05-22T21:13:20.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.691+0000 7f61cd600640 1 --2- 172.21.15.161:0/746932682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61c815d410 0x7f61c815b6c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.691+0000 7f61cd600640 1 --2- 172.21.15.161:0/746932682 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61c815e130 0x7f61c815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.691+0000 7f61cd600640 1 -- 172.21.15.161:0/746932682 >> 172.21.15.161:0/746932682 conn(0x7f61c80b0bc0 msgr2=0x7f61c80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:20.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.691+0000 7f61cd600640 1 -- 172.21.15.161:0/746932682 shutdown_connections 2024-05-22T21:13:20.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.691+0000 7f61cd600640 1 -- 172.21.15.161:0/746932682 wait complete. 2024-05-22T21:13:20.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.691+0000 7f61cd600640 1 Processor -- start 2024-05-22T21:13:20.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.691+0000 7f61cd600640 1 -- start start 2024-05-22T21:13:20.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.691+0000 7f61cd600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61c815d410 0x7f61c80b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61cd600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61c815dd50 0x7f61c814d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61cd600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61c815e130 0x7f61c80b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61cd600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80b3fe0 con 0x7f61c815e130 2024-05-22T21:13:20.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61cd600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80b3610 con 0x7f61c815dd50 2024-05-22T21:13:20.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61cd600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80b2ce0 con 0x7f61c815d410 2024-05-22T21:13:20.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61c7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61c815e130 0x7f61c80b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61c7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61c815e130 0x7f61c80b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41306/0 (socket says 172.21.15.161:41306) 2024-05-22T21:13:20.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61c7400640 1 -- 172.21.15.161:0/659231022 learned_addr learned my addr 172.21.15.161:0/659231022 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:20.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61c6a00640 1 --2- 172.21.15.161:0/659231022 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61c815d410 0x7f61c80b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61c6000640 1 --2- 172.21.15.161:0/659231022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61c815dd50 0x7f61c814d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61c7400640 1 -- 172.21.15.161:0/659231022 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61c815d410 msgr2=0x7f61c80b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61c7400640 1 --2- 172.21.15.161:0/659231022 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61c815d410 0x7f61c80b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61c7400640 1 -- 172.21.15.161:0/659231022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61c815dd50 msgr2=0x7f61c814d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61c7400640 1 --2- 172.21.15.161:0/659231022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61c815dd50 0x7f61c814d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61c7400640 1 -- 172.21.15.161:0/659231022 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b4009420 con 0x7f61c815e130 2024-05-22T21:13:20.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61c7400640 1 --2- 172.21.15.161:0/659231022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61c815e130 0x7f61c80b3aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f61c00af040 tx=0x7f61c0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:20.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61bea00640 1 -- 172.21.15.161:0/659231022 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00b8670 con 0x7f61c815e130 2024-05-22T21:13:20.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61bea00640 1 -- 172.21.15.161:0/659231022 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c00b87d0 con 0x7f61c815e130 2024-05-22T21:13:20.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61bea00640 1 -- 172.21.15.161:0/659231022 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00b8b60 con 0x7f61c815e130 2024-05-22T21:13:20.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61cd600640 1 -- 172.21.15.161:0/659231022 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c0009b30 con 0x7f61c815e130 2024-05-22T21:13:20.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61cd600640 1 -- 172.21.15.161:0/659231022 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c80b32a0 con 0x7f61c815e130 2024-05-22T21:13:20.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61cd600640 1 -- 172.21.15.161:0/659231022 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c80b37a0 con 0x7f61c815e130 2024-05-22T21:13:20.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.695+0000 7f61bea00640 1 -- 172.21.15.161:0/659231022 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f61c00b8cc0 con 0x7f61c815e130 2024-05-22T21:13:20.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.699+0000 7f61bea00640 1 --2- 172.21.15.161:0/659231022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61a8066690 0x7f61a8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:20.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.699+0000 7f61bea00640 1 -- 172.21.15.161:0/659231022 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f61c011f350 con 0x7f61c815e130 2024-05-22T21:13:20.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.699+0000 7f61c6a00640 1 --2- 172.21.15.161:0/659231022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61a8066690 0x7f61a8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:20.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.699+0000 7f61c6a00640 1 --2- 172.21.15.161:0/659231022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61a8066690 0x7f61a8068b50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f61b8007fd0 tx=0x7f61b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:20.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.703+0000 7f61bea00640 1 -- 172.21.15.161:0/659231022 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f61c00ecc00 con 0x7f61c815e130 2024-05-22T21:13:20.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.855+0000 7f61cd600640 1 -- 172.21.15.161:0/659231022 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f61c8067220 con 0x7f61c815e130 2024-05-22T21:13:20.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.855+0000 7f61bea00640 1 -- 172.21.15.161:0/659231022 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f61c00f0220 con 0x7f61c815e130 2024-05-22T21:13:20.859 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:20.859 INFO:teuthology.orchestra.run.smithi161.stdout:[ 2024-05-22T21:13:20.859 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:13:20.859 INFO:teuthology.orchestra.run.smithi161.stdout: "poolnum": 1, 2024-05-22T21:13:20.859 INFO:teuthology.orchestra.run.smithi161.stdout: "poolname": ".mgr" 2024-05-22T21:13:20.859 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:13:20.860 INFO:teuthology.orchestra.run.smithi161.stdout:] 2024-05-22T21:13:20.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.859+0000 7f61cd600640 1 -- 172.21.15.161:0/659231022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61a8066690 msgr2=0x7f61a8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.859+0000 7f61cd600640 1 --2- 172.21.15.161:0/659231022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61a8066690 0x7f61a8068b50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f61b8007fd0 tx=0x7f61b8009040 comp rx=0 tx=0).stop 2024-05-22T21:13:20.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.859+0000 7f61cd600640 1 -- 172.21.15.161:0/659231022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61c815e130 msgr2=0x7f61c80b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:20.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.859+0000 7f61cd600640 1 --2- 172.21.15.161:0/659231022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61c815e130 0x7f61c80b3aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f61c00af040 tx=0x7f61c0019710 comp rx=0 tx=0).stop 2024-05-22T21:13:20.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.859+0000 7f61cd600640 1 -- 172.21.15.161:0/659231022 shutdown_connections 2024-05-22T21:13:20.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.859+0000 7f61cd600640 1 --2- 172.21.15.161:0/659231022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61a8066690 0x7f61a8068b50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.859+0000 7f61cd600640 1 --2- 172.21.15.161:0/659231022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61c815e130 0x7f61c80b3aa0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.859+0000 7f61cd600640 1 --2- 172.21.15.161:0/659231022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61c815dd50 0x7f61c814d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.859+0000 7f61cd600640 1 --2- 172.21.15.161:0/659231022 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61c815d410 0x7f61c80b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:20.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.859+0000 7f61cd600640 1 -- 172.21.15.161:0/659231022 >> 172.21.15.161:0/659231022 conn(0x7f61c80b0bc0 msgr2=0x7f61c81591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:20.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.859+0000 7f61cd600640 1 -- 172.21.15.161:0/659231022 shutdown_connections 2024-05-22T21:13:20.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:20.859+0000 7f61cd600640 1 -- 172.21.15.161:0/659231022 wait complete. 2024-05-22T21:13:20.879 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-22T21:13:20.879 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-22T21:13:21.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.015+0000 7f1c1dc00640 1 Processor -- start 2024-05-22T21:13:21.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.015+0000 7f1c1dc00640 1 -- start start 2024-05-22T21:13:21.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.019+0000 7f1c1dc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c1815d350 0x7f1c18071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:21.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.019+0000 7f1c1dc00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c1806dfe0 con 0x7f1c1806a5e0 2024-05-22T21:13:21.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.019+0000 7f1c1dc00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c180d0c70 con 0x7f1c18071d50 2024-05-22T21:13:21.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.019+0000 7f1c1dc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c180cfbd0 con 0x7f1c1815d350 2024-05-22T21:13:21.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.019+0000 7f1c17400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1c1806a5e0 0x7f1c18166f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51186/0 (socket says 172.21.15.161:51186) 2024-05-22T21:13:21.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.019+0000 7f1c17400640 1 -- 172.21.15.161:0/3419153471 learned_addr learned my addr 172.21.15.161:0/3419153471 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:21.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.019+0000 7f1c16a00640 1 --2- 172.21.15.161:0/3419153471 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c1815d350 0x7f1c18071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:21.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.019+0000 7f1c16000640 1 -- 172.21.15.161:0/3419153471 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984713610 0 0) 0x7f1c1806dfe0 con 0x7f1c1806a5e0 2024-05-22T21:13:21.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.019+0000 7f1c16000640 1 -- 172.21.15.161:0/3419153471 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bfc003530 con 0x7f1c1806a5e0 2024-05-22T21:13:21.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.019+0000 7f1c16a00640 1 -- 172.21.15.161:0/3419153471 >> v1:172.21.15.179:6789/0 conn(0x7f1c18071d50 legacy=0x7f1c18069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:21.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.019+0000 7f1c16a00640 1 -- 172.21.15.161:0/3419153471 >> v1:172.21.15.161:6789/0 conn(0x7f1c1806a5e0 legacy=0x7f1c18166f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:21.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.019+0000 7f1c16a00640 1 -- 172.21.15.161:0/3419153471 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c180cfeb0 con 0x7f1c1815d350 2024-05-22T21:13:21.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.023+0000 7f1c16a00640 1 --2- 172.21.15.161:0/3419153471 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c1815d350 0x7f1c18071810 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1c080097f0 tx=0x7f1c0801c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45785bbfabbed1b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:21.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.023+0000 7f1c16000640 1 -- 172.21.15.161:0/3419153471 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c080ab620 con 0x7f1c1815d350 2024-05-22T21:13:21.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.023+0000 7f1c16000640 1 -- 172.21.15.161:0/3419153471 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c080ab780 con 0x7f1c1815d350 2024-05-22T21:13:21.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.023+0000 7f1c16000640 1 -- 172.21.15.161:0/3419153471 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c080abb10 con 0x7f1c1815d350 2024-05-22T21:13:21.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.023+0000 7f1c1dc00640 1 -- 172.21.15.161:0/3419153471 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c1815d350 msgr2=0x7f1c18071810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:21.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.023+0000 7f1c1dc00640 1 --2- 172.21.15.161:0/3419153471 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c1815d350 0x7f1c18071810 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1c080097f0 tx=0x7f1c0801c420 comp rx=0 tx=0).stop 2024-05-22T21:13:21.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.023+0000 7f1c1dc00640 1 -- 172.21.15.161:0/3419153471 shutdown_connections 2024-05-22T21:13:21.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.023+0000 7f1c1dc00640 1 --2- 172.21.15.161:0/3419153471 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c1815d350 0x7f1c18071810 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:21.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.023+0000 7f1c1dc00640 1 -- 172.21.15.161:0/3419153471 >> 172.21.15.161:0/3419153471 conn(0x7f1c1806afe0 msgr2=0x7f1c1806b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:21.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.023+0000 7f1c1dc00640 1 -- 172.21.15.161:0/3419153471 shutdown_connections 2024-05-22T21:13:21.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.023+0000 7f1c1dc00640 1 -- 172.21.15.161:0/3419153471 wait complete. 2024-05-22T21:13:21.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c1dc00640 1 Processor -- start 2024-05-22T21:13:21.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c1dc00640 1 -- start start 2024-05-22T21:13:21.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c1dc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c1806a5e0 0x7f1c1814dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:21.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c1dc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c18071d50 0x7f1c18155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:21.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c1dc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c1815d350 0x7f1c180cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:21.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c1dc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c18153520 con 0x7f1c18071d50 2024-05-22T21:13:21.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c1dc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c181537c0 con 0x7f1c1806a5e0 2024-05-22T21:13:21.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c1dc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c18153aa0 con 0x7f1c1815d350 2024-05-22T21:13:21.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c0fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c18071d50 0x7f1c18155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:21.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c0fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c18071d50 0x7f1c18155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41322/0 (socket says 172.21.15.161:41322) 2024-05-22T21:13:21.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c0fe00640 1 -- 172.21.15.161:0/3817286746 learned_addr learned my addr 172.21.15.161:0/3817286746 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:21.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c16a00640 1 --2- 172.21.15.161:0/3817286746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c1806a5e0 0x7f1c1814dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:21.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c17400640 1 --2- 172.21.15.161:0/3817286746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c1815d350 0x7f1c180cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:21.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c0fe00640 1 -- 172.21.15.161:0/3817286746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c1815d350 msgr2=0x7f1c180cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:21.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c0fe00640 1 --2- 172.21.15.161:0/3817286746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c1815d350 0x7f1c180cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:21.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c0fe00640 1 -- 172.21.15.161:0/3817286746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c1806a5e0 msgr2=0x7f1c1814dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:21.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c0fe00640 1 --2- 172.21.15.161:0/3817286746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c1806a5e0 0x7f1c1814dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:21.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c0fe00640 1 -- 172.21.15.161:0/3817286746 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c08009420 con 0x7f1c18071d50 2024-05-22T21:13:21.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c17400640 1 --2- 172.21.15.161:0/3817286746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c1815d350 0x7f1c180cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:21.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c0fe00640 1 --2- 172.21.15.161:0/3817286746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c18071d50 0x7f1c18155260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000c500 tx=0x7f1c0000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:21.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c0ea00640 1 -- 172.21.15.161:0/3817286746 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00005220 con 0x7f1c18071d50 2024-05-22T21:13:21.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c0ea00640 1 -- 172.21.15.161:0/3817286746 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c00005380 con 0x7f1c18071d50 2024-05-22T21:13:21.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c1dc00640 1 -- 172.21.15.161:0/3817286746 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1815ebc0 con 0x7f1c18071d50 2024-05-22T21:13:21.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.027+0000 7f1c0ea00640 1 -- 172.21.15.161:0/3817286746 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00007270 con 0x7f1c18071d50 2024-05-22T21:13:21.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.031+0000 7f1c1dc00640 1 -- 172.21.15.161:0/3817286746 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1814e1c0 con 0x7f1c18071d50 2024-05-22T21:13:21.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.031+0000 7f1c1dc00640 1 -- 172.21.15.161:0/3817286746 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1815edb0 con 0x7f1c18071d50 2024-05-22T21:13:21.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.031+0000 7f1c0ea00640 1 -- 172.21.15.161:0/3817286746 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1c000073d0 con 0x7f1c18071d50 2024-05-22T21:13:21.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.031+0000 7f1c0ea00640 1 --2- 172.21.15.161:0/3817286746 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bf4066690 0x7f1bf4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:21.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.031+0000 7f1c0ea00640 1 -- 172.21.15.161:0/3817286746 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1c00083740 con 0x7f1c18071d50 2024-05-22T21:13:21.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.031+0000 7f1c16a00640 1 --2- 172.21.15.161:0/3817286746 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bf4066690 0x7f1bf4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:21.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.031+0000 7f1c16a00640 1 --2- 172.21.15.161:0/3817286746 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bf4066690 0x7f1bf4068b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1c080af000 tx=0x7f1c080a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:21.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.035+0000 7f1c0ea00640 1 -- 172.21.15.161:0/3817286746 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1c00050d70 con 0x7f1c18071d50 2024-05-22T21:13:21.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:21.191+0000 7f1c1dc00640 1 -- 172.21.15.161:0/3817286746 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1c18067220 con 0x7f1c18071d50 2024-05-22T21:13:22.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.031+0000 7f1c0ea00640 1 -- 172.21.15.161:0/3817286746 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1c00054390 con 0x7f1c18071d50 2024-05-22T21:13:22.032 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' created 2024-05-22T21:13:22.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.035+0000 7f1c1dc00640 1 -- 172.21.15.161:0/3817286746 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bf4066690 msgr2=0x7f1bf4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:22.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.035+0000 7f1c1dc00640 1 --2- 172.21.15.161:0/3817286746 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bf4066690 0x7f1bf4068b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1c080af000 tx=0x7f1c080a7000 comp rx=0 tx=0).stop 2024-05-22T21:13:22.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.035+0000 7f1c1dc00640 1 -- 172.21.15.161:0/3817286746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c18071d50 msgr2=0x7f1c18155260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:22.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.035+0000 7f1c1dc00640 1 --2- 172.21.15.161:0/3817286746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c18071d50 0x7f1c18155260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000c500 tx=0x7f1c0000c9c0 comp rx=0 tx=0).stop 2024-05-22T21:13:22.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.035+0000 7f1c1dc00640 1 -- 172.21.15.161:0/3817286746 shutdown_connections 2024-05-22T21:13:22.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.035+0000 7f1c1dc00640 1 --2- 172.21.15.161:0/3817286746 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bf4066690 0x7f1bf4068b50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:22.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.035+0000 7f1c1dc00640 1 --2- 172.21.15.161:0/3817286746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c1815d350 0x7f1c180cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:22.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.035+0000 7f1c1dc00640 1 --2- 172.21.15.161:0/3817286746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c18071d50 0x7f1c18155260 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:22.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.035+0000 7f1c1dc00640 1 --2- 172.21.15.161:0/3817286746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c1806a5e0 0x7f1c1814dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:22.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.035+0000 7f1c1dc00640 1 -- 172.21.15.161:0/3817286746 >> 172.21.15.161:0/3817286746 conn(0x7f1c1806afe0 msgr2=0x7f1c180ac990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:22.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.035+0000 7f1c1dc00640 1 -- 172.21.15.161:0/3817286746 shutdown_connections 2024-05-22T21:13:22.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.035+0000 7f1c1dc00640 1 -- 172.21.15.161:0/3817286746 wait complete. 2024-05-22T21:13:22.060 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-22T21:13:22.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.203+0000 7fc23a200640 1 Processor -- start 2024-05-22T21:13:22.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.203+0000 7fc23a200640 1 -- start start 2024-05-22T21:13:22.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.207+0000 7fc23a200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc23415e530 0x7fc2340ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:22.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.207+0000 7fc23a200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc2340b22e0 0x7fc2340cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:22.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.207+0000 7fc23a200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2340b1470 con 0x7fc2340d18c0 2024-05-22T21:13:22.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.207+0000 7fc23a200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2340b1750 con 0x7fc2340b22e0 2024-05-22T21:13:22.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.207+0000 7fc23a200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2340b1a30 con 0x7fc23415e530 2024-05-22T21:13:22.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.207+0000 7fc22be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc2340b22e0 0x7fc2340cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:22.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.207+0000 7fc22be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc2340b22e0 0x7fc2340cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41330/0 (socket says 172.21.15.161:41330) 2024-05-22T21:13:22.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.207+0000 7fc22be00640 1 -- 172.21.15.161:0/1599747560 learned_addr learned my addr 172.21.15.161:0/1599747560 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:22.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.207+0000 7fc238e00640 1 --2- 172.21.15.161:0/1599747560 >> v2:172.21.15.179:3300/0 conn(0x7fc23415e530 0x7fc2340ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:22.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.207+0000 7fc22be00640 1 -- 172.21.15.161:0/1599747560 >> v2:172.21.15.179:3300/0 conn(0x7fc23415e530 msgr2=0x7fc2340ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:22.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.207+0000 7fc22be00640 1 --2- 172.21.15.161:0/1599747560 >> v2:172.21.15.179:3300/0 conn(0x7fc23415e530 0x7fc2340ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:22.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.207+0000 7fc22be00640 1 -- 172.21.15.161:0/1599747560 >> v1:172.21.15.179:6789/0 conn(0x7fc2340d18c0 legacy=0x7fc23415ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:22.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.207+0000 7fc22be00640 1 -- 172.21.15.161:0/1599747560 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2340adba0 con 0x7fc2340b22e0 2024-05-22T21:13:22.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.207+0000 7fc22be00640 1 --2- 172.21.15.161:0/1599747560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2340b22e0 0x7fc2340cf280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc23406a9b0 tx=0x7fc21c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10724e5ebf2596fc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:22.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.207+0000 7fc233e00640 1 -- 172.21.15.161:0/1599747560 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0ac620 con 0x7fc2340b22e0 2024-05-22T21:13:22.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc233e00640 1 -- 172.21.15.161:0/1599747560 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21c0ac780 con 0x7fc2340b22e0 2024-05-22T21:13:22.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc233e00640 1 -- 172.21.15.161:0/1599747560 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0acb10 con 0x7fc2340b22e0 2024-05-22T21:13:22.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc23a200640 1 -- 172.21.15.161:0/1599747560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2340b22e0 msgr2=0x7fc2340cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:22.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc23a200640 1 --2- 172.21.15.161:0/1599747560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2340b22e0 0x7fc2340cf280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc23406a9b0 tx=0x7fc21c01c4b0 comp rx=0 tx=0).stop 2024-05-22T21:13:22.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc23a200640 1 -- 172.21.15.161:0/1599747560 shutdown_connections 2024-05-22T21:13:22.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc23a200640 1 --2- 172.21.15.161:0/1599747560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2340b22e0 0x7fc2340cf280 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:22.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc23a200640 1 --2- 172.21.15.161:0/1599747560 >> v2:172.21.15.179:3300/0 conn(0x7fc23415e530 0x7fc2340ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:22.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc23a200640 1 -- 172.21.15.161:0/1599747560 >> 172.21.15.161:0/1599747560 conn(0x7fc2340d0dd0 msgr2=0x7fc2340d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:22.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc23a200640 1 -- 172.21.15.161:0/1599747560 shutdown_connections 2024-05-22T21:13:22.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc23a200640 1 -- 172.21.15.161:0/1599747560 wait complete. 2024-05-22T21:13:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc23a200640 1 Processor -- start 2024-05-22T21:13:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc23a200640 1 -- start start 2024-05-22T21:13:22.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc23a200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2340b22e0 0x7fc2340c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:22.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc23a200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2340d18c0 0x7fc23415a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:22.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc23a200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc23415e530 0x7fc23415a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:22.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc23a200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2340c6fb0 con 0x7fc23415e530 2024-05-22T21:13:22.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc23a200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2340c7290 con 0x7fc2340d18c0 2024-05-22T21:13:22.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc23a200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23406afe0 con 0x7fc2340b22e0 2024-05-22T21:13:22.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc239800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc23415e530 0x7fc23415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:22.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc239800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc23415e530 0x7fc23415a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41332/0 (socket says 172.21.15.161:41332) 2024-05-22T21:13:22.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc239800640 1 -- 172.21.15.161:0/2517031953 learned_addr learned my addr 172.21.15.161:0/2517031953 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:22.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc238e00640 1 --2- 172.21.15.161:0/2517031953 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2340b22e0 0x7fc2340c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:22.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc22be00640 1 --2- 172.21.15.161:0/2517031953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2340d18c0 0x7fc23415a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:22.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc239800640 1 -- 172.21.15.161:0/2517031953 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2340b22e0 msgr2=0x7fc2340c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:22.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc239800640 1 --2- 172.21.15.161:0/2517031953 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2340b22e0 0x7fc2340c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:22.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc239800640 1 -- 172.21.15.161:0/2517031953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2340d18c0 msgr2=0x7fc23415a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:22.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc239800640 1 --2- 172.21.15.161:0/2517031953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2340d18c0 0x7fc23415a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:22.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.211+0000 7fc239800640 1 -- 172.21.15.161:0/2517031953 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c0094d0 con 0x7fc23415e530 2024-05-22T21:13:22.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.215+0000 7fc239800640 1 --2- 172.21.15.161:0/2517031953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc23415e530 0x7fc23415a670 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00c780 tx=0x7fc22c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:22.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.215+0000 7fc231600640 1 -- 172.21.15.161:0/2517031953 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c004b70 con 0x7fc23415e530 2024-05-22T21:13:22.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.215+0000 7fc231600640 1 -- 172.21.15.161:0/2517031953 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22c00ce90 con 0x7fc23415e530 2024-05-22T21:13:22.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.215+0000 7fc23a200640 1 -- 172.21.15.161:0/2517031953 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23406b2c0 con 0x7fc23415e530 2024-05-22T21:13:22.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.215+0000 7fc231600640 1 -- 172.21.15.161:0/2517031953 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c00a850 con 0x7fc23415e530 2024-05-22T21:13:22.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.215+0000 7fc23a200640 1 -- 172.21.15.161:0/2517031953 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc234165980 con 0x7fc23415e530 2024-05-22T21:13:22.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.215+0000 7fc23a200640 1 -- 172.21.15.161:0/2517031953 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc234067070 con 0x7fc23415e530 2024-05-22T21:13:22.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.215+0000 7fc231600640 1 -- 172.21.15.161:0/2517031953 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc22c00a9b0 con 0x7fc23415e530 2024-05-22T21:13:22.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.215+0000 7fc231600640 1 --2- 172.21.15.161:0/2517031953 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc210066690 0x7fc210068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:22.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.215+0000 7fc231600640 1 -- 172.21.15.161:0/2517031953 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fc22c084200 con 0x7fc23415e530 2024-05-22T21:13:22.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.215+0000 7fc238e00640 1 --2- 172.21.15.161:0/2517031953 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc210066690 0x7fc210068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:22.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.219+0000 7fc238e00640 1 --2- 172.21.15.161:0/2517031953 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc210066690 0x7fc210068b50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc22400b930 tx=0x7fc224008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:22.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.219+0000 7fc231600640 1 -- 172.21.15.161:0/2517031953 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc22c0516b0 con 0x7fc23415e530 2024-05-22T21:13:22.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:22.379+0000 7fc23a200640 1 -- 172.21.15.161:0/2517031953 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc23406b770 con 0x7fc23415e530 2024-05-22T21:13:22.856 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:13:22.859 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:13:23.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.035+0000 7fc231600640 1 -- 172.21.15.161:0/2517031953 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc22c054cd0 con 0x7fc23415e530 2024-05-22T21:13:23.039 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' created 2024-05-22T21:13:23.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.043+0000 7fc23a200640 1 -- 172.21.15.161:0/2517031953 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc210066690 msgr2=0x7fc210068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.043+0000 7fc23a200640 1 --2- 172.21.15.161:0/2517031953 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc210066690 0x7fc210068b50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc22400b930 tx=0x7fc224008040 comp rx=0 tx=0).stop 2024-05-22T21:13:23.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.043+0000 7fc23a200640 1 -- 172.21.15.161:0/2517031953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc23415e530 msgr2=0x7fc23415a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.043+0000 7fc23a200640 1 --2- 172.21.15.161:0/2517031953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc23415e530 0x7fc23415a670 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00c780 tx=0x7fc22c00cc40 comp rx=0 tx=0).stop 2024-05-22T21:13:23.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.043+0000 7fc23a200640 1 -- 172.21.15.161:0/2517031953 shutdown_connections 2024-05-22T21:13:23.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.043+0000 7fc23a200640 1 --2- 172.21.15.161:0/2517031953 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc210066690 0x7fc210068b50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.043+0000 7fc23a200640 1 --2- 172.21.15.161:0/2517031953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc23415e530 0x7fc23415a670 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.043+0000 7fc23a200640 1 --2- 172.21.15.161:0/2517031953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2340d18c0 0x7fc23415a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.043+0000 7fc23a200640 1 --2- 172.21.15.161:0/2517031953 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2340b22e0 0x7fc2340c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.043+0000 7fc23a200640 1 -- 172.21.15.161:0/2517031953 >> 172.21.15.161:0/2517031953 conn(0x7fc2340d0dd0 msgr2=0x7fc2340ae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:23.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.047+0000 7fc23a200640 1 -- 172.21.15.161:0/2517031953 shutdown_connections 2024-05-22T21:13:23.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.047+0000 7fc23a200640 1 -- 172.21.15.161:0/2517031953 wait complete. 2024-05-22T21:13:23.073 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-22T21:13:23.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77d5800640 1 Processor -- start 2024-05-22T21:13:23.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77d5800640 1 -- start start 2024-05-22T21:13:23.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77d5800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77d013cfc0 0x7f77d01402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77d5800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f77d015af60 0x7f77d013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77d5800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d01400d0 con 0x7f77d013fc80 2024-05-22T21:13:23.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77d5800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d013d3a0 con 0x7f77d015af60 2024-05-22T21:13:23.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77d5800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00cd920 con 0x7f77d013cfc0 2024-05-22T21:13:23.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77ce000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f77d015af60 0x7f77d013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77ce000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f77d015af60 0x7f77d013f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41342/0 (socket says 172.21.15.161:41342) 2024-05-22T21:13:23.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77ce000640 1 -- 172.21.15.161:0/2778550768 learned_addr learned my addr 172.21.15.161:0/2778550768 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:23.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77cf400640 1 --1- 172.21.15.161:0/2778550768 >> v1:172.21.15.161:6789/0 conn(0x7f77d013fc80 0x7f77d015dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51190/0 (socket says 172.21.15.161:51190) 2024-05-22T21:13:23.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77cea00640 1 --2- 172.21.15.161:0/2778550768 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77d013cfc0 0x7f77d01402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77ce000640 1 -- 172.21.15.161:0/2778550768 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77d013cfc0 msgr2=0x7f77d01402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:23.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77ce000640 1 --2- 172.21.15.161:0/2778550768 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77d013cfc0 0x7f77d01402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77ce000640 1 -- 172.21.15.161:0/2778550768 >> v1:172.21.15.161:6789/0 conn(0x7f77d013fc80 legacy=0x7f77d015dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77ce000640 1 -- 172.21.15.161:0/2778550768 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d00cdc00 con 0x7f77d015af60 2024-05-22T21:13:23.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.211+0000 7f77cd600640 1 -- 172.21.15.161:0/2778550768 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3109898776 0 0) 0x7f77c80014a0 con 0x7f77d013fc80 2024-05-22T21:13:23.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.215+0000 7f77ce000640 1 --2- 172.21.15.161:0/2778550768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d015af60 0x7f77d013f740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009870 tx=0x7f77bc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6840514f0c69330 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:23.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.215+0000 7f77cd600640 1 -- 172.21.15.161:0/2778550768 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0ab600 con 0x7f77d015af60 2024-05-22T21:13:23.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.215+0000 7f77cd600640 1 -- 172.21.15.161:0/2778550768 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77bc0a7050 con 0x7f77d015af60 2024-05-22T21:13:23.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.215+0000 7f77cd600640 1 -- 172.21.15.161:0/2778550768 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0ab980 con 0x7f77d015af60 2024-05-22T21:13:23.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.215+0000 7f77d5800640 1 -- 172.21.15.161:0/2778550768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d015af60 msgr2=0x7f77d013f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.215+0000 7f77d5800640 1 --2- 172.21.15.161:0/2778550768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d015af60 0x7f77d013f740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009870 tx=0x7f77bc01c4d0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.215+0000 7f77d5800640 1 -- 172.21.15.161:0/2778550768 shutdown_connections 2024-05-22T21:13:23.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.215+0000 7f77d5800640 1 --2- 172.21.15.161:0/2778550768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d015af60 0x7f77d013f740 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.215+0000 7f77d5800640 1 --2- 172.21.15.161:0/2778550768 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77d013cfc0 0x7f77d01402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.215+0000 7f77d5800640 1 -- 172.21.15.161:0/2778550768 >> 172.21.15.161:0/2778550768 conn(0x7f77d006a2d0 msgr2=0x7f77d006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:23.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.215+0000 7f77d5800640 1 -- 172.21.15.161:0/2778550768 shutdown_connections 2024-05-22T21:13:23.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.215+0000 7f77d5800640 1 -- 172.21.15.161:0/2778550768 wait complete. 2024-05-22T21:13:23.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.215+0000 7f77d5800640 1 Processor -- start 2024-05-22T21:13:23.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.215+0000 7f77d5800640 1 -- start start 2024-05-22T21:13:23.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77d5800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d013cfc0 0x7f77d0140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77d5800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77d013fc80 0x7f77d01f7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77d5800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77d015af60 0x7f77d0157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77d5800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d01407b0 con 0x7f77d013cfc0 2024-05-22T21:13:23.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77d5800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d0137e70 con 0x7f77d013fc80 2024-05-22T21:13:23.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77d5800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00adca0 con 0x7f77d015af60 2024-05-22T21:13:23.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77cea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d013cfc0 0x7f77d0140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77cea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d013cfc0 0x7f77d0140270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41346/0 (socket says 172.21.15.161:41346) 2024-05-22T21:13:23.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77cea00640 1 -- 172.21.15.161:0/1031259659 learned_addr learned my addr 172.21.15.161:0/1031259659 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:23.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77cf400640 1 --2- 172.21.15.161:0/1031259659 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77d015af60 0x7f77d0157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77ce000640 1 --2- 172.21.15.161:0/1031259659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77d013fc80 0x7f77d01f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77cea00640 1 -- 172.21.15.161:0/1031259659 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77d015af60 msgr2=0x7f77d0157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77cea00640 1 --2- 172.21.15.161:0/1031259659 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77d015af60 0x7f77d0157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77cea00640 1 -- 172.21.15.161:0/1031259659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77d013fc80 msgr2=0x7f77d01f7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77cea00640 1 --2- 172.21.15.161:0/1031259659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77d013fc80 0x7f77d01f7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77cea00640 1 -- 172.21.15.161:0/1031259659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc0094d0 con 0x7f77d013cfc0 2024-05-22T21:13:23.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77cea00640 1 --2- 172.21.15.161:0/1031259659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d013cfc0 0x7f77d0140270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f77c000d5f0 tx=0x7f77c000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:23.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77c6a00640 1 -- 172.21.15.161:0/1031259659 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c00125f0 con 0x7f77d013cfc0 2024-05-22T21:13:23.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77c6a00640 1 -- 172.21.15.161:0/1031259659 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c0012750 con 0x7f77d013cfc0 2024-05-22T21:13:23.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77c6a00640 1 -- 172.21.15.161:0/1031259659 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0012ae0 con 0x7f77d013cfc0 2024-05-22T21:13:23.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77d5800640 1 -- 172.21.15.161:0/1031259659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d00adf80 con 0x7f77d013cfc0 2024-05-22T21:13:23.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77d5800640 1 -- 172.21.15.161:0/1031259659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d01f76d0 con 0x7f77d013cfc0 2024-05-22T21:13:23.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.219+0000 7f77d5800640 1 -- 172.21.15.161:0/1031259659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d0067220 con 0x7f77d013cfc0 2024-05-22T21:13:23.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.223+0000 7f77c6a00640 1 -- 172.21.15.161:0/1031259659 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f77c0012c40 con 0x7f77d013cfc0 2024-05-22T21:13:23.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.223+0000 7f77c6a00640 1 --2- 172.21.15.161:0/1031259659 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f77b0066690 0x7f77b0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.223+0000 7f77c6a00640 1 -- 172.21.15.161:0/1031259659 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f77c00845b0 con 0x7f77d013cfc0 2024-05-22T21:13:23.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.223+0000 7f77ce000640 1 --2- 172.21.15.161:0/1031259659 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f77b0066690 0x7f77b0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.223+0000 7f77ce000640 1 --2- 172.21.15.161:0/1031259659 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f77b0066690 0x7f77b0068b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0ad000 tx=0x7f77bc0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:23.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.227+0000 7f77c6a00640 1 -- 172.21.15.161:0/1031259659 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f77c0051990 con 0x7f77d013cfc0 2024-05-22T21:13:23.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.415+0000 7f77d5800640 1 -- 172.21.15.161:0/1031259659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f77d0071670 con 0x7f77d013cfc0 2024-05-22T21:13:23.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.423+0000 7f77c6a00640 1 -- 172.21.15.161:0/1031259659 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f77c0054fb0 con 0x7f77d013cfc0 2024-05-22T21:13:23.426 INFO:teuthology.orchestra.run.smithi161.stderr:new fs with metadata pool 2 and data pool 3 2024-05-22T21:13:23.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.427+0000 7f77d5800640 1 -- 172.21.15.161:0/1031259659 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f77b0066690 msgr2=0x7f77b0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.427+0000 7f77d5800640 1 --2- 172.21.15.161:0/1031259659 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f77b0066690 0x7f77b0068b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0ad000 tx=0x7f77bc0a6000 comp rx=0 tx=0).stop 2024-05-22T21:13:23.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.427+0000 7f77d5800640 1 -- 172.21.15.161:0/1031259659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d013cfc0 msgr2=0x7f77d0140270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.427+0000 7f77d5800640 1 --2- 172.21.15.161:0/1031259659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d013cfc0 0x7f77d0140270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f77c000d5f0 tx=0x7f77c000dab0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.427+0000 7f77d5800640 1 -- 172.21.15.161:0/1031259659 shutdown_connections 2024-05-22T21:13:23.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.427+0000 7f77d5800640 1 --2- 172.21.15.161:0/1031259659 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f77b0066690 0x7f77b0068b50 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0ad000 tx=0x7f77bc0a6000 comp rx=0 tx=0).stop 2024-05-22T21:13:23.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.427+0000 7f77d5800640 1 --2- 172.21.15.161:0/1031259659 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77d015af60 0x7f77d0157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.427+0000 7f77d5800640 1 --2- 172.21.15.161:0/1031259659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77d013fc80 0x7f77d01f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.427+0000 7f77d5800640 1 --2- 172.21.15.161:0/1031259659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77d013cfc0 0x7f77d0140270 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.427+0000 7f77d5800640 1 -- 172.21.15.161:0/1031259659 >> 172.21.15.161:0/1031259659 conn(0x7f77d006a2d0 msgr2=0x7f77d01382e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:23.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.431+0000 7f77d5800640 1 -- 172.21.15.161:0/1031259659 shutdown_connections 2024-05-22T21:13:23.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.431+0000 7f77d5800640 1 -- 172.21.15.161:0/1031259659 wait complete. 2024-05-22T21:13:23.459 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:23.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.591+0000 7f97a8600640 1 Processor -- start 2024-05-22T21:13:23.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.591+0000 7f97a8600640 1 -- start start 2024-05-22T21:13:23.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.591+0000 7f97a8600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97a0143ec0 0x7f97a015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.591+0000 7f97a8600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f97a015ec90 0x7f97a015f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.591+0000 7f97a8600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a01442a0 con 0x7f97a00715a0 2024-05-22T21:13:23.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.591+0000 7f97a8600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a015f680 con 0x7f97a015ec90 2024-05-22T21:13:23.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.591+0000 7f97a8600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a0070500 con 0x7f97a0143ec0 2024-05-22T21:13:23.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.591+0000 7f97a6600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f97a015ec90 0x7f97a015f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.591+0000 7f97a5c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97a0143ec0 0x7f97a015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.591+0000 7f97a6600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f97a015ec90 0x7f97a015f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:49964/0 (socket says 172.21.15.161:49964) 2024-05-22T21:13:23.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.591+0000 7f97a6600640 1 -- 172.21.15.161:0/1852716318 learned_addr learned my addr 172.21.15.161:0/1852716318 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:23.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f97a6600640 1 -- 172.21.15.161:0/1852716318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97a0143ec0 msgr2=0x7f97a015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:23.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f97a6600640 1 --2- 172.21.15.161:0/1852716318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97a0143ec0 0x7f97a015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f97a6600640 1 -- 172.21.15.161:0/1852716318 >> v1:172.21.15.179:6789/0 conn(0x7f97a00715a0 legacy=0x7f97a006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f97a6600640 1 -- 172.21.15.161:0/1852716318 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a00707e0 con 0x7f97a015ec90 2024-05-22T21:13:23.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f97a5c00640 1 --2- 172.21.15.161:0/1852716318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97a0143ec0 0x7f97a015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:23.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f97a6600640 1 --2- 172.21.15.161:0/1852716318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97a015ec90 0x7f97a015f140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9798009870 tx=0x7f979801c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14c512f3d9aaad7d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:23.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f979fe00640 1 -- 172.21.15.161:0/1852716318 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97980a7070 con 0x7f97a015ec90 2024-05-22T21:13:23.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f979fe00640 1 -- 172.21.15.161:0/1852716318 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97980ab430 con 0x7f97a015ec90 2024-05-22T21:13:23.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f979fe00640 1 -- 172.21.15.161:0/1852716318 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97980ab7c0 con 0x7f97a015ec90 2024-05-22T21:13:23.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f97a8600640 1 -- 172.21.15.161:0/1852716318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97a015ec90 msgr2=0x7f97a015f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f97a8600640 1 --2- 172.21.15.161:0/1852716318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97a015ec90 0x7f97a015f140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9798009870 tx=0x7f979801c530 comp rx=0 tx=0).stop 2024-05-22T21:13:23.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f97a8600640 1 -- 172.21.15.161:0/1852716318 shutdown_connections 2024-05-22T21:13:23.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f97a8600640 1 --2- 172.21.15.161:0/1852716318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97a015ec90 0x7f97a015f140 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f97a8600640 1 --2- 172.21.15.161:0/1852716318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97a0143ec0 0x7f97a015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f97a8600640 1 -- 172.21.15.161:0/1852716318 >> 172.21.15.161:0/1852716318 conn(0x7f97a00b0bc0 msgr2=0x7f97a00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:23.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f97a8600640 1 -- 172.21.15.161:0/1852716318 shutdown_connections 2024-05-22T21:13:23.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.595+0000 7f97a8600640 1 -- 172.21.15.161:0/1852716318 wait complete. 2024-05-22T21:13:23.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a8600640 1 Processor -- start 2024-05-22T21:13:23.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a8600640 1 -- start start 2024-05-22T21:13:23.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a8600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97a0143ec0 0x7f97a0141770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a8600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97a015ec90 0x7f97a0144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a8600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a01462e0 0x7f97a013b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a8600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a0145240 con 0x7f97a01462e0 2024-05-22T21:13:23.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a8600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a0145520 con 0x7f97a0143ec0 2024-05-22T21:13:23.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a8600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a0145800 con 0x7f97a015ec90 2024-05-22T21:13:23.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a6600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a01462e0 0x7f97a013b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a6600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a01462e0 0x7f97a013b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41360/0 (socket says 172.21.15.161:41360) 2024-05-22T21:13:23.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a6600640 1 -- 172.21.15.161:0/4274705108 learned_addr learned my addr 172.21.15.161:0/4274705108 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:23.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a5200640 1 --2- 172.21.15.161:0/4274705108 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97a015ec90 0x7f97a0144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a5c00640 1 --2- 172.21.15.161:0/4274705108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97a0143ec0 0x7f97a0141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a6600640 1 -- 172.21.15.161:0/4274705108 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97a015ec90 msgr2=0x7f97a0144780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a6600640 1 --2- 172.21.15.161:0/4274705108 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97a015ec90 0x7f97a0144780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a6600640 1 -- 172.21.15.161:0/4274705108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97a0143ec0 msgr2=0x7f97a0141770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a6600640 1 --2- 172.21.15.161:0/4274705108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97a0143ec0 0x7f97a0141770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a6600640 1 -- 172.21.15.161:0/4274705108 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9794009a70 con 0x7f97a01462e0 2024-05-22T21:13:23.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a6600640 1 --2- 172.21.15.161:0/4274705108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a01462e0 0x7f97a013b940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f97980ad040 tx=0x7f97980192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:23.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f979d600640 1 -- 172.21.15.161:0/4274705108 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97980b4620 con 0x7f97a01462e0 2024-05-22T21:13:23.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f979d600640 1 -- 172.21.15.161:0/4274705108 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97980abe90 con 0x7f97a01462e0 2024-05-22T21:13:23.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f979d600640 1 -- 172.21.15.161:0/4274705108 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97980b4980 con 0x7f97a01462e0 2024-05-22T21:13:23.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.599+0000 7f97a8600640 1 -- 172.21.15.161:0/4274705108 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97980094d0 con 0x7f97a01462e0 2024-05-22T21:13:23.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.603+0000 7f97a8600640 1 -- 172.21.15.161:0/4274705108 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a0145dc0 con 0x7f97a01462e0 2024-05-22T21:13:23.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.603+0000 7f97a8600640 1 -- 172.21.15.161:0/4274705108 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a0145fb0 con 0x7f97a01462e0 2024-05-22T21:13:23.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.603+0000 7f979d600640 1 -- 172.21.15.161:0/4274705108 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f97980b4ae0 con 0x7f97a01462e0 2024-05-22T21:13:23.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.603+0000 7f979d600640 1 --2- 172.21.15.161:0/4274705108 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97800666c0 0x7f9780068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.603+0000 7f979d600640 1 -- 172.21.15.161:0/4274705108 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f979811e2f0 con 0x7f97a01462e0 2024-05-22T21:13:23.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.603+0000 7f97a5c00640 1 --2- 172.21.15.161:0/4274705108 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97800666c0 0x7f9780068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.603+0000 7f97a5c00640 1 --2- 172.21.15.161:0/4274705108 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97800666c0 0x7f9780068b80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9794009fd0 tx=0x7f979400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:23.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.607+0000 7f979d600640 1 -- 172.21.15.161:0/4274705108 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f97980eb670 con 0x7f97a01462e0 2024-05-22T21:13:23.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.763+0000 7f97a8600640 1 -- 172.21.15.161:0/4274705108 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97a0067220 con 0x7f97a01462e0 2024-05-22T21:13:23.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.763+0000 7f979d600640 1 -- 172.21.15.161:0/4274705108 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16098 (secure 0 0 0) 0x7f97980eec90 con 0x7f97a01462e0 2024-05-22T21:13:23.767 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:23.767 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":20,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:23.418070+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:23.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.767+0000 7f97a8600640 1 -- 172.21.15.161:0/4274705108 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97800666c0 msgr2=0x7f9780068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.767+0000 7f97a8600640 1 --2- 172.21.15.161:0/4274705108 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97800666c0 0x7f9780068b80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9794009fd0 tx=0x7f979400b040 comp rx=0 tx=0).stop 2024-05-22T21:13:23.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.767+0000 7f97a8600640 1 -- 172.21.15.161:0/4274705108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a01462e0 msgr2=0x7f97a013b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.767+0000 7f97a8600640 1 --2- 172.21.15.161:0/4274705108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a01462e0 0x7f97a013b940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f97980ad040 tx=0x7f97980192d0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.767+0000 7f97a8600640 1 -- 172.21.15.161:0/4274705108 shutdown_connections 2024-05-22T21:13:23.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.767+0000 7f97a8600640 1 --2- 172.21.15.161:0/4274705108 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97800666c0 0x7f9780068b80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.767+0000 7f97a8600640 1 --2- 172.21.15.161:0/4274705108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97a01462e0 0x7f97a013b940 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.767+0000 7f97a8600640 1 --2- 172.21.15.161:0/4274705108 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97a015ec90 0x7f97a0144780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.767+0000 7f97a8600640 1 --2- 172.21.15.161:0/4274705108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97a0143ec0 0x7f97a0141770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.767+0000 7f97a8600640 1 -- 172.21.15.161:0/4274705108 >> 172.21.15.161:0/4274705108 conn(0x7f97a00b0bc0 msgr2=0x7f97a015bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:23.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.767+0000 7f97a8600640 1 -- 172.21.15.161:0/4274705108 shutdown_connections 2024-05-22T21:13:23.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.767+0000 7f97a8600640 1 -- 172.21.15.161:0/4274705108 wait complete. 2024-05-22T21:13:23.789 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:23.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea33e00640 1 Processor -- start 2024-05-22T21:13:23.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea33e00640 1 -- start start 2024-05-22T21:13:23.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea33e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2c0715a0 0x7fea2c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea33e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fea2c15ec90 0x7fea2c15f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea33e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2c1442a0 con 0x7fea2c143ec0 2024-05-22T21:13:23.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea33e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2c15f660 con 0x7fea2c15ec90 2024-05-22T21:13:23.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea33e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c070500 con 0x7fea2c0715a0 2024-05-22T21:13:23.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea31400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fea2c143ec0 0x7fea2c06fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51194/0 (socket says 172.21.15.161:51194) 2024-05-22T21:13:23.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea31400640 1 -- 172.21.15.161:0/3169315517 learned_addr learned my addr 172.21.15.161:0/3169315517 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:23.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea31e00640 1 --2- 172.21.15.161:0/3169315517 >> v2:172.21.15.179:3300/0 conn(0x7fea2c15ec90 0x7fea2c15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea30a00640 1 --2- 172.21.15.161:0/3169315517 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2c0715a0 0x7fea2c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea2be00640 1 -- 172.21.15.161:0/3169315517 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310496946 0 0) 0x7fea2c1442a0 con 0x7fea2c143ec0 2024-05-22T21:13:23.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea2be00640 1 -- 172.21.15.161:0/3169315517 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea14003530 con 0x7fea2c143ec0 2024-05-22T21:13:23.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea31e00640 1 -- 172.21.15.161:0/3169315517 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2c0715a0 msgr2=0x7fea2c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:23.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea31e00640 1 --2- 172.21.15.161:0/3169315517 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2c0715a0 0x7fea2c15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea31e00640 1 -- 172.21.15.161:0/3169315517 >> v1:172.21.15.161:6789/0 conn(0x7fea2c143ec0 legacy=0x7fea2c06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.923+0000 7fea31e00640 1 -- 172.21.15.161:0/3169315517 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c0707e0 con 0x7fea2c15ec90 2024-05-22T21:13:23.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.927+0000 7fea31e00640 1 --2- 172.21.15.161:0/3169315517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea2c15ec90 0x7fea2c15f120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea24009d90 tx=0x7fea2401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=751157d636f7e7d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:23.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.927+0000 7fea2be00640 1 -- 172.21.15.161:0/3169315517 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea240a5900 con 0x7fea2c15ec90 2024-05-22T21:13:23.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.927+0000 7fea2be00640 1 -- 172.21.15.161:0/3169315517 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea240a5a60 con 0x7fea2c15ec90 2024-05-22T21:13:23.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.927+0000 7fea2be00640 1 -- 172.21.15.161:0/3169315517 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea240a5df0 con 0x7fea2c15ec90 2024-05-22T21:13:23.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.927+0000 7fea33e00640 1 -- 172.21.15.161:0/3169315517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea2c15ec90 msgr2=0x7fea2c15f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.927+0000 7fea33e00640 1 --2- 172.21.15.161:0/3169315517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea2c15ec90 0x7fea2c15f120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea24009d90 tx=0x7fea2401c940 comp rx=0 tx=0).stop 2024-05-22T21:13:23.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.927+0000 7fea33e00640 1 -- 172.21.15.161:0/3169315517 shutdown_connections 2024-05-22T21:13:23.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.927+0000 7fea33e00640 1 --2- 172.21.15.161:0/3169315517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea2c15ec90 0x7fea2c15f120 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.927+0000 7fea33e00640 1 --2- 172.21.15.161:0/3169315517 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2c0715a0 0x7fea2c15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.927+0000 7fea33e00640 1 -- 172.21.15.161:0/3169315517 >> 172.21.15.161:0/3169315517 conn(0x7fea2c0b0bc0 msgr2=0x7fea2c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:23.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.927+0000 7fea33e00640 1 -- 172.21.15.161:0/3169315517 shutdown_connections 2024-05-22T21:13:23.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.927+0000 7fea33e00640 1 -- 172.21.15.161:0/3169315517 wait complete. 2024-05-22T21:13:23.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea33e00640 1 Processor -- start 2024-05-22T21:13:23.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea33e00640 1 -- start start 2024-05-22T21:13:23.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea33e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2c143ec0 0x7fea2c142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea33e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea2c15ec90 0x7fea2c15db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea33e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea2c13b540 0x7fea2c13b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea33e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c141ad0 con 0x7fea2c15ec90 2024-05-22T21:13:23.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea33e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c141db0 con 0x7fea2c13b540 2024-05-22T21:13:23.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea33e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c142090 con 0x7fea2c143ec0 2024-05-22T21:13:23.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea30a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea2c15ec90 0x7fea2c15db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea30a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea2c15ec90 0x7fea2c15db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41374/0 (socket says 172.21.15.161:41374) 2024-05-22T21:13:23.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea30a00640 1 -- 172.21.15.161:0/56743980 learned_addr learned my addr 172.21.15.161:0/56743980 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:23.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea31400640 1 --2- 172.21.15.161:0/56743980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2c143ec0 0x7fea2c142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea31e00640 1 --2- 172.21.15.161:0/56743980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea2c13b540 0x7fea2c13b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea30a00640 1 -- 172.21.15.161:0/56743980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2c143ec0 msgr2=0x7fea2c142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea30a00640 1 --2- 172.21.15.161:0/56743980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2c143ec0 0x7fea2c142b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea30a00640 1 -- 172.21.15.161:0/56743980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea2c13b540 msgr2=0x7fea2c13b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:23.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea30a00640 1 --2- 172.21.15.161:0/56743980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea2c13b540 0x7fea2c13b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:23.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea30a00640 1 -- 172.21.15.161:0/56743980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea24009a70 con 0x7fea2c15ec90 2024-05-22T21:13:23.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea31400640 1 --2- 172.21.15.161:0/56743980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2c143ec0 0x7fea2c142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:23.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea31e00640 1 --2- 172.21.15.161:0/56743980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea2c13b540 0x7fea2c13b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:23.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea30a00640 1 --2- 172.21.15.161:0/56743980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea2c15ec90 0x7fea2c15db40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea1800e930 tx=0x7fea1800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:23.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea29600640 1 -- 172.21.15.161:0/56743980 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18009690 con 0x7fea2c15ec90 2024-05-22T21:13:23.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea29600640 1 -- 172.21.15.161:0/56743980 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1800ca00 con 0x7fea2c15ec90 2024-05-22T21:13:23.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea29600640 1 -- 172.21.15.161:0/56743980 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1800cd70 con 0x7fea2c15ec90 2024-05-22T21:13:23.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea33e00640 1 -- 172.21.15.161:0/56743980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2c142370 con 0x7fea2c15ec90 2024-05-22T21:13:23.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea33e00640 1 -- 172.21.15.161:0/56743980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2c13bf10 con 0x7fea2c15ec90 2024-05-22T21:13:23.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.931+0000 7fea33e00640 1 -- 172.21.15.161:0/56743980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2c067220 con 0x7fea2c15ec90 2024-05-22T21:13:23.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.935+0000 7fea29600640 1 -- 172.21.15.161:0/56743980 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fea18013470 con 0x7fea2c15ec90 2024-05-22T21:13:23.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.935+0000 7fea29600640 1 --2- 172.21.15.161:0/56743980 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea0c066690 0x7fea0c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:23.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.935+0000 7fea29600640 1 -- 172.21.15.161:0/56743980 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea18084400 con 0x7fea2c15ec90 2024-05-22T21:13:23.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.935+0000 7fea31400640 1 --2- 172.21.15.161:0/56743980 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea0c066690 0x7fea0c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:23.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.935+0000 7fea31400640 1 --2- 172.21.15.161:0/56743980 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea0c066690 0x7fea0c068b50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fea200011f0 tx=0x7fea20009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:23.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:23.939+0000 7fea29600640 1 -- 172.21.15.161:0/56743980 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fea18051820 con 0x7fea2c15ec90 2024-05-22T21:13:24.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.095+0000 7fea33e00640 1 -- 172.21.15.161:0/56743980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea2c06dcc0 con 0x7fea2c15ec90 2024-05-22T21:13:24.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.095+0000 7fea29600640 1 -- 172.21.15.161:0/56743980 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16098 (secure 0 0 0) 0x7fea18054e40 con 0x7fea2c15ec90 2024-05-22T21:13:24.098 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:24.099 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":20,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:23.418070+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.099+0000 7fea33e00640 1 -- 172.21.15.161:0/56743980 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea0c066690 msgr2=0x7fea0c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.099+0000 7fea33e00640 1 --2- 172.21.15.161:0/56743980 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea0c066690 0x7fea0c068b50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fea200011f0 tx=0x7fea20009040 comp rx=0 tx=0).stop 2024-05-22T21:13:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.099+0000 7fea33e00640 1 -- 172.21.15.161:0/56743980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea2c15ec90 msgr2=0x7fea2c15db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.099+0000 7fea33e00640 1 --2- 172.21.15.161:0/56743980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea2c15ec90 0x7fea2c15db40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea1800e930 tx=0x7fea1800edf0 comp rx=0 tx=0).stop 2024-05-22T21:13:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.099+0000 7fea33e00640 1 -- 172.21.15.161:0/56743980 shutdown_connections 2024-05-22T21:13:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.099+0000 7fea33e00640 1 --2- 172.21.15.161:0/56743980 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea0c066690 0x7fea0c068b50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.099+0000 7fea33e00640 1 --2- 172.21.15.161:0/56743980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea2c13b540 0x7fea2c13b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.099+0000 7fea33e00640 1 --2- 172.21.15.161:0/56743980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea2c15ec90 0x7fea2c15db40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.099+0000 7fea33e00640 1 --2- 172.21.15.161:0/56743980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2c143ec0 0x7fea2c142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.099+0000 7fea33e00640 1 -- 172.21.15.161:0/56743980 >> 172.21.15.161:0/56743980 conn(0x7fea2c0b0bc0 msgr2=0x7fea2c06bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.099+0000 7fea33e00640 1 -- 172.21.15.161:0/56743980 shutdown_connections 2024-05-22T21:13:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.099+0000 7fea33e00640 1 -- 172.21.15.161:0/56743980 wait complete. 2024-05-22T21:13:24.120 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-22T21:13:24.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.255+0000 7f705a600640 1 Processor -- start 2024-05-22T21:13:24.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.255+0000 7f705a600640 1 -- start start 2024-05-22T21:13:24.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.255+0000 7f705a600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7054143c40 0x7f705415b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:24.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.255+0000 7f705a600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f70541444a0 0x7f705406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:24.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.255+0000 7f705a600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705415efb0 con 0x7f7054144900 2024-05-22T21:13:24.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.255+0000 7f705a600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705415f290 con 0x7f70541444a0 2024-05-22T21:13:24.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.255+0000 7f705a600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705415f570 con 0x7f7054143c40 2024-05-22T21:13:24.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.255+0000 7f7052a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f70541444a0 0x7f705406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:24.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.255+0000 7f7052a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f70541444a0 0x7f705406c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41390/0 (socket says 172.21.15.161:41390) 2024-05-22T21:13:24.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.255+0000 7f7052a00640 1 -- 172.21.15.161:0/708783681 learned_addr learned my addr 172.21.15.161:0/708783681 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:24.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.255+0000 7f7053e00640 1 --1- 172.21.15.161:0/708783681 >> v1:172.21.15.161:6789/0 conn(0x7f7054144900 0x7f70540b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51198/0 (socket says 172.21.15.161:51198) 2024-05-22T21:13:24.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f7053400640 1 --2- 172.21.15.161:0/708783681 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7054143c40 0x7f705415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:24.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f7052a00640 1 -- 172.21.15.161:0/708783681 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7054143c40 msgr2=0x7f705415b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:24.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f7052a00640 1 --2- 172.21.15.161:0/708783681 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7054143c40 0x7f705415b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:24.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f7052a00640 1 -- 172.21.15.161:0/708783681 >> v1:172.21.15.161:6789/0 conn(0x7f7054144900 legacy=0x7f70540b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:24.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f7052a00640 1 -- 172.21.15.161:0/708783681 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705415f850 con 0x7f70541444a0 2024-05-22T21:13:24.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f7052000640 1 -- 172.21.15.161:0/708783681 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812630279 0 0) 0x7f705415efb0 con 0x7f7054144900 2024-05-22T21:13:24.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f7052a00640 1 --2- 172.21.15.161:0/708783681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70541444a0 0x7f705406c2a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7040009870 tx=0x7f704001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85af2bc2c8098264 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:24.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f7052000640 1 -- 172.21.15.161:0/708783681 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70400ab670 con 0x7f70541444a0 2024-05-22T21:13:24.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f7052000640 1 -- 172.21.15.161:0/708783681 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70400ab7d0 con 0x7f70541444a0 2024-05-22T21:13:24.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f7052000640 1 -- 172.21.15.161:0/708783681 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70400abb60 con 0x7f70541444a0 2024-05-22T21:13:24.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f705a600640 1 -- 172.21.15.161:0/708783681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70541444a0 msgr2=0x7f705406c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:24.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f705a600640 1 --2- 172.21.15.161:0/708783681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70541444a0 0x7f705406c2a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7040009870 tx=0x7f704001c560 comp rx=0 tx=0).stop 2024-05-22T21:13:24.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f705a600640 1 -- 172.21.15.161:0/708783681 shutdown_connections 2024-05-22T21:13:24.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f705a600640 1 --2- 172.21.15.161:0/708783681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70541444a0 0x7f705406c2a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:24.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f705a600640 1 --2- 172.21.15.161:0/708783681 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7054143c40 0x7f705415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:24.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f705a600640 1 -- 172.21.15.161:0/708783681 >> 172.21.15.161:0/708783681 conn(0x7f705406a2d0 msgr2=0x7f705406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:24.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f705a600640 1 -- 172.21.15.161:0/708783681 shutdown_connections 2024-05-22T21:13:24.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.259+0000 7f705a600640 1 -- 172.21.15.161:0/708783681 wait complete. 2024-05-22T21:13:24.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f705a600640 1 Processor -- start 2024-05-22T21:13:24.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f705a600640 1 -- start start 2024-05-22T21:13:24.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f705a600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7054143c40 0x7f7054159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:24.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f705a600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70541444a0 0x7f70540ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:24.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f705a600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7054144900 0x7f70540adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:24.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f705a600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705415e2e0 con 0x7f7054144900 2024-05-22T21:13:24.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f705a600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705415d430 con 0x7f7054143c40 2024-05-22T21:13:24.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f705a600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705415d6d0 con 0x7f70541444a0 2024-05-22T21:13:24.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f7053e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7054144900 0x7f70540adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:24.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f7053e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7054144900 0x7f70540adf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41398/0 (socket says 172.21.15.161:41398) 2024-05-22T21:13:24.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f7053e00640 1 -- 172.21.15.161:0/246863382 learned_addr learned my addr 172.21.15.161:0/246863382 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:24.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f7052a00640 1 --2- 172.21.15.161:0/246863382 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70541444a0 0x7f70540ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:24.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f7053400640 1 --2- 172.21.15.161:0/246863382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7054143c40 0x7f7054159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:24.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f7053e00640 1 -- 172.21.15.161:0/246863382 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70541444a0 msgr2=0x7f70540ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:24.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f7053e00640 1 --2- 172.21.15.161:0/246863382 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70541444a0 0x7f70540ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:24.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f7053e00640 1 -- 172.21.15.161:0/246863382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7054143c40 msgr2=0x7f7054159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:24.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f7053e00640 1 --2- 172.21.15.161:0/246863382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7054143c40 0x7f7054159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:24.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f7053e00640 1 -- 172.21.15.161:0/246863382 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70400094d0 con 0x7f7054144900 2024-05-22T21:13:24.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f7053400640 1 --2- 172.21.15.161:0/246863382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7054143c40 0x7f7054159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:24.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f7053e00640 1 --2- 172.21.15.161:0/246863382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7054144900 0x7f70540adf90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f704c00c3e0 tx=0x7f704c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:24.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f704b400640 1 -- 172.21.15.161:0/246863382 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c0074e0 con 0x7f7054144900 2024-05-22T21:13:24.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f704b400640 1 -- 172.21.15.161:0/246863382 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c007660 con 0x7f7054144900 2024-05-22T21:13:24.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f704b400640 1 -- 172.21.15.161:0/246863382 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c004a30 con 0x7f7054144900 2024-05-22T21:13:24.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f705a600640 1 -- 172.21.15.161:0/246863382 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705415d9b0 con 0x7f7054144900 2024-05-22T21:13:24.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.263+0000 7f705a600640 1 -- 172.21.15.161:0/246863382 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705415df70 con 0x7f7054144900 2024-05-22T21:13:24.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.267+0000 7f705a600640 1 -- 172.21.15.161:0/246863382 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7054067220 con 0x7f7054144900 2024-05-22T21:13:24.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.267+0000 7f704b400640 1 -- 172.21.15.161:0/246863382 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f704c004b90 con 0x7f7054144900 2024-05-22T21:13:24.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.267+0000 7f704b400640 1 --2- 172.21.15.161:0/246863382 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7034066690 0x7f7034068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:24.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.267+0000 7f704b400640 1 -- 172.21.15.161:0/246863382 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f704c083f60 con 0x7f7054144900 2024-05-22T21:13:24.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.267+0000 7f7053400640 1 --2- 172.21.15.161:0/246863382 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7034066690 0x7f7034068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:24.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.267+0000 7f7053400640 1 --2- 172.21.15.161:0/246863382 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7034066690 0x7f7034068b50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7044003fd0 tx=0x7f70440039e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:24.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.271+0000 7f704b400640 1 -- 172.21.15.161:0/246863382 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f704c054c30 con 0x7f7054144900 2024-05-22T21:13:24.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:24.463+0000 7f705a600640 1 -- 172.21.15.161:0/246863382 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f70540ceaa0 con 0x7f7054144900 2024-05-22T21:13:25.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.439+0000 7f704b400640 1 -- 172.21.15.161:0/246863382 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f704c001090 con 0x7f7054144900 2024-05-22T21:13:25.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.443+0000 7f705a600640 1 -- 172.21.15.161:0/246863382 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7034066690 msgr2=0x7f7034068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:25.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.443+0000 7f705a600640 1 --2- 172.21.15.161:0/246863382 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7034066690 0x7f7034068b50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7044003fd0 tx=0x7f70440039e0 comp rx=0 tx=0).stop 2024-05-22T21:13:25.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.443+0000 7f705a600640 1 -- 172.21.15.161:0/246863382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7054144900 msgr2=0x7f70540adf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:25.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.443+0000 7f705a600640 1 --2- 172.21.15.161:0/246863382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7054144900 0x7f70540adf90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f704c00c3e0 tx=0x7f704c00c8a0 comp rx=0 tx=0).stop 2024-05-22T21:13:25.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.443+0000 7f705a600640 1 -- 172.21.15.161:0/246863382 shutdown_connections 2024-05-22T21:13:25.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.443+0000 7f705a600640 1 --2- 172.21.15.161:0/246863382 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7034066690 0x7f7034068b50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:25.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.443+0000 7f705a600640 1 --2- 172.21.15.161:0/246863382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7054144900 0x7f70540adf90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:25.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.443+0000 7f705a600640 1 --2- 172.21.15.161:0/246863382 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70541444a0 0x7f70540ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:25.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.443+0000 7f705a600640 1 --2- 172.21.15.161:0/246863382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7054143c40 0x7f7054159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:25.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.443+0000 7f705a600640 1 -- 172.21.15.161:0/246863382 >> 172.21.15.161:0/246863382 conn(0x7f705406a2d0 msgr2=0x7f70540afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:25.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.443+0000 7f705a600640 1 -- 172.21.15.161:0/246863382 shutdown_connections 2024-05-22T21:13:25.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.443+0000 7f705a600640 1 -- 172.21.15.161:0/246863382 wait complete. 2024-05-22T21:13:25.467 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']} 2024-05-22T21:13:25.467 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-22T21:13:25.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75b4e00640 1 Processor -- start 2024-05-22T21:13:25.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75b4e00640 1 -- start start 2024-05-22T21:13:25.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75b4e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75b013cfc0 0x7f75b01402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:25.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75b4e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f75b015af60 0x7f75b013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:25.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75b4e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f75b013fc80 0x7f75b015dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:25.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75b4e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b01400d0 con 0x7f75b015af60 2024-05-22T21:13:25.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75b4e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b013d3a0 con 0x7f75b013fc80 2024-05-22T21:13:25.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75b4e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00cd920 con 0x7f75b013cfc0 2024-05-22T21:13:25.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75a7e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f75b015af60 0x7f75b013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:25.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75a7e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f75b015af60 0x7f75b013f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41406/0 (socket says 172.21.15.161:41406) 2024-05-22T21:13:25.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75a7e00640 1 -- 172.21.15.161:0/2804675864 learned_addr learned my addr 172.21.15.161:0/2804675864 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:25.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75ae800640 1 --2- 172.21.15.161:0/2804675864 >> v2:172.21.15.179:3300/0 conn(0x7f75b013fc80 0x7f75b015dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:25.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75ade00640 1 --2- 172.21.15.161:0/2804675864 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75b013cfc0 0x7f75b01402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:25.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75a7e00640 1 -- 172.21.15.161:0/2804675864 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75b013cfc0 msgr2=0x7f75b01402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:25.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75a7e00640 1 --2- 172.21.15.161:0/2804675864 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75b013cfc0 0x7f75b01402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:25.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75a7e00640 1 -- 172.21.15.161:0/2804675864 >> v2:172.21.15.179:3300/0 conn(0x7f75b013fc80 msgr2=0x7f75b015dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:25.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75a7e00640 1 --2- 172.21.15.161:0/2804675864 >> v2:172.21.15.179:3300/0 conn(0x7f75b013fc80 0x7f75b015dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:25.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75a7e00640 1 -- 172.21.15.161:0/2804675864 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b00cdc00 con 0x7f75b015af60 2024-05-22T21:13:25.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75ade00640 1 --2- 172.21.15.161:0/2804675864 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75b013cfc0 0x7f75b01402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:25.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.607+0000 7f75ae800640 1 --2- 172.21.15.161:0/2804675864 >> v2:172.21.15.179:3300/0 conn(0x7f75b013fc80 0x7f75b015dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:25.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.611+0000 7f75a7e00640 1 --2- 172.21.15.161:0/2804675864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75b015af60 0x7f75b013f740 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f75b006de40 tx=0x7f759801c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367e8271fd52c626 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:25.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.611+0000 7f75ad400640 1 -- 172.21.15.161:0/2804675864 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75980ab600 con 0x7f75b015af60 2024-05-22T21:13:25.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.611+0000 7f75ad400640 1 -- 172.21.15.161:0/2804675864 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75980a7050 con 0x7f75b015af60 2024-05-22T21:13:25.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.611+0000 7f75ad400640 1 -- 172.21.15.161:0/2804675864 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75980ab980 con 0x7f75b015af60 2024-05-22T21:13:25.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.611+0000 7f75b4e00640 1 -- 172.21.15.161:0/2804675864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75b015af60 msgr2=0x7f75b013f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:25.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.611+0000 7f75b4e00640 1 --2- 172.21.15.161:0/2804675864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75b015af60 0x7f75b013f740 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f75b006de40 tx=0x7f759801c1a0 comp rx=0 tx=0).stop 2024-05-22T21:13:25.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.611+0000 7f75b4e00640 1 -- 172.21.15.161:0/2804675864 shutdown_connections 2024-05-22T21:13:25.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.611+0000 7f75b4e00640 1 --2- 172.21.15.161:0/2804675864 >> v2:172.21.15.179:3300/0 conn(0x7f75b013fc80 0x7f75b015dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:25.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.611+0000 7f75b4e00640 1 --2- 172.21.15.161:0/2804675864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75b015af60 0x7f75b013f740 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:25.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.611+0000 7f75b4e00640 1 --2- 172.21.15.161:0/2804675864 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75b013cfc0 0x7f75b01402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:25.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.611+0000 7f75b4e00640 1 -- 172.21.15.161:0/2804675864 >> 172.21.15.161:0/2804675864 conn(0x7f75b006a2d0 msgr2=0x7f75b006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:25.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.611+0000 7f75b4e00640 1 -- 172.21.15.161:0/2804675864 shutdown_connections 2024-05-22T21:13:25.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.611+0000 7f75b4e00640 1 -- 172.21.15.161:0/2804675864 wait complete. 2024-05-22T21:13:25.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.611+0000 7f75b4e00640 1 Processor -- start 2024-05-22T21:13:25.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.611+0000 7f75b4e00640 1 -- start start 2024-05-22T21:13:25.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75b4e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75b013cfc0 0x7f75b01560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:25.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75b4e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75b013fc80 0x7f75b01565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:25.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75b4e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75b015af60 0x7f75b013a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:25.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75b4e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b0158100 con 0x7f75b013fc80 2024-05-22T21:13:25.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75b4e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00b4070 con 0x7f75b015af60 2024-05-22T21:13:25.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75b4e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b0159dc0 con 0x7f75b013cfc0 2024-05-22T21:13:25.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75a7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75b013fc80 0x7f75b01565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:25.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75a7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75b013fc80 0x7f75b01565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41416/0 (socket says 172.21.15.161:41416) 2024-05-22T21:13:25.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75a7e00640 1 -- 172.21.15.161:0/626804710 learned_addr learned my addr 172.21.15.161:0/626804710 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:25.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75ade00640 1 --2- 172.21.15.161:0/626804710 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75b013cfc0 0x7f75b01560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:25.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75ae800640 1 --2- 172.21.15.161:0/626804710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75b015af60 0x7f75b013a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:25.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75a7e00640 1 -- 172.21.15.161:0/626804710 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75b013cfc0 msgr2=0x7f75b01560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:25.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75a7e00640 1 --2- 172.21.15.161:0/626804710 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75b013cfc0 0x7f75b01560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:25.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75a7e00640 1 -- 172.21.15.161:0/626804710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75b015af60 msgr2=0x7f75b013a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:25.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75a7e00640 1 --2- 172.21.15.161:0/626804710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75b015af60 0x7f75b013a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:25.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75a7e00640 1 -- 172.21.15.161:0/626804710 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75980094d0 con 0x7f75b013fc80 2024-05-22T21:13:25.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75a7e00640 1 --2- 172.21.15.161:0/626804710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75b013fc80 0x7f75b01565f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f75980ae040 tx=0x7f7598018f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:25.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75a6000640 1 -- 172.21.15.161:0/626804710 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75980ace70 con 0x7f75b013fc80 2024-05-22T21:13:25.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75a6000640 1 -- 172.21.15.161:0/626804710 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75980b6420 con 0x7f75b013fc80 2024-05-22T21:13:25.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75b4e00640 1 -- 172.21.15.161:0/626804710 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b0158350 con 0x7f75b013fc80 2024-05-22T21:13:25.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75a6000640 1 -- 172.21.15.161:0/626804710 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75980b67d0 con 0x7f75b013fc80 2024-05-22T21:13:25.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75b4e00640 1 -- 172.21.15.161:0/626804710 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b0158910 con 0x7f75b013fc80 2024-05-22T21:13:25.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75b4e00640 1 -- 172.21.15.161:0/626804710 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b0067220 con 0x7f75b013fc80 2024-05-22T21:13:25.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.615+0000 7f75a6000640 1 -- 172.21.15.161:0/626804710 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f75980b2040 con 0x7f75b013fc80 2024-05-22T21:13:25.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.619+0000 7f75a6000640 1 --2- 172.21.15.161:0/626804710 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f758c0666c0 0x7f758c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:25.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.619+0000 7f75a6000640 1 -- 172.21.15.161:0/626804710 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f759811e4c0 con 0x7f75b013fc80 2024-05-22T21:13:25.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.619+0000 7f75ade00640 1 --2- 172.21.15.161:0/626804710 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f758c0666c0 0x7f758c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:25.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.619+0000 7f75ade00640 1 --2- 172.21.15.161:0/626804710 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f758c0666c0 0x7f758c068b80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f75b01f70d0 tx=0x7f75a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:25.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.623+0000 7f75a6000640 1 -- 172.21.15.161:0/626804710 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f75980eb790 con 0x7f75b013fc80 2024-05-22T21:13:25.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:25.807+0000 7f75b4e00640 1 -- 172.21.15.161:0/626804710 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f75b0071670 con 0x7f75b013fc80 2024-05-22T21:13:26.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.443+0000 7f75a6000640 1 -- 172.21.15.161:0/626804710 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f75980eedb0 con 0x7f75b013fc80 2024-05-22T21:13:26.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.443+0000 7f75b4e00640 1 -- 172.21.15.161:0/626804710 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f758c0666c0 msgr2=0x7f758c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:26.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.443+0000 7f75b4e00640 1 --2- 172.21.15.161:0/626804710 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f758c0666c0 0x7f758c068b80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f75b01f70d0 tx=0x7f75a0008040 comp rx=0 tx=0).stop 2024-05-22T21:13:26.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.443+0000 7f75b4e00640 1 -- 172.21.15.161:0/626804710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75b013fc80 msgr2=0x7f75b01565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:26.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.443+0000 7f75b4e00640 1 --2- 172.21.15.161:0/626804710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75b013fc80 0x7f75b01565f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f75980ae040 tx=0x7f7598018f40 comp rx=0 tx=0).stop 2024-05-22T21:13:26.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.443+0000 7f75b4e00640 1 -- 172.21.15.161:0/626804710 shutdown_connections 2024-05-22T21:13:26.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.443+0000 7f75b4e00640 1 --2- 172.21.15.161:0/626804710 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f758c0666c0 0x7f758c068b80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.443+0000 7f75b4e00640 1 --2- 172.21.15.161:0/626804710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75b015af60 0x7f75b013a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.443+0000 7f75b4e00640 1 --2- 172.21.15.161:0/626804710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75b013fc80 0x7f75b01565f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.443+0000 7f75b4e00640 1 --2- 172.21.15.161:0/626804710 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75b013cfc0 0x7f75b01560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.443+0000 7f75b4e00640 1 -- 172.21.15.161:0/626804710 >> 172.21.15.161:0/626804710 conn(0x7f75b006a2d0 msgr2=0x7f75b00af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:26.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.443+0000 7f75b4e00640 1 -- 172.21.15.161:0/626804710 shutdown_connections 2024-05-22T21:13:26.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.443+0000 7f75b4e00640 1 -- 172.21.15.161:0/626804710 wait complete. 2024-05-22T21:13:26.468 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-22T21:13:26.468 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:13:26.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.603+0000 7f6d28200640 1 Processor -- start 2024-05-22T21:13:26.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.603+0000 7f6d28200640 1 -- start start 2024-05-22T21:13:26.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.603+0000 7f6d28200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6d20143ec0 0x7f6d2015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:26.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.603+0000 7f6d28200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6d200715a0 0x7f6d2006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:26.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.603+0000 7f6d28200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d201442a0 con 0x7f6d2015ec90 2024-05-22T21:13:26.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.603+0000 7f6d28200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d20070260 con 0x7f6d200715a0 2024-05-22T21:13:26.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.603+0000 7f6d28200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d20070500 con 0x7f6d20143ec0 2024-05-22T21:13:26.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.603+0000 7f6d24e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6d200715a0 0x7f6d2006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:26.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.603+0000 7f6d26200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f6d2015ec90 0x7f6d2006bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56090/0 (socket says 172.21.15.161:56090) 2024-05-22T21:13:26.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.603+0000 7f6d26200640 1 -- 172.21.15.161:0/2835947706 learned_addr learned my addr 172.21.15.161:0/2835947706 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:26.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.603+0000 7f6d24e00640 1 -- 172.21.15.161:0/2835947706 >> v2:172.21.15.179:3300/0 conn(0x7f6d20143ec0 msgr2=0x7f6d2015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.603+0000 7f6d24e00640 1 --2- 172.21.15.161:0/2835947706 >> v2:172.21.15.179:3300/0 conn(0x7f6d20143ec0 0x7f6d2015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.603+0000 7f6d24e00640 1 -- 172.21.15.161:0/2835947706 >> v1:172.21.15.161:6789/0 conn(0x7f6d2015ec90 legacy=0x7f6d2006bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.603+0000 7f6d24e00640 1 -- 172.21.15.161:0/2835947706 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d200707e0 con 0x7f6d200715a0 2024-05-22T21:13:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.607+0000 7f6d24e00640 1 --2- 172.21.15.161:0/2835947706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d200715a0 0x7f6d2006fd20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6d200724e0 tx=0x7f6d0c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da4fad2be0b3c86 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.607+0000 7f6d1fe00640 1 -- 172.21.15.161:0/2835947706 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c0ab650 con 0x7f6d200715a0 2024-05-22T21:13:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.607+0000 7f6d1fe00640 1 -- 172.21.15.161:0/2835947706 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c0ab7b0 con 0x7f6d200715a0 2024-05-22T21:13:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.607+0000 7f6d1fe00640 1 -- 172.21.15.161:0/2835947706 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c0abb40 con 0x7f6d200715a0 2024-05-22T21:13:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.607+0000 7f6d28200640 1 -- 172.21.15.161:0/2835947706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d200715a0 msgr2=0x7f6d2006fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.607+0000 7f6d28200640 1 --2- 172.21.15.161:0/2835947706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d200715a0 0x7f6d2006fd20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6d200724e0 tx=0x7f6d0c01c450 comp rx=0 tx=0).stop 2024-05-22T21:13:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.607+0000 7f6d28200640 1 -- 172.21.15.161:0/2835947706 shutdown_connections 2024-05-22T21:13:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.607+0000 7f6d28200640 1 --2- 172.21.15.161:0/2835947706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d200715a0 0x7f6d2006fd20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.607+0000 7f6d28200640 1 --2- 172.21.15.161:0/2835947706 >> v2:172.21.15.179:3300/0 conn(0x7f6d20143ec0 0x7f6d2015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.607+0000 7f6d28200640 1 -- 172.21.15.161:0/2835947706 >> 172.21.15.161:0/2835947706 conn(0x7f6d200b0bc0 msgr2=0x7f6d200b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:26.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.607+0000 7f6d28200640 1 -- 172.21.15.161:0/2835947706 shutdown_connections 2024-05-22T21:13:26.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.607+0000 7f6d28200640 1 -- 172.21.15.161:0/2835947706 wait complete. 2024-05-22T21:13:26.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d28200640 1 Processor -- start 2024-05-22T21:13:26.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d28200640 1 -- start start 2024-05-22T21:13:26.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d28200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6d20143ec0 0x7f6d200b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:26.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d28200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d2015ec90 0x7f6d2013a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:26.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d28200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d2015a280 0x7f6d20159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:26.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d28200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d200b26b0 con 0x7f6d2015a280 2024-05-22T21:13:26.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d28200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d200adf80 con 0x7f6d2015ec90 2024-05-22T21:13:26.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d28200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d200adca0 con 0x7f6d20143ec0 2024-05-22T21:13:26.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d26200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d2015a280 0x7f6d20159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:26.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d26200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d2015a280 0x7f6d20159580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48902/0 (socket says 172.21.15.161:48902) 2024-05-22T21:13:26.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d26200640 1 -- 172.21.15.161:0/3884232757 learned_addr learned my addr 172.21.15.161:0/3884232757 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:26.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d26200640 1 -- 172.21.15.161:0/3884232757 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6d20143ec0 msgr2=0x7f6d200b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:26.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d26200640 1 --2- 172.21.15.161:0/3884232757 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6d20143ec0 0x7f6d200b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d26200640 1 -- 172.21.15.161:0/3884232757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d2015ec90 msgr2=0x7f6d2013a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:26.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d26200640 1 --2- 172.21.15.161:0/3884232757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d2015ec90 0x7f6d2013a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d26200640 1 -- 172.21.15.161:0/3884232757 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0c009420 con 0x7f6d2015a280 2024-05-22T21:13:26.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d26200640 1 --2- 172.21.15.161:0/3884232757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d2015a280 0x7f6d20159580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800ca80 tx=0x7f6d1800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:26.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d1d600640 1 -- 172.21.15.161:0/3884232757 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1800ac30 con 0x7f6d2015a280 2024-05-22T21:13:26.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d1d600640 1 -- 172.21.15.161:0/3884232757 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1800ad90 con 0x7f6d2015a280 2024-05-22T21:13:26.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d1d600640 1 -- 172.21.15.161:0/3884232757 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18012600 con 0x7f6d2015a280 2024-05-22T21:13:26.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d28200640 1 -- 172.21.15.161:0/3884232757 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d2015f630 con 0x7f6d2015a280 2024-05-22T21:13:26.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.611+0000 7f6d28200640 1 -- 172.21.15.161:0/3884232757 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d200b2340 con 0x7f6d2015a280 2024-05-22T21:13:26.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.615+0000 7f6d28200640 1 -- 172.21.15.161:0/3884232757 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d20067220 con 0x7f6d2015a280 2024-05-22T21:13:26.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.615+0000 7f6d1d600640 1 -- 172.21.15.161:0/3884232757 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6d18007070 con 0x7f6d2015a280 2024-05-22T21:13:26.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.615+0000 7f6d1d600640 1 --2- 172.21.15.161:0/3884232757 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6d000666c0 0x7f6d00068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:26.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.615+0000 7f6d1d600640 1 -- 172.21.15.161:0/3884232757 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d18083fc0 con 0x7f6d2015a280 2024-05-22T21:13:26.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.615+0000 7f6d25800640 1 --2- 172.21.15.161:0/3884232757 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6d000666c0 0x7f6d00068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:26.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.615+0000 7f6d25800640 1 --2- 172.21.15.161:0/3884232757 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6d000666c0 0x7f6d00068b80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009730 tx=0x7f6d14006ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:26.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.619+0000 7f6d1d600640 1 -- 172.21.15.161:0/3884232757 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6d18051340 con 0x7f6d2015a280 2024-05-22T21:13:26.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.803+0000 7f6d28200640 1 -- 172.21.15.161:0/3884232757 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d2006dcc0 con 0x7f6d2015a280 2024-05-22T21:13:26.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.803+0000 7f6d1d600640 1 -- 172.21.15.161:0/3884232757 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4740 (secure 0 0 0) 0x7f6d18017090 con 0x7f6d2015a280 2024-05-22T21:13:26.807 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:26.807 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3179612470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3179612470},{"type":"v1","addr":"172.21.15.179:6834","nonce":3179612470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2196055309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2196055309},{"type":"v1","addr":"172.21.15.161:6835","nonce":2196055309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1793681759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1793681759},{"type":"v1","addr":"172.21.15.161:6837","nonce":1793681759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:13:23.418412+0000","modified":"2024-05-22T21:13:26.442354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/3095599688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3095599688},{"type":"v1","addr":"172.21.15.179:6835","nonce":3095599688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-22T21:13:26.808 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 8 2024-05-22T21:13:26.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.807+0000 7f6d28200640 1 -- 172.21.15.161:0/3884232757 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6d000666c0 msgr2=0x7f6d00068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:26.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.807+0000 7f6d28200640 1 --2- 172.21.15.161:0/3884232757 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6d000666c0 0x7f6d00068b80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009730 tx=0x7f6d14006ca0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.807+0000 7f6d28200640 1 -- 172.21.15.161:0/3884232757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d2015a280 msgr2=0x7f6d20159580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:26.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.807+0000 7f6d28200640 1 --2- 172.21.15.161:0/3884232757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d2015a280 0x7f6d20159580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800ca80 tx=0x7f6d1800cf40 comp rx=0 tx=0).stop 2024-05-22T21:13:26.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.807+0000 7f6d28200640 1 -- 172.21.15.161:0/3884232757 shutdown_connections 2024-05-22T21:13:26.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.807+0000 7f6d28200640 1 --2- 172.21.15.161:0/3884232757 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6d000666c0 0x7f6d00068b80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.807+0000 7f6d28200640 1 --2- 172.21.15.161:0/3884232757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d2015a280 0x7f6d20159580 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.807+0000 7f6d28200640 1 --2- 172.21.15.161:0/3884232757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d2015ec90 0x7f6d2013a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.807+0000 7f6d28200640 1 --2- 172.21.15.161:0/3884232757 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6d20143ec0 0x7f6d200b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.807+0000 7f6d28200640 1 -- 172.21.15.161:0/3884232757 >> 172.21.15.161:0/3884232757 conn(0x7f6d200b0bc0 msgr2=0x7f6d20142110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:26.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.807+0000 7f6d28200640 1 -- 172.21.15.161:0/3884232757 shutdown_connections 2024-05-22T21:13:26.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.807+0000 7f6d28200640 1 -- 172.21.15.161:0/3884232757 wait complete. 2024-05-22T21:13:26.827 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:26.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.959+0000 7f8f61a00640 1 Processor -- start 2024-05-22T21:13:26.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.959+0000 7f8f61a00640 1 -- start start 2024-05-22T21:13:26.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.959+0000 7f8f61a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8f5c15dd50 0x7f8f5c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:26.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.959+0000 7f8f61a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8f5c15e130 0x7f8f5c15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:26.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.959+0000 7f8f61a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f5c15d410 0x7f8f5c15a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:26.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.959+0000 7f8f61a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f5c071670 con 0x7f8f5c15e130 2024-05-22T21:13:26.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.959+0000 7f8f61a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f5c15d8a0 con 0x7f8f5c15dd50 2024-05-22T21:13:26.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.959+0000 7f8f61a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c06de40 con 0x7f8f5c15d410 2024-05-22T21:13:26.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.959+0000 7f8f5a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8f5c15e130 0x7f8f5c15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:26.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.959+0000 7f8f5a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8f5c15e130 0x7f8f5c15ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48916/0 (socket says 172.21.15.161:48916) 2024-05-22T21:13:26.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.959+0000 7f8f5a000640 1 -- 172.21.15.161:0/3318899480 learned_addr learned my addr 172.21.15.161:0/3318899480 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:26.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f5aa00640 1 --2- 172.21.15.161:0/3318899480 >> v2:172.21.15.179:3300/0 conn(0x7f8f5c15dd50 0x7f8f5c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:26.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f5b400640 1 --2- 172.21.15.161:0/3318899480 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f5c15d410 0x7f8f5c15a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:26.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f5a000640 1 -- 172.21.15.161:0/3318899480 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f5c15d410 msgr2=0x7f8f5c15a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:26.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f5a000640 1 --2- 172.21.15.161:0/3318899480 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f5c15d410 0x7f8f5c15a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f5a000640 1 -- 172.21.15.161:0/3318899480 >> v2:172.21.15.179:3300/0 conn(0x7f8f5c15dd50 msgr2=0x7f8f5c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:26.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f5a000640 1 --2- 172.21.15.161:0/3318899480 >> v2:172.21.15.179:3300/0 conn(0x7f8f5c15dd50 0x7f8f5c06d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f5a000640 1 -- 172.21.15.161:0/3318899480 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c070a60 con 0x7f8f5c15e130 2024-05-22T21:13:26.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f5a000640 1 --2- 172.21.15.161:0/3318899480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c15e130 0x7f8f5c15ced0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c072330 tx=0x7f8f4801ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8b91584bbac99d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:26.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f59600640 1 -- 172.21.15.161:0/3318899480 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f480a7c50 con 0x7f8f5c15e130 2024-05-22T21:13:26.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f59600640 1 -- 172.21.15.161:0/3318899480 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f480a7db0 con 0x7f8f5c15e130 2024-05-22T21:13:26.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f59600640 1 -- 172.21.15.161:0/3318899480 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f480af660 con 0x7f8f5c15e130 2024-05-22T21:13:26.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f61a00640 1 -- 172.21.15.161:0/3318899480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c15e130 msgr2=0x7f8f5c15ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:26.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f61a00640 1 --2- 172.21.15.161:0/3318899480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c15e130 0x7f8f5c15ced0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c072330 tx=0x7f8f4801ebc0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f61a00640 1 -- 172.21.15.161:0/3318899480 shutdown_connections 2024-05-22T21:13:26.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f61a00640 1 --2- 172.21.15.161:0/3318899480 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f5c15d410 0x7f8f5c15a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f61a00640 1 --2- 172.21.15.161:0/3318899480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c15e130 0x7f8f5c15ced0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f61a00640 1 --2- 172.21.15.161:0/3318899480 >> v2:172.21.15.179:3300/0 conn(0x7f8f5c15dd50 0x7f8f5c06d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f61a00640 1 -- 172.21.15.161:0/3318899480 >> 172.21.15.161:0/3318899480 conn(0x7f8f5c0b0bc0 msgr2=0x7f8f5c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:26.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.963+0000 7f8f61a00640 1 -- 172.21.15.161:0/3318899480 shutdown_connections 2024-05-22T21:13:26.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f61a00640 1 -- 172.21.15.161:0/3318899480 wait complete. 2024-05-22T21:13:26.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f61a00640 1 Processor -- start 2024-05-22T21:13:26.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f61a00640 1 -- start start 2024-05-22T21:13:26.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f61a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f5c15d410 0x7f8f5c0b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:26.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f61a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f5c15dd50 0x7f8f5c1542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:26.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f61a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c15e130 0x7f8f5c156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:26.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f61a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c15f630 con 0x7f8f5c15e130 2024-05-22T21:13:26.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f61a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c15fac0 con 0x7f8f5c15dd50 2024-05-22T21:13:26.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f61a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c0cdc40 con 0x7f8f5c15d410 2024-05-22T21:13:26.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f5b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c15e130 0x7f8f5c156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:26.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f5aa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f5c15d410 0x7f8f5c0b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:26.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f5b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c15e130 0x7f8f5c156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48930/0 (socket says 172.21.15.161:48930) 2024-05-22T21:13:26.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f5b400640 1 -- 172.21.15.161:0/1468898528 learned_addr learned my addr 172.21.15.161:0/1468898528 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:26.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f5a000640 1 --2- 172.21.15.161:0/1468898528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f5c15dd50 0x7f8f5c1542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:26.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f5b400640 1 -- 172.21.15.161:0/1468898528 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f5c15d410 msgr2=0x7f8f5c0b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:26.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f5b400640 1 --2- 172.21.15.161:0/1468898528 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f5c15d410 0x7f8f5c0b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f5b400640 1 -- 172.21.15.161:0/1468898528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f5c15dd50 msgr2=0x7f8f5c1542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:26.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f5b400640 1 --2- 172.21.15.161:0/1468898528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f5c15dd50 0x7f8f5c1542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:26.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f5b400640 1 -- 172.21.15.161:0/1468898528 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4800b770 con 0x7f8f5c15e130 2024-05-22T21:13:26.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f5b400640 1 --2- 172.21.15.161:0/1468898528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c15e130 0x7f8f5c156580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f54007e40 tx=0x7f8f5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:26.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f52a00640 1 -- 172.21.15.161:0/1468898528 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54014620 con 0x7f8f5c15e130 2024-05-22T21:13:26.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f52a00640 1 -- 172.21.15.161:0/1468898528 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5400ee90 con 0x7f8f5c15e130 2024-05-22T21:13:26.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f52a00640 1 -- 172.21.15.161:0/1468898528 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f540149e0 con 0x7f8f5c15e130 2024-05-22T21:13:26.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.967+0000 7f8f61a00640 1 -- 172.21.15.161:0/1468898528 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5c06d780 con 0x7f8f5c15e130 2024-05-22T21:13:26.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.971+0000 7f8f61a00640 1 -- 172.21.15.161:0/1468898528 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5c154a30 con 0x7f8f5c15e130 2024-05-22T21:13:26.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.971+0000 7f8f61a00640 1 -- 172.21.15.161:0/1468898528 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f5c15f400 con 0x7f8f5c15e130 2024-05-22T21:13:26.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.971+0000 7f8f52a00640 1 -- 172.21.15.161:0/1468898528 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8f54014b40 con 0x7f8f5c15e130 2024-05-22T21:13:26.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.971+0000 7f8f52a00640 1 --2- 172.21.15.161:0/1468898528 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8f3c0666e0 0x7f8f3c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:26.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.971+0000 7f8f52a00640 1 -- 172.21.15.161:0/1468898528 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f54084a10 con 0x7f8f5c15e130 2024-05-22T21:13:26.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.971+0000 7f8f5aa00640 1 --2- 172.21.15.161:0/1468898528 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8f3c0666e0 0x7f8f3c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:26.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.971+0000 7f8f5aa00640 1 --2- 172.21.15.161:0/1468898528 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8f3c0666e0 0x7f8f3c068ba0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0079e0 tx=0x7f8f4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:26.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:26.975+0000 7f8f52a00640 1 -- 172.21.15.161:0/1468898528 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8f5c15f400 con 0x7f8f5c15e130 2024-05-22T21:13:27.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.131+0000 7f8f61a00640 1 -- 172.21.15.161:0/1468898528 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f5c15f400 con 0x7f8f5c15e130 2024-05-22T21:13:27.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.131+0000 7f8f52a00640 1 -- 172.21.15.161:0/1468898528 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f8f5c15f400 con 0x7f8f5c15e130 2024-05-22T21:13:27.135 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:27.136 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:27.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.135+0000 7f8f61a00640 1 -- 172.21.15.161:0/1468898528 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8f3c0666e0 msgr2=0x7f8f3c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:27.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.135+0000 7f8f61a00640 1 --2- 172.21.15.161:0/1468898528 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8f3c0666e0 0x7f8f3c068ba0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0079e0 tx=0x7f8f4c008040 comp rx=0 tx=0).stop 2024-05-22T21:13:27.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.135+0000 7f8f61a00640 1 -- 172.21.15.161:0/1468898528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c15e130 msgr2=0x7f8f5c156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:27.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.135+0000 7f8f61a00640 1 --2- 172.21.15.161:0/1468898528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c15e130 0x7f8f5c156580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f54007e40 tx=0x7f8f5400e4b0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.135+0000 7f8f61a00640 1 -- 172.21.15.161:0/1468898528 shutdown_connections 2024-05-22T21:13:27.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.135+0000 7f8f61a00640 1 --2- 172.21.15.161:0/1468898528 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8f3c0666e0 0x7f8f3c068ba0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.135+0000 7f8f61a00640 1 --2- 172.21.15.161:0/1468898528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c15e130 0x7f8f5c156580 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.135+0000 7f8f61a00640 1 --2- 172.21.15.161:0/1468898528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f5c15dd50 0x7f8f5c1542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.135+0000 7f8f61a00640 1 --2- 172.21.15.161:0/1468898528 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f5c15d410 0x7f8f5c0b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.135+0000 7f8f61a00640 1 -- 172.21.15.161:0/1468898528 >> 172.21.15.161:0/1468898528 conn(0x7f8f5c0b0bc0 msgr2=0x7f8f5c0ad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:27.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.135+0000 7f8f61a00640 1 -- 172.21.15.161:0/1468898528 shutdown_connections 2024-05-22T21:13:27.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.135+0000 7f8f61a00640 1 -- 172.21.15.161:0/1468898528 wait complete. 2024-05-22T21:13:27.157 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-22T21:13:27.157 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:13:27.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.291+0000 7f7698400640 1 Processor -- start 2024-05-22T21:13:27.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.291+0000 7f7698400640 1 -- start start 2024-05-22T21:13:27.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.291+0000 7f7698400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7690143ec0 0x7f769015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:27.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.291+0000 7f7698400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f76900715a0 0x7f769006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:27.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.291+0000 7f7698400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f769015ec90 0x7f769015f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:27.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.291+0000 7f7698400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769015f660 con 0x7f7690143ec0 2024-05-22T21:13:27.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.291+0000 7f7698400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76901442a0 con 0x7f76900715a0 2024-05-22T21:13:27.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.291+0000 7f7698400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7690070260 con 0x7f769015ec90 2024-05-22T21:13:27.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.291+0000 7f7695a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7690143ec0 0x7f769015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:27.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.291+0000 7f7695a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7690143ec0 0x7f769015f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48936/0 (socket says 172.21.15.161:48936) 2024-05-22T21:13:27.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.291+0000 7f7695a00640 1 -- 172.21.15.161:0/2339708114 learned_addr learned my addr 172.21.15.161:0/2339708114 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:27.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.291+0000 7f7696400640 1 --2- 172.21.15.161:0/2339708114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f769015ec90 0x7f769015f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:27.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.291+0000 7f7695000640 1 --2- 172.21.15.161:0/2339708114 >> v2:172.21.15.179:3300/0 conn(0x7f76900715a0 0x7f769006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:27.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.295+0000 7f7695a00640 1 -- 172.21.15.161:0/2339708114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f769015ec90 msgr2=0x7f769015f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:27.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.295+0000 7f7695a00640 1 --2- 172.21.15.161:0/2339708114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f769015ec90 0x7f769015f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.295+0000 7f7695a00640 1 -- 172.21.15.161:0/2339708114 >> v2:172.21.15.179:3300/0 conn(0x7f76900715a0 msgr2=0x7f769006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:27.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.295+0000 7f7695a00640 1 --2- 172.21.15.161:0/2339708114 >> v2:172.21.15.179:3300/0 conn(0x7f76900715a0 0x7f769006fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.295+0000 7f7695a00640 1 -- 172.21.15.161:0/2339708114 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690070500 con 0x7f7690143ec0 2024-05-22T21:13:27.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.295+0000 7f7695000640 1 --2- 172.21.15.161:0/2339708114 >> v2:172.21.15.179:3300/0 conn(0x7f76900715a0 0x7f769006fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:27.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.295+0000 7f7695a00640 1 --2- 172.21.15.161:0/2339708114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7690143ec0 0x7f769015f830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7690074430 tx=0x7f768401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a7b8307fbdc110 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:27.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.295+0000 7f768fe00640 1 -- 172.21.15.161:0/2339708114 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76840aa610 con 0x7f7690143ec0 2024-05-22T21:13:27.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.295+0000 7f768fe00640 1 -- 172.21.15.161:0/2339708114 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76840a6040 con 0x7f7690143ec0 2024-05-22T21:13:27.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.295+0000 7f768fe00640 1 -- 172.21.15.161:0/2339708114 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76840aa990 con 0x7f7690143ec0 2024-05-22T21:13:27.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.295+0000 7f7698400640 1 -- 172.21.15.161:0/2339708114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7690143ec0 msgr2=0x7f769015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:27.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.295+0000 7f7698400640 1 --2- 172.21.15.161:0/2339708114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7690143ec0 0x7f769015f830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7690074430 tx=0x7f768401c4f0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7698400640 1 -- 172.21.15.161:0/2339708114 shutdown_connections 2024-05-22T21:13:27.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7698400640 1 --2- 172.21.15.161:0/2339708114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f769015ec90 0x7f769015f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7698400640 1 --2- 172.21.15.161:0/2339708114 >> v2:172.21.15.179:3300/0 conn(0x7f76900715a0 0x7f769006fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7698400640 1 --2- 172.21.15.161:0/2339708114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7690143ec0 0x7f769015f830 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7698400640 1 -- 172.21.15.161:0/2339708114 >> 172.21.15.161:0/2339708114 conn(0x7f76900b0bc0 msgr2=0x7f76900b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:27.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7698400640 1 -- 172.21.15.161:0/2339708114 shutdown_connections 2024-05-22T21:13:27.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7698400640 1 -- 172.21.15.161:0/2339708114 wait complete. 2024-05-22T21:13:27.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7698400640 1 Processor -- start 2024-05-22T21:13:27.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7698400640 1 -- start start 2024-05-22T21:13:27.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7698400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7690143ec0 0x7f76900b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:27.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7698400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f769015ec90 0x7f7690156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:27.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7698400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76900adc80 0x7f7690158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:27.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7698400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76900b0a50 con 0x7f7690143ec0 2024-05-22T21:13:27.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7698400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76900afcf0 con 0x7f769015ec90 2024-05-22T21:13:27.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7698400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76900b3430 con 0x7f76900adc80 2024-05-22T21:13:27.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7695a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7690143ec0 0x7f76900b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:27.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7695a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7690143ec0 0x7f76900b0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48944/0 (socket says 172.21.15.161:48944) 2024-05-22T21:13:27.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7695a00640 1 -- 172.21.15.161:0/4120901537 learned_addr learned my addr 172.21.15.161:0/4120901537 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:27.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7695000640 1 --2- 172.21.15.161:0/4120901537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f769015ec90 0x7f7690156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:27.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7696400640 1 --2- 172.21.15.161:0/4120901537 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76900adc80 0x7f7690158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:27.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7695a00640 1 -- 172.21.15.161:0/4120901537 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76900adc80 msgr2=0x7f7690158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:27.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7695a00640 1 --2- 172.21.15.161:0/4120901537 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76900adc80 0x7f7690158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7695a00640 1 -- 172.21.15.161:0/4120901537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f769015ec90 msgr2=0x7f7690156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:27.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7695a00640 1 --2- 172.21.15.161:0/4120901537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f769015ec90 0x7f7690156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7695a00640 1 -- 172.21.15.161:0/4120901537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76840094d0 con 0x7f7690143ec0 2024-05-22T21:13:27.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.299+0000 7f7695a00640 1 --2- 172.21.15.161:0/4120901537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7690143ec0 0x7f76900b0510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f768401aba0 tx=0x7f768401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:27.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.303+0000 7f768d600640 1 -- 172.21.15.161:0/4120901537 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76840bd5f0 con 0x7f7690143ec0 2024-05-22T21:13:27.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.303+0000 7f768d600640 1 -- 172.21.15.161:0/4120901537 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76840bd750 con 0x7f7690143ec0 2024-05-22T21:13:27.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.303+0000 7f768d600640 1 -- 172.21.15.161:0/4120901537 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76840bdb00 con 0x7f7690143ec0 2024-05-22T21:13:27.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.303+0000 7f7698400640 1 -- 172.21.15.161:0/4120901537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76900ae1b0 con 0x7f7690143ec0 2024-05-22T21:13:27.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.303+0000 7f7698400640 1 -- 172.21.15.161:0/4120901537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76900b2b00 con 0x7f7690143ec0 2024-05-22T21:13:27.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.303+0000 7f7698400640 1 -- 172.21.15.161:0/4120901537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7690067220 con 0x7f7690143ec0 2024-05-22T21:13:27.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.303+0000 7f768d600640 1 -- 172.21.15.161:0/4120901537 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f76840b33f0 con 0x7f7690143ec0 2024-05-22T21:13:27.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.303+0000 7f768d600640 1 --2- 172.21.15.161:0/4120901537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76700666c0 0x7f7670068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:27.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.303+0000 7f768d600640 1 -- 172.21.15.161:0/4120901537 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f768411ee20 con 0x7f7690143ec0 2024-05-22T21:13:27.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.303+0000 7f7695000640 1 --2- 172.21.15.161:0/4120901537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76700666c0 0x7f7670068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:27.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.303+0000 7f7695000640 1 --2- 172.21.15.161:0/4120901537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76700666c0 0x7f7670068b80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f769015eb60 tx=0x7f767c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:27.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.307+0000 7f768d600640 1 -- 172.21.15.161:0/4120901537 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f76840ec230 con 0x7f7690143ec0 2024-05-22T21:13:27.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.459+0000 7f7698400640 1 -- 172.21.15.161:0/4120901537 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7690071080 con 0x7f76700666c0 2024-05-22T21:13:27.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.467+0000 7f768d600640 1 -- 172.21.15.161:0/4120901537 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248992 (secure 0 0 0) 0x7f767c00a670 con 0x7f76700666c0 2024-05-22T21:13:27.471 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:27.474 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:13:27.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.471+0000 7f7698400640 1 -- 172.21.15.161:0/4120901537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76700666c0 msgr2=0x7f7670068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:27.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.471+0000 7f7698400640 1 --2- 172.21.15.161:0/4120901537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76700666c0 0x7f7670068b80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f769015eb60 tx=0x7f767c008040 comp rx=0 tx=0).stop 2024-05-22T21:13:27.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.471+0000 7f7698400640 1 -- 172.21.15.161:0/4120901537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7690143ec0 msgr2=0x7f76900b0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:27.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.471+0000 7f7698400640 1 --2- 172.21.15.161:0/4120901537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7690143ec0 0x7f76900b0510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f768401aba0 tx=0x7f768401ad30 comp rx=0 tx=0).stop 2024-05-22T21:13:27.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.471+0000 7f7698400640 1 -- 172.21.15.161:0/4120901537 shutdown_connections 2024-05-22T21:13:27.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.471+0000 7f7698400640 1 --2- 172.21.15.161:0/4120901537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76700666c0 0x7f7670068b80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.471+0000 7f7698400640 1 --2- 172.21.15.161:0/4120901537 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76900adc80 0x7f7690158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.471+0000 7f7698400640 1 --2- 172.21.15.161:0/4120901537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f769015ec90 0x7f7690156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.471+0000 7f7698400640 1 --2- 172.21.15.161:0/4120901537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7690143ec0 0x7f76900b0510 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.471+0000 7f7698400640 1 -- 172.21.15.161:0/4120901537 >> 172.21.15.161:0/4120901537 conn(0x7f76900b0bc0 msgr2=0x7f7690142e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:27.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.471+0000 7f7698400640 1 -- 172.21.15.161:0/4120901537 shutdown_connections 2024-05-22T21:13:27.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.471+0000 7f7698400640 1 -- 172.21.15.161:0/4120901537 wait complete. 2024-05-22T21:13:27.494 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-05-22T21:13:26.392477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1544,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560955,"statfs":{"total":1600000000,"available":1598417947,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.359966"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-22T21:13:15.003438+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:13:15.003438+0000","last_peered":"2024-05-22T21:13:15.003438+0000","last_clean":"2024-05-22T21:13:15.003438+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:13:15.003438+0000","last_undegraded":"2024-05-22T21:13:15.003438+0000","last_fullsized":"2024-05-22T21:13:15.003438+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359390,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":624,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194687,"statfs":{"total":200000000,"available":199360009,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]}],"pool_statfs":[]}} 2024-05-22T21:13:27.498 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:13:27.597 INFO:tasks.ceph.mds.b.smithi179.stderr:2024-05-22T21:13:27.591+0000 7f4c24c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T21:13:27.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.671+0000 7f1e9b400640 1 Processor -- start 2024-05-22T21:13:27.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.671+0000 7f1e9b400640 1 -- start start 2024-05-22T21:13:27.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.671+0000 7f1e9b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1e9415d430 0x7f1e940707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:27.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.671+0000 7f1e9b400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e940accf0 0x7f1e94160e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:27.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.671+0000 7f1e9b400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e94071fd0 con 0x7f1e940ac8b0 2024-05-22T21:13:27.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.671+0000 7f1e9b400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e94070e80 con 0x7f1e9415d430 2024-05-22T21:13:27.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.671+0000 7f1e9b400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9406c510 con 0x7f1e940accf0 2024-05-22T21:13:27.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.671+0000 7f1e98a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1e9415d430 0x7f1e940707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:27.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.671+0000 7f1e98a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1e9415d430 0x7f1e940707c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48948/0 (socket says 172.21.15.161:48948) 2024-05-22T21:13:27.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.671+0000 7f1e98a00640 1 -- 172.21.15.161:0/3648764270 learned_addr learned my addr 172.21.15.161:0/3648764270 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:27.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.671+0000 7f1e93e00640 1 --1- 172.21.15.161:0/3648764270 >> v1:172.21.15.161:6789/0 conn(0x7f1e940ac8b0 0x7f1e9406cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56096/0 (socket says 172.21.15.161:56096) 2024-05-22T21:13:27.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.671+0000 7f1e99400640 1 --2- 172.21.15.161:0/3648764270 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e940accf0 0x7f1e94160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:27.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.671+0000 7f1e98a00640 1 -- 172.21.15.161:0/3648764270 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e940accf0 msgr2=0x7f1e94160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:27.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.671+0000 7f1e98a00640 1 --2- 172.21.15.161:0/3648764270 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e940accf0 0x7f1e94160e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.675+0000 7f1e98a00640 1 -- 172.21.15.161:0/3648764270 >> v1:172.21.15.161:6789/0 conn(0x7f1e940ac8b0 legacy=0x7f1e9406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:27.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.675+0000 7f1e93400640 1 -- 172.21.15.161:0/3648764270 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430478744 0 0) 0x7f1e94071fd0 con 0x7f1e940ac8b0 2024-05-22T21:13:27.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.675+0000 7f1e98a00640 1 -- 172.21.15.161:0/3648764270 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9406c7f0 con 0x7f1e9415d430 2024-05-22T21:13:27.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.675+0000 7f1e98a00640 1 --2- 172.21.15.161:0/3648764270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9415d430 0x7f1e940707c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009e50 tx=0x7f1e8401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80476c86a1308181 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:27.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.675+0000 7f1e93400640 1 -- 172.21.15.161:0/3648764270 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e840a5740 con 0x7f1e9415d430 2024-05-22T21:13:27.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.675+0000 7f1e93400640 1 -- 172.21.15.161:0/3648764270 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e840a58a0 con 0x7f1e9415d430 2024-05-22T21:13:27.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.675+0000 7f1e93400640 1 -- 172.21.15.161:0/3648764270 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e840a5c30 con 0x7f1e9415d430 2024-05-22T21:13:27.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.675+0000 7f1e9b400640 1 -- 172.21.15.161:0/3648764270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9415d430 msgr2=0x7f1e940707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:27.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.675+0000 7f1e9b400640 1 --2- 172.21.15.161:0/3648764270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9415d430 0x7f1e940707c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009e50 tx=0x7f1e8401c970 comp rx=0 tx=0).stop 2024-05-22T21:13:27.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.675+0000 7f1e9b400640 1 -- 172.21.15.161:0/3648764270 shutdown_connections 2024-05-22T21:13:27.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.675+0000 7f1e9b400640 1 --2- 172.21.15.161:0/3648764270 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e940accf0 0x7f1e94160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.675+0000 7f1e9b400640 1 --2- 172.21.15.161:0/3648764270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9415d430 0x7f1e940707c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.675+0000 7f1e9b400640 1 -- 172.21.15.161:0/3648764270 >> 172.21.15.161:0/3648764270 conn(0x7f1e940b0bc0 msgr2=0x7f1e940b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:27.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.675+0000 7f1e9b400640 1 -- 172.21.15.161:0/3648764270 shutdown_connections 2024-05-22T21:13:27.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.675+0000 7f1e9b400640 1 -- 172.21.15.161:0/3648764270 wait complete. 2024-05-22T21:13:27.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e9b400640 1 Processor -- start 2024-05-22T21:13:27.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e9b400640 1 -- start start 2024-05-22T21:13:27.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e9b400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e940ac8b0 0x7f1e9415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:27.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e9b400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e940accf0 0x7f1e94146810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:27.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e9b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9415d430 0x7f1e941f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:27.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e9b400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9415f920 con 0x7f1e9415d430 2024-05-22T21:13:27.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e9b400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9415eca0 con 0x7f1e940accf0 2024-05-22T21:13:27.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e9b400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9415ef80 con 0x7f1e940ac8b0 2024-05-22T21:13:27.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e99400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9415d430 0x7f1e941f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:27.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e99400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9415d430 0x7f1e941f6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48954/0 (socket says 172.21.15.161:48954) 2024-05-22T21:13:27.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e99400640 1 -- 172.21.15.161:0/3822115769 learned_addr learned my addr 172.21.15.161:0/3822115769 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:27.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e93e00640 1 --2- 172.21.15.161:0/3822115769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e940accf0 0x7f1e94146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:27.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e98a00640 1 --2- 172.21.15.161:0/3822115769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e940ac8b0 0x7f1e9415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:27.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e99400640 1 -- 172.21.15.161:0/3822115769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e940ac8b0 msgr2=0x7f1e9415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:27.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e99400640 1 --2- 172.21.15.161:0/3822115769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e940ac8b0 0x7f1e9415fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e99400640 1 -- 172.21.15.161:0/3822115769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e940accf0 msgr2=0x7f1e94146810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:27.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e99400640 1 --2- 172.21.15.161:0/3822115769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e940accf0 0x7f1e94146810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e99400640 1 -- 172.21.15.161:0/3822115769 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e84009b30 con 0x7f1e9415d430 2024-05-22T21:13:27.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e93e00640 1 --2- 172.21.15.161:0/3822115769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e940accf0 0x7f1e94146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:27.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e98a00640 1 --2- 172.21.15.161:0/3822115769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e940ac8b0 0x7f1e9415fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:27.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e99400640 1 --2- 172.21.15.161:0/3822115769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9415d430 0x7f1e941f6630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00d500 tx=0x7f1e8c00d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:27.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e90c00640 1 -- 172.21.15.161:0/3822115769 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c0125f0 con 0x7f1e9415d430 2024-05-22T21:13:27.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e90c00640 1 -- 172.21.15.161:0/3822115769 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8c012750 con 0x7f1e9415d430 2024-05-22T21:13:27.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e90c00640 1 -- 172.21.15.161:0/3822115769 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c012b00 con 0x7f1e9415d430 2024-05-22T21:13:27.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e9b400640 1 -- 172.21.15.161:0/3822115769 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9415f260 con 0x7f1e9415d430 2024-05-22T21:13:27.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.679+0000 7f1e9b400640 1 -- 172.21.15.161:0/3822115769 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e94160280 con 0x7f1e9415d430 2024-05-22T21:13:27.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.683+0000 7f1e9b400640 1 -- 172.21.15.161:0/3822115769 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e9415fa60 con 0x7f1e9415d430 2024-05-22T21:13:27.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.683+0000 7f1e90c00640 1 -- 172.21.15.161:0/3822115769 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1e8c012c60 con 0x7f1e9415d430 2024-05-22T21:13:27.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.683+0000 7f1e90c00640 1 --2- 172.21.15.161:0/3822115769 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e74066690 0x7f1e74068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:27.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.683+0000 7f1e90c00640 1 -- 172.21.15.161:0/3822115769 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e8c0843a0 con 0x7f1e9415d430 2024-05-22T21:13:27.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.683+0000 7f1e98a00640 1 --2- 172.21.15.161:0/3822115769 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e74066690 0x7f1e74068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:27.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.683+0000 7f1e98a00640 1 --2- 172.21.15.161:0/3822115769 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e74066690 0x7f1e74068b50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1e840ae000 tx=0x7f1e840a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:27.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.687+0000 7f1e90c00640 1 -- 172.21.15.161:0/3822115769 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1e9415fa60 con 0x7f1e9415d430 2024-05-22T21:13:27.838 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.835+0000 7f1e9b400640 1 -- 172.21.15.161:0/3822115769 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e9415fa60 con 0x7f1e74066690 2024-05-22T21:13:27.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.843+0000 7f1e90c00640 1 -- 172.21.15.161:0/3822115769 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248992 (secure 0 0 0) 0x7f1e9415fa60 con 0x7f1e74066690 2024-05-22T21:13:27.848 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:27.849 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:13:27.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.847+0000 7f1e9b400640 1 -- 172.21.15.161:0/3822115769 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e74066690 msgr2=0x7f1e74068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:27.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.847+0000 7f1e9b400640 1 --2- 172.21.15.161:0/3822115769 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e74066690 0x7f1e74068b50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1e840ae000 tx=0x7f1e840a7070 comp rx=0 tx=0).stop 2024-05-22T21:13:27.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.847+0000 7f1e9b400640 1 -- 172.21.15.161:0/3822115769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9415d430 msgr2=0x7f1e941f6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:27.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.847+0000 7f1e9b400640 1 --2- 172.21.15.161:0/3822115769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9415d430 0x7f1e941f6630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00d500 tx=0x7f1e8c00d9c0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.847+0000 7f1e9b400640 1 -- 172.21.15.161:0/3822115769 shutdown_connections 2024-05-22T21:13:27.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.847+0000 7f1e9b400640 1 --2- 172.21.15.161:0/3822115769 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e74066690 0x7f1e74068b50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.847+0000 7f1e9b400640 1 --2- 172.21.15.161:0/3822115769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e9415d430 0x7f1e941f6630 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.847+0000 7f1e9b400640 1 --2- 172.21.15.161:0/3822115769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e940accf0 0x7f1e94146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.847+0000 7f1e9b400640 1 --2- 172.21.15.161:0/3822115769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e940ac8b0 0x7f1e9415fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:27.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.847+0000 7f1e9b400640 1 -- 172.21.15.161:0/3822115769 >> 172.21.15.161:0/3822115769 conn(0x7f1e940b0bc0 msgr2=0x7f1e940adea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:27.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.847+0000 7f1e9b400640 1 -- 172.21.15.161:0/3822115769 shutdown_connections 2024-05-22T21:13:27.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:27.847+0000 7f1e9b400640 1 -- 172.21.15.161:0/3822115769 wait complete. 2024-05-22T21:13:27.868 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-05-22T21:13:26.392477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1544,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560955,"statfs":{"total":1600000000,"available":1598417947,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.359966"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-22T21:13:15.003438+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:13:15.003438+0000","last_peered":"2024-05-22T21:13:15.003438+0000","last_clean":"2024-05-22T21:13:15.003438+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:13:15.003438+0000","last_undegraded":"2024-05-22T21:13:15.003438+0000","last_fullsized":"2024-05-22T21:13:15.003438+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359390,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":624,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194687,"statfs":{"total":200000000,"available":199360009,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]}],"pool_statfs":[]}} 2024-05-22T21:13:27.872 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:13:28.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.043+0000 7f5bdae00640 1 Processor -- start 2024-05-22T21:13:28.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.043+0000 7f5bdae00640 1 -- start start 2024-05-22T21:13:28.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.043+0000 7f5bdae00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5bd40715a0 0x7f5bd415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.043+0000 7f5bdae00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd41442a0 con 0x7f5bd4143ec0 2024-05-22T21:13:28.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.043+0000 7f5bdae00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd4070500 con 0x7f5bd415ec90 2024-05-22T21:13:28.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.043+0000 7f5bdae00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd40707e0 con 0x7f5bd40715a0 2024-05-22T21:13:28.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.043+0000 7f5bd3e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f5bd4143ec0 0x7f5bd406fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56110/0 (socket says 172.21.15.161:56110) 2024-05-22T21:13:28.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.043+0000 7f5bd3e00640 1 -- 172.21.15.161:0/2816266839 learned_addr learned my addr 172.21.15.161:0/2816266839 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:28.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.043+0000 7f5bcb400640 1 --2- 172.21.15.161:0/2816266839 >> v2:172.21.15.179:3300/0 conn(0x7f5bd40715a0 0x7f5bd415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.043+0000 7f5bd3400640 1 -- 172.21.15.161:0/2816266839 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2868538280 0 0) 0x7f5bd41442a0 con 0x7f5bd4143ec0 2024-05-22T21:13:28.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.043+0000 7f5bd3400640 1 -- 172.21.15.161:0/2816266839 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bb4003530 con 0x7f5bd4143ec0 2024-05-22T21:13:28.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.043+0000 7f5bcb400640 1 -- 172.21.15.161:0/2816266839 >> v1:172.21.15.179:6789/0 conn(0x7f5bd415ec90 legacy=0x7f5bd40cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.043+0000 7f5bd3400640 1 -- 172.21.15.161:0/2816266839 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2075489011 0 0) 0x7f5bb4003530 con 0x7f5bd4143ec0 2024-05-22T21:13:28.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.043+0000 7f5bcb400640 1 -- 172.21.15.161:0/2816266839 >> v1:172.21.15.161:6789/0 conn(0x7f5bd4143ec0 legacy=0x7f5bd406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.043+0000 7f5bcb400640 1 -- 172.21.15.161:0/2816266839 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd4070ac0 con 0x7f5bd40715a0 2024-05-22T21:13:28.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.047+0000 7f5bcb400640 1 --2- 172.21.15.161:0/2816266839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bd40715a0 0x7f5bd415f830 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0097c0 tx=0x7f5bbc01c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1ac845323977c6e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:28.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.047+0000 7f5bd3400640 1 -- 172.21.15.161:0/2816266839 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4001680 con 0x7f5bd40715a0 2024-05-22T21:13:28.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.047+0000 7f5bd3400640 1 -- 172.21.15.161:0/2816266839 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bbc0ac440 con 0x7f5bd40715a0 2024-05-22T21:13:28.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.047+0000 7f5bd3400640 1 -- 172.21.15.161:0/2816266839 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc0ac7d0 con 0x7f5bd40715a0 2024-05-22T21:13:28.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.047+0000 7f5bdae00640 1 -- 172.21.15.161:0/2816266839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bd40715a0 msgr2=0x7f5bd415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.047+0000 7f5bdae00640 1 --2- 172.21.15.161:0/2816266839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bd40715a0 0x7f5bd415f830 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0097c0 tx=0x7f5bbc01c390 comp rx=0 tx=0).stop 2024-05-22T21:13:28.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.047+0000 7f5bdae00640 1 -- 172.21.15.161:0/2816266839 shutdown_connections 2024-05-22T21:13:28.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.047+0000 7f5bdae00640 1 --2- 172.21.15.161:0/2816266839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bd40715a0 0x7f5bd415f830 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.047+0000 7f5bdae00640 1 -- 172.21.15.161:0/2816266839 >> 172.21.15.161:0/2816266839 conn(0x7f5bd40b0bc0 msgr2=0x7f5bd40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:28.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.047+0000 7f5bdae00640 1 -- 172.21.15.161:0/2816266839 shutdown_connections 2024-05-22T21:13:28.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.047+0000 7f5bdae00640 1 -- 172.21.15.161:0/2816266839 wait complete. 2024-05-22T21:13:28.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bdae00640 1 Processor -- start 2024-05-22T21:13:28.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bdae00640 1 -- start start 2024-05-22T21:13:28.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bdae00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bd4143ec0 0x7f5bd413f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bdae00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bd415ec90 0x7f5bd413a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bdae00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5bd415a240 0x7f5bd4159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bdae00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd413ffe0 con 0x7f5bd4143ec0 2024-05-22T21:13:28.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bdae00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd41402c0 con 0x7f5bd415ec90 2024-05-22T21:13:28.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bdae00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd41405a0 con 0x7f5bd415a240 2024-05-22T21:13:28.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bd3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bd4143ec0 0x7f5bd413f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bd3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bd4143ec0 0x7f5bd413f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48964/0 (socket says 172.21.15.161:48964) 2024-05-22T21:13:28.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bd3e00640 1 -- 172.21.15.161:0/4080566165 learned_addr learned my addr 172.21.15.161:0/4080566165 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:28.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bcb400640 1 --2- 172.21.15.161:0/4080566165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bd415ec90 0x7f5bd413a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bd8e00640 1 --2- 172.21.15.161:0/4080566165 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5bd415a240 0x7f5bd4159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bd3e00640 1 -- 172.21.15.161:0/4080566165 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5bd415a240 msgr2=0x7f5bd4159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bd3e00640 1 --2- 172.21.15.161:0/4080566165 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5bd415a240 0x7f5bd4159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bd3e00640 1 -- 172.21.15.161:0/4080566165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bd415ec90 msgr2=0x7f5bd413a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bd3e00640 1 --2- 172.21.15.161:0/4080566165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bd415ec90 0x7f5bd413a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bd3e00640 1 -- 172.21.15.161:0/4080566165 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc009420 con 0x7f5bd4143ec0 2024-05-22T21:13:28.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bd3e00640 1 --2- 172.21.15.161:0/4080566165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bd4143ec0 0x7f5bd413f790 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400c760 tx=0x7f5bc400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:28.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bd0c00640 1 -- 172.21.15.161:0/4080566165 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4005ae0 con 0x7f5bd4143ec0 2024-05-22T21:13:28.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bd0c00640 1 -- 172.21.15.161:0/4080566165 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc4005c40 con 0x7f5bd4143ec0 2024-05-22T21:13:28.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bd0c00640 1 -- 172.21.15.161:0/4080566165 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4007070 con 0x7f5bd4143ec0 2024-05-22T21:13:28.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bdae00640 1 -- 172.21.15.161:0/4080566165 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd4140880 con 0x7f5bd4143ec0 2024-05-22T21:13:28.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.051+0000 7f5bdae00640 1 -- 172.21.15.161:0/4080566165 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd413fcd0 con 0x7f5bd4143ec0 2024-05-22T21:13:28.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.055+0000 7f5bdae00640 1 -- 172.21.15.161:0/4080566165 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd4067220 con 0x7f5bd4143ec0 2024-05-22T21:13:28.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.055+0000 7f5bd0c00640 1 -- 172.21.15.161:0/4080566165 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5bc40071d0 con 0x7f5bd4143ec0 2024-05-22T21:13:28.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.055+0000 7f5bd0c00640 1 --2- 172.21.15.161:0/4080566165 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5bb0066690 0x7f5bb0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.055+0000 7f5bd0c00640 1 -- 172.21.15.161:0/4080566165 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bc4083f20 con 0x7f5bd4143ec0 2024-05-22T21:13:28.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.055+0000 7f5bcb400640 1 --2- 172.21.15.161:0/4080566165 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5bb0066690 0x7f5bb0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.055+0000 7f5bcb400640 1 --2- 172.21.15.161:0/4080566165 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5bb0066690 0x7f5bb0068b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0ae000 tx=0x7f5bbc0a6070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:28.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.059+0000 7f5bd0c00640 1 -- 172.21.15.161:0/4080566165 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5bc4054dc0 con 0x7f5bd4143ec0 2024-05-22T21:13:28.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.211+0000 7f5bdae00640 1 -- 172.21.15.161:0/4080566165 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5bd40b1cb0 con 0x7f5bb0066690 2024-05-22T21:13:28.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.215+0000 7f5bd0c00640 1 -- 172.21.15.161:0/4080566165 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248992 (secure 0 0 0) 0x7f5bd40b1cb0 con 0x7f5bb0066690 2024-05-22T21:13:28.220 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:28.221 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:13:28.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.219+0000 7f5bdae00640 1 -- 172.21.15.161:0/4080566165 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5bb0066690 msgr2=0x7f5bb0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.219+0000 7f5bdae00640 1 --2- 172.21.15.161:0/4080566165 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5bb0066690 0x7f5bb0068b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0ae000 tx=0x7f5bbc0a6070 comp rx=0 tx=0).stop 2024-05-22T21:13:28.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.219+0000 7f5bdae00640 1 -- 172.21.15.161:0/4080566165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bd4143ec0 msgr2=0x7f5bd413f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.219+0000 7f5bdae00640 1 --2- 172.21.15.161:0/4080566165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bd4143ec0 0x7f5bd413f790 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400c760 tx=0x7f5bc400cc20 comp rx=0 tx=0).stop 2024-05-22T21:13:28.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.219+0000 7f5bdae00640 1 -- 172.21.15.161:0/4080566165 shutdown_connections 2024-05-22T21:13:28.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.219+0000 7f5bdae00640 1 --2- 172.21.15.161:0/4080566165 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5bb0066690 0x7f5bb0068b50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.219+0000 7f5bdae00640 1 --2- 172.21.15.161:0/4080566165 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5bd415a240 0x7f5bd4159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.219+0000 7f5bdae00640 1 --2- 172.21.15.161:0/4080566165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bd415ec90 0x7f5bd413a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.219+0000 7f5bdae00640 1 --2- 172.21.15.161:0/4080566165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bd4143ec0 0x7f5bd413f790 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.219+0000 7f5bdae00640 1 -- 172.21.15.161:0/4080566165 >> 172.21.15.161:0/4080566165 conn(0x7f5bd40b0bc0 msgr2=0x7f5bd40afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:28.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.219+0000 7f5bdae00640 1 -- 172.21.15.161:0/4080566165 shutdown_connections 2024-05-22T21:13:28.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.219+0000 7f5bdae00640 1 -- 172.21.15.161:0/4080566165 wait complete. 2024-05-22T21:13:28.240 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-05-22T21:13:26.392477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1544,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560955,"statfs":{"total":1600000000,"available":1598417947,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.359966"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-22T21:13:15.003438+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:13:15.003438+0000","last_peered":"2024-05-22T21:13:15.003438+0000","last_clean":"2024-05-22T21:13:15.003438+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:13:15.003438+0000","last_undegraded":"2024-05-22T21:13:15.003438+0000","last_fullsized":"2024-05-22T21:13:15.003438+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:23.032353+0000","last_change":"2024-05-22T21:13:23.032353+0000","last_active":"2024-05-22T21:13:23.032353+0000","last_peered":"2024-05-22T21:13:23.032353+0000","last_clean":"2024-05-22T21:13:23.032353+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:23.032353+0000","last_undegraded":"2024-05-22T21:13:23.032353+0000","last_fullsized":"2024-05-22T21:13:23.032353+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:13:22.024639+0000","last_change":"2024-05-22T21:13:22.024639+0000","last_active":"2024-05-22T21:13:22.024639+0000","last_peered":"2024-05-22T21:13:22.024639+0000","last_clean":"2024-05-22T21:13:22.024639+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:13:22.024639+0000","last_undegraded":"2024-05-22T21:13:22.024639+0000","last_fullsized":"2024-05-22T21:13:22.024639+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359390,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":624,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194687,"statfs":{"total":200000000,"available":199360009,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]}],"pool_statfs":[]}} 2024-05-22T21:13:28.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2024-05-22T21:13:28.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2024-05-22T21:13:28.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2024-05-22T21:13:28.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2024-05-22T21:13:28.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2024-05-22T21:13:28.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2024-05-22T21:13:28.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2024-05-22T21:13:28.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2024-05-22T21:13:28.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.249 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2024-05-22T21:13:28.249 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2024-05-22T21:13:28.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2024-05-22T21:13:28.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2024-05-22T21:13:28.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2024-05-22T21:13:28.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2024-05-22T21:13:28.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2024-05-22T21:13:28.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2024-05-22T21:13:28.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2024-05-22T21:13:28.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2024-05-22T21:13:28.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2024-05-22T21:13:28.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2024-05-22T21:13:28.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2024-05-22T21:13:28.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2024-05-22T21:13:28.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2024-05-22T21:13:28.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2024-05-22T21:13:28.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2024-05-22T21:13:28.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2024-05-22T21:13:28.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2024-05-22T21:13:28.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2024-05-22T21:13:28.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2024-05-22T21:13:28.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2024-05-22T21:13:28.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2024-05-22T21:13:28.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2024-05-22T21:13:28.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2024-05-22T21:13:28.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2024-05-22T21:13:28.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-05-22T21:13:28.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2024-05-22T21:13:28.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2024-05-22T21:13:28.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2024-05-22T21:13:28.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2024-05-22T21:13:28.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-05-22T21:13:28.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2024-05-22T21:13:28.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2024-05-22T21:13:28.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2024-05-22T21:13:28.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2024-05-22T21:13:28.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2024-05-22T21:13:28.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2024-05-22T21:13:28.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2024-05-22T21:13:28.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2024-05-22T21:13:28.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2024-05-22T21:13:28.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2024-05-22T21:13:28.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2024-05-22T21:13:28.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2024-05-22T21:13:28.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2024-05-22T21:13:28.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2024-05-22T21:13:28.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2024-05-22T21:13:28.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2024-05-22T21:13:28.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2024-05-22T21:13:28.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2024-05-22T21:13:28.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2024-05-22T21:13:28.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2024-05-22T21:13:28.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2024-05-22T21:13:28.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2024-05-22T21:13:28.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2024-05-22T21:13:28.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2024-05-22T21:13:28.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2024-05-22T21:13:28.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2024-05-22T21:13:28.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2024-05-22T21:13:28.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2024-05-22T21:13:28.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2024-05-22T21:13:28.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-05-22T21:13:28.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2024-05-22T21:13:28.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2024-05-22T21:13:28.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2024-05-22T21:13:28.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2024-05-22T21:13:28.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2024-05-22T21:13:28.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2024-05-22T21:13:28.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2024-05-22T21:13:28.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2024-05-22T21:13:28.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2024-05-22T21:13:28.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-05-22T21:13:28.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2024-05-22T21:13:28.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2024-05-22T21:13:28.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2024-05-22T21:13:28.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2024-05-22T21:13:28.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2024-05-22T21:13:28.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2024-05-22T21:13:28.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2024-05-22T21:13:28.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2024-05-22T21:13:28.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2024-05-22T21:13:28.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2024-05-22T21:13:28.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2024-05-22T21:13:28.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2024-05-22T21:13:28.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2024-05-22T21:13:28.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2024-05-22T21:13:28.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2024-05-22T21:13:28.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:23.032353+0000', 'last_change': '2024-05-22T21:13:23.032353+0000', 'last_active': '2024-05-22T21:13:23.032353+0000', 'last_peered': '2024-05-22T21:13:23.032353+0000', 'last_clean': '2024-05-22T21:13:23.032353+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:23.032353+0000', 'last_undegraded': '2024-05-22T21:13:23.032353+0000', 'last_fullsized': '2024-05-22T21:13:23.032353+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:23.032353+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2024-05-22T21:13:28.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-05-22T21:13:22.024639+0000', 'last_change': '2024-05-22T21:13:22.024639+0000', 'last_active': '2024-05-22T21:13:22.024639+0000', 'last_peered': '2024-05-22T21:13:22.024639+0000', 'last_clean': '2024-05-22T21:13:22.024639+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-22T21:13:22.024639+0000', 'last_undegraded': '2024-05-22T21:13:22.024639+0000', 'last_fullsized': '2024-05-22T21:13:22.024639+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'last_clean_scrub_stamp': '2024-05-22T21:13:22.024639+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-05-22T21:13:28.266 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-22T21:13:28.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.414+0000 7f2c15200640 1 Processor -- start 2024-05-22T21:13:28.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.414+0000 7f2c15200640 1 -- start start 2024-05-22T21:13:28.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.414+0000 7f2c15200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2c1015d430 0x7f2c100707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.414+0000 7f2c15200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c100accf0 0x7f2c10160e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.414+0000 7f2c15200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c10071fd0 con 0x7f2c100ac8b0 2024-05-22T21:13:28.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.414+0000 7f2c15200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c10070e80 con 0x7f2c1015d430 2024-05-22T21:13:28.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.414+0000 7f2c15200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1006c510 con 0x7f2c100accf0 2024-05-22T21:13:28.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.414+0000 7f2c0de00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2c1015d430 0x7f2c100707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.414+0000 7f2c0de00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2c1015d430 0x7f2c100707c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48978/0 (socket says 172.21.15.161:48978) 2024-05-22T21:13:28.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.414+0000 7f2c0de00640 1 -- 172.21.15.161:0/1475974340 learned_addr learned my addr 172.21.15.161:0/1475974340 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:28.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.414+0000 7f2c0e800640 1 --2- 172.21.15.161:0/1475974340 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c100accf0 0x7f2c10160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.414+0000 7f2c0de00640 1 -- 172.21.15.161:0/1475974340 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c100accf0 msgr2=0x7f2c10160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:28.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.414+0000 7f2c0de00640 1 --2- 172.21.15.161:0/1475974340 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c100accf0 0x7f2c10160e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.414+0000 7f2c0de00640 1 -- 172.21.15.161:0/1475974340 >> v1:172.21.15.179:6789/0 conn(0x7f2c100ac8b0 legacy=0x7f2c1006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.414+0000 7f2c0de00640 1 -- 172.21.15.161:0/1475974340 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1006c7f0 con 0x7f2c1015d430 2024-05-22T21:13:28.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.418+0000 7f2c0de00640 1 --2- 172.21.15.161:0/1475974340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c1015d430 0x7f2c100707c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009870 tx=0x7f2bf801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8b75e913a858f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:28.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.418+0000 7f2c0d400640 1 -- 172.21.15.161:0/1475974340 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf80ac410 con 0x7f2c1015d430 2024-05-22T21:13:28.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.418+0000 7f2c0d400640 1 -- 172.21.15.161:0/1475974340 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf80ac570 con 0x7f2c1015d430 2024-05-22T21:13:28.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.418+0000 7f2c0d400640 1 -- 172.21.15.161:0/1475974340 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf80ac900 con 0x7f2c1015d430 2024-05-22T21:13:28.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.418+0000 7f2c15200640 1 -- 172.21.15.161:0/1475974340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c1015d430 msgr2=0x7f2c100707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.418+0000 7f2c15200640 1 --2- 172.21.15.161:0/1475974340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c1015d430 0x7f2c100707c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009870 tx=0x7f2bf801c4a0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.418+0000 7f2c15200640 1 -- 172.21.15.161:0/1475974340 shutdown_connections 2024-05-22T21:13:28.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.418+0000 7f2c15200640 1 --2- 172.21.15.161:0/1475974340 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c100accf0 0x7f2c10160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.418+0000 7f2c15200640 1 --2- 172.21.15.161:0/1475974340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c1015d430 0x7f2c100707c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.418+0000 7f2c15200640 1 -- 172.21.15.161:0/1475974340 >> 172.21.15.161:0/1475974340 conn(0x7f2c100b0bc0 msgr2=0x7f2c100b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:28.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.418+0000 7f2c15200640 1 -- 172.21.15.161:0/1475974340 shutdown_connections 2024-05-22T21:13:28.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.418+0000 7f2c15200640 1 -- 172.21.15.161:0/1475974340 wait complete. 2024-05-22T21:13:28.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c15200640 1 Processor -- start 2024-05-22T21:13:28.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c15200640 1 -- start start 2024-05-22T21:13:28.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c15200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c100ac8b0 0x7f2c10155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c15200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c100accf0 0x7f2c100b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c15200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c1015d430 0x7f2c100b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c15200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c10154280 con 0x7f2c100accf0 2024-05-22T21:13:28.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c15200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c10154560 con 0x7f2c1015d430 2024-05-22T21:13:28.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c15200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c10154840 con 0x7f2c100ac8b0 2024-05-22T21:13:28.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c05400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c100accf0 0x7f2c100b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c05400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c100accf0 0x7f2c100b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48980/0 (socket says 172.21.15.161:48980) 2024-05-22T21:13:28.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c05400640 1 -- 172.21.15.161:0/367752919 learned_addr learned my addr 172.21.15.161:0/367752919 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c0de00640 1 --2- 172.21.15.161:0/367752919 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c100ac8b0 0x7f2c10155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c0e800640 1 --2- 172.21.15.161:0/367752919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c1015d430 0x7f2c100b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c05400640 1 -- 172.21.15.161:0/367752919 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c100ac8b0 msgr2=0x7f2c10155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c05400640 1 --2- 172.21.15.161:0/367752919 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c100ac8b0 0x7f2c10155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c05400640 1 -- 172.21.15.161:0/367752919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c1015d430 msgr2=0x7f2c100b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c05400640 1 --2- 172.21.15.161:0/367752919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c1015d430 0x7f2c100b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c05400640 1 -- 172.21.15.161:0/367752919 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf80094d0 con 0x7f2c100accf0 2024-05-22T21:13:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c0e800640 1 --2- 172.21.15.161:0/367752919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c1015d430 0x7f2c100b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c05400640 1 --2- 172.21.15.161:0/367752919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c100accf0 0x7f2c100b22c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000c3c0 tx=0x7f2c0000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c06a00640 1 -- 172.21.15.161:0/367752919 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c000052d0 con 0x7f2c100accf0 2024-05-22T21:13:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c06a00640 1 -- 172.21.15.161:0/367752919 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c00005430 con 0x7f2c100accf0 2024-05-22T21:13:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c06a00640 1 -- 172.21.15.161:0/367752919 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c00003840 con 0x7f2c100accf0 2024-05-22T21:13:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c15200640 1 -- 172.21.15.161:0/367752919 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c10154b20 con 0x7f2c100accf0 2024-05-22T21:13:28.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.422+0000 7f2c15200640 1 -- 172.21.15.161:0/367752919 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c10153df0 con 0x7f2c100accf0 2024-05-22T21:13:28.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.426+0000 7f2c15200640 1 -- 172.21.15.161:0/367752919 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c10067220 con 0x7f2c100accf0 2024-05-22T21:13:28.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.426+0000 7f2c06a00640 1 -- 172.21.15.161:0/367752919 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2c000039a0 con 0x7f2c100accf0 2024-05-22T21:13:28.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.426+0000 7f2c06a00640 1 --2- 172.21.15.161:0/367752919 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2bec066690 0x7f2bec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.426+0000 7f2c06a00640 1 -- 172.21.15.161:0/367752919 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c000836c0 con 0x7f2c100accf0 2024-05-22T21:13:28.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.426+0000 7f2c0de00640 1 --2- 172.21.15.161:0/367752919 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2bec066690 0x7f2bec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.426+0000 7f2c0de00640 1 --2- 172.21.15.161:0/367752919 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2bec066690 0x7f2bec068b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80ae000 tx=0x7f2bf8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:28.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.430+0000 7f2c06a00640 1 -- 172.21.15.161:0/367752919 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2c00050a40 con 0x7f2c100accf0 2024-05-22T21:13:28.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.638+0000 7f2c15200640 1 -- 172.21.15.161:0/367752919 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2c100ae7a0 con 0x7f2c100accf0 2024-05-22T21:13:28.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.638+0000 7f2c06a00640 1 -- 172.21.15.161:0/367752919 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1004 (secure 0 0 0) 0x7f2c00054060 con 0x7f2c100accf0 2024-05-22T21:13:28.643 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:28.643 INFO:teuthology.orchestra.run.smithi161.stdout:{"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":23,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1716412390,"num_in_osds":8,"osd_in_since":1716412387,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592730,"bytes_used":1788113,"bytes_avail":1598211887,"bytes_total":1600000000,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4282}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-05-22T21:13:05.416891+0000","services":{}},"progress_events":{}} 2024-05-22T21:13:28.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.642+0000 7f2c15200640 1 -- 172.21.15.161:0/367752919 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2bec066690 msgr2=0x7f2bec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.642+0000 7f2c15200640 1 --2- 172.21.15.161:0/367752919 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2bec066690 0x7f2bec068b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80ae000 tx=0x7f2bf8009040 comp rx=0 tx=0).stop 2024-05-22T21:13:28.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.642+0000 7f2c15200640 1 -- 172.21.15.161:0/367752919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c100accf0 msgr2=0x7f2c100b22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.642+0000 7f2c15200640 1 --2- 172.21.15.161:0/367752919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c100accf0 0x7f2c100b22c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000c3c0 tx=0x7f2c0000c880 comp rx=0 tx=0).stop 2024-05-22T21:13:28.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.642+0000 7f2c15200640 1 -- 172.21.15.161:0/367752919 shutdown_connections 2024-05-22T21:13:28.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.642+0000 7f2c15200640 1 --2- 172.21.15.161:0/367752919 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2bec066690 0x7f2bec068b50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.642+0000 7f2c15200640 1 --2- 172.21.15.161:0/367752919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c1015d430 0x7f2c100b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.642+0000 7f2c15200640 1 --2- 172.21.15.161:0/367752919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c100accf0 0x7f2c100b22c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.642+0000 7f2c15200640 1 --2- 172.21.15.161:0/367752919 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c100ac8b0 0x7f2c10155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.642+0000 7f2c15200640 1 -- 172.21.15.161:0/367752919 >> 172.21.15.161:0/367752919 conn(0x7f2c100b0bc0 msgr2=0x7f2c100ad120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:28.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.642+0000 7f2c15200640 1 -- 172.21.15.161:0/367752919 shutdown_connections 2024-05-22T21:13:28.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.642+0000 7f2c15200640 1 -- 172.21.15.161:0/367752919 wait complete. 2024-05-22T21:13:28.663 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-22T21:13:28.664 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:13:28.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.794+0000 7f3dd3a00640 1 Processor -- start 2024-05-22T21:13:28.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.794+0000 7f3dd3a00640 1 -- start start 2024-05-22T21:13:28.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.794+0000 7f3dd3a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dcc15f3a0 0x7f3dcc15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.794+0000 7f3dd3a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dcc1442a0 con 0x7f3dcc0715a0 2024-05-22T21:13:28.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.794+0000 7f3dd3a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dcc070500 con 0x7f3dcc143ec0 2024-05-22T21:13:28.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.794+0000 7f3dd3a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc0707e0 con 0x7f3dcc15f3a0 2024-05-22T21:13:28.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.794+0000 7f3dc3e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3dcc0715a0 0x7f3dcc15ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56124/0 (socket says 172.21.15.161:56124) 2024-05-22T21:13:28.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.794+0000 7f3dc3e00640 1 -- 172.21.15.161:0/98307700 learned_addr learned my addr 172.21.15.161:0/98307700 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:28.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dd1a00640 1 --2- 172.21.15.161:0/98307700 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dcc15f3a0 0x7f3dcc15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dcbe00640 1 -- 172.21.15.161:0/98307700 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173396372 0 0) 0x7f3dcc1442a0 con 0x7f3dcc0715a0 2024-05-22T21:13:28.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dcbe00640 1 -- 172.21.15.161:0/98307700 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3db0003530 con 0x7f3dcc0715a0 2024-05-22T21:13:28.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dcbe00640 1 -- 172.21.15.161:0/98307700 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4030534774 0 0) 0x7f3db0003530 con 0x7f3dcc0715a0 2024-05-22T21:13:28.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dcbe00640 1 -- 172.21.15.161:0/98307700 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dcc15f3a0 msgr2=0x7f3dcc15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:28.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dcbe00640 1 --2- 172.21.15.161:0/98307700 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dcc15f3a0 0x7f3dcc15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dcbe00640 1 -- 172.21.15.161:0/98307700 >> v1:172.21.15.179:6789/0 conn(0x7f3dcc143ec0 legacy=0x7f3dcc06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dcbe00640 1 -- 172.21.15.161:0/98307700 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc070ac0 con 0x7f3dcc0715a0 2024-05-22T21:13:28.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dcbe00640 1 -- 172.21.15.161:0/98307700 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f3db4001680 con 0x7f3dcc0715a0 2024-05-22T21:13:28.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dcbe00640 1 -- 172.21.15.161:0/98307700 >> v1:172.21.15.161:6789/0 conn(0x7f3dcc0715a0 legacy=0x7f3dcc15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dcbe00640 1 --2- 172.21.15.161:0/98307700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db00039b0 0x7f3db0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dcbe00640 1 --2- 172.21.15.161:0/98307700 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dcc15f3a0 0x7f3db00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dcbe00640 1 --2- 172.21.15.161:0/98307700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3db0004d10 0x7f3db0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dcbe00640 1 -- 172.21.15.161:0/98307700 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc0707e0 con 0x7f3db0004d10 2024-05-22T21:13:28.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dcbe00640 1 -- 172.21.15.161:0/98307700 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db000ba70 con 0x7f3db00039b0 2024-05-22T21:13:28.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dcbe00640 1 -- 172.21.15.161:0/98307700 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db000bc30 con 0x7f3dcc15f3a0 2024-05-22T21:13:28.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dd1a00640 1 --2- 172.21.15.161:0/98307700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3db0004d10 0x7f3db0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dc3e00640 1 --2- 172.21.15.161:0/98307700 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dcc15f3a0 0x7f3db00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dd1000640 1 --2- 172.21.15.161:0/98307700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db00039b0 0x7f3db0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dd1a00640 1 -- 172.21.15.161:0/98307700 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dcc15f3a0 msgr2=0x7f3db00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:28.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dd1a00640 1 --2- 172.21.15.161:0/98307700 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dcc15f3a0 0x7f3db00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dd1a00640 1 -- 172.21.15.161:0/98307700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db00039b0 msgr2=0x7f3db0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:28.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dd1a00640 1 --2- 172.21.15.161:0/98307700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db00039b0 0x7f3db0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.798+0000 7f3dd1a00640 1 -- 172.21.15.161:0/98307700 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db0003530 con 0x7f3db0004d10 2024-05-22T21:13:28.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd1a00640 1 --2- 172.21.15.161:0/98307700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3db0004d10 0x7f3db0005190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0cede0 tx=0x7f3dc401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6cdcd56a79ca29 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:28.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dcbe00640 1 -- 172.21.15.161:0/98307700 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc40a5c30 con 0x7f3db0004d10 2024-05-22T21:13:28.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dcbe00640 1 -- 172.21.15.161:0/98307700 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc400f040 con 0x7f3db0004d10 2024-05-22T21:13:28.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dcbe00640 1 -- 172.21.15.161:0/98307700 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc40ad440 con 0x7f3db0004d10 2024-05-22T21:13:28.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd3a00640 1 -- 172.21.15.161:0/98307700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3db0004d10 msgr2=0x7f3db0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd3a00640 1 --2- 172.21.15.161:0/98307700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3db0004d10 0x7f3db0005190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0cede0 tx=0x7f3dc401cc20 comp rx=0 tx=0).stop 2024-05-22T21:13:28.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd1a00640 1 -- 172.21.15.161:0/98307700 reap_dead start 2024-05-22T21:13:28.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd3a00640 1 -- 172.21.15.161:0/98307700 shutdown_connections 2024-05-22T21:13:28.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd3a00640 1 -- 172.21.15.161:0/98307700 >> 172.21.15.161:0/98307700 conn(0x7f3dcc0b0bc0 msgr2=0x7f3dcc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:28.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd3a00640 1 -- 172.21.15.161:0/98307700 shutdown_connections 2024-05-22T21:13:28.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd3a00640 1 -- 172.21.15.161:0/98307700 wait complete. 2024-05-22T21:13:28.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd3a00640 1 Processor -- start 2024-05-22T21:13:28.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd3a00640 1 -- start start 2024-05-22T21:13:28.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd3a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db0004d10 0x7f3dcc15e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd3a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dcc0ac8b0 0x7f3dcc13cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd3a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dcc15bef0 0x7f3dcc15c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd3a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc15c8b0 con 0x7f3dcc15bef0 2024-05-22T21:13:28.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd3a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc0ad020 con 0x7f3db0004d10 2024-05-22T21:13:28.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd3a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc0ad300 con 0x7f3dcc0ac8b0 2024-05-22T21:13:28.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd1a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dcc15bef0 0x7f3dcc15c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd1a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dcc15bef0 0x7f3dcc15c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48986/0 (socket says 172.21.15.161:48986) 2024-05-22T21:13:28.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd1a00640 1 -- 172.21.15.161:0/1667949361 learned_addr learned my addr 172.21.15.161:0/1667949361 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:28.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.802+0000 7f3dd1000640 1 --2- 172.21.15.161:0/1667949361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db0004d10 0x7f3dcc15e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dc3e00640 1 --2- 172.21.15.161:0/1667949361 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dcc0ac8b0 0x7f3dcc13cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dd1a00640 1 -- 172.21.15.161:0/1667949361 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dcc0ac8b0 msgr2=0x7f3dcc13cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dd1a00640 1 --2- 172.21.15.161:0/1667949361 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dcc0ac8b0 0x7f3dcc13cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dd1a00640 1 -- 172.21.15.161:0/1667949361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db0004d10 msgr2=0x7f3dcc15e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dd1a00640 1 --2- 172.21.15.161:0/1667949361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db0004d10 0x7f3dcc15e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dd1a00640 1 -- 172.21.15.161:0/1667949361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc0ad5e0 con 0x7f3dcc15bef0 2024-05-22T21:13:28.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dc3e00640 1 --2- 172.21.15.161:0/1667949361 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dcc0ac8b0 0x7f3dcc13cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:28.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dd1a00640 1 --2- 172.21.15.161:0/1667949361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dcc15bef0 0x7f3dcc15c370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4002240 tx=0x7f3dc4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:28.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dc9600640 1 -- 172.21.15.161:0/1667949361 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc40039a0 con 0x7f3dcc15bef0 2024-05-22T21:13:28.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dc9600640 1 -- 172.21.15.161:0/1667949361 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc4003b00 con 0x7f3dcc15bef0 2024-05-22T21:13:28.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dc9600640 1 -- 172.21.15.161:0/1667949361 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc40ae890 con 0x7f3dcc15bef0 2024-05-22T21:13:28.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dd3a00640 1 -- 172.21.15.161:0/1667949361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dcc0ad8c0 con 0x7f3dcc15bef0 2024-05-22T21:13:28.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dd3a00640 1 -- 172.21.15.161:0/1667949361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dcc0acd30 con 0x7f3dcc15bef0 2024-05-22T21:13:28.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dd3a00640 1 -- 172.21.15.161:0/1667949361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dcc067220 con 0x7f3dcc15bef0 2024-05-22T21:13:28.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dc9600640 1 -- 172.21.15.161:0/1667949361 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3dc40b4040 con 0x7f3dcc15bef0 2024-05-22T21:13:28.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dc9600640 1 --2- 172.21.15.161:0/1667949361 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3da8066690 0x7f3da8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:28.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dc9600640 1 -- 172.21.15.161:0/1667949361 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3dc411dfe0 con 0x7f3dcc15bef0 2024-05-22T21:13:28.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.806+0000 7f3dd1000640 1 --2- 172.21.15.161:0/1667949361 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3da8066690 0x7f3da8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:28.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.810+0000 7f3dd1000640 1 --2- 172.21.15.161:0/1667949361 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3da8066690 0x7f3da8068b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc007fd0 tx=0x7f3dbc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:28.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.810+0000 7f3dc9600640 1 -- 172.21.15.161:0/1667949361 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3dc40eb360 con 0x7f3dcc15bef0 2024-05-22T21:13:28.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.962+0000 7f3dd3a00640 1 -- 172.21.15.161:0/1667949361 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3dcc071080 con 0x7f3da8066690 2024-05-22T21:13:28.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.970+0000 7f3dc9600640 1 -- 172.21.15.161:0/1667949361 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260221 (secure 0 0 0) 0x7f3dbc012940 con 0x7f3da8066690 2024-05-22T21:13:28.974 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:28.976 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:13:28.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.970+0000 7f3dd3a00640 1 -- 172.21.15.161:0/1667949361 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3da8066690 msgr2=0x7f3da8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.970+0000 7f3dd3a00640 1 --2- 172.21.15.161:0/1667949361 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3da8066690 0x7f3da8068b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc007fd0 tx=0x7f3dbc009040 comp rx=0 tx=0).stop 2024-05-22T21:13:28.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.970+0000 7f3dd3a00640 1 -- 172.21.15.161:0/1667949361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dcc15bef0 msgr2=0x7f3dcc15c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:28.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.970+0000 7f3dd3a00640 1 --2- 172.21.15.161:0/1667949361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dcc15bef0 0x7f3dcc15c370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4002240 tx=0x7f3dc4009750 comp rx=0 tx=0).stop 2024-05-22T21:13:28.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.970+0000 7f3dd3a00640 1 -- 172.21.15.161:0/1667949361 shutdown_connections 2024-05-22T21:13:28.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.970+0000 7f3dd3a00640 1 --2- 172.21.15.161:0/1667949361 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3da8066690 0x7f3da8068b50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.970+0000 7f3dd3a00640 1 --2- 172.21.15.161:0/1667949361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dcc15bef0 0x7f3dcc15c370 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.970+0000 7f3dd3a00640 1 --2- 172.21.15.161:0/1667949361 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dcc0ac8b0 0x7f3dcc13cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.970+0000 7f3dd3a00640 1 --2- 172.21.15.161:0/1667949361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db0004d10 0x7f3dcc15e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:28.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.970+0000 7f3dd3a00640 1 -- 172.21.15.161:0/1667949361 >> 172.21.15.161:0/1667949361 conn(0x7f3dcc0b0bc0 msgr2=0x7f3dcc13fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:28.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.970+0000 7f3dd3a00640 1 -- 172.21.15.161:0/1667949361 shutdown_connections 2024-05-22T21:13:28.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:28.970+0000 7f3dd3a00640 1 -- 172.21.15.161:0/1667949361 wait complete. 2024-05-22T21:13:28.993 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-22T21:13:28.392912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1746,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560753,"statfs":{"total":1600000000,"available":1598211887,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001123"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428654+0000","last_change":"2024-05-22T21:13:23.445778+0000","last_active":"2024-05-22T21:13:24.428654+0000","last_peered":"2024-05-22T21:13:24.428654+0000","last_clean":"2024-05-22T21:13:24.428654+0000","last_became_active":"2024-05-22T21:13:23.445416+0000","last_became_peered":"2024-05-22T21:13:23.445416+0000","last_unstale":"2024-05-22T21:13:24.428654+0000","last_undegraded":"2024-05-22T21:13:24.428654+0000","last_fullsized":"2024-05-22T21:13:24.428654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:57:10.746221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423393+0000","last_change":"2024-05-22T21:13:23.429874+0000","last_active":"2024-05-22T21:13:24.423393+0000","last_peered":"2024-05-22T21:13:24.423393+0000","last_clean":"2024-05-22T21:13:24.423393+0000","last_became_active":"2024-05-22T21:13:23.428636+0000","last_became_peered":"2024-05-22T21:13:23.428636+0000","last_unstale":"2024-05-22T21:13:24.423393+0000","last_undegraded":"2024-05-22T21:13:24.423393+0000","last_fullsized":"2024-05-22T21:13:24.423393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:44:56.029080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015292+0000","last_change":"2024-05-22T21:13:23.434034+0000","last_active":"2024-05-22T21:13:25.015292+0000","last_peered":"2024-05-22T21:13:25.015292+0000","last_clean":"2024-05-22T21:13:25.015292+0000","last_became_active":"2024-05-22T21:13:23.432250+0000","last_became_peered":"2024-05-22T21:13:23.432250+0000","last_unstale":"2024-05-22T21:13:25.015292+0000","last_undegraded":"2024-05-22T21:13:25.015292+0000","last_fullsized":"2024-05-22T21:13:25.015292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:04:37.695966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919501+0000","last_change":"2024-05-22T21:13:23.440239+0000","last_active":"2024-05-22T21:13:24.919501+0000","last_peered":"2024-05-22T21:13:24.919501+0000","last_clean":"2024-05-22T21:13:24.919501+0000","last_became_active":"2024-05-22T21:13:23.433800+0000","last_became_peered":"2024-05-22T21:13:23.433800+0000","last_unstale":"2024-05-22T21:13:24.919501+0000","last_undegraded":"2024-05-22T21:13:24.919501+0000","last_fullsized":"2024-05-22T21:13:24.919501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:09:20.599937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429279+0000","last_change":"2024-05-22T21:13:23.436638+0000","last_active":"2024-05-22T21:13:24.429279+0000","last_peered":"2024-05-22T21:13:24.429279+0000","last_clean":"2024-05-22T21:13:24.429279+0000","last_became_active":"2024-05-22T21:13:23.436392+0000","last_became_peered":"2024-05-22T21:13:23.436392+0000","last_unstale":"2024-05-22T21:13:24.429279+0000","last_undegraded":"2024-05-22T21:13:24.429279+0000","last_fullsized":"2024-05-22T21:13:24.429279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:38:04.537935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919974+0000","last_change":"2024-05-22T21:13:23.442807+0000","last_active":"2024-05-22T21:13:24.919974+0000","last_peered":"2024-05-22T21:13:24.919974+0000","last_clean":"2024-05-22T21:13:24.919974+0000","last_became_active":"2024-05-22T21:13:23.441590+0000","last_became_peered":"2024-05-22T21:13:23.441590+0000","last_unstale":"2024-05-22T21:13:24.919974+0000","last_undegraded":"2024-05-22T21:13:24.919974+0000","last_fullsized":"2024-05-22T21:13:24.919974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:57:11.273220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429185+0000","last_change":"2024-05-22T21:13:23.436160+0000","last_active":"2024-05-22T21:13:24.429185+0000","last_peered":"2024-05-22T21:13:24.429185+0000","last_clean":"2024-05-22T21:13:24.429185+0000","last_became_active":"2024-05-22T21:13:23.435949+0000","last_became_peered":"2024-05-22T21:13:23.435949+0000","last_unstale":"2024-05-22T21:13:24.429185+0000","last_undegraded":"2024-05-22T21:13:24.429185+0000","last_fullsized":"2024-05-22T21:13:24.429185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:56:26.375232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014774+0000","last_change":"2024-05-22T21:13:23.437626+0000","last_active":"2024-05-22T21:13:25.014774+0000","last_peered":"2024-05-22T21:13:25.014774+0000","last_clean":"2024-05-22T21:13:25.014774+0000","last_became_active":"2024-05-22T21:13:23.437162+0000","last_became_peered":"2024-05-22T21:13:23.437162+0000","last_unstale":"2024-05-22T21:13:25.014774+0000","last_undegraded":"2024-05-22T21:13:25.014774+0000","last_fullsized":"2024-05-22T21:13:25.014774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:09:20.599937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428632+0000","last_change":"2024-05-22T21:13:23.446764+0000","last_active":"2024-05-22T21:13:24.428632+0000","last_peered":"2024-05-22T21:13:24.428632+0000","last_clean":"2024-05-22T21:13:24.428632+0000","last_became_active":"2024-05-22T21:13:23.446506+0000","last_became_peered":"2024-05-22T21:13:23.446506+0000","last_unstale":"2024-05-22T21:13:24.428632+0000","last_undegraded":"2024-05-22T21:13:24.428632+0000","last_fullsized":"2024-05-22T21:13:24.428632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:13:51.196274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014576+0000","last_change":"2024-05-22T21:13:23.450063+0000","last_active":"2024-05-22T21:13:25.014576+0000","last_peered":"2024-05-22T21:13:25.014576+0000","last_clean":"2024-05-22T21:13:25.014576+0000","last_became_active":"2024-05-22T21:13:23.449193+0000","last_became_peered":"2024-05-22T21:13:23.449193+0000","last_unstale":"2024-05-22T21:13:25.014576+0000","last_undegraded":"2024-05-22T21:13:25.014576+0000","last_fullsized":"2024-05-22T21:13:25.014576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:14:11.125539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014530+0000","last_change":"2024-05-22T21:13:23.449563+0000","last_active":"2024-05-22T21:13:25.014530+0000","last_peered":"2024-05-22T21:13:25.014530+0000","last_clean":"2024-05-22T21:13:25.014530+0000","last_became_active":"2024-05-22T21:13:23.449304+0000","last_became_peered":"2024-05-22T21:13:23.449304+0000","last_unstale":"2024-05-22T21:13:25.014530+0000","last_undegraded":"2024-05-22T21:13:25.014530+0000","last_fullsized":"2024-05-22T21:13:25.014530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:07:36.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428800+0000","last_change":"2024-05-22T21:13:23.444134+0000","last_active":"2024-05-22T21:13:24.428800+0000","last_peered":"2024-05-22T21:13:24.428800+0000","last_clean":"2024-05-22T21:13:24.428800+0000","last_became_active":"2024-05-22T21:13:23.443924+0000","last_became_peered":"2024-05-22T21:13:23.443924+0000","last_unstale":"2024-05-22T21:13:24.428800+0000","last_undegraded":"2024-05-22T21:13:24.428800+0000","last_fullsized":"2024-05-22T21:13:24.428800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:10.124782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014938+0000","last_change":"2024-05-22T21:13:23.447229+0000","last_active":"2024-05-22T21:13:25.014938+0000","last_peered":"2024-05-22T21:13:25.014938+0000","last_clean":"2024-05-22T21:13:25.014938+0000","last_became_active":"2024-05-22T21:13:23.447007+0000","last_became_peered":"2024-05-22T21:13:23.447007+0000","last_unstale":"2024-05-22T21:13:25.014938+0000","last_undegraded":"2024-05-22T21:13:25.014938+0000","last_fullsized":"2024-05-22T21:13:25.014938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:44:04.542487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014914+0000","last_change":"2024-05-22T21:13:23.433465+0000","last_active":"2024-05-22T21:13:25.014914+0000","last_peered":"2024-05-22T21:13:25.014914+0000","last_clean":"2024-05-22T21:13:25.014914+0000","last_became_active":"2024-05-22T21:13:23.432948+0000","last_became_peered":"2024-05-22T21:13:23.432948+0000","last_unstale":"2024-05-22T21:13:25.014914+0000","last_undegraded":"2024-05-22T21:13:25.014914+0000","last_fullsized":"2024-05-22T21:13:25.014914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:07:36.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423652+0000","last_change":"2024-05-22T21:13:23.426036+0000","last_active":"2024-05-22T21:13:24.423652+0000","last_peered":"2024-05-22T21:13:24.423652+0000","last_clean":"2024-05-22T21:13:24.423652+0000","last_became_active":"2024-05-22T21:13:23.425830+0000","last_became_peered":"2024-05-22T21:13:23.425830+0000","last_unstale":"2024-05-22T21:13:24.423652+0000","last_undegraded":"2024-05-22T21:13:24.423652+0000","last_fullsized":"2024-05-22T21:13:24.423652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:27.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919876+0000","last_change":"2024-05-22T21:13:23.442938+0000","last_active":"2024-05-22T21:13:24.919876+0000","last_peered":"2024-05-22T21:13:24.919876+0000","last_clean":"2024-05-22T21:13:24.919876+0000","last_became_active":"2024-05-22T21:13:23.441874+0000","last_became_peered":"2024-05-22T21:13:23.441874+0000","last_unstale":"2024-05-22T21:13:24.919876+0000","last_undegraded":"2024-05-22T21:13:24.919876+0000","last_fullsized":"2024-05-22T21:13:24.919876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:57:10.746221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919437+0000","last_change":"2024-05-22T21:13:23.444504+0000","last_active":"2024-05-22T21:13:24.919437+0000","last_peered":"2024-05-22T21:13:24.919437+0000","last_clean":"2024-05-22T21:13:24.919437+0000","last_became_active":"2024-05-22T21:13:23.443892+0000","last_became_peered":"2024-05-22T21:13:23.443892+0000","last_unstale":"2024-05-22T21:13:24.919437+0000","last_undegraded":"2024-05-22T21:13:24.919437+0000","last_fullsized":"2024-05-22T21:13:24.919437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:56:26.375232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014525+0000","last_change":"2024-05-22T21:13:23.447393+0000","last_active":"2024-05-22T21:13:25.014525+0000","last_peered":"2024-05-22T21:13:25.014525+0000","last_clean":"2024-05-22T21:13:25.014525+0000","last_became_active":"2024-05-22T21:13:23.446956+0000","last_became_peered":"2024-05-22T21:13:23.446956+0000","last_unstale":"2024-05-22T21:13:25.014525+0000","last_undegraded":"2024-05-22T21:13:25.014525+0000","last_fullsized":"2024-05-22T21:13:25.014525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:34:49.902120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428827+0000","last_change":"2024-05-22T21:13:23.437025+0000","last_active":"2024-05-22T21:13:24.428827+0000","last_peered":"2024-05-22T21:13:24.428827+0000","last_clean":"2024-05-22T21:13:24.428827+0000","last_became_active":"2024-05-22T21:13:23.436786+0000","last_became_peered":"2024-05-22T21:13:23.436786+0000","last_unstale":"2024-05-22T21:13:24.428827+0000","last_undegraded":"2024-05-22T21:13:24.428827+0000","last_fullsized":"2024-05-22T21:13:24.428827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:04:32.035089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014759+0000","last_change":"2024-05-22T21:13:23.444119+0000","last_active":"2024-05-22T21:13:25.014759+0000","last_peered":"2024-05-22T21:13:25.014759+0000","last_clean":"2024-05-22T21:13:25.014759+0000","last_became_active":"2024-05-22T21:13:23.443861+0000","last_became_peered":"2024-05-22T21:13:23.443861+0000","last_unstale":"2024-05-22T21:13:25.014759+0000","last_undegraded":"2024-05-22T21:13:25.014759+0000","last_fullsized":"2024-05-22T21:13:25.014759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:00:43.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015860+0000","last_change":"2024-05-22T21:13:23.434203+0000","last_active":"2024-05-22T21:13:25.015860+0000","last_peered":"2024-05-22T21:13:25.015860+0000","last_clean":"2024-05-22T21:13:25.015860+0000","last_became_active":"2024-05-22T21:13:23.433871+0000","last_became_peered":"2024-05-22T21:13:23.433871+0000","last_unstale":"2024-05-22T21:13:25.015860+0000","last_undegraded":"2024-05-22T21:13:25.015860+0000","last_fullsized":"2024-05-22T21:13:25.015860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:00:43.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919633+0000","last_change":"2024-05-22T21:13:23.444258+0000","last_active":"2024-05-22T21:13:24.919633+0000","last_peered":"2024-05-22T21:13:24.919633+0000","last_clean":"2024-05-22T21:13:24.919633+0000","last_became_active":"2024-05-22T21:13:23.443329+0000","last_became_peered":"2024-05-22T21:13:23.443329+0000","last_unstale":"2024-05-22T21:13:24.919633+0000","last_undegraded":"2024-05-22T21:13:24.919633+0000","last_fullsized":"2024-05-22T21:13:24.919633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015111+0000","last_change":"2024-05-22T21:13:23.437489+0000","last_active":"2024-05-22T21:13:25.015111+0000","last_peered":"2024-05-22T21:13:25.015111+0000","last_clean":"2024-05-22T21:13:25.015111+0000","last_became_active":"2024-05-22T21:13:23.436877+0000","last_became_peered":"2024-05-22T21:13:23.436877+0000","last_unstale":"2024-05-22T21:13:25.015111+0000","last_undegraded":"2024-05-22T21:13:25.015111+0000","last_fullsized":"2024-05-22T21:13:25.015111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:56:26.375232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919626+0000","last_change":"2024-05-22T21:13:23.434147+0000","last_active":"2024-05-22T21:13:24.919626+0000","last_peered":"2024-05-22T21:13:24.919626+0000","last_clean":"2024-05-22T21:13:24.919626+0000","last_became_active":"2024-05-22T21:13:23.433529+0000","last_became_peered":"2024-05-22T21:13:23.433529+0000","last_unstale":"2024-05-22T21:13:24.919626+0000","last_undegraded":"2024-05-22T21:13:24.919626+0000","last_fullsized":"2024-05-22T21:13:24.919626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:44:04.542487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015414+0000","last_change":"2024-05-22T21:13:23.435603+0000","last_active":"2024-05-22T21:13:25.015414+0000","last_peered":"2024-05-22T21:13:25.015414+0000","last_clean":"2024-05-22T21:13:25.015414+0000","last_became_active":"2024-05-22T21:13:23.435387+0000","last_became_peered":"2024-05-22T21:13:23.435387+0000","last_unstale":"2024-05-22T21:13:25.015414+0000","last_undegraded":"2024-05-22T21:13:25.015414+0000","last_fullsized":"2024-05-22T21:13:25.015414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:27.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014667+0000","last_change":"2024-05-22T21:13:23.444856+0000","last_active":"2024-05-22T21:13:25.014667+0000","last_peered":"2024-05-22T21:13:25.014667+0000","last_clean":"2024-05-22T21:13:25.014667+0000","last_became_active":"2024-05-22T21:13:23.444640+0000","last_became_peered":"2024-05-22T21:13:23.444640+0000","last_unstale":"2024-05-22T21:13:25.014667+0000","last_undegraded":"2024-05-22T21:13:25.014667+0000","last_fullsized":"2024-05-22T21:13:25.014667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:07:57.351276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423445+0000","last_change":"2024-05-22T21:13:23.430546+0000","last_active":"2024-05-22T21:13:24.423445+0000","last_peered":"2024-05-22T21:13:24.423445+0000","last_clean":"2024-05-22T21:13:24.423445+0000","last_became_active":"2024-05-22T21:13:23.430306+0000","last_became_peered":"2024-05-22T21:13:23.430306+0000","last_unstale":"2024-05-22T21:13:24.423445+0000","last_undegraded":"2024-05-22T21:13:24.423445+0000","last_fullsized":"2024-05-22T21:13:24.423445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:06.303665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015209+0000","last_change":"2024-05-22T21:13:23.437362+0000","last_active":"2024-05-22T21:13:25.015209+0000","last_peered":"2024-05-22T21:13:25.015209+0000","last_clean":"2024-05-22T21:13:25.015209+0000","last_became_active":"2024-05-22T21:13:23.436580+0000","last_became_peered":"2024-05-22T21:13:23.436580+0000","last_unstale":"2024-05-22T21:13:25.015209+0000","last_undegraded":"2024-05-22T21:13:25.015209+0000","last_fullsized":"2024-05-22T21:13:25.015209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:06.303665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015001+0000","last_change":"2024-05-22T21:13:23.433616+0000","last_active":"2024-05-22T21:13:25.015001+0000","last_peered":"2024-05-22T21:13:25.015001+0000","last_clean":"2024-05-22T21:13:25.015001+0000","last_became_active":"2024-05-22T21:13:23.433258+0000","last_became_peered":"2024-05-22T21:13:23.433258+0000","last_unstale":"2024-05-22T21:13:25.015001+0000","last_undegraded":"2024-05-22T21:13:25.015001+0000","last_fullsized":"2024-05-22T21:13:25.015001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:44:56.029080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919662+0000","last_change":"2024-05-22T21:13:23.435441+0000","last_active":"2024-05-22T21:13:24.919662+0000","last_peered":"2024-05-22T21:13:24.919662+0000","last_clean":"2024-05-22T21:13:24.919662+0000","last_became_active":"2024-05-22T21:13:23.435225+0000","last_became_peered":"2024-05-22T21:13:23.435225+0000","last_unstale":"2024-05-22T21:13:24.919662+0000","last_undegraded":"2024-05-22T21:13:24.919662+0000","last_fullsized":"2024-05-22T21:13:24.919662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:44:56.029080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014612+0000","last_change":"2024-05-22T21:13:23.446437+0000","last_active":"2024-05-22T21:13:25.014612+0000","last_peered":"2024-05-22T21:13:25.014612+0000","last_clean":"2024-05-22T21:13:25.014612+0000","last_became_active":"2024-05-22T21:13:23.446219+0000","last_became_peered":"2024-05-22T21:13:23.446219+0000","last_unstale":"2024-05-22T21:13:25.014612+0000","last_undegraded":"2024-05-22T21:13:25.014612+0000","last_fullsized":"2024-05-22T21:13:25.014612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:57:10.070412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014588+0000","last_change":"2024-05-22T21:13:23.443792+0000","last_active":"2024-05-22T21:13:25.014588+0000","last_peered":"2024-05-22T21:13:25.014588+0000","last_clean":"2024-05-22T21:13:25.014588+0000","last_became_active":"2024-05-22T21:13:23.443581+0000","last_became_peered":"2024-05-22T21:13:23.443581+0000","last_unstale":"2024-05-22T21:13:25.014588+0000","last_undegraded":"2024-05-22T21:13:25.014588+0000","last_fullsized":"2024-05-22T21:13:25.014588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.430114+0000","last_change":"2024-05-22T21:13:23.429085+0000","last_active":"2024-05-22T21:13:24.430114+0000","last_peered":"2024-05-22T21:13:24.430114+0000","last_clean":"2024-05-22T21:13:24.430114+0000","last_became_active":"2024-05-22T21:13:23.428826+0000","last_became_peered":"2024-05-22T21:13:23.428826+0000","last_unstale":"2024-05-22T21:13:24.430114+0000","last_undegraded":"2024-05-22T21:13:24.430114+0000","last_fullsized":"2024-05-22T21:13:24.430114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:17.109310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015138+0000","last_change":"2024-05-22T21:13:23.053644+0000","last_active":"2024-05-22T21:13:25.015138+0000","last_peered":"2024-05-22T21:13:25.015138+0000","last_clean":"2024-05-22T21:13:25.015138+0000","last_became_active":"2024-05-22T21:13:23.053129+0000","last_became_peered":"2024-05-22T21:13:23.053129+0000","last_unstale":"2024-05-22T21:13:25.015138+0000","last_undegraded":"2024-05-22T21:13:25.015138+0000","last_fullsized":"2024-05-22T21:13:25.015138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:02.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015372+0000","last_change":"2024-05-22T21:13:23.430851+0000","last_active":"2024-05-22T21:13:25.015372+0000","last_peered":"2024-05-22T21:13:25.015372+0000","last_clean":"2024-05-22T21:13:25.015372+0000","last_became_active":"2024-05-22T21:13:23.430614+0000","last_became_peered":"2024-05-22T21:13:23.430614+0000","last_unstale":"2024-05-22T21:13:25.015372+0000","last_undegraded":"2024-05-22T21:13:25.015372+0000","last_fullsized":"2024-05-22T21:13:25.015372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:14:11.125539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014815+0000","last_change":"2024-05-22T21:13:23.052090+0000","last_active":"2024-05-22T21:13:25.014815+0000","last_peered":"2024-05-22T21:13:25.014815+0000","last_clean":"2024-05-22T21:13:25.014815+0000","last_became_active":"2024-05-22T21:13:23.051762+0000","last_became_peered":"2024-05-22T21:13:23.051762+0000","last_unstale":"2024-05-22T21:13:25.014815+0000","last_undegraded":"2024-05-22T21:13:25.014815+0000","last_fullsized":"2024-05-22T21:13:25.014815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:54.533292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015255+0000","last_change":"2024-05-22T21:13:23.053832+0000","last_active":"2024-05-22T21:13:25.015255+0000","last_peered":"2024-05-22T21:13:25.015255+0000","last_clean":"2024-05-22T21:13:25.015255+0000","last_became_active":"2024-05-22T21:13:23.053170+0000","last_became_peered":"2024-05-22T21:13:23.053170+0000","last_unstale":"2024-05-22T21:13:25.015255+0000","last_undegraded":"2024-05-22T21:13:25.015255+0000","last_fullsized":"2024-05-22T21:13:25.015255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015495+0000","last_change":"2024-05-22T21:13:23.429829+0000","last_active":"2024-05-22T21:13:25.015495+0000","last_peered":"2024-05-22T21:13:25.015495+0000","last_clean":"2024-05-22T21:13:25.015495+0000","last_became_active":"2024-05-22T21:13:23.429616+0000","last_became_peered":"2024-05-22T21:13:23.429616+0000","last_unstale":"2024-05-22T21:13:25.015495+0000","last_undegraded":"2024-05-22T21:13:25.015495+0000","last_fullsized":"2024-05-22T21:13:25.015495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:00:48.798418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919805+0000","last_change":"2024-05-22T21:13:23.056945+0000","last_active":"2024-05-22T21:13:24.919805+0000","last_peered":"2024-05-22T21:13:24.919805+0000","last_clean":"2024-05-22T21:13:24.919805+0000","last_became_active":"2024-05-22T21:13:23.056643+0000","last_became_peered":"2024-05-22T21:13:23.056643+0000","last_unstale":"2024-05-22T21:13:24.919805+0000","last_undegraded":"2024-05-22T21:13:24.919805+0000","last_fullsized":"2024-05-22T21:13:24.919805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:02.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423490+0000","last_change":"2024-05-22T21:13:23.429959+0000","last_active":"2024-05-22T21:13:24.423490+0000","last_peered":"2024-05-22T21:13:24.423490+0000","last_clean":"2024-05-22T21:13:24.423490+0000","last_became_active":"2024-05-22T21:13:23.429752+0000","last_became_peered":"2024-05-22T21:13:23.429752+0000","last_unstale":"2024-05-22T21:13:24.423490+0000","last_undegraded":"2024-05-22T21:13:24.423490+0000","last_fullsized":"2024-05-22T21:13:24.423490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:14:11.125539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014851+0000","last_change":"2024-05-22T21:13:23.058878+0000","last_active":"2024-05-22T21:13:25.014851+0000","last_peered":"2024-05-22T21:13:25.014851+0000","last_clean":"2024-05-22T21:13:25.014851+0000","last_became_active":"2024-05-22T21:13:23.058648+0000","last_became_peered":"2024-05-22T21:13:23.058648+0000","last_unstale":"2024-05-22T21:13:25.014851+0000","last_undegraded":"2024-05-22T21:13:25.014851+0000","last_fullsized":"2024-05-22T21:13:25.014851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919552+0000","last_change":"2024-05-22T21:13:23.444383+0000","last_active":"2024-05-22T21:13:24.919552+0000","last_peered":"2024-05-22T21:13:24.919552+0000","last_clean":"2024-05-22T21:13:24.919552+0000","last_became_active":"2024-05-22T21:13:23.443612+0000","last_became_peered":"2024-05-22T21:13:23.443612+0000","last_unstale":"2024-05-22T21:13:24.919552+0000","last_undegraded":"2024-05-22T21:13:24.919552+0000","last_fullsized":"2024-05-22T21:13:24.919552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:27.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919883+0000","last_change":"2024-05-22T21:13:23.056793+0000","last_active":"2024-05-22T21:13:24.919883+0000","last_peered":"2024-05-22T21:13:24.919883+0000","last_clean":"2024-05-22T21:13:24.919883+0000","last_became_active":"2024-05-22T21:13:23.056450+0000","last_became_peered":"2024-05-22T21:13:23.056450+0000","last_unstale":"2024-05-22T21:13:24.919883+0000","last_undegraded":"2024-05-22T21:13:24.919883+0000","last_fullsized":"2024-05-22T21:13:24.919883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:34.410142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015942+0000","last_change":"2024-05-22T21:13:23.446398+0000","last_active":"2024-05-22T21:13:25.015942+0000","last_peered":"2024-05-22T21:13:25.015942+0000","last_clean":"2024-05-22T21:13:25.015942+0000","last_became_active":"2024-05-22T21:13:23.445907+0000","last_became_peered":"2024-05-22T21:13:23.445907+0000","last_unstale":"2024-05-22T21:13:25.015942+0000","last_undegraded":"2024-05-22T21:13:25.015942+0000","last_fullsized":"2024-05-22T21:13:25.015942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:06.303665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423552+0000","last_change":"2024-05-22T21:13:23.056570+0000","last_active":"2024-05-22T21:13:24.423552+0000","last_peered":"2024-05-22T21:13:24.423552+0000","last_clean":"2024-05-22T21:13:24.423552+0000","last_became_active":"2024-05-22T21:13:23.056357+0000","last_became_peered":"2024-05-22T21:13:23.056357+0000","last_unstale":"2024-05-22T21:13:24.423552+0000","last_undegraded":"2024-05-22T21:13:24.423552+0000","last_fullsized":"2024-05-22T21:13:24.423552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:30.101472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428611+0000","last_change":"2024-05-22T21:13:23.439128+0000","last_active":"2024-05-22T21:13:24.428611+0000","last_peered":"2024-05-22T21:13:24.428611+0000","last_clean":"2024-05-22T21:13:24.428611+0000","last_became_active":"2024-05-22T21:13:23.438898+0000","last_became_peered":"2024-05-22T21:13:23.438898+0000","last_unstale":"2024-05-22T21:13:24.428611+0000","last_undegraded":"2024-05-22T21:13:24.428611+0000","last_fullsized":"2024-05-22T21:13:24.428611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:51:29.569811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014610+0000","last_change":"2024-05-22T21:13:23.049600+0000","last_active":"2024-05-22T21:13:25.014610+0000","last_peered":"2024-05-22T21:13:25.014610+0000","last_clean":"2024-05-22T21:13:25.014610+0000","last_became_active":"2024-05-22T21:13:23.049395+0000","last_became_peered":"2024-05-22T21:13:23.049395+0000","last_unstale":"2024-05-22T21:13:25.014610+0000","last_undegraded":"2024-05-22T21:13:25.014610+0000","last_fullsized":"2024-05-22T21:13:25.014610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:24.879526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429576+0000","last_change":"2024-05-22T21:13:23.439228+0000","last_active":"2024-05-22T21:13:24.429576+0000","last_peered":"2024-05-22T21:13:24.429576+0000","last_clean":"2024-05-22T21:13:24.429576+0000","last_became_active":"2024-05-22T21:13:23.437292+0000","last_became_peered":"2024-05-22T21:13:23.437292+0000","last_unstale":"2024-05-22T21:13:24.429576+0000","last_undegraded":"2024-05-22T21:13:24.429576+0000","last_fullsized":"2024-05-22T21:13:24.429576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:44:04.542487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015531+0000","last_change":"2024-05-22T21:13:23.052267+0000","last_active":"2024-05-22T21:13:25.015531+0000","last_peered":"2024-05-22T21:13:25.015531+0000","last_clean":"2024-05-22T21:13:25.015531+0000","last_became_active":"2024-05-22T21:13:23.051663+0000","last_became_peered":"2024-05-22T21:13:23.051663+0000","last_unstale":"2024-05-22T21:13:25.015531+0000","last_undegraded":"2024-05-22T21:13:25.015531+0000","last_fullsized":"2024-05-22T21:13:25.015531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:54.533292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015989+0000","last_change":"2024-05-22T21:13:23.434082+0000","last_active":"2024-05-22T21:13:25.015989+0000","last_peered":"2024-05-22T21:13:25.015989+0000","last_clean":"2024-05-22T21:13:25.015989+0000","last_became_active":"2024-05-22T21:13:23.433591+0000","last_became_peered":"2024-05-22T21:13:23.433591+0000","last_unstale":"2024-05-22T21:13:25.015989+0000","last_undegraded":"2024-05-22T21:13:25.015989+0000","last_fullsized":"2024-05-22T21:13:25.015989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:08:18.923855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.920207+0000","last_change":"2024-05-22T21:13:23.058203+0000","last_active":"2024-05-22T21:13:24.920207+0000","last_peered":"2024-05-22T21:13:24.920207+0000","last_clean":"2024-05-22T21:13:24.920207+0000","last_became_active":"2024-05-22T21:13:23.057230+0000","last_became_peered":"2024-05-22T21:13:23.057230+0000","last_unstale":"2024-05-22T21:13:24.920207+0000","last_undegraded":"2024-05-22T21:13:24.920207+0000","last_fullsized":"2024-05-22T21:13:24.920207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:16.101596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429362+0000","last_change":"2024-05-22T21:13:23.437522+0000","last_active":"2024-05-22T21:13:24.429362+0000","last_peered":"2024-05-22T21:13:24.429362+0000","last_clean":"2024-05-22T21:13:24.429362+0000","last_became_active":"2024-05-22T21:13:23.436971+0000","last_became_peered":"2024-05-22T21:13:23.436971+0000","last_unstale":"2024-05-22T21:13:24.429362+0000","last_undegraded":"2024-05-22T21:13:24.429362+0000","last_fullsized":"2024-05-22T21:13:24.429362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:51:04.078274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014654+0000","last_change":"2024-05-22T21:13:23.056955+0000","last_active":"2024-05-22T21:13:25.014654+0000","last_peered":"2024-05-22T21:13:25.014654+0000","last_clean":"2024-05-22T21:13:25.014654+0000","last_became_active":"2024-05-22T21:13:23.056131+0000","last_became_peered":"2024-05-22T21:13:23.056131+0000","last_unstale":"2024-05-22T21:13:25.014654+0000","last_undegraded":"2024-05-22T21:13:25.014654+0000","last_fullsized":"2024-05-22T21:13:25.014654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:02.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014604+0000","last_change":"2024-05-22T21:13:23.445799+0000","last_active":"2024-05-22T21:13:25.014604+0000","last_peered":"2024-05-22T21:13:25.014604+0000","last_clean":"2024-05-22T21:13:25.014604+0000","last_became_active":"2024-05-22T21:13:23.445576+0000","last_became_peered":"2024-05-22T21:13:23.445576+0000","last_unstale":"2024-05-22T21:13:25.014604+0000","last_undegraded":"2024-05-22T21:13:25.014604+0000","last_fullsized":"2024-05-22T21:13:25.014604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:04:37.695966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014626+0000","last_change":"2024-05-22T21:13:23.046892+0000","last_active":"2024-05-22T21:13:25.014626+0000","last_peered":"2024-05-22T21:13:25.014626+0000","last_clean":"2024-05-22T21:13:25.014626+0000","last_became_active":"2024-05-22T21:13:23.046677+0000","last_became_peered":"2024-05-22T21:13:23.046677+0000","last_unstale":"2024-05-22T21:13:25.014626+0000","last_undegraded":"2024-05-22T21:13:25.014626+0000","last_fullsized":"2024-05-22T21:13:25.014626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:24.879526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429428+0000","last_change":"2024-05-22T21:13:23.437891+0000","last_active":"2024-05-22T21:13:24.429428+0000","last_peered":"2024-05-22T21:13:24.429428+0000","last_clean":"2024-05-22T21:13:24.429428+0000","last_became_active":"2024-05-22T21:13:23.436990+0000","last_became_peered":"2024-05-22T21:13:23.436990+0000","last_unstale":"2024-05-22T21:13:24.429428+0000","last_undegraded":"2024-05-22T21:13:24.429428+0000","last_fullsized":"2024-05-22T21:13:24.429428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:12:43.781854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428700+0000","last_change":"2024-05-22T21:13:23.054659+0000","last_active":"2024-05-22T21:13:24.428700+0000","last_peered":"2024-05-22T21:13:24.428700+0000","last_clean":"2024-05-22T21:13:24.428700+0000","last_became_active":"2024-05-22T21:13:23.054403+0000","last_became_peered":"2024-05-22T21:13:23.054403+0000","last_unstale":"2024-05-22T21:13:24.428700+0000","last_undegraded":"2024-05-22T21:13:24.428700+0000","last_fullsized":"2024-05-22T21:13:24.428700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:26:28.755372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429419+0000","last_change":"2024-05-22T21:13:23.437668+0000","last_active":"2024-05-22T21:13:24.429419+0000","last_peered":"2024-05-22T21:13:24.429419+0000","last_clean":"2024-05-22T21:13:24.429419+0000","last_became_active":"2024-05-22T21:13:23.437307+0000","last_became_peered":"2024-05-22T21:13:23.437307+0000","last_unstale":"2024-05-22T21:13:24.429419+0000","last_undegraded":"2024-05-22T21:13:24.429419+0000","last_fullsized":"2024-05-22T21:13:24.429419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:57:11.273220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014824+0000","last_change":"2024-05-22T21:13:23.047615+0000","last_active":"2024-05-22T21:13:25.014824+0000","last_peered":"2024-05-22T21:13:25.014824+0000","last_clean":"2024-05-22T21:13:25.014824+0000","last_became_active":"2024-05-22T21:13:23.047358+0000","last_became_peered":"2024-05-22T21:13:23.047358+0000","last_unstale":"2024-05-22T21:13:25.014824+0000","last_undegraded":"2024-05-22T21:13:25.014824+0000","last_fullsized":"2024-05-22T21:13:25.014824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:34.410142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428730+0000","last_change":"2024-05-22T21:13:23.443073+0000","last_active":"2024-05-22T21:13:24.428730+0000","last_peered":"2024-05-22T21:13:24.428730+0000","last_clean":"2024-05-22T21:13:24.428730+0000","last_became_active":"2024-05-22T21:13:23.442751+0000","last_became_peered":"2024-05-22T21:13:23.442751+0000","last_unstale":"2024-05-22T21:13:24.428730+0000","last_undegraded":"2024-05-22T21:13:24.428730+0000","last_fullsized":"2024-05-22T21:13:24.428730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:09:20.599937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423443+0000","last_change":"2024-05-22T21:13:23.050892+0000","last_active":"2024-05-22T21:13:24.423443+0000","last_peered":"2024-05-22T21:13:24.423443+0000","last_clean":"2024-05-22T21:13:24.423443+0000","last_became_active":"2024-05-22T21:13:23.050565+0000","last_became_peered":"2024-05-22T21:13:23.050565+0000","last_unstale":"2024-05-22T21:13:24.423443+0000","last_undegraded":"2024-05-22T21:13:24.423443+0000","last_fullsized":"2024-05-22T21:13:24.423443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:16.101596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919723+0000","last_change":"2024-05-22T21:13:23.437191+0000","last_active":"2024-05-22T21:13:24.919723+0000","last_peered":"2024-05-22T21:13:24.919723+0000","last_clean":"2024-05-22T21:13:24.919723+0000","last_became_active":"2024-05-22T21:13:23.436983+0000","last_became_peered":"2024-05-22T21:13:23.436983+0000","last_unstale":"2024-05-22T21:13:24.919723+0000","last_undegraded":"2024-05-22T21:13:24.919723+0000","last_fullsized":"2024-05-22T21:13:24.919723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:06.303665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014631+0000","last_change":"2024-05-22T21:13:23.056461+0000","last_active":"2024-05-22T21:13:25.014631+0000","last_peered":"2024-05-22T21:13:25.014631+0000","last_clean":"2024-05-22T21:13:25.014631+0000","last_became_active":"2024-05-22T21:13:23.056237+0000","last_became_peered":"2024-05-22T21:13:23.056237+0000","last_unstale":"2024-05-22T21:13:25.014631+0000","last_undegraded":"2024-05-22T21:13:25.014631+0000","last_fullsized":"2024-05-22T21:13:25.014631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428905+0000","last_change":"2024-05-22T21:13:23.448377+0000","last_active":"2024-05-22T21:13:24.428905+0000","last_peered":"2024-05-22T21:13:24.428905+0000","last_clean":"2024-05-22T21:13:24.428905+0000","last_became_active":"2024-05-22T21:13:23.447835+0000","last_became_peered":"2024-05-22T21:13:23.447835+0000","last_unstale":"2024-05-22T21:13:24.428905+0000","last_undegraded":"2024-05-22T21:13:24.428905+0000","last_fullsized":"2024-05-22T21:13:24.428905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:44:10.866789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-22T21:13:23.435118+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:13:23.435118+0000","last_peered":"2024-05-22T21:13:23.435118+0000","last_clean":"2024-05-22T21:13:23.435118+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:13:23.435118+0000","last_undegraded":"2024-05-22T21:13:23.435118+0000","last_fullsized":"2024-05-22T21:13:23.435118+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.920353+0000","last_change":"2024-05-22T21:13:23.058027+0000","last_active":"2024-05-22T21:13:24.920353+0000","last_peered":"2024-05-22T21:13:24.920353+0000","last_clean":"2024-05-22T21:13:24.920353+0000","last_became_active":"2024-05-22T21:13:23.056401+0000","last_became_peered":"2024-05-22T21:13:23.056401+0000","last_unstale":"2024-05-22T21:13:24.920353+0000","last_undegraded":"2024-05-22T21:13:24.920353+0000","last_fullsized":"2024-05-22T21:13:24.920353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:24.879526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014947+0000","last_change":"2024-05-22T21:13:23.440325+0000","last_active":"2024-05-22T21:13:25.014947+0000","last_peered":"2024-05-22T21:13:25.014947+0000","last_clean":"2024-05-22T21:13:25.014947+0000","last_became_active":"2024-05-22T21:13:23.440020+0000","last_became_peered":"2024-05-22T21:13:23.440020+0000","last_unstale":"2024-05-22T21:13:25.014947+0000","last_undegraded":"2024-05-22T21:13:25.014947+0000","last_fullsized":"2024-05-22T21:13:25.014947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:38:04.537935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429005+0000","last_change":"2024-05-22T21:13:23.050452+0000","last_active":"2024-05-22T21:13:24.429005+0000","last_peered":"2024-05-22T21:13:24.429005+0000","last_clean":"2024-05-22T21:13:24.429005+0000","last_became_active":"2024-05-22T21:13:23.049982+0000","last_became_peered":"2024-05-22T21:13:23.049982+0000","last_unstale":"2024-05-22T21:13:24.429005+0000","last_undegraded":"2024-05-22T21:13:24.429005+0000","last_fullsized":"2024-05-22T21:13:24.429005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:57:09.062698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015492+0000","last_change":"2024-05-22T21:13:23.434429+0000","last_active":"2024-05-22T21:13:25.015492+0000","last_peered":"2024-05-22T21:13:25.015492+0000","last_clean":"2024-05-22T21:13:25.015492+0000","last_became_active":"2024-05-22T21:13:23.433640+0000","last_became_peered":"2024-05-22T21:13:23.433640+0000","last_unstale":"2024-05-22T21:13:25.015492+0000","last_undegraded":"2024-05-22T21:13:25.015492+0000","last_fullsized":"2024-05-22T21:13:25.015492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014707+0000","last_change":"2024-05-22T21:13:23.051179+0000","last_active":"2024-05-22T21:13:25.014707+0000","last_peered":"2024-05-22T21:13:25.014707+0000","last_clean":"2024-05-22T21:13:25.014707+0000","last_became_active":"2024-05-22T21:13:23.050967+0000","last_became_peered":"2024-05-22T21:13:23.050967+0000","last_unstale":"2024-05-22T21:13:25.014707+0000","last_undegraded":"2024-05-22T21:13:25.014707+0000","last_fullsized":"2024-05-22T21:13:25.014707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:02.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423581+0000","last_change":"2024-05-22T21:13:23.426627+0000","last_active":"2024-05-22T21:13:24.423581+0000","last_peered":"2024-05-22T21:13:24.423581+0000","last_clean":"2024-05-22T21:13:24.423581+0000","last_became_active":"2024-05-22T21:13:23.426412+0000","last_became_peered":"2024-05-22T21:13:23.426412+0000","last_unstale":"2024-05-22T21:13:24.423581+0000","last_undegraded":"2024-05-22T21:13:24.423581+0000","last_fullsized":"2024-05-22T21:13:24.423581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919933+0000","last_change":"2024-05-22T21:13:23.055924+0000","last_active":"2024-05-22T21:13:24.919933+0000","last_peered":"2024-05-22T21:13:24.919933+0000","last_clean":"2024-05-22T21:13:24.919933+0000","last_became_active":"2024-05-22T21:13:23.055577+0000","last_became_peered":"2024-05-22T21:13:23.055577+0000","last_unstale":"2024-05-22T21:13:24.919933+0000","last_undegraded":"2024-05-22T21:13:24.919933+0000","last_fullsized":"2024-05-22T21:13:24.919933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.430340+0000","last_change":"2024-05-22T21:13:23.428954+0000","last_active":"2024-05-22T21:13:24.430340+0000","last_peered":"2024-05-22T21:13:24.430340+0000","last_clean":"2024-05-22T21:13:24.430340+0000","last_became_active":"2024-05-22T21:13:23.428628+0000","last_became_peered":"2024-05-22T21:13:23.428628+0000","last_unstale":"2024-05-22T21:13:24.430340+0000","last_undegraded":"2024-05-22T21:13:24.430340+0000","last_fullsized":"2024-05-22T21:13:24.430340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:31.109187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423797+0000","last_change":"2024-05-22T21:13:23.055046+0000","last_active":"2024-05-22T21:13:24.423797+0000","last_peered":"2024-05-22T21:13:24.423797+0000","last_clean":"2024-05-22T21:13:24.423797+0000","last_became_active":"2024-05-22T21:13:23.054670+0000","last_became_peered":"2024-05-22T21:13:23.054670+0000","last_unstale":"2024-05-22T21:13:24.423797+0000","last_undegraded":"2024-05-22T21:13:24.423797+0000","last_fullsized":"2024-05-22T21:13:24.423797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:24.879526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919722+0000","last_change":"2024-05-22T21:13:23.434008+0000","last_active":"2024-05-22T21:13:24.919722+0000","last_peered":"2024-05-22T21:13:24.919722+0000","last_clean":"2024-05-22T21:13:24.919722+0000","last_became_active":"2024-05-22T21:13:23.433233+0000","last_became_peered":"2024-05-22T21:13:23.433233+0000","last_unstale":"2024-05-22T21:13:24.919722+0000","last_undegraded":"2024-05-22T21:13:24.919722+0000","last_fullsized":"2024-05-22T21:13:24.919722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:38:04.537935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428911+0000","last_change":"2024-05-22T21:13:23.041563+0000","last_active":"2024-05-22T21:13:24.428911+0000","last_peered":"2024-05-22T21:13:24.428911+0000","last_clean":"2024-05-22T21:13:24.428911+0000","last_became_active":"2024-05-22T21:13:23.041302+0000","last_became_peered":"2024-05-22T21:13:23.041302+0000","last_unstale":"2024-05-22T21:13:24.428911+0000","last_undegraded":"2024-05-22T21:13:24.428911+0000","last_fullsized":"2024-05-22T21:13:24.428911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:51:28.562097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423568+0000","last_change":"2024-05-22T21:13:23.426584+0000","last_active":"2024-05-22T21:13:24.423568+0000","last_peered":"2024-05-22T21:13:24.423568+0000","last_clean":"2024-05-22T21:13:24.423568+0000","last_became_active":"2024-05-22T21:13:23.426367+0000","last_became_peered":"2024-05-22T21:13:23.426367+0000","last_unstale":"2024-05-22T21:13:24.423568+0000","last_undegraded":"2024-05-22T21:13:24.423568+0000","last_fullsized":"2024-05-22T21:13:24.423568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:07:36.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423622+0000","last_change":"2024-05-22T21:13:23.049729+0000","last_active":"2024-05-22T21:13:24.423622+0000","last_peered":"2024-05-22T21:13:24.423622+0000","last_clean":"2024-05-22T21:13:24.423622+0000","last_became_active":"2024-05-22T21:13:23.049480+0000","last_became_peered":"2024-05-22T21:13:23.049480+0000","last_unstale":"2024-05-22T21:13:24.423622+0000","last_undegraded":"2024-05-22T21:13:24.423622+0000","last_fullsized":"2024-05-22T21:13:24.423622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:51:28.562097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014848+0000","last_change":"2024-05-22T21:13:23.437853+0000","last_active":"2024-05-22T21:13:25.014848+0000","last_peered":"2024-05-22T21:13:25.014848+0000","last_clean":"2024-05-22T21:13:25.014848+0000","last_became_active":"2024-05-22T21:13:23.437676+0000","last_became_peered":"2024-05-22T21:13:23.437676+0000","last_unstale":"2024-05-22T21:13:25.014848+0000","last_undegraded":"2024-05-22T21:13:25.014848+0000","last_fullsized":"2024-05-22T21:13:25.014848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:14:11.125539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014750+0000","last_change":"2024-05-22T21:13:23.052969+0000","last_active":"2024-05-22T21:13:25.014750+0000","last_peered":"2024-05-22T21:13:25.014750+0000","last_clean":"2024-05-22T21:13:25.014750+0000","last_became_active":"2024-05-22T21:13:23.052306+0000","last_became_peered":"2024-05-22T21:13:23.052306+0000","last_unstale":"2024-05-22T21:13:25.014750+0000","last_undegraded":"2024-05-22T21:13:25.014750+0000","last_fullsized":"2024-05-22T21:13:25.014750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:34.410142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015367+0000","last_change":"2024-05-22T21:13:23.434301+0000","last_active":"2024-05-22T21:13:25.015367+0000","last_peered":"2024-05-22T21:13:25.015367+0000","last_clean":"2024-05-22T21:13:25.015367+0000","last_became_active":"2024-05-22T21:13:23.432584+0000","last_became_peered":"2024-05-22T21:13:23.432584+0000","last_unstale":"2024-05-22T21:13:25.015367+0000","last_undegraded":"2024-05-22T21:13:25.015367+0000","last_fullsized":"2024-05-22T21:13:25.015367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:53:57.578626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429187+0000","last_change":"2024-05-22T21:13:23.047297+0000","last_active":"2024-05-22T21:13:24.429187+0000","last_peered":"2024-05-22T21:13:24.429187+0000","last_clean":"2024-05-22T21:13:24.429187+0000","last_became_active":"2024-05-22T21:13:23.047021+0000","last_became_peered":"2024-05-22T21:13:23.047021+0000","last_unstale":"2024-05-22T21:13:24.429187+0000","last_undegraded":"2024-05-22T21:13:24.429187+0000","last_fullsized":"2024-05-22T21:13:24.429187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:09.117068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429366+0000","last_change":"2024-05-22T21:13:23.434654+0000","last_active":"2024-05-22T21:13:24.429366+0000","last_peered":"2024-05-22T21:13:24.429366+0000","last_clean":"2024-05-22T21:13:24.429366+0000","last_became_active":"2024-05-22T21:13:23.433684+0000","last_became_peered":"2024-05-22T21:13:23.433684+0000","last_unstale":"2024-05-22T21:13:24.429366+0000","last_undegraded":"2024-05-22T21:13:24.429366+0000","last_fullsized":"2024-05-22T21:13:24.429366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:03.482406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423727+0000","last_change":"2024-05-22T21:13:23.054907+0000","last_active":"2024-05-22T21:13:24.423727+0000","last_peered":"2024-05-22T21:13:24.423727+0000","last_clean":"2024-05-22T21:13:24.423727+0000","last_became_active":"2024-05-22T21:13:23.054307+0000","last_became_peered":"2024-05-22T21:13:23.054307+0000","last_unstale":"2024-05-22T21:13:24.423727+0000","last_undegraded":"2024-05-22T21:13:24.423727+0000","last_fullsized":"2024-05-22T21:13:24.423727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:09.117068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015009+0000","last_change":"2024-05-22T21:13:23.429322+0000","last_active":"2024-05-22T21:13:25.015009+0000","last_peered":"2024-05-22T21:13:25.015009+0000","last_clean":"2024-05-22T21:13:25.015009+0000","last_became_active":"2024-05-22T21:13:23.429091+0000","last_became_peered":"2024-05-22T21:13:23.429091+0000","last_unstale":"2024-05-22T21:13:25.015009+0000","last_undegraded":"2024-05-22T21:13:25.015009+0000","last_fullsized":"2024-05-22T21:13:25.015009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:44:56.029080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423657+0000","last_change":"2024-05-22T21:13:23.052267+0000","last_active":"2024-05-22T21:13:24.423657+0000","last_peered":"2024-05-22T21:13:24.423657+0000","last_clean":"2024-05-22T21:13:24.423657+0000","last_became_active":"2024-05-22T21:13:23.052062+0000","last_became_peered":"2024-05-22T21:13:23.052062+0000","last_unstale":"2024-05-22T21:13:24.423657+0000","last_undegraded":"2024-05-22T21:13:24.423657+0000","last_fullsized":"2024-05-22T21:13:24.423657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:34:48.894406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015300+0000","last_change":"2024-05-22T21:13:23.433439+0000","last_active":"2024-05-22T21:13:25.015300+0000","last_peered":"2024-05-22T21:13:25.015300+0000","last_clean":"2024-05-22T21:13:25.015300+0000","last_became_active":"2024-05-22T21:13:23.433016+0000","last_became_peered":"2024-05-22T21:13:23.433016+0000","last_unstale":"2024-05-22T21:13:25.015300+0000","last_undegraded":"2024-05-22T21:13:25.015300+0000","last_fullsized":"2024-05-22T21:13:25.015300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429655+0000","last_change":"2024-05-22T21:13:23.052977+0000","last_active":"2024-05-22T21:13:24.429655+0000","last_peered":"2024-05-22T21:13:24.429655+0000","last_clean":"2024-05-22T21:13:24.429655+0000","last_became_active":"2024-05-22T21:13:23.052490+0000","last_became_peered":"2024-05-22T21:13:23.052490+0000","last_unstale":"2024-05-22T21:13:24.429655+0000","last_undegraded":"2024-05-22T21:13:24.429655+0000","last_fullsized":"2024-05-22T21:13:24.429655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:16.101596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014517+0000","last_change":"2024-05-22T21:13:23.446126+0000","last_active":"2024-05-22T21:13:25.014517+0000","last_peered":"2024-05-22T21:13:25.014517+0000","last_clean":"2024-05-22T21:13:25.014517+0000","last_became_active":"2024-05-22T21:13:23.444835+0000","last_became_peered":"2024-05-22T21:13:23.444835+0000","last_unstale":"2024-05-22T21:13:25.014517+0000","last_undegraded":"2024-05-22T21:13:25.014517+0000","last_fullsized":"2024-05-22T21:13:25.014517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:53:57.578626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919747+0000","last_change":"2024-05-22T21:13:23.432060+0000","last_active":"2024-05-22T21:13:24.919747+0000","last_peered":"2024-05-22T21:13:24.919747+0000","last_clean":"2024-05-22T21:13:24.919747+0000","last_became_active":"2024-05-22T21:13:23.431845+0000","last_became_peered":"2024-05-22T21:13:23.431845+0000","last_unstale":"2024-05-22T21:13:24.919747+0000","last_undegraded":"2024-05-22T21:13:24.919747+0000","last_fullsized":"2024-05-22T21:13:24.919747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:44:10.866789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429092+0000","last_change":"2024-05-22T21:13:23.058221+0000","last_active":"2024-05-22T21:13:24.429092+0000","last_peered":"2024-05-22T21:13:24.429092+0000","last_clean":"2024-05-22T21:13:24.429092+0000","last_became_active":"2024-05-22T21:13:23.058012+0000","last_became_peered":"2024-05-22T21:13:23.058012+0000","last_unstale":"2024-05-22T21:13:24.429092+0000","last_undegraded":"2024-05-22T21:13:24.429092+0000","last_fullsized":"2024-05-22T21:13:24.429092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:34:48.894406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.920057+0000","last_change":"2024-05-22T21:13:23.431267+0000","last_active":"2024-05-22T21:13:24.920057+0000","last_peered":"2024-05-22T21:13:24.920057+0000","last_clean":"2024-05-22T21:13:24.920057+0000","last_became_active":"2024-05-22T21:13:23.431054+0000","last_became_peered":"2024-05-22T21:13:23.431054+0000","last_unstale":"2024-05-22T21:13:24.920057+0000","last_undegraded":"2024-05-22T21:13:24.920057+0000","last_fullsized":"2024-05-22T21:13:24.920057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:51:04.078274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429474+0000","last_change":"2024-05-22T21:13:23.051743+0000","last_active":"2024-05-22T21:13:24.429474+0000","last_peered":"2024-05-22T21:13:24.429474+0000","last_clean":"2024-05-22T21:13:24.429474+0000","last_became_active":"2024-05-22T21:13:23.051564+0000","last_became_peered":"2024-05-22T21:13:23.051564+0000","last_unstale":"2024-05-22T21:13:24.429474+0000","last_undegraded":"2024-05-22T21:13:24.429474+0000","last_fullsized":"2024-05-22T21:13:24.429474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:30.101472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015432+0000","last_change":"2024-05-22T21:13:23.433871+0000","last_active":"2024-05-22T21:13:25.015432+0000","last_peered":"2024-05-22T21:13:25.015432+0000","last_clean":"2024-05-22T21:13:25.015432+0000","last_became_active":"2024-05-22T21:13:23.431365+0000","last_became_peered":"2024-05-22T21:13:23.431365+0000","last_unstale":"2024-05-22T21:13:25.015432+0000","last_undegraded":"2024-05-22T21:13:25.015432+0000","last_fullsized":"2024-05-22T21:13:25.015432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:27.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015179+0000","last_change":"2024-05-22T21:13:23.046747+0000","last_active":"2024-05-22T21:13:25.015179+0000","last_peered":"2024-05-22T21:13:25.015179+0000","last_clean":"2024-05-22T21:13:25.015179+0000","last_became_active":"2024-05-22T21:13:23.046523+0000","last_became_peered":"2024-05-22T21:13:23.046523+0000","last_unstale":"2024-05-22T21:13:25.015179+0000","last_undegraded":"2024-05-22T21:13:25.015179+0000","last_fullsized":"2024-05-22T21:13:25.015179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:34.410142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015087+0000","last_change":"2024-05-22T21:13:23.447604+0000","last_active":"2024-05-22T21:13:25.015087+0000","last_peered":"2024-05-22T21:13:25.015087+0000","last_clean":"2024-05-22T21:13:25.015087+0000","last_became_active":"2024-05-22T21:13:23.447391+0000","last_became_peered":"2024-05-22T21:13:23.447391+0000","last_unstale":"2024-05-22T21:13:25.015087+0000","last_undegraded":"2024-05-22T21:13:25.015087+0000","last_fullsized":"2024-05-22T21:13:25.015087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:07:36.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015546+0000","last_change":"2024-05-22T21:13:23.043385+0000","last_active":"2024-05-22T21:13:25.015546+0000","last_peered":"2024-05-22T21:13:25.015546+0000","last_clean":"2024-05-22T21:13:25.015546+0000","last_became_active":"2024-05-22T21:13:23.043036+0000","last_became_peered":"2024-05-22T21:13:23.043036+0000","last_unstale":"2024-05-22T21:13:25.015546+0000","last_undegraded":"2024-05-22T21:13:25.015546+0000","last_fullsized":"2024-05-22T21:13:25.015546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194661,"statfs":{"total":200000000,"available":199333641,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":75,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923027,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924501,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924465,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":75,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923027,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924591,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924431,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199334204,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]}],"pool_statfs":[]}} 2024-05-22T21:13:31.998 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:13:32.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.142+0000 7f71c5800640 1 Processor -- start 2024-05-22T21:13:32.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.142+0000 7f71c5800640 1 -- start start 2024-05-22T21:13:32.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.142+0000 7f71c5800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f71c00cd3e0 0x7f71c00cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.142+0000 7f71c5800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f71c00cf5c0 0x7f71c0069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.142+0000 7f71c5800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c00712f0 con 0x7f71c006a410 2024-05-22T21:13:32.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.142+0000 7f71c5800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c006a8a0 con 0x7f71c00cd3e0 2024-05-22T21:13:32.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.142+0000 7f71c5800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c00710e0 con 0x7f71c00cf5c0 2024-05-22T21:13:32.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.142+0000 7f71bfe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f71c00cd3e0 0x7f71c00cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.142+0000 7f71bfe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f71c00cd3e0 0x7f71c00cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49000/0 (socket says 172.21.15.161:49000) 2024-05-22T21:13:32.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.142+0000 7f71bfe00640 1 -- 172.21.15.161:0/543119147 learned_addr learned my addr 172.21.15.161:0/543119147 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:32.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.142+0000 7f71bf400640 1 --2- 172.21.15.161:0/543119147 >> v2:172.21.15.179:3300/0 conn(0x7f71c00cf5c0 0x7f71c0069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.142+0000 7f71bfe00640 1 -- 172.21.15.161:0/543119147 >> v2:172.21.15.179:3300/0 conn(0x7f71c00cf5c0 msgr2=0x7f71c0069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:32.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.142+0000 7f71bfe00640 1 --2- 172.21.15.161:0/543119147 >> v2:172.21.15.179:3300/0 conn(0x7f71c00cf5c0 0x7f71c0069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.142+0000 7f71bfe00640 1 -- 172.21.15.161:0/543119147 >> v1:172.21.15.179:6789/0 conn(0x7f71c006a410 legacy=0x7f71c00cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.142+0000 7f71bfe00640 1 -- 172.21.15.161:0/543119147 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c00715d0 con 0x7f71c00cd3e0 2024-05-22T21:13:32.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.146+0000 7f71bfe00640 1 --2- 172.21.15.161:0/543119147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c00cd3e0 0x7f71c00cf080 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f71c00b4140 tx=0x7f71b001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5652c8fd50faaf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:32.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.146+0000 7f71bea00640 1 -- 172.21.15.161:0/543119147 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b00ab600 con 0x7f71c00cd3e0 2024-05-22T21:13:32.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.146+0000 7f71bea00640 1 -- 172.21.15.161:0/543119147 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b00a7050 con 0x7f71c00cd3e0 2024-05-22T21:13:32.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.146+0000 7f71bea00640 1 -- 172.21.15.161:0/543119147 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b00ab980 con 0x7f71c00cd3e0 2024-05-22T21:13:32.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.146+0000 7f71c5800640 1 -- 172.21.15.161:0/543119147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c00cd3e0 msgr2=0x7f71c00cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.146+0000 7f71c5800640 1 --2- 172.21.15.161:0/543119147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c00cd3e0 0x7f71c00cf080 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f71c00b4140 tx=0x7f71b001c490 comp rx=0 tx=0).stop 2024-05-22T21:13:32.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.146+0000 7f71c5800640 1 -- 172.21.15.161:0/543119147 shutdown_connections 2024-05-22T21:13:32.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.146+0000 7f71c5800640 1 --2- 172.21.15.161:0/543119147 >> v2:172.21.15.179:3300/0 conn(0x7f71c00cf5c0 0x7f71c0069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.146+0000 7f71c5800640 1 --2- 172.21.15.161:0/543119147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c00cd3e0 0x7f71c00cf080 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.146+0000 7f71c5800640 1 -- 172.21.15.161:0/543119147 >> 172.21.15.161:0/543119147 conn(0x7f71c00cfcc0 msgr2=0x7f71c00d00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:32.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.146+0000 7f71c5800640 1 -- 172.21.15.161:0/543119147 shutdown_connections 2024-05-22T21:13:32.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.146+0000 7f71c5800640 1 -- 172.21.15.161:0/543119147 wait complete. 2024-05-22T21:13:32.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71c5800640 1 Processor -- start 2024-05-22T21:13:32.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71c5800640 1 -- start start 2024-05-22T21:13:32.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71c5800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f71c006a410 0x7f71c014d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71c5800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c00cd3e0 0x7f71c014deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71c5800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71c00cf5c0 0x7f71c014e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71c5800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c014fd20 con 0x7f71c00cd3e0 2024-05-22T21:13:32.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71c5800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c014da50 con 0x7f71c00cf5c0 2024-05-22T21:13:32.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71c5800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c014dc80 con 0x7f71c006a410 2024-05-22T21:13:32.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71bf400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c00cd3e0 0x7f71c014deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71bf400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c00cd3e0 0x7f71c014deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49010/0 (socket says 172.21.15.161:49010) 2024-05-22T21:13:32.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71bf400640 1 -- 172.21.15.161:0/2655057356 learned_addr learned my addr 172.21.15.161:0/2655057356 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:32.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71bfe00640 1 --2- 172.21.15.161:0/2655057356 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f71c006a410 0x7f71c014d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71c4e00640 1 --2- 172.21.15.161:0/2655057356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71c00cf5c0 0x7f71c014e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71bf400640 1 -- 172.21.15.161:0/2655057356 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f71c006a410 msgr2=0x7f71c014d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71bf400640 1 --2- 172.21.15.161:0/2655057356 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f71c006a410 0x7f71c014d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71bf400640 1 -- 172.21.15.161:0/2655057356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71c00cf5c0 msgr2=0x7f71c014e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71bf400640 1 --2- 172.21.15.161:0/2655057356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71c00cf5c0 0x7f71c014e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71bf400640 1 -- 172.21.15.161:0/2655057356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b00094d0 con 0x7f71c00cd3e0 2024-05-22T21:13:32.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71bf400640 1 --2- 172.21.15.161:0/2655057356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c00cd3e0 0x7f71c014deb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00e9b0 tx=0x7f71ac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:32.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71b7e00640 1 -- 172.21.15.161:0/2655057356 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac0096b0 con 0x7f71c00cd3e0 2024-05-22T21:13:32.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71b7e00640 1 -- 172.21.15.161:0/2655057356 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ac00c9a0 con 0x7f71c00cd3e0 2024-05-22T21:13:32.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71b7e00640 1 -- 172.21.15.161:0/2655057356 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac00cd30 con 0x7f71c00cd3e0 2024-05-22T21:13:32.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71c5800640 1 -- 172.21.15.161:0/2655057356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c0150880 con 0x7f71c00cd3e0 2024-05-22T21:13:32.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71c5800640 1 -- 172.21.15.161:0/2655057356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c0150230 con 0x7f71c00cd3e0 2024-05-22T21:13:32.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.150+0000 7f71c5800640 1 -- 172.21.15.161:0/2655057356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c00676a0 con 0x7f71c00cd3e0 2024-05-22T21:13:32.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.154+0000 7f71b7e00640 1 -- 172.21.15.161:0/2655057356 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f71ac013470 con 0x7f71c00cd3e0 2024-05-22T21:13:32.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.154+0000 7f71b7e00640 1 --2- 172.21.15.161:0/2655057356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f71a0066690 0x7f71a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.154+0000 7f71b7e00640 1 -- 172.21.15.161:0/2655057356 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71ac0844e0 con 0x7f71c00cd3e0 2024-05-22T21:13:32.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.154+0000 7f71bfe00640 1 --2- 172.21.15.161:0/2655057356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f71a0066690 0x7f71a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.154+0000 7f71bfe00640 1 --2- 172.21.15.161:0/2655057356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f71a0066690 0x7f71a0068b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f71b00ad000 tx=0x7f71b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:32.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.158+0000 7f71b7e00640 1 -- 172.21.15.161:0/2655057356 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f71ac0517b0 con 0x7f71c00cd3e0 2024-05-22T21:13:32.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.310+0000 7f71c5800640 1 -- 172.21.15.161:0/2655057356 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f71c016a100 con 0x7f71a0066690 2024-05-22T21:13:32.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.314+0000 7f71b7e00640 1 -- 172.21.15.161:0/2655057356 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260221 (secure 0 0 0) 0x7f71b00ba090 con 0x7f71a0066690 2024-05-22T21:13:32.322 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:32.323 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:13:32.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.318+0000 7f71c5800640 1 -- 172.21.15.161:0/2655057356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f71a0066690 msgr2=0x7f71a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.318+0000 7f71c5800640 1 --2- 172.21.15.161:0/2655057356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f71a0066690 0x7f71a0068b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f71b00ad000 tx=0x7f71b0008040 comp rx=0 tx=0).stop 2024-05-22T21:13:32.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.318+0000 7f71c5800640 1 -- 172.21.15.161:0/2655057356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c00cd3e0 msgr2=0x7f71c014deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.318+0000 7f71c5800640 1 --2- 172.21.15.161:0/2655057356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c00cd3e0 0x7f71c014deb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00e9b0 tx=0x7f71ac00ee70 comp rx=0 tx=0).stop 2024-05-22T21:13:32.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.318+0000 7f71c5800640 1 -- 172.21.15.161:0/2655057356 shutdown_connections 2024-05-22T21:13:32.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.318+0000 7f71c5800640 1 --2- 172.21.15.161:0/2655057356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f71a0066690 0x7f71a0068b50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.318+0000 7f71c5800640 1 --2- 172.21.15.161:0/2655057356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71c00cf5c0 0x7f71c014e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.318+0000 7f71c5800640 1 --2- 172.21.15.161:0/2655057356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71c00cd3e0 0x7f71c014deb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.318+0000 7f71c5800640 1 --2- 172.21.15.161:0/2655057356 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f71c006a410 0x7f71c014d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.318+0000 7f71c5800640 1 -- 172.21.15.161:0/2655057356 >> 172.21.15.161:0/2655057356 conn(0x7f71c00cfcc0 msgr2=0x7f71c00b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:32.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.318+0000 7f71c5800640 1 -- 172.21.15.161:0/2655057356 shutdown_connections 2024-05-22T21:13:32.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.318+0000 7f71c5800640 1 -- 172.21.15.161:0/2655057356 wait complete. 2024-05-22T21:13:32.342 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-05-22T21:13:30.393427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1746,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560753,"statfs":{"total":1600000000,"available":1598211887,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.354676"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428654+0000","last_change":"2024-05-22T21:13:23.445778+0000","last_active":"2024-05-22T21:13:24.428654+0000","last_peered":"2024-05-22T21:13:24.428654+0000","last_clean":"2024-05-22T21:13:24.428654+0000","last_became_active":"2024-05-22T21:13:23.445416+0000","last_became_peered":"2024-05-22T21:13:23.445416+0000","last_unstale":"2024-05-22T21:13:24.428654+0000","last_undegraded":"2024-05-22T21:13:24.428654+0000","last_fullsized":"2024-05-22T21:13:24.428654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:57:10.746221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423393+0000","last_change":"2024-05-22T21:13:23.429874+0000","last_active":"2024-05-22T21:13:24.423393+0000","last_peered":"2024-05-22T21:13:24.423393+0000","last_clean":"2024-05-22T21:13:24.423393+0000","last_became_active":"2024-05-22T21:13:23.428636+0000","last_became_peered":"2024-05-22T21:13:23.428636+0000","last_unstale":"2024-05-22T21:13:24.423393+0000","last_undegraded":"2024-05-22T21:13:24.423393+0000","last_fullsized":"2024-05-22T21:13:24.423393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:44:56.029080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015292+0000","last_change":"2024-05-22T21:13:23.434034+0000","last_active":"2024-05-22T21:13:25.015292+0000","last_peered":"2024-05-22T21:13:25.015292+0000","last_clean":"2024-05-22T21:13:25.015292+0000","last_became_active":"2024-05-22T21:13:23.432250+0000","last_became_peered":"2024-05-22T21:13:23.432250+0000","last_unstale":"2024-05-22T21:13:25.015292+0000","last_undegraded":"2024-05-22T21:13:25.015292+0000","last_fullsized":"2024-05-22T21:13:25.015292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:04:37.695966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919501+0000","last_change":"2024-05-22T21:13:23.440239+0000","last_active":"2024-05-22T21:13:24.919501+0000","last_peered":"2024-05-22T21:13:24.919501+0000","last_clean":"2024-05-22T21:13:24.919501+0000","last_became_active":"2024-05-22T21:13:23.433800+0000","last_became_peered":"2024-05-22T21:13:23.433800+0000","last_unstale":"2024-05-22T21:13:24.919501+0000","last_undegraded":"2024-05-22T21:13:24.919501+0000","last_fullsized":"2024-05-22T21:13:24.919501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:09:20.599937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429279+0000","last_change":"2024-05-22T21:13:23.436638+0000","last_active":"2024-05-22T21:13:24.429279+0000","last_peered":"2024-05-22T21:13:24.429279+0000","last_clean":"2024-05-22T21:13:24.429279+0000","last_became_active":"2024-05-22T21:13:23.436392+0000","last_became_peered":"2024-05-22T21:13:23.436392+0000","last_unstale":"2024-05-22T21:13:24.429279+0000","last_undegraded":"2024-05-22T21:13:24.429279+0000","last_fullsized":"2024-05-22T21:13:24.429279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:38:04.537935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919974+0000","last_change":"2024-05-22T21:13:23.442807+0000","last_active":"2024-05-22T21:13:24.919974+0000","last_peered":"2024-05-22T21:13:24.919974+0000","last_clean":"2024-05-22T21:13:24.919974+0000","last_became_active":"2024-05-22T21:13:23.441590+0000","last_became_peered":"2024-05-22T21:13:23.441590+0000","last_unstale":"2024-05-22T21:13:24.919974+0000","last_undegraded":"2024-05-22T21:13:24.919974+0000","last_fullsized":"2024-05-22T21:13:24.919974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:57:11.273220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429185+0000","last_change":"2024-05-22T21:13:23.436160+0000","last_active":"2024-05-22T21:13:24.429185+0000","last_peered":"2024-05-22T21:13:24.429185+0000","last_clean":"2024-05-22T21:13:24.429185+0000","last_became_active":"2024-05-22T21:13:23.435949+0000","last_became_peered":"2024-05-22T21:13:23.435949+0000","last_unstale":"2024-05-22T21:13:24.429185+0000","last_undegraded":"2024-05-22T21:13:24.429185+0000","last_fullsized":"2024-05-22T21:13:24.429185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:56:26.375232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014774+0000","last_change":"2024-05-22T21:13:23.437626+0000","last_active":"2024-05-22T21:13:25.014774+0000","last_peered":"2024-05-22T21:13:25.014774+0000","last_clean":"2024-05-22T21:13:25.014774+0000","last_became_active":"2024-05-22T21:13:23.437162+0000","last_became_peered":"2024-05-22T21:13:23.437162+0000","last_unstale":"2024-05-22T21:13:25.014774+0000","last_undegraded":"2024-05-22T21:13:25.014774+0000","last_fullsized":"2024-05-22T21:13:25.014774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:09:20.599937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428632+0000","last_change":"2024-05-22T21:13:23.446764+0000","last_active":"2024-05-22T21:13:24.428632+0000","last_peered":"2024-05-22T21:13:24.428632+0000","last_clean":"2024-05-22T21:13:24.428632+0000","last_became_active":"2024-05-22T21:13:23.446506+0000","last_became_peered":"2024-05-22T21:13:23.446506+0000","last_unstale":"2024-05-22T21:13:24.428632+0000","last_undegraded":"2024-05-22T21:13:24.428632+0000","last_fullsized":"2024-05-22T21:13:24.428632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:13:51.196274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014576+0000","last_change":"2024-05-22T21:13:23.450063+0000","last_active":"2024-05-22T21:13:25.014576+0000","last_peered":"2024-05-22T21:13:25.014576+0000","last_clean":"2024-05-22T21:13:25.014576+0000","last_became_active":"2024-05-22T21:13:23.449193+0000","last_became_peered":"2024-05-22T21:13:23.449193+0000","last_unstale":"2024-05-22T21:13:25.014576+0000","last_undegraded":"2024-05-22T21:13:25.014576+0000","last_fullsized":"2024-05-22T21:13:25.014576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:14:11.125539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014530+0000","last_change":"2024-05-22T21:13:23.449563+0000","last_active":"2024-05-22T21:13:25.014530+0000","last_peered":"2024-05-22T21:13:25.014530+0000","last_clean":"2024-05-22T21:13:25.014530+0000","last_became_active":"2024-05-22T21:13:23.449304+0000","last_became_peered":"2024-05-22T21:13:23.449304+0000","last_unstale":"2024-05-22T21:13:25.014530+0000","last_undegraded":"2024-05-22T21:13:25.014530+0000","last_fullsized":"2024-05-22T21:13:25.014530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:07:36.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428800+0000","last_change":"2024-05-22T21:13:23.444134+0000","last_active":"2024-05-22T21:13:24.428800+0000","last_peered":"2024-05-22T21:13:24.428800+0000","last_clean":"2024-05-22T21:13:24.428800+0000","last_became_active":"2024-05-22T21:13:23.443924+0000","last_became_peered":"2024-05-22T21:13:23.443924+0000","last_unstale":"2024-05-22T21:13:24.428800+0000","last_undegraded":"2024-05-22T21:13:24.428800+0000","last_fullsized":"2024-05-22T21:13:24.428800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:10.124782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014938+0000","last_change":"2024-05-22T21:13:23.447229+0000","last_active":"2024-05-22T21:13:25.014938+0000","last_peered":"2024-05-22T21:13:25.014938+0000","last_clean":"2024-05-22T21:13:25.014938+0000","last_became_active":"2024-05-22T21:13:23.447007+0000","last_became_peered":"2024-05-22T21:13:23.447007+0000","last_unstale":"2024-05-22T21:13:25.014938+0000","last_undegraded":"2024-05-22T21:13:25.014938+0000","last_fullsized":"2024-05-22T21:13:25.014938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:44:04.542487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014914+0000","last_change":"2024-05-22T21:13:23.433465+0000","last_active":"2024-05-22T21:13:25.014914+0000","last_peered":"2024-05-22T21:13:25.014914+0000","last_clean":"2024-05-22T21:13:25.014914+0000","last_became_active":"2024-05-22T21:13:23.432948+0000","last_became_peered":"2024-05-22T21:13:23.432948+0000","last_unstale":"2024-05-22T21:13:25.014914+0000","last_undegraded":"2024-05-22T21:13:25.014914+0000","last_fullsized":"2024-05-22T21:13:25.014914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:07:36.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423652+0000","last_change":"2024-05-22T21:13:23.426036+0000","last_active":"2024-05-22T21:13:24.423652+0000","last_peered":"2024-05-22T21:13:24.423652+0000","last_clean":"2024-05-22T21:13:24.423652+0000","last_became_active":"2024-05-22T21:13:23.425830+0000","last_became_peered":"2024-05-22T21:13:23.425830+0000","last_unstale":"2024-05-22T21:13:24.423652+0000","last_undegraded":"2024-05-22T21:13:24.423652+0000","last_fullsized":"2024-05-22T21:13:24.423652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:27.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919876+0000","last_change":"2024-05-22T21:13:23.442938+0000","last_active":"2024-05-22T21:13:24.919876+0000","last_peered":"2024-05-22T21:13:24.919876+0000","last_clean":"2024-05-22T21:13:24.919876+0000","last_became_active":"2024-05-22T21:13:23.441874+0000","last_became_peered":"2024-05-22T21:13:23.441874+0000","last_unstale":"2024-05-22T21:13:24.919876+0000","last_undegraded":"2024-05-22T21:13:24.919876+0000","last_fullsized":"2024-05-22T21:13:24.919876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:57:10.746221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919437+0000","last_change":"2024-05-22T21:13:23.444504+0000","last_active":"2024-05-22T21:13:24.919437+0000","last_peered":"2024-05-22T21:13:24.919437+0000","last_clean":"2024-05-22T21:13:24.919437+0000","last_became_active":"2024-05-22T21:13:23.443892+0000","last_became_peered":"2024-05-22T21:13:23.443892+0000","last_unstale":"2024-05-22T21:13:24.919437+0000","last_undegraded":"2024-05-22T21:13:24.919437+0000","last_fullsized":"2024-05-22T21:13:24.919437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:56:26.375232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014525+0000","last_change":"2024-05-22T21:13:23.447393+0000","last_active":"2024-05-22T21:13:25.014525+0000","last_peered":"2024-05-22T21:13:25.014525+0000","last_clean":"2024-05-22T21:13:25.014525+0000","last_became_active":"2024-05-22T21:13:23.446956+0000","last_became_peered":"2024-05-22T21:13:23.446956+0000","last_unstale":"2024-05-22T21:13:25.014525+0000","last_undegraded":"2024-05-22T21:13:25.014525+0000","last_fullsized":"2024-05-22T21:13:25.014525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:34:49.902120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428827+0000","last_change":"2024-05-22T21:13:23.437025+0000","last_active":"2024-05-22T21:13:24.428827+0000","last_peered":"2024-05-22T21:13:24.428827+0000","last_clean":"2024-05-22T21:13:24.428827+0000","last_became_active":"2024-05-22T21:13:23.436786+0000","last_became_peered":"2024-05-22T21:13:23.436786+0000","last_unstale":"2024-05-22T21:13:24.428827+0000","last_undegraded":"2024-05-22T21:13:24.428827+0000","last_fullsized":"2024-05-22T21:13:24.428827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:04:32.035089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014759+0000","last_change":"2024-05-22T21:13:23.444119+0000","last_active":"2024-05-22T21:13:25.014759+0000","last_peered":"2024-05-22T21:13:25.014759+0000","last_clean":"2024-05-22T21:13:25.014759+0000","last_became_active":"2024-05-22T21:13:23.443861+0000","last_became_peered":"2024-05-22T21:13:23.443861+0000","last_unstale":"2024-05-22T21:13:25.014759+0000","last_undegraded":"2024-05-22T21:13:25.014759+0000","last_fullsized":"2024-05-22T21:13:25.014759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:00:43.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015860+0000","last_change":"2024-05-22T21:13:23.434203+0000","last_active":"2024-05-22T21:13:25.015860+0000","last_peered":"2024-05-22T21:13:25.015860+0000","last_clean":"2024-05-22T21:13:25.015860+0000","last_became_active":"2024-05-22T21:13:23.433871+0000","last_became_peered":"2024-05-22T21:13:23.433871+0000","last_unstale":"2024-05-22T21:13:25.015860+0000","last_undegraded":"2024-05-22T21:13:25.015860+0000","last_fullsized":"2024-05-22T21:13:25.015860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:00:43.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919633+0000","last_change":"2024-05-22T21:13:23.444258+0000","last_active":"2024-05-22T21:13:24.919633+0000","last_peered":"2024-05-22T21:13:24.919633+0000","last_clean":"2024-05-22T21:13:24.919633+0000","last_became_active":"2024-05-22T21:13:23.443329+0000","last_became_peered":"2024-05-22T21:13:23.443329+0000","last_unstale":"2024-05-22T21:13:24.919633+0000","last_undegraded":"2024-05-22T21:13:24.919633+0000","last_fullsized":"2024-05-22T21:13:24.919633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015111+0000","last_change":"2024-05-22T21:13:23.437489+0000","last_active":"2024-05-22T21:13:25.015111+0000","last_peered":"2024-05-22T21:13:25.015111+0000","last_clean":"2024-05-22T21:13:25.015111+0000","last_became_active":"2024-05-22T21:13:23.436877+0000","last_became_peered":"2024-05-22T21:13:23.436877+0000","last_unstale":"2024-05-22T21:13:25.015111+0000","last_undegraded":"2024-05-22T21:13:25.015111+0000","last_fullsized":"2024-05-22T21:13:25.015111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:56:26.375232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919626+0000","last_change":"2024-05-22T21:13:23.434147+0000","last_active":"2024-05-22T21:13:24.919626+0000","last_peered":"2024-05-22T21:13:24.919626+0000","last_clean":"2024-05-22T21:13:24.919626+0000","last_became_active":"2024-05-22T21:13:23.433529+0000","last_became_peered":"2024-05-22T21:13:23.433529+0000","last_unstale":"2024-05-22T21:13:24.919626+0000","last_undegraded":"2024-05-22T21:13:24.919626+0000","last_fullsized":"2024-05-22T21:13:24.919626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:44:04.542487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015414+0000","last_change":"2024-05-22T21:13:23.435603+0000","last_active":"2024-05-22T21:13:25.015414+0000","last_peered":"2024-05-22T21:13:25.015414+0000","last_clean":"2024-05-22T21:13:25.015414+0000","last_became_active":"2024-05-22T21:13:23.435387+0000","last_became_peered":"2024-05-22T21:13:23.435387+0000","last_unstale":"2024-05-22T21:13:25.015414+0000","last_undegraded":"2024-05-22T21:13:25.015414+0000","last_fullsized":"2024-05-22T21:13:25.015414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:27.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014667+0000","last_change":"2024-05-22T21:13:23.444856+0000","last_active":"2024-05-22T21:13:25.014667+0000","last_peered":"2024-05-22T21:13:25.014667+0000","last_clean":"2024-05-22T21:13:25.014667+0000","last_became_active":"2024-05-22T21:13:23.444640+0000","last_became_peered":"2024-05-22T21:13:23.444640+0000","last_unstale":"2024-05-22T21:13:25.014667+0000","last_undegraded":"2024-05-22T21:13:25.014667+0000","last_fullsized":"2024-05-22T21:13:25.014667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:07:57.351276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423445+0000","last_change":"2024-05-22T21:13:23.430546+0000","last_active":"2024-05-22T21:13:24.423445+0000","last_peered":"2024-05-22T21:13:24.423445+0000","last_clean":"2024-05-22T21:13:24.423445+0000","last_became_active":"2024-05-22T21:13:23.430306+0000","last_became_peered":"2024-05-22T21:13:23.430306+0000","last_unstale":"2024-05-22T21:13:24.423445+0000","last_undegraded":"2024-05-22T21:13:24.423445+0000","last_fullsized":"2024-05-22T21:13:24.423445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:06.303665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015209+0000","last_change":"2024-05-22T21:13:23.437362+0000","last_active":"2024-05-22T21:13:25.015209+0000","last_peered":"2024-05-22T21:13:25.015209+0000","last_clean":"2024-05-22T21:13:25.015209+0000","last_became_active":"2024-05-22T21:13:23.436580+0000","last_became_peered":"2024-05-22T21:13:23.436580+0000","last_unstale":"2024-05-22T21:13:25.015209+0000","last_undegraded":"2024-05-22T21:13:25.015209+0000","last_fullsized":"2024-05-22T21:13:25.015209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:06.303665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015001+0000","last_change":"2024-05-22T21:13:23.433616+0000","last_active":"2024-05-22T21:13:25.015001+0000","last_peered":"2024-05-22T21:13:25.015001+0000","last_clean":"2024-05-22T21:13:25.015001+0000","last_became_active":"2024-05-22T21:13:23.433258+0000","last_became_peered":"2024-05-22T21:13:23.433258+0000","last_unstale":"2024-05-22T21:13:25.015001+0000","last_undegraded":"2024-05-22T21:13:25.015001+0000","last_fullsized":"2024-05-22T21:13:25.015001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:44:56.029080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919662+0000","last_change":"2024-05-22T21:13:23.435441+0000","last_active":"2024-05-22T21:13:24.919662+0000","last_peered":"2024-05-22T21:13:24.919662+0000","last_clean":"2024-05-22T21:13:24.919662+0000","last_became_active":"2024-05-22T21:13:23.435225+0000","last_became_peered":"2024-05-22T21:13:23.435225+0000","last_unstale":"2024-05-22T21:13:24.919662+0000","last_undegraded":"2024-05-22T21:13:24.919662+0000","last_fullsized":"2024-05-22T21:13:24.919662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:44:56.029080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014612+0000","last_change":"2024-05-22T21:13:23.446437+0000","last_active":"2024-05-22T21:13:25.014612+0000","last_peered":"2024-05-22T21:13:25.014612+0000","last_clean":"2024-05-22T21:13:25.014612+0000","last_became_active":"2024-05-22T21:13:23.446219+0000","last_became_peered":"2024-05-22T21:13:23.446219+0000","last_unstale":"2024-05-22T21:13:25.014612+0000","last_undegraded":"2024-05-22T21:13:25.014612+0000","last_fullsized":"2024-05-22T21:13:25.014612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:57:10.070412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014588+0000","last_change":"2024-05-22T21:13:23.443792+0000","last_active":"2024-05-22T21:13:25.014588+0000","last_peered":"2024-05-22T21:13:25.014588+0000","last_clean":"2024-05-22T21:13:25.014588+0000","last_became_active":"2024-05-22T21:13:23.443581+0000","last_became_peered":"2024-05-22T21:13:23.443581+0000","last_unstale":"2024-05-22T21:13:25.014588+0000","last_undegraded":"2024-05-22T21:13:25.014588+0000","last_fullsized":"2024-05-22T21:13:25.014588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.430114+0000","last_change":"2024-05-22T21:13:23.429085+0000","last_active":"2024-05-22T21:13:24.430114+0000","last_peered":"2024-05-22T21:13:24.430114+0000","last_clean":"2024-05-22T21:13:24.430114+0000","last_became_active":"2024-05-22T21:13:23.428826+0000","last_became_peered":"2024-05-22T21:13:23.428826+0000","last_unstale":"2024-05-22T21:13:24.430114+0000","last_undegraded":"2024-05-22T21:13:24.430114+0000","last_fullsized":"2024-05-22T21:13:24.430114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:17.109310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015138+0000","last_change":"2024-05-22T21:13:23.053644+0000","last_active":"2024-05-22T21:13:25.015138+0000","last_peered":"2024-05-22T21:13:25.015138+0000","last_clean":"2024-05-22T21:13:25.015138+0000","last_became_active":"2024-05-22T21:13:23.053129+0000","last_became_peered":"2024-05-22T21:13:23.053129+0000","last_unstale":"2024-05-22T21:13:25.015138+0000","last_undegraded":"2024-05-22T21:13:25.015138+0000","last_fullsized":"2024-05-22T21:13:25.015138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:02.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015372+0000","last_change":"2024-05-22T21:13:23.430851+0000","last_active":"2024-05-22T21:13:25.015372+0000","last_peered":"2024-05-22T21:13:25.015372+0000","last_clean":"2024-05-22T21:13:25.015372+0000","last_became_active":"2024-05-22T21:13:23.430614+0000","last_became_peered":"2024-05-22T21:13:23.430614+0000","last_unstale":"2024-05-22T21:13:25.015372+0000","last_undegraded":"2024-05-22T21:13:25.015372+0000","last_fullsized":"2024-05-22T21:13:25.015372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:14:11.125539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014815+0000","last_change":"2024-05-22T21:13:23.052090+0000","last_active":"2024-05-22T21:13:25.014815+0000","last_peered":"2024-05-22T21:13:25.014815+0000","last_clean":"2024-05-22T21:13:25.014815+0000","last_became_active":"2024-05-22T21:13:23.051762+0000","last_became_peered":"2024-05-22T21:13:23.051762+0000","last_unstale":"2024-05-22T21:13:25.014815+0000","last_undegraded":"2024-05-22T21:13:25.014815+0000","last_fullsized":"2024-05-22T21:13:25.014815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:54.533292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015255+0000","last_change":"2024-05-22T21:13:23.053832+0000","last_active":"2024-05-22T21:13:25.015255+0000","last_peered":"2024-05-22T21:13:25.015255+0000","last_clean":"2024-05-22T21:13:25.015255+0000","last_became_active":"2024-05-22T21:13:23.053170+0000","last_became_peered":"2024-05-22T21:13:23.053170+0000","last_unstale":"2024-05-22T21:13:25.015255+0000","last_undegraded":"2024-05-22T21:13:25.015255+0000","last_fullsized":"2024-05-22T21:13:25.015255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015495+0000","last_change":"2024-05-22T21:13:23.429829+0000","last_active":"2024-05-22T21:13:25.015495+0000","last_peered":"2024-05-22T21:13:25.015495+0000","last_clean":"2024-05-22T21:13:25.015495+0000","last_became_active":"2024-05-22T21:13:23.429616+0000","last_became_peered":"2024-05-22T21:13:23.429616+0000","last_unstale":"2024-05-22T21:13:25.015495+0000","last_undegraded":"2024-05-22T21:13:25.015495+0000","last_fullsized":"2024-05-22T21:13:25.015495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:00:48.798418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919805+0000","last_change":"2024-05-22T21:13:23.056945+0000","last_active":"2024-05-22T21:13:24.919805+0000","last_peered":"2024-05-22T21:13:24.919805+0000","last_clean":"2024-05-22T21:13:24.919805+0000","last_became_active":"2024-05-22T21:13:23.056643+0000","last_became_peered":"2024-05-22T21:13:23.056643+0000","last_unstale":"2024-05-22T21:13:24.919805+0000","last_undegraded":"2024-05-22T21:13:24.919805+0000","last_fullsized":"2024-05-22T21:13:24.919805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:02.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423490+0000","last_change":"2024-05-22T21:13:23.429959+0000","last_active":"2024-05-22T21:13:24.423490+0000","last_peered":"2024-05-22T21:13:24.423490+0000","last_clean":"2024-05-22T21:13:24.423490+0000","last_became_active":"2024-05-22T21:13:23.429752+0000","last_became_peered":"2024-05-22T21:13:23.429752+0000","last_unstale":"2024-05-22T21:13:24.423490+0000","last_undegraded":"2024-05-22T21:13:24.423490+0000","last_fullsized":"2024-05-22T21:13:24.423490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:14:11.125539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014851+0000","last_change":"2024-05-22T21:13:23.058878+0000","last_active":"2024-05-22T21:13:25.014851+0000","last_peered":"2024-05-22T21:13:25.014851+0000","last_clean":"2024-05-22T21:13:25.014851+0000","last_became_active":"2024-05-22T21:13:23.058648+0000","last_became_peered":"2024-05-22T21:13:23.058648+0000","last_unstale":"2024-05-22T21:13:25.014851+0000","last_undegraded":"2024-05-22T21:13:25.014851+0000","last_fullsized":"2024-05-22T21:13:25.014851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919552+0000","last_change":"2024-05-22T21:13:23.444383+0000","last_active":"2024-05-22T21:13:24.919552+0000","last_peered":"2024-05-22T21:13:24.919552+0000","last_clean":"2024-05-22T21:13:24.919552+0000","last_became_active":"2024-05-22T21:13:23.443612+0000","last_became_peered":"2024-05-22T21:13:23.443612+0000","last_unstale":"2024-05-22T21:13:24.919552+0000","last_undegraded":"2024-05-22T21:13:24.919552+0000","last_fullsized":"2024-05-22T21:13:24.919552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:27.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919883+0000","last_change":"2024-05-22T21:13:23.056793+0000","last_active":"2024-05-22T21:13:24.919883+0000","last_peered":"2024-05-22T21:13:24.919883+0000","last_clean":"2024-05-22T21:13:24.919883+0000","last_became_active":"2024-05-22T21:13:23.056450+0000","last_became_peered":"2024-05-22T21:13:23.056450+0000","last_unstale":"2024-05-22T21:13:24.919883+0000","last_undegraded":"2024-05-22T21:13:24.919883+0000","last_fullsized":"2024-05-22T21:13:24.919883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:34.410142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015942+0000","last_change":"2024-05-22T21:13:23.446398+0000","last_active":"2024-05-22T21:13:25.015942+0000","last_peered":"2024-05-22T21:13:25.015942+0000","last_clean":"2024-05-22T21:13:25.015942+0000","last_became_active":"2024-05-22T21:13:23.445907+0000","last_became_peered":"2024-05-22T21:13:23.445907+0000","last_unstale":"2024-05-22T21:13:25.015942+0000","last_undegraded":"2024-05-22T21:13:25.015942+0000","last_fullsized":"2024-05-22T21:13:25.015942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:06.303665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423552+0000","last_change":"2024-05-22T21:13:23.056570+0000","last_active":"2024-05-22T21:13:24.423552+0000","last_peered":"2024-05-22T21:13:24.423552+0000","last_clean":"2024-05-22T21:13:24.423552+0000","last_became_active":"2024-05-22T21:13:23.056357+0000","last_became_peered":"2024-05-22T21:13:23.056357+0000","last_unstale":"2024-05-22T21:13:24.423552+0000","last_undegraded":"2024-05-22T21:13:24.423552+0000","last_fullsized":"2024-05-22T21:13:24.423552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:30.101472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428611+0000","last_change":"2024-05-22T21:13:23.439128+0000","last_active":"2024-05-22T21:13:24.428611+0000","last_peered":"2024-05-22T21:13:24.428611+0000","last_clean":"2024-05-22T21:13:24.428611+0000","last_became_active":"2024-05-22T21:13:23.438898+0000","last_became_peered":"2024-05-22T21:13:23.438898+0000","last_unstale":"2024-05-22T21:13:24.428611+0000","last_undegraded":"2024-05-22T21:13:24.428611+0000","last_fullsized":"2024-05-22T21:13:24.428611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:51:29.569811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014610+0000","last_change":"2024-05-22T21:13:23.049600+0000","last_active":"2024-05-22T21:13:25.014610+0000","last_peered":"2024-05-22T21:13:25.014610+0000","last_clean":"2024-05-22T21:13:25.014610+0000","last_became_active":"2024-05-22T21:13:23.049395+0000","last_became_peered":"2024-05-22T21:13:23.049395+0000","last_unstale":"2024-05-22T21:13:25.014610+0000","last_undegraded":"2024-05-22T21:13:25.014610+0000","last_fullsized":"2024-05-22T21:13:25.014610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:24.879526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429576+0000","last_change":"2024-05-22T21:13:23.439228+0000","last_active":"2024-05-22T21:13:24.429576+0000","last_peered":"2024-05-22T21:13:24.429576+0000","last_clean":"2024-05-22T21:13:24.429576+0000","last_became_active":"2024-05-22T21:13:23.437292+0000","last_became_peered":"2024-05-22T21:13:23.437292+0000","last_unstale":"2024-05-22T21:13:24.429576+0000","last_undegraded":"2024-05-22T21:13:24.429576+0000","last_fullsized":"2024-05-22T21:13:24.429576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:44:04.542487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015531+0000","last_change":"2024-05-22T21:13:23.052267+0000","last_active":"2024-05-22T21:13:25.015531+0000","last_peered":"2024-05-22T21:13:25.015531+0000","last_clean":"2024-05-22T21:13:25.015531+0000","last_became_active":"2024-05-22T21:13:23.051663+0000","last_became_peered":"2024-05-22T21:13:23.051663+0000","last_unstale":"2024-05-22T21:13:25.015531+0000","last_undegraded":"2024-05-22T21:13:25.015531+0000","last_fullsized":"2024-05-22T21:13:25.015531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:54.533292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015989+0000","last_change":"2024-05-22T21:13:23.434082+0000","last_active":"2024-05-22T21:13:25.015989+0000","last_peered":"2024-05-22T21:13:25.015989+0000","last_clean":"2024-05-22T21:13:25.015989+0000","last_became_active":"2024-05-22T21:13:23.433591+0000","last_became_peered":"2024-05-22T21:13:23.433591+0000","last_unstale":"2024-05-22T21:13:25.015989+0000","last_undegraded":"2024-05-22T21:13:25.015989+0000","last_fullsized":"2024-05-22T21:13:25.015989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:08:18.923855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.920207+0000","last_change":"2024-05-22T21:13:23.058203+0000","last_active":"2024-05-22T21:13:24.920207+0000","last_peered":"2024-05-22T21:13:24.920207+0000","last_clean":"2024-05-22T21:13:24.920207+0000","last_became_active":"2024-05-22T21:13:23.057230+0000","last_became_peered":"2024-05-22T21:13:23.057230+0000","last_unstale":"2024-05-22T21:13:24.920207+0000","last_undegraded":"2024-05-22T21:13:24.920207+0000","last_fullsized":"2024-05-22T21:13:24.920207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:16.101596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429362+0000","last_change":"2024-05-22T21:13:23.437522+0000","last_active":"2024-05-22T21:13:24.429362+0000","last_peered":"2024-05-22T21:13:24.429362+0000","last_clean":"2024-05-22T21:13:24.429362+0000","last_became_active":"2024-05-22T21:13:23.436971+0000","last_became_peered":"2024-05-22T21:13:23.436971+0000","last_unstale":"2024-05-22T21:13:24.429362+0000","last_undegraded":"2024-05-22T21:13:24.429362+0000","last_fullsized":"2024-05-22T21:13:24.429362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:51:04.078274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014654+0000","last_change":"2024-05-22T21:13:23.056955+0000","last_active":"2024-05-22T21:13:25.014654+0000","last_peered":"2024-05-22T21:13:25.014654+0000","last_clean":"2024-05-22T21:13:25.014654+0000","last_became_active":"2024-05-22T21:13:23.056131+0000","last_became_peered":"2024-05-22T21:13:23.056131+0000","last_unstale":"2024-05-22T21:13:25.014654+0000","last_undegraded":"2024-05-22T21:13:25.014654+0000","last_fullsized":"2024-05-22T21:13:25.014654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:02.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014604+0000","last_change":"2024-05-22T21:13:23.445799+0000","last_active":"2024-05-22T21:13:25.014604+0000","last_peered":"2024-05-22T21:13:25.014604+0000","last_clean":"2024-05-22T21:13:25.014604+0000","last_became_active":"2024-05-22T21:13:23.445576+0000","last_became_peered":"2024-05-22T21:13:23.445576+0000","last_unstale":"2024-05-22T21:13:25.014604+0000","last_undegraded":"2024-05-22T21:13:25.014604+0000","last_fullsized":"2024-05-22T21:13:25.014604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:04:37.695966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014626+0000","last_change":"2024-05-22T21:13:23.046892+0000","last_active":"2024-05-22T21:13:25.014626+0000","last_peered":"2024-05-22T21:13:25.014626+0000","last_clean":"2024-05-22T21:13:25.014626+0000","last_became_active":"2024-05-22T21:13:23.046677+0000","last_became_peered":"2024-05-22T21:13:23.046677+0000","last_unstale":"2024-05-22T21:13:25.014626+0000","last_undegraded":"2024-05-22T21:13:25.014626+0000","last_fullsized":"2024-05-22T21:13:25.014626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:24.879526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429428+0000","last_change":"2024-05-22T21:13:23.437891+0000","last_active":"2024-05-22T21:13:24.429428+0000","last_peered":"2024-05-22T21:13:24.429428+0000","last_clean":"2024-05-22T21:13:24.429428+0000","last_became_active":"2024-05-22T21:13:23.436990+0000","last_became_peered":"2024-05-22T21:13:23.436990+0000","last_unstale":"2024-05-22T21:13:24.429428+0000","last_undegraded":"2024-05-22T21:13:24.429428+0000","last_fullsized":"2024-05-22T21:13:24.429428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:12:43.781854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428700+0000","last_change":"2024-05-22T21:13:23.054659+0000","last_active":"2024-05-22T21:13:24.428700+0000","last_peered":"2024-05-22T21:13:24.428700+0000","last_clean":"2024-05-22T21:13:24.428700+0000","last_became_active":"2024-05-22T21:13:23.054403+0000","last_became_peered":"2024-05-22T21:13:23.054403+0000","last_unstale":"2024-05-22T21:13:24.428700+0000","last_undegraded":"2024-05-22T21:13:24.428700+0000","last_fullsized":"2024-05-22T21:13:24.428700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:26:28.755372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429419+0000","last_change":"2024-05-22T21:13:23.437668+0000","last_active":"2024-05-22T21:13:24.429419+0000","last_peered":"2024-05-22T21:13:24.429419+0000","last_clean":"2024-05-22T21:13:24.429419+0000","last_became_active":"2024-05-22T21:13:23.437307+0000","last_became_peered":"2024-05-22T21:13:23.437307+0000","last_unstale":"2024-05-22T21:13:24.429419+0000","last_undegraded":"2024-05-22T21:13:24.429419+0000","last_fullsized":"2024-05-22T21:13:24.429419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:57:11.273220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014824+0000","last_change":"2024-05-22T21:13:23.047615+0000","last_active":"2024-05-22T21:13:25.014824+0000","last_peered":"2024-05-22T21:13:25.014824+0000","last_clean":"2024-05-22T21:13:25.014824+0000","last_became_active":"2024-05-22T21:13:23.047358+0000","last_became_peered":"2024-05-22T21:13:23.047358+0000","last_unstale":"2024-05-22T21:13:25.014824+0000","last_undegraded":"2024-05-22T21:13:25.014824+0000","last_fullsized":"2024-05-22T21:13:25.014824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:34.410142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428730+0000","last_change":"2024-05-22T21:13:23.443073+0000","last_active":"2024-05-22T21:13:24.428730+0000","last_peered":"2024-05-22T21:13:24.428730+0000","last_clean":"2024-05-22T21:13:24.428730+0000","last_became_active":"2024-05-22T21:13:23.442751+0000","last_became_peered":"2024-05-22T21:13:23.442751+0000","last_unstale":"2024-05-22T21:13:24.428730+0000","last_undegraded":"2024-05-22T21:13:24.428730+0000","last_fullsized":"2024-05-22T21:13:24.428730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:09:20.599937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423443+0000","last_change":"2024-05-22T21:13:23.050892+0000","last_active":"2024-05-22T21:13:24.423443+0000","last_peered":"2024-05-22T21:13:24.423443+0000","last_clean":"2024-05-22T21:13:24.423443+0000","last_became_active":"2024-05-22T21:13:23.050565+0000","last_became_peered":"2024-05-22T21:13:23.050565+0000","last_unstale":"2024-05-22T21:13:24.423443+0000","last_undegraded":"2024-05-22T21:13:24.423443+0000","last_fullsized":"2024-05-22T21:13:24.423443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:16.101596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919723+0000","last_change":"2024-05-22T21:13:23.437191+0000","last_active":"2024-05-22T21:13:24.919723+0000","last_peered":"2024-05-22T21:13:24.919723+0000","last_clean":"2024-05-22T21:13:24.919723+0000","last_became_active":"2024-05-22T21:13:23.436983+0000","last_became_peered":"2024-05-22T21:13:23.436983+0000","last_unstale":"2024-05-22T21:13:24.919723+0000","last_undegraded":"2024-05-22T21:13:24.919723+0000","last_fullsized":"2024-05-22T21:13:24.919723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:06.303665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014631+0000","last_change":"2024-05-22T21:13:23.056461+0000","last_active":"2024-05-22T21:13:25.014631+0000","last_peered":"2024-05-22T21:13:25.014631+0000","last_clean":"2024-05-22T21:13:25.014631+0000","last_became_active":"2024-05-22T21:13:23.056237+0000","last_became_peered":"2024-05-22T21:13:23.056237+0000","last_unstale":"2024-05-22T21:13:25.014631+0000","last_undegraded":"2024-05-22T21:13:25.014631+0000","last_fullsized":"2024-05-22T21:13:25.014631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428905+0000","last_change":"2024-05-22T21:13:23.448377+0000","last_active":"2024-05-22T21:13:24.428905+0000","last_peered":"2024-05-22T21:13:24.428905+0000","last_clean":"2024-05-22T21:13:24.428905+0000","last_became_active":"2024-05-22T21:13:23.447835+0000","last_became_peered":"2024-05-22T21:13:23.447835+0000","last_unstale":"2024-05-22T21:13:24.428905+0000","last_undegraded":"2024-05-22T21:13:24.428905+0000","last_fullsized":"2024-05-22T21:13:24.428905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:44:10.866789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-22T21:13:23.435118+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:13:23.435118+0000","last_peered":"2024-05-22T21:13:23.435118+0000","last_clean":"2024-05-22T21:13:23.435118+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:13:23.435118+0000","last_undegraded":"2024-05-22T21:13:23.435118+0000","last_fullsized":"2024-05-22T21:13:23.435118+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.920353+0000","last_change":"2024-05-22T21:13:23.058027+0000","last_active":"2024-05-22T21:13:24.920353+0000","last_peered":"2024-05-22T21:13:24.920353+0000","last_clean":"2024-05-22T21:13:24.920353+0000","last_became_active":"2024-05-22T21:13:23.056401+0000","last_became_peered":"2024-05-22T21:13:23.056401+0000","last_unstale":"2024-05-22T21:13:24.920353+0000","last_undegraded":"2024-05-22T21:13:24.920353+0000","last_fullsized":"2024-05-22T21:13:24.920353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:24.879526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014947+0000","last_change":"2024-05-22T21:13:23.440325+0000","last_active":"2024-05-22T21:13:25.014947+0000","last_peered":"2024-05-22T21:13:25.014947+0000","last_clean":"2024-05-22T21:13:25.014947+0000","last_became_active":"2024-05-22T21:13:23.440020+0000","last_became_peered":"2024-05-22T21:13:23.440020+0000","last_unstale":"2024-05-22T21:13:25.014947+0000","last_undegraded":"2024-05-22T21:13:25.014947+0000","last_fullsized":"2024-05-22T21:13:25.014947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:38:04.537935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429005+0000","last_change":"2024-05-22T21:13:23.050452+0000","last_active":"2024-05-22T21:13:24.429005+0000","last_peered":"2024-05-22T21:13:24.429005+0000","last_clean":"2024-05-22T21:13:24.429005+0000","last_became_active":"2024-05-22T21:13:23.049982+0000","last_became_peered":"2024-05-22T21:13:23.049982+0000","last_unstale":"2024-05-22T21:13:24.429005+0000","last_undegraded":"2024-05-22T21:13:24.429005+0000","last_fullsized":"2024-05-22T21:13:24.429005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:57:09.062698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015492+0000","last_change":"2024-05-22T21:13:23.434429+0000","last_active":"2024-05-22T21:13:25.015492+0000","last_peered":"2024-05-22T21:13:25.015492+0000","last_clean":"2024-05-22T21:13:25.015492+0000","last_became_active":"2024-05-22T21:13:23.433640+0000","last_became_peered":"2024-05-22T21:13:23.433640+0000","last_unstale":"2024-05-22T21:13:25.015492+0000","last_undegraded":"2024-05-22T21:13:25.015492+0000","last_fullsized":"2024-05-22T21:13:25.015492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014707+0000","last_change":"2024-05-22T21:13:23.051179+0000","last_active":"2024-05-22T21:13:25.014707+0000","last_peered":"2024-05-22T21:13:25.014707+0000","last_clean":"2024-05-22T21:13:25.014707+0000","last_became_active":"2024-05-22T21:13:23.050967+0000","last_became_peered":"2024-05-22T21:13:23.050967+0000","last_unstale":"2024-05-22T21:13:25.014707+0000","last_undegraded":"2024-05-22T21:13:25.014707+0000","last_fullsized":"2024-05-22T21:13:25.014707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:02.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423581+0000","last_change":"2024-05-22T21:13:23.426627+0000","last_active":"2024-05-22T21:13:24.423581+0000","last_peered":"2024-05-22T21:13:24.423581+0000","last_clean":"2024-05-22T21:13:24.423581+0000","last_became_active":"2024-05-22T21:13:23.426412+0000","last_became_peered":"2024-05-22T21:13:23.426412+0000","last_unstale":"2024-05-22T21:13:24.423581+0000","last_undegraded":"2024-05-22T21:13:24.423581+0000","last_fullsized":"2024-05-22T21:13:24.423581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919933+0000","last_change":"2024-05-22T21:13:23.055924+0000","last_active":"2024-05-22T21:13:24.919933+0000","last_peered":"2024-05-22T21:13:24.919933+0000","last_clean":"2024-05-22T21:13:24.919933+0000","last_became_active":"2024-05-22T21:13:23.055577+0000","last_became_peered":"2024-05-22T21:13:23.055577+0000","last_unstale":"2024-05-22T21:13:24.919933+0000","last_undegraded":"2024-05-22T21:13:24.919933+0000","last_fullsized":"2024-05-22T21:13:24.919933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.430340+0000","last_change":"2024-05-22T21:13:23.428954+0000","last_active":"2024-05-22T21:13:24.430340+0000","last_peered":"2024-05-22T21:13:24.430340+0000","last_clean":"2024-05-22T21:13:24.430340+0000","last_became_active":"2024-05-22T21:13:23.428628+0000","last_became_peered":"2024-05-22T21:13:23.428628+0000","last_unstale":"2024-05-22T21:13:24.430340+0000","last_undegraded":"2024-05-22T21:13:24.430340+0000","last_fullsized":"2024-05-22T21:13:24.430340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:31.109187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423797+0000","last_change":"2024-05-22T21:13:23.055046+0000","last_active":"2024-05-22T21:13:24.423797+0000","last_peered":"2024-05-22T21:13:24.423797+0000","last_clean":"2024-05-22T21:13:24.423797+0000","last_became_active":"2024-05-22T21:13:23.054670+0000","last_became_peered":"2024-05-22T21:13:23.054670+0000","last_unstale":"2024-05-22T21:13:24.423797+0000","last_undegraded":"2024-05-22T21:13:24.423797+0000","last_fullsized":"2024-05-22T21:13:24.423797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:24.879526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919722+0000","last_change":"2024-05-22T21:13:23.434008+0000","last_active":"2024-05-22T21:13:24.919722+0000","last_peered":"2024-05-22T21:13:24.919722+0000","last_clean":"2024-05-22T21:13:24.919722+0000","last_became_active":"2024-05-22T21:13:23.433233+0000","last_became_peered":"2024-05-22T21:13:23.433233+0000","last_unstale":"2024-05-22T21:13:24.919722+0000","last_undegraded":"2024-05-22T21:13:24.919722+0000","last_fullsized":"2024-05-22T21:13:24.919722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:38:04.537935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428911+0000","last_change":"2024-05-22T21:13:23.041563+0000","last_active":"2024-05-22T21:13:24.428911+0000","last_peered":"2024-05-22T21:13:24.428911+0000","last_clean":"2024-05-22T21:13:24.428911+0000","last_became_active":"2024-05-22T21:13:23.041302+0000","last_became_peered":"2024-05-22T21:13:23.041302+0000","last_unstale":"2024-05-22T21:13:24.428911+0000","last_undegraded":"2024-05-22T21:13:24.428911+0000","last_fullsized":"2024-05-22T21:13:24.428911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:51:28.562097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423568+0000","last_change":"2024-05-22T21:13:23.426584+0000","last_active":"2024-05-22T21:13:24.423568+0000","last_peered":"2024-05-22T21:13:24.423568+0000","last_clean":"2024-05-22T21:13:24.423568+0000","last_became_active":"2024-05-22T21:13:23.426367+0000","last_became_peered":"2024-05-22T21:13:23.426367+0000","last_unstale":"2024-05-22T21:13:24.423568+0000","last_undegraded":"2024-05-22T21:13:24.423568+0000","last_fullsized":"2024-05-22T21:13:24.423568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:07:36.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423622+0000","last_change":"2024-05-22T21:13:23.049729+0000","last_active":"2024-05-22T21:13:24.423622+0000","last_peered":"2024-05-22T21:13:24.423622+0000","last_clean":"2024-05-22T21:13:24.423622+0000","last_became_active":"2024-05-22T21:13:23.049480+0000","last_became_peered":"2024-05-22T21:13:23.049480+0000","last_unstale":"2024-05-22T21:13:24.423622+0000","last_undegraded":"2024-05-22T21:13:24.423622+0000","last_fullsized":"2024-05-22T21:13:24.423622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:51:28.562097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014848+0000","last_change":"2024-05-22T21:13:23.437853+0000","last_active":"2024-05-22T21:13:25.014848+0000","last_peered":"2024-05-22T21:13:25.014848+0000","last_clean":"2024-05-22T21:13:25.014848+0000","last_became_active":"2024-05-22T21:13:23.437676+0000","last_became_peered":"2024-05-22T21:13:23.437676+0000","last_unstale":"2024-05-22T21:13:25.014848+0000","last_undegraded":"2024-05-22T21:13:25.014848+0000","last_fullsized":"2024-05-22T21:13:25.014848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:14:11.125539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014750+0000","last_change":"2024-05-22T21:13:23.052969+0000","last_active":"2024-05-22T21:13:25.014750+0000","last_peered":"2024-05-22T21:13:25.014750+0000","last_clean":"2024-05-22T21:13:25.014750+0000","last_became_active":"2024-05-22T21:13:23.052306+0000","last_became_peered":"2024-05-22T21:13:23.052306+0000","last_unstale":"2024-05-22T21:13:25.014750+0000","last_undegraded":"2024-05-22T21:13:25.014750+0000","last_fullsized":"2024-05-22T21:13:25.014750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:34.410142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015367+0000","last_change":"2024-05-22T21:13:23.434301+0000","last_active":"2024-05-22T21:13:25.015367+0000","last_peered":"2024-05-22T21:13:25.015367+0000","last_clean":"2024-05-22T21:13:25.015367+0000","last_became_active":"2024-05-22T21:13:23.432584+0000","last_became_peered":"2024-05-22T21:13:23.432584+0000","last_unstale":"2024-05-22T21:13:25.015367+0000","last_undegraded":"2024-05-22T21:13:25.015367+0000","last_fullsized":"2024-05-22T21:13:25.015367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:53:57.578626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429187+0000","last_change":"2024-05-22T21:13:23.047297+0000","last_active":"2024-05-22T21:13:24.429187+0000","last_peered":"2024-05-22T21:13:24.429187+0000","last_clean":"2024-05-22T21:13:24.429187+0000","last_became_active":"2024-05-22T21:13:23.047021+0000","last_became_peered":"2024-05-22T21:13:23.047021+0000","last_unstale":"2024-05-22T21:13:24.429187+0000","last_undegraded":"2024-05-22T21:13:24.429187+0000","last_fullsized":"2024-05-22T21:13:24.429187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:09.117068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429366+0000","last_change":"2024-05-22T21:13:23.434654+0000","last_active":"2024-05-22T21:13:24.429366+0000","last_peered":"2024-05-22T21:13:24.429366+0000","last_clean":"2024-05-22T21:13:24.429366+0000","last_became_active":"2024-05-22T21:13:23.433684+0000","last_became_peered":"2024-05-22T21:13:23.433684+0000","last_unstale":"2024-05-22T21:13:24.429366+0000","last_undegraded":"2024-05-22T21:13:24.429366+0000","last_fullsized":"2024-05-22T21:13:24.429366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:03.482406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423727+0000","last_change":"2024-05-22T21:13:23.054907+0000","last_active":"2024-05-22T21:13:24.423727+0000","last_peered":"2024-05-22T21:13:24.423727+0000","last_clean":"2024-05-22T21:13:24.423727+0000","last_became_active":"2024-05-22T21:13:23.054307+0000","last_became_peered":"2024-05-22T21:13:23.054307+0000","last_unstale":"2024-05-22T21:13:24.423727+0000","last_undegraded":"2024-05-22T21:13:24.423727+0000","last_fullsized":"2024-05-22T21:13:24.423727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:09.117068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015009+0000","last_change":"2024-05-22T21:13:23.429322+0000","last_active":"2024-05-22T21:13:25.015009+0000","last_peered":"2024-05-22T21:13:25.015009+0000","last_clean":"2024-05-22T21:13:25.015009+0000","last_became_active":"2024-05-22T21:13:23.429091+0000","last_became_peered":"2024-05-22T21:13:23.429091+0000","last_unstale":"2024-05-22T21:13:25.015009+0000","last_undegraded":"2024-05-22T21:13:25.015009+0000","last_fullsized":"2024-05-22T21:13:25.015009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:44:56.029080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423657+0000","last_change":"2024-05-22T21:13:23.052267+0000","last_active":"2024-05-22T21:13:24.423657+0000","last_peered":"2024-05-22T21:13:24.423657+0000","last_clean":"2024-05-22T21:13:24.423657+0000","last_became_active":"2024-05-22T21:13:23.052062+0000","last_became_peered":"2024-05-22T21:13:23.052062+0000","last_unstale":"2024-05-22T21:13:24.423657+0000","last_undegraded":"2024-05-22T21:13:24.423657+0000","last_fullsized":"2024-05-22T21:13:24.423657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:34:48.894406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015300+0000","last_change":"2024-05-22T21:13:23.433439+0000","last_active":"2024-05-22T21:13:25.015300+0000","last_peered":"2024-05-22T21:13:25.015300+0000","last_clean":"2024-05-22T21:13:25.015300+0000","last_became_active":"2024-05-22T21:13:23.433016+0000","last_became_peered":"2024-05-22T21:13:23.433016+0000","last_unstale":"2024-05-22T21:13:25.015300+0000","last_undegraded":"2024-05-22T21:13:25.015300+0000","last_fullsized":"2024-05-22T21:13:25.015300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429655+0000","last_change":"2024-05-22T21:13:23.052977+0000","last_active":"2024-05-22T21:13:24.429655+0000","last_peered":"2024-05-22T21:13:24.429655+0000","last_clean":"2024-05-22T21:13:24.429655+0000","last_became_active":"2024-05-22T21:13:23.052490+0000","last_became_peered":"2024-05-22T21:13:23.052490+0000","last_unstale":"2024-05-22T21:13:24.429655+0000","last_undegraded":"2024-05-22T21:13:24.429655+0000","last_fullsized":"2024-05-22T21:13:24.429655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:16.101596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014517+0000","last_change":"2024-05-22T21:13:23.446126+0000","last_active":"2024-05-22T21:13:25.014517+0000","last_peered":"2024-05-22T21:13:25.014517+0000","last_clean":"2024-05-22T21:13:25.014517+0000","last_became_active":"2024-05-22T21:13:23.444835+0000","last_became_peered":"2024-05-22T21:13:23.444835+0000","last_unstale":"2024-05-22T21:13:25.014517+0000","last_undegraded":"2024-05-22T21:13:25.014517+0000","last_fullsized":"2024-05-22T21:13:25.014517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:53:57.578626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919747+0000","last_change":"2024-05-22T21:13:23.432060+0000","last_active":"2024-05-22T21:13:24.919747+0000","last_peered":"2024-05-22T21:13:24.919747+0000","last_clean":"2024-05-22T21:13:24.919747+0000","last_became_active":"2024-05-22T21:13:23.431845+0000","last_became_peered":"2024-05-22T21:13:23.431845+0000","last_unstale":"2024-05-22T21:13:24.919747+0000","last_undegraded":"2024-05-22T21:13:24.919747+0000","last_fullsized":"2024-05-22T21:13:24.919747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:44:10.866789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429092+0000","last_change":"2024-05-22T21:13:23.058221+0000","last_active":"2024-05-22T21:13:24.429092+0000","last_peered":"2024-05-22T21:13:24.429092+0000","last_clean":"2024-05-22T21:13:24.429092+0000","last_became_active":"2024-05-22T21:13:23.058012+0000","last_became_peered":"2024-05-22T21:13:23.058012+0000","last_unstale":"2024-05-22T21:13:24.429092+0000","last_undegraded":"2024-05-22T21:13:24.429092+0000","last_fullsized":"2024-05-22T21:13:24.429092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:34:48.894406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.920057+0000","last_change":"2024-05-22T21:13:23.431267+0000","last_active":"2024-05-22T21:13:24.920057+0000","last_peered":"2024-05-22T21:13:24.920057+0000","last_clean":"2024-05-22T21:13:24.920057+0000","last_became_active":"2024-05-22T21:13:23.431054+0000","last_became_peered":"2024-05-22T21:13:23.431054+0000","last_unstale":"2024-05-22T21:13:24.920057+0000","last_undegraded":"2024-05-22T21:13:24.920057+0000","last_fullsized":"2024-05-22T21:13:24.920057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:51:04.078274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429474+0000","last_change":"2024-05-22T21:13:23.051743+0000","last_active":"2024-05-22T21:13:24.429474+0000","last_peered":"2024-05-22T21:13:24.429474+0000","last_clean":"2024-05-22T21:13:24.429474+0000","last_became_active":"2024-05-22T21:13:23.051564+0000","last_became_peered":"2024-05-22T21:13:23.051564+0000","last_unstale":"2024-05-22T21:13:24.429474+0000","last_undegraded":"2024-05-22T21:13:24.429474+0000","last_fullsized":"2024-05-22T21:13:24.429474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:30.101472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015432+0000","last_change":"2024-05-22T21:13:23.433871+0000","last_active":"2024-05-22T21:13:25.015432+0000","last_peered":"2024-05-22T21:13:25.015432+0000","last_clean":"2024-05-22T21:13:25.015432+0000","last_became_active":"2024-05-22T21:13:23.431365+0000","last_became_peered":"2024-05-22T21:13:23.431365+0000","last_unstale":"2024-05-22T21:13:25.015432+0000","last_undegraded":"2024-05-22T21:13:25.015432+0000","last_fullsized":"2024-05-22T21:13:25.015432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:27.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015179+0000","last_change":"2024-05-22T21:13:23.046747+0000","last_active":"2024-05-22T21:13:25.015179+0000","last_peered":"2024-05-22T21:13:25.015179+0000","last_clean":"2024-05-22T21:13:25.015179+0000","last_became_active":"2024-05-22T21:13:23.046523+0000","last_became_peered":"2024-05-22T21:13:23.046523+0000","last_unstale":"2024-05-22T21:13:25.015179+0000","last_undegraded":"2024-05-22T21:13:25.015179+0000","last_fullsized":"2024-05-22T21:13:25.015179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:34.410142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015087+0000","last_change":"2024-05-22T21:13:23.447604+0000","last_active":"2024-05-22T21:13:25.015087+0000","last_peered":"2024-05-22T21:13:25.015087+0000","last_clean":"2024-05-22T21:13:25.015087+0000","last_became_active":"2024-05-22T21:13:23.447391+0000","last_became_peered":"2024-05-22T21:13:23.447391+0000","last_unstale":"2024-05-22T21:13:25.015087+0000","last_undegraded":"2024-05-22T21:13:25.015087+0000","last_fullsized":"2024-05-22T21:13:25.015087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:07:36.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015546+0000","last_change":"2024-05-22T21:13:23.043385+0000","last_active":"2024-05-22T21:13:25.015546+0000","last_peered":"2024-05-22T21:13:25.015546+0000","last_clean":"2024-05-22T21:13:25.015546+0000","last_became_active":"2024-05-22T21:13:23.043036+0000","last_became_peered":"2024-05-22T21:13:23.043036+0000","last_unstale":"2024-05-22T21:13:25.015546+0000","last_undegraded":"2024-05-22T21:13:25.015546+0000","last_fullsized":"2024-05-22T21:13:25.015546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194661,"statfs":{"total":200000000,"available":199333641,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":75,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923027,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924501,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924465,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":75,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923027,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924591,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924431,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199334204,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]}],"pool_statfs":[]}} 2024-05-22T21:13:32.347 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-22T21:13:32.347 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-22T21:13:32.347 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-22T21:13:32.347 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-22T21:13:32.348 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-22T21:13:32.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.514+0000 7f85c9000640 1 Processor -- start 2024-05-22T21:13:32.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.514+0000 7f85c9000640 1 -- start start 2024-05-22T21:13:32.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c9000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85c415e7d0 0x7f85c40b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c9000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c415eed0 0x7f85c415f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c9000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c40b3560 con 0x7f85c415f890 2024-05-22T21:13:32.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c9000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c40cf530 con 0x7f85c415e7d0 2024-05-22T21:13:32.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c9000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c415fd20 con 0x7f85c415eed0 2024-05-22T21:13:32.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c2800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f85c415f890 0x7f85c4099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56128/0 (socket says 172.21.15.161:56128) 2024-05-22T21:13:32.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c2800640 1 -- 172.21.15.161:0/4274089857 learned_addr learned my addr 172.21.15.161:0/4274089857 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:32.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1e00640 1 --2- 172.21.15.161:0/4274089857 >> v2:172.21.15.179:3300/0 conn(0x7f85c415e7d0 0x7f85c40b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85b9400640 1 --2- 172.21.15.161:0/4274089857 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c415eed0 0x7f85c415f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1400640 1 -- 172.21.15.161:0/4274089857 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10594320 0 0) 0x7f85c40b3560 con 0x7f85c415f890 2024-05-22T21:13:32.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1400640 1 -- 172.21.15.161:0/4274089857 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85a8003530 con 0x7f85c415f890 2024-05-22T21:13:32.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1400640 1 -- 172.21.15.161:0/4274089857 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3126264421 0 0) 0x7f85a8003530 con 0x7f85c415f890 2024-05-22T21:13:32.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1400640 1 -- 172.21.15.161:0/4274089857 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c415eed0 msgr2=0x7f85c415f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:32.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1400640 1 --2- 172.21.15.161:0/4274089857 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c415eed0 0x7f85c415f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1400640 1 -- 172.21.15.161:0/4274089857 >> v2:172.21.15.179:3300/0 conn(0x7f85c415e7d0 msgr2=0x7f85c40b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:32.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1400640 1 --2- 172.21.15.161:0/4274089857 >> v2:172.21.15.179:3300/0 conn(0x7f85c415e7d0 0x7f85c40b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1400640 1 -- 172.21.15.161:0/4274089857 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c40ccdb0 con 0x7f85c415f890 2024-05-22T21:13:32.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1400640 1 -- 172.21.15.161:0/4274089857 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f85bc001680 con 0x7f85c415f890 2024-05-22T21:13:32.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1e00640 1 --2- 172.21.15.161:0/4274089857 >> v2:172.21.15.179:3300/0 conn(0x7f85c415e7d0 0x7f85c40b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:32.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1400640 1 -- 172.21.15.161:0/4274089857 >> v1:172.21.15.161:6789/0 conn(0x7f85c415f890 legacy=0x7f85c4099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1400640 1 --2- 172.21.15.161:0/4274089857 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c415eed0 0x7f85a8003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1400640 1 --2- 172.21.15.161:0/4274089857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85a8003f00 0x7f85a80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1400640 1 --2- 172.21.15.161:0/4274089857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85a8004d10 0x7f85a8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1400640 1 -- 172.21.15.161:0/4274089857 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c40cf530 con 0x7f85a8003f00 2024-05-22T21:13:32.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1400640 1 -- 172.21.15.161:0/4274089857 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c415fd20 con 0x7f85a8004d10 2024-05-22T21:13:32.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1400640 1 -- 172.21.15.161:0/4274089857 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a800b990 con 0x7f85c415eed0 2024-05-22T21:13:32.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85b9400640 1 --2- 172.21.15.161:0/4274089857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85a8003f00 0x7f85a80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c1e00640 1 --2- 172.21.15.161:0/4274089857 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c415eed0 0x7f85a8003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85c2800640 1 --2- 172.21.15.161:0/4274089857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85a8004d10 0x7f85a8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85b9400640 1 -- 172.21.15.161:0/4274089857 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c415eed0 msgr2=0x7f85a8003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:32.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85b9400640 1 --2- 172.21.15.161:0/4274089857 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c415eed0 0x7f85a8003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85b9400640 1 -- 172.21.15.161:0/4274089857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85a8004d10 msgr2=0x7f85a8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:32.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85b9400640 1 --2- 172.21.15.161:0/4274089857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85a8004d10 0x7f85a8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.518+0000 7f85b9400640 1 -- 172.21.15.161:0/4274089857 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a8003530 con 0x7f85a8003f00 2024-05-22T21:13:32.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.522+0000 7f85b9400640 1 --2- 172.21.15.161:0/4274089857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85a8003f00 0x7f85a80047d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009ae0 tx=0x7f85ac01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f23730e844d210 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:32.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.522+0000 7f85c1400640 1 -- 172.21.15.161:0/4274089857 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0a5550 con 0x7f85a8003f00 2024-05-22T21:13:32.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.522+0000 7f85c1400640 1 -- 172.21.15.161:0/4274089857 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac0a56b0 con 0x7f85a8003f00 2024-05-22T21:13:32.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.522+0000 7f85c1400640 1 -- 172.21.15.161:0/4274089857 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0a5a60 con 0x7f85a8003f00 2024-05-22T21:13:32.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.522+0000 7f85c9000640 1 -- 172.21.15.161:0/4274089857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85a8003f00 msgr2=0x7f85a80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.522+0000 7f85c9000640 1 --2- 172.21.15.161:0/4274089857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85a8003f00 0x7f85a80047d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009ae0 tx=0x7f85ac01ca10 comp rx=0 tx=0).stop 2024-05-22T21:13:32.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.522+0000 7f85c2800640 1 -- 172.21.15.161:0/4274089857 reap_dead start 2024-05-22T21:13:32.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.522+0000 7f85c9000640 1 -- 172.21.15.161:0/4274089857 shutdown_connections 2024-05-22T21:13:32.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.522+0000 7f85c9000640 1 -- 172.21.15.161:0/4274089857 >> 172.21.15.161:0/4274089857 conn(0x7f85c40b2a40 msgr2=0x7f85c40b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:32.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.522+0000 7f85c9000640 1 -- 172.21.15.161:0/4274089857 shutdown_connections 2024-05-22T21:13:32.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.522+0000 7f85c9000640 1 -- 172.21.15.161:0/4274089857 wait complete. 2024-05-22T21:13:32.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85c9000640 1 Processor -- start 2024-05-22T21:13:32.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85c9000640 1 -- start start 2024-05-22T21:13:32.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85c9000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85a8003f00 0x7f85c414d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85c9000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c415b0b0 0x7f85c40c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85c9000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c415b530 0x7f85c40c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85c9000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c406b3a0 con 0x7f85c415b0b0 2024-05-22T21:13:32.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85c9000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c406b680 con 0x7f85a8003f00 2024-05-22T21:13:32.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85c9000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c406b960 con 0x7f85c415b530 2024-05-22T21:13:32.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85b9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c415b0b0 0x7f85c40c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85b9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c415b0b0 0x7f85c40c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49036/0 (socket says 172.21.15.161:49036) 2024-05-22T21:13:32.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85b9400640 1 -- 172.21.15.161:0/3101322697 learned_addr learned my addr 172.21.15.161:0/3101322697 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:32.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85c1e00640 1 --2- 172.21.15.161:0/3101322697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85a8003f00 0x7f85c414d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85c2800640 1 --2- 172.21.15.161:0/3101322697 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c415b530 0x7f85c40c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85b9400640 1 -- 172.21.15.161:0/3101322697 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c415b530 msgr2=0x7f85c40c6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85b9400640 1 --2- 172.21.15.161:0/3101322697 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c415b530 0x7f85c40c6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85b9400640 1 -- 172.21.15.161:0/3101322697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85a8003f00 msgr2=0x7f85c414d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85b9400640 1 --2- 172.21.15.161:0/3101322697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85a8003f00 0x7f85c414d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85b9400640 1 -- 172.21.15.161:0/3101322697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c406bc40 con 0x7f85c415b0b0 2024-05-22T21:13:32.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85c1e00640 1 --2- 172.21.15.161:0/3101322697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85a8003f00 0x7f85c414d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:32.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85b9400640 1 --2- 172.21.15.161:0/3101322697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c415b0b0 0x7f85c40c5710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009a00 tx=0x7f85a8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:32.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85baa00640 1 -- 172.21.15.161:0/3101322697 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac004220 con 0x7f85c415b0b0 2024-05-22T21:13:32.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85baa00640 1 -- 172.21.15.161:0/3101322697 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac0043c0 con 0x7f85c415b0b0 2024-05-22T21:13:32.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85c9000640 1 -- 172.21.15.161:0/3101322697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c406bf20 con 0x7f85c415b0b0 2024-05-22T21:13:32.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85baa00640 1 -- 172.21.15.161:0/3101322697 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac003860 con 0x7f85c415b0b0 2024-05-22T21:13:32.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85c9000640 1 -- 172.21.15.161:0/3101322697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c406afe0 con 0x7f85c415b0b0 2024-05-22T21:13:32.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.526+0000 7f85c9000640 1 -- 172.21.15.161:0/3101322697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c406c0b0 con 0x7f85c415b0b0 2024-05-22T21:13:32.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.530+0000 7f85baa00640 1 -- 172.21.15.161:0/3101322697 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f85ac0b4040 con 0x7f85c415b0b0 2024-05-22T21:13:32.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.530+0000 7f85baa00640 1 --2- 172.21.15.161:0/3101322697 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85a0066690 0x7f85a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.530+0000 7f85baa00640 1 -- 172.21.15.161:0/3101322697 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85ac11e890 con 0x7f85c415b0b0 2024-05-22T21:13:32.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.530+0000 7f85c1e00640 1 --2- 172.21.15.161:0/3101322697 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85a0066690 0x7f85a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.530+0000 7f85c1e00640 1 --2- 172.21.15.161:0/3101322697 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85a0066690 0x7f85a0068b50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f85b4007660 tx=0x7f85b400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:32.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.534+0000 7f85baa00640 1 -- 172.21.15.161:0/3101322697 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f85ac0ebd20 con 0x7f85c415b0b0 2024-05-22T21:13:32.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.726+0000 7f85c9000640 1 -- 172.21.15.161:0/3101322697 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f85c4067220 con 0x7f85c415b0b0 2024-05-22T21:13:32.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.726+0000 7f85baa00640 1 -- 172.21.15.161:0/3101322697 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164890 (secure 0 0 0) 0x7f85ac0ef340 con 0x7f85c415b0b0 2024-05-22T21:13:32.733 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:32.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.730+0000 7f85c9000640 1 -- 172.21.15.161:0/3101322697 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85a0066690 msgr2=0x7f85a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.730+0000 7f85c9000640 1 --2- 172.21.15.161:0/3101322697 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85a0066690 0x7f85a0068b50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f85b4007660 tx=0x7f85b400b040 comp rx=0 tx=0).stop 2024-05-22T21:13:32.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.730+0000 7f85c9000640 1 -- 172.21.15.161:0/3101322697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c415b0b0 msgr2=0x7f85c40c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.730+0000 7f85c9000640 1 --2- 172.21.15.161:0/3101322697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c415b0b0 0x7f85c40c5710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009a00 tx=0x7f85a8005800 comp rx=0 tx=0).stop 2024-05-22T21:13:32.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.730+0000 7f85c9000640 1 -- 172.21.15.161:0/3101322697 shutdown_connections 2024-05-22T21:13:32.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.730+0000 7f85c9000640 1 --2- 172.21.15.161:0/3101322697 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85a0066690 0x7f85a0068b50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.730+0000 7f85c9000640 1 --2- 172.21.15.161:0/3101322697 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c415b530 0x7f85c40c6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.730+0000 7f85c9000640 1 --2- 172.21.15.161:0/3101322697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c415b0b0 0x7f85c40c5710 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.730+0000 7f85c9000640 1 --2- 172.21.15.161:0/3101322697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85a8003f00 0x7f85c414d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.730+0000 7f85c9000640 1 -- 172.21.15.161:0/3101322697 >> 172.21.15.161:0/3101322697 conn(0x7f85c40b2a40 msgr2=0x7f85c4158fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:32.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.730+0000 7f85c9000640 1 -- 172.21.15.161:0/3101322697 shutdown_connections 2024-05-22T21:13:32.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.730+0000 7f85c9000640 1 -- 172.21.15.161:0/3101322697 wait complete. 2024-05-22T21:13:32.753 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":23334},{"type":"v1","addr":"172.21.15.161:6833","nonce":23334}]},"active_addr":"172.21.15.161:6833/23334","active_change":"2024-05-22T21:13:10.379947+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4098,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":501340783}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":2026678546}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":1821158309}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":161839883}]}]} 2024-05-22T21:13:32.756 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-22T21:13:32.756 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-22T21:13:32.756 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:32.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.886+0000 7f8a35000640 1 Processor -- start 2024-05-22T21:13:32.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.886+0000 7f8a35000640 1 -- start start 2024-05-22T21:13:32.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.890+0000 7f8a35000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8a3015e130 0x7f8a3015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.890+0000 7f8a35000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a30071670 con 0x7f8a3015d410 2024-05-22T21:13:32.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.890+0000 7f8a35000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a3015d870 con 0x7f8a3015dd50 2024-05-22T21:13:32.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.890+0000 7f8a35000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a30070a60 con 0x7f8a3015e130 2024-05-22T21:13:32.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.890+0000 7f8a2d400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8a3015e130 0x7f8a3015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.890+0000 7f8a2de00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8a3015dd50 0x7f8a3006d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:60684/0 (socket says 172.21.15.161:60684) 2024-05-22T21:13:32.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.890+0000 7f8a2de00640 1 -- 172.21.15.161:0/2712542107 learned_addr learned my addr 172.21.15.161:0/2712542107 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:32.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.890+0000 7f8a2d400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8a3015e130 0x7f8a3015ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49040/0 (socket says 172.21.15.161:49040) 2024-05-22T21:13:32.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.890+0000 7f8a2e800640 1 --1- 172.21.15.161:0/2712542107 >> v1:172.21.15.161:6789/0 conn(0x7f8a3015d410 0x7f8a3015b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56144/0 (socket says 172.21.15.161:56144) 2024-05-22T21:13:32.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.890+0000 7f8a2ca00640 1 -- 172.21.15.161:0/2712542107 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024850248 0 0) 0x7f8a3015d870 con 0x7f8a3015dd50 2024-05-22T21:13:32.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.890+0000 7f8a2d400640 1 -- 172.21.15.161:0/2712542107 >> v1:172.21.15.179:6789/0 conn(0x7f8a3015dd50 legacy=0x7f8a3006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.890+0000 7f8a2d400640 1 -- 172.21.15.161:0/2712542107 >> v1:172.21.15.161:6789/0 conn(0x7f8a3015d410 legacy=0x7f8a3015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.890+0000 7f8a2d400640 1 -- 172.21.15.161:0/2712542107 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a30070d40 con 0x7f8a3015e130 2024-05-22T21:13:32.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a2d400640 1 --2- 172.21.15.161:0/2712542107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3015e130 0x7f8a3015ced0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0097c0 tx=0x7f8a1c01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a725a3de973b1a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:32.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a2ca00640 1 -- 172.21.15.161:0/2712542107 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0ab600 con 0x7f8a3015e130 2024-05-22T21:13:32.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a2ca00640 1 -- 172.21.15.161:0/2712542107 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1c0a7050 con 0x7f8a3015e130 2024-05-22T21:13:32.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a2ca00640 1 -- 172.21.15.161:0/2712542107 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0ab980 con 0x7f8a3015e130 2024-05-22T21:13:32.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a35000640 1 -- 172.21.15.161:0/2712542107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3015e130 msgr2=0x7f8a3015ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a35000640 1 --2- 172.21.15.161:0/2712542107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3015e130 0x7f8a3015ced0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0097c0 tx=0x7f8a1c01c330 comp rx=0 tx=0).stop 2024-05-22T21:13:32.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a35000640 1 -- 172.21.15.161:0/2712542107 shutdown_connections 2024-05-22T21:13:32.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a35000640 1 --2- 172.21.15.161:0/2712542107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3015e130 0x7f8a3015ced0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a35000640 1 -- 172.21.15.161:0/2712542107 >> 172.21.15.161:0/2712542107 conn(0x7f8a300b0bc0 msgr2=0x7f8a300b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:32.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a35000640 1 -- 172.21.15.161:0/2712542107 shutdown_connections 2024-05-22T21:13:32.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a35000640 1 -- 172.21.15.161:0/2712542107 wait complete. 2024-05-22T21:13:32.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a35000640 1 Processor -- start 2024-05-22T21:13:32.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a35000640 1 -- start start 2024-05-22T21:13:32.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a35000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3015d410 0x7f8a300b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a35000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a3015dd50 0x7f8a30153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a35000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a3015e130 0x7f8a30158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a35000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a30154190 con 0x7f8a3015d410 2024-05-22T21:13:32.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a35000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a300b2500 con 0x7f8a3015e130 2024-05-22T21:13:32.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a35000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a300b27e0 con 0x7f8a3015dd50 2024-05-22T21:13:32.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a2de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3015d410 0x7f8a300b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.894+0000 7f8a2de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3015d410 0x7f8a300b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49042/0 (socket says 172.21.15.161:49042) 2024-05-22T21:13:32.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a2de00640 1 -- 172.21.15.161:0/464111278 learned_addr learned my addr 172.21.15.161:0/464111278 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:32.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a2e800640 1 --2- 172.21.15.161:0/464111278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a3015e130 0x7f8a30158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a2d400640 1 --2- 172.21.15.161:0/464111278 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a3015dd50 0x7f8a30153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a2de00640 1 -- 172.21.15.161:0/464111278 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a3015dd50 msgr2=0x7f8a30153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a2de00640 1 --2- 172.21.15.161:0/464111278 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a3015dd50 0x7f8a30153c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a2de00640 1 -- 172.21.15.161:0/464111278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a3015e130 msgr2=0x7f8a30158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:32.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a2de00640 1 --2- 172.21.15.161:0/464111278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a3015e130 0x7f8a30158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:32.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a2de00640 1 -- 172.21.15.161:0/464111278 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c009420 con 0x7f8a3015d410 2024-05-22T21:13:32.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a2de00640 1 --2- 172.21.15.161:0/464111278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3015d410 0x7f8a300b35a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8a200011f0 tx=0x7f8a2000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:32.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a26000640 1 -- 172.21.15.161:0/464111278 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a200074e0 con 0x7f8a3015d410 2024-05-22T21:13:32.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a26000640 1 -- 172.21.15.161:0/464111278 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a20007640 con 0x7f8a3015d410 2024-05-22T21:13:32.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a26000640 1 -- 172.21.15.161:0/464111278 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a20004a10 con 0x7f8a3015d410 2024-05-22T21:13:32.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a35000640 1 -- 172.21.15.161:0/464111278 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a300b2ac0 con 0x7f8a3015d410 2024-05-22T21:13:32.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a35000640 1 -- 172.21.15.161:0/464111278 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a300b3080 con 0x7f8a3015d410 2024-05-22T21:13:32.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a35000640 1 -- 172.21.15.161:0/464111278 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a30067220 con 0x7f8a3015d410 2024-05-22T21:13:32.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a26000640 1 -- 172.21.15.161:0/464111278 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8a20004b70 con 0x7f8a3015d410 2024-05-22T21:13:32.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a26000640 1 --2- 172.21.15.161:0/464111278 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a10066690 0x7f8a10068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:32.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.898+0000 7f8a26000640 1 -- 172.21.15.161:0/464111278 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a20083510 con 0x7f8a3015d410 2024-05-22T21:13:32.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.902+0000 7f8a2d400640 1 --2- 172.21.15.161:0/464111278 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a10066690 0x7f8a10068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:32.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.902+0000 7f8a2d400640 1 --2- 172.21.15.161:0/464111278 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a10066690 0x7f8a10068b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0ad000 tx=0x7f8a1c0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:32.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:32.906+0000 7f8a26000640 1 -- 172.21.15.161:0/464111278 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8a200507e0 con 0x7f8a3015d410 2024-05-22T21:13:33.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.062+0000 7f8a35000640 1 -- 172.21.15.161:0/464111278 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a3006c310 con 0x7f8a3015d410 2024-05-22T21:13:33.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.066+0000 7f8a26000640 1 -- 172.21.15.161:0/464111278 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f8a20018090 con 0x7f8a3015d410 2024-05-22T21:13:33.069 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:33.069 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:33.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.066+0000 7f8a35000640 1 -- 172.21.15.161:0/464111278 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a10066690 msgr2=0x7f8a10068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.066+0000 7f8a35000640 1 --2- 172.21.15.161:0/464111278 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a10066690 0x7f8a10068b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0ad000 tx=0x7f8a1c0a6000 comp rx=0 tx=0).stop 2024-05-22T21:13:33.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.066+0000 7f8a35000640 1 -- 172.21.15.161:0/464111278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3015d410 msgr2=0x7f8a300b35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.066+0000 7f8a35000640 1 --2- 172.21.15.161:0/464111278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3015d410 0x7f8a300b35a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8a200011f0 tx=0x7f8a2000c9b0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.066+0000 7f8a35000640 1 -- 172.21.15.161:0/464111278 shutdown_connections 2024-05-22T21:13:33.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.066+0000 7f8a35000640 1 --2- 172.21.15.161:0/464111278 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a10066690 0x7f8a10068b50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.066+0000 7f8a35000640 1 --2- 172.21.15.161:0/464111278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a3015e130 0x7f8a30158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.066+0000 7f8a35000640 1 --2- 172.21.15.161:0/464111278 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a3015dd50 0x7f8a30153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.066+0000 7f8a35000640 1 --2- 172.21.15.161:0/464111278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3015d410 0x7f8a300b35a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.066+0000 7f8a35000640 1 -- 172.21.15.161:0/464111278 >> 172.21.15.161:0/464111278 conn(0x7f8a300b0bc0 msgr2=0x7f8a3015b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.066+0000 7f8a35000640 1 -- 172.21.15.161:0/464111278 shutdown_connections 2024-05-22T21:13:33.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.066+0000 7f8a35000640 1 -- 172.21.15.161:0/464111278 wait complete. 2024-05-22T21:13:33.090 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-22T21:13:33.091 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:33.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.222+0000 7ff2e4800640 1 Processor -- start 2024-05-22T21:13:33.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.222+0000 7ff2e4800640 1 -- start start 2024-05-22T21:13:33.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.222+0000 7ff2e4800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2dc143ec0 0x7ff2dc15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.222+0000 7ff2e4800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2dc1442a0 con 0x7ff2dc0715a0 2024-05-22T21:13:33.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.222+0000 7ff2e4800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2dc070500 con 0x7ff2dc15ec90 2024-05-22T21:13:33.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.222+0000 7ff2e4800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0707e0 con 0x7ff2dc143ec0 2024-05-22T21:13:33.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.222+0000 7ff2e1400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff2dc0715a0 0x7ff2dc06fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56146/0 (socket says 172.21.15.161:56146) 2024-05-22T21:13:33.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.222+0000 7ff2e1400640 1 -- 172.21.15.161:0/988871904 learned_addr learned my addr 172.21.15.161:0/988871904 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:33.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.222+0000 7ff2e1e00640 1 --2- 172.21.15.161:0/988871904 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2dc143ec0 0x7ff2dc15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.222+0000 7ff2e0a00640 1 -- 172.21.15.161:0/988871904 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741668462 0 0) 0x7ff2dc1442a0 con 0x7ff2dc0715a0 2024-05-22T21:13:33.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.222+0000 7ff2e0a00640 1 -- 172.21.15.161:0/988871904 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c4003530 con 0x7ff2dc0715a0 2024-05-22T21:13:33.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e0a00640 1 -- 172.21.15.161:0/988871904 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761080609 0 0) 0x7ff2dc070500 con 0x7ff2dc15ec90 2024-05-22T21:13:33.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e0a00640 1 -- 172.21.15.161:0/988871904 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2dc1442a0 con 0x7ff2dc15ec90 2024-05-22T21:13:33.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e0a00640 1 -- 172.21.15.161:0/988871904 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 858191134 0 0) 0x7ff2c4003530 con 0x7ff2dc0715a0 2024-05-22T21:13:33.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e0a00640 1 -- 172.21.15.161:0/988871904 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2dc143ec0 msgr2=0x7ff2dc15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:33.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e0a00640 1 --2- 172.21.15.161:0/988871904 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2dc143ec0 0x7ff2dc15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e0a00640 1 -- 172.21.15.161:0/988871904 >> v1:172.21.15.179:6789/0 conn(0x7ff2dc15ec90 legacy=0x7ff2dc0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e0a00640 1 -- 172.21.15.161:0/988871904 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc070ac0 con 0x7ff2dc0715a0 2024-05-22T21:13:33.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e0a00640 1 -- 172.21.15.161:0/988871904 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7ff2cc001680 con 0x7ff2dc0715a0 2024-05-22T21:13:33.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e0a00640 1 -- 172.21.15.161:0/988871904 >> v1:172.21.15.161:6789/0 conn(0x7ff2dc0715a0 legacy=0x7ff2dc06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e0a00640 1 --2- 172.21.15.161:0/988871904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2c4003b40 0x7ff2c4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e0a00640 1 --2- 172.21.15.161:0/988871904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c40047d0 0x7ff2c4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e0a00640 1 --2- 172.21.15.161:0/988871904 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2dc143ec0 0x7ff2c4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e0a00640 1 -- 172.21.15.161:0/988871904 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0707e0 con 0x7ff2c40047d0 2024-05-22T21:13:33.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e0a00640 1 -- 172.21.15.161:0/988871904 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c400ba40 con 0x7ff2c4003b40 2024-05-22T21:13:33.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e0a00640 1 -- 172.21.15.161:0/988871904 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c400bbd0 con 0x7ff2dc143ec0 2024-05-22T21:13:33.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e1400640 1 --2- 172.21.15.161:0/988871904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c40047d0 0x7ff2c4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e1e00640 1 --2- 172.21.15.161:0/988871904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2c4003b40 0x7ff2c4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e2800640 1 --2- 172.21.15.161:0/988871904 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2dc143ec0 0x7ff2c4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e1400640 1 -- 172.21.15.161:0/988871904 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2dc143ec0 msgr2=0x7ff2c4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:33.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e1400640 1 --2- 172.21.15.161:0/988871904 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2dc143ec0 0x7ff2c4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e1400640 1 -- 172.21.15.161:0/988871904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2c4003b40 msgr2=0x7ff2c4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:33.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e1400640 1 --2- 172.21.15.161:0/988871904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2c4003b40 0x7ff2c4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e1400640 1 -- 172.21.15.161:0/988871904 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c4003530 con 0x7ff2c40047d0 2024-05-22T21:13:33.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.226+0000 7ff2e1400640 1 --2- 172.21.15.161:0/988871904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c40047d0 0x7ff2c4004c50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc070430 tx=0x7ff2cc01cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1728b8ec889fc14 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e0a00640 1 -- 172.21.15.161:0/988871904 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0058e0 con 0x7ff2c40047d0 2024-05-22T21:13:33.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e0a00640 1 -- 172.21.15.161:0/988871904 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc005a40 con 0x7ff2c40047d0 2024-05-22T21:13:33.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e0a00640 1 -- 172.21.15.161:0/988871904 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc005e10 con 0x7ff2c40047d0 2024-05-22T21:13:33.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e4800640 1 -- 172.21.15.161:0/988871904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c40047d0 msgr2=0x7ff2c4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e4800640 1 --2- 172.21.15.161:0/988871904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c40047d0 0x7ff2c4004c50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc070430 tx=0x7ff2cc01cb40 comp rx=0 tx=0).stop 2024-05-22T21:13:33.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e2800640 1 -- 172.21.15.161:0/988871904 reap_dead start 2024-05-22T21:13:33.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e4800640 1 -- 172.21.15.161:0/988871904 shutdown_connections 2024-05-22T21:13:33.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e4800640 1 -- 172.21.15.161:0/988871904 >> 172.21.15.161:0/988871904 conn(0x7ff2dc0b0bc0 msgr2=0x7ff2dc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e4800640 1 -- 172.21.15.161:0/988871904 shutdown_connections 2024-05-22T21:13:33.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e4800640 1 -- 172.21.15.161:0/988871904 wait complete. 2024-05-22T21:13:33.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e4800640 1 Processor -- start 2024-05-22T21:13:33.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e4800640 1 -- start start 2024-05-22T21:13:33.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e4800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2c40047d0 0x7ff2dc0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e4800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2dc15d0e0 0x7ff2dc15a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e4800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc0ae710 0x7ff2dc0aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e4800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0af100 con 0x7ff2dc0ae710 2024-05-22T21:13:33.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e4800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc15d810 con 0x7ff2dc15d0e0 2024-05-22T21:13:33.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e4800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0a84a0 con 0x7ff2c40047d0 2024-05-22T21:13:33.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e2800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc0ae710 0x7ff2dc0aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e2800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc0ae710 0x7ff2dc0aebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49066/0 (socket says 172.21.15.161:49066) 2024-05-22T21:13:33.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e2800640 1 -- 172.21.15.161:0/258396664 learned_addr learned my addr 172.21.15.161:0/258396664 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:33.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e1400640 1 --2- 172.21.15.161:0/258396664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2dc15d0e0 0x7ff2dc15a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.230+0000 7ff2e1e00640 1 --2- 172.21.15.161:0/258396664 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2c40047d0 0x7ff2dc0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.234+0000 7ff2e2800640 1 -- 172.21.15.161:0/258396664 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2c40047d0 msgr2=0x7ff2dc0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.234+0000 7ff2e2800640 1 --2- 172.21.15.161:0/258396664 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2c40047d0 0x7ff2dc0ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.234+0000 7ff2e2800640 1 -- 172.21.15.161:0/258396664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2dc15d0e0 msgr2=0x7ff2dc15a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.234+0000 7ff2e2800640 1 --2- 172.21.15.161:0/258396664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2dc15d0e0 0x7ff2dc15a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.234+0000 7ff2e2800640 1 -- 172.21.15.161:0/258396664 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc0a8780 con 0x7ff2dc0ae710 2024-05-22T21:13:33.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.234+0000 7ff2e2800640 1 --2- 172.21.15.161:0/258396664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc0ae710 0x7ff2dc0aebc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80059b0 tx=0x7ff2d8012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.234+0000 7ff2d6000640 1 -- 172.21.15.161:0/258396664 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8012ee0 con 0x7ff2dc0ae710 2024-05-22T21:13:33.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.234+0000 7ff2d6000640 1 -- 172.21.15.161:0/258396664 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d8010b20 con 0x7ff2dc0ae710 2024-05-22T21:13:33.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.234+0000 7ff2e4800640 1 -- 172.21.15.161:0/258396664 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2dc0a8a60 con 0x7ff2dc0ae710 2024-05-22T21:13:33.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.234+0000 7ff2d6000640 1 -- 172.21.15.161:0/258396664 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8010e90 con 0x7ff2dc0ae710 2024-05-22T21:13:33.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.234+0000 7ff2e4800640 1 -- 172.21.15.161:0/258396664 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2dc15d520 con 0x7ff2dc0ae710 2024-05-22T21:13:33.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.234+0000 7ff2e4800640 1 -- 172.21.15.161:0/258396664 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2dc0a8f80 con 0x7ff2dc0ae710 2024-05-22T21:13:33.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.234+0000 7ff2d6000640 1 -- 172.21.15.161:0/258396664 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff2d8016480 con 0x7ff2dc0ae710 2024-05-22T21:13:33.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.234+0000 7ff2d6000640 1 --2- 172.21.15.161:0/258396664 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff2c00666c0 0x7ff2c0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.234+0000 7ff2d6000640 1 -- 172.21.15.161:0/258396664 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2d80877e0 con 0x7ff2dc0ae710 2024-05-22T21:13:33.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.234+0000 7ff2e1e00640 1 --2- 172.21.15.161:0/258396664 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff2c00666c0 0x7ff2c0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.238+0000 7ff2e1e00640 1 --2- 172.21.15.161:0/258396664 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff2c00666c0 0x7ff2c0068b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00099e0 tx=0x7ff2d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.238+0000 7ff2d6000640 1 -- 172.21.15.161:0/258396664 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff2dc0a8f80 con 0x7ff2dc0ae710 2024-05-22T21:13:33.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.394+0000 7ff2e4800640 1 -- 172.21.15.161:0/258396664 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2dc0a8f80 con 0x7ff2dc0ae710 2024-05-22T21:13:33.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.394+0000 7ff2d6000640 1 -- 172.21.15.161:0/258396664 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7ff2dc0a8f80 con 0x7ff2dc0ae710 2024-05-22T21:13:33.398 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:33.398 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:33.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.394+0000 7ff2e4800640 1 -- 172.21.15.161:0/258396664 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff2c00666c0 msgr2=0x7ff2c0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.394+0000 7ff2e4800640 1 --2- 172.21.15.161:0/258396664 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff2c00666c0 0x7ff2c0068b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00099e0 tx=0x7ff2d0008040 comp rx=0 tx=0).stop 2024-05-22T21:13:33.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.394+0000 7ff2e4800640 1 -- 172.21.15.161:0/258396664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc0ae710 msgr2=0x7ff2dc0aebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.394+0000 7ff2e4800640 1 --2- 172.21.15.161:0/258396664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc0ae710 0x7ff2dc0aebc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80059b0 tx=0x7ff2d8012c90 comp rx=0 tx=0).stop 2024-05-22T21:13:33.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.394+0000 7ff2e4800640 1 -- 172.21.15.161:0/258396664 shutdown_connections 2024-05-22T21:13:33.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.394+0000 7ff2e4800640 1 --2- 172.21.15.161:0/258396664 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff2c00666c0 0x7ff2c0068b80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.394+0000 7ff2e4800640 1 --2- 172.21.15.161:0/258396664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2dc0ae710 0x7ff2dc0aebc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.394+0000 7ff2e4800640 1 --2- 172.21.15.161:0/258396664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2dc15d0e0 0x7ff2dc15a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.394+0000 7ff2e4800640 1 --2- 172.21.15.161:0/258396664 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2c40047d0 0x7ff2dc0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.394+0000 7ff2e4800640 1 -- 172.21.15.161:0/258396664 >> 172.21.15.161:0/258396664 conn(0x7ff2dc0b0bc0 msgr2=0x7ff2dc142e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.394+0000 7ff2e4800640 1 -- 172.21.15.161:0/258396664 shutdown_connections 2024-05-22T21:13:33.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.394+0000 7ff2e4800640 1 -- 172.21.15.161:0/258396664 wait complete. 2024-05-22T21:13:33.419 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-22T21:13:33.419 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-22T21:13:33.420 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-22T21:13:33.420 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-22T21:13:33.420 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-22T21:13:33.421 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-22T21:13:33.421 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-22T21:13:33.421 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-22T21:13:33.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.654+0000 7f8e73200640 1 Processor -- start 2024-05-22T21:13:33.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.654+0000 7fb32de00640 1 Processor -- start 2024-05-22T21:13:33.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.654+0000 7f8e73200640 1 -- start start 2024-05-22T21:13:33.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7f8e73200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8e7415ac70 0x7f8e740b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7f8e73200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e740b3120 0x7f8e740ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7f8e73200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e740b2140 con 0x7f8e740acdf0 2024-05-22T21:13:33.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7f8e73200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e7406fed0 con 0x7f8e7415ac70 2024-05-22T21:13:33.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7f8e73200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e740b23e0 con 0x7f8e740b3120 2024-05-22T21:13:33.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.654+0000 7fb32de00640 1 -- start start 2024-05-22T21:13:33.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7fb32de00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb320047bf0 0x7fb3200eb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7fb32de00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb320005e70 0x7fb3200062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7fb32de00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb320037b90 con 0x7fb320033e10 2024-05-22T21:13:33.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7fb32de00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3200501e0 con 0x7fb320047bf0 2024-05-22T21:13:33.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7fb32de00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32004f060 con 0x7fb320005e70 2024-05-22T21:13:33.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7f8e71400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e740b3120 0x7f8e740ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7f8e71400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e740b3120 0x7f8e740ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:53288/0 (socket says 172.21.15.161:53288) 2024-05-22T21:13:33.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7f8e71400640 1 -- 172.21.15.161:0/3183526785 learned_addr learned my addr 172.21.15.161:0/3183526785 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:33.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7f8e71e00640 1 --2- 172.21.15.161:0/3183526785 >> v2:172.21.15.179:3300/0 conn(0x7f8e7415ac70 0x7f8e740b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7fb32ca00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb320033e10 0x7fb3200eb150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:60698/0 (socket says 172.21.15.161:60698) 2024-05-22T21:13:33.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7fb32ca00640 1 -- 172.21.15.161:0/141452515 learned_addr learned my addr 172.21.15.161:0/141452515 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:33.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7fb327e00640 1 --2- 172.21.15.161:0/141452515 >> v2:172.21.15.161:3300/0 conn(0x7fb320047bf0 0x7fb3200eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7f8e71400640 1 -- 172.21.15.161:0/3183526785 >> v2:172.21.15.179:3300/0 conn(0x7f8e7415ac70 msgr2=0x7f8e740b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:33.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7fb32d400640 1 --2- 172.21.15.161:0/141452515 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb320005e70 0x7fb3200062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7fb327e00640 1 -- 172.21.15.161:0/141452515 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb320005e70 msgr2=0x7fb3200062a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:33.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7fb327e00640 1 --2- 172.21.15.161:0/141452515 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb320005e70 0x7fb3200062a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7fb327e00640 1 -- 172.21.15.161:0/141452515 >> v1:172.21.15.179:6789/0 conn(0x7fb320033e10 legacy=0x7fb3200eb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7fb327e00640 1 -- 172.21.15.161:0/141452515 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32004fd50 con 0x7fb320047bf0 2024-05-22T21:13:33.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.658+0000 7f8e71400640 1 --2- 172.21.15.161:0/3183526785 >> v2:172.21.15.179:3300/0 conn(0x7f8e7415ac70 0x7f8e740b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.662+0000 7f8e71400640 1 -- 172.21.15.161:0/3183526785 >> v1:172.21.15.179:6789/0 conn(0x7f8e740acdf0 legacy=0x7f8e7415d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.662+0000 7f8e71400640 1 -- 172.21.15.161:0/3183526785 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e740b26c0 con 0x7f8e740b3120 2024-05-22T21:13:33.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f8e71400640 1 --2- 172.21.15.161:0/3183526785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e740b3120 0x7f8e740ac8b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009870 tx=0x7f8e6401c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=249a896da900300 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f5e7d800640 1 Processor -- start 2024-05-22T21:13:33.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f8e70a00640 1 -- 172.21.15.161:0/3183526785 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e640ab600 con 0x7f8e740b3120 2024-05-22T21:13:33.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f8e70a00640 1 -- 172.21.15.161:0/3183526785 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e640a7050 con 0x7f8e740b3120 2024-05-22T21:13:33.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f8e70a00640 1 -- 172.21.15.161:0/3183526785 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e640ab980 con 0x7f8e740b3120 2024-05-22T21:13:33.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f5e7d800640 1 -- start start 2024-05-22T21:13:33.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7fb327e00640 1 --2- 172.21.15.161:0/141452515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb320047bf0 0x7fb3200eb860 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb3140097c0 tx=0x7fb31401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de87305b80ec912 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7fa6baa00640 1 Processor -- start 2024-05-22T21:13:33.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7fb327400640 1 -- 172.21.15.161:0/141452515 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3140a87d0 con 0x7fb320047bf0 2024-05-22T21:13:33.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7fb327400640 1 -- 172.21.15.161:0/141452515 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3140a8930 con 0x7fb320047bf0 2024-05-22T21:13:33.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f8e73200640 1 -- 172.21.15.161:0/3183526785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e740b3120 msgr2=0x7f8e740ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f8e73200640 1 --2- 172.21.15.161:0/3183526785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e740b3120 0x7f8e740ac8b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009870 tx=0x7f8e6401c360 comp rx=0 tx=0).stop 2024-05-22T21:13:33.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f8e73200640 1 -- 172.21.15.161:0/3183526785 shutdown_connections 2024-05-22T21:13:33.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f8e73200640 1 --2- 172.21.15.161:0/3183526785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e740b3120 0x7f8e740ac8b0 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009870 tx=0x7f8e6401c360 comp rx=0 tx=0).stop 2024-05-22T21:13:33.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f8e73200640 1 --2- 172.21.15.161:0/3183526785 >> v2:172.21.15.179:3300/0 conn(0x7f8e7415ac70 0x7f8e740b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f8e73200640 1 -- 172.21.15.161:0/3183526785 >> 172.21.15.161:0/3183526785 conn(0x7f8e740cec00 msgr2=0x7f8e740cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7fb32de00640 1 -- 172.21.15.161:0/141452515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb320047bf0 msgr2=0x7fb3200eb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7fb32de00640 1 --2- 172.21.15.161:0/141452515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb320047bf0 0x7fb3200eb860 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb3140097c0 tx=0x7fb31401ec50 comp rx=0 tx=0).stop 2024-05-22T21:13:33.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7fa6baa00640 1 -- start start 2024-05-22T21:13:33.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f5e7d800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5e780acd60 0x7f5e781613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7fa6baa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6b4070d80 0x7fa6b406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f5e77e00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f5e7815ac70 0x7f5e780b2be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:60700/0 (socket says 172.21.15.161:60700) 2024-05-22T21:13:33.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f5e7d800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e780b2140 con 0x7f5e780ac8b0 2024-05-22T21:13:33.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f5e7d800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7806fed0 con 0x7f5e7815ac70 2024-05-22T21:13:33.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f5e7d800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e780b23e0 con 0x7f5e780acd60 2024-05-22T21:13:33.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f5e77e00640 1 -- 172.21.15.161:0/2749376233 learned_addr learned my addr 172.21.15.161:0/2749376233 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:33.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7fa6baa00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b40b2710 con 0x7fa6b406c7e0 2024-05-22T21:13:33.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7fa6baa00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b40b29f0 con 0x7fa6b414b1c0 2024-05-22T21:13:33.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7fa6baa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b406cc70 con 0x7fa6b4070d80 2024-05-22T21:13:33.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fa6b8a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa6b406c7e0 0x7fa6b40ccd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56170/0 (socket says 172.21.15.161:56170) 2024-05-22T21:13:33.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fa6b8a00640 1 -- 172.21.15.161:0/3168522196 learned_addr learned my addr 172.21.15.161:0/3168522196 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:33.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fe492200640 1 Processor -- start 2024-05-22T21:13:33.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fb32de00640 1 -- 172.21.15.161:0/141452515 shutdown_connections 2024-05-22T21:13:33.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fb32de00640 1 --2- 172.21.15.161:0/141452515 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb320005e70 0x7fb3200062a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fb32de00640 1 --2- 172.21.15.161:0/141452515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb320047bf0 0x7fb3200eb860 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fb32de00640 1 -- 172.21.15.161:0/141452515 >> 172.21.15.161:0/141452515 conn(0x7fb320009f90 msgr2=0x7fb32000a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 -- 172.21.15.161:0/2749376233 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746418601 0 0) 0x7f5e7806fed0 con 0x7f5e7815ac70 2024-05-22T21:13:33.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 -- 172.21.15.161:0/2749376233 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e5c003530 con 0x7f5e7815ac70 2024-05-22T21:13:33.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e7ce00640 1 --2- 172.21.15.161:0/2749376233 >> v2:172.21.15.179:3300/0 conn(0x7f5e780acd60 0x7f5e781613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f8e73200640 1 -- 172.21.15.161:0/3183526785 shutdown_connections 2024-05-22T21:13:33.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.666+0000 7f8e73200640 1 -- 172.21.15.161:0/3183526785 wait complete. 2024-05-22T21:13:33.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fe492200640 1 -- start start 2024-05-22T21:13:33.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fb32de00640 1 -- 172.21.15.161:0/141452515 shutdown_connections 2024-05-22T21:13:33.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fa6b3400640 1 --2- 172.21.15.161:0/3168522196 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6b4070d80 0x7fa6b406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fa6b2a00640 1 -- 172.21.15.161:0/3168522196 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956147150 0 0) 0x7fa6b40b2710 con 0x7fa6b406c7e0 2024-05-22T21:13:33.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 -- 172.21.15.161:0/2749376233 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2200228964 0 0) 0x7f5e780b2140 con 0x7f5e780ac8b0 2024-05-22T21:13:33.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fb32de00640 1 -- 172.21.15.161:0/141452515 wait complete. 2024-05-22T21:13:33.678 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fa6b2a00640 1 -- 172.21.15.161:0/3168522196 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa698003530 con 0x7fa6b406c7e0 2024-05-22T21:13:33.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 -- 172.21.15.161:0/2749376233 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e7806fed0 con 0x7f5e780ac8b0 2024-05-22T21:13:33.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 -- 172.21.15.161:0/2749376233 <== mon.1 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2783679100 0 0) 0x7f5e5c003530 con 0x7f5e7815ac70 2024-05-22T21:13:33.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 -- 172.21.15.161:0/2749376233 >> v2:172.21.15.179:3300/0 conn(0x7f5e780acd60 msgr2=0x7f5e781613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:33.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 --2- 172.21.15.161:0/2749376233 >> v2:172.21.15.179:3300/0 conn(0x7f5e780acd60 0x7f5e781613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 -- 172.21.15.161:0/2749376233 >> v1:172.21.15.161:6789/0 conn(0x7f5e780ac8b0 legacy=0x7f5e78160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 -- 172.21.15.161:0/2749376233 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e780b26c0 con 0x7f5e7815ac70 2024-05-22T21:13:33.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 -- 172.21.15.161:0/2749376233 <== mon.1 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f5e68001680 con 0x7f5e7815ac70 2024-05-22T21:13:33.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 -- 172.21.15.161:0/2749376233 >> v1:172.21.15.179:6789/0 conn(0x7f5e7815ac70 legacy=0x7f5e780b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 --2- 172.21.15.161:0/2749376233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e5c003b10 0x7f5e5c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 --2- 172.21.15.161:0/2749376233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e5c0047d0 0x7f5e5c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 --2- 172.21.15.161:0/2749376233 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e5c005190 0x7f5e5c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 -- 172.21.15.161:0/2749376233 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780b23e0 con 0x7f5e5c003b10 2024-05-22T21:13:33.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e77e00640 1 --2- 172.21.15.161:0/2749376233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e5c003b10 0x7f5e5c003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f8e73200640 1 Processor -- start 2024-05-22T21:13:33.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e77400640 1 --2- 172.21.15.161:0/2749376233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e5c0047d0 0x7f5e5c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 -- 172.21.15.161:0/2749376233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c00c010 con 0x7f5e5c0047d0 2024-05-22T21:13:33.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e7ce00640 1 --2- 172.21.15.161:0/2749376233 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e5c005190 0x7f5e5c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e76a00640 1 -- 172.21.15.161:0/2749376233 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c00c1d0 con 0x7f5e5c005190 2024-05-22T21:13:33.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fe492200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe48c13f6b0 0x7fe48c0b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f9fba600640 1 Processor -- start 2024-05-22T21:13:33.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f8e73200640 1 -- start start 2024-05-22T21:13:33.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e7ce00640 1 -- 172.21.15.161:0/2749376233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e5c0047d0 msgr2=0x7f5e5c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:33.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f9fba600640 1 -- start start 2024-05-22T21:13:33.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fe492200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe48c15d450 0x7fe48c06c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fe492200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe48c142dd0 con 0x7fe48c0b1b10 2024-05-22T21:13:33.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7fe491800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe48c15d450 0x7fe48c06c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7fe491800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe48c15d450 0x7fe48c06c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:39486/0 (socket says 172.21.15.161:39486) 2024-05-22T21:13:33.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7fe491800640 1 -- 172.21.15.161:0/3072810906 learned_addr learned my addr 172.21.15.161:0/3072810906 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:33.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7fe492200640 1 -- 172.21.15.161:0/3072810906 --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe48c06bae0 con 0x7fe48c15d450 2024-05-22T21:13:33.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7fe492200640 1 -- 172.21.15.161:0/3072810906 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c13ff60 con 0x7fe48c13f6b0 2024-05-22T21:13:33.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fa6b3400640 1 -- 172.21.15.161:0/3168522196 >> v1:172.21.15.179:6789/0 conn(0x7fa6b414b1c0 legacy=0x7fa6b40b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fa6b3400640 1 -- 172.21.15.161:0/3168522196 >> v1:172.21.15.161:6789/0 conn(0x7fa6b406c7e0 legacy=0x7fa6b40ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fa6b3400640 1 -- 172.21.15.161:0/3168522196 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b40b2cd0 con 0x7fa6b4070d80 2024-05-22T21:13:33.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7fa6b2a00640 1 -- 172.21.15.161:0/3168522196 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393113929 0 0) 0x7fa6b40b29f0 con 0x7fa6b414b1c0 2024-05-22T21:13:33.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7fe490e00640 1 --2- 172.21.15.161:0/3072810906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe48c13f6b0 0x7fe48c0b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7fe490e00640 1 -- 172.21.15.161:0/3072810906 >> v2:172.21.15.179:3300/0 conn(0x7fe48c15d450 msgr2=0x7fe48c06c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:33.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7fb32de00640 1 Processor -- start 2024-05-22T21:13:33.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e7ce00640 1 --2- 172.21.15.161:0/2749376233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e5c0047d0 0x7f5e5c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e7ce00640 1 -- 172.21.15.161:0/2749376233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e5c003b10 msgr2=0x7f5e5c003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:33.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e7ce00640 1 --2- 172.21.15.161:0/2749376233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e5c003b10 0x7f5e5c003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.670+0000 7f5e7ce00640 1 -- 172.21.15.161:0/2749376233 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c003530 con 0x7f5e5c005190 2024-05-22T21:13:33.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f8e73200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e740acdf0 0x7f8e7414d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f5e77400640 1 --2- 172.21.15.161:0/2749376233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e5c0047d0 0x7f5e5c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:33.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f8e73200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e7415ac70 0x7f8e740cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7fe48b400640 1 -- 172.21.15.161:0/3072810906 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808017693 0 0) 0x7fe478001120 con 0x7fe48c0b1b10 2024-05-22T21:13:33.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7fb32de00640 1 -- start start 2024-05-22T21:13:33.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f8e71e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e740acdf0 0x7f8e7414d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7fe490e00640 1 --2- 172.21.15.161:0/3072810906 >> v2:172.21.15.179:3300/0 conn(0x7fe48c15d450 0x7fe48c06c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f8e71e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e740acdf0 0x7f8e7414d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:39490/0 (socket says 172.21.15.161:39490) 2024-05-22T21:13:33.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f9fba600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9fb40ce3c0 0x7f9fb40b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7fe490e00640 1 -- 172.21.15.161:0/3072810906 >> v1:172.21.15.179:6789/0 conn(0x7fe48c0b1b10 legacy=0x7fe48c15cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f8e73200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e740cdfe0 0x7f8e74069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7fe490e00640 1 -- 172.21.15.161:0/3072810906 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48c0b1fa0 con 0x7fe48c13f6b0 2024-05-22T21:13:33.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f8e71e00640 1 -- 172.21.15.161:0/62662299 learned_addr learned my addr 172.21.15.161:0/62662299 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:33.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f9fba600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9fb406cae0 0x7f9fb40b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f8e73200640 1 -- 172.21.15.161:0/62662299 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7414daa0 con 0x7f8e7415ac70 2024-05-22T21:13:33.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f8e73200640 1 -- 172.21.15.161:0/62662299 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e740cbb50 con 0x7f8e740acdf0 2024-05-22T21:13:33.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f8e73200640 1 -- 172.21.15.161:0/62662299 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e740cbd60 con 0x7f8e740cdfe0 2024-05-22T21:13:33.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f8e72800640 1 --2- 172.21.15.161:0/62662299 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e740cdfe0 0x7f8e74069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f9fba600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb40cdf20 con 0x7f9fb41591f0 2024-05-22T21:13:33.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f9fba600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb40b23d0 con 0x7f9fb40ce3c0 2024-05-22T21:13:33.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7fe491800640 1 --2- 172.21.15.161:0/3072810906 >> v2:172.21.15.179:3300/0 conn(0x7fe48c15d450 0x7fe48c06c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:33.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f9fba600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb40cd960 con 0x7f9fb406cae0 2024-05-22T21:13:33.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f9fb3e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9fb406cae0 0x7f9fb40b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f9fb3e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9fb406cae0 0x7f9fb40b2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:53320/0 (socket says 172.21.15.161:53320) 2024-05-22T21:13:33.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f9fb3e00640 1 -- 172.21.15.161:0/657704867 learned_addr learned my addr 172.21.15.161:0/657704867 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:33.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f8e72800640 1 -- 172.21.15.161:0/62662299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e740acdf0 msgr2=0x7f8e7414d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f8e72800640 1 --2- 172.21.15.161:0/62662299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e740acdf0 0x7f8e7414d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f8e72800640 1 -- 172.21.15.161:0/62662299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e7415ac70 msgr2=0x7f8e740cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f8e72800640 1 --2- 172.21.15.161:0/62662299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e7415ac70 0x7f8e740cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.674+0000 7f8e72800640 1 -- 172.21.15.161:0/62662299 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e640094d0 con 0x7f8e740cdfe0 2024-05-22T21:13:33.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f8e72800640 1 --2- 172.21.15.161:0/62662299 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e740cdfe0 0x7f8e74069ed0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8e60009dc0 tx=0x7f8e6000c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb32de00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb320005e70 0x7fb3200e7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb32de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb320033e10 0x7fb3200e54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb32de00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb320047bf0 0x7fb3200e59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f9fb3e00640 1 -- 172.21.15.161:0/657704867 >> v2:172.21.15.161:3300/0 conn(0x7f9fb40ce3c0 msgr2=0x7f9fb40b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:33.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f9fb3e00640 1 --2- 172.21.15.161:0/657704867 >> v2:172.21.15.161:3300/0 conn(0x7f9fb40ce3c0 0x7f9fb40b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb32de00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3200d30b0 con 0x7fb320033e10 2024-05-22T21:13:33.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f9fb3e00640 1 -- 172.21.15.161:0/657704867 >> v1:172.21.15.161:6789/0 conn(0x7f9fb41591f0 legacy=0x7f9fb406c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:33.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f9fb3e00640 1 -- 172.21.15.161:0/657704867 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb40cdc40 con 0x7f9fb406cae0 2024-05-22T21:13:33.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb32de00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3200e6ed0 con 0x7fb320047bf0 2024-05-22T21:13:33.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb32de00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3200d0830 con 0x7fb320005e70 2024-05-22T21:13:33.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fa6b3400640 1 --2- 172.21.15.161:0/3168522196 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6b4070d80 0x7fa6b406c2a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00097f0 tx=0x7fa6a001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=215f2cf7d9864b92 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb327e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb320033e10 0x7fb3200e54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb327e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb320033e10 0x7fb3200e54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49108/0 (socket says 172.21.15.161:49108) 2024-05-22T21:13:33.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb327e00640 1 -- 172.21.15.161:0/3421796721 learned_addr learned my addr 172.21.15.161:0/3421796721 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:33.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f5e77e00640 1 --2- 172.21.15.161:0/2749376233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e5c003b10 0x7f5e5c003ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:33.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f5e7ce00640 1 -- 172.21.15.161:0/2749376233 reap_dead start 2024-05-22T21:13:33.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb32ca00640 1 --2- 172.21.15.161:0/3421796721 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb320005e70 0x7fb3200e7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb32d400640 1 --2- 172.21.15.161:0/3421796721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb320047bf0 0x7fb3200e59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fa6b2a00640 1 -- 172.21.15.161:0/3168522196 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a00ab670 con 0x7fa6b4070d80 2024-05-22T21:13:33.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fa6b2a00640 1 -- 172.21.15.161:0/3168522196 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a00ab7d0 con 0x7fa6b4070d80 2024-05-22T21:13:33.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fa6b2a00640 1 -- 172.21.15.161:0/3168522196 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a00abb60 con 0x7fa6b4070d80 2024-05-22T21:13:33.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f5e7ce00640 1 --2- 172.21.15.161:0/2749376233 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e5c005190 0x7f5e5c00b990 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009fc0 tx=0x7f5e7001cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35893d5934114296 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb327e00640 1 -- 172.21.15.161:0/3421796721 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb320005e70 msgr2=0x7fb3200e7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f8e6a000640 1 -- 172.21.15.161:0/62662299 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e60018070 con 0x7f8e740cdfe0 2024-05-22T21:13:33.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f8e6a000640 1 -- 172.21.15.161:0/62662299 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6000ccf0 con 0x7f8e740cdfe0 2024-05-22T21:13:33.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f8e6a000640 1 -- 172.21.15.161:0/62662299 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6000cea0 con 0x7f8e740cdfe0 2024-05-22T21:13:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb327e00640 1 --2- 172.21.15.161:0/3421796721 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb320005e70 0x7fb3200e7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb327e00640 1 -- 172.21.15.161:0/3421796721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb320047bf0 msgr2=0x7fb3200e59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb327e00640 1 --2- 172.21.15.161:0/3421796721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb320047bf0 0x7fb3200e59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb327e00640 1 -- 172.21.15.161:0/3421796721 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb314009420 con 0x7fb320033e10 2024-05-22T21:13:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f5e76a00640 1 -- 172.21.15.161:0/2749376233 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e700a5c80 con 0x7f5e5c005190 2024-05-22T21:13:33.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f8e73200640 1 -- 172.21.15.161:0/62662299 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e740cc040 con 0x7f8e740cdfe0 2024-05-22T21:13:33.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f8e73200640 1 -- 172.21.15.161:0/62662299 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7415bad0 con 0x7f8e740cdfe0 2024-05-22T21:13:33.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb327e00640 1 --2- 172.21.15.161:0/3421796721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb320033e10 0x7fb3200e54b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb3140b3040 tx=0x7fb31401b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fa6baa00640 1 -- 172.21.15.161:0/3168522196 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6b4070d80 msgr2=0x7fa6b406c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fa6baa00640 1 --2- 172.21.15.161:0/3168522196 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6b4070d80 0x7fa6b406c2a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00097f0 tx=0x7fa6a001c4b0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f5e76a00640 1 -- 172.21.15.161:0/2749376233 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7000f040 con 0x7f5e5c005190 2024-05-22T21:13:33.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f5e7d800640 1 -- 172.21.15.161:0/2749376233 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e5c005190 msgr2=0x7f5e5c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f5e7d800640 1 --2- 172.21.15.161:0/2749376233 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e5c005190 0x7f5e5c00b990 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009fc0 tx=0x7f5e7001cc90 comp rx=0 tx=0).stop 2024-05-22T21:13:33.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f5e7d800640 1 -- 172.21.15.161:0/2749376233 shutdown_connections 2024-05-22T21:13:33.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f5e7d800640 1 --2- 172.21.15.161:0/2749376233 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e5c005190 0x7f5e5c00b990 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f5e7d800640 1 -- 172.21.15.161:0/2749376233 >> 172.21.15.161:0/2749376233 conn(0x7f5e780cec00 msgr2=0x7f5e780cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7ffa74c00640 1 Processor -- start 2024-05-22T21:13:33.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb324c00640 1 -- 172.21.15.161:0/3421796721 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314003620 con 0x7fb320033e10 2024-05-22T21:13:33.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb324c00640 1 -- 172.21.15.161:0/3421796721 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb314003780 con 0x7fb320033e10 2024-05-22T21:13:33.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fb324c00640 1 -- 172.21.15.161:0/3421796721 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314003b30 con 0x7fb320033e10 2024-05-22T21:13:33.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fa6baa00640 1 -- 172.21.15.161:0/3168522196 shutdown_connections 2024-05-22T21:13:33.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fa6baa00640 1 --2- 172.21.15.161:0/3168522196 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6b4070d80 0x7fa6b406c2a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7fa6baa00640 1 -- 172.21.15.161:0/3168522196 >> 172.21.15.161:0/3168522196 conn(0x7fa6b406a2d0 msgr2=0x7fa6b406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f27d7a00640 1 Processor -- start 2024-05-22T21:13:33.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fa6baa00640 1 -- 172.21.15.161:0/3168522196 shutdown_connections 2024-05-22T21:13:33.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fa6baa00640 1 -- 172.21.15.161:0/3168522196 wait complete. 2024-05-22T21:13:33.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f27d7a00640 1 -- start start 2024-05-22T21:13:33.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7f5e7d800640 1 -- 172.21.15.161:0/2749376233 shutdown_connections 2024-05-22T21:13:33.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fe490e00640 1 --2- 172.21.15.161:0/3072810906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe48c13f6b0 0x7fe48c0b2620 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0097c0 tx=0x7fe47c01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84af9016317d3c01 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fb32de00640 1 -- 172.21.15.161:0/3421796721 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3200e6560 con 0x7fb320033e10 2024-05-22T21:13:33.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f8e73200640 1 -- 172.21.15.161:0/62662299 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8e4c000fd0 con 0x7f8e740cdfe0 2024-05-22T21:13:33.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fb32de00640 1 -- 172.21.15.161:0/3421796721 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3200d3300 con 0x7fb320033e10 2024-05-22T21:13:33.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f5e7d800640 1 -- 172.21.15.161:0/2749376233 wait complete. 2024-05-22T21:13:33.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.678+0000 7ffa74c00640 1 -- start start 2024-05-22T21:13:33.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fe48b400640 1 -- 172.21.15.161:0/3072810906 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0ab670 con 0x7fe48c13f6b0 2024-05-22T21:13:33.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f8e6a000640 1 -- 172.21.15.161:0/62662299 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8e60004260 con 0x7f8e740cdfe0 2024-05-22T21:13:33.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fa6baa00640 1 Processor -- start 2024-05-22T21:13:33.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fe48b400640 1 -- 172.21.15.161:0/3072810906 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c0ab7d0 con 0x7fe48c13f6b0 2024-05-22T21:13:33.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fe48b400640 1 -- 172.21.15.161:0/3072810906 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0ab670 con 0x7fe48c13f6b0 2024-05-22T21:13:33.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fe492200640 1 -- 172.21.15.161:0/3072810906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe48c13f6b0 msgr2=0x7fe48c0b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fe492200640 1 --2- 172.21.15.161:0/3072810906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe48c13f6b0 0x7fe48c0b2620 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0097c0 tx=0x7fe47c01c470 comp rx=0 tx=0).stop 2024-05-22T21:13:33.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7ffa74c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffa7015cd40 0x7ffa7015d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7ffa74c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa70142dd0 con 0x7ffa700b1b10 2024-05-22T21:13:33.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7ffa74c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7006bae0 con 0x7ffa700cf720 2024-05-22T21:13:33.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7ffa74c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa700b2d30 con 0x7ffa7015cd40 2024-05-22T21:13:33.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fb32de00640 1 -- 172.21.15.161:0/3421796721 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb32004f2b0 con 0x7fb320033e10 2024-05-22T21:13:33.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f8e6a000640 1 --2- 172.21.15.161:0/62662299 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e54066760 0x7f8e54068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f8e6a000640 1 -- 172.21.15.161:0/62662299 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e60086200 con 0x7f8e740cdfe0 2024-05-22T21:13:33.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fa6baa00640 1 -- start start 2024-05-22T21:13:33.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f27d7a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f27d00b16e0 0x7f27d00b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f27d7a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f27d006bea0 0x7f27d0158f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f27d7a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d00cca10 con 0x7f27d01594d0 2024-05-22T21:13:33.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f27d7a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27d00b3c00 con 0x7f27d006bea0 2024-05-22T21:13:33.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f27d7a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27d00ccc40 con 0x7f27d00b16e0 2024-05-22T21:13:33.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f5e7d800640 1 Processor -- start 2024-05-22T21:13:33.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f8e6a000640 1 --2- 172.21.15.161:0/62662299 >> [v2:172.21.15.161:6808/23413,v1:172.21.15.161:6809/23413] conn(0x7f8e5406e980 0x7f8e54070da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f8e6a000640 1 -- 172.21.15.161:0/62662299 --> [v2:172.21.15.161:6808/23413,v1:172.21.15.161:6809/23413] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e54071450 con 0x7f8e5406e980 2024-05-22T21:13:33.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f8e6a000640 1 -- 172.21.15.161:0/62662299 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e60088820 con 0x7f8e740cdfe0 2024-05-22T21:13:33.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f8e71e00640 1 --2- 172.21.15.161:0/62662299 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e54066760 0x7f8e54068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f5e7d800640 1 -- start start 2024-05-22T21:13:33.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f9fb3e00640 1 --2- 172.21.15.161:0/657704867 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9fb406cae0 0x7f9fb40b2d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00d1e0 tx=0x7f9fac01ebf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3c808b7544c37bd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fe492200640 1 -- 172.21.15.161:0/3072810906 shutdown_connections 2024-05-22T21:13:33.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fe492200640 1 --2- 172.21.15.161:0/3072810906 >> v2:172.21.15.179:3300/0 conn(0x7fe48c15d450 0x7fe48c06c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fe492200640 1 --2- 172.21.15.161:0/3072810906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe48c13f6b0 0x7fe48c0b2620 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7fe492200640 1 -- 172.21.15.161:0/3072810906 >> 172.21.15.161:0/3072810906 conn(0x7fe48c0cec00 msgr2=0x7fe48c0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7ffa6ea00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ffa700cf720 0x7ffa7013f450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:60740/0 (socket says 172.21.15.161:60740) 2024-05-22T21:13:33.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7ffa6ea00640 1 -- 172.21.15.161:0/2790760083 learned_addr learned my addr 172.21.15.161:0/2790760083 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:33.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7ffa6fe00640 1 --2- 172.21.15.161:0/2790760083 >> v2:172.21.15.179:3300/0 conn(0x7ffa7015cd40 0x7ffa7015d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f27d5000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f27d00b16e0 0x7f27d00b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f27d5000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f27d00b16e0 0x7f27d00b28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:39514/0 (socket says 172.21.15.161:39514) 2024-05-22T21:13:33.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.682+0000 7f27d5000640 1 -- 172.21.15.161:0/3029555720 learned_addr learned my addr 172.21.15.161:0/3029555720 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:33.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f8e71400640 1 --2- 172.21.15.161:0/62662299 >> [v2:172.21.15.161:6808/23413,v1:172.21.15.161:6809/23413] conn(0x7f8e5406e980 0x7f8e54070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7ffa6fe00640 1 -- 172.21.15.161:0/2790760083 >> v1:172.21.15.179:6789/0 conn(0x7ffa700cf720 legacy=0x7ffa7013f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7ffa6fe00640 1 -- 172.21.15.161:0/2790760083 >> v1:172.21.15.161:6789/0 conn(0x7ffa700b1b10 legacy=0x7ffa700b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:33.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7ffa6fe00640 1 -- 172.21.15.161:0/2790760083 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7015d6b0 con 0x7ffa7015cd40 2024-05-22T21:13:33.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f8e71e00640 1 --2- 172.21.15.161:0/62662299 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e54066760 0x7f8e54068c20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c003b90 tx=0x7f8e6c0098d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fb324c00640 1 -- 172.21.15.161:0/3421796721 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb3140b7040 con 0x7fb320033e10 2024-05-22T21:13:33.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6baa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6b406c7e0 0x7fa6b4160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6baa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6b4070d80 0x7fa6b4146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6baa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6b414b1c0 0x7fa6b40af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6baa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b4146270 con 0x7fa6b4070d80 2024-05-22T21:13:33.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6baa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b4160ee0 con 0x7fa6b414b1c0 2024-05-22T21:13:33.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6baa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b41455f0 con 0x7fa6b406c7e0 2024-05-22T21:13:33.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f5e7d800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e5c005190 0x7f5e78148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f5e7d800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e78144890 0x7f5e781494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f5e7d800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e7814acf0 0x7f5e78154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f5e7d800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e78147dc0 con 0x7f5e78144890 2024-05-22T21:13:33.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f5e7d800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e78149a20 con 0x7f5e5c005190 2024-05-22T21:13:33.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f5e7d800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7814a860 con 0x7f5e7814acf0 2024-05-22T21:13:33.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f9fb2000640 1 -- 172.21.15.161:0/657704867 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0a7be0 con 0x7f9fb406cae0 2024-05-22T21:13:33.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f9fb2000640 1 -- 172.21.15.161:0/657704867 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac0ab070 con 0x7f9fb406cae0 2024-05-22T21:13:33.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f9fb2000640 1 -- 172.21.15.161:0/657704867 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0b1400 con 0x7f9fb406cae0 2024-05-22T21:13:33.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fe492200640 1 -- 172.21.15.161:0/3072810906 shutdown_connections 2024-05-22T21:13:33.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f27cfe00640 1 --2- 172.21.15.161:0/3029555720 >> v2:172.21.15.161:3300/0 conn(0x7f27d006bea0 0x7f27d0158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fb324c00640 1 --2- 172.21.15.161:0/3421796721 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3080666e0 0x7fb308068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6b8a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6b414b1c0 0x7fa6b40af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6b8a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6b414b1c0 0x7fa6b40af1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:39522/0 (socket says 172.21.15.161:39522) 2024-05-22T21:13:33.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6b8a00640 1 -- 172.21.15.161:0/671494772 learned_addr learned my addr 172.21.15.161:0/671494772 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:33.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6b3e00640 1 --2- 172.21.15.161:0/671494772 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6b406c7e0 0x7fa6b4160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f5e77e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e5c005190 0x7f5e78148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f5e77e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e5c005190 0x7f5e78148950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:39532/0 (socket says 172.21.15.161:39532) 2024-05-22T21:13:33.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f5e77e00640 1 -- 172.21.15.161:0/128147225 learned_addr learned my addr 172.21.15.161:0/128147225 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:33.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f5e7ce00640 1 --2- 172.21.15.161:0/128147225 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e7814acf0 0x7f5e78154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f9fba600640 1 -- 172.21.15.161:0/657704867 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9fb406cae0 msgr2=0x7f9fb40b2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f9fba600640 1 --2- 172.21.15.161:0/657704867 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9fb406cae0 0x7f9fb40b2d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00d1e0 tx=0x7f9fac01ebf0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f9fba600640 1 -- 172.21.15.161:0/657704867 shutdown_connections 2024-05-22T21:13:33.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f9fba600640 1 --2- 172.21.15.161:0/657704867 >> v2:172.21.15.161:3300/0 conn(0x7f9fb40ce3c0 0x7f9fb40b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f9fba600640 1 --2- 172.21.15.161:0/657704867 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9fb406cae0 0x7f9fb40b2d30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f9fba600640 1 -- 172.21.15.161:0/657704867 >> 172.21.15.161:0/657704867 conn(0x7f9fb406a2d0 msgr2=0x7f9fb406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fe492200640 1 -- 172.21.15.161:0/3072810906 wait complete. 2024-05-22T21:13:33.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f8e71400640 1 --2- 172.21.15.161:0/62662299 >> [v2:172.21.15.161:6808/23413,v1:172.21.15.161:6809/23413] conn(0x7f8e5406e980 0x7f8e54070da0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fb324c00640 1 -- 172.21.15.161:0/3421796721 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb314121a20 con 0x7fb320033e10 2024-05-22T21:13:33.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f27d5000640 1 -- 172.21.15.161:0/3029555720 >> v2:172.21.15.161:3300/0 conn(0x7f27d006bea0 msgr2=0x7f27d0158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:33.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f27d5000640 1 --2- 172.21.15.161:0/3029555720 >> v2:172.21.15.161:3300/0 conn(0x7f27d006bea0 0x7f27d0158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f27d5000640 1 -- 172.21.15.161:0/3029555720 >> v1:172.21.15.161:6789/0 conn(0x7f27d01594d0 legacy=0x7f27d0070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:33.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f5e77e00640 1 -- 172.21.15.161:0/128147225 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e7814acf0 msgr2=0x7f5e78154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f5e77e00640 1 --2- 172.21.15.161:0/128147225 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e7814acf0 0x7f5e78154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f9fba600640 1 -- 172.21.15.161:0/657704867 shutdown_connections 2024-05-22T21:13:33.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f5e77e00640 1 -- 172.21.15.161:0/128147225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e78144890 msgr2=0x7f5e781494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f5e77e00640 1 --2- 172.21.15.161:0/128147225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e78144890 0x7f5e781494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fb324c00640 1 --2- 172.21.15.161:0/3421796721 >> [v2:172.21.15.161:6800/23412,v1:172.21.15.161:6801/23412] conn(0x7fb30806e900 0x7fb308070d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fb324c00640 1 -- 172.21.15.161:0/3421796721 --> [v2:172.21.15.161:6800/23412,v1:172.21.15.161:6801/23412] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3080713d0 con 0x7fb30806e900 2024-05-22T21:13:33.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fb324c00640 1 -- 172.21.15.161:0/3421796721 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3141262a0 con 0x7fb320033e10 2024-05-22T21:13:33.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6b3e00640 1 -- 172.21.15.161:0/671494772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6b414b1c0 msgr2=0x7fa6b40af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6b3e00640 1 --2- 172.21.15.161:0/671494772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6b414b1c0 0x7fa6b40af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6b3e00640 1 -- 172.21.15.161:0/671494772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6b4070d80 msgr2=0x7fa6b4146690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6b3e00640 1 --2- 172.21.15.161:0/671494772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6b4070d80 0x7fa6b4146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6b3e00640 1 -- 172.21.15.161:0/671494772 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a0009420 con 0x7fa6b406c7e0 2024-05-22T21:13:33.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fb32ca00640 1 --2- 172.21.15.161:0/3421796721 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3080666e0 0x7fb308068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6b3e00640 1 --2- 172.21.15.161:0/671494772 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6b406c7e0 0x7fa6b4160610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400c430 tx=0x7fa6a400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f5e77e00640 1 -- 172.21.15.161:0/128147225 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e78149c50 con 0x7f5e5c005190 2024-05-22T21:13:33.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f9fba600640 1 -- 172.21.15.161:0/657704867 wait complete. 2024-05-22T21:13:33.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6abe00640 1 -- 172.21.15.161:0/671494772 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a40054e0 con 0x7fa6b406c7e0 2024-05-22T21:13:33.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6abe00640 1 -- 172.21.15.161:0/671494772 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a4003600 con 0x7fa6b406c7e0 2024-05-22T21:13:33.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6abe00640 1 -- 172.21.15.161:0/671494772 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a40039b0 con 0x7fa6b406c7e0 2024-05-22T21:13:33.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6baa00640 1 -- 172.21.15.161:0/671494772 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b41458d0 con 0x7fa6b406c7e0 2024-05-22T21:13:33.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f8e6a000640 1 -- 172.21.15.161:0/62662299 <== osd.0 v2:172.21.15.161:6808/23413 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24459 (crc 0 0 0) 0x7f8e54071450 con 0x7f8e5406e980 2024-05-22T21:13:33.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f27d5000640 1 -- 172.21.15.161:0/3029555720 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d00b3e10 con 0x7f27d00b16e0 2024-05-22T21:13:33.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7fa6baa00640 1 -- 172.21.15.161:0/671494772 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b4145e90 con 0x7fa6b406c7e0 2024-05-22T21:13:33.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7fb32d400640 1 --2- 172.21.15.161:0/3421796721 >> [v2:172.21.15.161:6800/23412,v1:172.21.15.161:6801/23412] conn(0x7fb30806e900 0x7fb308070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.686+0000 7f5e77e00640 1 --2- 172.21.15.161:0/128147225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e5c005190 0x7f5e78148950 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5e680017b0 tx=0x7f5e68001c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7f5e6fe00640 1 -- 172.21.15.161:0/128147225 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6800e930 con 0x7f5e5c005190 2024-05-22T21:13:33.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7f5e6fe00640 1 -- 172.21.15.161:0/128147225 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6800ea90 con 0x7f5e5c005190 2024-05-22T21:13:33.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7f5e6fe00640 1 -- 172.21.15.161:0/128147225 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6800ee00 con 0x7f5e5c005190 2024-05-22T21:13:33.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7f5e7d800640 1 -- 172.21.15.161:0/128147225 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e78149f30 con 0x7f5e5c005190 2024-05-22T21:13:33.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7f5e7d800640 1 -- 172.21.15.161:0/128147225 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7814a4f0 con 0x7f5e5c005190 2024-05-22T21:13:33.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7f9fba600640 1 Processor -- start 2024-05-22T21:13:33.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7fe492200640 1 Processor -- start 2024-05-22T21:13:33.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7fb32ca00640 1 --2- 172.21.15.161:0/3421796721 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3080666e0 0x7fb308068ba0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb32802c980 tx=0x7fb328061a30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7ffa6fe00640 1 --2- 172.21.15.161:0/2790760083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7015cd40 0x7ffa7015d170 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffa600097c0 tx=0x7ffa6001ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3910984ffd25d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7fa6baa00640 1 -- 172.21.15.161:0/671494772 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa6b416f1e0 con 0x7fa6b406c7e0 2024-05-22T21:13:33.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7fe492200640 1 -- start start 2024-05-22T21:13:33.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7fb32d400640 1 --2- 172.21.15.161:0/3421796721 >> [v2:172.21.15.161:6800/23412,v1:172.21.15.161:6801/23412] conn(0x7fb30806e900 0x7fb308070d20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7fb324c00640 1 -- 172.21.15.161:0/3421796721 <== osd.1 v2:172.21.15.161:6800/23412 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24459 (crc 0 0 0) 0x7fb3080713d0 con 0x7fb30806e900 2024-05-22T21:13:33.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7fa6abe00640 1 -- 172.21.15.161:0/671494772 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa6a4003b10 con 0x7fa6b406c7e0 2024-05-22T21:13:33.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7f5e7d800640 1 -- 172.21.15.161:0/128147225 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5e7806fa80 con 0x7f5e5c005190 2024-05-22T21:13:33.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7f9fba600640 1 -- start start 2024-05-22T21:13:33.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7ffa6e000640 1 -- 172.21.15.161:0/2790760083 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa600a8950 con 0x7ffa7015cd40 2024-05-22T21:13:33.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7ffa6e000640 1 -- 172.21.15.161:0/2790760083 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa600a8ab0 con 0x7ffa7015cd40 2024-05-22T21:13:33.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7ffa6e000640 1 -- 172.21.15.161:0/2790760083 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa600a8e40 con 0x7ffa7015cd40 2024-05-22T21:13:33.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7f9fba600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fb406cae0 0x7f9fb41f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7f9fba600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9fb40ce3c0 0x7f9fb4154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7f9fba600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb41591f0 0x7f9fb415a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7fe492200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe48c13f6b0 0x7fe48c0cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7fe492200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe48c15d450 0x7fe48c159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7fe492200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe48c070820 0x7fe48c155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7fe492200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c13d0b0 con 0x7fe48c070820 2024-05-22T21:13:33.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7fe492200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c15bec0 con 0x7fe48c15d450 2024-05-22T21:13:33.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7fe492200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c15c1a0 con 0x7fe48c13f6b0 2024-05-22T21:13:33.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.690+0000 7fa6abe00640 1 --2- 172.21.15.161:0/671494772 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6940666e0 0x7fa694068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fa6abe00640 1 -- 172.21.15.161:0/671494772 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6a4083730 con 0x7fa6b406c7e0 2024-05-22T21:13:33.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fa6abe00640 1 --2- 172.21.15.161:0/671494772 >> [v2:172.21.15.161:6824/23414,v1:172.21.15.161:6825/23414] conn(0x7fa69406e900 0x7fa694070d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fba600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb4159a80 con 0x7f9fb406cae0 2024-05-22T21:13:33.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fb2a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9fb40ce3c0 0x7f9fb4154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fba600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb4159c10 con 0x7f9fb41591f0 2024-05-22T21:13:33.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fba600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb4155670 con 0x7f9fb40ce3c0 2024-05-22T21:13:33.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fb3e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb41591f0 0x7f9fb415a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fb2a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9fb40ce3c0 0x7f9fb4154c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:53364/0 (socket says 172.21.15.161:53364) 2024-05-22T21:13:33.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fb2a00640 1 -- 172.21.15.161:0/2921226238 learned_addr learned my addr 172.21.15.161:0/2921226238 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:33.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fe490e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe48c13f6b0 0x7fe48c0cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fe490e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe48c13f6b0 0x7fe48c0cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:53362/0 (socket says 172.21.15.161:53362) 2024-05-22T21:13:33.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fe490e00640 1 -- 172.21.15.161:0/298307011 learned_addr learned my addr 172.21.15.161:0/298307011 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:33.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7ffa74c00640 1 -- 172.21.15.161:0/2790760083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7015cd40 msgr2=0x7ffa7015d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7ffa74c00640 1 --2- 172.21.15.161:0/2790760083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7015cd40 0x7ffa7015d170 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffa600097c0 tx=0x7ffa6001ec50 comp rx=0 tx=0).stop 2024-05-22T21:13:33.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7ffa74c00640 1 -- 172.21.15.161:0/2790760083 shutdown_connections 2024-05-22T21:13:33.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7ffa74c00640 1 --2- 172.21.15.161:0/2790760083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7015cd40 0x7ffa7015d170 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7ffa74c00640 1 -- 172.21.15.161:0/2790760083 >> 172.21.15.161:0/2790760083 conn(0x7ffa700cec00 msgr2=0x7ffa700cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f27d5000640 1 --2- 172.21.15.161:0/3029555720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d00b16e0 0x7f27d00b28d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f27d006add0 tx=0x7f27c001ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48a369171e0476c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fa6abe00640 1 -- 172.21.15.161:0/671494772 --> [v2:172.21.15.161:6824/23414,v1:172.21.15.161:6825/23414] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6940713d0 con 0x7fa69406e900 2024-05-22T21:13:33.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fa6abe00640 1 -- 172.21.15.161:0/671494772 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6a4085d10 con 0x7fa6b406c7e0 2024-05-22T21:13:33.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7ffa74c00640 1 -- 172.21.15.161:0/2790760083 shutdown_connections 2024-05-22T21:13:33.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fe48be00640 1 --2- 172.21.15.161:0/298307011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe48c15d450 0x7fe48c159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fe491800640 1 --2- 172.21.15.161:0/298307011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe48c070820 0x7fe48c155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7ffa74c00640 1 -- 172.21.15.161:0/2790760083 wait complete. 2024-05-22T21:13:33.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fa6b3400640 1 --2- 172.21.15.161:0/671494772 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6940666e0 0x7fa694068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f5e6fe00640 1 -- 172.21.15.161:0/128147225 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5e68014470 con 0x7f5e5c005190 2024-05-22T21:13:33.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fb3e00640 1 -- 172.21.15.161:0/2921226238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9fb40ce3c0 msgr2=0x7f9fb4154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fb3e00640 1 --2- 172.21.15.161:0/2921226238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9fb40ce3c0 0x7f9fb4154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fb3e00640 1 -- 172.21.15.161:0/2921226238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fb406cae0 msgr2=0x7f9fb41f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fb3e00640 1 --2- 172.21.15.161:0/2921226238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fb406cae0 0x7f9fb41f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fb3e00640 1 -- 172.21.15.161:0/2921226238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fac0079d0 con 0x7f9fb41591f0 2024-05-22T21:13:33.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fe490e00640 1 -- 172.21.15.161:0/298307011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe48c15d450 msgr2=0x7fe48c159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fe490e00640 1 --2- 172.21.15.161:0/298307011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe48c15d450 0x7fe48c159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fe490e00640 1 -- 172.21.15.161:0/298307011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe48c070820 msgr2=0x7fe48c155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fe490e00640 1 --2- 172.21.15.161:0/298307011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe48c070820 0x7fe48c155eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fe490e00640 1 -- 172.21.15.161:0/298307011 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48400c050 con 0x7fe48c13f6b0 2024-05-22T21:13:33.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f27cf400640 1 -- 172.21.15.161:0/3029555720 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c00a89d0 con 0x7f27d00b16e0 2024-05-22T21:13:33.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f27cf400640 1 -- 172.21.15.161:0/3029555720 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c00a8b30 con 0x7f27d00b16e0 2024-05-22T21:13:33.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f27cf400640 1 -- 172.21.15.161:0/3029555720 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c00b0440 con 0x7f27d00b16e0 2024-05-22T21:13:33.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fa6b8a00640 1 --2- 172.21.15.161:0/671494772 >> [v2:172.21.15.161:6824/23414,v1:172.21.15.161:6825/23414] conn(0x7fa69406e900 0x7fa694070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fb3e00640 1 --2- 172.21.15.161:0/2921226238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb41591f0 0x7f9fb415a0a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0af000 tx=0x7f9fac01b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fab400640 1 -- 172.21.15.161:0/2921226238 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0bf070 con 0x7f9fb41591f0 2024-05-22T21:13:33.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fab400640 1 -- 172.21.15.161:0/2921226238 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac003e20 con 0x7f9fb41591f0 2024-05-22T21:13:33.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fab400640 1 -- 172.21.15.161:0/2921226238 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac004260 con 0x7f9fb41591f0 2024-05-22T21:13:33.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f27d7a00640 1 -- 172.21.15.161:0/3029555720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d00b16e0 msgr2=0x7f27d00b28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f27d7a00640 1 --2- 172.21.15.161:0/3029555720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d00b16e0 0x7f27d00b28d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f27d006add0 tx=0x7f27c001ecd0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f5e6fe00640 1 --2- 172.21.15.161:0/128147225 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5e58066760 0x7f5e58068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f5e6fe00640 1 -- 172.21.15.161:0/128147225 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e680846e0 con 0x7f5e5c005190 2024-05-22T21:13:33.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f5e6fe00640 1 --2- 172.21.15.161:0/128147225 >> [v2:172.21.15.161:6816/23415,v1:172.21.15.161:6817/23415] conn(0x7f5e5806e980 0x7f5e58070da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f5e6fe00640 1 -- 172.21.15.161:0/128147225 --> [v2:172.21.15.161:6816/23415,v1:172.21.15.161:6817/23415] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e58071450 con 0x7f5e5806e980 2024-05-22T21:13:33.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f5e6fe00640 1 -- 172.21.15.161:0/128147225 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e68084a90 con 0x7f5e5c005190 2024-05-22T21:13:33.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f27d7a00640 1 -- 172.21.15.161:0/3029555720 shutdown_connections 2024-05-22T21:13:33.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f27d7a00640 1 --2- 172.21.15.161:0/3029555720 >> v2:172.21.15.161:3300/0 conn(0x7f27d006bea0 0x7f27d0158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f27d7a00640 1 --2- 172.21.15.161:0/3029555720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d00b16e0 0x7f27d00b28d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f27d7a00640 1 -- 172.21.15.161:0/3029555720 >> 172.21.15.161:0/3029555720 conn(0x7f27d00b0bc0 msgr2=0x7f27d00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fa6b3400640 1 --2- 172.21.15.161:0/671494772 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6940666e0 0x7fa694068ba0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00af000 tx=0x7fa6a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f5e7ce00640 1 --2- 172.21.15.161:0/128147225 >> [v2:172.21.15.161:6816/23415,v1:172.21.15.161:6817/23415] conn(0x7f5e5806e980 0x7f5e58070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fba600640 1 -- 172.21.15.161:0/2921226238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb4155950 con 0x7f9fb41591f0 2024-05-22T21:13:33.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f9fba600640 1 -- 172.21.15.161:0/2921226238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb4155f10 con 0x7f9fb41591f0 2024-05-22T21:13:33.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fe490e00640 1 --2- 172.21.15.161:0/298307011 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe48c13f6b0 0x7fe48c0cb610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0ad040 tx=0x7fe47c019230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7ffa74c00640 1 Processor -- start 2024-05-22T21:13:33.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f27d7a00640 1 -- 172.21.15.161:0/3029555720 shutdown_connections 2024-05-22T21:13:33.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7ffa74c00640 1 -- start start 2024-05-22T21:13:33.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7f5e77400640 1 --2- 172.21.15.161:0/128147225 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5e58066760 0x7f5e58068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fe488c00640 1 -- 172.21.15.161:0/298307011 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c004510 con 0x7fe48c13f6b0 2024-05-22T21:13:33.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7fe488c00640 1 -- 172.21.15.161:0/298307011 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c0ab940 con 0x7fe48c13f6b0 2024-05-22T21:13:33.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7fe488c00640 1 -- 172.21.15.161:0/298307011 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0abcf0 con 0x7fe48c13f6b0 2024-05-22T21:13:33.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7fe492200640 1 -- 172.21.15.161:0/298307011 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47c009420 con 0x7fe48c13f6b0 2024-05-22T21:13:33.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7fe492200640 1 -- 172.21.15.161:0/298307011 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48c0b2ce0 con 0x7fe48c13f6b0 2024-05-22T21:13:33.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7f5e77400640 1 --2- 172.21.15.161:0/128147225 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5e58066760 0x7f5e58068c20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e7814c1a0 tx=0x7f5e64009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7f5e7ce00640 1 --2- 172.21.15.161:0/128147225 >> [v2:172.21.15.161:6816/23415,v1:172.21.15.161:6817/23415] conn(0x7f5e5806e980 0x7f5e58070da0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7fe482000640 1 -- 172.21.15.161:0/298307011 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe460000fd0 con 0x7fe48c13f6b0 2024-05-22T21:13:33.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7f9fa8c00640 1 -- 172.21.15.161:0/2921226238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9f88000fd0 con 0x7f9fb41591f0 2024-05-22T21:13:33.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7ffa74c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa700cf720 0x7ffa70135070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7ffa74c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7015cd40 0x7ffa700cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7ffa74c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffa700ce2a0 0x7ffa70158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7ffa74c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa70159be0 con 0x7ffa700cf720 2024-05-22T21:13:33.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7ffa74c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa70159ec0 con 0x7ffa7015cd40 2024-05-22T21:13:33.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7ffa74c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7015a1a0 con 0x7ffa700ce2a0 2024-05-22T21:13:33.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.694+0000 7f27d7a00640 1 -- 172.21.15.161:0/3029555720 wait complete. 2024-05-22T21:13:33.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7f27d7a00640 1 Processor -- start 2024-05-22T21:13:33.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7fa6b8a00640 1 --2- 172.21.15.161:0/671494772 >> [v2:172.21.15.161:6824/23414,v1:172.21.15.161:6825/23414] conn(0x7fa69406e900 0x7fa694070d20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7f27d7a00640 1 -- start start 2024-05-22T21:13:33.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7ffa6ea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7015cd40 0x7ffa700cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7ffa6f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa700cf720 0x7ffa70135070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7ffa6ea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7015cd40 0x7ffa700cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:39554/0 (socket says 172.21.15.161:39554) 2024-05-22T21:13:33.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7ffa6ea00640 1 -- 172.21.15.161:0/3424307681 learned_addr learned my addr 172.21.15.161:0/3424307681 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:33.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7ffa6fe00640 1 --2- 172.21.15.161:0/3424307681 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffa700ce2a0 0x7ffa70158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7ffa6f400640 1 -- 172.21.15.161:0/3424307681 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffa700ce2a0 msgr2=0x7ffa70158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7fa6abe00640 1 -- 172.21.15.161:0/671494772 <== osd.2 v2:172.21.15.161:6824/23414 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24459 (crc 0 0 0) 0x7fa6940713d0 con 0x7fa69406e900 2024-05-22T21:13:33.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7fe488c00640 1 -- 172.21.15.161:0/298307011 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe47c003620 con 0x7fe48c13f6b0 2024-05-22T21:13:33.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.698+0000 7f5e6fe00640 1 -- 172.21.15.161:0/128147225 <== osd.3 v2:172.21.15.161:6816/23415 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24459 (crc 0 0 0) 0x7f5e58071450 con 0x7f5e5806e980 2024-05-22T21:13:33.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f9fab400640 1 -- 172.21.15.161:0/2921226238 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9fac0043c0 con 0x7f9fb41591f0 2024-05-22T21:13:33.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7ffa6f400640 1 --2- 172.21.15.161:0/3424307681 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffa700ce2a0 0x7ffa70158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7ffa6f400640 1 -- 172.21.15.161:0/3424307681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7015cd40 msgr2=0x7ffa700cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7ffa6f400640 1 --2- 172.21.15.161:0/3424307681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7015cd40 0x7ffa700cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7ffa6f400640 1 -- 172.21.15.161:0/3424307681 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa60009420 con 0x7ffa700cf720 2024-05-22T21:13:33.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d7a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d006bea0 0x7f27d0147060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d7a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d00b16e0 0x7f27d00afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d7a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f27d01427d0 0x7f27d0137ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d7a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d01419e0 con 0x7f27d006bea0 2024-05-22T21:13:33.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d7a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d0137490 con 0x7f27d00b16e0 2024-05-22T21:13:33.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d7a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d0137a50 con 0x7f27d01427d0 2024-05-22T21:13:33.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7ffa6ea00640 1 --2- 172.21.15.161:0/3424307681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa7015cd40 0x7ffa700cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:33.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7ffa6fe00640 1 --2- 172.21.15.161:0/3424307681 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffa700ce2a0 0x7ffa70158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:33.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d5000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d006bea0 0x7f27d0147060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d5000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d006bea0 0x7f27d0147060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49172/0 (socket says 172.21.15.161:49172) 2024-05-22T21:13:33.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d5000640 1 -- 172.21.15.161:0/1399278674 learned_addr learned my addr 172.21.15.161:0/1399278674 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:33.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d5a00640 1 --2- 172.21.15.161:0/1399278674 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f27d01427d0 0x7f27d0137ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d5000640 1 -- 172.21.15.161:0/1399278674 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f27d01427d0 msgr2=0x7f27d0137ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d5000640 1 --2- 172.21.15.161:0/1399278674 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f27d01427d0 0x7f27d0137ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d5000640 1 -- 172.21.15.161:0/1399278674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d00b16e0 msgr2=0x7f27d00afdc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T21:13:33.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7fe488c00640 1 --2- 172.21.15.161:0/298307011 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe46c066710 0x7fe46c068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7fe488c00640 1 -- 172.21.15.161:0/298307011 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe47c11e580 con 0x7fe48c13f6b0 2024-05-22T21:13:33.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7fe488c00640 1 --2- 172.21.15.161:0/298307011 >> [v2:172.21.15.179:6808/23607,v1:172.21.15.179:6809/23607] conn(0x7fe46c06e930 0x7fe46c070d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7fe488c00640 1 -- 172.21.15.161:0/298307011 --> [v2:172.21.15.179:6808/23607,v1:172.21.15.179:6809/23607] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe46c071400 con 0x7fe46c06e930 2024-05-22T21:13:33.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7fe488c00640 1 -- 172.21.15.161:0/298307011 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe47c120c90 con 0x7fe48c13f6b0 2024-05-22T21:13:33.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7fe48be00640 1 --2- 172.21.15.161:0/298307011 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe46c066710 0x7fe46c068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d5000640 1 --2- 172.21.15.161:0/1399278674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27d00b16e0 0x7f27d00afdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d5000640 1 -- 172.21.15.161:0/1399278674 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c00094d0 con 0x7f27d006bea0 2024-05-22T21:13:33.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f9fab400640 1 --2- 172.21.15.161:0/2921226238 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9f94066760 0x7f9f94068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f9fab400640 1 -- 172.21.15.161:0/2921226238 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fac11ea40 con 0x7f9fb41591f0 2024-05-22T21:13:33.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f9fab400640 1 --2- 172.21.15.161:0/2921226238 >> [v2:172.21.15.179:6800/23606,v1:172.21.15.179:6801/23606] conn(0x7f9f9406e980 0x7f9f94070da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f9fab400640 1 -- 172.21.15.161:0/2921226238 --> [v2:172.21.15.179:6800/23606,v1:172.21.15.179:6801/23606] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9f94071450 con 0x7f9f9406e980 2024-05-22T21:13:33.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f9fab400640 1 -- 172.21.15.161:0/2921226238 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fac123240 con 0x7f9fb41591f0 2024-05-22T21:13:33.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7fe491800640 1 --2- 172.21.15.161:0/298307011 >> [v2:172.21.15.179:6808/23607,v1:172.21.15.179:6809/23607] conn(0x7fe46c06e930 0x7fe46c070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7fe48be00640 1 --2- 172.21.15.161:0/298307011 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe46c066710 0x7fe46c068bd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe478000f70 tx=0x7fe478009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d5000640 1 --2- 172.21.15.161:0/1399278674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d006bea0 0x7f27d0147060 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f27c001b8e0 tx=0x7f27c001ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f9fb3400640 1 --2- 172.21.15.161:0/2921226238 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9f94066760 0x7f9f94068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f9fb2a00640 1 --2- 172.21.15.161:0/2921226238 >> [v2:172.21.15.179:6800/23606,v1:172.21.15.179:6801/23606] conn(0x7f9f9406e980 0x7f9f94070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7ffa6f400640 1 --2- 172.21.15.161:0/3424307681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa700cf720 0x7ffa70135070 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800c700 tx=0x7ffa6800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27ccc00640 1 -- 172.21.15.161:0/1399278674 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c00b1d90 con 0x7f27d006bea0 2024-05-22T21:13:33.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d7a00640 1 -- 172.21.15.161:0/1399278674 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d0141c30 con 0x7f27d006bea0 2024-05-22T21:13:33.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27d7a00640 1 -- 172.21.15.161:0/1399278674 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d01376e0 con 0x7f27d006bea0 2024-05-22T21:13:33.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27ccc00640 1 -- 172.21.15.161:0/1399278674 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c00bb470 con 0x7f27d006bea0 2024-05-22T21:13:33.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f27ccc00640 1 -- 172.21.15.161:0/1399278674 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c00bb800 con 0x7f27d006bea0 2024-05-22T21:13:33.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7f9fb3400640 1 --2- 172.21.15.161:0/2921226238 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9f94066760 0x7f9f94068c20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009650 tx=0x7f9fa4006bb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7ffa67400640 1 -- 172.21.15.161:0/3424307681 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6800aa50 con 0x7ffa700cf720 2024-05-22T21:13:33.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.702+0000 7fe491800640 1 --2- 172.21.15.161:0/298307011 >> [v2:172.21.15.179:6808/23607,v1:172.21.15.179:6809/23607] conn(0x7fe46c06e930 0x7fe46c070d50 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.706+0000 7ffa67400640 1 -- 172.21.15.161:0/3424307681 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6800abb0 con 0x7ffa700cf720 2024-05-22T21:13:33.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.706+0000 7ffa67400640 1 -- 172.21.15.161:0/3424307681 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6800ce10 con 0x7ffa700cf720 2024-05-22T21:13:33.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.706+0000 7f9fb2a00640 1 --2- 172.21.15.161:0/2921226238 >> [v2:172.21.15.179:6800/23606,v1:172.21.15.179:6801/23606] conn(0x7f9f9406e980 0x7f9f94070da0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.706+0000 7ffa74c00640 1 -- 172.21.15.161:0/3424307681 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa7015a480 con 0x7ffa700cf720 2024-05-22T21:13:33.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.706+0000 7ffa74c00640 1 -- 172.21.15.161:0/3424307681 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa70159840 con 0x7ffa700cf720 2024-05-22T21:13:33.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.706+0000 7fe488c00640 1 -- 172.21.15.161:0/298307011 <== osd.5 v2:172.21.15.179:6808/23607 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24459 (crc 0 0 0) 0x7fe46c071400 con 0x7fe46c06e930 2024-05-22T21:13:33.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.706+0000 7ffa64c00640 1 -- 172.21.15.161:0/3424307681 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa700b2280 con 0x7ffa700cf720 2024-05-22T21:13:33.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.706+0000 7f9fab400640 1 -- 172.21.15.161:0/2921226238 <== osd.4 v2:172.21.15.179:6800/23606 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24459 (crc 0 0 0) 0x7f9f94071450 con 0x7f9f9406e980 2024-05-22T21:13:33.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.706+0000 7f27ccc00640 1 -- 172.21.15.161:0/1399278674 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f27c00b7070 con 0x7f27d006bea0 2024-05-22T21:13:33.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.710+0000 7f27d7a00640 1 -- 172.21.15.161:0/1399278674 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f27a8000fd0 con 0x7f27d006bea0 2024-05-22T21:13:33.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.710+0000 7f27ccc00640 1 --2- 172.21.15.161:0/1399278674 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f27ac066710 0x7f27ac068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.710+0000 7f27ccc00640 1 -- 172.21.15.161:0/1399278674 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27c00f38d0 con 0x7f27d006bea0 2024-05-22T21:13:33.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.710+0000 7f27ccc00640 1 --2- 172.21.15.161:0/1399278674 >> [v2:172.21.15.179:6816/23609,v1:172.21.15.179:6817/23609] conn(0x7f27ac06f540 0x7f27ac0719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.710+0000 7f27ccc00640 1 -- 172.21.15.161:0/1399278674 --> [v2:172.21.15.179:6816/23609,v1:172.21.15.179:6817/23609] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27ac0720b0 con 0x7f27ac06f540 2024-05-22T21:13:33.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.710+0000 7f27ccc00640 1 -- 172.21.15.161:0/1399278674 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c00f2fa0 con 0x7f27d006bea0 2024-05-22T21:13:33.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.710+0000 7ffa67400640 1 -- 172.21.15.161:0/3424307681 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffa68012420 con 0x7ffa700cf720 2024-05-22T21:13:33.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.710+0000 7ffa67400640 1 --2- 172.21.15.161:0/3424307681 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffa500666e0 0x7ffa50068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.710+0000 7ffa67400640 1 -- 172.21.15.161:0/3424307681 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa68055bf0 con 0x7ffa700cf720 2024-05-22T21:13:33.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.714+0000 7f27d5a00640 1 --2- 172.21.15.161:0/1399278674 >> [v2:172.21.15.179:6816/23609,v1:172.21.15.179:6817/23609] conn(0x7f27ac06f540 0x7f27ac0719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.714+0000 7f27d5a00640 1 --2- 172.21.15.161:0/1399278674 >> [v2:172.21.15.179:6816/23609,v1:172.21.15.179:6817/23609] conn(0x7f27ac06f540 0x7f27ac0719e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.714+0000 7ffa6ea00640 1 --2- 172.21.15.161:0/3424307681 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffa500666e0 0x7ffa50068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.714+0000 7ffa67400640 1 --2- 172.21.15.161:0/3424307681 >> [v2:172.21.15.179:6820/23608,v1:172.21.15.179:6822/23608] conn(0x7ffa5006e900 0x7ffa50070d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:33.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.714+0000 7ffa67400640 1 -- 172.21.15.161:0/3424307681 --> [v2:172.21.15.179:6820/23608,v1:172.21.15.179:6822/23608] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa500713d0 con 0x7ffa5006e900 2024-05-22T21:13:33.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.714+0000 7ffa67400640 1 -- 172.21.15.161:0/3424307681 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa68003ee0 con 0x7ffa700cf720 2024-05-22T21:13:33.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.714+0000 7ffa6ea00640 1 --2- 172.21.15.161:0/3424307681 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffa500666e0 0x7ffa50068ba0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa700aed00 tx=0x7ffa5c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.714+0000 7f27cfe00640 1 --2- 172.21.15.161:0/1399278674 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f27ac066710 0x7f27ac068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.714+0000 7ffa6fe00640 1 --2- 172.21.15.161:0/3424307681 >> [v2:172.21.15.179:6820/23608,v1:172.21.15.179:6822/23608] conn(0x7ffa5006e900 0x7ffa50070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:33.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.714+0000 7f27ccc00640 1 -- 172.21.15.161:0/1399278674 <== osd.7 v2:172.21.15.179:6816/23609 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24459 (crc 0 0 0) 0x7f27ac0720b0 con 0x7f27ac06f540 2024-05-22T21:13:33.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.714+0000 7f8e73200640 1 -- 172.21.15.161:0/62662299 --> [v2:172.21.15.161:6808/23413,v1:172.21.15.161:6809/23413] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8e4c002cc0 con 0x7f8e5406e980 2024-05-22T21:13:33.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.714+0000 7f27cfe00640 1 --2- 172.21.15.161:0/1399278674 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f27ac066710 0x7f27ac068bd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f27d01391b0 tx=0x7f27b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.714+0000 7f8e6a000640 1 -- 172.21.15.161:0/62662299 <== osd.0 v2:172.21.15.161:6808/23413 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8e4c002cc0 con 0x7f8e5406e980 2024-05-22T21:13:33.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.714+0000 7ffa6fe00640 1 --2- 172.21.15.161:0/3424307681 >> [v2:172.21.15.179:6820/23608,v1:172.21.15.179:6822/23608] conn(0x7ffa5006e900 0x7ffa50070d20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:33.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.714+0000 7fb32de00640 1 -- 172.21.15.161:0/3421796721 --> [v2:172.21.15.161:6800/23412,v1:172.21.15.161:6801/23412] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb3200e91b0 con 0x7fb30806e900 2024-05-22T21:13:33.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7f8e73200640 1 -- 172.21.15.161:0/62662299 >> [v2:172.21.15.161:6808/23413,v1:172.21.15.161:6809/23413] conn(0x7f8e5406e980 msgr2=0x7f8e54070da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7f8e73200640 1 --2- 172.21.15.161:0/62662299 >> [v2:172.21.15.161:6808/23413,v1:172.21.15.161:6809/23413] conn(0x7f8e5406e980 0x7f8e54070da0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7fb324c00640 1 -- 172.21.15.161:0/3421796721 <== osd.1 v2:172.21.15.161:6800/23412 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb3200e91b0 con 0x7fb30806e900 2024-05-22T21:13:33.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7ffa67400640 1 -- 172.21.15.161:0/3424307681 <== osd.6 v2:172.21.15.179:6820/23608 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24459 (crc 0 0 0) 0x7ffa500713d0 con 0x7ffa5006e900 2024-05-22T21:13:33.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7f8e73200640 1 -- 172.21.15.161:0/62662299 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e54066760 msgr2=0x7f8e54068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7f8e73200640 1 --2- 172.21.15.161:0/62662299 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e54066760 0x7f8e54068c20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c003b90 tx=0x7f8e6c0098d0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7f8e73200640 1 -- 172.21.15.161:0/62662299 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e740cdfe0 msgr2=0x7f8e74069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7f8e73200640 1 --2- 172.21.15.161:0/62662299 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e740cdfe0 0x7f8e74069ed0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8e60009dc0 tx=0x7f8e6000c430 comp rx=0 tx=0).stop 2024-05-22T21:13:33.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7f8e72800640 1 -- 172.21.15.161:0/62662299 reap_dead start 2024-05-22T21:13:33.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7f8e73200640 1 -- 172.21.15.161:0/62662299 shutdown_connections 2024-05-22T21:13:33.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7f8e73200640 1 -- 172.21.15.161:0/62662299 >> 172.21.15.161:0/62662299 conn(0x7f8e740cec00 msgr2=0x7f8e74158610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7f8e73200640 1 -- 172.21.15.161:0/62662299 shutdown_connections 2024-05-22T21:13:33.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7f8e73200640 1 -- 172.21.15.161:0/62662299 wait complete. 2024-05-22T21:13:33.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7fb32de00640 1 -- 172.21.15.161:0/3421796721 >> [v2:172.21.15.161:6800/23412,v1:172.21.15.161:6801/23412] conn(0x7fb30806e900 msgr2=0x7fb308070d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7fb32de00640 1 --2- 172.21.15.161:0/3421796721 >> [v2:172.21.15.161:6800/23412,v1:172.21.15.161:6801/23412] conn(0x7fb30806e900 0x7fb308070d20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7fb32de00640 1 -- 172.21.15.161:0/3421796721 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3080666e0 msgr2=0x7fb308068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7fb32de00640 1 --2- 172.21.15.161:0/3421796721 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3080666e0 0x7fb308068ba0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb32802c980 tx=0x7fb328061a30 comp rx=0 tx=0).stop 2024-05-22T21:13:33.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7fb32de00640 1 -- 172.21.15.161:0/3421796721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb320033e10 msgr2=0x7fb3200e54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7fb32de00640 1 --2- 172.21.15.161:0/3421796721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb320033e10 0x7fb3200e54b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb3140b3040 tx=0x7fb31401b9f0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7fb32d400640 1 -- 172.21.15.161:0/3421796721 reap_dead start 2024-05-22T21:13:33.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7fb32de00640 1 -- 172.21.15.161:0/3421796721 shutdown_connections 2024-05-22T21:13:33.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7fb32de00640 1 -- 172.21.15.161:0/3421796721 >> 172.21.15.161:0/3421796721 conn(0x7fb320009f90 msgr2=0x7fb320039860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.718+0000 7fb32de00640 1 -- 172.21.15.161:0/3421796721 shutdown_connections 2024-05-22T21:13:33.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.722+0000 7fb32de00640 1 -- 172.21.15.161:0/3421796721 wait complete. 2024-05-22T21:13:33.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.726+0000 7fa6baa00640 1 -- 172.21.15.161:0/671494772 --> [v2:172.21.15.161:6824/23414,v1:172.21.15.161:6825/23414] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa6b4067220 con 0x7fa69406e900 2024-05-22T21:13:33.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.726+0000 7fa6abe00640 1 -- 172.21.15.161:0/671494772 <== osd.2 v2:172.21.15.161:6824/23414 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa6b4067220 con 0x7fa69406e900 2024-05-22T21:13:33.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.726+0000 7f5e7d800640 1 -- 172.21.15.161:0/128147225 --> [v2:172.21.15.161:6816/23415,v1:172.21.15.161:6817/23415] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5e7814a9a0 con 0x7f5e5806e980 2024-05-22T21:13:33.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.726+0000 7fa6baa00640 1 -- 172.21.15.161:0/671494772 >> [v2:172.21.15.161:6824/23414,v1:172.21.15.161:6825/23414] conn(0x7fa69406e900 msgr2=0x7fa694070d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.726+0000 7fa6baa00640 1 --2- 172.21.15.161:0/671494772 >> [v2:172.21.15.161:6824/23414,v1:172.21.15.161:6825/23414] conn(0x7fa69406e900 0x7fa694070d20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7f5e6fe00640 1 -- 172.21.15.161:0/128147225 <== osd.3 v2:172.21.15.161:6816/23415 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5e700a5a80 con 0x7f5e5806e980 2024-05-22T21:13:33.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7fa6baa00640 1 -- 172.21.15.161:0/671494772 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6940666e0 msgr2=0x7fa694068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7fa6baa00640 1 --2- 172.21.15.161:0/671494772 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6940666e0 0x7fa694068ba0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00af000 tx=0x7fa6a0008040 comp rx=0 tx=0).stop 2024-05-22T21:13:33.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7fa6baa00640 1 -- 172.21.15.161:0/671494772 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6b406c7e0 msgr2=0x7fa6b4160610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7fa6baa00640 1 --2- 172.21.15.161:0/671494772 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6b406c7e0 0x7fa6b4160610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400c430 tx=0x7fa6a400c8f0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7fa6b8a00640 1 -- 172.21.15.161:0/671494772 reap_dead start 2024-05-22T21:13:33.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7f5e7d800640 1 -- 172.21.15.161:0/128147225 >> [v2:172.21.15.161:6816/23415,v1:172.21.15.161:6817/23415] conn(0x7f5e5806e980 msgr2=0x7f5e58070da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7f5e7d800640 1 --2- 172.21.15.161:0/128147225 >> [v2:172.21.15.161:6816/23415,v1:172.21.15.161:6817/23415] conn(0x7f5e5806e980 0x7f5e58070da0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7f5e7d800640 1 -- 172.21.15.161:0/128147225 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5e58066760 msgr2=0x7f5e58068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7f5e7d800640 1 --2- 172.21.15.161:0/128147225 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5e58066760 0x7f5e58068c20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e7814c1a0 tx=0x7f5e64009040 comp rx=0 tx=0).stop 2024-05-22T21:13:33.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7f5e7d800640 1 -- 172.21.15.161:0/128147225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e5c005190 msgr2=0x7f5e78148950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7f5e7d800640 1 --2- 172.21.15.161:0/128147225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e5c005190 0x7f5e78148950 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5e680017b0 tx=0x7f5e68001c70 comp rx=0 tx=0).stop 2024-05-22T21:13:33.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7f5e7ce00640 1 -- 172.21.15.161:0/128147225 reap_dead start 2024-05-22T21:13:33.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7f5e7d800640 1 -- 172.21.15.161:0/128147225 shutdown_connections 2024-05-22T21:13:33.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7f5e7d800640 1 -- 172.21.15.161:0/128147225 >> 172.21.15.161:0/128147225 conn(0x7f5e780cec00 msgr2=0x7f5e78150060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7fa6baa00640 1 -- 172.21.15.161:0/671494772 shutdown_connections 2024-05-22T21:13:33.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7fa6baa00640 1 -- 172.21.15.161:0/671494772 >> 172.21.15.161:0/671494772 conn(0x7fa6b406a2d0 msgr2=0x7fa6b40b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7fa6baa00640 1 -- 172.21.15.161:0/671494772 shutdown_connections 2024-05-22T21:13:33.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7f5e7d800640 1 -- 172.21.15.161:0/128147225 shutdown_connections 2024-05-22T21:13:33.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7f5e7d800640 1 -- 172.21.15.161:0/128147225 wait complete. 2024-05-22T21:13:33.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7fa6baa00640 1 -- 172.21.15.161:0/671494772 wait complete. 2024-05-22T21:13:33.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.730+0000 7f9fa8c00640 1 -- 172.21.15.161:0/2921226238 --> [v2:172.21.15.179:6800/23606,v1:172.21.15.179:6801/23606] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9f88002cc0 con 0x7f9f9406e980 2024-05-22T21:13:33.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7fe482000640 1 -- 172.21.15.161:0/298307011 --> [v2:172.21.15.179:6808/23607,v1:172.21.15.179:6809/23607] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe460002cc0 con 0x7fe46c06e930 2024-05-22T21:13:33.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7f9fab400640 1 -- 172.21.15.161:0/2921226238 <== osd.4 v2:172.21.15.179:6800/23606 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9f88002cc0 con 0x7f9f9406e980 2024-05-22T21:13:33.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7fe488c00640 1 -- 172.21.15.161:0/298307011 <== osd.5 v2:172.21.15.179:6808/23607 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe460002cc0 con 0x7fe46c06e930 2024-05-22T21:13:33.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7fe482000640 1 -- 172.21.15.161:0/298307011 >> [v2:172.21.15.179:6808/23607,v1:172.21.15.179:6809/23607] conn(0x7fe46c06e930 msgr2=0x7fe46c070d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7fe482000640 1 --2- 172.21.15.161:0/298307011 >> [v2:172.21.15.179:6808/23607,v1:172.21.15.179:6809/23607] conn(0x7fe46c06e930 0x7fe46c070d50 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7f9fa8c00640 1 -- 172.21.15.161:0/2921226238 >> [v2:172.21.15.179:6800/23606,v1:172.21.15.179:6801/23606] conn(0x7f9f9406e980 msgr2=0x7f9f94070da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7f9fa8c00640 1 --2- 172.21.15.161:0/2921226238 >> [v2:172.21.15.179:6800/23606,v1:172.21.15.179:6801/23606] conn(0x7f9f9406e980 0x7f9f94070da0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7fe482000640 1 -- 172.21.15.161:0/298307011 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe46c066710 msgr2=0x7fe46c068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7fe482000640 1 --2- 172.21.15.161:0/298307011 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe46c066710 0x7fe46c068bd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe478000f70 tx=0x7fe478009040 comp rx=0 tx=0).stop 2024-05-22T21:13:33.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7fe482000640 1 -- 172.21.15.161:0/298307011 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe48c13f6b0 msgr2=0x7fe48c0cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7fe482000640 1 --2- 172.21.15.161:0/298307011 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe48c13f6b0 0x7fe48c0cb610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0ad040 tx=0x7fe47c019230 comp rx=0 tx=0).stop 2024-05-22T21:13:33.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7f9fa8c00640 1 -- 172.21.15.161:0/2921226238 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9f94066760 msgr2=0x7f9f94068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7f9fa8c00640 1 --2- 172.21.15.161:0/2921226238 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9f94066760 0x7f9f94068c20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009650 tx=0x7f9fa4006bb0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7f9fa8c00640 1 -- 172.21.15.161:0/2921226238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb41591f0 msgr2=0x7f9fb415a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7f9fa8c00640 1 --2- 172.21.15.161:0/2921226238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fb41591f0 0x7f9fb415a0a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0af000 tx=0x7f9fac01b990 comp rx=0 tx=0).stop 2024-05-22T21:13:33.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7f9fb3e00640 1 -- 172.21.15.161:0/2921226238 reap_dead start 2024-05-22T21:13:33.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7f9fa8c00640 1 -- 172.21.15.161:0/2921226238 shutdown_connections 2024-05-22T21:13:33.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7f9fa8c00640 1 -- 172.21.15.161:0/2921226238 >> 172.21.15.161:0/2921226238 conn(0x7f9fb406a2d0 msgr2=0x7f9fb40add10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7fe491800640 1 -- 172.21.15.161:0/298307011 reap_dead start 2024-05-22T21:13:33.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7fe482000640 1 -- 172.21.15.161:0/298307011 shutdown_connections 2024-05-22T21:13:33.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7fe482000640 1 -- 172.21.15.161:0/298307011 >> 172.21.15.161:0/298307011 conn(0x7fe48c0cec00 msgr2=0x7fe48c0afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7fe482000640 1 -- 172.21.15.161:0/298307011 shutdown_connections 2024-05-22T21:13:33.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7fe482000640 1 -- 172.21.15.161:0/298307011 wait complete. 2024-05-22T21:13:33.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7f9fa8c00640 1 -- 172.21.15.161:0/2921226238 shutdown_connections 2024-05-22T21:13:33.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.734+0000 7f9fa8c00640 1 -- 172.21.15.161:0/2921226238 wait complete. 2024-05-22T21:13:33.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.738+0000 7f27d7a00640 1 -- 172.21.15.161:0/1399278674 --> [v2:172.21.15.179:6816/23609,v1:172.21.15.179:6817/23609] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f27a8002cc0 con 0x7f27ac06f540 2024-05-22T21:13:33.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.738+0000 7f27ccc00640 1 -- 172.21.15.161:0/1399278674 <== osd.7 v2:172.21.15.179:6816/23609 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f27a8002cc0 con 0x7f27ac06f540 2024-05-22T21:13:33.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.738+0000 7f27c6000640 1 -- 172.21.15.161:0/1399278674 >> [v2:172.21.15.179:6816/23609,v1:172.21.15.179:6817/23609] conn(0x7f27ac06f540 msgr2=0x7f27ac0719e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.738+0000 7f27c6000640 1 --2- 172.21.15.161:0/1399278674 >> [v2:172.21.15.179:6816/23609,v1:172.21.15.179:6817/23609] conn(0x7f27ac06f540 0x7f27ac0719e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7f27c6000640 1 -- 172.21.15.161:0/1399278674 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f27ac066710 msgr2=0x7f27ac068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7f27c6000640 1 --2- 172.21.15.161:0/1399278674 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f27ac066710 0x7f27ac068bd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f27d01391b0 tx=0x7f27b8008040 comp rx=0 tx=0).stop 2024-05-22T21:13:33.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7f27c6000640 1 -- 172.21.15.161:0/1399278674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d006bea0 msgr2=0x7f27d0147060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7f27c6000640 1 --2- 172.21.15.161:0/1399278674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d006bea0 0x7f27d0147060 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f27c001b8e0 tx=0x7f27c001ba70 comp rx=0 tx=0).stop 2024-05-22T21:13:33.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7f27d5a00640 1 -- 172.21.15.161:0/1399278674 reap_dead start 2024-05-22T21:13:33.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7f27c6000640 1 -- 172.21.15.161:0/1399278674 shutdown_connections 2024-05-22T21:13:33.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7f27c6000640 1 -- 172.21.15.161:0/1399278674 >> 172.21.15.161:0/1399278674 conn(0x7f27d00b0bc0 msgr2=0x7f27d00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7ffa64c00640 1 -- 172.21.15.161:0/3424307681 --> [v2:172.21.15.179:6820/23608,v1:172.21.15.179:6822/23608] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffa7013fb60 con 0x7ffa5006e900 2024-05-22T21:13:33.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7f27c6000640 1 -- 172.21.15.161:0/1399278674 shutdown_connections 2024-05-22T21:13:33.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7f27c6000640 1 -- 172.21.15.161:0/1399278674 wait complete. 2024-05-22T21:13:33.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7ffa67400640 1 -- 172.21.15.161:0/3424307681 <== osd.6 v2:172.21.15.179:6820/23608 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffa600b07f0 con 0x7ffa5006e900 2024-05-22T21:13:33.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7ffa64c00640 1 -- 172.21.15.161:0/3424307681 >> [v2:172.21.15.179:6820/23608,v1:172.21.15.179:6822/23608] conn(0x7ffa5006e900 msgr2=0x7ffa50070d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7ffa64c00640 1 --2- 172.21.15.161:0/3424307681 >> [v2:172.21.15.179:6820/23608,v1:172.21.15.179:6822/23608] conn(0x7ffa5006e900 0x7ffa50070d20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7ffa64c00640 1 -- 172.21.15.161:0/3424307681 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffa500666e0 msgr2=0x7ffa50068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7ffa64c00640 1 --2- 172.21.15.161:0/3424307681 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffa500666e0 0x7ffa50068ba0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa700aed00 tx=0x7ffa5c009040 comp rx=0 tx=0).stop 2024-05-22T21:13:33.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7ffa64c00640 1 -- 172.21.15.161:0/3424307681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa700cf720 msgr2=0x7ffa70135070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:33.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7ffa64c00640 1 --2- 172.21.15.161:0/3424307681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa700cf720 0x7ffa70135070 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800c700 tx=0x7ffa6800cbc0 comp rx=0 tx=0).stop 2024-05-22T21:13:33.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.742+0000 7ffa6fe00640 1 -- 172.21.15.161:0/3424307681 reap_dead start 2024-05-22T21:13:33.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.746+0000 7ffa64c00640 1 -- 172.21.15.161:0/3424307681 shutdown_connections 2024-05-22T21:13:33.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.746+0000 7ffa64c00640 1 -- 172.21.15.161:0/3424307681 >> 172.21.15.161:0/3424307681 conn(0x7ffa700cec00 msgr2=0x7ffa70141860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:33.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.746+0000 7ffa64c00640 1 -- 172.21.15.161:0/3424307681 shutdown_connections 2024-05-22T21:13:33.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:33.746+0000 7ffa64c00640 1 -- 172.21.15.161:0/3424307681 wait complete. 2024-05-22T21:13:33.785 INFO:teuthology.orchestra.run.smithi161.stdout:55834574854 2024-05-22T21:13:33.786 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-22T21:13:33.786 INFO:teuthology.orchestra.run.smithi161.stdout:55834574854 2024-05-22T21:13:33.787 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-22T21:13:33.788 INFO:teuthology.orchestra.run.smithi161.stdout:55834574854 2024-05-22T21:13:33.788 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-22T21:13:33.789 INFO:teuthology.orchestra.run.smithi161.stdout:55834574854 2024-05-22T21:13:33.789 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-22T21:13:33.790 INFO:teuthology.orchestra.run.smithi161.stdout:55834574854 2024-05-22T21:13:33.790 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-22T21:13:33.791 INFO:teuthology.orchestra.run.smithi161.stdout:55834574854 2024-05-22T21:13:33.791 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-22T21:13:33.792 INFO:teuthology.orchestra.run.smithi161.stdout:55834574854 2024-05-22T21:13:33.792 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-22T21:13:33.793 INFO:teuthology.orchestra.run.smithi161.stdout:55834574854 2024-05-22T21:13:33.793 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-22T21:13:34.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.058+0000 7fc8c1a00640 1 Processor -- start 2024-05-22T21:13:34.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.058+0000 7fc8c1a00640 1 -- start start 2024-05-22T21:13:34.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.062+0000 7fc8c1a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8b400aab0 0x7fc8b40df280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.062+0000 7fc8c1a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc8b4033cd0 0x7fc8b404fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.062+0000 7fc8c1a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc8b404ffb0 0x7fc8b40e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.062+0000 7fc8bb400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc8b4033cd0 0x7fc8b404fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.062+0000 7fc8bb400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc8b4033cd0 0x7fc8b404fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49182/0 (socket says 172.21.15.161:49182) 2024-05-22T21:13:34.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.062+0000 7fc8bb400640 1 -- 172.21.15.161:0/2465415293 learned_addr learned my addr 172.21.15.161:0/2465415293 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:34.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.062+0000 7fc8c1000640 1 --2- 172.21.15.161:0/2465415293 >> v2:172.21.15.179:3300/0 conn(0x7fc8b404ffb0 0x7fc8b40e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.062+0000 7fc8c1a00640 1 -- 172.21.15.161:0/2465415293 --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b4050440 con 0x7fc8b4033cd0 2024-05-22T21:13:34.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.062+0000 7fc8c1a00640 1 -- 172.21.15.161:0/2465415293 --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b4038060 con 0x7fc8b404ffb0 2024-05-22T21:13:34.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.062+0000 7fc8c1a00640 1 -- 172.21.15.161:0/2465415293 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40540f0 con 0x7fc8b400aab0 2024-05-22T21:13:34.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.062+0000 7fc8bbe00640 1 --2- 172.21.15.161:0/2465415293 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8b400aab0 0x7fc8b40df280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.062+0000 7fc8bb400640 1 -- 172.21.15.161:0/2465415293 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8b400aab0 msgr2=0x7fc8b40df280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:34.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.062+0000 7fc8bb400640 1 --2- 172.21.15.161:0/2465415293 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8b400aab0 0x7fc8b40df280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.062+0000 7fc8bb400640 1 -- 172.21.15.161:0/2465415293 >> v2:172.21.15.179:3300/0 conn(0x7fc8b404ffb0 msgr2=0x7fc8b40e2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:34.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.066+0000 7fc8bb400640 1 --2- 172.21.15.161:0/2465415293 >> v2:172.21.15.179:3300/0 conn(0x7fc8b404ffb0 0x7fc8b40e2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.066+0000 7fc8bb400640 1 -- 172.21.15.161:0/2465415293 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b40543d0 con 0x7fc8b4033cd0 2024-05-22T21:13:34.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.066+0000 7fc8c1000640 1 --2- 172.21.15.161:0/2465415293 >> v2:172.21.15.179:3300/0 conn(0x7fc8b404ffb0 0x7fc8b40e2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:34.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.070+0000 7f01c7800640 1 Processor -- start 2024-05-22T21:13:34.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.070+0000 7f01c7800640 1 -- start start 2024-05-22T21:13:34.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.074+0000 7fc8bb400640 1 --2- 172.21.15.161:0/2465415293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8b4033cd0 0x7fc8b404fa70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400b2c0 tx=0x7fc8a801c110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c03ac875a5b90d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.074+0000 7fc8baa00640 1 -- 172.21.15.161:0/2465415293 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a80ab600 con 0x7fc8b4033cd0 2024-05-22T21:13:34.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.074+0000 7fc8baa00640 1 -- 172.21.15.161:0/2465415293 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a80a7050 con 0x7fc8b4033cd0 2024-05-22T21:13:34.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.074+0000 7fb3f7800640 1 Processor -- start 2024-05-22T21:13:34.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.074+0000 7fc8baa00640 1 -- 172.21.15.161:0/2465415293 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a80ab980 con 0x7fc8b4033cd0 2024-05-22T21:13:34.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.074+0000 7fb3f7800640 1 -- start start 2024-05-22T21:13:34.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.074+0000 7f01c4e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f01c014b1c0 0x7f01c00b1ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56202/0 (socket says 172.21.15.161:56202) 2024-05-22T21:13:34.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.074+0000 7f01c4e00640 1 -- 172.21.15.161:0/4214978811 learned_addr learned my addr 172.21.15.161:0/4214978811 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:34.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.074+0000 7f01c7800640 1 --2- 172.21.15.161:0/4214978811 >> v2:172.21.15.161:3300/0 conn(0x7f01c0070d80 0x7f01c006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.074+0000 7f01c7800640 1 --2- 172.21.15.161:0/4214978811 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f01c006c7e0 0x7f01c00ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.074+0000 7fc8c1a00640 1 -- 172.21.15.161:0/2465415293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8b4033cd0 msgr2=0x7fc8b404fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.074+0000 7fc8c1a00640 1 --2- 172.21.15.161:0/2465415293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8b4033cd0 0x7fc8b404fa70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400b2c0 tx=0x7fc8a801c110 comp rx=0 tx=0).stop 2024-05-22T21:13:34.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01c7800640 1 -- 172.21.15.161:0/4214978811 --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c00b2710 con 0x7f01c014b1c0 2024-05-22T21:13:34.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01c7800640 1 -- 172.21.15.161:0/4214978811 --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01c006cc70 con 0x7f01c0070d80 2024-05-22T21:13:34.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bf400640 1 -- 172.21.15.161:0/4214978811 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058306214 0 0) 0x7f01c00b2710 con 0x7f01c014b1c0 2024-05-22T21:13:34.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01c7800640 1 -- 172.21.15.161:0/4214978811 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c00b29f0 con 0x7f01c006c7e0 2024-05-22T21:13:34.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01c5800640 1 --2- 172.21.15.161:0/4214978811 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f01c006c7e0 0x7f01c00ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bf400640 1 -- 172.21.15.161:0/4214978811 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01a4003530 con 0x7f01c014b1c0 2024-05-22T21:13:34.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fb3f7800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3f015b600 0x7fb3f006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fb3f7800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb3f006c7e0 0x7fb3f00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fb3f7800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f0144bb0 con 0x7fb3f0143c40 2024-05-22T21:13:34.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fb3f7800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f015ba60 con 0x7fb3f006c7e0 2024-05-22T21:13:34.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fb3f7800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f006cc70 con 0x7fb3f015b600 2024-05-22T21:13:34.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bf400640 1 -- 172.21.15.161:0/4214978811 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 638759834 0 0) 0x7f01a4003530 con 0x7f01c014b1c0 2024-05-22T21:13:34.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bf400640 1 -- 172.21.15.161:0/4214978811 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f01c006c7e0 msgr2=0x7f01c00ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:34.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bf400640 1 --2- 172.21.15.161:0/4214978811 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f01c006c7e0 0x7f01c00ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bf400640 1 -- 172.21.15.161:0/4214978811 >> v2:172.21.15.161:3300/0 conn(0x7f01c0070d80 msgr2=0x7f01c006c2a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-22T21:13:34.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bf400640 1 --2- 172.21.15.161:0/4214978811 >> v2:172.21.15.161:3300/0 conn(0x7f01c0070d80 0x7f01c006c2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bf400640 1 -- 172.21.15.161:0/4214978811 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c00b2cd0 con 0x7f01c014b1c0 2024-05-22T21:13:34.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bf400640 1 -- 172.21.15.161:0/4214978811 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f01ac001680 con 0x7f01c014b1c0 2024-05-22T21:13:34.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bf400640 1 -- 172.21.15.161:0/4214978811 >> v1:172.21.15.161:6789/0 conn(0x7f01c014b1c0 legacy=0x7f01c00b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bf400640 1 --2- 172.21.15.161:0/4214978811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01a4003980 0x7f01a4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fb3f4e00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb3f0143c40 0x7fb3f01444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:60742/0 (socket says 172.21.15.161:60742) 2024-05-22T21:13:34.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fb3f4e00640 1 -- 172.21.15.161:0/1147843161 learned_addr learned my addr 172.21.15.161:0/1147843161 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:34.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bf400640 1 --2- 172.21.15.161:0/4214978811 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f01c006c7e0 0x7f01a40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bf400640 1 --2- 172.21.15.161:0/4214978811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01a4004d10 0x7f01a4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bf400640 1 -- 172.21.15.161:0/4214978811 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c006cc70 con 0x7f01a4003980 2024-05-22T21:13:34.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bf400640 1 -- 172.21.15.161:0/4214978811 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c00b29f0 con 0x7f01a4004d10 2024-05-22T21:13:34.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bf400640 1 -- 172.21.15.161:0/4214978811 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a400ba90 con 0x7f01c006c7e0 2024-05-22T21:13:34.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fb3efe00640 1 --2- 172.21.15.161:0/1147843161 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3f015b600 0x7fb3f006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fb3f5800640 1 --2- 172.21.15.161:0/1147843161 >> v2:172.21.15.161:3300/0 conn(0x7fb3f006c7e0 0x7fb3f00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01c5800640 1 --2- 172.21.15.161:0/4214978811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01a4004d10 0x7f01a4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01c4e00640 1 --2- 172.21.15.161:0/4214978811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01a4003980 0x7f01a4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01bfe00640 1 --2- 172.21.15.161:0/4214978811 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f01c006c7e0 0x7f01a40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fb3ef400640 1 -- 172.21.15.161:0/1147843161 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398031393 0 0) 0x7fb3f0144bb0 con 0x7fb3f0143c40 2024-05-22T21:13:34.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f83faa00640 1 Processor -- start 2024-05-22T21:13:34.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01c5800640 1 -- 172.21.15.161:0/4214978811 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f01c006c7e0 msgr2=0x7f01a40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:34.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01c5800640 1 --2- 172.21.15.161:0/4214978811 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f01c006c7e0 0x7f01a40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01c5800640 1 -- 172.21.15.161:0/4214978811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01a4003980 msgr2=0x7f01a4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:34.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01c5800640 1 --2- 172.21.15.161:0/4214978811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01a4003980 0x7f01a4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01c5800640 1 -- 172.21.15.161:0/4214978811 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c00b2cd0 con 0x7f01a4004d10 2024-05-22T21:13:34.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fb3ef400640 1 -- 172.21.15.161:0/1147843161 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3d8003530 con 0x7fb3f0143c40 2024-05-22T21:13:34.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f01c4e00640 1 --2- 172.21.15.161:0/4214978811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01a4003980 0x7f01a4003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:34.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fb3f5800640 1 -- 172.21.15.161:0/1147843161 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3f015b600 msgr2=0x7fb3f006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:34.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fb3f5800640 1 --2- 172.21.15.161:0/1147843161 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3f015b600 0x7fb3f006c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fb3f5800640 1 -- 172.21.15.161:0/1147843161 >> v1:172.21.15.179:6789/0 conn(0x7fb3f0143c40 legacy=0x7fb3f01444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fb3f5800640 1 -- 172.21.15.161:0/1147843161 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f015efb0 con 0x7fb3f006c7e0 2024-05-22T21:13:34.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f173f400640 1 Processor -- start 2024-05-22T21:13:34.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fb3efe00640 1 --2- 172.21.15.161:0/1147843161 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3f015b600 0x7fb3f006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:34.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f83faa00640 1 -- start start 2024-05-22T21:13:34.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fc8c1a00640 1 -- 172.21.15.161:0/2465415293 shutdown_connections 2024-05-22T21:13:34.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fc8c1a00640 1 --2- 172.21.15.161:0/2465415293 >> v2:172.21.15.179:3300/0 conn(0x7fc8b404ffb0 0x7fc8b40e2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fc8c1a00640 1 --2- 172.21.15.161:0/2465415293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8b4033cd0 0x7fc8b404fa70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fc8c1a00640 1 --2- 172.21.15.161:0/2465415293 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8b400aab0 0x7fc8b40df280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fc8c1a00640 1 -- 172.21.15.161:0/2465415293 >> 172.21.15.161:0/2465415293 conn(0x7fc8b4009f90 msgr2=0x7fc8b400a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:34.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f173f400640 1 -- start start 2024-05-22T21:13:34.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7f64d7a00640 1 Processor -- start 2024-05-22T21:13:34.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fc8c1a00640 1 -- 172.21.15.161:0/2465415293 shutdown_connections 2024-05-22T21:13:34.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f64d7a00640 1 -- start start 2024-05-22T21:13:34.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f83faa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83f414b1c0 0x7f83f40b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f83faa00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f83f4070d80 0x7f83f406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f83faa00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f40b2710 con 0x7f83f406c7e0 2024-05-22T21:13:34.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f83faa00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f406cc70 con 0x7f83f4070d80 2024-05-22T21:13:34.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f83faa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40b29f0 con 0x7f83f414b1c0 2024-05-22T21:13:34.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f83f3e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83f414b1c0 0x7f83f40b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.078+0000 7fc8c1a00640 1 -- 172.21.15.161:0/2465415293 wait complete. 2024-05-22T21:13:34.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7fc8c1a00640 1 Processor -- start 2024-05-22T21:13:34.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f173f400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f17380676a0 0x7f1738159380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f173f400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f17380b2760 0x7f17380afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f173f400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17380ac030 con 0x7f17380aeb30 2024-05-22T21:13:34.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f173f400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17380b03c0 con 0x7f17380b2760 2024-05-22T21:13:34.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f173f400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17380b06a0 con 0x7f17380676a0 2024-05-22T21:13:34.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f64d7a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f64d00ad4d0 0x7f64d00afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f64d7a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d00ac030 con 0x7f64d00b2050 2024-05-22T21:13:34.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f64d7a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d00b00e0 con 0x7f64d00aeb30 2024-05-22T21:13:34.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f64d7a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d00b03c0 con 0x7f64d00ad4d0 2024-05-22T21:13:34.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7fc8c1a00640 1 -- start start 2024-05-22T21:13:34.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f1737e00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f17380aeb30 0x7f17380b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:60756/0 (socket says 172.21.15.161:60756) 2024-05-22T21:13:34.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f1737e00640 1 -- 172.21.15.161:0/3506175013 learned_addr learned my addr 172.21.15.161:0/3506175013 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:34.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f173ca00640 1 --2- 172.21.15.161:0/3506175013 >> v2:172.21.15.179:3300/0 conn(0x7f17380676a0 0x7f1738159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f173d400640 1 --2- 172.21.15.161:0/3506175013 >> v2:172.21.15.161:3300/0 conn(0x7f17380b2760 0x7f17380afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f83f8a00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f83f406c7e0 0x7f83f406fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:60746/0 (socket says 172.21.15.161:60746) 2024-05-22T21:13:34.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f83f8a00640 1 -- 172.21.15.161:0/2059490378 learned_addr learned my addr 172.21.15.161:0/2059490378 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:34.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f83f3400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f83f4070d80 0x7f83f406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f83f3e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83f414b1c0 0x7f83f40b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:53440/0 (socket says 172.21.15.161:53440) 2024-05-22T21:13:34.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f64d5a00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f64d00aeb30 0x7f64d0162ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:60758/0 (socket says 172.21.15.161:60758) 2024-05-22T21:13:34.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f64d5a00640 1 -- 172.21.15.161:0/2083748262 learned_addr learned my addr 172.21.15.161:0/2083748262 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:34.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f83f2a00640 1 -- 172.21.15.161:0/2059490378 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1544499479 0 0) 0x7f83f40b2710 con 0x7f83f406c7e0 2024-05-22T21:13:34.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f173ca00640 1 -- 172.21.15.161:0/3506175013 >> v2:172.21.15.161:3300/0 conn(0x7f17380b2760 msgr2=0x7f17380afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:34.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f173ca00640 1 --2- 172.21.15.161:0/3506175013 >> v2:172.21.15.161:3300/0 conn(0x7f17380b2760 0x7f17380afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f64cf400640 1 -- 172.21.15.161:0/2083748262 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745498833 0 0) 0x7f64d00b00e0 con 0x7f64d00aeb30 2024-05-22T21:13:34.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f64cf400640 1 -- 172.21.15.161:0/2083748262 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64b8003530 con 0x7f64d00aeb30 2024-05-22T21:13:34.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f173ca00640 1 -- 172.21.15.161:0/3506175013 >> v1:172.21.15.179:6789/0 conn(0x7f17380aeb30 legacy=0x7f17380b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:34.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f83f3400640 1 -- 172.21.15.161:0/2059490378 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83f414b1c0 msgr2=0x7f83f40b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:34.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f83f3400640 1 --2- 172.21.15.161:0/2059490378 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83f414b1c0 0x7f83f40b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f83f3400640 1 -- 172.21.15.161:0/2059490378 >> v1:172.21.15.179:6789/0 conn(0x7f83f406c7e0 legacy=0x7f83f406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f83f3400640 1 -- 172.21.15.161:0/2059490378 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f40b2cd0 con 0x7f83f4070d80 2024-05-22T21:13:34.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.082+0000 7f64cfe00640 1 --2- 172.21.15.161:0/2083748262 >> v2:172.21.15.161:3300/0 conn(0x7f64d00ad4d0 0x7f64d00afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7f01c5800640 1 --2- 172.21.15.161:0/4214978811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01a4004d10 0x7f01a4005190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f01b0009ae0 tx=0x7f01b001ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51e4b847366af41f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7f64cfe00640 1 -- 172.21.15.161:0/2083748262 >> v1:172.21.15.179:6789/0 conn(0x7f64d00aeb30 legacy=0x7f64d0162ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7f64cfe00640 1 -- 172.21.15.161:0/2083748262 >> v1:172.21.15.161:6789/0 conn(0x7f64d00b2050 legacy=0x7f64d00b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:34.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7f64cfe00640 1 -- 172.21.15.161:0/2083748262 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d00b06a0 con 0x7f64d00ad4d0 2024-05-22T21:13:34.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7f173ca00640 1 -- 172.21.15.161:0/3506175013 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17380b0980 con 0x7f17380676a0 2024-05-22T21:13:34.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7f01bf400640 1 -- 172.21.15.161:0/4214978811 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b00a55b0 con 0x7f01a4004d10 2024-05-22T21:13:34.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7f01bf400640 1 -- 172.21.15.161:0/4214978811 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b00a5710 con 0x7f01a4004d10 2024-05-22T21:13:34.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fd3b8000640 1 Processor -- start 2024-05-22T21:13:34.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7f01bf400640 1 -- 172.21.15.161:0/4214978811 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b00a55b0 con 0x7f01a4004d10 2024-05-22T21:13:34.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7f01c7800640 1 -- 172.21.15.161:0/4214978811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01a4004d10 msgr2=0x7f01a4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7f01c7800640 1 --2- 172.21.15.161:0/4214978811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01a4004d10 0x7f01a4005190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f01b0009ae0 tx=0x7f01b001ca40 comp rx=0 tx=0).stop 2024-05-22T21:13:34.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7f01c5800640 1 -- 172.21.15.161:0/4214978811 reap_dead start 2024-05-22T21:13:34.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8c1a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8b400aab0 0x7fc8b4039da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8c1a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8b4033cd0 0x7fc8b40d6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8c1a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8b404ffb0 0x7fc8b40cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8c1a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40cc6d0 con 0x7fc8b400aab0 2024-05-22T21:13:34.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8c1a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40d7050 con 0x7fc8b4033cd0 2024-05-22T21:13:34.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8c1a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40d7240 con 0x7fc8b404ffb0 2024-05-22T21:13:34.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8bbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8b400aab0 0x7fc8b4039da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8bbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8b400aab0 0x7fc8b4039da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49220/0 (socket says 172.21.15.161:49220) 2024-05-22T21:13:34.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8bbe00640 1 -- 172.21.15.161:0/3317886474 learned_addr learned my addr 172.21.15.161:0/3317886474 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:34.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fb3f5800640 1 --2- 172.21.15.161:0/1147843161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3f006c7e0 0x7fb3f00b2160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009d90 tx=0x7fb3e801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0f867be44f0350 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fe54ee00640 1 Processor -- start 2024-05-22T21:13:34.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fe54ee00640 1 -- start start 2024-05-22T21:13:34.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7f01c7800640 1 -- 172.21.15.161:0/4214978811 shutdown_connections 2024-05-22T21:13:34.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7f01c7800640 1 -- 172.21.15.161:0/4214978811 >> 172.21.15.161:0/4214978811 conn(0x7f01c006a2d0 msgr2=0x7f01c006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:34.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8c1000640 1 --2- 172.21.15.161:0/3317886474 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8b404ffb0 0x7fc8b40cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fd3b8000640 1 -- start start 2024-05-22T21:13:34.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7f01c7800640 1 -- 172.21.15.161:0/4214978811 shutdown_connections 2024-05-22T21:13:34.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8c1000640 1 -- 172.21.15.161:0/3317886474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8b4033cd0 msgr2=0x7fc8b40d6b10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-22T21:13:34.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8c1000640 1 --2- 172.21.15.161:0/3317886474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8b4033cd0 0x7fc8b40d6b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8c1000640 1 -- 172.21.15.161:0/3317886474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8b400aab0 msgr2=0x7fc8b4039da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8c1000640 1 --2- 172.21.15.161:0/3317886474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8b400aab0 0x7fc8b4039da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8c1000640 1 -- 172.21.15.161:0/3317886474 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b0009420 con 0x7fc8b404ffb0 2024-05-22T21:13:34.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8c1000640 1 --2- 172.21.15.161:0/3317886474 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8b404ffb0 0x7fc8b40cbb70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000e7d0 tx=0x7fc8b000ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fb3ef400640 1 -- 172.21.15.161:0/1147843161 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80a5a30 con 0x7fb3f006c7e0 2024-05-22T21:13:34.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fb3ef400640 1 -- 172.21.15.161:0/1147843161 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e80a5b90 con 0x7fb3f006c7e0 2024-05-22T21:13:34.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fb3ef400640 1 -- 172.21.15.161:0/1147843161 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80ad440 con 0x7fb3f006c7e0 2024-05-22T21:13:34.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7f01c7800640 1 -- 172.21.15.161:0/4214978811 wait complete. 2024-05-22T21:13:34.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8afe00640 1 -- 172.21.15.161:0/3317886474 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000eee0 con 0x7fc8b404ffb0 2024-05-22T21:13:34.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8afe00640 1 -- 172.21.15.161:0/3317886474 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b0004020 con 0x7fc8b404ffb0 2024-05-22T21:13:34.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8afe00640 1 -- 172.21.15.161:0/3317886474 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b00043b0 con 0x7fc8b404ffb0 2024-05-22T21:13:34.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8c1a00640 1 -- 172.21.15.161:0/3317886474 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a80094d0 con 0x7fc8b404ffb0 2024-05-22T21:13:34.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fc8c1a00640 1 -- 172.21.15.161:0/3317886474 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b40d7800 con 0x7fc8b404ffb0 2024-05-22T21:13:34.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.086+0000 7fd3b8000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd3b006fad0 0x7fd3b006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f173ca00640 1 --2- 172.21.15.161:0/3506175013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17380676a0 0x7f1738159380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1738066e70 tx=0x7f172801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f872b370834c37 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fd3b8000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b006f030 con 0x7fd3b015e5f0 2024-05-22T21:13:34.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fd3b8000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b015efc0 con 0x7fd3b00cf000 2024-05-22T21:13:34.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fd3b8000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b006f840 con 0x7fd3b006fad0 2024-05-22T21:13:34.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fd3b5600640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd3b015e5f0 0x7fd3b015f450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56230/0 (socket says 172.21.15.161:56230) 2024-05-22T21:13:34.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fd3b5600640 1 -- 172.21.15.161:0/2609445117 learned_addr learned my addr 172.21.15.161:0/2609445117 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:34.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe54ee00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe54806c330 0x7fe5480cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fb3f7800640 1 -- 172.21.15.161:0/1147843161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3f006c7e0 msgr2=0x7fb3f00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fd3b4c00640 1 --2- 172.21.15.161:0/2609445117 >> v2:172.21.15.179:3300/0 conn(0x7fd3b006fad0 0x7fd3b006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fb3f7800640 1 --2- 172.21.15.161:0/1147843161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3f006c7e0 0x7fb3f00b2160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009d90 tx=0x7fb3e801c970 comp rx=0 tx=0).stop 2024-05-22T21:13:34.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f83f3400640 1 --2- 172.21.15.161:0/2059490378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83f4070d80 0x7f83f406c2a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f83e00097c0 tx=0x7f83e001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73e1ad457ae865b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe547e00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe5481591f0 0x7fe5480b28d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:60768/0 (socket says 172.21.15.161:60768) 2024-05-22T21:13:34.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe547e00640 1 -- 172.21.15.161:0/4093986142 learned_addr learned my addr 172.21.15.161:0/4093986142 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:34.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe547400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe54806bea0 0x7fe548070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56240/0 (socket says 172.21.15.161:56240) 2024-05-22T21:13:34.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe54ce00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe54806c330 0x7fe5480cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fc8c1a00640 1 -- 172.21.15.161:0/3317886474 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b40531c0 con 0x7fc8b404ffb0 2024-05-22T21:13:34.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe54ee00640 1 -- 172.21.15.161:0/4093986142 --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5480cca10 con 0x7fe54806bea0 2024-05-22T21:13:34.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f1737400640 1 -- 172.21.15.161:0/3506175013 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17280ac410 con 0x7f17380676a0 2024-05-22T21:13:34.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f1737400640 1 -- 172.21.15.161:0/3506175013 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17280ac570 con 0x7f17380676a0 2024-05-22T21:13:34.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f1737400640 1 -- 172.21.15.161:0/3506175013 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17280ac900 con 0x7f17380676a0 2024-05-22T21:13:34.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f64cfe00640 1 --2- 172.21.15.161:0/2083748262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d00ad4d0 0x7f64d00afa20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009740 tx=0x7f64bc01c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e1f2a8147e7ec5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fd3afe00640 1 -- 172.21.15.161:0/2609445117 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627671937 0 0) 0x7fd3b006f030 con 0x7fd3b015e5f0 2024-05-22T21:13:34.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe54ee00640 1 -- 172.21.15.161:0/4093986142 --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5480b43d0 con 0x7fe5481591f0 2024-05-22T21:13:34.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe54ee00640 1 -- 172.21.15.161:0/4093986142 --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5480b3c00 con 0x7fe54806c330 2024-05-22T21:13:34.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fd3afe00640 1 -- 172.21.15.161:0/2609445117 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd398003530 con 0x7fd3b015e5f0 2024-05-22T21:13:34.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fd3b4c00640 1 -- 172.21.15.161:0/2609445117 >> v1:172.21.15.179:6789/0 conn(0x7fd3b00cf000 legacy=0x7fd3b00cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fd3b4c00640 1 -- 172.21.15.161:0/2609445117 >> v1:172.21.15.161:6789/0 conn(0x7fd3b015e5f0 legacy=0x7fd3b015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fd3b4c00640 1 -- 172.21.15.161:0/2609445117 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b00cfcc0 con 0x7fd3b006fad0 2024-05-22T21:13:34.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f83f2a00640 1 -- 172.21.15.161:0/2059490378 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e00a7070 con 0x7f83f4070d80 2024-05-22T21:13:34.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f83f2a00640 1 -- 172.21.15.161:0/2059490378 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e00ab430 con 0x7f83f4070d80 2024-05-22T21:13:34.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f83f2a00640 1 -- 172.21.15.161:0/2059490378 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e00ab7c0 con 0x7f83f4070d80 2024-05-22T21:13:34.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 -- 172.21.15.161:0/4093986142 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848845078 0 0) 0x7fe5480b43d0 con 0x7fe5481591f0 2024-05-22T21:13:34.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 -- 172.21.15.161:0/4093986142 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe52c003530 con 0x7fe5481591f0 2024-05-22T21:13:34.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fb3f7800640 1 -- 172.21.15.161:0/1147843161 shutdown_connections 2024-05-22T21:13:34.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fb3f7800640 1 --2- 172.21.15.161:0/1147843161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3f006c7e0 0x7fb3f00b2160 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fb3f7800640 1 --2- 172.21.15.161:0/1147843161 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3f015b600 0x7fb3f006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fb3f7800640 1 -- 172.21.15.161:0/1147843161 >> 172.21.15.161:0/1147843161 conn(0x7fb3f006a2d0 msgr2=0x7fb3f006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:34.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fb3f7800640 1 -- 172.21.15.161:0/1147843161 shutdown_connections 2024-05-22T21:13:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 -- 172.21.15.161:0/4093986142 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2234794096 0 0) 0x7fe5480cca10 con 0x7fe54806bea0 2024-05-22T21:13:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f173f400640 1 -- 172.21.15.161:0/3506175013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17380676a0 msgr2=0x7f1738159380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f173f400640 1 --2- 172.21.15.161:0/3506175013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17380676a0 0x7f1738159380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1738066e70 tx=0x7f172801c480 comp rx=0 tx=0).stop 2024-05-22T21:13:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 -- 172.21.15.161:0/4093986142 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5480b43d0 con 0x7fe54806bea0 2024-05-22T21:13:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f83faa00640 1 -- 172.21.15.161:0/2059490378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83f4070d80 msgr2=0x7f83f406c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f83faa00640 1 --2- 172.21.15.161:0/2059490378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83f4070d80 0x7f83f406c2a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f83e00097c0 tx=0x7f83e001c470 comp rx=0 tx=0).stop 2024-05-22T21:13:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f83faa00640 1 -- 172.21.15.161:0/2059490378 shutdown_connections 2024-05-22T21:13:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f83faa00640 1 --2- 172.21.15.161:0/2059490378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83f4070d80 0x7f83f406c2a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f83faa00640 1 --2- 172.21.15.161:0/2059490378 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83f414b1c0 0x7f83f40b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f83faa00640 1 -- 172.21.15.161:0/2059490378 >> 172.21.15.161:0/2059490378 conn(0x7f83f406a2d0 msgr2=0x7f83f406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fb3f7800640 1 -- 172.21.15.161:0/1147843161 wait complete. 2024-05-22T21:13:34.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 -- 172.21.15.161:0/4093986142 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1098413271 0 0) 0x7fe5480b43d0 con 0x7fe54806bea0 2024-05-22T21:13:34.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 -- 172.21.15.161:0/4093986142 >> v2:172.21.15.161:3300/0 conn(0x7fe54806c330 msgr2=0x7fe5480cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:34.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 --2- 172.21.15.161:0/4093986142 >> v2:172.21.15.161:3300/0 conn(0x7fe54806c330 0x7fe5480cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 -- 172.21.15.161:0/4093986142 >> v1:172.21.15.179:6789/0 conn(0x7fe5481591f0 legacy=0x7fe5480b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 -- 172.21.15.161:0/4093986142 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5480ccc40 con 0x7fe54806bea0 2024-05-22T21:13:34.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f83faa00640 1 -- 172.21.15.161:0/2059490378 shutdown_connections 2024-05-22T21:13:34.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7f83faa00640 1 -- 172.21.15.161:0/2059490378 wait complete. 2024-05-22T21:13:34.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 -- 172.21.15.161:0/4093986142 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fe538001840 con 0x7fe54806bea0 2024-05-22T21:13:34.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 -- 172.21.15.161:0/4093986142 >> v1:172.21.15.161:6789/0 conn(0x7fe54806bea0 legacy=0x7fe548070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 --2- 172.21.15.161:0/4093986142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe52c003b10 0x7fe52c003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 --2- 172.21.15.161:0/4093986142 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe52c0047d0 0x7fe52c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 --2- 172.21.15.161:0/4093986142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c005190 0x7fe52c00fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 -- 172.21.15.161:0/4093986142 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5480b3c00 con 0x7fe52c005190 2024-05-22T21:13:34.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 -- 172.21.15.161:0/4093986142 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c0103c0 con 0x7fe52c003b10 2024-05-22T21:13:34.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.090+0000 7fe546a00640 1 -- 172.21.15.161:0/4093986142 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c010580 con 0x7fe52c0047d0 2024-05-22T21:13:34.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f01c7800640 1 Processor -- start 2024-05-22T21:13:34.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fe547400640 1 --2- 172.21.15.161:0/4093986142 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe52c0047d0 0x7fe52c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f173f400640 1 -- 172.21.15.161:0/3506175013 shutdown_connections 2024-05-22T21:13:34.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f173f400640 1 --2- 172.21.15.161:0/3506175013 >> v2:172.21.15.161:3300/0 conn(0x7f17380b2760 0x7f17380afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f173f400640 1 --2- 172.21.15.161:0/3506175013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17380676a0 0x7f1738159380 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f173f400640 1 -- 172.21.15.161:0/3506175013 >> 172.21.15.161:0/3506175013 conn(0x7f17380714a0 msgr2=0x7f17380718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:34.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f01c7800640 1 -- start start 2024-05-22T21:13:34.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f173f400640 1 -- 172.21.15.161:0/3506175013 shutdown_connections 2024-05-22T21:13:34.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fe547e00640 1 --2- 172.21.15.161:0/4093986142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe52c003b10 0x7fe52c003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f173f400640 1 -- 172.21.15.161:0/3506175013 wait complete. 2024-05-22T21:13:34.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f64cf400640 1 -- 172.21.15.161:0/2083748262 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8001a00 con 0x7f64d00ad4d0 2024-05-22T21:13:34.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f64cf400640 1 -- 172.21.15.161:0/2083748262 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64bc0a7050 con 0x7f64d00ad4d0 2024-05-22T21:13:34.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f64cf400640 1 -- 172.21.15.161:0/2083748262 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc0ab620 con 0x7f64d00ad4d0 2024-05-22T21:13:34.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f7800640 1 Processor -- start 2024-05-22T21:13:34.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f7800640 1 -- start start 2024-05-22T21:13:34.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fe54ce00640 1 --2- 172.21.15.161:0/4093986142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c005190 0x7fe52c00fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fc8afe00640 1 -- 172.21.15.161:0/3317886474 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc8b000c940 con 0x7fc8b404ffb0 2024-05-22T21:13:34.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f83faa00640 1 Processor -- start 2024-05-22T21:13:34.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f83faa00640 1 -- start start 2024-05-22T21:13:34.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fd3b4c00640 1 --2- 172.21.15.161:0/2609445117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3b006fad0 0x7fd3b006e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40097c0 tx=0x7fd3a401c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fc58ec8472c41b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fc8afe00640 1 --2- 172.21.15.161:0/3317886474 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc89c066760 0x7fc89c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f7800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3f006c7e0 0x7fb3f0159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f7800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3f0143c40 0x7fb3f01371c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fd3afe00640 1 -- 172.21.15.161:0/2609445117 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4004230 con 0x7fd3b006fad0 2024-05-22T21:13:34.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fd3afe00640 1 -- 172.21.15.161:0/2609445117 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a40a7050 con 0x7fd3b006fad0 2024-05-22T21:13:34.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fd3afe00640 1 -- 172.21.15.161:0/2609445117 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4003620 con 0x7fd3b006fad0 2024-05-22T21:13:34.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f7800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3f015b600 0x7fb3f0138fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f7800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f015a3d0 con 0x7fb3f0143c40 2024-05-22T21:13:34.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f7800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f0137ba0 con 0x7fb3f006c7e0 2024-05-22T21:13:34.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f7800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f015bf90 con 0x7fb3f015b600 2024-05-22T21:13:34.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fe547400640 1 -- 172.21.15.161:0/4093986142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe52c003b10 msgr2=0x7fe52c003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:34.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fe547400640 1 --2- 172.21.15.161:0/4093986142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe52c003b10 0x7fe52c003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fe547400640 1 -- 172.21.15.161:0/4093986142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c005190 msgr2=0x7fe52c00fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:34.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fe547400640 1 --2- 172.21.15.161:0/4093986142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c005190 0x7fe52c00fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fe547400640 1 -- 172.21.15.161:0/4093986142 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5480ccc40 con 0x7fe52c0047d0 2024-05-22T21:13:34.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fc8afe00640 1 -- 172.21.15.161:0/3317886474 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8b00842a0 con 0x7fc8b404ffb0 2024-05-22T21:13:34.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fe54ce00640 1 -- 172.21.15.161:0/4093986142 reap_dead start 2024-05-22T21:13:34.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fd3b8000640 1 -- 172.21.15.161:0/2609445117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3b006fad0 msgr2=0x7fd3b006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f01c7800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01a4004d10 0x7f01c01f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f01c7800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01c00abd10 0x7f01c00ac190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f01c7800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f01c00ada60 0x7f01c00adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f01c7800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c01fe110 con 0x7f01c00abd10 2024-05-22T21:13:34.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f01c7800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c01fd590 con 0x7f01a4004d10 2024-05-22T21:13:34.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f01c7800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c01fd870 con 0x7f01c00ada60 2024-05-22T21:13:34.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f01c4e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01a4004d10 0x7f01c01f93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f01c4e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01a4004d10 0x7f01c01f93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:39618/0 (socket says 172.21.15.161:39618) 2024-05-22T21:13:34.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f01c4e00640 1 -- 172.21.15.161:0/2714584894 learned_addr learned my addr 172.21.15.161:0/2714584894 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:34.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f01c5800640 1 --2- 172.21.15.161:0/2714584894 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f01c00ada60 0x7f01c00adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f01c5800640 1 -- 172.21.15.161:0/2714584894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01a4004d10 msgr2=0x7f01c01f93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f5800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3f015b600 0x7fb3f0138fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f5800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3f015b600 0x7fb3f0138fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:53482/0 (socket says 172.21.15.161:53482) 2024-05-22T21:13:34.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f5800640 1 -- 172.21.15.161:0/2068324114 learned_addr learned my addr 172.21.15.161:0/2068324114 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:34.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f4e00640 1 --2- 172.21.15.161:0/2068324114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3f006c7e0 0x7fb3f0159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fc8bbe00640 1 --2- 172.21.15.161:0/3317886474 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc89c066760 0x7fc89c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f5800640 1 -- 172.21.15.161:0/2068324114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3f006c7e0 msgr2=0x7fb3f0159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f5800640 1 --2- 172.21.15.161:0/2068324114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3f006c7e0 0x7fb3f0159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f5800640 1 -- 172.21.15.161:0/2068324114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3f0143c40 msgr2=0x7fb3f01371c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-22T21:13:34.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f5800640 1 --2- 172.21.15.161:0/2068324114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3f0143c40 0x7fb3f01371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fb3f5800640 1 -- 172.21.15.161:0/2068324114 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3dc009420 con 0x7fb3f015b600 2024-05-22T21:13:34.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f173f400640 1 Processor -- start 2024-05-22T21:13:34.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f83faa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83f406c7e0 0x7f83f4141fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f83faa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83f4070d80 0x7f83f4149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f83faa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f414b1c0 0x7f83f4149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f83faa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f41424e0 con 0x7f83f414b1c0 2024-05-22T21:13:34.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f83faa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f4147b80 con 0x7f83f406c7e0 2024-05-22T21:13:34.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f01bfe00640 1 --2- 172.21.15.161:0/2714584894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01c00abd10 0x7f01c00ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f83faa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f4147e00 con 0x7f83f4070d80 2024-05-22T21:13:34.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f83f3e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83f406c7e0 0x7f83f4141fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7f83f3400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83f4070d80 0x7f83f4149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f83f3400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83f4070d80 0x7f83f4149960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:53486/0 (socket says 172.21.15.161:53486) 2024-05-22T21:13:34.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f83f3400640 1 -- 172.21.15.161:0/150769577 learned_addr learned my addr 172.21.15.161:0/150769577 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:34.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f64d7a00640 1 -- 172.21.15.161:0/2083748262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d00ad4d0 msgr2=0x7f64d00afa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f64d7a00640 1 --2- 172.21.15.161:0/2083748262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d00ad4d0 0x7f64d00afa20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009740 tx=0x7f64bc01c250 comp rx=0 tx=0).stop 2024-05-22T21:13:34.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173f400640 1 -- start start 2024-05-22T21:13:34.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.094+0000 7fd3b8000640 1 --2- 172.21.15.161:0/2609445117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3b006fad0 0x7fd3b006e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40097c0 tx=0x7fd3a401c350 comp rx=0 tx=0).stop 2024-05-22T21:13:34.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f01c5800640 1 --2- 172.21.15.161:0/2714584894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01a4004d10 0x7f01c01f93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f01c5800640 1 -- 172.21.15.161:0/2714584894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01c00abd10 msgr2=0x7f01c00ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f01c5800640 1 --2- 172.21.15.161:0/2714584894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01c00abd10 0x7f01c00ac190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f01c5800640 1 -- 172.21.15.161:0/2714584894 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c01fdb50 con 0x7f01c00ada60 2024-05-22T21:13:34.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f64d7a00640 1 -- 172.21.15.161:0/2083748262 shutdown_connections 2024-05-22T21:13:34.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f64d7a00640 1 --2- 172.21.15.161:0/2083748262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d00ad4d0 0x7f64d00afa20 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009740 tx=0x7f64bc01c250 comp rx=0 tx=0).stop 2024-05-22T21:13:34.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f64d7a00640 1 -- 172.21.15.161:0/2083748262 >> 172.21.15.161:0/2083748262 conn(0x7f64d00714a0 msgr2=0x7f64d00718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:34.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7fd3b8000640 1 -- 172.21.15.161:0/2609445117 shutdown_connections 2024-05-22T21:13:34.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7fd3b8000640 1 --2- 172.21.15.161:0/2609445117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3b006fad0 0x7fd3b006e9a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7fd3b8000640 1 -- 172.21.15.161:0/2609445117 >> 172.21.15.161:0/2609445117 conn(0x7fd3b00b2a80 msgr2=0x7fd3b00b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:34.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f83f3400640 1 -- 172.21.15.161:0/150769577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83f406c7e0 msgr2=0x7f83f4141fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f83f3400640 1 --2- 172.21.15.161:0/150769577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83f406c7e0 0x7f83f4141fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f83f3400640 1 -- 172.21.15.161:0/150769577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f414b1c0 msgr2=0x7f83f4149ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f83f3400640 1 --2- 172.21.15.161:0/150769577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f414b1c0 0x7f83f4149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f83f3400640 1 -- 172.21.15.161:0/150769577 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e0009420 con 0x7f83f4070d80 2024-05-22T21:13:34.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f64d7a00640 1 -- 172.21.15.161:0/2083748262 shutdown_connections 2024-05-22T21:13:34.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f64d7a00640 1 -- 172.21.15.161:0/2083748262 wait complete. 2024-05-22T21:13:34.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7fb3f5800640 1 --2- 172.21.15.161:0/2068324114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3f015b600 0x7fb3f0138fc0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3e801a5a0 tx=0x7fb3e8019710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f01c5800640 1 --2- 172.21.15.161:0/2714584894 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f01c00ada60 0x7f01c00adee0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f01b00ac000 tx=0x7f01a4005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7fd3b8000640 1 -- 172.21.15.161:0/2609445117 shutdown_connections 2024-05-22T21:13:34.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7fd3b8000640 1 -- 172.21.15.161:0/2609445117 wait complete. 2024-05-22T21:13:34.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7fc8bbe00640 1 --2- 172.21.15.161:0/3317886474 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc89c066760 0x7fc89c068c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc0686a0 tx=0x7fc8bc08d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173f400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17380676a0 0x7f173816c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17380aeb30 0x7f1738148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173f400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17380b2760 0x7f173806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173f400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173816a8b0 con 0x7f17380aeb30 2024-05-22T21:13:34.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173f400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1738148f70 con 0x7f17380676a0 2024-05-22T21:13:34.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173f400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1738149250 con 0x7f17380b2760 2024-05-22T21:13:34.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173ca00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17380676a0 0x7f173816c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f1737e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17380aeb30 0x7f1738148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173ca00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17380676a0 0x7f173816c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:39640/0 (socket says 172.21.15.161:39640) 2024-05-22T21:13:34.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173ca00640 1 -- 172.21.15.161:0/2749540699 learned_addr learned my addr 172.21.15.161:0/2749540699 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:34.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173d400640 1 --2- 172.21.15.161:0/2749540699 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17380b2760 0x7f173806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173ca00640 1 -- 172.21.15.161:0/2749540699 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17380b2760 msgr2=0x7f173806cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173ca00640 1 --2- 172.21.15.161:0/2749540699 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17380b2760 0x7f173806cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173ca00640 1 -- 172.21.15.161:0/2749540699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17380aeb30 msgr2=0x7f1738148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173ca00640 1 --2- 172.21.15.161:0/2749540699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17380aeb30 0x7f1738148590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173ca00640 1 -- 172.21.15.161:0/2749540699 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1728009420 con 0x7f17380676a0 2024-05-22T21:13:34.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f64d7a00640 1 Processor -- start 2024-05-22T21:13:34.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f173ca00640 1 --2- 172.21.15.161:0/2749540699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17380676a0 0x7f173816c810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f17280af040 tx=0x7f1728019220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f64d7a00640 1 -- start start 2024-05-22T21:13:34.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7fd3b8000640 1 Processor -- start 2024-05-22T21:13:34.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7fd3b8000640 1 -- start start 2024-05-22T21:13:34.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.098+0000 7f1734c00640 1 -- 172.21.15.161:0/2749540699 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17280ace30 con 0x7f17380676a0 2024-05-22T21:13:34.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f1734c00640 1 -- 172.21.15.161:0/2749540699 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17280b5620 con 0x7f17380676a0 2024-05-22T21:13:34.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f1734c00640 1 -- 172.21.15.161:0/2749540699 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17280b59d0 con 0x7f17380676a0 2024-05-22T21:13:34.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f173f400640 1 -- 172.21.15.161:0/2749540699 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1738149530 con 0x7f17380676a0 2024-05-22T21:13:34.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f173f400640 1 -- 172.21.15.161:0/2749540699 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173816cd50 con 0x7f17380676a0 2024-05-22T21:13:34.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64d7a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64d00aeb30 0x7f64d014cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64d7a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d00b2050 0x7f64d0148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64d7a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64d014ac90 0x7f64d01461b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64d7a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d01480c0 con 0x7f64d00b2050 2024-05-22T21:13:34.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64d7a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d0151ae0 con 0x7f64d014ac90 2024-05-22T21:13:34.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64d7a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d014b120 con 0x7f64d00aeb30 2024-05-22T21:13:34.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64cfe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d00b2050 0x7f64d0148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64cfe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d00b2050 0x7f64d0148e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49280/0 (socket says 172.21.15.161:49280) 2024-05-22T21:13:34.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64cfe00640 1 -- 172.21.15.161:0/3420110941 learned_addr learned my addr 172.21.15.161:0/3420110941 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:34.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64d5000640 1 --2- 172.21.15.161:0/3420110941 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64d00aeb30 0x7f64d014cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b8000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3b006fad0 0x7fd3b00c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b8000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3b00cf000 0x7fd3b00ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b8000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3b015e5f0 0x7fd3b0152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f173f400640 1 -- 172.21.15.161:0/2749540699 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1738067220 con 0x7f17380676a0 2024-05-22T21:13:34.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b8000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b00c9bd0 con 0x7fd3b006fad0 2024-05-22T21:13:34.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b8000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b00ca190 con 0x7fd3b00cf000 2024-05-22T21:13:34.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b8000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b00c9370 con 0x7fd3b015e5f0 2024-05-22T21:13:34.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64d5a00640 1 --2- 172.21.15.161:0/3420110941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64d014ac90 0x7f64d01461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b5600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3b006fad0 0x7fd3b00c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b5600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3b006fad0 0x7fd3b00c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49284/0 (socket says 172.21.15.161:49284) 2024-05-22T21:13:34.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b5600640 1 -- 172.21.15.161:0/197147425 learned_addr learned my addr 172.21.15.161:0/197147425 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:34.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64d5000640 1 -- 172.21.15.161:0/3420110941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64d014ac90 msgr2=0x7f64d01461b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b4c00640 1 --2- 172.21.15.161:0/197147425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3b00cf000 0x7fd3b00ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64d5000640 1 --2- 172.21.15.161:0/3420110941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64d014ac90 0x7f64d01461b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64d5000640 1 -- 172.21.15.161:0/3420110941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d00b2050 msgr2=0x7f64d0148e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64d5000640 1 --2- 172.21.15.161:0/3420110941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d00b2050 0x7f64d0148e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64d5000640 1 -- 172.21.15.161:0/3420110941 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64bc009420 con 0x7f64d00aeb30 2024-05-22T21:13:34.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64cfe00640 1 --2- 172.21.15.161:0/3420110941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d00b2050 0x7f64d0148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:34.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f64d5000640 1 --2- 172.21.15.161:0/3420110941 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64d00aeb30 0x7f64d014cdf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f64c400b410 tx=0x7f64c400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b4c00640 1 -- 172.21.15.161:0/197147425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3b015e5f0 msgr2=0x7fd3b0152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b4c00640 1 --2- 172.21.15.161:0/197147425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3b015e5f0 0x7fd3b0152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b4c00640 1 -- 172.21.15.161:0/197147425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3b006fad0 msgr2=0x7fd3b00c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b4c00640 1 --2- 172.21.15.161:0/197147425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3b006fad0 0x7fd3b00c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b4c00640 1 -- 172.21.15.161:0/197147425 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a4009420 con 0x7fd3b00cf000 2024-05-22T21:13:34.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b5600640 1 --2- 172.21.15.161:0/197147425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3b006fad0 0x7fd3b00c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:34.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f1734c00640 1 -- 172.21.15.161:0/2749540699 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f17280b3040 con 0x7f17380676a0 2024-05-22T21:13:34.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7fd3b4c00640 1 --2- 172.21.15.161:0/197147425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3b00cf000 0x7fd3b00ca620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4004000 tx=0x7fd3a40190f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f1734c00640 1 --2- 172.21.15.161:0/2749540699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f17180666c0 0x7f1718068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f1734c00640 1 -- 172.21.15.161:0/2749540699 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f172811e550 con 0x7f17380676a0 2024-05-22T21:13:34.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.102+0000 7f1737e00640 1 --2- 172.21.15.161:0/2749540699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f17180666c0 0x7f1718068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.106+0000 7f1737e00640 1 --2- 172.21.15.161:0/2749540699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f17180666c0 0x7f1718068b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f173809a360 tx=0x7f1724009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.106+0000 7fd3ad600640 1 -- 172.21.15.161:0/197147425 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4003af0 con 0x7fd3b00cf000 2024-05-22T21:13:34.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.106+0000 7fd3ad600640 1 -- 172.21.15.161:0/197147425 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a4003c50 con 0x7fd3b00cf000 2024-05-22T21:13:34.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.106+0000 7fd3ad600640 1 -- 172.21.15.161:0/197147425 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a40ab440 con 0x7fd3b00cf000 2024-05-22T21:13:34.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.106+0000 7fd3b8000640 1 -- 172.21.15.161:0/197147425 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b00cf9a0 con 0x7fd3b00cf000 2024-05-22T21:13:34.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.106+0000 7fd3b8000640 1 -- 172.21.15.161:0/197147425 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b00c9610 con 0x7fd3b00cf000 2024-05-22T21:13:34.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.106+0000 7f1734c00640 1 -- 172.21.15.161:0/2749540699 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f17280eb950 con 0x7f17380676a0 2024-05-22T21:13:34.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.110+0000 7fd3b8000640 1 -- 172.21.15.161:0/197147425 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b0073ca0 con 0x7fd3b00cf000 2024-05-22T21:13:34.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.110+0000 7fd3ad600640 1 -- 172.21.15.161:0/197147425 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd3a40b2040 con 0x7fd3b00cf000 2024-05-22T21:13:34.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.110+0000 7fd3ad600640 1 --2- 172.21.15.161:0/197147425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd390066690 0x7fd390068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.110+0000 7fd3ad600640 1 -- 172.21.15.161:0/197147425 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3a411df70 con 0x7fd3b00cf000 2024-05-22T21:13:34.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.110+0000 7fd3b5600640 1 --2- 172.21.15.161:0/197147425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd390066690 0x7fd390068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.110+0000 7fd3b5600640 1 --2- 172.21.15.161:0/197147425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd390066690 0x7fd390068b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0011f0 tx=0x7fd39c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.110+0000 7f83f3400640 1 --2- 172.21.15.161:0/150769577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83f4070d80 0x7f83f4149960 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f83e00ad040 tx=0x7f83e0019210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.110+0000 7fe547400640 1 --2- 172.21.15.161:0/4093986142 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe52c0047d0 0x7fe52c004c50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe53800a860 tx=0x7fe53801d1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aad6658d17f33e2f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7fd3ad600640 1 -- 172.21.15.161:0/197147425 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd3a40eb2d0 con 0x7fd3b00cf000 2024-05-22T21:13:34.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7f83ebe00640 1 -- 172.21.15.161:0/150769577 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e00bb070 con 0x7f83f4070d80 2024-05-22T21:13:34.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7fe546a00640 1 -- 172.21.15.161:0/4093986142 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5380ac600 con 0x7fe52c0047d0 2024-05-22T21:13:34.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7fb3ecc00640 1 -- 172.21.15.161:0/2068324114 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80aec30 con 0x7fb3f015b600 2024-05-22T21:13:34.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7fb3ecc00640 1 -- 172.21.15.161:0/2068324114 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e80aedb0 con 0x7fb3f015b600 2024-05-22T21:13:34.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7f83faa00640 1 -- 172.21.15.161:0/150769577 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f41480e0 con 0x7f83f4070d80 2024-05-22T21:13:34.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7f83ebe00640 1 -- 172.21.15.161:0/150769577 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e00b43f0 con 0x7f83f4070d80 2024-05-22T21:13:34.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7fb3ecc00640 1 -- 172.21.15.161:0/2068324114 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80b7650 con 0x7fb3f015b600 2024-05-22T21:13:34.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7f01bcc00640 1 -- 172.21.15.161:0/2714584894 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b00a55b0 con 0x7f01c00ada60 2024-05-22T21:13:34.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7f01bcc00640 1 -- 172.21.15.161:0/2714584894 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b0004220 con 0x7f01c00ada60 2024-05-22T21:13:34.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7f01bcc00640 1 -- 172.21.15.161:0/2714584894 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b00a55b0 con 0x7f01c00ada60 2024-05-22T21:13:34.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7f83ebe00640 1 -- 172.21.15.161:0/150769577 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e00b47c0 con 0x7f83f4070d80 2024-05-22T21:13:34.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7f83faa00640 1 -- 172.21.15.161:0/150769577 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f41486a0 con 0x7f83f4070d80 2024-05-22T21:13:34.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7f64ccc00640 1 -- 172.21.15.161:0/3420110941 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4011830 con 0x7f64d00aeb30 2024-05-22T21:13:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7f64ccc00640 1 -- 172.21.15.161:0/3420110941 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c4011990 con 0x7f64d00aeb30 2024-05-22T21:13:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7f64ccc00640 1 -- 172.21.15.161:0/3420110941 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4011d40 con 0x7f64d00aeb30 2024-05-22T21:13:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7fe546a00640 1 -- 172.21.15.161:0/4093986142 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5380a8050 con 0x7fe52c0047d0 2024-05-22T21:13:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7fe546a00640 1 -- 172.21.15.161:0/4093986142 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5380ac9a0 con 0x7fe52c0047d0 2024-05-22T21:13:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7f01c7800640 1 -- 172.21.15.161:0/2714584894 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c01fde30 con 0x7f01c00ada60 2024-05-22T21:13:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7f01c7800640 1 -- 172.21.15.161:0/2714584894 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c01fd220 con 0x7f01c00ada60 2024-05-22T21:13:34.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7fb3f7800640 1 -- 172.21.15.161:0/2068324114 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e8009a70 con 0x7fb3f015b600 2024-05-22T21:13:34.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.114+0000 7fb3f7800640 1 -- 172.21.15.161:0/2068324114 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f013f150 con 0x7fb3f015b600 2024-05-22T21:13:34.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7fc8afe00640 1 -- 172.21.15.161:0/3317886474 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc8b0051680 con 0x7fc8b404ffb0 2024-05-22T21:13:34.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7fe54ee00640 1 -- 172.21.15.161:0/4093986142 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe52c0047d0 msgr2=0x7fe52c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7fe54ee00640 1 --2- 172.21.15.161:0/4093986142 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe52c0047d0 0x7fe52c004c50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe53800a860 tx=0x7fe53801d1c0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7fe54ee00640 1 -- 172.21.15.161:0/4093986142 shutdown_connections 2024-05-22T21:13:34.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7f64d7a00640 1 -- 172.21.15.161:0/3420110941 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d014b400 con 0x7f64d00aeb30 2024-05-22T21:13:34.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7f64d7a00640 1 -- 172.21.15.161:0/3420110941 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d014b9c0 con 0x7f64d00aeb30 2024-05-22T21:13:34.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7fb3f7800640 1 -- 172.21.15.161:0/2068324114 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f0067220 con 0x7fb3f015b600 2024-05-22T21:13:34.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7fe54ee00640 1 --2- 172.21.15.161:0/4093986142 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe52c0047d0 0x7fe52c004c50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7f83faa00640 1 -- 172.21.15.161:0/150769577 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f4148890 con 0x7f83f4070d80 2024-05-22T21:13:34.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7fe54ee00640 1 -- 172.21.15.161:0/4093986142 >> 172.21.15.161:0/4093986142 conn(0x7fe5480b0bc0 msgr2=0x7fe5480b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:34.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7f01b6000640 1 -- 172.21.15.161:0/2714584894 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0194005260 con 0x7f01c00ada60 2024-05-22T21:13:34.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7fe54ee00640 1 -- 172.21.15.161:0/4093986142 shutdown_connections 2024-05-22T21:13:34.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7fe54ee00640 1 -- 172.21.15.161:0/4093986142 wait complete. 2024-05-22T21:13:34.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7f83ebe00640 1 -- 172.21.15.161:0/150769577 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f83e00b4920 con 0x7f83f4070d80 2024-05-22T21:13:34.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7f64d7a00640 1 -- 172.21.15.161:0/3420110941 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d0067220 con 0x7f64d00aeb30 2024-05-22T21:13:34.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7fb3ecc00640 1 -- 172.21.15.161:0/2068324114 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb3e80b4050 con 0x7fb3f015b600 2024-05-22T21:13:34.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7f83ebe00640 1 --2- 172.21.15.161:0/150769577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f83d4066690 0x7f83d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.118+0000 7f83ebe00640 1 -- 172.21.15.161:0/150769577 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83e011dba0 con 0x7f83f4070d80 2024-05-22T21:13:34.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7fb3ecc00640 1 --2- 172.21.15.161:0/2068324114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3d0066690 0x7fb3d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7fb3ecc00640 1 -- 172.21.15.161:0/2068324114 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3e811f070 con 0x7fb3f015b600 2024-05-22T21:13:34.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7f83f3e00640 1 --2- 172.21.15.161:0/150769577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f83d4066690 0x7f83d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7fb3f4e00640 1 --2- 172.21.15.161:0/2068324114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3d0066690 0x7fb3d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7f64ccc00640 1 -- 172.21.15.161:0/3420110941 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f64c401a400 con 0x7f64d00aeb30 2024-05-22T21:13:34.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7fe54ee00640 1 Processor -- start 2024-05-22T21:13:34.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7f01bcc00640 1 -- 172.21.15.161:0/2714584894 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f01b00b4070 con 0x7f01c00ada60 2024-05-22T21:13:34.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7f01bcc00640 1 --2- 172.21.15.161:0/2714584894 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f01a0066720 0x7f01a0068be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7f01bcc00640 1 -- 172.21.15.161:0/2714584894 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01b011db00 con 0x7f01c00ada60 2024-05-22T21:13:34.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7fe54ee00640 1 -- start start 2024-05-22T21:13:34.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7f01c4e00640 1 --2- 172.21.15.161:0/2714584894 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f01a0066720 0x7f01a0068be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7fb3f4e00640 1 --2- 172.21.15.161:0/2068324114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3d0066690 0x7fb3d0068b50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4005890 tx=0x7fb3e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7f83f3e00640 1 --2- 172.21.15.161:0/150769577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f83d4066690 0x7f83d4068b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f83f4158a00 tx=0x7f83e4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7f64ccc00640 1 --2- 172.21.15.161:0/3420110941 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f64b00666e0 0x7f64b0068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7f64ccc00640 1 -- 172.21.15.161:0/3420110941 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64c4083fb0 con 0x7f64d00aeb30 2024-05-22T21:13:34.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7f01c4e00640 1 --2- 172.21.15.161:0/2714584894 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f01a0066720 0x7f01a0068be0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f01ac001ec0 tx=0x7f01ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.122+0000 7f64cfe00640 1 --2- 172.21.15.161:0/3420110941 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f64b00666e0 0x7f64b0068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7f83ebe00640 1 -- 172.21.15.161:0/150769577 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f83e00eaf00 con 0x7f83f4070d80 2024-05-22T21:13:34.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe54ee00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c0047d0 0x7fe5481392d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7f64cfe00640 1 --2- 172.21.15.161:0/3420110941 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f64b00666e0 0x7f64b0068ba0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f64bc019e80 tx=0x7f64bc0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe54ee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe54813f5c0 0x7fe5481edd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe54ee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe5481ee270 0x7fe5481f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe54ee00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54814cc40 con 0x7fe52c0047d0 2024-05-22T21:13:34.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe54ee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54814cf20 con 0x7fe54813f5c0 2024-05-22T21:13:34.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe54ee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54814d200 con 0x7fe5481ee270 2024-05-22T21:13:34.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe547e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c0047d0 0x7fe5481392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe547e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c0047d0 0x7fe5481392d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49290/0 (socket says 172.21.15.161:49290) 2024-05-22T21:13:34.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe547e00640 1 -- 172.21.15.161:0/1762326416 learned_addr learned my addr 172.21.15.161:0/1762326416 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:34.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe54ce00640 1 --2- 172.21.15.161:0/1762326416 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe5481ee270 0x7fe5481f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe547400640 1 --2- 172.21.15.161:0/1762326416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe54813f5c0 0x7fe5481edd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe54ce00640 1 -- 172.21.15.161:0/1762326416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe54813f5c0 msgr2=0x7fe5481edd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe54ce00640 1 --2- 172.21.15.161:0/1762326416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe54813f5c0 0x7fe5481edd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe54ce00640 1 -- 172.21.15.161:0/1762326416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c0047d0 msgr2=0x7fe5481392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe54ce00640 1 --2- 172.21.15.161:0/1762326416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c0047d0 0x7fe5481392d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe54ce00640 1 -- 172.21.15.161:0/1762326416 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54814d4e0 con 0x7fe5481ee270 2024-05-22T21:13:34.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe547e00640 1 --2- 172.21.15.161:0/1762326416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c0047d0 0x7fe5481392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:34.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.126+0000 7fe54ce00640 1 --2- 172.21.15.161:0/1762326416 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe5481ee270 0x7fe5481f2130 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe548071840 tx=0x7fe530011cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.130+0000 7f01bcc00640 1 -- 172.21.15.161:0/2714584894 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f01b00eadc0 con 0x7f01c00ada60 2024-05-22T21:13:34.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.134+0000 7fb3ecc00640 1 -- 172.21.15.161:0/2068324114 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb3e80ec340 con 0x7fb3f015b600 2024-05-22T21:13:34.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.138+0000 7f64ccc00640 1 -- 172.21.15.161:0/3420110941 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f64c4051540 con 0x7f64d00aeb30 2024-05-22T21:13:34.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.138+0000 7fe53fe00640 1 -- 172.21.15.161:0/1762326416 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530004240 con 0x7fe5481ee270 2024-05-22T21:13:34.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.146+0000 7fe53fe00640 1 -- 172.21.15.161:0/1762326416 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5300043a0 con 0x7fe5481ee270 2024-05-22T21:13:34.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.146+0000 7fe53fe00640 1 -- 172.21.15.161:0/1762326416 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53000f910 con 0x7fe5481ee270 2024-05-22T21:13:34.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.146+0000 7fe54ee00640 1 -- 172.21.15.161:0/1762326416 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54814d7c0 con 0x7fe5481ee270 2024-05-22T21:13:34.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.146+0000 7fe54ee00640 1 -- 172.21.15.161:0/1762326416 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54814c860 con 0x7fe5481ee270 2024-05-22T21:13:34.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.146+0000 7fe54ee00640 1 -- 172.21.15.161:0/1762326416 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe548067220 con 0x7fe5481ee270 2024-05-22T21:13:34.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.154+0000 7fe53fe00640 1 -- 172.21.15.161:0/1762326416 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe53000fa70 con 0x7fe5481ee270 2024-05-22T21:13:34.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.154+0000 7fe53fe00640 1 --2- 172.21.15.161:0/1762326416 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe5240666c0 0x7fe524068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:34.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.154+0000 7fe53fe00640 1 -- 172.21.15.161:0/1762326416 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe530055360 con 0x7fe5481ee270 2024-05-22T21:13:34.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.154+0000 7fe53fe00640 1 -- 172.21.15.161:0/1762326416 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe530051e30 con 0x7fe5481ee270 2024-05-22T21:13:34.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.154+0000 7fe547e00640 1 --2- 172.21.15.161:0/1762326416 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe5240666c0 0x7fe524068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:34.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.158+0000 7fe547e00640 1 --2- 172.21.15.161:0/1762326416 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe5240666c0 0x7fe524068b80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe5400015b0 tx=0x7fe540007d80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:34.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.422+0000 7f173f400640 1 -- 172.21.15.161:0/2749540699 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f173809a390 con 0x7f17380676a0 2024-05-22T21:13:34.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.422+0000 7f1734c00640 1 -- 172.21.15.161:0/2749540699 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f17280eef70 con 0x7f17380676a0 2024-05-22T21:13:34.426 INFO:teuthology.orchestra.run.smithi161.stdout:55834574853 2024-05-22T21:13:34.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.426+0000 7f173f400640 1 -- 172.21.15.161:0/2749540699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f17180666c0 msgr2=0x7f1718068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.426+0000 7f173f400640 1 --2- 172.21.15.161:0/2749540699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f17180666c0 0x7f1718068b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f173809a360 tx=0x7f1724009040 comp rx=0 tx=0).stop 2024-05-22T21:13:34.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.426+0000 7f173f400640 1 -- 172.21.15.161:0/2749540699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17380676a0 msgr2=0x7f173816c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.426+0000 7f173f400640 1 --2- 172.21.15.161:0/2749540699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17380676a0 0x7f173816c810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f17280af040 tx=0x7f1728019220 comp rx=0 tx=0).stop 2024-05-22T21:13:34.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.426+0000 7f173f400640 1 -- 172.21.15.161:0/2749540699 shutdown_connections 2024-05-22T21:13:34.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.426+0000 7f173f400640 1 --2- 172.21.15.161:0/2749540699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f17180666c0 0x7f1718068b80 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f173809a360 tx=0x7f1724009040 comp rx=0 tx=0).stop 2024-05-22T21:13:34.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.426+0000 7f173f400640 1 --2- 172.21.15.161:0/2749540699 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17380b2760 0x7f173806cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.426+0000 7f173f400640 1 --2- 172.21.15.161:0/2749540699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17380aeb30 0x7f1738148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.426+0000 7f173f400640 1 --2- 172.21.15.161:0/2749540699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17380676a0 0x7f173816c810 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.426+0000 7f173f400640 1 -- 172.21.15.161:0/2749540699 >> 172.21.15.161:0/2749540699 conn(0x7f17380714a0 msgr2=0x7f17380cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:34.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.426+0000 7f173f400640 1 -- 172.21.15.161:0/2749540699 shutdown_connections 2024-05-22T21:13:34.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.426+0000 7f173f400640 1 -- 172.21.15.161:0/2749540699 wait complete. 2024-05-22T21:13:34.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.438+0000 7fc8c1a00640 1 -- 172.21.15.161:0/3317886474 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc8b40089b0 con 0x7fc8b404ffb0 2024-05-22T21:13:34.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.438+0000 7fc8afe00640 1 -- 172.21.15.161:0/3317886474 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc8b0054ca0 con 0x7fc8b404ffb0 2024-05-22T21:13:34.444 INFO:teuthology.orchestra.run.smithi161.stdout:55834574853 2024-05-22T21:13:34.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.442+0000 7fc8c1a00640 1 -- 172.21.15.161:0/3317886474 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc89c066760 msgr2=0x7fc89c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.442+0000 7fc8c1a00640 1 --2- 172.21.15.161:0/3317886474 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc89c066760 0x7fc89c068c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc0686a0 tx=0x7fc8bc08d040 comp rx=0 tx=0).stop 2024-05-22T21:13:34.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.442+0000 7fc8c1a00640 1 -- 172.21.15.161:0/3317886474 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8b404ffb0 msgr2=0x7fc8b40cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.442+0000 7fc8c1a00640 1 --2- 172.21.15.161:0/3317886474 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8b404ffb0 0x7fc8b40cbb70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000e7d0 tx=0x7fc8b000ec90 comp rx=0 tx=0).stop 2024-05-22T21:13:34.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.442+0000 7fc8c1a00640 1 -- 172.21.15.161:0/3317886474 shutdown_connections 2024-05-22T21:13:34.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.442+0000 7fc8c1a00640 1 --2- 172.21.15.161:0/3317886474 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc89c066760 0x7fc89c068c20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.442+0000 7fc8c1a00640 1 --2- 172.21.15.161:0/3317886474 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8b404ffb0 0x7fc8b40cbb70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.442+0000 7fc8c1a00640 1 --2- 172.21.15.161:0/3317886474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8b4033cd0 0x7fc8b40d6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.442+0000 7fc8c1a00640 1 --2- 172.21.15.161:0/3317886474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8b400aab0 0x7fc8b4039da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.442+0000 7fc8c1a00640 1 -- 172.21.15.161:0/3317886474 >> 172.21.15.161:0/3317886474 conn(0x7fc8b4009f90 msgr2=0x7fc8b4036570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:34.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.446+0000 7fd3b8000640 1 -- 172.21.15.161:0/197147425 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fd3b0067220 con 0x7fd3b00cf000 2024-05-22T21:13:34.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.446+0000 7fc8c1a00640 1 -- 172.21.15.161:0/3317886474 shutdown_connections 2024-05-22T21:13:34.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.446+0000 7fc8c1a00640 1 -- 172.21.15.161:0/3317886474 wait complete. 2024-05-22T21:13:34.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.446+0000 7fd3ad600640 1 -- 172.21.15.161:0/197147425 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd3a40ee8f0 con 0x7fd3b00cf000 2024-05-22T21:13:34.452 INFO:teuthology.orchestra.run.smithi161.stdout:55834574853 2024-05-22T21:13:34.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.450+0000 7fd3b8000640 1 -- 172.21.15.161:0/197147425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd390066690 msgr2=0x7fd390068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.450+0000 7fd3b8000640 1 --2- 172.21.15.161:0/197147425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd390066690 0x7fd390068b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0011f0 tx=0x7fd39c009040 comp rx=0 tx=0).stop 2024-05-22T21:13:34.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.450+0000 7fd3b8000640 1 -- 172.21.15.161:0/197147425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3b00cf000 msgr2=0x7fd3b00ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.450+0000 7fd3b8000640 1 --2- 172.21.15.161:0/197147425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3b00cf000 0x7fd3b00ca620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4004000 tx=0x7fd3a40190f0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.454+0000 7f83faa00640 1 -- 172.21.15.161:0/150769577 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f83f4067220 con 0x7f83f4070d80 2024-05-22T21:13:34.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.454+0000 7fd3b8000640 1 -- 172.21.15.161:0/197147425 shutdown_connections 2024-05-22T21:13:34.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.454+0000 7fd3b8000640 1 --2- 172.21.15.161:0/197147425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd390066690 0x7fd390068b50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.454+0000 7fd3b8000640 1 --2- 172.21.15.161:0/197147425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3b015e5f0 0x7fd3b0152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.454+0000 7fd3b8000640 1 --2- 172.21.15.161:0/197147425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3b00cf000 0x7fd3b00ca620 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.454+0000 7fd3b8000640 1 --2- 172.21.15.161:0/197147425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3b006fad0 0x7fd3b00c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.454+0000 7fd3b8000640 1 -- 172.21.15.161:0/197147425 >> 172.21.15.161:0/197147425 conn(0x7fd3b00b2a80 msgr2=0x7fd3b006df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:34.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.454+0000 7fd3b8000640 1 -- 172.21.15.161:0/197147425 shutdown_connections 2024-05-22T21:13:34.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.454+0000 7fd3b8000640 1 -- 172.21.15.161:0/197147425 wait complete. 2024-05-22T21:13:34.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.458+0000 7f83ebe00640 1 -- 172.21.15.161:0/150769577 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f83e00ee520 con 0x7f83f4070d80 2024-05-22T21:13:34.460 INFO:teuthology.orchestra.run.smithi161.stdout:55834574853 2024-05-22T21:13:34.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.458+0000 7f83faa00640 1 -- 172.21.15.161:0/150769577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f83d4066690 msgr2=0x7f83d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.458+0000 7f83faa00640 1 --2- 172.21.15.161:0/150769577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f83d4066690 0x7f83d4068b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f83f4158a00 tx=0x7f83e4009190 comp rx=0 tx=0).stop 2024-05-22T21:13:34.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.458+0000 7f83faa00640 1 -- 172.21.15.161:0/150769577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83f4070d80 msgr2=0x7f83f4149960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.458+0000 7f83faa00640 1 --2- 172.21.15.161:0/150769577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83f4070d80 0x7f83f4149960 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f83e00ad040 tx=0x7f83e0019210 comp rx=0 tx=0).stop 2024-05-22T21:13:34.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.462+0000 7f83faa00640 1 -- 172.21.15.161:0/150769577 shutdown_connections 2024-05-22T21:13:34.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.462+0000 7f83faa00640 1 --2- 172.21.15.161:0/150769577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f83d4066690 0x7f83d4068b50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.462+0000 7f83faa00640 1 --2- 172.21.15.161:0/150769577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f414b1c0 0x7f83f4149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.462+0000 7f83faa00640 1 --2- 172.21.15.161:0/150769577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83f4070d80 0x7f83f4149960 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.462+0000 7f83faa00640 1 --2- 172.21.15.161:0/150769577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83f406c7e0 0x7f83f4141fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.462+0000 7f83faa00640 1 -- 172.21.15.161:0/150769577 >> 172.21.15.161:0/150769577 conn(0x7f83f406a2d0 msgr2=0x7f83f41451f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:34.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.462+0000 7f64d7a00640 1 -- 172.21.15.161:0/3420110941 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f64d014d330 con 0x7f64d00aeb30 2024-05-22T21:13:34.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.462+0000 7f83faa00640 1 -- 172.21.15.161:0/150769577 shutdown_connections 2024-05-22T21:13:34.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.462+0000 7f83faa00640 1 -- 172.21.15.161:0/150769577 wait complete. 2024-05-22T21:13:34.465 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-05-22T21:13:34.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.462+0000 7f64ccc00640 1 -- 172.21.15.161:0/3420110941 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f64c4015090 con 0x7f64d00aeb30 2024-05-22T21:13:34.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.462+0000 7f01b6000640 1 -- 172.21.15.161:0/2714584894 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f01940057e0 con 0x7f01c00ada60 2024-05-22T21:13:34.466 INFO:teuthology.orchestra.run.smithi161.stdout:55834574853 2024-05-22T21:13:34.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.462+0000 7f01bcc00640 1 -- 172.21.15.161:0/2714584894 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f01b00b2090 con 0x7f01c00ada60 2024-05-22T21:13:34.467 INFO:teuthology.orchestra.run.smithi161.stdout:55834574853 2024-05-22T21:13:34.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f64d7a00640 1 -- 172.21.15.161:0/3420110941 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f64b00666e0 msgr2=0x7f64b0068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f64d7a00640 1 --2- 172.21.15.161:0/3420110941 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f64b00666e0 0x7f64b0068ba0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f64bc019e80 tx=0x7f64bc0a6000 comp rx=0 tx=0).stop 2024-05-22T21:13:34.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f64d7a00640 1 -- 172.21.15.161:0/3420110941 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64d00aeb30 msgr2=0x7f64d014cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f64d7a00640 1 --2- 172.21.15.161:0/3420110941 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64d00aeb30 0x7f64d014cdf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f64c400b410 tx=0x7f64c400b8d0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f01b6000640 1 -- 172.21.15.161:0/2714584894 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f01a0066720 msgr2=0x7f01a0068be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f01b6000640 1 --2- 172.21.15.161:0/2714584894 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f01a0066720 0x7f01a0068be0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f01ac001ec0 tx=0x7f01ac009040 comp rx=0 tx=0).stop 2024-05-22T21:13:34.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f01b6000640 1 -- 172.21.15.161:0/2714584894 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f01c00ada60 msgr2=0x7f01c00adee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f01b6000640 1 --2- 172.21.15.161:0/2714584894 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f01c00ada60 0x7f01c00adee0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f01b00ac000 tx=0x7f01a4005800 comp rx=0 tx=0).stop 2024-05-22T21:13:34.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f64d7a00640 1 -- 172.21.15.161:0/3420110941 shutdown_connections 2024-05-22T21:13:34.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f64d7a00640 1 --2- 172.21.15.161:0/3420110941 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f64b00666e0 0x7f64b0068ba0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f64d7a00640 1 --2- 172.21.15.161:0/3420110941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64d014ac90 0x7f64d01461b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f64d7a00640 1 --2- 172.21.15.161:0/3420110941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64d00b2050 0x7f64d0148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f64d7a00640 1 --2- 172.21.15.161:0/3420110941 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64d00aeb30 0x7f64d014cdf0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f64d7a00640 1 -- 172.21.15.161:0/3420110941 >> 172.21.15.161:0/3420110941 conn(0x7f64d00714a0 msgr2=0x7f64d006d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f01b6000640 1 -- 172.21.15.161:0/2714584894 shutdown_connections 2024-05-22T21:13:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f01b6000640 1 --2- 172.21.15.161:0/2714584894 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f01a0066720 0x7f01a0068be0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f01b6000640 1 --2- 172.21.15.161:0/2714584894 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f01c00ada60 0x7f01c00adee0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f01b6000640 1 --2- 172.21.15.161:0/2714584894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01c00abd10 0x7f01c00ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f01b6000640 1 --2- 172.21.15.161:0/2714584894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01a4004d10 0x7f01c01f93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f01b6000640 1 -- 172.21.15.161:0/2714584894 >> 172.21.15.161:0/2714584894 conn(0x7f01c006a2d0 msgr2=0x7f01c00b1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f01b6000640 1 -- 172.21.15.161:0/2714584894 shutdown_connections 2024-05-22T21:13:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f01b6000640 1 -- 172.21.15.161:0/2714584894 wait complete. 2024-05-22T21:13:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f64d7a00640 1 -- 172.21.15.161:0/3420110941 shutdown_connections 2024-05-22T21:13:34.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.466+0000 7f64d7a00640 1 -- 172.21.15.161:0/3420110941 wait complete. 2024-05-22T21:13:34.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.478+0000 7fb3f7800640 1 -- 172.21.15.161:0/2068324114 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb3f015c0d0 con 0x7fb3f015b600 2024-05-22T21:13:34.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.478+0000 7fb3ecc00640 1 -- 172.21.15.161:0/2068324114 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb3e80b2020 con 0x7fb3f015b600 2024-05-22T21:13:34.483 INFO:teuthology.orchestra.run.smithi161.stdout:55834574854 2024-05-22T21:13:34.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.482+0000 7fb3f7800640 1 -- 172.21.15.161:0/2068324114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3d0066690 msgr2=0x7fb3d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.482+0000 7fb3f7800640 1 --2- 172.21.15.161:0/2068324114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3d0066690 0x7fb3d0068b50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4005890 tx=0x7fb3e4009040 comp rx=0 tx=0).stop 2024-05-22T21:13:34.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.482+0000 7fb3f7800640 1 -- 172.21.15.161:0/2068324114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3f015b600 msgr2=0x7fb3f0138fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.482+0000 7fb3f7800640 1 --2- 172.21.15.161:0/2068324114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3f015b600 0x7fb3f0138fc0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3e801a5a0 tx=0x7fb3e8019710 comp rx=0 tx=0).stop 2024-05-22T21:13:34.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.482+0000 7fb3f7800640 1 -- 172.21.15.161:0/2068324114 shutdown_connections 2024-05-22T21:13:34.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.482+0000 7fb3f7800640 1 --2- 172.21.15.161:0/2068324114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3d0066690 0x7fb3d0068b50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.482+0000 7fb3f7800640 1 --2- 172.21.15.161:0/2068324114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3f015b600 0x7fb3f0138fc0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.482+0000 7fb3f7800640 1 --2- 172.21.15.161:0/2068324114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3f0143c40 0x7fb3f01371c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.482+0000 7fb3f7800640 1 --2- 172.21.15.161:0/2068324114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3f006c7e0 0x7fb3f0159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.482+0000 7fb3f7800640 1 -- 172.21.15.161:0/2068324114 >> 172.21.15.161:0/2068324114 conn(0x7fb3f006a2d0 msgr2=0x7fb3f00af6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:34.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.482+0000 7fb3f7800640 1 -- 172.21.15.161:0/2068324114 shutdown_connections 2024-05-22T21:13:34.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.482+0000 7fb3f7800640 1 -- 172.21.15.161:0/2068324114 wait complete. 2024-05-22T21:13:34.488 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-05-22T21:13:34.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.486+0000 7fe54ee00640 1 -- 172.21.15.161:0/1762326416 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe54814ede0 con 0x7fe5481ee270 2024-05-22T21:13:34.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.486+0000 7fe53fe00640 1 -- 172.21.15.161:0/1762326416 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe530051e30 con 0x7fe5481ee270 2024-05-22T21:13:34.492 INFO:teuthology.orchestra.run.smithi161.stdout:55834574854 2024-05-22T21:13:34.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.494+0000 7fe54ee00640 1 -- 172.21.15.161:0/1762326416 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe5240666c0 msgr2=0x7fe524068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.494+0000 7fe54ee00640 1 --2- 172.21.15.161:0/1762326416 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe5240666c0 0x7fe524068b80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe5400015b0 tx=0x7fe540007d80 comp rx=0 tx=0).stop 2024-05-22T21:13:34.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.494+0000 7fe54ee00640 1 -- 172.21.15.161:0/1762326416 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe5481ee270 msgr2=0x7fe5481f2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:34.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.494+0000 7fe54ee00640 1 --2- 172.21.15.161:0/1762326416 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe5481ee270 0x7fe5481f2130 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe548071840 tx=0x7fe530011cf0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.494+0000 7fe54ee00640 1 -- 172.21.15.161:0/1762326416 shutdown_connections 2024-05-22T21:13:34.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.494+0000 7fe54ee00640 1 --2- 172.21.15.161:0/1762326416 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe5240666c0 0x7fe524068b80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.494+0000 7fe54ee00640 1 --2- 172.21.15.161:0/1762326416 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe5481ee270 0x7fe5481f2130 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.494+0000 7fe54ee00640 1 --2- 172.21.15.161:0/1762326416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe54813f5c0 0x7fe5481edd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.494+0000 7fe54ee00640 1 --2- 172.21.15.161:0/1762326416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe52c0047d0 0x7fe5481392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:34.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.494+0000 7fe54ee00640 1 -- 172.21.15.161:0/1762326416 >> 172.21.15.161:0/1762326416 conn(0x7fe5480b0bc0 msgr2=0x7fe548071000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:34.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.494+0000 7fe54ee00640 1 -- 172.21.15.161:0/1762326416 shutdown_connections 2024-05-22T21:13:34.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:34.494+0000 7fe54ee00640 1 -- 172.21.15.161:0/1762326416 wait complete. 2024-05-22T21:13:34.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-05-22T21:13:34.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-05-22T21:13:34.506 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-05-22T21:13:34.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-05-22T21:13:34.522 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-05-22T21:13:34.523 DEBUG:teuthology.parallel:result is None 2024-05-22T21:13:34.527 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-05-22T21:13:34.527 DEBUG:teuthology.parallel:result is None 2024-05-22T21:13:35.466 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-22T21:13:35.489 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-22T21:13:35.505 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-22T21:13:35.506 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-22T21:13:35.507 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-22T21:13:35.513 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-22T21:13:35.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.666+0000 7f3292c00640 1 Processor -- start 2024-05-22T21:13:35.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.666+0000 7f3292c00640 1 -- start start 2024-05-22T21:13:35.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.666+0000 7f3292c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f328c161840 0x7f328c0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.666+0000 7f3292c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f328c0c4670 0x7f328c0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.666+0000 7f3292c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328c161f70 con 0x7f328c0b3630 2024-05-22T21:13:35.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.666+0000 7f3292c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f328c0b0870 con 0x7f328c0c4670 2024-05-22T21:13:35.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.666+0000 7f3292c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c162250 con 0x7f328c161840 2024-05-22T21:13:35.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.666+0000 7f328b400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f328c0b3630 0x7f328c0c3f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56256/0 (socket says 172.21.15.161:56256) 2024-05-22T21:13:35.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.666+0000 7f328b400640 1 -- 172.21.15.161:0/452275174 learned_addr learned my addr 172.21.15.161:0/452275174 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:35.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.666+0000 7f328be00640 1 --2- 172.21.15.161:0/452275174 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f328c161840 0x7f328c0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.666+0000 7f3290c00640 1 --2- 172.21.15.161:0/452275174 >> v2:172.21.15.179:3300/0 conn(0x7f328c0c4670 0x7f328c0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.670+0000 7f328be00640 1 -- 172.21.15.161:0/452275174 >> v2:172.21.15.179:3300/0 conn(0x7f328c0c4670 msgr2=0x7f328c0b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:35.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.670+0000 7f328be00640 1 --2- 172.21.15.161:0/452275174 >> v2:172.21.15.179:3300/0 conn(0x7f328c0c4670 0x7f328c0b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.670+0000 7f328be00640 1 -- 172.21.15.161:0/452275174 >> v1:172.21.15.161:6789/0 conn(0x7f328c0b3630 legacy=0x7f328c0c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:35.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.670+0000 7f328be00640 1 -- 172.21.15.161:0/452275174 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c162530 con 0x7f328c161840 2024-05-22T21:13:35.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.670+0000 7f328be00640 1 --2- 172.21.15.161:0/452275174 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f328c161840 0x7f328c0cb5e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f327c0097c0 tx=0x7f327c01c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c798303e12b10925 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.674+0000 7f328aa00640 1 -- 172.21.15.161:0/452275174 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c0ac410 con 0x7f328c161840 2024-05-22T21:13:35.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.674+0000 7f328aa00640 1 -- 172.21.15.161:0/452275174 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c0ac570 con 0x7f328c161840 2024-05-22T21:13:35.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.674+0000 7f328aa00640 1 -- 172.21.15.161:0/452275174 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c0ac900 con 0x7f328c161840 2024-05-22T21:13:35.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.674+0000 7f3292c00640 1 -- 172.21.15.161:0/452275174 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f328c161840 msgr2=0x7f328c0cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.674+0000 7f3292c00640 1 --2- 172.21.15.161:0/452275174 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f328c161840 0x7f328c0cb5e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f327c0097c0 tx=0x7f327c01c3d0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.674+0000 7f3292c00640 1 -- 172.21.15.161:0/452275174 shutdown_connections 2024-05-22T21:13:35.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.674+0000 7f3292c00640 1 --2- 172.21.15.161:0/452275174 >> v2:172.21.15.179:3300/0 conn(0x7f328c0c4670 0x7f328c0b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.674+0000 7f3292c00640 1 --2- 172.21.15.161:0/452275174 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f328c161840 0x7f328c0cb5e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.674+0000 7f3292c00640 1 -- 172.21.15.161:0/452275174 >> 172.21.15.161:0/452275174 conn(0x7f328c0b2a80 msgr2=0x7f328c0b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:35.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.674+0000 7f3292c00640 1 -- 172.21.15.161:0/452275174 shutdown_connections 2024-05-22T21:13:35.678 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.674+0000 7f3292c00640 1 -- 172.21.15.161:0/452275174 wait complete. 2024-05-22T21:13:35.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.674+0000 7f3292c00640 1 Processor -- start 2024-05-22T21:13:35.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.674+0000 7f3292c00640 1 -- start start 2024-05-22T21:13:35.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3292c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f328c0b3630 0x7f328c154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3292c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f328c0c4670 0x7f328c0cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3292c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c161840 0x7f328c0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3292c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0cf970 con 0x7f328c161840 2024-05-22T21:13:35.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3292c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0cfc50 con 0x7f328c0b3630 2024-05-22T21:13:35.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3292c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0cff30 con 0x7f328c0c4670 2024-05-22T21:13:35.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3290c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c161840 0x7f328c0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3290c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c161840 0x7f328c0ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49300/0 (socket says 172.21.15.161:49300) 2024-05-22T21:13:35.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3290c00640 1 -- 172.21.15.161:0/2704597109 learned_addr learned my addr 172.21.15.161:0/2704597109 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:35.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f328be00640 1 --2- 172.21.15.161:0/2704597109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f328c0b3630 0x7f328c154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f328b400640 1 --2- 172.21.15.161:0/2704597109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f328c0c4670 0x7f328c0cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f6912000640 1 Processor -- start 2024-05-22T21:13:35.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3290c00640 1 -- 172.21.15.161:0/2704597109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f328c0c4670 msgr2=0x7f328c0cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3290c00640 1 --2- 172.21.15.161:0/2704597109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f328c0c4670 0x7f328c0cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3290c00640 1 -- 172.21.15.161:0/2704597109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f328c0b3630 msgr2=0x7f328c154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3290c00640 1 --2- 172.21.15.161:0/2704597109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f328c0b3630 0x7f328c154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3290c00640 1 -- 172.21.15.161:0/2704597109 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c009420 con 0x7f328c161840 2024-05-22T21:13:35.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f328b400640 1 --2- 172.21.15.161:0/2704597109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f328c0c4670 0x7f328c0cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:35.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f6912000640 1 -- start start 2024-05-22T21:13:35.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3290c00640 1 --2- 172.21.15.161:0/2704597109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c161840 0x7f328c0ccd00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f328400efc0 tx=0x7f328400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3283e00640 1 -- 172.21.15.161:0/2704597109 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284014480 con 0x7f328c161840 2024-05-22T21:13:35.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3283e00640 1 -- 172.21.15.161:0/2704597109 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328400f040 con 0x7f328c161840 2024-05-22T21:13:35.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3283e00640 1 -- 172.21.15.161:0/2704597109 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284014820 con 0x7f328c161840 2024-05-22T21:13:35.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3292c00640 1 -- 172.21.15.161:0/2704597109 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328c0d0210 con 0x7f328c161840 2024-05-22T21:13:35.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f3292c00640 1 -- 172.21.15.161:0/2704597109 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328c154990 con 0x7f328c161840 2024-05-22T21:13:35.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f6912000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f690c15e5f0 0x7f690c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f6912000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f690c0cf000 0x7f690c0cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f6912000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690c06f030 con 0x7f690c06fad0 2024-05-22T21:13:35.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f6912000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690c0cf9d0 con 0x7f690c15e5f0 2024-05-22T21:13:35.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.678+0000 7f6912000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c06f840 con 0x7f690c0cf000 2024-05-22T21:13:35.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.682+0000 7f6903e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f690c06fad0 0x7f690c15f450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56272/0 (socket says 172.21.15.161:56272) 2024-05-22T21:13:35.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.682+0000 7f6903e00640 1 -- 172.21.15.161:0/2133103134 learned_addr learned my addr 172.21.15.161:0/2133103134 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:35.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.682+0000 7f690b400640 1 --2- 172.21.15.161:0/2133103134 >> v2:172.21.15.161:3300/0 conn(0x7f690c15e5f0 0x7f690c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.682+0000 7f690be00640 1 --2- 172.21.15.161:0/2133103134 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f690c0cf000 0x7f690c0cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.682+0000 7f690aa00640 1 -- 172.21.15.161:0/2133103134 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491123178 0 0) 0x7f690c06f030 con 0x7f690c06fad0 2024-05-22T21:13:35.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.682+0000 7f690aa00640 1 -- 172.21.15.161:0/2133103134 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68f0003530 con 0x7f690c06fad0 2024-05-22T21:13:35.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.682+0000 7f690b400640 1 -- 172.21.15.161:0/2133103134 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f690c0cf000 msgr2=0x7f690c0cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:35.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.682+0000 7f690b400640 1 --2- 172.21.15.161:0/2133103134 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f690c0cf000 0x7f690c0cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.682+0000 7f690b400640 1 -- 172.21.15.161:0/2133103134 >> v1:172.21.15.161:6789/0 conn(0x7f690c06fad0 legacy=0x7f690c15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.682+0000 7f690b400640 1 -- 172.21.15.161:0/2133103134 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c15efc0 con 0x7f690c15e5f0 2024-05-22T21:13:35.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.682+0000 7f3283e00640 1 -- 172.21.15.161:0/2704597109 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3284014980 con 0x7f328c161840 2024-05-22T21:13:35.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.682+0000 7f3292c00640 1 -- 172.21.15.161:0/2704597109 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3264005260 con 0x7f328c161840 2024-05-22T21:13:35.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.682+0000 7f3283e00640 1 --2- 172.21.15.161:0/2704597109 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f326c066690 0x7f326c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.682+0000 7f3283e00640 1 -- 172.21.15.161:0/2704597109 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3284084f40 con 0x7f328c161840 2024-05-22T21:13:35.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.686+0000 7f328be00640 1 --2- 172.21.15.161:0/2704597109 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f326c066690 0x7f326c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.686+0000 7f690b400640 1 --2- 172.21.15.161:0/2133103134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f690c15e5f0 0x7f690c06e9a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6904009d90 tx=0x7f690401c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d4f34dd2120747 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.686+0000 7f328be00640 1 --2- 172.21.15.161:0/2704597109 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f326c066690 0x7f326c068b50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f327c0ae000 tx=0x7f327c0a6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.686+0000 7f550d200640 1 Processor -- start 2024-05-22T21:13:35.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.686+0000 7f550d200640 1 -- start start 2024-05-22T21:13:35.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.686+0000 7f550d200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5508160ad0 0x7f5508099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.686+0000 7f550d200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f55080b37b0 0x7f5508099f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f550d200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550806e4a0 con 0x7f55080d0bb0 2024-05-22T21:13:35.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f550d200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f550806c690 con 0x7f55080b37b0 2024-05-22T21:13:35.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f550d200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f550806c0f0 con 0x7f5508160ad0 2024-05-22T21:13:35.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f5506800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f55080b37b0 0x7f5508099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f5506800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f55080b37b0 0x7f5508099f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49326/0 (socket says 172.21.15.161:49326) 2024-05-22T21:13:35.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f5506800640 1 -- 172.21.15.161:0/3321649396 learned_addr learned my addr 172.21.15.161:0/3321649396 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:35.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f5505e00640 1 --2- 172.21.15.161:0/3321649396 >> v2:172.21.15.179:3300/0 conn(0x7f5508160ad0 0x7f5508099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f5506800640 1 -- 172.21.15.161:0/3321649396 >> v2:172.21.15.179:3300/0 conn(0x7f5508160ad0 msgr2=0x7f5508099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:35.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f5506800640 1 --2- 172.21.15.161:0/3321649396 >> v2:172.21.15.179:3300/0 conn(0x7f5508160ad0 0x7f5508099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f5506800640 1 -- 172.21.15.161:0/3321649396 >> v1:172.21.15.161:6789/0 conn(0x7f55080d0bb0 legacy=0x7f5508160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f5506800640 1 -- 172.21.15.161:0/3321649396 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550806cf50 con 0x7f55080b37b0 2024-05-22T21:13:35.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f690aa00640 1 -- 172.21.15.161:0/2133103134 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69040a5670 con 0x7f690c15e5f0 2024-05-22T21:13:35.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f690aa00640 1 -- 172.21.15.161:0/2133103134 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69040a57d0 con 0x7f690c15e5f0 2024-05-22T21:13:35.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f3283e00640 1 -- 172.21.15.161:0/2704597109 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f32840522b0 con 0x7f328c161840 2024-05-22T21:13:35.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f690aa00640 1 -- 172.21.15.161:0/2133103134 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69040a5b60 con 0x7f690c15e5f0 2024-05-22T21:13:35.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f6912000640 1 -- 172.21.15.161:0/2133103134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f690c15e5f0 msgr2=0x7f690c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f6912000640 1 --2- 172.21.15.161:0/2133103134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f690c15e5f0 0x7f690c06e9a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6904009d90 tx=0x7f690401c7f0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f6912000640 1 -- 172.21.15.161:0/2133103134 shutdown_connections 2024-05-22T21:13:35.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f6912000640 1 --2- 172.21.15.161:0/2133103134 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f690c0cf000 0x7f690c0cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f6912000640 1 --2- 172.21.15.161:0/2133103134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f690c15e5f0 0x7f690c06e9a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f6912000640 1 -- 172.21.15.161:0/2133103134 >> 172.21.15.161:0/2133103134 conn(0x7f690c0b2a80 msgr2=0x7f690c0b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:35.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f6912000640 1 -- 172.21.15.161:0/2133103134 shutdown_connections 2024-05-22T21:13:35.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.690+0000 7f6912000640 1 -- 172.21.15.161:0/2133103134 wait complete. 2024-05-22T21:13:35.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f5506800640 1 --2- 172.21.15.161:0/3321649396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55080b37b0 0x7f5508099f80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f55080701e0 tx=0x7f550001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2990c77219e9aae3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f6912000640 1 Processor -- start 2024-05-22T21:13:35.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f6912000640 1 -- start start 2024-05-22T21:13:35.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f5505400640 1 -- 172.21.15.161:0/3321649396 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55000a58b0 con 0x7f55080b37b0 2024-05-22T21:13:35.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f6912000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c06fad0 0x7f690c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f6912000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f690c0cf000 0x7f690c0b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f6912000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f690c15e5f0 0x7f690c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f6912000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c0c8060 con 0x7f690c0cf000 2024-05-22T21:13:35.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f6912000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c0c8ec0 con 0x7f690c06fad0 2024-05-22T21:13:35.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f6912000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c0c82b0 con 0x7f690c15e5f0 2024-05-22T21:13:35.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f5505400640 1 -- 172.21.15.161:0/3321649396 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55000a5a10 con 0x7f55080b37b0 2024-05-22T21:13:35.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f5505400640 1 -- 172.21.15.161:0/3321649396 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55000a5da0 con 0x7f55080b37b0 2024-05-22T21:13:35.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f550d200640 1 -- 172.21.15.161:0/3321649396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55080b37b0 msgr2=0x7f5508099f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f550d200640 1 --2- 172.21.15.161:0/3321649396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55080b37b0 0x7f5508099f80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f55080701e0 tx=0x7f550001c9f0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f550d200640 1 -- 172.21.15.161:0/3321649396 shutdown_connections 2024-05-22T21:13:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f550d200640 1 --2- 172.21.15.161:0/3321649396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55080b37b0 0x7f5508099f80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f550d200640 1 --2- 172.21.15.161:0/3321649396 >> v2:172.21.15.179:3300/0 conn(0x7f5508160ad0 0x7f5508099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f550d200640 1 -- 172.21.15.161:0/3321649396 >> 172.21.15.161:0/3321649396 conn(0x7f55080b2a80 msgr2=0x7f55080b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.694+0000 7f550d200640 1 -- 172.21.15.161:0/3321649396 shutdown_connections 2024-05-22T21:13:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f690b400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c06fad0 0x7f690c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f6903e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f690c0cf000 0x7f690c0b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f690b400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c06fad0 0x7f690c0c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:39708/0 (socket says 172.21.15.161:39708) 2024-05-22T21:13:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f690b400640 1 -- 172.21.15.161:0/1137407405 learned_addr learned my addr 172.21.15.161:0/1137407405 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f690be00640 1 --2- 172.21.15.161:0/1137407405 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f690c15e5f0 0x7f690c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f550d200640 1 -- 172.21.15.161:0/3321649396 wait complete. 2024-05-22T21:13:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f690b400640 1 -- 172.21.15.161:0/1137407405 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f690c15e5f0 msgr2=0x7f690c0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f690b400640 1 --2- 172.21.15.161:0/1137407405 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f690c15e5f0 0x7f690c0b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f690b400640 1 -- 172.21.15.161:0/1137407405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f690c0cf000 msgr2=0x7f690c0b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f690b400640 1 --2- 172.21.15.161:0/1137407405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f690c0cf000 0x7f690c0b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f690b400640 1 -- 172.21.15.161:0/1137407405 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6904009a70 con 0x7f690c06fad0 2024-05-22T21:13:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f6903e00640 1 --2- 172.21.15.161:0/1137407405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f690c0cf000 0x7f690c0b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f690be00640 1 --2- 172.21.15.161:0/1137407405 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f690c15e5f0 0x7f690c0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f690b400640 1 --2- 172.21.15.161:0/1137407405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c06fad0 0x7f690c0c4830 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f690401aea0 tx=0x7f6904019590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f6903400640 1 -- 172.21.15.161:0/1137407405 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69040b9860 con 0x7f690c06fad0 2024-05-22T21:13:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f6903400640 1 -- 172.21.15.161:0/1137407405 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69040b99c0 con 0x7f690c06fad0 2024-05-22T21:13:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f6903400640 1 -- 172.21.15.161:0/1137407405 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69040b9d50 con 0x7f690c06fad0 2024-05-22T21:13:35.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f6912000640 1 -- 172.21.15.161:0/1137407405 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690c0c8590 con 0x7f690c06fad0 2024-05-22T21:13:35.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f6912000640 1 -- 172.21.15.161:0/1137407405 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690c0c8b50 con 0x7f690c06fad0 2024-05-22T21:13:35.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f550d200640 1 Processor -- start 2024-05-22T21:13:35.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f550d200640 1 -- start start 2024-05-22T21:13:35.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.698+0000 7f6912000640 1 -- 172.21.15.161:0/1137407405 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690c0d0c40 con 0x7f690c06fad0 2024-05-22T21:13:35.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f550d200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55080b37b0 0x7f55080c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f550d200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55080d0bb0 0x7f55080ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f6903400640 1 -- 172.21.15.161:0/1137407405 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f69040ad440 con 0x7f690c06fad0 2024-05-22T21:13:35.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f550d200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5508160ad0 0x7f55080b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f550d200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550814f550 con 0x7f55080b37b0 2024-05-22T21:13:35.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f550d200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55080ca680 con 0x7f55080d0bb0 2024-05-22T21:13:35.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f550d200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550814f7a0 con 0x7f5508160ad0 2024-05-22T21:13:35.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f54ffe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55080d0bb0 0x7f55080ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f54ffe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55080d0bb0 0x7f55080ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:39720/0 (socket says 172.21.15.161:39720) 2024-05-22T21:13:35.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f54ffe00640 1 -- 172.21.15.161:0/1791956354 learned_addr learned my addr 172.21.15.161:0/1791956354 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:35.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f5506800640 1 --2- 172.21.15.161:0/1791956354 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5508160ad0 0x7f55080b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f5505e00640 1 --2- 172.21.15.161:0/1791956354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55080b37b0 0x7f55080c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f54ffe00640 1 -- 172.21.15.161:0/1791956354 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5508160ad0 msgr2=0x7f55080b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f54ffe00640 1 --2- 172.21.15.161:0/1791956354 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5508160ad0 0x7f55080b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f54ffe00640 1 -- 172.21.15.161:0/1791956354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55080b37b0 msgr2=0x7f55080c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f54ffe00640 1 --2- 172.21.15.161:0/1791956354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55080b37b0 0x7f55080c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f54ffe00640 1 -- 172.21.15.161:0/1791956354 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5500009b30 con 0x7f55080d0bb0 2024-05-22T21:13:35.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f6903400640 1 --2- 172.21.15.161:0/1137407405 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f68e8066690 0x7f68e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f6903400640 1 -- 172.21.15.161:0/1137407405 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f690411f5e0 con 0x7f690c06fad0 2024-05-22T21:13:35.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f54ffe00640 1 --2- 172.21.15.161:0/1791956354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55080d0bb0 0x7f55080ca050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f54f000c400 tx=0x7f54f000c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f6903e00640 1 --2- 172.21.15.161:0/1137407405 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f68e8066690 0x7f68e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.702+0000 7f6903e00640 1 --2- 172.21.15.161:0/1137407405 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f68e8066690 0x7f68e8068b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f690c166000 tx=0x7f68f4007c70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.706+0000 7f54fe000640 1 -- 172.21.15.161:0/1791956354 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f00052d0 con 0x7f55080d0bb0 2024-05-22T21:13:35.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.706+0000 7f54fe000640 1 -- 172.21.15.161:0/1791956354 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f0005430 con 0x7f55080d0bb0 2024-05-22T21:13:35.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.706+0000 7f550d200640 1 -- 172.21.15.161:0/1791956354 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550814fa80 con 0x7f55080d0bb0 2024-05-22T21:13:35.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.706+0000 7f6903400640 1 -- 172.21.15.161:0/1137407405 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f69040ec9e0 con 0x7f690c06fad0 2024-05-22T21:13:35.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.706+0000 7f54fe000640 1 -- 172.21.15.161:0/1791956354 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0003840 con 0x7f55080d0bb0 2024-05-22T21:13:35.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.706+0000 7f550d200640 1 -- 172.21.15.161:0/1791956354 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55080c72d0 con 0x7f55080d0bb0 2024-05-22T21:13:35.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.706+0000 7f550d200640 1 -- 172.21.15.161:0/1791956354 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5508073ca0 con 0x7f55080d0bb0 2024-05-22T21:13:35.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.710+0000 7f54fe000640 1 -- 172.21.15.161:0/1791956354 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f54f00039a0 con 0x7f55080d0bb0 2024-05-22T21:13:35.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.710+0000 7f54fe000640 1 --2- 172.21.15.161:0/1791956354 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f54e4066690 0x7f54e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.710+0000 7f54fe000640 1 -- 172.21.15.161:0/1791956354 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54f0083630 con 0x7f55080d0bb0 2024-05-22T21:13:35.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.710+0000 7f5505e00640 1 --2- 172.21.15.161:0/1791956354 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f54e4066690 0x7f54e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.710+0000 7f5505e00640 1 --2- 172.21.15.161:0/1791956354 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f54e4066690 0x7f54e4068b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009ac0 tx=0x7f54f8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.714+0000 7f54fe000640 1 -- 172.21.15.161:0/1791956354 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f54f0050ac0 con 0x7f55080d0bb0 2024-05-22T21:13:35.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.734+0000 7faf83e00640 1 Processor -- start 2024-05-22T21:13:35.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.734+0000 7faf83e00640 1 -- start start 2024-05-22T21:13:35.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.734+0000 7faf83e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7faf840afce0 0x7faf841661f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.734+0000 7faf83e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf8415fce0 con 0x7faf8415e860 2024-05-22T21:13:35.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.738+0000 7faf83e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf8415ec40 con 0x7faf840adba0 2024-05-22T21:13:35.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.738+0000 7faf83e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf840cd320 con 0x7faf840afce0 2024-05-22T21:13:35.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.738+0000 7faf82000640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7faf840adba0 0x7faf8415f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:60790/0 (socket says 172.21.15.161:60790) 2024-05-22T21:13:35.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.738+0000 7faf82000640 1 -- 172.21.15.161:0/2441936111 learned_addr learned my addr 172.21.15.161:0/2441936111 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:35.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.738+0000 7faf83400640 1 --2- 172.21.15.161:0/2441936111 >> v2:172.21.15.161:3300/0 conn(0x7faf840afce0 0x7faf841661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.738+0000 7faf81600640 1 -- 172.21.15.161:0/2441936111 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3713354861 0 0) 0x7faf8415ec40 con 0x7faf840adba0 2024-05-22T21:13:35.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.738+0000 7faf81600640 1 -- 172.21.15.161:0/2441936111 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf68003530 con 0x7faf840adba0 2024-05-22T21:13:35.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.738+0000 7faf81600640 1 -- 172.21.15.161:0/2441936111 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836783570 0 0) 0x7faf8415fce0 con 0x7faf8415e860 2024-05-22T21:13:35.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.738+0000 7faf83400640 1 -- 172.21.15.161:0/2441936111 >> v1:172.21.15.179:6789/0 conn(0x7faf840adba0 legacy=0x7faf8415f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.738+0000 7faf83400640 1 -- 172.21.15.161:0/2441936111 >> v1:172.21.15.161:6789/0 conn(0x7faf8415e860 legacy=0x7faf8415eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.738+0000 7faf83400640 1 -- 172.21.15.161:0/2441936111 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf840ccd50 con 0x7faf840afce0 2024-05-22T21:13:35.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.742+0000 7faf83400640 1 --2- 172.21.15.161:0/2441936111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf840afce0 0x7faf841661f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009d90 tx=0x7faf7c01c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bf818e2aaae82e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.742+0000 7faf81600640 1 -- 172.21.15.161:0/2441936111 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70001600 con 0x7faf840afce0 2024-05-22T21:13:35.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.742+0000 7faf81600640 1 -- 172.21.15.161:0/2441936111 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7c0a5640 con 0x7faf840afce0 2024-05-22T21:13:35.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.742+0000 7faf81600640 1 -- 172.21.15.161:0/2441936111 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c0a59b0 con 0x7faf840afce0 2024-05-22T21:13:35.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.746+0000 7faf83e00640 1 -- 172.21.15.161:0/2441936111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf840afce0 msgr2=0x7faf841661f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.746+0000 7faf83e00640 1 --2- 172.21.15.161:0/2441936111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf840afce0 0x7faf841661f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009d90 tx=0x7faf7c01c7c0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.746+0000 7faf83e00640 1 -- 172.21.15.161:0/2441936111 shutdown_connections 2024-05-22T21:13:35.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.746+0000 7faf83e00640 1 --2- 172.21.15.161:0/2441936111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf840afce0 0x7faf841661f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.746+0000 7faf83e00640 1 -- 172.21.15.161:0/2441936111 >> 172.21.15.161:0/2441936111 conn(0x7faf840d0dd0 msgr2=0x7faf840d11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:35.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.746+0000 7faf83e00640 1 -- 172.21.15.161:0/2441936111 shutdown_connections 2024-05-22T21:13:35.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.746+0000 7faf83e00640 1 -- 172.21.15.161:0/2441936111 wait complete. 2024-05-22T21:13:35.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.746+0000 7faf83e00640 1 Processor -- start 2024-05-22T21:13:35.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.746+0000 7faf83e00640 1 -- start start 2024-05-22T21:13:35.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf83e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf840adba0 0x7faf840b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf83e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faf840afce0 0x7faf8415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf83e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf8415e860 0x7faf840c3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf83e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf840b15e0 con 0x7faf840adba0 2024-05-22T21:13:35.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf83e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf84150f70 con 0x7faf8415e860 2024-05-22T21:13:35.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf83e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf840b0d40 con 0x7faf840afce0 2024-05-22T21:13:35.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7ff06e400640 1 Processor -- start 2024-05-22T21:13:35.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf82a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf840adba0 0x7faf840b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf82a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf840adba0 0x7faf840b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49370/0 (socket says 172.21.15.161:49370) 2024-05-22T21:13:35.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf82a00640 1 -- 172.21.15.161:0/3149958734 learned_addr learned my addr 172.21.15.161:0/3149958734 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:35.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7ff06e400640 1 -- start start 2024-05-22T21:13:35.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf82000640 1 --2- 172.21.15.161:0/3149958734 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faf840afce0 0x7faf8415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf83400640 1 --2- 172.21.15.161:0/3149958734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf8415e860 0x7faf840c3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf82a00640 1 -- 172.21.15.161:0/3149958734 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faf840afce0 msgr2=0x7faf8415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf82a00640 1 --2- 172.21.15.161:0/3149958734 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faf840afce0 0x7faf8415dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf82a00640 1 -- 172.21.15.161:0/3149958734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf8415e860 msgr2=0x7faf840c3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf82a00640 1 --2- 172.21.15.161:0/3149958734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf8415e860 0x7faf840c3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf82a00640 1 -- 172.21.15.161:0/3149958734 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7c009a70 con 0x7faf840adba0 2024-05-22T21:13:35.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf82a00640 1 --2- 172.21.15.161:0/3149958734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf840adba0 0x7faf840b0660 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faf7400c420 tx=0x7faf7400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf7aa00640 1 -- 172.21.15.161:0/3149958734 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf740054e0 con 0x7faf840adba0 2024-05-22T21:13:35.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf7aa00640 1 -- 172.21.15.161:0/3149958734 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf74003600 con 0x7faf840adba0 2024-05-22T21:13:35.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf7aa00640 1 -- 172.21.15.161:0/3149958734 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf74003990 con 0x7faf840adba0 2024-05-22T21:13:35.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7ff06e400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff068161840 0x7ff0680cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf83e00640 1 -- 172.21.15.161:0/3149958734 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf840b1020 con 0x7faf840adba0 2024-05-22T21:13:35.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7faf83e00640 1 -- 172.21.15.161:0/3149958734 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf840c43c0 con 0x7faf840adba0 2024-05-22T21:13:35.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7ff06e400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff0680c4670 0x7ff0680b0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7ff06e400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff068161f70 con 0x7ff0680b3630 2024-05-22T21:13:35.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7ff06e400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0680b0870 con 0x7ff068161840 2024-05-22T21:13:35.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7ff06e400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff068162250 con 0x7ff0680c4670 2024-05-22T21:13:35.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7ff067e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff0680c4670 0x7ff0680b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7ff067e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff0680c4670 0x7ff0680b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:39740/0 (socket says 172.21.15.161:39740) 2024-05-22T21:13:35.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7ff067e00640 1 -- 172.21.15.161:0/2815894306 learned_addr learned my addr 172.21.15.161:0/2815894306 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:35.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.750+0000 7f96b2000640 1 Processor -- start 2024-05-22T21:13:35.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7f96b2000640 1 -- start start 2024-05-22T21:13:35.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7ff067e00640 1 -- 172.21.15.161:0/2815894306 >> v2:172.21.15.161:3300/0 conn(0x7ff068161840 msgr2=0x7ff0680cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:35.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7ff067e00640 1 --2- 172.21.15.161:0/2815894306 >> v2:172.21.15.161:3300/0 conn(0x7ff068161840 0x7ff0680cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7ff067e00640 1 -- 172.21.15.161:0/2815894306 >> v1:172.21.15.179:6789/0 conn(0x7ff0680b3630 legacy=0x7ff0680c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:35.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7ff067e00640 1 -- 172.21.15.161:0/2815894306 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff068162530 con 0x7ff0680c4670 2024-05-22T21:13:35.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7faf83e00640 1 -- 172.21.15.161:0/3149958734 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf840676a0 con 0x7faf840adba0 2024-05-22T21:13:35.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7f96b2000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96ac0b2210 0x7f96ac0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7f96b2000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f96ac0b25f0 0x7f96ac0cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7f96b2000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ac0cea50 con 0x7f96ac0c6470 2024-05-22T21:13:35.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7f96b2000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ac0ced30 con 0x7f96ac0b25f0 2024-05-22T21:13:35.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7f96b2000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac0cf010 con 0x7f96ac0b2210 2024-05-22T21:13:35.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7f96ab400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96ac0b2210 0x7f96ac0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7f96ab400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96ac0b2210 0x7f96ac0d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:53578/0 (socket says 172.21.15.161:53578) 2024-05-22T21:13:35.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7f96ab400640 1 -- 172.21.15.161:0/2694973522 learned_addr learned my addr 172.21.15.161:0/2694973522 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:35.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7f96aaa00640 1 --2- 172.21.15.161:0/2694973522 >> v2:172.21.15.161:3300/0 conn(0x7f96ac0b25f0 0x7f96ac0cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7f96ab400640 1 -- 172.21.15.161:0/2694973522 >> v2:172.21.15.161:3300/0 conn(0x7f96ac0b25f0 msgr2=0x7f96ac0cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:35.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7f96ab400640 1 --2- 172.21.15.161:0/2694973522 >> v2:172.21.15.161:3300/0 conn(0x7f96ac0b25f0 0x7f96ac0cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7f96ab400640 1 -- 172.21.15.161:0/2694973522 >> v1:172.21.15.179:6789/0 conn(0x7f96ac0c6470 legacy=0x7f96ac0d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:35.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7f96ab400640 1 -- 172.21.15.161:0/2694973522 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ac0cf2f0 con 0x7f96ac0b2210 2024-05-22T21:13:35.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7f96aaa00640 1 --2- 172.21.15.161:0/2694973522 >> v2:172.21.15.161:3300/0 conn(0x7f96ac0b25f0 0x7f96ac0cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:35.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.754+0000 7faf7aa00640 1 -- 172.21.15.161:0/3149958734 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faf74003af0 con 0x7faf840adba0 2024-05-22T21:13:35.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.758+0000 7faf7aa00640 1 --2- 172.21.15.161:0/3149958734 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faf64066690 0x7faf64068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.758+0000 7faf7aa00640 1 -- 172.21.15.161:0/3149958734 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf74083bf0 con 0x7faf840adba0 2024-05-22T21:13:35.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.758+0000 7faf82000640 1 --2- 172.21.15.161:0/3149958734 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faf64066690 0x7faf64068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.758+0000 7ff067e00640 1 --2- 172.21.15.161:0/2815894306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0680c4670 0x7ff0680b0330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff0680721b0 tx=0x7ff06001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e06b6917d5d8a99 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.758+0000 7ff066000640 1 -- 172.21.15.161:0/2815894306 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0600a5800 con 0x7ff0680c4670 2024-05-22T21:13:35.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.758+0000 7ff066000640 1 -- 172.21.15.161:0/2815894306 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0600a5960 con 0x7ff0680c4670 2024-05-22T21:13:35.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.758+0000 7ff066000640 1 -- 172.21.15.161:0/2815894306 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0600a5cf0 con 0x7ff0680c4670 2024-05-22T21:13:35.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.758+0000 7faf82000640 1 --2- 172.21.15.161:0/3149958734 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faf64066690 0x7faf64068b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faf840cae80 tx=0x7faf700018c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.758+0000 7ff06e400640 1 -- 172.21.15.161:0/2815894306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0680c4670 msgr2=0x7ff0680b0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.758+0000 7ff06e400640 1 --2- 172.21.15.161:0/2815894306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0680c4670 0x7ff0680b0330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff0680721b0 tx=0x7ff06001c920 comp rx=0 tx=0).stop 2024-05-22T21:13:35.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7ff06e400640 1 -- 172.21.15.161:0/2815894306 shutdown_connections 2024-05-22T21:13:35.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7ff06e400640 1 --2- 172.21.15.161:0/2815894306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0680c4670 0x7ff0680b0330 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7ff06e400640 1 --2- 172.21.15.161:0/2815894306 >> v2:172.21.15.161:3300/0 conn(0x7ff068161840 0x7ff0680cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7ff06e400640 1 -- 172.21.15.161:0/2815894306 >> 172.21.15.161:0/2815894306 conn(0x7ff0680b2a80 msgr2=0x7ff0680b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:35.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7ff06e400640 1 -- 172.21.15.161:0/2815894306 shutdown_connections 2024-05-22T21:13:35.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7ff06e400640 1 -- 172.21.15.161:0/2815894306 wait complete. 2024-05-22T21:13:35.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7f96ab400640 1 --2- 172.21.15.161:0/2694973522 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96ac0b2210 0x7f96ac0d38a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009e50 tx=0x7f96a401c9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95e72a199f5f9d2d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7f96aa000640 1 -- 172.21.15.161:0/2694973522 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a40a5890 con 0x7f96ac0b2210 2024-05-22T21:13:35.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7f96aa000640 1 -- 172.21.15.161:0/2694973522 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a40a59f0 con 0x7f96ac0b2210 2024-05-22T21:13:35.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7f96aa000640 1 -- 172.21.15.161:0/2694973522 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a40a5d80 con 0x7f96ac0b2210 2024-05-22T21:13:35.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7f96b2000640 1 -- 172.21.15.161:0/2694973522 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96ac0b2210 msgr2=0x7f96ac0d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7f96b2000640 1 --2- 172.21.15.161:0/2694973522 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96ac0b2210 0x7f96ac0d38a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009e50 tx=0x7f96a401c9b0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7f96b2000640 1 -- 172.21.15.161:0/2694973522 shutdown_connections 2024-05-22T21:13:35.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7f96b2000640 1 --2- 172.21.15.161:0/2694973522 >> v2:172.21.15.161:3300/0 conn(0x7f96ac0b25f0 0x7f96ac0cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7f96b2000640 1 --2- 172.21.15.161:0/2694973522 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96ac0b2210 0x7f96ac0d38a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7f96b2000640 1 -- 172.21.15.161:0/2694973522 >> 172.21.15.161:0/2694973522 conn(0x7f96ac09c470 msgr2=0x7f96ac0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:35.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7f96b2000640 1 -- 172.21.15.161:0/2694973522 shutdown_connections 2024-05-22T21:13:35.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7ff06e400640 1 Processor -- start 2024-05-22T21:13:35.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7f96b2000640 1 -- 172.21.15.161:0/2694973522 wait complete. 2024-05-22T21:13:35.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7ff06e400640 1 -- start start 2024-05-22T21:13:35.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.762+0000 7faf7aa00640 1 -- 172.21.15.161:0/3149958734 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7faf74050fd0 con 0x7faf840adba0 2024-05-22T21:13:35.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff06e400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0680b3630 0x7ff068155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff06e400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0680c4670 0x7ff06815ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff06e400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff068161840 0x7ff06815c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff06e400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff068155550 con 0x7ff0680b3630 2024-05-22T21:13:35.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff06e400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff068161100 con 0x7ff0680c4670 2024-05-22T21:13:35.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff06e400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff068157530 con 0x7ff068161840 2024-05-22T21:13:35.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff066a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0680c4670 0x7ff06815ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff066a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0680c4670 0x7ff06815ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:39750/0 (socket says 172.21.15.161:39750) 2024-05-22T21:13:35.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff066a00640 1 -- 172.21.15.161:0/39104945 learned_addr learned my addr 172.21.15.161:0/39104945 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:35.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff067400640 1 --2- 172.21.15.161:0/39104945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0680b3630 0x7ff068155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7f96b2000640 1 Processor -- start 2024-05-22T21:13:35.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff067e00640 1 --2- 172.21.15.161:0/39104945 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff068161840 0x7ff06815c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7f96b2000640 1 -- start start 2024-05-22T21:13:35.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff066a00640 1 -- 172.21.15.161:0/39104945 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff068161840 msgr2=0x7ff06815c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff066a00640 1 --2- 172.21.15.161:0/39104945 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff068161840 0x7ff06815c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff066a00640 1 -- 172.21.15.161:0/39104945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0680b3630 msgr2=0x7ff068155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff066a00640 1 --2- 172.21.15.161:0/39104945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0680b3630 0x7ff068155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff066a00640 1 -- 172.21.15.161:0/39104945 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff060009b30 con 0x7ff0680c4670 2024-05-22T21:13:35.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff066a00640 1 --2- 172.21.15.161:0/39104945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0680c4670 0x7ff06815ee40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff05400c410 tx=0x7ff05400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff05f400640 1 -- 172.21.15.161:0/39104945 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054011790 con 0x7ff0680c4670 2024-05-22T21:13:35.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7f96b2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96ac0b2210 0x7f96ac0b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7f96b2000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96ac0b25f0 0x7f96ac1494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7f96b2000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96ac0c6470 0x7f96ac157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7f96b2000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac0b62b0 con 0x7f96ac0b2210 2024-05-22T21:13:35.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7f96b2000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac158ea0 con 0x7f96ac0b25f0 2024-05-22T21:13:35.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7f96b2000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac149a30 con 0x7f96ac0c6470 2024-05-22T21:13:35.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff05f400640 1 -- 172.21.15.161:0/39104945 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0540118f0 con 0x7ff0680c4670 2024-05-22T21:13:35.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff05f400640 1 -- 172.21.15.161:0/39104945 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054011ce0 con 0x7ff0680c4670 2024-05-22T21:13:35.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff06e400640 1 -- 172.21.15.161:0/39104945 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff068157810 con 0x7ff0680c4670 2024-05-22T21:13:35.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7ff06e400640 1 -- 172.21.15.161:0/39104945 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff068157dd0 con 0x7ff0680c4670 2024-05-22T21:13:35.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7f96aaa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96ac0b25f0 0x7f96ac1494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7f96aaa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96ac0b25f0 0x7f96ac1494f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:39754/0 (socket says 172.21.15.161:39754) 2024-05-22T21:13:35.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7f96aaa00640 1 -- 172.21.15.161:0/2332671737 learned_addr learned my addr 172.21.15.161:0/2332671737 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:35.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7f96abe00640 1 --2- 172.21.15.161:0/2332671737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96ac0c6470 0x7f96ac157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7f96ab400640 1 --2- 172.21.15.161:0/2332671737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96ac0b2210 0x7f96ac0b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7f96aaa00640 1 -- 172.21.15.161:0/2332671737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96ac0c6470 msgr2=0x7f96ac157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7f96aaa00640 1 --2- 172.21.15.161:0/2332671737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96ac0c6470 0x7f96ac157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.766+0000 7f96aaa00640 1 -- 172.21.15.161:0/2332671737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96ac0b2210 msgr2=0x7f96ac0b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.770+0000 7ff06e400640 1 -- 172.21.15.161:0/39104945 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff068073ca0 con 0x7ff0680c4670 2024-05-22T21:13:35.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.770+0000 7ff05f400640 1 -- 172.21.15.161:0/39104945 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff05401c460 con 0x7ff0680c4670 2024-05-22T21:13:35.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.770+0000 7f96aaa00640 1 --2- 172.21.15.161:0/2332671737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96ac0b2210 0x7f96ac0b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.770+0000 7f96aaa00640 1 -- 172.21.15.161:0/2332671737 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9694009420 con 0x7f96ac0b25f0 2024-05-22T21:13:35.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.770+0000 7f96ab400640 1 --2- 172.21.15.161:0/2332671737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96ac0b2210 0x7f96ac0b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:35.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.770+0000 7f96aaa00640 1 --2- 172.21.15.161:0/2332671737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96ac0b25f0 0x7f96ac1494f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f969400ebe0 tx=0x7f969400c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.770+0000 7ff05f400640 1 --2- 172.21.15.161:0/39104945 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff048066690 0x7ff048068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.770+0000 7ff05f400640 1 -- 172.21.15.161:0/39104945 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff054083df0 con 0x7ff0680c4670 2024-05-22T21:13:35.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.770+0000 7ff067400640 1 --2- 172.21.15.161:0/39104945 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff048066690 0x7ff048068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.774+0000 7ff067400640 1 --2- 172.21.15.161:0/39104945 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff048066690 0x7ff048068b50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff06809f070 tx=0x7ff058006f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.774+0000 7f96a3400640 1 -- 172.21.15.161:0/2332671737 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969400eee0 con 0x7f96ac0b25f0 2024-05-22T21:13:35.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.774+0000 7f96a3400640 1 -- 172.21.15.161:0/2332671737 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969400cbd0 con 0x7f96ac0b25f0 2024-05-22T21:13:35.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.774+0000 7f96b2000640 1 -- 172.21.15.161:0/2332671737 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a4009b30 con 0x7f96ac0b25f0 2024-05-22T21:13:35.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.774+0000 7f96a3400640 1 -- 172.21.15.161:0/2332671737 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694013470 con 0x7f96ac0b25f0 2024-05-22T21:13:35.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.774+0000 7f96b2000640 1 -- 172.21.15.161:0/2332671737 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ac0b67a0 con 0x7f96ac0b25f0 2024-05-22T21:13:35.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.774+0000 7ff05f400640 1 -- 172.21.15.161:0/39104945 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff054051170 con 0x7ff0680c4670 2024-05-22T21:13:35.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.778+0000 7f96b2000640 1 -- 172.21.15.161:0/2332671737 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ac067220 con 0x7f96ac0b25f0 2024-05-22T21:13:35.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.778+0000 7f96a3400640 1 -- 172.21.15.161:0/2332671737 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f96940135d0 con 0x7f96ac0b25f0 2024-05-22T21:13:35.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.778+0000 7f96a3400640 1 --2- 172.21.15.161:0/2332671737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f96880666c0 0x7f9688068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:35.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.778+0000 7f96a3400640 1 -- 172.21.15.161:0/2332671737 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9694084550 con 0x7f96ac0b25f0 2024-05-22T21:13:35.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.782+0000 7f96ab400640 1 --2- 172.21.15.161:0/2332671737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f96880666c0 0x7f9688068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:35.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.782+0000 7f96ab400640 1 --2- 172.21.15.161:0/2332671737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f96880666c0 0x7f9688068b80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f96a40ae000 tx=0x7f96a40a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:35.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.782+0000 7f96a3400640 1 -- 172.21.15.161:0/2332671737 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f96940517c0 con 0x7f96ac0b25f0 2024-05-22T21:13:35.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.890+0000 7f6912000640 1 -- 172.21.15.161:0/1137407405 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f690c073ca0 con 0x7f690c06fad0 2024-05-22T21:13:35.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.890+0000 7f6903400640 1 -- 172.21.15.161:0/1137407405 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f69040f0000 con 0x7f690c06fad0 2024-05-22T21:13:35.892 INFO:teuthology.orchestra.run.smithi161.stdout:55834574854 2024-05-22T21:13:35.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.890+0000 7f6912000640 1 -- 172.21.15.161:0/1137407405 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f68e8066690 msgr2=0x7f68e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.890+0000 7f6912000640 1 --2- 172.21.15.161:0/1137407405 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f68e8066690 0x7f68e8068b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f690c166000 tx=0x7f68f4007c70 comp rx=0 tx=0).stop 2024-05-22T21:13:35.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.890+0000 7f6912000640 1 -- 172.21.15.161:0/1137407405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c06fad0 msgr2=0x7f690c0c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.890+0000 7f6912000640 1 --2- 172.21.15.161:0/1137407405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c06fad0 0x7f690c0c4830 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f690401aea0 tx=0x7f6904019590 comp rx=0 tx=0).stop 2024-05-22T21:13:35.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.890+0000 7f6912000640 1 -- 172.21.15.161:0/1137407405 shutdown_connections 2024-05-22T21:13:35.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.890+0000 7f6912000640 1 --2- 172.21.15.161:0/1137407405 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f68e8066690 0x7f68e8068b50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.890+0000 7f6912000640 1 --2- 172.21.15.161:0/1137407405 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f690c15e5f0 0x7f690c0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.890+0000 7f6912000640 1 --2- 172.21.15.161:0/1137407405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f690c0cf000 0x7f690c0b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.890+0000 7f6912000640 1 --2- 172.21.15.161:0/1137407405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f690c06fad0 0x7f690c0c4830 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.890+0000 7f6912000640 1 -- 172.21.15.161:0/1137407405 >> 172.21.15.161:0/1137407405 conn(0x7f690c0b2a80 msgr2=0x7f690c0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:35.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.890+0000 7f6912000640 1 -- 172.21.15.161:0/1137407405 shutdown_connections 2024-05-22T21:13:35.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.890+0000 7f6912000640 1 -- 172.21.15.161:0/1137407405 wait complete. 2024-05-22T21:13:35.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.906+0000 7f550d200640 1 -- 172.21.15.161:0/1791956354 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5508067220 con 0x7f55080d0bb0 2024-05-22T21:13:35.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.906+0000 7f54fe000640 1 -- 172.21.15.161:0/1791956354 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f54f00540e0 con 0x7f55080d0bb0 2024-05-22T21:13:35.911 INFO:teuthology.orchestra.run.smithi161.stdout:55834574854 2024-05-22T21:13:35.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.910+0000 7f550d200640 1 -- 172.21.15.161:0/1791956354 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f54e4066690 msgr2=0x7f54e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.910+0000 7f550d200640 1 --2- 172.21.15.161:0/1791956354 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f54e4066690 0x7f54e4068b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009ac0 tx=0x7f54f8009190 comp rx=0 tx=0).stop 2024-05-22T21:13:35.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.910+0000 7f550d200640 1 -- 172.21.15.161:0/1791956354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55080d0bb0 msgr2=0x7f55080ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:35.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.910+0000 7f550d200640 1 --2- 172.21.15.161:0/1791956354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55080d0bb0 0x7f55080ca050 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f54f000c400 tx=0x7f54f000c8c0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.910+0000 7f550d200640 1 -- 172.21.15.161:0/1791956354 shutdown_connections 2024-05-22T21:13:35.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.910+0000 7f550d200640 1 --2- 172.21.15.161:0/1791956354 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f54e4066690 0x7f54e4068b50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.910+0000 7f550d200640 1 --2- 172.21.15.161:0/1791956354 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5508160ad0 0x7f55080b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.910+0000 7f550d200640 1 --2- 172.21.15.161:0/1791956354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55080d0bb0 0x7f55080ca050 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.910+0000 7f550d200640 1 --2- 172.21.15.161:0/1791956354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55080b37b0 0x7f55080c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:35.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.910+0000 7f550d200640 1 -- 172.21.15.161:0/1791956354 >> 172.21.15.161:0/1791956354 conn(0x7f55080b2a80 msgr2=0x7f55080aee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:35.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.910+0000 7f550d200640 1 -- 172.21.15.161:0/1791956354 shutdown_connections 2024-05-22T21:13:35.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.910+0000 7f550d200640 1 -- 172.21.15.161:0/1791956354 wait complete. 2024-05-22T21:13:35.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-05-22T21:13:35.916 DEBUG:teuthology.parallel:result is None 2024-05-22T21:13:35.935 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-05-22T21:13:35.935 DEBUG:teuthology.parallel:result is None 2024-05-22T21:13:36.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.998+0000 7f3292c00640 1 -- 172.21.15.161:0/2704597109 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f32640057e0 con 0x7f328c161840 2024-05-22T21:13:36.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:35.998+0000 7f3283e00640 1 -- 172.21.15.161:0/2704597109 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3284018090 con 0x7f328c161840 2024-05-22T21:13:36.001 INFO:teuthology.orchestra.run.smithi161.stdout:55834574854 2024-05-22T21:13:36.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.002+0000 7f3292c00640 1 -- 172.21.15.161:0/2704597109 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f326c066690 msgr2=0x7f326c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.002+0000 7f3292c00640 1 --2- 172.21.15.161:0/2704597109 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f326c066690 0x7f326c068b50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f327c0ae000 tx=0x7f327c0a6040 comp rx=0 tx=0).stop 2024-05-22T21:13:36.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.002+0000 7f3292c00640 1 -- 172.21.15.161:0/2704597109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c161840 msgr2=0x7f328c0ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.002+0000 7f3292c00640 1 --2- 172.21.15.161:0/2704597109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c161840 0x7f328c0ccd00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f328400efc0 tx=0x7f328400c490 comp rx=0 tx=0).stop 2024-05-22T21:13:36.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.002+0000 7f3292c00640 1 -- 172.21.15.161:0/2704597109 shutdown_connections 2024-05-22T21:13:36.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.002+0000 7f3292c00640 1 --2- 172.21.15.161:0/2704597109 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f326c066690 0x7f326c068b50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.002+0000 7f3292c00640 1 --2- 172.21.15.161:0/2704597109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c161840 0x7f328c0ccd00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.002+0000 7f3292c00640 1 --2- 172.21.15.161:0/2704597109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f328c0c4670 0x7f328c0cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.002+0000 7f3292c00640 1 --2- 172.21.15.161:0/2704597109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f328c0b3630 0x7f328c154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.002+0000 7f3292c00640 1 -- 172.21.15.161:0/2704597109 >> 172.21.15.161:0/2704597109 conn(0x7f328c0b2a80 msgr2=0x7f328c157540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:36.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.002+0000 7f3292c00640 1 -- 172.21.15.161:0/2704597109 shutdown_connections 2024-05-22T21:13:36.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.002+0000 7f3292c00640 1 -- 172.21.15.161:0/2704597109 wait complete. 2024-05-22T21:13:36.024 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-05-22T21:13:36.024 DEBUG:teuthology.parallel:result is None 2024-05-22T21:13:36.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.046+0000 7faf83e00640 1 -- 172.21.15.161:0/3149958734 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faf840b6d50 con 0x7faf840adba0 2024-05-22T21:13:36.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.046+0000 7faf7aa00640 1 -- 172.21.15.161:0/3149958734 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faf74017070 con 0x7faf840adba0 2024-05-22T21:13:36.049 INFO:teuthology.orchestra.run.smithi161.stdout:55834574854 2024-05-22T21:13:36.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.046+0000 7faf83e00640 1 -- 172.21.15.161:0/3149958734 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faf64066690 msgr2=0x7faf64068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.046+0000 7faf83e00640 1 --2- 172.21.15.161:0/3149958734 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faf64066690 0x7faf64068b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faf840cae80 tx=0x7faf700018c0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.046+0000 7faf83e00640 1 -- 172.21.15.161:0/3149958734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf840adba0 msgr2=0x7faf840b0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.046+0000 7faf83e00640 1 --2- 172.21.15.161:0/3149958734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf840adba0 0x7faf840b0660 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faf7400c420 tx=0x7faf7400c8e0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.046+0000 7faf83e00640 1 -- 172.21.15.161:0/3149958734 shutdown_connections 2024-05-22T21:13:36.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.046+0000 7faf83e00640 1 --2- 172.21.15.161:0/3149958734 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faf64066690 0x7faf64068b50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.046+0000 7faf83e00640 1 --2- 172.21.15.161:0/3149958734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf8415e860 0x7faf840c3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.046+0000 7faf83e00640 1 --2- 172.21.15.161:0/3149958734 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faf840afce0 0x7faf8415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.046+0000 7faf83e00640 1 --2- 172.21.15.161:0/3149958734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf840adba0 0x7faf840b0660 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.046+0000 7faf83e00640 1 -- 172.21.15.161:0/3149958734 >> 172.21.15.161:0/3149958734 conn(0x7faf840d0dd0 msgr2=0x7faf84155e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:36.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.050+0000 7faf83e00640 1 -- 172.21.15.161:0/3149958734 shutdown_connections 2024-05-22T21:13:36.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.050+0000 7faf83e00640 1 -- 172.21.15.161:0/3149958734 wait complete. 2024-05-22T21:13:36.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-05-22T21:13:36.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.066+0000 7ff06e400640 1 -- 172.21.15.161:0/39104945 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff068067220 con 0x7ff0680c4670 2024-05-22T21:13:36.072 DEBUG:teuthology.parallel:result is None 2024-05-22T21:13:36.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.070+0000 7ff05f400640 1 -- 172.21.15.161:0/39104945 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff054054790 con 0x7ff0680c4670 2024-05-22T21:13:36.072 INFO:teuthology.orchestra.run.smithi161.stdout:55834574854 2024-05-22T21:13:36.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.070+0000 7ff06e400640 1 -- 172.21.15.161:0/39104945 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff048066690 msgr2=0x7ff048068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.070+0000 7ff06e400640 1 --2- 172.21.15.161:0/39104945 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff048066690 0x7ff048068b50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff06809f070 tx=0x7ff058006f90 comp rx=0 tx=0).stop 2024-05-22T21:13:36.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.070+0000 7ff06e400640 1 -- 172.21.15.161:0/39104945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0680c4670 msgr2=0x7ff06815ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.070+0000 7ff06e400640 1 --2- 172.21.15.161:0/39104945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0680c4670 0x7ff06815ee40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff05400c410 tx=0x7ff05400c8d0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.070+0000 7ff06e400640 1 -- 172.21.15.161:0/39104945 shutdown_connections 2024-05-22T21:13:36.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.070+0000 7ff06e400640 1 --2- 172.21.15.161:0/39104945 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff048066690 0x7ff048068b50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.070+0000 7ff06e400640 1 --2- 172.21.15.161:0/39104945 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff068161840 0x7ff06815c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.070+0000 7ff06e400640 1 --2- 172.21.15.161:0/39104945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0680c4670 0x7ff06815ee40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.070+0000 7ff06e400640 1 --2- 172.21.15.161:0/39104945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0680b3630 0x7ff068155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.070+0000 7ff06e400640 1 -- 172.21.15.161:0/39104945 >> 172.21.15.161:0/39104945 conn(0x7ff0680b2a80 msgr2=0x7ff0680cd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:36.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.070+0000 7ff06e400640 1 -- 172.21.15.161:0/39104945 shutdown_connections 2024-05-22T21:13:36.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.070+0000 7ff06e400640 1 -- 172.21.15.161:0/39104945 wait complete. 2024-05-22T21:13:36.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.074+0000 7f96b2000640 1 -- 172.21.15.161:0/2332671737 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f96ac06c590 con 0x7f96ac0b25f0 2024-05-22T21:13:36.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.074+0000 7f96a3400640 1 -- 172.21.15.161:0/2332671737 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9694054de0 con 0x7f96ac0b25f0 2024-05-22T21:13:36.078 INFO:teuthology.orchestra.run.smithi161.stdout:55834574854 2024-05-22T21:13:36.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.078+0000 7f96b2000640 1 -- 172.21.15.161:0/2332671737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f96880666c0 msgr2=0x7f9688068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.078+0000 7f96b2000640 1 --2- 172.21.15.161:0/2332671737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f96880666c0 0x7f9688068b80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f96a40ae000 tx=0x7f96a40a7070 comp rx=0 tx=0).stop 2024-05-22T21:13:36.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.078+0000 7f96b2000640 1 -- 172.21.15.161:0/2332671737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96ac0b25f0 msgr2=0x7f96ac1494f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.078+0000 7f96b2000640 1 --2- 172.21.15.161:0/2332671737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96ac0b25f0 0x7f96ac1494f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f969400ebe0 tx=0x7f969400c520 comp rx=0 tx=0).stop 2024-05-22T21:13:36.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.078+0000 7f96b2000640 1 -- 172.21.15.161:0/2332671737 shutdown_connections 2024-05-22T21:13:36.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.078+0000 7f96b2000640 1 --2- 172.21.15.161:0/2332671737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f96880666c0 0x7f9688068b80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.078+0000 7f96b2000640 1 --2- 172.21.15.161:0/2332671737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96ac0c6470 0x7f96ac157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.078+0000 7f96b2000640 1 --2- 172.21.15.161:0/2332671737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96ac0b25f0 0x7f96ac1494f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.078+0000 7f96b2000640 1 --2- 172.21.15.161:0/2332671737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96ac0b2210 0x7f96ac0b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.078+0000 7f96b2000640 1 -- 172.21.15.161:0/2332671737 >> 172.21.15.161:0/2332671737 conn(0x7f96ac09c470 msgr2=0x7f96ac0afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:36.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.078+0000 7f96b2000640 1 -- 172.21.15.161:0/2332671737 shutdown_connections 2024-05-22T21:13:36.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.078+0000 7f96b2000640 1 -- 172.21.15.161:0/2332671737 wait complete. 2024-05-22T21:13:36.097 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-05-22T21:13:36.097 DEBUG:teuthology.parallel:result is None 2024-05-22T21:13:36.101 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-05-22T21:13:36.101 DEBUG:teuthology.parallel:result is None 2024-05-22T21:13:36.101 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-22T21:13:36.101 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:13:36.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3640c00640 1 Processor -- start 2024-05-22T21:13:36.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3640c00640 1 -- start start 2024-05-22T21:13:36.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3640c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f363c15dd50 0x7f363c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:36.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3640c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f363c15e130 0x7f363c15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:36.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3640c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f363c15d410 0x7f363c15a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:36.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3640c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c071670 con 0x7f363c15dd50 2024-05-22T21:13:36.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3640c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c15d8a0 con 0x7f363c15e130 2024-05-22T21:13:36.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3640c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c06de40 con 0x7f363c15d410 2024-05-22T21:13:36.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3639e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f363c15dd50 0x7f363c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:36.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3639e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f363c15dd50 0x7f363c06d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54624/0 (socket says 172.21.15.161:54624) 2024-05-22T21:13:36.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3639e00640 1 -- 172.21.15.161:0/1511018410 learned_addr learned my addr 172.21.15.161:0/1511018410 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:36.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f363a800640 1 --2- 172.21.15.161:0/1511018410 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f363c15d410 0x7f363c15a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:36.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3631400640 1 --2- 172.21.15.161:0/1511018410 >> v2:172.21.15.179:3300/0 conn(0x7f363c15e130 0x7f363c15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:36.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3639e00640 1 -- 172.21.15.161:0/1511018410 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f363c15d410 msgr2=0x7f363c15a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:36.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3639e00640 1 --2- 172.21.15.161:0/1511018410 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f363c15d410 0x7f363c15a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3639e00640 1 -- 172.21.15.161:0/1511018410 >> v2:172.21.15.179:3300/0 conn(0x7f363c15e130 msgr2=0x7f363c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:36.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3639e00640 1 --2- 172.21.15.161:0/1511018410 >> v2:172.21.15.179:3300/0 conn(0x7f363c15e130 0x7f363c15ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.274+0000 7f3639e00640 1 -- 172.21.15.161:0/1511018410 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c070a60 con 0x7f363c15dd50 2024-05-22T21:13:36.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.278+0000 7f3639e00640 1 --2- 172.21.15.161:0/1511018410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f363c15dd50 0x7f363c06d900 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f363c06c7c0 tx=0x7f362c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ca22fad2569fa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:36.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.278+0000 7f3639400640 1 -- 172.21.15.161:0/1511018410 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c0aa610 con 0x7f363c15dd50 2024-05-22T21:13:36.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.278+0000 7f3639400640 1 -- 172.21.15.161:0/1511018410 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c0a6040 con 0x7f363c15dd50 2024-05-22T21:13:36.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.278+0000 7f3639400640 1 -- 172.21.15.161:0/1511018410 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c0aa990 con 0x7f363c15dd50 2024-05-22T21:13:36.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.278+0000 7f3640c00640 1 -- 172.21.15.161:0/1511018410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f363c15dd50 msgr2=0x7f363c06d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.278+0000 7f3640c00640 1 --2- 172.21.15.161:0/1511018410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f363c15dd50 0x7f363c06d900 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f363c06c7c0 tx=0x7f362c01c4f0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.278+0000 7f3640c00640 1 -- 172.21.15.161:0/1511018410 shutdown_connections 2024-05-22T21:13:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.278+0000 7f3640c00640 1 --2- 172.21.15.161:0/1511018410 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f363c15d410 0x7f363c15a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.278+0000 7f3640c00640 1 --2- 172.21.15.161:0/1511018410 >> v2:172.21.15.179:3300/0 conn(0x7f363c15e130 0x7f363c15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.278+0000 7f3640c00640 1 --2- 172.21.15.161:0/1511018410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f363c15dd50 0x7f363c06d900 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.278+0000 7f3640c00640 1 -- 172.21.15.161:0/1511018410 >> 172.21.15.161:0/1511018410 conn(0x7f363c0b0bc0 msgr2=0x7f363c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.278+0000 7f3640c00640 1 -- 172.21.15.161:0/1511018410 shutdown_connections 2024-05-22T21:13:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.278+0000 7f3640c00640 1 -- 172.21.15.161:0/1511018410 wait complete. 2024-05-22T21:13:36.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f3640c00640 1 Processor -- start 2024-05-22T21:13:36.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f3640c00640 1 -- start start 2024-05-22T21:13:36.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f3640c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f363c15d410 0x7f363c14fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:36.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f3640c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f363c15dd50 0x7f363c0b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:36.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f3640c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f363c15e130 0x7f363c14e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:36.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f3640c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c0cc840 con 0x7f363c15e130 2024-05-22T21:13:36.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f3640c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c14f6b0 con 0x7f363c15dd50 2024-05-22T21:13:36.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f3640c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c150430 con 0x7f363c15d410 2024-05-22T21:13:36.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f363a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f363c15e130 0x7f363c14e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:36.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f363a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f363c15e130 0x7f363c14e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54628/0 (socket says 172.21.15.161:54628) 2024-05-22T21:13:36.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f363a800640 1 -- 172.21.15.161:0/1670796022 learned_addr learned my addr 172.21.15.161:0/1670796022 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:36.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f3631400640 1 --2- 172.21.15.161:0/1670796022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f363c15dd50 0x7f363c0b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:36.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f3639e00640 1 --2- 172.21.15.161:0/1670796022 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f363c15d410 0x7f363c14fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:36.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f363a800640 1 -- 172.21.15.161:0/1670796022 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f363c15d410 msgr2=0x7f363c14fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f363a800640 1 --2- 172.21.15.161:0/1670796022 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f363c15d410 0x7f363c14fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f363a800640 1 -- 172.21.15.161:0/1670796022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f363c15dd50 msgr2=0x7f363c0b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f363a800640 1 --2- 172.21.15.161:0/1670796022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f363c15dd50 0x7f363c0b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f363a800640 1 -- 172.21.15.161:0/1670796022 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c0094d0 con 0x7f363c15e130 2024-05-22T21:13:36.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f363a800640 1 --2- 172.21.15.161:0/1670796022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f363c15e130 0x7f363c14e4c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3634007e70 tx=0x7f363400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:36.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f3632a00640 1 -- 172.21.15.161:0/1670796022 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634014620 con 0x7f363c15e130 2024-05-22T21:13:36.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f3632a00640 1 -- 172.21.15.161:0/1670796022 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36340147a0 con 0x7f363c15e130 2024-05-22T21:13:36.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f3632a00640 1 -- 172.21.15.161:0/1670796022 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634014b50 con 0x7f363c15e130 2024-05-22T21:13:36.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f3640c00640 1 -- 172.21.15.161:0/1670796022 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363c143a60 con 0x7f363c15e130 2024-05-22T21:13:36.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.282+0000 7f3640c00640 1 -- 172.21.15.161:0/1670796022 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c14f060 con 0x7f363c15e130 2024-05-22T21:13:36.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.286+0000 7f3640c00640 1 -- 172.21.15.161:0/1670796022 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363c14f860 con 0x7f363c15e130 2024-05-22T21:13:36.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.286+0000 7f3632a00640 1 -- 172.21.15.161:0/1670796022 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3634014cb0 con 0x7f363c15e130 2024-05-22T21:13:36.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.286+0000 7f3632a00640 1 --2- 172.21.15.161:0/1670796022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3618066690 0x7f3618068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:36.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.286+0000 7f3632a00640 1 -- 172.21.15.161:0/1670796022 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3634084ac0 con 0x7f363c15e130 2024-05-22T21:13:36.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.286+0000 7f3639e00640 1 --2- 172.21.15.161:0/1670796022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3618066690 0x7f3618068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:36.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.286+0000 7f3639e00640 1 --2- 172.21.15.161:0/1670796022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3618066690 0x7f3618068b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f362c0ae000 tx=0x7f362c0b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:36.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.290+0000 7f3632a00640 1 -- 172.21.15.161:0/1670796022 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f363c14f860 con 0x7f363c15e130 2024-05-22T21:13:36.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.442+0000 7f3640c00640 1 -- 172.21.15.161:0/1670796022 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f363c15e890 con 0x7f3618066690 2024-05-22T21:13:36.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.450+0000 7f3632a00640 1 -- 172.21.15.161:0/1670796022 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260224 (secure 0 0 0) 0x7f362c0aa630 con 0x7f3618066690 2024-05-22T21:13:36.456 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:36.458 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:13:36.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.454+0000 7f3640c00640 1 -- 172.21.15.161:0/1670796022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3618066690 msgr2=0x7f3618068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.454+0000 7f3640c00640 1 --2- 172.21.15.161:0/1670796022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3618066690 0x7f3618068b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f362c0ae000 tx=0x7f362c0b5040 comp rx=0 tx=0).stop 2024-05-22T21:13:36.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.454+0000 7f3640c00640 1 -- 172.21.15.161:0/1670796022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f363c15e130 msgr2=0x7f363c14e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.454+0000 7f3640c00640 1 --2- 172.21.15.161:0/1670796022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f363c15e130 0x7f363c14e4c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3634007e70 tx=0x7f363400e4b0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.454+0000 7f3640c00640 1 -- 172.21.15.161:0/1670796022 shutdown_connections 2024-05-22T21:13:36.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.454+0000 7f3640c00640 1 --2- 172.21.15.161:0/1670796022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3618066690 0x7f3618068b50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.454+0000 7f3640c00640 1 --2- 172.21.15.161:0/1670796022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f363c15e130 0x7f363c14e4c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.454+0000 7f3640c00640 1 --2- 172.21.15.161:0/1670796022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f363c15dd50 0x7f363c0b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.454+0000 7f3640c00640 1 --2- 172.21.15.161:0/1670796022 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f363c15d410 0x7f363c14fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.454+0000 7f3640c00640 1 -- 172.21.15.161:0/1670796022 >> 172.21.15.161:0/1670796022 conn(0x7f363c0b0bc0 msgr2=0x7f363c06ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:36.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.454+0000 7f3640c00640 1 -- 172.21.15.161:0/1670796022 shutdown_connections 2024-05-22T21:13:36.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.454+0000 7f3640c00640 1 -- 172.21.15.161:0/1670796022 wait complete. 2024-05-22T21:13:36.475 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-05-22T21:13:36.398143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1746,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560753,"statfs":{"total":1600000000,"available":1598211887,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.975307"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428654+0000","last_change":"2024-05-22T21:13:23.445778+0000","last_active":"2024-05-22T21:13:24.428654+0000","last_peered":"2024-05-22T21:13:24.428654+0000","last_clean":"2024-05-22T21:13:24.428654+0000","last_became_active":"2024-05-22T21:13:23.445416+0000","last_became_peered":"2024-05-22T21:13:23.445416+0000","last_unstale":"2024-05-22T21:13:24.428654+0000","last_undegraded":"2024-05-22T21:13:24.428654+0000","last_fullsized":"2024-05-22T21:13:24.428654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:57:10.746221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423393+0000","last_change":"2024-05-22T21:13:23.429874+0000","last_active":"2024-05-22T21:13:24.423393+0000","last_peered":"2024-05-22T21:13:24.423393+0000","last_clean":"2024-05-22T21:13:24.423393+0000","last_became_active":"2024-05-22T21:13:23.428636+0000","last_became_peered":"2024-05-22T21:13:23.428636+0000","last_unstale":"2024-05-22T21:13:24.423393+0000","last_undegraded":"2024-05-22T21:13:24.423393+0000","last_fullsized":"2024-05-22T21:13:24.423393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:44:56.029080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015292+0000","last_change":"2024-05-22T21:13:23.434034+0000","last_active":"2024-05-22T21:13:25.015292+0000","last_peered":"2024-05-22T21:13:25.015292+0000","last_clean":"2024-05-22T21:13:25.015292+0000","last_became_active":"2024-05-22T21:13:23.432250+0000","last_became_peered":"2024-05-22T21:13:23.432250+0000","last_unstale":"2024-05-22T21:13:25.015292+0000","last_undegraded":"2024-05-22T21:13:25.015292+0000","last_fullsized":"2024-05-22T21:13:25.015292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:04:37.695966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919501+0000","last_change":"2024-05-22T21:13:23.440239+0000","last_active":"2024-05-22T21:13:24.919501+0000","last_peered":"2024-05-22T21:13:24.919501+0000","last_clean":"2024-05-22T21:13:24.919501+0000","last_became_active":"2024-05-22T21:13:23.433800+0000","last_became_peered":"2024-05-22T21:13:23.433800+0000","last_unstale":"2024-05-22T21:13:24.919501+0000","last_undegraded":"2024-05-22T21:13:24.919501+0000","last_fullsized":"2024-05-22T21:13:24.919501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:09:20.599937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429279+0000","last_change":"2024-05-22T21:13:23.436638+0000","last_active":"2024-05-22T21:13:24.429279+0000","last_peered":"2024-05-22T21:13:24.429279+0000","last_clean":"2024-05-22T21:13:24.429279+0000","last_became_active":"2024-05-22T21:13:23.436392+0000","last_became_peered":"2024-05-22T21:13:23.436392+0000","last_unstale":"2024-05-22T21:13:24.429279+0000","last_undegraded":"2024-05-22T21:13:24.429279+0000","last_fullsized":"2024-05-22T21:13:24.429279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:38:04.537935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919974+0000","last_change":"2024-05-22T21:13:23.442807+0000","last_active":"2024-05-22T21:13:24.919974+0000","last_peered":"2024-05-22T21:13:24.919974+0000","last_clean":"2024-05-22T21:13:24.919974+0000","last_became_active":"2024-05-22T21:13:23.441590+0000","last_became_peered":"2024-05-22T21:13:23.441590+0000","last_unstale":"2024-05-22T21:13:24.919974+0000","last_undegraded":"2024-05-22T21:13:24.919974+0000","last_fullsized":"2024-05-22T21:13:24.919974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:57:11.273220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429185+0000","last_change":"2024-05-22T21:13:23.436160+0000","last_active":"2024-05-22T21:13:24.429185+0000","last_peered":"2024-05-22T21:13:24.429185+0000","last_clean":"2024-05-22T21:13:24.429185+0000","last_became_active":"2024-05-22T21:13:23.435949+0000","last_became_peered":"2024-05-22T21:13:23.435949+0000","last_unstale":"2024-05-22T21:13:24.429185+0000","last_undegraded":"2024-05-22T21:13:24.429185+0000","last_fullsized":"2024-05-22T21:13:24.429185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:56:26.375232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014774+0000","last_change":"2024-05-22T21:13:23.437626+0000","last_active":"2024-05-22T21:13:25.014774+0000","last_peered":"2024-05-22T21:13:25.014774+0000","last_clean":"2024-05-22T21:13:25.014774+0000","last_became_active":"2024-05-22T21:13:23.437162+0000","last_became_peered":"2024-05-22T21:13:23.437162+0000","last_unstale":"2024-05-22T21:13:25.014774+0000","last_undegraded":"2024-05-22T21:13:25.014774+0000","last_fullsized":"2024-05-22T21:13:25.014774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:09:20.599937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428632+0000","last_change":"2024-05-22T21:13:23.446764+0000","last_active":"2024-05-22T21:13:24.428632+0000","last_peered":"2024-05-22T21:13:24.428632+0000","last_clean":"2024-05-22T21:13:24.428632+0000","last_became_active":"2024-05-22T21:13:23.446506+0000","last_became_peered":"2024-05-22T21:13:23.446506+0000","last_unstale":"2024-05-22T21:13:24.428632+0000","last_undegraded":"2024-05-22T21:13:24.428632+0000","last_fullsized":"2024-05-22T21:13:24.428632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:13:51.196274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014576+0000","last_change":"2024-05-22T21:13:23.450063+0000","last_active":"2024-05-22T21:13:25.014576+0000","last_peered":"2024-05-22T21:13:25.014576+0000","last_clean":"2024-05-22T21:13:25.014576+0000","last_became_active":"2024-05-22T21:13:23.449193+0000","last_became_peered":"2024-05-22T21:13:23.449193+0000","last_unstale":"2024-05-22T21:13:25.014576+0000","last_undegraded":"2024-05-22T21:13:25.014576+0000","last_fullsized":"2024-05-22T21:13:25.014576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:14:11.125539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014530+0000","last_change":"2024-05-22T21:13:23.449563+0000","last_active":"2024-05-22T21:13:25.014530+0000","last_peered":"2024-05-22T21:13:25.014530+0000","last_clean":"2024-05-22T21:13:25.014530+0000","last_became_active":"2024-05-22T21:13:23.449304+0000","last_became_peered":"2024-05-22T21:13:23.449304+0000","last_unstale":"2024-05-22T21:13:25.014530+0000","last_undegraded":"2024-05-22T21:13:25.014530+0000","last_fullsized":"2024-05-22T21:13:25.014530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:07:36.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428800+0000","last_change":"2024-05-22T21:13:23.444134+0000","last_active":"2024-05-22T21:13:24.428800+0000","last_peered":"2024-05-22T21:13:24.428800+0000","last_clean":"2024-05-22T21:13:24.428800+0000","last_became_active":"2024-05-22T21:13:23.443924+0000","last_became_peered":"2024-05-22T21:13:23.443924+0000","last_unstale":"2024-05-22T21:13:24.428800+0000","last_undegraded":"2024-05-22T21:13:24.428800+0000","last_fullsized":"2024-05-22T21:13:24.428800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:10.124782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014938+0000","last_change":"2024-05-22T21:13:23.447229+0000","last_active":"2024-05-22T21:13:25.014938+0000","last_peered":"2024-05-22T21:13:25.014938+0000","last_clean":"2024-05-22T21:13:25.014938+0000","last_became_active":"2024-05-22T21:13:23.447007+0000","last_became_peered":"2024-05-22T21:13:23.447007+0000","last_unstale":"2024-05-22T21:13:25.014938+0000","last_undegraded":"2024-05-22T21:13:25.014938+0000","last_fullsized":"2024-05-22T21:13:25.014938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:44:04.542487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014914+0000","last_change":"2024-05-22T21:13:23.433465+0000","last_active":"2024-05-22T21:13:25.014914+0000","last_peered":"2024-05-22T21:13:25.014914+0000","last_clean":"2024-05-22T21:13:25.014914+0000","last_became_active":"2024-05-22T21:13:23.432948+0000","last_became_peered":"2024-05-22T21:13:23.432948+0000","last_unstale":"2024-05-22T21:13:25.014914+0000","last_undegraded":"2024-05-22T21:13:25.014914+0000","last_fullsized":"2024-05-22T21:13:25.014914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:07:36.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423652+0000","last_change":"2024-05-22T21:13:23.426036+0000","last_active":"2024-05-22T21:13:24.423652+0000","last_peered":"2024-05-22T21:13:24.423652+0000","last_clean":"2024-05-22T21:13:24.423652+0000","last_became_active":"2024-05-22T21:13:23.425830+0000","last_became_peered":"2024-05-22T21:13:23.425830+0000","last_unstale":"2024-05-22T21:13:24.423652+0000","last_undegraded":"2024-05-22T21:13:24.423652+0000","last_fullsized":"2024-05-22T21:13:24.423652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:27.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919876+0000","last_change":"2024-05-22T21:13:23.442938+0000","last_active":"2024-05-22T21:13:24.919876+0000","last_peered":"2024-05-22T21:13:24.919876+0000","last_clean":"2024-05-22T21:13:24.919876+0000","last_became_active":"2024-05-22T21:13:23.441874+0000","last_became_peered":"2024-05-22T21:13:23.441874+0000","last_unstale":"2024-05-22T21:13:24.919876+0000","last_undegraded":"2024-05-22T21:13:24.919876+0000","last_fullsized":"2024-05-22T21:13:24.919876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:57:10.746221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919437+0000","last_change":"2024-05-22T21:13:23.444504+0000","last_active":"2024-05-22T21:13:24.919437+0000","last_peered":"2024-05-22T21:13:24.919437+0000","last_clean":"2024-05-22T21:13:24.919437+0000","last_became_active":"2024-05-22T21:13:23.443892+0000","last_became_peered":"2024-05-22T21:13:23.443892+0000","last_unstale":"2024-05-22T21:13:24.919437+0000","last_undegraded":"2024-05-22T21:13:24.919437+0000","last_fullsized":"2024-05-22T21:13:24.919437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:56:26.375232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014525+0000","last_change":"2024-05-22T21:13:23.447393+0000","last_active":"2024-05-22T21:13:25.014525+0000","last_peered":"2024-05-22T21:13:25.014525+0000","last_clean":"2024-05-22T21:13:25.014525+0000","last_became_active":"2024-05-22T21:13:23.446956+0000","last_became_peered":"2024-05-22T21:13:23.446956+0000","last_unstale":"2024-05-22T21:13:25.014525+0000","last_undegraded":"2024-05-22T21:13:25.014525+0000","last_fullsized":"2024-05-22T21:13:25.014525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:34:49.902120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428827+0000","last_change":"2024-05-22T21:13:23.437025+0000","last_active":"2024-05-22T21:13:24.428827+0000","last_peered":"2024-05-22T21:13:24.428827+0000","last_clean":"2024-05-22T21:13:24.428827+0000","last_became_active":"2024-05-22T21:13:23.436786+0000","last_became_peered":"2024-05-22T21:13:23.436786+0000","last_unstale":"2024-05-22T21:13:24.428827+0000","last_undegraded":"2024-05-22T21:13:24.428827+0000","last_fullsized":"2024-05-22T21:13:24.428827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:04:32.035089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014759+0000","last_change":"2024-05-22T21:13:23.444119+0000","last_active":"2024-05-22T21:13:25.014759+0000","last_peered":"2024-05-22T21:13:25.014759+0000","last_clean":"2024-05-22T21:13:25.014759+0000","last_became_active":"2024-05-22T21:13:23.443861+0000","last_became_peered":"2024-05-22T21:13:23.443861+0000","last_unstale":"2024-05-22T21:13:25.014759+0000","last_undegraded":"2024-05-22T21:13:25.014759+0000","last_fullsized":"2024-05-22T21:13:25.014759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:00:43.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015860+0000","last_change":"2024-05-22T21:13:23.434203+0000","last_active":"2024-05-22T21:13:25.015860+0000","last_peered":"2024-05-22T21:13:25.015860+0000","last_clean":"2024-05-22T21:13:25.015860+0000","last_became_active":"2024-05-22T21:13:23.433871+0000","last_became_peered":"2024-05-22T21:13:23.433871+0000","last_unstale":"2024-05-22T21:13:25.015860+0000","last_undegraded":"2024-05-22T21:13:25.015860+0000","last_fullsized":"2024-05-22T21:13:25.015860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:00:43.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919633+0000","last_change":"2024-05-22T21:13:23.444258+0000","last_active":"2024-05-22T21:13:24.919633+0000","last_peered":"2024-05-22T21:13:24.919633+0000","last_clean":"2024-05-22T21:13:24.919633+0000","last_became_active":"2024-05-22T21:13:23.443329+0000","last_became_peered":"2024-05-22T21:13:23.443329+0000","last_unstale":"2024-05-22T21:13:24.919633+0000","last_undegraded":"2024-05-22T21:13:24.919633+0000","last_fullsized":"2024-05-22T21:13:24.919633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015111+0000","last_change":"2024-05-22T21:13:23.437489+0000","last_active":"2024-05-22T21:13:25.015111+0000","last_peered":"2024-05-22T21:13:25.015111+0000","last_clean":"2024-05-22T21:13:25.015111+0000","last_became_active":"2024-05-22T21:13:23.436877+0000","last_became_peered":"2024-05-22T21:13:23.436877+0000","last_unstale":"2024-05-22T21:13:25.015111+0000","last_undegraded":"2024-05-22T21:13:25.015111+0000","last_fullsized":"2024-05-22T21:13:25.015111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:56:26.375232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919626+0000","last_change":"2024-05-22T21:13:23.434147+0000","last_active":"2024-05-22T21:13:24.919626+0000","last_peered":"2024-05-22T21:13:24.919626+0000","last_clean":"2024-05-22T21:13:24.919626+0000","last_became_active":"2024-05-22T21:13:23.433529+0000","last_became_peered":"2024-05-22T21:13:23.433529+0000","last_unstale":"2024-05-22T21:13:24.919626+0000","last_undegraded":"2024-05-22T21:13:24.919626+0000","last_fullsized":"2024-05-22T21:13:24.919626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:44:04.542487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015414+0000","last_change":"2024-05-22T21:13:23.435603+0000","last_active":"2024-05-22T21:13:25.015414+0000","last_peered":"2024-05-22T21:13:25.015414+0000","last_clean":"2024-05-22T21:13:25.015414+0000","last_became_active":"2024-05-22T21:13:23.435387+0000","last_became_peered":"2024-05-22T21:13:23.435387+0000","last_unstale":"2024-05-22T21:13:25.015414+0000","last_undegraded":"2024-05-22T21:13:25.015414+0000","last_fullsized":"2024-05-22T21:13:25.015414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:27.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014667+0000","last_change":"2024-05-22T21:13:23.444856+0000","last_active":"2024-05-22T21:13:25.014667+0000","last_peered":"2024-05-22T21:13:25.014667+0000","last_clean":"2024-05-22T21:13:25.014667+0000","last_became_active":"2024-05-22T21:13:23.444640+0000","last_became_peered":"2024-05-22T21:13:23.444640+0000","last_unstale":"2024-05-22T21:13:25.014667+0000","last_undegraded":"2024-05-22T21:13:25.014667+0000","last_fullsized":"2024-05-22T21:13:25.014667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:07:57.351276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423445+0000","last_change":"2024-05-22T21:13:23.430546+0000","last_active":"2024-05-22T21:13:24.423445+0000","last_peered":"2024-05-22T21:13:24.423445+0000","last_clean":"2024-05-22T21:13:24.423445+0000","last_became_active":"2024-05-22T21:13:23.430306+0000","last_became_peered":"2024-05-22T21:13:23.430306+0000","last_unstale":"2024-05-22T21:13:24.423445+0000","last_undegraded":"2024-05-22T21:13:24.423445+0000","last_fullsized":"2024-05-22T21:13:24.423445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:06.303665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015209+0000","last_change":"2024-05-22T21:13:23.437362+0000","last_active":"2024-05-22T21:13:25.015209+0000","last_peered":"2024-05-22T21:13:25.015209+0000","last_clean":"2024-05-22T21:13:25.015209+0000","last_became_active":"2024-05-22T21:13:23.436580+0000","last_became_peered":"2024-05-22T21:13:23.436580+0000","last_unstale":"2024-05-22T21:13:25.015209+0000","last_undegraded":"2024-05-22T21:13:25.015209+0000","last_fullsized":"2024-05-22T21:13:25.015209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:06.303665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015001+0000","last_change":"2024-05-22T21:13:23.433616+0000","last_active":"2024-05-22T21:13:25.015001+0000","last_peered":"2024-05-22T21:13:25.015001+0000","last_clean":"2024-05-22T21:13:25.015001+0000","last_became_active":"2024-05-22T21:13:23.433258+0000","last_became_peered":"2024-05-22T21:13:23.433258+0000","last_unstale":"2024-05-22T21:13:25.015001+0000","last_undegraded":"2024-05-22T21:13:25.015001+0000","last_fullsized":"2024-05-22T21:13:25.015001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:44:56.029080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919662+0000","last_change":"2024-05-22T21:13:23.435441+0000","last_active":"2024-05-22T21:13:24.919662+0000","last_peered":"2024-05-22T21:13:24.919662+0000","last_clean":"2024-05-22T21:13:24.919662+0000","last_became_active":"2024-05-22T21:13:23.435225+0000","last_became_peered":"2024-05-22T21:13:23.435225+0000","last_unstale":"2024-05-22T21:13:24.919662+0000","last_undegraded":"2024-05-22T21:13:24.919662+0000","last_fullsized":"2024-05-22T21:13:24.919662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:44:56.029080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014612+0000","last_change":"2024-05-22T21:13:23.446437+0000","last_active":"2024-05-22T21:13:25.014612+0000","last_peered":"2024-05-22T21:13:25.014612+0000","last_clean":"2024-05-22T21:13:25.014612+0000","last_became_active":"2024-05-22T21:13:23.446219+0000","last_became_peered":"2024-05-22T21:13:23.446219+0000","last_unstale":"2024-05-22T21:13:25.014612+0000","last_undegraded":"2024-05-22T21:13:25.014612+0000","last_fullsized":"2024-05-22T21:13:25.014612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:57:10.070412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014588+0000","last_change":"2024-05-22T21:13:23.443792+0000","last_active":"2024-05-22T21:13:25.014588+0000","last_peered":"2024-05-22T21:13:25.014588+0000","last_clean":"2024-05-22T21:13:25.014588+0000","last_became_active":"2024-05-22T21:13:23.443581+0000","last_became_peered":"2024-05-22T21:13:23.443581+0000","last_unstale":"2024-05-22T21:13:25.014588+0000","last_undegraded":"2024-05-22T21:13:25.014588+0000","last_fullsized":"2024-05-22T21:13:25.014588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.430114+0000","last_change":"2024-05-22T21:13:23.429085+0000","last_active":"2024-05-22T21:13:24.430114+0000","last_peered":"2024-05-22T21:13:24.430114+0000","last_clean":"2024-05-22T21:13:24.430114+0000","last_became_active":"2024-05-22T21:13:23.428826+0000","last_became_peered":"2024-05-22T21:13:23.428826+0000","last_unstale":"2024-05-22T21:13:24.430114+0000","last_undegraded":"2024-05-22T21:13:24.430114+0000","last_fullsized":"2024-05-22T21:13:24.430114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:17.109310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015138+0000","last_change":"2024-05-22T21:13:23.053644+0000","last_active":"2024-05-22T21:13:25.015138+0000","last_peered":"2024-05-22T21:13:25.015138+0000","last_clean":"2024-05-22T21:13:25.015138+0000","last_became_active":"2024-05-22T21:13:23.053129+0000","last_became_peered":"2024-05-22T21:13:23.053129+0000","last_unstale":"2024-05-22T21:13:25.015138+0000","last_undegraded":"2024-05-22T21:13:25.015138+0000","last_fullsized":"2024-05-22T21:13:25.015138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:02.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015372+0000","last_change":"2024-05-22T21:13:23.430851+0000","last_active":"2024-05-22T21:13:25.015372+0000","last_peered":"2024-05-22T21:13:25.015372+0000","last_clean":"2024-05-22T21:13:25.015372+0000","last_became_active":"2024-05-22T21:13:23.430614+0000","last_became_peered":"2024-05-22T21:13:23.430614+0000","last_unstale":"2024-05-22T21:13:25.015372+0000","last_undegraded":"2024-05-22T21:13:25.015372+0000","last_fullsized":"2024-05-22T21:13:25.015372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:14:11.125539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014815+0000","last_change":"2024-05-22T21:13:23.052090+0000","last_active":"2024-05-22T21:13:25.014815+0000","last_peered":"2024-05-22T21:13:25.014815+0000","last_clean":"2024-05-22T21:13:25.014815+0000","last_became_active":"2024-05-22T21:13:23.051762+0000","last_became_peered":"2024-05-22T21:13:23.051762+0000","last_unstale":"2024-05-22T21:13:25.014815+0000","last_undegraded":"2024-05-22T21:13:25.014815+0000","last_fullsized":"2024-05-22T21:13:25.014815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:54.533292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015255+0000","last_change":"2024-05-22T21:13:23.053832+0000","last_active":"2024-05-22T21:13:25.015255+0000","last_peered":"2024-05-22T21:13:25.015255+0000","last_clean":"2024-05-22T21:13:25.015255+0000","last_became_active":"2024-05-22T21:13:23.053170+0000","last_became_peered":"2024-05-22T21:13:23.053170+0000","last_unstale":"2024-05-22T21:13:25.015255+0000","last_undegraded":"2024-05-22T21:13:25.015255+0000","last_fullsized":"2024-05-22T21:13:25.015255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015495+0000","last_change":"2024-05-22T21:13:23.429829+0000","last_active":"2024-05-22T21:13:25.015495+0000","last_peered":"2024-05-22T21:13:25.015495+0000","last_clean":"2024-05-22T21:13:25.015495+0000","last_became_active":"2024-05-22T21:13:23.429616+0000","last_became_peered":"2024-05-22T21:13:23.429616+0000","last_unstale":"2024-05-22T21:13:25.015495+0000","last_undegraded":"2024-05-22T21:13:25.015495+0000","last_fullsized":"2024-05-22T21:13:25.015495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:00:48.798418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919805+0000","last_change":"2024-05-22T21:13:23.056945+0000","last_active":"2024-05-22T21:13:24.919805+0000","last_peered":"2024-05-22T21:13:24.919805+0000","last_clean":"2024-05-22T21:13:24.919805+0000","last_became_active":"2024-05-22T21:13:23.056643+0000","last_became_peered":"2024-05-22T21:13:23.056643+0000","last_unstale":"2024-05-22T21:13:24.919805+0000","last_undegraded":"2024-05-22T21:13:24.919805+0000","last_fullsized":"2024-05-22T21:13:24.919805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:02.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423490+0000","last_change":"2024-05-22T21:13:23.429959+0000","last_active":"2024-05-22T21:13:24.423490+0000","last_peered":"2024-05-22T21:13:24.423490+0000","last_clean":"2024-05-22T21:13:24.423490+0000","last_became_active":"2024-05-22T21:13:23.429752+0000","last_became_peered":"2024-05-22T21:13:23.429752+0000","last_unstale":"2024-05-22T21:13:24.423490+0000","last_undegraded":"2024-05-22T21:13:24.423490+0000","last_fullsized":"2024-05-22T21:13:24.423490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:14:11.125539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014851+0000","last_change":"2024-05-22T21:13:23.058878+0000","last_active":"2024-05-22T21:13:25.014851+0000","last_peered":"2024-05-22T21:13:25.014851+0000","last_clean":"2024-05-22T21:13:25.014851+0000","last_became_active":"2024-05-22T21:13:23.058648+0000","last_became_peered":"2024-05-22T21:13:23.058648+0000","last_unstale":"2024-05-22T21:13:25.014851+0000","last_undegraded":"2024-05-22T21:13:25.014851+0000","last_fullsized":"2024-05-22T21:13:25.014851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919552+0000","last_change":"2024-05-22T21:13:23.444383+0000","last_active":"2024-05-22T21:13:24.919552+0000","last_peered":"2024-05-22T21:13:24.919552+0000","last_clean":"2024-05-22T21:13:24.919552+0000","last_became_active":"2024-05-22T21:13:23.443612+0000","last_became_peered":"2024-05-22T21:13:23.443612+0000","last_unstale":"2024-05-22T21:13:24.919552+0000","last_undegraded":"2024-05-22T21:13:24.919552+0000","last_fullsized":"2024-05-22T21:13:24.919552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:27.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919883+0000","last_change":"2024-05-22T21:13:23.056793+0000","last_active":"2024-05-22T21:13:24.919883+0000","last_peered":"2024-05-22T21:13:24.919883+0000","last_clean":"2024-05-22T21:13:24.919883+0000","last_became_active":"2024-05-22T21:13:23.056450+0000","last_became_peered":"2024-05-22T21:13:23.056450+0000","last_unstale":"2024-05-22T21:13:24.919883+0000","last_undegraded":"2024-05-22T21:13:24.919883+0000","last_fullsized":"2024-05-22T21:13:24.919883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:34.410142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015942+0000","last_change":"2024-05-22T21:13:23.446398+0000","last_active":"2024-05-22T21:13:25.015942+0000","last_peered":"2024-05-22T21:13:25.015942+0000","last_clean":"2024-05-22T21:13:25.015942+0000","last_became_active":"2024-05-22T21:13:23.445907+0000","last_became_peered":"2024-05-22T21:13:23.445907+0000","last_unstale":"2024-05-22T21:13:25.015942+0000","last_undegraded":"2024-05-22T21:13:25.015942+0000","last_fullsized":"2024-05-22T21:13:25.015942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:06.303665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423552+0000","last_change":"2024-05-22T21:13:23.056570+0000","last_active":"2024-05-22T21:13:24.423552+0000","last_peered":"2024-05-22T21:13:24.423552+0000","last_clean":"2024-05-22T21:13:24.423552+0000","last_became_active":"2024-05-22T21:13:23.056357+0000","last_became_peered":"2024-05-22T21:13:23.056357+0000","last_unstale":"2024-05-22T21:13:24.423552+0000","last_undegraded":"2024-05-22T21:13:24.423552+0000","last_fullsized":"2024-05-22T21:13:24.423552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:30.101472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428611+0000","last_change":"2024-05-22T21:13:23.439128+0000","last_active":"2024-05-22T21:13:24.428611+0000","last_peered":"2024-05-22T21:13:24.428611+0000","last_clean":"2024-05-22T21:13:24.428611+0000","last_became_active":"2024-05-22T21:13:23.438898+0000","last_became_peered":"2024-05-22T21:13:23.438898+0000","last_unstale":"2024-05-22T21:13:24.428611+0000","last_undegraded":"2024-05-22T21:13:24.428611+0000","last_fullsized":"2024-05-22T21:13:24.428611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:51:29.569811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014610+0000","last_change":"2024-05-22T21:13:23.049600+0000","last_active":"2024-05-22T21:13:25.014610+0000","last_peered":"2024-05-22T21:13:25.014610+0000","last_clean":"2024-05-22T21:13:25.014610+0000","last_became_active":"2024-05-22T21:13:23.049395+0000","last_became_peered":"2024-05-22T21:13:23.049395+0000","last_unstale":"2024-05-22T21:13:25.014610+0000","last_undegraded":"2024-05-22T21:13:25.014610+0000","last_fullsized":"2024-05-22T21:13:25.014610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:24.879526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429576+0000","last_change":"2024-05-22T21:13:23.439228+0000","last_active":"2024-05-22T21:13:24.429576+0000","last_peered":"2024-05-22T21:13:24.429576+0000","last_clean":"2024-05-22T21:13:24.429576+0000","last_became_active":"2024-05-22T21:13:23.437292+0000","last_became_peered":"2024-05-22T21:13:23.437292+0000","last_unstale":"2024-05-22T21:13:24.429576+0000","last_undegraded":"2024-05-22T21:13:24.429576+0000","last_fullsized":"2024-05-22T21:13:24.429576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:44:04.542487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015531+0000","last_change":"2024-05-22T21:13:23.052267+0000","last_active":"2024-05-22T21:13:25.015531+0000","last_peered":"2024-05-22T21:13:25.015531+0000","last_clean":"2024-05-22T21:13:25.015531+0000","last_became_active":"2024-05-22T21:13:23.051663+0000","last_became_peered":"2024-05-22T21:13:23.051663+0000","last_unstale":"2024-05-22T21:13:25.015531+0000","last_undegraded":"2024-05-22T21:13:25.015531+0000","last_fullsized":"2024-05-22T21:13:25.015531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:54.533292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015989+0000","last_change":"2024-05-22T21:13:23.434082+0000","last_active":"2024-05-22T21:13:25.015989+0000","last_peered":"2024-05-22T21:13:25.015989+0000","last_clean":"2024-05-22T21:13:25.015989+0000","last_became_active":"2024-05-22T21:13:23.433591+0000","last_became_peered":"2024-05-22T21:13:23.433591+0000","last_unstale":"2024-05-22T21:13:25.015989+0000","last_undegraded":"2024-05-22T21:13:25.015989+0000","last_fullsized":"2024-05-22T21:13:25.015989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:08:18.923855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.920207+0000","last_change":"2024-05-22T21:13:23.058203+0000","last_active":"2024-05-22T21:13:24.920207+0000","last_peered":"2024-05-22T21:13:24.920207+0000","last_clean":"2024-05-22T21:13:24.920207+0000","last_became_active":"2024-05-22T21:13:23.057230+0000","last_became_peered":"2024-05-22T21:13:23.057230+0000","last_unstale":"2024-05-22T21:13:24.920207+0000","last_undegraded":"2024-05-22T21:13:24.920207+0000","last_fullsized":"2024-05-22T21:13:24.920207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:16.101596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429362+0000","last_change":"2024-05-22T21:13:23.437522+0000","last_active":"2024-05-22T21:13:24.429362+0000","last_peered":"2024-05-22T21:13:24.429362+0000","last_clean":"2024-05-22T21:13:24.429362+0000","last_became_active":"2024-05-22T21:13:23.436971+0000","last_became_peered":"2024-05-22T21:13:23.436971+0000","last_unstale":"2024-05-22T21:13:24.429362+0000","last_undegraded":"2024-05-22T21:13:24.429362+0000","last_fullsized":"2024-05-22T21:13:24.429362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:51:04.078274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014654+0000","last_change":"2024-05-22T21:13:23.056955+0000","last_active":"2024-05-22T21:13:25.014654+0000","last_peered":"2024-05-22T21:13:25.014654+0000","last_clean":"2024-05-22T21:13:25.014654+0000","last_became_active":"2024-05-22T21:13:23.056131+0000","last_became_peered":"2024-05-22T21:13:23.056131+0000","last_unstale":"2024-05-22T21:13:25.014654+0000","last_undegraded":"2024-05-22T21:13:25.014654+0000","last_fullsized":"2024-05-22T21:13:25.014654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:02.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014604+0000","last_change":"2024-05-22T21:13:23.445799+0000","last_active":"2024-05-22T21:13:25.014604+0000","last_peered":"2024-05-22T21:13:25.014604+0000","last_clean":"2024-05-22T21:13:25.014604+0000","last_became_active":"2024-05-22T21:13:23.445576+0000","last_became_peered":"2024-05-22T21:13:23.445576+0000","last_unstale":"2024-05-22T21:13:25.014604+0000","last_undegraded":"2024-05-22T21:13:25.014604+0000","last_fullsized":"2024-05-22T21:13:25.014604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:04:37.695966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014626+0000","last_change":"2024-05-22T21:13:23.046892+0000","last_active":"2024-05-22T21:13:25.014626+0000","last_peered":"2024-05-22T21:13:25.014626+0000","last_clean":"2024-05-22T21:13:25.014626+0000","last_became_active":"2024-05-22T21:13:23.046677+0000","last_became_peered":"2024-05-22T21:13:23.046677+0000","last_unstale":"2024-05-22T21:13:25.014626+0000","last_undegraded":"2024-05-22T21:13:25.014626+0000","last_fullsized":"2024-05-22T21:13:25.014626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:24.879526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429428+0000","last_change":"2024-05-22T21:13:23.437891+0000","last_active":"2024-05-22T21:13:24.429428+0000","last_peered":"2024-05-22T21:13:24.429428+0000","last_clean":"2024-05-22T21:13:24.429428+0000","last_became_active":"2024-05-22T21:13:23.436990+0000","last_became_peered":"2024-05-22T21:13:23.436990+0000","last_unstale":"2024-05-22T21:13:24.429428+0000","last_undegraded":"2024-05-22T21:13:24.429428+0000","last_fullsized":"2024-05-22T21:13:24.429428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:12:43.781854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428700+0000","last_change":"2024-05-22T21:13:23.054659+0000","last_active":"2024-05-22T21:13:24.428700+0000","last_peered":"2024-05-22T21:13:24.428700+0000","last_clean":"2024-05-22T21:13:24.428700+0000","last_became_active":"2024-05-22T21:13:23.054403+0000","last_became_peered":"2024-05-22T21:13:23.054403+0000","last_unstale":"2024-05-22T21:13:24.428700+0000","last_undegraded":"2024-05-22T21:13:24.428700+0000","last_fullsized":"2024-05-22T21:13:24.428700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:26:28.755372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429419+0000","last_change":"2024-05-22T21:13:23.437668+0000","last_active":"2024-05-22T21:13:24.429419+0000","last_peered":"2024-05-22T21:13:24.429419+0000","last_clean":"2024-05-22T21:13:24.429419+0000","last_became_active":"2024-05-22T21:13:23.437307+0000","last_became_peered":"2024-05-22T21:13:23.437307+0000","last_unstale":"2024-05-22T21:13:24.429419+0000","last_undegraded":"2024-05-22T21:13:24.429419+0000","last_fullsized":"2024-05-22T21:13:24.429419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:57:11.273220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014824+0000","last_change":"2024-05-22T21:13:23.047615+0000","last_active":"2024-05-22T21:13:25.014824+0000","last_peered":"2024-05-22T21:13:25.014824+0000","last_clean":"2024-05-22T21:13:25.014824+0000","last_became_active":"2024-05-22T21:13:23.047358+0000","last_became_peered":"2024-05-22T21:13:23.047358+0000","last_unstale":"2024-05-22T21:13:25.014824+0000","last_undegraded":"2024-05-22T21:13:25.014824+0000","last_fullsized":"2024-05-22T21:13:25.014824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:34.410142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428730+0000","last_change":"2024-05-22T21:13:23.443073+0000","last_active":"2024-05-22T21:13:24.428730+0000","last_peered":"2024-05-22T21:13:24.428730+0000","last_clean":"2024-05-22T21:13:24.428730+0000","last_became_active":"2024-05-22T21:13:23.442751+0000","last_became_peered":"2024-05-22T21:13:23.442751+0000","last_unstale":"2024-05-22T21:13:24.428730+0000","last_undegraded":"2024-05-22T21:13:24.428730+0000","last_fullsized":"2024-05-22T21:13:24.428730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:09:20.599937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423443+0000","last_change":"2024-05-22T21:13:23.050892+0000","last_active":"2024-05-22T21:13:24.423443+0000","last_peered":"2024-05-22T21:13:24.423443+0000","last_clean":"2024-05-22T21:13:24.423443+0000","last_became_active":"2024-05-22T21:13:23.050565+0000","last_became_peered":"2024-05-22T21:13:23.050565+0000","last_unstale":"2024-05-22T21:13:24.423443+0000","last_undegraded":"2024-05-22T21:13:24.423443+0000","last_fullsized":"2024-05-22T21:13:24.423443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:16.101596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919723+0000","last_change":"2024-05-22T21:13:23.437191+0000","last_active":"2024-05-22T21:13:24.919723+0000","last_peered":"2024-05-22T21:13:24.919723+0000","last_clean":"2024-05-22T21:13:24.919723+0000","last_became_active":"2024-05-22T21:13:23.436983+0000","last_became_peered":"2024-05-22T21:13:23.436983+0000","last_unstale":"2024-05-22T21:13:24.919723+0000","last_undegraded":"2024-05-22T21:13:24.919723+0000","last_fullsized":"2024-05-22T21:13:24.919723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:06.303665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014631+0000","last_change":"2024-05-22T21:13:23.056461+0000","last_active":"2024-05-22T21:13:25.014631+0000","last_peered":"2024-05-22T21:13:25.014631+0000","last_clean":"2024-05-22T21:13:25.014631+0000","last_became_active":"2024-05-22T21:13:23.056237+0000","last_became_peered":"2024-05-22T21:13:23.056237+0000","last_unstale":"2024-05-22T21:13:25.014631+0000","last_undegraded":"2024-05-22T21:13:25.014631+0000","last_fullsized":"2024-05-22T21:13:25.014631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428905+0000","last_change":"2024-05-22T21:13:23.448377+0000","last_active":"2024-05-22T21:13:24.428905+0000","last_peered":"2024-05-22T21:13:24.428905+0000","last_clean":"2024-05-22T21:13:24.428905+0000","last_became_active":"2024-05-22T21:13:23.447835+0000","last_became_peered":"2024-05-22T21:13:23.447835+0000","last_unstale":"2024-05-22T21:13:24.428905+0000","last_undegraded":"2024-05-22T21:13:24.428905+0000","last_fullsized":"2024-05-22T21:13:24.428905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:44:10.866789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-22T21:13:23.435118+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:13:23.435118+0000","last_peered":"2024-05-22T21:13:23.435118+0000","last_clean":"2024-05-22T21:13:23.435118+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:13:23.435118+0000","last_undegraded":"2024-05-22T21:13:23.435118+0000","last_fullsized":"2024-05-22T21:13:23.435118+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.920353+0000","last_change":"2024-05-22T21:13:23.058027+0000","last_active":"2024-05-22T21:13:24.920353+0000","last_peered":"2024-05-22T21:13:24.920353+0000","last_clean":"2024-05-22T21:13:24.920353+0000","last_became_active":"2024-05-22T21:13:23.056401+0000","last_became_peered":"2024-05-22T21:13:23.056401+0000","last_unstale":"2024-05-22T21:13:24.920353+0000","last_undegraded":"2024-05-22T21:13:24.920353+0000","last_fullsized":"2024-05-22T21:13:24.920353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:24.879526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014947+0000","last_change":"2024-05-22T21:13:23.440325+0000","last_active":"2024-05-22T21:13:25.014947+0000","last_peered":"2024-05-22T21:13:25.014947+0000","last_clean":"2024-05-22T21:13:25.014947+0000","last_became_active":"2024-05-22T21:13:23.440020+0000","last_became_peered":"2024-05-22T21:13:23.440020+0000","last_unstale":"2024-05-22T21:13:25.014947+0000","last_undegraded":"2024-05-22T21:13:25.014947+0000","last_fullsized":"2024-05-22T21:13:25.014947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:38:04.537935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429005+0000","last_change":"2024-05-22T21:13:23.050452+0000","last_active":"2024-05-22T21:13:24.429005+0000","last_peered":"2024-05-22T21:13:24.429005+0000","last_clean":"2024-05-22T21:13:24.429005+0000","last_became_active":"2024-05-22T21:13:23.049982+0000","last_became_peered":"2024-05-22T21:13:23.049982+0000","last_unstale":"2024-05-22T21:13:24.429005+0000","last_undegraded":"2024-05-22T21:13:24.429005+0000","last_fullsized":"2024-05-22T21:13:24.429005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:57:09.062698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015492+0000","last_change":"2024-05-22T21:13:23.434429+0000","last_active":"2024-05-22T21:13:25.015492+0000","last_peered":"2024-05-22T21:13:25.015492+0000","last_clean":"2024-05-22T21:13:25.015492+0000","last_became_active":"2024-05-22T21:13:23.433640+0000","last_became_peered":"2024-05-22T21:13:23.433640+0000","last_unstale":"2024-05-22T21:13:25.015492+0000","last_undegraded":"2024-05-22T21:13:25.015492+0000","last_fullsized":"2024-05-22T21:13:25.015492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014707+0000","last_change":"2024-05-22T21:13:23.051179+0000","last_active":"2024-05-22T21:13:25.014707+0000","last_peered":"2024-05-22T21:13:25.014707+0000","last_clean":"2024-05-22T21:13:25.014707+0000","last_became_active":"2024-05-22T21:13:23.050967+0000","last_became_peered":"2024-05-22T21:13:23.050967+0000","last_unstale":"2024-05-22T21:13:25.014707+0000","last_undegraded":"2024-05-22T21:13:25.014707+0000","last_fullsized":"2024-05-22T21:13:25.014707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:02.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423581+0000","last_change":"2024-05-22T21:13:23.426627+0000","last_active":"2024-05-22T21:13:24.423581+0000","last_peered":"2024-05-22T21:13:24.423581+0000","last_clean":"2024-05-22T21:13:24.423581+0000","last_became_active":"2024-05-22T21:13:23.426412+0000","last_became_peered":"2024-05-22T21:13:23.426412+0000","last_unstale":"2024-05-22T21:13:24.423581+0000","last_undegraded":"2024-05-22T21:13:24.423581+0000","last_fullsized":"2024-05-22T21:13:24.423581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919933+0000","last_change":"2024-05-22T21:13:23.055924+0000","last_active":"2024-05-22T21:13:24.919933+0000","last_peered":"2024-05-22T21:13:24.919933+0000","last_clean":"2024-05-22T21:13:24.919933+0000","last_became_active":"2024-05-22T21:13:23.055577+0000","last_became_peered":"2024-05-22T21:13:23.055577+0000","last_unstale":"2024-05-22T21:13:24.919933+0000","last_undegraded":"2024-05-22T21:13:24.919933+0000","last_fullsized":"2024-05-22T21:13:24.919933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.430340+0000","last_change":"2024-05-22T21:13:23.428954+0000","last_active":"2024-05-22T21:13:24.430340+0000","last_peered":"2024-05-22T21:13:24.430340+0000","last_clean":"2024-05-22T21:13:24.430340+0000","last_became_active":"2024-05-22T21:13:23.428628+0000","last_became_peered":"2024-05-22T21:13:23.428628+0000","last_unstale":"2024-05-22T21:13:24.430340+0000","last_undegraded":"2024-05-22T21:13:24.430340+0000","last_fullsized":"2024-05-22T21:13:24.430340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:31.109187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423797+0000","last_change":"2024-05-22T21:13:23.055046+0000","last_active":"2024-05-22T21:13:24.423797+0000","last_peered":"2024-05-22T21:13:24.423797+0000","last_clean":"2024-05-22T21:13:24.423797+0000","last_became_active":"2024-05-22T21:13:23.054670+0000","last_became_peered":"2024-05-22T21:13:23.054670+0000","last_unstale":"2024-05-22T21:13:24.423797+0000","last_undegraded":"2024-05-22T21:13:24.423797+0000","last_fullsized":"2024-05-22T21:13:24.423797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:24.879526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919722+0000","last_change":"2024-05-22T21:13:23.434008+0000","last_active":"2024-05-22T21:13:24.919722+0000","last_peered":"2024-05-22T21:13:24.919722+0000","last_clean":"2024-05-22T21:13:24.919722+0000","last_became_active":"2024-05-22T21:13:23.433233+0000","last_became_peered":"2024-05-22T21:13:23.433233+0000","last_unstale":"2024-05-22T21:13:24.919722+0000","last_undegraded":"2024-05-22T21:13:24.919722+0000","last_fullsized":"2024-05-22T21:13:24.919722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:38:04.537935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428911+0000","last_change":"2024-05-22T21:13:23.041563+0000","last_active":"2024-05-22T21:13:24.428911+0000","last_peered":"2024-05-22T21:13:24.428911+0000","last_clean":"2024-05-22T21:13:24.428911+0000","last_became_active":"2024-05-22T21:13:23.041302+0000","last_became_peered":"2024-05-22T21:13:23.041302+0000","last_unstale":"2024-05-22T21:13:24.428911+0000","last_undegraded":"2024-05-22T21:13:24.428911+0000","last_fullsized":"2024-05-22T21:13:24.428911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:51:28.562097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423568+0000","last_change":"2024-05-22T21:13:23.426584+0000","last_active":"2024-05-22T21:13:24.423568+0000","last_peered":"2024-05-22T21:13:24.423568+0000","last_clean":"2024-05-22T21:13:24.423568+0000","last_became_active":"2024-05-22T21:13:23.426367+0000","last_became_peered":"2024-05-22T21:13:23.426367+0000","last_unstale":"2024-05-22T21:13:24.423568+0000","last_undegraded":"2024-05-22T21:13:24.423568+0000","last_fullsized":"2024-05-22T21:13:24.423568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:07:36.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:26.399531+0000","last_change":"2024-05-22T21:13:23.049729+0000","last_active":"2024-05-22T21:13:26.399531+0000","last_peered":"2024-05-22T21:13:26.399531+0000","last_clean":"2024-05-22T21:13:26.399531+0000","last_became_active":"2024-05-22T21:13:23.049480+0000","last_became_peered":"2024-05-22T21:13:23.049480+0000","last_unstale":"2024-05-22T21:13:26.399531+0000","last_undegraded":"2024-05-22T21:13:26.399531+0000","last_fullsized":"2024-05-22T21:13:26.399531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:51:28.562097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014848+0000","last_change":"2024-05-22T21:13:23.437853+0000","last_active":"2024-05-22T21:13:25.014848+0000","last_peered":"2024-05-22T21:13:25.014848+0000","last_clean":"2024-05-22T21:13:25.014848+0000","last_became_active":"2024-05-22T21:13:23.437676+0000","last_became_peered":"2024-05-22T21:13:23.437676+0000","last_unstale":"2024-05-22T21:13:25.014848+0000","last_undegraded":"2024-05-22T21:13:25.014848+0000","last_fullsized":"2024-05-22T21:13:25.014848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:14:11.125539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014750+0000","last_change":"2024-05-22T21:13:23.052969+0000","last_active":"2024-05-22T21:13:25.014750+0000","last_peered":"2024-05-22T21:13:25.014750+0000","last_clean":"2024-05-22T21:13:25.014750+0000","last_became_active":"2024-05-22T21:13:23.052306+0000","last_became_peered":"2024-05-22T21:13:23.052306+0000","last_unstale":"2024-05-22T21:13:25.014750+0000","last_undegraded":"2024-05-22T21:13:25.014750+0000","last_fullsized":"2024-05-22T21:13:25.014750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:34.410142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015367+0000","last_change":"2024-05-22T21:13:23.434301+0000","last_active":"2024-05-22T21:13:25.015367+0000","last_peered":"2024-05-22T21:13:25.015367+0000","last_clean":"2024-05-22T21:13:25.015367+0000","last_became_active":"2024-05-22T21:13:23.432584+0000","last_became_peered":"2024-05-22T21:13:23.432584+0000","last_unstale":"2024-05-22T21:13:25.015367+0000","last_undegraded":"2024-05-22T21:13:25.015367+0000","last_fullsized":"2024-05-22T21:13:25.015367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:53:57.578626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429187+0000","last_change":"2024-05-22T21:13:23.047297+0000","last_active":"2024-05-22T21:13:24.429187+0000","last_peered":"2024-05-22T21:13:24.429187+0000","last_clean":"2024-05-22T21:13:24.429187+0000","last_became_active":"2024-05-22T21:13:23.047021+0000","last_became_peered":"2024-05-22T21:13:23.047021+0000","last_unstale":"2024-05-22T21:13:24.429187+0000","last_undegraded":"2024-05-22T21:13:24.429187+0000","last_fullsized":"2024-05-22T21:13:24.429187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:09.117068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429366+0000","last_change":"2024-05-22T21:13:23.434654+0000","last_active":"2024-05-22T21:13:24.429366+0000","last_peered":"2024-05-22T21:13:24.429366+0000","last_clean":"2024-05-22T21:13:24.429366+0000","last_became_active":"2024-05-22T21:13:23.433684+0000","last_became_peered":"2024-05-22T21:13:23.433684+0000","last_unstale":"2024-05-22T21:13:24.429366+0000","last_undegraded":"2024-05-22T21:13:24.429366+0000","last_fullsized":"2024-05-22T21:13:24.429366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:03.482406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423727+0000","last_change":"2024-05-22T21:13:23.054907+0000","last_active":"2024-05-22T21:13:24.423727+0000","last_peered":"2024-05-22T21:13:24.423727+0000","last_clean":"2024-05-22T21:13:24.423727+0000","last_became_active":"2024-05-22T21:13:23.054307+0000","last_became_peered":"2024-05-22T21:13:23.054307+0000","last_unstale":"2024-05-22T21:13:24.423727+0000","last_undegraded":"2024-05-22T21:13:24.423727+0000","last_fullsized":"2024-05-22T21:13:24.423727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:09.117068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015009+0000","last_change":"2024-05-22T21:13:23.429322+0000","last_active":"2024-05-22T21:13:25.015009+0000","last_peered":"2024-05-22T21:13:25.015009+0000","last_clean":"2024-05-22T21:13:25.015009+0000","last_became_active":"2024-05-22T21:13:23.429091+0000","last_became_peered":"2024-05-22T21:13:23.429091+0000","last_unstale":"2024-05-22T21:13:25.015009+0000","last_undegraded":"2024-05-22T21:13:25.015009+0000","last_fullsized":"2024-05-22T21:13:25.015009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:44:56.029080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423657+0000","last_change":"2024-05-22T21:13:23.052267+0000","last_active":"2024-05-22T21:13:24.423657+0000","last_peered":"2024-05-22T21:13:24.423657+0000","last_clean":"2024-05-22T21:13:24.423657+0000","last_became_active":"2024-05-22T21:13:23.052062+0000","last_became_peered":"2024-05-22T21:13:23.052062+0000","last_unstale":"2024-05-22T21:13:24.423657+0000","last_undegraded":"2024-05-22T21:13:24.423657+0000","last_fullsized":"2024-05-22T21:13:24.423657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:34:48.894406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015300+0000","last_change":"2024-05-22T21:13:23.433439+0000","last_active":"2024-05-22T21:13:25.015300+0000","last_peered":"2024-05-22T21:13:25.015300+0000","last_clean":"2024-05-22T21:13:25.015300+0000","last_became_active":"2024-05-22T21:13:23.433016+0000","last_became_peered":"2024-05-22T21:13:23.433016+0000","last_unstale":"2024-05-22T21:13:25.015300+0000","last_undegraded":"2024-05-22T21:13:25.015300+0000","last_fullsized":"2024-05-22T21:13:25.015300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429655+0000","last_change":"2024-05-22T21:13:23.052977+0000","last_active":"2024-05-22T21:13:24.429655+0000","last_peered":"2024-05-22T21:13:24.429655+0000","last_clean":"2024-05-22T21:13:24.429655+0000","last_became_active":"2024-05-22T21:13:23.052490+0000","last_became_peered":"2024-05-22T21:13:23.052490+0000","last_unstale":"2024-05-22T21:13:24.429655+0000","last_undegraded":"2024-05-22T21:13:24.429655+0000","last_fullsized":"2024-05-22T21:13:24.429655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:16.101596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014517+0000","last_change":"2024-05-22T21:13:23.446126+0000","last_active":"2024-05-22T21:13:25.014517+0000","last_peered":"2024-05-22T21:13:25.014517+0000","last_clean":"2024-05-22T21:13:25.014517+0000","last_became_active":"2024-05-22T21:13:23.444835+0000","last_became_peered":"2024-05-22T21:13:23.444835+0000","last_unstale":"2024-05-22T21:13:25.014517+0000","last_undegraded":"2024-05-22T21:13:25.014517+0000","last_fullsized":"2024-05-22T21:13:25.014517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:53:57.578626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919747+0000","last_change":"2024-05-22T21:13:23.432060+0000","last_active":"2024-05-22T21:13:24.919747+0000","last_peered":"2024-05-22T21:13:24.919747+0000","last_clean":"2024-05-22T21:13:24.919747+0000","last_became_active":"2024-05-22T21:13:23.431845+0000","last_became_peered":"2024-05-22T21:13:23.431845+0000","last_unstale":"2024-05-22T21:13:24.919747+0000","last_undegraded":"2024-05-22T21:13:24.919747+0000","last_fullsized":"2024-05-22T21:13:24.919747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:44:10.866789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429092+0000","last_change":"2024-05-22T21:13:23.058221+0000","last_active":"2024-05-22T21:13:24.429092+0000","last_peered":"2024-05-22T21:13:24.429092+0000","last_clean":"2024-05-22T21:13:24.429092+0000","last_became_active":"2024-05-22T21:13:23.058012+0000","last_became_peered":"2024-05-22T21:13:23.058012+0000","last_unstale":"2024-05-22T21:13:24.429092+0000","last_undegraded":"2024-05-22T21:13:24.429092+0000","last_fullsized":"2024-05-22T21:13:24.429092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:34:48.894406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.920057+0000","last_change":"2024-05-22T21:13:23.431267+0000","last_active":"2024-05-22T21:13:24.920057+0000","last_peered":"2024-05-22T21:13:24.920057+0000","last_clean":"2024-05-22T21:13:24.920057+0000","last_became_active":"2024-05-22T21:13:23.431054+0000","last_became_peered":"2024-05-22T21:13:23.431054+0000","last_unstale":"2024-05-22T21:13:24.920057+0000","last_undegraded":"2024-05-22T21:13:24.920057+0000","last_fullsized":"2024-05-22T21:13:24.920057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:51:04.078274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429474+0000","last_change":"2024-05-22T21:13:23.051743+0000","last_active":"2024-05-22T21:13:24.429474+0000","last_peered":"2024-05-22T21:13:24.429474+0000","last_clean":"2024-05-22T21:13:24.429474+0000","last_became_active":"2024-05-22T21:13:23.051564+0000","last_became_peered":"2024-05-22T21:13:23.051564+0000","last_unstale":"2024-05-22T21:13:24.429474+0000","last_undegraded":"2024-05-22T21:13:24.429474+0000","last_fullsized":"2024-05-22T21:13:24.429474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:30.101472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015432+0000","last_change":"2024-05-22T21:13:23.433871+0000","last_active":"2024-05-22T21:13:25.015432+0000","last_peered":"2024-05-22T21:13:25.015432+0000","last_clean":"2024-05-22T21:13:25.015432+0000","last_became_active":"2024-05-22T21:13:23.431365+0000","last_became_peered":"2024-05-22T21:13:23.431365+0000","last_unstale":"2024-05-22T21:13:25.015432+0000","last_undegraded":"2024-05-22T21:13:25.015432+0000","last_fullsized":"2024-05-22T21:13:25.015432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:27.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015179+0000","last_change":"2024-05-22T21:13:23.046747+0000","last_active":"2024-05-22T21:13:25.015179+0000","last_peered":"2024-05-22T21:13:25.015179+0000","last_clean":"2024-05-22T21:13:25.015179+0000","last_became_active":"2024-05-22T21:13:23.046523+0000","last_became_peered":"2024-05-22T21:13:23.046523+0000","last_unstale":"2024-05-22T21:13:25.015179+0000","last_undegraded":"2024-05-22T21:13:25.015179+0000","last_fullsized":"2024-05-22T21:13:25.015179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:34.410142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015087+0000","last_change":"2024-05-22T21:13:23.447604+0000","last_active":"2024-05-22T21:13:25.015087+0000","last_peered":"2024-05-22T21:13:25.015087+0000","last_clean":"2024-05-22T21:13:25.015087+0000","last_became_active":"2024-05-22T21:13:23.447391+0000","last_became_peered":"2024-05-22T21:13:23.447391+0000","last_unstale":"2024-05-22T21:13:25.015087+0000","last_undegraded":"2024-05-22T21:13:25.015087+0000","last_fullsized":"2024-05-22T21:13:25.015087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:07:36.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015546+0000","last_change":"2024-05-22T21:13:23.043385+0000","last_active":"2024-05-22T21:13:25.015546+0000","last_peered":"2024-05-22T21:13:25.015546+0000","last_clean":"2024-05-22T21:13:25.015546+0000","last_became_active":"2024-05-22T21:13:23.043036+0000","last_became_peered":"2024-05-22T21:13:23.043036+0000","last_unstale":"2024-05-22T21:13:25.015546+0000","last_undegraded":"2024-05-22T21:13:25.015546+0000","last_fullsized":"2024-05-22T21:13:25.015546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194661,"statfs":{"total":200000000,"available":199333641,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0369999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0920000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.923}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":75,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923027,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924501,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924465,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":75,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923027,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924591,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924431,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199334204,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]}]}],"pool_statfs":[]}} 2024-05-22T21:13:36.480 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:13:36.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.650+0000 7fcd8c400640 1 Processor -- start 2024-05-22T21:13:36.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.650+0000 7fcd8c400640 1 -- start start 2024-05-22T21:13:36.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.650+0000 7fcd8c400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcd8406d700 0x7fcd84168210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:36.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.650+0000 7fcd8c400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd84071fd0 con 0x7fcd8415d430 2024-05-22T21:13:36.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.650+0000 7fcd8c400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd84070c00 con 0x7fcd840707c0 2024-05-22T21:13:36.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.650+0000 7fcd8c400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd840acfc0 con 0x7fcd8406d700 2024-05-22T21:13:36.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.650+0000 7fcd89a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fcd8415d430 0x7fcd840ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43490/0 (socket says 172.21.15.161:43490) 2024-05-22T21:13:36.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.650+0000 7fcd89a00640 1 -- 172.21.15.161:0/3693253752 learned_addr learned my addr 172.21.15.161:0/3693253752 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:36.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.654+0000 7fcd8a400640 1 --2- 172.21.15.161:0/3693253752 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcd8406d700 0x7fcd84168210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:36.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.654+0000 7fcd83e00640 1 -- 172.21.15.161:0/3693253752 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154314371 0 0) 0x7fcd84071fd0 con 0x7fcd8415d430 2024-05-22T21:13:36.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.654+0000 7fcd83e00640 1 -- 172.21.15.161:0/3693253752 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd6c003530 con 0x7fcd8415d430 2024-05-22T21:13:36.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.654+0000 7fcd83e00640 1 -- 172.21.15.161:0/3693253752 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 826757950 0 0) 0x7fcd84070c00 con 0x7fcd840707c0 2024-05-22T21:13:36.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.654+0000 7fcd83e00640 1 -- 172.21.15.161:0/3693253752 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd84071fd0 con 0x7fcd840707c0 2024-05-22T21:13:36.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.654+0000 7fcd8a400640 1 -- 172.21.15.161:0/3693253752 >> v1:172.21.15.179:6789/0 conn(0x7fcd840707c0 legacy=0x7fcd8406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.654+0000 7fcd8a400640 1 -- 172.21.15.161:0/3693253752 >> v1:172.21.15.161:6789/0 conn(0x7fcd8415d430 legacy=0x7fcd840ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.654+0000 7fcd83e00640 1 -- 172.21.15.161:0/3693253752 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1214211150 0 0) 0x7fcd6c003530 con 0x7fcd8415d430 2024-05-22T21:13:36.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.654+0000 7fcd8a400640 1 -- 172.21.15.161:0/3693253752 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd84070e80 con 0x7fcd8406d700 2024-05-22T21:13:36.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.654+0000 7fcd8a400640 1 --2- 172.21.15.161:0/3693253752 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcd8406d700 0x7fcd84168210 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009dc0 tx=0x7fcd7c01c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=449c9bbef8f90f30 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:36.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.654+0000 7fcd83e00640 1 -- 172.21.15.161:0/3693253752 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd78001680 con 0x7fcd8406d700 2024-05-22T21:13:36.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.654+0000 7fcd83e00640 1 -- 172.21.15.161:0/3693253752 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7c0a55a0 con 0x7fcd8406d700 2024-05-22T21:13:36.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.654+0000 7fcd83e00640 1 -- 172.21.15.161:0/3693253752 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c0a5930 con 0x7fcd8406d700 2024-05-22T21:13:36.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd8c400640 1 -- 172.21.15.161:0/3693253752 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcd8406d700 msgr2=0x7fcd84168210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd8c400640 1 --2- 172.21.15.161:0/3693253752 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcd8406d700 0x7fcd84168210 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009dc0 tx=0x7fcd7c01c760 comp rx=0 tx=0).stop 2024-05-22T21:13:36.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd8c400640 1 -- 172.21.15.161:0/3693253752 shutdown_connections 2024-05-22T21:13:36.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd8c400640 1 --2- 172.21.15.161:0/3693253752 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcd8406d700 0x7fcd84168210 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd8c400640 1 -- 172.21.15.161:0/3693253752 >> 172.21.15.161:0/3693253752 conn(0x7fcd840b0bc0 msgr2=0x7fcd840b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:36.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd8c400640 1 -- 172.21.15.161:0/3693253752 shutdown_connections 2024-05-22T21:13:36.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd8c400640 1 -- 172.21.15.161:0/3693253752 wait complete. 2024-05-22T21:13:36.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd8c400640 1 Processor -- start 2024-05-22T21:13:36.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd8c400640 1 -- start start 2024-05-22T21:13:36.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd8c400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd8406d700 0x7fcd840b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:36.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd8c400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd840707c0 0x7fcd84151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:36.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd8c400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcd8415d430 0x7fcd841f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:36.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd8c400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8414edc0 con 0x7fcd8406d700 2024-05-22T21:13:36.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd8c400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8414eae0 con 0x7fcd840707c0 2024-05-22T21:13:36.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd8c400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd840af570 con 0x7fcd8415d430 2024-05-22T21:13:36.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd89a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd8406d700 0x7fcd840b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:36.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd89a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd8406d700 0x7fcd840b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54638/0 (socket says 172.21.15.161:54638) 2024-05-22T21:13:36.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd89a00640 1 -- 172.21.15.161:0/1406020119 learned_addr learned my addr 172.21.15.161:0/1406020119 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:36.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd8a400640 1 --2- 172.21.15.161:0/1406020119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcd8415d430 0x7fcd841f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:36.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd89000640 1 --2- 172.21.15.161:0/1406020119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd840707c0 0x7fcd84151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:36.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd89a00640 1 -- 172.21.15.161:0/1406020119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcd8415d430 msgr2=0x7fcd841f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd89a00640 1 --2- 172.21.15.161:0/1406020119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcd8415d430 0x7fcd841f96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd89a00640 1 -- 172.21.15.161:0/1406020119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd840707c0 msgr2=0x7fcd84151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd89a00640 1 --2- 172.21.15.161:0/1406020119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd840707c0 0x7fcd84151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.658+0000 7fcd89a00640 1 -- 172.21.15.161:0/1406020119 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c009a70 con 0x7fcd8406d700 2024-05-22T21:13:36.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.662+0000 7fcd89a00640 1 --2- 172.21.15.161:0/1406020119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd8406d700 0x7fcd840b0050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800c750 tx=0x7fcd7800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:36.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.662+0000 7fcd81600640 1 -- 172.21.15.161:0/1406020119 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd78005ae0 con 0x7fcd8406d700 2024-05-22T21:13:36.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.662+0000 7fcd81600640 1 -- 172.21.15.161:0/1406020119 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd78005c40 con 0x7fcd8406d700 2024-05-22T21:13:36.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.662+0000 7fcd81600640 1 -- 172.21.15.161:0/1406020119 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd78007070 con 0x7fcd8406d700 2024-05-22T21:13:36.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.662+0000 7fcd8c400640 1 -- 172.21.15.161:0/1406020119 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd840af850 con 0x7fcd8406d700 2024-05-22T21:13:36.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.662+0000 7fcd8c400640 1 -- 172.21.15.161:0/1406020119 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd84151ab0 con 0x7fcd8406d700 2024-05-22T21:13:36.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.662+0000 7fcd8c400640 1 -- 172.21.15.161:0/1406020119 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd84067220 con 0x7fcd8406d700 2024-05-22T21:13:36.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.662+0000 7fcd81600640 1 -- 172.21.15.161:0/1406020119 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcd780071d0 con 0x7fcd8406d700 2024-05-22T21:13:36.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.662+0000 7fcd81600640 1 --2- 172.21.15.161:0/1406020119 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcd64066690 0x7fcd64068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:36.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.662+0000 7fcd81600640 1 -- 172.21.15.161:0/1406020119 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd78083e30 con 0x7fcd8406d700 2024-05-22T21:13:36.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.662+0000 7fcd89000640 1 --2- 172.21.15.161:0/1406020119 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcd64066690 0x7fcd64068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:36.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.666+0000 7fcd89000640 1 --2- 172.21.15.161:0/1406020119 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcd64066690 0x7fcd64068b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcd8414e620 tx=0x7fcd70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:36.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.666+0000 7fcd81600640 1 -- 172.21.15.161:0/1406020119 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcd78051230 con 0x7fcd8406d700 2024-05-22T21:13:36.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.818+0000 7fcd8c400640 1 -- 172.21.15.161:0/1406020119 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd84071160 con 0x7fcd64066690 2024-05-22T21:13:36.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.822+0000 7fcd81600640 1 -- 172.21.15.161:0/1406020119 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260224 (secure 0 0 0) 0x7fcd700050d0 con 0x7fcd64066690 2024-05-22T21:13:36.830 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:36.831 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:13:36.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.826+0000 7fcd8c400640 1 -- 172.21.15.161:0/1406020119 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcd64066690 msgr2=0x7fcd64068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.826+0000 7fcd8c400640 1 --2- 172.21.15.161:0/1406020119 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcd64066690 0x7fcd64068b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcd8414e620 tx=0x7fcd70009040 comp rx=0 tx=0).stop 2024-05-22T21:13:36.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.826+0000 7fcd8c400640 1 -- 172.21.15.161:0/1406020119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd8406d700 msgr2=0x7fcd840b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:36.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.826+0000 7fcd8c400640 1 --2- 172.21.15.161:0/1406020119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd8406d700 0x7fcd840b0050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800c750 tx=0x7fcd7800cc10 comp rx=0 tx=0).stop 2024-05-22T21:13:36.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.826+0000 7fcd8c400640 1 -- 172.21.15.161:0/1406020119 shutdown_connections 2024-05-22T21:13:36.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.826+0000 7fcd8c400640 1 --2- 172.21.15.161:0/1406020119 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcd64066690 0x7fcd64068b50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.826+0000 7fcd8c400640 1 --2- 172.21.15.161:0/1406020119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcd8415d430 0x7fcd841f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.826+0000 7fcd8c400640 1 --2- 172.21.15.161:0/1406020119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd840707c0 0x7fcd84151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.826+0000 7fcd8c400640 1 --2- 172.21.15.161:0/1406020119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd8406d700 0x7fcd840b0050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:36.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.826+0000 7fcd8c400640 1 -- 172.21.15.161:0/1406020119 >> 172.21.15.161:0/1406020119 conn(0x7fcd840b0bc0 msgr2=0x7fcd840b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:36.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.826+0000 7fcd8c400640 1 -- 172.21.15.161:0/1406020119 shutdown_connections 2024-05-22T21:13:36.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:36.830+0000 7fcd8c400640 1 -- 172.21.15.161:0/1406020119 wait complete. 2024-05-22T21:13:36.850 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-05-22T21:13:36.398143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1746,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560753,"statfs":{"total":1600000000,"available":1598211887,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.975307"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428654+0000","last_change":"2024-05-22T21:13:23.445778+0000","last_active":"2024-05-22T21:13:24.428654+0000","last_peered":"2024-05-22T21:13:24.428654+0000","last_clean":"2024-05-22T21:13:24.428654+0000","last_became_active":"2024-05-22T21:13:23.445416+0000","last_became_peered":"2024-05-22T21:13:23.445416+0000","last_unstale":"2024-05-22T21:13:24.428654+0000","last_undegraded":"2024-05-22T21:13:24.428654+0000","last_fullsized":"2024-05-22T21:13:24.428654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:57:10.746221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423393+0000","last_change":"2024-05-22T21:13:23.429874+0000","last_active":"2024-05-22T21:13:24.423393+0000","last_peered":"2024-05-22T21:13:24.423393+0000","last_clean":"2024-05-22T21:13:24.423393+0000","last_became_active":"2024-05-22T21:13:23.428636+0000","last_became_peered":"2024-05-22T21:13:23.428636+0000","last_unstale":"2024-05-22T21:13:24.423393+0000","last_undegraded":"2024-05-22T21:13:24.423393+0000","last_fullsized":"2024-05-22T21:13:24.423393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:44:56.029080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015292+0000","last_change":"2024-05-22T21:13:23.434034+0000","last_active":"2024-05-22T21:13:25.015292+0000","last_peered":"2024-05-22T21:13:25.015292+0000","last_clean":"2024-05-22T21:13:25.015292+0000","last_became_active":"2024-05-22T21:13:23.432250+0000","last_became_peered":"2024-05-22T21:13:23.432250+0000","last_unstale":"2024-05-22T21:13:25.015292+0000","last_undegraded":"2024-05-22T21:13:25.015292+0000","last_fullsized":"2024-05-22T21:13:25.015292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:04:37.695966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919501+0000","last_change":"2024-05-22T21:13:23.440239+0000","last_active":"2024-05-22T21:13:24.919501+0000","last_peered":"2024-05-22T21:13:24.919501+0000","last_clean":"2024-05-22T21:13:24.919501+0000","last_became_active":"2024-05-22T21:13:23.433800+0000","last_became_peered":"2024-05-22T21:13:23.433800+0000","last_unstale":"2024-05-22T21:13:24.919501+0000","last_undegraded":"2024-05-22T21:13:24.919501+0000","last_fullsized":"2024-05-22T21:13:24.919501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:09:20.599937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429279+0000","last_change":"2024-05-22T21:13:23.436638+0000","last_active":"2024-05-22T21:13:24.429279+0000","last_peered":"2024-05-22T21:13:24.429279+0000","last_clean":"2024-05-22T21:13:24.429279+0000","last_became_active":"2024-05-22T21:13:23.436392+0000","last_became_peered":"2024-05-22T21:13:23.436392+0000","last_unstale":"2024-05-22T21:13:24.429279+0000","last_undegraded":"2024-05-22T21:13:24.429279+0000","last_fullsized":"2024-05-22T21:13:24.429279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:38:04.537935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919974+0000","last_change":"2024-05-22T21:13:23.442807+0000","last_active":"2024-05-22T21:13:24.919974+0000","last_peered":"2024-05-22T21:13:24.919974+0000","last_clean":"2024-05-22T21:13:24.919974+0000","last_became_active":"2024-05-22T21:13:23.441590+0000","last_became_peered":"2024-05-22T21:13:23.441590+0000","last_unstale":"2024-05-22T21:13:24.919974+0000","last_undegraded":"2024-05-22T21:13:24.919974+0000","last_fullsized":"2024-05-22T21:13:24.919974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:57:11.273220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429185+0000","last_change":"2024-05-22T21:13:23.436160+0000","last_active":"2024-05-22T21:13:24.429185+0000","last_peered":"2024-05-22T21:13:24.429185+0000","last_clean":"2024-05-22T21:13:24.429185+0000","last_became_active":"2024-05-22T21:13:23.435949+0000","last_became_peered":"2024-05-22T21:13:23.435949+0000","last_unstale":"2024-05-22T21:13:24.429185+0000","last_undegraded":"2024-05-22T21:13:24.429185+0000","last_fullsized":"2024-05-22T21:13:24.429185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:56:26.375232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014774+0000","last_change":"2024-05-22T21:13:23.437626+0000","last_active":"2024-05-22T21:13:25.014774+0000","last_peered":"2024-05-22T21:13:25.014774+0000","last_clean":"2024-05-22T21:13:25.014774+0000","last_became_active":"2024-05-22T21:13:23.437162+0000","last_became_peered":"2024-05-22T21:13:23.437162+0000","last_unstale":"2024-05-22T21:13:25.014774+0000","last_undegraded":"2024-05-22T21:13:25.014774+0000","last_fullsized":"2024-05-22T21:13:25.014774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:09:20.599937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428632+0000","last_change":"2024-05-22T21:13:23.446764+0000","last_active":"2024-05-22T21:13:24.428632+0000","last_peered":"2024-05-22T21:13:24.428632+0000","last_clean":"2024-05-22T21:13:24.428632+0000","last_became_active":"2024-05-22T21:13:23.446506+0000","last_became_peered":"2024-05-22T21:13:23.446506+0000","last_unstale":"2024-05-22T21:13:24.428632+0000","last_undegraded":"2024-05-22T21:13:24.428632+0000","last_fullsized":"2024-05-22T21:13:24.428632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:13:51.196274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014576+0000","last_change":"2024-05-22T21:13:23.450063+0000","last_active":"2024-05-22T21:13:25.014576+0000","last_peered":"2024-05-22T21:13:25.014576+0000","last_clean":"2024-05-22T21:13:25.014576+0000","last_became_active":"2024-05-22T21:13:23.449193+0000","last_became_peered":"2024-05-22T21:13:23.449193+0000","last_unstale":"2024-05-22T21:13:25.014576+0000","last_undegraded":"2024-05-22T21:13:25.014576+0000","last_fullsized":"2024-05-22T21:13:25.014576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:14:11.125539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014530+0000","last_change":"2024-05-22T21:13:23.449563+0000","last_active":"2024-05-22T21:13:25.014530+0000","last_peered":"2024-05-22T21:13:25.014530+0000","last_clean":"2024-05-22T21:13:25.014530+0000","last_became_active":"2024-05-22T21:13:23.449304+0000","last_became_peered":"2024-05-22T21:13:23.449304+0000","last_unstale":"2024-05-22T21:13:25.014530+0000","last_undegraded":"2024-05-22T21:13:25.014530+0000","last_fullsized":"2024-05-22T21:13:25.014530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:07:36.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428800+0000","last_change":"2024-05-22T21:13:23.444134+0000","last_active":"2024-05-22T21:13:24.428800+0000","last_peered":"2024-05-22T21:13:24.428800+0000","last_clean":"2024-05-22T21:13:24.428800+0000","last_became_active":"2024-05-22T21:13:23.443924+0000","last_became_peered":"2024-05-22T21:13:23.443924+0000","last_unstale":"2024-05-22T21:13:24.428800+0000","last_undegraded":"2024-05-22T21:13:24.428800+0000","last_fullsized":"2024-05-22T21:13:24.428800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:10.124782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014938+0000","last_change":"2024-05-22T21:13:23.447229+0000","last_active":"2024-05-22T21:13:25.014938+0000","last_peered":"2024-05-22T21:13:25.014938+0000","last_clean":"2024-05-22T21:13:25.014938+0000","last_became_active":"2024-05-22T21:13:23.447007+0000","last_became_peered":"2024-05-22T21:13:23.447007+0000","last_unstale":"2024-05-22T21:13:25.014938+0000","last_undegraded":"2024-05-22T21:13:25.014938+0000","last_fullsized":"2024-05-22T21:13:25.014938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:44:04.542487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014914+0000","last_change":"2024-05-22T21:13:23.433465+0000","last_active":"2024-05-22T21:13:25.014914+0000","last_peered":"2024-05-22T21:13:25.014914+0000","last_clean":"2024-05-22T21:13:25.014914+0000","last_became_active":"2024-05-22T21:13:23.432948+0000","last_became_peered":"2024-05-22T21:13:23.432948+0000","last_unstale":"2024-05-22T21:13:25.014914+0000","last_undegraded":"2024-05-22T21:13:25.014914+0000","last_fullsized":"2024-05-22T21:13:25.014914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:07:36.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423652+0000","last_change":"2024-05-22T21:13:23.426036+0000","last_active":"2024-05-22T21:13:24.423652+0000","last_peered":"2024-05-22T21:13:24.423652+0000","last_clean":"2024-05-22T21:13:24.423652+0000","last_became_active":"2024-05-22T21:13:23.425830+0000","last_became_peered":"2024-05-22T21:13:23.425830+0000","last_unstale":"2024-05-22T21:13:24.423652+0000","last_undegraded":"2024-05-22T21:13:24.423652+0000","last_fullsized":"2024-05-22T21:13:24.423652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:27.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919876+0000","last_change":"2024-05-22T21:13:23.442938+0000","last_active":"2024-05-22T21:13:24.919876+0000","last_peered":"2024-05-22T21:13:24.919876+0000","last_clean":"2024-05-22T21:13:24.919876+0000","last_became_active":"2024-05-22T21:13:23.441874+0000","last_became_peered":"2024-05-22T21:13:23.441874+0000","last_unstale":"2024-05-22T21:13:24.919876+0000","last_undegraded":"2024-05-22T21:13:24.919876+0000","last_fullsized":"2024-05-22T21:13:24.919876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:57:10.746221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919437+0000","last_change":"2024-05-22T21:13:23.444504+0000","last_active":"2024-05-22T21:13:24.919437+0000","last_peered":"2024-05-22T21:13:24.919437+0000","last_clean":"2024-05-22T21:13:24.919437+0000","last_became_active":"2024-05-22T21:13:23.443892+0000","last_became_peered":"2024-05-22T21:13:23.443892+0000","last_unstale":"2024-05-22T21:13:24.919437+0000","last_undegraded":"2024-05-22T21:13:24.919437+0000","last_fullsized":"2024-05-22T21:13:24.919437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:56:26.375232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014525+0000","last_change":"2024-05-22T21:13:23.447393+0000","last_active":"2024-05-22T21:13:25.014525+0000","last_peered":"2024-05-22T21:13:25.014525+0000","last_clean":"2024-05-22T21:13:25.014525+0000","last_became_active":"2024-05-22T21:13:23.446956+0000","last_became_peered":"2024-05-22T21:13:23.446956+0000","last_unstale":"2024-05-22T21:13:25.014525+0000","last_undegraded":"2024-05-22T21:13:25.014525+0000","last_fullsized":"2024-05-22T21:13:25.014525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:34:49.902120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428827+0000","last_change":"2024-05-22T21:13:23.437025+0000","last_active":"2024-05-22T21:13:24.428827+0000","last_peered":"2024-05-22T21:13:24.428827+0000","last_clean":"2024-05-22T21:13:24.428827+0000","last_became_active":"2024-05-22T21:13:23.436786+0000","last_became_peered":"2024-05-22T21:13:23.436786+0000","last_unstale":"2024-05-22T21:13:24.428827+0000","last_undegraded":"2024-05-22T21:13:24.428827+0000","last_fullsized":"2024-05-22T21:13:24.428827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:04:32.035089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014759+0000","last_change":"2024-05-22T21:13:23.444119+0000","last_active":"2024-05-22T21:13:25.014759+0000","last_peered":"2024-05-22T21:13:25.014759+0000","last_clean":"2024-05-22T21:13:25.014759+0000","last_became_active":"2024-05-22T21:13:23.443861+0000","last_became_peered":"2024-05-22T21:13:23.443861+0000","last_unstale":"2024-05-22T21:13:25.014759+0000","last_undegraded":"2024-05-22T21:13:25.014759+0000","last_fullsized":"2024-05-22T21:13:25.014759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:00:43.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015860+0000","last_change":"2024-05-22T21:13:23.434203+0000","last_active":"2024-05-22T21:13:25.015860+0000","last_peered":"2024-05-22T21:13:25.015860+0000","last_clean":"2024-05-22T21:13:25.015860+0000","last_became_active":"2024-05-22T21:13:23.433871+0000","last_became_peered":"2024-05-22T21:13:23.433871+0000","last_unstale":"2024-05-22T21:13:25.015860+0000","last_undegraded":"2024-05-22T21:13:25.015860+0000","last_fullsized":"2024-05-22T21:13:25.015860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:00:43.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919633+0000","last_change":"2024-05-22T21:13:23.444258+0000","last_active":"2024-05-22T21:13:24.919633+0000","last_peered":"2024-05-22T21:13:24.919633+0000","last_clean":"2024-05-22T21:13:24.919633+0000","last_became_active":"2024-05-22T21:13:23.443329+0000","last_became_peered":"2024-05-22T21:13:23.443329+0000","last_unstale":"2024-05-22T21:13:24.919633+0000","last_undegraded":"2024-05-22T21:13:24.919633+0000","last_fullsized":"2024-05-22T21:13:24.919633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015111+0000","last_change":"2024-05-22T21:13:23.437489+0000","last_active":"2024-05-22T21:13:25.015111+0000","last_peered":"2024-05-22T21:13:25.015111+0000","last_clean":"2024-05-22T21:13:25.015111+0000","last_became_active":"2024-05-22T21:13:23.436877+0000","last_became_peered":"2024-05-22T21:13:23.436877+0000","last_unstale":"2024-05-22T21:13:25.015111+0000","last_undegraded":"2024-05-22T21:13:25.015111+0000","last_fullsized":"2024-05-22T21:13:25.015111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:56:26.375232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919626+0000","last_change":"2024-05-22T21:13:23.434147+0000","last_active":"2024-05-22T21:13:24.919626+0000","last_peered":"2024-05-22T21:13:24.919626+0000","last_clean":"2024-05-22T21:13:24.919626+0000","last_became_active":"2024-05-22T21:13:23.433529+0000","last_became_peered":"2024-05-22T21:13:23.433529+0000","last_unstale":"2024-05-22T21:13:24.919626+0000","last_undegraded":"2024-05-22T21:13:24.919626+0000","last_fullsized":"2024-05-22T21:13:24.919626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:44:04.542487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015414+0000","last_change":"2024-05-22T21:13:23.435603+0000","last_active":"2024-05-22T21:13:25.015414+0000","last_peered":"2024-05-22T21:13:25.015414+0000","last_clean":"2024-05-22T21:13:25.015414+0000","last_became_active":"2024-05-22T21:13:23.435387+0000","last_became_peered":"2024-05-22T21:13:23.435387+0000","last_unstale":"2024-05-22T21:13:25.015414+0000","last_undegraded":"2024-05-22T21:13:25.015414+0000","last_fullsized":"2024-05-22T21:13:25.015414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:27.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014667+0000","last_change":"2024-05-22T21:13:23.444856+0000","last_active":"2024-05-22T21:13:25.014667+0000","last_peered":"2024-05-22T21:13:25.014667+0000","last_clean":"2024-05-22T21:13:25.014667+0000","last_became_active":"2024-05-22T21:13:23.444640+0000","last_became_peered":"2024-05-22T21:13:23.444640+0000","last_unstale":"2024-05-22T21:13:25.014667+0000","last_undegraded":"2024-05-22T21:13:25.014667+0000","last_fullsized":"2024-05-22T21:13:25.014667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:07:57.351276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423445+0000","last_change":"2024-05-22T21:13:23.430546+0000","last_active":"2024-05-22T21:13:24.423445+0000","last_peered":"2024-05-22T21:13:24.423445+0000","last_clean":"2024-05-22T21:13:24.423445+0000","last_became_active":"2024-05-22T21:13:23.430306+0000","last_became_peered":"2024-05-22T21:13:23.430306+0000","last_unstale":"2024-05-22T21:13:24.423445+0000","last_undegraded":"2024-05-22T21:13:24.423445+0000","last_fullsized":"2024-05-22T21:13:24.423445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:06.303665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015209+0000","last_change":"2024-05-22T21:13:23.437362+0000","last_active":"2024-05-22T21:13:25.015209+0000","last_peered":"2024-05-22T21:13:25.015209+0000","last_clean":"2024-05-22T21:13:25.015209+0000","last_became_active":"2024-05-22T21:13:23.436580+0000","last_became_peered":"2024-05-22T21:13:23.436580+0000","last_unstale":"2024-05-22T21:13:25.015209+0000","last_undegraded":"2024-05-22T21:13:25.015209+0000","last_fullsized":"2024-05-22T21:13:25.015209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:06.303665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015001+0000","last_change":"2024-05-22T21:13:23.433616+0000","last_active":"2024-05-22T21:13:25.015001+0000","last_peered":"2024-05-22T21:13:25.015001+0000","last_clean":"2024-05-22T21:13:25.015001+0000","last_became_active":"2024-05-22T21:13:23.433258+0000","last_became_peered":"2024-05-22T21:13:23.433258+0000","last_unstale":"2024-05-22T21:13:25.015001+0000","last_undegraded":"2024-05-22T21:13:25.015001+0000","last_fullsized":"2024-05-22T21:13:25.015001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:44:56.029080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919662+0000","last_change":"2024-05-22T21:13:23.435441+0000","last_active":"2024-05-22T21:13:24.919662+0000","last_peered":"2024-05-22T21:13:24.919662+0000","last_clean":"2024-05-22T21:13:24.919662+0000","last_became_active":"2024-05-22T21:13:23.435225+0000","last_became_peered":"2024-05-22T21:13:23.435225+0000","last_unstale":"2024-05-22T21:13:24.919662+0000","last_undegraded":"2024-05-22T21:13:24.919662+0000","last_fullsized":"2024-05-22T21:13:24.919662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:44:56.029080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014612+0000","last_change":"2024-05-22T21:13:23.446437+0000","last_active":"2024-05-22T21:13:25.014612+0000","last_peered":"2024-05-22T21:13:25.014612+0000","last_clean":"2024-05-22T21:13:25.014612+0000","last_became_active":"2024-05-22T21:13:23.446219+0000","last_became_peered":"2024-05-22T21:13:23.446219+0000","last_unstale":"2024-05-22T21:13:25.014612+0000","last_undegraded":"2024-05-22T21:13:25.014612+0000","last_fullsized":"2024-05-22T21:13:25.014612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:57:10.070412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014588+0000","last_change":"2024-05-22T21:13:23.443792+0000","last_active":"2024-05-22T21:13:25.014588+0000","last_peered":"2024-05-22T21:13:25.014588+0000","last_clean":"2024-05-22T21:13:25.014588+0000","last_became_active":"2024-05-22T21:13:23.443581+0000","last_became_peered":"2024-05-22T21:13:23.443581+0000","last_unstale":"2024-05-22T21:13:25.014588+0000","last_undegraded":"2024-05-22T21:13:25.014588+0000","last_fullsized":"2024-05-22T21:13:25.014588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.430114+0000","last_change":"2024-05-22T21:13:23.429085+0000","last_active":"2024-05-22T21:13:24.430114+0000","last_peered":"2024-05-22T21:13:24.430114+0000","last_clean":"2024-05-22T21:13:24.430114+0000","last_became_active":"2024-05-22T21:13:23.428826+0000","last_became_peered":"2024-05-22T21:13:23.428826+0000","last_unstale":"2024-05-22T21:13:24.430114+0000","last_undegraded":"2024-05-22T21:13:24.430114+0000","last_fullsized":"2024-05-22T21:13:24.430114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:17.109310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015138+0000","last_change":"2024-05-22T21:13:23.053644+0000","last_active":"2024-05-22T21:13:25.015138+0000","last_peered":"2024-05-22T21:13:25.015138+0000","last_clean":"2024-05-22T21:13:25.015138+0000","last_became_active":"2024-05-22T21:13:23.053129+0000","last_became_peered":"2024-05-22T21:13:23.053129+0000","last_unstale":"2024-05-22T21:13:25.015138+0000","last_undegraded":"2024-05-22T21:13:25.015138+0000","last_fullsized":"2024-05-22T21:13:25.015138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:02.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015372+0000","last_change":"2024-05-22T21:13:23.430851+0000","last_active":"2024-05-22T21:13:25.015372+0000","last_peered":"2024-05-22T21:13:25.015372+0000","last_clean":"2024-05-22T21:13:25.015372+0000","last_became_active":"2024-05-22T21:13:23.430614+0000","last_became_peered":"2024-05-22T21:13:23.430614+0000","last_unstale":"2024-05-22T21:13:25.015372+0000","last_undegraded":"2024-05-22T21:13:25.015372+0000","last_fullsized":"2024-05-22T21:13:25.015372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:14:11.125539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014815+0000","last_change":"2024-05-22T21:13:23.052090+0000","last_active":"2024-05-22T21:13:25.014815+0000","last_peered":"2024-05-22T21:13:25.014815+0000","last_clean":"2024-05-22T21:13:25.014815+0000","last_became_active":"2024-05-22T21:13:23.051762+0000","last_became_peered":"2024-05-22T21:13:23.051762+0000","last_unstale":"2024-05-22T21:13:25.014815+0000","last_undegraded":"2024-05-22T21:13:25.014815+0000","last_fullsized":"2024-05-22T21:13:25.014815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:54.533292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015255+0000","last_change":"2024-05-22T21:13:23.053832+0000","last_active":"2024-05-22T21:13:25.015255+0000","last_peered":"2024-05-22T21:13:25.015255+0000","last_clean":"2024-05-22T21:13:25.015255+0000","last_became_active":"2024-05-22T21:13:23.053170+0000","last_became_peered":"2024-05-22T21:13:23.053170+0000","last_unstale":"2024-05-22T21:13:25.015255+0000","last_undegraded":"2024-05-22T21:13:25.015255+0000","last_fullsized":"2024-05-22T21:13:25.015255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015495+0000","last_change":"2024-05-22T21:13:23.429829+0000","last_active":"2024-05-22T21:13:25.015495+0000","last_peered":"2024-05-22T21:13:25.015495+0000","last_clean":"2024-05-22T21:13:25.015495+0000","last_became_active":"2024-05-22T21:13:23.429616+0000","last_became_peered":"2024-05-22T21:13:23.429616+0000","last_unstale":"2024-05-22T21:13:25.015495+0000","last_undegraded":"2024-05-22T21:13:25.015495+0000","last_fullsized":"2024-05-22T21:13:25.015495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:00:48.798418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919805+0000","last_change":"2024-05-22T21:13:23.056945+0000","last_active":"2024-05-22T21:13:24.919805+0000","last_peered":"2024-05-22T21:13:24.919805+0000","last_clean":"2024-05-22T21:13:24.919805+0000","last_became_active":"2024-05-22T21:13:23.056643+0000","last_became_peered":"2024-05-22T21:13:23.056643+0000","last_unstale":"2024-05-22T21:13:24.919805+0000","last_undegraded":"2024-05-22T21:13:24.919805+0000","last_fullsized":"2024-05-22T21:13:24.919805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:02.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423490+0000","last_change":"2024-05-22T21:13:23.429959+0000","last_active":"2024-05-22T21:13:24.423490+0000","last_peered":"2024-05-22T21:13:24.423490+0000","last_clean":"2024-05-22T21:13:24.423490+0000","last_became_active":"2024-05-22T21:13:23.429752+0000","last_became_peered":"2024-05-22T21:13:23.429752+0000","last_unstale":"2024-05-22T21:13:24.423490+0000","last_undegraded":"2024-05-22T21:13:24.423490+0000","last_fullsized":"2024-05-22T21:13:24.423490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:14:11.125539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014851+0000","last_change":"2024-05-22T21:13:23.058878+0000","last_active":"2024-05-22T21:13:25.014851+0000","last_peered":"2024-05-22T21:13:25.014851+0000","last_clean":"2024-05-22T21:13:25.014851+0000","last_became_active":"2024-05-22T21:13:23.058648+0000","last_became_peered":"2024-05-22T21:13:23.058648+0000","last_unstale":"2024-05-22T21:13:25.014851+0000","last_undegraded":"2024-05-22T21:13:25.014851+0000","last_fullsized":"2024-05-22T21:13:25.014851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919552+0000","last_change":"2024-05-22T21:13:23.444383+0000","last_active":"2024-05-22T21:13:24.919552+0000","last_peered":"2024-05-22T21:13:24.919552+0000","last_clean":"2024-05-22T21:13:24.919552+0000","last_became_active":"2024-05-22T21:13:23.443612+0000","last_became_peered":"2024-05-22T21:13:23.443612+0000","last_unstale":"2024-05-22T21:13:24.919552+0000","last_undegraded":"2024-05-22T21:13:24.919552+0000","last_fullsized":"2024-05-22T21:13:24.919552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:27.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919883+0000","last_change":"2024-05-22T21:13:23.056793+0000","last_active":"2024-05-22T21:13:24.919883+0000","last_peered":"2024-05-22T21:13:24.919883+0000","last_clean":"2024-05-22T21:13:24.919883+0000","last_became_active":"2024-05-22T21:13:23.056450+0000","last_became_peered":"2024-05-22T21:13:23.056450+0000","last_unstale":"2024-05-22T21:13:24.919883+0000","last_undegraded":"2024-05-22T21:13:24.919883+0000","last_fullsized":"2024-05-22T21:13:24.919883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:34.410142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015942+0000","last_change":"2024-05-22T21:13:23.446398+0000","last_active":"2024-05-22T21:13:25.015942+0000","last_peered":"2024-05-22T21:13:25.015942+0000","last_clean":"2024-05-22T21:13:25.015942+0000","last_became_active":"2024-05-22T21:13:23.445907+0000","last_became_peered":"2024-05-22T21:13:23.445907+0000","last_unstale":"2024-05-22T21:13:25.015942+0000","last_undegraded":"2024-05-22T21:13:25.015942+0000","last_fullsized":"2024-05-22T21:13:25.015942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:06.303665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423552+0000","last_change":"2024-05-22T21:13:23.056570+0000","last_active":"2024-05-22T21:13:24.423552+0000","last_peered":"2024-05-22T21:13:24.423552+0000","last_clean":"2024-05-22T21:13:24.423552+0000","last_became_active":"2024-05-22T21:13:23.056357+0000","last_became_peered":"2024-05-22T21:13:23.056357+0000","last_unstale":"2024-05-22T21:13:24.423552+0000","last_undegraded":"2024-05-22T21:13:24.423552+0000","last_fullsized":"2024-05-22T21:13:24.423552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:30.101472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428611+0000","last_change":"2024-05-22T21:13:23.439128+0000","last_active":"2024-05-22T21:13:24.428611+0000","last_peered":"2024-05-22T21:13:24.428611+0000","last_clean":"2024-05-22T21:13:24.428611+0000","last_became_active":"2024-05-22T21:13:23.438898+0000","last_became_peered":"2024-05-22T21:13:23.438898+0000","last_unstale":"2024-05-22T21:13:24.428611+0000","last_undegraded":"2024-05-22T21:13:24.428611+0000","last_fullsized":"2024-05-22T21:13:24.428611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:51:29.569811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014610+0000","last_change":"2024-05-22T21:13:23.049600+0000","last_active":"2024-05-22T21:13:25.014610+0000","last_peered":"2024-05-22T21:13:25.014610+0000","last_clean":"2024-05-22T21:13:25.014610+0000","last_became_active":"2024-05-22T21:13:23.049395+0000","last_became_peered":"2024-05-22T21:13:23.049395+0000","last_unstale":"2024-05-22T21:13:25.014610+0000","last_undegraded":"2024-05-22T21:13:25.014610+0000","last_fullsized":"2024-05-22T21:13:25.014610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:24.879526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429576+0000","last_change":"2024-05-22T21:13:23.439228+0000","last_active":"2024-05-22T21:13:24.429576+0000","last_peered":"2024-05-22T21:13:24.429576+0000","last_clean":"2024-05-22T21:13:24.429576+0000","last_became_active":"2024-05-22T21:13:23.437292+0000","last_became_peered":"2024-05-22T21:13:23.437292+0000","last_unstale":"2024-05-22T21:13:24.429576+0000","last_undegraded":"2024-05-22T21:13:24.429576+0000","last_fullsized":"2024-05-22T21:13:24.429576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:44:04.542487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015531+0000","last_change":"2024-05-22T21:13:23.052267+0000","last_active":"2024-05-22T21:13:25.015531+0000","last_peered":"2024-05-22T21:13:25.015531+0000","last_clean":"2024-05-22T21:13:25.015531+0000","last_became_active":"2024-05-22T21:13:23.051663+0000","last_became_peered":"2024-05-22T21:13:23.051663+0000","last_unstale":"2024-05-22T21:13:25.015531+0000","last_undegraded":"2024-05-22T21:13:25.015531+0000","last_fullsized":"2024-05-22T21:13:25.015531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:54.533292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015989+0000","last_change":"2024-05-22T21:13:23.434082+0000","last_active":"2024-05-22T21:13:25.015989+0000","last_peered":"2024-05-22T21:13:25.015989+0000","last_clean":"2024-05-22T21:13:25.015989+0000","last_became_active":"2024-05-22T21:13:23.433591+0000","last_became_peered":"2024-05-22T21:13:23.433591+0000","last_unstale":"2024-05-22T21:13:25.015989+0000","last_undegraded":"2024-05-22T21:13:25.015989+0000","last_fullsized":"2024-05-22T21:13:25.015989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:08:18.923855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.920207+0000","last_change":"2024-05-22T21:13:23.058203+0000","last_active":"2024-05-22T21:13:24.920207+0000","last_peered":"2024-05-22T21:13:24.920207+0000","last_clean":"2024-05-22T21:13:24.920207+0000","last_became_active":"2024-05-22T21:13:23.057230+0000","last_became_peered":"2024-05-22T21:13:23.057230+0000","last_unstale":"2024-05-22T21:13:24.920207+0000","last_undegraded":"2024-05-22T21:13:24.920207+0000","last_fullsized":"2024-05-22T21:13:24.920207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:16.101596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429362+0000","last_change":"2024-05-22T21:13:23.437522+0000","last_active":"2024-05-22T21:13:24.429362+0000","last_peered":"2024-05-22T21:13:24.429362+0000","last_clean":"2024-05-22T21:13:24.429362+0000","last_became_active":"2024-05-22T21:13:23.436971+0000","last_became_peered":"2024-05-22T21:13:23.436971+0000","last_unstale":"2024-05-22T21:13:24.429362+0000","last_undegraded":"2024-05-22T21:13:24.429362+0000","last_fullsized":"2024-05-22T21:13:24.429362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:51:04.078274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014654+0000","last_change":"2024-05-22T21:13:23.056955+0000","last_active":"2024-05-22T21:13:25.014654+0000","last_peered":"2024-05-22T21:13:25.014654+0000","last_clean":"2024-05-22T21:13:25.014654+0000","last_became_active":"2024-05-22T21:13:23.056131+0000","last_became_peered":"2024-05-22T21:13:23.056131+0000","last_unstale":"2024-05-22T21:13:25.014654+0000","last_undegraded":"2024-05-22T21:13:25.014654+0000","last_fullsized":"2024-05-22T21:13:25.014654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:02.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014604+0000","last_change":"2024-05-22T21:13:23.445799+0000","last_active":"2024-05-22T21:13:25.014604+0000","last_peered":"2024-05-22T21:13:25.014604+0000","last_clean":"2024-05-22T21:13:25.014604+0000","last_became_active":"2024-05-22T21:13:23.445576+0000","last_became_peered":"2024-05-22T21:13:23.445576+0000","last_unstale":"2024-05-22T21:13:25.014604+0000","last_undegraded":"2024-05-22T21:13:25.014604+0000","last_fullsized":"2024-05-22T21:13:25.014604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:04:37.695966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014626+0000","last_change":"2024-05-22T21:13:23.046892+0000","last_active":"2024-05-22T21:13:25.014626+0000","last_peered":"2024-05-22T21:13:25.014626+0000","last_clean":"2024-05-22T21:13:25.014626+0000","last_became_active":"2024-05-22T21:13:23.046677+0000","last_became_peered":"2024-05-22T21:13:23.046677+0000","last_unstale":"2024-05-22T21:13:25.014626+0000","last_undegraded":"2024-05-22T21:13:25.014626+0000","last_fullsized":"2024-05-22T21:13:25.014626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:24.879526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429428+0000","last_change":"2024-05-22T21:13:23.437891+0000","last_active":"2024-05-22T21:13:24.429428+0000","last_peered":"2024-05-22T21:13:24.429428+0000","last_clean":"2024-05-22T21:13:24.429428+0000","last_became_active":"2024-05-22T21:13:23.436990+0000","last_became_peered":"2024-05-22T21:13:23.436990+0000","last_unstale":"2024-05-22T21:13:24.429428+0000","last_undegraded":"2024-05-22T21:13:24.429428+0000","last_fullsized":"2024-05-22T21:13:24.429428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:12:43.781854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428700+0000","last_change":"2024-05-22T21:13:23.054659+0000","last_active":"2024-05-22T21:13:24.428700+0000","last_peered":"2024-05-22T21:13:24.428700+0000","last_clean":"2024-05-22T21:13:24.428700+0000","last_became_active":"2024-05-22T21:13:23.054403+0000","last_became_peered":"2024-05-22T21:13:23.054403+0000","last_unstale":"2024-05-22T21:13:24.428700+0000","last_undegraded":"2024-05-22T21:13:24.428700+0000","last_fullsized":"2024-05-22T21:13:24.428700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:26:28.755372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429419+0000","last_change":"2024-05-22T21:13:23.437668+0000","last_active":"2024-05-22T21:13:24.429419+0000","last_peered":"2024-05-22T21:13:24.429419+0000","last_clean":"2024-05-22T21:13:24.429419+0000","last_became_active":"2024-05-22T21:13:23.437307+0000","last_became_peered":"2024-05-22T21:13:23.437307+0000","last_unstale":"2024-05-22T21:13:24.429419+0000","last_undegraded":"2024-05-22T21:13:24.429419+0000","last_fullsized":"2024-05-22T21:13:24.429419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:57:11.273220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014824+0000","last_change":"2024-05-22T21:13:23.047615+0000","last_active":"2024-05-22T21:13:25.014824+0000","last_peered":"2024-05-22T21:13:25.014824+0000","last_clean":"2024-05-22T21:13:25.014824+0000","last_became_active":"2024-05-22T21:13:23.047358+0000","last_became_peered":"2024-05-22T21:13:23.047358+0000","last_unstale":"2024-05-22T21:13:25.014824+0000","last_undegraded":"2024-05-22T21:13:25.014824+0000","last_fullsized":"2024-05-22T21:13:25.014824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:34.410142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428730+0000","last_change":"2024-05-22T21:13:23.443073+0000","last_active":"2024-05-22T21:13:24.428730+0000","last_peered":"2024-05-22T21:13:24.428730+0000","last_clean":"2024-05-22T21:13:24.428730+0000","last_became_active":"2024-05-22T21:13:23.442751+0000","last_became_peered":"2024-05-22T21:13:23.442751+0000","last_unstale":"2024-05-22T21:13:24.428730+0000","last_undegraded":"2024-05-22T21:13:24.428730+0000","last_fullsized":"2024-05-22T21:13:24.428730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:09:20.599937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423443+0000","last_change":"2024-05-22T21:13:23.050892+0000","last_active":"2024-05-22T21:13:24.423443+0000","last_peered":"2024-05-22T21:13:24.423443+0000","last_clean":"2024-05-22T21:13:24.423443+0000","last_became_active":"2024-05-22T21:13:23.050565+0000","last_became_peered":"2024-05-22T21:13:23.050565+0000","last_unstale":"2024-05-22T21:13:24.423443+0000","last_undegraded":"2024-05-22T21:13:24.423443+0000","last_fullsized":"2024-05-22T21:13:24.423443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:16.101596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919723+0000","last_change":"2024-05-22T21:13:23.437191+0000","last_active":"2024-05-22T21:13:24.919723+0000","last_peered":"2024-05-22T21:13:24.919723+0000","last_clean":"2024-05-22T21:13:24.919723+0000","last_became_active":"2024-05-22T21:13:23.436983+0000","last_became_peered":"2024-05-22T21:13:23.436983+0000","last_unstale":"2024-05-22T21:13:24.919723+0000","last_undegraded":"2024-05-22T21:13:24.919723+0000","last_fullsized":"2024-05-22T21:13:24.919723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:06.303665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014631+0000","last_change":"2024-05-22T21:13:23.056461+0000","last_active":"2024-05-22T21:13:25.014631+0000","last_peered":"2024-05-22T21:13:25.014631+0000","last_clean":"2024-05-22T21:13:25.014631+0000","last_became_active":"2024-05-22T21:13:23.056237+0000","last_became_peered":"2024-05-22T21:13:23.056237+0000","last_unstale":"2024-05-22T21:13:25.014631+0000","last_undegraded":"2024-05-22T21:13:25.014631+0000","last_fullsized":"2024-05-22T21:13:25.014631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428905+0000","last_change":"2024-05-22T21:13:23.448377+0000","last_active":"2024-05-22T21:13:24.428905+0000","last_peered":"2024-05-22T21:13:24.428905+0000","last_clean":"2024-05-22T21:13:24.428905+0000","last_became_active":"2024-05-22T21:13:23.447835+0000","last_became_peered":"2024-05-22T21:13:23.447835+0000","last_unstale":"2024-05-22T21:13:24.428905+0000","last_undegraded":"2024-05-22T21:13:24.428905+0000","last_fullsized":"2024-05-22T21:13:24.428905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:44:10.866789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-22T21:13:23.435118+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:13:23.435118+0000","last_peered":"2024-05-22T21:13:23.435118+0000","last_clean":"2024-05-22T21:13:23.435118+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:13:23.435118+0000","last_undegraded":"2024-05-22T21:13:23.435118+0000","last_fullsized":"2024-05-22T21:13:23.435118+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.920353+0000","last_change":"2024-05-22T21:13:23.058027+0000","last_active":"2024-05-22T21:13:24.920353+0000","last_peered":"2024-05-22T21:13:24.920353+0000","last_clean":"2024-05-22T21:13:24.920353+0000","last_became_active":"2024-05-22T21:13:23.056401+0000","last_became_peered":"2024-05-22T21:13:23.056401+0000","last_unstale":"2024-05-22T21:13:24.920353+0000","last_undegraded":"2024-05-22T21:13:24.920353+0000","last_fullsized":"2024-05-22T21:13:24.920353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:24.879526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014947+0000","last_change":"2024-05-22T21:13:23.440325+0000","last_active":"2024-05-22T21:13:25.014947+0000","last_peered":"2024-05-22T21:13:25.014947+0000","last_clean":"2024-05-22T21:13:25.014947+0000","last_became_active":"2024-05-22T21:13:23.440020+0000","last_became_peered":"2024-05-22T21:13:23.440020+0000","last_unstale":"2024-05-22T21:13:25.014947+0000","last_undegraded":"2024-05-22T21:13:25.014947+0000","last_fullsized":"2024-05-22T21:13:25.014947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:38:04.537935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429005+0000","last_change":"2024-05-22T21:13:23.050452+0000","last_active":"2024-05-22T21:13:24.429005+0000","last_peered":"2024-05-22T21:13:24.429005+0000","last_clean":"2024-05-22T21:13:24.429005+0000","last_became_active":"2024-05-22T21:13:23.049982+0000","last_became_peered":"2024-05-22T21:13:23.049982+0000","last_unstale":"2024-05-22T21:13:24.429005+0000","last_undegraded":"2024-05-22T21:13:24.429005+0000","last_fullsized":"2024-05-22T21:13:24.429005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:57:09.062698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015492+0000","last_change":"2024-05-22T21:13:23.434429+0000","last_active":"2024-05-22T21:13:25.015492+0000","last_peered":"2024-05-22T21:13:25.015492+0000","last_clean":"2024-05-22T21:13:25.015492+0000","last_became_active":"2024-05-22T21:13:23.433640+0000","last_became_peered":"2024-05-22T21:13:23.433640+0000","last_unstale":"2024-05-22T21:13:25.015492+0000","last_undegraded":"2024-05-22T21:13:25.015492+0000","last_fullsized":"2024-05-22T21:13:25.015492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014707+0000","last_change":"2024-05-22T21:13:23.051179+0000","last_active":"2024-05-22T21:13:25.014707+0000","last_peered":"2024-05-22T21:13:25.014707+0000","last_clean":"2024-05-22T21:13:25.014707+0000","last_became_active":"2024-05-22T21:13:23.050967+0000","last_became_peered":"2024-05-22T21:13:23.050967+0000","last_unstale":"2024-05-22T21:13:25.014707+0000","last_undegraded":"2024-05-22T21:13:25.014707+0000","last_fullsized":"2024-05-22T21:13:25.014707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:02.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423581+0000","last_change":"2024-05-22T21:13:23.426627+0000","last_active":"2024-05-22T21:13:24.423581+0000","last_peered":"2024-05-22T21:13:24.423581+0000","last_clean":"2024-05-22T21:13:24.423581+0000","last_became_active":"2024-05-22T21:13:23.426412+0000","last_became_peered":"2024-05-22T21:13:23.426412+0000","last_unstale":"2024-05-22T21:13:24.423581+0000","last_undegraded":"2024-05-22T21:13:24.423581+0000","last_fullsized":"2024-05-22T21:13:24.423581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919933+0000","last_change":"2024-05-22T21:13:23.055924+0000","last_active":"2024-05-22T21:13:24.919933+0000","last_peered":"2024-05-22T21:13:24.919933+0000","last_clean":"2024-05-22T21:13:24.919933+0000","last_became_active":"2024-05-22T21:13:23.055577+0000","last_became_peered":"2024-05-22T21:13:23.055577+0000","last_unstale":"2024-05-22T21:13:24.919933+0000","last_undegraded":"2024-05-22T21:13:24.919933+0000","last_fullsized":"2024-05-22T21:13:24.919933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.430340+0000","last_change":"2024-05-22T21:13:23.428954+0000","last_active":"2024-05-22T21:13:24.430340+0000","last_peered":"2024-05-22T21:13:24.430340+0000","last_clean":"2024-05-22T21:13:24.430340+0000","last_became_active":"2024-05-22T21:13:23.428628+0000","last_became_peered":"2024-05-22T21:13:23.428628+0000","last_unstale":"2024-05-22T21:13:24.430340+0000","last_undegraded":"2024-05-22T21:13:24.430340+0000","last_fullsized":"2024-05-22T21:13:24.430340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:31.109187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423797+0000","last_change":"2024-05-22T21:13:23.055046+0000","last_active":"2024-05-22T21:13:24.423797+0000","last_peered":"2024-05-22T21:13:24.423797+0000","last_clean":"2024-05-22T21:13:24.423797+0000","last_became_active":"2024-05-22T21:13:23.054670+0000","last_became_peered":"2024-05-22T21:13:23.054670+0000","last_unstale":"2024-05-22T21:13:24.423797+0000","last_undegraded":"2024-05-22T21:13:24.423797+0000","last_fullsized":"2024-05-22T21:13:24.423797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:15:24.879526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919722+0000","last_change":"2024-05-22T21:13:23.434008+0000","last_active":"2024-05-22T21:13:24.919722+0000","last_peered":"2024-05-22T21:13:24.919722+0000","last_clean":"2024-05-22T21:13:24.919722+0000","last_became_active":"2024-05-22T21:13:23.433233+0000","last_became_peered":"2024-05-22T21:13:23.433233+0000","last_unstale":"2024-05-22T21:13:24.919722+0000","last_undegraded":"2024-05-22T21:13:24.919722+0000","last_fullsized":"2024-05-22T21:13:24.919722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:38:04.537935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.428911+0000","last_change":"2024-05-22T21:13:23.041563+0000","last_active":"2024-05-22T21:13:24.428911+0000","last_peered":"2024-05-22T21:13:24.428911+0000","last_clean":"2024-05-22T21:13:24.428911+0000","last_became_active":"2024-05-22T21:13:23.041302+0000","last_became_peered":"2024-05-22T21:13:23.041302+0000","last_unstale":"2024-05-22T21:13:24.428911+0000","last_undegraded":"2024-05-22T21:13:24.428911+0000","last_fullsized":"2024-05-22T21:13:24.428911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:51:28.562097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423568+0000","last_change":"2024-05-22T21:13:23.426584+0000","last_active":"2024-05-22T21:13:24.423568+0000","last_peered":"2024-05-22T21:13:24.423568+0000","last_clean":"2024-05-22T21:13:24.423568+0000","last_became_active":"2024-05-22T21:13:23.426367+0000","last_became_peered":"2024-05-22T21:13:23.426367+0000","last_unstale":"2024-05-22T21:13:24.423568+0000","last_undegraded":"2024-05-22T21:13:24.423568+0000","last_fullsized":"2024-05-22T21:13:24.423568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:07:36.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:26.399531+0000","last_change":"2024-05-22T21:13:23.049729+0000","last_active":"2024-05-22T21:13:26.399531+0000","last_peered":"2024-05-22T21:13:26.399531+0000","last_clean":"2024-05-22T21:13:26.399531+0000","last_became_active":"2024-05-22T21:13:23.049480+0000","last_became_peered":"2024-05-22T21:13:23.049480+0000","last_unstale":"2024-05-22T21:13:26.399531+0000","last_undegraded":"2024-05-22T21:13:26.399531+0000","last_fullsized":"2024-05-22T21:13:26.399531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:51:28.562097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014848+0000","last_change":"2024-05-22T21:13:23.437853+0000","last_active":"2024-05-22T21:13:25.014848+0000","last_peered":"2024-05-22T21:13:25.014848+0000","last_clean":"2024-05-22T21:13:25.014848+0000","last_became_active":"2024-05-22T21:13:23.437676+0000","last_became_peered":"2024-05-22T21:13:23.437676+0000","last_unstale":"2024-05-22T21:13:25.014848+0000","last_undegraded":"2024-05-22T21:13:25.014848+0000","last_fullsized":"2024-05-22T21:13:25.014848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:14:11.125539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014750+0000","last_change":"2024-05-22T21:13:23.052969+0000","last_active":"2024-05-22T21:13:25.014750+0000","last_peered":"2024-05-22T21:13:25.014750+0000","last_clean":"2024-05-22T21:13:25.014750+0000","last_became_active":"2024-05-22T21:13:23.052306+0000","last_became_peered":"2024-05-22T21:13:23.052306+0000","last_unstale":"2024-05-22T21:13:25.014750+0000","last_undegraded":"2024-05-22T21:13:25.014750+0000","last_fullsized":"2024-05-22T21:13:25.014750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:34.410142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015367+0000","last_change":"2024-05-22T21:13:23.434301+0000","last_active":"2024-05-22T21:13:25.015367+0000","last_peered":"2024-05-22T21:13:25.015367+0000","last_clean":"2024-05-22T21:13:25.015367+0000","last_became_active":"2024-05-22T21:13:23.432584+0000","last_became_peered":"2024-05-22T21:13:23.432584+0000","last_unstale":"2024-05-22T21:13:25.015367+0000","last_undegraded":"2024-05-22T21:13:25.015367+0000","last_fullsized":"2024-05-22T21:13:25.015367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:53:57.578626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429187+0000","last_change":"2024-05-22T21:13:23.047297+0000","last_active":"2024-05-22T21:13:24.429187+0000","last_peered":"2024-05-22T21:13:24.429187+0000","last_clean":"2024-05-22T21:13:24.429187+0000","last_became_active":"2024-05-22T21:13:23.047021+0000","last_became_peered":"2024-05-22T21:13:23.047021+0000","last_unstale":"2024-05-22T21:13:24.429187+0000","last_undegraded":"2024-05-22T21:13:24.429187+0000","last_fullsized":"2024-05-22T21:13:24.429187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:09.117068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429366+0000","last_change":"2024-05-22T21:13:23.434654+0000","last_active":"2024-05-22T21:13:24.429366+0000","last_peered":"2024-05-22T21:13:24.429366+0000","last_clean":"2024-05-22T21:13:24.429366+0000","last_became_active":"2024-05-22T21:13:23.433684+0000","last_became_peered":"2024-05-22T21:13:23.433684+0000","last_unstale":"2024-05-22T21:13:24.429366+0000","last_undegraded":"2024-05-22T21:13:24.429366+0000","last_fullsized":"2024-05-22T21:13:24.429366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:03.482406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423727+0000","last_change":"2024-05-22T21:13:23.054907+0000","last_active":"2024-05-22T21:13:24.423727+0000","last_peered":"2024-05-22T21:13:24.423727+0000","last_clean":"2024-05-22T21:13:24.423727+0000","last_became_active":"2024-05-22T21:13:23.054307+0000","last_became_peered":"2024-05-22T21:13:23.054307+0000","last_unstale":"2024-05-22T21:13:24.423727+0000","last_undegraded":"2024-05-22T21:13:24.423727+0000","last_fullsized":"2024-05-22T21:13:24.423727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:09.117068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015009+0000","last_change":"2024-05-22T21:13:23.429322+0000","last_active":"2024-05-22T21:13:25.015009+0000","last_peered":"2024-05-22T21:13:25.015009+0000","last_clean":"2024-05-22T21:13:25.015009+0000","last_became_active":"2024-05-22T21:13:23.429091+0000","last_became_peered":"2024-05-22T21:13:23.429091+0000","last_unstale":"2024-05-22T21:13:25.015009+0000","last_undegraded":"2024-05-22T21:13:25.015009+0000","last_fullsized":"2024-05-22T21:13:25.015009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:44:56.029080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.423657+0000","last_change":"2024-05-22T21:13:23.052267+0000","last_active":"2024-05-22T21:13:24.423657+0000","last_peered":"2024-05-22T21:13:24.423657+0000","last_clean":"2024-05-22T21:13:24.423657+0000","last_became_active":"2024-05-22T21:13:23.052062+0000","last_became_peered":"2024-05-22T21:13:23.052062+0000","last_unstale":"2024-05-22T21:13:24.423657+0000","last_undegraded":"2024-05-22T21:13:24.423657+0000","last_fullsized":"2024-05-22T21:13:24.423657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:34:48.894406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015300+0000","last_change":"2024-05-22T21:13:23.433439+0000","last_active":"2024-05-22T21:13:25.015300+0000","last_peered":"2024-05-22T21:13:25.015300+0000","last_clean":"2024-05-22T21:13:25.015300+0000","last_became_active":"2024-05-22T21:13:23.433016+0000","last_became_peered":"2024-05-22T21:13:23.433016+0000","last_unstale":"2024-05-22T21:13:25.015300+0000","last_undegraded":"2024-05-22T21:13:25.015300+0000","last_fullsized":"2024-05-22T21:13:25.015300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:42:10.333828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429655+0000","last_change":"2024-05-22T21:13:23.052977+0000","last_active":"2024-05-22T21:13:24.429655+0000","last_peered":"2024-05-22T21:13:24.429655+0000","last_clean":"2024-05-22T21:13:24.429655+0000","last_became_active":"2024-05-22T21:13:23.052490+0000","last_became_peered":"2024-05-22T21:13:23.052490+0000","last_unstale":"2024-05-22T21:13:24.429655+0000","last_undegraded":"2024-05-22T21:13:24.429655+0000","last_fullsized":"2024-05-22T21:13:24.429655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:28:16.101596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.014517+0000","last_change":"2024-05-22T21:13:23.446126+0000","last_active":"2024-05-22T21:13:25.014517+0000","last_peered":"2024-05-22T21:13:25.014517+0000","last_clean":"2024-05-22T21:13:25.014517+0000","last_became_active":"2024-05-22T21:13:23.444835+0000","last_became_peered":"2024-05-22T21:13:23.444835+0000","last_unstale":"2024-05-22T21:13:25.014517+0000","last_undegraded":"2024-05-22T21:13:25.014517+0000","last_fullsized":"2024-05-22T21:13:25.014517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:53:57.578626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.919747+0000","last_change":"2024-05-22T21:13:23.432060+0000","last_active":"2024-05-22T21:13:24.919747+0000","last_peered":"2024-05-22T21:13:24.919747+0000","last_clean":"2024-05-22T21:13:24.919747+0000","last_became_active":"2024-05-22T21:13:23.431845+0000","last_became_peered":"2024-05-22T21:13:23.431845+0000","last_unstale":"2024-05-22T21:13:24.919747+0000","last_undegraded":"2024-05-22T21:13:24.919747+0000","last_fullsized":"2024-05-22T21:13:24.919747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:44:10.866789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429092+0000","last_change":"2024-05-22T21:13:23.058221+0000","last_active":"2024-05-22T21:13:24.429092+0000","last_peered":"2024-05-22T21:13:24.429092+0000","last_clean":"2024-05-22T21:13:24.429092+0000","last_became_active":"2024-05-22T21:13:23.058012+0000","last_became_peered":"2024-05-22T21:13:23.058012+0000","last_unstale":"2024-05-22T21:13:24.429092+0000","last_undegraded":"2024-05-22T21:13:24.429092+0000","last_fullsized":"2024-05-22T21:13:24.429092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:34:48.894406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.920057+0000","last_change":"2024-05-22T21:13:23.431267+0000","last_active":"2024-05-22T21:13:24.920057+0000","last_peered":"2024-05-22T21:13:24.920057+0000","last_clean":"2024-05-22T21:13:24.920057+0000","last_became_active":"2024-05-22T21:13:23.431054+0000","last_became_peered":"2024-05-22T21:13:23.431054+0000","last_unstale":"2024-05-22T21:13:24.920057+0000","last_undegraded":"2024-05-22T21:13:24.920057+0000","last_fullsized":"2024-05-22T21:13:24.920057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:51:04.078274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:24.429474+0000","last_change":"2024-05-22T21:13:23.051743+0000","last_active":"2024-05-22T21:13:24.429474+0000","last_peered":"2024-05-22T21:13:24.429474+0000","last_clean":"2024-05-22T21:13:24.429474+0000","last_became_active":"2024-05-22T21:13:23.051564+0000","last_became_peered":"2024-05-22T21:13:23.051564+0000","last_unstale":"2024-05-22T21:13:24.429474+0000","last_undegraded":"2024-05-22T21:13:24.429474+0000","last_fullsized":"2024-05-22T21:13:24.429474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:30.101472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015432+0000","last_change":"2024-05-22T21:13:23.433871+0000","last_active":"2024-05-22T21:13:25.015432+0000","last_peered":"2024-05-22T21:13:25.015432+0000","last_clean":"2024-05-22T21:13:25.015432+0000","last_became_active":"2024-05-22T21:13:23.431365+0000","last_became_peered":"2024-05-22T21:13:23.431365+0000","last_unstale":"2024-05-22T21:13:25.015432+0000","last_undegraded":"2024-05-22T21:13:25.015432+0000","last_fullsized":"2024-05-22T21:13:25.015432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:27.057629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015179+0000","last_change":"2024-05-22T21:13:23.046747+0000","last_active":"2024-05-22T21:13:25.015179+0000","last_peered":"2024-05-22T21:13:25.015179+0000","last_clean":"2024-05-22T21:13:25.015179+0000","last_became_active":"2024-05-22T21:13:23.046523+0000","last_became_peered":"2024-05-22T21:13:23.046523+0000","last_unstale":"2024-05-22T21:13:25.015179+0000","last_undegraded":"2024-05-22T21:13:25.015179+0000","last_fullsized":"2024-05-22T21:13:25.015179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:34.410142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015087+0000","last_change":"2024-05-22T21:13:23.447604+0000","last_active":"2024-05-22T21:13:25.015087+0000","last_peered":"2024-05-22T21:13:25.015087+0000","last_clean":"2024-05-22T21:13:25.015087+0000","last_became_active":"2024-05-22T21:13:23.447391+0000","last_became_peered":"2024-05-22T21:13:23.447391+0000","last_unstale":"2024-05-22T21:13:25.015087+0000","last_undegraded":"2024-05-22T21:13:25.015087+0000","last_fullsized":"2024-05-22T21:13:25.015087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:23.032353+0000","last_clean_scrub_stamp":"2024-05-22T21:13:23.032353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:07:36.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-22T21:13:25.015546+0000","last_change":"2024-05-22T21:13:23.043385+0000","last_active":"2024-05-22T21:13:25.015546+0000","last_peered":"2024-05-22T21:13:25.015546+0000","last_clean":"2024-05-22T21:13:25.015546+0000","last_became_active":"2024-05-22T21:13:23.043036+0000","last_became_peered":"2024-05-22T21:13:23.043036+0000","last_unstale":"2024-05-22T21:13:25.015546+0000","last_undegraded":"2024-05-22T21:13:25.015546+0000","last_fullsized":"2024-05-22T21:13:25.015546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:22.024639+0000","last_clean_scrub_stamp":"2024-05-22T21:13:22.024639+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:26:49.981453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194661,"statfs":{"total":200000000,"available":199333641,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0369999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0920000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.923}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":75,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923027,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924501,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924465,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":75,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923027,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924591,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924431,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199334204,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]}]}],"pool_statfs":[]}} 2024-05-22T21:13:36.854 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-22T21:13:36.854 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-22T21:13:36.854 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-22T21:13:36.855 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-22T21:13:37.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.022+0000 7fdbc3800640 1 Processor -- start 2024-05-22T21:13:37.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.022+0000 7fdbc3800640 1 -- start start 2024-05-22T21:13:37.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.022+0000 7fdbc3800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdbbc15d4a0 0x7fdbbc099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.022+0000 7fdbc3800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbbc0b0490 0x7fdbbc099f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.022+0000 7fdbc3800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbbc06c440 con 0x7fdbbc06bfb0 2024-05-22T21:13:37.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.022+0000 7fdbc3800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbbc15d880 con 0x7fdbbc15d4a0 2024-05-22T21:13:37.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.022+0000 7fdbc3800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc06b440 con 0x7fdbbc0b0490 2024-05-22T21:13:37.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.022+0000 7fdbc0e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdbbc15d4a0 0x7fdbbc099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.022+0000 7fdbb3e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbbc0b0490 0x7fdbbc099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.022+0000 7fdbc0e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdbbc15d4a0 0x7fdbbc099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:49498/0 (socket says 172.21.15.161:49498) 2024-05-22T21:13:37.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.022+0000 7fdbc0e00640 1 -- 172.21.15.161:0/1720364020 learned_addr learned my addr 172.21.15.161:0/1720364020 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:37.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.022+0000 7fdbc0e00640 1 -- 172.21.15.161:0/1720364020 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbbc0b0490 msgr2=0x7fdbbc099f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:37.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.022+0000 7fdbc0e00640 1 --2- 172.21.15.161:0/1720364020 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbbc0b0490 0x7fdbbc099f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.022+0000 7fdbc0e00640 1 -- 172.21.15.161:0/1720364020 >> v1:172.21.15.179:6789/0 conn(0x7fdbbc06bfb0 legacy=0x7fdbbc160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.022+0000 7fdbc0e00640 1 -- 172.21.15.161:0/1720364020 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc0cf190 con 0x7fdbbc15d4a0 2024-05-22T21:13:37.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.022+0000 7fdbb3e00640 1 --2- 172.21.15.161:0/1720364020 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbbc0b0490 0x7fdbbc099f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:37.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.026+0000 7fdbc0e00640 1 --2- 172.21.15.161:0/1720364020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbbc15d4a0 0x7fdbbc099a40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009870 tx=0x7fdbac01c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8df435938004544a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:37.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.026+0000 7fdbbbe00640 1 -- 172.21.15.161:0/1720364020 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0a95f0 con 0x7fdbbc15d4a0 2024-05-22T21:13:37.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.026+0000 7fdbbbe00640 1 -- 172.21.15.161:0/1720364020 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbac0a9750 con 0x7fdbbc15d4a0 2024-05-22T21:13:37.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.026+0000 7fdbbbe00640 1 -- 172.21.15.161:0/1720364020 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0a9ae0 con 0x7fdbbc15d4a0 2024-05-22T21:13:37.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.026+0000 7fdbc3800640 1 -- 172.21.15.161:0/1720364020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbbc15d4a0 msgr2=0x7fdbbc099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.026+0000 7fdbc3800640 1 --2- 172.21.15.161:0/1720364020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbbc15d4a0 0x7fdbbc099a40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009870 tx=0x7fdbac01c530 comp rx=0 tx=0).stop 2024-05-22T21:13:37.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.026+0000 7fdbc3800640 1 -- 172.21.15.161:0/1720364020 shutdown_connections 2024-05-22T21:13:37.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.026+0000 7fdbc3800640 1 --2- 172.21.15.161:0/1720364020 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbbc0b0490 0x7fdbbc099f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.026+0000 7fdbc3800640 1 --2- 172.21.15.161:0/1720364020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbbc15d4a0 0x7fdbbc099a40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.026+0000 7fdbc3800640 1 -- 172.21.15.161:0/1720364020 >> 172.21.15.161:0/1720364020 conn(0x7fdbbc0b2a40 msgr2=0x7fdbbc0b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:37.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.026+0000 7fdbc3800640 1 -- 172.21.15.161:0/1720364020 shutdown_connections 2024-05-22T21:13:37.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.026+0000 7fdbc3800640 1 -- 172.21.15.161:0/1720364020 wait complete. 2024-05-22T21:13:37.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc3800640 1 Processor -- start 2024-05-22T21:13:37.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc3800640 1 -- start start 2024-05-22T21:13:37.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc3800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbbc0b0490 0x7fdbbc160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc3800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbbc15d4a0 0x7fdbbc15dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc3800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbbc0aede0 0x7fdbbc0af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc3800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc0af7d0 con 0x7fdbbc0aede0 2024-05-22T21:13:37.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc3800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc160b50 con 0x7fdbbc15d4a0 2024-05-22T21:13:37.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc3800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc142c00 con 0x7fdbbc0b0490 2024-05-22T21:13:37.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc1800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbbc0aede0 0x7fdbbc0af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc0e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbbc0b0490 0x7fdbbc160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc1800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbbc0aede0 0x7fdbbc0af290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54644/0 (socket says 172.21.15.161:54644) 2024-05-22T21:13:37.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc1800640 1 -- 172.21.15.161:0/2820041263 learned_addr learned my addr 172.21.15.161:0/2820041263 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:37.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbb3e00640 1 --2- 172.21.15.161:0/2820041263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbbc15d4a0 0x7fdbbc15dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc1800640 1 -- 172.21.15.161:0/2820041263 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbbc0b0490 msgr2=0x7fdbbc160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc1800640 1 --2- 172.21.15.161:0/2820041263 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbbc0b0490 0x7fdbbc160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc1800640 1 -- 172.21.15.161:0/2820041263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbbc15d4a0 msgr2=0x7fdbbc15dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc1800640 1 --2- 172.21.15.161:0/2820041263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbbc15d4a0 0x7fdbbc15dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc1800640 1 -- 172.21.15.161:0/2820041263 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba4009420 con 0x7fdbbc0aede0 2024-05-22T21:13:37.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc1800640 1 --2- 172.21.15.161:0/2820041263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbbc0aede0 0x7fdbbc0af290 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400c700 tx=0x7fdbb400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:37.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbb9600640 1 -- 172.21.15.161:0/2820041263 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4003b80 con 0x7fdbbc0aede0 2024-05-22T21:13:37.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbb9600640 1 -- 172.21.15.161:0/2820041263 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb4003ce0 con 0x7fdbbc0aede0 2024-05-22T21:13:37.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbb9600640 1 -- 172.21.15.161:0/2820041263 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb400aa30 con 0x7fdbbc0aede0 2024-05-22T21:13:37.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc3800640 1 -- 172.21.15.161:0/2820041263 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbac0094d0 con 0x7fdbbc0aede0 2024-05-22T21:13:37.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc3800640 1 -- 172.21.15.161:0/2820041263 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbbc0cb7b0 con 0x7fdbbc0aede0 2024-05-22T21:13:37.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.030+0000 7fdbc3800640 1 -- 172.21.15.161:0/2820041263 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbbc067220 con 0x7fdbbc0aede0 2024-05-22T21:13:37.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.034+0000 7fdbb9600640 1 -- 172.21.15.161:0/2820041263 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdbb400ab90 con 0x7fdbbc0aede0 2024-05-22T21:13:37.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.034+0000 7fdbb9600640 1 --2- 172.21.15.161:0/2820041263 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdb980666c0 0x7fdb98068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.034+0000 7fdbb9600640 1 -- 172.21.15.161:0/2820041263 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbb4083cf0 con 0x7fdbbc0aede0 2024-05-22T21:13:37.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.034+0000 7fdbc0e00640 1 --2- 172.21.15.161:0/2820041263 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdb980666c0 0x7fdb98068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.034+0000 7fdbc0e00640 1 --2- 172.21.15.161:0/2820041263 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdb980666c0 0x7fdb98068b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0ae000 tx=0x7fdbac0b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:37.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.038+0000 7fdbb9600640 1 -- 172.21.15.161:0/2820041263 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdbb4051070 con 0x7fdbbc0aede0 2024-05-22T21:13:37.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.222+0000 7fdbc3800640 1 -- 172.21.15.161:0/2820041263 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fdbbc06c440 con 0x7fdbbc0aede0 2024-05-22T21:13:37.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.222+0000 7fdbb9600640 1 -- 172.21.15.161:0/2820041263 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fdbbc06c440 con 0x7fdbbc0aede0 2024-05-22T21:13:37.226 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:37.227 INFO:teuthology.orchestra.run.smithi161.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-22T21:13:37.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.226+0000 7fdbc3800640 1 -- 172.21.15.161:0/2820041263 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdb980666c0 msgr2=0x7fdb98068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.226+0000 7fdbc3800640 1 --2- 172.21.15.161:0/2820041263 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdb980666c0 0x7fdb98068b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0ae000 tx=0x7fdbac0b3040 comp rx=0 tx=0).stop 2024-05-22T21:13:37.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.226+0000 7fdbc3800640 1 -- 172.21.15.161:0/2820041263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbbc0aede0 msgr2=0x7fdbbc0af290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.226+0000 7fdbc3800640 1 --2- 172.21.15.161:0/2820041263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbbc0aede0 0x7fdbbc0af290 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400c700 tx=0x7fdbb400cbc0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.226+0000 7fdbc3800640 1 -- 172.21.15.161:0/2820041263 shutdown_connections 2024-05-22T21:13:37.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.226+0000 7fdbc3800640 1 --2- 172.21.15.161:0/2820041263 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdb980666c0 0x7fdb98068b80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.226+0000 7fdbc3800640 1 --2- 172.21.15.161:0/2820041263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbbc0aede0 0x7fdbbc0af290 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.226+0000 7fdbc3800640 1 --2- 172.21.15.161:0/2820041263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbbc15d4a0 0x7fdbbc15dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.226+0000 7fdbc3800640 1 --2- 172.21.15.161:0/2820041263 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbbc0b0490 0x7fdbbc160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.226+0000 7fdbc3800640 1 -- 172.21.15.161:0/2820041263 >> 172.21.15.161:0/2820041263 conn(0x7fdbbc0b2a40 msgr2=0x7fdbbc06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:37.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.226+0000 7fdbc3800640 1 -- 172.21.15.161:0/2820041263 shutdown_connections 2024-05-22T21:13:37.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.226+0000 7fdbc3800640 1 -- 172.21.15.161:0/2820041263 wait complete. 2024-05-22T21:13:37.248 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-22T21:13:37.248 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.378+0000 7f1f3d800640 1 Processor -- start 2024-05-22T21:13:37.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.378+0000 7f1f3d800640 1 -- start start 2024-05-22T21:13:37.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.378+0000 7f1f3d800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1f3815dd50 0x7f1f3806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.378+0000 7f1f3d800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1f3815e130 0x7f1f3815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.378+0000 7f1f3d800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f38071670 con 0x7f1f3815d410 2024-05-22T21:13:37.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.378+0000 7f1f3d800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f3815d8a0 con 0x7f1f3815e130 2024-05-22T21:13:37.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.378+0000 7f1f3d800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f3806de40 con 0x7f1f3815dd50 2024-05-22T21:13:37.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.382+0000 7f1f2e000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1f3815e130 0x7f1f3815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.382+0000 7f1f2e000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1f3815e130 0x7f1f3815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54652/0 (socket says 172.21.15.161:54652) 2024-05-22T21:13:37.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.382+0000 7f1f2e000640 1 -- 172.21.15.161:0/764228124 learned_addr learned my addr 172.21.15.161:0/764228124 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:37.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.382+0000 7f1f36a00640 1 --2- 172.21.15.161:0/764228124 >> v2:172.21.15.179:3300/0 conn(0x7f1f3815dd50 0x7f1f3806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.382+0000 7f1f2e000640 1 -- 172.21.15.161:0/764228124 >> v2:172.21.15.179:3300/0 conn(0x7f1f3815dd50 msgr2=0x7f1f3806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:37.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.382+0000 7f1f2e000640 1 --2- 172.21.15.161:0/764228124 >> v2:172.21.15.179:3300/0 conn(0x7f1f3815dd50 0x7f1f3806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.382+0000 7f1f2e000640 1 -- 172.21.15.161:0/764228124 >> v1:172.21.15.179:6789/0 conn(0x7f1f3815d410 legacy=0x7f1f3815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:37.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.382+0000 7f1f2e000640 1 -- 172.21.15.161:0/764228124 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f38070a60 con 0x7f1f3815e130 2024-05-22T21:13:37.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.382+0000 7f1f2e000640 1 --2- 172.21.15.161:0/764228124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3815e130 0x7f1f3815ced0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1f38072330 tx=0x7f1f2001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328b00073d328fd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:37.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.382+0000 7f1f36000640 1 -- 172.21.15.161:0/764228124 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f200aa610 con 0x7f1f3815e130 2024-05-22T21:13:37.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.382+0000 7f1f36000640 1 -- 172.21.15.161:0/764228124 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f200a6040 con 0x7f1f3815e130 2024-05-22T21:13:37.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.382+0000 7f1f36000640 1 -- 172.21.15.161:0/764228124 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f200aa990 con 0x7f1f3815e130 2024-05-22T21:13:37.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f3d800640 1 -- 172.21.15.161:0/764228124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3815e130 msgr2=0x7f1f3815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f3d800640 1 --2- 172.21.15.161:0/764228124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3815e130 0x7f1f3815ced0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1f38072330 tx=0x7f1f2001c4b0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f3d800640 1 -- 172.21.15.161:0/764228124 shutdown_connections 2024-05-22T21:13:37.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f3d800640 1 --2- 172.21.15.161:0/764228124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3815e130 0x7f1f3815ced0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f3d800640 1 --2- 172.21.15.161:0/764228124 >> v2:172.21.15.179:3300/0 conn(0x7f1f3815dd50 0x7f1f3806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f3d800640 1 -- 172.21.15.161:0/764228124 >> 172.21.15.161:0/764228124 conn(0x7f1f380b0bc0 msgr2=0x7f1f380b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:37.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f3d800640 1 -- 172.21.15.161:0/764228124 shutdown_connections 2024-05-22T21:13:37.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f3d800640 1 -- 172.21.15.161:0/764228124 wait complete. 2024-05-22T21:13:37.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f3d800640 1 Processor -- start 2024-05-22T21:13:37.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f3d800640 1 -- start start 2024-05-22T21:13:37.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f3d800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f3815d410 0x7f1f380b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f3d800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f3815dd50 0x7f1f3814d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f3d800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3815e130 0x7f1f38157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f3d800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f38152e50 con 0x7f1f3815e130 2024-05-22T21:13:37.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f3d800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380b3530 con 0x7f1f3815dd50 2024-05-22T21:13:37.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f3d800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380b3810 con 0x7f1f3815d410 2024-05-22T21:13:37.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f37400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3815e130 0x7f1f38157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f37400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3815e130 0x7f1f38157210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54656/0 (socket says 172.21.15.161:54656) 2024-05-22T21:13:37.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f37400640 1 -- 172.21.15.161:0/2763921611 learned_addr learned my addr 172.21.15.161:0/2763921611 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:37.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f36a00640 1 --2- 172.21.15.161:0/2763921611 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f3815d410 0x7f1f380b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f2e000640 1 --2- 172.21.15.161:0/2763921611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f3815dd50 0x7f1f3814d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f37400640 1 -- 172.21.15.161:0/2763921611 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f3815d410 msgr2=0x7f1f380b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f37400640 1 --2- 172.21.15.161:0/2763921611 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f3815d410 0x7f1f380b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f37400640 1 -- 172.21.15.161:0/2763921611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f3815dd50 msgr2=0x7f1f3814d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f37400640 1 --2- 172.21.15.161:0/2763921611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f3815dd50 0x7f1f3814d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.386+0000 7f1f37400640 1 -- 172.21.15.161:0/2763921611 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f200094d0 con 0x7f1f3815e130 2024-05-22T21:13:37.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.390+0000 7f1f37400640 1 --2- 172.21.15.161:0/2763921611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3815e130 0x7f1f38157210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000c780 tx=0x7f1f3000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:37.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.390+0000 7f1f2f400640 1 -- 172.21.15.161:0/2763921611 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30004b70 con 0x7f1f3815e130 2024-05-22T21:13:37.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.390+0000 7f1f2f400640 1 -- 172.21.15.161:0/2763921611 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3000ce90 con 0x7f1f3815e130 2024-05-22T21:13:37.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.390+0000 7f1f3d800640 1 -- 172.21.15.161:0/2763921611 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f380b3af0 con 0x7f1f3815e130 2024-05-22T21:13:37.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.390+0000 7f1f2f400640 1 -- 172.21.15.161:0/2763921611 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3000a850 con 0x7f1f3815e130 2024-05-22T21:13:37.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.390+0000 7f1f3d800640 1 -- 172.21.15.161:0/2763921611 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3814d950 con 0x7f1f3815e130 2024-05-22T21:13:37.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.390+0000 7f1f3d800640 1 -- 172.21.15.161:0/2763921611 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f38153070 con 0x7f1f3815e130 2024-05-22T21:13:37.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.390+0000 7f1f2f400640 1 -- 172.21.15.161:0/2763921611 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1f3000a9b0 con 0x7f1f3815e130 2024-05-22T21:13:37.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.390+0000 7f1f2f400640 1 --2- 172.21.15.161:0/2763921611 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f14066690 0x7f1f14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.390+0000 7f1f2f400640 1 -- 172.21.15.161:0/2763921611 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f300841d0 con 0x7f1f3815e130 2024-05-22T21:13:37.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.390+0000 7f1f36a00640 1 --2- 172.21.15.161:0/2763921611 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f14066690 0x7f1f14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.394+0000 7f1f36a00640 1 --2- 172.21.15.161:0/2763921611 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f14066690 0x7f1f14068b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f28007a60 tx=0x7f1f28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:37.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.394+0000 7f1f2f400640 1 -- 172.21.15.161:0/2763921611 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1f38153070 con 0x7f1f3815e130 2024-05-22T21:13:37.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.550+0000 7f1f3d800640 1 -- 172.21.15.161:0/2763921611 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f38153070 con 0x7f1f3815e130 2024-05-22T21:13:37.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.550+0000 7f1f2f400640 1 -- 172.21.15.161:0/2763921611 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f1f38153070 con 0x7f1f3815e130 2024-05-22T21:13:37.556 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:37.556 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:37.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.554+0000 7f1f3d800640 1 -- 172.21.15.161:0/2763921611 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f14066690 msgr2=0x7f1f14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.554+0000 7f1f3d800640 1 --2- 172.21.15.161:0/2763921611 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f14066690 0x7f1f14068b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f28007a60 tx=0x7f1f28008040 comp rx=0 tx=0).stop 2024-05-22T21:13:37.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.554+0000 7f1f3d800640 1 -- 172.21.15.161:0/2763921611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3815e130 msgr2=0x7f1f38157210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.554+0000 7f1f3d800640 1 --2- 172.21.15.161:0/2763921611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3815e130 0x7f1f38157210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000c780 tx=0x7f1f3000cc40 comp rx=0 tx=0).stop 2024-05-22T21:13:37.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.554+0000 7f1f3d800640 1 -- 172.21.15.161:0/2763921611 shutdown_connections 2024-05-22T21:13:37.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.554+0000 7f1f3d800640 1 --2- 172.21.15.161:0/2763921611 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f14066690 0x7f1f14068b50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.554+0000 7f1f3d800640 1 --2- 172.21.15.161:0/2763921611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f3815e130 0x7f1f38157210 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.554+0000 7f1f3d800640 1 --2- 172.21.15.161:0/2763921611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f3815dd50 0x7f1f3814d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.554+0000 7f1f3d800640 1 --2- 172.21.15.161:0/2763921611 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f3815d410 0x7f1f380b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.554+0000 7f1f3d800640 1 -- 172.21.15.161:0/2763921611 >> 172.21.15.161:0/2763921611 conn(0x7f1f380b0bc0 msgr2=0x7f1f38155910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:37.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.554+0000 7f1f3d800640 1 -- 172.21.15.161:0/2763921611 shutdown_connections 2024-05-22T21:13:37.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.554+0000 7f1f3d800640 1 -- 172.21.15.161:0/2763921611 wait complete. 2024-05-22T21:13:37.577 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:13:21.237503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:13:22.425891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:13:37.577 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:13:37.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.710+0000 7f51ac400640 1 Processor -- start 2024-05-22T21:13:37.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.710+0000 7f51ac400640 1 -- start start 2024-05-22T21:13:37.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51ac400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f51a415d180 0x7f51a40b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51ac400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51a406d080 0x7f51a406aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51ac400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a4161c20 con 0x7f51a406b420 2024-05-22T21:13:37.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51ac400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a406c390 con 0x7f51a415d180 2024-05-22T21:13:37.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51ac400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a406b890 con 0x7f51a406d080 2024-05-22T21:13:37.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51aa400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f51a406b420 0x7f51a40ad570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43500/0 (socket says 172.21.15.161:43500) 2024-05-22T21:13:37.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51aa400640 1 -- 172.21.15.161:0/955446569 learned_addr learned my addr 172.21.15.161:0/955446569 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:37.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a9a00640 1 --2- 172.21.15.161:0/955446569 >> v2:172.21.15.179:3300/0 conn(0x7f51a415d180 0x7f51a40b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a9000640 1 --2- 172.21.15.161:0/955446569 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51a406d080 0x7f51a406aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a3e00640 1 -- 172.21.15.161:0/955446569 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 399836000 0 0) 0x7f51a4161c20 con 0x7f51a406b420 2024-05-22T21:13:37.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a3e00640 1 -- 172.21.15.161:0/955446569 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f518c003530 con 0x7f51a406b420 2024-05-22T21:13:37.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a3e00640 1 -- 172.21.15.161:0/955446569 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4123258402 0 0) 0x7f518c003530 con 0x7f51a406b420 2024-05-22T21:13:37.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a3e00640 1 -- 172.21.15.161:0/955446569 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51a406d080 msgr2=0x7f51a406aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:37.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a3e00640 1 --2- 172.21.15.161:0/955446569 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51a406d080 0x7f51a406aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a3e00640 1 -- 172.21.15.161:0/955446569 >> v2:172.21.15.179:3300/0 conn(0x7f51a415d180 msgr2=0x7f51a40b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:37.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a3e00640 1 --2- 172.21.15.161:0/955446569 >> v2:172.21.15.179:3300/0 conn(0x7f51a415d180 0x7f51a40b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a3e00640 1 -- 172.21.15.161:0/955446569 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a406c5a0 con 0x7f51a406b420 2024-05-22T21:13:37.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a3e00640 1 -- 172.21.15.161:0/955446569 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f519c001a00 con 0x7f51a406b420 2024-05-22T21:13:37.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a3e00640 1 -- 172.21.15.161:0/955446569 >> v1:172.21.15.161:6789/0 conn(0x7f51a406b420 legacy=0x7f51a40ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a3e00640 1 --2- 172.21.15.161:0/955446569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f518c003980 0x7f518c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a3e00640 1 --2- 172.21.15.161:0/955446569 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51a406d080 0x7f518c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a3e00640 1 --2- 172.21.15.161:0/955446569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f518c004d10 0x7f518c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a3e00640 1 -- 172.21.15.161:0/955446569 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a406c390 con 0x7f518c003980 2024-05-22T21:13:37.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a3e00640 1 -- 172.21.15.161:0/955446569 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a406b890 con 0x7f518c004d10 2024-05-22T21:13:37.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a3e00640 1 -- 172.21.15.161:0/955446569 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518c00b9c0 con 0x7f51a406d080 2024-05-22T21:13:37.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a9a00640 1 --2- 172.21.15.161:0/955446569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f518c003980 0x7f518c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a9000640 1 --2- 172.21.15.161:0/955446569 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51a406d080 0x7f518c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51aa400640 1 --2- 172.21.15.161:0/955446569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f518c004d10 0x7f518c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a9a00640 1 -- 172.21.15.161:0/955446569 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51a406d080 msgr2=0x7f518c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:37.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a9a00640 1 --2- 172.21.15.161:0/955446569 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51a406d080 0x7f518c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a9a00640 1 -- 172.21.15.161:0/955446569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f518c004d10 msgr2=0x7f518c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:37.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a9a00640 1 --2- 172.21.15.161:0/955446569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f518c004d10 0x7f518c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51a9a00640 1 -- 172.21.15.161:0/955446569 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c003530 con 0x7f518c003980 2024-05-22T21:13:37.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.714+0000 7f51aa400640 1 --2- 172.21.15.161:0/955446569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f518c004d10 0x7f518c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:37.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.718+0000 7f51a9a00640 1 --2- 172.21.15.161:0/955446569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f518c003980 0x7f518c003da0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5198009ae0 tx=0x7f519801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0850ec85d0b8e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:37.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.718+0000 7f51a3e00640 1 -- 172.21.15.161:0/955446569 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51980a6490 con 0x7f518c003980 2024-05-22T21:13:37.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.718+0000 7f51a3e00640 1 -- 172.21.15.161:0/955446569 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51980a65f0 con 0x7f518c003980 2024-05-22T21:13:37.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.718+0000 7f51a3e00640 1 -- 172.21.15.161:0/955446569 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51980a69a0 con 0x7f518c003980 2024-05-22T21:13:37.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.718+0000 7f51ac400640 1 -- 172.21.15.161:0/955446569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f518c003980 msgr2=0x7f518c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.718+0000 7f51ac400640 1 --2- 172.21.15.161:0/955446569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f518c003980 0x7f518c003da0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5198009ae0 tx=0x7f519801cab0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.718+0000 7f51aa400640 1 -- 172.21.15.161:0/955446569 reap_dead start 2024-05-22T21:13:37.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.718+0000 7f51ac400640 1 -- 172.21.15.161:0/955446569 shutdown_connections 2024-05-22T21:13:37.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.718+0000 7f51ac400640 1 -- 172.21.15.161:0/955446569 >> 172.21.15.161:0/955446569 conn(0x7f51a4070610 msgr2=0x7f51a4070a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:37.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.718+0000 7f51ac400640 1 -- 172.21.15.161:0/955446569 shutdown_connections 2024-05-22T21:13:37.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.718+0000 7f51ac400640 1 -- 172.21.15.161:0/955446569 wait complete. 2024-05-22T21:13:37.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51ac400640 1 Processor -- start 2024-05-22T21:13:37.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51ac400640 1 -- start start 2024-05-22T21:13:37.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51ac400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f518c003980 0x7f51a414d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51ac400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51a414e0e0 0x7f51a40cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51ac400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51a40cc4b0 0x7f51a4156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51ac400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a4156140 con 0x7f518c003980 2024-05-22T21:13:37.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51ac400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a414dea0 con 0x7f51a414e0e0 2024-05-22T21:13:37.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51ac400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a4155550 con 0x7f51a40cc4b0 2024-05-22T21:13:37.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51a9a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f518c003980 0x7f51a414d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51a9a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f518c003980 0x7f51a414d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54678/0 (socket says 172.21.15.161:54678) 2024-05-22T21:13:37.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51a9a00640 1 -- 172.21.15.161:0/1785326502 learned_addr learned my addr 172.21.15.161:0/1785326502 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:37.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51aa400640 1 --2- 172.21.15.161:0/1785326502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51a40cc4b0 0x7f51a4156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51a9000640 1 --2- 172.21.15.161:0/1785326502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51a414e0e0 0x7f51a40cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51a9a00640 1 -- 172.21.15.161:0/1785326502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51a40cc4b0 msgr2=0x7f51a4156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51a9a00640 1 --2- 172.21.15.161:0/1785326502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51a40cc4b0 0x7f51a4156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51a9a00640 1 -- 172.21.15.161:0/1785326502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51a414e0e0 msgr2=0x7f51a40cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51a9a00640 1 --2- 172.21.15.161:0/1785326502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51a414e0e0 0x7f51a40cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51a9a00640 1 -- 172.21.15.161:0/1785326502 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a41557a0 con 0x7f518c003980 2024-05-22T21:13:37.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51a9000640 1 --2- 172.21.15.161:0/1785326502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51a414e0e0 0x7f51a40cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:37.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51a9a00640 1 --2- 172.21.15.161:0/1785326502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f518c003980 0x7f51a414d960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f519801c8e0 tx=0x7f518c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:37.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51a1600640 1 -- 172.21.15.161:0/1785326502 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51980a6de0 con 0x7f518c003980 2024-05-22T21:13:37.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51a1600640 1 -- 172.21.15.161:0/1785326502 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519800f000 con 0x7f518c003980 2024-05-22T21:13:37.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51a1600640 1 -- 172.21.15.161:0/1785326502 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51980a7690 con 0x7f518c003980 2024-05-22T21:13:37.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51ac400640 1 -- 172.21.15.161:0/1785326502 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a4155a80 con 0x7f518c003980 2024-05-22T21:13:37.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51ac400640 1 -- 172.21.15.161:0/1785326502 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a4156aa0 con 0x7f518c003980 2024-05-22T21:13:37.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.722+0000 7f51ac400640 1 -- 172.21.15.161:0/1785326502 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a4067220 con 0x7f518c003980 2024-05-22T21:13:37.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.726+0000 7f51a1600640 1 -- 172.21.15.161:0/1785326502 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f51980a77f0 con 0x7f518c003980 2024-05-22T21:13:37.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.726+0000 7f51a1600640 1 --2- 172.21.15.161:0/1785326502 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5184066690 0x7f5184068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:37.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.726+0000 7f51a1600640 1 -- 172.21.15.161:0/1785326502 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f519811db90 con 0x7f518c003980 2024-05-22T21:13:37.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.726+0000 7f51a9000640 1 --2- 172.21.15.161:0/1785326502 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5184066690 0x7f5184068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:37.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.726+0000 7f51a9000640 1 --2- 172.21.15.161:0/1785326502 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5184066690 0x7f5184068b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f51a41543f0 tx=0x7f5190008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:37.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.730+0000 7f51a1600640 1 -- 172.21.15.161:0/1785326502 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f51980eafa0 con 0x7f518c003980 2024-05-22T21:13:37.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.886+0000 7f51ac400640 1 -- 172.21.15.161:0/1785326502 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51a406dd70 con 0x7f518c003980 2024-05-22T21:13:37.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.886+0000 7f51a1600640 1 -- 172.21.15.161:0/1785326502 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51980be070 con 0x7f518c003980 2024-05-22T21:13:37.890 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:13:37.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.886+0000 7f51ac400640 1 -- 172.21.15.161:0/1785326502 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5184066690 msgr2=0x7f5184068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.886+0000 7f51ac400640 1 --2- 172.21.15.161:0/1785326502 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5184066690 0x7f5184068b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f51a41543f0 tx=0x7f5190008040 comp rx=0 tx=0).stop 2024-05-22T21:13:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.886+0000 7f51ac400640 1 -- 172.21.15.161:0/1785326502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f518c003980 msgr2=0x7f51a414d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.886+0000 7f51ac400640 1 --2- 172.21.15.161:0/1785326502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f518c003980 0x7f51a414d960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f519801c8e0 tx=0x7f518c005800 comp rx=0 tx=0).stop 2024-05-22T21:13:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.886+0000 7f51ac400640 1 -- 172.21.15.161:0/1785326502 shutdown_connections 2024-05-22T21:13:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.886+0000 7f51ac400640 1 --2- 172.21.15.161:0/1785326502 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5184066690 0x7f5184068b50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.886+0000 7f51ac400640 1 --2- 172.21.15.161:0/1785326502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51a40cc4b0 0x7f51a4156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.886+0000 7f51ac400640 1 --2- 172.21.15.161:0/1785326502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51a414e0e0 0x7f51a40cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.886+0000 7f51ac400640 1 --2- 172.21.15.161:0/1785326502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f518c003980 0x7f51a414d960 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.886+0000 7f51ac400640 1 -- 172.21.15.161:0/1785326502 >> 172.21.15.161:0/1785326502 conn(0x7f51a4070610 msgr2=0x7f51a4160e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.886+0000 7f51ac400640 1 -- 172.21.15.161:0/1785326502 shutdown_connections 2024-05-22T21:13:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:37.886+0000 7f51ac400640 1 -- 172.21.15.161:0/1785326502 wait complete. 2024-05-22T21:13:37.910 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:13:38.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.042+0000 7fe7f7800640 1 Processor -- start 2024-05-22T21:13:38.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.046+0000 7fe7f7800640 1 -- start start 2024-05-22T21:13:38.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.046+0000 7fe7f7800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe7f0143c40 0x7fe7f015b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.046+0000 7fe7f7800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe7f006c9b0 0x7fe7f00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.046+0000 7fe7f7800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f015efb0 con 0x7fe7f01444a0 2024-05-22T21:13:38.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.046+0000 7fe7f7800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f015f290 con 0x7fe7f006c9b0 2024-05-22T21:13:38.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.046+0000 7fe7f7800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f015f570 con 0x7fe7f0143c40 2024-05-22T21:13:38.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.046+0000 7fe7f5800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe7f006c9b0 0x7fe7f00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.046+0000 7fe7f4e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe7f0143c40 0x7fe7f015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.046+0000 7fe7f5800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe7f006c9b0 0x7fe7f00b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:49558/0 (socket says 172.21.15.161:49558) 2024-05-22T21:13:38.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.046+0000 7fe7f5800640 1 -- 172.21.15.161:0/3218882034 learned_addr learned my addr 172.21.15.161:0/3218882034 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:38.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.046+0000 7fe7f5800640 1 -- 172.21.15.161:0/3218882034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe7f0143c40 msgr2=0x7fe7f015b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:38.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.046+0000 7fe7f5800640 1 --2- 172.21.15.161:0/3218882034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe7f0143c40 0x7fe7f015b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.046+0000 7fe7f5800640 1 -- 172.21.15.161:0/3218882034 >> v1:172.21.15.179:6789/0 conn(0x7fe7f01444a0 legacy=0x7fe7f006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.046+0000 7fe7f5800640 1 -- 172.21.15.161:0/3218882034 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f015f850 con 0x7fe7f006c9b0 2024-05-22T21:13:38.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.046+0000 7fe7f4e00640 1 --2- 172.21.15.161:0/3218882034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe7f0143c40 0x7fe7f015b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:38.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.050+0000 7fe7f5800640 1 --2- 172.21.15.161:0/3218882034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7f006c9b0 0x7fe7f00b2160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009870 tx=0x7fe7e801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72e82ac90502f736 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:38.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.050+0000 7fe7ef400640 1 -- 172.21.15.161:0/3218882034 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80aa620 con 0x7fe7f006c9b0 2024-05-22T21:13:38.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.050+0000 7fe7ef400640 1 -- 172.21.15.161:0/3218882034 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e80aa780 con 0x7fe7f006c9b0 2024-05-22T21:13:38.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.050+0000 7fe7ef400640 1 -- 172.21.15.161:0/3218882034 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80aab10 con 0x7fe7f006c9b0 2024-05-22T21:13:38.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.050+0000 7fe7f7800640 1 -- 172.21.15.161:0/3218882034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7f006c9b0 msgr2=0x7fe7f00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.050+0000 7fe7f7800640 1 --2- 172.21.15.161:0/3218882034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7f006c9b0 0x7fe7f00b2160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009870 tx=0x7fe7e801c560 comp rx=0 tx=0).stop 2024-05-22T21:13:38.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.050+0000 7fe7f7800640 1 -- 172.21.15.161:0/3218882034 shutdown_connections 2024-05-22T21:13:38.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.050+0000 7fe7f7800640 1 --2- 172.21.15.161:0/3218882034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7f006c9b0 0x7fe7f00b2160 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.050+0000 7fe7f7800640 1 --2- 172.21.15.161:0/3218882034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe7f0143c40 0x7fe7f015b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.050+0000 7fe7f7800640 1 -- 172.21.15.161:0/3218882034 >> 172.21.15.161:0/3218882034 conn(0x7fe7f006a2d0 msgr2=0x7fe7f006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:38.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.050+0000 7fe7f7800640 1 -- 172.21.15.161:0/3218882034 shutdown_connections 2024-05-22T21:13:38.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.050+0000 7fe7f7800640 1 -- 172.21.15.161:0/3218882034 wait complete. 2024-05-22T21:13:38.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.050+0000 7fe7f7800640 1 Processor -- start 2024-05-22T21:13:38.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.050+0000 7fe7f7800640 1 -- start start 2024-05-22T21:13:38.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f7800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7f006c9b0 0x7fe7f013f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f7800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe7f0143c40 0x7fe7f0159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f7800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7f01444a0 0x7fe7f013dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f7800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f013a740 con 0x7fe7f006c9b0 2024-05-22T21:13:38.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f7800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f013afe0 con 0x7fe7f01444a0 2024-05-22T21:13:38.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f7800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f013fcb0 con 0x7fe7f0143c40 2024-05-22T21:13:38.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f4e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7f006c9b0 0x7fe7f013f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f4e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7f006c9b0 0x7fe7f013f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54692/0 (socket says 172.21.15.161:54692) 2024-05-22T21:13:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f4e00640 1 -- 172.21.15.161:0/732528220 learned_addr learned my addr 172.21.15.161:0/732528220 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f4e00640 1 -- 172.21.15.161:0/732528220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe7f0143c40 msgr2=0x7fe7f0159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7efe00640 1 --2- 172.21.15.161:0/732528220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe7f0143c40 0x7fe7f0159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f5800640 1 --2- 172.21.15.161:0/732528220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7f01444a0 0x7fe7f013dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f4e00640 1 --2- 172.21.15.161:0/732528220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe7f0143c40 0x7fe7f0159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f4e00640 1 -- 172.21.15.161:0/732528220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7f01444a0 msgr2=0x7fe7f013dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f4e00640 1 --2- 172.21.15.161:0/732528220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7f01444a0 0x7fe7f013dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f4e00640 1 -- 172.21.15.161:0/732528220 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e0009a70 con 0x7fe7f006c9b0 2024-05-22T21:13:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7efe00640 1 --2- 172.21.15.161:0/732528220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe7f0143c40 0x7fe7f0159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f5800640 1 --2- 172.21.15.161:0/732528220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7f01444a0 0x7fe7f013dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f4e00640 1 --2- 172.21.15.161:0/732528220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7f006c9b0 0x7fe7f013f770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000efc0 tx=0x7fe7e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7ecc00640 1 -- 172.21.15.161:0/732528220 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0014680 con 0x7fe7f006c9b0 2024-05-22T21:13:38.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7ecc00640 1 -- 172.21.15.161:0/732528220 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e000f040 con 0x7fe7f006c9b0 2024-05-22T21:13:38.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7ecc00640 1 -- 172.21.15.161:0/732528220 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0014a00 con 0x7fe7f006c9b0 2024-05-22T21:13:38.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f7800640 1 -- 172.21.15.161:0/732528220 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e80094d0 con 0x7fe7f006c9b0 2024-05-22T21:13:38.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f7800640 1 -- 172.21.15.161:0/732528220 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f013ac00 con 0x7fe7f006c9b0 2024-05-22T21:13:38.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.054+0000 7fe7f7800640 1 -- 172.21.15.161:0/732528220 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f013b250 con 0x7fe7f006c9b0 2024-05-22T21:13:38.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.058+0000 7fe7ecc00640 1 -- 172.21.15.161:0/732528220 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe7e0014b60 con 0x7fe7f006c9b0 2024-05-22T21:13:38.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.058+0000 7fe7ecc00640 1 --2- 172.21.15.161:0/732528220 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe7d0066690 0x7fe7d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.058+0000 7fe7ecc00640 1 -- 172.21.15.161:0/732528220 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7e0085200 con 0x7fe7f006c9b0 2024-05-22T21:13:38.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.058+0000 7fe7efe00640 1 --2- 172.21.15.161:0/732528220 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe7d0066690 0x7fe7d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.058+0000 7fe7efe00640 1 --2- 172.21.15.161:0/732528220 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe7d0066690 0x7fe7d0068b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe7f006c980 tx=0x7fe7dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:38.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.062+0000 7fe7ecc00640 1 -- 172.21.15.161:0/732528220 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe7e00524f0 con 0x7fe7f006c9b0 2024-05-22T21:13:38.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.214+0000 7fe7f7800640 1 -- 172.21.15.161:0/732528220 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe7f0067220 con 0x7fe7f006c9b0 2024-05-22T21:13:38.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.214+0000 7fe7ecc00640 1 -- 172.21.15.161:0/732528220 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe7e0055b10 con 0x7fe7f006c9b0 2024-05-22T21:13:38.219 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:13:38.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.218+0000 7fe7f7800640 1 -- 172.21.15.161:0/732528220 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe7d0066690 msgr2=0x7fe7d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.218+0000 7fe7f7800640 1 --2- 172.21.15.161:0/732528220 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe7d0066690 0x7fe7d0068b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe7f006c980 tx=0x7fe7dc009040 comp rx=0 tx=0).stop 2024-05-22T21:13:38.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.218+0000 7fe7f7800640 1 -- 172.21.15.161:0/732528220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7f006c9b0 msgr2=0x7fe7f013f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.218+0000 7fe7f7800640 1 --2- 172.21.15.161:0/732528220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7f006c9b0 0x7fe7f013f770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000efc0 tx=0x7fe7e000c490 comp rx=0 tx=0).stop 2024-05-22T21:13:38.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.218+0000 7fe7f7800640 1 -- 172.21.15.161:0/732528220 shutdown_connections 2024-05-22T21:13:38.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.218+0000 7fe7f7800640 1 --2- 172.21.15.161:0/732528220 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe7d0066690 0x7fe7d0068b50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.218+0000 7fe7f7800640 1 --2- 172.21.15.161:0/732528220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7f01444a0 0x7fe7f013dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.218+0000 7fe7f7800640 1 --2- 172.21.15.161:0/732528220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe7f0143c40 0x7fe7f0159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.218+0000 7fe7f7800640 1 --2- 172.21.15.161:0/732528220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7f006c9b0 0x7fe7f013f770 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.218+0000 7fe7f7800640 1 -- 172.21.15.161:0/732528220 >> 172.21.15.161:0/732528220 conn(0x7fe7f006a2d0 msgr2=0x7fe7f00aefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:38.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.218+0000 7fe7f7800640 1 -- 172.21.15.161:0/732528220 shutdown_connections 2024-05-22T21:13:38.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.218+0000 7fe7f7800640 1 -- 172.21.15.161:0/732528220 wait complete. 2024-05-22T21:13:38.239 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:13:38.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.370+0000 7fa264000640 1 Processor -- start 2024-05-22T21:13:38.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.370+0000 7fa264000640 1 -- start start 2024-05-22T21:13:38.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.374+0000 7fa264000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa25c143c40 0x7fa25c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.374+0000 7fa264000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa25c06c9b0 0x7fa25c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.374+0000 7fa264000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25c15efb0 con 0x7fa25c1444a0 2024-05-22T21:13:38.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.374+0000 7fa264000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa25c15f290 con 0x7fa25c143c40 2024-05-22T21:13:38.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.374+0000 7fa264000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa25c15f570 con 0x7fa25c06c9b0 2024-05-22T21:13:38.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.374+0000 7fa261600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa25c143c40 0x7fa25c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.374+0000 7fa261600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa25c143c40 0x7fa25c15b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54706/0 (socket says 172.21.15.161:54706) 2024-05-22T21:13:38.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.374+0000 7fa261600640 1 -- 172.21.15.161:0/2905617885 learned_addr learned my addr 172.21.15.161:0/2905617885 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:38.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.374+0000 7fa262000640 1 --2- 172.21.15.161:0/2905617885 >> v2:172.21.15.179:3300/0 conn(0x7fa25c06c9b0 0x7fa25c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.374+0000 7fa261600640 1 -- 172.21.15.161:0/2905617885 >> v2:172.21.15.179:3300/0 conn(0x7fa25c06c9b0 msgr2=0x7fa25c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:38.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.374+0000 7fa261600640 1 --2- 172.21.15.161:0/2905617885 >> v2:172.21.15.179:3300/0 conn(0x7fa25c06c9b0 0x7fa25c0b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.374+0000 7fa261600640 1 -- 172.21.15.161:0/2905617885 >> v1:172.21.15.179:6789/0 conn(0x7fa25c1444a0 legacy=0x7fa25c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:38.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.374+0000 7fa261600640 1 -- 172.21.15.161:0/2905617885 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c15f850 con 0x7fa25c143c40 2024-05-22T21:13:38.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.378+0000 7fa261600640 1 --2- 172.21.15.161:0/2905617885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa25c143c40 0x7fa25c15b600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0aead0 tx=0x7fa25001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e5f87d6aa4d8a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:38.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.378+0000 7fa25be00640 1 -- 172.21.15.161:0/2905617885 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2500ab640 con 0x7fa25c143c40 2024-05-22T21:13:38.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.378+0000 7fa25be00640 1 -- 172.21.15.161:0/2905617885 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2500a7070 con 0x7fa25c143c40 2024-05-22T21:13:38.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.378+0000 7fa25be00640 1 -- 172.21.15.161:0/2905617885 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2500ab9c0 con 0x7fa25c143c40 2024-05-22T21:13:38.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.378+0000 7fa264000640 1 -- 172.21.15.161:0/2905617885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa25c143c40 msgr2=0x7fa25c15b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.378+0000 7fa264000640 1 --2- 172.21.15.161:0/2905617885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa25c143c40 0x7fa25c15b600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0aead0 tx=0x7fa25001c4e0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.378+0000 7fa264000640 1 -- 172.21.15.161:0/2905617885 shutdown_connections 2024-05-22T21:13:38.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.378+0000 7fa264000640 1 --2- 172.21.15.161:0/2905617885 >> v2:172.21.15.179:3300/0 conn(0x7fa25c06c9b0 0x7fa25c0b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.378+0000 7fa264000640 1 --2- 172.21.15.161:0/2905617885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa25c143c40 0x7fa25c15b600 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.378+0000 7fa264000640 1 -- 172.21.15.161:0/2905617885 >> 172.21.15.161:0/2905617885 conn(0x7fa25c06a2d0 msgr2=0x7fa25c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:38.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.378+0000 7fa264000640 1 -- 172.21.15.161:0/2905617885 shutdown_connections 2024-05-22T21:13:38.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.378+0000 7fa264000640 1 -- 172.21.15.161:0/2905617885 wait complete. 2024-05-22T21:13:38.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.378+0000 7fa264000640 1 Processor -- start 2024-05-22T21:13:38.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.378+0000 7fa264000640 1 -- start start 2024-05-22T21:13:38.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa264000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa25c06c9b0 0x7fa25c1f9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa264000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa25c143c40 0x7fa25c0706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa264000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa25c1444a0 0x7fa25c0ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa264000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c158b10 con 0x7fa25c1444a0 2024-05-22T21:13:38.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa264000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c071060 con 0x7fa25c06c9b0 2024-05-22T21:13:38.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa264000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c071340 con 0x7fa25c143c40 2024-05-22T21:13:38.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa262000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa25c1444a0 0x7fa25c0ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa262000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa25c1444a0 0x7fa25c0ad900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54720/0 (socket says 172.21.15.161:54720) 2024-05-22T21:13:38.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa262000640 1 -- 172.21.15.161:0/1707948987 learned_addr learned my addr 172.21.15.161:0/1707948987 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:38.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa261600640 1 --2- 172.21.15.161:0/1707948987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa25c06c9b0 0x7fa25c1f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa260c00640 1 --2- 172.21.15.161:0/1707948987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa25c143c40 0x7fa25c0706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa262000640 1 -- 172.21.15.161:0/1707948987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa25c143c40 msgr2=0x7fa25c0706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa262000640 1 --2- 172.21.15.161:0/1707948987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa25c143c40 0x7fa25c0706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa262000640 1 -- 172.21.15.161:0/1707948987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa25c06c9b0 msgr2=0x7fa25c1f9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa262000640 1 --2- 172.21.15.161:0/1707948987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa25c06c9b0 0x7fa25c1f9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa262000640 1 -- 172.21.15.161:0/1707948987 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2500094d0 con 0x7fa25c1444a0 2024-05-22T21:13:38.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa261600640 1 --2- 172.21.15.161:0/1707948987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa25c06c9b0 0x7fa25c1f9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:38.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa262000640 1 --2- 172.21.15.161:0/1707948987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa25c1444a0 0x7fa25c0ad900 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa254007e40 tx=0x7fa25400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:38.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa259600640 1 -- 172.21.15.161:0/1707948987 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254014620 con 0x7fa25c1444a0 2024-05-22T21:13:38.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa259600640 1 -- 172.21.15.161:0/1707948987 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25400ee90 con 0x7fa25c1444a0 2024-05-22T21:13:38.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa264000640 1 -- 172.21.15.161:0/1707948987 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25c071620 con 0x7fa25c1444a0 2024-05-22T21:13:38.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa259600640 1 -- 172.21.15.161:0/1707948987 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2540149e0 con 0x7fa25c1444a0 2024-05-22T21:13:38.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa264000640 1 -- 172.21.15.161:0/1707948987 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25c1f9e90 con 0x7fa25c1444a0 2024-05-22T21:13:38.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa264000640 1 -- 172.21.15.161:0/1707948987 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25c158c80 con 0x7fa25c1444a0 2024-05-22T21:13:38.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.382+0000 7fa259600640 1 -- 172.21.15.161:0/1707948987 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa254014b40 con 0x7fa25c1444a0 2024-05-22T21:13:38.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.386+0000 7fa259600640 1 --2- 172.21.15.161:0/1707948987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa23c066690 0x7fa23c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.386+0000 7fa259600640 1 -- 172.21.15.161:0/1707948987 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa254084c20 con 0x7fa25c1444a0 2024-05-22T21:13:38.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.386+0000 7fa261600640 1 --2- 172.21.15.161:0/1707948987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa23c066690 0x7fa23c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.386+0000 7fa261600640 1 --2- 172.21.15.161:0/1707948987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa23c066690 0x7fa23c068b50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa2500af000 tx=0x7fa2500a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:38.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.390+0000 7fa259600640 1 -- 172.21.15.161:0/1707948987 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa25c158c80 con 0x7fa25c1444a0 2024-05-22T21:13:38.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.546+0000 7fa264000640 1 -- 172.21.15.161:0/1707948987 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa25c158c50 con 0x7fa25c1444a0 2024-05-22T21:13:38.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.546+0000 7fa259600640 1 -- 172.21.15.161:0/1707948987 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa254051ef0 con 0x7fa25c1444a0 2024-05-22T21:13:38.552 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:13:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.550+0000 7fa264000640 1 -- 172.21.15.161:0/1707948987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa23c066690 msgr2=0x7fa23c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.550+0000 7fa264000640 1 --2- 172.21.15.161:0/1707948987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa23c066690 0x7fa23c068b50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa2500af000 tx=0x7fa2500a7000 comp rx=0 tx=0).stop 2024-05-22T21:13:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.550+0000 7fa264000640 1 -- 172.21.15.161:0/1707948987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa25c1444a0 msgr2=0x7fa25c0ad900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.550+0000 7fa264000640 1 --2- 172.21.15.161:0/1707948987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa25c1444a0 0x7fa25c0ad900 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa254007e40 tx=0x7fa25400e4b0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.550+0000 7fa264000640 1 -- 172.21.15.161:0/1707948987 shutdown_connections 2024-05-22T21:13:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.550+0000 7fa264000640 1 --2- 172.21.15.161:0/1707948987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa23c066690 0x7fa23c068b50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.550+0000 7fa264000640 1 --2- 172.21.15.161:0/1707948987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa25c1444a0 0x7fa25c0ad900 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.550+0000 7fa264000640 1 --2- 172.21.15.161:0/1707948987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa25c143c40 0x7fa25c0706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.550+0000 7fa264000640 1 --2- 172.21.15.161:0/1707948987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa25c06c9b0 0x7fa25c1f9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.550+0000 7fa264000640 1 -- 172.21.15.161:0/1707948987 >> 172.21.15.161:0/1707948987 conn(0x7fa25c06a2d0 msgr2=0x7fa25c139e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:38.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.550+0000 7fa264000640 1 -- 172.21.15.161:0/1707948987 shutdown_connections 2024-05-22T21:13:38.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.550+0000 7fa264000640 1 -- 172.21.15.161:0/1707948987 wait complete. 2024-05-22T21:13:38.573 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:13:38.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.702+0000 7f7b66600640 1 Processor -- start 2024-05-22T21:13:38.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.702+0000 7f7b66600640 1 -- start start 2024-05-22T21:13:38.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.706+0000 7f7b66600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7b6015e130 0x7f7b6015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.706+0000 7f7b66600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7b6015d410 0x7f7b6015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.706+0000 7f7b66600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b60071670 con 0x7f7b6015dd50 2024-05-22T21:13:38.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.706+0000 7f7b66600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b6015d870 con 0x7f7b6015d410 2024-05-22T21:13:38.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.706+0000 7f7b66600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b60070a60 con 0x7f7b6015e130 2024-05-22T21:13:38.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.706+0000 7f7b5fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7b6015d410 0x7f7b6015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.706+0000 7f7b5f400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f7b6015dd50 0x7f7b6006d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43504/0 (socket says 172.21.15.161:43504) 2024-05-22T21:13:38.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.706+0000 7f7b5f400640 1 -- 172.21.15.161:0/1996618465 learned_addr learned my addr 172.21.15.161:0/1996618465 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:38.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.706+0000 7f7b5ea00640 1 --2- 172.21.15.161:0/1996618465 >> v2:172.21.15.179:3300/0 conn(0x7f7b6015e130 0x7f7b6015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.706+0000 7f7b5fe00640 1 -- 172.21.15.161:0/1996618465 >> v2:172.21.15.179:3300/0 conn(0x7f7b6015e130 msgr2=0x7f7b6015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:38.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.706+0000 7f7b5fe00640 1 --2- 172.21.15.161:0/1996618465 >> v2:172.21.15.179:3300/0 conn(0x7f7b6015e130 0x7f7b6015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.706+0000 7f7b5fe00640 1 -- 172.21.15.161:0/1996618465 >> v1:172.21.15.161:6789/0 conn(0x7f7b6015dd50 legacy=0x7f7b6006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.706+0000 7f7b5fe00640 1 -- 172.21.15.161:0/1996618465 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b60070d40 con 0x7f7b6015d410 2024-05-22T21:13:38.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.706+0000 7f7b5e000640 1 -- 172.21.15.161:0/1996618465 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513975159 0 0) 0x7f7b60071670 con 0x7f7b6015dd50 2024-05-22T21:13:38.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.706+0000 7f7b5ea00640 1 --2- 172.21.15.161:0/1996618465 >> v2:172.21.15.179:3300/0 conn(0x7f7b6015e130 0x7f7b6015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:38.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.710+0000 7f7b5fe00640 1 --2- 172.21.15.161:0/1996618465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6015d410 0x7f7b6015b6c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7b6006c7c0 tx=0x7f7b5801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1ac161fab3689c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:38.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.710+0000 7f7b5e000640 1 -- 172.21.15.161:0/1996618465 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b580a59c0 con 0x7f7b6015d410 2024-05-22T21:13:38.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.710+0000 7f7b5e000640 1 -- 172.21.15.161:0/1996618465 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b580a5b20 con 0x7f7b6015d410 2024-05-22T21:13:38.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.710+0000 7f7b5e000640 1 -- 172.21.15.161:0/1996618465 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b580ad440 con 0x7f7b6015d410 2024-05-22T21:13:38.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.710+0000 7f7b66600640 1 -- 172.21.15.161:0/1996618465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6015d410 msgr2=0x7f7b6015b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.710+0000 7f7b66600640 1 --2- 172.21.15.161:0/1996618465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6015d410 0x7f7b6015b6c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7b6006c7c0 tx=0x7f7b5801c900 comp rx=0 tx=0).stop 2024-05-22T21:13:38.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.710+0000 7f7b66600640 1 -- 172.21.15.161:0/1996618465 shutdown_connections 2024-05-22T21:13:38.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.710+0000 7f7b66600640 1 --2- 172.21.15.161:0/1996618465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6015d410 0x7f7b6015b6c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.710+0000 7f7b66600640 1 --2- 172.21.15.161:0/1996618465 >> v2:172.21.15.179:3300/0 conn(0x7f7b6015e130 0x7f7b6015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.710+0000 7f7b66600640 1 -- 172.21.15.161:0/1996618465 >> 172.21.15.161:0/1996618465 conn(0x7f7b600b0bc0 msgr2=0x7f7b600b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:38.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.710+0000 7f7b66600640 1 -- 172.21.15.161:0/1996618465 shutdown_connections 2024-05-22T21:13:38.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.710+0000 7f7b66600640 1 -- 172.21.15.161:0/1996618465 wait complete. 2024-05-22T21:13:38.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.710+0000 7f7b66600640 1 Processor -- start 2024-05-22T21:13:38.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.710+0000 7f7b66600640 1 -- start start 2024-05-22T21:13:38.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b66600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b6015d410 0x7f7b600aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b66600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7b6015dd50 0x7f7b6014d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b66600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6015e130 0x7f7b6014ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b66600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6014ec80 con 0x7f7b6015e130 2024-05-22T21:13:38.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b66600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6014e470 con 0x7f7b6015d410 2024-05-22T21:13:38.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b66600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b600b1480 con 0x7f7b6015dd50 2024-05-22T21:13:38.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b5fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6015e130 0x7f7b6014ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b5fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6015e130 0x7f7b6014ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54738/0 (socket says 172.21.15.161:54738) 2024-05-22T21:13:38.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b5fe00640 1 -- 172.21.15.161:0/4227113616 learned_addr learned my addr 172.21.15.161:0/4227113616 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:38.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b5f400640 1 --2- 172.21.15.161:0/4227113616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b6015d410 0x7f7b600aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b5ea00640 1 --2- 172.21.15.161:0/4227113616 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7b6015dd50 0x7f7b6014d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b5fe00640 1 -- 172.21.15.161:0/4227113616 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7b6015dd50 msgr2=0x7f7b6014d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b5fe00640 1 --2- 172.21.15.161:0/4227113616 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7b6015dd50 0x7f7b6014d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b5fe00640 1 -- 172.21.15.161:0/4227113616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b6015d410 msgr2=0x7f7b600aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b5fe00640 1 --2- 172.21.15.161:0/4227113616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b6015d410 0x7f7b600aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b5fe00640 1 -- 172.21.15.161:0/4227113616 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b58009a70 con 0x7f7b6015e130 2024-05-22T21:13:38.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b5f400640 1 --2- 172.21.15.161:0/4227113616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b6015d410 0x7f7b600aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:38.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b5fe00640 1 --2- 172.21.15.161:0/4227113616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6015e130 0x7f7b6014ddf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7b58019510 tx=0x7f7b580196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:38.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b57400640 1 -- 172.21.15.161:0/4227113616 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b580ade20 con 0x7f7b6015e130 2024-05-22T21:13:38.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b57400640 1 -- 172.21.15.161:0/4227113616 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b580b9450 con 0x7f7b6015e130 2024-05-22T21:13:38.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b66600640 1 -- 172.21.15.161:0/4227113616 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6014e6c0 con 0x7f7b6015e130 2024-05-22T21:13:38.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b57400640 1 -- 172.21.15.161:0/4227113616 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b580b9800 con 0x7f7b6015e130 2024-05-22T21:13:38.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b66600640 1 -- 172.21.15.161:0/4227113616 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b600af2d0 con 0x7f7b6015e130 2024-05-22T21:13:38.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b66600640 1 -- 172.21.15.161:0/4227113616 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b600af4c0 con 0x7f7b6015e130 2024-05-22T21:13:38.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.714+0000 7f7b57400640 1 -- 172.21.15.161:0/4227113616 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7b580b9960 con 0x7f7b6015e130 2024-05-22T21:13:38.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.718+0000 7f7b57400640 1 --2- 172.21.15.161:0/4227113616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7b3c066690 0x7f7b3c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:38.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.718+0000 7f7b57400640 1 -- 172.21.15.161:0/4227113616 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b5811fba0 con 0x7f7b6015e130 2024-05-22T21:13:38.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.718+0000 7f7b5f400640 1 --2- 172.21.15.161:0/4227113616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7b3c066690 0x7f7b3c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:38.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.718+0000 7f7b5f400640 1 --2- 172.21.15.161:0/4227113616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7b3c066690 0x7f7b3c068b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7b48007fd0 tx=0x7f7b48007c20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:38.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.722+0000 7f7b57400640 1 -- 172.21.15.161:0/4227113616 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7b600af4c0 con 0x7f7b6015e130 2024-05-22T21:13:38.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.906+0000 7f7b66600640 1 -- 172.21.15.161:0/4227113616 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b600af4c0 con 0x7f7b6015e130 2024-05-22T21:13:38.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.910+0000 7f7b57400640 1 -- 172.21.15.161:0/4227113616 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4740 (secure 0 0 0) 0x7f7b600af4c0 con 0x7f7b6015e130 2024-05-22T21:13:38.912 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:38.912 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3179612470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3179612470},{"type":"v1","addr":"172.21.15.179:6834","nonce":3179612470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2196055309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2196055309},{"type":"v1","addr":"172.21.15.161:6835","nonce":2196055309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1793681759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1793681759},{"type":"v1","addr":"172.21.15.161:6837","nonce":1793681759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:13:23.418412+0000","modified":"2024-05-22T21:13:26.442354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/3095599688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3095599688},{"type":"v1","addr":"172.21.15.179:6835","nonce":3095599688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-22T21:13:38.913 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 8 2024-05-22T21:13:38.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.910+0000 7f7b66600640 1 -- 172.21.15.161:0/4227113616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7b3c066690 msgr2=0x7f7b3c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.910+0000 7f7b66600640 1 --2- 172.21.15.161:0/4227113616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7b3c066690 0x7f7b3c068b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7b48007fd0 tx=0x7f7b48007c20 comp rx=0 tx=0).stop 2024-05-22T21:13:38.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.910+0000 7f7b66600640 1 -- 172.21.15.161:0/4227113616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6015e130 msgr2=0x7f7b6014ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:38.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.910+0000 7f7b66600640 1 --2- 172.21.15.161:0/4227113616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6015e130 0x7f7b6014ddf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7b58019510 tx=0x7f7b580196a0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.910+0000 7f7b66600640 1 -- 172.21.15.161:0/4227113616 shutdown_connections 2024-05-22T21:13:38.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.910+0000 7f7b66600640 1 --2- 172.21.15.161:0/4227113616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7b3c066690 0x7f7b3c068b50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.910+0000 7f7b66600640 1 --2- 172.21.15.161:0/4227113616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b6015e130 0x7f7b6014ddf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.910+0000 7f7b66600640 1 --2- 172.21.15.161:0/4227113616 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7b6015dd50 0x7f7b6014d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.910+0000 7f7b66600640 1 --2- 172.21.15.161:0/4227113616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b6015d410 0x7f7b600aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:38.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.910+0000 7f7b66600640 1 -- 172.21.15.161:0/4227113616 >> 172.21.15.161:0/4227113616 conn(0x7f7b600b0bc0 msgr2=0x7f7b60157360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:38.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.910+0000 7f7b66600640 1 -- 172.21.15.161:0/4227113616 shutdown_connections 2024-05-22T21:13:38.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:38.910+0000 7f7b66600640 1 -- 172.21.15.161:0/4227113616 wait complete. 2024-05-22T21:13:38.934 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:39.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.070+0000 7f4eeb600640 1 Processor -- start 2024-05-22T21:13:39.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.070+0000 7f4eeb600640 1 -- start start 2024-05-22T21:13:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.070+0000 7f4eeb600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4ee4160ae0 0x7f4ee40cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.070+0000 7f4eeb600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee40b2380 con 0x7f4ee40ced60 2024-05-22T21:13:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.070+0000 7f4eeb600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee40d04d0 con 0x7f4ee40d0090 2024-05-22T21:13:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.070+0000 7f4eeb600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee40d1970 con 0x7f4ee4160ae0 2024-05-22T21:13:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.070+0000 7f4ee9600640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f4ee40ced60 0x7f4ee40b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43508/0 (socket says 172.21.15.161:43508) 2024-05-22T21:13:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.070+0000 7f4ee9600640 1 -- 172.21.15.161:0/1659488687 learned_addr learned my addr 172.21.15.161:0/1659488687 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.070+0000 7f4ee8c00640 1 --2- 172.21.15.161:0/1659488687 >> v2:172.21.15.179:3300/0 conn(0x7f4ee4160ae0 0x7f4ee40cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.070+0000 7f4ee3400640 1 -- 172.21.15.161:0/1659488687 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2308224630 0 0) 0x7f4ee40b2380 con 0x7f4ee40ced60 2024-05-22T21:13:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.070+0000 7f4ee3400640 1 -- 172.21.15.161:0/1659488687 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ecc003530 con 0x7f4ee40ced60 2024-05-22T21:13:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.070+0000 7f4ee8c00640 1 -- 172.21.15.161:0/1659488687 >> v1:172.21.15.179:6789/0 conn(0x7f4ee40d0090 legacy=0x7f4ee40b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.070+0000 7f4ee8c00640 1 -- 172.21.15.161:0/1659488687 >> v1:172.21.15.161:6789/0 conn(0x7f4ee40ced60 legacy=0x7f4ee40b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.070+0000 7f4ee8c00640 1 -- 172.21.15.161:0/1659488687 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee40d1c50 con 0x7f4ee4160ae0 2024-05-22T21:13:39.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.074+0000 7f4ee8c00640 1 --2- 172.21.15.161:0/1659488687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee4160ae0 0x7f4ee40cfb50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80097c0 tx=0x7f4ed801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a97fa6a5512f55bf server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:39.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.074+0000 7f4ee3400640 1 -- 172.21.15.161:0/1659488687 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed80ac640 con 0x7f4ee4160ae0 2024-05-22T21:13:39.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.074+0000 7f4ee3400640 1 -- 172.21.15.161:0/1659488687 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed80ac7a0 con 0x7f4ee4160ae0 2024-05-22T21:13:39.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.074+0000 7f4ee3400640 1 -- 172.21.15.161:0/1659488687 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed80acb30 con 0x7f4ee4160ae0 2024-05-22T21:13:39.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.074+0000 7f4eeb600640 1 -- 172.21.15.161:0/1659488687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee4160ae0 msgr2=0x7f4ee40cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.074+0000 7f4eeb600640 1 --2- 172.21.15.161:0/1659488687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee4160ae0 0x7f4ee40cfb50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80097c0 tx=0x7f4ed801c3e0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.074+0000 7f4eeb600640 1 -- 172.21.15.161:0/1659488687 shutdown_connections 2024-05-22T21:13:39.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.074+0000 7f4eeb600640 1 --2- 172.21.15.161:0/1659488687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee4160ae0 0x7f4ee40cfb50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.074+0000 7f4eeb600640 1 -- 172.21.15.161:0/1659488687 >> 172.21.15.161:0/1659488687 conn(0x7f4ee409c490 msgr2=0x7f4ee40d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:39.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.074+0000 7f4eeb600640 1 -- 172.21.15.161:0/1659488687 shutdown_connections 2024-05-22T21:13:39.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.074+0000 7f4eeb600640 1 -- 172.21.15.161:0/1659488687 wait complete. 2024-05-22T21:13:39.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.074+0000 7f4eeb600640 1 Processor -- start 2024-05-22T21:13:39.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4eeb600640 1 -- start start 2024-05-22T21:13:39.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4eeb600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee40ced60 0x7f4ee414f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4eeb600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee40d0090 0x7f4ee40b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4eeb600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4ee4160ae0 0x7f4ee40c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4eeb600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee4152260 con 0x7f4ee40d0090 2024-05-22T21:13:39.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4eeb600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee414f720 con 0x7f4ee40ced60 2024-05-22T21:13:39.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4eeb600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee40b8c30 con 0x7f4ee4160ae0 2024-05-22T21:13:39.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee40d0090 0x7f4ee40b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4ee8c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee40ced60 0x7f4ee414f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee40d0090 0x7f4ee40b86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54746/0 (socket says 172.21.15.161:54746) 2024-05-22T21:13:39.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4ee3e00640 1 -- 172.21.15.161:0/3143045103 learned_addr learned my addr 172.21.15.161:0/3143045103 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:39.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4ee9600640 1 --2- 172.21.15.161:0/3143045103 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4ee4160ae0 0x7f4ee40c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4ee8c00640 1 -- 172.21.15.161:0/3143045103 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4ee4160ae0 msgr2=0x7f4ee40c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4ee8c00640 1 --2- 172.21.15.161:0/3143045103 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4ee4160ae0 0x7f4ee40c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4ee8c00640 1 -- 172.21.15.161:0/3143045103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee40d0090 msgr2=0x7f4ee40b86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4ee8c00640 1 --2- 172.21.15.161:0/3143045103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee40d0090 0x7f4ee40b86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4ee8c00640 1 -- 172.21.15.161:0/3143045103 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed8009420 con 0x7f4ee40ced60 2024-05-22T21:13:39.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4ee3e00640 1 --2- 172.21.15.161:0/3143045103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee40d0090 0x7f4ee40b86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:39.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4ee8c00640 1 --2- 172.21.15.161:0/3143045103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee40ced60 0x7f4ee414f1e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80ac000 tx=0x7f4ed801ac20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:39.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4ee0c00640 1 -- 172.21.15.161:0/3143045103 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed80bd070 con 0x7f4ee40ced60 2024-05-22T21:13:39.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4ee0c00640 1 -- 172.21.15.161:0/3143045103 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed80b7420 con 0x7f4ee40ced60 2024-05-22T21:13:39.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4ee0c00640 1 -- 172.21.15.161:0/3143045103 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed80b77d0 con 0x7f4ee40ced60 2024-05-22T21:13:39.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4eeb600640 1 -- 172.21.15.161:0/3143045103 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee41518c0 con 0x7f4ee40ced60 2024-05-22T21:13:39.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4eeb600640 1 -- 172.21.15.161:0/3143045103 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee4151e80 con 0x7f4ee40ced60 2024-05-22T21:13:39.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.078+0000 7f4eeb600640 1 -- 172.21.15.161:0/3143045103 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee4067220 con 0x7f4ee40ced60 2024-05-22T21:13:39.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.082+0000 7f4ee0c00640 1 -- 172.21.15.161:0/3143045103 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4ed80b3040 con 0x7f4ee40ced60 2024-05-22T21:13:39.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.082+0000 7f4ee0c00640 1 --2- 172.21.15.161:0/3143045103 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ec4066690 0x7f4ec4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.082+0000 7f4ee0c00640 1 -- 172.21.15.161:0/3143045103 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ed811e860 con 0x7f4ee40ced60 2024-05-22T21:13:39.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.082+0000 7f4ee3e00640 1 --2- 172.21.15.161:0/3143045103 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ec4066690 0x7f4ec4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.082+0000 7f4ee3e00640 1 --2- 172.21.15.161:0/3143045103 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ec4066690 0x7f4ec4068b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4ee41f9d90 tx=0x7f4ed0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:39.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.086+0000 7f4ee0c00640 1 -- 172.21.15.161:0/3143045103 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4ed80ebcf0 con 0x7f4ee40ced60 2024-05-22T21:13:39.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.238+0000 7f4eeb600640 1 -- 172.21.15.161:0/3143045103 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ee40d4070 con 0x7f4ee40ced60 2024-05-22T21:13:39.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.238+0000 7f4ee0c00640 1 -- 172.21.15.161:0/3143045103 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f4ed80ef310 con 0x7f4ee40ced60 2024-05-22T21:13:39.244 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:39.244 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:39.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.242+0000 7f4eeb600640 1 -- 172.21.15.161:0/3143045103 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ec4066690 msgr2=0x7f4ec4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.242+0000 7f4eeb600640 1 --2- 172.21.15.161:0/3143045103 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ec4066690 0x7f4ec4068b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4ee41f9d90 tx=0x7f4ed0009040 comp rx=0 tx=0).stop 2024-05-22T21:13:39.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.242+0000 7f4eeb600640 1 -- 172.21.15.161:0/3143045103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee40ced60 msgr2=0x7f4ee414f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.242+0000 7f4eeb600640 1 --2- 172.21.15.161:0/3143045103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee40ced60 0x7f4ee414f1e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80ac000 tx=0x7f4ed801ac20 comp rx=0 tx=0).stop 2024-05-22T21:13:39.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.242+0000 7f4eeb600640 1 -- 172.21.15.161:0/3143045103 shutdown_connections 2024-05-22T21:13:39.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.242+0000 7f4eeb600640 1 --2- 172.21.15.161:0/3143045103 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ec4066690 0x7f4ec4068b50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.242+0000 7f4eeb600640 1 --2- 172.21.15.161:0/3143045103 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4ee4160ae0 0x7f4ee40c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.242+0000 7f4eeb600640 1 --2- 172.21.15.161:0/3143045103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee40d0090 0x7f4ee40b86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.242+0000 7f4eeb600640 1 --2- 172.21.15.161:0/3143045103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee40ced60 0x7f4ee414f1e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.242+0000 7f4eeb600640 1 -- 172.21.15.161:0/3143045103 >> 172.21.15.161:0/3143045103 conn(0x7f4ee409c490 msgr2=0x7f4ee40afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:39.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.242+0000 7f4eeb600640 1 -- 172.21.15.161:0/3143045103 shutdown_connections 2024-05-22T21:13:39.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.242+0000 7f4eeb600640 1 -- 172.21.15.161:0/3143045103 wait complete. 2024-05-22T21:13:39.265 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:13:21.237503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:13:22.425891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:13:39.265 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:13:39.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.398+0000 7f926a400640 1 Processor -- start 2024-05-22T21:13:39.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.402+0000 7f926a400640 1 -- start start 2024-05-22T21:13:39.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.402+0000 7f926a400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f926415b600 0x7f926406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.402+0000 7f926a400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f926406c7e0 0x7f92640b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.402+0000 7f926a400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9264144bb0 con 0x7f9264143c40 2024-05-22T21:13:39.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.402+0000 7f926a400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926415ba60 con 0x7f926406c7e0 2024-05-22T21:13:39.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.402+0000 7f926a400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926406cc70 con 0x7f926415b600 2024-05-22T21:13:39.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.402+0000 7f9263400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f9264143c40 0x7f92641444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43512/0 (socket says 172.21.15.161:43512) 2024-05-22T21:13:39.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.402+0000 7f9263400640 1 -- 172.21.15.161:0/1556199830 learned_addr learned my addr 172.21.15.161:0/1556199830 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:39.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.402+0000 7f9263e00640 1 --2- 172.21.15.161:0/1556199830 >> v2:172.21.15.179:3300/0 conn(0x7f926406c7e0 0x7f92640b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.402+0000 7f925aa00640 1 --2- 172.21.15.161:0/1556199830 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f926415b600 0x7f926406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.402+0000 7f9262a00640 1 -- 172.21.15.161:0/1556199830 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1418643004 0 0) 0x7f9264144bb0 con 0x7f9264143c40 2024-05-22T21:13:39.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.402+0000 7f9262a00640 1 -- 172.21.15.161:0/1556199830 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9248003530 con 0x7f9264143c40 2024-05-22T21:13:39.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.402+0000 7f9263e00640 1 -- 172.21.15.161:0/1556199830 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f926415b600 msgr2=0x7f926406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:39.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.402+0000 7f9263e00640 1 --2- 172.21.15.161:0/1556199830 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f926415b600 0x7f926406c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.402+0000 7f9263e00640 1 -- 172.21.15.161:0/1556199830 >> v1:172.21.15.161:6789/0 conn(0x7f9264143c40 legacy=0x7f92641444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.402+0000 7f9263e00640 1 -- 172.21.15.161:0/1556199830 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926415efb0 con 0x7f926406c7e0 2024-05-22T21:13:39.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.406+0000 7f9263e00640 1 --2- 172.21.15.161:0/1556199830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926406c7e0 0x7f92640b2160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f925c009d90 tx=0x7f925c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73714c52d8d047a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:39.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.406+0000 7f9262a00640 1 -- 172.21.15.161:0/1556199830 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c0a5a30 con 0x7f926406c7e0 2024-05-22T21:13:39.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.406+0000 7f9262a00640 1 -- 172.21.15.161:0/1556199830 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925c0a5b90 con 0x7f926406c7e0 2024-05-22T21:13:39.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.406+0000 7f9262a00640 1 -- 172.21.15.161:0/1556199830 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c0ad440 con 0x7f926406c7e0 2024-05-22T21:13:39.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.406+0000 7f926a400640 1 -- 172.21.15.161:0/1556199830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926406c7e0 msgr2=0x7f92640b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.406+0000 7f926a400640 1 --2- 172.21.15.161:0/1556199830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926406c7e0 0x7f92640b2160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f925c009d90 tx=0x7f925c01c970 comp rx=0 tx=0).stop 2024-05-22T21:13:39.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.406+0000 7f926a400640 1 -- 172.21.15.161:0/1556199830 shutdown_connections 2024-05-22T21:13:39.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.406+0000 7f926a400640 1 --2- 172.21.15.161:0/1556199830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926406c7e0 0x7f92640b2160 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.406+0000 7f926a400640 1 --2- 172.21.15.161:0/1556199830 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f926415b600 0x7f926406c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.406+0000 7f926a400640 1 -- 172.21.15.161:0/1556199830 >> 172.21.15.161:0/1556199830 conn(0x7f926406a2d0 msgr2=0x7f926406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:39.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.406+0000 7f926a400640 1 -- 172.21.15.161:0/1556199830 shutdown_connections 2024-05-22T21:13:39.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.406+0000 7f926a400640 1 -- 172.21.15.161:0/1556199830 wait complete. 2024-05-22T21:13:39.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.406+0000 7f926a400640 1 Processor -- start 2024-05-22T21:13:39.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.406+0000 7f926a400640 1 -- start start 2024-05-22T21:13:39.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f926a400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f926406c7e0 0x7f9264139980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f926a400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9264143c40 0x7f926415e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f926a400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926415b600 0x7f926415a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f926a400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92641394f0 con 0x7f9264143c40 2024-05-22T21:13:39.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f926a400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92641388e0 con 0x7f926415b600 2024-05-22T21:13:39.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f926a400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9264138bc0 con 0x7f926406c7e0 2024-05-22T21:13:39.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f925aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9264143c40 0x7f926415e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f925aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9264143c40 0x7f926415e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54750/0 (socket says 172.21.15.161:54750) 2024-05-22T21:13:39.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f925aa00640 1 -- 172.21.15.161:0/3035197570 learned_addr learned my addr 172.21.15.161:0/3035197570 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:39.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f9263400640 1 --2- 172.21.15.161:0/3035197570 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f926406c7e0 0x7f9264139980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f9263e00640 1 --2- 172.21.15.161:0/3035197570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926415b600 0x7f926415a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f925aa00640 1 -- 172.21.15.161:0/3035197570 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f926406c7e0 msgr2=0x7f9264139980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f925aa00640 1 --2- 172.21.15.161:0/3035197570 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f926406c7e0 0x7f9264139980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f925aa00640 1 -- 172.21.15.161:0/3035197570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926415b600 msgr2=0x7f926415a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f925aa00640 1 --2- 172.21.15.161:0/3035197570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926415b600 0x7f926415a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f925aa00640 1 -- 172.21.15.161:0/3035197570 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c009a70 con 0x7f9264143c40 2024-05-22T21:13:39.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f925aa00640 1 --2- 172.21.15.161:0/3035197570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9264143c40 0x7f926415e140 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f924c00e870 tx=0x7f924c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:39.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f925be00640 1 -- 172.21.15.161:0/3035197570 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c009510 con 0x7f9264143c40 2024-05-22T21:13:39.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f925be00640 1 -- 172.21.15.161:0/3035197570 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924c00eec0 con 0x7f9264143c40 2024-05-22T21:13:39.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f925be00640 1 -- 172.21.15.161:0/3035197570 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c00cb40 con 0x7f9264143c40 2024-05-22T21:13:39.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f926a400640 1 -- 172.21.15.161:0/3035197570 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9264138ea0 con 0x7f9264143c40 2024-05-22T21:13:39.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f926a400640 1 -- 172.21.15.161:0/3035197570 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926415e680 con 0x7f9264143c40 2024-05-22T21:13:39.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.410+0000 7f926a400640 1 -- 172.21.15.161:0/3035197570 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92641396a0 con 0x7f9264143c40 2024-05-22T21:13:39.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.414+0000 7f925be00640 1 -- 172.21.15.161:0/3035197570 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f924c00cca0 con 0x7f9264143c40 2024-05-22T21:13:39.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.414+0000 7f925be00640 1 --2- 172.21.15.161:0/3035197570 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f92400666c0 0x7f9240068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.414+0000 7f925be00640 1 -- 172.21.15.161:0/3035197570 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f924c084310 con 0x7f9264143c40 2024-05-22T21:13:39.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.414+0000 7f9263400640 1 --2- 172.21.15.161:0/3035197570 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f92400666c0 0x7f9240068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.414+0000 7f9263400640 1 --2- 172.21.15.161:0/3035197570 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f92400666c0 0x7f9240068b80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92540011f0 tx=0x7f9254009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:39.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.418+0000 7f925be00640 1 -- 172.21.15.161:0/3035197570 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f92641396a0 con 0x7f9264143c40 2024-05-22T21:13:39.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.578+0000 7f926a400640 1 -- 172.21.15.161:0/3035197570 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9264067220 con 0x7f9264143c40 2024-05-22T21:13:39.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.578+0000 7f925be00640 1 -- 172.21.15.161:0/3035197570 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f924c0515e0 con 0x7f9264143c40 2024-05-22T21:13:39.581 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:13:39.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.578+0000 7f926a400640 1 -- 172.21.15.161:0/3035197570 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f92400666c0 msgr2=0x7f9240068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.578+0000 7f926a400640 1 --2- 172.21.15.161:0/3035197570 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f92400666c0 0x7f9240068b80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92540011f0 tx=0x7f9254009040 comp rx=0 tx=0).stop 2024-05-22T21:13:39.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.578+0000 7f926a400640 1 -- 172.21.15.161:0/3035197570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9264143c40 msgr2=0x7f926415e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.578+0000 7f926a400640 1 --2- 172.21.15.161:0/3035197570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9264143c40 0x7f926415e140 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f924c00e870 tx=0x7f924c00ec70 comp rx=0 tx=0).stop 2024-05-22T21:13:39.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.578+0000 7f926a400640 1 -- 172.21.15.161:0/3035197570 shutdown_connections 2024-05-22T21:13:39.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.578+0000 7f926a400640 1 --2- 172.21.15.161:0/3035197570 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f92400666c0 0x7f9240068b80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.578+0000 7f926a400640 1 --2- 172.21.15.161:0/3035197570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f926415b600 0x7f926415a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.578+0000 7f926a400640 1 --2- 172.21.15.161:0/3035197570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9264143c40 0x7f926415e140 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.578+0000 7f926a400640 1 --2- 172.21.15.161:0/3035197570 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f926406c7e0 0x7f9264139980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.578+0000 7f926a400640 1 -- 172.21.15.161:0/3035197570 >> 172.21.15.161:0/3035197570 conn(0x7f926406a2d0 msgr2=0x7f92640b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:39.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.578+0000 7f926a400640 1 -- 172.21.15.161:0/3035197570 shutdown_connections 2024-05-22T21:13:39.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.578+0000 7f926a400640 1 -- 172.21.15.161:0/3035197570 wait complete. 2024-05-22T21:13:39.603 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:13:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6827600640 1 Processor -- start 2024-05-22T21:13:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6827600640 1 -- start start 2024-05-22T21:13:39.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6827600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f682006c2a0 0x7f682006c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6827600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f68200b3370 0x7f68200b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6827600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682015d9d0 con 0x7f68201402b0 2024-05-22T21:13:39.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6827600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f682015d7c0 con 0x7f68200b3370 2024-05-22T21:13:39.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6827600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682006cc60 con 0x7f682006c2a0 2024-05-22T21:13:39.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6824c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f68201402b0 0x7f6820070590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43522/0 (socket says 172.21.15.161:43522) 2024-05-22T21:13:39.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6824c00640 1 -- 172.21.15.161:0/2215869882 learned_addr learned my addr 172.21.15.161:0/2215869882 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:39.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6825600640 1 --2- 172.21.15.161:0/2215869882 >> v2:172.21.15.179:3300/0 conn(0x7f68200b3370 0x7f68200b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681f400640 1 -- 172.21.15.161:0/2215869882 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142061240 0 0) 0x7f682015d9d0 con 0x7f68201402b0 2024-05-22T21:13:39.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681fe00640 1 --2- 172.21.15.161:0/2215869882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f682006c2a0 0x7f682006c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681f400640 1 -- 172.21.15.161:0/2215869882 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6808003530 con 0x7f68201402b0 2024-05-22T21:13:39.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681f400640 1 -- 172.21.15.161:0/2215869882 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 91570046 0 0) 0x7f6808003530 con 0x7f68201402b0 2024-05-22T21:13:39.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681f400640 1 -- 172.21.15.161:0/2215869882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f682006c2a0 msgr2=0x7f682006c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:39.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681f400640 1 --2- 172.21.15.161:0/2215869882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f682006c2a0 0x7f682006c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681f400640 1 -- 172.21.15.161:0/2215869882 >> v2:172.21.15.179:3300/0 conn(0x7f68200b3370 msgr2=0x7f68200b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:39.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681f400640 1 --2- 172.21.15.161:0/2215869882 >> v2:172.21.15.179:3300/0 conn(0x7f68200b3370 0x7f68200b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681f400640 1 -- 172.21.15.161:0/2215869882 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682015dcb0 con 0x7f68201402b0 2024-05-22T21:13:39.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681f400640 1 -- 172.21.15.161:0/2215869882 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f6814001680 con 0x7f68201402b0 2024-05-22T21:13:39.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681f400640 1 -- 172.21.15.161:0/2215869882 >> v1:172.21.15.161:6789/0 conn(0x7f68201402b0 legacy=0x7f6820070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681f400640 1 --2- 172.21.15.161:0/2215869882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f682006c2a0 0x7f6808003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681f400640 1 --2- 172.21.15.161:0/2215869882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6808003f00 0x7f68080047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681f400640 1 --2- 172.21.15.161:0/2215869882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6808004d10 0x7f6808005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681f400640 1 -- 172.21.15.161:0/2215869882 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682015d7c0 con 0x7f6808004d10 2024-05-22T21:13:39.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681f400640 1 -- 172.21.15.161:0/2215869882 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682006cc60 con 0x7f6808003f00 2024-05-22T21:13:39.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681f400640 1 -- 172.21.15.161:0/2215869882 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680800b990 con 0x7f682006c2a0 2024-05-22T21:13:39.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6825600640 1 --2- 172.21.15.161:0/2215869882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6808004d10 0x7f6808005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f681fe00640 1 --2- 172.21.15.161:0/2215869882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6808003f00 0x7f68080047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6824c00640 1 --2- 172.21.15.161:0/2215869882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f682006c2a0 0x7f6808003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6825600640 1 -- 172.21.15.161:0/2215869882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f682006c2a0 msgr2=0x7f6808003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:39.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6825600640 1 --2- 172.21.15.161:0/2215869882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f682006c2a0 0x7f6808003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6825600640 1 -- 172.21.15.161:0/2215869882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6808003f00 msgr2=0x7f68080047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:39.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6825600640 1 --2- 172.21.15.161:0/2215869882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6808003f00 0x7f68080047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6825600640 1 -- 172.21.15.161:0/2215869882 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6808003530 con 0x7f6808004d10 2024-05-22T21:13:39.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.738+0000 7f6824c00640 1 --2- 172.21.15.161:0/2215869882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f682006c2a0 0x7f6808003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:39.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.742+0000 7f6825600640 1 --2- 172.21.15.161:0/2215869882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6808004d10 0x7f6808005190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6818009fc0 tx=0x7f681801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273b8522cc55cf03 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:39.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.742+0000 7f681f400640 1 -- 172.21.15.161:0/2215869882 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68180a5cd0 con 0x7f6808004d10 2024-05-22T21:13:39.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.742+0000 7f681f400640 1 -- 172.21.15.161:0/2215869882 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681800f040 con 0x7f6808004d10 2024-05-22T21:13:39.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.742+0000 7f681f400640 1 -- 172.21.15.161:0/2215869882 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68180ad680 con 0x7f6808004d10 2024-05-22T21:13:39.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.742+0000 7f6827600640 1 -- 172.21.15.161:0/2215869882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6808004d10 msgr2=0x7f6808005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.742+0000 7f6827600640 1 --2- 172.21.15.161:0/2215869882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6808004d10 0x7f6808005190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6818009fc0 tx=0x7f681801cc60 comp rx=0 tx=0).stop 2024-05-22T21:13:39.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.742+0000 7f6825600640 1 -- 172.21.15.161:0/2215869882 reap_dead start 2024-05-22T21:13:39.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.742+0000 7f6827600640 1 -- 172.21.15.161:0/2215869882 shutdown_connections 2024-05-22T21:13:39.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.742+0000 7f6827600640 1 -- 172.21.15.161:0/2215869882 >> 172.21.15.161:0/2215869882 conn(0x7f682006a2d0 msgr2=0x7f682006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:39.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.742+0000 7f6827600640 1 -- 172.21.15.161:0/2215869882 shutdown_connections 2024-05-22T21:13:39.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.742+0000 7f6827600640 1 -- 172.21.15.161:0/2215869882 wait complete. 2024-05-22T21:13:39.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6827600640 1 Processor -- start 2024-05-22T21:13:39.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6827600640 1 -- start start 2024-05-22T21:13:39.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6827600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6808004d10 0x7f6820157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6827600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f682015a8c0 0x7f6820158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6827600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68201592c0 0x7f6820141b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6827600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6820159820 con 0x7f6808004d10 2024-05-22T21:13:39.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6827600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6820159b00 con 0x7f68201592c0 2024-05-22T21:13:39.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6827600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6820159de0 con 0x7f682015a8c0 2024-05-22T21:13:39.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6824c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6808004d10 0x7f6820157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6824c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6808004d10 0x7f6820157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54770/0 (socket says 172.21.15.161:54770) 2024-05-22T21:13:39.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6824c00640 1 -- 172.21.15.161:0/3516568438 learned_addr learned my addr 172.21.15.161:0/3516568438 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:39.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6825600640 1 --2- 172.21.15.161:0/3516568438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68201592c0 0x7f6820141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f681fe00640 1 --2- 172.21.15.161:0/3516568438 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f682015a8c0 0x7f6820158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6824c00640 1 -- 172.21.15.161:0/3516568438 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f682015a8c0 msgr2=0x7f6820158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6824c00640 1 --2- 172.21.15.161:0/3516568438 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f682015a8c0 0x7f6820158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6824c00640 1 -- 172.21.15.161:0/3516568438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68201592c0 msgr2=0x7f6820141b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6824c00640 1 --2- 172.21.15.161:0/3516568438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68201592c0 0x7f6820141b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6824c00640 1 -- 172.21.15.161:0/3516568438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682015a0c0 con 0x7f6808004d10 2024-05-22T21:13:39.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6824c00640 1 --2- 172.21.15.161:0/3516568438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6808004d10 0x7f6820157330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f681400e8f0 tx=0x7f681400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:39.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f681cc00640 1 -- 172.21.15.161:0/3516568438 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814004a00 con 0x7f6808004d10 2024-05-22T21:13:39.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f681cc00640 1 -- 172.21.15.161:0/3516568438 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6814004b60 con 0x7f6808004d10 2024-05-22T21:13:39.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6827600640 1 -- 172.21.15.161:0/3516568438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f682015a3a0 con 0x7f6808004d10 2024-05-22T21:13:39.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f681cc00640 1 -- 172.21.15.161:0/3516568438 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814005030 con 0x7f6808004d10 2024-05-22T21:13:39.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6827600640 1 -- 172.21.15.161:0/3516568438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6820157870 con 0x7f6808004d10 2024-05-22T21:13:39.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.746+0000 7f6827600640 1 -- 172.21.15.161:0/3516568438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f682015a5e0 con 0x7f6808004d10 2024-05-22T21:13:39.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.750+0000 7f681cc00640 1 -- 172.21.15.161:0/3516568438 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6814005190 con 0x7f6808004d10 2024-05-22T21:13:39.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.750+0000 7f681cc00640 1 --2- 172.21.15.161:0/3516568438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6800066690 0x7f6800068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:39.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.750+0000 7f681cc00640 1 -- 172.21.15.161:0/3516568438 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6814083d00 con 0x7f6808004d10 2024-05-22T21:13:39.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.750+0000 7f681fe00640 1 --2- 172.21.15.161:0/3516568438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6800066690 0x7f6800068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:39.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.750+0000 7f681fe00640 1 --2- 172.21.15.161:0/3516568438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6800066690 0x7f6800068b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f682015ad20 tx=0x7f680c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:39.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.754+0000 7f681cc00640 1 -- 172.21.15.161:0/3516568438 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f682015a5e0 con 0x7f6808004d10 2024-05-22T21:13:39.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.906+0000 7f6827600640 1 -- 172.21.15.161:0/3516568438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f682015a5e0 con 0x7f6808004d10 2024-05-22T21:13:39.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.906+0000 7f681cc00640 1 -- 172.21.15.161:0/3516568438 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f68140190a0 con 0x7f6808004d10 2024-05-22T21:13:39.910 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:13:39.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.910+0000 7f6827600640 1 -- 172.21.15.161:0/3516568438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6800066690 msgr2=0x7f6800068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.910+0000 7f6827600640 1 --2- 172.21.15.161:0/3516568438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6800066690 0x7f6800068b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f682015ad20 tx=0x7f680c009190 comp rx=0 tx=0).stop 2024-05-22T21:13:39.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.910+0000 7f6827600640 1 -- 172.21.15.161:0/3516568438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6808004d10 msgr2=0x7f6820157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:39.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.910+0000 7f6827600640 1 --2- 172.21.15.161:0/3516568438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6808004d10 0x7f6820157330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f681400e8f0 tx=0x7f681400ecf0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.910+0000 7f6827600640 1 -- 172.21.15.161:0/3516568438 shutdown_connections 2024-05-22T21:13:39.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.910+0000 7f6827600640 1 --2- 172.21.15.161:0/3516568438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6800066690 0x7f6800068b50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.910+0000 7f6827600640 1 --2- 172.21.15.161:0/3516568438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68201592c0 0x7f6820141b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.910+0000 7f6827600640 1 --2- 172.21.15.161:0/3516568438 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f682015a8c0 0x7f6820158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.910+0000 7f6827600640 1 --2- 172.21.15.161:0/3516568438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6808004d10 0x7f6820157330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:39.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.910+0000 7f6827600640 1 -- 172.21.15.161:0/3516568438 >> 172.21.15.161:0/3516568438 conn(0x7f682006a2d0 msgr2=0x7f682013ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:39.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.910+0000 7f6827600640 1 -- 172.21.15.161:0/3516568438 shutdown_connections 2024-05-22T21:13:39.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:39.910+0000 7f6827600640 1 -- 172.21.15.161:0/3516568438 wait complete. 2024-05-22T21:13:39.930 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:13:40.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f5936600640 1 Processor -- start 2024-05-22T21:13:40.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f5936600640 1 -- start start 2024-05-22T21:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f5936600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5930143c40 0x7f593015b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f5936600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59301444a0 0x7f593006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f5936600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f593015efb0 con 0x7f5930144900 2024-05-22T21:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f5936600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f593015f290 con 0x7f5930143c40 2024-05-22T21:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f5936600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593015f570 con 0x7f59301444a0 2024-05-22T21:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f592f400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5930143c40 0x7f593015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f592fe00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f5930144900 0x7f59300b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43528/0 (socket says 172.21.15.161:43528) 2024-05-22T21:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f592f400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5930143c40 0x7f593015b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54774/0 (socket says 172.21.15.161:54774) 2024-05-22T21:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f592fe00640 1 -- 172.21.15.161:0/1679679238 learned_addr learned my addr 172.21.15.161:0/1679679238 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f5926a00640 1 --2- 172.21.15.161:0/1679679238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59301444a0 0x7f593006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f592f400640 1 -- 172.21.15.161:0/1679679238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59301444a0 msgr2=0x7f593006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f592f400640 1 --2- 172.21.15.161:0/1679679238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59301444a0 0x7f593006c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f592f400640 1 -- 172.21.15.161:0/1679679238 >> v1:172.21.15.161:6789/0 conn(0x7f5930144900 legacy=0x7f59300b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f592f400640 1 -- 172.21.15.161:0/1679679238 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593015f850 con 0x7f5930143c40 2024-05-22T21:13:40.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.070+0000 7f592ea00640 1 -- 172.21.15.161:0/1679679238 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658785547 0 0) 0x7f593015efb0 con 0x7f5930144900 2024-05-22T21:13:40.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.074+0000 7f592f400640 1 --2- 172.21.15.161:0/1679679238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5930143c40 0x7f593015b600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f59200097c0 tx=0x7f592001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71deea70fc6954b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.074+0000 7f592ea00640 1 -- 172.21.15.161:0/1679679238 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200ac410 con 0x7f5930143c40 2024-05-22T21:13:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.074+0000 7f592ea00640 1 -- 172.21.15.161:0/1679679238 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59200ac570 con 0x7f5930143c40 2024-05-22T21:13:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.074+0000 7f592ea00640 1 -- 172.21.15.161:0/1679679238 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200ac900 con 0x7f5930143c40 2024-05-22T21:13:40.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.074+0000 7f5936600640 1 -- 172.21.15.161:0/1679679238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5930143c40 msgr2=0x7f593015b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.074+0000 7f5936600640 1 --2- 172.21.15.161:0/1679679238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5930143c40 0x7f593015b600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f59200097c0 tx=0x7f592001c4a0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.074+0000 7f5936600640 1 -- 172.21.15.161:0/1679679238 shutdown_connections 2024-05-22T21:13:40.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.074+0000 7f5936600640 1 --2- 172.21.15.161:0/1679679238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59301444a0 0x7f593006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.074+0000 7f5936600640 1 --2- 172.21.15.161:0/1679679238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5930143c40 0x7f593015b600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.074+0000 7f5936600640 1 -- 172.21.15.161:0/1679679238 >> 172.21.15.161:0/1679679238 conn(0x7f593006a2d0 msgr2=0x7f593006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:40.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.074+0000 7f5936600640 1 -- 172.21.15.161:0/1679679238 shutdown_connections 2024-05-22T21:13:40.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.074+0000 7f5936600640 1 -- 172.21.15.161:0/1679679238 wait complete. 2024-05-22T21:13:40.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f5936600640 1 Processor -- start 2024-05-22T21:13:40.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f5936600640 1 -- start start 2024-05-22T21:13:40.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f5936600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5930143c40 0x7f5930139560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f5936600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59301444a0 0x7f59301f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f5936600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5930144900 0x7f5930156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f5936600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5930156f90 con 0x7f5930143c40 2024-05-22T21:13:40.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f5936600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59300ac9b0 con 0x7f5930144900 2024-05-22T21:13:40.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f5936600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59300acc90 con 0x7f59301444a0 2024-05-22T21:13:40.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f592f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5930143c40 0x7f5930139560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:40.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f592f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5930143c40 0x7f5930139560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54788/0 (socket says 172.21.15.161:54788) 2024-05-22T21:13:40.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f592f400640 1 -- 172.21.15.161:0/2175078819 learned_addr learned my addr 172.21.15.161:0/2175078819 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:40.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f5926a00640 1 --2- 172.21.15.161:0/2175078819 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59301444a0 0x7f59301f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:40.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f592fe00640 1 --2- 172.21.15.161:0/2175078819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5930144900 0x7f5930156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:40.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f592f400640 1 -- 172.21.15.161:0/2175078819 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59301444a0 msgr2=0x7f59301f4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f592f400640 1 --2- 172.21.15.161:0/2175078819 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59301444a0 0x7f59301f4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f592f400640 1 -- 172.21.15.161:0/2175078819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5930144900 msgr2=0x7f5930156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f592f400640 1 --2- 172.21.15.161:0/2175078819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5930144900 0x7f5930156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f592f400640 1 -- 172.21.15.161:0/2175078819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5920009420 con 0x7f5930143c40 2024-05-22T21:13:40.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f5926a00640 1 --2- 172.21.15.161:0/2175078819 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59301444a0 0x7f59301f4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:40.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f592fe00640 1 --2- 172.21.15.161:0/2175078819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5930144900 0x7f5930156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:40.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f592f400640 1 --2- 172.21.15.161:0/2175078819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5930143c40 0x7f5930139560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f592001a0d0 tx=0x7f5920019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:40.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f5927e00640 1 -- 172.21.15.161:0/2175078819 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920006e20 con 0x7f5930143c40 2024-05-22T21:13:40.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f5927e00640 1 -- 172.21.15.161:0/2175078819 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59200b5620 con 0x7f5930143c40 2024-05-22T21:13:40.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f5927e00640 1 -- 172.21.15.161:0/2175078819 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200b59b0 con 0x7f5930143c40 2024-05-22T21:13:40.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f5936600640 1 -- 172.21.15.161:0/2175078819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59300acf70 con 0x7f5930143c40 2024-05-22T21:13:40.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f5936600640 1 -- 172.21.15.161:0/2175078819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59300ac660 con 0x7f5930143c40 2024-05-22T21:13:40.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.078+0000 7f5936600640 1 -- 172.21.15.161:0/2175078819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5930067220 con 0x7f5930143c40 2024-05-22T21:13:40.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.082+0000 7f5927e00640 1 -- 172.21.15.161:0/2175078819 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59200b3040 con 0x7f5930143c40 2024-05-22T21:13:40.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.082+0000 7f5927e00640 1 --2- 172.21.15.161:0/2175078819 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f590c066690 0x7f590c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.082+0000 7f5927e00640 1 -- 172.21.15.161:0/2175078819 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f592011e100 con 0x7f5930143c40 2024-05-22T21:13:40.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.082+0000 7f5926a00640 1 --2- 172.21.15.161:0/2175078819 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f590c066690 0x7f590c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:40.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.082+0000 7f5926a00640 1 --2- 172.21.15.161:0/2175078819 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f590c066690 0x7f590c068b50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5918009840 tx=0x7f5918008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:40.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.086+0000 7f5927e00640 1 -- 172.21.15.161:0/2175078819 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f59200eb3d0 con 0x7f5930143c40 2024-05-22T21:13:40.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.238+0000 7f5936600640 1 -- 172.21.15.161:0/2175078819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f59300ceaa0 con 0x7f5930143c40 2024-05-22T21:13:40.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.238+0000 7f5927e00640 1 -- 172.21.15.161:0/2175078819 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f59200ee9f0 con 0x7f5930143c40 2024-05-22T21:13:40.243 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:13:40.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.242+0000 7f5936600640 1 -- 172.21.15.161:0/2175078819 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f590c066690 msgr2=0x7f590c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.242+0000 7f5936600640 1 --2- 172.21.15.161:0/2175078819 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f590c066690 0x7f590c068b50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5918009840 tx=0x7f5918008040 comp rx=0 tx=0).stop 2024-05-22T21:13:40.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.242+0000 7f5936600640 1 -- 172.21.15.161:0/2175078819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5930143c40 msgr2=0x7f5930139560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.242+0000 7f5936600640 1 --2- 172.21.15.161:0/2175078819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5930143c40 0x7f5930139560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f592001a0d0 tx=0x7f5920019240 comp rx=0 tx=0).stop 2024-05-22T21:13:40.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.242+0000 7f5936600640 1 -- 172.21.15.161:0/2175078819 shutdown_connections 2024-05-22T21:13:40.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.242+0000 7f5936600640 1 --2- 172.21.15.161:0/2175078819 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f590c066690 0x7f590c068b50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.242+0000 7f5936600640 1 --2- 172.21.15.161:0/2175078819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5930144900 0x7f5930156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.242+0000 7f5936600640 1 --2- 172.21.15.161:0/2175078819 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59301444a0 0x7f59301f4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.242+0000 7f5936600640 1 --2- 172.21.15.161:0/2175078819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5930143c40 0x7f5930139560 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.242+0000 7f5936600640 1 -- 172.21.15.161:0/2175078819 >> 172.21.15.161:0/2175078819 conn(0x7f593006a2d0 msgr2=0x7f593015d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:40.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.242+0000 7f5936600640 1 -- 172.21.15.161:0/2175078819 shutdown_connections 2024-05-22T21:13:40.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.242+0000 7f5936600640 1 -- 172.21.15.161:0/2175078819 wait complete. 2024-05-22T21:13:40.265 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:13:40.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.394+0000 7fe80e600640 1 Processor -- start 2024-05-22T21:13:40.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.398+0000 7fe80e600640 1 -- start start 2024-05-22T21:13:40.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.398+0000 7fe80e600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8080715a0 0x7fe80815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.398+0000 7fe80e600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe80815ec90 0x7fe80815f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.398+0000 7fe80e600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8081442a0 con 0x7fe808143ec0 2024-05-22T21:13:40.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.398+0000 7fe80e600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe80815f660 con 0x7fe80815ec90 2024-05-22T21:13:40.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.398+0000 7fe80e600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe808070500 con 0x7fe8080715a0 2024-05-22T21:13:40.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.398+0000 7fe807e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe80815ec90 0x7fe80815f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:40.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.398+0000 7fe807e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe80815ec90 0x7fe80815f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54792/0 (socket says 172.21.15.161:54792) 2024-05-22T21:13:40.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.398+0000 7fe807e00640 1 -- 172.21.15.161:0/673429046 learned_addr learned my addr 172.21.15.161:0/673429046 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:40.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.398+0000 7fe7ffe00640 1 --2- 172.21.15.161:0/673429046 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8080715a0 0x7fe80815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:40.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.398+0000 7fe807e00640 1 -- 172.21.15.161:0/673429046 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8080715a0 msgr2=0x7fe80815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:40.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.398+0000 7fe807e00640 1 --2- 172.21.15.161:0/673429046 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8080715a0 0x7fe80815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.398+0000 7fe807e00640 1 -- 172.21.15.161:0/673429046 >> v1:172.21.15.161:6789/0 conn(0x7fe808143ec0 legacy=0x7fe80806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.398+0000 7fe807e00640 1 -- 172.21.15.161:0/673429046 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8080707e0 con 0x7fe80815ec90 2024-05-22T21:13:40.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.398+0000 7fe806a00640 1 -- 172.21.15.161:0/673429046 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590808310 0 0) 0x7fe8081442a0 con 0x7fe808143ec0 2024-05-22T21:13:40.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.402+0000 7fe807e00640 1 --2- 172.21.15.161:0/673429046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80815ec90 0x7fe80815f120 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe800009e50 tx=0x7fe80001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3271e442dce4033 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:40.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.402+0000 7fe806a00640 1 -- 172.21.15.161:0/673429046 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8000a57d0 con 0x7fe80815ec90 2024-05-22T21:13:40.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.402+0000 7fe806a00640 1 -- 172.21.15.161:0/673429046 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8000a5930 con 0x7fe80815ec90 2024-05-22T21:13:40.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.402+0000 7fe806a00640 1 -- 172.21.15.161:0/673429046 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8000a5cc0 con 0x7fe80815ec90 2024-05-22T21:13:40.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.402+0000 7fe80e600640 1 -- 172.21.15.161:0/673429046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80815ec90 msgr2=0x7fe80815f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.402+0000 7fe80e600640 1 --2- 172.21.15.161:0/673429046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80815ec90 0x7fe80815f120 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe800009e50 tx=0x7fe80001ca00 comp rx=0 tx=0).stop 2024-05-22T21:13:40.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.402+0000 7fe80e600640 1 -- 172.21.15.161:0/673429046 shutdown_connections 2024-05-22T21:13:40.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.402+0000 7fe80e600640 1 --2- 172.21.15.161:0/673429046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80815ec90 0x7fe80815f120 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.402+0000 7fe80e600640 1 --2- 172.21.15.161:0/673429046 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8080715a0 0x7fe80815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.402+0000 7fe80e600640 1 -- 172.21.15.161:0/673429046 >> 172.21.15.161:0/673429046 conn(0x7fe8080b0bc0 msgr2=0x7fe8080b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:40.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.402+0000 7fe80e600640 1 -- 172.21.15.161:0/673429046 shutdown_connections 2024-05-22T21:13:40.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.402+0000 7fe80e600640 1 -- 172.21.15.161:0/673429046 wait complete. 2024-05-22T21:13:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.402+0000 7fe80e600640 1 Processor -- start 2024-05-22T21:13:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.402+0000 7fe80e600640 1 -- start start 2024-05-22T21:13:40.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe80e600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe808143ec0 0x7fe80813f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe80e600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80815ec90 0x7fe8081601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe80e600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe80815d890 0x7fe80813fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe80e600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8081613e0 con 0x7fe80815ec90 2024-05-22T21:13:40.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe80e600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe808162bc0 con 0x7fe80815d890 2024-05-22T21:13:40.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe80e600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80813fdc0 con 0x7fe808143ec0 2024-05-22T21:13:40.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe7ffe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80815ec90 0x7fe8081601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:40.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe7ffe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80815ec90 0x7fe8081601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54796/0 (socket says 172.21.15.161:54796) 2024-05-22T21:13:40.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe7ffe00640 1 -- 172.21.15.161:0/3125565093 learned_addr learned my addr 172.21.15.161:0/3125565093 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:40.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe7ffe00640 1 -- 172.21.15.161:0/3125565093 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe808143ec0 msgr2=0x7fe80813f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe7ffe00640 1 --2- 172.21.15.161:0/3125565093 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe808143ec0 0x7fe80813f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe7ffe00640 1 -- 172.21.15.161:0/3125565093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe80815d890 msgr2=0x7fe80813fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe7ffe00640 1 --2- 172.21.15.161:0/3125565093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe80815d890 0x7fe80813fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe7ffe00640 1 -- 172.21.15.161:0/3125565093 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe800009b30 con 0x7fe80815ec90 2024-05-22T21:13:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe7ffe00640 1 --2- 172.21.15.161:0/3125565093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80815ec90 0x7fe8081601f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000e960 tx=0x7fe7f000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe7ff400640 1 -- 172.21.15.161:0/3125565093 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0009690 con 0x7fe80815ec90 2024-05-22T21:13:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe7ff400640 1 -- 172.21.15.161:0/3125565093 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f000c9a0 con 0x7fe80815ec90 2024-05-22T21:13:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe7ff400640 1 -- 172.21.15.161:0/3125565093 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f000cd10 con 0x7fe80815ec90 2024-05-22T21:13:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe80e600640 1 -- 172.21.15.161:0/3125565093 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe808160b40 con 0x7fe80815ec90 2024-05-22T21:13:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe80e600640 1 -- 172.21.15.161:0/3125565093 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe808161100 con 0x7fe80815ec90 2024-05-22T21:13:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe80e600640 1 -- 172.21.15.161:0/3125565093 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe808067220 con 0x7fe80815ec90 2024-05-22T21:13:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.406+0000 7fe7ff400640 1 -- 172.21.15.161:0/3125565093 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe7f0013470 con 0x7fe80815ec90 2024-05-22T21:13:40.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.410+0000 7fe7ff400640 1 --2- 172.21.15.161:0/3125565093 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe7e40666c0 0x7fe7e4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.410+0000 7fe7ff400640 1 -- 172.21.15.161:0/3125565093 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7f0084380 con 0x7fe80815ec90 2024-05-22T21:13:40.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.410+0000 7fe807400640 1 --2- 172.21.15.161:0/3125565093 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe7e40666c0 0x7fe7e4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:40.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.410+0000 7fe807400640 1 --2- 172.21.15.161:0/3125565093 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe7e40666c0 0x7fe7e4068b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8007fd0 tx=0x7fe7f8007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:40.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.414+0000 7fe7ff400640 1 -- 172.21.15.161:0/3125565093 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe7f00516e0 con 0x7fe80815ec90 2024-05-22T21:13:40.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.606+0000 7fe80e600640 1 -- 172.21.15.161:0/3125565093 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe80806dcc0 con 0x7fe80815ec90 2024-05-22T21:13:40.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.606+0000 7fe7ff400640 1 -- 172.21.15.161:0/3125565093 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4740 (secure 0 0 0) 0x7fe7f0054d00 con 0x7fe80815ec90 2024-05-22T21:13:40.610 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:40.610 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3179612470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3179612470},{"type":"v1","addr":"172.21.15.179:6834","nonce":3179612470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2196055309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2196055309},{"type":"v1","addr":"172.21.15.161:6835","nonce":2196055309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1793681759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1793681759},{"type":"v1","addr":"172.21.15.161:6837","nonce":1793681759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:13:23.418412+0000","modified":"2024-05-22T21:13:26.442354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/3095599688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3095599688},{"type":"v1","addr":"172.21.15.179:6835","nonce":3095599688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-22T21:13:40.610 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 8 2024-05-22T21:13:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.606+0000 7fe80e600640 1 -- 172.21.15.161:0/3125565093 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe7e40666c0 msgr2=0x7fe7e4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.606+0000 7fe80e600640 1 --2- 172.21.15.161:0/3125565093 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe7e40666c0 0x7fe7e4068b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8007fd0 tx=0x7fe7f8007920 comp rx=0 tx=0).stop 2024-05-22T21:13:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.606+0000 7fe80e600640 1 -- 172.21.15.161:0/3125565093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80815ec90 msgr2=0x7fe8081601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.606+0000 7fe80e600640 1 --2- 172.21.15.161:0/3125565093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80815ec90 0x7fe8081601f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000e960 tx=0x7fe7f000ee20 comp rx=0 tx=0).stop 2024-05-22T21:13:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.606+0000 7fe80e600640 1 -- 172.21.15.161:0/3125565093 shutdown_connections 2024-05-22T21:13:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.606+0000 7fe80e600640 1 --2- 172.21.15.161:0/3125565093 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe7e40666c0 0x7fe7e4068b80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.606+0000 7fe80e600640 1 --2- 172.21.15.161:0/3125565093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe80815d890 0x7fe80813fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.606+0000 7fe80e600640 1 --2- 172.21.15.161:0/3125565093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80815ec90 0x7fe8081601f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.606+0000 7fe80e600640 1 --2- 172.21.15.161:0/3125565093 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe808143ec0 0x7fe80813f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.606+0000 7fe80e600640 1 -- 172.21.15.161:0/3125565093 >> 172.21.15.161:0/3125565093 conn(0x7fe8080b0bc0 msgr2=0x7fe8080adbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:40.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.606+0000 7fe80e600640 1 -- 172.21.15.161:0/3125565093 shutdown_connections 2024-05-22T21:13:40.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.606+0000 7fe80e600640 1 -- 172.21.15.161:0/3125565093 wait complete. 2024-05-22T21:13:40.631 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:40.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.762+0000 7f6e45800640 1 Processor -- start 2024-05-22T21:13:40.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.762+0000 7f6e45800640 1 -- start start 2024-05-22T21:13:40.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.762+0000 7f6e45800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e400b0490 0x7f6e400cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.762+0000 7f6e45800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6e4006bfb0 0x7f6e400cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.762+0000 7f6e45800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4006c440 con 0x7f6e4015d4a0 2024-05-22T21:13:40.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.762+0000 7f6e45800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4015d880 con 0x7f6e4006bfb0 2024-05-22T21:13:40.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.762+0000 7f6e45800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4006b440 con 0x7f6e400b0490 2024-05-22T21:13:40.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.762+0000 7f6e3f400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6e4006bfb0 0x7f6e400cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:40.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.762+0000 7f6e3e000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e400b0490 0x7f6e400cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:40.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.762+0000 7f6e3e000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e400b0490 0x7f6e400cf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:52430/0 (socket says 172.21.15.161:52430) 2024-05-22T21:13:40.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.762+0000 7f6e3e000640 1 -- 172.21.15.161:0/4098821136 learned_addr learned my addr 172.21.15.161:0/4098821136 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:40.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.762+0000 7f6e3e000640 1 -- 172.21.15.161:0/4098821136 >> v2:172.21.15.179:3300/0 conn(0x7f6e4006bfb0 msgr2=0x7f6e400cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:40.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.762+0000 7f6e3e000640 1 --2- 172.21.15.161:0/4098821136 >> v2:172.21.15.179:3300/0 conn(0x7f6e4006bfb0 0x7f6e400cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.762+0000 7f6e3e000640 1 -- 172.21.15.161:0/4098821136 >> v1:172.21.15.179:6789/0 conn(0x7f6e4015d4a0 legacy=0x7f6e40099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.762+0000 7f6e3e000640 1 -- 172.21.15.161:0/4098821136 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e400cf190 con 0x7f6e400b0490 2024-05-22T21:13:40.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.762+0000 7f6e3f400640 1 --2- 172.21.15.161:0/4098821136 >> v2:172.21.15.179:3300/0 conn(0x7f6e4006bfb0 0x7f6e400cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:40.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.766+0000 7f6e3e000640 1 --2- 172.21.15.161:0/4098821136 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e400b0490 0x7f6e400cf670 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009870 tx=0x7f6e2801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97e651a6c45e76a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:40.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.766+0000 7f6e3d600640 1 -- 172.21.15.161:0/4098821136 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e280a95f0 con 0x7f6e400b0490 2024-05-22T21:13:40.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.766+0000 7f6e3d600640 1 -- 172.21.15.161:0/4098821136 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e280a9750 con 0x7f6e400b0490 2024-05-22T21:13:40.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.766+0000 7f6e3d600640 1 -- 172.21.15.161:0/4098821136 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e280a9ae0 con 0x7f6e400b0490 2024-05-22T21:13:40.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.766+0000 7f6e45800640 1 -- 172.21.15.161:0/4098821136 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e400b0490 msgr2=0x7f6e400cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.766+0000 7f6e45800640 1 --2- 172.21.15.161:0/4098821136 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e400b0490 0x7f6e400cf670 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009870 tx=0x7f6e2801c530 comp rx=0 tx=0).stop 2024-05-22T21:13:40.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.766+0000 7f6e45800640 1 -- 172.21.15.161:0/4098821136 shutdown_connections 2024-05-22T21:13:40.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.766+0000 7f6e45800640 1 --2- 172.21.15.161:0/4098821136 >> v2:172.21.15.179:3300/0 conn(0x7f6e4006bfb0 0x7f6e400cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.766+0000 7f6e45800640 1 --2- 172.21.15.161:0/4098821136 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e400b0490 0x7f6e400cf670 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.766+0000 7f6e45800640 1 -- 172.21.15.161:0/4098821136 >> 172.21.15.161:0/4098821136 conn(0x7f6e400b2a40 msgr2=0x7f6e400b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:40.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.766+0000 7f6e45800640 1 -- 172.21.15.161:0/4098821136 shutdown_connections 2024-05-22T21:13:40.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.766+0000 7f6e45800640 1 -- 172.21.15.161:0/4098821136 wait complete. 2024-05-22T21:13:40.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e45800640 1 Processor -- start 2024-05-22T21:13:40.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e45800640 1 -- start start 2024-05-22T21:13:40.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e45800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e400b0490 0x7f6e400b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e45800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e4015d4a0 0x7f6e4015b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e45800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e401567f0 0x7f6e40156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e45800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e401571c0 con 0x7f6e400b0490 2024-05-22T21:13:40.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e45800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e401452c0 con 0x7f6e4015d4a0 2024-05-22T21:13:40.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e45800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e401455a0 con 0x7f6e401567f0 2024-05-22T21:13:40.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e3ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e400b0490 0x7f6e400b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:40.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e3ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e400b0490 0x7f6e400b50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54800/0 (socket says 172.21.15.161:54800) 2024-05-22T21:13:40.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e3ea00640 1 -- 172.21.15.161:0/2286363171 learned_addr learned my addr 172.21.15.161:0/2286363171 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:40.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e3e000640 1 --2- 172.21.15.161:0/2286363171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e4015d4a0 0x7f6e4015b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:40.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e3f400640 1 --2- 172.21.15.161:0/2286363171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e401567f0 0x7f6e40156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:40.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e3ea00640 1 -- 172.21.15.161:0/2286363171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e401567f0 msgr2=0x7f6e40156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e3ea00640 1 --2- 172.21.15.161:0/2286363171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e401567f0 0x7f6e40156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e3ea00640 1 -- 172.21.15.161:0/2286363171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e4015d4a0 msgr2=0x7f6e4015b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e3ea00640 1 --2- 172.21.15.161:0/2286363171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e4015d4a0 0x7f6e4015b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e3ea00640 1 -- 172.21.15.161:0/2286363171 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e38009420 con 0x7f6e400b0490 2024-05-22T21:13:40.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e3ea00640 1 --2- 172.21.15.161:0/2286363171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e400b0490 0x7f6e400b50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000c7a0 tx=0x7f6e3000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:40.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e36a00640 1 -- 172.21.15.161:0/2286363171 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30003b80 con 0x7f6e400b0490 2024-05-22T21:13:40.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e36a00640 1 -- 172.21.15.161:0/2286363171 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e30003ce0 con 0x7f6e400b0490 2024-05-22T21:13:40.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e36a00640 1 -- 172.21.15.161:0/2286363171 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3000aa90 con 0x7f6e400b0490 2024-05-22T21:13:40.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e45800640 1 -- 172.21.15.161:0/2286363171 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e280094d0 con 0x7f6e400b0490 2024-05-22T21:13:40.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.770+0000 7f6e45800640 1 -- 172.21.15.161:0/2286363171 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e400cb7b0 con 0x7f6e400b0490 2024-05-22T21:13:40.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.774+0000 7f6e45800640 1 -- 172.21.15.161:0/2286363171 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4015f150 con 0x7f6e400b0490 2024-05-22T21:13:40.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.774+0000 7f6e36a00640 1 -- 172.21.15.161:0/2286363171 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6e3000abf0 con 0x7f6e400b0490 2024-05-22T21:13:40.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.774+0000 7f6e36a00640 1 --2- 172.21.15.161:0/2286363171 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e1c066690 0x7f6e1c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:40.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.774+0000 7f6e36a00640 1 -- 172.21.15.161:0/2286363171 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e30083cf0 con 0x7f6e400b0490 2024-05-22T21:13:40.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.774+0000 7f6e3e000640 1 --2- 172.21.15.161:0/2286363171 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e1c066690 0x7f6e1c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:40.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.774+0000 7f6e3e000640 1 --2- 172.21.15.161:0/2286363171 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e1c066690 0x7f6e1c068b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e280ae000 tx=0x7f6e280b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:40.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.778+0000 7f6e36a00640 1 -- 172.21.15.161:0/2286363171 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6e4015f150 con 0x7f6e400b0490 2024-05-22T21:13:40.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.930+0000 7f6e45800640 1 -- 172.21.15.161:0/2286363171 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e4006c440 con 0x7f6e400b0490 2024-05-22T21:13:40.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.934+0000 7f6e36a00640 1 -- 172.21.15.161:0/2286363171 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f6e4006c440 con 0x7f6e400b0490 2024-05-22T21:13:40.937 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:40.937 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:40.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.934+0000 7f6e45800640 1 -- 172.21.15.161:0/2286363171 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e1c066690 msgr2=0x7f6e1c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.934+0000 7f6e45800640 1 --2- 172.21.15.161:0/2286363171 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e1c066690 0x7f6e1c068b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e280ae000 tx=0x7f6e280b3040 comp rx=0 tx=0).stop 2024-05-22T21:13:40.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.934+0000 7f6e45800640 1 -- 172.21.15.161:0/2286363171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e400b0490 msgr2=0x7f6e400b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:40.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.934+0000 7f6e45800640 1 --2- 172.21.15.161:0/2286363171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e400b0490 0x7f6e400b50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000c7a0 tx=0x7f6e3000cc60 comp rx=0 tx=0).stop 2024-05-22T21:13:40.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.934+0000 7f6e45800640 1 -- 172.21.15.161:0/2286363171 shutdown_connections 2024-05-22T21:13:40.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.934+0000 7f6e45800640 1 --2- 172.21.15.161:0/2286363171 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e1c066690 0x7f6e1c068b50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.934+0000 7f6e45800640 1 --2- 172.21.15.161:0/2286363171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e401567f0 0x7f6e40156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.934+0000 7f6e45800640 1 --2- 172.21.15.161:0/2286363171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e4015d4a0 0x7f6e4015b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.934+0000 7f6e45800640 1 --2- 172.21.15.161:0/2286363171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e400b0490 0x7f6e400b50c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:40.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.934+0000 7f6e45800640 1 -- 172.21.15.161:0/2286363171 >> 172.21.15.161:0/2286363171 conn(0x7f6e400b2a40 msgr2=0x7f6e4015c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:40.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.934+0000 7f6e45800640 1 -- 172.21.15.161:0/2286363171 shutdown_connections 2024-05-22T21:13:40.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:40.934+0000 7f6e45800640 1 -- 172.21.15.161:0/2286363171 wait complete. 2024-05-22T21:13:40.958 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:13:41.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.090+0000 7f97d4a00640 1 Processor -- start 2024-05-22T21:13:41.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.090+0000 7f97d4a00640 1 -- start start 2024-05-22T21:13:41.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.090+0000 7f97d4a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f97d00715a0 0x7f97d015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:41.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.090+0000 7f97d4a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d01442a0 con 0x7f97d0143ec0 2024-05-22T21:13:41.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.090+0000 7f97d4a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d0070500 con 0x7f97d015ec90 2024-05-22T21:13:41.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.090+0000 7f97d4a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d00707e0 con 0x7f97d00715a0 2024-05-22T21:13:41.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.090+0000 7f97cde00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f97d0143ec0 0x7f97d006fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43540/0 (socket says 172.21.15.161:43540) 2024-05-22T21:13:41.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.090+0000 7f97c5400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f97d00715a0 0x7f97d015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:41.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.090+0000 7f97cde00640 1 -- 172.21.15.161:0/699152860 learned_addr learned my addr 172.21.15.161:0/699152860 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:41.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.090+0000 7f97cd400640 1 -- 172.21.15.161:0/699152860 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3448390525 0 0) 0x7f97d01442a0 con 0x7f97d0143ec0 2024-05-22T21:13:41.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.090+0000 7f97cd400640 1 -- 172.21.15.161:0/699152860 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97b4003530 con 0x7f97d0143ec0 2024-05-22T21:13:41.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.090+0000 7f97c5400640 1 -- 172.21.15.161:0/699152860 >> v1:172.21.15.179:6789/0 conn(0x7f97d015ec90 legacy=0x7f97d00cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:41.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.090+0000 7f97c5400640 1 -- 172.21.15.161:0/699152860 >> v1:172.21.15.161:6789/0 conn(0x7f97d0143ec0 legacy=0x7f97d006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:41.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.090+0000 7f97c5400640 1 -- 172.21.15.161:0/699152860 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d0070ac0 con 0x7f97d00715a0 2024-05-22T21:13:41.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.094+0000 7f97c5400640 1 --2- 172.21.15.161:0/699152860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d00715a0 0x7f97d015f830 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f97c00097c0 tx=0x7f97c001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712f8c320c7dd8cc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:41.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.094+0000 7f97cd400640 1 -- 172.21.15.161:0/699152860 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c00ab670 con 0x7f97d00715a0 2024-05-22T21:13:41.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.094+0000 7f97cd400640 1 -- 172.21.15.161:0/699152860 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c00ab7d0 con 0x7f97d00715a0 2024-05-22T21:13:41.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.094+0000 7f97cd400640 1 -- 172.21.15.161:0/699152860 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c00abb60 con 0x7f97d00715a0 2024-05-22T21:13:41.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.094+0000 7f97d4a00640 1 -- 172.21.15.161:0/699152860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d00715a0 msgr2=0x7f97d015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:41.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.094+0000 7f97d4a00640 1 --2- 172.21.15.161:0/699152860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d00715a0 0x7f97d015f830 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f97c00097c0 tx=0x7f97c001c470 comp rx=0 tx=0).stop 2024-05-22T21:13:41.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.094+0000 7f97d4a00640 1 -- 172.21.15.161:0/699152860 shutdown_connections 2024-05-22T21:13:41.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.094+0000 7f97d4a00640 1 --2- 172.21.15.161:0/699152860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d00715a0 0x7f97d015f830 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:41.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.094+0000 7f97d4a00640 1 -- 172.21.15.161:0/699152860 >> 172.21.15.161:0/699152860 conn(0x7f97d00b0bc0 msgr2=0x7f97d00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:41.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.094+0000 7f97d4a00640 1 -- 172.21.15.161:0/699152860 shutdown_connections 2024-05-22T21:13:41.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.094+0000 7f97d4a00640 1 -- 172.21.15.161:0/699152860 wait complete. 2024-05-22T21:13:41.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97d4a00640 1 Processor -- start 2024-05-22T21:13:41.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97d4a00640 1 -- start start 2024-05-22T21:13:41.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97d4a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d0143ec0 0x7f97d0157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:41.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97d4a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97d015ec90 0x7f97d013e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:41.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97d4a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97d015d740 0x7f97d013e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:41.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97d4a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d015dee0 con 0x7f97d0143ec0 2024-05-22T21:13:41.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97d4a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d013f7b0 con 0x7f97d015ec90 2024-05-22T21:13:41.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97d4a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d015dc00 con 0x7f97d015d740 2024-05-22T21:13:41.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97cde00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d0143ec0 0x7f97d0157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:41.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97cde00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d0143ec0 0x7f97d0157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54818/0 (socket says 172.21.15.161:54818) 2024-05-22T21:13:41.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97cde00640 1 -- 172.21.15.161:0/3719915021 learned_addr learned my addr 172.21.15.161:0/3719915021 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:41.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97c5400640 1 --2- 172.21.15.161:0/3719915021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97d015ec90 0x7f97d013e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:41.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97ce800640 1 --2- 172.21.15.161:0/3719915021 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97d015d740 0x7f97d013e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:41.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97cde00640 1 -- 172.21.15.161:0/3719915021 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97d015d740 msgr2=0x7f97d013e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:41.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97cde00640 1 --2- 172.21.15.161:0/3719915021 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97d015d740 0x7f97d013e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:41.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97cde00640 1 -- 172.21.15.161:0/3719915021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97d015ec90 msgr2=0x7f97d013e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:41.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97cde00640 1 --2- 172.21.15.161:0/3719915021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97d015ec90 0x7f97d013e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:41.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97cde00640 1 -- 172.21.15.161:0/3719915021 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c0009420 con 0x7f97d0143ec0 2024-05-22T21:13:41.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97ce800640 1 --2- 172.21.15.161:0/3719915021 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97d015d740 0x7f97d013e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:41.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97cde00640 1 --2- 172.21.15.161:0/3719915021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d0143ec0 0x7f97d0157170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f97b800c4d0 tx=0x7f97b800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:41.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97c6a00640 1 -- 172.21.15.161:0/3719915021 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8005cf0 con 0x7f97d0143ec0 2024-05-22T21:13:41.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97c6a00640 1 -- 172.21.15.161:0/3719915021 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b8005e50 con 0x7f97d0143ec0 2024-05-22T21:13:41.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97d4a00640 1 -- 172.21.15.161:0/3719915021 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d013f160 con 0x7f97d0143ec0 2024-05-22T21:13:41.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97c6a00640 1 -- 172.21.15.161:0/3719915021 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8007270 con 0x7f97d0143ec0 2024-05-22T21:13:41.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.098+0000 7f97d4a00640 1 -- 172.21.15.161:0/3719915021 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d01576b0 con 0x7f97d0143ec0 2024-05-22T21:13:41.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.102+0000 7f97d4a00640 1 -- 172.21.15.161:0/3719915021 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d013efc0 con 0x7f97d0143ec0 2024-05-22T21:13:41.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.102+0000 7f97c6a00640 1 -- 172.21.15.161:0/3719915021 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f97b80073d0 con 0x7f97d0143ec0 2024-05-22T21:13:41.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.102+0000 7f97c6a00640 1 --2- 172.21.15.161:0/3719915021 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97ac066690 0x7f97ac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:41.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.102+0000 7f97c6a00640 1 -- 172.21.15.161:0/3719915021 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97b80837a0 con 0x7f97d0143ec0 2024-05-22T21:13:41.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.102+0000 7f97c5400640 1 --2- 172.21.15.161:0/3719915021 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97ac066690 0x7f97ac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:41.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.102+0000 7f97c5400640 1 --2- 172.21.15.161:0/3719915021 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97ac066690 0x7f97ac068b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f97c00af000 tx=0x7f97c00a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:41.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.106+0000 7f97c6a00640 1 -- 172.21.15.161:0/3719915021 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f97d013efc0 con 0x7f97d0143ec0 2024-05-22T21:13:41.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.290+0000 7f97d4a00640 1 -- 172.21.15.161:0/3719915021 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97d013efc0 con 0x7f97d0143ec0 2024-05-22T21:13:41.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.290+0000 7f97c6a00640 1 -- 172.21.15.161:0/3719915021 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4740 (secure 0 0 0) 0x7f97d013efc0 con 0x7f97d0143ec0 2024-05-22T21:13:41.294 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:41.294 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3179612470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3179612470},{"type":"v1","addr":"172.21.15.179:6834","nonce":3179612470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2196055309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2196055309},{"type":"v1","addr":"172.21.15.161:6835","nonce":2196055309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1793681759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1793681759},{"type":"v1","addr":"172.21.15.161:6837","nonce":1793681759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:13:23.418412+0000","modified":"2024-05-22T21:13:26.442354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/3095599688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3095599688},{"type":"v1","addr":"172.21.15.179:6835","nonce":3095599688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-22T21:13:41.294 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 8 2024-05-22T21:13:41.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.294+0000 7f97d4a00640 1 -- 172.21.15.161:0/3719915021 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97ac066690 msgr2=0x7f97ac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:41.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.294+0000 7f97d4a00640 1 --2- 172.21.15.161:0/3719915021 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97ac066690 0x7f97ac068b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f97c00af000 tx=0x7f97c00a7000 comp rx=0 tx=0).stop 2024-05-22T21:13:41.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.294+0000 7f97d4a00640 1 -- 172.21.15.161:0/3719915021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d0143ec0 msgr2=0x7f97d0157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:41.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.294+0000 7f97d4a00640 1 --2- 172.21.15.161:0/3719915021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d0143ec0 0x7f97d0157170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f97b800c4d0 tx=0x7f97b800c990 comp rx=0 tx=0).stop 2024-05-22T21:13:41.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.294+0000 7f97d4a00640 1 -- 172.21.15.161:0/3719915021 shutdown_connections 2024-05-22T21:13:41.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.294+0000 7f97d4a00640 1 --2- 172.21.15.161:0/3719915021 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97ac066690 0x7f97ac068b50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:41.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.294+0000 7f97d4a00640 1 --2- 172.21.15.161:0/3719915021 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97d015d740 0x7f97d013e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:41.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.294+0000 7f97d4a00640 1 --2- 172.21.15.161:0/3719915021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97d015ec90 0x7f97d013e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:41.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.294+0000 7f97d4a00640 1 --2- 172.21.15.161:0/3719915021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d0143ec0 0x7f97d0157170 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:41.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.294+0000 7f97d4a00640 1 -- 172.21.15.161:0/3719915021 >> 172.21.15.161:0/3719915021 conn(0x7f97d00b0bc0 msgr2=0x7f97d0141dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:41.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.294+0000 7f97d4a00640 1 -- 172.21.15.161:0/3719915021 shutdown_connections 2024-05-22T21:13:41.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:41.294+0000 7f97d4a00640 1 -- 172.21.15.161:0/3719915021 wait complete. 2024-05-22T21:13:41.718 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-22T21:13:23.418412+0000', 'modified': '2024-05-22T21:13:26.442354+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.179:6835/3095599688', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6833', 'nonce': 3095599688}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 3095599688}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-22T21:13:41.921 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-22T21:13:41.921 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-22T21:13:42.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.058+0000 7f85c5c00640 1 Processor -- start 2024-05-22T21:13:42.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.058+0000 7f85c5c00640 1 -- start start 2024-05-22T21:13:42.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.058+0000 7f85c5c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85c013cfc0 0x7f85c01402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.058+0000 7f85c5c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f85c015af60 0x7f85c013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.058+0000 7f85c5c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c0140090 con 0x7f85c013fc80 2024-05-22T21:13:42.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.058+0000 7f85c5c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c013d3a0 con 0x7f85c015af60 2024-05-22T21:13:42.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.058+0000 7f85c5c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c00cd920 con 0x7f85c013cfc0 2024-05-22T21:13:42.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.058+0000 7f85be000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f85c015af60 0x7f85c013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.058+0000 7f85be000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f85c015af60 0x7f85c013f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54820/0 (socket says 172.21.15.161:54820) 2024-05-22T21:13:42.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.058+0000 7f85be000640 1 -- 172.21.15.161:0/869215624 learned_addr learned my addr 172.21.15.161:0/869215624 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:42.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.062+0000 7f85bea00640 1 --2- 172.21.15.161:0/869215624 >> v2:172.21.15.179:3300/0 conn(0x7f85c013cfc0 0x7f85c01402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.062+0000 7f85be000640 1 -- 172.21.15.161:0/869215624 >> v2:172.21.15.179:3300/0 conn(0x7f85c013cfc0 msgr2=0x7f85c01402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:42.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.062+0000 7f85be000640 1 --2- 172.21.15.161:0/869215624 >> v2:172.21.15.179:3300/0 conn(0x7f85c013cfc0 0x7f85c01402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.062+0000 7f85be000640 1 -- 172.21.15.161:0/869215624 >> v1:172.21.15.161:6789/0 conn(0x7f85c013fc80 legacy=0x7f85c015dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:42.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.062+0000 7f85be000640 1 -- 172.21.15.161:0/869215624 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c00cdc00 con 0x7f85c015af60 2024-05-22T21:13:42.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.062+0000 7f85bd600640 1 -- 172.21.15.161:0/869215624 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053020584 0 0) 0x7f85c0140090 con 0x7f85c013fc80 2024-05-22T21:13:42.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.062+0000 7f85be000640 1 --2- 172.21.15.161:0/869215624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c015af60 0x7f85c013f740 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f85c006de90 tx=0x7f85ac01eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fac25061eec7b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:42.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.062+0000 7f85bd600640 1 -- 172.21.15.161:0/869215624 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0a7a40 con 0x7f85c015af60 2024-05-22T21:13:42.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.062+0000 7f85bd600640 1 -- 172.21.15.161:0/869215624 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac0a7ba0 con 0x7f85c015af60 2024-05-22T21:13:42.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.062+0000 7f85bd600640 1 -- 172.21.15.161:0/869215624 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0af440 con 0x7f85c015af60 2024-05-22T21:13:42.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.062+0000 7f85c5c00640 1 -- 172.21.15.161:0/869215624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c015af60 msgr2=0x7f85c013f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:42.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.062+0000 7f85c5c00640 1 --2- 172.21.15.161:0/869215624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c015af60 0x7f85c013f740 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f85c006de90 tx=0x7f85ac01eb60 comp rx=0 tx=0).stop 2024-05-22T21:13:42.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.062+0000 7f85c5c00640 1 -- 172.21.15.161:0/869215624 shutdown_connections 2024-05-22T21:13:42.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.062+0000 7f85c5c00640 1 --2- 172.21.15.161:0/869215624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c015af60 0x7f85c013f740 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.062+0000 7f85c5c00640 1 --2- 172.21.15.161:0/869215624 >> v2:172.21.15.179:3300/0 conn(0x7f85c013cfc0 0x7f85c01402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.062+0000 7f85c5c00640 1 -- 172.21.15.161:0/869215624 >> 172.21.15.161:0/869215624 conn(0x7f85c006a2d0 msgr2=0x7f85c006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:42.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85c5c00640 1 -- 172.21.15.161:0/869215624 shutdown_connections 2024-05-22T21:13:42.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85c5c00640 1 -- 172.21.15.161:0/869215624 wait complete. 2024-05-22T21:13:42.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85c5c00640 1 Processor -- start 2024-05-22T21:13:42.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85c5c00640 1 -- start start 2024-05-22T21:13:42.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85c5c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c013cfc0 0x7f85c0142320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85c5c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c013fc80 0x7f85c00acff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85c5c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85c015af60 0x7f85c0156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85c5c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c01429b0 con 0x7f85c013fc80 2024-05-22T21:13:42.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85c5c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c013d990 con 0x7f85c015af60 2024-05-22T21:13:42.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85c5c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c013dc30 con 0x7f85c013cfc0 2024-05-22T21:13:42.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85be000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c013fc80 0x7f85c00acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85be000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c013fc80 0x7f85c00acff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54830/0 (socket says 172.21.15.161:54830) 2024-05-22T21:13:42.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85be000640 1 -- 172.21.15.161:0/2305159708 learned_addr learned my addr 172.21.15.161:0/2305159708 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:42.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85be000640 1 -- 172.21.15.161:0/2305159708 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c013cfc0 msgr2=0x7f85c0142320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:42.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85be000640 1 --2- 172.21.15.161:0/2305159708 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c013cfc0 0x7f85c0142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85be000640 1 -- 172.21.15.161:0/2305159708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85c015af60 msgr2=0x7f85c0156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:42.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85be000640 1 --2- 172.21.15.161:0/2305159708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85c015af60 0x7f85c0156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85be000640 1 -- 172.21.15.161:0/2305159708 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ac00b770 con 0x7f85c013fc80 2024-05-22T21:13:42.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85be000640 1 --2- 172.21.15.161:0/2305159708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c013fc80 0x7f85c00acff0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f85ac01b770 tx=0x7f85ac01b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:42.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85b6a00640 1 -- 172.21.15.161:0/2305159708 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0ba660 con 0x7f85c013fc80 2024-05-22T21:13:42.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85b6a00640 1 -- 172.21.15.161:0/2305159708 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac0ba7c0 con 0x7f85c013fc80 2024-05-22T21:13:42.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85b6a00640 1 -- 172.21.15.161:0/2305159708 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0bab70 con 0x7f85c013fc80 2024-05-22T21:13:42.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85c5c00640 1 -- 172.21.15.161:0/2305159708 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c0157a90 con 0x7f85c013fc80 2024-05-22T21:13:42.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.066+0000 7f85c5c00640 1 -- 172.21.15.161:0/2305159708 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c013e1b0 con 0x7f85c013fc80 2024-05-22T21:13:42.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.070+0000 7f85b6a00640 1 -- 172.21.15.161:0/2305159708 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f85ac0c2360 con 0x7f85c013fc80 2024-05-22T21:13:42.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.070+0000 7f85b6a00640 1 --2- 172.21.15.161:0/2305159708 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85a0066690 0x7f85a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.070+0000 7f85b6a00640 1 -- 172.21.15.161:0/2305159708 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85ac11f070 con 0x7f85c013fc80 2024-05-22T21:13:42.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.070+0000 7f85c90ab000 1 Processor -- start 2024-05-22T21:13:42.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.070+0000 7f85bea00640 1 --2- 172.21.15.161:0/2305159708 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85a0066690 0x7f85a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.070+0000 7f85c90ab000 1 -- start start 2024-05-22T21:13:42.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85bea00640 1 --2- 172.21.15.161:0/2305159708 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85a0066690 0x7f85a0068b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f85b0006aa0 tx=0x7f85b0006a30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:42.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7ba5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7ba62e0 0x55d5e7b5b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7b5bec0 0x55d5e7b5c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e7a23e80 con 0x55d5e7ba59c0 2024-05-22T21:13:42.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e79b1af0 con 0x55d5e7ba62e0 2024-05-22T21:13:42.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5e79af0c0 con 0x55d5e7b5bec0 2024-05-22T21:13:42.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85bea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7ba5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85bea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7ba5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54836/0 (socket says 172.21.15.161:54836) 2024-05-22T21:13:42.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85bea00640 1 -- 172.21.15.161:0/304619542 learned_addr learned my addr 172.21.15.161:0/304619542 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:42.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85be000640 1 --2- 172.21.15.161:0/304619542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7ba62e0 0x55d5e7b5b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85bea00640 1 -- 172.21.15.161:0/304619542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7b5bec0 msgr2=0x55d5e7b5c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:42.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85bf400640 1 --2- 172.21.15.161:0/304619542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7b5bec0 0x55d5e7b5c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85bea00640 1 --2- 172.21.15.161:0/304619542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7b5bec0 0x55d5e7b5c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85bea00640 1 -- 172.21.15.161:0/304619542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7ba62e0 msgr2=0x55d5e7b5b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:42.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85bea00640 1 --2- 172.21.15.161:0/304619542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7ba62e0 0x55d5e7b5b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85bea00640 1 -- 172.21.15.161:0/304619542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5e76a6890 con 0x55d5e7ba59c0 2024-05-22T21:13:42.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85bf400640 1 --2- 172.21.15.161:0/304619542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7b5bec0 0x55d5e7b5c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:42.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85be000640 1 --2- 172.21.15.161:0/304619542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7ba62e0 0x55d5e7b5b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:42.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85bea00640 1 --2- 172.21.15.161:0/304619542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7ba5da0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f85b000eb50 tx=0x7f85b0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1cd090d5104371 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:42.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85ab400640 1 -- 172.21.15.161:0/304619542 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b0015a90 con 0x55d5e7ba59c0 2024-05-22T21:13:42.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85ab400640 1 -- 172.21.15.161:0/304619542 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b0015bf0 con 0x55d5e7ba59c0 2024-05-22T21:13:42.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85ab400640 1 -- 172.21.15.161:0/304619542 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b0013450 con 0x55d5e7ba59c0 2024-05-22T21:13:42.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 -- 172.21.15.161:0/304619542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 msgr2=0x55d5e7ba5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:42.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 --2- 172.21.15.161:0/304619542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7ba5da0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f85b000eb50 tx=0x7f85b0011000 comp rx=0 tx=0).stop 2024-05-22T21:13:42.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 -- 172.21.15.161:0/304619542 shutdown_connections 2024-05-22T21:13:42.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 --2- 172.21.15.161:0/304619542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7b5bec0 0x55d5e7b5c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 --2- 172.21.15.161:0/304619542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7ba62e0 0x55d5e7b5b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 --2- 172.21.15.161:0/304619542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7ba5da0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 -- 172.21.15.161:0/304619542 >> 172.21.15.161:0/304619542 conn(0x55d5e7b62440 msgr2=0x55d5e7b5a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:42.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 -- 172.21.15.161:0/304619542 shutdown_connections 2024-05-22T21:13:42.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 -- 172.21.15.161:0/304619542 wait complete. 2024-05-22T21:13:42.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 -- start start 2024-05-22T21:13:42.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.074+0000 7f85c90ab000 1 Processor -- start 2024-05-22T21:13:42.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85c90ab000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7b5bec0 0x55d5e7b60e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85c90ab000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7b66fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85c90ab000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7ba62e0 0x55d5e7b674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85c90ab000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e79aa9b0 con 0x55d5e7ba59c0 2024-05-22T21:13:42.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85c90ab000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e7949c00 con 0x55d5e7ba62e0 2024-05-22T21:13:42.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85c90ab000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5e7968070 con 0x55d5e7b5bec0 2024-05-22T21:13:42.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85c90ab000 10 client.0 _ll_register_callbacks cb 0x55d5e7b47ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:13:42.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85be000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7b66fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85be000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7b66fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54842/0 (socket says 172.21.15.161:54842) 2024-05-22T21:13:42.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85be000640 1 -- 172.21.15.161:0/2962164990 learned_addr learned my addr 172.21.15.161:0/2962164990 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:42.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85bf400640 1 --2- 172.21.15.161:0/2962164990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7ba62e0 0x55d5e7b674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85bea00640 1 --2- 172.21.15.161:0/2962164990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7b5bec0 0x55d5e7b60e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85be000640 1 -- 172.21.15.161:0/2962164990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7b5bec0 msgr2=0x55d5e7b60e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:42.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85be000640 1 --2- 172.21.15.161:0/2962164990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7b5bec0 0x55d5e7b60e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85be000640 1 -- 172.21.15.161:0/2962164990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7ba62e0 msgr2=0x55d5e7b674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:42.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85be000640 1 --2- 172.21.15.161:0/2962164990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7ba62e0 0x55d5e7b674f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85be000640 1 -- 172.21.15.161:0/2962164990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b000e830 con 0x55d5e7ba59c0 2024-05-22T21:13:42.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85bea00640 1 --2- 172.21.15.161:0/2962164990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7b5bec0 0x55d5e7b60e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:42.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85be000640 1 --2- 172.21.15.161:0/2962164990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7b66fb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f85ac025070 tx=0x7f85ac0efac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd64fbd16c21805 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:42.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f859fe00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:13:42.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f859fe00640 1 -- 172.21.15.161:0/2962164990 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac107070 con 0x55d5e7ba59c0 2024-05-22T21:13:42.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f859fe00640 1 -- 172.21.15.161:0/2962164990 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85ac0f3ed0 con 0x55d5e7ba59c0 2024-05-22T21:13:42.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85c90ab000 1 -- 172.21.15.161:0/2962164990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5e79675f0 con 0x55d5e7ba59c0 2024-05-22T21:13:42.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f859fe00640 1 -- 172.21.15.161:0/2962164990 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85ac0f8490 con 0x55d5e7ba59c0 2024-05-22T21:13:42.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85c90ab000 10 client.4647 fetch_fsmap learned FSMap version 8 2024-05-22T21:13:42.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85c90ab000 1 -- 172.21.15.161:0/2962164990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55d5e796fab0 con 0x55d5e7ba59c0 2024-05-22T21:13:42.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f859fe00640 1 -- 172.21.15.161:0/2962164990 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d5e79675f0 con 0x55d5e7ba59c0 2024-05-22T21:13:42.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f859fe00640 1 -- 172.21.15.161:0/2962164990 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f85ac11d170 con 0x55d5e7ba59c0 2024-05-22T21:13:42.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85c90ab000 10 client.4647 fetch_fsmap finished waiting for FSMap version 8 2024-05-22T21:13:42.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85c90ab000 10 client.4647 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-22T21:13:42.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85c90ab000 20 client.4647 populate_metadata read hostname 'smithi161' 2024-05-22T21:13:42.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85c90ab000 1 --2- 172.21.15.161:0/2962164990 >> [v2:172.21.15.179:6833/3095599688,v1:172.21.15.179:6835/3095599688] conn(0x55d5e7bb5130 0x55d5e7bb7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85c90ab000 4 client.4647 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T21:13:42.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85c90ab000 1 -- 172.21.15.161:0/2962164990 --> [v2:172.21.15.179:6833/3095599688,v1:172.21.15.179:6835/3095599688] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5e79a7790 con 0x55d5e7bb5130 2024-05-22T21:13:42.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85bea00640 1 --2- 172.21.15.161:0/2962164990 >> [v2:172.21.15.179:6833/3095599688,v1:172.21.15.179:6835/3095599688] conn(0x55d5e7bb5130 0x55d5e7bb7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f85bea00640 1 --2- 172.21.15.161:0/2962164990 >> [v2:172.21.15.179:6833/3095599688,v1:172.21.15.179:6835/3095599688] conn(0x55d5e7bb5130 0x55d5e7bb7520 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e06ace4d868f2b6 server_cookie=194924133c9ac3ba in_seq=0 out_seq=0 2024-05-22T21:13:42.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f859fe00640 10 client.4647 ms_handle_connect on v2:172.21.15.179:6833/3095599688 2024-05-22T21:13:42.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f859fe00640 1 -- 172.21.15.161:0/2962164990 <== mds.0 v2:172.21.15.179:6833/3095599688 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d5e79a7790 con 0x55d5e7bb5130 2024-05-22T21:13:42.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.078+0000 7f859fe00640 10 client.4647 handle_command_reply: tid=0 2024-05-22T21:13:42.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.082+0000 7f85c90ab000 1 client.4647 shutdown 2024-05-22T21:13:42.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.082+0000 7f85c90ab000 1 -- 172.21.15.161:0/2962164990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 msgr2=0x55d5e7b66fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:42.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.082+0000 7f85c90ab000 1 --2- 172.21.15.161:0/2962164990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7b66fb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f85ac025070 tx=0x7f85ac0efac0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.082+0000 7f85c90ab000 1 -- 172.21.15.161:0/2962164990 shutdown_connections 2024-05-22T21:13:42.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.082+0000 7f85c90ab000 1 --2- 172.21.15.161:0/2962164990 >> [v2:172.21.15.179:6833/3095599688,v1:172.21.15.179:6835/3095599688] conn(0x55d5e7bb5130 0x55d5e7bb7520 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.082+0000 7f85c90ab000 1 --2- 172.21.15.161:0/2962164990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7ba62e0 0x55d5e7b674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.082+0000 7f85c90ab000 1 --2- 172.21.15.161:0/2962164990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7b66fb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.082+0000 7f859fe00640 0 client.4647 ms_handle_reset on v2:172.21.15.179:6833/3095599688 2024-05-22T21:13:42.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.082+0000 7f85c90ab000 1 --2- 172.21.15.161:0/2962164990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7b5bec0 0x55d5e7b60e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.082+0000 7f85c90ab000 1 -- 172.21.15.161:0/2962164990 >> 172.21.15.161:0/2962164990 conn(0x55d5e7b62440 msgr2=0x55d5e7b5a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:42.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.082+0000 7f85c90ab000 1 -- 172.21.15.161:0/2962164990 shutdown_connections 2024-05-22T21:13:42.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.082+0000 7f85c90ab000 1 -- 172.21.15.161:0/2962164990 wait complete. 2024-05-22T21:13:42.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.082+0000 7f85c90ab000 20 client.4647 trim_cache size 0 max 16384 2024-05-22T21:13:42.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.094+0000 7f85c90ab000 1 Processor -- start 2024-05-22T21:13:42.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.094+0000 7f85c90ab000 1 -- start start 2024-05-22T21:13:42.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7b5bec0 0x55d5e7bb6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7ba59c0 0x55d5e7bb6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba62e0 0x55d5e7bb6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e7667940 con 0x55d5e7ba62e0 2024-05-22T21:13:42.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e79a9840 con 0x55d5e7ba59c0 2024-05-22T21:13:42.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5e79ad510 con 0x55d5e7b5bec0 2024-05-22T21:13:42.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85bf400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba62e0 0x55d5e7bb6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85bf400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba62e0 0x55d5e7bb6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54848/0 (socket says 172.21.15.161:54848) 2024-05-22T21:13:42.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85bf400640 1 -- 172.21.15.161:0/183229462 learned_addr learned my addr 172.21.15.161:0/183229462 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:42.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85bf400640 1 -- 172.21.15.161:0/183229462 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7b5bec0 msgr2=0x55d5e7bb6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:42.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85bf400640 1 --2- 172.21.15.161:0/183229462 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7b5bec0 0x55d5e7bb6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85bf400640 1 -- 172.21.15.161:0/183229462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7ba59c0 msgr2=0x55d5e7bb6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:42.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85be000640 1 --2- 172.21.15.161:0/183229462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7ba59c0 0x55d5e7bb6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85bf400640 1 --2- 172.21.15.161:0/183229462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7ba59c0 0x55d5e7bb6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85bf400640 1 -- 172.21.15.161:0/183229462 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5e7955c00 con 0x55d5e7ba62e0 2024-05-22T21:13:42.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85bf400640 1 --2- 172.21.15.161:0/183229462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba62e0 0x55d5e7bb6fa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f85b8001dd0 tx=0x7f85b8010610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ffbf11f4b388cb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:42.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85abe00640 1 -- 172.21.15.161:0/183229462 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8001950 con 0x55d5e7ba62e0 2024-05-22T21:13:42.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85abe00640 1 -- 172.21.15.161:0/183229462 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b800d050 con 0x55d5e7ba62e0 2024-05-22T21:13:42.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85abe00640 1 -- 172.21.15.161:0/183229462 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8003bc0 con 0x55d5e7ba62e0 2024-05-22T21:13:42.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 -- 172.21.15.161:0/183229462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba62e0 msgr2=0x55d5e7bb6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:42.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 --2- 172.21.15.161:0/183229462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba62e0 0x55d5e7bb6fa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f85b8001dd0 tx=0x7f85b8010610 comp rx=0 tx=0).stop 2024-05-22T21:13:42.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 -- 172.21.15.161:0/183229462 shutdown_connections 2024-05-22T21:13:42.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 --2- 172.21.15.161:0/183229462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba62e0 0x55d5e7bb6fa0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 --2- 172.21.15.161:0/183229462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7ba59c0 0x55d5e7bb6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 --2- 172.21.15.161:0/183229462 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7b5bec0 0x55d5e7bb6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 -- 172.21.15.161:0/183229462 >> 172.21.15.161:0/183229462 conn(0x55d5e7b62440 msgr2=0x55d5e7ba4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:42.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 -- 172.21.15.161:0/183229462 shutdown_connections 2024-05-22T21:13:42.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 -- 172.21.15.161:0/183229462 wait complete. 2024-05-22T21:13:42.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 -- start start 2024-05-22T21:13:42.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.098+0000 7f85c90ab000 1 Processor -- start 2024-05-22T21:13:42.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7b5bec0 0x55d5e7bb97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7bea620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7ba62e0 0x55d5e7beab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e7b67740 con 0x55d5e7ba59c0 2024-05-22T21:13:42.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e7bbc210 con 0x55d5e7b5bec0 2024-05-22T21:13:42.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5e79ae9b0 con 0x55d5e7ba62e0 2024-05-22T21:13:42.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 10 client.0 _ll_register_callbacks cb 0x7f85b0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:13:42.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85be000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7bea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85be000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7bea620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54862/0 (socket says 172.21.15.161:54862) 2024-05-22T21:13:42.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85be000640 1 -- 172.21.15.161:0/4111416031 learned_addr learned my addr 172.21.15.161:0/4111416031 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:42.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85bea00640 1 --2- 172.21.15.161:0/4111416031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7b5bec0 0x55d5e7bb97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85bf400640 1 --2- 172.21.15.161:0/4111416031 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7ba62e0 0x55d5e7beab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85be000640 1 -- 172.21.15.161:0/4111416031 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7ba62e0 msgr2=0x55d5e7beab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:42.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85be000640 1 --2- 172.21.15.161:0/4111416031 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7ba62e0 0x55d5e7beab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85be000640 1 -- 172.21.15.161:0/4111416031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7b5bec0 msgr2=0x55d5e7bb97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85be000640 1 --2- 172.21.15.161:0/4111416031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7b5bec0 0x55d5e7bb97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85be000640 1 -- 172.21.15.161:0/4111416031 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b8001ab0 con 0x55d5e7ba59c0 2024-05-22T21:13:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85be000640 1 --2- 172.21.15.161:0/4111416031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7bea620 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f85ac000c10 tx=0x7f85ac11d470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926be45089079281 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f859fe00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:13:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f859fe00640 1 -- 172.21.15.161:0/4111416031 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac107040 con 0x55d5e7ba59c0 2024-05-22T21:13:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f859fe00640 1 -- 172.21.15.161:0/4111416031 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85ac0f3ed0 con 0x55d5e7ba59c0 2024-05-22T21:13:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 -- 172.21.15.161:0/4111416031 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5e7b60360 con 0x55d5e7ba59c0 2024-05-22T21:13:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f859fe00640 1 -- 172.21.15.161:0/4111416031 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85ac0f60b0 con 0x55d5e7ba59c0 2024-05-22T21:13:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 10 client.4653 fetch_fsmap learned FSMap version 8 2024-05-22T21:13:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 -- 172.21.15.161:0/4111416031 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55d5e76a6cc0 con 0x55d5e7ba59c0 2024-05-22T21:13:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f859fe00640 1 -- 172.21.15.161:0/4111416031 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d5e7b60360 con 0x55d5e7ba59c0 2024-05-22T21:13:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f859fe00640 1 -- 172.21.15.161:0/4111416031 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f85ac0ebe90 con 0x55d5e7ba59c0 2024-05-22T21:13:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 10 client.4653 fetch_fsmap finished waiting for FSMap version 8 2024-05-22T21:13:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 10 client.4653 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-22T21:13:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 20 client.4653 populate_metadata read hostname 'smithi161' 2024-05-22T21:13:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 --2- 172.21.15.161:0/4111416031 >> [v2:172.21.15.179:6833/3095599688,v1:172.21.15.179:6835/3095599688] conn(0x55d5e7beb490 0x55d5e7beb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 4 client.4653 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-22T21:13:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 -- 172.21.15.161:0/4111416031 --> [v2:172.21.15.179:6833/3095599688,v1:172.21.15.179:6835/3095599688] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d5e797a460 con 0x55d5e7beb490 2024-05-22T21:13:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85bea00640 1 --2- 172.21.15.161:0/4111416031 >> [v2:172.21.15.179:6833/3095599688,v1:172.21.15.179:6835/3095599688] conn(0x55d5e7beb490 0x55d5e7beb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85bea00640 1 --2- 172.21.15.161:0/4111416031 >> [v2:172.21.15.179:6833/3095599688,v1:172.21.15.179:6835/3095599688] conn(0x55d5e7beb490 0x55d5e7beb870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c572f20ae539c64 server_cookie=30a428b8904ae406 in_seq=0 out_seq=0 2024-05-22T21:13:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f859fe00640 10 client.4653 ms_handle_connect on v2:172.21.15.179:6833/3095599688 2024-05-22T21:13:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f859fe00640 1 -- 172.21.15.161:0/4111416031 <== mds.0 v2:172.21.15.179:6833/3095599688 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55d5e797a460 con 0x55d5e7beb490 2024-05-22T21:13:42.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f859fe00640 10 client.4653 handle_command_reply: tid=0 2024-05-22T21:13:42.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 client.4653 shutdown 2024-05-22T21:13:42.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 -- 172.21.15.161:0/4111416031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 msgr2=0x55d5e7bea620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:42.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 --2- 172.21.15.161:0/4111416031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7bea620 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f85ac000c10 tx=0x7f85ac11d470 comp rx=0 tx=0).stop 2024-05-22T21:13:42.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 -- 172.21.15.161:0/4111416031 shutdown_connections 2024-05-22T21:13:42.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 --2- 172.21.15.161:0/4111416031 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d5e7ba62e0 0x55d5e7beab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 --2- 172.21.15.161:0/4111416031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d5e7ba59c0 0x55d5e7bea620 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 --2- 172.21.15.161:0/4111416031 >> [v2:172.21.15.179:6833/3095599688,v1:172.21.15.179:6835/3095599688] conn(0x55d5e7beb490 0x55d5e7beb870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 --2- 172.21.15.161:0/4111416031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e7b5bec0 0x55d5e7bb97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f859fe00640 0 client.4653 ms_handle_reset on v2:172.21.15.179:6833/3095599688 2024-05-22T21:13:42.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 -- 172.21.15.161:0/4111416031 >> 172.21.15.161:0/4111416031 conn(0x55d5e7b62440 msgr2=0x55d5e7bb5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 -- 172.21.15.161:0/4111416031 shutdown_connections 2024-05-22T21:13:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 1 -- 172.21.15.161:0/4111416031 wait complete. 2024-05-22T21:13:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.102+0000 7f85c90ab000 20 client.4653 trim_cache size 0 max 16384 2024-05-22T21:13:42.114 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:13:42.114 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:13:42.114 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 0, 2024-05-22T21:13:42.114 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 4282, 2024-05-22T21:13:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2024-05-22T21:13:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2024-05-22T21:13:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2024-05-22T21:13:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 18.685572010000001, 2024-05-22T21:13:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 8, 2024-05-22T21:13:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 21, 2024-05-22T21:13:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 21, 2024-05-22T21:13:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 25.516154239999999 2024-05-22T21:13:42.115 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:13:42.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.106+0000 7f85c5c00640 1 -- 172.21.15.161:0/2305159708 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85a0066690 msgr2=0x7f85a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:42.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.106+0000 7f85c5c00640 1 --2- 172.21.15.161:0/2305159708 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85a0066690 0x7f85a0068b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f85b0006aa0 tx=0x7f85b0006a30 comp rx=0 tx=0).stop 2024-05-22T21:13:42.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.106+0000 7f85c5c00640 1 -- 172.21.15.161:0/2305159708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c013fc80 msgr2=0x7f85c00acff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:42.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.106+0000 7f85c5c00640 1 --2- 172.21.15.161:0/2305159708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c013fc80 0x7f85c00acff0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f85ac01b770 tx=0x7f85ac01b900 comp rx=0 tx=0).stop 2024-05-22T21:13:42.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.106+0000 7f85c5c00640 1 -- 172.21.15.161:0/2305159708 shutdown_connections 2024-05-22T21:13:42.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.106+0000 7f85c5c00640 1 --2- 172.21.15.161:0/2305159708 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85a0066690 0x7f85a0068b50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.106+0000 7f85c5c00640 1 --2- 172.21.15.161:0/2305159708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85c015af60 0x7f85c0156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.106+0000 7f85c5c00640 1 --2- 172.21.15.161:0/2305159708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85c013fc80 0x7f85c00acff0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.106+0000 7f85c5c00640 1 --2- 172.21.15.161:0/2305159708 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85c013cfc0 0x7f85c0142320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.106+0000 7f85c5c00640 1 -- 172.21.15.161:0/2305159708 >> 172.21.15.161:0/2305159708 conn(0x7f85c006a2d0 msgr2=0x7f85c00b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:42.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.106+0000 7f85c5c00640 1 -- 172.21.15.161:0/2305159708 shutdown_connections 2024-05-22T21:13:42.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.106+0000 7f85c5c00640 1 -- 172.21.15.161:0/2305159708 wait complete. 2024-05-22T21:13:42.127 INFO:teuthology.run_tasks:Running task kclient... 2024-05-22T21:13:42.159 INFO:tasks.kclient:Mounting kernel clients... 2024-05-22T21:13:42.159 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-22T21:13:42.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:42.159 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-05-22T21:13:42.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:42.169 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2024-05-22T21:13:42.233 INFO:teuthology.orchestra.run.smithi161.stderr:Cannot find device "ceph-brx" 2024-05-22T21:13:42.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:13:42.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:42.236 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-05-22T21:13:42.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:42.288 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2024-05-22T21:13:42.353 INFO:teuthology.orchestra.run.smithi161.stderr:Cannot find device "ceph-brx" 2024-05-22T21:13:42.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:13:42.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:42.355 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-22T21:13:42.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:42.366 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link delete ceph-brx 2024-05-22T21:13:42.434 INFO:teuthology.orchestra.run.smithi179.stderr:Cannot find device "ceph-brx" 2024-05-22T21:13:42.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:13:42.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:42.437 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-22T21:13:42.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:42.488 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link delete ceph-brx 2024-05-22T21:13:42.554 INFO:teuthology.orchestra.run.smithi179.stderr:Cannot find device "ceph-brx" 2024-05-22T21:13:42.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:13:42.556 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-05-22T21:13:42.556 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-22T21:13:42.556 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-22T21:13:42.556 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:13:42.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0ba3a00640 1 Processor -- start 2024-05-22T21:13:42.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0ba3a00640 1 -- start start 2024-05-22T21:13:42.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0ba3a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b9c0cbfa0 0x7f0b9c15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0ba3a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0b9c0d0ba0 0x7f0b9c0cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0ba3a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9c15c650 con 0x7f0b9c06cff0 2024-05-22T21:13:42.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0ba3a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b9c0711f0 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0ba3a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1590f0 con 0x7f0b9c0cbfa0 2024-05-22T21:13:42.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0ba1a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0b9c06cff0 0x7f0b9c071680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43566/0 (socket says 172.21.15.161:43566) 2024-05-22T21:13:42.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0ba1a00640 1 -- 172.21.15.161:0/3536932363 learned_addr learned my addr 172.21.15.161:0/3536932363 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:42.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0b93e00640 1 --2- 172.21.15.161:0/3536932363 >> v2:172.21.15.179:3300/0 conn(0x7f0b9c0d0ba0 0x7f0b9c0cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0ba1000640 1 --2- 172.21.15.161:0/3536932363 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b9c0cbfa0 0x7f0b9c15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0b9be00640 1 -- 172.21.15.161:0/3536932363 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1444578387 0 0) 0x7f0b9c15c650 con 0x7f0b9c06cff0 2024-05-22T21:13:42.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0b9be00640 1 -- 172.21.15.161:0/3536932363 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b80003530 con 0x7f0b9c06cff0 2024-05-22T21:13:42.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0b93e00640 1 -- 172.21.15.161:0/3536932363 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b9c0cbfa0 msgr2=0x7f0b9c15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:42.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0b93e00640 1 --2- 172.21.15.161:0/3536932363 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b9c0cbfa0 0x7f0b9c15b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0b93e00640 1 -- 172.21.15.161:0/3536932363 >> v1:172.21.15.161:6789/0 conn(0x7f0b9c06cff0 legacy=0x7f0b9c071680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:42.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.690+0000 7f0b93e00640 1 -- 172.21.15.161:0/3536932363 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c0cc8c0 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.694+0000 7f0b93e00640 1 --2- 172.21.15.161:0/3536932363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b9c0d0ba0 0x7f0b9c0cc380 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b840097c0 tx=0x7f0b8401c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=541cac40fc706aaa server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:42.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.694+0000 7f0b9be00640 1 -- 172.21.15.161:0/3536932363 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b840ab600 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.694+0000 7f0b9be00640 1 -- 172.21.15.161:0/3536932363 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b840a7050 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.694+0000 7f0b9be00640 1 -- 172.21.15.161:0/3536932363 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b840ab960 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.694+0000 7f0ba3a00640 1 -- 172.21.15.161:0/3536932363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b9c0d0ba0 msgr2=0x7f0b9c0cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:42.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.694+0000 7f0ba3a00640 1 --2- 172.21.15.161:0/3536932363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b9c0d0ba0 0x7f0b9c0cc380 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b840097c0 tx=0x7f0b8401c380 comp rx=0 tx=0).stop 2024-05-22T21:13:42.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.694+0000 7f0ba3a00640 1 -- 172.21.15.161:0/3536932363 shutdown_connections 2024-05-22T21:13:42.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.694+0000 7f0ba3a00640 1 --2- 172.21.15.161:0/3536932363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b9c0d0ba0 0x7f0b9c0cc380 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.694+0000 7f0ba3a00640 1 --2- 172.21.15.161:0/3536932363 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b9c0cbfa0 0x7f0b9c15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.694+0000 7f0ba3a00640 1 -- 172.21.15.161:0/3536932363 >> 172.21.15.161:0/3536932363 conn(0x7f0b9c0d00b0 msgr2=0x7f0b9c0d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:42.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.694+0000 7f0ba3a00640 1 -- 172.21.15.161:0/3536932363 shutdown_connections 2024-05-22T21:13:42.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.694+0000 7f0ba3a00640 1 -- 172.21.15.161:0/3536932363 wait complete. 2024-05-22T21:13:42.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba3a00640 1 Processor -- start 2024-05-22T21:13:42.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba3a00640 1 -- start start 2024-05-22T21:13:42.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba3a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b9c06cff0 0x7f0b9c0cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba3a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b9c0cbfa0 0x7f0b9c14ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba3a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b9c0d0ba0 0x7f0b9c0cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba3a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0c6bd0 con 0x7f0b9c0cbfa0 2024-05-22T21:13:42.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba3a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0c7180 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba3a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c0c6610 con 0x7f0b9c06cff0 2024-05-22T21:13:42.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0b93e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b9c0cbfa0 0x7f0b9c14ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba1a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b9c0d0ba0 0x7f0b9c0cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba1000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b9c06cff0 0x7f0b9c0cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0b93e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b9c0cbfa0 0x7f0b9c14ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54866/0 (socket says 172.21.15.161:54866) 2024-05-22T21:13:42.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0b93e00640 1 -- 172.21.15.161:0/1367882801 learned_addr learned my addr 172.21.15.161:0/1367882801 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:42.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba1a00640 1 -- 172.21.15.161:0/1367882801 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b9c06cff0 msgr2=0x7f0b9c0cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:42.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba1a00640 1 --2- 172.21.15.161:0/1367882801 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b9c06cff0 0x7f0b9c0cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba1a00640 1 -- 172.21.15.161:0/1367882801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b9c0cbfa0 msgr2=0x7f0b9c14ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:42.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba1a00640 1 --2- 172.21.15.161:0/1367882801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b9c0cbfa0 0x7f0b9c14ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba1a00640 1 -- 172.21.15.161:0/1367882801 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b84009420 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba1a00640 1 --2- 172.21.15.161:0/1367882801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b9c0d0ba0 0x7f0b9c0cdc80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400caa0 tx=0x7f0b9400cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:42.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0b99600640 1 -- 172.21.15.161:0/1367882801 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9400ab40 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0b99600640 1 -- 172.21.15.161:0/1367882801 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9400aca0 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0b99600640 1 -- 172.21.15.161:0/1367882801 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94012600 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba3a00640 1 -- 172.21.15.161:0/1367882801 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9c0c68f0 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.698+0000 7f0ba3a00640 1 -- 172.21.15.161:0/1367882801 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c0b3110 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.702+0000 7f0ba3a00640 1 -- 172.21.15.161:0/1367882801 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9c0b3300 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.702+0000 7f0b99600640 1 -- 172.21.15.161:0/1367882801 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0b94007070 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.702+0000 7f0b99600640 1 --2- 172.21.15.161:0/1367882801 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b78066690 0x7f0b78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:42.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.702+0000 7f0b99600640 1 -- 172.21.15.161:0/1367882801 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b94083fe0 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.702+0000 7f0ba1000640 1 --2- 172.21.15.161:0/1367882801 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b78066690 0x7f0b78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:42.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.702+0000 7f0ba1000640 1 --2- 172.21.15.161:0/1367882801 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b78066690 0x7f0b78068b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009ac0 tx=0x7f0b8c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:42.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.706+0000 7f0b99600640 1 -- 172.21.15.161:0/1367882801 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0b94051380 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.890+0000 7f0ba3a00640 1 -- 172.21.15.161:0/1367882801 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0b9c067220 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.894+0000 7f0b99600640 1 -- 172.21.15.161:0/1367882801 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0b94017070 con 0x7f0b9c0d0ba0 2024-05-22T21:13:42.896 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:13:42.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.894+0000 7f0ba3a00640 1 -- 172.21.15.161:0/1367882801 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b78066690 msgr2=0x7f0b78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:42.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.894+0000 7f0ba3a00640 1 --2- 172.21.15.161:0/1367882801 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b78066690 0x7f0b78068b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009ac0 tx=0x7f0b8c009190 comp rx=0 tx=0).stop 2024-05-22T21:13:42.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.894+0000 7f0ba3a00640 1 -- 172.21.15.161:0/1367882801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b9c0d0ba0 msgr2=0x7f0b9c0cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:42.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.894+0000 7f0ba3a00640 1 --2- 172.21.15.161:0/1367882801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b9c0d0ba0 0x7f0b9c0cdc80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400caa0 tx=0x7f0b9400cea0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.894+0000 7f0ba3a00640 1 -- 172.21.15.161:0/1367882801 shutdown_connections 2024-05-22T21:13:42.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.894+0000 7f0ba3a00640 1 --2- 172.21.15.161:0/1367882801 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b78066690 0x7f0b78068b50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.894+0000 7f0ba3a00640 1 --2- 172.21.15.161:0/1367882801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b9c0d0ba0 0x7f0b9c0cdc80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.894+0000 7f0ba3a00640 1 --2- 172.21.15.161:0/1367882801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b9c0cbfa0 0x7f0b9c14ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.894+0000 7f0ba3a00640 1 --2- 172.21.15.161:0/1367882801 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b9c06cff0 0x7f0b9c0cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:42.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.894+0000 7f0ba3a00640 1 -- 172.21.15.161:0/1367882801 >> 172.21.15.161:0/1367882801 conn(0x7f0b9c0d00b0 msgr2=0x7f0b9c159ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:42.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.894+0000 7f0ba3a00640 1 -- 172.21.15.161:0/1367882801 shutdown_connections 2024-05-22T21:13:42.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:42.894+0000 7f0ba3a00640 1 -- 172.21.15.161:0/1367882801 wait complete. 2024-05-22T21:13:42.917 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-22T21:13:42.917 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:13:42.917 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-05-22T21:13:42.917 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-22T21:13:42.917 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-22T21:13:42.917 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-22T21:13:42.917 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-22T21:13:42.917 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:13:42.918 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:13:42.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:42.918 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-05-22T21:13:42.926 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:13:42.927 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:13:42.927 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:13:42.927 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:42.927 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-05-22T21:13:42.927 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:42.927 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:13:42.927 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:42.927 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2024-05-22T21:13:42.927 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:13:42.927 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:42.927 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2024-05-22T21:13:42.927 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:13:42.928 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:42.928 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:13:42.928 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:42.928 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:13:42.928 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42469sec preferred_lft 42469sec 2024-05-22T21:13:42.928 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-05-22T21:13:42.928 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:42.928 INFO:teuthology.orchestra.run.smithi161.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:13:42.928 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 02:42:b9:0f:57:df brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:42.928 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:13:42.928 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:42.929 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-22T21:13:42.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:42.930 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:13:42.930 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:13:42.930 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link add name ceph-brx type bridge 2024-05-22T21:13:42.930 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip addr flush dev ceph-brx 2024-05-22T21:13:42.930 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set ceph-brx up 2024-05-22T21:13:42.930 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-22T21:13:42.930 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:13:43.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:43.041 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:13:43.059 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-05-22T21:13:43.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:43.061 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-05-22T21:13:43.163 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-05-22T21:13:43.163 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:13:43.163 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:13:43.163 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:13:43.164 INFO:teuthology.orchestra.run.smithi161.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:13:43.164 INFO:teuthology.orchestra.run.smithi161.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:13:43.164 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:13:43.164 INFO:teuthology.orchestra.run.smithi161.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:13:43.164 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:13:43.164 INFO:teuthology.orchestra.run.smithi161.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:13:43.164 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:13:43.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:43.164 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:13:43.165 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:13:43.165 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:13:43.165 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:13:43.165 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:13:43.165 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:13:43.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:43.212 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-05-22T21:13:43.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:43.222 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list-id 2024-05-22T21:13:43.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:43.275 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:13:43.276 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:13:43.276 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-22T21:13:43.276 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-22T21:13:43.276 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:13:43.357 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-22T21:13:43.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:43.358 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:13:43.358 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:13:43.358 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-22T21:13:43.358 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-22T21:13:43.358 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-22T21:13:43.358 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-22T21:13:43.358 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-22T21:13:43.358 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:13:43.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:43.519 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:13:43.519 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:13:43.519 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2024-05-22T21:13:43.519 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set dev brx.0 master ceph-brx 2024-05-22T21:13:43.519 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:13:43.557 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:13:43.557 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-22T21:13:43.611 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:13:43.611 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:13:43.612 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:13:43.667 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:13:43.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:43.667 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:13:43.755 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:13:43.756 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:13:43.756 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:13:44.428 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:13:44.428 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:13:44.428 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:13:44.428 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:13:44.428 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:13:44.440 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:13:44.440 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:13:44.440 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:13:44.440 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:13:44.441 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:13:44.441 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:13:44.441 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:13:44.441 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:13:44.445 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:13:44.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:44.445 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:13:44.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:44.467 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:13:44.544 INFO:teuthology.orchestra.run.smithi161.stdout:client4662 2024-05-22T21:13:44.544 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:44.544 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:13:44.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:44.596 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:13:44.666 INFO:teuthology.orchestra.run.smithi161.stdout:client4662 2024-05-22T21:13:44.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:44.666 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:13:44.727 INFO:teuthology.orchestra.run.smithi161.stdout:client4662 2024-05-22T21:13:44.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:44.728 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d6fa8528-b1bf-41b6-9f76-c0d715394087.client4662/status') 2024-05-22T21:13:44.798 INFO:teuthology.orchestra.run.smithi161.stdout:instance: client.4662 (3)192.168.0.1:0/833193947 2024-05-22T21:13:44.798 INFO:teuthology.orchestra.run.smithi161.stdout:blocklisted: false 2024-05-22T21:13:44.799 INFO:teuthology.orchestra.run.smithi161.stderr:0+1 records in 2024-05-22T21:13:44.799 INFO:teuthology.orchestra.run.smithi161.stderr:0+1 records out 2024-05-22T21:13:44.799 INFO:teuthology.orchestra.run.smithi161.stderr:68 bytes copied, 6.754e-05 s, 1.0 MB/s 2024-05-22T21:13:44.800 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-05-22T21:13:44.800 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-22T21:13:44.800 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-22T21:13:44.801 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:13:44.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.978+0000 7f0b9e200640 1 Processor -- start 2024-05-22T21:13:44.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.978+0000 7f0b9e200640 1 -- start start 2024-05-22T21:13:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.978+0000 7f0b9e200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0b98144550 0x7f0b980cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.978+0000 7f0b9e200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0b9806be70 0x7f0b9806c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.978+0000 7f0b9e200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9806de20 con 0x7f0b9815b420 2024-05-22T21:13:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.978+0000 7f0b9e200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b9806ba50 con 0x7f0b98144550 2024-05-22T21:13:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.978+0000 7f0b9e200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b980cdd20 con 0x7f0b9806be70 2024-05-22T21:13:44.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.978+0000 7f0b9ce00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0b98144550 0x7f0b980cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:44.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.978+0000 7f0b9ce00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0b98144550 0x7f0b980cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54872/0 (socket says 172.21.15.161:54872) 2024-05-22T21:13:44.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.978+0000 7f0b9ce00640 1 -- 172.21.15.161:0/1554976479 learned_addr learned my addr 172.21.15.161:0/1554976479 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:44.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.978+0000 7f0b9d800640 1 --2- 172.21.15.161:0/1554976479 >> v2:172.21.15.179:3300/0 conn(0x7f0b9806be70 0x7f0b9806c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:44.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.982+0000 7f0b9ce00640 1 -- 172.21.15.161:0/1554976479 >> v2:172.21.15.179:3300/0 conn(0x7f0b9806be70 msgr2=0x7f0b9806c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:44.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.982+0000 7f0b9ce00640 1 --2- 172.21.15.161:0/1554976479 >> v2:172.21.15.179:3300/0 conn(0x7f0b9806be70 0x7f0b9806c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:44.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.982+0000 7f0b9ce00640 1 -- 172.21.15.161:0/1554976479 >> v1:172.21.15.179:6789/0 conn(0x7f0b9815b420 legacy=0x7f0b9815b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:44.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.982+0000 7f0b9ce00640 1 -- 172.21.15.161:0/1554976479 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b980ce000 con 0x7f0b98144550 2024-05-22T21:13:44.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.982+0000 7f0b9ce00640 1 --2- 172.21.15.161:0/1554976479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b98144550 0x7f0b980cb5a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0b980b10d0 tx=0x7f0b8c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcaac11b2a3015b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:44.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.982+0000 7f0b97400640 1 -- 172.21.15.161:0/1554976479 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c0ab620 con 0x7f0b98144550 2024-05-22T21:13:44.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.982+0000 7f0b97400640 1 -- 172.21.15.161:0/1554976479 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8c0ab780 con 0x7f0b98144550 2024-05-22T21:13:44.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.982+0000 7f0b97400640 1 -- 172.21.15.161:0/1554976479 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c0abb10 con 0x7f0b98144550 2024-05-22T21:13:44.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.982+0000 7f0b9e200640 1 -- 172.21.15.161:0/1554976479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b98144550 msgr2=0x7f0b980cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:44.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.982+0000 7f0b9e200640 1 --2- 172.21.15.161:0/1554976479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b98144550 0x7f0b980cb5a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0b980b10d0 tx=0x7f0b8c01c420 comp rx=0 tx=0).stop 2024-05-22T21:13:44.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.982+0000 7f0b9e200640 1 -- 172.21.15.161:0/1554976479 shutdown_connections 2024-05-22T21:13:44.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.982+0000 7f0b9e200640 1 --2- 172.21.15.161:0/1554976479 >> v2:172.21.15.179:3300/0 conn(0x7f0b9806be70 0x7f0b9806c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:44.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.982+0000 7f0b9e200640 1 --2- 172.21.15.161:0/1554976479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b98144550 0x7f0b980cb5a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:44.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.982+0000 7f0b9e200640 1 -- 172.21.15.161:0/1554976479 >> 172.21.15.161:0/1554976479 conn(0x7f0b9806a4d0 msgr2=0x7f0b980b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:44.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.982+0000 7f0b9e200640 1 -- 172.21.15.161:0/1554976479 shutdown_connections 2024-05-22T21:13:44.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.982+0000 7f0b9e200640 1 -- 172.21.15.161:0/1554976479 wait complete. 2024-05-22T21:13:44.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9e200640 1 Processor -- start 2024-05-22T21:13:44.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9e200640 1 -- start start 2024-05-22T21:13:44.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9e200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b9806be70 0x7f0b98070da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:44.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9e200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b98144550 0x7f0b9815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:44.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9e200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b9815b420 0x7f0b9815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:44.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9e200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980c8ff0 con 0x7f0b9806be70 2024-05-22T21:13:44.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9e200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980c92d0 con 0x7f0b98144550 2024-05-22T21:13:44.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9e200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980c95b0 con 0x7f0b9815b420 2024-05-22T21:13:44.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b9806be70 0x7f0b98070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:44.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b9806be70 0x7f0b98070da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54876/0 (socket says 172.21.15.161:54876) 2024-05-22T21:13:44.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9ce00640 1 -- 172.21.15.161:0/281870260 learned_addr learned my addr 172.21.15.161:0/281870260 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:44.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b97e00640 1 --2- 172.21.15.161:0/281870260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b98144550 0x7f0b9815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:44.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9d800640 1 --2- 172.21.15.161:0/281870260 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b9815b420 0x7f0b9815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:44.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9ce00640 1 -- 172.21.15.161:0/281870260 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b9815b420 msgr2=0x7f0b9815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:44.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9ce00640 1 --2- 172.21.15.161:0/281870260 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b9815b420 0x7f0b9815a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:44.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9ce00640 1 -- 172.21.15.161:0/281870260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b98144550 msgr2=0x7f0b9815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:44.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9ce00640 1 --2- 172.21.15.161:0/281870260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b98144550 0x7f0b9815a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:44.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9ce00640 1 -- 172.21.15.161:0/281870260 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c0094d0 con 0x7f0b9806be70 2024-05-22T21:13:44.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9ce00640 1 --2- 172.21.15.161:0/281870260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b9806be70 0x7f0b98070da0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c01a050 tx=0x7f0b8c01ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:44.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b94c00640 1 -- 172.21.15.161:0/281870260 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c0be3f0 con 0x7f0b9806be70 2024-05-22T21:13:44.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b94c00640 1 -- 172.21.15.161:0/281870260 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8c0be550 con 0x7f0b9806be70 2024-05-22T21:13:44.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b94c00640 1 -- 172.21.15.161:0/281870260 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c0be8e0 con 0x7f0b9806be70 2024-05-22T21:13:44.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9e200640 1 -- 172.21.15.161:0/281870260 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b980c9890 con 0x7f0b9806be70 2024-05-22T21:13:44.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.986+0000 7f0b9e200640 1 -- 172.21.15.161:0/281870260 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b981351f0 con 0x7f0b9806be70 2024-05-22T21:13:44.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.990+0000 7f0b9e200640 1 -- 172.21.15.161:0/281870260 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b98067070 con 0x7f0b9806be70 2024-05-22T21:13:44.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.990+0000 7f0b94c00640 1 -- 172.21.15.161:0/281870260 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0b8c0b43f0 con 0x7f0b9806be70 2024-05-22T21:13:44.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.990+0000 7f0b94c00640 1 --2- 172.21.15.161:0/281870260 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b78066690 0x7f0b78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:44.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.990+0000 7f0b94c00640 1 -- 172.21.15.161:0/281870260 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b8c11e630 con 0x7f0b9806be70 2024-05-22T21:13:44.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.990+0000 7f0b97e00640 1 --2- 172.21.15.161:0/281870260 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b78066690 0x7f0b78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:44.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.990+0000 7f0b97e00640 1 --2- 172.21.15.161:0/281870260 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b78066690 0x7f0b78068b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0b9813dfa0 tx=0x7f0b84009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:44.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:44.994+0000 7f0b94c00640 1 -- 172.21.15.161:0/281870260 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0b8c0eb9b0 con 0x7f0b9806be70 2024-05-22T21:13:45.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:45.182+0000 7f0b9e200640 1 -- 172.21.15.161:0/281870260 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0b980b13c0 con 0x7f0b9806be70 2024-05-22T21:13:45.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:45.182+0000 7f0b94c00640 1 -- 172.21.15.161:0/281870260 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0b8c0eefd0 con 0x7f0b9806be70 2024-05-22T21:13:45.187 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:13:45.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:45.186+0000 7f0b9e200640 1 -- 172.21.15.161:0/281870260 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b78066690 msgr2=0x7f0b78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:45.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:45.186+0000 7f0b9e200640 1 --2- 172.21.15.161:0/281870260 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b78066690 0x7f0b78068b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0b9813dfa0 tx=0x7f0b84009060 comp rx=0 tx=0).stop 2024-05-22T21:13:45.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:45.186+0000 7f0b9e200640 1 -- 172.21.15.161:0/281870260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b9806be70 msgr2=0x7f0b98070da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:45.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:45.186+0000 7f0b9e200640 1 --2- 172.21.15.161:0/281870260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b9806be70 0x7f0b98070da0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c01a050 tx=0x7f0b8c01ac60 comp rx=0 tx=0).stop 2024-05-22T21:13:45.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:45.186+0000 7f0b9e200640 1 -- 172.21.15.161:0/281870260 shutdown_connections 2024-05-22T21:13:45.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:45.186+0000 7f0b9e200640 1 --2- 172.21.15.161:0/281870260 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b78066690 0x7f0b78068b50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:45.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:45.186+0000 7f0b9e200640 1 --2- 172.21.15.161:0/281870260 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b9815b420 0x7f0b9815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:45.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:45.186+0000 7f0b9e200640 1 --2- 172.21.15.161:0/281870260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b98144550 0x7f0b9815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:45.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:45.186+0000 7f0b9e200640 1 --2- 172.21.15.161:0/281870260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b9806be70 0x7f0b98070da0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:45.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:45.186+0000 7f0b9e200640 1 -- 172.21.15.161:0/281870260 >> 172.21.15.161:0/281870260 conn(0x7f0b9806a4d0 msgr2=0x7f0b980b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:45.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:45.186+0000 7f0b9e200640 1 -- 172.21.15.161:0/281870260 shutdown_connections 2024-05-22T21:13:45.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:45.186+0000 7f0b9e200640 1 -- 172.21.15.161:0/281870260 wait complete. 2024-05-22T21:13:45.208 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-22T21:13:45.208 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:13:45.208 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-05-22T21:13:45.208 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-22T21:13:45.208 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-22T21:13:45.208 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-22T21:13:45.208 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-22T21:13:45.208 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:13:45.208 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-22T21:13:45.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:45.209 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-05-22T21:13:45.218 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:13:45.218 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:13:45.218 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:13:45.218 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:45.218 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-05-22T21:13:45.218 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:45.218 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:13:45.218 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:45.218 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2024-05-22T21:13:45.218 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:13:45.218 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:45.219 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2024-05-22T21:13:45.219 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:13:45.219 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:45.219 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:13:45.219 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:45.219 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:13:45.219 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42467sec preferred_lft 42467sec 2024-05-22T21:13:45.219 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-05-22T21:13:45.219 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:45.219 INFO:teuthology.orchestra.run.smithi161.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:13:45.219 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 02:42:b9:0f:57:df brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:45.219 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:13:45.219 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:45.219 INFO:teuthology.orchestra.run.smithi161.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:13:45.220 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:45.220 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:13:45.220 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:45.220 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:13:45.220 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:45.220 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:13:45.220 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-22T21:13:45.220 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:13:45.220 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:45.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:45.221 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:13:45.281 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-05-22T21:13:45.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:45.283 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-05-22T21:13:45.339 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-05-22T21:13:45.339 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:13:45.339 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:13:45.339 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:13:45.339 INFO:teuthology.orchestra.run.smithi161.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:13:45.339 INFO:teuthology.orchestra.run.smithi161.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:13:45.339 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:13:45.340 INFO:teuthology.orchestra.run.smithi161.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:13:45.340 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:13:45.340 INFO:teuthology.orchestra.run.smithi161.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:13:45.340 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:13:45.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:45.340 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:13:45.340 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:13:45.340 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:13:45.340 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:13:45.340 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:13:45.340 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:13:45.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:45.427 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-05-22T21:13:45.436 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-22T21:13:45.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:45.437 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list-id 2024-05-22T21:13:45.491 INFO:teuthology.orchestra.run.smithi161.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-22T21:13:45.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:45.492 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:13:45.492 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:13:45.492 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-22T21:13:45.492 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-22T21:13:45.492 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:13:45.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:45.577 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-22T21:13:45.639 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:13:45.639 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:13:45.639 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:13:45.639 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:45.639 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-05-22T21:13:45.639 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:45.639 INFO:teuthology.orchestra.run.smithi161.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:13:45.640 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 6e:99:68:b8:ad:40 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-22T21:13:45.640 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-22T21:13:45.640 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:45.640 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::6c99:68ff:feb8:ad40/64 scope link 2024-05-22T21:13:45.640 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:45.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:45.641 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-22T21:13:45.702 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:13:45.703 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:13:45.703 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:13:45.703 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:45.703 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-05-22T21:13:45.703 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:45.703 INFO:teuthology.orchestra.run.smithi161.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:13:45.703 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 6e:99:68:b8:ad:40 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-22T21:13:45.703 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-22T21:13:45.703 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:45.703 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::6c99:68ff:feb8:ad40/64 scope link 2024-05-22T21:13:45.703 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:45.705 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-22T21:13:45.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:45.705 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:13:45.705 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:13:45.705 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-22T21:13:45.705 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-22T21:13:45.705 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-22T21:13:45.706 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-22T21:13:45.706 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-22T21:13:45.706 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:13:45.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:45.839 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:13:45.839 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:13:45.839 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.1 up 2024-05-22T21:13:45.839 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set dev brx.1 master ceph-brx 2024-05-22T21:13:45.839 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:13:45.876 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:13:45.876 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-22T21:13:45.927 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-22T21:13:45.927 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:13:45.927 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-22T21:13:45.979 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:13:45.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:45.979 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-22T21:13:46.066 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-22T21:13:46.067 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-22T21:13:46.067 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:13:46.081 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-22T21:13:46.081 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-22T21:13:46.082 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:13:46.082 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:13:46.082 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:13:46.090 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-22T21:13:46.090 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-22T21:13:46.090 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:13:46.090 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:13:46.091 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:13:46.091 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:13:46.091 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:13:46.091 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:13:46.095 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:13:46.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:46.096 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-22T21:13:46.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:46.115 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:13:46.167 INFO:teuthology.orchestra.run.smithi161.stdout:client4671 2024-05-22T21:13:46.167 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:46.167 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:13:46.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:46.219 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:13:46.274 INFO:teuthology.orchestra.run.smithi161.stdout:client4671 2024-05-22T21:13:46.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:46.275 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:13:46.326 INFO:teuthology.orchestra.run.smithi161.stdout:client4671 2024-05-22T21:13:46.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:46.327 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d6fa8528-b1bf-41b6-9f76-c0d715394087.client4671/status') 2024-05-22T21:13:46.386 INFO:teuthology.orchestra.run.smithi161.stdout:instance: client.4671 (3)192.168.0.2:0/273678278 2024-05-22T21:13:46.386 INFO:teuthology.orchestra.run.smithi161.stdout:blocklisted: false 2024-05-22T21:13:46.387 INFO:teuthology.orchestra.run.smithi161.stderr:0+1 records in 2024-05-22T21:13:46.387 INFO:teuthology.orchestra.run.smithi161.stderr:0+1 records out 2024-05-22T21:13:46.387 INFO:teuthology.orchestra.run.smithi161.stderr:68 bytes copied, 4.9877e-05 s, 1.4 MB/s 2024-05-22T21:13:46.388 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-05-22T21:13:46.389 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-22T21:13:46.389 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-22T21:13:46.389 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:13:46.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8480a00640 1 Processor -- start 2024-05-22T21:13:46.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8480a00640 1 -- start start 2024-05-22T21:13:46.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8480a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f847c15dd50 0x7f847c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:46.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8480a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c15d5e0 0x7f847c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:46.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8480a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847c071670 con 0x7f847c15e130 2024-05-22T21:13:46.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8480a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f847c06de40 con 0x7f847c15dd50 2024-05-22T21:13:46.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8480a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c070a60 con 0x7f847c15d5e0 2024-05-22T21:13:46.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8479400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f847c15e130 0x7f847c15ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51972/0 (socket says 172.21.15.161:51972) 2024-05-22T21:13:46.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8479400640 1 -- 172.21.15.161:0/1150930514 learned_addr learned my addr 172.21.15.161:0/1150930514 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f847a800640 1 --2- 172.21.15.161:0/1150930514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c15d5e0 0x7f847c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8478a00640 1 -- 172.21.15.161:0/1150930514 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960471062 0 0) 0x7f847c071670 con 0x7f847c15e130 2024-05-22T21:13:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8479e00640 1 --2- 172.21.15.161:0/1150930514 >> v2:172.21.15.179:3300/0 conn(0x7f847c15dd50 0x7f847c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8478a00640 1 -- 172.21.15.161:0/1150930514 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8460003530 con 0x7f847c15e130 2024-05-22T21:13:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8478a00640 1 -- 172.21.15.161:0/1150930514 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2841032374 0 0) 0x7f8460003530 con 0x7f847c15e130 2024-05-22T21:13:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8478a00640 1 -- 172.21.15.161:0/1150930514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c15d5e0 msgr2=0x7f847c15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8478a00640 1 --2- 172.21.15.161:0/1150930514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c15d5e0 0x7f847c15b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8478a00640 1 -- 172.21.15.161:0/1150930514 >> v2:172.21.15.179:3300/0 conn(0x7f847c15dd50 msgr2=0x7f847c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8478a00640 1 --2- 172.21.15.161:0/1150930514 >> v2:172.21.15.179:3300/0 conn(0x7f847c15dd50 0x7f847c06d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8478a00640 1 -- 172.21.15.161:0/1150930514 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c070d40 con 0x7f847c15e130 2024-05-22T21:13:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8478a00640 1 -- 172.21.15.161:0/1150930514 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f8468001680 con 0x7f847c15e130 2024-05-22T21:13:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.566+0000 7f8478a00640 1 -- 172.21.15.161:0/1150930514 >> v1:172.21.15.161:6789/0 conn(0x7f847c15e130 legacy=0x7f847c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8478a00640 1 --2- 172.21.15.161:0/1150930514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460003980 0x7f8460003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8478a00640 1 --2- 172.21.15.161:0/1150930514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c15d5e0 0x7f84600047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8478a00640 1 --2- 172.21.15.161:0/1150930514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8460004d10 0x7f8460005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8478a00640 1 -- 172.21.15.161:0/1150930514 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c06de40 con 0x7f8460003980 2024-05-22T21:13:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8478a00640 1 -- 172.21.15.161:0/1150930514 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c070a60 con 0x7f8460004d10 2024-05-22T21:13:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8478a00640 1 -- 172.21.15.161:0/1150930514 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846000b990 con 0x7f847c15d5e0 2024-05-22T21:13:46.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8479e00640 1 --2- 172.21.15.161:0/1150930514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460003980 0x7f8460003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:46.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8479400640 1 --2- 172.21.15.161:0/1150930514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c15d5e0 0x7f84600047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:46.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f847a800640 1 --2- 172.21.15.161:0/1150930514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8460004d10 0x7f8460005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:46.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8479e00640 1 -- 172.21.15.161:0/1150930514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c15d5e0 msgr2=0x7f84600047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:46.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8479e00640 1 --2- 172.21.15.161:0/1150930514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c15d5e0 0x7f84600047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:46.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8479e00640 1 -- 172.21.15.161:0/1150930514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8460004d10 msgr2=0x7f8460005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:46.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8479e00640 1 --2- 172.21.15.161:0/1150930514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8460004d10 0x7f8460005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:46.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8479e00640 1 -- 172.21.15.161:0/1150930514 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8460003530 con 0x7f8460003980 2024-05-22T21:13:46.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f847a800640 1 --2- 172.21.15.161:0/1150930514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8460004d10 0x7f8460005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:46.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8479e00640 1 --2- 172.21.15.161:0/1150930514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460003980 0x7f8460003da0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f846c010170 tx=0x7f846c01ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5094954956e945 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:46.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8478a00640 1 -- 172.21.15.161:0/1150930514 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c0a7b20 con 0x7f8460003980 2024-05-22T21:13:46.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8478a00640 1 -- 172.21.15.161:0/1150930514 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c011040 con 0x7f8460003980 2024-05-22T21:13:46.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.570+0000 7f8478a00640 1 -- 172.21.15.161:0/1150930514 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c0af440 con 0x7f8460003980 2024-05-22T21:13:46.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8480a00640 1 -- 172.21.15.161:0/1150930514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460003980 msgr2=0x7f8460003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:46.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8480a00640 1 --2- 172.21.15.161:0/1150930514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460003980 0x7f8460003da0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f846c010170 tx=0x7f846c01ec30 comp rx=0 tx=0).stop 2024-05-22T21:13:46.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f847a800640 1 -- 172.21.15.161:0/1150930514 reap_dead start 2024-05-22T21:13:46.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8480a00640 1 -- 172.21.15.161:0/1150930514 shutdown_connections 2024-05-22T21:13:46.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8480a00640 1 -- 172.21.15.161:0/1150930514 >> 172.21.15.161:0/1150930514 conn(0x7f847c0b0bc0 msgr2=0x7f847c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:46.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8480a00640 1 -- 172.21.15.161:0/1150930514 shutdown_connections 2024-05-22T21:13:46.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8480a00640 1 -- 172.21.15.161:0/1150930514 wait complete. 2024-05-22T21:13:46.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8480a00640 1 Processor -- start 2024-05-22T21:13:46.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8480a00640 1 -- start start 2024-05-22T21:13:46.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8480a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8460003980 0x7f847c14ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:46.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8480a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c0b3f10 0x7f847c0ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:46.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8480a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f847c0b1930 0x7f847c0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:46.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8480a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c15b110 con 0x7f847c0b1930 2024-05-22T21:13:46.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8480a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c0b22f0 con 0x7f8460003980 2024-05-22T21:13:46.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8480a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c14f340 con 0x7f847c0b3f10 2024-05-22T21:13:46.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f847a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f847c0b1930 0x7f847c0b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:46.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8479e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8460003980 0x7f847c14ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:46.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f847a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f847c0b1930 0x7f847c0b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46766/0 (socket says 172.21.15.161:46766) 2024-05-22T21:13:46.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f847a800640 1 -- 172.21.15.161:0/2885905685 learned_addr learned my addr 172.21.15.161:0/2885905685 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:46.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8479400640 1 --2- 172.21.15.161:0/2885905685 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c0b3f10 0x7f847c0ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:46.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f847a800640 1 -- 172.21.15.161:0/2885905685 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c0b3f10 msgr2=0x7f847c0ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:46.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f847a800640 1 --2- 172.21.15.161:0/2885905685 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c0b3f10 0x7f847c0ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:46.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f847a800640 1 -- 172.21.15.161:0/2885905685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8460003980 msgr2=0x7f847c14ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:46.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f847a800640 1 --2- 172.21.15.161:0/2885905685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8460003980 0x7f847c14ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:46.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f847a800640 1 -- 172.21.15.161:0/2885905685 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c0adbc0 con 0x7f847c0b1930 2024-05-22T21:13:46.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.574+0000 7f8479e00640 1 --2- 172.21.15.161:0/2885905685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8460003980 0x7f847c14ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:46.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.578+0000 7f847a800640 1 --2- 172.21.15.161:0/2885905685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f847c0b1930 0x7f847c0b1db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f847c073ce0 tx=0x7f8474011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:46.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.578+0000 7f8472000640 1 -- 172.21.15.161:0/2885905685 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8474004230 con 0x7f847c0b1930 2024-05-22T21:13:46.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.578+0000 7f8472000640 1 -- 172.21.15.161:0/2885905685 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84740043b0 con 0x7f847c0b1930 2024-05-22T21:13:46.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.578+0000 7f8472000640 1 -- 172.21.15.161:0/2885905685 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84740158b0 con 0x7f847c0b1930 2024-05-22T21:13:46.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.578+0000 7f8480a00640 1 -- 172.21.15.161:0/2885905685 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847c0adea0 con 0x7f847c0b1930 2024-05-22T21:13:46.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.578+0000 7f8480a00640 1 -- 172.21.15.161:0/2885905685 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847c15ad30 con 0x7f847c0b1930 2024-05-22T21:13:46.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.578+0000 7f8480a00640 1 -- 172.21.15.161:0/2885905685 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847c15b250 con 0x7f847c0b1930 2024-05-22T21:13:46.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.578+0000 7f8472000640 1 -- 172.21.15.161:0/2885905685 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8474015a10 con 0x7f847c0b1930 2024-05-22T21:13:46.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.578+0000 7f8472000640 1 --2- 172.21.15.161:0/2885905685 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f845c0666c0 0x7f845c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:46.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.578+0000 7f8472000640 1 -- 172.21.15.161:0/2885905685 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8474083f40 con 0x7f847c0b1930 2024-05-22T21:13:46.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.578+0000 7f8479e00640 1 --2- 172.21.15.161:0/2885905685 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f845c0666c0 0x7f845c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:46.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.582+0000 7f8479e00640 1 --2- 172.21.15.161:0/2885905685 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f845c0666c0 0x7f845c068b80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f846c0b1000 tx=0x7f846c007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:46.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.582+0000 7f8472000640 1 -- 172.21.15.161:0/2885905685 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f847c15b250 con 0x7f847c0b1930 2024-05-22T21:13:46.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.770+0000 7f8480a00640 1 -- 172.21.15.161:0/2885905685 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f847c15b250 con 0x7f847c0b1930 2024-05-22T21:13:46.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.770+0000 7f8472000640 1 -- 172.21.15.161:0/2885905685 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f847c15b250 con 0x7f847c0b1930 2024-05-22T21:13:46.774 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:13:46.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.770+0000 7f8480a00640 1 -- 172.21.15.161:0/2885905685 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f845c0666c0 msgr2=0x7f845c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:46.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.770+0000 7f8480a00640 1 --2- 172.21.15.161:0/2885905685 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f845c0666c0 0x7f845c068b80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f846c0b1000 tx=0x7f846c007040 comp rx=0 tx=0).stop 2024-05-22T21:13:46.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.770+0000 7f8480a00640 1 -- 172.21.15.161:0/2885905685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f847c0b1930 msgr2=0x7f847c0b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:46.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.770+0000 7f8480a00640 1 --2- 172.21.15.161:0/2885905685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f847c0b1930 0x7f847c0b1db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f847c073ce0 tx=0x7f8474011860 comp rx=0 tx=0).stop 2024-05-22T21:13:46.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.770+0000 7f8480a00640 1 -- 172.21.15.161:0/2885905685 shutdown_connections 2024-05-22T21:13:46.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.770+0000 7f8480a00640 1 --2- 172.21.15.161:0/2885905685 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f845c0666c0 0x7f845c068b80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:46.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.770+0000 7f8480a00640 1 --2- 172.21.15.161:0/2885905685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f847c0b1930 0x7f847c0b1db0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:46.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.770+0000 7f8480a00640 1 --2- 172.21.15.161:0/2885905685 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c0b3f10 0x7f847c0ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:46.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.770+0000 7f8480a00640 1 --2- 172.21.15.161:0/2885905685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8460003980 0x7f847c14ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:46.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.770+0000 7f8480a00640 1 -- 172.21.15.161:0/2885905685 >> 172.21.15.161:0/2885905685 conn(0x7f847c0b0bc0 msgr2=0x7f847c0b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:46.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.770+0000 7f8480a00640 1 -- 172.21.15.161:0/2885905685 shutdown_connections 2024-05-22T21:13:46.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:46.770+0000 7f8480a00640 1 -- 172.21.15.161:0/2885905685 wait complete. 2024-05-22T21:13:46.795 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-22T21:13:46.795 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:13:46.795 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-22T21:13:46.795 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-22T21:13:46.795 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-22T21:13:46.795 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-22T21:13:46.795 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-22T21:13:46.796 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:13:46.796 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:13:46.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:46.796 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-22T21:13:46.806 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:13:46.806 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:13:46.806 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:13:46.806 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:46.807 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-22T21:13:46.807 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:46.807 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:13:46.807 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:46.807 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f0 2024-05-22T21:13:46.807 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:13:46.807 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:46.807 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f1 2024-05-22T21:13:46.807 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:13:46.808 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:46.808 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:13:46.808 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:46.808 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:13:46.808 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42458sec preferred_lft 42458sec 2024-05-22T21:13:46.808 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-22T21:13:46.808 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:46.808 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:13:46.809 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:bd:4e:25:3d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:46.809 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:13:46.809 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:46.810 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-22T21:13:46.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:46.811 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:13:46.811 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:13:46.811 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link add name ceph-brx type bridge 2024-05-22T21:13:46.811 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip addr flush dev ceph-brx 2024-05-22T21:13:46.811 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set ceph-brx up 2024-05-22T21:13:46.811 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-22T21:13:46.811 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:13:46.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:46.933 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:13:46.952 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-22T21:13:46.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:46.955 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-22T21:13:47.308 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-22T21:13:47.308 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:13:47.308 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:13:47.308 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:13:47.309 INFO:teuthology.orchestra.run.smithi179.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:13:47.309 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:13:47.309 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:13:47.309 INFO:teuthology.orchestra.run.smithi179.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:13:47.309 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:13:47.309 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:13:47.309 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:13:47.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:47.309 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:13:47.309 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:13:47.310 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:13:47.310 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:13:47.310 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:13:47.310 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:13:47.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:47.361 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-22T21:13:47.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:47.411 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list-id 2024-05-22T21:13:47.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:47.462 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:13:47.462 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:13:47.463 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-22T21:13:47.463 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-22T21:13:47.463 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:13:47.547 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-22T21:13:47.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:47.547 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:13:47.547 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:13:47.547 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-22T21:13:47.548 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-22T21:13:47.548 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-22T21:13:47.548 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-22T21:13:47.548 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-22T21:13:47.548 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:13:47.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:47.736 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:13:47.736 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:13:47.736 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-22T21:13:47.736 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set dev brx.0 master ceph-brx 2024-05-22T21:13:47.736 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:13:47.775 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:13:47.775 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-22T21:13:47.825 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:13:47.826 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:13:47.826 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:13:47.877 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:13:47.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:47.878 DEBUG:teuthology.orchestra.run.smithi179:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:13:47.970 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:13:47.970 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:13:47.970 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:13:48.550 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:13:48.551 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:13:48.551 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:13:48.551 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:13:48.551 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:13:48.561 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:13:48.561 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:13:48.562 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:13:48.562 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:13:48.562 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:13:48.562 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:13:48.562 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:13:48.562 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:13:48.565 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:13:48.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:48.566 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:13:48.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:48.585 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:13:48.666 INFO:teuthology.orchestra.run.smithi179.stdout:client4523 2024-05-22T21:13:48.666 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:48.667 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:13:48.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:48.722 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:13:48.776 INFO:teuthology.orchestra.run.smithi179.stdout:client4523 2024-05-22T21:13:48.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:48.776 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:13:48.828 INFO:teuthology.orchestra.run.smithi179.stdout:client4523 2024-05-22T21:13:48.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:48.828 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d6fa8528-b1bf-41b6-9f76-c0d715394087.client4523/status') 2024-05-22T21:13:48.890 INFO:teuthology.orchestra.run.smithi179.stdout:instance: client.4523 (3)192.168.0.1:0/213480485 2024-05-22T21:13:48.890 INFO:teuthology.orchestra.run.smithi179.stdout:blocklisted: false 2024-05-22T21:13:48.891 INFO:teuthology.orchestra.run.smithi179.stderr:0+1 records in 2024-05-22T21:13:48.891 INFO:teuthology.orchestra.run.smithi179.stderr:0+1 records out 2024-05-22T21:13:48.892 INFO:teuthology.orchestra.run.smithi179.stderr:68 bytes copied, 5.2575e-05 s, 1.3 MB/s 2024-05-22T21:13:48.892 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-05-22T21:13:48.892 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-22T21:13:48.892 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-22T21:13:48.892 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:13:49.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.034+0000 7f1bef600640 1 Processor -- start 2024-05-22T21:13:49.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.034+0000 7f1bef600640 1 -- start start 2024-05-22T21:13:49.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.034+0000 7f1bef600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1be815dd50 0x7f1be806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:49.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.034+0000 7f1bef600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be8071670 con 0x7f1be815d5e0 2024-05-22T21:13:49.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.034+0000 7f1bef600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be806de40 con 0x7f1be815e130 2024-05-22T21:13:49.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.034+0000 7f1bef600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be8070a60 con 0x7f1be815dd50 2024-05-22T21:13:49.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.034+0000 7f1bed600640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1be815d5e0 0x7f1be815b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51988/0 (socket says 172.21.15.161:51988) 2024-05-22T21:13:49.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.034+0000 7f1bed600640 1 -- 172.21.15.161:0/3270769317 learned_addr learned my addr 172.21.15.161:0/3270769317 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:49.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.034+0000 7f1becc00640 1 --2- 172.21.15.161:0/3270769317 >> v2:172.21.15.179:3300/0 conn(0x7f1be815dd50 0x7f1be806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:49.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.038+0000 7f1be7400640 1 -- 172.21.15.161:0/3270769317 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800035379 0 0) 0x7f1be8071670 con 0x7f1be815d5e0 2024-05-22T21:13:49.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.038+0000 7f1be7400640 1 -- 172.21.15.161:0/3270769317 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bcc003530 con 0x7f1be815d5e0 2024-05-22T21:13:49.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.038+0000 7f1becc00640 1 -- 172.21.15.161:0/3270769317 >> v1:172.21.15.179:6789/0 conn(0x7f1be815e130 legacy=0x7f1be815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:49.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.038+0000 7f1becc00640 1 -- 172.21.15.161:0/3270769317 >> v1:172.21.15.161:6789/0 conn(0x7f1be815d5e0 legacy=0x7f1be815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:49.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.038+0000 7f1becc00640 1 -- 172.21.15.161:0/3270769317 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be8070d40 con 0x7f1be815dd50 2024-05-22T21:13:49.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.038+0000 7f1becc00640 1 --2- 172.21.15.161:0/3270769317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be815dd50 0x7f1be806d900 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009d90 tx=0x7f1bd801c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd17868bf2a9fbbe server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:49.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.038+0000 7f1be7400640 1 -- 172.21.15.161:0/3270769317 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80a5a00 con 0x7f1be815dd50 2024-05-22T21:13:49.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.038+0000 7f1be7400640 1 -- 172.21.15.161:0/3270769317 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd80a5b60 con 0x7f1be815dd50 2024-05-22T21:13:49.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.038+0000 7f1be7400640 1 -- 172.21.15.161:0/3270769317 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80ad440 con 0x7f1be815dd50 2024-05-22T21:13:49.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1bef600640 1 -- 172.21.15.161:0/3270769317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be815dd50 msgr2=0x7f1be806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:49.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1bef600640 1 --2- 172.21.15.161:0/3270769317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be815dd50 0x7f1be806d900 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009d90 tx=0x7f1bd801c940 comp rx=0 tx=0).stop 2024-05-22T21:13:49.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1bef600640 1 -- 172.21.15.161:0/3270769317 shutdown_connections 2024-05-22T21:13:49.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1bef600640 1 --2- 172.21.15.161:0/3270769317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be815dd50 0x7f1be806d900 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:49.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1bef600640 1 -- 172.21.15.161:0/3270769317 >> 172.21.15.161:0/3270769317 conn(0x7f1be80b0bc0 msgr2=0x7f1be80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:49.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1bef600640 1 -- 172.21.15.161:0/3270769317 shutdown_connections 2024-05-22T21:13:49.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1bef600640 1 -- 172.21.15.161:0/3270769317 wait complete. 2024-05-22T21:13:49.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1bef600640 1 Processor -- start 2024-05-22T21:13:49.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1bef600640 1 -- start start 2024-05-22T21:13:49.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1bef600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be815d5e0 0x7f1be80b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:49.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1bef600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1be815dd50 0x7f1be8142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:49.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1bef600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1be815e130 0x7f1be814f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:49.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1bef600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be80af5e0 con 0x7f1be815dd50 2024-05-22T21:13:49.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1bef600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be80afdf0 con 0x7f1be815d5e0 2024-05-22T21:13:49.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1bef600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be80af830 con 0x7f1be815e130 2024-05-22T21:13:49.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1be7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1be815dd50 0x7f1be8142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:49.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1becc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be815d5e0 0x7f1be80b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:49.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1be7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1be815dd50 0x7f1be8142db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46768/0 (socket says 172.21.15.161:46768) 2024-05-22T21:13:49.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1be7e00640 1 -- 172.21.15.161:0/2809287682 learned_addr learned my addr 172.21.15.161:0/2809287682 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:49.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1bed600640 1 --2- 172.21.15.161:0/2809287682 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1be815e130 0x7f1be814f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:49.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1be7e00640 1 -- 172.21.15.161:0/2809287682 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1be815e130 msgr2=0x7f1be814f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:49.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1be7e00640 1 --2- 172.21.15.161:0/2809287682 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1be815e130 0x7f1be814f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:49.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1be7e00640 1 -- 172.21.15.161:0/2809287682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be815d5e0 msgr2=0x7f1be80b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:49.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1be7e00640 1 --2- 172.21.15.161:0/2809287682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be815d5e0 0x7f1be80b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:49.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1be7e00640 1 -- 172.21.15.161:0/2809287682 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd8009a70 con 0x7f1be815dd50 2024-05-22T21:13:49.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.042+0000 7f1be7e00640 1 --2- 172.21.15.161:0/2809287682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1be815dd50 0x7f1be8142db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400c3c0 tx=0x7f1bd400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:49.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.046+0000 7f1be4c00640 1 -- 172.21.15.161:0/2809287682 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd40052d0 con 0x7f1be815dd50 2024-05-22T21:13:49.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.046+0000 7f1be4c00640 1 -- 172.21.15.161:0/2809287682 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd4005430 con 0x7f1be815dd50 2024-05-22T21:13:49.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.046+0000 7f1be4c00640 1 -- 172.21.15.161:0/2809287682 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4003840 con 0x7f1be815dd50 2024-05-22T21:13:49.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.046+0000 7f1bef600640 1 -- 172.21.15.161:0/2809287682 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be80afb10 con 0x7f1be815dd50 2024-05-22T21:13:49.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.046+0000 7f1bef600640 1 -- 172.21.15.161:0/2809287682 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be8143b70 con 0x7f1be815dd50 2024-05-22T21:13:49.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.046+0000 7f1bef600640 1 -- 172.21.15.161:0/2809287682 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be80706b0 con 0x7f1be815dd50 2024-05-22T21:13:49.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.046+0000 7f1be4c00640 1 -- 172.21.15.161:0/2809287682 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1bd40039a0 con 0x7f1be815dd50 2024-05-22T21:13:49.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.046+0000 7f1be4c00640 1 --2- 172.21.15.161:0/2809287682 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bc8066690 0x7f1bc8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:49.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.046+0000 7f1be4c00640 1 -- 172.21.15.161:0/2809287682 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bd4083620 con 0x7f1be815dd50 2024-05-22T21:13:49.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.046+0000 7f1becc00640 1 --2- 172.21.15.161:0/2809287682 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bc8066690 0x7f1bc8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:49.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.050+0000 7f1becc00640 1 --2- 172.21.15.161:0/2809287682 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bc8066690 0x7f1bc8068b50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80af000 tx=0x7f1bd80a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:49.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.050+0000 7f1be4c00640 1 -- 172.21.15.161:0/2809287682 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1bd4050a30 con 0x7f1be815dd50 2024-05-22T21:13:49.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.238+0000 7f1bef600640 1 -- 172.21.15.161:0/2809287682 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1be8067220 con 0x7f1be815dd50 2024-05-22T21:13:49.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.238+0000 7f1be4c00640 1 -- 172.21.15.161:0/2809287682 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1bd4054050 con 0x7f1be815dd50 2024-05-22T21:13:49.244 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:13:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.242+0000 7f1bef600640 1 -- 172.21.15.161:0/2809287682 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bc8066690 msgr2=0x7f1bc8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.242+0000 7f1bef600640 1 --2- 172.21.15.161:0/2809287682 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bc8066690 0x7f1bc8068b50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80af000 tx=0x7f1bd80a7070 comp rx=0 tx=0).stop 2024-05-22T21:13:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.242+0000 7f1bef600640 1 -- 172.21.15.161:0/2809287682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1be815dd50 msgr2=0x7f1be8142db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.242+0000 7f1bef600640 1 --2- 172.21.15.161:0/2809287682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1be815dd50 0x7f1be8142db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400c3c0 tx=0x7f1bd400c880 comp rx=0 tx=0).stop 2024-05-22T21:13:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.242+0000 7f1bef600640 1 -- 172.21.15.161:0/2809287682 shutdown_connections 2024-05-22T21:13:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.242+0000 7f1bef600640 1 --2- 172.21.15.161:0/2809287682 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bc8066690 0x7f1bc8068b50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.242+0000 7f1bef600640 1 --2- 172.21.15.161:0/2809287682 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1be815e130 0x7f1be814f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.242+0000 7f1bef600640 1 --2- 172.21.15.161:0/2809287682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1be815dd50 0x7f1be8142db0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.242+0000 7f1bef600640 1 --2- 172.21.15.161:0/2809287682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be815d5e0 0x7f1be80b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.242+0000 7f1bef600640 1 -- 172.21.15.161:0/2809287682 >> 172.21.15.161:0/2809287682 conn(0x7f1be80b0bc0 msgr2=0x7f1be806d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.242+0000 7f1bef600640 1 -- 172.21.15.161:0/2809287682 shutdown_connections 2024-05-22T21:13:49.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:49.242+0000 7f1bef600640 1 -- 172.21.15.161:0/2809287682 wait complete. 2024-05-22T21:13:49.265 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-22T21:13:49.265 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:13:49.265 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-22T21:13:49.265 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-22T21:13:49.265 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-22T21:13:49.265 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-22T21:13:49.265 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-22T21:13:49.266 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:13:49.266 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-22T21:13:49.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:49.266 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-22T21:13:49.275 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:13:49.275 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:13:49.275 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:13:49.275 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:49.275 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-22T21:13:49.275 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:49.275 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:13:49.276 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:49.276 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f0 2024-05-22T21:13:49.276 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:13:49.276 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:49.276 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f1 2024-05-22T21:13:49.276 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:13:49.276 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:49.276 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:13:49.276 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:49.276 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:13:49.277 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42455sec preferred_lft 42455sec 2024-05-22T21:13:49.277 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-22T21:13:49.277 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:49.277 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:13:49.277 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:bd:4e:25:3d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:49.277 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:13:49.277 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:49.277 INFO:teuthology.orchestra.run.smithi179.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:13:49.277 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:13:49.277 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:13:49.277 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:49.277 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:13:49.277 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:49.278 INFO:teuthology.orchestra.run.smithi179.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:13:49.278 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-22T21:13:49.278 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-05-22T21:13:49.278 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:49.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:49.278 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:13:49.335 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-22T21:13:49.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:49.337 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-22T21:13:49.388 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-22T21:13:49.388 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:13:49.388 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:13:49.389 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:13:49.389 INFO:teuthology.orchestra.run.smithi179.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:13:49.389 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:13:49.389 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:13:49.389 INFO:teuthology.orchestra.run.smithi179.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:13:49.389 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:13:49.389 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:13:49.389 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:13:49.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:49.389 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:13:49.389 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:13:49.390 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:13:49.390 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:13:49.390 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:13:49.390 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:13:49.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:49.478 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-22T21:13:49.486 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-22T21:13:49.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:49.487 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list-id 2024-05-22T21:13:49.537 INFO:teuthology.orchestra.run.smithi179.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-22T21:13:49.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:49.538 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:13:49.538 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:13:49.538 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-22T21:13:49.538 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-22T21:13:49.538 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:13:49.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:49.619 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-22T21:13:49.681 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:13:49.681 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:13:49.681 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:13:49.681 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:49.682 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-22T21:13:49.682 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:49.682 INFO:teuthology.orchestra.run.smithi179.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:13:49.682 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether c6:3c:98:1c:f5:a5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-22T21:13:49.682 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-22T21:13:49.682 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:49.682 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c43c:98ff:fe1c:f5a5/64 scope link 2024-05-22T21:13:49.682 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:49.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:49.683 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-22T21:13:49.745 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:13:49.745 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:13:49.745 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:13:49.745 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:49.745 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-22T21:13:49.745 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:49.745 INFO:teuthology.orchestra.run.smithi179.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:13:49.745 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether c6:3c:98:1c:f5:a5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-22T21:13:49.745 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-22T21:13:49.746 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:49.746 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c43c:98ff:fe1c:f5a5/64 scope link 2024-05-22T21:13:49.746 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:13:49.747 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-22T21:13:49.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:49.747 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:13:49.747 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:13:49.747 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-22T21:13:49.747 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-22T21:13:49.747 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-22T21:13:49.747 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-22T21:13:49.748 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-22T21:13:49.748 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:13:49.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:49.882 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:13:49.882 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:13:49.882 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.1 up 2024-05-22T21:13:49.882 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set dev brx.1 master ceph-brx 2024-05-22T21:13:49.882 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:13:49.920 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:13:49.921 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-22T21:13:49.969 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-22T21:13:49.970 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:13:49.970 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-22T21:13:50.024 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:13:50.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:50.025 DEBUG:teuthology.orchestra.run.smithi179:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-22T21:13:50.116 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-22T21:13:50.117 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-22T21:13:50.117 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:13:50.131 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-22T21:13:50.131 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-22T21:13:50.131 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:13:50.132 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:13:50.132 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:13:50.139 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-22T21:13:50.139 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-22T21:13:50.140 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:13:50.140 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:13:50.140 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:13:50.140 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:13:50.140 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:13:50.140 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:13:50.143 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:13:50.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:50.144 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-22T21:13:50.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:50.163 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:13:50.216 INFO:teuthology.orchestra.run.smithi179.stdout:client4686 2024-05-22T21:13:50.216 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:50.216 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:13:50.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:50.266 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:13:50.324 INFO:teuthology.orchestra.run.smithi179.stdout:client4686 2024-05-22T21:13:50.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:50.324 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:13:50.375 INFO:teuthology.orchestra.run.smithi179.stdout:client4686 2024-05-22T21:13:50.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:50.376 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d6fa8528-b1bf-41b6-9f76-c0d715394087.client4686/status') 2024-05-22T21:13:50.436 INFO:teuthology.orchestra.run.smithi179.stdout:instance: client.4686 (3)172.21.15.179:0/2550674302 2024-05-22T21:13:50.436 INFO:teuthology.orchestra.run.smithi179.stdout:blocklisted: false 2024-05-22T21:13:50.436 INFO:teuthology.orchestra.run.smithi179.stderr:0+1 records in 2024-05-22T21:13:50.437 INFO:teuthology.orchestra.run.smithi179.stderr:0+1 records out 2024-05-22T21:13:50.437 INFO:teuthology.orchestra.run.smithi179.stderr:71 bytes copied, 4.8531e-05 s, 1.5 MB/s 2024-05-22T21:13:50.439 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-22T21:13:50.462 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:50.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.598+0000 7f62b1e00640 1 Processor -- start 2024-05-22T21:13:50.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.598+0000 7f62b1e00640 1 -- start start 2024-05-22T21:13:50.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.598+0000 7f62b1e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62ac15d040 0x7f62ac0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:50.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.598+0000 7f62b1e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f62ac15e430 0x7f62ac0cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:50.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.598+0000 7f62b1e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ac0b13d0 con 0x7f62ac0d0cf0 2024-05-22T21:13:50.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.598+0000 7f62b1e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ac0b0aa0 con 0x7f62ac15e430 2024-05-22T21:13:50.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.598+0000 7f62b1e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0b0d80 con 0x7f62ac15d040 2024-05-22T21:13:50.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.598+0000 7f62b1400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f62ac15e430 0x7f62ac0cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:50.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.598+0000 7f62b1400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f62ac15e430 0x7f62ac0cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46780/0 (socket says 172.21.15.161:46780) 2024-05-22T21:13:50.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.598+0000 7f62b1400640 1 -- 172.21.15.161:0/4106451060 learned_addr learned my addr 172.21.15.161:0/4106451060 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:50.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.598+0000 7f62b0a00640 1 --2- 172.21.15.161:0/4106451060 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62ac15d040 0x7f62ac0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:50.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.598+0000 7f62b1400640 1 -- 172.21.15.161:0/4106451060 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62ac15d040 msgr2=0x7f62ac0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:50.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.598+0000 7f62b1400640 1 --2- 172.21.15.161:0/4106451060 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62ac15d040 0x7f62ac0b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:50.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.598+0000 7f62b1400640 1 -- 172.21.15.161:0/4106451060 >> v1:172.21.15.179:6789/0 conn(0x7f62ac0d0cf0 legacy=0x7f62ac0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:50.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.598+0000 7f62b1400640 1 -- 172.21.15.161:0/4106451060 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac0b1060 con 0x7f62ac15e430 2024-05-22T21:13:50.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.598+0000 7f62b0a00640 1 --2- 172.21.15.161:0/4106451060 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62ac15d040 0x7f62ac0b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:50.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.602+0000 7f62b1400640 1 --2- 172.21.15.161:0/4106451060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62ac15e430 0x7f62ac0cdf50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009e50 tx=0x7f62a401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e42387be23a307 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:50.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.602+0000 7f62abe00640 1 -- 172.21.15.161:0/4106451060 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0d16f0 con 0x7f62ac15e430 2024-05-22T21:13:50.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.602+0000 7f62abe00640 1 -- 172.21.15.161:0/4106451060 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a40a5820 con 0x7f62ac15e430 2024-05-22T21:13:50.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.602+0000 7f62abe00640 1 -- 172.21.15.161:0/4106451060 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a40a5bb0 con 0x7f62ac15e430 2024-05-22T21:13:50.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.602+0000 7f62b1e00640 1 -- 172.21.15.161:0/4106451060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62ac15e430 msgr2=0x7f62ac0cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:50.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.602+0000 7f62b1e00640 1 --2- 172.21.15.161:0/4106451060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62ac15e430 0x7f62ac0cdf50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009e50 tx=0x7f62a401ca30 comp rx=0 tx=0).stop 2024-05-22T21:13:50.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.602+0000 7f62b1e00640 1 -- 172.21.15.161:0/4106451060 shutdown_connections 2024-05-22T21:13:50.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.602+0000 7f62b1e00640 1 --2- 172.21.15.161:0/4106451060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62ac15e430 0x7f62ac0cdf50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:50.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.602+0000 7f62b1e00640 1 --2- 172.21.15.161:0/4106451060 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62ac15d040 0x7f62ac0b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:50.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.602+0000 7f62b1e00640 1 -- 172.21.15.161:0/4106451060 >> 172.21.15.161:0/4106451060 conn(0x7f62ac099a40 msgr2=0x7f62ac099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:50.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.602+0000 7f62b1e00640 1 -- 172.21.15.161:0/4106451060 shutdown_connections 2024-05-22T21:13:50.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.602+0000 7f62b1e00640 1 -- 172.21.15.161:0/4106451060 wait complete. 2024-05-22T21:13:50.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.602+0000 7f62b1e00640 1 Processor -- start 2024-05-22T21:13:50.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.602+0000 7f62b1e00640 1 -- start start 2024-05-22T21:13:50.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62ac0d0cf0 0x7f62ac15a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:50.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62ac15d040 0x7f62ac0c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:50.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62ac15e430 0x7f62ac0b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:50.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac142920 con 0x7f62ac15e430 2024-05-22T21:13:50.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac141d10 con 0x7f62ac0d0cf0 2024-05-22T21:13:50.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac141ff0 con 0x7f62ac15d040 2024-05-22T21:13:50.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62ac15e430 0x7f62ac0b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:50.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62ac15e430 0x7f62ac0b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46790/0 (socket says 172.21.15.161:46790) 2024-05-22T21:13:50.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b0a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62ac0d0cf0 0x7f62ac15a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:50.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1400640 1 -- 172.21.15.161:0/434160768 learned_addr learned my addr 172.21.15.161:0/434160768 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:50.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62a3e00640 1 --2- 172.21.15.161:0/434160768 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62ac15d040 0x7f62ac0c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:50.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1400640 1 -- 172.21.15.161:0/434160768 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62ac15d040 msgr2=0x7f62ac0c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:50.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1400640 1 --2- 172.21.15.161:0/434160768 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62ac15d040 0x7f62ac0c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:50.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1400640 1 -- 172.21.15.161:0/434160768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62ac0d0cf0 msgr2=0x7f62ac15a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:50.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1400640 1 --2- 172.21.15.161:0/434160768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62ac0d0cf0 0x7f62ac15a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:50.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1400640 1 -- 172.21.15.161:0/434160768 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a4009b30 con 0x7f62ac15e430 2024-05-22T21:13:50.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b0a00640 1 --2- 172.21.15.161:0/434160768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62ac0d0cf0 0x7f62ac15a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:50.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1400640 1 --2- 172.21.15.161:0/434160768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62ac15e430 0x7f62ac0b2860 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f62a401a660 tx=0x7f62a40197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:50.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62a9600640 1 -- 172.21.15.161:0/434160768 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a40b8640 con 0x7f62ac15e430 2024-05-22T21:13:50.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62a9600640 1 -- 172.21.15.161:0/434160768 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a40b87c0 con 0x7f62ac15e430 2024-05-22T21:13:50.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62a9600640 1 -- 172.21.15.161:0/434160768 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a40b8b70 con 0x7f62ac15e430 2024-05-22T21:13:50.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1e00640 1 -- 172.21.15.161:0/434160768 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ac06bd00 con 0x7f62ac15e430 2024-05-22T21:13:50.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1e00640 1 -- 172.21.15.161:0/434160768 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ac1425b0 con 0x7f62ac15e430 2024-05-22T21:13:50.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62b1e00640 1 -- 172.21.15.161:0/434160768 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ac142a90 con 0x7f62ac15e430 2024-05-22T21:13:50.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.606+0000 7f62a9600640 1 -- 172.21.15.161:0/434160768 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f62a40b8cd0 con 0x7f62ac15e430 2024-05-22T21:13:50.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.610+0000 7f62a9600640 1 --2- 172.21.15.161:0/434160768 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6288066690 0x7f6288068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:50.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.610+0000 7f62a9600640 1 -- 172.21.15.161:0/434160768 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62a411f4b0 con 0x7f62ac15e430 2024-05-22T21:13:50.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.610+0000 7f62b0a00640 1 --2- 172.21.15.161:0/434160768 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6288066690 0x7f6288068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:50.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.610+0000 7f62b0a00640 1 --2- 172.21.15.161:0/434160768 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6288066690 0x7f6288068b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f62ac06fa80 tx=0x7f629c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:50.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.614+0000 7f62a9600640 1 -- 172.21.15.161:0/434160768 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f62a40ec9e0 con 0x7f62ac15e430 2024-05-22T21:13:50.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.766+0000 7f62b1e00640 1 -- 172.21.15.161:0/434160768 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62ac067070 con 0x7f62ac15e430 2024-05-22T21:13:50.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.770+0000 7f62a9600640 1 -- 172.21.15.161:0/434160768 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f62a40f0000 con 0x7f62ac15e430 2024-05-22T21:13:50.772 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:50.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.770+0000 7f62b1e00640 1 -- 172.21.15.161:0/434160768 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6288066690 msgr2=0x7f6288068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:50.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.770+0000 7f62b1e00640 1 --2- 172.21.15.161:0/434160768 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6288066690 0x7f6288068b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f62ac06fa80 tx=0x7f629c008040 comp rx=0 tx=0).stop 2024-05-22T21:13:50.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.770+0000 7f62b1e00640 1 -- 172.21.15.161:0/434160768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62ac15e430 msgr2=0x7f62ac0b2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:50.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.770+0000 7f62b1e00640 1 --2- 172.21.15.161:0/434160768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62ac15e430 0x7f62ac0b2860 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f62a401a660 tx=0x7f62a40197d0 comp rx=0 tx=0).stop 2024-05-22T21:13:50.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.770+0000 7f62b1e00640 1 -- 172.21.15.161:0/434160768 shutdown_connections 2024-05-22T21:13:50.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.770+0000 7f62b1e00640 1 --2- 172.21.15.161:0/434160768 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6288066690 0x7f6288068b50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:50.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.770+0000 7f62b1e00640 1 --2- 172.21.15.161:0/434160768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62ac15e430 0x7f62ac0b2860 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:50.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.770+0000 7f62b1e00640 1 --2- 172.21.15.161:0/434160768 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62ac15d040 0x7f62ac0c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:50.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.770+0000 7f62b1e00640 1 --2- 172.21.15.161:0/434160768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62ac0d0cf0 0x7f62ac15a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:50.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.770+0000 7f62b1e00640 1 -- 172.21.15.161:0/434160768 >> 172.21.15.161:0/434160768 conn(0x7f62ac099a40 msgr2=0x7f62ac1595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:50.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.770+0000 7f62b1e00640 1 -- 172.21.15.161:0/434160768 shutdown_connections 2024-05-22T21:13:50.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.770+0000 7f62b1e00640 1 -- 172.21.15.161:0/434160768 wait complete. 2024-05-22T21:13:50.793 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:13:21.237503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:13:22.425891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:13:50.794 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:13:50.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.926+0000 7fcbd9800640 1 Processor -- start 2024-05-22T21:13:50.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.926+0000 7fcbd9800640 1 -- start start 2024-05-22T21:13:50.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.930+0000 7fcbd9800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcbd413cfc0 0x7fcbd41402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:50.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.930+0000 7fcbd9800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd413d3a0 con 0x7fcbd413fe50 2024-05-22T21:13:50.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.930+0000 7fcbd9800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd40cd920 con 0x7fcbd415af60 2024-05-22T21:13:50.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.930+0000 7fcbd9800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40cdc00 con 0x7fcbd413cfc0 2024-05-22T21:13:50.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.930+0000 7fcbd3400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fcbd413fe50 0x7fcbd406fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:52004/0 (socket says 172.21.15.161:52004) 2024-05-22T21:13:50.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.930+0000 7fcbd3400640 1 -- 172.21.15.161:0/63983097 learned_addr learned my addr 172.21.15.161:0/63983097 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:50.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.930+0000 7fcbd2a00640 1 --2- 172.21.15.161:0/63983097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcbd413cfc0 0x7fcbd41402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:50.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.930+0000 7fcbd2000640 1 -- 172.21.15.161:0/63983097 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283339656 0 0) 0x7fcbd413d3a0 con 0x7fcbd413fe50 2024-05-22T21:13:50.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.930+0000 7fcbd2000640 1 -- 172.21.15.161:0/63983097 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbb8005740 con 0x7fcbd413fe50 2024-05-22T21:13:50.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.930+0000 7fcbd2a00640 1 -- 172.21.15.161:0/63983097 >> v1:172.21.15.179:6789/0 conn(0x7fcbd415af60 legacy=0x7fcbd413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:50.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.930+0000 7fcbd2000640 1 -- 172.21.15.161:0/63983097 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1319546854 0 0) 0x7fcbb8005740 con 0x7fcbd413fe50 2024-05-22T21:13:50.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.930+0000 7fcbd2a00640 1 -- 172.21.15.161:0/63983097 >> v1:172.21.15.161:6789/0 conn(0x7fcbd413fe50 legacy=0x7fcbd406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:50.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.930+0000 7fcbd2a00640 1 -- 172.21.15.161:0/63983097 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd40cdee0 con 0x7fcbd413cfc0 2024-05-22T21:13:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.934+0000 7fcbd2a00640 1 --2- 172.21.15.161:0/63983097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcbd413cfc0 0x7fcbd41402e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40097f0 tx=0x7fcbc401c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2414df2f1c728a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.934+0000 7fcbd2000640 1 -- 172.21.15.161:0/63983097 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc001a00 con 0x7fcbd413cfc0 2024-05-22T21:13:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.934+0000 7fcbd2000640 1 -- 172.21.15.161:0/63983097 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc40a7050 con 0x7fcbd413cfc0 2024-05-22T21:13:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.934+0000 7fcbd2000640 1 -- 172.21.15.161:0/63983097 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc40ab400 con 0x7fcbd413cfc0 2024-05-22T21:13:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.934+0000 7fcbd9800640 1 -- 172.21.15.161:0/63983097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcbd413cfc0 msgr2=0x7fcbd41402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:50.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.934+0000 7fcbd9800640 1 --2- 172.21.15.161:0/63983097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcbd413cfc0 0x7fcbd41402e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40097f0 tx=0x7fcbc401c340 comp rx=0 tx=0).stop 2024-05-22T21:13:50.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.934+0000 7fcbd9800640 1 -- 172.21.15.161:0/63983097 shutdown_connections 2024-05-22T21:13:50.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.934+0000 7fcbd9800640 1 --2- 172.21.15.161:0/63983097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcbd413cfc0 0x7fcbd41402e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:50.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.934+0000 7fcbd9800640 1 -- 172.21.15.161:0/63983097 >> 172.21.15.161:0/63983097 conn(0x7fcbd406a2d0 msgr2=0x7fcbd406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:50.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.934+0000 7fcbd9800640 1 -- 172.21.15.161:0/63983097 shutdown_connections 2024-05-22T21:13:50.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.934+0000 7fcbd9800640 1 -- 172.21.15.161:0/63983097 wait complete. 2024-05-22T21:13:50.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.934+0000 7fcbd9800640 1 Processor -- start 2024-05-22T21:13:50.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.934+0000 7fcbd9800640 1 -- start start 2024-05-22T21:13:50.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbd9800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcbd413cfc0 0x7fcbd41578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:50.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbd9800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbd413fe50 0x7fcbd41426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:50.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbd9800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd415af60 0x7fcbd4158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:50.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbd9800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd413dcf0 con 0x7fcbd413fe50 2024-05-22T21:13:50.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbd9800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd413faa0 con 0x7fcbd415af60 2024-05-22T21:13:50.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbd9800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd413d6e0 con 0x7fcbd413cfc0 2024-05-22T21:13:50.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbca000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbd413fe50 0x7fcbd41426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:50.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbca000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbd413fe50 0x7fcbd41426e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46804/0 (socket says 172.21.15.161:46804) 2024-05-22T21:13:50.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbca000640 1 -- 172.21.15.161:0/3963598662 learned_addr learned my addr 172.21.15.161:0/3963598662 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:50.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbd2a00640 1 --2- 172.21.15.161:0/3963598662 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcbd413cfc0 0x7fcbd41578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:50.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbd3400640 1 --2- 172.21.15.161:0/3963598662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd415af60 0x7fcbd4158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:50.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbca000640 1 -- 172.21.15.161:0/3963598662 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcbd413cfc0 msgr2=0x7fcbd41578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:50.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbca000640 1 --2- 172.21.15.161:0/3963598662 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcbd413cfc0 0x7fcbd41578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:50.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbca000640 1 -- 172.21.15.161:0/3963598662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd415af60 msgr2=0x7fcbd4158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:50.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbca000640 1 --2- 172.21.15.161:0/3963598662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd415af60 0x7fcbd4158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:50.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbca000640 1 -- 172.21.15.161:0/3963598662 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc4009420 con 0x7fcbd413fe50 2024-05-22T21:13:50.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbca000640 1 --2- 172.21.15.161:0/3963598662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbd413fe50 0x7fcbd41426e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00c3c0 tx=0x7fcbbc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:50.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbcb400640 1 -- 172.21.15.161:0/3963598662 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc0052d0 con 0x7fcbd413fe50 2024-05-22T21:13:50.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbcb400640 1 -- 172.21.15.161:0/3963598662 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc005430 con 0x7fcbd413fe50 2024-05-22T21:13:50.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbcb400640 1 -- 172.21.15.161:0/3963598662 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc003840 con 0x7fcbd413fe50 2024-05-22T21:13:50.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbd9800640 1 -- 172.21.15.161:0/3963598662 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd4157df0 con 0x7fcbd413fe50 2024-05-22T21:13:50.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbd9800640 1 -- 172.21.15.161:0/3963598662 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd413f7c0 con 0x7fcbd413fe50 2024-05-22T21:13:50.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.938+0000 7fcbd9800640 1 -- 172.21.15.161:0/3963598662 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd413de60 con 0x7fcbd413fe50 2024-05-22T21:13:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.942+0000 7fcbcb400640 1 -- 172.21.15.161:0/3963598662 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcbbc0039a0 con 0x7fcbd413fe50 2024-05-22T21:13:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.942+0000 7fcbcb400640 1 --2- 172.21.15.161:0/3963598662 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcbb0066690 0x7fcbb0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.942+0000 7fcbcb400640 1 -- 172.21.15.161:0/3963598662 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbbc083630 con 0x7fcbd413fe50 2024-05-22T21:13:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.942+0000 7fcbd2a00640 1 --2- 172.21.15.161:0/3963598662 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcbb0066690 0x7fcbb0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.942+0000 7fcbd2a00640 1 --2- 172.21.15.161:0/3963598662 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcbb0066690 0x7fcbb0068b50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40ad000 tx=0x7fcbc40a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:50.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:50.946+0000 7fcbcb400640 1 -- 172.21.15.161:0/3963598662 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcbbc050a40 con 0x7fcbd413fe50 2024-05-22T21:13:51.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.102+0000 7fcbd9800640 1 -- 172.21.15.161:0/3963598662 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcbd4067220 con 0x7fcbd413fe50 2024-05-22T21:13:51.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.102+0000 7fcbcb400640 1 -- 172.21.15.161:0/3963598662 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcbbc054060 con 0x7fcbd413fe50 2024-05-22T21:13:51.105 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:13:51.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.102+0000 7fcbd9800640 1 -- 172.21.15.161:0/3963598662 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcbb0066690 msgr2=0x7fcbb0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.102+0000 7fcbd9800640 1 --2- 172.21.15.161:0/3963598662 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcbb0066690 0x7fcbb0068b50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40ad000 tx=0x7fcbc40a6000 comp rx=0 tx=0).stop 2024-05-22T21:13:51.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.102+0000 7fcbd9800640 1 -- 172.21.15.161:0/3963598662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbd413fe50 msgr2=0x7fcbd41426e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.102+0000 7fcbd9800640 1 --2- 172.21.15.161:0/3963598662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbd413fe50 0x7fcbd41426e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00c3c0 tx=0x7fcbbc00c880 comp rx=0 tx=0).stop 2024-05-22T21:13:51.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.102+0000 7fcbd9800640 1 -- 172.21.15.161:0/3963598662 shutdown_connections 2024-05-22T21:13:51.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.102+0000 7fcbd9800640 1 --2- 172.21.15.161:0/3963598662 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcbb0066690 0x7fcbb0068b50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.102+0000 7fcbd9800640 1 --2- 172.21.15.161:0/3963598662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd415af60 0x7fcbd4158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.102+0000 7fcbd9800640 1 --2- 172.21.15.161:0/3963598662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbd413fe50 0x7fcbd41426e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.102+0000 7fcbd9800640 1 --2- 172.21.15.161:0/3963598662 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcbd413cfc0 0x7fcbd41578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.102+0000 7fcbd9800640 1 -- 172.21.15.161:0/3963598662 >> 172.21.15.161:0/3963598662 conn(0x7fcbd406a2d0 msgr2=0x7fcbd406a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:51.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.102+0000 7fcbd9800640 1 -- 172.21.15.161:0/3963598662 shutdown_connections 2024-05-22T21:13:51.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.102+0000 7fcbd9800640 1 -- 172.21.15.161:0/3963598662 wait complete. 2024-05-22T21:13:51.127 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:13:51.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.262+0000 7fc4b6e00640 1 Processor -- start 2024-05-22T21:13:51.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.262+0000 7fc4b6e00640 1 -- start start 2024-05-22T21:13:51.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.262+0000 7fc4b6e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc4b01402b0 0x7fc4b0070590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.262+0000 7fc4b6e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc4b006c750 0x7fc4b00716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.262+0000 7fc4b6e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b006cbc0 con 0x7fc4b006c2a0 2024-05-22T21:13:51.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.262+0000 7fc4b6e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b015d7c0 con 0x7fc4b006c750 2024-05-22T21:13:51.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.262+0000 7fc4b6e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b015d9d0 con 0x7fc4b01402b0 2024-05-22T21:13:51.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.262+0000 7fc4b4e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc4b006c750 0x7fc4b00716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.262+0000 7fc4b4e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc4b006c750 0x7fc4b00716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46810/0 (socket says 172.21.15.161:46810) 2024-05-22T21:13:51.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.262+0000 7fc4b4e00640 1 -- 172.21.15.161:0/3175792114 learned_addr learned my addr 172.21.15.161:0/3175792114 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:51.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.262+0000 7fc4afe00640 1 --2- 172.21.15.161:0/3175792114 >> v2:172.21.15.179:3300/0 conn(0x7fc4b01402b0 0x7fc4b0070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.262+0000 7fc4b4e00640 1 -- 172.21.15.161:0/3175792114 >> v2:172.21.15.179:3300/0 conn(0x7fc4b01402b0 msgr2=0x7fc4b0070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:51.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.262+0000 7fc4b4e00640 1 --2- 172.21.15.161:0/3175792114 >> v2:172.21.15.179:3300/0 conn(0x7fc4b01402b0 0x7fc4b0070590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.262+0000 7fc4b4e00640 1 -- 172.21.15.161:0/3175792114 >> v1:172.21.15.179:6789/0 conn(0x7fc4b006c2a0 legacy=0x7fc4b0070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.262+0000 7fc4b4e00640 1 -- 172.21.15.161:0/3175792114 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b015dcb0 con 0x7fc4b006c750 2024-05-22T21:13:51.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.266+0000 7fc4b4e00640 1 --2- 172.21.15.161:0/3175792114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4b006c750 0x7fc4b00716a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc4b006dec0 tx=0x7fc4a801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495df6e6ca82a3fb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:51.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.266+0000 7fc4af400640 1 -- 172.21.15.161:0/3175792114 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a80a58b0 con 0x7fc4b006c750 2024-05-22T21:13:51.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.266+0000 7fc4af400640 1 -- 172.21.15.161:0/3175792114 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a80a5a10 con 0x7fc4b006c750 2024-05-22T21:13:51.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.266+0000 7fc4af400640 1 -- 172.21.15.161:0/3175792114 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a80a5da0 con 0x7fc4b006c750 2024-05-22T21:13:51.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.266+0000 7fc4b6e00640 1 -- 172.21.15.161:0/3175792114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4b006c750 msgr2=0x7fc4b00716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.266+0000 7fc4b6e00640 1 --2- 172.21.15.161:0/3175792114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4b006c750 0x7fc4b00716a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc4b006dec0 tx=0x7fc4a801c9f0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.266+0000 7fc4b6e00640 1 -- 172.21.15.161:0/3175792114 shutdown_connections 2024-05-22T21:13:51.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.266+0000 7fc4b6e00640 1 --2- 172.21.15.161:0/3175792114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4b006c750 0x7fc4b00716a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.266+0000 7fc4b6e00640 1 --2- 172.21.15.161:0/3175792114 >> v2:172.21.15.179:3300/0 conn(0x7fc4b01402b0 0x7fc4b0070590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.266+0000 7fc4b6e00640 1 -- 172.21.15.161:0/3175792114 >> 172.21.15.161:0/3175792114 conn(0x7fc4b006a2d0 msgr2=0x7fc4b006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:51.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.266+0000 7fc4b6e00640 1 -- 172.21.15.161:0/3175792114 shutdown_connections 2024-05-22T21:13:51.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.266+0000 7fc4b6e00640 1 -- 172.21.15.161:0/3175792114 wait complete. 2024-05-22T21:13:51.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4b6e00640 1 Processor -- start 2024-05-22T21:13:51.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4b6e00640 1 -- start start 2024-05-22T21:13:51.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4b6e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4b006c2a0 0x7fc4b01573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4b6e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4b006c750 0x7fc4b015f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4b6e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4b01402b0 0x7fc4b0138e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4b6e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b0160300 con 0x7fc4b006c2a0 2024-05-22T21:13:51.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4b6e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b013d750 con 0x7fc4b01402b0 2024-05-22T21:13:51.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4b6e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b013da30 con 0x7fc4b006c750 2024-05-22T21:13:51.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4afe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4b006c2a0 0x7fc4b01573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4afe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4b006c2a0 0x7fc4b01573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46826/0 (socket says 172.21.15.161:46826) 2024-05-22T21:13:51.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4afe00640 1 -- 172.21.15.161:0/4224878737 learned_addr learned my addr 172.21.15.161:0/4224878737 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:51.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4a7400640 1 --2- 172.21.15.161:0/4224878737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4b006c750 0x7fc4b015f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4b4e00640 1 --2- 172.21.15.161:0/4224878737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4b01402b0 0x7fc4b0138e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4afe00640 1 -- 172.21.15.161:0/4224878737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4b006c750 msgr2=0x7fc4b015f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4afe00640 1 --2- 172.21.15.161:0/4224878737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4b006c750 0x7fc4b015f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4afe00640 1 -- 172.21.15.161:0/4224878737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4b01402b0 msgr2=0x7fc4b0138e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4afe00640 1 --2- 172.21.15.161:0/4224878737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4b01402b0 0x7fc4b0138e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4afe00640 1 -- 172.21.15.161:0/4224878737 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a8009b30 con 0x7fc4b006c2a0 2024-05-22T21:13:51.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4a7400640 1 --2- 172.21.15.161:0/4224878737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4b006c750 0x7fc4b015f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:51.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4afe00640 1 --2- 172.21.15.161:0/4224878737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4b006c2a0 0x7fc4b01573a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000e870 tx=0x7fc4a000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:51.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4acc00640 1 -- 172.21.15.161:0/4224878737 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0009530 con 0x7fc4b006c2a0 2024-05-22T21:13:51.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4acc00640 1 -- 172.21.15.161:0/4224878737 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a000c940 con 0x7fc4b006c2a0 2024-05-22T21:13:51.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4acc00640 1 -- 172.21.15.161:0/4224878737 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a000ccd0 con 0x7fc4b006c2a0 2024-05-22T21:13:51.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4b6e00640 1 -- 172.21.15.161:0/4224878737 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b013dd10 con 0x7fc4b006c2a0 2024-05-22T21:13:51.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4b6e00640 1 -- 172.21.15.161:0/4224878737 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b015eb90 con 0x7fc4b006c2a0 2024-05-22T21:13:51.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.270+0000 7fc4b6e00640 1 -- 172.21.15.161:0/4224878737 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b0067220 con 0x7fc4b006c2a0 2024-05-22T21:13:51.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.274+0000 7fc4acc00640 1 -- 172.21.15.161:0/4224878737 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc4a0013470 con 0x7fc4b006c2a0 2024-05-22T21:13:51.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.274+0000 7fc4acc00640 1 --2- 172.21.15.161:0/4224878737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc48c0666c0 0x7fc48c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.274+0000 7fc4acc00640 1 -- 172.21.15.161:0/4224878737 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4a0084320 con 0x7fc4b006c2a0 2024-05-22T21:13:51.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.274+0000 7fc4a7400640 1 --2- 172.21.15.161:0/4224878737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc48c0666c0 0x7fc48c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.274+0000 7fc4a7400640 1 --2- 172.21.15.161:0/4224878737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc48c0666c0 0x7fc48c068b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc4b01604f0 tx=0x7fc498009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:51.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.278+0000 7fc4acc00640 1 -- 172.21.15.161:0/4224878737 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc4a0051600 con 0x7fc4b006c2a0 2024-05-22T21:13:51.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.430+0000 7fc4b6e00640 1 -- 172.21.15.161:0/4224878737 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc4b013cf10 con 0x7fc4b006c2a0 2024-05-22T21:13:51.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.430+0000 7fc4acc00640 1 -- 172.21.15.161:0/4224878737 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc4a0054c20 con 0x7fc4b006c2a0 2024-05-22T21:13:51.435 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:13:51.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.434+0000 7fc4b6e00640 1 -- 172.21.15.161:0/4224878737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc48c0666c0 msgr2=0x7fc48c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.434+0000 7fc4b6e00640 1 --2- 172.21.15.161:0/4224878737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc48c0666c0 0x7fc48c068b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc4b01604f0 tx=0x7fc498009040 comp rx=0 tx=0).stop 2024-05-22T21:13:51.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.434+0000 7fc4b6e00640 1 -- 172.21.15.161:0/4224878737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4b006c2a0 msgr2=0x7fc4b01573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.434+0000 7fc4b6e00640 1 --2- 172.21.15.161:0/4224878737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4b006c2a0 0x7fc4b01573a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000e870 tx=0x7fc4a000ed30 comp rx=0 tx=0).stop 2024-05-22T21:13:51.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.434+0000 7fc4b6e00640 1 -- 172.21.15.161:0/4224878737 shutdown_connections 2024-05-22T21:13:51.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.434+0000 7fc4b6e00640 1 --2- 172.21.15.161:0/4224878737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc48c0666c0 0x7fc48c068b80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.434+0000 7fc4b6e00640 1 --2- 172.21.15.161:0/4224878737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4b01402b0 0x7fc4b0138e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.434+0000 7fc4b6e00640 1 --2- 172.21.15.161:0/4224878737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4b006c750 0x7fc4b015f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.434+0000 7fc4b6e00640 1 --2- 172.21.15.161:0/4224878737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4b006c2a0 0x7fc4b01573a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.434+0000 7fc4b6e00640 1 -- 172.21.15.161:0/4224878737 >> 172.21.15.161:0/4224878737 conn(0x7fc4b006a2d0 msgr2=0x7fc4b006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:51.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.434+0000 7fc4b6e00640 1 -- 172.21.15.161:0/4224878737 shutdown_connections 2024-05-22T21:13:51.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.434+0000 7fc4b6e00640 1 -- 172.21.15.161:0/4224878737 wait complete. 2024-05-22T21:13:51.456 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:13:51.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.590+0000 7fc0d2600640 1 Processor -- start 2024-05-22T21:13:51.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.590+0000 7fc0d2600640 1 -- start start 2024-05-22T21:13:51.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.594+0000 7fc0d2600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc0cc13cfc0 0x7fc0cc1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.594+0000 7fc0d2600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc0cc13fe50 0x7fc0cc06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.594+0000 7fc0d2600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0cc13d3a0 con 0x7fc0cc15af60 2024-05-22T21:13:51.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.594+0000 7fc0d2600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0cc0cd920 con 0x7fc0cc13fe50 2024-05-22T21:13:51.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.594+0000 7fc0d2600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0cc0cdc00 con 0x7fc0cc13cfc0 2024-05-22T21:13:51.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.594+0000 7fc0cbe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc0cc13fe50 0x7fc0cc06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.594+0000 7fc0cbe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc0cc13fe50 0x7fc0cc06fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46838/0 (socket says 172.21.15.161:46838) 2024-05-22T21:13:51.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.594+0000 7fc0cbe00640 1 -- 172.21.15.161:0/2719124294 learned_addr learned my addr 172.21.15.161:0/2719124294 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:51.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.594+0000 7fc0c2a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc0cc15af60 0x7fc0cc13f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:52008/0 (socket says 172.21.15.161:52008) 2024-05-22T21:13:51.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.594+0000 7fc0cb400640 1 --2- 172.21.15.161:0/2719124294 >> v2:172.21.15.179:3300/0 conn(0x7fc0cc13cfc0 0x7fc0cc1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.594+0000 7fc0cbe00640 1 -- 172.21.15.161:0/2719124294 >> v2:172.21.15.179:3300/0 conn(0x7fc0cc13cfc0 msgr2=0x7fc0cc1402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:51.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.594+0000 7fc0cbe00640 1 --2- 172.21.15.161:0/2719124294 >> v2:172.21.15.179:3300/0 conn(0x7fc0cc13cfc0 0x7fc0cc1402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.594+0000 7fc0cbe00640 1 -- 172.21.15.161:0/2719124294 >> v1:172.21.15.161:6789/0 conn(0x7fc0cc15af60 legacy=0x7fc0cc13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.594+0000 7fc0cbe00640 1 -- 172.21.15.161:0/2719124294 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0cc0cdee0 con 0x7fc0cc13fe50 2024-05-22T21:13:51.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.594+0000 7fc0caa00640 1 -- 172.21.15.161:0/2719124294 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 931373113 0 0) 0x7fc0cc13d3a0 con 0x7fc0cc15af60 2024-05-22T21:13:51.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0cbe00640 1 --2- 172.21.15.161:0/2719124294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0cc13fe50 0x7fc0cc06fce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc06de90 tx=0x7fc0c401eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab66daf12dfb26b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:51.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0caa00640 1 -- 172.21.15.161:0/2719124294 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c400b2d0 con 0x7fc0cc13fe50 2024-05-22T21:13:51.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0caa00640 1 -- 172.21.15.161:0/2719124294 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c400b430 con 0x7fc0cc13fe50 2024-05-22T21:13:51.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0caa00640 1 -- 172.21.15.161:0/2719124294 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c40a7850 con 0x7fc0cc13fe50 2024-05-22T21:13:51.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0d2600640 1 -- 172.21.15.161:0/2719124294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0cc13fe50 msgr2=0x7fc0cc06fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0d2600640 1 --2- 172.21.15.161:0/2719124294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0cc13fe50 0x7fc0cc06fce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc06de90 tx=0x7fc0c401eb30 comp rx=0 tx=0).stop 2024-05-22T21:13:51.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0d2600640 1 -- 172.21.15.161:0/2719124294 shutdown_connections 2024-05-22T21:13:51.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0d2600640 1 --2- 172.21.15.161:0/2719124294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0cc13fe50 0x7fc0cc06fce0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0d2600640 1 --2- 172.21.15.161:0/2719124294 >> v2:172.21.15.179:3300/0 conn(0x7fc0cc13cfc0 0x7fc0cc1402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0d2600640 1 -- 172.21.15.161:0/2719124294 >> 172.21.15.161:0/2719124294 conn(0x7fc0cc06a2d0 msgr2=0x7fc0cc06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:51.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0d2600640 1 -- 172.21.15.161:0/2719124294 shutdown_connections 2024-05-22T21:13:51.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0d2600640 1 -- 172.21.15.161:0/2719124294 wait complete. 2024-05-22T21:13:51.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0d2600640 1 Processor -- start 2024-05-22T21:13:51.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0d2600640 1 -- start start 2024-05-22T21:13:51.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0d2600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0cc13cfc0 0x7fc0cc141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0d2600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0cc13fe50 0x7fc0cc15bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0d2600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc0cc15af60 0x7fc0cc139720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0d2600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc13fb90 con 0x7fc0cc13fe50 2024-05-22T21:13:51.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0d2600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc13f2f0 con 0x7fc0cc15af60 2024-05-22T21:13:51.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.598+0000 7fc0d2600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc13f5d0 con 0x7fc0cc13cfc0 2024-05-22T21:13:51.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0c2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0cc13fe50 0x7fc0cc15bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0c2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0cc13fe50 0x7fc0cc15bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46854/0 (socket says 172.21.15.161:46854) 2024-05-22T21:13:51.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0c2a00640 1 -- 172.21.15.161:0/4272196006 learned_addr learned my addr 172.21.15.161:0/4272196006 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:51.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0cb400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0cc13cfc0 0x7fc0cc141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0cbe00640 1 --2- 172.21.15.161:0/4272196006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc0cc15af60 0x7fc0cc139720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0c2a00640 1 -- 172.21.15.161:0/4272196006 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0cc13cfc0 msgr2=0x7fc0cc141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0c2a00640 1 --2- 172.21.15.161:0/4272196006 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0cc13cfc0 0x7fc0cc141e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0c2a00640 1 -- 172.21.15.161:0/4272196006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc0cc15af60 msgr2=0x7fc0cc139720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0c2a00640 1 --2- 172.21.15.161:0/4272196006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc0cc15af60 0x7fc0cc139720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0c2a00640 1 -- 172.21.15.161:0/4272196006 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c400bd40 con 0x7fc0cc13fe50 2024-05-22T21:13:51.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0c2a00640 1 --2- 172.21.15.161:0/4272196006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0cc13fe50 0x7fc0cc15bba0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400c480 tx=0x7fc0b400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:51.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0c3e00640 1 -- 172.21.15.161:0/4272196006 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b40074e0 con 0x7fc0cc13fe50 2024-05-22T21:13:51.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0c3e00640 1 -- 172.21.15.161:0/4272196006 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b4007660 con 0x7fc0cc13fe50 2024-05-22T21:13:51.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0c3e00640 1 -- 172.21.15.161:0/4272196006 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4004a30 con 0x7fc0cc13fe50 2024-05-22T21:13:51.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0d2600640 1 -- 172.21.15.161:0/4272196006 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0cc13f8b0 con 0x7fc0cc13fe50 2024-05-22T21:13:51.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0d2600640 1 -- 172.21.15.161:0/4272196006 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0cc139c60 con 0x7fc0cc13fe50 2024-05-22T21:13:51.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0d2600640 1 -- 172.21.15.161:0/4272196006 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0cc155f40 con 0x7fc0cc13fe50 2024-05-22T21:13:51.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0c3e00640 1 -- 172.21.15.161:0/4272196006 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc0b4004b90 con 0x7fc0cc13fe50 2024-05-22T21:13:51.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.602+0000 7fc0c3e00640 1 --2- 172.21.15.161:0/4272196006 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc0a8066690 0x7fc0a8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.606+0000 7fc0c3e00640 1 -- 172.21.15.161:0/4272196006 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0b40832d0 con 0x7fc0cc13fe50 2024-05-22T21:13:51.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.606+0000 7fc0cb400640 1 --2- 172.21.15.161:0/4272196006 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc0a8066690 0x7fc0a8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.606+0000 7fc0cb400640 1 --2- 172.21.15.161:0/4272196006 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc0a8066690 0x7fc0a8068b50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc1427a0 tx=0x7fc0bc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:51.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.610+0000 7fc0c3e00640 1 -- 172.21.15.161:0/4272196006 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc0cc155f40 con 0x7fc0cc13fe50 2024-05-22T21:13:51.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.762+0000 7fc0d2600640 1 -- 172.21.15.161:0/4272196006 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc0cc156320 con 0x7fc0cc13fe50 2024-05-22T21:13:51.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.762+0000 7fc0c3e00640 1 -- 172.21.15.161:0/4272196006 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc0b4018020 con 0x7fc0cc13fe50 2024-05-22T21:13:51.766 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:13:51.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.762+0000 7fc0d2600640 1 -- 172.21.15.161:0/4272196006 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc0a8066690 msgr2=0x7fc0a8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.762+0000 7fc0d2600640 1 --2- 172.21.15.161:0/4272196006 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc0a8066690 0x7fc0a8068b50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc1427a0 tx=0x7fc0bc009190 comp rx=0 tx=0).stop 2024-05-22T21:13:51.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.762+0000 7fc0d2600640 1 -- 172.21.15.161:0/4272196006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0cc13fe50 msgr2=0x7fc0cc15bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.762+0000 7fc0d2600640 1 --2- 172.21.15.161:0/4272196006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0cc13fe50 0x7fc0cc15bba0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400c480 tx=0x7fc0b400c940 comp rx=0 tx=0).stop 2024-05-22T21:13:51.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.762+0000 7fc0d2600640 1 -- 172.21.15.161:0/4272196006 shutdown_connections 2024-05-22T21:13:51.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.762+0000 7fc0d2600640 1 --2- 172.21.15.161:0/4272196006 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc0a8066690 0x7fc0a8068b50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.762+0000 7fc0d2600640 1 --2- 172.21.15.161:0/4272196006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc0cc15af60 0x7fc0cc139720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.762+0000 7fc0d2600640 1 --2- 172.21.15.161:0/4272196006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0cc13fe50 0x7fc0cc15bba0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.762+0000 7fc0d2600640 1 --2- 172.21.15.161:0/4272196006 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0cc13cfc0 0x7fc0cc141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.762+0000 7fc0d2600640 1 -- 172.21.15.161:0/4272196006 >> 172.21.15.161:0/4272196006 conn(0x7fc0cc06a2d0 msgr2=0x7fc0cc0ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:51.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.762+0000 7fc0d2600640 1 -- 172.21.15.161:0/4272196006 shutdown_connections 2024-05-22T21:13:51.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.762+0000 7fc0d2600640 1 -- 172.21.15.161:0/4272196006 wait complete. 2024-05-22T21:13:51.786 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:51.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.918+0000 7f1713200640 1 Processor -- start 2024-05-22T21:13:51.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.918+0000 7f1713200640 1 -- start start 2024-05-22T21:13:51.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.918+0000 7f1713200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f17140d0a40 0x7f17140d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.918+0000 7f1713200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17140cc320 0x7f17140cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.918+0000 7f1713200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17140cb8a0 con 0x7f171415af00 2024-05-22T21:13:51.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.918+0000 7f1713200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17140d13f0 con 0x7f17140d0a40 2024-05-22T21:13:51.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.918+0000 7f1713200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171415f530 con 0x7f17140cc320 2024-05-22T21:13:51.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.918+0000 7f1709400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f17140d0a40 0x7f17140d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.918+0000 7f1709400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f17140d0a40 0x7f17140d0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46866/0 (socket says 172.21.15.161:46866) 2024-05-22T21:13:51.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.918+0000 7f1709400640 1 -- 172.21.15.161:0/3386643466 learned_addr learned my addr 172.21.15.161:0/3386643466 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:51.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.918+0000 7f1712800640 1 --2- 172.21.15.161:0/3386643466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17140cc320 0x7f17140cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.922+0000 7f1709400640 1 -- 172.21.15.161:0/3386643466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17140cc320 msgr2=0x7f17140cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:51.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.922+0000 7f1709400640 1 --2- 172.21.15.161:0/3386643466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17140cc320 0x7f17140cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.922+0000 7f1709400640 1 -- 172.21.15.161:0/3386643466 >> v1:172.21.15.179:6789/0 conn(0x7f171415af00 legacy=0x7f171415eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.922+0000 7f1709400640 1 -- 172.21.15.161:0/3386643466 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171415f810 con 0x7f17140d0a40 2024-05-22T21:13:51.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.922+0000 7f1709400640 1 --2- 172.21.15.161:0/3386643466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17140d0a40 0x7f17140d0eb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009870 tx=0x7f16fc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db46d1ef9fe9f0ca server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:51.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.922+0000 7f1711400640 1 -- 172.21.15.161:0/3386643466 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0aa680 con 0x7f17140d0a40 2024-05-22T21:13:51.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.922+0000 7f1711400640 1 -- 172.21.15.161:0/3386643466 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc0aa7e0 con 0x7f17140d0a40 2024-05-22T21:13:51.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.922+0000 7f1711400640 1 -- 172.21.15.161:0/3386643466 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0aab70 con 0x7f17140d0a40 2024-05-22T21:13:51.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.922+0000 7f1713200640 1 -- 172.21.15.161:0/3386643466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17140d0a40 msgr2=0x7f17140d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.922+0000 7f1713200640 1 --2- 172.21.15.161:0/3386643466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17140d0a40 0x7f17140d0eb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009870 tx=0x7f16fc01c520 comp rx=0 tx=0).stop 2024-05-22T21:13:51.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.922+0000 7f1713200640 1 -- 172.21.15.161:0/3386643466 shutdown_connections 2024-05-22T21:13:51.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.922+0000 7f1713200640 1 --2- 172.21.15.161:0/3386643466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17140cc320 0x7f17140cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.922+0000 7f1713200640 1 --2- 172.21.15.161:0/3386643466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17140d0a40 0x7f17140d0eb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.922+0000 7f1713200640 1 -- 172.21.15.161:0/3386643466 >> 172.21.15.161:0/3386643466 conn(0x7f1714099a40 msgr2=0x7f1714099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:51.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.922+0000 7f1713200640 1 -- 172.21.15.161:0/3386643466 shutdown_connections 2024-05-22T21:13:51.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.922+0000 7f1713200640 1 -- 172.21.15.161:0/3386643466 wait complete. 2024-05-22T21:13:51.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1713200640 1 Processor -- start 2024-05-22T21:13:51.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1713200640 1 -- start start 2024-05-22T21:13:51.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1713200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17140cc320 0x7f171415b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1713200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17140d0a40 0x7f17140c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1713200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f171415af00 0x7f171406e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1713200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171415bc70 con 0x7f17140d0a40 2024-05-22T21:13:51.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1713200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171415d6a0 con 0x7f171415af00 2024-05-22T21:13:51.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1713200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171415d980 con 0x7f17140cc320 2024-05-22T21:13:51.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1709400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17140d0a40 0x7f17140c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1709400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17140d0a40 0x7f17140c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46878/0 (socket says 172.21.15.161:46878) 2024-05-22T21:13:51.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1709400640 1 -- 172.21.15.161:0/2141893331 learned_addr learned my addr 172.21.15.161:0/2141893331 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:51.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1711e00640 1 --2- 172.21.15.161:0/2141893331 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17140cc320 0x7f171415b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1712800640 1 --2- 172.21.15.161:0/2141893331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f171415af00 0x7f171406e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1709400640 1 -- 172.21.15.161:0/2141893331 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17140cc320 msgr2=0x7f171415b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1709400640 1 --2- 172.21.15.161:0/2141893331 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17140cc320 0x7f171415b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1709400640 1 -- 172.21.15.161:0/2141893331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f171415af00 msgr2=0x7f171406e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:51.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1709400640 1 --2- 172.21.15.161:0/2141893331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f171415af00 0x7f171406e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:51.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1709400640 1 -- 172.21.15.161:0/2141893331 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc0094d0 con 0x7f17140d0a40 2024-05-22T21:13:51.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1709400640 1 --2- 172.21.15.161:0/2141893331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17140d0a40 0x7f17140c8330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f16fc01a150 tx=0x7f16fc01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:51.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f170aa00640 1 -- 172.21.15.161:0/2141893331 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0b3800 con 0x7f17140d0a40 2024-05-22T21:13:51.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f170aa00640 1 -- 172.21.15.161:0/2141893331 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc0b3980 con 0x7f17140d0a40 2024-05-22T21:13:51.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f170aa00640 1 -- 172.21.15.161:0/2141893331 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0b3d30 con 0x7f17140d0a40 2024-05-22T21:13:51.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.926+0000 7f1713200640 1 -- 172.21.15.161:0/2141893331 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171415dc60 con 0x7f17140d0a40 2024-05-22T21:13:51.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.930+0000 7f1713200640 1 -- 172.21.15.161:0/2141893331 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171415d2c0 con 0x7f17140d0a40 2024-05-22T21:13:51.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.930+0000 7f1713200640 1 -- 172.21.15.161:0/2141893331 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171415d4b0 con 0x7f17140d0a40 2024-05-22T21:13:51.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.930+0000 7f170aa00640 1 -- 172.21.15.161:0/2141893331 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f16fc0b2380 con 0x7f17140d0a40 2024-05-22T21:13:51.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.930+0000 7f170aa00640 1 --2- 172.21.15.161:0/2141893331 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16f0066690 0x7f16f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:51.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.930+0000 7f170aa00640 1 -- 172.21.15.161:0/2141893331 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16fc11e870 con 0x7f17140d0a40 2024-05-22T21:13:51.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.930+0000 7f1711e00640 1 --2- 172.21.15.161:0/2141893331 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16f0066690 0x7f16f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:51.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.930+0000 7f1711e00640 1 --2- 172.21.15.161:0/2141893331 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16f0066690 0x7f16f0068b50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1704007fd0 tx=0x7f1704009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:51.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:51.934+0000 7f170aa00640 1 -- 172.21.15.161:0/2141893331 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f16fc0ebd10 con 0x7f17140d0a40 2024-05-22T21:13:52.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.090+0000 7f1713200640 1 -- 172.21.15.161:0/2141893331 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1714067070 con 0x7f17140d0a40 2024-05-22T21:13:52.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.090+0000 7f170aa00640 1 -- 172.21.15.161:0/2141893331 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f16fc0ef330 con 0x7f17140d0a40 2024-05-22T21:13:52.096 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:52.096 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:52.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.094+0000 7f1713200640 1 -- 172.21.15.161:0/2141893331 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16f0066690 msgr2=0x7f16f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:52.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.094+0000 7f1713200640 1 --2- 172.21.15.161:0/2141893331 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16f0066690 0x7f16f0068b50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1704007fd0 tx=0x7f1704009040 comp rx=0 tx=0).stop 2024-05-22T21:13:52.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.094+0000 7f1713200640 1 -- 172.21.15.161:0/2141893331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17140d0a40 msgr2=0x7f17140c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:52.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.094+0000 7f1713200640 1 --2- 172.21.15.161:0/2141893331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17140d0a40 0x7f17140c8330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f16fc01a150 tx=0x7f16fc01ad60 comp rx=0 tx=0).stop 2024-05-22T21:13:52.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.094+0000 7f1713200640 1 -- 172.21.15.161:0/2141893331 shutdown_connections 2024-05-22T21:13:52.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.094+0000 7f1713200640 1 --2- 172.21.15.161:0/2141893331 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16f0066690 0x7f16f0068b50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.094+0000 7f1713200640 1 --2- 172.21.15.161:0/2141893331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f171415af00 0x7f171406e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.094+0000 7f1713200640 1 --2- 172.21.15.161:0/2141893331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17140d0a40 0x7f17140c8330 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.094+0000 7f1713200640 1 --2- 172.21.15.161:0/2141893331 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17140cc320 0x7f171415b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.094+0000 7f1713200640 1 -- 172.21.15.161:0/2141893331 >> 172.21.15.161:0/2141893331 conn(0x7f1714099a40 msgr2=0x7f17140b6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:52.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.094+0000 7f1713200640 1 -- 172.21.15.161:0/2141893331 shutdown_connections 2024-05-22T21:13:52.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.094+0000 7f1713200640 1 -- 172.21.15.161:0/2141893331 wait complete. 2024-05-22T21:13:52.117 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:13:21.237503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:13:22.425891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:13:52.117 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:13:52.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.250+0000 7fc058e00640 1 Processor -- start 2024-05-22T21:13:52.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.250+0000 7fc058e00640 1 -- start start 2024-05-22T21:13:52.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.254+0000 7fc058e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc05413cfc0 0x7fc0541402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.254+0000 7fc058e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc05413fe50 0x7fc05406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.254+0000 7fc058e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc05413d3a0 con 0x7fc05415af60 2024-05-22T21:13:52.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.254+0000 7fc058e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0540cd920 con 0x7fc05413cfc0 2024-05-22T21:13:52.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.254+0000 7fc058e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0540cdc00 con 0x7fc05413fe50 2024-05-22T21:13:52.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.254+0000 7fc051e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc05413cfc0 0x7fc0541402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.254+0000 7fc051e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc05413cfc0 0x7fc0541402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46886/0 (socket says 172.21.15.161:46886) 2024-05-22T21:13:52.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.254+0000 7fc051e00640 1 -- 172.21.15.161:0/3302585452 learned_addr learned my addr 172.21.15.161:0/3302585452 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:52.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.254+0000 7fc052800640 1 --2- 172.21.15.161:0/3302585452 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc05413fe50 0x7fc05406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.254+0000 7fc051e00640 1 -- 172.21.15.161:0/3302585452 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc05413fe50 msgr2=0x7fc05406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:52.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.254+0000 7fc051e00640 1 --2- 172.21.15.161:0/3302585452 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc05413fe50 0x7fc05406fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.254+0000 7fc051e00640 1 -- 172.21.15.161:0/3302585452 >> v1:172.21.15.161:6789/0 conn(0x7fc05415af60 legacy=0x7fc05413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:52.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.254+0000 7fc051e00640 1 -- 172.21.15.161:0/3302585452 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0540cdee0 con 0x7fc05413cfc0 2024-05-22T21:13:52.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.258+0000 7fc051e00640 1 --2- 172.21.15.161:0/3302585452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc05413cfc0 0x7fc0541402e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc044009870 tx=0x7fc04401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f7621bd2b974dc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:52.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.258+0000 7fc051400640 1 -- 172.21.15.161:0/3302585452 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0440ab600 con 0x7fc05413cfc0 2024-05-22T21:13:52.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.258+0000 7fc051400640 1 -- 172.21.15.161:0/3302585452 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0440a7050 con 0x7fc05413cfc0 2024-05-22T21:13:52.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.258+0000 7fc051400640 1 -- 172.21.15.161:0/3302585452 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0440ab980 con 0x7fc05413cfc0 2024-05-22T21:13:52.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.258+0000 7fc058e00640 1 -- 172.21.15.161:0/3302585452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc05413cfc0 msgr2=0x7fc0541402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:52.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.258+0000 7fc058e00640 1 --2- 172.21.15.161:0/3302585452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc05413cfc0 0x7fc0541402e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc044009870 tx=0x7fc04401c4a0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.258+0000 7fc058e00640 1 -- 172.21.15.161:0/3302585452 shutdown_connections 2024-05-22T21:13:52.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.258+0000 7fc058e00640 1 --2- 172.21.15.161:0/3302585452 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc05413fe50 0x7fc05406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.258+0000 7fc058e00640 1 --2- 172.21.15.161:0/3302585452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc05413cfc0 0x7fc0541402e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.258+0000 7fc058e00640 1 -- 172.21.15.161:0/3302585452 >> 172.21.15.161:0/3302585452 conn(0x7fc05406a2d0 msgr2=0x7fc05406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:52.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.258+0000 7fc058e00640 1 -- 172.21.15.161:0/3302585452 shutdown_connections 2024-05-22T21:13:52.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.258+0000 7fc058e00640 1 -- 172.21.15.161:0/3302585452 wait complete. 2024-05-22T21:13:52.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.258+0000 7fc058e00640 1 Processor -- start 2024-05-22T21:13:52.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.258+0000 7fc058e00640 1 -- start start 2024-05-22T21:13:52.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc058e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc05413cfc0 0x7fc054157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc058e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc05413fe50 0x7fc05406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc058e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc05415af60 0x7fc05406d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc058e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc054159c40 con 0x7fc05413fe50 2024-05-22T21:13:52.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc058e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0540b2650 con 0x7fc05415af60 2024-05-22T21:13:52.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc058e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0540b2320 con 0x7fc05413cfc0 2024-05-22T21:13:52.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc049400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc05413fe50 0x7fc05406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc049400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc05413fe50 0x7fc05406cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46898/0 (socket says 172.21.15.161:46898) 2024-05-22T21:13:52.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc049400640 1 -- 172.21.15.161:0/3813013734 learned_addr learned my addr 172.21.15.161:0/3813013734 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:52.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc052800640 1 --2- 172.21.15.161:0/3813013734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc05415af60 0x7fc05406d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc051e00640 1 --2- 172.21.15.161:0/3813013734 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc05413cfc0 0x7fc054157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc049400640 1 -- 172.21.15.161:0/3813013734 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc05413cfc0 msgr2=0x7fc054157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:52.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc049400640 1 --2- 172.21.15.161:0/3813013734 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc05413cfc0 0x7fc054157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc049400640 1 -- 172.21.15.161:0/3813013734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc05415af60 msgr2=0x7fc05406d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:52.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc049400640 1 --2- 172.21.15.161:0/3813013734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc05415af60 0x7fc05406d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc049400640 1 -- 172.21.15.161:0/3813013734 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0440094d0 con 0x7fc05413fe50 2024-05-22T21:13:52.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc049400640 1 --2- 172.21.15.161:0/3813013734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc05413fe50 0x7fc05406cb30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00c3c0 tx=0x7fc03c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:52.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc04aa00640 1 -- 172.21.15.161:0/3813013734 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c0052d0 con 0x7fc05413fe50 2024-05-22T21:13:52.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc04aa00640 1 -- 172.21.15.161:0/3813013734 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c005430 con 0x7fc05413fe50 2024-05-22T21:13:52.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc04aa00640 1 -- 172.21.15.161:0/3813013734 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c003840 con 0x7fc05413fe50 2024-05-22T21:13:52.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc058e00640 1 -- 172.21.15.161:0/3813013734 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc054159680 con 0x7fc05413fe50 2024-05-22T21:13:52.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc058e00640 1 -- 172.21.15.161:0/3813013734 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc054157ac0 con 0x7fc05413fe50 2024-05-22T21:13:52.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc058e00640 1 -- 172.21.15.161:0/3813013734 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0540b2180 con 0x7fc05413fe50 2024-05-22T21:13:52.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.262+0000 7fc04aa00640 1 -- 172.21.15.161:0/3813013734 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc03c0039a0 con 0x7fc05413fe50 2024-05-22T21:13:52.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.266+0000 7fc04aa00640 1 --2- 172.21.15.161:0/3813013734 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc030066690 0x7fc030068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.266+0000 7fc04aa00640 1 -- 172.21.15.161:0/3813013734 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc03c083690 con 0x7fc05413fe50 2024-05-22T21:13:52.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.266+0000 7fc051e00640 1 --2- 172.21.15.161:0/3813013734 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc030066690 0x7fc030068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.266+0000 7fc051e00640 1 --2- 172.21.15.161:0/3813013734 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc030066690 0x7fc030068b50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc0440ad000 tx=0x7fc044008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:52.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.270+0000 7fc04aa00640 1 -- 172.21.15.161:0/3813013734 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc0540b2180 con 0x7fc05413fe50 2024-05-22T21:13:52.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.422+0000 7fc058e00640 1 -- 172.21.15.161:0/3813013734 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0540b2180 con 0x7fc05413fe50 2024-05-22T21:13:52.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.422+0000 7fc04aa00640 1 -- 172.21.15.161:0/3813013734 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc0540b2180 con 0x7fc05413fe50 2024-05-22T21:13:52.427 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:13:52.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.426+0000 7fc058e00640 1 -- 172.21.15.161:0/3813013734 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc030066690 msgr2=0x7fc030068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:52.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.426+0000 7fc058e00640 1 --2- 172.21.15.161:0/3813013734 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc030066690 0x7fc030068b50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc0440ad000 tx=0x7fc044008040 comp rx=0 tx=0).stop 2024-05-22T21:13:52.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.426+0000 7fc058e00640 1 -- 172.21.15.161:0/3813013734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc05413fe50 msgr2=0x7fc05406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:52.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.426+0000 7fc058e00640 1 --2- 172.21.15.161:0/3813013734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc05413fe50 0x7fc05406cb30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00c3c0 tx=0x7fc03c00c880 comp rx=0 tx=0).stop 2024-05-22T21:13:52.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.426+0000 7fc058e00640 1 -- 172.21.15.161:0/3813013734 shutdown_connections 2024-05-22T21:13:52.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.426+0000 7fc058e00640 1 --2- 172.21.15.161:0/3813013734 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc030066690 0x7fc030068b50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.426+0000 7fc058e00640 1 --2- 172.21.15.161:0/3813013734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc05415af60 0x7fc05406d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.426+0000 7fc058e00640 1 --2- 172.21.15.161:0/3813013734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc05413fe50 0x7fc05406cb30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.426+0000 7fc058e00640 1 --2- 172.21.15.161:0/3813013734 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc05413cfc0 0x7fc054157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.426+0000 7fc058e00640 1 -- 172.21.15.161:0/3813013734 >> 172.21.15.161:0/3813013734 conn(0x7fc05406a2d0 msgr2=0x7fc05406a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:52.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.426+0000 7fc058e00640 1 -- 172.21.15.161:0/3813013734 shutdown_connections 2024-05-22T21:13:52.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.426+0000 7fc058e00640 1 -- 172.21.15.161:0/3813013734 wait complete. 2024-05-22T21:13:52.448 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:13:52.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.582+0000 7f9e52c00640 1 Processor -- start 2024-05-22T21:13:52.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.582+0000 7f9e52c00640 1 -- start start 2024-05-22T21:13:52.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.582+0000 7f9e52c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9e4c143c40 0x7f9e4c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.582+0000 7f9e52c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9e4c1444a0 0x7f9e4c06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.582+0000 7f9e52c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4c15efb0 con 0x7f9e4c144900 2024-05-22T21:13:52.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.582+0000 7f9e52c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4c15f290 con 0x7f9e4c143c40 2024-05-22T21:13:52.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.582+0000 7f9e52c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4c15f570 con 0x7f9e4c1444a0 2024-05-22T21:13:52.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.582+0000 7f9e4be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9e4c143c40 0x7f9e4c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.586+0000 7f9e4be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9e4c143c40 0x7f9e4c15b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46912/0 (socket says 172.21.15.161:46912) 2024-05-22T21:13:52.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.586+0000 7f9e4be00640 1 -- 172.21.15.161:0/2868926481 learned_addr learned my addr 172.21.15.161:0/2868926481 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:52.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.586+0000 7f9e43e00640 1 --2- 172.21.15.161:0/2868926481 >> v2:172.21.15.179:3300/0 conn(0x7f9e4c1444a0 0x7f9e4c06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.586+0000 7f9e4be00640 1 -- 172.21.15.161:0/2868926481 >> v2:172.21.15.179:3300/0 conn(0x7f9e4c1444a0 msgr2=0x7f9e4c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:52.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.586+0000 7f9e4be00640 1 --2- 172.21.15.161:0/2868926481 >> v2:172.21.15.179:3300/0 conn(0x7f9e4c1444a0 0x7f9e4c06c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.586+0000 7f9e4be00640 1 -- 172.21.15.161:0/2868926481 >> v1:172.21.15.179:6789/0 conn(0x7f9e4c144900 legacy=0x7f9e4c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:52.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.586+0000 7f9e4be00640 1 -- 172.21.15.161:0/2868926481 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c15f850 con 0x7f9e4c143c40 2024-05-22T21:13:52.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.586+0000 7f9e43e00640 1 --2- 172.21.15.161:0/2868926481 >> v2:172.21.15.179:3300/0 conn(0x7f9e4c1444a0 0x7f9e4c06c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:52.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.586+0000 7f9e4be00640 1 --2- 172.21.15.161:0/2868926481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c143c40 0x7f9e4c15b600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0b1c80 tx=0x7f9e3c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdd3f6ced6b8c08 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:52.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.586+0000 7f9e4b400640 1 -- 172.21.15.161:0/2868926481 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0ab5f0 con 0x7f9e4c143c40 2024-05-22T21:13:52.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.586+0000 7f9e4b400640 1 -- 172.21.15.161:0/2868926481 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c0ab750 con 0x7f9e4c143c40 2024-05-22T21:13:52.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.586+0000 7f9e4b400640 1 -- 172.21.15.161:0/2868926481 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0abae0 con 0x7f9e4c143c40 2024-05-22T21:13:52.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e52c00640 1 -- 172.21.15.161:0/2868926481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c143c40 msgr2=0x7f9e4c15b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:52.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e52c00640 1 --2- 172.21.15.161:0/2868926481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c143c40 0x7f9e4c15b600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0b1c80 tx=0x7f9e3c01c4e0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e52c00640 1 -- 172.21.15.161:0/2868926481 shutdown_connections 2024-05-22T21:13:52.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e52c00640 1 --2- 172.21.15.161:0/2868926481 >> v2:172.21.15.179:3300/0 conn(0x7f9e4c1444a0 0x7f9e4c06c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e52c00640 1 --2- 172.21.15.161:0/2868926481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c143c40 0x7f9e4c15b600 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e52c00640 1 -- 172.21.15.161:0/2868926481 >> 172.21.15.161:0/2868926481 conn(0x7f9e4c06a2d0 msgr2=0x7f9e4c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:52.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e52c00640 1 -- 172.21.15.161:0/2868926481 shutdown_connections 2024-05-22T21:13:52.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e52c00640 1 -- 172.21.15.161:0/2868926481 wait complete. 2024-05-22T21:13:52.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e52c00640 1 Processor -- start 2024-05-22T21:13:52.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e52c00640 1 -- start start 2024-05-22T21:13:52.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e52c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c143c40 0x7f9e4c139800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e52c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e4c1444a0 0x7f9e4c15db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e52c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e4c144900 0x7f9e4c1531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e52c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c13a2f0 con 0x7f9e4c143c40 2024-05-22T21:13:52.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e52c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c15a060 con 0x7f9e4c1444a0 2024-05-22T21:13:52.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e52c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c139d40 con 0x7f9e4c144900 2024-05-22T21:13:52.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e4be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c143c40 0x7f9e4c139800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e4be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c143c40 0x7f9e4c139800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46924/0 (socket says 172.21.15.161:46924) 2024-05-22T21:13:52.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e4be00640 1 -- 172.21.15.161:0/1638641499 learned_addr learned my addr 172.21.15.161:0/1638641499 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:52.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e43e00640 1 --2- 172.21.15.161:0/1638641499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e4c1444a0 0x7f9e4c15db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e50c00640 1 --2- 172.21.15.161:0/1638641499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e4c144900 0x7f9e4c1531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e4be00640 1 -- 172.21.15.161:0/1638641499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e4c144900 msgr2=0x7f9e4c1531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:52.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e4be00640 1 --2- 172.21.15.161:0/1638641499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e4c144900 0x7f9e4c1531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e4be00640 1 -- 172.21.15.161:0/1638641499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e4c1444a0 msgr2=0x7f9e4c15db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:52.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e4be00640 1 --2- 172.21.15.161:0/1638641499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e4c1444a0 0x7f9e4c15db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e4be00640 1 -- 172.21.15.161:0/1638641499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0094d0 con 0x7f9e4c143c40 2024-05-22T21:13:52.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.590+0000 7f9e43e00640 1 --2- 172.21.15.161:0/1638641499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e4c1444a0 0x7f9e4c15db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:52.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.594+0000 7f9e4be00640 1 --2- 172.21.15.161:0/1638641499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c143c40 0x7f9e4c139800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c01a110 tx=0x7f9e3c01ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:52.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.594+0000 7f9e48c00640 1 -- 172.21.15.161:0/1638641499 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0b4800 con 0x7f9e4c143c40 2024-05-22T21:13:52.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.594+0000 7f9e48c00640 1 -- 172.21.15.161:0/1638641499 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c0b4960 con 0x7f9e4c143c40 2024-05-22T21:13:52.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.594+0000 7f9e52c00640 1 -- 172.21.15.161:0/1638641499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4c159450 con 0x7f9e4c143c40 2024-05-22T21:13:52.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.594+0000 7f9e48c00640 1 -- 172.21.15.161:0/1638641499 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0b4d10 con 0x7f9e4c143c40 2024-05-22T21:13:52.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.594+0000 7f9e52c00640 1 -- 172.21.15.161:0/1638641499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4c159a10 con 0x7f9e4c143c40 2024-05-22T21:13:52.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.594+0000 7f9e52c00640 1 -- 172.21.15.161:0/1638641499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4c067220 con 0x7f9e4c143c40 2024-05-22T21:13:52.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.594+0000 7f9e48c00640 1 -- 172.21.15.161:0/1638641499 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9e3c0b3380 con 0x7f9e4c143c40 2024-05-22T21:13:52.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.594+0000 7f9e48c00640 1 --2- 172.21.15.161:0/1638641499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e28066690 0x7f9e28068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.594+0000 7f9e48c00640 1 -- 172.21.15.161:0/1638641499 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e3c0a8050 con 0x7f9e4c143c40 2024-05-22T21:13:52.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.594+0000 7f9e43e00640 1 --2- 172.21.15.161:0/1638641499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e28066690 0x7f9e28068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.594+0000 7f9e43e00640 1 --2- 172.21.15.161:0/1638641499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e28066690 0x7f9e28068b50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9e34006fd0 tx=0x7f9e34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:52.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.598+0000 7f9e48c00640 1 -- 172.21.15.161:0/1638641499 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9e3c0ebc50 con 0x7f9e4c143c40 2024-05-22T21:13:52.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.750+0000 7f9e52c00640 1 -- 172.21.15.161:0/1638641499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e4c0ceaa0 con 0x7f9e4c143c40 2024-05-22T21:13:52.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.750+0000 7f9e48c00640 1 -- 172.21.15.161:0/1638641499 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e3c0ef270 con 0x7f9e4c143c40 2024-05-22T21:13:52.755 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:13:52.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.754+0000 7f9e52c00640 1 -- 172.21.15.161:0/1638641499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e28066690 msgr2=0x7f9e28068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:52.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.754+0000 7f9e52c00640 1 --2- 172.21.15.161:0/1638641499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e28066690 0x7f9e28068b50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9e34006fd0 tx=0x7f9e34008040 comp rx=0 tx=0).stop 2024-05-22T21:13:52.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.754+0000 7f9e52c00640 1 -- 172.21.15.161:0/1638641499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c143c40 msgr2=0x7f9e4c139800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:52.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.754+0000 7f9e52c00640 1 --2- 172.21.15.161:0/1638641499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c143c40 0x7f9e4c139800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c01a110 tx=0x7f9e3c01ad20 comp rx=0 tx=0).stop 2024-05-22T21:13:52.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.754+0000 7f9e52c00640 1 -- 172.21.15.161:0/1638641499 shutdown_connections 2024-05-22T21:13:52.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.754+0000 7f9e52c00640 1 --2- 172.21.15.161:0/1638641499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e28066690 0x7f9e28068b50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.754+0000 7f9e52c00640 1 --2- 172.21.15.161:0/1638641499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e4c144900 0x7f9e4c1531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.754+0000 7f9e52c00640 1 --2- 172.21.15.161:0/1638641499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e4c1444a0 0x7f9e4c15db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.754+0000 7f9e52c00640 1 --2- 172.21.15.161:0/1638641499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c143c40 0x7f9e4c139800 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.754+0000 7f9e52c00640 1 -- 172.21.15.161:0/1638641499 >> 172.21.15.161:0/1638641499 conn(0x7f9e4c06a2d0 msgr2=0x7f9e4c15be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:52.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.754+0000 7f9e52c00640 1 -- 172.21.15.161:0/1638641499 shutdown_connections 2024-05-22T21:13:52.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.754+0000 7f9e52c00640 1 -- 172.21.15.161:0/1638641499 wait complete. 2024-05-22T21:13:52.776 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:13:52.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.910+0000 7f1fb0e00640 1 Processor -- start 2024-05-22T21:13:52.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.910+0000 7f1fb0e00640 1 -- start start 2024-05-22T21:13:52.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.910+0000 7f1fb0e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1fac13cfc0 0x7f1fac1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.910+0000 7f1fb0e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1fac13fe50 0x7f1fac06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.910+0000 7f1fb0e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fac13d3a0 con 0x7f1fac15af60 2024-05-22T21:13:52.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.910+0000 7f1fb0e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fac0cd920 con 0x7f1fac13cfc0 2024-05-22T21:13:52.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.910+0000 7f1fb0e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fac0cdc00 con 0x7f1fac13fe50 2024-05-22T21:13:52.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.910+0000 7f1fa9e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1fac13cfc0 0x7f1fac1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.910+0000 7f1fa9e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1fac13cfc0 0x7f1fac1402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46936/0 (socket says 172.21.15.161:46936) 2024-05-22T21:13:52.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.910+0000 7f1fa9e00640 1 -- 172.21.15.161:0/3754544034 learned_addr learned my addr 172.21.15.161:0/3754544034 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:52.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.910+0000 7f1faa800640 1 --2- 172.21.15.161:0/3754544034 >> v2:172.21.15.179:3300/0 conn(0x7f1fac13fe50 0x7f1fac06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.914+0000 7f1fa9e00640 1 -- 172.21.15.161:0/3754544034 >> v2:172.21.15.179:3300/0 conn(0x7f1fac13fe50 msgr2=0x7f1fac06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:52.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.914+0000 7f1fa9e00640 1 --2- 172.21.15.161:0/3754544034 >> v2:172.21.15.179:3300/0 conn(0x7f1fac13fe50 0x7f1fac06fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.914+0000 7f1fa9e00640 1 -- 172.21.15.161:0/3754544034 >> v1:172.21.15.179:6789/0 conn(0x7f1fac15af60 legacy=0x7f1fac13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:52.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.914+0000 7f1fa9e00640 1 -- 172.21.15.161:0/3754544034 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac0cdee0 con 0x7f1fac13cfc0 2024-05-22T21:13:52.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.914+0000 7f1fa9e00640 1 --2- 172.21.15.161:0/3754544034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fac13cfc0 0x7f1fac1402e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1fac06c5e0 tx=0x7f1f9c01e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27074ed40573f09d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:52.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.914+0000 7f1fa9400640 1 -- 172.21.15.161:0/3754544034 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0ad600 con 0x7f1fac13cfc0 2024-05-22T21:13:52.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.914+0000 7f1fa9400640 1 -- 172.21.15.161:0/3754544034 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c0a9050 con 0x7f1fac13cfc0 2024-05-22T21:13:52.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.914+0000 7f1fa9400640 1 -- 172.21.15.161:0/3754544034 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0ad980 con 0x7f1fac13cfc0 2024-05-22T21:13:52.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.914+0000 7f1fb0e00640 1 -- 172.21.15.161:0/3754544034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fac13cfc0 msgr2=0x7f1fac1402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:52.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.914+0000 7f1fb0e00640 1 --2- 172.21.15.161:0/3754544034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fac13cfc0 0x7f1fac1402e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1fac06c5e0 tx=0x7f1f9c01e670 comp rx=0 tx=0).stop 2024-05-22T21:13:52.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.914+0000 7f1fb0e00640 1 -- 172.21.15.161:0/3754544034 shutdown_connections 2024-05-22T21:13:52.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.914+0000 7f1fb0e00640 1 --2- 172.21.15.161:0/3754544034 >> v2:172.21.15.179:3300/0 conn(0x7f1fac13fe50 0x7f1fac06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.914+0000 7f1fb0e00640 1 --2- 172.21.15.161:0/3754544034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fac13cfc0 0x7f1fac1402e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.914+0000 7f1fb0e00640 1 -- 172.21.15.161:0/3754544034 >> 172.21.15.161:0/3754544034 conn(0x7f1fac06a2d0 msgr2=0x7f1fac06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:52.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.914+0000 7f1fb0e00640 1 -- 172.21.15.161:0/3754544034 shutdown_connections 2024-05-22T21:13:52.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.914+0000 7f1fb0e00640 1 -- 172.21.15.161:0/3754544034 wait complete. 2024-05-22T21:13:52.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1fb0e00640 1 Processor -- start 2024-05-22T21:13:52.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1fb0e00640 1 -- start start 2024-05-22T21:13:52.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1fb0e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fac13cfc0 0x7f1fac1575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1fb0e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fac13fe50 0x7f1fac0ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1fb0e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fac15af60 0x7f1fac13b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1fb0e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0b2180 con 0x7f1fac15af60 2024-05-22T21:13:52.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1fb0e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0b2420 con 0x7f1fac13cfc0 2024-05-22T21:13:52.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1fb0e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0ada80 con 0x7f1fac13fe50 2024-05-22T21:13:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1faa800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fac15af60 0x7f1fac13b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1faa800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fac15af60 0x7f1fac13b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46938/0 (socket says 172.21.15.161:46938) 2024-05-22T21:13:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1faa800640 1 -- 172.21.15.161:0/3185982043 learned_addr learned my addr 172.21.15.161:0/3185982043 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1fa9e00640 1 --2- 172.21.15.161:0/3185982043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fac13cfc0 0x7f1fac1575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1fa1400640 1 --2- 172.21.15.161:0/3185982043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fac13fe50 0x7f1fac0ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1faa800640 1 -- 172.21.15.161:0/3185982043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fac13fe50 msgr2=0x7f1fac0ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1faa800640 1 --2- 172.21.15.161:0/3185982043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fac13fe50 0x7f1fac0ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1faa800640 1 -- 172.21.15.161:0/3185982043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fac13cfc0 msgr2=0x7f1fac1575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1faa800640 1 --2- 172.21.15.161:0/3185982043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fac13cfc0 0x7f1fac1575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1faa800640 1 -- 172.21.15.161:0/3185982043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c0094d0 con 0x7f1fac15af60 2024-05-22T21:13:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1faa800640 1 --2- 172.21.15.161:0/3185982043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fac15af60 0x7f1fac13b920 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4007e40 tx=0x7f1fa400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1fa2a00640 1 -- 172.21.15.161:0/3185982043 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4014620 con 0x7f1fac15af60 2024-05-22T21:13:52.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1fa2a00640 1 -- 172.21.15.161:0/3185982043 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa400ee90 con 0x7f1fac15af60 2024-05-22T21:13:52.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1fa2a00640 1 -- 172.21.15.161:0/3185982043 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa40149e0 con 0x7f1fac15af60 2024-05-22T21:13:52.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1fb0e00640 1 -- 172.21.15.161:0/3185982043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fac0add60 con 0x7f1fac15af60 2024-05-22T21:13:52.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.918+0000 7f1fb0e00640 1 -- 172.21.15.161:0/3185982043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fac157b10 con 0x7f1fac15af60 2024-05-22T21:13:52.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.922+0000 7f1fb0e00640 1 -- 172.21.15.161:0/3185982043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fac0b25b0 con 0x7f1fac15af60 2024-05-22T21:13:52.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.922+0000 7f1fa2a00640 1 -- 172.21.15.161:0/3185982043 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1fa4014b40 con 0x7f1fac15af60 2024-05-22T21:13:52.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.922+0000 7f1fa2a00640 1 --2- 172.21.15.161:0/3185982043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f880666c0 0x7f1f88068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:52.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.922+0000 7f1fa2a00640 1 -- 172.21.15.161:0/3185982043 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fa4084a90 con 0x7f1fac15af60 2024-05-22T21:13:52.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.922+0000 7f1fa9e00640 1 --2- 172.21.15.161:0/3185982043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f880666c0 0x7f1f88068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:52.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.922+0000 7f1fa9e00640 1 --2- 172.21.15.161:0/3185982043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f880666c0 0x7f1f88068b80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0af000 tx=0x7f1f9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:52.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:52.926+0000 7f1fa2a00640 1 -- 172.21.15.161:0/3185982043 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1fa4051e10 con 0x7f1fac15af60 2024-05-22T21:13:52.944 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:13:52.947 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:13:53.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.078+0000 7f1fb0e00640 1 -- 172.21.15.161:0/3185982043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1fac067220 con 0x7f1fac15af60 2024-05-22T21:13:53.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.078+0000 7f1fa2a00640 1 -- 172.21.15.161:0/3185982043 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1fa4055430 con 0x7f1fac15af60 2024-05-22T21:13:53.084 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:13:53.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.082+0000 7f1fb0e00640 1 -- 172.21.15.161:0/3185982043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f880666c0 msgr2=0x7f1f88068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.082+0000 7f1fb0e00640 1 --2- 172.21.15.161:0/3185982043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f880666c0 0x7f1f88068b80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0af000 tx=0x7f1f9c008040 comp rx=0 tx=0).stop 2024-05-22T21:13:53.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.082+0000 7f1fb0e00640 1 -- 172.21.15.161:0/3185982043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fac15af60 msgr2=0x7f1fac13b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.082+0000 7f1fb0e00640 1 --2- 172.21.15.161:0/3185982043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fac15af60 0x7f1fac13b920 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4007e40 tx=0x7f1fa400e4b0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.082+0000 7f1fb0e00640 1 -- 172.21.15.161:0/3185982043 shutdown_connections 2024-05-22T21:13:53.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.082+0000 7f1fb0e00640 1 --2- 172.21.15.161:0/3185982043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f880666c0 0x7f1f88068b80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.082+0000 7f1fb0e00640 1 --2- 172.21.15.161:0/3185982043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fac15af60 0x7f1fac13b920 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.082+0000 7f1fb0e00640 1 --2- 172.21.15.161:0/3185982043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fac13fe50 0x7f1fac0ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.082+0000 7f1fb0e00640 1 --2- 172.21.15.161:0/3185982043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fac13cfc0 0x7f1fac1575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.082+0000 7f1fb0e00640 1 -- 172.21.15.161:0/3185982043 >> 172.21.15.161:0/3185982043 conn(0x7f1fac06a2d0 msgr2=0x7f1fac15da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:53.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.082+0000 7f1fb0e00640 1 -- 172.21.15.161:0/3185982043 shutdown_connections 2024-05-22T21:13:53.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.082+0000 7f1fb0e00640 1 -- 172.21.15.161:0/3185982043 wait complete. 2024-05-22T21:13:53.105 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:53.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.234+0000 7f52fb600640 1 Processor -- start 2024-05-22T21:13:53.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.234+0000 7f52fb600640 1 -- start start 2024-05-22T21:13:53.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52fb600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f4143ec0 0x7f52f415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52fb600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f41442a0 con 0x7f52f40715a0 2024-05-22T21:13:53.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52fb600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f4070500 con 0x7f52f415ec90 2024-05-22T21:13:53.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52fb600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40707e0 con 0x7f52f4143ec0 2024-05-22T21:13:53.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52ebe00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f52f40715a0 0x7f52f406fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:52022/0 (socket says 172.21.15.161:52022) 2024-05-22T21:13:53.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52ebe00640 1 -- 172.21.15.161:0/2749161692 learned_addr learned my addr 172.21.15.161:0/2749161692 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:53.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f3e00640 1 -- 172.21.15.161:0/2749161692 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87543717 0 0) 0x7f52f41442a0 con 0x7f52f40715a0 2024-05-22T21:13:53.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f8c00640 1 --2- 172.21.15.161:0/2749161692 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f4143ec0 0x7f52f415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f3e00640 1 -- 172.21.15.161:0/2749161692 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52d8003530 con 0x7f52f40715a0 2024-05-22T21:13:53.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f3e00640 1 -- 172.21.15.161:0/2749161692 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 610074437 0 0) 0x7f52d8003530 con 0x7f52f40715a0 2024-05-22T21:13:53.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f3e00640 1 -- 172.21.15.161:0/2749161692 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f4143ec0 msgr2=0x7f52f415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:53.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f3e00640 1 --2- 172.21.15.161:0/2749161692 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f4143ec0 0x7f52f415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f3e00640 1 -- 172.21.15.161:0/2749161692 >> v1:172.21.15.179:6789/0 conn(0x7f52f415ec90 legacy=0x7f52f40cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f3e00640 1 -- 172.21.15.161:0/2749161692 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f4070ac0 con 0x7f52f40715a0 2024-05-22T21:13:53.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f3e00640 1 -- 172.21.15.161:0/2749161692 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f52dc001680 con 0x7f52f40715a0 2024-05-22T21:13:53.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f3e00640 1 -- 172.21.15.161:0/2749161692 >> v1:172.21.15.161:6789/0 conn(0x7f52f40715a0 legacy=0x7f52f406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f3e00640 1 --2- 172.21.15.161:0/2749161692 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f4143ec0 0x7f52d80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f3e00640 1 --2- 172.21.15.161:0/2749161692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d8003f30 0x7f52d80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f3e00640 1 --2- 172.21.15.161:0/2749161692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d8004d10 0x7f52d8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f3e00640 1 -- 172.21.15.161:0/2749161692 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40707e0 con 0x7f52d8003f30 2024-05-22T21:13:53.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f3e00640 1 -- 172.21.15.161:0/2749161692 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d800ba70 con 0x7f52d8004d10 2024-05-22T21:13:53.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f3e00640 1 -- 172.21.15.161:0/2749161692 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d800bc30 con 0x7f52f4143ec0 2024-05-22T21:13:53.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52ebe00640 1 --2- 172.21.15.161:0/2749161692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d8003f30 0x7f52d80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f8c00640 1 --2- 172.21.15.161:0/2749161692 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f4143ec0 0x7f52d80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52f9600640 1 --2- 172.21.15.161:0/2749161692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d8004d10 0x7f52d8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52ebe00640 1 -- 172.21.15.161:0/2749161692 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f4143ec0 msgr2=0x7f52d80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:53.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52ebe00640 1 --2- 172.21.15.161:0/2749161692 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f4143ec0 0x7f52d80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52ebe00640 1 -- 172.21.15.161:0/2749161692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d8004d10 msgr2=0x7f52d8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:53.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52ebe00640 1 --2- 172.21.15.161:0/2749161692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d8004d10 0x7f52d8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.238+0000 7f52ebe00640 1 -- 172.21.15.161:0/2749161692 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d8003530 con 0x7f52d8003f30 2024-05-22T21:13:53.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.242+0000 7f52ebe00640 1 --2- 172.21.15.161:0/2749161692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d8003f30 0x7f52d80047d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00c210 tx=0x7f52dc01f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd7b544561d3816 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:53.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.242+0000 7f52f3e00640 1 -- 172.21.15.161:0/2749161692 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc005ae0 con 0x7f52d8003f30 2024-05-22T21:13:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.242+0000 7f52f3e00640 1 -- 172.21.15.161:0/2749161692 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc0ab070 con 0x7f52d8003f30 2024-05-22T21:13:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.242+0000 7f52f3e00640 1 -- 172.21.15.161:0/2749161692 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc007070 con 0x7f52d8003f30 2024-05-22T21:13:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.242+0000 7f52fb600640 1 -- 172.21.15.161:0/2749161692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d8003f30 msgr2=0x7f52d80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.242+0000 7f52fb600640 1 --2- 172.21.15.161:0/2749161692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d8003f30 0x7f52d80047d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00c210 tx=0x7f52dc01f390 comp rx=0 tx=0).stop 2024-05-22T21:13:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.242+0000 7f52f9600640 1 -- 172.21.15.161:0/2749161692 reap_dead start 2024-05-22T21:13:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.242+0000 7f52fb600640 1 -- 172.21.15.161:0/2749161692 shutdown_connections 2024-05-22T21:13:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.242+0000 7f52fb600640 1 -- 172.21.15.161:0/2749161692 >> 172.21.15.161:0/2749161692 conn(0x7f52f40b0bc0 msgr2=0x7f52f40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.242+0000 7f52fb600640 1 -- 172.21.15.161:0/2749161692 shutdown_connections 2024-05-22T21:13:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.242+0000 7f52fb600640 1 -- 172.21.15.161:0/2749161692 wait complete. 2024-05-22T21:13:53.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52fb600640 1 Processor -- start 2024-05-22T21:13:53.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52fb600640 1 -- start start 2024-05-22T21:13:53.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52fb600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d8003f30 0x7f52f415e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52fb600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f415bdd0 0x7f52f4142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52fb600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52f4142870 0x7f52f4151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52fb600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40ad210 con 0x7f52f4142870 2024-05-22T21:13:53.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52fb600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40ac950 con 0x7f52d8003f30 2024-05-22T21:13:53.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52fb600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40acbc0 con 0x7f52f415bdd0 2024-05-22T21:13:53.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52f9600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52f4142870 0x7f52f4151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52f9600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52f4142870 0x7f52f4151730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46960/0 (socket says 172.21.15.161:46960) 2024-05-22T21:13:53.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52f9600640 1 -- 172.21.15.161:0/3159867987 learned_addr learned my addr 172.21.15.161:0/3159867987 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:53.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52f9600640 1 -- 172.21.15.161:0/3159867987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f415bdd0 msgr2=0x7f52f4142330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52ebe00640 1 --2- 172.21.15.161:0/3159867987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f415bdd0 0x7f52f4142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52f9600640 1 --2- 172.21.15.161:0/3159867987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f415bdd0 0x7f52f4142330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52f9600640 1 -- 172.21.15.161:0/3159867987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d8003f30 msgr2=0x7f52f415e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52f8c00640 1 --2- 172.21.15.161:0/3159867987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d8003f30 0x7f52f415e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52f9600640 1 --2- 172.21.15.161:0/3159867987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d8003f30 0x7f52f415e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52f9600640 1 -- 172.21.15.161:0/3159867987 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f40acea0 con 0x7f52f4142870 2024-05-22T21:13:53.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52f8c00640 1 --2- 172.21.15.161:0/3159867987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d8003f30 0x7f52f415e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:53.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52f9600640 1 --2- 172.21.15.161:0/3159867987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52f4142870 0x7f52f4151730 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f52f406fcb0 tx=0x7f52ec00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:53.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52f1600640 1 -- 172.21.15.161:0/3159867987 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec003980 con 0x7f52f4142870 2024-05-22T21:13:53.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52f1600640 1 -- 172.21.15.161:0/3159867987 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec003ae0 con 0x7f52f4142870 2024-05-22T21:13:53.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52f1600640 1 -- 172.21.15.161:0/3159867987 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec003e90 con 0x7f52f4142870 2024-05-22T21:13:53.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52fb600640 1 -- 172.21.15.161:0/3159867987 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f4159fc0 con 0x7f52f4142870 2024-05-22T21:13:53.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52fb600640 1 -- 172.21.15.161:0/3159867987 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f415c340 con 0x7f52f4142870 2024-05-22T21:13:53.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.246+0000 7f52fb600640 1 -- 172.21.15.161:0/3159867987 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f40ad380 con 0x7f52f4142870 2024-05-22T21:13:53.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.250+0000 7f52f1600640 1 -- 172.21.15.161:0/3159867987 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f52ec0050d0 con 0x7f52f4142870 2024-05-22T21:13:53.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.250+0000 7f52f1600640 1 --2- 172.21.15.161:0/3159867987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52d0066690 0x7f52d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.250+0000 7f52f1600640 1 -- 172.21.15.161:0/3159867987 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52ec083ff0 con 0x7f52f4142870 2024-05-22T21:13:53.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.250+0000 7f52f8c00640 1 --2- 172.21.15.161:0/3159867987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52d0066690 0x7f52d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.250+0000 7f52f8c00640 1 --2- 172.21.15.161:0/3159867987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52d0066690 0x7f52d0068b50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009ad0 tx=0x7f52e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:53.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.254+0000 7f52f1600640 1 -- 172.21.15.161:0/3159867987 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f52ec0512e0 con 0x7f52f4142870 2024-05-22T21:13:53.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.414+0000 7f52fb600640 1 -- 172.21.15.161:0/3159867987 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52f4067220 con 0x7f52f4142870 2024-05-22T21:13:53.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.414+0000 7f52f1600640 1 -- 172.21.15.161:0/3159867987 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f52ec054900 con 0x7f52f4142870 2024-05-22T21:13:53.418 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:53.418 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:53.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.414+0000 7f52fb600640 1 -- 172.21.15.161:0/3159867987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52d0066690 msgr2=0x7f52d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.414+0000 7f52fb600640 1 --2- 172.21.15.161:0/3159867987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52d0066690 0x7f52d0068b50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009ad0 tx=0x7f52e4008040 comp rx=0 tx=0).stop 2024-05-22T21:13:53.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.414+0000 7f52fb600640 1 -- 172.21.15.161:0/3159867987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52f4142870 msgr2=0x7f52f4151730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.414+0000 7f52fb600640 1 --2- 172.21.15.161:0/3159867987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52f4142870 0x7f52f4151730 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f52f406fcb0 tx=0x7f52ec00ec10 comp rx=0 tx=0).stop 2024-05-22T21:13:53.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.414+0000 7f52fb600640 1 -- 172.21.15.161:0/3159867987 shutdown_connections 2024-05-22T21:13:53.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.414+0000 7f52fb600640 1 --2- 172.21.15.161:0/3159867987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52d0066690 0x7f52d0068b50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.414+0000 7f52fb600640 1 --2- 172.21.15.161:0/3159867987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52f4142870 0x7f52f4151730 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.414+0000 7f52fb600640 1 --2- 172.21.15.161:0/3159867987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f415bdd0 0x7f52f4142330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.414+0000 7f52fb600640 1 --2- 172.21.15.161:0/3159867987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d8003f30 0x7f52f415e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.414+0000 7f52fb600640 1 -- 172.21.15.161:0/3159867987 >> 172.21.15.161:0/3159867987 conn(0x7f52f40b0bc0 msgr2=0x7f52f4140470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:53.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.414+0000 7f52fb600640 1 -- 172.21.15.161:0/3159867987 shutdown_connections 2024-05-22T21:13:53.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.414+0000 7f52fb600640 1 -- 172.21.15.161:0/3159867987 wait complete. 2024-05-22T21:13:53.440 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:13:21.237503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:13:22.425891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:13:53.440 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:13:53.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d3ea00640 1 Processor -- start 2024-05-22T21:13:53.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d3ea00640 1 -- start start 2024-05-22T21:13:53.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d3ea00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d38143c40 0x7f4d3815b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d3ea00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4d381444a0 0x7f4d3806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d3ea00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3815efb0 con 0x7f4d38144900 2024-05-22T21:13:53.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d3ea00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3815f290 con 0x7f4d381444a0 2024-05-22T21:13:53.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d3ea00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3815f570 con 0x7f4d38143c40 2024-05-22T21:13:53.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d3ca00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f4d38144900 0x7f4d380b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:52028/0 (socket says 172.21.15.161:52028) 2024-05-22T21:13:53.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d3ca00640 1 -- 172.21.15.161:0/3271817249 learned_addr learned my addr 172.21.15.161:0/3271817249 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:53.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d2f400640 1 --2- 172.21.15.161:0/3271817249 >> v2:172.21.15.179:3300/0 conn(0x7f4d381444a0 0x7f4d3806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d37e00640 1 --2- 172.21.15.161:0/3271817249 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d38143c40 0x7f4d3815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d37400640 1 -- 172.21.15.161:0/3271817249 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1777591502 0 0) 0x7f4d3815efb0 con 0x7f4d38144900 2024-05-22T21:13:53.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d37400640 1 -- 172.21.15.161:0/3271817249 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d1c003530 con 0x7f4d38144900 2024-05-22T21:13:53.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d37400640 1 -- 172.21.15.161:0/3271817249 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1046204098 0 0) 0x7f4d1c003530 con 0x7f4d38144900 2024-05-22T21:13:53.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d37400640 1 -- 172.21.15.161:0/3271817249 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d38143c40 msgr2=0x7f4d3815b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:53.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d37400640 1 --2- 172.21.15.161:0/3271817249 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d38143c40 0x7f4d3815b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d37400640 1 -- 172.21.15.161:0/3271817249 >> v2:172.21.15.179:3300/0 conn(0x7f4d381444a0 msgr2=0x7f4d3806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:53.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d37400640 1 --2- 172.21.15.161:0/3271817249 >> v2:172.21.15.179:3300/0 conn(0x7f4d381444a0 0x7f4d3806c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d37400640 1 -- 172.21.15.161:0/3271817249 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3815f850 con 0x7f4d38144900 2024-05-22T21:13:53.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d37400640 1 -- 172.21.15.161:0/3271817249 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f4d30001a00 con 0x7f4d38144900 2024-05-22T21:13:53.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d37400640 1 -- 172.21.15.161:0/3271817249 >> v1:172.21.15.161:6789/0 conn(0x7f4d38144900 legacy=0x7f4d380b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d37400640 1 --2- 172.21.15.161:0/3271817249 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d38143c40 0x7f4d1c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.578+0000 7f4d37400640 1 --2- 172.21.15.161:0/3271817249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d1c003f00 0x7f4d1c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.582+0000 7f4d37400640 1 --2- 172.21.15.161:0/3271817249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d1c004d10 0x7f4d1c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.582+0000 7f4d37400640 1 -- 172.21.15.161:0/3271817249 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3815f290 con 0x7f4d1c003f00 2024-05-22T21:13:53.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.582+0000 7f4d37400640 1 -- 172.21.15.161:0/3271817249 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3815f570 con 0x7f4d1c004d10 2024-05-22T21:13:53.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.582+0000 7f4d37400640 1 -- 172.21.15.161:0/3271817249 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c00b990 con 0x7f4d38143c40 2024-05-22T21:13:53.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.582+0000 7f4d2f400640 1 --2- 172.21.15.161:0/3271817249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d1c003f00 0x7f4d1c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.582+0000 7f4d37e00640 1 --2- 172.21.15.161:0/3271817249 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d38143c40 0x7f4d1c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.582+0000 7f4d3ca00640 1 --2- 172.21.15.161:0/3271817249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d1c004d10 0x7f4d1c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.582+0000 7f4d2f400640 1 -- 172.21.15.161:0/3271817249 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d38143c40 msgr2=0x7f4d1c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:53.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.582+0000 7f4d2f400640 1 --2- 172.21.15.161:0/3271817249 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d38143c40 0x7f4d1c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.582+0000 7f4d2f400640 1 -- 172.21.15.161:0/3271817249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d1c004d10 msgr2=0x7f4d1c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:53.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.582+0000 7f4d2f400640 1 --2- 172.21.15.161:0/3271817249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d1c004d10 0x7f4d1c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.582+0000 7f4d2f400640 1 -- 172.21.15.161:0/3271817249 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c003530 con 0x7f4d1c003f00 2024-05-22T21:13:53.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.582+0000 7f4d2f400640 1 --2- 172.21.15.161:0/3271817249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d1c003f00 0x7f4d1c0047d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009ae0 tx=0x7f4d2001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72c687acc503fb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:53.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.582+0000 7f4d37400640 1 -- 172.21.15.161:0/3271817249 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d200a5590 con 0x7f4d1c003f00 2024-05-22T21:13:53.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.582+0000 7f4d37400640 1 -- 172.21.15.161:0/3271817249 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d200a56f0 con 0x7f4d1c003f00 2024-05-22T21:13:53.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.582+0000 7f4d37400640 1 -- 172.21.15.161:0/3271817249 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d200a5aa0 con 0x7f4d1c003f00 2024-05-22T21:13:53.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d3ea00640 1 -- 172.21.15.161:0/3271817249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d1c003f00 msgr2=0x7f4d1c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d3ea00640 1 --2- 172.21.15.161:0/3271817249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d1c003f00 0x7f4d1c0047d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009ae0 tx=0x7f4d2001ca50 comp rx=0 tx=0).stop 2024-05-22T21:13:53.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d3ca00640 1 -- 172.21.15.161:0/3271817249 reap_dead start 2024-05-22T21:13:53.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d3ea00640 1 -- 172.21.15.161:0/3271817249 shutdown_connections 2024-05-22T21:13:53.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d3ea00640 1 -- 172.21.15.161:0/3271817249 >> 172.21.15.161:0/3271817249 conn(0x7f4d3806a2d0 msgr2=0x7f4d3806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:53.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d3ea00640 1 -- 172.21.15.161:0/3271817249 shutdown_connections 2024-05-22T21:13:53.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d3ea00640 1 -- 172.21.15.161:0/3271817249 wait complete. 2024-05-22T21:13:53.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d3ea00640 1 Processor -- start 2024-05-22T21:13:53.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d3ea00640 1 -- start start 2024-05-22T21:13:53.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d3ea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d1c003f00 0x7f4d3815bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d3ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d3815a6a0 0x7f4d38157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d3ea00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d381420e0 0x7f4d38142560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d3ea00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3815cf70 con 0x7f4d3815a6a0 2024-05-22T21:13:53.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d3ea00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d38142aa0 con 0x7f4d1c003f00 2024-05-22T21:13:53.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d3ea00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3815c0c0 con 0x7f4d381420e0 2024-05-22T21:13:53.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d2f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d3815a6a0 0x7f4d38157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d2f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d3815a6a0 0x7f4d38157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46988/0 (socket says 172.21.15.161:46988) 2024-05-22T21:13:53.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d2f400640 1 -- 172.21.15.161:0/2801768233 learned_addr learned my addr 172.21.15.161:0/2801768233 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:53.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d37e00640 1 --2- 172.21.15.161:0/2801768233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d1c003f00 0x7f4d3815bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d3ca00640 1 --2- 172.21.15.161:0/2801768233 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d381420e0 0x7f4d38142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d2f400640 1 -- 172.21.15.161:0/2801768233 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d381420e0 msgr2=0x7f4d38142560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d2f400640 1 --2- 172.21.15.161:0/2801768233 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d381420e0 0x7f4d38142560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d2f400640 1 -- 172.21.15.161:0/2801768233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d1c003f00 msgr2=0x7f4d3815bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d2f400640 1 --2- 172.21.15.161:0/2801768233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d1c003f00 0x7f4d3815bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.586+0000 7f4d2f400640 1 -- 172.21.15.161:0/2801768233 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3815c2f0 con 0x7f4d3815a6a0 2024-05-22T21:13:53.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.590+0000 7f4d2f400640 1 --2- 172.21.15.161:0/2801768233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d3815a6a0 0x7f4d38157850 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009ae0 tx=0x7f4d1c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:53.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.590+0000 7f4d34c00640 1 -- 172.21.15.161:0/2801768233 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20004220 con 0x7f4d3815a6a0 2024-05-22T21:13:53.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.590+0000 7f4d34c00640 1 -- 172.21.15.161:0/2801768233 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d20004380 con 0x7f4d3815a6a0 2024-05-22T21:13:53.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.590+0000 7f4d34c00640 1 -- 172.21.15.161:0/2801768233 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20003820 con 0x7f4d3815a6a0 2024-05-22T21:13:53.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.590+0000 7f4d3ea00640 1 -- 172.21.15.161:0/2801768233 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3815c5d0 con 0x7f4d3815a6a0 2024-05-22T21:13:53.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.590+0000 7f4d3ea00640 1 -- 172.21.15.161:0/2801768233 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3815cb90 con 0x7f4d3815a6a0 2024-05-22T21:13:53.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.590+0000 7f4d3ea00640 1 -- 172.21.15.161:0/2801768233 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d38067220 con 0x7f4d3815a6a0 2024-05-22T21:13:53.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.590+0000 7f4d34c00640 1 -- 172.21.15.161:0/2801768233 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4d200b4040 con 0x7f4d3815a6a0 2024-05-22T21:13:53.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.590+0000 7f4d34c00640 1 --2- 172.21.15.161:0/2801768233 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4d140666c0 0x7f4d14068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.590+0000 7f4d34c00640 1 -- 172.21.15.161:0/2801768233 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d2011e520 con 0x7f4d3815a6a0 2024-05-22T21:13:53.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.590+0000 7f4d37e00640 1 --2- 172.21.15.161:0/2801768233 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4d140666c0 0x7f4d14068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.594+0000 7f4d37e00640 1 --2- 172.21.15.161:0/2801768233 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4d140666c0 0x7f4d14068b80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009b00 tx=0x7f4d28009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:53.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.594+0000 7f4d34c00640 1 -- 172.21.15.161:0/2801768233 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4d200eb970 con 0x7f4d3815a6a0 2024-05-22T21:13:53.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.750+0000 7f4d3ea00640 1 -- 172.21.15.161:0/2801768233 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4d3815b5d0 con 0x7f4d3815a6a0 2024-05-22T21:13:53.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.750+0000 7f4d34c00640 1 -- 172.21.15.161:0/2801768233 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4d200eef90 con 0x7f4d3815a6a0 2024-05-22T21:13:53.752 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:13:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.750+0000 7f4d3ea00640 1 -- 172.21.15.161:0/2801768233 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4d140666c0 msgr2=0x7f4d14068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.750+0000 7f4d3ea00640 1 --2- 172.21.15.161:0/2801768233 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4d140666c0 0x7f4d14068b80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009b00 tx=0x7f4d28009190 comp rx=0 tx=0).stop 2024-05-22T21:13:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.750+0000 7f4d3ea00640 1 -- 172.21.15.161:0/2801768233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d3815a6a0 msgr2=0x7f4d38157850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.750+0000 7f4d3ea00640 1 --2- 172.21.15.161:0/2801768233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d3815a6a0 0x7f4d38157850 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009ae0 tx=0x7f4d1c005800 comp rx=0 tx=0).stop 2024-05-22T21:13:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.750+0000 7f4d3ea00640 1 -- 172.21.15.161:0/2801768233 shutdown_connections 2024-05-22T21:13:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.750+0000 7f4d3ea00640 1 --2- 172.21.15.161:0/2801768233 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4d140666c0 0x7f4d14068b80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.750+0000 7f4d3ea00640 1 --2- 172.21.15.161:0/2801768233 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d381420e0 0x7f4d38142560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.750+0000 7f4d3ea00640 1 --2- 172.21.15.161:0/2801768233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d3815a6a0 0x7f4d38157850 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.750+0000 7f4d3ea00640 1 --2- 172.21.15.161:0/2801768233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d1c003f00 0x7f4d3815bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.750+0000 7f4d3ea00640 1 -- 172.21.15.161:0/2801768233 >> 172.21.15.161:0/2801768233 conn(0x7f4d3806a2d0 msgr2=0x7f4d3806c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.750+0000 7f4d3ea00640 1 -- 172.21.15.161:0/2801768233 shutdown_connections 2024-05-22T21:13:53.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.750+0000 7f4d3ea00640 1 -- 172.21.15.161:0/2801768233 wait complete. 2024-05-22T21:13:53.774 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:13:53.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.906+0000 7fef02a00640 1 Processor -- start 2024-05-22T21:13:53.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.906+0000 7fef02a00640 1 -- start start 2024-05-22T21:13:53.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.910+0000 7fef02a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feefc143c40 0x7feefc15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.910+0000 7fef02a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc15efb0 con 0x7feefc1444a0 2024-05-22T21:13:53.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.910+0000 7fef02a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc15f290 con 0x7feefc06c9b0 2024-05-22T21:13:53.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.910+0000 7fef02a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc15f570 con 0x7feefc143c40 2024-05-22T21:13:53.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.910+0000 7feefbe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feefc143c40 0x7feefc15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.910+0000 7feefbe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feefc143c40 0x7feefc15b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47000/0 (socket says 172.21.15.161:47000) 2024-05-22T21:13:53.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.910+0000 7feefbe00640 1 -- 172.21.15.161:0/353830450 learned_addr learned my addr 172.21.15.161:0/353830450 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:53.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.910+0000 7feef3400640 1 --1- 172.21.15.161:0/353830450 >> v1:172.21.15.161:6789/0 conn(0x7feefc1444a0 0x7feefc06c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:52036/0 (socket says 172.21.15.161:52036) 2024-05-22T21:13:53.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.910+0000 7feefbe00640 1 -- 172.21.15.161:0/353830450 >> v1:172.21.15.179:6789/0 conn(0x7feefc06c9b0 legacy=0x7feefc0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.910+0000 7feefbe00640 1 -- 172.21.15.161:0/353830450 >> v1:172.21.15.161:6789/0 conn(0x7feefc1444a0 legacy=0x7feefc06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.910+0000 7feefbe00640 1 -- 172.21.15.161:0/353830450 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc15f850 con 0x7feefc143c40 2024-05-22T21:13:53.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7feefbe00640 1 --2- 172.21.15.161:0/353830450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feefc143c40 0x7feefc15b600 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feeec009870 tx=0x7feeec01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fa7ae0ef4a6d67 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:53.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7feefb400640 1 -- 172.21.15.161:0/353830450 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0aa620 con 0x7feefc143c40 2024-05-22T21:13:53.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7feefb400640 1 -- 172.21.15.161:0/353830450 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec0aa780 con 0x7feefc143c40 2024-05-22T21:13:53.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7feefb400640 1 -- 172.21.15.161:0/353830450 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0aab10 con 0x7feefc143c40 2024-05-22T21:13:53.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7fef02a00640 1 -- 172.21.15.161:0/353830450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feefc143c40 msgr2=0x7feefc15b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7fef02a00640 1 --2- 172.21.15.161:0/353830450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feefc143c40 0x7feefc15b600 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feeec009870 tx=0x7feeec01c560 comp rx=0 tx=0).stop 2024-05-22T21:13:53.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7fef02a00640 1 -- 172.21.15.161:0/353830450 shutdown_connections 2024-05-22T21:13:53.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7fef02a00640 1 --2- 172.21.15.161:0/353830450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feefc143c40 0x7feefc15b600 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7fef02a00640 1 -- 172.21.15.161:0/353830450 >> 172.21.15.161:0/353830450 conn(0x7feefc06a2d0 msgr2=0x7feefc06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:53.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7fef02a00640 1 -- 172.21.15.161:0/353830450 shutdown_connections 2024-05-22T21:13:53.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7fef02a00640 1 -- 172.21.15.161:0/353830450 wait complete. 2024-05-22T21:13:53.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7fef02a00640 1 Processor -- start 2024-05-22T21:13:53.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7fef02a00640 1 -- start start 2024-05-22T21:13:53.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7fef02a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feefc06c9b0 0x7feefc1395e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7fef02a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feefc143c40 0x7feefc141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7fef02a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feefc1444a0 0x7feefc142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7fef02a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc152d30 con 0x7feefc1444a0 2024-05-22T21:13:53.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7fef02a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc139b20 con 0x7feefc06c9b0 2024-05-22T21:13:53.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.914+0000 7fef02a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc152690 con 0x7feefc143c40 2024-05-22T21:13:53.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7fef00a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feefc1444a0 0x7feefc142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7fef00a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feefc1444a0 0x7feefc142060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47008/0 (socket says 172.21.15.161:47008) 2024-05-22T21:13:53.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7fef00a00640 1 -- 172.21.15.161:0/3475939322 learned_addr learned my addr 172.21.15.161:0/3475939322 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:53.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7feef3400640 1 --2- 172.21.15.161:0/3475939322 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feefc143c40 0x7feefc141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7feefbe00640 1 --2- 172.21.15.161:0/3475939322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feefc06c9b0 0x7feefc1395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7fef00a00640 1 -- 172.21.15.161:0/3475939322 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feefc143c40 msgr2=0x7feefc141b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7fef00a00640 1 --2- 172.21.15.161:0/3475939322 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feefc143c40 0x7feefc141b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7fef00a00640 1 -- 172.21.15.161:0/3475939322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feefc06c9b0 msgr2=0x7feefc1395e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:53.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7fef00a00640 1 --2- 172.21.15.161:0/3475939322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feefc06c9b0 0x7feefc1395e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:53.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7fef00a00640 1 -- 172.21.15.161:0/3475939322 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec0094d0 con 0x7feefc1444a0 2024-05-22T21:13:53.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7feefbe00640 1 --2- 172.21.15.161:0/3475939322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feefc06c9b0 0x7feefc1395e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:53.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7feef3400640 1 --2- 172.21.15.161:0/3475939322 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feefc143c40 0x7feefc141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:53.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7fef00a00640 1 --2- 172.21.15.161:0/3475939322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feefc1444a0 0x7feefc142060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feef400c770 tx=0x7feef400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:53.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7feef8c00640 1 -- 172.21.15.161:0/3475939322 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4003b80 con 0x7feefc1444a0 2024-05-22T21:13:53.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7feef8c00640 1 -- 172.21.15.161:0/3475939322 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef4003ce0 con 0x7feefc1444a0 2024-05-22T21:13:53.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7feef8c00640 1 -- 172.21.15.161:0/3475939322 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef400aa90 con 0x7feefc1444a0 2024-05-22T21:13:53.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7fef02a00640 1 -- 172.21.15.161:0/3475939322 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc1426a0 con 0x7feefc1444a0 2024-05-22T21:13:53.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7fef02a00640 1 -- 172.21.15.161:0/3475939322 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc152950 con 0x7feefc1444a0 2024-05-22T21:13:53.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7fef02a00640 1 -- 172.21.15.161:0/3475939322 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc152e70 con 0x7feefc1444a0 2024-05-22T21:13:53.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7feef8c00640 1 -- 172.21.15.161:0/3475939322 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feef400abf0 con 0x7feefc1444a0 2024-05-22T21:13:53.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7feef8c00640 1 --2- 172.21.15.161:0/3475939322 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feed8066690 0x7feed8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:53.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.918+0000 7feef8c00640 1 -- 172.21.15.161:0/3475939322 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feef4083ff0 con 0x7feefc1444a0 2024-05-22T21:13:53.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.922+0000 7feefbe00640 1 --2- 172.21.15.161:0/3475939322 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feed8066690 0x7feed8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:53.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.922+0000 7feefbe00640 1 --2- 172.21.15.161:0/3475939322 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feed8066690 0x7feed8068b50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feeec0ae000 tx=0x7feeec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:53.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:53.922+0000 7feef8c00640 1 -- 172.21.15.161:0/3475939322 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7feef40b5cf0 con 0x7feefc1444a0 2024-05-22T21:13:54.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.077+0000 7fef02a00640 1 -- 172.21.15.161:0/3475939322 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feefc067220 con 0x7feefc1444a0 2024-05-22T21:13:54.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.077+0000 7feef8c00640 1 -- 172.21.15.161:0/3475939322 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7feefc152e70 con 0x7feefc1444a0 2024-05-22T21:13:54.084 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:13:54.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.081+0000 7fef02a00640 1 -- 172.21.15.161:0/3475939322 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feed8066690 msgr2=0x7feed8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.081+0000 7fef02a00640 1 --2- 172.21.15.161:0/3475939322 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feed8066690 0x7feed8068b50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feeec0ae000 tx=0x7feeec008040 comp rx=0 tx=0).stop 2024-05-22T21:13:54.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.081+0000 7fef02a00640 1 -- 172.21.15.161:0/3475939322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feefc1444a0 msgr2=0x7feefc142060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.081+0000 7fef02a00640 1 --2- 172.21.15.161:0/3475939322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feefc1444a0 0x7feefc142060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feef400c770 tx=0x7feef400cc30 comp rx=0 tx=0).stop 2024-05-22T21:13:54.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.081+0000 7fef02a00640 1 -- 172.21.15.161:0/3475939322 shutdown_connections 2024-05-22T21:13:54.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.081+0000 7fef02a00640 1 --2- 172.21.15.161:0/3475939322 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feed8066690 0x7feed8068b50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.081+0000 7fef02a00640 1 --2- 172.21.15.161:0/3475939322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feefc1444a0 0x7feefc142060 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.081+0000 7fef02a00640 1 --2- 172.21.15.161:0/3475939322 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feefc143c40 0x7feefc141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.081+0000 7fef02a00640 1 --2- 172.21.15.161:0/3475939322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feefc06c9b0 0x7feefc1395e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.081+0000 7fef02a00640 1 -- 172.21.15.161:0/3475939322 >> 172.21.15.161:0/3475939322 conn(0x7feefc06a2d0 msgr2=0x7feefc15d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:54.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.081+0000 7fef02a00640 1 -- 172.21.15.161:0/3475939322 shutdown_connections 2024-05-22T21:13:54.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.081+0000 7fef02a00640 1 -- 172.21.15.161:0/3475939322 wait complete. 2024-05-22T21:13:54.104 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:13:54.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.237+0000 7f783de00640 1 Processor -- start 2024-05-22T21:13:54.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.237+0000 7f783de00640 1 -- start start 2024-05-22T21:13:54.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.237+0000 7f783de00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f783806c9b0 0x7f78380b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.237+0000 7f783de00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7838144bb0 con 0x7f7838143c40 2024-05-22T21:13:54.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.237+0000 7f783de00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783815efb0 con 0x7f783815b600 2024-05-22T21:13:54.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.237+0000 7f783de00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f783815ba60 con 0x7f783806c9b0 2024-05-22T21:13:54.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.237+0000 7f7836a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f7838143c40 0x7f78381444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:52040/0 (socket says 172.21.15.161:52040) 2024-05-22T21:13:54.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.237+0000 7f7836a00640 1 -- 172.21.15.161:0/1330040604 learned_addr learned my addr 172.21.15.161:0/1330040604 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:54.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.237+0000 7f7837400640 1 --2- 172.21.15.161:0/1330040604 >> v2:172.21.15.179:3300/0 conn(0x7f783806c9b0 0x7f78380b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.237+0000 7f7836000640 1 -- 172.21.15.161:0/1330040604 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763220899 0 0) 0x7f7838144bb0 con 0x7f7838143c40 2024-05-22T21:13:54.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.237+0000 7f7836000640 1 -- 172.21.15.161:0/1330040604 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f781c003530 con 0x7f7838143c40 2024-05-22T21:13:54.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.241+0000 7f7837400640 1 -- 172.21.15.161:0/1330040604 >> v1:172.21.15.179:6789/0 conn(0x7f783815b600 legacy=0x7f783806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.241+0000 7f7837400640 1 -- 172.21.15.161:0/1330040604 >> v1:172.21.15.161:6789/0 conn(0x7f7838143c40 legacy=0x7f78381444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.241+0000 7f7837400640 1 -- 172.21.15.161:0/1330040604 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783815f290 con 0x7f783806c9b0 2024-05-22T21:13:54.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.241+0000 7f7837400640 1 --2- 172.21.15.161:0/1330040604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f783806c9b0 0x7f78380b2160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7830009d90 tx=0x7f783001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2001c56428bdc67 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:54.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.241+0000 7f7836000640 1 -- 172.21.15.161:0/1330040604 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78300a5a30 con 0x7f783806c9b0 2024-05-22T21:13:54.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.241+0000 7f7836000640 1 -- 172.21.15.161:0/1330040604 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78300a5b90 con 0x7f783806c9b0 2024-05-22T21:13:54.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.241+0000 7f7836000640 1 -- 172.21.15.161:0/1330040604 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78300ad440 con 0x7f783806c9b0 2024-05-22T21:13:54.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.241+0000 7f783de00640 1 -- 172.21.15.161:0/1330040604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f783806c9b0 msgr2=0x7f78380b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.241+0000 7f783de00640 1 --2- 172.21.15.161:0/1330040604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f783806c9b0 0x7f78380b2160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7830009d90 tx=0x7f783001c970 comp rx=0 tx=0).stop 2024-05-22T21:13:54.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.241+0000 7f783de00640 1 -- 172.21.15.161:0/1330040604 shutdown_connections 2024-05-22T21:13:54.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.241+0000 7f783de00640 1 --2- 172.21.15.161:0/1330040604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f783806c9b0 0x7f78380b2160 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.241+0000 7f783de00640 1 -- 172.21.15.161:0/1330040604 >> 172.21.15.161:0/1330040604 conn(0x7f783806a2d0 msgr2=0x7f783806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:54.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.241+0000 7f783de00640 1 -- 172.21.15.161:0/1330040604 shutdown_connections 2024-05-22T21:13:54.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f783de00640 1 -- 172.21.15.161:0/1330040604 wait complete. 2024-05-22T21:13:54.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f783de00640 1 Processor -- start 2024-05-22T21:13:54.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f783de00640 1 -- start start 2024-05-22T21:13:54.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f783de00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f783806c9b0 0x7f78381364c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f783de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7838143c40 0x7f7838142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f783de00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f783815b600 0x7f783815bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f783de00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7838153660 con 0x7f7838143c40 2024-05-22T21:13:54.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f783de00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7838153940 con 0x7f783815b600 2024-05-22T21:13:54.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f783de00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7838153c20 con 0x7f783806c9b0 2024-05-22T21:13:54.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f782e000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7838143c40 0x7f7838142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f782e000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7838143c40 0x7f7838142d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47024/0 (socket says 172.21.15.161:47024) 2024-05-22T21:13:54.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f782e000640 1 -- 172.21.15.161:0/3927016040 learned_addr learned my addr 172.21.15.161:0/3927016040 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:54.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f7836a00640 1 --2- 172.21.15.161:0/3927016040 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f783806c9b0 0x7f78381364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f7837400640 1 --2- 172.21.15.161:0/3927016040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f783815b600 0x7f783815bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f782e000640 1 -- 172.21.15.161:0/3927016040 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f783806c9b0 msgr2=0x7f78381364c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f782e000640 1 --2- 172.21.15.161:0/3927016040 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f783806c9b0 0x7f78381364c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f782e000640 1 -- 172.21.15.161:0/3927016040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f783815b600 msgr2=0x7f783815bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f782e000640 1 --2- 172.21.15.161:0/3927016040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f783815b600 0x7f783815bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f782e000640 1 -- 172.21.15.161:0/3927016040 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7830009a70 con 0x7f7838143c40 2024-05-22T21:13:54.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f7837400640 1 --2- 172.21.15.161:0/3927016040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f783815b600 0x7f783815bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:54.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f782e000640 1 --2- 172.21.15.161:0/3927016040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7838143c40 0x7f7838142d50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f782000c370 tx=0x7f782000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:54.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.245+0000 7f782fe00640 1 -- 172.21.15.161:0/3927016040 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78200052d0 con 0x7f7838143c40 2024-05-22T21:13:54.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.249+0000 7f782fe00640 1 -- 172.21.15.161:0/3927016040 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7820005430 con 0x7f7838143c40 2024-05-22T21:13:54.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.249+0000 7f783de00640 1 -- 172.21.15.161:0/3927016040 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7838153f00 con 0x7f7838143c40 2024-05-22T21:13:54.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.249+0000 7f782fe00640 1 -- 172.21.15.161:0/3927016040 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820007260 con 0x7f7838143c40 2024-05-22T21:13:54.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.249+0000 7f783de00640 1 -- 172.21.15.161:0/3927016040 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78381544c0 con 0x7f7838143c40 2024-05-22T21:13:54.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.249+0000 7f783de00640 1 -- 172.21.15.161:0/3927016040 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7838067220 con 0x7f7838143c40 2024-05-22T21:13:54.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.249+0000 7f782fe00640 1 -- 172.21.15.161:0/3927016040 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f78200073c0 con 0x7f7838143c40 2024-05-22T21:13:54.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.249+0000 7f782fe00640 1 --2- 172.21.15.161:0/3927016040 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7814066690 0x7f7814068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.249+0000 7f782fe00640 1 -- 172.21.15.161:0/3927016040 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78200835c0 con 0x7f7838143c40 2024-05-22T21:13:54.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.249+0000 7f7836a00640 1 --2- 172.21.15.161:0/3927016040 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7814066690 0x7f7814068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.249+0000 7f7836a00640 1 --2- 172.21.15.161:0/3927016040 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7814066690 0x7f7814068b50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f78280011f0 tx=0x7f7828009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:54.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.253+0000 7f782fe00640 1 -- 172.21.15.161:0/3927016040 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7820050890 con 0x7f7838143c40 2024-05-22T21:13:54.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.405+0000 7f783de00640 1 -- 172.21.15.161:0/3927016040 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7838153440 con 0x7f7838143c40 2024-05-22T21:13:54.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.405+0000 7f782fe00640 1 -- 172.21.15.161:0/3927016040 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7820053eb0 con 0x7f7838143c40 2024-05-22T21:13:54.411 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:13:54.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.409+0000 7f783de00640 1 -- 172.21.15.161:0/3927016040 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7814066690 msgr2=0x7f7814068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.409+0000 7f783de00640 1 --2- 172.21.15.161:0/3927016040 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7814066690 0x7f7814068b50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f78280011f0 tx=0x7f7828009040 comp rx=0 tx=0).stop 2024-05-22T21:13:54.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.409+0000 7f783de00640 1 -- 172.21.15.161:0/3927016040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7838143c40 msgr2=0x7f7838142d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.409+0000 7f783de00640 1 --2- 172.21.15.161:0/3927016040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7838143c40 0x7f7838142d50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f782000c370 tx=0x7f782000c830 comp rx=0 tx=0).stop 2024-05-22T21:13:54.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.409+0000 7f783de00640 1 -- 172.21.15.161:0/3927016040 shutdown_connections 2024-05-22T21:13:54.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.409+0000 7f783de00640 1 --2- 172.21.15.161:0/3927016040 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7814066690 0x7f7814068b50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.409+0000 7f783de00640 1 --2- 172.21.15.161:0/3927016040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f783815b600 0x7f783815bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.409+0000 7f783de00640 1 --2- 172.21.15.161:0/3927016040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7838143c40 0x7f7838142d50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.409+0000 7f783de00640 1 --2- 172.21.15.161:0/3927016040 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f783806c9b0 0x7f78381364c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.409+0000 7f783de00640 1 -- 172.21.15.161:0/3927016040 >> 172.21.15.161:0/3927016040 conn(0x7f783806a2d0 msgr2=0x7f7838159340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:54.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.409+0000 7f783de00640 1 -- 172.21.15.161:0/3927016040 shutdown_connections 2024-05-22T21:13:54.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.409+0000 7f783de00640 1 -- 172.21.15.161:0/3927016040 wait complete. 2024-05-22T21:13:54.432 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:54.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.565+0000 7fabbee00640 1 Processor -- start 2024-05-22T21:13:54.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.565+0000 7fabbee00640 1 -- start start 2024-05-22T21:13:54.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.565+0000 7fabbee00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fabb815e7d0 0x7fabb80b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.565+0000 7fabbee00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fabb815fa60 0x7fabb80cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.565+0000 7fabbee00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb80b3560 con 0x7fabb815eed0 2024-05-22T21:13:54.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.565+0000 7fabbee00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb80cf530 con 0x7fabb815e7d0 2024-05-22T21:13:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.565+0000 7fabbee00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb80b37b0 con 0x7fabb815fa60 2024-05-22T21:13:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.565+0000 7fabb7e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fabb815e7d0 0x7fabb80b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.565+0000 7fabb7e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fabb815e7d0 0x7fabb80b1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47030/0 (socket says 172.21.15.161:47030) 2024-05-22T21:13:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.565+0000 7fabb7e00640 1 -- 172.21.15.161:0/1107613679 learned_addr learned my addr 172.21.15.161:0/1107613679 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.565+0000 7fabbce00640 1 --2- 172.21.15.161:0/1107613679 >> v2:172.21.15.179:3300/0 conn(0x7fabb815fa60 0x7fabb80cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.565+0000 7fabb7e00640 1 -- 172.21.15.161:0/1107613679 >> v2:172.21.15.179:3300/0 conn(0x7fabb815fa60 msgr2=0x7fabb80cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.565+0000 7fabb7e00640 1 --2- 172.21.15.161:0/1107613679 >> v2:172.21.15.179:3300/0 conn(0x7fabb815fa60 0x7fabb80cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.565+0000 7fabb7e00640 1 -- 172.21.15.161:0/1107613679 >> v1:172.21.15.179:6789/0 conn(0x7fabb815eed0 legacy=0x7fabb815f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.565+0000 7fabb7e00640 1 -- 172.21.15.161:0/1107613679 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb80cf780 con 0x7fabb815e7d0 2024-05-22T21:13:54.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.569+0000 7fabb7e00640 1 --2- 172.21.15.161:0/1107613679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb815e7d0 0x7fabb80b1df0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fabb80b2720 tx=0x7faba801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20da5fc33e211c5d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:54.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.569+0000 7fabb7400640 1 -- 172.21.15.161:0/1107613679 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80aa430 con 0x7fabb815e7d0 2024-05-22T21:13:54.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.569+0000 7fabb7400640 1 -- 172.21.15.161:0/1107613679 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba80aa590 con 0x7fabb815e7d0 2024-05-22T21:13:54.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.569+0000 7fabb7400640 1 -- 172.21.15.161:0/1107613679 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80aa920 con 0x7fabb815e7d0 2024-05-22T21:13:54.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.569+0000 7fabbee00640 1 -- 172.21.15.161:0/1107613679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb815e7d0 msgr2=0x7fabb80b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.569+0000 7fabbee00640 1 --2- 172.21.15.161:0/1107613679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb815e7d0 0x7fabb80b1df0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fabb80b2720 tx=0x7faba801c4b0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.569+0000 7fabbee00640 1 -- 172.21.15.161:0/1107613679 shutdown_connections 2024-05-22T21:13:54.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.569+0000 7fabbee00640 1 --2- 172.21.15.161:0/1107613679 >> v2:172.21.15.179:3300/0 conn(0x7fabb815fa60 0x7fabb80cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.569+0000 7fabbee00640 1 --2- 172.21.15.161:0/1107613679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb815e7d0 0x7fabb80b1df0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.569+0000 7fabbee00640 1 -- 172.21.15.161:0/1107613679 >> 172.21.15.161:0/1107613679 conn(0x7fabb80b2a40 msgr2=0x7fabb80b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:54.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.569+0000 7fabbee00640 1 -- 172.21.15.161:0/1107613679 shutdown_connections 2024-05-22T21:13:54.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.569+0000 7fabbee00640 1 -- 172.21.15.161:0/1107613679 wait complete. 2024-05-22T21:13:54.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.569+0000 7fabbee00640 1 Processor -- start 2024-05-22T21:13:54.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabbee00640 1 -- start start 2024-05-22T21:13:54.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabbee00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb815e7d0 0x7fabb806f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabbee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabb815eed0 0x7fabb814c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabbee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabb815fa60 0x7fabb814d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabbee00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb806b350 con 0x7fabb815e7d0 2024-05-22T21:13:54.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabbee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb806f3b0 con 0x7fabb815eed0 2024-05-22T21:13:54.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabbee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb806f080 con 0x7fabb815fa60 2024-05-22T21:13:54.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabb7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb815e7d0 0x7fabb806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabb7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb815e7d0 0x7fabb806f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47036/0 (socket says 172.21.15.161:47036) 2024-05-22T21:13:54.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabb7e00640 1 -- 172.21.15.161:0/4220990617 learned_addr learned my addr 172.21.15.161:0/4220990617 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:54.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabafe00640 1 --2- 172.21.15.161:0/4220990617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabb815eed0 0x7fabb814c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabbce00640 1 --2- 172.21.15.161:0/4220990617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabb815fa60 0x7fabb814d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabb7e00640 1 -- 172.21.15.161:0/4220990617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabb815fa60 msgr2=0x7fabb814d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabb7e00640 1 --2- 172.21.15.161:0/4220990617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabb815fa60 0x7fabb814d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabb7e00640 1 -- 172.21.15.161:0/4220990617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabb815eed0 msgr2=0x7fabb814c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabb7e00640 1 --2- 172.21.15.161:0/4220990617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabb815eed0 0x7fabb814c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabb7e00640 1 -- 172.21.15.161:0/4220990617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba80094d0 con 0x7fabb815e7d0 2024-05-22T21:13:54.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabafe00640 1 --2- 172.21.15.161:0/4220990617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabb815eed0 0x7fabb814c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:54.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabb7e00640 1 --2- 172.21.15.161:0/4220990617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb815e7d0 0x7fabb806f840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faba801a0e0 tx=0x7faba801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:54.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabb4c00640 1 -- 172.21.15.161:0/4220990617 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80b3800 con 0x7fabb815e7d0 2024-05-22T21:13:54.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabb4c00640 1 -- 172.21.15.161:0/4220990617 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba80b3960 con 0x7fabb815e7d0 2024-05-22T21:13:54.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabb4c00640 1 -- 172.21.15.161:0/4220990617 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80b3cf0 con 0x7fabb815e7d0 2024-05-22T21:13:54.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabbee00640 1 -- 172.21.15.161:0/4220990617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb806fd80 con 0x7fabb815e7d0 2024-05-22T21:13:54.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabbee00640 1 -- 172.21.15.161:0/4220990617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb815da30 con 0x7fabb815e7d0 2024-05-22T21:13:54.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.573+0000 7fabbee00640 1 -- 172.21.15.161:0/4220990617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb8067220 con 0x7fabb815e7d0 2024-05-22T21:13:54.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.577+0000 7fabb4c00640 1 -- 172.21.15.161:0/4220990617 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faba80b2380 con 0x7fabb815e7d0 2024-05-22T21:13:54.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.577+0000 7fabb4c00640 1 --2- 172.21.15.161:0/4220990617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fab94066690 0x7fab94068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.577+0000 7fabb4c00640 1 -- 172.21.15.161:0/4220990617 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faba811e700 con 0x7fabb815e7d0 2024-05-22T21:13:54.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.577+0000 7fabafe00640 1 --2- 172.21.15.161:0/4220990617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fab94066690 0x7fab94068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.577+0000 7fabafe00640 1 --2- 172.21.15.161:0/4220990617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fab94066690 0x7fab94068b50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fabb80c8040 tx=0x7faba0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:54.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.581+0000 7fabb4c00640 1 -- 172.21.15.161:0/4220990617 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7faba80eba80 con 0x7fabb815e7d0 2024-05-22T21:13:54.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.733+0000 7fabbee00640 1 -- 172.21.15.161:0/4220990617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabb80ce7a0 con 0x7fabb815e7d0 2024-05-22T21:13:54.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.737+0000 7fabb4c00640 1 -- 172.21.15.161:0/4220990617 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7faba80ef0a0 con 0x7fabb815e7d0 2024-05-22T21:13:54.740 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:54.741 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:54.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.737+0000 7fabbee00640 1 -- 172.21.15.161:0/4220990617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fab94066690 msgr2=0x7fab94068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.737+0000 7fabbee00640 1 --2- 172.21.15.161:0/4220990617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fab94066690 0x7fab94068b50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fabb80c8040 tx=0x7faba0009040 comp rx=0 tx=0).stop 2024-05-22T21:13:54.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.737+0000 7fabbee00640 1 -- 172.21.15.161:0/4220990617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb815e7d0 msgr2=0x7fabb806f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.737+0000 7fabbee00640 1 --2- 172.21.15.161:0/4220990617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb815e7d0 0x7fabb806f840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faba801a0e0 tx=0x7faba801acf0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.737+0000 7fabbee00640 1 -- 172.21.15.161:0/4220990617 shutdown_connections 2024-05-22T21:13:54.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.737+0000 7fabbee00640 1 --2- 172.21.15.161:0/4220990617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fab94066690 0x7fab94068b50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.737+0000 7fabbee00640 1 --2- 172.21.15.161:0/4220990617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabb815fa60 0x7fabb814d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.737+0000 7fabbee00640 1 --2- 172.21.15.161:0/4220990617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabb815eed0 0x7fabb814c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.737+0000 7fabbee00640 1 --2- 172.21.15.161:0/4220990617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb815e7d0 0x7fabb806f840 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.737+0000 7fabbee00640 1 -- 172.21.15.161:0/4220990617 >> 172.21.15.161:0/4220990617 conn(0x7fabb80b2a40 msgr2=0x7fabb80ceb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:54.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.737+0000 7fabbee00640 1 -- 172.21.15.161:0/4220990617 shutdown_connections 2024-05-22T21:13:54.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.737+0000 7fabbee00640 1 -- 172.21.15.161:0/4220990617 wait complete. 2024-05-22T21:13:54.761 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:13:21.237503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:13:22.425891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:13:54.761 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:13:54.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.893+0000 7fdc95400640 1 Processor -- start 2024-05-22T21:13:54.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.893+0000 7fdc95400640 1 -- start start 2024-05-22T21:13:54.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.897+0000 7fdc95400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdc9013cfc0 0x7fdc901402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.897+0000 7fdc95400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdc9015af60 0x7fdc9013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.897+0000 7fdc95400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc90140090 con 0x7fdc9013fc80 2024-05-22T21:13:54.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.897+0000 7fdc95400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9013d3a0 con 0x7fdc9013cfc0 2024-05-22T21:13:54.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.897+0000 7fdc95400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc900cd920 con 0x7fdc9015af60 2024-05-22T21:13:54.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.897+0000 7fdc8e800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdc9013cfc0 0x7fdc901402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.897+0000 7fdc8e800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdc9013cfc0 0x7fdc901402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47046/0 (socket says 172.21.15.161:47046) 2024-05-22T21:13:54.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.897+0000 7fdc8f200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fdc9013fc80 0x7fdc9015dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:52046/0 (socket says 172.21.15.161:52046) 2024-05-22T21:13:54.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.897+0000 7fdc8e800640 1 -- 172.21.15.161:0/3577728271 learned_addr learned my addr 172.21.15.161:0/3577728271 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:54.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.897+0000 7fdc8de00640 1 --2- 172.21.15.161:0/3577728271 >> v2:172.21.15.179:3300/0 conn(0x7fdc9015af60 0x7fdc9013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.897+0000 7fdc8e800640 1 -- 172.21.15.161:0/3577728271 >> v2:172.21.15.179:3300/0 conn(0x7fdc9015af60 msgr2=0x7fdc9013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:54.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.897+0000 7fdc8e800640 1 --2- 172.21.15.161:0/3577728271 >> v2:172.21.15.179:3300/0 conn(0x7fdc9015af60 0x7fdc9013f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.897+0000 7fdc8e800640 1 -- 172.21.15.161:0/3577728271 >> v1:172.21.15.161:6789/0 conn(0x7fdc9013fc80 legacy=0x7fdc9015dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.897+0000 7fdc8e800640 1 -- 172.21.15.161:0/3577728271 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc900cdc00 con 0x7fdc9013cfc0 2024-05-22T21:13:54.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.897+0000 7fdc8d400640 1 -- 172.21.15.161:0/3577728271 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3819111560 0 0) 0x7fdc90140090 con 0x7fdc9013fc80 2024-05-22T21:13:54.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc8e800640 1 --2- 172.21.15.161:0/3577728271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc9013cfc0 0x7fdc901402e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdc9006e1d0 tx=0x7fdc8001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc379544d9b244f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:54.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc8d400640 1 -- 172.21.15.161:0/3577728271 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc800a5630 con 0x7fdc9013cfc0 2024-05-22T21:13:54.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc8d400640 1 -- 172.21.15.161:0/3577728271 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc800a5790 con 0x7fdc9013cfc0 2024-05-22T21:13:54.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc8d400640 1 -- 172.21.15.161:0/3577728271 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc800a5b20 con 0x7fdc9013cfc0 2024-05-22T21:13:54.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc95400640 1 -- 172.21.15.161:0/3577728271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc9013cfc0 msgr2=0x7fdc901402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc95400640 1 --2- 172.21.15.161:0/3577728271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc9013cfc0 0x7fdc901402e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdc9006e1d0 tx=0x7fdc8001c960 comp rx=0 tx=0).stop 2024-05-22T21:13:54.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc95400640 1 -- 172.21.15.161:0/3577728271 shutdown_connections 2024-05-22T21:13:54.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc95400640 1 --2- 172.21.15.161:0/3577728271 >> v2:172.21.15.179:3300/0 conn(0x7fdc9015af60 0x7fdc9013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc95400640 1 --2- 172.21.15.161:0/3577728271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc9013cfc0 0x7fdc901402e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc95400640 1 -- 172.21.15.161:0/3577728271 >> 172.21.15.161:0/3577728271 conn(0x7fdc9006a2d0 msgr2=0x7fdc9006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:54.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc95400640 1 -- 172.21.15.161:0/3577728271 shutdown_connections 2024-05-22T21:13:54.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc95400640 1 -- 172.21.15.161:0/3577728271 wait complete. 2024-05-22T21:13:54.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc95400640 1 Processor -- start 2024-05-22T21:13:54.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc95400640 1 -- start start 2024-05-22T21:13:54.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc95400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc9013cfc0 0x7fdc90139520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc95400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc9013fc80 0x7fdc901576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc95400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc9015af60 0x7fdc901414d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc95400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc90139a60 con 0x7fdc9013fc80 2024-05-22T21:13:54.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc95400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc90157be0 con 0x7fdc9013cfc0 2024-05-22T21:13:54.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc95400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc90142770 con 0x7fdc9015af60 2024-05-22T21:13:54.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc8de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc9013fc80 0x7fdc901576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc8de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc9013fc80 0x7fdc901576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47058/0 (socket says 172.21.15.161:47058) 2024-05-22T21:13:54.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.901+0000 7fdc8de00640 1 -- 172.21.15.161:0/3486602081 learned_addr learned my addr 172.21.15.161:0/3486602081 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:54.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc8e800640 1 --2- 172.21.15.161:0/3486602081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc9013cfc0 0x7fdc90139520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc8f200640 1 --2- 172.21.15.161:0/3486602081 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc9015af60 0x7fdc901414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc8de00640 1 -- 172.21.15.161:0/3486602081 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc9015af60 msgr2=0x7fdc901414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc8de00640 1 --2- 172.21.15.161:0/3486602081 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc9015af60 0x7fdc901414d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc8de00640 1 -- 172.21.15.161:0/3486602081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc9013cfc0 msgr2=0x7fdc90139520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:54.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc8de00640 1 --2- 172.21.15.161:0/3486602081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc9013cfc0 0x7fdc90139520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:54.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc8de00640 1 -- 172.21.15.161:0/3486602081 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc80009b30 con 0x7fdc9013fc80 2024-05-22T21:13:54.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc8de00640 1 --2- 172.21.15.161:0/3486602081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc9013fc80 0x7fdc901576a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00e930 tx=0x7fdc7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:54.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc86a00640 1 -- 172.21.15.161:0/3486602081 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c009690 con 0x7fdc9013fc80 2024-05-22T21:13:54.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc86a00640 1 -- 172.21.15.161:0/3486602081 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7c00ca00 con 0x7fdc9013fc80 2024-05-22T21:13:54.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc86a00640 1 -- 172.21.15.161:0/3486602081 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c00cd70 con 0x7fdc9013fc80 2024-05-22T21:13:54.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc95400640 1 -- 172.21.15.161:0/3486602081 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9015b4f0 con 0x7fdc9013fc80 2024-05-22T21:13:54.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc95400640 1 -- 172.21.15.161:0/3486602081 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc901421b0 con 0x7fdc9013fc80 2024-05-22T21:13:54.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc95400640 1 -- 172.21.15.161:0/3486602081 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc90067220 con 0x7fdc9013fc80 2024-05-22T21:13:54.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc86a00640 1 -- 172.21.15.161:0/3486602081 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdc7c013470 con 0x7fdc9013fc80 2024-05-22T21:13:54.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc86a00640 1 --2- 172.21.15.161:0/3486602081 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc70066690 0x7fdc70068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:54.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.905+0000 7fdc86a00640 1 -- 172.21.15.161:0/3486602081 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc7c084400 con 0x7fdc9013fc80 2024-05-22T21:13:54.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.909+0000 7fdc8e800640 1 --2- 172.21.15.161:0/3486602081 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc70066690 0x7fdc70068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:54.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.909+0000 7fdc8e800640 1 --2- 172.21.15.161:0/3486602081 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc70066690 0x7fdc70068b50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdc800ae000 tx=0x7fdc800a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:54.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:54.909+0000 7fdc86a00640 1 -- 172.21.15.161:0/3486602081 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdc7c051820 con 0x7fdc9013fc80 2024-05-22T21:13:55.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.069+0000 7fdc95400640 1 -- 172.21.15.161:0/3486602081 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdc90071d00 con 0x7fdc9013fc80 2024-05-22T21:13:55.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.069+0000 7fdc86a00640 1 -- 172.21.15.161:0/3486602081 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdc7c054e40 con 0x7fdc9013fc80 2024-05-22T21:13:55.073 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:13:55.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.069+0000 7fdc95400640 1 -- 172.21.15.161:0/3486602081 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc70066690 msgr2=0x7fdc70068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.069+0000 7fdc95400640 1 --2- 172.21.15.161:0/3486602081 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc70066690 0x7fdc70068b50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdc800ae000 tx=0x7fdc800a7070 comp rx=0 tx=0).stop 2024-05-22T21:13:55.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.069+0000 7fdc95400640 1 -- 172.21.15.161:0/3486602081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc9013fc80 msgr2=0x7fdc901576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.069+0000 7fdc95400640 1 --2- 172.21.15.161:0/3486602081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc9013fc80 0x7fdc901576a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00e930 tx=0x7fdc7c00edf0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.069+0000 7fdc95400640 1 -- 172.21.15.161:0/3486602081 shutdown_connections 2024-05-22T21:13:55.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.069+0000 7fdc95400640 1 --2- 172.21.15.161:0/3486602081 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc70066690 0x7fdc70068b50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.069+0000 7fdc95400640 1 --2- 172.21.15.161:0/3486602081 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc9015af60 0x7fdc901414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.069+0000 7fdc95400640 1 --2- 172.21.15.161:0/3486602081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc9013fc80 0x7fdc901576a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.069+0000 7fdc95400640 1 --2- 172.21.15.161:0/3486602081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc9013cfc0 0x7fdc90139520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.069+0000 7fdc95400640 1 -- 172.21.15.161:0/3486602081 >> 172.21.15.161:0/3486602081 conn(0x7fdc9006a2d0 msgr2=0x7fdc9006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:55.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.069+0000 7fdc95400640 1 -- 172.21.15.161:0/3486602081 shutdown_connections 2024-05-22T21:13:55.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.069+0000 7fdc95400640 1 -- 172.21.15.161:0/3486602081 wait complete. 2024-05-22T21:13:55.094 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:13:55.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.225+0000 7f4778c00640 1 Processor -- start 2024-05-22T21:13:55.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.225+0000 7f4778c00640 1 -- start start 2024-05-22T21:13:55.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4778c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f477415b600 0x7f477406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4778c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477406cc40 con 0x7f477406c7e0 2024-05-22T21:13:55.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4778c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4774144bb0 con 0x7f4774143c40 2024-05-22T21:13:55.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4778c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477415ba60 con 0x7f477415b600 2024-05-22T21:13:55.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4772800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f477406c7e0 0x7f47740b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:52058/0 (socket says 172.21.15.161:52058) 2024-05-22T21:13:55.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4772800640 1 -- 172.21.15.161:0/548935683 learned_addr learned my addr 172.21.15.161:0/548935683 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:55.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4771400640 1 --2- 172.21.15.161:0/548935683 >> v2:172.21.15.179:3300/0 conn(0x7f477415b600 0x7f477406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 -- 172.21.15.161:0/548935683 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104671122 0 0) 0x7f477406cc40 con 0x7f477406c7e0 2024-05-22T21:13:55.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 -- 172.21.15.161:0/548935683 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4758003530 con 0x7f477406c7e0 2024-05-22T21:13:55.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 -- 172.21.15.161:0/548935683 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578943380 0 0) 0x7f4774144bb0 con 0x7f4774143c40 2024-05-22T21:13:55.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 -- 172.21.15.161:0/548935683 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f477406cc40 con 0x7f4774143c40 2024-05-22T21:13:55.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 -- 172.21.15.161:0/548935683 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 458736088 0 0) 0x7f4758003530 con 0x7f477406c7e0 2024-05-22T21:13:55.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 -- 172.21.15.161:0/548935683 >> v2:172.21.15.179:3300/0 conn(0x7f477415b600 msgr2=0x7f477406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:55.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 --2- 172.21.15.161:0/548935683 >> v2:172.21.15.179:3300/0 conn(0x7f477415b600 0x7f477406c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 -- 172.21.15.161:0/548935683 >> v1:172.21.15.179:6789/0 conn(0x7f4774143c40 legacy=0x7f47741444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 -- 172.21.15.161:0/548935683 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477415efb0 con 0x7f477406c7e0 2024-05-22T21:13:55.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 -- 172.21.15.161:0/548935683 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f476c001680 con 0x7f477406c7e0 2024-05-22T21:13:55.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4771400640 1 --2- 172.21.15.161:0/548935683 >> v2:172.21.15.179:3300/0 conn(0x7f477415b600 0x7f477406c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:55.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 -- 172.21.15.161:0/548935683 >> v1:172.21.15.161:6789/0 conn(0x7f477406c7e0 legacy=0x7f47740b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 --2- 172.21.15.161:0/548935683 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4758003b10 0x7f4758003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 --2- 172.21.15.161:0/548935683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47580047d0 0x7f4758004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 --2- 172.21.15.161:0/548935683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4758005190 0x7f475800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 -- 172.21.15.161:0/548935683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477415ba60 con 0x7f47580047d0 2024-05-22T21:13:55.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 -- 172.21.15.161:0/548935683 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475800c010 con 0x7f4758005190 2024-05-22T21:13:55.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4770a00640 1 -- 172.21.15.161:0/548935683 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475800c1d0 con 0x7f4758003b10 2024-05-22T21:13:55.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4771400640 1 --2- 172.21.15.161:0/548935683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47580047d0 0x7f4758004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4771e00640 1 --2- 172.21.15.161:0/548935683 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4758003b10 0x7f4758003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4772800640 1 --2- 172.21.15.161:0/548935683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4758005190 0x7f475800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4771400640 1 -- 172.21.15.161:0/548935683 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4758003b10 msgr2=0x7f4758003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:55.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4771400640 1 --2- 172.21.15.161:0/548935683 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4758003b10 0x7f4758003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4771400640 1 -- 172.21.15.161:0/548935683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4758005190 msgr2=0x7f475800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:55.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4771400640 1 --2- 172.21.15.161:0/548935683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4758005190 0x7f475800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4771400640 1 -- 172.21.15.161:0/548935683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4758003530 con 0x7f47580047d0 2024-05-22T21:13:55.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.229+0000 7f4772800640 1 -- 172.21.15.161:0/548935683 reap_dead start 2024-05-22T21:13:55.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.233+0000 7f4771400640 1 --2- 172.21.15.161:0/548935683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47580047d0 0x7f4758004c50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4760009cb0 tx=0x7f476001cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379b3bf8028ffa06 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:55.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.233+0000 7f4770a00640 1 -- 172.21.15.161:0/548935683 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47600a58d0 con 0x7f47580047d0 2024-05-22T21:13:55.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.233+0000 7f4770a00640 1 -- 172.21.15.161:0/548935683 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47600a5a30 con 0x7f47580047d0 2024-05-22T21:13:55.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.233+0000 7f4770a00640 1 -- 172.21.15.161:0/548935683 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47600a5e20 con 0x7f47580047d0 2024-05-22T21:13:55.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.233+0000 7f4778c00640 1 -- 172.21.15.161:0/548935683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47580047d0 msgr2=0x7f4758004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.233+0000 7f4778c00640 1 --2- 172.21.15.161:0/548935683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47580047d0 0x7f4758004c50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4760009cb0 tx=0x7f476001cc10 comp rx=0 tx=0).stop 2024-05-22T21:13:55.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.233+0000 7f4778c00640 1 -- 172.21.15.161:0/548935683 shutdown_connections 2024-05-22T21:13:55.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.233+0000 7f4778c00640 1 --2- 172.21.15.161:0/548935683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47580047d0 0x7f4758004c50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.233+0000 7f4778c00640 1 -- 172.21.15.161:0/548935683 >> 172.21.15.161:0/548935683 conn(0x7f477406a2d0 msgr2=0x7f477406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:55.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.233+0000 7f4778c00640 1 -- 172.21.15.161:0/548935683 shutdown_connections 2024-05-22T21:13:55.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.233+0000 7f4778c00640 1 -- 172.21.15.161:0/548935683 wait complete. 2024-05-22T21:13:55.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.233+0000 7f4778c00640 1 Processor -- start 2024-05-22T21:13:55.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.233+0000 7f4778c00640 1 -- start start 2024-05-22T21:13:55.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4778c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47580047d0 0x7f4774152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4778c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f477415a990 0x7f47740addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4778c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4774070820 0x7f477415da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4778c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47740ae310 con 0x7f47580047d0 2024-05-22T21:13:55.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4778c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477406fd80 con 0x7f4774070820 2024-05-22T21:13:55.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4778c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4774071360 con 0x7f477415a990 2024-05-22T21:13:55.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4771e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47580047d0 0x7f4774152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4771e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47580047d0 0x7f4774152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47072/0 (socket says 172.21.15.161:47072) 2024-05-22T21:13:55.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4771e00640 1 -- 172.21.15.161:0/3998529315 learned_addr learned my addr 172.21.15.161:0/3998529315 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:55.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4771400640 1 --2- 172.21.15.161:0/3998529315 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f477415a990 0x7f47740addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4772800640 1 --2- 172.21.15.161:0/3998529315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4774070820 0x7f477415da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4771e00640 1 -- 172.21.15.161:0/3998529315 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f477415a990 msgr2=0x7f47740addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4771e00640 1 --2- 172.21.15.161:0/3998529315 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f477415a990 0x7f47740addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4771e00640 1 -- 172.21.15.161:0/3998529315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4774070820 msgr2=0x7f477415da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4771e00640 1 --2- 172.21.15.161:0/3998529315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4774070820 0x7f477415da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4771e00640 1 -- 172.21.15.161:0/3998529315 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4774070020 con 0x7f47580047d0 2024-05-22T21:13:55.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4771e00640 1 --2- 172.21.15.161:0/3998529315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47580047d0 0x7f4774152980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4764011c60 tx=0x7f476400f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:55.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f476a000640 1 -- 172.21.15.161:0/3998529315 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476400fe50 con 0x7f47580047d0 2024-05-22T21:13:55.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f476a000640 1 -- 172.21.15.161:0/3998529315 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4764015410 con 0x7f47580047d0 2024-05-22T21:13:55.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f476a000640 1 -- 172.21.15.161:0/3998529315 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640157c0 con 0x7f47580047d0 2024-05-22T21:13:55.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4778c00640 1 -- 172.21.15.161:0/3998529315 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4774070300 con 0x7f47580047d0 2024-05-22T21:13:55.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4778c00640 1 -- 172.21.15.161:0/3998529315 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4774070cd0 con 0x7f47580047d0 2024-05-22T21:13:55.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.237+0000 7f4778c00640 1 -- 172.21.15.161:0/3998529315 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47740704f0 con 0x7f47580047d0 2024-05-22T21:13:55.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.241+0000 7f476a000640 1 -- 172.21.15.161:0/3998529315 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4764015920 con 0x7f47580047d0 2024-05-22T21:13:55.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.241+0000 7f476a000640 1 --2- 172.21.15.161:0/3998529315 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4754066690 0x7f4754068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.241+0000 7f476a000640 1 -- 172.21.15.161:0/3998529315 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47640877e0 con 0x7f47580047d0 2024-05-22T21:13:55.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.241+0000 7f4771400640 1 --2- 172.21.15.161:0/3998529315 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4754066690 0x7f4754068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.241+0000 7f4771400640 1 --2- 172.21.15.161:0/3998529315 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4754066690 0x7f4754068b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f47600af000 tx=0x7f47600046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:55.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.245+0000 7f476a000640 1 -- 172.21.15.161:0/3998529315 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4764054b50 con 0x7f47580047d0 2024-05-22T21:13:55.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.397+0000 7f4778c00640 1 -- 172.21.15.161:0/3998529315 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4774067220 con 0x7f47580047d0 2024-05-22T21:13:55.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.397+0000 7f476a000640 1 -- 172.21.15.161:0/3998529315 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4764058170 con 0x7f47580047d0 2024-05-22T21:13:55.403 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:13:55.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.401+0000 7f4778c00640 1 -- 172.21.15.161:0/3998529315 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4754066690 msgr2=0x7f4754068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.401+0000 7f4778c00640 1 --2- 172.21.15.161:0/3998529315 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4754066690 0x7f4754068b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f47600af000 tx=0x7f47600046d0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.401+0000 7f4778c00640 1 -- 172.21.15.161:0/3998529315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47580047d0 msgr2=0x7f4774152980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.401+0000 7f4778c00640 1 --2- 172.21.15.161:0/3998529315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47580047d0 0x7f4774152980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4764011c60 tx=0x7f476400f540 comp rx=0 tx=0).stop 2024-05-22T21:13:55.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.401+0000 7f4778c00640 1 -- 172.21.15.161:0/3998529315 shutdown_connections 2024-05-22T21:13:55.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.401+0000 7f4778c00640 1 --2- 172.21.15.161:0/3998529315 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4754066690 0x7f4754068b50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.401+0000 7f4778c00640 1 --2- 172.21.15.161:0/3998529315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4774070820 0x7f477415da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.401+0000 7f4778c00640 1 --2- 172.21.15.161:0/3998529315 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f477415a990 0x7f47740addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.401+0000 7f4778c00640 1 --2- 172.21.15.161:0/3998529315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47580047d0 0x7f4774152980 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.401+0000 7f4778c00640 1 -- 172.21.15.161:0/3998529315 >> 172.21.15.161:0/3998529315 conn(0x7f477406a2d0 msgr2=0x7f47740afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:55.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.401+0000 7f4778c00640 1 -- 172.21.15.161:0/3998529315 shutdown_connections 2024-05-22T21:13:55.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.401+0000 7f4778c00640 1 -- 172.21.15.161:0/3998529315 wait complete. 2024-05-22T21:13:55.423 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:13:55.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.561+0000 7fc8e6400640 1 Processor -- start 2024-05-22T21:13:55.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.561+0000 7fc8e6400640 1 -- start start 2024-05-22T21:13:55.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.561+0000 7fc8e6400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc8e0160ad0 0x7fc8e0099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.561+0000 7fc8e6400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc8e00b37b0 0x7fc8e0099f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.561+0000 7fc8e6400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e006e4a0 con 0x7fc8e00d0bb0 2024-05-22T21:13:55.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.561+0000 7fc8e6400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e006c690 con 0x7fc8e0160ad0 2024-05-22T21:13:55.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.561+0000 7fc8e6400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e006c0f0 con 0x7fc8e00b37b0 2024-05-22T21:13:55.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.561+0000 7fc8df400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc8e0160ad0 0x7fc8e0099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.561+0000 7fc8df400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc8e0160ad0 0x7fc8e0099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47078/0 (socket says 172.21.15.161:47078) 2024-05-22T21:13:55.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.561+0000 7fc8df400640 1 -- 172.21.15.161:0/4087013613 learned_addr learned my addr 172.21.15.161:0/4087013613 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:55.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.561+0000 7fc8dfe00640 1 --2- 172.21.15.161:0/4087013613 >> v2:172.21.15.179:3300/0 conn(0x7fc8e00b37b0 0x7fc8e0099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.561+0000 7fc8df400640 1 -- 172.21.15.161:0/4087013613 >> v2:172.21.15.179:3300/0 conn(0x7fc8e00b37b0 msgr2=0x7fc8e0099f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:55.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.561+0000 7fc8df400640 1 --2- 172.21.15.161:0/4087013613 >> v2:172.21.15.179:3300/0 conn(0x7fc8e00b37b0 0x7fc8e0099f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.561+0000 7fc8df400640 1 -- 172.21.15.161:0/4087013613 >> v1:172.21.15.179:6789/0 conn(0x7fc8e00d0bb0 legacy=0x7fc8e0160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.561+0000 7fc8df400640 1 -- 172.21.15.161:0/4087013613 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e006cf50 con 0x7fc8e0160ad0 2024-05-22T21:13:55.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.565+0000 7fc8df400640 1 --2- 172.21.15.161:0/4087013613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e0160ad0 0x7fc8e0099a40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc8e006d7f0 tx=0x7fc8d001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e5672b8e1cc4d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:55.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.565+0000 7fc8de000640 1 -- 172.21.15.161:0/4087013613 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d00aa680 con 0x7fc8e0160ad0 2024-05-22T21:13:55.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.565+0000 7fc8de000640 1 -- 172.21.15.161:0/4087013613 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d00aa7e0 con 0x7fc8e0160ad0 2024-05-22T21:13:55.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.565+0000 7fc8de000640 1 -- 172.21.15.161:0/4087013613 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d00aab70 con 0x7fc8e0160ad0 2024-05-22T21:13:55.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.565+0000 7fc8e6400640 1 -- 172.21.15.161:0/4087013613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e0160ad0 msgr2=0x7fc8e0099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.565+0000 7fc8e6400640 1 --2- 172.21.15.161:0/4087013613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e0160ad0 0x7fc8e0099a40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc8e006d7f0 tx=0x7fc8d001c520 comp rx=0 tx=0).stop 2024-05-22T21:13:55.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.565+0000 7fc8e6400640 1 -- 172.21.15.161:0/4087013613 shutdown_connections 2024-05-22T21:13:55.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.565+0000 7fc8e6400640 1 --2- 172.21.15.161:0/4087013613 >> v2:172.21.15.179:3300/0 conn(0x7fc8e00b37b0 0x7fc8e0099f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.565+0000 7fc8e6400640 1 --2- 172.21.15.161:0/4087013613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e0160ad0 0x7fc8e0099a40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.565+0000 7fc8e6400640 1 -- 172.21.15.161:0/4087013613 >> 172.21.15.161:0/4087013613 conn(0x7fc8e00b2a80 msgr2=0x7fc8e00b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:55.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.565+0000 7fc8e6400640 1 -- 172.21.15.161:0/4087013613 shutdown_connections 2024-05-22T21:13:55.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.565+0000 7fc8e6400640 1 -- 172.21.15.161:0/4087013613 wait complete. 2024-05-22T21:13:55.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8e6400640 1 Processor -- start 2024-05-22T21:13:55.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8e6400640 1 -- start start 2024-05-22T21:13:55.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8e6400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8e00b37b0 0x7fc8e00c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8e6400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e00d0bb0 0x7fc8e014afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8e6400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8e0160ad0 0x7fc8e014b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8e6400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e00c5b00 con 0x7fc8e00d0bb0 2024-05-22T21:13:55.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8e6400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e00c5de0 con 0x7fc8e00b37b0 2024-05-22T21:13:55.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8e6400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e00c60c0 con 0x7fc8e0160ad0 2024-05-22T21:13:55.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8dea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e00d0bb0 0x7fc8e014afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8dea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e00d0bb0 0x7fc8e014afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47092/0 (socket says 172.21.15.161:47092) 2024-05-22T21:13:55.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8dea00640 1 -- 172.21.15.161:0/3815362665 learned_addr learned my addr 172.21.15.161:0/3815362665 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:55.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8df400640 1 --2- 172.21.15.161:0/3815362665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8e00b37b0 0x7fc8e00c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8dfe00640 1 --2- 172.21.15.161:0/3815362665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8e0160ad0 0x7fc8e014b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8dea00640 1 -- 172.21.15.161:0/3815362665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8e0160ad0 msgr2=0x7fc8e014b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8dea00640 1 --2- 172.21.15.161:0/3815362665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8e0160ad0 0x7fc8e014b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8dea00640 1 -- 172.21.15.161:0/3815362665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8e00b37b0 msgr2=0x7fc8e00c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8dea00640 1 --2- 172.21.15.161:0/3815362665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8e00b37b0 0x7fc8e00c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8dea00640 1 -- 172.21.15.161:0/3815362665 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d00094d0 con 0x7fc8e00d0bb0 2024-05-22T21:13:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8dfe00640 1 --2- 172.21.15.161:0/3815362665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8e0160ad0 0x7fc8e014b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8dea00640 1 --2- 172.21.15.161:0/3815362665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e00d0bb0 0x7fc8e014afa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00c400 tx=0x7fc8cc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8d7400640 1 -- 172.21.15.161:0/3815362665 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc0052d0 con 0x7fc8e00d0bb0 2024-05-22T21:13:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8d7400640 1 -- 172.21.15.161:0/3815362665 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8cc005430 con 0x7fc8e00d0bb0 2024-05-22T21:13:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8d7400640 1 -- 172.21.15.161:0/3815362665 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc003840 con 0x7fc8e00d0bb0 2024-05-22T21:13:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8e6400640 1 -- 172.21.15.161:0/3815362665 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e00c63a0 con 0x7fc8e00d0bb0 2024-05-22T21:13:55.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8e6400640 1 -- 172.21.15.161:0/3815362665 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e00ca090 con 0x7fc8e00d0bb0 2024-05-22T21:13:55.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.569+0000 7fc8e6400640 1 -- 172.21.15.161:0/3815362665 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e0073ca0 con 0x7fc8e00d0bb0 2024-05-22T21:13:55.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.573+0000 7fc8d7400640 1 -- 172.21.15.161:0/3815362665 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc8cc0039a0 con 0x7fc8e00d0bb0 2024-05-22T21:13:55.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.573+0000 7fc8d7400640 1 --2- 172.21.15.161:0/3815362665 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc8c00666c0 0x7fc8c0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.573+0000 7fc8d7400640 1 -- 172.21.15.161:0/3815362665 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8cc083750 con 0x7fc8e00d0bb0 2024-05-22T21:13:55.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.573+0000 7fc8df400640 1 --2- 172.21.15.161:0/3815362665 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc8c00666c0 0x7fc8c0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.573+0000 7fc8df400640 1 --2- 172.21.15.161:0/3815362665 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc8c00666c0 0x7fc8c0068b80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00ae000 tx=0x7fc8d00b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:55.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.577+0000 7fc8d7400640 1 -- 172.21.15.161:0/3815362665 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc8cc050ad0 con 0x7fc8e00d0bb0 2024-05-22T21:13:55.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.733+0000 7fc8e6400640 1 -- 172.21.15.161:0/3815362665 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8e00721d0 con 0x7fc8e00d0bb0 2024-05-22T21:13:55.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.733+0000 7fc8d7400640 1 -- 172.21.15.161:0/3815362665 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc8cc0540f0 con 0x7fc8e00d0bb0 2024-05-22T21:13:55.740 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:13:55.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.737+0000 7fc8e6400640 1 -- 172.21.15.161:0/3815362665 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc8c00666c0 msgr2=0x7fc8c0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.737+0000 7fc8e6400640 1 --2- 172.21.15.161:0/3815362665 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc8c00666c0 0x7fc8c0068b80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00ae000 tx=0x7fc8d00b5040 comp rx=0 tx=0).stop 2024-05-22T21:13:55.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.737+0000 7fc8e6400640 1 -- 172.21.15.161:0/3815362665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e00d0bb0 msgr2=0x7fc8e014afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.737+0000 7fc8e6400640 1 --2- 172.21.15.161:0/3815362665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e00d0bb0 0x7fc8e014afa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00c400 tx=0x7fc8cc00c8c0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.737+0000 7fc8e6400640 1 -- 172.21.15.161:0/3815362665 shutdown_connections 2024-05-22T21:13:55.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.737+0000 7fc8e6400640 1 --2- 172.21.15.161:0/3815362665 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc8c00666c0 0x7fc8c0068b80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.737+0000 7fc8e6400640 1 --2- 172.21.15.161:0/3815362665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8e0160ad0 0x7fc8e014b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.737+0000 7fc8e6400640 1 --2- 172.21.15.161:0/3815362665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e00d0bb0 0x7fc8e014afa0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.737+0000 7fc8e6400640 1 --2- 172.21.15.161:0/3815362665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8e00b37b0 0x7fc8e00c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.737+0000 7fc8e6400640 1 -- 172.21.15.161:0/3815362665 >> 172.21.15.161:0/3815362665 conn(0x7fc8e00b2a80 msgr2=0x7fc8e00b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:55.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.737+0000 7fc8e6400640 1 -- 172.21.15.161:0/3815362665 shutdown_connections 2024-05-22T21:13:55.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.737+0000 7fc8e6400640 1 -- 172.21.15.161:0/3815362665 wait complete. 2024-05-22T21:13:55.771 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestClusterFull) That when an OSD epoch barrier is set on an MDS, subsequently 2024-05-22T21:13:55.771 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestClusterFull) 2024-05-22T21:13:55.772 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2024-05-22T21:13:55.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.901+0000 7f3af8e00640 1 Processor -- start 2024-05-22T21:13:55.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.901+0000 7f3af8e00640 1 -- start start 2024-05-22T21:13:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af8e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3af415c6d0 0x7f3af4164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af8e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af4071670 con 0x7f3af415dd50 2024-05-22T21:13:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af8e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af4070a60 con 0x7f3af415e130 2024-05-22T21:13:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af8e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af4070d40 con 0x7f3af415c6d0 2024-05-22T21:13:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af1e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3af415dd50 0x7f3af406d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33668/0 (socket says 172.21.15.161:33668) 2024-05-22T21:13:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af1e00640 1 -- 172.21.15.161:0/1088346980 learned_addr learned my addr 172.21.15.161:0/1088346980 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:55.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af0a00640 1 -- 172.21.15.161:0/1088346980 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1177317827 0 0) 0x7f3af4071670 con 0x7f3af415dd50 2024-05-22T21:13:55.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af0a00640 1 -- 172.21.15.161:0/1088346980 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ad8005740 con 0x7f3af415dd50 2024-05-22T21:13:55.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af2800640 1 --2- 172.21.15.161:0/1088346980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3af415c6d0 0x7f3af4164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af0a00640 1 -- 172.21.15.161:0/1088346980 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3879145837 0 0) 0x7f3ad8005740 con 0x7f3af415dd50 2024-05-22T21:13:55.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af0a00640 1 -- 172.21.15.161:0/1088346980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3af415c6d0 msgr2=0x7f3af4164e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:55.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af0a00640 1 --2- 172.21.15.161:0/1088346980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3af415c6d0 0x7f3af4164e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af0a00640 1 -- 172.21.15.161:0/1088346980 >> v1:172.21.15.179:6789/0 conn(0x7f3af415e130 legacy=0x7f3af415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af0a00640 1 -- 172.21.15.161:0/1088346980 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af4071020 con 0x7f3af415dd50 2024-05-22T21:13:55.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af0a00640 1 -- 172.21.15.161:0/1088346980 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f3ae4001680 con 0x7f3af415dd50 2024-05-22T21:13:55.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af0a00640 1 -- 172.21.15.161:0/1088346980 >> v1:172.21.15.161:6789/0 conn(0x7f3af415dd50 legacy=0x7f3af406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af0a00640 1 --2- 172.21.15.161:0/1088346980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3af415c6d0 0x7f3ad8005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af0a00640 1 --2- 172.21.15.161:0/1088346980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ad80061f0 0x7f3ad80069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af0a00640 1 --2- 172.21.15.161:0/1088346980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad8006f20 0x7f3ad80073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af0a00640 1 -- 172.21.15.161:0/1088346980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af4070d40 con 0x7f3ad80061f0 2024-05-22T21:13:55.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af0a00640 1 -- 172.21.15.161:0/1088346980 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad800dd60 con 0x7f3ad8006f20 2024-05-22T21:13:55.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af0a00640 1 -- 172.21.15.161:0/1088346980 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad800df20 con 0x7f3af415c6d0 2024-05-22T21:13:55.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af1400640 1 --2- 172.21.15.161:0/1088346980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ad80061f0 0x7f3ad80069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af1e00640 1 --2- 172.21.15.161:0/1088346980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3af415c6d0 0x7f3ad8005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af2800640 1 --2- 172.21.15.161:0/1088346980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad8006f20 0x7f3ad80073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af1400640 1 -- 172.21.15.161:0/1088346980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3af415c6d0 msgr2=0x7f3ad8005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af1400640 1 --2- 172.21.15.161:0/1088346980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3af415c6d0 0x7f3ad8005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af1400640 1 -- 172.21.15.161:0/1088346980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad8006f20 msgr2=0x7f3ad80073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af1400640 1 --2- 172.21.15.161:0/1088346980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad8006f20 0x7f3ad80073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af1400640 1 -- 172.21.15.161:0/1088346980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad8005740 con 0x7f3ad80061f0 2024-05-22T21:13:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.905+0000 7f3af2800640 1 --2- 172.21.15.161:0/1088346980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad8006f20 0x7f3ad80073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:55.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.909+0000 7f3af1400640 1 --2- 172.21.15.161:0/1088346980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ad80061f0 0x7f3ad80069e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0001780 tx=0x7f3ae001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb363a607d8fa254 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:55.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.909+0000 7f3af0a00640 1 -- 172.21.15.161:0/1088346980 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae00052d0 con 0x7f3ad80061f0 2024-05-22T21:13:55.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.909+0000 7f3af0a00640 1 -- 172.21.15.161:0/1088346980 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae00a7050 con 0x7f3ad80061f0 2024-05-22T21:13:55.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.909+0000 7f3af0a00640 1 -- 172.21.15.161:0/1088346980 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0003600 con 0x7f3ad80061f0 2024-05-22T21:13:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.909+0000 7f3af8e00640 1 -- 172.21.15.161:0/1088346980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ad80061f0 msgr2=0x7f3ad80069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.909+0000 7f3af8e00640 1 --2- 172.21.15.161:0/1088346980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ad80061f0 0x7f3ad80069e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0001780 tx=0x7f3ae001c420 comp rx=0 tx=0).stop 2024-05-22T21:13:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.909+0000 7f3af2800640 1 -- 172.21.15.161:0/1088346980 reap_dead start 2024-05-22T21:13:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.909+0000 7f3af8e00640 1 -- 172.21.15.161:0/1088346980 shutdown_connections 2024-05-22T21:13:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.909+0000 7f3af8e00640 1 -- 172.21.15.161:0/1088346980 >> 172.21.15.161:0/1088346980 conn(0x7f3af40b0bc0 msgr2=0x7f3af40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.909+0000 7f3af8e00640 1 -- 172.21.15.161:0/1088346980 shutdown_connections 2024-05-22T21:13:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.909+0000 7f3af8e00640 1 -- 172.21.15.161:0/1088346980 wait complete. 2024-05-22T21:13:55.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af8e00640 1 Processor -- start 2024-05-22T21:13:55.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af8e00640 1 -- start start 2024-05-22T21:13:55.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af8e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ad80061f0 0x7f3af40b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af8e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3af4159360 0x7f3af414ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af8e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40cd910 0x7f3af40cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af8e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af414d590 con 0x7f3af4159360 2024-05-22T21:13:55.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af8e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af40ce2d0 con 0x7f3af40cd910 2024-05-22T21:13:55.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af8e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af40b3610 con 0x7f3ad80061f0 2024-05-22T21:13:55.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af1400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3af4159360 0x7f3af414ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af1400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3af4159360 0x7f3af414ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58980/0 (socket says 172.21.15.161:58980) 2024-05-22T21:13:55.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af1400640 1 -- 172.21.15.161:0/2038519153 learned_addr learned my addr 172.21.15.161:0/2038519153 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:55.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af1e00640 1 --2- 172.21.15.161:0/2038519153 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ad80061f0 0x7f3af40b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af2800640 1 --2- 172.21.15.161:0/2038519153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40cd910 0x7f3af40cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af1400640 1 -- 172.21.15.161:0/2038519153 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ad80061f0 msgr2=0x7f3af40b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af1400640 1 --2- 172.21.15.161:0/2038519153 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ad80061f0 0x7f3af40b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af1400640 1 -- 172.21.15.161:0/2038519153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40cd910 msgr2=0x7f3af40cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:55.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af1400640 1 --2- 172.21.15.161:0/2038519153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40cd910 0x7f3af40cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:55.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af1400640 1 -- 172.21.15.161:0/2038519153 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af40b38f0 con 0x7f3af4159360 2024-05-22T21:13:55.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af2800640 1 --2- 172.21.15.161:0/2038519153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40cd910 0x7f3af40cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:55.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af1400640 1 --2- 172.21.15.161:0/2038519153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3af4159360 0x7f3af414ae30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0005040 tx=0x7f3ad8007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:55.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3aea000640 1 -- 172.21.15.161:0/2038519153 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0007390 con 0x7f3af4159360 2024-05-22T21:13:55.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3aea000640 1 -- 172.21.15.161:0/2038519153 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae00074f0 con 0x7f3af4159360 2024-05-22T21:13:55.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af8e00640 1 -- 172.21.15.161:0/2038519153 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af40b3bd0 con 0x7f3af4159360 2024-05-22T21:13:55.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3aea000640 1 -- 172.21.15.161:0/2038519153 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0006a20 con 0x7f3af4159360 2024-05-22T21:13:55.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af8e00640 1 -- 172.21.15.161:0/2038519153 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af414b370 con 0x7f3af4159360 2024-05-22T21:13:55.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.913+0000 7f3af8e00640 1 -- 172.21.15.161:0/2038519153 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af40b3e10 con 0x7f3af4159360 2024-05-22T21:13:55.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.917+0000 7f3aea000640 1 -- 172.21.15.161:0/2038519153 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3ae0006b80 con 0x7f3af4159360 2024-05-22T21:13:55.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.917+0000 7f3aea000640 1 --2- 172.21.15.161:0/2038519153 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ad40666c0 0x7f3ad4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:55.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.917+0000 7f3aea000640 1 -- 172.21.15.161:0/2038519153 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ae011e480 con 0x7f3af4159360 2024-05-22T21:13:55.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.917+0000 7f3af1e00640 1 --2- 172.21.15.161:0/2038519153 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ad40666c0 0x7f3ad4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:55.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.917+0000 7f3af1e00640 1 --2- 172.21.15.161:0/2038519153 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ad40666c0 0x7f3ad4068b80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4001060 tx=0x7f3ae4001c20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:55.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:55.921+0000 7f3aea000640 1 -- 172.21.15.161:0/2038519153 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3af40b3e10 con 0x7f3af4159360 2024-05-22T21:13:56.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.129+0000 7f3af8e00640 1 -- 172.21.15.161:0/2038519153 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7f3af40b3e10 con 0x7f3af4159360 2024-05-22T21:13:56.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.569+0000 7f3aea000640 1 -- 172.21.15.161:0/2038519153 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v50) v1 ==== 133+0+0 (secure 0 0 0) 0x7f3af40b3e10 con 0x7f3af4159360 2024-05-22T21:13:56.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.569+0000 7f3af8e00640 1 -- 172.21.15.161:0/2038519153 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ad40666c0 msgr2=0x7f3ad4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:56.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.569+0000 7f3af8e00640 1 --2- 172.21.15.161:0/2038519153 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ad40666c0 0x7f3ad4068b80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4001060 tx=0x7f3ae4001c20 comp rx=0 tx=0).stop 2024-05-22T21:13:56.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.569+0000 7f3af8e00640 1 -- 172.21.15.161:0/2038519153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3af4159360 msgr2=0x7f3af414ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:56.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.569+0000 7f3af8e00640 1 --2- 172.21.15.161:0/2038519153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3af4159360 0x7f3af414ae30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0005040 tx=0x7f3ad8007a10 comp rx=0 tx=0).stop 2024-05-22T21:13:56.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.569+0000 7f3af8e00640 1 -- 172.21.15.161:0/2038519153 shutdown_connections 2024-05-22T21:13:56.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.569+0000 7f3af8e00640 1 --2- 172.21.15.161:0/2038519153 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ad40666c0 0x7f3ad4068b80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:56.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.569+0000 7f3af8e00640 1 --2- 172.21.15.161:0/2038519153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3af40cd910 0x7f3af40cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:56.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.569+0000 7f3af8e00640 1 --2- 172.21.15.161:0/2038519153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3af4159360 0x7f3af414ae30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:56.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.569+0000 7f3af8e00640 1 --2- 172.21.15.161:0/2038519153 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ad80061f0 0x7f3af40b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:56.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.569+0000 7f3af8e00640 1 -- 172.21.15.161:0/2038519153 >> 172.21.15.161:0/2038519153 conn(0x7f3af40b0bc0 msgr2=0x7f3af4155a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:56.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.569+0000 7f3af8e00640 1 -- 172.21.15.161:0/2038519153 shutdown_connections 2024-05-22T21:13:56.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.569+0000 7f3af8e00640 1 -- 172.21.15.161:0/2038519153 wait complete. 2024-05-22T21:13:56.595 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-05-22T21:13:56.746 INFO:teuthology.orchestra.run.smithi161.stdout:{"osd_objectstore":"memstore"} 2024-05-22T21:13:56.747 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-05-22T21:13:56.747 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-22T21:13:56.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.881+0000 7fa7bc000640 1 Processor -- start 2024-05-22T21:13:56.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.881+0000 7fa7bc000640 1 -- start start 2024-05-22T21:13:56.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.881+0000 7fa7bc000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa7b415e5f0 0x7fa7b406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:56.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.881+0000 7fa7bc000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7b406fad0 0x7fa7b415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:56.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.881+0000 7fa7bc000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b406f030 con 0x7fa7b415f990 2024-05-22T21:13:56.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.881+0000 7fa7bc000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b406f840 con 0x7fa7b415e5f0 2024-05-22T21:13:56.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.881+0000 7fa7bc000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b415efc0 con 0x7fa7b406fad0 2024-05-22T21:13:56.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.881+0000 7fa7b9600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa7b415e5f0 0x7fa7b406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:56.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.881+0000 7fa7b9600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa7b415e5f0 0x7fa7b406e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:33480/0 (socket says 172.21.15.161:33480) 2024-05-22T21:13:56.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.881+0000 7fa7b9600640 1 -- 172.21.15.161:0/1116009954 learned_addr learned my addr 172.21.15.161:0/1116009954 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:56.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.881+0000 7fa7b8c00640 1 --2- 172.21.15.161:0/1116009954 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7b406fad0 0x7fa7b415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:56.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.881+0000 7fa7b9600640 1 -- 172.21.15.161:0/1116009954 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7b406fad0 msgr2=0x7fa7b415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:56.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.881+0000 7fa7b9600640 1 --2- 172.21.15.161:0/1116009954 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7b406fad0 0x7fa7b415f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:56.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.881+0000 7fa7b9600640 1 -- 172.21.15.161:0/1116009954 >> v1:172.21.15.179:6789/0 conn(0x7fa7b415f990 legacy=0x7fa7b40cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:56.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.881+0000 7fa7b9600640 1 -- 172.21.15.161:0/1116009954 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b40cfcc0 con 0x7fa7b415e5f0 2024-05-22T21:13:56.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.885+0000 7fa7b9600640 1 --2- 172.21.15.161:0/1116009954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7b415e5f0 0x7fa7b406e9a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009870 tx=0x7fa7a801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bed55afe573f5a01 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:56.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.885+0000 7fa7b3e00640 1 -- 172.21.15.161:0/1116009954 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a80ab610 con 0x7fa7b415e5f0 2024-05-22T21:13:56.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.885+0000 7fa7b3e00640 1 -- 172.21.15.161:0/1116009954 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a80ab770 con 0x7fa7b415e5f0 2024-05-22T21:13:56.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.885+0000 7fa7b3e00640 1 -- 172.21.15.161:0/1116009954 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a80abb00 con 0x7fa7b415e5f0 2024-05-22T21:13:56.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.885+0000 7fa7bc000640 1 -- 172.21.15.161:0/1116009954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7b415e5f0 msgr2=0x7fa7b406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:56.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.885+0000 7fa7bc000640 1 --2- 172.21.15.161:0/1116009954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7b415e5f0 0x7fa7b406e9a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009870 tx=0x7fa7a801c410 comp rx=0 tx=0).stop 2024-05-22T21:13:56.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.885+0000 7fa7bc000640 1 -- 172.21.15.161:0/1116009954 shutdown_connections 2024-05-22T21:13:56.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.885+0000 7fa7bc000640 1 --2- 172.21.15.161:0/1116009954 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7b406fad0 0x7fa7b415f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:56.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.885+0000 7fa7bc000640 1 --2- 172.21.15.161:0/1116009954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7b415e5f0 0x7fa7b406e9a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:56.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.885+0000 7fa7bc000640 1 -- 172.21.15.161:0/1116009954 >> 172.21.15.161:0/1116009954 conn(0x7fa7b40b2a80 msgr2=0x7fa7b40b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:56.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.885+0000 7fa7bc000640 1 -- 172.21.15.161:0/1116009954 shutdown_connections 2024-05-22T21:13:56.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.885+0000 7fa7bc000640 1 -- 172.21.15.161:0/1116009954 wait complete. 2024-05-22T21:13:56.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7bc000640 1 Processor -- start 2024-05-22T21:13:56.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7bc000640 1 -- start start 2024-05-22T21:13:56.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7bc000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7b406fad0 0x7fa7b4152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:56.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7bc000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7b415e5f0 0x7fa7b406c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:56.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7bc000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b415f990 0x7fa7b41588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:56.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7bc000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b4153270 con 0x7fa7b415f990 2024-05-22T21:13:56.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7bc000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b415ac80 con 0x7fa7b406fad0 2024-05-22T21:13:56.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7bc000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b415a440 con 0x7fa7b415e5f0 2024-05-22T21:13:56.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7ba000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b415f990 0x7fa7b41588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:56.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7ba000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b415f990 0x7fa7b41588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58996/0 (socket says 172.21.15.161:58996) 2024-05-22T21:13:56.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7ba000640 1 -- 172.21.15.161:0/90548702 learned_addr learned my addr 172.21.15.161:0/90548702 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:56.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7b9600640 1 --2- 172.21.15.161:0/90548702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7b406fad0 0x7fa7b4152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:56.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7b8c00640 1 --2- 172.21.15.161:0/90548702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7b415e5f0 0x7fa7b406c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:56.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7ba000640 1 -- 172.21.15.161:0/90548702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7b415e5f0 msgr2=0x7fa7b406c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:56.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7ba000640 1 --2- 172.21.15.161:0/90548702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7b415e5f0 0x7fa7b406c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:56.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7ba000640 1 -- 172.21.15.161:0/90548702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7b406fad0 msgr2=0x7fa7b4152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:56.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7ba000640 1 --2- 172.21.15.161:0/90548702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7b406fad0 0x7fa7b4152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:56.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7ba000640 1 -- 172.21.15.161:0/90548702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a80094d0 con 0x7fa7b415f990 2024-05-22T21:13:56.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7ba000640 1 --2- 172.21.15.161:0/90548702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b415f990 0x7fa7b41588e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00c730 tx=0x7fa7ac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:56.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7b1600640 1 -- 172.21.15.161:0/90548702 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac003b80 con 0x7fa7b415f990 2024-05-22T21:13:56.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7b1600640 1 -- 172.21.15.161:0/90548702 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac003ce0 con 0x7fa7b415f990 2024-05-22T21:13:56.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7b1600640 1 -- 172.21.15.161:0/90548702 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac00aa30 con 0x7fa7b415f990 2024-05-22T21:13:56.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7bc000640 1 -- 172.21.15.161:0/90548702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b415a6c0 con 0x7fa7b415f990 2024-05-22T21:13:56.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7bc000640 1 -- 172.21.15.161:0/90548702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b406bb20 con 0x7fa7b415f990 2024-05-22T21:13:56.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.889+0000 7fa7bc000640 1 -- 172.21.15.161:0/90548702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b40d0c40 con 0x7fa7b415f990 2024-05-22T21:13:56.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.893+0000 7fa7b1600640 1 -- 172.21.15.161:0/90548702 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa7ac00ab90 con 0x7fa7b415f990 2024-05-22T21:13:56.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.893+0000 7fa7b1600640 1 --2- 172.21.15.161:0/90548702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7940666c0 0x7fa794068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:56.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.893+0000 7fa7b1600640 1 -- 172.21.15.161:0/90548702 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7ac083dc0 con 0x7fa7b415f990 2024-05-22T21:13:56.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.893+0000 7fa7b9600640 1 --2- 172.21.15.161:0/90548702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7940666c0 0x7fa794068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:56.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.893+0000 7fa7b9600640 1 --2- 172.21.15.161:0/90548702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7940666c0 0x7fa794068b80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80af000 tx=0x7fa7a80b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:56.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:56.897+0000 7fa7b1600640 1 -- 172.21.15.161:0/90548702 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa7ac0511d0 con 0x7fa7b415f990 2024-05-22T21:13:57.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.049+0000 7fa7bc000640 1 -- 172.21.15.161:0/90548702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa7b4073ca0 con 0x7fa7b415f990 2024-05-22T21:13:57.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.053+0000 7fa7b1600640 1 -- 172.21.15.161:0/90548702 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa7ac0547f0 con 0x7fa7b415f990 2024-05-22T21:13:57.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.053+0000 7fa7bc000640 1 -- 172.21.15.161:0/90548702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7940666c0 msgr2=0x7fa794068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:57.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.053+0000 7fa7bc000640 1 --2- 172.21.15.161:0/90548702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7940666c0 0x7fa794068b80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80af000 tx=0x7fa7a80b6040 comp rx=0 tx=0).stop 2024-05-22T21:13:57.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.053+0000 7fa7bc000640 1 -- 172.21.15.161:0/90548702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b415f990 msgr2=0x7fa7b41588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:57.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.053+0000 7fa7bc000640 1 --2- 172.21.15.161:0/90548702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b415f990 0x7fa7b41588e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00c730 tx=0x7fa7ac00cbf0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.053+0000 7fa7bc000640 1 -- 172.21.15.161:0/90548702 shutdown_connections 2024-05-22T21:13:57.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.053+0000 7fa7bc000640 1 --2- 172.21.15.161:0/90548702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7940666c0 0x7fa794068b80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.053+0000 7fa7bc000640 1 --2- 172.21.15.161:0/90548702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b415f990 0x7fa7b41588e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.053+0000 7fa7bc000640 1 --2- 172.21.15.161:0/90548702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7b415e5f0 0x7fa7b406c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.053+0000 7fa7bc000640 1 --2- 172.21.15.161:0/90548702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7b406fad0 0x7fa7b4152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.053+0000 7fa7bc000640 1 -- 172.21.15.161:0/90548702 >> 172.21.15.161:0/90548702 conn(0x7fa7b40b2a80 msgr2=0x7fa7b4072f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:57.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.053+0000 7fa7bc000640 1 -- 172.21.15.161:0/90548702 shutdown_connections 2024-05-22T21:13:57.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.053+0000 7fa7bc000640 1 -- 172.21.15.161:0/90548702 wait complete. 2024-05-22T21:13:57.077 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:13:57.077 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:13:57.158 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:13:57.230 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:13:57.230 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:57.230 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:13:57.239 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:57.239 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:13:57.287 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:13:57.287 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:13:57.339 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:57.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.513+0000 7f1e85600640 1 Processor -- start 2024-05-22T21:13:57.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.513+0000 7f1e85600640 1 -- start start 2024-05-22T21:13:57.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.513+0000 7f1e85600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1e8015d040 0x7f1e800b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:57.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.513+0000 7f1e85600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1e800d0cf0 0x7f1e800d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:57.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.513+0000 7f1e85600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e800d16c0 con 0x7f1e800cd840 2024-05-22T21:13:57.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.513+0000 7f1e85600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e800ce390 con 0x7f1e8015d040 2024-05-22T21:13:57.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.513+0000 7f1e85600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e800b13d0 con 0x7f1e800d0cf0 2024-05-22T21:13:57.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.513+0000 7f1e7fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1e8015d040 0x7f1e800b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:57.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.513+0000 7f1e7fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1e8015d040 0x7f1e800b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59012/0 (socket says 172.21.15.161:59012) 2024-05-22T21:13:57.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.513+0000 7f1e7fe00640 1 -- 172.21.15.161:0/209660913 learned_addr learned my addr 172.21.15.161:0/209660913 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:57.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.513+0000 7f1e7fe00640 1 -- 172.21.15.161:0/209660913 >> v2:172.21.15.179:3300/0 conn(0x7f1e800d0cf0 msgr2=0x7f1e800d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:57.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.513+0000 7f1e7fe00640 1 --2- 172.21.15.161:0/209660913 >> v2:172.21.15.179:3300/0 conn(0x7f1e800d0cf0 0x7f1e800d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.513+0000 7f1e7fe00640 1 -- 172.21.15.161:0/209660913 >> v1:172.21.15.161:6789/0 conn(0x7f1e800cd840 legacy=0x7f1e800cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:57.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.513+0000 7f1e7fe00640 1 -- 172.21.15.161:0/209660913 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e800b0aa0 con 0x7f1e8015d040 2024-05-22T21:13:57.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.513+0000 7f1e7f400640 1 -- 172.21.15.161:0/209660913 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972463208 0 0) 0x7f1e800d16c0 con 0x7f1e800cd840 2024-05-22T21:13:57.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.517+0000 7f1e7fe00640 1 --2- 172.21.15.161:0/209660913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e8015d040 0x7f1e800b1860 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1e800b38f0 tx=0x7f1e7001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978aee796633829c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:57.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.517+0000 7f1e7f400640 1 -- 172.21.15.161:0/209660913 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e700aa650 con 0x7f1e8015d040 2024-05-22T21:13:57.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.517+0000 7f1e7f400640 1 -- 172.21.15.161:0/209660913 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e700aa7b0 con 0x7f1e8015d040 2024-05-22T21:13:57.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.517+0000 7f1e7f400640 1 -- 172.21.15.161:0/209660913 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e700aab40 con 0x7f1e8015d040 2024-05-22T21:13:57.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.517+0000 7f1e85600640 1 -- 172.21.15.161:0/209660913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e8015d040 msgr2=0x7f1e800b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:57.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.517+0000 7f1e85600640 1 --2- 172.21.15.161:0/209660913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e8015d040 0x7f1e800b1860 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1e800b38f0 tx=0x7f1e7001c4f0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.517+0000 7f1e85600640 1 -- 172.21.15.161:0/209660913 shutdown_connections 2024-05-22T21:13:57.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.517+0000 7f1e85600640 1 --2- 172.21.15.161:0/209660913 >> v2:172.21.15.179:3300/0 conn(0x7f1e800d0cf0 0x7f1e800d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.517+0000 7f1e85600640 1 --2- 172.21.15.161:0/209660913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e8015d040 0x7f1e800b1860 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.517+0000 7f1e85600640 1 -- 172.21.15.161:0/209660913 >> 172.21.15.161:0/209660913 conn(0x7f1e80099a40 msgr2=0x7f1e80099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:57.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.517+0000 7f1e85600640 1 -- 172.21.15.161:0/209660913 shutdown_connections 2024-05-22T21:13:57.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.517+0000 7f1e85600640 1 -- 172.21.15.161:0/209660913 wait complete. 2024-05-22T21:13:57.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e85600640 1 Processor -- start 2024-05-22T21:13:57.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e85600640 1 -- start start 2024-05-22T21:13:57.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e85600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e800cd840 0x7f1e80153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:57.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e85600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e800d0cf0 0x7f1e800aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:57.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e85600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e8015d040 0x7f1e800af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:57.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e85600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e800caa50 con 0x7f1e800d0cf0 2024-05-22T21:13:57.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e85600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8015b460 con 0x7f1e8015d040 2024-05-22T21:13:57.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e85600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e800b5440 con 0x7f1e800cd840 2024-05-22T21:13:57.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e77400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e800d0cf0 0x7f1e800aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:57.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e77400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e800d0cf0 0x7f1e800aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59014/0 (socket says 172.21.15.161:59014) 2024-05-22T21:13:57.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e77400640 1 -- 172.21.15.161:0/2497559156 learned_addr learned my addr 172.21.15.161:0/2497559156 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:57.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e7fe00640 1 --2- 172.21.15.161:0/2497559156 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e800cd840 0x7f1e80153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:57.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e84c00640 1 --2- 172.21.15.161:0/2497559156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e8015d040 0x7f1e800af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:57.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e77400640 1 -- 172.21.15.161:0/2497559156 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e800cd840 msgr2=0x7f1e80153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:57.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e77400640 1 --2- 172.21.15.161:0/2497559156 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e800cd840 0x7f1e80153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e77400640 1 -- 172.21.15.161:0/2497559156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e8015d040 msgr2=0x7f1e800af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:57.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e77400640 1 --2- 172.21.15.161:0/2497559156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e8015d040 0x7f1e800af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e77400640 1 -- 172.21.15.161:0/2497559156 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e700094d0 con 0x7f1e800d0cf0 2024-05-22T21:13:57.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e84c00640 1 --2- 172.21.15.161:0/2497559156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e8015d040 0x7f1e800af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:57.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e77400640 1 --2- 172.21.15.161:0/2497559156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e800d0cf0 0x7f1e800aeb10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800b550 tx=0x7f1e6800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:57.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e7cc00640 1 -- 172.21.15.161:0/2497559156 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68010870 con 0x7f1e800d0cf0 2024-05-22T21:13:57.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e7cc00640 1 -- 172.21.15.161:0/2497559156 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e680109f0 con 0x7f1e800d0cf0 2024-05-22T21:13:57.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e7cc00640 1 -- 172.21.15.161:0/2497559156 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68010da0 con 0x7f1e800d0cf0 2024-05-22T21:13:57.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e85600640 1 -- 172.21.15.161:0/2497559156 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e800b4e80 con 0x7f1e800d0cf0 2024-05-22T21:13:57.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e85600640 1 -- 172.21.15.161:0/2497559156 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8015b0f0 con 0x7f1e800d0cf0 2024-05-22T21:13:57.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.521+0000 7f1e85600640 1 -- 172.21.15.161:0/2497559156 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e8015af50 con 0x7f1e800d0cf0 2024-05-22T21:13:57.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.525+0000 7f1e7cc00640 1 -- 172.21.15.161:0/2497559156 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1e6800f3c0 con 0x7f1e800d0cf0 2024-05-22T21:13:57.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.525+0000 7f1e7cc00640 1 --2- 172.21.15.161:0/2497559156 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e5c066690 0x7f1e5c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:57.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.525+0000 7f1e7cc00640 1 -- 172.21.15.161:0/2497559156 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e68084190 con 0x7f1e800d0cf0 2024-05-22T21:13:57.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.525+0000 7f1e7fe00640 1 --2- 172.21.15.161:0/2497559156 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e5c066690 0x7f1e5c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:57.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.525+0000 7f1e7fe00640 1 --2- 172.21.15.161:0/2497559156 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e5c066690 0x7f1e5c068b50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1e700ae000 tx=0x7f1e700b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:57.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.529+0000 7f1e7cc00640 1 -- 172.21.15.161:0/2497559156 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1e8015af50 con 0x7f1e800d0cf0 2024-05-22T21:13:57.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.681+0000 7f1e85600640 1 -- 172.21.15.161:0/2497559156 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e8015af50 con 0x7f1e800d0cf0 2024-05-22T21:13:57.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.685+0000 7f1e7cc00640 1 -- 172.21.15.161:0/2497559156 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f1e8015af50 con 0x7f1e800d0cf0 2024-05-22T21:13:57.689 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:57.689 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:57.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.685+0000 7f1e85600640 1 -- 172.21.15.161:0/2497559156 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e5c066690 msgr2=0x7f1e5c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:57.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.685+0000 7f1e85600640 1 --2- 172.21.15.161:0/2497559156 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e5c066690 0x7f1e5c068b50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1e700ae000 tx=0x7f1e700b5040 comp rx=0 tx=0).stop 2024-05-22T21:13:57.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.685+0000 7f1e85600640 1 -- 172.21.15.161:0/2497559156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e800d0cf0 msgr2=0x7f1e800aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:57.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.685+0000 7f1e85600640 1 --2- 172.21.15.161:0/2497559156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e800d0cf0 0x7f1e800aeb10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800b550 tx=0x7f1e6800ba10 comp rx=0 tx=0).stop 2024-05-22T21:13:57.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.685+0000 7f1e85600640 1 -- 172.21.15.161:0/2497559156 shutdown_connections 2024-05-22T21:13:57.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.685+0000 7f1e85600640 1 --2- 172.21.15.161:0/2497559156 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e5c066690 0x7f1e5c068b50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.685+0000 7f1e85600640 1 --2- 172.21.15.161:0/2497559156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e8015d040 0x7f1e800af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.685+0000 7f1e85600640 1 --2- 172.21.15.161:0/2497559156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e800d0cf0 0x7f1e800aeb10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.685+0000 7f1e85600640 1 --2- 172.21.15.161:0/2497559156 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e800cd840 0x7f1e80153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.685+0000 7f1e85600640 1 -- 172.21.15.161:0/2497559156 >> 172.21.15.161:0/2497559156 conn(0x7f1e80099a40 msgr2=0x7f1e800cc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:57.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.685+0000 7f1e85600640 1 -- 172.21.15.161:0/2497559156 shutdown_connections 2024-05-22T21:13:57.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.685+0000 7f1e85600640 1 -- 172.21.15.161:0/2497559156 wait complete. 2024-05-22T21:13:57.709 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:13:21.237503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:13:22.425891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:13:57.710 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:13:57.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.841+0000 7f9a53600640 1 Processor -- start 2024-05-22T21:13:57.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.841+0000 7f9a53600640 1 -- start start 2024-05-22T21:13:57.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.845+0000 7f9a53600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9a4c06c9b0 0x7f9a4c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:57.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.845+0000 7f9a53600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4c144bb0 con 0x7f9a4c143c40 2024-05-22T21:13:57.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.845+0000 7f9a53600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4c15efb0 con 0x7f9a4c15b600 2024-05-22T21:13:57.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.845+0000 7f9a53600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a4c15ba60 con 0x7f9a4c06c9b0 2024-05-22T21:13:57.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.845+0000 7f9a50c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f9a4c143c40 0x7f9a4c1444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33696/0 (socket says 172.21.15.161:33696) 2024-05-22T21:13:57.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.845+0000 7f9a50c00640 1 -- 172.21.15.161:0/2233164247 learned_addr learned my addr 172.21.15.161:0/2233164247 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:57.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.845+0000 7f9a51600640 1 --2- 172.21.15.161:0/2233164247 >> v2:172.21.15.179:3300/0 conn(0x7f9a4c06c9b0 0x7f9a4c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:57.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.845+0000 7f9a4b400640 1 -- 172.21.15.161:0/2233164247 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283330565 0 0) 0x7f9a4c144bb0 con 0x7f9a4c143c40 2024-05-22T21:13:57.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.845+0000 7f9a4b400640 1 -- 172.21.15.161:0/2233164247 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a30003530 con 0x7f9a4c143c40 2024-05-22T21:13:57.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.845+0000 7f9a4b400640 1 -- 172.21.15.161:0/2233164247 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1145980569 0 0) 0x7f9a30003530 con 0x7f9a4c143c40 2024-05-22T21:13:57.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.845+0000 7f9a51600640 1 -- 172.21.15.161:0/2233164247 >> v1:172.21.15.179:6789/0 conn(0x7f9a4c15b600 legacy=0x7f9a4c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:57.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.845+0000 7f9a51600640 1 -- 172.21.15.161:0/2233164247 >> v1:172.21.15.161:6789/0 conn(0x7f9a4c143c40 legacy=0x7f9a4c1444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:57.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.845+0000 7f9a51600640 1 -- 172.21.15.161:0/2233164247 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c15f290 con 0x7f9a4c06c9b0 2024-05-22T21:13:57.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.849+0000 7f9a51600640 1 --2- 172.21.15.161:0/2233164247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a4c06c9b0 0x7f9a4c0b2160 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9a440097c0 tx=0x7f9a4401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc094189173a22e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:57.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.849+0000 7f9a4b400640 1 -- 172.21.15.161:0/2233164247 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c001a00 con 0x7f9a4c06c9b0 2024-05-22T21:13:57.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.849+0000 7f9a4b400640 1 -- 172.21.15.161:0/2233164247 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a440a7050 con 0x7f9a4c06c9b0 2024-05-22T21:13:57.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.849+0000 7f9a4b400640 1 -- 172.21.15.161:0/2233164247 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a440ab620 con 0x7f9a4c06c9b0 2024-05-22T21:13:57.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.849+0000 7f9a53600640 1 -- 172.21.15.161:0/2233164247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a4c06c9b0 msgr2=0x7f9a4c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:57.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.849+0000 7f9a53600640 1 --2- 172.21.15.161:0/2233164247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a4c06c9b0 0x7f9a4c0b2160 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9a440097c0 tx=0x7f9a4401c3c0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.849+0000 7f9a53600640 1 -- 172.21.15.161:0/2233164247 shutdown_connections 2024-05-22T21:13:57.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.849+0000 7f9a53600640 1 --2- 172.21.15.161:0/2233164247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a4c06c9b0 0x7f9a4c0b2160 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.849+0000 7f9a53600640 1 -- 172.21.15.161:0/2233164247 >> 172.21.15.161:0/2233164247 conn(0x7f9a4c06a2d0 msgr2=0x7f9a4c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:57.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.849+0000 7f9a53600640 1 -- 172.21.15.161:0/2233164247 shutdown_connections 2024-05-22T21:13:57.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.849+0000 7f9a53600640 1 -- 172.21.15.161:0/2233164247 wait complete. 2024-05-22T21:13:57.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.849+0000 7f9a53600640 1 Processor -- start 2024-05-22T21:13:57.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.849+0000 7f9a53600640 1 -- start start 2024-05-22T21:13:57.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a53600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a4c06c9b0 0x7f9a4c136860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:57.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a53600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a4c143c40 0x7f9a4c0aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:57.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a53600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a4c15b600 0x7f9a4c156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:57.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a53600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0af7f0 con 0x7f9a4c15b600 2024-05-22T21:13:57.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a53600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0afad0 con 0x7f9a4c143c40 2024-05-22T21:13:57.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a53600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0afdb0 con 0x7f9a4c06c9b0 2024-05-22T21:13:57.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a51600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a4c15b600 0x7f9a4c156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:57.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a51600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a4c15b600 0x7f9a4c156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59022/0 (socket says 172.21.15.161:59022) 2024-05-22T21:13:57.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a51600640 1 -- 172.21.15.161:0/3372211927 learned_addr learned my addr 172.21.15.161:0/3372211927 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:57.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a4be00640 1 --2- 172.21.15.161:0/3372211927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a4c143c40 0x7f9a4c0aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:57.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a50c00640 1 --2- 172.21.15.161:0/3372211927 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a4c06c9b0 0x7f9a4c136860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:57.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a51600640 1 -- 172.21.15.161:0/3372211927 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a4c06c9b0 msgr2=0x7f9a4c136860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:57.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a51600640 1 --2- 172.21.15.161:0/3372211927 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a4c06c9b0 0x7f9a4c136860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a51600640 1 -- 172.21.15.161:0/3372211927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a4c143c40 msgr2=0x7f9a4c0aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:57.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a51600640 1 --2- 172.21.15.161:0/3372211927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a4c143c40 0x7f9a4c0aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:57.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a51600640 1 -- 172.21.15.161:0/3372211927 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a44009420 con 0x7f9a4c15b600 2024-05-22T21:13:57.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a51600640 1 --2- 172.21.15.161:0/3372211927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a4c15b600 0x7f9a4c156c90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009710 tx=0x7f9a44019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:57.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a48c00640 1 -- 172.21.15.161:0/3372211927 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a440ba070 con 0x7f9a4c15b600 2024-05-22T21:13:57.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a48c00640 1 -- 172.21.15.161:0/3372211927 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a440abd40 con 0x7f9a4c15b600 2024-05-22T21:13:57.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a48c00640 1 -- 172.21.15.161:0/3372211927 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a440b4600 con 0x7f9a4c15b600 2024-05-22T21:13:57.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a53600640 1 -- 172.21.15.161:0/3372211927 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4c0b0090 con 0x7f9a4c15b600 2024-05-22T21:13:57.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a53600640 1 -- 172.21.15.161:0/3372211927 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4c136da0 con 0x7f9a4c15b600 2024-05-22T21:13:57.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a53600640 1 -- 172.21.15.161:0/3372211927 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4c067220 con 0x7f9a4c15b600 2024-05-22T21:13:57.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.853+0000 7f9a48c00640 1 -- 172.21.15.161:0/3372211927 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9a440b2040 con 0x7f9a4c15b600 2024-05-22T21:13:57.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.857+0000 7f9a48c00640 1 --2- 172.21.15.161:0/3372211927 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a2c066690 0x7f9a2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:57.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.857+0000 7f9a48c00640 1 -- 172.21.15.161:0/3372211927 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a4411d9b0 con 0x7f9a4c15b600 2024-05-22T21:13:57.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.857+0000 7f9a50c00640 1 --2- 172.21.15.161:0/3372211927 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a2c066690 0x7f9a2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:57.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.857+0000 7f9a50c00640 1 --2- 172.21.15.161:0/3372211927 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a2c066690 0x7f9a2c068b50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c0012f0 tx=0x7f9a3c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:57.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:57.861+0000 7f9a48c00640 1 -- 172.21.15.161:0/3372211927 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9a440eabf0 con 0x7f9a4c15b600 2024-05-22T21:13:58.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.013+0000 7f9a53600640 1 -- 172.21.15.161:0/3372211927 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a4c0b0540 con 0x7f9a4c15b600 2024-05-22T21:13:58.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.013+0000 7f9a48c00640 1 -- 172.21.15.161:0/3372211927 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a440b0070 con 0x7f9a4c15b600 2024-05-22T21:13:58.019 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:13:58.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.017+0000 7f9a53600640 1 -- 172.21.15.161:0/3372211927 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a2c066690 msgr2=0x7f9a2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.017+0000 7f9a53600640 1 --2- 172.21.15.161:0/3372211927 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a2c066690 0x7f9a2c068b50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c0012f0 tx=0x7f9a3c009040 comp rx=0 tx=0).stop 2024-05-22T21:13:58.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.017+0000 7f9a53600640 1 -- 172.21.15.161:0/3372211927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a4c15b600 msgr2=0x7f9a4c156c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.017+0000 7f9a53600640 1 --2- 172.21.15.161:0/3372211927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a4c15b600 0x7f9a4c156c90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009710 tx=0x7f9a44019160 comp rx=0 tx=0).stop 2024-05-22T21:13:58.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.017+0000 7f9a53600640 1 -- 172.21.15.161:0/3372211927 shutdown_connections 2024-05-22T21:13:58.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.017+0000 7f9a53600640 1 --2- 172.21.15.161:0/3372211927 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a2c066690 0x7f9a2c068b50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.017+0000 7f9a53600640 1 --2- 172.21.15.161:0/3372211927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a4c15b600 0x7f9a4c156c90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.017+0000 7f9a53600640 1 --2- 172.21.15.161:0/3372211927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a4c143c40 0x7f9a4c0aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.017+0000 7f9a53600640 1 --2- 172.21.15.161:0/3372211927 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a4c06c9b0 0x7f9a4c136860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.017+0000 7f9a53600640 1 -- 172.21.15.161:0/3372211927 >> 172.21.15.161:0/3372211927 conn(0x7f9a4c06a2d0 msgr2=0x7f9a4c1595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:58.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.017+0000 7f9a53600640 1 -- 172.21.15.161:0/3372211927 shutdown_connections 2024-05-22T21:13:58.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.017+0000 7f9a53600640 1 -- 172.21.15.161:0/3372211927 wait complete. 2024-05-22T21:13:58.039 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:13:58.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.173+0000 7fa1fe400640 1 Processor -- start 2024-05-22T21:13:58.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.173+0000 7fa1fe400640 1 -- start start 2024-05-22T21:13:58.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.173+0000 7fa1fe400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa1f81402e0 0x7fa1f813f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.173+0000 7fa1fe400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa1f813fc80 0x7fa1f806fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.173+0000 7fa1fe400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f8140090 con 0x7fa1f813cfc0 2024-05-22T21:13:58.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.173+0000 7fa1fe400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f813d3a0 con 0x7fa1f813fc80 2024-05-22T21:13:58.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.173+0000 7fa1fe400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f80cd920 con 0x7fa1f81402e0 2024-05-22T21:13:58.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.173+0000 7fa1f7e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa1f813fc80 0x7fa1f806fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.173+0000 7fa1f7e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa1f813fc80 0x7fa1f806fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59036/0 (socket says 172.21.15.161:59036) 2024-05-22T21:13:58.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.173+0000 7fa1f7e00640 1 -- 172.21.15.161:0/3707454567 learned_addr learned my addr 172.21.15.161:0/3707454567 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:58.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.173+0000 7fa1f6a00640 1 --2- 172.21.15.161:0/3707454567 >> v2:172.21.15.179:3300/0 conn(0x7fa1f81402e0 0x7fa1f813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.173+0000 7fa1f7e00640 1 -- 172.21.15.161:0/3707454567 >> v2:172.21.15.179:3300/0 conn(0x7fa1f81402e0 msgr2=0x7fa1f813f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:58.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.173+0000 7fa1f7e00640 1 --2- 172.21.15.161:0/3707454567 >> v2:172.21.15.179:3300/0 conn(0x7fa1f81402e0 0x7fa1f813f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.173+0000 7fa1f7e00640 1 -- 172.21.15.161:0/3707454567 >> v1:172.21.15.179:6789/0 conn(0x7fa1f813cfc0 legacy=0x7fa1f815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.173+0000 7fa1f7e00640 1 -- 172.21.15.161:0/3707454567 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f80cdc00 con 0x7fa1f813fc80 2024-05-22T21:13:58.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.177+0000 7fa1f7e00640 1 --2- 172.21.15.161:0/3707454567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f813fc80 0x7fa1f806fce0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa1f806e1d0 tx=0x7fa1f001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef53478acbb296ce server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:58.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.177+0000 7fa1f6000640 1 -- 172.21.15.161:0/3707454567 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00a9440 con 0x7fa1f813fc80 2024-05-22T21:13:58.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.177+0000 7fa1f6000640 1 -- 172.21.15.161:0/3707454567 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f00a95a0 con 0x7fa1f813fc80 2024-05-22T21:13:58.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.177+0000 7fa1f6000640 1 -- 172.21.15.161:0/3707454567 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00a9930 con 0x7fa1f813fc80 2024-05-22T21:13:58.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.177+0000 7fa1fe400640 1 -- 172.21.15.161:0/3707454567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f813fc80 msgr2=0x7fa1f806fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.177+0000 7fa1fe400640 1 --2- 172.21.15.161:0/3707454567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f813fc80 0x7fa1f806fce0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa1f806e1d0 tx=0x7fa1f001c930 comp rx=0 tx=0).stop 2024-05-22T21:13:58.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.177+0000 7fa1fe400640 1 -- 172.21.15.161:0/3707454567 shutdown_connections 2024-05-22T21:13:58.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.177+0000 7fa1fe400640 1 --2- 172.21.15.161:0/3707454567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f813fc80 0x7fa1f806fce0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.177+0000 7fa1fe400640 1 --2- 172.21.15.161:0/3707454567 >> v2:172.21.15.179:3300/0 conn(0x7fa1f81402e0 0x7fa1f813f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.177+0000 7fa1fe400640 1 -- 172.21.15.161:0/3707454567 >> 172.21.15.161:0/3707454567 conn(0x7fa1f806a2d0 msgr2=0x7fa1f806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:58.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.177+0000 7fa1fe400640 1 -- 172.21.15.161:0/3707454567 shutdown_connections 2024-05-22T21:13:58.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.177+0000 7fa1fe400640 1 -- 172.21.15.161:0/3707454567 wait complete. 2024-05-22T21:13:58.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.177+0000 7fa1fe400640 1 Processor -- start 2024-05-22T21:13:58.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1fe400640 1 -- start start 2024-05-22T21:13:58.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1fe400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1f813cfc0 0x7fa1f813c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1fe400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f813fc80 0x7fa1f815ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1fe400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1f81402e0 0x7fa1f815b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1fe400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f813f3e0 con 0x7fa1f813fc80 2024-05-22T21:13:58.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1fe400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f813e7d0 con 0x7fa1f81402e0 2024-05-22T21:13:58.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1fe400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f813eab0 con 0x7fa1f813cfc0 2024-05-22T21:13:58.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1f6a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f813fc80 0x7fa1f815ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1f6a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f813fc80 0x7fa1f815ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59052/0 (socket says 172.21.15.161:59052) 2024-05-22T21:13:58.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1f6a00640 1 -- 172.21.15.161:0/1238324609 learned_addr learned my addr 172.21.15.161:0/1238324609 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:58.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1f7400640 1 --2- 172.21.15.161:0/1238324609 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1f813cfc0 0x7fa1f813c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1f7e00640 1 --2- 172.21.15.161:0/1238324609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1f81402e0 0x7fa1f815b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1f6a00640 1 -- 172.21.15.161:0/1238324609 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1f813cfc0 msgr2=0x7fa1f813c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1f6a00640 1 --2- 172.21.15.161:0/1238324609 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1f813cfc0 0x7fa1f813c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1f6a00640 1 -- 172.21.15.161:0/1238324609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1f81402e0 msgr2=0x7fa1f815b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1f6a00640 1 --2- 172.21.15.161:0/1238324609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1f81402e0 0x7fa1f815b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1f6a00640 1 -- 172.21.15.161:0/1238324609 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f0009b30 con 0x7fa1f813fc80 2024-05-22T21:13:58.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1f7e00640 1 --2- 172.21.15.161:0/1238324609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1f81402e0 0x7fa1f815b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:58.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1f6a00640 1 --2- 172.21.15.161:0/1238324609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f813fc80 0x7fa1f815ada0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400e9b0 tx=0x7fa1e400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:58.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1ef400640 1 -- 172.21.15.161:0/1238324609 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40096b0 con 0x7fa1f813fc80 2024-05-22T21:13:58.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1ef400640 1 -- 172.21.15.161:0/1238324609 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e400c9a0 con 0x7fa1f813fc80 2024-05-22T21:13:58.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1fe400640 1 -- 172.21.15.161:0/1238324609 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f813ed90 con 0x7fa1f813fc80 2024-05-22T21:13:58.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1ef400640 1 -- 172.21.15.161:0/1238324609 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e400cd30 con 0x7fa1f813fc80 2024-05-22T21:13:58.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1fe400640 1 -- 172.21.15.161:0/1238324609 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f813f870 con 0x7fa1f813fc80 2024-05-22T21:13:58.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.181+0000 7fa1fe400640 1 -- 172.21.15.161:0/1238324609 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f8067220 con 0x7fa1f813fc80 2024-05-22T21:13:58.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.185+0000 7fa1ef400640 1 -- 172.21.15.161:0/1238324609 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa1e4013470 con 0x7fa1f813fc80 2024-05-22T21:13:58.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.185+0000 7fa1ef400640 1 --2- 172.21.15.161:0/1238324609 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1d8066690 0x7fa1d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.185+0000 7fa1ef400640 1 -- 172.21.15.161:0/1238324609 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1e40843f0 con 0x7fa1f813fc80 2024-05-22T21:13:58.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.185+0000 7fa1f7400640 1 --2- 172.21.15.161:0/1238324609 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1d8066690 0x7fa1d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.185+0000 7fa1f7400640 1 --2- 172.21.15.161:0/1238324609 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1d8066690 0x7fa1d8068b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8007fd0 tx=0x7fa1e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:58.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.189+0000 7fa1ef400640 1 -- 172.21.15.161:0/1238324609 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa1e4051770 con 0x7fa1f813fc80 2024-05-22T21:13:58.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.345+0000 7fa1fe400640 1 -- 172.21.15.161:0/1238324609 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1f8071d00 con 0x7fa1f813fc80 2024-05-22T21:13:58.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.345+0000 7fa1ef400640 1 -- 172.21.15.161:0/1238324609 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa1e4054d90 con 0x7fa1f813fc80 2024-05-22T21:13:58.350 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:13:58.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.349+0000 7fa1fe400640 1 -- 172.21.15.161:0/1238324609 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1d8066690 msgr2=0x7fa1d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.349+0000 7fa1fe400640 1 --2- 172.21.15.161:0/1238324609 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1d8066690 0x7fa1d8068b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8007fd0 tx=0x7fa1e8009040 comp rx=0 tx=0).stop 2024-05-22T21:13:58.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.349+0000 7fa1fe400640 1 -- 172.21.15.161:0/1238324609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f813fc80 msgr2=0x7fa1f815ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.349+0000 7fa1fe400640 1 --2- 172.21.15.161:0/1238324609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f813fc80 0x7fa1f815ada0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400e9b0 tx=0x7fa1e400ee70 comp rx=0 tx=0).stop 2024-05-22T21:13:58.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.349+0000 7fa1fe400640 1 -- 172.21.15.161:0/1238324609 shutdown_connections 2024-05-22T21:13:58.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.349+0000 7fa1fe400640 1 --2- 172.21.15.161:0/1238324609 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1d8066690 0x7fa1d8068b50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.349+0000 7fa1fe400640 1 --2- 172.21.15.161:0/1238324609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1f81402e0 0x7fa1f815b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.349+0000 7fa1fe400640 1 --2- 172.21.15.161:0/1238324609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f813fc80 0x7fa1f815ada0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.349+0000 7fa1fe400640 1 --2- 172.21.15.161:0/1238324609 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1f813cfc0 0x7fa1f813c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.349+0000 7fa1fe400640 1 -- 172.21.15.161:0/1238324609 >> 172.21.15.161:0/1238324609 conn(0x7fa1f806a2d0 msgr2=0x7fa1f806a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:58.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.349+0000 7fa1fe400640 1 -- 172.21.15.161:0/1238324609 shutdown_connections 2024-05-22T21:13:58.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.349+0000 7fa1fe400640 1 -- 172.21.15.161:0/1238324609 wait complete. 2024-05-22T21:13:58.372 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:13:58.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.505+0000 7fc573c00640 1 Processor -- start 2024-05-22T21:13:58.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.505+0000 7fc573c00640 1 -- start start 2024-05-22T21:13:58.513 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.509+0000 7fc573c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc56c1596b0 0x7fc56c071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.513 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.509+0000 7fc573c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc56c071db0 0x7fc56c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.513 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.509+0000 7fc573c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56c0b3110 con 0x7fc56c06d070 2024-05-22T21:13:58.513 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.509+0000 7fc573c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56c06d4e0 con 0x7fc56c1596b0 2024-05-22T21:13:58.513 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.509+0000 7fc573c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c15d250 con 0x7fc56c071db0 2024-05-22T21:13:58.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.509+0000 7fc571200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc56c1596b0 0x7fc56c071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.509+0000 7fc571200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc56c1596b0 0x7fc56c071870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59066/0 (socket says 172.21.15.161:59066) 2024-05-22T21:13:58.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.509+0000 7fc571200640 1 -- 172.21.15.161:0/102779423 learned_addr learned my addr 172.21.15.161:0/102779423 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:58.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.509+0000 7fc56be00640 1 --2- 172.21.15.161:0/102779423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc56c071db0 0x7fc56c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.509+0000 7fc571200640 1 -- 172.21.15.161:0/102779423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc56c071db0 msgr2=0x7fc56c06cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:58.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.509+0000 7fc571200640 1 --2- 172.21.15.161:0/102779423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc56c071db0 0x7fc56c06cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.509+0000 7fc571200640 1 -- 172.21.15.161:0/102779423 >> v1:172.21.15.179:6789/0 conn(0x7fc56c06d070 legacy=0x7fc56c06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:58.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.509+0000 7fc571200640 1 -- 172.21.15.161:0/102779423 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c159a90 con 0x7fc56c1596b0 2024-05-22T21:13:58.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc571200640 1 --2- 172.21.15.161:0/102779423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c1596b0 0x7fc56c071870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc55c009e50 tx=0x7fc55c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857b7b26c5b8a117 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:58.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc56b400640 1 -- 172.21.15.161:0/102779423 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c0a5790 con 0x7fc56c1596b0 2024-05-22T21:13:58.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc56b400640 1 -- 172.21.15.161:0/102779423 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55c0a58f0 con 0x7fc56c1596b0 2024-05-22T21:13:58.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc56b400640 1 -- 172.21.15.161:0/102779423 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c0a5c80 con 0x7fc56c1596b0 2024-05-22T21:13:58.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc573c00640 1 -- 172.21.15.161:0/102779423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c1596b0 msgr2=0x7fc56c071870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc573c00640 1 --2- 172.21.15.161:0/102779423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c1596b0 0x7fc56c071870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc55c009e50 tx=0x7fc55c01ca30 comp rx=0 tx=0).stop 2024-05-22T21:13:58.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc573c00640 1 -- 172.21.15.161:0/102779423 shutdown_connections 2024-05-22T21:13:58.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc573c00640 1 --2- 172.21.15.161:0/102779423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc56c071db0 0x7fc56c06cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc573c00640 1 --2- 172.21.15.161:0/102779423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c1596b0 0x7fc56c071870 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc573c00640 1 -- 172.21.15.161:0/102779423 >> 172.21.15.161:0/102779423 conn(0x7fc56c06a2d0 msgr2=0x7fc56c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:58.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc573c00640 1 -- 172.21.15.161:0/102779423 shutdown_connections 2024-05-22T21:13:58.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc573c00640 1 -- 172.21.15.161:0/102779423 wait complete. 2024-05-22T21:13:58.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc573c00640 1 Processor -- start 2024-05-22T21:13:58.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc573c00640 1 -- start start 2024-05-22T21:13:58.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc573c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c06d070 0x7fc56c156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc573c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc56c071db0 0x7fc56c150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc573c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc56c1596b0 0x7fc56c158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc573c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c1570a0 con 0x7fc56c06d070 2024-05-22T21:13:58.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc573c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c157ee0 con 0x7fc56c1596b0 2024-05-22T21:13:58.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.513+0000 7fc573c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c150a60 con 0x7fc56c071db0 2024-05-22T21:13:58.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc571200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c06d070 0x7fc56c156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc571200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c06d070 0x7fc56c156b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59076/0 (socket says 172.21.15.161:59076) 2024-05-22T21:13:58.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc571200640 1 -- 172.21.15.161:0/2642450099 learned_addr learned my addr 172.21.15.161:0/2642450099 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:58.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc56be00640 1 --2- 172.21.15.161:0/2642450099 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc56c071db0 0x7fc56c150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc571c00640 1 --2- 172.21.15.161:0/2642450099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc56c1596b0 0x7fc56c158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc571200640 1 -- 172.21.15.161:0/2642450099 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc56c071db0 msgr2=0x7fc56c150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc571200640 1 --2- 172.21.15.161:0/2642450099 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc56c071db0 0x7fc56c150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc571200640 1 -- 172.21.15.161:0/2642450099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc56c1596b0 msgr2=0x7fc56c158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc571200640 1 --2- 172.21.15.161:0/2642450099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc56c1596b0 0x7fc56c158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc571200640 1 -- 172.21.15.161:0/2642450099 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c009b30 con 0x7fc56c06d070 2024-05-22T21:13:58.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc571200640 1 --2- 172.21.15.161:0/2642450099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c06d070 0x7fc56c156b60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc55c01a660 tx=0x7fc55c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:58.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc568c00640 1 -- 172.21.15.161:0/2642450099 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c0b8670 con 0x7fc56c06d070 2024-05-22T21:13:58.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc568c00640 1 -- 172.21.15.161:0/2642450099 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55c0b87d0 con 0x7fc56c06d070 2024-05-22T21:13:58.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc573c00640 1 -- 172.21.15.161:0/2642450099 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56c157330 con 0x7fc56c06d070 2024-05-22T21:13:58.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc568c00640 1 -- 172.21.15.161:0/2642450099 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c0b8b80 con 0x7fc56c06d070 2024-05-22T21:13:58.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc573c00640 1 -- 172.21.15.161:0/2642450099 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56c157b70 con 0x7fc56c06d070 2024-05-22T21:13:58.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc573c00640 1 -- 172.21.15.161:0/2642450099 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56c1576d0 con 0x7fc56c06d070 2024-05-22T21:13:58.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc568c00640 1 -- 172.21.15.161:0/2642450099 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc55c0b8ce0 con 0x7fc56c06d070 2024-05-22T21:13:58.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc568c00640 1 --2- 172.21.15.161:0/2642450099 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc54c066690 0x7fc54c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.517+0000 7fc568c00640 1 -- 172.21.15.161:0/2642450099 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc55c11f5f0 con 0x7fc56c06d070 2024-05-22T21:13:58.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.521+0000 7fc56be00640 1 --2- 172.21.15.161:0/2642450099 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc54c066690 0x7fc54c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.521+0000 7fc56be00640 1 --2- 172.21.15.161:0/2642450099 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc54c066690 0x7fc54c068b50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc56c14a000 tx=0x7fc558008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:58.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.521+0000 7fc568c00640 1 -- 172.21.15.161:0/2642450099 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc55c0ec970 con 0x7fc56c06d070 2024-05-22T21:13:58.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.681+0000 7fc573c00640 1 -- 172.21.15.161:0/2642450099 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc56c067220 con 0x7fc56c06d070 2024-05-22T21:13:58.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.681+0000 7fc568c00640 1 -- 172.21.15.161:0/2642450099 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc55c0eff90 con 0x7fc56c06d070 2024-05-22T21:13:58.685 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:13:58.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.681+0000 7fc573c00640 1 -- 172.21.15.161:0/2642450099 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc54c066690 msgr2=0x7fc54c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.681+0000 7fc573c00640 1 --2- 172.21.15.161:0/2642450099 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc54c066690 0x7fc54c068b50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc56c14a000 tx=0x7fc558008040 comp rx=0 tx=0).stop 2024-05-22T21:13:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.681+0000 7fc573c00640 1 -- 172.21.15.161:0/2642450099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c06d070 msgr2=0x7fc56c156b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.681+0000 7fc573c00640 1 --2- 172.21.15.161:0/2642450099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c06d070 0x7fc56c156b60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc55c01a660 tx=0x7fc55c0197d0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.681+0000 7fc573c00640 1 -- 172.21.15.161:0/2642450099 shutdown_connections 2024-05-22T21:13:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.681+0000 7fc573c00640 1 --2- 172.21.15.161:0/2642450099 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc54c066690 0x7fc54c068b50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.681+0000 7fc573c00640 1 --2- 172.21.15.161:0/2642450099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc56c1596b0 0x7fc56c158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.681+0000 7fc573c00640 1 --2- 172.21.15.161:0/2642450099 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc56c071db0 0x7fc56c150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.681+0000 7fc573c00640 1 --2- 172.21.15.161:0/2642450099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c06d070 0x7fc56c156b60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.681+0000 7fc573c00640 1 -- 172.21.15.161:0/2642450099 >> 172.21.15.161:0/2642450099 conn(0x7fc56c06a2d0 msgr2=0x7fc56c14a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.681+0000 7fc573c00640 1 -- 172.21.15.161:0/2642450099 shutdown_connections 2024-05-22T21:13:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.681+0000 7fc573c00640 1 -- 172.21.15.161:0/2642450099 wait complete. 2024-05-22T21:13:58.705 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:13:58.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f5955200640 1 Processor -- start 2024-05-22T21:13:58.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f5955200640 1 -- start start 2024-05-22T21:13:58.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f5955200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5950143ed0 0x7f59501442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f5955200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59500ce830 con 0x7f59501448e0 2024-05-22T21:13:58.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f5955200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59500b2e60 con 0x7f59500ce370 2024-05-22T21:13:58.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f5955200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59500b3140 con 0x7f5950143ed0 2024-05-22T21:13:58.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594ea00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f59501448e0 0x7f59500b20e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33698/0 (socket says 172.21.15.161:33698) 2024-05-22T21:13:58.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594ea00640 1 -- 172.21.15.161:0/584778423 learned_addr learned my addr 172.21.15.161:0/584778423 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:58.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594f400640 1 --2- 172.21.15.161:0/584778423 >> v2:172.21.15.179:3300/0 conn(0x7f5950143ed0 0x7f59501442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594e000640 1 -- 172.21.15.161:0/584778423 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775338037 0 0) 0x7f59500ce830 con 0x7f59501448e0 2024-05-22T21:13:58.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594e000640 1 -- 172.21.15.161:0/584778423 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5934003530 con 0x7f59501448e0 2024-05-22T21:13:58.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594e000640 1 -- 172.21.15.161:0/584778423 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 219659193 0 0) 0x7f5934003530 con 0x7f59501448e0 2024-05-22T21:13:58.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594e000640 1 -- 172.21.15.161:0/584778423 >> v2:172.21.15.179:3300/0 conn(0x7f5950143ed0 msgr2=0x7f59501442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:58.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594e000640 1 --2- 172.21.15.161:0/584778423 >> v2:172.21.15.179:3300/0 conn(0x7f5950143ed0 0x7f59501442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594e000640 1 -- 172.21.15.161:0/584778423 >> v1:172.21.15.179:6789/0 conn(0x7f59500ce370 legacy=0x7f595006bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594e000640 1 -- 172.21.15.161:0/584778423 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59500b3420 con 0x7f59501448e0 2024-05-22T21:13:58.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594e000640 1 -- 172.21.15.161:0/584778423 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f593c001680 con 0x7f59501448e0 2024-05-22T21:13:58.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594f400640 1 --2- 172.21.15.161:0/584778423 >> v2:172.21.15.179:3300/0 conn(0x7f5950143ed0 0x7f59501442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:13:58.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594e000640 1 -- 172.21.15.161:0/584778423 >> v1:172.21.15.161:6789/0 conn(0x7f59501448e0 legacy=0x7f59500b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594e000640 1 --2- 172.21.15.161:0/584778423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5934003980 0x7f5934003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594e000640 1 --2- 172.21.15.161:0/584778423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59340047d0 0x7f5934004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594e000640 1 --2- 172.21.15.161:0/584778423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5934005190 0x7f593400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594e000640 1 -- 172.21.15.161:0/584778423 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59500b3140 con 0x7f59340047d0 2024-05-22T21:13:58.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594e000640 1 -- 172.21.15.161:0/584778423 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593400c130 con 0x7f5934005190 2024-05-22T21:13:58.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594e000640 1 -- 172.21.15.161:0/584778423 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593400c2f0 con 0x7f5934003980 2024-05-22T21:13:58.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594ea00640 1 --2- 172.21.15.161:0/584778423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59340047d0 0x7f5934004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594f400640 1 --2- 172.21.15.161:0/584778423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5934003980 0x7f5934003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.837+0000 7f594fe00640 1 --2- 172.21.15.161:0/584778423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5934005190 0x7f593400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f594ea00640 1 -- 172.21.15.161:0/584778423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5934003980 msgr2=0x7f5934003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:58.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f594ea00640 1 --2- 172.21.15.161:0/584778423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5934003980 0x7f5934003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f594ea00640 1 -- 172.21.15.161:0/584778423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5934005190 msgr2=0x7f593400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:58.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f594ea00640 1 --2- 172.21.15.161:0/584778423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5934005190 0x7f593400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f594ea00640 1 -- 172.21.15.161:0/584778423 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5934003530 con 0x7f59340047d0 2024-05-22T21:13:58.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f594fe00640 1 --2- 172.21.15.161:0/584778423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5934005190 0x7f593400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:58.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f594fe00640 1 -- 172.21.15.161:0/584778423 reap_dead start 2024-05-22T21:13:58.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f594ea00640 1 --2- 172.21.15.161:0/584778423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59340047d0 0x7f5934004c50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f593c00c240 tx=0x7f593c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22ec6a3512e8e42 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:58.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f594e000640 1 -- 172.21.15.161:0/584778423 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c0a5b60 con 0x7f59340047d0 2024-05-22T21:13:58.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f594e000640 1 -- 172.21.15.161:0/584778423 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c0a5cc0 con 0x7f59340047d0 2024-05-22T21:13:58.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f594e000640 1 -- 172.21.15.161:0/584778423 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c005b20 con 0x7f59340047d0 2024-05-22T21:13:58.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f5955200640 1 -- 172.21.15.161:0/584778423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59340047d0 msgr2=0x7f5934004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f5955200640 1 --2- 172.21.15.161:0/584778423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59340047d0 0x7f5934004c50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f593c00c240 tx=0x7f593c01cc60 comp rx=0 tx=0).stop 2024-05-22T21:13:58.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f5955200640 1 -- 172.21.15.161:0/584778423 shutdown_connections 2024-05-22T21:13:58.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f5955200640 1 --2- 172.21.15.161:0/584778423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59340047d0 0x7f5934004c50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f5955200640 1 -- 172.21.15.161:0/584778423 >> 172.21.15.161:0/584778423 conn(0x7f59500cec00 msgr2=0x7f59500cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:58.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f5955200640 1 -- 172.21.15.161:0/584778423 shutdown_connections 2024-05-22T21:13:58.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.841+0000 7f5955200640 1 -- 172.21.15.161:0/584778423 wait complete. 2024-05-22T21:13:58.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f5955200640 1 Processor -- start 2024-05-22T21:13:58.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f5955200640 1 -- start start 2024-05-22T21:13:58.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f5955200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59340047d0 0x7f59501f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f5955200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59501f9c80 0x7f5950070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f5955200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59500710c0 0x7f595015c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f5955200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59500706f0 con 0x7f59340047d0 2024-05-22T21:13:58.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f5955200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595006fb20 con 0x7f59500710c0 2024-05-22T21:13:58.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f5955200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595006fdc0 con 0x7f59501f9c80 2024-05-22T21:13:58.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f594f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59340047d0 0x7f59501f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f594f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59340047d0 0x7f59501f8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59094/0 (socket says 172.21.15.161:59094) 2024-05-22T21:13:58.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f594f400640 1 -- 172.21.15.161:0/4081827922 learned_addr learned my addr 172.21.15.161:0/4081827922 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:58.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f594fe00640 1 --2- 172.21.15.161:0/4081827922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59500710c0 0x7f595015c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f594ea00640 1 --2- 172.21.15.161:0/4081827922 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59501f9c80 0x7f5950070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f594f400640 1 -- 172.21.15.161:0/4081827922 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59501f9c80 msgr2=0x7f5950070b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f594f400640 1 --2- 172.21.15.161:0/4081827922 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59501f9c80 0x7f5950070b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f594f400640 1 -- 172.21.15.161:0/4081827922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59500710c0 msgr2=0x7f595015c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:58.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f594f400640 1 --2- 172.21.15.161:0/4081827922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59500710c0 0x7f595015c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:58.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f594f400640 1 -- 172.21.15.161:0/4081827922 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59500700a0 con 0x7f59340047d0 2024-05-22T21:13:58.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f594f400640 1 --2- 172.21.15.161:0/4081827922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59340047d0 0x7f59501f8710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f594000f430 tx=0x7f594000f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:58.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f5947400640 1 -- 172.21.15.161:0/4081827922 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594000e040 con 0x7f59340047d0 2024-05-22T21:13:58.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f5947400640 1 -- 172.21.15.161:0/4081827922 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594000f040 con 0x7f59340047d0 2024-05-22T21:13:58.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f5947400640 1 -- 172.21.15.161:0/4081827922 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940003e00 con 0x7f59340047d0 2024-05-22T21:13:58.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f5955200640 1 -- 172.21.15.161:0/4081827922 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5950070380 con 0x7f59340047d0 2024-05-22T21:13:58.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.845+0000 7f5955200640 1 -- 172.21.15.161:0/4081827922 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595015ca60 con 0x7f59340047d0 2024-05-22T21:13:58.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.849+0000 7f5955200640 1 -- 172.21.15.161:0/4081827922 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5950067070 con 0x7f59340047d0 2024-05-22T21:13:58.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.849+0000 7f5947400640 1 -- 172.21.15.161:0/4081827922 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5940004030 con 0x7f59340047d0 2024-05-22T21:13:58.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.849+0000 7f5947400640 1 --2- 172.21.15.161:0/4081827922 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5930066690 0x7f5930068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:58.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.849+0000 7f5947400640 1 -- 172.21.15.161:0/4081827922 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5940084e90 con 0x7f59340047d0 2024-05-22T21:13:58.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.849+0000 7f594ea00640 1 --2- 172.21.15.161:0/4081827922 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5930066690 0x7f5930068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:58.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.849+0000 7f594ea00640 1 --2- 172.21.15.161:0/4081827922 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5930066690 0x7f5930068b50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f593c0ab000 tx=0x7f593c0b0040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:58.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:58.853+0000 7f5947400640 1 -- 172.21.15.161:0/4081827922 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5940052210 con 0x7f59340047d0 2024-05-22T21:13:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.017+0000 7f5955200640 1 -- 172.21.15.161:0/4081827922 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5950140990 con 0x7f59340047d0 2024-05-22T21:13:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.017+0000 7f5947400640 1 -- 172.21.15.161:0/4081827922 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5940055830 con 0x7f59340047d0 2024-05-22T21:13:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:13:59.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.017+0000 7f5955200640 1 -- 172.21.15.161:0/4081827922 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5930066690 msgr2=0x7f5930068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:59.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.017+0000 7f5955200640 1 --2- 172.21.15.161:0/4081827922 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5930066690 0x7f5930068b50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f593c0ab000 tx=0x7f593c0b0040 comp rx=0 tx=0).stop 2024-05-22T21:13:59.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.017+0000 7f5955200640 1 -- 172.21.15.161:0/4081827922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59340047d0 msgr2=0x7f59501f8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:59.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.017+0000 7f5955200640 1 --2- 172.21.15.161:0/4081827922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59340047d0 0x7f59501f8710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f594000f430 tx=0x7f594000f830 comp rx=0 tx=0).stop 2024-05-22T21:13:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.017+0000 7f5955200640 1 -- 172.21.15.161:0/4081827922 shutdown_connections 2024-05-22T21:13:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.017+0000 7f5955200640 1 --2- 172.21.15.161:0/4081827922 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5930066690 0x7f5930068b50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.017+0000 7f5955200640 1 --2- 172.21.15.161:0/4081827922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59500710c0 0x7f595015c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.017+0000 7f5955200640 1 --2- 172.21.15.161:0/4081827922 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59501f9c80 0x7f5950070b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.017+0000 7f5955200640 1 --2- 172.21.15.161:0/4081827922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59340047d0 0x7f59501f8710 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.017+0000 7f5955200640 1 -- 172.21.15.161:0/4081827922 >> 172.21.15.161:0/4081827922 conn(0x7f59500cec00 msgr2=0x7f5950142ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.017+0000 7f5955200640 1 -- 172.21.15.161:0/4081827922 shutdown_connections 2024-05-22T21:13:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.017+0000 7f5955200640 1 -- 172.21.15.161:0/4081827922 wait complete. 2024-05-22T21:13:59.041 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-22T21:13:59.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:59.042 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-05-22T21:13:59.084 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:13:59.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:13:59.085 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-22T21:13:59.142 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:59.142 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:13:59.152 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:59.152 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:13:59.206 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:13:59.206 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:13:59.258 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:13:59.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f12e00640 1 Processor -- start 2024-05-22T21:13:59.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f12e00640 1 -- start start 2024-05-22T21:13:59.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f12e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7f0c0b0050 0x7f7f0c06b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f12e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f0c09a560 con 0x7f7f0c0b08e0 2024-05-22T21:13:59.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f12e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f0c09a840 con 0x7f7f0c15d710 2024-05-22T21:13:59.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f12e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f0c0cfc40 con 0x7f7f0c0b0050 2024-05-22T21:13:59.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f10e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f7f0c0b08e0 0x7f7f0c0b0d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33708/0 (socket says 172.21.15.161:33708) 2024-05-22T21:13:59.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f10e00640 1 -- 172.21.15.161:0/972313230 learned_addr learned my addr 172.21.15.161:0/972313230 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:59.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0b400640 1 --2- 172.21.15.161:0/972313230 >> v2:172.21.15.179:3300/0 conn(0x7f7f0c0b0050 0x7f7f0c06b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0aa00640 1 -- 172.21.15.161:0/972313230 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901001440 0 0) 0x7f7f0c09a560 con 0x7f7f0c0b08e0 2024-05-22T21:13:59.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0aa00640 1 -- 172.21.15.161:0/972313230 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ef0003530 con 0x7f7f0c0b08e0 2024-05-22T21:13:59.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0aa00640 1 -- 172.21.15.161:0/972313230 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2643591402 0 0) 0x7f7ef0003530 con 0x7f7f0c0b08e0 2024-05-22T21:13:59.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0aa00640 1 -- 172.21.15.161:0/972313230 >> v2:172.21.15.179:3300/0 conn(0x7f7f0c0b0050 msgr2=0x7f7f0c06b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:59.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0aa00640 1 --2- 172.21.15.161:0/972313230 >> v2:172.21.15.179:3300/0 conn(0x7f7f0c0b0050 0x7f7f0c06b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0aa00640 1 -- 172.21.15.161:0/972313230 >> v1:172.21.15.179:6789/0 conn(0x7f7f0c15d710 legacy=0x7f7f0c06afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:59.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0aa00640 1 -- 172.21.15.161:0/972313230 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c0cff20 con 0x7f7f0c0b08e0 2024-05-22T21:13:59.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0aa00640 1 -- 172.21.15.161:0/972313230 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f7f04001680 con 0x7f7f0c0b08e0 2024-05-22T21:13:59.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0aa00640 1 -- 172.21.15.161:0/972313230 >> v1:172.21.15.161:6789/0 conn(0x7f7f0c0b08e0 legacy=0x7f7f0c0b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:59.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0aa00640 1 --2- 172.21.15.161:0/972313230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef0003980 0x7f7ef0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0aa00640 1 --2- 172.21.15.161:0/972313230 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ef00047d0 0x7f7ef0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0aa00640 1 --2- 172.21.15.161:0/972313230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ef0005190 0x7f7ef000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0aa00640 1 -- 172.21.15.161:0/972313230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c0cfc40 con 0x7f7ef0003980 2024-05-22T21:13:59.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0aa00640 1 -- 172.21.15.161:0/972313230 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef000c030 con 0x7f7ef0005190 2024-05-22T21:13:59.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0aa00640 1 -- 172.21.15.161:0/972313230 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef000c1f0 con 0x7f7ef00047d0 2024-05-22T21:13:59.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0be00640 1 --2- 172.21.15.161:0/972313230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef0003980 0x7f7ef0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f0b400640 1 --2- 172.21.15.161:0/972313230 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ef00047d0 0x7f7ef0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.389+0000 7f7f10e00640 1 --2- 172.21.15.161:0/972313230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ef0005190 0x7f7ef000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f0be00640 1 -- 172.21.15.161:0/972313230 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ef00047d0 msgr2=0x7f7ef0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:59.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f0be00640 1 --2- 172.21.15.161:0/972313230 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ef00047d0 0x7f7ef0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f0be00640 1 -- 172.21.15.161:0/972313230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ef0005190 msgr2=0x7f7ef000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:59.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f0be00640 1 --2- 172.21.15.161:0/972313230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ef0005190 0x7f7ef000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f0be00640 1 -- 172.21.15.161:0/972313230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef0003530 con 0x7f7ef0003980 2024-05-22T21:13:59.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f10e00640 1 --2- 172.21.15.161:0/972313230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ef0005190 0x7f7ef000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:59.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f10e00640 1 -- 172.21.15.161:0/972313230 reap_dead start 2024-05-22T21:13:59.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f0be00640 1 --2- 172.21.15.161:0/972313230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef0003980 0x7f7ef0003da0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7efc001e70 tx=0x7f7efc01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13490eab75a05582 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:59.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f0aa00640 1 -- 172.21.15.161:0/972313230 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc003b80 con 0x7f7ef0003980 2024-05-22T21:13:59.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f0aa00640 1 -- 172.21.15.161:0/972313230 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc003ce0 con 0x7f7ef0003980 2024-05-22T21:13:59.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f0aa00640 1 -- 172.21.15.161:0/972313230 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc0052c0 con 0x7f7ef0003980 2024-05-22T21:13:59.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f12e00640 1 -- 172.21.15.161:0/972313230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef0003980 msgr2=0x7f7ef0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:59.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f12e00640 1 --2- 172.21.15.161:0/972313230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef0003980 0x7f7ef0003da0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7efc001e70 tx=0x7f7efc01c760 comp rx=0 tx=0).stop 2024-05-22T21:13:59.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f12e00640 1 -- 172.21.15.161:0/972313230 shutdown_connections 2024-05-22T21:13:59.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f12e00640 1 --2- 172.21.15.161:0/972313230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef0003980 0x7f7ef0003da0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f12e00640 1 -- 172.21.15.161:0/972313230 >> 172.21.15.161:0/972313230 conn(0x7f7f0c099a40 msgr2=0x7f7f0c099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:59.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f12e00640 1 -- 172.21.15.161:0/972313230 shutdown_connections 2024-05-22T21:13:59.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.393+0000 7f7f12e00640 1 -- 172.21.15.161:0/972313230 wait complete. 2024-05-22T21:13:59.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f12e00640 1 Processor -- start 2024-05-22T21:13:59.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f12e00640 1 -- start start 2024-05-22T21:13:59.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f12e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ef0003980 0x7f7f0c0c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f12e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f0c0af090 0x7f7f0c1f9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f12e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7f0c0af510 0x7f7f0c1f9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f12e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c1fa130 con 0x7f7f0c0af090 2024-05-22T21:13:59.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f12e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c1f9e50 con 0x7f7ef0003980 2024-05-22T21:13:59.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f12e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c0c8970 con 0x7f7f0c0af510 2024-05-22T21:13:59.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f0b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f0c0af090 0x7f7f0c1f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f0b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f0c0af090 0x7f7f0c1f9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59116/0 (socket says 172.21.15.161:59116) 2024-05-22T21:13:59.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f0b400640 1 -- 172.21.15.161:0/1970171951 learned_addr learned my addr 172.21.15.161:0/1970171951 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:59.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f10e00640 1 --2- 172.21.15.161:0/1970171951 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7f0c0af510 0x7f7f0c1f9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f0be00640 1 --2- 172.21.15.161:0/1970171951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ef0003980 0x7f7f0c0c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f0b400640 1 -- 172.21.15.161:0/1970171951 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7f0c0af510 msgr2=0x7f7f0c1f9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:59.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f0b400640 1 --2- 172.21.15.161:0/1970171951 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7f0c0af510 0x7f7f0c1f9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f0b400640 1 -- 172.21.15.161:0/1970171951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ef0003980 msgr2=0x7f7f0c0c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:59.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f0b400640 1 --2- 172.21.15.161:0/1970171951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ef0003980 0x7f7f0c0c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f0b400640 1 -- 172.21.15.161:0/1970171951 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c0c8c50 con 0x7f7f0c0af090 2024-05-22T21:13:59.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f0b400640 1 --2- 172.21.15.161:0/1970171951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f0c0af090 0x7f7f0c1f9200 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8011660 tx=0x7f7ef8011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:59.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f03e00640 1 -- 172.21.15.161:0/1970171951 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef800f9c0 con 0x7f7f0c0af090 2024-05-22T21:13:59.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f03e00640 1 -- 172.21.15.161:0/1970171951 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef800fb20 con 0x7f7f0c0af090 2024-05-22T21:13:59.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f03e00640 1 -- 172.21.15.161:0/1970171951 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef800fe90 con 0x7f7f0c0af090 2024-05-22T21:13:59.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f12e00640 1 -- 172.21.15.161:0/1970171951 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0c0c8f30 con 0x7f7f0c0af090 2024-05-22T21:13:59.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.397+0000 7f7f12e00640 1 -- 172.21.15.161:0/1970171951 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0c1f6030 con 0x7f7f0c0af090 2024-05-22T21:13:59.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.401+0000 7f7f12e00640 1 -- 172.21.15.161:0/1970171951 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0c0c87d0 con 0x7f7f0c0af090 2024-05-22T21:13:59.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.401+0000 7f7f03e00640 1 -- 172.21.15.161:0/1970171951 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7ef8015460 con 0x7f7f0c0af090 2024-05-22T21:13:59.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.401+0000 7f7f03e00640 1 --2- 172.21.15.161:0/1970171951 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7eec0666c0 0x7f7eec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.401+0000 7f7f03e00640 1 -- 172.21.15.161:0/1970171951 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ef80846e0 con 0x7f7f0c0af090 2024-05-22T21:13:59.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.401+0000 7f7f0be00640 1 --2- 172.21.15.161:0/1970171951 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7eec0666c0 0x7f7eec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.401+0000 7f7f0be00640 1 --2- 172.21.15.161:0/1970171951 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7eec0666c0 0x7f7eec068b80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7efc005000 tx=0x7f7efc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:59.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.405+0000 7f7f03e00640 1 -- 172.21.15.161:0/1970171951 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7f0c0c87d0 con 0x7f7f0c0af090 2024-05-22T21:13:59.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.561+0000 7f7f12e00640 1 -- 172.21.15.161:0/1970171951 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f0c0c87d0 con 0x7f7f0c0af090 2024-05-22T21:13:59.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.565+0000 7f7f03e00640 1 -- 172.21.15.161:0/1970171951 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f7f0c0c87d0 con 0x7f7f0c0af090 2024-05-22T21:13:59.568 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:13:59.568 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:13:59.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.565+0000 7f7f12e00640 1 -- 172.21.15.161:0/1970171951 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7eec0666c0 msgr2=0x7f7eec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:59.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.565+0000 7f7f12e00640 1 --2- 172.21.15.161:0/1970171951 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7eec0666c0 0x7f7eec068b80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7efc005000 tx=0x7f7efc009040 comp rx=0 tx=0).stop 2024-05-22T21:13:59.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.565+0000 7f7f12e00640 1 -- 172.21.15.161:0/1970171951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f0c0af090 msgr2=0x7f7f0c1f9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:59.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.565+0000 7f7f12e00640 1 --2- 172.21.15.161:0/1970171951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f0c0af090 0x7f7f0c1f9200 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8011660 tx=0x7f7ef8011b20 comp rx=0 tx=0).stop 2024-05-22T21:13:59.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.565+0000 7f7f12e00640 1 -- 172.21.15.161:0/1970171951 shutdown_connections 2024-05-22T21:13:59.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.565+0000 7f7f12e00640 1 --2- 172.21.15.161:0/1970171951 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7eec0666c0 0x7f7eec068b80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.565+0000 7f7f12e00640 1 --2- 172.21.15.161:0/1970171951 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7f0c0af510 0x7f7f0c1f9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.565+0000 7f7f12e00640 1 --2- 172.21.15.161:0/1970171951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f0c0af090 0x7f7f0c1f9200 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.565+0000 7f7f12e00640 1 --2- 172.21.15.161:0/1970171951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ef0003980 0x7f7f0c0c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.565+0000 7f7f12e00640 1 -- 172.21.15.161:0/1970171951 >> 172.21.15.161:0/1970171951 conn(0x7f7f0c099a40 msgr2=0x7f7f0c0cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:59.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.565+0000 7f7f12e00640 1 -- 172.21.15.161:0/1970171951 shutdown_connections 2024-05-22T21:13:59.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.565+0000 7f7f12e00640 1 -- 172.21.15.161:0/1970171951 wait complete. 2024-05-22T21:13:59.589 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:13:21.237503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:13:22.425891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:13:59.590 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:13:59.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.721+0000 7f65fc400640 1 Processor -- start 2024-05-22T21:13:59.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.721+0000 7f65fc400640 1 -- start start 2024-05-22T21:13:59.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65fc400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65f41402e0 0x7f65f413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65fc400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f65f413fc80 0x7f65f406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65fc400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f41400d0 con 0x7f65f413cfc0 2024-05-22T21:13:59.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65fc400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f413d3a0 con 0x7f65f413fc80 2024-05-22T21:13:59.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65fc400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f40cd920 con 0x7f65f41402e0 2024-05-22T21:13:59.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f9a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f65f413cfc0 0x7f65f415af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33712/0 (socket says 172.21.15.161:33712) 2024-05-22T21:13:59.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f9a00640 1 -- 172.21.15.161:0/2533092702 learned_addr learned my addr 172.21.15.161:0/2533092702 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:13:59.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65fa400640 1 --2- 172.21.15.161:0/2533092702 >> v2:172.21.15.179:3300/0 conn(0x7f65f413fc80 0x7f65f406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f9000640 1 --2- 172.21.15.161:0/2533092702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65f41402e0 0x7f65f413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f3e00640 1 -- 172.21.15.161:0/2533092702 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504851711 0 0) 0x7f65e8001120 con 0x7f65f413cfc0 2024-05-22T21:13:59.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f3e00640 1 -- 172.21.15.161:0/2533092702 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65dc003530 con 0x7f65f413cfc0 2024-05-22T21:13:59.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f3e00640 1 -- 172.21.15.161:0/2533092702 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 142131878 0 0) 0x7f65dc003530 con 0x7f65f413cfc0 2024-05-22T21:13:59.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f3e00640 1 -- 172.21.15.161:0/2533092702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65f41402e0 msgr2=0x7f65f413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:59.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f3e00640 1 --2- 172.21.15.161:0/2533092702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65f41402e0 0x7f65f413f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f3e00640 1 -- 172.21.15.161:0/2533092702 >> v2:172.21.15.179:3300/0 conn(0x7f65f413fc80 msgr2=0x7f65f406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:59.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f3e00640 1 --2- 172.21.15.161:0/2533092702 >> v2:172.21.15.179:3300/0 conn(0x7f65f413fc80 0x7f65f406fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f3e00640 1 -- 172.21.15.161:0/2533092702 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f40cdc00 con 0x7f65f413cfc0 2024-05-22T21:13:59.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f3e00640 1 -- 172.21.15.161:0/2533092702 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f65e8001810 con 0x7f65f413cfc0 2024-05-22T21:13:59.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f3e00640 1 -- 172.21.15.161:0/2533092702 >> v1:172.21.15.161:6789/0 conn(0x7f65f413cfc0 legacy=0x7f65f415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:59.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f3e00640 1 --2- 172.21.15.161:0/2533092702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65dc003980 0x7f65dc003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f3e00640 1 --2- 172.21.15.161:0/2533092702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65dc004330 0x7f65dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f3e00640 1 --2- 172.21.15.161:0/2533092702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65f41402e0 0x7f65dc004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f3e00640 1 -- 172.21.15.161:0/2533092702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f413d3a0 con 0x7f65dc004330 2024-05-22T21:13:59.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f3e00640 1 -- 172.21.15.161:0/2533092702 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f40cd920 con 0x7f65dc003980 2024-05-22T21:13:59.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f3e00640 1 -- 172.21.15.161:0/2533092702 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc00b990 con 0x7f65f41402e0 2024-05-22T21:13:59.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f9000640 1 --2- 172.21.15.161:0/2533092702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65dc004330 0x7f65dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f9a00640 1 --2- 172.21.15.161:0/2533092702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65dc003980 0x7f65dc003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65fa400640 1 --2- 172.21.15.161:0/2533092702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65f41402e0 0x7f65dc004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f9000640 1 -- 172.21.15.161:0/2533092702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65f41402e0 msgr2=0x7f65dc004f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:59.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f9000640 1 --2- 172.21.15.161:0/2533092702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65f41402e0 0x7f65dc004f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f9000640 1 -- 172.21.15.161:0/2533092702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65dc003980 msgr2=0x7f65dc003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:13:59.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f9000640 1 --2- 172.21.15.161:0/2533092702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65dc003980 0x7f65dc003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f9000640 1 -- 172.21.15.161:0/2533092702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc003530 con 0x7f65dc004330 2024-05-22T21:13:59.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65fa400640 1 --2- 172.21.15.161:0/2533092702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65f41402e0 0x7f65dc004f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:59.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.725+0000 7f65f9a00640 1 --2- 172.21.15.161:0/2533092702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65dc003980 0x7f65dc003d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:13:59.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.729+0000 7f65f9000640 1 --2- 172.21.15.161:0/2533092702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65dc004330 0x7f65dc004810 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009ae0 tx=0x7f65e001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964880810ab98dfc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:59.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.729+0000 7f65f3e00640 1 -- 172.21.15.161:0/2533092702 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e00ac400 con 0x7f65dc004330 2024-05-22T21:13:59.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.729+0000 7f65f3e00640 1 -- 172.21.15.161:0/2533092702 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e00a8050 con 0x7f65dc004330 2024-05-22T21:13:59.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.729+0000 7f65f3e00640 1 -- 172.21.15.161:0/2533092702 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e00ac7a0 con 0x7f65dc004330 2024-05-22T21:13:59.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.729+0000 7f65fc400640 1 -- 172.21.15.161:0/2533092702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65dc004330 msgr2=0x7f65dc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:59.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.729+0000 7f65fc400640 1 --2- 172.21.15.161:0/2533092702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65dc004330 0x7f65dc004810 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009ae0 tx=0x7f65e001ca40 comp rx=0 tx=0).stop 2024-05-22T21:13:59.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.729+0000 7f65fa400640 1 -- 172.21.15.161:0/2533092702 reap_dead start 2024-05-22T21:13:59.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.729+0000 7f65fc400640 1 -- 172.21.15.161:0/2533092702 shutdown_connections 2024-05-22T21:13:59.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.729+0000 7f65fc400640 1 -- 172.21.15.161:0/2533092702 >> 172.21.15.161:0/2533092702 conn(0x7f65f406a2d0 msgr2=0x7f65f406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:59.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.729+0000 7f65fc400640 1 -- 172.21.15.161:0/2533092702 shutdown_connections 2024-05-22T21:13:59.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.729+0000 7f65fc400640 1 -- 172.21.15.161:0/2533092702 wait complete. 2024-05-22T21:13:59.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fc400640 1 Processor -- start 2024-05-22T21:13:59.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fc400640 1 -- start start 2024-05-22T21:13:59.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fc400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65dc004330 0x7f65f413c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fc400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65f415ad30 0x7f65f4157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fc400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65f412fb60 0x7f65f412cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fc400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f415be40 con 0x7f65f412fb60 2024-05-22T21:13:59.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fc400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f4157ea0 con 0x7f65dc004330 2024-05-22T21:13:59.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fc400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f415b1c0 con 0x7f65f415ad30 2024-05-22T21:13:59.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fa400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65f412fb60 0x7f65f412cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fa400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65f412fb60 0x7f65f412cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59132/0 (socket says 172.21.15.161:59132) 2024-05-22T21:13:59.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fa400640 1 -- 172.21.15.161:0/1574935839 learned_addr learned my addr 172.21.15.161:0/1574935839 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:13:59.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65f9a00640 1 --2- 172.21.15.161:0/1574935839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65dc004330 0x7f65f413c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65f9000640 1 --2- 172.21.15.161:0/1574935839 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65f415ad30 0x7f65f4157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fa400640 1 -- 172.21.15.161:0/1574935839 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65f415ad30 msgr2=0x7f65f4157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:59.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fa400640 1 --2- 172.21.15.161:0/1574935839 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65f415ad30 0x7f65f4157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fa400640 1 -- 172.21.15.161:0/1574935839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65dc004330 msgr2=0x7f65f413c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:59.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fa400640 1 --2- 172.21.15.161:0/1574935839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65dc004330 0x7f65f413c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fa400640 1 -- 172.21.15.161:0/1574935839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f415b4a0 con 0x7f65f412fb60 2024-05-22T21:13:59.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65f9a00640 1 --2- 172.21.15.161:0/1574935839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65dc004330 0x7f65f413c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:59.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65f9000640 1 --2- 172.21.15.161:0/1574935839 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65f415ad30 0x7f65f4157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:13:59.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fa400640 1 --2- 172.21.15.161:0/1574935839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65f412fb60 0x7f65f412cf00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f65f40b2150 tx=0x7f65ec012590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:59.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65f1600640 1 -- 172.21.15.161:0/1574935839 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec010070 con 0x7f65f412fb60 2024-05-22T21:13:59.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65f1600640 1 -- 172.21.15.161:0/1574935839 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ec011040 con 0x7f65f412fb60 2024-05-22T21:13:59.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65f1600640 1 -- 172.21.15.161:0/1574935839 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec0039a0 con 0x7f65f412fb60 2024-05-22T21:13:59.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fc400640 1 -- 172.21.15.161:0/1574935839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f415b780 con 0x7f65f412fb60 2024-05-22T21:13:59.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fc400640 1 -- 172.21.15.161:0/1574935839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f412d440 con 0x7f65f412fb60 2024-05-22T21:13:59.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.733+0000 7f65fc400640 1 -- 172.21.15.161:0/1574935839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f415bf80 con 0x7f65f412fb60 2024-05-22T21:13:59.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.737+0000 7f65f1600640 1 -- 172.21.15.161:0/1574935839 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f65ec020460 con 0x7f65f412fb60 2024-05-22T21:13:59.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.737+0000 7f65f1600640 1 --2- 172.21.15.161:0/1574935839 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65d4066690 0x7f65d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:13:59.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.737+0000 7f65f1600640 1 -- 172.21.15.161:0/1574935839 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65ec0864c0 con 0x7f65f412fb60 2024-05-22T21:13:59.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.737+0000 7f65f9a00640 1 --2- 172.21.15.161:0/1574935839 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65d4066690 0x7f65d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:13:59.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.737+0000 7f65f9a00640 1 --2- 172.21.15.161:0/1574935839 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65d4066690 0x7f65d4068b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f65e8000fc0 tx=0x7f65e800a040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:13:59.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.741+0000 7f65f1600640 1 -- 172.21.15.161:0/1574935839 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f65f415bf80 con 0x7f65f412fb60 2024-05-22T21:13:59.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.893+0000 7f65fc400640 1 -- 172.21.15.161:0/1574935839 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65f415bf80 con 0x7f65f412fb60 2024-05-22T21:13:59.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.897+0000 7f65f1600640 1 -- 172.21.15.161:0/1574935839 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65f415bf80 con 0x7f65f412fb60 2024-05-22T21:13:59.900 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:13:59.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.897+0000 7f65fc400640 1 -- 172.21.15.161:0/1574935839 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65d4066690 msgr2=0x7f65d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:59.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.897+0000 7f65fc400640 1 --2- 172.21.15.161:0/1574935839 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65d4066690 0x7f65d4068b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f65e8000fc0 tx=0x7f65e800a040 comp rx=0 tx=0).stop 2024-05-22T21:13:59.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.897+0000 7f65fc400640 1 -- 172.21.15.161:0/1574935839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65f412fb60 msgr2=0x7f65f412cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:13:59.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.897+0000 7f65fc400640 1 --2- 172.21.15.161:0/1574935839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65f412fb60 0x7f65f412cf00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f65f40b2150 tx=0x7f65ec012590 comp rx=0 tx=0).stop 2024-05-22T21:13:59.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.897+0000 7f65fc400640 1 -- 172.21.15.161:0/1574935839 shutdown_connections 2024-05-22T21:13:59.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.897+0000 7f65fc400640 1 --2- 172.21.15.161:0/1574935839 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65d4066690 0x7f65d4068b50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.897+0000 7f65fc400640 1 --2- 172.21.15.161:0/1574935839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65f412fb60 0x7f65f412cf00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.897+0000 7f65fc400640 1 --2- 172.21.15.161:0/1574935839 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65f415ad30 0x7f65f4157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.897+0000 7f65fc400640 1 --2- 172.21.15.161:0/1574935839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65dc004330 0x7f65f413c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:13:59.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.897+0000 7f65fc400640 1 -- 172.21.15.161:0/1574935839 >> 172.21.15.161:0/1574935839 conn(0x7f65f406a2d0 msgr2=0x7f65f406cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:13:59.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.897+0000 7f65fc400640 1 -- 172.21.15.161:0/1574935839 shutdown_connections 2024-05-22T21:13:59.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:13:59.897+0000 7f65fc400640 1 -- 172.21.15.161:0/1574935839 wait complete. 2024-05-22T21:13:59.922 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:14:00.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.057+0000 7f45bb000640 1 Processor -- start 2024-05-22T21:14:00.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.057+0000 7f45bb000640 1 -- start start 2024-05-22T21:14:00.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.061+0000 7f45bb000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f45b41402e0 0x7f45b413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:00.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.061+0000 7f45bb000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b4140090 con 0x7f45b413fc80 2024-05-22T21:14:00.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.061+0000 7f45bb000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b413d3a0 con 0x7f45b413cfc0 2024-05-22T21:14:00.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.061+0000 7f45bb000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b40cd920 con 0x7f45b41402e0 2024-05-22T21:14:00.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.061+0000 7f45b9000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f45b413fc80 0x7f45b406fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33714/0 (socket says 172.21.15.161:33714) 2024-05-22T21:14:00.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.061+0000 7f45b9000640 1 -- 172.21.15.161:0/2594037679 learned_addr learned my addr 172.21.15.161:0/2594037679 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:00.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.061+0000 7f45ab400640 1 --2- 172.21.15.161:0/2594037679 >> v2:172.21.15.179:3300/0 conn(0x7f45b41402e0 0x7f45b413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:00.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.061+0000 7f45b3400640 1 -- 172.21.15.161:0/2594037679 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743914965 0 0) 0x7f45b4140090 con 0x7f45b413fc80 2024-05-22T21:14:00.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.061+0000 7f45b3400640 1 -- 172.21.15.161:0/2594037679 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4598003530 con 0x7f45b413fc80 2024-05-22T21:14:00.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.061+0000 7f45ab400640 1 -- 172.21.15.161:0/2594037679 >> v1:172.21.15.179:6789/0 conn(0x7f45b413cfc0 legacy=0x7f45b415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.061+0000 7f45ab400640 1 -- 172.21.15.161:0/2594037679 >> v1:172.21.15.161:6789/0 conn(0x7f45b413fc80 legacy=0x7f45b406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.061+0000 7f45ab400640 1 -- 172.21.15.161:0/2594037679 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b40cdc00 con 0x7f45b41402e0 2024-05-22T21:14:00.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45ab400640 1 --2- 172.21.15.161:0/2594037679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b41402e0 0x7f45b413f740 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f459c0097c0 tx=0x7f459c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6dd97cba0ce66e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:00.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45b3400640 1 -- 172.21.15.161:0/2594037679 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c0ac440 con 0x7f45b41402e0 2024-05-22T21:14:00.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45b3400640 1 -- 172.21.15.161:0/2594037679 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c0ac5a0 con 0x7f45b41402e0 2024-05-22T21:14:00.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45b3400640 1 -- 172.21.15.161:0/2594037679 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c0ac930 con 0x7f45b41402e0 2024-05-22T21:14:00.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45bb000640 1 -- 172.21.15.161:0/2594037679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b41402e0 msgr2=0x7f45b413f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45bb000640 1 --2- 172.21.15.161:0/2594037679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b41402e0 0x7f45b413f740 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f459c0097c0 tx=0x7f459c01c3e0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45bb000640 1 -- 172.21.15.161:0/2594037679 shutdown_connections 2024-05-22T21:14:00.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45bb000640 1 --2- 172.21.15.161:0/2594037679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b41402e0 0x7f45b413f740 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45bb000640 1 -- 172.21.15.161:0/2594037679 >> 172.21.15.161:0/2594037679 conn(0x7f45b406a2d0 msgr2=0x7f45b406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:00.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45bb000640 1 -- 172.21.15.161:0/2594037679 shutdown_connections 2024-05-22T21:14:00.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45bb000640 1 -- 172.21.15.161:0/2594037679 wait complete. 2024-05-22T21:14:00.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45bb000640 1 Processor -- start 2024-05-22T21:14:00.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45bb000640 1 -- start start 2024-05-22T21:14:00.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45bb000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b413cfc0 0x7f45b415ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:00.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45bb000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45b413fc80 0x7f45b40b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:00.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45bb000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f45b41402e0 0x7f45b41422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:00.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45bb000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b40b3d40 con 0x7f45b413fc80 2024-05-22T21:14:00.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45bb000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b415bc60 con 0x7f45b413cfc0 2024-05-22T21:14:00.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.065+0000 7f45bb000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b415b3c0 con 0x7f45b41402e0 2024-05-22T21:14:00.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45ab400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45b413fc80 0x7f45b40b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:00.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45ab400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45b413fc80 0x7f45b40b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59138/0 (socket says 172.21.15.161:59138) 2024-05-22T21:14:00.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45ab400640 1 -- 172.21.15.161:0/4160994225 learned_addr learned my addr 172.21.15.161:0/4160994225 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:00.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45b3e00640 1 --2- 172.21.15.161:0/4160994225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b413cfc0 0x7f45b415ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:00.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45b9000640 1 --2- 172.21.15.161:0/4160994225 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f45b41402e0 0x7f45b41422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:00.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45ab400640 1 -- 172.21.15.161:0/4160994225 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f45b41402e0 msgr2=0x7f45b41422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45ab400640 1 --2- 172.21.15.161:0/4160994225 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f45b41402e0 0x7f45b41422f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45ab400640 1 -- 172.21.15.161:0/4160994225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b413cfc0 msgr2=0x7f45b415ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45ab400640 1 --2- 172.21.15.161:0/4160994225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b413cfc0 0x7f45b415ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45ab400640 1 -- 172.21.15.161:0/4160994225 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c009420 con 0x7f45b413fc80 2024-05-22T21:14:00.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45ab400640 1 --2- 172.21.15.161:0/4160994225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45b413fc80 0x7f45b40b3800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f459c018ff0 tx=0x7f459c019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:00.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45b0c00640 1 -- 172.21.15.161:0/4160994225 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c0b5810 con 0x7f45b413fc80 2024-05-22T21:14:00.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45b0c00640 1 -- 172.21.15.161:0/4160994225 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c0b59b0 con 0x7f45b413fc80 2024-05-22T21:14:00.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45b0c00640 1 -- 172.21.15.161:0/4160994225 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c0b5d60 con 0x7f45b413fc80 2024-05-22T21:14:00.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45bb000640 1 -- 172.21.15.161:0/4160994225 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b415b6a0 con 0x7f45b413fc80 2024-05-22T21:14:00.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45bb000640 1 -- 172.21.15.161:0/4160994225 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b4140930 con 0x7f45b413fc80 2024-05-22T21:14:00.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45bb000640 1 -- 172.21.15.161:0/4160994225 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b4140b20 con 0x7f45b413fc80 2024-05-22T21:14:00.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45b0c00640 1 -- 172.21.15.161:0/4160994225 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f459c0b3040 con 0x7f45b413fc80 2024-05-22T21:14:00.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45b0c00640 1 --2- 172.21.15.161:0/4160994225 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4590066690 0x7f4590068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:00.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.069+0000 7f45b0c00640 1 -- 172.21.15.161:0/4160994225 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f459c11e430 con 0x7f45b413fc80 2024-05-22T21:14:00.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.073+0000 7f45b3e00640 1 --2- 172.21.15.161:0/4160994225 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4590066690 0x7f4590068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:00.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.073+0000 7f45b3e00640 1 --2- 172.21.15.161:0/4160994225 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4590066690 0x7f4590068b50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f45b406ca70 tx=0x7f45a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:00.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.077+0000 7f45b0c00640 1 -- 172.21.15.161:0/4160994225 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f459c0eb840 con 0x7f45b413fc80 2024-05-22T21:14:00.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.229+0000 7f45bb000640 1 -- 172.21.15.161:0/4160994225 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45b4067220 con 0x7f45b413fc80 2024-05-22T21:14:00.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.229+0000 7f45b0c00640 1 -- 172.21.15.161:0/4160994225 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f459c0eee60 con 0x7f45b413fc80 2024-05-22T21:14:00.234 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:14:00.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.233+0000 7f45bb000640 1 -- 172.21.15.161:0/4160994225 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4590066690 msgr2=0x7f4590068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.233+0000 7f45bb000640 1 --2- 172.21.15.161:0/4160994225 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4590066690 0x7f4590068b50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f45b406ca70 tx=0x7f45a4009040 comp rx=0 tx=0).stop 2024-05-22T21:14:00.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.233+0000 7f45bb000640 1 -- 172.21.15.161:0/4160994225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45b413fc80 msgr2=0x7f45b40b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.233+0000 7f45bb000640 1 --2- 172.21.15.161:0/4160994225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45b413fc80 0x7f45b40b3800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f459c018ff0 tx=0x7f459c019180 comp rx=0 tx=0).stop 2024-05-22T21:14:00.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.233+0000 7f45bb000640 1 -- 172.21.15.161:0/4160994225 shutdown_connections 2024-05-22T21:14:00.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.233+0000 7f45bb000640 1 --2- 172.21.15.161:0/4160994225 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4590066690 0x7f4590068b50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.233+0000 7f45bb000640 1 --2- 172.21.15.161:0/4160994225 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f45b41402e0 0x7f45b41422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.233+0000 7f45bb000640 1 --2- 172.21.15.161:0/4160994225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45b413fc80 0x7f45b40b3800 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.233+0000 7f45bb000640 1 --2- 172.21.15.161:0/4160994225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b413cfc0 0x7f45b415ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.233+0000 7f45bb000640 1 -- 172.21.15.161:0/4160994225 >> 172.21.15.161:0/4160994225 conn(0x7f45b406a2d0 msgr2=0x7f45b40aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:00.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.233+0000 7f45bb000640 1 -- 172.21.15.161:0/4160994225 shutdown_connections 2024-05-22T21:14:00.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.233+0000 7f45bb000640 1 -- 172.21.15.161:0/4160994225 wait complete. 2024-05-22T21:14:00.255 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:14:00.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.389+0000 7fb33e400640 1 Processor -- start 2024-05-22T21:14:00.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.389+0000 7fb33e400640 1 -- start start 2024-05-22T21:14:00.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.389+0000 7fb33e400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb338143c40 0x7fb33815b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:00.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.389+0000 7fb33e400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb33815efb0 con 0x7fb33806c9b0 2024-05-22T21:14:00.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.389+0000 7fb33e400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb33815f290 con 0x7fb3381444a0 2024-05-22T21:14:00.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.389+0000 7fb33e400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33815f570 con 0x7fb338143c40 2024-05-22T21:14:00.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.389+0000 7fb337e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb33806c9b0 0x7fb3380b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33726/0 (socket says 172.21.15.161:33726) 2024-05-22T21:14:00.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.389+0000 7fb337e00640 1 -- 172.21.15.161:0/3452245323 learned_addr learned my addr 172.21.15.161:0/3452245323 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:00.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.389+0000 7fb337400640 1 --2- 172.21.15.161:0/3452245323 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb338143c40 0x7fb33815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:00.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.389+0000 7fb336000640 1 -- 172.21.15.161:0/3452245323 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332759235 0 0) 0x7fb33815efb0 con 0x7fb33806c9b0 2024-05-22T21:14:00.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.389+0000 7fb336000640 1 -- 172.21.15.161:0/3452245323 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb31c003530 con 0x7fb33806c9b0 2024-05-22T21:14:00.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.389+0000 7fb336000640 1 -- 172.21.15.161:0/3452245323 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4116573511 0 0) 0x7fb33815f290 con 0x7fb3381444a0 2024-05-22T21:14:00.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.389+0000 7fb336000640 1 -- 172.21.15.161:0/3452245323 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb33815efb0 con 0x7fb3381444a0 2024-05-22T21:14:00.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.389+0000 7fb337400640 1 -- 172.21.15.161:0/3452245323 >> v1:172.21.15.179:6789/0 conn(0x7fb3381444a0 legacy=0x7fb33806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.389+0000 7fb337400640 1 -- 172.21.15.161:0/3452245323 >> v1:172.21.15.161:6789/0 conn(0x7fb33806c9b0 legacy=0x7fb3380b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.389+0000 7fb337400640 1 -- 172.21.15.161:0/3452245323 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33815f850 con 0x7fb338143c40 2024-05-22T21:14:00.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.393+0000 7fb337400640 1 --2- 172.21.15.161:0/3452245323 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb338143c40 0x7fb33815b600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3280097f0 tx=0x7fb32801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c01bdde04a9b184b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:00.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.393+0000 7fb336000640 1 -- 172.21.15.161:0/3452245323 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3280ab640 con 0x7fb338143c40 2024-05-22T21:14:00.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.393+0000 7fb336000640 1 -- 172.21.15.161:0/3452245323 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3280a7070 con 0x7fb338143c40 2024-05-22T21:14:00.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.393+0000 7fb336000640 1 -- 172.21.15.161:0/3452245323 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3280ab9c0 con 0x7fb338143c40 2024-05-22T21:14:00.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.393+0000 7fb33e400640 1 -- 172.21.15.161:0/3452245323 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb338143c40 msgr2=0x7fb33815b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.393+0000 7fb33e400640 1 --2- 172.21.15.161:0/3452245323 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb338143c40 0x7fb33815b600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3280097f0 tx=0x7fb32801c4e0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.393+0000 7fb33e400640 1 -- 172.21.15.161:0/3452245323 shutdown_connections 2024-05-22T21:14:00.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.393+0000 7fb33e400640 1 --2- 172.21.15.161:0/3452245323 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb338143c40 0x7fb33815b600 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.393+0000 7fb33e400640 1 -- 172.21.15.161:0/3452245323 >> 172.21.15.161:0/3452245323 conn(0x7fb33806a2d0 msgr2=0x7fb33806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:00.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.393+0000 7fb33e400640 1 -- 172.21.15.161:0/3452245323 shutdown_connections 2024-05-22T21:14:00.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.393+0000 7fb33e400640 1 -- 172.21.15.161:0/3452245323 wait complete. 2024-05-22T21:14:00.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb33e400640 1 Processor -- start 2024-05-22T21:14:00.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb33e400640 1 -- start start 2024-05-22T21:14:00.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb33e400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb33806c9b0 0x7fb338157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:00.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb33e400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb338143c40 0x7fb3381282e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:00.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb33e400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3381444a0 0x7fb33806fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:00.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb33e400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380adf80 con 0x7fb33806c9b0 2024-05-22T21:14:00.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb33e400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb338127e50 con 0x7fb3381444a0 2024-05-22T21:14:00.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb33e400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb338127840 con 0x7fb338143c40 2024-05-22T21:14:00.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb337400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb33806c9b0 0x7fb338157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:00.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb337400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb33806c9b0 0x7fb338157820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59154/0 (socket says 172.21.15.161:59154) 2024-05-22T21:14:00.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb337400640 1 -- 172.21.15.161:0/628083771 learned_addr learned my addr 172.21.15.161:0/628083771 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:00.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb337e00640 1 --2- 172.21.15.161:0/628083771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3381444a0 0x7fb33806fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:00.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb336a00640 1 --2- 172.21.15.161:0/628083771 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb338143c40 0x7fb3381282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:00.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb337400640 1 -- 172.21.15.161:0/628083771 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb338143c40 msgr2=0x7fb3381282e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb337400640 1 --2- 172.21.15.161:0/628083771 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb338143c40 0x7fb3381282e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb337400640 1 -- 172.21.15.161:0/628083771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3381444a0 msgr2=0x7fb33806fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb337400640 1 --2- 172.21.15.161:0/628083771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3381444a0 0x7fb33806fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb337400640 1 -- 172.21.15.161:0/628083771 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb328009420 con 0x7fb33806c9b0 2024-05-22T21:14:00.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb336a00640 1 --2- 172.21.15.161:0/628083771 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb338143c40 0x7fb3381282e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:00.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb337400640 1 --2- 172.21.15.161:0/628083771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb33806c9b0 0x7fb338157820 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb3280190f0 tx=0x7fb32801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:00.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb32f400640 1 -- 172.21.15.161:0/628083771 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3280abe30 con 0x7fb33806c9b0 2024-05-22T21:14:00.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb32f400640 1 -- 172.21.15.161:0/628083771 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3280bf410 con 0x7fb33806c9b0 2024-05-22T21:14:00.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb32f400640 1 -- 172.21.15.161:0/628083771 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3280bf7c0 con 0x7fb33806c9b0 2024-05-22T21:14:00.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb33e400640 1 -- 172.21.15.161:0/628083771 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3380adca0 con 0x7fb33806c9b0 2024-05-22T21:14:00.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.397+0000 7fb33e400640 1 -- 172.21.15.161:0/628083771 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb338157d60 con 0x7fb33806c9b0 2024-05-22T21:14:00.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.401+0000 7fb33e400640 1 -- 172.21.15.161:0/628083771 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3381276a0 con 0x7fb33806c9b0 2024-05-22T21:14:00.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.401+0000 7fb32f400640 1 -- 172.21.15.161:0/628083771 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb3280b5400 con 0x7fb33806c9b0 2024-05-22T21:14:00.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.401+0000 7fb32f400640 1 --2- 172.21.15.161:0/628083771 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb318066690 0x7fb318068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:00.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.401+0000 7fb32f400640 1 -- 172.21.15.161:0/628083771 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb32811e310 con 0x7fb33806c9b0 2024-05-22T21:14:00.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.401+0000 7fb336a00640 1 --2- 172.21.15.161:0/628083771 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb318066690 0x7fb318068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:00.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.401+0000 7fb336a00640 1 --2- 172.21.15.161:0/628083771 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb318066690 0x7fb318068b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb33815b320 tx=0x7fb324009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:00.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.405+0000 7fb32f400640 1 -- 172.21.15.161:0/628083771 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb3381276a0 con 0x7fb33806c9b0 2024-05-22T21:14:00.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.557+0000 7fb33e400640 1 -- 172.21.15.161:0/628083771 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb3381276a0 con 0x7fb33806c9b0 2024-05-22T21:14:00.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.561+0000 7fb32f400640 1 -- 172.21.15.161:0/628083771 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb3381276a0 con 0x7fb33806c9b0 2024-05-22T21:14:00.564 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:14:00.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.561+0000 7fb33e400640 1 -- 172.21.15.161:0/628083771 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb318066690 msgr2=0x7fb318068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.561+0000 7fb33e400640 1 --2- 172.21.15.161:0/628083771 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb318066690 0x7fb318068b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb33815b320 tx=0x7fb324009040 comp rx=0 tx=0).stop 2024-05-22T21:14:00.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.561+0000 7fb33e400640 1 -- 172.21.15.161:0/628083771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb33806c9b0 msgr2=0x7fb338157820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.561+0000 7fb33e400640 1 --2- 172.21.15.161:0/628083771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb33806c9b0 0x7fb338157820 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb3280190f0 tx=0x7fb32801ad20 comp rx=0 tx=0).stop 2024-05-22T21:14:00.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.561+0000 7fb33e400640 1 -- 172.21.15.161:0/628083771 shutdown_connections 2024-05-22T21:14:00.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.561+0000 7fb33e400640 1 --2- 172.21.15.161:0/628083771 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb318066690 0x7fb318068b50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.561+0000 7fb33e400640 1 --2- 172.21.15.161:0/628083771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3381444a0 0x7fb33806fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.561+0000 7fb33e400640 1 --2- 172.21.15.161:0/628083771 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb338143c40 0x7fb3381282e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.561+0000 7fb33e400640 1 --2- 172.21.15.161:0/628083771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb33806c9b0 0x7fb338157820 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.561+0000 7fb33e400640 1 -- 172.21.15.161:0/628083771 >> 172.21.15.161:0/628083771 conn(0x7fb33806a2d0 msgr2=0x7fb33815b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:00.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.561+0000 7fb33e400640 1 -- 172.21.15.161:0/628083771 shutdown_connections 2024-05-22T21:14:00.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.561+0000 7fb33e400640 1 -- 172.21.15.161:0/628083771 wait complete. 2024-05-22T21:14:00.586 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:14:00.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.717+0000 7f04d2c00640 1 Processor -- start 2024-05-22T21:14:00.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.717+0000 7f04d2c00640 1 -- start start 2024-05-22T21:14:00.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.717+0000 7f04d2c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f04cc06c9b0 0x7f04cc0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:00.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.717+0000 7f04d2c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04cc144bb0 con 0x7f04cc143c40 2024-05-22T21:14:00.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.717+0000 7f04d2c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04cc15efb0 con 0x7f04cc15b600 2024-05-22T21:14:00.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.717+0000 7f04d2c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04cc15ba60 con 0x7f04cc06c9b0 2024-05-22T21:14:00.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.717+0000 7f04cbe00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f04cc143c40 0x7f04cc1444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33730/0 (socket says 172.21.15.161:33730) 2024-05-22T21:14:00.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.717+0000 7f04cbe00640 1 -- 172.21.15.161:0/3870660414 learned_addr learned my addr 172.21.15.161:0/3870660414 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:00.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.717+0000 7f04d0c00640 1 --2- 172.21.15.161:0/3870660414 >> v2:172.21.15.161:3300/0 conn(0x7f04cc06c9b0 0x7f04cc0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:00.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.721+0000 7f04d0c00640 1 -- 172.21.15.161:0/3870660414 >> v1:172.21.15.179:6789/0 conn(0x7f04cc15b600 legacy=0x7f04cc06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:00.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.721+0000 7f04d0c00640 1 -- 172.21.15.161:0/3870660414 >> v1:172.21.15.161:6789/0 conn(0x7f04cc143c40 legacy=0x7f04cc1444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.721+0000 7f04d0c00640 1 -- 172.21.15.161:0/3870660414 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc15f290 con 0x7f04cc06c9b0 2024-05-22T21:14:00.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.721+0000 7f04d0c00640 1 --2- 172.21.15.161:0/3870660414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04cc06c9b0 0x7f04cc0b2160 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009d90 tx=0x7f04c401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ac5fe402e82c79 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:00.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.721+0000 7f04cb400640 1 -- 172.21.15.161:0/3870660414 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c40a59f0 con 0x7f04cc06c9b0 2024-05-22T21:14:00.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.721+0000 7f04cb400640 1 -- 172.21.15.161:0/3870660414 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c40a5b50 con 0x7f04cc06c9b0 2024-05-22T21:14:00.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.721+0000 7f04cb400640 1 -- 172.21.15.161:0/3870660414 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c40ad440 con 0x7f04cc06c9b0 2024-05-22T21:14:00.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.721+0000 7f04d2c00640 1 -- 172.21.15.161:0/3870660414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04cc06c9b0 msgr2=0x7f04cc0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.721+0000 7f04d2c00640 1 --2- 172.21.15.161:0/3870660414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04cc06c9b0 0x7f04cc0b2160 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009d90 tx=0x7f04c401c930 comp rx=0 tx=0).stop 2024-05-22T21:14:00.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d2c00640 1 -- 172.21.15.161:0/3870660414 shutdown_connections 2024-05-22T21:14:00.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d2c00640 1 --2- 172.21.15.161:0/3870660414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04cc06c9b0 0x7f04cc0b2160 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d2c00640 1 -- 172.21.15.161:0/3870660414 >> 172.21.15.161:0/3870660414 conn(0x7f04cc06a2d0 msgr2=0x7f04cc06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:00.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d2c00640 1 -- 172.21.15.161:0/3870660414 shutdown_connections 2024-05-22T21:14:00.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d2c00640 1 -- 172.21.15.161:0/3870660414 wait complete. 2024-05-22T21:14:00.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d2c00640 1 Processor -- start 2024-05-22T21:14:00.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d2c00640 1 -- start start 2024-05-22T21:14:00.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d2c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04cc06c9b0 0x7f04cc136a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:00.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d2c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04cc143c40 0x7f04cc1597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:00.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d2c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04cc15b600 0x7f04cc159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:00.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d2c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc1569c0 con 0x7f04cc15b600 2024-05-22T21:14:00.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d2c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc136fc0 con 0x7f04cc06c9b0 2024-05-22T21:14:00.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d2c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc070ab0 con 0x7f04cc143c40 2024-05-22T21:14:00.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d0c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04cc15b600 0x7f04cc159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:00.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d0c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04cc15b600 0x7f04cc159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59176/0 (socket says 172.21.15.161:59176) 2024-05-22T21:14:00.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d0c00640 1 -- 172.21.15.161:0/1919744883 learned_addr learned my addr 172.21.15.161:0/1919744883 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:00.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04cbe00640 1 --2- 172.21.15.161:0/1919744883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04cc06c9b0 0x7f04cc136a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:00.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04c3400640 1 --2- 172.21.15.161:0/1919744883 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04cc143c40 0x7f04cc1597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:00.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d0c00640 1 -- 172.21.15.161:0/1919744883 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04cc143c40 msgr2=0x7f04cc1597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d0c00640 1 --2- 172.21.15.161:0/1919744883 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04cc143c40 0x7f04cc1597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d0c00640 1 -- 172.21.15.161:0/1919744883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04cc06c9b0 msgr2=0x7f04cc136a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d0c00640 1 --2- 172.21.15.161:0/1919744883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04cc06c9b0 0x7f04cc136a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d0c00640 1 -- 172.21.15.161:0/1919744883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c4009a70 con 0x7f04cc15b600 2024-05-22T21:14:00.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04cbe00640 1 --2- 172.21.15.161:0/1919744883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04cc06c9b0 0x7f04cc136a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:00.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04c3400640 1 --2- 172.21.15.161:0/1919744883 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04cc143c40 0x7f04cc1597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:00.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.725+0000 7f04d0c00640 1 --2- 172.21.15.161:0/1919744883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04cc15b600 0x7f04cc159d20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f04c401a560 tx=0x7f04c40196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:00.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.729+0000 7f04c8c00640 1 -- 172.21.15.161:0/1919744883 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c40ade20 con 0x7f04cc15b600 2024-05-22T21:14:00.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.729+0000 7f04c8c00640 1 -- 172.21.15.161:0/1919744883 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c40b9450 con 0x7f04cc15b600 2024-05-22T21:14:00.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.729+0000 7f04d2c00640 1 -- 172.21.15.161:0/1919744883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04cc070d90 con 0x7f04cc15b600 2024-05-22T21:14:00.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.729+0000 7f04c8c00640 1 -- 172.21.15.161:0/1919744883 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c40b9800 con 0x7f04cc15b600 2024-05-22T21:14:00.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.729+0000 7f04d2c00640 1 -- 172.21.15.161:0/1919744883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04cc156650 con 0x7f04cc15b600 2024-05-22T21:14:00.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.729+0000 7f04d2c00640 1 -- 172.21.15.161:0/1919744883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04cc1564b0 con 0x7f04cc15b600 2024-05-22T21:14:00.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.729+0000 7f04c8c00640 1 -- 172.21.15.161:0/1919744883 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f04c40b9960 con 0x7f04cc15b600 2024-05-22T21:14:00.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.729+0000 7f04c8c00640 1 --2- 172.21.15.161:0/1919744883 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f04a8066690 0x7f04a8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:00.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.729+0000 7f04c8c00640 1 -- 172.21.15.161:0/1919744883 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04c411fa50 con 0x7f04cc15b600 2024-05-22T21:14:00.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.729+0000 7f04cbe00640 1 --2- 172.21.15.161:0/1919744883 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f04a8066690 0x7f04a8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:00.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.729+0000 7f04cbe00640 1 --2- 172.21.15.161:0/1919744883 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f04a8066690 0x7f04a8068b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f04b4005890 tx=0x7f04b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:00.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.733+0000 7f04c8c00640 1 -- 172.21.15.161:0/1919744883 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f04cc1564b0 con 0x7f04cc15b600 2024-05-22T21:14:00.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.901+0000 7f04d2c00640 1 -- 172.21.15.161:0/1919744883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f04cc1564b0 con 0x7f04cc15b600 2024-05-22T21:14:00.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.901+0000 7f04c8c00640 1 -- 172.21.15.161:0/1919744883 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f04cc1564b0 con 0x7f04cc15b600 2024-05-22T21:14:00.907 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:14:00.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.905+0000 7f04d2c00640 1 -- 172.21.15.161:0/1919744883 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f04a8066690 msgr2=0x7f04a8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.905+0000 7f04d2c00640 1 --2- 172.21.15.161:0/1919744883 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f04a8066690 0x7f04a8068b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f04b4005890 tx=0x7f04b4009040 comp rx=0 tx=0).stop 2024-05-22T21:14:00.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.905+0000 7f04d2c00640 1 -- 172.21.15.161:0/1919744883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04cc15b600 msgr2=0x7f04cc159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:00.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.905+0000 7f04d2c00640 1 --2- 172.21.15.161:0/1919744883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04cc15b600 0x7f04cc159d20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f04c401a560 tx=0x7f04c40196d0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.905+0000 7f04d2c00640 1 -- 172.21.15.161:0/1919744883 shutdown_connections 2024-05-22T21:14:00.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.905+0000 7f04d2c00640 1 --2- 172.21.15.161:0/1919744883 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f04a8066690 0x7f04a8068b50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.905+0000 7f04d2c00640 1 --2- 172.21.15.161:0/1919744883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04cc15b600 0x7f04cc159d20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.905+0000 7f04d2c00640 1 --2- 172.21.15.161:0/1919744883 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04cc143c40 0x7f04cc1597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.905+0000 7f04d2c00640 1 --2- 172.21.15.161:0/1919744883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04cc06c9b0 0x7f04cc136a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:00.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.905+0000 7f04d2c00640 1 -- 172.21.15.161:0/1919744883 >> 172.21.15.161:0/1919744883 conn(0x7f04cc06a2d0 msgr2=0x7f04cc142d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:00.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.905+0000 7f04d2c00640 1 -- 172.21.15.161:0/1919744883 shutdown_connections 2024-05-22T21:14:00.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:00.905+0000 7f04d2c00640 1 -- 172.21.15.161:0/1919744883 wait complete. 2024-05-22T21:14:00.929 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-22T21:14:00.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:00.929 DEBUG:teuthology.orchestra.run.smithi179:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-05-22T21:14:00.963 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:14:00.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:00.963 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-22T21:14:01.026 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:14:01.026 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:01.035 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:14:01.035 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:01.083 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:14:01.083 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:01.131 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:01.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f952b000640 1 Processor -- start 2024-05-22T21:14:01.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f952b000640 1 -- start start 2024-05-22T21:14:01.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f952b000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f95241449f0 0x7f952415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f952b000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952415bc50 con 0x7f9524071180 2024-05-22T21:14:01.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f952b000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952415f340 con 0x7f952415b7a0 2024-05-22T21:14:01.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f952b000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952415f620 con 0x7f95241449f0 2024-05-22T21:14:01.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9529000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f9524071180 0x7f9524071560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33738/0 (socket says 172.21.15.161:33738) 2024-05-22T21:14:01.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9529000640 1 -- 172.21.15.161:0/3749520714 learned_addr learned my addr 172.21.15.161:0/3749520714 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:01.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9523e00640 1 --2- 172.21.15.161:0/3749520714 >> v2:172.21.15.179:3300/0 conn(0x7f95241449f0 0x7f952415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9523400640 1 -- 172.21.15.161:0/3749520714 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997799908 0 0) 0x7f952415bc50 con 0x7f9524071180 2024-05-22T21:14:01.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9523400640 1 -- 172.21.15.161:0/3749520714 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9508003530 con 0x7f9524071180 2024-05-22T21:14:01.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9523400640 1 -- 172.21.15.161:0/3749520714 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2979846859 0 0) 0x7f9508003530 con 0x7f9524071180 2024-05-22T21:14:01.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9523400640 1 -- 172.21.15.161:0/3749520714 >> v2:172.21.15.179:3300/0 conn(0x7f95241449f0 msgr2=0x7f952415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:01.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9523400640 1 --2- 172.21.15.161:0/3749520714 >> v2:172.21.15.179:3300/0 conn(0x7f95241449f0 0x7f952415ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9523400640 1 -- 172.21.15.161:0/3749520714 >> v1:172.21.15.179:6789/0 conn(0x7f952415b7a0 legacy=0x7f952406a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:01.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9523400640 1 -- 172.21.15.161:0/3749520714 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952415f900 con 0x7f9524071180 2024-05-22T21:14:01.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9523400640 1 -- 172.21.15.161:0/3749520714 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f951c001680 con 0x7f9524071180 2024-05-22T21:14:01.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9523400640 1 -- 172.21.15.161:0/3749520714 >> v1:172.21.15.161:6789/0 conn(0x7f9524071180 legacy=0x7f9524071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:01.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9523400640 1 --2- 172.21.15.161:0/3749520714 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9508003980 0x7f9508003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9523400640 1 --2- 172.21.15.161:0/3749520714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95080047d0 0x7f9508004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9523400640 1 --2- 172.21.15.161:0/3749520714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9508005190 0x7f950800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9523400640 1 -- 172.21.15.161:0/3749520714 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952415f620 con 0x7f95080047d0 2024-05-22T21:14:01.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9523400640 1 -- 172.21.15.161:0/3749520714 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950800c130 con 0x7f9508005190 2024-05-22T21:14:01.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.305+0000 7f9523400640 1 -- 172.21.15.161:0/3749520714 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f950800c2f0 con 0x7f9508003980 2024-05-22T21:14:01.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.309+0000 7f951b400640 1 --2- 172.21.15.161:0/3749520714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95080047d0 0x7f9508004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.309+0000 7f9529000640 1 --2- 172.21.15.161:0/3749520714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9508005190 0x7f950800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.309+0000 7f9523e00640 1 --2- 172.21.15.161:0/3749520714 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9508003980 0x7f9508003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.309+0000 7f951b400640 1 -- 172.21.15.161:0/3749520714 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9508003980 msgr2=0x7f9508003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:01.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.309+0000 7f951b400640 1 --2- 172.21.15.161:0/3749520714 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9508003980 0x7f9508003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.309+0000 7f951b400640 1 -- 172.21.15.161:0/3749520714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9508005190 msgr2=0x7f950800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:01.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.309+0000 7f951b400640 1 --2- 172.21.15.161:0/3749520714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9508005190 0x7f950800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.309+0000 7f951b400640 1 -- 172.21.15.161:0/3749520714 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9508003530 con 0x7f95080047d0 2024-05-22T21:14:01.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.309+0000 7f9529000640 1 --2- 172.21.15.161:0/3749520714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9508005190 0x7f950800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:01.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.309+0000 7f9529000640 1 -- 172.21.15.161:0/3749520714 reap_dead start 2024-05-22T21:14:01.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.309+0000 7f951b400640 1 --2- 172.21.15.161:0/3749520714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95080047d0 0x7f9508004c50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f950c001780 tx=0x7f950c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcff6be7d610beb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:01.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.309+0000 7f9523400640 1 -- 172.21.15.161:0/3749520714 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c0052d0 con 0x7f95080047d0 2024-05-22T21:14:01.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.309+0000 7f9523400640 1 -- 172.21.15.161:0/3749520714 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950c005450 con 0x7f95080047d0 2024-05-22T21:14:01.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.309+0000 7f9523400640 1 -- 172.21.15.161:0/3749520714 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c003840 con 0x7f95080047d0 2024-05-22T21:14:01.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f952b000640 1 -- 172.21.15.161:0/3749520714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95080047d0 msgr2=0x7f9508004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:01.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f952b000640 1 --2- 172.21.15.161:0/3749520714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95080047d0 0x7f9508004c50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f950c001780 tx=0x7f950c01c3d0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f952b000640 1 -- 172.21.15.161:0/3749520714 shutdown_connections 2024-05-22T21:14:01.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f952b000640 1 --2- 172.21.15.161:0/3749520714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95080047d0 0x7f9508004c50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f952b000640 1 -- 172.21.15.161:0/3749520714 >> 172.21.15.161:0/3749520714 conn(0x7f95240b1c40 msgr2=0x7f95240b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:01.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f952b000640 1 -- 172.21.15.161:0/3749520714 shutdown_connections 2024-05-22T21:14:01.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f952b000640 1 -- 172.21.15.161:0/3749520714 wait complete. 2024-05-22T21:14:01.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f952b000640 1 Processor -- start 2024-05-22T21:14:01.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f952b000640 1 -- start start 2024-05-22T21:14:01.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f952b000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95080047d0 0x7f952413d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f952b000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f952415a5b0 0x7f952413e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f952b000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f952413ec70 0x7f952413f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f952b000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9524139240 con 0x7f952415a5b0 2024-05-22T21:14:01.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f952b000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952415a3f0 con 0x7f952413ec70 2024-05-22T21:14:01.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f952b000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952413d550 con 0x7f95080047d0 2024-05-22T21:14:01.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f951b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f952415a5b0 0x7f952413e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f951b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f952415a5b0 0x7f952413e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59202/0 (socket says 172.21.15.161:59202) 2024-05-22T21:14:01.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f951b400640 1 -- 172.21.15.161:0/4058065017 learned_addr learned my addr 172.21.15.161:0/4058065017 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:01.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f9523e00640 1 --2- 172.21.15.161:0/4058065017 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95080047d0 0x7f952413d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f9529000640 1 --2- 172.21.15.161:0/4058065017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f952413ec70 0x7f952413f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f951b400640 1 -- 172.21.15.161:0/4058065017 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95080047d0 msgr2=0x7f952413d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:01.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f951b400640 1 --2- 172.21.15.161:0/4058065017 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95080047d0 0x7f952413d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f951b400640 1 -- 172.21.15.161:0/4058065017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f952413ec70 msgr2=0x7f952413f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:01.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f951b400640 1 --2- 172.21.15.161:0/4058065017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f952413ec70 0x7f952413f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f951b400640 1 -- 172.21.15.161:0/4058065017 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952413e070 con 0x7f952415a5b0 2024-05-22T21:14:01.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f951b400640 1 --2- 172.21.15.161:0/4058065017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f952415a5b0 0x7f952413e730 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f950c01c200 tx=0x7f9508005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:01.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.313+0000 7f9520c00640 1 -- 172.21.15.161:0/4058065017 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c007a40 con 0x7f952415a5b0 2024-05-22T21:14:01.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.317+0000 7f9520c00640 1 -- 172.21.15.161:0/4058065017 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950c0050d0 con 0x7f952415a5b0 2024-05-22T21:14:01.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.317+0000 7f9520c00640 1 -- 172.21.15.161:0/4058065017 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c0067e0 con 0x7f952415a5b0 2024-05-22T21:14:01.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.317+0000 7f952b000640 1 -- 172.21.15.161:0/4058065017 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952413d7d0 con 0x7f952415a5b0 2024-05-22T21:14:01.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.317+0000 7f952b000640 1 -- 172.21.15.161:0/4058065017 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952413dd90 con 0x7f952415a5b0 2024-05-22T21:14:01.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.317+0000 7f952b000640 1 -- 172.21.15.161:0/4058065017 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95241393b0 con 0x7f952415a5b0 2024-05-22T21:14:01.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.317+0000 7f9520c00640 1 -- 172.21.15.161:0/4058065017 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f950c006940 con 0x7f952415a5b0 2024-05-22T21:14:01.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.317+0000 7f9520c00640 1 --2- 172.21.15.161:0/4058065017 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9500066690 0x7f9500068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.317+0000 7f9520c00640 1 -- 172.21.15.161:0/4058065017 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f950c11deb0 con 0x7f952415a5b0 2024-05-22T21:14:01.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.317+0000 7f9523e00640 1 --2- 172.21.15.161:0/4058065017 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9500066690 0x7f9500068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.317+0000 7f9523e00640 1 --2- 172.21.15.161:0/4058065017 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9500066690 0x7f9500068b50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9514007520 tx=0x7f951400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:01.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.321+0000 7f9520c00640 1 -- 172.21.15.161:0/4058065017 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f95241393b0 con 0x7f952415a5b0 2024-05-22T21:14:01.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.477+0000 7f952b000640 1 -- 172.21.15.161:0/4058065017 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95241393b0 con 0x7f952415a5b0 2024-05-22T21:14:01.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.481+0000 7f9520c00640 1 -- 172.21.15.161:0/4058065017 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f95241393b0 con 0x7f952415a5b0 2024-05-22T21:14:01.484 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:01.484 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:01.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.481+0000 7f952b000640 1 -- 172.21.15.161:0/4058065017 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9500066690 msgr2=0x7f9500068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:01.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.481+0000 7f952b000640 1 --2- 172.21.15.161:0/4058065017 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9500066690 0x7f9500068b50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9514007520 tx=0x7f951400b040 comp rx=0 tx=0).stop 2024-05-22T21:14:01.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.481+0000 7f952b000640 1 -- 172.21.15.161:0/4058065017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f952415a5b0 msgr2=0x7f952413e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:01.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.481+0000 7f952b000640 1 --2- 172.21.15.161:0/4058065017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f952415a5b0 0x7f952413e730 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f950c01c200 tx=0x7f9508005800 comp rx=0 tx=0).stop 2024-05-22T21:14:01.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.481+0000 7f952b000640 1 -- 172.21.15.161:0/4058065017 shutdown_connections 2024-05-22T21:14:01.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.481+0000 7f952b000640 1 --2- 172.21.15.161:0/4058065017 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9500066690 0x7f9500068b50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.481+0000 7f952b000640 1 --2- 172.21.15.161:0/4058065017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f952413ec70 0x7f952413f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.481+0000 7f952b000640 1 --2- 172.21.15.161:0/4058065017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f952415a5b0 0x7f952413e730 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.481+0000 7f952b000640 1 --2- 172.21.15.161:0/4058065017 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95080047d0 0x7f952413d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.481+0000 7f952b000640 1 -- 172.21.15.161:0/4058065017 >> 172.21.15.161:0/4058065017 conn(0x7f95240b1c40 msgr2=0x7f952415bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:01.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.481+0000 7f952b000640 1 -- 172.21.15.161:0/4058065017 shutdown_connections 2024-05-22T21:14:01.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.481+0000 7f952b000640 1 -- 172.21.15.161:0/4058065017 wait complete. 2024-05-22T21:14:01.505 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:13:21.237503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:13:22.425891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:14:01.506 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:14:01.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.637+0000 7f9d50400640 1 Processor -- start 2024-05-22T21:14:01.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.637+0000 7f9d50400640 1 -- start start 2024-05-22T21:14:01.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.641+0000 7f9d50400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9d4806c750 0x7f9d480b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.641+0000 7f9d50400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d4806cbc0 con 0x7f9d4806c2a0 2024-05-22T21:14:01.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.641+0000 7f9d50400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d4815d9d0 con 0x7f9d481402b0 2024-05-22T21:14:01.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.641+0000 7f9d50400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d4815d7c0 con 0x7f9d4806c750 2024-05-22T21:14:01.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.641+0000 7f9d4e400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9d4806c750 0x7f9d480b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.641+0000 7f9d4e400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9d4806c750 0x7f9d480b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59216/0 (socket says 172.21.15.161:59216) 2024-05-22T21:14:01.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.641+0000 7f9d4e400640 1 -- 172.21.15.161:0/71788108 learned_addr learned my addr 172.21.15.161:0/71788108 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:01.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.641+0000 7f9d4e400640 1 -- 172.21.15.161:0/71788108 >> v1:172.21.15.179:6789/0 conn(0x7f9d481402b0 legacy=0x7f9d48070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:01.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.641+0000 7f9d4e400640 1 -- 172.21.15.161:0/71788108 >> v1:172.21.15.161:6789/0 conn(0x7f9d4806c2a0 legacy=0x7f9d480b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:01.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.641+0000 7f9d4e400640 1 -- 172.21.15.161:0/71788108 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4815dcb0 con 0x7f9d4806c750 2024-05-22T21:14:01.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.645+0000 7f9d4e400640 1 --2- 172.21.15.161:0/71788108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d4806c750 0x7f9d480b3a80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009870 tx=0x7f9d4001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb9e2404a82ca57 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:01.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.645+0000 7f9d47e00640 1 -- 172.21.15.161:0/71788108 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d400a7070 con 0x7f9d4806c750 2024-05-22T21:14:01.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.645+0000 7f9d47e00640 1 -- 172.21.15.161:0/71788108 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d400ab430 con 0x7f9d4806c750 2024-05-22T21:14:01.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.645+0000 7f9d47e00640 1 -- 172.21.15.161:0/71788108 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d400ab7c0 con 0x7f9d4806c750 2024-05-22T21:14:01.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.645+0000 7f9d50400640 1 -- 172.21.15.161:0/71788108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d4806c750 msgr2=0x7f9d480b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:01.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.645+0000 7f9d50400640 1 --2- 172.21.15.161:0/71788108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d4806c750 0x7f9d480b3a80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009870 tx=0x7f9d4001c520 comp rx=0 tx=0).stop 2024-05-22T21:14:01.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.645+0000 7f9d50400640 1 -- 172.21.15.161:0/71788108 shutdown_connections 2024-05-22T21:14:01.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.645+0000 7f9d50400640 1 --2- 172.21.15.161:0/71788108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d4806c750 0x7f9d480b3a80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.645+0000 7f9d50400640 1 -- 172.21.15.161:0/71788108 >> 172.21.15.161:0/71788108 conn(0x7f9d4806a2d0 msgr2=0x7f9d4806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:01.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.645+0000 7f9d50400640 1 -- 172.21.15.161:0/71788108 shutdown_connections 2024-05-22T21:14:01.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.645+0000 7f9d50400640 1 -- 172.21.15.161:0/71788108 wait complete. 2024-05-22T21:14:01.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.645+0000 7f9d50400640 1 Processor -- start 2024-05-22T21:14:01.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.645+0000 7f9d50400640 1 -- start start 2024-05-22T21:14:01.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.645+0000 7f9d50400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d4806c2a0 0x7f9d48160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d50400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d4806c750 0x7f9d48139180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d50400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d481402b0 0x7f9d48153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d50400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d48160950 con 0x7f9d4806c750 2024-05-22T21:14:01.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d50400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4814e030 con 0x7f9d481402b0 2024-05-22T21:14:01.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d50400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d48150630 con 0x7f9d4806c2a0 2024-05-22T21:14:01.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d4d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d4806c750 0x7f9d48139180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d4d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d4806c750 0x7f9d48139180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59218/0 (socket says 172.21.15.161:59218) 2024-05-22T21:14:01.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d4d000640 1 -- 172.21.15.161:0/3596313438 learned_addr learned my addr 172.21.15.161:0/3596313438 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:01.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d4da00640 1 --2- 172.21.15.161:0/3596313438 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d4806c2a0 0x7f9d48160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d4e400640 1 --2- 172.21.15.161:0/3596313438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d481402b0 0x7f9d48153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d4d000640 1 -- 172.21.15.161:0/3596313438 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d4806c2a0 msgr2=0x7f9d48160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:01.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d4d000640 1 --2- 172.21.15.161:0/3596313438 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d4806c2a0 0x7f9d48160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d4d000640 1 -- 172.21.15.161:0/3596313438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d481402b0 msgr2=0x7f9d48153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:01.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d4d000640 1 --2- 172.21.15.161:0/3596313438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d481402b0 0x7f9d48153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d4d000640 1 -- 172.21.15.161:0/3596313438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d400094d0 con 0x7f9d4806c750 2024-05-22T21:14:01.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d4da00640 1 --2- 172.21.15.161:0/3596313438 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d4806c2a0 0x7f9d48160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:01.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d4d000640 1 --2- 172.21.15.161:0/3596313438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d4806c750 0x7f9d48139180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9d3400c3c0 tx=0x7f9d3400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:01.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d45600640 1 -- 172.21.15.161:0/3596313438 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d340052d0 con 0x7f9d4806c750 2024-05-22T21:14:01.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d45600640 1 -- 172.21.15.161:0/3596313438 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d34005430 con 0x7f9d4806c750 2024-05-22T21:14:01.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d50400640 1 -- 172.21.15.161:0/3596313438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d48150910 con 0x7f9d4806c750 2024-05-22T21:14:01.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d45600640 1 -- 172.21.15.161:0/3596313438 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d34003840 con 0x7f9d4806c750 2024-05-22T21:14:01.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d50400640 1 -- 172.21.15.161:0/3596313438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d48150ed0 con 0x7f9d4806c750 2024-05-22T21:14:01.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d50400640 1 -- 172.21.15.161:0/3596313438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d48067220 con 0x7f9d4806c750 2024-05-22T21:14:01.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.649+0000 7f9d45600640 1 -- 172.21.15.161:0/3596313438 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9d340039a0 con 0x7f9d4806c750 2024-05-22T21:14:01.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.653+0000 7f9d45600640 1 --2- 172.21.15.161:0/3596313438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d28066690 0x7f9d28068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.653+0000 7f9d45600640 1 -- 172.21.15.161:0/3596313438 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d34083690 con 0x7f9d4806c750 2024-05-22T21:14:01.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.653+0000 7f9d4da00640 1 --2- 172.21.15.161:0/3596313438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d28066690 0x7f9d28068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.653+0000 7f9d4da00640 1 --2- 172.21.15.161:0/3596313438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d28066690 0x7f9d28068b50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0045c0 tx=0x7f9d3c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:01.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.653+0000 7f9d45600640 1 -- 172.21.15.161:0/3596313438 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9d34050a10 con 0x7f9d4806c750 2024-05-22T21:14:01.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.809+0000 7f9d50400640 1 -- 172.21.15.161:0/3596313438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9d48153990 con 0x7f9d4806c750 2024-05-22T21:14:01.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.809+0000 7f9d45600640 1 -- 172.21.15.161:0/3596313438 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d34054030 con 0x7f9d4806c750 2024-05-22T21:14:01.815 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:14:01.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.813+0000 7f9d50400640 1 -- 172.21.15.161:0/3596313438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d28066690 msgr2=0x7f9d28068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:01.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.813+0000 7f9d50400640 1 --2- 172.21.15.161:0/3596313438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d28066690 0x7f9d28068b50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0045c0 tx=0x7f9d3c009040 comp rx=0 tx=0).stop 2024-05-22T21:14:01.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.813+0000 7f9d50400640 1 -- 172.21.15.161:0/3596313438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d4806c750 msgr2=0x7f9d48139180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:01.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.813+0000 7f9d50400640 1 --2- 172.21.15.161:0/3596313438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d4806c750 0x7f9d48139180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9d3400c3c0 tx=0x7f9d3400c880 comp rx=0 tx=0).stop 2024-05-22T21:14:01.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.813+0000 7f9d50400640 1 -- 172.21.15.161:0/3596313438 shutdown_connections 2024-05-22T21:14:01.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.813+0000 7f9d50400640 1 --2- 172.21.15.161:0/3596313438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d28066690 0x7f9d28068b50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.813+0000 7f9d50400640 1 --2- 172.21.15.161:0/3596313438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d481402b0 0x7f9d48153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.813+0000 7f9d50400640 1 --2- 172.21.15.161:0/3596313438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d4806c750 0x7f9d48139180 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.813+0000 7f9d50400640 1 --2- 172.21.15.161:0/3596313438 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d4806c2a0 0x7f9d48160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.813+0000 7f9d50400640 1 -- 172.21.15.161:0/3596313438 >> 172.21.15.161:0/3596313438 conn(0x7f9d4806a2d0 msgr2=0x7f9d4815bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:01.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.813+0000 7f9d50400640 1 -- 172.21.15.161:0/3596313438 shutdown_connections 2024-05-22T21:14:01.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.813+0000 7f9d50400640 1 -- 172.21.15.161:0/3596313438 wait complete. 2024-05-22T21:14:01.837 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:14:01.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.969+0000 7f3f79400640 1 Processor -- start 2024-05-22T21:14:01.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.969+0000 7f3f79400640 1 -- start start 2024-05-22T21:14:01.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.969+0000 7f3f79400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f7413cfc0 0x7f3f741402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.969+0000 7f3f79400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3f7415af60 0x7f3f7413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.969+0000 7f3f79400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3f7413fc80 0x7f3f7415dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.969+0000 7f3f79400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f741400d0 con 0x7f3f7413fc80 2024-05-22T21:14:01.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.969+0000 7f3f79400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f7413d3a0 con 0x7f3f7415af60 2024-05-22T21:14:01.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.969+0000 7f3f79400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f740cd920 con 0x7f3f7413cfc0 2024-05-22T21:14:01.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.969+0000 7f3f73200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3f7413fc80 0x7f3f7415dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.969+0000 7f3f73200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3f7413fc80 0x7f3f7415dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59228/0 (socket says 172.21.15.161:59228) 2024-05-22T21:14:01.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.969+0000 7f3f73200640 1 -- 172.21.15.161:0/3080747993 learned_addr learned my addr 172.21.15.161:0/3080747993 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:01.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f71e00640 1 --2- 172.21.15.161:0/3080747993 >> v2:172.21.15.179:3300/0 conn(0x7f3f7415af60 0x7f3f7413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f72800640 1 --2- 172.21.15.161:0/3080747993 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f7413cfc0 0x7f3f741402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f73200640 1 -- 172.21.15.161:0/3080747993 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f7413cfc0 msgr2=0x7f3f741402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:01.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f73200640 1 --2- 172.21.15.161:0/3080747993 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f7413cfc0 0x7f3f741402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f73200640 1 -- 172.21.15.161:0/3080747993 >> v2:172.21.15.179:3300/0 conn(0x7f3f7415af60 msgr2=0x7f3f7413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:01.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f73200640 1 --2- 172.21.15.161:0/3080747993 >> v2:172.21.15.179:3300/0 conn(0x7f3f7415af60 0x7f3f7413f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f73200640 1 -- 172.21.15.161:0/3080747993 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f740cdc00 con 0x7f3f7413fc80 2024-05-22T21:14:01.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f72800640 1 --2- 172.21.15.161:0/3080747993 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f7413cfc0 0x7f3f741402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:01.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f71e00640 1 --2- 172.21.15.161:0/3080747993 >> v2:172.21.15.179:3300/0 conn(0x7f3f7415af60 0x7f3f7413f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:01.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f73200640 1 --2- 172.21.15.161:0/3080747993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f7413fc80 0x7f3f7415dad0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3f740723a0 tx=0x7f3f6c01c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffeccc1726e6ddf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:01.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f71400640 1 -- 172.21.15.161:0/3080747993 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0ab600 con 0x7f3f7413fc80 2024-05-22T21:14:01.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f71400640 1 -- 172.21.15.161:0/3080747993 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c0a7050 con 0x7f3f7413fc80 2024-05-22T21:14:01.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f71400640 1 -- 172.21.15.161:0/3080747993 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0ab980 con 0x7f3f7413fc80 2024-05-22T21:14:01.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f79400640 1 -- 172.21.15.161:0/3080747993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f7413fc80 msgr2=0x7f3f7415dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:01.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f79400640 1 --2- 172.21.15.161:0/3080747993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f7413fc80 0x7f3f7415dad0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3f740723a0 tx=0x7f3f6c01c1a0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f79400640 1 -- 172.21.15.161:0/3080747993 shutdown_connections 2024-05-22T21:14:01.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f79400640 1 --2- 172.21.15.161:0/3080747993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f7413fc80 0x7f3f7415dad0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f79400640 1 --2- 172.21.15.161:0/3080747993 >> v2:172.21.15.179:3300/0 conn(0x7f3f7415af60 0x7f3f7413f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f79400640 1 --2- 172.21.15.161:0/3080747993 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f7413cfc0 0x7f3f741402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f79400640 1 -- 172.21.15.161:0/3080747993 >> 172.21.15.161:0/3080747993 conn(0x7f3f7406a2d0 msgr2=0x7f3f7406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:01.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.973+0000 7f3f79400640 1 -- 172.21.15.161:0/3080747993 shutdown_connections 2024-05-22T21:14:01.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f79400640 1 -- 172.21.15.161:0/3080747993 wait complete. 2024-05-22T21:14:01.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f79400640 1 Processor -- start 2024-05-22T21:14:01.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f79400640 1 -- start start 2024-05-22T21:14:01.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f79400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f7413cfc0 0x7f3f74156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f79400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f7413fc80 0x7f3f7413bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f79400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f7415af60 0x7f3f7413c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f79400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7415d630 con 0x7f3f7415af60 2024-05-22T21:14:01.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f79400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7413c5b0 con 0x7f3f7413fc80 2024-05-22T21:14:01.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f79400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f74136820 con 0x7f3f7413cfc0 2024-05-22T21:14:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f73200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f7415af60 0x7f3f7413c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f73200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f7415af60 0x7f3f7413c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59244/0 (socket says 172.21.15.161:59244) 2024-05-22T21:14:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f73200640 1 -- 172.21.15.161:0/2832176680 learned_addr learned my addr 172.21.15.161:0/2832176680 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f72800640 1 --2- 172.21.15.161:0/2832176680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f7413cfc0 0x7f3f74156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f71e00640 1 --2- 172.21.15.161:0/2832176680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f7413fc80 0x7f3f7413bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f73200640 1 -- 172.21.15.161:0/2832176680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f7413cfc0 msgr2=0x7f3f74156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f73200640 1 --2- 172.21.15.161:0/2832176680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f7413cfc0 0x7f3f74156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f73200640 1 -- 172.21.15.161:0/2832176680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f7413fc80 msgr2=0x7f3f7413bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f73200640 1 --2- 172.21.15.161:0/2832176680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f7413fc80 0x7f3f7413bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f73200640 1 -- 172.21.15.161:0/2832176680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c0094d0 con 0x7f3f7415af60 2024-05-22T21:14:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f73200640 1 --2- 172.21.15.161:0/2832176680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f7415af60 0x7f3f7413c070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0ae040 tx=0x7f3f6c018f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f6aa00640 1 -- 172.21.15.161:0/2832176680 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0b46f0 con 0x7f3f7415af60 2024-05-22T21:14:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f6aa00640 1 -- 172.21.15.161:0/2832176680 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c0b4870 con 0x7f3f7415af60 2024-05-22T21:14:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f6aa00640 1 -- 172.21.15.161:0/2832176680 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0b4c20 con 0x7f3f7415af60 2024-05-22T21:14:01.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.977+0000 7f3f79400640 1 -- 172.21.15.161:0/2832176680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f7413a390 con 0x7f3f7415af60 2024-05-22T21:14:01.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.981+0000 7f3f79400640 1 -- 172.21.15.161:0/2832176680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f74136440 con 0x7f3f7415af60 2024-05-22T21:14:01.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.981+0000 7f3f79400640 1 -- 172.21.15.161:0/2832176680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f74136960 con 0x7f3f7415af60 2024-05-22T21:14:01.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.981+0000 7f3f6aa00640 1 -- 172.21.15.161:0/2832176680 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3f6c0b2040 con 0x7f3f7415af60 2024-05-22T21:14:01.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.981+0000 7f3f6aa00640 1 --2- 172.21.15.161:0/2832176680 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3f54066690 0x7f3f54068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:01.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.981+0000 7f3f6aa00640 1 -- 172.21.15.161:0/2832176680 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f6c11e090 con 0x7f3f7415af60 2024-05-22T21:14:01.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.981+0000 7f3f72800640 1 --2- 172.21.15.161:0/2832176680 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3f54066690 0x7f3f54068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:01.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.981+0000 7f3f72800640 1 --2- 172.21.15.161:0/2832176680 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3f54066690 0x7f3f54068b50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400f500 tx=0x7f3f64007770 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:01.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:01.985+0000 7f3f6aa00640 1 -- 172.21.15.161:0/2832176680 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3f74136960 con 0x7f3f7415af60 2024-05-22T21:14:02.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.137+0000 7f3f79400640 1 -- 172.21.15.161:0/2832176680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f74136960 con 0x7f3f7415af60 2024-05-22T21:14:02.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.137+0000 7f3f6aa00640 1 -- 172.21.15.161:0/2832176680 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3f74136960 con 0x7f3f7415af60 2024-05-22T21:14:02.144 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:14:02.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.141+0000 7f3f79400640 1 -- 172.21.15.161:0/2832176680 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3f54066690 msgr2=0x7f3f54068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:02.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.141+0000 7f3f79400640 1 --2- 172.21.15.161:0/2832176680 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3f54066690 0x7f3f54068b50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400f500 tx=0x7f3f64007770 comp rx=0 tx=0).stop 2024-05-22T21:14:02.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.141+0000 7f3f79400640 1 -- 172.21.15.161:0/2832176680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f7415af60 msgr2=0x7f3f7413c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:02.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.141+0000 7f3f79400640 1 --2- 172.21.15.161:0/2832176680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f7415af60 0x7f3f7413c070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0ae040 tx=0x7f3f6c018f40 comp rx=0 tx=0).stop 2024-05-22T21:14:02.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.141+0000 7f3f79400640 1 -- 172.21.15.161:0/2832176680 shutdown_connections 2024-05-22T21:14:02.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.141+0000 7f3f79400640 1 --2- 172.21.15.161:0/2832176680 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3f54066690 0x7f3f54068b50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.141+0000 7f3f79400640 1 --2- 172.21.15.161:0/2832176680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f7415af60 0x7f3f7413c070 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.141+0000 7f3f79400640 1 --2- 172.21.15.161:0/2832176680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f7413fc80 0x7f3f7413bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.141+0000 7f3f79400640 1 --2- 172.21.15.161:0/2832176680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f7413cfc0 0x7f3f74156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.141+0000 7f3f79400640 1 -- 172.21.15.161:0/2832176680 >> 172.21.15.161:0/2832176680 conn(0x7f3f7406a2d0 msgr2=0x7f3f74070680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:02.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.141+0000 7f3f79400640 1 -- 172.21.15.161:0/2832176680 shutdown_connections 2024-05-22T21:14:02.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.141+0000 7f3f79400640 1 -- 172.21.15.161:0/2832176680 wait complete. 2024-05-22T21:14:02.164 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:14:02.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.297+0000 7fa9f7a00640 1 Processor -- start 2024-05-22T21:14:02.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.297+0000 7fa9f7a00640 1 -- start start 2024-05-22T21:14:02.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.297+0000 7fa9f7a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa9f013cfc0 0x7fa9f01402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:02.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.297+0000 7fa9f7a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9f013fe50 0x7fa9f006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:02.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.297+0000 7fa9f7a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f013d3a0 con 0x7fa9f015af60 2024-05-22T21:14:02.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.297+0000 7fa9f7a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f00cd920 con 0x7fa9f013cfc0 2024-05-22T21:14:02.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.297+0000 7fa9f7a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00cdc00 con 0x7fa9f013fe50 2024-05-22T21:14:02.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.301+0000 7fa9efe00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa9f015af60 0x7fa9f013f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33752/0 (socket says 172.21.15.161:33752) 2024-05-22T21:14:02.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.301+0000 7fa9efe00640 1 -- 172.21.15.161:0/307299096 learned_addr learned my addr 172.21.15.161:0/307299096 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:02.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.301+0000 7fa9f5000640 1 --2- 172.21.15.161:0/307299096 >> v2:172.21.15.179:3300/0 conn(0x7fa9f013cfc0 0x7fa9f01402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:02.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.301+0000 7fa9f5a00640 1 --2- 172.21.15.161:0/307299096 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9f013fe50 0x7fa9f006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:02.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.301+0000 7fa9ef400640 1 -- 172.21.15.161:0/307299096 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567737606 0 0) 0x7fa9f013d3a0 con 0x7fa9f015af60 2024-05-22T21:14:02.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.301+0000 7fa9ef400640 1 -- 172.21.15.161:0/307299096 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9d4003530 con 0x7fa9f015af60 2024-05-22T21:14:02.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.301+0000 7fa9ef400640 1 -- 172.21.15.161:0/307299096 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3232641670 0 0) 0x7fa9d4003530 con 0x7fa9f015af60 2024-05-22T21:14:02.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.301+0000 7fa9f5a00640 1 -- 172.21.15.161:0/307299096 >> v2:172.21.15.179:3300/0 conn(0x7fa9f013cfc0 msgr2=0x7fa9f01402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:02.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.301+0000 7fa9f5a00640 1 --2- 172.21.15.161:0/307299096 >> v2:172.21.15.179:3300/0 conn(0x7fa9f013cfc0 0x7fa9f01402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.301+0000 7fa9f5a00640 1 -- 172.21.15.161:0/307299096 >> v1:172.21.15.161:6789/0 conn(0x7fa9f015af60 legacy=0x7fa9f013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:02.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.301+0000 7fa9f5a00640 1 -- 172.21.15.161:0/307299096 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f00cdee0 con 0x7fa9f013fe50 2024-05-22T21:14:02.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.301+0000 7fa9f5000640 1 --2- 172.21.15.161:0/307299096 >> v2:172.21.15.179:3300/0 conn(0x7fa9f013cfc0 0x7fa9f01402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:02.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.301+0000 7fa9f5a00640 1 --2- 172.21.15.161:0/307299096 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9f013fe50 0x7fa9f006fce0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80097c0 tx=0x7fa9e801c300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97ea685d5d2c6ae6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:02.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9ef400640 1 -- 172.21.15.161:0/307299096 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc001680 con 0x7fa9f013fe50 2024-05-22T21:14:02.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9ef400640 1 -- 172.21.15.161:0/307299096 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e80a4c20 con 0x7fa9f013fe50 2024-05-22T21:14:02.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9ef400640 1 -- 172.21.15.161:0/307299096 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80ac440 con 0x7fa9f013fe50 2024-05-22T21:14:02.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f7a00640 1 -- 172.21.15.161:0/307299096 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9f013fe50 msgr2=0x7fa9f006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:02.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f7a00640 1 --2- 172.21.15.161:0/307299096 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9f013fe50 0x7fa9f006fce0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80097c0 tx=0x7fa9e801c300 comp rx=0 tx=0).stop 2024-05-22T21:14:02.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f7a00640 1 -- 172.21.15.161:0/307299096 shutdown_connections 2024-05-22T21:14:02.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f7a00640 1 --2- 172.21.15.161:0/307299096 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9f013fe50 0x7fa9f006fce0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f7a00640 1 --2- 172.21.15.161:0/307299096 >> v2:172.21.15.179:3300/0 conn(0x7fa9f013cfc0 0x7fa9f01402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f7a00640 1 -- 172.21.15.161:0/307299096 >> 172.21.15.161:0/307299096 conn(0x7fa9f006a2d0 msgr2=0x7fa9f006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:02.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f7a00640 1 -- 172.21.15.161:0/307299096 shutdown_connections 2024-05-22T21:14:02.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f7a00640 1 -- 172.21.15.161:0/307299096 wait complete. 2024-05-22T21:14:02.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f7a00640 1 Processor -- start 2024-05-22T21:14:02.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f7a00640 1 -- start start 2024-05-22T21:14:02.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f7a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9f013cfc0 0x7fa9f0157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:02.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f7a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9f013fe50 0x7fa9f015d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:02.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f7a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9f015af60 0x7fa9f00b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f7a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f013bdb0 con 0x7fa9f013fe50 2024-05-22T21:14:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f7a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f013c1d0 con 0x7fa9f015af60 2024-05-22T21:14:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f7a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f015e1a0 con 0x7fa9f013cfc0 2024-05-22T21:14:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9efe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9f013fe50 0x7fa9f015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9efe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9f013fe50 0x7fa9f015d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59246/0 (socket says 172.21.15.161:59246) 2024-05-22T21:14:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9efe00640 1 -- 172.21.15.161:0/4085393818 learned_addr learned my addr 172.21.15.161:0/4085393818 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f5a00640 1 --2- 172.21.15.161:0/4085393818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9f015af60 0x7fa9f00b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.305+0000 7fa9f5000640 1 --2- 172.21.15.161:0/4085393818 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9f013cfc0 0x7fa9f0157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.309+0000 7fa9efe00640 1 -- 172.21.15.161:0/4085393818 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9f013cfc0 msgr2=0x7fa9f0157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.309+0000 7fa9efe00640 1 --2- 172.21.15.161:0/4085393818 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9f013cfc0 0x7fa9f0157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.309+0000 7fa9efe00640 1 -- 172.21.15.161:0/4085393818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9f015af60 msgr2=0x7fa9f00b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.309+0000 7fa9efe00640 1 --2- 172.21.15.161:0/4085393818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9f015af60 0x7fa9f00b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.309+0000 7fa9efe00640 1 -- 172.21.15.161:0/4085393818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e0009a70 con 0x7fa9f013fe50 2024-05-22T21:14:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.309+0000 7fa9efe00640 1 --2- 172.21.15.161:0/4085393818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9f013fe50 0x7fa9f015d020 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00c750 tx=0x7fa9dc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.309+0000 7fa9ecc00640 1 -- 172.21.15.161:0/4085393818 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc005ae0 con 0x7fa9f013fe50 2024-05-22T21:14:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.309+0000 7fa9ecc00640 1 -- 172.21.15.161:0/4085393818 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc005c40 con 0x7fa9f013fe50 2024-05-22T21:14:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.309+0000 7fa9f7a00640 1 -- 172.21.15.161:0/4085393818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e8009420 con 0x7fa9f013fe50 2024-05-22T21:14:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.309+0000 7fa9ecc00640 1 -- 172.21.15.161:0/4085393818 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc007070 con 0x7fa9f013fe50 2024-05-22T21:14:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.309+0000 7fa9f7a00640 1 -- 172.21.15.161:0/4085393818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f015dbe0 con 0x7fa9f013fe50 2024-05-22T21:14:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.309+0000 7fa9f7a00640 1 -- 172.21.15.161:0/4085393818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f013bef0 con 0x7fa9f013fe50 2024-05-22T21:14:02.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.309+0000 7fa9ecc00640 1 -- 172.21.15.161:0/4085393818 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa9dc0071f0 con 0x7fa9f013fe50 2024-05-22T21:14:02.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.309+0000 7fa9ecc00640 1 --2- 172.21.15.161:0/4085393818 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa9d0066690 0x7fa9d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:02.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.309+0000 7fa9ecc00640 1 -- 172.21.15.161:0/4085393818 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9dc083ea0 con 0x7fa9f013fe50 2024-05-22T21:14:02.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.309+0000 7fa9f5000640 1 --2- 172.21.15.161:0/4085393818 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa9d0066690 0x7fa9d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:02.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.313+0000 7fa9f5000640 1 --2- 172.21.15.161:0/4085393818 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa9d0066690 0x7fa9d0068b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0007600 tx=0x7fa9e000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:02.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.313+0000 7fa9ecc00640 1 -- 172.21.15.161:0/4085393818 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa9f013bef0 con 0x7fa9f013fe50 2024-05-22T21:14:02.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.469+0000 7fa9f7a00640 1 -- 172.21.15.161:0/4085393818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa9f013bef0 con 0x7fa9f013fe50 2024-05-22T21:14:02.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.469+0000 7fa9ecc00640 1 -- 172.21.15.161:0/4085393818 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa9f013bef0 con 0x7fa9f013fe50 2024-05-22T21:14:02.474 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:14:02.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.473+0000 7fa9f7a00640 1 -- 172.21.15.161:0/4085393818 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa9d0066690 msgr2=0x7fa9d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:02.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.473+0000 7fa9f7a00640 1 --2- 172.21.15.161:0/4085393818 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa9d0066690 0x7fa9d0068b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0007600 tx=0x7fa9e000b040 comp rx=0 tx=0).stop 2024-05-22T21:14:02.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.473+0000 7fa9f7a00640 1 -- 172.21.15.161:0/4085393818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9f013fe50 msgr2=0x7fa9f015d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:02.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.473+0000 7fa9f7a00640 1 --2- 172.21.15.161:0/4085393818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9f013fe50 0x7fa9f015d020 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00c750 tx=0x7fa9dc00cc10 comp rx=0 tx=0).stop 2024-05-22T21:14:02.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.473+0000 7fa9f7a00640 1 -- 172.21.15.161:0/4085393818 shutdown_connections 2024-05-22T21:14:02.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.473+0000 7fa9f7a00640 1 --2- 172.21.15.161:0/4085393818 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa9d0066690 0x7fa9d0068b50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.473+0000 7fa9f7a00640 1 --2- 172.21.15.161:0/4085393818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9f015af60 0x7fa9f00b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.473+0000 7fa9f7a00640 1 --2- 172.21.15.161:0/4085393818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9f013fe50 0x7fa9f015d020 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.473+0000 7fa9f7a00640 1 --2- 172.21.15.161:0/4085393818 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9f013cfc0 0x7fa9f0157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.473+0000 7fa9f7a00640 1 -- 172.21.15.161:0/4085393818 >> 172.21.15.161:0/4085393818 conn(0x7fa9f006a2d0 msgr2=0x7fa9f006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:02.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.473+0000 7fa9f7a00640 1 -- 172.21.15.161:0/4085393818 shutdown_connections 2024-05-22T21:14:02.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.473+0000 7fa9f7a00640 1 -- 172.21.15.161:0/4085393818 wait complete. 2024-05-22T21:14:02.495 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:14:02.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.629+0000 7f90b4800640 1 Processor -- start 2024-05-22T21:14:02.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.629+0000 7f90b4800640 1 -- start start 2024-05-22T21:14:02.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.629+0000 7f90b4800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f90ac13fe50 0x7f90ac164e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:02.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.629+0000 7f90b4800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90ac13d3a0 con 0x7f90ac1402e0 2024-05-22T21:14:02.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.629+0000 7f90b4800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90ac0cd920 con 0x7f90ac13cfc0 2024-05-22T21:14:02.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.629+0000 7f90b4800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90ac0cdc00 con 0x7f90ac13fe50 2024-05-22T21:14:02.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.629+0000 7f90b1400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f90ac1402e0 0x7f90ac13f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33756/0 (socket says 172.21.15.161:33756) 2024-05-22T21:14:02.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.629+0000 7f90b1400640 1 -- 172.21.15.161:0/1490477114 learned_addr learned my addr 172.21.15.161:0/1490477114 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:02.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.629+0000 7f90b2800640 1 --2- 172.21.15.161:0/1490477114 >> v2:172.21.15.161:3300/0 conn(0x7f90ac13fe50 0x7f90ac164e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:02.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.629+0000 7f90b2800640 1 -- 172.21.15.161:0/1490477114 >> v1:172.21.15.179:6789/0 conn(0x7f90ac13cfc0 legacy=0x7f90ac15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:02.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.629+0000 7f90b2800640 1 -- 172.21.15.161:0/1490477114 >> v1:172.21.15.161:6789/0 conn(0x7f90ac1402e0 legacy=0x7f90ac13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:02.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.629+0000 7f90b2800640 1 -- 172.21.15.161:0/1490477114 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ac0cdee0 con 0x7f90ac13fe50 2024-05-22T21:14:02.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.633+0000 7f90b2800640 1 --2- 172.21.15.161:0/1490477114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90ac13fe50 0x7f90ac164e60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009d90 tx=0x7f90a801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5615b0527b808e3f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:02.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.633+0000 7f90b0a00640 1 -- 172.21.15.161:0/1490477114 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a80a57a0 con 0x7f90ac13fe50 2024-05-22T21:14:02.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.633+0000 7f90b0a00640 1 -- 172.21.15.161:0/1490477114 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a80a5900 con 0x7f90ac13fe50 2024-05-22T21:14:02.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.633+0000 7f90b0a00640 1 -- 172.21.15.161:0/1490477114 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a80a5c90 con 0x7f90ac13fe50 2024-05-22T21:14:02.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.633+0000 7f90b4800640 1 -- 172.21.15.161:0/1490477114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90ac13fe50 msgr2=0x7f90ac164e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:02.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.633+0000 7f90b4800640 1 --2- 172.21.15.161:0/1490477114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90ac13fe50 0x7f90ac164e60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009d90 tx=0x7f90a801c8e0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.633+0000 7f90b4800640 1 -- 172.21.15.161:0/1490477114 shutdown_connections 2024-05-22T21:14:02.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.633+0000 7f90b4800640 1 --2- 172.21.15.161:0/1490477114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90ac13fe50 0x7f90ac164e60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.633+0000 7f90b4800640 1 -- 172.21.15.161:0/1490477114 >> 172.21.15.161:0/1490477114 conn(0x7f90ac06a2d0 msgr2=0x7f90ac06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:02.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.633+0000 7f90b4800640 1 -- 172.21.15.161:0/1490477114 shutdown_connections 2024-05-22T21:14:02.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.633+0000 7f90b4800640 1 -- 172.21.15.161:0/1490477114 wait complete. 2024-05-22T21:14:02.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b4800640 1 Processor -- start 2024-05-22T21:14:02.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b4800640 1 -- start start 2024-05-22T21:14:02.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b4800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90ac13cfc0 0x7f90ac13c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:02.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b4800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90ac13fe50 0x7f90ac156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:02.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b4800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90ac1402e0 0x7f90ac139510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:02.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b4800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac13c150 con 0x7f90ac13fe50 2024-05-22T21:14:02.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b4800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac156dd0 con 0x7f90ac13cfc0 2024-05-22T21:14:02.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b4800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac0ae300 con 0x7f90ac1402e0 2024-05-22T21:14:02.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b1400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90ac13fe50 0x7f90ac156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:02.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b1400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90ac13fe50 0x7f90ac156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59266/0 (socket says 172.21.15.161:59266) 2024-05-22T21:14:02.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b1400640 1 -- 172.21.15.161:0/266653986 learned_addr learned my addr 172.21.15.161:0/266653986 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:02.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b1e00640 1 --2- 172.21.15.161:0/266653986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90ac13cfc0 0x7f90ac13c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:02.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b2800640 1 --2- 172.21.15.161:0/266653986 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90ac1402e0 0x7f90ac139510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:02.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b1400640 1 -- 172.21.15.161:0/266653986 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90ac1402e0 msgr2=0x7f90ac139510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:02.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b1400640 1 --2- 172.21.15.161:0/266653986 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90ac1402e0 0x7f90ac139510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b1400640 1 -- 172.21.15.161:0/266653986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90ac13cfc0 msgr2=0x7f90ac13c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:02.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b1400640 1 --2- 172.21.15.161:0/266653986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90ac13cfc0 0x7f90ac13c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b1400640 1 -- 172.21.15.161:0/266653986 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a8009a70 con 0x7f90ac13fe50 2024-05-22T21:14:02.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b2800640 1 --2- 172.21.15.161:0/266653986 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90ac1402e0 0x7f90ac139510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:02.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b1400640 1 --2- 172.21.15.161:0/266653986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90ac13fe50 0x7f90ac156890 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f909c00c530 tx=0x7f909c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:02.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90a6000640 1 -- 172.21.15.161:0/266653986 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c005cf0 con 0x7f90ac13fe50 2024-05-22T21:14:02.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90a6000640 1 -- 172.21.15.161:0/266653986 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909c005e50 con 0x7f90ac13fe50 2024-05-22T21:14:02.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90a6000640 1 -- 172.21.15.161:0/266653986 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c007270 con 0x7f90ac13fe50 2024-05-22T21:14:02.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b4800640 1 -- 172.21.15.161:0/266653986 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90ac0af0c0 con 0x7f90ac13fe50 2024-05-22T21:14:02.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b4800640 1 -- 172.21.15.161:0/266653986 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90ac13bd70 con 0x7f90ac13fe50 2024-05-22T21:14:02.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.637+0000 7f90b4800640 1 -- 172.21.15.161:0/266653986 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90ac13bbd0 con 0x7f90ac13fe50 2024-05-22T21:14:02.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.641+0000 7f90a6000640 1 -- 172.21.15.161:0/266653986 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f909c0073d0 con 0x7f90ac13fe50 2024-05-22T21:14:02.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.641+0000 7f90a6000640 1 --2- 172.21.15.161:0/266653986 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9090066690 0x7f9090068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:02.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.641+0000 7f90a6000640 1 -- 172.21.15.161:0/266653986 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f909c0835a0 con 0x7f90ac13fe50 2024-05-22T21:14:02.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.641+0000 7f90b1e00640 1 --2- 172.21.15.161:0/266653986 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9090066690 0x7f9090068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:02.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.641+0000 7f90b1e00640 1 --2- 172.21.15.161:0/266653986 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9090066690 0x7f9090068b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f90a0000f70 tx=0x7f90a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:02.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.645+0000 7f90a6000640 1 -- 172.21.15.161:0/266653986 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f90ac13bbd0 con 0x7f90ac13fe50 2024-05-22T21:14:02.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.809+0000 7f90b4800640 1 -- 172.21.15.161:0/266653986 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f90ac13c290 con 0x7f90ac13fe50 2024-05-22T21:14:02.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.809+0000 7f90a6000640 1 -- 172.21.15.161:0/266653986 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f90ac13c290 con 0x7f90ac13fe50 2024-05-22T21:14:02.813 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:14:02.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.809+0000 7f90b4800640 1 -- 172.21.15.161:0/266653986 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9090066690 msgr2=0x7f9090068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:02.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.809+0000 7f90b4800640 1 --2- 172.21.15.161:0/266653986 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9090066690 0x7f9090068b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f90a0000f70 tx=0x7f90a0009040 comp rx=0 tx=0).stop 2024-05-22T21:14:02.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.809+0000 7f90b4800640 1 -- 172.21.15.161:0/266653986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90ac13fe50 msgr2=0x7f90ac156890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:02.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.809+0000 7f90b4800640 1 --2- 172.21.15.161:0/266653986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90ac13fe50 0x7f90ac156890 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f909c00c530 tx=0x7f909c00c9f0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.809+0000 7f90b4800640 1 -- 172.21.15.161:0/266653986 shutdown_connections 2024-05-22T21:14:02.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.809+0000 7f90b4800640 1 --2- 172.21.15.161:0/266653986 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9090066690 0x7f9090068b50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.809+0000 7f90b4800640 1 --2- 172.21.15.161:0/266653986 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90ac1402e0 0x7f90ac139510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.809+0000 7f90b4800640 1 --2- 172.21.15.161:0/266653986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90ac13fe50 0x7f90ac156890 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.809+0000 7f90b4800640 1 --2- 172.21.15.161:0/266653986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90ac13cfc0 0x7f90ac13c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:02.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.809+0000 7f90b4800640 1 -- 172.21.15.161:0/266653986 >> 172.21.15.161:0/266653986 conn(0x7f90ac06a2d0 msgr2=0x7f90ac070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:02.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.809+0000 7f90b4800640 1 -- 172.21.15.161:0/266653986 shutdown_connections 2024-05-22T21:14:02.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:02.809+0000 7f90b4800640 1 -- 172.21.15.161:0/266653986 wait complete. 2024-05-22T21:14:02.833 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-05-22T21:14:02.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:02.833 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-05-22T21:14:02.855 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:14:02.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:02.855 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-22T21:14:02.908 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:14:02.908 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:02.917 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:14:02.917 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:02.966 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:14:02.966 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:03.019 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:03.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.149+0000 7f3ad2c00640 1 Processor -- start 2024-05-22T21:14:03.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.149+0000 7f3ad2c00640 1 -- start start 2024-05-22T21:14:03.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3ad2c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3acc15dd50 0x7f3acc06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3ad2c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc15e130 0x7f3acc15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3ad2c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3acc071670 con 0x7f3acc15d410 2024-05-22T21:14:03.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3ad2c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3acc15d8a0 con 0x7f3acc15dd50 2024-05-22T21:14:03.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3ad2c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc06de40 con 0x7f3acc15e130 2024-05-22T21:14:03.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3ad0c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3acc15d410 0x7f3acc15a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33766/0 (socket says 172.21.15.161:33766) 2024-05-22T21:14:03.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3ad0c00640 1 -- 172.21.15.161:0/3752953159 learned_addr learned my addr 172.21.15.161:0/3752953159 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:03.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acbe00640 1 --2- 172.21.15.161:0/3752953159 >> v2:172.21.15.179:3300/0 conn(0x7f3acc15dd50 0x7f3acc06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3ac3400640 1 --2- 172.21.15.161:0/3752953159 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc15e130 0x7f3acc15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acb400640 1 -- 172.21.15.161:0/3752953159 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3166234337 0 0) 0x7f3acc071670 con 0x7f3acc15d410 2024-05-22T21:14:03.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acb400640 1 -- 172.21.15.161:0/3752953159 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ab0005740 con 0x7f3acc15d410 2024-05-22T21:14:03.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acb400640 1 -- 172.21.15.161:0/3752953159 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1901045706 0 0) 0x7f3ab0005740 con 0x7f3acc15d410 2024-05-22T21:14:03.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acb400640 1 -- 172.21.15.161:0/3752953159 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc15e130 msgr2=0x7f3acc15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:03.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acb400640 1 --2- 172.21.15.161:0/3752953159 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc15e130 0x7f3acc15ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acb400640 1 -- 172.21.15.161:0/3752953159 >> v2:172.21.15.179:3300/0 conn(0x7f3acc15dd50 msgr2=0x7f3acc06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:03.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acb400640 1 --2- 172.21.15.161:0/3752953159 >> v2:172.21.15.179:3300/0 conn(0x7f3acc15dd50 0x7f3acc06d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acb400640 1 -- 172.21.15.161:0/3752953159 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc070a60 con 0x7f3acc15d410 2024-05-22T21:14:03.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acb400640 1 -- 172.21.15.161:0/3752953159 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f3ac4001a00 con 0x7f3acc15d410 2024-05-22T21:14:03.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acb400640 1 -- 172.21.15.161:0/3752953159 >> v1:172.21.15.161:6789/0 conn(0x7f3acc15d410 legacy=0x7f3acc15a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acb400640 1 --2- 172.21.15.161:0/3752953159 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc15e130 0x7f3ab0005b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acb400640 1 --2- 172.21.15.161:0/3752953159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab0006110 0x7f3ab00069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acb400640 1 --2- 172.21.15.161:0/3752953159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab0006f20 0x7f3ab00073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acb400640 1 -- 172.21.15.161:0/3752953159 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc15d8a0 con 0x7f3ab0006110 2024-05-22T21:14:03.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acb400640 1 -- 172.21.15.161:0/3752953159 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc06de40 con 0x7f3ab0006f20 2024-05-22T21:14:03.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acb400640 1 -- 172.21.15.161:0/3752953159 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab000dbf0 con 0x7f3acc15e130 2024-05-22T21:14:03.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acbe00640 1 --2- 172.21.15.161:0/3752953159 >> v2:172.21.15.179:3300/0 conn(0x7f3acc15dd50 0x7f3acc06d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:03.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3ac3400640 1 --2- 172.21.15.161:0/3752953159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab0006110 0x7f3ab00069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3ad0c00640 1 --2- 172.21.15.161:0/3752953159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab0006f20 0x7f3ab00073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3acbe00640 1 --2- 172.21.15.161:0/3752953159 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc15e130 0x7f3ab0005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3ac3400640 1 -- 172.21.15.161:0/3752953159 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc15e130 msgr2=0x7f3ab0005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:03.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3ac3400640 1 --2- 172.21.15.161:0/3752953159 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc15e130 0x7f3ab0005b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3ac3400640 1 -- 172.21.15.161:0/3752953159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab0006f20 msgr2=0x7f3ab00073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:03.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3ac3400640 1 --2- 172.21.15.161:0/3752953159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab0006f20 0x7f3ab00073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.153+0000 7f3ac3400640 1 -- 172.21.15.161:0/3752953159 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab0005740 con 0x7f3ab0006110 2024-05-22T21:14:03.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.157+0000 7f3ac3400640 1 --2- 172.21.15.161:0/3752953159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab0006110 0x7f3ab00069e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009b10 tx=0x7f3ab401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff907fb8d1246ec2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:03.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.157+0000 7f3acb400640 1 -- 172.21.15.161:0/3752953159 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab40a5520 con 0x7f3ab0006110 2024-05-22T21:14:03.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.157+0000 7f3acb400640 1 -- 172.21.15.161:0/3752953159 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab40a5680 con 0x7f3ab0006110 2024-05-22T21:14:03.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.157+0000 7f3acb400640 1 -- 172.21.15.161:0/3752953159 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab40a5a30 con 0x7f3ab0006110 2024-05-22T21:14:03.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.157+0000 7f3ad2c00640 1 -- 172.21.15.161:0/3752953159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab0006110 msgr2=0x7f3ab00069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.157+0000 7f3ad2c00640 1 --2- 172.21.15.161:0/3752953159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab0006110 0x7f3ab00069e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009b10 tx=0x7f3ab401ca40 comp rx=0 tx=0).stop 2024-05-22T21:14:03.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.157+0000 7f3ad0c00640 1 -- 172.21.15.161:0/3752953159 reap_dead start 2024-05-22T21:14:03.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.157+0000 7f3ad2c00640 1 -- 172.21.15.161:0/3752953159 shutdown_connections 2024-05-22T21:14:03.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.157+0000 7f3ad2c00640 1 -- 172.21.15.161:0/3752953159 >> 172.21.15.161:0/3752953159 conn(0x7f3acc0b0bc0 msgr2=0x7f3acc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:03.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.157+0000 7f3ad2c00640 1 -- 172.21.15.161:0/3752953159 shutdown_connections 2024-05-22T21:14:03.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.157+0000 7f3ad2c00640 1 -- 172.21.15.161:0/3752953159 wait complete. 2024-05-22T21:14:03.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad2c00640 1 Processor -- start 2024-05-22T21:14:03.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad2c00640 1 -- start start 2024-05-22T21:14:03.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad2c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab0006110 0x7f3acc14d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad2c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc14db90 0x7f3acc160570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad2c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3acc14e200 0x7f3acc14e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad2c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc15f730 con 0x7f3acc14e200 2024-05-22T21:14:03.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad2c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc15fb50 con 0x7f3ab0006110 2024-05-22T21:14:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad2c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc14ebc0 con 0x7f3acc14db90 2024-05-22T21:14:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad0c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3acc14e200 0x7f3acc14e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad0c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3acc14e200 0x7f3acc14e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59288/0 (socket says 172.21.15.161:59288) 2024-05-22T21:14:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad0c00640 1 -- 172.21.15.161:0/3912283348 learned_addr learned my addr 172.21.15.161:0/3912283348 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3acbe00640 1 --2- 172.21.15.161:0/3912283348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab0006110 0x7f3acc14d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ac3400640 1 --2- 172.21.15.161:0/3912283348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc14db90 0x7f3acc160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad0c00640 1 -- 172.21.15.161:0/3912283348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc14db90 msgr2=0x7f3acc160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad0c00640 1 --2- 172.21.15.161:0/3912283348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc14db90 0x7f3acc160570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad0c00640 1 -- 172.21.15.161:0/3912283348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab0006110 msgr2=0x7f3acc14d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad0c00640 1 --2- 172.21.15.161:0/3912283348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab0006110 0x7f3acc14d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad0c00640 1 -- 172.21.15.161:0/3912283348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc15eab0 con 0x7f3acc14e200 2024-05-22T21:14:03.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ac3400640 1 --2- 172.21.15.161:0/3912283348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc14db90 0x7f3acc160570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:03.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3acbe00640 1 --2- 172.21.15.161:0/3912283348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab0006110 0x7f3acc14d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:03.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad0c00640 1 --2- 172.21.15.161:0/3912283348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3acc14e200 0x7f3acc14e680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3acc06df80 tx=0x7f3ac4007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:03.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ac8c00640 1 -- 172.21.15.161:0/3912283348 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac400ab60 con 0x7f3acc14e200 2024-05-22T21:14:03.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ac8c00640 1 -- 172.21.15.161:0/3912283348 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac400ad00 con 0x7f3acc14e200 2024-05-22T21:14:03.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ac8c00640 1 -- 172.21.15.161:0/3912283348 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4013630 con 0x7f3acc14e200 2024-05-22T21:14:03.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad2c00640 1 -- 172.21.15.161:0/3912283348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3acc15ed90 con 0x7f3acc14e200 2024-05-22T21:14:03.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad2c00640 1 -- 172.21.15.161:0/3912283348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3acc15f350 con 0x7f3acc14e200 2024-05-22T21:14:03.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.161+0000 7f3ad2c00640 1 -- 172.21.15.161:0/3912283348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3acc15f870 con 0x7f3acc14e200 2024-05-22T21:14:03.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.165+0000 7f3ac8c00640 1 -- 172.21.15.161:0/3912283348 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3ac4013790 con 0x7f3acc14e200 2024-05-22T21:14:03.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.165+0000 7f3ac8c00640 1 --2- 172.21.15.161:0/3912283348 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3aa80666c0 0x7f3aa8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.165+0000 7f3ac8c00640 1 -- 172.21.15.161:0/3912283348 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ac4084800 con 0x7f3acc14e200 2024-05-22T21:14:03.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.165+0000 7f3acbe00640 1 --2- 172.21.15.161:0/3912283348 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3aa80666c0 0x7f3aa8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.165+0000 7f3acbe00640 1 --2- 172.21.15.161:0/3912283348 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3aa80666c0 0x7f3aa8068b80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009d00 tx=0x7f3abc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:03.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.169+0000 7f3ac8c00640 1 -- 172.21.15.161:0/3912283348 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3acc15f870 con 0x7f3acc14e200 2024-05-22T21:14:03.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.321+0000 7f3ad2c00640 1 -- 172.21.15.161:0/3912283348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3acc15f870 con 0x7f3acc14e200 2024-05-22T21:14:03.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.321+0000 7f3ac8c00640 1 -- 172.21.15.161:0/3912283348 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f3acc15f870 con 0x7f3acc14e200 2024-05-22T21:14:03.328 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:03.328 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:03.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.325+0000 7f3ad2c00640 1 -- 172.21.15.161:0/3912283348 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3aa80666c0 msgr2=0x7f3aa8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.325+0000 7f3ad2c00640 1 --2- 172.21.15.161:0/3912283348 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3aa80666c0 0x7f3aa8068b80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009d00 tx=0x7f3abc008040 comp rx=0 tx=0).stop 2024-05-22T21:14:03.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.325+0000 7f3ad2c00640 1 -- 172.21.15.161:0/3912283348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3acc14e200 msgr2=0x7f3acc14e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.325+0000 7f3ad2c00640 1 --2- 172.21.15.161:0/3912283348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3acc14e200 0x7f3acc14e680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3acc06df80 tx=0x7f3ac4007a90 comp rx=0 tx=0).stop 2024-05-22T21:14:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.325+0000 7f3ad2c00640 1 -- 172.21.15.161:0/3912283348 shutdown_connections 2024-05-22T21:14:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.325+0000 7f3ad2c00640 1 --2- 172.21.15.161:0/3912283348 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3aa80666c0 0x7f3aa8068b80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.325+0000 7f3ad2c00640 1 --2- 172.21.15.161:0/3912283348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3acc14e200 0x7f3acc14e680 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.325+0000 7f3ad2c00640 1 --2- 172.21.15.161:0/3912283348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc14db90 0x7f3acc160570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.325+0000 7f3ad2c00640 1 --2- 172.21.15.161:0/3912283348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ab0006110 0x7f3acc14d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.325+0000 7f3ad2c00640 1 -- 172.21.15.161:0/3912283348 >> 172.21.15.161:0/3912283348 conn(0x7f3acc0b0bc0 msgr2=0x7f3acc0afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.325+0000 7f3ad2c00640 1 -- 172.21.15.161:0/3912283348 shutdown_connections 2024-05-22T21:14:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.325+0000 7f3ad2c00640 1 -- 172.21.15.161:0/3912283348 wait complete. 2024-05-22T21:14:03.348 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:13:21.237503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:13:22.425891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:14:03.349 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:14:03.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.485+0000 7fc08b600640 1 Processor -- start 2024-05-22T21:14:03.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.485+0000 7fc08b600640 1 -- start start 2024-05-22T21:14:03.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.485+0000 7fc08b600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc084143c40 0x7fc08415b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.485+0000 7fc08b600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08415efb0 con 0x7fc0841444a0 2024-05-22T21:14:03.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.485+0000 7fc08b600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08415f290 con 0x7fc08406c9b0 2024-05-22T21:14:03.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.485+0000 7fc08b600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc08415f570 con 0x7fc084143c40 2024-05-22T21:14:03.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.489+0000 7fc088c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc084143c40 0x7fc08415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.489+0000 7fc088c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc084143c40 0x7fc08415b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59296/0 (socket says 172.21.15.161:59296) 2024-05-22T21:14:03.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.489+0000 7fc088c00640 1 -- 172.21.15.161:0/1953564346 learned_addr learned my addr 172.21.15.161:0/1953564346 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:03.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.489+0000 7fc088c00640 1 -- 172.21.15.161:0/1953564346 >> v1:172.21.15.179:6789/0 conn(0x7fc08406c9b0 legacy=0x7fc0840b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.489+0000 7fc088c00640 1 -- 172.21.15.161:0/1953564346 >> v1:172.21.15.161:6789/0 conn(0x7fc0841444a0 legacy=0x7fc08406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.489+0000 7fc088c00640 1 -- 172.21.15.161:0/1953564346 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08415f850 con 0x7fc084143c40 2024-05-22T21:14:03.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.489+0000 7fc083400640 1 -- 172.21.15.161:0/1953564346 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287922952 0 0) 0x7fc08415efb0 con 0x7fc0841444a0 2024-05-22T21:14:03.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.489+0000 7fc088c00640 1 --2- 172.21.15.161:0/1953564346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc084143c40 0x7fc08415b600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc074009870 tx=0x7fc07401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab00dd86df9eda9f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:03.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.489+0000 7fc083400640 1 -- 172.21.15.161:0/1953564346 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0740aa620 con 0x7fc084143c40 2024-05-22T21:14:03.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.489+0000 7fc083400640 1 -- 172.21.15.161:0/1953564346 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0740aa780 con 0x7fc084143c40 2024-05-22T21:14:03.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.489+0000 7fc083400640 1 -- 172.21.15.161:0/1953564346 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0740aab10 con 0x7fc084143c40 2024-05-22T21:14:03.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc08b600640 1 -- 172.21.15.161:0/1953564346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc084143c40 msgr2=0x7fc08415b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc08b600640 1 --2- 172.21.15.161:0/1953564346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc084143c40 0x7fc08415b600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc074009870 tx=0x7fc07401c560 comp rx=0 tx=0).stop 2024-05-22T21:14:03.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc08b600640 1 -- 172.21.15.161:0/1953564346 shutdown_connections 2024-05-22T21:14:03.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc08b600640 1 --2- 172.21.15.161:0/1953564346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc084143c40 0x7fc08415b600 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc08b600640 1 -- 172.21.15.161:0/1953564346 >> 172.21.15.161:0/1953564346 conn(0x7fc08406a2d0 msgr2=0x7fc08406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:03.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc08b600640 1 -- 172.21.15.161:0/1953564346 shutdown_connections 2024-05-22T21:14:03.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc08b600640 1 -- 172.21.15.161:0/1953564346 wait complete. 2024-05-22T21:14:03.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc08b600640 1 Processor -- start 2024-05-22T21:14:03.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc08b600640 1 -- start start 2024-05-22T21:14:03.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc08b600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc08406c9b0 0x7fc084139590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc08b600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc084143c40 0x7fc084139010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc08b600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc0841444a0 0x7fc08413eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc08b600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc084152630 con 0x7fc084143c40 2024-05-22T21:14:03.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc08b600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc084071320 con 0x7fc0841444a0 2024-05-22T21:14:03.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc08b600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc084071600 con 0x7fc08406c9b0 2024-05-22T21:14:03.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc083e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc084143c40 0x7fc084139010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc083e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc084143c40 0x7fc084139010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59310/0 (socket says 172.21.15.161:59310) 2024-05-22T21:14:03.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc083e00640 1 -- 172.21.15.161:0/3338107514 learned_addr learned my addr 172.21.15.161:0/3338107514 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:03.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc088c00640 1 --2- 172.21.15.161:0/3338107514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc08406c9b0 0x7fc084139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc089600640 1 --2- 172.21.15.161:0/3338107514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc0841444a0 0x7fc08413eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc083e00640 1 -- 172.21.15.161:0/3338107514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc08406c9b0 msgr2=0x7fc084139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc083e00640 1 --2- 172.21.15.161:0/3338107514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc08406c9b0 0x7fc084139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc083e00640 1 -- 172.21.15.161:0/3338107514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc0841444a0 msgr2=0x7fc08413eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc083e00640 1 --2- 172.21.15.161:0/3338107514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc0841444a0 0x7fc08413eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc083e00640 1 -- 172.21.15.161:0/3338107514 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0740094d0 con 0x7fc084143c40 2024-05-22T21:14:03.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc083e00640 1 --2- 172.21.15.161:0/3338107514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc084143c40 0x7fc084139010 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc07000c410 tx=0x7fc07000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:03.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.493+0000 7fc080c00640 1 -- 172.21.15.161:0/3338107514 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0700054e0 con 0x7fc084143c40 2024-05-22T21:14:03.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.497+0000 7fc080c00640 1 -- 172.21.15.161:0/3338107514 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc070003600 con 0x7fc084143c40 2024-05-22T21:14:03.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.497+0000 7fc080c00640 1 -- 172.21.15.161:0/3338107514 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0700039b0 con 0x7fc084143c40 2024-05-22T21:14:03.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.497+0000 7fc08b600640 1 -- 172.21.15.161:0/3338107514 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc084152840 con 0x7fc084143c40 2024-05-22T21:14:03.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.497+0000 7fc08b600640 1 -- 172.21.15.161:0/3338107514 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc084139ca0 con 0x7fc084143c40 2024-05-22T21:14:03.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.497+0000 7fc08b600640 1 -- 172.21.15.161:0/3338107514 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc084067220 con 0x7fc084143c40 2024-05-22T21:14:03.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.497+0000 7fc080c00640 1 -- 172.21.15.161:0/3338107514 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc070003b10 con 0x7fc084143c40 2024-05-22T21:14:03.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.497+0000 7fc080c00640 1 --2- 172.21.15.161:0/3338107514 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc064066690 0x7fc064068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.497+0000 7fc080c00640 1 -- 172.21.15.161:0/3338107514 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc070083700 con 0x7fc084143c40 2024-05-22T21:14:03.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.497+0000 7fc088c00640 1 --2- 172.21.15.161:0/3338107514 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc064066690 0x7fc064068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.501+0000 7fc088c00640 1 --2- 172.21.15.161:0/3338107514 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc064066690 0x7fc064068b50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc0740ae000 tx=0x7fc0740b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:03.507 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.501+0000 7fc080c00640 1 -- 172.21.15.161:0/3338107514 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc070050b00 con 0x7fc084143c40 2024-05-22T21:14:03.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.657+0000 7fc08b600640 1 -- 172.21.15.161:0/3338107514 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc084152cf0 con 0x7fc084143c40 2024-05-22T21:14:03.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.657+0000 7fc080c00640 1 -- 172.21.15.161:0/3338107514 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc070017020 con 0x7fc084143c40 2024-05-22T21:14:03.663 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:14:03.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.661+0000 7fc08b600640 1 -- 172.21.15.161:0/3338107514 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc064066690 msgr2=0x7fc064068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.661+0000 7fc08b600640 1 --2- 172.21.15.161:0/3338107514 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc064066690 0x7fc064068b50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc0740ae000 tx=0x7fc0740b5040 comp rx=0 tx=0).stop 2024-05-22T21:14:03.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.661+0000 7fc08b600640 1 -- 172.21.15.161:0/3338107514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc084143c40 msgr2=0x7fc084139010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.661+0000 7fc08b600640 1 --2- 172.21.15.161:0/3338107514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc084143c40 0x7fc084139010 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc07000c410 tx=0x7fc07000c8d0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.661+0000 7fc08b600640 1 -- 172.21.15.161:0/3338107514 shutdown_connections 2024-05-22T21:14:03.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.661+0000 7fc08b600640 1 --2- 172.21.15.161:0/3338107514 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc064066690 0x7fc064068b50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.661+0000 7fc08b600640 1 --2- 172.21.15.161:0/3338107514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc0841444a0 0x7fc08413eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.661+0000 7fc08b600640 1 --2- 172.21.15.161:0/3338107514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc084143c40 0x7fc084139010 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.661+0000 7fc08b600640 1 --2- 172.21.15.161:0/3338107514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc08406c9b0 0x7fc084139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.661+0000 7fc08b600640 1 -- 172.21.15.161:0/3338107514 >> 172.21.15.161:0/3338107514 conn(0x7fc08406a2d0 msgr2=0x7fc08415cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:03.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.661+0000 7fc08b600640 1 -- 172.21.15.161:0/3338107514 shutdown_connections 2024-05-22T21:14:03.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.661+0000 7fc08b600640 1 -- 172.21.15.161:0/3338107514 wait complete. 2024-05-22T21:14:03.684 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:14:03.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdec1400640 1 Processor -- start 2024-05-22T21:14:03.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdec1400640 1 -- start start 2024-05-22T21:14:03.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdec1400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdebc1402e0 0x7fdebc13f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdec1400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdebc1400d0 con 0x7fdebc13cfc0 2024-05-22T21:14:03.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdec1400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdebc13d3a0 con 0x7fdebc13fc80 2024-05-22T21:14:03.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdec1400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc0cd920 con 0x7fdebc1402e0 2024-05-22T21:14:03.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeba800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fdebc13cfc0 0x7fdebc15af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33786/0 (socket says 172.21.15.161:33786) 2024-05-22T21:14:03.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeba800640 1 -- 172.21.15.161:0/478510850 learned_addr learned my addr 172.21.15.161:0/478510850 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:03.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9400640 1 -- 172.21.15.161:0/478510850 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796655991 0 0) 0x7fdeac001120 con 0x7fdebc13cfc0 2024-05-22T21:14:03.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9e00640 1 --2- 172.21.15.161:0/478510850 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdebc1402e0 0x7fdebc13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9400640 1 -- 172.21.15.161:0/478510850 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdea0005740 con 0x7fdebc13cfc0 2024-05-22T21:14:03.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9400640 1 -- 172.21.15.161:0/478510850 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1167927082 0 0) 0x7fdea0005740 con 0x7fdebc13cfc0 2024-05-22T21:14:03.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9400640 1 -- 172.21.15.161:0/478510850 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdebc1402e0 msgr2=0x7fdebc13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:03.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9400640 1 --2- 172.21.15.161:0/478510850 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdebc1402e0 0x7fdebc13f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9400640 1 -- 172.21.15.161:0/478510850 >> v1:172.21.15.179:6789/0 conn(0x7fdebc13fc80 legacy=0x7fdebc06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9400640 1 -- 172.21.15.161:0/478510850 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc0cdc00 con 0x7fdebc13cfc0 2024-05-22T21:14:03.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9400640 1 -- 172.21.15.161:0/478510850 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fdeac001810 con 0x7fdebc13cfc0 2024-05-22T21:14:03.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9400640 1 -- 172.21.15.161:0/478510850 >> v1:172.21.15.161:6789/0 conn(0x7fdebc13cfc0 legacy=0x7fdebc15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9400640 1 --2- 172.21.15.161:0/478510850 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdebc1402e0 0x7fdea0005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9400640 1 --2- 172.21.15.161:0/478510850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdea0006140 0x7fdea00069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9400640 1 --2- 172.21.15.161:0/478510850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdea0006f20 0x7fdea00073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9400640 1 -- 172.21.15.161:0/478510850 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc0cd920 con 0x7fdea0006140 2024-05-22T21:14:03.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9400640 1 -- 172.21.15.161:0/478510850 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea000dd30 con 0x7fdea0006f20 2024-05-22T21:14:03.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9400640 1 -- 172.21.15.161:0/478510850 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea000def0 con 0x7fdebc1402e0 2024-05-22T21:14:03.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9e00640 1 --2- 172.21.15.161:0/478510850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdea0006140 0x7fdea00069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeba800640 1 --2- 172.21.15.161:0/478510850 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdebc1402e0 0x7fdea0005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdebb200640 1 --2- 172.21.15.161:0/478510850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdea0006f20 0x7fdea00073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9e00640 1 -- 172.21.15.161:0/478510850 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdebc1402e0 msgr2=0x7fdea0005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:03.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9e00640 1 --2- 172.21.15.161:0/478510850 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdebc1402e0 0x7fdea0005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9e00640 1 -- 172.21.15.161:0/478510850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdea0006f20 msgr2=0x7fdea00073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:03.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9e00640 1 --2- 172.21.15.161:0/478510850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdea0006f20 0x7fdea00073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.817+0000 7fdeb9e00640 1 -- 172.21.15.161:0/478510850 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea0005740 con 0x7fdea0006140 2024-05-22T21:14:03.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.821+0000 7fdeb9e00640 1 --2- 172.21.15.161:0/478510850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdea0006140 0x7fdea00069e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009ae0 tx=0x7fdea801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cd8eef4593600b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:03.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.821+0000 7fdeb9400640 1 -- 172.21.15.161:0/478510850 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea80a6500 con 0x7fdea0006140 2024-05-22T21:14:03.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.821+0000 7fdeb9400640 1 -- 172.21.15.161:0/478510850 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea80a6660 con 0x7fdea0006140 2024-05-22T21:14:03.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.821+0000 7fdeb9400640 1 -- 172.21.15.161:0/478510850 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea80a6810 con 0x7fdea0006140 2024-05-22T21:14:03.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.821+0000 7fdec1400640 1 -- 172.21.15.161:0/478510850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdea0006140 msgr2=0x7fdea00069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.821+0000 7fdec1400640 1 --2- 172.21.15.161:0/478510850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdea0006140 0x7fdea00069e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009ae0 tx=0x7fdea801ca40 comp rx=0 tx=0).stop 2024-05-22T21:14:03.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.821+0000 7fdebb200640 1 -- 172.21.15.161:0/478510850 reap_dead start 2024-05-22T21:14:03.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.821+0000 7fdec1400640 1 -- 172.21.15.161:0/478510850 shutdown_connections 2024-05-22T21:14:03.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.821+0000 7fdec1400640 1 -- 172.21.15.161:0/478510850 >> 172.21.15.161:0/478510850 conn(0x7fdebc06a2d0 msgr2=0x7fdebc06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:03.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.821+0000 7fdec1400640 1 -- 172.21.15.161:0/478510850 shutdown_connections 2024-05-22T21:14:03.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.821+0000 7fdec1400640 1 -- 172.21.15.161:0/478510850 wait complete. 2024-05-22T21:14:03.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdec1400640 1 Processor -- start 2024-05-22T21:14:03.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdec1400640 1 -- start start 2024-05-22T21:14:03.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdec1400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdea0006140 0x7fdebc1578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdec1400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdebc15c0e0 0x7fdebc1371a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdec1400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdebc1376e0 0x7fdebc137b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdec1400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc157e10 con 0x7fdebc1376e0 2024-05-22T21:14:03.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdec1400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc15c570 con 0x7fdea0006140 2024-05-22T21:14:03.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdec1400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc15c850 con 0x7fdebc15c0e0 2024-05-22T21:14:03.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdebb200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdebc1376e0 0x7fdebc137b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdebb200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdebc1376e0 0x7fdebc137b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59320/0 (socket says 172.21.15.161:59320) 2024-05-22T21:14:03.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdebb200640 1 -- 172.21.15.161:0/499212422 learned_addr learned my addr 172.21.15.161:0/499212422 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:03.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdeba800640 1 --2- 172.21.15.161:0/499212422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdea0006140 0x7fdebc1578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdeb9e00640 1 --2- 172.21.15.161:0/499212422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdebc15c0e0 0x7fdebc1371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdebb200640 1 -- 172.21.15.161:0/499212422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdebc15c0e0 msgr2=0x7fdebc1371a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdebb200640 1 --2- 172.21.15.161:0/499212422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdebc15c0e0 0x7fdebc1371a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdebb200640 1 -- 172.21.15.161:0/499212422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdea0006140 msgr2=0x7fdebc1578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdebb200640 1 --2- 172.21.15.161:0/499212422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdea0006140 0x7fdebc1578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdebb200640 1 -- 172.21.15.161:0/499212422 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc15cb30 con 0x7fdebc1376e0 2024-05-22T21:14:03.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdeb9e00640 1 --2- 172.21.15.161:0/499212422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdebc15c0e0 0x7fdebc1371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:03.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdebb200640 1 --2- 172.21.15.161:0/499212422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdebc1376e0 0x7fdebc137b70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdebc06c5e0 tx=0x7fdeb400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:03.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdeb2a00640 1 -- 172.21.15.161:0/499212422 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4017070 con 0x7fdebc1376e0 2024-05-22T21:14:03.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdeb2a00640 1 -- 172.21.15.161:0/499212422 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb4006b60 con 0x7fdebc1376e0 2024-05-22T21:14:03.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdeb2a00640 1 -- 172.21.15.161:0/499212422 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4006cf0 con 0x7fdebc1376e0 2024-05-22T21:14:03.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdec1400640 1 -- 172.21.15.161:0/499212422 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdebc15ce10 con 0x7fdebc1376e0 2024-05-22T21:14:03.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.825+0000 7fdec1400640 1 -- 172.21.15.161:0/499212422 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdebc13aee0 con 0x7fdebc1376e0 2024-05-22T21:14:03.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.829+0000 7fdec1400640 1 -- 172.21.15.161:0/499212422 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdebc067220 con 0x7fdebc1376e0 2024-05-22T21:14:03.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.829+0000 7fdeb2a00640 1 -- 172.21.15.161:0/499212422 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdeb4003980 con 0x7fdebc1376e0 2024-05-22T21:14:03.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.829+0000 7fdeb2a00640 1 --2- 172.21.15.161:0/499212422 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fde9c0666c0 0x7fde9c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:03.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.829+0000 7fdeb2a00640 1 -- 172.21.15.161:0/499212422 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdeb40838f0 con 0x7fdebc1376e0 2024-05-22T21:14:03.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.829+0000 7fdeba800640 1 --2- 172.21.15.161:0/499212422 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fde9c0666c0 0x7fde9c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:03.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.829+0000 7fdeba800640 1 --2- 172.21.15.161:0/499212422 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fde9c0666c0 0x7fde9c068b80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdeac001e30 tx=0x7fdeac007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:03.838 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.833+0000 7fdeb2a00640 1 -- 172.21.15.161:0/499212422 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdeb4050bc0 con 0x7fdebc1376e0 2024-05-22T21:14:03.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.985+0000 7fdec1400640 1 -- 172.21.15.161:0/499212422 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdebc071670 con 0x7fdebc1376e0 2024-05-22T21:14:03.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.985+0000 7fdeb2a00640 1 -- 172.21.15.161:0/499212422 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdeb40541e0 con 0x7fdebc1376e0 2024-05-22T21:14:03.991 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:14:03.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.989+0000 7fdec1400640 1 -- 172.21.15.161:0/499212422 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fde9c0666c0 msgr2=0x7fde9c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.989+0000 7fdec1400640 1 --2- 172.21.15.161:0/499212422 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fde9c0666c0 0x7fde9c068b80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdeac001e30 tx=0x7fdeac007040 comp rx=0 tx=0).stop 2024-05-22T21:14:03.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.989+0000 7fdec1400640 1 -- 172.21.15.161:0/499212422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdebc1376e0 msgr2=0x7fdebc137b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:03.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.989+0000 7fdec1400640 1 --2- 172.21.15.161:0/499212422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdebc1376e0 0x7fdebc137b70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdebc06c5e0 tx=0x7fdeb400ec10 comp rx=0 tx=0).stop 2024-05-22T21:14:03.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.989+0000 7fdec1400640 1 -- 172.21.15.161:0/499212422 shutdown_connections 2024-05-22T21:14:03.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.989+0000 7fdec1400640 1 --2- 172.21.15.161:0/499212422 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fde9c0666c0 0x7fde9c068b80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.989+0000 7fdec1400640 1 --2- 172.21.15.161:0/499212422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdebc1376e0 0x7fdebc137b70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.989+0000 7fdec1400640 1 --2- 172.21.15.161:0/499212422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdebc15c0e0 0x7fdebc1371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.989+0000 7fdec1400640 1 --2- 172.21.15.161:0/499212422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdea0006140 0x7fdebc1578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:03.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.989+0000 7fdec1400640 1 -- 172.21.15.161:0/499212422 >> 172.21.15.161:0/499212422 conn(0x7fdebc06a2d0 msgr2=0x7fdebc0b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:03.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.989+0000 7fdec1400640 1 -- 172.21.15.161:0/499212422 shutdown_connections 2024-05-22T21:14:03.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:03.989+0000 7fdec1400640 1 -- 172.21.15.161:0/499212422 wait complete. 2024-05-22T21:14:04.012 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:14:04.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4982c00640 1 Processor -- start 2024-05-22T21:14:04.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4982c00640 1 -- start start 2024-05-22T21:14:04.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4982c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f497c1436d0 0x7f497c0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4982c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f497c06fa80 0x7f497c0b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4982c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f497c0b1b10 0x7f497c0b3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4982c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f497c0cc720 con 0x7f497c1436d0 2024-05-22T21:14:04.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4982c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f497c070b70 con 0x7f497c0b1b10 2024-05-22T21:14:04.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4982c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497c06ff60 con 0x7f497c06fa80 2024-05-22T21:14:04.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4981800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f497c1436d0 0x7f497c0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4981800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f497c1436d0 0x7f497c0cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59322/0 (socket says 172.21.15.161:59322) 2024-05-22T21:14:04.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4981800640 1 -- 172.21.15.161:0/457895641 learned_addr learned my addr 172.21.15.161:0/457895641 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:04.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4982200640 1 --2- 172.21.15.161:0/457895641 >> v2:172.21.15.179:3300/0 conn(0x7f497c0b1b10 0x7f497c0b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4981800640 1 -- 172.21.15.161:0/457895641 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f497c06fa80 msgr2=0x7f497c0b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:04.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4980e00640 1 --2- 172.21.15.161:0/457895641 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f497c06fa80 0x7f497c0b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4981800640 1 --2- 172.21.15.161:0/457895641 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f497c06fa80 0x7f497c0b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4981800640 1 -- 172.21.15.161:0/457895641 >> v2:172.21.15.179:3300/0 conn(0x7f497c0b1b10 msgr2=0x7f497c0b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:04.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4981800640 1 --2- 172.21.15.161:0/457895641 >> v2:172.21.15.179:3300/0 conn(0x7f497c0b1b10 0x7f497c0b3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4981800640 1 -- 172.21.15.161:0/457895641 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497c070240 con 0x7f497c1436d0 2024-05-22T21:14:04.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4982200640 1 --2- 172.21.15.161:0/457895641 >> v2:172.21.15.179:3300/0 conn(0x7f497c0b1b10 0x7f497c0b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:04.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.145+0000 7f4980e00640 1 --2- 172.21.15.161:0/457895641 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f497c06fa80 0x7f497c0b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:04.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.149+0000 7f4981800640 1 --2- 172.21.15.161:0/457895641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c1436d0 0x7f497c0cc1e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f497c06f3a0 tx=0x7f497001eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46c6e6503d84e0b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:04.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.149+0000 7f497be00640 1 -- 172.21.15.161:0/457895641 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49700a7bc0 con 0x7f497c1436d0 2024-05-22T21:14:04.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.149+0000 7f497be00640 1 -- 172.21.15.161:0/457895641 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49700a7d20 con 0x7f497c1436d0 2024-05-22T21:14:04.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.149+0000 7f497be00640 1 -- 172.21.15.161:0/457895641 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49700af660 con 0x7f497c1436d0 2024-05-22T21:14:04.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.149+0000 7f4982c00640 1 -- 172.21.15.161:0/457895641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c1436d0 msgr2=0x7f497c0cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:04.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.149+0000 7f4982c00640 1 --2- 172.21.15.161:0/457895641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c1436d0 0x7f497c0cc1e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f497c06f3a0 tx=0x7f497001eb30 comp rx=0 tx=0).stop 2024-05-22T21:14:04.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.149+0000 7f4982c00640 1 -- 172.21.15.161:0/457895641 shutdown_connections 2024-05-22T21:14:04.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.149+0000 7f4982c00640 1 --2- 172.21.15.161:0/457895641 >> v2:172.21.15.179:3300/0 conn(0x7f497c0b1b10 0x7f497c0b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.149+0000 7f4982c00640 1 --2- 172.21.15.161:0/457895641 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f497c06fa80 0x7f497c0b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.149+0000 7f4982c00640 1 --2- 172.21.15.161:0/457895641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c1436d0 0x7f497c0cc1e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.149+0000 7f4982c00640 1 -- 172.21.15.161:0/457895641 >> 172.21.15.161:0/457895641 conn(0x7f497c0cec00 msgr2=0x7f497c0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:04.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.149+0000 7f4982c00640 1 -- 172.21.15.161:0/457895641 shutdown_connections 2024-05-22T21:14:04.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.149+0000 7f4982c00640 1 -- 172.21.15.161:0/457895641 wait complete. 2024-05-22T21:14:04.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982c00640 1 Processor -- start 2024-05-22T21:14:04.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982c00640 1 -- start start 2024-05-22T21:14:04.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f497c06fa80 0x7f497c13e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f497c0b1b10 0x7f497c1550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c1436d0 0x7f497c155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c15d880 con 0x7f497c1436d0 2024-05-22T21:14:04.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c155d00 con 0x7f497c0b1b10 2024-05-22T21:14:04.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497c155fe0 con 0x7f497c06fa80 2024-05-22T21:14:04.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c1436d0 0x7f497c155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c1436d0 0x7f497c155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59326/0 (socket says 172.21.15.161:59326) 2024-05-22T21:14:04.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982200640 1 -- 172.21.15.161:0/1316643559 learned_addr learned my addr 172.21.15.161:0/1316643559 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:04.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4980e00640 1 --2- 172.21.15.161:0/1316643559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f497c0b1b10 0x7f497c1550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4981800640 1 --2- 172.21.15.161:0/1316643559 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f497c06fa80 0x7f497c13e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982200640 1 -- 172.21.15.161:0/1316643559 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f497c06fa80 msgr2=0x7f497c13e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:04.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982200640 1 --2- 172.21.15.161:0/1316643559 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f497c06fa80 0x7f497c13e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982200640 1 -- 172.21.15.161:0/1316643559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f497c0b1b10 msgr2=0x7f497c1550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:04.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982200640 1 --2- 172.21.15.161:0/1316643559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f497c0b1b10 0x7f497c1550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982200640 1 -- 172.21.15.161:0/1316643559 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497000b770 con 0x7f497c1436d0 2024-05-22T21:14:04.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982200640 1 --2- 172.21.15.161:0/1316643559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c1436d0 0x7f497c155630 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f497400cef0 tx=0x7f497400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:04.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4979600640 1 -- 172.21.15.161:0/1316643559 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400d840 con 0x7f497c1436d0 2024-05-22T21:14:04.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4979600640 1 -- 172.21.15.161:0/1316643559 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497400d9c0 con 0x7f497c1436d0 2024-05-22T21:14:04.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982c00640 1 -- 172.21.15.161:0/1316643559 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497c1562c0 con 0x7f497c1436d0 2024-05-22T21:14:04.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4979600640 1 -- 172.21.15.161:0/1316643559 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400dd70 con 0x7f497c1436d0 2024-05-22T21:14:04.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.153+0000 7f4982c00640 1 -- 172.21.15.161:0/1316643559 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497c1565a0 con 0x7f497c1436d0 2024-05-22T21:14:04.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.157+0000 7f4982c00640 1 -- 172.21.15.161:0/1316643559 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497c067070 con 0x7f497c1436d0 2024-05-22T21:14:04.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.157+0000 7f4979600640 1 -- 172.21.15.161:0/1316643559 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f497401e3b0 con 0x7f497c1436d0 2024-05-22T21:14:04.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.157+0000 7f4979600640 1 --2- 172.21.15.161:0/1316643559 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f495c066690 0x7f495c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.157+0000 7f4979600640 1 -- 172.21.15.161:0/1316643559 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4974084bb0 con 0x7f497c1436d0 2024-05-22T21:14:04.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.157+0000 7f4981800640 1 --2- 172.21.15.161:0/1316643559 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f495c066690 0x7f495c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.157+0000 7f4981800640 1 --2- 172.21.15.161:0/1316643559 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f495c066690 0x7f495c068b50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f49700b1000 tx=0x7f49700a9070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:04.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.161+0000 7f4979600640 1 -- 172.21.15.161:0/1316643559 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4974052080 con 0x7f497c1436d0 2024-05-22T21:14:04.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.309+0000 7f4982c00640 1 -- 172.21.15.161:0/1316643559 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f497c1407f0 con 0x7f497c1436d0 2024-05-22T21:14:04.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.313+0000 7f4979600640 1 -- 172.21.15.161:0/1316643559 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f497401a020 con 0x7f497c1436d0 2024-05-22T21:14:04.316 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:14:04.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.313+0000 7f4982c00640 1 -- 172.21.15.161:0/1316643559 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f495c066690 msgr2=0x7f495c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:04.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.313+0000 7f4982c00640 1 --2- 172.21.15.161:0/1316643559 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f495c066690 0x7f495c068b50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f49700b1000 tx=0x7f49700a9070 comp rx=0 tx=0).stop 2024-05-22T21:14:04.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.313+0000 7f4982c00640 1 -- 172.21.15.161:0/1316643559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c1436d0 msgr2=0x7f497c155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:04.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.313+0000 7f4982c00640 1 --2- 172.21.15.161:0/1316643559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c1436d0 0x7f497c155630 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f497400cef0 tx=0x7f497400a710 comp rx=0 tx=0).stop 2024-05-22T21:14:04.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.313+0000 7f4982c00640 1 -- 172.21.15.161:0/1316643559 shutdown_connections 2024-05-22T21:14:04.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.313+0000 7f4982c00640 1 --2- 172.21.15.161:0/1316643559 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f495c066690 0x7f495c068b50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.313+0000 7f4982c00640 1 --2- 172.21.15.161:0/1316643559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f497c1436d0 0x7f497c155630 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.313+0000 7f4982c00640 1 --2- 172.21.15.161:0/1316643559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f497c0b1b10 0x7f497c1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.313+0000 7f4982c00640 1 --2- 172.21.15.161:0/1316643559 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f497c06fa80 0x7f497c13e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.313+0000 7f4982c00640 1 -- 172.21.15.161:0/1316643559 >> 172.21.15.161:0/1316643559 conn(0x7f497c0cec00 msgr2=0x7f497c0b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:04.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.313+0000 7f4982c00640 1 -- 172.21.15.161:0/1316643559 shutdown_connections 2024-05-22T21:14:04.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.313+0000 7f4982c00640 1 -- 172.21.15.161:0/1316643559 wait complete. 2024-05-22T21:14:04.338 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:14:04.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.469+0000 7fad23800640 1 Processor -- start 2024-05-22T21:14:04.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.469+0000 7fad23800640 1 -- start start 2024-05-22T21:14:04.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.469+0000 7fad23800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c143c40 0x7fad1c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.469+0000 7fad23800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fad1c06c9b0 0x7fad1c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.469+0000 7fad23800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1c15efb0 con 0x7fad1c1444a0 2024-05-22T21:14:04.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.469+0000 7fad23800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1c15f290 con 0x7fad1c06c9b0 2024-05-22T21:14:04.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.469+0000 7fad23800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c15f570 con 0x7fad1c143c40 2024-05-22T21:14:04.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.473+0000 7fad21800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fad1c06c9b0 0x7fad1c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.473+0000 7fad21800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fad1c06c9b0 0x7fad1c0b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59336/0 (socket says 172.21.15.161:59336) 2024-05-22T21:14:04.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.473+0000 7fad21800640 1 -- 172.21.15.161:0/1867784542 learned_addr learned my addr 172.21.15.161:0/1867784542 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:04.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.473+0000 7fad20e00640 1 --2- 172.21.15.161:0/1867784542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c143c40 0x7fad1c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.473+0000 7fad21800640 1 -- 172.21.15.161:0/1867784542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c143c40 msgr2=0x7fad1c15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:04.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.473+0000 7fad21800640 1 --2- 172.21.15.161:0/1867784542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c143c40 0x7fad1c15b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.473+0000 7fad21800640 1 -- 172.21.15.161:0/1867784542 >> v1:172.21.15.179:6789/0 conn(0x7fad1c1444a0 legacy=0x7fad1c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:04.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.473+0000 7fad21800640 1 -- 172.21.15.161:0/1867784542 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c15f850 con 0x7fad1c06c9b0 2024-05-22T21:14:04.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.473+0000 7fad20e00640 1 --2- 172.21.15.161:0/1867784542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c143c40 0x7fad1c15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:04.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.473+0000 7fad21800640 1 --2- 172.21.15.161:0/1867784542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c06c9b0 0x7fad1c0b2160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fad14009e50 tx=0x7fad1401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696c3945da99149e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:04.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.473+0000 7fad1be00640 1 -- 172.21.15.161:0/1867784542 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad140a5860 con 0x7fad1c06c9b0 2024-05-22T21:14:04.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.473+0000 7fad1be00640 1 -- 172.21.15.161:0/1867784542 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad140a59c0 con 0x7fad1c06c9b0 2024-05-22T21:14:04.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.473+0000 7fad1be00640 1 -- 172.21.15.161:0/1867784542 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad140a5d50 con 0x7fad1c06c9b0 2024-05-22T21:14:04.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad23800640 1 -- 172.21.15.161:0/1867784542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c06c9b0 msgr2=0x7fad1c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:04.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad23800640 1 --2- 172.21.15.161:0/1867784542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c06c9b0 0x7fad1c0b2160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fad14009e50 tx=0x7fad1401ca30 comp rx=0 tx=0).stop 2024-05-22T21:14:04.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad23800640 1 -- 172.21.15.161:0/1867784542 shutdown_connections 2024-05-22T21:14:04.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad23800640 1 --2- 172.21.15.161:0/1867784542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c06c9b0 0x7fad1c0b2160 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad23800640 1 --2- 172.21.15.161:0/1867784542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c143c40 0x7fad1c15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad23800640 1 -- 172.21.15.161:0/1867784542 >> 172.21.15.161:0/1867784542 conn(0x7fad1c06a2d0 msgr2=0x7fad1c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:04.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad23800640 1 -- 172.21.15.161:0/1867784542 shutdown_connections 2024-05-22T21:14:04.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad23800640 1 -- 172.21.15.161:0/1867784542 wait complete. 2024-05-22T21:14:04.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad23800640 1 Processor -- start 2024-05-22T21:14:04.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad23800640 1 -- start start 2024-05-22T21:14:04.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad23800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c06c9b0 0x7fad1c140a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad23800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c143c40 0x7fad1c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad23800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c1444a0 0x7fad1c0afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad23800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c15b720 con 0x7fad1c1444a0 2024-05-22T21:14:04.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad23800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c15af50 con 0x7fad1c143c40 2024-05-22T21:14:04.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad23800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c13ce20 con 0x7fad1c06c9b0 2024-05-22T21:14:04.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad21800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c1444a0 0x7fad1c0afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad21800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c1444a0 0x7fad1c0afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59340/0 (socket says 172.21.15.161:59340) 2024-05-22T21:14:04.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad21800640 1 -- 172.21.15.161:0/2125421522 learned_addr learned my addr 172.21.15.161:0/2125421522 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:04.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad13e00640 1 --2- 172.21.15.161:0/2125421522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c143c40 0x7fad1c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad20e00640 1 --2- 172.21.15.161:0/2125421522 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c06c9b0 0x7fad1c140a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad21800640 1 -- 172.21.15.161:0/2125421522 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c06c9b0 msgr2=0x7fad1c140a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:04.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad21800640 1 --2- 172.21.15.161:0/2125421522 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c06c9b0 0x7fad1c140a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad21800640 1 -- 172.21.15.161:0/2125421522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c143c40 msgr2=0x7fad1c159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:04.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad21800640 1 --2- 172.21.15.161:0/2125421522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c143c40 0x7fad1c159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad21800640 1 -- 172.21.15.161:0/2125421522 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad14009b30 con 0x7fad1c1444a0 2024-05-22T21:14:04.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.477+0000 7fad20e00640 1 --2- 172.21.15.161:0/2125421522 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c06c9b0 0x7fad1c140a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:04.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.481+0000 7fad21800640 1 --2- 172.21.15.161:0/2125421522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c1444a0 0x7fad1c0afd70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fad140af040 tx=0x7fad140197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.481+0000 7fad19600640 1 -- 172.21.15.161:0/2125421522 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad140b8670 con 0x7fad1c1444a0 2024-05-22T21:14:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.481+0000 7fad19600640 1 -- 172.21.15.161:0/2125421522 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad140b87f0 con 0x7fad1c1444a0 2024-05-22T21:14:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.481+0000 7fad19600640 1 -- 172.21.15.161:0/2125421522 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad140b8ba0 con 0x7fad1c1444a0 2024-05-22T21:14:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.481+0000 7fad23800640 1 -- 172.21.15.161:0/2125421522 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1c13d3e0 con 0x7fad1c1444a0 2024-05-22T21:14:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.481+0000 7fad23800640 1 -- 172.21.15.161:0/2125421522 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1c15b160 con 0x7fad1c1444a0 2024-05-22T21:14:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.481+0000 7fad23800640 1 -- 172.21.15.161:0/2125421522 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1c13d570 con 0x7fad1c1444a0 2024-05-22T21:14:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.481+0000 7fad19600640 1 -- 172.21.15.161:0/2125421522 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fad140b8d00 con 0x7fad1c1444a0 2024-05-22T21:14:04.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.481+0000 7fad19600640 1 --2- 172.21.15.161:0/2125421522 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7facf8066690 0x7facf8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.481+0000 7fad19600640 1 -- 172.21.15.161:0/2125421522 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad1411f780 con 0x7fad1c1444a0 2024-05-22T21:14:04.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.481+0000 7fad20e00640 1 --2- 172.21.15.161:0/2125421522 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7facf8066690 0x7facf8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.485+0000 7fad20e00640 1 --2- 172.21.15.161:0/2125421522 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7facf8066690 0x7facf8068b50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009980 tx=0x7fad0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:04.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.485+0000 7fad19600640 1 -- 172.21.15.161:0/2125421522 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fad140eca70 con 0x7fad1c1444a0 2024-05-22T21:14:04.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.657+0000 7fad23800640 1 -- 172.21.15.161:0/2125421522 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fad1c067220 con 0x7fad1c1444a0 2024-05-22T21:14:04.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.657+0000 7fad19600640 1 -- 172.21.15.161:0/2125421522 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fad140f0090 con 0x7fad1c1444a0 2024-05-22T21:14:04.661 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:14:04.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.657+0000 7fad23800640 1 -- 172.21.15.161:0/2125421522 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7facf8066690 msgr2=0x7facf8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:04.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.657+0000 7fad23800640 1 --2- 172.21.15.161:0/2125421522 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7facf8066690 0x7facf8068b50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009980 tx=0x7fad0c008040 comp rx=0 tx=0).stop 2024-05-22T21:14:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.657+0000 7fad23800640 1 -- 172.21.15.161:0/2125421522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c1444a0 msgr2=0x7fad1c0afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.657+0000 7fad23800640 1 --2- 172.21.15.161:0/2125421522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c1444a0 0x7fad1c0afd70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fad140af040 tx=0x7fad140197d0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.657+0000 7fad23800640 1 -- 172.21.15.161:0/2125421522 shutdown_connections 2024-05-22T21:14:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.657+0000 7fad23800640 1 --2- 172.21.15.161:0/2125421522 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7facf8066690 0x7facf8068b50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.657+0000 7fad23800640 1 --2- 172.21.15.161:0/2125421522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c1444a0 0x7fad1c0afd70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.657+0000 7fad23800640 1 --2- 172.21.15.161:0/2125421522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c143c40 0x7fad1c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.657+0000 7fad23800640 1 --2- 172.21.15.161:0/2125421522 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c06c9b0 0x7fad1c140a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.657+0000 7fad23800640 1 -- 172.21.15.161:0/2125421522 >> 172.21.15.161:0/2125421522 conn(0x7fad1c06a2d0 msgr2=0x7fad1c06c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.657+0000 7fad23800640 1 -- 172.21.15.161:0/2125421522 shutdown_connections 2024-05-22T21:14:04.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.657+0000 7fad23800640 1 -- 172.21.15.161:0/2125421522 wait complete. 2024-05-22T21:14:04.683 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-05-22T21:14:04.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:04.683 DEBUG:teuthology.orchestra.run.smithi179:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-05-22T21:14:04.709 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:14:04.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:04.710 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-22T21:14:04.759 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-22T21:14:04.759 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:14:04.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.889+0000 7f8936a00640 1 Processor -- start 2024-05-22T21:14:04.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.889+0000 7f8936a00640 1 -- start start 2024-05-22T21:14:04.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.889+0000 7f8936a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f893015e130 0x7f893015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.889+0000 7f8936a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f893015d410 0x7f893015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.889+0000 7f8936a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8930071670 con 0x7f893015dd50 2024-05-22T21:14:04.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.889+0000 7f8936a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f893015d870 con 0x7f893015d410 2024-05-22T21:14:04.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.889+0000 7f8936a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8930070a60 con 0x7f893015e130 2024-05-22T21:14:04.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.889+0000 7f8934a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f893015d410 0x7f893015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.889+0000 7f8934a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f893015d410 0x7f893015b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59342/0 (socket says 172.21.15.161:59342) 2024-05-22T21:14:04.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.889+0000 7f8934a00640 1 -- 172.21.15.161:0/204261020 learned_addr learned my addr 172.21.15.161:0/204261020 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:04.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.889+0000 7f8927400640 1 --2- 172.21.15.161:0/204261020 >> v2:172.21.15.179:3300/0 conn(0x7f893015e130 0x7f893015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.889+0000 7f8934a00640 1 -- 172.21.15.161:0/204261020 >> v2:172.21.15.179:3300/0 conn(0x7f893015e130 msgr2=0x7f893015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:04.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.889+0000 7f8934a00640 1 --2- 172.21.15.161:0/204261020 >> v2:172.21.15.179:3300/0 conn(0x7f893015e130 0x7f893015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.889+0000 7f8934a00640 1 -- 172.21.15.161:0/204261020 >> v1:172.21.15.179:6789/0 conn(0x7f893015dd50 legacy=0x7f893006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:04.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.889+0000 7f8934a00640 1 -- 172.21.15.161:0/204261020 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8930070d40 con 0x7f893015d410 2024-05-22T21:14:04.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.893+0000 7f8934a00640 1 --2- 172.21.15.161:0/204261020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f893015d410 0x7f893015b6c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f893006c7c0 tx=0x7f892801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758c00041e7bcdf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:04.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.893+0000 7f892f400640 1 -- 172.21.15.161:0/204261020 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89280a5880 con 0x7f893015d410 2024-05-22T21:14:04.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.893+0000 7f892f400640 1 -- 172.21.15.161:0/204261020 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89280a59e0 con 0x7f893015d410 2024-05-22T21:14:04.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.893+0000 7f892f400640 1 -- 172.21.15.161:0/204261020 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89280a5d70 con 0x7f893015d410 2024-05-22T21:14:04.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.893+0000 7f8936a00640 1 -- 172.21.15.161:0/204261020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f893015d410 msgr2=0x7f893015b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:04.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.893+0000 7f8936a00640 1 --2- 172.21.15.161:0/204261020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f893015d410 0x7f893015b6c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f893006c7c0 tx=0x7f892801c9c0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.893+0000 7f8936a00640 1 -- 172.21.15.161:0/204261020 shutdown_connections 2024-05-22T21:14:04.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.893+0000 7f8936a00640 1 --2- 172.21.15.161:0/204261020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f893015d410 0x7f893015b6c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.893+0000 7f8936a00640 1 --2- 172.21.15.161:0/204261020 >> v2:172.21.15.179:3300/0 conn(0x7f893015e130 0x7f893015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.893+0000 7f8936a00640 1 -- 172.21.15.161:0/204261020 >> 172.21.15.161:0/204261020 conn(0x7f89300b0bc0 msgr2=0x7f89300b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:04.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.893+0000 7f8936a00640 1 -- 172.21.15.161:0/204261020 shutdown_connections 2024-05-22T21:14:04.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.893+0000 7f8936a00640 1 -- 172.21.15.161:0/204261020 wait complete. 2024-05-22T21:14:04.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8936a00640 1 Processor -- start 2024-05-22T21:14:04.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8936a00640 1 -- start start 2024-05-22T21:14:04.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8936a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f893015d410 0x7f893014ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8936a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f893015dd50 0x7f89301525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8936a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f893015e130 0x7f89300b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8936a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8930157160 con 0x7f893015e130 2024-05-22T21:14:04.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8936a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8930156e80 con 0x7f893015d410 2024-05-22T21:14:04.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8936a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8930151ad0 con 0x7f893015dd50 2024-05-22T21:14:04.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8934a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f893015e130 0x7f89300b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8934a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f893015e130 0x7f89300b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59352/0 (socket says 172.21.15.161:59352) 2024-05-22T21:14:04.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8934a00640 1 -- 172.21.15.161:0/4096683866 learned_addr learned my addr 172.21.15.161:0/4096683866 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:04.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f892fe00640 1 --2- 172.21.15.161:0/4096683866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f893015d410 0x7f893014ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8927400640 1 --2- 172.21.15.161:0/4096683866 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f893015dd50 0x7f89301525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8934a00640 1 -- 172.21.15.161:0/4096683866 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f893015dd50 msgr2=0x7f89301525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:04.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8934a00640 1 --2- 172.21.15.161:0/4096683866 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f893015dd50 0x7f89301525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8934a00640 1 -- 172.21.15.161:0/4096683866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f893015d410 msgr2=0x7f893014ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:04.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8934a00640 1 --2- 172.21.15.161:0/4096683866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f893015d410 0x7f893014ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:04.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8934a00640 1 -- 172.21.15.161:0/4096683866 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8928009b30 con 0x7f893015e130 2024-05-22T21:14:04.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8927400640 1 --2- 172.21.15.161:0/4096683866 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f893015dd50 0x7f89301525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:04.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8934a00640 1 --2- 172.21.15.161:0/4096683866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f893015e130 0x7f89300b0fa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f89280af040 tx=0x7f8928019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:04.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f892cc00640 1 -- 172.21.15.161:0/4096683866 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89280b8670 con 0x7f893015e130 2024-05-22T21:14:04.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f892cc00640 1 -- 172.21.15.161:0/4096683866 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89280b87d0 con 0x7f893015e130 2024-05-22T21:14:04.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f892cc00640 1 -- 172.21.15.161:0/4096683866 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89280b8b80 con 0x7f893015e130 2024-05-22T21:14:04.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8936a00640 1 -- 172.21.15.161:0/4096683866 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8930151db0 con 0x7f893015e130 2024-05-22T21:14:04.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.897+0000 7f8936a00640 1 -- 172.21.15.161:0/4096683866 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893014dbc0 con 0x7f893015e130 2024-05-22T21:14:04.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.901+0000 7f8936a00640 1 -- 172.21.15.161:0/4096683866 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8930151c10 con 0x7f893015e130 2024-05-22T21:14:04.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.901+0000 7f892cc00640 1 -- 172.21.15.161:0/4096683866 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f89280b8ce0 con 0x7f893015e130 2024-05-22T21:14:04.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.901+0000 7f892cc00640 1 --2- 172.21.15.161:0/4096683866 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f890c066690 0x7f890c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:04.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.901+0000 7f892cc00640 1 -- 172.21.15.161:0/4096683866 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f892811f9c0 con 0x7f893015e130 2024-05-22T21:14:04.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.901+0000 7f892fe00640 1 --2- 172.21.15.161:0/4096683866 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f890c066690 0x7f890c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:04.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.901+0000 7f892fe00640 1 --2- 172.21.15.161:0/4096683866 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f890c066690 0x7f890c068b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8930151900 tx=0x7f8920009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:04.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:04.905+0000 7f892cc00640 1 -- 172.21.15.161:0/4096683866 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8930151c10 con 0x7f893015e130 2024-05-22T21:14:05.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.093+0000 7f8936a00640 1 -- 172.21.15.161:0/4096683866 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8930151c10 con 0x7f893015e130 2024-05-22T21:14:05.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.093+0000 7f892cc00640 1 -- 172.21.15.161:0/4096683866 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4740 (secure 0 0 0) 0x7f8930151c10 con 0x7f893015e130 2024-05-22T21:14:05.100 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:05.100 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3179612470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3179612470},{"type":"v1","addr":"172.21.15.179:6834","nonce":3179612470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2196055309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2196055309},{"type":"v1","addr":"172.21.15.161:6835","nonce":2196055309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1793681759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1793681759},{"type":"v1","addr":"172.21.15.161:6837","nonce":1793681759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:13:23.418412+0000","modified":"2024-05-22T21:13:26.442354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/3095599688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3095599688},{"type":"v1","addr":"172.21.15.179:6835","nonce":3095599688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-22T21:14:05.100 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 8 2024-05-22T21:14:05.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.097+0000 7f8936a00640 1 -- 172.21.15.161:0/4096683866 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f890c066690 msgr2=0x7f890c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.097+0000 7f8936a00640 1 --2- 172.21.15.161:0/4096683866 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f890c066690 0x7f890c068b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8930151900 tx=0x7f8920009040 comp rx=0 tx=0).stop 2024-05-22T21:14:05.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.097+0000 7f8936a00640 1 -- 172.21.15.161:0/4096683866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f893015e130 msgr2=0x7f89300b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.097+0000 7f8936a00640 1 --2- 172.21.15.161:0/4096683866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f893015e130 0x7f89300b0fa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f89280af040 tx=0x7f8928019760 comp rx=0 tx=0).stop 2024-05-22T21:14:05.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.097+0000 7f8936a00640 1 -- 172.21.15.161:0/4096683866 shutdown_connections 2024-05-22T21:14:05.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.097+0000 7f8936a00640 1 --2- 172.21.15.161:0/4096683866 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f890c066690 0x7f890c068b50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.097+0000 7f8936a00640 1 --2- 172.21.15.161:0/4096683866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f893015e130 0x7f89300b0fa0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.097+0000 7f8936a00640 1 --2- 172.21.15.161:0/4096683866 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f893015dd50 0x7f89301525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.097+0000 7f8936a00640 1 --2- 172.21.15.161:0/4096683866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f893015d410 0x7f893014ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.097+0000 7f8936a00640 1 -- 172.21.15.161:0/4096683866 >> 172.21.15.161:0/4096683866 conn(0x7f89300b0bc0 msgr2=0x7f89300aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:05.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.097+0000 7f8936a00640 1 -- 172.21.15.161:0/4096683866 shutdown_connections 2024-05-22T21:14:05.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.097+0000 7f8936a00640 1 -- 172.21.15.161:0/4096683866 wait complete. 2024-05-22T21:14:05.121 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:05.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.249+0000 7fa7a4a00640 1 Processor -- start 2024-05-22T21:14:05.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.253+0000 7fa7a4a00640 1 -- start start 2024-05-22T21:14:05.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.253+0000 7fa7a4a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7a0071770 0x7fa7a0069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.253+0000 7fa7a4a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a00b34a0 con 0x7fa7a0146970 2024-05-22T21:14:05.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.253+0000 7fa7a4a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a006a830 con 0x7fa7a006a410 2024-05-22T21:14:05.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.253+0000 7fa7a4a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a0146d50 con 0x7fa7a0071770 2024-05-22T21:14:05.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.253+0000 7fa79de00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7a0071770 0x7fa7a0069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.253+0000 7fa79de00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7a0071770 0x7fa7a0069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59358/0 (socket says 172.21.15.161:59358) 2024-05-22T21:14:05.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.253+0000 7fa79de00640 1 -- 172.21.15.161:0/596738626 learned_addr learned my addr 172.21.15.161:0/596738626 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:05.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.253+0000 7fa79e800640 1 --1- 172.21.15.161:0/596738626 >> v1:172.21.15.161:6789/0 conn(0x7fa7a0146970 0x7fa7a0160be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33792/0 (socket says 172.21.15.161:33792) 2024-05-22T21:14:05.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.253+0000 7fa79de00640 1 -- 172.21.15.161:0/596738626 >> v1:172.21.15.179:6789/0 conn(0x7fa7a006a410 legacy=0x7fa7a00b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:05.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.253+0000 7fa79de00640 1 -- 172.21.15.161:0/596738626 >> v1:172.21.15.161:6789/0 conn(0x7fa7a0146970 legacy=0x7fa7a0160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.253+0000 7fa79de00640 1 -- 172.21.15.161:0/596738626 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a01614a0 con 0x7fa7a0071770 2024-05-22T21:14:05.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.253+0000 7fa79d400640 1 -- 172.21.15.161:0/596738626 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964037350 0 0) 0x7fa7a00b34a0 con 0x7fa7a0146970 2024-05-22T21:14:05.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.257+0000 7fa79de00640 1 --2- 172.21.15.161:0/596738626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a0071770 0x7fa7a0069ed0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009870 tx=0x7fa78c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7799d2e44b36e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:05.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.257+0000 7fa79d400640 1 -- 172.21.15.161:0/596738626 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c0ab610 con 0x7fa7a0071770 2024-05-22T21:14:05.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.257+0000 7fa79d400640 1 -- 172.21.15.161:0/596738626 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c0ab770 con 0x7fa7a0071770 2024-05-22T21:14:05.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.257+0000 7fa79d400640 1 -- 172.21.15.161:0/596738626 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c0abb00 con 0x7fa7a0071770 2024-05-22T21:14:05.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.257+0000 7fa7a4a00640 1 -- 172.21.15.161:0/596738626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a0071770 msgr2=0x7fa7a0069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.257+0000 7fa7a4a00640 1 --2- 172.21.15.161:0/596738626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a0071770 0x7fa7a0069ed0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009870 tx=0x7fa78c01c520 comp rx=0 tx=0).stop 2024-05-22T21:14:05.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.257+0000 7fa7a4a00640 1 -- 172.21.15.161:0/596738626 shutdown_connections 2024-05-22T21:14:05.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.257+0000 7fa7a4a00640 1 --2- 172.21.15.161:0/596738626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a0071770 0x7fa7a0069ed0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.257+0000 7fa7a4a00640 1 -- 172.21.15.161:0/596738626 >> 172.21.15.161:0/596738626 conn(0x7fa7a006ba00 msgr2=0x7fa7a00b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:05.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.257+0000 7fa7a4a00640 1 -- 172.21.15.161:0/596738626 shutdown_connections 2024-05-22T21:14:05.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.257+0000 7fa7a4a00640 1 -- 172.21.15.161:0/596738626 wait complete. 2024-05-22T21:14:05.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.257+0000 7fa7a4a00640 1 Processor -- start 2024-05-22T21:14:05.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.257+0000 7fa7a4a00640 1 -- start start 2024-05-22T21:14:05.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa7a4a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7a006a410 0x7fa7a015fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa7a4a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7a0071770 0x7fa7a015a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa7a4a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a0146970 0x7fa7a0133530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa7a4a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0160610 con 0x7fa7a0146970 2024-05-22T21:14:05.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa7a4a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0160420 con 0x7fa7a0071770 2024-05-22T21:14:05.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa7a4a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0134c30 con 0x7fa7a006a410 2024-05-22T21:14:05.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa79f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a0146970 0x7fa7a0133530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa79f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a0146970 0x7fa7a0133530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59372/0 (socket says 172.21.15.161:59372) 2024-05-22T21:14:05.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa79f200640 1 -- 172.21.15.161:0/2167554267 learned_addr learned my addr 172.21.15.161:0/2167554267 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:05.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa79e800640 1 --2- 172.21.15.161:0/2167554267 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7a006a410 0x7fa7a015fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa79de00640 1 --2- 172.21.15.161:0/2167554267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7a0071770 0x7fa7a015a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa79f200640 1 -- 172.21.15.161:0/2167554267 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7a006a410 msgr2=0x7fa7a015fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa79f200640 1 --2- 172.21.15.161:0/2167554267 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7a006a410 0x7fa7a015fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa79f200640 1 -- 172.21.15.161:0/2167554267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7a0071770 msgr2=0x7fa7a015a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa79f200640 1 --2- 172.21.15.161:0/2167554267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7a0071770 0x7fa7a015a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa79f200640 1 -- 172.21.15.161:0/2167554267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c0094d0 con 0x7fa7a0146970 2024-05-22T21:14:05.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa79f200640 1 --2- 172.21.15.161:0/2167554267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a0146970 0x7fa7a0133530 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa79800c780 tx=0x7fa79800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:05.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa796a00640 1 -- 172.21.15.161:0/2167554267 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798004b70 con 0x7fa7a0146970 2024-05-22T21:14:05.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa796a00640 1 -- 172.21.15.161:0/2167554267 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79800ce90 con 0x7fa7a0146970 2024-05-22T21:14:05.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa796a00640 1 -- 172.21.15.161:0/2167554267 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79800a850 con 0x7fa7a0146970 2024-05-22T21:14:05.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa7a4a00640 1 -- 172.21.15.161:0/2167554267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a0134300 con 0x7fa7a0146970 2024-05-22T21:14:05.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa7a4a00640 1 -- 172.21.15.161:0/2167554267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a01348c0 con 0x7fa7a0146970 2024-05-22T21:14:05.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa7a4a00640 1 -- 172.21.15.161:0/2167554267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a0067070 con 0x7fa7a0146970 2024-05-22T21:14:05.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.261+0000 7fa796a00640 1 -- 172.21.15.161:0/2167554267 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa79800a9b0 con 0x7fa7a0146970 2024-05-22T21:14:05.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.265+0000 7fa796a00640 1 --2- 172.21.15.161:0/2167554267 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa780066690 0x7fa780068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.265+0000 7fa796a00640 1 -- 172.21.15.161:0/2167554267 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7980842a0 con 0x7fa7a0146970 2024-05-22T21:14:05.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.265+0000 7fa79e800640 1 --2- 172.21.15.161:0/2167554267 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa780066690 0x7fa780068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.265+0000 7fa79e800640 1 --2- 172.21.15.161:0/2167554267 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa780066690 0x7fa780068b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa790000f70 tx=0x7fa790007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:05.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.269+0000 7fa796a00640 1 -- 172.21.15.161:0/2167554267 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa798051620 con 0x7fa7a0146970 2024-05-22T21:14:05.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.421+0000 7fa7a4a00640 1 -- 172.21.15.161:0/2167554267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7a00b07a0 con 0x7fa7a0146970 2024-05-22T21:14:05.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.425+0000 7fa796a00640 1 -- 172.21.15.161:0/2167554267 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7fa798054c40 con 0x7fa7a0146970 2024-05-22T21:14:05.428 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:05.428 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:05.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.425+0000 7fa7a4a00640 1 -- 172.21.15.161:0/2167554267 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa780066690 msgr2=0x7fa780068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.425+0000 7fa7a4a00640 1 --2- 172.21.15.161:0/2167554267 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa780066690 0x7fa780068b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa790000f70 tx=0x7fa790007df0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.425+0000 7fa7a4a00640 1 -- 172.21.15.161:0/2167554267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a0146970 msgr2=0x7fa7a0133530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.425+0000 7fa7a4a00640 1 --2- 172.21.15.161:0/2167554267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a0146970 0x7fa7a0133530 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa79800c780 tx=0x7fa79800cc40 comp rx=0 tx=0).stop 2024-05-22T21:14:05.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.425+0000 7fa7a4a00640 1 -- 172.21.15.161:0/2167554267 shutdown_connections 2024-05-22T21:14:05.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.425+0000 7fa7a4a00640 1 --2- 172.21.15.161:0/2167554267 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa780066690 0x7fa780068b50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.425+0000 7fa7a4a00640 1 --2- 172.21.15.161:0/2167554267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7a0146970 0x7fa7a0133530 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.425+0000 7fa7a4a00640 1 --2- 172.21.15.161:0/2167554267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7a0071770 0x7fa7a015a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.425+0000 7fa7a4a00640 1 --2- 172.21.15.161:0/2167554267 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7a006a410 0x7fa7a015fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.425+0000 7fa7a4a00640 1 -- 172.21.15.161:0/2167554267 >> 172.21.15.161:0/2167554267 conn(0x7fa7a006ba00 msgr2=0x7fa7a013aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:05.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.425+0000 7fa7a4a00640 1 -- 172.21.15.161:0/2167554267 shutdown_connections 2024-05-22T21:14:05.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.425+0000 7fa7a4a00640 1 -- 172.21.15.161:0/2167554267 wait complete. 2024-05-22T21:14:05.450 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:13:21.237503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:13:22.425891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:14:05.450 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:14:05.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.585+0000 7fee27e00640 1 Processor -- start 2024-05-22T21:14:05.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.585+0000 7fee27e00640 1 -- start start 2024-05-22T21:14:05.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.585+0000 7fee27e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fee2006c2a0 0x7fee2006c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.585+0000 7fee27e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fee200b3370 0x7fee200b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.585+0000 7fee27e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee2015d9d0 con 0x7fee201402b0 2024-05-22T21:14:05.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.585+0000 7fee27e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee2006cc90 con 0x7fee200b3370 2024-05-22T21:14:05.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.585+0000 7fee27e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee2015d7c0 con 0x7fee2006c2a0 2024-05-22T21:14:05.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.585+0000 7fee25e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fee200b3370 0x7fee200b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.585+0000 7fee25400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fee201402b0 0x7fee20070590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33798/0 (socket says 172.21.15.161:33798) 2024-05-22T21:14:05.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.585+0000 7fee25400640 1 -- 172.21.15.161:0/1604144522 learned_addr learned my addr 172.21.15.161:0/1604144522 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:05.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.585+0000 7fee24a00640 1 --2- 172.21.15.161:0/1604144522 >> v2:172.21.15.179:3300/0 conn(0x7fee2006c2a0 0x7fee2006c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.585+0000 7fee25e00640 1 -- 172.21.15.161:0/1604144522 >> v2:172.21.15.179:3300/0 conn(0x7fee2006c2a0 msgr2=0x7fee2006c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:05.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.585+0000 7fee25e00640 1 --2- 172.21.15.161:0/1604144522 >> v2:172.21.15.179:3300/0 conn(0x7fee2006c2a0 0x7fee2006c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.585+0000 7fee25e00640 1 -- 172.21.15.161:0/1604144522 >> v1:172.21.15.161:6789/0 conn(0x7fee201402b0 legacy=0x7fee20070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.585+0000 7fee25e00640 1 -- 172.21.15.161:0/1604144522 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2015dcb0 con 0x7fee200b3370 2024-05-22T21:14:05.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.585+0000 7fee1fe00640 1 -- 172.21.15.161:0/1604144522 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016905184 0 0) 0x7fee2015d9d0 con 0x7fee201402b0 2024-05-22T21:14:05.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.589+0000 7fee25e00640 1 --2- 172.21.15.161:0/1604144522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee200b3370 0x7fee200b37e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fee20157df0 tx=0x7fee1801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7a94ad4186e130 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:05.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.589+0000 7fee1fe00640 1 -- 172.21.15.161:0/1604144522 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee180ab5f0 con 0x7fee200b3370 2024-05-22T21:14:05.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.589+0000 7fee1fe00640 1 -- 172.21.15.161:0/1604144522 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee180ab750 con 0x7fee200b3370 2024-05-22T21:14:05.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.589+0000 7fee1fe00640 1 -- 172.21.15.161:0/1604144522 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee180abae0 con 0x7fee200b3370 2024-05-22T21:14:05.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.589+0000 7fee27e00640 1 -- 172.21.15.161:0/1604144522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee200b3370 msgr2=0x7fee200b37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.589+0000 7fee27e00640 1 --2- 172.21.15.161:0/1604144522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee200b3370 0x7fee200b37e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fee20157df0 tx=0x7fee1801c3f0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.589+0000 7fee27e00640 1 -- 172.21.15.161:0/1604144522 shutdown_connections 2024-05-22T21:14:05.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.589+0000 7fee27e00640 1 --2- 172.21.15.161:0/1604144522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee200b3370 0x7fee200b37e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.589+0000 7fee27e00640 1 --2- 172.21.15.161:0/1604144522 >> v2:172.21.15.179:3300/0 conn(0x7fee2006c2a0 0x7fee2006c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.589+0000 7fee27e00640 1 -- 172.21.15.161:0/1604144522 >> 172.21.15.161:0/1604144522 conn(0x7fee2006a2d0 msgr2=0x7fee2006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:05.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.589+0000 7fee27e00640 1 -- 172.21.15.161:0/1604144522 shutdown_connections 2024-05-22T21:14:05.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.589+0000 7fee27e00640 1 -- 172.21.15.161:0/1604144522 wait complete. 2024-05-22T21:14:05.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.589+0000 7fee27e00640 1 Processor -- start 2024-05-22T21:14:05.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.589+0000 7fee27e00640 1 -- start start 2024-05-22T21:14:05.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee27e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee2006c2a0 0x7fee20142560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee27e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee200b3370 0x7fee2015efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee27e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee201402b0 0x7fee20138470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee27e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee201608f0 con 0x7fee2006c2a0 2024-05-22T21:14:05.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee27e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee20160050 con 0x7fee200b3370 2024-05-22T21:14:05.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee27e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee20160330 con 0x7fee201402b0 2024-05-22T21:14:05.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee25400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee2006c2a0 0x7fee20142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee25400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee2006c2a0 0x7fee20142560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59388/0 (socket says 172.21.15.161:59388) 2024-05-22T21:14:05.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee25400640 1 -- 172.21.15.161:0/223274549 learned_addr learned my addr 172.21.15.161:0/223274549 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:05.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee24a00640 1 --2- 172.21.15.161:0/223274549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee200b3370 0x7fee2015efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee25e00640 1 --2- 172.21.15.161:0/223274549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee201402b0 0x7fee20138470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee25400640 1 -- 172.21.15.161:0/223274549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee201402b0 msgr2=0x7fee20138470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee25400640 1 --2- 172.21.15.161:0/223274549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee201402b0 0x7fee20138470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee25400640 1 -- 172.21.15.161:0/223274549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee200b3370 msgr2=0x7fee2015efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee25400640 1 --2- 172.21.15.161:0/223274549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee200b3370 0x7fee2015efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee25400640 1 -- 172.21.15.161:0/223274549 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee18009420 con 0x7fee2006c2a0 2024-05-22T21:14:05.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee25400640 1 --2- 172.21.15.161:0/223274549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee2006c2a0 0x7fee20142560 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fee14001570 tx=0x7fee1400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:05.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee1d600640 1 -- 172.21.15.161:0/223274549 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1400a640 con 0x7fee2006c2a0 2024-05-22T21:14:05.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee1d600640 1 -- 172.21.15.161:0/223274549 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1400a7c0 con 0x7fee2006c2a0 2024-05-22T21:14:05.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee1d600640 1 -- 172.21.15.161:0/223274549 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1400ab70 con 0x7fee2006c2a0 2024-05-22T21:14:05.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee27e00640 1 -- 172.21.15.161:0/223274549 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee20160610 con 0x7fee2006c2a0 2024-05-22T21:14:05.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee27e00640 1 -- 172.21.15.161:0/223274549 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee20142aa0 con 0x7fee2006c2a0 2024-05-22T21:14:05.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee27e00640 1 -- 172.21.15.161:0/223274549 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2015e920 con 0x7fee2006c2a0 2024-05-22T21:14:05.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.593+0000 7fee1d600640 1 -- 172.21.15.161:0/223274549 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fee1400acd0 con 0x7fee2006c2a0 2024-05-22T21:14:05.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.597+0000 7fee1d600640 1 --2- 172.21.15.161:0/223274549 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fee00066690 0x7fee00068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.597+0000 7fee1d600640 1 -- 172.21.15.161:0/223274549 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee14083890 con 0x7fee2006c2a0 2024-05-22T21:14:05.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.597+0000 7fee24a00640 1 --2- 172.21.15.161:0/223274549 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fee00066690 0x7fee00068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.597+0000 7fee24a00640 1 --2- 172.21.15.161:0/223274549 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fee00066690 0x7fee00068b50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fee200ce2e0 tx=0x7fee0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:05.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.601+0000 7fee1d600640 1 -- 172.21.15.161:0/223274549 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fee2015e920 con 0x7fee2006c2a0 2024-05-22T21:14:05.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.753+0000 7fee27e00640 1 -- 172.21.15.161:0/223274549 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee2015e920 con 0x7fee2006c2a0 2024-05-22T21:14:05.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.753+0000 7fee1d600640 1 -- 172.21.15.161:0/223274549 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee2015e920 con 0x7fee2006c2a0 2024-05-22T21:14:05.758 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:14:05.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.753+0000 7fee27e00640 1 -- 172.21.15.161:0/223274549 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fee00066690 msgr2=0x7fee00068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.753+0000 7fee27e00640 1 --2- 172.21.15.161:0/223274549 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fee00066690 0x7fee00068b50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fee200ce2e0 tx=0x7fee0c008040 comp rx=0 tx=0).stop 2024-05-22T21:14:05.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.753+0000 7fee27e00640 1 -- 172.21.15.161:0/223274549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee2006c2a0 msgr2=0x7fee20142560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.753+0000 7fee27e00640 1 --2- 172.21.15.161:0/223274549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee2006c2a0 0x7fee20142560 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fee14001570 tx=0x7fee1400cd20 comp rx=0 tx=0).stop 2024-05-22T21:14:05.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.753+0000 7fee27e00640 1 -- 172.21.15.161:0/223274549 shutdown_connections 2024-05-22T21:14:05.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.753+0000 7fee27e00640 1 --2- 172.21.15.161:0/223274549 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fee00066690 0x7fee00068b50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.753+0000 7fee27e00640 1 --2- 172.21.15.161:0/223274549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee201402b0 0x7fee20138470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.753+0000 7fee27e00640 1 --2- 172.21.15.161:0/223274549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee200b3370 0x7fee2015efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.753+0000 7fee27e00640 1 --2- 172.21.15.161:0/223274549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee2006c2a0 0x7fee20142560 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.753+0000 7fee27e00640 1 -- 172.21.15.161:0/223274549 >> 172.21.15.161:0/223274549 conn(0x7fee2006a2d0 msgr2=0x7fee2015c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:05.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.753+0000 7fee27e00640 1 -- 172.21.15.161:0/223274549 shutdown_connections 2024-05-22T21:14:05.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.753+0000 7fee27e00640 1 -- 172.21.15.161:0/223274549 wait complete. 2024-05-22T21:14:05.779 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:14:05.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f0385c00640 1 Processor -- start 2024-05-22T21:14:05.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f0385c00640 1 -- start start 2024-05-22T21:14:05.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f0385c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f038013cfc0 0x7f03801402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f0385c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f038015af60 0x7f038013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f0385c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03801400d0 con 0x7f038013fc80 2024-05-22T21:14:05.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f0385c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f038013d3a0 con 0x7f038015af60 2024-05-22T21:14:05.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f0385c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03800cd920 con 0x7f038013cfc0 2024-05-22T21:14:05.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f037e000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f038015af60 0x7f038013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f037e000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f038015af60 0x7f038013f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:43690/0 (socket says 172.21.15.161:43690) 2024-05-22T21:14:05.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f037e000640 1 -- 172.21.15.161:0/1608767531 learned_addr learned my addr 172.21.15.161:0/1608767531 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:05.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f037ea00640 1 --2- 172.21.15.161:0/1608767531 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f038013cfc0 0x7f03801402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f037e000640 1 -- 172.21.15.161:0/1608767531 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f038013cfc0 msgr2=0x7f03801402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:05.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f037e000640 1 --2- 172.21.15.161:0/1608767531 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f038013cfc0 0x7f03801402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f037e000640 1 -- 172.21.15.161:0/1608767531 >> v1:172.21.15.179:6789/0 conn(0x7f038013fc80 legacy=0x7f038015dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f037e000640 1 -- 172.21.15.161:0/1608767531 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03800cdc00 con 0x7f038015af60 2024-05-22T21:14:05.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f037d600640 1 -- 172.21.15.161:0/1608767531 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831692225 0 0) 0x7f03780014a0 con 0x7f038013fc80 2024-05-22T21:14:05.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.913+0000 7f037ea00640 1 --2- 172.21.15.161:0/1608767531 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f038013cfc0 0x7f03801402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:05.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.917+0000 7f037e000640 1 --2- 172.21.15.161:0/1608767531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f038015af60 0x7f038013f740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f036c009870 tx=0x7f036c01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebfbc4378c32e2bd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:05.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.917+0000 7f037d600640 1 -- 172.21.15.161:0/1608767531 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c0ab600 con 0x7f038015af60 2024-05-22T21:14:05.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.917+0000 7f037d600640 1 -- 172.21.15.161:0/1608767531 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036c0a7050 con 0x7f038015af60 2024-05-22T21:14:05.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.917+0000 7f037d600640 1 -- 172.21.15.161:0/1608767531 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c0ab980 con 0x7f038015af60 2024-05-22T21:14:05.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.917+0000 7f0385c00640 1 -- 172.21.15.161:0/1608767531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f038015af60 msgr2=0x7f038013f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.917+0000 7f0385c00640 1 --2- 172.21.15.161:0/1608767531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f038015af60 0x7f038013f740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f036c009870 tx=0x7f036c01c4d0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.917+0000 7f0385c00640 1 -- 172.21.15.161:0/1608767531 shutdown_connections 2024-05-22T21:14:05.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.917+0000 7f0385c00640 1 --2- 172.21.15.161:0/1608767531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f038015af60 0x7f038013f740 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.917+0000 7f0385c00640 1 --2- 172.21.15.161:0/1608767531 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f038013cfc0 0x7f03801402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.917+0000 7f0385c00640 1 -- 172.21.15.161:0/1608767531 >> 172.21.15.161:0/1608767531 conn(0x7f038006a2d0 msgr2=0x7f038006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:05.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.917+0000 7f0385c00640 1 -- 172.21.15.161:0/1608767531 shutdown_connections 2024-05-22T21:14:05.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.917+0000 7f0385c00640 1 -- 172.21.15.161:0/1608767531 wait complete. 2024-05-22T21:14:05.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f0385c00640 1 Processor -- start 2024-05-22T21:14:05.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f0385c00640 1 -- start start 2024-05-22T21:14:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f0385c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f038013cfc0 0x7f038013a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f0385c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f038013fc80 0x7f038013e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f0385c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f038015af60 0x7f03800ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f0385c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0380141fb0 con 0x7f038013fc80 2024-05-22T21:14:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f0385c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038013ea40 con 0x7f038015af60 2024-05-22T21:14:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f0385c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038013aa20 con 0x7f038013cfc0 2024-05-22T21:14:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f037e000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f038013fc80 0x7f038013e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f037e000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f038013fc80 0x7f038013e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39622/0 (socket says 172.21.15.161:39622) 2024-05-22T21:14:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f037e000640 1 -- 172.21.15.161:0/1140237797 learned_addr learned my addr 172.21.15.161:0/1140237797 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f037ea00640 1 --2- 172.21.15.161:0/1140237797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f038013cfc0 0x7f038013a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f037f400640 1 --2- 172.21.15.161:0/1140237797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f038015af60 0x7f03800ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f037e000640 1 -- 172.21.15.161:0/1140237797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f038013cfc0 msgr2=0x7f038013a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f037e000640 1 --2- 172.21.15.161:0/1140237797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f038013cfc0 0x7f038013a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f037e000640 1 -- 172.21.15.161:0/1140237797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f038015af60 msgr2=0x7f03800ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f037e000640 1 --2- 172.21.15.161:0/1140237797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f038015af60 0x7f03800ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f037e000640 1 -- 172.21.15.161:0/1140237797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0370009420 con 0x7f038013fc80 2024-05-22T21:14:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f037f400640 1 --2- 172.21.15.161:0/1140237797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f038015af60 0x7f03800ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f037e000640 1 --2- 172.21.15.161:0/1140237797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f038013fc80 0x7f038013e1f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f036c0ae040 tx=0x7f036c019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f0376a00640 1 -- 172.21.15.161:0/1140237797 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c0b6420 con 0x7f038013fc80 2024-05-22T21:14:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f0376a00640 1 -- 172.21.15.161:0/1140237797 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036c0b65a0 con 0x7f038013fc80 2024-05-22T21:14:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f0385c00640 1 -- 172.21.15.161:0/1140237797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036c0094d0 con 0x7f038013fc80 2024-05-22T21:14:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.921+0000 7f0376a00640 1 -- 172.21.15.161:0/1140237797 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c0b6950 con 0x7f038013fc80 2024-05-22T21:14:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.925+0000 7f0385c00640 1 -- 172.21.15.161:0/1140237797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0380142a00 con 0x7f038013fc80 2024-05-22T21:14:05.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.925+0000 7f0385c00640 1 -- 172.21.15.161:0/1140237797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0380067220 con 0x7f038013fc80 2024-05-22T21:14:05.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.925+0000 7f0376a00640 1 -- 172.21.15.161:0/1140237797 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f036c0b2040 con 0x7f038013fc80 2024-05-22T21:14:05.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.925+0000 7f0376a00640 1 --2- 172.21.15.161:0/1140237797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0360066690 0x7f0360068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:05.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.925+0000 7f0376a00640 1 -- 172.21.15.161:0/1140237797 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f036c11e560 con 0x7f038013fc80 2024-05-22T21:14:05.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.925+0000 7f037ea00640 1 --2- 172.21.15.161:0/1140237797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0360066690 0x7f0360068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:05.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.925+0000 7f037ea00640 1 --2- 172.21.15.161:0/1140237797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0360066690 0x7f0360068b50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0370009c50 tx=0x7f0370009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:05.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:05.929+0000 7f0376a00640 1 -- 172.21.15.161:0/1140237797 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f036c0eb8e0 con 0x7f038013fc80 2024-05-22T21:14:06.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.081+0000 7f0385c00640 1 -- 172.21.15.161:0/1140237797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0380071670 con 0x7f038013fc80 2024-05-22T21:14:06.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.085+0000 7f0376a00640 1 -- 172.21.15.161:0/1140237797 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f036c0eef00 con 0x7f038013fc80 2024-05-22T21:14:06.088 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:14:06.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.085+0000 7f0385c00640 1 -- 172.21.15.161:0/1140237797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0360066690 msgr2=0x7f0360068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.085+0000 7f0385c00640 1 --2- 172.21.15.161:0/1140237797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0360066690 0x7f0360068b50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0370009c50 tx=0x7f0370009190 comp rx=0 tx=0).stop 2024-05-22T21:14:06.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.085+0000 7f0385c00640 1 -- 172.21.15.161:0/1140237797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f038013fc80 msgr2=0x7f038013e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.085+0000 7f0385c00640 1 --2- 172.21.15.161:0/1140237797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f038013fc80 0x7f038013e1f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f036c0ae040 tx=0x7f036c019270 comp rx=0 tx=0).stop 2024-05-22T21:14:06.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.085+0000 7f0385c00640 1 -- 172.21.15.161:0/1140237797 shutdown_connections 2024-05-22T21:14:06.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.085+0000 7f0385c00640 1 --2- 172.21.15.161:0/1140237797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0360066690 0x7f0360068b50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.085+0000 7f0385c00640 1 --2- 172.21.15.161:0/1140237797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f038015af60 0x7f03800ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.085+0000 7f0385c00640 1 --2- 172.21.15.161:0/1140237797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f038013fc80 0x7f038013e1f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.085+0000 7f0385c00640 1 --2- 172.21.15.161:0/1140237797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f038013cfc0 0x7f038013a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.085+0000 7f0385c00640 1 -- 172.21.15.161:0/1140237797 >> 172.21.15.161:0/1140237797 conn(0x7f038006a2d0 msgr2=0x7f03800ad810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:06.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.085+0000 7f0385c00640 1 -- 172.21.15.161:0/1140237797 shutdown_connections 2024-05-22T21:14:06.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.085+0000 7f0385c00640 1 -- 172.21.15.161:0/1140237797 wait complete. 2024-05-22T21:14:06.109 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:14:06.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.241+0000 7f1b57a00640 1 Processor -- start 2024-05-22T21:14:06.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.241+0000 7f1b57a00640 1 -- start start 2024-05-22T21:14:06.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.241+0000 7f1b57a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1b50143c40 0x7f1b5015b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.241+0000 7f1b57a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b5015efb0 con 0x7f1b5006c9b0 2024-05-22T21:14:06.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.241+0000 7f1b57a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b5015f290 con 0x7f1b501444a0 2024-05-22T21:14:06.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.241+0000 7f1b57a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5015f570 con 0x7f1b50143c40 2024-05-22T21:14:06.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.241+0000 7f1b55a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1b5006c9b0 0x7f1b500b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59162/0 (socket says 172.21.15.161:59162) 2024-05-22T21:14:06.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.241+0000 7f1b55a00640 1 -- 172.21.15.161:0/3831837326 learned_addr learned my addr 172.21.15.161:0/3831837326 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:06.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.241+0000 7f1b55000640 1 --2- 172.21.15.161:0/3831837326 >> v2:172.21.15.179:3300/0 conn(0x7f1b50143c40 0x7f1b5015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.241+0000 7f1b4fe00640 1 -- 172.21.15.161:0/3831837326 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191493641 0 0) 0x7f1b5015efb0 con 0x7f1b5006c9b0 2024-05-22T21:14:06.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.241+0000 7f1b4fe00640 1 -- 172.21.15.161:0/3831837326 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b34003530 con 0x7f1b5006c9b0 2024-05-22T21:14:06.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.245+0000 7f1b55000640 1 -- 172.21.15.161:0/3831837326 >> v1:172.21.15.179:6789/0 conn(0x7f1b501444a0 legacy=0x7f1b5006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.245+0000 7f1b55000640 1 -- 172.21.15.161:0/3831837326 >> v1:172.21.15.161:6789/0 conn(0x7f1b5006c9b0 legacy=0x7f1b500b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.245+0000 7f1b55000640 1 -- 172.21.15.161:0/3831837326 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5015f850 con 0x7f1b50143c40 2024-05-22T21:14:06.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.245+0000 7f1b55000640 1 --2- 172.21.15.161:0/3831837326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b50143c40 0x7f1b5015b600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1b400097c0 tx=0x7f1b4001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d03e442e0f0b0f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:06.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.245+0000 7f1b4fe00640 1 -- 172.21.15.161:0/3831837326 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b400ab600 con 0x7f1b50143c40 2024-05-22T21:14:06.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.245+0000 7f1b4fe00640 1 -- 172.21.15.161:0/3831837326 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b400ab760 con 0x7f1b50143c40 2024-05-22T21:14:06.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.245+0000 7f1b4fe00640 1 -- 172.21.15.161:0/3831837326 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b400abaf0 con 0x7f1b50143c40 2024-05-22T21:14:06.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.245+0000 7f1b57a00640 1 -- 172.21.15.161:0/3831837326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b50143c40 msgr2=0x7f1b5015b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.245+0000 7f1b57a00640 1 --2- 172.21.15.161:0/3831837326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b50143c40 0x7f1b5015b600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1b400097c0 tx=0x7f1b4001c4a0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.245+0000 7f1b57a00640 1 -- 172.21.15.161:0/3831837326 shutdown_connections 2024-05-22T21:14:06.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.245+0000 7f1b57a00640 1 --2- 172.21.15.161:0/3831837326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b50143c40 0x7f1b5015b600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.245+0000 7f1b57a00640 1 -- 172.21.15.161:0/3831837326 >> 172.21.15.161:0/3831837326 conn(0x7f1b5006a2d0 msgr2=0x7f1b5006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:06.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b57a00640 1 -- 172.21.15.161:0/3831837326 shutdown_connections 2024-05-22T21:14:06.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b57a00640 1 -- 172.21.15.161:0/3831837326 wait complete. 2024-05-22T21:14:06.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b57a00640 1 Processor -- start 2024-05-22T21:14:06.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b57a00640 1 -- start start 2024-05-22T21:14:06.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b57a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b5006c9b0 0x7f1b500a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b57a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1b50143c40 0x7f1b50136ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b57a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b501444a0 0x7f1b50137710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b57a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b50139ab0 con 0x7f1b5006c9b0 2024-05-22T21:14:06.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b57a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b50137530 con 0x7f1b501444a0 2024-05-22T21:14:06.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b57a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500a8b30 con 0x7f1b50143c40 2024-05-22T21:14:06.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b55000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b5006c9b0 0x7f1b500a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b55000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b5006c9b0 0x7f1b500a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39628/0 (socket says 172.21.15.161:39628) 2024-05-22T21:14:06.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b55000640 1 -- 172.21.15.161:0/147533098 learned_addr learned my addr 172.21.15.161:0/147533098 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:06.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b55a00640 1 --2- 172.21.15.161:0/147533098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b501444a0 0x7f1b50137710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b47e00640 1 --2- 172.21.15.161:0/147533098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1b50143c40 0x7f1b50136ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b55000640 1 -- 172.21.15.161:0/147533098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1b50143c40 msgr2=0x7f1b50136ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b55000640 1 --2- 172.21.15.161:0/147533098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1b50143c40 0x7f1b50136ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b55000640 1 -- 172.21.15.161:0/147533098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b501444a0 msgr2=0x7f1b50137710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b55000640 1 --2- 172.21.15.161:0/147533098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b501444a0 0x7f1b50137710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b55000640 1 -- 172.21.15.161:0/147533098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b40009420 con 0x7f1b5006c9b0 2024-05-22T21:14:06.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b55000640 1 --2- 172.21.15.161:0/147533098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b5006c9b0 0x7f1b500a84a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1b400ad040 tx=0x7f1b4001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:06.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b4d600640 1 -- 172.21.15.161:0/147533098 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b400bf610 con 0x7f1b5006c9b0 2024-05-22T21:14:06.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b4d600640 1 -- 172.21.15.161:0/147533098 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b400bf7b0 con 0x7f1b5006c9b0 2024-05-22T21:14:06.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b57a00640 1 -- 172.21.15.161:0/147533098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b500a8e10 con 0x7f1b5006c9b0 2024-05-22T21:14:06.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.249+0000 7f1b4d600640 1 -- 172.21.15.161:0/147533098 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b400bfb60 con 0x7f1b5006c9b0 2024-05-22T21:14:06.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.253+0000 7f1b57a00640 1 -- 172.21.15.161:0/147533098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b500a93d0 con 0x7f1b5006c9b0 2024-05-22T21:14:06.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.253+0000 7f1b57a00640 1 -- 172.21.15.161:0/147533098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b50067220 con 0x7f1b5006c9b0 2024-05-22T21:14:06.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.253+0000 7f1b4d600640 1 -- 172.21.15.161:0/147533098 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1b400b5400 con 0x7f1b5006c9b0 2024-05-22T21:14:06.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.253+0000 7f1b4d600640 1 --2- 172.21.15.161:0/147533098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1b2c066690 0x7f1b2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.253+0000 7f1b4d600640 1 -- 172.21.15.161:0/147533098 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b4011eba0 con 0x7f1b5006c9b0 2024-05-22T21:14:06.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.253+0000 7f1b47e00640 1 --2- 172.21.15.161:0/147533098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1b2c066690 0x7f1b2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.253+0000 7f1b47e00640 1 --2- 172.21.15.161:0/147533098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1b2c066690 0x7f1b2c068b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1b5015a9f0 tx=0x7f1b38009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:06.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.257+0000 7f1b4d600640 1 -- 172.21.15.161:0/147533098 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1b40123070 con 0x7f1b5006c9b0 2024-05-22T21:14:06.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.413+0000 7f1b57a00640 1 -- 172.21.15.161:0/147533098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b500ceaa0 con 0x7f1b5006c9b0 2024-05-22T21:14:06.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.413+0000 7f1b4d600640 1 -- 172.21.15.161:0/147533098 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b400ebf20 con 0x7f1b5006c9b0 2024-05-22T21:14:06.417 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:14:06.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.413+0000 7f1b57a00640 1 -- 172.21.15.161:0/147533098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1b2c066690 msgr2=0x7f1b2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.413+0000 7f1b57a00640 1 --2- 172.21.15.161:0/147533098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1b2c066690 0x7f1b2c068b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1b5015a9f0 tx=0x7f1b38009040 comp rx=0 tx=0).stop 2024-05-22T21:14:06.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.413+0000 7f1b57a00640 1 -- 172.21.15.161:0/147533098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b5006c9b0 msgr2=0x7f1b500a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.413+0000 7f1b57a00640 1 --2- 172.21.15.161:0/147533098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b5006c9b0 0x7f1b500a84a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1b400ad040 tx=0x7f1b4001ace0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.413+0000 7f1b57a00640 1 -- 172.21.15.161:0/147533098 shutdown_connections 2024-05-22T21:14:06.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.413+0000 7f1b57a00640 1 --2- 172.21.15.161:0/147533098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1b2c066690 0x7f1b2c068b50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.413+0000 7f1b57a00640 1 --2- 172.21.15.161:0/147533098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b501444a0 0x7f1b50137710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.413+0000 7f1b57a00640 1 --2- 172.21.15.161:0/147533098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1b50143c40 0x7f1b50136ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.413+0000 7f1b57a00640 1 --2- 172.21.15.161:0/147533098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b5006c9b0 0x7f1b500a84a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.413+0000 7f1b57a00640 1 -- 172.21.15.161:0/147533098 >> 172.21.15.161:0/147533098 conn(0x7f1b5006a2d0 msgr2=0x7f1b500b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:06.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.413+0000 7f1b57a00640 1 -- 172.21.15.161:0/147533098 shutdown_connections 2024-05-22T21:14:06.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.413+0000 7f1b57a00640 1 -- 172.21.15.161:0/147533098 wait complete. 2024-05-22T21:14:06.438 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:14:06.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.569+0000 7fa1f3a00640 1 Processor -- start 2024-05-22T21:14:06.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.569+0000 7fa1f3a00640 1 -- start start 2024-05-22T21:14:06.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.569+0000 7fa1f3a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa1ec143ec0 0x7fa1ec15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.569+0000 7fa1f3a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa1ec0715a0 0x7fa1ec06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.569+0000 7fa1f3a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ec1442a0 con 0x7fa1ec15ec90 2024-05-22T21:14:06.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.569+0000 7fa1f3a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1ec070260 con 0x7fa1ec143ec0 2024-05-22T21:14:06.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.569+0000 7fa1f3a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1ec070500 con 0x7fa1ec0715a0 2024-05-22T21:14:06.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.569+0000 7fa1f1000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa1ec143ec0 0x7fa1ec15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.569+0000 7fa1f1000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa1ec143ec0 0x7fa1ec15f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39642/0 (socket says 172.21.15.161:39642) 2024-05-22T21:14:06.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.569+0000 7fa1f1000640 1 -- 172.21.15.161:0/392513782 learned_addr learned my addr 172.21.15.161:0/392513782 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:06.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.569+0000 7fa1ebe00640 1 --2- 172.21.15.161:0/392513782 >> v2:172.21.15.179:3300/0 conn(0x7fa1ec0715a0 0x7fa1ec06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.569+0000 7fa1f1000640 1 -- 172.21.15.161:0/392513782 >> v2:172.21.15.179:3300/0 conn(0x7fa1ec0715a0 msgr2=0x7fa1ec06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:06.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.569+0000 7fa1f1000640 1 --2- 172.21.15.161:0/392513782 >> v2:172.21.15.179:3300/0 conn(0x7fa1ec0715a0 0x7fa1ec06fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.569+0000 7fa1f1000640 1 -- 172.21.15.161:0/392513782 >> v1:172.21.15.179:6789/0 conn(0x7fa1ec15ec90 legacy=0x7fa1ec06bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:06.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.569+0000 7fa1f1000640 1 -- 172.21.15.161:0/392513782 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec0707e0 con 0x7fa1ec143ec0 2024-05-22T21:14:06.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.569+0000 7fa1ebe00640 1 --2- 172.21.15.161:0/392513782 >> v2:172.21.15.179:3300/0 conn(0x7fa1ec0715a0 0x7fa1ec06fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:06.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.573+0000 7fa1f1000640 1 --2- 172.21.15.161:0/392513782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1ec143ec0 0x7fa1ec15f830 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec074430 tx=0x7fa1e001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c148b0c4117abb68 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:06.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.573+0000 7fa1eb400640 1 -- 172.21.15.161:0/392513782 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e00aa610 con 0x7fa1ec143ec0 2024-05-22T21:14:06.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.573+0000 7fa1eb400640 1 -- 172.21.15.161:0/392513782 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e00a6040 con 0x7fa1ec143ec0 2024-05-22T21:14:06.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.573+0000 7fa1eb400640 1 -- 172.21.15.161:0/392513782 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e00aa990 con 0x7fa1ec143ec0 2024-05-22T21:14:06.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.573+0000 7fa1f3a00640 1 -- 172.21.15.161:0/392513782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1ec143ec0 msgr2=0x7fa1ec15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.573+0000 7fa1f3a00640 1 --2- 172.21.15.161:0/392513782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1ec143ec0 0x7fa1ec15f830 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec074430 tx=0x7fa1e001c4b0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.573+0000 7fa1f3a00640 1 -- 172.21.15.161:0/392513782 shutdown_connections 2024-05-22T21:14:06.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.573+0000 7fa1f3a00640 1 --2- 172.21.15.161:0/392513782 >> v2:172.21.15.179:3300/0 conn(0x7fa1ec0715a0 0x7fa1ec06fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.573+0000 7fa1f3a00640 1 --2- 172.21.15.161:0/392513782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1ec143ec0 0x7fa1ec15f830 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.573+0000 7fa1f3a00640 1 -- 172.21.15.161:0/392513782 >> 172.21.15.161:0/392513782 conn(0x7fa1ec0b0bc0 msgr2=0x7fa1ec0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:06.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.573+0000 7fa1f3a00640 1 -- 172.21.15.161:0/392513782 shutdown_connections 2024-05-22T21:14:06.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.573+0000 7fa1f3a00640 1 -- 172.21.15.161:0/392513782 wait complete. 2024-05-22T21:14:06.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f3a00640 1 Processor -- start 2024-05-22T21:14:06.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f3a00640 1 -- start start 2024-05-22T21:14:06.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f3a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1ec143ec0 0x7fa1ec159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f3a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1ec15ec90 0x7fa1ec1f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f3a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1ec0ade70 0x7fa1ec15d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f3a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec138570 con 0x7fa1ec143ec0 2024-05-22T21:14:06.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f3a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec138850 con 0x7fa1ec15ec90 2024-05-22T21:14:06.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f3a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec138b30 con 0x7fa1ec0ade70 2024-05-22T21:14:06.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f1000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1ec143ec0 0x7fa1ec159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f1000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1ec143ec0 0x7fa1ec159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39652/0 (socket says 172.21.15.161:39652) 2024-05-22T21:14:06.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f1000640 1 -- 172.21.15.161:0/1768669921 learned_addr learned my addr 172.21.15.161:0/1768669921 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:06.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1ebe00640 1 --2- 172.21.15.161:0/1768669921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1ec15ec90 0x7fa1ec1f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f1a00640 1 --2- 172.21.15.161:0/1768669921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1ec0ade70 0x7fa1ec15d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f1000640 1 -- 172.21.15.161:0/1768669921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1ec0ade70 msgr2=0x7fa1ec15d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f1000640 1 --2- 172.21.15.161:0/1768669921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1ec0ade70 0x7fa1ec15d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f1000640 1 -- 172.21.15.161:0/1768669921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1ec15ec90 msgr2=0x7fa1ec1f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f1000640 1 --2- 172.21.15.161:0/1768669921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1ec15ec90 0x7fa1ec1f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f1000640 1 -- 172.21.15.161:0/1768669921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e00094d0 con 0x7fa1ec143ec0 2024-05-22T21:14:06.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f1000640 1 --2- 172.21.15.161:0/1768669921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1ec143ec0 0x7fa1ec159ef0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa1e001a0e0 tx=0x7fa1e001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:06.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1e8c00640 1 -- 172.21.15.161:0/1768669921 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e00b3600 con 0x7fa1ec143ec0 2024-05-22T21:14:06.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1e8c00640 1 -- 172.21.15.161:0/1768669921 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e00b3760 con 0x7fa1ec143ec0 2024-05-22T21:14:06.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f3a00640 1 -- 172.21.15.161:0/1768669921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ec138e10 con 0x7fa1ec143ec0 2024-05-22T21:14:06.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1e8c00640 1 -- 172.21.15.161:0/1768669921 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e00b3b10 con 0x7fa1ec143ec0 2024-05-22T21:14:06.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f3a00640 1 -- 172.21.15.161:0/1768669921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ec15a430 con 0x7fa1ec143ec0 2024-05-22T21:14:06.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.577+0000 7fa1f3a00640 1 -- 172.21.15.161:0/1768669921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ec0740e0 con 0x7fa1ec143ec0 2024-05-22T21:14:06.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.581+0000 7fa1e8c00640 1 -- 172.21.15.161:0/1768669921 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa1e00b3c70 con 0x7fa1ec143ec0 2024-05-22T21:14:06.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.581+0000 7fa1e8c00640 1 --2- 172.21.15.161:0/1768669921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1cc066690 0x7fa1cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.581+0000 7fa1e8c00640 1 -- 172.21.15.161:0/1768669921 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1ec06ce80 con 0x7fa1ec143ec0 2024-05-22T21:14:06.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.581+0000 7fa1ebe00640 1 --2- 172.21.15.161:0/1768669921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1cc066690 0x7fa1cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.581+0000 7fa1ebe00640 1 --2- 172.21.15.161:0/1768669921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1cc066690 0x7fa1cc068b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0cc660 tx=0x7fa1d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:06.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.585+0000 7fa1e8c00640 1 -- 172.21.15.161:0/1768669921 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa1ec0740e0 con 0x7fa1ec143ec0 2024-05-22T21:14:06.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.773+0000 7fa1f3a00640 1 -- 172.21.15.161:0/1768669921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1ec0740e0 con 0x7fa1ec143ec0 2024-05-22T21:14:06.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.777+0000 7fa1e8c00640 1 -- 172.21.15.161:0/1768669921 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4740 (secure 0 0 0) 0x7fa1ec0740e0 con 0x7fa1ec143ec0 2024-05-22T21:14:06.780 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:06.781 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3179612470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3179612470},{"type":"v1","addr":"172.21.15.179:6834","nonce":3179612470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2196055309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2196055309},{"type":"v1","addr":"172.21.15.161:6835","nonce":2196055309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1793681759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1793681759},{"type":"v1","addr":"172.21.15.161:6837","nonce":1793681759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:13:23.418412+0000","modified":"2024-05-22T21:13:26.442354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/3095599688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3095599688},{"type":"v1","addr":"172.21.15.179:6835","nonce":3095599688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-22T21:14:06.781 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 8 2024-05-22T21:14:06.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.777+0000 7fa1f3a00640 1 -- 172.21.15.161:0/1768669921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1cc066690 msgr2=0x7fa1cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.777+0000 7fa1f3a00640 1 --2- 172.21.15.161:0/1768669921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1cc066690 0x7fa1cc068b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0cc660 tx=0x7fa1d8008040 comp rx=0 tx=0).stop 2024-05-22T21:14:06.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.777+0000 7fa1f3a00640 1 -- 172.21.15.161:0/1768669921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1ec143ec0 msgr2=0x7fa1ec159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.777+0000 7fa1f3a00640 1 --2- 172.21.15.161:0/1768669921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1ec143ec0 0x7fa1ec159ef0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa1e001a0e0 tx=0x7fa1e001acf0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.777+0000 7fa1f3a00640 1 -- 172.21.15.161:0/1768669921 shutdown_connections 2024-05-22T21:14:06.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.777+0000 7fa1f3a00640 1 --2- 172.21.15.161:0/1768669921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1cc066690 0x7fa1cc068b50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.777+0000 7fa1f3a00640 1 --2- 172.21.15.161:0/1768669921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1ec0ade70 0x7fa1ec15d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.777+0000 7fa1f3a00640 1 --2- 172.21.15.161:0/1768669921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1ec15ec90 0x7fa1ec1f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.777+0000 7fa1f3a00640 1 --2- 172.21.15.161:0/1768669921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1ec143ec0 0x7fa1ec159ef0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.777+0000 7fa1f3a00640 1 -- 172.21.15.161:0/1768669921 >> 172.21.15.161:0/1768669921 conn(0x7fa1ec0b0bc0 msgr2=0x7fa1ec141050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:06.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.777+0000 7fa1f3a00640 1 -- 172.21.15.161:0/1768669921 shutdown_connections 2024-05-22T21:14:06.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.777+0000 7fa1f3a00640 1 -- 172.21.15.161:0/1768669921 wait complete. 2024-05-22T21:14:06.799 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:06.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.929+0000 7efdab000640 1 Processor -- start 2024-05-22T21:14:06.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.929+0000 7efdab000640 1 -- start start 2024-05-22T21:14:06.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.929+0000 7efdab000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efda40b4420 0x7efda406aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.929+0000 7efdab000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efda406b420 0x7efda406c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.929+0000 7efdab000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda415db70 con 0x7efda406c8f0 2024-05-22T21:14:06.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.929+0000 7efdab000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda406b8b0 con 0x7efda406b420 2024-05-22T21:14:06.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.929+0000 7efdab000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda40ad610 con 0x7efda40b4420 2024-05-22T21:14:06.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.929+0000 7efda3400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efda406b420 0x7efda406c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.929+0000 7efda3400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efda406b420 0x7efda406c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39654/0 (socket says 172.21.15.161:39654) 2024-05-22T21:14:06.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.929+0000 7efda3400640 1 -- 172.21.15.161:0/3951938282 learned_addr learned my addr 172.21.15.161:0/3951938282 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:06.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.929+0000 7efda3e00640 1 --2- 172.21.15.161:0/3951938282 >> v2:172.21.15.179:3300/0 conn(0x7efda40b4420 0x7efda406aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.929+0000 7efda3400640 1 -- 172.21.15.161:0/3951938282 >> v2:172.21.15.179:3300/0 conn(0x7efda40b4420 msgr2=0x7efda406aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:06.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.929+0000 7efda3400640 1 --2- 172.21.15.161:0/3951938282 >> v2:172.21.15.179:3300/0 conn(0x7efda40b4420 0x7efda406aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.929+0000 7efda3400640 1 -- 172.21.15.161:0/3951938282 >> v1:172.21.15.179:6789/0 conn(0x7efda406c8f0 legacy=0x7efda4160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:06.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.929+0000 7efda3400640 1 -- 172.21.15.161:0/3951938282 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda40b26f0 con 0x7efda406b420 2024-05-22T21:14:06.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.933+0000 7efda3400640 1 --2- 172.21.15.161:0/3951938282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda406b420 0x7efda406c3b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efda406ba70 tx=0x7efd9001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b3002ca0d88d09 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:06.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.933+0000 7efda2a00640 1 -- 172.21.15.161:0/3951938282 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd900ab620 con 0x7efda406b420 2024-05-22T21:14:06.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.933+0000 7efda2a00640 1 -- 172.21.15.161:0/3951938282 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd900ab780 con 0x7efda406b420 2024-05-22T21:14:06.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.933+0000 7efda2a00640 1 -- 172.21.15.161:0/3951938282 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd900abb10 con 0x7efda406b420 2024-05-22T21:14:06.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.933+0000 7efdab000640 1 -- 172.21.15.161:0/3951938282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda406b420 msgr2=0x7efda406c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.933+0000 7efdab000640 1 --2- 172.21.15.161:0/3951938282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda406b420 0x7efda406c3b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efda406ba70 tx=0x7efd9001c4e0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.933+0000 7efdab000640 1 -- 172.21.15.161:0/3951938282 shutdown_connections 2024-05-22T21:14:06.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.933+0000 7efdab000640 1 --2- 172.21.15.161:0/3951938282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda406b420 0x7efda406c3b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.933+0000 7efdab000640 1 --2- 172.21.15.161:0/3951938282 >> v2:172.21.15.179:3300/0 conn(0x7efda40b4420 0x7efda406aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.933+0000 7efdab000640 1 -- 172.21.15.161:0/3951938282 >> 172.21.15.161:0/3951938282 conn(0x7efda4070630 msgr2=0x7efda4070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:06.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.933+0000 7efdab000640 1 -- 172.21.15.161:0/3951938282 shutdown_connections 2024-05-22T21:14:06.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.933+0000 7efdab000640 1 -- 172.21.15.161:0/3951938282 wait complete. 2024-05-22T21:14:06.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efdab000640 1 Processor -- start 2024-05-22T21:14:06.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efdab000640 1 -- start start 2024-05-22T21:14:06.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efdab000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efda406b420 0x7efda40adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efdab000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda406c8f0 0x7efda40ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efdab000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efda40b4420 0x7efda414cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efdab000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda40cd050 con 0x7efda406c8f0 2024-05-22T21:14:06.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efdab000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda40cc440 con 0x7efda40b4420 2024-05-22T21:14:06.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efdab000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda40cc720 con 0x7efda406b420 2024-05-22T21:14:06.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efda3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda406c8f0 0x7efda40ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efda3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda406c8f0 0x7efda40ae490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39668/0 (socket says 172.21.15.161:39668) 2024-05-22T21:14:06.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efda3400640 1 -- 172.21.15.161:0/1141016713 learned_addr learned my addr 172.21.15.161:0/1141016713 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:06.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efda9000640 1 --2- 172.21.15.161:0/1141016713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efda40b4420 0x7efda414cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efda3e00640 1 --2- 172.21.15.161:0/1141016713 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efda406b420 0x7efda40adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efda3400640 1 -- 172.21.15.161:0/1141016713 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efda406b420 msgr2=0x7efda40adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efda3400640 1 --2- 172.21.15.161:0/1141016713 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efda406b420 0x7efda40adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efda3400640 1 -- 172.21.15.161:0/1141016713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efda40b4420 msgr2=0x7efda414cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:06.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efda3400640 1 --2- 172.21.15.161:0/1141016713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efda40b4420 0x7efda414cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:06.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efda3400640 1 -- 172.21.15.161:0/1141016713 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd900094d0 con 0x7efda406c8f0 2024-05-22T21:14:06.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efda3400640 1 --2- 172.21.15.161:0/1141016713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda406c8f0 0x7efda40ae490 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efd9001a110 tx=0x7efd9001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:06.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efd9be00640 1 -- 172.21.15.161:0/1141016713 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd900b4800 con 0x7efda406c8f0 2024-05-22T21:14:06.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efd9be00640 1 -- 172.21.15.161:0/1141016713 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd900b4960 con 0x7efda406c8f0 2024-05-22T21:14:06.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efd9be00640 1 -- 172.21.15.161:0/1141016713 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd900b4d10 con 0x7efda406c8f0 2024-05-22T21:14:06.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efdab000640 1 -- 172.21.15.161:0/1141016713 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda40cca00 con 0x7efda406c8f0 2024-05-22T21:14:06.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.937+0000 7efdab000640 1 -- 172.21.15.161:0/1141016713 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda414c560 con 0x7efda406c8f0 2024-05-22T21:14:06.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.941+0000 7efdab000640 1 -- 172.21.15.161:0/1141016713 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda40cd1c0 con 0x7efda406c8f0 2024-05-22T21:14:06.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.941+0000 7efd9be00640 1 -- 172.21.15.161:0/1141016713 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efd900b3380 con 0x7efda406c8f0 2024-05-22T21:14:06.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.941+0000 7efd9be00640 1 --2- 172.21.15.161:0/1141016713 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd840666c0 0x7efd84068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:06.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.941+0000 7efd9be00640 1 -- 172.21.15.161:0/1141016713 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd9011e7c0 con 0x7efda406c8f0 2024-05-22T21:14:06.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.941+0000 7efda3e00640 1 --2- 172.21.15.161:0/1141016713 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd840666c0 0x7efd84068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:06.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.941+0000 7efda3e00640 1 --2- 172.21.15.161:0/1141016713 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd840666c0 0x7efd84068b80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efd94003fd0 tx=0x7efd940039a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:06.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:06.945+0000 7efd9be00640 1 -- 172.21.15.161:0/1141016713 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efd900ebcf0 con 0x7efda406c8f0 2024-05-22T21:14:07.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.101+0000 7efdab000640 1 -- 172.21.15.161:0/1141016713 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efda4067220 con 0x7efda406c8f0 2024-05-22T21:14:07.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.101+0000 7efd9be00640 1 -- 172.21.15.161:0/1141016713 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7efd900ef310 con 0x7efda406c8f0 2024-05-22T21:14:07.106 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:07.106 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:07.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.105+0000 7efdab000640 1 -- 172.21.15.161:0/1141016713 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd840666c0 msgr2=0x7efd84068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.105+0000 7efdab000640 1 --2- 172.21.15.161:0/1141016713 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd840666c0 0x7efd84068b80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efd94003fd0 tx=0x7efd940039a0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.105+0000 7efdab000640 1 -- 172.21.15.161:0/1141016713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda406c8f0 msgr2=0x7efda40ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.105+0000 7efdab000640 1 --2- 172.21.15.161:0/1141016713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda406c8f0 0x7efda40ae490 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efd9001a110 tx=0x7efd9001ad20 comp rx=0 tx=0).stop 2024-05-22T21:14:07.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.105+0000 7efdab000640 1 -- 172.21.15.161:0/1141016713 shutdown_connections 2024-05-22T21:14:07.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.105+0000 7efdab000640 1 --2- 172.21.15.161:0/1141016713 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd840666c0 0x7efd84068b80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.105+0000 7efdab000640 1 --2- 172.21.15.161:0/1141016713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efda40b4420 0x7efda414cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.105+0000 7efdab000640 1 --2- 172.21.15.161:0/1141016713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efda406c8f0 0x7efda40ae490 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.105+0000 7efdab000640 1 --2- 172.21.15.161:0/1141016713 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efda406b420 0x7efda40adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.105+0000 7efdab000640 1 -- 172.21.15.161:0/1141016713 >> 172.21.15.161:0/1141016713 conn(0x7efda4070630 msgr2=0x7efda40afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:07.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.105+0000 7efdab000640 1 -- 172.21.15.161:0/1141016713 shutdown_connections 2024-05-22T21:14:07.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.105+0000 7efdab000640 1 -- 172.21.15.161:0/1141016713 wait complete. 2024-05-22T21:14:07.128 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-22T21:14:07.128 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:14:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.257+0000 7f3a10800640 1 Processor -- start 2024-05-22T21:14:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.257+0000 7f3a10800640 1 -- start start 2024-05-22T21:14:07.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.257+0000 7f3a10800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a0815dd50 0x7f3a0806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.257+0000 7f3a10800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3a0815e130 0x7f3a0815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.257+0000 7f3a10800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a08071670 con 0x7f3a0815d410 2024-05-22T21:14:07.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.257+0000 7f3a10800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0815d8a0 con 0x7f3a0815e130 2024-05-22T21:14:07.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.257+0000 7f3a10800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0806de40 con 0x7f3a0815dd50 2024-05-22T21:14:07.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.257+0000 7f3a0e800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3a0815d410 0x7f3a0815a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59172/0 (socket says 172.21.15.161:59172) 2024-05-22T21:14:07.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.257+0000 7f3a0e800640 1 -- 172.21.15.161:0/1162863659 learned_addr learned my addr 172.21.15.161:0/1162863659 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:07.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.257+0000 7f3a0d400640 1 --2- 172.21.15.161:0/1162863659 >> v2:172.21.15.179:3300/0 conn(0x7f3a0815e130 0x7f3a0815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.257+0000 7f3a0de00640 1 --2- 172.21.15.161:0/1162863659 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a0815dd50 0x7f3a0806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.261+0000 7f3a0ca00640 1 -- 172.21.15.161:0/1162863659 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388773398 0 0) 0x7f3a08071670 con 0x7f3a0815d410 2024-05-22T21:14:07.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.261+0000 7f3a0ca00640 1 -- 172.21.15.161:0/1162863659 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39f0003530 con 0x7f3a0815d410 2024-05-22T21:14:07.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.261+0000 7f3a0d400640 1 -- 172.21.15.161:0/1162863659 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a0815dd50 msgr2=0x7f3a0806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:07.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.261+0000 7f3a0d400640 1 --2- 172.21.15.161:0/1162863659 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a0815dd50 0x7f3a0806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.261+0000 7f3a0d400640 1 -- 172.21.15.161:0/1162863659 >> v1:172.21.15.161:6789/0 conn(0x7f3a0815d410 legacy=0x7f3a0815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.261+0000 7f3a0d400640 1 -- 172.21.15.161:0/1162863659 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a08070a60 con 0x7f3a0815e130 2024-05-22T21:14:07.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.261+0000 7f3a0d400640 1 --2- 172.21.15.161:0/1162863659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a0815e130 0x7f3a0815ced0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f39f80097c0 tx=0x7f39f801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cbeb2398454a8e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:07.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a0ca00640 1 -- 172.21.15.161:0/1162863659 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80ac640 con 0x7f3a0815e130 2024-05-22T21:14:07.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a0ca00640 1 -- 172.21.15.161:0/1162863659 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f80ac7a0 con 0x7f3a0815e130 2024-05-22T21:14:07.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a0ca00640 1 -- 172.21.15.161:0/1162863659 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80acb30 con 0x7f3a0815e130 2024-05-22T21:14:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a10800640 1 -- 172.21.15.161:0/1162863659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a0815e130 msgr2=0x7f3a0815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a10800640 1 --2- 172.21.15.161:0/1162863659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a0815e130 0x7f3a0815ced0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f39f80097c0 tx=0x7f39f801c3e0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a10800640 1 -- 172.21.15.161:0/1162863659 shutdown_connections 2024-05-22T21:14:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a10800640 1 --2- 172.21.15.161:0/1162863659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a0815e130 0x7f3a0815ced0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a10800640 1 --2- 172.21.15.161:0/1162863659 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a0815dd50 0x7f3a0806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a10800640 1 -- 172.21.15.161:0/1162863659 >> 172.21.15.161:0/1162863659 conn(0x7f3a080b0bc0 msgr2=0x7f3a080b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a10800640 1 -- 172.21.15.161:0/1162863659 shutdown_connections 2024-05-22T21:14:07.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a10800640 1 -- 172.21.15.161:0/1162863659 wait complete. 2024-05-22T21:14:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a10800640 1 Processor -- start 2024-05-22T21:14:07.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a10800640 1 -- start start 2024-05-22T21:14:07.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a10800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0815d410 0x7f3a08157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a10800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a0815dd50 0x7f3a08152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a10800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a0815e130 0x7f3a081567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a10800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a08149220 con 0x7f3a0815d410 2024-05-22T21:14:07.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a10800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a08154c70 con 0x7f3a0815e130 2024-05-22T21:14:07.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a10800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a08154f50 con 0x7f3a0815dd50 2024-05-22T21:14:07.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a0de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0815d410 0x7f3a08157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a0de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0815d410 0x7f3a08157140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39676/0 (socket says 172.21.15.161:39676) 2024-05-22T21:14:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.265+0000 7f3a0de00640 1 -- 172.21.15.161:0/1991023051 learned_addr learned my addr 172.21.15.161:0/1991023051 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a0d400640 1 --2- 172.21.15.161:0/1991023051 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a0815dd50 0x7f3a08152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a0e800640 1 --2- 172.21.15.161:0/1991023051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a0815e130 0x7f3a081567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a0de00640 1 -- 172.21.15.161:0/1991023051 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a0815dd50 msgr2=0x7f3a08152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a0de00640 1 --2- 172.21.15.161:0/1991023051 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a0815dd50 0x7f3a08152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a0de00640 1 -- 172.21.15.161:0/1991023051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a0815e130 msgr2=0x7f3a081567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a0de00640 1 --2- 172.21.15.161:0/1991023051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a0815e130 0x7f3a081567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a0de00640 1 -- 172.21.15.161:0/1991023051 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f8009420 con 0x7f3a0815d410 2024-05-22T21:14:07.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a0de00640 1 --2- 172.21.15.161:0/1991023051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0815d410 0x7f3a08157140 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00e870 tx=0x7f39fc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:07.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a02000640 1 -- 172.21.15.161:0/1991023051 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc009510 con 0x7f3a0815d410 2024-05-22T21:14:07.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a02000640 1 -- 172.21.15.161:0/1991023051 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc00eec0 con 0x7f3a0815d410 2024-05-22T21:14:07.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a10800640 1 -- 172.21.15.161:0/1991023051 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a081494c0 con 0x7f3a0815d410 2024-05-22T21:14:07.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a02000640 1 -- 172.21.15.161:0/1991023051 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc00cb40 con 0x7f3a0815d410 2024-05-22T21:14:07.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a10800640 1 -- 172.21.15.161:0/1991023051 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a08155230 con 0x7f3a0815d410 2024-05-22T21:14:07.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a10800640 1 -- 172.21.15.161:0/1991023051 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a08067220 con 0x7f3a0815d410 2024-05-22T21:14:07.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a02000640 1 -- 172.21.15.161:0/1991023051 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f39fc00cca0 con 0x7f3a0815d410 2024-05-22T21:14:07.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a02000640 1 --2- 172.21.15.161:0/1991023051 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f39ec066690 0x7f39ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.269+0000 7f3a02000640 1 -- 172.21.15.161:0/1991023051 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39fc084310 con 0x7f3a0815d410 2024-05-22T21:14:07.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.273+0000 7f3a0d400640 1 --2- 172.21.15.161:0/1991023051 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f39ec066690 0x7f39ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.273+0000 7f3a0d400640 1 --2- 172.21.15.161:0/1991023051 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f39ec066690 0x7f39ec068b50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f39f80ae000 tx=0x7f39f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:07.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.273+0000 7f3a02000640 1 -- 172.21.15.161:0/1991023051 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f39fc051600 con 0x7f3a0815d410 2024-05-22T21:14:07.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.457+0000 7f3a10800640 1 -- 172.21.15.161:0/1991023051 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a0806c310 con 0x7f3a0815d410 2024-05-22T21:14:07.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.461+0000 7f3a02000640 1 -- 172.21.15.161:0/1991023051 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4740 (secure 0 0 0) 0x7f39fc054c20 con 0x7f3a0815d410 2024-05-22T21:14:07.464 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:07.465 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3179612470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3179612470},{"type":"v1","addr":"172.21.15.179:6834","nonce":3179612470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2196055309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2196055309},{"type":"v1","addr":"172.21.15.161:6835","nonce":2196055309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1793681759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1793681759},{"type":"v1","addr":"172.21.15.161:6837","nonce":1793681759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:13:23.418412+0000","modified":"2024-05-22T21:13:26.442354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/3095599688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3095599688},{"type":"v1","addr":"172.21.15.179:6835","nonce":3095599688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-22T21:14:07.465 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 8 2024-05-22T21:14:07.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.461+0000 7f3a10800640 1 -- 172.21.15.161:0/1991023051 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f39ec066690 msgr2=0x7f39ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.461+0000 7f3a10800640 1 --2- 172.21.15.161:0/1991023051 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f39ec066690 0x7f39ec068b50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f39f80ae000 tx=0x7f39f8008040 comp rx=0 tx=0).stop 2024-05-22T21:14:07.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.461+0000 7f3a10800640 1 -- 172.21.15.161:0/1991023051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0815d410 msgr2=0x7f3a08157140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.461+0000 7f3a10800640 1 --2- 172.21.15.161:0/1991023051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0815d410 0x7f3a08157140 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00e870 tx=0x7f39fc00ec70 comp rx=0 tx=0).stop 2024-05-22T21:14:07.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.461+0000 7f3a10800640 1 -- 172.21.15.161:0/1991023051 shutdown_connections 2024-05-22T21:14:07.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.461+0000 7f3a10800640 1 --2- 172.21.15.161:0/1991023051 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f39ec066690 0x7f39ec068b50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.461+0000 7f3a10800640 1 --2- 172.21.15.161:0/1991023051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a0815e130 0x7f3a081567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.461+0000 7f3a10800640 1 --2- 172.21.15.161:0/1991023051 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a0815dd50 0x7f3a08152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.461+0000 7f3a10800640 1 --2- 172.21.15.161:0/1991023051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0815d410 0x7f3a08157140 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.461+0000 7f3a10800640 1 -- 172.21.15.161:0/1991023051 >> 172.21.15.161:0/1991023051 conn(0x7f3a080b0bc0 msgr2=0x7f3a080adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:07.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.461+0000 7f3a10800640 1 -- 172.21.15.161:0/1991023051 shutdown_connections 2024-05-22T21:14:07.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.461+0000 7f3a10800640 1 -- 172.21.15.161:0/1991023051 wait complete. 2024-05-22T21:14:07.486 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:14:07.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.617+0000 7f412f200640 1 Processor -- start 2024-05-22T21:14:07.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.617+0000 7f412f200640 1 -- start start 2024-05-22T21:14:07.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.617+0000 7f412f200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f41281608b0 0x7f41280d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.617+0000 7f412f200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41280cec30 con 0x7f41280d1df0 2024-05-22T21:14:07.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.617+0000 7f412f200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41280cf0c0 con 0x7f41280afcf0 2024-05-22T21:14:07.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.617+0000 7f412f200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41280d0a30 con 0x7f41281608b0 2024-05-22T21:14:07.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.617+0000 7f411f400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f41280d1df0 0x7f41280d11d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59180/0 (socket says 172.21.15.161:59180) 2024-05-22T21:14:07.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.617+0000 7f411f400640 1 -- 172.21.15.161:0/2456223602 learned_addr learned my addr 172.21.15.161:0/2456223602 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:07.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.617+0000 7f4127e00640 1 --2- 172.21.15.161:0/2456223602 >> v2:172.21.15.179:3300/0 conn(0x7f41281608b0 0x7f41280d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.617+0000 7f4127400640 1 -- 172.21.15.161:0/2456223602 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594473728 0 0) 0x7f41280cec30 con 0x7f41280d1df0 2024-05-22T21:14:07.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.617+0000 7f4127400640 1 -- 172.21.15.161:0/2456223602 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f410c003530 con 0x7f41280d1df0 2024-05-22T21:14:07.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.617+0000 7f4127e00640 1 -- 172.21.15.161:0/2456223602 >> v1:172.21.15.179:6789/0 conn(0x7f41280afcf0 legacy=0x7f41280b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.617+0000 7f4127e00640 1 -- 172.21.15.161:0/2456223602 >> v1:172.21.15.161:6789/0 conn(0x7f41280d1df0 legacy=0x7f41280d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.617+0000 7f4127400640 1 -- 172.21.15.161:0/2456223602 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1080414935 0 0) 0x7f410c003530 con 0x7f41280d1df0 2024-05-22T21:14:07.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.617+0000 7f4127e00640 1 -- 172.21.15.161:0/2456223602 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4128160c90 con 0x7f41281608b0 2024-05-22T21:14:07.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.621+0000 7f4127e00640 1 --2- 172.21.15.161:0/2456223602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41281608b0 0x7f41280d0c90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f41180097c0 tx=0x7f411801c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=500b5dd2744d2f0d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:07.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.621+0000 7f4127400640 1 -- 172.21.15.161:0/2456223602 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110001680 con 0x7f41281608b0 2024-05-22T21:14:07.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.621+0000 7f4127400640 1 -- 172.21.15.161:0/2456223602 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41180ac440 con 0x7f41281608b0 2024-05-22T21:14:07.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.621+0000 7f4127400640 1 -- 172.21.15.161:0/2456223602 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41180ac7d0 con 0x7f41281608b0 2024-05-22T21:14:07.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.621+0000 7f412f200640 1 -- 172.21.15.161:0/2456223602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41281608b0 msgr2=0x7f41280d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.621+0000 7f412f200640 1 --2- 172.21.15.161:0/2456223602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41281608b0 0x7f41280d0c90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f41180097c0 tx=0x7f411801c390 comp rx=0 tx=0).stop 2024-05-22T21:14:07.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.621+0000 7f412f200640 1 -- 172.21.15.161:0/2456223602 shutdown_connections 2024-05-22T21:14:07.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.621+0000 7f412f200640 1 --2- 172.21.15.161:0/2456223602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41281608b0 0x7f41280d0c90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.621+0000 7f412f200640 1 -- 172.21.15.161:0/2456223602 >> 172.21.15.161:0/2456223602 conn(0x7f412809c400 msgr2=0x7f41280d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:07.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.621+0000 7f412f200640 1 -- 172.21.15.161:0/2456223602 shutdown_connections 2024-05-22T21:14:07.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.621+0000 7f412f200640 1 -- 172.21.15.161:0/2456223602 wait complete. 2024-05-22T21:14:07.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f412f200640 1 Processor -- start 2024-05-22T21:14:07.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f412f200640 1 -- start start 2024-05-22T21:14:07.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f412f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41280afcf0 0x7f41280c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f412f200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41280d1df0 0x7f41280b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f412f200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41281608b0 0x7f41280b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f412f200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41280cb5b0 con 0x7f41280afcf0 2024-05-22T21:14:07.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f412f200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41280c62d0 con 0x7f41280d1df0 2024-05-22T21:14:07.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f412f200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41280cb7e0 con 0x7f41281608b0 2024-05-22T21:14:07.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f4127e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41280afcf0 0x7f41280c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f4127e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41280afcf0 0x7f41280c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39678/0 (socket says 172.21.15.161:39678) 2024-05-22T21:14:07.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f4127e00640 1 -- 172.21.15.161:0/2522728026 learned_addr learned my addr 172.21.15.161:0/2522728026 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:07.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f411f400640 1 --2- 172.21.15.161:0/2522728026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41280d1df0 0x7f41280b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f412d200640 1 --2- 172.21.15.161:0/2522728026 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41281608b0 0x7f41280b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f4127e00640 1 -- 172.21.15.161:0/2522728026 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41281608b0 msgr2=0x7f41280b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f4127e00640 1 --2- 172.21.15.161:0/2522728026 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41281608b0 0x7f41280b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f4127e00640 1 -- 172.21.15.161:0/2522728026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41280d1df0 msgr2=0x7f41280b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f4127e00640 1 --2- 172.21.15.161:0/2522728026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41280d1df0 0x7f41280b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f4127e00640 1 -- 172.21.15.161:0/2522728026 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4118009420 con 0x7f41280afcf0 2024-05-22T21:14:07.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f4127e00640 1 --2- 172.21.15.161:0/2522728026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41280afcf0 0x7f41280c6770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f41180097c0 tx=0x7f411801abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:07.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f4124c00640 1 -- 172.21.15.161:0/2522728026 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118004220 con 0x7f41280afcf0 2024-05-22T21:14:07.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f4124c00640 1 -- 172.21.15.161:0/2522728026 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4118004380 con 0x7f41280afcf0 2024-05-22T21:14:07.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f4124c00640 1 -- 172.21.15.161:0/2522728026 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118003860 con 0x7f41280afcf0 2024-05-22T21:14:07.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f412f200640 1 -- 172.21.15.161:0/2522728026 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41280cba80 con 0x7f41280afcf0 2024-05-22T21:14:07.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.625+0000 7f412f200640 1 -- 172.21.15.161:0/2522728026 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41280cc040 con 0x7f41280afcf0 2024-05-22T21:14:07.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.629+0000 7f412f200640 1 -- 172.21.15.161:0/2522728026 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4128067220 con 0x7f41280afcf0 2024-05-22T21:14:07.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.629+0000 7f4124c00640 1 -- 172.21.15.161:0/2522728026 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f41180039c0 con 0x7f41280afcf0 2024-05-22T21:14:07.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.629+0000 7f4124c00640 1 --2- 172.21.15.161:0/2522728026 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4104066690 0x7f4104068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.629+0000 7f4124c00640 1 -- 172.21.15.161:0/2522728026 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f411811dd30 con 0x7f41280afcf0 2024-05-22T21:14:07.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.629+0000 7f411f400640 1 --2- 172.21.15.161:0/2522728026 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4104066690 0x7f4104068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.629+0000 7f411f400640 1 --2- 172.21.15.161:0/2522728026 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4104066690 0x7f4104068b50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f41280c97e0 tx=0x7f4110001e10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:07.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.633+0000 7f4124c00640 1 -- 172.21.15.161:0/2522728026 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f41180eb030 con 0x7f41280afcf0 2024-05-22T21:14:07.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.817+0000 7f412f200640 1 -- 172.21.15.161:0/2522728026 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41280cf5f0 con 0x7f41280afcf0 2024-05-22T21:14:07.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.817+0000 7f4124c00640 1 -- 172.21.15.161:0/2522728026 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4740 (secure 0 0 0) 0x7f41180ee650 con 0x7f41280afcf0 2024-05-22T21:14:07.823 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:07.823 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3179612470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3179612470},{"type":"v1","addr":"172.21.15.179:6834","nonce":3179612470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2196055309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2196055309},{"type":"v1","addr":"172.21.15.161:6835","nonce":2196055309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1793681759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1793681759},{"type":"v1","addr":"172.21.15.161:6837","nonce":1793681759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:13:23.418412+0000","modified":"2024-05-22T21:13:26.442354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/3095599688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3095599688},{"type":"v1","addr":"172.21.15.179:6835","nonce":3095599688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-22T21:14:07.823 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 8 2024-05-22T21:14:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.821+0000 7f412f200640 1 -- 172.21.15.161:0/2522728026 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4104066690 msgr2=0x7f4104068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.821+0000 7f412f200640 1 --2- 172.21.15.161:0/2522728026 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4104066690 0x7f4104068b50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f41280c97e0 tx=0x7f4110001e10 comp rx=0 tx=0).stop 2024-05-22T21:14:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.821+0000 7f412f200640 1 -- 172.21.15.161:0/2522728026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41280afcf0 msgr2=0x7f41280c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.821+0000 7f412f200640 1 --2- 172.21.15.161:0/2522728026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41280afcf0 0x7f41280c6770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f41180097c0 tx=0x7f411801abd0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.821+0000 7f412f200640 1 -- 172.21.15.161:0/2522728026 shutdown_connections 2024-05-22T21:14:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.821+0000 7f412f200640 1 --2- 172.21.15.161:0/2522728026 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4104066690 0x7f4104068b50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.821+0000 7f412f200640 1 --2- 172.21.15.161:0/2522728026 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41281608b0 0x7f41280b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.821+0000 7f412f200640 1 --2- 172.21.15.161:0/2522728026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41280d1df0 0x7f41280b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.821+0000 7f412f200640 1 --2- 172.21.15.161:0/2522728026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41280afcf0 0x7f41280c6770 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.821+0000 7f412f200640 1 -- 172.21.15.161:0/2522728026 >> 172.21.15.161:0/2522728026 conn(0x7f412809c400 msgr2=0x7f41280b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.821+0000 7f412f200640 1 -- 172.21.15.161:0/2522728026 shutdown_connections 2024-05-22T21:14:07.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.821+0000 7f412f200640 1 -- 172.21.15.161:0/2522728026 wait complete. 2024-05-22T21:14:07.844 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:07.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.973+0000 7f2880200640 1 Processor -- start 2024-05-22T21:14:07.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.973+0000 7f2880200640 1 -- start start 2024-05-22T21:14:07.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.973+0000 7f2880200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2878143ec0 0x7f287815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.973+0000 7f2880200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f287815ec90 0x7f287815f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.973+0000 7f2880200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28781442a0 con 0x7f28780715a0 2024-05-22T21:14:07.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.973+0000 7f2880200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287815f680 con 0x7f287815ec90 2024-05-22T21:14:07.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.973+0000 7f2880200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2878070500 con 0x7f2878143ec0 2024-05-22T21:14:07.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.973+0000 7f287e200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f287815ec90 0x7f287815f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.973+0000 7f287ce00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f28780715a0 0x7f287806fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59196/0 (socket says 172.21.15.161:59196) 2024-05-22T21:14:07.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.973+0000 7f287ce00640 1 -- 172.21.15.161:0/2779447246 learned_addr learned my addr 172.21.15.161:0/2779447246 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:07.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.977+0000 7f287d800640 1 --2- 172.21.15.161:0/2779447246 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2878143ec0 0x7f287815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.977+0000 7f2877e00640 1 -- 172.21.15.161:0/2779447246 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631341585 0 0) 0x7f28781442a0 con 0x7f28780715a0 2024-05-22T21:14:07.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.977+0000 7f287e200640 1 -- 172.21.15.161:0/2779447246 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2878143ec0 msgr2=0x7f287815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:07.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.977+0000 7f287e200640 1 --2- 172.21.15.161:0/2779447246 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2878143ec0 0x7f287815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.977+0000 7f287e200640 1 -- 172.21.15.161:0/2779447246 >> v1:172.21.15.161:6789/0 conn(0x7f28780715a0 legacy=0x7f287806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.977+0000 7f287e200640 1 -- 172.21.15.161:0/2779447246 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28780707e0 con 0x7f287815ec90 2024-05-22T21:14:07.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.977+0000 7f287e200640 1 --2- 172.21.15.161:0/2779447246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f287815ec90 0x7f287815f140 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2870009d90 tx=0x7f287001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4cc781964afc67 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:07.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.977+0000 7f2877e00640 1 -- 172.21.15.161:0/2779447246 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28700a5900 con 0x7f287815ec90 2024-05-22T21:14:07.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.977+0000 7f2877e00640 1 -- 172.21.15.161:0/2779447246 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28700a5a60 con 0x7f287815ec90 2024-05-22T21:14:07.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.977+0000 7f2877e00640 1 -- 172.21.15.161:0/2779447246 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28700a5df0 con 0x7f287815ec90 2024-05-22T21:14:07.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.977+0000 7f2880200640 1 -- 172.21.15.161:0/2779447246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f287815ec90 msgr2=0x7f287815f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.977+0000 7f2880200640 1 --2- 172.21.15.161:0/2779447246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f287815ec90 0x7f287815f140 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2870009d90 tx=0x7f287001c940 comp rx=0 tx=0).stop 2024-05-22T21:14:07.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f2880200640 1 -- 172.21.15.161:0/2779447246 shutdown_connections 2024-05-22T21:14:07.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f2880200640 1 --2- 172.21.15.161:0/2779447246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f287815ec90 0x7f287815f140 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f2880200640 1 --2- 172.21.15.161:0/2779447246 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2878143ec0 0x7f287815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f2880200640 1 -- 172.21.15.161:0/2779447246 >> 172.21.15.161:0/2779447246 conn(0x7f28780b0bc0 msgr2=0x7f28780b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:07.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f2880200640 1 -- 172.21.15.161:0/2779447246 shutdown_connections 2024-05-22T21:14:07.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f2880200640 1 -- 172.21.15.161:0/2779447246 wait complete. 2024-05-22T21:14:07.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f2880200640 1 Processor -- start 2024-05-22T21:14:07.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f2880200640 1 -- start start 2024-05-22T21:14:07.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f2880200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2878143ec0 0x7f287813f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f2880200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f287815ec90 0x7f28780ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f2880200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f28780b3950 0x7f28780af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f2880200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2878140150 con 0x7f287815ec90 2024-05-22T21:14:07.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f2880200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287813ff20 con 0x7f2878143ec0 2024-05-22T21:14:07.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f2880200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28780ae360 con 0x7f28780b3950 2024-05-22T21:14:07.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f287ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f287815ec90 0x7f28780ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f287ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f287815ec90 0x7f28780ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39698/0 (socket says 172.21.15.161:39698) 2024-05-22T21:14:07.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f287ce00640 1 -- 172.21.15.161:0/653470360 learned_addr learned my addr 172.21.15.161:0/653470360 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:07.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f287d800640 1 --2- 172.21.15.161:0/653470360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2878143ec0 0x7f287813f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f287e200640 1 --2- 172.21.15.161:0/653470360 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f28780b3950 0x7f28780af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f287ce00640 1 -- 172.21.15.161:0/653470360 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f28780b3950 msgr2=0x7f28780af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f287ce00640 1 --2- 172.21.15.161:0/653470360 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f28780b3950 0x7f28780af0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f287ce00640 1 -- 172.21.15.161:0/653470360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2878143ec0 msgr2=0x7f287813f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:07.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f287ce00640 1 --2- 172.21.15.161:0/653470360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2878143ec0 0x7f287813f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:07.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f287ce00640 1 -- 172.21.15.161:0/653470360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2870009a70 con 0x7f287815ec90 2024-05-22T21:14:07.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f287d800640 1 --2- 172.21.15.161:0/653470360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2878143ec0 0x7f287813f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:07.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f287e200640 1 --2- 172.21.15.161:0/653470360 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f28780b3950 0x7f28780af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:07.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.981+0000 7f287ce00640 1 --2- 172.21.15.161:0/653470360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f287815ec90 0x7f28780ade20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f286400c4b0 tx=0x7f286400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:07.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.985+0000 7f2875600640 1 -- 172.21.15.161:0/653470360 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28640074e0 con 0x7f287815ec90 2024-05-22T21:14:07.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.985+0000 7f2875600640 1 -- 172.21.15.161:0/653470360 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2864007660 con 0x7f287815ec90 2024-05-22T21:14:07.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.985+0000 7f2875600640 1 -- 172.21.15.161:0/653470360 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864004a30 con 0x7f287815ec90 2024-05-22T21:14:07.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.985+0000 7f2880200640 1 -- 172.21.15.161:0/653470360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2878149200 con 0x7f287815ec90 2024-05-22T21:14:07.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.985+0000 7f2880200640 1 -- 172.21.15.161:0/653470360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28781488d0 con 0x7f287815ec90 2024-05-22T21:14:07.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.985+0000 7f2880200640 1 -- 172.21.15.161:0/653470360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2878149390 con 0x7f287815ec90 2024-05-22T21:14:07.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.985+0000 7f2875600640 1 -- 172.21.15.161:0/653470360 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2864004b90 con 0x7f287815ec90 2024-05-22T21:14:07.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.985+0000 7f2875600640 1 --2- 172.21.15.161:0/653470360 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2858066690 0x7f2858068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:07.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.985+0000 7f2875600640 1 -- 172.21.15.161:0/653470360 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2864083430 con 0x7f287815ec90 2024-05-22T21:14:07.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.985+0000 7f287d800640 1 --2- 172.21.15.161:0/653470360 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2858066690 0x7f2858068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:07.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.985+0000 7f287d800640 1 --2- 172.21.15.161:0/653470360 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2858066690 0x7f2858068b50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f286c009ae0 tx=0x7f286c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:07.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:07.989+0000 7f2875600640 1 -- 172.21.15.161:0/653470360 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f28640507c0 con 0x7f287815ec90 2024-05-22T21:14:08.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.145+0000 7f2880200640 1 -- 172.21.15.161:0/653470360 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2878067220 con 0x7f287815ec90 2024-05-22T21:14:08.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.145+0000 7f2875600640 1 -- 172.21.15.161:0/653470360 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f2864018090 con 0x7f287815ec90 2024-05-22T21:14:08.150 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:08.150 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":21,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:13:24.420638+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:13:21.237503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-22T21:13:22.425891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:08.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.149+0000 7f2880200640 1 -- 172.21.15.161:0/653470360 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2858066690 msgr2=0x7f2858068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:08.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.149+0000 7f2880200640 1 --2- 172.21.15.161:0/653470360 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2858066690 0x7f2858068b50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f286c009ae0 tx=0x7f286c008040 comp rx=0 tx=0).stop 2024-05-22T21:14:08.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.149+0000 7f2880200640 1 -- 172.21.15.161:0/653470360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f287815ec90 msgr2=0x7f28780ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:08.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.149+0000 7f2880200640 1 --2- 172.21.15.161:0/653470360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f287815ec90 0x7f28780ade20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f286400c4b0 tx=0x7f286400c970 comp rx=0 tx=0).stop 2024-05-22T21:14:08.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.149+0000 7f2880200640 1 -- 172.21.15.161:0/653470360 shutdown_connections 2024-05-22T21:14:08.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.149+0000 7f2880200640 1 --2- 172.21.15.161:0/653470360 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2858066690 0x7f2858068b50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.149+0000 7f2880200640 1 --2- 172.21.15.161:0/653470360 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f28780b3950 0x7f28780af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.149+0000 7f2880200640 1 --2- 172.21.15.161:0/653470360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f287815ec90 0x7f28780ade20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.149+0000 7f2880200640 1 --2- 172.21.15.161:0/653470360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2878143ec0 0x7f287813f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.149+0000 7f2880200640 1 -- 172.21.15.161:0/653470360 >> 172.21.15.161:0/653470360 conn(0x7f28780b0bc0 msgr2=0x7f2878139060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:08.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.149+0000 7f2880200640 1 -- 172.21.15.161:0/653470360 shutdown_connections 2024-05-22T21:14:08.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.149+0000 7f2880200640 1 -- 172.21.15.161:0/653470360 wait complete. 2024-05-22T21:14:08.172 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-22T21:14:08.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.301+0000 7f8cb1000640 1 Processor -- start 2024-05-22T21:14:08.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.301+0000 7f8cb1000640 1 -- start start 2024-05-22T21:14:08.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.305+0000 7f8cb1000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8cac143c40 0x7f8cac15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:08.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.305+0000 7f8cb1000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cac1444a0 0x7f8cac06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:08.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.305+0000 7f8cb1000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cac15efb0 con 0x7f8cac144900 2024-05-22T21:14:08.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.305+0000 7f8cb1000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac15f290 con 0x7f8cac143c40 2024-05-22T21:14:08.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.305+0000 7f8cb1000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac15f570 con 0x7f8cac1444a0 2024-05-22T21:14:08.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.305+0000 7f8caa800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8cac144900 0x7f8cac0b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59198/0 (socket says 172.21.15.161:59198) 2024-05-22T21:14:08.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.305+0000 7f8caa800640 1 -- 172.21.15.161:0/2408387865 learned_addr learned my addr 172.21.15.161:0/2408387865 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:08.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.305+0000 7f8ca9400640 1 --2- 172.21.15.161:0/2408387865 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cac1444a0 0x7f8cac06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:08.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.305+0000 7f8ca9e00640 1 --2- 172.21.15.161:0/2408387865 >> v2:172.21.15.179:3300/0 conn(0x7f8cac143c40 0x7f8cac15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:08.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.305+0000 7f8ca8a00640 1 -- 172.21.15.161:0/2408387865 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945327967 0 0) 0x7f8cac15efb0 con 0x7f8cac144900 2024-05-22T21:14:08.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.305+0000 7f8ca8a00640 1 -- 172.21.15.161:0/2408387865 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c90003530 con 0x7f8cac144900 2024-05-22T21:14:08.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.305+0000 7f8ca9400640 1 -- 172.21.15.161:0/2408387865 >> v2:172.21.15.179:3300/0 conn(0x7f8cac143c40 msgr2=0x7f8cac15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:08.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.305+0000 7f8ca9400640 1 --2- 172.21.15.161:0/2408387865 >> v2:172.21.15.179:3300/0 conn(0x7f8cac143c40 0x7f8cac15b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.305+0000 7f8ca9400640 1 -- 172.21.15.161:0/2408387865 >> v1:172.21.15.161:6789/0 conn(0x7f8cac144900 legacy=0x7f8cac0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:08.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.305+0000 7f8ca9400640 1 -- 172.21.15.161:0/2408387865 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac15f850 con 0x7f8cac1444a0 2024-05-22T21:14:08.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.305+0000 7f8ca9e00640 1 --2- 172.21.15.161:0/2408387865 >> v2:172.21.15.179:3300/0 conn(0x7f8cac143c40 0x7f8cac15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:08.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.309+0000 7f8ca9400640 1 --2- 172.21.15.161:0/2408387865 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cac1444a0 0x7f8cac06c2a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8c980097c0 tx=0x7f8c9801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=496b10230235e506 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:08.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.309+0000 7f8ca8a00640 1 -- 172.21.15.161:0/2408387865 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c980ac410 con 0x7f8cac1444a0 2024-05-22T21:14:08.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.309+0000 7f8ca8a00640 1 -- 172.21.15.161:0/2408387865 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c980ac570 con 0x7f8cac1444a0 2024-05-22T21:14:08.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.309+0000 7f8ca8a00640 1 -- 172.21.15.161:0/2408387865 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c980ac900 con 0x7f8cac1444a0 2024-05-22T21:14:08.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.309+0000 7f8cb1000640 1 -- 172.21.15.161:0/2408387865 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cac1444a0 msgr2=0x7f8cac06c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:08.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.309+0000 7f8cb1000640 1 --2- 172.21.15.161:0/2408387865 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cac1444a0 0x7f8cac06c2a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8c980097c0 tx=0x7f8c9801c4a0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.309+0000 7f8cb1000640 1 -- 172.21.15.161:0/2408387865 shutdown_connections 2024-05-22T21:14:08.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.309+0000 7f8cb1000640 1 --2- 172.21.15.161:0/2408387865 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cac1444a0 0x7f8cac06c2a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.309+0000 7f8cb1000640 1 --2- 172.21.15.161:0/2408387865 >> v2:172.21.15.179:3300/0 conn(0x7f8cac143c40 0x7f8cac15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.309+0000 7f8cb1000640 1 -- 172.21.15.161:0/2408387865 >> 172.21.15.161:0/2408387865 conn(0x7f8cac06a2d0 msgr2=0x7f8cac06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:08.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.309+0000 7f8cb1000640 1 -- 172.21.15.161:0/2408387865 shutdown_connections 2024-05-22T21:14:08.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.309+0000 7f8cb1000640 1 -- 172.21.15.161:0/2408387865 wait complete. 2024-05-22T21:14:08.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.309+0000 7f8cb1000640 1 Processor -- start 2024-05-22T21:14:08.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.309+0000 7f8cb1000640 1 -- start start 2024-05-22T21:14:08.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8cb1000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cac143c40 0x7f8cac152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:08.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8cb1000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cac1444a0 0x7f8cac15e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:08.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8cb1000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cac144900 0x7f8cac13dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:08.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8cb1000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac159130 con 0x7f8cac143c40 2024-05-22T21:14:08.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8cb1000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac159410 con 0x7f8cac1444a0 2024-05-22T21:14:08.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8cb1000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac1596f0 con 0x7f8cac144900 2024-05-22T21:14:08.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8ca9e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cac143c40 0x7f8cac152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:08.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8ca9e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cac143c40 0x7f8cac152410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39700/0 (socket says 172.21.15.161:39700) 2024-05-22T21:14:08.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8ca9e00640 1 -- 172.21.15.161:0/938702086 learned_addr learned my addr 172.21.15.161:0/938702086 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:08.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8ca9e00640 1 -- 172.21.15.161:0/938702086 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cac144900 msgr2=0x7f8cac13dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:08.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8ca9e00640 1 --2- 172.21.15.161:0/938702086 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cac144900 0x7f8cac13dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8ca9e00640 1 -- 172.21.15.161:0/938702086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cac1444a0 msgr2=0x7f8cac15e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:08.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8ca9e00640 1 --2- 172.21.15.161:0/938702086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cac1444a0 0x7f8cac15e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8ca9e00640 1 -- 172.21.15.161:0/938702086 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c009a70 con 0x7f8cac143c40 2024-05-22T21:14:08.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8ca9e00640 1 --2- 172.21.15.161:0/938702086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cac143c40 0x7f8cac152410 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00efc0 tx=0x7f8c9c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:08.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8ca2000640 1 -- 172.21.15.161:0/938702086 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c014680 con 0x7f8cac143c40 2024-05-22T21:14:08.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8ca2000640 1 -- 172.21.15.161:0/938702086 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c00f040 con 0x7f8cac143c40 2024-05-22T21:14:08.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8ca2000640 1 -- 172.21.15.161:0/938702086 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0149e0 con 0x7f8cac143c40 2024-05-22T21:14:08.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8cb1000640 1 -- 172.21.15.161:0/938702086 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c98009420 con 0x7f8cac143c40 2024-05-22T21:14:08.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8cb1000640 1 -- 172.21.15.161:0/938702086 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac152950 con 0x7f8cac143c40 2024-05-22T21:14:08.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.313+0000 7f8cb1000640 1 -- 172.21.15.161:0/938702086 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cac152ba0 con 0x7f8cac143c40 2024-05-22T21:14:08.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.317+0000 7f8ca2000640 1 -- 172.21.15.161:0/938702086 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8c9c014b40 con 0x7f8cac143c40 2024-05-22T21:14:08.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.317+0000 7f8ca2000640 1 --2- 172.21.15.161:0/938702086 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c8c066690 0x7f8c8c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:08.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.317+0000 7f8ca2000640 1 -- 172.21.15.161:0/938702086 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c9c085270 con 0x7f8cac143c40 2024-05-22T21:14:08.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.317+0000 7f8ca9400640 1 --2- 172.21.15.161:0/938702086 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c8c066690 0x7f8c8c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:08.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.317+0000 7f8ca9400640 1 --2- 172.21.15.161:0/938702086 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c8c066690 0x7f8c8c068b50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8c980ae000 tx=0x7f8c98004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:08.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.321+0000 7f8ca2000640 1 -- 172.21.15.161:0/938702086 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8c9c0525d0 con 0x7f8cac143c40 2024-05-22T21:14:08.512 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.509+0000 7f8cb1000640 1 -- 172.21.15.161:0/938702086 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8cac067220 con 0x7f8cac143c40 2024-05-22T21:14:08.513 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:14:08.509+0000 7f7d32800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-22T21:14:08.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.517+0000 7f8ca2000640 1 -- 172.21.15.161:0/938702086 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8c9c055bf0 con 0x7f8cac143c40 2024-05-22T21:14:08.520 INFO:teuthology.orchestra.run.smithi161.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-22T21:14:08.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.517+0000 7f8cb1000640 1 -- 172.21.15.161:0/938702086 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c8c066690 msgr2=0x7f8c8c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:08.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.517+0000 7f8cb1000640 1 --2- 172.21.15.161:0/938702086 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c8c066690 0x7f8c8c068b50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8c980ae000 tx=0x7f8c98004740 comp rx=0 tx=0).stop 2024-05-22T21:14:08.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.517+0000 7f8cb1000640 1 -- 172.21.15.161:0/938702086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cac143c40 msgr2=0x7f8cac152410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:08.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.517+0000 7f8cb1000640 1 --2- 172.21.15.161:0/938702086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cac143c40 0x7f8cac152410 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00efc0 tx=0x7f8c9c00c490 comp rx=0 tx=0).stop 2024-05-22T21:14:08.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.517+0000 7f8cb1000640 1 -- 172.21.15.161:0/938702086 shutdown_connections 2024-05-22T21:14:08.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.517+0000 7f8cb1000640 1 --2- 172.21.15.161:0/938702086 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c8c066690 0x7f8c8c068b50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.517+0000 7f8cb1000640 1 --2- 172.21.15.161:0/938702086 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cac144900 0x7f8cac13dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.517+0000 7f8cb1000640 1 --2- 172.21.15.161:0/938702086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cac1444a0 0x7f8cac15e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.517+0000 7f8cb1000640 1 --2- 172.21.15.161:0/938702086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cac143c40 0x7f8cac152410 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.517+0000 7f8cb1000640 1 -- 172.21.15.161:0/938702086 >> 172.21.15.161:0/938702086 conn(0x7f8cac06a2d0 msgr2=0x7f8cac0afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:08.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.517+0000 7f8cb1000640 1 -- 172.21.15.161:0/938702086 shutdown_connections 2024-05-22T21:14:08.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.517+0000 7f8cb1000640 1 -- 172.21.15.161:0/938702086 wait complete. 2024-05-22T21:14:08.534 INFO:tasks.ceph.mds.b.smithi179.stderr: -6735> 2024-05-22T21:13:27.591+0000 7f4c24c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T21:14:08.542 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-22T21:14:08.585 INFO:tasks.ceph.mds.b.smithi179.stdout:starting mds.b at 2024-05-22T21:14:08.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.677+0000 7f70b8000640 1 Processor -- start 2024-05-22T21:14:08.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.677+0000 7f70b8000640 1 -- start start 2024-05-22T21:14:08.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.677+0000 7f70b8000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f70b013cfc0 0x7f70b01402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:08.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.677+0000 7f70b8000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b013d3a0 con 0x7f70b015af60 2024-05-22T21:14:08.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.677+0000 7f70b8000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b00cd920 con 0x7f70b013fe50 2024-05-22T21:14:08.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.677+0000 7f70b8000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b00cdc00 con 0x7f70b013cfc0 2024-05-22T21:14:08.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.677+0000 7f70b4c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f70b015af60 0x7f70b013f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59206/0 (socket says 172.21.15.161:59206) 2024-05-22T21:14:08.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.677+0000 7f70b4c00640 1 -- 172.21.15.161:0/2218319744 learned_addr learned my addr 172.21.15.161:0/2218319744 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:08.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.677+0000 7f70b5600640 1 --2- 172.21.15.161:0/2218319744 >> v2:172.21.15.179:3300/0 conn(0x7f70b013cfc0 0x7f70b01402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:08.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.677+0000 7f70afe00640 1 -- 172.21.15.161:0/2218319744 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836902231 0 0) 0x7f70b013d3a0 con 0x7f70b015af60 2024-05-22T21:14:08.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.677+0000 7f70afe00640 1 -- 172.21.15.161:0/2218319744 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7098003530 con 0x7f70b015af60 2024-05-22T21:14:08.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.677+0000 7f70afe00640 1 -- 172.21.15.161:0/2218319744 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825822550 0 0) 0x7f70b00cd920 con 0x7f70b013fe50 2024-05-22T21:14:08.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.677+0000 7f70afe00640 1 -- 172.21.15.161:0/2218319744 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70b013d3a0 con 0x7f70b013fe50 2024-05-22T21:14:08.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.677+0000 7f70b5600640 1 -- 172.21.15.161:0/2218319744 >> v1:172.21.15.179:6789/0 conn(0x7f70b013fe50 legacy=0x7f70b006fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:08.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.677+0000 7f70b5600640 1 -- 172.21.15.161:0/2218319744 >> v1:172.21.15.161:6789/0 conn(0x7f70b015af60 legacy=0x7f70b013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:08.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.677+0000 7f70b5600640 1 -- 172.21.15.161:0/2218319744 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b00cdee0 con 0x7f70b013cfc0 2024-05-22T21:14:08.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.681+0000 7f70b5600640 1 --2- 172.21.15.161:0/2218319744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70b013cfc0 0x7f70b01402e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f70a40097c0 tx=0x7f70a401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=455cfe53a3078c62 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:08.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.681+0000 7f70afe00640 1 -- 172.21.15.161:0/2218319744 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a40ac440 con 0x7f70b013cfc0 2024-05-22T21:14:08.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.681+0000 7f70afe00640 1 -- 172.21.15.161:0/2218319744 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a40ac5a0 con 0x7f70b013cfc0 2024-05-22T21:14:08.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.681+0000 7f70afe00640 1 -- 172.21.15.161:0/2218319744 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a40ac930 con 0x7f70b013cfc0 2024-05-22T21:14:08.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.681+0000 7f70b8000640 1 -- 172.21.15.161:0/2218319744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70b013cfc0 msgr2=0x7f70b01402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:08.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.681+0000 7f70b8000640 1 --2- 172.21.15.161:0/2218319744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70b013cfc0 0x7f70b01402e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f70a40097c0 tx=0x7f70a401c3e0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.681+0000 7f70b8000640 1 -- 172.21.15.161:0/2218319744 shutdown_connections 2024-05-22T21:14:08.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.681+0000 7f70b8000640 1 --2- 172.21.15.161:0/2218319744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70b013cfc0 0x7f70b01402e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.681+0000 7f70b8000640 1 -- 172.21.15.161:0/2218319744 >> 172.21.15.161:0/2218319744 conn(0x7f70b006a2d0 msgr2=0x7f70b006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:08.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.681+0000 7f70b8000640 1 -- 172.21.15.161:0/2218319744 shutdown_connections 2024-05-22T21:14:08.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.681+0000 7f70b8000640 1 -- 172.21.15.161:0/2218319744 wait complete. 2024-05-22T21:14:08.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b8000640 1 Processor -- start 2024-05-22T21:14:08.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b8000640 1 -- start start 2024-05-22T21:14:08.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b8000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b013cfc0 0x7f70b01fa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:08.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b8000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70b013fe50 0x7f70b01fd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:08.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b8000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70b015af60 0x7f70b0141280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:08.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b8000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b013e0d0 con 0x7f70b013cfc0 2024-05-22T21:14:08.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b8000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b01fd740 con 0x7f70b015af60 2024-05-22T21:14:08.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b8000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b013d730 con 0x7f70b013fe50 2024-05-22T21:14:08.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b5600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b013cfc0 0x7f70b01fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:08.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b5600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b013cfc0 0x7f70b01fa340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39702/0 (socket says 172.21.15.161:39702) 2024-05-22T21:14:08.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b5600640 1 -- 172.21.15.161:0/534743033 learned_addr learned my addr 172.21.15.161:0/534743033 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:08.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b4c00640 1 --2- 172.21.15.161:0/534743033 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70b013fe50 0x7f70b01fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:08.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b5600640 1 -- 172.21.15.161:0/534743033 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70b013fe50 msgr2=0x7f70b01fd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:08.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b6000640 1 --2- 172.21.15.161:0/534743033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70b015af60 0x7f70b0141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:08.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b5600640 1 --2- 172.21.15.161:0/534743033 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70b013fe50 0x7f70b01fd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b5600640 1 -- 172.21.15.161:0/534743033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70b015af60 msgr2=0x7f70b0141280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:08.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b5600640 1 --2- 172.21.15.161:0/534743033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70b015af60 0x7f70b0141280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:08.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b5600640 1 -- 172.21.15.161:0/534743033 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a4009420 con 0x7f70b013cfc0 2024-05-22T21:14:08.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b6000640 1 --2- 172.21.15.161:0/534743033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70b015af60 0x7f70b0141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:08.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b4c00640 1 --2- 172.21.15.161:0/534743033 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70b013fe50 0x7f70b01fd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:08.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b5600640 1 --2- 172.21.15.161:0/534743033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b013cfc0 0x7f70b01fa340 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f70a401a010 tx=0x7f70a4019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:08.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70ad600640 1 -- 172.21.15.161:0/534743033 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a40b5600 con 0x7f70b013cfc0 2024-05-22T21:14:08.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70ad600640 1 -- 172.21.15.161:0/534743033 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a40b5760 con 0x7f70b013cfc0 2024-05-22T21:14:08.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70ad600640 1 -- 172.21.15.161:0/534743033 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a40b5b10 con 0x7f70b013cfc0 2024-05-22T21:14:08.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.685+0000 7f70b8000640 1 -- 172.21.15.161:0/534743033 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b013da10 con 0x7f70b013cfc0 2024-05-22T21:14:08.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.689+0000 7f70b8000640 1 -- 172.21.15.161:0/534743033 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b01417c0 con 0x7f70b013cfc0 2024-05-22T21:14:08.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.689+0000 7f70b8000640 1 -- 172.21.15.161:0/534743033 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b0067220 con 0x7f70b013cfc0 2024-05-22T21:14:08.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.689+0000 7f70ad600640 1 -- 172.21.15.161:0/534743033 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f70a40b3040 con 0x7f70b013cfc0 2024-05-22T21:14:08.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.689+0000 7f70ad600640 1 --2- 172.21.15.161:0/534743033 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7090066690 0x7f7090068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:08.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.689+0000 7f70ad600640 1 -- 172.21.15.161:0/534743033 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f70a411e490 con 0x7f70b013cfc0 2024-05-22T21:14:08.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.689+0000 7f70b4c00640 1 --2- 172.21.15.161:0/534743033 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7090066690 0x7f7090068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:08.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.689+0000 7f70b4c00640 1 --2- 172.21.15.161:0/534743033 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7090066690 0x7f7090068b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f70b015d1a0 tx=0x7f709c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:08.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.693+0000 7f70ad600640 1 -- 172.21.15.161:0/534743033 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f70a40eb700 con 0x7f70b013cfc0 2024-05-22T21:14:08.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:08.881+0000 7f70b8000640 1 -- 172.21.15.161:0/534743033 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f70b0157e60 con 0x7f70b013cfc0 2024-05-22T21:14:09.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.517+0000 7f70ad600640 1 -- 172.21.15.161:0/534743033 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f70a40eed20 con 0x7f70b013cfc0 2024-05-22T21:14:09.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.521+0000 7f70b8000640 1 -- 172.21.15.161:0/534743033 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7090066690 msgr2=0x7f7090068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:09.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.521+0000 7f70b8000640 1 --2- 172.21.15.161:0/534743033 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7090066690 0x7f7090068b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f70b015d1a0 tx=0x7f709c009040 comp rx=0 tx=0).stop 2024-05-22T21:14:09.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.521+0000 7f70b8000640 1 -- 172.21.15.161:0/534743033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b013cfc0 msgr2=0x7f70b01fa340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:09.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.521+0000 7f70b8000640 1 --2- 172.21.15.161:0/534743033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b013cfc0 0x7f70b01fa340 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f70a401a010 tx=0x7f70a4019180 comp rx=0 tx=0).stop 2024-05-22T21:14:09.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.521+0000 7f70b8000640 1 -- 172.21.15.161:0/534743033 shutdown_connections 2024-05-22T21:14:09.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.521+0000 7f70b8000640 1 --2- 172.21.15.161:0/534743033 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7090066690 0x7f7090068b50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:09.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.521+0000 7f70b8000640 1 --2- 172.21.15.161:0/534743033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70b015af60 0x7f70b0141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:09.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.521+0000 7f70b8000640 1 --2- 172.21.15.161:0/534743033 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70b013fe50 0x7f70b01fd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:09.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.521+0000 7f70b8000640 1 --2- 172.21.15.161:0/534743033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70b013cfc0 0x7f70b01fa340 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:09.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.521+0000 7f70b8000640 1 -- 172.21.15.161:0/534743033 >> 172.21.15.161:0/534743033 conn(0x7f70b006a2d0 msgr2=0x7f70b00b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:09.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.521+0000 7f70b8000640 1 -- 172.21.15.161:0/534743033 shutdown_connections 2024-05-22T21:14:09.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.521+0000 7f70b8000640 1 -- 172.21.15.161:0/534743033 wait complete. 2024-05-22T21:14:09.548 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-22T21:14:09.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.685+0000 7f63f1a00640 1 Processor -- start 2024-05-22T21:14:09.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.685+0000 7f63f1a00640 1 -- start start 2024-05-22T21:14:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.689+0000 7f63f1a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f63ec144980 0x7f63ec15c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.689+0000 7f63f1a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f63ec0ccc10 0x7f63ec15c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.689+0000 7f63f1a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ec06d0d0 con 0x7f63ec06d560 2024-05-22T21:14:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.689+0000 7f63f1a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ec15c680 con 0x7f63ec144980 2024-05-22T21:14:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.689+0000 7f63f1a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec06c4c0 con 0x7f63ec0ccc10 2024-05-22T21:14:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.689+0000 7f63eaa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f63ec144980 0x7f63ec15c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.689+0000 7f63eaa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f63ec144980 0x7f63ec15c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39710/0 (socket says 172.21.15.161:39710) 2024-05-22T21:14:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.689+0000 7f63eaa00640 1 -- 172.21.15.161:0/914394636 learned_addr learned my addr 172.21.15.161:0/914394636 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.689+0000 7f63ea000640 1 --1- 172.21.15.161:0/914394636 >> v1:172.21.15.161:6789/0 conn(0x7f63ec06d560 0x7f63ec0cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59212/0 (socket says 172.21.15.161:59212) 2024-05-22T21:14:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.689+0000 7f63eaa00640 1 -- 172.21.15.161:0/914394636 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f63ec0ccc10 msgr2=0x7f63ec15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.689+0000 7f63eaa00640 1 --2- 172.21.15.161:0/914394636 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f63ec0ccc10 0x7f63ec15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.689+0000 7f63eaa00640 1 -- 172.21.15.161:0/914394636 >> v1:172.21.15.161:6789/0 conn(0x7f63ec06d560 legacy=0x7f63ec0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.689+0000 7f63eaa00640 1 -- 172.21.15.161:0/914394636 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ec06c7a0 con 0x7f63ec144980 2024-05-22T21:14:09.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.689+0000 7f63e9600640 1 -- 172.21.15.161:0/914394636 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3389942249 0 0) 0x7f63ec06d0d0 con 0x7f63ec06d560 2024-05-22T21:14:09.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.693+0000 7f63eaa00640 1 --2- 172.21.15.161:0/914394636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63ec144980 0x7f63ec15c140 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009e50 tx=0x7f63dc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb83492902668218 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:09.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.693+0000 7f63e9600640 1 -- 172.21.15.161:0/914394636 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0a57d0 con 0x7f63ec144980 2024-05-22T21:14:09.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.693+0000 7f63e9600640 1 -- 172.21.15.161:0/914394636 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63dc0a5930 con 0x7f63ec144980 2024-05-22T21:14:09.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.693+0000 7f63e9600640 1 -- 172.21.15.161:0/914394636 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0a5cc0 con 0x7f63ec144980 2024-05-22T21:14:09.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.693+0000 7f63f1a00640 1 -- 172.21.15.161:0/914394636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63ec144980 msgr2=0x7f63ec15c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:09.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.693+0000 7f63f1a00640 1 --2- 172.21.15.161:0/914394636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63ec144980 0x7f63ec15c140 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009e50 tx=0x7f63dc01ca50 comp rx=0 tx=0).stop 2024-05-22T21:14:09.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.693+0000 7f63f1a00640 1 -- 172.21.15.161:0/914394636 shutdown_connections 2024-05-22T21:14:09.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.693+0000 7f63f1a00640 1 --2- 172.21.15.161:0/914394636 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f63ec0ccc10 0x7f63ec15c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:09.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.693+0000 7f63f1a00640 1 --2- 172.21.15.161:0/914394636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63ec144980 0x7f63ec15c140 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:09.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.693+0000 7f63f1a00640 1 -- 172.21.15.161:0/914394636 >> 172.21.15.161:0/914394636 conn(0x7f63ec06a0a0 msgr2=0x7f63ec06a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:09.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.693+0000 7f63f1a00640 1 -- 172.21.15.161:0/914394636 shutdown_connections 2024-05-22T21:14:09.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.693+0000 7f63f1a00640 1 -- 172.21.15.161:0/914394636 wait complete. 2024-05-22T21:14:09.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.693+0000 7f63f1a00640 1 Processor -- start 2024-05-22T21:14:09.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.693+0000 7f63f1a00640 1 -- start start 2024-05-22T21:14:09.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63f1a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63ec06d560 0x7f63ec12d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:09.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63f1a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f63ec0ccc10 0x7f63ec071690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:09.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63f1a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ec144980 0x7f63ec155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:09.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63f1a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec1412e0 con 0x7f63ec06d560 2024-05-22T21:14:09.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63f1a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec12d900 con 0x7f63ec144980 2024-05-22T21:14:09.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63f1a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec140a40 con 0x7f63ec0ccc10 2024-05-22T21:14:09.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63eaa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63ec06d560 0x7f63ec12d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:09.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63eaa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63ec06d560 0x7f63ec12d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39712/0 (socket says 172.21.15.161:39712) 2024-05-22T21:14:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63eaa00640 1 -- 172.21.15.161:0/2751730070 learned_addr learned my addr 172.21.15.161:0/2751730070 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63ea000640 1 --2- 172.21.15.161:0/2751730070 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f63ec0ccc10 0x7f63ec071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63eb400640 1 --2- 172.21.15.161:0/2751730070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ec144980 0x7f63ec155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63eaa00640 1 -- 172.21.15.161:0/2751730070 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f63ec0ccc10 msgr2=0x7f63ec071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63eaa00640 1 --2- 172.21.15.161:0/2751730070 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f63ec0ccc10 0x7f63ec071690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63eaa00640 1 -- 172.21.15.161:0/2751730070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ec144980 msgr2=0x7f63ec155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63eaa00640 1 --2- 172.21.15.161:0/2751730070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ec144980 0x7f63ec155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63eaa00640 1 -- 172.21.15.161:0/2751730070 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63dc009b30 con 0x7f63ec06d560 2024-05-22T21:14:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63eb400640 1 --2- 172.21.15.161:0/2751730070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ec144980 0x7f63ec155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63eaa00640 1 --2- 172.21.15.161:0/2751730070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63ec06d560 0x7f63ec12d3c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f63dc01b100 tx=0x7f63dc0197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:09.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63e2a00640 1 -- 172.21.15.161:0/2751730070 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0b8680 con 0x7f63ec06d560 2024-05-22T21:14:09.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63e2a00640 1 -- 172.21.15.161:0/2751730070 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63dc0b8800 con 0x7f63ec06d560 2024-05-22T21:14:09.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63f1a00640 1 -- 172.21.15.161:0/2751730070 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ec06db50 con 0x7f63ec06d560 2024-05-22T21:14:09.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63e2a00640 1 -- 172.21.15.161:0/2751730070 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0b8bb0 con 0x7f63ec06d560 2024-05-22T21:14:09.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63f1a00640 1 -- 172.21.15.161:0/2751730070 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ec141000 con 0x7f63ec06d560 2024-05-22T21:14:09.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63f1a00640 1 -- 172.21.15.161:0/2751730070 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ec06d990 con 0x7f63ec06d560 2024-05-22T21:14:09.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.697+0000 7f63e2a00640 1 -- 172.21.15.161:0/2751730070 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f63dc0b8d10 con 0x7f63ec06d560 2024-05-22T21:14:09.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.701+0000 7f63e2a00640 1 --2- 172.21.15.161:0/2751730070 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f63cc066690 0x7f63cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:09.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.701+0000 7f63e2a00640 1 -- 172.21.15.161:0/2751730070 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f63dc11f900 con 0x7f63ec06d560 2024-05-22T21:14:09.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.701+0000 7f63ea000640 1 --2- 172.21.15.161:0/2751730070 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f63cc066690 0x7f63cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:09.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.701+0000 7f63ea000640 1 --2- 172.21.15.161:0/2751730070 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f63cc066690 0x7f63cc068b50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f63ec12dc60 tx=0x7f63d8007b60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:09.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.705+0000 7f63e2a00640 1 -- 172.21.15.161:0/2751730070 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f63dc0ecc20 con 0x7f63ec06d560 2024-05-22T21:14:09.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:09.857+0000 7f63f1a00640 1 -- 172.21.15.161:0/2751730070 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f63ec067220 con 0x7f63ec06d560 2024-05-22T21:14:10.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.625+0000 7f63e2a00640 1 -- 172.21.15.161:0/2751730070 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7f63dc0f0240 con 0x7f63ec06d560 2024-05-22T21:14:10.632 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' removed 2024-05-22T21:14:10.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.633+0000 7f63f1a00640 1 -- 172.21.15.161:0/2751730070 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f63cc066690 msgr2=0x7f63cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:10.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.633+0000 7f63f1a00640 1 --2- 172.21.15.161:0/2751730070 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f63cc066690 0x7f63cc068b50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f63ec12dc60 tx=0x7f63d8007b60 comp rx=0 tx=0).stop 2024-05-22T21:14:10.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.633+0000 7f63f1a00640 1 -- 172.21.15.161:0/2751730070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63ec06d560 msgr2=0x7f63ec12d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:10.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.633+0000 7f63f1a00640 1 --2- 172.21.15.161:0/2751730070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63ec06d560 0x7f63ec12d3c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f63dc01b100 tx=0x7f63dc0197f0 comp rx=0 tx=0).stop 2024-05-22T21:14:10.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.633+0000 7f63f1a00640 1 -- 172.21.15.161:0/2751730070 shutdown_connections 2024-05-22T21:14:10.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.633+0000 7f63f1a00640 1 --2- 172.21.15.161:0/2751730070 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f63cc066690 0x7f63cc068b50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:10.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.633+0000 7f63f1a00640 1 --2- 172.21.15.161:0/2751730070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63ec144980 0x7f63ec155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:10.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.633+0000 7f63f1a00640 1 --2- 172.21.15.161:0/2751730070 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f63ec0ccc10 0x7f63ec071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:10.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.633+0000 7f63f1a00640 1 --2- 172.21.15.161:0/2751730070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63ec06d560 0x7f63ec12d3c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:10.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.633+0000 7f63f1a00640 1 -- 172.21.15.161:0/2751730070 >> 172.21.15.161:0/2751730070 conn(0x7f63ec06a0a0 msgr2=0x7f63ec1437d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:10.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.633+0000 7f63f1a00640 1 -- 172.21.15.161:0/2751730070 shutdown_connections 2024-05-22T21:14:10.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.633+0000 7f63f1a00640 1 -- 172.21.15.161:0/2751730070 wait complete. 2024-05-22T21:14:10.662 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-22T21:14:10.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.797+0000 7f7988000640 1 Processor -- start 2024-05-22T21:14:10.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.797+0000 7f7988000640 1 -- start start 2024-05-22T21:14:10.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.797+0000 7f7988000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7980143630 0x7f798006d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:10.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.797+0000 7f7988000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f798006d290 con 0x7f7980161510 2024-05-22T21:14:10.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.797+0000 7f7988000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79800adf60 con 0x7f7980143a10 2024-05-22T21:14:10.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.797+0000 7f7988000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798006cc40 con 0x7f7980143630 2024-05-22T21:14:10.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.797+0000 7f7986000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f7980161510 0x7f798006fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59222/0 (socket says 172.21.15.161:59222) 2024-05-22T21:14:10.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.797+0000 7f7986000640 1 -- 172.21.15.161:0/779144136 learned_addr learned my addr 172.21.15.161:0/779144136 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:10.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.797+0000 7f7985600640 1 --2- 172.21.15.161:0/779144136 >> v2:172.21.15.179:3300/0 conn(0x7f7980143630 0x7f798006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:10.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.801+0000 7f797fe00640 1 -- 172.21.15.161:0/779144136 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805578536 0 0) 0x7f798006d290 con 0x7f7980161510 2024-05-22T21:14:10.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.801+0000 7f797fe00640 1 -- 172.21.15.161:0/779144136 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7968003530 con 0x7f7980161510 2024-05-22T21:14:10.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.801+0000 7f7985600640 1 -- 172.21.15.161:0/779144136 >> v1:172.21.15.179:6789/0 conn(0x7f7980143a10 legacy=0x7f7980160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:10.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.801+0000 7f797fe00640 1 -- 172.21.15.161:0/779144136 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1567199121 0 0) 0x7f7968003530 con 0x7f7980161510 2024-05-22T21:14:10.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.801+0000 7f7985600640 1 -- 172.21.15.161:0/779144136 >> v1:172.21.15.161:6789/0 conn(0x7f7980161510 legacy=0x7f798006fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:10.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.801+0000 7f7985600640 1 -- 172.21.15.161:0/779144136 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798006cf20 con 0x7f7980143630 2024-05-22T21:14:10.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.801+0000 7f7985600640 1 --2- 172.21.15.161:0/779144136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980143630 0x7f798006d720 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f79740097c0 tx=0x7f797401c270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ca579bbf520c361 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:10.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.801+0000 7f797fe00640 1 -- 172.21.15.161:0/779144136 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978001a00 con 0x7f7980143630 2024-05-22T21:14:10.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.801+0000 7f797fe00640 1 -- 172.21.15.161:0/779144136 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79740a7050 con 0x7f7980143630 2024-05-22T21:14:10.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.801+0000 7f797fe00640 1 -- 172.21.15.161:0/779144136 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79740ab600 con 0x7f7980143630 2024-05-22T21:14:10.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7988000640 1 -- 172.21.15.161:0/779144136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980143630 msgr2=0x7f798006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:10.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7988000640 1 --2- 172.21.15.161:0/779144136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980143630 0x7f798006d720 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f79740097c0 tx=0x7f797401c270 comp rx=0 tx=0).stop 2024-05-22T21:14:10.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7988000640 1 -- 172.21.15.161:0/779144136 shutdown_connections 2024-05-22T21:14:10.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7988000640 1 --2- 172.21.15.161:0/779144136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980143630 0x7f798006d720 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:10.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7988000640 1 -- 172.21.15.161:0/779144136 >> 172.21.15.161:0/779144136 conn(0x7f798006a0a0 msgr2=0x7f798006a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:10.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7988000640 1 -- 172.21.15.161:0/779144136 shutdown_connections 2024-05-22T21:14:10.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7988000640 1 -- 172.21.15.161:0/779144136 wait complete. 2024-05-22T21:14:10.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7988000640 1 Processor -- start 2024-05-22T21:14:10.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7988000640 1 -- start start 2024-05-22T21:14:10.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7988000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980143630 0x7f798015c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:10.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7988000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7980143a10 0x7f79801452d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:10.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7988000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7980161510 0x7f7980138e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:10.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7988000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798015e700 con 0x7f7980161510 2024-05-22T21:14:10.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7988000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7980145cb0 con 0x7f7980143630 2024-05-22T21:14:10.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7988000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7980145f90 con 0x7f7980143a10 2024-05-22T21:14:10.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7986000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7980161510 0x7f7980138e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:10.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7986000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7980161510 0x7f7980138e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39726/0 (socket says 172.21.15.161:39726) 2024-05-22T21:14:10.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7986000640 1 -- 172.21.15.161:0/295695875 learned_addr learned my addr 172.21.15.161:0/295695875 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:10.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7985600640 1 --2- 172.21.15.161:0/295695875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980143630 0x7f798015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:10.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7986000640 1 -- 172.21.15.161:0/295695875 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7980143a10 msgr2=0x7f79801452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:10.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7986000640 1 --2- 172.21.15.161:0/295695875 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7980143a10 0x7f79801452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:10.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7986000640 1 -- 172.21.15.161:0/295695875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980143630 msgr2=0x7f798015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:10.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7986000640 1 --2- 172.21.15.161:0/295695875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980143630 0x7f798015c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:10.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7986000640 1 -- 172.21.15.161:0/295695875 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7974009420 con 0x7f7980161510 2024-05-22T21:14:10.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7985600640 1 --2- 172.21.15.161:0/295695875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980143630 0x7f798015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:10.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.805+0000 7f7986000640 1 --2- 172.21.15.161:0/295695875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7980161510 0x7f7980138e20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f797800c810 tx=0x7f797800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:10.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.809+0000 7f797d600640 1 -- 172.21.15.161:0/295695875 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797800a640 con 0x7f7980161510 2024-05-22T21:14:10.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.809+0000 7f797d600640 1 -- 172.21.15.161:0/295695875 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797800a7a0 con 0x7f7980161510 2024-05-22T21:14:10.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.809+0000 7f797d600640 1 -- 172.21.15.161:0/295695875 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797800ab50 con 0x7f7980161510 2024-05-22T21:14:10.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.809+0000 7f7988000640 1 -- 172.21.15.161:0/295695875 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f798015e9a0 con 0x7f7980161510 2024-05-22T21:14:10.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.809+0000 7f7988000640 1 -- 172.21.15.161:0/295695875 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f798015c880 con 0x7f7980161510 2024-05-22T21:14:10.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.809+0000 7f7988000640 1 -- 172.21.15.161:0/295695875 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7980067220 con 0x7f7980161510 2024-05-22T21:14:10.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.809+0000 7f797d600640 1 -- 172.21.15.161:0/295695875 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f797800acb0 con 0x7f7980161510 2024-05-22T21:14:10.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.809+0000 7f797d600640 1 --2- 172.21.15.161:0/295695875 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f79600666e0 0x7f7960068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:10.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.809+0000 7f797d600640 1 -- 172.21.15.161:0/295695875 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f79780839f0 con 0x7f7980161510 2024-05-22T21:14:10.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.809+0000 7f7985600640 1 --2- 172.21.15.161:0/295695875 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f79600666e0 0x7f7960068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:10.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.809+0000 7f7985600640 1 --2- 172.21.15.161:0/295695875 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f79600666e0 0x7f7960068ba0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f79740ad000 tx=0x7f79740a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:10.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.813+0000 7f797d600640 1 -- 172.21.15.161:0/295695875 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7978085e60 con 0x7f7980161510 2024-05-22T21:14:10.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:10.965+0000 7f7988000640 1 -- 172.21.15.161:0/295695875 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7980160b70 con 0x7f7980161510 2024-05-22T21:14:11.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.633+0000 7f797d600640 1 -- 172.21.15.161:0/295695875 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7978018070 con 0x7f7980161510 2024-05-22T21:14:11.642 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' removed 2024-05-22T21:14:11.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.641+0000 7f7988000640 1 -- 172.21.15.161:0/295695875 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f79600666e0 msgr2=0x7f7960068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:11.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.641+0000 7f7988000640 1 --2- 172.21.15.161:0/295695875 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f79600666e0 0x7f7960068ba0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f79740ad000 tx=0x7f79740a6000 comp rx=0 tx=0).stop 2024-05-22T21:14:11.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.641+0000 7f7988000640 1 -- 172.21.15.161:0/295695875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7980161510 msgr2=0x7f7980138e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:11.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.641+0000 7f7988000640 1 --2- 172.21.15.161:0/295695875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7980161510 0x7f7980138e20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f797800c810 tx=0x7f797800ccd0 comp rx=0 tx=0).stop 2024-05-22T21:14:11.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.641+0000 7f7988000640 1 -- 172.21.15.161:0/295695875 shutdown_connections 2024-05-22T21:14:11.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.641+0000 7f7988000640 1 --2- 172.21.15.161:0/295695875 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f79600666e0 0x7f7960068ba0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:11.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.641+0000 7f7988000640 1 --2- 172.21.15.161:0/295695875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7980161510 0x7f7980138e20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:11.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.641+0000 7f7988000640 1 --2- 172.21.15.161:0/295695875 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7980143a10 0x7f79801452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:11.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.641+0000 7f7988000640 1 --2- 172.21.15.161:0/295695875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980143630 0x7f798015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:11.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.641+0000 7f7988000640 1 -- 172.21.15.161:0/295695875 >> 172.21.15.161:0/295695875 conn(0x7f798006a0a0 msgr2=0x7f798006c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:11.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.641+0000 7f7988000640 1 -- 172.21.15.161:0/295695875 shutdown_connections 2024-05-22T21:14:11.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.641+0000 7f7988000640 1 -- 172.21.15.161:0/295695875 wait complete. 2024-05-22T21:14:11.667 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-22T21:14:11.667 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-22T21:14:11.667 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-22T21:14:11.668 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:14:11.668 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-22T21:14:11.668 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-22T21:14:11.668 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-22T21:14:11.668 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:14:11.668 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-22T21:14:11.668 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-22T21:14:11.669 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-22T21:14:11.669 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:14:11.669 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-22T21:14:11.669 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-22T21:14:11.669 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-22T21:14:11.669 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:14:11.680 INFO:tasks.ceph.mds.c:Stopped 2024-05-22T21:14:11.680 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-22T21:14:11.681 INFO:tasks.ceph.mds.a:Stopped 2024-05-22T21:14:11.681 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-22T21:14:11.682 INFO:tasks.ceph.mds.b:Stopped 2024-05-22T21:14:11.682 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-22T21:14:11.683 INFO:tasks.ceph.mds.d:Stopped 2024-05-22T21:14:11.683 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-22T21:14:11.726 INFO:tasks.ceph.mds.c:Started 2024-05-22T21:14:11.726 DEBUG:teuthology.parallel:result is None 2024-05-22T21:14:11.727 INFO:tasks.ceph.mds.a:Started 2024-05-22T21:14:11.728 DEBUG:teuthology.parallel:result is None 2024-05-22T21:14:11.729 INFO:tasks.ceph.mds.b:Started 2024-05-22T21:14:11.729 DEBUG:teuthology.parallel:result is None 2024-05-22T21:14:11.731 INFO:tasks.ceph.mds.d:Started 2024-05-22T21:14:11.731 DEBUG:teuthology.parallel:result is None 2024-05-22T21:14:11.731 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-22T21:14:11.813 INFO:tasks.ceph.mds.a.smithi161.stdout:starting mds.a at 2024-05-22T21:14:11.814 INFO:tasks.ceph.mds.c.smithi161.stdout:starting mds.c at 2024-05-22T21:14:11.816 INFO:tasks.ceph.mds.b.smithi179.stdout:starting mds.b at 2024-05-22T21:14:11.817 INFO:tasks.ceph.mds.d.smithi179.stdout:starting mds.d at 2024-05-22T21:14:11.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.873+0000 7f0c18800640 1 Processor -- start 2024-05-22T21:14:11.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.873+0000 7f0c18800640 1 -- start start 2024-05-22T21:14:11.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.873+0000 7f0c18800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c1013cfc0 0x7f0c101402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:11.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.873+0000 7f0c18800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0c1013fe50 0x7f0c1006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:11.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.873+0000 7f0c18800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1013d3a0 con 0x7f0c1015af60 2024-05-22T21:14:11.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.873+0000 7f0c18800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c100cd920 con 0x7f0c1013fe50 2024-05-22T21:14:11.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.873+0000 7f0c18800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c100cdc00 con 0x7f0c1013cfc0 2024-05-22T21:14:11.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.873+0000 7f0c15e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c1013cfc0 0x7f0c101402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:11.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.873+0000 7f0c16800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0c1013fe50 0x7f0c1006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:11.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.873+0000 7f0c15e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c1013cfc0 0x7f0c101402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:47234/0 (socket says 172.21.15.161:47234) 2024-05-22T21:14:11.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.873+0000 7f0c15e00640 1 -- 172.21.15.161:0/2234415827 learned_addr learned my addr 172.21.15.161:0/2234415827 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:11.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.873+0000 7f0c16800640 1 --2- 172.21.15.161:0/2234415827 >> v2:172.21.15.179:3300/0 conn(0x7f0c1013fe50 0x7f0c1006fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:43912/0 (socket says 172.21.15.161:43912) 2024-05-22T21:14:11.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.873+0000 7f0c15400640 1 --1- 172.21.15.161:0/2234415827 >> v1:172.21.15.179:6789/0 conn(0x7f0c1015af60 0x7f0c1013f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:53566/0 (socket says 172.21.15.161:53566) 2024-05-22T21:14:11.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.877+0000 7f0c14a00640 1 -- 172.21.15.161:0/2234415827 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425309215 0 0) 0x7f0c1013d3a0 con 0x7f0c1015af60 2024-05-22T21:14:11.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.877+0000 7f0c16800640 1 -- 172.21.15.161:0/2234415827 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c1013cfc0 msgr2=0x7f0c101402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:11.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.877+0000 7f0c16800640 1 --2- 172.21.15.161:0/2234415827 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c1013cfc0 0x7f0c101402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:11.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.877+0000 7f0c16800640 1 -- 172.21.15.161:0/2234415827 >> v1:172.21.15.179:6789/0 conn(0x7f0c1015af60 legacy=0x7f0c1013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:11.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.877+0000 7f0c16800640 1 -- 172.21.15.161:0/2234415827 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c100cdee0 con 0x7f0c1013fe50 2024-05-22T21:14:11.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.877+0000 7f0c15e00640 1 --2- 172.21.15.161:0/2234415827 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c1013cfc0 0x7f0c101402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:11.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.877+0000 7f0c16800640 1 --2- 172.21.15.161:0/2234415827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c1013fe50 0x7f0c1006fce0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009d90 tx=0x7f0c0c01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf7cb49cca7bf1b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:11.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.877+0000 7f0c14a00640 1 -- 172.21.15.161:0/2234415827 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0a8610 con 0x7f0c1013fe50 2024-05-22T21:14:11.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.877+0000 7f0c14a00640 1 -- 172.21.15.161:0/2234415827 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c0a8770 con 0x7f0c1013fe50 2024-05-22T21:14:11.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.877+0000 7f0c14a00640 1 -- 172.21.15.161:0/2234415827 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0a8b00 con 0x7f0c1013fe50 2024-05-22T21:14:11.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c18800640 1 -- 172.21.15.161:0/2234415827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c1013fe50 msgr2=0x7f0c1006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:11.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c18800640 1 --2- 172.21.15.161:0/2234415827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c1013fe50 0x7f0c1006fce0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009d90 tx=0x7f0c0c01c8b0 comp rx=0 tx=0).stop 2024-05-22T21:14:11.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c18800640 1 -- 172.21.15.161:0/2234415827 shutdown_connections 2024-05-22T21:14:11.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c18800640 1 --2- 172.21.15.161:0/2234415827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c1013fe50 0x7f0c1006fce0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:11.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c18800640 1 --2- 172.21.15.161:0/2234415827 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c1013cfc0 0x7f0c101402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:11.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c18800640 1 -- 172.21.15.161:0/2234415827 >> 172.21.15.161:0/2234415827 conn(0x7f0c1006a2d0 msgr2=0x7f0c1006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:11.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c18800640 1 -- 172.21.15.161:0/2234415827 shutdown_connections 2024-05-22T21:14:11.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c18800640 1 -- 172.21.15.161:0/2234415827 wait complete. 2024-05-22T21:14:11.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c18800640 1 Processor -- start 2024-05-22T21:14:11.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c18800640 1 -- start start 2024-05-22T21:14:11.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c18800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c1013cfc0 0x7f0c10141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:11.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c18800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c1013fe50 0x7f0c1013d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:11.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c18800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c1015af60 0x7f0c1013b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:11.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c18800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c101424d0 con 0x7f0c1013fe50 2024-05-22T21:14:11.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c18800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1013f320 con 0x7f0c1015af60 2024-05-22T21:14:11.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c18800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1013f600 con 0x7f0c1013cfc0 2024-05-22T21:14:11.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c15400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c1013fe50 0x7f0c1013d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:11.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c15400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c1013fe50 0x7f0c1013d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39762/0 (socket says 172.21.15.161:39762) 2024-05-22T21:14:11.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c15400640 1 -- 172.21.15.161:0/3827923356 learned_addr learned my addr 172.21.15.161:0/3827923356 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:11.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c15e00640 1 --2- 172.21.15.161:0/3827923356 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c1013cfc0 0x7f0c10141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:11.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c16800640 1 --2- 172.21.15.161:0/3827923356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c1015af60 0x7f0c1013b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:11.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c15400640 1 -- 172.21.15.161:0/3827923356 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c1013cfc0 msgr2=0x7f0c10141ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:11.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c15400640 1 --2- 172.21.15.161:0/3827923356 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c1013cfc0 0x7f0c10141ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:11.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c15400640 1 -- 172.21.15.161:0/3827923356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c1015af60 msgr2=0x7f0c1013b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:11.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c15400640 1 --2- 172.21.15.161:0/3827923356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c1015af60 0x7f0c1013b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:11.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c15400640 1 -- 172.21.15.161:0/3827923356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c040094d0 con 0x7f0c1013fe50 2024-05-22T21:14:11.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c16800640 1 --2- 172.21.15.161:0/3827923356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c1015af60 0x7f0c1013b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:11.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.881+0000 7f0c15400640 1 --2- 172.21.15.161:0/3827923356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c1013fe50 0x7f0c1013d5b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000c450 tx=0x7f0c0000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:11.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.885+0000 7f0c0a000640 1 -- 172.21.15.161:0/3827923356 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00006480 con 0x7f0c1013fe50 2024-05-22T21:14:11.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.885+0000 7f0c0a000640 1 -- 172.21.15.161:0/3827923356 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c00006600 con 0x7f0c1013fe50 2024-05-22T21:14:11.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.885+0000 7f0c0a000640 1 -- 172.21.15.161:0/3827923356 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00004a10 con 0x7f0c1013fe50 2024-05-22T21:14:11.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.885+0000 7f0c18800640 1 -- 172.21.15.161:0/3827923356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0c009a70 con 0x7f0c1013fe50 2024-05-22T21:14:11.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.885+0000 7f0c18800640 1 -- 172.21.15.161:0/3827923356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c101f5e50 con 0x7f0c1013fe50 2024-05-22T21:14:11.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.885+0000 7f0c18800640 1 -- 172.21.15.161:0/3827923356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1013fb20 con 0x7f0c1013fe50 2024-05-22T21:14:11.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.885+0000 7f0c0a000640 1 -- 172.21.15.161:0/3827923356 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0c00004b70 con 0x7f0c1013fe50 2024-05-22T21:14:11.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.885+0000 7f0c0a000640 1 --2- 172.21.15.161:0/3827923356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0bf4066690 0x7f0bf4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:11.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.885+0000 7f0c0a000640 1 -- 172.21.15.161:0/3827923356 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0c000834b0 con 0x7f0c1013fe50 2024-05-22T21:14:11.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.885+0000 7f0c15e00640 1 --2- 172.21.15.161:0/3827923356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0bf4066690 0x7f0bf4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:11.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.889+0000 7f0c15e00640 1 --2- 172.21.15.161:0/3827923356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0bf4066690 0x7f0bf4068b50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009c70 tx=0x7f0c04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:11.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:11.889+0000 7f0c0a000640 1 -- 172.21.15.161:0/3827923356 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0c00050a80 con 0x7f0c1013fe50 2024-05-22T21:14:12.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:12.053+0000 7f0c18800640 1 -- 172.21.15.161:0/3827923356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0c10067220 con 0x7f0c1013fe50 2024-05-22T21:14:12.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:12.845+0000 7f0c0a000640 1 -- 172.21.15.161:0/3827923356 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0c00017020 con 0x7f0c1013fe50 2024-05-22T21:14:12.849 INFO:teuthology.orchestra.run.smithi161.stderr: removed all blocklist entries 2024-05-22T21:14:12.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:12.845+0000 7f0c18800640 1 -- 172.21.15.161:0/3827923356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0bf4066690 msgr2=0x7f0bf4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:12.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:12.845+0000 7f0c18800640 1 --2- 172.21.15.161:0/3827923356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0bf4066690 0x7f0bf4068b50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009c70 tx=0x7f0c04008040 comp rx=0 tx=0).stop 2024-05-22T21:14:12.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:12.845+0000 7f0c18800640 1 -- 172.21.15.161:0/3827923356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c1013fe50 msgr2=0x7f0c1013d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:12.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:12.845+0000 7f0c18800640 1 --2- 172.21.15.161:0/3827923356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c1013fe50 0x7f0c1013d5b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000c450 tx=0x7f0c0000c910 comp rx=0 tx=0).stop 2024-05-22T21:14:12.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:12.849+0000 7f0c18800640 1 -- 172.21.15.161:0/3827923356 shutdown_connections 2024-05-22T21:14:12.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:12.849+0000 7f0c18800640 1 --2- 172.21.15.161:0/3827923356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0bf4066690 0x7f0bf4068b50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:12.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:12.849+0000 7f0c18800640 1 --2- 172.21.15.161:0/3827923356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c1015af60 0x7f0c1013b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:12.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:12.849+0000 7f0c18800640 1 --2- 172.21.15.161:0/3827923356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c1013fe50 0x7f0c1013d5b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:12.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:12.849+0000 7f0c18800640 1 --2- 172.21.15.161:0/3827923356 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c1013cfc0 0x7f0c10141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:12.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:12.849+0000 7f0c18800640 1 -- 172.21.15.161:0/3827923356 >> 172.21.15.161:0/3827923356 conn(0x7f0c1006a2d0 msgr2=0x7f0c100acb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:12.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:12.849+0000 7f0c18800640 1 -- 172.21.15.161:0/3827923356 shutdown_connections 2024-05-22T21:14:12.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:12.849+0000 7f0c18800640 1 -- 172.21.15.161:0/3827923356 wait complete. 2024-05-22T21:14:12.875 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-22T21:14:13.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.009+0000 7fd6a0800640 1 Processor -- start 2024-05-22T21:14:13.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.009+0000 7fd6a0800640 1 -- start start 2024-05-22T21:14:13.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.009+0000 7fd6a0800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd698160ae0 0x7fd6980cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.009+0000 7fd6a0800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6980b2380 con 0x7fd6980d0090 2024-05-22T21:14:13.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.009+0000 7fd6a0800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6980d04d0 con 0x7fd6980ced60 2024-05-22T21:14:13.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.009+0000 7fd6a0800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6980d1970 con 0x7fd698160ae0 2024-05-22T21:14:13.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.009+0000 7fd69d400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd6980d0090 0x7fd6980b3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59256/0 (socket says 172.21.15.161:59256) 2024-05-22T21:14:13.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.009+0000 7fd69d400640 1 -- 172.21.15.161:0/1146501061 learned_addr learned my addr 172.21.15.161:0/1146501061 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:13.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.009+0000 7fd69de00640 1 --2- 172.21.15.161:0/1146501061 >> v2:172.21.15.179:3300/0 conn(0x7fd698160ae0 0x7fd6980cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.009+0000 7fd69ca00640 1 -- 172.21.15.161:0/1146501061 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019312233 0 0) 0x7fd6980b2380 con 0x7fd6980d0090 2024-05-22T21:14:13.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.009+0000 7fd69ca00640 1 -- 172.21.15.161:0/1146501061 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd680003530 con 0x7fd6980d0090 2024-05-22T21:14:13.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.009+0000 7fd69de00640 1 -- 172.21.15.161:0/1146501061 >> v1:172.21.15.179:6789/0 conn(0x7fd6980ced60 legacy=0x7fd6980b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.009+0000 7fd69ca00640 1 -- 172.21.15.161:0/1146501061 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843468526 0 0) 0x7fd6980d04d0 con 0x7fd6980ced60 2024-05-22T21:14:13.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.009+0000 7fd69de00640 1 -- 172.21.15.161:0/1146501061 >> v1:172.21.15.161:6789/0 conn(0x7fd6980d0090 legacy=0x7fd6980b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.009+0000 7fd69de00640 1 -- 172.21.15.161:0/1146501061 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6980d1c50 con 0x7fd698160ae0 2024-05-22T21:14:13.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.013+0000 7fd69de00640 1 --2- 172.21.15.161:0/1146501061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd698160ae0 0x7fd6980cfb50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0097c0 tx=0x7fd68c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb3ba8bbc69a19a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:13.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.013+0000 7fd69ca00640 1 -- 172.21.15.161:0/1146501061 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c0ac640 con 0x7fd698160ae0 2024-05-22T21:14:13.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.013+0000 7fd69ca00640 1 -- 172.21.15.161:0/1146501061 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c0ac7a0 con 0x7fd698160ae0 2024-05-22T21:14:13.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.013+0000 7fd69ca00640 1 -- 172.21.15.161:0/1146501061 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c0acb30 con 0x7fd698160ae0 2024-05-22T21:14:13.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.013+0000 7fd6a0800640 1 -- 172.21.15.161:0/1146501061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd698160ae0 msgr2=0x7fd6980cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.013+0000 7fd6a0800640 1 --2- 172.21.15.161:0/1146501061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd698160ae0 0x7fd6980cfb50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0097c0 tx=0x7fd68c01c3e0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.013+0000 7fd6a0800640 1 -- 172.21.15.161:0/1146501061 shutdown_connections 2024-05-22T21:14:13.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.013+0000 7fd6a0800640 1 --2- 172.21.15.161:0/1146501061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd698160ae0 0x7fd6980cfb50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.013+0000 7fd6a0800640 1 -- 172.21.15.161:0/1146501061 >> 172.21.15.161:0/1146501061 conn(0x7fd69809c490 msgr2=0x7fd6980d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:13.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.013+0000 7fd6a0800640 1 -- 172.21.15.161:0/1146501061 shutdown_connections 2024-05-22T21:14:13.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.013+0000 7fd6a0800640 1 -- 172.21.15.161:0/1146501061 wait complete. 2024-05-22T21:14:13.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd6a0800640 1 Processor -- start 2024-05-22T21:14:13.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd6a0800640 1 -- start start 2024-05-22T21:14:13.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd6a0800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6980ced60 0x7fd6980b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd6a0800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6980d0090 0x7fd6980b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd6a0800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd698160ae0 0x7fd6980afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd6a0800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980b7370 con 0x7fd698160ae0 2024-05-22T21:14:13.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd6a0800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980cb880 con 0x7fd6980ced60 2024-05-22T21:14:13.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd6a0800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980b67f0 con 0x7fd6980d0090 2024-05-22T21:14:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd69e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd698160ae0 0x7fd6980afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd69e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd698160ae0 0x7fd6980afeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39766/0 (socket says 172.21.15.161:39766) 2024-05-22T21:14:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd69e800640 1 -- 172.21.15.161:0/1637429399 learned_addr learned my addr 172.21.15.161:0/1637429399 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd69de00640 1 --2- 172.21.15.161:0/1637429399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6980ced60 0x7fd6980b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd69d400640 1 --2- 172.21.15.161:0/1637429399 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6980d0090 0x7fd6980b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd69e800640 1 -- 172.21.15.161:0/1637429399 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6980d0090 msgr2=0x7fd6980b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd69e800640 1 --2- 172.21.15.161:0/1637429399 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6980d0090 0x7fd6980b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd69e800640 1 -- 172.21.15.161:0/1637429399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6980ced60 msgr2=0x7fd6980b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd69e800640 1 --2- 172.21.15.161:0/1637429399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6980ced60 0x7fd6980b85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd69e800640 1 -- 172.21.15.161:0/1637429399 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c009420 con 0x7fd698160ae0 2024-05-22T21:14:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd69e800640 1 --2- 172.21.15.161:0/1637429399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd698160ae0 0x7fd6980afeb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd694001570 tx=0x7fd69400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd692000640 1 -- 172.21.15.161:0/1637429399 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694003db0 con 0x7fd698160ae0 2024-05-22T21:14:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd692000640 1 -- 172.21.15.161:0/1637429399 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd694005070 con 0x7fd698160ae0 2024-05-22T21:14:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd6a0800640 1 -- 172.21.15.161:0/1637429399 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6980b6ad0 con 0x7fd698160ae0 2024-05-22T21:14:13.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd692000640 1 -- 172.21.15.161:0/1637429399 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694005420 con 0x7fd698160ae0 2024-05-22T21:14:13.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd6a0800640 1 -- 172.21.15.161:0/1637429399 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6980b7090 con 0x7fd698160ae0 2024-05-22T21:14:13.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.017+0000 7fd6a0800640 1 -- 172.21.15.161:0/1637429399 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6980cba40 con 0x7fd698160ae0 2024-05-22T21:14:13.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.021+0000 7fd692000640 1 -- 172.21.15.161:0/1637429399 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd694005580 con 0x7fd698160ae0 2024-05-22T21:14:13.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.021+0000 7fd692000640 1 --2- 172.21.15.161:0/1637429399 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd67c066690 0x7fd67c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.021+0000 7fd692000640 1 -- 172.21.15.161:0/1637429399 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd694083740 con 0x7fd698160ae0 2024-05-22T21:14:13.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.021+0000 7fd69de00640 1 --2- 172.21.15.161:0/1637429399 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd67c066690 0x7fd67c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.021+0000 7fd69de00640 1 --2- 172.21.15.161:0/1637429399 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd67c066690 0x7fd67c068b50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0ae000 tx=0x7fd68c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:13.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.025+0000 7fd692000640 1 -- 172.21.15.161:0/1637429399 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd694050e10 con 0x7fd698160ae0 2024-05-22T21:14:13.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.229+0000 7fd6a0800640 1 -- 172.21.15.161:0/1637429399 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd698067220 con 0x7fd698160ae0 2024-05-22T21:14:13.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.233+0000 7fd692000640 1 -- 172.21.15.161:0/1637429399 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fd694017020 con 0x7fd698160ae0 2024-05-22T21:14:13.236 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:13.236 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:14:13.236 INFO:teuthology.orchestra.run.smithi161.stdout: "auth_dump": [ 2024-05-22T21:14:13.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.236 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.a", 2024-05-22T21:14:13.237 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m00QTFRAAPSRrpBRY75FlcWf3QxOOfg==", 2024-05-22T21:14:13.237 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.237 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:14:13.237 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:14:13.237 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:14:13.237 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:13.237 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.237 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.b", 2024-05-22T21:14:13.237 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m/ipAIRAA4gj5jGqTpNCr1K2OU/bsAQ==", 2024-05-22T21:14:13.237 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.237 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:14:13.237 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:14:13.238 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:14:13.238 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:13.238 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.238 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.c", 2024-05-22T21:14:13.238 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mWlM2HBAAEmJJWMDlTG9E7Qzld2gd5w==", 2024-05-22T21:14:13.238 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.238 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:14:13.238 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:14:13.238 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:14:13.238 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:13.238 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.239 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.d", 2024-05-22T21:14:13.239 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mmsuaKBAAqpCYRZZUGK5ZuxflUBcPgg==", 2024-05-22T21:14:13.239 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.239 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:14:13.239 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:14:13.239 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:14:13.239 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:13.239 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.239 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.0", 2024-05-22T21:14:13.239 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05mJZokLhAAh2vbQnRX25iYUIbvVmURuQ==", 2024-05-22T21:14:13.239 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.240 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:14:13.240 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:14:13.240 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:13.240 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.240 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.1", 2024-05-22T21:14:13.240 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05m8kWNNhAA6aaVT2xshFkqnk4G4gOttA==", 2024-05-22T21:14:13.240 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.240 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:14:13.240 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:14:13.240 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:13.240 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.241 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.2", 2024-05-22T21:14:13.241 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05ml1k9AxAAtL3XaJJI6cilpmvurh18Pw==", 2024-05-22T21:14:13.241 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.241 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:14:13.241 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:14:13.241 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:13.241 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.241 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.3", 2024-05-22T21:14:13.241 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05mbNBgCxAAas1L+QR4Aez+H1G0EVltfg==", 2024-05-22T21:14:13.241 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.242 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:14:13.242 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:14:13.242 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:13.242 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.242 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.4", 2024-05-22T21:14:13.242 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mOPLPABAAZBGIMF/FasNB27n/7di0AA==", 2024-05-22T21:14:13.242 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.242 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:14:13.242 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:14:13.242 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:13.243 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.243 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.5", 2024-05-22T21:14:13.243 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mT1poCRAASXm4tdP01pA1VOviqeoLVw==", 2024-05-22T21:14:13.243 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.243 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:14:13.243 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:14:13.243 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:13.243 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.243 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.6", 2024-05-22T21:14:13.244 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05m1ZPRERAA2S7WeTwysLCZm1OB/qCrAA==", 2024-05-22T21:14:13.244 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.244 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:14:13.244 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:14:13.244 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:13.244 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.244 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.7", 2024-05-22T21:14:13.244 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mCrUhGhAAvXoFWV7X3303rfXaxmfWqA==", 2024-05-22T21:14:13.244 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.244 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:14:13.244 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:14:13.244 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:13.245 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.245 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.0", 2024-05-22T21:14:13.245 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mh9B7LBAAevhvj5KpwP9pEIpJcxUvFA==", 2024-05-22T21:14:13.245 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.245 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:14:13.245 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:14:13.245 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:14:13.245 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:14:13.245 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.245 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.1", 2024-05-22T21:14:13.245 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mMnHtNRAAg/a/Mb1oRqx73d9efIHCfA==", 2024-05-22T21:14:13.246 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.246 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:14:13.246 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:14:13.246 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:14:13.246 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:14:13.246 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.246 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.2", 2024-05-22T21:14:13.246 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mOzdkMhAAbTiNdvyxqIyrluMiZW0cOw==", 2024-05-22T21:14:13.246 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.246 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:14:13.246 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:14:13.246 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:14:13.247 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:14:13.247 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.247 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.247 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.3", 2024-05-22T21:14:13.247 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDYX05mr4oxABAA1+bQQbxhoeU+kb0OCa+Wyw==", 2024-05-22T21:14:13.247 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.247 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:14:13.247 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:14:13.247 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:14:13.247 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:14:13.247 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.248 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.admin", 2024-05-22T21:14:13.248 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDWX05mHZY0NRAAiBvMDkrCZ+MgG2iwBH+bcg==", 2024-05-22T21:14:13.248 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.248 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:14:13.248 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:14:13.248 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:14:13.248 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:13.248 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.248 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.bootstrap-mds", 2024-05-22T21:14:13.248 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDhX05mXi3OGBAABVot3WZJ+bM+TU8vGuEefw==", 2024-05-22T21:14:13.248 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.248 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile bootstrap-mds" 2024-05-22T21:14:13.249 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.249 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.bootstrap-mgr", 2024-05-22T21:14:13.249 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDhX05m0TzOGBAAXSIzpv2pCrE5efMqJeRdcA==", 2024-05-22T21:14:13.249 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.249 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-22T21:14:13.249 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.249 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.bootstrap-osd", 2024-05-22T21:14:13.249 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDhX05ms0rOGBAAvPk52R1KOpz93wOFIiSJYA==", 2024-05-22T21:14:13.249 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.249 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile bootstrap-osd" 2024-05-22T21:14:13.249 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.250 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.bootstrap-rbd", 2024-05-22T21:14:13.250 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDhX05mQVnOGBAAJTDyIQVThCJ+I9L1Mur9SA==", 2024-05-22T21:14:13.250 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.250 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-22T21:14:13.250 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.250 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-22T21:14:13.250 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDhX05mPGfOGBAAc6l8Mwa5e5mxkXy2GVyI3w==", 2024-05-22T21:14:13.250 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.250 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-22T21:14:13.250 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.251 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.251 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.251 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.bootstrap-rgw", 2024-05-22T21:14:13.251 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDhX05mW3XOGBAAr1OsrfoFDB9JiUy/bx3/7w==", 2024-05-22T21:14:13.251 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.251 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-22T21:14:13.251 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.251 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.251 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.251 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.x", 2024-05-22T21:14:13.251 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mH2kuERAAgigMUWK3M3WaMXlqGUcNrw==", 2024-05-22T21:14:13.251 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.251 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:14:13.251 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:14:13.252 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:13.252 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.252 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:13.252 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:13.252 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.y", 2024-05-22T21:14:13.252 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mZRdiDRAA0vUwD+rPSfyhjgA+9XPp5w==", 2024-05-22T21:14:13.252 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:13.252 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:14:13.252 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:14:13.252 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:13.252 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.252 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:13.252 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:14:13.252 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:14:13.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.233+0000 7fd6a0800640 1 -- 172.21.15.161:0/1637429399 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd67c066690 msgr2=0x7fd67c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.233+0000 7fd6a0800640 1 --2- 172.21.15.161:0/1637429399 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd67c066690 0x7fd67c068b50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0ae000 tx=0x7fd68c0a7000 comp rx=0 tx=0).stop 2024-05-22T21:14:13.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.233+0000 7fd6a0800640 1 -- 172.21.15.161:0/1637429399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd698160ae0 msgr2=0x7fd6980afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.233+0000 7fd6a0800640 1 --2- 172.21.15.161:0/1637429399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd698160ae0 0x7fd6980afeb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd694001570 tx=0x7fd69400cd20 comp rx=0 tx=0).stop 2024-05-22T21:14:13.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.233+0000 7fd6a0800640 1 -- 172.21.15.161:0/1637429399 shutdown_connections 2024-05-22T21:14:13.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.233+0000 7fd6a0800640 1 --2- 172.21.15.161:0/1637429399 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd67c066690 0x7fd67c068b50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.233+0000 7fd6a0800640 1 --2- 172.21.15.161:0/1637429399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd698160ae0 0x7fd6980afeb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.233+0000 7fd6a0800640 1 --2- 172.21.15.161:0/1637429399 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6980d0090 0x7fd6980b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.233+0000 7fd6a0800640 1 --2- 172.21.15.161:0/1637429399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6980ced60 0x7fd6980b85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.233+0000 7fd6a0800640 1 -- 172.21.15.161:0/1637429399 >> 172.21.15.161:0/1637429399 conn(0x7fd69809c490 msgr2=0x7fd69806c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:13.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.233+0000 7fd6a0800640 1 -- 172.21.15.161:0/1637429399 shutdown_connections 2024-05-22T21:14:13.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.233+0000 7fd6a0800640 1 -- 172.21.15.161:0/1637429399 wait complete. 2024-05-22T21:14:13.257 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-22T21:14:13.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.389+0000 7f09b3200640 1 Processor -- start 2024-05-22T21:14:13.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.389+0000 7f09b3200640 1 -- start start 2024-05-22T21:14:13.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.389+0000 7f09b3200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f09ac13cfc0 0x7f09ac1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.389+0000 7f09b3200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f09ac13fe50 0x7f09ac06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.389+0000 7f09b3200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ac13d3a0 con 0x7f09ac15af60 2024-05-22T21:14:13.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.389+0000 7f09b3200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09ac0cd920 con 0x7f09ac13cfc0 2024-05-22T21:14:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.389+0000 7f09b3200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac0cdc00 con 0x7f09ac13fe50 2024-05-22T21:14:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.389+0000 7f09abe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f09ac13cfc0 0x7f09ac1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.389+0000 7f09abe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f09ac13cfc0 0x7f09ac1402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39780/0 (socket says 172.21.15.161:39780) 2024-05-22T21:14:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.389+0000 7f09abe00640 1 -- 172.21.15.161:0/3676993079 learned_addr learned my addr 172.21.15.161:0/3676993079 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.389+0000 7f09b1200640 1 --2- 172.21.15.161:0/3676993079 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f09ac13fe50 0x7f09ac06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.389+0000 7f09abe00640 1 -- 172.21.15.161:0/3676993079 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f09ac13fe50 msgr2=0x7f09ac06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:13.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.389+0000 7f09abe00640 1 --2- 172.21.15.161:0/3676993079 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f09ac13fe50 0x7f09ac06fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.389+0000 7f09abe00640 1 -- 172.21.15.161:0/3676993079 >> v1:172.21.15.161:6789/0 conn(0x7f09ac15af60 legacy=0x7f09ac13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.389+0000 7f09abe00640 1 -- 172.21.15.161:0/3676993079 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac0cdee0 con 0x7f09ac13cfc0 2024-05-22T21:14:13.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.393+0000 7f09abe00640 1 --2- 172.21.15.161:0/3676993079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac13cfc0 0x7f09ac1402e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f099c009870 tx=0x7f099c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3604e32b1da96623 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:13.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.393+0000 7f09ab400640 1 -- 172.21.15.161:0/3676993079 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c0ab600 con 0x7f09ac13cfc0 2024-05-22T21:14:13.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.393+0000 7f09ab400640 1 -- 172.21.15.161:0/3676993079 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c0a7050 con 0x7f09ac13cfc0 2024-05-22T21:14:13.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.393+0000 7f09ab400640 1 -- 172.21.15.161:0/3676993079 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c0ab980 con 0x7f09ac13cfc0 2024-05-22T21:14:13.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.393+0000 7f09b3200640 1 -- 172.21.15.161:0/3676993079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac13cfc0 msgr2=0x7f09ac1402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.393+0000 7f09b3200640 1 --2- 172.21.15.161:0/3676993079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac13cfc0 0x7f09ac1402e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f099c009870 tx=0x7f099c01c4a0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.393+0000 7f09b3200640 1 -- 172.21.15.161:0/3676993079 shutdown_connections 2024-05-22T21:14:13.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.393+0000 7f09b3200640 1 --2- 172.21.15.161:0/3676993079 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f09ac13fe50 0x7f09ac06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.393+0000 7f09b3200640 1 --2- 172.21.15.161:0/3676993079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac13cfc0 0x7f09ac1402e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.393+0000 7f09b3200640 1 -- 172.21.15.161:0/3676993079 >> 172.21.15.161:0/3676993079 conn(0x7f09ac06a2d0 msgr2=0x7f09ac06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:13.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.393+0000 7f09b3200640 1 -- 172.21.15.161:0/3676993079 shutdown_connections 2024-05-22T21:14:13.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.393+0000 7f09b3200640 1 -- 172.21.15.161:0/3676993079 wait complete. 2024-05-22T21:14:13.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09b3200640 1 Processor -- start 2024-05-22T21:14:13.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09b3200640 1 -- start start 2024-05-22T21:14:13.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09b3200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f09ac13cfc0 0x7f09ac138480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09b3200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac13fe50 0x7f09ac141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09b3200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09ac15af60 0x7f09ac1342b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09b3200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac15b580 con 0x7f09ac13fe50 2024-05-22T21:14:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09b3200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac15c3a0 con 0x7f09ac15af60 2024-05-22T21:14:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09b3200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac15b790 con 0x7f09ac13cfc0 2024-05-22T21:14:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09a3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac13fe50 0x7f09ac141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09a3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac13fe50 0x7f09ac141870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39796/0 (socket says 172.21.15.161:39796) 2024-05-22T21:14:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09a3400640 1 -- 172.21.15.161:0/1407657542 learned_addr learned my addr 172.21.15.161:0/1407657542 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09abe00640 1 --2- 172.21.15.161:0/1407657542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f09ac13cfc0 0x7f09ac138480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09b1200640 1 --2- 172.21.15.161:0/1407657542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09ac15af60 0x7f09ac1342b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09a3400640 1 -- 172.21.15.161:0/1407657542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f09ac13cfc0 msgr2=0x7f09ac138480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09a3400640 1 --2- 172.21.15.161:0/1407657542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f09ac13cfc0 0x7f09ac138480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09a3400640 1 -- 172.21.15.161:0/1407657542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09ac15af60 msgr2=0x7f09ac1342b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09a3400640 1 --2- 172.21.15.161:0/1407657542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09ac15af60 0x7f09ac1342b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09a3400640 1 -- 172.21.15.161:0/1407657542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c0094d0 con 0x7f09ac13fe50 2024-05-22T21:14:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09b1200640 1 --2- 172.21.15.161:0/1407657542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09ac15af60 0x7f09ac1342b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:13.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09a3400640 1 --2- 172.21.15.161:0/1407657542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac13fe50 0x7f09ac141870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f099400c3c0 tx=0x7f099400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:13.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09a8c00640 1 -- 172.21.15.161:0/1407657542 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09940052d0 con 0x7f09ac13fe50 2024-05-22T21:14:13.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09a8c00640 1 -- 172.21.15.161:0/1407657542 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0994005430 con 0x7f09ac13fe50 2024-05-22T21:14:13.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09a8c00640 1 -- 172.21.15.161:0/1407657542 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994003840 con 0x7f09ac13fe50 2024-05-22T21:14:13.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09b3200640 1 -- 172.21.15.161:0/1407657542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ac15ba70 con 0x7f09ac13fe50 2024-05-22T21:14:13.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.397+0000 7f09b3200640 1 -- 172.21.15.161:0/1407657542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ac15c030 con 0x7f09ac13fe50 2024-05-22T21:14:13.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.401+0000 7f09b3200640 1 -- 172.21.15.161:0/1407657542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ac15c550 con 0x7f09ac13fe50 2024-05-22T21:14:13.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.401+0000 7f09a8c00640 1 -- 172.21.15.161:0/1407657542 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f09940039a0 con 0x7f09ac13fe50 2024-05-22T21:14:13.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.401+0000 7f09a8c00640 1 --2- 172.21.15.161:0/1407657542 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f09880666c0 0x7f0988068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.401+0000 7f09a8c00640 1 -- 172.21.15.161:0/1407657542 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0994083450 con 0x7f09ac13fe50 2024-05-22T21:14:13.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.401+0000 7f09abe00640 1 --2- 172.21.15.161:0/1407657542 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f09880666c0 0x7f0988068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.401+0000 7f09abe00640 1 --2- 172.21.15.161:0/1407657542 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f09880666c0 0x7f0988068b80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f099c0ad000 tx=0x7f099c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:13.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.405+0000 7f09a8c00640 1 -- 172.21.15.161:0/1407657542 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f09ac15c550 con 0x7f09ac13fe50 2024-05-22T21:14:13.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.613+0000 7f09b3200640 1 -- 172.21.15.161:0/1407657542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f09ac15c510 con 0x7f09ac13fe50 2024-05-22T21:14:13.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.617+0000 7f09a8c00640 1 -- 172.21.15.161:0/1407657542 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f09940509e0 con 0x7f09ac13fe50 2024-05-22T21:14:13.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.621+0000 7f09b3200640 1 -- 172.21.15.161:0/1407657542 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f09880666c0 msgr2=0x7f0988068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.621+0000 7f09b3200640 1 --2- 172.21.15.161:0/1407657542 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f09880666c0 0x7f0988068b80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f099c0ad000 tx=0x7f099c008040 comp rx=0 tx=0).stop 2024-05-22T21:14:13.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.621+0000 7f09b3200640 1 -- 172.21.15.161:0/1407657542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac13fe50 msgr2=0x7f09ac141870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.621+0000 7f09b3200640 1 --2- 172.21.15.161:0/1407657542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac13fe50 0x7f09ac141870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f099400c3c0 tx=0x7f099400c880 comp rx=0 tx=0).stop 2024-05-22T21:14:13.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.621+0000 7f09b3200640 1 -- 172.21.15.161:0/1407657542 shutdown_connections 2024-05-22T21:14:13.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.621+0000 7f09b3200640 1 --2- 172.21.15.161:0/1407657542 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f09880666c0 0x7f0988068b80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.621+0000 7f09b3200640 1 --2- 172.21.15.161:0/1407657542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09ac15af60 0x7f09ac1342b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.621+0000 7f09b3200640 1 --2- 172.21.15.161:0/1407657542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac13fe50 0x7f09ac141870 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.621+0000 7f09b3200640 1 --2- 172.21.15.161:0/1407657542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f09ac13cfc0 0x7f09ac138480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.621+0000 7f09b3200640 1 -- 172.21.15.161:0/1407657542 >> 172.21.15.161:0/1407657542 conn(0x7f09ac06a2d0 msgr2=0x7f09ac06a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:13.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.621+0000 7f09b3200640 1 -- 172.21.15.161:0/1407657542 shutdown_connections 2024-05-22T21:14:13.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.621+0000 7f09b3200640 1 -- 172.21.15.161:0/1407657542 wait complete. 2024-05-22T21:14:13.646 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-22T21:14:13.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc9000640 1 Processor -- start 2024-05-22T21:14:13.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc9000640 1 -- start start 2024-05-22T21:14:13.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc9000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fccc406cb30 0x7fccc406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc9000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fccc40b2100 0x7fccc40b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc9000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc415d250 con 0x7fccc41596b0 2024-05-22T21:14:13.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc9000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc406d500 con 0x7fccc406cb30 2024-05-22T21:14:13.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc9000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc4159a90 con 0x7fccc40b2100 2024-05-22T21:14:13.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc1e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fccc41596b0 0x7fccc4071870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59264/0 (socket says 172.21.15.161:59264) 2024-05-22T21:14:13.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc1e00640 1 -- 172.21.15.161:0/652826230 learned_addr learned my addr 172.21.15.161:0/652826230 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:13.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccb9400640 1 --2- 172.21.15.161:0/652826230 >> v2:172.21.15.179:3300/0 conn(0x7fccc406cb30 0x7fccc406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc2800640 1 --2- 172.21.15.161:0/652826230 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fccc40b2100 0x7fccc40b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc1400640 1 -- 172.21.15.161:0/652826230 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4132405469 0 0) 0x7fccc415d250 con 0x7fccc41596b0 2024-05-22T21:14:13.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc1400640 1 -- 172.21.15.161:0/652826230 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcca8003530 con 0x7fccc41596b0 2024-05-22T21:14:13.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc1400640 1 -- 172.21.15.161:0/652826230 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2872059439 0 0) 0x7fcca8003530 con 0x7fccc41596b0 2024-05-22T21:14:13.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc1400640 1 -- 172.21.15.161:0/652826230 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fccc40b2100 msgr2=0x7fccc40b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:13.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc1400640 1 --2- 172.21.15.161:0/652826230 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fccc40b2100 0x7fccc40b2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc1400640 1 -- 172.21.15.161:0/652826230 >> v2:172.21.15.179:3300/0 conn(0x7fccc406cb30 msgr2=0x7fccc406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:13.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc1400640 1 --2- 172.21.15.161:0/652826230 >> v2:172.21.15.179:3300/0 conn(0x7fccc406cb30 0x7fccc406cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc1400640 1 -- 172.21.15.161:0/652826230 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc415d510 con 0x7fccc41596b0 2024-05-22T21:14:13.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc1400640 1 -- 172.21.15.161:0/652826230 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fccb4001680 con 0x7fccc41596b0 2024-05-22T21:14:13.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc1400640 1 -- 172.21.15.161:0/652826230 >> v1:172.21.15.161:6789/0 conn(0x7fccc41596b0 legacy=0x7fccc4071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.777+0000 7fccc1400640 1 --2- 172.21.15.161:0/652826230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcca80039b0 0x7fcca8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.781+0000 7fccc1400640 1 --2- 172.21.15.161:0/652826230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca80047d0 0x7fcca8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.781+0000 7fccc1400640 1 --2- 172.21.15.161:0/652826230 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fccc40b2100 0x7fcca8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.781+0000 7fccc1400640 1 -- 172.21.15.161:0/652826230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc406d500 con 0x7fcca80047d0 2024-05-22T21:14:13.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.781+0000 7fccc1400640 1 -- 172.21.15.161:0/652826230 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc4159a90 con 0x7fcca80039b0 2024-05-22T21:14:13.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.781+0000 7fccc1400640 1 -- 172.21.15.161:0/652826230 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca800b990 con 0x7fccc40b2100 2024-05-22T21:14:13.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.781+0000 7fccc1e00640 1 --2- 172.21.15.161:0/652826230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcca80039b0 0x7fcca8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.781+0000 7fccb9400640 1 --2- 172.21.15.161:0/652826230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca80047d0 0x7fcca8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.781+0000 7fccc2800640 1 --2- 172.21.15.161:0/652826230 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fccc40b2100 0x7fcca8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.781+0000 7fccc1e00640 1 -- 172.21.15.161:0/652826230 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fccc40b2100 msgr2=0x7fcca8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:13.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.781+0000 7fccc1e00640 1 --2- 172.21.15.161:0/652826230 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fccc40b2100 0x7fcca8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.781+0000 7fccc1e00640 1 -- 172.21.15.161:0/652826230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca80047d0 msgr2=0x7fcca8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:13.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.781+0000 7fccc1e00640 1 --2- 172.21.15.161:0/652826230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca80047d0 0x7fcca8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.781+0000 7fccc1e00640 1 -- 172.21.15.161:0/652826230 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca8003530 con 0x7fcca80039b0 2024-05-22T21:14:13.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.781+0000 7fccc1e00640 1 --2- 172.21.15.161:0/652826230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcca80039b0 0x7fcca8003dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fccb400c240 tx=0x7fccb401f3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=343f7f7a18970028 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:13.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc1400640 1 -- 172.21.15.161:0/652826230 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb40058e0 con 0x7fcca80039b0 2024-05-22T21:14:13.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc1400640 1 -- 172.21.15.161:0/652826230 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb4005a60 con 0x7fcca80039b0 2024-05-22T21:14:13.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc1400640 1 -- 172.21.15.161:0/652826230 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb4005e10 con 0x7fcca80039b0 2024-05-22T21:14:13.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc9000640 1 -- 172.21.15.161:0/652826230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcca80039b0 msgr2=0x7fcca8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc9000640 1 --2- 172.21.15.161:0/652826230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcca80039b0 0x7fcca8003dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fccb400c240 tx=0x7fccb401f3d0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc2800640 1 -- 172.21.15.161:0/652826230 reap_dead start 2024-05-22T21:14:13.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc9000640 1 -- 172.21.15.161:0/652826230 shutdown_connections 2024-05-22T21:14:13.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc9000640 1 -- 172.21.15.161:0/652826230 >> 172.21.15.161:0/652826230 conn(0x7fccc406a2d0 msgr2=0x7fccc406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:13.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc9000640 1 -- 172.21.15.161:0/652826230 shutdown_connections 2024-05-22T21:14:13.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc9000640 1 -- 172.21.15.161:0/652826230 wait complete. 2024-05-22T21:14:13.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc9000640 1 Processor -- start 2024-05-22T21:14:13.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc9000640 1 -- start start 2024-05-22T21:14:13.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc9000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca80039b0 0x7fccc41469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc9000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc4159ee0 0x7fccc415adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc9000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fccc414a830 0x7fccc415bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc9000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc415b670 con 0x7fcca80039b0 2024-05-22T21:14:13.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc9000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc4149350 con 0x7fccc4159ee0 2024-05-22T21:14:13.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc9000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc4146f30 con 0x7fccc414a830 2024-05-22T21:14:13.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca80039b0 0x7fccc41469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca80039b0 0x7fccc41469f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39810/0 (socket says 172.21.15.161:39810) 2024-05-22T21:14:13.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc1e00640 1 -- 172.21.15.161:0/2744579365 learned_addr learned my addr 172.21.15.161:0/2744579365 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:13.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccc2800640 1 --2- 172.21.15.161:0/2744579365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fccc414a830 0x7fccc415bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.785+0000 7fccb9400640 1 --2- 172.21.15.161:0/2744579365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc4159ee0 0x7fccc415adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccb9400640 1 -- 172.21.15.161:0/2744579365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fccc414a830 msgr2=0x7fccc415bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccb9400640 1 --2- 172.21.15.161:0/2744579365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fccc414a830 0x7fccc415bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccb9400640 1 -- 172.21.15.161:0/2744579365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca80039b0 msgr2=0x7fccc41469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:13.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccb9400640 1 --2- 172.21.15.161:0/2744579365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca80039b0 0x7fccc41469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:13.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccb9400640 1 -- 172.21.15.161:0/2744579365 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc4156e50 con 0x7fccc4159ee0 2024-05-22T21:14:13.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccc1e00640 1 --2- 172.21.15.161:0/2744579365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca80039b0 0x7fccc41469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:13.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccb9400640 1 --2- 172.21.15.161:0/2744579365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc4159ee0 0x7fccc415adf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fccac011990 tx=0x7fccac011e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:13.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccbaa00640 1 -- 172.21.15.161:0/2744579365 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac00f9b0 con 0x7fccc4159ee0 2024-05-22T21:14:13.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccbaa00640 1 -- 172.21.15.161:0/2744579365 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccac00fb10 con 0x7fccc4159ee0 2024-05-22T21:14:13.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccc9000640 1 -- 172.21.15.161:0/2744579365 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc4157130 con 0x7fccc4159ee0 2024-05-22T21:14:13.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccbaa00640 1 -- 172.21.15.161:0/2744579365 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac00fea0 con 0x7fccc4159ee0 2024-05-22T21:14:13.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccc9000640 1 -- 172.21.15.161:0/2744579365 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc41576f0 con 0x7fccc4159ee0 2024-05-22T21:14:13.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccc9000640 1 -- 172.21.15.161:0/2744579365 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc4157910 con 0x7fccc4159ee0 2024-05-22T21:14:13.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccbaa00640 1 -- 172.21.15.161:0/2744579365 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fccac014440 con 0x7fccc4159ee0 2024-05-22T21:14:13.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccbaa00640 1 --2- 172.21.15.161:0/2744579365 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcca0066690 0x7fcca0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:13.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccbaa00640 1 -- 172.21.15.161:0/2744579365 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fccac0843b0 con 0x7fccc4159ee0 2024-05-22T21:14:13.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.789+0000 7fccc1e00640 1 --2- 172.21.15.161:0/2744579365 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcca0066690 0x7fcca0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:13.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.793+0000 7fccc1e00640 1 --2- 172.21.15.161:0/2744579365 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcca0066690 0x7fcca0068b50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fccb401f410 tx=0x7fccb40aa070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:13.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:13.797+0000 7fccbaa00640 1 -- 172.21.15.161:0/2744579365 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fccac051a80 con 0x7fccc4159ee0 2024-05-22T21:14:14.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.001+0000 7fccc9000640 1 -- 172.21.15.161:0/2744579365 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fccc4067220 con 0x7fccc4159ee0 2024-05-22T21:14:14.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.005+0000 7fccbaa00640 1 -- 172.21.15.161:0/2744579365 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fccac0550a0 con 0x7fccc4159ee0 2024-05-22T21:14:14.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.009+0000 7fccc9000640 1 -- 172.21.15.161:0/2744579365 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcca0066690 msgr2=0x7fcca0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.009+0000 7fccc9000640 1 --2- 172.21.15.161:0/2744579365 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcca0066690 0x7fcca0068b50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fccb401f410 tx=0x7fccb40aa070 comp rx=0 tx=0).stop 2024-05-22T21:14:14.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.009+0000 7fccc9000640 1 -- 172.21.15.161:0/2744579365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc4159ee0 msgr2=0x7fccc415adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.009+0000 7fccc9000640 1 --2- 172.21.15.161:0/2744579365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc4159ee0 0x7fccc415adf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fccac011990 tx=0x7fccac011e50 comp rx=0 tx=0).stop 2024-05-22T21:14:14.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.009+0000 7fccc9000640 1 -- 172.21.15.161:0/2744579365 shutdown_connections 2024-05-22T21:14:14.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.009+0000 7fccc9000640 1 --2- 172.21.15.161:0/2744579365 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcca0066690 0x7fcca0068b50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.009+0000 7fccc9000640 1 --2- 172.21.15.161:0/2744579365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fccc414a830 0x7fccc415bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.009+0000 7fccc9000640 1 --2- 172.21.15.161:0/2744579365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fccc4159ee0 0x7fccc415adf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.009+0000 7fccc9000640 1 --2- 172.21.15.161:0/2744579365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcca80039b0 0x7fccc41469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.009+0000 7fccc9000640 1 -- 172.21.15.161:0/2744579365 >> 172.21.15.161:0/2744579365 conn(0x7fccc406a2d0 msgr2=0x7fccc4155b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:14.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.009+0000 7fccc9000640 1 -- 172.21.15.161:0/2744579365 shutdown_connections 2024-05-22T21:14:14.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.009+0000 7fccc9000640 1 -- 172.21.15.161:0/2744579365 wait complete. 2024-05-22T21:14:14.033 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-22T21:14:14.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa86b200640 1 Processor -- start 2024-05-22T21:14:14.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa86b200640 1 -- start start 2024-05-22T21:14:14.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa86b200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa86c15d190 0x7fa86c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa86b200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa86c0b3450 0x7fa86c06b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa86b200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c0cfa40 con 0x7fa86c06b970 2024-05-22T21:14:14.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa86b200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86c06b1a0 con 0x7fa86c0b3450 2024-05-22T21:14:14.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa86b200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c06be00 con 0x7fa86c15d190 2024-05-22T21:14:14.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa863e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa86c0b3450 0x7fa86c06b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa86a800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa86c06b970 0x7fa86c160d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59274/0 (socket says 172.21.15.161:59274) 2024-05-22T21:14:14.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa863e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa86c0b3450 0x7fa86c06b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39822/0 (socket says 172.21.15.161:39822) 2024-05-22T21:14:14.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa86a800640 1 -- 172.21.15.161:0/3096914777 learned_addr learned my addr 172.21.15.161:0/3096914777 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:14.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa869e00640 1 --2- 172.21.15.161:0/3096914777 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa86c15d190 0x7fa86c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa863e00640 1 -- 172.21.15.161:0/3096914777 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa86c15d190 msgr2=0x7fa86c0b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:14.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa863e00640 1 --2- 172.21.15.161:0/3096914777 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa86c15d190 0x7fa86c0b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa863e00640 1 -- 172.21.15.161:0/3096914777 >> v1:172.21.15.161:6789/0 conn(0x7fa86c06b970 legacy=0x7fa86c160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa863e00640 1 -- 172.21.15.161:0/3096914777 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c0cfce0 con 0x7fa86c0b3450 2024-05-22T21:14:14.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.165+0000 7fa869400640 1 -- 172.21.15.161:0/3096914777 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2324072310 0 0) 0x7fa86c0cfa40 con 0x7fa86c06b970 2024-05-22T21:14:14.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.169+0000 7fa863e00640 1 --2- 172.21.15.161:0/3096914777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa86c0b3450 0x7fa86c06b430 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa8540097c0 tx=0x7fa85401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf2a6c7337f6e43 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:14.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.169+0000 7fa869400640 1 -- 172.21.15.161:0/3096914777 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8540ac410 con 0x7fa86c0b3450 2024-05-22T21:14:14.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.169+0000 7fa869400640 1 -- 172.21.15.161:0/3096914777 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8540ac570 con 0x7fa86c0b3450 2024-05-22T21:14:14.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.169+0000 7fa869400640 1 -- 172.21.15.161:0/3096914777 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8540ac900 con 0x7fa86c0b3450 2024-05-22T21:14:14.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.169+0000 7fa86b200640 1 -- 172.21.15.161:0/3096914777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa86c0b3450 msgr2=0x7fa86c06b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.169+0000 7fa86b200640 1 --2- 172.21.15.161:0/3096914777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa86c0b3450 0x7fa86c06b430 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa8540097c0 tx=0x7fa85401c4a0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.169+0000 7fa86b200640 1 -- 172.21.15.161:0/3096914777 shutdown_connections 2024-05-22T21:14:14.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.169+0000 7fa86b200640 1 --2- 172.21.15.161:0/3096914777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa86c0b3450 0x7fa86c06b430 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.169+0000 7fa86b200640 1 --2- 172.21.15.161:0/3096914777 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa86c15d190 0x7fa86c0b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.169+0000 7fa86b200640 1 -- 172.21.15.161:0/3096914777 >> 172.21.15.161:0/3096914777 conn(0x7fa86c0b2930 msgr2=0x7fa86c0b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:14.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.169+0000 7fa86b200640 1 -- 172.21.15.161:0/3096914777 shutdown_connections 2024-05-22T21:14:14.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.169+0000 7fa86b200640 1 -- 172.21.15.161:0/3096914777 wait complete. 2024-05-22T21:14:14.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86b200640 1 Processor -- start 2024-05-22T21:14:14.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86b200640 1 -- start start 2024-05-22T21:14:14.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86b200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa86c06b970 0x7fa86c0c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86b200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa86c0b3450 0x7fa86c15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa86c15d190 0x7fa86c15ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86b200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0c52e0 con 0x7fa86c15d190 2024-05-22T21:14:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86b200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0d1b60 con 0x7fa86c06b970 2024-05-22T21:14:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86b200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0c49b0 con 0x7fa86c0b3450 2024-05-22T21:14:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa86c15d190 0x7fa86c15ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa86c15d190 0x7fa86c15ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39838/0 (socket says 172.21.15.161:39838) 2024-05-22T21:14:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86a800640 1 -- 172.21.15.161:0/1725768480 learned_addr learned my addr 172.21.15.161:0/1725768480 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa869e00640 1 --2- 172.21.15.161:0/1725768480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa86c06b970 0x7fa86c0c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa863e00640 1 --2- 172.21.15.161:0/1725768480 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa86c0b3450 0x7fa86c15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86a800640 1 -- 172.21.15.161:0/1725768480 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa86c0b3450 msgr2=0x7fa86c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86a800640 1 --2- 172.21.15.161:0/1725768480 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa86c0b3450 0x7fa86c15e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86a800640 1 -- 172.21.15.161:0/1725768480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa86c06b970 msgr2=0x7fa86c0c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86a800640 1 --2- 172.21.15.161:0/1725768480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa86c06b970 0x7fa86c0c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86a800640 1 -- 172.21.15.161:0/1725768480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa854009420 con 0x7fa86c15d190 2024-05-22T21:14:14.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86a800640 1 --2- 172.21.15.161:0/1725768480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa86c15d190 0x7fa86c15ede0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa86400c780 tx=0x7fa86400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:14.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa862000640 1 -- 172.21.15.161:0/1725768480 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86400ce90 con 0x7fa86c15d190 2024-05-22T21:14:14.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa862000640 1 -- 172.21.15.161:0/1725768480 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86400a640 con 0x7fa86c15d190 2024-05-22T21:14:14.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa862000640 1 -- 172.21.15.161:0/1725768480 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86400a9d0 con 0x7fa86c15d190 2024-05-22T21:14:14.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86b200640 1 -- 172.21.15.161:0/1725768480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c0c4c90 con 0x7fa86c15d190 2024-05-22T21:14:14.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86b200640 1 -- 172.21.15.161:0/1725768480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c154f90 con 0x7fa86c15d190 2024-05-22T21:14:14.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.173+0000 7fa86b200640 1 -- 172.21.15.161:0/1725768480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86c0c5450 con 0x7fa86c15d190 2024-05-22T21:14:14.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.177+0000 7fa862000640 1 -- 172.21.15.161:0/1725768480 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa86400ab30 con 0x7fa86c15d190 2024-05-22T21:14:14.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.177+0000 7fa862000640 1 --2- 172.21.15.161:0/1725768480 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8480666c0 0x7fa848068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.177+0000 7fa862000640 1 -- 172.21.15.161:0/1725768480 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa864083290 con 0x7fa86c15d190 2024-05-22T21:14:14.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.177+0000 7fa869e00640 1 --2- 172.21.15.161:0/1725768480 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8480666c0 0x7fa848068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.177+0000 7fa869e00640 1 --2- 172.21.15.161:0/1725768480 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8480666c0 0x7fa848068b80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00b9d0 tx=0x7fa85c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:14.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.181+0000 7fa862000640 1 -- 172.21.15.161:0/1725768480 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa8640508b0 con 0x7fa86c15d190 2024-05-22T21:14:14.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.397+0000 7fa86b200640 1 -- 172.21.15.161:0/1725768480 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fa86c0676a0 con 0x7fa86c15d190 2024-05-22T21:14:14.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.401+0000 7fa862000640 1 -- 172.21.15.161:0/1725768480 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa864053ed0 con 0x7fa86c15d190 2024-05-22T21:14:14.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.405+0000 7fa86b200640 1 -- 172.21.15.161:0/1725768480 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8480666c0 msgr2=0x7fa848068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.405+0000 7fa86b200640 1 --2- 172.21.15.161:0/1725768480 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8480666c0 0x7fa848068b80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00b9d0 tx=0x7fa85c008040 comp rx=0 tx=0).stop 2024-05-22T21:14:14.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.405+0000 7fa86b200640 1 -- 172.21.15.161:0/1725768480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa86c15d190 msgr2=0x7fa86c15ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.405+0000 7fa86b200640 1 --2- 172.21.15.161:0/1725768480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa86c15d190 0x7fa86c15ede0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa86400c780 tx=0x7fa86400cc40 comp rx=0 tx=0).stop 2024-05-22T21:14:14.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.405+0000 7fa86b200640 1 -- 172.21.15.161:0/1725768480 shutdown_connections 2024-05-22T21:14:14.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.405+0000 7fa86b200640 1 --2- 172.21.15.161:0/1725768480 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8480666c0 0x7fa848068b80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.405+0000 7fa86b200640 1 --2- 172.21.15.161:0/1725768480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa86c15d190 0x7fa86c15ede0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.405+0000 7fa86b200640 1 --2- 172.21.15.161:0/1725768480 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa86c0b3450 0x7fa86c15e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.405+0000 7fa86b200640 1 --2- 172.21.15.161:0/1725768480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa86c06b970 0x7fa86c0c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.405+0000 7fa86b200640 1 -- 172.21.15.161:0/1725768480 >> 172.21.15.161:0/1725768480 conn(0x7fa86c0b2930 msgr2=0x7fa86c0b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:14.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.405+0000 7fa86b200640 1 -- 172.21.15.161:0/1725768480 shutdown_connections 2024-05-22T21:14:14.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.405+0000 7fa86b200640 1 -- 172.21.15.161:0/1725768480 wait complete. 2024-05-22T21:14:14.429 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-22T21:14:14.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc260c00640 1 Processor -- start 2024-05-22T21:14:14.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc260c00640 1 -- start start 2024-05-22T21:14:14.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc260c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc25c0d0b20 0x7fc25c0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc260c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc25c06b4a0 0x7fc25c0d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc260c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc25c160f10 0x7fc25c1613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc260c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25c06ca20 con 0x7fc25c06b4a0 2024-05-22T21:14:14.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc260c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25c06b980 con 0x7fc25c160f10 2024-05-22T21:14:14.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc260c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c06bc60 con 0x7fc25c0d0b20 2024-05-22T21:14:14.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc25aa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc25c06b4a0 0x7fc25c0d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc25aa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc25c06b4a0 0x7fc25c0d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39848/0 (socket says 172.21.15.161:39848) 2024-05-22T21:14:14.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc25aa00640 1 -- 172.21.15.161:0/2507235399 learned_addr learned my addr 172.21.15.161:0/2507235399 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:14.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc25be00640 1 --2- 172.21.15.161:0/2507235399 >> v2:172.21.15.179:3300/0 conn(0x7fc25c160f10 0x7fc25c1613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc25b400640 1 --2- 172.21.15.161:0/2507235399 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc25c0d0b20 0x7fc25c0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc25aa00640 1 -- 172.21.15.161:0/2507235399 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc25c0d0b20 msgr2=0x7fc25c0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:14.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc25aa00640 1 --2- 172.21.15.161:0/2507235399 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc25c0d0b20 0x7fc25c0ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc25aa00640 1 -- 172.21.15.161:0/2507235399 >> v2:172.21.15.179:3300/0 conn(0x7fc25c160f10 msgr2=0x7fc25c1613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:14.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc25aa00640 1 --2- 172.21.15.161:0/2507235399 >> v2:172.21.15.179:3300/0 conn(0x7fc25c160f10 0x7fc25c1613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc25aa00640 1 -- 172.21.15.161:0/2507235399 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c06bf40 con 0x7fc25c06b4a0 2024-05-22T21:14:14.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.561+0000 7fc25b400640 1 --2- 172.21.15.161:0/2507235399 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc25c0d0b20 0x7fc25c0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:14.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.565+0000 7fc25aa00640 1 --2- 172.21.15.161:0/2507235399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25c06b4a0 0x7fc25c0d0f00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc25c09a6e0 tx=0x7fc24801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf33d97bcd4a2329 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:14.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.565+0000 7fc25a000640 1 -- 172.21.15.161:0/2507235399 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2480aa430 con 0x7fc25c06b4a0 2024-05-22T21:14:14.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.565+0000 7fc25a000640 1 -- 172.21.15.161:0/2507235399 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2480aa590 con 0x7fc25c06b4a0 2024-05-22T21:14:14.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.565+0000 7fc25a000640 1 -- 172.21.15.161:0/2507235399 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2480aa920 con 0x7fc25c06b4a0 2024-05-22T21:14:14.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.565+0000 7fc260c00640 1 -- 172.21.15.161:0/2507235399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25c06b4a0 msgr2=0x7fc25c0d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.565+0000 7fc260c00640 1 --2- 172.21.15.161:0/2507235399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25c06b4a0 0x7fc25c0d0f00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc25c09a6e0 tx=0x7fc24801c4f0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.565+0000 7fc260c00640 1 -- 172.21.15.161:0/2507235399 shutdown_connections 2024-05-22T21:14:14.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.565+0000 7fc260c00640 1 --2- 172.21.15.161:0/2507235399 >> v2:172.21.15.179:3300/0 conn(0x7fc25c160f10 0x7fc25c1613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.565+0000 7fc260c00640 1 --2- 172.21.15.161:0/2507235399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25c06b4a0 0x7fc25c0d0f00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.565+0000 7fc260c00640 1 --2- 172.21.15.161:0/2507235399 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc25c0d0b20 0x7fc25c0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.565+0000 7fc260c00640 1 -- 172.21.15.161:0/2507235399 >> 172.21.15.161:0/2507235399 conn(0x7fc25c099a40 msgr2=0x7fc25c099e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:14.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.565+0000 7fc260c00640 1 -- 172.21.15.161:0/2507235399 shutdown_connections 2024-05-22T21:14:14.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.565+0000 7fc260c00640 1 -- 172.21.15.161:0/2507235399 wait complete. 2024-05-22T21:14:14.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc260c00640 1 Processor -- start 2024-05-22T21:14:14.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc260c00640 1 -- start start 2024-05-22T21:14:14.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc260c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc25c06b4a0 0x7fc25c156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc260c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc25c0d0b20 0x7fc25c1572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc260c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25c160f10 0x7fc25c152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc260c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c153cd0 con 0x7fc25c160f10 2024-05-22T21:14:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc260c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c0ca3c0 con 0x7fc25c06b4a0 2024-05-22T21:14:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc260c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c159ae0 con 0x7fc25c0d0b20 2024-05-22T21:14:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc25be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25c160f10 0x7fc25c152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc25be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25c160f10 0x7fc25c152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39862/0 (socket says 172.21.15.161:39862) 2024-05-22T21:14:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc25be00640 1 -- 172.21.15.161:0/2605695947 learned_addr learned my addr 172.21.15.161:0/2605695947 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc25b400640 1 --2- 172.21.15.161:0/2605695947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc25c06b4a0 0x7fc25c156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc25aa00640 1 --2- 172.21.15.161:0/2605695947 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc25c0d0b20 0x7fc25c1572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc25be00640 1 -- 172.21.15.161:0/2605695947 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc25c0d0b20 msgr2=0x7fc25c1572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc25be00640 1 --2- 172.21.15.161:0/2605695947 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc25c0d0b20 0x7fc25c1572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc25be00640 1 -- 172.21.15.161:0/2605695947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc25c06b4a0 msgr2=0x7fc25c156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc25be00640 1 --2- 172.21.15.161:0/2605695947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc25c06b4a0 0x7fc25c156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc25be00640 1 -- 172.21.15.161:0/2605695947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2480094d0 con 0x7fc25c160f10 2024-05-22T21:14:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc25be00640 1 --2- 172.21.15.161:0/2605695947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25c160f10 0x7fc25c152ce0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc25400d560 tx=0x7fc25400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc253400640 1 -- 172.21.15.161:0/2605695947 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2540125f0 con 0x7fc25c160f10 2024-05-22T21:14:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc253400640 1 -- 172.21.15.161:0/2605695947 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc254012770 con 0x7fc25c160f10 2024-05-22T21:14:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc260c00640 1 -- 172.21.15.161:0/2605695947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25c0ca890 con 0x7fc25c160f10 2024-05-22T21:14:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc253400640 1 -- 172.21.15.161:0/2605695947 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254012b20 con 0x7fc25c160f10 2024-05-22T21:14:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc260c00640 1 -- 172.21.15.161:0/2605695947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25c0cae50 con 0x7fc25c160f10 2024-05-22T21:14:14.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.569+0000 7fc260c00640 1 -- 172.21.15.161:0/2605695947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25c153e10 con 0x7fc25c160f10 2024-05-22T21:14:14.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.573+0000 7fc253400640 1 -- 172.21.15.161:0/2605695947 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc254012c80 con 0x7fc25c160f10 2024-05-22T21:14:14.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.573+0000 7fc253400640 1 --2- 172.21.15.161:0/2605695947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc23c066690 0x7fc23c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.573+0000 7fc253400640 1 -- 172.21.15.161:0/2605695947 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc2540841b0 con 0x7fc25c160f10 2024-05-22T21:14:14.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.573+0000 7fc25b400640 1 --2- 172.21.15.161:0/2605695947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc23c066690 0x7fc23c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.573+0000 7fc25b400640 1 --2- 172.21.15.161:0/2605695947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc23c066690 0x7fc23c068b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc24c007130 tx=0x7fc24c0077c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:14.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.577+0000 7fc253400640 1 -- 172.21.15.161:0/2605695947 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc25c153e10 con 0x7fc25c160f10 2024-05-22T21:14:14.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.785+0000 7fc260c00640 1 -- 172.21.15.161:0/2605695947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fc25c153e10 con 0x7fc25c160f10 2024-05-22T21:14:14.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.793+0000 7fc253400640 1 -- 172.21.15.161:0/2605695947 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc25c153e10 con 0x7fc25c160f10 2024-05-22T21:14:14.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.793+0000 7fc260c00640 1 -- 172.21.15.161:0/2605695947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc23c066690 msgr2=0x7fc23c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.793+0000 7fc260c00640 1 --2- 172.21.15.161:0/2605695947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc23c066690 0x7fc23c068b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc24c007130 tx=0x7fc24c0077c0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.793+0000 7fc260c00640 1 -- 172.21.15.161:0/2605695947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25c160f10 msgr2=0x7fc25c152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.793+0000 7fc260c00640 1 --2- 172.21.15.161:0/2605695947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25c160f10 0x7fc25c152ce0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc25400d560 tx=0x7fc25400da20 comp rx=0 tx=0).stop 2024-05-22T21:14:14.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.793+0000 7fc260c00640 1 -- 172.21.15.161:0/2605695947 shutdown_connections 2024-05-22T21:14:14.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.793+0000 7fc260c00640 1 --2- 172.21.15.161:0/2605695947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc23c066690 0x7fc23c068b50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.793+0000 7fc260c00640 1 --2- 172.21.15.161:0/2605695947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25c160f10 0x7fc25c152ce0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.793+0000 7fc260c00640 1 --2- 172.21.15.161:0/2605695947 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc25c0d0b20 0x7fc25c1572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.793+0000 7fc260c00640 1 --2- 172.21.15.161:0/2605695947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc25c06b4a0 0x7fc25c156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.793+0000 7fc260c00640 1 -- 172.21.15.161:0/2605695947 >> 172.21.15.161:0/2605695947 conn(0x7fc25c099a40 msgr2=0x7fc25c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:14.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.793+0000 7fc260c00640 1 -- 172.21.15.161:0/2605695947 shutdown_connections 2024-05-22T21:14:14.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.793+0000 7fc260c00640 1 -- 172.21.15.161:0/2605695947 wait complete. 2024-05-22T21:14:14.815 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-22T21:14:14.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.949+0000 7ffb9d800640 1 Processor -- start 2024-05-22T21:14:14.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.949+0000 7ffb9d800640 1 -- start start 2024-05-22T21:14:14.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.953+0000 7ffb9d800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb9806cfc0 0x7ffb980b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.953+0000 7ffb9d800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9815d250 con 0x7ffb9806cb30 2024-05-22T21:14:14.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.953+0000 7ffb9d800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb98159a90 con 0x7ffb981596b0 2024-05-22T21:14:14.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.953+0000 7ffb9d800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9815d510 con 0x7ffb9806cfc0 2024-05-22T21:14:14.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.953+0000 7ffb96000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ffb9806cb30 0x7ffb980b2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59280/0 (socket says 172.21.15.161:59280) 2024-05-22T21:14:14.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.953+0000 7ffb96000640 1 -- 172.21.15.161:0/1618454500 learned_addr learned my addr 172.21.15.161:0/1618454500 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:14.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.953+0000 7ffb97400640 1 --2- 172.21.15.161:0/1618454500 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb9806cfc0 0x7ffb980b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.953+0000 7ffb95600640 1 -- 172.21.15.161:0/1618454500 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2711104894 0 0) 0x7ffb9815d250 con 0x7ffb9806cb30 2024-05-22T21:14:14.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.953+0000 7ffb95600640 1 -- 172.21.15.161:0/1618454500 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb7c003450 con 0x7ffb9806cb30 2024-05-22T21:14:14.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.953+0000 7ffb95600640 1 -- 172.21.15.161:0/1618454500 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040721806 0 0) 0x7ffb98159a90 con 0x7ffb981596b0 2024-05-22T21:14:14.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.953+0000 7ffb95600640 1 -- 172.21.15.161:0/1618454500 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb9815d250 con 0x7ffb981596b0 2024-05-22T21:14:14.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.953+0000 7ffb97400640 1 -- 172.21.15.161:0/1618454500 >> v1:172.21.15.179:6789/0 conn(0x7ffb981596b0 legacy=0x7ffb98071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.953+0000 7ffb97400640 1 -- 172.21.15.161:0/1618454500 >> v1:172.21.15.161:6789/0 conn(0x7ffb9806cb30 legacy=0x7ffb980b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.953+0000 7ffb97400640 1 -- 172.21.15.161:0/1618454500 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9815d7f0 con 0x7ffb9806cfc0 2024-05-22T21:14:14.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb97400640 1 --2- 172.21.15.161:0/1618454500 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb9806cfc0 0x7ffb980b2810 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009dc0 tx=0x7ffb9001c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=217b0deba5b3f9d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:14.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb95600640 1 -- 172.21.15.161:0/1618454500 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb900a58e0 con 0x7ffb9806cfc0 2024-05-22T21:14:14.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb95600640 1 -- 172.21.15.161:0/1618454500 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb900a5a40 con 0x7ffb9806cfc0 2024-05-22T21:14:14.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb95600640 1 -- 172.21.15.161:0/1618454500 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb900a5dd0 con 0x7ffb9806cfc0 2024-05-22T21:14:14.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb9d800640 1 -- 172.21.15.161:0/1618454500 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb9806cfc0 msgr2=0x7ffb980b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb9d800640 1 --2- 172.21.15.161:0/1618454500 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb9806cfc0 0x7ffb980b2810 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009dc0 tx=0x7ffb9001c990 comp rx=0 tx=0).stop 2024-05-22T21:14:14.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb9d800640 1 -- 172.21.15.161:0/1618454500 shutdown_connections 2024-05-22T21:14:14.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb9d800640 1 --2- 172.21.15.161:0/1618454500 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb9806cfc0 0x7ffb980b2810 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb9d800640 1 -- 172.21.15.161:0/1618454500 >> 172.21.15.161:0/1618454500 conn(0x7ffb9806a2d0 msgr2=0x7ffb9806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:14.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb9d800640 1 -- 172.21.15.161:0/1618454500 shutdown_connections 2024-05-22T21:14:14.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb9d800640 1 -- 172.21.15.161:0/1618454500 wait complete. 2024-05-22T21:14:14.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb9d800640 1 Processor -- start 2024-05-22T21:14:14.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb9d800640 1 -- start start 2024-05-22T21:14:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb9d800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb9806cb30 0x7ffb9815ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb9d800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9806cfc0 0x7ffb98146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb9d800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb981596b0 0x7ffb981f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb9d800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9815b380 con 0x7ffb9806cfc0 2024-05-22T21:14:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb9d800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9815b660 con 0x7ffb981596b0 2024-05-22T21:14:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb9d800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9815b940 con 0x7ffb9806cb30 2024-05-22T21:14:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb96000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9806cfc0 0x7ffb98146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb96000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9806cfc0 0x7ffb98146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39870/0 (socket says 172.21.15.161:39870) 2024-05-22T21:14:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.957+0000 7ffb96000640 1 -- 172.21.15.161:0/755972007 learned_addr learned my addr 172.21.15.161:0/755972007 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb97400640 1 --2- 172.21.15.161:0/755972007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb981596b0 0x7ffb981f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb96a00640 1 --2- 172.21.15.161:0/755972007 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb9806cb30 0x7ffb9815ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb96000640 1 -- 172.21.15.161:0/755972007 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb9806cb30 msgr2=0x7ffb9815ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb96000640 1 --2- 172.21.15.161:0/755972007 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb9806cb30 0x7ffb9815ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb96000640 1 -- 172.21.15.161:0/755972007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb981596b0 msgr2=0x7ffb981f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb96000640 1 --2- 172.21.15.161:0/755972007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb981596b0 0x7ffb981f6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb96000640 1 -- 172.21.15.161:0/755972007 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb90009a70 con 0x7ffb9806cfc0 2024-05-22T21:14:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb96000640 1 --2- 172.21.15.161:0/755972007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9806cfc0 0x7ffb98146d80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400c4d0 tx=0x7ffb8400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb8ea00640 1 -- 172.21.15.161:0/755972007 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb84005cf0 con 0x7ffb9806cfc0 2024-05-22T21:14:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb8ea00640 1 -- 172.21.15.161:0/755972007 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb84005e50 con 0x7ffb9806cfc0 2024-05-22T21:14:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb9d800640 1 -- 172.21.15.161:0/755972007 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9815bc20 con 0x7ffb9806cfc0 2024-05-22T21:14:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb8ea00640 1 -- 172.21.15.161:0/755972007 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb840072b0 con 0x7ffb9806cfc0 2024-05-22T21:14:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb9d800640 1 -- 172.21.15.161:0/755972007 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb981472c0 con 0x7ffb9806cfc0 2024-05-22T21:14:14.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb9d800640 1 -- 172.21.15.161:0/755972007 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb98067220 con 0x7ffb9806cfc0 2024-05-22T21:14:14.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb8ea00640 1 -- 172.21.15.161:0/755972007 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffb84007410 con 0x7ffb9806cfc0 2024-05-22T21:14:14.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb8ea00640 1 --2- 172.21.15.161:0/755972007 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb78066690 0x7ffb78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:14.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb8ea00640 1 -- 172.21.15.161:0/755972007 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ffb840833e0 con 0x7ffb9806cfc0 2024-05-22T21:14:14.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.961+0000 7ffb96a00640 1 --2- 172.21.15.161:0/755972007 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb78066690 0x7ffb78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:14.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.965+0000 7ffb96a00640 1 --2- 172.21.15.161:0/755972007 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb78066690 0x7ffb78068b50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffb880011f0 tx=0x7ffb88009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:14.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:14.965+0000 7ffb8ea00640 1 -- 172.21.15.161:0/755972007 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffb84050b40 con 0x7ffb9806cfc0 2024-05-22T21:14:15.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.173+0000 7ffb9d800640 1 -- 172.21.15.161:0/755972007 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7ffb9815c0d0 con 0x7ffb9806cfc0 2024-05-22T21:14:15.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.177+0000 7ffb8ea00640 1 -- 172.21.15.161:0/755972007 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffb84054160 con 0x7ffb9806cfc0 2024-05-22T21:14:15.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.181+0000 7ffb9d800640 1 -- 172.21.15.161:0/755972007 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb78066690 msgr2=0x7ffb78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:15.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.181+0000 7ffb9d800640 1 --2- 172.21.15.161:0/755972007 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb78066690 0x7ffb78068b50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffb880011f0 tx=0x7ffb88009040 comp rx=0 tx=0).stop 2024-05-22T21:14:15.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.181+0000 7ffb9d800640 1 -- 172.21.15.161:0/755972007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9806cfc0 msgr2=0x7ffb98146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:15.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.181+0000 7ffb9d800640 1 --2- 172.21.15.161:0/755972007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9806cfc0 0x7ffb98146d80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400c4d0 tx=0x7ffb8400c990 comp rx=0 tx=0).stop 2024-05-22T21:14:15.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.181+0000 7ffb9d800640 1 -- 172.21.15.161:0/755972007 shutdown_connections 2024-05-22T21:14:15.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.181+0000 7ffb9d800640 1 --2- 172.21.15.161:0/755972007 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb78066690 0x7ffb78068b50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.181+0000 7ffb9d800640 1 --2- 172.21.15.161:0/755972007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb981596b0 0x7ffb981f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.181+0000 7ffb9d800640 1 --2- 172.21.15.161:0/755972007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9806cfc0 0x7ffb98146d80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.181+0000 7ffb9d800640 1 --2- 172.21.15.161:0/755972007 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb9806cb30 0x7ffb9815ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.181+0000 7ffb9d800640 1 -- 172.21.15.161:0/755972007 >> 172.21.15.161:0/755972007 conn(0x7ffb9806a2d0 msgr2=0x7ffb98147bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:15.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.181+0000 7ffb9d800640 1 -- 172.21.15.161:0/755972007 shutdown_connections 2024-05-22T21:14:15.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.181+0000 7ffb9d800640 1 -- 172.21.15.161:0/755972007 wait complete. 2024-05-22T21:14:15.200 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-22T21:14:15.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.333+0000 7f1be9400640 1 Processor -- start 2024-05-22T21:14:15.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.333+0000 7f1be9400640 1 -- start start 2024-05-22T21:14:15.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1be9400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1be4143c40 0x7f1be415b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:15.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1be9400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1be41444a0 0x7f1be406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:15.337 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1be9400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be415efb0 con 0x7f1be4144900 2024-05-22T21:14:15.337 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1be9400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be415f290 con 0x7f1be41444a0 2024-05-22T21:14:15.337 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1be9400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be415f570 con 0x7f1be4143c40 2024-05-22T21:14:15.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1be3200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1be4144900 0x7f1be40b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59288/0 (socket says 172.21.15.161:59288) 2024-05-22T21:14:15.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1be3200640 1 -- 172.21.15.161:0/1387837236 learned_addr learned my addr 172.21.15.161:0/1387837236 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:15.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1bd9e00640 1 --2- 172.21.15.161:0/1387837236 >> v2:172.21.15.179:3300/0 conn(0x7f1be41444a0 0x7f1be406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:15.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1be2800640 1 --2- 172.21.15.161:0/1387837236 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1be4143c40 0x7f1be415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:15.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1be1e00640 1 -- 172.21.15.161:0/1387837236 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219134047 0 0) 0x7f1be415efb0 con 0x7f1be4144900 2024-05-22T21:14:15.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1be1e00640 1 -- 172.21.15.161:0/1387837236 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bc8003530 con 0x7f1be4144900 2024-05-22T21:14:15.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1be1e00640 1 -- 172.21.15.161:0/1387837236 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3405428861 0 0) 0x7f1bc8003530 con 0x7f1be4144900 2024-05-22T21:14:15.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1bd9e00640 1 -- 172.21.15.161:0/1387837236 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1be4143c40 msgr2=0x7f1be415b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:15.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1bd9e00640 1 --2- 172.21.15.161:0/1387837236 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1be4143c40 0x7f1be415b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1bd9e00640 1 -- 172.21.15.161:0/1387837236 >> v1:172.21.15.161:6789/0 conn(0x7f1be4144900 legacy=0x7f1be40b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:15.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1bd9e00640 1 -- 172.21.15.161:0/1387837236 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be415f850 con 0x7f1be41444a0 2024-05-22T21:14:15.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.337+0000 7f1be2800640 1 --2- 172.21.15.161:0/1387837236 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1be4143c40 0x7f1be415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:15.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.341+0000 7f1bd9e00640 1 --2- 172.21.15.161:0/1387837236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be41444a0 0x7f1be406c2a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0097c0 tx=0x7f1bcc01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=866d1cda31503a72 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:15.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.341+0000 7f1be1e00640 1 -- 172.21.15.161:0/1387837236 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0a4ea0 con 0x7f1be41444a0 2024-05-22T21:14:15.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.341+0000 7f1be1e00640 1 -- 172.21.15.161:0/1387837236 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc0a7050 con 0x7f1be41444a0 2024-05-22T21:14:15.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.341+0000 7f1be1e00640 1 -- 172.21.15.161:0/1387837236 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc001680 con 0x7f1be41444a0 2024-05-22T21:14:15.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.341+0000 7f1be9400640 1 -- 172.21.15.161:0/1387837236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be41444a0 msgr2=0x7f1be406c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:15.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.341+0000 7f1be9400640 1 --2- 172.21.15.161:0/1387837236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be41444a0 0x7f1be406c2a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0097c0 tx=0x7f1bcc01c3c0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.341+0000 7f1be9400640 1 -- 172.21.15.161:0/1387837236 shutdown_connections 2024-05-22T21:14:15.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.341+0000 7f1be9400640 1 --2- 172.21.15.161:0/1387837236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be41444a0 0x7f1be406c2a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.341+0000 7f1be9400640 1 --2- 172.21.15.161:0/1387837236 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1be4143c40 0x7f1be415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.341+0000 7f1be9400640 1 -- 172.21.15.161:0/1387837236 >> 172.21.15.161:0/1387837236 conn(0x7f1be406a2d0 msgr2=0x7f1be406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:15.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.341+0000 7f1be9400640 1 -- 172.21.15.161:0/1387837236 shutdown_connections 2024-05-22T21:14:15.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.341+0000 7f1be9400640 1 -- 172.21.15.161:0/1387837236 wait complete. 2024-05-22T21:14:15.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.341+0000 7f1be9400640 1 Processor -- start 2024-05-22T21:14:15.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.341+0000 7f1be9400640 1 -- start start 2024-05-22T21:14:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1be4143c40 0x7f1be4136760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be9400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be41444a0 0x7f1be4158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be9400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1be4144900 0x7f1be4142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be9400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be4136ca0 con 0x7f1be4143c40 2024-05-22T21:14:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be9400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be415dca0 con 0x7f1be41444a0 2024-05-22T21:14:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be9400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be415df50 con 0x7f1be4144900 2024-05-22T21:14:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be2800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1be4143c40 0x7f1be4136760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be2800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1be4143c40 0x7f1be4136760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39882/0 (socket says 172.21.15.161:39882) 2024-05-22T21:14:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be2800640 1 -- 172.21.15.161:0/1230093920 learned_addr learned my addr 172.21.15.161:0/1230093920 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be2800640 1 -- 172.21.15.161:0/1230093920 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1be4144900 msgr2=0x7f1be4142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:15.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be2800640 1 --2- 172.21.15.161:0/1230093920 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1be4144900 0x7f1be4142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be2800640 1 -- 172.21.15.161:0/1230093920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be41444a0 msgr2=0x7f1be4158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:15.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be2800640 1 --2- 172.21.15.161:0/1230093920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be41444a0 0x7f1be4158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be2800640 1 -- 172.21.15.161:0/1230093920 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd4009a70 con 0x7f1be4143c40 2024-05-22T21:14:15.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be2800640 1 --2- 172.21.15.161:0/1230093920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1be4143c40 0x7f1be4136760 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400efc0 tx=0x7f1bd400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:15.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1bdb400640 1 -- 172.21.15.161:0/1230093920 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4014680 con 0x7f1be4143c40 2024-05-22T21:14:15.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be9400640 1 -- 172.21.15.161:0/1230093920 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc009420 con 0x7f1be4143c40 2024-05-22T21:14:15.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1bdb400640 1 -- 172.21.15.161:0/1230093920 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd400f040 con 0x7f1be4143c40 2024-05-22T21:14:15.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1bdb400640 1 -- 172.21.15.161:0/1230093920 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4014a00 con 0x7f1be4143c40 2024-05-22T21:14:15.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be9400640 1 -- 172.21.15.161:0/1230093920 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be415e510 con 0x7f1be4143c40 2024-05-22T21:14:15.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1be9400640 1 -- 172.21.15.161:0/1230093920 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be4067220 con 0x7f1be4143c40 2024-05-22T21:14:15.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.345+0000 7f1bdb400640 1 -- 172.21.15.161:0/1230093920 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1bd4014b60 con 0x7f1be4143c40 2024-05-22T21:14:15.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.349+0000 7f1bdb400640 1 --2- 172.21.15.161:0/1230093920 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bc0066690 0x7f1bc0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:15.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.349+0000 7f1bdb400640 1 -- 172.21.15.161:0/1230093920 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1bd4084e50 con 0x7f1be4143c40 2024-05-22T21:14:15.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.349+0000 7f1bd9e00640 1 --2- 172.21.15.161:0/1230093920 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bc0066690 0x7f1bc0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:15.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.349+0000 7f1bd9e00640 1 --2- 172.21.15.161:0/1230093920 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bc0066690 0x7f1bc0068b50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0ad000 tx=0x7f1bcc004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:15.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.353+0000 7f1bdb400640 1 -- 172.21.15.161:0/1230093920 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1bd4052520 con 0x7f1be4143c40 2024-05-22T21:14:15.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.561+0000 7f1be9400640 1 -- 172.21.15.161:0/1230093920 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f1be40ad360 con 0x7f1be4143c40 2024-05-22T21:14:15.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.565+0000 7f1bdb400640 1 -- 172.21.15.161:0/1230093920 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1bd4055b40 con 0x7f1be4143c40 2024-05-22T21:14:15.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.569+0000 7f1be9400640 1 -- 172.21.15.161:0/1230093920 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bc0066690 msgr2=0x7f1bc0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:15.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.569+0000 7f1be9400640 1 --2- 172.21.15.161:0/1230093920 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bc0066690 0x7f1bc0068b50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0ad000 tx=0x7f1bcc004740 comp rx=0 tx=0).stop 2024-05-22T21:14:15.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.569+0000 7f1be9400640 1 -- 172.21.15.161:0/1230093920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1be4143c40 msgr2=0x7f1be4136760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:15.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.569+0000 7f1be9400640 1 --2- 172.21.15.161:0/1230093920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1be4143c40 0x7f1be4136760 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400efc0 tx=0x7f1bd400c490 comp rx=0 tx=0).stop 2024-05-22T21:14:15.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.569+0000 7f1be9400640 1 -- 172.21.15.161:0/1230093920 shutdown_connections 2024-05-22T21:14:15.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.569+0000 7f1be9400640 1 --2- 172.21.15.161:0/1230093920 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bc0066690 0x7f1bc0068b50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.569+0000 7f1be9400640 1 --2- 172.21.15.161:0/1230093920 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1be4144900 0x7f1be4142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.569+0000 7f1be9400640 1 --2- 172.21.15.161:0/1230093920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be41444a0 0x7f1be4158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.569+0000 7f1be9400640 1 --2- 172.21.15.161:0/1230093920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1be4143c40 0x7f1be4136760 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.569+0000 7f1be9400640 1 -- 172.21.15.161:0/1230093920 >> 172.21.15.161:0/1230093920 conn(0x7f1be406a2d0 msgr2=0x7f1be40ad580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:15.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.569+0000 7f1be9400640 1 -- 172.21.15.161:0/1230093920 shutdown_connections 2024-05-22T21:14:15.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.569+0000 7f1be9400640 1 -- 172.21.15.161:0/1230093920 wait complete. 2024-05-22T21:14:15.589 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:15.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52de800640 1 Processor -- start 2024-05-22T21:14:15.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52de800640 1 -- start start 2024-05-22T21:14:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52de800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d806b5f0 0x7f52d806c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52de800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d815db70 con 0x7f52d80b4420 2024-05-22T21:14:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52de800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d80ad610 con 0x7f52d806c8f0 2024-05-22T21:14:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52de800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d806cd00 con 0x7f52d806b5f0 2024-05-22T21:14:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d7400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f52d80b4420 0x7f52d806aee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59292/0 (socket says 172.21.15.161:59292) 2024-05-22T21:14:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d7400640 1 -- 172.21.15.161:0/1994126291 learned_addr learned my addr 172.21.15.161:0/1994126291 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:15.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6000640 1 -- 172.21.15.161:0/1994126291 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874506507 0 0) 0x7f52d815db70 con 0x7f52d80b4420 2024-05-22T21:14:15.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6000640 1 -- 172.21.15.161:0/1994126291 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52bc003530 con 0x7f52d80b4420 2024-05-22T21:14:15.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6a00640 1 --2- 172.21.15.161:0/1994126291 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d806b5f0 0x7f52d806c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6000640 1 -- 172.21.15.161:0/1994126291 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3448051471 0 0) 0x7f52bc003530 con 0x7f52d80b4420 2024-05-22T21:14:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6000640 1 -- 172.21.15.161:0/1994126291 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d806b5f0 msgr2=0x7f52d806c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6000640 1 --2- 172.21.15.161:0/1994126291 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d806b5f0 0x7f52d806c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6000640 1 -- 172.21.15.161:0/1994126291 >> v1:172.21.15.179:6789/0 conn(0x7f52d806c8f0 legacy=0x7f52d80ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6000640 1 -- 172.21.15.161:0/1994126291 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d80b26f0 con 0x7f52d80b4420 2024-05-22T21:14:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6000640 1 -- 172.21.15.161:0/1994126291 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f52c0001680 con 0x7f52d80b4420 2024-05-22T21:14:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6000640 1 -- 172.21.15.161:0/1994126291 >> v1:172.21.15.161:6789/0 conn(0x7f52d80b4420 legacy=0x7f52d806aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6000640 1 --2- 172.21.15.161:0/1994126291 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d806b5f0 0x7f52bc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6000640 1 --2- 172.21.15.161:0/1994126291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52bc003fe0 0x7f52bc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6000640 1 --2- 172.21.15.161:0/1994126291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52bc004d10 0x7f52bc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6000640 1 -- 172.21.15.161:0/1994126291 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d806cd00 con 0x7f52bc003fe0 2024-05-22T21:14:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6000640 1 -- 172.21.15.161:0/1994126291 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc00bb50 con 0x7f52bc004d10 2024-05-22T21:14:15.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6000640 1 -- 172.21.15.161:0/1994126291 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc00bd10 con 0x7f52d806b5f0 2024-05-22T21:14:15.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6a00640 1 --2- 172.21.15.161:0/1994126291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52bc003fe0 0x7f52bc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:15.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d7400640 1 --2- 172.21.15.161:0/1994126291 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d806b5f0 0x7f52bc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:15.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d7e00640 1 --2- 172.21.15.161:0/1994126291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52bc004d10 0x7f52bc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:15.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6a00640 1 -- 172.21.15.161:0/1994126291 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d806b5f0 msgr2=0x7f52bc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:15.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6a00640 1 --2- 172.21.15.161:0/1994126291 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d806b5f0 0x7f52bc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6a00640 1 -- 172.21.15.161:0/1994126291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52bc004d10 msgr2=0x7f52bc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:15.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6a00640 1 --2- 172.21.15.161:0/1994126291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52bc004d10 0x7f52bc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d6a00640 1 -- 172.21.15.161:0/1994126291 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc003530 con 0x7f52bc003fe0 2024-05-22T21:14:15.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d7e00640 1 --2- 172.21.15.161:0/1994126291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52bc004d10 0x7f52bc005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:15.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.725+0000 7f52d7400640 1 --2- 172.21.15.161:0/1994126291 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d806b5f0 0x7f52bc0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:15.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.729+0000 7f52d6a00640 1 --2- 172.21.15.161:0/1994126291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52bc003fe0 0x7f52bc0047d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f52c800ba30 tx=0x7f52c801c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66bb748d9f01e69 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:15.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.729+0000 7f52d6000640 1 -- 172.21.15.161:0/1994126291 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c80ac440 con 0x7f52bc003fe0 2024-05-22T21:14:15.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.729+0000 7f52d6000640 1 -- 172.21.15.161:0/1994126291 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c80ac5a0 con 0x7f52bc003fe0 2024-05-22T21:14:15.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.729+0000 7f52d6000640 1 -- 172.21.15.161:0/1994126291 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c80ac950 con 0x7f52bc003fe0 2024-05-22T21:14:15.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.729+0000 7f52de800640 1 -- 172.21.15.161:0/1994126291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52bc003fe0 msgr2=0x7f52bc0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:15.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.729+0000 7f52de800640 1 --2- 172.21.15.161:0/1994126291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52bc003fe0 0x7f52bc0047d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f52c800ba30 tx=0x7f52c801c6a0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.729+0000 7f52d7e00640 1 -- 172.21.15.161:0/1994126291 reap_dead start 2024-05-22T21:14:15.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.729+0000 7f52de800640 1 -- 172.21.15.161:0/1994126291 shutdown_connections 2024-05-22T21:14:15.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.729+0000 7f52de800640 1 -- 172.21.15.161:0/1994126291 >> 172.21.15.161:0/1994126291 conn(0x7f52d8070630 msgr2=0x7f52d8070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:15.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.729+0000 7f52de800640 1 -- 172.21.15.161:0/1994126291 shutdown_connections 2024-05-22T21:14:15.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.729+0000 7f52de800640 1 -- 172.21.15.161:0/1994126291 wait complete. 2024-05-22T21:14:15.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52de800640 1 Processor -- start 2024-05-22T21:14:15.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52de800640 1 -- start start 2024-05-22T21:14:15.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52de800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52bc003fe0 0x7f52d81448a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:15.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52de800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d81473e0 0x7f52d8153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:15.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52de800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d80caeb0 0x7f52d80ce340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:15.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52de800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d80c52e0 con 0x7f52d80caeb0 2024-05-22T21:14:15.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52de800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d814e690 con 0x7f52d81473e0 2024-05-22T21:14:15.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52de800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d80c4660 con 0x7f52bc003fe0 2024-05-22T21:14:15.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52d7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d80caeb0 0x7f52d80ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:15.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52d7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d80caeb0 0x7f52d80ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39900/0 (socket says 172.21.15.161:39900) 2024-05-22T21:14:15.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52d7e00640 1 -- 172.21.15.161:0/2212266963 learned_addr learned my addr 172.21.15.161:0/2212266963 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:15.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52d7400640 1 --2- 172.21.15.161:0/2212266963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52bc003fe0 0x7f52d81448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:15.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52d6a00640 1 --2- 172.21.15.161:0/2212266963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d81473e0 0x7f52d8153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:15.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52d7e00640 1 -- 172.21.15.161:0/2212266963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52bc003fe0 msgr2=0x7f52d81448a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:15.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52d7e00640 1 --2- 172.21.15.161:0/2212266963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52bc003fe0 0x7f52d81448a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52d7e00640 1 -- 172.21.15.161:0/2212266963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d81473e0 msgr2=0x7f52d8153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:15.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52d7e00640 1 --2- 172.21.15.161:0/2212266963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d81473e0 0x7f52d8153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52d7e00640 1 -- 172.21.15.161:0/2212266963 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d80c4940 con 0x7f52d80caeb0 2024-05-22T21:14:15.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52d6a00640 1 --2- 172.21.15.161:0/2212266963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d81473e0 0x7f52d8153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:15.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52d7e00640 1 --2- 172.21.15.161:0/2212266963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d80caeb0 0x7f52d80ce340 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f52d0000f90 tx=0x7f52d000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:15.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52cf400640 1 -- 172.21.15.161:0/2212266963 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d0003980 con 0x7f52d80caeb0 2024-05-22T21:14:15.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52cf400640 1 -- 172.21.15.161:0/2212266963 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d0003ae0 con 0x7f52d80caeb0 2024-05-22T21:14:15.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52de800640 1 -- 172.21.15.161:0/2212266963 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d80c4c20 con 0x7f52d80caeb0 2024-05-22T21:14:15.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52cf400640 1 -- 172.21.15.161:0/2212266963 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d0003e70 con 0x7f52d80caeb0 2024-05-22T21:14:15.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.733+0000 7f52de800640 1 -- 172.21.15.161:0/2212266963 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d80cb330 con 0x7f52d80caeb0 2024-05-22T21:14:15.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.737+0000 7f52de800640 1 -- 172.21.15.161:0/2212266963 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d80c5420 con 0x7f52d80caeb0 2024-05-22T21:14:15.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.737+0000 7f52cf400640 1 -- 172.21.15.161:0/2212266963 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f52d00050d0 con 0x7f52d80caeb0 2024-05-22T21:14:15.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.737+0000 7f52cf400640 1 --2- 172.21.15.161:0/2212266963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52b40666c0 0x7f52b4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:15.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.737+0000 7f52cf400640 1 -- 172.21.15.161:0/2212266963 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f52d0083c00 con 0x7f52d80caeb0 2024-05-22T21:14:15.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.737+0000 7f52d7400640 1 --2- 172.21.15.161:0/2212266963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52b40666c0 0x7f52b4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:15.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.737+0000 7f52d7400640 1 --2- 172.21.15.161:0/2212266963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52b40666c0 0x7f52b4068b80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f52c0005360 tx=0x7f52c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:15.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.741+0000 7f52cf400640 1 -- 172.21.15.161:0/2212266963 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f52d80c5420 con 0x7f52d80caeb0 2024-05-22T21:14:15.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.897+0000 7f52de800640 1 -- 172.21.15.161:0/2212266963 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52d80c5420 con 0x7f52d80caeb0 2024-05-22T21:14:15.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.897+0000 7f52cf400640 1 -- 172.21.15.161:0/2212266963 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12437 (secure 0 0 0) 0x7f52d80c5420 con 0x7f52d80caeb0 2024-05-22T21:14:15.899 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:15.899 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":29,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:14:12.841744+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:15.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.901+0000 7f52de800640 1 -- 172.21.15.161:0/2212266963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52b40666c0 msgr2=0x7f52b4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:15.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.901+0000 7f52de800640 1 --2- 172.21.15.161:0/2212266963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52b40666c0 0x7f52b4068b80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f52c0005360 tx=0x7f52c0009040 comp rx=0 tx=0).stop 2024-05-22T21:14:15.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.901+0000 7f52de800640 1 -- 172.21.15.161:0/2212266963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d80caeb0 msgr2=0x7f52d80ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:15.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.901+0000 7f52de800640 1 --2- 172.21.15.161:0/2212266963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d80caeb0 0x7f52d80ce340 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f52d0000f90 tx=0x7f52d000eb90 comp rx=0 tx=0).stop 2024-05-22T21:14:15.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.901+0000 7f52de800640 1 -- 172.21.15.161:0/2212266963 shutdown_connections 2024-05-22T21:14:15.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.901+0000 7f52de800640 1 --2- 172.21.15.161:0/2212266963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52b40666c0 0x7f52b4068b80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.901+0000 7f52de800640 1 --2- 172.21.15.161:0/2212266963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d80caeb0 0x7f52d80ce340 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.901+0000 7f52de800640 1 --2- 172.21.15.161:0/2212266963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d81473e0 0x7f52d8153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.901+0000 7f52de800640 1 --2- 172.21.15.161:0/2212266963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52bc003fe0 0x7f52d81448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:15.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.901+0000 7f52de800640 1 -- 172.21.15.161:0/2212266963 >> 172.21.15.161:0/2212266963 conn(0x7f52d8070630 msgr2=0x7f52d80b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:15.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.901+0000 7f52de800640 1 -- 172.21.15.161:0/2212266963 shutdown_connections 2024-05-22T21:14:15.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:15.901+0000 7f52de800640 1 -- 172.21.15.161:0/2212266963 wait complete. 2024-05-22T21:14:15.920 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:14:15.920 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:14:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.057+0000 7f3d19c00640 1 Processor -- start 2024-05-22T21:14:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.057+0000 7f3d19c00640 1 -- start start 2024-05-22T21:14:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d19c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3d1413cfc0 0x7f3d141402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d19c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d1415af60 0x7f3d1413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d19c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3d1413fc80 0x7f3d1415dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d19c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d141400d0 con 0x7f3d1413fc80 2024-05-22T21:14:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d19c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1413d3a0 con 0x7f3d1413cfc0 2024-05-22T21:14:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d19c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d140cd920 con 0x7f3d1415af60 2024-05-22T21:14:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d13400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3d1413fc80 0x7f3d1415dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d13400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3d1413fc80 0x7f3d1415dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48818/0 (socket says 172.21.15.161:48818) 2024-05-22T21:14:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d13400640 1 -- 172.21.15.161:0/3576032782 learned_addr learned my addr 172.21.15.161:0/3576032782 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d12000640 1 --2- 172.21.15.161:0/3576032782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d1415af60 0x7f3d1413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d12a00640 1 --2- 172.21.15.161:0/3576032782 >> v2:172.21.15.179:3300/0 conn(0x7f3d1413cfc0 0x7f3d141402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d13400640 1 -- 172.21.15.161:0/3576032782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d1415af60 msgr2=0x7f3d1413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:16.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d13400640 1 --2- 172.21.15.161:0/3576032782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d1415af60 0x7f3d1413f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d13400640 1 -- 172.21.15.161:0/3576032782 >> v2:172.21.15.179:3300/0 conn(0x7f3d1413cfc0 msgr2=0x7f3d141402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:16.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d13400640 1 --2- 172.21.15.161:0/3576032782 >> v2:172.21.15.179:3300/0 conn(0x7f3d1413cfc0 0x7f3d141402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d13400640 1 -- 172.21.15.161:0/3576032782 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d140cdc00 con 0x7f3d1413fc80 2024-05-22T21:14:16.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.061+0000 7f3d13400640 1 --2- 172.21.15.161:0/3576032782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d1413fc80 0x7f3d1415dad0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3d1406de90 tx=0x7f3d0c01c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0fabcc2f71622b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:16.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d11600640 1 -- 172.21.15.161:0/3576032782 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c0a6500 con 0x7f3d1413fc80 2024-05-22T21:14:16.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d11600640 1 -- 172.21.15.161:0/3576032782 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c0a6660 con 0x7f3d1413fc80 2024-05-22T21:14:16.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d11600640 1 -- 172.21.15.161:0/3576032782 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c0a69f0 con 0x7f3d1413fc80 2024-05-22T21:14:16.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 -- 172.21.15.161:0/3576032782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d1413fc80 msgr2=0x7f3d1415dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:16.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 --2- 172.21.15.161:0/3576032782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d1413fc80 0x7f3d1415dad0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3d1406de90 tx=0x7f3d0c01c670 comp rx=0 tx=0).stop 2024-05-22T21:14:16.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 -- 172.21.15.161:0/3576032782 shutdown_connections 2024-05-22T21:14:16.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 --2- 172.21.15.161:0/3576032782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d1413fc80 0x7f3d1415dad0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 --2- 172.21.15.161:0/3576032782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d1415af60 0x7f3d1413f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 --2- 172.21.15.161:0/3576032782 >> v2:172.21.15.179:3300/0 conn(0x7f3d1413cfc0 0x7f3d141402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 -- 172.21.15.161:0/3576032782 >> 172.21.15.161:0/3576032782 conn(0x7f3d1406a2d0 msgr2=0x7f3d1406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:16.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 -- 172.21.15.161:0/3576032782 shutdown_connections 2024-05-22T21:14:16.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 -- 172.21.15.161:0/3576032782 wait complete. 2024-05-22T21:14:16.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 Processor -- start 2024-05-22T21:14:16.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 -- start start 2024-05-22T21:14:16.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d1413cfc0 0x7f3d1413a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d1413fc80 0x7f3d14137610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d1415af60 0x7f3d141570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1413a7f0 con 0x7f3d1415af60 2024-05-22T21:14:16.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1413aad0 con 0x7f3d1413cfc0 2024-05-22T21:14:16.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d19c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1413adb0 con 0x7f3d1413fc80 2024-05-22T21:14:16.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d13400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d1415af60 0x7f3d141570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d13400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d1415af60 0x7f3d141570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48826/0 (socket says 172.21.15.161:48826) 2024-05-22T21:14:16.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d13400640 1 -- 172.21.15.161:0/287491718 learned_addr learned my addr 172.21.15.161:0/287491718 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:16.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d12000640 1 --2- 172.21.15.161:0/287491718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d1413fc80 0x7f3d14137610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.065+0000 7f3d12a00640 1 --2- 172.21.15.161:0/287491718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d1413cfc0 0x7f3d1413a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d13400640 1 -- 172.21.15.161:0/287491718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d1413fc80 msgr2=0x7f3d14137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:16.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d13400640 1 --2- 172.21.15.161:0/287491718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d1413fc80 0x7f3d14137610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d13400640 1 -- 172.21.15.161:0/287491718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d1413cfc0 msgr2=0x7f3d1413a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:16.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d13400640 1 --2- 172.21.15.161:0/287491718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d1413cfc0 0x7f3d1413a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d13400640 1 -- 172.21.15.161:0/287491718 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c009b30 con 0x7f3d1415af60 2024-05-22T21:14:16.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d12a00640 1 --2- 172.21.15.161:0/287491718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d1413cfc0 0x7f3d1413a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:16.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d12000640 1 --2- 172.21.15.161:0/287491718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d1413fc80 0x7f3d14137610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:16.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d13400640 1 --2- 172.21.15.161:0/287491718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d1415af60 0x7f3d141570a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c01ad20 tx=0x7f3d0c019410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:16.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d0aa00640 1 -- 172.21.15.161:0/287491718 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c0a69d0 con 0x7f3d1415af60 2024-05-22T21:14:16.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d0aa00640 1 -- 172.21.15.161:0/287491718 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c0a6b30 con 0x7f3d1415af60 2024-05-22T21:14:16.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d19c00640 1 -- 172.21.15.161:0/287491718 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1413b090 con 0x7f3d1415af60 2024-05-22T21:14:16.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d0aa00640 1 -- 172.21.15.161:0/287491718 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c0af480 con 0x7f3d1415af60 2024-05-22T21:14:16.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d19c00640 1 -- 172.21.15.161:0/287491718 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d14137b50 con 0x7f3d1415af60 2024-05-22T21:14:16.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d19c00640 1 -- 172.21.15.161:0/287491718 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d14067220 con 0x7f3d1415af60 2024-05-22T21:14:16.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d0aa00640 1 -- 172.21.15.161:0/287491718 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3d0c0af5e0 con 0x7f3d1415af60 2024-05-22T21:14:16.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d0aa00640 1 --2- 172.21.15.161:0/287491718 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3cf4066690 0x7f3cf4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.069+0000 7f3d0aa00640 1 -- 172.21.15.161:0/287491718 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3d0c0aa070 con 0x7f3d1415af60 2024-05-22T21:14:16.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.073+0000 7f3d12a00640 1 --2- 172.21.15.161:0/287491718 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3cf4066690 0x7f3cf4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.073+0000 7f3d12a00640 1 --2- 172.21.15.161:0/287491718 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3cf4066690 0x7f3cf4068b50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3d1415b8c0 tx=0x7f3d04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:16.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.073+0000 7f3d0aa00640 1 -- 172.21.15.161:0/287491718 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3d0c0eb800 con 0x7f3d1415af60 2024-05-22T21:14:16.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.229+0000 7f3d19c00640 1 -- 172.21.15.161:0/287491718 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d14071670 con 0x7f3d1415af60 2024-05-22T21:14:16.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.229+0000 7f3d0aa00640 1 -- 172.21.15.161:0/287491718 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d0c0eee20 con 0x7f3d1415af60 2024-05-22T21:14:16.231 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:14:16.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.233+0000 7f3d19c00640 1 -- 172.21.15.161:0/287491718 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3cf4066690 msgr2=0x7f3cf4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:16.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.233+0000 7f3d19c00640 1 --2- 172.21.15.161:0/287491718 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3cf4066690 0x7f3cf4068b50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3d1415b8c0 tx=0x7f3d04008040 comp rx=0 tx=0).stop 2024-05-22T21:14:16.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.233+0000 7f3d19c00640 1 -- 172.21.15.161:0/287491718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d1415af60 msgr2=0x7f3d141570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:16.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.233+0000 7f3d19c00640 1 --2- 172.21.15.161:0/287491718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d1415af60 0x7f3d141570a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c01ad20 tx=0x7f3d0c019410 comp rx=0 tx=0).stop 2024-05-22T21:14:16.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.233+0000 7f3d19c00640 1 -- 172.21.15.161:0/287491718 shutdown_connections 2024-05-22T21:14:16.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.233+0000 7f3d19c00640 1 --2- 172.21.15.161:0/287491718 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3cf4066690 0x7f3cf4068b50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.233+0000 7f3d19c00640 1 --2- 172.21.15.161:0/287491718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d1415af60 0x7f3d141570a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.233+0000 7f3d19c00640 1 --2- 172.21.15.161:0/287491718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d1413fc80 0x7f3d14137610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.233+0000 7f3d19c00640 1 --2- 172.21.15.161:0/287491718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d1413cfc0 0x7f3d1413a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.233+0000 7f3d19c00640 1 -- 172.21.15.161:0/287491718 >> 172.21.15.161:0/287491718 conn(0x7f3d1406a2d0 msgr2=0x7f3d140af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:16.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.233+0000 7f3d19c00640 1 -- 172.21.15.161:0/287491718 shutdown_connections 2024-05-22T21:14:16.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.233+0000 7f3d19c00640 1 -- 172.21.15.161:0/287491718 wait complete. 2024-05-22T21:14:16.252 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-22T21:14:16.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.385+0000 7f0df7200640 1 Processor -- start 2024-05-22T21:14:16.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.385+0000 7f0df7200640 1 -- start start 2024-05-22T21:14:16.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0df7200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0df015d4a0 0x7f0df0099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0df7200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0df00b0490 0x7f0df0099f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0df7200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0df006bfb0 0x7f0df0160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0df7200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df015d880 con 0x7f0df00b0490 2024-05-22T21:14:16.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0df7200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df006b440 con 0x7f0df015d4a0 2024-05-22T21:14:16.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0df7200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df006c440 con 0x7f0df006bfb0 2024-05-22T21:14:16.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0def400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0df00b0490 0x7f0df0099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0def400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0df00b0490 0x7f0df0099f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48834/0 (socket says 172.21.15.161:48834) 2024-05-22T21:14:16.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0def400640 1 -- 172.21.15.161:0/3280553542 learned_addr learned my addr 172.21.15.161:0/3280553542 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:16.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0df5200640 1 --2- 172.21.15.161:0/3280553542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0df006bfb0 0x7f0df0160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0defe00640 1 --2- 172.21.15.161:0/3280553542 >> v2:172.21.15.179:3300/0 conn(0x7f0df015d4a0 0x7f0df0099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0def400640 1 -- 172.21.15.161:0/3280553542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0df006bfb0 msgr2=0x7f0df0160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:16.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0def400640 1 --2- 172.21.15.161:0/3280553542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0df006bfb0 0x7f0df0160ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0def400640 1 -- 172.21.15.161:0/3280553542 >> v2:172.21.15.179:3300/0 conn(0x7f0df015d4a0 msgr2=0x7f0df0099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:16.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0def400640 1 --2- 172.21.15.161:0/3280553542 >> v2:172.21.15.179:3300/0 conn(0x7f0df015d4a0 0x7f0df0099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0def400640 1 -- 172.21.15.161:0/3280553542 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df00cf190 con 0x7f0df00b0490 2024-05-22T21:14:16.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.389+0000 7f0defe00640 1 --2- 172.21.15.161:0/3280553542 >> v2:172.21.15.179:3300/0 conn(0x7f0df015d4a0 0x7f0df0099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:16.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.393+0000 7f0def400640 1 --2- 172.21.15.161:0/3280553542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df00b0490 0x7f0df0099f80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0df00708c0 tx=0x7f0ddc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8836d9a3716daae server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:16.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.393+0000 7f0deea00640 1 -- 172.21.15.161:0/3280553542 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0aa610 con 0x7f0df00b0490 2024-05-22T21:14:16.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.393+0000 7f0deea00640 1 -- 172.21.15.161:0/3280553542 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc0a6040 con 0x7f0df00b0490 2024-05-22T21:14:16.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.393+0000 7f0deea00640 1 -- 172.21.15.161:0/3280553542 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0aa990 con 0x7f0df00b0490 2024-05-22T21:14:16.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.393+0000 7f0df7200640 1 -- 172.21.15.161:0/3280553542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df00b0490 msgr2=0x7f0df0099f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:16.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.393+0000 7f0df7200640 1 --2- 172.21.15.161:0/3280553542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df00b0490 0x7f0df0099f80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0df00708c0 tx=0x7f0ddc01c4f0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.393+0000 7f0df7200640 1 -- 172.21.15.161:0/3280553542 shutdown_connections 2024-05-22T21:14:16.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.393+0000 7f0df7200640 1 --2- 172.21.15.161:0/3280553542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0df006bfb0 0x7f0df0160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.393+0000 7f0df7200640 1 --2- 172.21.15.161:0/3280553542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df00b0490 0x7f0df0099f80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.393+0000 7f0df7200640 1 --2- 172.21.15.161:0/3280553542 >> v2:172.21.15.179:3300/0 conn(0x7f0df015d4a0 0x7f0df0099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.393+0000 7f0df7200640 1 -- 172.21.15.161:0/3280553542 >> 172.21.15.161:0/3280553542 conn(0x7f0df00b2a40 msgr2=0x7f0df00b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:16.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.393+0000 7f0df7200640 1 -- 172.21.15.161:0/3280553542 shutdown_connections 2024-05-22T21:14:16.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.393+0000 7f0df7200640 1 -- 172.21.15.161:0/3280553542 wait complete. 2024-05-22T21:14:16.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.393+0000 7f0df7200640 1 Processor -- start 2024-05-22T21:14:16.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.393+0000 7f0df7200640 1 -- start start 2024-05-22T21:14:16.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0df7200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0df00b0490 0x7f0df00b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0df7200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df015d4a0 0x7f0df015d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0df7200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0df00b14a0 0x7f0df015df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0df7200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df0157ee0 con 0x7f0df015d4a0 2024-05-22T21:14:16.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0df7200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df01572d0 con 0x7f0df00b0490 2024-05-22T21:14:16.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0df7200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df01575b0 con 0x7f0df00b14a0 2024-05-22T21:14:16.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0def400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df015d4a0 0x7f0df015d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0def400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df015d4a0 0x7f0df015d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48842/0 (socket says 172.21.15.161:48842) 2024-05-22T21:14:16.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0def400640 1 -- 172.21.15.161:0/1700227686 learned_addr learned my addr 172.21.15.161:0/1700227686 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:16.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0defe00640 1 --2- 172.21.15.161:0/1700227686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0df00b0490 0x7f0df00b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0df5200640 1 --2- 172.21.15.161:0/1700227686 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0df00b14a0 0x7f0df015df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0def400640 1 -- 172.21.15.161:0/1700227686 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0df00b14a0 msgr2=0x7f0df015df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:16.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0def400640 1 --2- 172.21.15.161:0/1700227686 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0df00b14a0 0x7f0df015df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0def400640 1 -- 172.21.15.161:0/1700227686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0df00b0490 msgr2=0x7f0df00b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:16.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0def400640 1 --2- 172.21.15.161:0/1700227686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0df00b0490 0x7f0df00b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0def400640 1 -- 172.21.15.161:0/1700227686 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc0094d0 con 0x7f0df015d4a0 2024-05-22T21:14:16.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0def400640 1 --2- 172.21.15.161:0/1700227686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df015d4a0 0x7f0df015d9c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc01aba0 tx=0x7f0ddc01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:16.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0de7e00640 1 -- 172.21.15.161:0/1700227686 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0bd5f0 con 0x7f0df015d4a0 2024-05-22T21:14:16.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0de7e00640 1 -- 172.21.15.161:0/1700227686 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc0bd750 con 0x7f0df015d4a0 2024-05-22T21:14:16.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0df7200640 1 -- 172.21.15.161:0/1700227686 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df0157890 con 0x7f0df015d4a0 2024-05-22T21:14:16.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0de7e00640 1 -- 172.21.15.161:0/1700227686 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0bdb00 con 0x7f0df015d4a0 2024-05-22T21:14:16.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0df7200640 1 -- 172.21.15.161:0/1700227686 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df0158370 con 0x7f0df015d4a0 2024-05-22T21:14:16.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0df7200640 1 -- 172.21.15.161:0/1700227686 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df0158560 con 0x7f0df015d4a0 2024-05-22T21:14:16.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.397+0000 7f0de7e00640 1 -- 172.21.15.161:0/1700227686 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0ddc0b33f0 con 0x7f0df015d4a0 2024-05-22T21:14:16.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.401+0000 7f0de7e00640 1 --2- 172.21.15.161:0/1700227686 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0dd0066690 0x7f0dd0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.401+0000 7f0de7e00640 1 -- 172.21.15.161:0/1700227686 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0ddc11ee40 con 0x7f0df015d4a0 2024-05-22T21:14:16.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.401+0000 7f0defe00640 1 --2- 172.21.15.161:0/1700227686 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0dd0066690 0x7f0dd0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.401+0000 7f0defe00640 1 --2- 172.21.15.161:0/1700227686 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0dd0066690 0x7f0dd0068b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0de0006e90 tx=0x7f0de0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:16.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.405+0000 7f0de7e00640 1 -- 172.21.15.161:0/1700227686 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0ddc0ec520 con 0x7f0df015d4a0 2024-05-22T21:14:16.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.561+0000 7f0df7200640 1 -- 172.21.15.161:0/1700227686 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0df0067220 con 0x7f0df015d4a0 2024-05-22T21:14:16.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.561+0000 7f0de7e00640 1 -- 172.21.15.161:0/1700227686 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0ddc0efb40 con 0x7f0df015d4a0 2024-05-22T21:14:16.564 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:16.564 INFO:teuthology.orchestra.run.smithi161.stdout:[ 2024-05-22T21:14:16.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:16.564 INFO:teuthology.orchestra.run.smithi161.stdout: "poolnum": 1, 2024-05-22T21:14:16.564 INFO:teuthology.orchestra.run.smithi161.stdout: "poolname": ".mgr" 2024-05-22T21:14:16.564 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:16.564 INFO:teuthology.orchestra.run.smithi161.stdout:] 2024-05-22T21:14:16.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.565+0000 7f0df7200640 1 -- 172.21.15.161:0/1700227686 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0dd0066690 msgr2=0x7f0dd0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:16.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.565+0000 7f0df7200640 1 --2- 172.21.15.161:0/1700227686 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0dd0066690 0x7f0dd0068b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0de0006e90 tx=0x7f0de0008040 comp rx=0 tx=0).stop 2024-05-22T21:14:16.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.565+0000 7f0df7200640 1 -- 172.21.15.161:0/1700227686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df015d4a0 msgr2=0x7f0df015d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:16.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.565+0000 7f0df7200640 1 --2- 172.21.15.161:0/1700227686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df015d4a0 0x7f0df015d9c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc01aba0 tx=0x7f0ddc01ad30 comp rx=0 tx=0).stop 2024-05-22T21:14:16.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.565+0000 7f0df7200640 1 -- 172.21.15.161:0/1700227686 shutdown_connections 2024-05-22T21:14:16.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.565+0000 7f0df7200640 1 --2- 172.21.15.161:0/1700227686 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0dd0066690 0x7f0dd0068b50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.565+0000 7f0df7200640 1 --2- 172.21.15.161:0/1700227686 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0df00b14a0 0x7f0df015df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.565+0000 7f0df7200640 1 --2- 172.21.15.161:0/1700227686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0df015d4a0 0x7f0df015d9c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.565+0000 7f0df7200640 1 --2- 172.21.15.161:0/1700227686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0df00b0490 0x7f0df00b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.565+0000 7f0df7200640 1 -- 172.21.15.161:0/1700227686 >> 172.21.15.161:0/1700227686 conn(0x7f0df00b2a40 msgr2=0x7f0df00cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:16.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.565+0000 7f0df7200640 1 -- 172.21.15.161:0/1700227686 shutdown_connections 2024-05-22T21:14:16.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.565+0000 7f0df7200640 1 -- 172.21.15.161:0/1700227686 wait complete. 2024-05-22T21:14:16.584 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-22T21:14:16.584 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-22T21:14:16.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f844b200640 1 Processor -- start 2024-05-22T21:14:16.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f844b200640 1 -- start start 2024-05-22T21:14:16.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f844b200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f844c15d040 0x7f844c0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f844b200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f844c0d0cf0 0x7f844c0d1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f844b200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f844c0b13d0 con 0x7f844c0cd840 2024-05-22T21:14:16.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f844b200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f844c0d1690 con 0x7f844c0d0cf0 2024-05-22T21:14:16.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f844b200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844c0b0aa0 con 0x7f844c15d040 2024-05-22T21:14:16.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f8449400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f844c0d0cf0 0x7f844c0d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f8449400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f844c0d0cf0 0x7f844c0d1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48850/0 (socket says 172.21.15.161:48850) 2024-05-22T21:14:16.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f844a800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f844c0cd840 0x7f844c0cdcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:38300/0 (socket says 172.21.15.161:38300) 2024-05-22T21:14:16.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f8449400640 1 -- 172.21.15.161:0/3876922027 learned_addr learned my addr 172.21.15.161:0/3876922027 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:16.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f8449e00640 1 --2- 172.21.15.161:0/3876922027 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f844c15d040 0x7f844c0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f8449400640 1 -- 172.21.15.161:0/3876922027 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f844c15d040 msgr2=0x7f844c0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:16.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f8449400640 1 --2- 172.21.15.161:0/3876922027 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f844c15d040 0x7f844c0b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f8449400640 1 -- 172.21.15.161:0/3876922027 >> v1:172.21.15.161:6789/0 conn(0x7f844c0cd840 legacy=0x7f844c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:16.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f8448a00640 1 -- 172.21.15.161:0/3876922027 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573682546 0 0) 0x7f844c0b13d0 con 0x7f844c0cd840 2024-05-22T21:14:16.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.725+0000 7f8449400640 1 -- 172.21.15.161:0/3876922027 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844c0b0d80 con 0x7f844c0d0cf0 2024-05-22T21:14:16.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.729+0000 7f8449400640 1 --2- 172.21.15.161:0/3876922027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c0d0cf0 0x7f844c0d1150 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f843c009870 tx=0x7f843c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4babe646d68bd0e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:16.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.729+0000 7f8448a00640 1 -- 172.21.15.161:0/3876922027 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c0ab600 con 0x7f844c0d0cf0 2024-05-22T21:14:16.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.729+0000 7f8448a00640 1 -- 172.21.15.161:0/3876922027 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843c0a7050 con 0x7f844c0d0cf0 2024-05-22T21:14:16.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.729+0000 7f8448a00640 1 -- 172.21.15.161:0/3876922027 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c0ab980 con 0x7f844c0d0cf0 2024-05-22T21:14:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.729+0000 7f844b200640 1 -- 172.21.15.161:0/3876922027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c0d0cf0 msgr2=0x7f844c0d1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.729+0000 7f844b200640 1 --2- 172.21.15.161:0/3876922027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c0d0cf0 0x7f844c0d1150 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f843c009870 tx=0x7f843c01c560 comp rx=0 tx=0).stop 2024-05-22T21:14:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.729+0000 7f844b200640 1 -- 172.21.15.161:0/3876922027 shutdown_connections 2024-05-22T21:14:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.729+0000 7f844b200640 1 --2- 172.21.15.161:0/3876922027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c0d0cf0 0x7f844c0d1150 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.729+0000 7f844b200640 1 --2- 172.21.15.161:0/3876922027 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f844c15d040 0x7f844c0b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.729+0000 7f844b200640 1 -- 172.21.15.161:0/3876922027 >> 172.21.15.161:0/3876922027 conn(0x7f844c099a40 msgr2=0x7f844c099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.729+0000 7f844b200640 1 -- 172.21.15.161:0/3876922027 shutdown_connections 2024-05-22T21:14:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.729+0000 7f844b200640 1 -- 172.21.15.161:0/3876922027 wait complete. 2024-05-22T21:14:16.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.729+0000 7f844b200640 1 Processor -- start 2024-05-22T21:14:16.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.729+0000 7f844b200640 1 -- start start 2024-05-22T21:14:16.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f844b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c0cd840 0x7f844c15aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f844b200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f844c0d0cf0 0x7f844c157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f844b200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f844c15d040 0x7f844c0b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f844b200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844c0b5ba0 con 0x7f844c0cd840 2024-05-22T21:14:16.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f844b200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844c159f50 con 0x7f844c0d0cf0 2024-05-22T21:14:16.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f844b200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844c15a230 con 0x7f844c15d040 2024-05-22T21:14:16.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f8449e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c0cd840 0x7f844c15aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f8449e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c0cd840 0x7f844c15aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48858/0 (socket says 172.21.15.161:48858) 2024-05-22T21:14:16.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f8449e00640 1 -- 172.21.15.161:0/2248671053 learned_addr learned my addr 172.21.15.161:0/2248671053 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:16.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f8449400640 1 --2- 172.21.15.161:0/2248671053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f844c0d0cf0 0x7f844c157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f844a800640 1 --2- 172.21.15.161:0/2248671053 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f844c15d040 0x7f844c0b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f8449e00640 1 -- 172.21.15.161:0/2248671053 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f844c15d040 msgr2=0x7f844c0b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:16.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f8449e00640 1 --2- 172.21.15.161:0/2248671053 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f844c15d040 0x7f844c0b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f8449e00640 1 -- 172.21.15.161:0/2248671053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f844c0d0cf0 msgr2=0x7f844c157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:16.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f8449e00640 1 --2- 172.21.15.161:0/2248671053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f844c0d0cf0 0x7f844c157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:16.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f8449e00640 1 -- 172.21.15.161:0/2248671053 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c0094d0 con 0x7f844c0cd840 2024-05-22T21:14:16.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f8449e00640 1 --2- 172.21.15.161:0/2248671053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c0cd840 0x7f844c15aa30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f843400d5f0 tx=0x7f843400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:16.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f8442000640 1 -- 172.21.15.161:0/2248671053 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84340125f0 con 0x7f844c0cd840 2024-05-22T21:14:16.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f8442000640 1 -- 172.21.15.161:0/2248671053 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8434012750 con 0x7f844c0cd840 2024-05-22T21:14:16.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f8442000640 1 -- 172.21.15.161:0/2248671053 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434012b00 con 0x7f844c0cd840 2024-05-22T21:14:16.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f844b200640 1 -- 172.21.15.161:0/2248671053 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f844c15a510 con 0x7f844c0cd840 2024-05-22T21:14:16.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f844b200640 1 -- 172.21.15.161:0/2248671053 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f844c159560 con 0x7f844c0cd840 2024-05-22T21:14:16.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f844b200640 1 -- 172.21.15.161:0/2248671053 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f844c067110 con 0x7f844c0cd840 2024-05-22T21:14:16.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.733+0000 7f8442000640 1 -- 172.21.15.161:0/2248671053 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8434012c60 con 0x7f844c0cd840 2024-05-22T21:14:16.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.737+0000 7f8442000640 1 --2- 172.21.15.161:0/2248671053 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84280666c0 0x7f8428068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:16.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.737+0000 7f8442000640 1 -- 172.21.15.161:0/2248671053 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8434084240 con 0x7f844c0cd840 2024-05-22T21:14:16.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.737+0000 7f8449400640 1 --2- 172.21.15.161:0/2248671053 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84280666c0 0x7f8428068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:16.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.737+0000 7f8449400640 1 --2- 172.21.15.161:0/2248671053 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84280666c0 0x7f8428068b80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f843c0ad000 tx=0x7f843c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:16.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.741+0000 7f8442000640 1 -- 172.21.15.161:0/2248671053 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f84340519a0 con 0x7f844c0cd840 2024-05-22T21:14:16.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:16.893+0000 7f844b200640 1 -- 172.21.15.161:0/2248671053 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f844c0b6d10 con 0x7f844c0cd840 2024-05-22T21:14:17.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.665+0000 7f8442000640 1 -- 172.21.15.161:0/2248671053 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8434054fc0 con 0x7f844c0cd840 2024-05-22T21:14:17.666 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' created 2024-05-22T21:14:17.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.669+0000 7f844b200640 1 -- 172.21.15.161:0/2248671053 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84280666c0 msgr2=0x7f8428068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:17.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.669+0000 7f844b200640 1 --2- 172.21.15.161:0/2248671053 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84280666c0 0x7f8428068b80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f843c0ad000 tx=0x7f843c008040 comp rx=0 tx=0).stop 2024-05-22T21:14:17.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.669+0000 7f844b200640 1 -- 172.21.15.161:0/2248671053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c0cd840 msgr2=0x7f844c15aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:17.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.669+0000 7f844b200640 1 --2- 172.21.15.161:0/2248671053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c0cd840 0x7f844c15aa30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f843400d5f0 tx=0x7f843400dab0 comp rx=0 tx=0).stop 2024-05-22T21:14:17.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.669+0000 7f844b200640 1 -- 172.21.15.161:0/2248671053 shutdown_connections 2024-05-22T21:14:17.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.669+0000 7f844b200640 1 --2- 172.21.15.161:0/2248671053 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84280666c0 0x7f8428068b80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:17.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.669+0000 7f844b200640 1 --2- 172.21.15.161:0/2248671053 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f844c15d040 0x7f844c0b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:17.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.669+0000 7f844b200640 1 --2- 172.21.15.161:0/2248671053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f844c0d0cf0 0x7f844c157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:17.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.669+0000 7f844b200640 1 --2- 172.21.15.161:0/2248671053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c0cd840 0x7f844c15aa30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:17.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.669+0000 7f844b200640 1 -- 172.21.15.161:0/2248671053 >> 172.21.15.161:0/2248671053 conn(0x7f844c099a40 msgr2=0x7f844c0aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:17.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.669+0000 7f844b200640 1 -- 172.21.15.161:0/2248671053 shutdown_connections 2024-05-22T21:14:17.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.669+0000 7f844b200640 1 -- 172.21.15.161:0/2248671053 wait complete. 2024-05-22T21:14:17.694 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-22T21:14:17.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.837+0000 7fd823000640 1 Processor -- start 2024-05-22T21:14:17.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.837+0000 7fd823000640 1 -- start start 2024-05-22T21:14:17.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd823000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81c15a110 0x7fd81c06c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:17.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd823000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd81c06c8b0 0x7fd81c15d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:17.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd823000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd81c15d5b0 0x7fd81c15da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:17.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd823000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81c06cd10 con 0x7fd81c15d5b0 2024-05-22T21:14:17.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd823000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81c0b2b90 con 0x7fd81c06c8b0 2024-05-22T21:14:17.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd823000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0b3020 con 0x7fd81c15a110 2024-05-22T21:14:17.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd822600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd81c15d5b0 0x7fd81c15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:17.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd822600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd81c15d5b0 0x7fd81c15da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48870/0 (socket says 172.21.15.161:48870) 2024-05-22T21:14:17.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd822600640 1 -- 172.21.15.161:0/1921966430 learned_addr learned my addr 172.21.15.161:0/1921966430 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:17.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd821c00640 1 --2- 172.21.15.161:0/1921966430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81c15a110 0x7fd81c06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:17.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd821200640 1 --2- 172.21.15.161:0/1921966430 >> v2:172.21.15.179:3300/0 conn(0x7fd81c06c8b0 0x7fd81c15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:17.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd822600640 1 -- 172.21.15.161:0/1921966430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81c15a110 msgr2=0x7fd81c06c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:17.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd822600640 1 --2- 172.21.15.161:0/1921966430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81c15a110 0x7fd81c06c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:17.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd822600640 1 -- 172.21.15.161:0/1921966430 >> v2:172.21.15.179:3300/0 conn(0x7fd81c06c8b0 msgr2=0x7fd81c15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:17.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd822600640 1 --2- 172.21.15.161:0/1921966430 >> v2:172.21.15.179:3300/0 conn(0x7fd81c06c8b0 0x7fd81c15d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:17.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd822600640 1 -- 172.21.15.161:0/1921966430 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c0b1f80 con 0x7fd81c15d5b0 2024-05-22T21:14:17.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.841+0000 7fd822600640 1 --2- 172.21.15.161:0/1921966430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81c15d5b0 0x7fd81c15da20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd81c06f450 tx=0x7fd81401daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10518b4171d517d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:17.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd81be00640 1 -- 172.21.15.161:0/1921966430 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81400a2d0 con 0x7fd81c15d5b0 2024-05-22T21:14:17.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd81be00640 1 -- 172.21.15.161:0/1921966430 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8140a6820 con 0x7fd81c15d5b0 2024-05-22T21:14:17.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd81be00640 1 -- 172.21.15.161:0/1921966430 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8140a6bb0 con 0x7fd81c15d5b0 2024-05-22T21:14:17.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 -- 172.21.15.161:0/1921966430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81c15d5b0 msgr2=0x7fd81c15da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:17.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 --2- 172.21.15.161:0/1921966430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81c15d5b0 0x7fd81c15da20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd81c06f450 tx=0x7fd81401daf0 comp rx=0 tx=0).stop 2024-05-22T21:14:17.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 -- 172.21.15.161:0/1921966430 shutdown_connections 2024-05-22T21:14:17.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 --2- 172.21.15.161:0/1921966430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81c15d5b0 0x7fd81c15da20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:17.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 --2- 172.21.15.161:0/1921966430 >> v2:172.21.15.179:3300/0 conn(0x7fd81c06c8b0 0x7fd81c15d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:17.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 --2- 172.21.15.161:0/1921966430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81c15a110 0x7fd81c06c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:17.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 -- 172.21.15.161:0/1921966430 >> 172.21.15.161:0/1921966430 conn(0x7fd81c0cec00 msgr2=0x7fd81c0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:17.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 -- 172.21.15.161:0/1921966430 shutdown_connections 2024-05-22T21:14:17.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 -- 172.21.15.161:0/1921966430 wait complete. 2024-05-22T21:14:17.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 Processor -- start 2024-05-22T21:14:17.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 -- start start 2024-05-22T21:14:17.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81c06c8b0 0x7fd81c154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:17.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81c15d5b0 0x7fd81c146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:17.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd81c1467b0 0x7fd81c1558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:17.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c146c40 con 0x7fd81c06c8b0 2024-05-22T21:14:17.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c156570 con 0x7fd81c1467b0 2024-05-22T21:14:17.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd823000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c156850 con 0x7fd81c15d5b0 2024-05-22T21:14:17.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd821c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81c06c8b0 0x7fd81c154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:17.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd821c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81c06c8b0 0x7fd81c154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48872/0 (socket says 172.21.15.161:48872) 2024-05-22T21:14:17.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.845+0000 7fd821c00640 1 -- 172.21.15.161:0/4257079125 learned_addr learned my addr 172.21.15.161:0/4257079125 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:17.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd821200640 1 --2- 172.21.15.161:0/4257079125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81c15d5b0 0x7fd81c146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:17.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd822600640 1 --2- 172.21.15.161:0/4257079125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd81c1467b0 0x7fd81c1558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:17.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd821c00640 1 -- 172.21.15.161:0/4257079125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81c15d5b0 msgr2=0x7fd81c146270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:17.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd821c00640 1 --2- 172.21.15.161:0/4257079125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81c15d5b0 0x7fd81c146270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:17.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd821c00640 1 -- 172.21.15.161:0/4257079125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd81c1467b0 msgr2=0x7fd81c1558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:17.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd821c00640 1 --2- 172.21.15.161:0/4257079125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd81c1467b0 0x7fd81c1558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:17.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd821c00640 1 -- 172.21.15.161:0/4257079125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81400ac40 con 0x7fd81c06c8b0 2024-05-22T21:14:17.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd821c00640 1 --2- 172.21.15.161:0/4257079125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81c06c8b0 0x7fd81c154d70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd81000d800 tx=0x7fd81000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:17.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd819600640 1 -- 172.21.15.161:0/4257079125 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81000b6c0 con 0x7fd81c06c8b0 2024-05-22T21:14:17.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd819600640 1 -- 172.21.15.161:0/4257079125 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81000b860 con 0x7fd81c06c8b0 2024-05-22T21:14:17.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd819600640 1 -- 172.21.15.161:0/4257079125 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81000bc10 con 0x7fd81c06c8b0 2024-05-22T21:14:17.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd823000640 1 -- 172.21.15.161:0/4257079125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81c156b30 con 0x7fd81c06c8b0 2024-05-22T21:14:17.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd823000640 1 -- 172.21.15.161:0/4257079125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81c1570f0 con 0x7fd81c06c8b0 2024-05-22T21:14:17.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd823000640 1 -- 172.21.15.161:0/4257079125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81c1f5bb0 con 0x7fd81c06c8b0 2024-05-22T21:14:17.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd819600640 1 -- 172.21.15.161:0/4257079125 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd81001e460 con 0x7fd81c06c8b0 2024-05-22T21:14:17.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd819600640 1 --2- 172.21.15.161:0/4257079125 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7fc066690 0x7fd7fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:17.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.849+0000 7fd819600640 1 -- 172.21.15.161:0/4257079125 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fd810085110 con 0x7fd81c06c8b0 2024-05-22T21:14:17.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.853+0000 7fd821200640 1 --2- 172.21.15.161:0/4257079125 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7fc066690 0x7fd7fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:17.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.853+0000 7fd821200640 1 --2- 172.21.15.161:0/4257079125 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7fc066690 0x7fd7fc068b50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd81c1598b0 tx=0x7fd808008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:17.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:17.853+0000 7fd819600640 1 -- 172.21.15.161:0/4257079125 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd8100525b0 con 0x7fd81c06c8b0 2024-05-22T21:14:18.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.009+0000 7fd823000640 1 -- 172.21.15.161:0/4257079125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd81c067070 con 0x7fd81c06c8b0 2024-05-22T21:14:18.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.673+0000 7fd819600640 1 -- 172.21.15.161:0/4257079125 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd810055bd0 con 0x7fd81c06c8b0 2024-05-22T21:14:18.675 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' created 2024-05-22T21:14:18.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.681+0000 7fd823000640 1 -- 172.21.15.161:0/4257079125 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7fc066690 msgr2=0x7fd7fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:18.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.681+0000 7fd823000640 1 --2- 172.21.15.161:0/4257079125 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7fc066690 0x7fd7fc068b50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd81c1598b0 tx=0x7fd808008040 comp rx=0 tx=0).stop 2024-05-22T21:14:18.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.681+0000 7fd823000640 1 -- 172.21.15.161:0/4257079125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81c06c8b0 msgr2=0x7fd81c154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:18.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.681+0000 7fd823000640 1 --2- 172.21.15.161:0/4257079125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81c06c8b0 0x7fd81c154d70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd81000d800 tx=0x7fd81000dcc0 comp rx=0 tx=0).stop 2024-05-22T21:14:18.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.685+0000 7fd823000640 1 -- 172.21.15.161:0/4257079125 shutdown_connections 2024-05-22T21:14:18.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.685+0000 7fd823000640 1 --2- 172.21.15.161:0/4257079125 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7fc066690 0x7fd7fc068b50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:18.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.685+0000 7fd823000640 1 --2- 172.21.15.161:0/4257079125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd81c1467b0 0x7fd81c1558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:18.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.685+0000 7fd823000640 1 --2- 172.21.15.161:0/4257079125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81c15d5b0 0x7fd81c146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:18.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.685+0000 7fd823000640 1 --2- 172.21.15.161:0/4257079125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81c06c8b0 0x7fd81c154d70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:18.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.685+0000 7fd823000640 1 -- 172.21.15.161:0/4257079125 >> 172.21.15.161:0/4257079125 conn(0x7fd81c0cec00 msgr2=0x7fd81c153220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:18.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.685+0000 7fd823000640 1 -- 172.21.15.161:0/4257079125 shutdown_connections 2024-05-22T21:14:18.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.685+0000 7fd823000640 1 -- 172.21.15.161:0/4257079125 wait complete. 2024-05-22T21:14:18.709 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-22T21:14:18.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.845+0000 7fda21000640 1 Processor -- start 2024-05-22T21:14:18.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.845+0000 7fda21000640 1 -- start start 2024-05-22T21:14:18.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.845+0000 7fda21000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fda1c15b600 0x7fda1c06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:18.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.845+0000 7fda21000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fda1c06c7e0 0x7fda1c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:18.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.845+0000 7fda21000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1c06cc40 con 0x7fda1c143c40 2024-05-22T21:14:18.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.845+0000 7fda21000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda1c15ba60 con 0x7fda1c15b600 2024-05-22T21:14:18.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.845+0000 7fda21000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda1c144bb0 con 0x7fda1c06c7e0 2024-05-22T21:14:18.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.845+0000 7fda19e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fda1c143c40 0x7fda1c1444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:38310/0 (socket says 172.21.15.161:38310) 2024-05-22T21:14:18.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.845+0000 7fda19e00640 1 -- 172.21.15.161:0/1345037510 learned_addr learned my addr 172.21.15.161:0/1345037510 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:18.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.845+0000 7fda11400640 1 --2- 172.21.15.161:0/1345037510 >> v2:172.21.15.161:3300/0 conn(0x7fda1c15b600 0x7fda1c06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:18.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.849+0000 7fda1a800640 1 --2- 172.21.15.161:0/1345037510 >> v2:172.21.15.179:3300/0 conn(0x7fda1c06c7e0 0x7fda1c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:18.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.849+0000 7fda11400640 1 -- 172.21.15.161:0/1345037510 >> v2:172.21.15.179:3300/0 conn(0x7fda1c06c7e0 msgr2=0x7fda1c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:18.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.849+0000 7fda11400640 1 --2- 172.21.15.161:0/1345037510 >> v2:172.21.15.179:3300/0 conn(0x7fda1c06c7e0 0x7fda1c0b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:18.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.849+0000 7fda11400640 1 -- 172.21.15.161:0/1345037510 >> v1:172.21.15.161:6789/0 conn(0x7fda1c143c40 legacy=0x7fda1c1444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:18.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.849+0000 7fda11400640 1 -- 172.21.15.161:0/1345037510 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c15efb0 con 0x7fda1c15b600 2024-05-22T21:14:18.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.849+0000 7fda19400640 1 -- 172.21.15.161:0/1345037510 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945173428 0 0) 0x7fda1c06cc40 con 0x7fda1c143c40 2024-05-22T21:14:18.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.849+0000 7fda11400640 1 --2- 172.21.15.161:0/1345037510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c15b600 0x7fda1c06c2a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0aead0 tx=0x7fda0401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de13cbb0d2f7e873 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:18.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.849+0000 7fda19400640 1 -- 172.21.15.161:0/1345037510 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda040ab680 con 0x7fda1c15b600 2024-05-22T21:14:18.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.849+0000 7fda19400640 1 -- 172.21.15.161:0/1345037510 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda040ab7e0 con 0x7fda1c15b600 2024-05-22T21:14:18.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.849+0000 7fda19400640 1 -- 172.21.15.161:0/1345037510 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda040abb70 con 0x7fda1c15b600 2024-05-22T21:14:18.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda21000640 1 -- 172.21.15.161:0/1345037510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c15b600 msgr2=0x7fda1c06c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:18.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda21000640 1 --2- 172.21.15.161:0/1345037510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c15b600 0x7fda1c06c2a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0aead0 tx=0x7fda0401c480 comp rx=0 tx=0).stop 2024-05-22T21:14:18.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda21000640 1 -- 172.21.15.161:0/1345037510 shutdown_connections 2024-05-22T21:14:18.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda21000640 1 --2- 172.21.15.161:0/1345037510 >> v2:172.21.15.179:3300/0 conn(0x7fda1c06c7e0 0x7fda1c0b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:18.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda21000640 1 --2- 172.21.15.161:0/1345037510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c15b600 0x7fda1c06c2a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:18.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda21000640 1 -- 172.21.15.161:0/1345037510 >> 172.21.15.161:0/1345037510 conn(0x7fda1c06a2d0 msgr2=0x7fda1c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:18.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda21000640 1 -- 172.21.15.161:0/1345037510 shutdown_connections 2024-05-22T21:14:18.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda21000640 1 -- 172.21.15.161:0/1345037510 wait complete. 2024-05-22T21:14:18.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda21000640 1 Processor -- start 2024-05-22T21:14:18.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda21000640 1 -- start start 2024-05-22T21:14:18.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda21000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda1c06c7e0 0x7fda1c13a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:18.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda21000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda1c143c40 0x7fda1c15e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:18.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda21000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c15b600 0x7fda1c1393f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:18.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda21000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c15eaa0 con 0x7fda1c15b600 2024-05-22T21:14:18.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda21000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c15c4c0 con 0x7fda1c06c7e0 2024-05-22T21:14:18.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda21000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c13ed70 con 0x7fda1c143c40 2024-05-22T21:14:18.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda1a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c15b600 0x7fda1c1393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:18.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda1a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c15b600 0x7fda1c1393f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48904/0 (socket says 172.21.15.161:48904) 2024-05-22T21:14:18.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda1a800640 1 -- 172.21.15.161:0/2006924987 learned_addr learned my addr 172.21.15.161:0/2006924987 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:18.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda19e00640 1 --2- 172.21.15.161:0/2006924987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda1c06c7e0 0x7fda1c13a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:18.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda11400640 1 --2- 172.21.15.161:0/2006924987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda1c143c40 0x7fda1c15e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:18.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda1a800640 1 -- 172.21.15.161:0/2006924987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda1c143c40 msgr2=0x7fda1c15e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:18.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda1a800640 1 --2- 172.21.15.161:0/2006924987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda1c143c40 0x7fda1c15e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:18.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda1a800640 1 -- 172.21.15.161:0/2006924987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda1c06c7e0 msgr2=0x7fda1c13a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:18.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda1a800640 1 --2- 172.21.15.161:0/2006924987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda1c06c7e0 0x7fda1c13a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:18.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda1a800640 1 -- 172.21.15.161:0/2006924987 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda04009420 con 0x7fda1c15b600 2024-05-22T21:14:18.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.853+0000 7fda11400640 1 --2- 172.21.15.161:0/2006924987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda1c143c40 0x7fda1c15e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:18.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.857+0000 7fda1a800640 1 --2- 172.21.15.161:0/2006924987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c15b600 0x7fda1c1393f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fda14007e60 tx=0x7fda1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:18.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.857+0000 7fda12a00640 1 -- 172.21.15.161:0/2006924987 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14014420 con 0x7fda1c15b600 2024-05-22T21:14:18.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.857+0000 7fda12a00640 1 -- 172.21.15.161:0/2006924987 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda140145a0 con 0x7fda1c15b600 2024-05-22T21:14:18.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.857+0000 7fda12a00640 1 -- 172.21.15.161:0/2006924987 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14014950 con 0x7fda1c15b600 2024-05-22T21:14:18.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.857+0000 7fda21000640 1 -- 172.21.15.161:0/2006924987 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1c13f050 con 0x7fda1c15b600 2024-05-22T21:14:18.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.857+0000 7fda21000640 1 -- 172.21.15.161:0/2006924987 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1c13f610 con 0x7fda1c15b600 2024-05-22T21:14:18.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.857+0000 7fda21000640 1 -- 172.21.15.161:0/2006924987 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1c15c650 con 0x7fda1c15b600 2024-05-22T21:14:18.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.857+0000 7fda12a00640 1 -- 172.21.15.161:0/2006924987 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fda14014ab0 con 0x7fda1c15b600 2024-05-22T21:14:18.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.857+0000 7fda12a00640 1 --2- 172.21.15.161:0/2006924987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd9f8066690 0x7fd9f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:18.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.857+0000 7fda12a00640 1 -- 172.21.15.161:0/2006924987 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fda14084aa0 con 0x7fda1c15b600 2024-05-22T21:14:18.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.857+0000 7fda19e00640 1 --2- 172.21.15.161:0/2006924987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd9f8066690 0x7fd9f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:18.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.861+0000 7fda19e00640 1 --2- 172.21.15.161:0/2006924987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd9f8066690 0x7fd9f8068b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fda0c000f70 tx=0x7fda0c007b60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:18.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:18.861+0000 7fda12a00640 1 -- 172.21.15.161:0/2006924987 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fda14051e60 con 0x7fda1c15b600 2024-05-22T21:14:19.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.049+0000 7fda21000640 1 -- 172.21.15.161:0/2006924987 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fda1c067220 con 0x7fda1c15b600 2024-05-22T21:14:19.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.057+0000 7fda12a00640 1 -- 172.21.15.161:0/2006924987 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7fda14055480 con 0x7fda1c15b600 2024-05-22T21:14:19.067 INFO:teuthology.orchestra.run.smithi161.stderr:new fs with metadata pool 4 and data pool 5 2024-05-22T21:14:19.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.069+0000 7fda21000640 1 -- 172.21.15.161:0/2006924987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd9f8066690 msgr2=0x7fd9f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.069+0000 7fda21000640 1 --2- 172.21.15.161:0/2006924987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd9f8066690 0x7fd9f8068b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fda0c000f70 tx=0x7fda0c007b60 comp rx=0 tx=0).stop 2024-05-22T21:14:19.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.069+0000 7fda21000640 1 -- 172.21.15.161:0/2006924987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c15b600 msgr2=0x7fda1c1393f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.069+0000 7fda21000640 1 --2- 172.21.15.161:0/2006924987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c15b600 0x7fda1c1393f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fda14007e60 tx=0x7fda1400e4b0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.073+0000 7fda21000640 1 -- 172.21.15.161:0/2006924987 shutdown_connections 2024-05-22T21:14:19.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.073+0000 7fda21000640 1 --2- 172.21.15.161:0/2006924987 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd9f8066690 0x7fd9f8068b50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.073+0000 7fda21000640 1 --2- 172.21.15.161:0/2006924987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c15b600 0x7fda1c1393f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.073+0000 7fda21000640 1 --2- 172.21.15.161:0/2006924987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda1c143c40 0x7fda1c15e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.073+0000 7fda21000640 1 --2- 172.21.15.161:0/2006924987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda1c06c7e0 0x7fda1c13a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.073+0000 7fda21000640 1 -- 172.21.15.161:0/2006924987 >> 172.21.15.161:0/2006924987 conn(0x7fda1c06a2d0 msgr2=0x7fda1c06a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:19.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.073+0000 7fda21000640 1 -- 172.21.15.161:0/2006924987 shutdown_connections 2024-05-22T21:14:19.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.073+0000 7fda21000640 1 -- 172.21.15.161:0/2006924987 wait complete. 2024-05-22T21:14:19.097 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:19.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.233+0000 7f8fca000640 1 Processor -- start 2024-05-22T21:14:19.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.233+0000 7f8fca000640 1 -- start start 2024-05-22T21:14:19.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.233+0000 7f8fca000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8fc4099ca0 0x7f8fc406eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.233+0000 7f8fca000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8fc40b3560 0x7f8fc409a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.233+0000 7f8fca000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8fc40cf0e0 0x7f8fc40cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.233+0000 7f8fca000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc40c7ea0 con 0x7f8fc4099ca0 2024-05-22T21:14:19.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.233+0000 7f8fca000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc406df70 con 0x7f8fc40cf0e0 2024-05-22T21:14:19.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.233+0000 7f8fca000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc4161c60 con 0x7f8fc40b3560 2024-05-22T21:14:19.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.233+0000 7f8fc3400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8fc4099ca0 0x7f8fc406eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:19.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.233+0000 7f8fc3400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8fc4099ca0 0x7f8fc406eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48908/0 (socket says 172.21.15.161:48908) 2024-05-22T21:14:19.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.233+0000 7f8fc3400640 1 -- 172.21.15.161:0/1210912985 learned_addr learned my addr 172.21.15.161:0/1210912985 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:19.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.233+0000 7f8fc3400640 1 -- 172.21.15.161:0/1210912985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8fc40b3560 msgr2=0x7f8fc409a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:19.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.233+0000 7f8fc3400640 1 --2- 172.21.15.161:0/1210912985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8fc40b3560 0x7f8fc409a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.233+0000 7f8fc3400640 1 -- 172.21.15.161:0/1210912985 >> v2:172.21.15.179:3300/0 conn(0x7f8fc40cf0e0 msgr2=0x7f8fc40cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:19.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.233+0000 7f8fc3400640 1 --2- 172.21.15.161:0/1210912985 >> v2:172.21.15.179:3300/0 conn(0x7f8fc40cf0e0 0x7f8fc40cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.233+0000 7f8fc3400640 1 -- 172.21.15.161:0/1210912985 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc406e1c0 con 0x7f8fc4099ca0 2024-05-22T21:14:19.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.237+0000 7f8fc3400640 1 --2- 172.21.15.161:0/1210912985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc4099ca0 0x7f8fc406eca0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8fc406d9f0 tx=0x7f8fb401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f08dba7a1129a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:19.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.237+0000 7f8fc2000640 1 -- 172.21.15.161:0/1210912985 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40a5840 con 0x7f8fc4099ca0 2024-05-22T21:14:19.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.237+0000 7f8fc2000640 1 -- 172.21.15.161:0/1210912985 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb40a59a0 con 0x7f8fc4099ca0 2024-05-22T21:14:19.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.237+0000 7f8fc2000640 1 -- 172.21.15.161:0/1210912985 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40a5d30 con 0x7f8fc4099ca0 2024-05-22T21:14:19.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.237+0000 7f8fca000640 1 -- 172.21.15.161:0/1210912985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc4099ca0 msgr2=0x7f8fc406eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.237+0000 7f8fca000640 1 --2- 172.21.15.161:0/1210912985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc4099ca0 0x7f8fc406eca0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8fc406d9f0 tx=0x7f8fb401c9c0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.237+0000 7f8fca000640 1 -- 172.21.15.161:0/1210912985 shutdown_connections 2024-05-22T21:14:19.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.237+0000 7f8fca000640 1 --2- 172.21.15.161:0/1210912985 >> v2:172.21.15.179:3300/0 conn(0x7f8fc40cf0e0 0x7f8fc40cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.237+0000 7f8fca000640 1 --2- 172.21.15.161:0/1210912985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8fc40b3560 0x7f8fc409a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.237+0000 7f8fca000640 1 --2- 172.21.15.161:0/1210912985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc4099ca0 0x7f8fc406eca0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.237+0000 7f8fca000640 1 -- 172.21.15.161:0/1210912985 >> 172.21.15.161:0/1210912985 conn(0x7f8fc40b2a40 msgr2=0x7f8fc40b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:19.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.237+0000 7f8fca000640 1 -- 172.21.15.161:0/1210912985 shutdown_connections 2024-05-22T21:14:19.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.237+0000 7f8fca000640 1 -- 172.21.15.161:0/1210912985 wait complete. 2024-05-22T21:14:19.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fca000640 1 Processor -- start 2024-05-22T21:14:19.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fca000640 1 -- start start 2024-05-22T21:14:19.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fca000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc4099ca0 0x7f8fc40cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fca000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8fc40b3560 0x7f8fc4153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fca000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fc40cf0e0 0x7f8fc40c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fca000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc4154900 con 0x7f8fc4099ca0 2024-05-22T21:14:19.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fca000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc4154be0 con 0x7f8fc40cf0e0 2024-05-22T21:14:19.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fca000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc4154ec0 con 0x7f8fc40b3560 2024-05-22T21:14:19.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fc3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc4099ca0 0x7f8fc40cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:19.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fc3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc4099ca0 0x7f8fc40cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48922/0 (socket says 172.21.15.161:48922) 2024-05-22T21:14:19.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fc3400640 1 -- 172.21.15.161:0/2805112266 learned_addr learned my addr 172.21.15.161:0/2805112266 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:19.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fc3400640 1 -- 172.21.15.161:0/2805112266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8fc40b3560 msgr2=0x7f8fc4153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fc3400640 1 --2- 172.21.15.161:0/2805112266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8fc40b3560 0x7f8fc4153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fc3400640 1 -- 172.21.15.161:0/2805112266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fc40cf0e0 msgr2=0x7f8fc40c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fc3400640 1 --2- 172.21.15.161:0/2805112266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fc40cf0e0 0x7f8fc40c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fc3400640 1 -- 172.21.15.161:0/2805112266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb4009b30 con 0x7f8fc4099ca0 2024-05-22T21:14:19.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fc3400640 1 --2- 172.21.15.161:0/2805112266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc4099ca0 0x7f8fc40cb560 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40af040 tx=0x7f8fb4019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:19.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fbb400640 1 -- 172.21.15.161:0/2805112266 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40b8670 con 0x7f8fc4099ca0 2024-05-22T21:14:19.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fbb400640 1 -- 172.21.15.161:0/2805112266 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb40b87f0 con 0x7f8fc4099ca0 2024-05-22T21:14:19.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fbb400640 1 -- 172.21.15.161:0/2805112266 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40b8ba0 con 0x7f8fc4099ca0 2024-05-22T21:14:19.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fca000640 1 -- 172.21.15.161:0/2805112266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc40c9a40 con 0x7f8fc4099ca0 2024-05-22T21:14:19.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fca000640 1 -- 172.21.15.161:0/2805112266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc41553b0 con 0x7f8fc4099ca0 2024-05-22T21:14:19.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.241+0000 7f8fca000640 1 -- 172.21.15.161:0/2805112266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc40c98a0 con 0x7f8fc4099ca0 2024-05-22T21:14:19.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.245+0000 7f8fbb400640 1 -- 172.21.15.161:0/2805112266 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8fb40b8d00 con 0x7f8fc4099ca0 2024-05-22T21:14:19.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.245+0000 7f8fbb400640 1 --2- 172.21.15.161:0/2805112266 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8fa4066690 0x7f8fa4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.245+0000 7f8fbb400640 1 -- 172.21.15.161:0/2805112266 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fb411fad0 con 0x7f8fc4099ca0 2024-05-22T21:14:19.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.245+0000 7f8fc2a00640 1 --2- 172.21.15.161:0/2805112266 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8fa4066690 0x7f8fa4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:19.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.245+0000 7f8fc2a00640 1 --2- 172.21.15.161:0/2805112266 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8fa4066690 0x7f8fa4068b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40c6540 tx=0x7f8fb0006c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:19.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.249+0000 7f8fbb400640 1 -- 172.21.15.161:0/2805112266 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8fc40c98a0 con 0x7f8fc4099ca0 2024-05-22T21:14:19.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.401+0000 7f8fca000640 1 -- 172.21.15.161:0/2805112266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fc40c98a0 con 0x7f8fc4099ca0 2024-05-22T21:14:19.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.405+0000 7f8fbb400640 1 -- 172.21.15.161:0/2805112266 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16162 (secure 0 0 0) 0x7f8fc40c98a0 con 0x7f8fc4099ca0 2024-05-22T21:14:19.406 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:19.406 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":32,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:14:19.052912+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:16.938769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:18.054579+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:19.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.405+0000 7f8fca000640 1 -- 172.21.15.161:0/2805112266 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8fa4066690 msgr2=0x7f8fa4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.405+0000 7f8fca000640 1 --2- 172.21.15.161:0/2805112266 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8fa4066690 0x7f8fa4068b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40c6540 tx=0x7f8fb0006c00 comp rx=0 tx=0).stop 2024-05-22T21:14:19.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.405+0000 7f8fca000640 1 -- 172.21.15.161:0/2805112266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc4099ca0 msgr2=0x7f8fc40cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.405+0000 7f8fca000640 1 --2- 172.21.15.161:0/2805112266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc4099ca0 0x7f8fc40cb560 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40af040 tx=0x7f8fb4019760 comp rx=0 tx=0).stop 2024-05-22T21:14:19.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.405+0000 7f8fca000640 1 -- 172.21.15.161:0/2805112266 shutdown_connections 2024-05-22T21:14:19.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.405+0000 7f8fca000640 1 --2- 172.21.15.161:0/2805112266 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8fa4066690 0x7f8fa4068b50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.405+0000 7f8fca000640 1 --2- 172.21.15.161:0/2805112266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fc40cf0e0 0x7f8fc40c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.405+0000 7f8fca000640 1 --2- 172.21.15.161:0/2805112266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8fc40b3560 0x7f8fc4153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.405+0000 7f8fca000640 1 --2- 172.21.15.161:0/2805112266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fc4099ca0 0x7f8fc40cb560 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.405+0000 7f8fca000640 1 -- 172.21.15.161:0/2805112266 >> 172.21.15.161:0/2805112266 conn(0x7f8fc40b2a40 msgr2=0x7f8fc40b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:19.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.405+0000 7f8fca000640 1 -- 172.21.15.161:0/2805112266 shutdown_connections 2024-05-22T21:14:19.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.405+0000 7f8fca000640 1 -- 172.21.15.161:0/2805112266 wait complete. 2024-05-22T21:14:19.426 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:19.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c41600640 1 Processor -- start 2024-05-22T21:14:19.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c41600640 1 -- start start 2024-05-22T21:14:19.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c41600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9c3c15dd50 0x7f9c3c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c41600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c3c15d5e0 0x7f9c3c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c41600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3c071670 con 0x7f9c3c15e130 2024-05-22T21:14:19.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c41600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c06de40 con 0x7f9c3c15dd50 2024-05-22T21:14:19.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c41600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c070a60 con 0x7f9c3c15d5e0 2024-05-22T21:14:19.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c3b400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c3c15d5e0 0x7f9c3c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:19.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c3aa00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9c3c15dd50 0x7f9c3c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:19.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c32000640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f9c3c15e130 0x7f9c3c15ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:50078/0 (socket says 172.21.15.161:50078) 2024-05-22T21:14:19.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c32000640 1 -- 172.21.15.161:0/3521097282 learned_addr learned my addr 172.21.15.161:0/3521097282 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:19.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c3a000640 1 -- 172.21.15.161:0/3521097282 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188296772 0 0) 0x7f9c3c071670 con 0x7f9c3c15e130 2024-05-22T21:14:19.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c3a000640 1 -- 172.21.15.161:0/3521097282 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c20003530 con 0x7f9c3c15e130 2024-05-22T21:14:19.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c3b400640 1 -- 172.21.15.161:0/3521097282 >> v2:172.21.15.179:3300/0 conn(0x7f9c3c15dd50 msgr2=0x7f9c3c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:19.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c3b400640 1 --2- 172.21.15.161:0/3521097282 >> v2:172.21.15.179:3300/0 conn(0x7f9c3c15dd50 0x7f9c3c06d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c3b400640 1 -- 172.21.15.161:0/3521097282 >> v1:172.21.15.179:6789/0 conn(0x7f9c3c15e130 legacy=0x7f9c3c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.561+0000 7f9c3b400640 1 -- 172.21.15.161:0/3521097282 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c070d40 con 0x7f9c3c15d5e0 2024-05-22T21:14:19.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.565+0000 7f9c3b400640 1 --2- 172.21.15.161:0/3521097282 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c3c15d5e0 0x7f9c3c15b6c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9c340097c0 tx=0x7f9c3401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eedec5210b59d43 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:19.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.565+0000 7f9c3a000640 1 -- 172.21.15.161:0/3521097282 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c340ab670 con 0x7f9c3c15d5e0 2024-05-22T21:14:19.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.565+0000 7f9c3a000640 1 -- 172.21.15.161:0/3521097282 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c340ab7d0 con 0x7f9c3c15d5e0 2024-05-22T21:14:19.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.565+0000 7f9c3a000640 1 -- 172.21.15.161:0/3521097282 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c340abb40 con 0x7f9c3c15d5e0 2024-05-22T21:14:19.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.565+0000 7f9c41600640 1 -- 172.21.15.161:0/3521097282 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c3c15d5e0 msgr2=0x7f9c3c15b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.565+0000 7f9c41600640 1 --2- 172.21.15.161:0/3521097282 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c3c15d5e0 0x7f9c3c15b6c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9c340097c0 tx=0x7f9c3401c470 comp rx=0 tx=0).stop 2024-05-22T21:14:19.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.565+0000 7f9c41600640 1 -- 172.21.15.161:0/3521097282 shutdown_connections 2024-05-22T21:14:19.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.565+0000 7f9c41600640 1 --2- 172.21.15.161:0/3521097282 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c3c15d5e0 0x7f9c3c15b6c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.565+0000 7f9c41600640 1 --2- 172.21.15.161:0/3521097282 >> v2:172.21.15.179:3300/0 conn(0x7f9c3c15dd50 0x7f9c3c06d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.565+0000 7f9c41600640 1 -- 172.21.15.161:0/3521097282 >> 172.21.15.161:0/3521097282 conn(0x7f9c3c0b0bc0 msgr2=0x7f9c3c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:19.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.565+0000 7f9c41600640 1 -- 172.21.15.161:0/3521097282 shutdown_connections 2024-05-22T21:14:19.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.565+0000 7f9c41600640 1 -- 172.21.15.161:0/3521097282 wait complete. 2024-05-22T21:14:19.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c41600640 1 Processor -- start 2024-05-22T21:14:19.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c41600640 1 -- start start 2024-05-22T21:14:19.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c41600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c3c15d5e0 0x7f9c3c135870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c41600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c3c15dd50 0x7f9c3c1495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c41600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c3c15e130 0x7f9c3c0cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c41600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c14ad10 con 0x7f9c3c15dd50 2024-05-22T21:14:19.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c41600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c14a090 con 0x7f9c3c15e130 2024-05-22T21:14:19.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c41600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c14a370 con 0x7f9c3c15d5e0 2024-05-22T21:14:19.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c32000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c3c15dd50 0x7f9c3c1495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:19.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c32000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c3c15dd50 0x7f9c3c1495d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48932/0 (socket says 172.21.15.161:48932) 2024-05-22T21:14:19.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c32000640 1 -- 172.21.15.161:0/2053914597 learned_addr learned my addr 172.21.15.161:0/2053914597 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:19.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c3aa00640 1 --2- 172.21.15.161:0/2053914597 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c3c15d5e0 0x7f9c3c135870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:19.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c3b400640 1 --2- 172.21.15.161:0/2053914597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c3c15e130 0x7f9c3c0cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:19.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c32000640 1 -- 172.21.15.161:0/2053914597 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c3c15d5e0 msgr2=0x7f9c3c135870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c32000640 1 --2- 172.21.15.161:0/2053914597 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c3c15d5e0 0x7f9c3c135870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c32000640 1 -- 172.21.15.161:0/2053914597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c3c15e130 msgr2=0x7f9c3c0cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c32000640 1 --2- 172.21.15.161:0/2053914597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c3c15e130 0x7f9c3c0cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c32000640 1 -- 172.21.15.161:0/2053914597 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c34009420 con 0x7f9c3c15dd50 2024-05-22T21:14:19.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c32000640 1 --2- 172.21.15.161:0/2053914597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c3c15dd50 0x7f9c3c1495d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400c3c0 tx=0x7f9c2400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:19.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c33400640 1 -- 172.21.15.161:0/2053914597 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24005cf0 con 0x7f9c3c15dd50 2024-05-22T21:14:19.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c33400640 1 -- 172.21.15.161:0/2053914597 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c24005e50 con 0x7f9c3c15dd50 2024-05-22T21:14:19.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c33400640 1 -- 172.21.15.161:0/2053914597 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c240072f0 con 0x7f9c3c15dd50 2024-05-22T21:14:19.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c41600640 1 -- 172.21.15.161:0/2053914597 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c14a650 con 0x7f9c3c15dd50 2024-05-22T21:14:19.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.569+0000 7f9c41600640 1 -- 172.21.15.161:0/2053914597 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c15b3f0 con 0x7f9c3c15dd50 2024-05-22T21:14:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.573+0000 7f9c41600640 1 -- 172.21.15.161:0/2053914597 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c14ae50 con 0x7f9c3c15dd50 2024-05-22T21:14:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.573+0000 7f9c33400640 1 -- 172.21.15.161:0/2053914597 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9c24007450 con 0x7f9c3c15dd50 2024-05-22T21:14:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.573+0000 7f9c33400640 1 --2- 172.21.15.161:0/2053914597 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9c18066690 0x7f9c18068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.573+0000 7f9c33400640 1 -- 172.21.15.161:0/2053914597 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c240835f0 con 0x7f9c3c15dd50 2024-05-22T21:14:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.573+0000 7f9c3aa00640 1 --2- 172.21.15.161:0/2053914597 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9c18066690 0x7f9c18068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.573+0000 7f9c3aa00640 1 --2- 172.21.15.161:0/2053914597 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9c18066690 0x7f9c18068b50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0113e0 tx=0x7f9c2c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:19.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.577+0000 7f9c33400640 1 -- 172.21.15.161:0/2053914597 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9c3c14ae50 con 0x7f9c3c15dd50 2024-05-22T21:14:19.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.733+0000 7f9c41600640 1 -- 172.21.15.161:0/2053914597 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c3c14ae50 con 0x7f9c3c15dd50 2024-05-22T21:14:19.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.733+0000 7f9c33400640 1 -- 172.21.15.161:0/2053914597 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16162 (secure 0 0 0) 0x7f9c3c14ae50 con 0x7f9c3c15dd50 2024-05-22T21:14:19.736 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:19.736 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":32,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:14:19.052912+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:16.938769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:18.054579+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:19.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.737+0000 7f9c41600640 1 -- 172.21.15.161:0/2053914597 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9c18066690 msgr2=0x7f9c18068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.737+0000 7f9c41600640 1 --2- 172.21.15.161:0/2053914597 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9c18066690 0x7f9c18068b50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0113e0 tx=0x7f9c2c00b040 comp rx=0 tx=0).stop 2024-05-22T21:14:19.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.737+0000 7f9c41600640 1 -- 172.21.15.161:0/2053914597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c3c15dd50 msgr2=0x7f9c3c1495d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.737+0000 7f9c41600640 1 --2- 172.21.15.161:0/2053914597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c3c15dd50 0x7f9c3c1495d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400c3c0 tx=0x7f9c2400c880 comp rx=0 tx=0).stop 2024-05-22T21:14:19.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.737+0000 7f9c41600640 1 -- 172.21.15.161:0/2053914597 shutdown_connections 2024-05-22T21:14:19.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.737+0000 7f9c41600640 1 --2- 172.21.15.161:0/2053914597 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9c18066690 0x7f9c18068b50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.737+0000 7f9c41600640 1 --2- 172.21.15.161:0/2053914597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c3c15e130 0x7f9c3c0cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.737+0000 7f9c41600640 1 --2- 172.21.15.161:0/2053914597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c3c15dd50 0x7f9c3c1495d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.737+0000 7f9c41600640 1 --2- 172.21.15.161:0/2053914597 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c3c15d5e0 0x7f9c3c135870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.737+0000 7f9c41600640 1 -- 172.21.15.161:0/2053914597 >> 172.21.15.161:0/2053914597 conn(0x7f9c3c0b0bc0 msgr2=0x7f9c3c0b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:19.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.737+0000 7f9c41600640 1 -- 172.21.15.161:0/2053914597 shutdown_connections 2024-05-22T21:14:19.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.737+0000 7f9c41600640 1 -- 172.21.15.161:0/2053914597 wait complete. 2024-05-22T21:14:19.758 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-22T21:14:19.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.897+0000 7fa1f0000640 1 Processor -- start 2024-05-22T21:14:19.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.897+0000 7fa1f0000640 1 -- start start 2024-05-22T21:14:19.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.897+0000 7fa1f0000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa1e81402b0 0x7fa1e8070590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.897+0000 7fa1f0000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1e806c750 0x7fa1e80716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.897+0000 7fa1f0000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e806cbc0 con 0x7fa1e806c2a0 2024-05-22T21:14:19.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.897+0000 7fa1f0000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e815d7c0 con 0x7fa1e81402b0 2024-05-22T21:14:19.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.897+0000 7fa1f0000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e815d9d0 con 0x7fa1e806c750 2024-05-22T21:14:19.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.897+0000 7fa1ed600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa1e81402b0 0x7fa1e8070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:19.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.897+0000 7fa1ed600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa1e81402b0 0x7fa1e8070590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48946/0 (socket says 172.21.15.161:48946) 2024-05-22T21:14:19.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.897+0000 7fa1ed600640 1 -- 172.21.15.161:0/1799668193 learned_addr learned my addr 172.21.15.161:0/1799668193 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:19.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.897+0000 7fa1ee000640 1 --2- 172.21.15.161:0/1799668193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1e806c750 0x7fa1e80716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:19.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.897+0000 7fa1ed600640 1 -- 172.21.15.161:0/1799668193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1e806c750 msgr2=0x7fa1e80716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:19.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.897+0000 7fa1ed600640 1 --2- 172.21.15.161:0/1799668193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1e806c750 0x7fa1e80716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.897+0000 7fa1ed600640 1 -- 172.21.15.161:0/1799668193 >> v1:172.21.15.161:6789/0 conn(0x7fa1e806c2a0 legacy=0x7fa1e8070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.897+0000 7fa1ed600640 1 -- 172.21.15.161:0/1799668193 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e815dcb0 con 0x7fa1e81402b0 2024-05-22T21:14:19.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.897+0000 7fa1e7e00640 1 -- 172.21.15.161:0/1799668193 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275508646 0 0) 0x7fa1e806cbc0 con 0x7fa1e806c2a0 2024-05-22T21:14:19.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.901+0000 7fa1ed600640 1 --2- 172.21.15.161:0/1799668193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e81402b0 0x7fa1e8070590 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009870 tx=0x7fa1dc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2677c497e83946b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:19.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.901+0000 7fa1e7e00640 1 -- 172.21.15.161:0/1799668193 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0a9620 con 0x7fa1e81402b0 2024-05-22T21:14:19.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.901+0000 7fa1e7e00640 1 -- 172.21.15.161:0/1799668193 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc0a9780 con 0x7fa1e81402b0 2024-05-22T21:14:19.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.901+0000 7fa1e7e00640 1 -- 172.21.15.161:0/1799668193 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0a9b10 con 0x7fa1e81402b0 2024-05-22T21:14:19.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.901+0000 7fa1f0000640 1 -- 172.21.15.161:0/1799668193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e81402b0 msgr2=0x7fa1e8070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.901+0000 7fa1f0000640 1 --2- 172.21.15.161:0/1799668193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e81402b0 0x7fa1e8070590 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009870 tx=0x7fa1dc01c560 comp rx=0 tx=0).stop 2024-05-22T21:14:19.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.901+0000 7fa1f0000640 1 -- 172.21.15.161:0/1799668193 shutdown_connections 2024-05-22T21:14:19.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.901+0000 7fa1f0000640 1 --2- 172.21.15.161:0/1799668193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1e806c750 0x7fa1e80716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.901+0000 7fa1f0000640 1 --2- 172.21.15.161:0/1799668193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e81402b0 0x7fa1e8070590 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.901+0000 7fa1f0000640 1 -- 172.21.15.161:0/1799668193 >> 172.21.15.161:0/1799668193 conn(0x7fa1e806a2d0 msgr2=0x7fa1e806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:19.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.901+0000 7fa1f0000640 1 -- 172.21.15.161:0/1799668193 shutdown_connections 2024-05-22T21:14:19.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.901+0000 7fa1f0000640 1 -- 172.21.15.161:0/1799668193 wait complete. 2024-05-22T21:14:19.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.901+0000 7fa1f0000640 1 Processor -- start 2024-05-22T21:14:19.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.901+0000 7fa1f0000640 1 -- start start 2024-05-22T21:14:19.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1f0000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e806c2a0 0x7fa1e813dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1f0000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1e806c750 0x7fa1e80ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1f0000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1e81402b0 0x7fa1e8154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1f0000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e813e310 con 0x7fa1e806c2a0 2024-05-22T21:14:19.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1f0000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e813e5f0 con 0x7fa1e806c750 2024-05-22T21:14:19.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1f0000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e813a900 con 0x7fa1e81402b0 2024-05-22T21:14:19.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1ed600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e806c2a0 0x7fa1e813dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:19.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1ed600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e806c2a0 0x7fa1e813dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48960/0 (socket says 172.21.15.161:48960) 2024-05-22T21:14:19.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1ed600640 1 -- 172.21.15.161:0/2705539537 learned_addr learned my addr 172.21.15.161:0/2705539537 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:19.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1ee000640 1 --2- 172.21.15.161:0/2705539537 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1e81402b0 0x7fa1e8154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:19.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1ecc00640 1 --2- 172.21.15.161:0/2705539537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1e806c750 0x7fa1e80ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:19.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1ed600640 1 -- 172.21.15.161:0/2705539537 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1e81402b0 msgr2=0x7fa1e8154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1ed600640 1 --2- 172.21.15.161:0/2705539537 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1e81402b0 0x7fa1e8154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1ed600640 1 -- 172.21.15.161:0/2705539537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1e806c750 msgr2=0x7fa1e80ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:19.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1ed600640 1 --2- 172.21.15.161:0/2705539537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1e806c750 0x7fa1e80ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:19.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1ed600640 1 -- 172.21.15.161:0/2705539537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc0094d0 con 0x7fa1e806c2a0 2024-05-22T21:14:19.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1ed600640 1 --2- 172.21.15.161:0/2705539537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e806c2a0 0x7fa1e813dd30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc01ac10 tx=0x7fa1dc019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:19.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1e5600640 1 -- 172.21.15.161:0/2705539537 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0bb3f0 con 0x7fa1e806c2a0 2024-05-22T21:14:19.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1e5600640 1 -- 172.21.15.161:0/2705539537 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc0bb570 con 0x7fa1e806c2a0 2024-05-22T21:14:19.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1e5600640 1 -- 172.21.15.161:0/2705539537 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0bb920 con 0x7fa1e806c2a0 2024-05-22T21:14:19.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1f0000640 1 -- 172.21.15.161:0/2705539537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e813abe0 con 0x7fa1e806c2a0 2024-05-22T21:14:19.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1f0000640 1 -- 172.21.15.161:0/2705539537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e813a5d0 con 0x7fa1e806c2a0 2024-05-22T21:14:19.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.905+0000 7fa1f0000640 1 -- 172.21.15.161:0/2705539537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e8067220 con 0x7fa1e806c2a0 2024-05-22T21:14:19.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.909+0000 7fa1e5600640 1 -- 172.21.15.161:0/2705539537 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa1dc0b13f0 con 0x7fa1e806c2a0 2024-05-22T21:14:19.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.909+0000 7fa1e5600640 1 --2- 172.21.15.161:0/2705539537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1cc066690 0x7fa1cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:19.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.909+0000 7fa1e5600640 1 -- 172.21.15.161:0/2705539537 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1dc11e340 con 0x7fa1e806c2a0 2024-05-22T21:14:19.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.909+0000 7fa1ecc00640 1 --2- 172.21.15.161:0/2705539537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1cc066690 0x7fa1cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:19.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.909+0000 7fa1ecc00640 1 --2- 172.21.15.161:0/2705539537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1cc066690 0x7fa1cc068b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8132e70 tx=0x7fa1d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:19.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:19.913+0000 7fa1e5600640 1 -- 172.21.15.161:0/2705539537 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa1dc0eb720 con 0x7fa1e806c2a0 2024-05-22T21:14:20.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:20.101+0000 7fa1f0000640 1 -- 172.21.15.161:0/2705539537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa1e80ce8f0 con 0x7fa1e806c2a0 2024-05-22T21:14:21.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.077+0000 7fa1e5600640 1 -- 172.21.15.161:0/2705539537 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa1dc0eed40 con 0x7fa1e806c2a0 2024-05-22T21:14:21.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.077+0000 7fa1f0000640 1 -- 172.21.15.161:0/2705539537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1cc066690 msgr2=0x7fa1cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:21.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.077+0000 7fa1f0000640 1 --2- 172.21.15.161:0/2705539537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1cc066690 0x7fa1cc068b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8132e70 tx=0x7fa1d8009040 comp rx=0 tx=0).stop 2024-05-22T21:14:21.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.077+0000 7fa1f0000640 1 -- 172.21.15.161:0/2705539537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e806c2a0 msgr2=0x7fa1e813dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:21.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.077+0000 7fa1f0000640 1 --2- 172.21.15.161:0/2705539537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e806c2a0 0x7fa1e813dd30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc01ac10 tx=0x7fa1dc019300 comp rx=0 tx=0).stop 2024-05-22T21:14:21.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.077+0000 7fa1f0000640 1 -- 172.21.15.161:0/2705539537 shutdown_connections 2024-05-22T21:14:21.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.077+0000 7fa1f0000640 1 --2- 172.21.15.161:0/2705539537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1cc066690 0x7fa1cc068b50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:21.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.077+0000 7fa1f0000640 1 --2- 172.21.15.161:0/2705539537 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1e81402b0 0x7fa1e8154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:21.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.077+0000 7fa1f0000640 1 --2- 172.21.15.161:0/2705539537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1e806c750 0x7fa1e80ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:21.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.077+0000 7fa1f0000640 1 --2- 172.21.15.161:0/2705539537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1e806c2a0 0x7fa1e813dd30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:21.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.077+0000 7fa1f0000640 1 -- 172.21.15.161:0/2705539537 >> 172.21.15.161:0/2705539537 conn(0x7fa1e806a2d0 msgr2=0x7fa1e815bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:21.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.077+0000 7fa1f0000640 1 -- 172.21.15.161:0/2705539537 shutdown_connections 2024-05-22T21:14:21.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.077+0000 7fa1f0000640 1 -- 172.21.15.161:0/2705539537 wait complete. 2024-05-22T21:14:21.105 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-22T21:14:21.105 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-22T21:14:21.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.249+0000 7f2a32a00640 1 Processor -- start 2024-05-22T21:14:21.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.249+0000 7f2a32a00640 1 -- start start 2024-05-22T21:14:21.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.249+0000 7f2a32a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2a2c0d38a0 0x7f2a2c0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:21.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.249+0000 7f2a32a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a2c0cf060 con 0x7f2a2c0d1e60 2024-05-22T21:14:21.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.249+0000 7f2a32a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a2c0d0f00 con 0x7f2a2c15d500 2024-05-22T21:14:21.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.249+0000 7f2a32a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a2c0d11e0 con 0x7f2a2c0d38a0 2024-05-22T21:14:21.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.249+0000 7f2a2b400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f2a2c0d1e60 0x7f2a2c0c7580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:38330/0 (socket says 172.21.15.161:38330) 2024-05-22T21:14:21.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.249+0000 7f2a2b400640 1 -- 172.21.15.161:0/84869232 learned_addr learned my addr 172.21.15.161:0/84869232 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:21.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.249+0000 7f2a30a00640 1 --2- 172.21.15.161:0/84869232 >> v2:172.21.15.161:3300/0 conn(0x7f2a2c0d38a0 0x7f2a2c0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:21.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.249+0000 7f2a2aa00640 1 -- 172.21.15.161:0/84869232 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018988114 0 0) 0x7f2a2c0cf060 con 0x7f2a2c0d1e60 2024-05-22T21:14:21.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.249+0000 7f2a2aa00640 1 -- 172.21.15.161:0/84869232 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a10003530 con 0x7f2a2c0d1e60 2024-05-22T21:14:21.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.249+0000 7f2a30a00640 1 -- 172.21.15.161:0/84869232 >> v1:172.21.15.179:6789/0 conn(0x7f2a2c15d500 legacy=0x7f2a2c0d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:21.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.249+0000 7f2a2aa00640 1 -- 172.21.15.161:0/84869232 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514377223 0 0) 0x7f2a2c0d0f00 con 0x7f2a2c15d500 2024-05-22T21:14:21.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.249+0000 7f2a30a00640 1 -- 172.21.15.161:0/84869232 >> v1:172.21.15.161:6789/0 conn(0x7f2a2c0d1e60 legacy=0x7f2a2c0c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:21.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.249+0000 7f2a30a00640 1 -- 172.21.15.161:0/84869232 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c0d14c0 con 0x7f2a2c0d38a0 2024-05-22T21:14:21.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.253+0000 7f2a30a00640 1 --2- 172.21.15.161:0/84869232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c0d38a0 0x7f2a2c0d03e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009d90 tx=0x7f2a2401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff5b339f7bde5cb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:21.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.253+0000 7f2a2aa00640 1 -- 172.21.15.161:0/84869232 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a240a5970 con 0x7f2a2c0d38a0 2024-05-22T21:14:21.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.253+0000 7f2a2aa00640 1 -- 172.21.15.161:0/84869232 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a240a5ad0 con 0x7f2a2c0d38a0 2024-05-22T21:14:21.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.253+0000 7f2a2aa00640 1 -- 172.21.15.161:0/84869232 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a240a5e60 con 0x7f2a2c0d38a0 2024-05-22T21:14:21.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.253+0000 7f2a32a00640 1 -- 172.21.15.161:0/84869232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c0d38a0 msgr2=0x7f2a2c0d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:21.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.253+0000 7f2a32a00640 1 --2- 172.21.15.161:0/84869232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c0d38a0 0x7f2a2c0d03e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009d90 tx=0x7f2a2401c8b0 comp rx=0 tx=0).stop 2024-05-22T21:14:21.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.253+0000 7f2a32a00640 1 -- 172.21.15.161:0/84869232 shutdown_connections 2024-05-22T21:14:21.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.253+0000 7f2a32a00640 1 --2- 172.21.15.161:0/84869232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c0d38a0 0x7f2a2c0d03e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:21.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.253+0000 7f2a32a00640 1 -- 172.21.15.161:0/84869232 >> 172.21.15.161:0/84869232 conn(0x7f2a2c09c470 msgr2=0x7f2a2c0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:21.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.253+0000 7f2a32a00640 1 -- 172.21.15.161:0/84869232 shutdown_connections 2024-05-22T21:14:21.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.253+0000 7f2a32a00640 1 -- 172.21.15.161:0/84869232 wait complete. 2024-05-22T21:14:21.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a32a00640 1 Processor -- start 2024-05-22T21:14:21.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a32a00640 1 -- start start 2024-05-22T21:14:21.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a32a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a2c0d1e60 0x7f2a2c14e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:21.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a32a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c0d38a0 0x7f2a2c0ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:21.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a32a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a2c0c5e10 0x7f2a2c1f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:21.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a32a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c153020 con 0x7f2a2c0d38a0 2024-05-22T21:14:21.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a32a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c153fd0 con 0x7f2a2c0c5e10 2024-05-22T21:14:21.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a32a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c153b40 con 0x7f2a2c0d1e60 2024-05-22T21:14:21.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a2b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c0d38a0 0x7f2a2c0ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:21.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a2b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c0d38a0 0x7f2a2c0ca4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48984/0 (socket says 172.21.15.161:48984) 2024-05-22T21:14:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a2b400640 1 -- 172.21.15.161:0/3509417138 learned_addr learned my addr 172.21.15.161:0/3509417138 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a30a00640 1 --2- 172.21.15.161:0/3509417138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a2c0c5e10 0x7f2a2c1f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a2be00640 1 --2- 172.21.15.161:0/3509417138 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a2c0d1e60 0x7f2a2c14e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a2b400640 1 -- 172.21.15.161:0/3509417138 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a2c0d1e60 msgr2=0x7f2a2c14e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a2b400640 1 --2- 172.21.15.161:0/3509417138 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a2c0d1e60 0x7f2a2c14e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a2b400640 1 -- 172.21.15.161:0/3509417138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a2c0c5e10 msgr2=0x7f2a2c1f8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a2b400640 1 --2- 172.21.15.161:0/3509417138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a2c0c5e10 0x7f2a2c1f8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a2b400640 1 -- 172.21.15.161:0/3509417138 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a24009a70 con 0x7f2a2c0d38a0 2024-05-22T21:14:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a2b400640 1 --2- 172.21.15.161:0/3509417138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c0d38a0 0x7f2a2c0ca4c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2a180010f0 tx=0x7f2a1800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a23e00640 1 -- 172.21.15.161:0/3509417138 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18005cf0 con 0x7f2a2c0d38a0 2024-05-22T21:14:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a23e00640 1 -- 172.21.15.161:0/3509417138 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a18005e70 con 0x7f2a2c0d38a0 2024-05-22T21:14:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a23e00640 1 -- 172.21.15.161:0/3509417138 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a180072d0 con 0x7f2a2c0d38a0 2024-05-22T21:14:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a32a00640 1 -- 172.21.15.161:0/3509417138 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c14e6f0 con 0x7f2a2c0d38a0 2024-05-22T21:14:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.257+0000 7f2a32a00640 1 -- 172.21.15.161:0/3509417138 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c1534f0 con 0x7f2a2c0d38a0 2024-05-22T21:14:21.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.261+0000 7f2a32a00640 1 -- 172.21.15.161:0/3509417138 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2c153cb0 con 0x7f2a2c0d38a0 2024-05-22T21:14:21.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.261+0000 7f2a23e00640 1 -- 172.21.15.161:0/3509417138 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2a18007430 con 0x7f2a2c0d38a0 2024-05-22T21:14:21.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.261+0000 7f2a23e00640 1 --2- 172.21.15.161:0/3509417138 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a0c0666c0 0x7f2a0c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:21.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.261+0000 7f2a23e00640 1 -- 172.21.15.161:0/3509417138 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a18083850 con 0x7f2a2c0d38a0 2024-05-22T21:14:21.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.261+0000 7f2a2be00640 1 --2- 172.21.15.161:0/3509417138 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a0c0666c0 0x7f2a0c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:21.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.261+0000 7f2a2be00640 1 --2- 172.21.15.161:0/3509417138 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a0c0666c0 0x7f2a0c068b80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c000f70 tx=0x7f2a1c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:21.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.265+0000 7f2a23e00640 1 -- 172.21.15.161:0/3509417138 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2a18050b20 con 0x7f2a2c0d38a0 2024-05-22T21:14:21.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:21.453+0000 7f2a32a00640 1 -- 172.21.15.161:0/3509417138 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2a2c067220 con 0x7f2a2c0d38a0 2024-05-22T21:14:22.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.077+0000 7f2a23e00640 1 -- 172.21.15.161:0/3509417138 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2a18054140 con 0x7f2a2c0d38a0 2024-05-22T21:14:22.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.081+0000 7f2a32a00640 1 -- 172.21.15.161:0/3509417138 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a0c0666c0 msgr2=0x7f2a0c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.081+0000 7f2a32a00640 1 --2- 172.21.15.161:0/3509417138 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a0c0666c0 0x7f2a0c068b80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c000f70 tx=0x7f2a1c009040 comp rx=0 tx=0).stop 2024-05-22T21:14:22.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.081+0000 7f2a32a00640 1 -- 172.21.15.161:0/3509417138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c0d38a0 msgr2=0x7f2a2c0ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.081+0000 7f2a32a00640 1 --2- 172.21.15.161:0/3509417138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c0d38a0 0x7f2a2c0ca4c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2a180010f0 tx=0x7f2a1800c8f0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.081+0000 7f2a32a00640 1 -- 172.21.15.161:0/3509417138 shutdown_connections 2024-05-22T21:14:22.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.081+0000 7f2a32a00640 1 --2- 172.21.15.161:0/3509417138 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a0c0666c0 0x7f2a0c068b80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.081+0000 7f2a32a00640 1 --2- 172.21.15.161:0/3509417138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a2c0c5e10 0x7f2a2c1f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.081+0000 7f2a32a00640 1 --2- 172.21.15.161:0/3509417138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c0d38a0 0x7f2a2c0ca4c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.081+0000 7f2a32a00640 1 --2- 172.21.15.161:0/3509417138 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a2c0d1e60 0x7f2a2c14e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.081+0000 7f2a32a00640 1 -- 172.21.15.161:0/3509417138 >> 172.21.15.161:0/3509417138 conn(0x7f2a2c09c470 msgr2=0x7f2a2c15c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:22.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.081+0000 7f2a32a00640 1 -- 172.21.15.161:0/3509417138 shutdown_connections 2024-05-22T21:14:22.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.081+0000 7f2a32a00640 1 -- 172.21.15.161:0/3509417138 wait complete. 2024-05-22T21:14:22.105 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-22T21:14:22.105 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:14:22.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.245+0000 7ff51f200640 1 Processor -- start 2024-05-22T21:14:22.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.245+0000 7ff51f200640 1 -- start start 2024-05-22T21:14:22.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.245+0000 7ff51f200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff52015b420 0x7ff52015b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.245+0000 7ff51f200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52006be70 0x7ff52006c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.245+0000 7ff51f200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52006de20 con 0x7ff520144550 2024-05-22T21:14:22.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.245+0000 7ff51f200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52006ba50 con 0x7ff52015b420 2024-05-22T21:14:22.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.245+0000 7ff51f200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200cbcb0 con 0x7ff52006be70 2024-05-22T21:14:22.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.245+0000 7ff51e800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52006be70 0x7ff52006c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.245+0000 7ff51e800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52006be70 0x7ff52006c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:51240/0 (socket says 172.21.15.161:51240) 2024-05-22T21:14:22.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.245+0000 7ff51e800640 1 -- 172.21.15.161:0/2731621835 learned_addr learned my addr 172.21.15.161:0/2731621835 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:22.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.245+0000 7ff517e00640 1 --2- 172.21.15.161:0/2731621835 >> v2:172.21.15.179:3300/0 conn(0x7ff52015b420 0x7ff52015b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.245+0000 7ff51e800640 1 -- 172.21.15.161:0/2731621835 >> v2:172.21.15.179:3300/0 conn(0x7ff52015b420 msgr2=0x7ff52015b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:22.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.245+0000 7ff51e800640 1 --2- 172.21.15.161:0/2731621835 >> v2:172.21.15.179:3300/0 conn(0x7ff52015b420 0x7ff52015b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.245+0000 7ff51e800640 1 -- 172.21.15.161:0/2731621835 >> v1:172.21.15.179:6789/0 conn(0x7ff520144550 legacy=0x7ff5200cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.245+0000 7ff51e800640 1 -- 172.21.15.161:0/2731621835 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52015be10 con 0x7ff52006be70 2024-05-22T21:14:22.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.249+0000 7ff51e800640 1 --2- 172.21.15.161:0/2731621835 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52006be70 0x7ff52006c2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff518009e50 tx=0x7ff51801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=939995af4200762d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:22.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.249+0000 7ff51d400640 1 -- 172.21.15.161:0/2731621835 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5180a5830 con 0x7ff52006be70 2024-05-22T21:14:22.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.249+0000 7ff51d400640 1 -- 172.21.15.161:0/2731621835 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5180a5990 con 0x7ff52006be70 2024-05-22T21:14:22.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.249+0000 7ff51d400640 1 -- 172.21.15.161:0/2731621835 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5180a5d20 con 0x7ff52006be70 2024-05-22T21:14:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.249+0000 7ff51f200640 1 -- 172.21.15.161:0/2731621835 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52006be70 msgr2=0x7ff52006c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.249+0000 7ff51f200640 1 --2- 172.21.15.161:0/2731621835 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52006be70 0x7ff52006c2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff518009e50 tx=0x7ff51801c970 comp rx=0 tx=0).stop 2024-05-22T21:14:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.249+0000 7ff51f200640 1 -- 172.21.15.161:0/2731621835 shutdown_connections 2024-05-22T21:14:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.249+0000 7ff51f200640 1 --2- 172.21.15.161:0/2731621835 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52006be70 0x7ff52006c2e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.249+0000 7ff51f200640 1 --2- 172.21.15.161:0/2731621835 >> v2:172.21.15.179:3300/0 conn(0x7ff52015b420 0x7ff52015b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.249+0000 7ff51f200640 1 -- 172.21.15.161:0/2731621835 >> 172.21.15.161:0/2731621835 conn(0x7ff52006a4d0 msgr2=0x7ff5200b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.249+0000 7ff51f200640 1 -- 172.21.15.161:0/2731621835 shutdown_connections 2024-05-22T21:14:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.249+0000 7ff51f200640 1 -- 172.21.15.161:0/2731621835 wait complete. 2024-05-22T21:14:22.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.249+0000 7ff51f200640 1 Processor -- start 2024-05-22T21:14:22.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff51f200640 1 -- start start 2024-05-22T21:14:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff51f200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff52006be70 0x7ff5200ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff51f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff520144550 0x7ff5200b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff51f200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52015b420 0x7ff52015a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff51f200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200b3620 con 0x7ff520144550 2024-05-22T21:14:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff51f200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200b38c0 con 0x7ff52006be70 2024-05-22T21:14:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff51f200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200b3ba0 con 0x7ff52015b420 2024-05-22T21:14:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff517e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff520144550 0x7ff5200b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff517e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff520144550 0x7ff5200b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48988/0 (socket says 172.21.15.161:48988) 2024-05-22T21:14:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff517e00640 1 -- 172.21.15.161:0/3089164577 learned_addr learned my addr 172.21.15.161:0/3089164577 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff51e800640 1 --2- 172.21.15.161:0/3089164577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52015b420 0x7ff52015a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff51de00640 1 --2- 172.21.15.161:0/3089164577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff52006be70 0x7ff5200ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff517e00640 1 -- 172.21.15.161:0/3089164577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52015b420 msgr2=0x7ff52015a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff517e00640 1 --2- 172.21.15.161:0/3089164577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52015b420 0x7ff52015a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff517e00640 1 -- 172.21.15.161:0/3089164577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff52006be70 msgr2=0x7ff5200ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff517e00640 1 --2- 172.21.15.161:0/3089164577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff52006be70 0x7ff5200ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff517e00640 1 -- 172.21.15.161:0/3089164577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff518009b30 con 0x7ff520144550 2024-05-22T21:14:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff51de00640 1 --2- 172.21.15.161:0/3089164577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff52006be70 0x7ff5200ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff517e00640 1 --2- 172.21.15.161:0/3089164577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff520144550 0x7ff5200b30e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff50800e930 tx=0x7ff50800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff516000640 1 -- 172.21.15.161:0/3089164577 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508009690 con 0x7ff520144550 2024-05-22T21:14:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff516000640 1 -- 172.21.15.161:0/3089164577 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50800ca00 con 0x7ff520144550 2024-05-22T21:14:22.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff51f200640 1 -- 172.21.15.161:0/3089164577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5200b1c80 con 0x7ff520144550 2024-05-22T21:14:22.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff516000640 1 -- 172.21.15.161:0/3089164577 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50800cd70 con 0x7ff520144550 2024-05-22T21:14:22.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff51f200640 1 -- 172.21.15.161:0/3089164577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5200b2240 con 0x7ff520144550 2024-05-22T21:14:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.253+0000 7ff51f200640 1 -- 172.21.15.161:0/3089164577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff520067070 con 0x7ff520144550 2024-05-22T21:14:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.257+0000 7ff516000640 1 -- 172.21.15.161:0/3089164577 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff508013470 con 0x7ff520144550 2024-05-22T21:14:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.257+0000 7ff516000640 1 --2- 172.21.15.161:0/3089164577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff4fc066690 0x7ff4fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.257+0000 7ff516000640 1 -- 172.21.15.161:0/3089164577 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff508084430 con 0x7ff520144550 2024-05-22T21:14:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.257+0000 7ff51de00640 1 --2- 172.21.15.161:0/3089164577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff4fc066690 0x7ff4fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.257+0000 7ff51de00640 1 --2- 172.21.15.161:0/3089164577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff4fc066690 0x7ff4fc068b50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff510000f70 tx=0x7ff510009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:22.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.261+0000 7ff516000640 1 -- 172.21.15.161:0/3089164577 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff508051870 con 0x7ff520144550 2024-05-22T21:14:22.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.449+0000 7ff51f200640 1 -- 172.21.15.161:0/3089164577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5200b13c0 con 0x7ff520144550 2024-05-22T21:14:22.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.449+0000 7ff516000640 1 -- 172.21.15.161:0/3089164577 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4743 (secure 0 0 0) 0x7ff508054e90 con 0x7ff520144550 2024-05-22T21:14:22.449 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:22.449 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3742018925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":3742018925},{"type":"v1","addr":"172.21.15.161:6837","nonce":3742018925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2553366378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2553366378},{"type":"v1","addr":"172.21.15.179:6834","nonce":2553366378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/509468338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":509468338},{"type":"v1","addr":"172.21.15.179:6835","nonce":509468338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:19.053259+0000","modified":"2024-05-22T21:14:22.076119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6836/1635458417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1635458417},{"type":"v1","addr":"172.21.15.161:6836","nonce":1635458417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-22T21:14:22.450 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 16 2024-05-22T21:14:22.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.449+0000 7ff51f200640 1 -- 172.21.15.161:0/3089164577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff4fc066690 msgr2=0x7ff4fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.449+0000 7ff51f200640 1 --2- 172.21.15.161:0/3089164577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff4fc066690 0x7ff4fc068b50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff510000f70 tx=0x7ff510009040 comp rx=0 tx=0).stop 2024-05-22T21:14:22.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.449+0000 7ff51f200640 1 -- 172.21.15.161:0/3089164577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff520144550 msgr2=0x7ff5200b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.449+0000 7ff51f200640 1 --2- 172.21.15.161:0/3089164577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff520144550 0x7ff5200b30e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff50800e930 tx=0x7ff50800edf0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.449+0000 7ff51f200640 1 -- 172.21.15.161:0/3089164577 shutdown_connections 2024-05-22T21:14:22.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.449+0000 7ff51f200640 1 --2- 172.21.15.161:0/3089164577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff4fc066690 0x7ff4fc068b50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.449+0000 7ff51f200640 1 --2- 172.21.15.161:0/3089164577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52015b420 0x7ff52015a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.449+0000 7ff51f200640 1 --2- 172.21.15.161:0/3089164577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff520144550 0x7ff5200b30e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.449+0000 7ff51f200640 1 --2- 172.21.15.161:0/3089164577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff52006be70 0x7ff5200ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.449+0000 7ff51f200640 1 -- 172.21.15.161:0/3089164577 >> 172.21.15.161:0/3089164577 conn(0x7ff52006a4d0 msgr2=0x7ff52013e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:22.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.449+0000 7ff51f200640 1 -- 172.21.15.161:0/3089164577 shutdown_connections 2024-05-22T21:14:22.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.449+0000 7ff51f200640 1 -- 172.21.15.161:0/3089164577 wait complete. 2024-05-22T21:14:22.471 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:22.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.605+0000 7f555fa00640 1 Processor -- start 2024-05-22T21:14:22.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.605+0000 7f555fa00640 1 -- start start 2024-05-22T21:14:22.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f555fa00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f555815c6d0 0x7f5558164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f555fa00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5558071670 con 0x7f555815dd50 2024-05-22T21:14:22.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f555fa00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5558070a60 con 0x7f555815e130 2024-05-22T21:14:22.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f555fa00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5558070d40 con 0x7f555815c6d0 2024-05-22T21:14:22.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f555d000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f555815dd50 0x7f555806d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:38340/0 (socket says 172.21.15.161:38340) 2024-05-22T21:14:22.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f555d000640 1 -- 172.21.15.161:0/3944815036 learned_addr learned my addr 172.21.15.161:0/3944815036 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:22.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557400640 1 -- 172.21.15.161:0/3944815036 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784388158 0 0) 0x7f5558071670 con 0x7f555815dd50 2024-05-22T21:14:22.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557400640 1 -- 172.21.15.161:0/3944815036 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f553c003530 con 0x7f555815dd50 2024-05-22T21:14:22.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f555da00640 1 --2- 172.21.15.161:0/3944815036 >> v2:172.21.15.179:3300/0 conn(0x7f555815c6d0 0x7f5558164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557400640 1 -- 172.21.15.161:0/3944815036 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3876975304 0 0) 0x7f553c003530 con 0x7f555815dd50 2024-05-22T21:14:22.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557400640 1 -- 172.21.15.161:0/3944815036 >> v2:172.21.15.179:3300/0 conn(0x7f555815c6d0 msgr2=0x7f5558164e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:22.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557400640 1 --2- 172.21.15.161:0/3944815036 >> v2:172.21.15.179:3300/0 conn(0x7f555815c6d0 0x7f5558164e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557400640 1 -- 172.21.15.161:0/3944815036 >> v1:172.21.15.179:6789/0 conn(0x7f555815e130 legacy=0x7f555815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557400640 1 -- 172.21.15.161:0/3944815036 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5558071020 con 0x7f555815dd50 2024-05-22T21:14:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557400640 1 -- 172.21.15.161:0/3944815036 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f5548001680 con 0x7f555815dd50 2024-05-22T21:14:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557400640 1 -- 172.21.15.161:0/3944815036 >> v1:172.21.15.161:6789/0 conn(0x7f555815dd50 legacy=0x7f555806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557400640 1 --2- 172.21.15.161:0/3944815036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f553c003980 0x7f553c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557400640 1 --2- 172.21.15.161:0/3944815036 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f553c0047d0 0x7f553c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557400640 1 --2- 172.21.15.161:0/3944815036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f553c005190 0x7f553c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557400640 1 -- 172.21.15.161:0/3944815036 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5558070d40 con 0x7f553c003980 2024-05-22T21:14:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557400640 1 -- 172.21.15.161:0/3944815036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c00c0e0 con 0x7f553c005190 2024-05-22T21:14:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557400640 1 -- 172.21.15.161:0/3944815036 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553c00e4b0 con 0x7f553c0047d0 2024-05-22T21:14:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f555d000640 1 --2- 172.21.15.161:0/3944815036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f553c003980 0x7f553c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f555da00640 1 --2- 172.21.15.161:0/3944815036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f553c005190 0x7f553c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557e00640 1 --2- 172.21.15.161:0/3944815036 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f553c0047d0 0x7f553c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f555d000640 1 -- 172.21.15.161:0/3944815036 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f553c0047d0 msgr2=0x7f553c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f555d000640 1 --2- 172.21.15.161:0/3944815036 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f553c0047d0 0x7f553c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f555d000640 1 -- 172.21.15.161:0/3944815036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f553c005190 msgr2=0x7f553c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f555d000640 1 --2- 172.21.15.161:0/3944815036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f553c005190 0x7f553c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f555d000640 1 -- 172.21.15.161:0/3944815036 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553c003530 con 0x7f553c003980 2024-05-22T21:14:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f555da00640 1 --2- 172.21.15.161:0/3944815036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f553c005190 0x7f553c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f5557e00640 1 --2- 172.21.15.161:0/3944815036 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f553c0047d0 0x7f553c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.609+0000 7f555da00640 1 -- 172.21.15.161:0/3944815036 reap_dead start 2024-05-22T21:14:22.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.613+0000 7f555d000640 1 --2- 172.21.15.161:0/3944815036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f553c003980 0x7f553c003da0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f554800c240 tx=0x7f554801cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f89cede02ef01c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:22.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.613+0000 7f5557400640 1 -- 172.21.15.161:0/3944815036 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5548005ae0 con 0x7f553c003980 2024-05-22T21:14:22.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.613+0000 7f5557400640 1 -- 172.21.15.161:0/3944815036 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5548005c40 con 0x7f553c003980 2024-05-22T21:14:22.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.613+0000 7f5557400640 1 -- 172.21.15.161:0/3944815036 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55480a5a30 con 0x7f553c003980 2024-05-22T21:14:22.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.613+0000 7f555fa00640 1 -- 172.21.15.161:0/3944815036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f553c003980 msgr2=0x7f553c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.613+0000 7f555fa00640 1 --2- 172.21.15.161:0/3944815036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f553c003980 0x7f553c003da0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f554800c240 tx=0x7f554801cc30 comp rx=0 tx=0).stop 2024-05-22T21:14:22.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.613+0000 7f555fa00640 1 -- 172.21.15.161:0/3944815036 shutdown_connections 2024-05-22T21:14:22.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.613+0000 7f555fa00640 1 --2- 172.21.15.161:0/3944815036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f553c003980 0x7f553c003da0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.613+0000 7f555fa00640 1 -- 172.21.15.161:0/3944815036 >> 172.21.15.161:0/3944815036 conn(0x7f55580b0bc0 msgr2=0x7f55580b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:22.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.613+0000 7f555fa00640 1 -- 172.21.15.161:0/3944815036 shutdown_connections 2024-05-22T21:14:22.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.613+0000 7f555fa00640 1 -- 172.21.15.161:0/3944815036 wait complete. 2024-05-22T21:14:22.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.613+0000 7f555fa00640 1 Processor -- start 2024-05-22T21:14:22.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.613+0000 7f555fa00640 1 -- start start 2024-05-22T21:14:22.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555fa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f553c003980 0x7f5558158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555fa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5558159ae0 0x7f5558153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555fa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f555814fbb0 0x7f5558150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555fa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55581581c0 con 0x7f553c003980 2024-05-22T21:14:22.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555fa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5558151030 con 0x7f5558159ae0 2024-05-22T21:14:22.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555fa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5558150570 con 0x7f555814fbb0 2024-05-22T21:14:22.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f553c003980 0x7f5558158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f553c003980 0x7f5558158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49002/0 (socket says 172.21.15.161:49002) 2024-05-22T21:14:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555d000640 1 -- 172.21.15.161:0/467240499 learned_addr learned my addr 172.21.15.161:0/467240499 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555da00640 1 --2- 172.21.15.161:0/467240499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f555814fbb0 0x7f5558150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f5557e00640 1 --2- 172.21.15.161:0/467240499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5558159ae0 0x7f5558153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555d000640 1 -- 172.21.15.161:0/467240499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f555814fbb0 msgr2=0x7f5558150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555d000640 1 --2- 172.21.15.161:0/467240499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f555814fbb0 0x7f5558150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555d000640 1 -- 172.21.15.161:0/467240499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5558159ae0 msgr2=0x7f5558153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555d000640 1 --2- 172.21.15.161:0/467240499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5558159ae0 0x7f5558153790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555d000640 1 -- 172.21.15.161:0/467240499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5558152730 con 0x7f553c003980 2024-05-22T21:14:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555d000640 1 --2- 172.21.15.161:0/467240499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f553c003980 0x7f5558158e80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f554801ca60 tx=0x7f553c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:22.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f5554c00640 1 -- 172.21.15.161:0/467240499 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55480b4070 con 0x7f553c003980 2024-05-22T21:14:22.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f5554c00640 1 -- 172.21.15.161:0/467240499 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5548001440 con 0x7f553c003980 2024-05-22T21:14:22.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555fa00640 1 -- 172.21.15.161:0/467240499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55581583a0 con 0x7f553c003980 2024-05-22T21:14:22.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f5554c00640 1 -- 172.21.15.161:0/467240499 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55480a57f0 con 0x7f553c003980 2024-05-22T21:14:22.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555fa00640 1 -- 172.21.15.161:0/467240499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5558158960 con 0x7f553c003980 2024-05-22T21:14:22.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f555fa00640 1 -- 172.21.15.161:0/467240499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5558067220 con 0x7f553c003980 2024-05-22T21:14:22.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.617+0000 7f5554c00640 1 -- 172.21.15.161:0/467240499 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f55480a5970 con 0x7f553c003980 2024-05-22T21:14:22.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.621+0000 7f5554c00640 1 --2- 172.21.15.161:0/467240499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f55380666c0 0x7f5538068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.621+0000 7f5554c00640 1 -- 172.21.15.161:0/467240499 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f554811d980 con 0x7f553c003980 2024-05-22T21:14:22.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.621+0000 7f5557e00640 1 --2- 172.21.15.161:0/467240499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f55380666c0 0x7f5538068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.621+0000 7f5557e00640 1 --2- 172.21.15.161:0/467240499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f55380666c0 0x7f5538068b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f55581f96a0 tx=0x7f5544009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:22.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.625+0000 7f5554c00640 1 -- 172.21.15.161:0/467240499 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f55480eac50 con 0x7f553c003980 2024-05-22T21:14:22.648 INFO:tasks.ceph.mds.c.smithi161.stderr:2024-05-22T21:14:22.645+0000 7f884b800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T21:14:22.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.776+0000 7f555fa00640 1 -- 172.21.15.161:0/467240499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f555806c310 con 0x7f553c003980 2024-05-22T21:14:22.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.780+0000 7f5554c00640 1 -- 172.21.15.161:0/467240499 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16149 (secure 0 0 0) 0x7f55480ee270 con 0x7f553c003980 2024-05-22T21:14:22.781 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:22.781 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":33,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:14:20.055431+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:16.938769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:18.054579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:22.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.780+0000 7f555fa00640 1 -- 172.21.15.161:0/467240499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f55380666c0 msgr2=0x7f5538068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.780+0000 7f555fa00640 1 --2- 172.21.15.161:0/467240499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f55380666c0 0x7f5538068b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f55581f96a0 tx=0x7f5544009040 comp rx=0 tx=0).stop 2024-05-22T21:14:22.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.780+0000 7f555fa00640 1 -- 172.21.15.161:0/467240499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f553c003980 msgr2=0x7f5558158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.780+0000 7f555fa00640 1 --2- 172.21.15.161:0/467240499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f553c003980 0x7f5558158e80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f554801ca60 tx=0x7f553c005800 comp rx=0 tx=0).stop 2024-05-22T21:14:22.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.780+0000 7f555fa00640 1 -- 172.21.15.161:0/467240499 shutdown_connections 2024-05-22T21:14:22.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.780+0000 7f555fa00640 1 --2- 172.21.15.161:0/467240499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f55380666c0 0x7f5538068b80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.780+0000 7f555fa00640 1 --2- 172.21.15.161:0/467240499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f555814fbb0 0x7f5558150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.780+0000 7f555fa00640 1 --2- 172.21.15.161:0/467240499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5558159ae0 0x7f5558153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.780+0000 7f555fa00640 1 --2- 172.21.15.161:0/467240499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f553c003980 0x7f5558158e80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.780+0000 7f555fa00640 1 -- 172.21.15.161:0/467240499 >> 172.21.15.161:0/467240499 conn(0x7f55580b0bc0 msgr2=0x7f55580aebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:22.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.780+0000 7f555fa00640 1 -- 172.21.15.161:0/467240499 shutdown_connections 2024-05-22T21:14:22.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.780+0000 7f555fa00640 1 -- 172.21.15.161:0/467240499 wait complete. 2024-05-22T21:14:22.802 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-22T21:14:22.802 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:14:22.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.932+0000 7fbb73200640 1 Processor -- start 2024-05-22T21:14:22.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.932+0000 7fbb73200640 1 -- start start 2024-05-22T21:14:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.936+0000 7fbb73200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbb6c15dd50 0x7fbb6c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.936+0000 7fbb73200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb6c15e130 0x7fbb6c15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.936+0000 7fbb73200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbb6c15d410 0x7fbb6c15a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.936+0000 7fbb73200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6c071670 con 0x7fbb6c15d410 2024-05-22T21:14:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.936+0000 7fbb73200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6c15d8a0 con 0x7fbb6c15dd50 2024-05-22T21:14:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.936+0000 7fbb73200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c06de40 con 0x7fbb6c15e130 2024-05-22T21:14:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.936+0000 7fbb71200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbb6c15d410 0x7fbb6c15a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.936+0000 7fbb71200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbb6c15d410 0x7fbb6c15a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49008/0 (socket says 172.21.15.161:49008) 2024-05-22T21:14:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.936+0000 7fbb71200640 1 -- 172.21.15.161:0/711506180 learned_addr learned my addr 172.21.15.161:0/711506180 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:22.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.936+0000 7fbb6be00640 1 --2- 172.21.15.161:0/711506180 >> v2:172.21.15.179:3300/0 conn(0x7fbb6c15dd50 0x7fbb6c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.936+0000 7fbb63400640 1 --2- 172.21.15.161:0/711506180 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb6c15e130 0x7fbb6c15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.936+0000 7fbb71200640 1 -- 172.21.15.161:0/711506180 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb6c15e130 msgr2=0x7fbb6c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:22.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.936+0000 7fbb71200640 1 --2- 172.21.15.161:0/711506180 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb6c15e130 0x7fbb6c15ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.936+0000 7fbb71200640 1 -- 172.21.15.161:0/711506180 >> v2:172.21.15.179:3300/0 conn(0x7fbb6c15dd50 msgr2=0x7fbb6c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:22.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.936+0000 7fbb71200640 1 --2- 172.21.15.161:0/711506180 >> v2:172.21.15.179:3300/0 conn(0x7fbb6c15dd50 0x7fbb6c06d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.936+0000 7fbb71200640 1 -- 172.21.15.161:0/711506180 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c070a60 con 0x7fbb6c15d410 2024-05-22T21:14:22.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.940+0000 7fbb71200640 1 --2- 172.21.15.161:0/711506180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb6c15d410 0x7fbb6c15a500 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c072330 tx=0x7fbb6401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f07b34108f5cdec server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:22.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.940+0000 7fbb6b400640 1 -- 172.21.15.161:0/711506180 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb640ab620 con 0x7fbb6c15d410 2024-05-22T21:14:22.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.940+0000 7fbb6b400640 1 -- 172.21.15.161:0/711506180 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb640ab780 con 0x7fbb6c15d410 2024-05-22T21:14:22.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.940+0000 7fbb6b400640 1 -- 172.21.15.161:0/711506180 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb640abb10 con 0x7fbb6c15d410 2024-05-22T21:14:22.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.940+0000 7fbb73200640 1 -- 172.21.15.161:0/711506180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb6c15d410 msgr2=0x7fbb6c15a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.940+0000 7fbb73200640 1 --2- 172.21.15.161:0/711506180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb6c15d410 0x7fbb6c15a500 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c072330 tx=0x7fbb6401c460 comp rx=0 tx=0).stop 2024-05-22T21:14:22.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.940+0000 7fbb73200640 1 -- 172.21.15.161:0/711506180 shutdown_connections 2024-05-22T21:14:22.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.940+0000 7fbb73200640 1 --2- 172.21.15.161:0/711506180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb6c15d410 0x7fbb6c15a500 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.940+0000 7fbb73200640 1 --2- 172.21.15.161:0/711506180 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb6c15e130 0x7fbb6c15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.940+0000 7fbb73200640 1 --2- 172.21.15.161:0/711506180 >> v2:172.21.15.179:3300/0 conn(0x7fbb6c15dd50 0x7fbb6c06d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.940+0000 7fbb73200640 1 -- 172.21.15.161:0/711506180 >> 172.21.15.161:0/711506180 conn(0x7fbb6c0b0bc0 msgr2=0x7fbb6c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:22.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.940+0000 7fbb73200640 1 -- 172.21.15.161:0/711506180 shutdown_connections 2024-05-22T21:14:22.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.940+0000 7fbb73200640 1 -- 172.21.15.161:0/711506180 wait complete. 2024-05-22T21:14:22.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.940+0000 7fbb73200640 1 Processor -- start 2024-05-22T21:14:22.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.940+0000 7fbb73200640 1 -- start start 2024-05-22T21:14:22.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb73200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb6c15d410 0x7fbb6c158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb73200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb6c15dd50 0x7fbb6c14cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb73200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb6c15e130 0x7fbb6c159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb73200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c159c30 con 0x7fbb6c15e130 2024-05-22T21:14:22.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb73200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c14ddb0 con 0x7fbb6c15d410 2024-05-22T21:14:22.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb73200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c159950 con 0x7fbb6c15dd50 2024-05-22T21:14:22.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb71200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb6c15e130 0x7fbb6c159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb71200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb6c15e130 0x7fbb6c159250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49026/0 (socket says 172.21.15.161:49026) 2024-05-22T21:14:22.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb71200640 1 -- 172.21.15.161:0/3970751759 learned_addr learned my addr 172.21.15.161:0/3970751759 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:22.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb6be00640 1 --2- 172.21.15.161:0/3970751759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb6c15d410 0x7fbb6c158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb63400640 1 --2- 172.21.15.161:0/3970751759 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb6c15dd50 0x7fbb6c14cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb71200640 1 -- 172.21.15.161:0/3970751759 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb6c15dd50 msgr2=0x7fbb6c14cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb71200640 1 --2- 172.21.15.161:0/3970751759 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb6c15dd50 0x7fbb6c14cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb71200640 1 -- 172.21.15.161:0/3970751759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb6c15d410 msgr2=0x7fbb6c158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:22.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb71200640 1 --2- 172.21.15.161:0/3970751759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb6c15d410 0x7fbb6c158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:22.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb71200640 1 -- 172.21.15.161:0/3970751759 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb640094d0 con 0x7fbb6c15e130 2024-05-22T21:14:22.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb71200640 1 --2- 172.21.15.161:0/3970751759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb6c15e130 0x7fbb6c159250 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbb6401ab10 tx=0x7fbb6401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:22.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb68c00640 1 -- 172.21.15.161:0/3970751759 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb640abd00 con 0x7fbb6c15e130 2024-05-22T21:14:22.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb68c00640 1 -- 172.21.15.161:0/3970751759 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb640abe60 con 0x7fbb6c15e130 2024-05-22T21:14:22.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb68c00640 1 -- 172.21.15.161:0/3970751759 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb640be630 con 0x7fbb6c15e130 2024-05-22T21:14:22.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb73200640 1 -- 172.21.15.161:0/3970751759 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6c157730 con 0x7fbb6c15e130 2024-05-22T21:14:22.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb73200640 1 -- 172.21.15.161:0/3970751759 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6c157cf0 con 0x7fbb6c15e130 2024-05-22T21:14:22.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb73200640 1 -- 172.21.15.161:0/3970751759 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6c157590 con 0x7fbb6c15e130 2024-05-22T21:14:22.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.944+0000 7fbb68c00640 1 -- 172.21.15.161:0/3970751759 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbb640b43f0 con 0x7fbb6c15e130 2024-05-22T21:14:22.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.948+0000 7fbb68c00640 1 --2- 172.21.15.161:0/3970751759 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb48066690 0x7fbb48068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:22.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.948+0000 7fbb68c00640 1 -- 172.21.15.161:0/3970751759 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb6411e7a0 con 0x7fbb6c15e130 2024-05-22T21:14:22.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.948+0000 7fbb6be00640 1 --2- 172.21.15.161:0/3970751759 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb48066690 0x7fbb48068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:22.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.948+0000 7fbb6be00640 1 --2- 172.21.15.161:0/3970751759 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb48066690 0x7fbb48068b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0113e0 tx=0x7fbb5c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:22.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:22.952+0000 7fbb68c00640 1 -- 172.21.15.161:0/3970751759 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbb6c157590 con 0x7fbb6c15e130 2024-05-22T21:14:22.997 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:14:23.001 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:14:23.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.104+0000 7fbb73200640 1 -- 172.21.15.161:0/3970751759 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb6c070730 con 0x7fbb48066690 2024-05-22T21:14:23.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.112+0000 7fbb68c00640 1 -- 172.21.15.161:0/3970751759 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284490 (secure 0 0 0) 0x7fbb5c00f040 con 0x7fbb48066690 2024-05-22T21:14:23.117 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:23.121 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:14:23.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.120+0000 7fbb73200640 1 -- 172.21.15.161:0/3970751759 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb48066690 msgr2=0x7fbb48068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:23.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.120+0000 7fbb73200640 1 --2- 172.21.15.161:0/3970751759 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb48066690 0x7fbb48068b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0113e0 tx=0x7fbb5c00b040 comp rx=0 tx=0).stop 2024-05-22T21:14:23.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.120+0000 7fbb73200640 1 -- 172.21.15.161:0/3970751759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb6c15e130 msgr2=0x7fbb6c159250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:23.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.120+0000 7fbb73200640 1 --2- 172.21.15.161:0/3970751759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb6c15e130 0x7fbb6c159250 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbb6401ab10 tx=0x7fbb6401aca0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.120+0000 7fbb73200640 1 -- 172.21.15.161:0/3970751759 shutdown_connections 2024-05-22T21:14:23.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.120+0000 7fbb73200640 1 --2- 172.21.15.161:0/3970751759 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb48066690 0x7fbb48068b50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.120+0000 7fbb73200640 1 --2- 172.21.15.161:0/3970751759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb6c15e130 0x7fbb6c159250 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.120+0000 7fbb73200640 1 --2- 172.21.15.161:0/3970751759 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb6c15dd50 0x7fbb6c14cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.120+0000 7fbb73200640 1 --2- 172.21.15.161:0/3970751759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb6c15d410 0x7fbb6c158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.120+0000 7fbb73200640 1 -- 172.21.15.161:0/3970751759 >> 172.21.15.161:0/3970751759 conn(0x7fbb6c0b0bc0 msgr2=0x7fbb6c0aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:23.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.120+0000 7fbb73200640 1 -- 172.21.15.161:0/3970751759 shutdown_connections 2024-05-22T21:14:23.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.120+0000 7fbb73200640 1 -- 172.21.15.161:0/3970751759 wait complete. 2024-05-22T21:14:23.141 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-05-22T21:14:22.406717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2284,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560215,"statfs":{"total":1600000000,"available":1597661175,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.740437"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061559+0000","last_change":"2024-05-22T21:14:19.075809+0000","last_active":"2024-05-22T21:14:20.061559+0000","last_peered":"2024-05-22T21:14:20.061559+0000","last_clean":"2024-05-22T21:14:20.061559+0000","last_became_active":"2024-05-22T21:14:19.075583+0000","last_became_peered":"2024-05-22T21:14:19.075583+0000","last_unstale":"2024-05-22T21:14:20.061559+0000","last_undegraded":"2024-05-22T21:14:20.061559+0000","last_fullsized":"2024-05-22T21:14:20.061559+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:16:57.714956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061731+0000","last_change":"2024-05-22T21:14:19.069739+0000","last_active":"2024-05-22T21:14:20.061731+0000","last_peered":"2024-05-22T21:14:20.061731+0000","last_clean":"2024-05-22T21:14:20.061731+0000","last_became_active":"2024-05-22T21:14:19.069232+0000","last_became_peered":"2024-05-22T21:14:19.069232+0000","last_unstale":"2024-05-22T21:14:20.061731+0000","last_undegraded":"2024-05-22T21:14:20.061731+0000","last_fullsized":"2024-05-22T21:14:20.061731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:49:08.296970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062432+0000","last_change":"2024-05-22T21:14:19.078965+0000","last_active":"2024-05-22T21:14:20.062432+0000","last_peered":"2024-05-22T21:14:20.062432+0000","last_clean":"2024-05-22T21:14:20.062432+0000","last_became_active":"2024-05-22T21:14:19.077828+0000","last_became_peered":"2024-05-22T21:14:19.077828+0000","last_unstale":"2024-05-22T21:14:20.062432+0000","last_undegraded":"2024-05-22T21:14:20.062432+0000","last_fullsized":"2024-05-22T21:14:20.062432+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:53:03.531926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061761+0000","last_change":"2024-05-22T21:14:19.074084+0000","last_active":"2024-05-22T21:14:20.061761+0000","last_peered":"2024-05-22T21:14:20.061761+0000","last_clean":"2024-05-22T21:14:20.061761+0000","last_became_active":"2024-05-22T21:14:19.069975+0000","last_became_peered":"2024-05-22T21:14:19.069975+0000","last_unstale":"2024-05-22T21:14:20.061761+0000","last_undegraded":"2024-05-22T21:14:20.061761+0000","last_fullsized":"2024-05-22T21:14:20.061761+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:24:45.072721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061560+0000","last_change":"2024-05-22T21:14:19.070712+0000","last_active":"2024-05-22T21:14:20.061560+0000","last_peered":"2024-05-22T21:14:20.061560+0000","last_clean":"2024-05-22T21:14:20.061560+0000","last_became_active":"2024-05-22T21:14:19.070166+0000","last_became_peered":"2024-05-22T21:14:19.070166+0000","last_unstale":"2024-05-22T21:14:20.061560+0000","last_undegraded":"2024-05-22T21:14:20.061560+0000","last_fullsized":"2024-05-22T21:14:20.061560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:31:44.124278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062880+0000","last_change":"2024-05-22T21:14:19.073061+0000","last_active":"2024-05-22T21:14:20.062880+0000","last_peered":"2024-05-22T21:14:20.062880+0000","last_clean":"2024-05-22T21:14:20.062880+0000","last_became_active":"2024-05-22T21:14:19.072746+0000","last_became_peered":"2024-05-22T21:14:19.072746+0000","last_unstale":"2024-05-22T21:14:20.062880+0000","last_undegraded":"2024-05-22T21:14:20.062880+0000","last_fullsized":"2024-05-22T21:14:20.062880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:53:03.531926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061196+0000","last_change":"2024-05-22T21:14:19.067746+0000","last_active":"2024-05-22T21:14:20.061196+0000","last_peered":"2024-05-22T21:14:20.061196+0000","last_clean":"2024-05-22T21:14:20.061196+0000","last_became_active":"2024-05-22T21:14:19.067268+0000","last_became_peered":"2024-05-22T21:14:19.067268+0000","last_unstale":"2024-05-22T21:14:20.061196+0000","last_undegraded":"2024-05-22T21:14:20.061196+0000","last_fullsized":"2024-05-22T21:14:20.061196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:15:46.140480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061203+0000","last_change":"2024-05-22T21:14:19.066178+0000","last_active":"2024-05-22T21:14:20.061203+0000","last_peered":"2024-05-22T21:14:20.061203+0000","last_clean":"2024-05-22T21:14:20.061203+0000","last_became_active":"2024-05-22T21:14:19.065931+0000","last_became_peered":"2024-05-22T21:14:19.065931+0000","last_unstale":"2024-05-22T21:14:20.061203+0000","last_undegraded":"2024-05-22T21:14:20.061203+0000","last_fullsized":"2024-05-22T21:14:20.061203+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:05:45.991300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063496+0000","last_change":"2024-05-22T21:14:19.065468+0000","last_active":"2024-05-22T21:14:20.063496+0000","last_peered":"2024-05-22T21:14:20.063496+0000","last_clean":"2024-05-22T21:14:20.063496+0000","last_became_active":"2024-05-22T21:14:19.065221+0000","last_became_peered":"2024-05-22T21:14:19.065221+0000","last_unstale":"2024-05-22T21:14:20.063496+0000","last_undegraded":"2024-05-22T21:14:20.063496+0000","last_fullsized":"2024-05-22T21:14:20.063496+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:34:53.068191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061737+0000","last_change":"2024-05-22T21:14:19.070117+0000","last_active":"2024-05-22T21:14:20.061737+0000","last_peered":"2024-05-22T21:14:20.061737+0000","last_clean":"2024-05-22T21:14:20.061737+0000","last_became_active":"2024-05-22T21:14:19.068399+0000","last_became_peered":"2024-05-22T21:14:19.068399+0000","last_unstale":"2024-05-22T21:14:20.061737+0000","last_undegraded":"2024-05-22T21:14:20.061737+0000","last_fullsized":"2024-05-22T21:14:20.061737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:54:26.183700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061757+0000","last_change":"2024-05-22T21:14:19.066353+0000","last_active":"2024-05-22T21:14:20.061757+0000","last_peered":"2024-05-22T21:14:20.061757+0000","last_clean":"2024-05-22T21:14:20.061757+0000","last_became_active":"2024-05-22T21:14:19.065776+0000","last_became_peered":"2024-05-22T21:14:19.065776+0000","last_unstale":"2024-05-22T21:14:20.061757+0000","last_undegraded":"2024-05-22T21:14:20.061757+0000","last_fullsized":"2024-05-22T21:14:20.061757+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:24:45.072721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062356+0000","last_change":"2024-05-22T21:14:19.072669+0000","last_active":"2024-05-22T21:14:20.062356+0000","last_peered":"2024-05-22T21:14:20.062356+0000","last_clean":"2024-05-22T21:14:20.062356+0000","last_became_active":"2024-05-22T21:14:19.072116+0000","last_became_peered":"2024-05-22T21:14:19.072116+0000","last_unstale":"2024-05-22T21:14:20.062356+0000","last_undegraded":"2024-05-22T21:14:20.062356+0000","last_fullsized":"2024-05-22T21:14:20.062356+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:37:11.967585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061279+0000","last_change":"2024-05-22T21:14:19.075444+0000","last_active":"2024-05-22T21:14:20.061279+0000","last_peered":"2024-05-22T21:14:20.061279+0000","last_clean":"2024-05-22T21:14:20.061279+0000","last_became_active":"2024-05-22T21:14:19.074141+0000","last_became_peered":"2024-05-22T21:14:19.074141+0000","last_unstale":"2024-05-22T21:14:20.061279+0000","last_undegraded":"2024-05-22T21:14:20.061279+0000","last_fullsized":"2024-05-22T21:14:20.061279+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:34:13.649968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062305+0000","last_change":"2024-05-22T21:14:19.063930+0000","last_active":"2024-05-22T21:14:20.062305+0000","last_peered":"2024-05-22T21:14:20.062305+0000","last_clean":"2024-05-22T21:14:20.062305+0000","last_became_active":"2024-05-22T21:14:19.063748+0000","last_became_peered":"2024-05-22T21:14:19.063748+0000","last_unstale":"2024-05-22T21:14:20.062305+0000","last_undegraded":"2024-05-22T21:14:20.062305+0000","last_fullsized":"2024-05-22T21:14:20.062305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:05:45.991300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062721+0000","last_change":"2024-05-22T21:14:19.068633+0000","last_active":"2024-05-22T21:14:20.062721+0000","last_peered":"2024-05-22T21:14:20.062721+0000","last_clean":"2024-05-22T21:14:20.062721+0000","last_became_active":"2024-05-22T21:14:19.068418+0000","last_became_peered":"2024-05-22T21:14:19.068418+0000","last_unstale":"2024-05-22T21:14:20.062721+0000","last_undegraded":"2024-05-22T21:14:20.062721+0000","last_fullsized":"2024-05-22T21:14:20.062721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:49:14.649560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061841+0000","last_change":"2024-05-22T21:14:19.068078+0000","last_active":"2024-05-22T21:14:20.061841+0000","last_peered":"2024-05-22T21:14:20.061841+0000","last_clean":"2024-05-22T21:14:20.061841+0000","last_became_active":"2024-05-22T21:14:19.066115+0000","last_became_peered":"2024-05-22T21:14:19.066115+0000","last_unstale":"2024-05-22T21:14:20.061841+0000","last_undegraded":"2024-05-22T21:14:20.061841+0000","last_fullsized":"2024-05-22T21:14:20.061841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:49:08.296970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061288+0000","last_change":"2024-05-22T21:14:19.067547+0000","last_active":"2024-05-22T21:14:20.061288+0000","last_peered":"2024-05-22T21:14:20.061288+0000","last_clean":"2024-05-22T21:14:20.061288+0000","last_became_active":"2024-05-22T21:14:19.066896+0000","last_became_peered":"2024-05-22T21:14:19.066896+0000","last_unstale":"2024-05-22T21:14:20.061288+0000","last_undegraded":"2024-05-22T21:14:20.061288+0000","last_fullsized":"2024-05-22T21:14:20.061288+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:53:03.531926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062296+0000","last_change":"2024-05-22T21:14:19.064735+0000","last_active":"2024-05-22T21:14:20.062296+0000","last_peered":"2024-05-22T21:14:20.062296+0000","last_clean":"2024-05-22T21:14:20.062296+0000","last_became_active":"2024-05-22T21:14:19.064532+0000","last_became_peered":"2024-05-22T21:14:19.064532+0000","last_unstale":"2024-05-22T21:14:20.062296+0000","last_undegraded":"2024-05-22T21:14:20.062296+0000","last_fullsized":"2024-05-22T21:14:20.062296+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:58:20.679986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062076+0000","last_change":"2024-05-22T21:14:19.071883+0000","last_active":"2024-05-22T21:14:20.062076+0000","last_peered":"2024-05-22T21:14:20.062076+0000","last_clean":"2024-05-22T21:14:20.062076+0000","last_became_active":"2024-05-22T21:14:19.064966+0000","last_became_peered":"2024-05-22T21:14:19.064966+0000","last_unstale":"2024-05-22T21:14:20.062076+0000","last_undegraded":"2024-05-22T21:14:20.062076+0000","last_fullsized":"2024-05-22T21:14:20.062076+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:01:07.386423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061637+0000","last_change":"2024-05-22T21:14:19.076484+0000","last_active":"2024-05-22T21:14:20.061637+0000","last_peered":"2024-05-22T21:14:20.061637+0000","last_clean":"2024-05-22T21:14:20.061637+0000","last_became_active":"2024-05-22T21:14:19.076078+0000","last_became_peered":"2024-05-22T21:14:19.076078+0000","last_unstale":"2024-05-22T21:14:20.061637+0000","last_undegraded":"2024-05-22T21:14:20.061637+0000","last_fullsized":"2024-05-22T21:14:20.061637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:50:41.893275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062920+0000","last_change":"2024-05-22T21:14:19.072360+0000","last_active":"2024-05-22T21:14:20.062920+0000","last_peered":"2024-05-22T21:14:20.062920+0000","last_clean":"2024-05-22T21:14:20.062920+0000","last_became_active":"2024-05-22T21:14:19.072078+0000","last_became_peered":"2024-05-22T21:14:19.072078+0000","last_unstale":"2024-05-22T21:14:20.062920+0000","last_undegraded":"2024-05-22T21:14:20.062920+0000","last_fullsized":"2024-05-22T21:14:20.062920+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:16:57.714956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063301+0000","last_change":"2024-05-22T21:14:19.064748+0000","last_active":"2024-05-22T21:14:20.063301+0000","last_peered":"2024-05-22T21:14:20.063301+0000","last_clean":"2024-05-22T21:14:20.063301+0000","last_became_active":"2024-05-22T21:14:19.063910+0000","last_became_peered":"2024-05-22T21:14:19.063910+0000","last_unstale":"2024-05-22T21:14:20.063301+0000","last_undegraded":"2024-05-22T21:14:20.063301+0000","last_fullsized":"2024-05-22T21:14:20.063301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:18:23.818225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058362+0000","last_change":"2024-05-22T21:14:19.069656+0000","last_active":"2024-05-22T21:14:20.058362+0000","last_peered":"2024-05-22T21:14:20.058362+0000","last_clean":"2024-05-22T21:14:20.058362+0000","last_became_active":"2024-05-22T21:14:19.068999+0000","last_became_peered":"2024-05-22T21:14:19.068999+0000","last_unstale":"2024-05-22T21:14:20.058362+0000","last_undegraded":"2024-05-22T21:14:20.058362+0000","last_fullsized":"2024-05-22T21:14:20.058362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:16:57.714956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062294+0000","last_change":"2024-05-22T21:14:19.076134+0000","last_active":"2024-05-22T21:14:20.062294+0000","last_peered":"2024-05-22T21:14:20.062294+0000","last_clean":"2024-05-22T21:14:20.062294+0000","last_became_active":"2024-05-22T21:14:19.075829+0000","last_became_peered":"2024-05-22T21:14:19.075829+0000","last_unstale":"2024-05-22T21:14:20.062294+0000","last_undegraded":"2024-05-22T21:14:20.062294+0000","last_fullsized":"2024-05-22T21:14:20.062294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:49:08.296970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061824+0000","last_change":"2024-05-22T21:14:19.070539+0000","last_active":"2024-05-22T21:14:20.061824+0000","last_peered":"2024-05-22T21:14:20.061824+0000","last_clean":"2024-05-22T21:14:20.061824+0000","last_became_active":"2024-05-22T21:14:19.070335+0000","last_became_peered":"2024-05-22T21:14:19.070335+0000","last_unstale":"2024-05-22T21:14:20.061824+0000","last_undegraded":"2024-05-22T21:14:20.061824+0000","last_fullsized":"2024-05-22T21:14:20.061824+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:45:18.511639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062960+0000","last_change":"2024-05-22T21:14:19.063816+0000","last_active":"2024-05-22T21:14:20.062960+0000","last_peered":"2024-05-22T21:14:20.062960+0000","last_clean":"2024-05-22T21:14:20.062960+0000","last_became_active":"2024-05-22T21:14:19.063622+0000","last_became_peered":"2024-05-22T21:14:19.063622+0000","last_unstale":"2024-05-22T21:14:20.062960+0000","last_undegraded":"2024-05-22T21:14:20.062960+0000","last_fullsized":"2024-05-22T21:14:20.062960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:34:07.543158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062931+0000","last_change":"2024-05-22T21:14:19.072106+0000","last_active":"2024-05-22T21:14:20.062931+0000","last_peered":"2024-05-22T21:14:20.062931+0000","last_clean":"2024-05-22T21:14:20.062931+0000","last_became_active":"2024-05-22T21:14:19.071872+0000","last_became_peered":"2024-05-22T21:14:19.071872+0000","last_unstale":"2024-05-22T21:14:20.062931+0000","last_undegraded":"2024-05-22T21:14:20.062931+0000","last_fullsized":"2024-05-22T21:14:20.062931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:58:20.679986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062010+0000","last_change":"2024-05-22T21:14:19.074296+0000","last_active":"2024-05-22T21:14:20.062010+0000","last_peered":"2024-05-22T21:14:20.062010+0000","last_clean":"2024-05-22T21:14:20.062010+0000","last_became_active":"2024-05-22T21:14:19.073949+0000","last_became_peered":"2024-05-22T21:14:19.073949+0000","last_unstale":"2024-05-22T21:14:20.062010+0000","last_undegraded":"2024-05-22T21:14:20.062010+0000","last_fullsized":"2024-05-22T21:14:20.062010+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:34:07.543158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063604+0000","last_change":"2024-05-22T21:14:19.072137+0000","last_active":"2024-05-22T21:14:20.063604+0000","last_peered":"2024-05-22T21:14:20.063604+0000","last_clean":"2024-05-22T21:14:20.063604+0000","last_became_active":"2024-05-22T21:14:19.071797+0000","last_became_peered":"2024-05-22T21:14:19.071797+0000","last_unstale":"2024-05-22T21:14:20.063604+0000","last_undegraded":"2024-05-22T21:14:20.063604+0000","last_fullsized":"2024-05-22T21:14:20.063604+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:57:23.815795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061929+0000","last_change":"2024-05-22T21:14:19.068998+0000","last_active":"2024-05-22T21:14:20.061929+0000","last_peered":"2024-05-22T21:14:20.061929+0000","last_clean":"2024-05-22T21:14:20.061929+0000","last_became_active":"2024-05-22T21:14:19.068628+0000","last_became_peered":"2024-05-22T21:14:19.068628+0000","last_unstale":"2024-05-22T21:14:20.061929+0000","last_undegraded":"2024-05-22T21:14:20.061929+0000","last_fullsized":"2024-05-22T21:14:20.061929+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:34:07.543158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061647+0000","last_change":"2024-05-22T21:14:19.069251+0000","last_active":"2024-05-22T21:14:20.061647+0000","last_peered":"2024-05-22T21:14:20.061647+0000","last_clean":"2024-05-22T21:14:20.061647+0000","last_became_active":"2024-05-22T21:14:19.068914+0000","last_became_peered":"2024-05-22T21:14:19.068914+0000","last_unstale":"2024-05-22T21:14:20.061647+0000","last_undegraded":"2024-05-22T21:14:20.061647+0000","last_fullsized":"2024-05-22T21:14:20.061647+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:21:48.071027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061385+0000","last_change":"2024-05-22T21:14:19.062575+0000","last_active":"2024-05-22T21:14:20.061385+0000","last_peered":"2024-05-22T21:14:20.061385+0000","last_clean":"2024-05-22T21:14:20.061385+0000","last_became_active":"2024-05-22T21:14:19.062253+0000","last_became_peered":"2024-05-22T21:14:19.062253+0000","last_unstale":"2024-05-22T21:14:20.061385+0000","last_undegraded":"2024-05-22T21:14:20.061385+0000","last_fullsized":"2024-05-22T21:14:20.061385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:31:30.588601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058420+0000","last_change":"2024-05-22T21:14:19.073511+0000","last_active":"2024-05-22T21:14:20.058420+0000","last_peered":"2024-05-22T21:14:20.058420+0000","last_clean":"2024-05-22T21:14:20.058420+0000","last_became_active":"2024-05-22T21:14:19.073297+0000","last_became_peered":"2024-05-22T21:14:19.073297+0000","last_unstale":"2024-05-22T21:14:20.058420+0000","last_undegraded":"2024-05-22T21:14:20.058420+0000","last_fullsized":"2024-05-22T21:14:20.058420+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:21:48.071027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061044+0000","last_change":"2024-05-22T21:14:18.687958+0000","last_active":"2024-05-22T21:14:20.061044+0000","last_peered":"2024-05-22T21:14:20.061044+0000","last_clean":"2024-05-22T21:14:20.061044+0000","last_became_active":"2024-05-22T21:14:18.687753+0000","last_became_peered":"2024-05-22T21:14:18.687753+0000","last_unstale":"2024-05-22T21:14:20.061044+0000","last_undegraded":"2024-05-22T21:14:20.061044+0000","last_fullsized":"2024-05-22T21:14:20.061044+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:11:50.017938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.060946+0000","last_change":"2024-05-22T21:14:19.071430+0000","last_active":"2024-05-22T21:14:20.060946+0000","last_peered":"2024-05-22T21:14:20.060946+0000","last_clean":"2024-05-22T21:14:20.060946+0000","last_became_active":"2024-05-22T21:14:19.071148+0000","last_became_peered":"2024-05-22T21:14:19.071148+0000","last_unstale":"2024-05-22T21:14:20.060946+0000","last_undegraded":"2024-05-22T21:14:20.060946+0000","last_fullsized":"2024-05-22T21:14:20.060946+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:18:23.818225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058324+0000","last_change":"2024-05-22T21:14:18.682562+0000","last_active":"2024-05-22T21:14:20.058324+0000","last_peered":"2024-05-22T21:14:20.058324+0000","last_clean":"2024-05-22T21:14:20.058324+0000","last_became_active":"2024-05-22T21:14:18.681508+0000","last_became_peered":"2024-05-22T21:14:18.681508+0000","last_unstale":"2024-05-22T21:14:20.058324+0000","last_undegraded":"2024-05-22T21:14:20.058324+0000","last_fullsized":"2024-05-22T21:14:20.058324+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:16:48.710431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063061+0000","last_change":"2024-05-22T21:14:19.065676+0000","last_active":"2024-05-22T21:14:20.063061+0000","last_peered":"2024-05-22T21:14:20.063061+0000","last_clean":"2024-05-22T21:14:20.063061+0000","last_became_active":"2024-05-22T21:14:19.065397+0000","last_became_peered":"2024-05-22T21:14:19.065397+0000","last_unstale":"2024-05-22T21:14:20.063061+0000","last_undegraded":"2024-05-22T21:14:20.063061+0000","last_fullsized":"2024-05-22T21:14:20.063061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:53:56.966937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062376+0000","last_change":"2024-05-22T21:14:18.685300+0000","last_active":"2024-05-22T21:14:20.062376+0000","last_peered":"2024-05-22T21:14:20.062376+0000","last_clean":"2024-05-22T21:14:20.062376+0000","last_became_active":"2024-05-22T21:14:18.685102+0000","last_became_peered":"2024-05-22T21:14:18.685102+0000","last_unstale":"2024-05-22T21:14:20.062376+0000","last_undegraded":"2024-05-22T21:14:20.062376+0000","last_fullsized":"2024-05-22T21:14:20.062376+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:35:24.776954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061917+0000","last_change":"2024-05-22T21:14:19.076460+0000","last_active":"2024-05-22T21:14:20.061917+0000","last_peered":"2024-05-22T21:14:20.061917+0000","last_clean":"2024-05-22T21:14:20.061917+0000","last_became_active":"2024-05-22T21:14:19.072447+0000","last_became_peered":"2024-05-22T21:14:19.072447+0000","last_unstale":"2024-05-22T21:14:20.061917+0000","last_undegraded":"2024-05-22T21:14:20.061917+0000","last_fullsized":"2024-05-22T21:14:20.061917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:31:30.588601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062606+0000","last_change":"2024-05-22T21:14:18.687785+0000","last_active":"2024-05-22T21:14:20.062606+0000","last_peered":"2024-05-22T21:14:20.062606+0000","last_clean":"2024-05-22T21:14:20.062606+0000","last_became_active":"2024-05-22T21:14:18.687525+0000","last_became_peered":"2024-05-22T21:14:18.687525+0000","last_unstale":"2024-05-22T21:14:20.062606+0000","last_undegraded":"2024-05-22T21:14:20.062606+0000","last_fullsized":"2024-05-22T21:14:20.062606+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:38:56.042952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061358+0000","last_change":"2024-05-22T21:14:19.064148+0000","last_active":"2024-05-22T21:14:20.061358+0000","last_peered":"2024-05-22T21:14:20.061358+0000","last_clean":"2024-05-22T21:14:20.061358+0000","last_became_active":"2024-05-22T21:14:19.062245+0000","last_became_peered":"2024-05-22T21:14:19.062245+0000","last_unstale":"2024-05-22T21:14:20.061358+0000","last_undegraded":"2024-05-22T21:14:20.061358+0000","last_fullsized":"2024-05-22T21:14:20.061358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:22:06.703835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063240+0000","last_change":"2024-05-22T21:14:18.679232+0000","last_active":"2024-05-22T21:14:20.063240+0000","last_peered":"2024-05-22T21:14:20.063240+0000","last_clean":"2024-05-22T21:14:20.063240+0000","last_became_active":"2024-05-22T21:14:18.678977+0000","last_became_peered":"2024-05-22T21:14:18.678977+0000","last_unstale":"2024-05-22T21:14:20.063240+0000","last_undegraded":"2024-05-22T21:14:20.063240+0000","last_fullsized":"2024-05-22T21:14:20.063240+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:47:53.263656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062459+0000","last_change":"2024-05-22T21:14:19.073308+0000","last_active":"2024-05-22T21:14:20.062459+0000","last_peered":"2024-05-22T21:14:20.062459+0000","last_clean":"2024-05-22T21:14:20.062459+0000","last_became_active":"2024-05-22T21:14:19.072924+0000","last_became_peered":"2024-05-22T21:14:19.072924+0000","last_unstale":"2024-05-22T21:14:20.062459+0000","last_undegraded":"2024-05-22T21:14:20.062459+0000","last_fullsized":"2024-05-22T21:14:20.062459+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:01:07.386423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062441+0000","last_change":"2024-05-22T21:14:18.682839+0000","last_active":"2024-05-22T21:14:20.062441+0000","last_peered":"2024-05-22T21:14:20.062441+0000","last_clean":"2024-05-22T21:14:20.062441+0000","last_became_active":"2024-05-22T21:14:18.682559+0000","last_became_peered":"2024-05-22T21:14:18.682559+0000","last_unstale":"2024-05-22T21:14:20.062441+0000","last_undegraded":"2024-05-22T21:14:20.062441+0000","last_fullsized":"2024-05-22T21:14:20.062441+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:34:45.136012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062882+0000","last_change":"2024-05-22T21:14:19.070174+0000","last_active":"2024-05-22T21:14:20.062882+0000","last_peered":"2024-05-22T21:14:20.062882+0000","last_clean":"2024-05-22T21:14:20.062882+0000","last_became_active":"2024-05-22T21:14:19.069916+0000","last_became_peered":"2024-05-22T21:14:19.069916+0000","last_unstale":"2024-05-22T21:14:20.062882+0000","last_undegraded":"2024-05-22T21:14:20.062882+0000","last_fullsized":"2024-05-22T21:14:20.062882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:33:32.237400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062371+0000","last_change":"2024-05-22T21:14:18.688013+0000","last_active":"2024-05-22T21:14:20.062371+0000","last_peered":"2024-05-22T21:14:20.062371+0000","last_clean":"2024-05-22T21:14:20.062371+0000","last_became_active":"2024-05-22T21:14:18.682394+0000","last_became_peered":"2024-05-22T21:14:18.682394+0000","last_unstale":"2024-05-22T21:14:20.062371+0000","last_undegraded":"2024-05-22T21:14:20.062371+0000","last_fullsized":"2024-05-22T21:14:20.062371+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:40:57.258501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061843+0000","last_change":"2024-05-22T21:14:19.074879+0000","last_active":"2024-05-22T21:14:20.061843+0000","last_peered":"2024-05-22T21:14:20.061843+0000","last_clean":"2024-05-22T21:14:20.061843+0000","last_became_active":"2024-05-22T21:14:19.074290+0000","last_became_peered":"2024-05-22T21:14:19.074290+0000","last_unstale":"2024-05-22T21:14:20.061843+0000","last_undegraded":"2024-05-22T21:14:20.061843+0000","last_fullsized":"2024-05-22T21:14:20.061843+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:22:06.703835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063272+0000","last_change":"2024-05-22T21:14:18.685836+0000","last_active":"2024-05-22T21:14:20.063272+0000","last_peered":"2024-05-22T21:14:20.063272+0000","last_clean":"2024-05-22T21:14:20.063272+0000","last_became_active":"2024-05-22T21:14:18.684837+0000","last_became_peered":"2024-05-22T21:14:18.684837+0000","last_unstale":"2024-05-22T21:14:20.063272+0000","last_undegraded":"2024-05-22T21:14:20.063272+0000","last_fullsized":"2024-05-22T21:14:20.063272+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:43:45.138067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062736+0000","last_change":"2024-05-22T21:14:19.072396+0000","last_active":"2024-05-22T21:14:20.062736+0000","last_peered":"2024-05-22T21:14:20.062736+0000","last_clean":"2024-05-22T21:14:20.062736+0000","last_became_active":"2024-05-22T21:14:19.071611+0000","last_became_peered":"2024-05-22T21:14:19.071611+0000","last_unstale":"2024-05-22T21:14:20.062736+0000","last_undegraded":"2024-05-22T21:14:20.062736+0000","last_fullsized":"2024-05-22T21:14:20.062736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:35:41.126663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063086+0000","last_change":"2024-05-22T21:14:18.683506+0000","last_active":"2024-05-22T21:14:20.063086+0000","last_peered":"2024-05-22T21:14:20.063086+0000","last_clean":"2024-05-22T21:14:20.063086+0000","last_became_active":"2024-05-22T21:14:18.683251+0000","last_became_peered":"2024-05-22T21:14:18.683251+0000","last_unstale":"2024-05-22T21:14:20.063086+0000","last_undegraded":"2024-05-22T21:14:20.063086+0000","last_fullsized":"2024-05-22T21:14:20.063086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:34:52.060776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063037+0000","last_change":"2024-05-22T21:14:19.065214+0000","last_active":"2024-05-22T21:14:20.063037+0000","last_peered":"2024-05-22T21:14:20.063037+0000","last_clean":"2024-05-22T21:14:20.063037+0000","last_became_active":"2024-05-22T21:14:19.064985+0000","last_became_peered":"2024-05-22T21:14:19.064985+0000","last_unstale":"2024-05-22T21:14:20.063037+0000","last_undegraded":"2024-05-22T21:14:20.063037+0000","last_fullsized":"2024-05-22T21:14:20.063037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:35:41.126663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058462+0000","last_change":"2024-05-22T21:14:18.678981+0000","last_active":"2024-05-22T21:14:20.058462+0000","last_peered":"2024-05-22T21:14:20.058462+0000","last_clean":"2024-05-22T21:14:20.058462+0000","last_became_active":"2024-05-22T21:14:18.678745+0000","last_became_peered":"2024-05-22T21:14:18.678745+0000","last_unstale":"2024-05-22T21:14:20.058462+0000","last_undegraded":"2024-05-22T21:14:20.058462+0000","last_fullsized":"2024-05-22T21:14:20.058462+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:40:57.258501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062524+0000","last_change":"2024-05-22T21:14:19.061923+0000","last_active":"2024-05-22T21:14:20.062524+0000","last_peered":"2024-05-22T21:14:20.062524+0000","last_clean":"2024-05-22T21:14:20.062524+0000","last_became_active":"2024-05-22T21:14:19.061703+0000","last_became_peered":"2024-05-22T21:14:19.061703+0000","last_unstale":"2024-05-22T21:14:20.062524+0000","last_undegraded":"2024-05-22T21:14:20.062524+0000","last_fullsized":"2024-05-22T21:14:20.062524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:15:46.140480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062508+0000","last_change":"2024-05-22T21:14:18.689586+0000","last_active":"2024-05-22T21:14:20.062508+0000","last_peered":"2024-05-22T21:14:20.062508+0000","last_clean":"2024-05-22T21:14:20.062508+0000","last_became_active":"2024-05-22T21:14:18.689146+0000","last_became_peered":"2024-05-22T21:14:18.689146+0000","last_unstale":"2024-05-22T21:14:20.062508+0000","last_undegraded":"2024-05-22T21:14:20.062508+0000","last_fullsized":"2024-05-22T21:14:20.062508+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:57:22.808381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062182+0000","last_change":"2024-05-22T21:14:19.079226+0000","last_active":"2024-05-22T21:14:20.062182+0000","last_peered":"2024-05-22T21:14:20.062182+0000","last_clean":"2024-05-22T21:14:20.062182+0000","last_became_active":"2024-05-22T21:14:19.078091+0000","last_became_peered":"2024-05-22T21:14:19.078091+0000","last_unstale":"2024-05-22T21:14:20.062182+0000","last_undegraded":"2024-05-22T21:14:20.062182+0000","last_fullsized":"2024-05-22T21:14:20.062182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:03:35.024855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062444+0000","last_change":"2024-05-22T21:14:18.689130+0000","last_active":"2024-05-22T21:14:20.062444+0000","last_peered":"2024-05-22T21:14:20.062444+0000","last_clean":"2024-05-22T21:14:20.062444+0000","last_became_active":"2024-05-22T21:14:18.688397+0000","last_became_peered":"2024-05-22T21:14:18.688397+0000","last_unstale":"2024-05-22T21:14:20.062444+0000","last_undegraded":"2024-05-22T21:14:20.062444+0000","last_fullsized":"2024-05-22T21:14:20.062444+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:18:22.810811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061980+0000","last_change":"2024-05-22T21:14:19.074374+0000","last_active":"2024-05-22T21:14:20.061980+0000","last_peered":"2024-05-22T21:14:20.061980+0000","last_clean":"2024-05-22T21:14:20.061980+0000","last_became_active":"2024-05-22T21:14:19.074147+0000","last_became_peered":"2024-05-22T21:14:19.074147+0000","last_unstale":"2024-05-22T21:14:20.061980+0000","last_undegraded":"2024-05-22T21:14:20.061980+0000","last_fullsized":"2024-05-22T21:14:20.061980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:05:45.991300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063366+0000","last_change":"2024-05-22T21:14:18.685500+0000","last_active":"2024-05-22T21:14:20.063366+0000","last_peered":"2024-05-22T21:14:20.063366+0000","last_clean":"2024-05-22T21:14:20.063366+0000","last_became_active":"2024-05-22T21:14:18.684509+0000","last_became_peered":"2024-05-22T21:14:18.684509+0000","last_unstale":"2024-05-22T21:14:20.063366+0000","last_undegraded":"2024-05-22T21:14:20.063366+0000","last_fullsized":"2024-05-22T21:14:20.063366+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:31:43.116864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062222+0000","last_change":"2024-05-22T21:14:19.073238+0000","last_active":"2024-05-22T21:14:20.062222+0000","last_peered":"2024-05-22T21:14:20.062222+0000","last_clean":"2024-05-22T21:14:20.062222+0000","last_became_active":"2024-05-22T21:14:19.072721+0000","last_became_peered":"2024-05-22T21:14:19.072721+0000","last_unstale":"2024-05-22T21:14:20.062222+0000","last_undegraded":"2024-05-22T21:14:20.062222+0000","last_fullsized":"2024-05-22T21:14:20.062222+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:03:35.024855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062125+0000","last_change":"2024-05-22T21:14:18.689238+0000","last_active":"2024-05-22T21:14:20.062125+0000","last_peered":"2024-05-22T21:14:20.062125+0000","last_clean":"2024-05-22T21:14:20.062125+0000","last_became_active":"2024-05-22T21:14:18.687071+0000","last_became_peered":"2024-05-22T21:14:18.687071+0000","last_unstale":"2024-05-22T21:14:20.062125+0000","last_undegraded":"2024-05-22T21:14:20.062125+0000","last_fullsized":"2024-05-22T21:14:20.062125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:34:27.211979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061443+0000","last_change":"2024-05-22T21:14:19.064281+0000","last_active":"2024-05-22T21:14:20.061443+0000","last_peered":"2024-05-22T21:14:20.061443+0000","last_clean":"2024-05-22T21:14:20.061443+0000","last_became_active":"2024-05-22T21:14:19.063254+0000","last_became_peered":"2024-05-22T21:14:19.063254+0000","last_unstale":"2024-05-22T21:14:20.061443+0000","last_undegraded":"2024-05-22T21:14:20.061443+0000","last_fullsized":"2024-05-22T21:14:20.061443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:54:26.183700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062042+0000","last_change":"2024-05-22T21:14:18.688356+0000","last_active":"2024-05-22T21:14:20.062042+0000","last_peered":"2024-05-22T21:14:20.062042+0000","last_clean":"2024-05-22T21:14:20.062042+0000","last_became_active":"2024-05-22T21:14:18.687837+0000","last_became_peered":"2024-05-22T21:14:18.687837+0000","last_unstale":"2024-05-22T21:14:20.062042+0000","last_undegraded":"2024-05-22T21:14:20.062042+0000","last_fullsized":"2024-05-22T21:14:20.062042+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:34:27.211979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061291+0000","last_change":"2024-05-22T21:14:19.074572+0000","last_active":"2024-05-22T21:14:20.061291+0000","last_peered":"2024-05-22T21:14:20.061291+0000","last_clean":"2024-05-22T21:14:20.061291+0000","last_became_active":"2024-05-22T21:14:19.074017+0000","last_became_peered":"2024-05-22T21:14:19.074017+0000","last_unstale":"2024-05-22T21:14:20.061291+0000","last_undegraded":"2024-05-22T21:14:20.061291+0000","last_fullsized":"2024-05-22T21:14:20.061291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:57:23.815795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062434+0000","last_change":"2024-05-22T21:14:18.683288+0000","last_active":"2024-05-22T21:14:20.062434+0000","last_peered":"2024-05-22T21:14:20.062434+0000","last_clean":"2024-05-22T21:14:20.062434+0000","last_became_active":"2024-05-22T21:14:18.682999+0000","last_became_peered":"2024-05-22T21:14:18.682999+0000","last_unstale":"2024-05-22T21:14:20.062434+0000","last_undegraded":"2024-05-22T21:14:20.062434+0000","last_fullsized":"2024-05-22T21:14:20.062434+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:04:58.088978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063153+0000","last_change":"2024-05-22T21:14:19.070769+0000","last_active":"2024-05-22T21:14:20.063153+0000","last_peered":"2024-05-22T21:14:20.063153+0000","last_clean":"2024-05-22T21:14:20.063153+0000","last_became_active":"2024-05-22T21:14:19.070536+0000","last_became_peered":"2024-05-22T21:14:19.070536+0000","last_unstale":"2024-05-22T21:14:20.063153+0000","last_undegraded":"2024-05-22T21:14:20.063153+0000","last_fullsized":"2024-05-22T21:14:20.063153+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:28:03.263405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061994+0000","last_change":"2024-05-22T21:14:18.684858+0000","last_active":"2024-05-22T21:14:20.061994+0000","last_peered":"2024-05-22T21:14:20.061994+0000","last_clean":"2024-05-22T21:14:20.061994+0000","last_became_active":"2024-05-22T21:14:18.684211+0000","last_became_peered":"2024-05-22T21:14:18.684211+0000","last_unstale":"2024-05-22T21:14:20.061994+0000","last_undegraded":"2024-05-22T21:14:20.061994+0000","last_fullsized":"2024-05-22T21:14:20.061994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:34:52.060776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062408+0000","last_change":"2024-05-22T21:14:19.074599+0000","last_active":"2024-05-22T21:14:20.062408+0000","last_peered":"2024-05-22T21:14:20.062408+0000","last_clean":"2024-05-22T21:14:20.062408+0000","last_became_active":"2024-05-22T21:14:19.074152+0000","last_became_peered":"2024-05-22T21:14:19.074152+0000","last_unstale":"2024-05-22T21:14:20.062408+0000","last_undegraded":"2024-05-22T21:14:20.062408+0000","last_fullsized":"2024-05-22T21:14:20.062408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:22:06.703835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063449+0000","last_change":"2024-05-22T21:14:18.686006+0000","last_active":"2024-05-22T21:14:20.063449+0000","last_peered":"2024-05-22T21:14:20.063449+0000","last_clean":"2024-05-22T21:14:20.063449+0000","last_became_active":"2024-05-22T21:14:18.685235+0000","last_became_peered":"2024-05-22T21:14:18.685235+0000","last_unstale":"2024-05-22T21:14:20.063449+0000","last_undegraded":"2024-05-22T21:14:20.063449+0000","last_fullsized":"2024-05-22T21:14:20.063449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:18:12.891326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062768+0000","last_change":"2024-05-22T21:14:19.069069+0000","last_active":"2024-05-22T21:14:20.062768+0000","last_peered":"2024-05-22T21:14:20.062768+0000","last_clean":"2024-05-22T21:14:20.062768+0000","last_became_active":"2024-05-22T21:14:19.068855+0000","last_became_peered":"2024-05-22T21:14:19.068855+0000","last_unstale":"2024-05-22T21:14:20.062768+0000","last_undegraded":"2024-05-22T21:14:20.062768+0000","last_fullsized":"2024-05-22T21:14:20.062768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:49:08.296970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063173+0000","last_change":"2024-05-22T21:14:18.682049+0000","last_active":"2024-05-22T21:14:20.063173+0000","last_peered":"2024-05-22T21:14:20.063173+0000","last_clean":"2024-05-22T21:14:20.063173+0000","last_became_active":"2024-05-22T21:14:18.681799+0000","last_became_peered":"2024-05-22T21:14:18.681799+0000","last_unstale":"2024-05-22T21:14:20.063173+0000","last_undegraded":"2024-05-22T21:14:20.063173+0000","last_fullsized":"2024-05-22T21:14:20.063173+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:42:39.208248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061234+0000","last_change":"2024-05-22T21:14:19.073723+0000","last_active":"2024-05-22T21:14:20.061234+0000","last_peered":"2024-05-22T21:14:20.061234+0000","last_clean":"2024-05-22T21:14:20.061234+0000","last_became_active":"2024-05-22T21:14:19.073484+0000","last_became_peered":"2024-05-22T21:14:19.073484+0000","last_unstale":"2024-05-22T21:14:20.061234+0000","last_undegraded":"2024-05-22T21:14:20.061234+0000","last_fullsized":"2024-05-22T21:14:20.061234+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:43:46.145481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063352+0000","last_change":"2024-05-22T21:14:18.680460+0000","last_active":"2024-05-22T21:14:20.063352+0000","last_peered":"2024-05-22T21:14:20.063352+0000","last_clean":"2024-05-22T21:14:20.063352+0000","last_became_active":"2024-05-22T21:14:18.679883+0000","last_became_peered":"2024-05-22T21:14:18.679883+0000","last_unstale":"2024-05-22T21:14:20.063352+0000","last_undegraded":"2024-05-22T21:14:20.063352+0000","last_fullsized":"2024-05-22T21:14:20.063352+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:44:36.545411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061375+0000","last_change":"2024-05-22T21:14:19.064604+0000","last_active":"2024-05-22T21:14:20.061375+0000","last_peered":"2024-05-22T21:14:20.061375+0000","last_clean":"2024-05-22T21:14:20.061375+0000","last_became_active":"2024-05-22T21:14:19.064264+0000","last_became_peered":"2024-05-22T21:14:19.064264+0000","last_unstale":"2024-05-22T21:14:20.061375+0000","last_undegraded":"2024-05-22T21:14:20.061375+0000","last_fullsized":"2024-05-22T21:14:20.061375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:10:29.781440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058454+0000","last_change":"2024-05-22T21:14:18.683390+0000","last_active":"2024-05-22T21:14:20.058454+0000","last_peered":"2024-05-22T21:14:20.058454+0000","last_clean":"2024-05-22T21:14:20.058454+0000","last_became_active":"2024-05-22T21:14:18.682551+0000","last_became_peered":"2024-05-22T21:14:18.682551+0000","last_unstale":"2024-05-22T21:14:20.058454+0000","last_undegraded":"2024-05-22T21:14:20.058454+0000","last_fullsized":"2024-05-22T21:14:20.058454+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:02:14.974395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061374+0000","last_change":"2024-05-22T21:14:19.074423+0000","last_active":"2024-05-22T21:14:20.061374+0000","last_peered":"2024-05-22T21:14:20.061374+0000","last_clean":"2024-05-22T21:14:20.061374+0000","last_became_active":"2024-05-22T21:14:19.073863+0000","last_became_peered":"2024-05-22T21:14:19.073863+0000","last_unstale":"2024-05-22T21:14:20.061374+0000","last_undegraded":"2024-05-22T21:14:20.061374+0000","last_fullsized":"2024-05-22T21:14:20.061374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:11:51.025353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.060846+0000","last_change":"2024-05-22T21:14:18.683770+0000","last_active":"2024-05-22T21:14:20.060846+0000","last_peered":"2024-05-22T21:14:20.060846+0000","last_clean":"2024-05-22T21:14:20.060846+0000","last_became_active":"2024-05-22T21:14:18.683515+0000","last_became_peered":"2024-05-22T21:14:18.683515+0000","last_unstale":"2024-05-22T21:14:20.060846+0000","last_undegraded":"2024-05-22T21:14:20.060846+0000","last_fullsized":"2024-05-22T21:14:20.060846+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:16:48.710431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061326+0000","last_change":"2024-05-22T21:14:19.072479+0000","last_active":"2024-05-22T21:14:20.061326+0000","last_peered":"2024-05-22T21:14:20.061326+0000","last_clean":"2024-05-22T21:14:20.061326+0000","last_became_active":"2024-05-22T21:14:19.072272+0000","last_became_peered":"2024-05-22T21:14:19.072272+0000","last_unstale":"2024-05-22T21:14:20.061326+0000","last_undegraded":"2024-05-22T21:14:20.061326+0000","last_fullsized":"2024-05-22T21:14:20.061326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:02:15.981809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063503+0000","last_change":"2024-05-22T21:14:18.687665+0000","last_active":"2024-05-22T21:14:20.063503+0000","last_peered":"2024-05-22T21:14:20.063503+0000","last_clean":"2024-05-22T21:14:20.063503+0000","last_became_active":"2024-05-22T21:14:18.687439+0000","last_became_peered":"2024-05-22T21:14:18.687439+0000","last_unstale":"2024-05-22T21:14:20.063503+0000","last_undegraded":"2024-05-22T21:14:20.063503+0000","last_fullsized":"2024-05-22T21:14:20.063503+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:41:19.249505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062005+0000","last_change":"2024-05-22T21:14:19.068841+0000","last_active":"2024-05-22T21:14:20.062005+0000","last_peered":"2024-05-22T21:14:20.062005+0000","last_clean":"2024-05-22T21:14:20.062005+0000","last_became_active":"2024-05-22T21:14:19.068115+0000","last_became_peered":"2024-05-22T21:14:19.068115+0000","last_unstale":"2024-05-22T21:14:20.062005+0000","last_undegraded":"2024-05-22T21:14:20.062005+0000","last_fullsized":"2024-05-22T21:14:20.062005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:58:20.679986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062714+0000","last_change":"2024-05-22T21:14:18.689681+0000","last_active":"2024-05-22T21:14:20.062714+0000","last_peered":"2024-05-22T21:14:20.062714+0000","last_clean":"2024-05-22T21:14:20.062714+0000","last_became_active":"2024-05-22T21:14:18.689354+0000","last_became_peered":"2024-05-22T21:14:18.689354+0000","last_unstale":"2024-05-22T21:14:20.062714+0000","last_undegraded":"2024-05-22T21:14:20.062714+0000","last_fullsized":"2024-05-22T21:14:20.062714+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:35:24.776954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061458+0000","last_change":"2024-05-22T21:14:19.071264+0000","last_active":"2024-05-22T21:14:20.061458+0000","last_peered":"2024-05-22T21:14:20.061458+0000","last_clean":"2024-05-22T21:14:20.061458+0000","last_became_active":"2024-05-22T21:14:19.071022+0000","last_became_peered":"2024-05-22T21:14:19.071022+0000","last_unstale":"2024-05-22T21:14:20.061458+0000","last_undegraded":"2024-05-22T21:14:20.061458+0000","last_fullsized":"2024-05-22T21:14:20.061458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:16:49.717845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062848+0000","last_change":"2024-05-22T21:14:18.689595+0000","last_active":"2024-05-22T21:14:20.062848+0000","last_peered":"2024-05-22T21:14:20.062848+0000","last_clean":"2024-05-22T21:14:20.062848+0000","last_became_active":"2024-05-22T21:14:18.688892+0000","last_became_peered":"2024-05-22T21:14:18.688892+0000","last_unstale":"2024-05-22T21:14:20.062848+0000","last_undegraded":"2024-05-22T21:14:20.062848+0000","last_fullsized":"2024-05-22T21:14:20.062848+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:40:57.258501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062571+0000","last_change":"2024-05-22T21:14:19.072541+0000","last_active":"2024-05-22T21:14:20.062571+0000","last_peered":"2024-05-22T21:14:20.062571+0000","last_clean":"2024-05-22T21:14:20.062571+0000","last_became_active":"2024-05-22T21:14:19.071855+0000","last_became_peered":"2024-05-22T21:14:19.071855+0000","last_unstale":"2024-05-22T21:14:20.062571+0000","last_undegraded":"2024-05-22T21:14:20.062571+0000","last_fullsized":"2024-05-22T21:14:20.062571+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:15:46.140480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058527+0000","last_change":"2024-05-22T21:14:18.683247+0000","last_active":"2024-05-22T21:14:20.058527+0000","last_peered":"2024-05-22T21:14:20.058527+0000","last_clean":"2024-05-22T21:14:20.058527+0000","last_became_active":"2024-05-22T21:14:18.682247+0000","last_became_peered":"2024-05-22T21:14:18.682247+0000","last_unstale":"2024-05-22T21:14:20.058527+0000","last_undegraded":"2024-05-22T21:14:20.058527+0000","last_fullsized":"2024-05-22T21:14:20.058527+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:11:50.017938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058224+0000","last_change":"2024-05-22T21:14:19.070412+0000","last_active":"2024-05-22T21:14:20.058224+0000","last_peered":"2024-05-22T21:14:20.058224+0000","last_clean":"2024-05-22T21:14:20.058224+0000","last_became_active":"2024-05-22T21:14:19.070206+0000","last_became_peered":"2024-05-22T21:14:19.070206+0000","last_unstale":"2024-05-22T21:14:20.058224+0000","last_undegraded":"2024-05-22T21:14:20.058224+0000","last_fullsized":"2024-05-22T21:14:20.058224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:57:23.815795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062047+0000","last_change":"2024-05-22T21:14:18.682898+0000","last_active":"2024-05-22T21:14:20.062047+0000","last_peered":"2024-05-22T21:14:20.062047+0000","last_clean":"2024-05-22T21:14:20.062047+0000","last_became_active":"2024-05-22T21:14:18.682581+0000","last_became_peered":"2024-05-22T21:14:18.682581+0000","last_unstale":"2024-05-22T21:14:20.062047+0000","last_undegraded":"2024-05-22T21:14:20.062047+0000","last_fullsized":"2024-05-22T21:14:20.062047+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:50:40.885861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061378+0000","last_change":"2024-05-22T21:14:19.076098+0000","last_active":"2024-05-22T21:14:20.061378+0000","last_peered":"2024-05-22T21:14:20.061378+0000","last_clean":"2024-05-22T21:14:20.061378+0000","last_became_active":"2024-05-22T21:14:19.074328+0000","last_became_peered":"2024-05-22T21:14:19.074328+0000","last_unstale":"2024-05-22T21:14:20.061378+0000","last_undegraded":"2024-05-22T21:14:20.061378+0000","last_fullsized":"2024-05-22T21:14:20.061378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:35:25.784368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058518+0000","last_change":"2024-05-22T21:14:18.689372+0000","last_active":"2024-05-22T21:14:20.058518+0000","last_peered":"2024-05-22T21:14:20.058518+0000","last_clean":"2024-05-22T21:14:20.058518+0000","last_became_active":"2024-05-22T21:14:18.689134+0000","last_became_peered":"2024-05-22T21:14:18.689134+0000","last_unstale":"2024-05-22T21:14:20.058518+0000","last_undegraded":"2024-05-22T21:14:20.058518+0000","last_fullsized":"2024-05-22T21:14:20.058518+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:50:40.885861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058373+0000","last_change":"2024-05-22T21:14:19.073743+0000","last_active":"2024-05-22T21:14:20.058373+0000","last_peered":"2024-05-22T21:14:20.058373+0000","last_clean":"2024-05-22T21:14:20.058373+0000","last_became_active":"2024-05-22T21:14:19.073475+0000","last_became_peered":"2024-05-22T21:14:19.073475+0000","last_unstale":"2024-05-22T21:14:20.058373+0000","last_undegraded":"2024-05-22T21:14:20.058373+0000","last_fullsized":"2024-05-22T21:14:20.058373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:35:25.784368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063004+0000","last_change":"2024-05-22T21:14:18.689431+0000","last_active":"2024-05-22T21:14:20.063004+0000","last_peered":"2024-05-22T21:14:20.063004+0000","last_clean":"2024-05-22T21:14:20.063004+0000","last_became_active":"2024-05-22T21:14:18.688601+0000","last_became_peered":"2024-05-22T21:14:18.688601+0000","last_unstale":"2024-05-22T21:14:20.063004+0000","last_undegraded":"2024-05-22T21:14:20.063004+0000","last_fullsized":"2024-05-22T21:14:20.063004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:16:48.710431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062161+0000","last_change":"2024-05-22T21:14:19.071723+0000","last_active":"2024-05-22T21:14:20.062161+0000","last_peered":"2024-05-22T21:14:20.062161+0000","last_clean":"2024-05-22T21:14:20.062161+0000","last_became_active":"2024-05-22T21:14:19.064677+0000","last_became_peered":"2024-05-22T21:14:19.064677+0000","last_unstale":"2024-05-22T21:14:20.062161+0000","last_undegraded":"2024-05-22T21:14:20.062161+0000","last_fullsized":"2024-05-22T21:14:20.062161+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:49:14.649560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062576+0000","last_change":"2024-05-22T21:14:18.688659+0000","last_active":"2024-05-22T21:14:20.062576+0000","last_peered":"2024-05-22T21:14:20.062576+0000","last_clean":"2024-05-22T21:14:20.062576+0000","last_became_active":"2024-05-22T21:14:18.688437+0000","last_became_peered":"2024-05-22T21:14:18.688437+0000","last_unstale":"2024-05-22T21:14:20.062576+0000","last_undegraded":"2024-05-22T21:14:20.062576+0000","last_fullsized":"2024-05-22T21:14:20.062576+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:04:53.771604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062354+0000","last_change":"2024-05-22T21:14:19.069813+0000","last_active":"2024-05-22T21:14:20.062354+0000","last_peered":"2024-05-22T21:14:20.062354+0000","last_clean":"2024-05-22T21:14:20.062354+0000","last_became_active":"2024-05-22T21:14:19.066103+0000","last_became_peered":"2024-05-22T21:14:19.066103+0000","last_unstale":"2024-05-22T21:14:20.062354+0000","last_undegraded":"2024-05-22T21:14:20.062354+0000","last_fullsized":"2024-05-22T21:14:20.062354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:21:48.071027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062599+0000","last_change":"2024-05-22T21:14:18.688154+0000","last_active":"2024-05-22T21:14:20.062599+0000","last_peered":"2024-05-22T21:14:20.062599+0000","last_clean":"2024-05-22T21:14:20.062599+0000","last_became_active":"2024-05-22T21:14:18.683773+0000","last_became_peered":"2024-05-22T21:14:18.683773+0000","last_unstale":"2024-05-22T21:14:20.062599+0000","last_undegraded":"2024-05-22T21:14:20.062599+0000","last_fullsized":"2024-05-22T21:14:20.062599+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:53:55.959523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062080+0000","last_change":"2024-05-22T21:14:19.071943+0000","last_active":"2024-05-22T21:14:20.062080+0000","last_peered":"2024-05-22T21:14:20.062080+0000","last_clean":"2024-05-22T21:14:20.062080+0000","last_became_active":"2024-05-22T21:14:19.071565+0000","last_became_peered":"2024-05-22T21:14:19.071565+0000","last_unstale":"2024-05-22T21:14:20.062080+0000","last_undegraded":"2024-05-22T21:14:20.062080+0000","last_fullsized":"2024-05-22T21:14:20.062080+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:45:18.511639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.060942+0000","last_change":"2024-05-22T21:14:18.685487+0000","last_active":"2024-05-22T21:14:20.060942+0000","last_peered":"2024-05-22T21:14:20.060942+0000","last_clean":"2024-05-22T21:14:20.060942+0000","last_became_active":"2024-05-22T21:14:18.685270+0000","last_became_peered":"2024-05-22T21:14:18.685270+0000","last_unstale":"2024-05-22T21:14:20.060942+0000","last_undegraded":"2024-05-22T21:14:20.060942+0000","last_fullsized":"2024-05-22T21:14:20.060942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:02:14.974395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-22T21:14:19.057371+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:14:19.057371+0000","last_peered":"2024-05-22T21:14:19.057371+0000","last_clean":"2024-05-22T21:14:19.057371+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:14:19.057371+0000","last_undegraded":"2024-05-22T21:14:19.057371+0000","last_fullsized":"2024-05-22T21:14:19.057371+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":717,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199265294,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.44700000000000001}]},{"osd":1,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":3.0920000000000001,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":3.1499999999999999,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.39900000000000002}]},{"osd":2,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.46999999999999997}]},{"osd":3,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.54600000000000004}]},{"osd":4,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.625},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.67000000000000004}]},{"osd":5,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.504}]},{"osd":6,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.63900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":141,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195171,"statfs":{"total":200000000,"available":199855136,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.73899999999999999}]},{"osd":1,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.82099999999999995}]},{"osd":2,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.80500000000000005}]},{"osd":3,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.61899999999999999}]},{"osd":4,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.752},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.65500000000000003}]},{"osd":7,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.83399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":141,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195170,"statfs":{"total":200000000,"available":199855080,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.82099999999999995}]},{"osd":1,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.70699999999999996}]},{"osd":2,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.747},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.76900000000000002}]},{"osd":3,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.66300000000000003}]},{"osd":6,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.629}]},{"osd":7,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.68799999999999994}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":140,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195171,"statfs":{"total":200000000,"available":199855648,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.52900000000000003}]},{"osd":1,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.50600000000000001}]},{"osd":2,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.58399999999999996}]},{"osd":3,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.629},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.48999999999999999}]},{"osd":6,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.56299999999999994}]},{"osd":7,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.61599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":142,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195170,"statfs":{"total":200000000,"available":199854166,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.245},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.32800000000000001}]},{"osd":1,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.35399999999999998}]},{"osd":2,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.45900000000000002}]},{"osd":4,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.75600000000000001}]},{"osd":5,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.60199999999999998}]},{"osd":6,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.65700000000000003}]},{"osd":7,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.749}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":140,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195171,"statfs":{"total":200000000,"available":199855682,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.29099999999999998}]},{"osd":1,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.316},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.41599999999999998}]},{"osd":3,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.38800000000000001}]},{"osd":4,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65500000000000003}]},{"osd":5,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.58099999999999996}]},{"osd":6,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.56399999999999995}]},{"osd":7,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.628}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":142,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195170,"statfs":{"total":200000000,"available":199854166,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.42999999999999999}]},{"osd":2,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.49099999999999999}]},{"osd":3,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.378},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47799999999999998}]},{"osd":4,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.68000000000000005}]},{"osd":5,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.748}]},{"osd":6,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.76200000000000001}]},{"osd":7,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.69599999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":716,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199266003,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.436}]},{"osd":2,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.45800000000000002}]},{"osd":3,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.751}]},{"osd":5,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.89000000000000001}]},{"osd":6,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.998}]},{"osd":7,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.77200000000000002}]}]}],"pool_statfs":[]}} 2024-05-22T21:14:23.146 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:14:23.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.316+0000 7ff5d0000640 1 Processor -- start 2024-05-22T21:14:23.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.316+0000 7ff5d0000640 1 -- start start 2024-05-22T21:14:23.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.316+0000 7ff5d0000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff5c80b1df0 0x7ff5c815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:23.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.316+0000 7ff5d0000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c80b3560 con 0x7ff5c80cbda0 2024-05-22T21:14:23.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.316+0000 7ff5d0000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c80cf530 con 0x7ff5c815e7d0 2024-05-22T21:14:23.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.316+0000 7ff5d0000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c80b37b0 con 0x7ff5c80b1df0 2024-05-22T21:14:23.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.316+0000 7ff5ccc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff5c80b1df0 0x7ff5c815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:23.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.316+0000 7ff5ce000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff5c80cbda0 0x7ff5c80cc230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:38350/0 (socket says 172.21.15.161:38350) 2024-05-22T21:14:23.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.316+0000 7ff5ccc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff5c80b1df0 0x7ff5c815f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49030/0 (socket says 172.21.15.161:49030) 2024-05-22T21:14:23.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.316+0000 7ff5ce000640 1 -- 172.21.15.161:0/3545630340 learned_addr learned my addr 172.21.15.161:0/3545630340 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:23.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.316+0000 7ff5ccc00640 1 -- 172.21.15.161:0/3545630340 >> v1:172.21.15.179:6789/0 conn(0x7ff5c815e7d0 legacy=0x7ff5c815eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:23.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.316+0000 7ff5ccc00640 1 -- 172.21.15.161:0/3545630340 >> v1:172.21.15.161:6789/0 conn(0x7ff5c80cbda0 legacy=0x7ff5c80cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:23.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.316+0000 7ff5ccc00640 1 -- 172.21.15.161:0/3545630340 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c80cf780 con 0x7ff5c80b1df0 2024-05-22T21:14:23.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.316+0000 7ff5c7e00640 1 -- 172.21.15.161:0/3545630340 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164681954 0 0) 0x7ff5c80b3560 con 0x7ff5c80cbda0 2024-05-22T21:14:23.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.320+0000 7ff5ccc00640 1 --2- 172.21.15.161:0/3545630340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c80b1df0 0x7ff5c815f5e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40097c0 tx=0x7ff5b401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57abd9544821ba7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:23.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.320+0000 7ff5c7e00640 1 -- 172.21.15.161:0/3545630340 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40ab670 con 0x7ff5c80b1df0 2024-05-22T21:14:23.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.320+0000 7ff5c7e00640 1 -- 172.21.15.161:0/3545630340 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b40ab7d0 con 0x7ff5c80b1df0 2024-05-22T21:14:23.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.320+0000 7ff5c7e00640 1 -- 172.21.15.161:0/3545630340 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40abb60 con 0x7ff5c80b1df0 2024-05-22T21:14:23.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.320+0000 7ff5d0000640 1 -- 172.21.15.161:0/3545630340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c80b1df0 msgr2=0x7ff5c815f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:23.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.320+0000 7ff5d0000640 1 --2- 172.21.15.161:0/3545630340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c80b1df0 0x7ff5c815f5e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40097c0 tx=0x7ff5b401c470 comp rx=0 tx=0).stop 2024-05-22T21:14:23.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.320+0000 7ff5d0000640 1 -- 172.21.15.161:0/3545630340 shutdown_connections 2024-05-22T21:14:23.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.320+0000 7ff5d0000640 1 --2- 172.21.15.161:0/3545630340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c80b1df0 0x7ff5c815f5e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.320+0000 7ff5d0000640 1 -- 172.21.15.161:0/3545630340 >> 172.21.15.161:0/3545630340 conn(0x7ff5c80b2a40 msgr2=0x7ff5c80b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:23.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.320+0000 7ff5d0000640 1 -- 172.21.15.161:0/3545630340 shutdown_connections 2024-05-22T21:14:23.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.320+0000 7ff5d0000640 1 -- 172.21.15.161:0/3545630340 wait complete. 2024-05-22T21:14:23.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5d0000640 1 Processor -- start 2024-05-22T21:14:23.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5d0000640 1 -- start start 2024-05-22T21:14:23.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5d0000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c80b1df0 0x7ff5c8156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:23.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5d0000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5c80cbda0 0x7ff5c8157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:23.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5d0000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5c815e7d0 0x7ff5c8155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:23.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5d0000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c8163310 con 0x7ff5c80b1df0 2024-05-22T21:14:23.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5d0000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80b5580 con 0x7ff5c80cbda0 2024-05-22T21:14:23.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5d0000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c81635b0 con 0x7ff5c815e7d0 2024-05-22T21:14:23.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5cd600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c80b1df0 0x7ff5c8156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:23.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5cd600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c80b1df0 0x7ff5c8156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49038/0 (socket says 172.21.15.161:49038) 2024-05-22T21:14:23.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5cd600640 1 -- 172.21.15.161:0/276008914 learned_addr learned my addr 172.21.15.161:0/276008914 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:23.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5ccc00640 1 --2- 172.21.15.161:0/276008914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5c80cbda0 0x7ff5c8157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:23.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5ce000640 1 --2- 172.21.15.161:0/276008914 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5c815e7d0 0x7ff5c8155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:23.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5cd600640 1 -- 172.21.15.161:0/276008914 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5c815e7d0 msgr2=0x7ff5c8155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:23.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5cd600640 1 --2- 172.21.15.161:0/276008914 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5c815e7d0 0x7ff5c8155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5cd600640 1 -- 172.21.15.161:0/276008914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5c80cbda0 msgr2=0x7ff5c8157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:23.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5cd600640 1 --2- 172.21.15.161:0/276008914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5c80cbda0 0x7ff5c8157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5cd600640 1 -- 172.21.15.161:0/276008914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b4009420 con 0x7ff5c80b1df0 2024-05-22T21:14:23.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5cd600640 1 --2- 172.21.15.161:0/276008914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c80b1df0 0x7ff5c8156c30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00c500 tx=0x7ff5bc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:23.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5c5600640 1 -- 172.21.15.161:0/276008914 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc005220 con 0x7ff5c80b1df0 2024-05-22T21:14:23.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5c5600640 1 -- 172.21.15.161:0/276008914 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc005380 con 0x7ff5c80b1df0 2024-05-22T21:14:23.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5c5600640 1 -- 172.21.15.161:0/276008914 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc007270 con 0x7ff5c80b1df0 2024-05-22T21:14:23.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5d0000640 1 -- 172.21.15.161:0/276008914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c8163890 con 0x7ff5c80b1df0 2024-05-22T21:14:23.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.324+0000 7ff5d0000640 1 -- 172.21.15.161:0/276008914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c8157170 con 0x7ff5c80b1df0 2024-05-22T21:14:23.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.328+0000 7ff5d0000640 1 -- 172.21.15.161:0/276008914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c8067220 con 0x7ff5c80b1df0 2024-05-22T21:14:23.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.328+0000 7ff5c5600640 1 -- 172.21.15.161:0/276008914 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff5bc0073d0 con 0x7ff5c80b1df0 2024-05-22T21:14:23.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.328+0000 7ff5c5600640 1 --2- 172.21.15.161:0/276008914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff5a8066690 0x7ff5a8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:23.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.328+0000 7ff5c5600640 1 -- 172.21.15.161:0/276008914 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5bc0839e0 con 0x7ff5c80b1df0 2024-05-22T21:14:23.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.328+0000 7ff5ccc00640 1 --2- 172.21.15.161:0/276008914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff5a8066690 0x7ff5a8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:23.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.328+0000 7ff5ccc00640 1 --2- 172.21.15.161:0/276008914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff5a8066690 0x7ff5a8068b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40af000 tx=0x7ff5b40a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:23.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.332+0000 7ff5c5600640 1 -- 172.21.15.161:0/276008914 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff5bc050d70 con 0x7ff5c80b1df0 2024-05-22T21:14:23.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.484+0000 7ff5d0000640 1 -- 172.21.15.161:0/276008914 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5c80cfd40 con 0x7ff5a8066690 2024-05-22T21:14:23.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.492+0000 7ff5c5600640 1 -- 172.21.15.161:0/276008914 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284490 (secure 0 0 0) 0x7ff5b40ab9c0 con 0x7ff5a8066690 2024-05-22T21:14:23.496 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:23.497 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:14:23.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.496+0000 7ff5d0000640 1 -- 172.21.15.161:0/276008914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff5a8066690 msgr2=0x7ff5a8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:23.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.496+0000 7ff5d0000640 1 --2- 172.21.15.161:0/276008914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff5a8066690 0x7ff5a8068b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40af000 tx=0x7ff5b40a7000 comp rx=0 tx=0).stop 2024-05-22T21:14:23.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.496+0000 7ff5d0000640 1 -- 172.21.15.161:0/276008914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c80b1df0 msgr2=0x7ff5c8156c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:23.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.496+0000 7ff5d0000640 1 --2- 172.21.15.161:0/276008914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c80b1df0 0x7ff5c8156c30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00c500 tx=0x7ff5bc00c9c0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.496+0000 7ff5d0000640 1 -- 172.21.15.161:0/276008914 shutdown_connections 2024-05-22T21:14:23.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.496+0000 7ff5d0000640 1 --2- 172.21.15.161:0/276008914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff5a8066690 0x7ff5a8068b50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.496+0000 7ff5d0000640 1 --2- 172.21.15.161:0/276008914 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5c815e7d0 0x7ff5c8155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.496+0000 7ff5d0000640 1 --2- 172.21.15.161:0/276008914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5c80cbda0 0x7ff5c8157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.496+0000 7ff5d0000640 1 --2- 172.21.15.161:0/276008914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c80b1df0 0x7ff5c8156c30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.496+0000 7ff5d0000640 1 -- 172.21.15.161:0/276008914 >> 172.21.15.161:0/276008914 conn(0x7ff5c80b2a40 msgr2=0x7ff5c815b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:23.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.496+0000 7ff5d0000640 1 -- 172.21.15.161:0/276008914 shutdown_connections 2024-05-22T21:14:23.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.496+0000 7ff5d0000640 1 -- 172.21.15.161:0/276008914 wait complete. 2024-05-22T21:14:23.517 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-05-22T21:14:22.406717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2284,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560215,"statfs":{"total":1600000000,"available":1597661175,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.740437"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061559+0000","last_change":"2024-05-22T21:14:19.075809+0000","last_active":"2024-05-22T21:14:20.061559+0000","last_peered":"2024-05-22T21:14:20.061559+0000","last_clean":"2024-05-22T21:14:20.061559+0000","last_became_active":"2024-05-22T21:14:19.075583+0000","last_became_peered":"2024-05-22T21:14:19.075583+0000","last_unstale":"2024-05-22T21:14:20.061559+0000","last_undegraded":"2024-05-22T21:14:20.061559+0000","last_fullsized":"2024-05-22T21:14:20.061559+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:16:57.714956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061731+0000","last_change":"2024-05-22T21:14:19.069739+0000","last_active":"2024-05-22T21:14:20.061731+0000","last_peered":"2024-05-22T21:14:20.061731+0000","last_clean":"2024-05-22T21:14:20.061731+0000","last_became_active":"2024-05-22T21:14:19.069232+0000","last_became_peered":"2024-05-22T21:14:19.069232+0000","last_unstale":"2024-05-22T21:14:20.061731+0000","last_undegraded":"2024-05-22T21:14:20.061731+0000","last_fullsized":"2024-05-22T21:14:20.061731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:49:08.296970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062432+0000","last_change":"2024-05-22T21:14:19.078965+0000","last_active":"2024-05-22T21:14:20.062432+0000","last_peered":"2024-05-22T21:14:20.062432+0000","last_clean":"2024-05-22T21:14:20.062432+0000","last_became_active":"2024-05-22T21:14:19.077828+0000","last_became_peered":"2024-05-22T21:14:19.077828+0000","last_unstale":"2024-05-22T21:14:20.062432+0000","last_undegraded":"2024-05-22T21:14:20.062432+0000","last_fullsized":"2024-05-22T21:14:20.062432+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:53:03.531926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061761+0000","last_change":"2024-05-22T21:14:19.074084+0000","last_active":"2024-05-22T21:14:20.061761+0000","last_peered":"2024-05-22T21:14:20.061761+0000","last_clean":"2024-05-22T21:14:20.061761+0000","last_became_active":"2024-05-22T21:14:19.069975+0000","last_became_peered":"2024-05-22T21:14:19.069975+0000","last_unstale":"2024-05-22T21:14:20.061761+0000","last_undegraded":"2024-05-22T21:14:20.061761+0000","last_fullsized":"2024-05-22T21:14:20.061761+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:24:45.072721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061560+0000","last_change":"2024-05-22T21:14:19.070712+0000","last_active":"2024-05-22T21:14:20.061560+0000","last_peered":"2024-05-22T21:14:20.061560+0000","last_clean":"2024-05-22T21:14:20.061560+0000","last_became_active":"2024-05-22T21:14:19.070166+0000","last_became_peered":"2024-05-22T21:14:19.070166+0000","last_unstale":"2024-05-22T21:14:20.061560+0000","last_undegraded":"2024-05-22T21:14:20.061560+0000","last_fullsized":"2024-05-22T21:14:20.061560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:31:44.124278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062880+0000","last_change":"2024-05-22T21:14:19.073061+0000","last_active":"2024-05-22T21:14:20.062880+0000","last_peered":"2024-05-22T21:14:20.062880+0000","last_clean":"2024-05-22T21:14:20.062880+0000","last_became_active":"2024-05-22T21:14:19.072746+0000","last_became_peered":"2024-05-22T21:14:19.072746+0000","last_unstale":"2024-05-22T21:14:20.062880+0000","last_undegraded":"2024-05-22T21:14:20.062880+0000","last_fullsized":"2024-05-22T21:14:20.062880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:53:03.531926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061196+0000","last_change":"2024-05-22T21:14:19.067746+0000","last_active":"2024-05-22T21:14:20.061196+0000","last_peered":"2024-05-22T21:14:20.061196+0000","last_clean":"2024-05-22T21:14:20.061196+0000","last_became_active":"2024-05-22T21:14:19.067268+0000","last_became_peered":"2024-05-22T21:14:19.067268+0000","last_unstale":"2024-05-22T21:14:20.061196+0000","last_undegraded":"2024-05-22T21:14:20.061196+0000","last_fullsized":"2024-05-22T21:14:20.061196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:15:46.140480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061203+0000","last_change":"2024-05-22T21:14:19.066178+0000","last_active":"2024-05-22T21:14:20.061203+0000","last_peered":"2024-05-22T21:14:20.061203+0000","last_clean":"2024-05-22T21:14:20.061203+0000","last_became_active":"2024-05-22T21:14:19.065931+0000","last_became_peered":"2024-05-22T21:14:19.065931+0000","last_unstale":"2024-05-22T21:14:20.061203+0000","last_undegraded":"2024-05-22T21:14:20.061203+0000","last_fullsized":"2024-05-22T21:14:20.061203+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:05:45.991300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063496+0000","last_change":"2024-05-22T21:14:19.065468+0000","last_active":"2024-05-22T21:14:20.063496+0000","last_peered":"2024-05-22T21:14:20.063496+0000","last_clean":"2024-05-22T21:14:20.063496+0000","last_became_active":"2024-05-22T21:14:19.065221+0000","last_became_peered":"2024-05-22T21:14:19.065221+0000","last_unstale":"2024-05-22T21:14:20.063496+0000","last_undegraded":"2024-05-22T21:14:20.063496+0000","last_fullsized":"2024-05-22T21:14:20.063496+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:34:53.068191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061737+0000","last_change":"2024-05-22T21:14:19.070117+0000","last_active":"2024-05-22T21:14:20.061737+0000","last_peered":"2024-05-22T21:14:20.061737+0000","last_clean":"2024-05-22T21:14:20.061737+0000","last_became_active":"2024-05-22T21:14:19.068399+0000","last_became_peered":"2024-05-22T21:14:19.068399+0000","last_unstale":"2024-05-22T21:14:20.061737+0000","last_undegraded":"2024-05-22T21:14:20.061737+0000","last_fullsized":"2024-05-22T21:14:20.061737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:54:26.183700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061757+0000","last_change":"2024-05-22T21:14:19.066353+0000","last_active":"2024-05-22T21:14:20.061757+0000","last_peered":"2024-05-22T21:14:20.061757+0000","last_clean":"2024-05-22T21:14:20.061757+0000","last_became_active":"2024-05-22T21:14:19.065776+0000","last_became_peered":"2024-05-22T21:14:19.065776+0000","last_unstale":"2024-05-22T21:14:20.061757+0000","last_undegraded":"2024-05-22T21:14:20.061757+0000","last_fullsized":"2024-05-22T21:14:20.061757+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:24:45.072721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062356+0000","last_change":"2024-05-22T21:14:19.072669+0000","last_active":"2024-05-22T21:14:20.062356+0000","last_peered":"2024-05-22T21:14:20.062356+0000","last_clean":"2024-05-22T21:14:20.062356+0000","last_became_active":"2024-05-22T21:14:19.072116+0000","last_became_peered":"2024-05-22T21:14:19.072116+0000","last_unstale":"2024-05-22T21:14:20.062356+0000","last_undegraded":"2024-05-22T21:14:20.062356+0000","last_fullsized":"2024-05-22T21:14:20.062356+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:37:11.967585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061279+0000","last_change":"2024-05-22T21:14:19.075444+0000","last_active":"2024-05-22T21:14:20.061279+0000","last_peered":"2024-05-22T21:14:20.061279+0000","last_clean":"2024-05-22T21:14:20.061279+0000","last_became_active":"2024-05-22T21:14:19.074141+0000","last_became_peered":"2024-05-22T21:14:19.074141+0000","last_unstale":"2024-05-22T21:14:20.061279+0000","last_undegraded":"2024-05-22T21:14:20.061279+0000","last_fullsized":"2024-05-22T21:14:20.061279+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:34:13.649968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062305+0000","last_change":"2024-05-22T21:14:19.063930+0000","last_active":"2024-05-22T21:14:20.062305+0000","last_peered":"2024-05-22T21:14:20.062305+0000","last_clean":"2024-05-22T21:14:20.062305+0000","last_became_active":"2024-05-22T21:14:19.063748+0000","last_became_peered":"2024-05-22T21:14:19.063748+0000","last_unstale":"2024-05-22T21:14:20.062305+0000","last_undegraded":"2024-05-22T21:14:20.062305+0000","last_fullsized":"2024-05-22T21:14:20.062305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:05:45.991300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062721+0000","last_change":"2024-05-22T21:14:19.068633+0000","last_active":"2024-05-22T21:14:20.062721+0000","last_peered":"2024-05-22T21:14:20.062721+0000","last_clean":"2024-05-22T21:14:20.062721+0000","last_became_active":"2024-05-22T21:14:19.068418+0000","last_became_peered":"2024-05-22T21:14:19.068418+0000","last_unstale":"2024-05-22T21:14:20.062721+0000","last_undegraded":"2024-05-22T21:14:20.062721+0000","last_fullsized":"2024-05-22T21:14:20.062721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:49:14.649560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061841+0000","last_change":"2024-05-22T21:14:19.068078+0000","last_active":"2024-05-22T21:14:20.061841+0000","last_peered":"2024-05-22T21:14:20.061841+0000","last_clean":"2024-05-22T21:14:20.061841+0000","last_became_active":"2024-05-22T21:14:19.066115+0000","last_became_peered":"2024-05-22T21:14:19.066115+0000","last_unstale":"2024-05-22T21:14:20.061841+0000","last_undegraded":"2024-05-22T21:14:20.061841+0000","last_fullsized":"2024-05-22T21:14:20.061841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:49:08.296970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061288+0000","last_change":"2024-05-22T21:14:19.067547+0000","last_active":"2024-05-22T21:14:20.061288+0000","last_peered":"2024-05-22T21:14:20.061288+0000","last_clean":"2024-05-22T21:14:20.061288+0000","last_became_active":"2024-05-22T21:14:19.066896+0000","last_became_peered":"2024-05-22T21:14:19.066896+0000","last_unstale":"2024-05-22T21:14:20.061288+0000","last_undegraded":"2024-05-22T21:14:20.061288+0000","last_fullsized":"2024-05-22T21:14:20.061288+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:53:03.531926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062296+0000","last_change":"2024-05-22T21:14:19.064735+0000","last_active":"2024-05-22T21:14:20.062296+0000","last_peered":"2024-05-22T21:14:20.062296+0000","last_clean":"2024-05-22T21:14:20.062296+0000","last_became_active":"2024-05-22T21:14:19.064532+0000","last_became_peered":"2024-05-22T21:14:19.064532+0000","last_unstale":"2024-05-22T21:14:20.062296+0000","last_undegraded":"2024-05-22T21:14:20.062296+0000","last_fullsized":"2024-05-22T21:14:20.062296+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:58:20.679986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062076+0000","last_change":"2024-05-22T21:14:19.071883+0000","last_active":"2024-05-22T21:14:20.062076+0000","last_peered":"2024-05-22T21:14:20.062076+0000","last_clean":"2024-05-22T21:14:20.062076+0000","last_became_active":"2024-05-22T21:14:19.064966+0000","last_became_peered":"2024-05-22T21:14:19.064966+0000","last_unstale":"2024-05-22T21:14:20.062076+0000","last_undegraded":"2024-05-22T21:14:20.062076+0000","last_fullsized":"2024-05-22T21:14:20.062076+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:01:07.386423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061637+0000","last_change":"2024-05-22T21:14:19.076484+0000","last_active":"2024-05-22T21:14:20.061637+0000","last_peered":"2024-05-22T21:14:20.061637+0000","last_clean":"2024-05-22T21:14:20.061637+0000","last_became_active":"2024-05-22T21:14:19.076078+0000","last_became_peered":"2024-05-22T21:14:19.076078+0000","last_unstale":"2024-05-22T21:14:20.061637+0000","last_undegraded":"2024-05-22T21:14:20.061637+0000","last_fullsized":"2024-05-22T21:14:20.061637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:50:41.893275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062920+0000","last_change":"2024-05-22T21:14:19.072360+0000","last_active":"2024-05-22T21:14:20.062920+0000","last_peered":"2024-05-22T21:14:20.062920+0000","last_clean":"2024-05-22T21:14:20.062920+0000","last_became_active":"2024-05-22T21:14:19.072078+0000","last_became_peered":"2024-05-22T21:14:19.072078+0000","last_unstale":"2024-05-22T21:14:20.062920+0000","last_undegraded":"2024-05-22T21:14:20.062920+0000","last_fullsized":"2024-05-22T21:14:20.062920+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:16:57.714956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063301+0000","last_change":"2024-05-22T21:14:19.064748+0000","last_active":"2024-05-22T21:14:20.063301+0000","last_peered":"2024-05-22T21:14:20.063301+0000","last_clean":"2024-05-22T21:14:20.063301+0000","last_became_active":"2024-05-22T21:14:19.063910+0000","last_became_peered":"2024-05-22T21:14:19.063910+0000","last_unstale":"2024-05-22T21:14:20.063301+0000","last_undegraded":"2024-05-22T21:14:20.063301+0000","last_fullsized":"2024-05-22T21:14:20.063301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:18:23.818225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058362+0000","last_change":"2024-05-22T21:14:19.069656+0000","last_active":"2024-05-22T21:14:20.058362+0000","last_peered":"2024-05-22T21:14:20.058362+0000","last_clean":"2024-05-22T21:14:20.058362+0000","last_became_active":"2024-05-22T21:14:19.068999+0000","last_became_peered":"2024-05-22T21:14:19.068999+0000","last_unstale":"2024-05-22T21:14:20.058362+0000","last_undegraded":"2024-05-22T21:14:20.058362+0000","last_fullsized":"2024-05-22T21:14:20.058362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:16:57.714956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062294+0000","last_change":"2024-05-22T21:14:19.076134+0000","last_active":"2024-05-22T21:14:20.062294+0000","last_peered":"2024-05-22T21:14:20.062294+0000","last_clean":"2024-05-22T21:14:20.062294+0000","last_became_active":"2024-05-22T21:14:19.075829+0000","last_became_peered":"2024-05-22T21:14:19.075829+0000","last_unstale":"2024-05-22T21:14:20.062294+0000","last_undegraded":"2024-05-22T21:14:20.062294+0000","last_fullsized":"2024-05-22T21:14:20.062294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:49:08.296970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061824+0000","last_change":"2024-05-22T21:14:19.070539+0000","last_active":"2024-05-22T21:14:20.061824+0000","last_peered":"2024-05-22T21:14:20.061824+0000","last_clean":"2024-05-22T21:14:20.061824+0000","last_became_active":"2024-05-22T21:14:19.070335+0000","last_became_peered":"2024-05-22T21:14:19.070335+0000","last_unstale":"2024-05-22T21:14:20.061824+0000","last_undegraded":"2024-05-22T21:14:20.061824+0000","last_fullsized":"2024-05-22T21:14:20.061824+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:45:18.511639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062960+0000","last_change":"2024-05-22T21:14:19.063816+0000","last_active":"2024-05-22T21:14:20.062960+0000","last_peered":"2024-05-22T21:14:20.062960+0000","last_clean":"2024-05-22T21:14:20.062960+0000","last_became_active":"2024-05-22T21:14:19.063622+0000","last_became_peered":"2024-05-22T21:14:19.063622+0000","last_unstale":"2024-05-22T21:14:20.062960+0000","last_undegraded":"2024-05-22T21:14:20.062960+0000","last_fullsized":"2024-05-22T21:14:20.062960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:34:07.543158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062931+0000","last_change":"2024-05-22T21:14:19.072106+0000","last_active":"2024-05-22T21:14:20.062931+0000","last_peered":"2024-05-22T21:14:20.062931+0000","last_clean":"2024-05-22T21:14:20.062931+0000","last_became_active":"2024-05-22T21:14:19.071872+0000","last_became_peered":"2024-05-22T21:14:19.071872+0000","last_unstale":"2024-05-22T21:14:20.062931+0000","last_undegraded":"2024-05-22T21:14:20.062931+0000","last_fullsized":"2024-05-22T21:14:20.062931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:58:20.679986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062010+0000","last_change":"2024-05-22T21:14:19.074296+0000","last_active":"2024-05-22T21:14:20.062010+0000","last_peered":"2024-05-22T21:14:20.062010+0000","last_clean":"2024-05-22T21:14:20.062010+0000","last_became_active":"2024-05-22T21:14:19.073949+0000","last_became_peered":"2024-05-22T21:14:19.073949+0000","last_unstale":"2024-05-22T21:14:20.062010+0000","last_undegraded":"2024-05-22T21:14:20.062010+0000","last_fullsized":"2024-05-22T21:14:20.062010+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:34:07.543158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063604+0000","last_change":"2024-05-22T21:14:19.072137+0000","last_active":"2024-05-22T21:14:20.063604+0000","last_peered":"2024-05-22T21:14:20.063604+0000","last_clean":"2024-05-22T21:14:20.063604+0000","last_became_active":"2024-05-22T21:14:19.071797+0000","last_became_peered":"2024-05-22T21:14:19.071797+0000","last_unstale":"2024-05-22T21:14:20.063604+0000","last_undegraded":"2024-05-22T21:14:20.063604+0000","last_fullsized":"2024-05-22T21:14:20.063604+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:57:23.815795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061929+0000","last_change":"2024-05-22T21:14:19.068998+0000","last_active":"2024-05-22T21:14:20.061929+0000","last_peered":"2024-05-22T21:14:20.061929+0000","last_clean":"2024-05-22T21:14:20.061929+0000","last_became_active":"2024-05-22T21:14:19.068628+0000","last_became_peered":"2024-05-22T21:14:19.068628+0000","last_unstale":"2024-05-22T21:14:20.061929+0000","last_undegraded":"2024-05-22T21:14:20.061929+0000","last_fullsized":"2024-05-22T21:14:20.061929+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:34:07.543158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061647+0000","last_change":"2024-05-22T21:14:19.069251+0000","last_active":"2024-05-22T21:14:20.061647+0000","last_peered":"2024-05-22T21:14:20.061647+0000","last_clean":"2024-05-22T21:14:20.061647+0000","last_became_active":"2024-05-22T21:14:19.068914+0000","last_became_peered":"2024-05-22T21:14:19.068914+0000","last_unstale":"2024-05-22T21:14:20.061647+0000","last_undegraded":"2024-05-22T21:14:20.061647+0000","last_fullsized":"2024-05-22T21:14:20.061647+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:21:48.071027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061385+0000","last_change":"2024-05-22T21:14:19.062575+0000","last_active":"2024-05-22T21:14:20.061385+0000","last_peered":"2024-05-22T21:14:20.061385+0000","last_clean":"2024-05-22T21:14:20.061385+0000","last_became_active":"2024-05-22T21:14:19.062253+0000","last_became_peered":"2024-05-22T21:14:19.062253+0000","last_unstale":"2024-05-22T21:14:20.061385+0000","last_undegraded":"2024-05-22T21:14:20.061385+0000","last_fullsized":"2024-05-22T21:14:20.061385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:31:30.588601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058420+0000","last_change":"2024-05-22T21:14:19.073511+0000","last_active":"2024-05-22T21:14:20.058420+0000","last_peered":"2024-05-22T21:14:20.058420+0000","last_clean":"2024-05-22T21:14:20.058420+0000","last_became_active":"2024-05-22T21:14:19.073297+0000","last_became_peered":"2024-05-22T21:14:19.073297+0000","last_unstale":"2024-05-22T21:14:20.058420+0000","last_undegraded":"2024-05-22T21:14:20.058420+0000","last_fullsized":"2024-05-22T21:14:20.058420+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:21:48.071027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061044+0000","last_change":"2024-05-22T21:14:18.687958+0000","last_active":"2024-05-22T21:14:20.061044+0000","last_peered":"2024-05-22T21:14:20.061044+0000","last_clean":"2024-05-22T21:14:20.061044+0000","last_became_active":"2024-05-22T21:14:18.687753+0000","last_became_peered":"2024-05-22T21:14:18.687753+0000","last_unstale":"2024-05-22T21:14:20.061044+0000","last_undegraded":"2024-05-22T21:14:20.061044+0000","last_fullsized":"2024-05-22T21:14:20.061044+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:11:50.017938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.060946+0000","last_change":"2024-05-22T21:14:19.071430+0000","last_active":"2024-05-22T21:14:20.060946+0000","last_peered":"2024-05-22T21:14:20.060946+0000","last_clean":"2024-05-22T21:14:20.060946+0000","last_became_active":"2024-05-22T21:14:19.071148+0000","last_became_peered":"2024-05-22T21:14:19.071148+0000","last_unstale":"2024-05-22T21:14:20.060946+0000","last_undegraded":"2024-05-22T21:14:20.060946+0000","last_fullsized":"2024-05-22T21:14:20.060946+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:18:23.818225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058324+0000","last_change":"2024-05-22T21:14:18.682562+0000","last_active":"2024-05-22T21:14:20.058324+0000","last_peered":"2024-05-22T21:14:20.058324+0000","last_clean":"2024-05-22T21:14:20.058324+0000","last_became_active":"2024-05-22T21:14:18.681508+0000","last_became_peered":"2024-05-22T21:14:18.681508+0000","last_unstale":"2024-05-22T21:14:20.058324+0000","last_undegraded":"2024-05-22T21:14:20.058324+0000","last_fullsized":"2024-05-22T21:14:20.058324+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:16:48.710431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063061+0000","last_change":"2024-05-22T21:14:19.065676+0000","last_active":"2024-05-22T21:14:20.063061+0000","last_peered":"2024-05-22T21:14:20.063061+0000","last_clean":"2024-05-22T21:14:20.063061+0000","last_became_active":"2024-05-22T21:14:19.065397+0000","last_became_peered":"2024-05-22T21:14:19.065397+0000","last_unstale":"2024-05-22T21:14:20.063061+0000","last_undegraded":"2024-05-22T21:14:20.063061+0000","last_fullsized":"2024-05-22T21:14:20.063061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:53:56.966937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062376+0000","last_change":"2024-05-22T21:14:18.685300+0000","last_active":"2024-05-22T21:14:20.062376+0000","last_peered":"2024-05-22T21:14:20.062376+0000","last_clean":"2024-05-22T21:14:20.062376+0000","last_became_active":"2024-05-22T21:14:18.685102+0000","last_became_peered":"2024-05-22T21:14:18.685102+0000","last_unstale":"2024-05-22T21:14:20.062376+0000","last_undegraded":"2024-05-22T21:14:20.062376+0000","last_fullsized":"2024-05-22T21:14:20.062376+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:35:24.776954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061917+0000","last_change":"2024-05-22T21:14:19.076460+0000","last_active":"2024-05-22T21:14:20.061917+0000","last_peered":"2024-05-22T21:14:20.061917+0000","last_clean":"2024-05-22T21:14:20.061917+0000","last_became_active":"2024-05-22T21:14:19.072447+0000","last_became_peered":"2024-05-22T21:14:19.072447+0000","last_unstale":"2024-05-22T21:14:20.061917+0000","last_undegraded":"2024-05-22T21:14:20.061917+0000","last_fullsized":"2024-05-22T21:14:20.061917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:31:30.588601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062606+0000","last_change":"2024-05-22T21:14:18.687785+0000","last_active":"2024-05-22T21:14:20.062606+0000","last_peered":"2024-05-22T21:14:20.062606+0000","last_clean":"2024-05-22T21:14:20.062606+0000","last_became_active":"2024-05-22T21:14:18.687525+0000","last_became_peered":"2024-05-22T21:14:18.687525+0000","last_unstale":"2024-05-22T21:14:20.062606+0000","last_undegraded":"2024-05-22T21:14:20.062606+0000","last_fullsized":"2024-05-22T21:14:20.062606+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:38:56.042952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061358+0000","last_change":"2024-05-22T21:14:19.064148+0000","last_active":"2024-05-22T21:14:20.061358+0000","last_peered":"2024-05-22T21:14:20.061358+0000","last_clean":"2024-05-22T21:14:20.061358+0000","last_became_active":"2024-05-22T21:14:19.062245+0000","last_became_peered":"2024-05-22T21:14:19.062245+0000","last_unstale":"2024-05-22T21:14:20.061358+0000","last_undegraded":"2024-05-22T21:14:20.061358+0000","last_fullsized":"2024-05-22T21:14:20.061358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:22:06.703835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063240+0000","last_change":"2024-05-22T21:14:18.679232+0000","last_active":"2024-05-22T21:14:20.063240+0000","last_peered":"2024-05-22T21:14:20.063240+0000","last_clean":"2024-05-22T21:14:20.063240+0000","last_became_active":"2024-05-22T21:14:18.678977+0000","last_became_peered":"2024-05-22T21:14:18.678977+0000","last_unstale":"2024-05-22T21:14:20.063240+0000","last_undegraded":"2024-05-22T21:14:20.063240+0000","last_fullsized":"2024-05-22T21:14:20.063240+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:47:53.263656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062459+0000","last_change":"2024-05-22T21:14:19.073308+0000","last_active":"2024-05-22T21:14:20.062459+0000","last_peered":"2024-05-22T21:14:20.062459+0000","last_clean":"2024-05-22T21:14:20.062459+0000","last_became_active":"2024-05-22T21:14:19.072924+0000","last_became_peered":"2024-05-22T21:14:19.072924+0000","last_unstale":"2024-05-22T21:14:20.062459+0000","last_undegraded":"2024-05-22T21:14:20.062459+0000","last_fullsized":"2024-05-22T21:14:20.062459+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:01:07.386423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062441+0000","last_change":"2024-05-22T21:14:18.682839+0000","last_active":"2024-05-22T21:14:20.062441+0000","last_peered":"2024-05-22T21:14:20.062441+0000","last_clean":"2024-05-22T21:14:20.062441+0000","last_became_active":"2024-05-22T21:14:18.682559+0000","last_became_peered":"2024-05-22T21:14:18.682559+0000","last_unstale":"2024-05-22T21:14:20.062441+0000","last_undegraded":"2024-05-22T21:14:20.062441+0000","last_fullsized":"2024-05-22T21:14:20.062441+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:34:45.136012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062882+0000","last_change":"2024-05-22T21:14:19.070174+0000","last_active":"2024-05-22T21:14:20.062882+0000","last_peered":"2024-05-22T21:14:20.062882+0000","last_clean":"2024-05-22T21:14:20.062882+0000","last_became_active":"2024-05-22T21:14:19.069916+0000","last_became_peered":"2024-05-22T21:14:19.069916+0000","last_unstale":"2024-05-22T21:14:20.062882+0000","last_undegraded":"2024-05-22T21:14:20.062882+0000","last_fullsized":"2024-05-22T21:14:20.062882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:33:32.237400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062371+0000","last_change":"2024-05-22T21:14:18.688013+0000","last_active":"2024-05-22T21:14:20.062371+0000","last_peered":"2024-05-22T21:14:20.062371+0000","last_clean":"2024-05-22T21:14:20.062371+0000","last_became_active":"2024-05-22T21:14:18.682394+0000","last_became_peered":"2024-05-22T21:14:18.682394+0000","last_unstale":"2024-05-22T21:14:20.062371+0000","last_undegraded":"2024-05-22T21:14:20.062371+0000","last_fullsized":"2024-05-22T21:14:20.062371+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:40:57.258501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061843+0000","last_change":"2024-05-22T21:14:19.074879+0000","last_active":"2024-05-22T21:14:20.061843+0000","last_peered":"2024-05-22T21:14:20.061843+0000","last_clean":"2024-05-22T21:14:20.061843+0000","last_became_active":"2024-05-22T21:14:19.074290+0000","last_became_peered":"2024-05-22T21:14:19.074290+0000","last_unstale":"2024-05-22T21:14:20.061843+0000","last_undegraded":"2024-05-22T21:14:20.061843+0000","last_fullsized":"2024-05-22T21:14:20.061843+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:22:06.703835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063272+0000","last_change":"2024-05-22T21:14:18.685836+0000","last_active":"2024-05-22T21:14:20.063272+0000","last_peered":"2024-05-22T21:14:20.063272+0000","last_clean":"2024-05-22T21:14:20.063272+0000","last_became_active":"2024-05-22T21:14:18.684837+0000","last_became_peered":"2024-05-22T21:14:18.684837+0000","last_unstale":"2024-05-22T21:14:20.063272+0000","last_undegraded":"2024-05-22T21:14:20.063272+0000","last_fullsized":"2024-05-22T21:14:20.063272+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:43:45.138067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062736+0000","last_change":"2024-05-22T21:14:19.072396+0000","last_active":"2024-05-22T21:14:20.062736+0000","last_peered":"2024-05-22T21:14:20.062736+0000","last_clean":"2024-05-22T21:14:20.062736+0000","last_became_active":"2024-05-22T21:14:19.071611+0000","last_became_peered":"2024-05-22T21:14:19.071611+0000","last_unstale":"2024-05-22T21:14:20.062736+0000","last_undegraded":"2024-05-22T21:14:20.062736+0000","last_fullsized":"2024-05-22T21:14:20.062736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:35:41.126663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063086+0000","last_change":"2024-05-22T21:14:18.683506+0000","last_active":"2024-05-22T21:14:20.063086+0000","last_peered":"2024-05-22T21:14:20.063086+0000","last_clean":"2024-05-22T21:14:20.063086+0000","last_became_active":"2024-05-22T21:14:18.683251+0000","last_became_peered":"2024-05-22T21:14:18.683251+0000","last_unstale":"2024-05-22T21:14:20.063086+0000","last_undegraded":"2024-05-22T21:14:20.063086+0000","last_fullsized":"2024-05-22T21:14:20.063086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:34:52.060776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063037+0000","last_change":"2024-05-22T21:14:19.065214+0000","last_active":"2024-05-22T21:14:20.063037+0000","last_peered":"2024-05-22T21:14:20.063037+0000","last_clean":"2024-05-22T21:14:20.063037+0000","last_became_active":"2024-05-22T21:14:19.064985+0000","last_became_peered":"2024-05-22T21:14:19.064985+0000","last_unstale":"2024-05-22T21:14:20.063037+0000","last_undegraded":"2024-05-22T21:14:20.063037+0000","last_fullsized":"2024-05-22T21:14:20.063037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:35:41.126663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058462+0000","last_change":"2024-05-22T21:14:18.678981+0000","last_active":"2024-05-22T21:14:20.058462+0000","last_peered":"2024-05-22T21:14:20.058462+0000","last_clean":"2024-05-22T21:14:20.058462+0000","last_became_active":"2024-05-22T21:14:18.678745+0000","last_became_peered":"2024-05-22T21:14:18.678745+0000","last_unstale":"2024-05-22T21:14:20.058462+0000","last_undegraded":"2024-05-22T21:14:20.058462+0000","last_fullsized":"2024-05-22T21:14:20.058462+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:40:57.258501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062524+0000","last_change":"2024-05-22T21:14:19.061923+0000","last_active":"2024-05-22T21:14:20.062524+0000","last_peered":"2024-05-22T21:14:20.062524+0000","last_clean":"2024-05-22T21:14:20.062524+0000","last_became_active":"2024-05-22T21:14:19.061703+0000","last_became_peered":"2024-05-22T21:14:19.061703+0000","last_unstale":"2024-05-22T21:14:20.062524+0000","last_undegraded":"2024-05-22T21:14:20.062524+0000","last_fullsized":"2024-05-22T21:14:20.062524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:15:46.140480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062508+0000","last_change":"2024-05-22T21:14:18.689586+0000","last_active":"2024-05-22T21:14:20.062508+0000","last_peered":"2024-05-22T21:14:20.062508+0000","last_clean":"2024-05-22T21:14:20.062508+0000","last_became_active":"2024-05-22T21:14:18.689146+0000","last_became_peered":"2024-05-22T21:14:18.689146+0000","last_unstale":"2024-05-22T21:14:20.062508+0000","last_undegraded":"2024-05-22T21:14:20.062508+0000","last_fullsized":"2024-05-22T21:14:20.062508+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:57:22.808381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062182+0000","last_change":"2024-05-22T21:14:19.079226+0000","last_active":"2024-05-22T21:14:20.062182+0000","last_peered":"2024-05-22T21:14:20.062182+0000","last_clean":"2024-05-22T21:14:20.062182+0000","last_became_active":"2024-05-22T21:14:19.078091+0000","last_became_peered":"2024-05-22T21:14:19.078091+0000","last_unstale":"2024-05-22T21:14:20.062182+0000","last_undegraded":"2024-05-22T21:14:20.062182+0000","last_fullsized":"2024-05-22T21:14:20.062182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:03:35.024855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062444+0000","last_change":"2024-05-22T21:14:18.689130+0000","last_active":"2024-05-22T21:14:20.062444+0000","last_peered":"2024-05-22T21:14:20.062444+0000","last_clean":"2024-05-22T21:14:20.062444+0000","last_became_active":"2024-05-22T21:14:18.688397+0000","last_became_peered":"2024-05-22T21:14:18.688397+0000","last_unstale":"2024-05-22T21:14:20.062444+0000","last_undegraded":"2024-05-22T21:14:20.062444+0000","last_fullsized":"2024-05-22T21:14:20.062444+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:18:22.810811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061980+0000","last_change":"2024-05-22T21:14:19.074374+0000","last_active":"2024-05-22T21:14:20.061980+0000","last_peered":"2024-05-22T21:14:20.061980+0000","last_clean":"2024-05-22T21:14:20.061980+0000","last_became_active":"2024-05-22T21:14:19.074147+0000","last_became_peered":"2024-05-22T21:14:19.074147+0000","last_unstale":"2024-05-22T21:14:20.061980+0000","last_undegraded":"2024-05-22T21:14:20.061980+0000","last_fullsized":"2024-05-22T21:14:20.061980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:05:45.991300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063366+0000","last_change":"2024-05-22T21:14:18.685500+0000","last_active":"2024-05-22T21:14:20.063366+0000","last_peered":"2024-05-22T21:14:20.063366+0000","last_clean":"2024-05-22T21:14:20.063366+0000","last_became_active":"2024-05-22T21:14:18.684509+0000","last_became_peered":"2024-05-22T21:14:18.684509+0000","last_unstale":"2024-05-22T21:14:20.063366+0000","last_undegraded":"2024-05-22T21:14:20.063366+0000","last_fullsized":"2024-05-22T21:14:20.063366+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:31:43.116864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062222+0000","last_change":"2024-05-22T21:14:19.073238+0000","last_active":"2024-05-22T21:14:20.062222+0000","last_peered":"2024-05-22T21:14:20.062222+0000","last_clean":"2024-05-22T21:14:20.062222+0000","last_became_active":"2024-05-22T21:14:19.072721+0000","last_became_peered":"2024-05-22T21:14:19.072721+0000","last_unstale":"2024-05-22T21:14:20.062222+0000","last_undegraded":"2024-05-22T21:14:20.062222+0000","last_fullsized":"2024-05-22T21:14:20.062222+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:03:35.024855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062125+0000","last_change":"2024-05-22T21:14:18.689238+0000","last_active":"2024-05-22T21:14:20.062125+0000","last_peered":"2024-05-22T21:14:20.062125+0000","last_clean":"2024-05-22T21:14:20.062125+0000","last_became_active":"2024-05-22T21:14:18.687071+0000","last_became_peered":"2024-05-22T21:14:18.687071+0000","last_unstale":"2024-05-22T21:14:20.062125+0000","last_undegraded":"2024-05-22T21:14:20.062125+0000","last_fullsized":"2024-05-22T21:14:20.062125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:34:27.211979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061443+0000","last_change":"2024-05-22T21:14:19.064281+0000","last_active":"2024-05-22T21:14:20.061443+0000","last_peered":"2024-05-22T21:14:20.061443+0000","last_clean":"2024-05-22T21:14:20.061443+0000","last_became_active":"2024-05-22T21:14:19.063254+0000","last_became_peered":"2024-05-22T21:14:19.063254+0000","last_unstale":"2024-05-22T21:14:20.061443+0000","last_undegraded":"2024-05-22T21:14:20.061443+0000","last_fullsized":"2024-05-22T21:14:20.061443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:54:26.183700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062042+0000","last_change":"2024-05-22T21:14:18.688356+0000","last_active":"2024-05-22T21:14:20.062042+0000","last_peered":"2024-05-22T21:14:20.062042+0000","last_clean":"2024-05-22T21:14:20.062042+0000","last_became_active":"2024-05-22T21:14:18.687837+0000","last_became_peered":"2024-05-22T21:14:18.687837+0000","last_unstale":"2024-05-22T21:14:20.062042+0000","last_undegraded":"2024-05-22T21:14:20.062042+0000","last_fullsized":"2024-05-22T21:14:20.062042+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:34:27.211979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061291+0000","last_change":"2024-05-22T21:14:19.074572+0000","last_active":"2024-05-22T21:14:20.061291+0000","last_peered":"2024-05-22T21:14:20.061291+0000","last_clean":"2024-05-22T21:14:20.061291+0000","last_became_active":"2024-05-22T21:14:19.074017+0000","last_became_peered":"2024-05-22T21:14:19.074017+0000","last_unstale":"2024-05-22T21:14:20.061291+0000","last_undegraded":"2024-05-22T21:14:20.061291+0000","last_fullsized":"2024-05-22T21:14:20.061291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:57:23.815795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062434+0000","last_change":"2024-05-22T21:14:18.683288+0000","last_active":"2024-05-22T21:14:20.062434+0000","last_peered":"2024-05-22T21:14:20.062434+0000","last_clean":"2024-05-22T21:14:20.062434+0000","last_became_active":"2024-05-22T21:14:18.682999+0000","last_became_peered":"2024-05-22T21:14:18.682999+0000","last_unstale":"2024-05-22T21:14:20.062434+0000","last_undegraded":"2024-05-22T21:14:20.062434+0000","last_fullsized":"2024-05-22T21:14:20.062434+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:04:58.088978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063153+0000","last_change":"2024-05-22T21:14:19.070769+0000","last_active":"2024-05-22T21:14:20.063153+0000","last_peered":"2024-05-22T21:14:20.063153+0000","last_clean":"2024-05-22T21:14:20.063153+0000","last_became_active":"2024-05-22T21:14:19.070536+0000","last_became_peered":"2024-05-22T21:14:19.070536+0000","last_unstale":"2024-05-22T21:14:20.063153+0000","last_undegraded":"2024-05-22T21:14:20.063153+0000","last_fullsized":"2024-05-22T21:14:20.063153+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:28:03.263405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061994+0000","last_change":"2024-05-22T21:14:18.684858+0000","last_active":"2024-05-22T21:14:20.061994+0000","last_peered":"2024-05-22T21:14:20.061994+0000","last_clean":"2024-05-22T21:14:20.061994+0000","last_became_active":"2024-05-22T21:14:18.684211+0000","last_became_peered":"2024-05-22T21:14:18.684211+0000","last_unstale":"2024-05-22T21:14:20.061994+0000","last_undegraded":"2024-05-22T21:14:20.061994+0000","last_fullsized":"2024-05-22T21:14:20.061994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:34:52.060776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062408+0000","last_change":"2024-05-22T21:14:19.074599+0000","last_active":"2024-05-22T21:14:20.062408+0000","last_peered":"2024-05-22T21:14:20.062408+0000","last_clean":"2024-05-22T21:14:20.062408+0000","last_became_active":"2024-05-22T21:14:19.074152+0000","last_became_peered":"2024-05-22T21:14:19.074152+0000","last_unstale":"2024-05-22T21:14:20.062408+0000","last_undegraded":"2024-05-22T21:14:20.062408+0000","last_fullsized":"2024-05-22T21:14:20.062408+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:22:06.703835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063449+0000","last_change":"2024-05-22T21:14:18.686006+0000","last_active":"2024-05-22T21:14:20.063449+0000","last_peered":"2024-05-22T21:14:20.063449+0000","last_clean":"2024-05-22T21:14:20.063449+0000","last_became_active":"2024-05-22T21:14:18.685235+0000","last_became_peered":"2024-05-22T21:14:18.685235+0000","last_unstale":"2024-05-22T21:14:20.063449+0000","last_undegraded":"2024-05-22T21:14:20.063449+0000","last_fullsized":"2024-05-22T21:14:20.063449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:18:12.891326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062768+0000","last_change":"2024-05-22T21:14:19.069069+0000","last_active":"2024-05-22T21:14:20.062768+0000","last_peered":"2024-05-22T21:14:20.062768+0000","last_clean":"2024-05-22T21:14:20.062768+0000","last_became_active":"2024-05-22T21:14:19.068855+0000","last_became_peered":"2024-05-22T21:14:19.068855+0000","last_unstale":"2024-05-22T21:14:20.062768+0000","last_undegraded":"2024-05-22T21:14:20.062768+0000","last_fullsized":"2024-05-22T21:14:20.062768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:49:08.296970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063173+0000","last_change":"2024-05-22T21:14:18.682049+0000","last_active":"2024-05-22T21:14:20.063173+0000","last_peered":"2024-05-22T21:14:20.063173+0000","last_clean":"2024-05-22T21:14:20.063173+0000","last_became_active":"2024-05-22T21:14:18.681799+0000","last_became_peered":"2024-05-22T21:14:18.681799+0000","last_unstale":"2024-05-22T21:14:20.063173+0000","last_undegraded":"2024-05-22T21:14:20.063173+0000","last_fullsized":"2024-05-22T21:14:20.063173+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:42:39.208248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061234+0000","last_change":"2024-05-22T21:14:19.073723+0000","last_active":"2024-05-22T21:14:20.061234+0000","last_peered":"2024-05-22T21:14:20.061234+0000","last_clean":"2024-05-22T21:14:20.061234+0000","last_became_active":"2024-05-22T21:14:19.073484+0000","last_became_peered":"2024-05-22T21:14:19.073484+0000","last_unstale":"2024-05-22T21:14:20.061234+0000","last_undegraded":"2024-05-22T21:14:20.061234+0000","last_fullsized":"2024-05-22T21:14:20.061234+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:43:46.145481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063352+0000","last_change":"2024-05-22T21:14:18.680460+0000","last_active":"2024-05-22T21:14:20.063352+0000","last_peered":"2024-05-22T21:14:20.063352+0000","last_clean":"2024-05-22T21:14:20.063352+0000","last_became_active":"2024-05-22T21:14:18.679883+0000","last_became_peered":"2024-05-22T21:14:18.679883+0000","last_unstale":"2024-05-22T21:14:20.063352+0000","last_undegraded":"2024-05-22T21:14:20.063352+0000","last_fullsized":"2024-05-22T21:14:20.063352+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:44:36.545411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061375+0000","last_change":"2024-05-22T21:14:19.064604+0000","last_active":"2024-05-22T21:14:20.061375+0000","last_peered":"2024-05-22T21:14:20.061375+0000","last_clean":"2024-05-22T21:14:20.061375+0000","last_became_active":"2024-05-22T21:14:19.064264+0000","last_became_peered":"2024-05-22T21:14:19.064264+0000","last_unstale":"2024-05-22T21:14:20.061375+0000","last_undegraded":"2024-05-22T21:14:20.061375+0000","last_fullsized":"2024-05-22T21:14:20.061375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:10:29.781440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058454+0000","last_change":"2024-05-22T21:14:18.683390+0000","last_active":"2024-05-22T21:14:20.058454+0000","last_peered":"2024-05-22T21:14:20.058454+0000","last_clean":"2024-05-22T21:14:20.058454+0000","last_became_active":"2024-05-22T21:14:18.682551+0000","last_became_peered":"2024-05-22T21:14:18.682551+0000","last_unstale":"2024-05-22T21:14:20.058454+0000","last_undegraded":"2024-05-22T21:14:20.058454+0000","last_fullsized":"2024-05-22T21:14:20.058454+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:02:14.974395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061374+0000","last_change":"2024-05-22T21:14:19.074423+0000","last_active":"2024-05-22T21:14:20.061374+0000","last_peered":"2024-05-22T21:14:20.061374+0000","last_clean":"2024-05-22T21:14:20.061374+0000","last_became_active":"2024-05-22T21:14:19.073863+0000","last_became_peered":"2024-05-22T21:14:19.073863+0000","last_unstale":"2024-05-22T21:14:20.061374+0000","last_undegraded":"2024-05-22T21:14:20.061374+0000","last_fullsized":"2024-05-22T21:14:20.061374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:11:51.025353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.060846+0000","last_change":"2024-05-22T21:14:18.683770+0000","last_active":"2024-05-22T21:14:20.060846+0000","last_peered":"2024-05-22T21:14:20.060846+0000","last_clean":"2024-05-22T21:14:20.060846+0000","last_became_active":"2024-05-22T21:14:18.683515+0000","last_became_peered":"2024-05-22T21:14:18.683515+0000","last_unstale":"2024-05-22T21:14:20.060846+0000","last_undegraded":"2024-05-22T21:14:20.060846+0000","last_fullsized":"2024-05-22T21:14:20.060846+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:16:48.710431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061326+0000","last_change":"2024-05-22T21:14:19.072479+0000","last_active":"2024-05-22T21:14:20.061326+0000","last_peered":"2024-05-22T21:14:20.061326+0000","last_clean":"2024-05-22T21:14:20.061326+0000","last_became_active":"2024-05-22T21:14:19.072272+0000","last_became_peered":"2024-05-22T21:14:19.072272+0000","last_unstale":"2024-05-22T21:14:20.061326+0000","last_undegraded":"2024-05-22T21:14:20.061326+0000","last_fullsized":"2024-05-22T21:14:20.061326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:02:15.981809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063503+0000","last_change":"2024-05-22T21:14:18.687665+0000","last_active":"2024-05-22T21:14:20.063503+0000","last_peered":"2024-05-22T21:14:20.063503+0000","last_clean":"2024-05-22T21:14:20.063503+0000","last_became_active":"2024-05-22T21:14:18.687439+0000","last_became_peered":"2024-05-22T21:14:18.687439+0000","last_unstale":"2024-05-22T21:14:20.063503+0000","last_undegraded":"2024-05-22T21:14:20.063503+0000","last_fullsized":"2024-05-22T21:14:20.063503+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:41:19.249505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062005+0000","last_change":"2024-05-22T21:14:19.068841+0000","last_active":"2024-05-22T21:14:20.062005+0000","last_peered":"2024-05-22T21:14:20.062005+0000","last_clean":"2024-05-22T21:14:20.062005+0000","last_became_active":"2024-05-22T21:14:19.068115+0000","last_became_peered":"2024-05-22T21:14:19.068115+0000","last_unstale":"2024-05-22T21:14:20.062005+0000","last_undegraded":"2024-05-22T21:14:20.062005+0000","last_fullsized":"2024-05-22T21:14:20.062005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:58:20.679986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062714+0000","last_change":"2024-05-22T21:14:18.689681+0000","last_active":"2024-05-22T21:14:20.062714+0000","last_peered":"2024-05-22T21:14:20.062714+0000","last_clean":"2024-05-22T21:14:20.062714+0000","last_became_active":"2024-05-22T21:14:18.689354+0000","last_became_peered":"2024-05-22T21:14:18.689354+0000","last_unstale":"2024-05-22T21:14:20.062714+0000","last_undegraded":"2024-05-22T21:14:20.062714+0000","last_fullsized":"2024-05-22T21:14:20.062714+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:35:24.776954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061458+0000","last_change":"2024-05-22T21:14:19.071264+0000","last_active":"2024-05-22T21:14:20.061458+0000","last_peered":"2024-05-22T21:14:20.061458+0000","last_clean":"2024-05-22T21:14:20.061458+0000","last_became_active":"2024-05-22T21:14:19.071022+0000","last_became_peered":"2024-05-22T21:14:19.071022+0000","last_unstale":"2024-05-22T21:14:20.061458+0000","last_undegraded":"2024-05-22T21:14:20.061458+0000","last_fullsized":"2024-05-22T21:14:20.061458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:16:49.717845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062848+0000","last_change":"2024-05-22T21:14:18.689595+0000","last_active":"2024-05-22T21:14:20.062848+0000","last_peered":"2024-05-22T21:14:20.062848+0000","last_clean":"2024-05-22T21:14:20.062848+0000","last_became_active":"2024-05-22T21:14:18.688892+0000","last_became_peered":"2024-05-22T21:14:18.688892+0000","last_unstale":"2024-05-22T21:14:20.062848+0000","last_undegraded":"2024-05-22T21:14:20.062848+0000","last_fullsized":"2024-05-22T21:14:20.062848+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:40:57.258501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062571+0000","last_change":"2024-05-22T21:14:19.072541+0000","last_active":"2024-05-22T21:14:20.062571+0000","last_peered":"2024-05-22T21:14:20.062571+0000","last_clean":"2024-05-22T21:14:20.062571+0000","last_became_active":"2024-05-22T21:14:19.071855+0000","last_became_peered":"2024-05-22T21:14:19.071855+0000","last_unstale":"2024-05-22T21:14:20.062571+0000","last_undegraded":"2024-05-22T21:14:20.062571+0000","last_fullsized":"2024-05-22T21:14:20.062571+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:15:46.140480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058527+0000","last_change":"2024-05-22T21:14:18.683247+0000","last_active":"2024-05-22T21:14:20.058527+0000","last_peered":"2024-05-22T21:14:20.058527+0000","last_clean":"2024-05-22T21:14:20.058527+0000","last_became_active":"2024-05-22T21:14:18.682247+0000","last_became_peered":"2024-05-22T21:14:18.682247+0000","last_unstale":"2024-05-22T21:14:20.058527+0000","last_undegraded":"2024-05-22T21:14:20.058527+0000","last_fullsized":"2024-05-22T21:14:20.058527+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:11:50.017938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058224+0000","last_change":"2024-05-22T21:14:19.070412+0000","last_active":"2024-05-22T21:14:20.058224+0000","last_peered":"2024-05-22T21:14:20.058224+0000","last_clean":"2024-05-22T21:14:20.058224+0000","last_became_active":"2024-05-22T21:14:19.070206+0000","last_became_peered":"2024-05-22T21:14:19.070206+0000","last_unstale":"2024-05-22T21:14:20.058224+0000","last_undegraded":"2024-05-22T21:14:20.058224+0000","last_fullsized":"2024-05-22T21:14:20.058224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:57:23.815795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062047+0000","last_change":"2024-05-22T21:14:18.682898+0000","last_active":"2024-05-22T21:14:20.062047+0000","last_peered":"2024-05-22T21:14:20.062047+0000","last_clean":"2024-05-22T21:14:20.062047+0000","last_became_active":"2024-05-22T21:14:18.682581+0000","last_became_peered":"2024-05-22T21:14:18.682581+0000","last_unstale":"2024-05-22T21:14:20.062047+0000","last_undegraded":"2024-05-22T21:14:20.062047+0000","last_fullsized":"2024-05-22T21:14:20.062047+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:50:40.885861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.061378+0000","last_change":"2024-05-22T21:14:19.076098+0000","last_active":"2024-05-22T21:14:20.061378+0000","last_peered":"2024-05-22T21:14:20.061378+0000","last_clean":"2024-05-22T21:14:20.061378+0000","last_became_active":"2024-05-22T21:14:19.074328+0000","last_became_peered":"2024-05-22T21:14:19.074328+0000","last_unstale":"2024-05-22T21:14:20.061378+0000","last_undegraded":"2024-05-22T21:14:20.061378+0000","last_fullsized":"2024-05-22T21:14:20.061378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:35:25.784368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058518+0000","last_change":"2024-05-22T21:14:18.689372+0000","last_active":"2024-05-22T21:14:20.058518+0000","last_peered":"2024-05-22T21:14:20.058518+0000","last_clean":"2024-05-22T21:14:20.058518+0000","last_became_active":"2024-05-22T21:14:18.689134+0000","last_became_peered":"2024-05-22T21:14:18.689134+0000","last_unstale":"2024-05-22T21:14:20.058518+0000","last_undegraded":"2024-05-22T21:14:20.058518+0000","last_fullsized":"2024-05-22T21:14:20.058518+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:50:40.885861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.058373+0000","last_change":"2024-05-22T21:14:19.073743+0000","last_active":"2024-05-22T21:14:20.058373+0000","last_peered":"2024-05-22T21:14:20.058373+0000","last_clean":"2024-05-22T21:14:20.058373+0000","last_became_active":"2024-05-22T21:14:19.073475+0000","last_became_peered":"2024-05-22T21:14:19.073475+0000","last_unstale":"2024-05-22T21:14:20.058373+0000","last_undegraded":"2024-05-22T21:14:20.058373+0000","last_fullsized":"2024-05-22T21:14:20.058373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:35:25.784368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.063004+0000","last_change":"2024-05-22T21:14:18.689431+0000","last_active":"2024-05-22T21:14:20.063004+0000","last_peered":"2024-05-22T21:14:20.063004+0000","last_clean":"2024-05-22T21:14:20.063004+0000","last_became_active":"2024-05-22T21:14:18.688601+0000","last_became_peered":"2024-05-22T21:14:18.688601+0000","last_unstale":"2024-05-22T21:14:20.063004+0000","last_undegraded":"2024-05-22T21:14:20.063004+0000","last_fullsized":"2024-05-22T21:14:20.063004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:16:48.710431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062161+0000","last_change":"2024-05-22T21:14:19.071723+0000","last_active":"2024-05-22T21:14:20.062161+0000","last_peered":"2024-05-22T21:14:20.062161+0000","last_clean":"2024-05-22T21:14:20.062161+0000","last_became_active":"2024-05-22T21:14:19.064677+0000","last_became_peered":"2024-05-22T21:14:19.064677+0000","last_unstale":"2024-05-22T21:14:20.062161+0000","last_undegraded":"2024-05-22T21:14:20.062161+0000","last_fullsized":"2024-05-22T21:14:20.062161+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:49:14.649560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062576+0000","last_change":"2024-05-22T21:14:18.688659+0000","last_active":"2024-05-22T21:14:20.062576+0000","last_peered":"2024-05-22T21:14:20.062576+0000","last_clean":"2024-05-22T21:14:20.062576+0000","last_became_active":"2024-05-22T21:14:18.688437+0000","last_became_peered":"2024-05-22T21:14:18.688437+0000","last_unstale":"2024-05-22T21:14:20.062576+0000","last_undegraded":"2024-05-22T21:14:20.062576+0000","last_fullsized":"2024-05-22T21:14:20.062576+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:04:53.771604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062354+0000","last_change":"2024-05-22T21:14:19.069813+0000","last_active":"2024-05-22T21:14:20.062354+0000","last_peered":"2024-05-22T21:14:20.062354+0000","last_clean":"2024-05-22T21:14:20.062354+0000","last_became_active":"2024-05-22T21:14:19.066103+0000","last_became_peered":"2024-05-22T21:14:19.066103+0000","last_unstale":"2024-05-22T21:14:20.062354+0000","last_undegraded":"2024-05-22T21:14:20.062354+0000","last_fullsized":"2024-05-22T21:14:20.062354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:21:48.071027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062599+0000","last_change":"2024-05-22T21:14:18.688154+0000","last_active":"2024-05-22T21:14:20.062599+0000","last_peered":"2024-05-22T21:14:20.062599+0000","last_clean":"2024-05-22T21:14:20.062599+0000","last_became_active":"2024-05-22T21:14:18.683773+0000","last_became_peered":"2024-05-22T21:14:18.683773+0000","last_unstale":"2024-05-22T21:14:20.062599+0000","last_undegraded":"2024-05-22T21:14:20.062599+0000","last_fullsized":"2024-05-22T21:14:20.062599+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:53:55.959523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.062080+0000","last_change":"2024-05-22T21:14:19.071943+0000","last_active":"2024-05-22T21:14:20.062080+0000","last_peered":"2024-05-22T21:14:20.062080+0000","last_clean":"2024-05-22T21:14:20.062080+0000","last_became_active":"2024-05-22T21:14:19.071565+0000","last_became_peered":"2024-05-22T21:14:19.071565+0000","last_unstale":"2024-05-22T21:14:20.062080+0000","last_undegraded":"2024-05-22T21:14:20.062080+0000","last_fullsized":"2024-05-22T21:14:20.062080+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:18.663785+0000","last_clean_scrub_stamp":"2024-05-22T21:14:18.663785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:45:18.511639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-22T21:14:20.060942+0000","last_change":"2024-05-22T21:14:18.685487+0000","last_active":"2024-05-22T21:14:20.060942+0000","last_peered":"2024-05-22T21:14:20.060942+0000","last_clean":"2024-05-22T21:14:20.060942+0000","last_became_active":"2024-05-22T21:14:18.685270+0000","last_became_peered":"2024-05-22T21:14:18.685270+0000","last_unstale":"2024-05-22T21:14:20.060942+0000","last_undegraded":"2024-05-22T21:14:20.060942+0000","last_fullsized":"2024-05-22T21:14:20.060942+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:17.656370+0000","last_clean_scrub_stamp":"2024-05-22T21:14:17.656370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:02:14.974395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-22T21:14:19.057371+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:14:19.057371+0000","last_peered":"2024-05-22T21:14:19.057371+0000","last_clean":"2024-05-22T21:14:19.057371+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:14:19.057371+0000","last_undegraded":"2024-05-22T21:14:19.057371+0000","last_fullsized":"2024-05-22T21:14:19.057371+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":717,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199265294,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.44700000000000001}]},{"osd":1,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":3.0920000000000001,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":3.1499999999999999,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.39900000000000002}]},{"osd":2,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.46999999999999997}]},{"osd":3,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.54600000000000004}]},{"osd":4,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.625},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.67000000000000004}]},{"osd":5,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.504}]},{"osd":6,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.63900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":141,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195171,"statfs":{"total":200000000,"available":199855136,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.73899999999999999}]},{"osd":1,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.82099999999999995}]},{"osd":2,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.80500000000000005}]},{"osd":3,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.61899999999999999}]},{"osd":4,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.752},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.65500000000000003}]},{"osd":7,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.83399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":141,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195170,"statfs":{"total":200000000,"available":199855080,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.82099999999999995}]},{"osd":1,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.70699999999999996}]},{"osd":2,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.747},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.76900000000000002}]},{"osd":3,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.66300000000000003}]},{"osd":6,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.629}]},{"osd":7,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.68799999999999994}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":140,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195171,"statfs":{"total":200000000,"available":199855648,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.52900000000000003}]},{"osd":1,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.50600000000000001}]},{"osd":2,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.58399999999999996}]},{"osd":3,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.629},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.48999999999999999}]},{"osd":6,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.56299999999999994}]},{"osd":7,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.61599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":142,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195170,"statfs":{"total":200000000,"available":199854166,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.245},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.32800000000000001}]},{"osd":1,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.35399999999999998}]},{"osd":2,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.45900000000000002}]},{"osd":4,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.75600000000000001}]},{"osd":5,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.60199999999999998}]},{"osd":6,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.65700000000000003}]},{"osd":7,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.749}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":140,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195171,"statfs":{"total":200000000,"available":199855682,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.29099999999999998}]},{"osd":1,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.316},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.41599999999999998}]},{"osd":3,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.38800000000000001}]},{"osd":4,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65500000000000003}]},{"osd":5,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.58099999999999996}]},{"osd":6,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.56399999999999995}]},{"osd":7,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.628}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":142,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195170,"statfs":{"total":200000000,"available":199854166,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.42999999999999999}]},{"osd":2,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.49099999999999999}]},{"osd":3,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.378},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47799999999999998}]},{"osd":4,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.68000000000000005}]},{"osd":5,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.748}]},{"osd":6,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.76200000000000001}]},{"osd":7,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.69599999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":716,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199266003,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.436}]},{"osd":2,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.45800000000000002}]},{"osd":3,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.751}]},{"osd":5,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.89000000000000001}]},{"osd":6,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.998}]},{"osd":7,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.77200000000000002}]}]}],"pool_statfs":[]}} 2024-05-22T21:14:23.521 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-22T21:14:23.521 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-22T21:14:23.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.704+0000 7f65aee00640 1 Processor -- start 2024-05-22T21:14:23.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.704+0000 7f65aee00640 1 -- start start 2024-05-22T21:14:23.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.704+0000 7f65aee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65a806c2a0 0x7f65a806c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:23.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.704+0000 7f65aee00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f65a80b3370 0x7f65a80b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:23.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.704+0000 7f65aee00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a815d9d0 con 0x7f65a81402b0 2024-05-22T21:14:23.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.704+0000 7f65aee00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a815d7c0 con 0x7f65a80b3370 2024-05-22T21:14:23.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.704+0000 7f65aee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a806cc60 con 0x7f65a806c2a0 2024-05-22T21:14:23.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.704+0000 7f65ace00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f65a80b3370 0x7f65a80b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:23.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.704+0000 7f65ace00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f65a80b3370 0x7f65a80b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49044/0 (socket says 172.21.15.161:49044) 2024-05-22T21:14:23.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.704+0000 7f65ace00640 1 -- 172.21.15.161:0/3822362466 learned_addr learned my addr 172.21.15.161:0/3822362466 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:23.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.704+0000 7f65a7400640 1 --2- 172.21.15.161:0/3822362466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65a806c2a0 0x7f65a806c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:23.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.704+0000 7f65ace00640 1 -- 172.21.15.161:0/3822362466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65a806c2a0 msgr2=0x7f65a806c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:23.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.704+0000 7f65ace00640 1 --2- 172.21.15.161:0/3822362466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65a806c2a0 0x7f65a806c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.704+0000 7f65ace00640 1 -- 172.21.15.161:0/3822362466 >> v1:172.21.15.179:6789/0 conn(0x7f65a81402b0 legacy=0x7f65a8070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:23.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.704+0000 7f65ace00640 1 -- 172.21.15.161:0/3822362466 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a815dcb0 con 0x7f65a80b3370 2024-05-22T21:14:23.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.708+0000 7f65ace00640 1 --2- 172.21.15.161:0/3822362466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65a80b3370 0x7f65a80b3800 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009e50 tx=0x7f65a001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57a3835001a25a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:23.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.708+0000 7f65a6a00640 1 -- 172.21.15.161:0/3822362466 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a00a5900 con 0x7f65a80b3370 2024-05-22T21:14:23.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.708+0000 7f65a6a00640 1 -- 172.21.15.161:0/3822362466 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a00a5a60 con 0x7f65a80b3370 2024-05-22T21:14:23.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.708+0000 7f65a6a00640 1 -- 172.21.15.161:0/3822362466 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a00a5df0 con 0x7f65a80b3370 2024-05-22T21:14:23.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.708+0000 7f65aee00640 1 -- 172.21.15.161:0/3822362466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65a80b3370 msgr2=0x7f65a80b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:23.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.708+0000 7f65aee00640 1 --2- 172.21.15.161:0/3822362466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65a80b3370 0x7f65a80b3800 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009e50 tx=0x7f65a001ca30 comp rx=0 tx=0).stop 2024-05-22T21:14:23.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.708+0000 7f65aee00640 1 -- 172.21.15.161:0/3822362466 shutdown_connections 2024-05-22T21:14:23.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.708+0000 7f65aee00640 1 --2- 172.21.15.161:0/3822362466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65a80b3370 0x7f65a80b3800 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.708+0000 7f65aee00640 1 --2- 172.21.15.161:0/3822362466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65a806c2a0 0x7f65a806c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.708+0000 7f65aee00640 1 -- 172.21.15.161:0/3822362466 >> 172.21.15.161:0/3822362466 conn(0x7f65a806a2d0 msgr2=0x7f65a806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:23.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.708+0000 7f65aee00640 1 -- 172.21.15.161:0/3822362466 shutdown_connections 2024-05-22T21:14:23.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.708+0000 7f65aee00640 1 -- 172.21.15.161:0/3822362466 wait complete. 2024-05-22T21:14:23.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.708+0000 7f65aee00640 1 Processor -- start 2024-05-22T21:14:23.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65aee00640 1 -- start start 2024-05-22T21:14:23.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65aee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65a806c2a0 0x7f65a81562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:23.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65aee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65a80b3370 0x7f65a8139ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:23.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65aee00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65a81402b0 0x7f65a8141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:23.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65aee00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a813bda0 con 0x7f65a81402b0 2024-05-22T21:14:23.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65aee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a813c080 con 0x7f65a80b3370 2024-05-22T21:14:23.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65aee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a813c360 con 0x7f65a806c2a0 2024-05-22T21:14:23.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65ace00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65a81402b0 0x7f65a8141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:23.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65ace00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65a81402b0 0x7f65a8141a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49060/0 (socket says 172.21.15.161:49060) 2024-05-22T21:14:23.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65ace00640 1 -- 172.21.15.161:0/1242204582 learned_addr learned my addr 172.21.15.161:0/1242204582 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:23.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65a7e00640 1 --2- 172.21.15.161:0/1242204582 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65a806c2a0 0x7f65a81562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:23.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65a7400640 1 --2- 172.21.15.161:0/1242204582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65a80b3370 0x7f65a8139ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:23.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65ace00640 1 -- 172.21.15.161:0/1242204582 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65a806c2a0 msgr2=0x7f65a81562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:23.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65ace00640 1 --2- 172.21.15.161:0/1242204582 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65a806c2a0 0x7f65a81562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65ace00640 1 -- 172.21.15.161:0/1242204582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65a80b3370 msgr2=0x7f65a8139ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:23.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65ace00640 1 --2- 172.21.15.161:0/1242204582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65a80b3370 0x7f65a8139ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65ace00640 1 -- 172.21.15.161:0/1242204582 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a0009b30 con 0x7f65a81402b0 2024-05-22T21:14:23.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65ace00640 1 --2- 172.21.15.161:0/1242204582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65a81402b0 0x7f65a8141a00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f65a001a660 tx=0x7f65a80b4550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:23.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f659fe00640 1 -- 172.21.15.161:0/1242204582 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a00b8670 con 0x7f65a81402b0 2024-05-22T21:14:23.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f659fe00640 1 -- 172.21.15.161:0/1242204582 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a00b87d0 con 0x7f65a81402b0 2024-05-22T21:14:23.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f659fe00640 1 -- 172.21.15.161:0/1242204582 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a00b8b60 con 0x7f65a81402b0 2024-05-22T21:14:23.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65aee00640 1 -- 172.21.15.161:0/1242204582 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a813c640 con 0x7f65a81402b0 2024-05-22T21:14:23.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65aee00640 1 -- 172.21.15.161:0/1242204582 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a8156820 con 0x7f65a81402b0 2024-05-22T21:14:23.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.712+0000 7f65aee00640 1 -- 172.21.15.161:0/1242204582 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a813caf0 con 0x7f65a81402b0 2024-05-22T21:14:23.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.716+0000 7f659fe00640 1 -- 172.21.15.161:0/1242204582 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f65a00b8cc0 con 0x7f65a81402b0 2024-05-22T21:14:23.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.716+0000 7f659fe00640 1 --2- 172.21.15.161:0/1242204582 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6588066690 0x7f6588068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:23.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.716+0000 7f659fe00640 1 -- 172.21.15.161:0/1242204582 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65a011f590 con 0x7f65a81402b0 2024-05-22T21:14:23.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.716+0000 7f65a7e00640 1 --2- 172.21.15.161:0/1242204582 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6588066690 0x7f6588068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:23.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.716+0000 7f65a7e00640 1 --2- 172.21.15.161:0/1242204582 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6588066690 0x7f6588068b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6598007fd0 tx=0x7f6598009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:23.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.720+0000 7f659fe00640 1 -- 172.21.15.161:0/1242204582 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f65a00ec880 con 0x7f65a81402b0 2024-05-22T21:14:23.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.924+0000 7f65aee00640 1 -- 172.21.15.161:0/1242204582 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f65a8067220 con 0x7f65a81402b0 2024-05-22T21:14:23.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.932+0000 7f659fe00640 1 -- 172.21.15.161:0/1242204582 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f65a00efea0 con 0x7f65a81402b0 2024-05-22T21:14:23.932 INFO:teuthology.orchestra.run.smithi161.stderr:updated caps for client.0 2024-05-22T21:14:23.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.932+0000 7f65aee00640 1 -- 172.21.15.161:0/1242204582 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6588066690 msgr2=0x7f6588068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:23.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.932+0000 7f65aee00640 1 --2- 172.21.15.161:0/1242204582 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6588066690 0x7f6588068b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6598007fd0 tx=0x7f6598009040 comp rx=0 tx=0).stop 2024-05-22T21:14:23.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.932+0000 7f65aee00640 1 -- 172.21.15.161:0/1242204582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65a81402b0 msgr2=0x7f65a8141a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:23.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.932+0000 7f65aee00640 1 --2- 172.21.15.161:0/1242204582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65a81402b0 0x7f65a8141a00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f65a001a660 tx=0x7f65a80b4550 comp rx=0 tx=0).stop 2024-05-22T21:14:23.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.932+0000 7f65aee00640 1 -- 172.21.15.161:0/1242204582 shutdown_connections 2024-05-22T21:14:23.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.932+0000 7f65aee00640 1 --2- 172.21.15.161:0/1242204582 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6588066690 0x7f6588068b50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.932+0000 7f65aee00640 1 --2- 172.21.15.161:0/1242204582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65a81402b0 0x7f65a8141a00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.932+0000 7f65aee00640 1 --2- 172.21.15.161:0/1242204582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65a80b3370 0x7f65a8139ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.932+0000 7f65aee00640 1 --2- 172.21.15.161:0/1242204582 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65a806c2a0 0x7f65a81562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:23.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.932+0000 7f65aee00640 1 -- 172.21.15.161:0/1242204582 >> 172.21.15.161:0/1242204582 conn(0x7f65a806a2d0 msgr2=0x7f65a80b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:23.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.932+0000 7f65aee00640 1 -- 172.21.15.161:0/1242204582 shutdown_connections 2024-05-22T21:14:23.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:23.932+0000 7f65aee00640 1 -- 172.21.15.161:0/1242204582 wait complete. 2024-05-22T21:14:23.954 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:14:24.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.088+0000 7f9d0b200640 1 Processor -- start 2024-05-22T21:14:24.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.088+0000 7f9d0b200640 1 -- start start 2024-05-22T21:14:24.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.088+0000 7f9d0b200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9d0c15d040 0x7f9d0c0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.088+0000 7f9d0b200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d0c15e430 0x7f9d0c0cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.088+0000 7f9d0b200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c0b13d0 con 0x7f9d0c0d0cf0 2024-05-22T21:14:24.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.088+0000 7f9d0b200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0c0b0aa0 con 0x7f9d0c15d040 2024-05-22T21:14:24.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.088+0000 7f9d0b200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0b0d80 con 0x7f9d0c15e430 2024-05-22T21:14:24.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.088+0000 7f9d09e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9d0c15d040 0x7f9d0c0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.088+0000 7f9d09e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9d0c15d040 0x7f9d0c0b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49072/0 (socket says 172.21.15.161:49072) 2024-05-22T21:14:24.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.088+0000 7f9d09e00640 1 -- 172.21.15.161:0/3028105849 learned_addr learned my addr 172.21.15.161:0/3028105849 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:24.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.088+0000 7f9d0a800640 1 --2- 172.21.15.161:0/3028105849 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d0c15e430 0x7f9d0c0cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.088+0000 7f9d09e00640 1 -- 172.21.15.161:0/3028105849 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d0c15e430 msgr2=0x7f9d0c0cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:24.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.088+0000 7f9d09e00640 1 --2- 172.21.15.161:0/3028105849 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d0c15e430 0x7f9d0c0cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.088+0000 7f9d09e00640 1 -- 172.21.15.161:0/3028105849 >> v1:172.21.15.179:6789/0 conn(0x7f9d0c0d0cf0 legacy=0x7f9d0c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:24.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.088+0000 7f9d09e00640 1 -- 172.21.15.161:0/3028105849 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0b1060 con 0x7f9d0c15d040 2024-05-22T21:14:24.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.092+0000 7f9d09e00640 1 --2- 172.21.15.161:0/3028105849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d0c15d040 0x7f9d0c0b1860 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009e50 tx=0x7f9cfc01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f8d7e61c69709f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:24.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.092+0000 7f9d09400640 1 -- 172.21.15.161:0/3028105849 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0a5820 con 0x7f9d0c15d040 2024-05-22T21:14:24.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.092+0000 7f9d09400640 1 -- 172.21.15.161:0/3028105849 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc0a5980 con 0x7f9d0c15d040 2024-05-22T21:14:24.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.092+0000 7f9d09400640 1 -- 172.21.15.161:0/3028105849 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0a5d10 con 0x7f9d0c15d040 2024-05-22T21:14:24.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.092+0000 7f9d0b200640 1 -- 172.21.15.161:0/3028105849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d0c15d040 msgr2=0x7f9d0c0b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:24.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.092+0000 7f9d0b200640 1 --2- 172.21.15.161:0/3028105849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d0c15d040 0x7f9d0c0b1860 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009e50 tx=0x7f9cfc01ca30 comp rx=0 tx=0).stop 2024-05-22T21:14:24.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.092+0000 7f9d0b200640 1 -- 172.21.15.161:0/3028105849 shutdown_connections 2024-05-22T21:14:24.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.092+0000 7f9d0b200640 1 --2- 172.21.15.161:0/3028105849 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d0c15e430 0x7f9d0c0cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.092+0000 7f9d0b200640 1 --2- 172.21.15.161:0/3028105849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d0c15d040 0x7f9d0c0b1860 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.092+0000 7f9d0b200640 1 -- 172.21.15.161:0/3028105849 >> 172.21.15.161:0/3028105849 conn(0x7f9d0c099a40 msgr2=0x7f9d0c099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:24.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.092+0000 7f9d0b200640 1 -- 172.21.15.161:0/3028105849 shutdown_connections 2024-05-22T21:14:24.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.092+0000 7f9d0b200640 1 -- 172.21.15.161:0/3028105849 wait complete. 2024-05-22T21:14:24.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0b200640 1 Processor -- start 2024-05-22T21:14:24.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0b200640 1 -- start start 2024-05-22T21:14:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0b200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d0c0d0cf0 0x7f9d0c15b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0b200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d0c15d040 0x7f9d0c1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d0c15e430 0x7f9d0c0b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0b200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c155810 con 0x7f9d0c15e430 2024-05-22T21:14:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0b200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0ae820 con 0x7f9d0c0d0cf0 2024-05-22T21:14:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0b200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0aeb00 con 0x7f9d0c15d040 2024-05-22T21:14:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d0c15e430 0x7f9d0c0b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d0c15e430 0x7f9d0c0b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49078/0 (socket says 172.21.15.161:49078) 2024-05-22T21:14:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0a800640 1 -- 172.21.15.161:0/2926270947 learned_addr learned my addr 172.21.15.161:0/2926270947 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d09e00640 1 --2- 172.21.15.161:0/2926270947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d0c0d0cf0 0x7f9d0c15b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d01400640 1 --2- 172.21.15.161:0/2926270947 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d0c15d040 0x7f9d0c1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0a800640 1 -- 172.21.15.161:0/2926270947 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d0c15d040 msgr2=0x7f9d0c1552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:24.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0a800640 1 --2- 172.21.15.161:0/2926270947 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d0c15d040 0x7f9d0c1552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0a800640 1 -- 172.21.15.161:0/2926270947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d0c0d0cf0 msgr2=0x7f9d0c15b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:24.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0a800640 1 --2- 172.21.15.161:0/2926270947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d0c0d0cf0 0x7f9d0c15b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0a800640 1 -- 172.21.15.161:0/2926270947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc009b30 con 0x7f9d0c15e430 2024-05-22T21:14:24.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0a800640 1 --2- 172.21.15.161:0/2926270947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d0c15e430 0x7f9d0c0b0130 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400e7f0 tx=0x7f9d0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:24.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d02a00640 1 -- 172.21.15.161:0/2926270947 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04009530 con 0x7f9d0c15e430 2024-05-22T21:14:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d02a00640 1 -- 172.21.15.161:0/2926270947 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0400eea0 con 0x7f9d0c15e430 2024-05-22T21:14:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d02a00640 1 -- 172.21.15.161:0/2926270947 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0400cb60 con 0x7f9d0c15e430 2024-05-22T21:14:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0b200640 1 -- 172.21.15.161:0/2926270947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c15aef0 con 0x7f9d0c15e430 2024-05-22T21:14:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0b200640 1 -- 172.21.15.161:0/2926270947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c15b4b0 con 0x7f9d0c15e430 2024-05-22T21:14:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.096+0000 7f9d0b200640 1 -- 172.21.15.161:0/2926270947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c15ad50 con 0x7f9d0c15e430 2024-05-22T21:14:24.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.100+0000 7f9d02a00640 1 -- 172.21.15.161:0/2926270947 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9d0400ccc0 con 0x7f9d0c15e430 2024-05-22T21:14:24.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.100+0000 7f9d02a00640 1 --2- 172.21.15.161:0/2926270947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ce8066690 0x7f9ce8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.100+0000 7f9d02a00640 1 -- 172.21.15.161:0/2926270947 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d040841a0 con 0x7f9d0c15e430 2024-05-22T21:14:24.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.100+0000 7f9d09e00640 1 --2- 172.21.15.161:0/2926270947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ce8066690 0x7f9ce8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.100+0000 7f9d09e00640 1 --2- 172.21.15.161:0/2926270947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ce8066690 0x7f9ce8068b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0ae000 tx=0x7f9cfc0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:24.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.104+0000 7f9d02a00640 1 -- 172.21.15.161:0/2926270947 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9d0c15ad50 con 0x7f9d0c15e430 2024-05-22T21:14:24.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.288+0000 7f9d0b200640 1 -- 172.21.15.161:0/2926270947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d0c15ad50 con 0x7f9d0c15e430 2024-05-22T21:14:24.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.288+0000 7f9d02a00640 1 -- 172.21.15.161:0/2926270947 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f9d0c15ad50 con 0x7f9d0c15e430 2024-05-22T21:14:24.292 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:24.292 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3742018925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":3742018925},{"type":"v1","addr":"172.21.15.161:6837","nonce":3742018925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2553366378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2553366378},{"type":"v1","addr":"172.21.15.179:6834","nonce":2553366378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/509468338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":509468338},{"type":"v1","addr":"172.21.15.179:6835","nonce":509468338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:19.053259+0000","modified":"2024-05-22T21:14:22.076119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6836/1635458417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1635458417},{"type":"v1","addr":"172.21.15.161:6836","nonce":1635458417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-22T21:14:24.292 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 16 2024-05-22T21:14:24.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.292+0000 7f9d0b200640 1 -- 172.21.15.161:0/2926270947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ce8066690 msgr2=0x7f9ce8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:24.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.292+0000 7f9d0b200640 1 --2- 172.21.15.161:0/2926270947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ce8066690 0x7f9ce8068b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0ae000 tx=0x7f9cfc0a7070 comp rx=0 tx=0).stop 2024-05-22T21:14:24.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.292+0000 7f9d0b200640 1 -- 172.21.15.161:0/2926270947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d0c15e430 msgr2=0x7f9d0c0b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:24.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.292+0000 7f9d0b200640 1 --2- 172.21.15.161:0/2926270947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d0c15e430 0x7f9d0c0b0130 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400e7f0 tx=0x7f9d0400ecb0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.292+0000 7f9d0b200640 1 -- 172.21.15.161:0/2926270947 shutdown_connections 2024-05-22T21:14:24.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.292+0000 7f9d0b200640 1 --2- 172.21.15.161:0/2926270947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ce8066690 0x7f9ce8068b50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.292+0000 7f9d0b200640 1 --2- 172.21.15.161:0/2926270947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d0c15e430 0x7f9d0c0b0130 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.292+0000 7f9d0b200640 1 --2- 172.21.15.161:0/2926270947 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d0c15d040 0x7f9d0c1552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.292+0000 7f9d0b200640 1 --2- 172.21.15.161:0/2926270947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d0c0d0cf0 0x7f9d0c15b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.292+0000 7f9d0b200640 1 -- 172.21.15.161:0/2926270947 >> 172.21.15.161:0/2926270947 conn(0x7f9d0c099a40 msgr2=0x7f9d0c0cc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:24.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.292+0000 7f9d0b200640 1 -- 172.21.15.161:0/2926270947 shutdown_connections 2024-05-22T21:14:24.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.292+0000 7f9d0b200640 1 -- 172.21.15.161:0/2926270947 wait complete. 2024-05-22T21:14:24.715 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-22T21:14:19.053259+0000', 'modified': '2024-05-22T21:14:22.076119+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5055}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5055': {'gid': 5055, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6836/1635458417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 1635458417}, {'type': 'v1', 'addr': '172.21.15.161:6836', 'nonce': 1635458417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-22T21:14:24.715 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-22T21:14:24.715 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-22T21:14:24.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6ea5e00640 1 Processor -- start 2024-05-22T21:14:24.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6ea5e00640 1 -- start start 2024-05-22T21:14:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6ea5e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6ea01402e0 0x7f6ea013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6ea5e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6ea013fc80 0x7f6ea006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6ea5e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea0140090 con 0x7f6ea013cfc0 2024-05-22T21:14:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6ea5e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea013d3a0 con 0x7f6ea013fc80 2024-05-22T21:14:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6ea5e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea00cd920 con 0x7f6ea01402e0 2024-05-22T21:14:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6e9ea00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f6ea013cfc0 0x7f6ea015af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:38366/0 (socket says 172.21.15.161:38366) 2024-05-22T21:14:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6e9ea00640 1 -- 172.21.15.161:0/1097763977 learned_addr learned my addr 172.21.15.161:0/1097763977 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6e9f400640 1 --2- 172.21.15.161:0/1097763977 >> v2:172.21.15.161:3300/0 conn(0x7f6ea013fc80 0x7f6ea006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6e9e000640 1 --2- 172.21.15.161:0/1097763977 >> v2:172.21.15.179:3300/0 conn(0x7f6ea01402e0 0x7f6ea013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6e9d600640 1 -- 172.21.15.161:0/1097763977 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276063511 0 0) 0x7f6ea0140090 con 0x7f6ea013cfc0 2024-05-22T21:14:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6e9d600640 1 -- 172.21.15.161:0/1097763977 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e84003530 con 0x7f6ea013cfc0 2024-05-22T21:14:24.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6e9f400640 1 -- 172.21.15.161:0/1097763977 >> v2:172.21.15.179:3300/0 conn(0x7f6ea01402e0 msgr2=0x7f6ea013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:24.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6e9f400640 1 --2- 172.21.15.161:0/1097763977 >> v2:172.21.15.179:3300/0 conn(0x7f6ea01402e0 0x7f6ea013f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6e9f400640 1 -- 172.21.15.161:0/1097763977 >> v1:172.21.15.161:6789/0 conn(0x7f6ea013cfc0 legacy=0x7f6ea015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:24.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6e9f400640 1 -- 172.21.15.161:0/1097763977 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea00cdc00 con 0x7f6ea013fc80 2024-05-22T21:14:24.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.852+0000 7f6e9e000640 1 --2- 172.21.15.161:0/1097763977 >> v2:172.21.15.179:3300/0 conn(0x7f6ea01402e0 0x7f6ea013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:24.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.856+0000 7f6e9f400640 1 --2- 172.21.15.161:0/1097763977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea013fc80 0x7f6ea006fce0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6ea006e1d0 tx=0x7f6e9801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812040b8d4736ed3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:24.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.856+0000 7f6e9d600640 1 -- 172.21.15.161:0/1097763977 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e980aa620 con 0x7f6ea013fc80 2024-05-22T21:14:24.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.856+0000 7f6e9d600640 1 -- 172.21.15.161:0/1097763977 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e980aa780 con 0x7f6ea013fc80 2024-05-22T21:14:24.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.856+0000 7f6e9d600640 1 -- 172.21.15.161:0/1097763977 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e980aab10 con 0x7f6ea013fc80 2024-05-22T21:14:24.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.856+0000 7f6ea5e00640 1 -- 172.21.15.161:0/1097763977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea013fc80 msgr2=0x7f6ea006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:24.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.856+0000 7f6ea5e00640 1 --2- 172.21.15.161:0/1097763977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea013fc80 0x7f6ea006fce0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6ea006e1d0 tx=0x7f6e9801c870 comp rx=0 tx=0).stop 2024-05-22T21:14:24.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.856+0000 7f6ea5e00640 1 -- 172.21.15.161:0/1097763977 shutdown_connections 2024-05-22T21:14:24.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.856+0000 7f6ea5e00640 1 --2- 172.21.15.161:0/1097763977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea013fc80 0x7f6ea006fce0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.856+0000 7f6ea5e00640 1 --2- 172.21.15.161:0/1097763977 >> v2:172.21.15.179:3300/0 conn(0x7f6ea01402e0 0x7f6ea013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.856+0000 7f6ea5e00640 1 -- 172.21.15.161:0/1097763977 >> 172.21.15.161:0/1097763977 conn(0x7f6ea006a2d0 msgr2=0x7f6ea006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:24.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.856+0000 7f6ea5e00640 1 -- 172.21.15.161:0/1097763977 shutdown_connections 2024-05-22T21:14:24.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.856+0000 7f6ea5e00640 1 -- 172.21.15.161:0/1097763977 wait complete. 2024-05-22T21:14:24.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.856+0000 7f6ea5e00640 1 Processor -- start 2024-05-22T21:14:24.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6ea5e00640 1 -- start start 2024-05-22T21:14:24.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6ea5e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ea013cfc0 0x7f6ea013bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6ea5e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ea013fc80 0x7f6ea013e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6ea5e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea01402e0 0x7f6ea00ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6ea5e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea015c6c0 con 0x7f6ea01402e0 2024-05-22T21:14:24.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6ea5e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea013c960 con 0x7f6ea013cfc0 2024-05-22T21:14:24.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6ea5e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea015bab0 con 0x7f6ea013fc80 2024-05-22T21:14:24.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6e9f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea01402e0 0x7f6ea00ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6e9f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea01402e0 0x7f6ea00ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49102/0 (socket says 172.21.15.161:49102) 2024-05-22T21:14:24.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6e9f400640 1 -- 172.21.15.161:0/3725321810 learned_addr learned my addr 172.21.15.161:0/3725321810 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:24.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6e9e000640 1 --2- 172.21.15.161:0/3725321810 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ea013fc80 0x7f6ea013e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6e9ea00640 1 --2- 172.21.15.161:0/3725321810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ea013cfc0 0x7f6ea013bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6e9f400640 1 -- 172.21.15.161:0/3725321810 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ea013fc80 msgr2=0x7f6ea013e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:24.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6e9f400640 1 --2- 172.21.15.161:0/3725321810 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ea013fc80 0x7f6ea013e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6e9f400640 1 -- 172.21.15.161:0/3725321810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ea013cfc0 msgr2=0x7f6ea013bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:24.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6e9f400640 1 --2- 172.21.15.161:0/3725321810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ea013cfc0 0x7f6ea013bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6e9f400640 1 -- 172.21.15.161:0/3725321810 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e98009a70 con 0x7f6ea01402e0 2024-05-22T21:14:24.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6e9f400640 1 --2- 172.21.15.161:0/3725321810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea01402e0 0x7f6ea00ad8a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6e9801a4a0 tx=0x7f6e98019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:24.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6e97400640 1 -- 172.21.15.161:0/3725321810 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e980aad00 con 0x7f6ea01402e0 2024-05-22T21:14:24.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6e97400640 1 -- 172.21.15.161:0/3725321810 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e980aae60 con 0x7f6ea01402e0 2024-05-22T21:14:24.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6e97400640 1 -- 172.21.15.161:0/3725321810 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e980bf690 con 0x7f6ea01402e0 2024-05-22T21:14:24.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6ea5e00640 1 -- 172.21.15.161:0/3725321810 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea015bd90 con 0x7f6ea01402e0 2024-05-22T21:14:24.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.860+0000 7f6ea5e00640 1 -- 172.21.15.161:0/3725321810 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea015c350 con 0x7f6ea01402e0 2024-05-22T21:14:24.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6e97400640 1 -- 172.21.15.161:0/3725321810 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6e980bf7f0 con 0x7f6ea01402e0 2024-05-22T21:14:24.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6e97400640 1 --2- 172.21.15.161:0/3725321810 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e80066690 0x7f6e80068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6e97400640 1 -- 172.21.15.161:0/3725321810 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e9811f050 con 0x7f6ea01402e0 2024-05-22T21:14:24.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6ea9195000 1 Processor -- start 2024-05-22T21:14:24.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6e9ea00640 1 --2- 172.21.15.161:0/3725321810 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e80066690 0x7f6e80068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6ea9195000 1 -- start start 2024-05-22T21:14:24.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6e9ea00640 1 --2- 172.21.15.161:0/3725321810 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e80066690 0x7f6e80068b50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6e900011f0 tx=0x7f6e90009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:24.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6ea9195000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9d9c0 0x55bdbac9dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6ea9195000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9e2e0 0x55bdbac53980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6ea9195000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbac542a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6ea9195000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbab1be80 con 0x55bdbac53ec0 2024-05-22T21:14:24.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6ea9195000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbaaa9af0 con 0x55bdbac9d9c0 2024-05-22T21:14:24.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6ea9195000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdbaaa70c0 con 0x55bdbac9e2e0 2024-05-22T21:14:24.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6e9f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbac542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6e9f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbac542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49116/0 (socket says 172.21.15.161:49116) 2024-05-22T21:14:24.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6e9f400640 1 -- 172.21.15.161:0/4128926715 learned_addr learned my addr 172.21.15.161:0/4128926715 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:24.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6e9ea00640 1 --2- 172.21.15.161:0/4128926715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9d9c0 0x55bdbac9dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6e9e000640 1 --2- 172.21.15.161:0/4128926715 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9e2e0 0x55bdbac53980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6e9f400640 1 -- 172.21.15.161:0/4128926715 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9e2e0 msgr2=0x55bdbac53980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:24.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6e9f400640 1 --2- 172.21.15.161:0/4128926715 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9e2e0 0x55bdbac53980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6e9f400640 1 -- 172.21.15.161:0/4128926715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9d9c0 msgr2=0x55bdbac9dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:24.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6e9f400640 1 --2- 172.21.15.161:0/4128926715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9d9c0 0x55bdbac9dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6e9f400640 1 -- 172.21.15.161:0/4128926715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdba79e890 con 0x55bdbac53ec0 2024-05-22T21:14:24.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.864+0000 7f6e9ea00640 1 --2- 172.21.15.161:0/4128926715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9d9c0 0x55bdbac9dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:24.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e9f400640 1 --2- 172.21.15.161:0/4128926715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbac542a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6e980f7ea0 tx=0x7f6e980f0820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4333dc4cfa3854 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:24.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e8be00640 1 -- 172.21.15.161:0/4128926715 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e980f60f0 con 0x55bdbac53ec0 2024-05-22T21:14:24.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e8be00640 1 -- 172.21.15.161:0/4128926715 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e980f83d0 con 0x55bdbac53ec0 2024-05-22T21:14:24.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e8be00640 1 -- 172.21.15.161:0/4128926715 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e980ec6f0 con 0x55bdbac53ec0 2024-05-22T21:14:24.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 -- 172.21.15.161:0/4128926715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 msgr2=0x55bdbac542a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:24.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 --2- 172.21.15.161:0/4128926715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbac542a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6e980f7ea0 tx=0x7f6e980f0820 comp rx=0 tx=0).stop 2024-05-22T21:14:24.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 -- 172.21.15.161:0/4128926715 shutdown_connections 2024-05-22T21:14:24.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 --2- 172.21.15.161:0/4128926715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbac542a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 --2- 172.21.15.161:0/4128926715 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9e2e0 0x55bdbac53980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 --2- 172.21.15.161:0/4128926715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9d9c0 0x55bdbac9dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 -- 172.21.15.161:0/4128926715 >> 172.21.15.161:0/4128926715 conn(0x55bdbac5a440 msgr2=0x55bdbac52260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:24.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 -- 172.21.15.161:0/4128926715 shutdown_connections 2024-05-22T21:14:24.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 -- 172.21.15.161:0/4128926715 wait complete. 2024-05-22T21:14:24.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 -- start start 2024-05-22T21:14:24.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 Processor -- start 2024-05-22T21:14:24.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbac58dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9d9c0 0x55bdbac5efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9e2e0 0x55bdbac5f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbaaa29b0 con 0x55bdbac53ec0 2024-05-22T21:14:24.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbaa41c00 con 0x55bdbac9d9c0 2024-05-22T21:14:24.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdbaa60070 con 0x55bdbac9e2e0 2024-05-22T21:14:24.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6ea9195000 10 client.0 _ll_register_callbacks cb 0x55bdbac3fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:14:24.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e9ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbac58dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e9ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbac58dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49126/0 (socket says 172.21.15.161:49126) 2024-05-22T21:14:24.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e9ea00640 1 -- 172.21.15.161:0/1781447755 learned_addr learned my addr 172.21.15.161:0/1781447755 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:24.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e9e000640 1 --2- 172.21.15.161:0/1781447755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9d9c0 0x55bdbac5efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e9f400640 1 --2- 172.21.15.161:0/1781447755 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9e2e0 0x55bdbac5f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e9ea00640 1 -- 172.21.15.161:0/1781447755 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9e2e0 msgr2=0x55bdbac5f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:24.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e9ea00640 1 --2- 172.21.15.161:0/1781447755 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9e2e0 0x55bdbac5f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e9ea00640 1 -- 172.21.15.161:0/1781447755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9d9c0 msgr2=0x55bdbac5efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:24.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e9ea00640 1 --2- 172.21.15.161:0/1781447755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9d9c0 0x55bdbac5efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e9ea00640 1 -- 172.21.15.161:0/1781447755 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea015c350 con 0x55bdbac53ec0 2024-05-22T21:14:24.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e9e000640 1 --2- 172.21.15.161:0/1781447755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9d9c0 0x55bdbac5efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:24.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e9ea00640 1 --2- 172.21.15.161:0/1781447755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbac58dc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000e720 tx=0x7f6e90000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4aeab461c251ac2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:24.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.868+0000 7f6e88c00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:14:24.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6e88c00640 1 -- 172.21.15.161:0/1781447755 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90007560 con 0x55bdbac53ec0 2024-05-22T21:14:24.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6e88c00640 1 -- 172.21.15.161:0/1781447755 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e900076c0 con 0x55bdbac53ec0 2024-05-22T21:14:24.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 1 -- 172.21.15.161:0/1781447755 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdbaa5f5f0 con 0x55bdbac53ec0 2024-05-22T21:14:24.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6e88c00640 1 -- 172.21.15.161:0/1781447755 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e90013420 con 0x55bdbac53ec0 2024-05-22T21:14:24.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 10 client.5235 fetch_fsmap learned FSMap version 16 2024-05-22T21:14:24.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 1 -- 172.21.15.161:0/1781447755 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55bdbaa67ab0 con 0x55bdbac53ec0 2024-05-22T21:14:24.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6e88c00640 1 -- 172.21.15.161:0/1781447755 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bdbaa5f5f0 con 0x55bdbac53ec0 2024-05-22T21:14:24.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6e88c00640 1 -- 172.21.15.161:0/1781447755 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6e90013580 con 0x55bdbac53ec0 2024-05-22T21:14:24.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 10 client.5235 fetch_fsmap finished waiting for FSMap version 16 2024-05-22T21:14:24.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 10 client.5235 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-22T21:14:24.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 20 client.5235 populate_metadata read hostname 'smithi161' 2024-05-22T21:14:24.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 1 --2- 172.21.15.161:0/1781447755 >> [v2:172.21.15.161:6834/1635458417,v1:172.21.15.161:6836/1635458417] conn(0x55bdbaca4720 0x55bdbaca4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 4 client.5235 mds_command: new command op to 5055 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T21:14:24.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 1 -- 172.21.15.161:0/1781447755 --> [v2:172.21.15.161:6834/1635458417,v1:172.21.15.161:6836/1635458417] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdbaa9f790 con 0x55bdbaca4720 2024-05-22T21:14:24.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6e9e000640 1 --2- 172.21.15.161:0/1781447755 >> [v2:172.21.15.161:6834/1635458417,v1:172.21.15.161:6836/1635458417] conn(0x55bdbaca4720 0x55bdbaca4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6e9e000640 1 --2- 172.21.15.161:0/1781447755 >> [v2:172.21.15.161:6834/1635458417,v1:172.21.15.161:6836/1635458417] conn(0x55bdbaca4720 0x55bdbaca4b00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa447e9b4498a5a5 server_cookie=f2f6b1637818fc9e in_seq=0 out_seq=0 2024-05-22T21:14:24.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6e88c00640 10 client.5235 ms_handle_connect on v2:172.21.15.161:6834/1635458417 2024-05-22T21:14:24.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6e88c00640 1 -- 172.21.15.161:0/1781447755 <== mds.0 v2:172.21.15.161:6834/1635458417 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bdbaa9f790 con 0x55bdbaca4720 2024-05-22T21:14:24.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6e88c00640 10 client.5235 handle_command_reply: tid=0 2024-05-22T21:14:24.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 1 client.5235 shutdown 2024-05-22T21:14:24.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 1 -- 172.21.15.161:0/1781447755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 msgr2=0x55bdbac58dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:24.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 1 --2- 172.21.15.161:0/1781447755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbac58dc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000e720 tx=0x7f6e90000ee0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 1 -- 172.21.15.161:0/1781447755 shutdown_connections 2024-05-22T21:14:24.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 1 --2- 172.21.15.161:0/1781447755 >> [v2:172.21.15.161:6834/1635458417,v1:172.21.15.161:6836/1635458417] conn(0x55bdbaca4720 0x55bdbaca4b00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 1 --2- 172.21.15.161:0/1781447755 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9e2e0 0x55bdbac5f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 1 --2- 172.21.15.161:0/1781447755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9d9c0 0x55bdbac5efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6e88c00640 0 client.5235 ms_handle_reset on v2:172.21.15.161:6834/1635458417 2024-05-22T21:14:24.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 1 --2- 172.21.15.161:0/1781447755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbac58dc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 1 -- 172.21.15.161:0/1781447755 >> 172.21.15.161:0/1781447755 conn(0x55bdbac5a440 msgr2=0x55bdbac52960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:24.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 1 -- 172.21.15.161:0/1781447755 shutdown_connections 2024-05-22T21:14:24.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 1 -- 172.21.15.161:0/1781447755 wait complete. 2024-05-22T21:14:24.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.872+0000 7f6ea9195000 20 client.5235 trim_cache size 0 max 16384 2024-05-22T21:14:24.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6ea9195000 1 Processor -- start 2024-05-22T21:14:24.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6ea9195000 1 -- start start 2024-05-22T21:14:24.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6ea9195000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac53ec0 0x55bdbad02400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6ea9195000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac9d9c0 0x55bdbad02940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6ea9195000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9e2e0 0x55bdbad02e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6ea9195000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdba75f940 con 0x55bdbac9d9c0 2024-05-22T21:14:24.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6ea9195000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbaaa69b0 con 0x55bdbac53ec0 2024-05-22T21:14:24.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6ea9195000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdbac9c9a0 con 0x55bdbac9e2e0 2024-05-22T21:14:24.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6e9e000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac9d9c0 0x55bdbad02940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6e9e000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac9d9c0 0x55bdbad02940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49142/0 (socket says 172.21.15.161:49142) 2024-05-22T21:14:24.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6e9e000640 1 -- 172.21.15.161:0/2817324836 learned_addr learned my addr 172.21.15.161:0/2817324836 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:24.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6e9ea00640 1 --2- 172.21.15.161:0/2817324836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac53ec0 0x55bdbad02400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6e9f400640 1 --2- 172.21.15.161:0/2817324836 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9e2e0 0x55bdbad02e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6e9e000640 1 -- 172.21.15.161:0/2817324836 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9e2e0 msgr2=0x55bdbad02e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:24.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6e9e000640 1 --2- 172.21.15.161:0/2817324836 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9e2e0 0x55bdbad02e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6e9e000640 1 -- 172.21.15.161:0/2817324836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac53ec0 msgr2=0x55bdbad02400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:24.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6e9e000640 1 --2- 172.21.15.161:0/2817324836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac53ec0 0x55bdbad02400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6e9e000640 1 -- 172.21.15.161:0/2817324836 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdbaa4dc00 con 0x55bdbac9d9c0 2024-05-22T21:14:24.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.888+0000 7f6e9ea00640 1 --2- 172.21.15.161:0/2817324836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac53ec0 0x55bdbad02400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:24.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e9e000640 1 --2- 172.21.15.161:0/2817324836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac9d9c0 0x55bdbad02940 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00db90 tx=0x7f6e8c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a586c26dce794f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:24.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e94c00640 1 -- 172.21.15.161:0/2817324836 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c01c070 con 0x55bdbac9d9c0 2024-05-22T21:14:24.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e94c00640 1 -- 172.21.15.161:0/2817324836 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c00dde0 con 0x55bdbac9d9c0 2024-05-22T21:14:24.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e94c00640 1 -- 172.21.15.161:0/2817324836 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c004260 con 0x55bdbac9d9c0 2024-05-22T21:14:24.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 -- 172.21.15.161:0/2817324836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac9d9c0 msgr2=0x55bdbad02940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:24.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 --2- 172.21.15.161:0/2817324836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac9d9c0 0x55bdbad02940 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00db90 tx=0x7f6e8c008040 comp rx=0 tx=0).stop 2024-05-22T21:14:24.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 -- 172.21.15.161:0/2817324836 shutdown_connections 2024-05-22T21:14:24.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 --2- 172.21.15.161:0/2817324836 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9e2e0 0x55bdbad02e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 --2- 172.21.15.161:0/2817324836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac9d9c0 0x55bdbad02940 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 --2- 172.21.15.161:0/2817324836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac53ec0 0x55bdbad02400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 -- 172.21.15.161:0/2817324836 >> 172.21.15.161:0/2817324836 conn(0x55bdbac5a440 msgr2=0x55bdbac59470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:24.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 -- 172.21.15.161:0/2817324836 shutdown_connections 2024-05-22T21:14:24.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 -- 172.21.15.161:0/2817324836 wait complete. 2024-05-22T21:14:24.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 -- start start 2024-05-22T21:14:24.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 Processor -- start 2024-05-22T21:14:24.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbad08060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9d9c0 0x55bdbad085a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9e2e0 0x55bdbacebdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbaaa1660 con 0x55bdbac53ec0 2024-05-22T21:14:24.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbaa974d0 con 0x55bdbac9e2e0 2024-05-22T21:14:24.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdbaa3fc10 con 0x55bdbac9d9c0 2024-05-22T21:14:24.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 10 client.0 _ll_register_callbacks cb 0x7f6e980ec3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:14:24.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e9ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbad08060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e9ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbad08060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49144/0 (socket says 172.21.15.161:49144) 2024-05-22T21:14:24.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e9ea00640 1 -- 172.21.15.161:0/1836903053 learned_addr learned my addr 172.21.15.161:0/1836903053 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:24.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e9f400640 1 --2- 172.21.15.161:0/1836903053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9e2e0 0x55bdbacebdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e9e000640 1 --2- 172.21.15.161:0/1836903053 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9d9c0 0x55bdbad085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e9ea00640 1 -- 172.21.15.161:0/1836903053 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9d9c0 msgr2=0x55bdbad085a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:24.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e9ea00640 1 --2- 172.21.15.161:0/1836903053 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9d9c0 0x55bdbad085a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e9ea00640 1 -- 172.21.15.161:0/1836903053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9e2e0 msgr2=0x55bdbacebdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:24.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e9ea00640 1 --2- 172.21.15.161:0/1836903053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9e2e0 0x55bdbacebdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e9ea00640 1 -- 172.21.15.161:0/1836903053 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e8c00e040 con 0x55bdbac53ec0 2024-05-22T21:14:24.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e9e000640 1 --2- 172.21.15.161:0/1836903053 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9d9c0 0x55bdbad085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:24.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e9ea00640 1 --2- 172.21.15.161:0/1836903053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbad08060 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000e6f0 tx=0x7f6e90007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99eb845403054b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:24.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e88c00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:14:24.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e88c00640 1 -- 172.21.15.161:0/1836903053 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90001cf0 con 0x55bdbac53ec0 2024-05-22T21:14:24.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e88c00640 1 -- 172.21.15.161:0/1836903053 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e90013cf0 con 0x55bdbac53ec0 2024-05-22T21:14:24.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6ea9195000 1 -- 172.21.15.161:0/1836903053 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdbaa72460 con 0x55bdbac53ec0 2024-05-22T21:14:24.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.892+0000 7f6e88c00640 1 -- 172.21.15.161:0/1836903053 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e90007560 con 0x55bdbac53ec0 2024-05-22T21:14:24.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 10 client.5241 fetch_fsmap learned FSMap version 16 2024-05-22T21:14:24.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 1 -- 172.21.15.161:0/1836903053 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55bdba79ecc0 con 0x55bdbac53ec0 2024-05-22T21:14:24.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6e88c00640 1 -- 172.21.15.161:0/1836903053 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bdbaa72460 con 0x55bdbac53ec0 2024-05-22T21:14:24.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6e88c00640 1 -- 172.21.15.161:0/1836903053 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6e90013420 con 0x55bdbac53ec0 2024-05-22T21:14:24.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 10 client.5241 fetch_fsmap finished waiting for FSMap version 16 2024-05-22T21:14:24.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 10 client.5241 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-22T21:14:24.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 20 client.5241 populate_metadata read hostname 'smithi161' 2024-05-22T21:14:24.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 1 --2- 172.21.15.161:0/1836903053 >> [v2:172.21.15.161:6834/1635458417,v1:172.21.15.161:6836/1635458417] conn(0x55bdbacec300 0x55bdbacec6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:24.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 4 client.5241 mds_command: new command op to 5055 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-22T21:14:24.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 1 -- 172.21.15.161:0/1836903053 --> [v2:172.21.15.161:6834/1635458417,v1:172.21.15.161:6836/1635458417] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bdbaa6b9c0 con 0x55bdbacec300 2024-05-22T21:14:24.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6e9e000640 1 --2- 172.21.15.161:0/1836903053 >> [v2:172.21.15.161:6834/1635458417,v1:172.21.15.161:6836/1635458417] conn(0x55bdbacec300 0x55bdbacec6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:24.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6e88c00640 10 client.5241 ms_handle_connect on v2:172.21.15.161:6834/1635458417 2024-05-22T21:14:24.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6e9e000640 1 --2- 172.21.15.161:0/1836903053 >> [v2:172.21.15.161:6834/1635458417,v1:172.21.15.161:6836/1635458417] conn(0x55bdbacec300 0x55bdbacec6e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5cfdd9bac804ee7 server_cookie=9bf397d53b1e8fba in_seq=0 out_seq=0 2024-05-22T21:14:24.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6e88c00640 1 -- 172.21.15.161:0/1836903053 <== mds.0 v2:172.21.15.161:6834/1635458417 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55bdbaa6b9c0 con 0x55bdbacec300 2024-05-22T21:14:24.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6e88c00640 10 client.5241 handle_command_reply: tid=0 2024-05-22T21:14:24.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 1 client.5241 shutdown 2024-05-22T21:14:24.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 1 -- 172.21.15.161:0/1836903053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 msgr2=0x55bdbad08060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:24.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 1 --2- 172.21.15.161:0/1836903053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbad08060 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000e6f0 tx=0x7f6e90007000 comp rx=0 tx=0).stop 2024-05-22T21:14:24.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 1 -- 172.21.15.161:0/1836903053 shutdown_connections 2024-05-22T21:14:24.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 1 --2- 172.21.15.161:0/1836903053 >> [v2:172.21.15.161:6834/1635458417,v1:172.21.15.161:6836/1635458417] conn(0x55bdbacec300 0x55bdbacec6e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 1 --2- 172.21.15.161:0/1836903053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdbac9e2e0 0x55bdbacebdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 1 --2- 172.21.15.161:0/1836903053 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bdbac9d9c0 0x55bdbad085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6e88c00640 0 client.5241 ms_handle_reset on v2:172.21.15.161:6834/1635458417 2024-05-22T21:14:24.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 1 --2- 172.21.15.161:0/1836903053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bdbac53ec0 0x55bdbad08060 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 1 -- 172.21.15.161:0/1836903053 >> 172.21.15.161:0/1836903053 conn(0x55bdbac5a440 msgr2=0x55bdbac58be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:24.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 1 -- 172.21.15.161:0/1836903053 shutdown_connections 2024-05-22T21:14:24.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 1 -- 172.21.15.161:0/1836903053 wait complete. 2024-05-22T21:14:24.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea9195000 20 client.5241 trim_cache size 0 max 16384 2024-05-22T21:14:24.904 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:14:24.904 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:14:24.904 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 0, 2024-05-22T21:14:24.904 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 5055, 2024-05-22T21:14:24.905 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2024-05-22T21:14:24.905 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2024-05-22T21:14:24.905 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2024-05-22T21:14:24.905 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 5.8425777630000004, 2024-05-22T21:14:24.905 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 16, 2024-05-22T21:14:24.905 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 33, 2024-05-22T21:14:24.905 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 33, 2024-05-22T21:14:24.905 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 13.08038277 2024-05-22T21:14:24.905 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:14:24.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea5e00640 1 -- 172.21.15.161:0/3725321810 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e80066690 msgr2=0x7f6e80068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:24.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea5e00640 1 --2- 172.21.15.161:0/3725321810 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e80066690 0x7f6e80068b50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6e900011f0 tx=0x7f6e90009040 comp rx=0 tx=0).stop 2024-05-22T21:14:24.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea5e00640 1 -- 172.21.15.161:0/3725321810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea01402e0 msgr2=0x7f6ea00ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:24.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea5e00640 1 --2- 172.21.15.161:0/3725321810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea01402e0 0x7f6ea00ad8a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6e9801a4a0 tx=0x7f6e98019610 comp rx=0 tx=0).stop 2024-05-22T21:14:24.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea5e00640 1 -- 172.21.15.161:0/3725321810 shutdown_connections 2024-05-22T21:14:24.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea5e00640 1 --2- 172.21.15.161:0/3725321810 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e80066690 0x7f6e80068b50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea5e00640 1 --2- 172.21.15.161:0/3725321810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea01402e0 0x7f6ea00ad8a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea5e00640 1 --2- 172.21.15.161:0/3725321810 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ea013fc80 0x7f6ea013e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea5e00640 1 --2- 172.21.15.161:0/3725321810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ea013cfc0 0x7f6ea013bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:24.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea5e00640 1 -- 172.21.15.161:0/3725321810 >> 172.21.15.161:0/3725321810 conn(0x7f6ea006a2d0 msgr2=0x7f6ea015da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:24.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea5e00640 1 -- 172.21.15.161:0/3725321810 shutdown_connections 2024-05-22T21:14:24.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:24.896+0000 7f6ea5e00640 1 -- 172.21.15.161:0/3725321810 wait complete. 2024-05-22T21:14:24.917 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:14:25.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.048+0000 7ff5da200640 1 Processor -- start 2024-05-22T21:14:25.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.048+0000 7ff5da200640 1 -- start start 2024-05-22T21:14:25.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.052+0000 7ff5da200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5d415e130 0x7ff5d415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:25.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.052+0000 7ff5da200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff5d415d410 0x7ff5d415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:25.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.052+0000 7ff5da200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d4071670 con 0x7ff5d415dd50 2024-05-22T21:14:25.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.052+0000 7ff5da200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d415d870 con 0x7ff5d415d410 2024-05-22T21:14:25.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.052+0000 7ff5da200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d4070a60 con 0x7ff5d415e130 2024-05-22T21:14:25.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.052+0000 7ff5d3e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff5d415d410 0x7ff5d415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:25.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.052+0000 7ff5caa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5d415e130 0x7ff5d415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:25.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.052+0000 7ff5d3400640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff5d415dd50 0x7ff5d406d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:50148/0 (socket says 172.21.15.161:50148) 2024-05-22T21:14:25.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.052+0000 7ff5d3400640 1 -- 172.21.15.161:0/3932769362 learned_addr learned my addr 172.21.15.161:0/3932769362 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:25.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.052+0000 7ff5caa00640 1 -- 172.21.15.161:0/3932769362 >> v2:172.21.15.179:3300/0 conn(0x7ff5d415d410 msgr2=0x7ff5d415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:25.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.052+0000 7ff5caa00640 1 --2- 172.21.15.161:0/3932769362 >> v2:172.21.15.179:3300/0 conn(0x7ff5d415d410 0x7ff5d415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:25.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.052+0000 7ff5caa00640 1 -- 172.21.15.161:0/3932769362 >> v1:172.21.15.179:6789/0 conn(0x7ff5d415dd50 legacy=0x7ff5d406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:25.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.052+0000 7ff5caa00640 1 -- 172.21.15.161:0/3932769362 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d4070d40 con 0x7ff5d415e130 2024-05-22T21:14:25.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.056+0000 7ff5caa00640 1 --2- 172.21.15.161:0/3932769362 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5d415e130 0x7ff5d415ced0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40097c0 tx=0x7ff5c401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d2a73eb27976425 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:25.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.056+0000 7ff5d2a00640 1 -- 172.21.15.161:0/3932769362 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40ac640 con 0x7ff5d415e130 2024-05-22T21:14:25.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.056+0000 7ff5d2a00640 1 -- 172.21.15.161:0/3932769362 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c40ac7a0 con 0x7ff5d415e130 2024-05-22T21:14:25.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.056+0000 7ff5d2a00640 1 -- 172.21.15.161:0/3932769362 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40acb30 con 0x7ff5d415e130 2024-05-22T21:14:25.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.056+0000 7ff5da200640 1 -- 172.21.15.161:0/3932769362 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5d415e130 msgr2=0x7ff5d415ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:25.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.056+0000 7ff5da200640 1 --2- 172.21.15.161:0/3932769362 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5d415e130 0x7ff5d415ced0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40097c0 tx=0x7ff5c401c3e0 comp rx=0 tx=0).stop 2024-05-22T21:14:25.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.056+0000 7ff5da200640 1 -- 172.21.15.161:0/3932769362 shutdown_connections 2024-05-22T21:14:25.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.056+0000 7ff5da200640 1 --2- 172.21.15.161:0/3932769362 >> v2:172.21.15.179:3300/0 conn(0x7ff5d415d410 0x7ff5d415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:25.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.056+0000 7ff5da200640 1 --2- 172.21.15.161:0/3932769362 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5d415e130 0x7ff5d415ced0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:25.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.056+0000 7ff5da200640 1 -- 172.21.15.161:0/3932769362 >> 172.21.15.161:0/3932769362 conn(0x7ff5d40b0bc0 msgr2=0x7ff5d40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:25.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.056+0000 7ff5da200640 1 -- 172.21.15.161:0/3932769362 shutdown_connections 2024-05-22T21:14:25.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.056+0000 7ff5da200640 1 -- 172.21.15.161:0/3932769362 wait complete. 2024-05-22T21:14:25.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.056+0000 7ff5da200640 1 Processor -- start 2024-05-22T21:14:25.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.056+0000 7ff5da200640 1 -- start start 2024-05-22T21:14:25.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5da200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5d415d410 0x7ff5d4158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:25.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5da200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5d415dd50 0x7ff5d4157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:25.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5da200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5d415e130 0x7ff5d41531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:25.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5da200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d4159100 con 0x7ff5d415dd50 2024-05-22T21:14:25.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5da200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d4157470 con 0x7ff5d415d410 2024-05-22T21:14:25.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5da200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d4158170 con 0x7ff5d415e130 2024-05-22T21:14:25.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5caa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5d415dd50 0x7ff5d4157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:25.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5caa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5d415dd50 0x7ff5d4157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49160/0 (socket says 172.21.15.161:49160) 2024-05-22T21:14:25.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5caa00640 1 -- 172.21.15.161:0/3395400830 learned_addr learned my addr 172.21.15.161:0/3395400830 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:25.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5d3400640 1 --2- 172.21.15.161:0/3395400830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5d415d410 0x7ff5d4158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:25.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5d3e00640 1 --2- 172.21.15.161:0/3395400830 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5d415e130 0x7ff5d41531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:25.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5caa00640 1 -- 172.21.15.161:0/3395400830 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5d415e130 msgr2=0x7ff5d41531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:25.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5caa00640 1 --2- 172.21.15.161:0/3395400830 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5d415e130 0x7ff5d41531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:25.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5caa00640 1 -- 172.21.15.161:0/3395400830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5d415d410 msgr2=0x7ff5d4158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:25.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5caa00640 1 --2- 172.21.15.161:0/3395400830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5d415d410 0x7ff5d4158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:25.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5caa00640 1 -- 172.21.15.161:0/3395400830 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c4009420 con 0x7ff5d415dd50 2024-05-22T21:14:25.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5caa00640 1 --2- 172.21.15.161:0/3395400830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5d415dd50 0x7ff5d4157c30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff5c401aa90 tx=0x7ff5c4019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:25.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5cbe00640 1 -- 172.21.15.161:0/3395400830 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40b5600 con 0x7ff5d415dd50 2024-05-22T21:14:25.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5cbe00640 1 -- 172.21.15.161:0/3395400830 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c40b5760 con 0x7ff5d415dd50 2024-05-22T21:14:25.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5da200640 1 -- 172.21.15.161:0/3395400830 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d4158b40 con 0x7ff5d415dd50 2024-05-22T21:14:25.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5cbe00640 1 -- 172.21.15.161:0/3395400830 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40b5b10 con 0x7ff5d415dd50 2024-05-22T21:14:25.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5da200640 1 -- 172.21.15.161:0/3395400830 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d4157710 con 0x7ff5d415dd50 2024-05-22T21:14:25.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5da200640 1 -- 172.21.15.161:0/3395400830 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d41572d0 con 0x7ff5d415dd50 2024-05-22T21:14:25.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.060+0000 7ff5cbe00640 1 -- 172.21.15.161:0/3395400830 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff5c40b3040 con 0x7ff5d415dd50 2024-05-22T21:14:25.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.064+0000 7ff5cbe00640 1 --2- 172.21.15.161:0/3395400830 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff5b0066690 0x7ff5b0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:25.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.064+0000 7ff5cbe00640 1 -- 172.21.15.161:0/3395400830 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5c411e720 con 0x7ff5d415dd50 2024-05-22T21:14:25.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.064+0000 7ff5d3400640 1 --2- 172.21.15.161:0/3395400830 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff5b0066690 0x7ff5b0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:25.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.064+0000 7ff5d3400640 1 --2- 172.21.15.161:0/3395400830 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff5b0066690 0x7ff5b0068b50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc005890 tx=0x7ff5bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:25.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.068+0000 7ff5cbe00640 1 -- 172.21.15.161:0/3395400830 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff5d41572d0 con 0x7ff5d415dd50 2024-05-22T21:14:25.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.256+0000 7ff5da200640 1 -- 172.21.15.161:0/3395400830 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff5d41572d0 con 0x7ff5d415dd50 2024-05-22T21:14:25.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.256+0000 7ff5cbe00640 1 -- 172.21.15.161:0/3395400830 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff5d41572d0 con 0x7ff5d415dd50 2024-05-22T21:14:25.259 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:14:25.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.260+0000 7ff5da200640 1 -- 172.21.15.161:0/3395400830 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff5b0066690 msgr2=0x7ff5b0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:25.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.260+0000 7ff5da200640 1 --2- 172.21.15.161:0/3395400830 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff5b0066690 0x7ff5b0068b50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc005890 tx=0x7ff5bc009040 comp rx=0 tx=0).stop 2024-05-22T21:14:25.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.260+0000 7ff5da200640 1 -- 172.21.15.161:0/3395400830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5d415dd50 msgr2=0x7ff5d4157c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:25.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.260+0000 7ff5da200640 1 --2- 172.21.15.161:0/3395400830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5d415dd50 0x7ff5d4157c30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff5c401aa90 tx=0x7ff5c4019180 comp rx=0 tx=0).stop 2024-05-22T21:14:25.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.260+0000 7ff5da200640 1 -- 172.21.15.161:0/3395400830 shutdown_connections 2024-05-22T21:14:25.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.260+0000 7ff5da200640 1 --2- 172.21.15.161:0/3395400830 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff5b0066690 0x7ff5b0068b50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:25.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.260+0000 7ff5da200640 1 --2- 172.21.15.161:0/3395400830 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5d415e130 0x7ff5d41531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:25.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.260+0000 7ff5da200640 1 --2- 172.21.15.161:0/3395400830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5d415dd50 0x7ff5d4157c30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:25.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.260+0000 7ff5da200640 1 --2- 172.21.15.161:0/3395400830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5d415d410 0x7ff5d4158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:25.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.260+0000 7ff5da200640 1 -- 172.21.15.161:0/3395400830 >> 172.21.15.161:0/3395400830 conn(0x7ff5d40b0bc0 msgr2=0x7ff5d40aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:25.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.260+0000 7ff5da200640 1 -- 172.21.15.161:0/3395400830 shutdown_connections 2024-05-22T21:14:25.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:25.260+0000 7ff5da200640 1 -- 172.21.15.161:0/3395400830 wait complete. 2024-05-22T21:14:25.279 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:14:25.280 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:14:25.280 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-05-22T21:14:25.280 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-22T21:14:25.280 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-22T21:14:25.280 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:14:25.280 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:14:25.280 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:14:25.280 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:14:25.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:25.281 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-05-22T21:14:25.289 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:14:25.289 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:14:25.289 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:14:25.289 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:25.290 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-05-22T21:14:25.290 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:25.290 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:14:25.290 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:25.290 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2024-05-22T21:14:25.290 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:14:25.290 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:25.290 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2024-05-22T21:14:25.290 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:14:25.290 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:25.291 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:14:25.291 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:25.291 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:14:25.291 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42427sec preferred_lft 42427sec 2024-05-22T21:14:25.291 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-05-22T21:14:25.291 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:25.291 INFO:teuthology.orchestra.run.smithi161.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:14:25.291 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 02:42:b9:0f:57:df brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:25.291 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:14:25.291 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:25.291 INFO:teuthology.orchestra.run.smithi161.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:14:25.291 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:25.291 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:14:25.291 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:25.292 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:14:25.292 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:25.292 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:14:25.292 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-22T21:14:25.292 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:14:25.292 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:25.292 INFO:teuthology.orchestra.run.smithi161.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:14:25.292 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-22T21:14:25.292 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:14:25.292 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:25.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:25.294 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:14:25.354 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-05-22T21:14:25.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:25.356 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-05-22T21:14:25.411 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-05-22T21:14:25.412 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:14:25.412 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:14:25.412 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:14:25.412 INFO:teuthology.orchestra.run.smithi161.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:14:25.412 INFO:teuthology.orchestra.run.smithi161.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:14:25.412 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:14:25.412 INFO:teuthology.orchestra.run.smithi161.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:14:25.412 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:14:25.412 INFO:teuthology.orchestra.run.smithi161.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:14:25.412 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:14:25.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:25.413 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:14:25.413 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:14:25.413 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:14:25.413 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:14:25.413 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:14:25.413 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:14:25.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:25.501 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-05-22T21:14:25.510 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-22T21:14:25.510 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-22T21:14:25.510 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:14:25.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:25.510 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2024-05-22T21:14:25.571 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:14:25.572 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-22T21:14:25.620 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:14:25.620 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:14:25.620 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:14:25.671 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:14:25.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:25.672 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:14:25.759 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:14:25.759 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:14:25.759 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:14:25.774 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:14:25.774 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:14:25.774 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:14:25.774 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:14:25.774 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:14:25.783 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:14:25.783 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:14:25.783 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:14:25.783 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:14:25.783 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:14:25.783 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:14:25.784 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:14:25.784 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:14:25.788 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:14:25.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:25.788 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:14:25.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:25.807 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:14:25.870 INFO:teuthology.orchestra.run.smithi161.stdout:client4874 2024-05-22T21:14:25.870 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:14:25.870 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:25.920 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:14:26.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.096+0000 7f9abd400640 1 Processor -- start 2024-05-22T21:14:26.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.096+0000 7f9abd400640 1 -- start start 2024-05-22T21:14:26.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.096+0000 7f9abd400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9ab815dd50 0x7f9ab806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:26.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.096+0000 7f9abd400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9ab815e130 0x7f9ab815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:26.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.096+0000 7f9abd400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9ab815d410 0x7f9ab815a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:26.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.096+0000 7f9abd400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab8071670 con 0x7f9ab815dd50 2024-05-22T21:14:26.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.096+0000 7f9abd400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab815d8a0 con 0x7f9ab815d410 2024-05-22T21:14:26.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.096+0000 7f9abd400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab806de40 con 0x7f9ab815e130 2024-05-22T21:14:26.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.096+0000 7f9ab6800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9ab815dd50 0x7f9ab806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:26.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.100+0000 7f9ab6800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9ab815dd50 0x7f9ab806d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55606/0 (socket says 172.21.15.161:55606) 2024-05-22T21:14:26.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.100+0000 7f9ab6800640 1 -- 172.21.15.161:0/457005509 learned_addr learned my addr 172.21.15.161:0/457005509 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:26.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.100+0000 7f9aade00640 1 --2- 172.21.15.161:0/457005509 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9ab815e130 0x7f9ab815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:26.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.100+0000 7f9ab7200640 1 --2- 172.21.15.161:0/457005509 >> v2:172.21.15.179:3300/0 conn(0x7f9ab815d410 0x7f9ab815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:26.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.100+0000 7f9ab6800640 1 -- 172.21.15.161:0/457005509 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9ab815e130 msgr2=0x7f9ab815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:26.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.100+0000 7f9ab6800640 1 --2- 172.21.15.161:0/457005509 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9ab815e130 0x7f9ab815ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:26.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.100+0000 7f9ab6800640 1 -- 172.21.15.161:0/457005509 >> v2:172.21.15.179:3300/0 conn(0x7f9ab815d410 msgr2=0x7f9ab815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:26.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.100+0000 7f9ab6800640 1 --2- 172.21.15.161:0/457005509 >> v2:172.21.15.179:3300/0 conn(0x7f9ab815d410 0x7f9ab815a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:26.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.100+0000 7f9ab6800640 1 -- 172.21.15.161:0/457005509 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab8070a60 con 0x7f9ab815dd50 2024-05-22T21:14:26.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.100+0000 7f9ab6800640 1 --2- 172.21.15.161:0/457005509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ab815dd50 0x7f9ab806d900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80721d0 tx=0x7f9aa801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799302ab5ede4d06 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:26.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.100+0000 7f9ab5e00640 1 -- 172.21.15.161:0/457005509 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa80ab620 con 0x7f9ab815dd50 2024-05-22T21:14:26.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.100+0000 7f9ab5e00640 1 -- 172.21.15.161:0/457005509 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa80ab780 con 0x7f9ab815dd50 2024-05-22T21:14:26.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.100+0000 7f9ab5e00640 1 -- 172.21.15.161:0/457005509 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa80abb10 con 0x7f9ab815dd50 2024-05-22T21:14:26.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 -- 172.21.15.161:0/457005509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ab815dd50 msgr2=0x7f9ab806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:26.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 --2- 172.21.15.161:0/457005509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ab815dd50 0x7f9ab806d900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80721d0 tx=0x7f9aa801c460 comp rx=0 tx=0).stop 2024-05-22T21:14:26.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 -- 172.21.15.161:0/457005509 shutdown_connections 2024-05-22T21:14:26.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 --2- 172.21.15.161:0/457005509 >> v2:172.21.15.179:3300/0 conn(0x7f9ab815d410 0x7f9ab815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:26.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 --2- 172.21.15.161:0/457005509 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9ab815e130 0x7f9ab815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:26.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 --2- 172.21.15.161:0/457005509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ab815dd50 0x7f9ab806d900 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:26.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 -- 172.21.15.161:0/457005509 >> 172.21.15.161:0/457005509 conn(0x7f9ab80b0bc0 msgr2=0x7f9ab80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:26.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 -- 172.21.15.161:0/457005509 shutdown_connections 2024-05-22T21:14:26.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 -- 172.21.15.161:0/457005509 wait complete. 2024-05-22T21:14:26.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 Processor -- start 2024-05-22T21:14:26.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 -- start start 2024-05-22T21:14:26.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9ab815d410 0x7f9ab814fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:26.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ab815dd50 0x7f9ab814b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:26.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ab815e130 0x7f9ab8151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:26.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab814b8c0 con 0x7f9ab815dd50 2024-05-22T21:14:26.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab81518d0 con 0x7f9ab815e130 2024-05-22T21:14:26.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9abd400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab81502f0 con 0x7f9ab815d410 2024-05-22T21:14:26.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9aade00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ab815dd50 0x7f9ab814b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:26.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9aade00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ab815dd50 0x7f9ab814b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55612/0 (socket says 172.21.15.161:55612) 2024-05-22T21:14:26.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9aade00640 1 -- 172.21.15.161:0/203632016 learned_addr learned my addr 172.21.15.161:0/203632016 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:26.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9ab6800640 1 --2- 172.21.15.161:0/203632016 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9ab815d410 0x7f9ab814fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:26.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9ab7200640 1 --2- 172.21.15.161:0/203632016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ab815e130 0x7f9ab8151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:26.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9aade00640 1 -- 172.21.15.161:0/203632016 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9ab815d410 msgr2=0x7f9ab814fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:26.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9aade00640 1 --2- 172.21.15.161:0/203632016 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9ab815d410 0x7f9ab814fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:26.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9aade00640 1 -- 172.21.15.161:0/203632016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ab815e130 msgr2=0x7f9ab8151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:26.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9aade00640 1 --2- 172.21.15.161:0/203632016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ab815e130 0x7f9ab8151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:26.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9aade00640 1 -- 172.21.15.161:0/203632016 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa80094d0 con 0x7f9ab815dd50 2024-05-22T21:14:26.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.104+0000 7f9aade00640 1 --2- 172.21.15.161:0/203632016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ab815dd50 0x7f9ab814b380 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000e870 tx=0x7f9aa000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:26.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.108+0000 7f9aaf400640 1 -- 172.21.15.161:0/203632016 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0009530 con 0x7f9ab815dd50 2024-05-22T21:14:26.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.108+0000 7f9aaf400640 1 -- 172.21.15.161:0/203632016 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa000c940 con 0x7f9ab815dd50 2024-05-22T21:14:26.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.108+0000 7f9abd400640 1 -- 172.21.15.161:0/203632016 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab81505d0 con 0x7f9ab815dd50 2024-05-22T21:14:26.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.108+0000 7f9aaf400640 1 -- 172.21.15.161:0/203632016 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa000ccd0 con 0x7f9ab815dd50 2024-05-22T21:14:26.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.108+0000 7f9abd400640 1 -- 172.21.15.161:0/203632016 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab8150b90 con 0x7f9ab815dd50 2024-05-22T21:14:26.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.108+0000 7f9abd400640 1 -- 172.21.15.161:0/203632016 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab8067220 con 0x7f9ab815dd50 2024-05-22T21:14:26.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.108+0000 7f9aaf400640 1 -- 172.21.15.161:0/203632016 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9aa0013470 con 0x7f9ab815dd50 2024-05-22T21:14:26.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.108+0000 7f9aaf400640 1 --2- 172.21.15.161:0/203632016 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a94066690 0x7f9a94068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:26.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.108+0000 7f9aaf400640 1 -- 172.21.15.161:0/203632016 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9aa0084210 con 0x7f9ab815dd50 2024-05-22T21:14:26.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.108+0000 7f9ab6800640 1 --2- 172.21.15.161:0/203632016 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a94066690 0x7f9a94068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:26.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.112+0000 7f9ab6800640 1 --2- 172.21.15.161:0/203632016 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a94066690 0x7f9a94068b50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80af000 tx=0x7f9aa80b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:26.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.112+0000 7f9aaf400640 1 -- 172.21.15.161:0/203632016 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9aa0051630 con 0x7f9ab815dd50 2024-05-22T21:14:26.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.300+0000 7f9abd400640 1 -- 172.21.15.161:0/203632016 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9ab8152490 con 0x7f9ab815dd50 2024-05-22T21:14:26.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.300+0000 7f9aaf400640 1 -- 172.21.15.161:0/203632016 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9aa0054c50 con 0x7f9ab815dd50 2024-05-22T21:14:26.304 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:14:26.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.304+0000 7f9abd400640 1 -- 172.21.15.161:0/203632016 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a94066690 msgr2=0x7f9a94068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:26.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.304+0000 7f9abd400640 1 --2- 172.21.15.161:0/203632016 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a94066690 0x7f9a94068b50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80af000 tx=0x7f9aa80b6040 comp rx=0 tx=0).stop 2024-05-22T21:14:26.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.304+0000 7f9abd400640 1 -- 172.21.15.161:0/203632016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ab815dd50 msgr2=0x7f9ab814b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:26.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.304+0000 7f9abd400640 1 --2- 172.21.15.161:0/203632016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ab815dd50 0x7f9ab814b380 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000e870 tx=0x7f9aa000ed30 comp rx=0 tx=0).stop 2024-05-22T21:14:26.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.304+0000 7f9abd400640 1 -- 172.21.15.161:0/203632016 shutdown_connections 2024-05-22T21:14:26.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.304+0000 7f9abd400640 1 --2- 172.21.15.161:0/203632016 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a94066690 0x7f9a94068b50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:26.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.304+0000 7f9abd400640 1 --2- 172.21.15.161:0/203632016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ab815e130 0x7f9ab8151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:26.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.304+0000 7f9abd400640 1 --2- 172.21.15.161:0/203632016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ab815dd50 0x7f9ab814b380 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:26.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.304+0000 7f9abd400640 1 --2- 172.21.15.161:0/203632016 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9ab815d410 0x7f9ab814fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:26.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.304+0000 7f9abd400640 1 -- 172.21.15.161:0/203632016 >> 172.21.15.161:0/203632016 conn(0x7f9ab80b0bc0 msgr2=0x7f9ab80cc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:26.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.304+0000 7f9abd400640 1 -- 172.21.15.161:0/203632016 shutdown_connections 2024-05-22T21:14:26.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:26.304+0000 7f9abd400640 1 -- 172.21.15.161:0/203632016 wait complete. 2024-05-22T21:14:26.325 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:14:26.325 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:14:26.326 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-22T21:14:26.326 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-22T21:14:26.326 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-22T21:14:26.326 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:14:26.326 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:14:26.326 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:14:26.326 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:14:26.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:26.327 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-22T21:14:26.336 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:14:26.337 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:14:26.337 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:14:26.337 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:26.337 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-22T21:14:26.337 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:26.337 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:14:26.337 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:26.337 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f0 2024-05-22T21:14:26.337 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:14:26.338 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:26.338 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f1 2024-05-22T21:14:26.338 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:14:26.338 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:26.338 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:14:26.338 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:26.338 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:14:26.338 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42418sec preferred_lft 42418sec 2024-05-22T21:14:26.338 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-22T21:14:26.338 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:26.339 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:14:26.339 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:bd:4e:25:3d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:26.339 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:14:26.339 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:26.339 INFO:teuthology.orchestra.run.smithi179.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:14:26.339 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:26.339 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:14:26.339 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:26.339 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:14:26.339 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:26.339 INFO:teuthology.orchestra.run.smithi179.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:14:26.339 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-22T21:14:26.339 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:14:26.340 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:26.340 INFO:teuthology.orchestra.run.smithi179.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:14:26.340 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-22T21:14:26.340 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:14:26.340 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:26.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:26.341 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:14:26.402 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-22T21:14:26.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:26.404 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-22T21:14:26.459 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-22T21:14:26.459 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:14:26.459 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:14:26.459 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:14:26.460 INFO:teuthology.orchestra.run.smithi179.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:14:26.460 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:14:26.460 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:14:26.460 INFO:teuthology.orchestra.run.smithi179.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:14:26.460 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:14:26.460 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:14:26.460 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:14:26.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:26.460 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:14:26.460 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:14:26.460 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:14:26.461 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:14:26.461 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:14:26.461 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:14:26.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:26.555 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-22T21:14:26.563 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-22T21:14:26.563 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-22T21:14:26.564 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:14:26.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:26.564 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-22T21:14:26.627 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:14:26.627 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-22T21:14:26.678 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:14:26.679 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:14:26.679 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:14:26.730 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:14:26.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:26.730 DEBUG:teuthology.orchestra.run.smithi179:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:14:26.818 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:14:26.818 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:14:26.819 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:14:26.833 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:14:26.834 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:14:26.834 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:14:26.834 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:14:26.834 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:14:26.841 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:14:26.842 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:14:26.842 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:14:26.842 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:14:26.842 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:14:26.842 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:14:26.842 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:14:26.842 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:14:26.845 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:14:26.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:26.846 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:14:26.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:26.865 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:14:26.917 INFO:teuthology.orchestra.run.smithi179.stdout:client4883 2024-05-22T21:14:26.918 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:14:26.918 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:26.970 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:14:27.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.108+0000 7f922be00640 1 Processor -- start 2024-05-22T21:14:27.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.108+0000 7f922be00640 1 -- start start 2024-05-22T21:14:27.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.112+0000 7f922be00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f922c143760 0x7f922c070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:27.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.112+0000 7f922be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f922c06bfc0 0x7f922c071180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:27.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.112+0000 7f922be00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922c143b40 con 0x7f922c15e640 2024-05-22T21:14:27.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.112+0000 7f922be00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922c06c7c0 con 0x7f922c143760 2024-05-22T21:14:27.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.112+0000 7f922be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922c0b3380 con 0x7f922c06bfc0 2024-05-22T21:14:27.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.112+0000 7f922aa00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f922c143760 0x7f922c070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:27.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.112+0000 7f922a000640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f922c15e640 0x7f922c06b8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:41950/0 (socket says 172.21.15.161:41950) 2024-05-22T21:14:27.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.112+0000 7f922a000640 1 -- 172.21.15.161:0/3641916475 learned_addr learned my addr 172.21.15.161:0/3641916475 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:27.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.112+0000 7f922b400640 1 --2- 172.21.15.161:0/3641916475 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f922c06bfc0 0x7f922c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:27.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.112+0000 7f922aa00640 1 -- 172.21.15.161:0/3641916475 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f922c06bfc0 msgr2=0x7f922c071180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:27.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.112+0000 7f9229600640 1 -- 172.21.15.161:0/3641916475 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131524872 0 0) 0x7f922c143b40 con 0x7f922c15e640 2024-05-22T21:14:27.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.112+0000 7f922aa00640 1 --2- 172.21.15.161:0/3641916475 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f922c06bfc0 0x7f922c071180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.112+0000 7f922aa00640 1 -- 172.21.15.161:0/3641916475 >> v1:172.21.15.179:6789/0 conn(0x7f922c15e640 legacy=0x7f922c06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:27.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.112+0000 7f922aa00640 1 -- 172.21.15.161:0/3641916475 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c06c4e0 con 0x7f922c143760 2024-05-22T21:14:27.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.112+0000 7f922b400640 1 --2- 172.21.15.161:0/3641916475 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f922c06bfc0 0x7f922c071180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:27.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.116+0000 7f922aa00640 1 --2- 172.21.15.161:0/3641916475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f922c143760 0x7f922c070c40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f921c0097c0 tx=0x7f921c01c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89d6f068d52b5de server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:27.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.116+0000 7f9229600640 1 -- 172.21.15.161:0/3641916475 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c0ac410 con 0x7f922c143760 2024-05-22T21:14:27.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.116+0000 7f9229600640 1 -- 172.21.15.161:0/3641916475 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c0ac570 con 0x7f922c143760 2024-05-22T21:14:27.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.116+0000 7f9229600640 1 -- 172.21.15.161:0/3641916475 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c0ac900 con 0x7f922c143760 2024-05-22T21:14:27.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.116+0000 7f922be00640 1 -- 172.21.15.161:0/3641916475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f922c143760 msgr2=0x7f922c070c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:27.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.116+0000 7f922be00640 1 --2- 172.21.15.161:0/3641916475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f922c143760 0x7f922c070c40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f921c0097c0 tx=0x7f921c01c3b0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.116+0000 7f922be00640 1 -- 172.21.15.161:0/3641916475 shutdown_connections 2024-05-22T21:14:27.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.116+0000 7f922be00640 1 --2- 172.21.15.161:0/3641916475 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f922c06bfc0 0x7f922c071180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.116+0000 7f922be00640 1 --2- 172.21.15.161:0/3641916475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f922c143760 0x7f922c070c40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.116+0000 7f922be00640 1 -- 172.21.15.161:0/3641916475 >> 172.21.15.161:0/3641916475 conn(0x7f922c06a4d0 msgr2=0x7f922c0b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:27.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.116+0000 7f922be00640 1 -- 172.21.15.161:0/3641916475 shutdown_connections 2024-05-22T21:14:27.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.116+0000 7f922be00640 1 -- 172.21.15.161:0/3641916475 wait complete. 2024-05-22T21:14:27.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.116+0000 7f922be00640 1 Processor -- start 2024-05-22T21:14:27.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.116+0000 7f922be00640 1 -- start start 2024-05-22T21:14:27.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f922c06bfc0 0x7f922c1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:27.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f922c143760 0x7f922c0b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:27.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f922c15e640 0x7f922c0b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:27.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922be00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c15c2b0 con 0x7f922c06bfc0 2024-05-22T21:14:27.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922be00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c15b6a0 con 0x7f922c143760 2024-05-22T21:14:27.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922c15b980 con 0x7f922c15e640 2024-05-22T21:14:27.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f922c06bfc0 0x7f922c1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:27.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f922c06bfc0 0x7f922c1597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55620/0 (socket says 172.21.15.161:55620) 2024-05-22T21:14:27.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922aa00640 1 -- 172.21.15.161:0/2647758667 learned_addr learned my addr 172.21.15.161:0/2647758667 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:27.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922a000640 1 --2- 172.21.15.161:0/2647758667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f922c143760 0x7f922c0b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:27.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922b400640 1 --2- 172.21.15.161:0/2647758667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f922c15e640 0x7f922c0b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:27.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922aa00640 1 -- 172.21.15.161:0/2647758667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f922c15e640 msgr2=0x7f922c0b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:27.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922aa00640 1 --2- 172.21.15.161:0/2647758667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f922c15e640 0x7f922c0b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922aa00640 1 -- 172.21.15.161:0/2647758667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f922c143760 msgr2=0x7f922c0b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:27.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922aa00640 1 --2- 172.21.15.161:0/2647758667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f922c143760 0x7f922c0b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922aa00640 1 -- 172.21.15.161:0/2647758667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9224009a70 con 0x7f922c06bfc0 2024-05-22T21:14:27.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922aa00640 1 --2- 172.21.15.161:0/2647758667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f922c06bfc0 0x7f922c1597f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f921c019fe0 tx=0x7f921c01abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:27.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f9222a00640 1 -- 172.21.15.161:0/2647758667 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c0ac410 con 0x7f922c06bfc0 2024-05-22T21:14:27.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f9222a00640 1 -- 172.21.15.161:0/2647758667 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c0ace90 con 0x7f922c06bfc0 2024-05-22T21:14:27.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f9222a00640 1 -- 172.21.15.161:0/2647758667 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c004230 con 0x7f922c06bfc0 2024-05-22T21:14:27.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922be00640 1 -- 172.21.15.161:0/2647758667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921c009420 con 0x7f922c06bfc0 2024-05-22T21:14:27.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922be00640 1 -- 172.21.15.161:0/2647758667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922c15bf40 con 0x7f922c06bfc0 2024-05-22T21:14:27.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.120+0000 7f922be00640 1 -- 172.21.15.161:0/2647758667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922c067070 con 0x7f922c06bfc0 2024-05-22T21:14:27.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.124+0000 7f9222a00640 1 -- 172.21.15.161:0/2647758667 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f921c004390 con 0x7f922c06bfc0 2024-05-22T21:14:27.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.124+0000 7f9222a00640 1 --2- 172.21.15.161:0/2647758667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f920c066690 0x7f920c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:27.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.124+0000 7f9222a00640 1 -- 172.21.15.161:0/2647758667 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f921c11e460 con 0x7f922c06bfc0 2024-05-22T21:14:27.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.124+0000 7f922a000640 1 --2- 172.21.15.161:0/2647758667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f920c066690 0x7f920c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:27.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.124+0000 7f922a000640 1 --2- 172.21.15.161:0/2647758667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f920c066690 0x7f920c068b50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f922c0b15e0 tx=0x7f9218007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:27.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.128+0000 7f9222a00640 1 -- 172.21.15.161:0/2647758667 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f921c0eb990 con 0x7f922c06bfc0 2024-05-22T21:14:27.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.312+0000 7f922be00640 1 -- 172.21.15.161:0/2647758667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f922c0cd200 con 0x7f922c06bfc0 2024-05-22T21:14:27.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.312+0000 7f9222a00640 1 -- 172.21.15.161:0/2647758667 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f921c006e20 con 0x7f922c06bfc0 2024-05-22T21:14:27.315 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:27.315 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3742018925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":3742018925},{"type":"v1","addr":"172.21.15.161:6837","nonce":3742018925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2553366378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2553366378},{"type":"v1","addr":"172.21.15.179:6834","nonce":2553366378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/509468338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":509468338},{"type":"v1","addr":"172.21.15.179:6835","nonce":509468338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:19.053259+0000","modified":"2024-05-22T21:14:22.076119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6836/1635458417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1635458417},{"type":"v1","addr":"172.21.15.161:6836","nonce":1635458417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-22T21:14:27.315 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 16 2024-05-22T21:14:27.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.316+0000 7f922be00640 1 -- 172.21.15.161:0/2647758667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f920c066690 msgr2=0x7f920c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:27.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.316+0000 7f922be00640 1 --2- 172.21.15.161:0/2647758667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f920c066690 0x7f920c068b50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f922c0b15e0 tx=0x7f9218007df0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.316+0000 7f922be00640 1 -- 172.21.15.161:0/2647758667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f922c06bfc0 msgr2=0x7f922c1597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:27.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.316+0000 7f922be00640 1 --2- 172.21.15.161:0/2647758667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f922c06bfc0 0x7f922c1597f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f921c019fe0 tx=0x7f921c01abf0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.316+0000 7f922be00640 1 -- 172.21.15.161:0/2647758667 shutdown_connections 2024-05-22T21:14:27.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.316+0000 7f922be00640 1 --2- 172.21.15.161:0/2647758667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f920c066690 0x7f920c068b50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.316+0000 7f922be00640 1 --2- 172.21.15.161:0/2647758667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f922c15e640 0x7f922c0b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.316+0000 7f922be00640 1 --2- 172.21.15.161:0/2647758667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f922c143760 0x7f922c0b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.316+0000 7f922be00640 1 --2- 172.21.15.161:0/2647758667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f922c06bfc0 0x7f922c1597f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.316+0000 7f922be00640 1 -- 172.21.15.161:0/2647758667 >> 172.21.15.161:0/2647758667 conn(0x7f922c06a4d0 msgr2=0x7f922c0cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:27.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.316+0000 7f922be00640 1 -- 172.21.15.161:0/2647758667 shutdown_connections 2024-05-22T21:14:27.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.316+0000 7f922be00640 1 -- 172.21.15.161:0/2647758667 wait complete. 2024-05-22T21:14:27.337 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-05-22T21:14:27.492 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":5055,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.4217318090000006,"mdsmap_epoch":16,"osdmap_epoch":33,"osdmap_epoch_barrier":33,"uptime":15.659523516} 2024-05-22T21:14:27.493 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 5055, "mdsmap_epoch": 16, "osdmap_epoch": 33, "osdmap_epoch_barrier": 33, "rank_uptime": 8.421731809, "state": "up:active", "uptime": 15.659523516, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:14:27.493 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2024-05-22T21:14:27.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.624+0000 7f284a400640 1 Processor -- start 2024-05-22T21:14:27.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.624+0000 7f284a400640 1 -- start start 2024-05-22T21:14:27.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.628+0000 7f284a400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f284415e130 0x7f284415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:27.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.628+0000 7f284a400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2844071670 con 0x7f284415dd50 2024-05-22T21:14:27.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.628+0000 7f284a400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284415d870 con 0x7f284415d410 2024-05-22T21:14:27.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.628+0000 7f284a400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2844070a60 con 0x7f284415e130 2024-05-22T21:14:27.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.628+0000 7f2842a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f284415e130 0x7f284415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:27.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.628+0000 7f2843400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f284415dd50 0x7f284406d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47580/0 (socket says 172.21.15.161:47580) 2024-05-22T21:14:27.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.628+0000 7f2843400640 1 -- 172.21.15.161:0/779298339 learned_addr learned my addr 172.21.15.161:0/779298339 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:27.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.628+0000 7f2842a00640 1 -- 172.21.15.161:0/779298339 >> v1:172.21.15.179:6789/0 conn(0x7f284415d410 legacy=0x7f284415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:27.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.628+0000 7f2842a00640 1 -- 172.21.15.161:0/779298339 >> v1:172.21.15.161:6789/0 conn(0x7f284415dd50 legacy=0x7f284406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:27.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.628+0000 7f2842a00640 1 -- 172.21.15.161:0/779298339 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2844070d40 con 0x7f284415e130 2024-05-22T21:14:27.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.628+0000 7f2842000640 1 -- 172.21.15.161:0/779298339 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942732077 0 0) 0x7f2844071670 con 0x7f284415dd50 2024-05-22T21:14:27.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.628+0000 7f2842a00640 1 --2- 172.21.15.161:0/779298339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f284415e130 0x7f284415ced0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f28300097c0 tx=0x7f283001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c6c57258801891 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:27.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.628+0000 7f2842000640 1 -- 172.21.15.161:0/779298339 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28300a7070 con 0x7f284415e130 2024-05-22T21:14:27.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.628+0000 7f2842000640 1 -- 172.21.15.161:0/779298339 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28300ab430 con 0x7f284415e130 2024-05-22T21:14:27.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.628+0000 7f2842000640 1 -- 172.21.15.161:0/779298339 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28300ab7c0 con 0x7f284415e130 2024-05-22T21:14:27.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f284a400640 1 -- 172.21.15.161:0/779298339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f284415e130 msgr2=0x7f284415ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:27.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f284a400640 1 --2- 172.21.15.161:0/779298339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f284415e130 0x7f284415ced0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f28300097c0 tx=0x7f283001c430 comp rx=0 tx=0).stop 2024-05-22T21:14:27.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f284a400640 1 -- 172.21.15.161:0/779298339 shutdown_connections 2024-05-22T21:14:27.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f284a400640 1 --2- 172.21.15.161:0/779298339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f284415e130 0x7f284415ced0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f284a400640 1 -- 172.21.15.161:0/779298339 >> 172.21.15.161:0/779298339 conn(0x7f28440b0bc0 msgr2=0x7f28440b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:27.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f284a400640 1 -- 172.21.15.161:0/779298339 shutdown_connections 2024-05-22T21:14:27.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f284a400640 1 -- 172.21.15.161:0/779298339 wait complete. 2024-05-22T21:14:27.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f284a400640 1 Processor -- start 2024-05-22T21:14:27.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f284a400640 1 -- start start 2024-05-22T21:14:27.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f284a400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f284415d410 0x7f284415eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:27.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f284a400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f284415dd50 0x7f28441f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:27.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f284a400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f284415e130 0x7f284415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:27.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f284a400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28440b3040 con 0x7f284415dd50 2024-05-22T21:14:27.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f284a400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28440b3320 con 0x7f284415e130 2024-05-22T21:14:27.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f284a400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28440b3600 con 0x7f284415d410 2024-05-22T21:14:27.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f2842a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f284415dd50 0x7f28441f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:27.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f2842a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f284415dd50 0x7f28441f5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55642/0 (socket says 172.21.15.161:55642) 2024-05-22T21:14:27.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f2842a00640 1 -- 172.21.15.161:0/3657766046 learned_addr learned my addr 172.21.15.161:0/3657766046 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:27.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f2843400640 1 --2- 172.21.15.161:0/3657766046 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f284415d410 0x7f284415eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:27.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.632+0000 7f2843e00640 1 --2- 172.21.15.161:0/3657766046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f284415e130 0x7f284415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:27.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.636+0000 7f2842a00640 1 -- 172.21.15.161:0/3657766046 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f284415d410 msgr2=0x7f284415eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:27.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.636+0000 7f2842a00640 1 --2- 172.21.15.161:0/3657766046 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f284415d410 0x7f284415eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.636+0000 7f2842a00640 1 -- 172.21.15.161:0/3657766046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f284415e130 msgr2=0x7f284415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:27.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.636+0000 7f2842a00640 1 --2- 172.21.15.161:0/3657766046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f284415e130 0x7f284415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.636+0000 7f2842a00640 1 -- 172.21.15.161:0/3657766046 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2830009420 con 0x7f284415dd50 2024-05-22T21:14:27.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.636+0000 7f2842a00640 1 --2- 172.21.15.161:0/3657766046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f284415dd50 0x7f28441f5150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f283001a060 tx=0x7f28300191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:27.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.636+0000 7f283b400640 1 -- 172.21.15.161:0/3657766046 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28300bf450 con 0x7f284415dd50 2024-05-22T21:14:27.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.636+0000 7f283b400640 1 -- 172.21.15.161:0/3657766046 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28300bf5d0 con 0x7f284415dd50 2024-05-22T21:14:27.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.636+0000 7f283b400640 1 -- 172.21.15.161:0/3657766046 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28300bf980 con 0x7f284415dd50 2024-05-22T21:14:27.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.636+0000 7f284a400640 1 -- 172.21.15.161:0/3657766046 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28440b38e0 con 0x7f284415dd50 2024-05-22T21:14:27.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.636+0000 7f284a400640 1 -- 172.21.15.161:0/3657766046 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28441f5690 con 0x7f284415dd50 2024-05-22T21:14:27.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.636+0000 7f284a400640 1 -- 172.21.15.161:0/3657766046 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28440b3740 con 0x7f284415dd50 2024-05-22T21:14:27.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.636+0000 7f283b400640 1 -- 172.21.15.161:0/3657766046 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f28300b43f0 con 0x7f284415dd50 2024-05-22T21:14:27.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.636+0000 7f283b400640 1 --2- 172.21.15.161:0/3657766046 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2824066690 0x7f2824068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:27.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.636+0000 7f283b400640 1 -- 172.21.15.161:0/3657766046 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f283011ec10 con 0x7f284415dd50 2024-05-22T21:14:27.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.636+0000 7f2843400640 1 --2- 172.21.15.161:0/3657766046 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2824066690 0x7f2824068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:27.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.640+0000 7f2843400640 1 --2- 172.21.15.161:0/3657766046 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2824066690 0x7f2824068b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f28340012f0 tx=0x7f2834001690 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:27.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.640+0000 7f283b400640 1 -- 172.21.15.161:0/3657766046 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f28440b3740 con 0x7f284415dd50 2024-05-22T21:14:27.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.852+0000 7f284a400640 1 -- 172.21.15.161:0/3657766046 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f28440b3740 con 0x7f284415dd50 2024-05-22T21:14:27.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.852+0000 7f283b400640 1 -- 172.21.15.161:0/3657766046 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1618 (secure 0 0 0) 0x7f28440b3740 con 0x7f284415dd50 2024-05-22T21:14:27.853 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:27.854 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:14:27.854 INFO:teuthology.orchestra.run.smithi161.stdout: "stats": { 2024-05-22T21:14:27.854 INFO:teuthology.orchestra.run.smithi161.stdout: "total_bytes": 1600000000, 2024-05-22T21:14:27.854 INFO:teuthology.orchestra.run.smithi161.stdout: "total_avail_bytes": 1597661175, 2024-05-22T21:14:27.854 INFO:teuthology.orchestra.run.smithi161.stdout: "total_used_bytes": 2338825, 2024-05-22T21:14:27.854 INFO:teuthology.orchestra.run.smithi161.stdout: "total_used_raw_bytes": 2338825, 2024-05-22T21:14:27.854 INFO:teuthology.orchestra.run.smithi161.stdout: "total_used_raw_ratio": 0.001461765612475574, 2024-05-22T21:14:27.854 INFO:teuthology.orchestra.run.smithi161.stdout: "num_osds": 8, 2024-05-22T21:14:27.854 INFO:teuthology.orchestra.run.smithi161.stdout: "num_per_pool_osds": 0, 2024-05-22T21:14:27.854 INFO:teuthology.orchestra.run.smithi161.stdout: "num_per_pool_omap_osds": 0 2024-05-22T21:14:27.854 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:27.855 INFO:teuthology.orchestra.run.smithi161.stdout: "stats_by_class": { 2024-05-22T21:14:27.855 INFO:teuthology.orchestra.run.smithi161.stdout: "hdd": { 2024-05-22T21:14:27.855 INFO:teuthology.orchestra.run.smithi161.stdout: "total_bytes": 1600000000, 2024-05-22T21:14:27.855 INFO:teuthology.orchestra.run.smithi161.stdout: "total_avail_bytes": 1597661175, 2024-05-22T21:14:27.855 INFO:teuthology.orchestra.run.smithi161.stdout: "total_used_bytes": 2338825, 2024-05-22T21:14:27.855 INFO:teuthology.orchestra.run.smithi161.stdout: "total_used_raw_bytes": 2338825, 2024-05-22T21:14:27.855 INFO:teuthology.orchestra.run.smithi161.stdout: "total_used_raw_ratio": 0.001461765612475574 2024-05-22T21:14:27.855 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:27.855 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:27.856 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:14:27.856 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:27.856 INFO:teuthology.orchestra.run.smithi161.stdout: "name": ".mgr", 2024-05-22T21:14:27.856 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 1, 2024-05-22T21:14:27.856 INFO:teuthology.orchestra.run.smithi161.stdout: "stats": { 2024-05-22T21:14:27.856 INFO:teuthology.orchestra.run.smithi161.stdout: "stored": 590368, 2024-05-22T21:14:27.856 INFO:teuthology.orchestra.run.smithi161.stdout: "objects": 2, 2024-05-22T21:14:27.856 INFO:teuthology.orchestra.run.smithi161.stdout: "kb_used": 577, 2024-05-22T21:14:27.856 INFO:teuthology.orchestra.run.smithi161.stdout: "bytes_used": 590368, 2024-05-22T21:14:27.857 INFO:teuthology.orchestra.run.smithi161.stdout: "percent_used": 0.00052961398614570498, 2024-05-22T21:14:27.857 INFO:teuthology.orchestra.run.smithi161.stdout: "max_avail": 557061696 2024-05-22T21:14:27.857 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:27.857 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:27.857 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:27.857 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "cephfs_metadata", 2024-05-22T21:14:27.857 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 4, 2024-05-22T21:14:27.857 INFO:teuthology.orchestra.run.smithi161.stdout: "stats": { 2024-05-22T21:14:27.857 INFO:teuthology.orchestra.run.smithi161.stdout: "stored": 2620, 2024-05-22T21:14:27.857 INFO:teuthology.orchestra.run.smithi161.stdout: "objects": 22, 2024-05-22T21:14:27.857 INFO:teuthology.orchestra.run.smithi161.stdout: "kb_used": 3, 2024-05-22T21:14:27.857 INFO:teuthology.orchestra.run.smithi161.stdout: "bytes_used": 2620, 2024-05-22T21:14:27.857 INFO:teuthology.orchestra.run.smithi161.stdout: "percent_used": 2.3516192868555663e-06, 2024-05-22T21:14:27.857 INFO:teuthology.orchestra.run.smithi161.stdout: "max_avail": 557061696 2024-05-22T21:14:27.858 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:27.858 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:27.858 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:27.858 INFO:teuthology.orchestra.run.smithi161.stdout: "name": "cephfs_data", 2024-05-22T21:14:27.858 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 5, 2024-05-22T21:14:27.858 INFO:teuthology.orchestra.run.smithi161.stdout: "stats": { 2024-05-22T21:14:27.858 INFO:teuthology.orchestra.run.smithi161.stdout: "stored": 0, 2024-05-22T21:14:27.858 INFO:teuthology.orchestra.run.smithi161.stdout: "objects": 0, 2024-05-22T21:14:27.858 INFO:teuthology.orchestra.run.smithi161.stdout: "kb_used": 0, 2024-05-22T21:14:27.858 INFO:teuthology.orchestra.run.smithi161.stdout: "bytes_used": 0, 2024-05-22T21:14:27.858 INFO:teuthology.orchestra.run.smithi161.stdout: "percent_used": 0, 2024-05-22T21:14:27.858 INFO:teuthology.orchestra.run.smithi161.stdout: "max_avail": 557061696 2024-05-22T21:14:27.858 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:27.859 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:27.859 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:14:27.859 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:14:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.852+0000 7f284a400640 1 -- 172.21.15.161:0/3657766046 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2824066690 msgr2=0x7f2824068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.852+0000 7f284a400640 1 --2- 172.21.15.161:0/3657766046 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2824066690 0x7f2824068b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f28340012f0 tx=0x7f2834001690 comp rx=0 tx=0).stop 2024-05-22T21:14:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.852+0000 7f284a400640 1 -- 172.21.15.161:0/3657766046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f284415dd50 msgr2=0x7f28441f5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.852+0000 7f284a400640 1 --2- 172.21.15.161:0/3657766046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f284415dd50 0x7f28441f5150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f283001a060 tx=0x7f28300191d0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.852+0000 7f284a400640 1 -- 172.21.15.161:0/3657766046 shutdown_connections 2024-05-22T21:14:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.852+0000 7f284a400640 1 --2- 172.21.15.161:0/3657766046 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2824066690 0x7f2824068b50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.852+0000 7f284a400640 1 --2- 172.21.15.161:0/3657766046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f284415e130 0x7f284415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.852+0000 7f284a400640 1 --2- 172.21.15.161:0/3657766046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f284415dd50 0x7f28441f5150 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.852+0000 7f284a400640 1 --2- 172.21.15.161:0/3657766046 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f284415d410 0x7f284415eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.852+0000 7f284a400640 1 -- 172.21.15.161:0/3657766046 >> 172.21.15.161:0/3657766046 conn(0x7f28440b0bc0 msgr2=0x7f28440ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.852+0000 7f284a400640 1 -- 172.21.15.161:0/3657766046 shutdown_connections 2024-05-22T21:14:27.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:27.852+0000 7f284a400640 1 -- 172.21.15.161:0/3657766046 wait complete. 2024-05-22T21:14:27.874 DEBUG:teuthology.orchestra.run.smithi161:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-22T21:14:27.874 DEBUG:teuthology.orchestra.run.smithi161:> import os 2024-05-22T21:14:27.874 DEBUG:teuthology.orchestra.run.smithi161:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-05-22T21:14:27.875 DEBUG:teuthology.orchestra.run.smithi161:> os.fchmod(fd, 0O666) 2024-05-22T21:14:27.875 DEBUG:teuthology.orchestra.run.smithi161:> os.fsync(fd) 2024-05-22T21:14:27.875 DEBUG:teuthology.orchestra.run.smithi161:> os.close(fd) 2024-05-22T21:14:27.875 DEBUG:teuthology.orchestra.run.smithi161:> ' 2024-05-22T21:14:28.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:28.126 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:14:28.179 INFO:teuthology.orchestra.run.smithi161.stdout:client4874 2024-05-22T21:14:28.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:28.180 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d6fa8528-b1bf-41b6-9f76-c0d715394087.client4874/osdmap') 2024-05-22T21:14:28.242 INFO:teuthology.orchestra.run.smithi161.stdout:epoch 33 barrier 33 flags 0x588000 2024-05-22T21:14:28.243 INFO:teuthology.orchestra.run.smithi161.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:14:28.243 INFO:teuthology.orchestra.run.smithi161.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:14:28.243 INFO:teuthology.orchestra.run.smithi161.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:14:28.243 INFO:teuthology.orchestra.run.smithi161.stdout:osd0 (1)172.21.15.161:6809 100% (exists, up) 100% -1 2024-05-22T21:14:28.243 INFO:teuthology.orchestra.run.smithi161.stdout:osd1 (1)172.21.15.161:6801 100% (exists, up) 100% -1 2024-05-22T21:14:28.243 INFO:teuthology.orchestra.run.smithi161.stdout:osd2 (1)172.21.15.161:6825 100% (exists, up) 100% -1 2024-05-22T21:14:28.243 INFO:teuthology.orchestra.run.smithi161.stdout:osd3 (1)172.21.15.161:6817 100% (exists, up) 100% -1 2024-05-22T21:14:28.243 INFO:teuthology.orchestra.run.smithi161.stdout:osd4 (1)172.21.15.179:6801 100% (exists, up) 100% -1 2024-05-22T21:14:28.243 INFO:teuthology.orchestra.run.smithi161.stdout:osd5 (1)172.21.15.179:6809 100% (exists, up) 100% -1 2024-05-22T21:14:28.243 INFO:teuthology.orchestra.run.smithi161.stdout:osd6 (1)172.21.15.179:6822 100% (exists, up) 100% -1 2024-05-22T21:14:28.243 INFO:teuthology.orchestra.run.smithi161.stdout:osd7 (1)172.21.15.179:6817 100% (exists, up) 100% -1 2024-05-22T21:14:28.243 INFO:teuthology.orchestra.run.smithi161.stderr:1+1 records in 2024-05-22T21:14:28.244 INFO:teuthology.orchestra.run.smithi161.stderr:1+1 records out 2024-05-22T21:14:28.244 INFO:teuthology.orchestra.run.smithi161.stderr:799 bytes copied, 0.000101555 s, 7.9 MB/s 2024-05-22T21:14:28.245 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2024-05-22T21:14:28.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f331e400640 1 Processor -- start 2024-05-22T21:14:28.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f331e400640 1 -- start start 2024-05-22T21:14:28.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f331e400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f331813cfc0 0x7f33181402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:28.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f331e400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f331815af60 0x7f331813f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:28.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f331e400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33181400d0 con 0x7f331813fc80 2024-05-22T21:14:28.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f331e400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331813d3a0 con 0x7f331815af60 2024-05-22T21:14:28.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f331e400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33180cd920 con 0x7f331813cfc0 2024-05-22T21:14:28.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f3316a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f331815af60 0x7f331813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:28.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f3316a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f331815af60 0x7f331813f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55656/0 (socket says 172.21.15.161:55656) 2024-05-22T21:14:28.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f3316a00640 1 -- 172.21.15.161:0/3106009150 learned_addr learned my addr 172.21.15.161:0/3106009150 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:28.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f3317e00640 1 --1- 172.21.15.161:0/3106009150 >> v1:172.21.15.161:6789/0 conn(0x7f331813fc80 0x7f331815dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47584/0 (socket says 172.21.15.161:47584) 2024-05-22T21:14:28.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f3317400640 1 --2- 172.21.15.161:0/3106009150 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f331813cfc0 0x7f33181402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:28.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f3316a00640 1 -- 172.21.15.161:0/3106009150 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f331813cfc0 msgr2=0x7f33181402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:28.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f3316a00640 1 --2- 172.21.15.161:0/3106009150 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f331813cfc0 0x7f33181402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:28.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f3316a00640 1 -- 172.21.15.161:0/3106009150 >> v1:172.21.15.161:6789/0 conn(0x7f331813fc80 legacy=0x7f331815dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:28.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f3316a00640 1 -- 172.21.15.161:0/3106009150 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33180cdc00 con 0x7f331815af60 2024-05-22T21:14:28.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.428+0000 7f3316000640 1 -- 172.21.15.161:0/3106009150 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4007389483 0 0) 0x7f33100014a0 con 0x7f331813fc80 2024-05-22T21:14:28.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.432+0000 7f3316a00640 1 --2- 172.21.15.161:0/3106009150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331815af60 0x7f331813f740 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3304009870 tx=0x7f330401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f101a74e031b0fe5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:28.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.432+0000 7f3316000640 1 -- 172.21.15.161:0/3106009150 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33040ab600 con 0x7f331815af60 2024-05-22T21:14:28.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.432+0000 7f3316000640 1 -- 172.21.15.161:0/3106009150 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33040a7050 con 0x7f331815af60 2024-05-22T21:14:28.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.432+0000 7f3316000640 1 -- 172.21.15.161:0/3106009150 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33040ab980 con 0x7f331815af60 2024-05-22T21:14:28.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.432+0000 7f331e400640 1 -- 172.21.15.161:0/3106009150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331815af60 msgr2=0x7f331813f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:28.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.432+0000 7f331e400640 1 --2- 172.21.15.161:0/3106009150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331815af60 0x7f331813f740 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3304009870 tx=0x7f330401c4d0 comp rx=0 tx=0).stop 2024-05-22T21:14:28.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.432+0000 7f331e400640 1 -- 172.21.15.161:0/3106009150 shutdown_connections 2024-05-22T21:14:28.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.432+0000 7f331e400640 1 --2- 172.21.15.161:0/3106009150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331815af60 0x7f331813f740 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:28.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.432+0000 7f331e400640 1 --2- 172.21.15.161:0/3106009150 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f331813cfc0 0x7f33181402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:28.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.432+0000 7f331e400640 1 -- 172.21.15.161:0/3106009150 >> 172.21.15.161:0/3106009150 conn(0x7f331806a2d0 msgr2=0x7f331806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:28.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.432+0000 7f331e400640 1 -- 172.21.15.161:0/3106009150 shutdown_connections 2024-05-22T21:14:28.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.432+0000 7f331e400640 1 -- 172.21.15.161:0/3106009150 wait complete. 2024-05-22T21:14:28.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.432+0000 7f331e400640 1 Processor -- start 2024-05-22T21:14:28.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.432+0000 7f331e400640 1 -- start start 2024-05-22T21:14:28.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f331e400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f331813cfc0 0x7f33180cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:28.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f331e400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f331813fc80 0x7f3318140c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:28.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f331e400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331815af60 0x7f331813ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:28.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f331e400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331815d460 con 0x7f331815af60 2024-05-22T21:14:28.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f331e400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331813d9f0 con 0x7f331813fc80 2024-05-22T21:14:28.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f331e400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3318141360 con 0x7f331813cfc0 2024-05-22T21:14:28.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f3317e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331815af60 0x7f331813ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:28.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f3317e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331815af60 0x7f331813ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55666/0 (socket says 172.21.15.161:55666) 2024-05-22T21:14:28.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f3317e00640 1 -- 172.21.15.161:0/557605566 learned_addr learned my addr 172.21.15.161:0/557605566 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:28.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f3317400640 1 --2- 172.21.15.161:0/557605566 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f331813cfc0 0x7f33180cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:28.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f3316a00640 1 --2- 172.21.15.161:0/557605566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f331813fc80 0x7f3318140c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:28.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f3317e00640 1 -- 172.21.15.161:0/557605566 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f331813cfc0 msgr2=0x7f33180cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:28.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f3317e00640 1 --2- 172.21.15.161:0/557605566 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f331813cfc0 0x7f33180cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:28.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f3317e00640 1 -- 172.21.15.161:0/557605566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f331813fc80 msgr2=0x7f3318140c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:28.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f3317e00640 1 --2- 172.21.15.161:0/557605566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f331813fc80 0x7f3318140c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:28.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f3317e00640 1 -- 172.21.15.161:0/557605566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33040094d0 con 0x7f331815af60 2024-05-22T21:14:28.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f3317e00640 1 --2- 172.21.15.161:0/557605566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331815af60 0x7f331813ea50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f331000c7e0 tx=0x7f331000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:28.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f330f400640 1 -- 172.21.15.161:0/557605566 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331000a640 con 0x7f331815af60 2024-05-22T21:14:28.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f330f400640 1 -- 172.21.15.161:0/557605566 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331000a7c0 con 0x7f331815af60 2024-05-22T21:14:28.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f330f400640 1 -- 172.21.15.161:0/557605566 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331000ab70 con 0x7f331815af60 2024-05-22T21:14:28.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f331e400640 1 -- 172.21.15.161:0/557605566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331813dc90 con 0x7f331815af60 2024-05-22T21:14:28.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f331e400640 1 -- 172.21.15.161:0/557605566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331813e250 con 0x7f331815af60 2024-05-22T21:14:28.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.436+0000 7f331e400640 1 -- 172.21.15.161:0/557605566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3318067220 con 0x7f331815af60 2024-05-22T21:14:28.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.440+0000 7f330f400640 1 -- 172.21.15.161:0/557605566 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f331000acd0 con 0x7f331815af60 2024-05-22T21:14:28.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.440+0000 7f330f400640 1 --2- 172.21.15.161:0/557605566 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f32f8066690 0x7f32f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:28.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.440+0000 7f330f400640 1 -- 172.21.15.161:0/557605566 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33100834d0 con 0x7f331815af60 2024-05-22T21:14:28.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.440+0000 7f3317400640 1 --2- 172.21.15.161:0/557605566 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f32f8066690 0x7f32f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:28.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.440+0000 7f3317400640 1 --2- 172.21.15.161:0/557605566 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f32f8066690 0x7f32f8068b50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3308009ac0 tx=0x7f3308009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:28.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.444+0000 7f330f400640 1 -- 172.21.15.161:0/557605566 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3310050960 con 0x7f331815af60 2024-05-22T21:14:28.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:28.608+0000 7f331e400640 1 -- 172.21.15.161:0/557605566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7f33181400d0 con 0x7f331815af60 2024-05-22T21:14:29.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.108+0000 7f330f400640 1 -- 172.21.15.161:0/557605566 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v34) v1 ==== 93+0+0 (secure 0 0 0) 0x7f33181400d0 con 0x7f331815af60 2024-05-22T21:14:29.109 INFO:teuthology.orchestra.run.smithi161.stderr:pauserd,pausewr is set 2024-05-22T21:14:29.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.108+0000 7f331e400640 1 -- 172.21.15.161:0/557605566 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f32f8066690 msgr2=0x7f32f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:29.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.108+0000 7f331e400640 1 --2- 172.21.15.161:0/557605566 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f32f8066690 0x7f32f8068b50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3308009ac0 tx=0x7f3308009190 comp rx=0 tx=0).stop 2024-05-22T21:14:29.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.108+0000 7f331e400640 1 -- 172.21.15.161:0/557605566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331815af60 msgr2=0x7f331813ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:29.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.108+0000 7f331e400640 1 --2- 172.21.15.161:0/557605566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331815af60 0x7f331813ea50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f331000c7e0 tx=0x7f331000cca0 comp rx=0 tx=0).stop 2024-05-22T21:14:29.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.108+0000 7f331e400640 1 -- 172.21.15.161:0/557605566 shutdown_connections 2024-05-22T21:14:29.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.108+0000 7f331e400640 1 --2- 172.21.15.161:0/557605566 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f32f8066690 0x7f32f8068b50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:29.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.108+0000 7f331e400640 1 --2- 172.21.15.161:0/557605566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f331815af60 0x7f331813ea50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:29.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.108+0000 7f331e400640 1 --2- 172.21.15.161:0/557605566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f331813fc80 0x7f3318140c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:29.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.108+0000 7f331e400640 1 --2- 172.21.15.161:0/557605566 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f331813cfc0 0x7f33180cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:29.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.108+0000 7f331e400640 1 -- 172.21.15.161:0/557605566 >> 172.21.15.161:0/557605566 conn(0x7f331806a2d0 msgr2=0x7f331806a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:29.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.108+0000 7f331e400640 1 -- 172.21.15.161:0/557605566 shutdown_connections 2024-05-22T21:14:29.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.108+0000 7f331e400640 1 -- 172.21.15.161:0/557605566 wait complete. 2024-05-22T21:14:29.131 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2024-05-22T21:14:29.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.264+0000 7f8b3d000640 1 Processor -- start 2024-05-22T21:14:29.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.264+0000 7f8b3d000640 1 -- start start 2024-05-22T21:14:29.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b3d000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8b3815d180 0x7f8b3806f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:29.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b3d000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b380af070 0x7f8b38069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:29.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b3d000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b3806a410 0x7f8b38070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:29.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b3d000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b380af500 con 0x7f8b3815d180 2024-05-22T21:14:29.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b3d000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3806a860 con 0x7f8b380af070 2024-05-22T21:14:29.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b3d000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3815d560 con 0x7f8b3806a410 2024-05-22T21:14:29.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b37400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8b3815d180 0x7f8b3806f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:29.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b37400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8b3815d180 0x7f8b3806f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55682/0 (socket says 172.21.15.161:55682) 2024-05-22T21:14:29.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b37400640 1 -- 172.21.15.161:0/1670333387 learned_addr learned my addr 172.21.15.161:0/1670333387 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:29.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b37e00640 1 --2- 172.21.15.161:0/1670333387 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b3806a410 0x7f8b38070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:29.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b36a00640 1 --2- 172.21.15.161:0/1670333387 >> v2:172.21.15.179:3300/0 conn(0x7f8b380af070 0x7f8b38069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:29.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b37400640 1 -- 172.21.15.161:0/1670333387 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b3806a410 msgr2=0x7f8b38070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:29.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b37400640 1 --2- 172.21.15.161:0/1670333387 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b3806a410 0x7f8b38070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:29.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b37400640 1 -- 172.21.15.161:0/1670333387 >> v2:172.21.15.179:3300/0 conn(0x7f8b380af070 msgr2=0x7f8b38069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:29.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b37400640 1 --2- 172.21.15.161:0/1670333387 >> v2:172.21.15.179:3300/0 conn(0x7f8b380af070 0x7f8b38069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:29.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b37400640 1 -- 172.21.15.161:0/1670333387 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b380adb00 con 0x7f8b3815d180 2024-05-22T21:14:29.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b36a00640 1 --2- 172.21.15.161:0/1670333387 >> v2:172.21.15.179:3300/0 conn(0x7f8b380af070 0x7f8b38069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:29.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b37e00640 1 --2- 172.21.15.161:0/1670333387 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b3806a410 0x7f8b38070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:29.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b37400640 1 --2- 172.21.15.161:0/1670333387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3815d180 0x7f8b3806f2b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8b380af8c0 tx=0x7f8b2801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fc6e72c9803f90 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:29.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b36000640 1 -- 172.21.15.161:0/1670333387 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b280ab600 con 0x7f8b3815d180 2024-05-22T21:14:29.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b36000640 1 -- 172.21.15.161:0/1670333387 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b280a7050 con 0x7f8b3815d180 2024-05-22T21:14:29.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.268+0000 7f8b36000640 1 -- 172.21.15.161:0/1670333387 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b280ab980 con 0x7f8b3815d180 2024-05-22T21:14:29.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 -- 172.21.15.161:0/1670333387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3815d180 msgr2=0x7f8b3806f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:29.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 --2- 172.21.15.161:0/1670333387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3815d180 0x7f8b3806f2b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8b380af8c0 tx=0x7f8b2801c230 comp rx=0 tx=0).stop 2024-05-22T21:14:29.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 -- 172.21.15.161:0/1670333387 shutdown_connections 2024-05-22T21:14:29.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 --2- 172.21.15.161:0/1670333387 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b3806a410 0x7f8b38070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:29.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 --2- 172.21.15.161:0/1670333387 >> v2:172.21.15.179:3300/0 conn(0x7f8b380af070 0x7f8b38069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:29.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 --2- 172.21.15.161:0/1670333387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3815d180 0x7f8b3806f2b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:29.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 -- 172.21.15.161:0/1670333387 >> 172.21.15.161:0/1670333387 conn(0x7f8b3806b9d0 msgr2=0x7f8b380b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:29.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 -- 172.21.15.161:0/1670333387 shutdown_connections 2024-05-22T21:14:29.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 -- 172.21.15.161:0/1670333387 wait complete. 2024-05-22T21:14:29.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 Processor -- start 2024-05-22T21:14:29.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 -- start start 2024-05-22T21:14:29.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3806a410 0x7f8b3815bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:29.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b380af070 0x7f8b381587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:29.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b3815d180 0x7f8b3814eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:29.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b38157e20 con 0x7f8b3806a410 2024-05-22T21:14:29.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3814e670 con 0x7f8b380af070 2024-05-22T21:14:29.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b3d000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b38157b40 con 0x7f8b3815d180 2024-05-22T21:14:29.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b37400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3806a410 0x7f8b3815bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:29.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b37400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3806a410 0x7f8b3815bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55692/0 (socket says 172.21.15.161:55692) 2024-05-22T21:14:29.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b37400640 1 -- 172.21.15.161:0/3175659342 learned_addr learned my addr 172.21.15.161:0/3175659342 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:29.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b37e00640 1 --2- 172.21.15.161:0/3175659342 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b3815d180 0x7f8b3814eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:29.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b36a00640 1 --2- 172.21.15.161:0/3175659342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b380af070 0x7f8b381587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:29.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b37400640 1 -- 172.21.15.161:0/3175659342 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b3815d180 msgr2=0x7f8b3814eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:29.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b37400640 1 --2- 172.21.15.161:0/3175659342 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b3815d180 0x7f8b3814eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:29.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b37400640 1 -- 172.21.15.161:0/3175659342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b380af070 msgr2=0x7f8b381587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:29.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b37400640 1 --2- 172.21.15.161:0/3175659342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b380af070 0x7f8b381587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:29.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b37400640 1 -- 172.21.15.161:0/3175659342 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b280094d0 con 0x7f8b3806a410 2024-05-22T21:14:29.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.272+0000 7f8b36a00640 1 --2- 172.21.15.161:0/3175659342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b380af070 0x7f8b381587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:29.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.276+0000 7f8b37400640 1 --2- 172.21.15.161:0/3175659342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3806a410 0x7f8b3815bf80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8b280ae040 tx=0x7f8b28018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:29.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.276+0000 7f8b2f400640 1 -- 172.21.15.161:0/3175659342 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b280b37d0 con 0x7f8b3806a410 2024-05-22T21:14:29.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.276+0000 7f8b2f400640 1 -- 172.21.15.161:0/3175659342 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b280b3930 con 0x7f8b3806a410 2024-05-22T21:14:29.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.276+0000 7f8b2f400640 1 -- 172.21.15.161:0/3175659342 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b280b3ce0 con 0x7f8b3806a410 2024-05-22T21:14:29.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.276+0000 7f8b3d000640 1 -- 172.21.15.161:0/3175659342 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3814e020 con 0x7f8b3806a410 2024-05-22T21:14:29.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.276+0000 7f8b3d000640 1 -- 172.21.15.161:0/3175659342 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b381577d0 con 0x7f8b3806a410 2024-05-22T21:14:29.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.276+0000 7f8b3d000640 1 -- 172.21.15.161:0/3175659342 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b38067070 con 0x7f8b3806a410 2024-05-22T21:14:29.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.276+0000 7f8b2f400640 1 -- 172.21.15.161:0/3175659342 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b280b2070 con 0x7f8b3806a410 2024-05-22T21:14:29.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.276+0000 7f8b2f400640 1 --2- 172.21.15.161:0/3175659342 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8b18066690 0x7f8b18068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:29.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.276+0000 7f8b2f400640 1 -- 172.21.15.161:0/3175659342 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b2811e5b0 con 0x7f8b3806a410 2024-05-22T21:14:29.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.276+0000 7f8b36a00640 1 --2- 172.21.15.161:0/3175659342 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8b18066690 0x7f8b18068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:29.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.276+0000 7f8b2f400640 1 -- 172.21.15.161:0/3175659342 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=35+}) v3 -- 0x7f8b1806e1c0 con 0x7f8b3806a410 2024-05-22T21:14:29.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.276+0000 7f8b36a00640 1 --2- 172.21.15.161:0/3175659342 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8b18066690 0x7f8b18068b50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8b38152630 tx=0x7f8b24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:29.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.280+0000 7f8b2f400640 1 -- 172.21.15.161:0/3175659342 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8b2811ce60 con 0x7f8b3806a410 2024-05-22T21:14:29.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:29.444+0000 7f8b3d000640 1 -- 172.21.15.161:0/3175659342 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7f8b38153400 con 0x7f8b3806a410 2024-05-22T21:14:30.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.108+0000 7f8b2f400640 1 -- 172.21.15.161:0/3175659342 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(35..35 src has 1..35) v4 ==== 288+0+0 (secure 0 0 0) 0x7f8b280efcd0 con 0x7f8b3806a410 2024-05-22T21:14:30.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.108+0000 7f8b2f400640 1 -- 172.21.15.161:0/3175659342 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=36}) v3 -- 0x7f8b1806ea00 con 0x7f8b3806a410 2024-05-22T21:14:30.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.108+0000 7f8b2f400640 1 -- 172.21.15.161:0/3175659342 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v35) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8b280b0020 con 0x7f8b3806a410 2024-05-22T21:14:30.111 INFO:teuthology.orchestra.run.smithi161.stderr:pauserd,pausewr is unset 2024-05-22T21:14:30.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.112+0000 7f8b3d000640 1 -- 172.21.15.161:0/3175659342 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8b18066690 msgr2=0x7f8b18068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:30.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.112+0000 7f8b3d000640 1 --2- 172.21.15.161:0/3175659342 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8b18066690 0x7f8b18068b50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8b38152630 tx=0x7f8b24008040 comp rx=0 tx=0).stop 2024-05-22T21:14:30.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.112+0000 7f8b3d000640 1 -- 172.21.15.161:0/3175659342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3806a410 msgr2=0x7f8b3815bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:30.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.112+0000 7f8b3d000640 1 --2- 172.21.15.161:0/3175659342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3806a410 0x7f8b3815bf80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8b280ae040 tx=0x7f8b28018fd0 comp rx=0 tx=0).stop 2024-05-22T21:14:30.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.112+0000 7f8b3d000640 1 -- 172.21.15.161:0/3175659342 shutdown_connections 2024-05-22T21:14:30.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.112+0000 7f8b3d000640 1 --2- 172.21.15.161:0/3175659342 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8b18066690 0x7f8b18068b50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:30.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.112+0000 7f8b3d000640 1 --2- 172.21.15.161:0/3175659342 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b3815d180 0x7f8b3814eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:30.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.112+0000 7f8b3d000640 1 --2- 172.21.15.161:0/3175659342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b380af070 0x7f8b381587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:30.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.112+0000 7f8b3d000640 1 --2- 172.21.15.161:0/3175659342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3806a410 0x7f8b3815bf80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:30.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.112+0000 7f8b3d000640 1 -- 172.21.15.161:0/3175659342 >> 172.21.15.161:0/3175659342 conn(0x7f8b3806b9d0 msgr2=0x7f8b380b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:30.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.112+0000 7f8b3d000640 1 -- 172.21.15.161:0/3175659342 shutdown_connections 2024-05-22T21:14:30.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.112+0000 7f8b3d000640 1 -- 172.21.15.161:0/3175659342 wait complete. 2024-05-22T21:14:30.137 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:30.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.272+0000 7f9d75200640 1 Processor -- start 2024-05-22T21:14:30.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.272+0000 7f9d75200640 1 -- start start 2024-05-22T21:14:30.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.272+0000 7f9d75200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9d70143ec0 0x7f9d7015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:30.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.272+0000 7f9d75200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d701442a0 con 0x7f9d700715a0 2024-05-22T21:14:30.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.272+0000 7f9d75200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d70070500 con 0x7f9d7015ec90 2024-05-22T21:14:30.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.272+0000 7f9d75200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d700707e0 con 0x7f9d70143ec0 2024-05-22T21:14:30.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.272+0000 7f9d6d400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f9d700715a0 0x7f9d7006fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47590/0 (socket says 172.21.15.161:47590) 2024-05-22T21:14:30.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.272+0000 7f9d6d400640 1 -- 172.21.15.161:0/2627271844 learned_addr learned my addr 172.21.15.161:0/2627271844 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:30.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.272+0000 7f9d6de00640 1 --2- 172.21.15.161:0/2627271844 >> v2:172.21.15.179:3300/0 conn(0x7f9d70143ec0 0x7f9d7015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:30.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.272+0000 7f9d6ca00640 1 -- 172.21.15.161:0/2627271844 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247944952 0 0) 0x7f9d701442a0 con 0x7f9d700715a0 2024-05-22T21:14:30.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.272+0000 7f9d6ca00640 1 -- 172.21.15.161:0/2627271844 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d54003530 con 0x7f9d700715a0 2024-05-22T21:14:30.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.272+0000 7f9d6ca00640 1 -- 172.21.15.161:0/2627271844 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620526993 0 0) 0x7f9d70070500 con 0x7f9d7015ec90 2024-05-22T21:14:30.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.272+0000 7f9d6ca00640 1 -- 172.21.15.161:0/2627271844 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d701442a0 con 0x7f9d7015ec90 2024-05-22T21:14:30.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.272+0000 7f9d6de00640 1 -- 172.21.15.161:0/2627271844 >> v1:172.21.15.179:6789/0 conn(0x7f9d7015ec90 legacy=0x7f9d700cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:30.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.272+0000 7f9d6de00640 1 -- 172.21.15.161:0/2627271844 >> v1:172.21.15.161:6789/0 conn(0x7f9d700715a0 legacy=0x7f9d7006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:30.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.272+0000 7f9d6de00640 1 -- 172.21.15.161:0/2627271844 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d70070ac0 con 0x7f9d70143ec0 2024-05-22T21:14:30.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.276+0000 7f9d6de00640 1 --2- 172.21.15.161:0/2627271844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d70143ec0 0x7f9d7015f830 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009d90 tx=0x7f9d6001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5c09fe29b81e698 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:30.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.276+0000 7f9d6ca00640 1 -- 172.21.15.161:0/2627271844 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d600a5a00 con 0x7f9d70143ec0 2024-05-22T21:14:30.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.276+0000 7f9d6ca00640 1 -- 172.21.15.161:0/2627271844 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d600a5b60 con 0x7f9d70143ec0 2024-05-22T21:14:30.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.276+0000 7f9d6ca00640 1 -- 172.21.15.161:0/2627271844 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d600ad440 con 0x7f9d70143ec0 2024-05-22T21:14:30.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.276+0000 7f9d75200640 1 -- 172.21.15.161:0/2627271844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d70143ec0 msgr2=0x7f9d7015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:30.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.276+0000 7f9d75200640 1 --2- 172.21.15.161:0/2627271844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d70143ec0 0x7f9d7015f830 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009d90 tx=0x7f9d6001c940 comp rx=0 tx=0).stop 2024-05-22T21:14:30.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.276+0000 7f9d75200640 1 -- 172.21.15.161:0/2627271844 shutdown_connections 2024-05-22T21:14:30.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.276+0000 7f9d75200640 1 --2- 172.21.15.161:0/2627271844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d70143ec0 0x7f9d7015f830 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:30.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.276+0000 7f9d75200640 1 -- 172.21.15.161:0/2627271844 >> 172.21.15.161:0/2627271844 conn(0x7f9d700b0bc0 msgr2=0x7f9d700b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:30.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.276+0000 7f9d75200640 1 -- 172.21.15.161:0/2627271844 shutdown_connections 2024-05-22T21:14:30.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.276+0000 7f9d75200640 1 -- 172.21.15.161:0/2627271844 wait complete. 2024-05-22T21:14:30.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d75200640 1 Processor -- start 2024-05-22T21:14:30.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d75200640 1 -- start start 2024-05-22T21:14:30.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d75200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d70143ec0 0x7f9d70159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:30.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d75200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d7015ec90 0x7f9d701612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:30.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d75200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d70161800 0x7f9d700b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:30.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d75200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7015a450 con 0x7f9d70143ec0 2024-05-22T21:14:30.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d75200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d700b2090 con 0x7f9d70161800 2024-05-22T21:14:30.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d75200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d700b2370 con 0x7f9d7015ec90 2024-05-22T21:14:30.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d6de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d70143ec0 0x7f9d70159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:30.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d6de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d70143ec0 0x7f9d70159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55700/0 (socket says 172.21.15.161:55700) 2024-05-22T21:14:30.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d6de00640 1 -- 172.21.15.161:0/4011285847 learned_addr learned my addr 172.21.15.161:0/4011285847 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:30.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d6d400640 1 --2- 172.21.15.161:0/4011285847 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d7015ec90 0x7f9d701612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:30.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d6de00640 1 -- 172.21.15.161:0/4011285847 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d7015ec90 msgr2=0x7f9d701612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:30.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d6de00640 1 --2- 172.21.15.161:0/4011285847 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d7015ec90 0x7f9d701612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:30.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d6e800640 1 --2- 172.21.15.161:0/4011285847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d70161800 0x7f9d700b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:30.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d6de00640 1 -- 172.21.15.161:0/4011285847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d70161800 msgr2=0x7f9d700b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:30.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d6de00640 1 --2- 172.21.15.161:0/4011285847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d70161800 0x7f9d700b3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:30.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d6de00640 1 -- 172.21.15.161:0/4011285847 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d60009a70 con 0x7f9d70143ec0 2024-05-22T21:14:30.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d6e800640 1 --2- 172.21.15.161:0/4011285847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d70161800 0x7f9d700b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:30.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d6de00640 1 --2- 172.21.15.161:0/4011285847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d70143ec0 0x7f9d70159f10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9d60019550 tx=0x7f9d600196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:30.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d66000640 1 -- 172.21.15.161:0/4011285847 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d600add80 con 0x7f9d70143ec0 2024-05-22T21:14:30.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d66000640 1 -- 172.21.15.161:0/4011285847 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d60009450 con 0x7f9d70143ec0 2024-05-22T21:14:30.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d75200640 1 -- 172.21.15.161:0/4011285847 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d700b2650 con 0x7f9d70143ec0 2024-05-22T21:14:30.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d66000640 1 -- 172.21.15.161:0/4011285847 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d600b7650 con 0x7f9d70143ec0 2024-05-22T21:14:30.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.280+0000 7f9d75200640 1 -- 172.21.15.161:0/4011285847 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d700b2c10 con 0x7f9d70143ec0 2024-05-22T21:14:30.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.284+0000 7f9d75200640 1 -- 172.21.15.161:0/4011285847 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d70067220 con 0x7f9d70143ec0 2024-05-22T21:14:30.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.284+0000 7f9d66000640 1 -- 172.21.15.161:0/4011285847 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9d600b4050 con 0x7f9d70143ec0 2024-05-22T21:14:30.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.284+0000 7f9d66000640 1 --2- 172.21.15.161:0/4011285847 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d50066690 0x7f9d50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:30.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.284+0000 7f9d66000640 1 -- 172.21.15.161:0/4011285847 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d6011f850 con 0x7f9d70143ec0 2024-05-22T21:14:30.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.284+0000 7f9d6d400640 1 --2- 172.21.15.161:0/4011285847 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d50066690 0x7f9d50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:30.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.284+0000 7f9d6d400640 1 --2- 172.21.15.161:0/4011285847 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d50066690 0x7f9d50068b50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9d700adfa0 tx=0x7f9d5c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:30.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.288+0000 7f9d66000640 1 -- 172.21.15.161:0/4011285847 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9d600ecd70 con 0x7f9d70143ec0 2024-05-22T21:14:30.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.444+0000 7f9d75200640 1 -- 172.21.15.161:0/4011285847 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d701610a0 con 0x7f9d70143ec0 2024-05-22T21:14:30.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.448+0000 7f9d66000640 1 -- 172.21.15.161:0/4011285847 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16149 (secure 0 0 0) 0x7f9d600c7090 con 0x7f9d70143ec0 2024-05-22T21:14:30.449 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:30.449 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":35,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:14:30.106879+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:16.938769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:18.054579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:30.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.448+0000 7f9d75200640 1 -- 172.21.15.161:0/4011285847 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d50066690 msgr2=0x7f9d50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:30.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.448+0000 7f9d75200640 1 --2- 172.21.15.161:0/4011285847 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d50066690 0x7f9d50068b50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9d700adfa0 tx=0x7f9d5c009040 comp rx=0 tx=0).stop 2024-05-22T21:14:30.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.448+0000 7f9d75200640 1 -- 172.21.15.161:0/4011285847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d70143ec0 msgr2=0x7f9d70159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:30.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.448+0000 7f9d75200640 1 --2- 172.21.15.161:0/4011285847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d70143ec0 0x7f9d70159f10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9d60019550 tx=0x7f9d600196e0 comp rx=0 tx=0).stop 2024-05-22T21:14:30.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.448+0000 7f9d75200640 1 -- 172.21.15.161:0/4011285847 shutdown_connections 2024-05-22T21:14:30.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.448+0000 7f9d75200640 1 --2- 172.21.15.161:0/4011285847 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d50066690 0x7f9d50068b50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:30.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.448+0000 7f9d75200640 1 --2- 172.21.15.161:0/4011285847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d70161800 0x7f9d700b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:30.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.448+0000 7f9d75200640 1 --2- 172.21.15.161:0/4011285847 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d7015ec90 0x7f9d701612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:30.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.448+0000 7f9d75200640 1 --2- 172.21.15.161:0/4011285847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d70143ec0 0x7f9d70159f10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:30.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.448+0000 7f9d75200640 1 -- 172.21.15.161:0/4011285847 >> 172.21.15.161:0/4011285847 conn(0x7f9d700b0bc0 msgr2=0x7f9d700b11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:30.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.448+0000 7f9d75200640 1 -- 172.21.15.161:0/4011285847 shutdown_connections 2024-05-22T21:14:30.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:30.448+0000 7f9d75200640 1 -- 172.21.15.161:0/4011285847 wait complete. 2024-05-22T21:14:30.470 DEBUG:teuthology.orchestra.run.smithi161:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-22T21:14:30.470 DEBUG:teuthology.orchestra.run.smithi161:> import os 2024-05-22T21:14:30.470 DEBUG:teuthology.orchestra.run.smithi161:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-05-22T21:14:30.470 DEBUG:teuthology.orchestra.run.smithi161:> os.fchmod(fd, 0O666) 2024-05-22T21:14:30.470 DEBUG:teuthology.orchestra.run.smithi161:> os.fsync(fd) 2024-05-22T21:14:30.470 DEBUG:teuthology.orchestra.run.smithi161:> os.close(fd) 2024-05-22T21:14:30.470 DEBUG:teuthology.orchestra.run.smithi161:> ' 2024-05-22T21:14:30.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:30.722 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:14:30.779 INFO:teuthology.orchestra.run.smithi161.stdout:client4874 2024-05-22T21:14:30.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:30.779 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d6fa8528-b1bf-41b6-9f76-c0d715394087.client4874/osdmap') 2024-05-22T21:14:30.840 INFO:teuthology.orchestra.run.smithi161.stdout:epoch 33 barrier 33 flags 0x588000 2024-05-22T21:14:30.840 INFO:teuthology.orchestra.run.smithi161.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:14:30.840 INFO:teuthology.orchestra.run.smithi161.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:14:30.840 INFO:teuthology.orchestra.run.smithi161.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:14:30.840 INFO:teuthology.orchestra.run.smithi161.stdout:osd0 (1)172.21.15.161:6809 100% (exists, up) 100% -1 2024-05-22T21:14:30.840 INFO:teuthology.orchestra.run.smithi161.stdout:osd1 (1)172.21.15.161:6801 100% (exists, up) 100% -1 2024-05-22T21:14:30.840 INFO:teuthology.orchestra.run.smithi161.stdout:osd2 (1)172.21.15.161:6825 100% (exists, up) 100% -1 2024-05-22T21:14:30.841 INFO:teuthology.orchestra.run.smithi161.stdout:osd3 (1)172.21.15.161:6817 100% (exists, up) 100% -1 2024-05-22T21:14:30.841 INFO:teuthology.orchestra.run.smithi161.stdout:osd4 (1)172.21.15.179:6801 100% (exists, up) 100% -1 2024-05-22T21:14:30.841 INFO:teuthology.orchestra.run.smithi161.stdout:osd5 (1)172.21.15.179:6809 100% (exists, up) 100% -1 2024-05-22T21:14:30.841 INFO:teuthology.orchestra.run.smithi161.stdout:osd6 (1)172.21.15.179:6822 100% (exists, up) 100% -1 2024-05-22T21:14:30.841 INFO:teuthology.orchestra.run.smithi161.stdout:osd7 (1)172.21.15.179:6817 100% (exists, up) 100% -1 2024-05-22T21:14:30.841 INFO:teuthology.orchestra.run.smithi161.stderr:1+1 records in 2024-05-22T21:14:30.841 INFO:teuthology.orchestra.run.smithi161.stderr:1+1 records out 2024-05-22T21:14:30.841 INFO:teuthology.orchestra.run.smithi161.stderr:799 bytes copied, 8.1831e-05 s, 9.8 MB/s 2024-05-22T21:14:30.842 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:14:31.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f79a00640 1 Processor -- start 2024-05-22T21:14:31.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f79a00640 1 -- start start 2024-05-22T21:14:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f79a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f74099ca0 0x7f6f740b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f79a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6f740cba60 0x7f6f7409a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f79a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6f7409a5c0 0x7f6f7406ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f79a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f74070050 con 0x7f6f740cba60 2024-05-22T21:14:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f79a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f7406e760 con 0x7f6f7409a5c0 2024-05-22T21:14:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f79a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7406fab0 con 0x7f6f74099ca0 2024-05-22T21:14:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f6a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6f740cba60 0x7f6f7409a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f6a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6f740cba60 0x7f6f7409a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55702/0 (socket says 172.21.15.161:55702) 2024-05-22T21:14:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f6a000640 1 -- 172.21.15.161:0/957218180 learned_addr learned my addr 172.21.15.161:0/957218180 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f73400640 1 --2- 172.21.15.161:0/957218180 >> v2:172.21.15.179:3300/0 conn(0x7f6f7409a5c0 0x7f6f7406ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:31.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f72a00640 1 --2- 172.21.15.161:0/957218180 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f74099ca0 0x7f6f740b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:31.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f6a000640 1 -- 172.21.15.161:0/957218180 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f74099ca0 msgr2=0x7f6f740b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:31.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f6a000640 1 --2- 172.21.15.161:0/957218180 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f74099ca0 0x7f6f740b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:31.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f6a000640 1 -- 172.21.15.161:0/957218180 >> v2:172.21.15.179:3300/0 conn(0x7f6f7409a5c0 msgr2=0x7f6f7406ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:31.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f6a000640 1 --2- 172.21.15.161:0/957218180 >> v2:172.21.15.179:3300/0 conn(0x7f6f7409a5c0 0x7f6f7406ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:31.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.016+0000 7f6f6a000640 1 -- 172.21.15.161:0/957218180 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7406fd70 con 0x7f6f740cba60 2024-05-22T21:14:31.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.020+0000 7f6f6a000640 1 --2- 172.21.15.161:0/957218180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f740cba60 0x7f6f7409a080 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6f74066ae0 tx=0x7f6f5c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efe475d1406e1a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:31.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.020+0000 7f6f72000640 1 -- 172.21.15.161:0/957218180 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0aa610 con 0x7f6f740cba60 2024-05-22T21:14:31.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.020+0000 7f6f72000640 1 -- 172.21.15.161:0/957218180 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c0a6040 con 0x7f6f740cba60 2024-05-22T21:14:31.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.020+0000 7f6f72000640 1 -- 172.21.15.161:0/957218180 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0aa990 con 0x7f6f740cba60 2024-05-22T21:14:31.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.020+0000 7f6f79a00640 1 -- 172.21.15.161:0/957218180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f740cba60 msgr2=0x7f6f7409a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:31.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.020+0000 7f6f79a00640 1 --2- 172.21.15.161:0/957218180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f740cba60 0x7f6f7409a080 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6f74066ae0 tx=0x7f6f5c01c4f0 comp rx=0 tx=0).stop 2024-05-22T21:14:31.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.020+0000 7f6f79a00640 1 -- 172.21.15.161:0/957218180 shutdown_connections 2024-05-22T21:14:31.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.020+0000 7f6f79a00640 1 --2- 172.21.15.161:0/957218180 >> v2:172.21.15.179:3300/0 conn(0x7f6f7409a5c0 0x7f6f7406ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:31.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.020+0000 7f6f79a00640 1 --2- 172.21.15.161:0/957218180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f740cba60 0x7f6f7409a080 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:31.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.020+0000 7f6f79a00640 1 --2- 172.21.15.161:0/957218180 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f74099ca0 0x7f6f740b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:31.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.020+0000 7f6f79a00640 1 -- 172.21.15.161:0/957218180 >> 172.21.15.161:0/957218180 conn(0x7f6f740b2a40 msgr2=0x7f6f740b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:31.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.020+0000 7f6f79a00640 1 -- 172.21.15.161:0/957218180 shutdown_connections 2024-05-22T21:14:31.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.020+0000 7f6f79a00640 1 -- 172.21.15.161:0/957218180 wait complete. 2024-05-22T21:14:31.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f79a00640 1 Processor -- start 2024-05-22T21:14:31.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f79a00640 1 -- start start 2024-05-22T21:14:31.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f79a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f74099ca0 0x7f6f7414a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:31.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f79a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f7409a5c0 0x7f6f740cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:31.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f79a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f740cba60 0x7f6f740c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:31.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f79a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740c56b0 con 0x7f6f7409a5c0 2024-05-22T21:14:31.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f79a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7414a680 con 0x7f6f740cba60 2024-05-22T21:14:31.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f79a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740c4da0 con 0x7f6f74099ca0 2024-05-22T21:14:31.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f6a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f7409a5c0 0x7f6f740cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:31.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f6a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f7409a5c0 0x7f6f740cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55710/0 (socket says 172.21.15.161:55710) 2024-05-22T21:14:31.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f6a000640 1 -- 172.21.15.161:0/3289551317 learned_addr learned my addr 172.21.15.161:0/3289551317 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:31.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f72a00640 1 --2- 172.21.15.161:0/3289551317 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f74099ca0 0x7f6f7414a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:31.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f73400640 1 --2- 172.21.15.161:0/3289551317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f740cba60 0x7f6f740c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:31.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f6a000640 1 -- 172.21.15.161:0/3289551317 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f74099ca0 msgr2=0x7f6f7414a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:31.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f6a000640 1 --2- 172.21.15.161:0/3289551317 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f74099ca0 0x7f6f7414a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:31.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f6a000640 1 -- 172.21.15.161:0/3289551317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f740cba60 msgr2=0x7f6f740c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:31.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f6a000640 1 --2- 172.21.15.161:0/3289551317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f740cba60 0x7f6f740c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:31.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f6a000640 1 -- 172.21.15.161:0/3289551317 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c0094d0 con 0x7f6f7409a5c0 2024-05-22T21:14:31.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f6a000640 1 --2- 172.21.15.161:0/3289551317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f7409a5c0 0x7f6f740cd3c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c01aba0 tx=0x7f6f5c019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:31.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f6b400640 1 -- 172.21.15.161:0/3289551317 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0be610 con 0x7f6f7409a5c0 2024-05-22T21:14:31.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f6b400640 1 -- 172.21.15.161:0/3289551317 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c0be790 con 0x7f6f7409a5c0 2024-05-22T21:14:31.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f6b400640 1 -- 172.21.15.161:0/3289551317 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0beb40 con 0x7f6f7409a5c0 2024-05-22T21:14:31.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f79a00640 1 -- 172.21.15.161:0/3289551317 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f740cdd80 con 0x7f6f7409a5c0 2024-05-22T21:14:31.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.024+0000 7f6f79a00640 1 -- 172.21.15.161:0/3289551317 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f740c5340 con 0x7f6f7409a5c0 2024-05-22T21:14:31.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.028+0000 7f6f79a00640 1 -- 172.21.15.161:0/3289551317 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f740c5860 con 0x7f6f7409a5c0 2024-05-22T21:14:31.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.028+0000 7f6f6b400640 1 -- 172.21.15.161:0/3289551317 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6f5c0b4400 con 0x7f6f7409a5c0 2024-05-22T21:14:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.028+0000 7f6f6b400640 1 --2- 172.21.15.161:0/3289551317 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6f50066690 0x7f6f50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.028+0000 7f6f6b400640 1 -- 172.21.15.161:0/3289551317 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f5c11e900 con 0x7f6f7409a5c0 2024-05-22T21:14:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.028+0000 7f6f72a00640 1 --2- 172.21.15.161:0/3289551317 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6f50066690 0x7f6f50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.028+0000 7f6f72a00640 1 --2- 172.21.15.161:0/3289551317 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6f50066690 0x7f6f50068b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6f64003ab0 tx=0x7f6f64009810 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:31.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.032+0000 7f6f6b400640 1 -- 172.21.15.161:0/3289551317 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6f740c5860 con 0x7f6f7409a5c0 2024-05-22T21:14:31.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.220+0000 7f6f79a00640 1 -- 172.21.15.161:0/3289551317 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f740c5860 con 0x7f6f7409a5c0 2024-05-22T21:14:31.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.220+0000 7f6f6b400640 1 -- 172.21.15.161:0/3289551317 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f6f740c5860 con 0x7f6f7409a5c0 2024-05-22T21:14:31.222 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:31.222 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3742018925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":3742018925},{"type":"v1","addr":"172.21.15.161:6837","nonce":3742018925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2553366378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2553366378},{"type":"v1","addr":"172.21.15.179:6834","nonce":2553366378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/509468338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":509468338},{"type":"v1","addr":"172.21.15.179:6835","nonce":509468338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:19.053259+0000","modified":"2024-05-22T21:14:22.076119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6836/1635458417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1635458417},{"type":"v1","addr":"172.21.15.161:6836","nonce":1635458417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-22T21:14:31.222 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 16 2024-05-22T21:14:31.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.220+0000 7f6f79a00640 1 -- 172.21.15.161:0/3289551317 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6f50066690 msgr2=0x7f6f50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:31.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.220+0000 7f6f79a00640 1 --2- 172.21.15.161:0/3289551317 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6f50066690 0x7f6f50068b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6f64003ab0 tx=0x7f6f64009810 comp rx=0 tx=0).stop 2024-05-22T21:14:31.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.220+0000 7f6f79a00640 1 -- 172.21.15.161:0/3289551317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f7409a5c0 msgr2=0x7f6f740cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:31.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.220+0000 7f6f79a00640 1 --2- 172.21.15.161:0/3289551317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f7409a5c0 0x7f6f740cd3c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c01aba0 tx=0x7f6f5c019290 comp rx=0 tx=0).stop 2024-05-22T21:14:31.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.220+0000 7f6f79a00640 1 -- 172.21.15.161:0/3289551317 shutdown_connections 2024-05-22T21:14:31.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.220+0000 7f6f79a00640 1 --2- 172.21.15.161:0/3289551317 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6f50066690 0x7f6f50068b50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:31.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.220+0000 7f6f79a00640 1 --2- 172.21.15.161:0/3289551317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f740cba60 0x7f6f740c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:31.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.220+0000 7f6f79a00640 1 --2- 172.21.15.161:0/3289551317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f7409a5c0 0x7f6f740cd3c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:31.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.220+0000 7f6f79a00640 1 --2- 172.21.15.161:0/3289551317 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f74099ca0 0x7f6f7414a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:31.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.220+0000 7f6f79a00640 1 -- 172.21.15.161:0/3289551317 >> 172.21.15.161:0/3289551317 conn(0x7f6f740b2a40 msgr2=0x7f6f740ae090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:31.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.220+0000 7f6f79a00640 1 -- 172.21.15.161:0/3289551317 shutdown_connections 2024-05-22T21:14:31.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:31.220+0000 7f6f79a00640 1 -- 172.21.15.161:0/3289551317 wait complete. 2024-05-22T21:14:31.244 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json osdmap barrier 35 2024-05-22T21:14:31.399 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-05-22T21:14:31.399 DEBUG:teuthology.orchestra.run.smithi161:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-22T21:14:31.400 DEBUG:teuthology.orchestra.run.smithi161:> import os 2024-05-22T21:14:31.400 DEBUG:teuthology.orchestra.run.smithi161:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2024-05-22T21:14:31.400 DEBUG:teuthology.orchestra.run.smithi161:> os.fchmod(fd, 0O666) 2024-05-22T21:14:31.400 DEBUG:teuthology.orchestra.run.smithi161:> os.fsync(fd) 2024-05-22T21:14:31.400 DEBUG:teuthology.orchestra.run.smithi161:> os.close(fd) 2024-05-22T21:14:31.400 DEBUG:teuthology.orchestra.run.smithi161:> ' 2024-05-22T21:14:31.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:31.653 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:14:31.708 INFO:teuthology.orchestra.run.smithi161.stdout:client4874 2024-05-22T21:14:31.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:31.708 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d6fa8528-b1bf-41b6-9f76-c0d715394087.client4874/osdmap') 2024-05-22T21:14:31.767 INFO:teuthology.orchestra.run.smithi161.stdout:epoch 35 barrier 35 flags 0x588000 2024-05-22T21:14:31.767 INFO:teuthology.orchestra.run.smithi161.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:14:31.768 INFO:teuthology.orchestra.run.smithi161.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:14:31.768 INFO:teuthology.orchestra.run.smithi161.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:14:31.768 INFO:teuthology.orchestra.run.smithi161.stdout:osd0 (1)172.21.15.161:6809 100% (exists, up) 100% -1 2024-05-22T21:14:31.768 INFO:teuthology.orchestra.run.smithi161.stdout:osd1 (1)172.21.15.161:6801 100% (exists, up) 100% -1 2024-05-22T21:14:31.768 INFO:teuthology.orchestra.run.smithi161.stdout:osd2 (1)172.21.15.161:6825 100% (exists, up) 100% -1 2024-05-22T21:14:31.768 INFO:teuthology.orchestra.run.smithi161.stdout:osd3 (1)172.21.15.161:6817 100% (exists, up) 100% -1 2024-05-22T21:14:31.768 INFO:teuthology.orchestra.run.smithi161.stdout:osd4 (1)172.21.15.179:6801 100% (exists, up) 100% -1 2024-05-22T21:14:31.768 INFO:teuthology.orchestra.run.smithi161.stdout:osd5 (1)172.21.15.179:6809 100% (exists, up) 100% -1 2024-05-22T21:14:31.768 INFO:teuthology.orchestra.run.smithi161.stdout:osd6 (1)172.21.15.179:6822 100% (exists, up) 100% -1 2024-05-22T21:14:31.768 INFO:teuthology.orchestra.run.smithi161.stdout:osd7 (1)172.21.15.179:6817 100% (exists, up) 100% -1 2024-05-22T21:14:31.768 INFO:teuthology.orchestra.run.smithi161.stderr:1+1 records in 2024-05-22T21:14:31.769 INFO:teuthology.orchestra.run.smithi161.stderr:1+1 records out 2024-05-22T21:14:31.769 INFO:teuthology.orchestra.run.smithi161.stderr:799 bytes copied, 7.9094e-05 s, 10.1 MB/s 2024-05-22T21:14:31.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:31.770 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:14:31.827 INFO:teuthology.orchestra.run.smithi161.stdout:client4874 2024-05-22T21:14:31.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:31.827 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d6fa8528-b1bf-41b6-9f76-c0d715394087.client4874/osdmap') 2024-05-22T21:14:31.887 INFO:teuthology.orchestra.run.smithi161.stdout:epoch 35 barrier 35 flags 0x588000 2024-05-22T21:14:31.887 INFO:teuthology.orchestra.run.smithi161.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:14:31.887 INFO:teuthology.orchestra.run.smithi161.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:14:31.887 INFO:teuthology.orchestra.run.smithi161.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:14:31.887 INFO:teuthology.orchestra.run.smithi161.stdout:osd0 (1)172.21.15.161:6809 100% (exists, up) 100% -1 2024-05-22T21:14:31.888 INFO:teuthology.orchestra.run.smithi161.stdout:osd1 (1)172.21.15.161:6801 100% (exists, up) 100% -1 2024-05-22T21:14:31.888 INFO:teuthology.orchestra.run.smithi161.stdout:osd2 (1)172.21.15.161:6825 100% (exists, up) 100% -1 2024-05-22T21:14:31.888 INFO:teuthology.orchestra.run.smithi161.stdout:osd3 (1)172.21.15.161:6817 100% (exists, up) 100% -1 2024-05-22T21:14:31.888 INFO:teuthology.orchestra.run.smithi161.stdout:osd4 (1)172.21.15.179:6801 100% (exists, up) 100% -1 2024-05-22T21:14:31.888 INFO:teuthology.orchestra.run.smithi161.stdout:osd5 (1)172.21.15.179:6809 100% (exists, up) 100% -1 2024-05-22T21:14:31.888 INFO:teuthology.orchestra.run.smithi161.stdout:osd6 (1)172.21.15.179:6822 100% (exists, up) 100% -1 2024-05-22T21:14:31.888 INFO:teuthology.orchestra.run.smithi161.stdout:osd7 (1)172.21.15.179:6817 100% (exists, up) 100% -1 2024-05-22T21:14:31.888 INFO:teuthology.orchestra.run.smithi161.stderr:1+1 records in 2024-05-22T21:14:31.888 INFO:teuthology.orchestra.run.smithi161.stderr:1+1 records out 2024-05-22T21:14:31.888 INFO:teuthology.orchestra.run.smithi161.stderr:799 bytes copied, 6.7697e-05 s, 11.8 MB/s 2024-05-22T21:14:31.889 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-22T21:14:31.889 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:14:31.890 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:14:31.996 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:14:32.078 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:14:32.078 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:14:32.078 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:32.086 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:14:32.086 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:32.140 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:32.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.312+0000 7ff943200640 1 Processor -- start 2024-05-22T21:14:32.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.312+0000 7ff943200640 1 -- start start 2024-05-22T21:14:32.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.312+0000 7ff943200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff94415e640 0x7ff944164b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.312+0000 7ff943200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff944143b40 con 0x7ff944143760 2024-05-22T21:14:32.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.312+0000 7ff943200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff94406c4e0 con 0x7ff944071350 2024-05-22T21:14:32.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.312+0000 7ff943200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94406c7c0 con 0x7ff94415e640 2024-05-22T21:14:32.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.312+0000 7ff941e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff944143760 0x7ff944070c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47592/0 (socket says 172.21.15.161:47592) 2024-05-22T21:14:32.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.312+0000 7ff941e00640 1 -- 172.21.15.161:0/38235295 learned_addr learned my addr 172.21.15.161:0/38235295 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:32.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.312+0000 7ff942800640 1 --2- 172.21.15.161:0/38235295 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff94415e640 0x7ff944164b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.312+0000 7ff941400640 1 -- 172.21.15.161:0/38235295 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874604333 0 0) 0x7ff944143b40 con 0x7ff944143760 2024-05-22T21:14:32.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.312+0000 7ff941400640 1 -- 172.21.15.161:0/38235295 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff928005740 con 0x7ff944143760 2024-05-22T21:14:32.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.316+0000 7ff941400640 1 -- 172.21.15.161:0/38235295 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131379879 0 0) 0x7ff94406c4e0 con 0x7ff944071350 2024-05-22T21:14:32.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.316+0000 7ff941400640 1 -- 172.21.15.161:0/38235295 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff944143b40 con 0x7ff944071350 2024-05-22T21:14:32.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.316+0000 7ff941400640 1 -- 172.21.15.161:0/38235295 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3038072724 0 0) 0x7ff928005740 con 0x7ff944143760 2024-05-22T21:14:32.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.316+0000 7ff942800640 1 -- 172.21.15.161:0/38235295 >> v1:172.21.15.179:6789/0 conn(0x7ff944071350 legacy=0x7ff94406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:32.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.316+0000 7ff942800640 1 -- 172.21.15.161:0/38235295 >> v1:172.21.15.161:6789/0 conn(0x7ff944143760 legacy=0x7ff944070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:32.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.316+0000 7ff942800640 1 -- 172.21.15.161:0/38235295 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94415db60 con 0x7ff94415e640 2024-05-22T21:14:32.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.316+0000 7ff942800640 1 --2- 172.21.15.161:0/38235295 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff94415e640 0x7ff944164b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0097f0 tx=0x7ff93c01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e7a7892fb1c8198 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:32.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.316+0000 7ff941400640 1 -- 172.21.15.161:0/38235295 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934001a20 con 0x7ff94415e640 2024-05-22T21:14:32.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.316+0000 7ff941400640 1 -- 172.21.15.161:0/38235295 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93c0a7050 con 0x7ff94415e640 2024-05-22T21:14:32.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.316+0000 7ff941400640 1 -- 172.21.15.161:0/38235295 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0ab400 con 0x7ff94415e640 2024-05-22T21:14:32.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff943200640 1 -- 172.21.15.161:0/38235295 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff94415e640 msgr2=0x7ff944164b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:32.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff943200640 1 --2- 172.21.15.161:0/38235295 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff94415e640 0x7ff944164b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0097f0 tx=0x7ff93c01c400 comp rx=0 tx=0).stop 2024-05-22T21:14:32.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff943200640 1 -- 172.21.15.161:0/38235295 shutdown_connections 2024-05-22T21:14:32.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff943200640 1 --2- 172.21.15.161:0/38235295 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff94415e640 0x7ff944164b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff943200640 1 -- 172.21.15.161:0/38235295 >> 172.21.15.161:0/38235295 conn(0x7ff94406a4d0 msgr2=0x7ff9440b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:32.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff943200640 1 -- 172.21.15.161:0/38235295 shutdown_connections 2024-05-22T21:14:32.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff943200640 1 -- 172.21.15.161:0/38235295 wait complete. 2024-05-22T21:14:32.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff943200640 1 Processor -- start 2024-05-22T21:14:32.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff943200640 1 -- start start 2024-05-22T21:14:32.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff943200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff944071350 0x7ff9441599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff943200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff944143760 0x7ff94413e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff943200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff94415e640 0x7ff944163fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff943200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff944159f20 con 0x7ff94415e640 2024-05-22T21:14:32.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff943200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff944139930 con 0x7ff944071350 2024-05-22T21:14:32.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff943200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff944163cf0 con 0x7ff944143760 2024-05-22T21:14:32.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff942800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff94415e640 0x7ff944163fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff942800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff94415e640 0x7ff944163fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55724/0 (socket says 172.21.15.161:55724) 2024-05-22T21:14:32.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff942800640 1 -- 172.21.15.161:0/1903715273 learned_addr learned my addr 172.21.15.161:0/1903715273 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:32.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff941e00640 1 --2- 172.21.15.161:0/1903715273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff944071350 0x7ff9441599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff939400640 1 --2- 172.21.15.161:0/1903715273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff944143760 0x7ff94413e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff942800640 1 -- 172.21.15.161:0/1903715273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff944143760 msgr2=0x7ff94413e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:32.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff942800640 1 --2- 172.21.15.161:0/1903715273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff944143760 0x7ff94413e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff942800640 1 -- 172.21.15.161:0/1903715273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff944071350 msgr2=0x7ff9441599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:32.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff942800640 1 --2- 172.21.15.161:0/1903715273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff944071350 0x7ff9441599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.320+0000 7ff942800640 1 -- 172.21.15.161:0/1903715273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c009420 con 0x7ff94415e640 2024-05-22T21:14:32.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.324+0000 7ff942800640 1 --2- 172.21.15.161:0/1903715273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff94415e640 0x7ff944163fa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0097f0 tx=0x7ff93c0191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:32.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.324+0000 7ff93aa00640 1 -- 172.21.15.161:0/1903715273 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0abc80 con 0x7ff94415e640 2024-05-22T21:14:32.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.324+0000 7ff93aa00640 1 -- 172.21.15.161:0/1903715273 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93c0abde0 con 0x7ff94415e640 2024-05-22T21:14:32.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.324+0000 7ff93aa00640 1 -- 172.21.15.161:0/1903715273 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0b5690 con 0x7ff94415e640 2024-05-22T21:14:32.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.324+0000 7ff943200640 1 -- 172.21.15.161:0/1903715273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff944139000 con 0x7ff94415e640 2024-05-22T21:14:32.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.324+0000 7ff943200640 1 -- 172.21.15.161:0/1903715273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9441395c0 con 0x7ff94415e640 2024-05-22T21:14:32.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.324+0000 7ff943200640 1 -- 172.21.15.161:0/1903715273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff944139aa0 con 0x7ff94415e640 2024-05-22T21:14:32.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.324+0000 7ff93aa00640 1 -- 172.21.15.161:0/1903715273 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff93c0b2040 con 0x7ff94415e640 2024-05-22T21:14:32.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.324+0000 7ff93aa00640 1 --2- 172.21.15.161:0/1903715273 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff920066690 0x7ff920068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.324+0000 7ff93aa00640 1 -- 172.21.15.161:0/1903715273 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff93c11e2b0 con 0x7ff94415e640 2024-05-22T21:14:32.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.324+0000 7ff941e00640 1 --2- 172.21.15.161:0/1903715273 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff920066690 0x7ff920068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.324+0000 7ff941e00640 1 --2- 172.21.15.161:0/1903715273 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff920066690 0x7ff920068b50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff9340012f0 tx=0x7ff934007ba0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:32.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.328+0000 7ff93aa00640 1 -- 172.21.15.161:0/1903715273 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff93c0eb5a0 con 0x7ff94415e640 2024-05-22T21:14:32.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.488+0000 7ff943200640 1 -- 172.21.15.161:0/1903715273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff944067070 con 0x7ff94415e640 2024-05-22T21:14:32.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.488+0000 7ff93aa00640 1 -- 172.21.15.161:0/1903715273 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16149 (secure 0 0 0) 0x7ff93c0b0090 con 0x7ff94415e640 2024-05-22T21:14:32.491 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:32.491 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":35,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:14:30.106879+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:16.938769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:18.054579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:32.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.488+0000 7ff943200640 1 -- 172.21.15.161:0/1903715273 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff920066690 msgr2=0x7ff920068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:32.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.488+0000 7ff943200640 1 --2- 172.21.15.161:0/1903715273 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff920066690 0x7ff920068b50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff9340012f0 tx=0x7ff934007ba0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.488+0000 7ff943200640 1 -- 172.21.15.161:0/1903715273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff94415e640 msgr2=0x7ff944163fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:32.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.488+0000 7ff943200640 1 --2- 172.21.15.161:0/1903715273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff94415e640 0x7ff944163fa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0097f0 tx=0x7ff93c0191a0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.488+0000 7ff943200640 1 -- 172.21.15.161:0/1903715273 shutdown_connections 2024-05-22T21:14:32.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.488+0000 7ff943200640 1 --2- 172.21.15.161:0/1903715273 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff920066690 0x7ff920068b50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.488+0000 7ff943200640 1 --2- 172.21.15.161:0/1903715273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff94415e640 0x7ff944163fa0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.488+0000 7ff943200640 1 --2- 172.21.15.161:0/1903715273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff944143760 0x7ff94413e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.488+0000 7ff943200640 1 --2- 172.21.15.161:0/1903715273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff944071350 0x7ff9441599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.488+0000 7ff943200640 1 -- 172.21.15.161:0/1903715273 >> 172.21.15.161:0/1903715273 conn(0x7ff94406a4d0 msgr2=0x7ff944070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:32.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.492+0000 7ff943200640 1 -- 172.21.15.161:0/1903715273 shutdown_connections 2024-05-22T21:14:32.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.492+0000 7ff943200640 1 -- 172.21.15.161:0/1903715273 wait complete. 2024-05-22T21:14:32.511 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:14:16.938769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:14:18.054579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:14:32.512 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:14:32.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4be600640 1 Processor -- start 2024-05-22T21:14:32.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4be600640 1 -- start start 2024-05-22T21:14:32.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4be600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff4b8143c40 0x7ff4b815b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4be600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff4b81444a0 0x7ff4b806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4be600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4b8144900 0x7ff4b806c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4be600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b815efb0 con 0x7ff4b8143c40 2024-05-22T21:14:32.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4be600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b815f290 con 0x7ff4b81444a0 2024-05-22T21:14:32.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4be600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b815f570 con 0x7ff4b8144900 2024-05-22T21:14:32.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4b7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff4b8143c40 0x7ff4b815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4b7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff4b8143c40 0x7ff4b815b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55736/0 (socket says 172.21.15.161:55736) 2024-05-22T21:14:32.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4b7400640 1 -- 172.21.15.161:0/345102240 learned_addr learned my addr 172.21.15.161:0/345102240 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:32.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4b7e00640 1 --2- 172.21.15.161:0/345102240 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4b8144900 0x7ff4b806c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4afe00640 1 --2- 172.21.15.161:0/345102240 >> v2:172.21.15.179:3300/0 conn(0x7ff4b81444a0 0x7ff4b806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4b7400640 1 -- 172.21.15.161:0/345102240 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4b8144900 msgr2=0x7ff4b806c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:32.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4b7400640 1 --2- 172.21.15.161:0/345102240 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4b8144900 0x7ff4b806c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4b7400640 1 -- 172.21.15.161:0/345102240 >> v2:172.21.15.179:3300/0 conn(0x7ff4b81444a0 msgr2=0x7ff4b806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:32.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4b7400640 1 --2- 172.21.15.161:0/345102240 >> v2:172.21.15.179:3300/0 conn(0x7ff4b81444a0 0x7ff4b806c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.648+0000 7ff4b7400640 1 -- 172.21.15.161:0/345102240 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b815f850 con 0x7ff4b8143c40 2024-05-22T21:14:32.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.652+0000 7ff4b7400640 1 --2- 172.21.15.161:0/345102240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b8143c40 0x7ff4b815b600 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80b1c80 tx=0x7ff4a801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f521b2a91a54a66 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:32.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.652+0000 7ff4b6a00640 1 -- 172.21.15.161:0/345102240 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80ab640 con 0x7ff4b8143c40 2024-05-22T21:14:32.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.652+0000 7ff4b6a00640 1 -- 172.21.15.161:0/345102240 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a80a7070 con 0x7ff4b8143c40 2024-05-22T21:14:32.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.652+0000 7ff4b6a00640 1 -- 172.21.15.161:0/345102240 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80ab9c0 con 0x7ff4b8143c40 2024-05-22T21:14:32.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.652+0000 7ff4be600640 1 -- 172.21.15.161:0/345102240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b8143c40 msgr2=0x7ff4b815b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:32.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.652+0000 7ff4be600640 1 --2- 172.21.15.161:0/345102240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b8143c40 0x7ff4b815b600 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80b1c80 tx=0x7ff4a801c520 comp rx=0 tx=0).stop 2024-05-22T21:14:32.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.652+0000 7ff4be600640 1 -- 172.21.15.161:0/345102240 shutdown_connections 2024-05-22T21:14:32.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.652+0000 7ff4be600640 1 --2- 172.21.15.161:0/345102240 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4b8144900 0x7ff4b806c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.652+0000 7ff4be600640 1 --2- 172.21.15.161:0/345102240 >> v2:172.21.15.179:3300/0 conn(0x7ff4b81444a0 0x7ff4b806c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.652+0000 7ff4be600640 1 --2- 172.21.15.161:0/345102240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b8143c40 0x7ff4b815b600 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.652+0000 7ff4be600640 1 -- 172.21.15.161:0/345102240 >> 172.21.15.161:0/345102240 conn(0x7ff4b806a2d0 msgr2=0x7ff4b806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:32.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.652+0000 7ff4be600640 1 -- 172.21.15.161:0/345102240 shutdown_connections 2024-05-22T21:14:32.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.652+0000 7ff4be600640 1 -- 172.21.15.161:0/345102240 wait complete. 2024-05-22T21:14:32.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4be600640 1 Processor -- start 2024-05-22T21:14:32.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4be600640 1 -- start start 2024-05-22T21:14:32.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4be600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b8143c40 0x7ff4b8138e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4be600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4b81444a0 0x7ff4b806c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4be600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4b8144900 0x7ff4b806c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4be600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b815eae0 con 0x7ff4b8143c40 2024-05-22T21:14:32.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4be600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b815e520 con 0x7ff4b81444a0 2024-05-22T21:14:32.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4be600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b815e800 con 0x7ff4b8144900 2024-05-22T21:14:32.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4b7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b8143c40 0x7ff4b8138e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4b7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b8143c40 0x7ff4b8138e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55750/0 (socket says 172.21.15.161:55750) 2024-05-22T21:14:32.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4b7400640 1 -- 172.21.15.161:0/3373288970 learned_addr learned my addr 172.21.15.161:0/3373288970 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:32.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4b7e00640 1 --2- 172.21.15.161:0/3373288970 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4b8144900 0x7ff4b806c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4afe00640 1 --2- 172.21.15.161:0/3373288970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4b81444a0 0x7ff4b806c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4b7400640 1 -- 172.21.15.161:0/3373288970 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4b8144900 msgr2=0x7ff4b806c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:32.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4b7400640 1 --2- 172.21.15.161:0/3373288970 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4b8144900 0x7ff4b806c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4b7400640 1 -- 172.21.15.161:0/3373288970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4b81444a0 msgr2=0x7ff4b806c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:32.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4b7400640 1 --2- 172.21.15.161:0/3373288970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4b81444a0 0x7ff4b806c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4b7400640 1 -- 172.21.15.161:0/3373288970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a80094d0 con 0x7ff4b8143c40 2024-05-22T21:14:32.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4b7400640 1 --2- 172.21.15.161:0/3373288970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b8143c40 0x7ff4b8138e80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff4a801abd0 tx=0x7ff4a801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:32.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4af400640 1 -- 172.21.15.161:0/3373288970 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80abdf0 con 0x7ff4b8143c40 2024-05-22T21:14:32.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4af400640 1 -- 172.21.15.161:0/3373288970 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a80be3f0 con 0x7ff4b8143c40 2024-05-22T21:14:32.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4be600640 1 -- 172.21.15.161:0/3373288970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b815c750 con 0x7ff4b8143c40 2024-05-22T21:14:32.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4af400640 1 -- 172.21.15.161:0/3373288970 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80be780 con 0x7ff4b8143c40 2024-05-22T21:14:32.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.656+0000 7ff4be600640 1 -- 172.21.15.161:0/3373288970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b815e230 con 0x7ff4b8143c40 2024-05-22T21:14:32.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.660+0000 7ff4be600640 1 -- 172.21.15.161:0/3373288970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b81570e0 con 0x7ff4b8143c40 2024-05-22T21:14:32.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.660+0000 7ff4af400640 1 -- 172.21.15.161:0/3373288970 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff4a80b43f0 con 0x7ff4b8143c40 2024-05-22T21:14:32.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.660+0000 7ff4af400640 1 --2- 172.21.15.161:0/3373288970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff494066690 0x7ff494068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.660+0000 7ff4af400640 1 -- 172.21.15.161:0/3373288970 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4a811e7a0 con 0x7ff4b8143c40 2024-05-22T21:14:32.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.660+0000 7ff4afe00640 1 --2- 172.21.15.161:0/3373288970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff494066690 0x7ff494068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.660+0000 7ff4afe00640 1 --2- 172.21.15.161:0/3373288970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff494066690 0x7ff494068b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff4b81393c0 tx=0x7ff4a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:32.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.664+0000 7ff4af400640 1 -- 172.21.15.161:0/3373288970 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff4a80ebb90 con 0x7ff4b8143c40 2024-05-22T21:14:32.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.816+0000 7ff4be600640 1 -- 172.21.15.161:0/3373288970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4b815c5b0 con 0x7ff4b8143c40 2024-05-22T21:14:32.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.816+0000 7ff4af400640 1 -- 172.21.15.161:0/3373288970 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff4b815c5b0 con 0x7ff4b8143c40 2024-05-22T21:14:32.819 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:14:32.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.816+0000 7ff4be600640 1 -- 172.21.15.161:0/3373288970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff494066690 msgr2=0x7ff494068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:32.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.816+0000 7ff4be600640 1 --2- 172.21.15.161:0/3373288970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff494066690 0x7ff494068b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff4b81393c0 tx=0x7ff4a0008040 comp rx=0 tx=0).stop 2024-05-22T21:14:32.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.820+0000 7ff4be600640 1 -- 172.21.15.161:0/3373288970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b8143c40 msgr2=0x7ff4b8138e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:32.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.820+0000 7ff4be600640 1 --2- 172.21.15.161:0/3373288970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b8143c40 0x7ff4b8138e80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff4a801abd0 tx=0x7ff4a801ad60 comp rx=0 tx=0).stop 2024-05-22T21:14:32.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.820+0000 7ff4be600640 1 -- 172.21.15.161:0/3373288970 shutdown_connections 2024-05-22T21:14:32.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.820+0000 7ff4be600640 1 --2- 172.21.15.161:0/3373288970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff494066690 0x7ff494068b50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.820+0000 7ff4be600640 1 --2- 172.21.15.161:0/3373288970 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4b8144900 0x7ff4b806c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.820+0000 7ff4be600640 1 --2- 172.21.15.161:0/3373288970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4b81444a0 0x7ff4b806c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.820+0000 7ff4be600640 1 --2- 172.21.15.161:0/3373288970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4b8143c40 0x7ff4b8138e80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.820+0000 7ff4be600640 1 -- 172.21.15.161:0/3373288970 >> 172.21.15.161:0/3373288970 conn(0x7ff4b806a2d0 msgr2=0x7ff4b815a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:32.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.820+0000 7ff4be600640 1 -- 172.21.15.161:0/3373288970 shutdown_connections 2024-05-22T21:14:32.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.820+0000 7ff4be600640 1 -- 172.21.15.161:0/3373288970 wait complete. 2024-05-22T21:14:32.841 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:14:32.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.976+0000 7fa5fee00640 1 Processor -- start 2024-05-22T21:14:32.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.976+0000 7fa5fee00640 1 -- start start 2024-05-22T21:14:32.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.976+0000 7fa5fee00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa5f8143c40 0x7fa5f815b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.976+0000 7fa5fee00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa5f81444a0 0x7fa5f806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.976+0000 7fa5fee00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f815efb0 con 0x7fa5f8144900 2024-05-22T21:14:32.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.976+0000 7fa5fee00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f815f290 con 0x7fa5f8143c40 2024-05-22T21:14:32.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.976+0000 7fa5fee00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f815f570 con 0x7fa5f81444a0 2024-05-22T21:14:32.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.976+0000 7fa5f7e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa5f8143c40 0x7fa5f815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.976+0000 7fa5f7e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa5f8143c40 0x7fa5f815b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55762/0 (socket says 172.21.15.161:55762) 2024-05-22T21:14:32.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.976+0000 7fa5f7e00640 1 -- 172.21.15.161:0/2387079649 learned_addr learned my addr 172.21.15.161:0/2387079649 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:32.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.976+0000 7fa5ef400640 1 --2- 172.21.15.161:0/2387079649 >> v2:172.21.15.179:3300/0 conn(0x7fa5f81444a0 0x7fa5f806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.976+0000 7fa5f7e00640 1 -- 172.21.15.161:0/2387079649 >> v2:172.21.15.179:3300/0 conn(0x7fa5f81444a0 msgr2=0x7fa5f806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:32.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.976+0000 7fa5f7e00640 1 --2- 172.21.15.161:0/2387079649 >> v2:172.21.15.179:3300/0 conn(0x7fa5f81444a0 0x7fa5f806c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.976+0000 7fa5f7e00640 1 -- 172.21.15.161:0/2387079649 >> v1:172.21.15.179:6789/0 conn(0x7fa5f8144900 legacy=0x7fa5f80b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:32.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.976+0000 7fa5f7e00640 1 -- 172.21.15.161:0/2387079649 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f815f850 con 0x7fa5f8143c40 2024-05-22T21:14:32.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.980+0000 7fa5f7e00640 1 --2- 172.21.15.161:0/2387079649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f8143c40 0x7fa5f815b600 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80b1c80 tx=0x7fa5e801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c3bb9541947c39 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:32.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.980+0000 7fa5f7400640 1 -- 172.21.15.161:0/2387079649 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80ab5f0 con 0x7fa5f8143c40 2024-05-22T21:14:32.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.980+0000 7fa5f7400640 1 -- 172.21.15.161:0/2387079649 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e80ab750 con 0x7fa5f8143c40 2024-05-22T21:14:32.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.980+0000 7fa5f7400640 1 -- 172.21.15.161:0/2387079649 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80abae0 con 0x7fa5f8143c40 2024-05-22T21:14:32.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.980+0000 7fa5fee00640 1 -- 172.21.15.161:0/2387079649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f8143c40 msgr2=0x7fa5f815b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:32.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.980+0000 7fa5fee00640 1 --2- 172.21.15.161:0/2387079649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f8143c40 0x7fa5f815b600 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80b1c80 tx=0x7fa5e801c4e0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.980+0000 7fa5fee00640 1 -- 172.21.15.161:0/2387079649 shutdown_connections 2024-05-22T21:14:32.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.980+0000 7fa5fee00640 1 --2- 172.21.15.161:0/2387079649 >> v2:172.21.15.179:3300/0 conn(0x7fa5f81444a0 0x7fa5f806c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.980+0000 7fa5fee00640 1 --2- 172.21.15.161:0/2387079649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f8143c40 0x7fa5f815b600 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.980+0000 7fa5fee00640 1 -- 172.21.15.161:0/2387079649 >> 172.21.15.161:0/2387079649 conn(0x7fa5f806a2d0 msgr2=0x7fa5f806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:32.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.980+0000 7fa5fee00640 1 -- 172.21.15.161:0/2387079649 shutdown_connections 2024-05-22T21:14:32.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.980+0000 7fa5fee00640 1 -- 172.21.15.161:0/2387079649 wait complete. 2024-05-22T21:14:32.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fee00640 1 Processor -- start 2024-05-22T21:14:32.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fee00640 1 -- start start 2024-05-22T21:14:32.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa5f8143c40 0x7fa5f8155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5f81444a0 0x7fa5f81395e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fee00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f8144900 0x7fa5f8142b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fee00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f8156430 con 0x7fa5f8144900 2024-05-22T21:14:32.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f8156710 con 0x7fa5f81444a0 2024-05-22T21:14:32.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f81569f0 con 0x7fa5f8143c40 2024-05-22T21:14:32.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f8144900 0x7fa5f8142b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f8144900 0x7fa5f8142b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55768/0 (socket says 172.21.15.161:55768) 2024-05-22T21:14:32.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fce00640 1 -- 172.21.15.161:0/1619674895 learned_addr learned my addr 172.21.15.161:0/1619674895 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:32.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5f7e00640 1 --2- 172.21.15.161:0/1619674895 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa5f8143c40 0x7fa5f8155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5ef400640 1 --2- 172.21.15.161:0/1619674895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5f81444a0 0x7fa5f81395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fce00640 1 -- 172.21.15.161:0/1619674895 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa5f8143c40 msgr2=0x7fa5f8155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:32.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fce00640 1 --2- 172.21.15.161:0/1619674895 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa5f8143c40 0x7fa5f8155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fce00640 1 -- 172.21.15.161:0/1619674895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5f81444a0 msgr2=0x7fa5f81395e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:32.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fce00640 1 --2- 172.21.15.161:0/1619674895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5f81444a0 0x7fa5f81395e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:32.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fce00640 1 -- 172.21.15.161:0/1619674895 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e80094d0 con 0x7fa5f8144900 2024-05-22T21:14:32.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5ef400640 1 --2- 172.21.15.161:0/1619674895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5f81444a0 0x7fa5f81395e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:32.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fce00640 1 --2- 172.21.15.161:0/1619674895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f8144900 0x7fa5f8142b60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000b9e0 tx=0x7fa5f000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:32.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5f4c00640 1 -- 172.21.15.161:0/1619674895 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0003b80 con 0x7fa5f8144900 2024-05-22T21:14:32.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5f4c00640 1 -- 172.21.15.161:0/1619674895 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f0003ce0 con 0x7fa5f8144900 2024-05-22T21:14:32.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fee00640 1 -- 172.21.15.161:0/1619674895 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f8156cd0 con 0x7fa5f8144900 2024-05-22T21:14:32.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5f4c00640 1 -- 172.21.15.161:0/1619674895 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000cb60 con 0x7fa5f8144900 2024-05-22T21:14:32.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.984+0000 7fa5fee00640 1 -- 172.21.15.161:0/1619674895 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f81560e0 con 0x7fa5f8144900 2024-05-22T21:14:32.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.988+0000 7fa5fee00640 1 -- 172.21.15.161:0/1619674895 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f8067220 con 0x7fa5f8144900 2024-05-22T21:14:32.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.988+0000 7fa5f4c00640 1 -- 172.21.15.161:0/1619674895 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa5f000ccc0 con 0x7fa5f8144900 2024-05-22T21:14:32.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.988+0000 7fa5f4c00640 1 --2- 172.21.15.161:0/1619674895 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa5d4066690 0x7fa5d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:32.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.988+0000 7fa5f4c00640 1 -- 172.21.15.161:0/1619674895 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5f0083e30 con 0x7fa5f8144900 2024-05-22T21:14:32.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.988+0000 7fa5f7e00640 1 --2- 172.21.15.161:0/1619674895 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa5d4066690 0x7fa5d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:32.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.988+0000 7fa5f7e00640 1 --2- 172.21.15.161:0/1619674895 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa5d4066690 0x7fa5d4068b50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80af000 tx=0x7fa5e80b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:32.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:32.992+0000 7fa5f4c00640 1 -- 172.21.15.161:0/1619674895 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa5f0051240 con 0x7fa5f8144900 2024-05-22T21:14:33.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.148+0000 7fa5fee00640 1 -- 172.21.15.161:0/1619674895 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa5f80ceaa0 con 0x7fa5f8144900 2024-05-22T21:14:33.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.148+0000 7fa5f4c00640 1 -- 172.21.15.161:0/1619674895 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa5f0054860 con 0x7fa5f8144900 2024-05-22T21:14:33.150 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:14:33.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.148+0000 7fa5fee00640 1 -- 172.21.15.161:0/1619674895 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa5d4066690 msgr2=0x7fa5d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:33.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.148+0000 7fa5fee00640 1 --2- 172.21.15.161:0/1619674895 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa5d4066690 0x7fa5d4068b50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80af000 tx=0x7fa5e80b6040 comp rx=0 tx=0).stop 2024-05-22T21:14:33.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.148+0000 7fa5fee00640 1 -- 172.21.15.161:0/1619674895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f8144900 msgr2=0x7fa5f8142b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:33.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.148+0000 7fa5fee00640 1 --2- 172.21.15.161:0/1619674895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f8144900 0x7fa5f8142b60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000b9e0 tx=0x7fa5f000bea0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.148+0000 7fa5fee00640 1 -- 172.21.15.161:0/1619674895 shutdown_connections 2024-05-22T21:14:33.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.148+0000 7fa5fee00640 1 --2- 172.21.15.161:0/1619674895 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa5d4066690 0x7fa5d4068b50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.148+0000 7fa5fee00640 1 --2- 172.21.15.161:0/1619674895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f8144900 0x7fa5f8142b60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.148+0000 7fa5fee00640 1 --2- 172.21.15.161:0/1619674895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5f81444a0 0x7fa5f81395e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.148+0000 7fa5fee00640 1 --2- 172.21.15.161:0/1619674895 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa5f8143c40 0x7fa5f8155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.148+0000 7fa5fee00640 1 -- 172.21.15.161:0/1619674895 >> 172.21.15.161:0/1619674895 conn(0x7fa5f806a2d0 msgr2=0x7fa5f806a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:33.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.148+0000 7fa5fee00640 1 -- 172.21.15.161:0/1619674895 shutdown_connections 2024-05-22T21:14:33.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.148+0000 7fa5fee00640 1 -- 172.21.15.161:0/1619674895 wait complete. 2024-05-22T21:14:33.171 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:14:33.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.308+0000 7fa573a00640 1 Processor -- start 2024-05-22T21:14:33.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.308+0000 7fa573a00640 1 -- start start 2024-05-22T21:14:33.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.308+0000 7fa573a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa56c1596b0 0x7fa56c071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:33.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.308+0000 7fa573a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa56c06d240 0x7fa56c0b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:33.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.308+0000 7fa573a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56c15d250 con 0x7fa56c071db0 2024-05-22T21:14:33.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.308+0000 7fa573a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56c159a90 con 0x7fa56c06d240 2024-05-22T21:14:33.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.308+0000 7fa573a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c15d510 con 0x7fa56c1596b0 2024-05-22T21:14:33.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.308+0000 7fa571a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa56c06d240 0x7fa56c0b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:33.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.308+0000 7fa571a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa56c06d240 0x7fa56c0b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55774/0 (socket says 172.21.15.161:55774) 2024-05-22T21:14:33.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.308+0000 7fa571a00640 1 -- 172.21.15.161:0/1718614392 learned_addr learned my addr 172.21.15.161:0/1718614392 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:33.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.308+0000 7fa571000640 1 --2- 172.21.15.161:0/1718614392 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa56c1596b0 0x7fa56c071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:33.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.308+0000 7fa571a00640 1 -- 172.21.15.161:0/1718614392 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa56c1596b0 msgr2=0x7fa56c071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:33.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.308+0000 7fa571a00640 1 --2- 172.21.15.161:0/1718614392 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa56c1596b0 0x7fa56c071870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.308+0000 7fa571a00640 1 -- 172.21.15.161:0/1718614392 >> v1:172.21.15.179:6789/0 conn(0x7fa56c071db0 legacy=0x7fa56c06cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:33.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.308+0000 7fa571a00640 1 -- 172.21.15.161:0/1718614392 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c15d7f0 con 0x7fa56c06d240 2024-05-22T21:14:33.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.308+0000 7fa571000640 1 --2- 172.21.15.161:0/1718614392 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa56c1596b0 0x7fa56c071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:33.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.312+0000 7fa571a00640 1 --2- 172.21.15.161:0/1718614392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56c06d240 0x7fa56c0b2100 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa56400d150 tx=0x7fa56401ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90b67243315f6fe server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:33.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.312+0000 7fa56b400640 1 -- 172.21.15.161:0/1718614392 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5640a7b90 con 0x7fa56c06d240 2024-05-22T21:14:33.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.312+0000 7fa56b400640 1 -- 172.21.15.161:0/1718614392 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5640ab070 con 0x7fa56c06d240 2024-05-22T21:14:33.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.312+0000 7fa56b400640 1 -- 172.21.15.161:0/1718614392 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5640b1400 con 0x7fa56c06d240 2024-05-22T21:14:33.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.312+0000 7fa573a00640 1 -- 172.21.15.161:0/1718614392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56c06d240 msgr2=0x7fa56c0b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:33.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.312+0000 7fa573a00640 1 --2- 172.21.15.161:0/1718614392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56c06d240 0x7fa56c0b2100 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa56400d150 tx=0x7fa56401ec30 comp rx=0 tx=0).stop 2024-05-22T21:14:33.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.312+0000 7fa573a00640 1 -- 172.21.15.161:0/1718614392 shutdown_connections 2024-05-22T21:14:33.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.312+0000 7fa573a00640 1 --2- 172.21.15.161:0/1718614392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56c06d240 0x7fa56c0b2100 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.312+0000 7fa573a00640 1 --2- 172.21.15.161:0/1718614392 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa56c1596b0 0x7fa56c071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.312+0000 7fa573a00640 1 -- 172.21.15.161:0/1718614392 >> 172.21.15.161:0/1718614392 conn(0x7fa56c06a2d0 msgr2=0x7fa56c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:33.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.312+0000 7fa573a00640 1 -- 172.21.15.161:0/1718614392 shutdown_connections 2024-05-22T21:14:33.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.312+0000 7fa573a00640 1 -- 172.21.15.161:0/1718614392 wait complete. 2024-05-22T21:14:33.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.312+0000 7fa573a00640 1 Processor -- start 2024-05-22T21:14:33.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa573a00640 1 -- start start 2024-05-22T21:14:33.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa573a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa56c06d240 0x7fa56c1f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:33.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa573a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56c071db0 0x7fa56c14a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:33.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa573a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa56c1596b0 0x7fa56c15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:33.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa573a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c14c510 con 0x7fa56c071db0 2024-05-22T21:14:33.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa573a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c14c080 con 0x7fa56c1596b0 2024-05-22T21:14:33.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa573a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c1f6020 con 0x7fa56c06d240 2024-05-22T21:14:33.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa56be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56c071db0 0x7fa56c14a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:33.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa56be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56c071db0 0x7fa56c14a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55776/0 (socket says 172.21.15.161:55776) 2024-05-22T21:14:33.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa56be00640 1 -- 172.21.15.161:0/1852310320 learned_addr learned my addr 172.21.15.161:0/1852310320 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:33.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa571000640 1 --2- 172.21.15.161:0/1852310320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa56c06d240 0x7fa56c1f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:33.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa571a00640 1 --2- 172.21.15.161:0/1852310320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa56c1596b0 0x7fa56c15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:33.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa56be00640 1 -- 172.21.15.161:0/1852310320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa56c06d240 msgr2=0x7fa56c1f5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:33.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa56be00640 1 --2- 172.21.15.161:0/1852310320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa56c06d240 0x7fa56c1f5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa56be00640 1 -- 172.21.15.161:0/1852310320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa56c1596b0 msgr2=0x7fa56c15a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:33.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa56be00640 1 --2- 172.21.15.161:0/1852310320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa56c1596b0 0x7fa56c15a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa56be00640 1 -- 172.21.15.161:0/1852310320 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56400bd40 con 0x7fa56c071db0 2024-05-22T21:14:33.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa56be00640 1 --2- 172.21.15.161:0/1852310320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56c071db0 0x7fa56c14a9f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00c420 tx=0x7fa55c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:33.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa568c00640 1 -- 172.21.15.161:0/1852310320 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0052d0 con 0x7fa56c071db0 2024-05-22T21:14:33.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa568c00640 1 -- 172.21.15.161:0/1852310320 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c005430 con 0x7fa56c071db0 2024-05-22T21:14:33.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa568c00640 1 -- 172.21.15.161:0/1852310320 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c003820 con 0x7fa56c071db0 2024-05-22T21:14:33.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa573a00640 1 -- 172.21.15.161:0/1852310320 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56c14b470 con 0x7fa56c071db0 2024-05-22T21:14:33.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa573a00640 1 -- 172.21.15.161:0/1852310320 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56c14ba30 con 0x7fa56c071db0 2024-05-22T21:14:33.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.316+0000 7fa573a00640 1 -- 172.21.15.161:0/1852310320 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56c151330 con 0x7fa56c071db0 2024-05-22T21:14:33.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.320+0000 7fa568c00640 1 -- 172.21.15.161:0/1852310320 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa55c003980 con 0x7fa56c071db0 2024-05-22T21:14:33.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.320+0000 7fa568c00640 1 --2- 172.21.15.161:0/1852310320 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa548066690 0x7fa548068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:33.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.320+0000 7fa568c00640 1 -- 172.21.15.161:0/1852310320 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa55c083640 con 0x7fa56c071db0 2024-05-22T21:14:33.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.320+0000 7fa571000640 1 --2- 172.21.15.161:0/1852310320 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa548066690 0x7fa548068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:33.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.320+0000 7fa571000640 1 --2- 172.21.15.161:0/1852310320 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa548066690 0x7fa548068b50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa56c156b20 tx=0x7fa554008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:33.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.324+0000 7fa568c00640 1 -- 172.21.15.161:0/1852310320 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa55c050a50 con 0x7fa56c071db0 2024-05-22T21:14:33.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.480+0000 7fa573a00640 1 -- 172.21.15.161:0/1852310320 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa56c14c1f0 con 0x7fa56c071db0 2024-05-22T21:14:33.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.480+0000 7fa568c00640 1 -- 172.21.15.161:0/1852310320 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa55c054070 con 0x7fa56c071db0 2024-05-22T21:14:33.482 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:14:33.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.480+0000 7fa573a00640 1 -- 172.21.15.161:0/1852310320 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa548066690 msgr2=0x7fa548068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:33.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.480+0000 7fa573a00640 1 --2- 172.21.15.161:0/1852310320 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa548066690 0x7fa548068b50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa56c156b20 tx=0x7fa554008040 comp rx=0 tx=0).stop 2024-05-22T21:14:33.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.480+0000 7fa573a00640 1 -- 172.21.15.161:0/1852310320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56c071db0 msgr2=0x7fa56c14a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:33.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.480+0000 7fa573a00640 1 --2- 172.21.15.161:0/1852310320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56c071db0 0x7fa56c14a9f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00c420 tx=0x7fa55c00c8e0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.480+0000 7fa573a00640 1 -- 172.21.15.161:0/1852310320 shutdown_connections 2024-05-22T21:14:33.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.480+0000 7fa573a00640 1 --2- 172.21.15.161:0/1852310320 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa548066690 0x7fa548068b50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.480+0000 7fa573a00640 1 --2- 172.21.15.161:0/1852310320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa56c1596b0 0x7fa56c15a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.480+0000 7fa573a00640 1 --2- 172.21.15.161:0/1852310320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa56c071db0 0x7fa56c14a9f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.480+0000 7fa573a00640 1 --2- 172.21.15.161:0/1852310320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa56c06d240 0x7fa56c1f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.480+0000 7fa573a00640 1 -- 172.21.15.161:0/1852310320 >> 172.21.15.161:0/1852310320 conn(0x7fa56c06a2d0 msgr2=0x7fa56c14f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:33.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.480+0000 7fa573a00640 1 -- 172.21.15.161:0/1852310320 shutdown_connections 2024-05-22T21:14:33.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.480+0000 7fa573a00640 1 -- 172.21.15.161:0/1852310320 wait complete. 2024-05-22T21:14:33.502 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:14:33.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.636+0000 7fb0ed400640 1 Processor -- start 2024-05-22T21:14:33.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.636+0000 7fb0ed400640 1 -- start start 2024-05-22T21:14:33.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.636+0000 7fb0ed400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0e813cfc0 0x7fb0e81402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:33.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.636+0000 7fb0ed400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb0e813fe50 0x7fb0e806fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:33.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.636+0000 7fb0ed400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e813d3a0 con 0x7fb0e815af60 2024-05-22T21:14:33.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.636+0000 7fb0ed400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e80cd920 con 0x7fb0e813fe50 2024-05-22T21:14:33.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.636+0000 7fb0ed400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e80cdc00 con 0x7fb0e813cfc0 2024-05-22T21:14:33.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.636+0000 7fb0e7200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb0e813fe50 0x7fb0e806fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:33.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.636+0000 7fb0dde00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb0e815af60 0x7fb0e813f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47596/0 (socket says 172.21.15.161:47596) 2024-05-22T21:14:33.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.636+0000 7fb0dde00640 1 -- 172.21.15.161:0/1690345680 learned_addr learned my addr 172.21.15.161:0/1690345680 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:33.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.636+0000 7fb0e6800640 1 --2- 172.21.15.161:0/1690345680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0e813cfc0 0x7fb0e81402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:33.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.636+0000 7fb0e7200640 1 -- 172.21.15.161:0/1690345680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0e813cfc0 msgr2=0x7fb0e81402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:33.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.636+0000 7fb0e7200640 1 --2- 172.21.15.161:0/1690345680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0e813cfc0 0x7fb0e81402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.636+0000 7fb0e7200640 1 -- 172.21.15.161:0/1690345680 >> v1:172.21.15.161:6789/0 conn(0x7fb0e815af60 legacy=0x7fb0e813f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:33.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.636+0000 7fb0e7200640 1 -- 172.21.15.161:0/1690345680 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e80cdee0 con 0x7fb0e813fe50 2024-05-22T21:14:33.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.636+0000 7fb0e5e00640 1 -- 172.21.15.161:0/1690345680 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181125876 0 0) 0x7fb0e813d3a0 con 0x7fb0e815af60 2024-05-22T21:14:33.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.640+0000 7fb0e7200640 1 --2- 172.21.15.161:0/1690345680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0e813fe50 0x7fb0e806fce0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00097c0 tx=0x7fb0e001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ded70bca754b494 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:33.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.640+0000 7fb0e5e00640 1 -- 172.21.15.161:0/1690345680 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00ab600 con 0x7fb0e813fe50 2024-05-22T21:14:33.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.640+0000 7fb0e5e00640 1 -- 172.21.15.161:0/1690345680 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e00a7050 con 0x7fb0e813fe50 2024-05-22T21:14:33.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.640+0000 7fb0e5e00640 1 -- 172.21.15.161:0/1690345680 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00ab980 con 0x7fb0e813fe50 2024-05-22T21:14:33.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.640+0000 7fb0ed400640 1 -- 172.21.15.161:0/1690345680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0e813fe50 msgr2=0x7fb0e806fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:33.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.640+0000 7fb0ed400640 1 --2- 172.21.15.161:0/1690345680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0e813fe50 0x7fb0e806fce0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00097c0 tx=0x7fb0e001c3e0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.640+0000 7fb0ed400640 1 -- 172.21.15.161:0/1690345680 shutdown_connections 2024-05-22T21:14:33.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.640+0000 7fb0ed400640 1 --2- 172.21.15.161:0/1690345680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0e813fe50 0x7fb0e806fce0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.640+0000 7fb0ed400640 1 --2- 172.21.15.161:0/1690345680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0e813cfc0 0x7fb0e81402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.640+0000 7fb0ed400640 1 -- 172.21.15.161:0/1690345680 >> 172.21.15.161:0/1690345680 conn(0x7fb0e806a2d0 msgr2=0x7fb0e806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:33.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.640+0000 7fb0ed400640 1 -- 172.21.15.161:0/1690345680 shutdown_connections 2024-05-22T21:14:33.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.640+0000 7fb0ed400640 1 -- 172.21.15.161:0/1690345680 wait complete. 2024-05-22T21:14:33.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0ed400640 1 Processor -- start 2024-05-22T21:14:33.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0ed400640 1 -- start start 2024-05-22T21:14:33.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0ed400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0e813cfc0 0x7fb0e813df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:33.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0ed400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0e813fe50 0x7fb0e813a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:33.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0ed400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0e815af60 0x7fb0e80b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:33.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0ed400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8140920 con 0x7fb0e815af60 2024-05-22T21:14:33.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0ed400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8140610 con 0x7fb0e813fe50 2024-05-22T21:14:33.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0ed400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e813e490 con 0x7fb0e813cfc0 2024-05-22T21:14:33.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0e7200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0e815af60 0x7fb0e80b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:33.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0e7200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0e815af60 0x7fb0e80b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55794/0 (socket says 172.21.15.161:55794) 2024-05-22T21:14:33.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0e7200640 1 -- 172.21.15.161:0/3438828520 learned_addr learned my addr 172.21.15.161:0/3438828520 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:33.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0e7200640 1 -- 172.21.15.161:0/3438828520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0e813cfc0 msgr2=0x7fb0e813df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:33.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0e6800640 1 --2- 172.21.15.161:0/3438828520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0e813cfc0 0x7fb0e813df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:33.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0dde00640 1 --2- 172.21.15.161:0/3438828520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0e813fe50 0x7fb0e813a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:33.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0e7200640 1 --2- 172.21.15.161:0/3438828520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0e813cfc0 0x7fb0e813df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0e7200640 1 -- 172.21.15.161:0/3438828520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0e813fe50 msgr2=0x7fb0e813a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:33.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0e7200640 1 --2- 172.21.15.161:0/3438828520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0e813fe50 0x7fb0e813a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0e7200640 1 -- 172.21.15.161:0/3438828520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e0009420 con 0x7fb0e815af60 2024-05-22T21:14:33.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0e6800640 1 --2- 172.21.15.161:0/3438828520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0e813cfc0 0x7fb0e813df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:33.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0dde00640 1 --2- 172.21.15.161:0/3438828520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0e813fe50 0x7fb0e813a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:33.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0e7200640 1 --2- 172.21.15.161:0/3438828520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0e815af60 0x7fb0e80b25b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00097c0 tx=0x7fb0e0019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:33.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0df400640 1 -- 172.21.15.161:0/3438828520 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00b4430 con 0x7fb0e815af60 2024-05-22T21:14:33.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0df400640 1 -- 172.21.15.161:0/3438828520 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e00b4590 con 0x7fb0e815af60 2024-05-22T21:14:33.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0df400640 1 -- 172.21.15.161:0/3438828520 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00b4920 con 0x7fb0e815af60 2024-05-22T21:14:33.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0ed400640 1 -- 172.21.15.161:0/3438828520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e813e710 con 0x7fb0e815af60 2024-05-22T21:14:33.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.644+0000 7fb0ed400640 1 -- 172.21.15.161:0/3438828520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e813ecd0 con 0x7fb0e815af60 2024-05-22T21:14:33.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.648+0000 7fb0ed400640 1 -- 172.21.15.161:0/3438828520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e8140ad0 con 0x7fb0e815af60 2024-05-22T21:14:33.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.648+0000 7fb0df400640 1 -- 172.21.15.161:0/3438828520 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb0e00b2040 con 0x7fb0e815af60 2024-05-22T21:14:33.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.648+0000 7fb0df400640 1 --2- 172.21.15.161:0/3438828520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0c4066690 0x7fb0c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:33.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.648+0000 7fb0df400640 1 -- 172.21.15.161:0/3438828520 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0e011e150 con 0x7fb0e815af60 2024-05-22T21:14:33.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.648+0000 7fb0e6800640 1 --2- 172.21.15.161:0/3438828520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0c4066690 0x7fb0c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:33.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.648+0000 7fb0e6800640 1 --2- 172.21.15.161:0/3438828520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0c4066690 0x7fb0c4068b50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8007520 tx=0x7fb0d800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:33.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.652+0000 7fb0df400640 1 -- 172.21.15.161:0/3438828520 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb0e8140ad0 con 0x7fb0e815af60 2024-05-22T21:14:33.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.816+0000 7fb0ed400640 1 -- 172.21.15.161:0/3438828520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb0e8140ad0 con 0x7fb0e815af60 2024-05-22T21:14:33.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.816+0000 7fb0df400640 1 -- 172.21.15.161:0/3438828520 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb0e8140ad0 con 0x7fb0e815af60 2024-05-22T21:14:33.818 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:14:33.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.816+0000 7fb0ed400640 1 -- 172.21.15.161:0/3438828520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0c4066690 msgr2=0x7fb0c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:33.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.816+0000 7fb0ed400640 1 --2- 172.21.15.161:0/3438828520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0c4066690 0x7fb0c4068b50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8007520 tx=0x7fb0d800b040 comp rx=0 tx=0).stop 2024-05-22T21:14:33.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.816+0000 7fb0ed400640 1 -- 172.21.15.161:0/3438828520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0e815af60 msgr2=0x7fb0e80b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:33.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.816+0000 7fb0ed400640 1 --2- 172.21.15.161:0/3438828520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0e815af60 0x7fb0e80b25b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00097c0 tx=0x7fb0e0019180 comp rx=0 tx=0).stop 2024-05-22T21:14:33.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.816+0000 7fb0ed400640 1 -- 172.21.15.161:0/3438828520 shutdown_connections 2024-05-22T21:14:33.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.816+0000 7fb0ed400640 1 --2- 172.21.15.161:0/3438828520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0c4066690 0x7fb0c4068b50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.816+0000 7fb0ed400640 1 --2- 172.21.15.161:0/3438828520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0e815af60 0x7fb0e80b25b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.816+0000 7fb0ed400640 1 --2- 172.21.15.161:0/3438828520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0e813fe50 0x7fb0e813a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.816+0000 7fb0ed400640 1 --2- 172.21.15.161:0/3438828520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0e813cfc0 0x7fb0e813df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:33.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.816+0000 7fb0ed400640 1 -- 172.21.15.161:0/3438828520 >> 172.21.15.161:0/3438828520 conn(0x7fb0e806a2d0 msgr2=0x7fb0e806a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:33.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.816+0000 7fb0ed400640 1 -- 172.21.15.161:0/3438828520 shutdown_connections 2024-05-22T21:14:33.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:33.816+0000 7fb0ed400640 1 -- 172.21.15.161:0/3438828520 wait complete. 2024-05-22T21:14:33.839 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-22T21:14:33.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:33.839 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-22T21:14:33.891 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:14:33.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:33.891 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-22T21:14:33.903 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:14:33.903 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:33.912 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:14:33.912 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:33.967 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:34.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.100+0000 7f98a5400640 1 Processor -- start 2024-05-22T21:14:34.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.100+0000 7f98a5400640 1 -- start start 2024-05-22T21:14:34.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f98a5400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98a015dd50 0x7f98a006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f98a5400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f98a015e130 0x7f98a015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f98a5400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98a015d410 0x7f98a015a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f98a5400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a0071670 con 0x7f98a015e130 2024-05-22T21:14:34.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f98a5400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a015d8a0 con 0x7f98a015d410 2024-05-22T21:14:34.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f98a5400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a006de40 con 0x7f98a015dd50 2024-05-22T21:14:34.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f9895e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f98a015e130 0x7f98a015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f9895e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f98a015e130 0x7f98a015ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55810/0 (socket says 172.21.15.161:55810) 2024-05-22T21:14:34.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f9895e00640 1 -- 172.21.15.161:0/3297653207 learned_addr learned my addr 172.21.15.161:0/3297653207 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:34.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f9895e00640 1 -- 172.21.15.161:0/3297653207 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98a015dd50 msgr2=0x7f98a006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:34.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f9895e00640 1 --2- 172.21.15.161:0/3297653207 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98a015dd50 0x7f98a006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f9895e00640 1 -- 172.21.15.161:0/3297653207 >> v2:172.21.15.179:3300/0 conn(0x7f98a015d410 msgr2=0x7f98a015a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:34.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f989f200640 1 --2- 172.21.15.161:0/3297653207 >> v2:172.21.15.179:3300/0 conn(0x7f98a015d410 0x7f98a015a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f9895e00640 1 --2- 172.21.15.161:0/3297653207 >> v2:172.21.15.179:3300/0 conn(0x7f98a015d410 0x7f98a015a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f9895e00640 1 -- 172.21.15.161:0/3297653207 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a0070a60 con 0x7f98a015e130 2024-05-22T21:14:34.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f989f200640 1 --2- 172.21.15.161:0/3297653207 >> v2:172.21.15.179:3300/0 conn(0x7f98a015d410 0x7f98a015a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:34.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f9895e00640 1 --2- 172.21.15.161:0/3297653207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98a015e130 0x7f98a015ced0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f98a00721d0 tx=0x7f988801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b831f20d4d4faa88 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:34.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.104+0000 7f989de00640 1 -- 172.21.15.161:0/3297653207 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98880ab620 con 0x7f98a015e130 2024-05-22T21:14:34.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f989de00640 1 -- 172.21.15.161:0/3297653207 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98880ab780 con 0x7f98a015e130 2024-05-22T21:14:34.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f989de00640 1 -- 172.21.15.161:0/3297653207 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98880abb10 con 0x7f98a015e130 2024-05-22T21:14:34.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 -- 172.21.15.161:0/3297653207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98a015e130 msgr2=0x7f98a015ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 --2- 172.21.15.161:0/3297653207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98a015e130 0x7f98a015ced0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f98a00721d0 tx=0x7f988801c460 comp rx=0 tx=0).stop 2024-05-22T21:14:34.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 -- 172.21.15.161:0/3297653207 shutdown_connections 2024-05-22T21:14:34.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 --2- 172.21.15.161:0/3297653207 >> v2:172.21.15.179:3300/0 conn(0x7f98a015d410 0x7f98a015a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 --2- 172.21.15.161:0/3297653207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98a015e130 0x7f98a015ced0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 --2- 172.21.15.161:0/3297653207 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98a015dd50 0x7f98a006d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 -- 172.21.15.161:0/3297653207 >> 172.21.15.161:0/3297653207 conn(0x7f98a00b0bc0 msgr2=0x7f98a00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:34.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 -- 172.21.15.161:0/3297653207 shutdown_connections 2024-05-22T21:14:34.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 -- 172.21.15.161:0/3297653207 wait complete. 2024-05-22T21:14:34.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 Processor -- start 2024-05-22T21:14:34.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 -- start start 2024-05-22T21:14:34.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98a015d410 0x7f98a00c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a015dd50 0x7f98a0154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98a015e130 0x7f98a00b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00c4b20 con 0x7f98a015d410 2024-05-22T21:14:34.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a0158a00 con 0x7f98a015dd50 2024-05-22T21:14:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f98a5400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a0158ce0 con 0x7f98a015e130 2024-05-22T21:14:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f989e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98a015d410 0x7f98a00c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f989e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98a015d410 0x7f98a00c45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55826/0 (socket says 172.21.15.161:55826) 2024-05-22T21:14:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f989e800640 1 -- 172.21.15.161:0/1398142986 learned_addr learned my addr 172.21.15.161:0/1398142986 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f9895e00640 1 --2- 172.21.15.161:0/1398142986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a015dd50 0x7f98a0154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f989f200640 1 --2- 172.21.15.161:0/1398142986 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98a015e130 0x7f98a00b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f989e800640 1 -- 172.21.15.161:0/1398142986 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98a015e130 msgr2=0x7f98a00b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f989e800640 1 --2- 172.21.15.161:0/1398142986 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98a015e130 0x7f98a00b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f989e800640 1 -- 172.21.15.161:0/1398142986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a015dd50 msgr2=0x7f98a0154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f989e800640 1 --2- 172.21.15.161:0/1398142986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a015dd50 0x7f98a0154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.108+0000 7f989e800640 1 -- 172.21.15.161:0/1398142986 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98880094d0 con 0x7f98a015d410 2024-05-22T21:14:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.112+0000 7f989e800640 1 --2- 172.21.15.161:0/1398142986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98a015d410 0x7f98a00c45e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f989000b550 tx=0x7f989000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:34.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.112+0000 7f9897400640 1 -- 172.21.15.161:0/1398142986 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890010870 con 0x7f98a015d410 2024-05-22T21:14:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.112+0000 7f9897400640 1 -- 172.21.15.161:0/1398142986 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98900109f0 con 0x7f98a015d410 2024-05-22T21:14:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.112+0000 7f9897400640 1 -- 172.21.15.161:0/1398142986 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890010da0 con 0x7f98a015d410 2024-05-22T21:14:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.112+0000 7f98a5400640 1 -- 172.21.15.161:0/1398142986 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a0158fc0 con 0x7f98a015d410 2024-05-22T21:14:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.112+0000 7f98a5400640 1 -- 172.21.15.161:0/1398142986 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a0159580 con 0x7f98a015d410 2024-05-22T21:14:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.112+0000 7f98a5400640 1 -- 172.21.15.161:0/1398142986 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a00706b0 con 0x7f98a015d410 2024-05-22T21:14:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.112+0000 7f9897400640 1 -- 172.21.15.161:0/1398142986 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f989000f3c0 con 0x7f98a015d410 2024-05-22T21:14:34.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.112+0000 7f9897400640 1 --2- 172.21.15.161:0/1398142986 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f987c066690 0x7f987c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.112+0000 7f9897400640 1 -- 172.21.15.161:0/1398142986 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9890084190 con 0x7f98a015d410 2024-05-22T21:14:34.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.112+0000 7f9895e00640 1 --2- 172.21.15.161:0/1398142986 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f987c066690 0x7f987c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.116+0000 7f9895e00640 1 --2- 172.21.15.161:0/1398142986 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f987c066690 0x7f987c068b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f98880af000 tx=0x7f98880b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:34.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.116+0000 7f9897400640 1 -- 172.21.15.161:0/1398142986 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9890051510 con 0x7f98a015d410 2024-05-22T21:14:34.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.276+0000 7f98a5400640 1 -- 172.21.15.161:0/1398142986 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98a0067220 con 0x7f98a015d410 2024-05-22T21:14:34.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.276+0000 7f9897400640 1 -- 172.21.15.161:0/1398142986 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16149 (secure 0 0 0) 0x7f9890054b30 con 0x7f98a015d410 2024-05-22T21:14:34.280 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:34.281 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":35,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:14:30.106879+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:16.938769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:18.054579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.280+0000 7f98a5400640 1 -- 172.21.15.161:0/1398142986 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f987c066690 msgr2=0x7f987c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.280+0000 7f98a5400640 1 --2- 172.21.15.161:0/1398142986 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f987c066690 0x7f987c068b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f98880af000 tx=0x7f98880b6040 comp rx=0 tx=0).stop 2024-05-22T21:14:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.280+0000 7f98a5400640 1 -- 172.21.15.161:0/1398142986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98a015d410 msgr2=0x7f98a00c45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.280+0000 7f98a5400640 1 --2- 172.21.15.161:0/1398142986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98a015d410 0x7f98a00c45e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f989000b550 tx=0x7f989000ba10 comp rx=0 tx=0).stop 2024-05-22T21:14:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.280+0000 7f98a5400640 1 -- 172.21.15.161:0/1398142986 shutdown_connections 2024-05-22T21:14:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.280+0000 7f98a5400640 1 --2- 172.21.15.161:0/1398142986 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f987c066690 0x7f987c068b50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.280+0000 7f98a5400640 1 --2- 172.21.15.161:0/1398142986 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98a015e130 0x7f98a00b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.280+0000 7f98a5400640 1 --2- 172.21.15.161:0/1398142986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a015dd50 0x7f98a0154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.280+0000 7f98a5400640 1 --2- 172.21.15.161:0/1398142986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98a015d410 0x7f98a00c45e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.280+0000 7f98a5400640 1 -- 172.21.15.161:0/1398142986 >> 172.21.15.161:0/1398142986 conn(0x7f98a00b0bc0 msgr2=0x7f98a00aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.280+0000 7f98a5400640 1 -- 172.21.15.161:0/1398142986 shutdown_connections 2024-05-22T21:14:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.280+0000 7f98a5400640 1 -- 172.21.15.161:0/1398142986 wait complete. 2024-05-22T21:14:34.301 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:14:16.938769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:14:18.054579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:14:34.301 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:14:34.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.436+0000 7fd4bf000640 1 Processor -- start 2024-05-22T21:14:34.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.436+0000 7fd4bf000640 1 -- start start 2024-05-22T21:14:34.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.440+0000 7fd4bf000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd4b81596b0 0x7fd4b8071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.440+0000 7fd4bf000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd4b8071db0 0x7fd4b806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.440+0000 7fd4bf000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b806d480 con 0x7fd4b806d070 2024-05-22T21:14:34.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.440+0000 7fd4bf000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b80b3110 con 0x7fd4b8071db0 2024-05-22T21:14:34.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.440+0000 7fd4bf000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b815d250 con 0x7fd4b81596b0 2024-05-22T21:14:34.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.440+0000 7fd4af400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd4b8071db0 0x7fd4b806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.440+0000 7fd4bd000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd4b806d070 0x7fd4b806fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47612/0 (socket says 172.21.15.161:47612) 2024-05-22T21:14:34.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.440+0000 7fd4bd000640 1 -- 172.21.15.161:0/2107042648 learned_addr learned my addr 172.21.15.161:0/2107042648 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:34.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.440+0000 7fd4af400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd4b8071db0 0x7fd4b806cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55828/0 (socket says 172.21.15.161:55828) 2024-05-22T21:14:34.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.440+0000 7fd4b7e00640 1 --2- 172.21.15.161:0/2107042648 >> v2:172.21.15.179:3300/0 conn(0x7fd4b81596b0 0x7fd4b8071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.440+0000 7fd4af400640 1 -- 172.21.15.161:0/2107042648 >> v2:172.21.15.179:3300/0 conn(0x7fd4b81596b0 msgr2=0x7fd4b8071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:34.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.440+0000 7fd4af400640 1 --2- 172.21.15.161:0/2107042648 >> v2:172.21.15.179:3300/0 conn(0x7fd4b81596b0 0x7fd4b8071870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.440+0000 7fd4af400640 1 -- 172.21.15.161:0/2107042648 >> v1:172.21.15.161:6789/0 conn(0x7fd4b806d070 legacy=0x7fd4b806fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.440+0000 7fd4af400640 1 -- 172.21.15.161:0/2107042648 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b8159a90 con 0x7fd4b8071db0 2024-05-22T21:14:34.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.440+0000 7fd4b7400640 1 -- 172.21.15.161:0/2107042648 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451649231 0 0) 0x7fd4b806d480 con 0x7fd4b806d070 2024-05-22T21:14:34.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.440+0000 7fd4b7e00640 1 --2- 172.21.15.161:0/2107042648 >> v2:172.21.15.179:3300/0 conn(0x7fd4b81596b0 0x7fd4b8071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:34.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4af400640 1 --2- 172.21.15.161:0/2107042648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4b8071db0 0x7fd4b806cb30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80724b0 tx=0x7fd4a001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402526c90d97597f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:34.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4b7400640 1 -- 172.21.15.161:0/2107042648 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a00ab600 con 0x7fd4b8071db0 2024-05-22T21:14:34.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4b7400640 1 -- 172.21.15.161:0/2107042648 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a00a7050 con 0x7fd4b8071db0 2024-05-22T21:14:34.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4b7400640 1 -- 172.21.15.161:0/2107042648 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a00ab980 con 0x7fd4b8071db0 2024-05-22T21:14:34.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4bf000640 1 -- 172.21.15.161:0/2107042648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4b8071db0 msgr2=0x7fd4b806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4bf000640 1 --2- 172.21.15.161:0/2107042648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4b8071db0 0x7fd4b806cb30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80724b0 tx=0x7fd4a001c480 comp rx=0 tx=0).stop 2024-05-22T21:14:34.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4bf000640 1 -- 172.21.15.161:0/2107042648 shutdown_connections 2024-05-22T21:14:34.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4bf000640 1 --2- 172.21.15.161:0/2107042648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4b8071db0 0x7fd4b806cb30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4bf000640 1 --2- 172.21.15.161:0/2107042648 >> v2:172.21.15.179:3300/0 conn(0x7fd4b81596b0 0x7fd4b8071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4bf000640 1 -- 172.21.15.161:0/2107042648 >> 172.21.15.161:0/2107042648 conn(0x7fd4b806a2d0 msgr2=0x7fd4b806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:34.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4bf000640 1 -- 172.21.15.161:0/2107042648 shutdown_connections 2024-05-22T21:14:34.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4bf000640 1 -- 172.21.15.161:0/2107042648 wait complete. 2024-05-22T21:14:34.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4bf000640 1 Processor -- start 2024-05-22T21:14:34.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4bf000640 1 -- start start 2024-05-22T21:14:34.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4bf000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4b806d070 0x7fd4b8144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4bf000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd4b8071db0 0x7fd4b80ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4bf000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4b81596b0 0x7fd4b80adb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4bf000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b80ac8b0 con 0x7fd4b806d070 2024-05-22T21:14:34.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4bf000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b80cd910 con 0x7fd4b81596b0 2024-05-22T21:14:34.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4bf000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b80cdbf0 con 0x7fd4b8071db0 2024-05-22T21:14:34.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4b7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4b806d070 0x7fd4b8144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4b7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4b806d070 0x7fd4b8144360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55838/0 (socket says 172.21.15.161:55838) 2024-05-22T21:14:34.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.444+0000 7fd4b7e00640 1 -- 172.21.15.161:0/1047925381 learned_addr learned my addr 172.21.15.161:0/1047925381 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:34.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4af400640 1 --2- 172.21.15.161:0/1047925381 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd4b8071db0 0x7fd4b80ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4bd000640 1 --2- 172.21.15.161:0/1047925381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4b81596b0 0x7fd4b80adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4b7e00640 1 -- 172.21.15.161:0/1047925381 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd4b8071db0 msgr2=0x7fd4b80ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4b7e00640 1 --2- 172.21.15.161:0/1047925381 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd4b8071db0 0x7fd4b80ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4b7e00640 1 -- 172.21.15.161:0/1047925381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4b81596b0 msgr2=0x7fd4b80adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4b7e00640 1 --2- 172.21.15.161:0/1047925381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4b81596b0 0x7fd4b80adb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4b7e00640 1 -- 172.21.15.161:0/1047925381 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a0009420 con 0x7fd4b806d070 2024-05-22T21:14:34.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4bd000640 1 --2- 172.21.15.161:0/1047925381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4b81596b0 0x7fd4b80adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:34.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4b7e00640 1 --2- 172.21.15.161:0/1047925381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4b806d070 0x7fd4b8144360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800c7f0 tx=0x7fd4a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:34.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4b4c00640 1 -- 172.21.15.161:0/1047925381 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8007820 con 0x7fd4b806d070 2024-05-22T21:14:34.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4b4c00640 1 -- 172.21.15.161:0/1047925381 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a8007980 con 0x7fd4b806d070 2024-05-22T21:14:34.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4bf000640 1 -- 172.21.15.161:0/1047925381 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b80cded0 con 0x7fd4b806d070 2024-05-22T21:14:34.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4b4c00640 1 -- 172.21.15.161:0/1047925381 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8007d30 con 0x7fd4b806d070 2024-05-22T21:14:34.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4bf000640 1 -- 172.21.15.161:0/1047925381 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b81448a0 con 0x7fd4b806d070 2024-05-22T21:14:34.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4bf000640 1 -- 172.21.15.161:0/1047925381 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b8067220 con 0x7fd4b806d070 2024-05-22T21:14:34.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4b4c00640 1 -- 172.21.15.161:0/1047925381 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd4a8024480 con 0x7fd4b806d070 2024-05-22T21:14:34.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4b4c00640 1 --2- 172.21.15.161:0/1047925381 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd494066690 0x7fd494068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.448+0000 7fd4b4c00640 1 -- 172.21.15.161:0/1047925381 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4a80848e0 con 0x7fd4b806d070 2024-05-22T21:14:34.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.452+0000 7fd4af400640 1 --2- 172.21.15.161:0/1047925381 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd494066690 0x7fd494068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.452+0000 7fd4af400640 1 --2- 172.21.15.161:0/1047925381 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd494066690 0x7fd494068b50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00ad000 tx=0x7fd4a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:34.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.452+0000 7fd4b4c00640 1 -- 172.21.15.161:0/1047925381 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd4a8051cc0 con 0x7fd4b806d070 2024-05-22T21:14:34.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.604+0000 7fd4bf000640 1 -- 172.21.15.161:0/1047925381 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4b80aff00 con 0x7fd4b806d070 2024-05-22T21:14:34.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.608+0000 7fd4b4c00640 1 -- 172.21.15.161:0/1047925381 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4a80552e0 con 0x7fd4b806d070 2024-05-22T21:14:34.609 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:14:34.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.608+0000 7fd4bf000640 1 -- 172.21.15.161:0/1047925381 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd494066690 msgr2=0x7fd494068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.608+0000 7fd4bf000640 1 --2- 172.21.15.161:0/1047925381 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd494066690 0x7fd494068b50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00ad000 tx=0x7fd4a0008040 comp rx=0 tx=0).stop 2024-05-22T21:14:34.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.608+0000 7fd4bf000640 1 -- 172.21.15.161:0/1047925381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4b806d070 msgr2=0x7fd4b8144360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.608+0000 7fd4bf000640 1 --2- 172.21.15.161:0/1047925381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4b806d070 0x7fd4b8144360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800c7f0 tx=0x7fd4a800ccb0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.608+0000 7fd4bf000640 1 -- 172.21.15.161:0/1047925381 shutdown_connections 2024-05-22T21:14:34.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.608+0000 7fd4bf000640 1 --2- 172.21.15.161:0/1047925381 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd494066690 0x7fd494068b50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.608+0000 7fd4bf000640 1 --2- 172.21.15.161:0/1047925381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4b81596b0 0x7fd4b80adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.608+0000 7fd4bf000640 1 --2- 172.21.15.161:0/1047925381 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd4b8071db0 0x7fd4b80ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.608+0000 7fd4bf000640 1 --2- 172.21.15.161:0/1047925381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4b806d070 0x7fd4b8144360 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.608+0000 7fd4bf000640 1 -- 172.21.15.161:0/1047925381 >> 172.21.15.161:0/1047925381 conn(0x7fd4b806a2d0 msgr2=0x7fd4b814efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:34.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.608+0000 7fd4bf000640 1 -- 172.21.15.161:0/1047925381 shutdown_connections 2024-05-22T21:14:34.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.608+0000 7fd4bf000640 1 -- 172.21.15.161:0/1047925381 wait complete. 2024-05-22T21:14:34.630 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:14:34.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.764+0000 7fa233000640 1 Processor -- start 2024-05-22T21:14:34.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.764+0000 7fa233000640 1 -- start start 2024-05-22T21:14:34.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa233000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa22c143c40 0x7fa22c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa233000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa22c15efb0 con 0x7fa22c06c9b0 2024-05-22T21:14:34.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa233000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa22c15f290 con 0x7fa22c1444a0 2024-05-22T21:14:34.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa233000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c15f570 con 0x7fa22c143c40 2024-05-22T21:14:34.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa231000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa22c06c9b0 0x7fa22c0b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47614/0 (socket says 172.21.15.161:47614) 2024-05-22T21:14:34.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa231000640 1 -- 172.21.15.161:0/1982178782 learned_addr learned my addr 172.21.15.161:0/1982178782 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:34.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22be00640 1 --2- 172.21.15.161:0/1982178782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa22c143c40 0x7fa22c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22b400640 1 -- 172.21.15.161:0/1982178782 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406184068 0 0) 0x7fa22c15efb0 con 0x7fa22c06c9b0 2024-05-22T21:14:34.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22b400640 1 -- 172.21.15.161:0/1982178782 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa210003530 con 0x7fa22c06c9b0 2024-05-22T21:14:34.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22b400640 1 -- 172.21.15.161:0/1982178782 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2761737063 0 0) 0x7fa210003530 con 0x7fa22c06c9b0 2024-05-22T21:14:34.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22b400640 1 -- 172.21.15.161:0/1982178782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa22c143c40 msgr2=0x7fa22c15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:34.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22b400640 1 --2- 172.21.15.161:0/1982178782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa22c143c40 0x7fa22c15b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22b400640 1 -- 172.21.15.161:0/1982178782 >> v1:172.21.15.179:6789/0 conn(0x7fa22c1444a0 legacy=0x7fa22c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22b400640 1 -- 172.21.15.161:0/1982178782 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c15f850 con 0x7fa22c06c9b0 2024-05-22T21:14:34.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22b400640 1 -- 172.21.15.161:0/1982178782 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fa224001680 con 0x7fa22c06c9b0 2024-05-22T21:14:34.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22b400640 1 -- 172.21.15.161:0/1982178782 >> v1:172.21.15.161:6789/0 conn(0x7fa22c06c9b0 legacy=0x7fa22c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22b400640 1 --2- 172.21.15.161:0/1982178782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa22c143c40 0x7fa2100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22b400640 1 --2- 172.21.15.161:0/1982178782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa210003ef0 0x7fa2100047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22b400640 1 --2- 172.21.15.161:0/1982178782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa210004d10 0x7fa210005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22b400640 1 -- 172.21.15.161:0/1982178782 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c15f570 con 0x7fa210003ef0 2024-05-22T21:14:34.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22b400640 1 -- 172.21.15.161:0/1982178782 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21000b9f0 con 0x7fa210004d10 2024-05-22T21:14:34.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22b400640 1 -- 172.21.15.161:0/1982178782 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21000bbb0 con 0x7fa22c143c40 2024-05-22T21:14:34.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa223400640 1 --2- 172.21.15.161:0/1982178782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa210003ef0 0x7fa2100047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa22be00640 1 --2- 172.21.15.161:0/1982178782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa22c143c40 0x7fa2100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa231000640 1 --2- 172.21.15.161:0/1982178782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa210004d10 0x7fa210005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa223400640 1 -- 172.21.15.161:0/1982178782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa22c143c40 msgr2=0x7fa2100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:34.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa223400640 1 --2- 172.21.15.161:0/1982178782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa22c143c40 0x7fa2100039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa223400640 1 -- 172.21.15.161:0/1982178782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa210004d10 msgr2=0x7fa210005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:34.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa223400640 1 --2- 172.21.15.161:0/1982178782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa210004d10 0x7fa210005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.768+0000 7fa223400640 1 -- 172.21.15.161:0/1982178782 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa210003530 con 0x7fa210003ef0 2024-05-22T21:14:34.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.772+0000 7fa223400640 1 --2- 172.21.15.161:0/1982178782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa210003ef0 0x7fa2100047d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa214001860 tx=0x7fa21401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8e78d0130314bf server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:34.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.772+0000 7fa22b400640 1 -- 172.21.15.161:0/1982178782 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2140ab600 con 0x7fa210003ef0 2024-05-22T21:14:34.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.772+0000 7fa22b400640 1 -- 172.21.15.161:0/1982178782 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2140a7050 con 0x7fa210003ef0 2024-05-22T21:14:34.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.772+0000 7fa22b400640 1 -- 172.21.15.161:0/1982178782 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2140ab9e0 con 0x7fa210003ef0 2024-05-22T21:14:34.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.772+0000 7fa233000640 1 -- 172.21.15.161:0/1982178782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa210003ef0 msgr2=0x7fa2100047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.772+0000 7fa233000640 1 --2- 172.21.15.161:0/1982178782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa210003ef0 0x7fa2100047d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa214001860 tx=0x7fa21401c440 comp rx=0 tx=0).stop 2024-05-22T21:14:34.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.772+0000 7fa231000640 1 -- 172.21.15.161:0/1982178782 reap_dead start 2024-05-22T21:14:34.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.772+0000 7fa233000640 1 -- 172.21.15.161:0/1982178782 shutdown_connections 2024-05-22T21:14:34.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.772+0000 7fa233000640 1 -- 172.21.15.161:0/1982178782 >> 172.21.15.161:0/1982178782 conn(0x7fa22c06a2d0 msgr2=0x7fa22c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:34.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.772+0000 7fa233000640 1 -- 172.21.15.161:0/1982178782 shutdown_connections 2024-05-22T21:14:34.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.772+0000 7fa233000640 1 -- 172.21.15.161:0/1982178782 wait complete. 2024-05-22T21:14:34.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa233000640 1 Processor -- start 2024-05-22T21:14:34.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa233000640 1 -- start start 2024-05-22T21:14:34.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa233000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa210003ef0 0x7fa22c154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa233000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c139570 0x7fa22c138400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa233000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa22c156410 0x7fa22c138940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa233000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c1624d0 con 0x7fa22c139570 2024-05-22T21:14:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa233000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c138e80 con 0x7fa210003ef0 2024-05-22T21:14:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa233000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c139320 con 0x7fa22c156410 2024-05-22T21:14:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa223400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c139570 0x7fa22c138400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa22be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa210003ef0 0x7fa22c154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa223400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c139570 0x7fa22c138400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55858/0 (socket says 172.21.15.161:55858) 2024-05-22T21:14:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa223400640 1 -- 172.21.15.161:0/1953372569 learned_addr learned my addr 172.21.15.161:0/1953372569 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa231000640 1 --2- 172.21.15.161:0/1953372569 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa22c156410 0x7fa22c138940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa223400640 1 -- 172.21.15.161:0/1953372569 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa22c156410 msgr2=0x7fa22c138940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa223400640 1 --2- 172.21.15.161:0/1953372569 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa22c156410 0x7fa22c138940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa223400640 1 -- 172.21.15.161:0/1953372569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa210003ef0 msgr2=0x7fa22c154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa223400640 1 --2- 172.21.15.161:0/1953372569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa210003ef0 0x7fa22c154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa223400640 1 -- 172.21.15.161:0/1953372569 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c139040 con 0x7fa22c139570 2024-05-22T21:14:34.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa223400640 1 --2- 172.21.15.161:0/1953372569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c139570 0x7fa22c138400 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa21401c270 tx=0x7fa210005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:34.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa228c00640 1 -- 172.21.15.161:0/1953372569 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214005390 con 0x7fa22c139570 2024-05-22T21:14:34.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa228c00640 1 -- 172.21.15.161:0/1953372569 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2140aba40 con 0x7fa22c139570 2024-05-22T21:14:34.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa233000640 1 -- 172.21.15.161:0/1953372569 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22c13ca00 con 0x7fa22c139570 2024-05-22T21:14:34.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa228c00640 1 -- 172.21.15.161:0/1953372569 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2140abdb0 con 0x7fa22c139570 2024-05-22T21:14:34.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa233000640 1 -- 172.21.15.161:0/1953372569 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22c161fb0 con 0x7fa22c139570 2024-05-22T21:14:34.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.776+0000 7fa233000640 1 -- 172.21.15.161:0/1953372569 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22c067220 con 0x7fa22c139570 2024-05-22T21:14:34.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.780+0000 7fa228c00640 1 -- 172.21.15.161:0/1953372569 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa2140b2050 con 0x7fa22c139570 2024-05-22T21:14:34.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.780+0000 7fa228c00640 1 --2- 172.21.15.161:0/1953372569 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa2080666c0 0x7fa208068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:34.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.780+0000 7fa228c00640 1 -- 172.21.15.161:0/1953372569 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa21411dd70 con 0x7fa22c139570 2024-05-22T21:14:34.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.780+0000 7fa22be00640 1 --2- 172.21.15.161:0/1953372569 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa2080666c0 0x7fa208068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:34.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.780+0000 7fa22be00640 1 --2- 172.21.15.161:0/1953372569 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa2080666c0 0x7fa208068b80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa22c15e580 tx=0x7fa21c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:34.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.784+0000 7fa228c00640 1 -- 172.21.15.161:0/1953372569 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa2140eb180 con 0x7fa22c139570 2024-05-22T21:14:34.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.936+0000 7fa233000640 1 -- 172.21.15.161:0/1953372569 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa22c0ceaa0 con 0x7fa22c139570 2024-05-22T21:14:34.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.936+0000 7fa228c00640 1 -- 172.21.15.161:0/1953372569 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa2140ee7a0 con 0x7fa22c139570 2024-05-22T21:14:34.939 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:14:34.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.936+0000 7fa233000640 1 -- 172.21.15.161:0/1953372569 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa2080666c0 msgr2=0x7fa208068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.936+0000 7fa233000640 1 --2- 172.21.15.161:0/1953372569 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa2080666c0 0x7fa208068b80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa22c15e580 tx=0x7fa21c00b040 comp rx=0 tx=0).stop 2024-05-22T21:14:34.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.936+0000 7fa233000640 1 -- 172.21.15.161:0/1953372569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c139570 msgr2=0x7fa22c138400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:34.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.936+0000 7fa233000640 1 --2- 172.21.15.161:0/1953372569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c139570 0x7fa22c138400 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa21401c270 tx=0x7fa210005800 comp rx=0 tx=0).stop 2024-05-22T21:14:34.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.936+0000 7fa233000640 1 -- 172.21.15.161:0/1953372569 shutdown_connections 2024-05-22T21:14:34.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.936+0000 7fa233000640 1 --2- 172.21.15.161:0/1953372569 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa2080666c0 0x7fa208068b80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.936+0000 7fa233000640 1 --2- 172.21.15.161:0/1953372569 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa22c156410 0x7fa22c138940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.936+0000 7fa233000640 1 --2- 172.21.15.161:0/1953372569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c139570 0x7fa22c138400 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.936+0000 7fa233000640 1 --2- 172.21.15.161:0/1953372569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa210003ef0 0x7fa22c154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:34.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.936+0000 7fa233000640 1 -- 172.21.15.161:0/1953372569 >> 172.21.15.161:0/1953372569 conn(0x7fa22c06a2d0 msgr2=0x7fa22c15bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:34.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.936+0000 7fa233000640 1 -- 172.21.15.161:0/1953372569 shutdown_connections 2024-05-22T21:14:34.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:34.936+0000 7fa233000640 1 -- 172.21.15.161:0/1953372569 wait complete. 2024-05-22T21:14:34.960 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:14:35.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.096+0000 7f254f000640 1 Processor -- start 2024-05-22T21:14:35.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.096+0000 7f254f000640 1 -- start start 2024-05-22T21:14:35.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.100+0000 7f254f000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f25480ce370 0x7f25481442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.100+0000 7f254f000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25481448e0 0x7f25480b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.100+0000 7f254f000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25480ce830 con 0x7f2548143ed0 2024-05-22T21:14:35.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.100+0000 7f254f000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25480b2e60 con 0x7f25480ce370 2024-05-22T21:14:35.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.100+0000 7f254f000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25480b3140 con 0x7f25481448e0 2024-05-22T21:14:35.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.100+0000 7f254e600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25481448e0 0x7f25480b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.100+0000 7f254dc00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f2548143ed0 0x7f25480b20e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:42008/0 (socket says 172.21.15.161:42008) 2024-05-22T21:14:35.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.100+0000 7f254dc00640 1 -- 172.21.15.161:0/3498255014 learned_addr learned my addr 172.21.15.161:0/3498255014 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:35.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.100+0000 7f254d200640 1 --2- 172.21.15.161:0/3498255014 >> v2:172.21.15.179:3300/0 conn(0x7f25480ce370 0x7f25481442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.100+0000 7f254e600640 1 -- 172.21.15.161:0/3498255014 >> v2:172.21.15.179:3300/0 conn(0x7f25480ce370 msgr2=0x7f25481442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:35.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.100+0000 7f254e600640 1 --2- 172.21.15.161:0/3498255014 >> v2:172.21.15.179:3300/0 conn(0x7f25480ce370 0x7f25481442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.100+0000 7f254e600640 1 -- 172.21.15.161:0/3498255014 >> v1:172.21.15.179:6789/0 conn(0x7f2548143ed0 legacy=0x7f25480b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:35.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.100+0000 7f2547e00640 1 -- 172.21.15.161:0/3498255014 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909037880 0 0) 0x7f25480ce830 con 0x7f2548143ed0 2024-05-22T21:14:35.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.100+0000 7f254e600640 1 -- 172.21.15.161:0/3498255014 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25480b3420 con 0x7f25481448e0 2024-05-22T21:14:35.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.104+0000 7f254e600640 1 --2- 172.21.15.161:0/3498255014 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25481448e0 0x7f25480b27f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2540009d90 tx=0x7f254001c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f17b1599c9ab390 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:35.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.104+0000 7f2547e00640 1 -- 172.21.15.161:0/3498255014 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25400a5930 con 0x7f25481448e0 2024-05-22T21:14:35.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.104+0000 7f2547e00640 1 -- 172.21.15.161:0/3498255014 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25400a5a90 con 0x7f25481448e0 2024-05-22T21:14:35.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.104+0000 7f2547e00640 1 -- 172.21.15.161:0/3498255014 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25400a5e20 con 0x7f25481448e0 2024-05-22T21:14:35.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.104+0000 7f254f000640 1 -- 172.21.15.161:0/3498255014 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25481448e0 msgr2=0x7f25480b27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:35.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.104+0000 7f254f000640 1 --2- 172.21.15.161:0/3498255014 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25481448e0 0x7f25480b27f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2540009d90 tx=0x7f254001c850 comp rx=0 tx=0).stop 2024-05-22T21:14:35.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.104+0000 7f254f000640 1 -- 172.21.15.161:0/3498255014 shutdown_connections 2024-05-22T21:14:35.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.104+0000 7f254f000640 1 --2- 172.21.15.161:0/3498255014 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25481448e0 0x7f25480b27f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.104+0000 7f254f000640 1 --2- 172.21.15.161:0/3498255014 >> v2:172.21.15.179:3300/0 conn(0x7f25480ce370 0x7f25481442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.104+0000 7f254f000640 1 -- 172.21.15.161:0/3498255014 >> 172.21.15.161:0/3498255014 conn(0x7f25480cec00 msgr2=0x7f25480cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:35.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.104+0000 7f254f000640 1 -- 172.21.15.161:0/3498255014 shutdown_connections 2024-05-22T21:14:35.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.104+0000 7f254f000640 1 -- 172.21.15.161:0/3498255014 wait complete. 2024-05-22T21:14:35.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.104+0000 7f254f000640 1 Processor -- start 2024-05-22T21:14:35.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.104+0000 7f254f000640 1 -- start start 2024-05-22T21:14:35.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.104+0000 7f254f000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25480ce370 0x7f25481285e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254f000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2548143ed0 0x7f2548071010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254f000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25481448e0 0x7f25480aa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254f000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2548129b20 con 0x7f2548143ed0 2024-05-22T21:14:35.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254f000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2548128f10 con 0x7f25481448e0 2024-05-22T21:14:35.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254f000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25481291f0 con 0x7f25480ce370 2024-05-22T21:14:35.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254d200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2548143ed0 0x7f2548071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254d200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2548143ed0 0x7f2548071010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55860/0 (socket says 172.21.15.161:55860) 2024-05-22T21:14:35.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254d200640 1 -- 172.21.15.161:0/945383599 learned_addr learned my addr 172.21.15.161:0/945383599 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:35.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254dc00640 1 --2- 172.21.15.161:0/945383599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25480ce370 0x7f25481285e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254e600640 1 --2- 172.21.15.161:0/945383599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25481448e0 0x7f25480aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254d200640 1 -- 172.21.15.161:0/945383599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25480ce370 msgr2=0x7f25481285e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:35.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254d200640 1 --2- 172.21.15.161:0/945383599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25480ce370 0x7f25481285e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254d200640 1 -- 172.21.15.161:0/945383599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25481448e0 msgr2=0x7f25480aa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:35.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254d200640 1 --2- 172.21.15.161:0/945383599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25481448e0 0x7f25480aa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254d200640 1 -- 172.21.15.161:0/945383599 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2540009a70 con 0x7f2548143ed0 2024-05-22T21:14:35.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254dc00640 1 --2- 172.21.15.161:0/945383599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25480ce370 0x7f25481285e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:35.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254d200640 1 --2- 172.21.15.161:0/945383599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2548143ed0 0x7f2548071010 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f253400e930 tx=0x7f253400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:35.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f2545600640 1 -- 172.21.15.161:0/945383599 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534009690 con 0x7f2548143ed0 2024-05-22T21:14:35.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f2545600640 1 -- 172.21.15.161:0/945383599 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253400ca00 con 0x7f2548143ed0 2024-05-22T21:14:35.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f2545600640 1 -- 172.21.15.161:0/945383599 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253400cd70 con 0x7f2548143ed0 2024-05-22T21:14:35.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254f000640 1 -- 172.21.15.161:0/945383599 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25481294d0 con 0x7f2548143ed0 2024-05-22T21:14:35.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254f000640 1 -- 172.21.15.161:0/945383599 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2548128b20 con 0x7f2548143ed0 2024-05-22T21:14:35.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f254f000640 1 -- 172.21.15.161:0/945383599 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2548067070 con 0x7f2548143ed0 2024-05-22T21:14:35.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.108+0000 7f2545600640 1 -- 172.21.15.161:0/945383599 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2534013470 con 0x7f2548143ed0 2024-05-22T21:14:35.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.112+0000 7f2545600640 1 --2- 172.21.15.161:0/945383599 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f25280666c0 0x7f2528068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.112+0000 7f2545600640 1 -- 172.21.15.161:0/945383599 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25340844a0 con 0x7f2548143ed0 2024-05-22T21:14:35.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.112+0000 7f254dc00640 1 --2- 172.21.15.161:0/945383599 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f25280666c0 0x7f2528068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.112+0000 7f254dc00640 1 --2- 172.21.15.161:0/945383599 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f25280666c0 0x7f2528068b80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f253c007fd0 tx=0x7f253c007c20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:35.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.112+0000 7f2545600640 1 -- 172.21.15.161:0/945383599 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2534051830 con 0x7f2548143ed0 2024-05-22T21:14:35.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.268+0000 7f254f000640 1 -- 172.21.15.161:0/945383599 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2548129c60 con 0x7f2548143ed0 2024-05-22T21:14:35.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.268+0000 7f2545600640 1 -- 172.21.15.161:0/945383599 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2548129c60 con 0x7f2548143ed0 2024-05-22T21:14:35.270 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:14:35.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.268+0000 7f254f000640 1 -- 172.21.15.161:0/945383599 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f25280666c0 msgr2=0x7f2528068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:35.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.268+0000 7f254f000640 1 --2- 172.21.15.161:0/945383599 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f25280666c0 0x7f2528068b80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f253c007fd0 tx=0x7f253c007c20 comp rx=0 tx=0).stop 2024-05-22T21:14:35.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.268+0000 7f254f000640 1 -- 172.21.15.161:0/945383599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2548143ed0 msgr2=0x7f2548071010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:35.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.268+0000 7f254f000640 1 --2- 172.21.15.161:0/945383599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2548143ed0 0x7f2548071010 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f253400e930 tx=0x7f253400edf0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.268+0000 7f254f000640 1 -- 172.21.15.161:0/945383599 shutdown_connections 2024-05-22T21:14:35.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.268+0000 7f254f000640 1 --2- 172.21.15.161:0/945383599 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f25280666c0 0x7f2528068b80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.268+0000 7f254f000640 1 --2- 172.21.15.161:0/945383599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25481448e0 0x7f25480aa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.268+0000 7f254f000640 1 --2- 172.21.15.161:0/945383599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2548143ed0 0x7f2548071010 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.268+0000 7f254f000640 1 --2- 172.21.15.161:0/945383599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25480ce370 0x7f25481285e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.268+0000 7f254f000640 1 -- 172.21.15.161:0/945383599 >> 172.21.15.161:0/945383599 conn(0x7f25480cec00 msgr2=0x7f2548142690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:35.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.268+0000 7f254f000640 1 -- 172.21.15.161:0/945383599 shutdown_connections 2024-05-22T21:14:35.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.268+0000 7f254f000640 1 -- 172.21.15.161:0/945383599 wait complete. 2024-05-22T21:14:35.292 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:14:35.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.424+0000 7fb360c00640 1 Processor -- start 2024-05-22T21:14:35.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.424+0000 7fb360c00640 1 -- start start 2024-05-22T21:14:35.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.424+0000 7fb360c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb35c13cfc0 0x7fb35c1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.424+0000 7fb360c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb35c13fe50 0x7fb35c06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.424+0000 7fb360c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35c13d3a0 con 0x7fb35c15af60 2024-05-22T21:14:35.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.424+0000 7fb360c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb35c0cd920 con 0x7fb35c13cfc0 2024-05-22T21:14:35.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.424+0000 7fb360c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c0cdc00 con 0x7fb35c13fe50 2024-05-22T21:14:35.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.424+0000 7fb359e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb35c13cfc0 0x7fb35c1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.424+0000 7fb359e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb35c13cfc0 0x7fb35c1402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55866/0 (socket says 172.21.15.161:55866) 2024-05-22T21:14:35.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.424+0000 7fb359e00640 1 -- 172.21.15.161:0/1869149582 learned_addr learned my addr 172.21.15.161:0/1869149582 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:35.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.428+0000 7fb35a800640 1 --2- 172.21.15.161:0/1869149582 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb35c13fe50 0x7fb35c06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.428+0000 7fb359e00640 1 -- 172.21.15.161:0/1869149582 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb35c13fe50 msgr2=0x7fb35c06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:35.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.428+0000 7fb359e00640 1 --2- 172.21.15.161:0/1869149582 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb35c13fe50 0x7fb35c06fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.428+0000 7fb359e00640 1 -- 172.21.15.161:0/1869149582 >> v1:172.21.15.179:6789/0 conn(0x7fb35c15af60 legacy=0x7fb35c13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:35.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.428+0000 7fb359e00640 1 -- 172.21.15.161:0/1869149582 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35c0cdee0 con 0x7fb35c13cfc0 2024-05-22T21:14:35.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.428+0000 7fb359e00640 1 --2- 172.21.15.161:0/1869149582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb35c13cfc0 0x7fb35c1402e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00bb10 tx=0x7fb34c01eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15784fd1459f9ebc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:35.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.428+0000 7fb359400640 1 -- 172.21.15.161:0/1869149582 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c0a7870 con 0x7fb35c13cfc0 2024-05-22T21:14:35.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.428+0000 7fb359400640 1 -- 172.21.15.161:0/1869149582 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34c0a79d0 con 0x7fb35c13cfc0 2024-05-22T21:14:35.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.428+0000 7fb359400640 1 -- 172.21.15.161:0/1869149582 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c0a7d60 con 0x7fb35c13cfc0 2024-05-22T21:14:35.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.428+0000 7fb360c00640 1 -- 172.21.15.161:0/1869149582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb35c13cfc0 msgr2=0x7fb35c1402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:35.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.428+0000 7fb360c00640 1 --2- 172.21.15.161:0/1869149582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb35c13cfc0 0x7fb35c1402e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00bb10 tx=0x7fb34c01eb70 comp rx=0 tx=0).stop 2024-05-22T21:14:35.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.428+0000 7fb360c00640 1 -- 172.21.15.161:0/1869149582 shutdown_connections 2024-05-22T21:14:35.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.428+0000 7fb360c00640 1 --2- 172.21.15.161:0/1869149582 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb35c13fe50 0x7fb35c06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.428+0000 7fb360c00640 1 --2- 172.21.15.161:0/1869149582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb35c13cfc0 0x7fb35c1402e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.428+0000 7fb360c00640 1 -- 172.21.15.161:0/1869149582 >> 172.21.15.161:0/1869149582 conn(0x7fb35c06a2d0 msgr2=0x7fb35c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:35.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb360c00640 1 -- 172.21.15.161:0/1869149582 shutdown_connections 2024-05-22T21:14:35.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb360c00640 1 -- 172.21.15.161:0/1869149582 wait complete. 2024-05-22T21:14:35.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb360c00640 1 Processor -- start 2024-05-22T21:14:35.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb360c00640 1 -- start start 2024-05-22T21:14:35.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb360c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb35c13cfc0 0x7fb35c13c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb360c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb35c13fe50 0x7fb35c0cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb360c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb35c15af60 0x7fb35c0cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb360c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c142900 con 0x7fb35c13cfc0 2024-05-22T21:14:35.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb360c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c0b3300 con 0x7fb35c15af60 2024-05-22T21:14:35.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb360c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c0b35e0 con 0x7fb35c13fe50 2024-05-22T21:14:35.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb359e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb35c13cfc0 0x7fb35c13c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb359e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb35c13cfc0 0x7fb35c13c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55874/0 (socket says 172.21.15.161:55874) 2024-05-22T21:14:35.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb359e00640 1 -- 172.21.15.161:0/1822321179 learned_addr learned my addr 172.21.15.161:0/1822321179 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:35.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb351400640 1 --2- 172.21.15.161:0/1822321179 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb35c13fe50 0x7fb35c0cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb35a800640 1 --2- 172.21.15.161:0/1822321179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb35c15af60 0x7fb35c0cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb359e00640 1 -- 172.21.15.161:0/1822321179 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb35c13fe50 msgr2=0x7fb35c0cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:35.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb359e00640 1 --2- 172.21.15.161:0/1822321179 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb35c13fe50 0x7fb35c0cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb359e00640 1 -- 172.21.15.161:0/1822321179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb35c15af60 msgr2=0x7fb35c0cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:35.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb359e00640 1 --2- 172.21.15.161:0/1822321179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb35c15af60 0x7fb35c0cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb359e00640 1 -- 172.21.15.161:0/1822321179 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c00b770 con 0x7fb35c13cfc0 2024-05-22T21:14:35.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb359e00640 1 --2- 172.21.15.161:0/1822321179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb35c13cfc0 0x7fb35c13c390 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00b2c0 tx=0x7fb34c01d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:35.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb352a00640 1 -- 172.21.15.161:0/1822321179 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c0af8d0 con 0x7fb35c13cfc0 2024-05-22T21:14:35.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb352a00640 1 -- 172.21.15.161:0/1822321179 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34c0afa30 con 0x7fb35c13cfc0 2024-05-22T21:14:35.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb360c00640 1 -- 172.21.15.161:0/1822321179 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35c0b38c0 con 0x7fb35c13cfc0 2024-05-22T21:14:35.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.432+0000 7fb352a00640 1 -- 172.21.15.161:0/1822321179 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c0afde0 con 0x7fb35c13cfc0 2024-05-22T21:14:35.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.436+0000 7fb360c00640 1 -- 172.21.15.161:0/1822321179 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35c0b3e80 con 0x7fb35c13cfc0 2024-05-22T21:14:35.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.436+0000 7fb360c00640 1 -- 172.21.15.161:0/1822321179 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35c067220 con 0x7fb35c13cfc0 2024-05-22T21:14:35.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.436+0000 7fb352a00640 1 -- 172.21.15.161:0/1822321179 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb34c0ba480 con 0x7fb35c13cfc0 2024-05-22T21:14:35.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.436+0000 7fb352a00640 1 --2- 172.21.15.161:0/1822321179 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb338066690 0x7fb338068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.436+0000 7fb352a00640 1 -- 172.21.15.161:0/1822321179 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb34c11f320 con 0x7fb35c13cfc0 2024-05-22T21:14:35.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.436+0000 7fb351400640 1 --2- 172.21.15.161:0/1822321179 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb338066690 0x7fb338068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.436+0000 7fb351400640 1 --2- 172.21.15.161:0/1822321179 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb338066690 0x7fb338068b50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb35c1f6050 tx=0x7fb344009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:35.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.440+0000 7fb352a00640 1 -- 172.21.15.161:0/1822321179 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb34c0ec680 con 0x7fb35c13cfc0 2024-05-22T21:14:35.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.604+0000 7fb360c00640 1 -- 172.21.15.161:0/1822321179 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb35c164c40 con 0x7fb35c13cfc0 2024-05-22T21:14:35.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.604+0000 7fb352a00640 1 -- 172.21.15.161:0/1822321179 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb34c0b4080 con 0x7fb35c13cfc0 2024-05-22T21:14:35.608 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:14:35.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.608+0000 7fb360c00640 1 -- 172.21.15.161:0/1822321179 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb338066690 msgr2=0x7fb338068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:35.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.608+0000 7fb360c00640 1 --2- 172.21.15.161:0/1822321179 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb338066690 0x7fb338068b50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb35c1f6050 tx=0x7fb344009040 comp rx=0 tx=0).stop 2024-05-22T21:14:35.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.608+0000 7fb360c00640 1 -- 172.21.15.161:0/1822321179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb35c13cfc0 msgr2=0x7fb35c13c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:35.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.608+0000 7fb360c00640 1 --2- 172.21.15.161:0/1822321179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb35c13cfc0 0x7fb35c13c390 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00b2c0 tx=0x7fb34c01d3b0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.608+0000 7fb360c00640 1 -- 172.21.15.161:0/1822321179 shutdown_connections 2024-05-22T21:14:35.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.608+0000 7fb360c00640 1 --2- 172.21.15.161:0/1822321179 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb338066690 0x7fb338068b50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.608+0000 7fb360c00640 1 --2- 172.21.15.161:0/1822321179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb35c15af60 0x7fb35c0cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.608+0000 7fb360c00640 1 --2- 172.21.15.161:0/1822321179 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb35c13fe50 0x7fb35c0cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.608+0000 7fb360c00640 1 --2- 172.21.15.161:0/1822321179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb35c13cfc0 0x7fb35c13c390 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.608+0000 7fb360c00640 1 -- 172.21.15.161:0/1822321179 >> 172.21.15.161:0/1822321179 conn(0x7fb35c06a2d0 msgr2=0x7fb35c0b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:35.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.608+0000 7fb360c00640 1 -- 172.21.15.161:0/1822321179 shutdown_connections 2024-05-22T21:14:35.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.608+0000 7fb360c00640 1 -- 172.21.15.161:0/1822321179 wait complete. 2024-05-22T21:14:35.629 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-22T21:14:35.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:35.629 DEBUG:teuthology.orchestra.run.smithi179:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-22T21:14:35.659 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:14:35.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:35.659 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-22T21:14:35.716 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:14:35.716 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:35.725 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:14:35.725 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:14:35.725 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:35.771 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:14:35.771 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:14:35.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.904+0000 7fc893c00640 1 Processor -- start 2024-05-22T21:14:35.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.904+0000 7fc893c00640 1 -- start start 2024-05-22T21:14:35.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.904+0000 7fc893c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc88c1597f0 0x7fc88c0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.904+0000 7fc893c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc88c070be0 0x7fc88c0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.904+0000 7fc893c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88c071040 con 0x7fc88c0b2ea0 2024-05-22T21:14:35.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.904+0000 7fc893c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc88c159bd0 con 0x7fc88c1597f0 2024-05-22T21:14:35.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.904+0000 7fc893c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0cbdd0 con 0x7fc88c070be0 2024-05-22T21:14:35.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.904+0000 7fc891200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc88c1597f0 0x7fc88c0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.904+0000 7fc891c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc88c0b2ea0 0x7fc88c06c7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41842/0 (socket says 172.21.15.161:41842) 2024-05-22T21:14:35.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.904+0000 7fc891c00640 1 -- 172.21.15.161:0/327461425 learned_addr learned my addr 172.21.15.161:0/327461425 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:35.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc88be00640 1 --2- 172.21.15.161:0/327461425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc88c070be0 0x7fc88c0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc891200640 1 -- 172.21.15.161:0/327461425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc88c070be0 msgr2=0x7fc88c0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:35.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc88b400640 1 -- 172.21.15.161:0/327461425 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1370587985 0 0) 0x7fc88c071040 con 0x7fc88c0b2ea0 2024-05-22T21:14:35.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc891200640 1 --2- 172.21.15.161:0/327461425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc88c070be0 0x7fc88c0b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc891200640 1 -- 172.21.15.161:0/327461425 >> v1:172.21.15.161:6789/0 conn(0x7fc88c0b2ea0 legacy=0x7fc88c06c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:35.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc891200640 1 -- 172.21.15.161:0/327461425 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c0cc050 con 0x7fc88c1597f0 2024-05-22T21:14:35.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc88be00640 1 --2- 172.21.15.161:0/327461425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc88c070be0 0x7fc88c0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:35.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc891200640 1 --2- 172.21.15.161:0/327461425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc88c1597f0 0x7fc88c0706a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00aea0 tx=0x7fc87c01da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d8e6e5dcddd733 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:35.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc88b400640 1 -- 172.21.15.161:0/327461425 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c00a2d0 con 0x7fc88c1597f0 2024-05-22T21:14:35.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc88b400640 1 -- 172.21.15.161:0/327461425 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87c0a68b0 con 0x7fc88c1597f0 2024-05-22T21:14:35.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc88b400640 1 -- 172.21.15.161:0/327461425 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c0a6c40 con 0x7fc88c1597f0 2024-05-22T21:14:35.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc893c00640 1 -- 172.21.15.161:0/327461425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc88c1597f0 msgr2=0x7fc88c0706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:35.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc893c00640 1 --2- 172.21.15.161:0/327461425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc88c1597f0 0x7fc88c0706a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00aea0 tx=0x7fc87c01da70 comp rx=0 tx=0).stop 2024-05-22T21:14:35.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc893c00640 1 -- 172.21.15.161:0/327461425 shutdown_connections 2024-05-22T21:14:35.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc893c00640 1 --2- 172.21.15.161:0/327461425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc88c070be0 0x7fc88c0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc893c00640 1 --2- 172.21.15.161:0/327461425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc88c1597f0 0x7fc88c0706a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.908+0000 7fc893c00640 1 -- 172.21.15.161:0/327461425 >> 172.21.15.161:0/327461425 conn(0x7fc88c0b0bc0 msgr2=0x7fc88c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:35.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc893c00640 1 -- 172.21.15.161:0/327461425 shutdown_connections 2024-05-22T21:14:35.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc893c00640 1 -- 172.21.15.161:0/327461425 wait complete. 2024-05-22T21:14:35.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc893c00640 1 Processor -- start 2024-05-22T21:14:35.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc893c00640 1 -- start start 2024-05-22T21:14:35.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc893c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc88c070be0 0x7fc88c159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc893c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc88c0b2ea0 0x7fc88c150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc893c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc88c1597f0 0x7fc88c157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc893c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c15a2d0 con 0x7fc88c070be0 2024-05-22T21:14:35.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc893c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c15a5a0 con 0x7fc88c1597f0 2024-05-22T21:14:35.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc893c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c15a880 con 0x7fc88c0b2ea0 2024-05-22T21:14:35.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc891200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc88c070be0 0x7fc88c159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc891200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc88c070be0 0x7fc88c159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57010/0 (socket says 172.21.15.161:57010) 2024-05-22T21:14:35.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc891200640 1 -- 172.21.15.161:0/1595841947 learned_addr learned my addr 172.21.15.161:0/1595841947 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:35.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc88be00640 1 --2- 172.21.15.161:0/1595841947 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc88c0b2ea0 0x7fc88c150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc891c00640 1 --2- 172.21.15.161:0/1595841947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc88c1597f0 0x7fc88c157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc891200640 1 -- 172.21.15.161:0/1595841947 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc88c0b2ea0 msgr2=0x7fc88c150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:35.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc891200640 1 --2- 172.21.15.161:0/1595841947 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc88c0b2ea0 0x7fc88c150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc891200640 1 -- 172.21.15.161:0/1595841947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc88c1597f0 msgr2=0x7fc88c157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:35.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc891200640 1 --2- 172.21.15.161:0/1595841947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc88c1597f0 0x7fc88c157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:35.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc891200640 1 -- 172.21.15.161:0/1595841947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c00ab80 con 0x7fc88c070be0 2024-05-22T21:14:35.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc891c00640 1 --2- 172.21.15.161:0/1595841947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc88c1597f0 0x7fc88c157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:35.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc891200640 1 --2- 172.21.15.161:0/1595841947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc88c070be0 0x7fc88c159d90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc87c01b6a0 tx=0x7fc87c01a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:35.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc888c00640 1 -- 172.21.15.161:0/1595841947 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c0bc070 con 0x7fc88c070be0 2024-05-22T21:14:35.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc888c00640 1 -- 172.21.15.161:0/1595841947 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87c0b7690 con 0x7fc88c070be0 2024-05-22T21:14:35.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc888c00640 1 -- 172.21.15.161:0/1595841947 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c0b7a20 con 0x7fc88c070be0 2024-05-22T21:14:35.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc893c00640 1 -- 172.21.15.161:0/1595841947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88c15ab60 con 0x7fc88c070be0 2024-05-22T21:14:35.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.912+0000 7fc893c00640 1 -- 172.21.15.161:0/1595841947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88c15b120 con 0x7fc88c070be0 2024-05-22T21:14:35.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.916+0000 7fc893c00640 1 -- 172.21.15.161:0/1595841947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88c06cee0 con 0x7fc88c070be0 2024-05-22T21:14:35.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.916+0000 7fc888c00640 1 -- 172.21.15.161:0/1595841947 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc87c0b7b80 con 0x7fc88c070be0 2024-05-22T21:14:35.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.916+0000 7fc888c00640 1 --2- 172.21.15.161:0/1595841947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc86c066690 0x7fc86c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:35.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.916+0000 7fc888c00640 1 -- 172.21.15.161:0/1595841947 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc87c11efe0 con 0x7fc88c070be0 2024-05-22T21:14:35.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.916+0000 7fc88be00640 1 --2- 172.21.15.161:0/1595841947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc86c066690 0x7fc86c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:35.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.916+0000 7fc88be00640 1 --2- 172.21.15.161:0/1595841947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc86c066690 0x7fc86c068b50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0b24e0 tx=0x7fc878008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:35.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:35.920+0000 7fc888c00640 1 -- 172.21.15.161:0/1595841947 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc88c06cee0 con 0x7fc88c070be0 2024-05-22T21:14:36.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.104+0000 7fc893c00640 1 -- 172.21.15.161:0/1595841947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc88c06cee0 con 0x7fc88c070be0 2024-05-22T21:14:36.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.108+0000 7fc888c00640 1 -- 172.21.15.161:0/1595841947 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4743 (secure 0 0 0) 0x7fc88c06cee0 con 0x7fc88c070be0 2024-05-22T21:14:36.109 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:36.109 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3742018925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":3742018925},{"type":"v1","addr":"172.21.15.161:6837","nonce":3742018925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2553366378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2553366378},{"type":"v1","addr":"172.21.15.179:6834","nonce":2553366378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/509468338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":509468338},{"type":"v1","addr":"172.21.15.179:6835","nonce":509468338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:19.053259+0000","modified":"2024-05-22T21:14:22.076119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6836/1635458417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1635458417},{"type":"v1","addr":"172.21.15.161:6836","nonce":1635458417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-22T21:14:36.110 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 16 2024-05-22T21:14:36.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.108+0000 7fc893c00640 1 -- 172.21.15.161:0/1595841947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc86c066690 msgr2=0x7fc86c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.108+0000 7fc893c00640 1 --2- 172.21.15.161:0/1595841947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc86c066690 0x7fc86c068b50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0b24e0 tx=0x7fc878008040 comp rx=0 tx=0).stop 2024-05-22T21:14:36.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.108+0000 7fc893c00640 1 -- 172.21.15.161:0/1595841947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc88c070be0 msgr2=0x7fc88c159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.108+0000 7fc893c00640 1 --2- 172.21.15.161:0/1595841947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc88c070be0 0x7fc88c159d90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc87c01b6a0 tx=0x7fc87c01a810 comp rx=0 tx=0).stop 2024-05-22T21:14:36.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.108+0000 7fc893c00640 1 -- 172.21.15.161:0/1595841947 shutdown_connections 2024-05-22T21:14:36.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.108+0000 7fc893c00640 1 --2- 172.21.15.161:0/1595841947 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc86c066690 0x7fc86c068b50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.108+0000 7fc893c00640 1 --2- 172.21.15.161:0/1595841947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc88c1597f0 0x7fc88c157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.108+0000 7fc893c00640 1 --2- 172.21.15.161:0/1595841947 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc88c0b2ea0 0x7fc88c150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.108+0000 7fc893c00640 1 --2- 172.21.15.161:0/1595841947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc88c070be0 0x7fc88c159d90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.108+0000 7fc893c00640 1 -- 172.21.15.161:0/1595841947 >> 172.21.15.161:0/1595841947 conn(0x7fc88c0b0bc0 msgr2=0x7fc88c0ce380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:36.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.108+0000 7fc893c00640 1 -- 172.21.15.161:0/1595841947 shutdown_connections 2024-05-22T21:14:36.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.108+0000 7fc893c00640 1 -- 172.21.15.161:0/1595841947 wait complete. 2024-05-22T21:14:36.130 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:36.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.264+0000 7f9f87a00640 1 Processor -- start 2024-05-22T21:14:36.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.264+0000 7f9f87a00640 1 -- start start 2024-05-22T21:14:36.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.264+0000 7f9f87a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9f801438e0 0x7f9f800714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:36.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.264+0000 7f9f87a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9f800b1510 0x7f9f80163af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:36.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.264+0000 7f9f87a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f80143cc0 con 0x7f9f80070f30 2024-05-22T21:14:36.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.264+0000 7f9f87a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f80070a50 con 0x7f9f801438e0 2024-05-22T21:14:36.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.264+0000 7f9f87a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f8006fdd0 con 0x7f9f800b1510 2024-05-22T21:14:36.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.264+0000 7f9f85000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9f801438e0 0x7f9f800714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:36.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.264+0000 7f9f85000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9f801438e0 0x7f9f800714f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57022/0 (socket says 172.21.15.161:57022) 2024-05-22T21:14:36.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.264+0000 7f9f85000640 1 -- 172.21.15.161:0/2480584221 learned_addr learned my addr 172.21.15.161:0/2480584221 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:36.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.264+0000 7f9f85a00640 1 --2- 172.21.15.161:0/2480584221 >> v2:172.21.15.179:3300/0 conn(0x7f9f800b1510 0x7f9f80163af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:36.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.264+0000 7f9f85000640 1 -- 172.21.15.161:0/2480584221 >> v2:172.21.15.179:3300/0 conn(0x7f9f800b1510 msgr2=0x7f9f80163af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:36.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.264+0000 7f9f85000640 1 --2- 172.21.15.161:0/2480584221 >> v2:172.21.15.179:3300/0 conn(0x7f9f800b1510 0x7f9f80163af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.264+0000 7f9f85000640 1 -- 172.21.15.161:0/2480584221 >> v1:172.21.15.179:6789/0 conn(0x7f9f80070f30 legacy=0x7f9f800b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.264+0000 7f9f85000640 1 -- 172.21.15.161:0/2480584221 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f800700b0 con 0x7f9f801438e0 2024-05-22T21:14:36.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.268+0000 7f9f85000640 1 --2- 172.21.15.161:0/2480584221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f801438e0 0x7f9f800714f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9f800ce9a0 tx=0x7f9f7001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497f8ff4058aca85 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:36.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.268+0000 7f9f7f400640 1 -- 172.21.15.161:0/2480584221 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7000b2d0 con 0x7f9f801438e0 2024-05-22T21:14:36.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.268+0000 7f9f7f400640 1 -- 172.21.15.161:0/2480584221 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7000b430 con 0x7f9f801438e0 2024-05-22T21:14:36.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.268+0000 7f9f7f400640 1 -- 172.21.15.161:0/2480584221 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f700a7a40 con 0x7f9f801438e0 2024-05-22T21:14:36.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.268+0000 7f9f87a00640 1 -- 172.21.15.161:0/2480584221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f801438e0 msgr2=0x7f9f800714f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.268+0000 7f9f87a00640 1 --2- 172.21.15.161:0/2480584221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f801438e0 0x7f9f800714f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9f800ce9a0 tx=0x7f9f7001eb60 comp rx=0 tx=0).stop 2024-05-22T21:14:36.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.268+0000 7f9f87a00640 1 -- 172.21.15.161:0/2480584221 shutdown_connections 2024-05-22T21:14:36.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.268+0000 7f9f87a00640 1 --2- 172.21.15.161:0/2480584221 >> v2:172.21.15.179:3300/0 conn(0x7f9f800b1510 0x7f9f80163af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.268+0000 7f9f87a00640 1 --2- 172.21.15.161:0/2480584221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f801438e0 0x7f9f800714f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.268+0000 7f9f87a00640 1 -- 172.21.15.161:0/2480584221 >> 172.21.15.161:0/2480584221 conn(0x7f9f8006a0a0 msgr2=0x7f9f8006a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:36.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.268+0000 7f9f87a00640 1 -- 172.21.15.161:0/2480584221 shutdown_connections 2024-05-22T21:14:36.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.268+0000 7f9f87a00640 1 -- 172.21.15.161:0/2480584221 wait complete. 2024-05-22T21:14:36.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f87a00640 1 Processor -- start 2024-05-22T21:14:36.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f87a00640 1 -- start start 2024-05-22T21:14:36.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f87a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f80070f30 0x7f9f80141190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:36.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f87a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f800b1510 0x7f9f801421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:36.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f87a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f801438e0 0x7f9f801593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:36.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f87a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f80142720 con 0x7f9f801438e0 2024-05-22T21:14:36.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f87a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f80140410 con 0x7f9f800b1510 2024-05-22T21:14:36.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f87a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8013fc20 con 0x7f9f80070f30 2024-05-22T21:14:36.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f85a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f801438e0 0x7f9f801593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:36.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f85000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f80070f30 0x7f9f80141190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:36.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f85a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f801438e0 0x7f9f801593a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57024/0 (socket says 172.21.15.161:57024) 2024-05-22T21:14:36.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f85a00640 1 -- 172.21.15.161:0/2591026385 learned_addr learned my addr 172.21.15.161:0/2591026385 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:36.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f7fe00640 1 --2- 172.21.15.161:0/2591026385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f800b1510 0x7f9f801421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:36.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f85000640 1 -- 172.21.15.161:0/2591026385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f800b1510 msgr2=0x7f9f801421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f85000640 1 --2- 172.21.15.161:0/2591026385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f800b1510 0x7f9f801421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f85000640 1 -- 172.21.15.161:0/2591026385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f801438e0 msgr2=0x7f9f801593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f85000640 1 --2- 172.21.15.161:0/2591026385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f801438e0 0x7f9f801593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f85000640 1 -- 172.21.15.161:0/2591026385 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7000bd40 con 0x7f9f80070f30 2024-05-22T21:14:36.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f85a00640 1 --2- 172.21.15.161:0/2591026385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f801438e0 0x7f9f801593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:36.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f85000640 1 --2- 172.21.15.161:0/2591026385 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f80070f30 0x7f9f80141190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9f80143e50 tx=0x7f9f7001b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:36.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f7cc00640 1 -- 172.21.15.161:0/2591026385 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f70009800 con 0x7f9f80070f30 2024-05-22T21:14:36.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f7cc00640 1 -- 172.21.15.161:0/2591026385 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f70009980 con 0x7f9f80070f30 2024-05-22T21:14:36.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.272+0000 7f9f7cc00640 1 -- 172.21.15.161:0/2591026385 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f70009d30 con 0x7f9f80070f30 2024-05-22T21:14:36.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.276+0000 7f9f87a00640 1 -- 172.21.15.161:0/2591026385 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f801406b0 con 0x7f9f80070f30 2024-05-22T21:14:36.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.276+0000 7f9f87a00640 1 -- 172.21.15.161:0/2591026385 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f80140c70 con 0x7f9f80070f30 2024-05-22T21:14:36.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.276+0000 7f9f87a00640 1 -- 172.21.15.161:0/2591026385 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f80067220 con 0x7f9f80070f30 2024-05-22T21:14:36.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.276+0000 7f9f7cc00640 1 -- 172.21.15.161:0/2591026385 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9f700b9430 con 0x7f9f80070f30 2024-05-22T21:14:36.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.276+0000 7f9f7cc00640 1 --2- 172.21.15.161:0/2591026385 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9f60066690 0x7f9f60068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:36.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.276+0000 7f9f7cc00640 1 -- 172.21.15.161:0/2591026385 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f7011ffb0 con 0x7f9f80070f30 2024-05-22T21:14:36.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.276+0000 7f9f7fe00640 1 --2- 172.21.15.161:0/2591026385 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9f60066690 0x7f9f60068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:36.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.276+0000 7f9f7fe00640 1 --2- 172.21.15.161:0/2591026385 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9f60066690 0x7f9f60068b50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9f800b08b0 tx=0x7f9f6c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:36.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.280+0000 7f9f7cc00640 1 -- 172.21.15.161:0/2591026385 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9f700ed5d0 con 0x7f9f80070f30 2024-05-22T21:14:36.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.436+0000 7f9f87a00640 1 -- 172.21.15.161:0/2591026385 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f800adec0 con 0x7f9f80070f30 2024-05-22T21:14:36.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.440+0000 7f9f7cc00640 1 -- 172.21.15.161:0/2591026385 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16149 (secure 0 0 0) 0x7f9f700f0bf0 con 0x7f9f80070f30 2024-05-22T21:14:36.441 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:36.441 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":35,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:14:30.106879+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:16.938769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:18.054579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:36.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.440+0000 7f9f87a00640 1 -- 172.21.15.161:0/2591026385 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9f60066690 msgr2=0x7f9f60068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.440+0000 7f9f87a00640 1 --2- 172.21.15.161:0/2591026385 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9f60066690 0x7f9f60068b50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9f800b08b0 tx=0x7f9f6c009040 comp rx=0 tx=0).stop 2024-05-22T21:14:36.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.440+0000 7f9f87a00640 1 -- 172.21.15.161:0/2591026385 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f80070f30 msgr2=0x7f9f80141190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.440+0000 7f9f87a00640 1 --2- 172.21.15.161:0/2591026385 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f80070f30 0x7f9f80141190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9f80143e50 tx=0x7f9f7001b900 comp rx=0 tx=0).stop 2024-05-22T21:14:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.440+0000 7f9f87a00640 1 -- 172.21.15.161:0/2591026385 shutdown_connections 2024-05-22T21:14:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.440+0000 7f9f87a00640 1 --2- 172.21.15.161:0/2591026385 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9f60066690 0x7f9f60068b50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.440+0000 7f9f87a00640 1 --2- 172.21.15.161:0/2591026385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f801438e0 0x7f9f801593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.440+0000 7f9f87a00640 1 --2- 172.21.15.161:0/2591026385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f800b1510 0x7f9f801421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.440+0000 7f9f87a00640 1 --2- 172.21.15.161:0/2591026385 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f80070f30 0x7f9f80141190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.440+0000 7f9f87a00640 1 -- 172.21.15.161:0/2591026385 >> 172.21.15.161:0/2591026385 conn(0x7f9f8006a0a0 msgr2=0x7f9f800b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.440+0000 7f9f87a00640 1 -- 172.21.15.161:0/2591026385 shutdown_connections 2024-05-22T21:14:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.440+0000 7f9f87a00640 1 -- 172.21.15.161:0/2591026385 wait complete. 2024-05-22T21:14:36.462 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:14:16.938769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:14:18.054579+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:14:36.463 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:14:36.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.596+0000 7f8b10c00640 1 Processor -- start 2024-05-22T21:14:36.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.596+0000 7f8b10c00640 1 -- start start 2024-05-22T21:14:36.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.600+0000 7f8b10c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8b0c06c9b0 0x7f8b0c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:36.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.600+0000 7f8b10c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0c144bb0 con 0x7f8b0c143c40 2024-05-22T21:14:36.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.600+0000 7f8b10c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0c15efb0 con 0x7f8b0c15b600 2024-05-22T21:14:36.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.600+0000 7f8b10c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b0c15ba60 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.600+0000 7f8b09e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8b0c143c40 0x7f8b0c1444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41848/0 (socket says 172.21.15.161:41848) 2024-05-22T21:14:36.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.600+0000 7f8b09e00640 1 -- 172.21.15.161:0/830831286 learned_addr learned my addr 172.21.15.161:0/830831286 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:36.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.600+0000 7f8b0a800640 1 --2- 172.21.15.161:0/830831286 >> v2:172.21.15.161:3300/0 conn(0x7f8b0c06c9b0 0x7f8b0c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:36.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.600+0000 7f8b09400640 1 -- 172.21.15.161:0/830831286 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924239516 0 0) 0x7f8b0c144bb0 con 0x7f8b0c143c40 2024-05-22T21:14:36.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.600+0000 7f8b09400640 1 -- 172.21.15.161:0/830831286 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8af0003530 con 0x7f8b0c143c40 2024-05-22T21:14:36.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.600+0000 7f8b0a800640 1 -- 172.21.15.161:0/830831286 >> v1:172.21.15.179:6789/0 conn(0x7f8b0c15b600 legacy=0x7f8b0c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.600+0000 7f8b0a800640 1 -- 172.21.15.161:0/830831286 >> v1:172.21.15.161:6789/0 conn(0x7f8b0c143c40 legacy=0x7f8b0c1444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.600+0000 7f8b0a800640 1 -- 172.21.15.161:0/830831286 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c15f290 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.600+0000 7f8b0a800640 1 --2- 172.21.15.161:0/830831286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c06c9b0 0x7f8b0c0b2160 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009d90 tx=0x7f8b0401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98d0de370e4cd7e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:36.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b09400640 1 -- 172.21.15.161:0/830831286 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b040a5a30 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b09400640 1 -- 172.21.15.161:0/830831286 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b040a5b90 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b09400640 1 -- 172.21.15.161:0/830831286 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b040ad440 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b10c00640 1 -- 172.21.15.161:0/830831286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c06c9b0 msgr2=0x7f8b0c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b10c00640 1 --2- 172.21.15.161:0/830831286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c06c9b0 0x7f8b0c0b2160 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009d90 tx=0x7f8b0401c970 comp rx=0 tx=0).stop 2024-05-22T21:14:36.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b10c00640 1 -- 172.21.15.161:0/830831286 shutdown_connections 2024-05-22T21:14:36.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b10c00640 1 --2- 172.21.15.161:0/830831286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c06c9b0 0x7f8b0c0b2160 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b10c00640 1 -- 172.21.15.161:0/830831286 >> 172.21.15.161:0/830831286 conn(0x7f8b0c06a2d0 msgr2=0x7f8b0c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:36.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b10c00640 1 -- 172.21.15.161:0/830831286 shutdown_connections 2024-05-22T21:14:36.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b10c00640 1 -- 172.21.15.161:0/830831286 wait complete. 2024-05-22T21:14:36.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b10c00640 1 Processor -- start 2024-05-22T21:14:36.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b10c00640 1 -- start start 2024-05-22T21:14:36.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b10c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c06c9b0 0x7f8b0c1429f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:36.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b10c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b0c143c40 0x7f8b0c06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:36.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b10c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b0c15b600 0x7f8b0c151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:36.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b10c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c13f720 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b10c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c142f30 con 0x7f8b0c143c40 2024-05-22T21:14:36.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b10c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c13f970 con 0x7f8b0c15b600 2024-05-22T21:14:36.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b09e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c06c9b0 0x7f8b0c1429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:36.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b09e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c06c9b0 0x7f8b0c1429f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57042/0 (socket says 172.21.15.161:57042) 2024-05-22T21:14:36.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b09e00640 1 -- 172.21.15.161:0/3305146004 learned_addr learned my addr 172.21.15.161:0/3305146004 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:36.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b0a800640 1 --2- 172.21.15.161:0/3305146004 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b0c15b600 0x7f8b0c151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:36.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.604+0000 7f8b01400640 1 --2- 172.21.15.161:0/3305146004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b0c143c40 0x7f8b0c06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:36.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.608+0000 7f8b09e00640 1 -- 172.21.15.161:0/3305146004 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b0c15b600 msgr2=0x7f8b0c151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.608+0000 7f8b09e00640 1 --2- 172.21.15.161:0/3305146004 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b0c15b600 0x7f8b0c151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.608+0000 7f8b09e00640 1 -- 172.21.15.161:0/3305146004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b0c143c40 msgr2=0x7f8b0c06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.608+0000 7f8b09e00640 1 --2- 172.21.15.161:0/3305146004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b0c143c40 0x7f8b0c06fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.608+0000 7f8b09e00640 1 -- 172.21.15.161:0/3305146004 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b04009a70 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.608+0000 7f8b0a800640 1 --2- 172.21.15.161:0/3305146004 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b0c15b600 0x7f8b0c151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:36.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.608+0000 7f8b01400640 1 --2- 172.21.15.161:0/3305146004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b0c143c40 0x7f8b0c06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:36.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.608+0000 7f8b09e00640 1 --2- 172.21.15.161:0/3305146004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c06c9b0 0x7f8b0c1429f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00c4d0 tx=0x7f8afc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:36.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.608+0000 7f8b02a00640 1 -- 172.21.15.161:0/3305146004 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc005cf0 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.608+0000 7f8b02a00640 1 -- 172.21.15.161:0/3305146004 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8afc005e50 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.608+0000 7f8b02a00640 1 -- 172.21.15.161:0/3305146004 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc007270 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.608+0000 7f8b10c00640 1 -- 172.21.15.161:0/3305146004 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0c13fc50 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.608+0000 7f8b10c00640 1 -- 172.21.15.161:0/3305146004 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0c0afa20 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.608+0000 7f8b10c00640 1 -- 172.21.15.161:0/3305146004 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0c13fde0 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.608+0000 7f8b02a00640 1 -- 172.21.15.161:0/3305146004 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8afc0073d0 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.608+0000 7f8b02a00640 1 --2- 172.21.15.161:0/3305146004 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ae8066690 0x7f8ae8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:36.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.612+0000 7f8b02a00640 1 -- 172.21.15.161:0/3305146004 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8afc083600 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.612+0000 7f8b01400640 1 --2- 172.21.15.161:0/3305146004 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ae8066690 0x7f8ae8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:36.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.612+0000 7f8b01400640 1 --2- 172.21.15.161:0/3305146004 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ae8066690 0x7f8ae8068b50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0cc340 tx=0x7f8af4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:36.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.616+0000 7f8b02a00640 1 -- 172.21.15.161:0/3305146004 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8afc050ab0 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.768+0000 7f8b10c00640 1 -- 172.21.15.161:0/3305146004 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b0c067220 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.768+0000 7f8b02a00640 1 -- 172.21.15.161:0/3305146004 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8afc0540d0 con 0x7f8b0c06c9b0 2024-05-22T21:14:36.770 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:14:36.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.768+0000 7f8b10c00640 1 -- 172.21.15.161:0/3305146004 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ae8066690 msgr2=0x7f8ae8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.768+0000 7f8b10c00640 1 --2- 172.21.15.161:0/3305146004 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ae8066690 0x7f8ae8068b50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0cc340 tx=0x7f8af4009040 comp rx=0 tx=0).stop 2024-05-22T21:14:36.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.768+0000 7f8b10c00640 1 -- 172.21.15.161:0/3305146004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c06c9b0 msgr2=0x7f8b0c1429f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.768+0000 7f8b10c00640 1 --2- 172.21.15.161:0/3305146004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c06c9b0 0x7f8b0c1429f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00c4d0 tx=0x7f8afc00c990 comp rx=0 tx=0).stop 2024-05-22T21:14:36.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.768+0000 7f8b10c00640 1 -- 172.21.15.161:0/3305146004 shutdown_connections 2024-05-22T21:14:36.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.768+0000 7f8b10c00640 1 --2- 172.21.15.161:0/3305146004 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ae8066690 0x7f8ae8068b50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.768+0000 7f8b10c00640 1 --2- 172.21.15.161:0/3305146004 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b0c15b600 0x7f8b0c151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.768+0000 7f8b10c00640 1 --2- 172.21.15.161:0/3305146004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b0c143c40 0x7f8b0c06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.768+0000 7f8b10c00640 1 --2- 172.21.15.161:0/3305146004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b0c06c9b0 0x7f8b0c1429f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.768+0000 7f8b10c00640 1 -- 172.21.15.161:0/3305146004 >> 172.21.15.161:0/3305146004 conn(0x7f8b0c06a2d0 msgr2=0x7f8b0c141ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:36.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.768+0000 7f8b10c00640 1 -- 172.21.15.161:0/3305146004 shutdown_connections 2024-05-22T21:14:36.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.768+0000 7f8b10c00640 1 -- 172.21.15.161:0/3305146004 wait complete. 2024-05-22T21:14:36.791 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:14:36.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.928+0000 7fdaaac00640 1 Processor -- start 2024-05-22T21:14:36.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.928+0000 7fdaaac00640 1 -- start start 2024-05-22T21:14:36.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.928+0000 7fdaaac00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdaa415b600 0x7fdaa406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:36.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.928+0000 7fdaaac00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa406cc40 con 0x7fdaa406c7e0 2024-05-22T21:14:36.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.928+0000 7fdaaac00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa4144bb0 con 0x7fdaa4143c40 2024-05-22T21:14:36.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.928+0000 7fdaaac00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa415ba60 con 0x7fdaa415b600 2024-05-22T21:14:36.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.928+0000 7fdaa3400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdaa415b600 0x7fdaa406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:36.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.928+0000 7fdaa8c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fdaa406c7e0 0x7fdaa40b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41852/0 (socket says 172.21.15.161:41852) 2024-05-22T21:14:36.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.928+0000 7fdaa8c00640 1 -- 172.21.15.161:0/196112208 learned_addr learned my addr 172.21.15.161:0/196112208 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:36.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.928+0000 7fdaa3400640 1 -- 172.21.15.161:0/196112208 >> v1:172.21.15.179:6789/0 conn(0x7fdaa4143c40 legacy=0x7fdaa41444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.928+0000 7fdaa3400640 1 -- 172.21.15.161:0/196112208 >> v1:172.21.15.161:6789/0 conn(0x7fdaa406c7e0 legacy=0x7fdaa40b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.928+0000 7fdaa2a00640 1 -- 172.21.15.161:0/196112208 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889348367 0 0) 0x7fdaa406cc40 con 0x7fdaa406c7e0 2024-05-22T21:14:36.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.928+0000 7fdaa3400640 1 -- 172.21.15.161:0/196112208 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa415efb0 con 0x7fdaa415b600 2024-05-22T21:14:36.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.932+0000 7fdaa3400640 1 --2- 172.21.15.161:0/196112208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa415b600 0x7fdaa406c2a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fda900097c0 tx=0x7fda9001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f8e2903fe22af5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:36.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.932+0000 7fdaa2a00640 1 -- 172.21.15.161:0/196112208 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda900a6040 con 0x7fdaa415b600 2024-05-22T21:14:36.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.932+0000 7fdaa2a00640 1 -- 172.21.15.161:0/196112208 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda900aa600 con 0x7fdaa415b600 2024-05-22T21:14:36.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.932+0000 7fdaa2a00640 1 -- 172.21.15.161:0/196112208 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda900aa990 con 0x7fdaa415b600 2024-05-22T21:14:36.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.932+0000 7fdaaac00640 1 -- 172.21.15.161:0/196112208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa415b600 msgr2=0x7fdaa406c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.932+0000 7fdaaac00640 1 --2- 172.21.15.161:0/196112208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa415b600 0x7fdaa406c2a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fda900097c0 tx=0x7fda9001c4a0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.932+0000 7fdaaac00640 1 -- 172.21.15.161:0/196112208 shutdown_connections 2024-05-22T21:14:36.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.932+0000 7fdaaac00640 1 --2- 172.21.15.161:0/196112208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa415b600 0x7fdaa406c2a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.932+0000 7fdaaac00640 1 -- 172.21.15.161:0/196112208 >> 172.21.15.161:0/196112208 conn(0x7fdaa406a2d0 msgr2=0x7fdaa406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:36.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.932+0000 7fdaaac00640 1 -- 172.21.15.161:0/196112208 shutdown_connections 2024-05-22T21:14:36.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.932+0000 7fdaaac00640 1 -- 172.21.15.161:0/196112208 wait complete. 2024-05-22T21:14:36.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.932+0000 7fdaaac00640 1 Processor -- start 2024-05-22T21:14:36.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.932+0000 7fdaaac00640 1 -- start start 2024-05-22T21:14:36.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaaac00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa406c7e0 0x7fdaa41526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaaac00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa4143c40 0x7fdaa4154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaaac00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdaa415b600 0x7fdaa4154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaaac00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa41552a0 con 0x7fdaa406c7e0 2024-05-22T21:14:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaaac00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa4154fc0 con 0x7fdaa4143c40 2024-05-22T21:14:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaaac00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa4126e10 con 0x7fdaa415b600 2024-05-22T21:14:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaa3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa406c7e0 0x7fdaa41526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaa3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa406c7e0 0x7fdaa41526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57070/0 (socket says 172.21.15.161:57070) 2024-05-22T21:14:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaa3e00640 1 -- 172.21.15.161:0/3293692018 learned_addr learned my addr 172.21.15.161:0/3293692018 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaa8c00640 1 --2- 172.21.15.161:0/3293692018 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdaa415b600 0x7fdaa4154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaa3400640 1 --2- 172.21.15.161:0/3293692018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa4143c40 0x7fdaa4154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaa3e00640 1 -- 172.21.15.161:0/3293692018 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdaa415b600 msgr2=0x7fdaa4154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaa3e00640 1 --2- 172.21.15.161:0/3293692018 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdaa415b600 0x7fdaa4154980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaa3e00640 1 -- 172.21.15.161:0/3293692018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa4143c40 msgr2=0x7fdaa4154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:36.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaa3e00640 1 --2- 172.21.15.161:0/3293692018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa4143c40 0x7fdaa4154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:36.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaa3e00640 1 -- 172.21.15.161:0/3293692018 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda90009420 con 0x7fdaa406c7e0 2024-05-22T21:14:36.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaa3e00640 1 --2- 172.21.15.161:0/3293692018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa406c7e0 0x7fdaa41526d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fda9400c500 tx=0x7fda9400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:36.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fda9be00640 1 -- 172.21.15.161:0/3293692018 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94005220 con 0x7fdaa406c7e0 2024-05-22T21:14:36.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fda9be00640 1 -- 172.21.15.161:0/3293692018 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda94005380 con 0x7fdaa406c7e0 2024-05-22T21:14:36.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fda9be00640 1 -- 172.21.15.161:0/3293692018 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94007270 con 0x7fdaa406c7e0 2024-05-22T21:14:36.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaaac00640 1 -- 172.21.15.161:0/3293692018 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa41270f0 con 0x7fdaa406c7e0 2024-05-22T21:14:36.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaaac00640 1 -- 172.21.15.161:0/3293692018 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa4152c10 con 0x7fdaa406c7e0 2024-05-22T21:14:36.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fdaaac00640 1 -- 172.21.15.161:0/3293692018 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa4067220 con 0x7fdaa406c7e0 2024-05-22T21:14:36.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.936+0000 7fda9be00640 1 -- 172.21.15.161:0/3293692018 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fda940073d0 con 0x7fdaa406c7e0 2024-05-22T21:14:36.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.940+0000 7fda9be00640 1 --2- 172.21.15.161:0/3293692018 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda84066690 0x7fda84068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:36.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.940+0000 7fda9be00640 1 -- 172.21.15.161:0/3293692018 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda94083b20 con 0x7fdaa406c7e0 2024-05-22T21:14:36.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.940+0000 7fdaa3400640 1 --2- 172.21.15.161:0/3293692018 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda84066690 0x7fda84068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:36.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.940+0000 7fdaa3400640 1 --2- 172.21.15.161:0/3293692018 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda84066690 0x7fda84068b50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fda900ae000 tx=0x7fda900a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:36.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:36.944+0000 7fda9be00640 1 -- 172.21.15.161:0/3293692018 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fda94050e00 con 0x7fdaa406c7e0 2024-05-22T21:14:37.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.096+0000 7fdaaac00640 1 -- 172.21.15.161:0/3293692018 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdaa40ceaa0 con 0x7fdaa406c7e0 2024-05-22T21:14:37.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.096+0000 7fda9be00640 1 -- 172.21.15.161:0/3293692018 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7fda94054420 con 0x7fdaa406c7e0 2024-05-22T21:14:37.099 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:14:37.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.100+0000 7fdaaac00640 1 -- 172.21.15.161:0/3293692018 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda84066690 msgr2=0x7fda84068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.100+0000 7fdaaac00640 1 --2- 172.21.15.161:0/3293692018 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda84066690 0x7fda84068b50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fda900ae000 tx=0x7fda900a7000 comp rx=0 tx=0).stop 2024-05-22T21:14:37.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.100+0000 7fdaaac00640 1 -- 172.21.15.161:0/3293692018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa406c7e0 msgr2=0x7fdaa41526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.100+0000 7fdaaac00640 1 --2- 172.21.15.161:0/3293692018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa406c7e0 0x7fdaa41526d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fda9400c500 tx=0x7fda9400c9c0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.100+0000 7fdaaac00640 1 -- 172.21.15.161:0/3293692018 shutdown_connections 2024-05-22T21:14:37.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.100+0000 7fdaaac00640 1 --2- 172.21.15.161:0/3293692018 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda84066690 0x7fda84068b50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.100+0000 7fdaaac00640 1 --2- 172.21.15.161:0/3293692018 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdaa415b600 0x7fdaa4154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.100+0000 7fdaaac00640 1 --2- 172.21.15.161:0/3293692018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa4143c40 0x7fdaa4154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.100+0000 7fdaaac00640 1 --2- 172.21.15.161:0/3293692018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa406c7e0 0x7fdaa41526d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.100+0000 7fdaaac00640 1 -- 172.21.15.161:0/3293692018 >> 172.21.15.161:0/3293692018 conn(0x7fdaa406a2d0 msgr2=0x7fdaa40cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:37.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.100+0000 7fdaaac00640 1 -- 172.21.15.161:0/3293692018 shutdown_connections 2024-05-22T21:14:37.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.100+0000 7fdaaac00640 1 -- 172.21.15.161:0/3293692018 wait complete. 2024-05-22T21:14:37.120 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:14:37.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3499e00640 1 Processor -- start 2024-05-22T21:14:37.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3499e00640 1 -- start start 2024-05-22T21:14:37.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3499e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f34940b26f0 0x7f349406aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3499e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f349406b420 0x7f34940ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3499e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f349406b860 con 0x7f349415d180 2024-05-22T21:14:37.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3499e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349406c390 con 0x7f34940b26f0 2024-05-22T21:14:37.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3499e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34940b2b80 con 0x7f349406b420 2024-05-22T21:14:37.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3493400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f349406b420 0x7f34940ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3493400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f349406b420 0x7f34940ad570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:35158/0 (socket says 172.21.15.161:35158) 2024-05-22T21:14:37.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3493400640 1 -- 172.21.15.161:0/2759782058 learned_addr learned my addr 172.21.15.161:0/2759782058 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:37.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3492000640 1 --2- 172.21.15.161:0/2759782058 >> v2:172.21.15.179:3300/0 conn(0x7f34940b26f0 0x7f349406aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3491600640 1 -- 172.21.15.161:0/2759782058 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3692361092 0 0) 0x7f349406b860 con 0x7f349415d180 2024-05-22T21:14:37.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3491600640 1 -- 172.21.15.161:0/2759782058 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3478003530 con 0x7f349415d180 2024-05-22T21:14:37.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3493400640 1 -- 172.21.15.161:0/2759782058 >> v2:172.21.15.179:3300/0 conn(0x7f34940b26f0 msgr2=0x7f349406aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:37.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3493400640 1 --2- 172.21.15.161:0/2759782058 >> v2:172.21.15.179:3300/0 conn(0x7f34940b26f0 0x7f349406aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3493400640 1 -- 172.21.15.161:0/2759782058 >> v1:172.21.15.179:6789/0 conn(0x7f349415d180 legacy=0x7f349406d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.256+0000 7f3493400640 1 -- 172.21.15.161:0/2759782058 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3494161c20 con 0x7f349406b420 2024-05-22T21:14:37.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.260+0000 7f3493400640 1 --2- 172.21.15.161:0/2759782058 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f349406b420 0x7f34940ad570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f348c009870 tx=0x7f348c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a993c8fd24944b44 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:37.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.260+0000 7f3491600640 1 -- 172.21.15.161:0/2759782058 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c0ad640 con 0x7f349406b420 2024-05-22T21:14:37.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.260+0000 7f3491600640 1 -- 172.21.15.161:0/2759782058 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c0ad7a0 con 0x7f349406b420 2024-05-22T21:14:37.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.260+0000 7f3491600640 1 -- 172.21.15.161:0/2759782058 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c0adb30 con 0x7f349406b420 2024-05-22T21:14:37.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.260+0000 7f3499e00640 1 -- 172.21.15.161:0/2759782058 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f349406b420 msgr2=0x7f34940ad570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.260+0000 7f3499e00640 1 --2- 172.21.15.161:0/2759782058 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f349406b420 0x7f34940ad570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f348c009870 tx=0x7f348c01c4a0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.260+0000 7f3499e00640 1 -- 172.21.15.161:0/2759782058 shutdown_connections 2024-05-22T21:14:37.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.260+0000 7f3499e00640 1 --2- 172.21.15.161:0/2759782058 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f349406b420 0x7f34940ad570 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.260+0000 7f3499e00640 1 --2- 172.21.15.161:0/2759782058 >> v2:172.21.15.179:3300/0 conn(0x7f34940b26f0 0x7f349406aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.260+0000 7f3499e00640 1 -- 172.21.15.161:0/2759782058 >> 172.21.15.161:0/2759782058 conn(0x7f3494070610 msgr2=0x7f3494070a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:37.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.260+0000 7f3499e00640 1 -- 172.21.15.161:0/2759782058 shutdown_connections 2024-05-22T21:14:37.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.260+0000 7f3499e00640 1 -- 172.21.15.161:0/2759782058 wait complete. 2024-05-22T21:14:37.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3499e00640 1 Processor -- start 2024-05-22T21:14:37.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3499e00640 1 -- start start 2024-05-22T21:14:37.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3499e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f349406b420 0x7f34941551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3499e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34940b26f0 0x7f349415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3499e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f349415d180 0x7f34941f9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3499e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349415eb50 con 0x7f349406b420 2024-05-22T21:14:37.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3499e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349415df10 con 0x7f349415d180 2024-05-22T21:14:37.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3499e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349415e1b0 con 0x7f34940b26f0 2024-05-22T21:14:37.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3492a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f349406b420 0x7f34941551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3492a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f349406b420 0x7f34941551d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57084/0 (socket says 172.21.15.161:57084) 2024-05-22T21:14:37.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3492a00640 1 -- 172.21.15.161:0/3986472494 learned_addr learned my addr 172.21.15.161:0/3986472494 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:37.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3492000640 1 --2- 172.21.15.161:0/3986472494 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34940b26f0 0x7f349415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3493400640 1 --2- 172.21.15.161:0/3986472494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f349415d180 0x7f34941f9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3492a00640 1 -- 172.21.15.161:0/3986472494 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34940b26f0 msgr2=0x7f349415b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3492a00640 1 --2- 172.21.15.161:0/3986472494 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34940b26f0 0x7f349415b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3492a00640 1 -- 172.21.15.161:0/3986472494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f349415d180 msgr2=0x7f34941f9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3492a00640 1 --2- 172.21.15.161:0/3986472494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f349415d180 0x7f34941f9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3492a00640 1 -- 172.21.15.161:0/3986472494 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c0094d0 con 0x7f349406b420 2024-05-22T21:14:37.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3493400640 1 --2- 172.21.15.161:0/3986472494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f349415d180 0x7f34941f9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:37.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3492a00640 1 --2- 172.21.15.161:0/3986472494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f349406b420 0x7f34941551d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f348400c770 tx=0x7f348400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:37.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f348b400640 1 -- 172.21.15.161:0/3986472494 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484003b80 con 0x7f349406b420 2024-05-22T21:14:37.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f348b400640 1 -- 172.21.15.161:0/3986472494 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3484003ce0 con 0x7f349406b420 2024-05-22T21:14:37.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f348b400640 1 -- 172.21.15.161:0/3986472494 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348400aa90 con 0x7f349406b420 2024-05-22T21:14:37.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3499e00640 1 -- 172.21.15.161:0/3986472494 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349415e490 con 0x7f349406b420 2024-05-22T21:14:37.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.264+0000 7f3499e00640 1 -- 172.21.15.161:0/3986472494 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34941639d0 con 0x7f349406b420 2024-05-22T21:14:37.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.268+0000 7f3499e00640 1 -- 172.21.15.161:0/3986472494 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3494067220 con 0x7f349406b420 2024-05-22T21:14:37.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.268+0000 7f348b400640 1 -- 172.21.15.161:0/3986472494 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f348400abf0 con 0x7f349406b420 2024-05-22T21:14:37.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.268+0000 7f348b400640 1 --2- 172.21.15.161:0/3986472494 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3470066690 0x7f3470068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.268+0000 7f348b400640 1 -- 172.21.15.161:0/3986472494 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34840840b0 con 0x7f349406b420 2024-05-22T21:14:37.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.268+0000 7f3492000640 1 --2- 172.21.15.161:0/3986472494 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3470066690 0x7f3470068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.268+0000 7f3492000640 1 --2- 172.21.15.161:0/3986472494 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3470066690 0x7f3470068b50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3494153c40 tx=0x7f347c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:37.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.272+0000 7f348b400640 1 -- 172.21.15.161:0/3986472494 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3484051410 con 0x7f349406b420 2024-05-22T21:14:37.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.424+0000 7f3499e00640 1 -- 172.21.15.161:0/3986472494 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f349406dd70 con 0x7f349406b420 2024-05-22T21:14:37.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.424+0000 7f348b400640 1 -- 172.21.15.161:0/3986472494 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3484054a30 con 0x7f349406b420 2024-05-22T21:14:37.427 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:14:37.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.428+0000 7f3499e00640 1 -- 172.21.15.161:0/3986472494 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3470066690 msgr2=0x7f3470068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.428+0000 7f3499e00640 1 --2- 172.21.15.161:0/3986472494 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3470066690 0x7f3470068b50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3494153c40 tx=0x7f347c008040 comp rx=0 tx=0).stop 2024-05-22T21:14:37.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.428+0000 7f3499e00640 1 -- 172.21.15.161:0/3986472494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f349406b420 msgr2=0x7f34941551d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.428+0000 7f3499e00640 1 --2- 172.21.15.161:0/3986472494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f349406b420 0x7f34941551d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f348400c770 tx=0x7f348400cc30 comp rx=0 tx=0).stop 2024-05-22T21:14:37.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.428+0000 7f3499e00640 1 -- 172.21.15.161:0/3986472494 shutdown_connections 2024-05-22T21:14:37.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.428+0000 7f3499e00640 1 --2- 172.21.15.161:0/3986472494 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3470066690 0x7f3470068b50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.428+0000 7f3499e00640 1 --2- 172.21.15.161:0/3986472494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f349415d180 0x7f34941f9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.428+0000 7f3499e00640 1 --2- 172.21.15.161:0/3986472494 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34940b26f0 0x7f349415b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.428+0000 7f3499e00640 1 --2- 172.21.15.161:0/3986472494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f349406b420 0x7f34941551d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.428+0000 7f3499e00640 1 -- 172.21.15.161:0/3986472494 >> 172.21.15.161:0/3986472494 conn(0x7f3494070610 msgr2=0x7f34940b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:37.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.428+0000 7f3499e00640 1 -- 172.21.15.161:0/3986472494 shutdown_connections 2024-05-22T21:14:37.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.428+0000 7f3499e00640 1 -- 172.21.15.161:0/3986472494 wait complete. 2024-05-22T21:14:37.449 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:14:37.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.580+0000 7f0f9a400640 1 Processor -- start 2024-05-22T21:14:37.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.580+0000 7f0f9a400640 1 -- start start 2024-05-22T21:14:37.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.584+0000 7f0f9a400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0f9415e130 0x7f0f9415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.584+0000 7f0f9a400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0f9415d410 0x7f0f9415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.584+0000 7f0f9a400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f94071670 con 0x7f0f9415dd50 2024-05-22T21:14:37.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.584+0000 7f0f9a400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f9415d870 con 0x7f0f9415e130 2024-05-22T21:14:37.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.584+0000 7f0f9a400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f94070a60 con 0x7f0f9415d410 2024-05-22T21:14:37.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.584+0000 7f0f8aa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0f9415e130 0x7f0f9415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.584+0000 7f0f8aa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0f9415e130 0x7f0f9415ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57098/0 (socket says 172.21.15.161:57098) 2024-05-22T21:14:37.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.584+0000 7f0f93400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0f9415dd50 0x7f0f9406d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41854/0 (socket says 172.21.15.161:41854) 2024-05-22T21:14:37.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.584+0000 7f0f8aa00640 1 -- 172.21.15.161:0/2402245446 learned_addr learned my addr 172.21.15.161:0/2402245446 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:37.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.584+0000 7f0f93e00640 1 --2- 172.21.15.161:0/2402245446 >> v2:172.21.15.179:3300/0 conn(0x7f0f9415d410 0x7f0f9415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.584+0000 7f0f8aa00640 1 -- 172.21.15.161:0/2402245446 >> v2:172.21.15.179:3300/0 conn(0x7f0f9415d410 msgr2=0x7f0f9415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:37.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.584+0000 7f0f8aa00640 1 --2- 172.21.15.161:0/2402245446 >> v2:172.21.15.179:3300/0 conn(0x7f0f9415d410 0x7f0f9415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.584+0000 7f0f8aa00640 1 -- 172.21.15.161:0/2402245446 >> v1:172.21.15.161:6789/0 conn(0x7f0f9415dd50 legacy=0x7f0f9406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.584+0000 7f0f8aa00640 1 -- 172.21.15.161:0/2402245446 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f94070d40 con 0x7f0f9415e130 2024-05-22T21:14:37.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.584+0000 7f0f92a00640 1 -- 172.21.15.161:0/2402245446 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601903337 0 0) 0x7f0f94071670 con 0x7f0f9415dd50 2024-05-22T21:14:37.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f8aa00640 1 --2- 172.21.15.161:0/2402245446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9415e130 0x7f0f9415ced0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0f9415e5c0 tx=0x7f0f7c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d596e1f573415d0b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:37.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f92a00640 1 -- 172.21.15.161:0/2402245446 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0aa650 con 0x7f0f9415e130 2024-05-22T21:14:37.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f92a00640 1 -- 172.21.15.161:0/2402245446 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7c0aa7b0 con 0x7f0f9415e130 2024-05-22T21:14:37.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f92a00640 1 -- 172.21.15.161:0/2402245446 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0aab40 con 0x7f0f9415e130 2024-05-22T21:14:37.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f9a400640 1 -- 172.21.15.161:0/2402245446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9415e130 msgr2=0x7f0f9415ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f9a400640 1 --2- 172.21.15.161:0/2402245446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9415e130 0x7f0f9415ced0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0f9415e5c0 tx=0x7f0f7c01c4f0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f9a400640 1 -- 172.21.15.161:0/2402245446 shutdown_connections 2024-05-22T21:14:37.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f9a400640 1 --2- 172.21.15.161:0/2402245446 >> v2:172.21.15.179:3300/0 conn(0x7f0f9415d410 0x7f0f9415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f9a400640 1 --2- 172.21.15.161:0/2402245446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9415e130 0x7f0f9415ced0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f9a400640 1 -- 172.21.15.161:0/2402245446 >> 172.21.15.161:0/2402245446 conn(0x7f0f940b0bc0 msgr2=0x7f0f940b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:37.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f9a400640 1 -- 172.21.15.161:0/2402245446 shutdown_connections 2024-05-22T21:14:37.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f9a400640 1 -- 172.21.15.161:0/2402245446 wait complete. 2024-05-22T21:14:37.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f9a400640 1 Processor -- start 2024-05-22T21:14:37.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f9a400640 1 -- start start 2024-05-22T21:14:37.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f9a400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9415d410 0x7f0f941f8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f9a400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f9415dd50 0x7f0f94158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f9a400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f9415e130 0x7f0f94150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f9a400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f94158c90 con 0x7f0f9415d410 2024-05-22T21:14:37.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f9a400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f941507c0 con 0x7f0f9415e130 2024-05-22T21:14:37.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.588+0000 7f0f9a400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f94150ec0 con 0x7f0f9415dd50 2024-05-22T21:14:37.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f93400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9415d410 0x7f0f941f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f93400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9415d410 0x7f0f941f8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57110/0 (socket says 172.21.15.161:57110) 2024-05-22T21:14:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f93400640 1 -- 172.21.15.161:0/3661826100 learned_addr learned my addr 172.21.15.161:0/3661826100 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f93e00640 1 --2- 172.21.15.161:0/3661826100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f9415e130 0x7f0f94150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f8aa00640 1 --2- 172.21.15.161:0/3661826100 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f9415dd50 0x7f0f94158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f93400640 1 -- 172.21.15.161:0/3661826100 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f9415dd50 msgr2=0x7f0f94158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f93400640 1 --2- 172.21.15.161:0/3661826100 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f9415dd50 0x7f0f94158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f93400640 1 -- 172.21.15.161:0/3661826100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f9415e130 msgr2=0x7f0f94150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f93400640 1 --2- 172.21.15.161:0/3661826100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f9415e130 0x7f0f94150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f93400640 1 -- 172.21.15.161:0/3661826100 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c0094d0 con 0x7f0f9415d410 2024-05-22T21:14:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f93400640 1 --2- 172.21.15.161:0/3661826100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9415d410 0x7f0f941f8dc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0f840011f0 tx=0x7f0f8400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f8be00640 1 -- 172.21.15.161:0/3661826100 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f840074e0 con 0x7f0f9415d410 2024-05-22T21:14:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f8be00640 1 -- 172.21.15.161:0/3661826100 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f84007640 con 0x7f0f9415d410 2024-05-22T21:14:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f9a400640 1 -- 172.21.15.161:0/3661826100 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9414ea00 con 0x7f0f9415d410 2024-05-22T21:14:37.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f8be00640 1 -- 172.21.15.161:0/3661826100 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84004a10 con 0x7f0f9415d410 2024-05-22T21:14:37.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f9a400640 1 -- 172.21.15.161:0/3661826100 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f941f9450 con 0x7f0f9415d410 2024-05-22T21:14:37.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f9a400640 1 -- 172.21.15.161:0/3661826100 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f94067220 con 0x7f0f9415d410 2024-05-22T21:14:37.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f8be00640 1 -- 172.21.15.161:0/3661826100 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0f84004b70 con 0x7f0f9415d410 2024-05-22T21:14:37.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.592+0000 7f0f8be00640 1 --2- 172.21.15.161:0/3661826100 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f70066690 0x7f0f70068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.596+0000 7f0f8be00640 1 -- 172.21.15.161:0/3661826100 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f840836f0 con 0x7f0f9415d410 2024-05-22T21:14:37.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.596+0000 7f0f8aa00640 1 --2- 172.21.15.161:0/3661826100 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f70066690 0x7f0f70068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.596+0000 7f0f8aa00640 1 --2- 172.21.15.161:0/3661826100 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f70066690 0x7f0f70068b50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0ae000 tx=0x7f0f7c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:37.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.600+0000 7f0f8be00640 1 -- 172.21.15.161:0/3661826100 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0f84050a60 con 0x7f0f9415d410 2024-05-22T21:14:37.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.784+0000 7f0f9a400640 1 -- 172.21.15.161:0/3661826100 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f94071e60 con 0x7f0f9415d410 2024-05-22T21:14:37.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.784+0000 7f0f8be00640 1 -- 172.21.15.161:0/3661826100 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f0f84018090 con 0x7f0f9415d410 2024-05-22T21:14:37.788 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:37.789 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3742018925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":3742018925},{"type":"v1","addr":"172.21.15.161:6837","nonce":3742018925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2553366378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2553366378},{"type":"v1","addr":"172.21.15.179:6834","nonce":2553366378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/509468338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":509468338},{"type":"v1","addr":"172.21.15.179:6835","nonce":509468338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:19.053259+0000","modified":"2024-05-22T21:14:22.076119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6836/1635458417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1635458417},{"type":"v1","addr":"172.21.15.161:6836","nonce":1635458417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-22T21:14:37.789 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 16 2024-05-22T21:14:37.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.788+0000 7f0f9a400640 1 -- 172.21.15.161:0/3661826100 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f70066690 msgr2=0x7f0f70068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.788+0000 7f0f9a400640 1 --2- 172.21.15.161:0/3661826100 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f70066690 0x7f0f70068b50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0ae000 tx=0x7f0f7c0a7000 comp rx=0 tx=0).stop 2024-05-22T21:14:37.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.788+0000 7f0f9a400640 1 -- 172.21.15.161:0/3661826100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9415d410 msgr2=0x7f0f941f8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.788+0000 7f0f9a400640 1 --2- 172.21.15.161:0/3661826100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9415d410 0x7f0f941f8dc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0f840011f0 tx=0x7f0f8400c9b0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.788+0000 7f0f9a400640 1 -- 172.21.15.161:0/3661826100 shutdown_connections 2024-05-22T21:14:37.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.788+0000 7f0f9a400640 1 --2- 172.21.15.161:0/3661826100 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f70066690 0x7f0f70068b50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.788+0000 7f0f9a400640 1 --2- 172.21.15.161:0/3661826100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f9415e130 0x7f0f94150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.788+0000 7f0f9a400640 1 --2- 172.21.15.161:0/3661826100 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f9415dd50 0x7f0f94158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.788+0000 7f0f9a400640 1 --2- 172.21.15.161:0/3661826100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f9415d410 0x7f0f941f8dc0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.788+0000 7f0f9a400640 1 -- 172.21.15.161:0/3661826100 >> 172.21.15.161:0/3661826100 conn(0x7f0f940b0bc0 msgr2=0x7f0f94153a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:37.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.788+0000 7f0f9a400640 1 -- 172.21.15.161:0/3661826100 shutdown_connections 2024-05-22T21:14:37.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.788+0000 7f0f9a400640 1 -- 172.21.15.161:0/3661826100 wait complete. 2024-05-22T21:14:37.809 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:37.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.940+0000 7fbd59000640 1 Processor -- start 2024-05-22T21:14:37.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.940+0000 7fbd59000640 1 -- start start 2024-05-22T21:14:37.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.940+0000 7fbd59000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbd541597f0 0x7fbd540706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.940+0000 7fbd59000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd54071040 con 0x7fbd540b3070 2024-05-22T21:14:37.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.940+0000 7fbd59000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd540b34e0 con 0x7fbd54070be0 2024-05-22T21:14:37.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.940+0000 7fbd59000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd54159bd0 con 0x7fbd541597f0 2024-05-22T21:14:37.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.944+0000 7fbd52800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fbd540b3070 0x7fbd540cc550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41864/0 (socket says 172.21.15.161:41864) 2024-05-22T21:14:37.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.944+0000 7fbd52800640 1 -- 172.21.15.161:0/3172780477 learned_addr learned my addr 172.21.15.161:0/3172780477 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:37.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.944+0000 7fbd51e00640 1 --2- 172.21.15.161:0/3172780477 >> v2:172.21.15.179:3300/0 conn(0x7fbd541597f0 0x7fbd540706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.944+0000 7fbd50a00640 1 -- 172.21.15.161:0/3172780477 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623765392 0 0) 0x7fbd54071040 con 0x7fbd540b3070 2024-05-22T21:14:37.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.944+0000 7fbd50a00640 1 -- 172.21.15.161:0/3172780477 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd38003450 con 0x7fbd540b3070 2024-05-22T21:14:37.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.944+0000 7fbd50a00640 1 -- 172.21.15.161:0/3172780477 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351370999 0 0) 0x7fbd40001120 con 0x7fbd54070be0 2024-05-22T21:14:37.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.944+0000 7fbd50a00640 1 -- 172.21.15.161:0/3172780477 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd54071040 con 0x7fbd54070be0 2024-05-22T21:14:37.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.944+0000 7fbd51e00640 1 -- 172.21.15.161:0/3172780477 >> v1:172.21.15.179:6789/0 conn(0x7fbd54070be0 legacy=0x7fbd540b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.944+0000 7fbd51e00640 1 -- 172.21.15.161:0/3172780477 >> v1:172.21.15.161:6789/0 conn(0x7fbd540b3070 legacy=0x7fbd540cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.944+0000 7fbd50a00640 1 -- 172.21.15.161:0/3172780477 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3148331651 0 0) 0x7fbd38003450 con 0x7fbd540b3070 2024-05-22T21:14:37.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.944+0000 7fbd51e00640 1 -- 172.21.15.161:0/3172780477 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd540cbdd0 con 0x7fbd541597f0 2024-05-22T21:14:37.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.944+0000 7fbd51e00640 1 --2- 172.21.15.161:0/3172780477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd541597f0 0x7fbd540706a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009d90 tx=0x7fbd4401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce181eb9eab573a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:37.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd50a00640 1 -- 172.21.15.161:0/3172780477 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c001680 con 0x7fbd541597f0 2024-05-22T21:14:37.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd50a00640 1 -- 172.21.15.161:0/3172780477 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd440a5650 con 0x7fbd541597f0 2024-05-22T21:14:37.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd50a00640 1 -- 172.21.15.161:0/3172780477 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd440a59e0 con 0x7fbd541597f0 2024-05-22T21:14:37.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd59000640 1 -- 172.21.15.161:0/3172780477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd541597f0 msgr2=0x7fbd540706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd59000640 1 --2- 172.21.15.161:0/3172780477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd541597f0 0x7fbd540706a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009d90 tx=0x7fbd4401c890 comp rx=0 tx=0).stop 2024-05-22T21:14:37.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd59000640 1 -- 172.21.15.161:0/3172780477 shutdown_connections 2024-05-22T21:14:37.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd59000640 1 --2- 172.21.15.161:0/3172780477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd541597f0 0x7fbd540706a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd59000640 1 -- 172.21.15.161:0/3172780477 >> 172.21.15.161:0/3172780477 conn(0x7fbd540b0bc0 msgr2=0x7fbd540b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:37.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd59000640 1 -- 172.21.15.161:0/3172780477 shutdown_connections 2024-05-22T21:14:37.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd59000640 1 -- 172.21.15.161:0/3172780477 wait complete. 2024-05-22T21:14:37.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd59000640 1 Processor -- start 2024-05-22T21:14:37.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd59000640 1 -- start start 2024-05-22T21:14:37.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd59000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd54070be0 0x7fbd54140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd59000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd540b3070 0x7fbd541429e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd59000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbd541597f0 0x7fbd54159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd59000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd541466a0 con 0x7fbd540b3070 2024-05-22T21:14:37.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd59000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd54141220 con 0x7fbd54070be0 2024-05-22T21:14:37.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd59000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd54142f20 con 0x7fbd541597f0 2024-05-22T21:14:37.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd51400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd540b3070 0x7fbd541429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd51400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd540b3070 0x7fbd541429e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57120/0 (socket says 172.21.15.161:57120) 2024-05-22T21:14:37.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd51400640 1 -- 172.21.15.161:0/523162194 learned_addr learned my addr 172.21.15.161:0/523162194 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:37.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd52800640 1 --2- 172.21.15.161:0/523162194 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbd541597f0 0x7fbd54159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.948+0000 7fbd51e00640 1 --2- 172.21.15.161:0/523162194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd54070be0 0x7fbd54140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.952+0000 7fbd51400640 1 -- 172.21.15.161:0/523162194 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbd541597f0 msgr2=0x7fbd54159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.952+0000 7fbd51400640 1 --2- 172.21.15.161:0/523162194 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbd541597f0 0x7fbd54159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.952+0000 7fbd51400640 1 -- 172.21.15.161:0/523162194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd54070be0 msgr2=0x7fbd54140650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:37.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.952+0000 7fbd51400640 1 --2- 172.21.15.161:0/523162194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd54070be0 0x7fbd54140650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:37.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.952+0000 7fbd51400640 1 -- 172.21.15.161:0/523162194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd44009a70 con 0x7fbd540b3070 2024-05-22T21:14:37.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.952+0000 7fbd51e00640 1 --2- 172.21.15.161:0/523162194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd54070be0 0x7fbd54140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:37.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.952+0000 7fbd51400640 1 --2- 172.21.15.161:0/523162194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd540b3070 0x7fbd541429e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000c700 tx=0x7fbd4000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:37.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.952+0000 7fbd4a000640 1 -- 172.21.15.161:0/523162194 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd400052d0 con 0x7fbd540b3070 2024-05-22T21:14:37.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.952+0000 7fbd4a000640 1 -- 172.21.15.161:0/523162194 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd40005450 con 0x7fbd540b3070 2024-05-22T21:14:37.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.952+0000 7fbd59000640 1 -- 172.21.15.161:0/523162194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd54140c60 con 0x7fbd540b3070 2024-05-22T21:14:37.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.952+0000 7fbd4a000640 1 -- 172.21.15.161:0/523162194 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd40003820 con 0x7fbd540b3070 2024-05-22T21:14:37.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.952+0000 7fbd59000640 1 -- 172.21.15.161:0/523162194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd54146880 con 0x7fbd540b3070 2024-05-22T21:14:37.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.952+0000 7fbd59000640 1 -- 172.21.15.161:0/523162194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd54067220 con 0x7fbd540b3070 2024-05-22T21:14:37.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.952+0000 7fbd4a000640 1 -- 172.21.15.161:0/523162194 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbd40003980 con 0x7fbd540b3070 2024-05-22T21:14:37.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.952+0000 7fbd4a000640 1 --2- 172.21.15.161:0/523162194 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbd340666c0 0x7fbd34068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:37.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.952+0000 7fbd4a000640 1 -- 172.21.15.161:0/523162194 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd40083c40 con 0x7fbd540b3070 2024-05-22T21:14:37.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.956+0000 7fbd51e00640 1 --2- 172.21.15.161:0/523162194 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbd340666c0 0x7fbd34068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:37.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.956+0000 7fbd51e00640 1 --2- 172.21.15.161:0/523162194 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbd340666c0 0x7fbd34068b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbd440af000 tx=0x7fbd440a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:37.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:37.956+0000 7fbd4a000640 1 -- 172.21.15.161:0/523162194 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbd40050e80 con 0x7fbd540b3070 2024-05-22T21:14:38.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.108+0000 7fbd59000640 1 -- 172.21.15.161:0/523162194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd540b34e0 con 0x7fbd540b3070 2024-05-22T21:14:38.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.112+0000 7fbd4a000640 1 -- 172.21.15.161:0/523162194 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16149 (secure 0 0 0) 0x7fbd540b34e0 con 0x7fbd540b3070 2024-05-22T21:14:38.113 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:38.114 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":35,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:14:30.106879+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:16.938769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:18.054579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:38.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.112+0000 7fbd59000640 1 -- 172.21.15.161:0/523162194 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbd340666c0 msgr2=0x7fbd34068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:38.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.112+0000 7fbd59000640 1 --2- 172.21.15.161:0/523162194 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbd340666c0 0x7fbd34068b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbd440af000 tx=0x7fbd440a7070 comp rx=0 tx=0).stop 2024-05-22T21:14:38.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.112+0000 7fbd59000640 1 -- 172.21.15.161:0/523162194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd540b3070 msgr2=0x7fbd541429e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:38.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.112+0000 7fbd59000640 1 --2- 172.21.15.161:0/523162194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd540b3070 0x7fbd541429e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000c700 tx=0x7fbd4000cbc0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.112+0000 7fbd59000640 1 -- 172.21.15.161:0/523162194 shutdown_connections 2024-05-22T21:14:38.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.112+0000 7fbd59000640 1 --2- 172.21.15.161:0/523162194 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbd340666c0 0x7fbd34068b80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.112+0000 7fbd59000640 1 --2- 172.21.15.161:0/523162194 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbd541597f0 0x7fbd54159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.112+0000 7fbd59000640 1 --2- 172.21.15.161:0/523162194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd540b3070 0x7fbd541429e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.112+0000 7fbd59000640 1 --2- 172.21.15.161:0/523162194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd54070be0 0x7fbd54140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.112+0000 7fbd59000640 1 -- 172.21.15.161:0/523162194 >> 172.21.15.161:0/523162194 conn(0x7fbd540b0bc0 msgr2=0x7fbd540aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:38.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.112+0000 7fbd59000640 1 -- 172.21.15.161:0/523162194 shutdown_connections 2024-05-22T21:14:38.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.112+0000 7fbd59000640 1 -- 172.21.15.161:0/523162194 wait complete. 2024-05-22T21:14:38.134 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-22T21:14:38.135 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:14:38.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.268+0000 7f3ddec00640 1 Processor -- start 2024-05-22T21:14:38.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.268+0000 7f3ddec00640 1 -- start start 2024-05-22T21:14:38.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.268+0000 7f3ddec00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd80715a0 0x7f3dd815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:38.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.268+0000 7f3ddec00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3dd815ec90 0x7f3dd815f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:38.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.268+0000 7f3ddec00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd81442a0 con 0x7f3dd8143ec0 2024-05-22T21:14:38.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.268+0000 7f3ddec00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd815f660 con 0x7f3dd815ec90 2024-05-22T21:14:38.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.268+0000 7f3ddec00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd8070500 con 0x7f3dd80715a0 2024-05-22T21:14:38.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.272+0000 7f3ddcc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3dd815ec90 0x7f3dd815f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:38.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.272+0000 7f3ddcc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3dd815ec90 0x7f3dd815f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57122/0 (socket says 172.21.15.161:57122) 2024-05-22T21:14:38.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.272+0000 7f3ddcc00640 1 -- 172.21.15.161:0/2525252974 learned_addr learned my addr 172.21.15.161:0/2525252974 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:38.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.272+0000 7f3dcf400640 1 --2- 172.21.15.161:0/2525252974 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd80715a0 0x7f3dd815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:38.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.272+0000 7f3ddcc00640 1 -- 172.21.15.161:0/2525252974 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd80715a0 msgr2=0x7f3dd815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:38.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.272+0000 7f3ddcc00640 1 --2- 172.21.15.161:0/2525252974 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd80715a0 0x7f3dd815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.272+0000 7f3ddcc00640 1 -- 172.21.15.161:0/2525252974 >> v1:172.21.15.179:6789/0 conn(0x7f3dd8143ec0 legacy=0x7f3dd806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:38.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.272+0000 7f3ddcc00640 1 -- 172.21.15.161:0/2525252974 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd80707e0 con 0x7f3dd815ec90 2024-05-22T21:14:38.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.272+0000 7f3ddcc00640 1 --2- 172.21.15.161:0/2525252974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd815ec90 0x7f3dd815f120 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009e50 tx=0x7f3dd001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5726901c6ce59dbb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:38.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.272+0000 7f3dd7400640 1 -- 172.21.15.161:0/2525252974 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd00a57d0 con 0x7f3dd815ec90 2024-05-22T21:14:38.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.272+0000 7f3dd7400640 1 -- 172.21.15.161:0/2525252974 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd00a5930 con 0x7f3dd815ec90 2024-05-22T21:14:38.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.272+0000 7f3dd7400640 1 -- 172.21.15.161:0/2525252974 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd00a5cc0 con 0x7f3dd815ec90 2024-05-22T21:14:38.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddec00640 1 -- 172.21.15.161:0/2525252974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd815ec90 msgr2=0x7f3dd815f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:38.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddec00640 1 --2- 172.21.15.161:0/2525252974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd815ec90 0x7f3dd815f120 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009e50 tx=0x7f3dd001ca00 comp rx=0 tx=0).stop 2024-05-22T21:14:38.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddec00640 1 -- 172.21.15.161:0/2525252974 shutdown_connections 2024-05-22T21:14:38.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddec00640 1 --2- 172.21.15.161:0/2525252974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd815ec90 0x7f3dd815f120 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddec00640 1 --2- 172.21.15.161:0/2525252974 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd80715a0 0x7f3dd815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddec00640 1 -- 172.21.15.161:0/2525252974 >> 172.21.15.161:0/2525252974 conn(0x7f3dd80b0bc0 msgr2=0x7f3dd80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:38.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddec00640 1 -- 172.21.15.161:0/2525252974 shutdown_connections 2024-05-22T21:14:38.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddec00640 1 -- 172.21.15.161:0/2525252974 wait complete. 2024-05-22T21:14:38.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddec00640 1 Processor -- start 2024-05-22T21:14:38.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddec00640 1 -- start start 2024-05-22T21:14:38.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddec00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd8143ec0 0x7f3dd8142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:38.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddec00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd815ec90 0x7f3dd813ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:38.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddec00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd81f84a0 0x7f3dd813a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:38.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddec00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd8142590 con 0x7f3dd815ec90 2024-05-22T21:14:38.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddec00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd813f290 con 0x7f3dd8143ec0 2024-05-22T21:14:38.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddec00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd813f4a0 con 0x7f3dd81f84a0 2024-05-22T21:14:38.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3dcf400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd815ec90 0x7f3dd813ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:38.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3dcf400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd815ec90 0x7f3dd813ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57128/0 (socket says 172.21.15.161:57128) 2024-05-22T21:14:38.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3dcf400640 1 -- 172.21.15.161:0/3670384641 learned_addr learned my addr 172.21.15.161:0/3670384641 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:38.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3dd7e00640 1 --2- 172.21.15.161:0/3670384641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd8143ec0 0x7f3dd8142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:38.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3ddcc00640 1 --2- 172.21.15.161:0/3670384641 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd81f84a0 0x7f3dd813a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:38.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3dcf400640 1 -- 172.21.15.161:0/3670384641 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd81f84a0 msgr2=0x7f3dd813a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:38.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3dcf400640 1 --2- 172.21.15.161:0/3670384641 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd81f84a0 0x7f3dd813a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3dcf400640 1 -- 172.21.15.161:0/3670384641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd8143ec0 msgr2=0x7f3dd8142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:38.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3dcf400640 1 --2- 172.21.15.161:0/3670384641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd8143ec0 0x7f3dd8142050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3dcf400640 1 -- 172.21.15.161:0/3670384641 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd0009b30 con 0x7f3dd815ec90 2024-05-22T21:14:38.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.276+0000 7f3dd7e00640 1 --2- 172.21.15.161:0/3670384641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd8143ec0 0x7f3dd8142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:38.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.280+0000 7f3dcf400640 1 --2- 172.21.15.161:0/3670384641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd815ec90 0x7f3dd813ed50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000d800 tx=0x7f3dc000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:38.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.280+0000 7f3dd4c00640 1 -- 172.21.15.161:0/3670384641 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc000b6c0 con 0x7f3dd815ec90 2024-05-22T21:14:38.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.280+0000 7f3dd4c00640 1 -- 172.21.15.161:0/3670384641 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc000b820 con 0x7f3dd815ec90 2024-05-22T21:14:38.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.280+0000 7f3dd4c00640 1 -- 172.21.15.161:0/3670384641 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc000bb90 con 0x7f3dd815ec90 2024-05-22T21:14:38.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.280+0000 7f3ddec00640 1 -- 172.21.15.161:0/3670384641 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd813f780 con 0x7f3dd815ec90 2024-05-22T21:14:38.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.280+0000 7f3ddec00640 1 -- 172.21.15.161:0/3670384641 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd813d880 con 0x7f3dd815ec90 2024-05-22T21:14:38.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.280+0000 7f3ddec00640 1 -- 172.21.15.161:0/3670384641 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd8067220 con 0x7f3dd815ec90 2024-05-22T21:14:38.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.280+0000 7f3dd4c00640 1 -- 172.21.15.161:0/3670384641 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3dc000bcf0 con 0x7f3dd815ec90 2024-05-22T21:14:38.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.280+0000 7f3dd4c00640 1 --2- 172.21.15.161:0/3670384641 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3db4066690 0x7f3db4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:38.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.280+0000 7f3dd4c00640 1 -- 172.21.15.161:0/3670384641 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3dc00846a0 con 0x7f3dd815ec90 2024-05-22T21:14:38.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.280+0000 7f3dd7e00640 1 --2- 172.21.15.161:0/3670384641 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3db4066690 0x7f3db4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:38.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.280+0000 7f3dd7e00640 1 --2- 172.21.15.161:0/3670384641 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3db4066690 0x7f3db4068b50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8000f70 tx=0x7f3dc8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:38.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.284+0000 7f3dd4c00640 1 -- 172.21.15.161:0/3670384641 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3dc0051aa0 con 0x7f3dd815ec90 2024-05-22T21:14:38.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.472+0000 7f3ddec00640 1 -- 172.21.15.161:0/3670384641 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3dd806dcc0 con 0x7f3dd815ec90 2024-05-22T21:14:38.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.472+0000 7f3dd4c00640 1 -- 172.21.15.161:0/3670384641 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f3dc00550c0 con 0x7f3dd815ec90 2024-05-22T21:14:38.474 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:38.474 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3742018925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":3742018925},{"type":"v1","addr":"172.21.15.161:6837","nonce":3742018925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2553366378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2553366378},{"type":"v1","addr":"172.21.15.179:6834","nonce":2553366378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/509468338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":509468338},{"type":"v1","addr":"172.21.15.179:6835","nonce":509468338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:19.053259+0000","modified":"2024-05-22T21:14:22.076119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6836/1635458417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1635458417},{"type":"v1","addr":"172.21.15.161:6836","nonce":1635458417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-22T21:14:38.475 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 16 2024-05-22T21:14:38.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.472+0000 7f3ddec00640 1 -- 172.21.15.161:0/3670384641 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3db4066690 msgr2=0x7f3db4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:38.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.472+0000 7f3ddec00640 1 --2- 172.21.15.161:0/3670384641 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3db4066690 0x7f3db4068b50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8000f70 tx=0x7f3dc8009040 comp rx=0 tx=0).stop 2024-05-22T21:14:38.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.472+0000 7f3ddec00640 1 -- 172.21.15.161:0/3670384641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd815ec90 msgr2=0x7f3dd813ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.472+0000 7f3ddec00640 1 --2- 172.21.15.161:0/3670384641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd815ec90 0x7f3dd813ed50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000d800 tx=0x7f3dc000dcc0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.472+0000 7f3ddec00640 1 -- 172.21.15.161:0/3670384641 shutdown_connections 2024-05-22T21:14:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.472+0000 7f3ddec00640 1 --2- 172.21.15.161:0/3670384641 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3db4066690 0x7f3db4068b50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.472+0000 7f3ddec00640 1 --2- 172.21.15.161:0/3670384641 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd81f84a0 0x7f3dd813a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.472+0000 7f3ddec00640 1 --2- 172.21.15.161:0/3670384641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd815ec90 0x7f3dd813ed50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.472+0000 7f3ddec00640 1 --2- 172.21.15.161:0/3670384641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd8143ec0 0x7f3dd8142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.472+0000 7f3ddec00640 1 -- 172.21.15.161:0/3670384641 >> 172.21.15.161:0/3670384641 conn(0x7f3dd80b0bc0 msgr2=0x7f3dd806c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.472+0000 7f3ddec00640 1 -- 172.21.15.161:0/3670384641 shutdown_connections 2024-05-22T21:14:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.472+0000 7f3ddec00640 1 -- 172.21.15.161:0/3670384641 wait complete. 2024-05-22T21:14:38.496 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:14:38.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.628+0000 7fd7cbe00640 1 Processor -- start 2024-05-22T21:14:38.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.628+0000 7fd7cbe00640 1 -- start start 2024-05-22T21:14:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.628+0000 7fd7cbe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd7c40b3330 0x7fd7c406d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.628+0000 7fd7cbe00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c406c7b0 con 0x7fd7c41618b0 2024-05-22T21:14:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.628+0000 7fd7cbe00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c406ca90 con 0x7fd7c406fd20 2024-05-22T21:14:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.628+0000 7fd7cbe00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c406cd70 con 0x7fd7c40b3330 2024-05-22T21:14:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.632+0000 7fd7c8a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd7c40b3330 0x7fd7c406d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.632+0000 7fd7c8a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd7c40b3330 0x7fd7c406d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57138/0 (socket says 172.21.15.161:57138) 2024-05-22T21:14:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.632+0000 7fd7c8a00640 1 -- 172.21.15.161:0/3316065490 learned_addr learned my addr 172.21.15.161:0/3316065490 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.632+0000 7fd7c9400640 1 --1- 172.21.15.161:0/3316065490 >> v1:172.21.15.161:6789/0 conn(0x7fd7c41618b0 0x7fd7c406bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41866/0 (socket says 172.21.15.161:41866) 2024-05-22T21:14:38.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.632+0000 7fd7c8a00640 1 -- 172.21.15.161:0/3316065490 >> v1:172.21.15.179:6789/0 conn(0x7fd7c406fd20 legacy=0x7fd7c4070130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:38.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.632+0000 7fd7c8a00640 1 -- 172.21.15.161:0/3316065490 >> v1:172.21.15.161:6789/0 conn(0x7fd7c41618b0 legacy=0x7fd7c406bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:38.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.632+0000 7fd7c8a00640 1 -- 172.21.15.161:0/3316065490 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c406d050 con 0x7fd7c40b3330 2024-05-22T21:14:38.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.632+0000 7fd7c3e00640 1 -- 172.21.15.161:0/3316065490 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429679006 0 0) 0x7fd7c406c7b0 con 0x7fd7c41618b0 2024-05-22T21:14:38.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.632+0000 7fd7c8a00640 1 --2- 172.21.15.161:0/3316065490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c40b3330 0x7fd7c406d850 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009870 tx=0x7fd7b001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ce30f250a9a966 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:38.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.632+0000 7fd7c3e00640 1 -- 172.21.15.161:0/3316065490 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b00aa670 con 0x7fd7c40b3330 2024-05-22T21:14:38.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.632+0000 7fd7c3e00640 1 -- 172.21.15.161:0/3316065490 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b00aa7d0 con 0x7fd7c40b3330 2024-05-22T21:14:38.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.632+0000 7fd7c3e00640 1 -- 172.21.15.161:0/3316065490 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b00aab60 con 0x7fd7c40b3330 2024-05-22T21:14:38.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7cbe00640 1 -- 172.21.15.161:0/3316065490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c40b3330 msgr2=0x7fd7c406d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:38.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7cbe00640 1 --2- 172.21.15.161:0/3316065490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c40b3330 0x7fd7c406d850 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009870 tx=0x7fd7b001c4f0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7cbe00640 1 -- 172.21.15.161:0/3316065490 shutdown_connections 2024-05-22T21:14:38.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7cbe00640 1 --2- 172.21.15.161:0/3316065490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c40b3330 0x7fd7c406d850 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7cbe00640 1 -- 172.21.15.161:0/3316065490 >> 172.21.15.161:0/3316065490 conn(0x7fd7c40b0bc0 msgr2=0x7fd7c40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:38.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7cbe00640 1 -- 172.21.15.161:0/3316065490 shutdown_connections 2024-05-22T21:14:38.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7cbe00640 1 -- 172.21.15.161:0/3316065490 wait complete. 2024-05-22T21:14:38.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7cbe00640 1 Processor -- start 2024-05-22T21:14:38.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7cbe00640 1 -- start start 2024-05-22T21:14:38.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7cbe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7c406fd20 0x7fd7c4152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:38.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7cbe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7c40b3330 0x7fd7c41548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:38.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7cbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c41618b0 0x7fd7c4154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:38.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7cbe00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c41560e0 con 0x7fd7c41618b0 2024-05-22T21:14:38.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7cbe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c415ecd0 con 0x7fd7c406fd20 2024-05-22T21:14:38.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7cbe00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c4153560 con 0x7fd7c40b3330 2024-05-22T21:14:38.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7c9e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c41618b0 0x7fd7c4154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:38.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7c9e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c41618b0 0x7fd7c4154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57144/0 (socket says 172.21.15.161:57144) 2024-05-22T21:14:38.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7c9e00640 1 -- 172.21.15.161:0/3655110549 learned_addr learned my addr 172.21.15.161:0/3655110549 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:38.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7c9400640 1 --2- 172.21.15.161:0/3655110549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7c406fd20 0x7fd7c4152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:38.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7c8a00640 1 --2- 172.21.15.161:0/3655110549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7c40b3330 0x7fd7c41548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:38.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7c9e00640 1 -- 172.21.15.161:0/3655110549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7c40b3330 msgr2=0x7fd7c41548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:38.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7c9e00640 1 --2- 172.21.15.161:0/3655110549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7c40b3330 0x7fd7c41548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7c9e00640 1 -- 172.21.15.161:0/3655110549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7c406fd20 msgr2=0x7fd7c4152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:38.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7c9e00640 1 --2- 172.21.15.161:0/3655110549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7c406fd20 0x7fd7c4152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7c9e00640 1 -- 172.21.15.161:0/3655110549 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b00094d0 con 0x7fd7c41618b0 2024-05-22T21:14:38.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7c8a00640 1 --2- 172.21.15.161:0/3655110549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7c40b3330 0x7fd7c41548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:38.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7c9400640 1 --2- 172.21.15.161:0/3655110549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7c406fd20 0x7fd7c4152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:38.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.636+0000 7fd7c9e00640 1 --2- 172.21.15.161:0/3655110549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c41618b0 0x7fd7c4154e20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00b970 tx=0x7fd7bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:38.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.640+0000 7fd7c1600640 1 -- 172.21.15.161:0/3655110549 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc00cb40 con 0x7fd7c41618b0 2024-05-22T21:14:38.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.640+0000 7fd7c1600640 1 -- 172.21.15.161:0/3655110549 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc00cca0 con 0x7fd7c41618b0 2024-05-22T21:14:38.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.640+0000 7fd7c1600640 1 -- 172.21.15.161:0/3655110549 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc014680 con 0x7fd7c41618b0 2024-05-22T21:14:38.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.640+0000 7fd7cbe00640 1 -- 172.21.15.161:0/3655110549 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c415e9f0 con 0x7fd7c41618b0 2024-05-22T21:14:38.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.640+0000 7fd7cbe00640 1 -- 172.21.15.161:0/3655110549 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c4155d00 con 0x7fd7c41618b0 2024-05-22T21:14:38.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.640+0000 7fd7cbe00640 1 -- 172.21.15.161:0/3655110549 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c4067220 con 0x7fd7c41618b0 2024-05-22T21:14:38.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.640+0000 7fd7c1600640 1 -- 172.21.15.161:0/3655110549 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd7bc01d3b0 con 0x7fd7c41618b0 2024-05-22T21:14:38.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.640+0000 7fd7c1600640 1 --2- 172.21.15.161:0/3655110549 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7a4066690 0x7fd7a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:38.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.640+0000 7fd7c1600640 1 -- 172.21.15.161:0/3655110549 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7bc084070 con 0x7fd7c41618b0 2024-05-22T21:14:38.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.640+0000 7fd7c9400640 1 --2- 172.21.15.161:0/3655110549 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7a4066690 0x7fd7a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:38.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.640+0000 7fd7c9400640 1 --2- 172.21.15.161:0/3655110549 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7a4066690 0x7fd7a4068b50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8000f70 tx=0x7fd7b8007c40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:38.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.644+0000 7fd7c1600640 1 -- 172.21.15.161:0/3655110549 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd7bc0b5cf0 con 0x7fd7c41618b0 2024-05-22T21:14:38.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.832+0000 7fd7cbe00640 1 -- 172.21.15.161:0/3655110549 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7c40cebf0 con 0x7fd7c41618b0 2024-05-22T21:14:38.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.832+0000 7fd7c1600640 1 -- 172.21.15.161:0/3655110549 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4743 (secure 0 0 0) 0x7fd7bc051340 con 0x7fd7c41618b0 2024-05-22T21:14:38.834 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:38.835 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3742018925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":3742018925},{"type":"v1","addr":"172.21.15.161:6837","nonce":3742018925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2553366378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2553366378},{"type":"v1","addr":"172.21.15.179:6834","nonce":2553366378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/509468338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":509468338},{"type":"v1","addr":"172.21.15.179:6835","nonce":509468338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:19.053259+0000","modified":"2024-05-22T21:14:22.076119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6836/1635458417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1635458417},{"type":"v1","addr":"172.21.15.161:6836","nonce":1635458417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-22T21:14:38.835 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 16 2024-05-22T21:14:38.837 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.832+0000 7fd7cbe00640 1 -- 172.21.15.161:0/3655110549 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7a4066690 msgr2=0x7fd7a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:38.837 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.832+0000 7fd7cbe00640 1 --2- 172.21.15.161:0/3655110549 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7a4066690 0x7fd7a4068b50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8000f70 tx=0x7fd7b8007c40 comp rx=0 tx=0).stop 2024-05-22T21:14:38.837 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.832+0000 7fd7cbe00640 1 -- 172.21.15.161:0/3655110549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c41618b0 msgr2=0x7fd7c4154e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:38.837 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.832+0000 7fd7cbe00640 1 --2- 172.21.15.161:0/3655110549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c41618b0 0x7fd7c4154e20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00b970 tx=0x7fd7bc00be30 comp rx=0 tx=0).stop 2024-05-22T21:14:38.837 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.832+0000 7fd7cbe00640 1 -- 172.21.15.161:0/3655110549 shutdown_connections 2024-05-22T21:14:38.837 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.832+0000 7fd7cbe00640 1 --2- 172.21.15.161:0/3655110549 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7a4066690 0x7fd7a4068b50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.837 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.832+0000 7fd7cbe00640 1 --2- 172.21.15.161:0/3655110549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c41618b0 0x7fd7c4154e20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.837 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.832+0000 7fd7cbe00640 1 --2- 172.21.15.161:0/3655110549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7c40b3330 0x7fd7c41548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.837 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.832+0000 7fd7cbe00640 1 --2- 172.21.15.161:0/3655110549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd7c406fd20 0x7fd7c4152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.837 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.832+0000 7fd7cbe00640 1 -- 172.21.15.161:0/3655110549 >> 172.21.15.161:0/3655110549 conn(0x7fd7c40b0bc0 msgr2=0x7fd7c40aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:38.837 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.832+0000 7fd7cbe00640 1 -- 172.21.15.161:0/3655110549 shutdown_connections 2024-05-22T21:14:38.837 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.832+0000 7fd7cbe00640 1 -- 172.21.15.161:0/3655110549 wait complete. 2024-05-22T21:14:38.856 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:38.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.988+0000 7efea7600640 1 Processor -- start 2024-05-22T21:14:38.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.988+0000 7efea7600640 1 -- start start 2024-05-22T21:14:38.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.988+0000 7efea7600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efea015dd50 0x7efea006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:38.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.988+0000 7efea7600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efea015e130 0x7efea015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:38.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.988+0000 7efea7600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efea015d410 0x7efea015a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:38.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.988+0000 7efea7600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea0071670 con 0x7efea015e130 2024-05-22T21:14:38.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.988+0000 7efea7600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea015d8a0 con 0x7efea015d410 2024-05-22T21:14:38.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.988+0000 7efea7600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea006de40 con 0x7efea015dd50 2024-05-22T21:14:38.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.992+0000 7efe9fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efea015e130 0x7efea015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:38.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.992+0000 7efea5600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efea015d410 0x7efea015a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:38.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.992+0000 7efe9fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efea015e130 0x7efea015ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57146/0 (socket says 172.21.15.161:57146) 2024-05-22T21:14:38.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.992+0000 7efe9fe00640 1 -- 172.21.15.161:0/1350518324 learned_addr learned my addr 172.21.15.161:0/1350518324 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:38.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.992+0000 7efea4c00640 1 --2- 172.21.15.161:0/1350518324 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efea015dd50 0x7efea006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:38.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.992+0000 7efe9fe00640 1 -- 172.21.15.161:0/1350518324 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efea015dd50 msgr2=0x7efea006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:38.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.992+0000 7efe9fe00640 1 --2- 172.21.15.161:0/1350518324 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efea015dd50 0x7efea006d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.992+0000 7efe9fe00640 1 -- 172.21.15.161:0/1350518324 >> v2:172.21.15.179:3300/0 conn(0x7efea015d410 msgr2=0x7efea015a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:38.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.992+0000 7efe9fe00640 1 --2- 172.21.15.161:0/1350518324 >> v2:172.21.15.179:3300/0 conn(0x7efea015d410 0x7efea015a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.992+0000 7efe9fe00640 1 -- 172.21.15.161:0/1350518324 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea0070a60 con 0x7efea015e130 2024-05-22T21:14:38.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.992+0000 7efe9fe00640 1 --2- 172.21.15.161:0/1350518324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea015e130 0x7efea015ced0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efea00721d0 tx=0x7efe8c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5268bef76c3beeb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:38.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.992+0000 7efe9f400640 1 -- 172.21.15.161:0/1350518324 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0ab620 con 0x7efea015e130 2024-05-22T21:14:38.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.992+0000 7efe9f400640 1 -- 172.21.15.161:0/1350518324 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8c0ab780 con 0x7efea015e130 2024-05-22T21:14:38.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.992+0000 7efe9f400640 1 -- 172.21.15.161:0/1350518324 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0abb10 con 0x7efea015e130 2024-05-22T21:14:38.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 -- 172.21.15.161:0/1350518324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea015e130 msgr2=0x7efea015ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:38.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 --2- 172.21.15.161:0/1350518324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea015e130 0x7efea015ced0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efea00721d0 tx=0x7efe8c01c460 comp rx=0 tx=0).stop 2024-05-22T21:14:38.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 -- 172.21.15.161:0/1350518324 shutdown_connections 2024-05-22T21:14:38.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 --2- 172.21.15.161:0/1350518324 >> v2:172.21.15.179:3300/0 conn(0x7efea015d410 0x7efea015a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 --2- 172.21.15.161:0/1350518324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea015e130 0x7efea015ced0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 --2- 172.21.15.161:0/1350518324 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efea015dd50 0x7efea006d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:38.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 -- 172.21.15.161:0/1350518324 >> 172.21.15.161:0/1350518324 conn(0x7efea00b0bc0 msgr2=0x7efea00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:38.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 -- 172.21.15.161:0/1350518324 shutdown_connections 2024-05-22T21:14:38.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 -- 172.21.15.161:0/1350518324 wait complete. 2024-05-22T21:14:38.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 Processor -- start 2024-05-22T21:14:38.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 -- start start 2024-05-22T21:14:39.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efea015d410 0x7efea0155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:39.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efea015dd50 0x7efea015f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:39.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea015e130 0x7efea00b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:39.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea0156240 con 0x7efea015e130 2024-05-22T21:14:39.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea00b0fa0 con 0x7efea015dd50 2024-05-22T21:14:39.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea7600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea00b1280 con 0x7efea015d410 2024-05-22T21:14:39.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea5600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea015e130 0x7efea00b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:39.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea5600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea015e130 0x7efea00b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57154/0 (socket says 172.21.15.161:57154) 2024-05-22T21:14:39.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea5600640 1 -- 172.21.15.161:0/2296734425 learned_addr learned my addr 172.21.15.161:0/2296734425 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:39.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efe9fe00640 1 --2- 172.21.15.161:0/2296734425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efea015dd50 0x7efea015f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:39.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea4c00640 1 --2- 172.21.15.161:0/2296734425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efea015d410 0x7efea0155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:39.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea5600640 1 -- 172.21.15.161:0/2296734425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efea015d410 msgr2=0x7efea0155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:39.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea5600640 1 --2- 172.21.15.161:0/2296734425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efea015d410 0x7efea0155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea5600640 1 -- 172.21.15.161:0/2296734425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efea015dd50 msgr2=0x7efea015f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:39.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:38.996+0000 7efea5600640 1 --2- 172.21.15.161:0/2296734425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efea015dd50 0x7efea015f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.000+0000 7efea5600640 1 -- 172.21.15.161:0/2296734425 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c0094d0 con 0x7efea015e130 2024-05-22T21:14:39.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.000+0000 7efea4c00640 1 --2- 172.21.15.161:0/2296734425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efea015d410 0x7efea0155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:39.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.000+0000 7efea5600640 1 --2- 172.21.15.161:0/2296734425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea015e130 0x7efea00b2320 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7efe9800efc0 tx=0x7efe9800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:39.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.000+0000 7efe9cc00640 1 -- 172.21.15.161:0/2296734425 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98014480 con 0x7efea015e130 2024-05-22T21:14:39.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.000+0000 7efe9cc00640 1 -- 172.21.15.161:0/2296734425 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9800f040 con 0x7efea015e130 2024-05-22T21:14:39.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.000+0000 7efe9cc00640 1 -- 172.21.15.161:0/2296734425 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98014820 con 0x7efea015e130 2024-05-22T21:14:39.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.000+0000 7efea7600640 1 -- 172.21.15.161:0/2296734425 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea00b1560 con 0x7efea015e130 2024-05-22T21:14:39.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.000+0000 7efea7600640 1 -- 172.21.15.161:0/2296734425 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea00b1b20 con 0x7efea015e130 2024-05-22T21:14:39.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.000+0000 7efea7600640 1 -- 172.21.15.161:0/2296734425 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea0067220 con 0x7efea015e130 2024-05-22T21:14:39.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.000+0000 7efe9cc00640 1 -- 172.21.15.161:0/2296734425 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efe98014980 con 0x7efea015e130 2024-05-22T21:14:39.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.000+0000 7efe9cc00640 1 --2- 172.21.15.161:0/2296734425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efe80066690 0x7efe80068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:39.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.000+0000 7efe9cc00640 1 -- 172.21.15.161:0/2296734425 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe98084d70 con 0x7efea015e130 2024-05-22T21:14:39.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.000+0000 7efea4c00640 1 --2- 172.21.15.161:0/2296734425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efe80066690 0x7efe80068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:39.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.004+0000 7efea4c00640 1 --2- 172.21.15.161:0/2296734425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efe80066690 0x7efe80068b50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efea00b3fa0 tx=0x7efe90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:39.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.004+0000 7efe9cc00640 1 -- 172.21.15.161:0/2296734425 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efe98052040 con 0x7efea015e130 2024-05-22T21:14:39.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.159+0000 7efea7600640 1 -- 172.21.15.161:0/2296734425 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efea015a650 con 0x7efea015e130 2024-05-22T21:14:39.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.163+0000 7efe9cc00640 1 -- 172.21.15.161:0/2296734425 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16149 (secure 0 0 0) 0x7efe98018090 con 0x7efea015e130 2024-05-22T21:14:39.165 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:39.165 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":35,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:14:30.106879+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:16.938769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:18.054579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:39.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.163+0000 7efea7600640 1 -- 172.21.15.161:0/2296734425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efe80066690 msgr2=0x7efe80068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:39.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.163+0000 7efea7600640 1 --2- 172.21.15.161:0/2296734425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efe80066690 0x7efe80068b50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efea00b3fa0 tx=0x7efe90008040 comp rx=0 tx=0).stop 2024-05-22T21:14:39.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.163+0000 7efea7600640 1 -- 172.21.15.161:0/2296734425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea015e130 msgr2=0x7efea00b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:39.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.163+0000 7efea7600640 1 --2- 172.21.15.161:0/2296734425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea015e130 0x7efea00b2320 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7efe9800efc0 tx=0x7efe9800c490 comp rx=0 tx=0).stop 2024-05-22T21:14:39.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.163+0000 7efea7600640 1 -- 172.21.15.161:0/2296734425 shutdown_connections 2024-05-22T21:14:39.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.163+0000 7efea7600640 1 --2- 172.21.15.161:0/2296734425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efe80066690 0x7efe80068b50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.163+0000 7efea7600640 1 --2- 172.21.15.161:0/2296734425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efea015e130 0x7efea00b2320 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.163+0000 7efea7600640 1 --2- 172.21.15.161:0/2296734425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efea015dd50 0x7efea015f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.163+0000 7efea7600640 1 --2- 172.21.15.161:0/2296734425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efea015d410 0x7efea0155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.163+0000 7efea7600640 1 -- 172.21.15.161:0/2296734425 >> 172.21.15.161:0/2296734425 conn(0x7efea00b0bc0 msgr2=0x7efea00aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:39.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.163+0000 7efea7600640 1 -- 172.21.15.161:0/2296734425 shutdown_connections 2024-05-22T21:14:39.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.163+0000 7efea7600640 1 -- 172.21.15.161:0/2296734425 wait complete. 2024-05-22T21:14:39.186 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-22T21:14:39.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.319+0000 7fb418600640 1 Processor -- start 2024-05-22T21:14:39.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.319+0000 7fb418600640 1 -- start start 2024-05-22T21:14:39.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.319+0000 7fb418600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb41013cfc0 0x7fb4101402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:39.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.319+0000 7fb418600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb41013fe50 0x7fb41006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:39.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.319+0000 7fb418600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41013d3a0 con 0x7fb41015af60 2024-05-22T21:14:39.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.319+0000 7fb418600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4100cd920 con 0x7fb41013cfc0 2024-05-22T21:14:39.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.319+0000 7fb418600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4100cdc00 con 0x7fb41013fe50 2024-05-22T21:14:39.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.319+0000 7fb416600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb41013fe50 0x7fb41006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:39.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.319+0000 7fb416600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb41013fe50 0x7fb41006fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:35252/0 (socket says 172.21.15.161:35252) 2024-05-22T21:14:39.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.319+0000 7fb416600640 1 -- 172.21.15.161:0/2475368656 learned_addr learned my addr 172.21.15.161:0/2475368656 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:39.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.319+0000 7fb415c00640 1 --2- 172.21.15.161:0/2475368656 >> v2:172.21.15.179:3300/0 conn(0x7fb41013cfc0 0x7fb4101402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:39.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.319+0000 7fb416600640 1 -- 172.21.15.161:0/2475368656 >> v2:172.21.15.179:3300/0 conn(0x7fb41013cfc0 msgr2=0x7fb4101402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:39.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.319+0000 7fb416600640 1 --2- 172.21.15.161:0/2475368656 >> v2:172.21.15.179:3300/0 conn(0x7fb41013cfc0 0x7fb4101402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.319+0000 7fb416600640 1 -- 172.21.15.161:0/2475368656 >> v1:172.21.15.179:6789/0 conn(0x7fb41015af60 legacy=0x7fb41013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:39.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.319+0000 7fb416600640 1 -- 172.21.15.161:0/2475368656 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4100cdee0 con 0x7fb41013fe50 2024-05-22T21:14:39.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.319+0000 7fb415c00640 1 --2- 172.21.15.161:0/2475368656 >> v2:172.21.15.179:3300/0 conn(0x7fb41013cfc0 0x7fb4101402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:39.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.323+0000 7fb416600640 1 --2- 172.21.15.161:0/2475368656 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb41013fe50 0x7fb41006fce0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb408009e50 tx=0x7fb40801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf9a07c55483b507 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:39.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.323+0000 7fb40fe00640 1 -- 172.21.15.161:0/2475368656 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080a6470 con 0x7fb41013fe50 2024-05-22T21:14:39.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.323+0000 7fb40fe00640 1 -- 172.21.15.161:0/2475368656 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4080a65d0 con 0x7fb41013fe50 2024-05-22T21:14:39.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.323+0000 7fb40fe00640 1 -- 172.21.15.161:0/2475368656 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080a6960 con 0x7fb41013fe50 2024-05-22T21:14:39.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.323+0000 7fb418600640 1 -- 172.21.15.161:0/2475368656 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb41013fe50 msgr2=0x7fb41006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:39.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.323+0000 7fb418600640 1 --2- 172.21.15.161:0/2475368656 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb41013fe50 0x7fb41006fce0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb408009e50 tx=0x7fb40801c970 comp rx=0 tx=0).stop 2024-05-22T21:14:39.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.323+0000 7fb418600640 1 -- 172.21.15.161:0/2475368656 shutdown_connections 2024-05-22T21:14:39.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.323+0000 7fb418600640 1 --2- 172.21.15.161:0/2475368656 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb41013fe50 0x7fb41006fce0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.323+0000 7fb418600640 1 --2- 172.21.15.161:0/2475368656 >> v2:172.21.15.179:3300/0 conn(0x7fb41013cfc0 0x7fb4101402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.323+0000 7fb418600640 1 -- 172.21.15.161:0/2475368656 >> 172.21.15.161:0/2475368656 conn(0x7fb41006a2d0 msgr2=0x7fb41006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:39.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.323+0000 7fb418600640 1 -- 172.21.15.161:0/2475368656 shutdown_connections 2024-05-22T21:14:39.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.323+0000 7fb418600640 1 -- 172.21.15.161:0/2475368656 wait complete. 2024-05-22T21:14:39.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb418600640 1 Processor -- start 2024-05-22T21:14:39.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb418600640 1 -- start start 2024-05-22T21:14:39.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb418600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb41013cfc0 0x7fb410142080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:39.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb418600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb41013fe50 0x7fb41013abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:39.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb418600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb41015af60 0x7fb41013e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:39.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb418600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb410141c60 con 0x7fb41015af60 2024-05-22T21:14:39.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb418600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb410140fe0 con 0x7fb41013fe50 2024-05-22T21:14:39.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb418600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4101412c0 con 0x7fb41013cfc0 2024-05-22T21:14:39.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb416600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb41015af60 0x7fb41013e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:39.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb416600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb41015af60 0x7fb41013e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57162/0 (socket says 172.21.15.161:57162) 2024-05-22T21:14:39.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb416600640 1 -- 172.21.15.161:0/3378753345 learned_addr learned my addr 172.21.15.161:0/3378753345 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:39.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb415200640 1 --2- 172.21.15.161:0/3378753345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb41013fe50 0x7fb41013abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:39.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb415c00640 1 --2- 172.21.15.161:0/3378753345 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb41013cfc0 0x7fb410142080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:39.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb416600640 1 -- 172.21.15.161:0/3378753345 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb41013cfc0 msgr2=0x7fb410142080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:39.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb416600640 1 --2- 172.21.15.161:0/3378753345 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb41013cfc0 0x7fb410142080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb416600640 1 -- 172.21.15.161:0/3378753345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb41013fe50 msgr2=0x7fb41013abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:39.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb416600640 1 --2- 172.21.15.161:0/3378753345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb41013fe50 0x7fb41013abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb416600640 1 -- 172.21.15.161:0/3378753345 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb404009420 con 0x7fb41015af60 2024-05-22T21:14:39.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb416600640 1 --2- 172.21.15.161:0/3378753345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb41015af60 0x7fb41013e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb40801b020 tx=0x7fb408019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:39.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb40d600640 1 -- 172.21.15.161:0/3378753345 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080a6470 con 0x7fb41015af60 2024-05-22T21:14:39.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb40d600640 1 -- 172.21.15.161:0/3378753345 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4080a7650 con 0x7fb41015af60 2024-05-22T21:14:39.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.327+0000 7fb418600640 1 -- 172.21.15.161:0/3378753345 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb408009b30 con 0x7fb41015af60 2024-05-22T21:14:39.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.331+0000 7fb40d600640 1 -- 172.21.15.161:0/3378753345 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080a79e0 con 0x7fb41015af60 2024-05-22T21:14:39.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.331+0000 7fb418600640 1 -- 172.21.15.161:0/3378753345 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb410141880 con 0x7fb41015af60 2024-05-22T21:14:39.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.331+0000 7fb418600640 1 -- 172.21.15.161:0/3378753345 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb410141da0 con 0x7fb41015af60 2024-05-22T21:14:39.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.331+0000 7fb40d600640 1 -- 172.21.15.161:0/3378753345 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb4080a7b40 con 0x7fb41015af60 2024-05-22T21:14:39.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.331+0000 7fb40d600640 1 --2- 172.21.15.161:0/3378753345 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3f0066690 0x7fb3f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:39.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.331+0000 7fb40d600640 1 -- 172.21.15.161:0/3378753345 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb40811de70 con 0x7fb41015af60 2024-05-22T21:14:39.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.331+0000 7fb415c00640 1 --2- 172.21.15.161:0/3378753345 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3f0066690 0x7fb3f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:39.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.331+0000 7fb415c00640 1 --2- 172.21.15.161:0/3378753345 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3f0066690 0x7fb3f0068b50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb404009cf0 tx=0x7fb404009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:39.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.335+0000 7fb40d600640 1 -- 172.21.15.161:0/3378753345 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb410141da0 con 0x7fb41015af60 2024-05-22T21:14:39.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.519+0000 7fb418600640 1 -- 172.21.15.161:0/3378753345 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb410141da0 con 0x7fb41015af60 2024-05-22T21:14:39.525 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:14:39.523+0000 7f7d32800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-22T21:14:39.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.535+0000 7fb40d600640 1 -- 172.21.15.161:0/3378753345 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb410141da0 con 0x7fb41015af60 2024-05-22T21:14:39.537 INFO:teuthology.orchestra.run.smithi161.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-22T21:14:39.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.535+0000 7fb418600640 1 -- 172.21.15.161:0/3378753345 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3f0066690 msgr2=0x7fb3f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:39.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.535+0000 7fb418600640 1 --2- 172.21.15.161:0/3378753345 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3f0066690 0x7fb3f0068b50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb404009cf0 tx=0x7fb404009190 comp rx=0 tx=0).stop 2024-05-22T21:14:39.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.535+0000 7fb418600640 1 -- 172.21.15.161:0/3378753345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb41015af60 msgr2=0x7fb41013e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:39.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.535+0000 7fb418600640 1 --2- 172.21.15.161:0/3378753345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb41015af60 0x7fb41013e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb40801b020 tx=0x7fb408019710 comp rx=0 tx=0).stop 2024-05-22T21:14:39.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.535+0000 7fb418600640 1 -- 172.21.15.161:0/3378753345 shutdown_connections 2024-05-22T21:14:39.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.535+0000 7fb418600640 1 --2- 172.21.15.161:0/3378753345 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb3f0066690 0x7fb3f0068b50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.535+0000 7fb418600640 1 --2- 172.21.15.161:0/3378753345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb41015af60 0x7fb41013e2f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.535+0000 7fb418600640 1 --2- 172.21.15.161:0/3378753345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb41013fe50 0x7fb41013abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.535+0000 7fb418600640 1 --2- 172.21.15.161:0/3378753345 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb41013cfc0 0x7fb410142080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.535+0000 7fb418600640 1 -- 172.21.15.161:0/3378753345 >> 172.21.15.161:0/3378753345 conn(0x7fb41006a2d0 msgr2=0x7fb4100b02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:39.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.535+0000 7fb418600640 1 -- 172.21.15.161:0/3378753345 shutdown_connections 2024-05-22T21:14:39.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.535+0000 7fb418600640 1 -- 172.21.15.161:0/3378753345 wait complete. 2024-05-22T21:14:39.546 INFO:tasks.ceph.mds.c.smithi161.stderr: -4667> 2024-05-22T21:14:22.645+0000 7f884b800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T21:14:39.559 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-22T21:14:39.597 INFO:tasks.ceph.mds.c.smithi161.stdout:starting mds.c at 2024-05-22T21:14:39.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.695+0000 7fd6cf400640 1 Processor -- start 2024-05-22T21:14:39.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.695+0000 7fd6cf400640 1 -- start start 2024-05-22T21:14:39.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.695+0000 7fd6cf400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd6c81402e0 0x7fd6c813f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:39.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.695+0000 7fd6cf400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c8140090 con 0x7fd6c813fc80 2024-05-22T21:14:39.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.695+0000 7fd6cf400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c813d3a0 con 0x7fd6c813cfc0 2024-05-22T21:14:39.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.695+0000 7fd6cf400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c80cd920 con 0x7fd6c81402e0 2024-05-22T21:14:39.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.695+0000 7fd6bfe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd6c81402e0 0x7fd6c813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:39.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.695+0000 7fd6cd400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd6c813fc80 0x7fd6c806fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41890/0 (socket says 172.21.15.161:41890) 2024-05-22T21:14:39.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.695+0000 7fd6cd400640 1 -- 172.21.15.161:0/3600268430 learned_addr learned my addr 172.21.15.161:0/3600268430 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:39.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.695+0000 7fd6bfe00640 1 -- 172.21.15.161:0/3600268430 >> v1:172.21.15.179:6789/0 conn(0x7fd6c813cfc0 legacy=0x7fd6c815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:39.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.695+0000 7fd6bfe00640 1 -- 172.21.15.161:0/3600268430 >> v1:172.21.15.161:6789/0 conn(0x7fd6c813fc80 legacy=0x7fd6c806fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:39.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.695+0000 7fd6bfe00640 1 -- 172.21.15.161:0/3600268430 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c80cdc00 con 0x7fd6c81402e0 2024-05-22T21:14:39.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.695+0000 7fd6c7e00640 1 -- 172.21.15.161:0/3600268430 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865208973 0 0) 0x7fd6c8140090 con 0x7fd6c813fc80 2024-05-22T21:14:39.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.699+0000 7fd6bfe00640 1 --2- 172.21.15.161:0/3600268430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c81402e0 0x7fd6c813f740 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00097c0 tx=0x7fd6b001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd4fcb392e3f6eb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:39.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.699+0000 7fd6c7e00640 1 -- 172.21.15.161:0/3600268430 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b00ac440 con 0x7fd6c81402e0 2024-05-22T21:14:39.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.699+0000 7fd6c7e00640 1 -- 172.21.15.161:0/3600268430 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b00ac5a0 con 0x7fd6c81402e0 2024-05-22T21:14:39.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.699+0000 7fd6c7e00640 1 -- 172.21.15.161:0/3600268430 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b00ac930 con 0x7fd6c81402e0 2024-05-22T21:14:39.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.699+0000 7fd6cf400640 1 -- 172.21.15.161:0/3600268430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c81402e0 msgr2=0x7fd6c813f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:39.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.699+0000 7fd6cf400640 1 --2- 172.21.15.161:0/3600268430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c81402e0 0x7fd6c813f740 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00097c0 tx=0x7fd6b001c3e0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.699+0000 7fd6cf400640 1 -- 172.21.15.161:0/3600268430 shutdown_connections 2024-05-22T21:14:39.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.699+0000 7fd6cf400640 1 --2- 172.21.15.161:0/3600268430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c81402e0 0x7fd6c813f740 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.699+0000 7fd6cf400640 1 -- 172.21.15.161:0/3600268430 >> 172.21.15.161:0/3600268430 conn(0x7fd6c806a2d0 msgr2=0x7fd6c806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:39.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.699+0000 7fd6cf400640 1 -- 172.21.15.161:0/3600268430 shutdown_connections 2024-05-22T21:14:39.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.699+0000 7fd6cf400640 1 -- 172.21.15.161:0/3600268430 wait complete. 2024-05-22T21:14:39.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cf400640 1 Processor -- start 2024-05-22T21:14:39.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cf400640 1 -- start start 2024-05-22T21:14:39.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cf400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c813cfc0 0x7fd6c815b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:39.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cf400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6c813fc80 0x7fd6c8157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:39.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cf400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6c81402e0 0x7fd6c81413a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:39.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cf400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c8157d70 con 0x7fd6c813cfc0 2024-05-22T21:14:39.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cf400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c815bb30 con 0x7fd6c81402e0 2024-05-22T21:14:39.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cf400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c815be10 con 0x7fd6c813fc80 2024-05-22T21:14:39.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cca00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c813cfc0 0x7fd6c815b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:39.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cca00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c813cfc0 0x7fd6c815b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57212/0 (socket says 172.21.15.161:57212) 2024-05-22T21:14:39.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cca00640 1 -- 172.21.15.161:0/1132895952 learned_addr learned my addr 172.21.15.161:0/1132895952 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:39.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cd400640 1 --2- 172.21.15.161:0/1132895952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6c81402e0 0x7fd6c81413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:39.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6bfe00640 1 --2- 172.21.15.161:0/1132895952 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6c813fc80 0x7fd6c8157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:39.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cca00640 1 -- 172.21.15.161:0/1132895952 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6c813fc80 msgr2=0x7fd6c8157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:39.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cca00640 1 --2- 172.21.15.161:0/1132895952 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6c813fc80 0x7fd6c8157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cca00640 1 -- 172.21.15.161:0/1132895952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6c81402e0 msgr2=0x7fd6c81413a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:39.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cca00640 1 --2- 172.21.15.161:0/1132895952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6c81402e0 0x7fd6c81413a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:39.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cca00640 1 -- 172.21.15.161:0/1132895952 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b0009420 con 0x7fd6c813cfc0 2024-05-22T21:14:39.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cd400640 1 --2- 172.21.15.161:0/1132895952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6c81402e0 0x7fd6c81413a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:39.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6bfe00640 1 --2- 172.21.15.161:0/1132895952 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6c813fc80 0x7fd6c8157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:39.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cca00640 1 --2- 172.21.15.161:0/1132895952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c813cfc0 0x7fd6c815b490 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800c3c0 tx=0x7fd6b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:39.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6c5600640 1 -- 172.21.15.161:0/1132895952 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b80052d0 con 0x7fd6c813cfc0 2024-05-22T21:14:39.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6c5600640 1 -- 172.21.15.161:0/1132895952 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b8005430 con 0x7fd6c813cfc0 2024-05-22T21:14:39.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cf400640 1 -- 172.21.15.161:0/1132895952 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c815c0f0 con 0x7fd6c813cfc0 2024-05-22T21:14:39.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6c5600640 1 -- 172.21.15.161:0/1132895952 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8003840 con 0x7fd6c813cfc0 2024-05-22T21:14:39.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.703+0000 7fd6cf400640 1 -- 172.21.15.161:0/1132895952 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c815c6b0 con 0x7fd6c813cfc0 2024-05-22T21:14:39.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.707+0000 7fd6cf400640 1 -- 172.21.15.161:0/1132895952 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c8157ee0 con 0x7fd6c813cfc0 2024-05-22T21:14:39.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.707+0000 7fd6c5600640 1 -- 172.21.15.161:0/1132895952 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd6b80039a0 con 0x7fd6c813cfc0 2024-05-22T21:14:39.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.707+0000 7fd6c5600640 1 --2- 172.21.15.161:0/1132895952 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd6a4066690 0x7fd6a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:39.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.707+0000 7fd6c5600640 1 -- 172.21.15.161:0/1132895952 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd6b80837c0 con 0x7fd6c813cfc0 2024-05-22T21:14:39.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.707+0000 7fd6bfe00640 1 --2- 172.21.15.161:0/1132895952 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd6a4066690 0x7fd6a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:39.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.707+0000 7fd6bfe00640 1 --2- 172.21.15.161:0/1132895952 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd6a4066690 0x7fd6a4068b50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00ae000 tx=0x7fd6b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:39.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.711+0000 7fd6c5600640 1 -- 172.21.15.161:0/1132895952 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd6c8157ee0 con 0x7fd6c813cfc0 2024-05-22T21:14:39.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:39.899+0000 7fd6cf400640 1 -- 172.21.15.161:0/1132895952 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd6c815c8a0 con 0x7fd6c813cfc0 2024-05-22T21:14:40.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.535+0000 7fd6c5600640 1 -- 172.21.15.161:0/1132895952 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd6b8050a30 con 0x7fd6c813cfc0 2024-05-22T21:14:40.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.539+0000 7fd6cf400640 1 -- 172.21.15.161:0/1132895952 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd6a4066690 msgr2=0x7fd6a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:40.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.539+0000 7fd6cf400640 1 --2- 172.21.15.161:0/1132895952 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd6a4066690 0x7fd6a4068b50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00ae000 tx=0x7fd6b0008040 comp rx=0 tx=0).stop 2024-05-22T21:14:40.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.539+0000 7fd6cf400640 1 -- 172.21.15.161:0/1132895952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c813cfc0 msgr2=0x7fd6c815b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:40.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.539+0000 7fd6cf400640 1 --2- 172.21.15.161:0/1132895952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c813cfc0 0x7fd6c815b490 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800c3c0 tx=0x7fd6b800c880 comp rx=0 tx=0).stop 2024-05-22T21:14:40.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.539+0000 7fd6cf400640 1 -- 172.21.15.161:0/1132895952 shutdown_connections 2024-05-22T21:14:40.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.539+0000 7fd6cf400640 1 --2- 172.21.15.161:0/1132895952 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd6a4066690 0x7fd6a4068b50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:40.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.539+0000 7fd6cf400640 1 --2- 172.21.15.161:0/1132895952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6c81402e0 0x7fd6c81413a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:40.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.539+0000 7fd6cf400640 1 --2- 172.21.15.161:0/1132895952 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6c813fc80 0x7fd6c8157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:40.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.539+0000 7fd6cf400640 1 --2- 172.21.15.161:0/1132895952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6c813cfc0 0x7fd6c815b490 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:40.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.539+0000 7fd6cf400640 1 -- 172.21.15.161:0/1132895952 >> 172.21.15.161:0/1132895952 conn(0x7fd6c806a2d0 msgr2=0x7fd6c806cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:40.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.539+0000 7fd6cf400640 1 -- 172.21.15.161:0/1132895952 shutdown_connections 2024-05-22T21:14:40.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.539+0000 7fd6cf400640 1 -- 172.21.15.161:0/1132895952 wait complete. 2024-05-22T21:14:40.564 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-22T21:14:40.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.747+0000 7f0b77000640 1 Processor -- start 2024-05-22T21:14:40.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.747+0000 7f0b77000640 1 -- start start 2024-05-22T21:14:40.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.747+0000 7f0b77000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0b7006d560 0x7f0b700cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:40.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.747+0000 7f0b77000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b700cce90 con 0x7f0b70144980 2024-05-22T21:14:40.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.747+0000 7f0b77000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b7006d0d0 con 0x7f0b700cca40 2024-05-22T21:14:40.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.747+0000 7f0b77000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b7006c4c0 con 0x7f0b7006d560 2024-05-22T21:14:40.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.747+0000 7f0b67400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0b7006d560 0x7f0b700cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:40.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.747+0000 7f0b67400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0b7006d560 0x7f0b700cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57218/0 (socket says 172.21.15.161:57218) 2024-05-22T21:14:40.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.747+0000 7f0b67400640 1 -- 172.21.15.161:0/3291341119 learned_addr learned my addr 172.21.15.161:0/3291341119 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:40.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.747+0000 7f0b6fe00640 1 --1- 172.21.15.161:0/3291341119 >> v1:172.21.15.161:6789/0 conn(0x7f0b70144980 0x7f0b7015c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41900/0 (socket says 172.21.15.161:41900) 2024-05-22T21:14:40.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.747+0000 7f0b67400640 1 -- 172.21.15.161:0/3291341119 >> v1:172.21.15.179:6789/0 conn(0x7f0b700cca40 legacy=0x7f0b7015c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:40.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.747+0000 7f0b67400640 1 -- 172.21.15.161:0/3291341119 >> v1:172.21.15.161:6789/0 conn(0x7f0b70144980 legacy=0x7f0b7015c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:40.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.747+0000 7f0b67400640 1 -- 172.21.15.161:0/3291341119 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7006c7a0 con 0x7f0b7006d560 2024-05-22T21:14:40.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.747+0000 7f0b6f400640 1 -- 172.21.15.161:0/3291341119 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099820462 0 0) 0x7f0b60001120 con 0x7f0b70144980 2024-05-22T21:14:40.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.751+0000 7f0b67400640 1 --2- 172.21.15.161:0/3291341119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b7006d560 0x7f0b700cc500 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009870 tx=0x7f0b5801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b883f19b5ac933e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:40.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.751+0000 7f0b6f400640 1 -- 172.21.15.161:0/3291341119 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b580ab650 con 0x7f0b7006d560 2024-05-22T21:14:40.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.751+0000 7f0b6f400640 1 -- 172.21.15.161:0/3291341119 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b580ab7b0 con 0x7f0b7006d560 2024-05-22T21:14:40.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.751+0000 7f0b6f400640 1 -- 172.21.15.161:0/3291341119 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b580abb40 con 0x7f0b7006d560 2024-05-22T21:14:40.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.751+0000 7f0b77000640 1 -- 172.21.15.161:0/3291341119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b7006d560 msgr2=0x7f0b700cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:40.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.751+0000 7f0b77000640 1 --2- 172.21.15.161:0/3291341119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b7006d560 0x7f0b700cc500 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009870 tx=0x7f0b5801c560 comp rx=0 tx=0).stop 2024-05-22T21:14:40.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.751+0000 7f0b77000640 1 -- 172.21.15.161:0/3291341119 shutdown_connections 2024-05-22T21:14:40.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.751+0000 7f0b77000640 1 --2- 172.21.15.161:0/3291341119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b7006d560 0x7f0b700cc500 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:40.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.751+0000 7f0b77000640 1 -- 172.21.15.161:0/3291341119 >> 172.21.15.161:0/3291341119 conn(0x7f0b7006a0a0 msgr2=0x7f0b7006a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:40.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.751+0000 7f0b77000640 1 -- 172.21.15.161:0/3291341119 shutdown_connections 2024-05-22T21:14:40.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.751+0000 7f0b77000640 1 -- 172.21.15.161:0/3291341119 wait complete. 2024-05-22T21:14:40.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.751+0000 7f0b77000640 1 Processor -- start 2024-05-22T21:14:40.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.751+0000 7f0b77000640 1 -- start start 2024-05-22T21:14:40.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b77000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b7006d560 0x7f0b700718d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:40.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b77000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b700cca40 0x7f0b7013c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:40.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b77000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b70144980 0x7f0b701fa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:40.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b77000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700a9ef0 con 0x7f0b700cca40 2024-05-22T21:14:40.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b77000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700aa6c0 con 0x7f0b7006d560 2024-05-22T21:14:40.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b77000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b701f9cc0 con 0x7f0b70144980 2024-05-22T21:14:40.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b67400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b700cca40 0x7f0b7013c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:40.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b67400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b700cca40 0x7f0b7013c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57232/0 (socket says 172.21.15.161:57232) 2024-05-22T21:14:40.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b67400640 1 -- 172.21.15.161:0/3454210994 learned_addr learned my addr 172.21.15.161:0/3454210994 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:40.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b67400640 1 -- 172.21.15.161:0/3454210994 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b70144980 msgr2=0x7f0b701fa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:40.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b67400640 1 --2- 172.21.15.161:0/3454210994 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b70144980 0x7f0b701fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:40.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b67400640 1 -- 172.21.15.161:0/3454210994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b7006d560 msgr2=0x7f0b700718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:40.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b67400640 1 --2- 172.21.15.161:0/3454210994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b7006d560 0x7f0b700718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:40.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b67400640 1 -- 172.21.15.161:0/3454210994 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b580094d0 con 0x7f0b700cca40 2024-05-22T21:14:40.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b67400640 1 --2- 172.21.15.161:0/3454210994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b700cca40 0x7f0b7013c390 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0b5801a190 tx=0x7f0b5801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:40.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b6cc00640 1 -- 172.21.15.161:0/3454210994 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b580b4800 con 0x7f0b700cca40 2024-05-22T21:14:40.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b6cc00640 1 -- 172.21.15.161:0/3454210994 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b580b4980 con 0x7f0b700cca40 2024-05-22T21:14:40.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b6cc00640 1 -- 172.21.15.161:0/3454210994 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b580b4d30 con 0x7f0b700cca40 2024-05-22T21:14:40.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b77000640 1 -- 172.21.15.161:0/3454210994 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b7013c8d0 con 0x7f0b700cca40 2024-05-22T21:14:40.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b77000640 1 -- 172.21.15.161:0/3454210994 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b700aa3e0 con 0x7f0b700cca40 2024-05-22T21:14:40.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b77000640 1 -- 172.21.15.161:0/3454210994 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b700b2a80 con 0x7f0b700cca40 2024-05-22T21:14:40.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.755+0000 7f0b6cc00640 1 -- 172.21.15.161:0/3454210994 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0b580c7440 con 0x7f0b700cca40 2024-05-22T21:14:40.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.759+0000 7f0b6cc00640 1 --2- 172.21.15.161:0/3454210994 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b4c066690 0x7f0b4c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:40.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.759+0000 7f0b6cc00640 1 -- 172.21.15.161:0/3454210994 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0b5811efd0 con 0x7f0b700cca40 2024-05-22T21:14:40.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.759+0000 7f0b6fe00640 1 --2- 172.21.15.161:0/3454210994 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b4c066690 0x7f0b4c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:40.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.759+0000 7f0b6fe00640 1 --2- 172.21.15.161:0/3454210994 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b4c066690 0x7f0b4c068b50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0b60000c10 tx=0x7f0b600079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:40.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.763+0000 7f0b6cc00640 1 -- 172.21.15.161:0/3454210994 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0b580ec370 con 0x7f0b700cca40 2024-05-22T21:14:40.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:40.919+0000 7f0b77000640 1 -- 172.21.15.161:0/3454210994 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0b70067220 con 0x7f0b700cca40 2024-05-22T21:14:41.542 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.539+0000 7f0b6cc00640 1 -- 172.21.15.161:0/3454210994 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0b580ef990 con 0x7f0b700cca40 2024-05-22T21:14:41.542 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' removed 2024-05-22T21:14:41.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.539+0000 7f0b77000640 1 -- 172.21.15.161:0/3454210994 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b4c066690 msgr2=0x7f0b4c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:41.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.539+0000 7f0b77000640 1 --2- 172.21.15.161:0/3454210994 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b4c066690 0x7f0b4c068b50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0b60000c10 tx=0x7f0b600079d0 comp rx=0 tx=0).stop 2024-05-22T21:14:41.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.539+0000 7f0b77000640 1 -- 172.21.15.161:0/3454210994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b700cca40 msgr2=0x7f0b7013c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:41.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.539+0000 7f0b77000640 1 --2- 172.21.15.161:0/3454210994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b700cca40 0x7f0b7013c390 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0b5801a190 tx=0x7f0b5801ada0 comp rx=0 tx=0).stop 2024-05-22T21:14:41.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.539+0000 7f0b77000640 1 -- 172.21.15.161:0/3454210994 shutdown_connections 2024-05-22T21:14:41.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.539+0000 7f0b77000640 1 --2- 172.21.15.161:0/3454210994 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b4c066690 0x7f0b4c068b50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:41.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.539+0000 7f0b77000640 1 --2- 172.21.15.161:0/3454210994 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b70144980 0x7f0b701fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:41.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.539+0000 7f0b77000640 1 --2- 172.21.15.161:0/3454210994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b700cca40 0x7f0b7013c390 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:41.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.539+0000 7f0b77000640 1 --2- 172.21.15.161:0/3454210994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b7006d560 0x7f0b700718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:41.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.539+0000 7f0b77000640 1 -- 172.21.15.161:0/3454210994 >> 172.21.15.161:0/3454210994 conn(0x7f0b7006a0a0 msgr2=0x7f0b7015c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:41.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.539+0000 7f0b77000640 1 -- 172.21.15.161:0/3454210994 shutdown_connections 2024-05-22T21:14:41.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.539+0000 7f0b77000640 1 -- 172.21.15.161:0/3454210994 wait complete. 2024-05-22T21:14:41.565 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-22T21:14:41.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.699+0000 7f0c47000640 1 Processor -- start 2024-05-22T21:14:41.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.699+0000 7f0c47000640 1 -- start start 2024-05-22T21:14:41.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c47000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0c400b3b70 0x7f0c4006d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:41.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c47000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0c4015af90 0x7f0c4006da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:41.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c47000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c40071630 con 0x7f0c401542e0 2024-05-22T21:14:41.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c47000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4015ab00 con 0x7f0c400b3b70 2024-05-22T21:14:41.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c47000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c40071350 con 0x7f0c4015af90 2024-05-22T21:14:41.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c45200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0c400b3b70 0x7f0c4006d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:41.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c45c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0c401542e0 0x7f0c40070900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41914/0 (socket says 172.21.15.161:41914) 2024-05-22T21:14:41.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c45c00640 1 -- 172.21.15.161:0/3396733922 learned_addr learned my addr 172.21.15.161:0/3396733922 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:41.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c46600640 1 --2- 172.21.15.161:0/3396733922 >> v2:172.21.15.179:3300/0 conn(0x7f0c4015af90 0x7f0c4006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:41.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c45200640 1 -- 172.21.15.161:0/3396733922 >> v2:172.21.15.179:3300/0 conn(0x7f0c4015af90 msgr2=0x7f0c4006da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:41.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c45200640 1 --2- 172.21.15.161:0/3396733922 >> v2:172.21.15.179:3300/0 conn(0x7f0c4015af90 0x7f0c4006da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:41.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c45200640 1 -- 172.21.15.161:0/3396733922 >> v1:172.21.15.161:6789/0 conn(0x7f0c401542e0 legacy=0x7f0c40070900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:41.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c45200640 1 -- 172.21.15.161:0/3396733922 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4015a4b0 con 0x7f0c400b3b70 2024-05-22T21:14:41.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c3fe00640 1 -- 172.21.15.161:0/3396733922 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460040889 0 0) 0x7f0c40071630 con 0x7f0c401542e0 2024-05-22T21:14:41.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c45200640 1 --2- 172.21.15.161:0/3396733922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c400b3b70 0x7f0c4006d550 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0c40070010 tx=0x7f0c2c01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6651413227db1fa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:41.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c3fe00640 1 -- 172.21.15.161:0/3396733922 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c0ab600 con 0x7f0c400b3b70 2024-05-22T21:14:41.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c3fe00640 1 -- 172.21.15.161:0/3396733922 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c0a7050 con 0x7f0c400b3b70 2024-05-22T21:14:41.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.703+0000 7f0c3fe00640 1 -- 172.21.15.161:0/3396733922 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c0ab980 con 0x7f0c400b3b70 2024-05-22T21:14:41.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c47000640 1 -- 172.21.15.161:0/3396733922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c400b3b70 msgr2=0x7f0c4006d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:41.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c47000640 1 --2- 172.21.15.161:0/3396733922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c400b3b70 0x7f0c4006d550 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0c40070010 tx=0x7f0c2c01c330 comp rx=0 tx=0).stop 2024-05-22T21:14:41.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c47000640 1 -- 172.21.15.161:0/3396733922 shutdown_connections 2024-05-22T21:14:41.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c47000640 1 --2- 172.21.15.161:0/3396733922 >> v2:172.21.15.179:3300/0 conn(0x7f0c4015af90 0x7f0c4006da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:41.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c47000640 1 --2- 172.21.15.161:0/3396733922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c400b3b70 0x7f0c4006d550 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:41.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c47000640 1 -- 172.21.15.161:0/3396733922 >> 172.21.15.161:0/3396733922 conn(0x7f0c400cebc0 msgr2=0x7f0c400cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:41.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c47000640 1 -- 172.21.15.161:0/3396733922 shutdown_connections 2024-05-22T21:14:41.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c47000640 1 -- 172.21.15.161:0/3396733922 wait complete. 2024-05-22T21:14:41.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c47000640 1 Processor -- start 2024-05-22T21:14:41.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c47000640 1 -- start start 2024-05-22T21:14:41.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c47000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c400b3b70 0x7f0c40146ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:41.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c47000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c401542e0 0x7f0c4014f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:41.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c47000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c4015af90 0x7f0c400af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:41.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c47000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c400cd170 con 0x7f0c401542e0 2024-05-22T21:14:41.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c47000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c401486a0 con 0x7f0c400b3b70 2024-05-22T21:14:41.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c47000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c400cc5f0 con 0x7f0c4015af90 2024-05-22T21:14:41.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c45200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c401542e0 0x7f0c4014f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:41.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c45200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c401542e0 0x7f0c4014f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57256/0 (socket says 172.21.15.161:57256) 2024-05-22T21:14:41.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c45200640 1 -- 172.21.15.161:0/974793750 learned_addr learned my addr 172.21.15.161:0/974793750 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:41.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c45c00640 1 --2- 172.21.15.161:0/974793750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c400b3b70 0x7f0c40146ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:41.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c46600640 1 --2- 172.21.15.161:0/974793750 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c4015af90 0x7f0c400af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:41.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c45200640 1 -- 172.21.15.161:0/974793750 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c4015af90 msgr2=0x7f0c400af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:41.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c45200640 1 --2- 172.21.15.161:0/974793750 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c4015af90 0x7f0c400af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:41.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c45200640 1 -- 172.21.15.161:0/974793750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c400b3b70 msgr2=0x7f0c40146ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:41.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c45200640 1 --2- 172.21.15.161:0/974793750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c400b3b70 0x7f0c40146ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:41.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.707+0000 7f0c45200640 1 -- 172.21.15.161:0/974793750 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c009420 con 0x7f0c401542e0 2024-05-22T21:14:41.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.711+0000 7f0c45200640 1 --2- 172.21.15.161:0/974793750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c401542e0 0x7f0c4014f680 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c0ae040 tx=0x7f0c2c0190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:41.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.711+0000 7f0c3d600640 1 -- 172.21.15.161:0/974793750 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c0b4610 con 0x7f0c401542e0 2024-05-22T21:14:41.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.711+0000 7f0c3d600640 1 -- 172.21.15.161:0/974793750 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c0b4770 con 0x7f0c401542e0 2024-05-22T21:14:41.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.711+0000 7f0c3d600640 1 -- 172.21.15.161:0/974793750 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c0b4b20 con 0x7f0c401542e0 2024-05-22T21:14:41.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.711+0000 7f0c47000640 1 -- 172.21.15.161:0/974793750 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c400cc8d0 con 0x7f0c401542e0 2024-05-22T21:14:41.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.711+0000 7f0c47000640 1 -- 172.21.15.161:0/974793750 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c400cce90 con 0x7f0c401542e0 2024-05-22T21:14:41.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.711+0000 7f0c47000640 1 -- 172.21.15.161:0/974793750 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c40067110 con 0x7f0c401542e0 2024-05-22T21:14:41.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.711+0000 7f0c3d600640 1 -- 172.21.15.161:0/974793750 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0c2c0b2040 con 0x7f0c401542e0 2024-05-22T21:14:41.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.711+0000 7f0c3d600640 1 --2- 172.21.15.161:0/974793750 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0c20066690 0x7f0c20068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:41.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.711+0000 7f0c3d600640 1 -- 172.21.15.161:0/974793750 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f0c2c11e1c0 con 0x7f0c401542e0 2024-05-22T21:14:41.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.711+0000 7f0c45c00640 1 --2- 172.21.15.161:0/974793750 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0c20066690 0x7f0c20068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:41.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.715+0000 7f0c45c00640 1 --2- 172.21.15.161:0/974793750 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0c20066690 0x7f0c20068b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0c34000f70 tx=0x7f0c34007b60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:41.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.715+0000 7f0c3d600640 1 -- 172.21.15.161:0/974793750 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0c2c0eb690 con 0x7f0c401542e0 2024-05-22T21:14:41.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:41.871+0000 7f0c47000640 1 -- 172.21.15.161:0/974793750 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0c40148810 con 0x7f0c401542e0 2024-05-22T21:14:42.550 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.547+0000 7f0c3d600640 1 -- 172.21.15.161:0/974793750 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0c2c0eecb0 con 0x7f0c401542e0 2024-05-22T21:14:42.551 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' removed 2024-05-22T21:14:42.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.547+0000 7f0c47000640 1 -- 172.21.15.161:0/974793750 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0c20066690 msgr2=0x7f0c20068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:42.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.547+0000 7f0c47000640 1 --2- 172.21.15.161:0/974793750 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0c20066690 0x7f0c20068b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0c34000f70 tx=0x7f0c34007b60 comp rx=0 tx=0).stop 2024-05-22T21:14:42.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.547+0000 7f0c47000640 1 -- 172.21.15.161:0/974793750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c401542e0 msgr2=0x7f0c4014f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:42.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.547+0000 7f0c47000640 1 --2- 172.21.15.161:0/974793750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c401542e0 0x7f0c4014f680 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c0ae040 tx=0x7f0c2c0190d0 comp rx=0 tx=0).stop 2024-05-22T21:14:42.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.547+0000 7f0c47000640 1 -- 172.21.15.161:0/974793750 shutdown_connections 2024-05-22T21:14:42.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.547+0000 7f0c47000640 1 --2- 172.21.15.161:0/974793750 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0c20066690 0x7f0c20068b50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:42.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.547+0000 7f0c47000640 1 --2- 172.21.15.161:0/974793750 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0c4015af90 0x7f0c400af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:42.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.547+0000 7f0c47000640 1 --2- 172.21.15.161:0/974793750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c401542e0 0x7f0c4014f680 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:42.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.547+0000 7f0c47000640 1 --2- 172.21.15.161:0/974793750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c400b3b70 0x7f0c40146ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:42.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.547+0000 7f0c47000640 1 -- 172.21.15.161:0/974793750 >> 172.21.15.161:0/974793750 conn(0x7f0c400cebc0 msgr2=0x7f0c400b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:42.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.547+0000 7f0c47000640 1 -- 172.21.15.161:0/974793750 shutdown_connections 2024-05-22T21:14:42.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.547+0000 7f0c47000640 1 -- 172.21.15.161:0/974793750 wait complete. 2024-05-22T21:14:42.573 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-22T21:14:42.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.707+0000 7f8c77200640 1 Processor -- start 2024-05-22T21:14:42.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.707+0000 7f8c77200640 1 -- start start 2024-05-22T21:14:42.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.707+0000 7f8c77200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c780ce370 0x7f8c781442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:42.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.707+0000 7f8c77200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c781448e0 0x7f8c780b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:42.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.707+0000 7f8c77200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c780ce830 con 0x7f8c78143ed0 2024-05-22T21:14:42.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.707+0000 7f8c77200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c780b2e60 con 0x7f8c780ce370 2024-05-22T21:14:42.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.707+0000 7f8c77200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c780b3140 con 0x7f8c781448e0 2024-05-22T21:14:42.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.707+0000 7f8c75400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c780ce370 0x7f8c781442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:42.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.707+0000 7f8c75400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c780ce370 0x7f8c781442b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57262/0 (socket says 172.21.15.161:57262) 2024-05-22T21:14:42.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.707+0000 7f8c75400640 1 -- 172.21.15.161:0/3760310957 learned_addr learned my addr 172.21.15.161:0/3760310957 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:42.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.707+0000 7f8c75400640 1 -- 172.21.15.161:0/3760310957 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c781448e0 msgr2=0x7f8c780b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:42.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.707+0000 7f8c76800640 1 --2- 172.21.15.161:0/3760310957 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c781448e0 0x7f8c780b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:42.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.707+0000 7f8c75400640 1 --2- 172.21.15.161:0/3760310957 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c781448e0 0x7f8c780b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:42.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.707+0000 7f8c75400640 1 -- 172.21.15.161:0/3760310957 >> v1:172.21.15.179:6789/0 conn(0x7f8c78143ed0 legacy=0x7f8c780b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:42.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.707+0000 7f8c75400640 1 -- 172.21.15.161:0/3760310957 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c780b3420 con 0x7f8c780ce370 2024-05-22T21:14:42.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.711+0000 7f8c76800640 1 --2- 172.21.15.161:0/3760310957 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c781448e0 0x7f8c780b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:42.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.711+0000 7f8c75400640 1 --2- 172.21.15.161:0/3760310957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c780ce370 0x7f8c781442b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009870 tx=0x7f8c6401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e85018382396d87 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:42.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.711+0000 7f8c74a00640 1 -- 172.21.15.161:0/3760310957 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c640ac640 con 0x7f8c780ce370 2024-05-22T21:14:42.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.711+0000 7f8c74a00640 1 -- 172.21.15.161:0/3760310957 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c640ac7a0 con 0x7f8c780ce370 2024-05-22T21:14:42.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.711+0000 7f8c74a00640 1 -- 172.21.15.161:0/3760310957 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c640acb30 con 0x7f8c780ce370 2024-05-22T21:14:42.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.711+0000 7f8c77200640 1 -- 172.21.15.161:0/3760310957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c780ce370 msgr2=0x7f8c781442b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:42.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.711+0000 7f8c77200640 1 --2- 172.21.15.161:0/3760310957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c780ce370 0x7f8c781442b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009870 tx=0x7f8c6401c400 comp rx=0 tx=0).stop 2024-05-22T21:14:42.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.711+0000 7f8c77200640 1 -- 172.21.15.161:0/3760310957 shutdown_connections 2024-05-22T21:14:42.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.711+0000 7f8c77200640 1 --2- 172.21.15.161:0/3760310957 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c781448e0 0x7f8c780b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:42.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.711+0000 7f8c77200640 1 --2- 172.21.15.161:0/3760310957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c780ce370 0x7f8c781442b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:42.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.711+0000 7f8c77200640 1 -- 172.21.15.161:0/3760310957 >> 172.21.15.161:0/3760310957 conn(0x7f8c780cec00 msgr2=0x7f8c780cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:42.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.711+0000 7f8c77200640 1 -- 172.21.15.161:0/3760310957 shutdown_connections 2024-05-22T21:14:42.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.711+0000 7f8c77200640 1 -- 172.21.15.161:0/3760310957 wait complete. 2024-05-22T21:14:42.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c77200640 1 Processor -- start 2024-05-22T21:14:42.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c77200640 1 -- start start 2024-05-22T21:14:42.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c77200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c780ce370 0x7f8c7813dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:42.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c77200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c78143ed0 0x7f8c78158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:42.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c77200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c781448e0 0x7f8c78137b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:42.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c77200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c78132880 con 0x7f8c780ce370 2024-05-22T21:14:42.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c77200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c78136f60 con 0x7f8c781448e0 2024-05-22T21:14:42.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c77200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c781325a0 con 0x7f8c78143ed0 2024-05-22T21:14:42.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c75e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c780ce370 0x7f8c7813dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:42.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c75e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c780ce370 0x7f8c7813dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57266/0 (socket says 172.21.15.161:57266) 2024-05-22T21:14:42.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c75e00640 1 -- 172.21.15.161:0/2041575702 learned_addr learned my addr 172.21.15.161:0/2041575702 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:42.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c76800640 1 --2- 172.21.15.161:0/2041575702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c781448e0 0x7f8c78137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:42.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c75400640 1 --2- 172.21.15.161:0/2041575702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c78143ed0 0x7f8c78158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:42.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c75e00640 1 -- 172.21.15.161:0/2041575702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c78143ed0 msgr2=0x7f8c78158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:42.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c75e00640 1 --2- 172.21.15.161:0/2041575702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c78143ed0 0x7f8c78158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:42.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c75e00640 1 -- 172.21.15.161:0/2041575702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c781448e0 msgr2=0x7f8c78137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:42.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c75e00640 1 --2- 172.21.15.161:0/2041575702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c781448e0 0x7f8c78137b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:42.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c75e00640 1 -- 172.21.15.161:0/2041575702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c640094d0 con 0x7f8c780ce370 2024-05-22T21:14:42.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c75400640 1 --2- 172.21.15.161:0/2041575702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c78143ed0 0x7f8c78158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:42.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c76800640 1 --2- 172.21.15.161:0/2041575702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c781448e0 0x7f8c78137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:42.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c75e00640 1 --2- 172.21.15.161:0/2041575702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c780ce370 0x7f8c7813dee0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800c7c0 tx=0x7f8c6800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:42.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c6e000640 1 -- 172.21.15.161:0/2041575702 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6800aa50 con 0x7f8c780ce370 2024-05-22T21:14:42.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c6e000640 1 -- 172.21.15.161:0/2041575702 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6800abb0 con 0x7f8c780ce370 2024-05-22T21:14:42.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c77200640 1 -- 172.21.15.161:0/2041575702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c781368a0 con 0x7f8c780ce370 2024-05-22T21:14:42.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c6e000640 1 -- 172.21.15.161:0/2041575702 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c68012420 con 0x7f8c780ce370 2024-05-22T21:14:42.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.715+0000 7f8c77200640 1 -- 172.21.15.161:0/2041575702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c781591f0 con 0x7f8c780ce370 2024-05-22T21:14:42.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.719+0000 7f8c77200640 1 -- 172.21.15.161:0/2041575702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c78136700 con 0x7f8c780ce370 2024-05-22T21:14:42.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.719+0000 7f8c6e000640 1 -- 172.21.15.161:0/2041575702 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8c68012580 con 0x7f8c780ce370 2024-05-22T21:14:42.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.719+0000 7f8c6e000640 1 --2- 172.21.15.161:0/2041575702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c58066690 0x7f8c58068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:42.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.719+0000 7f8c6e000640 1 -- 172.21.15.161:0/2041575702 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8c680840f0 con 0x7f8c780ce370 2024-05-22T21:14:42.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.719+0000 7f8c75400640 1 --2- 172.21.15.161:0/2041575702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c58066690 0x7f8c58068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:42.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.719+0000 7f8c75400640 1 --2- 172.21.15.161:0/2041575702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c58066690 0x7f8c58068b50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8c640ae000 tx=0x7f8c64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:42.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.723+0000 7f8c6e000640 1 -- 172.21.15.161:0/2041575702 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8c78136700 con 0x7f8c780ce370 2024-05-22T21:14:42.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.875+0000 7f8c77200640 1 -- 172.21.15.161:0/2041575702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8c78136700 con 0x7f8c780ce370 2024-05-22T21:14:42.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.883+0000 7f8c6e000640 1 -- 172.21.15.161:0/2041575702 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8c78136700 con 0x7f8c780ce370 2024-05-22T21:14:42.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.883+0000 7f8c77200640 1 -- 172.21.15.161:0/2041575702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c58066690 msgr2=0x7f8c58068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:42.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.883+0000 7f8c77200640 1 --2- 172.21.15.161:0/2041575702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c58066690 0x7f8c58068b50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8c640ae000 tx=0x7f8c64008040 comp rx=0 tx=0).stop 2024-05-22T21:14:42.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.883+0000 7f8c77200640 1 -- 172.21.15.161:0/2041575702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c780ce370 msgr2=0x7f8c7813dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:42.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.883+0000 7f8c77200640 1 --2- 172.21.15.161:0/2041575702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c780ce370 0x7f8c7813dee0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800c7c0 tx=0x7f8c6800cc80 comp rx=0 tx=0).stop 2024-05-22T21:14:42.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.883+0000 7f8c77200640 1 -- 172.21.15.161:0/2041575702 shutdown_connections 2024-05-22T21:14:42.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.883+0000 7f8c77200640 1 --2- 172.21.15.161:0/2041575702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c58066690 0x7f8c58068b50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:42.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.883+0000 7f8c77200640 1 --2- 172.21.15.161:0/2041575702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c781448e0 0x7f8c78137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:42.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.883+0000 7f8c77200640 1 --2- 172.21.15.161:0/2041575702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c78143ed0 0x7f8c78158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:42.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.883+0000 7f8c77200640 1 --2- 172.21.15.161:0/2041575702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c780ce370 0x7f8c7813dee0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:42.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.883+0000 7f8c77200640 1 -- 172.21.15.161:0/2041575702 >> 172.21.15.161:0/2041575702 conn(0x7f8c780cec00 msgr2=0x7f8c7806cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:42.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.883+0000 7f8c77200640 1 -- 172.21.15.161:0/2041575702 shutdown_connections 2024-05-22T21:14:42.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:42.883+0000 7f8c77200640 1 -- 172.21.15.161:0/2041575702 wait complete. 2024-05-22T21:14:42.909 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2024-05-22T21:14:43.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.039+0000 7ff489e00640 1 Processor -- start 2024-05-22T21:14:43.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.039+0000 7ff489e00640 1 -- start start 2024-05-22T21:14:43.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.043+0000 7ff489e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff484160df0 0x7ff4840cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:43.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.043+0000 7ff489e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff48406f120 0x7ff48415f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:43.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.043+0000 7ff489e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4840d0460 con 0x7ff48415ec30 2024-05-22T21:14:43.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.043+0000 7ff489e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4840d0180 con 0x7ff48406f120 2024-05-22T21:14:43.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.043+0000 7ff489e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48406df30 con 0x7ff484160df0 2024-05-22T21:14:43.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.043+0000 7ff489400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff48406f120 0x7ff48415f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:43.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.043+0000 7ff483e00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff48415ec30 0x7ff48406ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:57968/0 (socket says 172.21.15.161:57968) 2024-05-22T21:14:43.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.043+0000 7ff483e00640 1 -- 172.21.15.161:0/3113152155 learned_addr learned my addr 172.21.15.161:0/3113152155 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:43.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.043+0000 7ff488a00640 1 --2- 172.21.15.161:0/3113152155 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff484160df0 0x7ff4840cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.043+0000 7ff489400640 1 -- 172.21.15.161:0/3113152155 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff484160df0 msgr2=0x7ff4840cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.043+0000 7ff489400640 1 --2- 172.21.15.161:0/3113152155 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff484160df0 0x7ff4840cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.043+0000 7ff489400640 1 -- 172.21.15.161:0/3113152155 >> v1:172.21.15.179:6789/0 conn(0x7ff48415ec30 legacy=0x7ff48406ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.043+0000 7ff489400640 1 -- 172.21.15.161:0/3113152155 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48406e210 con 0x7ff48406f120 2024-05-22T21:14:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.043+0000 7ff488a00640 1 --2- 172.21.15.161:0/3113152155 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff484160df0 0x7ff4840cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.043+0000 7ff483400640 1 -- 172.21.15.161:0/3113152155 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3315678026 0 0) 0x7ff4840d0460 con 0x7ff48415ec30 2024-05-22T21:14:43.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489400640 1 --2- 172.21.15.161:0/3113152155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff48406f120 0x7ff48415f090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0097c0 tx=0x7ff47c01c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94ec7e4610fcb689 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:43.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff483400640 1 -- 172.21.15.161:0/3113152155 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0ac640 con 0x7ff48406f120 2024-05-22T21:14:43.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff483400640 1 -- 172.21.15.161:0/3113152155 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c0ac7a0 con 0x7ff48406f120 2024-05-22T21:14:43.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff483400640 1 -- 172.21.15.161:0/3113152155 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0acb30 con 0x7ff48406f120 2024-05-22T21:14:43.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489e00640 1 -- 172.21.15.161:0/3113152155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff48406f120 msgr2=0x7ff48415f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:43.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489e00640 1 --2- 172.21.15.161:0/3113152155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff48406f120 0x7ff48415f090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0097c0 tx=0x7ff47c01c350 comp rx=0 tx=0).stop 2024-05-22T21:14:43.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489e00640 1 -- 172.21.15.161:0/3113152155 shutdown_connections 2024-05-22T21:14:43.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489e00640 1 --2- 172.21.15.161:0/3113152155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff48406f120 0x7ff48415f090 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:43.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489e00640 1 --2- 172.21.15.161:0/3113152155 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff484160df0 0x7ff4840cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:43.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489e00640 1 -- 172.21.15.161:0/3113152155 >> 172.21.15.161:0/3113152155 conn(0x7ff4840b2930 msgr2=0x7ff4840b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:43.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489e00640 1 -- 172.21.15.161:0/3113152155 shutdown_connections 2024-05-22T21:14:43.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489e00640 1 -- 172.21.15.161:0/3113152155 wait complete. 2024-05-22T21:14:43.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489e00640 1 Processor -- start 2024-05-22T21:14:43.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489e00640 1 -- start start 2024-05-22T21:14:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff48406f120 0x7ff484155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff48415ec30 0x7ff4841533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff484160df0 0x7ff4840c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4841595d0 con 0x7ff484160df0 2024-05-22T21:14:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff484159010 con 0x7ff48415ec30 2024-05-22T21:14:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.047+0000 7ff489e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4841592f0 con 0x7ff48406f120 2024-05-22T21:14:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff489400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff484160df0 0x7ff4840c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:43.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff489400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff484160df0 0x7ff4840c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57278/0 (socket says 172.21.15.161:57278) 2024-05-22T21:14:43.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff489400640 1 -- 172.21.15.161:0/67112885 learned_addr learned my addr 172.21.15.161:0/67112885 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:43.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff483e00640 1 --2- 172.21.15.161:0/67112885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff48415ec30 0x7ff4841533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:43.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff488a00640 1 --2- 172.21.15.161:0/67112885 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff48406f120 0x7ff484155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff489400640 1 -- 172.21.15.161:0/67112885 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff48406f120 msgr2=0x7ff484155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff489400640 1 --2- 172.21.15.161:0/67112885 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff48406f120 0x7ff484155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff489400640 1 -- 172.21.15.161:0/67112885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff48415ec30 msgr2=0x7ff4841533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff489400640 1 --2- 172.21.15.161:0/67112885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff48415ec30 0x7ff4841533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff489400640 1 -- 172.21.15.161:0/67112885 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff474009a70 con 0x7ff484160df0 2024-05-22T21:14:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff488a00640 1 --2- 172.21.15.161:0/67112885 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff48406f120 0x7ff484155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff489400640 1 --2- 172.21.15.161:0/67112885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff484160df0 0x7ff4840c5ec0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff47c019f80 tx=0x7ff47c0190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff480c00640 1 -- 172.21.15.161:0/67112885 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0b5610 con 0x7ff484160df0 2024-05-22T21:14:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff480c00640 1 -- 172.21.15.161:0/67112885 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c0b5770 con 0x7ff484160df0 2024-05-22T21:14:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff480c00640 1 -- 172.21.15.161:0/67112885 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0b5b00 con 0x7ff484160df0 2024-05-22T21:14:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff489e00640 1 -- 172.21.15.161:0/67112885 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47c009420 con 0x7ff484160df0 2024-05-22T21:14:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff489e00640 1 -- 172.21.15.161:0/67112885 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4840c6f50 con 0x7ff484160df0 2024-05-22T21:14:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff489e00640 1 -- 172.21.15.161:0/67112885 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff484159430 con 0x7ff484160df0 2024-05-22T21:14:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.051+0000 7ff480c00640 1 -- 172.21.15.161:0/67112885 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff47c0b3040 con 0x7ff484160df0 2024-05-22T21:14:43.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.055+0000 7ff480c00640 1 --2- 172.21.15.161:0/67112885 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff464066690 0x7ff464068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:43.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.055+0000 7ff480c00640 1 -- 172.21.15.161:0/67112885 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff47c11e0a0 con 0x7ff484160df0 2024-05-22T21:14:43.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.055+0000 7ff488a00640 1 --2- 172.21.15.161:0/67112885 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff464066690 0x7ff464068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:43.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.055+0000 7ff488a00640 1 --2- 172.21.15.161:0/67112885 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff464066690 0x7ff464068b50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff474009fd0 tx=0x7ff47400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:43.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.059+0000 7ff480c00640 1 -- 172.21.15.161:0/67112885 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff484159430 con 0x7ff484160df0 2024-05-22T21:14:43.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.271+0000 7ff489e00640 1 -- 172.21.15.161:0/67112885 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7ff484159430 con 0x7ff484160df0 2024-05-22T21:14:43.550 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.547+0000 7ff480c00640 1 -- 172.21.15.161:0/67112885 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v96) v1 ==== 130+0+0 (secure 0 0 0) 0x7ff484159430 con 0x7ff484160df0 2024-05-22T21:14:43.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.547+0000 7ff489e00640 1 -- 172.21.15.161:0/67112885 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff464066690 msgr2=0x7ff464068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:43.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.547+0000 7ff489e00640 1 --2- 172.21.15.161:0/67112885 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff464066690 0x7ff464068b50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff474009fd0 tx=0x7ff47400b040 comp rx=0 tx=0).stop 2024-05-22T21:14:43.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.547+0000 7ff489e00640 1 -- 172.21.15.161:0/67112885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff484160df0 msgr2=0x7ff4840c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:43.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.547+0000 7ff489e00640 1 --2- 172.21.15.161:0/67112885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff484160df0 0x7ff4840c5ec0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff47c019f80 tx=0x7ff47c0190f0 comp rx=0 tx=0).stop 2024-05-22T21:14:43.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.547+0000 7ff489e00640 1 -- 172.21.15.161:0/67112885 shutdown_connections 2024-05-22T21:14:43.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.547+0000 7ff489e00640 1 --2- 172.21.15.161:0/67112885 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff464066690 0x7ff464068b50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:43.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.547+0000 7ff489e00640 1 --2- 172.21.15.161:0/67112885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff484160df0 0x7ff4840c5ec0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:43.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.547+0000 7ff489e00640 1 --2- 172.21.15.161:0/67112885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff48415ec30 0x7ff4841533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:43.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.547+0000 7ff489e00640 1 --2- 172.21.15.161:0/67112885 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff48406f120 0x7ff484155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:43.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.547+0000 7ff489e00640 1 -- 172.21.15.161:0/67112885 >> 172.21.15.161:0/67112885 conn(0x7ff4840b2930 msgr2=0x7ff4840b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:43.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.547+0000 7ff489e00640 1 -- 172.21.15.161:0/67112885 shutdown_connections 2024-05-22T21:14:43.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.547+0000 7ff489e00640 1 -- 172.21.15.161:0/67112885 wait complete. 2024-05-22T21:14:43.571 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2024-05-22T21:14:43.572 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestClusterFull) 2024-05-22T21:14:43.572 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2024-05-22T21:14:43.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40d9800640 1 Processor -- start 2024-05-22T21:14:43.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40d9800640 1 -- start start 2024-05-22T21:14:43.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40d9800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f40d40d0a40 0x7f40d40d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:43.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40d9800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f40d40cc320 0x7f40d40cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:43.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40d9800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d40cb8a0 con 0x7f40d415af00 2024-05-22T21:14:43.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40d9800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d40d13f0 con 0x7f40d40cc320 2024-05-22T21:14:43.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40d9800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d415f530 con 0x7f40d40d0a40 2024-05-22T21:14:43.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40d8e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f40d40cc320 0x7f40d40cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:43.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40d8e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f40d40cc320 0x7f40d40cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57294/0 (socket says 172.21.15.161:57294) 2024-05-22T21:14:43.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40d8e00640 1 -- 172.21.15.161:0/3291162395 learned_addr learned my addr 172.21.15.161:0/3291162395 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:43.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40d3e00640 1 --1- 172.21.15.161:0/3291162395 >> v1:172.21.15.161:6789/0 conn(0x7f40d415af00 0x7f40d415eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41922/0 (socket says 172.21.15.161:41922) 2024-05-22T21:14:43.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40cb400640 1 --2- 172.21.15.161:0/3291162395 >> v2:172.21.15.179:3300/0 conn(0x7f40d40d0a40 0x7f40d40d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:43.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40d8e00640 1 -- 172.21.15.161:0/3291162395 >> v2:172.21.15.179:3300/0 conn(0x7f40d40d0a40 msgr2=0x7f40d40d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:43.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40d8e00640 1 --2- 172.21.15.161:0/3291162395 >> v2:172.21.15.179:3300/0 conn(0x7f40d40d0a40 0x7f40d40d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:43.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40d8e00640 1 -- 172.21.15.161:0/3291162395 >> v1:172.21.15.161:6789/0 conn(0x7f40d415af00 legacy=0x7f40d415eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:43.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40d3400640 1 -- 172.21.15.161:0/3291162395 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956304954 0 0) 0x7f40d40cb8a0 con 0x7f40d415af00 2024-05-22T21:14:43.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.703+0000 7f40d8e00640 1 -- 172.21.15.161:0/3291162395 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d415f810 con 0x7f40d40cc320 2024-05-22T21:14:43.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.707+0000 7f40d8e00640 1 --2- 172.21.15.161:0/3291162395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d40cc320 0x7f40d40cc700 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f40d415ea40 tx=0x7f40cc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e1f957f5ea7389 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:43.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.707+0000 7f40d3400640 1 -- 172.21.15.161:0/3291162395 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0aa680 con 0x7f40d40cc320 2024-05-22T21:14:43.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.707+0000 7f40d3400640 1 -- 172.21.15.161:0/3291162395 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc0aa7e0 con 0x7f40d40cc320 2024-05-22T21:14:43.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.707+0000 7f40d3400640 1 -- 172.21.15.161:0/3291162395 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0aab70 con 0x7f40d40cc320 2024-05-22T21:14:43.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.707+0000 7f40d9800640 1 -- 172.21.15.161:0/3291162395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d40cc320 msgr2=0x7f40d40cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:43.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.707+0000 7f40d9800640 1 --2- 172.21.15.161:0/3291162395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d40cc320 0x7f40d40cc700 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f40d415ea40 tx=0x7f40cc01c520 comp rx=0 tx=0).stop 2024-05-22T21:14:43.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.707+0000 7f40d9800640 1 -- 172.21.15.161:0/3291162395 shutdown_connections 2024-05-22T21:14:43.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.707+0000 7f40d9800640 1 --2- 172.21.15.161:0/3291162395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d40cc320 0x7f40d40cc700 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:43.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.707+0000 7f40d9800640 1 --2- 172.21.15.161:0/3291162395 >> v2:172.21.15.179:3300/0 conn(0x7f40d40d0a40 0x7f40d40d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:43.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.707+0000 7f40d9800640 1 -- 172.21.15.161:0/3291162395 >> 172.21.15.161:0/3291162395 conn(0x7f40d4099a40 msgr2=0x7f40d4099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:43.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.707+0000 7f40d9800640 1 -- 172.21.15.161:0/3291162395 shutdown_connections 2024-05-22T21:14:43.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.707+0000 7f40d9800640 1 -- 172.21.15.161:0/3291162395 wait complete. 2024-05-22T21:14:43.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d9800640 1 Processor -- start 2024-05-22T21:14:43.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d9800640 1 -- start start 2024-05-22T21:14:43.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d9800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d40cc320 0x7f40d40c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:43.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d9800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40d40d0a40 0x7f40d4153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:43.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d9800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40d415af00 0x7f40d40c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:43.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d9800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d415cd30 con 0x7f40d40cc320 2024-05-22T21:14:43.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d9800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d4154480 con 0x7f40d415af00 2024-05-22T21:14:43.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d9800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d415c6b0 con 0x7f40d40d0a40 2024-05-22T21:14:43.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d40cc320 0x7f40d40c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:43.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d40cc320 0x7f40d40c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57302/0 (socket says 172.21.15.161:57302) 2024-05-22T21:14:43.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d3e00640 1 -- 172.21.15.161:0/3823523898 learned_addr learned my addr 172.21.15.161:0/3823523898 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:43.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d8e00640 1 --2- 172.21.15.161:0/3823523898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40d415af00 0x7f40d40c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:43.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40cb400640 1 --2- 172.21.15.161:0/3823523898 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40d40d0a40 0x7f40d4153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:43.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d3e00640 1 -- 172.21.15.161:0/3823523898 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40d40d0a40 msgr2=0x7f40d4153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:43.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d3e00640 1 --2- 172.21.15.161:0/3823523898 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40d40d0a40 0x7f40d4153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:43.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d3e00640 1 -- 172.21.15.161:0/3823523898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40d415af00 msgr2=0x7f40d40c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:43.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d3e00640 1 --2- 172.21.15.161:0/3823523898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40d415af00 0x7f40d40c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:43.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d3e00640 1 -- 172.21.15.161:0/3823523898 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc0094d0 con 0x7f40d40cc320 2024-05-22T21:14:43.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d3e00640 1 --2- 172.21.15.161:0/3823523898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d40cc320 0x7f40d40c7e60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f40c400c890 tx=0x7f40c400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:43.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d0c00640 1 -- 172.21.15.161:0/3823523898 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c400a640 con 0x7f40d40cc320 2024-05-22T21:14:43.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d0c00640 1 -- 172.21.15.161:0/3823523898 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c400a7c0 con 0x7f40d40cc320 2024-05-22T21:14:43.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d0c00640 1 -- 172.21.15.161:0/3823523898 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c400ab70 con 0x7f40d40cc320 2024-05-22T21:14:43.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d9800640 1 -- 172.21.15.161:0/3823523898 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d415e0a0 con 0x7f40d40cc320 2024-05-22T21:14:43.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.711+0000 7f40d9800640 1 -- 172.21.15.161:0/3823523898 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d415e380 con 0x7f40d40cc320 2024-05-22T21:14:43.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.715+0000 7f40d9800640 1 -- 172.21.15.161:0/3823523898 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d40c48c0 con 0x7f40d40cc320 2024-05-22T21:14:43.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.715+0000 7f40d0c00640 1 -- 172.21.15.161:0/3823523898 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f40c400acd0 con 0x7f40d40cc320 2024-05-22T21:14:43.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.715+0000 7f40d0c00640 1 --2- 172.21.15.161:0/3823523898 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f40b0066690 0x7f40b0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:43.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.715+0000 7f40d0c00640 1 -- 172.21.15.161:0/3823523898 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f40c4083350 con 0x7f40d40cc320 2024-05-22T21:14:43.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.715+0000 7f40cb400640 1 --2- 172.21.15.161:0/3823523898 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f40b0066690 0x7f40b0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:43.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.715+0000 7f40cb400640 1 --2- 172.21.15.161:0/3823523898 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f40b0066690 0x7f40b0068b50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f40d415a8b0 tx=0x7f40bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:43.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.719+0000 7f40d0c00640 1 -- 172.21.15.161:0/3823523898 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f40d40c48c0 con 0x7f40d40cc320 2024-05-22T21:14:43.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:43.931+0000 7f40d9800640 1 -- 172.21.15.161:0/3823523898 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f40d40c48c0 con 0x7f40d40cc320 2024-05-22T21:14:44.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.555+0000 7f40d0c00640 1 -- 172.21.15.161:0/3823523898 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v97) v1 ==== 145+0+0 (secure 0 0 0) 0x7f40d40c48c0 con 0x7f40d40cc320 2024-05-22T21:14:44.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.559+0000 7f40d9800640 1 -- 172.21.15.161:0/3823523898 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f40b0066690 msgr2=0x7f40b0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:44.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.559+0000 7f40d9800640 1 --2- 172.21.15.161:0/3823523898 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f40b0066690 0x7f40b0068b50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f40d415a8b0 tx=0x7f40bc008040 comp rx=0 tx=0).stop 2024-05-22T21:14:44.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.559+0000 7f40d9800640 1 -- 172.21.15.161:0/3823523898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d40cc320 msgr2=0x7f40d40c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:44.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.559+0000 7f40d9800640 1 --2- 172.21.15.161:0/3823523898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d40cc320 0x7f40d40c7e60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f40c400c890 tx=0x7f40c400cd50 comp rx=0 tx=0).stop 2024-05-22T21:14:44.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.559+0000 7f40d9800640 1 -- 172.21.15.161:0/3823523898 shutdown_connections 2024-05-22T21:14:44.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.559+0000 7f40d9800640 1 --2- 172.21.15.161:0/3823523898 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f40b0066690 0x7f40b0068b50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:44.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.559+0000 7f40d9800640 1 --2- 172.21.15.161:0/3823523898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40d415af00 0x7f40d40c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:44.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.559+0000 7f40d9800640 1 --2- 172.21.15.161:0/3823523898 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40d40d0a40 0x7f40d4153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:44.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.559+0000 7f40d9800640 1 --2- 172.21.15.161:0/3823523898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d40cc320 0x7f40d40c7e60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:44.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.559+0000 7f40d9800640 1 -- 172.21.15.161:0/3823523898 >> 172.21.15.161:0/3823523898 conn(0x7f40d4099a40 msgr2=0x7f40d406dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:44.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.559+0000 7f40d9800640 1 -- 172.21.15.161:0/3823523898 shutdown_connections 2024-05-22T21:14:44.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.559+0000 7f40d9800640 1 -- 172.21.15.161:0/3823523898 wait complete. 2024-05-22T21:14:44.583 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-05-22T21:14:44.735 INFO:teuthology.orchestra.run.smithi161.stdout:{"osd_objectstore":"memstore"} 2024-05-22T21:14:44.735 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-05-22T21:14:44.735 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-22T21:14:44.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.867+0000 7fc7e5a00640 1 Processor -- start 2024-05-22T21:14:44.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.867+0000 7fc7e5a00640 1 -- start start 2024-05-22T21:14:44.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.871+0000 7fc7e5a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc7e0143c40 0x7fc7e015b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:44.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.871+0000 7fc7e5a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc7e01444a0 0x7fc7e006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:44.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.871+0000 7fc7e5a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e015efb0 con 0x7fc7e0144900 2024-05-22T21:14:44.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.871+0000 7fc7e5a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e015f290 con 0x7fc7e0143c40 2024-05-22T21:14:44.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.871+0000 7fc7e5a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e015f570 con 0x7fc7e01444a0 2024-05-22T21:14:44.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.871+0000 7fc7dea00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc7e0143c40 0x7fc7e015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:44.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.871+0000 7fc7dea00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc7e0143c40 0x7fc7e015b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57304/0 (socket says 172.21.15.161:57304) 2024-05-22T21:14:44.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.871+0000 7fc7dea00640 1 -- 172.21.15.161:0/1935677345 learned_addr learned my addr 172.21.15.161:0/1935677345 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:44.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.871+0000 7fc7dea00640 1 -- 172.21.15.161:0/1935677345 >> v2:172.21.15.179:3300/0 conn(0x7fc7e01444a0 msgr2=0x7fc7e006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:44.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.871+0000 7fc7d6000640 1 --2- 172.21.15.161:0/1935677345 >> v2:172.21.15.179:3300/0 conn(0x7fc7e01444a0 0x7fc7e006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:44.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.871+0000 7fc7dea00640 1 --2- 172.21.15.161:0/1935677345 >> v2:172.21.15.179:3300/0 conn(0x7fc7e01444a0 0x7fc7e006c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:44.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.871+0000 7fc7dea00640 1 -- 172.21.15.161:0/1935677345 >> v1:172.21.15.179:6789/0 conn(0x7fc7e0144900 legacy=0x7fc7e00b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:44.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.871+0000 7fc7dea00640 1 -- 172.21.15.161:0/1935677345 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e015f850 con 0x7fc7e0143c40 2024-05-22T21:14:44.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.871+0000 7fc7d6000640 1 --2- 172.21.15.161:0/1935677345 >> v2:172.21.15.179:3300/0 conn(0x7fc7e01444a0 0x7fc7e006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:44.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.875+0000 7fc7dea00640 1 --2- 172.21.15.161:0/1935677345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7e0143c40 0x7fc7e015b600 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00b1c80 tx=0x7fc7d001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d7d98e18ba5038 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:44.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.875+0000 7fc7de000640 1 -- 172.21.15.161:0/1935677345 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d00ab5f0 con 0x7fc7e0143c40 2024-05-22T21:14:44.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.875+0000 7fc7de000640 1 -- 172.21.15.161:0/1935677345 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d00ab750 con 0x7fc7e0143c40 2024-05-22T21:14:44.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.875+0000 7fc7de000640 1 -- 172.21.15.161:0/1935677345 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d00abae0 con 0x7fc7e0143c40 2024-05-22T21:14:44.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.875+0000 7fc7e5a00640 1 -- 172.21.15.161:0/1935677345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7e0143c40 msgr2=0x7fc7e015b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:44.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.875+0000 7fc7e5a00640 1 --2- 172.21.15.161:0/1935677345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7e0143c40 0x7fc7e015b600 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00b1c80 tx=0x7fc7d001c4e0 comp rx=0 tx=0).stop 2024-05-22T21:14:44.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.875+0000 7fc7e5a00640 1 -- 172.21.15.161:0/1935677345 shutdown_connections 2024-05-22T21:14:44.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.875+0000 7fc7e5a00640 1 --2- 172.21.15.161:0/1935677345 >> v2:172.21.15.179:3300/0 conn(0x7fc7e01444a0 0x7fc7e006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:44.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.875+0000 7fc7e5a00640 1 --2- 172.21.15.161:0/1935677345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7e0143c40 0x7fc7e015b600 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:44.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.875+0000 7fc7e5a00640 1 -- 172.21.15.161:0/1935677345 >> 172.21.15.161:0/1935677345 conn(0x7fc7e006a2d0 msgr2=0x7fc7e006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:44.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.875+0000 7fc7e5a00640 1 -- 172.21.15.161:0/1935677345 shutdown_connections 2024-05-22T21:14:44.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.875+0000 7fc7e5a00640 1 -- 172.21.15.161:0/1935677345 wait complete. 2024-05-22T21:14:44.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.875+0000 7fc7e5a00640 1 Processor -- start 2024-05-22T21:14:44.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.875+0000 7fc7e5a00640 1 -- start start 2024-05-22T21:14:44.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7e5a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7e0143c40 0x7fc7e015b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:44.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7e5a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc7e01444a0 0x7fc7e0137a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:44.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7e5a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7e0144900 0x7fc7e01341c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:44.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7e5a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e0153df0 con 0x7fc7e0144900 2024-05-22T21:14:44.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7e5a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e0152830 con 0x7fc7e0143c40 2024-05-22T21:14:44.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7e5a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e0152220 con 0x7fc7e01444a0 2024-05-22T21:14:44.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7df400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7e0144900 0x7fc7e01341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:44.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7df400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7e0144900 0x7fc7e01341c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57314/0 (socket says 172.21.15.161:57314) 2024-05-22T21:14:44.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7df400640 1 -- 172.21.15.161:0/4053593846 learned_addr learned my addr 172.21.15.161:0/4053593846 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:44.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7dea00640 1 --2- 172.21.15.161:0/4053593846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7e0143c40 0x7fc7e015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:44.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7d6000640 1 --2- 172.21.15.161:0/4053593846 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc7e01444a0 0x7fc7e0137a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:44.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7df400640 1 -- 172.21.15.161:0/4053593846 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc7e01444a0 msgr2=0x7fc7e0137a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:44.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7df400640 1 --2- 172.21.15.161:0/4053593846 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc7e01444a0 0x7fc7e0137a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:44.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7df400640 1 -- 172.21.15.161:0/4053593846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7e0143c40 msgr2=0x7fc7e015b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:44.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7df400640 1 --2- 172.21.15.161:0/4053593846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7e0143c40 0x7fc7e015b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:44.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7df400640 1 -- 172.21.15.161:0/4053593846 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d00094d0 con 0x7fc7e0144900 2024-05-22T21:14:44.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7d6000640 1 --2- 172.21.15.161:0/4053593846 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc7e01444a0 0x7fc7e0137a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:44.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7df400640 1 --2- 172.21.15.161:0/4053593846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7e0144900 0x7fc7e01341c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800b9e0 tx=0x7fc7d800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:44.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7d7400640 1 -- 172.21.15.161:0/4053593846 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8003b80 con 0x7fc7e0144900 2024-05-22T21:14:44.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7d7400640 1 -- 172.21.15.161:0/4053593846 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d8003ce0 con 0x7fc7e0144900 2024-05-22T21:14:44.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7d7400640 1 -- 172.21.15.161:0/4053593846 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d800cb60 con 0x7fc7e0144900 2024-05-22T21:14:44.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7e5a00640 1 -- 172.21.15.161:0/4053593846 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e0154020 con 0x7fc7e0144900 2024-05-22T21:14:44.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7e5a00640 1 -- 172.21.15.161:0/4053593846 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e0154300 con 0x7fc7e0144900 2024-05-22T21:14:44.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7e5a00640 1 -- 172.21.15.161:0/4053593846 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e0067220 con 0x7fc7e0144900 2024-05-22T21:14:44.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.879+0000 7fc7d7400640 1 -- 172.21.15.161:0/4053593846 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc7d800ccc0 con 0x7fc7e0144900 2024-05-22T21:14:44.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.883+0000 7fc7d7400640 1 --2- 172.21.15.161:0/4053593846 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7bc066690 0x7fc7bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:44.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.883+0000 7fc7d7400640 1 -- 172.21.15.161:0/4053593846 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc7d8083b90 con 0x7fc7e0144900 2024-05-22T21:14:44.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.883+0000 7fc7dea00640 1 --2- 172.21.15.161:0/4053593846 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7bc066690 0x7fc7bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:44.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.883+0000 7fc7dea00640 1 --2- 172.21.15.161:0/4053593846 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7bc066690 0x7fc7bc068b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00af000 tx=0x7fc7d00b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:44.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:44.887+0000 7fc7d7400640 1 -- 172.21.15.161:0/4053593846 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc7d8051210 con 0x7fc7e0144900 2024-05-22T21:14:45.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.035+0000 7fc7e5a00640 1 -- 172.21.15.161:0/4053593846 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc7e00ceaa0 con 0x7fc7e0144900 2024-05-22T21:14:45.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.043+0000 7fc7d7400640 1 -- 172.21.15.161:0/4053593846 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc7d8054830 con 0x7fc7e0144900 2024-05-22T21:14:45.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.043+0000 7fc7e5a00640 1 -- 172.21.15.161:0/4053593846 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7bc066690 msgr2=0x7fc7bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:45.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.043+0000 7fc7e5a00640 1 --2- 172.21.15.161:0/4053593846 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7bc066690 0x7fc7bc068b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00af000 tx=0x7fc7d00b6040 comp rx=0 tx=0).stop 2024-05-22T21:14:45.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.043+0000 7fc7e5a00640 1 -- 172.21.15.161:0/4053593846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7e0144900 msgr2=0x7fc7e01341c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:45.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.043+0000 7fc7e5a00640 1 --2- 172.21.15.161:0/4053593846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7e0144900 0x7fc7e01341c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800b9e0 tx=0x7fc7d800bea0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.043+0000 7fc7e5a00640 1 -- 172.21.15.161:0/4053593846 shutdown_connections 2024-05-22T21:14:45.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.043+0000 7fc7e5a00640 1 --2- 172.21.15.161:0/4053593846 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7bc066690 0x7fc7bc068b50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.043+0000 7fc7e5a00640 1 --2- 172.21.15.161:0/4053593846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7e0144900 0x7fc7e01341c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.043+0000 7fc7e5a00640 1 --2- 172.21.15.161:0/4053593846 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc7e01444a0 0x7fc7e0137a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.043+0000 7fc7e5a00640 1 --2- 172.21.15.161:0/4053593846 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7e0143c40 0x7fc7e015b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.043+0000 7fc7e5a00640 1 -- 172.21.15.161:0/4053593846 >> 172.21.15.161:0/4053593846 conn(0x7fc7e006a2d0 msgr2=0x7fc7e015cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:45.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.047+0000 7fc7e5a00640 1 -- 172.21.15.161:0/4053593846 shutdown_connections 2024-05-22T21:14:45.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.047+0000 7fc7e5a00640 1 -- 172.21.15.161:0/4053593846 wait complete. 2024-05-22T21:14:45.068 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:14:45.069 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:14:45.152 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:14:45.198 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:14:45.198 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:14:45.198 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:45.207 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:14:45.207 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:14:45.207 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:45.247 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:14:45.247 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:14:45.247 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:45.256 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:14:45.257 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:14:45.257 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:45.299 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:14:45.299 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-22T21:14:45.299 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:14:45.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.431+0000 7f2ab7a00640 1 Processor -- start 2024-05-22T21:14:45.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.431+0000 7f2ab7a00640 1 -- start start 2024-05-22T21:14:45.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.431+0000 7f2ab7a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2ab01597f0 0x7f2ab00706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:45.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.431+0000 7f2ab7a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab0071040 con 0x7f2ab0070be0 2024-05-22T21:14:45.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.431+0000 7f2ab7a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab00b34e0 con 0x7f2ab00b3070 2024-05-22T21:14:45.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.431+0000 7f2ab7a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab0159bd0 con 0x7f2ab01597f0 2024-05-22T21:14:45.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.431+0000 7f2aafe00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f2ab0070be0 0x7f2ab00b2960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41926/0 (socket says 172.21.15.161:41926) 2024-05-22T21:14:45.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.431+0000 7f2aafe00640 1 -- 172.21.15.161:0/3102752999 learned_addr learned my addr 172.21.15.161:0/3102752999 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:45.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.431+0000 7f2aaf400640 1 -- 172.21.15.161:0/3102752999 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556308010 0 0) 0x7f2ab0071040 con 0x7f2ab0070be0 2024-05-22T21:14:45.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.431+0000 7f2ab5000640 1 --2- 172.21.15.161:0/3102752999 >> v2:172.21.15.179:3300/0 conn(0x7f2ab01597f0 0x7f2ab00706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:45.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2aaf400640 1 -- 172.21.15.161:0/3102752999 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a94003530 con 0x7f2ab0070be0 2024-05-22T21:14:45.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2aaf400640 1 -- 172.21.15.161:0/3102752999 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1186436174 0 0) 0x7f2a94003530 con 0x7f2ab0070be0 2024-05-22T21:14:45.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2aaf400640 1 -- 172.21.15.161:0/3102752999 >> v2:172.21.15.179:3300/0 conn(0x7f2ab01597f0 msgr2=0x7f2ab00706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:45.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2aaf400640 1 --2- 172.21.15.161:0/3102752999 >> v2:172.21.15.179:3300/0 conn(0x7f2ab01597f0 0x7f2ab00706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2aaf400640 1 -- 172.21.15.161:0/3102752999 >> v1:172.21.15.179:6789/0 conn(0x7f2ab00b3070 legacy=0x7f2ab00cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:45.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2aaf400640 1 -- 172.21.15.161:0/3102752999 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab00cbdd0 con 0x7f2ab0070be0 2024-05-22T21:14:45.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2aaf400640 1 -- 172.21.15.161:0/3102752999 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f2a9c001680 con 0x7f2ab0070be0 2024-05-22T21:14:45.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2aaf400640 1 -- 172.21.15.161:0/3102752999 >> v1:172.21.15.161:6789/0 conn(0x7f2ab0070be0 legacy=0x7f2ab00b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:45.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2aaf400640 1 --2- 172.21.15.161:0/3102752999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94003b50 0x7f2a94003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:45.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2aaf400640 1 --2- 172.21.15.161:0/3102752999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a94008820 0x7f2a9400ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:45.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2aaf400640 1 --2- 172.21.15.161:0/3102752999 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a9400b490 0x7f2a9400d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:45.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2aaf400640 1 -- 172.21.15.161:0/3102752999 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0159bd0 con 0x7f2a94003b50 2024-05-22T21:14:45.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2aaf400640 1 -- 172.21.15.161:0/3102752999 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9400e230 con 0x7f2a94008820 2024-05-22T21:14:45.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2aaf400640 1 -- 172.21.15.161:0/3102752999 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9400e3f0 con 0x7f2a9400b490 2024-05-22T21:14:45.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2ab5000640 1 --2- 172.21.15.161:0/3102752999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94003b50 0x7f2a94003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:45.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2aafe00640 1 --2- 172.21.15.161:0/3102752999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a94008820 0x7f2a9400ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:45.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2ab5a00640 1 --2- 172.21.15.161:0/3102752999 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a9400b490 0x7f2a9400d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:45.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2ab5000640 1 -- 172.21.15.161:0/3102752999 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a9400b490 msgr2=0x7f2a9400d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:45.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2ab5000640 1 --2- 172.21.15.161:0/3102752999 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a9400b490 0x7f2a9400d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2ab5000640 1 -- 172.21.15.161:0/3102752999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a94008820 msgr2=0x7f2a9400ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:45.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2ab5000640 1 --2- 172.21.15.161:0/3102752999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a94008820 0x7f2a9400ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2ab5000640 1 -- 172.21.15.161:0/3102752999 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a94003530 con 0x7f2a94003b50 2024-05-22T21:14:45.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2aafe00640 1 --2- 172.21.15.161:0/3102752999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a94008820 0x7f2a9400ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:45.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2ab5a00640 1 -- 172.21.15.161:0/3102752999 reap_dead start 2024-05-22T21:14:45.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.435+0000 7f2ab5000640 1 --2- 172.21.15.161:0/3102752999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94003b50 0x7f2a94003f70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009ae0 tx=0x7f2aa001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a65fb9a515590b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:45.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2aaf400640 1 -- 172.21.15.161:0/3102752999 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa00a5550 con 0x7f2a94003b50 2024-05-22T21:14:45.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2aaf400640 1 -- 172.21.15.161:0/3102752999 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa00a56b0 con 0x7f2a94003b50 2024-05-22T21:14:45.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2aaf400640 1 -- 172.21.15.161:0/3102752999 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa00a5a60 con 0x7f2a94003b50 2024-05-22T21:14:45.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab7a00640 1 -- 172.21.15.161:0/3102752999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94003b50 msgr2=0x7f2a94003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:45.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab7a00640 1 --2- 172.21.15.161:0/3102752999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94003b50 0x7f2a94003f70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009ae0 tx=0x7f2aa001ca70 comp rx=0 tx=0).stop 2024-05-22T21:14:45.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab7a00640 1 -- 172.21.15.161:0/3102752999 shutdown_connections 2024-05-22T21:14:45.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab7a00640 1 --2- 172.21.15.161:0/3102752999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94003b50 0x7f2a94003f70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab7a00640 1 -- 172.21.15.161:0/3102752999 >> 172.21.15.161:0/3102752999 conn(0x7f2ab00b0bc0 msgr2=0x7f2ab00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:45.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab7a00640 1 -- 172.21.15.161:0/3102752999 shutdown_connections 2024-05-22T21:14:45.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab7a00640 1 -- 172.21.15.161:0/3102752999 wait complete. 2024-05-22T21:14:45.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab7a00640 1 Processor -- start 2024-05-22T21:14:45.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab7a00640 1 -- start start 2024-05-22T21:14:45.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab7a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94003b50 0x7f2ab013f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:45.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab7a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ab0140320 0x7f2ab01407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:45.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab7a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab015c1d0 0x7f2ab015c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:45.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab7a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab015cb90 con 0x7f2a94003b50 2024-05-22T21:14:45.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab7a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0155bc0 con 0x7f2ab015c1d0 2024-05-22T21:14:45.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab7a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab013fde0 con 0x7f2ab0140320 2024-05-22T21:14:45.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab5000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94003b50 0x7f2ab013f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:45.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab5000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94003b50 0x7f2ab013f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57324/0 (socket says 172.21.15.161:57324) 2024-05-22T21:14:45.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab5000640 1 -- 172.21.15.161:0/2602241988 learned_addr learned my addr 172.21.15.161:0/2602241988 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:45.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab5a00640 1 --2- 172.21.15.161:0/2602241988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab015c1d0 0x7f2ab015c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:45.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2aafe00640 1 --2- 172.21.15.161:0/2602241988 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ab0140320 0x7f2ab01407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:45.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab5000640 1 -- 172.21.15.161:0/2602241988 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ab0140320 msgr2=0x7f2ab01407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:45.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab5000640 1 --2- 172.21.15.161:0/2602241988 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ab0140320 0x7f2ab01407a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab5000640 1 -- 172.21.15.161:0/2602241988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab015c1d0 msgr2=0x7f2ab015c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:45.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.439+0000 7f2ab5000640 1 --2- 172.21.15.161:0/2602241988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab015c1d0 0x7f2ab015c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.443+0000 7f2ab5000640 1 -- 172.21.15.161:0/2602241988 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab015b130 con 0x7f2a94003b50 2024-05-22T21:14:45.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.443+0000 7f2aafe00640 1 --2- 172.21.15.161:0/2602241988 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ab0140320 0x7f2ab01407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:45.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.443+0000 7f2ab5000640 1 --2- 172.21.15.161:0/2602241988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94003b50 0x7f2ab013f8a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00b0040 tx=0x7f2aa0019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:45.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.443+0000 7f2aacc00640 1 -- 172.21.15.161:0/2602241988 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa00a5550 con 0x7f2a94003b50 2024-05-22T21:14:45.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.443+0000 7f2aacc00640 1 -- 172.21.15.161:0/2602241988 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa0004260 con 0x7f2a94003b50 2024-05-22T21:14:45.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.443+0000 7f2ab7a00640 1 -- 172.21.15.161:0/2602241988 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab015b410 con 0x7f2a94003b50 2024-05-22T21:14:45.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.443+0000 7f2aacc00640 1 -- 172.21.15.161:0/2602241988 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0003620 con 0x7f2a94003b50 2024-05-22T21:14:45.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.443+0000 7f2ab7a00640 1 -- 172.21.15.161:0/2602241988 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab015b9d0 con 0x7f2a94003b50 2024-05-22T21:14:45.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.443+0000 7f2ab7a00640 1 -- 172.21.15.161:0/2602241988 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab0067220 con 0x7f2a94003b50 2024-05-22T21:14:45.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.443+0000 7f2aacc00640 1 -- 172.21.15.161:0/2602241988 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2aa00b4070 con 0x7f2a94003b50 2024-05-22T21:14:45.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.443+0000 7f2aacc00640 1 --2- 172.21.15.161:0/2602241988 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a900666c0 0x7f2a90068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:45.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.443+0000 7f2aacc00640 1 -- 172.21.15.161:0/2602241988 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2aa011db20 con 0x7f2a94003b50 2024-05-22T21:14:45.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.447+0000 7f2aafe00640 1 --2- 172.21.15.161:0/2602241988 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a900666c0 0x7f2a90068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:45.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.447+0000 7f2aafe00640 1 --2- 172.21.15.161:0/2602241988 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a900666c0 0x7f2a90068b80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0143540 tx=0x7f2a9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:45.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.451+0000 7f2aacc00640 1 -- 172.21.15.161:0/2602241988 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2aa00eb1b0 con 0x7f2a94003b50 2024-05-22T21:14:45.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.635+0000 7f2ab7a00640 1 -- 172.21.15.161:0/2602241988 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ab00cf7c0 con 0x7f2a94003b50 2024-05-22T21:14:45.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.639+0000 7f2aacc00640 1 -- 172.21.15.161:0/2602241988 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f2aa00b2070 con 0x7f2a94003b50 2024-05-22T21:14:45.642 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:45.642 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4990,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3742018925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":3742018925},{"type":"v1","addr":"172.21.15.161:6837","nonce":3742018925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2553366378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2553366378},{"type":"v1","addr":"172.21.15.179:6834","nonce":2553366378}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/509468338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":509468338},{"type":"v1","addr":"172.21.15.179:6835","nonce":509468338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5433,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/140021855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":140021855},{"type":"v1","addr":"172.21.15.161:6836","nonce":140021855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2024-05-22T21:14:45.642 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 18 2024-05-22T21:14:45.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.639+0000 7f2ab7a00640 1 -- 172.21.15.161:0/2602241988 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a900666c0 msgr2=0x7f2a90068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:45.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.639+0000 7f2ab7a00640 1 --2- 172.21.15.161:0/2602241988 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a900666c0 0x7f2a90068b80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0143540 tx=0x7f2a9c009040 comp rx=0 tx=0).stop 2024-05-22T21:14:45.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.639+0000 7f2ab7a00640 1 -- 172.21.15.161:0/2602241988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94003b50 msgr2=0x7f2ab013f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:45.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.639+0000 7f2ab7a00640 1 --2- 172.21.15.161:0/2602241988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94003b50 0x7f2ab013f8a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00b0040 tx=0x7f2aa0019810 comp rx=0 tx=0).stop 2024-05-22T21:14:45.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.639+0000 7f2ab7a00640 1 -- 172.21.15.161:0/2602241988 shutdown_connections 2024-05-22T21:14:45.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.639+0000 7f2ab7a00640 1 --2- 172.21.15.161:0/2602241988 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a900666c0 0x7f2a90068b80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.639+0000 7f2ab7a00640 1 --2- 172.21.15.161:0/2602241988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab015c1d0 0x7f2ab015c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.639+0000 7f2ab7a00640 1 --2- 172.21.15.161:0/2602241988 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ab0140320 0x7f2ab01407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.639+0000 7f2ab7a00640 1 --2- 172.21.15.161:0/2602241988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a94003b50 0x7f2ab013f8a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.639+0000 7f2ab7a00640 1 -- 172.21.15.161:0/2602241988 >> 172.21.15.161:0/2602241988 conn(0x7f2ab00b0bc0 msgr2=0x7f2ab014f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:45.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.639+0000 7f2ab7a00640 1 -- 172.21.15.161:0/2602241988 shutdown_connections 2024-05-22T21:14:45.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.639+0000 7f2ab7a00640 1 -- 172.21.15.161:0/2602241988 wait complete. 2024-05-22T21:14:45.662 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-22T21:14:45.663 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-22T21:14:45.663 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-22T21:14:45.663 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:14:45.663 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-22T21:14:45.663 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-22T21:14:45.663 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-22T21:14:45.663 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:14:45.664 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-22T21:14:45.664 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-22T21:14:45.664 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-22T21:14:45.664 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:14:45.664 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-22T21:14:45.664 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-22T21:14:45.664 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-22T21:14:45.665 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:14:45.679 INFO:tasks.ceph.mds.a:Stopped 2024-05-22T21:14:45.679 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-22T21:14:45.679 INFO:tasks.ceph.mds.c:Stopped 2024-05-22T21:14:45.680 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-22T21:14:45.680 INFO:tasks.ceph.mds.b:Stopped 2024-05-22T21:14:45.681 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-22T21:14:45.681 INFO:tasks.ceph.mds.d:Stopped 2024-05-22T21:14:45.682 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-22T21:14:45.723 INFO:tasks.ceph.mds.a:Started 2024-05-22T21:14:45.724 DEBUG:teuthology.parallel:result is None 2024-05-22T21:14:45.725 INFO:tasks.ceph.mds.c:Started 2024-05-22T21:14:45.725 DEBUG:teuthology.parallel:result is None 2024-05-22T21:14:45.727 INFO:tasks.ceph.mds.b:Started 2024-05-22T21:14:45.727 DEBUG:teuthology.parallel:result is None 2024-05-22T21:14:45.728 INFO:tasks.ceph.mds.d:Started 2024-05-22T21:14:45.729 DEBUG:teuthology.parallel:result is None 2024-05-22T21:14:45.729 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-22T21:14:45.809 INFO:tasks.ceph.mds.c.smithi161.stdout:starting mds.c at 2024-05-22T21:14:45.809 INFO:tasks.ceph.mds.a.smithi161.stdout:starting mds.a at 2024-05-22T21:14:45.825 INFO:tasks.ceph.mds.b.smithi179.stdout:starting mds.b at 2024-05-22T21:14:45.825 INFO:tasks.ceph.mds.d.smithi179.stdout:starting mds.d at 2024-05-22T21:14:45.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.871+0000 7f80bd000640 1 Processor -- start 2024-05-22T21:14:45.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.871+0000 7f80bd000640 1 -- start start 2024-05-22T21:14:45.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.871+0000 7f80bd000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80b806b4a0 0x7f80b80d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:45.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.871+0000 7f80bd000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b806b980 con 0x7f80b80cb4a0 2024-05-22T21:14:45.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.871+0000 7f80bd000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b806bc60 con 0x7f80b80d0b20 2024-05-22T21:14:45.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.871+0000 7f80bd000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b806ca20 con 0x7f80b806b4a0 2024-05-22T21:14:45.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.871+0000 7f80b7e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f80b80cb4a0 0x7f80b80cb950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:57636/0 (socket says 172.21.15.161:57636) 2024-05-22T21:14:45.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.871+0000 7f80b7e00640 1 -- 172.21.15.161:0/3559960414 learned_addr learned my addr 172.21.15.161:0/3559960414 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:45.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.871+0000 7f80b6a00640 1 --2- 172.21.15.161:0/3559960414 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80b806b4a0 0x7f80b80d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:45.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.871+0000 7f80b6000640 1 -- 172.21.15.161:0/3559960414 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79544675 0 0) 0x7f80b806b980 con 0x7f80b80cb4a0 2024-05-22T21:14:45.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.871+0000 7f80b6000640 1 -- 172.21.15.161:0/3559960414 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f809c003530 con 0x7f80b80cb4a0 2024-05-22T21:14:45.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.871+0000 7f80b6000640 1 -- 172.21.15.161:0/3559960414 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999149513 0 0) 0x7f80b806bc60 con 0x7f80b80d0b20 2024-05-22T21:14:45.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.871+0000 7f80b6000640 1 -- 172.21.15.161:0/3559960414 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80b806b980 con 0x7f80b80d0b20 2024-05-22T21:14:45.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.871+0000 7f80b6a00640 1 -- 172.21.15.161:0/3559960414 >> v1:172.21.15.179:6789/0 conn(0x7f80b80d0b20 legacy=0x7f80b80ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:45.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.871+0000 7f80b6a00640 1 -- 172.21.15.161:0/3559960414 >> v1:172.21.15.161:6789/0 conn(0x7f80b80cb4a0 legacy=0x7f80b80cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:45.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.871+0000 7f80b6a00640 1 -- 172.21.15.161:0/3559960414 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b806bf40 con 0x7f80b806b4a0 2024-05-22T21:14:45.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.875+0000 7f80b6a00640 1 --2- 172.21.15.161:0/3559960414 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80b806b4a0 0x7f80b80d0f00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f80a40097f0 tx=0x7f80a401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ac58b3cd35bda57 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:45.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.875+0000 7f80b6000640 1 -- 172.21.15.161:0/3559960414 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a40aa610 con 0x7f80b806b4a0 2024-05-22T21:14:45.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.875+0000 7f80b6000640 1 -- 172.21.15.161:0/3559960414 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a40a6040 con 0x7f80b806b4a0 2024-05-22T21:14:45.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.875+0000 7f80b6000640 1 -- 172.21.15.161:0/3559960414 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a40aa990 con 0x7f80b806b4a0 2024-05-22T21:14:45.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.875+0000 7f80bd000640 1 -- 172.21.15.161:0/3559960414 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80b806b4a0 msgr2=0x7f80b80d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:45.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.875+0000 7f80bd000640 1 --2- 172.21.15.161:0/3559960414 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80b806b4a0 0x7f80b80d0f00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f80a40097f0 tx=0x7f80a401c4b0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.875+0000 7f80bd000640 1 -- 172.21.15.161:0/3559960414 shutdown_connections 2024-05-22T21:14:45.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.875+0000 7f80bd000640 1 --2- 172.21.15.161:0/3559960414 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80b806b4a0 0x7f80b80d0f00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.875+0000 7f80bd000640 1 -- 172.21.15.161:0/3559960414 >> 172.21.15.161:0/3559960414 conn(0x7f80b8099a40 msgr2=0x7f80b8099e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:45.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.875+0000 7f80bd000640 1 -- 172.21.15.161:0/3559960414 shutdown_connections 2024-05-22T21:14:45.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.875+0000 7f80bd000640 1 -- 172.21.15.161:0/3559960414 wait complete. 2024-05-22T21:14:45.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80bd000640 1 Processor -- start 2024-05-22T21:14:45.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80bd000640 1 -- start start 2024-05-22T21:14:45.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80bd000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b806b4a0 0x7f80b80b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:45.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80bd000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80b80cb4a0 0x7f80b81f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:45.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80bd000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80b80d0b20 0x7f80b80ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:45.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80bd000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80c9e80 con 0x7f80b806b4a0 2024-05-22T21:14:45.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80bd000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80c7610 con 0x7f80b80d0b20 2024-05-22T21:14:45.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80bd000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80c98c0 con 0x7f80b80cb4a0 2024-05-22T21:14:45.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80b7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b806b4a0 0x7f80b80b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:45.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80b7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b806b4a0 0x7f80b80b5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55848/0 (socket says 172.21.15.161:55848) 2024-05-22T21:14:45.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80b7400640 1 -- 172.21.15.161:0/2337325376 learned_addr learned my addr 172.21.15.161:0/2337325376 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:45.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80b6a00640 1 --2- 172.21.15.161:0/2337325376 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80b80cb4a0 0x7f80b81f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:45.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80b7e00640 1 --2- 172.21.15.161:0/2337325376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80b80d0b20 0x7f80b80ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:45.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80b7400640 1 -- 172.21.15.161:0/2337325376 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80b80cb4a0 msgr2=0x7f80b81f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:45.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80b7400640 1 --2- 172.21.15.161:0/2337325376 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80b80cb4a0 0x7f80b81f9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80b7400640 1 -- 172.21.15.161:0/2337325376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80b80d0b20 msgr2=0x7f80b80ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:45.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80b7400640 1 --2- 172.21.15.161:0/2337325376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80b80d0b20 0x7f80b80ca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:45.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80b7400640 1 -- 172.21.15.161:0/2337325376 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a4009420 con 0x7f80b806b4a0 2024-05-22T21:14:45.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80b7400640 1 --2- 172.21.15.161:0/2337325376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b806b4a0 0x7f80b80b5ea0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f80a800c4c0 tx=0x7f80a800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:45.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80af400640 1 -- 172.21.15.161:0/2337325376 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a80052d0 con 0x7f80b806b4a0 2024-05-22T21:14:45.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80af400640 1 -- 172.21.15.161:0/2337325376 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a8005430 con 0x7f80b806b4a0 2024-05-22T21:14:45.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80bd000640 1 -- 172.21.15.161:0/2337325376 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b81fa0c0 con 0x7f80b806b4a0 2024-05-22T21:14:45.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80af400640 1 -- 172.21.15.161:0/2337325376 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8012870 con 0x7f80b806b4a0 2024-05-22T21:14:45.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.879+0000 7f80bd000640 1 -- 172.21.15.161:0/2337325376 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b815c7c0 con 0x7f80b806b4a0 2024-05-22T21:14:45.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.883+0000 7f80bd000640 1 -- 172.21.15.161:0/2337325376 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b8067070 con 0x7f80b806b4a0 2024-05-22T21:14:45.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.883+0000 7f80af400640 1 -- 172.21.15.161:0/2337325376 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f80a80129d0 con 0x7f80b806b4a0 2024-05-22T21:14:45.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.883+0000 7f80af400640 1 --2- 172.21.15.161:0/2337325376 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8098066690 0x7f8098068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:45.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.883+0000 7f80af400640 1 -- 172.21.15.161:0/2337325376 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f80a8083880 con 0x7f80b806b4a0 2024-05-22T21:14:45.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.883+0000 7f80b6a00640 1 --2- 172.21.15.161:0/2337325376 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8098066690 0x7f8098068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:45.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.883+0000 7f80b6a00640 1 --2- 172.21.15.161:0/2337325376 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8098066690 0x7f8098068b50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f80a40ae000 tx=0x7f80a40a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:45.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:45.887+0000 7f80af400640 1 -- 172.21.15.161:0/2337325376 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f80a8051030 con 0x7f80b806b4a0 2024-05-22T21:14:46.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:46.055+0000 7f80bd000640 1 -- 172.21.15.161:0/2337325376 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f80b80cada0 con 0x7f80b806b4a0 2024-05-22T21:14:46.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:46.863+0000 7f80af400640 1 -- 172.21.15.161:0/2337325376 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v43) v1 ==== 97+0+0 (secure 0 0 0) 0x7f80a8054650 con 0x7f80b806b4a0 2024-05-22T21:14:46.868 INFO:teuthology.orchestra.run.smithi161.stderr: removed all blocklist entries 2024-05-22T21:14:46.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:46.867+0000 7f80bd000640 1 -- 172.21.15.161:0/2337325376 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8098066690 msgr2=0x7f8098068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:46.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:46.867+0000 7f80bd000640 1 --2- 172.21.15.161:0/2337325376 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8098066690 0x7f8098068b50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f80a40ae000 tx=0x7f80a40a7000 comp rx=0 tx=0).stop 2024-05-22T21:14:46.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:46.867+0000 7f80bd000640 1 -- 172.21.15.161:0/2337325376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b806b4a0 msgr2=0x7f80b80b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:46.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:46.867+0000 7f80bd000640 1 --2- 172.21.15.161:0/2337325376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b806b4a0 0x7f80b80b5ea0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f80a800c4c0 tx=0x7f80a800c980 comp rx=0 tx=0).stop 2024-05-22T21:14:46.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:46.867+0000 7f80bd000640 1 -- 172.21.15.161:0/2337325376 shutdown_connections 2024-05-22T21:14:46.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:46.867+0000 7f80bd000640 1 --2- 172.21.15.161:0/2337325376 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8098066690 0x7f8098068b50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:46.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:46.867+0000 7f80bd000640 1 --2- 172.21.15.161:0/2337325376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80b80d0b20 0x7f80b80ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:46.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:46.867+0000 7f80bd000640 1 --2- 172.21.15.161:0/2337325376 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80b80cb4a0 0x7f80b81f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:46.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:46.867+0000 7f80bd000640 1 --2- 172.21.15.161:0/2337325376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80b806b4a0 0x7f80b80b5ea0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:46.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:46.867+0000 7f80bd000640 1 -- 172.21.15.161:0/2337325376 >> 172.21.15.161:0/2337325376 conn(0x7f80b8099a40 msgr2=0x7f80b80cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:46.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:46.867+0000 7f80bd000640 1 -- 172.21.15.161:0/2337325376 shutdown_connections 2024-05-22T21:14:46.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:46.867+0000 7f80bd000640 1 -- 172.21.15.161:0/2337325376 wait complete. 2024-05-22T21:14:46.891 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-22T21:14:47.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3ac600640 1 Processor -- start 2024-05-22T21:14:47.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3ac600640 1 -- start start 2024-05-22T21:14:47.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3ac600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3a415dd50 0x7fa3a406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3ac600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa3a415e130 0x7fa3a415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3ac600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3a4071670 con 0x7fa3a415d410 2024-05-22T21:14:47.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3ac600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3a415d8a0 con 0x7fa3a415e130 2024-05-22T21:14:47.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3ac600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a406de40 con 0x7fa3a415dd50 2024-05-22T21:14:47.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3aa600640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa3a415d410 0x7fa3a415a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:57646/0 (socket says 172.21.15.161:57646) 2024-05-22T21:14:47.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3aa600640 1 -- 172.21.15.161:0/2709836472 learned_addr learned my addr 172.21.15.161:0/2709836472 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:47.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a3e00640 1 -- 172.21.15.161:0/2709836472 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634724165 0 0) 0x7fa3a4071670 con 0x7fa3a415d410 2024-05-22T21:14:47.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a9200640 1 --2- 172.21.15.161:0/2709836472 >> v2:172.21.15.179:3300/0 conn(0x7fa3a415e130 0x7fa3a415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a9c00640 1 --2- 172.21.15.161:0/2709836472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3a415dd50 0x7fa3a406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a3e00640 1 -- 172.21.15.161:0/2709836472 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa38c003530 con 0x7fa3a415d410 2024-05-22T21:14:47.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a3e00640 1 -- 172.21.15.161:0/2709836472 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4217268276 0 0) 0x7fa38c003530 con 0x7fa3a415d410 2024-05-22T21:14:47.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a3e00640 1 -- 172.21.15.161:0/2709836472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3a415dd50 msgr2=0x7fa3a406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:47.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a3e00640 1 --2- 172.21.15.161:0/2709836472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3a415dd50 0x7fa3a406d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a3e00640 1 -- 172.21.15.161:0/2709836472 >> v2:172.21.15.179:3300/0 conn(0x7fa3a415e130 msgr2=0x7fa3a415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:47.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a3e00640 1 --2- 172.21.15.161:0/2709836472 >> v2:172.21.15.179:3300/0 conn(0x7fa3a415e130 0x7fa3a415ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a3e00640 1 -- 172.21.15.161:0/2709836472 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a4070a60 con 0x7fa3a415d410 2024-05-22T21:14:47.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a3e00640 1 -- 172.21.15.161:0/2709836472 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fa39c001a00 con 0x7fa3a415d410 2024-05-22T21:14:47.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a3e00640 1 -- 172.21.15.161:0/2709836472 >> v1:172.21.15.161:6789/0 conn(0x7fa3a415d410 legacy=0x7fa3a415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a3e00640 1 --2- 172.21.15.161:0/2709836472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3a415dd50 0x7fa38c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a3e00640 1 --2- 172.21.15.161:0/2709836472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa38c003ec0 0x7fa38c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a3e00640 1 --2- 172.21.15.161:0/2709836472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa38c004d10 0x7fa38c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a3e00640 1 -- 172.21.15.161:0/2709836472 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a415d8a0 con 0x7fa38c004d10 2024-05-22T21:14:47.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a3e00640 1 -- 172.21.15.161:0/2709836472 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a406de40 con 0x7fa38c003ec0 2024-05-22T21:14:47.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.023+0000 7fa3a3e00640 1 -- 172.21.15.161:0/2709836472 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c00b990 con 0x7fa3a415dd50 2024-05-22T21:14:47.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3aa600640 1 --2- 172.21.15.161:0/2709836472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa38c004d10 0x7fa38c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3a9200640 1 --2- 172.21.15.161:0/2709836472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa38c003ec0 0x7fa38c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3a9c00640 1 --2- 172.21.15.161:0/2709836472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3a415dd50 0x7fa38c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3aa600640 1 -- 172.21.15.161:0/2709836472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3a415dd50 msgr2=0x7fa38c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:47.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3aa600640 1 --2- 172.21.15.161:0/2709836472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3a415dd50 0x7fa38c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3aa600640 1 -- 172.21.15.161:0/2709836472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa38c003ec0 msgr2=0x7fa38c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:47.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3aa600640 1 --2- 172.21.15.161:0/2709836472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa38c003ec0 0x7fa38c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3aa600640 1 -- 172.21.15.161:0/2709836472 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38c003530 con 0x7fa38c004d10 2024-05-22T21:14:47.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3a9c00640 1 --2- 172.21.15.161:0/2709836472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3a415dd50 0x7fa38c003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:47.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3aa600640 1 --2- 172.21.15.161:0/2709836472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa38c004d10 0x7fa38c005190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa39c005af0 tx=0x7fa39c020110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe2ecbed6c09bc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:47.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3a3e00640 1 -- 172.21.15.161:0/2709836472 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c0ad030 con 0x7fa38c004d10 2024-05-22T21:14:47.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3a3e00640 1 -- 172.21.15.161:0/2709836472 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c0a9d50 con 0x7fa38c004d10 2024-05-22T21:14:47.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3a3e00640 1 -- 172.21.15.161:0/2709836472 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c0083f0 con 0x7fa38c004d10 2024-05-22T21:14:47.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3ac600640 1 -- 172.21.15.161:0/2709836472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa38c004d10 msgr2=0x7fa38c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3ac600640 1 --2- 172.21.15.161:0/2709836472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa38c004d10 0x7fa38c005190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa39c005af0 tx=0x7fa39c020110 comp rx=0 tx=0).stop 2024-05-22T21:14:47.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3aa600640 1 -- 172.21.15.161:0/2709836472 reap_dead start 2024-05-22T21:14:47.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3ac600640 1 -- 172.21.15.161:0/2709836472 shutdown_connections 2024-05-22T21:14:47.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.027+0000 7fa3ac600640 1 -- 172.21.15.161:0/2709836472 >> 172.21.15.161:0/2709836472 conn(0x7fa3a40b0bc0 msgr2=0x7fa3a40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:47.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3ac600640 1 -- 172.21.15.161:0/2709836472 shutdown_connections 2024-05-22T21:14:47.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3ac600640 1 -- 172.21.15.161:0/2709836472 wait complete. 2024-05-22T21:14:47.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3ac600640 1 Processor -- start 2024-05-22T21:14:47.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3ac600640 1 -- start start 2024-05-22T21:14:47.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3ac600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa38c004d10 0x7fa3a4144d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3ac600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3a40ce860 0x7fa3a40cece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3ac600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3a40af760 0x7fa3a40afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3ac600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a40b2610 con 0x7fa3a40af760 2024-05-22T21:14:47.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3ac600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a40b0120 con 0x7fa3a40ce860 2024-05-22T21:14:47.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3ac600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a40cf340 con 0x7fa38c004d10 2024-05-22T21:14:47.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3aa600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3a40af760 0x7fa3a40afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3aa600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3a40af760 0x7fa3a40afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55856/0 (socket says 172.21.15.161:55856) 2024-05-22T21:14:47.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3aa600640 1 -- 172.21.15.161:0/546741126 learned_addr learned my addr 172.21.15.161:0/546741126 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:47.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3a9200640 1 --2- 172.21.15.161:0/546741126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3a40ce860 0x7fa3a40cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3a9c00640 1 --2- 172.21.15.161:0/546741126 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa38c004d10 0x7fa3a4144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3aa600640 1 -- 172.21.15.161:0/546741126 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa38c004d10 msgr2=0x7fa3a4144d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3aa600640 1 --2- 172.21.15.161:0/546741126 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa38c004d10 0x7fa3a4144d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3aa600640 1 -- 172.21.15.161:0/546741126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3a40ce860 msgr2=0x7fa3a40cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3aa600640 1 --2- 172.21.15.161:0/546741126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3a40ce860 0x7fa3a40cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3aa600640 1 -- 172.21.15.161:0/546741126 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a40b1c70 con 0x7fa3a40af760 2024-05-22T21:14:47.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3aa600640 1 --2- 172.21.15.161:0/546741126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3a40af760 0x7fa3a40afbe0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa39c01ff40 tx=0x7fa38c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:47.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3a1600640 1 -- 172.21.15.161:0/546741126 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c0ad040 con 0x7fa3a40af760 2024-05-22T21:14:47.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3a1600640 1 -- 172.21.15.161:0/546741126 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c004ff0 con 0x7fa3a40af760 2024-05-22T21:14:47.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3a1600640 1 -- 172.21.15.161:0/546741126 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c001210 con 0x7fa3a40af760 2024-05-22T21:14:47.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.031+0000 7fa3ac600640 1 -- 172.21.15.161:0/546741126 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a40b1f50 con 0x7fa3a40af760 2024-05-22T21:14:47.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.035+0000 7fa3ac600640 1 -- 172.21.15.161:0/546741126 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a40ae050 con 0x7fa3a40af760 2024-05-22T21:14:47.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.035+0000 7fa3ac600640 1 -- 172.21.15.161:0/546741126 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a4067220 con 0x7fa3a40af760 2024-05-22T21:14:47.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.035+0000 7fa3a1600640 1 -- 172.21.15.161:0/546741126 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa39c0b9020 con 0x7fa3a40af760 2024-05-22T21:14:47.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.035+0000 7fa3a1600640 1 --2- 172.21.15.161:0/546741126 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa3840666c0 0x7fa384068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.035+0000 7fa3a1600640 1 -- 172.21.15.161:0/546741126 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa39c0ae070 con 0x7fa3a40af760 2024-05-22T21:14:47.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.035+0000 7fa3a9c00640 1 --2- 172.21.15.161:0/546741126 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa3840666c0 0x7fa384068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.035+0000 7fa3a9c00640 1 --2- 172.21.15.161:0/546741126 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa3840666c0 0x7fa384068b80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40b1960 tx=0x7fa398008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:47.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.039+0000 7fa3a1600640 1 -- 172.21.15.161:0/546741126 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa39c0ee980 con 0x7fa3a40af760 2024-05-22T21:14:47.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.251+0000 7fa3ac600640 1 -- 172.21.15.161:0/546741126 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa3a406bf60 con 0x7fa3a40af760 2024-05-22T21:14:47.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.251+0000 7fa3a1600640 1 -- 172.21.15.161:0/546741126 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa39c0c1070 con 0x7fa3a40af760 2024-05-22T21:14:47.255 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:47.255 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:14:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "auth_dump": [ 2024-05-22T21:14:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.a", 2024-05-22T21:14:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m00QTFRAAPSRrpBRY75FlcWf3QxOOfg==", 2024-05-22T21:14:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:14:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:14:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:14:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.256 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.b", 2024-05-22T21:14:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m/ipAIRAA4gj5jGqTpNCr1K2OU/bsAQ==", 2024-05-22T21:14:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:14:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:14:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:14:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.257 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.c", 2024-05-22T21:14:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mWlM2HBAAEmJJWMDlTG9E7Qzld2gd5w==", 2024-05-22T21:14:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:14:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:14:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:14:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.d", 2024-05-22T21:14:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mmsuaKBAAqpCYRZZUGK5ZuxflUBcPgg==", 2024-05-22T21:14:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:14:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:14:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:14:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:47.259 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.0", 2024-05-22T21:14:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05mJZokLhAAh2vbQnRX25iYUIbvVmURuQ==", 2024-05-22T21:14:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:14:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:14:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.260 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.1", 2024-05-22T21:14:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05m8kWNNhAA6aaVT2xshFkqnk4G4gOttA==", 2024-05-22T21:14:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:14:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:14:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.2", 2024-05-22T21:14:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05ml1k9AxAAtL3XaJJI6cilpmvurh18Pw==", 2024-05-22T21:14:47.261 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:14:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:14:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.3", 2024-05-22T21:14:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05mbNBgCxAAas1L+QR4Aez+H1G0EVltfg==", 2024-05-22T21:14:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.262 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:14:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:14:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.4", 2024-05-22T21:14:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mOPLPABAAZBGIMF/FasNB27n/7di0AA==", 2024-05-22T21:14:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:14:47.263 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:14:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.5", 2024-05-22T21:14:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mT1poCRAASXm4tdP01pA1VOviqeoLVw==", 2024-05-22T21:14:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:14:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:14:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:47.264 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.6", 2024-05-22T21:14:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05m1ZPRERAA2S7WeTwysLCZm1OB/qCrAA==", 2024-05-22T21:14:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:14:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:14:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.7", 2024-05-22T21:14:47.266 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mCrUhGhAAvXoFWV7X3303rfXaxmfWqA==", 2024-05-22T21:14:47.266 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.266 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:14:47.266 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:14:47.266 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:47.266 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.266 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.266 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.266 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.0", 2024-05-22T21:14:47.266 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mh9B7LBAAevhvj5KpwP9pEIpJcxUvFA==", 2024-05-22T21:14:47.266 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.267 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:14:47.267 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow r", 2024-05-22T21:14:47.267 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-22T21:14:47.267 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.267 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.267 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.267 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.1", 2024-05-22T21:14:47.267 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mMnHtNRAAg/a/Mb1oRqx73d9efIHCfA==", 2024-05-22T21:14:47.267 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.267 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:14:47.268 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:14:47.268 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:14:47.268 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:14:47.268 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.268 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.268 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.268 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.2", 2024-05-22T21:14:47.268 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mOzdkMhAAbTiNdvyxqIyrluMiZW0cOw==", 2024-05-22T21:14:47.268 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.268 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:14:47.268 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:14:47.269 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:14:47.269 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:14:47.269 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.269 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.269 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.269 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.3", 2024-05-22T21:14:47.269 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDYX05mr4oxABAA1+bQQbxhoeU+kb0OCa+Wyw==", 2024-05-22T21:14:47.269 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.269 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:14:47.269 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:14:47.269 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:14:47.270 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:14:47.270 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.270 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.270 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.270 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.admin", 2024-05-22T21:14:47.270 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDWX05mHZY0NRAAiBvMDkrCZ+MgG2iwBH+bcg==", 2024-05-22T21:14:47.270 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.270 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:14:47.270 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:14:47.270 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:14:47.270 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:47.271 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.271 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.271 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.271 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.x", 2024-05-22T21:14:47.271 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mH2kuERAAgigMUWK3M3WaMXlqGUcNrw==", 2024-05-22T21:14:47.271 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.271 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:14:47.271 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:14:47.271 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:47.271 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.271 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:14:47.272 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:47.272 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.y", 2024-05-22T21:14:47.272 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mZRdiDRAA0vUwD+rPSfyhjgA+9XPp5w==", 2024-05-22T21:14:47.272 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:14:47.272 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:14:47.272 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:14:47.272 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:14:47.272 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.272 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:47.272 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:14:47.272 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:14:47.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.251+0000 7fa3ac600640 1 -- 172.21.15.161:0/546741126 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa3840666c0 msgr2=0x7fa384068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.251+0000 7fa3ac600640 1 --2- 172.21.15.161:0/546741126 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa3840666c0 0x7fa384068b80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40b1960 tx=0x7fa398008040 comp rx=0 tx=0).stop 2024-05-22T21:14:47.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.251+0000 7fa3ac600640 1 -- 172.21.15.161:0/546741126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3a40af760 msgr2=0x7fa3a40afbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.251+0000 7fa3ac600640 1 --2- 172.21.15.161:0/546741126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3a40af760 0x7fa3a40afbe0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa39c01ff40 tx=0x7fa38c005800 comp rx=0 tx=0).stop 2024-05-22T21:14:47.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.251+0000 7fa3ac600640 1 -- 172.21.15.161:0/546741126 shutdown_connections 2024-05-22T21:14:47.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.251+0000 7fa3ac600640 1 --2- 172.21.15.161:0/546741126 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa3840666c0 0x7fa384068b80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.251+0000 7fa3ac600640 1 --2- 172.21.15.161:0/546741126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3a40af760 0x7fa3a40afbe0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.251+0000 7fa3ac600640 1 --2- 172.21.15.161:0/546741126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3a40ce860 0x7fa3a40cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.251+0000 7fa3ac600640 1 --2- 172.21.15.161:0/546741126 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa38c004d10 0x7fa3a4144d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.251+0000 7fa3ac600640 1 -- 172.21.15.161:0/546741126 >> 172.21.15.161:0/546741126 conn(0x7fa3a40b0bc0 msgr2=0x7fa3a4155bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:47.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.255+0000 7fa3ac600640 1 -- 172.21.15.161:0/546741126 shutdown_connections 2024-05-22T21:14:47.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.255+0000 7fa3ac600640 1 -- 172.21.15.161:0/546741126 wait complete. 2024-05-22T21:14:47.277 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:47.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.407+0000 7f8ccd600640 1 Processor -- start 2024-05-22T21:14:47.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.407+0000 7f8ccd600640 1 -- start start 2024-05-22T21:14:47.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.407+0000 7f8ccd600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cc80cc260 0x7f8cc8162b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.407+0000 7f8ccd600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8cc806dd40 0x7f8cc81650b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.407+0000 7f8ccd600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc80cf670 con 0x7f8cc8069ed0 2024-05-22T21:14:47.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.407+0000 7f8ccd600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc80cf120 con 0x7f8cc806dd40 2024-05-22T21:14:47.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.407+0000 7f8ccd600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc8071b00 con 0x7f8cc80cc260 2024-05-22T21:14:47.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.407+0000 7f8cc7e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8cc8069ed0 0x7f8cc806a2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:57662/0 (socket says 172.21.15.161:57662) 2024-05-22T21:14:47.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.407+0000 7f8cc7e00640 1 -- 172.21.15.161:0/1734530665 learned_addr learned my addr 172.21.15.161:0/1734530665 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:47.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.407+0000 7f8cccc00640 1 --2- 172.21.15.161:0/1734530665 >> v2:172.21.15.179:3300/0 conn(0x7f8cc806dd40 0x7f8cc81650b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.407+0000 7f8cc7400640 1 --2- 172.21.15.161:0/1734530665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cc80cc260 0x7f8cc8162b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.407+0000 7f8cc6a00640 1 -- 172.21.15.161:0/1734530665 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431480071 0 0) 0x7f8cc80cf670 con 0x7f8cc8069ed0 2024-05-22T21:14:47.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.407+0000 7f8cc6a00640 1 -- 172.21.15.161:0/1734530665 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cac003530 con 0x7f8cc8069ed0 2024-05-22T21:14:47.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.411+0000 7f8cccc00640 1 -- 172.21.15.161:0/1734530665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cc80cc260 msgr2=0x7f8cc8162b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:47.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.411+0000 7f8cccc00640 1 --2- 172.21.15.161:0/1734530665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cc80cc260 0x7f8cc8162b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.411+0000 7f8cccc00640 1 -- 172.21.15.161:0/1734530665 >> v1:172.21.15.161:6789/0 conn(0x7f8cc8069ed0 legacy=0x7f8cc806a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.411+0000 7f8cc6a00640 1 -- 172.21.15.161:0/1734530665 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 499413212 0 0) 0x7f8cac003530 con 0x7f8cc8069ed0 2024-05-22T21:14:47.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.411+0000 7f8cccc00640 1 -- 172.21.15.161:0/1734530665 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc815c070 con 0x7f8cc806dd40 2024-05-22T21:14:47.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.411+0000 7f8cccc00640 1 --2- 172.21.15.161:0/1734530665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cc806dd40 0x7f8cc81650b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00097c0 tx=0x7f8cc001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=155a015c3ab711cb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:47.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.411+0000 7f8cc6a00640 1 -- 172.21.15.161:0/1734530665 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00ab670 con 0x7f8cc806dd40 2024-05-22T21:14:47.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.411+0000 7f8cc6a00640 1 -- 172.21.15.161:0/1734530665 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc00ab7d0 con 0x7f8cc806dd40 2024-05-22T21:14:47.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.411+0000 7f8cc6a00640 1 -- 172.21.15.161:0/1734530665 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00abb60 con 0x7f8cc806dd40 2024-05-22T21:14:47.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.411+0000 7f8ccd600640 1 -- 172.21.15.161:0/1734530665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cc806dd40 msgr2=0x7f8cc81650b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.411+0000 7f8ccd600640 1 --2- 172.21.15.161:0/1734530665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cc806dd40 0x7f8cc81650b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00097c0 tx=0x7f8cc001c470 comp rx=0 tx=0).stop 2024-05-22T21:14:47.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8ccd600640 1 -- 172.21.15.161:0/1734530665 shutdown_connections 2024-05-22T21:14:47.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8ccd600640 1 --2- 172.21.15.161:0/1734530665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cc806dd40 0x7f8cc81650b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8ccd600640 1 --2- 172.21.15.161:0/1734530665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cc80cc260 0x7f8cc8162b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8ccd600640 1 -- 172.21.15.161:0/1734530665 >> 172.21.15.161:0/1734530665 conn(0x7f8cc80cfc20 msgr2=0x7f8cc80d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:47.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8ccd600640 1 -- 172.21.15.161:0/1734530665 shutdown_connections 2024-05-22T21:14:47.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8ccd600640 1 -- 172.21.15.161:0/1734530665 wait complete. 2024-05-22T21:14:47.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8ccd600640 1 Processor -- start 2024-05-22T21:14:47.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8ccd600640 1 -- start start 2024-05-22T21:14:47.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8ccd600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cc8069ed0 0x7f8cc813d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8ccd600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cc806dd40 0x7f8cc813de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8ccd600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cc80cc260 0x7f8cc80c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8ccd600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc8140550 con 0x7f8cc8069ed0 2024-05-22T21:14:47.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8ccd600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc813f680 con 0x7f8cc806dd40 2024-05-22T21:14:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8ccd600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc813f940 con 0x7f8cc80cc260 2024-05-22T21:14:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8cc7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cc8069ed0 0x7f8cc813d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8cc7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cc8069ed0 0x7f8cc813d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55864/0 (socket says 172.21.15.161:55864) 2024-05-22T21:14:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8cc7e00640 1 -- 172.21.15.161:0/443377276 learned_addr learned my addr 172.21.15.161:0/443377276 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8cc7400640 1 --2- 172.21.15.161:0/443377276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cc806dd40 0x7f8cc813de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8cc7e00640 1 -- 172.21.15.161:0/443377276 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cc80cc260 msgr2=0x7f8cc80c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8cc7e00640 1 --2- 172.21.15.161:0/443377276 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cc80cc260 0x7f8cc80c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8cc7e00640 1 -- 172.21.15.161:0/443377276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cc806dd40 msgr2=0x7f8cc813de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8cc7e00640 1 --2- 172.21.15.161:0/443377276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cc806dd40 0x7f8cc813de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8cc7e00640 1 -- 172.21.15.161:0/443377276 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc0009420 con 0x7f8cc8069ed0 2024-05-22T21:14:47.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8cc7400640 1 --2- 172.21.15.161:0/443377276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cc806dd40 0x7f8cc813de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:47.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8cc7e00640 1 --2- 172.21.15.161:0/443377276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cc8069ed0 0x7f8cc813d8c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800cc10 tx=0x7f8cb800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:47.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8cbfe00640 1 -- 172.21.15.161:0/443377276 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800ad40 con 0x7f8cc8069ed0 2024-05-22T21:14:47.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8cbfe00640 1 -- 172.21.15.161:0/443377276 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb8012400 con 0x7f8cc8069ed0 2024-05-22T21:14:47.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8cbfe00640 1 -- 172.21.15.161:0/443377276 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800aea0 con 0x7f8cc8069ed0 2024-05-22T21:14:47.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8ccd600640 1 -- 172.21.15.161:0/443377276 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc8144e30 con 0x7f8cc8069ed0 2024-05-22T21:14:47.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.415+0000 7f8ccd600640 1 -- 172.21.15.161:0/443377276 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc813ff00 con 0x7f8cc8069ed0 2024-05-22T21:14:47.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.419+0000 7f8ccd600640 1 -- 172.21.15.161:0/443377276 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc8067070 con 0x7f8cc8069ed0 2024-05-22T21:14:47.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.419+0000 7f8cbfe00640 1 -- 172.21.15.161:0/443377276 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8cb80127d0 con 0x7f8cc8069ed0 2024-05-22T21:14:47.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.419+0000 7f8cbfe00640 1 --2- 172.21.15.161:0/443377276 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ca80666c0 0x7f8ca8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.419+0000 7f8cbfe00640 1 -- 172.21.15.161:0/443377276 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8cb8084480 con 0x7f8cc8069ed0 2024-05-22T21:14:47.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.419+0000 7f8cc7400640 1 --2- 172.21.15.161:0/443377276 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ca80666c0 0x7f8ca8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.419+0000 7f8cc7400640 1 --2- 172.21.15.161:0/443377276 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ca80666c0 0x7f8ca8068b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8067ad0 tx=0x7f8cb4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:47.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.423+0000 7f8cbfe00640 1 -- 172.21.15.161:0/443377276 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8cb8051b30 con 0x7f8cc8069ed0 2024-05-22T21:14:47.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.579+0000 7f8ccd600640 1 -- 172.21.15.161:0/443377276 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cc813a260 con 0x7f8cc8069ed0 2024-05-22T21:14:47.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.579+0000 7f8cbfe00640 1 -- 172.21.15.161:0/443377276 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+12437 (secure 0 0 0) 0x7f8cb8055150 con 0x7f8cc8069ed0 2024-05-22T21:14:47.586 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:47.587 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":43,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:14:46.859323+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:47.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.583+0000 7f8ccd600640 1 -- 172.21.15.161:0/443377276 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ca80666c0 msgr2=0x7f8ca8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.583+0000 7f8ccd600640 1 --2- 172.21.15.161:0/443377276 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ca80666c0 0x7f8ca8068b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8067ad0 tx=0x7f8cb4008040 comp rx=0 tx=0).stop 2024-05-22T21:14:47.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.583+0000 7f8ccd600640 1 -- 172.21.15.161:0/443377276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cc8069ed0 msgr2=0x7f8cc813d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.583+0000 7f8ccd600640 1 --2- 172.21.15.161:0/443377276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cc8069ed0 0x7f8cc813d8c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800cc10 tx=0x7f8cb800a430 comp rx=0 tx=0).stop 2024-05-22T21:14:47.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.583+0000 7f8ccd600640 1 -- 172.21.15.161:0/443377276 shutdown_connections 2024-05-22T21:14:47.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.583+0000 7f8ccd600640 1 --2- 172.21.15.161:0/443377276 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ca80666c0 0x7f8ca8068b80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.583+0000 7f8ccd600640 1 --2- 172.21.15.161:0/443377276 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8cc80cc260 0x7f8cc80c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.583+0000 7f8ccd600640 1 --2- 172.21.15.161:0/443377276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cc806dd40 0x7f8cc813de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.583+0000 7f8ccd600640 1 --2- 172.21.15.161:0/443377276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cc8069ed0 0x7f8cc813d8c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.583+0000 7f8ccd600640 1 -- 172.21.15.161:0/443377276 >> 172.21.15.161:0/443377276 conn(0x7f8cc80cfc20 msgr2=0x7f8cc813a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:47.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.583+0000 7f8ccd600640 1 -- 172.21.15.161:0/443377276 shutdown_connections 2024-05-22T21:14:47.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.583+0000 7f8ccd600640 1 -- 172.21.15.161:0/443377276 wait complete. 2024-05-22T21:14:47.605 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:14:47.606 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:14:47.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.739+0000 7f4c53000640 1 Processor -- start 2024-05-22T21:14:47.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.739+0000 7f4c53000640 1 -- start start 2024-05-22T21:14:47.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.739+0000 7f4c53000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4c4c143c40 0x7f4c4c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.739+0000 7f4c53000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4c4c1444a0 0x7f4c4c06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.739+0000 7f4c53000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4c15efb0 con 0x7f4c4c144900 2024-05-22T21:14:47.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.739+0000 7f4c53000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4c15f290 con 0x7f4c4c1444a0 2024-05-22T21:14:47.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.739+0000 7f4c53000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4c15f570 con 0x7f4c4c143c40 2024-05-22T21:14:47.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.739+0000 7f4c4b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4c4c1444a0 0x7f4c4c06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.739+0000 7f4c4b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4c4c1444a0 0x7f4c4c06c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55880/0 (socket says 172.21.15.161:55880) 2024-05-22T21:14:47.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.739+0000 7f4c4b400640 1 -- 172.21.15.161:0/3841618946 learned_addr learned my addr 172.21.15.161:0/3841618946 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:47.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.739+0000 7f4c4be00640 1 --2- 172.21.15.161:0/3841618946 >> v2:172.21.15.179:3300/0 conn(0x7f4c4c143c40 0x7f4c4c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.739+0000 7f4c4b400640 1 -- 172.21.15.161:0/3841618946 >> v2:172.21.15.179:3300/0 conn(0x7f4c4c143c40 msgr2=0x7f4c4c15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:47.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.739+0000 7f4c4b400640 1 --2- 172.21.15.161:0/3841618946 >> v2:172.21.15.179:3300/0 conn(0x7f4c4c143c40 0x7f4c4c15b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.739+0000 7f4c4b400640 1 -- 172.21.15.161:0/3841618946 >> v1:172.21.15.179:6789/0 conn(0x7f4c4c144900 legacy=0x7f4c4c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.739+0000 7f4c4b400640 1 -- 172.21.15.161:0/3841618946 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c15f850 con 0x7f4c4c1444a0 2024-05-22T21:14:47.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.743+0000 7f4c4b400640 1 --2- 172.21.15.161:0/3841618946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c4c1444a0 0x7f4c4c06c2a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0724b0 tx=0x7f4c3801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29cfd745713d78c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:47.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.743+0000 7f4c4aa00640 1 -- 172.21.15.161:0/3841618946 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c380a7070 con 0x7f4c4c1444a0 2024-05-22T21:14:47.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.743+0000 7f4c4aa00640 1 -- 172.21.15.161:0/3841618946 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c380ab430 con 0x7f4c4c1444a0 2024-05-22T21:14:47.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.743+0000 7f4c4aa00640 1 -- 172.21.15.161:0/3841618946 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c380ab7c0 con 0x7f4c4c1444a0 2024-05-22T21:14:47.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.743+0000 7f4c53000640 1 -- 172.21.15.161:0/3841618946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c4c1444a0 msgr2=0x7f4c4c06c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.743+0000 7f4c53000640 1 --2- 172.21.15.161:0/3841618946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c4c1444a0 0x7f4c4c06c2a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0724b0 tx=0x7f4c3801c520 comp rx=0 tx=0).stop 2024-05-22T21:14:47.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.743+0000 7f4c53000640 1 -- 172.21.15.161:0/3841618946 shutdown_connections 2024-05-22T21:14:47.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.743+0000 7f4c53000640 1 --2- 172.21.15.161:0/3841618946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c4c1444a0 0x7f4c4c06c2a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.743+0000 7f4c53000640 1 --2- 172.21.15.161:0/3841618946 >> v2:172.21.15.179:3300/0 conn(0x7f4c4c143c40 0x7f4c4c15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.743+0000 7f4c53000640 1 -- 172.21.15.161:0/3841618946 >> 172.21.15.161:0/3841618946 conn(0x7f4c4c06a2d0 msgr2=0x7f4c4c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:47.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.743+0000 7f4c53000640 1 -- 172.21.15.161:0/3841618946 shutdown_connections 2024-05-22T21:14:47.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.743+0000 7f4c53000640 1 -- 172.21.15.161:0/3841618946 wait complete. 2024-05-22T21:14:47.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c53000640 1 Processor -- start 2024-05-22T21:14:47.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c53000640 1 -- start start 2024-05-22T21:14:47.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c53000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c4c143c40 0x7f4c4c13ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c53000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4c4c1444a0 0x7f4c4c0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c53000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c4c144900 0x7f4c4c159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c53000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c0ad290 con 0x7f4c4c143c40 2024-05-22T21:14:47.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c53000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c0ad570 con 0x7f4c4c144900 2024-05-22T21:14:47.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c53000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c06fd80 con 0x7f4c4c1444a0 2024-05-22T21:14:47.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c4be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c4c143c40 0x7f4c4c13ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c4be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c4c143c40 0x7f4c4c13ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55882/0 (socket says 172.21.15.161:55882) 2024-05-22T21:14:47.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c4be00640 1 -- 172.21.15.161:0/233633599 learned_addr learned my addr 172.21.15.161:0/233633599 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:47.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c51000640 1 --2- 172.21.15.161:0/233633599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c4c144900 0x7f4c4c159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c4b400640 1 --2- 172.21.15.161:0/233633599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4c4c1444a0 0x7f4c4c0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c4be00640 1 -- 172.21.15.161:0/233633599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4c4c1444a0 msgr2=0x7f4c4c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c4be00640 1 --2- 172.21.15.161:0/233633599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4c4c1444a0 0x7f4c4c0ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c4be00640 1 -- 172.21.15.161:0/233633599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c4c144900 msgr2=0x7f4c4c159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c4be00640 1 --2- 172.21.15.161:0/233633599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c4c144900 0x7f4c4c159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c4be00640 1 -- 172.21.15.161:0/233633599 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c380094d0 con 0x7f4c4c143c40 2024-05-22T21:14:47.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c4b400640 1 --2- 172.21.15.161:0/233633599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4c4c1444a0 0x7f4c4c0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:47.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c4be00640 1 --2- 172.21.15.161:0/233633599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c4c143c40 0x7f4c4c13ccf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00d800 tx=0x7f4c3c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:47.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c43e00640 1 -- 172.21.15.161:0/233633599 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00b6c0 con 0x7f4c4c143c40 2024-05-22T21:14:47.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c43e00640 1 -- 172.21.15.161:0/233633599 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c00b820 con 0x7f4c4c143c40 2024-05-22T21:14:47.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c43e00640 1 -- 172.21.15.161:0/233633599 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00bb90 con 0x7f4c4c143c40 2024-05-22T21:14:47.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c53000640 1 -- 172.21.15.161:0/233633599 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4c070060 con 0x7f4c4c143c40 2024-05-22T21:14:47.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c53000640 1 -- 172.21.15.161:0/233633599 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4c0ad850 con 0x7f4c4c143c40 2024-05-22T21:14:47.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.747+0000 7f4c53000640 1 -- 172.21.15.161:0/233633599 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4c067220 con 0x7f4c4c143c40 2024-05-22T21:14:47.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.751+0000 7f4c43e00640 1 -- 172.21.15.161:0/233633599 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4c3c00bcf0 con 0x7f4c4c143c40 2024-05-22T21:14:47.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.751+0000 7f4c43e00640 1 --2- 172.21.15.161:0/233633599 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4c2c066690 0x7f4c2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:47.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.751+0000 7f4c43e00640 1 -- 172.21.15.161:0/233633599 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4c3c084470 con 0x7f4c4c143c40 2024-05-22T21:14:47.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.751+0000 7f4c4b400640 1 --2- 172.21.15.161:0/233633599 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4c2c066690 0x7f4c2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:47.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.751+0000 7f4c4b400640 1 --2- 172.21.15.161:0/233633599 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4c2c066690 0x7f4c2c068b50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4c380af000 tx=0x7f4c380a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:47.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.755+0000 7f4c43e00640 1 -- 172.21.15.161:0/233633599 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4c3c051a00 con 0x7f4c4c143c40 2024-05-22T21:14:47.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.907+0000 7f4c53000640 1 -- 172.21.15.161:0/233633599 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c4c0ceaa0 con 0x7f4c4c143c40 2024-05-22T21:14:47.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.907+0000 7f4c43e00640 1 -- 172.21.15.161:0/233633599 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c3c055020 con 0x7f4c4c143c40 2024-05-22T21:14:47.911 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:14:47.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.907+0000 7f4c53000640 1 -- 172.21.15.161:0/233633599 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4c2c066690 msgr2=0x7f4c2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.907+0000 7f4c53000640 1 --2- 172.21.15.161:0/233633599 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4c2c066690 0x7f4c2c068b50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4c380af000 tx=0x7f4c380a7000 comp rx=0 tx=0).stop 2024-05-22T21:14:47.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.907+0000 7f4c53000640 1 -- 172.21.15.161:0/233633599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c4c143c40 msgr2=0x7f4c4c13ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:47.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.907+0000 7f4c53000640 1 --2- 172.21.15.161:0/233633599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c4c143c40 0x7f4c4c13ccf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00d800 tx=0x7f4c3c00dcc0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.907+0000 7f4c53000640 1 -- 172.21.15.161:0/233633599 shutdown_connections 2024-05-22T21:14:47.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.907+0000 7f4c53000640 1 --2- 172.21.15.161:0/233633599 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4c2c066690 0x7f4c2c068b50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.907+0000 7f4c53000640 1 --2- 172.21.15.161:0/233633599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c4c144900 0x7f4c4c159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.907+0000 7f4c53000640 1 --2- 172.21.15.161:0/233633599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4c4c1444a0 0x7f4c4c0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.907+0000 7f4c53000640 1 --2- 172.21.15.161:0/233633599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c4c143c40 0x7f4c4c13ccf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:47.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.907+0000 7f4c53000640 1 -- 172.21.15.161:0/233633599 >> 172.21.15.161:0/233633599 conn(0x7f4c4c06a2d0 msgr2=0x7f4c4c0cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:47.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.907+0000 7f4c53000640 1 -- 172.21.15.161:0/233633599 shutdown_connections 2024-05-22T21:14:47.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:47.911+0000 7f4c53000640 1 -- 172.21.15.161:0/233633599 wait complete. 2024-05-22T21:14:47.932 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-22T21:14:48.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.063+0000 7ff95e000640 1 Processor -- start 2024-05-22T21:14:48.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.063+0000 7ff95e000640 1 -- start start 2024-05-22T21:14:48.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.063+0000 7ff95e000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff9580b4fa0 0x7ff95815e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.063+0000 7ff95e000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff958160f70 0x7ff958161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.063+0000 7ff95e000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff95815e000 con 0x7ff958069ed0 2024-05-22T21:14:48.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.063+0000 7ff95e000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95815e9d0 con 0x7ff9580b4fa0 2024-05-22T21:14:48.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.063+0000 7ff95e000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff958161a30 con 0x7ff958160f70 2024-05-22T21:14:48.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.063+0000 7ff957400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff9580b4fa0 0x7ff95815e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.063+0000 7ff957400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff9580b4fa0 0x7ff95815e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55894/0 (socket says 172.21.15.161:55894) 2024-05-22T21:14:48.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.063+0000 7ff957400640 1 -- 172.21.15.161:0/1085280719 learned_addr learned my addr 172.21.15.161:0/1085280719 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:48.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.063+0000 7ff957400640 1 -- 172.21.15.161:0/1085280719 >> v2:172.21.15.179:3300/0 conn(0x7ff958160f70 msgr2=0x7ff958161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:48.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.063+0000 7ff957400640 1 --2- 172.21.15.161:0/1085280719 >> v2:172.21.15.179:3300/0 conn(0x7ff958160f70 0x7ff958161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.063+0000 7ff957400640 1 -- 172.21.15.161:0/1085280719 >> v1:172.21.15.179:6789/0 conn(0x7ff958069ed0 legacy=0x7ff9580cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:48.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.063+0000 7ff957400640 1 -- 172.21.15.161:0/1085280719 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff958161d10 con 0x7ff9580b4fa0 2024-05-22T21:14:48.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.067+0000 7ff957400640 1 --2- 172.21.15.161:0/1085280719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9580b4fa0 0x7ff95815e490 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff958071d20 tx=0x7ff94801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809659ab9b5afd84 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:48.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.067+0000 7ff956000640 1 -- 172.21.15.161:0/1085280719 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9480ab670 con 0x7ff9580b4fa0 2024-05-22T21:14:48.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.067+0000 7ff956000640 1 -- 172.21.15.161:0/1085280719 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9480ab7d0 con 0x7ff9580b4fa0 2024-05-22T21:14:48.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.067+0000 7ff956000640 1 -- 172.21.15.161:0/1085280719 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9480abb60 con 0x7ff9580b4fa0 2024-05-22T21:14:48.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.067+0000 7ff95e000640 1 -- 172.21.15.161:0/1085280719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9580b4fa0 msgr2=0x7ff95815e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:48.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.067+0000 7ff95e000640 1 --2- 172.21.15.161:0/1085280719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9580b4fa0 0x7ff95815e490 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff958071d20 tx=0x7ff94801c4b0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.067+0000 7ff95e000640 1 -- 172.21.15.161:0/1085280719 shutdown_connections 2024-05-22T21:14:48.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.067+0000 7ff95e000640 1 --2- 172.21.15.161:0/1085280719 >> v2:172.21.15.179:3300/0 conn(0x7ff958160f70 0x7ff958161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.067+0000 7ff95e000640 1 --2- 172.21.15.161:0/1085280719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9580b4fa0 0x7ff95815e490 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.067+0000 7ff95e000640 1 -- 172.21.15.161:0/1085280719 >> 172.21.15.161:0/1085280719 conn(0x7ff9580b1c40 msgr2=0x7ff9580b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:48.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.067+0000 7ff95e000640 1 -- 172.21.15.161:0/1085280719 shutdown_connections 2024-05-22T21:14:48.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.067+0000 7ff95e000640 1 -- 172.21.15.161:0/1085280719 wait complete. 2024-05-22T21:14:48.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff95e000640 1 Processor -- start 2024-05-22T21:14:48.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff95e000640 1 -- start start 2024-05-22T21:14:48.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff95e000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff958069ed0 0x7ff95813c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff95e000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9580b4fa0 0x7ff958140050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff95e000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff958160f70 0x7ff958145050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff95e000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95813e780 con 0x7ff958069ed0 2024-05-22T21:14:48.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff95e000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95813d120 con 0x7ff958160f70 2024-05-22T21:14:48.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff95e000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95813d400 con 0x7ff9580b4fa0 2024-05-22T21:14:48.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff957400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff958069ed0 0x7ff95813c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff957400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff958069ed0 0x7ff95813c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55910/0 (socket says 172.21.15.161:55910) 2024-05-22T21:14:48.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff957400640 1 -- 172.21.15.161:0/3474377514 learned_addr learned my addr 172.21.15.161:0/3474377514 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:48.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff956a00640 1 --2- 172.21.15.161:0/3474377514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9580b4fa0 0x7ff958140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff957e00640 1 --2- 172.21.15.161:0/3474377514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff958160f70 0x7ff958145050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff957400640 1 -- 172.21.15.161:0/3474377514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9580b4fa0 msgr2=0x7ff958140050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:48.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff957400640 1 --2- 172.21.15.161:0/3474377514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9580b4fa0 0x7ff958140050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff957400640 1 -- 172.21.15.161:0/3474377514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff958160f70 msgr2=0x7ff958145050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:48.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff957400640 1 --2- 172.21.15.161:0/3474377514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff958160f70 0x7ff958145050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff957400640 1 -- 172.21.15.161:0/3474377514 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9480094d0 con 0x7ff958069ed0 2024-05-22T21:14:48.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff957e00640 1 --2- 172.21.15.161:0/3474377514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff958160f70 0x7ff958145050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:48.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff956a00640 1 --2- 172.21.15.161:0/3474377514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9580b4fa0 0x7ff958140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:48.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff957400640 1 --2- 172.21.15.161:0/3474377514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff958069ed0 0x7ff95813c740 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff9480ad040 tx=0x7ff94801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:48.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff94f400640 1 -- 172.21.15.161:0/3474377514 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9480b4800 con 0x7ff958069ed0 2024-05-22T21:14:48.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff94f400640 1 -- 172.21.15.161:0/3474377514 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9480b4980 con 0x7ff958069ed0 2024-05-22T21:14:48.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff94f400640 1 -- 172.21.15.161:0/3474377514 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9480b4d30 con 0x7ff958069ed0 2024-05-22T21:14:48.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff95e000640 1 -- 172.21.15.161:0/3474377514 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95813d6e0 con 0x7ff958069ed0 2024-05-22T21:14:48.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.071+0000 7ff95e000640 1 -- 172.21.15.161:0/3474377514 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95813dca0 con 0x7ff958069ed0 2024-05-22T21:14:48.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.075+0000 7ff95e000640 1 -- 172.21.15.161:0/3474377514 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff95813de90 con 0x7ff958069ed0 2024-05-22T21:14:48.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.075+0000 7ff94f400640 1 -- 172.21.15.161:0/3474377514 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff9480b3380 con 0x7ff958069ed0 2024-05-22T21:14:48.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.075+0000 7ff94f400640 1 --2- 172.21.15.161:0/3474377514 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff938066690 0x7ff938068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.075+0000 7ff94f400640 1 -- 172.21.15.161:0/3474377514 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff94811e6a0 con 0x7ff958069ed0 2024-05-22T21:14:48.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.075+0000 7ff956a00640 1 --2- 172.21.15.161:0/3474377514 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff938066690 0x7ff938068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.075+0000 7ff956a00640 1 --2- 172.21.15.161:0/3474377514 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff938066690 0x7ff938068b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff9581609c0 tx=0x7ff944008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:48.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.079+0000 7ff94f400640 1 -- 172.21.15.161:0/3474377514 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff9480ebe10 con 0x7ff958069ed0 2024-05-22T21:14:48.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.235+0000 7ff95e000640 1 -- 172.21.15.161:0/3474377514 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff958067220 con 0x7ff958069ed0 2024-05-22T21:14:48.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.235+0000 7ff94f400640 1 -- 172.21.15.161:0/3474377514 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff9480ef430 con 0x7ff958069ed0 2024-05-22T21:14:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:48.238 INFO:teuthology.orchestra.run.smithi161.stdout:[ 2024-05-22T21:14:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:14:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: "poolnum": 1, 2024-05-22T21:14:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: "poolname": ".mgr" 2024-05-22T21:14:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:14:48.239 INFO:teuthology.orchestra.run.smithi161.stdout:] 2024-05-22T21:14:48.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.235+0000 7ff95e000640 1 -- 172.21.15.161:0/3474377514 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff938066690 msgr2=0x7ff938068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:48.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.235+0000 7ff95e000640 1 --2- 172.21.15.161:0/3474377514 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff938066690 0x7ff938068b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff9581609c0 tx=0x7ff944008040 comp rx=0 tx=0).stop 2024-05-22T21:14:48.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.235+0000 7ff95e000640 1 -- 172.21.15.161:0/3474377514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff958069ed0 msgr2=0x7ff95813c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:48.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.235+0000 7ff95e000640 1 --2- 172.21.15.161:0/3474377514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff958069ed0 0x7ff95813c740 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff9480ad040 tx=0x7ff94801acf0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.235+0000 7ff95e000640 1 -- 172.21.15.161:0/3474377514 shutdown_connections 2024-05-22T21:14:48.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.235+0000 7ff95e000640 1 --2- 172.21.15.161:0/3474377514 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff938066690 0x7ff938068b50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.235+0000 7ff95e000640 1 --2- 172.21.15.161:0/3474377514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff958160f70 0x7ff958145050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.235+0000 7ff95e000640 1 --2- 172.21.15.161:0/3474377514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9580b4fa0 0x7ff958140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.235+0000 7ff95e000640 1 --2- 172.21.15.161:0/3474377514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff958069ed0 0x7ff95813c740 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.235+0000 7ff95e000640 1 -- 172.21.15.161:0/3474377514 >> 172.21.15.161:0/3474377514 conn(0x7ff9580b1c40 msgr2=0x7ff9580cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:48.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.235+0000 7ff95e000640 1 -- 172.21.15.161:0/3474377514 shutdown_connections 2024-05-22T21:14:48.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.235+0000 7ff95e000640 1 -- 172.21.15.161:0/3474377514 wait complete. 2024-05-22T21:14:48.258 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-22T21:14:48.259 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-22T21:14:48.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.395+0000 7fdc6ce00640 1 Processor -- start 2024-05-22T21:14:48.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.395+0000 7fdc6ce00640 1 -- start start 2024-05-22T21:14:48.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.395+0000 7fdc6ce00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdc680ccc10 0x7fdc6815c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.395+0000 7fdc6ce00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6806d0d0 con 0x7fdc6806d560 2024-05-22T21:14:48.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.395+0000 7fdc6ce00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6806c4c0 con 0x7fdc68144980 2024-05-22T21:14:48.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.395+0000 7fdc6ce00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6806c7a0 con 0x7fdc680ccc10 2024-05-22T21:14:48.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.395+0000 7fdc5fe00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fdc6806d560 0x7fdc680cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:57666/0 (socket says 172.21.15.161:57666) 2024-05-22T21:14:48.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.395+0000 7fdc5fe00640 1 -- 172.21.15.161:0/181337203 learned_addr learned my addr 172.21.15.161:0/181337203 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:48.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.395+0000 7fdc66800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdc680ccc10 0x7fdc6815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.395+0000 7fdc66800640 1 -- 172.21.15.161:0/181337203 >> v1:172.21.15.179:6789/0 conn(0x7fdc68144980 legacy=0x7fdc6815c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:48.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.395+0000 7fdc66800640 1 -- 172.21.15.161:0/181337203 >> v1:172.21.15.161:6789/0 conn(0x7fdc6806d560 legacy=0x7fdc680cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:48.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.395+0000 7fdc66800640 1 -- 172.21.15.161:0/181337203 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6806ca80 con 0x7fdc680ccc10 2024-05-22T21:14:48.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.395+0000 7fdc65400640 1 -- 172.21.15.161:0/181337203 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362003235 0 0) 0x7fdc6806d0d0 con 0x7fdc6806d560 2024-05-22T21:14:48.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.399+0000 7fdc66800640 1 --2- 172.21.15.161:0/181337203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680ccc10 0x7fdc6815c880 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009d90 tx=0x7fdc6001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa432f32b8ccac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:48.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.399+0000 7fdc65400640 1 -- 172.21.15.161:0/181337203 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc600a58c0 con 0x7fdc680ccc10 2024-05-22T21:14:48.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.399+0000 7fdc65400640 1 -- 172.21.15.161:0/181337203 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc600a5a20 con 0x7fdc680ccc10 2024-05-22T21:14:48.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.399+0000 7fdc65400640 1 -- 172.21.15.161:0/181337203 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc600a5db0 con 0x7fdc680ccc10 2024-05-22T21:14:48.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.399+0000 7fdc6ce00640 1 -- 172.21.15.161:0/181337203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680ccc10 msgr2=0x7fdc6815c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:48.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.399+0000 7fdc6ce00640 1 --2- 172.21.15.161:0/181337203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680ccc10 0x7fdc6815c880 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009d90 tx=0x7fdc6001c970 comp rx=0 tx=0).stop 2024-05-22T21:14:48.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.399+0000 7fdc6ce00640 1 -- 172.21.15.161:0/181337203 shutdown_connections 2024-05-22T21:14:48.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.399+0000 7fdc6ce00640 1 --2- 172.21.15.161:0/181337203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680ccc10 0x7fdc6815c880 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.399+0000 7fdc6ce00640 1 -- 172.21.15.161:0/181337203 >> 172.21.15.161:0/181337203 conn(0x7fdc6806a0a0 msgr2=0x7fdc6806a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:48.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.399+0000 7fdc6ce00640 1 -- 172.21.15.161:0/181337203 shutdown_connections 2024-05-22T21:14:48.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.399+0000 7fdc6ce00640 1 -- 172.21.15.161:0/181337203 wait complete. 2024-05-22T21:14:48.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.399+0000 7fdc6ce00640 1 Processor -- start 2024-05-22T21:14:48.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc6ce00640 1 -- start start 2024-05-22T21:14:48.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc6ce00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc6806d560 0x7fdc681360b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc6ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680ccc10 0x7fdc6813e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc6ce00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc68144980 0x7fdc6815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc6ce00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc68137430 con 0x7fdc680ccc10 2024-05-22T21:14:48.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc6ce00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc68155590 con 0x7fdc68144980 2024-05-22T21:14:48.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc6ce00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6813e8b0 con 0x7fdc6806d560 2024-05-22T21:14:48.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc5fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680ccc10 0x7fdc6813e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc5fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680ccc10 0x7fdc6813e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55936/0 (socket says 172.21.15.161:55936) 2024-05-22T21:14:48.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc5fe00640 1 -- 172.21.15.161:0/2405797232 learned_addr learned my addr 172.21.15.161:0/2405797232 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:48.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc66800640 1 --2- 172.21.15.161:0/2405797232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc68144980 0x7fdc6815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc65e00640 1 --2- 172.21.15.161:0/2405797232 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc6806d560 0x7fdc681360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc5fe00640 1 -- 172.21.15.161:0/2405797232 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc6806d560 msgr2=0x7fdc681360b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:48.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc5fe00640 1 --2- 172.21.15.161:0/2405797232 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc6806d560 0x7fdc681360b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc5fe00640 1 -- 172.21.15.161:0/2405797232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc68144980 msgr2=0x7fdc6815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:48.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc5fe00640 1 --2- 172.21.15.161:0/2405797232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc68144980 0x7fdc6815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc5fe00640 1 -- 172.21.15.161:0/2405797232 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc60009a70 con 0x7fdc680ccc10 2024-05-22T21:14:48.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc65e00640 1 --2- 172.21.15.161:0/2405797232 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc6806d560 0x7fdc681360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:48.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc5fe00640 1 --2- 172.21.15.161:0/2405797232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680ccc10 0x7fdc6813e370 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c4a0 tx=0x7fdc5000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:48.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc5e000640 1 -- 172.21.15.161:0/2405797232 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500074e0 con 0x7fdc680ccc10 2024-05-22T21:14:48.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc5e000640 1 -- 172.21.15.161:0/2405797232 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc50007640 con 0x7fdc680ccc10 2024-05-22T21:14:48.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc5e000640 1 -- 172.21.15.161:0/2405797232 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50004a30 con 0x7fdc680ccc10 2024-05-22T21:14:48.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc6ce00640 1 -- 172.21.15.161:0/2405797232 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc681367b0 con 0x7fdc680ccc10 2024-05-22T21:14:48.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc6ce00640 1 -- 172.21.15.161:0/2405797232 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc68136d70 con 0x7fdc680ccc10 2024-05-22T21:14:48.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.403+0000 7fdc6ce00640 1 -- 172.21.15.161:0/2405797232 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc68137570 con 0x7fdc680ccc10 2024-05-22T21:14:48.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.407+0000 7fdc5e000640 1 -- 172.21.15.161:0/2405797232 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdc50004b90 con 0x7fdc680ccc10 2024-05-22T21:14:48.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.407+0000 7fdc5e000640 1 --2- 172.21.15.161:0/2405797232 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc44066690 0x7fdc44068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.407+0000 7fdc5e000640 1 -- 172.21.15.161:0/2405797232 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdc50082f50 con 0x7fdc680ccc10 2024-05-22T21:14:48.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.407+0000 7fdc65e00640 1 --2- 172.21.15.161:0/2405797232 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc44066690 0x7fdc44068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.407+0000 7fdc65e00640 1 --2- 172.21.15.161:0/2405797232 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc44066690 0x7fdc44068b50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdc68155770 tx=0x7fdc58009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:48.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.411+0000 7fdc5e000640 1 -- 172.21.15.161:0/2405797232 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdc68137570 con 0x7fdc680ccc10 2024-05-22T21:14:48.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.563+0000 7fdc6ce00640 1 -- 172.21.15.161:0/2405797232 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdc68137570 con 0x7fdc680ccc10 2024-05-22T21:14:48.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.667+0000 7fdc5e000640 1 -- 172.21.15.161:0/2405797232 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdc68137570 con 0x7fdc680ccc10 2024-05-22T21:14:48.670 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' created 2024-05-22T21:14:48.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.671+0000 7fdc6ce00640 1 -- 172.21.15.161:0/2405797232 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc44066690 msgr2=0x7fdc44068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:48.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.671+0000 7fdc6ce00640 1 --2- 172.21.15.161:0/2405797232 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc44066690 0x7fdc44068b50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdc68155770 tx=0x7fdc58009040 comp rx=0 tx=0).stop 2024-05-22T21:14:48.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.671+0000 7fdc6ce00640 1 -- 172.21.15.161:0/2405797232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680ccc10 msgr2=0x7fdc6813e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:48.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.671+0000 7fdc6ce00640 1 --2- 172.21.15.161:0/2405797232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680ccc10 0x7fdc6813e370 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c4a0 tx=0x7fdc5000c960 comp rx=0 tx=0).stop 2024-05-22T21:14:48.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.671+0000 7fdc6ce00640 1 -- 172.21.15.161:0/2405797232 shutdown_connections 2024-05-22T21:14:48.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.671+0000 7fdc6ce00640 1 --2- 172.21.15.161:0/2405797232 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc44066690 0x7fdc44068b50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.671+0000 7fdc6ce00640 1 --2- 172.21.15.161:0/2405797232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc68144980 0x7fdc6815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.671+0000 7fdc6ce00640 1 --2- 172.21.15.161:0/2405797232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680ccc10 0x7fdc6813e370 secure :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c4a0 tx=0x7fdc5000c960 comp rx=0 tx=0).stop 2024-05-22T21:14:48.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.671+0000 7fdc6ce00640 1 --2- 172.21.15.161:0/2405797232 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc6806d560 0x7fdc681360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.671+0000 7fdc6ce00640 1 -- 172.21.15.161:0/2405797232 >> 172.21.15.161:0/2405797232 conn(0x7fdc6806a0a0 msgr2=0x7fdc68143b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:48.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.671+0000 7fdc6ce00640 1 -- 172.21.15.161:0/2405797232 shutdown_connections 2024-05-22T21:14:48.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.671+0000 7fdc6ce00640 1 -- 172.21.15.161:0/2405797232 wait complete. 2024-05-22T21:14:48.694 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-22T21:14:48.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.831+0000 7f6e4ca00640 1 Processor -- start 2024-05-22T21:14:48.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.831+0000 7f6e4ca00640 1 -- start start 2024-05-22T21:14:48.837 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e4ca00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6e480b33d0 0x7f6e480cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.837 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e4ca00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e48070b70 con 0x7f6e4806fa80 2024-05-22T21:14:48.837 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e4ca00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4806ff60 con 0x7f6e481436d0 2024-05-22T21:14:48.838 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e4ca00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e480cc720 con 0x7f6e480b33d0 2024-05-22T21:14:48.838 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e47200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f6e4806fa80 0x7f6e48071000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:57680/0 (socket says 172.21.15.161:57680) 2024-05-22T21:14:48.838 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e47200640 1 -- 172.21.15.161:0/1300123350 learned_addr learned my addr 172.21.15.161:0/1300123350 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:48.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e3de00640 1 --2- 172.21.15.161:0/1300123350 >> v2:172.21.15.179:3300/0 conn(0x7f6e480b33d0 0x7f6e480cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e45e00640 1 -- 172.21.15.161:0/1300123350 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832550404 0 0) 0x7f6e48070b70 con 0x7f6e4806fa80 2024-05-22T21:14:48.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e45e00640 1 -- 172.21.15.161:0/1300123350 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e2c003530 con 0x7f6e4806fa80 2024-05-22T21:14:48.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e45e00640 1 -- 172.21.15.161:0/1300123350 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 520682823 0 0) 0x7f6e2c003530 con 0x7f6e4806fa80 2024-05-22T21:14:48.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e45e00640 1 -- 172.21.15.161:0/1300123350 >> v2:172.21.15.179:3300/0 conn(0x7f6e480b33d0 msgr2=0x7f6e480cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:48.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e45e00640 1 --2- 172.21.15.161:0/1300123350 >> v2:172.21.15.179:3300/0 conn(0x7f6e480b33d0 0x7f6e480cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e45e00640 1 -- 172.21.15.161:0/1300123350 >> v1:172.21.15.179:6789/0 conn(0x7f6e481436d0 legacy=0x7f6e480b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:48.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e45e00640 1 -- 172.21.15.161:0/1300123350 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e48070240 con 0x7f6e4806fa80 2024-05-22T21:14:48.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e45e00640 1 -- 172.21.15.161:0/1300123350 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f6e40001680 con 0x7f6e4806fa80 2024-05-22T21:14:48.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e3de00640 1 --2- 172.21.15.161:0/1300123350 >> v2:172.21.15.179:3300/0 conn(0x7f6e480b33d0 0x7f6e480cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:48.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e45e00640 1 -- 172.21.15.161:0/1300123350 >> v1:172.21.15.161:6789/0 conn(0x7f6e4806fa80 legacy=0x7f6e48071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:48.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e45e00640 1 --2- 172.21.15.161:0/1300123350 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e2c003980 0x7f6e2c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e45e00640 1 --2- 172.21.15.161:0/1300123350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e2c0047d0 0x7f6e2c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e45e00640 1 --2- 172.21.15.161:0/1300123350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e2c005190 0x7f6e2c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e45e00640 1 -- 172.21.15.161:0/1300123350 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e480cc720 con 0x7f6e2c005190 2024-05-22T21:14:48.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e45e00640 1 -- 172.21.15.161:0/1300123350 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c00c130 con 0x7f6e2c0047d0 2024-05-22T21:14:48.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e45e00640 1 -- 172.21.15.161:0/1300123350 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c00c2f0 con 0x7f6e2c003980 2024-05-22T21:14:48.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e47200640 1 --2- 172.21.15.161:0/1300123350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e2c005190 0x7f6e2c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e3de00640 1 --2- 172.21.15.161:0/1300123350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e2c0047d0 0x7f6e2c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e46800640 1 --2- 172.21.15.161:0/1300123350 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e2c003980 0x7f6e2c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e47200640 1 -- 172.21.15.161:0/1300123350 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e2c003980 msgr2=0x7f6e2c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:48.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e47200640 1 --2- 172.21.15.161:0/1300123350 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e2c003980 0x7f6e2c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e47200640 1 -- 172.21.15.161:0/1300123350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e2c0047d0 msgr2=0x7f6e2c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:48.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e47200640 1 --2- 172.21.15.161:0/1300123350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e2c0047d0 0x7f6e2c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e47200640 1 -- 172.21.15.161:0/1300123350 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c003530 con 0x7f6e2c005190 2024-05-22T21:14:48.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.835+0000 7f6e3de00640 1 --2- 172.21.15.161:0/1300123350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e2c0047d0 0x7f6e2c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:48.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.839+0000 7f6e47200640 1 -- 172.21.15.161:0/1300123350 reap_dead start 2024-05-22T21:14:48.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.839+0000 7f6e47200640 1 --2- 172.21.15.161:0/1300123350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e2c005190 0x7f6e2c00b990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6e4806fe90 tx=0x7f6e4001f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be0fff59206f87d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:48.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.839+0000 7f6e45e00640 1 -- 172.21.15.161:0/1300123350 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40005ae0 con 0x7f6e2c005190 2024-05-22T21:14:48.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.839+0000 7f6e45e00640 1 -- 172.21.15.161:0/1300123350 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e400ab070 con 0x7f6e2c005190 2024-05-22T21:14:48.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.839+0000 7f6e45e00640 1 -- 172.21.15.161:0/1300123350 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40005e80 con 0x7f6e2c005190 2024-05-22T21:14:48.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.839+0000 7f6e4ca00640 1 -- 172.21.15.161:0/1300123350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e2c005190 msgr2=0x7f6e2c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:48.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.839+0000 7f6e4ca00640 1 --2- 172.21.15.161:0/1300123350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e2c005190 0x7f6e2c00b990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6e4806fe90 tx=0x7f6e4001f370 comp rx=0 tx=0).stop 2024-05-22T21:14:48.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.839+0000 7f6e4ca00640 1 -- 172.21.15.161:0/1300123350 shutdown_connections 2024-05-22T21:14:48.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.839+0000 7f6e4ca00640 1 --2- 172.21.15.161:0/1300123350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e2c005190 0x7f6e2c00b990 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.839+0000 7f6e4ca00640 1 -- 172.21.15.161:0/1300123350 >> 172.21.15.161:0/1300123350 conn(0x7f6e480cec00 msgr2=0x7f6e480cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:48.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.839+0000 7f6e4ca00640 1 -- 172.21.15.161:0/1300123350 shutdown_connections 2024-05-22T21:14:48.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.839+0000 7f6e4ca00640 1 -- 172.21.15.161:0/1300123350 wait complete. 2024-05-22T21:14:48.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.839+0000 7f6e4ca00640 1 Processor -- start 2024-05-22T21:14:48.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.839+0000 7f6e4ca00640 1 -- start start 2024-05-22T21:14:48.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e4ca00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e2c005190 0x7f6e481549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e4ca00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e480c4470 0x7f6e480acc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e4ca00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e480cefe0 0x7f6e480cf460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e4ca00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e480cf9a0 con 0x7f6e480cefe0 2024-05-22T21:14:48.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e4ca00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e480ad510 con 0x7f6e480c4470 2024-05-22T21:14:48.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e4ca00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e480ad7f0 con 0x7f6e2c005190 2024-05-22T21:14:48.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e47200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e480cefe0 0x7f6e480cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e47200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e480cefe0 0x7f6e480cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55942/0 (socket says 172.21.15.161:55942) 2024-05-22T21:14:48.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e47200640 1 -- 172.21.15.161:0/2703306774 learned_addr learned my addr 172.21.15.161:0/2703306774 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:48.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e3de00640 1 --2- 172.21.15.161:0/2703306774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e480c4470 0x7f6e480acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e46800640 1 --2- 172.21.15.161:0/2703306774 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e2c005190 0x7f6e481549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e47200640 1 -- 172.21.15.161:0/2703306774 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e2c005190 msgr2=0x7f6e481549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:48.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e47200640 1 --2- 172.21.15.161:0/2703306774 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e2c005190 0x7f6e481549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e47200640 1 -- 172.21.15.161:0/2703306774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e480c4470 msgr2=0x7f6e480acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:48.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e47200640 1 --2- 172.21.15.161:0/2703306774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e480c4470 0x7f6e480acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:48.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e47200640 1 -- 172.21.15.161:0/2703306774 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e480adad0 con 0x7f6e480cefe0 2024-05-22T21:14:48.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e3de00640 1 --2- 172.21.15.161:0/2703306774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e480c4470 0x7f6e480acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:48.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e47200640 1 --2- 172.21.15.161:0/2703306774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e480cefe0 0x7f6e480cf460 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6e4001f1a0 tx=0x7f6e2c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:48.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e3f400640 1 -- 172.21.15.161:0/2703306774 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40001640 con 0x7f6e480cefe0 2024-05-22T21:14:48.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e3f400640 1 -- 172.21.15.161:0/2703306774 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e400b6040 con 0x7f6e480cefe0 2024-05-22T21:14:48.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e4ca00640 1 -- 172.21.15.161:0/2703306774 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e480addb0 con 0x7f6e480cefe0 2024-05-22T21:14:48.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e3f400640 1 -- 172.21.15.161:0/2703306774 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e400049f0 con 0x7f6e480cefe0 2024-05-22T21:14:48.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e4ca00640 1 -- 172.21.15.161:0/2703306774 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e480ad150 con 0x7f6e480cefe0 2024-05-22T21:14:48.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e4ca00640 1 -- 172.21.15.161:0/2703306774 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e48067070 con 0x7f6e480cefe0 2024-05-22T21:14:48.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.843+0000 7f6e3f400640 1 -- 172.21.15.161:0/2703306774 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6e400ad030 con 0x7f6e480cefe0 2024-05-22T21:14:48.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.847+0000 7f6e3f400640 1 --2- 172.21.15.161:0/2703306774 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e24066690 0x7f6e24068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:48.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.847+0000 7f6e3f400640 1 -- 172.21.15.161:0/2703306774 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6e401208c0 con 0x7f6e480cefe0 2024-05-22T21:14:48.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.847+0000 7f6e46800640 1 --2- 172.21.15.161:0/2703306774 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e24066690 0x7f6e24068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:48.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.847+0000 7f6e46800640 1 --2- 172.21.15.161:0/2703306774 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e24066690 0x7f6e24068b50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6e38003f40 tx=0x7f6e38009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:48.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:48.851+0000 7f6e3f400640 1 -- 172.21.15.161:0/2703306774 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6e400edd70 con 0x7f6e480cefe0 2024-05-22T21:14:49.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.007+0000 7f6e4ca00640 1 -- 172.21.15.161:0/2703306774 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6e480ae260 con 0x7f6e480cefe0 2024-05-22T21:14:49.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.671+0000 7f6e3f400640 1 -- 172.21.15.161:0/2703306774 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6e400c0070 con 0x7f6e480cefe0 2024-05-22T21:14:49.678 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' created 2024-05-22T21:14:49.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.679+0000 7f6e4ca00640 1 -- 172.21.15.161:0/2703306774 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e24066690 msgr2=0x7f6e24068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:49.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.679+0000 7f6e4ca00640 1 --2- 172.21.15.161:0/2703306774 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e24066690 0x7f6e24068b50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6e38003f40 tx=0x7f6e38009040 comp rx=0 tx=0).stop 2024-05-22T21:14:49.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.679+0000 7f6e4ca00640 1 -- 172.21.15.161:0/2703306774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e480cefe0 msgr2=0x7f6e480cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:49.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.679+0000 7f6e4ca00640 1 --2- 172.21.15.161:0/2703306774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e480cefe0 0x7f6e480cf460 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6e4001f1a0 tx=0x7f6e2c005800 comp rx=0 tx=0).stop 2024-05-22T21:14:49.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.679+0000 7f6e4ca00640 1 -- 172.21.15.161:0/2703306774 shutdown_connections 2024-05-22T21:14:49.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.679+0000 7f6e4ca00640 1 --2- 172.21.15.161:0/2703306774 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e24066690 0x7f6e24068b50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:49.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.679+0000 7f6e4ca00640 1 --2- 172.21.15.161:0/2703306774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e480cefe0 0x7f6e480cf460 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:49.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.679+0000 7f6e4ca00640 1 --2- 172.21.15.161:0/2703306774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e480c4470 0x7f6e480acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:49.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.679+0000 7f6e4ca00640 1 --2- 172.21.15.161:0/2703306774 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e2c005190 0x7f6e481549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:49.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.679+0000 7f6e4ca00640 1 -- 172.21.15.161:0/2703306774 >> 172.21.15.161:0/2703306774 conn(0x7f6e480cec00 msgr2=0x7f6e4806a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:49.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.679+0000 7f6e4ca00640 1 -- 172.21.15.161:0/2703306774 shutdown_connections 2024-05-22T21:14:49.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.683+0000 7f6e4ca00640 1 -- 172.21.15.161:0/2703306774 wait complete. 2024-05-22T21:14:49.710 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-22T21:14:49.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.843+0000 7fce20e00640 1 Processor -- start 2024-05-22T21:14:49.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.843+0000 7fce20e00640 1 -- start start 2024-05-22T21:14:49.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.843+0000 7fce20e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fce1c13cfc0 0x7fce1c1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:49.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.847+0000 7fce20e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fce1c13fe50 0x7fce1c06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:49.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.847+0000 7fce20e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce1c13d3a0 con 0x7fce1c15af60 2024-05-22T21:14:49.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.847+0000 7fce20e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce1c0cd920 con 0x7fce1c13fe50 2024-05-22T21:14:49.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.847+0000 7fce20e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c0cdc00 con 0x7fce1c13cfc0 2024-05-22T21:14:49.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.847+0000 7fce1a800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fce1c13fe50 0x7fce1c06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:49.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.847+0000 7fce11400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fce1c15af60 0x7fce1c13f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:57694/0 (socket says 172.21.15.161:57694) 2024-05-22T21:14:49.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.847+0000 7fce11400640 1 -- 172.21.15.161:0/3909877739 learned_addr learned my addr 172.21.15.161:0/3909877739 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:49.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.847+0000 7fce19e00640 1 --2- 172.21.15.161:0/3909877739 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fce1c13cfc0 0x7fce1c1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:49.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.847+0000 7fce1a800640 1 -- 172.21.15.161:0/3909877739 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fce1c13cfc0 msgr2=0x7fce1c1402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:49.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.847+0000 7fce1a800640 1 --2- 172.21.15.161:0/3909877739 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fce1c13cfc0 0x7fce1c1402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:49.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.847+0000 7fce1a800640 1 -- 172.21.15.161:0/3909877739 >> v1:172.21.15.161:6789/0 conn(0x7fce1c15af60 legacy=0x7fce1c13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:49.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.847+0000 7fce1a800640 1 -- 172.21.15.161:0/3909877739 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c0cdee0 con 0x7fce1c13fe50 2024-05-22T21:14:49.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.847+0000 7fce1a800640 1 --2- 172.21.15.161:0/3909877739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce1c13fe50 0x7fce1c06fce0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fce14009d90 tx=0x7fce1401c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45cfcc1183b8f2b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:49.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce19400640 1 -- 172.21.15.161:0/3909877739 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce140ac620 con 0x7fce1c13fe50 2024-05-22T21:14:49.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce19400640 1 -- 172.21.15.161:0/3909877739 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce140ac780 con 0x7fce1c13fe50 2024-05-22T21:14:49.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce19400640 1 -- 172.21.15.161:0/3909877739 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce140acb10 con 0x7fce1c13fe50 2024-05-22T21:14:49.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce20e00640 1 -- 172.21.15.161:0/3909877739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce1c13fe50 msgr2=0x7fce1c06fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:49.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce20e00640 1 --2- 172.21.15.161:0/3909877739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce1c13fe50 0x7fce1c06fce0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fce14009d90 tx=0x7fce1401c740 comp rx=0 tx=0).stop 2024-05-22T21:14:49.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce20e00640 1 -- 172.21.15.161:0/3909877739 shutdown_connections 2024-05-22T21:14:49.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce20e00640 1 --2- 172.21.15.161:0/3909877739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce1c13fe50 0x7fce1c06fce0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:49.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce20e00640 1 --2- 172.21.15.161:0/3909877739 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fce1c13cfc0 0x7fce1c1402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:49.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce20e00640 1 -- 172.21.15.161:0/3909877739 >> 172.21.15.161:0/3909877739 conn(0x7fce1c06a2d0 msgr2=0x7fce1c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:49.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce20e00640 1 -- 172.21.15.161:0/3909877739 shutdown_connections 2024-05-22T21:14:49.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce20e00640 1 -- 172.21.15.161:0/3909877739 wait complete. 2024-05-22T21:14:49.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce20e00640 1 Processor -- start 2024-05-22T21:14:49.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce20e00640 1 -- start start 2024-05-22T21:14:49.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce20e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce1c13cfc0 0x7fce1c156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:49.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce20e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce1c13fe50 0x7fce1c1375b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:49.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce20e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fce1c15af60 0x7fce1c0b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:49.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce20e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c0b2650 con 0x7fce1c13fe50 2024-05-22T21:14:49.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce20e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c0b2320 con 0x7fce1c13cfc0 2024-05-22T21:14:49.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce20e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c135660 con 0x7fce1c15af60 2024-05-22T21:14:49.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce11400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce1c13fe50 0x7fce1c1375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:49.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce11400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce1c13fe50 0x7fce1c1375b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55974/0 (socket says 172.21.15.161:55974) 2024-05-22T21:14:49.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce11400640 1 -- 172.21.15.161:0/877514413 learned_addr learned my addr 172.21.15.161:0/877514413 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:49.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce19e00640 1 --2- 172.21.15.161:0/877514413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce1c13cfc0 0x7fce1c156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:49.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.851+0000 7fce1a800640 1 --2- 172.21.15.161:0/877514413 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fce1c15af60 0x7fce1c0b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:49.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.855+0000 7fce11400640 1 -- 172.21.15.161:0/877514413 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fce1c15af60 msgr2=0x7fce1c0b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:49.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.855+0000 7fce11400640 1 --2- 172.21.15.161:0/877514413 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fce1c15af60 0x7fce1c0b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:49.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.855+0000 7fce11400640 1 -- 172.21.15.161:0/877514413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce1c13cfc0 msgr2=0x7fce1c156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:49.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.855+0000 7fce11400640 1 --2- 172.21.15.161:0/877514413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce1c13cfc0 0x7fce1c156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:49.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.855+0000 7fce11400640 1 -- 172.21.15.161:0/877514413 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce14009a70 con 0x7fce1c13fe50 2024-05-22T21:14:49.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.855+0000 7fce11400640 1 --2- 172.21.15.161:0/877514413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce1c13fe50 0x7fce1c1375b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fce0400c780 tx=0x7fce0400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:49.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.855+0000 7fce12a00640 1 -- 172.21.15.161:0/877514413 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce040072d0 con 0x7fce1c13fe50 2024-05-22T21:14:49.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.855+0000 7fce12a00640 1 -- 172.21.15.161:0/877514413 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce04007430 con 0x7fce1c13fe50 2024-05-22T21:14:49.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.855+0000 7fce12a00640 1 -- 172.21.15.161:0/877514413 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce040047f0 con 0x7fce1c13fe50 2024-05-22T21:14:49.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.855+0000 7fce20e00640 1 -- 172.21.15.161:0/877514413 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1c1350a0 con 0x7fce1c13fe50 2024-05-22T21:14:49.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.855+0000 7fce20e00640 1 -- 172.21.15.161:0/877514413 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1c156c70 con 0x7fce1c13fe50 2024-05-22T21:14:49.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.855+0000 7fce20e00640 1 -- 172.21.15.161:0/877514413 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1c0b2180 con 0x7fce1c13fe50 2024-05-22T21:14:49.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.855+0000 7fce12a00640 1 -- 172.21.15.161:0/877514413 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fce0400a640 con 0x7fce1c13fe50 2024-05-22T21:14:49.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.855+0000 7fce12a00640 1 --2- 172.21.15.161:0/877514413 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcdf8066690 0x7fcdf8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:49.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.859+0000 7fce12a00640 1 -- 172.21.15.161:0/877514413 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fce04083610 con 0x7fce1c13fe50 2024-05-22T21:14:49.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.859+0000 7fce19e00640 1 --2- 172.21.15.161:0/877514413 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcdf8066690 0x7fcdf8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:49.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.859+0000 7fce19e00640 1 --2- 172.21.15.161:0/877514413 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcdf8066690 0x7fcdf8068b50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fce0c009b60 tx=0x7fce0c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:49.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:49.859+0000 7fce12a00640 1 -- 172.21.15.161:0/877514413 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fce04050b70 con 0x7fce1c13fe50 2024-05-22T21:14:50.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.047+0000 7fce20e00640 1 -- 172.21.15.161:0/877514413 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fce1c0b2180 con 0x7fce1c13fe50 2024-05-22T21:14:50.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.055+0000 7fce12a00640 1 -- 172.21.15.161:0/877514413 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7fce1c0b2180 con 0x7fce1c13fe50 2024-05-22T21:14:50.060 INFO:teuthology.orchestra.run.smithi161.stderr:new fs with metadata pool 6 and data pool 7 2024-05-22T21:14:50.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.059+0000 7fce20e00640 1 -- 172.21.15.161:0/877514413 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcdf8066690 msgr2=0x7fcdf8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.059+0000 7fce20e00640 1 --2- 172.21.15.161:0/877514413 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcdf8066690 0x7fcdf8068b50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fce0c009b60 tx=0x7fce0c009190 comp rx=0 tx=0).stop 2024-05-22T21:14:50.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.059+0000 7fce20e00640 1 -- 172.21.15.161:0/877514413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce1c13fe50 msgr2=0x7fce1c1375b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.059+0000 7fce20e00640 1 --2- 172.21.15.161:0/877514413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce1c13fe50 0x7fce1c1375b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fce0400c780 tx=0x7fce0400cc40 comp rx=0 tx=0).stop 2024-05-22T21:14:50.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.067+0000 7fce20e00640 1 -- 172.21.15.161:0/877514413 shutdown_connections 2024-05-22T21:14:50.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.067+0000 7fce20e00640 1 --2- 172.21.15.161:0/877514413 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcdf8066690 0x7fcdf8068b50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.067+0000 7fce20e00640 1 --2- 172.21.15.161:0/877514413 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fce1c15af60 0x7fce1c0b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.067+0000 7fce20e00640 1 --2- 172.21.15.161:0/877514413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce1c13fe50 0x7fce1c1375b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.067+0000 7fce20e00640 1 --2- 172.21.15.161:0/877514413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce1c13cfc0 0x7fce1c156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.067+0000 7fce20e00640 1 -- 172.21.15.161:0/877514413 >> 172.21.15.161:0/877514413 conn(0x7fce1c06a2d0 msgr2=0x7fce1c06a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:50.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.071+0000 7fce20e00640 1 -- 172.21.15.161:0/877514413 shutdown_connections 2024-05-22T21:14:50.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.071+0000 7fce20e00640 1 -- 172.21.15.161:0/877514413 wait complete. 2024-05-22T21:14:50.098 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:50.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.231+0000 7f4f18a00640 1 Processor -- start 2024-05-22T21:14:50.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.231+0000 7f4f18a00640 1 -- start start 2024-05-22T21:14:50.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.235+0000 7f4f18a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4f1415bd20 0x7f4f1415c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.235+0000 7f4f18a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4f140b3420 0x7f4f140cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.235+0000 7f4f18a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f140b3880 con 0x7f4f140cf3a0 2024-05-22T21:14:50.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.235+0000 7f4f18a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f140cf830 con 0x7f4f140b3420 2024-05-22T21:14:50.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.235+0000 7f4f18a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f140cc970 con 0x7f4f1415bd20 2024-05-22T21:14:50.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.235+0000 7f4f13e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f4f140cf3a0 0x7f4f1415ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:57702/0 (socket says 172.21.15.161:57702) 2024-05-22T21:14:50.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.235+0000 7f4f13e00640 1 -- 172.21.15.161:0/97290040 learned_addr learned my addr 172.21.15.161:0/97290040 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:50.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.235+0000 7f4f13400640 1 --2- 172.21.15.161:0/97290040 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4f1415bd20 0x7f4f1415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.235+0000 7f4f0aa00640 1 --2- 172.21.15.161:0/97290040 >> v2:172.21.15.179:3300/0 conn(0x7f4f140b3420 0x7f4f140cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.235+0000 7f4f12a00640 1 -- 172.21.15.161:0/97290040 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069365820 0 0) 0x7f4f0c0014a0 con 0x7f4f140cf3a0 2024-05-22T21:14:50.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.235+0000 7f4f12a00640 1 -- 172.21.15.161:0/97290040 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ef8003530 con 0x7f4f140cf3a0 2024-05-22T21:14:50.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.235+0000 7f4f13400640 1 -- 172.21.15.161:0/97290040 >> v2:172.21.15.179:3300/0 conn(0x7f4f140b3420 msgr2=0x7f4f140cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:50.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.235+0000 7f4f13400640 1 --2- 172.21.15.161:0/97290040 >> v2:172.21.15.179:3300/0 conn(0x7f4f140b3420 0x7f4f140cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.235+0000 7f4f13400640 1 -- 172.21.15.161:0/97290040 >> v1:172.21.15.161:6789/0 conn(0x7f4f140cf3a0 legacy=0x7f4f1415ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.235+0000 7f4f13400640 1 -- 172.21.15.161:0/97290040 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f140cfcf0 con 0x7f4f1415bd20 2024-05-22T21:14:50.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.235+0000 7f4f0aa00640 1 --2- 172.21.15.161:0/97290040 >> v2:172.21.15.179:3300/0 conn(0x7f4f140b3420 0x7f4f140cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:50.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.239+0000 7f4f13400640 1 --2- 172.21.15.161:0/97290040 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4f1415bd20 0x7f4f1415c100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4f040097c0 tx=0x7f4f0401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=852298dab03a52c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:50.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.239+0000 7f4f12a00640 1 -- 172.21.15.161:0/97290040 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f040ac640 con 0x7f4f1415bd20 2024-05-22T21:14:50.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.239+0000 7f4f12a00640 1 -- 172.21.15.161:0/97290040 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f040ac7a0 con 0x7f4f1415bd20 2024-05-22T21:14:50.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.239+0000 7f4f12a00640 1 -- 172.21.15.161:0/97290040 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f040acb30 con 0x7f4f1415bd20 2024-05-22T21:14:50.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.239+0000 7f4f18a00640 1 -- 172.21.15.161:0/97290040 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4f1415bd20 msgr2=0x7f4f1415c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.239+0000 7f4f18a00640 1 --2- 172.21.15.161:0/97290040 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4f1415bd20 0x7f4f1415c100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4f040097c0 tx=0x7f4f0401c3e0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.239+0000 7f4f18a00640 1 -- 172.21.15.161:0/97290040 shutdown_connections 2024-05-22T21:14:50.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.239+0000 7f4f18a00640 1 --2- 172.21.15.161:0/97290040 >> v2:172.21.15.179:3300/0 conn(0x7f4f140b3420 0x7f4f140cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.239+0000 7f4f18a00640 1 --2- 172.21.15.161:0/97290040 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4f1415bd20 0x7f4f1415c100 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.239+0000 7f4f18a00640 1 -- 172.21.15.161:0/97290040 >> 172.21.15.161:0/97290040 conn(0x7f4f140b2930 msgr2=0x7f4f140b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:50.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.239+0000 7f4f18a00640 1 -- 172.21.15.161:0/97290040 shutdown_connections 2024-05-22T21:14:50.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.239+0000 7f4f18a00640 1 -- 172.21.15.161:0/97290040 wait complete. 2024-05-22T21:14:50.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.239+0000 7f4f18a00640 1 Processor -- start 2024-05-22T21:14:50.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.239+0000 7f4f18a00640 1 -- start start 2024-05-22T21:14:50.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f18a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4f140b3420 0x7f4f14150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f18a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f140cf3a0 0x7f4f1414c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f18a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f1415bd20 0x7f4f140ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f18a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f140adc10 con 0x7f4f1415bd20 2024-05-22T21:14:50.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f18a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f14150590 con 0x7f4f140cf3a0 2024-05-22T21:14:50.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f18a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f14150860 con 0x7f4f140b3420 2024-05-22T21:14:50.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f13e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f1415bd20 0x7f4f140ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f13e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f1415bd20 0x7f4f140ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55988/0 (socket says 172.21.15.161:55988) 2024-05-22T21:14:50.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f13e00640 1 -- 172.21.15.161:0/375129439 learned_addr learned my addr 172.21.15.161:0/375129439 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:50.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f0aa00640 1 --2- 172.21.15.161:0/375129439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f140cf3a0 0x7f4f1414c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f13400640 1 --2- 172.21.15.161:0/375129439 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4f140b3420 0x7f4f14150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f13e00640 1 -- 172.21.15.161:0/375129439 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4f140b3420 msgr2=0x7f4f14150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f13e00640 1 --2- 172.21.15.161:0/375129439 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4f140b3420 0x7f4f14150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f13e00640 1 -- 172.21.15.161:0/375129439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f140cf3a0 msgr2=0x7f4f1414c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f13e00640 1 --2- 172.21.15.161:0/375129439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f140cf3a0 0x7f4f1414c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f13e00640 1 -- 172.21.15.161:0/375129439 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc009420 con 0x7f4f1415bd20 2024-05-22T21:14:50.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f13e00640 1 --2- 172.21.15.161:0/375129439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f1415bd20 0x7f4f140ae340 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00cae0 tx=0x7f4f0c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:50.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f0be00640 1 -- 172.21.15.161:0/375129439 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c00ac40 con 0x7f4f1415bd20 2024-05-22T21:14:50.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f0be00640 1 -- 172.21.15.161:0/375129439 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0c00ada0 con 0x7f4f1415bd20 2024-05-22T21:14:50.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f0be00640 1 -- 172.21.15.161:0/375129439 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c012620 con 0x7f4f1415bd20 2024-05-22T21:14:50.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f18a00640 1 -- 172.21.15.161:0/375129439 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f04009420 con 0x7f4f1415bd20 2024-05-22T21:14:50.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f18a00640 1 -- 172.21.15.161:0/375129439 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f14150e20 con 0x7f4f1415bd20 2024-05-22T21:14:50.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f18a00640 1 -- 172.21.15.161:0/375129439 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f140afe80 con 0x7f4f1415bd20 2024-05-22T21:14:50.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.243+0000 7f4f0be00640 1 -- 172.21.15.161:0/375129439 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4f0c007070 con 0x7f4f1415bd20 2024-05-22T21:14:50.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.247+0000 7f4f0be00640 1 --2- 172.21.15.161:0/375129439 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ef0066690 0x7f4ef0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.247+0000 7f4f0be00640 1 -- 172.21.15.161:0/375129439 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f0c083f70 con 0x7f4f1415bd20 2024-05-22T21:14:50.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.247+0000 7f4f13400640 1 --2- 172.21.15.161:0/375129439 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ef0066690 0x7f4ef0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.247+0000 7f4f13400640 1 --2- 172.21.15.161:0/375129439 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ef0066690 0x7f4ef0068b50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4f040ae000 tx=0x7f4f040a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:50.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.251+0000 7f4f0be00640 1 -- 172.21.15.161:0/375129439 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4f0c0512f0 con 0x7f4f1415bd20 2024-05-22T21:14:50.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.403+0000 7f4f18a00640 1 -- 172.21.15.161:0/375129439 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f140b3880 con 0x7f4f1415bd20 2024-05-22T21:14:50.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.407+0000 7f4f0be00640 1 -- 172.21.15.161:0/375129439 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16106 (secure 0 0 0) 0x7f4f140b3880 con 0x7f4f1415bd20 2024-05-22T21:14:50.410 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:50.410 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":46,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:14:50.051974+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:48.612726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:49.053022+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:50.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.407+0000 7f4f18a00640 1 -- 172.21.15.161:0/375129439 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ef0066690 msgr2=0x7f4ef0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.407+0000 7f4f18a00640 1 --2- 172.21.15.161:0/375129439 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ef0066690 0x7f4ef0068b50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4f040ae000 tx=0x7f4f040a7000 comp rx=0 tx=0).stop 2024-05-22T21:14:50.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.407+0000 7f4f18a00640 1 -- 172.21.15.161:0/375129439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f1415bd20 msgr2=0x7f4f140ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.407+0000 7f4f18a00640 1 --2- 172.21.15.161:0/375129439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f1415bd20 0x7f4f140ae340 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00cae0 tx=0x7f4f0c00a060 comp rx=0 tx=0).stop 2024-05-22T21:14:50.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.407+0000 7f4f18a00640 1 -- 172.21.15.161:0/375129439 shutdown_connections 2024-05-22T21:14:50.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.407+0000 7f4f18a00640 1 --2- 172.21.15.161:0/375129439 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ef0066690 0x7f4ef0068b50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.407+0000 7f4f18a00640 1 --2- 172.21.15.161:0/375129439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f1415bd20 0x7f4f140ae340 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.407+0000 7f4f18a00640 1 --2- 172.21.15.161:0/375129439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f140cf3a0 0x7f4f1414c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.407+0000 7f4f18a00640 1 --2- 172.21.15.161:0/375129439 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4f140b3420 0x7f4f14150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.407+0000 7f4f18a00640 1 -- 172.21.15.161:0/375129439 >> 172.21.15.161:0/375129439 conn(0x7f4f140b2930 msgr2=0x7f4f140b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:50.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.407+0000 7f4f18a00640 1 -- 172.21.15.161:0/375129439 shutdown_connections 2024-05-22T21:14:50.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.407+0000 7f4f18a00640 1 -- 172.21.15.161:0/375129439 wait complete. 2024-05-22T21:14:50.431 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:50.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.563+0000 7f215c000640 1 Processor -- start 2024-05-22T21:14:50.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.563+0000 7f215c000640 1 -- start start 2024-05-22T21:14:50.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.563+0000 7f215c000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f21540715a0 0x7f215415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.563+0000 7f215c000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f215415ec90 0x7f215415f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.563+0000 7f215c000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21541442a0 con 0x7f2154143ec0 2024-05-22T21:14:50.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.563+0000 7f215c000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f215415f660 con 0x7f215415ec90 2024-05-22T21:14:50.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.563+0000 7f215c000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2154070500 con 0x7f21540715a0 2024-05-22T21:14:50.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.563+0000 7f2159600640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f2154143ec0 0x7f215406fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:57706/0 (socket says 172.21.15.161:57706) 2024-05-22T21:14:50.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.563+0000 7f2159600640 1 -- 172.21.15.161:0/1530872037 learned_addr learned my addr 172.21.15.161:0/1530872037 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:50.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f215a000640 1 --2- 172.21.15.161:0/1530872037 >> v2:172.21.15.179:3300/0 conn(0x7f215415ec90 0x7f215415f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2158c00640 1 --2- 172.21.15.161:0/1530872037 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f21540715a0 0x7f215415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2153e00640 1 -- 172.21.15.161:0/1530872037 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577923645 0 0) 0x7f21541442a0 con 0x7f2154143ec0 2024-05-22T21:14:50.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2153e00640 1 -- 172.21.15.161:0/1530872037 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f213c003530 con 0x7f2154143ec0 2024-05-22T21:14:50.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2153e00640 1 -- 172.21.15.161:0/1530872037 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1352748146 0 0) 0x7f213c003530 con 0x7f2154143ec0 2024-05-22T21:14:50.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2153e00640 1 -- 172.21.15.161:0/1530872037 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f21540715a0 msgr2=0x7f215415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:50.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2153e00640 1 --2- 172.21.15.161:0/1530872037 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f21540715a0 0x7f215415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2153e00640 1 -- 172.21.15.161:0/1530872037 >> v2:172.21.15.179:3300/0 conn(0x7f215415ec90 msgr2=0x7f215415f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:50.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2153e00640 1 --2- 172.21.15.161:0/1530872037 >> v2:172.21.15.179:3300/0 conn(0x7f215415ec90 0x7f215415f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2153e00640 1 -- 172.21.15.161:0/1530872037 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21540707e0 con 0x7f2154143ec0 2024-05-22T21:14:50.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2153e00640 1 -- 172.21.15.161:0/1530872037 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f2148001680 con 0x7f2154143ec0 2024-05-22T21:14:50.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f215a000640 1 --2- 172.21.15.161:0/1530872037 >> v2:172.21.15.179:3300/0 conn(0x7f215415ec90 0x7f215415f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:50.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2153e00640 1 -- 172.21.15.161:0/1530872037 >> v1:172.21.15.161:6789/0 conn(0x7f2154143ec0 legacy=0x7f215406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2153e00640 1 --2- 172.21.15.161:0/1530872037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f213c003980 0x7f213c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2153e00640 1 --2- 172.21.15.161:0/1530872037 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f213c0047d0 0x7f213c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2153e00640 1 --2- 172.21.15.161:0/1530872037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c005150 0x7f213c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2153e00640 1 -- 172.21.15.161:0/1530872037 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215415f660 con 0x7f213c005150 2024-05-22T21:14:50.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2153e00640 1 -- 172.21.15.161:0/1530872037 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2154070500 con 0x7f213c003980 2024-05-22T21:14:50.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2153e00640 1 -- 172.21.15.161:0/1530872037 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213c0045d0 con 0x7f213c0047d0 2024-05-22T21:14:50.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f215a000640 1 --2- 172.21.15.161:0/1530872037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c005150 0x7f213c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2159600640 1 --2- 172.21.15.161:0/1530872037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f213c003980 0x7f213c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f2158c00640 1 --2- 172.21.15.161:0/1530872037 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f213c0047d0 0x7f213c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f215a000640 1 -- 172.21.15.161:0/1530872037 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f213c0047d0 msgr2=0x7f213c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:50.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f215a000640 1 --2- 172.21.15.161:0/1530872037 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f213c0047d0 0x7f213c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f215a000640 1 -- 172.21.15.161:0/1530872037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f213c003980 msgr2=0x7f213c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:50.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f215a000640 1 --2- 172.21.15.161:0/1530872037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f213c003980 0x7f213c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.567+0000 7f215a000640 1 -- 172.21.15.161:0/1530872037 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c003530 con 0x7f213c005150 2024-05-22T21:14:50.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.571+0000 7f215a000640 1 --2- 172.21.15.161:0/1530872037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c005150 0x7f213c00b990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f214c00e350 tx=0x7f214c01ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ead3ffa25d21f71 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:50.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.571+0000 7f2153e00640 1 -- 172.21.15.161:0/1530872037 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c0a5e90 con 0x7f213c005150 2024-05-22T21:14:50.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.571+0000 7f2153e00640 1 -- 172.21.15.161:0/1530872037 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c00f040 con 0x7f213c005150 2024-05-22T21:14:50.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.571+0000 7f2153e00640 1 -- 172.21.15.161:0/1530872037 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c0ad640 con 0x7f213c005150 2024-05-22T21:14:50.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.571+0000 7f215c000640 1 -- 172.21.15.161:0/1530872037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c005150 msgr2=0x7f213c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.571+0000 7f215c000640 1 --2- 172.21.15.161:0/1530872037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f213c005150 0x7f213c00b990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f214c00e350 tx=0x7f214c01ce80 comp rx=0 tx=0).stop 2024-05-22T21:14:50.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.571+0000 7f215a000640 1 -- 172.21.15.161:0/1530872037 reap_dead start 2024-05-22T21:14:50.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.571+0000 7f215c000640 1 -- 172.21.15.161:0/1530872037 shutdown_connections 2024-05-22T21:14:50.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.571+0000 7f215c000640 1 -- 172.21.15.161:0/1530872037 >> 172.21.15.161:0/1530872037 conn(0x7f21540b0bc0 msgr2=0x7f21540b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:50.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.571+0000 7f215c000640 1 -- 172.21.15.161:0/1530872037 shutdown_connections 2024-05-22T21:14:50.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.571+0000 7f215c000640 1 -- 172.21.15.161:0/1530872037 wait complete. 2024-05-22T21:14:50.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.571+0000 7f215c000640 1 Processor -- start 2024-05-22T21:14:50.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.571+0000 7f215c000640 1 -- start start 2024-05-22T21:14:50.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215c000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f213c005150 0x7f2154159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215c000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f215415c760 0x7f21540aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215c000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215415cbe0 0x7f21540ab490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215c000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21540ab9d0 con 0x7f215415cbe0 2024-05-22T21:14:50.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215c000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215415a3a0 con 0x7f213c005150 2024-05-22T21:14:50.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215c000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215415ddc0 con 0x7f215415c760 2024-05-22T21:14:50.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215415cbe0 0x7f21540ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215415cbe0 0x7f21540ab490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55998/0 (socket says 172.21.15.161:55998) 2024-05-22T21:14:50.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215a000640 1 -- 172.21.15.161:0/3199612900 learned_addr learned my addr 172.21.15.161:0/3199612900 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:50.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f2158c00640 1 --2- 172.21.15.161:0/3199612900 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f215415c760 0x7f21540aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f2159600640 1 --2- 172.21.15.161:0/3199612900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f213c005150 0x7f2154159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215a000640 1 -- 172.21.15.161:0/3199612900 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f215415c760 msgr2=0x7f21540aaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215a000640 1 --2- 172.21.15.161:0/3199612900 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f215415c760 0x7f21540aaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215a000640 1 -- 172.21.15.161:0/3199612900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f213c005150 msgr2=0x7f2154159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215a000640 1 --2- 172.21.15.161:0/3199612900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f213c005150 0x7f2154159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215a000640 1 -- 172.21.15.161:0/3199612900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215415d1b0 con 0x7f215415cbe0 2024-05-22T21:14:50.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215a000640 1 --2- 172.21.15.161:0/3199612900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215415cbe0 0x7f21540ab490 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f214c0b0040 tx=0x7f214c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:50.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f2151600640 1 -- 172.21.15.161:0/3199612900 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c0aee10 con 0x7f215415cbe0 2024-05-22T21:14:50.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f2151600640 1 -- 172.21.15.161:0/3199612900 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c0039a0 con 0x7f215415cbe0 2024-05-22T21:14:50.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f2151600640 1 -- 172.21.15.161:0/3199612900 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c003d50 con 0x7f215415cbe0 2024-05-22T21:14:50.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215c000640 1 -- 172.21.15.161:0/3199612900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215415d490 con 0x7f215415cbe0 2024-05-22T21:14:50.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215c000640 1 -- 172.21.15.161:0/3199612900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f215415da50 con 0x7f215415cbe0 2024-05-22T21:14:50.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f215c000640 1 -- 172.21.15.161:0/3199612900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2154067220 con 0x7f215415cbe0 2024-05-22T21:14:50.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.575+0000 7f2151600640 1 -- 172.21.15.161:0/3199612900 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f214c0b4040 con 0x7f215415cbe0 2024-05-22T21:14:50.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.579+0000 7f2151600640 1 --2- 172.21.15.161:0/3199612900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2134066690 0x7f2134068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.579+0000 7f2151600640 1 -- 172.21.15.161:0/3199612900 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f214c11e840 con 0x7f215415cbe0 2024-05-22T21:14:50.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.579+0000 7f2159600640 1 --2- 172.21.15.161:0/3199612900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2134066690 0x7f2134068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.579+0000 7f2159600640 1 --2- 172.21.15.161:0/3199612900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2134066690 0x7f2134068b50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2148000c10 tx=0x7f2148009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:50.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.583+0000 7f2151600640 1 -- 172.21.15.161:0/3199612900 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f214c0ebb10 con 0x7f215415cbe0 2024-05-22T21:14:50.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.739+0000 7f215c000640 1 -- 172.21.15.161:0/3199612900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f215415df00 con 0x7f215415cbe0 2024-05-22T21:14:50.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.739+0000 7f2151600640 1 -- 172.21.15.161:0/3199612900 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16106 (secure 0 0 0) 0x7f214c009de0 con 0x7f215415cbe0 2024-05-22T21:14:50.743 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:50.743 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":46,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:14:50.051974+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:48.612726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:49.053022+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:50.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.739+0000 7f215c000640 1 -- 172.21.15.161:0/3199612900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2134066690 msgr2=0x7f2134068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.739+0000 7f215c000640 1 --2- 172.21.15.161:0/3199612900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2134066690 0x7f2134068b50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2148000c10 tx=0x7f2148009040 comp rx=0 tx=0).stop 2024-05-22T21:14:50.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.739+0000 7f215c000640 1 -- 172.21.15.161:0/3199612900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215415cbe0 msgr2=0x7f21540ab490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.739+0000 7f215c000640 1 --2- 172.21.15.161:0/3199612900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215415cbe0 0x7f21540ab490 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f214c0b0040 tx=0x7f214c009750 comp rx=0 tx=0).stop 2024-05-22T21:14:50.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.739+0000 7f215c000640 1 -- 172.21.15.161:0/3199612900 shutdown_connections 2024-05-22T21:14:50.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.739+0000 7f215c000640 1 --2- 172.21.15.161:0/3199612900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2134066690 0x7f2134068b50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.739+0000 7f215c000640 1 --2- 172.21.15.161:0/3199612900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f215415cbe0 0x7f21540ab490 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.739+0000 7f215c000640 1 --2- 172.21.15.161:0/3199612900 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f215415c760 0x7f21540aaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.743+0000 7f215c000640 1 --2- 172.21.15.161:0/3199612900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f213c005150 0x7f2154159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.743+0000 7f215c000640 1 -- 172.21.15.161:0/3199612900 >> 172.21.15.161:0/3199612900 conn(0x7f21540b0bc0 msgr2=0x7f215413fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:50.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.743+0000 7f215c000640 1 -- 172.21.15.161:0/3199612900 shutdown_connections 2024-05-22T21:14:50.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.743+0000 7f215c000640 1 -- 172.21.15.161:0/3199612900 wait complete. 2024-05-22T21:14:50.765 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-22T21:14:50.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.899+0000 7fea25200640 1 Processor -- start 2024-05-22T21:14:50.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.899+0000 7fea25200640 1 -- start start 2024-05-22T21:14:50.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.899+0000 7fea25200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fea2013cfc0 0x7fea201402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.899+0000 7fea25200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2015af60 0x7fea2013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.899+0000 7fea25200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea201400d0 con 0x7fea2013fc80 2024-05-22T21:14:50.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.899+0000 7fea25200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2013d3a0 con 0x7fea2013cfc0 2024-05-22T21:14:50.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.899+0000 7fea25200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea200cd920 con 0x7fea2015af60 2024-05-22T21:14:50.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.899+0000 7fea1d400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2015af60 0x7fea2013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.899+0000 7fea1d400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2015af60 0x7fea2013f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:40670/0 (socket says 172.21.15.161:40670) 2024-05-22T21:14:50.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.899+0000 7fea1d400640 1 -- 172.21.15.161:0/1207037114 learned_addr learned my addr 172.21.15.161:0/1207037114 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:50.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.899+0000 7fea1de00640 1 --2- 172.21.15.161:0/1207037114 >> v2:172.21.15.179:3300/0 conn(0x7fea2013cfc0 0x7fea201402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.899+0000 7fea1d400640 1 -- 172.21.15.161:0/1207037114 >> v2:172.21.15.179:3300/0 conn(0x7fea2013cfc0 msgr2=0x7fea201402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:50.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.899+0000 7fea1d400640 1 --2- 172.21.15.161:0/1207037114 >> v2:172.21.15.179:3300/0 conn(0x7fea2013cfc0 0x7fea201402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.899+0000 7fea1d400640 1 -- 172.21.15.161:0/1207037114 >> v1:172.21.15.179:6789/0 conn(0x7fea2013fc80 legacy=0x7fea2015dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.899+0000 7fea1d400640 1 -- 172.21.15.161:0/1207037114 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea200cdc00 con 0x7fea2015af60 2024-05-22T21:14:50.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.903+0000 7fea1d400640 1 --2- 172.21.15.161:0/1207037114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2015af60 0x7fea2013f740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009870 tx=0x7fea0c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f05b3f2f2bb91944 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:50.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.903+0000 7fea1ca00640 1 -- 172.21.15.161:0/1207037114 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0ab600 con 0x7fea2015af60 2024-05-22T21:14:50.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.903+0000 7fea1ca00640 1 -- 172.21.15.161:0/1207037114 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c0a7050 con 0x7fea2015af60 2024-05-22T21:14:50.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.903+0000 7fea1ca00640 1 -- 172.21.15.161:0/1207037114 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0ab980 con 0x7fea2015af60 2024-05-22T21:14:50.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.903+0000 7fea25200640 1 -- 172.21.15.161:0/1207037114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2015af60 msgr2=0x7fea2013f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.903+0000 7fea25200640 1 --2- 172.21.15.161:0/1207037114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2015af60 0x7fea2013f740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009870 tx=0x7fea0c01c4d0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea25200640 1 -- 172.21.15.161:0/1207037114 shutdown_connections 2024-05-22T21:14:50.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea25200640 1 --2- 172.21.15.161:0/1207037114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2015af60 0x7fea2013f740 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea25200640 1 --2- 172.21.15.161:0/1207037114 >> v2:172.21.15.179:3300/0 conn(0x7fea2013cfc0 0x7fea201402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea25200640 1 -- 172.21.15.161:0/1207037114 >> 172.21.15.161:0/1207037114 conn(0x7fea2006a2d0 msgr2=0x7fea2006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:50.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea25200640 1 -- 172.21.15.161:0/1207037114 shutdown_connections 2024-05-22T21:14:50.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea25200640 1 -- 172.21.15.161:0/1207037114 wait complete. 2024-05-22T21:14:50.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea25200640 1 Processor -- start 2024-05-22T21:14:50.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea25200640 1 -- start start 2024-05-22T21:14:50.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea25200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea2013cfc0 0x7fea201577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea25200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2013fc80 0x7fea20134f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea25200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea2015af60 0x7fea20135480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea25200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea200af890 con 0x7fea2015af60 2024-05-22T21:14:50.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea25200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea200b08b0 con 0x7fea2013cfc0 2024-05-22T21:14:50.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea25200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea200af4f0 con 0x7fea2013fc80 2024-05-22T21:14:50.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea1e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea2015af60 0x7fea20135480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea1e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea2015af60 0x7fea20135480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:56010/0 (socket says 172.21.15.161:56010) 2024-05-22T21:14:50.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea1e800640 1 -- 172.21.15.161:0/4014247869 learned_addr learned my addr 172.21.15.161:0/4014247869 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:50.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea1d400640 1 --2- 172.21.15.161:0/4014247869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2013fc80 0x7fea20134f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea1de00640 1 --2- 172.21.15.161:0/4014247869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea2013cfc0 0x7fea201577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea1e800640 1 -- 172.21.15.161:0/4014247869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2013fc80 msgr2=0x7fea20134f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea1e800640 1 --2- 172.21.15.161:0/4014247869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2013fc80 0x7fea20134f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea1e800640 1 -- 172.21.15.161:0/4014247869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea2013cfc0 msgr2=0x7fea201577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:50.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea1e800640 1 --2- 172.21.15.161:0/4014247869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea2013cfc0 0x7fea201577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:50.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea1e800640 1 -- 172.21.15.161:0/4014247869 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c0094d0 con 0x7fea2015af60 2024-05-22T21:14:50.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea1e800640 1 --2- 172.21.15.161:0/4014247869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea2015af60 0x7fea20135480 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c770 tx=0x7fea1800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:50.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.907+0000 7fea16000640 1 -- 172.21.15.161:0/4014247869 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18003b80 con 0x7fea2015af60 2024-05-22T21:14:50.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.911+0000 7fea25200640 1 -- 172.21.15.161:0/4014247869 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2015b670 con 0x7fea2015af60 2024-05-22T21:14:50.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.911+0000 7fea16000640 1 -- 172.21.15.161:0/4014247869 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea18003ce0 con 0x7fea2015af60 2024-05-22T21:14:50.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.911+0000 7fea16000640 1 -- 172.21.15.161:0/4014247869 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1800aa90 con 0x7fea2015af60 2024-05-22T21:14:50.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.911+0000 7fea25200640 1 -- 172.21.15.161:0/4014247869 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea200b0560 con 0x7fea2015af60 2024-05-22T21:14:50.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.911+0000 7fea25200640 1 -- 172.21.15.161:0/4014247869 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea20067220 con 0x7fea2015af60 2024-05-22T21:14:50.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.911+0000 7fea16000640 1 -- 172.21.15.161:0/4014247869 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fea1800abf0 con 0x7fea2015af60 2024-05-22T21:14:50.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.911+0000 7fea16000640 1 --2- 172.21.15.161:0/4014247869 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea00066690 0x7fea00068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:50.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.911+0000 7fea16000640 1 -- 172.21.15.161:0/4014247869 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea18083d70 con 0x7fea2015af60 2024-05-22T21:14:50.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.911+0000 7fea1de00640 1 --2- 172.21.15.161:0/4014247869 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea00066690 0x7fea00068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:50.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.911+0000 7fea1de00640 1 --2- 172.21.15.161:0/4014247869 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea00066690 0x7fea00068b50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fea10009ac0 tx=0x7fea10009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:50.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:50.915+0000 7fea16000640 1 -- 172.21.15.161:0/4014247869 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fea18051100 con 0x7fea2015af60 2024-05-22T21:14:51.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:51.107+0000 7fea25200640 1 -- 172.21.15.161:0/4014247869 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fea201400d0 con 0x7fea2015af60 2024-05-22T21:14:51.577 INFO:tasks.ceph.mds.b.smithi179.stderr:2024-05-22T21:14:51.571+0000 7f1127400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T21:14:52.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.055+0000 7fea16000640 1 -- 172.21.15.161:0/4014247869 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7fea201400d0 con 0x7fea2015af60 2024-05-22T21:14:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.059+0000 7fea25200640 1 -- 172.21.15.161:0/4014247869 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea00066690 msgr2=0x7fea00068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.059+0000 7fea25200640 1 --2- 172.21.15.161:0/4014247869 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea00066690 0x7fea00068b50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fea10009ac0 tx=0x7fea10009190 comp rx=0 tx=0).stop 2024-05-22T21:14:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.059+0000 7fea25200640 1 -- 172.21.15.161:0/4014247869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea2015af60 msgr2=0x7fea20135480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.059+0000 7fea25200640 1 --2- 172.21.15.161:0/4014247869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea2015af60 0x7fea20135480 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c770 tx=0x7fea1800cc30 comp rx=0 tx=0).stop 2024-05-22T21:14:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.059+0000 7fea25200640 1 -- 172.21.15.161:0/4014247869 shutdown_connections 2024-05-22T21:14:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.059+0000 7fea25200640 1 --2- 172.21.15.161:0/4014247869 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea00066690 0x7fea00068b50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.059+0000 7fea25200640 1 --2- 172.21.15.161:0/4014247869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea2015af60 0x7fea20135480 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.059+0000 7fea25200640 1 --2- 172.21.15.161:0/4014247869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea2013fc80 0x7fea20134f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.059+0000 7fea25200640 1 --2- 172.21.15.161:0/4014247869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea2013cfc0 0x7fea201577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.059+0000 7fea25200640 1 -- 172.21.15.161:0/4014247869 >> 172.21.15.161:0/4014247869 conn(0x7fea2006a2d0 msgr2=0x7fea2006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:52.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.059+0000 7fea25200640 1 -- 172.21.15.161:0/4014247869 shutdown_connections 2024-05-22T21:14:52.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.059+0000 7fea25200640 1 -- 172.21.15.161:0/4014247869 wait complete. 2024-05-22T21:14:52.084 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-22T21:14:52.085 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-22T21:14:52.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf97600640 1 Processor -- start 2024-05-22T21:14:52.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf97600640 1 -- start start 2024-05-22T21:14:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf97600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf9006c2a0 0x7fcf9006c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf97600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf9015d9d0 con 0x7fcf901402b0 2024-05-22T21:14:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf97600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf9015dcb0 con 0x7fcf900b3370 2024-05-22T21:14:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf97600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9015d7c0 con 0x7fcf9006c2a0 2024-05-22T21:14:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf94c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fcf901402b0 0x7fcf90070590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:57716/0 (socket says 172.21.15.161:57716) 2024-05-22T21:14:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf94c00640 1 -- 172.21.15.161:0/2118202423 learned_addr learned my addr 172.21.15.161:0/2118202423 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8fe00640 1 --2- 172.21.15.161:0/2118202423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf9006c2a0 0x7fcf9006c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:52.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 -- 172.21.15.161:0/2118202423 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838932228 0 0) 0x7fcf9015d9d0 con 0x7fcf901402b0 2024-05-22T21:14:52.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 -- 172.21.15.161:0/2118202423 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf78003530 con 0x7fcf901402b0 2024-05-22T21:14:52.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 -- 172.21.15.161:0/2118202423 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 848178967 0 0) 0x7fcf9015dcb0 con 0x7fcf900b3370 2024-05-22T21:14:52.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 -- 172.21.15.161:0/2118202423 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf9015d9d0 con 0x7fcf900b3370 2024-05-22T21:14:52.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 -- 172.21.15.161:0/2118202423 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2083263747 0 0) 0x7fcf78003530 con 0x7fcf901402b0 2024-05-22T21:14:52.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 -- 172.21.15.161:0/2118202423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf9006c2a0 msgr2=0x7fcf9006c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:52.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 --2- 172.21.15.161:0/2118202423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf9006c2a0 0x7fcf9006c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:52.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 -- 172.21.15.161:0/2118202423 >> v1:172.21.15.179:6789/0 conn(0x7fcf900b3370 legacy=0x7fcf900b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:52.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 -- 172.21.15.161:0/2118202423 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9015df90 con 0x7fcf901402b0 2024-05-22T21:14:52.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 -- 172.21.15.161:0/2118202423 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fcf84001680 con 0x7fcf901402b0 2024-05-22T21:14:52.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 -- 172.21.15.161:0/2118202423 >> v1:172.21.15.161:6789/0 conn(0x7fcf901402b0 legacy=0x7fcf90070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:52.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 --2- 172.21.15.161:0/2118202423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf78003b40 0x7fcf78003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:52.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 --2- 172.21.15.161:0/2118202423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf780047d0 0x7fcf78004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:52.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 --2- 172.21.15.161:0/2118202423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf78005190 0x7fcf7800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:52.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 -- 172.21.15.161:0/2118202423 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9015d7c0 con 0x7fcf780047d0 2024-05-22T21:14:52.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 -- 172.21.15.161:0/2118202423 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7800bf80 con 0x7fcf78005190 2024-05-22T21:14:52.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8f400640 1 -- 172.21.15.161:0/2118202423 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7800c140 con 0x7fcf78003b40 2024-05-22T21:14:52.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8fe00640 1 --2- 172.21.15.161:0/2118202423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf9006c2a0 0x7fcf9006c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:52.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf95600640 1 --2- 172.21.15.161:0/2118202423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf78005190 0x7fcf7800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:52.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8fe00640 1 --2- 172.21.15.161:0/2118202423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf780047d0 0x7fcf78004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:52.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf94c00640 1 --2- 172.21.15.161:0/2118202423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf78003b40 0x7fcf78003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:52.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8fe00640 1 -- 172.21.15.161:0/2118202423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf78003b40 msgr2=0x7fcf78003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:52.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8fe00640 1 --2- 172.21.15.161:0/2118202423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf78003b40 0x7fcf78003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:52.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8fe00640 1 -- 172.21.15.161:0/2118202423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf78005190 msgr2=0x7fcf7800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:52.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8fe00640 1 --2- 172.21.15.161:0/2118202423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf78005190 0x7fcf7800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:52.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.223+0000 7fcf8fe00640 1 -- 172.21.15.161:0/2118202423 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf78003530 con 0x7fcf780047d0 2024-05-22T21:14:52.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.227+0000 7fcf8fe00640 1 --2- 172.21.15.161:0/2118202423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf780047d0 0x7fcf78004c50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009b20 tx=0x7fcf7c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6f06ce9e94f72a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:52.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.227+0000 7fcf8f400640 1 -- 172.21.15.161:0/2118202423 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c004030 con 0x7fcf780047d0 2024-05-22T21:14:52.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.227+0000 7fcf8f400640 1 -- 172.21.15.161:0/2118202423 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7c004190 con 0x7fcf780047d0 2024-05-22T21:14:52.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.227+0000 7fcf8f400640 1 -- 172.21.15.161:0/2118202423 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c004520 con 0x7fcf780047d0 2024-05-22T21:14:52.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.227+0000 7fcf97600640 1 -- 172.21.15.161:0/2118202423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf780047d0 msgr2=0x7fcf78004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:52.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.227+0000 7fcf97600640 1 --2- 172.21.15.161:0/2118202423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf780047d0 0x7fcf78004c50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009b20 tx=0x7fcf7c01cab0 comp rx=0 tx=0).stop 2024-05-22T21:14:52.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.227+0000 7fcf95600640 1 -- 172.21.15.161:0/2118202423 reap_dead start 2024-05-22T21:14:52.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.227+0000 7fcf97600640 1 -- 172.21.15.161:0/2118202423 shutdown_connections 2024-05-22T21:14:52.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.227+0000 7fcf97600640 1 -- 172.21.15.161:0/2118202423 >> 172.21.15.161:0/2118202423 conn(0x7fcf9006a2d0 msgr2=0x7fcf9006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:52.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.227+0000 7fcf97600640 1 -- 172.21.15.161:0/2118202423 shutdown_connections 2024-05-22T21:14:52.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.227+0000 7fcf97600640 1 -- 172.21.15.161:0/2118202423 wait complete. 2024-05-22T21:14:52.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf97600640 1 Processor -- start 2024-05-22T21:14:52.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf97600640 1 -- start start 2024-05-22T21:14:52.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf97600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf780047d0 0x7fcf90139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:52.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf97600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf900b15a0 0x7fcf90158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:52.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf97600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf9015a850 0x7fcf900af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:52.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf97600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf901597b0 con 0x7fcf900b15a0 2024-05-22T21:14:52.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf97600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf90159a90 con 0x7fcf9015a850 2024-05-22T21:14:52.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf97600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf90159d70 con 0x7fcf780047d0 2024-05-22T21:14:52.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf8fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf900b15a0 0x7fcf90158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:52.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf8fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf900b15a0 0x7fcf90158e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:56018/0 (socket says 172.21.15.161:56018) 2024-05-22T21:14:52.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf8fe00640 1 -- 172.21.15.161:0/82461427 learned_addr learned my addr 172.21.15.161:0/82461427 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:52.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf94c00640 1 --2- 172.21.15.161:0/82461427 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf780047d0 0x7fcf90139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:52.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf95600640 1 --2- 172.21.15.161:0/82461427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf9015a850 0x7fcf900af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:52.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf8fe00640 1 -- 172.21.15.161:0/82461427 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf780047d0 msgr2=0x7fcf90139920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:52.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf8fe00640 1 --2- 172.21.15.161:0/82461427 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf780047d0 0x7fcf90139920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:52.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf8fe00640 1 -- 172.21.15.161:0/82461427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf9015a850 msgr2=0x7fcf900af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:52.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf8fe00640 1 --2- 172.21.15.161:0/82461427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf9015a850 0x7fcf900af220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:52.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf8fe00640 1 -- 172.21.15.161:0/82461427 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9015a050 con 0x7fcf900b15a0 2024-05-22T21:14:52.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf8fe00640 1 --2- 172.21.15.161:0/82461427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf900b15a0 0x7fcf90158e50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009b20 tx=0x7fcf78005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:52.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf8cc00640 1 -- 172.21.15.161:0/82461427 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c004030 con 0x7fcf900b15a0 2024-05-22T21:14:52.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf8cc00640 1 -- 172.21.15.161:0/82461427 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7c004300 con 0x7fcf900b15a0 2024-05-22T21:14:52.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf97600640 1 -- 172.21.15.161:0/82461427 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf9015a330 con 0x7fcf900b15a0 2024-05-22T21:14:52.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf8cc00640 1 -- 172.21.15.161:0/82461427 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c003620 con 0x7fcf900b15a0 2024-05-22T21:14:52.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.231+0000 7fcf97600640 1 -- 172.21.15.161:0/82461427 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf90159390 con 0x7fcf900b15a0 2024-05-22T21:14:52.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.235+0000 7fcf97600640 1 -- 172.21.15.161:0/82461427 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf90067220 con 0x7fcf900b15a0 2024-05-22T21:14:52.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.235+0000 7fcf8cc00640 1 -- 172.21.15.161:0/82461427 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcf7c003780 con 0x7fcf900b15a0 2024-05-22T21:14:52.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.235+0000 7fcf8cc00640 1 --2- 172.21.15.161:0/82461427 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcf70066690 0x7fcf70068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:52.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.235+0000 7fcf8cc00640 1 -- 172.21.15.161:0/82461427 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf7c11dbf0 con 0x7fcf900b15a0 2024-05-22T21:14:52.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.235+0000 7fcf94c00640 1 --2- 172.21.15.161:0/82461427 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcf70066690 0x7fcf70068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:52.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.235+0000 7fcf94c00640 1 --2- 172.21.15.161:0/82461427 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcf70066690 0x7fcf70068b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcf84001c30 tx=0x7fcf84001e10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:52.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.239+0000 7fcf8cc00640 1 -- 172.21.15.161:0/82461427 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcf7c0eb000 con 0x7fcf900b15a0 2024-05-22T21:14:52.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:52.427+0000 7fcf97600640 1 -- 172.21.15.161:0/82461427 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcf90067220 con 0x7fcf900b15a0 2024-05-22T21:14:53.053 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:14:53.056 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:14:53.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.059+0000 7fcf8cc00640 1 -- 172.21.15.161:0/82461427 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcf7c0ee620 con 0x7fcf900b15a0 2024-05-22T21:14:53.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.059+0000 7fcf97600640 1 -- 172.21.15.161:0/82461427 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcf70066690 msgr2=0x7fcf70068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.059+0000 7fcf97600640 1 --2- 172.21.15.161:0/82461427 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcf70066690 0x7fcf70068b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcf84001c30 tx=0x7fcf84001e10 comp rx=0 tx=0).stop 2024-05-22T21:14:53.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.059+0000 7fcf97600640 1 -- 172.21.15.161:0/82461427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf900b15a0 msgr2=0x7fcf90158e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.059+0000 7fcf97600640 1 --2- 172.21.15.161:0/82461427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf900b15a0 0x7fcf90158e50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009b20 tx=0x7fcf78005800 comp rx=0 tx=0).stop 2024-05-22T21:14:53.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.059+0000 7fcf97600640 1 -- 172.21.15.161:0/82461427 shutdown_connections 2024-05-22T21:14:53.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.059+0000 7fcf97600640 1 --2- 172.21.15.161:0/82461427 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcf70066690 0x7fcf70068b50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.059+0000 7fcf97600640 1 --2- 172.21.15.161:0/82461427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf9015a850 0x7fcf900af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.059+0000 7fcf97600640 1 --2- 172.21.15.161:0/82461427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf900b15a0 0x7fcf90158e50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.059+0000 7fcf97600640 1 --2- 172.21.15.161:0/82461427 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf780047d0 0x7fcf90139920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.059+0000 7fcf97600640 1 -- 172.21.15.161:0/82461427 >> 172.21.15.161:0/82461427 conn(0x7fcf9006a2d0 msgr2=0x7fcf9015b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:53.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.059+0000 7fcf97600640 1 -- 172.21.15.161:0/82461427 shutdown_connections 2024-05-22T21:14:53.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.059+0000 7fcf97600640 1 -- 172.21.15.161:0/82461427 wait complete. 2024-05-22T21:14:53.087 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-22T21:14:53.087 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:14:53.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b3200640 1 Processor -- start 2024-05-22T21:14:53.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b3200640 1 -- start start 2024-05-22T21:14:53.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b3200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42ac146970 0x7f42ac160be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b3200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f42ac06a5e0 0x7f42ac0b3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b3200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ac0b34a0 con 0x7f42ac071770 2024-05-22T21:14:53.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b3200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ac146d50 con 0x7f42ac06a5e0 2024-05-22T21:14:53.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b3200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac1614a0 con 0x7f42ac146970 2024-05-22T21:14:53.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b2800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f42ac06a5e0 0x7f42ac0b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b1400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f42ac071770 0x7f42ac069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:57728/0 (socket says 172.21.15.161:57728) 2024-05-22T21:14:53.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b1400640 1 -- 172.21.15.161:0/3473754582 learned_addr learned my addr 172.21.15.161:0/3473754582 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:53.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b1e00640 1 --2- 172.21.15.161:0/3473754582 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42ac146970 0x7f42ac160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b0a00640 1 -- 172.21.15.161:0/3473754582 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034955319 0 0) 0x7f42ac0b34a0 con 0x7f42ac071770 2024-05-22T21:14:53.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b0a00640 1 -- 172.21.15.161:0/3473754582 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4294003530 con 0x7f42ac071770 2024-05-22T21:14:53.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b2800640 1 -- 172.21.15.161:0/3473754582 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42ac146970 msgr2=0x7f42ac160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:53.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b2800640 1 --2- 172.21.15.161:0/3473754582 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42ac146970 0x7f42ac160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b2800640 1 -- 172.21.15.161:0/3473754582 >> v1:172.21.15.161:6789/0 conn(0x7f42ac071770 legacy=0x7f42ac069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b2800640 1 -- 172.21.15.161:0/3473754582 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac161780 con 0x7f42ac06a5e0 2024-05-22T21:14:53.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.279+0000 7f42b1e00640 1 --2- 172.21.15.161:0/3473754582 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42ac146970 0x7f42ac160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:53.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.283+0000 7f42b2800640 1 --2- 172.21.15.161:0/3473754582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac06a5e0 0x7f42ac0b3930 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009d90 tx=0x7f42a801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d0e0765b3e9358 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:53.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.283+0000 7f42b0a00640 1 -- 172.21.15.161:0/3473754582 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a80a5a00 con 0x7f42ac06a5e0 2024-05-22T21:14:53.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.283+0000 7f42b0a00640 1 -- 172.21.15.161:0/3473754582 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a80a5b60 con 0x7f42ac06a5e0 2024-05-22T21:14:53.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.283+0000 7f42b0a00640 1 -- 172.21.15.161:0/3473754582 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a80ad440 con 0x7f42ac06a5e0 2024-05-22T21:14:53.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.283+0000 7f42b3200640 1 -- 172.21.15.161:0/3473754582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac06a5e0 msgr2=0x7f42ac0b3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.283+0000 7f42b3200640 1 --2- 172.21.15.161:0/3473754582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac06a5e0 0x7f42ac0b3930 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009d90 tx=0x7f42a801c940 comp rx=0 tx=0).stop 2024-05-22T21:14:53.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.283+0000 7f42b3200640 1 -- 172.21.15.161:0/3473754582 shutdown_connections 2024-05-22T21:14:53.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.283+0000 7f42b3200640 1 --2- 172.21.15.161:0/3473754582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac06a5e0 0x7f42ac0b3930 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.283+0000 7f42b3200640 1 --2- 172.21.15.161:0/3473754582 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42ac146970 0x7f42ac160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.283+0000 7f42b3200640 1 -- 172.21.15.161:0/3473754582 >> 172.21.15.161:0/3473754582 conn(0x7f42ac06ba00 msgr2=0x7f42ac0b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:53.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.283+0000 7f42b3200640 1 -- 172.21.15.161:0/3473754582 shutdown_connections 2024-05-22T21:14:53.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.283+0000 7f42b3200640 1 -- 172.21.15.161:0/3473754582 wait complete. 2024-05-22T21:14:53.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.283+0000 7f42b3200640 1 Processor -- start 2024-05-22T21:14:53.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.283+0000 7f42b3200640 1 -- start start 2024-05-22T21:14:53.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b3200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42ac06a5e0 0x7f42ac15cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b3200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac071770 0x7f42ac13be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b3200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42ac146970 0x7f42ac13c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b3200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac137670 con 0x7f42ac071770 2024-05-22T21:14:53.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b3200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac15e0a0 con 0x7f42ac06a5e0 2024-05-22T21:14:53.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b3200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac15e380 con 0x7f42ac146970 2024-05-22T21:14:53.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b1400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac071770 0x7f42ac13be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b1400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac071770 0x7f42ac13be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:56038/0 (socket says 172.21.15.161:56038) 2024-05-22T21:14:53.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b1400640 1 -- 172.21.15.161:0/2080773587 learned_addr learned my addr 172.21.15.161:0/2080773587 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:53.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b1e00640 1 --2- 172.21.15.161:0/2080773587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42ac06a5e0 0x7f42ac15cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b1400640 1 -- 172.21.15.161:0/2080773587 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42ac146970 msgr2=0x7f42ac13c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b2800640 1 --2- 172.21.15.161:0/2080773587 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42ac146970 0x7f42ac13c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b1400640 1 --2- 172.21.15.161:0/2080773587 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42ac146970 0x7f42ac13c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b1400640 1 -- 172.21.15.161:0/2080773587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42ac06a5e0 msgr2=0x7f42ac15cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b1400640 1 --2- 172.21.15.161:0/2080773587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42ac06a5e0 0x7f42ac15cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b1400640 1 -- 172.21.15.161:0/2080773587 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a0009420 con 0x7f42ac071770 2024-05-22T21:14:53.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b1e00640 1 --2- 172.21.15.161:0/2080773587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42ac06a5e0 0x7f42ac15cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:53.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b2800640 1 --2- 172.21.15.161:0/2080773587 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42ac146970 0x7f42ac13c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:14:53.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b1400640 1 --2- 172.21.15.161:0/2080773587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac071770 0x7f42ac13be20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f429c00c4d0 tx=0x7f429c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:53.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42a6000640 1 -- 172.21.15.161:0/2080773587 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c005cf0 con 0x7f42ac071770 2024-05-22T21:14:53.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42a6000640 1 -- 172.21.15.161:0/2080773587 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c005e50 con 0x7f42ac071770 2024-05-22T21:14:53.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42a6000640 1 -- 172.21.15.161:0/2080773587 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c007270 con 0x7f42ac071770 2024-05-22T21:14:53.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b3200640 1 -- 172.21.15.161:0/2080773587 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a8009a70 con 0x7f42ac071770 2024-05-22T21:14:53.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b3200640 1 -- 172.21.15.161:0/2080773587 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ac15e940 con 0x7f42ac071770 2024-05-22T21:14:53.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.287+0000 7f42b3200640 1 -- 172.21.15.161:0/2080773587 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ac137840 con 0x7f42ac071770 2024-05-22T21:14:53.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.291+0000 7f42a6000640 1 -- 172.21.15.161:0/2080773587 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f429c0073d0 con 0x7f42ac071770 2024-05-22T21:14:53.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.291+0000 7f42a6000640 1 --2- 172.21.15.161:0/2080773587 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4290066690 0x7f4290068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.291+0000 7f42a6000640 1 -- 172.21.15.161:0/2080773587 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f429c0837c0 con 0x7f42ac071770 2024-05-22T21:14:53.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.291+0000 7f42b1e00640 1 --2- 172.21.15.161:0/2080773587 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4290066690 0x7f4290068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.291+0000 7f42b1e00640 1 --2- 172.21.15.161:0/2080773587 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4290066690 0x7f4290068b50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009bd0 tx=0x7f42a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:53.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.295+0000 7f42a6000640 1 -- 172.21.15.161:0/2080773587 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f429c050ab0 con 0x7f42ac071770 2024-05-22T21:14:53.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.479+0000 7f42b3200640 1 -- 172.21.15.161:0/2080773587 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42ac067070 con 0x7f42ac071770 2024-05-22T21:14:53.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.479+0000 7f42a6000640 1 -- 172.21.15.161:0/2080773587 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f429c017020 con 0x7f42ac071770 2024-05-22T21:14:53.483 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:53.483 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/1767934947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1767934947},{"type":"v1","addr":"172.21.15.161:6836","nonce":1767934947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1873337287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1873337287},{"type":"v1","addr":"172.21.15.179:6834","nonce":1873337287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1815838236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":1815838236},{"type":"v1","addr":"172.21.15.161:6837","nonce":1815838236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:50.052292+0000","modified":"2024-05-22T21:14:53.058881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5374":{"gid":5374,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/389418050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":389418050},{"type":"v1","addr":"172.21.15.179:6835","nonce":389418050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-22T21:14:53.483 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 23 2024-05-22T21:14:53.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.479+0000 7f42b3200640 1 -- 172.21.15.161:0/2080773587 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4290066690 msgr2=0x7f4290068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.479+0000 7f42b3200640 1 --2- 172.21.15.161:0/2080773587 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4290066690 0x7f4290068b50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009bd0 tx=0x7f42a0008040 comp rx=0 tx=0).stop 2024-05-22T21:14:53.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.479+0000 7f42b3200640 1 -- 172.21.15.161:0/2080773587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac071770 msgr2=0x7f42ac13be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.479+0000 7f42b3200640 1 --2- 172.21.15.161:0/2080773587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac071770 0x7f42ac13be20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f429c00c4d0 tx=0x7f429c00c990 comp rx=0 tx=0).stop 2024-05-22T21:14:53.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.479+0000 7f42b3200640 1 -- 172.21.15.161:0/2080773587 shutdown_connections 2024-05-22T21:14:53.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.479+0000 7f42b3200640 1 --2- 172.21.15.161:0/2080773587 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4290066690 0x7f4290068b50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.479+0000 7f42b3200640 1 --2- 172.21.15.161:0/2080773587 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42ac146970 0x7f42ac13c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.479+0000 7f42b3200640 1 --2- 172.21.15.161:0/2080773587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42ac071770 0x7f42ac13be20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.479+0000 7f42b3200640 1 --2- 172.21.15.161:0/2080773587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42ac06a5e0 0x7f42ac15cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.479+0000 7f42b3200640 1 -- 172.21.15.161:0/2080773587 >> 172.21.15.161:0/2080773587 conn(0x7f42ac06ba00 msgr2=0x7f42ac06e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:53.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.479+0000 7f42b3200640 1 -- 172.21.15.161:0/2080773587 shutdown_connections 2024-05-22T21:14:53.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.479+0000 7f42b3200640 1 -- 172.21.15.161:0/2080773587 wait complete. 2024-05-22T21:14:53.505 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:14:53.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.635+0000 7fde06600640 1 Processor -- start 2024-05-22T21:14:53.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.635+0000 7fde06600640 1 -- start start 2024-05-22T21:14:53.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.635+0000 7fde06600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde001597f0 0x7fde000706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.635+0000 7fde06600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fde000b3070 0x7fde000cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.635+0000 7fde06600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde00071040 con 0x7fde00070be0 2024-05-22T21:14:53.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.635+0000 7fde06600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde000b3500 con 0x7fde000b3070 2024-05-22T21:14:53.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.635+0000 7fde06600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde00159bd0 con 0x7fde001597f0 2024-05-22T21:14:53.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.639+0000 7fddffe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fde000b3070 0x7fde000cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.639+0000 7fddfea00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fde00070be0 0x7fde000b2960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:57744/0 (socket says 172.21.15.161:57744) 2024-05-22T21:14:53.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.639+0000 7fddfea00640 1 -- 172.21.15.161:0/3484094819 learned_addr learned my addr 172.21.15.161:0/3484094819 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:53.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.639+0000 7fddff400640 1 --2- 172.21.15.161:0/3484094819 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde001597f0 0x7fde000706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.639+0000 7fddffe00640 1 -- 172.21.15.161:0/3484094819 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde001597f0 msgr2=0x7fde000706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:53.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.639+0000 7fddffe00640 1 --2- 172.21.15.161:0/3484094819 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde001597f0 0x7fde000706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.639+0000 7fddffe00640 1 -- 172.21.15.161:0/3484094819 >> v1:172.21.15.161:6789/0 conn(0x7fde00070be0 legacy=0x7fde000b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.639+0000 7fddffe00640 1 -- 172.21.15.161:0/3484094819 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde000cbdd0 con 0x7fde000b3070 2024-05-22T21:14:53.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.639+0000 7fddfe000640 1 -- 172.21.15.161:0/3484094819 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825273815 0 0) 0x7fde00071040 con 0x7fde00070be0 2024-05-22T21:14:53.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.639+0000 7fddffe00640 1 --2- 172.21.15.161:0/3484094819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde000b3070 0x7fde000cc550 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009d90 tx=0x7fddf801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21479635c08f7166 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:53.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.639+0000 7fddfe000640 1 -- 172.21.15.161:0/3484094819 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf80a59b0 con 0x7fde000b3070 2024-05-22T21:14:53.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.639+0000 7fddfe000640 1 -- 172.21.15.161:0/3484094819 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf80a5b10 con 0x7fde000b3070 2024-05-22T21:14:53.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.639+0000 7fddfe000640 1 -- 172.21.15.161:0/3484094819 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf80a5ea0 con 0x7fde000b3070 2024-05-22T21:14:53.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fde06600640 1 -- 172.21.15.161:0/3484094819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde000b3070 msgr2=0x7fde000cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fde06600640 1 --2- 172.21.15.161:0/3484094819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde000b3070 0x7fde000cc550 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009d90 tx=0x7fddf801c970 comp rx=0 tx=0).stop 2024-05-22T21:14:53.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fde06600640 1 -- 172.21.15.161:0/3484094819 shutdown_connections 2024-05-22T21:14:53.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fde06600640 1 --2- 172.21.15.161:0/3484094819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde000b3070 0x7fde000cc550 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fde06600640 1 --2- 172.21.15.161:0/3484094819 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde001597f0 0x7fde000706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fde06600640 1 -- 172.21.15.161:0/3484094819 >> 172.21.15.161:0/3484094819 conn(0x7fde000b0bc0 msgr2=0x7fde000b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:53.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fde06600640 1 -- 172.21.15.161:0/3484094819 shutdown_connections 2024-05-22T21:14:53.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fde06600640 1 -- 172.21.15.161:0/3484094819 wait complete. 2024-05-22T21:14:53.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fde06600640 1 Processor -- start 2024-05-22T21:14:53.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fde06600640 1 -- start start 2024-05-22T21:14:53.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fde06600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde00070be0 0x7fde0014c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fde06600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde000b3070 0x7fde00148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fde06600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde001597f0 0x7fde00148a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fde06600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0014de90 con 0x7fde000b3070 2024-05-22T21:14:53.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fde06600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0015ab40 con 0x7fde00070be0 2024-05-22T21:14:53.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fde06600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0015a860 con 0x7fde001597f0 2024-05-22T21:14:53.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fddfea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde000b3070 0x7fde00148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fddfea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde000b3070 0x7fde00148510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:56064/0 (socket says 172.21.15.161:56064) 2024-05-22T21:14:53.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fddfea00640 1 -- 172.21.15.161:0/1481906453 learned_addr learned my addr 172.21.15.161:0/1481906453 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:53.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fddff400640 1 --2- 172.21.15.161:0/1481906453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde00070be0 0x7fde0014c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fddffe00640 1 --2- 172.21.15.161:0/1481906453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde001597f0 0x7fde00148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fddfea00640 1 -- 172.21.15.161:0/1481906453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde001597f0 msgr2=0x7fde00148a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fddfea00640 1 --2- 172.21.15.161:0/1481906453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde001597f0 0x7fde00148a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fddfea00640 1 -- 172.21.15.161:0/1481906453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde00070be0 msgr2=0x7fde0014c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fddfea00640 1 --2- 172.21.15.161:0/1481906453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde00070be0 0x7fde0014c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.643+0000 7fddfea00640 1 -- 172.21.15.161:0/1481906453 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf8009a70 con 0x7fde000b3070 2024-05-22T21:14:53.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.647+0000 7fddfea00640 1 --2- 172.21.15.161:0/1481906453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde000b3070 0x7fde00148510 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fddec0011f0 tx=0x7fddec00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:53.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.647+0000 7fddf7400640 1 -- 172.21.15.161:0/1481906453 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec0074e0 con 0x7fde000b3070 2024-05-22T21:14:53.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.647+0000 7fddf7400640 1 -- 172.21.15.161:0/1481906453 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec007640 con 0x7fde000b3070 2024-05-22T21:14:53.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.647+0000 7fde06600640 1 -- 172.21.15.161:0/1481906453 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde00159f30 con 0x7fde000b3070 2024-05-22T21:14:53.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.647+0000 7fddf7400640 1 -- 172.21.15.161:0/1481906453 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec004a10 con 0x7fde000b3070 2024-05-22T21:14:53.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.647+0000 7fde06600640 1 -- 172.21.15.161:0/1481906453 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0015a4f0 con 0x7fde000b3070 2024-05-22T21:14:53.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.647+0000 7fde06600640 1 -- 172.21.15.161:0/1481906453 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0015acb0 con 0x7fde000b3070 2024-05-22T21:14:53.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.647+0000 7fddf7400640 1 -- 172.21.15.161:0/1481906453 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fddec004b70 con 0x7fde000b3070 2024-05-22T21:14:53.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.647+0000 7fddf7400640 1 --2- 172.21.15.161:0/1481906453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdde00666c0 0x7fdde0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.647+0000 7fddf7400640 1 -- 172.21.15.161:0/1481906453 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddec0836f0 con 0x7fde000b3070 2024-05-22T21:14:53.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.647+0000 7fddff400640 1 --2- 172.21.15.161:0/1481906453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdde00666c0 0x7fdde0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.651+0000 7fddff400640 1 --2- 172.21.15.161:0/1481906453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdde00666c0 0x7fdde0068b80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fddf000ab50 tx=0x7fddf0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:53.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.651+0000 7fddf7400640 1 -- 172.21.15.161:0/1481906453 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fddec050a60 con 0x7fde000b3070 2024-05-22T21:14:53.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.807+0000 7fde06600640 1 -- 172.21.15.161:0/1481906453 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde00067220 con 0x7fde000b3070 2024-05-22T21:14:53.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.807+0000 7fddf7400640 1 -- 172.21.15.161:0/1481906453 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16093 (secure 0 0 0) 0x7fddec018090 con 0x7fde000b3070 2024-05-22T21:14:53.811 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:53.811 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":47,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:14:51.054474+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:48.612726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:49.053022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:14:53.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.807+0000 7fde06600640 1 -- 172.21.15.161:0/1481906453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdde00666c0 msgr2=0x7fdde0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.807+0000 7fde06600640 1 --2- 172.21.15.161:0/1481906453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdde00666c0 0x7fdde0068b80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fddf000ab50 tx=0x7fddf0008040 comp rx=0 tx=0).stop 2024-05-22T21:14:53.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.807+0000 7fde06600640 1 -- 172.21.15.161:0/1481906453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde000b3070 msgr2=0x7fde00148510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.807+0000 7fde06600640 1 --2- 172.21.15.161:0/1481906453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde000b3070 0x7fde00148510 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fddec0011f0 tx=0x7fddec00c9b0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.807+0000 7fde06600640 1 -- 172.21.15.161:0/1481906453 shutdown_connections 2024-05-22T21:14:53.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.807+0000 7fde06600640 1 --2- 172.21.15.161:0/1481906453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdde00666c0 0x7fdde0068b80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.807+0000 7fde06600640 1 --2- 172.21.15.161:0/1481906453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde001597f0 0x7fde00148a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.807+0000 7fde06600640 1 --2- 172.21.15.161:0/1481906453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde000b3070 0x7fde00148510 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.807+0000 7fde06600640 1 --2- 172.21.15.161:0/1481906453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde00070be0 0x7fde0014c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.807+0000 7fde06600640 1 -- 172.21.15.161:0/1481906453 >> 172.21.15.161:0/1481906453 conn(0x7fde000b0bc0 msgr2=0x7fde00158760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:53.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.807+0000 7fde06600640 1 -- 172.21.15.161:0/1481906453 shutdown_connections 2024-05-22T21:14:53.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.811+0000 7fde06600640 1 -- 172.21.15.161:0/1481906453 wait complete. 2024-05-22T21:14:53.832 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-22T21:14:53.832 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:14:53.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.963+0000 7fc6a7800640 1 Processor -- start 2024-05-22T21:14:53.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.963+0000 7fc6a7800640 1 -- start start 2024-05-22T21:14:53.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.963+0000 7fc6a7800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc6a0143ec0 0x7fc6a015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.963+0000 7fc6a7800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc6a015ec90 0x7fc6a015f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.963+0000 7fc6a7800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a01442a0 con 0x7fc6a00715a0 2024-05-22T21:14:53.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.963+0000 7fc6a7800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a015f680 con 0x7fc6a015ec90 2024-05-22T21:14:53.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.963+0000 7fc6a7800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a0070500 con 0x7fc6a0143ec0 2024-05-22T21:14:53.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.963+0000 7fc6a5800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc6a015ec90 0x7fc6a015f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.963+0000 7fc6a5800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc6a015ec90 0x7fc6a015f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:56080/0 (socket says 172.21.15.161:56080) 2024-05-22T21:14:53.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.963+0000 7fc6a5800640 1 -- 172.21.15.161:0/3492671386 learned_addr learned my addr 172.21.15.161:0/3492671386 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:53.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.963+0000 7fc6a4e00640 1 --2- 172.21.15.161:0/3492671386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc6a0143ec0 0x7fc6a015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.963+0000 7fc6a5800640 1 -- 172.21.15.161:0/3492671386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc6a0143ec0 msgr2=0x7fc6a015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:53.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.963+0000 7fc6a5800640 1 --2- 172.21.15.161:0/3492671386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc6a0143ec0 0x7fc6a015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.963+0000 7fc6a5800640 1 -- 172.21.15.161:0/3492671386 >> v1:172.21.15.179:6789/0 conn(0x7fc6a00715a0 legacy=0x7fc6a006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.963+0000 7fc6a5800640 1 -- 172.21.15.161:0/3492671386 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a00707e0 con 0x7fc6a015ec90 2024-05-22T21:14:53.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.967+0000 7fc6a5800640 1 --2- 172.21.15.161:0/3492671386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a015ec90 0x7fc6a015f140 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc698009e50 tx=0x7fc69801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4e09bd678e3cc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:53.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.967+0000 7fc69f400640 1 -- 172.21.15.161:0/3492671386 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6980a57d0 con 0x7fc6a015ec90 2024-05-22T21:14:53.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.967+0000 7fc69f400640 1 -- 172.21.15.161:0/3492671386 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6980a5930 con 0x7fc6a015ec90 2024-05-22T21:14:53.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.967+0000 7fc69f400640 1 -- 172.21.15.161:0/3492671386 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6980a5cc0 con 0x7fc6a015ec90 2024-05-22T21:14:53.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.967+0000 7fc6a7800640 1 -- 172.21.15.161:0/3492671386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a015ec90 msgr2=0x7fc6a015f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.967+0000 7fc6a7800640 1 --2- 172.21.15.161:0/3492671386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a015ec90 0x7fc6a015f140 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc698009e50 tx=0x7fc69801ca00 comp rx=0 tx=0).stop 2024-05-22T21:14:53.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.967+0000 7fc6a7800640 1 -- 172.21.15.161:0/3492671386 shutdown_connections 2024-05-22T21:14:53.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.967+0000 7fc6a7800640 1 --2- 172.21.15.161:0/3492671386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a015ec90 0x7fc6a015f140 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.967+0000 7fc6a7800640 1 --2- 172.21.15.161:0/3492671386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc6a0143ec0 0x7fc6a015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.967+0000 7fc6a7800640 1 -- 172.21.15.161:0/3492671386 >> 172.21.15.161:0/3492671386 conn(0x7fc6a00b0bc0 msgr2=0x7fc6a00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:53.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.967+0000 7fc6a7800640 1 -- 172.21.15.161:0/3492671386 shutdown_connections 2024-05-22T21:14:53.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.967+0000 7fc6a7800640 1 -- 172.21.15.161:0/3492671386 wait complete. 2024-05-22T21:14:53.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc6a7800640 1 Processor -- start 2024-05-22T21:14:53.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc6a7800640 1 -- start start 2024-05-22T21:14:53.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc6a7800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc6a0143ec0 0x7fc6a0142830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc6a7800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a015ec90 0x7fc6a015a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc6a7800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6a013cca0 0x7fc6a013d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc6a7800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a00a8eb0 con 0x7fc6a015ec90 2024-05-22T21:14:53.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc6a7800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a013d670 con 0x7fc6a013cca0 2024-05-22T21:14:53.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc6a7800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a00a9ca0 con 0x7fc6a0143ec0 2024-05-22T21:14:53.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc69fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a015ec90 0x7fc6a015a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc69fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a015ec90 0x7fc6a015a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:56086/0 (socket says 172.21.15.161:56086) 2024-05-22T21:14:53.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc69fe00640 1 -- 172.21.15.161:0/1970277540 learned_addr learned my addr 172.21.15.161:0/1970277540 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:53.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc6a4e00640 1 --2- 172.21.15.161:0/1970277540 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc6a0143ec0 0x7fc6a0142830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc6a5800640 1 --2- 172.21.15.161:0/1970277540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6a013cca0 0x7fc6a013d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc69fe00640 1 -- 172.21.15.161:0/1970277540 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc6a0143ec0 msgr2=0x7fc6a0142830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc69fe00640 1 --2- 172.21.15.161:0/1970277540 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc6a0143ec0 0x7fc6a0142830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc69fe00640 1 -- 172.21.15.161:0/1970277540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6a013cca0 msgr2=0x7fc6a013d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:53.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc69fe00640 1 --2- 172.21.15.161:0/1970277540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6a013cca0 0x7fc6a013d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:53.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc69fe00640 1 -- 172.21.15.161:0/1970277540 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc698009b30 con 0x7fc6a015ec90 2024-05-22T21:14:53.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc69fe00640 1 --2- 172.21.15.161:0/1970277540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a015ec90 0x7fc6a015a5f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c500 tx=0x7fc68c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:53.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc69cc00640 1 -- 172.21.15.161:0/1970277540 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0052d0 con 0x7fc6a015ec90 2024-05-22T21:14:53.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc69cc00640 1 -- 172.21.15.161:0/1970277540 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c005430 con 0x7fc6a015ec90 2024-05-22T21:14:53.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc69cc00640 1 -- 172.21.15.161:0/1970277540 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c012830 con 0x7fc6a015ec90 2024-05-22T21:14:53.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc6a7800640 1 -- 172.21.15.161:0/1970277540 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a00aa130 con 0x7fc6a015ec90 2024-05-22T21:14:53.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.971+0000 7fc6a7800640 1 -- 172.21.15.161:0/1970277540 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a00a9370 con 0x7fc6a015ec90 2024-05-22T21:14:53.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.975+0000 7fc6a7800640 1 -- 172.21.15.161:0/1970277540 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a00a9e50 con 0x7fc6a015ec90 2024-05-22T21:14:53.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.975+0000 7fc69cc00640 1 -- 172.21.15.161:0/1970277540 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc68c012990 con 0x7fc6a015ec90 2024-05-22T21:14:53.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.975+0000 7fc69cc00640 1 --2- 172.21.15.161:0/1970277540 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc680066690 0x7fc680068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:53.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.975+0000 7fc69cc00640 1 -- 172.21.15.161:0/1970277540 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc68c083be0 con 0x7fc6a015ec90 2024-05-22T21:14:53.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.975+0000 7fc6a4e00640 1 --2- 172.21.15.161:0/1970277540 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc680066690 0x7fc680068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:53.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.975+0000 7fc6a4e00640 1 --2- 172.21.15.161:0/1970277540 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc680066690 0x7fc680068b50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00cc0d0 tx=0x7fc694008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:53.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:53.979+0000 7fc69cc00640 1 -- 172.21.15.161:0/1970277540 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc6a00a9e50 con 0x7fc6a015ec90 2024-05-22T21:14:54.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.131+0000 7fc6a7800640 1 -- 172.21.15.161:0/1970277540 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6a015b6d0 con 0x7fc680066690 2024-05-22T21:14:54.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.139+0000 7fc69cc00640 1 -- 172.21.15.161:0/1970277540 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284455 (secure 0 0 0) 0x7fc6a015b6d0 con 0x7fc680066690 2024-05-22T21:14:54.146 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:54.148 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:14:54.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.143+0000 7fc6a7800640 1 -- 172.21.15.161:0/1970277540 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc680066690 msgr2=0x7fc680068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:54.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.143+0000 7fc6a7800640 1 --2- 172.21.15.161:0/1970277540 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc680066690 0x7fc680068b50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00cc0d0 tx=0x7fc694008040 comp rx=0 tx=0).stop 2024-05-22T21:14:54.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.143+0000 7fc6a7800640 1 -- 172.21.15.161:0/1970277540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a015ec90 msgr2=0x7fc6a015a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:54.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.143+0000 7fc6a7800640 1 --2- 172.21.15.161:0/1970277540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a015ec90 0x7fc6a015a5f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c500 tx=0x7fc68c00c9c0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.143+0000 7fc6a7800640 1 -- 172.21.15.161:0/1970277540 shutdown_connections 2024-05-22T21:14:54.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.143+0000 7fc6a7800640 1 --2- 172.21.15.161:0/1970277540 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc680066690 0x7fc680068b50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.143+0000 7fc6a7800640 1 --2- 172.21.15.161:0/1970277540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6a013cca0 0x7fc6a013d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.143+0000 7fc6a7800640 1 --2- 172.21.15.161:0/1970277540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6a015ec90 0x7fc6a015a5f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.143+0000 7fc6a7800640 1 --2- 172.21.15.161:0/1970277540 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc6a0143ec0 0x7fc6a0142830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.143+0000 7fc6a7800640 1 -- 172.21.15.161:0/1970277540 >> 172.21.15.161:0/1970277540 conn(0x7fc6a00b0bc0 msgr2=0x7fc6a01430f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:54.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.143+0000 7fc6a7800640 1 -- 172.21.15.161:0/1970277540 shutdown_connections 2024-05-22T21:14:54.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.143+0000 7fc6a7800640 1 -- 172.21.15.161:0/1970277540 wait complete. 2024-05-22T21:14:54.166 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-05-22T21:14:52.411901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2874,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559625,"statfs":{"total":1600000000,"available":1597056375,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000576"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059198+0000","last_change":"2024-05-22T21:14:50.061770+0000","last_active":"2024-05-22T21:14:51.059198+0000","last_peered":"2024-05-22T21:14:51.059198+0000","last_clean":"2024-05-22T21:14:51.059198+0000","last_became_active":"2024-05-22T21:14:50.061152+0000","last_became_peered":"2024-05-22T21:14:50.061152+0000","last_unstale":"2024-05-22T21:14:51.059198+0000","last_undegraded":"2024-05-22T21:14:51.059198+0000","last_fullsized":"2024-05-22T21:14:51.059198+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:40:22.552291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.076346+0000","last_change":"2024-05-22T21:14:50.076275+0000","last_active":"2024-05-22T21:14:50.076346+0000","last_peered":"2024-05-22T21:14:50.076346+0000","last_clean":"2024-05-22T21:14:50.076346+0000","last_became_active":"2024-05-22T21:14:50.076061+0000","last_became_peered":"2024-05-22T21:14:50.076061+0000","last_unstale":"2024-05-22T21:14:50.076346+0000","last_undegraded":"2024-05-22T21:14:50.076346+0000","last_fullsized":"2024-05-22T21:14:50.076346+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:32:09.786594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058410+0000","last_change":"2024-05-22T21:14:50.066837+0000","last_active":"2024-05-22T21:14:51.058410+0000","last_peered":"2024-05-22T21:14:51.058410+0000","last_clean":"2024-05-22T21:14:51.058410+0000","last_became_active":"2024-05-22T21:14:50.066583+0000","last_became_peered":"2024-05-22T21:14:50.066583+0000","last_unstale":"2024-05-22T21:14:51.058410+0000","last_undegraded":"2024-05-22T21:14:51.058410+0000","last_fullsized":"2024-05-22T21:14:51.058410+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:12:27.713458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.076076+0000","last_change":"2024-05-22T21:14:50.075848+0000","last_active":"2024-05-22T21:14:50.076076+0000","last_peered":"2024-05-22T21:14:50.076076+0000","last_clean":"2024-05-22T21:14:50.076076+0000","last_became_active":"2024-05-22T21:14:50.075598+0000","last_became_peered":"2024-05-22T21:14:50.075598+0000","last_unstale":"2024-05-22T21:14:50.076076+0000","last_undegraded":"2024-05-22T21:14:50.076076+0000","last_fullsized":"2024-05-22T21:14:50.076076+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:12:27.713458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.077171+0000","last_change":"2024-05-22T21:14:50.076846+0000","last_active":"2024-05-22T21:14:50.077171+0000","last_peered":"2024-05-22T21:14:50.077171+0000","last_clean":"2024-05-22T21:14:50.077171+0000","last_became_active":"2024-05-22T21:14:50.076643+0000","last_became_peered":"2024-05-22T21:14:50.076643+0000","last_unstale":"2024-05-22T21:14:50.077171+0000","last_undegraded":"2024-05-22T21:14:50.077171+0000","last_fullsized":"2024-05-22T21:14:50.077171+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:36:31.712862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057952+0000","last_change":"2024-05-22T21:14:50.064710+0000","last_active":"2024-05-22T21:14:51.057952+0000","last_peered":"2024-05-22T21:14:51.057952+0000","last_clean":"2024-05-22T21:14:51.057952+0000","last_became_active":"2024-05-22T21:14:50.064441+0000","last_became_peered":"2024-05-22T21:14:50.064441+0000","last_unstale":"2024-05-22T21:14:51.057952+0000","last_undegraded":"2024-05-22T21:14:51.057952+0000","last_fullsized":"2024-05-22T21:14:51.057952+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:36:53.357957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058503+0000","last_change":"2024-05-22T21:14:50.064004+0000","last_active":"2024-05-22T21:14:51.058503+0000","last_peered":"2024-05-22T21:14:51.058503+0000","last_clean":"2024-05-22T21:14:51.058503+0000","last_became_active":"2024-05-22T21:14:50.063751+0000","last_became_peered":"2024-05-22T21:14:50.063751+0000","last_unstale":"2024-05-22T21:14:51.058503+0000","last_undegraded":"2024-05-22T21:14:51.058503+0000","last_fullsized":"2024-05-22T21:14:51.058503+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:39:03.840693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.073771+0000","last_change":"2024-05-22T21:14:50.073646+0000","last_active":"2024-05-22T21:14:50.073771+0000","last_peered":"2024-05-22T21:14:50.073771+0000","last_clean":"2024-05-22T21:14:50.073771+0000","last_became_active":"2024-05-22T21:14:50.073415+0000","last_became_peered":"2024-05-22T21:14:50.073415+0000","last_unstale":"2024-05-22T21:14:50.073771+0000","last_undegraded":"2024-05-22T21:14:50.073771+0000","last_fullsized":"2024-05-22T21:14:50.073771+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:43:40.200304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.069977+0000","last_change":"2024-05-22T21:14:50.068582+0000","last_active":"2024-05-22T21:14:50.069977+0000","last_peered":"2024-05-22T21:14:50.069977+0000","last_clean":"2024-05-22T21:14:50.069977+0000","last_became_active":"2024-05-22T21:14:50.067719+0000","last_became_peered":"2024-05-22T21:14:50.067719+0000","last_unstale":"2024-05-22T21:14:50.069977+0000","last_undegraded":"2024-05-22T21:14:50.069977+0000","last_fullsized":"2024-05-22T21:14:50.069977+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:40:22.552291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.074193+0000","last_change":"2024-05-22T21:14:50.074044+0000","last_active":"2024-05-22T21:14:50.074193+0000","last_peered":"2024-05-22T21:14:50.074193+0000","last_clean":"2024-05-22T21:14:50.074193+0000","last_became_active":"2024-05-22T21:14:50.072779+0000","last_became_peered":"2024-05-22T21:14:50.072779+0000","last_unstale":"2024-05-22T21:14:50.074193+0000","last_undegraded":"2024-05-22T21:14:50.074193+0000","last_fullsized":"2024-05-22T21:14:50.074193+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:36:31.712862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.072208+0000","last_change":"2024-05-22T21:14:50.072132+0000","last_active":"2024-05-22T21:14:50.072208+0000","last_peered":"2024-05-22T21:14:50.072208+0000","last_clean":"2024-05-22T21:14:50.072208+0000","last_became_active":"2024-05-22T21:14:50.071230+0000","last_became_peered":"2024-05-22T21:14:50.071230+0000","last_unstale":"2024-05-22T21:14:50.072208+0000","last_undegraded":"2024-05-22T21:14:50.072208+0000","last_fullsized":"2024-05-22T21:14:50.072208+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:33:20.372548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.070491+0000","last_change":"2024-05-22T21:14:50.070398+0000","last_active":"2024-05-22T21:14:50.070491+0000","last_peered":"2024-05-22T21:14:50.070491+0000","last_clean":"2024-05-22T21:14:50.070491+0000","last_became_active":"2024-05-22T21:14:50.070181+0000","last_became_peered":"2024-05-22T21:14:50.070181+0000","last_unstale":"2024-05-22T21:14:50.070491+0000","last_undegraded":"2024-05-22T21:14:50.070491+0000","last_fullsized":"2024-05-22T21:14:50.070491+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:39:03.840693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.066439+0000","last_change":"2024-05-22T21:14:50.066296+0000","last_active":"2024-05-22T21:14:50.066439+0000","last_peered":"2024-05-22T21:14:50.066439+0000","last_clean":"2024-05-22T21:14:50.066439+0000","last_became_active":"2024-05-22T21:14:50.065988+0000","last_became_peered":"2024-05-22T21:14:50.065988+0000","last_unstale":"2024-05-22T21:14:50.066439+0000","last_undegraded":"2024-05-22T21:14:50.066439+0000","last_fullsized":"2024-05-22T21:14:50.066439+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:17:20.539737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.070013+0000","last_change":"2024-05-22T21:14:50.069252+0000","last_active":"2024-05-22T21:14:50.070013+0000","last_peered":"2024-05-22T21:14:50.070013+0000","last_clean":"2024-05-22T21:14:50.070013+0000","last_became_active":"2024-05-22T21:14:50.068378+0000","last_became_peered":"2024-05-22T21:14:50.068378+0000","last_unstale":"2024-05-22T21:14:50.070013+0000","last_undegraded":"2024-05-22T21:14:50.070013+0000","last_fullsized":"2024-05-22T21:14:50.070013+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:37:51.679058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057855+0000","last_change":"2024-05-22T21:14:50.065207+0000","last_active":"2024-05-22T21:14:51.057855+0000","last_peered":"2024-05-22T21:14:51.057855+0000","last_clean":"2024-05-22T21:14:51.057855+0000","last_became_active":"2024-05-22T21:14:50.065013+0000","last_became_peered":"2024-05-22T21:14:50.065013+0000","last_unstale":"2024-05-22T21:14:51.057855+0000","last_undegraded":"2024-05-22T21:14:51.057855+0000","last_fullsized":"2024-05-22T21:14:51.057855+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:58:07.918046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.076690+0000","last_change":"2024-05-22T21:14:50.076578+0000","last_active":"2024-05-22T21:14:50.076690+0000","last_peered":"2024-05-22T21:14:50.076690+0000","last_clean":"2024-05-22T21:14:50.076690+0000","last_became_active":"2024-05-22T21:14:50.076214+0000","last_became_peered":"2024-05-22T21:14:50.076214+0000","last_unstale":"2024-05-22T21:14:50.076690+0000","last_undegraded":"2024-05-22T21:14:50.076690+0000","last_fullsized":"2024-05-22T21:14:50.076690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:15:52.936539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058493+0000","last_change":"2024-05-22T21:14:50.073211+0000","last_active":"2024-05-22T21:14:51.058493+0000","last_peered":"2024-05-22T21:14:51.058493+0000","last_clean":"2024-05-22T21:14:51.058493+0000","last_became_active":"2024-05-22T21:14:50.072982+0000","last_became_peered":"2024-05-22T21:14:50.072982+0000","last_unstale":"2024-05-22T21:14:51.058493+0000","last_undegraded":"2024-05-22T21:14:51.058493+0000","last_fullsized":"2024-05-22T21:14:51.058493+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:00:23.932047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058633+0000","last_change":"2024-05-22T21:14:50.071899+0000","last_active":"2024-05-22T21:14:51.058633+0000","last_peered":"2024-05-22T21:14:51.058633+0000","last_clean":"2024-05-22T21:14:51.058633+0000","last_became_active":"2024-05-22T21:14:50.068603+0000","last_became_peered":"2024-05-22T21:14:50.068603+0000","last_unstale":"2024-05-22T21:14:51.058633+0000","last_undegraded":"2024-05-22T21:14:51.058633+0000","last_fullsized":"2024-05-22T21:14:51.058633+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:03:55.018814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.079305+0000","last_change":"2024-05-22T21:14:50.079199+0000","last_active":"2024-05-22T21:14:50.079305+0000","last_peered":"2024-05-22T21:14:50.079305+0000","last_clean":"2024-05-22T21:14:50.079305+0000","last_became_active":"2024-05-22T21:14:50.078542+0000","last_became_peered":"2024-05-22T21:14:50.078542+0000","last_unstale":"2024-05-22T21:14:50.079305+0000","last_undegraded":"2024-05-22T21:14:50.079305+0000","last_fullsized":"2024-05-22T21:14:50.079305+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:50:42.819506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.066354+0000","last_change":"2024-05-22T21:14:50.065235+0000","last_active":"2024-05-22T21:14:50.066354+0000","last_peered":"2024-05-22T21:14:50.066354+0000","last_clean":"2024-05-22T21:14:50.066354+0000","last_became_active":"2024-05-22T21:14:50.064988+0000","last_became_peered":"2024-05-22T21:14:50.064988+0000","last_unstale":"2024-05-22T21:14:50.066354+0000","last_undegraded":"2024-05-22T21:14:50.066354+0000","last_fullsized":"2024-05-22T21:14:50.066354+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:43:51.085330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058035+0000","last_change":"2024-05-22T21:14:50.071100+0000","last_active":"2024-05-22T21:14:51.058035+0000","last_peered":"2024-05-22T21:14:51.058035+0000","last_clean":"2024-05-22T21:14:51.058035+0000","last_became_active":"2024-05-22T21:14:50.069813+0000","last_became_peered":"2024-05-22T21:14:50.069813+0000","last_unstale":"2024-05-22T21:14:51.058035+0000","last_undegraded":"2024-05-22T21:14:51.058035+0000","last_fullsized":"2024-05-22T21:14:51.058035+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:16:44.963153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.074672+0000","last_change":"2024-05-22T21:14:50.074482+0000","last_active":"2024-05-22T21:14:50.074672+0000","last_peered":"2024-05-22T21:14:50.074672+0000","last_clean":"2024-05-22T21:14:50.074672+0000","last_became_active":"2024-05-22T21:14:50.074279+0000","last_became_peered":"2024-05-22T21:14:50.074279+0000","last_unstale":"2024-05-22T21:14:50.074672+0000","last_undegraded":"2024-05-22T21:14:50.074672+0000","last_fullsized":"2024-05-22T21:14:50.074672+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:16:46.917494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.078403+0000","last_change":"2024-05-22T21:14:50.078289+0000","last_active":"2024-05-22T21:14:50.078403+0000","last_peered":"2024-05-22T21:14:50.078403+0000","last_clean":"2024-05-22T21:14:50.078403+0000","last_became_active":"2024-05-22T21:14:50.077892+0000","last_became_peered":"2024-05-22T21:14:50.077892+0000","last_unstale":"2024-05-22T21:14:50.078403+0000","last_undegraded":"2024-05-22T21:14:50.078403+0000","last_fullsized":"2024-05-22T21:14:50.078403+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:04:01.163527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.079288+0000","last_change":"2024-05-22T21:14:50.079071+0000","last_active":"2024-05-22T21:14:50.079288+0000","last_peered":"2024-05-22T21:14:50.079288+0000","last_clean":"2024-05-22T21:14:50.079288+0000","last_became_active":"2024-05-22T21:14:50.078196+0000","last_became_peered":"2024-05-22T21:14:50.078196+0000","last_unstale":"2024-05-22T21:14:50.079288+0000","last_undegraded":"2024-05-22T21:14:50.079288+0000","last_fullsized":"2024-05-22T21:14:50.079288+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:43:40.200304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.069560+0000","last_change":"2024-05-22T21:14:50.069171+0000","last_active":"2024-05-22T21:14:50.069560+0000","last_peered":"2024-05-22T21:14:50.069560+0000","last_clean":"2024-05-22T21:14:50.069560+0000","last_became_active":"2024-05-22T21:14:50.068937+0000","last_became_peered":"2024-05-22T21:14:50.068937+0000","last_unstale":"2024-05-22T21:14:50.069560+0000","last_undegraded":"2024-05-22T21:14:50.069560+0000","last_fullsized":"2024-05-22T21:14:50.069560+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:41:40.374906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.066640+0000","last_change":"2024-05-22T21:14:50.066534+0000","last_active":"2024-05-22T21:14:50.066640+0000","last_peered":"2024-05-22T21:14:50.066640+0000","last_clean":"2024-05-22T21:14:50.066640+0000","last_became_active":"2024-05-22T21:14:50.066036+0000","last_became_peered":"2024-05-22T21:14:50.066036+0000","last_unstale":"2024-05-22T21:14:50.066640+0000","last_undegraded":"2024-05-22T21:14:50.066640+0000","last_fullsized":"2024-05-22T21:14:50.066640+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:05:06.217169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058601+0000","last_change":"2024-05-22T21:14:50.075608+0000","last_active":"2024-05-22T21:14:51.058601+0000","last_peered":"2024-05-22T21:14:51.058601+0000","last_clean":"2024-05-22T21:14:51.058601+0000","last_became_active":"2024-05-22T21:14:50.075321+0000","last_became_peered":"2024-05-22T21:14:50.075321+0000","last_unstale":"2024-05-22T21:14:51.058601+0000","last_undegraded":"2024-05-22T21:14:51.058601+0000","last_fullsized":"2024-05-22T21:14:51.058601+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:15:52.936539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.071918+0000","last_change":"2024-05-22T21:14:50.071845+0000","last_active":"2024-05-22T21:14:50.071918+0000","last_peered":"2024-05-22T21:14:50.071918+0000","last_clean":"2024-05-22T21:14:50.071918+0000","last_became_active":"2024-05-22T21:14:50.071610+0000","last_became_peered":"2024-05-22T21:14:50.071610+0000","last_unstale":"2024-05-22T21:14:50.071918+0000","last_undegraded":"2024-05-22T21:14:50.071918+0000","last_fullsized":"2024-05-22T21:14:50.071918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:04:47.495150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.071290+0000","last_change":"2024-05-22T21:14:50.071126+0000","last_active":"2024-05-22T21:14:50.071290+0000","last_peered":"2024-05-22T21:14:50.071290+0000","last_clean":"2024-05-22T21:14:50.071290+0000","last_became_active":"2024-05-22T21:14:50.070856+0000","last_became_peered":"2024-05-22T21:14:50.070856+0000","last_unstale":"2024-05-22T21:14:50.071290+0000","last_undegraded":"2024-05-22T21:14:50.071290+0000","last_fullsized":"2024-05-22T21:14:50.071290+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:35:14.542505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058619+0000","last_change":"2024-05-22T21:14:50.062910+0000","last_active":"2024-05-22T21:14:51.058619+0000","last_peered":"2024-05-22T21:14:51.058619+0000","last_clean":"2024-05-22T21:14:51.058619+0000","last_became_active":"2024-05-22T21:14:50.060665+0000","last_became_peered":"2024-05-22T21:14:50.060665+0000","last_unstale":"2024-05-22T21:14:51.058619+0000","last_undegraded":"2024-05-22T21:14:51.058619+0000","last_fullsized":"2024-05-22T21:14:51.058619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:16:46.917494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058115+0000","last_change":"2024-05-22T21:14:50.071233+0000","last_active":"2024-05-22T21:14:51.058115+0000","last_peered":"2024-05-22T21:14:51.058115+0000","last_clean":"2024-05-22T21:14:51.058115+0000","last_became_active":"2024-05-22T21:14:50.070888+0000","last_became_peered":"2024-05-22T21:14:50.070888+0000","last_unstale":"2024-05-22T21:14:51.058115+0000","last_undegraded":"2024-05-22T21:14:51.058115+0000","last_fullsized":"2024-05-22T21:14:51.058115+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:46:38.590939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.063955+0000","last_change":"2024-05-22T21:14:50.063893+0000","last_active":"2024-05-22T21:14:50.063955+0000","last_peered":"2024-05-22T21:14:50.063955+0000","last_clean":"2024-05-22T21:14:50.063955+0000","last_became_active":"2024-05-22T21:14:50.063677+0000","last_became_peered":"2024-05-22T21:14:50.063677+0000","last_unstale":"2024-05-22T21:14:50.063955+0000","last_undegraded":"2024-05-22T21:14:50.063955+0000","last_fullsized":"2024-05-22T21:14:50.063955+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:45:06.407521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.063955+0000","last_change":"2024-05-22T21:14:50.063423+0000","last_active":"2024-05-22T21:14:50.063955+0000","last_peered":"2024-05-22T21:14:50.063955+0000","last_clean":"2024-05-22T21:14:50.063955+0000","last_became_active":"2024-05-22T21:14:50.063222+0000","last_became_peered":"2024-05-22T21:14:50.063222+0000","last_unstale":"2024-05-22T21:14:50.063955+0000","last_undegraded":"2024-05-22T21:14:50.063955+0000","last_fullsized":"2024-05-22T21:14:50.063955+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:16:46.917494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057819+0000","last_change":"2024-05-22T21:14:49.688612+0000","last_active":"2024-05-22T21:14:51.057819+0000","last_peered":"2024-05-22T21:14:51.057819+0000","last_clean":"2024-05-22T21:14:51.057819+0000","last_became_active":"2024-05-22T21:14:49.688041+0000","last_became_peered":"2024-05-22T21:14:49.688041+0000","last_unstale":"2024-05-22T21:14:51.057819+0000","last_undegraded":"2024-05-22T21:14:51.057819+0000","last_fullsized":"2024-05-22T21:14:51.057819+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:33:19.368954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.078179+0000","last_change":"2024-05-22T21:14:50.078095+0000","last_active":"2024-05-22T21:14:50.078179+0000","last_peered":"2024-05-22T21:14:50.078179+0000","last_clean":"2024-05-22T21:14:50.078179+0000","last_became_active":"2024-05-22T21:14:50.077160+0000","last_became_peered":"2024-05-22T21:14:50.077160+0000","last_unstale":"2024-05-22T21:14:50.078179+0000","last_undegraded":"2024-05-22T21:14:50.078179+0000","last_fullsized":"2024-05-22T21:14:50.078179+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:31:13.280796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058226+0000","last_change":"2024-05-22T21:14:49.686130+0000","last_active":"2024-05-22T21:14:51.058226+0000","last_peered":"2024-05-22T21:14:51.058226+0000","last_clean":"2024-05-22T21:14:51.058226+0000","last_became_active":"2024-05-22T21:14:49.685900+0000","last_became_peered":"2024-05-22T21:14:49.685900+0000","last_unstale":"2024-05-22T21:14:51.058226+0000","last_undegraded":"2024-05-22T21:14:51.058226+0000","last_fullsized":"2024-05-22T21:14:51.058226+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:46:32.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.071184+0000","last_change":"2024-05-22T21:14:50.070940+0000","last_active":"2024-05-22T21:14:50.071184+0000","last_peered":"2024-05-22T21:14:50.071184+0000","last_clean":"2024-05-22T21:14:50.071184+0000","last_became_active":"2024-05-22T21:14:50.070733+0000","last_became_peered":"2024-05-22T21:14:50.070733+0000","last_unstale":"2024-05-22T21:14:50.071184+0000","last_undegraded":"2024-05-22T21:14:50.071184+0000","last_fullsized":"2024-05-22T21:14:50.071184+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:08:05.259519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.081689+0000","last_change":"2024-05-22T21:14:49.680411+0000","last_active":"2024-05-22T21:14:50.081689+0000","last_peered":"2024-05-22T21:14:50.081689+0000","last_clean":"2024-05-22T21:14:50.081689+0000","last_became_active":"2024-05-22T21:14:49.679389+0000","last_became_peered":"2024-05-22T21:14:49.679389+0000","last_unstale":"2024-05-22T21:14:50.081689+0000","last_undegraded":"2024-05-22T21:14:50.081689+0000","last_fullsized":"2024-05-22T21:14:50.081689+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:15:51.932944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.069816+0000","last_change":"2024-05-22T21:14:50.069771+0000","last_active":"2024-05-22T21:14:50.069816+0000","last_peered":"2024-05-22T21:14:50.069816+0000","last_clean":"2024-05-22T21:14:50.069816+0000","last_became_active":"2024-05-22T21:14:50.069567+0000","last_became_peered":"2024-05-22T21:14:50.069567+0000","last_unstale":"2024-05-22T21:14:50.069816+0000","last_undegraded":"2024-05-22T21:14:50.069816+0000","last_fullsized":"2024-05-22T21:14:50.069816+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:50:42.819506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059382+0000","last_change":"2024-05-22T21:14:49.681679+0000","last_active":"2024-05-22T21:14:51.059382+0000","last_peered":"2024-05-22T21:14:51.059382+0000","last_clean":"2024-05-22T21:14:51.059382+0000","last_became_active":"2024-05-22T21:14:49.681446+0000","last_became_peered":"2024-05-22T21:14:49.681446+0000","last_unstale":"2024-05-22T21:14:51.059382+0000","last_undegraded":"2024-05-22T21:14:51.059382+0000","last_fullsized":"2024-05-22T21:14:51.059382+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:25:40.491524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058307+0000","last_change":"2024-05-22T21:14:50.075123+0000","last_active":"2024-05-22T21:14:51.058307+0000","last_peered":"2024-05-22T21:14:51.058307+0000","last_clean":"2024-05-22T21:14:51.058307+0000","last_became_active":"2024-05-22T21:14:50.074477+0000","last_became_peered":"2024-05-22T21:14:50.074477+0000","last_unstale":"2024-05-22T21:14:51.058307+0000","last_undegraded":"2024-05-22T21:14:51.058307+0000","last_fullsized":"2024-05-22T21:14:51.058307+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:25:41.495118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058061+0000","last_change":"2024-05-22T21:14:49.688420+0000","last_active":"2024-05-22T21:14:51.058061+0000","last_peered":"2024-05-22T21:14:51.058061+0000","last_clean":"2024-05-22T21:14:51.058061+0000","last_became_active":"2024-05-22T21:14:49.687817+0000","last_became_peered":"2024-05-22T21:14:49.687817+0000","last_unstale":"2024-05-22T21:14:51.058061+0000","last_undegraded":"2024-05-22T21:14:51.058061+0000","last_fullsized":"2024-05-22T21:14:51.058061+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:55:08.681604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.063680+0000","last_change":"2024-05-22T21:14:50.062829+0000","last_active":"2024-05-22T21:14:50.063680+0000","last_peered":"2024-05-22T21:14:50.063680+0000","last_clean":"2024-05-22T21:14:50.063680+0000","last_became_active":"2024-05-22T21:14:50.062609+0000","last_became_peered":"2024-05-22T21:14:50.062609+0000","last_unstale":"2024-05-22T21:14:50.063680+0000","last_undegraded":"2024-05-22T21:14:50.063680+0000","last_fullsized":"2024-05-22T21:14:50.063680+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:55:09.685198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058743+0000","last_change":"2024-05-22T21:14:49.683862+0000","last_active":"2024-05-22T21:14:51.058743+0000","last_peered":"2024-05-22T21:14:51.058743+0000","last_clean":"2024-05-22T21:14:51.058743+0000","last_became_active":"2024-05-22T21:14:49.683259+0000","last_became_peered":"2024-05-22T21:14:49.683259+0000","last_unstale":"2024-05-22T21:14:51.058743+0000","last_undegraded":"2024-05-22T21:14:51.058743+0000","last_fullsized":"2024-05-22T21:14:51.058743+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:46.067511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.073945+0000","last_change":"2024-05-22T21:14:50.073887+0000","last_active":"2024-05-22T21:14:50.073945+0000","last_peered":"2024-05-22T21:14:50.073945+0000","last_clean":"2024-05-22T21:14:50.073945+0000","last_became_active":"2024-05-22T21:14:50.072506+0000","last_became_peered":"2024-05-22T21:14:50.072506+0000","last_unstale":"2024-05-22T21:14:50.073945+0000","last_undegraded":"2024-05-22T21:14:50.073945+0000","last_fullsized":"2024-05-22T21:14:50.073945+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:08:05.259519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.065478+0000","last_change":"2024-05-22T21:14:49.676438+0000","last_active":"2024-05-22T21:14:50.065478+0000","last_peered":"2024-05-22T21:14:50.065478+0000","last_clean":"2024-05-22T21:14:50.065478+0000","last_became_active":"2024-05-22T21:14:49.676185+0000","last_became_peered":"2024-05-22T21:14:49.676185+0000","last_unstale":"2024-05-22T21:14:50.065478+0000","last_undegraded":"2024-05-22T21:14:50.065478+0000","last_fullsized":"2024-05-22T21:14:50.065478+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:12:26.709864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.079271+0000","last_change":"2024-05-22T21:14:50.078911+0000","last_active":"2024-05-22T21:14:50.079271+0000","last_peered":"2024-05-22T21:14:50.079271+0000","last_clean":"2024-05-22T21:14:50.079271+0000","last_became_active":"2024-05-22T21:14:50.077790+0000","last_became_peered":"2024-05-22T21:14:50.077790+0000","last_unstale":"2024-05-22T21:14:50.079271+0000","last_undegraded":"2024-05-22T21:14:50.079271+0000","last_fullsized":"2024-05-22T21:14:50.079271+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:16:46.917494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.062793+0000","last_change":"2024-05-22T21:14:49.690446+0000","last_active":"2024-05-22T21:14:50.062793+0000","last_peered":"2024-05-22T21:14:50.062793+0000","last_clean":"2024-05-22T21:14:50.062793+0000","last_became_active":"2024-05-22T21:14:49.689486+0000","last_became_peered":"2024-05-22T21:14:49.689486+0000","last_unstale":"2024-05-22T21:14:50.062793+0000","last_undegraded":"2024-05-22T21:14:50.062793+0000","last_fullsized":"2024-05-22T21:14:50.062793+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:06:35.897212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058712+0000","last_change":"2024-05-22T21:14:50.068317+0000","last_active":"2024-05-22T21:14:51.058712+0000","last_peered":"2024-05-22T21:14:51.058712+0000","last_clean":"2024-05-22T21:14:51.058712+0000","last_became_active":"2024-05-22T21:14:50.068098+0000","last_became_peered":"2024-05-22T21:14:50.068098+0000","last_unstale":"2024-05-22T21:14:51.058712+0000","last_undegraded":"2024-05-22T21:14:51.058712+0000","last_fullsized":"2024-05-22T21:14:51.058712+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:35:14.542505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058830+0000","last_change":"2024-05-22T21:14:49.685613+0000","last_active":"2024-05-22T21:14:51.058830+0000","last_peered":"2024-05-22T21:14:51.058830+0000","last_clean":"2024-05-22T21:14:51.058830+0000","last_became_active":"2024-05-22T21:14:49.685239+0000","last_became_peered":"2024-05-22T21:14:49.685239+0000","last_unstale":"2024-05-22T21:14:51.058830+0000","last_undegraded":"2024-05-22T21:14:51.058830+0000","last_fullsized":"2024-05-22T21:14:51.058830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:43:50.081735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059294+0000","last_change":"2024-05-22T21:14:50.073200+0000","last_active":"2024-05-22T21:14:51.059294+0000","last_peered":"2024-05-22T21:14:51.059294+0000","last_clean":"2024-05-22T21:14:51.059294+0000","last_became_active":"2024-05-22T21:14:50.072238+0000","last_became_peered":"2024-05-22T21:14:50.072238+0000","last_unstale":"2024-05-22T21:14:51.059294+0000","last_undegraded":"2024-05-22T21:14:51.059294+0000","last_fullsized":"2024-05-22T21:14:51.059294+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:08:05.259519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.062042+0000","last_change":"2024-05-22T21:14:49.687112+0000","last_active":"2024-05-22T21:14:50.062042+0000","last_peered":"2024-05-22T21:14:50.062042+0000","last_clean":"2024-05-22T21:14:50.062042+0000","last_became_active":"2024-05-22T21:14:49.686857+0000","last_became_peered":"2024-05-22T21:14:49.686857+0000","last_unstale":"2024-05-22T21:14:50.062042+0000","last_undegraded":"2024-05-22T21:14:50.062042+0000","last_fullsized":"2024-05-22T21:14:50.062042+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:17:19.536142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057859+0000","last_change":"2024-05-22T21:14:50.067586+0000","last_active":"2024-05-22T21:14:51.057859+0000","last_peered":"2024-05-22T21:14:51.057859+0000","last_clean":"2024-05-22T21:14:51.057859+0000","last_became_active":"2024-05-22T21:14:50.067098+0000","last_became_peered":"2024-05-22T21:14:50.067098+0000","last_unstale":"2024-05-22T21:14:51.057859+0000","last_undegraded":"2024-05-22T21:14:51.057859+0000","last_fullsized":"2024-05-22T21:14:51.057859+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:40:51.810150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059042+0000","last_change":"2024-05-22T21:14:49.685326+0000","last_active":"2024-05-22T21:14:51.059042+0000","last_peered":"2024-05-22T21:14:51.059042+0000","last_clean":"2024-05-22T21:14:51.059042+0000","last_became_active":"2024-05-22T21:14:49.685105+0000","last_became_peered":"2024-05-22T21:14:49.685105+0000","last_unstale":"2024-05-22T21:14:51.059042+0000","last_undegraded":"2024-05-22T21:14:51.059042+0000","last_fullsized":"2024-05-22T21:14:51.059042+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:35:13.538910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058919+0000","last_change":"2024-05-22T21:14:50.071196+0000","last_active":"2024-05-22T21:14:51.058919+0000","last_peered":"2024-05-22T21:14:51.058919+0000","last_clean":"2024-05-22T21:14:51.058919+0000","last_became_active":"2024-05-22T21:14:50.070169+0000","last_became_peered":"2024-05-22T21:14:50.070169+0000","last_unstale":"2024-05-22T21:14:51.058919+0000","last_undegraded":"2024-05-22T21:14:51.058919+0000","last_fullsized":"2024-05-22T21:14:51.058919+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:04:47.495150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.082149+0000","last_change":"2024-05-22T21:14:49.680428+0000","last_active":"2024-05-22T21:14:50.082149+0000","last_peered":"2024-05-22T21:14:50.082149+0000","last_clean":"2024-05-22T21:14:50.082149+0000","last_became_active":"2024-05-22T21:14:49.680131+0000","last_became_peered":"2024-05-22T21:14:49.680131+0000","last_unstale":"2024-05-22T21:14:50.082149+0000","last_undegraded":"2024-05-22T21:14:50.082149+0000","last_fullsized":"2024-05-22T21:14:50.082149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:56:44.859254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.076673+0000","last_change":"2024-05-22T21:14:50.076430+0000","last_active":"2024-05-22T21:14:50.076673+0000","last_peered":"2024-05-22T21:14:50.076673+0000","last_clean":"2024-05-22T21:14:50.076673+0000","last_became_active":"2024-05-22T21:14:50.075586+0000","last_became_peered":"2024-05-22T21:14:50.075586+0000","last_unstale":"2024-05-22T21:14:50.076673+0000","last_undegraded":"2024-05-22T21:14:50.076673+0000","last_fullsized":"2024-05-22T21:14:50.076673+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:37.575533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057962+0000","last_change":"2024-05-22T21:14:49.685529+0000","last_active":"2024-05-22T21:14:51.057962+0000","last_peered":"2024-05-22T21:14:51.057962+0000","last_clean":"2024-05-22T21:14:51.057962+0000","last_became_active":"2024-05-22T21:14:49.685294+0000","last_became_peered":"2024-05-22T21:14:49.685294+0000","last_unstale":"2024-05-22T21:14:51.057962+0000","last_undegraded":"2024-05-22T21:14:51.057962+0000","last_fullsized":"2024-05-22T21:14:51.057962+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:34:29.750559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.073563+0000","last_change":"2024-05-22T21:14:50.073433+0000","last_active":"2024-05-22T21:14:50.073563+0000","last_peered":"2024-05-22T21:14:50.073563+0000","last_clean":"2024-05-22T21:14:50.073563+0000","last_became_active":"2024-05-22T21:14:50.071865+0000","last_became_peered":"2024-05-22T21:14:50.071865+0000","last_unstale":"2024-05-22T21:14:50.073563+0000","last_undegraded":"2024-05-22T21:14:50.073563+0000","last_fullsized":"2024-05-22T21:14:50.073563+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:04:47.495150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059464+0000","last_change":"2024-05-22T21:14:49.675587+0000","last_active":"2024-05-22T21:14:51.059464+0000","last_peered":"2024-05-22T21:14:51.059464+0000","last_clean":"2024-05-22T21:14:51.059464+0000","last_became_active":"2024-05-22T21:14:49.675019+0000","last_became_peered":"2024-05-22T21:14:49.675019+0000","last_unstale":"2024-05-22T21:14:51.059464+0000","last_undegraded":"2024-05-22T21:14:51.059464+0000","last_fullsized":"2024-05-22T21:14:51.059464+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:43:39.196709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057723+0000","last_change":"2024-05-22T21:14:50.068120+0000","last_active":"2024-05-22T21:14:51.057723+0000","last_peered":"2024-05-22T21:14:51.057723+0000","last_clean":"2024-05-22T21:14:51.057723+0000","last_became_active":"2024-05-22T21:14:50.067920+0000","last_became_peered":"2024-05-22T21:14:50.067920+0000","last_unstale":"2024-05-22T21:14:51.057723+0000","last_undegraded":"2024-05-22T21:14:51.057723+0000","last_fullsized":"2024-05-22T21:14:51.057723+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:18:08.972692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058154+0000","last_change":"2024-05-22T21:14:49.685539+0000","last_active":"2024-05-22T21:14:51.058154+0000","last_peered":"2024-05-22T21:14:51.058154+0000","last_clean":"2024-05-22T21:14:51.058154+0000","last_became_active":"2024-05-22T21:14:49.685245+0000","last_became_peered":"2024-05-22T21:14:49.685245+0000","last_unstale":"2024-05-22T21:14:51.058154+0000","last_undegraded":"2024-05-22T21:14:51.058154+0000","last_fullsized":"2024-05-22T21:14:51.058154+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:48:46.286454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058595+0000","last_change":"2024-05-22T21:14:50.070568+0000","last_active":"2024-05-22T21:14:51.058595+0000","last_peered":"2024-05-22T21:14:51.058595+0000","last_clean":"2024-05-22T21:14:51.058595+0000","last_became_active":"2024-05-22T21:14:50.070268+0000","last_became_peered":"2024-05-22T21:14:50.070268+0000","last_unstale":"2024-05-22T21:14:51.058595+0000","last_undegraded":"2024-05-22T21:14:51.058595+0000","last_fullsized":"2024-05-22T21:14:51.058595+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:36:31.712862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.082657+0000","last_change":"2024-05-22T21:14:49.681995+0000","last_active":"2024-05-22T21:14:50.082657+0000","last_peered":"2024-05-22T21:14:50.082657+0000","last_clean":"2024-05-22T21:14:50.082657+0000","last_became_active":"2024-05-22T21:14:49.681696+0000","last_became_peered":"2024-05-22T21:14:49.681696+0000","last_unstale":"2024-05-22T21:14:50.082657+0000","last_undegraded":"2024-05-22T21:14:50.082657+0000","last_fullsized":"2024-05-22T21:14:50.082657+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:46:32.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.070709+0000","last_change":"2024-05-22T21:14:50.070588+0000","last_active":"2024-05-22T21:14:50.070709+0000","last_peered":"2024-05-22T21:14:50.070709+0000","last_clean":"2024-05-22T21:14:50.070709+0000","last_became_active":"2024-05-22T21:14:50.070386+0000","last_became_peered":"2024-05-22T21:14:50.070386+0000","last_unstale":"2024-05-22T21:14:50.070709+0000","last_undegraded":"2024-05-22T21:14:50.070709+0000","last_fullsized":"2024-05-22T21:14:50.070709+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:01:04.391319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.077979+0000","last_change":"2024-05-22T21:14:49.681331+0000","last_active":"2024-05-22T21:14:50.077979+0000","last_peered":"2024-05-22T21:14:50.077979+0000","last_clean":"2024-05-22T21:14:50.077979+0000","last_became_active":"2024-05-22T21:14:49.680665+0000","last_became_peered":"2024-05-22T21:14:49.680665+0000","last_unstale":"2024-05-22T21:14:50.077979+0000","last_undegraded":"2024-05-22T21:14:50.077979+0000","last_fullsized":"2024-05-22T21:14:50.077979+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:07:46.043707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.069993+0000","last_change":"2024-05-22T21:14:50.068771+0000","last_active":"2024-05-22T21:14:50.069993+0000","last_peered":"2024-05-22T21:14:50.069993+0000","last_clean":"2024-05-22T21:14:50.069993+0000","last_became_active":"2024-05-22T21:14:50.068014+0000","last_became_peered":"2024-05-22T21:14:50.068014+0000","last_unstale":"2024-05-22T21:14:50.069993+0000","last_undegraded":"2024-05-22T21:14:50.069993+0000","last_fullsized":"2024-05-22T21:14:50.069993+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:34:30.754154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058725+0000","last_change":"2024-05-22T21:14:49.684435+0000","last_active":"2024-05-22T21:14:51.058725+0000","last_peered":"2024-05-22T21:14:51.058725+0000","last_clean":"2024-05-22T21:14:51.058725+0000","last_became_active":"2024-05-22T21:14:49.684142+0000","last_became_peered":"2024-05-22T21:14:49.684142+0000","last_unstale":"2024-05-22T21:14:51.058725+0000","last_undegraded":"2024-05-22T21:14:51.058725+0000","last_fullsized":"2024-05-22T21:14:51.058725+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:07:55.343479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.072081+0000","last_change":"2024-05-22T21:14:50.071555+0000","last_active":"2024-05-22T21:14:50.072081+0000","last_peered":"2024-05-22T21:14:50.072081+0000","last_clean":"2024-05-22T21:14:50.072081+0000","last_became_active":"2024-05-22T21:14:50.071204+0000","last_became_peered":"2024-05-22T21:14:50.071204+0000","last_unstale":"2024-05-22T21:14:50.072081+0000","last_undegraded":"2024-05-22T21:14:50.072081+0000","last_fullsized":"2024-05-22T21:14:50.072081+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:31:49.864012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.080441+0000","last_change":"2024-05-22T21:14:49.677556+0000","last_active":"2024-05-22T21:14:50.080441+0000","last_peered":"2024-05-22T21:14:50.080441+0000","last_clean":"2024-05-22T21:14:50.080441+0000","last_became_active":"2024-05-22T21:14:49.677246+0000","last_became_peered":"2024-05-22T21:14:49.677246+0000","last_unstale":"2024-05-22T21:14:50.080441+0000","last_undegraded":"2024-05-22T21:14:50.080441+0000","last_fullsized":"2024-05-22T21:14:50.080441+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:43:39.196709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.072842+0000","last_change":"2024-05-22T21:14:50.072394+0000","last_active":"2024-05-22T21:14:50.072842+0000","last_peered":"2024-05-22T21:14:50.072842+0000","last_clean":"2024-05-22T21:14:50.072842+0000","last_became_active":"2024-05-22T21:14:50.071970+0000","last_became_peered":"2024-05-22T21:14:50.071970+0000","last_unstale":"2024-05-22T21:14:50.072842+0000","last_undegraded":"2024-05-22T21:14:50.072842+0000","last_fullsized":"2024-05-22T21:14:50.072842+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:34:50.721150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058233+0000","last_change":"2024-05-22T21:14:49.688564+0000","last_active":"2024-05-22T21:14:51.058233+0000","last_peered":"2024-05-22T21:14:51.058233+0000","last_clean":"2024-05-22T21:14:51.058233+0000","last_became_active":"2024-05-22T21:14:49.688183+0000","last_became_peered":"2024-05-22T21:14:49.688183+0000","last_unstale":"2024-05-22T21:14:51.058233+0000","last_undegraded":"2024-05-22T21:14:51.058233+0000","last_fullsized":"2024-05-22T21:14:51.058233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:03:54.015219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057772+0000","last_change":"2024-05-22T21:14:50.067453+0000","last_active":"2024-05-22T21:14:51.057772+0000","last_peered":"2024-05-22T21:14:51.057772+0000","last_clean":"2024-05-22T21:14:51.057772+0000","last_became_active":"2024-05-22T21:14:50.066818+0000","last_became_peered":"2024-05-22T21:14:50.066818+0000","last_unstale":"2024-05-22T21:14:51.057772+0000","last_undegraded":"2024-05-22T21:14:51.057772+0000","last_fullsized":"2024-05-22T21:14:51.057772+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:47:41.860954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057975+0000","last_change":"2024-05-22T21:14:49.688321+0000","last_active":"2024-05-22T21:14:51.057975+0000","last_peered":"2024-05-22T21:14:51.057975+0000","last_clean":"2024-05-22T21:14:51.057975+0000","last_became_active":"2024-05-22T21:14:49.687398+0000","last_became_peered":"2024-05-22T21:14:49.687398+0000","last_unstale":"2024-05-22T21:14:51.057975+0000","last_undegraded":"2024-05-22T21:14:51.057975+0000","last_fullsized":"2024-05-22T21:14:51.057975+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:48:15.025744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.070909+0000","last_change":"2024-05-22T21:14:50.070850+0000","last_active":"2024-05-22T21:14:50.070909+0000","last_peered":"2024-05-22T21:14:50.070909+0000","last_clean":"2024-05-22T21:14:50.070909+0000","last_became_active":"2024-05-22T21:14:50.070627+0000","last_became_peered":"2024-05-22T21:14:50.070627+0000","last_unstale":"2024-05-22T21:14:50.070909+0000","last_undegraded":"2024-05-22T21:14:50.070909+0000","last_fullsized":"2024-05-22T21:14:50.070909+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:35:14.542505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.065596+0000","last_change":"2024-05-22T21:14:49.685733+0000","last_active":"2024-05-22T21:14:50.065596+0000","last_peered":"2024-05-22T21:14:50.065596+0000","last_clean":"2024-05-22T21:14:50.065596+0000","last_became_active":"2024-05-22T21:14:49.685486+0000","last_became_peered":"2024-05-22T21:14:49.685486+0000","last_unstale":"2024-05-22T21:14:50.065596+0000","last_undegraded":"2024-05-22T21:14:50.065596+0000","last_fullsized":"2024-05-22T21:14:50.065596+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:25:40.491524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.066378+0000","last_change":"2024-05-22T21:14:50.065398+0000","last_active":"2024-05-22T21:14:50.066378+0000","last_peered":"2024-05-22T21:14:50.066378+0000","last_clean":"2024-05-22T21:14:50.066378+0000","last_became_active":"2024-05-22T21:14:50.064842+0000","last_became_peered":"2024-05-22T21:14:50.064842+0000","last_unstale":"2024-05-22T21:14:50.066378+0000","last_undegraded":"2024-05-22T21:14:50.066378+0000","last_fullsized":"2024-05-22T21:14:50.066378+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:57:31.660384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.062881+0000","last_change":"2024-05-22T21:14:49.690298+0000","last_active":"2024-05-22T21:14:50.062881+0000","last_peered":"2024-05-22T21:14:50.062881+0000","last_clean":"2024-05-22T21:14:50.062881+0000","last_became_active":"2024-05-22T21:14:49.689029+0000","last_became_peered":"2024-05-22T21:14:49.689029+0000","last_unstale":"2024-05-22T21:14:50.062881+0000","last_undegraded":"2024-05-22T21:14:50.062881+0000","last_fullsized":"2024-05-22T21:14:50.062881+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:43:50.081735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.073805+0000","last_change":"2024-05-22T21:14:50.073734+0000","last_active":"2024-05-22T21:14:50.073805+0000","last_peered":"2024-05-22T21:14:50.073805+0000","last_clean":"2024-05-22T21:14:50.073805+0000","last_became_active":"2024-05-22T21:14:50.072260+0000","last_became_peered":"2024-05-22T21:14:50.072260+0000","last_unstale":"2024-05-22T21:14:50.073805+0000","last_undegraded":"2024-05-22T21:14:50.073805+0000","last_fullsized":"2024-05-22T21:14:50.073805+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:39:03.840693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059536+0000","last_change":"2024-05-22T21:14:49.675747+0000","last_active":"2024-05-22T21:14:51.059536+0000","last_peered":"2024-05-22T21:14:51.059536+0000","last_clean":"2024-05-22T21:14:51.059536+0000","last_became_active":"2024-05-22T21:14:49.675349+0000","last_became_peered":"2024-05-22T21:14:49.675349+0000","last_unstale":"2024-05-22T21:14:51.059536+0000","last_undegraded":"2024-05-22T21:14:51.059536+0000","last_fullsized":"2024-05-22T21:14:51.059536+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:34:49.717555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.074875+0000","last_change":"2024-05-22T21:14:50.074808+0000","last_active":"2024-05-22T21:14:50.074875+0000","last_peered":"2024-05-22T21:14:50.074875+0000","last_clean":"2024-05-22T21:14:50.074875+0000","last_became_active":"2024-05-22T21:14:50.074376+0000","last_became_peered":"2024-05-22T21:14:50.074376+0000","last_unstale":"2024-05-22T21:14:50.074875+0000","last_undegraded":"2024-05-22T21:14:50.074875+0000","last_fullsized":"2024-05-22T21:14:50.074875+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:00:24.489860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058037+0000","last_change":"2024-05-22T21:14:49.682472+0000","last_active":"2024-05-22T21:14:51.058037+0000","last_peered":"2024-05-22T21:14:51.058037+0000","last_clean":"2024-05-22T21:14:51.058037+0000","last_became_active":"2024-05-22T21:14:49.682213+0000","last_became_peered":"2024-05-22T21:14:49.682213+0000","last_unstale":"2024-05-22T21:14:51.058037+0000","last_undegraded":"2024-05-22T21:14:51.058037+0000","last_fullsized":"2024-05-22T21:14:51.058037+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:12:26.709864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.063700+0000","last_change":"2024-05-22T21:14:50.063390+0000","last_active":"2024-05-22T21:14:50.063700+0000","last_peered":"2024-05-22T21:14:50.063700+0000","last_clean":"2024-05-22T21:14:50.063700+0000","last_became_active":"2024-05-22T21:14:50.063183+0000","last_became_peered":"2024-05-22T21:14:50.063183+0000","last_unstale":"2024-05-22T21:14:50.063700+0000","last_undegraded":"2024-05-22T21:14:50.063700+0000","last_fullsized":"2024-05-22T21:14:50.063700+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:48:16.029338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059239+0000","last_change":"2024-05-22T21:14:49.686228+0000","last_active":"2024-05-22T21:14:51.059239+0000","last_peered":"2024-05-22T21:14:51.059239+0000","last_clean":"2024-05-22T21:14:51.059239+0000","last_became_active":"2024-05-22T21:14:49.685982+0000","last_became_peered":"2024-05-22T21:14:49.685982+0000","last_unstale":"2024-05-22T21:14:51.059239+0000","last_undegraded":"2024-05-22T21:14:51.059239+0000","last_fullsized":"2024-05-22T21:14:51.059239+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:50:41.815912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059183+0000","last_change":"2024-05-22T21:14:50.070898+0000","last_active":"2024-05-22T21:14:51.059183+0000","last_peered":"2024-05-22T21:14:51.059183+0000","last_clean":"2024-05-22T21:14:51.059183+0000","last_became_active":"2024-05-22T21:14:50.069847+0000","last_became_peered":"2024-05-22T21:14:50.069847+0000","last_unstale":"2024-05-22T21:14:51.059183+0000","last_undegraded":"2024-05-22T21:14:51.059183+0000","last_fullsized":"2024-05-22T21:14:51.059183+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:48:47.290048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.055971+0000","last_change":"2024-05-22T21:14:49.681175+0000","last_active":"2024-05-22T21:14:50.055971+0000","last_peered":"2024-05-22T21:14:50.055971+0000","last_clean":"2024-05-22T21:14:50.055971+0000","last_became_active":"2024-05-22T21:14:49.679735+0000","last_became_peered":"2024-05-22T21:14:49.679735+0000","last_unstale":"2024-05-22T21:14:50.055971+0000","last_undegraded":"2024-05-22T21:14:50.055971+0000","last_fullsized":"2024-05-22T21:14:50.055971+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:31:12.277202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057831+0000","last_change":"2024-05-22T21:14:50.066398+0000","last_active":"2024-05-22T21:14:51.057831+0000","last_peered":"2024-05-22T21:14:51.057831+0000","last_clean":"2024-05-22T21:14:51.057831+0000","last_became_active":"2024-05-22T21:14:50.066197+0000","last_became_peered":"2024-05-22T21:14:50.066197+0000","last_unstale":"2024-05-22T21:14:51.057831+0000","last_undegraded":"2024-05-22T21:14:51.057831+0000","last_fullsized":"2024-05-22T21:14:51.057831+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:12:33.652523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059607+0000","last_change":"2024-05-22T21:14:49.687483+0000","last_active":"2024-05-22T21:14:51.059607+0000","last_peered":"2024-05-22T21:14:51.059607+0000","last_clean":"2024-05-22T21:14:51.059607+0000","last_became_active":"2024-05-22T21:14:49.687263+0000","last_became_peered":"2024-05-22T21:14:49.687263+0000","last_unstale":"2024-05-22T21:14:51.059607+0000","last_undegraded":"2024-05-22T21:14:51.059607+0000","last_fullsized":"2024-05-22T21:14:51.059607+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:16:45.913899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058190+0000","last_change":"2024-05-22T21:14:50.067728+0000","last_active":"2024-05-22T21:14:51.058190+0000","last_peered":"2024-05-22T21:14:51.058190+0000","last_clean":"2024-05-22T21:14:51.058190+0000","last_became_active":"2024-05-22T21:14:50.067469+0000","last_became_peered":"2024-05-22T21:14:50.067469+0000","last_unstale":"2024-05-22T21:14:51.058190+0000","last_undegraded":"2024-05-22T21:14:51.058190+0000","last_fullsized":"2024-05-22T21:14:51.058190+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:55:16.986795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058805+0000","last_change":"2024-05-22T21:14:49.682200+0000","last_active":"2024-05-22T21:14:51.058805+0000","last_peered":"2024-05-22T21:14:51.058805+0000","last_clean":"2024-05-22T21:14:51.058805+0000","last_became_active":"2024-05-22T21:14:49.681714+0000","last_became_peered":"2024-05-22T21:14:49.681714+0000","last_unstale":"2024-05-22T21:14:51.058805+0000","last_undegraded":"2024-05-22T21:14:51.058805+0000","last_fullsized":"2024-05-22T21:14:51.058805+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:06:35.897212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.073419+0000","last_change":"2024-05-22T21:14:50.072743+0000","last_active":"2024-05-22T21:14:50.073419+0000","last_peered":"2024-05-22T21:14:50.073419+0000","last_clean":"2024-05-22T21:14:50.073419+0000","last_became_active":"2024-05-22T21:14:50.072525+0000","last_became_peered":"2024-05-22T21:14:50.072525+0000","last_unstale":"2024-05-22T21:14:50.073419+0000","last_undegraded":"2024-05-22T21:14:50.073419+0000","last_fullsized":"2024-05-22T21:14:50.073419+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:12:02.664121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.055921+0000","last_change":"2024-05-22T21:14:49.686521+0000","last_active":"2024-05-22T21:14:50.055921+0000","last_peered":"2024-05-22T21:14:50.055921+0000","last_clean":"2024-05-22T21:14:50.055921+0000","last_became_active":"2024-05-22T21:14:49.686302+0000","last_became_peered":"2024-05-22T21:14:49.686302+0000","last_unstale":"2024-05-22T21:14:50.055921+0000","last_undegraded":"2024-05-22T21:14:50.055921+0000","last_fullsized":"2024-05-22T21:14:50.055921+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:36.571938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058518+0000","last_change":"2024-05-22T21:14:50.061488+0000","last_active":"2024-05-22T21:14:51.058518+0000","last_peered":"2024-05-22T21:14:51.058518+0000","last_clean":"2024-05-22T21:14:51.058518+0000","last_became_active":"2024-05-22T21:14:50.061268+0000","last_became_peered":"2024-05-22T21:14:50.061268+0000","last_unstale":"2024-05-22T21:14:51.058518+0000","last_undegraded":"2024-05-22T21:14:51.058518+0000","last_fullsized":"2024-05-22T21:14:51.058518+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:43:40.200304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.059672+0000","last_change":"2024-05-22T21:14:49.690092+0000","last_active":"2024-05-22T21:14:50.059672+0000","last_peered":"2024-05-22T21:14:50.059672+0000","last_clean":"2024-05-22T21:14:50.059672+0000","last_became_active":"2024-05-22T21:14:49.689053+0000","last_became_peered":"2024-05-22T21:14:49.689053+0000","last_unstale":"2024-05-22T21:14:50.059672+0000","last_undegraded":"2024-05-22T21:14:50.059672+0000","last_fullsized":"2024-05-22T21:14:50.059672+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:46.067511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.070028+0000","last_change":"2024-05-22T21:14:50.069612+0000","last_active":"2024-05-22T21:14:50.070028+0000","last_peered":"2024-05-22T21:14:50.070028+0000","last_clean":"2024-05-22T21:14:50.070028+0000","last_became_active":"2024-05-22T21:14:50.069366+0000","last_became_peered":"2024-05-22T21:14:50.069366+0000","last_unstale":"2024-05-22T21:14:50.070028+0000","last_undegraded":"2024-05-22T21:14:50.070028+0000","last_fullsized":"2024-05-22T21:14:50.070028+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:48:47.290048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.080999+0000","last_change":"2024-05-22T21:14:49.679526+0000","last_active":"2024-05-22T21:14:50.080999+0000","last_peered":"2024-05-22T21:14:50.080999+0000","last_clean":"2024-05-22T21:14:50.080999+0000","last_became_active":"2024-05-22T21:14:49.679296+0000","last_became_peered":"2024-05-22T21:14:49.679296+0000","last_unstale":"2024-05-22T21:14:50.080999+0000","last_undegraded":"2024-05-22T21:14:50.080999+0000","last_fullsized":"2024-05-22T21:14:50.080999+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:16:45.913899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.056749+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:14:50.056749+0000","last_peered":"2024-05-22T21:14:50.056749+0000","last_clean":"2024-05-22T21:14:50.056749+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:14:50.056749+0000","last_undegraded":"2024-05-22T21:14:50.056749+0000","last_fullsized":"2024-05-22T21:14:50.056749+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":789,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194522,"statfs":{"total":200000000,"available":199191269,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.434},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.495}]},{"osd":1,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":3.0920000000000001,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":3.1499999999999999,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.40799999999999997}]},{"osd":2,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.55500000000000005}]},{"osd":3,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.64700000000000002}]},{"osd":4,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.60499999999999998}]},{"osd":5,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.67300000000000004}]},{"osd":6,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.70699999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":213,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195098,"statfs":{"total":200000000,"available":199781055,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.90300000000000002}]},{"osd":1,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":1.02},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":1.0029999999999999}]},{"osd":2,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.94199999999999995}]},{"osd":3,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.97899999999999998}]},{"osd":4,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.76600000000000001}]},{"osd":7,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.83399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":213,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195099,"statfs":{"total":200000000,"available":199781657,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.71999999999999997}]},{"osd":1,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.65800000000000003}]},{"osd":2,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.61899999999999999}]},{"osd":4,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68999999999999995}]},{"osd":6,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.70599999999999996}]},{"osd":7,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.67500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":220,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195091,"statfs":{"total":200000000,"available":199773863,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56799999999999995}]},{"osd":1,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.59099999999999997}]},{"osd":2,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.626}]},{"osd":3,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.497},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.60599999999999998}]},{"osd":5,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.43099999999999999}]},{"osd":6,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.47699999999999998}]},{"osd":7,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.57699999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":219,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195092,"statfs":{"total":200000000,"available":199774789,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.33000000000000002}]},{"osd":1,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.35999999999999999}]},{"osd":2,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.70199999999999996}]},{"osd":5,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.81499999999999995}]},{"osd":6,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.629},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.66800000000000004}]},{"osd":7,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.80300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":214,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195097,"statfs":{"total":200000000,"available":199780141,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.501}]},{"osd":1,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.441}]},{"osd":3,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.41399999999999998}]},{"osd":4,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81200000000000006}]},{"osd":5,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.72699999999999998}]},{"osd":6,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.746}]},{"osd":7,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.78100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":213,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195099,"statfs":{"total":200000000,"available":199781657,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.53000000000000003}]},{"osd":2,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.40100000000000002}]},{"osd":3,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.434},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.49099999999999999}]},{"osd":4,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":1.03}]},{"osd":5,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.876}]},{"osd":6,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.0680000000000001}]},{"osd":7,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.0449999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":789,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194523,"statfs":{"total":200000000,"available":199191944,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.437},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.503}]},{"osd":2,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.38400000000000001}]},{"osd":3,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.54300000000000004}]},{"osd":4,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.67400000000000004}]},{"osd":5,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.60499999999999998}]},{"osd":6,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.69199999999999995}]},{"osd":7,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.63900000000000001}]}]}],"pool_statfs":[]}} 2024-05-22T21:14:54.171 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:14:54.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.339+0000 7f2a08200640 1 Processor -- start 2024-05-22T21:14:54.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.339+0000 7f2a08200640 1 -- start start 2024-05-22T21:14:54.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.339+0000 7f2a08200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2a0015dd50 0x7f2a0006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:54.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.339+0000 7f2a08200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a0015d5e0 0x7f2a0015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:54.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.339+0000 7f2a08200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a00071670 con 0x7f2a0015e130 2024-05-22T21:14:54.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.339+0000 7f2a08200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0006de40 con 0x7f2a0015dd50 2024-05-22T21:14:54.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.339+0000 7f2a08200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a00070a60 con 0x7f2a0015d5e0 2024-05-22T21:14:54.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.339+0000 7f2a05800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2a0015dd50 0x7f2a0006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:54.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.339+0000 7f2a05800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2a0015dd50 0x7f2a0006d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:56102/0 (socket says 172.21.15.161:56102) 2024-05-22T21:14:54.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.339+0000 7f2a05800640 1 -- 172.21.15.161:0/1772954775 learned_addr learned my addr 172.21.15.161:0/1772954775 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:54.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.343+0000 7f2a06200640 1 --2- 172.21.15.161:0/1772954775 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a0015d5e0 0x7f2a0015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:54.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.343+0000 7f2a05800640 1 -- 172.21.15.161:0/1772954775 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a0015d5e0 msgr2=0x7f2a0015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:54.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.343+0000 7f2a05800640 1 --2- 172.21.15.161:0/1772954775 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a0015d5e0 0x7f2a0015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.343+0000 7f2a05800640 1 -- 172.21.15.161:0/1772954775 >> v1:172.21.15.179:6789/0 conn(0x7f2a0015e130 legacy=0x7f2a0015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:54.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.343+0000 7f2a05800640 1 -- 172.21.15.161:0/1772954775 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a00070d40 con 0x7f2a0015dd50 2024-05-22T21:14:54.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.343+0000 7f2a05800640 1 --2- 172.21.15.161:0/1772954775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a0015dd50 0x7f2a0006d900 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f29f400bb10 tx=0x7f29f401ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45711a51f32adf88 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:54.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.343+0000 7f29ffe00640 1 -- 172.21.15.161:0/1772954775 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40a7c50 con 0x7f2a0015dd50 2024-05-22T21:14:54.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.343+0000 7f29ffe00640 1 -- 172.21.15.161:0/1772954775 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f40a7db0 con 0x7f2a0015dd50 2024-05-22T21:14:54.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.343+0000 7f29ffe00640 1 -- 172.21.15.161:0/1772954775 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40af660 con 0x7f2a0015dd50 2024-05-22T21:14:54.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.343+0000 7f2a08200640 1 -- 172.21.15.161:0/1772954775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a0015dd50 msgr2=0x7f2a0006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:54.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.343+0000 7f2a08200640 1 --2- 172.21.15.161:0/1772954775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a0015dd50 0x7f2a0006d900 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f29f400bb10 tx=0x7f29f401ebc0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.343+0000 7f2a08200640 1 -- 172.21.15.161:0/1772954775 shutdown_connections 2024-05-22T21:14:54.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.343+0000 7f2a08200640 1 --2- 172.21.15.161:0/1772954775 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a0015d5e0 0x7f2a0015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a08200640 1 --2- 172.21.15.161:0/1772954775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a0015dd50 0x7f2a0006d900 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a08200640 1 -- 172.21.15.161:0/1772954775 >> 172.21.15.161:0/1772954775 conn(0x7f2a000b0bc0 msgr2=0x7f2a000b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:54.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a08200640 1 -- 172.21.15.161:0/1772954775 shutdown_connections 2024-05-22T21:14:54.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a08200640 1 -- 172.21.15.161:0/1772954775 wait complete. 2024-05-22T21:14:54.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a08200640 1 Processor -- start 2024-05-22T21:14:54.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a08200640 1 -- start start 2024-05-22T21:14:54.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a08200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a0015d5e0 0x7f2a001f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:54.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a08200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a0015dd50 0x7f2a000ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:54.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a08200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a0015e130 0x7f2a000acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:54.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a08200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0013b540 con 0x7f2a0015d5e0 2024-05-22T21:14:54.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a08200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a00138660 con 0x7f2a0015e130 2024-05-22T21:14:54.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a08200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a00138940 con 0x7f2a0015dd50 2024-05-22T21:14:54.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a05800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a0015d5e0 0x7f2a001f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:54.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a05800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a0015d5e0 0x7f2a001f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:56108/0 (socket says 172.21.15.161:56108) 2024-05-22T21:14:54.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a05800640 1 -- 172.21.15.161:0/3311324238 learned_addr learned my addr 172.21.15.161:0/3311324238 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:54.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a06200640 1 --2- 172.21.15.161:0/3311324238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a0015e130 0x7f2a000acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:54.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a04e00640 1 --2- 172.21.15.161:0/3311324238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a0015dd50 0x7f2a000ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:54.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a05800640 1 -- 172.21.15.161:0/3311324238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a0015dd50 msgr2=0x7f2a000ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:54.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a05800640 1 --2- 172.21.15.161:0/3311324238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a0015dd50 0x7f2a000ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a05800640 1 -- 172.21.15.161:0/3311324238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a0015e130 msgr2=0x7f2a000acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:54.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a05800640 1 --2- 172.21.15.161:0/3311324238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a0015e130 0x7f2a000acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a05800640 1 -- 172.21.15.161:0/3311324238 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f400b770 con 0x7f2a0015d5e0 2024-05-22T21:14:54.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a04e00640 1 --2- 172.21.15.161:0/3311324238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a0015dd50 0x7f2a000ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:54.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.347+0000 7f2a05800640 1 --2- 172.21.15.161:0/3311324238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a0015d5e0 0x7f2a001f51c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f29f400b2c0 tx=0x7f29f401b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:54.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.351+0000 7f29fd600640 1 -- 172.21.15.161:0/3311324238 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40ba660 con 0x7f2a0015d5e0 2024-05-22T21:14:54.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.351+0000 7f29fd600640 1 -- 172.21.15.161:0/3311324238 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f40ba7c0 con 0x7f2a0015d5e0 2024-05-22T21:14:54.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.351+0000 7f2a08200640 1 -- 172.21.15.161:0/3311324238 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a00138c20 con 0x7f2a0015d5e0 2024-05-22T21:14:54.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.351+0000 7f29fd600640 1 -- 172.21.15.161:0/3311324238 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40bab70 con 0x7f2a0015d5e0 2024-05-22T21:14:54.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.351+0000 7f2a08200640 1 -- 172.21.15.161:0/3311324238 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a001f5700 con 0x7f2a0015d5e0 2024-05-22T21:14:54.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.351+0000 7f2a08200640 1 -- 172.21.15.161:0/3311324238 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a00067220 con 0x7f2a0015d5e0 2024-05-22T21:14:54.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.351+0000 7f29fd600640 1 -- 172.21.15.161:0/3311324238 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f29f40c33b0 con 0x7f2a0015d5e0 2024-05-22T21:14:54.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.351+0000 7f29fd600640 1 --2- 172.21.15.161:0/3311324238 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f29e00666c0 0x7f29e0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:54.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.351+0000 7f29fd600640 1 -- 172.21.15.161:0/3311324238 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29f411f1f0 con 0x7f2a0015d5e0 2024-05-22T21:14:54.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.351+0000 7f2a04e00640 1 --2- 172.21.15.161:0/3311324238 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f29e00666c0 0x7f29e0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:54.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.355+0000 7f2a04e00640 1 --2- 172.21.15.161:0/3311324238 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f29e00666c0 0x7f29e0068b80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2a000ad500 tx=0x7f29ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:54.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.355+0000 7f29fd600640 1 -- 172.21.15.161:0/3311324238 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f29f40ec6a0 con 0x7f2a0015d5e0 2024-05-22T21:14:54.513 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.507+0000 7f2a08200640 1 -- 172.21.15.161:0/3311324238 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a001387d0 con 0x7f29e00666c0 2024-05-22T21:14:54.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.515+0000 7f29fd600640 1 -- 172.21.15.161:0/3311324238 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284455 (secure 0 0 0) 0x7f2a001387d0 con 0x7f29e00666c0 2024-05-22T21:14:54.526 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:54.528 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:14:54.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.519+0000 7f2a08200640 1 -- 172.21.15.161:0/3311324238 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f29e00666c0 msgr2=0x7f29e0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:54.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.519+0000 7f2a08200640 1 --2- 172.21.15.161:0/3311324238 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f29e00666c0 0x7f29e0068b80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2a000ad500 tx=0x7f29ec008040 comp rx=0 tx=0).stop 2024-05-22T21:14:54.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.519+0000 7f2a08200640 1 -- 172.21.15.161:0/3311324238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a0015d5e0 msgr2=0x7f2a001f51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:54.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.519+0000 7f2a08200640 1 --2- 172.21.15.161:0/3311324238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a0015d5e0 0x7f2a001f51c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f29f400b2c0 tx=0x7f29f401b960 comp rx=0 tx=0).stop 2024-05-22T21:14:54.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.519+0000 7f2a08200640 1 -- 172.21.15.161:0/3311324238 shutdown_connections 2024-05-22T21:14:54.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.519+0000 7f2a08200640 1 --2- 172.21.15.161:0/3311324238 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f29e00666c0 0x7f29e0068b80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.519+0000 7f2a08200640 1 --2- 172.21.15.161:0/3311324238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a0015e130 0x7f2a000acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.519+0000 7f2a08200640 1 --2- 172.21.15.161:0/3311324238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a0015dd50 0x7f2a000ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.519+0000 7f2a08200640 1 --2- 172.21.15.161:0/3311324238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a0015d5e0 0x7f2a001f51c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.519+0000 7f2a08200640 1 -- 172.21.15.161:0/3311324238 >> 172.21.15.161:0/3311324238 conn(0x7f2a000b0bc0 msgr2=0x7f2a000afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:54.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.519+0000 7f2a08200640 1 -- 172.21.15.161:0/3311324238 shutdown_connections 2024-05-22T21:14:54.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.519+0000 7f2a08200640 1 -- 172.21.15.161:0/3311324238 wait complete. 2024-05-22T21:14:54.543 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-05-22T21:14:54.412297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2874,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559625,"statfs":{"total":1600000000,"available":1597056375,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.743176"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059198+0000","last_change":"2024-05-22T21:14:50.061770+0000","last_active":"2024-05-22T21:14:51.059198+0000","last_peered":"2024-05-22T21:14:51.059198+0000","last_clean":"2024-05-22T21:14:51.059198+0000","last_became_active":"2024-05-22T21:14:50.061152+0000","last_became_peered":"2024-05-22T21:14:50.061152+0000","last_unstale":"2024-05-22T21:14:51.059198+0000","last_undegraded":"2024-05-22T21:14:51.059198+0000","last_fullsized":"2024-05-22T21:14:51.059198+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:40:22.552291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.076346+0000","last_change":"2024-05-22T21:14:50.076275+0000","last_active":"2024-05-22T21:14:50.076346+0000","last_peered":"2024-05-22T21:14:50.076346+0000","last_clean":"2024-05-22T21:14:50.076346+0000","last_became_active":"2024-05-22T21:14:50.076061+0000","last_became_peered":"2024-05-22T21:14:50.076061+0000","last_unstale":"2024-05-22T21:14:50.076346+0000","last_undegraded":"2024-05-22T21:14:50.076346+0000","last_fullsized":"2024-05-22T21:14:50.076346+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:32:09.786594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058410+0000","last_change":"2024-05-22T21:14:50.066837+0000","last_active":"2024-05-22T21:14:51.058410+0000","last_peered":"2024-05-22T21:14:51.058410+0000","last_clean":"2024-05-22T21:14:51.058410+0000","last_became_active":"2024-05-22T21:14:50.066583+0000","last_became_peered":"2024-05-22T21:14:50.066583+0000","last_unstale":"2024-05-22T21:14:51.058410+0000","last_undegraded":"2024-05-22T21:14:51.058410+0000","last_fullsized":"2024-05-22T21:14:51.058410+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:12:27.713458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.076076+0000","last_change":"2024-05-22T21:14:50.075848+0000","last_active":"2024-05-22T21:14:50.076076+0000","last_peered":"2024-05-22T21:14:50.076076+0000","last_clean":"2024-05-22T21:14:50.076076+0000","last_became_active":"2024-05-22T21:14:50.075598+0000","last_became_peered":"2024-05-22T21:14:50.075598+0000","last_unstale":"2024-05-22T21:14:50.076076+0000","last_undegraded":"2024-05-22T21:14:50.076076+0000","last_fullsized":"2024-05-22T21:14:50.076076+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:12:27.713458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.077171+0000","last_change":"2024-05-22T21:14:50.076846+0000","last_active":"2024-05-22T21:14:50.077171+0000","last_peered":"2024-05-22T21:14:50.077171+0000","last_clean":"2024-05-22T21:14:50.077171+0000","last_became_active":"2024-05-22T21:14:50.076643+0000","last_became_peered":"2024-05-22T21:14:50.076643+0000","last_unstale":"2024-05-22T21:14:50.077171+0000","last_undegraded":"2024-05-22T21:14:50.077171+0000","last_fullsized":"2024-05-22T21:14:50.077171+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:36:31.712862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057952+0000","last_change":"2024-05-22T21:14:50.064710+0000","last_active":"2024-05-22T21:14:51.057952+0000","last_peered":"2024-05-22T21:14:51.057952+0000","last_clean":"2024-05-22T21:14:51.057952+0000","last_became_active":"2024-05-22T21:14:50.064441+0000","last_became_peered":"2024-05-22T21:14:50.064441+0000","last_unstale":"2024-05-22T21:14:51.057952+0000","last_undegraded":"2024-05-22T21:14:51.057952+0000","last_fullsized":"2024-05-22T21:14:51.057952+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:36:53.357957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058503+0000","last_change":"2024-05-22T21:14:50.064004+0000","last_active":"2024-05-22T21:14:51.058503+0000","last_peered":"2024-05-22T21:14:51.058503+0000","last_clean":"2024-05-22T21:14:51.058503+0000","last_became_active":"2024-05-22T21:14:50.063751+0000","last_became_peered":"2024-05-22T21:14:50.063751+0000","last_unstale":"2024-05-22T21:14:51.058503+0000","last_undegraded":"2024-05-22T21:14:51.058503+0000","last_fullsized":"2024-05-22T21:14:51.058503+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:39:03.840693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.073771+0000","last_change":"2024-05-22T21:14:50.073646+0000","last_active":"2024-05-22T21:14:50.073771+0000","last_peered":"2024-05-22T21:14:50.073771+0000","last_clean":"2024-05-22T21:14:50.073771+0000","last_became_active":"2024-05-22T21:14:50.073415+0000","last_became_peered":"2024-05-22T21:14:50.073415+0000","last_unstale":"2024-05-22T21:14:50.073771+0000","last_undegraded":"2024-05-22T21:14:50.073771+0000","last_fullsized":"2024-05-22T21:14:50.073771+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:43:40.200304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.069977+0000","last_change":"2024-05-22T21:14:50.068582+0000","last_active":"2024-05-22T21:14:50.069977+0000","last_peered":"2024-05-22T21:14:50.069977+0000","last_clean":"2024-05-22T21:14:50.069977+0000","last_became_active":"2024-05-22T21:14:50.067719+0000","last_became_peered":"2024-05-22T21:14:50.067719+0000","last_unstale":"2024-05-22T21:14:50.069977+0000","last_undegraded":"2024-05-22T21:14:50.069977+0000","last_fullsized":"2024-05-22T21:14:50.069977+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:40:22.552291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.074193+0000","last_change":"2024-05-22T21:14:50.074044+0000","last_active":"2024-05-22T21:14:50.074193+0000","last_peered":"2024-05-22T21:14:50.074193+0000","last_clean":"2024-05-22T21:14:50.074193+0000","last_became_active":"2024-05-22T21:14:50.072779+0000","last_became_peered":"2024-05-22T21:14:50.072779+0000","last_unstale":"2024-05-22T21:14:50.074193+0000","last_undegraded":"2024-05-22T21:14:50.074193+0000","last_fullsized":"2024-05-22T21:14:50.074193+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:36:31.712862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.072208+0000","last_change":"2024-05-22T21:14:50.072132+0000","last_active":"2024-05-22T21:14:50.072208+0000","last_peered":"2024-05-22T21:14:50.072208+0000","last_clean":"2024-05-22T21:14:50.072208+0000","last_became_active":"2024-05-22T21:14:50.071230+0000","last_became_peered":"2024-05-22T21:14:50.071230+0000","last_unstale":"2024-05-22T21:14:50.072208+0000","last_undegraded":"2024-05-22T21:14:50.072208+0000","last_fullsized":"2024-05-22T21:14:50.072208+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:33:20.372548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.070491+0000","last_change":"2024-05-22T21:14:50.070398+0000","last_active":"2024-05-22T21:14:50.070491+0000","last_peered":"2024-05-22T21:14:50.070491+0000","last_clean":"2024-05-22T21:14:50.070491+0000","last_became_active":"2024-05-22T21:14:50.070181+0000","last_became_peered":"2024-05-22T21:14:50.070181+0000","last_unstale":"2024-05-22T21:14:50.070491+0000","last_undegraded":"2024-05-22T21:14:50.070491+0000","last_fullsized":"2024-05-22T21:14:50.070491+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:39:03.840693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.066439+0000","last_change":"2024-05-22T21:14:50.066296+0000","last_active":"2024-05-22T21:14:50.066439+0000","last_peered":"2024-05-22T21:14:50.066439+0000","last_clean":"2024-05-22T21:14:50.066439+0000","last_became_active":"2024-05-22T21:14:50.065988+0000","last_became_peered":"2024-05-22T21:14:50.065988+0000","last_unstale":"2024-05-22T21:14:50.066439+0000","last_undegraded":"2024-05-22T21:14:50.066439+0000","last_fullsized":"2024-05-22T21:14:50.066439+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:17:20.539737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.070013+0000","last_change":"2024-05-22T21:14:50.069252+0000","last_active":"2024-05-22T21:14:50.070013+0000","last_peered":"2024-05-22T21:14:50.070013+0000","last_clean":"2024-05-22T21:14:50.070013+0000","last_became_active":"2024-05-22T21:14:50.068378+0000","last_became_peered":"2024-05-22T21:14:50.068378+0000","last_unstale":"2024-05-22T21:14:50.070013+0000","last_undegraded":"2024-05-22T21:14:50.070013+0000","last_fullsized":"2024-05-22T21:14:50.070013+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:37:51.679058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057855+0000","last_change":"2024-05-22T21:14:50.065207+0000","last_active":"2024-05-22T21:14:51.057855+0000","last_peered":"2024-05-22T21:14:51.057855+0000","last_clean":"2024-05-22T21:14:51.057855+0000","last_became_active":"2024-05-22T21:14:50.065013+0000","last_became_peered":"2024-05-22T21:14:50.065013+0000","last_unstale":"2024-05-22T21:14:51.057855+0000","last_undegraded":"2024-05-22T21:14:51.057855+0000","last_fullsized":"2024-05-22T21:14:51.057855+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:58:07.918046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.076690+0000","last_change":"2024-05-22T21:14:50.076578+0000","last_active":"2024-05-22T21:14:50.076690+0000","last_peered":"2024-05-22T21:14:50.076690+0000","last_clean":"2024-05-22T21:14:50.076690+0000","last_became_active":"2024-05-22T21:14:50.076214+0000","last_became_peered":"2024-05-22T21:14:50.076214+0000","last_unstale":"2024-05-22T21:14:50.076690+0000","last_undegraded":"2024-05-22T21:14:50.076690+0000","last_fullsized":"2024-05-22T21:14:50.076690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:15:52.936539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058493+0000","last_change":"2024-05-22T21:14:50.073211+0000","last_active":"2024-05-22T21:14:51.058493+0000","last_peered":"2024-05-22T21:14:51.058493+0000","last_clean":"2024-05-22T21:14:51.058493+0000","last_became_active":"2024-05-22T21:14:50.072982+0000","last_became_peered":"2024-05-22T21:14:50.072982+0000","last_unstale":"2024-05-22T21:14:51.058493+0000","last_undegraded":"2024-05-22T21:14:51.058493+0000","last_fullsized":"2024-05-22T21:14:51.058493+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:00:23.932047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058633+0000","last_change":"2024-05-22T21:14:50.071899+0000","last_active":"2024-05-22T21:14:51.058633+0000","last_peered":"2024-05-22T21:14:51.058633+0000","last_clean":"2024-05-22T21:14:51.058633+0000","last_became_active":"2024-05-22T21:14:50.068603+0000","last_became_peered":"2024-05-22T21:14:50.068603+0000","last_unstale":"2024-05-22T21:14:51.058633+0000","last_undegraded":"2024-05-22T21:14:51.058633+0000","last_fullsized":"2024-05-22T21:14:51.058633+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:03:55.018814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.079305+0000","last_change":"2024-05-22T21:14:50.079199+0000","last_active":"2024-05-22T21:14:50.079305+0000","last_peered":"2024-05-22T21:14:50.079305+0000","last_clean":"2024-05-22T21:14:50.079305+0000","last_became_active":"2024-05-22T21:14:50.078542+0000","last_became_peered":"2024-05-22T21:14:50.078542+0000","last_unstale":"2024-05-22T21:14:50.079305+0000","last_undegraded":"2024-05-22T21:14:50.079305+0000","last_fullsized":"2024-05-22T21:14:50.079305+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:50:42.819506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.066354+0000","last_change":"2024-05-22T21:14:50.065235+0000","last_active":"2024-05-22T21:14:50.066354+0000","last_peered":"2024-05-22T21:14:50.066354+0000","last_clean":"2024-05-22T21:14:50.066354+0000","last_became_active":"2024-05-22T21:14:50.064988+0000","last_became_peered":"2024-05-22T21:14:50.064988+0000","last_unstale":"2024-05-22T21:14:50.066354+0000","last_undegraded":"2024-05-22T21:14:50.066354+0000","last_fullsized":"2024-05-22T21:14:50.066354+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:43:51.085330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058035+0000","last_change":"2024-05-22T21:14:50.071100+0000","last_active":"2024-05-22T21:14:51.058035+0000","last_peered":"2024-05-22T21:14:51.058035+0000","last_clean":"2024-05-22T21:14:51.058035+0000","last_became_active":"2024-05-22T21:14:50.069813+0000","last_became_peered":"2024-05-22T21:14:50.069813+0000","last_unstale":"2024-05-22T21:14:51.058035+0000","last_undegraded":"2024-05-22T21:14:51.058035+0000","last_fullsized":"2024-05-22T21:14:51.058035+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:16:44.963153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.074672+0000","last_change":"2024-05-22T21:14:50.074482+0000","last_active":"2024-05-22T21:14:50.074672+0000","last_peered":"2024-05-22T21:14:50.074672+0000","last_clean":"2024-05-22T21:14:50.074672+0000","last_became_active":"2024-05-22T21:14:50.074279+0000","last_became_peered":"2024-05-22T21:14:50.074279+0000","last_unstale":"2024-05-22T21:14:50.074672+0000","last_undegraded":"2024-05-22T21:14:50.074672+0000","last_fullsized":"2024-05-22T21:14:50.074672+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:16:46.917494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.078403+0000","last_change":"2024-05-22T21:14:50.078289+0000","last_active":"2024-05-22T21:14:50.078403+0000","last_peered":"2024-05-22T21:14:50.078403+0000","last_clean":"2024-05-22T21:14:50.078403+0000","last_became_active":"2024-05-22T21:14:50.077892+0000","last_became_peered":"2024-05-22T21:14:50.077892+0000","last_unstale":"2024-05-22T21:14:50.078403+0000","last_undegraded":"2024-05-22T21:14:50.078403+0000","last_fullsized":"2024-05-22T21:14:50.078403+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:04:01.163527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.079288+0000","last_change":"2024-05-22T21:14:50.079071+0000","last_active":"2024-05-22T21:14:50.079288+0000","last_peered":"2024-05-22T21:14:50.079288+0000","last_clean":"2024-05-22T21:14:50.079288+0000","last_became_active":"2024-05-22T21:14:50.078196+0000","last_became_peered":"2024-05-22T21:14:50.078196+0000","last_unstale":"2024-05-22T21:14:50.079288+0000","last_undegraded":"2024-05-22T21:14:50.079288+0000","last_fullsized":"2024-05-22T21:14:50.079288+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:43:40.200304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.069560+0000","last_change":"2024-05-22T21:14:50.069171+0000","last_active":"2024-05-22T21:14:50.069560+0000","last_peered":"2024-05-22T21:14:50.069560+0000","last_clean":"2024-05-22T21:14:50.069560+0000","last_became_active":"2024-05-22T21:14:50.068937+0000","last_became_peered":"2024-05-22T21:14:50.068937+0000","last_unstale":"2024-05-22T21:14:50.069560+0000","last_undegraded":"2024-05-22T21:14:50.069560+0000","last_fullsized":"2024-05-22T21:14:50.069560+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:41:40.374906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.066640+0000","last_change":"2024-05-22T21:14:50.066534+0000","last_active":"2024-05-22T21:14:50.066640+0000","last_peered":"2024-05-22T21:14:50.066640+0000","last_clean":"2024-05-22T21:14:50.066640+0000","last_became_active":"2024-05-22T21:14:50.066036+0000","last_became_peered":"2024-05-22T21:14:50.066036+0000","last_unstale":"2024-05-22T21:14:50.066640+0000","last_undegraded":"2024-05-22T21:14:50.066640+0000","last_fullsized":"2024-05-22T21:14:50.066640+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:05:06.217169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058601+0000","last_change":"2024-05-22T21:14:50.075608+0000","last_active":"2024-05-22T21:14:51.058601+0000","last_peered":"2024-05-22T21:14:51.058601+0000","last_clean":"2024-05-22T21:14:51.058601+0000","last_became_active":"2024-05-22T21:14:50.075321+0000","last_became_peered":"2024-05-22T21:14:50.075321+0000","last_unstale":"2024-05-22T21:14:51.058601+0000","last_undegraded":"2024-05-22T21:14:51.058601+0000","last_fullsized":"2024-05-22T21:14:51.058601+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:15:52.936539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.071918+0000","last_change":"2024-05-22T21:14:50.071845+0000","last_active":"2024-05-22T21:14:50.071918+0000","last_peered":"2024-05-22T21:14:50.071918+0000","last_clean":"2024-05-22T21:14:50.071918+0000","last_became_active":"2024-05-22T21:14:50.071610+0000","last_became_peered":"2024-05-22T21:14:50.071610+0000","last_unstale":"2024-05-22T21:14:50.071918+0000","last_undegraded":"2024-05-22T21:14:50.071918+0000","last_fullsized":"2024-05-22T21:14:50.071918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:04:47.495150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.071290+0000","last_change":"2024-05-22T21:14:50.071126+0000","last_active":"2024-05-22T21:14:50.071290+0000","last_peered":"2024-05-22T21:14:50.071290+0000","last_clean":"2024-05-22T21:14:50.071290+0000","last_became_active":"2024-05-22T21:14:50.070856+0000","last_became_peered":"2024-05-22T21:14:50.070856+0000","last_unstale":"2024-05-22T21:14:50.071290+0000","last_undegraded":"2024-05-22T21:14:50.071290+0000","last_fullsized":"2024-05-22T21:14:50.071290+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:35:14.542505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058619+0000","last_change":"2024-05-22T21:14:50.062910+0000","last_active":"2024-05-22T21:14:51.058619+0000","last_peered":"2024-05-22T21:14:51.058619+0000","last_clean":"2024-05-22T21:14:51.058619+0000","last_became_active":"2024-05-22T21:14:50.060665+0000","last_became_peered":"2024-05-22T21:14:50.060665+0000","last_unstale":"2024-05-22T21:14:51.058619+0000","last_undegraded":"2024-05-22T21:14:51.058619+0000","last_fullsized":"2024-05-22T21:14:51.058619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:16:46.917494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058115+0000","last_change":"2024-05-22T21:14:50.071233+0000","last_active":"2024-05-22T21:14:51.058115+0000","last_peered":"2024-05-22T21:14:51.058115+0000","last_clean":"2024-05-22T21:14:51.058115+0000","last_became_active":"2024-05-22T21:14:50.070888+0000","last_became_peered":"2024-05-22T21:14:50.070888+0000","last_unstale":"2024-05-22T21:14:51.058115+0000","last_undegraded":"2024-05-22T21:14:51.058115+0000","last_fullsized":"2024-05-22T21:14:51.058115+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:46:38.590939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.063955+0000","last_change":"2024-05-22T21:14:50.063893+0000","last_active":"2024-05-22T21:14:50.063955+0000","last_peered":"2024-05-22T21:14:50.063955+0000","last_clean":"2024-05-22T21:14:50.063955+0000","last_became_active":"2024-05-22T21:14:50.063677+0000","last_became_peered":"2024-05-22T21:14:50.063677+0000","last_unstale":"2024-05-22T21:14:50.063955+0000","last_undegraded":"2024-05-22T21:14:50.063955+0000","last_fullsized":"2024-05-22T21:14:50.063955+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:45:06.407521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.063955+0000","last_change":"2024-05-22T21:14:50.063423+0000","last_active":"2024-05-22T21:14:50.063955+0000","last_peered":"2024-05-22T21:14:50.063955+0000","last_clean":"2024-05-22T21:14:50.063955+0000","last_became_active":"2024-05-22T21:14:50.063222+0000","last_became_peered":"2024-05-22T21:14:50.063222+0000","last_unstale":"2024-05-22T21:14:50.063955+0000","last_undegraded":"2024-05-22T21:14:50.063955+0000","last_fullsized":"2024-05-22T21:14:50.063955+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:16:46.917494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057819+0000","last_change":"2024-05-22T21:14:49.688612+0000","last_active":"2024-05-22T21:14:51.057819+0000","last_peered":"2024-05-22T21:14:51.057819+0000","last_clean":"2024-05-22T21:14:51.057819+0000","last_became_active":"2024-05-22T21:14:49.688041+0000","last_became_peered":"2024-05-22T21:14:49.688041+0000","last_unstale":"2024-05-22T21:14:51.057819+0000","last_undegraded":"2024-05-22T21:14:51.057819+0000","last_fullsized":"2024-05-22T21:14:51.057819+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:33:19.368954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.078179+0000","last_change":"2024-05-22T21:14:50.078095+0000","last_active":"2024-05-22T21:14:50.078179+0000","last_peered":"2024-05-22T21:14:50.078179+0000","last_clean":"2024-05-22T21:14:50.078179+0000","last_became_active":"2024-05-22T21:14:50.077160+0000","last_became_peered":"2024-05-22T21:14:50.077160+0000","last_unstale":"2024-05-22T21:14:50.078179+0000","last_undegraded":"2024-05-22T21:14:50.078179+0000","last_fullsized":"2024-05-22T21:14:50.078179+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:31:13.280796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058226+0000","last_change":"2024-05-22T21:14:49.686130+0000","last_active":"2024-05-22T21:14:51.058226+0000","last_peered":"2024-05-22T21:14:51.058226+0000","last_clean":"2024-05-22T21:14:51.058226+0000","last_became_active":"2024-05-22T21:14:49.685900+0000","last_became_peered":"2024-05-22T21:14:49.685900+0000","last_unstale":"2024-05-22T21:14:51.058226+0000","last_undegraded":"2024-05-22T21:14:51.058226+0000","last_fullsized":"2024-05-22T21:14:51.058226+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:46:32.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.071184+0000","last_change":"2024-05-22T21:14:50.070940+0000","last_active":"2024-05-22T21:14:50.071184+0000","last_peered":"2024-05-22T21:14:50.071184+0000","last_clean":"2024-05-22T21:14:50.071184+0000","last_became_active":"2024-05-22T21:14:50.070733+0000","last_became_peered":"2024-05-22T21:14:50.070733+0000","last_unstale":"2024-05-22T21:14:50.071184+0000","last_undegraded":"2024-05-22T21:14:50.071184+0000","last_fullsized":"2024-05-22T21:14:50.071184+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:08:05.259519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.081689+0000","last_change":"2024-05-22T21:14:49.680411+0000","last_active":"2024-05-22T21:14:50.081689+0000","last_peered":"2024-05-22T21:14:50.081689+0000","last_clean":"2024-05-22T21:14:50.081689+0000","last_became_active":"2024-05-22T21:14:49.679389+0000","last_became_peered":"2024-05-22T21:14:49.679389+0000","last_unstale":"2024-05-22T21:14:50.081689+0000","last_undegraded":"2024-05-22T21:14:50.081689+0000","last_fullsized":"2024-05-22T21:14:50.081689+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:15:51.932944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.069816+0000","last_change":"2024-05-22T21:14:50.069771+0000","last_active":"2024-05-22T21:14:50.069816+0000","last_peered":"2024-05-22T21:14:50.069816+0000","last_clean":"2024-05-22T21:14:50.069816+0000","last_became_active":"2024-05-22T21:14:50.069567+0000","last_became_peered":"2024-05-22T21:14:50.069567+0000","last_unstale":"2024-05-22T21:14:50.069816+0000","last_undegraded":"2024-05-22T21:14:50.069816+0000","last_fullsized":"2024-05-22T21:14:50.069816+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:50:42.819506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059382+0000","last_change":"2024-05-22T21:14:49.681679+0000","last_active":"2024-05-22T21:14:51.059382+0000","last_peered":"2024-05-22T21:14:51.059382+0000","last_clean":"2024-05-22T21:14:51.059382+0000","last_became_active":"2024-05-22T21:14:49.681446+0000","last_became_peered":"2024-05-22T21:14:49.681446+0000","last_unstale":"2024-05-22T21:14:51.059382+0000","last_undegraded":"2024-05-22T21:14:51.059382+0000","last_fullsized":"2024-05-22T21:14:51.059382+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:25:40.491524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058307+0000","last_change":"2024-05-22T21:14:50.075123+0000","last_active":"2024-05-22T21:14:51.058307+0000","last_peered":"2024-05-22T21:14:51.058307+0000","last_clean":"2024-05-22T21:14:51.058307+0000","last_became_active":"2024-05-22T21:14:50.074477+0000","last_became_peered":"2024-05-22T21:14:50.074477+0000","last_unstale":"2024-05-22T21:14:51.058307+0000","last_undegraded":"2024-05-22T21:14:51.058307+0000","last_fullsized":"2024-05-22T21:14:51.058307+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:25:41.495118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058061+0000","last_change":"2024-05-22T21:14:49.688420+0000","last_active":"2024-05-22T21:14:51.058061+0000","last_peered":"2024-05-22T21:14:51.058061+0000","last_clean":"2024-05-22T21:14:51.058061+0000","last_became_active":"2024-05-22T21:14:49.687817+0000","last_became_peered":"2024-05-22T21:14:49.687817+0000","last_unstale":"2024-05-22T21:14:51.058061+0000","last_undegraded":"2024-05-22T21:14:51.058061+0000","last_fullsized":"2024-05-22T21:14:51.058061+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:55:08.681604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.063680+0000","last_change":"2024-05-22T21:14:50.062829+0000","last_active":"2024-05-22T21:14:50.063680+0000","last_peered":"2024-05-22T21:14:50.063680+0000","last_clean":"2024-05-22T21:14:50.063680+0000","last_became_active":"2024-05-22T21:14:50.062609+0000","last_became_peered":"2024-05-22T21:14:50.062609+0000","last_unstale":"2024-05-22T21:14:50.063680+0000","last_undegraded":"2024-05-22T21:14:50.063680+0000","last_fullsized":"2024-05-22T21:14:50.063680+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:55:09.685198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058743+0000","last_change":"2024-05-22T21:14:49.683862+0000","last_active":"2024-05-22T21:14:51.058743+0000","last_peered":"2024-05-22T21:14:51.058743+0000","last_clean":"2024-05-22T21:14:51.058743+0000","last_became_active":"2024-05-22T21:14:49.683259+0000","last_became_peered":"2024-05-22T21:14:49.683259+0000","last_unstale":"2024-05-22T21:14:51.058743+0000","last_undegraded":"2024-05-22T21:14:51.058743+0000","last_fullsized":"2024-05-22T21:14:51.058743+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:46.067511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.073945+0000","last_change":"2024-05-22T21:14:50.073887+0000","last_active":"2024-05-22T21:14:50.073945+0000","last_peered":"2024-05-22T21:14:50.073945+0000","last_clean":"2024-05-22T21:14:50.073945+0000","last_became_active":"2024-05-22T21:14:50.072506+0000","last_became_peered":"2024-05-22T21:14:50.072506+0000","last_unstale":"2024-05-22T21:14:50.073945+0000","last_undegraded":"2024-05-22T21:14:50.073945+0000","last_fullsized":"2024-05-22T21:14:50.073945+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:08:05.259519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.065478+0000","last_change":"2024-05-22T21:14:49.676438+0000","last_active":"2024-05-22T21:14:50.065478+0000","last_peered":"2024-05-22T21:14:50.065478+0000","last_clean":"2024-05-22T21:14:50.065478+0000","last_became_active":"2024-05-22T21:14:49.676185+0000","last_became_peered":"2024-05-22T21:14:49.676185+0000","last_unstale":"2024-05-22T21:14:50.065478+0000","last_undegraded":"2024-05-22T21:14:50.065478+0000","last_fullsized":"2024-05-22T21:14:50.065478+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:12:26.709864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.079271+0000","last_change":"2024-05-22T21:14:50.078911+0000","last_active":"2024-05-22T21:14:50.079271+0000","last_peered":"2024-05-22T21:14:50.079271+0000","last_clean":"2024-05-22T21:14:50.079271+0000","last_became_active":"2024-05-22T21:14:50.077790+0000","last_became_peered":"2024-05-22T21:14:50.077790+0000","last_unstale":"2024-05-22T21:14:50.079271+0000","last_undegraded":"2024-05-22T21:14:50.079271+0000","last_fullsized":"2024-05-22T21:14:50.079271+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:16:46.917494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.062793+0000","last_change":"2024-05-22T21:14:49.690446+0000","last_active":"2024-05-22T21:14:50.062793+0000","last_peered":"2024-05-22T21:14:50.062793+0000","last_clean":"2024-05-22T21:14:50.062793+0000","last_became_active":"2024-05-22T21:14:49.689486+0000","last_became_peered":"2024-05-22T21:14:49.689486+0000","last_unstale":"2024-05-22T21:14:50.062793+0000","last_undegraded":"2024-05-22T21:14:50.062793+0000","last_fullsized":"2024-05-22T21:14:50.062793+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:06:35.897212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058712+0000","last_change":"2024-05-22T21:14:50.068317+0000","last_active":"2024-05-22T21:14:51.058712+0000","last_peered":"2024-05-22T21:14:51.058712+0000","last_clean":"2024-05-22T21:14:51.058712+0000","last_became_active":"2024-05-22T21:14:50.068098+0000","last_became_peered":"2024-05-22T21:14:50.068098+0000","last_unstale":"2024-05-22T21:14:51.058712+0000","last_undegraded":"2024-05-22T21:14:51.058712+0000","last_fullsized":"2024-05-22T21:14:51.058712+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:35:14.542505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058830+0000","last_change":"2024-05-22T21:14:49.685613+0000","last_active":"2024-05-22T21:14:51.058830+0000","last_peered":"2024-05-22T21:14:51.058830+0000","last_clean":"2024-05-22T21:14:51.058830+0000","last_became_active":"2024-05-22T21:14:49.685239+0000","last_became_peered":"2024-05-22T21:14:49.685239+0000","last_unstale":"2024-05-22T21:14:51.058830+0000","last_undegraded":"2024-05-22T21:14:51.058830+0000","last_fullsized":"2024-05-22T21:14:51.058830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:43:50.081735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059294+0000","last_change":"2024-05-22T21:14:50.073200+0000","last_active":"2024-05-22T21:14:51.059294+0000","last_peered":"2024-05-22T21:14:51.059294+0000","last_clean":"2024-05-22T21:14:51.059294+0000","last_became_active":"2024-05-22T21:14:50.072238+0000","last_became_peered":"2024-05-22T21:14:50.072238+0000","last_unstale":"2024-05-22T21:14:51.059294+0000","last_undegraded":"2024-05-22T21:14:51.059294+0000","last_fullsized":"2024-05-22T21:14:51.059294+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:08:05.259519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.062042+0000","last_change":"2024-05-22T21:14:49.687112+0000","last_active":"2024-05-22T21:14:50.062042+0000","last_peered":"2024-05-22T21:14:50.062042+0000","last_clean":"2024-05-22T21:14:50.062042+0000","last_became_active":"2024-05-22T21:14:49.686857+0000","last_became_peered":"2024-05-22T21:14:49.686857+0000","last_unstale":"2024-05-22T21:14:50.062042+0000","last_undegraded":"2024-05-22T21:14:50.062042+0000","last_fullsized":"2024-05-22T21:14:50.062042+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:17:19.536142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057859+0000","last_change":"2024-05-22T21:14:50.067586+0000","last_active":"2024-05-22T21:14:51.057859+0000","last_peered":"2024-05-22T21:14:51.057859+0000","last_clean":"2024-05-22T21:14:51.057859+0000","last_became_active":"2024-05-22T21:14:50.067098+0000","last_became_peered":"2024-05-22T21:14:50.067098+0000","last_unstale":"2024-05-22T21:14:51.057859+0000","last_undegraded":"2024-05-22T21:14:51.057859+0000","last_fullsized":"2024-05-22T21:14:51.057859+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:40:51.810150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059042+0000","last_change":"2024-05-22T21:14:49.685326+0000","last_active":"2024-05-22T21:14:51.059042+0000","last_peered":"2024-05-22T21:14:51.059042+0000","last_clean":"2024-05-22T21:14:51.059042+0000","last_became_active":"2024-05-22T21:14:49.685105+0000","last_became_peered":"2024-05-22T21:14:49.685105+0000","last_unstale":"2024-05-22T21:14:51.059042+0000","last_undegraded":"2024-05-22T21:14:51.059042+0000","last_fullsized":"2024-05-22T21:14:51.059042+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:35:13.538910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058919+0000","last_change":"2024-05-22T21:14:50.071196+0000","last_active":"2024-05-22T21:14:51.058919+0000","last_peered":"2024-05-22T21:14:51.058919+0000","last_clean":"2024-05-22T21:14:51.058919+0000","last_became_active":"2024-05-22T21:14:50.070169+0000","last_became_peered":"2024-05-22T21:14:50.070169+0000","last_unstale":"2024-05-22T21:14:51.058919+0000","last_undegraded":"2024-05-22T21:14:51.058919+0000","last_fullsized":"2024-05-22T21:14:51.058919+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:04:47.495150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.082149+0000","last_change":"2024-05-22T21:14:49.680428+0000","last_active":"2024-05-22T21:14:50.082149+0000","last_peered":"2024-05-22T21:14:50.082149+0000","last_clean":"2024-05-22T21:14:50.082149+0000","last_became_active":"2024-05-22T21:14:49.680131+0000","last_became_peered":"2024-05-22T21:14:49.680131+0000","last_unstale":"2024-05-22T21:14:50.082149+0000","last_undegraded":"2024-05-22T21:14:50.082149+0000","last_fullsized":"2024-05-22T21:14:50.082149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:56:44.859254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.076673+0000","last_change":"2024-05-22T21:14:50.076430+0000","last_active":"2024-05-22T21:14:50.076673+0000","last_peered":"2024-05-22T21:14:50.076673+0000","last_clean":"2024-05-22T21:14:50.076673+0000","last_became_active":"2024-05-22T21:14:50.075586+0000","last_became_peered":"2024-05-22T21:14:50.075586+0000","last_unstale":"2024-05-22T21:14:50.076673+0000","last_undegraded":"2024-05-22T21:14:50.076673+0000","last_fullsized":"2024-05-22T21:14:50.076673+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:37.575533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057962+0000","last_change":"2024-05-22T21:14:49.685529+0000","last_active":"2024-05-22T21:14:51.057962+0000","last_peered":"2024-05-22T21:14:51.057962+0000","last_clean":"2024-05-22T21:14:51.057962+0000","last_became_active":"2024-05-22T21:14:49.685294+0000","last_became_peered":"2024-05-22T21:14:49.685294+0000","last_unstale":"2024-05-22T21:14:51.057962+0000","last_undegraded":"2024-05-22T21:14:51.057962+0000","last_fullsized":"2024-05-22T21:14:51.057962+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:34:29.750559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.073563+0000","last_change":"2024-05-22T21:14:50.073433+0000","last_active":"2024-05-22T21:14:50.073563+0000","last_peered":"2024-05-22T21:14:50.073563+0000","last_clean":"2024-05-22T21:14:50.073563+0000","last_became_active":"2024-05-22T21:14:50.071865+0000","last_became_peered":"2024-05-22T21:14:50.071865+0000","last_unstale":"2024-05-22T21:14:50.073563+0000","last_undegraded":"2024-05-22T21:14:50.073563+0000","last_fullsized":"2024-05-22T21:14:50.073563+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:04:47.495150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059464+0000","last_change":"2024-05-22T21:14:49.675587+0000","last_active":"2024-05-22T21:14:51.059464+0000","last_peered":"2024-05-22T21:14:51.059464+0000","last_clean":"2024-05-22T21:14:51.059464+0000","last_became_active":"2024-05-22T21:14:49.675019+0000","last_became_peered":"2024-05-22T21:14:49.675019+0000","last_unstale":"2024-05-22T21:14:51.059464+0000","last_undegraded":"2024-05-22T21:14:51.059464+0000","last_fullsized":"2024-05-22T21:14:51.059464+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:43:39.196709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057723+0000","last_change":"2024-05-22T21:14:50.068120+0000","last_active":"2024-05-22T21:14:51.057723+0000","last_peered":"2024-05-22T21:14:51.057723+0000","last_clean":"2024-05-22T21:14:51.057723+0000","last_became_active":"2024-05-22T21:14:50.067920+0000","last_became_peered":"2024-05-22T21:14:50.067920+0000","last_unstale":"2024-05-22T21:14:51.057723+0000","last_undegraded":"2024-05-22T21:14:51.057723+0000","last_fullsized":"2024-05-22T21:14:51.057723+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:18:08.972692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058154+0000","last_change":"2024-05-22T21:14:49.685539+0000","last_active":"2024-05-22T21:14:51.058154+0000","last_peered":"2024-05-22T21:14:51.058154+0000","last_clean":"2024-05-22T21:14:51.058154+0000","last_became_active":"2024-05-22T21:14:49.685245+0000","last_became_peered":"2024-05-22T21:14:49.685245+0000","last_unstale":"2024-05-22T21:14:51.058154+0000","last_undegraded":"2024-05-22T21:14:51.058154+0000","last_fullsized":"2024-05-22T21:14:51.058154+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:48:46.286454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058595+0000","last_change":"2024-05-22T21:14:50.070568+0000","last_active":"2024-05-22T21:14:51.058595+0000","last_peered":"2024-05-22T21:14:51.058595+0000","last_clean":"2024-05-22T21:14:51.058595+0000","last_became_active":"2024-05-22T21:14:50.070268+0000","last_became_peered":"2024-05-22T21:14:50.070268+0000","last_unstale":"2024-05-22T21:14:51.058595+0000","last_undegraded":"2024-05-22T21:14:51.058595+0000","last_fullsized":"2024-05-22T21:14:51.058595+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:36:31.712862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.082657+0000","last_change":"2024-05-22T21:14:49.681995+0000","last_active":"2024-05-22T21:14:50.082657+0000","last_peered":"2024-05-22T21:14:50.082657+0000","last_clean":"2024-05-22T21:14:50.082657+0000","last_became_active":"2024-05-22T21:14:49.681696+0000","last_became_peered":"2024-05-22T21:14:49.681696+0000","last_unstale":"2024-05-22T21:14:50.082657+0000","last_undegraded":"2024-05-22T21:14:50.082657+0000","last_fullsized":"2024-05-22T21:14:50.082657+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:46:32.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.070709+0000","last_change":"2024-05-22T21:14:50.070588+0000","last_active":"2024-05-22T21:14:50.070709+0000","last_peered":"2024-05-22T21:14:50.070709+0000","last_clean":"2024-05-22T21:14:50.070709+0000","last_became_active":"2024-05-22T21:14:50.070386+0000","last_became_peered":"2024-05-22T21:14:50.070386+0000","last_unstale":"2024-05-22T21:14:50.070709+0000","last_undegraded":"2024-05-22T21:14:50.070709+0000","last_fullsized":"2024-05-22T21:14:50.070709+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:01:04.391319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.077979+0000","last_change":"2024-05-22T21:14:49.681331+0000","last_active":"2024-05-22T21:14:50.077979+0000","last_peered":"2024-05-22T21:14:50.077979+0000","last_clean":"2024-05-22T21:14:50.077979+0000","last_became_active":"2024-05-22T21:14:49.680665+0000","last_became_peered":"2024-05-22T21:14:49.680665+0000","last_unstale":"2024-05-22T21:14:50.077979+0000","last_undegraded":"2024-05-22T21:14:50.077979+0000","last_fullsized":"2024-05-22T21:14:50.077979+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:07:46.043707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.069993+0000","last_change":"2024-05-22T21:14:50.068771+0000","last_active":"2024-05-22T21:14:50.069993+0000","last_peered":"2024-05-22T21:14:50.069993+0000","last_clean":"2024-05-22T21:14:50.069993+0000","last_became_active":"2024-05-22T21:14:50.068014+0000","last_became_peered":"2024-05-22T21:14:50.068014+0000","last_unstale":"2024-05-22T21:14:50.069993+0000","last_undegraded":"2024-05-22T21:14:50.069993+0000","last_fullsized":"2024-05-22T21:14:50.069993+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:34:30.754154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058725+0000","last_change":"2024-05-22T21:14:49.684435+0000","last_active":"2024-05-22T21:14:51.058725+0000","last_peered":"2024-05-22T21:14:51.058725+0000","last_clean":"2024-05-22T21:14:51.058725+0000","last_became_active":"2024-05-22T21:14:49.684142+0000","last_became_peered":"2024-05-22T21:14:49.684142+0000","last_unstale":"2024-05-22T21:14:51.058725+0000","last_undegraded":"2024-05-22T21:14:51.058725+0000","last_fullsized":"2024-05-22T21:14:51.058725+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:07:55.343479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.072081+0000","last_change":"2024-05-22T21:14:50.071555+0000","last_active":"2024-05-22T21:14:50.072081+0000","last_peered":"2024-05-22T21:14:50.072081+0000","last_clean":"2024-05-22T21:14:50.072081+0000","last_became_active":"2024-05-22T21:14:50.071204+0000","last_became_peered":"2024-05-22T21:14:50.071204+0000","last_unstale":"2024-05-22T21:14:50.072081+0000","last_undegraded":"2024-05-22T21:14:50.072081+0000","last_fullsized":"2024-05-22T21:14:50.072081+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:31:49.864012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.080441+0000","last_change":"2024-05-22T21:14:49.677556+0000","last_active":"2024-05-22T21:14:50.080441+0000","last_peered":"2024-05-22T21:14:50.080441+0000","last_clean":"2024-05-22T21:14:50.080441+0000","last_became_active":"2024-05-22T21:14:49.677246+0000","last_became_peered":"2024-05-22T21:14:49.677246+0000","last_unstale":"2024-05-22T21:14:50.080441+0000","last_undegraded":"2024-05-22T21:14:50.080441+0000","last_fullsized":"2024-05-22T21:14:50.080441+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:43:39.196709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.072842+0000","last_change":"2024-05-22T21:14:50.072394+0000","last_active":"2024-05-22T21:14:50.072842+0000","last_peered":"2024-05-22T21:14:50.072842+0000","last_clean":"2024-05-22T21:14:50.072842+0000","last_became_active":"2024-05-22T21:14:50.071970+0000","last_became_peered":"2024-05-22T21:14:50.071970+0000","last_unstale":"2024-05-22T21:14:50.072842+0000","last_undegraded":"2024-05-22T21:14:50.072842+0000","last_fullsized":"2024-05-22T21:14:50.072842+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:34:50.721150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058233+0000","last_change":"2024-05-22T21:14:49.688564+0000","last_active":"2024-05-22T21:14:51.058233+0000","last_peered":"2024-05-22T21:14:51.058233+0000","last_clean":"2024-05-22T21:14:51.058233+0000","last_became_active":"2024-05-22T21:14:49.688183+0000","last_became_peered":"2024-05-22T21:14:49.688183+0000","last_unstale":"2024-05-22T21:14:51.058233+0000","last_undegraded":"2024-05-22T21:14:51.058233+0000","last_fullsized":"2024-05-22T21:14:51.058233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:03:54.015219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057772+0000","last_change":"2024-05-22T21:14:50.067453+0000","last_active":"2024-05-22T21:14:51.057772+0000","last_peered":"2024-05-22T21:14:51.057772+0000","last_clean":"2024-05-22T21:14:51.057772+0000","last_became_active":"2024-05-22T21:14:50.066818+0000","last_became_peered":"2024-05-22T21:14:50.066818+0000","last_unstale":"2024-05-22T21:14:51.057772+0000","last_undegraded":"2024-05-22T21:14:51.057772+0000","last_fullsized":"2024-05-22T21:14:51.057772+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:47:41.860954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057975+0000","last_change":"2024-05-22T21:14:49.688321+0000","last_active":"2024-05-22T21:14:51.057975+0000","last_peered":"2024-05-22T21:14:51.057975+0000","last_clean":"2024-05-22T21:14:51.057975+0000","last_became_active":"2024-05-22T21:14:49.687398+0000","last_became_peered":"2024-05-22T21:14:49.687398+0000","last_unstale":"2024-05-22T21:14:51.057975+0000","last_undegraded":"2024-05-22T21:14:51.057975+0000","last_fullsized":"2024-05-22T21:14:51.057975+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:48:15.025744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.070909+0000","last_change":"2024-05-22T21:14:50.070850+0000","last_active":"2024-05-22T21:14:50.070909+0000","last_peered":"2024-05-22T21:14:50.070909+0000","last_clean":"2024-05-22T21:14:50.070909+0000","last_became_active":"2024-05-22T21:14:50.070627+0000","last_became_peered":"2024-05-22T21:14:50.070627+0000","last_unstale":"2024-05-22T21:14:50.070909+0000","last_undegraded":"2024-05-22T21:14:50.070909+0000","last_fullsized":"2024-05-22T21:14:50.070909+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:35:14.542505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.065596+0000","last_change":"2024-05-22T21:14:49.685733+0000","last_active":"2024-05-22T21:14:50.065596+0000","last_peered":"2024-05-22T21:14:50.065596+0000","last_clean":"2024-05-22T21:14:50.065596+0000","last_became_active":"2024-05-22T21:14:49.685486+0000","last_became_peered":"2024-05-22T21:14:49.685486+0000","last_unstale":"2024-05-22T21:14:50.065596+0000","last_undegraded":"2024-05-22T21:14:50.065596+0000","last_fullsized":"2024-05-22T21:14:50.065596+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:25:40.491524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.066378+0000","last_change":"2024-05-22T21:14:50.065398+0000","last_active":"2024-05-22T21:14:50.066378+0000","last_peered":"2024-05-22T21:14:50.066378+0000","last_clean":"2024-05-22T21:14:50.066378+0000","last_became_active":"2024-05-22T21:14:50.064842+0000","last_became_peered":"2024-05-22T21:14:50.064842+0000","last_unstale":"2024-05-22T21:14:50.066378+0000","last_undegraded":"2024-05-22T21:14:50.066378+0000","last_fullsized":"2024-05-22T21:14:50.066378+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:57:31.660384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.062881+0000","last_change":"2024-05-22T21:14:49.690298+0000","last_active":"2024-05-22T21:14:50.062881+0000","last_peered":"2024-05-22T21:14:50.062881+0000","last_clean":"2024-05-22T21:14:50.062881+0000","last_became_active":"2024-05-22T21:14:49.689029+0000","last_became_peered":"2024-05-22T21:14:49.689029+0000","last_unstale":"2024-05-22T21:14:50.062881+0000","last_undegraded":"2024-05-22T21:14:50.062881+0000","last_fullsized":"2024-05-22T21:14:50.062881+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:43:50.081735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.073805+0000","last_change":"2024-05-22T21:14:50.073734+0000","last_active":"2024-05-22T21:14:50.073805+0000","last_peered":"2024-05-22T21:14:50.073805+0000","last_clean":"2024-05-22T21:14:50.073805+0000","last_became_active":"2024-05-22T21:14:50.072260+0000","last_became_peered":"2024-05-22T21:14:50.072260+0000","last_unstale":"2024-05-22T21:14:50.073805+0000","last_undegraded":"2024-05-22T21:14:50.073805+0000","last_fullsized":"2024-05-22T21:14:50.073805+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:39:03.840693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059536+0000","last_change":"2024-05-22T21:14:49.675747+0000","last_active":"2024-05-22T21:14:51.059536+0000","last_peered":"2024-05-22T21:14:51.059536+0000","last_clean":"2024-05-22T21:14:51.059536+0000","last_became_active":"2024-05-22T21:14:49.675349+0000","last_became_peered":"2024-05-22T21:14:49.675349+0000","last_unstale":"2024-05-22T21:14:51.059536+0000","last_undegraded":"2024-05-22T21:14:51.059536+0000","last_fullsized":"2024-05-22T21:14:51.059536+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:34:49.717555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.074875+0000","last_change":"2024-05-22T21:14:50.074808+0000","last_active":"2024-05-22T21:14:50.074875+0000","last_peered":"2024-05-22T21:14:50.074875+0000","last_clean":"2024-05-22T21:14:50.074875+0000","last_became_active":"2024-05-22T21:14:50.074376+0000","last_became_peered":"2024-05-22T21:14:50.074376+0000","last_unstale":"2024-05-22T21:14:50.074875+0000","last_undegraded":"2024-05-22T21:14:50.074875+0000","last_fullsized":"2024-05-22T21:14:50.074875+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:00:24.489860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058037+0000","last_change":"2024-05-22T21:14:49.682472+0000","last_active":"2024-05-22T21:14:51.058037+0000","last_peered":"2024-05-22T21:14:51.058037+0000","last_clean":"2024-05-22T21:14:51.058037+0000","last_became_active":"2024-05-22T21:14:49.682213+0000","last_became_peered":"2024-05-22T21:14:49.682213+0000","last_unstale":"2024-05-22T21:14:51.058037+0000","last_undegraded":"2024-05-22T21:14:51.058037+0000","last_fullsized":"2024-05-22T21:14:51.058037+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:12:26.709864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.063700+0000","last_change":"2024-05-22T21:14:50.063390+0000","last_active":"2024-05-22T21:14:50.063700+0000","last_peered":"2024-05-22T21:14:50.063700+0000","last_clean":"2024-05-22T21:14:50.063700+0000","last_became_active":"2024-05-22T21:14:50.063183+0000","last_became_peered":"2024-05-22T21:14:50.063183+0000","last_unstale":"2024-05-22T21:14:50.063700+0000","last_undegraded":"2024-05-22T21:14:50.063700+0000","last_fullsized":"2024-05-22T21:14:50.063700+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:48:16.029338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059239+0000","last_change":"2024-05-22T21:14:49.686228+0000","last_active":"2024-05-22T21:14:51.059239+0000","last_peered":"2024-05-22T21:14:51.059239+0000","last_clean":"2024-05-22T21:14:51.059239+0000","last_became_active":"2024-05-22T21:14:49.685982+0000","last_became_peered":"2024-05-22T21:14:49.685982+0000","last_unstale":"2024-05-22T21:14:51.059239+0000","last_undegraded":"2024-05-22T21:14:51.059239+0000","last_fullsized":"2024-05-22T21:14:51.059239+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:50:41.815912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059183+0000","last_change":"2024-05-22T21:14:50.070898+0000","last_active":"2024-05-22T21:14:51.059183+0000","last_peered":"2024-05-22T21:14:51.059183+0000","last_clean":"2024-05-22T21:14:51.059183+0000","last_became_active":"2024-05-22T21:14:50.069847+0000","last_became_peered":"2024-05-22T21:14:50.069847+0000","last_unstale":"2024-05-22T21:14:51.059183+0000","last_undegraded":"2024-05-22T21:14:51.059183+0000","last_fullsized":"2024-05-22T21:14:51.059183+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:48:47.290048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.055971+0000","last_change":"2024-05-22T21:14:49.681175+0000","last_active":"2024-05-22T21:14:50.055971+0000","last_peered":"2024-05-22T21:14:50.055971+0000","last_clean":"2024-05-22T21:14:50.055971+0000","last_became_active":"2024-05-22T21:14:49.679735+0000","last_became_peered":"2024-05-22T21:14:49.679735+0000","last_unstale":"2024-05-22T21:14:50.055971+0000","last_undegraded":"2024-05-22T21:14:50.055971+0000","last_fullsized":"2024-05-22T21:14:50.055971+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:31:12.277202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.057831+0000","last_change":"2024-05-22T21:14:50.066398+0000","last_active":"2024-05-22T21:14:51.057831+0000","last_peered":"2024-05-22T21:14:51.057831+0000","last_clean":"2024-05-22T21:14:51.057831+0000","last_became_active":"2024-05-22T21:14:50.066197+0000","last_became_peered":"2024-05-22T21:14:50.066197+0000","last_unstale":"2024-05-22T21:14:51.057831+0000","last_undegraded":"2024-05-22T21:14:51.057831+0000","last_fullsized":"2024-05-22T21:14:51.057831+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:12:33.652523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.059607+0000","last_change":"2024-05-22T21:14:49.687483+0000","last_active":"2024-05-22T21:14:51.059607+0000","last_peered":"2024-05-22T21:14:51.059607+0000","last_clean":"2024-05-22T21:14:51.059607+0000","last_became_active":"2024-05-22T21:14:49.687263+0000","last_became_peered":"2024-05-22T21:14:49.687263+0000","last_unstale":"2024-05-22T21:14:51.059607+0000","last_undegraded":"2024-05-22T21:14:51.059607+0000","last_fullsized":"2024-05-22T21:14:51.059607+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:16:45.913899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058190+0000","last_change":"2024-05-22T21:14:50.067728+0000","last_active":"2024-05-22T21:14:51.058190+0000","last_peered":"2024-05-22T21:14:51.058190+0000","last_clean":"2024-05-22T21:14:51.058190+0000","last_became_active":"2024-05-22T21:14:50.067469+0000","last_became_peered":"2024-05-22T21:14:50.067469+0000","last_unstale":"2024-05-22T21:14:51.058190+0000","last_undegraded":"2024-05-22T21:14:51.058190+0000","last_fullsized":"2024-05-22T21:14:51.058190+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:55:16.986795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058805+0000","last_change":"2024-05-22T21:14:49.682200+0000","last_active":"2024-05-22T21:14:51.058805+0000","last_peered":"2024-05-22T21:14:51.058805+0000","last_clean":"2024-05-22T21:14:51.058805+0000","last_became_active":"2024-05-22T21:14:49.681714+0000","last_became_peered":"2024-05-22T21:14:49.681714+0000","last_unstale":"2024-05-22T21:14:51.058805+0000","last_undegraded":"2024-05-22T21:14:51.058805+0000","last_fullsized":"2024-05-22T21:14:51.058805+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:06:35.897212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.073419+0000","last_change":"2024-05-22T21:14:50.072743+0000","last_active":"2024-05-22T21:14:50.073419+0000","last_peered":"2024-05-22T21:14:50.073419+0000","last_clean":"2024-05-22T21:14:50.073419+0000","last_became_active":"2024-05-22T21:14:50.072525+0000","last_became_peered":"2024-05-22T21:14:50.072525+0000","last_unstale":"2024-05-22T21:14:50.073419+0000","last_undegraded":"2024-05-22T21:14:50.073419+0000","last_fullsized":"2024-05-22T21:14:50.073419+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:12:02.664121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.055921+0000","last_change":"2024-05-22T21:14:49.686521+0000","last_active":"2024-05-22T21:14:50.055921+0000","last_peered":"2024-05-22T21:14:50.055921+0000","last_clean":"2024-05-22T21:14:50.055921+0000","last_became_active":"2024-05-22T21:14:49.686302+0000","last_became_peered":"2024-05-22T21:14:49.686302+0000","last_unstale":"2024-05-22T21:14:50.055921+0000","last_undegraded":"2024-05-22T21:14:50.055921+0000","last_fullsized":"2024-05-22T21:14:50.055921+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:36.571938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-22T21:14:51.058518+0000","last_change":"2024-05-22T21:14:50.061488+0000","last_active":"2024-05-22T21:14:51.058518+0000","last_peered":"2024-05-22T21:14:51.058518+0000","last_clean":"2024-05-22T21:14:51.058518+0000","last_became_active":"2024-05-22T21:14:50.061268+0000","last_became_peered":"2024-05-22T21:14:50.061268+0000","last_unstale":"2024-05-22T21:14:51.058518+0000","last_undegraded":"2024-05-22T21:14:51.058518+0000","last_fullsized":"2024-05-22T21:14:51.058518+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:43:40.200304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.059672+0000","last_change":"2024-05-22T21:14:49.690092+0000","last_active":"2024-05-22T21:14:50.059672+0000","last_peered":"2024-05-22T21:14:50.059672+0000","last_clean":"2024-05-22T21:14:50.059672+0000","last_became_active":"2024-05-22T21:14:49.689053+0000","last_became_peered":"2024-05-22T21:14:49.689053+0000","last_unstale":"2024-05-22T21:14:50.059672+0000","last_undegraded":"2024-05-22T21:14:50.059672+0000","last_fullsized":"2024-05-22T21:14:50.059672+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:46.067511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.070028+0000","last_change":"2024-05-22T21:14:50.069612+0000","last_active":"2024-05-22T21:14:50.070028+0000","last_peered":"2024-05-22T21:14:50.070028+0000","last_clean":"2024-05-22T21:14:50.070028+0000","last_became_active":"2024-05-22T21:14:50.069366+0000","last_became_peered":"2024-05-22T21:14:50.069366+0000","last_unstale":"2024-05-22T21:14:50.070028+0000","last_undegraded":"2024-05-22T21:14:50.070028+0000","last_fullsized":"2024-05-22T21:14:50.070028+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:49.666504+0000","last_clean_scrub_stamp":"2024-05-22T21:14:49.666504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:48:47.290048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.080999+0000","last_change":"2024-05-22T21:14:49.679526+0000","last_active":"2024-05-22T21:14:50.080999+0000","last_peered":"2024-05-22T21:14:50.080999+0000","last_clean":"2024-05-22T21:14:50.080999+0000","last_became_active":"2024-05-22T21:14:49.679296+0000","last_became_peered":"2024-05-22T21:14:49.679296+0000","last_unstale":"2024-05-22T21:14:50.080999+0000","last_undegraded":"2024-05-22T21:14:50.080999+0000","last_fullsized":"2024-05-22T21:14:50.080999+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:14:48.662909+0000","last_clean_scrub_stamp":"2024-05-22T21:14:48.662909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:16:45.913899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-22T21:14:50.056749+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:14:50.056749+0000","last_peered":"2024-05-22T21:14:50.056749+0000","last_clean":"2024-05-22T21:14:50.056749+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:14:50.056749+0000","last_undegraded":"2024-05-22T21:14:50.056749+0000","last_fullsized":"2024-05-22T21:14:50.056749+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":789,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194522,"statfs":{"total":200000000,"available":199191269,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.434},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.495}]},{"osd":1,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":3.0920000000000001,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":3.1499999999999999,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.40799999999999997}]},{"osd":2,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.55500000000000005}]},{"osd":3,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.64700000000000002}]},{"osd":4,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.60499999999999998}]},{"osd":5,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.67300000000000004}]},{"osd":6,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.70699999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":213,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195098,"statfs":{"total":200000000,"available":199781055,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.90300000000000002}]},{"osd":1,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":1.02},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":1.0029999999999999}]},{"osd":2,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.94199999999999995}]},{"osd":3,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.97899999999999998}]},{"osd":4,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.76600000000000001}]},{"osd":7,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.83399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":213,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195099,"statfs":{"total":200000000,"available":199781657,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.71999999999999997}]},{"osd":1,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.65800000000000003}]},{"osd":2,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.61899999999999999}]},{"osd":4,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68999999999999995}]},{"osd":6,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.70599999999999996}]},{"osd":7,"last update":"Wed May 22 21:14:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.67500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":220,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195091,"statfs":{"total":200000000,"available":199773863,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56799999999999995}]},{"osd":1,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.59099999999999997}]},{"osd":2,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.626}]},{"osd":3,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.497},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.60599999999999998}]},{"osd":5,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.43099999999999999}]},{"osd":6,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.47699999999999998}]},{"osd":7,"last update":"Wed May 22 21:14:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.57699999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":219,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195092,"statfs":{"total":200000000,"available":199774789,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.33000000000000002}]},{"osd":1,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.35999999999999999}]},{"osd":2,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.40300000000000002}]},{"osd":4,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.70199999999999996}]},{"osd":5,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.81499999999999995}]},{"osd":6,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.629},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.66800000000000004}]},{"osd":7,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.80300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":214,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195097,"statfs":{"total":200000000,"available":199780141,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.501}]},{"osd":1,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.441}]},{"osd":3,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.41399999999999998}]},{"osd":4,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81200000000000006}]},{"osd":5,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.72699999999999998}]},{"osd":6,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.746}]},{"osd":7,"last update":"Wed May 22 21:14:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.78100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":213,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195099,"statfs":{"total":200000000,"available":199781657,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.53000000000000003}]},{"osd":2,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.40100000000000002}]},{"osd":3,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.434},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.49099999999999999}]},{"osd":4,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":1.03}]},{"osd":5,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.876}]},{"osd":6,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.0680000000000001}]},{"osd":7,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.0449999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":789,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194523,"statfs":{"total":200000000,"available":199191944,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.437},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.503}]},{"osd":2,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.38400000000000001}]},{"osd":3,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.54300000000000004}]},{"osd":4,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.67400000000000004}]},{"osd":5,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.60499999999999998}]},{"osd":6,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.69199999999999995}]},{"osd":7,"last update":"Wed May 22 21:14:11 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.63900000000000001}]}]}],"pool_statfs":[]}} 2024-05-22T21:14:54.548 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-22T21:14:54.548 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-22T21:14:54.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7f7c00640 1 Processor -- start 2024-05-22T21:14:54.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7f7c00640 1 -- start start 2024-05-22T21:14:54.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7f7c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb7f01402b0 0x7fb7f0070590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:54.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7f7c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7f006c750 0x7fb7f00716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:54.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7f7c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f006cbc0 con 0x7fb7f006c2a0 2024-05-22T21:14:54.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7f7c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f015d7c0 con 0x7fb7f01402b0 2024-05-22T21:14:54.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7f7c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f015d9d0 con 0x7fb7f006c750 2024-05-22T21:14:54.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7e7e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb7f006c2a0 0x7fb7f0070f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:57758/0 (socket says 172.21.15.161:57758) 2024-05-22T21:14:54.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7e7e00640 1 -- 172.21.15.161:0/3748176389 learned_addr learned my addr 172.21.15.161:0/3748176389 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:54.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7f5200640 1 --2- 172.21.15.161:0/3748176389 >> v2:172.21.15.179:3300/0 conn(0x7fb7f01402b0 0x7fb7f0070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:54.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7f5c00640 1 --2- 172.21.15.161:0/3748176389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7f006c750 0x7fb7f00716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:54.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7efe00640 1 -- 172.21.15.161:0/3748176389 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029241540 0 0) 0x7fb7f006cbc0 con 0x7fb7f006c2a0 2024-05-22T21:14:54.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7efe00640 1 -- 172.21.15.161:0/3748176389 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7d4003530 con 0x7fb7f006c2a0 2024-05-22T21:14:54.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7efe00640 1 -- 172.21.15.161:0/3748176389 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2603169944 0 0) 0x7fb7d4003530 con 0x7fb7f006c2a0 2024-05-22T21:14:54.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7efe00640 1 -- 172.21.15.161:0/3748176389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7f006c750 msgr2=0x7fb7f00716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:54.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7efe00640 1 --2- 172.21.15.161:0/3748176389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7f006c750 0x7fb7f00716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7efe00640 1 -- 172.21.15.161:0/3748176389 >> v2:172.21.15.179:3300/0 conn(0x7fb7f01402b0 msgr2=0x7fb7f0070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:54.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7efe00640 1 --2- 172.21.15.161:0/3748176389 >> v2:172.21.15.179:3300/0 conn(0x7fb7f01402b0 0x7fb7f0070590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7efe00640 1 -- 172.21.15.161:0/3748176389 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f015dcb0 con 0x7fb7f006c2a0 2024-05-22T21:14:54.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7efe00640 1 -- 172.21.15.161:0/3748176389 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fb7d80016a0 con 0x7fb7f006c2a0 2024-05-22T21:14:54.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7efe00640 1 -- 172.21.15.161:0/3748176389 >> v1:172.21.15.161:6789/0 conn(0x7fb7f006c2a0 legacy=0x7fb7f0070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:54.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7efe00640 1 --2- 172.21.15.161:0/3748176389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7d4003980 0x7fb7d4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:54.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7efe00640 1 --2- 172.21.15.161:0/3748176389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7f006c750 0x7fb7d40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:54.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7efe00640 1 --2- 172.21.15.161:0/3748176389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d4004d10 0x7fb7d4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:54.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7efe00640 1 -- 172.21.15.161:0/3748176389 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f015d7c0 con 0x7fb7d4004d10 2024-05-22T21:14:54.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7efe00640 1 -- 172.21.15.161:0/3748176389 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f015d9d0 con 0x7fb7d4003980 2024-05-22T21:14:54.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7efe00640 1 -- 172.21.15.161:0/3748176389 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d400b990 con 0x7fb7f006c750 2024-05-22T21:14:54.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7f5c00640 1 --2- 172.21.15.161:0/3748176389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d4004d10 0x7fb7d4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:54.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7f5200640 1 --2- 172.21.15.161:0/3748176389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7d4003980 0x7fb7d4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:54.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7e7e00640 1 --2- 172.21.15.161:0/3748176389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7f006c750 0x7fb7d40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:54.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.727+0000 7fb7f5c00640 1 -- 172.21.15.161:0/3748176389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7f006c750 msgr2=0x7fb7d40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:54.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.731+0000 7fb7f5c00640 1 --2- 172.21.15.161:0/3748176389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7f006c750 0x7fb7d40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.731+0000 7fb7f5c00640 1 -- 172.21.15.161:0/3748176389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7d4003980 msgr2=0x7fb7d4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:54.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.731+0000 7fb7f5c00640 1 --2- 172.21.15.161:0/3748176389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7d4003980 0x7fb7d4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.731+0000 7fb7f5c00640 1 -- 172.21.15.161:0/3748176389 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d4003530 con 0x7fb7d4004d10 2024-05-22T21:14:54.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.731+0000 7fb7f5c00640 1 --2- 172.21.15.161:0/3748176389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d4004d10 0x7fb7d4005190 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800e170 tx=0x7fb7e801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d09125c0916f72 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:54.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.731+0000 7fb7efe00640 1 -- 172.21.15.161:0/3748176389 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e80a5c70 con 0x7fb7d4004d10 2024-05-22T21:14:54.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.731+0000 7fb7efe00640 1 -- 172.21.15.161:0/3748176389 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e800f040 con 0x7fb7d4004d10 2024-05-22T21:14:54.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.731+0000 7fb7efe00640 1 -- 172.21.15.161:0/3748176389 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e80ad440 con 0x7fb7d4004d10 2024-05-22T21:14:54.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.731+0000 7fb7f7c00640 1 -- 172.21.15.161:0/3748176389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d4004d10 msgr2=0x7fb7d4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:54.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.731+0000 7fb7f7c00640 1 --2- 172.21.15.161:0/3748176389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d4004d10 0x7fb7d4005190 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800e170 tx=0x7fb7e801cc60 comp rx=0 tx=0).stop 2024-05-22T21:14:54.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.731+0000 7fb7f5c00640 1 -- 172.21.15.161:0/3748176389 reap_dead start 2024-05-22T21:14:54.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.731+0000 7fb7f7c00640 1 -- 172.21.15.161:0/3748176389 shutdown_connections 2024-05-22T21:14:54.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.731+0000 7fb7f7c00640 1 -- 172.21.15.161:0/3748176389 >> 172.21.15.161:0/3748176389 conn(0x7fb7f006a2d0 msgr2=0x7fb7f006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:54.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.731+0000 7fb7f7c00640 1 -- 172.21.15.161:0/3748176389 shutdown_connections 2024-05-22T21:14:54.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.731+0000 7fb7f7c00640 1 -- 172.21.15.161:0/3748176389 wait complete. 2024-05-22T21:14:54.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7f7c00640 1 Processor -- start 2024-05-22T21:14:54.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7f7c00640 1 -- start start 2024-05-22T21:14:54.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7f7c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7d4004d10 0x7fb7f0140830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:54.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7f7c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7f01e7650 0x7fb7f00b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:54.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7f7c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7f00b10e0 0x7fb7f013db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:54.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7f7c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f00b1560 con 0x7fb7f01e7650 2024-05-22T21:14:54.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7f7c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f01f6200 con 0x7fb7f00b10e0 2024-05-22T21:14:54.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7f7c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f0139160 con 0x7fb7d4004d10 2024-05-22T21:14:54.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7e7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7f01e7650 0x7fb7f00b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:54.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7e7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7f01e7650 0x7fb7f00b0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:56122/0 (socket says 172.21.15.161:56122) 2024-05-22T21:14:54.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7e7e00640 1 -- 172.21.15.161:0/3790145303 learned_addr learned my addr 172.21.15.161:0/3790145303 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:54.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7f5200640 1 --2- 172.21.15.161:0/3790145303 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7d4004d10 0x7fb7f0140830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:54.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7f5c00640 1 --2- 172.21.15.161:0/3790145303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7f00b10e0 0x7fb7f013db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:54.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7e7e00640 1 -- 172.21.15.161:0/3790145303 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7d4004d10 msgr2=0x7fb7f0140830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:54.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7e7e00640 1 --2- 172.21.15.161:0/3790145303 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7d4004d10 0x7fb7f0140830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7e7e00640 1 -- 172.21.15.161:0/3790145303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7f00b10e0 msgr2=0x7fb7f013db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:54.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7e7e00640 1 --2- 172.21.15.161:0/3790145303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7f00b10e0 0x7fb7f013db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7e7e00640 1 -- 172.21.15.161:0/3790145303 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f01f64c0 con 0x7fb7f01e7650 2024-05-22T21:14:54.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7e7e00640 1 --2- 172.21.15.161:0/3790145303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7f01e7650 0x7fb7f00b0ba0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8007310 tx=0x7fb7d8007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:54.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7ed600640 1 -- 172.21.15.161:0/3790145303 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8007e40 con 0x7fb7f01e7650 2024-05-22T21:14:54.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7ed600640 1 -- 172.21.15.161:0/3790145303 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d800f9c0 con 0x7fb7f01e7650 2024-05-22T21:14:54.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7f7c00640 1 -- 172.21.15.161:0/3790145303 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f01f67a0 con 0x7fb7f01e7650 2024-05-22T21:14:54.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7ed600640 1 -- 172.21.15.161:0/3790145303 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d800fd50 con 0x7fb7f01e7650 2024-05-22T21:14:54.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.735+0000 7fb7f7c00640 1 -- 172.21.15.161:0/3790145303 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f01f6fa0 con 0x7fb7f01e7650 2024-05-22T21:14:54.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.739+0000 7fb7f7c00640 1 -- 172.21.15.161:0/3790145303 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f0067220 con 0x7fb7f01e7650 2024-05-22T21:14:54.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.739+0000 7fb7ed600640 1 -- 172.21.15.161:0/3790145303 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb7d801d460 con 0x7fb7f01e7650 2024-05-22T21:14:54.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.739+0000 7fb7ed600640 1 --2- 172.21.15.161:0/3790145303 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7cc066690 0x7fb7cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:54.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.739+0000 7fb7ed600640 1 -- 172.21.15.161:0/3790145303 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7d8084270 con 0x7fb7f01e7650 2024-05-22T21:14:54.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.739+0000 7fb7f5200640 1 --2- 172.21.15.161:0/3790145303 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7cc066690 0x7fb7cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:54.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.739+0000 7fb7f5200640 1 --2- 172.21.15.161:0/3790145303 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7cc066690 0x7fb7cc068b50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009c80 tx=0x7fb7e0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:54.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.743+0000 7fb7ed600640 1 -- 172.21.15.161:0/3790145303 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb7d8051540 con 0x7fb7f01e7650 2024-05-22T21:14:54.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.955+0000 7fb7f7c00640 1 -- 172.21.15.161:0/3790145303 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb7f01f6c50 con 0x7fb7f01e7650 2024-05-22T21:14:54.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.959+0000 7fb7ed600640 1 -- 172.21.15.161:0/3790145303 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb7d8054b60 con 0x7fb7f01e7650 2024-05-22T21:14:54.963 INFO:teuthology.orchestra.run.smithi161.stderr:updated caps for client.0 2024-05-22T21:14:54.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.959+0000 7fb7f7c00640 1 -- 172.21.15.161:0/3790145303 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7cc066690 msgr2=0x7fb7cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:54.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.959+0000 7fb7f7c00640 1 --2- 172.21.15.161:0/3790145303 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7cc066690 0x7fb7cc068b50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009c80 tx=0x7fb7e0009190 comp rx=0 tx=0).stop 2024-05-22T21:14:54.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.959+0000 7fb7f7c00640 1 -- 172.21.15.161:0/3790145303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7f01e7650 msgr2=0x7fb7f00b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:54.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.959+0000 7fb7f7c00640 1 --2- 172.21.15.161:0/3790145303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7f01e7650 0x7fb7f00b0ba0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8007310 tx=0x7fb7d8007710 comp rx=0 tx=0).stop 2024-05-22T21:14:54.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.959+0000 7fb7f7c00640 1 -- 172.21.15.161:0/3790145303 shutdown_connections 2024-05-22T21:14:54.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.959+0000 7fb7f7c00640 1 --2- 172.21.15.161:0/3790145303 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7cc066690 0x7fb7cc068b50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.959+0000 7fb7f7c00640 1 --2- 172.21.15.161:0/3790145303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7f00b10e0 0x7fb7f013db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.959+0000 7fb7f7c00640 1 --2- 172.21.15.161:0/3790145303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7f01e7650 0x7fb7f00b0ba0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.959+0000 7fb7f7c00640 1 --2- 172.21.15.161:0/3790145303 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7d4004d10 0x7fb7f0140830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:54.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.959+0000 7fb7f7c00640 1 -- 172.21.15.161:0/3790145303 >> 172.21.15.161:0/3790145303 conn(0x7fb7f006a2d0 msgr2=0x7fb7f00af4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:54.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.959+0000 7fb7f7c00640 1 -- 172.21.15.161:0/3790145303 shutdown_connections 2024-05-22T21:14:54.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:54.959+0000 7fb7f7c00640 1 -- 172.21.15.161:0/3790145303 wait complete. 2024-05-22T21:14:54.984 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:14:55.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f252dc00640 1 Processor -- start 2024-05-22T21:14:55.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f252dc00640 1 -- start start 2024-05-22T21:14:55.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f252dc00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f252806bfb0 0x7f25280cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f252dc00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252806c440 con 0x7f252815d4a0 2024-05-22T21:14:55.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f252dc00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25280cf190 con 0x7f25280b0490 2024-05-22T21:14:55.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f252dc00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f252815d880 con 0x7f252806bfb0 2024-05-22T21:14:55.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2526a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f252815d4a0 0x7f2528099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:57772/0 (socket says 172.21.15.161:57772) 2024-05-22T21:14:55.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2526a00640 1 -- 172.21.15.161:0/3348542679 learned_addr learned my addr 172.21.15.161:0/3348542679 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:55.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2527400640 1 --2- 172.21.15.161:0/3348542679 >> v2:172.21.15.179:3300/0 conn(0x7f252806bfb0 0x7f25280cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2525600640 1 -- 172.21.15.161:0/3348542679 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922512692 0 0) 0x7f25180014a0 con 0x7f252815d4a0 2024-05-22T21:14:55.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2525600640 1 -- 172.21.15.161:0/3348542679 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f250c003530 con 0x7f252815d4a0 2024-05-22T21:14:55.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2525600640 1 -- 172.21.15.161:0/3348542679 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2605800566 0 0) 0x7f250c003530 con 0x7f252815d4a0 2024-05-22T21:14:55.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2525600640 1 -- 172.21.15.161:0/3348542679 >> v2:172.21.15.179:3300/0 conn(0x7f252806bfb0 msgr2=0x7f25280cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:55.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2525600640 1 --2- 172.21.15.161:0/3348542679 >> v2:172.21.15.179:3300/0 conn(0x7f252806bfb0 0x7f25280cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2525600640 1 -- 172.21.15.161:0/3348542679 >> v1:172.21.15.179:6789/0 conn(0x7f25280b0490 legacy=0x7f25280cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2525600640 1 -- 172.21.15.161:0/3348542679 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252806b160 con 0x7f252815d4a0 2024-05-22T21:14:55.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2525600640 1 -- 172.21.15.161:0/3348542679 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f2518001b90 con 0x7f252815d4a0 2024-05-22T21:14:55.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2525600640 1 -- 172.21.15.161:0/3348542679 >> v1:172.21.15.161:6789/0 conn(0x7f252815d4a0 legacy=0x7f2528099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2525600640 1 --2- 172.21.15.161:0/3348542679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f250c003980 0x7f250c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2525600640 1 --2- 172.21.15.161:0/3348542679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f250c0047d0 0x7f250c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2525600640 1 --2- 172.21.15.161:0/3348542679 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f250c005190 0x7f250c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2525600640 1 -- 172.21.15.161:0/3348542679 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252815d880 con 0x7f250c0047d0 2024-05-22T21:14:55.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2525600640 1 -- 172.21.15.161:0/3348542679 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c00c030 con 0x7f250c003980 2024-05-22T21:14:55.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.119+0000 7f2525600640 1 -- 172.21.15.161:0/3348542679 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250c00c1f0 con 0x7f250c005190 2024-05-22T21:14:55.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.123+0000 7f2526000640 1 --2- 172.21.15.161:0/3348542679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f250c0047d0 0x7f250c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.123+0000 7f2527400640 1 --2- 172.21.15.161:0/3348542679 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f250c005190 0x7f250c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.123+0000 7f2526a00640 1 --2- 172.21.15.161:0/3348542679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f250c003980 0x7f250c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.123+0000 7f2526000640 1 -- 172.21.15.161:0/3348542679 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f250c005190 msgr2=0x7f250c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:55.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.123+0000 7f2526000640 1 --2- 172.21.15.161:0/3348542679 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f250c005190 0x7f250c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.123+0000 7f2526000640 1 -- 172.21.15.161:0/3348542679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f250c003980 msgr2=0x7f250c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:55.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.123+0000 7f2526000640 1 --2- 172.21.15.161:0/3348542679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f250c003980 0x7f250c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.123+0000 7f2526000640 1 -- 172.21.15.161:0/3348542679 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250c003530 con 0x7f250c0047d0 2024-05-22T21:14:55.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.123+0000 7f2527400640 1 -- 172.21.15.161:0/3348542679 reap_dead start 2024-05-22T21:14:55.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.123+0000 7f2526000640 1 --2- 172.21.15.161:0/3348542679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f250c0047d0 0x7f250c004c50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2514001780 tx=0x7f251401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784b7a28e5560e7d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:55.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.123+0000 7f2525600640 1 -- 172.21.15.161:0/3348542679 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25140050d0 con 0x7f250c0047d0 2024-05-22T21:14:55.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.123+0000 7f2525600640 1 -- 172.21.15.161:0/3348542679 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25140a7050 con 0x7f250c0047d0 2024-05-22T21:14:55.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.123+0000 7f2525600640 1 -- 172.21.15.161:0/3348542679 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514005490 con 0x7f250c0047d0 2024-05-22T21:14:55.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f252dc00640 1 -- 172.21.15.161:0/3348542679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f250c0047d0 msgr2=0x7f250c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f252dc00640 1 --2- 172.21.15.161:0/3348542679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f250c0047d0 0x7f250c004c50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2514001780 tx=0x7f251401c400 comp rx=0 tx=0).stop 2024-05-22T21:14:55.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f252dc00640 1 -- 172.21.15.161:0/3348542679 shutdown_connections 2024-05-22T21:14:55.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f252dc00640 1 --2- 172.21.15.161:0/3348542679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f250c0047d0 0x7f250c004c50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f252dc00640 1 -- 172.21.15.161:0/3348542679 >> 172.21.15.161:0/3348542679 conn(0x7f25280b2a40 msgr2=0x7f25280b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:55.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f252dc00640 1 -- 172.21.15.161:0/3348542679 shutdown_connections 2024-05-22T21:14:55.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f252dc00640 1 -- 172.21.15.161:0/3348542679 wait complete. 2024-05-22T21:14:55.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f252dc00640 1 Processor -- start 2024-05-22T21:14:55.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f252dc00640 1 -- start start 2024-05-22T21:14:55.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f252dc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f250c0047d0 0x7f25280c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f252dc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f252814b8a0 0x7f25280ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f252dc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2528149af0 0x7f2528149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f252dc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25280cad60 con 0x7f2528149af0 2024-05-22T21:14:55.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f252dc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2528148d30 con 0x7f252814b8a0 2024-05-22T21:14:55.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f252dc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2528149010 con 0x7f250c0047d0 2024-05-22T21:14:55.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f2527400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2528149af0 0x7f2528149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f2527400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2528149af0 0x7f2528149f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:56136/0 (socket says 172.21.15.161:56136) 2024-05-22T21:14:55.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f2527400640 1 -- 172.21.15.161:0/3971282329 learned_addr learned my addr 172.21.15.161:0/3971282329 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:55.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f2526a00640 1 --2- 172.21.15.161:0/3971282329 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f250c0047d0 0x7f25280c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f2526000640 1 --2- 172.21.15.161:0/3971282329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f252814b8a0 0x7f25280ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f2527400640 1 -- 172.21.15.161:0/3971282329 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f250c0047d0 msgr2=0x7f25280c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f2527400640 1 --2- 172.21.15.161:0/3971282329 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f250c0047d0 0x7f25280c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f2527400640 1 -- 172.21.15.161:0/3971282329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f252814b8a0 msgr2=0x7f25280ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f2527400640 1 --2- 172.21.15.161:0/3971282329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f252814b8a0 0x7f25280ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.127+0000 7f2527400640 1 -- 172.21.15.161:0/3971282329 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25281492f0 con 0x7f2528149af0 2024-05-22T21:14:55.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.131+0000 7f2527400640 1 --2- 172.21.15.161:0/3971282329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2528149af0 0x7f2528149f70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2528074240 tx=0x7f2520011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:55.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.131+0000 7f251ea00640 1 -- 172.21.15.161:0/3971282329 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2520004030 con 0x7f2528149af0 2024-05-22T21:14:55.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.131+0000 7f251ea00640 1 -- 172.21.15.161:0/3971282329 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2520004190 con 0x7f2528149af0 2024-05-22T21:14:55.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.131+0000 7f252dc00640 1 -- 172.21.15.161:0/3971282329 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25281495d0 con 0x7f2528149af0 2024-05-22T21:14:55.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.131+0000 7f251ea00640 1 -- 172.21.15.161:0/3971282329 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252000f770 con 0x7f2528149af0 2024-05-22T21:14:55.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.131+0000 7f252dc00640 1 -- 172.21.15.161:0/3971282329 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252814a4b0 con 0x7f2528149af0 2024-05-22T21:14:55.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.131+0000 7f252dc00640 1 -- 172.21.15.161:0/3971282329 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2528067220 con 0x7f2528149af0 2024-05-22T21:14:55.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.131+0000 7f251ea00640 1 -- 172.21.15.161:0/3971282329 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f252000f8d0 con 0x7f2528149af0 2024-05-22T21:14:55.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.131+0000 7f251ea00640 1 --2- 172.21.15.161:0/3971282329 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2508066690 0x7f2508068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.131+0000 7f251ea00640 1 -- 172.21.15.161:0/3971282329 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2520084240 con 0x7f2528149af0 2024-05-22T21:14:55.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.131+0000 7f2526a00640 1 --2- 172.21.15.161:0/3971282329 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2508066690 0x7f2508068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.135+0000 7f2526a00640 1 --2- 172.21.15.161:0/3971282329 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2508066690 0x7f2508068b50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2518001720 tx=0x7f2518001380 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:55.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.135+0000 7f251ea00640 1 -- 172.21.15.161:0/3971282329 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2520051660 con 0x7f2528149af0 2024-05-22T21:14:55.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.327+0000 7f252dc00640 1 -- 172.21.15.161:0/3971282329 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25280c7b10 con 0x7f2528149af0 2024-05-22T21:14:55.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.327+0000 7f251ea00640 1 -- 172.21.15.161:0/3971282329 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f2520054c80 con 0x7f2528149af0 2024-05-22T21:14:55.330 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:55.330 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/1767934947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1767934947},{"type":"v1","addr":"172.21.15.161:6836","nonce":1767934947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1873337287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1873337287},{"type":"v1","addr":"172.21.15.179:6834","nonce":1873337287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1815838236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":1815838236},{"type":"v1","addr":"172.21.15.161:6837","nonce":1815838236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:50.052292+0000","modified":"2024-05-22T21:14:53.058881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5374":{"gid":5374,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/389418050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":389418050},{"type":"v1","addr":"172.21.15.179:6835","nonce":389418050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-22T21:14:55.331 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 23 2024-05-22T21:14:55.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.327+0000 7f252dc00640 1 -- 172.21.15.161:0/3971282329 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2508066690 msgr2=0x7f2508068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.327+0000 7f252dc00640 1 --2- 172.21.15.161:0/3971282329 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2508066690 0x7f2508068b50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2518001720 tx=0x7f2518001380 comp rx=0 tx=0).stop 2024-05-22T21:14:55.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.327+0000 7f252dc00640 1 -- 172.21.15.161:0/3971282329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2528149af0 msgr2=0x7f2528149f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.327+0000 7f252dc00640 1 --2- 172.21.15.161:0/3971282329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2528149af0 0x7f2528149f70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2528074240 tx=0x7f2520011cf0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.327+0000 7f252dc00640 1 -- 172.21.15.161:0/3971282329 shutdown_connections 2024-05-22T21:14:55.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.327+0000 7f252dc00640 1 --2- 172.21.15.161:0/3971282329 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2508066690 0x7f2508068b50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.327+0000 7f252dc00640 1 --2- 172.21.15.161:0/3971282329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2528149af0 0x7f2528149f70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.327+0000 7f252dc00640 1 --2- 172.21.15.161:0/3971282329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f252814b8a0 0x7f25280ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.327+0000 7f252dc00640 1 --2- 172.21.15.161:0/3971282329 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f250c0047d0 0x7f25280c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.327+0000 7f252dc00640 1 -- 172.21.15.161:0/3971282329 >> 172.21.15.161:0/3971282329 conn(0x7f25280b2a40 msgr2=0x7f25280add20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:55.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.327+0000 7f252dc00640 1 -- 172.21.15.161:0/3971282329 shutdown_connections 2024-05-22T21:14:55.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.327+0000 7f252dc00640 1 -- 172.21.15.161:0/3971282329 wait complete. 2024-05-22T21:14:55.753 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-22T21:14:50.052292+0000', 'modified': '2024-05-22T21:14:53.058881+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5374}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5374': {'gid': 5374, 'name': 'b', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.179:6835/389418050', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6833', 'nonce': 389418050}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 389418050}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-22T21:14:55.753 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-22T21:14:55.754 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-22T21:14:55.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06bbe00640 1 Processor -- start 2024-05-22T21:14:55.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06bbe00640 1 -- start start 2024-05-22T21:14:55.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06bbe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06bc15eab0 0x7f06bc15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06bbe00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f06bc069ed0 0x7f06bc06a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06bbe00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06bc0ce7c0 con 0x7f06bc06b300 2024-05-22T21:14:55.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06bbe00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06bc0cf360 con 0x7f06bc069ed0 2024-05-22T21:14:55.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06bbe00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0b3620 con 0x7f06bc15eab0 2024-05-22T21:14:55.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06baa00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f06bc06b300 0x7f06bc0cec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56116/0 (socket says 172.21.15.161:56116) 2024-05-22T21:14:55.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06baa00640 1 -- 172.21.15.161:0/2652877502 learned_addr learned my addr 172.21.15.161:0/2652877502 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:55.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06ba000640 1 -- 172.21.15.161:0/2652877502 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74035870 0 0) 0x7f06bc0ce7c0 con 0x7f06bc06b300 2024-05-22T21:14:55.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06b2000640 1 --2- 172.21.15.161:0/2652877502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06bc15eab0 0x7f06bc15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06ba000640 1 -- 172.21.15.161:0/2652877502 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06a0003530 con 0x7f06bc06b300 2024-05-22T21:14:55.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06bb400640 1 --2- 172.21.15.161:0/2652877502 >> v2:172.21.15.179:3300/0 conn(0x7f06bc069ed0 0x7f06bc06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06ba000640 1 -- 172.21.15.161:0/2652877502 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2585624878 0 0) 0x7f06a0003530 con 0x7f06bc06b300 2024-05-22T21:14:55.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06ba000640 1 -- 172.21.15.161:0/2652877502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06bc15eab0 msgr2=0x7f06bc15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:55.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06ba000640 1 --2- 172.21.15.161:0/2652877502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06bc15eab0 0x7f06bc15eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06ba000640 1 -- 172.21.15.161:0/2652877502 >> v2:172.21.15.179:3300/0 conn(0x7f06bc069ed0 msgr2=0x7f06bc06a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:55.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06ba000640 1 --2- 172.21.15.161:0/2652877502 >> v2:172.21.15.179:3300/0 conn(0x7f06bc069ed0 0x7f06bc06a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06ba000640 1 -- 172.21.15.161:0/2652877502 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc15f430 con 0x7f06bc06b300 2024-05-22T21:14:55.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06ba000640 1 -- 172.21.15.161:0/2652877502 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f06ac001680 con 0x7f06bc06b300 2024-05-22T21:14:55.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06bb400640 1 --2- 172.21.15.161:0/2652877502 >> v2:172.21.15.179:3300/0 conn(0x7f06bc069ed0 0x7f06bc06a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:55.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06ba000640 1 -- 172.21.15.161:0/2652877502 >> v1:172.21.15.161:6789/0 conn(0x7f06bc06b300 legacy=0x7f06bc0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06ba000640 1 --2- 172.21.15.161:0/2652877502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06a0003980 0x7f06a0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06ba000640 1 --2- 172.21.15.161:0/2652877502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06bc15eab0 0x7f06a00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06ba000640 1 --2- 172.21.15.161:0/2652877502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a0004d10 0x7f06a0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06ba000640 1 -- 172.21.15.161:0/2652877502 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0cf360 con 0x7f06a0003980 2024-05-22T21:14:55.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06ba000640 1 -- 172.21.15.161:0/2652877502 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0b3620 con 0x7f06a0004d10 2024-05-22T21:14:55.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06ba000640 1 -- 172.21.15.161:0/2652877502 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a000ba70 con 0x7f06bc15eab0 2024-05-22T21:14:55.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.887+0000 7f06baa00640 1 --2- 172.21.15.161:0/2652877502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06a0003980 0x7f06a0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.891+0000 7f06b2000640 1 --2- 172.21.15.161:0/2652877502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06bc15eab0 0x7f06a00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.891+0000 7f06bb400640 1 --2- 172.21.15.161:0/2652877502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a0004d10 0x7f06a0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.891+0000 7f06baa00640 1 -- 172.21.15.161:0/2652877502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06bc15eab0 msgr2=0x7f06a00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:55.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.891+0000 7f06baa00640 1 --2- 172.21.15.161:0/2652877502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06bc15eab0 0x7f06a00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.891+0000 7f06baa00640 1 -- 172.21.15.161:0/2652877502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a0004d10 msgr2=0x7f06a0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:55.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.891+0000 7f06baa00640 1 --2- 172.21.15.161:0/2652877502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a0004d10 0x7f06a0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.891+0000 7f06baa00640 1 -- 172.21.15.161:0/2652877502 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a0003530 con 0x7f06a0003980 2024-05-22T21:14:55.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.891+0000 7f06baa00640 1 --2- 172.21.15.161:0/2652877502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06a0003980 0x7f06a0003da0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f06ac00c210 tx=0x7f06ac01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96eeacadc3991ac server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:55.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.891+0000 7f06ba000640 1 -- 172.21.15.161:0/2652877502 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac0058e0 con 0x7f06a0003980 2024-05-22T21:14:55.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.891+0000 7f06ba000640 1 -- 172.21.15.161:0/2652877502 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ac005a40 con 0x7f06a0003980 2024-05-22T21:14:55.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.891+0000 7f06ba000640 1 -- 172.21.15.161:0/2652877502 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac005e10 con 0x7f06a0003980 2024-05-22T21:14:55.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.891+0000 7f06bbe00640 1 -- 172.21.15.161:0/2652877502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06a0003980 msgr2=0x7f06a0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.891+0000 7f06bbe00640 1 --2- 172.21.15.161:0/2652877502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06a0003980 0x7f06a0003da0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f06ac00c210 tx=0x7f06ac01cc80 comp rx=0 tx=0).stop 2024-05-22T21:14:55.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.891+0000 7f06bb400640 1 -- 172.21.15.161:0/2652877502 reap_dead start 2024-05-22T21:14:55.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06bbe00640 1 -- 172.21.15.161:0/2652877502 shutdown_connections 2024-05-22T21:14:55.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06bbe00640 1 -- 172.21.15.161:0/2652877502 >> 172.21.15.161:0/2652877502 conn(0x7f06bc0cf4a0 msgr2=0x7f06bc0cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:55.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06bbe00640 1 -- 172.21.15.161:0/2652877502 shutdown_connections 2024-05-22T21:14:55.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06bbe00640 1 -- 172.21.15.161:0/2652877502 wait complete. 2024-05-22T21:14:55.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06bbe00640 1 Processor -- start 2024-05-22T21:14:55.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06bbe00640 1 -- start start 2024-05-22T21:14:55.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06bbe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06a0003980 0x7f06bc151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06bbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06bc14ed70 0x7f06bc14df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06bbe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06bc06c300 0x7f06bc159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06bbe00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc14f290 con 0x7f06bc14ed70 2024-05-22T21:14:55.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06bbe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0ac8b0 con 0x7f06bc06c300 2024-05-22T21:14:55.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06bbe00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0acb00 con 0x7f06a0003980 2024-05-22T21:14:55.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06b2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06bc14ed70 0x7f06bc14df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06b2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06bc14ed70 0x7f06bc14df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45852/0 (socket says 172.21.15.161:45852) 2024-05-22T21:14:55.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06b2000640 1 -- 172.21.15.161:0/662681408 learned_addr learned my addr 172.21.15.161:0/662681408 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:55.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06baa00640 1 --2- 172.21.15.161:0/662681408 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06a0003980 0x7f06bc151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06bb400640 1 --2- 172.21.15.161:0/662681408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06bc06c300 0x7f06bc159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06b2000640 1 -- 172.21.15.161:0/662681408 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06a0003980 msgr2=0x7f06bc151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06b2000640 1 --2- 172.21.15.161:0/662681408 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06a0003980 0x7f06bc151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06b2000640 1 -- 172.21.15.161:0/662681408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06bc06c300 msgr2=0x7f06bc159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06b2000640 1 --2- 172.21.15.161:0/662681408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06bc06c300 0x7f06bc159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06b2000640 1 -- 172.21.15.161:0/662681408 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc0acde0 con 0x7f06bc14ed70 2024-05-22T21:14:55.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06bb400640 1 --2- 172.21.15.161:0/662681408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06bc06c300 0x7f06bc159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:55.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06b2000640 1 --2- 172.21.15.161:0/662681408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06bc14ed70 0x7f06bc14df80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f06a40119d0 tx=0x7f06a4011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:55.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06b3400640 1 -- 172.21.15.161:0/662681408 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a400fb50 con 0x7f06bc14ed70 2024-05-22T21:14:55.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06b3400640 1 -- 172.21.15.161:0/662681408 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a400fcb0 con 0x7f06bc14ed70 2024-05-22T21:14:55.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06bbe00640 1 -- 172.21.15.161:0/662681408 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06bc0ad0c0 con 0x7f06bc14ed70 2024-05-22T21:14:55.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.895+0000 7f06b3400640 1 -- 172.21.15.161:0/662681408 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4014440 con 0x7f06bc14ed70 2024-05-22T21:14:55.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.899+0000 7f06bbe00640 1 -- 172.21.15.161:0/662681408 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06bc1628b0 con 0x7f06bc14ed70 2024-05-22T21:14:55.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.899+0000 7f06b3400640 1 -- 172.21.15.161:0/662681408 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f06a40145a0 con 0x7f06bc14ed70 2024-05-22T21:14:55.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.899+0000 7f06b3400640 1 --2- 172.21.15.161:0/662681408 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f06980666c0 0x7f0698068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.899+0000 7f06b3400640 1 -- 172.21.15.161:0/662681408 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06a4084aa0 con 0x7f06bc14ed70 2024-05-22T21:14:55.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.899+0000 7f06baa00640 1 --2- 172.21.15.161:0/662681408 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f06980666c0 0x7f0698068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.899+0000 7f06c4fea000 1 Processor -- start 2024-05-22T21:14:55.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.899+0000 7f06c4fea000 1 -- start start 2024-05-22T21:14:55.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.899+0000 7f06baa00640 1 --2- 172.21.15.161:0/662681408 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f06980666c0 0x7f0698068b80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0aa000 tx=0x7f06ac001a10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:55.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 0x55bb37b6bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b6c2e0 0x55bb37b21980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b21ec0 0x55bb37b222a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb379e9e80 con 0x55bb37b6c2e0 2024-05-22T21:14:55.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb37977af0 con 0x55bb37b21ec0 2024-05-22T21:14:55.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb379750c0 con 0x55bb37b6b9c0 2024-05-22T21:14:55.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06b2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b6c2e0 0x55bb37b21980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06b2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b6c2e0 0x55bb37b21980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45854/0 (socket says 172.21.15.161:45854) 2024-05-22T21:14:55.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06b2000640 1 -- 172.21.15.161:0/778250431 learned_addr learned my addr 172.21.15.161:0/778250431 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:55.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06bb400640 1 --2- 172.21.15.161:0/778250431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b21ec0 0x55bb37b222a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06baa00640 1 --2- 172.21.15.161:0/778250431 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 0x55bb37b6bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06b2000640 1 -- 172.21.15.161:0/778250431 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 msgr2=0x55bb37b6bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06b2000640 1 --2- 172.21.15.161:0/778250431 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 0x55bb37b6bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06b2000640 1 -- 172.21.15.161:0/778250431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b21ec0 msgr2=0x55bb37b222a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06b2000640 1 --2- 172.21.15.161:0/778250431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b21ec0 0x55bb37b222a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06b2000640 1 -- 172.21.15.161:0/778250431 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb3766c890 con 0x55bb37b6c2e0 2024-05-22T21:14:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06bb400640 1 --2- 172.21.15.161:0/778250431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b21ec0 0x55bb37b222a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06b2000640 1 --2- 172.21.15.161:0/778250431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b6c2e0 0x55bb37b21980 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f06a4055a80 tx=0x7f06a4055580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aebcf2bb32e2675 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06ab400640 1 -- 172.21.15.161:0/778250431 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a405dee0 con 0x55bb37b6c2e0 2024-05-22T21:14:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06ab400640 1 -- 172.21.15.161:0/778250431 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a4053e90 con 0x55bb37b6c2e0 2024-05-22T21:14:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06ab400640 1 -- 172.21.15.161:0/778250431 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a401b300 con 0x55bb37b6c2e0 2024-05-22T21:14:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 -- 172.21.15.161:0/778250431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b6c2e0 msgr2=0x55bb37b21980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 --2- 172.21.15.161:0/778250431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b6c2e0 0x55bb37b21980 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f06a4055a80 tx=0x7f06a4055580 comp rx=0 tx=0).stop 2024-05-22T21:14:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 -- 172.21.15.161:0/778250431 shutdown_connections 2024-05-22T21:14:55.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 --2- 172.21.15.161:0/778250431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b21ec0 0x55bb37b222a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 --2- 172.21.15.161:0/778250431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b6c2e0 0x55bb37b21980 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 --2- 172.21.15.161:0/778250431 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 0x55bb37b6bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 -- 172.21.15.161:0/778250431 >> 172.21.15.161:0/778250431 conn(0x55bb37b28440 msgr2=0x55bb37b20260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:55.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 -- 172.21.15.161:0/778250431 shutdown_connections 2024-05-22T21:14:55.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 -- 172.21.15.161:0/778250431 wait complete. 2024-05-22T21:14:55.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 -- start start 2024-05-22T21:14:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.903+0000 7f06c4fea000 1 Processor -- start 2024-05-22T21:14:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b21ec0 0x55bb37b26df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 0x55bb37b2cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b6c2e0 0x55bb37b2d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb379709b0 con 0x55bb37b6c2e0 2024-05-22T21:14:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3790fc00 con 0x55bb37b21ec0 2024-05-22T21:14:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb3792e070 con 0x55bb37b6b9c0 2024-05-22T21:14:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 10 client.0 _ll_register_callbacks cb 0x55bb37b0dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:14:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06bb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b6c2e0 0x55bb37b2d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06bb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b6c2e0 0x55bb37b2d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45864/0 (socket says 172.21.15.161:45864) 2024-05-22T21:14:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06baa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b21ec0 0x55bb37b26df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06bb400640 1 -- 172.21.15.161:0/805180145 learned_addr learned my addr 172.21.15.161:0/805180145 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:55.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06b2000640 1 --2- 172.21.15.161:0/805180145 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 0x55bb37b2cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06bb400640 1 -- 172.21.15.161:0/805180145 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 msgr2=0x55bb37b2cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:55.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06bb400640 1 --2- 172.21.15.161:0/805180145 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 0x55bb37b2cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06bb400640 1 -- 172.21.15.161:0/805180145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b21ec0 msgr2=0x55bb37b26df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:55.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06bb400640 1 --2- 172.21.15.161:0/805180145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b21ec0 0x55bb37b26df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06bb400640 1 -- 172.21.15.161:0/805180145 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06bc1628b0 con 0x55bb37b6c2e0 2024-05-22T21:14:55.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06b2000640 1 --2- 172.21.15.161:0/805180145 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 0x55bb37b2cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:55.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06bb400640 1 --2- 172.21.15.161:0/805180145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b6c2e0 0x55bb37b2d4f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0cd980 tx=0x7f06b4014ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb1f5f0ba4ee09c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:55.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f069fe00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:14:55.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f069fe00640 1 -- 172.21.15.161:0/805180145 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4007930 con 0x55bb37b6c2e0 2024-05-22T21:14:55.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f069fe00640 1 -- 172.21.15.161:0/805180145 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06b4007a90 con 0x55bb37b6c2e0 2024-05-22T21:14:55.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 1 -- 172.21.15.161:0/805180145 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb3792d5f0 con 0x55bb37b6c2e0 2024-05-22T21:14:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f069fe00640 1 -- 172.21.15.161:0/805180145 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06b4007c50 con 0x55bb37b6c2e0 2024-05-22T21:14:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 10 client.5640 fetch_fsmap learned FSMap version 23 2024-05-22T21:14:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 1 -- 172.21.15.161:0/805180145 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55bb37935ab0 con 0x55bb37b6c2e0 2024-05-22T21:14:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f069fe00640 1 -- 172.21.15.161:0/805180145 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bb3792d5f0 con 0x55bb37b6c2e0 2024-05-22T21:14:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f069fe00640 1 -- 172.21.15.161:0/805180145 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 23) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f06b4022440 con 0x55bb37b6c2e0 2024-05-22T21:14:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 10 client.5640 fetch_fsmap finished waiting for FSMap version 23 2024-05-22T21:14:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 10 client.5640 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-22T21:14:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 20 client.5640 populate_metadata read hostname 'smithi161' 2024-05-22T21:14:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 1 --2- 172.21.15.161:0/805180145 >> [v2:172.21.15.179:6833/389418050,v1:172.21.15.179:6835/389418050] conn(0x55bb37b72bf0 0x55bb37b72fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 4 client.5640 mds_command: new command op to 5374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T21:14:55.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 1 -- 172.21.15.161:0/805180145 --> [v2:172.21.15.179:6833/389418050,v1:172.21.15.179:6835/389418050] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb3796d790 con 0x55bb37b72bf0 2024-05-22T21:14:55.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06baa00640 1 --2- 172.21.15.161:0/805180145 >> [v2:172.21.15.179:6833/389418050,v1:172.21.15.179:6835/389418050] conn(0x55bb37b72bf0 0x55bb37b72fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06baa00640 1 --2- 172.21.15.161:0/805180145 >> [v2:172.21.15.179:6833/389418050,v1:172.21.15.179:6835/389418050] conn(0x55bb37b72bf0 0x55bb37b72fd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=581d9ec8a0d22216 server_cookie=3be04ba5958c90b4 in_seq=0 out_seq=0 2024-05-22T21:14:55.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f069fe00640 10 client.5640 ms_handle_connect on v2:172.21.15.179:6833/389418050 2024-05-22T21:14:55.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f069fe00640 1 -- 172.21.15.161:0/805180145 <== mds.0 v2:172.21.15.179:6833/389418050 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bb3796d790 con 0x55bb37b72bf0 2024-05-22T21:14:55.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f069fe00640 10 client.5640 handle_command_reply: tid=0 2024-05-22T21:14:55.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.907+0000 7f06c4fea000 1 client.5640 shutdown 2024-05-22T21:14:55.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.911+0000 7f06c4fea000 1 -- 172.21.15.161:0/805180145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b6c2e0 msgr2=0x55bb37b2d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.911+0000 7f06c4fea000 1 --2- 172.21.15.161:0/805180145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b6c2e0 0x55bb37b2d4f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0cd980 tx=0x7f06b4014ed0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.911+0000 7f06c4fea000 1 -- 172.21.15.161:0/805180145 shutdown_connections 2024-05-22T21:14:55.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.911+0000 7f06c4fea000 1 --2- 172.21.15.161:0/805180145 >> [v2:172.21.15.179:6833/389418050,v1:172.21.15.179:6835/389418050] conn(0x55bb37b72bf0 0x55bb37b72fd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.911+0000 7f06c4fea000 1 --2- 172.21.15.161:0/805180145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b6c2e0 0x55bb37b2d4f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.911+0000 7f06c4fea000 1 --2- 172.21.15.161:0/805180145 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 0x55bb37b2cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.911+0000 7f06c4fea000 1 --2- 172.21.15.161:0/805180145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b21ec0 0x55bb37b26df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.911+0000 7f069fe00640 0 client.5640 ms_handle_reset on v2:172.21.15.179:6833/389418050 2024-05-22T21:14:55.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.911+0000 7f06c4fea000 1 -- 172.21.15.161:0/805180145 >> 172.21.15.161:0/805180145 conn(0x55bb37b28440 msgr2=0x55bb37b20960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:55.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.911+0000 7f06c4fea000 1 -- 172.21.15.161:0/805180145 shutdown_connections 2024-05-22T21:14:55.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.911+0000 7f06c4fea000 1 -- 172.21.15.161:0/805180145 wait complete. 2024-05-22T21:14:55.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.911+0000 7f06c4fea000 20 client.5640 trim_cache size 0 max 16384 2024-05-22T21:14:55.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.923+0000 7f06c4fea000 1 Processor -- start 2024-05-22T21:14:55.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.923+0000 7f06c4fea000 1 -- start start 2024-05-22T21:14:55.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b28440 0x55bb37b25cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b21ec0 0x55bb37b26210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b6b9c0 0x55bb37b6f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3762d940 con 0x55bb37b21ec0 2024-05-22T21:14:55.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb37972fa0 con 0x55bb37b6b9c0 2024-05-22T21:14:55.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb37937c60 con 0x55bb37b28440 2024-05-22T21:14:55.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06b2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b21ec0 0x55bb37b26210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06b2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b21ec0 0x55bb37b26210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45868/0 (socket says 172.21.15.161:45868) 2024-05-22T21:14:55.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06b2000640 1 -- 172.21.15.161:0/3432819870 learned_addr learned my addr 172.21.15.161:0/3432819870 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:55.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06bb400640 1 --2- 172.21.15.161:0/3432819870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b6b9c0 0x55bb37b6f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06baa00640 1 --2- 172.21.15.161:0/3432819870 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b28440 0x55bb37b25cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06b2000640 1 -- 172.21.15.161:0/3432819870 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b28440 msgr2=0x55bb37b25cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:55.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06b2000640 1 --2- 172.21.15.161:0/3432819870 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b28440 0x55bb37b25cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06b2000640 1 -- 172.21.15.161:0/3432819870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b6b9c0 msgr2=0x55bb37b6f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:55.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06b2000640 1 --2- 172.21.15.161:0/3432819870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b6b9c0 0x55bb37b6f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06b2000640 1 -- 172.21.15.161:0/3432819870 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb3791bc00 con 0x55bb37b21ec0 2024-05-22T21:14:55.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06b2000640 1 --2- 172.21.15.161:0/3432819870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b21ec0 0x55bb37b26210 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f06a4055a80 tx=0x7f06a40559f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9981014de2f7b38c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:55.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06abe00640 1 -- 172.21.15.161:0/3432819870 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a405dee0 con 0x55bb37b21ec0 2024-05-22T21:14:55.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06abe00640 1 -- 172.21.15.161:0/3432819870 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a4053e90 con 0x55bb37b21ec0 2024-05-22T21:14:55.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06abe00640 1 -- 172.21.15.161:0/3432819870 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a401b300 con 0x55bb37b21ec0 2024-05-22T21:14:55.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 -- 172.21.15.161:0/3432819870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b21ec0 msgr2=0x55bb37b26210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 --2- 172.21.15.161:0/3432819870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b21ec0 0x55bb37b26210 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f06a4055a80 tx=0x7f06a40559f0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 -- 172.21.15.161:0/3432819870 shutdown_connections 2024-05-22T21:14:55.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 --2- 172.21.15.161:0/3432819870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b6b9c0 0x55bb37b6f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 --2- 172.21.15.161:0/3432819870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b21ec0 0x55bb37b26210 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 --2- 172.21.15.161:0/3432819870 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b28440 0x55bb37b25cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 -- 172.21.15.161:0/3432819870 >> 172.21.15.161:0/3432819870 conn(0x55bb37b6a560 msgr2=0x55bb37b255c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:55.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 -- 172.21.15.161:0/3432819870 shutdown_connections 2024-05-22T21:14:55.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 -- 172.21.15.161:0/3432819870 wait complete. 2024-05-22T21:14:55.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 -- start start 2024-05-22T21:14:55.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.927+0000 7f06c4fea000 1 Processor -- start 2024-05-22T21:14:55.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b21ec0 0x55bb37bb0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b28440 0x55bb37bb13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 0x55bb37bb18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb37978d80 con 0x55bb37b28440 2024-05-22T21:14:55.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb37b81ed0 con 0x55bb37b21ec0 2024-05-22T21:14:55.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb379c8ad0 con 0x55bb37b6b9c0 2024-05-22T21:14:55.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 10 client.0 _ll_register_callbacks cb 0x7f06a4051e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:14:55.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06b2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b28440 0x55bb37bb13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06b2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b28440 0x55bb37bb13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45874/0 (socket says 172.21.15.161:45874) 2024-05-22T21:14:55.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06b2000640 1 -- 172.21.15.161:0/1855467760 learned_addr learned my addr 172.21.15.161:0/1855467760 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:55.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06baa00640 1 --2- 172.21.15.161:0/1855467760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b21ec0 0x55bb37bb0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06bb400640 1 --2- 172.21.15.161:0/1855467760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 0x55bb37bb18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06b2000640 1 -- 172.21.15.161:0/1855467760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 msgr2=0x55bb37bb18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:55.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06b2000640 1 --2- 172.21.15.161:0/1855467760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 0x55bb37bb18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06b2000640 1 -- 172.21.15.161:0/1855467760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b21ec0 msgr2=0x55bb37bb0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:55.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06b2000640 1 --2- 172.21.15.161:0/1855467760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b21ec0 0x55bb37bb0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06b2000640 1 -- 172.21.15.161:0/1855467760 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb3766c890 con 0x55bb37b28440 2024-05-22T21:14:55.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06bb400640 1 --2- 172.21.15.161:0/1855467760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 0x55bb37bb18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:55.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06b2000640 1 --2- 172.21.15.161:0/1855467760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b28440 0x55bb37bb13a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f06a4051ff0 tx=0x7f06a4057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6ddf5ca71192dc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:55.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f069fe00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:14:55.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f069fe00640 1 -- 172.21.15.161:0/1855467760 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a401b500 con 0x55bb37b28440 2024-05-22T21:14:55.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f069fe00640 1 -- 172.21.15.161:0/1855467760 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06a4055d50 con 0x55bb37b28440 2024-05-22T21:14:55.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 -- 172.21.15.161:0/1855467760 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb37940460 con 0x55bb37b28440 2024-05-22T21:14:55.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f069fe00640 1 -- 172.21.15.161:0/1855467760 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06a4055f10 con 0x55bb37b28440 2024-05-22T21:14:55.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 10 client.5646 fetch_fsmap learned FSMap version 23 2024-05-22T21:14:55.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 -- 172.21.15.161:0/1855467760 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55bb3766ccc0 con 0x55bb37b28440 2024-05-22T21:14:55.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f069fe00640 1 -- 172.21.15.161:0/1855467760 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bb37940460 con 0x55bb37b28440 2024-05-22T21:14:55.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f069fe00640 1 -- 172.21.15.161:0/1855467760 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 23) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f06a401b020 con 0x55bb37b28440 2024-05-22T21:14:55.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 10 client.5646 fetch_fsmap finished waiting for FSMap version 23 2024-05-22T21:14:55.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 10 client.5646 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-22T21:14:55.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 20 client.5646 populate_metadata read hostname 'smithi161' 2024-05-22T21:14:55.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 --2- 172.21.15.161:0/1855467760 >> [v2:172.21.15.179:6833/389418050,v1:172.21.15.179:6835/389418050] conn(0x55bb37b6c2e0 0x55bb37bb1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:55.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 4 client.5646 mds_command: new command op to 5374 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-22T21:14:55.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 -- 172.21.15.161:0/1855467760 --> [v2:172.21.15.179:6833/389418050,v1:172.21.15.179:6835/389418050] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bb379399c0 con 0x55bb37b6c2e0 2024-05-22T21:14:55.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06baa00640 1 --2- 172.21.15.161:0/1855467760 >> [v2:172.21.15.179:6833/389418050,v1:172.21.15.179:6835/389418050] conn(0x55bb37b6c2e0 0x55bb37bb1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:55.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06baa00640 1 --2- 172.21.15.161:0/1855467760 >> [v2:172.21.15.179:6833/389418050,v1:172.21.15.179:6835/389418050] conn(0x55bb37b6c2e0 0x55bb37bb1e20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d06a1f4d8cb51b2c server_cookie=959378ceb3753dcf in_seq=0 out_seq=0 2024-05-22T21:14:55.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f069fe00640 10 client.5646 ms_handle_connect on v2:172.21.15.179:6833/389418050 2024-05-22T21:14:55.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f069fe00640 1 -- 172.21.15.161:0/1855467760 <== mds.0 v2:172.21.15.179:6833/389418050 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55bb379399c0 con 0x55bb37b6c2e0 2024-05-22T21:14:55.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f069fe00640 10 client.5646 handle_command_reply: tid=0 2024-05-22T21:14:55.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 client.5646 shutdown 2024-05-22T21:14:55.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 -- 172.21.15.161:0/1855467760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b28440 msgr2=0x55bb37bb13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 --2- 172.21.15.161:0/1855467760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b28440 0x55bb37bb13a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f06a4051ff0 tx=0x7f06a4057000 comp rx=0 tx=0).stop 2024-05-22T21:14:55.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 -- 172.21.15.161:0/1855467760 shutdown_connections 2024-05-22T21:14:55.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 --2- 172.21.15.161:0/1855467760 >> [v2:172.21.15.179:6833/389418050,v1:172.21.15.179:6835/389418050] conn(0x55bb37b6c2e0 0x55bb37bb1e20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 --2- 172.21.15.161:0/1855467760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55bb37b6b9c0 0x55bb37bb18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 --2- 172.21.15.161:0/1855467760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb37b28440 0x55bb37bb13a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 --2- 172.21.15.161:0/1855467760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb37b21ec0 0x55bb37bb0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f069fe00640 0 client.5646 ms_handle_reset on v2:172.21.15.179:6833/389418050 2024-05-22T21:14:55.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 -- 172.21.15.161:0/1855467760 >> 172.21.15.161:0/1855467760 conn(0x55bb37b6a560 msgr2=0x55bb37b25e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:55.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 -- 172.21.15.161:0/1855467760 shutdown_connections 2024-05-22T21:14:55.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 1 -- 172.21.15.161:0/1855467760 wait complete. 2024-05-22T21:14:55.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.931+0000 7f06c4fea000 20 client.5646 trim_cache size 0 max 16384 2024-05-22T21:14:55.943 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:14:55.943 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:14:55.943 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 0, 2024-05-22T21:14:55.943 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 5374, 2024-05-22T21:14:55.943 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2024-05-22T21:14:55.943 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2024-05-22T21:14:55.943 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2024-05-22T21:14:55.943 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 5.8802758710000003, 2024-05-22T21:14:55.943 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 23, 2024-05-22T21:14:55.943 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 47, 2024-05-22T21:14:55.944 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 47, 2024-05-22T21:14:55.944 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 10.106939689000001 2024-05-22T21:14:55.944 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:14:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.935+0000 7f06bbe00640 1 -- 172.21.15.161:0/662681408 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f06980666c0 msgr2=0x7f0698068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.935+0000 7f06bbe00640 1 --2- 172.21.15.161:0/662681408 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f06980666c0 0x7f0698068b80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0aa000 tx=0x7f06ac001a10 comp rx=0 tx=0).stop 2024-05-22T21:14:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.935+0000 7f06bbe00640 1 -- 172.21.15.161:0/662681408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06bc14ed70 msgr2=0x7f06bc14df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.935+0000 7f06bbe00640 1 --2- 172.21.15.161:0/662681408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06bc14ed70 0x7f06bc14df80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f06a40119d0 tx=0x7f06a4011e90 comp rx=0 tx=0).stop 2024-05-22T21:14:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.935+0000 7f06bbe00640 1 -- 172.21.15.161:0/662681408 shutdown_connections 2024-05-22T21:14:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.935+0000 7f06bbe00640 1 --2- 172.21.15.161:0/662681408 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f06980666c0 0x7f0698068b80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.935+0000 7f06bbe00640 1 --2- 172.21.15.161:0/662681408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06bc06c300 0x7f06bc159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.935+0000 7f06bbe00640 1 --2- 172.21.15.161:0/662681408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06bc14ed70 0x7f06bc14df80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.935+0000 7f06bbe00640 1 --2- 172.21.15.161:0/662681408 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06a0003980 0x7f06bc151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.935+0000 7f06bbe00640 1 -- 172.21.15.161:0/662681408 >> 172.21.15.161:0/662681408 conn(0x7f06bc0cf4a0 msgr2=0x7f06bc15c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.935+0000 7f06bbe00640 1 -- 172.21.15.161:0/662681408 shutdown_connections 2024-05-22T21:14:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:55.935+0000 7f06bbe00640 1 -- 172.21.15.161:0/662681408 wait complete. 2024-05-22T21:14:55.957 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:14:56.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.087+0000 7fc09aa00640 1 Processor -- start 2024-05-22T21:14:56.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.087+0000 7fc09aa00640 1 -- start start 2024-05-22T21:14:56.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.087+0000 7fc09aa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc09415dd50 0x7fc09406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:56.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc09aa00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc09415d5e0 0x7fc09415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:56.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc09aa00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc094071670 con 0x7fc09415e130 2024-05-22T21:14:56.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc09aa00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc09406de40 con 0x7fc09415d5e0 2024-05-22T21:14:56.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc09aa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc094070a60 con 0x7fc09415dd50 2024-05-22T21:14:56.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc093400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc09415e130 0x7fc09415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56118/0 (socket says 172.21.15.161:56118) 2024-05-22T21:14:56.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc093400640 1 -- 172.21.15.161:0/928010451 learned_addr learned my addr 172.21.15.161:0/928010451 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:56.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc098a00640 1 --2- 172.21.15.161:0/928010451 >> v2:172.21.15.179:3300/0 conn(0x7fc09415d5e0 0x7fc09415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:56.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc093e00640 1 --2- 172.21.15.161:0/928010451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc09415dd50 0x7fc09406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:56.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc092a00640 1 -- 172.21.15.161:0/928010451 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2774170444 0 0) 0x7fc094071670 con 0x7fc09415e130 2024-05-22T21:14:56.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc092a00640 1 -- 172.21.15.161:0/928010451 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc078003530 con 0x7fc09415e130 2024-05-22T21:14:56.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc092a00640 1 -- 172.21.15.161:0/928010451 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 71049047 0 0) 0x7fc078003530 con 0x7fc09415e130 2024-05-22T21:14:56.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc092a00640 1 -- 172.21.15.161:0/928010451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc09415dd50 msgr2=0x7fc09406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:56.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc092a00640 1 --2- 172.21.15.161:0/928010451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc09415dd50 0x7fc09406d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:56.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc092a00640 1 -- 172.21.15.161:0/928010451 >> v2:172.21.15.179:3300/0 conn(0x7fc09415d5e0 msgr2=0x7fc09415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:56.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc092a00640 1 --2- 172.21.15.161:0/928010451 >> v2:172.21.15.179:3300/0 conn(0x7fc09415d5e0 0x7fc09415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:56.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc092a00640 1 -- 172.21.15.161:0/928010451 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc094070d40 con 0x7fc09415e130 2024-05-22T21:14:56.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc092a00640 1 -- 172.21.15.161:0/928010451 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fc080001680 con 0x7fc09415e130 2024-05-22T21:14:56.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc093e00640 1 --2- 172.21.15.161:0/928010451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc09415dd50 0x7fc09406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:56.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc098a00640 1 --2- 172.21.15.161:0/928010451 >> v2:172.21.15.179:3300/0 conn(0x7fc09415d5e0 0x7fc09415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:56.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc092a00640 1 -- 172.21.15.161:0/928010451 >> v1:172.21.15.161:6789/0 conn(0x7fc09415e130 legacy=0x7fc09415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:56.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc092a00640 1 --2- 172.21.15.161:0/928010451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc09415dd50 0x7fc078003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:56.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc092a00640 1 --2- 172.21.15.161:0/928010451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc078003ec0 0x7fc0780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:56.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc092a00640 1 --2- 172.21.15.161:0/928010451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc078004d10 0x7fc078005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:56.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc092a00640 1 -- 172.21.15.161:0/928010451 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09406de40 con 0x7fc078004d10 2024-05-22T21:14:56.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc092a00640 1 -- 172.21.15.161:0/928010451 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc094070a60 con 0x7fc078003ec0 2024-05-22T21:14:56.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc092a00640 1 -- 172.21.15.161:0/928010451 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07800b990 con 0x7fc09415dd50 2024-05-22T21:14:56.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc098a00640 1 --2- 172.21.15.161:0/928010451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc078004d10 0x7fc078005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:56.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc093400640 1 --2- 172.21.15.161:0/928010451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc078003ec0 0x7fc0780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:56.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc093e00640 1 --2- 172.21.15.161:0/928010451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc09415dd50 0x7fc078003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:56.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc098a00640 1 -- 172.21.15.161:0/928010451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc09415dd50 msgr2=0x7fc078003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:56.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc098a00640 1 --2- 172.21.15.161:0/928010451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc09415dd50 0x7fc078003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:56.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc098a00640 1 -- 172.21.15.161:0/928010451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc078003ec0 msgr2=0x7fc0780047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:56.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc098a00640 1 --2- 172.21.15.161:0/928010451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc078003ec0 0x7fc0780047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:56.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.091+0000 7fc098a00640 1 -- 172.21.15.161:0/928010451 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc084009420 con 0x7fc078004d10 2024-05-22T21:14:56.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.095+0000 7fc098a00640 1 --2- 172.21.15.161:0/928010451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc078004d10 0x7fc078005190 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00e320 tx=0x7fc08c01ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff87f974f1f8d311 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:56.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.095+0000 7fc092a00640 1 -- 172.21.15.161:0/928010451 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0a5d40 con 0x7fc078004d10 2024-05-22T21:14:56.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.095+0000 7fc092a00640 1 -- 172.21.15.161:0/928010451 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c00f040 con 0x7fc078004d10 2024-05-22T21:14:56.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.095+0000 7fc092a00640 1 -- 172.21.15.161:0/928010451 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0ad640 con 0x7fc078004d10 2024-05-22T21:14:56.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.095+0000 7fc09aa00640 1 -- 172.21.15.161:0/928010451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc078004d10 msgr2=0x7fc078005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:56.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.095+0000 7fc09aa00640 1 --2- 172.21.15.161:0/928010451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc078004d10 0x7fc078005190 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00e320 tx=0x7fc08c01ce40 comp rx=0 tx=0).stop 2024-05-22T21:14:56.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.095+0000 7fc098a00640 1 -- 172.21.15.161:0/928010451 reap_dead start 2024-05-22T21:14:56.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.095+0000 7fc09aa00640 1 -- 172.21.15.161:0/928010451 shutdown_connections 2024-05-22T21:14:56.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.095+0000 7fc09aa00640 1 -- 172.21.15.161:0/928010451 >> 172.21.15.161:0/928010451 conn(0x7fc0940b0bc0 msgr2=0x7fc0940b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:56.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.095+0000 7fc09aa00640 1 -- 172.21.15.161:0/928010451 shutdown_connections 2024-05-22T21:14:56.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.095+0000 7fc09aa00640 1 -- 172.21.15.161:0/928010451 wait complete. 2024-05-22T21:14:56.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.095+0000 7fc09aa00640 1 Processor -- start 2024-05-22T21:14:56.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.095+0000 7fc09aa00640 1 -- start start 2024-05-22T21:14:56.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc09aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc078004d10 0x7fc094144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:56.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc09aa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc094148530 0x7fc0940cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:56.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc09aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0940cef80 0x7fc0941e41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:56.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc09aa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940cf430 con 0x7fc0940cef80 2024-05-22T21:14:56.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc09aa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc094144dc0 con 0x7fc078004d10 2024-05-22T21:14:56.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc09aa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc094158010 con 0x7fc094148530 2024-05-22T21:14:56.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc098a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0940cef80 0x7fc0941e41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:56.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc098a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0940cef80 0x7fc0941e41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45900/0 (socket says 172.21.15.161:45900) 2024-05-22T21:14:56.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc098a00640 1 -- 172.21.15.161:0/2828241237 learned_addr learned my addr 172.21.15.161:0/2828241237 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:56.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc093e00640 1 --2- 172.21.15.161:0/2828241237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc078004d10 0x7fc094144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:56.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc093400640 1 --2- 172.21.15.161:0/2828241237 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc094148530 0x7fc0940cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:56.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc098a00640 1 -- 172.21.15.161:0/2828241237 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc094148530 msgr2=0x7fc0940cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:56.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc098a00640 1 --2- 172.21.15.161:0/2828241237 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc094148530 0x7fc0940cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:56.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc098a00640 1 -- 172.21.15.161:0/2828241237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc078004d10 msgr2=0x7fc094144880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:56.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc098a00640 1 --2- 172.21.15.161:0/2828241237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc078004d10 0x7fc094144880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:56.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc098a00640 1 -- 172.21.15.161:0/2828241237 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0941582f0 con 0x7fc0940cef80 2024-05-22T21:14:56.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc098a00640 1 --2- 172.21.15.161:0/2828241237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0940cef80 0x7fc0941e41c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc08c002240 tx=0x7fc08c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:56.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc08be00640 1 -- 172.21.15.161:0/2828241237 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0bc070 con 0x7fc0940cef80 2024-05-22T21:14:56.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc08be00640 1 -- 172.21.15.161:0/2828241237 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c0039a0 con 0x7fc0940cef80 2024-05-22T21:14:56.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc09aa00640 1 -- 172.21.15.161:0/2828241237 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0941585d0 con 0x7fc0940cef80 2024-05-22T21:14:56.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc08be00640 1 -- 172.21.15.161:0/2828241237 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c003b30 con 0x7fc0940cef80 2024-05-22T21:14:56.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc09aa00640 1 -- 172.21.15.161:0/2828241237 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc094158b90 con 0x7fc0940cef80 2024-05-22T21:14:56.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc09aa00640 1 -- 172.21.15.161:0/2828241237 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0940cb1d0 con 0x7fc0940cef80 2024-05-22T21:14:56.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.099+0000 7fc08be00640 1 -- 172.21.15.161:0/2828241237 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc08c0b4040 con 0x7fc0940cef80 2024-05-22T21:14:56.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.103+0000 7fc08be00640 1 --2- 172.21.15.161:0/2828241237 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc0740666c0 0x7fc074068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:56.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.103+0000 7fc08be00640 1 -- 172.21.15.161:0/2828241237 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc08c11e270 con 0x7fc0940cef80 2024-05-22T21:14:56.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.103+0000 7fc093e00640 1 --2- 172.21.15.161:0/2828241237 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc0740666c0 0x7fc074068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:56.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.103+0000 7fc093e00640 1 --2- 172.21.15.161:0/2828241237 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc0740666c0 0x7fc074068b80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc0840093f0 tx=0x7fc084008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:56.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.107+0000 7fc08be00640 1 -- 172.21.15.161:0/2828241237 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc08c0eb600 con 0x7fc0940cef80 2024-05-22T21:14:56.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.295+0000 7fc09aa00640 1 -- 172.21.15.161:0/2828241237 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc0940cb1d0 con 0x7fc0940cef80 2024-05-22T21:14:56.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.295+0000 7fc08be00640 1 -- 172.21.15.161:0/2828241237 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc08c0b2060 con 0x7fc0940cef80 2024-05-22T21:14:56.299 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:14:56.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.295+0000 7fc09aa00640 1 -- 172.21.15.161:0/2828241237 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc0740666c0 msgr2=0x7fc074068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:56.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.295+0000 7fc09aa00640 1 --2- 172.21.15.161:0/2828241237 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc0740666c0 0x7fc074068b80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc0840093f0 tx=0x7fc084008040 comp rx=0 tx=0).stop 2024-05-22T21:14:56.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.295+0000 7fc09aa00640 1 -- 172.21.15.161:0/2828241237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0940cef80 msgr2=0x7fc0941e41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:56.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.295+0000 7fc09aa00640 1 --2- 172.21.15.161:0/2828241237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0940cef80 0x7fc0941e41c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc08c002240 tx=0x7fc08c009750 comp rx=0 tx=0).stop 2024-05-22T21:14:56.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.295+0000 7fc09aa00640 1 -- 172.21.15.161:0/2828241237 shutdown_connections 2024-05-22T21:14:56.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.295+0000 7fc09aa00640 1 --2- 172.21.15.161:0/2828241237 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc0740666c0 0x7fc074068b80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:56.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.295+0000 7fc09aa00640 1 --2- 172.21.15.161:0/2828241237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0940cef80 0x7fc0941e41c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:56.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.295+0000 7fc09aa00640 1 --2- 172.21.15.161:0/2828241237 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc094148530 0x7fc0940cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:56.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.295+0000 7fc09aa00640 1 --2- 172.21.15.161:0/2828241237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc078004d10 0x7fc094144880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:56.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.295+0000 7fc09aa00640 1 -- 172.21.15.161:0/2828241237 >> 172.21.15.161:0/2828241237 conn(0x7fc0940b0bc0 msgr2=0x7fc0940b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:56.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.295+0000 7fc09aa00640 1 -- 172.21.15.161:0/2828241237 shutdown_connections 2024-05-22T21:14:56.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:56.295+0000 7fc09aa00640 1 -- 172.21.15.161:0/2828241237 wait complete. 2024-05-22T21:14:56.320 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:14:56.320 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:14:56.320 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-05-22T21:14:56.321 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-22T21:14:56.321 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-22T21:14:56.321 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:14:56.321 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:14:56.321 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:14:56.321 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:14:56.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:56.321 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-05-22T21:14:56.330 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:14:56.330 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:14:56.330 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:14:56.330 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:56.330 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-05-22T21:14:56.330 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:56.330 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:14:56.330 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:56.331 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2024-05-22T21:14:56.331 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:14:56.331 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:56.331 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2024-05-22T21:14:56.331 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:14:56.331 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:56.331 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:14:56.331 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:56.331 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:14:56.331 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42396sec preferred_lft 42396sec 2024-05-22T21:14:56.331 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-05-22T21:14:56.331 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:56.331 INFO:teuthology.orchestra.run.smithi161.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:14:56.332 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 02:42:b9:0f:57:df brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:56.332 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:14:56.332 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:56.332 INFO:teuthology.orchestra.run.smithi161.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:14:56.332 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:56.332 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:14:56.332 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:56.332 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:14:56.332 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:56.332 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:14:56.332 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-22T21:14:56.332 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:14:56.332 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:56.332 INFO:teuthology.orchestra.run.smithi161.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:14:56.333 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-22T21:14:56.333 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:14:56.333 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:56.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:56.334 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:14:56.398 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-05-22T21:14:56.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:56.400 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-05-22T21:14:56.456 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-05-22T21:14:56.457 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:14:56.457 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:14:56.457 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:14:56.457 INFO:teuthology.orchestra.run.smithi161.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:14:56.457 INFO:teuthology.orchestra.run.smithi161.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:14:56.457 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:14:56.457 INFO:teuthology.orchestra.run.smithi161.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:14:56.457 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:14:56.457 INFO:teuthology.orchestra.run.smithi161.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:14:56.457 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:14:56.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:56.458 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:14:56.458 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:14:56.458 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:14:56.458 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:14:56.458 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:14:56.458 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:14:56.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:56.549 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-05-22T21:14:56.602 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-22T21:14:56.603 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-22T21:14:56.603 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:14:56.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:56.604 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2024-05-22T21:14:56.669 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:14:56.669 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-22T21:14:56.721 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:14:56.722 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:14:56.722 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:14:56.772 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:14:56.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:56.772 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:14:56.861 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:14:56.861 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:14:56.861 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:14:56.876 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:14:56.876 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:14:56.876 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:14:56.876 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:14:56.877 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:14:56.889 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:14:56.889 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:14:56.889 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:14:56.889 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:14:56.889 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:14:56.889 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:14:56.890 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:14:56.890 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:14:56.893 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:14:56.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:56.894 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:14:56.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:56.957 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:14:57.021 INFO:teuthology.orchestra.run.smithi161.stdout:client5500 2024-05-22T21:14:57.021 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:14:57.021 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:57.072 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:14:57.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.243+0000 7f0f61000640 1 Processor -- start 2024-05-22T21:14:57.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.243+0000 7f0f61000640 1 -- start start 2024-05-22T21:14:57.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.243+0000 7f0f61000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0f5c0715a0 0x7f0f5c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:57.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.243+0000 7f0f61000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5c1442a0 con 0x7f0f5c143ec0 2024-05-22T21:14:57.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.243+0000 7f0f61000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5c070500 con 0x7f0f5c15ec90 2024-05-22T21:14:57.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.243+0000 7f0f61000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5c0707e0 con 0x7f0f5c0715a0 2024-05-22T21:14:57.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0f5c143ec0 0x7f0f5c06fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56120/0 (socket says 172.21.15.161:56120) 2024-05-22T21:14:57.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59e00640 1 -- 172.21.15.161:0/77125485 learned_addr learned my addr 172.21.15.161:0/77125485 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:57.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f51400640 1 --2- 172.21.15.161:0/77125485 >> v2:172.21.15.179:3300/0 conn(0x7f0f5c0715a0 0x7f0f5c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:57.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 -- 172.21.15.161:0/77125485 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804988711 0 0) 0x7f0f5c1442a0 con 0x7f0f5c143ec0 2024-05-22T21:14:57.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 -- 172.21.15.161:0/77125485 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f40003530 con 0x7f0f5c143ec0 2024-05-22T21:14:57.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 -- 172.21.15.161:0/77125485 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1853830642 0 0) 0x7f0f5c070500 con 0x7f0f5c15ec90 2024-05-22T21:14:57.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 -- 172.21.15.161:0/77125485 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f5c1442a0 con 0x7f0f5c15ec90 2024-05-22T21:14:57.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 -- 172.21.15.161:0/77125485 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3068094679 0 0) 0x7f0f40003530 con 0x7f0f5c143ec0 2024-05-22T21:14:57.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 -- 172.21.15.161:0/77125485 >> v2:172.21.15.179:3300/0 conn(0x7f0f5c0715a0 msgr2=0x7f0f5c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:57.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 --2- 172.21.15.161:0/77125485 >> v2:172.21.15.179:3300/0 conn(0x7f0f5c0715a0 0x7f0f5c15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:57.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 -- 172.21.15.161:0/77125485 >> v1:172.21.15.179:6789/0 conn(0x7f0f5c15ec90 legacy=0x7f0f5c0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:57.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 -- 172.21.15.161:0/77125485 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c070ac0 con 0x7f0f5c143ec0 2024-05-22T21:14:57.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 -- 172.21.15.161:0/77125485 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f0f4c001980 con 0x7f0f5c143ec0 2024-05-22T21:14:57.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 -- 172.21.15.161:0/77125485 >> v1:172.21.15.161:6789/0 conn(0x7f0f5c143ec0 legacy=0x7f0f5c06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:57.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 --2- 172.21.15.161:0/77125485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f40003b60 0x7f0f40003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:57.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 --2- 172.21.15.161:0/77125485 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f400047d0 0x7f0f40004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:57.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 --2- 172.21.15.161:0/77125485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f40005190 0x7f0f4000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:57.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 -- 172.21.15.161:0/77125485 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0707e0 con 0x7f0f40005190 2024-05-22T21:14:57.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 -- 172.21.15.161:0/77125485 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4000c060 con 0x7f0f40003b60 2024-05-22T21:14:57.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59400640 1 -- 172.21.15.161:0/77125485 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4000c220 con 0x7f0f400047d0 2024-05-22T21:14:57.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f5a800640 1 --2- 172.21.15.161:0/77125485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f40005190 0x7f0f4000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:57.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f51400640 1 --2- 172.21.15.161:0/77125485 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f400047d0 0x7f0f40004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:57.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f59e00640 1 --2- 172.21.15.161:0/77125485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f40003b60 0x7f0f40003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:57.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f5a800640 1 -- 172.21.15.161:0/77125485 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f400047d0 msgr2=0x7f0f40004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:57.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f5a800640 1 --2- 172.21.15.161:0/77125485 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f400047d0 0x7f0f40004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:57.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f5a800640 1 -- 172.21.15.161:0/77125485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f40003b60 msgr2=0x7f0f40003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:57.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f5a800640 1 --2- 172.21.15.161:0/77125485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f40003b60 0x7f0f40003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:57.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.247+0000 7f0f5a800640 1 -- 172.21.15.161:0/77125485 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f40003530 con 0x7f0f40005190 2024-05-22T21:14:57.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.251+0000 7f0f5a800640 1 -- 172.21.15.161:0/77125485 reap_dead start 2024-05-22T21:14:57.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.251+0000 7f0f5a800640 1 --2- 172.21.15.161:0/77125485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f40005190 0x7f0f4000b990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0f540018d0 tx=0x7f0f5401c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85ded01bb72076b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:57.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.251+0000 7f0f59400640 1 -- 172.21.15.161:0/77125485 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f540050d0 con 0x7f0f40005190 2024-05-22T21:14:57.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.251+0000 7f0f59400640 1 -- 172.21.15.161:0/77125485 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f54005230 con 0x7f0f40005190 2024-05-22T21:14:57.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.251+0000 7f0f59400640 1 -- 172.21.15.161:0/77125485 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f54003600 con 0x7f0f40005190 2024-05-22T21:14:57.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.251+0000 7f0f61000640 1 -- 172.21.15.161:0/77125485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f40005190 msgr2=0x7f0f4000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:57.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.251+0000 7f0f61000640 1 --2- 172.21.15.161:0/77125485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f40005190 0x7f0f4000b990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0f540018d0 tx=0x7f0f5401c580 comp rx=0 tx=0).stop 2024-05-22T21:14:57.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.251+0000 7f0f61000640 1 -- 172.21.15.161:0/77125485 shutdown_connections 2024-05-22T21:14:57.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.251+0000 7f0f61000640 1 --2- 172.21.15.161:0/77125485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f40005190 0x7f0f4000b990 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:57.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.251+0000 7f0f61000640 1 -- 172.21.15.161:0/77125485 >> 172.21.15.161:0/77125485 conn(0x7f0f5c0b0bc0 msgr2=0x7f0f5c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:57.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.251+0000 7f0f61000640 1 -- 172.21.15.161:0/77125485 shutdown_connections 2024-05-22T21:14:57.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.251+0000 7f0f61000640 1 -- 172.21.15.161:0/77125485 wait complete. 2024-05-22T21:14:57.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.251+0000 7f0f61000640 1 Processor -- start 2024-05-22T21:14:57.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.251+0000 7f0f61000640 1 -- start start 2024-05-22T21:14:57.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f61000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f40005190 0x7f0f5c15d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:57.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f61000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f5c15a280 0x7f0f5c13e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:57.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f61000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5c13e890 0x7f0f5c0acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:57.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f61000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c15e8b0 con 0x7f0f5c13e890 2024-05-22T21:14:57.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f61000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c13ed10 con 0x7f0f5c15a280 2024-05-22T21:14:57.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f61000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c15cf70 con 0x7f0f40005190 2024-05-22T21:14:57.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f5a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5c13e890 0x7f0f5c0acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:57.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f5a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5c13e890 0x7f0f5c0acae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45914/0 (socket says 172.21.15.161:45914) 2024-05-22T21:14:57.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f5a800640 1 -- 172.21.15.161:0/257706667 learned_addr learned my addr 172.21.15.161:0/257706667 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:57.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f51400640 1 --2- 172.21.15.161:0/257706667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f5c15a280 0x7f0f5c13e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:57.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f59e00640 1 --2- 172.21.15.161:0/257706667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f40005190 0x7f0f5c15d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:57.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f5a800640 1 -- 172.21.15.161:0/257706667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f40005190 msgr2=0x7f0f5c15d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:57.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f5a800640 1 --2- 172.21.15.161:0/257706667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f40005190 0x7f0f5c15d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:57.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f5a800640 1 -- 172.21.15.161:0/257706667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f5c15a280 msgr2=0x7f0f5c13e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:57.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f5a800640 1 --2- 172.21.15.161:0/257706667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f5c15a280 0x7f0f5c13e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:57.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f5a800640 1 -- 172.21.15.161:0/257706667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c15d940 con 0x7f0f5c13e890 2024-05-22T21:14:57.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f59e00640 1 --2- 172.21.15.161:0/257706667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f40005190 0x7f0f5c15d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:57.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f5a800640 1 --2- 172.21.15.161:0/257706667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5c13e890 0x7f0f5c0acae0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0f5401ce20 tx=0x7f0f40005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:57.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f52a00640 1 -- 172.21.15.161:0/257706667 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f540b1070 con 0x7f0f5c13e890 2024-05-22T21:14:57.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f52a00640 1 -- 172.21.15.161:0/257706667 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f54004e80 con 0x7f0f5c13e890 2024-05-22T21:14:57.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f61000640 1 -- 172.21.15.161:0/257706667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c0ad850 con 0x7f0f5c13e890 2024-05-22T21:14:57.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f52a00640 1 -- 172.21.15.161:0/257706667 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f540050d0 con 0x7f0f5c13e890 2024-05-22T21:14:57.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f61000640 1 -- 172.21.15.161:0/257706667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c15c920 con 0x7f0f5c13e890 2024-05-22T21:14:57.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f61000640 1 -- 172.21.15.161:0/257706667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5c067220 con 0x7f0f5c13e890 2024-05-22T21:14:57.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.255+0000 7f0f52a00640 1 -- 172.21.15.161:0/257706667 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0f540b53c0 con 0x7f0f5c13e890 2024-05-22T21:14:57.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.259+0000 7f0f52a00640 1 --2- 172.21.15.161:0/257706667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f38066690 0x7f0f38068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:57.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.259+0000 7f0f52a00640 1 -- 172.21.15.161:0/257706667 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f5411d260 con 0x7f0f5c13e890 2024-05-22T21:14:57.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.259+0000 7f0f59e00640 1 --2- 172.21.15.161:0/257706667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f38066690 0x7f0f38068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:57.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.259+0000 7f0f59e00640 1 --2- 172.21.15.161:0/257706667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f38066690 0x7f0f38068b50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c000f90 tx=0x7f0f4c001260 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:57.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.263+0000 7f0f52a00640 1 -- 172.21.15.161:0/257706667 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0f540ea660 con 0x7f0f5c13e890 2024-05-22T21:14:57.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.447+0000 7f0f61000640 1 -- 172.21.15.161:0/257706667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0f5c15d0b0 con 0x7f0f5c13e890 2024-05-22T21:14:57.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.447+0000 7f0f52a00640 1 -- 172.21.15.161:0/257706667 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0f540edc80 con 0x7f0f5c13e890 2024-05-22T21:14:57.452 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:14:57.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.447+0000 7f0f61000640 1 -- 172.21.15.161:0/257706667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f38066690 msgr2=0x7f0f38068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:57.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.447+0000 7f0f61000640 1 --2- 172.21.15.161:0/257706667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f38066690 0x7f0f38068b50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c000f90 tx=0x7f0f4c001260 comp rx=0 tx=0).stop 2024-05-22T21:14:57.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.447+0000 7f0f61000640 1 -- 172.21.15.161:0/257706667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5c13e890 msgr2=0x7f0f5c0acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:57.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.447+0000 7f0f61000640 1 --2- 172.21.15.161:0/257706667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5c13e890 0x7f0f5c0acae0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0f5401ce20 tx=0x7f0f40005800 comp rx=0 tx=0).stop 2024-05-22T21:14:57.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.447+0000 7f0f61000640 1 -- 172.21.15.161:0/257706667 shutdown_connections 2024-05-22T21:14:57.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.447+0000 7f0f61000640 1 --2- 172.21.15.161:0/257706667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f38066690 0x7f0f38068b50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:57.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.447+0000 7f0f61000640 1 --2- 172.21.15.161:0/257706667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5c13e890 0x7f0f5c0acae0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:57.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.447+0000 7f0f61000640 1 --2- 172.21.15.161:0/257706667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f5c15a280 0x7f0f5c13e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:57.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.447+0000 7f0f61000640 1 --2- 172.21.15.161:0/257706667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f40005190 0x7f0f5c15d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:57.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.451+0000 7f0f61000640 1 -- 172.21.15.161:0/257706667 >> 172.21.15.161:0/257706667 conn(0x7f0f5c0b0bc0 msgr2=0x7f0f5c13f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:57.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.451+0000 7f0f61000640 1 -- 172.21.15.161:0/257706667 shutdown_connections 2024-05-22T21:14:57.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:57.451+0000 7f0f61000640 1 -- 172.21.15.161:0/257706667 wait complete. 2024-05-22T21:14:57.472 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:14:57.472 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:14:57.472 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-22T21:14:57.472 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-22T21:14:57.472 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-22T21:14:57.472 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:14:57.473 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:14:57.473 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:14:57.473 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:14:57.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:57.473 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-22T21:14:57.483 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:14:57.483 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:14:57.484 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:14:57.484 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:57.484 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-22T21:14:57.484 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:57.484 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:14:57.484 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:57.484 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f0 2024-05-22T21:14:57.484 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:14:57.484 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:57.484 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f1 2024-05-22T21:14:57.484 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:14:57.485 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:57.485 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:14:57.485 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:57.485 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:14:57.485 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42387sec preferred_lft 42387sec 2024-05-22T21:14:57.485 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-22T21:14:57.485 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:57.485 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:14:57.485 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:bd:4e:25:3d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:57.485 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:14:57.485 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:57.485 INFO:teuthology.orchestra.run.smithi179.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:14:57.485 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:14:57.486 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:14:57.486 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:57.486 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:14:57.486 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:57.486 INFO:teuthology.orchestra.run.smithi179.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:14:57.486 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-22T21:14:57.486 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:14:57.486 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:57.486 INFO:teuthology.orchestra.run.smithi179.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:14:57.486 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-22T21:14:57.486 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:14:57.486 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:14:57.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:57.487 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:14:57.550 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-22T21:14:57.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:57.553 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-22T21:14:57.609 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-22T21:14:57.609 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:14:57.609 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:14:57.609 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:14:57.609 INFO:teuthology.orchestra.run.smithi179.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:14:57.609 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:14:57.609 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:14:57.609 INFO:teuthology.orchestra.run.smithi179.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:14:57.609 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:14:57.609 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:14:57.610 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:14:57.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:57.610 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:14:57.610 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:14:57.610 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:14:57.610 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:14:57.610 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:14:57.610 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:14:57.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:57.707 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-22T21:14:57.716 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-22T21:14:57.716 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-22T21:14:57.717 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:14:57.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:57.717 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-22T21:14:57.779 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:14:57.780 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-22T21:14:57.831 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:14:57.831 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:14:57.832 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:14:57.882 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:14:57.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:57.883 DEBUG:teuthology.orchestra.run.smithi179:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:14:57.975 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:14:57.975 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:14:57.975 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:14:57.991 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:14:57.991 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:14:57.991 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:14:57.991 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:14:57.991 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:14:58.000 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:14:58.000 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:14:58.000 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:14:58.000 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:14:58.000 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:14:58.000 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:14:58.000 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:14:58.000 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:14:58.004 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:14:58.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:58.004 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:14:58.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:58.023 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:14:58.083 INFO:teuthology.orchestra.run.smithi179.stdout:client5661 2024-05-22T21:14:58.083 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:14:58.083 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:58.132 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:14:58.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.263+0000 7f2df9600640 1 Processor -- start 2024-05-22T21:14:58.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.263+0000 7f2df9600640 1 -- start start 2024-05-22T21:14:58.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.263+0000 7f2df9600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2df415dd50 0x7f2df406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:58.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.263+0000 7f2df9600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2df415e130 0x7f2df415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:58.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.263+0000 7f2df9600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df4071670 con 0x7f2df415d410 2024-05-22T21:14:58.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.263+0000 7f2df9600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df415d8a0 con 0x7f2df415dd50 2024-05-22T21:14:58.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.263+0000 7f2df9600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df406de40 con 0x7f2df415e130 2024-05-22T21:14:58.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.267+0000 7f2df2000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2df415e130 0x7f2df415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:58.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.267+0000 7f2df3400640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f2df415d410 0x7f2df415a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:35924/0 (socket says 172.21.15.161:35924) 2024-05-22T21:14:58.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.267+0000 7f2df3400640 1 -- 172.21.15.161:0/1185946106 learned_addr learned my addr 172.21.15.161:0/1185946106 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:14:58.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.267+0000 7f2df2000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2df415e130 0x7f2df415ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:43240/0 (socket says 172.21.15.161:43240) 2024-05-22T21:14:58.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.267+0000 7f2df2a00640 1 --2- 172.21.15.161:0/1185946106 >> v2:172.21.15.179:3300/0 conn(0x7f2df415dd50 0x7f2df406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:58.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.267+0000 7f2df1600640 1 -- 172.21.15.161:0/1185946106 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679019009 0 0) 0x7f2df4071670 con 0x7f2df415d410 2024-05-22T21:14:58.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.267+0000 7f2df1600640 1 -- 172.21.15.161:0/1185946106 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dd8003530 con 0x7f2df415d410 2024-05-22T21:14:58.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.267+0000 7f2df2000640 1 -- 172.21.15.161:0/1185946106 >> v2:172.21.15.179:3300/0 conn(0x7f2df415dd50 msgr2=0x7f2df406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:14:58.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.267+0000 7f2df2000640 1 --2- 172.21.15.161:0/1185946106 >> v2:172.21.15.179:3300/0 conn(0x7f2df415dd50 0x7f2df406d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:58.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.267+0000 7f2df2000640 1 -- 172.21.15.161:0/1185946106 >> v1:172.21.15.179:6789/0 conn(0x7f2df415d410 legacy=0x7f2df415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:58.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.267+0000 7f2df2000640 1 -- 172.21.15.161:0/1185946106 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df4070a60 con 0x7f2df415e130 2024-05-22T21:14:58.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.267+0000 7f2df2a00640 1 --2- 172.21.15.161:0/1185946106 >> v2:172.21.15.179:3300/0 conn(0x7f2df415dd50 0x7f2df406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:14:58.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.267+0000 7f2df2000640 1 --2- 172.21.15.161:0/1185946106 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2df415e130 0x7f2df415ced0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2de00097c0 tx=0x7f2de001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2b3c2e624b5d392 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:58.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df1600640 1 -- 172.21.15.161:0/1185946106 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de00ac640 con 0x7f2df415e130 2024-05-22T21:14:58.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df1600640 1 -- 172.21.15.161:0/1185946106 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de00ac7a0 con 0x7f2df415e130 2024-05-22T21:14:58.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df1600640 1 -- 172.21.15.161:0/1185946106 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de00acb30 con 0x7f2df415e130 2024-05-22T21:14:58.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df9600640 1 -- 172.21.15.161:0/1185946106 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2df415e130 msgr2=0x7f2df415ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:58.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df9600640 1 --2- 172.21.15.161:0/1185946106 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2df415e130 0x7f2df415ced0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2de00097c0 tx=0x7f2de001c3e0 comp rx=0 tx=0).stop 2024-05-22T21:14:58.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df9600640 1 -- 172.21.15.161:0/1185946106 shutdown_connections 2024-05-22T21:14:58.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df9600640 1 --2- 172.21.15.161:0/1185946106 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2df415e130 0x7f2df415ced0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:58.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df9600640 1 --2- 172.21.15.161:0/1185946106 >> v2:172.21.15.179:3300/0 conn(0x7f2df415dd50 0x7f2df406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:58.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df9600640 1 -- 172.21.15.161:0/1185946106 >> 172.21.15.161:0/1185946106 conn(0x7f2df40b0bc0 msgr2=0x7f2df40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:58.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df9600640 1 -- 172.21.15.161:0/1185946106 shutdown_connections 2024-05-22T21:14:58.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df9600640 1 -- 172.21.15.161:0/1185946106 wait complete. 2024-05-22T21:14:58.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df9600640 1 Processor -- start 2024-05-22T21:14:58.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df9600640 1 -- start start 2024-05-22T21:14:58.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df9600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2df415d410 0x7f2df414d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:58.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df9600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2df415dd50 0x7f2df414cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:58.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df9600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2df415e130 0x7f2df41551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:58.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df9600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40b2fa0 con 0x7f2df415d410 2024-05-22T21:14:58.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df9600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df414d290 con 0x7f2df415dd50 2024-05-22T21:14:58.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df9600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40b1ce0 con 0x7f2df415e130 2024-05-22T21:14:58.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2df415d410 0x7f2df414d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:58.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2df415d410 0x7f2df414d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45926/0 (socket says 172.21.15.161:45926) 2024-05-22T21:14:58.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.271+0000 7f2df2a00640 1 -- 172.21.15.161:0/469045829 learned_addr learned my addr 172.21.15.161:0/469045829 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:14:58.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.275+0000 7f2df2000640 1 --2- 172.21.15.161:0/469045829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2df415dd50 0x7f2df414cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:58.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.275+0000 7f2df3400640 1 --2- 172.21.15.161:0/469045829 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2df415e130 0x7f2df41551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:58.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.275+0000 7f2df2a00640 1 -- 172.21.15.161:0/469045829 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2df415e130 msgr2=0x7f2df41551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:58.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.275+0000 7f2df2a00640 1 --2- 172.21.15.161:0/469045829 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2df415e130 0x7f2df41551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:58.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.275+0000 7f2df2a00640 1 -- 172.21.15.161:0/469045829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2df415dd50 msgr2=0x7f2df414cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:58.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.275+0000 7f2df2a00640 1 --2- 172.21.15.161:0/469045829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2df415dd50 0x7f2df414cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:58.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.275+0000 7f2df2a00640 1 -- 172.21.15.161:0/469045829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de4009a70 con 0x7f2df415d410 2024-05-22T21:14:58.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.275+0000 7f2df3400640 1 --2- 172.21.15.161:0/469045829 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2df415e130 0x7f2df41551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:14:58.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.275+0000 7f2df2a00640 1 --2- 172.21.15.161:0/469045829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2df415d410 0x7f2df414d4d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2de400efc0 tx=0x7f2de400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:58.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.275+0000 7f2deaa00640 1 -- 172.21.15.161:0/469045829 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4014680 con 0x7f2df415d410 2024-05-22T21:14:58.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.275+0000 7f2deaa00640 1 -- 172.21.15.161:0/469045829 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de400f040 con 0x7f2df415d410 2024-05-22T21:14:58.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.275+0000 7f2df9600640 1 -- 172.21.15.161:0/469045829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de0009420 con 0x7f2df415d410 2024-05-22T21:14:58.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.275+0000 7f2deaa00640 1 -- 172.21.15.161:0/469045829 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4014a00 con 0x7f2df415d410 2024-05-22T21:14:58.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.275+0000 7f2df9600640 1 -- 172.21.15.161:0/469045829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df414f940 con 0x7f2df415d410 2024-05-22T21:14:58.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.275+0000 7f2df9600640 1 -- 172.21.15.161:0/469045829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df4067220 con 0x7f2df415d410 2024-05-22T21:14:58.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.275+0000 7f2deaa00640 1 -- 172.21.15.161:0/469045829 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2de4014b60 con 0x7f2df415d410 2024-05-22T21:14:58.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.279+0000 7f2deaa00640 1 --2- 172.21.15.161:0/469045829 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2dd4066690 0x7f2dd4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:14:58.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.279+0000 7f2deaa00640 1 -- 172.21.15.161:0/469045829 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2de4085100 con 0x7f2df415d410 2024-05-22T21:14:58.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.279+0000 7f2df2000640 1 --2- 172.21.15.161:0/469045829 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2dd4066690 0x7f2dd4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:14:58.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.279+0000 7f2df2000640 1 --2- 172.21.15.161:0/469045829 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2dd4066690 0x7f2dd4068b50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2de00ae000 tx=0x7f2de0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:14:58.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.283+0000 7f2deaa00640 1 -- 172.21.15.161:0/469045829 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2de4052460 con 0x7f2df415d410 2024-05-22T21:14:58.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.471+0000 7f2df9600640 1 -- 172.21.15.161:0/469045829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2df406c310 con 0x7f2df415d410 2024-05-22T21:14:58.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.471+0000 7f2deaa00640 1 -- 172.21.15.161:0/469045829 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f2de4055a80 con 0x7f2df415d410 2024-05-22T21:14:58.475 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:14:58.475 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/1767934947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1767934947},{"type":"v1","addr":"172.21.15.161:6836","nonce":1767934947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1873337287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1873337287},{"type":"v1","addr":"172.21.15.179:6834","nonce":1873337287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1815838236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":1815838236},{"type":"v1","addr":"172.21.15.161:6837","nonce":1815838236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:50.052292+0000","modified":"2024-05-22T21:14:53.058881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5374":{"gid":5374,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/389418050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":389418050},{"type":"v1","addr":"172.21.15.179:6835","nonce":389418050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-22T21:14:58.475 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 23 2024-05-22T21:14:58.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.471+0000 7f2df9600640 1 -- 172.21.15.161:0/469045829 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2dd4066690 msgr2=0x7f2dd4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:58.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.471+0000 7f2df9600640 1 --2- 172.21.15.161:0/469045829 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2dd4066690 0x7f2dd4068b50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2de00ae000 tx=0x7f2de0008040 comp rx=0 tx=0).stop 2024-05-22T21:14:58.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.471+0000 7f2df9600640 1 -- 172.21.15.161:0/469045829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2df415d410 msgr2=0x7f2df414d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:14:58.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.471+0000 7f2df9600640 1 --2- 172.21.15.161:0/469045829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2df415d410 0x7f2df414d4d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2de400efc0 tx=0x7f2de400c490 comp rx=0 tx=0).stop 2024-05-22T21:14:58.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.471+0000 7f2df9600640 1 -- 172.21.15.161:0/469045829 shutdown_connections 2024-05-22T21:14:58.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.471+0000 7f2df9600640 1 --2- 172.21.15.161:0/469045829 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2dd4066690 0x7f2dd4068b50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:58.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.471+0000 7f2df9600640 1 --2- 172.21.15.161:0/469045829 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2df415e130 0x7f2df41551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:58.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.471+0000 7f2df9600640 1 --2- 172.21.15.161:0/469045829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2df415dd50 0x7f2df414cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:58.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.471+0000 7f2df9600640 1 --2- 172.21.15.161:0/469045829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2df415d410 0x7f2df414d4d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:14:58.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.471+0000 7f2df9600640 1 -- 172.21.15.161:0/469045829 >> 172.21.15.161:0/469045829 conn(0x7f2df40b0bc0 msgr2=0x7f2df4151980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:14:58.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.471+0000 7f2df9600640 1 -- 172.21.15.161:0/469045829 shutdown_connections 2024-05-22T21:14:58.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:14:58.471+0000 7f2df9600640 1 -- 172.21.15.161:0/469045829 wait complete. 2024-05-22T21:14:58.496 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-05-22T21:14:58.653 INFO:teuthology.orchestra.run.smithi179.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":5374,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.5819551020000002,"mdsmap_epoch":23,"osdmap_epoch":47,"osdmap_epoch_barrier":47,"uptime":12.808620466000001} 2024-05-22T21:14:58.653 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 5374, "mdsmap_epoch": 23, "osdmap_epoch": 47, "osdmap_epoch_barrier": 47, "rank_uptime": 8.581955102, "state": "up:active", "uptime": 12.808620466, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:14:58.654 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-05-22T21:14:58.806 INFO:teuthology.orchestra.run.smithi161.stdout:{"osd_mon_report_interval":"5"} 2024-05-22T21:14:58.806 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-05-22T21:14:58.807 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2024-05-22T21:14:58.807 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:14:58.807 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:14:58.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:14:58.816 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2024-05-22T21:15:01.129 INFO:teuthology.orchestra.run.smithi161.stderr:265+0 records in 2024-05-22T21:15:01.129 INFO:teuthology.orchestra.run.smithi161.stderr:265+0 records out 2024-05-22T21:15:01.129 INFO:teuthology.orchestra.run.smithi161.stderr:277872640 bytes (278 MB, 265 MiB) copied, 2.2604 s, 123 MB/s 2024-05-22T21:15:01.130 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:15:01.131 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:15:01.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:01.139 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2024-05-22T21:15:03.569 INFO:teuthology.orchestra.run.smithi161.stderr:292+0 records in 2024-05-22T21:15:03.569 INFO:teuthology.orchestra.run.smithi161.stderr:292+0 records out 2024-05-22T21:15:03.569 INFO:teuthology.orchestra.run.smithi161.stderr:306184192 bytes (306 MB, 292 MiB) copied, 2.37811 s, 129 MB/s 2024-05-22T21:15:03.570 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-05-22T21:15:03.570 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:15:03.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.703+0000 7f4eaa600640 1 Processor -- start 2024-05-22T21:15:03.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.703+0000 7f4eaa600640 1 -- start start 2024-05-22T21:15:03.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.703+0000 7f4eaa600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4ea415f3a0 0x7f4ea415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:03.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.703+0000 7f4eaa600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea41442a0 con 0x7f4ea40715a0 2024-05-22T21:15:03.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.703+0000 7f4eaa600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea4070500 con 0x7f4ea4143ec0 2024-05-22T21:15:03.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.703+0000 7f4eaa600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea40707e0 con 0x7f4ea415f3a0 2024-05-22T21:15:03.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.703+0000 7f4ea2a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f4ea40715a0 0x7f4ea415ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56134/0 (socket says 172.21.15.161:56134) 2024-05-22T21:15:03.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.703+0000 7f4ea2a00640 1 -- 172.21.15.161:0/1506748849 learned_addr learned my addr 172.21.15.161:0/1506748849 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:03.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.703+0000 7f4ea3e00640 1 --2- 172.21.15.161:0/1506748849 >> v2:172.21.15.179:3300/0 conn(0x7f4ea415f3a0 0x7f4ea415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:03.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.703+0000 7f4ea2000640 1 -- 172.21.15.161:0/1506748849 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532856077 0 0) 0x7f4ea41442a0 con 0x7f4ea40715a0 2024-05-22T21:15:03.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.703+0000 7f4ea2000640 1 -- 172.21.15.161:0/1506748849 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e88003530 con 0x7f4ea40715a0 2024-05-22T21:15:03.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.703+0000 7f4ea3e00640 1 -- 172.21.15.161:0/1506748849 >> v1:172.21.15.179:6789/0 conn(0x7f4ea4143ec0 legacy=0x7f4ea406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:03.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.703+0000 7f4ea2000640 1 -- 172.21.15.161:0/1506748849 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214976489 0 0) 0x7f4ea4070500 con 0x7f4ea4143ec0 2024-05-22T21:15:03.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.703+0000 7f4ea3e00640 1 -- 172.21.15.161:0/1506748849 >> v1:172.21.15.161:6789/0 conn(0x7f4ea40715a0 legacy=0x7f4ea415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:03.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.703+0000 7f4ea3e00640 1 -- 172.21.15.161:0/1506748849 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea4070ac0 con 0x7f4ea415f3a0 2024-05-22T21:15:03.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.707+0000 7f4ea3e00640 1 --2- 172.21.15.161:0/1506748849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ea415f3a0 0x7f4ea415f830 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0097c0 tx=0x7f4e9c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=488a464539eeeff3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:03.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.707+0000 7f4ea2000640 1 -- 172.21.15.161:0/1506748849 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0ab670 con 0x7f4ea415f3a0 2024-05-22T21:15:03.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.707+0000 7f4ea2000640 1 -- 172.21.15.161:0/1506748849 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c0ab7d0 con 0x7f4ea415f3a0 2024-05-22T21:15:03.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.707+0000 7f4ea2000640 1 -- 172.21.15.161:0/1506748849 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0abb40 con 0x7f4ea415f3a0 2024-05-22T21:15:03.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.707+0000 7f4eaa600640 1 -- 172.21.15.161:0/1506748849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ea415f3a0 msgr2=0x7f4ea415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:03.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.707+0000 7f4eaa600640 1 --2- 172.21.15.161:0/1506748849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ea415f3a0 0x7f4ea415f830 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0097c0 tx=0x7f4e9c01c470 comp rx=0 tx=0).stop 2024-05-22T21:15:03.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.707+0000 7f4eaa600640 1 -- 172.21.15.161:0/1506748849 shutdown_connections 2024-05-22T21:15:03.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.707+0000 7f4eaa600640 1 --2- 172.21.15.161:0/1506748849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ea415f3a0 0x7f4ea415f830 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:03.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.707+0000 7f4eaa600640 1 -- 172.21.15.161:0/1506748849 >> 172.21.15.161:0/1506748849 conn(0x7f4ea40b0bc0 msgr2=0x7f4ea40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:03.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.707+0000 7f4eaa600640 1 -- 172.21.15.161:0/1506748849 shutdown_connections 2024-05-22T21:15:03.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.707+0000 7f4eaa600640 1 -- 172.21.15.161:0/1506748849 wait complete. 2024-05-22T21:15:03.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4eaa600640 1 Processor -- start 2024-05-22T21:15:03.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4eaa600640 1 -- start start 2024-05-22T21:15:03.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4eaa600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ea4143ec0 0x7f4ea4159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:03.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4eaa600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ea415f3a0 0x7f4ea415d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:03.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4eaa600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4ea4159f80 0x7f4ea415d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:03.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4eaa600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea415e070 con 0x7f4ea415f3a0 2024-05-22T21:15:03.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4eaa600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea415e350 con 0x7f4ea4143ec0 2024-05-22T21:15:03.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4eaa600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea415c760 con 0x7f4ea4159f80 2024-05-22T21:15:03.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4ea2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ea415f3a0 0x7f4ea415d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:03.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4ea3400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ea4143ec0 0x7f4ea4159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:03.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4ea2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ea415f3a0 0x7f4ea415d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45928/0 (socket says 172.21.15.161:45928) 2024-05-22T21:15:03.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4ea2a00640 1 -- 172.21.15.161:0/2156996632 learned_addr learned my addr 172.21.15.161:0/2156996632 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:03.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4ea3e00640 1 --2- 172.21.15.161:0/2156996632 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4ea4159f80 0x7f4ea415d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:03.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4ea2a00640 1 -- 172.21.15.161:0/2156996632 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4ea4159f80 msgr2=0x7f4ea415d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:03.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4ea2a00640 1 --2- 172.21.15.161:0/2156996632 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4ea4159f80 0x7f4ea415d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:03.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4ea2a00640 1 -- 172.21.15.161:0/2156996632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ea4143ec0 msgr2=0x7f4ea4159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:03.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4ea2a00640 1 --2- 172.21.15.161:0/2156996632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ea4143ec0 0x7f4ea4159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:03.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4ea2a00640 1 -- 172.21.15.161:0/2156996632 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c009420 con 0x7f4ea415f3a0 2024-05-22T21:15:03.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4ea2a00640 1 --2- 172.21.15.161:0/2156996632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ea415f3a0 0x7f4ea415d240 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000c450 tx=0x7f4e9000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:03.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4e9b400640 1 -- 172.21.15.161:0/2156996632 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90005cf0 con 0x7f4ea415f3a0 2024-05-22T21:15:03.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4e9b400640 1 -- 172.21.15.161:0/2156996632 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e90005e50 con 0x7f4ea415f3a0 2024-05-22T21:15:03.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4e9b400640 1 -- 172.21.15.161:0/2156996632 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90007290 con 0x7f4ea415f3a0 2024-05-22T21:15:03.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4eaa600640 1 -- 172.21.15.161:0/2156996632 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea415ca40 con 0x7f4ea415f3a0 2024-05-22T21:15:03.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4eaa600640 1 -- 172.21.15.161:0/2156996632 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea415dd60 con 0x7f4ea415f3a0 2024-05-22T21:15:03.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.711+0000 7f4eaa600640 1 -- 172.21.15.161:0/2156996632 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea415cf60 con 0x7f4ea415f3a0 2024-05-22T21:15:03.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.715+0000 7f4e9b400640 1 -- 172.21.15.161:0/2156996632 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4e900073f0 con 0x7f4ea415f3a0 2024-05-22T21:15:03.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.715+0000 7f4e9b400640 1 --2- 172.21.15.161:0/2156996632 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4e840666c0 0x7f4e84068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:03.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.715+0000 7f4e9b400640 1 -- 172.21.15.161:0/2156996632 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e90083730 con 0x7f4ea415f3a0 2024-05-22T21:15:03.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.715+0000 7f4ea3400640 1 --2- 172.21.15.161:0/2156996632 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4e840666c0 0x7f4e84068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:03.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.715+0000 7f4ea3400640 1 --2- 172.21.15.161:0/2156996632 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4e840666c0 0x7f4e84068b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4e940012f0 tx=0x7f4e94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:03.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.719+0000 7f4e9b400640 1 -- 172.21.15.161:0/2156996632 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4ea415cf60 con 0x7f4ea415f3a0 2024-05-22T21:15:03.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.875+0000 7f4eaa600640 1 -- 172.21.15.161:0/2156996632 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4ea415cf60 con 0x7f4ea415f3a0 2024-05-22T21:15:03.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.879+0000 7f4e9b400640 1 -- 172.21.15.161:0/2156996632 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v47) v1 ==== 81+0+33528 (secure 0 0 0) 0x7f4ea415cf60 con 0x7f4ea415f3a0 2024-05-22T21:15:03.883 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:03.883 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:15:03.883 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 47, 2024-05-22T21:15:03.883 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:15:03.884 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:15:03.884 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:14:51.054474+0000", 2024-05-22T21:15:03.884 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:15:03.884 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:15:03.884 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:15:03.884 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:15:03.884 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:15:03.884 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:15:03.884 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:15:03.884 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:15:03.884 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:15:03.884 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:03.884 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:15:03.885 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:15:03.885 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:15:03.885 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:15:03.885 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:15:03.885 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 7, 2024-05-22T21:15:03.885 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:15:03.885 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:15:03.885 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:15:03.885 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:15:03.885 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:15:03.885 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:15:03.885 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.885 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:15:03.886 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:15:03.886 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:15:03.886 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:15:03.886 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:15:03.886 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:03.886 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:03.886 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:03.886 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:03.886 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:03.886 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:03.886 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:03.886 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:03.887 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:03.887 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:03.887 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:15:03.887 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:15:03.887 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:15:03.887 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:15:03.887 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:15:03.887 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:03.887 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:03.887 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:03.887 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:03.887 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:03.887 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:03.887 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "17", 2024-05-22T21:15:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:03.888 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:03.889 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:03.889 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:03.889 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:03.889 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:03.889 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:03.889 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:03.889 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:03.889 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:03.889 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:03.889 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:03.889 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:03.889 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:03.889 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:03.889 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:03.890 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.890 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:03.890 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:03.890 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:03.890 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:03.890 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:03.890 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:03.890 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:03.890 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:03.890 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:03.890 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:03.890 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:03.891 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:03.891 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:15:03.891 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:15:03.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.891 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:03.891 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:15:03.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.891 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:03.891 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:15:03.891 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:15:03.892 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:15:03.892 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:15:03.892 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:15:03.892 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:03.892 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:03.892 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:03.892 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.892 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.892 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.892 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 6, 2024-05-22T21:15:03.893 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:15:03.893 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:14:48.612726+0000", 2024-05-22T21:15:03.893 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:15:03.893 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:15:03.893 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:03.893 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:03.893 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:03.893 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:03.893 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:03.893 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:03.893 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:03.893 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:03.893 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:03.893 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:03.894 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:15:03.894 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:15:03.894 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:15:03.894 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:15:03.894 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:15:03.894 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:03.894 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:03.894 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:03.894 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:03.894 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:03.894 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:03.894 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:03.895 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.895 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "46", 2024-05-22T21:15:03.895 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:03.895 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:03.895 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:03.895 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:03.895 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:03.895 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:03.895 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:03.895 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:03.895 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:03.895 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:03.895 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:03.895 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:03.896 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:03.896 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:03.896 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:03.896 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:03.896 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:03.896 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:03.896 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:03.896 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:03.896 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:03.896 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:03.896 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:03.896 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:03.896 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:03.896 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:03.897 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.897 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:03.897 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:03.897 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:03.897 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:03.897 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:03.897 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:03.897 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:03.897 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:03.897 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:03.897 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:03.897 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:03.897 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:03.898 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:15:03.898 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:15:03.898 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:15:03.898 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.898 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:03.898 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:15:03.898 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:15:03.898 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.898 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.898 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:03.898 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:15:03.898 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:15:03.898 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:15:03.899 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:15:03.899 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:15:03.899 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:03.899 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:03.899 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:03.899 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.899 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.899 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.899 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 7, 2024-05-22T21:15:03.899 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:15:03.899 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:14:49.053022+0000", 2024-05-22T21:15:03.899 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:15:03.899 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:15:03.899 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:03.899 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:03.900 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:03.900 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:03.900 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:03.900 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:03.900 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:03.900 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:03.900 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:03.900 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:03.900 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:15:03.900 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:15:03.900 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:15:03.900 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:15:03.900 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:15:03.901 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:03.901 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:03.901 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:03.901 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:03.901 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:03.901 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:03.901 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:03.901 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.901 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "47", 2024-05-22T21:15:03.901 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:03.901 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:03.901 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:03.901 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:03.902 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:03.902 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:03.902 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:03.902 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:03.902 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:03.902 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:03.902 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:03.902 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:03.902 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:03.902 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:03.902 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:03.902 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:03.903 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:03.903 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:03.903 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:03.903 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:03.903 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:03.903 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:03.903 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:03.903 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:03.903 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:03.903 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:03.903 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.903 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:03.903 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:03.903 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:03.904 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:03.904 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:03.904 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:03.904 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:03.904 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:03.904 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:03.904 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:03.904 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:03.904 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:03.904 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:15:03.904 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:15:03.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.904 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:03.904 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:15:03.905 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:15:03.905 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.905 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.905 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:03.905 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:15:03.905 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:15:03.905 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:15:03.905 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:15:03.905 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:15:03.905 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:03.905 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:03.905 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:03.905 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.906 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.906 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:03.906 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:15:03.906 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.906 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:15:03.906 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:03.906 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:03.906 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:03.906 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:03.906 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:03.906 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:03.906 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:03.906 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:03.906 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:03.907 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:03.907 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:03.907 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:03.907 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.907 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.907 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.907 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:15:03.907 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:03.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.907 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.907 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.907 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:15:03.907 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:03.907 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.908 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.908 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:03.908 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.908 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.908 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.908 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:15:03.908 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:03.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.908 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.908 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.908 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:15:03.908 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:03.908 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.908 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.909 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.909 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:03.909 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.909 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.909 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.909 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:15:03.909 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:03.909 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.909 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.909 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.909 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:15:03.909 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:03.909 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.909 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.909 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.910 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:03.910 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.910 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.910 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.910 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:15:03.910 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:03.910 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.910 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.910 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.910 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:15:03.910 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:03.910 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.911 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.911 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.911 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:15:03.911 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:15:03.911 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:15:03.911 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:15:03.911 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:03.911 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:03.911 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:03.911 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.911 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.911 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.911 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:15:03.911 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:03.912 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:03.912 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:03.912 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:03.912 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:03.912 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:03.912 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:03.912 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:03.912 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:03.912 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:03.912 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:03.912 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:03.912 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.912 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.912 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.913 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:15:03.913 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:03.913 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.913 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.913 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.913 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:15:03.913 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:03.913 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.913 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.913 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.913 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:03.913 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.913 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.914 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.914 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:15:03.914 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:03.914 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.914 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.914 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.914 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:15:03.914 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:03.914 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.914 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.914 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.914 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:03.914 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.914 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.915 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.915 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:15:03.915 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:03.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.915 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.915 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.915 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:15:03.915 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:03.915 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.915 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.915 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.915 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:03.915 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.915 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.916 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.916 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:15:03.916 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:03.916 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.916 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.916 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.916 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:15:03.916 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:03.916 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.916 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.916 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.916 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:15:03.916 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:15:03.916 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:15:03.917 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:15:03.917 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:03.917 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:03.917 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:03.917 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.917 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.917 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.917 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:15:03.917 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:03.917 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:03.917 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:03.917 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:03.917 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:03.918 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:03.918 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:03.918 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:03.918 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:03.918 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:03.918 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:03.918 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:03.918 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.918 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.918 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.918 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:15:03.918 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:03.918 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.919 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.919 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.919 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:15:03.919 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:03.919 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.919 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.919 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.919 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:03.919 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.919 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.919 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.919 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:15:03.919 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:03.919 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.919 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.920 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.920 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:15:03.920 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:03.920 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.920 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.920 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.920 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:03.920 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.920 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.920 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.920 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:15:03.920 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:03.921 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.921 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.921 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.921 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:15:03.921 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:03.921 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.921 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.921 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.921 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:03.921 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.921 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.921 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.921 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:15:03.921 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:03.922 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.922 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.922 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.922 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:15:03.922 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:03.922 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.922 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.922 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.922 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:15:03.922 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:15:03.922 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:15:03.922 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:15:03.922 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:03.923 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:03.923 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:03.923 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.923 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.923 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.923 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:15:03.923 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:03.923 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:03.923 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:03.923 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:03.923 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:15:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.924 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:15:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:15:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:03.925 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:15:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:15:03.926 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:15:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:15:03.927 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:03.928 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.928 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.928 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.928 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:15:03.928 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:03.928 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.928 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.928 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.928 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:15:03.928 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:15:03.928 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:15:03.928 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:15:03.928 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:03.928 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:03.928 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:03.929 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.929 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.929 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.929 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:15:03.929 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:03.929 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:03.929 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:03.929 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:03.929 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:03.929 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:03.929 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:03.929 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:03.929 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:03.929 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:03.930 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:03.930 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:03.930 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.930 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.930 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.930 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:15:03.930 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:03.930 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.930 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.930 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.930 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:15:03.930 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:03.930 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.931 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.931 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.931 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:03.931 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.931 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.931 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.931 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:15:03.931 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:03.931 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.931 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.931 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.931 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:15:03.931 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:03.932 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.932 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.932 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.932 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:03.932 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.932 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.932 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.932 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:15:03.932 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:03.932 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.932 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.932 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.932 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:15:03.932 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:03.932 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.933 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.933 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.933 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:03.933 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.933 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.933 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.933 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:15:03.933 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:03.933 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.933 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.933 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.933 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:15:03.933 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:15:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:15:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:15:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:15:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.934 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:15:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:03.935 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:03.936 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.936 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.936 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.936 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:15:03.936 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:03.936 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.936 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.936 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.936 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:15:03.936 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:03.936 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.936 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.937 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.937 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:03.937 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.937 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.937 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.937 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:15:03.937 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:03.937 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.937 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.937 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.937 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:15:03.937 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:03.938 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.938 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.938 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.938 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:03.938 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.938 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.938 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.938 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:15:03.938 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:03.938 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.938 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.938 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.938 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:15:03.939 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:03.939 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.939 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.939 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.939 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:03.939 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.939 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.939 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.939 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:15:03.939 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:03.939 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.939 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.939 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.940 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:15:03.940 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:03.940 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.940 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.940 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.940 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:15:03.940 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:15:03.940 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:15:03.940 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:15:03.940 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:03.940 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:03.940 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:03.940 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.941 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.941 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.941 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:15:03.941 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:03.941 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:03.941 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:03.941 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:03.941 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:03.941 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:03.941 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:03.941 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:03.941 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:03.941 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:03.942 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:03.942 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:03.942 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.942 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.942 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.942 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:15:03.942 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:03.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.942 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.942 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.942 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:15:03.942 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:03.942 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.943 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.943 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:03.943 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.943 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.943 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.943 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:15:03.943 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:03.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.943 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.943 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.943 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:15:03.943 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:03.943 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.944 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.944 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:03.944 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.944 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.944 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.944 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:15:03.944 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:03.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.944 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.944 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.944 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:15:03.944 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:03.944 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.945 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.945 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:03.945 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.945 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.945 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.945 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:15:03.945 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:03.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.945 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.945 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.945 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:15:03.945 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:03.945 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.945 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.946 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.946 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:15:03.946 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:15:03.946 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:15:03.946 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:15:03.946 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:03.946 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:03.946 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:03.946 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.946 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.946 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.946 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:15:03.947 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:03.947 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:03.947 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:03.947 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:03.947 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:03.947 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:03.947 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:03.947 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:03.947 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:03.947 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:03.947 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:03.947 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:03.947 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.947 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.948 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.948 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:15:03.948 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:03.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.948 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.948 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.948 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:15:03.948 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:03.948 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.948 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.948 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:03.948 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.948 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.949 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.949 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:15:03.949 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:03.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.949 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.949 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.949 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:15:03.949 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:03.949 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.949 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.949 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:03.949 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.949 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.949 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.950 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:15:03.950 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:03.950 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.950 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.950 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.950 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:15:03.950 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:03.950 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.950 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.950 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.950 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:03.950 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:03.951 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.951 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:03.951 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:15:03.951 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:03.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.951 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.951 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:03.951 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:15:03.951 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:03.951 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.951 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.951 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:15:03.951 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:15:03.951 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:15:03.952 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:15:03.952 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:03.952 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:03.952 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:03.952 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:03.952 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.952 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:03.952 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:15:03.952 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.952 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:15:03.952 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:03.952 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:03.952 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:15:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:15:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:15:03.953 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:15:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:15:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.954 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:15:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:15:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:15:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:03.955 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:15:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:15:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:03.956 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:15:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:15:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:15:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:03.957 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:15:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:15:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:15:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:15:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:15:03.958 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:15:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:15:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:15:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:15:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:15:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:15:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:15:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:15:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:15:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:15:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:15:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:15:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:15:03.959 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:15:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:15:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:15:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:15:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:15:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:15:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:15:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:15:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:15:03.960 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:03.960 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:15:03.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.879+0000 7f4eaa600640 1 -- 172.21.15.161:0/2156996632 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4e840666c0 msgr2=0x7f4e84068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:03.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.879+0000 7f4eaa600640 1 --2- 172.21.15.161:0/2156996632 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4e840666c0 0x7f4e84068b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4e940012f0 tx=0x7f4e94009040 comp rx=0 tx=0).stop 2024-05-22T21:15:03.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.879+0000 7f4eaa600640 1 -- 172.21.15.161:0/2156996632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ea415f3a0 msgr2=0x7f4ea415d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:03.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.879+0000 7f4eaa600640 1 --2- 172.21.15.161:0/2156996632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ea415f3a0 0x7f4ea415d240 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000c450 tx=0x7f4e9000c910 comp rx=0 tx=0).stop 2024-05-22T21:15:03.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.879+0000 7f4eaa600640 1 -- 172.21.15.161:0/2156996632 shutdown_connections 2024-05-22T21:15:03.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.879+0000 7f4eaa600640 1 --2- 172.21.15.161:0/2156996632 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4e840666c0 0x7f4e84068b80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:03.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.879+0000 7f4eaa600640 1 --2- 172.21.15.161:0/2156996632 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4ea4159f80 0x7f4ea415d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:03.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.879+0000 7f4eaa600640 1 --2- 172.21.15.161:0/2156996632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ea415f3a0 0x7f4ea415d240 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:03.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.879+0000 7f4eaa600640 1 --2- 172.21.15.161:0/2156996632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ea4143ec0 0x7f4ea4159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:03.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.879+0000 7f4eaa600640 1 -- 172.21.15.161:0/2156996632 >> 172.21.15.161:0/2156996632 conn(0x7f4ea40b0bc0 msgr2=0x7f4ea40ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:03.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.879+0000 7f4eaa600640 1 -- 172.21.15.161:0/2156996632 shutdown_connections 2024-05-22T21:15:03.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:03.883+0000 7f4eaa600640 1 -- 172.21.15.161:0/2156996632 wait complete. 2024-05-22T21:15:03.963 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-05-22T21:15:05.476 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:15:05.471+0000 7f7d35a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-05-22T21:15:08.965 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:15:09.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.106+0000 7f9b83200640 1 Processor -- start 2024-05-22T21:15:09.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.110+0000 7f9b83200640 1 -- start start 2024-05-22T21:15:09.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.110+0000 7f9b83200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b840cfb70 0x7f9b8406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:09.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.110+0000 7f9b83200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9b8406ef50 0x7f9b841611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:09.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.110+0000 7f9b83200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b840d0460 con 0x7f9b84160df0 2024-05-22T21:15:09.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.110+0000 7f9b83200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8406f3e0 con 0x7f9b8406ef50 2024-05-22T21:15:09.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.110+0000 7f9b83200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840d0180 con 0x7f9b840cfb70 2024-05-22T21:15:09.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.110+0000 7f9b82800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9b8406ef50 0x7f9b841611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:09.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.110+0000 7f9b82800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9b8406ef50 0x7f9b841611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35808/0 (socket says 172.21.15.161:35808) 2024-05-22T21:15:09.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.110+0000 7f9b82800640 1 -- 172.21.15.161:0/3427593114 learned_addr learned my addr 172.21.15.161:0/3427593114 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:09.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.110+0000 7f9b81400640 1 --2- 172.21.15.161:0/3427593114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b840cfb70 0x7f9b8406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:09.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.110+0000 7f9b82800640 1 -- 172.21.15.161:0/3427593114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b840cfb70 msgr2=0x7f9b8406ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:09.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.110+0000 7f9b82800640 1 --2- 172.21.15.161:0/3427593114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b840cfb70 0x7f9b8406ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.110+0000 7f9b82800640 1 -- 172.21.15.161:0/3427593114 >> v1:172.21.15.179:6789/0 conn(0x7f9b84160df0 legacy=0x7f9b8415ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:09.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.110+0000 7f9b82800640 1 -- 172.21.15.161:0/3427593114 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8406df30 con 0x7f9b8406ef50 2024-05-22T21:15:09.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.114+0000 7f9b82800640 1 --2- 172.21.15.161:0/3427593114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8406ef50 0x7f9b841611d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009e50 tx=0x7f9b7c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddca5a90a8d9d4e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:09.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.114+0000 7f9b80a00640 1 -- 172.21.15.161:0/3427593114 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0a56c0 con 0x7f9b8406ef50 2024-05-22T21:15:09.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.114+0000 7f9b80a00640 1 -- 172.21.15.161:0/3427593114 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7c0a5820 con 0x7f9b8406ef50 2024-05-22T21:15:09.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.114+0000 7f9b80a00640 1 -- 172.21.15.161:0/3427593114 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0a5bb0 con 0x7f9b8406ef50 2024-05-22T21:15:09.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.114+0000 7f9b83200640 1 -- 172.21.15.161:0/3427593114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8406ef50 msgr2=0x7f9b841611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:09.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.114+0000 7f9b83200640 1 --2- 172.21.15.161:0/3427593114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8406ef50 0x7f9b841611d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009e50 tx=0x7f9b7c01c910 comp rx=0 tx=0).stop 2024-05-22T21:15:09.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.114+0000 7f9b83200640 1 -- 172.21.15.161:0/3427593114 shutdown_connections 2024-05-22T21:15:09.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.114+0000 7f9b83200640 1 --2- 172.21.15.161:0/3427593114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8406ef50 0x7f9b841611d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.114+0000 7f9b83200640 1 --2- 172.21.15.161:0/3427593114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b840cfb70 0x7f9b8406ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.114+0000 7f9b83200640 1 -- 172.21.15.161:0/3427593114 >> 172.21.15.161:0/3427593114 conn(0x7f9b840b2930 msgr2=0x7f9b840b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:09.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.114+0000 7f9b83200640 1 -- 172.21.15.161:0/3427593114 shutdown_connections 2024-05-22T21:15:09.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.114+0000 7f9b83200640 1 -- 172.21.15.161:0/3427593114 wait complete. 2024-05-22T21:15:09.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.114+0000 7f9b83200640 1 Processor -- start 2024-05-22T21:15:09.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.114+0000 7f9b83200640 1 -- start start 2024-05-22T21:15:09.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b83200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8406ef50 0x7f9b84155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:09.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b83200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b840cfb70 0x7f9b840ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:09.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b83200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b84160df0 0x7f9b841545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:09.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b83200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b84155c40 con 0x7f9b8406ef50 2024-05-22T21:15:09.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b83200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840aee00 con 0x7f9b840cfb70 2024-05-22T21:15:09.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b83200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b84156710 con 0x7f9b84160df0 2024-05-22T21:15:09.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b81e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8406ef50 0x7f9b84155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:09.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b81e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8406ef50 0x7f9b84155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35810/0 (socket says 172.21.15.161:35810) 2024-05-22T21:15:09.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b81e00640 1 -- 172.21.15.161:0/4178549444 learned_addr learned my addr 172.21.15.161:0/4178549444 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:09.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b81400640 1 --2- 172.21.15.161:0/4178549444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b840cfb70 0x7f9b840ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:09.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b82800640 1 --2- 172.21.15.161:0/4178549444 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b84160df0 0x7f9b841545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:09.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b81e00640 1 -- 172.21.15.161:0/4178549444 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b84160df0 msgr2=0x7f9b841545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:09.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b81e00640 1 --2- 172.21.15.161:0/4178549444 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b84160df0 0x7f9b841545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b81e00640 1 -- 172.21.15.161:0/4178549444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b840cfb70 msgr2=0x7f9b840ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:09.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b81e00640 1 --2- 172.21.15.161:0/4178549444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b840cfb70 0x7f9b840ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b81e00640 1 -- 172.21.15.161:0/4178549444 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7c009b30 con 0x7f9b8406ef50 2024-05-22T21:15:09.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b81e00640 1 --2- 172.21.15.161:0/4178549444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8406ef50 0x7f9b84155700 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400c3c0 tx=0x7f9b7400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:09.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b7a000640 1 -- 172.21.15.161:0/4178549444 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b740052d0 con 0x7f9b8406ef50 2024-05-22T21:15:09.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b7a000640 1 -- 172.21.15.161:0/4178549444 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b74005430 con 0x7f9b8406ef50 2024-05-22T21:15:09.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b7a000640 1 -- 172.21.15.161:0/4178549444 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74003840 con 0x7f9b8406ef50 2024-05-22T21:15:09.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b83200640 1 -- 172.21.15.161:0/4178549444 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b84154b00 con 0x7f9b8406ef50 2024-05-22T21:15:09.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b83200640 1 -- 172.21.15.161:0/4178549444 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b84156150 con 0x7f9b8406ef50 2024-05-22T21:15:09.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.118+0000 7f9b83200640 1 -- 172.21.15.161:0/4178549444 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b84067070 con 0x7f9b8406ef50 2024-05-22T21:15:09.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.122+0000 7f9b7a000640 1 -- 172.21.15.161:0/4178549444 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9b740039a0 con 0x7f9b8406ef50 2024-05-22T21:15:09.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.122+0000 7f9b7a000640 1 --2- 172.21.15.161:0/4178549444 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9b64066690 0x7f9b64068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:09.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.122+0000 7f9b7a000640 1 -- 172.21.15.161:0/4178549444 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b740835a0 con 0x7f9b8406ef50 2024-05-22T21:15:09.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.122+0000 7f9b81400640 1 --2- 172.21.15.161:0/4178549444 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9b64066690 0x7f9b64068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:09.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.122+0000 7f9b7a000640 1 -- 172.21.15.161:0/4178549444 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=51}) v3 -- 0x7f9b6406e1c0 con 0x7f9b8406ef50 2024-05-22T21:15:09.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.122+0000 7f9b81400640 1 --2- 172.21.15.161:0/4178549444 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9b64066690 0x7f9b64068b50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9b841f95d0 tx=0x7f9b70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:09.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.126+0000 7f9b7a000640 1 -- 172.21.15.161:0/4178549444 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9b74050a30 con 0x7f9b8406ef50 2024-05-22T21:15:09.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.278+0000 7f9b83200640 1 -- 172.21.15.161:0/4178549444 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9b840669f0 con 0x7f9b8406ef50 2024-05-22T21:15:09.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.282+0000 7f9b7a000640 1 -- 172.21.15.161:0/4178549444 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) v1 ==== 81+0+33671 (secure 0 0 0) 0x7f9b74017030 con 0x7f9b8406ef50 2024-05-22T21:15:09.286 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:09.286 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:15:09.286 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 50, 2024-05-22T21:15:09.286 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:15:09.286 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:15:09.287 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:15:06.483153+0000", 2024-05-22T21:15:09.287 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:15:09.287 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:15:09.287 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:15:09.287 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:15:09.287 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:15:09.287 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:15:09.287 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:15:09.287 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:15:09.287 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:15:09.287 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:09.287 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:15:09.287 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:15:09.288 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:15:09.288 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:15:09.288 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:15:09.288 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 7, 2024-05-22T21:15:09.288 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:15:09.288 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:15:09.288 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:15:09.288 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:15:09.288 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:15:09.288 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:15:09.288 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.288 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:15:09.288 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:15:09.289 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:15:09.289 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:15:09.289 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:15:09.289 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:09.289 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:09.289 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:09.289 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:09.289 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:09.289 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:09.289 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:09.289 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:09.289 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:09.290 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:09.290 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:15:09.290 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:15:09.290 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:15:09.290 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:15:09.290 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:15:09.290 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:09.290 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:09.290 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:09.290 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:09.290 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:09.290 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:09.290 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:09.291 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.291 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "49", 2024-05-22T21:15:09.291 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:09.291 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:09.291 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:09.291 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:09.291 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:09.291 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:09.291 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:09.291 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:09.291 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:09.291 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:09.291 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:09.292 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:09.292 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:09.292 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:09.292 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:09.292 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:09.292 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:09.292 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:09.292 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:09.292 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:09.292 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:09.292 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:09.292 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:09.292 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:09.292 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:09.293 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:09.293 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.293 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:09.293 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:09.293 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:09.293 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:09.293 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:09.293 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:09.293 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:09.293 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:09.293 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:09.293 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:09.293 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:09.294 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:09.294 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:15:09.294 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:15:09.294 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.294 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:09.294 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:15:09.294 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.294 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:09.294 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:15:09.294 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:15:09.294 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:15:09.294 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:15:09.295 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:15:09.295 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:09.295 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:09.295 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:09.295 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.295 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.295 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.295 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 6, 2024-05-22T21:15:09.295 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:15:09.295 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:14:48.612726+0000", 2024-05-22T21:15:09.295 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:15:09.295 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:15:09.295 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:09.295 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:09.296 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:09.296 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:09.296 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:09.296 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:09.296 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:09.296 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:09.296 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:09.296 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:09.296 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:15:09.296 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:15:09.296 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:15:09.296 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:15:09.296 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:15:09.296 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:09.297 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:09.297 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:09.297 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:09.297 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:09.297 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:09.297 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:09.297 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.297 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "49", 2024-05-22T21:15:09.297 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:09.297 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:09.297 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:09.297 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:09.297 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:09.297 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:09.298 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:09.298 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:09.298 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:09.298 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:09.298 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:09.298 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:09.298 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:09.298 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:09.298 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:09.298 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:09.298 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:09.298 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:09.298 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:09.299 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:09.299 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:09.299 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:09.299 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:09.299 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:09.299 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:09.299 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:09.299 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.299 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:09.299 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:09.299 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:09.299 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:09.299 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:09.300 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:09.300 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:09.300 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:09.300 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:09.300 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:09.300 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:09.300 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:09.300 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:15:09.300 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:15:09.300 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:15:09.300 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.300 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:09.300 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:15:09.301 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:15:09.301 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.301 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.301 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:09.301 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:15:09.301 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:15:09.301 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:15:09.301 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:15:09.301 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:15:09.301 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:09.301 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:09.301 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:09.302 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.302 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.302 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.302 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 7, 2024-05-22T21:15:09.302 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:15:09.302 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:14:49.053022+0000", 2024-05-22T21:15:09.302 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:15:09.302 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:15:09.302 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:09.302 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:09.302 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:09.302 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:09.303 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:09.303 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:09.303 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:09.303 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:09.303 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:09.303 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:09.303 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:15:09.303 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:15:09.303 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:15:09.303 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:15:09.303 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:15:09.303 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:09.303 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:09.304 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:09.304 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:09.304 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:09.304 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:09.304 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:09.304 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.304 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "49", 2024-05-22T21:15:09.304 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:09.304 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:09.304 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:09.304 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:09.304 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:09.305 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:09.305 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:09.305 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:09.305 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:09.305 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:09.305 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:09.305 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:09.305 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:09.305 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:09.305 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:09.305 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:09.305 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:09.305 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:09.306 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:09.306 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:09.306 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:09.306 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:09.306 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:09.306 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:09.306 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:09.306 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:09.306 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.306 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:09.306 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:09.306 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:09.306 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:09.307 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:09.307 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:09.307 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:09.307 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:09.307 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:09.307 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:09.307 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:09.307 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:09.307 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:15:09.307 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:15:09.307 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.308 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:09.308 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:15:09.308 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:15:09.308 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.308 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.308 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:09.308 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:15:09.308 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:15:09.308 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:15:09.308 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:15:09.308 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:15:09.308 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:09.308 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:09.308 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:09.309 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.309 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.309 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:09.309 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:15:09.309 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.309 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:15:09.309 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:09.309 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:09.309 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:09.309 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:09.309 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:09.309 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:09.310 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:09.310 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:09.310 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:09.310 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:09.310 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:09.310 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:09.310 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.310 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.310 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.310 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:15:09.310 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:09.310 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.311 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.311 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.311 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:15:09.311 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:09.311 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.311 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.311 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.311 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:09.311 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.311 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.311 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.311 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:15:09.311 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:09.311 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.312 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.312 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.312 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:15:09.312 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:09.312 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.312 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.312 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.312 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:09.312 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.312 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.312 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.312 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:15:09.312 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:09.312 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.313 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.313 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.313 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:15:09.313 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:09.313 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.313 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.313 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.313 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:09.313 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.313 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.313 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.313 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:15:09.313 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:09.313 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.314 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.314 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.314 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:15:09.314 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:09.314 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.314 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.314 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.314 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:15:09.314 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:15:09.314 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:15:09.314 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:15:09.314 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:09.314 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:09.315 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:15:09.315 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:09.315 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.315 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.315 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.315 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:15:09.315 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:09.315 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:09.315 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:09.315 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:09.315 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:09.315 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:09.315 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:09.315 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:09.316 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:09.316 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:09.316 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:09.316 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:09.316 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.316 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.316 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.316 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:15:09.316 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:09.316 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.316 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.316 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.316 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:15:09.316 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:09.317 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.317 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.317 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.317 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:09.317 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.317 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.317 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.317 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:15:09.317 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:09.317 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.317 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.317 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.317 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:15:09.318 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:09.318 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.318 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.318 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.318 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:09.318 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.318 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.318 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.318 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:15:09.318 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:09.318 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.318 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.318 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.319 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:15:09.319 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:09.319 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.319 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.319 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.319 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:09.319 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.319 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.319 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.319 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:15:09.319 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:09.319 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.319 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.320 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.320 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:15:09.320 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:09.320 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.320 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.320 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.320 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:15:09.320 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:15:09.320 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:15:09.320 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:15:09.320 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:09.320 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:09.320 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:09.320 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.321 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.321 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.321 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:15:09.321 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:09.321 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:09.321 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:09.321 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:09.321 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:09.321 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:09.321 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:09.321 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:09.321 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:09.321 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:09.322 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:09.322 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:09.322 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.322 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.322 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.322 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:15:09.322 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:09.322 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.322 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.322 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.322 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:15:09.322 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:09.322 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.323 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.323 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.323 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:09.323 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.323 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.323 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.323 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:15:09.323 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:09.323 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.323 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.323 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.323 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:15:09.323 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:09.324 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.324 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.324 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.324 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:09.324 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.324 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.324 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.324 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:15:09.324 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:09.325 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.325 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.325 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.325 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:15:09.325 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:09.325 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.325 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.325 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.325 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:09.326 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.326 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.326 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.326 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:15:09.326 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:09.326 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.326 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.326 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.326 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:15:09.326 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:09.326 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.326 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.326 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.327 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:15:09.327 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:15:09.327 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:15:09.327 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:15:09.327 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:09.327 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:09.327 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:15:09.327 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:09.327 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.327 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.327 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.327 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:15:09.327 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:09.328 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:09.328 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:09.328 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:09.328 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:09.328 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:09.328 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:09.328 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:09.328 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:09.328 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:09.328 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:09.328 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:09.328 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.328 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.329 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.329 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:15:09.329 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:09.329 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.329 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.329 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.329 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:15:09.329 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:09.329 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.329 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.329 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.329 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:09.329 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.330 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.330 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.330 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:15:09.330 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:09.330 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.330 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.330 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.330 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:15:09.330 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:09.330 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.330 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.330 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.330 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:09.331 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.331 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.331 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.331 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:15:09.331 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:09.331 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.331 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.331 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.331 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:15:09.331 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:09.331 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.331 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.331 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.332 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:09.332 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.332 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.332 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.332 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:15:09.332 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:09.332 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.332 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.332 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.332 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:15:09.332 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:09.332 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.332 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.332 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.333 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:15:09.333 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:15:09.333 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:15:09.333 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:15:09.333 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:09.333 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull", 2024-05-22T21:15:09.333 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:09.333 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:09.333 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.333 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.333 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.333 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:15:09.333 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:09.334 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:09.334 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:09.334 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:09.334 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:09.334 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:09.334 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:09.334 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:09.334 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:09.334 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:09.334 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:09.334 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:09.334 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.334 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.335 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.335 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:15:09.335 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:09.335 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.335 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.335 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.335 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:15:09.335 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:09.335 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.335 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.335 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.335 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:09.335 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.335 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.336 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.336 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:15:09.336 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:09.336 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.336 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.336 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.336 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:15:09.336 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:09.336 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.337 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.337 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.337 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:09.337 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.337 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.337 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.337 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:15:09.337 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:09.337 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.337 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.337 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.338 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:15:09.338 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:09.338 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.338 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.338 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.338 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:09.338 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.338 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.338 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.338 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:15:09.338 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:09.338 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.339 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.339 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.339 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:15:09.339 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:09.339 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.339 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.339 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.339 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:15:09.339 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:15:09.339 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:15:09.339 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:15:09.339 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:09.339 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:09.339 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:09.340 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.340 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.340 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.340 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:15:09.340 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:09.340 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:09.340 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:09.340 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:09.340 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:09.340 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:09.340 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:09.340 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:09.340 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:09.341 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:09.341 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:09.341 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:09.341 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.341 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.341 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.341 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:15:09.341 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:09.341 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.341 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.341 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.341 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:15:09.341 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:09.342 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.342 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.342 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.342 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:09.342 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.342 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.342 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.342 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:15:09.342 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:09.342 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.342 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.342 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.342 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:15:09.343 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:09.343 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.343 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.343 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.343 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:09.343 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.343 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.343 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.343 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:15:09.343 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:09.343 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.343 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.343 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.343 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:15:09.344 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:09.344 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.344 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.344 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:09.344 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.344 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.344 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.344 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:15:09.344 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:09.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.344 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.344 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.345 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:15:09.345 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:09.345 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.345 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.345 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.345 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:15:09.345 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:15:09.345 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:15:09.345 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:15:09.345 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:09.345 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:09.345 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:09.345 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.346 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.346 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.346 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:15:09.346 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:09.346 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:09.346 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:09.346 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:09.346 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:09.346 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:09.346 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:09.346 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:09.346 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:09.346 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:09.347 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:09.347 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:09.347 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.347 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.347 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.347 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:15:09.347 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:09.347 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.347 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.347 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.347 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:15:09.347 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:09.347 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.348 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.348 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:09.348 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.348 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.348 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.348 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:15:09.348 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:09.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.348 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.348 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.348 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:15:09.349 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:09.349 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.349 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.349 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:09.349 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.349 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.349 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.349 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:15:09.349 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:09.349 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.349 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.349 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.350 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:15:09.350 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:09.350 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.350 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.350 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.350 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:09.350 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.350 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.350 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.350 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:15:09.350 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:09.350 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.350 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.351 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.351 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:15:09.351 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:09.351 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.351 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.351 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.351 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:15:09.351 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:15:09.351 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:15:09.351 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:15:09.351 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:09.351 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:09.351 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:15:09.352 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:09.352 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.352 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.352 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.352 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:15:09.352 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:09.352 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:09.352 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:09.352 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:09.352 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:09.352 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:09.352 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:09.352 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:09.353 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:09.353 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:09.353 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:09.353 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:09.353 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.353 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.353 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.353 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:15:09.353 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:09.353 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.353 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.353 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.353 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:15:09.353 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:09.354 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.354 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.354 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.354 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:09.354 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.354 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.354 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.354 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:15:09.354 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:09.354 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.354 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.354 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.355 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:15:09.355 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:09.355 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.355 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.355 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:09.355 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.355 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.355 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.355 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:15:09.355 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:09.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.356 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.356 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.356 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:15:09.356 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:09.356 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.356 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.356 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.356 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:09.356 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:09.356 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.356 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:09.356 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:15:09.356 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:09.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.357 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.357 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:09.357 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:15:09.357 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:09.357 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.357 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.357 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:15:09.357 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:15:09.357 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:15:09.357 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:15:09.357 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:09.357 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:09.357 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:15:09.358 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:09.358 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:09.358 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.358 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:09.358 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:15:09.358 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.358 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:15:09.358 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:09.358 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:09.358 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:09.358 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:09.358 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:09.359 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:15:09.359 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:09.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.359 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.359 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:15:09.359 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:09.359 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:09.359 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:09.359 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:09.359 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:09.359 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:15:09.359 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:09.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.360 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.360 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:15:09.360 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:09.360 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:09.360 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:09.360 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:09.360 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:09.360 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:15:09.360 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:09.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.360 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.360 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:15:09.360 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:09.360 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:09.361 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:09.361 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:09.361 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:09.361 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:15:09.361 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:09.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.361 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.361 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:15:09.361 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:09.361 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:09.361 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:09.361 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:09.361 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:09.362 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:15:09.362 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:09.362 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.362 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.362 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:15:09.362 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:09.362 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:09.362 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:09.362 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:09.362 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:09.362 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:15:09.362 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:09.362 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.363 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.363 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:15:09.363 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:09.363 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:09.363 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:09.363 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:09.363 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:09.363 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:15:09.363 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:09.363 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.363 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:09.363 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:15:09.363 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:09.363 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:09.364 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:09.364 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:09.364 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:09.364 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:15:09.364 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:09.364 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.364 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:09.364 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:15:09.364 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:15:09.364 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:15:09.364 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:15:09.364 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:15:09.364 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:15:09.365 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:15:09.365 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:15:09.365 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:15:09.365 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:15:09.365 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:15:09.365 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:15:09.365 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:15:09.365 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:15:09.365 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:09.365 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:15:09.365 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:15:09.365 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:15:09.365 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:15:09.365 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:15:09.366 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:15:09.366 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:15:09.366 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:15:09.366 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:15:09.366 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:15:09.366 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:15:09.366 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:09.366 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:15:09.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.282+0000 7f9b83200640 1 -- 172.21.15.161:0/4178549444 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9b64066690 msgr2=0x7f9b64068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:09.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.282+0000 7f9b83200640 1 --2- 172.21.15.161:0/4178549444 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9b64066690 0x7f9b64068b50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9b841f95d0 tx=0x7f9b70008040 comp rx=0 tx=0).stop 2024-05-22T21:15:09.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.282+0000 7f9b83200640 1 -- 172.21.15.161:0/4178549444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8406ef50 msgr2=0x7f9b84155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:09.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.282+0000 7f9b83200640 1 --2- 172.21.15.161:0/4178549444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8406ef50 0x7f9b84155700 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400c3c0 tx=0x7f9b7400c880 comp rx=0 tx=0).stop 2024-05-22T21:15:09.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.282+0000 7f9b83200640 1 -- 172.21.15.161:0/4178549444 shutdown_connections 2024-05-22T21:15:09.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.282+0000 7f9b83200640 1 --2- 172.21.15.161:0/4178549444 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9b64066690 0x7f9b64068b50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.282+0000 7f9b83200640 1 --2- 172.21.15.161:0/4178549444 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b84160df0 0x7f9b841545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.282+0000 7f9b83200640 1 --2- 172.21.15.161:0/4178549444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b840cfb70 0x7f9b840ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.282+0000 7f9b83200640 1 --2- 172.21.15.161:0/4178549444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8406ef50 0x7f9b84155700 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.282+0000 7f9b83200640 1 -- 172.21.15.161:0/4178549444 >> 172.21.15.161:0/4178549444 conn(0x7f9b840b2930 msgr2=0x7f9b840cd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:09.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.282+0000 7f9b83200640 1 -- 172.21.15.161:0/4178549444 shutdown_connections 2024-05-22T21:15:09.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.282+0000 7f9b83200640 1 -- 172.21.15.161:0/4178549444 wait complete. 2024-05-22T21:15:09.369 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-05-22T21:15:09.370 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:15:09.370 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:15:09.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:09.379 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2024-05-22T21:15:09.766 INFO:teuthology.orchestra.run.smithi161.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-05-22T21:15:09.766 INFO:teuthology.orchestra.run.smithi161.stderr:50+0 records in 2024-05-22T21:15:09.766 INFO:teuthology.orchestra.run.smithi161.stderr:50+0 records out 2024-05-22T21:15:09.766 INFO:teuthology.orchestra.run.smithi161.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.325859 s, 161 MB/s 2024-05-22T21:15:09.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:15:09.767 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:15:09.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbce2c00640 1 Processor -- start 2024-05-22T21:15:09.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbce2c00640 1 -- start start 2024-05-22T21:15:09.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbce2c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbcdc143ec0 0x7fbcdc15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:09.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbce2c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcdc0715a0 0x7fbcdc06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:09.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbce2c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbcdc15ec90 0x7fbcdc15f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:09.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbce2c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcdc15f660 con 0x7fbcdc15ec90 2024-05-22T21:15:09.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbce2c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcdc1442a0 con 0x7fbcdc143ec0 2024-05-22T21:15:09.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbce2c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc070260 con 0x7fbcdc0715a0 2024-05-22T21:15:09.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbce0c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbcdc15ec90 0x7fbcdc15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:09.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbce0c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbcdc15ec90 0x7fbcdc15f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35814/0 (socket says 172.21.15.161:35814) 2024-05-22T21:15:09.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbce0c00640 1 -- 172.21.15.161:0/2952507746 learned_addr learned my addr 172.21.15.161:0/2952507746 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:09.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbcdbe00640 1 --2- 172.21.15.161:0/2952507746 >> v2:172.21.15.179:3300/0 conn(0x7fbcdc143ec0 0x7fbcdc15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:09.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbcd3400640 1 --2- 172.21.15.161:0/2952507746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcdc0715a0 0x7fbcdc06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:09.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbce0c00640 1 -- 172.21.15.161:0/2952507746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcdc0715a0 msgr2=0x7fbcdc06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:09.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbce0c00640 1 --2- 172.21.15.161:0/2952507746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcdc0715a0 0x7fbcdc06fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbce0c00640 1 -- 172.21.15.161:0/2952507746 >> v2:172.21.15.179:3300/0 conn(0x7fbcdc143ec0 msgr2=0x7fbcdc15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:09.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbce0c00640 1 --2- 172.21.15.161:0/2952507746 >> v2:172.21.15.179:3300/0 conn(0x7fbcdc143ec0 0x7fbcdc15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbce0c00640 1 -- 172.21.15.161:0/2952507746 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcdc070500 con 0x7fbcdc15ec90 2024-05-22T21:15:09.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbcdbe00640 1 --2- 172.21.15.161:0/2952507746 >> v2:172.21.15.179:3300/0 conn(0x7fbcdc143ec0 0x7fbcdc15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:09.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.898+0000 7fbcd3400640 1 --2- 172.21.15.161:0/2952507746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcdc0715a0 0x7fbcdc06fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:09.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.902+0000 7fbce0c00640 1 --2- 172.21.15.161:0/2952507746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcdc15ec90 0x7fbcdc15f120 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0724e0 tx=0x7fbcd401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae3b5396b918b9a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:09.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.902+0000 7fbcdb400640 1 -- 172.21.15.161:0/2952507746 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40a5840 con 0x7fbcdc15ec90 2024-05-22T21:15:09.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.902+0000 7fbcdb400640 1 -- 172.21.15.161:0/2952507746 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd40a59a0 con 0x7fbcdc15ec90 2024-05-22T21:15:09.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.902+0000 7fbcdb400640 1 -- 172.21.15.161:0/2952507746 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40a5d30 con 0x7fbcdc15ec90 2024-05-22T21:15:09.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.902+0000 7fbce2c00640 1 -- 172.21.15.161:0/2952507746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcdc15ec90 msgr2=0x7fbcdc15f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:09.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.902+0000 7fbce2c00640 1 --2- 172.21.15.161:0/2952507746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcdc15ec90 0x7fbcdc15f120 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0724e0 tx=0x7fbcd401c9c0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.902+0000 7fbce2c00640 1 -- 172.21.15.161:0/2952507746 shutdown_connections 2024-05-22T21:15:09.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.902+0000 7fbce2c00640 1 --2- 172.21.15.161:0/2952507746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcdc15ec90 0x7fbcdc15f120 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.902+0000 7fbce2c00640 1 --2- 172.21.15.161:0/2952507746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcdc0715a0 0x7fbcdc06fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.902+0000 7fbce2c00640 1 --2- 172.21.15.161:0/2952507746 >> v2:172.21.15.179:3300/0 conn(0x7fbcdc143ec0 0x7fbcdc15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.902+0000 7fbce2c00640 1 -- 172.21.15.161:0/2952507746 >> 172.21.15.161:0/2952507746 conn(0x7fbcdc0b0bc0 msgr2=0x7fbcdc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:09.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.902+0000 7fbce2c00640 1 -- 172.21.15.161:0/2952507746 shutdown_connections 2024-05-22T21:15:09.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.902+0000 7fbce2c00640 1 -- 172.21.15.161:0/2952507746 wait complete. 2024-05-22T21:15:09.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce2c00640 1 Processor -- start 2024-05-22T21:15:09.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce2c00640 1 -- start start 2024-05-22T21:15:09.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce2c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcdc143ec0 0x7fbcdc154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:09.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce2c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcdc15ec90 0x7fbcdc158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:09.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce2c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcdc159020 0x7fbcdc0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:09.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce2c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc15a140 con 0x7fbcdc159020 2024-05-22T21:15:09.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce2c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc159530 con 0x7fbcdc15ec90 2024-05-22T21:15:09.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce2c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc159810 con 0x7fbcdc143ec0 2024-05-22T21:15:09.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce0c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcdc159020 0x7fbcdc0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:09.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce0c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcdc159020 0x7fbcdc0b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35816/0 (socket says 172.21.15.161:35816) 2024-05-22T21:15:09.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce0c00640 1 -- 172.21.15.161:0/3505546854 learned_addr learned my addr 172.21.15.161:0/3505546854 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:09.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbcdbe00640 1 --2- 172.21.15.161:0/3505546854 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcdc143ec0 0x7fbcdc154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:09.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbcd3400640 1 --2- 172.21.15.161:0/3505546854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcdc15ec90 0x7fbcdc158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:09.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce0c00640 1 -- 172.21.15.161:0/3505546854 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcdc143ec0 msgr2=0x7fbcdc154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:09.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce0c00640 1 --2- 172.21.15.161:0/3505546854 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcdc143ec0 0x7fbcdc154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce0c00640 1 -- 172.21.15.161:0/3505546854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcdc15ec90 msgr2=0x7fbcdc158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:09.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce0c00640 1 --2- 172.21.15.161:0/3505546854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcdc15ec90 0x7fbcdc158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:09.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce0c00640 1 -- 172.21.15.161:0/3505546854 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd4009b30 con 0x7fbcdc159020 2024-05-22T21:15:09.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbcd3400640 1 --2- 172.21.15.161:0/3505546854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcdc15ec90 0x7fbcdc158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:09.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbcdbe00640 1 --2- 172.21.15.161:0/3505546854 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcdc143ec0 0x7fbcdc154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:09.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce0c00640 1 --2- 172.21.15.161:0/3505546854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcdc159020 0x7fbcdc0b2b90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbcd401b070 tx=0x7fbcd4019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:09.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbcd8c00640 1 -- 172.21.15.161:0/3505546854 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40b8670 con 0x7fbcdc159020 2024-05-22T21:15:09.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbcd8c00640 1 -- 172.21.15.161:0/3505546854 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd40b87f0 con 0x7fbcdc159020 2024-05-22T21:15:09.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce2c00640 1 -- 172.21.15.161:0/3505546854 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcdc159af0 con 0x7fbcdc159020 2024-05-22T21:15:09.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbcd8c00640 1 -- 172.21.15.161:0/3505546854 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40b8ba0 con 0x7fbcdc159020 2024-05-22T21:15:09.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce2c00640 1 -- 172.21.15.161:0/3505546854 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcdc154fd0 con 0x7fbcdc159020 2024-05-22T21:15:09.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.906+0000 7fbce2c00640 1 -- 172.21.15.161:0/3505546854 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcdc067220 con 0x7fbcdc159020 2024-05-22T21:15:09.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.910+0000 7fbcd8c00640 1 -- 172.21.15.161:0/3505546854 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbcd40b8d00 con 0x7fbcdc159020 2024-05-22T21:15:09.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.910+0000 7fbcd8c00640 1 --2- 172.21.15.161:0/3505546854 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbcb80666c0 0x7fbcb8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:09.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.910+0000 7fbcd8c00640 1 -- 172.21.15.161:0/3505546854 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbcd411f410 con 0x7fbcdc159020 2024-05-22T21:15:09.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.910+0000 7fbcdbe00640 1 --2- 172.21.15.161:0/3505546854 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbcb80666c0 0x7fbcb8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:09.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.910+0000 7fbcd8c00640 1 -- 172.21.15.161:0/3505546854 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=51}) v3 -- 0x7fbcb806e1f0 con 0x7fbcdc159020 2024-05-22T21:15:09.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.910+0000 7fbcdbe00640 1 --2- 172.21.15.161:0/3505546854 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbcb80666c0 0x7fbcb8068b80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbccc006fd0 tx=0x7fbccc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:09.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:09.914+0000 7fbcd8c00640 1 -- 172.21.15.161:0/3505546854 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbcd40ec770 con 0x7fbcdc159020 2024-05-22T21:15:10.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.070+0000 7fbce2c00640 1 -- 172.21.15.161:0/3505546854 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fbcdc06dcc0 con 0x7fbcdc159020 2024-05-22T21:15:10.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.074+0000 7fbcd8c00640 1 -- 172.21.15.161:0/3505546854 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) v1 ==== 81+0+33671 (secure 0 0 0) 0x7fbcd40bc030 con 0x7fbcdc159020 2024-05-22T21:15:10.078 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:10.078 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:15:10.078 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 50, 2024-05-22T21:15:10.078 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:15:10.078 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:15:10.079 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:15:06.483153+0000", 2024-05-22T21:15:10.079 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:15:10.079 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:15:10.079 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:15:10.079 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:15:10.079 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:15:10.079 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:15:10.079 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:15:10.079 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:15:10.079 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:15:10.079 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:15:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:15:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:15:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:15:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:15:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 7, 2024-05-22T21:15:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:15:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:15:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:15:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:15:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:15:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:15:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:15:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:15:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:15:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:15:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:15:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:15:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:15:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:15:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:15:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:15:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:10.083 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:10.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.083 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "49", 2024-05-22T21:15:10.083 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:10.083 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:10.083 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:10.083 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:10.083 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:10.083 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:10.083 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:10.083 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:10.084 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:10.084 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:10.084 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:10.084 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:10.084 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:10.084 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:10.084 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:10.084 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:10.084 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:10.084 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:10.085 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:10.085 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:10.085 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:10.085 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:10.085 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:10.085 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:10.085 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:10.085 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:10.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.085 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:10.085 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:10.085 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:10.085 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:10.086 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:10.086 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:10.086 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:10.086 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:10.086 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:10.086 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:10.086 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:10.086 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:10.086 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:15:10.086 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:15:10.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.086 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:10.086 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:15:10.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.087 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:10.087 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:15:10.087 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:15:10.087 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:15:10.087 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:15:10.087 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:15:10.087 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:10.087 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:10.088 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:10.088 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.088 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 6, 2024-05-22T21:15:10.088 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:15:10.088 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:14:48.612726+0000", 2024-05-22T21:15:10.088 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:15:10.088 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:15:10.088 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:10.088 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:10.088 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:10.088 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:10.089 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:10.089 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:10.089 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:10.089 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:10.089 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:10.089 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:10.089 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:15:10.089 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:15:10.089 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:15:10.089 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:15:10.089 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:15:10.089 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:10.089 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:10.089 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:10.090 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:10.090 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:10.090 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:10.090 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:10.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.090 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "49", 2024-05-22T21:15:10.090 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:10.090 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:10.090 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:10.090 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:10.090 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:10.091 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:10.091 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:10.091 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:10.091 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:10.091 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:10.091 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:10.091 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:10.091 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:10.091 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:10.091 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:10.091 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:10.092 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:10.092 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:10.092 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:10.092 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:10.092 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:10.092 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:10.092 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:10.092 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:10.092 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:10.092 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:10.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.092 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:10.092 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:10.093 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:10.093 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:10.093 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:10.093 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:10.093 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:10.093 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:10.093 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:10.093 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:10.093 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:10.093 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:10.093 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:15:10.093 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:15:10.093 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:15:10.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.094 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:10.094 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:15:10.094 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:15:10.094 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.094 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:10.094 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:15:10.094 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:15:10.094 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:15:10.094 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:15:10.094 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:15:10.094 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:10.094 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:10.095 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:10.095 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.095 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 7, 2024-05-22T21:15:10.095 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:15:10.095 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:14:49.053022+0000", 2024-05-22T21:15:10.095 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:15:10.095 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:15:10.095 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:10.095 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:10.095 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:10.095 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:10.096 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:10.096 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:10.096 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:10.096 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:10.096 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:10.096 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:10.096 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:15:10.096 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:15:10.096 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:15:10.096 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:15:10.096 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:15:10.096 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:10.096 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:10.096 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:10.096 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:10.097 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:10.097 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:10.097 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:10.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.097 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "49", 2024-05-22T21:15:10.097 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:10.097 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:10.097 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:10.097 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:10.097 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:10.097 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:10.097 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:10.097 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:10.098 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:10.098 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:10.098 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:10.098 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:10.098 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:10.098 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:10.098 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:10.098 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:10.098 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:10.098 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:10.098 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:10.098 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:10.098 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:10.099 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:10.099 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:10.099 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:10.099 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:10.099 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:10.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.099 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:10.099 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:10.099 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:10.099 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:10.099 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:10.099 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:10.099 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:10.099 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:10.100 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:10.100 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:10.100 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:10.100 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:10.100 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:15:10.100 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:15:10.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.100 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:10.100 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:15:10.100 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:15:10.100 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.100 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:10.101 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:15:10.101 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:15:10.101 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:15:10.101 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:15:10.101 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:15:10.101 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:10.101 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:10.101 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:10.101 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.101 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.101 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:10.101 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:15:10.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.101 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:15:10.102 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:10.102 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:10.102 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:10.102 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:10.102 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:10.102 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:10.102 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:10.102 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:10.102 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:10.102 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:10.102 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:10.102 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:10.103 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.103 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.103 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:15:10.103 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:10.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.103 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.103 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:15:10.103 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:10.103 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.103 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.103 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:10.103 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.104 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.104 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:15:10.104 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:10.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.104 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.104 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:15:10.104 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:10.104 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.104 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.104 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:10.104 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.105 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.105 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:15:10.105 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:10.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.105 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.105 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:15:10.105 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:10.105 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.105 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.105 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:10.105 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.106 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.106 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:15:10.106 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:10.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.106 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.106 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:15:10.106 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:10.106 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.106 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.106 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:15:10.106 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:15:10.107 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:15:10.107 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:15:10.107 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:10.107 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:10.107 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:15:10.107 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:10.107 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.107 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:15:10.107 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:10.107 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:10.107 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:10.107 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:10.108 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:10.108 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:10.108 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:10.108 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:10.108 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:10.108 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:10.108 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:10.108 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:10.108 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.108 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.109 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:15:10.109 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:10.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.109 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.109 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:15:10.109 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:10.109 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.109 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.109 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:10.109 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.110 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.110 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:15:10.110 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:10.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.110 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.110 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:15:10.110 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:10.110 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.110 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.111 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:10.111 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.111 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.111 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:15:10.111 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:10.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.111 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.111 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:15:10.111 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:10.111 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.112 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.112 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:10.112 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.112 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.112 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:15:10.112 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:10.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.112 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.112 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.112 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:15:10.112 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:10.112 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.113 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.113 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:15:10.113 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:15:10.113 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:15:10.113 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:15:10.113 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:10.113 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:10.113 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:10.113 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.113 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:15:10.114 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:10.114 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:10.114 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:10.114 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:10.114 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:10.114 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:10.114 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:10.114 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:10.115 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:10.115 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:10.115 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:10.115 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:10.115 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.115 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.115 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:15:10.115 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:10.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.115 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.116 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:15:10.116 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:10.116 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.116 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.116 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:10.116 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.116 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.116 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:15:10.116 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:10.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.117 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.117 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:15:10.117 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:10.117 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.117 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.117 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:10.117 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.117 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.117 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:15:10.117 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:10.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.118 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.118 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:15:10.118 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:10.118 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.118 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.118 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:10.118 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.118 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.118 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.119 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:15:10.119 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:10.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.119 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.119 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:15:10.119 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:10.119 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.119 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.120 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:15:10.120 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:15:10.120 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:15:10.120 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:15:10.120 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:10.120 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:10.120 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:15:10.121 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:10.121 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.121 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:15:10.121 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:10.121 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:10.121 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:10.122 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:10.122 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:10.122 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:10.122 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:10.122 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:10.122 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:10.122 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:10.122 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:10.123 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:10.123 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.123 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.123 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:15:10.123 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:10.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.124 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.124 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:15:10.124 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:10.124 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.124 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.124 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:10.124 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.125 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.125 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:15:10.125 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:10.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.125 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.125 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:15:10.125 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:10.126 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.126 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.126 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:10.126 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.126 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.126 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:15:10.127 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:10.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.127 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.127 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:15:10.127 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:10.127 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.127 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.128 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:10.128 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.128 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.128 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:15:10.128 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:10.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.129 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.129 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:15:10.129 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:10.129 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.129 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.129 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:15:10.129 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:15:10.130 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:15:10.130 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:15:10.130 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:10.130 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull", 2024-05-22T21:15:10.130 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:10.130 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:10.130 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.131 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:15:10.131 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:10.131 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:10.131 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:10.131 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:10.131 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:10.131 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:10.131 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:10.132 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:10.132 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:10.132 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:10.132 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:10.132 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:10.132 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.132 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.133 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:15:10.133 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:10.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.133 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.133 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:15:10.133 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:10.133 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.133 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.134 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:10.134 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.134 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.134 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:15:10.134 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:10.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.134 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.134 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:15:10.134 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:10.134 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.135 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.135 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:10.135 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.135 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.135 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:15:10.135 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:10.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.135 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.135 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:15:10.135 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:10.135 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.135 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.136 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:10.136 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.136 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.136 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:15:10.136 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:10.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.136 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.136 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:15:10.136 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:10.136 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.136 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.137 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:15:10.137 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:15:10.137 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:15:10.137 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:15:10.137 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:10.137 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:10.137 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:10.137 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.137 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:15:10.137 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:10.137 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:10.137 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:10.137 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:10.138 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:10.138 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:10.138 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:10.138 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:10.138 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:10.138 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:10.138 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:10.138 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:10.138 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.138 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.138 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:15:10.139 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:10.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.139 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.139 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:15:10.139 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:10.139 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.139 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.139 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:10.139 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.139 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.139 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:15:10.140 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:10.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.140 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.140 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:15:10.140 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:10.140 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.140 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.140 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:10.140 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.140 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.141 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:15:10.141 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:10.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.141 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.141 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:15:10.141 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:10.141 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.141 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.141 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:10.141 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.141 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.142 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:15:10.142 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:10.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.142 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.142 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:15:10.142 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:10.142 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.142 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.143 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:15:10.143 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:15:10.143 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:15:10.143 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:15:10.143 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:10.143 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:10.143 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:10.143 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.143 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:15:10.143 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:10.143 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:10.144 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:10.144 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:10.144 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:10.144 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:10.144 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:10.144 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:10.144 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:10.144 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:10.144 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:10.144 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:10.144 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.144 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.144 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:15:10.145 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:10.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.145 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.145 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:15:10.145 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:10.145 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.145 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.145 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:10.145 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.145 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.145 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:15:10.145 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:10.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.146 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.146 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:15:10.146 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:10.146 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.146 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.146 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:10.146 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.146 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.146 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:15:10.147 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:10.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.147 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.147 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:15:10.147 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:10.147 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.147 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.147 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:10.147 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.147 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.147 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.147 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:15:10.148 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:10.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.148 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.148 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:15:10.148 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:10.148 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.148 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.148 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:15:10.148 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:15:10.148 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:15:10.148 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:15:10.148 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:10.148 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:10.149 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:15:10.149 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:10.149 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.149 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:15:10.149 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:10.149 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:10.149 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:10.149 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:10.149 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:10.149 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:10.149 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:10.149 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:10.150 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:10.150 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:10.150 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:10.150 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:10.150 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.150 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.150 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:15:10.150 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:10.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.150 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.151 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:15:10.151 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:10.151 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.151 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.151 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:10.151 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.151 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.151 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:15:10.151 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:10.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.151 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.151 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:15:10.152 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:10.152 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.152 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.152 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:10.152 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.152 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.152 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:15:10.152 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:10.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.152 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.152 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:15:10.152 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:10.153 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.153 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.153 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:10.153 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:10.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.153 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:10.153 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:15:10.153 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:10.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.153 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:10.153 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:15:10.153 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:10.154 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.154 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.154 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:15:10.154 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:15:10.154 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:15:10.154 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:15:10.154 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:10.154 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:10.154 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:15:10.154 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:10.154 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:10.154 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.155 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:10.155 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:15:10.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.155 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:15:10.155 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:10.155 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:10.155 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:10.155 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:10.155 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:10.155 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:15:10.155 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:10.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.155 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.156 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:15:10.156 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:10.156 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:10.156 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:10.156 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:10.156 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:10.156 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:15:10.156 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:10.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.156 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.156 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:15:10.156 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:10.156 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:10.157 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:10.157 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:10.157 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:10.157 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:15:10.157 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:10.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.157 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.157 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:15:10.157 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:10.157 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:10.157 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:10.157 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:10.157 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:10.157 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:15:10.158 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:10.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.158 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:15:10.158 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:10.158 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:10.158 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:10.158 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:10.158 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:10.158 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:15:10.158 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:10.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.158 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.159 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:15:10.159 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:10.159 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:10.159 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:10.159 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:10.159 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:10.159 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:15:10.159 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:10.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.159 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:15:10.159 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:10.159 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:10.160 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:10.160 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:10.160 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:10.160 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:15:10.160 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:10.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.160 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:10.160 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:15:10.160 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:10.160 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:10.160 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:10.160 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:10.160 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:10.161 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:15:10.161 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:10.161 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.161 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:10.161 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:15:10.161 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:15:10.161 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:15:10.161 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:15:10.161 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:15:10.161 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:15:10.161 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:15:10.161 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:15:10.161 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:15:10.162 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:15:10.162 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:15:10.162 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:15:10.162 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:15:10.162 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:15:10.162 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:10.162 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:15:10.162 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:15:10.162 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:15:10.162 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:15:10.162 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:15:10.163 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:15:10.163 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:15:10.163 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:15:10.163 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:15:10.163 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:15:10.163 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:15:10.163 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:10.163 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:15:10.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.074+0000 7fbce2c00640 1 -- 172.21.15.161:0/3505546854 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbcb80666c0 msgr2=0x7fbcb8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:10.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.074+0000 7fbce2c00640 1 --2- 172.21.15.161:0/3505546854 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbcb80666c0 0x7fbcb8068b80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbccc006fd0 tx=0x7fbccc008040 comp rx=0 tx=0).stop 2024-05-22T21:15:10.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.074+0000 7fbce2c00640 1 -- 172.21.15.161:0/3505546854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcdc159020 msgr2=0x7fbcdc0b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:10.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.074+0000 7fbce2c00640 1 --2- 172.21.15.161:0/3505546854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcdc159020 0x7fbcdc0b2b90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbcd401b070 tx=0x7fbcd4019760 comp rx=0 tx=0).stop 2024-05-22T21:15:10.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.074+0000 7fbce2c00640 1 -- 172.21.15.161:0/3505546854 shutdown_connections 2024-05-22T21:15:10.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.074+0000 7fbce2c00640 1 --2- 172.21.15.161:0/3505546854 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbcb80666c0 0x7fbcb8068b80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:10.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.074+0000 7fbce2c00640 1 --2- 172.21.15.161:0/3505546854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcdc159020 0x7fbcdc0b2b90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:10.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.074+0000 7fbce2c00640 1 --2- 172.21.15.161:0/3505546854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcdc15ec90 0x7fbcdc158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:10.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.074+0000 7fbce2c00640 1 --2- 172.21.15.161:0/3505546854 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcdc143ec0 0x7fbcdc154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:10.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.074+0000 7fbce2c00640 1 -- 172.21.15.161:0/3505546854 >> 172.21.15.161:0/3505546854 conn(0x7fbcdc0b0bc0 msgr2=0x7fbcdc0cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:10.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.074+0000 7fbce2c00640 1 -- 172.21.15.161:0/3505546854 shutdown_connections 2024-05-22T21:15:10.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.074+0000 7fbce2c00640 1 -- 172.21.15.161:0/3505546854 wait complete. 2024-05-22T21:15:10.166 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:15:10.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.294+0000 7f5183600640 1 Processor -- start 2024-05-22T21:15:10.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.294+0000 7f5183600640 1 -- start start 2024-05-22T21:15:10.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.298+0000 7f5183600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f517c06d700 0x7f517c168210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:10.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.298+0000 7f5183600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f517c071fd0 con 0x7f517c15d430 2024-05-22T21:15:10.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.298+0000 7f5183600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f517c070c00 con 0x7f517c0707c0 2024-05-22T21:15:10.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.298+0000 7f5183600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f517c0acfc0 con 0x7f517c06d700 2024-05-22T21:15:10.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.298+0000 7f5180c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f517c15d430 0x7f517c0ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:38124/0 (socket says 172.21.15.161:38124) 2024-05-22T21:15:10.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.298+0000 7f5180c00640 1 -- 172.21.15.161:0/4205883062 learned_addr learned my addr 172.21.15.161:0/4205883062 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:10.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.298+0000 7f5181600640 1 --2- 172.21.15.161:0/4205883062 >> v2:172.21.15.161:3300/0 conn(0x7f517c06d700 0x7f517c168210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:10.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.298+0000 7f517be00640 1 -- 172.21.15.161:0/4205883062 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462094540 0 0) 0x7f517c071fd0 con 0x7f517c15d430 2024-05-22T21:15:10.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.298+0000 7f5181600640 1 -- 172.21.15.161:0/4205883062 >> v1:172.21.15.179:6789/0 conn(0x7f517c0707c0 legacy=0x7f517c06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:10.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.298+0000 7f5181600640 1 -- 172.21.15.161:0/4205883062 >> v1:172.21.15.161:6789/0 conn(0x7f517c15d430 legacy=0x7f517c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:10.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.298+0000 7f5181600640 1 -- 172.21.15.161:0/4205883062 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c070e80 con 0x7f517c06d700 2024-05-22T21:15:10.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.298+0000 7f5181600640 1 --2- 172.21.15.161:0/4205883062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f517c06d700 0x7f517c168210 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5174009d90 tx=0x7f517401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15933b4913e5d5e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:10.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.298+0000 7f517be00640 1 -- 172.21.15.161:0/4205883062 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740a5970 con 0x7f517c06d700 2024-05-22T21:15:10.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.298+0000 7f517be00640 1 -- 172.21.15.161:0/4205883062 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51740a5ad0 con 0x7f517c06d700 2024-05-22T21:15:10.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.298+0000 7f517be00640 1 -- 172.21.15.161:0/4205883062 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740a5e60 con 0x7f517c06d700 2024-05-22T21:15:10.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5183600640 1 -- 172.21.15.161:0/4205883062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f517c06d700 msgr2=0x7f517c168210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:10.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5183600640 1 --2- 172.21.15.161:0/4205883062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f517c06d700 0x7f517c168210 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5174009d90 tx=0x7f517401c8b0 comp rx=0 tx=0).stop 2024-05-22T21:15:10.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5183600640 1 -- 172.21.15.161:0/4205883062 shutdown_connections 2024-05-22T21:15:10.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5183600640 1 --2- 172.21.15.161:0/4205883062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f517c06d700 0x7f517c168210 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:10.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5183600640 1 -- 172.21.15.161:0/4205883062 >> 172.21.15.161:0/4205883062 conn(0x7f517c0b0bc0 msgr2=0x7f517c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:10.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5183600640 1 -- 172.21.15.161:0/4205883062 shutdown_connections 2024-05-22T21:15:10.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5183600640 1 -- 172.21.15.161:0/4205883062 wait complete. 2024-05-22T21:15:10.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5183600640 1 Processor -- start 2024-05-22T21:15:10.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5183600640 1 -- start start 2024-05-22T21:15:10.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5183600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f517c06d700 0x7f517c137fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:10.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5183600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f517c0707c0 0x7f517c15b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:10.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5183600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f517c15d430 0x7f517c15bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:10.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5183600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c15b2a0 con 0x7f517c15d430 2024-05-22T21:15:10.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5183600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c1607a0 con 0x7f517c0707c0 2024-05-22T21:15:10.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5183600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517c153c30 con 0x7f517c06d700 2024-05-22T21:15:10.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5181600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f517c15d430 0x7f517c15bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:10.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5181600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f517c15d430 0x7f517c15bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35828/0 (socket says 172.21.15.161:35828) 2024-05-22T21:15:10.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5181600640 1 -- 172.21.15.161:0/2479745814 learned_addr learned my addr 172.21.15.161:0/2479745814 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:10.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5173e00640 1 --2- 172.21.15.161:0/2479745814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f517c0707c0 0x7f517c15b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:10.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5180c00640 1 --2- 172.21.15.161:0/2479745814 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f517c06d700 0x7f517c137fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:10.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5181600640 1 -- 172.21.15.161:0/2479745814 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f517c06d700 msgr2=0x7f517c137fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:10.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.302+0000 7f5181600640 1 --2- 172.21.15.161:0/2479745814 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f517c06d700 0x7f517c137fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:10.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.306+0000 7f5181600640 1 -- 172.21.15.161:0/2479745814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f517c0707c0 msgr2=0x7f517c15b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:10.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.306+0000 7f5181600640 1 --2- 172.21.15.161:0/2479745814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f517c0707c0 0x7f517c15b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:10.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.306+0000 7f5181600640 1 -- 172.21.15.161:0/2479745814 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5174009a70 con 0x7f517c15d430 2024-05-22T21:15:10.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.306+0000 7f5181600640 1 --2- 172.21.15.161:0/2479745814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f517c15d430 0x7f517c15bc60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f517401a4e0 tx=0x7f5174019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:10.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.306+0000 7f5179600640 1 -- 172.21.15.161:0/2479745814 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740addc0 con 0x7f517c15d430 2024-05-22T21:15:10.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.306+0000 7f5179600640 1 -- 172.21.15.161:0/2479745814 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51740b7410 con 0x7f517c15d430 2024-05-22T21:15:10.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.306+0000 7f5183600640 1 -- 172.21.15.161:0/2479745814 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517c156390 con 0x7f517c15d430 2024-05-22T21:15:10.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.306+0000 7f5179600640 1 -- 172.21.15.161:0/2479745814 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740b77a0 con 0x7f517c15d430 2024-05-22T21:15:10.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.306+0000 7f5183600640 1 -- 172.21.15.161:0/2479745814 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517c1604c0 con 0x7f517c15d430 2024-05-22T21:15:10.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.306+0000 7f5183600640 1 -- 172.21.15.161:0/2479745814 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517c160930 con 0x7f517c15d430 2024-05-22T21:15:10.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.306+0000 7f5179600640 1 -- 172.21.15.161:0/2479745814 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f51740b4050 con 0x7f517c15d430 2024-05-22T21:15:10.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.306+0000 7f5179600640 1 --2- 172.21.15.161:0/2479745814 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5158066690 0x7f5158068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:10.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.306+0000 7f5179600640 1 -- 172.21.15.161:0/2479745814 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f517411f5d0 con 0x7f517c15d430 2024-05-22T21:15:10.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.306+0000 7f5179600640 1 -- 172.21.15.161:0/2479745814 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=51}) v3 -- 0x7f515806e1c0 con 0x7f517c15d430 2024-05-22T21:15:10.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.306+0000 7f5180c00640 1 --2- 172.21.15.161:0/2479745814 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5158066690 0x7f5158068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:10.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.310+0000 7f5180c00640 1 --2- 172.21.15.161:0/2479745814 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5158066690 0x7f5158068b50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f516c007bc0 tx=0x7f516c007b10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:10.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.310+0000 7f5179600640 1 -- 172.21.15.161:0/2479745814 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f51740b2060 con 0x7f517c15d430 2024-05-22T21:15:10.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.502+0000 7f5183600640 1 -- 172.21.15.161:0/2479745814 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f517c15b440 con 0x7f517c15d430 2024-05-22T21:15:10.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.502+0000 7f5179600640 1 -- 172.21.15.161:0/2479745814 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f517c15b440 con 0x7f517c15d430 2024-05-22T21:15:10.508 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:10.509 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/1767934947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1767934947},{"type":"v1","addr":"172.21.15.161:6836","nonce":1767934947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1873337287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1873337287},{"type":"v1","addr":"172.21.15.179:6834","nonce":1873337287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1815838236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":1815838236},{"type":"v1","addr":"172.21.15.161:6837","nonce":1815838236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:50.052292+0000","modified":"2024-05-22T21:14:53.058881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5374":{"gid":5374,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/389418050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":389418050},{"type":"v1","addr":"172.21.15.179:6835","nonce":389418050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-22T21:15:10.509 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 23 2024-05-22T21:15:10.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.506+0000 7f5183600640 1 -- 172.21.15.161:0/2479745814 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5158066690 msgr2=0x7f5158068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:10.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.506+0000 7f5183600640 1 --2- 172.21.15.161:0/2479745814 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5158066690 0x7f5158068b50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f516c007bc0 tx=0x7f516c007b10 comp rx=0 tx=0).stop 2024-05-22T21:15:10.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.506+0000 7f5183600640 1 -- 172.21.15.161:0/2479745814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f517c15d430 msgr2=0x7f517c15bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:10.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.506+0000 7f5183600640 1 --2- 172.21.15.161:0/2479745814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f517c15d430 0x7f517c15bc60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f517401a4e0 tx=0x7f5174019650 comp rx=0 tx=0).stop 2024-05-22T21:15:10.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.506+0000 7f5183600640 1 -- 172.21.15.161:0/2479745814 shutdown_connections 2024-05-22T21:15:10.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.506+0000 7f5183600640 1 --2- 172.21.15.161:0/2479745814 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5158066690 0x7f5158068b50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:10.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.506+0000 7f5183600640 1 --2- 172.21.15.161:0/2479745814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f517c15d430 0x7f517c15bc60 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:10.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.506+0000 7f5183600640 1 --2- 172.21.15.161:0/2479745814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f517c0707c0 0x7f517c15b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:10.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.506+0000 7f5183600640 1 --2- 172.21.15.161:0/2479745814 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f517c06d700 0x7f517c137fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:10.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.506+0000 7f5183600640 1 -- 172.21.15.161:0/2479745814 >> 172.21.15.161:0/2479745814 conn(0x7f517c0b0bc0 msgr2=0x7f517c0b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:10.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.506+0000 7f5183600640 1 -- 172.21.15.161:0/2479745814 shutdown_connections 2024-05-22T21:15:10.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:10.506+0000 7f5183600640 1 -- 172.21.15.161:0/2479745814 wait complete. 2024-05-22T21:15:10.529 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-05-22T21:15:10.694 INFO:teuthology.orchestra.run.smithi179.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":5374,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":20.622591095000001,"mdsmap_epoch":23,"osdmap_epoch":50,"osdmap_epoch_barrier":50,"uptime":24.849257424000001} 2024-05-22T21:15:10.695 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 5374, "mdsmap_epoch": 23, "osdmap_epoch": 50, "osdmap_epoch_barrier": 50, "rank_uptime": 20.622591095, "state": "up:active", "uptime": 24.849257424, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:15:10.695 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-22T21:15:10.695 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:15:10.695 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:15:10.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:10.705 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2024-05-22T21:15:10.753 INFO:teuthology.orchestra.run.smithi161.stderr:dd: failed to open 'small_file_1': Permission denied 2024-05-22T21:15:10.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:15:10.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:10.755 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-05-22T21:15:10.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:10.809 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-05-22T21:15:10.861 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:15:11.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.034+0000 7f0e0bc00640 1 Processor -- start 2024-05-22T21:15:11.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.034+0000 7f0e0bc00640 1 -- start start 2024-05-22T21:15:11.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.038+0000 7f0e0bc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0e040715a0 0x7f0e0415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:11.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.038+0000 7f0e0bc00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0e0415ec90 0x7f0e0415f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:11.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.038+0000 7f0e0bc00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e041442a0 con 0x7f0e04143ec0 2024-05-22T21:15:11.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.038+0000 7f0e0bc00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e0415f640 con 0x7f0e040715a0 2024-05-22T21:15:11.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.038+0000 7f0e0bc00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e04070500 con 0x7f0e0415ec90 2024-05-22T21:15:11.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.038+0000 7f0dfbe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0e040715a0 0x7f0e0415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:11.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.038+0000 7f0dfbe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0e040715a0 0x7f0e0415f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35832/0 (socket says 172.21.15.161:35832) 2024-05-22T21:15:11.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.038+0000 7f0dfbe00640 1 -- 172.21.15.161:0/2855715563 learned_addr learned my addr 172.21.15.161:0/2855715563 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:11.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.038+0000 7f0e09c00640 1 --2- 172.21.15.161:0/2855715563 >> v2:172.21.15.179:3300/0 conn(0x7f0e0415ec90 0x7f0e0415f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:11.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.038+0000 7f0dfbe00640 1 -- 172.21.15.161:0/2855715563 >> v2:172.21.15.179:3300/0 conn(0x7f0e0415ec90 msgr2=0x7f0e0415f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:11.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.038+0000 7f0dfbe00640 1 --2- 172.21.15.161:0/2855715563 >> v2:172.21.15.179:3300/0 conn(0x7f0e0415ec90 0x7f0e0415f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:11.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.038+0000 7f0dfbe00640 1 -- 172.21.15.161:0/2855715563 >> v1:172.21.15.179:6789/0 conn(0x7f0e04143ec0 legacy=0x7f0e0406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:11.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.038+0000 7f0dfbe00640 1 -- 172.21.15.161:0/2855715563 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e040707e0 con 0x7f0e040715a0 2024-05-22T21:15:11.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.042+0000 7f0dfbe00640 1 --2- 172.21.15.161:0/2855715563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e040715a0 0x7f0e0415f830 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0e040cedc0 tx=0x7f0dec01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132215c378fdaa4a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:11.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.042+0000 7f0e03e00640 1 -- 172.21.15.161:0/2855715563 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0ab5f0 con 0x7f0e040715a0 2024-05-22T21:15:11.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.042+0000 7f0e03e00640 1 -- 172.21.15.161:0/2855715563 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dec0ab750 con 0x7f0e040715a0 2024-05-22T21:15:11.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.042+0000 7f0e03e00640 1 -- 172.21.15.161:0/2855715563 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0abae0 con 0x7f0e040715a0 2024-05-22T21:15:11.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.042+0000 7f0e0bc00640 1 -- 172.21.15.161:0/2855715563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e040715a0 msgr2=0x7f0e0415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:11.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.042+0000 7f0e0bc00640 1 --2- 172.21.15.161:0/2855715563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e040715a0 0x7f0e0415f830 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0e040cedc0 tx=0x7f0dec01c4f0 comp rx=0 tx=0).stop 2024-05-22T21:15:11.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.042+0000 7f0e0bc00640 1 -- 172.21.15.161:0/2855715563 shutdown_connections 2024-05-22T21:15:11.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.042+0000 7f0e0bc00640 1 --2- 172.21.15.161:0/2855715563 >> v2:172.21.15.179:3300/0 conn(0x7f0e0415ec90 0x7f0e0415f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:11.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.042+0000 7f0e0bc00640 1 --2- 172.21.15.161:0/2855715563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e040715a0 0x7f0e0415f830 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:11.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.042+0000 7f0e0bc00640 1 -- 172.21.15.161:0/2855715563 >> 172.21.15.161:0/2855715563 conn(0x7f0e040b0bc0 msgr2=0x7f0e040b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:11.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.042+0000 7f0e0bc00640 1 -- 172.21.15.161:0/2855715563 shutdown_connections 2024-05-22T21:15:11.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.042+0000 7f0e0bc00640 1 -- 172.21.15.161:0/2855715563 wait complete. 2024-05-22T21:15:11.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.042+0000 7f0e0bc00640 1 Processor -- start 2024-05-22T21:15:11.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.042+0000 7f0e0bc00640 1 -- start start 2024-05-22T21:15:11.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e0bc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e04143ec0 0x7f0e04158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:11.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e0bc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e0415ec90 0x7f0e0413aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:11.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e0bc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e0415b710 0x7f0e0406c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:11.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e0bc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e04158dd0 con 0x7f0e0415ec90 2024-05-22T21:15:11.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e0bc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0415d480 con 0x7f0e04143ec0 2024-05-22T21:15:11.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e0bc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e04158f30 con 0x7f0e0415b710 2024-05-22T21:15:11.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0dfbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e0415ec90 0x7f0e0413aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:11.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0dfbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e0415ec90 0x7f0e0413aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35840/0 (socket says 172.21.15.161:35840) 2024-05-22T21:15:11.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0dfbe00640 1 -- 172.21.15.161:0/2443501535 learned_addr learned my addr 172.21.15.161:0/2443501535 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:11.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e09200640 1 --2- 172.21.15.161:0/2443501535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e04143ec0 0x7f0e04158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:11.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e09c00640 1 --2- 172.21.15.161:0/2443501535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e0415b710 0x7f0e0406c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:11.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0dfbe00640 1 -- 172.21.15.161:0/2443501535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e0415b710 msgr2=0x7f0e0406c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:11.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0dfbe00640 1 --2- 172.21.15.161:0/2443501535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e0415b710 0x7f0e0406c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:11.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0dfbe00640 1 -- 172.21.15.161:0/2443501535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e04143ec0 msgr2=0x7f0e04158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:11.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0dfbe00640 1 --2- 172.21.15.161:0/2443501535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e04143ec0 0x7f0e04158890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:11.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0dfbe00640 1 -- 172.21.15.161:0/2443501535 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec0094d0 con 0x7f0e0415ec90 2024-05-22T21:15:11.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e09c00640 1 --2- 172.21.15.161:0/2443501535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e0415b710 0x7f0e0406c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:11.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0dfbe00640 1 --2- 172.21.15.161:0/2443501535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e0415ec90 0x7f0e0413aec0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0dec019100 tx=0x7f0dec01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:11.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e01600640 1 -- 172.21.15.161:0/2443501535 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0be5f0 con 0x7f0e0415ec90 2024-05-22T21:15:11.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e01600640 1 -- 172.21.15.161:0/2443501535 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dec0be750 con 0x7f0e0415ec90 2024-05-22T21:15:11.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e01600640 1 -- 172.21.15.161:0/2443501535 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0beb00 con 0x7f0e0415ec90 2024-05-22T21:15:11.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e0bc00640 1 -- 172.21.15.161:0/2443501535 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0415d6d0 con 0x7f0e0415ec90 2024-05-22T21:15:11.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e0bc00640 1 -- 172.21.15.161:0/2443501535 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0415dc90 con 0x7f0e0415ec90 2024-05-22T21:15:11.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e0bc00640 1 -- 172.21.15.161:0/2443501535 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e041590a0 con 0x7f0e0415ec90 2024-05-22T21:15:11.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e01600640 1 -- 172.21.15.161:0/2443501535 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0dec0b43f0 con 0x7f0e0415ec90 2024-05-22T21:15:11.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.046+0000 7f0e01600640 1 --2- 172.21.15.161:0/2443501535 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0de0066690 0x7f0de0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:11.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.050+0000 7f0e01600640 1 -- 172.21.15.161:0/2443501535 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0dec0a8050 con 0x7f0e0415ec90 2024-05-22T21:15:11.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.050+0000 7f0e09200640 1 --2- 172.21.15.161:0/2443501535 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0de0066690 0x7f0de0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:11.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.050+0000 7f0e01600640 1 -- 172.21.15.161:0/2443501535 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=51}) v3 -- 0x7f0de006e1c0 con 0x7f0e0415ec90 2024-05-22T21:15:11.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.050+0000 7f0e09200640 1 --2- 172.21.15.161:0/2443501535 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0de0066690 0x7f0de0068b50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0df4007fd0 tx=0x7f0df4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:11.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.050+0000 7f0e01600640 1 -- 172.21.15.161:0/2443501535 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0dec0c4060 con 0x7f0e0415ec90 2024-05-22T21:15:11.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.214+0000 7f0e0bc00640 1 -- 172.21.15.161:0/2443501535 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f0e04067220 con 0x7f0e0415ec90 2024-05-22T21:15:11.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.214+0000 7f0e01600640 1 -- 172.21.15.161:0/2443501535 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) v1 ==== 81+0+33671 (secure 0 0 0) 0x7f0dec0ec150 con 0x7f0e0415ec90 2024-05-22T21:15:11.221 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:11.222 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:15:11.222 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 50, 2024-05-22T21:15:11.222 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:15:11.222 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:15:11.222 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:15:06.483153+0000", 2024-05-22T21:15:11.222 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:15:11.222 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:15:11.222 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:15:11.222 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:15:11.222 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:15:11.222 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:15:11.223 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:15:11.223 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:15:11.223 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:15:11.223 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:11.223 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:15:11.223 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:15:11.223 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:15:11.223 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:15:11.223 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:15:11.223 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 7, 2024-05-22T21:15:11.223 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:15:11.223 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:15:11.223 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:15:11.224 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:15:11.224 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:15:11.224 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:15:11.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.224 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:15:11.224 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:15:11.224 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:15:11.224 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:15:11.224 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:15:11.224 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:11.224 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:11.224 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:11.224 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:11.225 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:11.225 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:11.225 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:11.225 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:11.225 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:11.225 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:11.225 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:15:11.225 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:15:11.225 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:15:11.225 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:15:11.225 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:15:11.225 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:11.225 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:11.226 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:11.226 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:11.226 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:11.226 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:11.226 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:11.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.226 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "49", 2024-05-22T21:15:11.226 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:11.226 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:11.226 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:11.226 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:11.226 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:11.226 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:11.227 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:11.227 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:11.227 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:11.227 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:11.227 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:11.227 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:11.227 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:11.227 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:11.227 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:11.227 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:11.227 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:11.227 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:11.227 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:11.228 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:11.228 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:11.228 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:11.228 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:11.228 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:11.228 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:11.228 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:11.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.228 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:11.228 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:11.228 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:11.228 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:11.228 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:11.228 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:11.229 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:11.229 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:11.229 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:11.229 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:11.229 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:11.229 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:11.229 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:15:11.229 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:15:11.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.229 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:11.229 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:15:11.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.229 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:11.229 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:15:11.230 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:15:11.230 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:15:11.230 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:15:11.230 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:15:11.230 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:11.230 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:11.230 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:11.230 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.230 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 6, 2024-05-22T21:15:11.230 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:15:11.230 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:14:48.612726+0000", 2024-05-22T21:15:11.231 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:15:11.231 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:15:11.231 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:11.231 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:11.231 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:11.231 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:11.231 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:11.231 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:11.231 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:11.231 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:11.231 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:11.231 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:11.231 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:15:11.232 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:15:11.232 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:15:11.232 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:15:11.232 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:15:11.232 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:11.232 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:11.232 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:11.232 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:11.232 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:11.232 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:11.232 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:11.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.232 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "49", 2024-05-22T21:15:11.232 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:11.233 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:11.233 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:11.233 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:11.233 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:11.233 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:11.233 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:11.233 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:11.233 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:11.233 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:11.233 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:11.233 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:11.233 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:11.233 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:11.233 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:11.234 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:11.234 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:11.234 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:11.234 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:11.234 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:11.234 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:11.234 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:11.234 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:11.234 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:11.234 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:11.234 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:11.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.234 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:11.235 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:11.235 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:11.235 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:11.235 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:11.235 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:11.235 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:11.235 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:11.235 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:11.235 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:11.235 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:11.235 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:11.235 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:15:11.235 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:15:11.235 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:15:11.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.236 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:11.236 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:15:11.236 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:15:11.236 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.236 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:11.236 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:15:11.236 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:15:11.236 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:15:11.236 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:15:11.236 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:15:11.236 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:11.237 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:11.237 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:11.237 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.237 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.237 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 7, 2024-05-22T21:15:11.237 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:15:11.237 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:14:49.053022+0000", 2024-05-22T21:15:11.237 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:15:11.237 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:15:11.237 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:11.237 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:11.237 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:11.238 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:11.238 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:11.238 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:11.238 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:11.238 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:11.238 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:11.238 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:11.238 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:15:11.238 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:15:11.238 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:15:11.238 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:15:11.238 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:15:11.238 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:11.239 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:11.239 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:11.239 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:11.239 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:11.239 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:11.239 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:11.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.239 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "49", 2024-05-22T21:15:11.239 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:11.239 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:11.239 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:11.239 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:11.239 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:11.240 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:11.240 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:11.240 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:11.240 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:11.240 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:11.240 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:11.240 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:11.240 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:11.240 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:11.240 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:11.240 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:11.240 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:11.240 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:11.241 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:11.241 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:11.241 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:11.241 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:11.241 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:11.241 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:11.241 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:11.241 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:11.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.241 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:11.241 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:11.241 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:11.241 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:11.242 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:11.242 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:11.242 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:11.242 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:11.242 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:11.242 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:11.242 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:11.242 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:11.242 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:15:11.242 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:15:11.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.242 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:11.243 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:15:11.243 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:15:11.243 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.243 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:11.243 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:15:11.243 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:15:11.243 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:15:11.243 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:15:11.243 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:15:11.243 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:11.243 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:11.244 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:11.244 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.244 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.244 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:11.244 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:15:11.244 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.244 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:15:11.244 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:11.244 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:11.244 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:11.244 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:11.244 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:11.244 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:11.245 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:11.245 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:11.245 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:11.245 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:11.245 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:11.245 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:11.245 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.245 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.245 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:15:11.245 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:11.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.245 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.246 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.246 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:15:11.246 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:11.246 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.246 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.246 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:11.246 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.246 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.246 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.246 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:15:11.246 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:11.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.247 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.247 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.247 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:15:11.247 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:11.247 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.247 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.247 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:11.247 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.247 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.247 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.247 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:15:11.248 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:11.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.248 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.248 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:15:11.248 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:11.248 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.248 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.248 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:11.248 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.248 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.248 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.249 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:15:11.249 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:11.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.249 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.249 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.249 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:15:11.249 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:11.249 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.249 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.249 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:15:11.249 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:15:11.249 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:15:11.249 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:15:11.250 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:11.250 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:11.250 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:15:11.250 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:11.250 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.250 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.250 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:15:11.250 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:11.250 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:11.250 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:11.250 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:11.251 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:11.251 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:11.251 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:11.251 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:11.251 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:11.251 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:11.251 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:11.251 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:11.251 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.251 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.251 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.251 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:15:11.252 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:11.252 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.252 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.252 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.252 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:15:11.252 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:11.252 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.252 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.252 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.252 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:11.252 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.252 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.252 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.253 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:15:11.253 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:11.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.253 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.253 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:15:11.253 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:11.253 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.253 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.253 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:11.253 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.254 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.254 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:15:11.254 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:11.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.254 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.254 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.254 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:15:11.254 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:11.254 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.254 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.254 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:11.254 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.255 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.255 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:15:11.255 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:11.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.255 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.255 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:15:11.255 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:11.255 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.255 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.255 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:15:11.256 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:15:11.256 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:15:11.256 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:15:11.256 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:11.256 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:11.256 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:11.256 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.256 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.256 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.256 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:15:11.256 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:11.256 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:11.256 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:11.256 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:11.257 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:11.257 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:11.257 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:11.257 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:11.257 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:11.257 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:11.257 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:11.257 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:11.257 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.257 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.257 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.257 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:15:11.257 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:11.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.258 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.258 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.258 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:15:11.258 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:11.258 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.258 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.258 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:11.258 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.258 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.258 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.258 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:15:11.259 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:11.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.259 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.259 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.259 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:15:11.259 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:11.259 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.259 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.259 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:11.259 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.259 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.259 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.260 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:15:11.260 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:11.260 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.260 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.260 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.260 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:15:11.260 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:11.260 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.260 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.260 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.260 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:11.260 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.260 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.261 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.261 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:15:11.261 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:11.261 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.261 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.261 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.261 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:15:11.261 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:11.261 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.261 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.261 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.261 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:15:11.261 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:15:11.262 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:15:11.262 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:15:11.262 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:11.262 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:11.262 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:15:11.262 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:11.262 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.262 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.262 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.262 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:15:11.262 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:11.262 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:11.263 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:11.263 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:11.263 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:11.263 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:11.263 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:11.263 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:11.263 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:11.263 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:11.263 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:11.263 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:11.263 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.263 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.263 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.263 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:15:11.264 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:11.264 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.264 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.264 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.264 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:15:11.264 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:11.264 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.264 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.264 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.264 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:11.264 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.264 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.264 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.265 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:15:11.265 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:11.265 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.265 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.265 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.265 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:15:11.265 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:11.265 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.265 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.265 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.265 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:11.265 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.265 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.266 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.266 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:15:11.266 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:11.266 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.266 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.266 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.266 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:15:11.266 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:11.266 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.266 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.266 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.266 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:11.267 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.267 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.267 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.267 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:15:11.267 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:11.267 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.267 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.267 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.267 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:15:11.267 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:11.267 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.267 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.267 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.267 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:15:11.268 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:15:11.268 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:15:11.268 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:15:11.268 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:11.268 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull", 2024-05-22T21:15:11.268 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:11.268 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:11.268 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.268 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.268 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.268 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:15:11.268 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:11.268 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:11.269 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:11.269 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:11.269 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:11.269 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:11.269 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:11.269 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:11.269 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:11.269 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:11.269 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:11.269 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:11.269 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.269 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.269 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.269 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:15:11.270 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:11.270 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.270 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.270 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.270 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:15:11.270 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:11.270 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.270 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.270 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.270 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:11.270 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.270 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.271 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.271 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:15:11.271 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:11.271 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.271 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.271 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.271 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:15:11.271 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:11.271 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.271 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.271 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.271 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:11.271 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.272 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.272 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.272 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:15:11.272 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:11.272 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.272 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.272 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.272 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:15:11.272 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:11.272 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.272 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.272 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.272 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:11.272 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.273 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.273 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.273 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:15:11.273 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:11.273 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.273 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.273 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.273 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:15:11.273 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:11.273 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.273 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.273 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.273 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:15:11.274 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:15:11.274 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:15:11.274 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:15:11.274 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:11.274 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:11.274 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:11.274 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.274 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.274 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.274 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:15:11.274 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:11.274 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:11.274 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.275 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:15:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:15:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.276 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.277 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:15:11.277 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:11.277 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.277 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.277 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.277 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:15:11.277 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:11.277 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.277 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.277 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.277 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:11.277 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.277 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.278 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.278 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:15:11.278 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:11.278 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.278 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.278 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.278 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:15:11.278 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:11.278 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.278 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.278 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.278 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:11.278 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.279 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.279 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.279 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:15:11.279 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:11.279 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.279 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.279 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.279 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:15:11.279 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:11.279 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.279 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.279 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.280 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:15:11.280 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:15:11.280 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:15:11.280 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:15:11.280 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:11.280 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:11.280 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:11.280 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.280 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.280 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.280 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:15:11.280 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:11.280 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:11.280 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:11.281 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:11.281 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:11.281 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:11.281 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:11.281 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:11.281 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:11.281 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:11.281 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:11.281 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:11.281 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.281 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.281 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.281 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:15:11.282 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:11.282 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.282 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.282 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.282 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:15:11.282 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:11.282 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.282 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.282 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.282 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:11.282 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.282 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.283 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.283 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:15:11.283 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:11.283 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.283 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.283 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.283 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:15:11.283 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:11.283 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.283 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.283 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.283 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:11.283 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.283 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.284 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.284 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:15:11.284 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:11.284 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.284 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.284 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.284 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:15:11.284 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:11.284 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.284 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.284 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.284 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:11.284 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.285 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.285 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.285 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:15:11.285 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:11.285 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.285 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.285 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.285 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:15:11.285 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:11.285 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.285 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.285 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.285 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:15:11.286 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:15:11.286 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:15:11.286 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:15:11.286 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:11.286 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:11.286 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:15:11.286 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:11.286 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.286 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.286 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.286 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:15:11.286 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:11.286 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:11.286 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:11.287 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:11.287 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:11.287 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:11.287 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:11.287 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:11.287 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:11.287 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:11.287 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:11.287 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:11.287 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.287 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.287 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.287 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:15:11.288 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:11.288 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.288 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.288 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.288 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:15:11.288 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:11.288 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.288 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.288 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.288 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:11.288 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.288 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.288 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.289 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:15:11.289 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:11.289 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.289 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.289 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.289 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:15:11.289 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:11.289 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.289 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.289 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.289 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:11.289 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.289 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.289 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.290 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:15:11.290 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:11.290 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.290 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.290 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.290 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:15:11.290 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:11.290 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.290 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.290 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.290 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:11.290 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:11.291 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.291 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:11.291 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:15:11.291 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:11.291 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.291 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.291 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:11.291 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:15:11.291 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:11.291 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.291 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.291 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.291 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:15:11.291 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:15:11.291 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:15:11.292 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:15:11.292 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:11.292 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:11.292 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:15:11.292 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:11.292 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:11.292 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.292 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:11.292 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:15:11.292 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.292 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:15:11.292 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:11.292 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:11.292 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:11.293 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:11.293 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:11.293 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:15:11.293 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:11.293 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.293 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.293 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:15:11.293 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:11.293 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:11.293 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:11.293 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:11.293 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:11.293 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:15:11.293 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:11.293 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.294 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.294 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:15:11.294 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:11.294 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:11.294 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:11.294 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:11.294 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:11.294 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:15:11.294 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:11.294 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.294 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.294 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:15:11.294 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:11.294 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:11.295 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:11.295 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:11.295 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:11.295 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:15:11.295 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:11.295 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.295 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.295 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:15:11.295 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:11.295 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:11.295 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:11.295 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:11.295 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:11.296 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:15:11.296 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:11.296 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.296 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.296 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:15:11.296 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:11.296 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:11.296 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:11.296 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:11.296 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:11.296 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:15:11.296 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:11.296 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.296 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.296 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:15:11.297 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:11.297 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:11.297 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:11.297 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:11.297 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:11.297 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:15:11.297 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:11.297 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.297 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:11.297 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:15:11.297 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:11.297 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:11.297 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:11.297 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:11.298 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:11.298 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:15:11.298 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:11.298 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.298 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:11.298 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:15:11.298 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:15:11.298 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:15:11.298 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:15:11.298 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:15:11.298 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:15:11.298 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:15:11.298 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:15:11.299 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:15:11.299 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:15:11.299 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:15:11.299 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:15:11.299 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:15:11.299 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:15:11.299 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.299 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:11.299 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:15:11.299 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:15:11.299 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:15:11.299 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:15:11.299 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:15:11.299 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:15:11.300 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:15:11.300 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:15:11.300 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:15:11.300 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:15:11.300 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:15:11.300 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:11.300 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:15:11.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.218+0000 7f0e0bc00640 1 -- 172.21.15.161:0/2443501535 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0de0066690 msgr2=0x7f0de0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:11.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.218+0000 7f0e0bc00640 1 --2- 172.21.15.161:0/2443501535 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0de0066690 0x7f0de0068b50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0df4007fd0 tx=0x7f0df4009040 comp rx=0 tx=0).stop 2024-05-22T21:15:11.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.218+0000 7f0e0bc00640 1 -- 172.21.15.161:0/2443501535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e0415ec90 msgr2=0x7f0e0413aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:11.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.218+0000 7f0e0bc00640 1 --2- 172.21.15.161:0/2443501535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e0415ec90 0x7f0e0413aec0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0dec019100 tx=0x7f0dec01ad30 comp rx=0 tx=0).stop 2024-05-22T21:15:11.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.218+0000 7f0e0bc00640 1 -- 172.21.15.161:0/2443501535 shutdown_connections 2024-05-22T21:15:11.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.218+0000 7f0e0bc00640 1 --2- 172.21.15.161:0/2443501535 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0de0066690 0x7f0de0068b50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:11.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.218+0000 7f0e0bc00640 1 --2- 172.21.15.161:0/2443501535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e0415b710 0x7f0e0406c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:11.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.218+0000 7f0e0bc00640 1 --2- 172.21.15.161:0/2443501535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e0415ec90 0x7f0e0413aec0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:11.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.218+0000 7f0e0bc00640 1 --2- 172.21.15.161:0/2443501535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e04143ec0 0x7f0e04158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:11.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.218+0000 7f0e0bc00640 1 -- 172.21.15.161:0/2443501535 >> 172.21.15.161:0/2443501535 conn(0x7f0e040b0bc0 msgr2=0x7f0e040ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:11.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.218+0000 7f0e0bc00640 1 -- 172.21.15.161:0/2443501535 shutdown_connections 2024-05-22T21:15:11.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:11.218+0000 7f0e0bc00640 1 -- 172.21.15.161:0/2443501535 wait complete. 2024-05-22T21:15:11.303 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-05-22T21:15:12.509 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:15:12.506+0000 7f7d35a00640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2024-05-22T21:15:16.304 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:15:16.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e15a00640 1 Processor -- start 2024-05-22T21:15:16.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e15a00640 1 -- start start 2024-05-22T21:15:16.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e15a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3e101597f0 0x7f3e100706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:16.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e15a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3e10070be0 0x7f3e100b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:16.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e15a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e100b2ea0 0x7f3e1006c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:16.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e15a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e10071040 con 0x7f3e101597f0 2024-05-22T21:15:16.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e15a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e10159bd0 con 0x7f3e10070be0 2024-05-22T21:15:16.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e15a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e100cbdd0 con 0x7f3e100b2ea0 2024-05-22T21:15:16.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e0ea00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3e101597f0 0x7f3e100706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:16.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e0ea00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3e101597f0 0x7f3e100706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51020/0 (socket says 172.21.15.161:51020) 2024-05-22T21:15:16.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e0ea00640 1 -- 172.21.15.161:0/3941274000 learned_addr learned my addr 172.21.15.161:0/3941274000 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:16.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e0ea00640 1 -- 172.21.15.161:0/3941274000 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e100b2ea0 msgr2=0x7f3e1006c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:16.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e06000640 1 --2- 172.21.15.161:0/3941274000 >> v2:172.21.15.179:3300/0 conn(0x7f3e10070be0 0x7f3e100b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:16.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e0f400640 1 --2- 172.21.15.161:0/3941274000 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e100b2ea0 0x7f3e1006c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:16.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e0ea00640 1 --2- 172.21.15.161:0/3941274000 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e100b2ea0 0x7f3e1006c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:16.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e0ea00640 1 -- 172.21.15.161:0/3941274000 >> v2:172.21.15.179:3300/0 conn(0x7f3e10070be0 msgr2=0x7f3e100b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:16.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e0ea00640 1 --2- 172.21.15.161:0/3941274000 >> v2:172.21.15.179:3300/0 conn(0x7f3e10070be0 0x7f3e100b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:16.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e0ea00640 1 -- 172.21.15.161:0/3941274000 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e100cc050 con 0x7f3e101597f0 2024-05-22T21:15:16.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e06000640 1 --2- 172.21.15.161:0/3941274000 >> v2:172.21.15.179:3300/0 conn(0x7f3e10070be0 0x7f3e100b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:16.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.446+0000 7f3e0f400640 1 --2- 172.21.15.161:0/3941274000 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e100b2ea0 0x7f3e1006c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:16.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.450+0000 7f3e0ea00640 1 --2- 172.21.15.161:0/3941274000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e101597f0 0x7f3e100706a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3e10073ce0 tx=0x7f3e0001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e4b5cae524c013 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:16.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.450+0000 7f3e0e000640 1 -- 172.21.15.161:0/3941274000 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e000ab660 con 0x7f3e101597f0 2024-05-22T21:15:16.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.450+0000 7f3e0e000640 1 -- 172.21.15.161:0/3941274000 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e000ab7c0 con 0x7f3e101597f0 2024-05-22T21:15:16.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.450+0000 7f3e0e000640 1 -- 172.21.15.161:0/3941274000 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e000abb50 con 0x7f3e101597f0 2024-05-22T21:15:16.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.450+0000 7f3e15a00640 1 -- 172.21.15.161:0/3941274000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e101597f0 msgr2=0x7f3e100706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:16.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.450+0000 7f3e15a00640 1 --2- 172.21.15.161:0/3941274000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e101597f0 0x7f3e100706a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3e10073ce0 tx=0x7f3e0001c520 comp rx=0 tx=0).stop 2024-05-22T21:15:16.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.450+0000 7f3e15a00640 1 -- 172.21.15.161:0/3941274000 shutdown_connections 2024-05-22T21:15:16.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.450+0000 7f3e15a00640 1 --2- 172.21.15.161:0/3941274000 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e100b2ea0 0x7f3e1006c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:16.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.450+0000 7f3e15a00640 1 --2- 172.21.15.161:0/3941274000 >> v2:172.21.15.179:3300/0 conn(0x7f3e10070be0 0x7f3e100b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:16.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.450+0000 7f3e15a00640 1 --2- 172.21.15.161:0/3941274000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e101597f0 0x7f3e100706a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:16.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.450+0000 7f3e15a00640 1 -- 172.21.15.161:0/3941274000 >> 172.21.15.161:0/3941274000 conn(0x7f3e100b0bc0 msgr2=0x7f3e100b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:16.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.450+0000 7f3e15a00640 1 -- 172.21.15.161:0/3941274000 shutdown_connections 2024-05-22T21:15:16.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.450+0000 7f3e15a00640 1 -- 172.21.15.161:0/3941274000 wait complete. 2024-05-22T21:15:16.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e15a00640 1 Processor -- start 2024-05-22T21:15:16.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e15a00640 1 -- start start 2024-05-22T21:15:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e15a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e10070be0 0x7f3e1013ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e15a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e100b2ea0 0x7f3e10142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e15a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e101597f0 0x7f3e101452d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e15a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e10147900 con 0x7f3e10070be0 2024-05-22T21:15:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e15a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e10143830 con 0x7f3e101597f0 2024-05-22T21:15:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e15a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e10143b10 con 0x7f3e100b2ea0 2024-05-22T21:15:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e0ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e10070be0 0x7f3e1013ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e0ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e10070be0 0x7f3e1013ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51022/0 (socket says 172.21.15.161:51022) 2024-05-22T21:15:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e0ea00640 1 -- 172.21.15.161:0/3401753281 learned_addr learned my addr 172.21.15.161:0/3401753281 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e06000640 1 --2- 172.21.15.161:0/3401753281 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e100b2ea0 0x7f3e10142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e0f400640 1 --2- 172.21.15.161:0/3401753281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e101597f0 0x7f3e101452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e0ea00640 1 -- 172.21.15.161:0/3401753281 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e100b2ea0 msgr2=0x7f3e10142d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e0ea00640 1 --2- 172.21.15.161:0/3401753281 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e100b2ea0 0x7f3e10142d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e0ea00640 1 -- 172.21.15.161:0/3401753281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e101597f0 msgr2=0x7f3e101452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e0ea00640 1 --2- 172.21.15.161:0/3401753281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e101597f0 0x7f3e101452d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e0ea00640 1 -- 172.21.15.161:0/3401753281 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e000094d0 con 0x7f3e10070be0 2024-05-22T21:15:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e06000640 1 --2- 172.21.15.161:0/3401753281 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e100b2ea0 0x7f3e10142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e0ea00640 1 --2- 172.21.15.161:0/3401753281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e10070be0 0x7f3e1013ef50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3e0001abd0 tx=0x7f3e0001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e07400640 1 -- 172.21.15.161:0/3401753281 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e000b4800 con 0x7f3e10070be0 2024-05-22T21:15:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e07400640 1 -- 172.21.15.161:0/3401753281 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e000b4980 con 0x7f3e10070be0 2024-05-22T21:15:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e15a00640 1 -- 172.21.15.161:0/3401753281 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e10143df0 con 0x7f3e10070be0 2024-05-22T21:15:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e07400640 1 -- 172.21.15.161:0/3401753281 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e000b4d30 con 0x7f3e10070be0 2024-05-22T21:15:16.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.454+0000 7f3e15a00640 1 -- 172.21.15.161:0/3401753281 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e101f1ab0 con 0x7f3e10070be0 2024-05-22T21:15:16.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.458+0000 7f3e15a00640 1 -- 172.21.15.161:0/3401753281 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e10067220 con 0x7f3e10070be0 2024-05-22T21:15:16.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.458+0000 7f3e07400640 1 -- 172.21.15.161:0/3401753281 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3e000a9070 con 0x7f3e10070be0 2024-05-22T21:15:16.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.458+0000 7f3e07400640 1 --2- 172.21.15.161:0/3401753281 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3dec066690 0x7f3dec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:16.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.458+0000 7f3e07400640 1 -- 172.21.15.161:0/3401753281 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3e0011ed00 con 0x7f3e10070be0 2024-05-22T21:15:16.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.458+0000 7f3e06000640 1 --2- 172.21.15.161:0/3401753281 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3dec066690 0x7f3dec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:16.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.458+0000 7f3e07400640 1 -- 172.21.15.161:0/3401753281 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=55}) v3 -- 0x7f3dec06e280 con 0x7f3e10070be0 2024-05-22T21:15:16.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.458+0000 7f3e06000640 1 --2- 172.21.15.161:0/3401753281 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3dec066690 0x7f3dec068b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3df8006fd0 tx=0x7f3df8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:16.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.462+0000 7f3e07400640 1 -- 172.21.15.161:0/3401753281 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3e000ec140 con 0x7f3e10070be0 2024-05-22T21:15:16.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.618+0000 7f3e15a00640 1 -- 172.21.15.161:0/3401753281 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f3e10067220 con 0x7f3e10070be0 2024-05-22T21:15:16.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.618+0000 7f3e07400640 1 -- 172.21.15.161:0/3401753281 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v54) v1 ==== 81+0+33936 (secure 0 0 0) 0x7f3e000ef760 con 0x7f3e10070be0 2024-05-22T21:15:16.625 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:16.625 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:15:16.625 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 54, 2024-05-22T21:15:16.625 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:15:16.625 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:15:16.625 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:15:15.761565+0000", 2024-05-22T21:15:16.625 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:15:16.626 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:15:16.626 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:15:16.626 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:15:16.626 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:15:16.626 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:15:16.626 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:15:16.626 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:15:16.626 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:15:16.626 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:16.626 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:15:16.626 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:15:16.627 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:15:16.627 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:15:16.627 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:15:16.627 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 7, 2024-05-22T21:15:16.627 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:15:16.627 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:15:16.627 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "luminous", 2024-05-22T21:15:16.627 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:15:16.627 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:15:16.627 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:15:16.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.627 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:15:16.627 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:15:16.627 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:15:16.628 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:15:16.628 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:15:16.628 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:16.628 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:16.628 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:16.628 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:16.628 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:16.628 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:16.628 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:16.628 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:16.628 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:16.628 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:16.628 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:15:16.629 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:15:16.629 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:15:16.629 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:15:16.629 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:15:16.629 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:16.629 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:16.629 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:16.629 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:16.629 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:16.629 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:16.629 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:16.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.630 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "49", 2024-05-22T21:15:16.630 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:16.630 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:16.630 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:16.630 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:16.630 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:16.630 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:16.630 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:16.630 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:16.630 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:16.630 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:16.630 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:16.631 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:16.631 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:16.631 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:16.631 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:16.631 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:16.631 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:16.631 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:16.631 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:16.631 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:16.631 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:16.631 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:16.631 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:16.631 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:16.631 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:16.632 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:16.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.632 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:16.632 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:16.632 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:16.632 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:16.632 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:16.632 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:16.632 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:16.632 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:16.632 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:16.632 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:16.632 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:16.633 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:16.633 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:15:16.633 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:15:16.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.633 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:16.633 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:15:16.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.633 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:16.633 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:15:16.633 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:15:16.633 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:15:16.633 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:15:16.633 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:15:16.633 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:16.634 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:16.634 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:16.634 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.634 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 6, 2024-05-22T21:15:16.634 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:15:16.634 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:14:48.612726+0000", 2024-05-22T21:15:16.634 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:15:16.634 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:15:16.634 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:16.634 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:16.634 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:16.635 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:16.635 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:16.635 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:16.635 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:16.635 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:16.635 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:16.635 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:16.635 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:15:16.635 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:15:16.635 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:15:16.635 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:15:16.635 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:15:16.635 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:16.636 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:16.636 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:16.636 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:16.636 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:16.636 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:16.636 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:16.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.636 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "49", 2024-05-22T21:15:16.636 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:16.636 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:16.636 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:16.636 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:16.636 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:16.636 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:16.637 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:16.637 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:16.637 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:16.637 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:16.637 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:16.637 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:16.637 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:16.637 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:16.637 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:16.637 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:16.637 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:16.637 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:16.637 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:16.637 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:16.638 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:16.638 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:16.638 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:16.638 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:16.638 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:16.638 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:16.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.638 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:16.638 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:16.638 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:16.638 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:16.638 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:16.638 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:16.638 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:16.639 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:16.639 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:16.639 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:16.639 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:16.639 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:16.639 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:15:16.639 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:15:16.639 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:15:16.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.639 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:16.639 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:15:16.639 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:15:16.639 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.640 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:16.640 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:15:16.640 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:15:16.640 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:15:16.640 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:15:16.640 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:15:16.640 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:16.640 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:16.640 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:16.640 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.640 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.640 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 7, 2024-05-22T21:15:16.641 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:15:16.641 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:14:49.053022+0000", 2024-05-22T21:15:16.641 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:15:16.641 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:15:16.641 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:16.641 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:16.641 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:16.641 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:16.641 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:16.641 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:16.641 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:16.641 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:16.641 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:16.642 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:16.642 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:15:16.642 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:15:16.642 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:15:16.642 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:15:16.642 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:15:16.642 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:16.642 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:16.642 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:16.642 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:16.642 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:16.642 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:16.643 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:16.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.643 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "49", 2024-05-22T21:15:16.643 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:16.643 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:16.643 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:16.643 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:16.643 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:16.643 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:16.643 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:16.643 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:16.643 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:16.643 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:16.643 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:16.644 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:16.644 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:16.644 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:16.644 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:16.644 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:16.644 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:16.644 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:16.644 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:16.644 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:16.644 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:16.644 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:16.644 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:16.644 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:16.644 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:16.645 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:16.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.645 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:16.645 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:16.645 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:16.645 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:16.645 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:16.645 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:16.645 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:16.645 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:16.645 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:16.645 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:16.645 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:16.646 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:16.646 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:15:16.646 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:15:16.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.646 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:16.646 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:15:16.646 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:15:16.646 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.646 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:16.646 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.5, 2024-05-22T21:15:16.646 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.5, 2024-05-22T21:15:16.647 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:15:16.647 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.5, 2024-05-22T21:15:16.647 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.5, 2024-05-22T21:15:16.647 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:16.647 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:16.647 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:16.647 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.647 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.647 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:16.647 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:15:16.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.647 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:15:16.647 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:16.647 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:16.648 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:16.648 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:16.648 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:16.648 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:16.648 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:16.648 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:16.648 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:16.648 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:16.648 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:16.648 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:16.648 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.648 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.649 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:15:16.649 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:16.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.649 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.649 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:15:16.649 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:16.649 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.649 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.649 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:16.649 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.649 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.650 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:15:16.650 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:16.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.650 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.650 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:15:16.650 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:16.650 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.650 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.650 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:16.650 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.651 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.651 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:15:16.651 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:16.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.651 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.651 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:15:16.651 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:16.651 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.651 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.652 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:16.652 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.652 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.652 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:15:16.652 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:16.652 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.652 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.652 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.652 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:15:16.652 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:16.652 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.652 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.653 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:15:16.653 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:15:16.653 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:15:16.653 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:15:16.653 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:16.653 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:16.653 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:16.653 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.653 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.653 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:15:16.653 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:16.654 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:16.654 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:16.654 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:16.654 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:16.654 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:16.654 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:16.654 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:16.654 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:16.654 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:16.654 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:16.654 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:16.654 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.655 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.655 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:15:16.655 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:16.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.655 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.655 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.655 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:15:16.655 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:16.656 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.656 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.656 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:16.656 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.656 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.656 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:15:16.656 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:16.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.656 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.656 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.657 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:15:16.657 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:16.657 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.657 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.657 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:16.657 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.657 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.657 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:15:16.657 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:16.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.657 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.658 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.658 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:15:16.658 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:16.658 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.658 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.658 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:16.658 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.658 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.658 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:15:16.658 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:16.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.659 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.659 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.659 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:15:16.659 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:16.659 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.659 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.659 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:15:16.659 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:15:16.659 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:15:16.659 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:15:16.659 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:16.659 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:16.660 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:16.660 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.660 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.660 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:15:16.660 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:16.660 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:16.660 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:16.660 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:16.660 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:16.660 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:16.660 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:16.660 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:16.660 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:16.661 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:16.661 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:16.661 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:16.661 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.661 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.661 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:15:16.661 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:16.661 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.661 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.661 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.661 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:15:16.661 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:16.661 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.662 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.662 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:16.662 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.662 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.662 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:15:16.662 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:16.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.662 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.662 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.662 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:15:16.663 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:16.663 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.663 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.663 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:16.663 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.663 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.663 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:15:16.663 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:16.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.663 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.663 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.664 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:15:16.664 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:16.664 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.664 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.664 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:16.664 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.664 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.664 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:15:16.664 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:16.664 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.665 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.665 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:15:16.665 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:16.665 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.665 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.665 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:15:16.665 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:15:16.665 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:15:16.665 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:15:16.665 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:16.665 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:16.665 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:15:16.666 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:16.666 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.666 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:15:16.666 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:16.666 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:16.666 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:16.666 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:16.666 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:16.666 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:16.666 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:16.667 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:16.667 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:16.667 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:16.667 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:16.667 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:16.667 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.667 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.667 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:15:16.667 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:16.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.667 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.668 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:15:16.668 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:16.668 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.668 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.668 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:16.668 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.668 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.668 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:15:16.668 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:16.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.669 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.669 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:15:16.669 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:16.669 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.669 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.669 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:16.669 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.669 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.669 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:15:16.669 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:16.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.670 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.670 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:15:16.670 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:16.670 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.670 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.670 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:16.670 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.670 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.670 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.670 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:15:16.670 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:16.670 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.671 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.671 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:15:16.671 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:16.671 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.671 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.671 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:15:16.671 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:15:16.671 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:15:16.671 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:15:16.671 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:16.671 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:16.671 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:16.672 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.672 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:15:16.672 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:16.672 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:16.672 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:16.672 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:16.672 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:16.672 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:16.672 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:16.672 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:16.672 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 51, 2024-05-22T21:15:16.673 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:16.673 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:16.673 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:16.673 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.673 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.673 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:15:16.673 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:16.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.673 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.673 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:15:16.674 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:16.674 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.674 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.674 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:16.674 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.674 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.674 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:15:16.674 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:16.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.674 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.675 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:15:16.675 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:16.675 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.675 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.675 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:16.675 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.675 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.675 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:15:16.675 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:16.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.676 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.676 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:15:16.676 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:16.676 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.676 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.676 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:16.676 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.676 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.676 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:15:16.676 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:16.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.677 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.677 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:15:16.677 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:16.677 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.677 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.677 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:15:16.677 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:15:16.677 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:15:16.677 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:15:16.677 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:16.678 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:16.678 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:16.678 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.678 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:15:16.678 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:16.678 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:16.678 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:16.678 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:16.678 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:16.678 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:16.678 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:16.679 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:16.679 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 51, 2024-05-22T21:15:16.679 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:16.679 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:16.679 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:16.679 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.679 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.679 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:15:16.679 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:16.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.679 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.679 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:15:16.679 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:16.680 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.680 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.680 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:16.680 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.680 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.680 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:15:16.680 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:16.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.680 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.680 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.680 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:15:16.681 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:16.681 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.681 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.681 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:16.681 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.681 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.681 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:15:16.681 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:16.681 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.681 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.681 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:15:16.682 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:16.682 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.682 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.682 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:16.682 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.682 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.682 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:15:16.682 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:16.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.682 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.683 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:15:16.683 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:16.683 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.683 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.683 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:15:16.683 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:15:16.683 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:15:16.683 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:15:16.683 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:16.683 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:16.683 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:16.683 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.684 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:15:16.684 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:16.684 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:16.684 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:16.684 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:16.684 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:16.684 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:16.684 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:16.684 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:16.684 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:16.684 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:16.684 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:16.684 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:16.685 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.685 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.685 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:15:16.685 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:16.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.685 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.685 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:15:16.685 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:16.685 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.685 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.685 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:16.686 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.686 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.686 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:15:16.686 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:16.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.686 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.686 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.686 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:15:16.686 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:16.686 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.686 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.686 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:16.687 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.687 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.687 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:15:16.687 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:16.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.687 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.687 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:15:16.687 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:16.687 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.687 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.687 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.687 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:16.688 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.688 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.688 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:15:16.688 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:16.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.688 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.688 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:15:16.688 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:16.688 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.688 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.688 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:15:16.689 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:15:16.689 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:15:16.689 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:15:16.689 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:16.689 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:16.689 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:16.689 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.689 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:15:16.689 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:16.689 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:16.689 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:16.690 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:16.690 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:16.690 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:16.690 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:16.690 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:16.690 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:16.690 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:16.690 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:16.690 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:16.690 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.690 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.691 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:15:16.691 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:16.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.691 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.691 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:15:16.691 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:16.691 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.691 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.691 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:16.691 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.692 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.692 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:15:16.692 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:16.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.692 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.692 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:15:16.692 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:16.692 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.692 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.692 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:16.692 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.693 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.693 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:15:16.693 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:16.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.693 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.693 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:15:16.693 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:16.693 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.693 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.693 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:16.693 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:16.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.694 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:16.694 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:15:16.694 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:16.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.694 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:16.694 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:15:16.694 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:16.694 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.694 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.695 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:15:16.695 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:15:16.695 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:15:16.695 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:15:16.695 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:16.695 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:16.695 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:16.695 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.695 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.695 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:16.695 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:15:16.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.695 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:15:16.696 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:16.696 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:16.696 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:16.696 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:16.696 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:16.696 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:15:16.696 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:16.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.696 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:15:16.696 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:16.696 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:16.696 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:16.696 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:16.697 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:16.697 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:15:16.697 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:16.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.697 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:15:16.697 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:16.697 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:16.697 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:16.697 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:16.697 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:16.697 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:15:16.697 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:16.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.698 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:15:16.698 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:16.698 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:16.698 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:16.698 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:16.698 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:16.698 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:15:16.698 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:16.698 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.698 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:15:16.699 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:16.699 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:16.699 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:16.699 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:16.699 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:16.699 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:15:16.699 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:16.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.699 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:15:16.699 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:16.699 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:16.699 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:16.700 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:16.700 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:16.700 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:15:16.700 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:16.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.700 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:15:16.700 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:16.700 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:16.700 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:16.700 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:16.700 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:16.700 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:15:16.701 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:16.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.701 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:15:16.701 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:16.701 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:16.701 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:16.701 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:16.701 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:16.701 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:15:16.701 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:16.701 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.701 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:16.702 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:15:16.702 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [ 2024-05-22T21:15:16.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.702 INFO:teuthology.orchestra.run.smithi161.stdout: "pgid": "7.6", 2024-05-22T21:15:16.702 INFO:teuthology.orchestra.run.smithi161.stdout: "mappings": [ 2024-05-22T21:15:16.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.702 INFO:teuthology.orchestra.run.smithi161.stdout: "from": 0, 2024-05-22T21:15:16.702 INFO:teuthology.orchestra.run.smithi161.stdout: "to": 4 2024-05-22T21:15:16.702 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.702 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.702 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.702 INFO:teuthology.orchestra.run.smithi161.stdout: "pgid": "7.9", 2024-05-22T21:15:16.703 INFO:teuthology.orchestra.run.smithi161.stdout: "mappings": [ 2024-05-22T21:15:16.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:16.703 INFO:teuthology.orchestra.run.smithi161.stdout: "from": 0, 2024-05-22T21:15:16.703 INFO:teuthology.orchestra.run.smithi161.stdout: "to": 5 2024-05-22T21:15:16.703 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.703 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:16.703 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.703 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:16.703 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:15:16.703 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:15:16.703 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:15:16.703 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:15:16.704 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:15:16.704 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:15:16.704 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:15:16.704 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:15:16.704 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:15:16.704 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:15:16.704 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:15:16.704 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:15:16.704 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:16.704 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:15:16.704 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:15:16.704 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:15:16.704 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:15:16.705 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:15:16.705 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:15:16.705 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:15:16.705 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:15:16.705 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:15:16.705 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:15:16.705 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:15:16.705 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:16.705 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:15:16.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.622+0000 7f3e15a00640 1 -- 172.21.15.161:0/3401753281 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3dec066690 msgr2=0x7f3dec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:16.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.622+0000 7f3e15a00640 1 --2- 172.21.15.161:0/3401753281 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3dec066690 0x7f3dec068b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3df8006fd0 tx=0x7f3df8008040 comp rx=0 tx=0).stop 2024-05-22T21:15:16.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.622+0000 7f3e15a00640 1 -- 172.21.15.161:0/3401753281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e10070be0 msgr2=0x7f3e1013ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:16.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.622+0000 7f3e15a00640 1 --2- 172.21.15.161:0/3401753281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e10070be0 0x7f3e1013ef50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3e0001abd0 tx=0x7f3e0001ad60 comp rx=0 tx=0).stop 2024-05-22T21:15:16.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.622+0000 7f3e15a00640 1 -- 172.21.15.161:0/3401753281 shutdown_connections 2024-05-22T21:15:16.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.622+0000 7f3e15a00640 1 --2- 172.21.15.161:0/3401753281 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3dec066690 0x7f3dec068b50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:16.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.622+0000 7f3e15a00640 1 --2- 172.21.15.161:0/3401753281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e101597f0 0x7f3e101452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:16.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.622+0000 7f3e15a00640 1 --2- 172.21.15.161:0/3401753281 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e100b2ea0 0x7f3e10142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:16.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.622+0000 7f3e15a00640 1 --2- 172.21.15.161:0/3401753281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e10070be0 0x7f3e1013ef50 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:16.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.622+0000 7f3e15a00640 1 -- 172.21.15.161:0/3401753281 >> 172.21.15.161:0/3401753281 conn(0x7f3e100b0bc0 msgr2=0x7f3e10156ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:16.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.622+0000 7f3e15a00640 1 -- 172.21.15.161:0/3401753281 shutdown_connections 2024-05-22T21:15:16.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:16.622+0000 7f3e15a00640 1 -- 172.21.15.161:0/3401753281 wait complete. 2024-05-22T21:15:16.708 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-05-22T21:15:16.764 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:15:16.758+0000 7f7d35a00640 -1 log_channel(cluster) log [ERR] : Health check failed: Full OSDs blocking recovery: 1 pg recovery_toofull (PG_RECOVERY_FULL) 2024-05-22T21:15:21.709 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:15:21.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.850+0000 7fa163600640 1 Processor -- start 2024-05-22T21:15:21.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.850+0000 7fa163600640 1 -- start start 2024-05-22T21:15:21.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.850+0000 7fa163600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa15c0715a0 0x7fa15c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:21.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.850+0000 7fa163600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa15c15ec90 0x7fa15c15f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:21.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.850+0000 7fa163600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15c1442a0 con 0x7fa15c143ec0 2024-05-22T21:15:21.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.850+0000 7fa163600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa15c15f640 con 0x7fa15c0715a0 2024-05-22T21:15:21.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.850+0000 7fa163600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa15c070500 con 0x7fa15c15ec90 2024-05-22T21:15:21.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.854+0000 7fa15be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa15c0715a0 0x7fa15c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:21.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.854+0000 7fa160c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa15c143ec0 0x7fa15c06fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53674/0 (socket says 172.21.15.161:53674) 2024-05-22T21:15:21.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.854+0000 7fa160c00640 1 -- 172.21.15.161:0/980232716 learned_addr learned my addr 172.21.15.161:0/980232716 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:21.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.854+0000 7fa15be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa15c0715a0 0x7fa15c15f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51028/0 (socket says 172.21.15.161:51028) 2024-05-22T21:15:21.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.854+0000 7fa161600640 1 --2- 172.21.15.161:0/980232716 >> v2:172.21.15.179:3300/0 conn(0x7fa15c15ec90 0x7fa15c15f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:21.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.854+0000 7fa15be00640 1 -- 172.21.15.161:0/980232716 >> v2:172.21.15.179:3300/0 conn(0x7fa15c15ec90 msgr2=0x7fa15c15f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:21.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.854+0000 7fa15be00640 1 --2- 172.21.15.161:0/980232716 >> v2:172.21.15.179:3300/0 conn(0x7fa15c15ec90 0x7fa15c15f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:21.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.854+0000 7fa15be00640 1 -- 172.21.15.161:0/980232716 >> v1:172.21.15.161:6789/0 conn(0x7fa15c143ec0 legacy=0x7fa15c06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:21.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.854+0000 7fa15be00640 1 -- 172.21.15.161:0/980232716 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c0707e0 con 0x7fa15c0715a0 2024-05-22T21:15:21.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.854+0000 7fa15b400640 1 -- 172.21.15.161:0/980232716 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810039521 0 0) 0x7fa15c1442a0 con 0x7fa15c143ec0 2024-05-22T21:15:21.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.854+0000 7fa15be00640 1 --2- 172.21.15.161:0/980232716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa15c0715a0 0x7fa15c15f830 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0cedc0 tx=0x7fa14801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9c9d47f3c41e7f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:21.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.854+0000 7fa15b400640 1 -- 172.21.15.161:0/980232716 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148004230 con 0x7fa15c0715a0 2024-05-22T21:15:21.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.854+0000 7fa15b400640 1 -- 172.21.15.161:0/980232716 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1480a7050 con 0x7fa15c0715a0 2024-05-22T21:15:21.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.854+0000 7fa15b400640 1 -- 172.21.15.161:0/980232716 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148003620 con 0x7fa15c0715a0 2024-05-22T21:15:21.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa163600640 1 -- 172.21.15.161:0/980232716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa15c0715a0 msgr2=0x7fa15c15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:21.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa163600640 1 --2- 172.21.15.161:0/980232716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa15c0715a0 0x7fa15c15f830 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0cedc0 tx=0x7fa14801c450 comp rx=0 tx=0).stop 2024-05-22T21:15:21.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa163600640 1 -- 172.21.15.161:0/980232716 shutdown_connections 2024-05-22T21:15:21.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa163600640 1 --2- 172.21.15.161:0/980232716 >> v2:172.21.15.179:3300/0 conn(0x7fa15c15ec90 0x7fa15c15f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:21.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa163600640 1 --2- 172.21.15.161:0/980232716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa15c0715a0 0x7fa15c15f830 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:21.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa163600640 1 -- 172.21.15.161:0/980232716 >> 172.21.15.161:0/980232716 conn(0x7fa15c0b0bc0 msgr2=0x7fa15c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:21.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa163600640 1 -- 172.21.15.161:0/980232716 shutdown_connections 2024-05-22T21:15:21.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa163600640 1 -- 172.21.15.161:0/980232716 wait complete. 2024-05-22T21:15:21.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa163600640 1 Processor -- start 2024-05-22T21:15:21.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa163600640 1 -- start start 2024-05-22T21:15:21.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa163600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa15c143ec0 0x7fa15c15b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:21.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa163600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa15c15ec90 0x7fa15c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:21.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa163600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa15c15d370 0x7fa15c159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:21.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa163600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c15fbf0 con 0x7fa15c15d370 2024-05-22T21:15:21.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa163600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c15c060 con 0x7fa15c15ec90 2024-05-22T21:15:21.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa163600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c15c340 con 0x7fa15c143ec0 2024-05-22T21:15:21.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa161600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa15c15d370 0x7fa15c159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:21.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa161600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa15c15d370 0x7fa15c159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51034/0 (socket says 172.21.15.161:51034) 2024-05-22T21:15:21.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa161600640 1 -- 172.21.15.161:0/3528829497 learned_addr learned my addr 172.21.15.161:0/3528829497 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:21.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa15be00640 1 --2- 172.21.15.161:0/3528829497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa15c15ec90 0x7fa15c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:21.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa160c00640 1 --2- 172.21.15.161:0/3528829497 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa15c143ec0 0x7fa15c15b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:21.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa161600640 1 -- 172.21.15.161:0/3528829497 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa15c143ec0 msgr2=0x7fa15c15b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:21.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa161600640 1 --2- 172.21.15.161:0/3528829497 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa15c143ec0 0x7fa15c15b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:21.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa161600640 1 -- 172.21.15.161:0/3528829497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa15c15ec90 msgr2=0x7fa15c0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:21.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa161600640 1 --2- 172.21.15.161:0/3528829497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa15c15ec90 0x7fa15c0af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:21.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa161600640 1 -- 172.21.15.161:0/3528829497 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa148009420 con 0x7fa15c15d370 2024-05-22T21:15:21.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.858+0000 7fa15be00640 1 --2- 172.21.15.161:0/3528829497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa15c15ec90 0x7fa15c0af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:21.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.862+0000 7fa161600640 1 --2- 172.21.15.161:0/3528829497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa15c15d370 0x7fa15c159e40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa15400d560 tx=0x7fa15400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:21.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.862+0000 7fa158c00640 1 -- 172.21.15.161:0/3528829497 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1540125f0 con 0x7fa15c15d370 2024-05-22T21:15:21.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.862+0000 7fa158c00640 1 -- 172.21.15.161:0/3528829497 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa154012770 con 0x7fa15c15d370 2024-05-22T21:15:21.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.862+0000 7fa158c00640 1 -- 172.21.15.161:0/3528829497 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154012b20 con 0x7fa15c15d370 2024-05-22T21:15:21.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.862+0000 7fa163600640 1 -- 172.21.15.161:0/3528829497 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15c15c620 con 0x7fa15c15d370 2024-05-22T21:15:21.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.862+0000 7fa163600640 1 -- 172.21.15.161:0/3528829497 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa15c15bc50 con 0x7fa15c15d370 2024-05-22T21:15:21.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.862+0000 7fa163600640 1 -- 172.21.15.161:0/3528829497 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa15c067220 con 0x7fa15c15d370 2024-05-22T21:15:21.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.862+0000 7fa158c00640 1 -- 172.21.15.161:0/3528829497 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa154012c80 con 0x7fa15c15d370 2024-05-22T21:15:21.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.862+0000 7fa158c00640 1 --2- 172.21.15.161:0/3528829497 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa13c066690 0x7fa13c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:21.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.862+0000 7fa158c00640 1 -- 172.21.15.161:0/3528829497 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa154084650 con 0x7fa15c15d370 2024-05-22T21:15:21.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.862+0000 7fa160c00640 1 --2- 172.21.15.161:0/3528829497 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa13c066690 0x7fa13c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:21.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.866+0000 7fa160c00640 1 --2- 172.21.15.161:0/3528829497 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa13c066690 0x7fa13c068b50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa14c007fd0 tx=0x7fa14c007ee0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:21.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:21.866+0000 7fa158c00640 1 -- 172.21.15.161:0/3528829497 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa1540519f0 con 0x7fa15c15d370 2024-05-22T21:15:22.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.022+0000 7fa163600640 1 -- 172.21.15.161:0/3528829497 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fa15c06dcc0 con 0x7fa15c15d370 2024-05-22T21:15:22.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.022+0000 7fa158c00640 1 -- 172.21.15.161:0/3528829497 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v55) v1 ==== 81+0+33897 (secure 0 0 0) 0x7fa154055010 con 0x7fa15c15d370 2024-05-22T21:15:22.028 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:22.029 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:15:22.029 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 55, 2024-05-22T21:15:22.029 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:15:22.029 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:15:22.029 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:15:17.769605+0000", 2024-05-22T21:15:22.029 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:15:22.029 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:15:22.029 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:15:22.029 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:15:22.029 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:15:22.029 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:15:22.030 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:15:22.030 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:15:22.030 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:15:22.030 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:22.030 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:15:22.030 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:15:22.030 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:15:22.030 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:15:22.030 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:15:22.030 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 7, 2024-05-22T21:15:22.030 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:15:22.031 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:15:22.031 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "luminous", 2024-05-22T21:15:22.031 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:15:22.031 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:15:22.031 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:15:22.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.031 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:15:22.031 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:15:22.031 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:15:22.031 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:15:22.031 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:15:22.031 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:22.032 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:22.032 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:22.032 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:22.032 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:22.032 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:22.032 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:22.032 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:22.032 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:22.032 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:22.032 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:15:22.032 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:15:22.032 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:15:22.032 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:15:22.033 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:15:22.033 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:22.033 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:22.033 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:22.033 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:22.033 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:22.033 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:22.033 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:22.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.033 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "55", 2024-05-22T21:15:22.033 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:22.033 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:22.034 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:22.034 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:22.034 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:22.034 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:22.034 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:22.034 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:22.034 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:22.034 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:22.034 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:22.034 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:22.034 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:22.034 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:22.035 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:22.035 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:22.035 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:22.035 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:22.035 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:22.035 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:22.035 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:22.035 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:22.035 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:22.035 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:22.035 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:22.035 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:22.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.036 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:22.036 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:22.036 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:22.036 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:22.036 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:22.036 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:22.036 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:22.036 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:22.036 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:22.036 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:22.036 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:22.037 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:22.037 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:15:22.037 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:15:22.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.037 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:22.037 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:15:22.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.037 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:22.037 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:15:22.037 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:15:22.037 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:15:22.037 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:15:22.037 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:15:22.038 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:22.038 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:22.038 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:22.038 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.038 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.038 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 6, 2024-05-22T21:15:22.038 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:15:22.038 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:14:48.612726+0000", 2024-05-22T21:15:22.038 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:15:22.038 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:15:22.038 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:22.038 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:22.039 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:22.039 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:22.039 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:22.039 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:22.039 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:22.039 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:22.039 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:22.039 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:22.039 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:15:22.039 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:15:22.039 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:15:22.039 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:15:22.039 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:15:22.039 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:22.039 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:22.040 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:22.040 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:22.040 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:22.040 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:22.040 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:22.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.040 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "55", 2024-05-22T21:15:22.040 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:22.040 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:22.040 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:22.040 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:22.040 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:22.040 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:22.040 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:22.040 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:22.041 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:22.041 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:22.041 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:22.041 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:22.041 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:22.041 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:22.041 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:22.041 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:22.041 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:22.041 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:22.041 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:22.041 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:22.041 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:22.041 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:22.042 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:22.042 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:22.042 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:22.042 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:22.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.042 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:22.042 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:22.042 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:22.042 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:22.042 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:22.042 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:22.043 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:22.043 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:22.043 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:22.043 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:22.043 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:22.043 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:22.043 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:15:22.043 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:15:22.043 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:15:22.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.043 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:22.043 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:15:22.043 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:15:22.043 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.044 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:22.044 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:15:22.044 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:15:22.044 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:15:22.044 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:15:22.044 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:15:22.044 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:22.044 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:22.044 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:22.044 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.044 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.045 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 7, 2024-05-22T21:15:22.045 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:15:22.045 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:14:49.053022+0000", 2024-05-22T21:15:22.045 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:15:22.045 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:15:22.045 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:22.045 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:22.045 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:22.045 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:22.045 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:22.045 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:22.045 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:22.045 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:22.046 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:22.046 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:22.046 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:15:22.046 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:15:22.046 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:15:22.046 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:15:22.046 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:15:22.046 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:22.046 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:22.046 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:22.046 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:22.046 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:22.046 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:22.047 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:22.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.047 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "55", 2024-05-22T21:15:22.047 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:22.047 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:22.047 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:22.047 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:22.047 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:22.047 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:22.047 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:22.047 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:22.047 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:22.047 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:22.048 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:22.048 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:22.048 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:22.048 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:22.048 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:22.048 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:22.048 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:22.048 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:22.048 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:22.048 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:22.048 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:22.048 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:22.048 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:22.048 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:22.049 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:22.049 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:22.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.049 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:22.049 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:22.049 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:22.049 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:22.049 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:22.049 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:22.049 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:22.049 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:22.049 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:22.049 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:22.050 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:22.050 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:22.050 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:15:22.050 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:15:22.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.050 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:22.050 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:15:22.050 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:15:22.050 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.050 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:22.050 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.5, 2024-05-22T21:15:22.051 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.5, 2024-05-22T21:15:22.051 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:15:22.051 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.5, 2024-05-22T21:15:22.051 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.5, 2024-05-22T21:15:22.051 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:22.051 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:22.051 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:22.051 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.051 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.051 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:22.051 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:15:22.051 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.051 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:15:22.052 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:22.052 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:22.052 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:22.052 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:22.052 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:22.052 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:22.052 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:22.052 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:22.052 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:22.052 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:22.052 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:22.052 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:22.052 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.052 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.053 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.053 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:15:22.053 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:22.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.053 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.053 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.053 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:15:22.053 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:22.053 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.053 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.053 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.053 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:22.054 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.054 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.054 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:15:22.054 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:22.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.054 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.054 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.054 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:15:22.054 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:22.054 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.055 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.055 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:22.055 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.055 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.055 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:15:22.055 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:22.055 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.055 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.055 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.056 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:15:22.056 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:22.056 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.056 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.056 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:22.056 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.056 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.056 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.056 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:15:22.056 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:22.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.056 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.056 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.056 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:15:22.057 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:22.057 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.057 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.057 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:15:22.057 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:15:22.057 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:15:22.057 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:15:22.057 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:22.057 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:22.057 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:22.057 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.057 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.057 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:15:22.058 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:22.058 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:22.058 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:22.058 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:22.058 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:22.058 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:22.058 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:22.058 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:22.058 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:22.058 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:22.058 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:22.058 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:22.058 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.059 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.059 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:15:22.059 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:22.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.059 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.059 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.059 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:15:22.059 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:22.059 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.059 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.059 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:22.059 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.060 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.060 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:15:22.060 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:22.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.060 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.060 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.060 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:15:22.060 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:22.060 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.060 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.060 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:22.060 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.061 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.061 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.061 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:15:22.061 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:22.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.061 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.061 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.061 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:15:22.061 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:22.061 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.061 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.061 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:22.061 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.062 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.062 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.062 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:15:22.062 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:22.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.062 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.062 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.062 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:15:22.062 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:22.062 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.062 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.063 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:15:22.063 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:15:22.063 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:15:22.063 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:15:22.063 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:22.063 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:22.063 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:22.063 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.063 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.063 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:15:22.063 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:22.063 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:22.064 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:22.064 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:22.064 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:22.064 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:22.064 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:22.064 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:22.064 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:22.064 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:22.064 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:22.064 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:22.064 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.064 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.065 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.065 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:15:22.065 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:22.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.065 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.065 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:15:22.065 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:22.065 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.065 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.065 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:22.065 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.066 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.066 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.066 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:15:22.066 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:22.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.066 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.066 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.066 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:15:22.066 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:22.066 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.066 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.067 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:22.067 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.067 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.067 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:15:22.067 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:22.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.067 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.067 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:15:22.067 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:22.067 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.067 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.068 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:22.068 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.068 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.068 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:15:22.068 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:22.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.068 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.068 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:15:22.068 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:22.068 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.069 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.069 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:15:22.069 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:15:22.069 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:15:22.069 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:15:22.069 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:22.069 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:22.069 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:22.069 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.069 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:15:22.070 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:22.070 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:22.070 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:22.070 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:22.070 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:22.070 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:22.070 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:22.070 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:22.070 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:22.070 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:22.070 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:22.071 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:22.071 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.071 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.071 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:15:22.071 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:22.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.071 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.071 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:15:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:15:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.072 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:15:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:15:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:15:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:15:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:22.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.075 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.075 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.075 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:15:22.075 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:22.075 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.075 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.075 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:15:22.075 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:15:22.075 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:15:22.075 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:15:22.075 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:15:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:22.076 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:22.077 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:22.077 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 51, 2024-05-22T21:15:22.077 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:22.077 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:22.077 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:22.077 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.077 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.077 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:15:22.077 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:22.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.077 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.078 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:15:22.078 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:22.078 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.078 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.078 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:22.078 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.078 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.078 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:15:22.078 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:22.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.079 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.079 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.079 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:15:22.079 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:22.079 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.079 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.079 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:22.079 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.079 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.079 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.079 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:15:22.079 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:22.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.080 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.080 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:15:22.080 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:22.080 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.080 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.080 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:22.080 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.080 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.080 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:15:22.081 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:22.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.081 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.081 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:15:22.081 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:22.081 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.081 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.081 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:15:22.081 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:15:22.081 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:15:22.081 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:15:22.082 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:22.082 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:22.082 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:22.082 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.082 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:15:22.082 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:22.082 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:22.082 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:22.082 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:22.082 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:22.083 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:22.083 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:22.083 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:22.083 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 51, 2024-05-22T21:15:22.083 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:22.083 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:22.083 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:22.083 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.083 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.083 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:15:22.083 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:22.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.084 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.084 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:15:22.084 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:22.084 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.084 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.084 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:22.084 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.084 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.084 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:15:22.084 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:22.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.085 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.085 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:15:22.085 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:22.085 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.085 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.085 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:22.085 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.085 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.085 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:15:22.085 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:22.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.086 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.086 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:15:22.086 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:22.086 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.086 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.086 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:22.086 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.086 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.086 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:15:22.087 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:22.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.087 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.087 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:15:22.087 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:22.087 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.087 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.087 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:15:22.087 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:15:22.087 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:15:22.087 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:15:22.088 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:22.088 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:22.088 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:22.088 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.088 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:15:22.088 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:22.088 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:22.088 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:22.088 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:22.088 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:22.088 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:22.089 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:22.089 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:22.089 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:22.089 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:22.089 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:22.089 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:22.089 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.089 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.089 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:15:22.089 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:22.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.089 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.090 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:15:22.090 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:22.090 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.090 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.090 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:22.090 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.090 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.090 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:15:22.090 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:22.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.091 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.091 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:15:22.091 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:22.091 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.091 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.091 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:22.091 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.091 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.091 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:15:22.091 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:22.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.092 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.092 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:15:22.092 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:22.092 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.092 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.092 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:22.092 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.092 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.092 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:15:22.093 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:22.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.093 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.093 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:15:22.093 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:22.093 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.093 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.093 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:15:22.093 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:15:22.093 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:15:22.093 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:15:22.094 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:22.094 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:22.094 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:22.094 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.094 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:15:22.094 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:22.094 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:22.094 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:22.094 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:22.095 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:22.095 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:22.095 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:22.095 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:22.095 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:22.095 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:22.095 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:22.095 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:22.095 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.095 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.095 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:15:22.095 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:22.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.096 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.096 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:15:22.096 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:22.096 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.096 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.096 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:22.096 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.096 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.096 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:15:22.096 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:22.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.097 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.097 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:15:22.097 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:22.097 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.097 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.097 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:22.097 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.097 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.097 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:15:22.098 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:22.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.098 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.098 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:15:22.098 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:22.098 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.098 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.098 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:22.098 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.099 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.099 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:15:22.099 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:22.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.099 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.099 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:15:22.099 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:22.099 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.099 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.099 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:15:22.099 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:15:22.100 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:15:22.100 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:15:22.100 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:22.100 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:22.100 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:22.100 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.100 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.100 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:22.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:15:22.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:15:22.100 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:22.100 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:22.101 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:22.101 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:22.101 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:22.101 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:15:22.101 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:22.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.101 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:15:22.101 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:22.101 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:22.101 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:22.101 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:22.101 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:22.102 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:15:22.102 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:22.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.102 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:15:22.102 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:22.102 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:22.102 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:22.102 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:22.102 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:22.102 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:15:22.102 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:22.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.103 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:15:22.103 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:22.103 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:22.103 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:22.103 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:22.103 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:22.103 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:15:22.103 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:22.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.103 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:15:22.104 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:22.104 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:22.104 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:22.104 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:22.104 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:22.104 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:15:22.104 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:22.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.104 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:15:22.104 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:22.104 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:22.104 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:22.105 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:22.105 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:22.105 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:15:22.105 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:22.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.105 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:15:22.105 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:22.105 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:22.105 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:22.105 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:22.105 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:22.105 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:15:22.105 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:22.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.106 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:15:22.106 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:22.106 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:22.106 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:22.106 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:22.106 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:22.106 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:15:22.106 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:22.106 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.106 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:22.106 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:15:22.106 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [ 2024-05-22T21:15:22.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.107 INFO:teuthology.orchestra.run.smithi161.stdout: "pgid": "7.6", 2024-05-22T21:15:22.107 INFO:teuthology.orchestra.run.smithi161.stdout: "mappings": [ 2024-05-22T21:15:22.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.107 INFO:teuthology.orchestra.run.smithi161.stdout: "from": 0, 2024-05-22T21:15:22.107 INFO:teuthology.orchestra.run.smithi161.stdout: "to": 4 2024-05-22T21:15:22.107 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.107 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.107 INFO:teuthology.orchestra.run.smithi161.stdout: "pgid": "7.9", 2024-05-22T21:15:22.107 INFO:teuthology.orchestra.run.smithi161.stdout: "mappings": [ 2024-05-22T21:15:22.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.108 INFO:teuthology.orchestra.run.smithi161.stdout: "from": 0, 2024-05-22T21:15:22.108 INFO:teuthology.orchestra.run.smithi161.stdout: "to": 5 2024-05-22T21:15:22.108 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.108 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.108 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.108 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:22.108 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:15:22.108 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:15:22.108 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:15:22.108 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:15:22.108 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:15:22.108 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:15:22.109 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:15:22.109 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:15:22.109 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:15:22.109 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:15:22.109 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:15:22.109 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:15:22.109 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.109 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:15:22.109 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:15:22.109 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:15:22.109 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:15:22.109 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:15:22.110 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:15:22.110 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:15:22.110 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:15:22.110 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:15:22.110 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:15:22.110 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:15:22.110 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.110 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:15:22.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.026+0000 7fa163600640 1 -- 172.21.15.161:0/3528829497 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa13c066690 msgr2=0x7fa13c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:22.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.026+0000 7fa163600640 1 --2- 172.21.15.161:0/3528829497 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa13c066690 0x7fa13c068b50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa14c007fd0 tx=0x7fa14c007ee0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.026+0000 7fa163600640 1 -- 172.21.15.161:0/3528829497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa15c15d370 msgr2=0x7fa15c159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:22.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.026+0000 7fa163600640 1 --2- 172.21.15.161:0/3528829497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa15c15d370 0x7fa15c159e40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa15400d560 tx=0x7fa15400da20 comp rx=0 tx=0).stop 2024-05-22T21:15:22.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.026+0000 7fa163600640 1 -- 172.21.15.161:0/3528829497 shutdown_connections 2024-05-22T21:15:22.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.026+0000 7fa163600640 1 --2- 172.21.15.161:0/3528829497 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa13c066690 0x7fa13c068b50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.026+0000 7fa163600640 1 --2- 172.21.15.161:0/3528829497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa15c15d370 0x7fa15c159e40 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.026+0000 7fa163600640 1 --2- 172.21.15.161:0/3528829497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa15c15ec90 0x7fa15c0af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.026+0000 7fa163600640 1 --2- 172.21.15.161:0/3528829497 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa15c143ec0 0x7fa15c15b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.026+0000 7fa163600640 1 -- 172.21.15.161:0/3528829497 >> 172.21.15.161:0/3528829497 conn(0x7fa15c0b0bc0 msgr2=0x7fa15c0b21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:22.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.026+0000 7fa163600640 1 -- 172.21.15.161:0/3528829497 shutdown_connections 2024-05-22T21:15:22.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.026+0000 7fa163600640 1 -- 172.21.15.161:0/3528829497 wait complete. 2024-05-22T21:15:22.114 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-05-22T21:15:22.114 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:15:22.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.242+0000 7feef3600640 1 Processor -- start 2024-05-22T21:15:22.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.242+0000 7feef3600640 1 -- start start 2024-05-22T21:15:22.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.246+0000 7feef3600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feeec070db0 0x7feeec0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:22.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.246+0000 7feef3600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feeec0b2ea0 0x7feeec0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.246+0000 7feef3600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeec159bd0 con 0x7feeec1597f0 2024-05-22T21:15:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.246+0000 7feef3600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeec0cbdd0 con 0x7feeec0b2ea0 2024-05-22T21:15:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.246+0000 7feef3600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0cc050 con 0x7feeec070db0 2024-05-22T21:15:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.246+0000 7feef1600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feeec0b2ea0 0x7feeec0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.246+0000 7feef1600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feeec0b2ea0 0x7feeec0cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51038/0 (socket says 172.21.15.161:51038) 2024-05-22T21:15:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.246+0000 7feef1600640 1 -- 172.21.15.161:0/3921293437 learned_addr learned my addr 172.21.15.161:0/3921293437 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.246+0000 7feeebe00640 1 --2- 172.21.15.161:0/3921293437 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feeec070db0 0x7feeec0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.246+0000 7feef1600640 1 -- 172.21.15.161:0/3921293437 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feeec070db0 msgr2=0x7feeec0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.246+0000 7feef1600640 1 --2- 172.21.15.161:0/3921293437 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feeec070db0 0x7feeec0b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.246+0000 7feef1600640 1 -- 172.21.15.161:0/3921293437 >> v1:172.21.15.179:6789/0 conn(0x7feeec1597f0 legacy=0x7feeec0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.246+0000 7feeeb400640 1 -- 172.21.15.161:0/3921293437 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2086377986 0 0) 0x7feeec159bd0 con 0x7feeec1597f0 2024-05-22T21:15:22.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.246+0000 7feef1600640 1 -- 172.21.15.161:0/3921293437 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec0b3720 con 0x7feeec0b2ea0 2024-05-22T21:15:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.246+0000 7feef1600640 1 --2- 172.21.15.161:0/3921293437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feeec0b2ea0 0x7feeec0cc550 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7feee4009e50 tx=0x7feee401db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1fe839d5102373 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feeeb400640 1 -- 172.21.15.161:0/3921293437 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee40a6980 con 0x7feeec0b2ea0 2024-05-22T21:15:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feeeb400640 1 -- 172.21.15.161:0/3921293437 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee40a6ae0 con 0x7feeec0b2ea0 2024-05-22T21:15:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feeeb400640 1 -- 172.21.15.161:0/3921293437 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee40a6e70 con 0x7feeec0b2ea0 2024-05-22T21:15:22.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef3600640 1 -- 172.21.15.161:0/3921293437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feeec0b2ea0 msgr2=0x7feeec0cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:22.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef3600640 1 --2- 172.21.15.161:0/3921293437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feeec0b2ea0 0x7feeec0cc550 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7feee4009e50 tx=0x7feee401db40 comp rx=0 tx=0).stop 2024-05-22T21:15:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef3600640 1 -- 172.21.15.161:0/3921293437 shutdown_connections 2024-05-22T21:15:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef3600640 1 --2- 172.21.15.161:0/3921293437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feeec0b2ea0 0x7feeec0cc550 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef3600640 1 --2- 172.21.15.161:0/3921293437 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feeec070db0 0x7feeec0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef3600640 1 -- 172.21.15.161:0/3921293437 >> 172.21.15.161:0/3921293437 conn(0x7feeec0b0bc0 msgr2=0x7feeec0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef3600640 1 -- 172.21.15.161:0/3921293437 shutdown_connections 2024-05-22T21:15:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef3600640 1 -- 172.21.15.161:0/3921293437 wait complete. 2024-05-22T21:15:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef3600640 1 Processor -- start 2024-05-22T21:15:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef3600640 1 -- start start 2024-05-22T21:15:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef3600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feeec070db0 0x7feeec0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef3600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feeec0b2ea0 0x7feeec0aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef3600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feeec1597f0 0x7feeec0af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef3600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec154e10 con 0x7feeec1597f0 2024-05-22T21:15:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef3600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec1550f0 con 0x7feeec0b2ea0 2024-05-22T21:15:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef3600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec1553d0 con 0x7feeec070db0 2024-05-22T21:15:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef1600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feeec1597f0 0x7feeec0af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef1600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feeec1597f0 0x7feeec0af070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51054/0 (socket says 172.21.15.161:51054) 2024-05-22T21:15:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feef1600640 1 -- 172.21.15.161:0/3476428696 learned_addr learned my addr 172.21.15.161:0/3476428696 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.250+0000 7feeebe00640 1 --2- 172.21.15.161:0/3476428696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feeec0b2ea0 0x7feeec0aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.254+0000 7feef1600640 1 -- 172.21.15.161:0/3476428696 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feeec070db0 msgr2=0x7feeec0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.254+0000 7feef0c00640 1 --2- 172.21.15.161:0/3476428696 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feeec070db0 0x7feeec0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.254+0000 7feef1600640 1 --2- 172.21.15.161:0/3476428696 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feeec070db0 0x7feeec0cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.254+0000 7feef1600640 1 -- 172.21.15.161:0/3476428696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feeec0b2ea0 msgr2=0x7feeec0aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.254+0000 7feef1600640 1 --2- 172.21.15.161:0/3476428696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feeec0b2ea0 0x7feeec0aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.254+0000 7feef1600640 1 -- 172.21.15.161:0/3476428696 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee4009b30 con 0x7feeec1597f0 2024-05-22T21:15:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.254+0000 7feeebe00640 1 --2- 172.21.15.161:0/3476428696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feeec0b2ea0 0x7feeec0aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.254+0000 7feef0c00640 1 --2- 172.21.15.161:0/3476428696 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feeec070db0 0x7feeec0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.254+0000 7feef1600640 1 --2- 172.21.15.161:0/3476428696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feeec1597f0 0x7feeec0af070 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7feee40b0040 tx=0x7feee401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.254+0000 7feee8c00640 1 -- 172.21.15.161:0/3476428696 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee40a6980 con 0x7feeec1597f0 2024-05-22T21:15:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.254+0000 7feee8c00640 1 -- 172.21.15.161:0/3476428696 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee4007df0 con 0x7feeec1597f0 2024-05-22T21:15:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.254+0000 7feee8c00640 1 -- 172.21.15.161:0/3476428696 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee40b8660 con 0x7feeec1597f0 2024-05-22T21:15:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.254+0000 7feef3600640 1 -- 172.21.15.161:0/3476428696 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeec1556b0 con 0x7feeec1597f0 2024-05-22T21:15:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.254+0000 7feef3600640 1 -- 172.21.15.161:0/3476428696 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeec0ce160 con 0x7feeec1597f0 2024-05-22T21:15:22.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.254+0000 7feef3600640 1 -- 172.21.15.161:0/3476428696 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeec0ce350 con 0x7feeec1597f0 2024-05-22T21:15:22.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.254+0000 7feee8c00640 1 -- 172.21.15.161:0/3476428696 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feee40b87c0 con 0x7feeec1597f0 2024-05-22T21:15:22.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.258+0000 7feee8c00640 1 --2- 172.21.15.161:0/3476428696 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feecc066690 0x7feecc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:22.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.258+0000 7feee8c00640 1 -- 172.21.15.161:0/3476428696 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feee4120170 con 0x7feeec1597f0 2024-05-22T21:15:22.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.258+0000 7feef0c00640 1 --2- 172.21.15.161:0/3476428696 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feecc066690 0x7feecc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:22.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.258+0000 7feef0c00640 1 --2- 172.21.15.161:0/3476428696 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feecc066690 0x7feecc068b50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feedc000f70 tx=0x7feedc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:22.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.258+0000 7feee8c00640 1 -- 172.21.15.161:0/3476428696 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7feee40ed3f0 con 0x7feeec1597f0 2024-05-22T21:15:22.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.414+0000 7feef3600640 1 -- 172.21.15.161:0/3476428696 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7feeec067220 con 0x7feeec1597f0 2024-05-22T21:15:22.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.414+0000 7feee8c00640 1 -- 172.21.15.161:0/3476428696 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v55) v1 ==== 81+0+33897 (secure 0 0 0) 0x7feee4009510 con 0x7feeec1597f0 2024-05-22T21:15:22.421 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:22.421 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:15:22.421 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 55, 2024-05-22T21:15:22.421 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:15:22.421 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:15:22.421 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:15:17.769605+0000", 2024-05-22T21:15:22.421 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:15:22.421 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:15:22.422 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:15:22.422 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:15:22.422 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:15:22.422 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:15:22.422 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:15:22.422 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:15:22.422 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:15:22.422 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:22.422 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:15:22.422 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:15:22.422 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:15:22.423 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:15:22.423 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:15:22.423 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 7, 2024-05-22T21:15:22.423 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:15:22.423 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:15:22.423 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "luminous", 2024-05-22T21:15:22.423 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:15:22.423 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:15:22.423 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:15:22.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.423 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:15:22.423 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:15:22.423 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:15:22.424 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:15:22.424 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:15:22.424 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:22.424 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:22.424 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:22.424 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:22.424 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:22.424 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:22.424 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:22.424 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:22.424 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:22.424 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:22.424 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:15:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:15:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:15:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:15:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:15:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "55", 2024-05-22T21:15:22.425 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:22.426 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:22.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.428 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:22.428 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:22.428 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:22.428 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:22.428 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:22.428 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:22.428 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:22.428 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:22.428 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:22.428 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:22.428 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:22.428 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:22.428 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:15:22.428 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:15:22.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.429 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:22.429 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:15:22.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.429 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:22.429 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:15:22.429 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:15:22.429 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:15:22.429 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:15:22.429 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:15:22.429 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:22.429 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:22.429 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:22.429 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.430 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.430 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 6, 2024-05-22T21:15:22.430 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:15:22.430 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:14:48.612726+0000", 2024-05-22T21:15:22.430 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:15:22.430 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:15:22.430 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:22.430 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:22.430 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:22.430 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:22.431 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:22.431 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:22.431 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:22.431 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:22.431 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:22.431 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:22.431 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:15:22.431 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:15:22.431 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:15:22.431 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:15:22.431 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:15:22.431 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:22.431 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:22.431 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:22.432 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:22.432 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:22.432 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:22.432 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:22.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.432 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "55", 2024-05-22T21:15:22.432 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:22.432 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:22.432 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:22.432 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:22.432 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:22.432 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:22.432 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:22.432 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:22.433 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:22.433 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:22.433 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:22.433 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:22.433 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:22.433 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:22.433 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:22.433 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:22.433 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:22.433 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:22.433 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:22.433 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:22.433 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:22.434 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:22.434 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:22.434 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:22.434 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:22.434 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:22.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.434 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:22.434 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:22.435 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:22.435 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:22.435 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:22.435 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:22.435 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:22.435 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:22.435 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:22.435 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:22.435 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:22.435 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:22.435 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:15:22.436 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:15:22.436 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:15:22.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.436 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:22.436 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:15:22.436 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:15:22.436 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.436 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:22.436 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:15:22.436 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:15:22.437 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:15:22.437 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:15:22.437 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:15:22.437 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:22.437 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:22.437 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:22.437 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.437 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 7, 2024-05-22T21:15:22.437 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:15:22.437 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:14:49.053022+0000", 2024-05-22T21:15:22.437 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:15:22.437 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:15:22.438 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:15:22.438 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:15:22.438 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:15:22.438 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:15:22.438 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:15:22.438 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:15:22.438 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:15:22.438 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:15:22.438 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:15:22.438 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:15:22.438 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:15:22.438 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:15:22.439 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:15:22.439 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:15:22.439 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:15:22.439 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:15:22.439 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:15:22.439 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:15:22.439 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:15:22.439 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:15:22.439 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:15:22.439 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:15:22.439 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.439 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "55", 2024-05-22T21:15:22.439 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:15:22.439 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:15:22.440 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:15:22.440 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:15:22.440 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:15:22.440 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:15:22.440 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:15:22.440 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:15:22.440 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:15:22.440 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:15:22.440 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:15:22.440 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:15:22.440 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:15:22.440 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:15:22.440 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:15:22.440 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:15:22.441 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:15:22.441 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:15:22.441 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:15:22.441 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:15:22.441 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:15:22.441 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:15:22.441 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:15:22.441 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:15:22.441 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:15:22.441 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:15:22.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.441 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:15:22.441 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:15:22.442 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:15:22.442 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:15:22.442 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:15:22.442 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:15:22.442 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:15:22.442 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:15:22.442 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:15:22.442 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:15:22.442 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:15:22.442 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:15:22.442 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:15:22.443 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:15:22.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.443 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:15:22.443 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:15:22.443 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:15:22.443 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.443 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:15:22.443 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.5, 2024-05-22T21:15:22.443 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.5, 2024-05-22T21:15:22.443 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:15:22.443 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.5, 2024-05-22T21:15:22.443 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.5, 2024-05-22T21:15:22.443 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:15:22.443 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:15:22.444 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:15:22.444 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.444 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.444 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:22.444 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:15:22.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.444 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:15:22.444 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:22.444 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:22.444 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:22.444 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:22.444 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:22.444 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:22.444 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:22.444 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:22.445 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:22.445 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:22.445 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:22.445 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:22.445 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.445 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.445 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:15:22.445 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:22.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.446 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.446 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:15:22.446 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:22.446 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.446 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.446 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:22.446 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.446 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.446 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:15:22.446 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:22.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.447 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.447 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.447 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:15:22.447 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:22.447 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.447 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.447 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:22.447 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.447 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.447 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.447 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:15:22.447 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:22.447 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.448 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.448 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:15:22.448 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:22.448 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.448 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.448 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:22.448 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.448 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.448 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.448 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:15:22.448 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:22.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.449 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.449 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.449 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:15:22.449 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:15:22.449 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.449 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.449 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:15:22.449 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:15:22.449 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:15:22.449 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:15:22.449 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:22.449 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:22.449 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:22.450 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.450 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.450 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.450 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:15:22.450 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:22.450 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:22.450 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:22.450 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:22.450 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:22.450 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:22.450 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:22.450 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:22.451 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:22.451 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:22.451 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:22.451 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:22.451 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.451 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.451 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:15:22.451 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:22.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.451 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.451 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.451 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:15:22.452 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:22.452 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.452 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.452 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:22.452 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.452 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.452 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.452 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:15:22.452 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:22.452 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.452 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.452 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.452 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:15:22.453 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:22.453 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.453 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.453 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:22.453 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.453 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.453 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.453 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:15:22.453 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:22.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.453 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.454 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.454 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:15:22.454 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:22.454 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.454 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.454 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.454 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:22.454 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.454 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.454 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.454 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:15:22.455 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:22.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.455 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.455 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.455 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:15:22.455 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:15:22.455 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.455 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.455 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:15:22.455 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:15:22.455 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:15:22.455 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:15:22.455 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:22.455 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:22.456 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:22.456 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.456 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.456 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.456 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:15:22.456 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:22.456 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:22.456 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:22.456 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:22.456 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:22.456 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:22.456 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:22.456 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:22.456 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:22.457 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:22.457 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:22.457 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:22.457 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.457 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.457 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:15:22.457 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:22.457 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.457 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.457 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.457 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:15:22.457 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:22.457 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.458 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.458 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:22.458 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.458 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.458 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.458 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:15:22.458 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:22.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.458 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.458 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.458 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:15:22.459 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:22.459 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.459 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.459 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:22.459 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.459 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.459 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.459 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:15:22.459 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:22.459 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.459 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.459 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.459 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:15:22.460 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:22.460 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.460 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.460 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.460 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:22.460 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.460 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.460 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.460 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:15:22.460 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:22.460 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.460 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.460 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.460 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:15:22.461 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:15:22.461 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.461 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.461 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:15:22.461 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:15:22.461 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:15:22.461 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:15:22.461 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:22.461 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:22.461 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:22.461 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.461 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.461 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.461 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:15:22.462 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:22.462 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:22.462 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:22.462 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:22.462 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:22.462 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:22.462 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:22.462 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:22.462 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:22.462 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:22.462 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:22.462 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:22.463 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.463 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.463 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:15:22.463 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:22.463 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.463 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.463 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.463 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:15:22.463 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:22.463 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.463 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.464 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:22.464 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.464 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.464 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.464 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:15:22.464 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:22.464 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.464 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.464 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.464 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:15:22.464 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:22.464 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.464 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.465 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:22.465 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.465 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.465 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.465 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:15:22.465 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:22.465 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.465 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.465 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.465 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:15:22.465 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:22.465 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.466 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.466 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:22.466 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.466 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.466 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.466 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:15:22.466 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:22.466 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.466 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.466 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.466 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:15:22.467 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:15:22.467 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.467 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.467 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:15:22.467 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:15:22.467 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:15:22.467 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:15:22.467 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:22.467 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:22.467 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:22.467 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.467 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.468 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.468 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:15:22.468 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:22.468 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:22.468 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:22.468 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:22.468 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:22.468 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:22.468 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:22.468 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:22.468 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 51, 2024-05-22T21:15:22.468 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:22.468 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:22.468 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:22.469 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.469 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.469 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:15:22.469 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:22.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.469 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.469 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.469 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:15:22.469 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:22.469 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.469 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.469 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.470 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:22.470 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.470 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.470 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.470 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:15:22.470 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:22.470 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.471 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.471 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.471 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:15:22.471 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:22.471 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.471 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.471 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.471 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:22.472 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.472 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.472 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:15:22.472 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:22.472 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.472 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.472 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.472 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:15:22.472 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:22.473 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.473 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.473 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:22.473 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.473 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.473 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.473 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:15:22.473 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:22.473 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.473 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.474 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.474 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:15:22.474 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:15:22.474 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.474 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.474 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.474 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:15:22.474 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:15:22.474 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:15:22.474 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:15:22.474 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:22.475 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:22.475 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:22.475 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.475 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.475 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.475 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:15:22.475 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:22.475 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:22.475 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:22.475 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:22.475 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:22.475 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:22.475 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:22.475 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:22.476 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 51, 2024-05-22T21:15:22.476 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:22.476 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:22.476 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:22.476 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.476 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.476 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.476 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:15:22.476 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:22.476 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.476 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.476 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.476 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:15:22.477 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:22.477 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.477 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.477 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:22.477 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.477 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.477 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:15:22.477 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:22.477 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.477 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.477 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.477 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:15:22.477 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:22.478 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.478 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.478 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:22.478 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.478 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.478 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.478 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:15:22.478 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:22.478 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.478 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.478 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.478 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:15:22.479 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:22.479 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.479 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.479 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:22.479 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.479 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.479 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.479 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:15:22.479 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:22.479 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.479 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.479 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.479 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:15:22.480 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:15:22.480 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.480 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.480 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:15:22.480 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:15:22.480 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:15:22.480 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:15:22.480 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:22.480 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:22.480 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:22.480 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.480 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.480 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.480 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:15:22.481 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:22.481 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:22.481 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:22.481 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:22.481 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:22.481 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:22.481 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:22.481 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:22.481 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:22.481 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:22.481 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:22.481 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:22.481 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.481 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.482 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.482 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:15:22.482 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:22.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.482 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.482 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:15:22.482 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:22.482 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.482 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.482 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.482 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:22.482 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.483 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.483 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.483 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:15:22.483 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:22.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.483 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.483 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.483 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:15:22.483 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:22.483 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.483 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.483 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.484 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:22.484 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.484 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.484 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.484 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:15:22.484 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:22.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.484 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.484 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.484 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:15:22.484 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:22.484 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.484 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.485 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.485 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:22.485 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.485 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.485 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:15:22.485 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:22.485 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.485 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.485 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.485 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:15:22.485 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:15:22.485 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.485 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.486 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:15:22.486 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:15:22.486 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:15:22.486 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:15:22.486 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:22.486 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:22.486 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:22.486 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.486 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.486 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.486 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:15:22.486 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:15:22.487 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:15:22.487 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:15:22.487 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:15:22.487 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:15:22.487 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:15:22.487 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:15:22.487 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:15:22.487 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 45, 2024-05-22T21:15:22.487 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:15:22.487 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:15:22.487 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:15:22.487 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.487 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.488 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.488 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:15:22.488 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:22.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.488 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.488 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.488 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:15:22.488 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:22.488 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.488 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.488 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:15:22.488 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.488 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.489 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.489 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:15:22.489 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:22.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.489 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.489 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.489 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:15:22.489 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:22.489 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.489 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.489 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:15:22.489 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.490 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.490 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:15:22.490 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:22.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.490 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.490 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:15:22.490 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:22.490 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.490 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.490 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:15:22.491 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:15:22.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.491 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:15:22.491 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:15:22.491 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:22.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.491 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:15:22.491 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:15:22.491 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:15:22.491 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.491 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.492 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:15:22.492 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:15:22.492 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:15:22.492 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:15:22.492 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:15:22.492 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:15:22.492 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:15:22.492 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.492 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.492 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:22.492 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:15:22.492 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.492 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:15:22.492 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:22.493 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:22.493 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:22.493 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:22.493 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:22.493 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:15:22.493 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:22.493 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.493 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.493 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:15:22.493 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:22.493 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:22.493 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:22.493 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:22.493 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:22.494 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:15:22.494 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:22.494 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.494 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.494 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:15:22.494 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:22.494 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:22.494 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:22.494 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:22.494 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:22.494 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:15:22.494 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:22.494 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.495 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:15:22.495 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:22.495 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:22.495 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:22.495 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:22.495 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:22.495 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:15:22.495 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:22.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.495 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:15:22.495 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:22.495 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:22.496 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:22.496 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:22.496 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:22.496 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:15:22.496 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:22.496 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.496 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.496 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:15:22.496 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:22.496 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:22.496 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:22.496 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:22.496 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:22.496 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:15:22.497 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:22.497 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.497 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.497 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:15:22.497 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:22.497 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:22.497 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:22.497 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:22.497 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:22.497 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:15:22.497 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:22.497 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.497 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.498 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:15:22.498 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:15:22.498 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:15:22.498 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:15:22.498 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:15:22.498 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:15:22.498 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:15:22.498 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:15:22.498 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.498 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:22.498 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:15:22.498 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [ 2024-05-22T21:15:22.499 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.499 INFO:teuthology.orchestra.run.smithi161.stdout: "pgid": "7.6", 2024-05-22T21:15:22.499 INFO:teuthology.orchestra.run.smithi161.stdout: "mappings": [ 2024-05-22T21:15:22.499 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.499 INFO:teuthology.orchestra.run.smithi161.stdout: "from": 0, 2024-05-22T21:15:22.499 INFO:teuthology.orchestra.run.smithi161.stdout: "to": 4 2024-05-22T21:15:22.499 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.499 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.499 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.499 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.499 INFO:teuthology.orchestra.run.smithi161.stdout: "pgid": "7.9", 2024-05-22T21:15:22.499 INFO:teuthology.orchestra.run.smithi161.stdout: "mappings": [ 2024-05-22T21:15:22.499 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:22.499 INFO:teuthology.orchestra.run.smithi161.stdout: "from": 0, 2024-05-22T21:15:22.500 INFO:teuthology.orchestra.run.smithi161.stdout: "to": 5 2024-05-22T21:15:22.500 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.500 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:22.500 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.500 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:15:22.500 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:15:22.500 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:15:22.500 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:15:22.500 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:15:22.500 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:15:22.500 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:15:22.500 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:15:22.501 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:15:22.501 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:15:22.501 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:15:22.501 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:15:22.501 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:15:22.501 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.501 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:22.501 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:15:22.501 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:15:22.502 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:15:22.502 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:15:22.502 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:15:22.502 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:15:22.502 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:15:22.502 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:15:22.502 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:15:22.503 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:15:22.503 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:15:22.503 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:22.503 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:15:22.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.418+0000 7feef3600640 1 -- 172.21.15.161:0/3476428696 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feecc066690 msgr2=0x7feecc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:22.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.418+0000 7feef3600640 1 --2- 172.21.15.161:0/3476428696 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feecc066690 0x7feecc068b50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feedc000f70 tx=0x7feedc009040 comp rx=0 tx=0).stop 2024-05-22T21:15:22.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.418+0000 7feef3600640 1 -- 172.21.15.161:0/3476428696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feeec1597f0 msgr2=0x7feeec0af070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:22.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.418+0000 7feef3600640 1 --2- 172.21.15.161:0/3476428696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feeec1597f0 0x7feeec0af070 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7feee40b0040 tx=0x7feee401c380 comp rx=0 tx=0).stop 2024-05-22T21:15:22.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.418+0000 7feef3600640 1 -- 172.21.15.161:0/3476428696 shutdown_connections 2024-05-22T21:15:22.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.418+0000 7feef3600640 1 --2- 172.21.15.161:0/3476428696 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feecc066690 0x7feecc068b50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.418+0000 7feef3600640 1 --2- 172.21.15.161:0/3476428696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feeec1597f0 0x7feeec0af070 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.418+0000 7feef3600640 1 --2- 172.21.15.161:0/3476428696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feeec0b2ea0 0x7feeec0aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.418+0000 7feef3600640 1 --2- 172.21.15.161:0/3476428696 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feeec070db0 0x7feeec0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.418+0000 7feef3600640 1 -- 172.21.15.161:0/3476428696 >> 172.21.15.161:0/3476428696 conn(0x7feeec0b0bc0 msgr2=0x7feeec06d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:22.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.418+0000 7feef3600640 1 -- 172.21.15.161:0/3476428696 shutdown_connections 2024-05-22T21:15:22.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.418+0000 7feef3600640 1 -- 172.21.15.161:0/3476428696 wait complete. 2024-05-22T21:15:22.507 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:15:22.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.638+0000 7f59c2000640 1 Processor -- start 2024-05-22T21:15:22.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.638+0000 7f59c2000640 1 -- start start 2024-05-22T21:15:22.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.638+0000 7f59c2000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f59bc15dd50 0x7f59bc06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:22.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.638+0000 7f59c2000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f59bc15e130 0x7f59bc15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:22.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.638+0000 7f59c2000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59bc071670 con 0x7f59bc15d410 2024-05-22T21:15:22.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.638+0000 7f59c2000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59bc15d8a0 con 0x7f59bc15e130 2024-05-22T21:15:22.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.638+0000 7f59c2000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59bc06de40 con 0x7f59bc15dd50 2024-05-22T21:15:22.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.638+0000 7f59b2a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f59bc15e130 0x7f59bc15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:22.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.638+0000 7f59b2a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f59bc15e130 0x7f59bc15ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51056/0 (socket says 172.21.15.161:51056) 2024-05-22T21:15:22.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.638+0000 7f59b2a00640 1 -- 172.21.15.161:0/314809068 learned_addr learned my addr 172.21.15.161:0/314809068 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:22.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.638+0000 7f59bb400640 1 --2- 172.21.15.161:0/314809068 >> v2:172.21.15.179:3300/0 conn(0x7f59bc15dd50 0x7f59bc06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:22.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.638+0000 7f59b2a00640 1 -- 172.21.15.161:0/314809068 >> v2:172.21.15.179:3300/0 conn(0x7f59bc15dd50 msgr2=0x7f59bc06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:22.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.638+0000 7f59b2a00640 1 --2- 172.21.15.161:0/314809068 >> v2:172.21.15.179:3300/0 conn(0x7f59bc15dd50 0x7f59bc06d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.638+0000 7f59b2a00640 1 -- 172.21.15.161:0/314809068 >> v1:172.21.15.179:6789/0 conn(0x7f59bc15d410 legacy=0x7f59bc15a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:22.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.638+0000 7f59b2a00640 1 -- 172.21.15.161:0/314809068 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc070a60 con 0x7f59bc15e130 2024-05-22T21:15:22.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.638+0000 7f59bb400640 1 --2- 172.21.15.161:0/314809068 >> v2:172.21.15.179:3300/0 conn(0x7f59bc15dd50 0x7f59bc06d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:22.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.642+0000 7f59b2a00640 1 --2- 172.21.15.161:0/314809068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59bc15e130 0x7f59bc15ced0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f59bc072330 tx=0x7f59a401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3be7cefcf026779 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:22.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.642+0000 7f59baa00640 1 -- 172.21.15.161:0/314809068 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a40aa610 con 0x7f59bc15e130 2024-05-22T21:15:22.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.642+0000 7f59baa00640 1 -- 172.21.15.161:0/314809068 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a40a6040 con 0x7f59bc15e130 2024-05-22T21:15:22.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.642+0000 7f59baa00640 1 -- 172.21.15.161:0/314809068 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a40aa990 con 0x7f59bc15e130 2024-05-22T21:15:22.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.642+0000 7f59c2000640 1 -- 172.21.15.161:0/314809068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59bc15e130 msgr2=0x7f59bc15ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:22.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.642+0000 7f59c2000640 1 --2- 172.21.15.161:0/314809068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59bc15e130 0x7f59bc15ced0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f59bc072330 tx=0x7f59a401c4b0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.642+0000 7f59c2000640 1 -- 172.21.15.161:0/314809068 shutdown_connections 2024-05-22T21:15:22.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.642+0000 7f59c2000640 1 --2- 172.21.15.161:0/314809068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59bc15e130 0x7f59bc15ced0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.642+0000 7f59c2000640 1 --2- 172.21.15.161:0/314809068 >> v2:172.21.15.179:3300/0 conn(0x7f59bc15dd50 0x7f59bc06d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.642+0000 7f59c2000640 1 -- 172.21.15.161:0/314809068 >> 172.21.15.161:0/314809068 conn(0x7f59bc0b0bc0 msgr2=0x7f59bc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:22.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.642+0000 7f59c2000640 1 -- 172.21.15.161:0/314809068 shutdown_connections 2024-05-22T21:15:22.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.642+0000 7f59c2000640 1 -- 172.21.15.161:0/314809068 wait complete. 2024-05-22T21:15:22.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59c2000640 1 Processor -- start 2024-05-22T21:15:22.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59c2000640 1 -- start start 2024-05-22T21:15:22.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59c2000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59bc15d410 0x7f59bc0b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:22.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59c2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59bc15dd50 0x7f59bc0cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:22.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59c2000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59bc15e130 0x7f59bc0ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:22.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59c2000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc1543c0 con 0x7f59bc15dd50 2024-05-22T21:15:22.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59c2000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc0cd5c0 con 0x7f59bc15d410 2024-05-22T21:15:22.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59c2000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc0cd8a0 con 0x7f59bc15e130 2024-05-22T21:15:22.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59b2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59bc15dd50 0x7f59bc0cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:22.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59b2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59bc15dd50 0x7f59bc0cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51066/0 (socket says 172.21.15.161:51066) 2024-05-22T21:15:22.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59b2a00640 1 -- 172.21.15.161:0/3747031438 learned_addr learned my addr 172.21.15.161:0/3747031438 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:22.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59bb400640 1 --2- 172.21.15.161:0/3747031438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59bc15d410 0x7f59bc0b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:22.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59bbe00640 1 --2- 172.21.15.161:0/3747031438 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59bc15e130 0x7f59bc0ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:22.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59b2a00640 1 -- 172.21.15.161:0/3747031438 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59bc15e130 msgr2=0x7f59bc0ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:22.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59b2a00640 1 --2- 172.21.15.161:0/3747031438 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59bc15e130 0x7f59bc0ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59b2a00640 1 -- 172.21.15.161:0/3747031438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59bc15d410 msgr2=0x7f59bc0b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:22.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59b2a00640 1 --2- 172.21.15.161:0/3747031438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59bc15d410 0x7f59bc0b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59b2a00640 1 -- 172.21.15.161:0/3747031438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a40094d0 con 0x7f59bc15dd50 2024-05-22T21:15:22.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59b2a00640 1 --2- 172.21.15.161:0/3747031438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59bc15dd50 0x7f59bc0cca50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f59a401a0e0 tx=0x7f59a401acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:22.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59b3e00640 1 -- 172.21.15.161:0/3747031438 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a40bd5f0 con 0x7f59bc15dd50 2024-05-22T21:15:22.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59b3e00640 1 -- 172.21.15.161:0/3747031438 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a40bd770 con 0x7f59bc15dd50 2024-05-22T21:15:22.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59b3e00640 1 -- 172.21.15.161:0/3747031438 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a40bdb20 con 0x7f59bc15dd50 2024-05-22T21:15:22.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59c2000640 1 -- 172.21.15.161:0/3747031438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc0cdb80 con 0x7f59bc15dd50 2024-05-22T21:15:22.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.646+0000 7f59c2000640 1 -- 172.21.15.161:0/3747031438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc1f3e50 con 0x7f59bc15dd50 2024-05-22T21:15:22.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.650+0000 7f59c2000640 1 -- 172.21.15.161:0/3747031438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59bc0ce030 con 0x7f59bc15dd50 2024-05-22T21:15:22.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.650+0000 7f59b3e00640 1 -- 172.21.15.161:0/3747031438 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59a40b33f0 con 0x7f59bc15dd50 2024-05-22T21:15:22.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.650+0000 7f59b3e00640 1 --2- 172.21.15.161:0/3747031438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5998066690 0x7f5998068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:22.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.650+0000 7f59b3e00640 1 -- 172.21.15.161:0/3747031438 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f59a411edf0 con 0x7f59bc15dd50 2024-05-22T21:15:22.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.650+0000 7f59bb400640 1 --2- 172.21.15.161:0/3747031438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5998066690 0x7f5998068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:22.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.650+0000 7f59bb400640 1 --2- 172.21.15.161:0/3747031438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5998066690 0x7f5998068b50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00ab60 tx=0x7f59ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:22.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.654+0000 7f59b3e00640 1 -- 172.21.15.161:0/3747031438 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f59a40ec130 con 0x7f59bc15dd50 2024-05-22T21:15:22.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.838+0000 7f59c2000640 1 -- 172.21.15.161:0/3747031438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59bc067220 con 0x7f59bc15dd50 2024-05-22T21:15:22.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.842+0000 7f59b3e00640 1 -- 172.21.15.161:0/3747031438 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f59a40c3090 con 0x7f59bc15dd50 2024-05-22T21:15:22.847 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:22.847 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/1767934947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1767934947},{"type":"v1","addr":"172.21.15.161:6836","nonce":1767934947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1873337287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1873337287},{"type":"v1","addr":"172.21.15.179:6834","nonce":1873337287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1815838236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":1815838236},{"type":"v1","addr":"172.21.15.161:6837","nonce":1815838236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:50.052292+0000","modified":"2024-05-22T21:14:53.058881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5374":{"gid":5374,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/389418050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":389418050},{"type":"v1","addr":"172.21.15.179:6835","nonce":389418050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-22T21:15:22.849 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 23 2024-05-22T21:15:22.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.842+0000 7f59c2000640 1 -- 172.21.15.161:0/3747031438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5998066690 msgr2=0x7f5998068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:22.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.842+0000 7f59c2000640 1 --2- 172.21.15.161:0/3747031438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5998066690 0x7f5998068b50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00ab60 tx=0x7f59ac008040 comp rx=0 tx=0).stop 2024-05-22T21:15:22.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.842+0000 7f59c2000640 1 -- 172.21.15.161:0/3747031438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59bc15dd50 msgr2=0x7f59bc0cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:22.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.842+0000 7f59c2000640 1 --2- 172.21.15.161:0/3747031438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59bc15dd50 0x7f59bc0cca50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f59a401a0e0 tx=0x7f59a401acf0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.842+0000 7f59c2000640 1 -- 172.21.15.161:0/3747031438 shutdown_connections 2024-05-22T21:15:22.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.842+0000 7f59c2000640 1 --2- 172.21.15.161:0/3747031438 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5998066690 0x7f5998068b50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.842+0000 7f59c2000640 1 --2- 172.21.15.161:0/3747031438 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59bc15e130 0x7f59bc0ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.842+0000 7f59c2000640 1 --2- 172.21.15.161:0/3747031438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59bc15dd50 0x7f59bc0cca50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.842+0000 7f59c2000640 1 --2- 172.21.15.161:0/3747031438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59bc15d410 0x7f59bc0b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:22.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.842+0000 7f59c2000640 1 -- 172.21.15.161:0/3747031438 >> 172.21.15.161:0/3747031438 conn(0x7f59bc0b0bc0 msgr2=0x7f59bc155910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:22.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.842+0000 7f59c2000640 1 -- 172.21.15.161:0/3747031438 shutdown_connections 2024-05-22T21:15:22.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:22.842+0000 7f59c2000640 1 -- 172.21.15.161:0/3747031438 wait complete. 2024-05-22T21:15:22.868 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-05-22T21:15:23.034 INFO:teuthology.orchestra.run.smithi179.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":5374,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":32.961997760999999,"mdsmap_epoch":23,"osdmap_epoch":55,"osdmap_epoch_barrier":50,"uptime":37.188663423999998} 2024-05-22T21:15:23.034 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 5374, "mdsmap_epoch": 23, "osdmap_epoch": 55, "osdmap_epoch_barrier": 50, "rank_uptime": 32.961997761, "state": "up:active", "uptime": 37.188663424, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:15:23.034 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-22T21:15:23.034 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:15:23.035 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:15:23.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:23.043 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-05-22T21:15:23.223 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:15:23.227 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:15:23.598 INFO:teuthology.orchestra.run.smithi161.stderr:50+0 records in 2024-05-22T21:15:23.598 INFO:teuthology.orchestra.run.smithi161.stderr:50+0 records out 2024-05-22T21:15:23.598 INFO:teuthology.orchestra.run.smithi161.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.499061 s, 105 MB/s 2024-05-22T21:15:23.599 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:15:23.599 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:15:23.668 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:15:23.746 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:15:23.746 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:15:23.746 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:15:23.755 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:15:23.755 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:15:23.805 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:15:23.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.982+0000 7f762fe00640 1 Processor -- start 2024-05-22T21:15:23.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.982+0000 7f762fe00640 1 -- start start 2024-05-22T21:15:23.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f762fe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f762815c6d0 0x7f7628164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:23.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f762fe00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7628071670 con 0x7f762815e130 2024-05-22T21:15:23.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f762fe00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7628070a60 con 0x7f762815dd50 2024-05-22T21:15:23.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f762fe00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7628070d40 con 0x7f762815c6d0 2024-05-22T21:15:23.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f762ca00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f762815e130 0x7f762815ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53678/0 (socket says 172.21.15.161:53678) 2024-05-22T21:15:23.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f762ca00640 1 -- 172.21.15.161:0/512916841 learned_addr learned my addr 172.21.15.161:0/512916841 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:23.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f762de00640 1 --2- 172.21.15.161:0/512916841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f762815c6d0 0x7f7628164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:23.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f7627e00640 1 -- 172.21.15.161:0/512916841 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824694172 0 0) 0x7f7628071670 con 0x7f762815e130 2024-05-22T21:15:23.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f7627e00640 1 -- 172.21.15.161:0/512916841 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7610003530 con 0x7f762815e130 2024-05-22T21:15:23.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f7627e00640 1 -- 172.21.15.161:0/512916841 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 924081583 0 0) 0x7f7610003530 con 0x7f762815e130 2024-05-22T21:15:23.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f7627e00640 1 -- 172.21.15.161:0/512916841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f762815c6d0 msgr2=0x7f7628164e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:23.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f7627e00640 1 --2- 172.21.15.161:0/512916841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f762815c6d0 0x7f7628164e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:23.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f7627e00640 1 -- 172.21.15.161:0/512916841 >> v1:172.21.15.179:6789/0 conn(0x7f762815dd50 legacy=0x7f762806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:23.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f7627e00640 1 -- 172.21.15.161:0/512916841 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7628071020 con 0x7f762815e130 2024-05-22T21:15:23.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f7627e00640 1 -- 172.21.15.161:0/512916841 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f7614001680 con 0x7f762815e130 2024-05-22T21:15:23.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f7627e00640 1 -- 172.21.15.161:0/512916841 >> v1:172.21.15.161:6789/0 conn(0x7f762815e130 legacy=0x7f762815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:23.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f7627e00640 1 --2- 172.21.15.161:0/512916841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f762815c6d0 0x7f76100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:23.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f7627e00640 1 --2- 172.21.15.161:0/512916841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7610003ef0 0x7f76100047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:23.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f7627e00640 1 --2- 172.21.15.161:0/512916841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7610004d10 0x7f7610005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:23.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f7627e00640 1 -- 172.21.15.161:0/512916841 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7628070d40 con 0x7f7610004d10 2024-05-22T21:15:23.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f7627e00640 1 -- 172.21.15.161:0/512916841 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761000b9f0 con 0x7f7610003ef0 2024-05-22T21:15:23.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f7627e00640 1 -- 172.21.15.161:0/512916841 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761000ddc0 con 0x7f762815c6d0 2024-05-22T21:15:23.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f762de00640 1 --2- 172.21.15.161:0/512916841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7610004d10 0x7f7610005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:23.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f762d400640 1 --2- 172.21.15.161:0/512916841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f762815c6d0 0x7f76100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:23.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f762ca00640 1 --2- 172.21.15.161:0/512916841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7610003ef0 0x7f76100047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:23.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f762de00640 1 -- 172.21.15.161:0/512916841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f762815c6d0 msgr2=0x7f76100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:23.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f762de00640 1 --2- 172.21.15.161:0/512916841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f762815c6d0 0x7f76100039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:23.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f762de00640 1 -- 172.21.15.161:0/512916841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7610003ef0 msgr2=0x7f76100047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:23.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f762de00640 1 --2- 172.21.15.161:0/512916841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7610003ef0 0x7f76100047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:23.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f762de00640 1 -- 172.21.15.161:0/512916841 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7610003530 con 0x7f7610004d10 2024-05-22T21:15:23.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.986+0000 7f762ca00640 1 --2- 172.21.15.161:0/512916841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7610003ef0 0x7f76100047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:23.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.990+0000 7f762de00640 1 --2- 172.21.15.161:0/512916841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7610004d10 0x7f7610005190 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7620009fc0 tx=0x7f762001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b6eb1b99305274 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:23.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.990+0000 7f7627e00640 1 -- 172.21.15.161:0/512916841 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76200a5c90 con 0x7f7610004d10 2024-05-22T21:15:23.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.990+0000 7f7627e00640 1 -- 172.21.15.161:0/512916841 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762000f040 con 0x7f7610004d10 2024-05-22T21:15:23.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.990+0000 7f7627e00640 1 -- 172.21.15.161:0/512916841 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76200ad440 con 0x7f7610004d10 2024-05-22T21:15:23.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.990+0000 7f762fe00640 1 -- 172.21.15.161:0/512916841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7610004d10 msgr2=0x7f7610005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:23.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.990+0000 7f762fe00640 1 --2- 172.21.15.161:0/512916841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7610004d10 0x7f7610005190 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7620009fc0 tx=0x7f762001cc80 comp rx=0 tx=0).stop 2024-05-22T21:15:23.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.990+0000 7f762de00640 1 -- 172.21.15.161:0/512916841 reap_dead start 2024-05-22T21:15:23.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.990+0000 7f762fe00640 1 -- 172.21.15.161:0/512916841 shutdown_connections 2024-05-22T21:15:23.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.990+0000 7f762fe00640 1 -- 172.21.15.161:0/512916841 >> 172.21.15.161:0/512916841 conn(0x7f76280b0bc0 msgr2=0x7f76280b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:23.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.990+0000 7f762fe00640 1 -- 172.21.15.161:0/512916841 shutdown_connections 2024-05-22T21:15:23.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.990+0000 7f762fe00640 1 -- 172.21.15.161:0/512916841 wait complete. 2024-05-22T21:15:23.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.990+0000 7f762fe00640 1 Processor -- start 2024-05-22T21:15:23.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.990+0000 7f762fe00640 1 -- start start 2024-05-22T21:15:23.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762fe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7628148830 0x7f76281490f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:23.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7628148c10 0x7f76280b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:23.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762fe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76281539d0 0x7f7628153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:23.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762fe00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7628156040 con 0x7f7628148c10 2024-05-22T21:15:23.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762fe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76281543a0 con 0x7f76281539d0 2024-05-22T21:15:23.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762fe00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7628149630 con 0x7f7628148830 2024-05-22T21:15:23.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762ca00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7628148c10 0x7f76280b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:23.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762ca00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7628148c10 0x7f76280b38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51084/0 (socket says 172.21.15.161:51084) 2024-05-22T21:15:23.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762ca00640 1 -- 172.21.15.161:0/793348526 learned_addr learned my addr 172.21.15.161:0/793348526 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:24.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762de00640 1 --2- 172.21.15.161:0/793348526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76281539d0 0x7f7628153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762d400640 1 --2- 172.21.15.161:0/793348526 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7628148830 0x7f76281490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762ca00640 1 -- 172.21.15.161:0/793348526 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7628148830 msgr2=0x7f76281490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762ca00640 1 --2- 172.21.15.161:0/793348526 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7628148830 0x7f76281490f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762ca00640 1 -- 172.21.15.161:0/793348526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76281539d0 msgr2=0x7f7628153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762ca00640 1 --2- 172.21.15.161:0/793348526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76281539d0 0x7f7628153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762ca00640 1 -- 172.21.15.161:0/793348526 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76281553c0 con 0x7f7628148c10 2024-05-22T21:15:24.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762de00640 1 --2- 172.21.15.161:0/793348526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76281539d0 0x7f7628153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:24.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762ca00640 1 --2- 172.21.15.161:0/793348526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7628148c10 0x7f76280b38f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7614011820 tx=0x7f7614011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:24.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f7625600640 1 -- 172.21.15.161:0/793348526 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761400fb60 con 0x7f7628148c10 2024-05-22T21:15:24.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f7625600640 1 -- 172.21.15.161:0/793348526 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761400fcc0 con 0x7f7628148c10 2024-05-22T21:15:24.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f7625600640 1 -- 172.21.15.161:0/793348526 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614015610 con 0x7f7628148c10 2024-05-22T21:15:24.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762fe00640 1 -- 172.21.15.161:0/793348526 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76281556a0 con 0x7f7628148c10 2024-05-22T21:15:24.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762fe00640 1 -- 172.21.15.161:0/793348526 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7628155c60 con 0x7f7628148c10 2024-05-22T21:15:24.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f762fe00640 1 -- 172.21.15.161:0/793348526 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76280706b0 con 0x7f7628148c10 2024-05-22T21:15:24.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.994+0000 7f7625600640 1 -- 172.21.15.161:0/793348526 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7614015770 con 0x7f7628148c10 2024-05-22T21:15:24.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.998+0000 7f7625600640 1 --2- 172.21.15.161:0/793348526 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7608066690 0x7f7608068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.998+0000 7f7625600640 1 -- 172.21.15.161:0/793348526 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7614087610 con 0x7f7628148c10 2024-05-22T21:15:24.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.998+0000 7f762d400640 1 --2- 172.21.15.161:0/793348526 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7608066690 0x7f7608068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:23.998+0000 7f762d400640 1 --2- 172.21.15.161:0/793348526 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7608066690 0x7f7608068b50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f761c007fd0 tx=0x7f761c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:24.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.002+0000 7f7625600640 1 -- 172.21.15.161:0/793348526 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7614054810 con 0x7f7628148c10 2024-05-22T21:15:24.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.154+0000 7f762fe00640 1 -- 172.21.15.161:0/793348526 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7628067220 con 0x7f7628148c10 2024-05-22T21:15:24.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.154+0000 7f7625600640 1 -- 172.21.15.161:0/793348526 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16183 (secure 0 0 0) 0x7f7614057e30 con 0x7f7628148c10 2024-05-22T21:15:24.161 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:24.162 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":55,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:15:17.769605+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:48.612726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:49.053022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.9","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:15:24.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.158+0000 7f762fe00640 1 -- 172.21.15.161:0/793348526 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7608066690 msgr2=0x7f7608068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.158+0000 7f762fe00640 1 --2- 172.21.15.161:0/793348526 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7608066690 0x7f7608068b50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f761c007fd0 tx=0x7f761c009040 comp rx=0 tx=0).stop 2024-05-22T21:15:24.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.158+0000 7f762fe00640 1 -- 172.21.15.161:0/793348526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7628148c10 msgr2=0x7f76280b38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.158+0000 7f762fe00640 1 --2- 172.21.15.161:0/793348526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7628148c10 0x7f76280b38f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7614011820 tx=0x7f7614011ce0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.158+0000 7f762fe00640 1 -- 172.21.15.161:0/793348526 shutdown_connections 2024-05-22T21:15:24.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.158+0000 7f762fe00640 1 --2- 172.21.15.161:0/793348526 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7608066690 0x7f7608068b50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.158+0000 7f762fe00640 1 --2- 172.21.15.161:0/793348526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76281539d0 0x7f7628153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.158+0000 7f762fe00640 1 --2- 172.21.15.161:0/793348526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7628148c10 0x7f76280b38f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.158+0000 7f762fe00640 1 --2- 172.21.15.161:0/793348526 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7628148830 0x7f76281490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.158+0000 7f762fe00640 1 -- 172.21.15.161:0/793348526 >> 172.21.15.161:0/793348526 conn(0x7f76280b0bc0 msgr2=0x7f76280af630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:24.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.158+0000 7f762fe00640 1 -- 172.21.15.161:0/793348526 shutdown_connections 2024-05-22T21:15:24.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.158+0000 7f762fe00640 1 -- 172.21.15.161:0/793348526 wait complete. 2024-05-22T21:15:24.183 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:14:48.612726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:14:49.053022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:15:24.183 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:15:24.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e7a400640 1 Processor -- start 2024-05-22T21:15:24.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e7a400640 1 -- start start 2024-05-22T21:15:24.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e7a400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7e74143c40 0x7f7e7415b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e7a400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7e7406c9b0 0x7f7e740b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e7a400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7415efb0 con 0x7f7e741444a0 2024-05-22T21:15:24.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e7a400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7415f290 con 0x7f7e7406c9b0 2024-05-22T21:15:24.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e7a400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7415f570 con 0x7f7e74143c40 2024-05-22T21:15:24.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e73e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7e7406c9b0 0x7f7e740b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e73e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7e7406c9b0 0x7f7e740b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51088/0 (socket says 172.21.15.161:51088) 2024-05-22T21:15:24.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e73e00640 1 -- 172.21.15.161:0/2465381358 learned_addr learned my addr 172.21.15.161:0/2465381358 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:24.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e73400640 1 --2- 172.21.15.161:0/2465381358 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7e74143c40 0x7f7e7415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e73e00640 1 -- 172.21.15.161:0/2465381358 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7e74143c40 msgr2=0x7f7e7415b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:24.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e73e00640 1 --2- 172.21.15.161:0/2465381358 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7e74143c40 0x7f7e7415b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e72000640 1 -- 172.21.15.161:0/2465381358 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731079898 0 0) 0x7f7e7415efb0 con 0x7f7e741444a0 2024-05-22T21:15:24.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e73e00640 1 -- 172.21.15.161:0/2465381358 >> v1:172.21.15.179:6789/0 conn(0x7f7e741444a0 legacy=0x7f7e7406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e73e00640 1 -- 172.21.15.161:0/2465381358 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7415f850 con 0x7f7e7406c9b0 2024-05-22T21:15:24.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.318+0000 7f7e73400640 1 --2- 172.21.15.161:0/2465381358 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7e74143c40 0x7f7e7415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:15:24.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.322+0000 7f7e73e00640 1 --2- 172.21.15.161:0/2465381358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e7406c9b0 0x7f7e740b2160 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009870 tx=0x7f7e6c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a560a912c4780dac server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:24.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.322+0000 7f7e72000640 1 -- 172.21.15.161:0/2465381358 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c0aa620 con 0x7f7e7406c9b0 2024-05-22T21:15:24.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.322+0000 7f7e72000640 1 -- 172.21.15.161:0/2465381358 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c0aa780 con 0x7f7e7406c9b0 2024-05-22T21:15:24.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.322+0000 7f7e72000640 1 -- 172.21.15.161:0/2465381358 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c0aab10 con 0x7f7e7406c9b0 2024-05-22T21:15:24.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.322+0000 7f7e7a400640 1 -- 172.21.15.161:0/2465381358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e7406c9b0 msgr2=0x7f7e740b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.322+0000 7f7e7a400640 1 --2- 172.21.15.161:0/2465381358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e7406c9b0 0x7f7e740b2160 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009870 tx=0x7f7e6c01c560 comp rx=0 tx=0).stop 2024-05-22T21:15:24.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.322+0000 7f7e7a400640 1 -- 172.21.15.161:0/2465381358 shutdown_connections 2024-05-22T21:15:24.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.322+0000 7f7e7a400640 1 --2- 172.21.15.161:0/2465381358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e7406c9b0 0x7f7e740b2160 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.322+0000 7f7e7a400640 1 --2- 172.21.15.161:0/2465381358 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7e74143c40 0x7f7e7415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.322+0000 7f7e7a400640 1 -- 172.21.15.161:0/2465381358 >> 172.21.15.161:0/2465381358 conn(0x7f7e7406a2d0 msgr2=0x7f7e7406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:24.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.322+0000 7f7e7a400640 1 -- 172.21.15.161:0/2465381358 shutdown_connections 2024-05-22T21:15:24.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.322+0000 7f7e7a400640 1 -- 172.21.15.161:0/2465381358 wait complete. 2024-05-22T21:15:24.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.322+0000 7f7e7a400640 1 Processor -- start 2024-05-22T21:15:24.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.322+0000 7f7e7a400640 1 -- start start 2024-05-22T21:15:24.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e7a400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7e7406c9b0 0x7f7e74070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e7a400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74143c40 0x7f7e7415a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e7a400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e741444a0 0x7f7e7415a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e7a400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740adbe0 con 0x7f7e741444a0 2024-05-22T21:15:24.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e7a400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7415ae10 con 0x7f7e74143c40 2024-05-22T21:15:24.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e7a400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7415b0f0 con 0x7f7e7406c9b0 2024-05-22T21:15:24.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e73e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e741444a0 0x7f7e7415a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e73e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e741444a0 0x7f7e7415a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51100/0 (socket says 172.21.15.161:51100) 2024-05-22T21:15:24.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e73400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7e7406c9b0 0x7f7e74070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e73e00640 1 -- 172.21.15.161:0/2315851938 learned_addr learned my addr 172.21.15.161:0/2315851938 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:24.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e72a00640 1 --2- 172.21.15.161:0/2315851938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74143c40 0x7f7e7415a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e73e00640 1 -- 172.21.15.161:0/2315851938 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7e7406c9b0 msgr2=0x7f7e74070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e73e00640 1 --2- 172.21.15.161:0/2315851938 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7e7406c9b0 0x7f7e74070ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e73e00640 1 -- 172.21.15.161:0/2315851938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74143c40 msgr2=0x7f7e7415a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e73e00640 1 --2- 172.21.15.161:0/2315851938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74143c40 0x7f7e7415a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e73e00640 1 -- 172.21.15.161:0/2315851938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e64009a70 con 0x7f7e741444a0 2024-05-22T21:15:24.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e73e00640 1 --2- 172.21.15.161:0/2315851938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e741444a0 0x7f7e7415a770 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0ac040 tx=0x7f7e6c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:24.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e6b400640 1 -- 172.21.15.161:0/2315851938 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c0aae00 con 0x7f7e741444a0 2024-05-22T21:15:24.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e6b400640 1 -- 172.21.15.161:0/2315851938 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c0bd3f0 con 0x7f7e741444a0 2024-05-22T21:15:24.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e7a400640 1 -- 172.21.15.161:0/2315851938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6c0094d0 con 0x7f7e741444a0 2024-05-22T21:15:24.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e6b400640 1 -- 172.21.15.161:0/2315851938 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c0bd7a0 con 0x7f7e741444a0 2024-05-22T21:15:24.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e7a400640 1 -- 172.21.15.161:0/2315851938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7415b6b0 con 0x7f7e741444a0 2024-05-22T21:15:24.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.326+0000 7f7e7a400640 1 -- 172.21.15.161:0/2315851938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e740ae980 con 0x7f7e741444a0 2024-05-22T21:15:24.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.330+0000 7f7e6b400640 1 -- 172.21.15.161:0/2315851938 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7e6c0b33f0 con 0x7f7e741444a0 2024-05-22T21:15:24.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.330+0000 7f7e6b400640 1 --2- 172.21.15.161:0/2315851938 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7e54066690 0x7f7e54068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.330+0000 7f7e6b400640 1 -- 172.21.15.161:0/2315851938 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7e6c11e660 con 0x7f7e741444a0 2024-05-22T21:15:24.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.330+0000 7f7e73400640 1 --2- 172.21.15.161:0/2315851938 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7e54066690 0x7f7e54068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.330+0000 7f7e73400640 1 --2- 172.21.15.161:0/2315851938 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7e54066690 0x7f7e54068b50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009fd0 tx=0x7f7e6400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:24.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.334+0000 7f7e6b400640 1 -- 172.21.15.161:0/2315851938 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7e6c0ebbf0 con 0x7f7e741444a0 2024-05-22T21:15:24.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.490+0000 7f7e7a400640 1 -- 172.21.15.161:0/2315851938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e7415b230 con 0x7f7e741444a0 2024-05-22T21:15:24.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.490+0000 7f7e6b400640 1 -- 172.21.15.161:0/2315851938 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e7415b230 con 0x7f7e741444a0 2024-05-22T21:15:24.494 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:15:24.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.490+0000 7f7e7a400640 1 -- 172.21.15.161:0/2315851938 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7e54066690 msgr2=0x7f7e54068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.490+0000 7f7e7a400640 1 --2- 172.21.15.161:0/2315851938 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7e54066690 0x7f7e54068b50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009fd0 tx=0x7f7e6400b040 comp rx=0 tx=0).stop 2024-05-22T21:15:24.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.490+0000 7f7e7a400640 1 -- 172.21.15.161:0/2315851938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e741444a0 msgr2=0x7f7e7415a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.490+0000 7f7e7a400640 1 --2- 172.21.15.161:0/2315851938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e741444a0 0x7f7e7415a770 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0ac040 tx=0x7f7e6c01ada0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.490+0000 7f7e7a400640 1 -- 172.21.15.161:0/2315851938 shutdown_connections 2024-05-22T21:15:24.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.490+0000 7f7e7a400640 1 --2- 172.21.15.161:0/2315851938 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7e54066690 0x7f7e54068b50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.490+0000 7f7e7a400640 1 --2- 172.21.15.161:0/2315851938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e741444a0 0x7f7e7415a770 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.490+0000 7f7e7a400640 1 --2- 172.21.15.161:0/2315851938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e74143c40 0x7f7e7415a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.490+0000 7f7e7a400640 1 --2- 172.21.15.161:0/2315851938 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7e7406c9b0 0x7f7e74070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.490+0000 7f7e7a400640 1 -- 172.21.15.161:0/2315851938 >> 172.21.15.161:0/2315851938 conn(0x7f7e7406a2d0 msgr2=0x7f7e7406c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:24.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.490+0000 7f7e7a400640 1 -- 172.21.15.161:0/2315851938 shutdown_connections 2024-05-22T21:15:24.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.490+0000 7f7e7a400640 1 -- 172.21.15.161:0/2315851938 wait complete. 2024-05-22T21:15:24.514 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:15:24.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.646+0000 7fd10b200640 1 Processor -- start 2024-05-22T21:15:24.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.646+0000 7fd10b200640 1 -- start start 2024-05-22T21:15:24.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.650+0000 7fd10b200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd10c15d180 0x7fd10c06f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.650+0000 7fd10b200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd10c0af070 0x7fd10c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.650+0000 7fd10b200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10c06a820 con 0x7fd10c06a410 2024-05-22T21:15:24.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.650+0000 7fd10b200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd10c15d560 con 0x7fd10c15d180 2024-05-22T21:15:24.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.650+0000 7fd10b200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c0adb00 con 0x7fd10c0af070 2024-05-22T21:15:24.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.650+0000 7fd101400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd10c0af070 0x7fd10c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.650+0000 7fd10a800640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd10c06a410 0x7fd10c070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:43650/0 (socket says 172.21.15.161:43650) 2024-05-22T21:15:24.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.650+0000 7fd101400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd10c0af070 0x7fd10c069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:55528/0 (socket says 172.21.15.161:55528) 2024-05-22T21:15:24.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.650+0000 7fd10a800640 1 -- 172.21.15.161:0/2594337520 learned_addr learned my addr 172.21.15.161:0/2594337520 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:24.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.650+0000 7fd109e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd10c15d180 0x7fd10c06f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.650+0000 7fd101400640 1 -- 172.21.15.161:0/2594337520 >> v2:172.21.15.179:3300/0 conn(0x7fd10c15d180 msgr2=0x7fd10c06f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:24.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.650+0000 7fd101400640 1 --2- 172.21.15.161:0/2594337520 >> v2:172.21.15.179:3300/0 conn(0x7fd10c15d180 0x7fd10c06f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.650+0000 7fd101400640 1 -- 172.21.15.161:0/2594337520 >> v1:172.21.15.179:6789/0 conn(0x7fd10c06a410 legacy=0x7fd10c070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.650+0000 7fd101400640 1 -- 172.21.15.161:0/2594337520 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c15d800 con 0x7fd10c0af070 2024-05-22T21:15:24.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.650+0000 7fd109400640 1 -- 172.21.15.161:0/2594337520 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419324725 0 0) 0x7fd10c06a820 con 0x7fd10c06a410 2024-05-22T21:15:24.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.654+0000 7fd101400640 1 --2- 172.21.15.161:0/2594337520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd10c0af070 0x7fd10c069ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0097c0 tx=0x7fd0fc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d11f76098ca91b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:24.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.654+0000 7fd109400640 1 -- 172.21.15.161:0/2594337520 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0ab600 con 0x7fd10c0af070 2024-05-22T21:15:24.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.654+0000 7fd109400640 1 -- 172.21.15.161:0/2594337520 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0fc0a7050 con 0x7fd10c0af070 2024-05-22T21:15:24.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.654+0000 7fd109400640 1 -- 172.21.15.161:0/2594337520 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0ab980 con 0x7fd10c0af070 2024-05-22T21:15:24.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.654+0000 7fd10b200640 1 -- 172.21.15.161:0/2594337520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd10c0af070 msgr2=0x7fd10c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.654+0000 7fd10b200640 1 --2- 172.21.15.161:0/2594337520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd10c0af070 0x7fd10c069ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0097c0 tx=0x7fd0fc01c4a0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.654+0000 7fd10b200640 1 -- 172.21.15.161:0/2594337520 shutdown_connections 2024-05-22T21:15:24.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.654+0000 7fd10b200640 1 --2- 172.21.15.161:0/2594337520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd10c0af070 0x7fd10c069ed0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.654+0000 7fd10b200640 1 --2- 172.21.15.161:0/2594337520 >> v2:172.21.15.179:3300/0 conn(0x7fd10c15d180 0x7fd10c06f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.654+0000 7fd10b200640 1 -- 172.21.15.161:0/2594337520 >> 172.21.15.161:0/2594337520 conn(0x7fd10c06b9d0 msgr2=0x7fd10c0b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:24.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.654+0000 7fd10b200640 1 -- 172.21.15.161:0/2594337520 shutdown_connections 2024-05-22T21:15:24.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.654+0000 7fd10b200640 1 -- 172.21.15.161:0/2594337520 wait complete. 2024-05-22T21:15:24.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.654+0000 7fd10b200640 1 Processor -- start 2024-05-22T21:15:24.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.654+0000 7fd10b200640 1 -- start start 2024-05-22T21:15:24.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd10b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd10c06a410 0x7fd10c14f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd10b200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd10c0af070 0x7fd10c155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd10b200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd10c15d180 0x7fd10c15f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd10b200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c15bc60 con 0x7fd10c06a410 2024-05-22T21:15:24.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd10b200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c15bf40 con 0x7fd10c15d180 2024-05-22T21:15:24.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd10b200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c15c220 con 0x7fd10c0af070 2024-05-22T21:15:24.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd109e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd10c06a410 0x7fd10c14f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd109e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd10c06a410 0x7fd10c14f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51108/0 (socket says 172.21.15.161:51108) 2024-05-22T21:15:24.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd109e00640 1 -- 172.21.15.161:0/3516334581 learned_addr learned my addr 172.21.15.161:0/3516334581 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:24.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd101400640 1 --2- 172.21.15.161:0/3516334581 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd10c0af070 0x7fd10c155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd10a800640 1 --2- 172.21.15.161:0/3516334581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd10c15d180 0x7fd10c15f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd109e00640 1 -- 172.21.15.161:0/3516334581 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd10c0af070 msgr2=0x7fd10c155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd109e00640 1 --2- 172.21.15.161:0/3516334581 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd10c0af070 0x7fd10c155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd109e00640 1 -- 172.21.15.161:0/3516334581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd10c15d180 msgr2=0x7fd10c15f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd109e00640 1 --2- 172.21.15.161:0/3516334581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd10c15d180 0x7fd10c15f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd109e00640 1 -- 172.21.15.161:0/3516334581 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0fc009420 con 0x7fd10c06a410 2024-05-22T21:15:24.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd10a800640 1 --2- 172.21.15.161:0/3516334581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd10c15d180 0x7fd10c15f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:24.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd109e00640 1 --2- 172.21.15.161:0/3516334581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd10c06a410 0x7fd10c14f840 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400e870 tx=0x7fd0f400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:24.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd102a00640 1 -- 172.21.15.161:0/3516334581 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4009510 con 0x7fd10c06a410 2024-05-22T21:15:24.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd102a00640 1 -- 172.21.15.161:0/3516334581 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f400eec0 con 0x7fd10c06a410 2024-05-22T21:15:24.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd10b200640 1 -- 172.21.15.161:0/3516334581 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10c15c500 con 0x7fd10c06a410 2024-05-22T21:15:24.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd102a00640 1 -- 172.21.15.161:0/3516334581 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400cb40 con 0x7fd10c06a410 2024-05-22T21:15:24.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd10b200640 1 -- 172.21.15.161:0/3516334581 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10c14fd80 con 0x7fd10c06a410 2024-05-22T21:15:24.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd10b200640 1 -- 172.21.15.161:0/3516334581 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd10c151bc0 con 0x7fd10c06a410 2024-05-22T21:15:24.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.658+0000 7fd102a00640 1 -- 172.21.15.161:0/3516334581 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd0f400cca0 con 0x7fd10c06a410 2024-05-22T21:15:24.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.662+0000 7fd102a00640 1 --2- 172.21.15.161:0/3516334581 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd0e80666c0 0x7fd0e8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.662+0000 7fd102a00640 1 -- 172.21.15.161:0/3516334581 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd0f4084210 con 0x7fd10c06a410 2024-05-22T21:15:24.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.662+0000 7fd101400640 1 --2- 172.21.15.161:0/3516334581 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd0e80666c0 0x7fd0e8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.662+0000 7fd101400640 1 --2- 172.21.15.161:0/3516334581 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd0e80666c0 0x7fd0e8068b80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0ad000 tx=0x7fd0fc0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:24.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.666+0000 7fd102a00640 1 -- 172.21.15.161:0/3516334581 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd0f40515e0 con 0x7fd10c06a410 2024-05-22T21:15:24.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.818+0000 7fd10b200640 1 -- 172.21.15.161:0/3516334581 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd10c067070 con 0x7fd10c06a410 2024-05-22T21:15:24.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.818+0000 7fd102a00640 1 -- 172.21.15.161:0/3516334581 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd0f4054c00 con 0x7fd10c06a410 2024-05-22T21:15:24.824 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:15:24.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.818+0000 7fd10b200640 1 -- 172.21.15.161:0/3516334581 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd0e80666c0 msgr2=0x7fd0e8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.818+0000 7fd10b200640 1 --2- 172.21.15.161:0/3516334581 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd0e80666c0 0x7fd0e8068b80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0ad000 tx=0x7fd0fc0a6000 comp rx=0 tx=0).stop 2024-05-22T21:15:24.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.818+0000 7fd10b200640 1 -- 172.21.15.161:0/3516334581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd10c06a410 msgr2=0x7fd10c14f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.818+0000 7fd10b200640 1 --2- 172.21.15.161:0/3516334581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd10c06a410 0x7fd10c14f840 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400e870 tx=0x7fd0f400ec70 comp rx=0 tx=0).stop 2024-05-22T21:15:24.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.818+0000 7fd10b200640 1 -- 172.21.15.161:0/3516334581 shutdown_connections 2024-05-22T21:15:24.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.818+0000 7fd10b200640 1 --2- 172.21.15.161:0/3516334581 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd0e80666c0 0x7fd0e8068b80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.818+0000 7fd10b200640 1 --2- 172.21.15.161:0/3516334581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd10c15d180 0x7fd10c15f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.818+0000 7fd10b200640 1 --2- 172.21.15.161:0/3516334581 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd10c0af070 0x7fd10c155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.818+0000 7fd10b200640 1 --2- 172.21.15.161:0/3516334581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd10c06a410 0x7fd10c14f840 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.818+0000 7fd10b200640 1 -- 172.21.15.161:0/3516334581 >> 172.21.15.161:0/3516334581 conn(0x7fd10c06b9d0 msgr2=0x7fd10c06d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:24.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.822+0000 7fd10b200640 1 -- 172.21.15.161:0/3516334581 shutdown_connections 2024-05-22T21:15:24.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.822+0000 7fd10b200640 1 -- 172.21.15.161:0/3516334581 wait complete. 2024-05-22T21:15:24.844 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:15:24.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16fc400640 1 Processor -- start 2024-05-22T21:15:24.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16fc400640 1 -- start start 2024-05-22T21:15:24.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16fc400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f16f41402e0 0x7f16f413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16fc400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f16f413fc80 0x7f16f406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16fc400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f41400d0 con 0x7f16f413cfc0 2024-05-22T21:15:24.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16fc400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f413d3a0 con 0x7f16f413fc80 2024-05-22T21:15:24.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16fc400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f40cd920 con 0x7f16f41402e0 2024-05-22T21:15:24.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f9a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f16f413cfc0 0x7f16f415af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53694/0 (socket says 172.21.15.161:53694) 2024-05-22T21:15:24.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f9a00640 1 -- 172.21.15.161:0/3302929571 learned_addr learned my addr 172.21.15.161:0/3302929571 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:24.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f9000640 1 --2- 172.21.15.161:0/3302929571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f16f41402e0 0x7f16f413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16fa400640 1 --2- 172.21.15.161:0/3302929571 >> v2:172.21.15.179:3300/0 conn(0x7f16f413fc80 0x7f16f406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f3e00640 1 -- 172.21.15.161:0/3302929571 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756647760 0 0) 0x7f16e8001120 con 0x7f16f413cfc0 2024-05-22T21:15:24.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f3e00640 1 -- 172.21.15.161:0/3302929571 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16dc003530 con 0x7f16f413cfc0 2024-05-22T21:15:24.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f3e00640 1 -- 172.21.15.161:0/3302929571 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 54551669 0 0) 0x7f16dc003530 con 0x7f16f413cfc0 2024-05-22T21:15:24.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f3e00640 1 -- 172.21.15.161:0/3302929571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f16f41402e0 msgr2=0x7f16f413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:24.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f3e00640 1 --2- 172.21.15.161:0/3302929571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f16f41402e0 0x7f16f413f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f3e00640 1 -- 172.21.15.161:0/3302929571 >> v2:172.21.15.179:3300/0 conn(0x7f16f413fc80 msgr2=0x7f16f406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:24.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f3e00640 1 --2- 172.21.15.161:0/3302929571 >> v2:172.21.15.179:3300/0 conn(0x7f16f413fc80 0x7f16f406fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f3e00640 1 -- 172.21.15.161:0/3302929571 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f40cdc00 con 0x7f16f413cfc0 2024-05-22T21:15:24.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f3e00640 1 -- 172.21.15.161:0/3302929571 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f16e8001810 con 0x7f16f413cfc0 2024-05-22T21:15:24.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f9000640 1 --2- 172.21.15.161:0/3302929571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f16f41402e0 0x7f16f413f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:15:24.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f3e00640 1 -- 172.21.15.161:0/3302929571 >> v1:172.21.15.161:6789/0 conn(0x7f16f413cfc0 legacy=0x7f16f415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f3e00640 1 --2- 172.21.15.161:0/3302929571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16dc003980 0x7f16dc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f3e00640 1 --2- 172.21.15.161:0/3302929571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f16f41402e0 0x7f16dc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f3e00640 1 --2- 172.21.15.161:0/3302929571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16dc004d10 0x7f16dc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f3e00640 1 -- 172.21.15.161:0/3302929571 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f413d3a0 con 0x7f16dc004d10 2024-05-22T21:15:24.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f3e00640 1 -- 172.21.15.161:0/3302929571 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f40cd920 con 0x7f16dc003980 2024-05-22T21:15:24.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f3e00640 1 -- 172.21.15.161:0/3302929571 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc00b990 con 0x7f16f41402e0 2024-05-22T21:15:24.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16fa400640 1 --2- 172.21.15.161:0/3302929571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16dc004d10 0x7f16dc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f9a00640 1 --2- 172.21.15.161:0/3302929571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16dc003980 0x7f16dc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16f9000640 1 --2- 172.21.15.161:0/3302929571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f16f41402e0 0x7f16dc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16fa400640 1 -- 172.21.15.161:0/3302929571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f16f41402e0 msgr2=0x7f16dc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:24.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16fa400640 1 --2- 172.21.15.161:0/3302929571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f16f41402e0 0x7f16dc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16fa400640 1 -- 172.21.15.161:0/3302929571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16dc003980 msgr2=0x7f16dc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:24.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16fa400640 1 --2- 172.21.15.161:0/3302929571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16dc003980 0x7f16dc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.978+0000 7f16fa400640 1 -- 172.21.15.161:0/3302929571 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e0009420 con 0x7f16dc004d10 2024-05-22T21:15:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.982+0000 7f16fa400640 1 --2- 172.21.15.161:0/3302929571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16dc004d10 0x7f16dc005190 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f16ec010350 tx=0x7f16ec01eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774e24ad795d2546 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.982+0000 7f16f3e00640 1 -- 172.21.15.161:0/3302929571 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec00b2d0 con 0x7f16dc004d10 2024-05-22T21:15:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.982+0000 7f16f3e00640 1 -- 172.21.15.161:0/3302929571 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ec011040 con 0x7f16dc004d10 2024-05-22T21:15:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.982+0000 7f16f3e00640 1 -- 172.21.15.161:0/3302929571 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0a7cf0 con 0x7f16dc004d10 2024-05-22T21:15:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.982+0000 7f16fc400640 1 -- 172.21.15.161:0/3302929571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16dc004d10 msgr2=0x7f16dc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.982+0000 7f16fc400640 1 --2- 172.21.15.161:0/3302929571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16dc004d10 0x7f16dc005190 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f16ec010350 tx=0x7f16ec01eec0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.982+0000 7f16fa400640 1 -- 172.21.15.161:0/3302929571 reap_dead start 2024-05-22T21:15:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.982+0000 7f16fc400640 1 -- 172.21.15.161:0/3302929571 shutdown_connections 2024-05-22T21:15:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.982+0000 7f16fc400640 1 -- 172.21.15.161:0/3302929571 >> 172.21.15.161:0/3302929571 conn(0x7f16f406a2d0 msgr2=0x7f16f406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.982+0000 7f16fc400640 1 -- 172.21.15.161:0/3302929571 shutdown_connections 2024-05-22T21:15:24.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.982+0000 7f16fc400640 1 -- 172.21.15.161:0/3302929571 wait complete. 2024-05-22T21:15:24.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fc400640 1 Processor -- start 2024-05-22T21:15:24.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fc400640 1 -- start start 2024-05-22T21:15:24.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fc400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16dc004d10 0x7f16f40b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fc400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f16f4157480 0x7f16f415cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fc400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16f40b32d0 0x7f16f415a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fc400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f415d500 con 0x7f16f40b32d0 2024-05-22T21:15:24.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fc400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f40b2230 con 0x7f16dc004d10 2024-05-22T21:15:24.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fc400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f40b2510 con 0x7f16f4157480 2024-05-22T21:15:24.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fa400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16f40b32d0 0x7f16f415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fa400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16f40b32d0 0x7f16f415a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51130/0 (socket says 172.21.15.161:51130) 2024-05-22T21:15:24.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fa400640 1 -- 172.21.15.161:0/2371859970 learned_addr learned my addr 172.21.15.161:0/2371859970 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:24.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16f9000640 1 --2- 172.21.15.161:0/2371859970 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f16f4157480 0x7f16f415cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16f9a00640 1 --2- 172.21.15.161:0/2371859970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16dc004d10 0x7f16f40b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fa400640 1 -- 172.21.15.161:0/2371859970 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f16f4157480 msgr2=0x7f16f415cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fa400640 1 --2- 172.21.15.161:0/2371859970 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f16f4157480 0x7f16f415cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fa400640 1 -- 172.21.15.161:0/2371859970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16dc004d10 msgr2=0x7f16f40b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:24.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fa400640 1 --2- 172.21.15.161:0/2371859970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16dc004d10 0x7f16f40b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:24.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fa400640 1 -- 172.21.15.161:0/2371859970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f40b27f0 con 0x7f16f40b32d0 2024-05-22T21:15:24.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16f9a00640 1 --2- 172.21.15.161:0/2371859970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16dc004d10 0x7f16f40b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:24.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fa400640 1 --2- 172.21.15.161:0/2371859970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16f40b32d0 0x7f16f415a370 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00b040 tx=0x7f16ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:24.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16f1600640 1 -- 172.21.15.161:0/2371859970 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0bb070 con 0x7f16f40b32d0 2024-05-22T21:15:24.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16f1600640 1 -- 172.21.15.161:0/2371859970 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ec0092e0 con 0x7f16f40b32d0 2024-05-22T21:15:24.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fc400640 1 -- 172.21.15.161:0/2371859970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f40b2ad0 con 0x7f16f40b32d0 2024-05-22T21:15:24.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16f1600640 1 -- 172.21.15.161:0/2371859970 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec009970 con 0x7f16f40b32d0 2024-05-22T21:15:24.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.986+0000 7f16fc400640 1 -- 172.21.15.161:0/2371859970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f41578e0 con 0x7f16f40b32d0 2024-05-22T21:15:24.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.990+0000 7f16fc400640 1 -- 172.21.15.161:0/2371859970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f4067220 con 0x7f16f40b32d0 2024-05-22T21:15:24.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.990+0000 7f16f1600640 1 -- 172.21.15.161:0/2371859970 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f16ec009ad0 con 0x7f16f40b32d0 2024-05-22T21:15:24.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.990+0000 7f16f1600640 1 --2- 172.21.15.161:0/2371859970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16d40666c0 0x7f16d4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:24.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.990+0000 7f16f1600640 1 -- 172.21.15.161:0/2371859970 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f16ec11e8e0 con 0x7f16f40b32d0 2024-05-22T21:15:24.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.990+0000 7f16f9a00640 1 --2- 172.21.15.161:0/2371859970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16d40666c0 0x7f16d4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:24.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.990+0000 7f16f9a00640 1 --2- 172.21.15.161:0/2371859970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16d40666c0 0x7f16d4068b80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f16e8001380 tx=0x7f16e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:25.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:24.994+0000 7f16f1600640 1 -- 172.21.15.161:0/2371859970 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f16ec0ebc00 con 0x7f16f40b32d0 2024-05-22T21:15:25.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.150+0000 7f16fc400640 1 -- 172.21.15.161:0/2371859970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f16f4071d00 con 0x7f16f40b32d0 2024-05-22T21:15:25.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.150+0000 7f16f1600640 1 -- 172.21.15.161:0/2371859970 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f16ec00be20 con 0x7f16f40b32d0 2024-05-22T21:15:25.154 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:15:25.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.150+0000 7f16fc400640 1 -- 172.21.15.161:0/2371859970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16d40666c0 msgr2=0x7f16d4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:25.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.150+0000 7f16fc400640 1 --2- 172.21.15.161:0/2371859970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16d40666c0 0x7f16d4068b80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f16e8001380 tx=0x7f16e8009040 comp rx=0 tx=0).stop 2024-05-22T21:15:25.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.150+0000 7f16fc400640 1 -- 172.21.15.161:0/2371859970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16f40b32d0 msgr2=0x7f16f415a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:25.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.150+0000 7f16fc400640 1 --2- 172.21.15.161:0/2371859970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16f40b32d0 0x7f16f415a370 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00b040 tx=0x7f16ec00b960 comp rx=0 tx=0).stop 2024-05-22T21:15:25.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.150+0000 7f16fc400640 1 -- 172.21.15.161:0/2371859970 shutdown_connections 2024-05-22T21:15:25.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.150+0000 7f16fc400640 1 --2- 172.21.15.161:0/2371859970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16d40666c0 0x7f16d4068b80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.150+0000 7f16fc400640 1 --2- 172.21.15.161:0/2371859970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16f40b32d0 0x7f16f415a370 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.150+0000 7f16fc400640 1 --2- 172.21.15.161:0/2371859970 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f16f4157480 0x7f16f415cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.150+0000 7f16fc400640 1 --2- 172.21.15.161:0/2371859970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16dc004d10 0x7f16f40b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.150+0000 7f16fc400640 1 -- 172.21.15.161:0/2371859970 >> 172.21.15.161:0/2371859970 conn(0x7f16f406a2d0 msgr2=0x7f16f406cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:25.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.150+0000 7f16fc400640 1 -- 172.21.15.161:0/2371859970 shutdown_connections 2024-05-22T21:15:25.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.150+0000 7f16fc400640 1 -- 172.21.15.161:0/2371859970 wait complete. 2024-05-22T21:15:25.175 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:15:25.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.310+0000 7f0437200640 1 Processor -- start 2024-05-22T21:15:25.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.310+0000 7f0437200640 1 -- start start 2024-05-22T21:15:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.310+0000 7f0437200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f04380cde80 0x7f04380d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.310+0000 7f0437200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04380cb8d0 con 0x7f04380b0c70 2024-05-22T21:15:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.310+0000 7f0437200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04380cf180 con 0x7f04380cdaa0 2024-05-22T21:15:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.310+0000 7f0437200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04380cb740 con 0x7f04380cde80 2024-05-22T21:15:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.310+0000 7f042d400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f04380cde80 0x7f04380d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.310+0000 7f042d400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f04380cde80 0x7f04380d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51144/0 (socket says 172.21.15.161:51144) 2024-05-22T21:15:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.310+0000 7f042d400640 1 -- 172.21.15.161:0/1978660125 learned_addr learned my addr 172.21.15.161:0/1978660125 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.310+0000 7f0436800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f04380b0c70 0x7f04380cbd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53708/0 (socket says 172.21.15.161:53708) 2024-05-22T21:15:25.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.310+0000 7f042d400640 1 -- 172.21.15.161:0/1978660125 >> v1:172.21.15.179:6789/0 conn(0x7f04380cdaa0 legacy=0x7f04380b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:25.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.310+0000 7f042d400640 1 -- 172.21.15.161:0/1978660125 >> v1:172.21.15.161:6789/0 conn(0x7f04380b0c70 legacy=0x7f04380cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:25.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.310+0000 7f042d400640 1 -- 172.21.15.161:0/1978660125 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04380cf460 con 0x7f04380cde80 2024-05-22T21:15:25.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.310+0000 7f0435400640 1 -- 172.21.15.161:0/1978660125 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713501946 0 0) 0x7f04380cb8d0 con 0x7f04380b0c70 2024-05-22T21:15:25.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.314+0000 7f042d400640 1 --2- 172.21.15.161:0/1978660125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04380cde80 0x7f04380d1950 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0420009870 tx=0x7f042001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e715aff0e550edd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:25.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.314+0000 7f0435400640 1 -- 172.21.15.161:0/1978660125 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04200a95f0 con 0x7f04380cde80 2024-05-22T21:15:25.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.314+0000 7f0435400640 1 -- 172.21.15.161:0/1978660125 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04200a9750 con 0x7f04380cde80 2024-05-22T21:15:25.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.314+0000 7f0435400640 1 -- 172.21.15.161:0/1978660125 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04200a9ae0 con 0x7f04380cde80 2024-05-22T21:15:25.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.314+0000 7f0437200640 1 -- 172.21.15.161:0/1978660125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04380cde80 msgr2=0x7f04380d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:25.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.314+0000 7f0437200640 1 --2- 172.21.15.161:0/1978660125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04380cde80 0x7f04380d1950 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0420009870 tx=0x7f042001c530 comp rx=0 tx=0).stop 2024-05-22T21:15:25.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.314+0000 7f0437200640 1 -- 172.21.15.161:0/1978660125 shutdown_connections 2024-05-22T21:15:25.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.314+0000 7f0437200640 1 --2- 172.21.15.161:0/1978660125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04380cde80 0x7f04380d1950 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.314+0000 7f0437200640 1 -- 172.21.15.161:0/1978660125 >> 172.21.15.161:0/1978660125 conn(0x7f04380d0e30 msgr2=0x7f04380d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:25.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.314+0000 7f0437200640 1 -- 172.21.15.161:0/1978660125 shutdown_connections 2024-05-22T21:15:25.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.314+0000 7f0437200640 1 -- 172.21.15.161:0/1978660125 wait complete. 2024-05-22T21:15:25.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0437200640 1 Processor -- start 2024-05-22T21:15:25.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0437200640 1 -- start start 2024-05-22T21:15:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0437200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04380b0c70 0x7f04380c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0437200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04380cdaa0 0x7f0438158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0437200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04380cde80 0x7f04381f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0437200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04380c9e40 con 0x7f04380cde80 2024-05-22T21:15:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0437200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04380c7ca0 con 0x7f04380cdaa0 2024-05-22T21:15:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0437200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04380c8f50 con 0x7f04380b0c70 2024-05-22T21:15:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0436800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04380cde80 0x7f04381f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0436800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04380cde80 0x7f04381f93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51158/0 (socket says 172.21.15.161:51158) 2024-05-22T21:15:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0436800640 1 -- 172.21.15.161:0/1474494936 learned_addr learned my addr 172.21.15.161:0/1474494936 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f042d400640 1 --2- 172.21.15.161:0/1474494936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04380cdaa0 0x7f0438158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0435e00640 1 --2- 172.21.15.161:0/1474494936 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04380b0c70 0x7f04380c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0436800640 1 -- 172.21.15.161:0/1474494936 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04380b0c70 msgr2=0x7f04380c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0436800640 1 --2- 172.21.15.161:0/1474494936 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04380b0c70 0x7f04380c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0436800640 1 -- 172.21.15.161:0/1474494936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04380cdaa0 msgr2=0x7f0438158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0436800640 1 --2- 172.21.15.161:0/1474494936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04380cdaa0 0x7f0438158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0436800640 1 -- 172.21.15.161:0/1474494936 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04200094d0 con 0x7f04380cde80 2024-05-22T21:15:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f042d400640 1 --2- 172.21.15.161:0/1474494936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04380cdaa0 0x7f0438158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0436800640 1 --2- 172.21.15.161:0/1474494936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04380cde80 0x7f04381f93f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0430001570 tx=0x7f043000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f042ea00640 1 -- 172.21.15.161:0/1474494936 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043000a640 con 0x7f04380cde80 2024-05-22T21:15:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f042ea00640 1 -- 172.21.15.161:0/1474494936 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043000a7c0 con 0x7f04380cde80 2024-05-22T21:15:25.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f042ea00640 1 -- 172.21.15.161:0/1474494936 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043000ab70 con 0x7f04380cde80 2024-05-22T21:15:25.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0437200640 1 -- 172.21.15.161:0/1474494936 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04380c9230 con 0x7f04380cde80 2024-05-22T21:15:25.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0437200640 1 -- 172.21.15.161:0/1474494936 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04380c97f0 con 0x7f04380cde80 2024-05-22T21:15:25.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.318+0000 7f0437200640 1 -- 172.21.15.161:0/1474494936 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04380676a0 con 0x7f04380cde80 2024-05-22T21:15:25.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.322+0000 7f042ea00640 1 -- 172.21.15.161:0/1474494936 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f043000acd0 con 0x7f04380cde80 2024-05-22T21:15:25.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.322+0000 7f042ea00640 1 --2- 172.21.15.161:0/1474494936 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f04140666c0 0x7f0414068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:25.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.322+0000 7f042ea00640 1 -- 172.21.15.161:0/1474494936 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0430083700 con 0x7f04380cde80 2024-05-22T21:15:25.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.322+0000 7f0435e00640 1 --2- 172.21.15.161:0/1474494936 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f04140666c0 0x7f0414068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:25.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.322+0000 7f0435e00640 1 --2- 172.21.15.161:0/1474494936 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f04140666c0 0x7f0414068b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0428000f70 tx=0x7f0428009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:25.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.326+0000 7f042ea00640 1 -- 172.21.15.161:0/1474494936 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0430050990 con 0x7f04380cde80 2024-05-22T21:15:25.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.494+0000 7f0437200640 1 -- 172.21.15.161:0/1474494936 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f043806c270 con 0x7f04380cde80 2024-05-22T21:15:25.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.494+0000 7f042ea00640 1 -- 172.21.15.161:0/1474494936 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0430053fb0 con 0x7f04380cde80 2024-05-22T21:15:25.499 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:15:25.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.494+0000 7f0437200640 1 -- 172.21.15.161:0/1474494936 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f04140666c0 msgr2=0x7f0414068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:25.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.494+0000 7f0437200640 1 --2- 172.21.15.161:0/1474494936 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f04140666c0 0x7f0414068b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0428000f70 tx=0x7f0428009040 comp rx=0 tx=0).stop 2024-05-22T21:15:25.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.494+0000 7f0437200640 1 -- 172.21.15.161:0/1474494936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04380cde80 msgr2=0x7f04381f93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:25.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.494+0000 7f0437200640 1 --2- 172.21.15.161:0/1474494936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04380cde80 0x7f04381f93f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0430001570 tx=0x7f043000cca0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.494+0000 7f0437200640 1 -- 172.21.15.161:0/1474494936 shutdown_connections 2024-05-22T21:15:25.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.494+0000 7f0437200640 1 --2- 172.21.15.161:0/1474494936 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f04140666c0 0x7f0414068b80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.494+0000 7f0437200640 1 --2- 172.21.15.161:0/1474494936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04380cde80 0x7f04381f93f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.494+0000 7f0437200640 1 --2- 172.21.15.161:0/1474494936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04380cdaa0 0x7f0438158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.494+0000 7f0437200640 1 --2- 172.21.15.161:0/1474494936 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04380b0c70 0x7f04380c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.494+0000 7f0437200640 1 -- 172.21.15.161:0/1474494936 >> 172.21.15.161:0/1474494936 conn(0x7f04380d0e30 msgr2=0x7f0438158fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:25.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.494+0000 7f0437200640 1 -- 172.21.15.161:0/1474494936 shutdown_connections 2024-05-22T21:15:25.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.494+0000 7f0437200640 1 -- 172.21.15.161:0/1474494936 wait complete. 2024-05-22T21:15:25.519 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-22T21:15:25.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:25.519 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-22T21:15:25.579 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:15:25.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:25.579 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-22T21:15:25.595 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:15:25.595 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:15:25.604 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:15:25.604 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:15:25.655 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:15:25.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.786+0000 7fd1fa200640 1 Processor -- start 2024-05-22T21:15:25.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.786+0000 7fd1fa200640 1 -- start start 2024-05-22T21:15:25.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.786+0000 7fd1fa200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd1f406b5f0 0x7fd1f406c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:25.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.786+0000 7fd1fa200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd1f406c8f0 0x7fd1f40ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:25.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.786+0000 7fd1fa200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f415db70 con 0x7fd1f40b4420 2024-05-22T21:15:25.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.786+0000 7fd1fa200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f40ad610 con 0x7fd1f406b5f0 2024-05-22T21:15:25.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.786+0000 7fd1fa200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f406cd20 con 0x7fd1f406c8f0 2024-05-22T21:15:25.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.786+0000 7fd1f2a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd1f406b5f0 0x7fd1f406c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:25.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.786+0000 7fd1f2a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd1f406b5f0 0x7fd1f406c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51172/0 (socket says 172.21.15.161:51172) 2024-05-22T21:15:25.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.786+0000 7fd1f2a00640 1 -- 172.21.15.161:0/1223631199 learned_addr learned my addr 172.21.15.161:0/1223631199 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:25.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.786+0000 7fd1f3400640 1 --1- 172.21.15.161:0/1223631199 >> v1:172.21.15.161:6789/0 conn(0x7fd1f40b4420 0x7fd1f406aee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53718/0 (socket says 172.21.15.161:53718) 2024-05-22T21:15:25.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.786+0000 7fd1f3e00640 1 --2- 172.21.15.161:0/1223631199 >> v2:172.21.15.179:3300/0 conn(0x7fd1f406c8f0 0x7fd1f40ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:25.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.786+0000 7fd1f2a00640 1 -- 172.21.15.161:0/1223631199 >> v2:172.21.15.179:3300/0 conn(0x7fd1f406c8f0 msgr2=0x7fd1f40ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:25.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.786+0000 7fd1f2a00640 1 --2- 172.21.15.161:0/1223631199 >> v2:172.21.15.179:3300/0 conn(0x7fd1f406c8f0 0x7fd1f40ae050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.786+0000 7fd1f2a00640 1 -- 172.21.15.161:0/1223631199 >> v1:172.21.15.161:6789/0 conn(0x7fd1f40b4420 legacy=0x7fd1f406aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:25.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.786+0000 7fd1f2a00640 1 -- 172.21.15.161:0/1223631199 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f40b26f0 con 0x7fd1f406b5f0 2024-05-22T21:15:25.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.790+0000 7fd1f2a00640 1 --2- 172.21.15.161:0/1223631199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1f406b5f0 0x7fd1f406c3b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd1f40717e0 tx=0x7fd1e001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624edbbc453e656d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:25.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.790+0000 7fd1f2000640 1 -- 172.21.15.161:0/1223631199 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00aa680 con 0x7fd1f406b5f0 2024-05-22T21:15:25.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.790+0000 7fd1f2000640 1 -- 172.21.15.161:0/1223631199 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e00aa7e0 con 0x7fd1f406b5f0 2024-05-22T21:15:25.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.790+0000 7fd1f2000640 1 -- 172.21.15.161:0/1223631199 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00aab70 con 0x7fd1f406b5f0 2024-05-22T21:15:25.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.790+0000 7fd1fa200640 1 -- 172.21.15.161:0/1223631199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1f406b5f0 msgr2=0x7fd1f406c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:25.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.790+0000 7fd1fa200640 1 --2- 172.21.15.161:0/1223631199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1f406b5f0 0x7fd1f406c3b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd1f40717e0 tx=0x7fd1e001c520 comp rx=0 tx=0).stop 2024-05-22T21:15:25.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.790+0000 7fd1fa200640 1 -- 172.21.15.161:0/1223631199 shutdown_connections 2024-05-22T21:15:25.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.790+0000 7fd1fa200640 1 --2- 172.21.15.161:0/1223631199 >> v2:172.21.15.179:3300/0 conn(0x7fd1f406c8f0 0x7fd1f40ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.790+0000 7fd1fa200640 1 --2- 172.21.15.161:0/1223631199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1f406b5f0 0x7fd1f406c3b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.790+0000 7fd1fa200640 1 -- 172.21.15.161:0/1223631199 >> 172.21.15.161:0/1223631199 conn(0x7fd1f4070630 msgr2=0x7fd1f4070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:25.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.790+0000 7fd1fa200640 1 -- 172.21.15.161:0/1223631199 shutdown_connections 2024-05-22T21:15:25.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.790+0000 7fd1fa200640 1 -- 172.21.15.161:0/1223631199 wait complete. 2024-05-22T21:15:25.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1fa200640 1 Processor -- start 2024-05-22T21:15:25.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1fa200640 1 -- start start 2024-05-22T21:15:25.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1fa200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1f406b5f0 0x7fd1f414a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:25.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1fa200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1f406c8f0 0x7fd1f4148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:25.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1fa200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1f40b4420 0x7fd1f4151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:25.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1fa200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f414b3b0 con 0x7fd1f406c8f0 2024-05-22T21:15:25.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1fa200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f414abd0 con 0x7fd1f40b4420 2024-05-22T21:15:25.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1fa200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f4148760 con 0x7fd1f406b5f0 2024-05-22T21:15:25.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1f2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1f406c8f0 0x7fd1f4148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:25.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1f2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1f406c8f0 0x7fd1f4148220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51178/0 (socket says 172.21.15.161:51178) 2024-05-22T21:15:25.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1f2a00640 1 -- 172.21.15.161:0/3843599111 learned_addr learned my addr 172.21.15.161:0/3843599111 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:25.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1f3400640 1 --2- 172.21.15.161:0/3843599111 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1f406b5f0 0x7fd1f414a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:25.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1f3e00640 1 --2- 172.21.15.161:0/3843599111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1f40b4420 0x7fd1f4151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:25.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1f2a00640 1 -- 172.21.15.161:0/3843599111 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1f406b5f0 msgr2=0x7fd1f414a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:25.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1f2a00640 1 --2- 172.21.15.161:0/3843599111 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1f406b5f0 0x7fd1f414a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1f2a00640 1 -- 172.21.15.161:0/3843599111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1f40b4420 msgr2=0x7fd1f4151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:25.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1f2a00640 1 --2- 172.21.15.161:0/3843599111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1f40b4420 0x7fd1f4151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1f2a00640 1 -- 172.21.15.161:0/3843599111 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e00094d0 con 0x7fd1f406c8f0 2024-05-22T21:15:25.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1f2a00640 1 --2- 172.21.15.161:0/3843599111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1f406c8f0 0x7fd1f4148220 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0019130 tx=0x7fd1e001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:25.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1eb400640 1 -- 172.21.15.161:0/3843599111 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00a99b0 con 0x7fd1f406c8f0 2024-05-22T21:15:25.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1eb400640 1 -- 172.21.15.161:0/3843599111 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e00a9b30 con 0x7fd1f406c8f0 2024-05-22T21:15:25.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1fa200640 1 -- 172.21.15.161:0/3843599111 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f414adf0 con 0x7fd1f406c8f0 2024-05-22T21:15:25.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1eb400640 1 -- 172.21.15.161:0/3843599111 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00a9ee0 con 0x7fd1f406c8f0 2024-05-22T21:15:25.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.794+0000 7fd1fa200640 1 -- 172.21.15.161:0/3843599111 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f4158fd0 con 0x7fd1f406c8f0 2024-05-22T21:15:25.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.798+0000 7fd1fa200640 1 -- 172.21.15.161:0/3843599111 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f4158e30 con 0x7fd1f406c8f0 2024-05-22T21:15:25.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.798+0000 7fd1eb400640 1 -- 172.21.15.161:0/3843599111 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd1e00b33f0 con 0x7fd1f406c8f0 2024-05-22T21:15:25.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.798+0000 7fd1eb400640 1 --2- 172.21.15.161:0/3843599111 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd1d4066690 0x7fd1d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:25.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.798+0000 7fd1eb400640 1 -- 172.21.15.161:0/3843599111 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd1e011eea0 con 0x7fd1f406c8f0 2024-05-22T21:15:25.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.798+0000 7fd1f3400640 1 --2- 172.21.15.161:0/3843599111 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd1d4066690 0x7fd1d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:25.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.798+0000 7fd1f3400640 1 --2- 172.21.15.161:0/3843599111 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd1d4066690 0x7fd1d4068b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40012f0 tx=0x7fd1e4001660 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:25.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.802+0000 7fd1eb400640 1 -- 172.21.15.161:0/3843599111 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd1f4158e30 con 0x7fd1f406c8f0 2024-05-22T21:15:25.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.962+0000 7fd1fa200640 1 -- 172.21.15.161:0/3843599111 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1f4158e30 con 0x7fd1f406c8f0 2024-05-22T21:15:25.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.962+0000 7fd1eb400640 1 -- 172.21.15.161:0/3843599111 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16183 (secure 0 0 0) 0x7fd1f4158e30 con 0x7fd1f406c8f0 2024-05-22T21:15:25.967 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:25.967 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":55,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:15:17.769605+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:48.612726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:49.053022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.9","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:15:25.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.962+0000 7fd1fa200640 1 -- 172.21.15.161:0/3843599111 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd1d4066690 msgr2=0x7fd1d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:25.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.962+0000 7fd1fa200640 1 --2- 172.21.15.161:0/3843599111 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd1d4066690 0x7fd1d4068b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40012f0 tx=0x7fd1e4001660 comp rx=0 tx=0).stop 2024-05-22T21:15:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.962+0000 7fd1fa200640 1 -- 172.21.15.161:0/3843599111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1f406c8f0 msgr2=0x7fd1f4148220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.962+0000 7fd1fa200640 1 --2- 172.21.15.161:0/3843599111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1f406c8f0 0x7fd1f4148220 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0019130 tx=0x7fd1e001ad60 comp rx=0 tx=0).stop 2024-05-22T21:15:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.962+0000 7fd1fa200640 1 -- 172.21.15.161:0/3843599111 shutdown_connections 2024-05-22T21:15:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.962+0000 7fd1fa200640 1 --2- 172.21.15.161:0/3843599111 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd1d4066690 0x7fd1d4068b50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.962+0000 7fd1fa200640 1 --2- 172.21.15.161:0/3843599111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1f40b4420 0x7fd1f4151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.962+0000 7fd1fa200640 1 --2- 172.21.15.161:0/3843599111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1f406c8f0 0x7fd1f4148220 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.962+0000 7fd1fa200640 1 --2- 172.21.15.161:0/3843599111 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1f406b5f0 0x7fd1f414a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.962+0000 7fd1fa200640 1 -- 172.21.15.161:0/3843599111 >> 172.21.15.161:0/3843599111 conn(0x7fd1f4070630 msgr2=0x7fd1f4157440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.962+0000 7fd1fa200640 1 -- 172.21.15.161:0/3843599111 shutdown_connections 2024-05-22T21:15:25.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:25.962+0000 7fd1fa200640 1 -- 172.21.15.161:0/3843599111 wait complete. 2024-05-22T21:15:25.989 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:14:48.612726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:14:49.053022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:15:25.989 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:15:26.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.122+0000 7f54d4800640 1 Processor -- start 2024-05-22T21:15:26.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.122+0000 7f54d4800640 1 -- start start 2024-05-22T21:15:26.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.122+0000 7f54d4800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f54cc13fe50 0x7f54cc164e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.122+0000 7f54d4800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54cc13d3a0 con 0x7f54cc1402e0 2024-05-22T21:15:26.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.122+0000 7f54d4800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54cc0cd920 con 0x7f54cc13cfc0 2024-05-22T21:15:26.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.122+0000 7f54d4800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54cc0cdc00 con 0x7f54cc13fe50 2024-05-22T21:15:26.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.122+0000 7f54d2800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f54cc13fe50 0x7f54cc164e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.122+0000 7f54d1400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f54cc1402e0 0x7f54cc13f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60436/0 (socket says 172.21.15.161:60436) 2024-05-22T21:15:26.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.122+0000 7f54d2800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f54cc13fe50 0x7f54cc164e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42860/0 (socket says 172.21.15.161:42860) 2024-05-22T21:15:26.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.122+0000 7f54d1400640 1 -- 172.21.15.161:0/2637629243 learned_addr learned my addr 172.21.15.161:0/2637629243 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:26.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.122+0000 7f54d2800640 1 -- 172.21.15.161:0/2637629243 >> v1:172.21.15.179:6789/0 conn(0x7f54cc13cfc0 legacy=0x7f54cc15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.122+0000 7f54d2800640 1 -- 172.21.15.161:0/2637629243 >> v1:172.21.15.161:6789/0 conn(0x7f54cc1402e0 legacy=0x7f54cc13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.122+0000 7f54d0a00640 1 -- 172.21.15.161:0/2637629243 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1133903074 0 0) 0x7f54cc13d3a0 con 0x7f54cc1402e0 2024-05-22T21:15:26.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.122+0000 7f54d2800640 1 -- 172.21.15.161:0/2637629243 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc0cdee0 con 0x7f54cc13fe50 2024-05-22T21:15:26.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.126+0000 7f54d2800640 1 --2- 172.21.15.161:0/2637629243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54cc13fe50 0x7f54cc164e60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009d90 tx=0x7f54c801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e74b1d764bf0c76 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:26.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.126+0000 7f54d0a00640 1 -- 172.21.15.161:0/2637629243 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c80a5890 con 0x7f54cc13fe50 2024-05-22T21:15:26.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.126+0000 7f54d0a00640 1 -- 172.21.15.161:0/2637629243 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c80a59f0 con 0x7f54cc13fe50 2024-05-22T21:15:26.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.126+0000 7f54d0a00640 1 -- 172.21.15.161:0/2637629243 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c80a5d80 con 0x7f54cc13fe50 2024-05-22T21:15:26.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.126+0000 7f54d4800640 1 -- 172.21.15.161:0/2637629243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54cc13fe50 msgr2=0x7f54cc164e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.126+0000 7f54d4800640 1 --2- 172.21.15.161:0/2637629243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54cc13fe50 0x7f54cc164e60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009d90 tx=0x7f54c801c8e0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.126+0000 7f54d4800640 1 -- 172.21.15.161:0/2637629243 shutdown_connections 2024-05-22T21:15:26.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.126+0000 7f54d4800640 1 --2- 172.21.15.161:0/2637629243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54cc13fe50 0x7f54cc164e60 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.126+0000 7f54d4800640 1 -- 172.21.15.161:0/2637629243 >> 172.21.15.161:0/2637629243 conn(0x7f54cc06a2d0 msgr2=0x7f54cc06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:26.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.126+0000 7f54d4800640 1 -- 172.21.15.161:0/2637629243 shutdown_connections 2024-05-22T21:15:26.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.126+0000 7f54d4800640 1 -- 172.21.15.161:0/2637629243 wait complete. 2024-05-22T21:15:26.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d4800640 1 Processor -- start 2024-05-22T21:15:26.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d4800640 1 -- start start 2024-05-22T21:15:26.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d4800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54cc13cfc0 0x7f54cc1576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d4800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54cc13fe50 0x7f54cc0addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d4800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54cc1402e0 0x7f54cc15b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d4800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc157be0 con 0x7f54cc13fe50 2024-05-22T21:15:26.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d4800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc0ae310 con 0x7f54cc13cfc0 2024-05-22T21:15:26.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d4800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc15b710 con 0x7f54cc1402e0 2024-05-22T21:15:26.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d1400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54cc13fe50 0x7f54cc0addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d1400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54cc13fe50 0x7f54cc0addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42866/0 (socket says 172.21.15.161:42866) 2024-05-22T21:15:26.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d1400640 1 -- 172.21.15.161:0/975869519 learned_addr learned my addr 172.21.15.161:0/975869519 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:26.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d2800640 1 --2- 172.21.15.161:0/975869519 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54cc1402e0 0x7f54cc15b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d1e00640 1 --2- 172.21.15.161:0/975869519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54cc13cfc0 0x7f54cc1576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d1400640 1 -- 172.21.15.161:0/975869519 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54cc1402e0 msgr2=0x7f54cc15b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d1400640 1 --2- 172.21.15.161:0/975869519 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54cc1402e0 0x7f54cc15b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d1400640 1 -- 172.21.15.161:0/975869519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54cc13cfc0 msgr2=0x7f54cc1576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d1400640 1 --2- 172.21.15.161:0/975869519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54cc13cfc0 0x7f54cc1576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d1400640 1 -- 172.21.15.161:0/975869519 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c8009a70 con 0x7f54cc13fe50 2024-05-22T21:15:26.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d1e00640 1 --2- 172.21.15.161:0/975869519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54cc13cfc0 0x7f54cc1576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:26.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d2800640 1 --2- 172.21.15.161:0/975869519 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54cc1402e0 0x7f54cc15b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:26.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d1400640 1 --2- 172.21.15.161:0/975869519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54cc13fe50 0x7f54cc0addd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00c4a0 tx=0x7f54bc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:26.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54c6000640 1 -- 172.21.15.161:0/975869519 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc0074e0 con 0x7f54cc13fe50 2024-05-22T21:15:26.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54c6000640 1 -- 172.21.15.161:0/975869519 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54bc007640 con 0x7f54cc13fe50 2024-05-22T21:15:26.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54c6000640 1 -- 172.21.15.161:0/975869519 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc004a30 con 0x7f54cc13fe50 2024-05-22T21:15:26.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d4800640 1 -- 172.21.15.161:0/975869519 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54cc15b9d0 con 0x7f54cc13fe50 2024-05-22T21:15:26.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.130+0000 7f54d4800640 1 -- 172.21.15.161:0/975869519 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54cc15bf90 con 0x7f54cc13fe50 2024-05-22T21:15:26.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.134+0000 7f54d4800640 1 -- 172.21.15.161:0/975869519 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54cc070970 con 0x7f54cc13fe50 2024-05-22T21:15:26.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.134+0000 7f54c6000640 1 -- 172.21.15.161:0/975869519 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f54bc004b90 con 0x7f54cc13fe50 2024-05-22T21:15:26.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.134+0000 7f54c6000640 1 --2- 172.21.15.161:0/975869519 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f54b0066690 0x7f54b0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.134+0000 7f54c6000640 1 -- 172.21.15.161:0/975869519 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f54bc083570 con 0x7f54cc13fe50 2024-05-22T21:15:26.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.134+0000 7f54d1e00640 1 --2- 172.21.15.161:0/975869519 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f54b0066690 0x7f54b0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.134+0000 7f54d1e00640 1 --2- 172.21.15.161:0/975869519 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f54b0066690 0x7f54b0068b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f54c0000f70 tx=0x7f54c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:26.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.138+0000 7f54c6000640 1 -- 172.21.15.161:0/975869519 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f54bc050770 con 0x7f54cc13fe50 2024-05-22T21:15:26.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.290+0000 7f54d4800640 1 -- 172.21.15.161:0/975869519 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54cc067220 con 0x7f54cc13fe50 2024-05-22T21:15:26.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.290+0000 7f54c6000640 1 -- 172.21.15.161:0/975869519 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f54bc018020 con 0x7f54cc13fe50 2024-05-22T21:15:26.297 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:15:26.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.294+0000 7f54d4800640 1 -- 172.21.15.161:0/975869519 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f54b0066690 msgr2=0x7f54b0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.294+0000 7f54d4800640 1 --2- 172.21.15.161:0/975869519 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f54b0066690 0x7f54b0068b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f54c0000f70 tx=0x7f54c0009040 comp rx=0 tx=0).stop 2024-05-22T21:15:26.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.294+0000 7f54d4800640 1 -- 172.21.15.161:0/975869519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54cc13fe50 msgr2=0x7f54cc0addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.294+0000 7f54d4800640 1 --2- 172.21.15.161:0/975869519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54cc13fe50 0x7f54cc0addd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00c4a0 tx=0x7f54bc00c960 comp rx=0 tx=0).stop 2024-05-22T21:15:26.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.294+0000 7f54d4800640 1 -- 172.21.15.161:0/975869519 shutdown_connections 2024-05-22T21:15:26.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.294+0000 7f54d4800640 1 --2- 172.21.15.161:0/975869519 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f54b0066690 0x7f54b0068b50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.294+0000 7f54d4800640 1 --2- 172.21.15.161:0/975869519 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54cc1402e0 0x7f54cc15b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.294+0000 7f54d4800640 1 --2- 172.21.15.161:0/975869519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54cc13fe50 0x7f54cc0addd0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.294+0000 7f54d4800640 1 --2- 172.21.15.161:0/975869519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54cc13cfc0 0x7f54cc1576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.294+0000 7f54d4800640 1 -- 172.21.15.161:0/975869519 >> 172.21.15.161:0/975869519 conn(0x7f54cc06a2d0 msgr2=0x7f54cc06cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:26.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.294+0000 7f54d4800640 1 -- 172.21.15.161:0/975869519 shutdown_connections 2024-05-22T21:15:26.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.294+0000 7f54d4800640 1 -- 172.21.15.161:0/975869519 wait complete. 2024-05-22T21:15:26.318 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:15:26.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.454+0000 7f477f200640 1 Processor -- start 2024-05-22T21:15:26.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.454+0000 7f477f200640 1 -- start start 2024-05-22T21:15:26.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.454+0000 7f477f200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f47781596b0 0x7f4778071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.454+0000 7f477f200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4778071db0 0x7f477806fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.454+0000 7f477f200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4778070220 0x7f4778070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.454+0000 7f477f200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477815d250 con 0x7f4778070220 2024-05-22T21:15:26.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.454+0000 7f477f200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477815d510 con 0x7f47781596b0 2024-05-22T21:15:26.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.454+0000 7f477f200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477815d7f0 con 0x7f4778071db0 2024-05-22T21:15:26.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.454+0000 7f477d200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4778070220 0x7f4778070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.454+0000 7f477d200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4778070220 0x7f4778070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42882/0 (socket says 172.21.15.161:42882) 2024-05-22T21:15:26.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.454+0000 7f477d200640 1 -- 172.21.15.161:0/3638616232 learned_addr learned my addr 172.21.15.161:0/3638616232 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:26.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.458+0000 7f4777e00640 1 --2- 172.21.15.161:0/3638616232 >> v2:172.21.15.179:3300/0 conn(0x7f47781596b0 0x7f4778071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.458+0000 7f476fe00640 1 --2- 172.21.15.161:0/3638616232 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4778071db0 0x7f477806fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.458+0000 7f477d200640 1 -- 172.21.15.161:0/3638616232 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4778071db0 msgr2=0x7f477806fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:26.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.458+0000 7f477d200640 1 --2- 172.21.15.161:0/3638616232 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4778071db0 0x7f477806fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.458+0000 7f477d200640 1 -- 172.21.15.161:0/3638616232 >> v2:172.21.15.179:3300/0 conn(0x7f47781596b0 msgr2=0x7f4778071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:26.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.458+0000 7f477d200640 1 --2- 172.21.15.161:0/3638616232 >> v2:172.21.15.179:3300/0 conn(0x7f47781596b0 0x7f4778071870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.458+0000 7f477d200640 1 -- 172.21.15.161:0/3638616232 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477815dad0 con 0x7f4778070220 2024-05-22T21:15:26.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.458+0000 7f476fe00640 1 --2- 172.21.15.161:0/3638616232 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4778071db0 0x7f477806fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:26.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.458+0000 7f4777e00640 1 --2- 172.21.15.161:0/3638616232 >> v2:172.21.15.179:3300/0 conn(0x7f47781596b0 0x7f4778071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:26.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.458+0000 7f477d200640 1 --2- 172.21.15.161:0/3638616232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4778070220 0x7f4778070690 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f47780724b0 tx=0x7f477001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c5e2c98e441683 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:26.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.458+0000 7f4777400640 1 -- 172.21.15.161:0/3638616232 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477000b2d0 con 0x7f4778070220 2024-05-22T21:15:26.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.458+0000 7f4777400640 1 -- 172.21.15.161:0/3638616232 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477000b430 con 0x7f4778070220 2024-05-22T21:15:26.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.458+0000 7f4777400640 1 -- 172.21.15.161:0/3638616232 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47700a7bb0 con 0x7f4778070220 2024-05-22T21:15:26.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 -- 172.21.15.161:0/3638616232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4778070220 msgr2=0x7f4778070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 --2- 172.21.15.161:0/3638616232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4778070220 0x7f4778070690 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f47780724b0 tx=0x7f477001ebf0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 -- 172.21.15.161:0/3638616232 shutdown_connections 2024-05-22T21:15:26.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 --2- 172.21.15.161:0/3638616232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4778070220 0x7f4778070690 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 --2- 172.21.15.161:0/3638616232 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4778071db0 0x7f477806fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 --2- 172.21.15.161:0/3638616232 >> v2:172.21.15.179:3300/0 conn(0x7f47781596b0 0x7f4778071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 -- 172.21.15.161:0/3638616232 >> 172.21.15.161:0/3638616232 conn(0x7f477806a2d0 msgr2=0x7f477806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:26.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 -- 172.21.15.161:0/3638616232 shutdown_connections 2024-05-22T21:15:26.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 -- 172.21.15.161:0/3638616232 wait complete. 2024-05-22T21:15:26.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 Processor -- start 2024-05-22T21:15:26.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 -- start start 2024-05-22T21:15:26.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4778070220 0x7f4778156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4778071db0 0x7f47781466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47781596b0 0x7f4778159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47781571f0 con 0x7f47781596b0 2024-05-22T21:15:26.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4778157400 con 0x7f4778071db0 2024-05-22T21:15:26.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477f200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47781576e0 con 0x7f4778070220 2024-05-22T21:15:26.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477d200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47781596b0 0x7f4778159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477d200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47781596b0 0x7f4778159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42884/0 (socket says 172.21.15.161:42884) 2024-05-22T21:15:26.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477d200640 1 -- 172.21.15.161:0/2699760403 learned_addr learned my addr 172.21.15.161:0/2699760403 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:26.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f4777e00640 1 --2- 172.21.15.161:0/2699760403 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4778070220 0x7f4778156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f476fe00640 1 --2- 172.21.15.161:0/2699760403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4778071db0 0x7f47781466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477d200640 1 -- 172.21.15.161:0/2699760403 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4778070220 msgr2=0x7f4778156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477d200640 1 --2- 172.21.15.161:0/2699760403 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4778070220 0x7f4778156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477d200640 1 -- 172.21.15.161:0/2699760403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4778071db0 msgr2=0x7f47781466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477d200640 1 --2- 172.21.15.161:0/2699760403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4778071db0 0x7f47781466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477d200640 1 -- 172.21.15.161:0/2699760403 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477000bd40 con 0x7f47781596b0 2024-05-22T21:15:26.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.462+0000 7f477d200640 1 --2- 172.21.15.161:0/2699760403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47781596b0 0x7f4778159cd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f477001d2a0 tx=0x7f477001b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:26.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.466+0000 7f4774c00640 1 -- 172.21.15.161:0/2699760403 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477000b2d0 con 0x7f47781596b0 2024-05-22T21:15:26.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.466+0000 7f4774c00640 1 -- 172.21.15.161:0/2699760403 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4770009600 con 0x7f47781596b0 2024-05-22T21:15:26.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.466+0000 7f4774c00640 1 -- 172.21.15.161:0/2699760403 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47700099d0 con 0x7f47781596b0 2024-05-22T21:15:26.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.466+0000 7f477f200640 1 -- 172.21.15.161:0/2699760403 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47781579c0 con 0x7f47781596b0 2024-05-22T21:15:26.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.466+0000 7f477f200640 1 -- 172.21.15.161:0/2699760403 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4778157f80 con 0x7f47781596b0 2024-05-22T21:15:26.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.466+0000 7f477f200640 1 -- 172.21.15.161:0/2699760403 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4778158170 con 0x7f47781596b0 2024-05-22T21:15:26.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.466+0000 7f4774c00640 1 -- 172.21.15.161:0/2699760403 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4770009b30 con 0x7f47781596b0 2024-05-22T21:15:26.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.466+0000 7f4774c00640 1 --2- 172.21.15.161:0/2699760403 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4754066690 0x7f4754068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.466+0000 7f4774c00640 1 -- 172.21.15.161:0/2699760403 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4770120090 con 0x7f47781596b0 2024-05-22T21:15:26.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.466+0000 7f4777e00640 1 --2- 172.21.15.161:0/2699760403 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4754066690 0x7f4754068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.466+0000 7f4777e00640 1 --2- 172.21.15.161:0/2699760403 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4754066690 0x7f4754068b50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f476800ba10 tx=0x7f4768008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:26.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.470+0000 7f4774c00640 1 -- 172.21.15.161:0/2699760403 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f47700ed560 con 0x7f47781596b0 2024-05-22T21:15:26.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.626+0000 7f477f200640 1 -- 172.21.15.161:0/2699760403 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4778067220 con 0x7f47781596b0 2024-05-22T21:15:26.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.626+0000 7f4774c00640 1 -- 172.21.15.161:0/2699760403 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f47700f0b80 con 0x7f47781596b0 2024-05-22T21:15:26.630 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:15:26.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.626+0000 7f477f200640 1 -- 172.21.15.161:0/2699760403 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4754066690 msgr2=0x7f4754068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.626+0000 7f477f200640 1 --2- 172.21.15.161:0/2699760403 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4754066690 0x7f4754068b50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f476800ba10 tx=0x7f4768008040 comp rx=0 tx=0).stop 2024-05-22T21:15:26.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.626+0000 7f477f200640 1 -- 172.21.15.161:0/2699760403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47781596b0 msgr2=0x7f4778159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.626+0000 7f477f200640 1 --2- 172.21.15.161:0/2699760403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47781596b0 0x7f4778159cd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f477001d2a0 tx=0x7f477001b990 comp rx=0 tx=0).stop 2024-05-22T21:15:26.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.626+0000 7f477f200640 1 -- 172.21.15.161:0/2699760403 shutdown_connections 2024-05-22T21:15:26.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.626+0000 7f477f200640 1 --2- 172.21.15.161:0/2699760403 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4754066690 0x7f4754068b50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.626+0000 7f477f200640 1 --2- 172.21.15.161:0/2699760403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47781596b0 0x7f4778159cd0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.626+0000 7f477f200640 1 --2- 172.21.15.161:0/2699760403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4778071db0 0x7f47781466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.626+0000 7f477f200640 1 --2- 172.21.15.161:0/2699760403 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4778070220 0x7f4778156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.626+0000 7f477f200640 1 -- 172.21.15.161:0/2699760403 >> 172.21.15.161:0/2699760403 conn(0x7f477806a2d0 msgr2=0x7f47780ad610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:26.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.626+0000 7f477f200640 1 -- 172.21.15.161:0/2699760403 shutdown_connections 2024-05-22T21:15:26.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.626+0000 7f477f200640 1 -- 172.21.15.161:0/2699760403 wait complete. 2024-05-22T21:15:26.652 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:15:26.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.786+0000 7fbfc0e00640 1 Processor -- start 2024-05-22T21:15:26.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.786+0000 7fbfc0e00640 1 -- start start 2024-05-22T21:15:26.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.786+0000 7fbfc0e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbfbc15b600 0x7fbfbc06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.786+0000 7fbfc0e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbfbc06c7e0 0x7fbfbc0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.786+0000 7fbfc0e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfbc06cc40 con 0x7fbfbc143c40 2024-05-22T21:15:26.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.786+0000 7fbfc0e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfbc15ba60 con 0x7fbfbc06c7e0 2024-05-22T21:15:26.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.786+0000 7fbfc0e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfbc144bb0 con 0x7fbfbc15b600 2024-05-22T21:15:26.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.786+0000 7fbfba800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbfbc06c7e0 0x7fbfbc0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.786+0000 7fbfba800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbfbc06c7e0 0x7fbfbc0b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42890/0 (socket says 172.21.15.161:42890) 2024-05-22T21:15:26.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.786+0000 7fbfba800640 1 -- 172.21.15.161:0/234384079 learned_addr learned my addr 172.21.15.161:0/234384079 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:26.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.786+0000 7fbfb9400640 1 --2- 172.21.15.161:0/234384079 >> v2:172.21.15.179:3300/0 conn(0x7fbfbc15b600 0x7fbfbc06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.786+0000 7fbfba800640 1 -- 172.21.15.161:0/234384079 >> v2:172.21.15.179:3300/0 conn(0x7fbfbc15b600 msgr2=0x7fbfbc06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:26.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.786+0000 7fbfba800640 1 --2- 172.21.15.161:0/234384079 >> v2:172.21.15.179:3300/0 conn(0x7fbfbc15b600 0x7fbfbc06c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.786+0000 7fbfba800640 1 -- 172.21.15.161:0/234384079 >> v1:172.21.15.179:6789/0 conn(0x7fbfbc143c40 legacy=0x7fbfbc1444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.786+0000 7fbfba800640 1 -- 172.21.15.161:0/234384079 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc15efb0 con 0x7fbfbc06c7e0 2024-05-22T21:15:26.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.790+0000 7fbfba800640 1 --2- 172.21.15.161:0/234384079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc06c7e0 0x7fbfbc0b2160 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0b1c80 tx=0x7fbfb401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8c6fa473ba2305 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:26.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.790+0000 7fbfb8a00640 1 -- 172.21.15.161:0/234384079 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb40a58b0 con 0x7fbfbc06c7e0 2024-05-22T21:15:26.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.790+0000 7fbfb8a00640 1 -- 172.21.15.161:0/234384079 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb40a5a10 con 0x7fbfbc06c7e0 2024-05-22T21:15:26.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.790+0000 7fbfb8a00640 1 -- 172.21.15.161:0/234384079 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb40a5da0 con 0x7fbfbc06c7e0 2024-05-22T21:15:26.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.790+0000 7fbfc0e00640 1 -- 172.21.15.161:0/234384079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc06c7e0 msgr2=0x7fbfbc0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.790+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/234384079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc06c7e0 0x7fbfbc0b2160 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0b1c80 tx=0x7fbfb401c9f0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.790+0000 7fbfc0e00640 1 -- 172.21.15.161:0/234384079 shutdown_connections 2024-05-22T21:15:26.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.790+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/234384079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc06c7e0 0x7fbfbc0b2160 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.790+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/234384079 >> v2:172.21.15.179:3300/0 conn(0x7fbfbc15b600 0x7fbfbc06c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.790+0000 7fbfc0e00640 1 -- 172.21.15.161:0/234384079 >> 172.21.15.161:0/234384079 conn(0x7fbfbc06a2d0 msgr2=0x7fbfbc06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:26.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.790+0000 7fbfc0e00640 1 -- 172.21.15.161:0/234384079 shutdown_connections 2024-05-22T21:15:26.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.790+0000 7fbfc0e00640 1 -- 172.21.15.161:0/234384079 wait complete. 2024-05-22T21:15:26.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfc0e00640 1 Processor -- start 2024-05-22T21:15:26.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfc0e00640 1 -- start start 2024-05-22T21:15:26.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfc0e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfbc06c7e0 0x7fbfbc06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfc0e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfbc143c40 0x7fbfbc139240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfc0e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc15b600 0x7fbfbc0afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfc0e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc139780 con 0x7fbfbc15b600 2024-05-22T21:15:26.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfc0e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc0b09d0 con 0x7fbfbc143c40 2024-05-22T21:15:26.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfc0e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc159310 con 0x7fbfbc06c7e0 2024-05-22T21:15:26.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfba800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc15b600 0x7fbfbc0afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfba800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc15b600 0x7fbfbc0afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42892/0 (socket says 172.21.15.161:42892) 2024-05-22T21:15:26.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfba800640 1 -- 172.21.15.161:0/4239407384 learned_addr learned my addr 172.21.15.161:0/4239407384 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:26.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfb9e00640 1 --2- 172.21.15.161:0/4239407384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfbc06c7e0 0x7fbfbc06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfb9400640 1 --2- 172.21.15.161:0/4239407384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfbc143c40 0x7fbfbc139240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfba800640 1 -- 172.21.15.161:0/4239407384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfbc06c7e0 msgr2=0x7fbfbc06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfba800640 1 --2- 172.21.15.161:0/4239407384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfbc06c7e0 0x7fbfbc06fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfba800640 1 -- 172.21.15.161:0/4239407384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfbc143c40 msgr2=0x7fbfbc139240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfba800640 1 --2- 172.21.15.161:0/4239407384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfbc143c40 0x7fbfbc139240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfba800640 1 -- 172.21.15.161:0/4239407384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb4009b30 con 0x7fbfbc15b600 2024-05-22T21:15:26.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfb9400640 1 --2- 172.21.15.161:0/4239407384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfbc143c40 0x7fbfbc139240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:26.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfba800640 1 --2- 172.21.15.161:0/4239407384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc15b600 0x7fbfbc0afd00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40af040 tx=0x7fbfb4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:26.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfb2000640 1 -- 172.21.15.161:0/4239407384 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb40b8880 con 0x7fbfbc15b600 2024-05-22T21:15:26.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfb2000640 1 -- 172.21.15.161:0/4239407384 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb40b89e0 con 0x7fbfbc15b600 2024-05-22T21:15:26.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfc0e00640 1 -- 172.21.15.161:0/4239407384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc0b06f0 con 0x7fbfbc15b600 2024-05-22T21:15:26.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfb2000640 1 -- 172.21.15.161:0/4239407384 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb40b8d90 con 0x7fbfbc15b600 2024-05-22T21:15:26.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfc0e00640 1 -- 172.21.15.161:0/4239407384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc159890 con 0x7fbfbc15b600 2024-05-22T21:15:26.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.794+0000 7fbfc0e00640 1 -- 172.21.15.161:0/4239407384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfbc159170 con 0x7fbfbc15b600 2024-05-22T21:15:26.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.798+0000 7fbfb2000640 1 -- 172.21.15.161:0/4239407384 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbfb40c3460 con 0x7fbfbc15b600 2024-05-22T21:15:26.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.798+0000 7fbfb2000640 1 --2- 172.21.15.161:0/4239407384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbf9c066690 0x7fbf9c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:26.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.798+0000 7fbfb2000640 1 -- 172.21.15.161:0/4239407384 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbfb411f740 con 0x7fbfbc15b600 2024-05-22T21:15:26.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.798+0000 7fbfb9e00640 1 --2- 172.21.15.161:0/4239407384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbf9c066690 0x7fbf9c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:26.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.798+0000 7fbfb9e00640 1 --2- 172.21.15.161:0/4239407384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbf9c066690 0x7fbf9c068b50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc1f94d0 tx=0x7fbfac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:26.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.802+0000 7fbfb2000640 1 -- 172.21.15.161:0/4239407384 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbfbc159170 con 0x7fbfbc15b600 2024-05-22T21:15:26.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.958+0000 7fbfc0e00640 1 -- 172.21.15.161:0/4239407384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbfbc159170 con 0x7fbfbc15b600 2024-05-22T21:15:26.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.958+0000 7fbfb2000640 1 -- 172.21.15.161:0/4239407384 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbfbc159170 con 0x7fbfbc15b600 2024-05-22T21:15:26.965 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:15:26.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.962+0000 7fbfc0e00640 1 -- 172.21.15.161:0/4239407384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbf9c066690 msgr2=0x7fbf9c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.962+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/4239407384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbf9c066690 0x7fbf9c068b50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc1f94d0 tx=0x7fbfac009040 comp rx=0 tx=0).stop 2024-05-22T21:15:26.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.962+0000 7fbfc0e00640 1 -- 172.21.15.161:0/4239407384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc15b600 msgr2=0x7fbfbc0afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:26.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.962+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/4239407384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc15b600 0x7fbfbc0afd00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40af040 tx=0x7fbfb4019790 comp rx=0 tx=0).stop 2024-05-22T21:15:26.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.962+0000 7fbfc0e00640 1 -- 172.21.15.161:0/4239407384 shutdown_connections 2024-05-22T21:15:26.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.962+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/4239407384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbf9c066690 0x7fbf9c068b50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.962+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/4239407384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc15b600 0x7fbfbc0afd00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.962+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/4239407384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfbc143c40 0x7fbfbc139240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.962+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/4239407384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfbc06c7e0 0x7fbfbc06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:26.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.962+0000 7fbfc0e00640 1 -- 172.21.15.161:0/4239407384 >> 172.21.15.161:0/4239407384 conn(0x7fbfbc06a2d0 msgr2=0x7fbfbc157b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:26.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.962+0000 7fbfc0e00640 1 -- 172.21.15.161:0/4239407384 shutdown_connections 2024-05-22T21:15:26.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:26.962+0000 7fbfc0e00640 1 -- 172.21.15.161:0/4239407384 wait complete. 2024-05-22T21:15:26.987 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:15:27.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.118+0000 7f938a000640 1 Processor -- start 2024-05-22T21:15:27.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.118+0000 7f938a000640 1 -- start start 2024-05-22T21:15:27.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.118+0000 7f938a000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f93841402e0 0x7f938413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.118+0000 7f938a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f938413fc80 0x7f938406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.118+0000 7f938a000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9384140090 con 0x7f938413cfc0 2024-05-22T21:15:27.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.118+0000 7f938a000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938413d3a0 con 0x7f938413fc80 2024-05-22T21:15:27.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.118+0000 7f938a000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93840cd920 con 0x7f93841402e0 2024-05-22T21:15:27.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.118+0000 7f9383400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f938413cfc0 0x7f938415af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60452/0 (socket says 172.21.15.161:60452) 2024-05-22T21:15:27.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.118+0000 7f9383400640 1 -- 172.21.15.161:0/3226928219 learned_addr learned my addr 172.21.15.161:0/3226928219 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:27.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.118+0000 7f9383e00640 1 --2- 172.21.15.161:0/3226928219 >> v2:172.21.15.161:3300/0 conn(0x7f938413fc80 0x7f938406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.122+0000 7f937aa00640 1 --2- 172.21.15.161:0/3226928219 >> v2:172.21.15.179:3300/0 conn(0x7f93841402e0 0x7f938413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.122+0000 7f9383e00640 1 -- 172.21.15.161:0/3226928219 >> v2:172.21.15.179:3300/0 conn(0x7f93841402e0 msgr2=0x7f938413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:27.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.122+0000 7f9383e00640 1 --2- 172.21.15.161:0/3226928219 >> v2:172.21.15.179:3300/0 conn(0x7f93841402e0 0x7f938413f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.122+0000 7f9383e00640 1 -- 172.21.15.161:0/3226928219 >> v1:172.21.15.161:6789/0 conn(0x7f938413cfc0 legacy=0x7f938415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:27.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.122+0000 7f9383e00640 1 -- 172.21.15.161:0/3226928219 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93840cdc00 con 0x7f938413fc80 2024-05-22T21:15:27.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.122+0000 7f9382a00640 1 -- 172.21.15.161:0/3226928219 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 201618452 0 0) 0x7f9384140090 con 0x7f938413cfc0 2024-05-22T21:15:27.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.122+0000 7f9383e00640 1 --2- 172.21.15.161:0/3226928219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f938413fc80 0x7f938406fce0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f938406e1d0 tx=0x7f937c01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2fef4df7507623 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:27.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.122+0000 7f9382a00640 1 -- 172.21.15.161:0/3226928219 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c0aa620 con 0x7f938413fc80 2024-05-22T21:15:27.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.122+0000 7f9382a00640 1 -- 172.21.15.161:0/3226928219 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937c0aa780 con 0x7f938413fc80 2024-05-22T21:15:27.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.122+0000 7f9382a00640 1 -- 172.21.15.161:0/3226928219 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c0aab10 con 0x7f938413fc80 2024-05-22T21:15:27.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.122+0000 7f938a000640 1 -- 172.21.15.161:0/3226928219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f938413fc80 msgr2=0x7f938406fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:27.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.122+0000 7f938a000640 1 --2- 172.21.15.161:0/3226928219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f938413fc80 0x7f938406fce0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f938406e1d0 tx=0x7f937c01c870 comp rx=0 tx=0).stop 2024-05-22T21:15:27.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.122+0000 7f938a000640 1 -- 172.21.15.161:0/3226928219 shutdown_connections 2024-05-22T21:15:27.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.122+0000 7f938a000640 1 --2- 172.21.15.161:0/3226928219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f938413fc80 0x7f938406fce0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.122+0000 7f938a000640 1 --2- 172.21.15.161:0/3226928219 >> v2:172.21.15.179:3300/0 conn(0x7f93841402e0 0x7f938413f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.122+0000 7f938a000640 1 -- 172.21.15.161:0/3226928219 >> 172.21.15.161:0/3226928219 conn(0x7f938406a2d0 msgr2=0x7f938406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:27.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f938a000640 1 -- 172.21.15.161:0/3226928219 shutdown_connections 2024-05-22T21:15:27.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f938a000640 1 -- 172.21.15.161:0/3226928219 wait complete. 2024-05-22T21:15:27.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f938a000640 1 Processor -- start 2024-05-22T21:15:27.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f938a000640 1 -- start start 2024-05-22T21:15:27.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f938a000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f938413cfc0 0x7f9384141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f938a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f938413fc80 0x7f93841f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f938a000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93841402e0 0x7f93841f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f938a000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938413a9f0 con 0x7f938413fc80 2024-05-22T21:15:27.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f938a000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93841f5a40 con 0x7f938413cfc0 2024-05-22T21:15:27.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f938a000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938415a370 con 0x7f93841402e0 2024-05-22T21:15:27.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f937aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f938413fc80 0x7f93841f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f937aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f938413fc80 0x7f93841f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42908/0 (socket says 172.21.15.161:42908) 2024-05-22T21:15:27.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f937aa00640 1 -- 172.21.15.161:0/1182137955 learned_addr learned my addr 172.21.15.161:0/1182137955 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:27.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f9383400640 1 --2- 172.21.15.161:0/1182137955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f938413cfc0 0x7f9384141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f9383e00640 1 --2- 172.21.15.161:0/1182137955 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93841402e0 0x7f93841f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f937aa00640 1 -- 172.21.15.161:0/1182137955 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93841402e0 msgr2=0x7f93841f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:27.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f937aa00640 1 --2- 172.21.15.161:0/1182137955 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93841402e0 0x7f93841f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f937aa00640 1 -- 172.21.15.161:0/1182137955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f938413cfc0 msgr2=0x7f9384141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:27.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f937aa00640 1 --2- 172.21.15.161:0/1182137955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f938413cfc0 0x7f9384141ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f937aa00640 1 -- 172.21.15.161:0/1182137955 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c009a70 con 0x7f938413fc80 2024-05-22T21:15:27.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f937aa00640 1 --2- 172.21.15.161:0/1182137955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f938413fc80 0x7f93841f4fc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f936c00d5f0 tx=0x7f936c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:27.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f937be00640 1 -- 172.21.15.161:0/1182137955 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c0125f0 con 0x7f938413fc80 2024-05-22T21:15:27.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f937be00640 1 -- 172.21.15.161:0/1182137955 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936c012750 con 0x7f938413fc80 2024-05-22T21:15:27.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f938a000640 1 -- 172.21.15.161:0/1182137955 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938415a650 con 0x7f938413fc80 2024-05-22T21:15:27.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.126+0000 7f937be00640 1 -- 172.21.15.161:0/1182137955 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c012ae0 con 0x7f938413fc80 2024-05-22T21:15:27.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.130+0000 7f938a000640 1 -- 172.21.15.161:0/1182137955 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938415ac10 con 0x7f938413fc80 2024-05-22T21:15:27.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.130+0000 7f938a000640 1 -- 172.21.15.161:0/1182137955 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9384067220 con 0x7f938413fc80 2024-05-22T21:15:27.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.130+0000 7f937be00640 1 -- 172.21.15.161:0/1182137955 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f936c012c40 con 0x7f938413fc80 2024-05-22T21:15:27.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.130+0000 7f937be00640 1 --2- 172.21.15.161:0/1182137955 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9360066690 0x7f9360068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.130+0000 7f937be00640 1 -- 172.21.15.161:0/1182137955 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f936c084680 con 0x7f938413fc80 2024-05-22T21:15:27.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.130+0000 7f9383400640 1 --2- 172.21.15.161:0/1182137955 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9360066690 0x7f9360068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.130+0000 7f9383400640 1 --2- 172.21.15.161:0/1182137955 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9360066690 0x7f9360068b50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9374000f70 tx=0x7f9374007b60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:27.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.134+0000 7f937be00640 1 -- 172.21.15.161:0/1182137955 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f936c0519a0 con 0x7f938413fc80 2024-05-22T21:15:27.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.298+0000 7f938a000640 1 -- 172.21.15.161:0/1182137955 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9384071d00 con 0x7f938413fc80 2024-05-22T21:15:27.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.298+0000 7f937be00640 1 -- 172.21.15.161:0/1182137955 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7f936c007380 con 0x7f938413fc80 2024-05-22T21:15:27.304 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:15:27.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.298+0000 7f938a000640 1 -- 172.21.15.161:0/1182137955 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9360066690 msgr2=0x7f9360068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:27.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.298+0000 7f938a000640 1 --2- 172.21.15.161:0/1182137955 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9360066690 0x7f9360068b50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9374000f70 tx=0x7f9374007b60 comp rx=0 tx=0).stop 2024-05-22T21:15:27.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.298+0000 7f938a000640 1 -- 172.21.15.161:0/1182137955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f938413fc80 msgr2=0x7f93841f4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:27.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.298+0000 7f938a000640 1 --2- 172.21.15.161:0/1182137955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f938413fc80 0x7f93841f4fc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f936c00d5f0 tx=0x7f936c00dab0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.302+0000 7f938a000640 1 -- 172.21.15.161:0/1182137955 shutdown_connections 2024-05-22T21:15:27.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.302+0000 7f938a000640 1 --2- 172.21.15.161:0/1182137955 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9360066690 0x7f9360068b50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.302+0000 7f938a000640 1 --2- 172.21.15.161:0/1182137955 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93841402e0 0x7f93841f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.302+0000 7f938a000640 1 --2- 172.21.15.161:0/1182137955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f938413fc80 0x7f93841f4fc0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.302+0000 7f938a000640 1 --2- 172.21.15.161:0/1182137955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f938413cfc0 0x7f9384141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.302+0000 7f938a000640 1 -- 172.21.15.161:0/1182137955 >> 172.21.15.161:0/1182137955 conn(0x7f938406a2d0 msgr2=0x7f938406a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:27.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.302+0000 7f938a000640 1 -- 172.21.15.161:0/1182137955 shutdown_connections 2024-05-22T21:15:27.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.302+0000 7f938a000640 1 -- 172.21.15.161:0/1182137955 wait complete. 2024-05-22T21:15:27.326 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-22T21:15:27.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:27.326 DEBUG:teuthology.orchestra.run.smithi179:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-22T21:15:27.355 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:15:27.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:27.355 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-22T21:15:27.408 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:15:27.408 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:15:27.417 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:15:27.417 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:15:27.417 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:15:27.461 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:15:27.461 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:15:27.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.590+0000 7f0b79c00640 1 Processor -- start 2024-05-22T21:15:27.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.590+0000 7f0b79c00640 1 -- start start 2024-05-22T21:15:27.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.594+0000 7f0b79c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0b74143ec0 0x7f0b7415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.594+0000 7f0b79c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b740715a0 0x7f0b7406fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.594+0000 7f0b79c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b741442a0 con 0x7f0b7415ec90 2024-05-22T21:15:27.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.594+0000 7f0b79c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b74070260 con 0x7f0b74143ec0 2024-05-22T21:15:27.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.594+0000 7f0b79c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b74070500 con 0x7f0b740715a0 2024-05-22T21:15:27.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.594+0000 7f0b72a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0b74143ec0 0x7f0b7415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.594+0000 7f0b72a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0b74143ec0 0x7f0b7415f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42924/0 (socket says 172.21.15.161:42924) 2024-05-22T21:15:27.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.594+0000 7f0b72a00640 1 -- 172.21.15.161:0/3785357787 learned_addr learned my addr 172.21.15.161:0/3785357787 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:27.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.594+0000 7f0b6be00640 1 --2- 172.21.15.161:0/3785357787 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b740715a0 0x7f0b7406fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.594+0000 7f0b72a00640 1 -- 172.21.15.161:0/3785357787 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b740715a0 msgr2=0x7f0b7406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:27.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.594+0000 7f0b72a00640 1 --2- 172.21.15.161:0/3785357787 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b740715a0 0x7f0b7406fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.594+0000 7f0b72a00640 1 -- 172.21.15.161:0/3785357787 >> v1:172.21.15.179:6789/0 conn(0x7f0b7415ec90 legacy=0x7f0b7406bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:27.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.594+0000 7f0b72a00640 1 -- 172.21.15.161:0/3785357787 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b740707e0 con 0x7f0b74143ec0 2024-05-22T21:15:27.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.594+0000 7f0b6be00640 1 --2- 172.21.15.161:0/3785357787 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b740715a0 0x7f0b7406fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:27.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.598+0000 7f0b72a00640 1 --2- 172.21.15.161:0/3785357787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b74143ec0 0x7f0b7415f830 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009870 tx=0x7f0b6401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985b36b5b75f5a6f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:27.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.598+0000 7f0b72000640 1 -- 172.21.15.161:0/3785357787 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b640aa610 con 0x7f0b74143ec0 2024-05-22T21:15:27.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.598+0000 7f0b72000640 1 -- 172.21.15.161:0/3785357787 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b640a6040 con 0x7f0b74143ec0 2024-05-22T21:15:27.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.598+0000 7f0b72000640 1 -- 172.21.15.161:0/3785357787 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b640aa990 con 0x7f0b74143ec0 2024-05-22T21:15:27.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.598+0000 7f0b79c00640 1 -- 172.21.15.161:0/3785357787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b74143ec0 msgr2=0x7f0b7415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:27.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.598+0000 7f0b79c00640 1 --2- 172.21.15.161:0/3785357787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b74143ec0 0x7f0b7415f830 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009870 tx=0x7f0b6401c4f0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.598+0000 7f0b79c00640 1 -- 172.21.15.161:0/3785357787 shutdown_connections 2024-05-22T21:15:27.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.598+0000 7f0b79c00640 1 --2- 172.21.15.161:0/3785357787 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b740715a0 0x7f0b7406fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.598+0000 7f0b79c00640 1 --2- 172.21.15.161:0/3785357787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b74143ec0 0x7f0b7415f830 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.598+0000 7f0b79c00640 1 -- 172.21.15.161:0/3785357787 >> 172.21.15.161:0/3785357787 conn(0x7f0b740b0bc0 msgr2=0x7f0b740b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:27.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.598+0000 7f0b79c00640 1 -- 172.21.15.161:0/3785357787 shutdown_connections 2024-05-22T21:15:27.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.598+0000 7f0b79c00640 1 -- 172.21.15.161:0/3785357787 wait complete. 2024-05-22T21:15:27.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.598+0000 7f0b79c00640 1 Processor -- start 2024-05-22T21:15:27.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.598+0000 7f0b79c00640 1 -- start start 2024-05-22T21:15:27.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b79c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b74143ec0 0x7f0b741f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b79c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b7415ec90 0x7f0b741f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b79c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b74160d10 0x7f0b74163eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b79c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b741fa3b0 con 0x7f0b74160d10 2024-05-22T21:15:27.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b79c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7415fc70 con 0x7f0b7415ec90 2024-05-22T21:15:27.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b79c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7415ff50 con 0x7f0b74143ec0 2024-05-22T21:15:27.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b73400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b74160d10 0x7f0b74163eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b73400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b74160d10 0x7f0b74163eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42926/0 (socket says 172.21.15.161:42926) 2024-05-22T21:15:27.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b73400640 1 -- 172.21.15.161:0/530840084 learned_addr learned my addr 172.21.15.161:0/530840084 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:27.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b6be00640 1 --2- 172.21.15.161:0/530840084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b7415ec90 0x7f0b741f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b72a00640 1 --2- 172.21.15.161:0/530840084 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b74143ec0 0x7f0b741f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b73400640 1 -- 172.21.15.161:0/530840084 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b74143ec0 msgr2=0x7f0b741f9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:27.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b73400640 1 --2- 172.21.15.161:0/530840084 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b74143ec0 0x7f0b741f9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b73400640 1 -- 172.21.15.161:0/530840084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b7415ec90 msgr2=0x7f0b741f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:27.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b73400640 1 --2- 172.21.15.161:0/530840084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b7415ec90 0x7f0b741f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b73400640 1 -- 172.21.15.161:0/530840084 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b640094d0 con 0x7f0b74160d10 2024-05-22T21:15:27.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b6be00640 1 --2- 172.21.15.161:0/530840084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b7415ec90 0x7f0b741f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:27.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b73400640 1 --2- 172.21.15.161:0/530840084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b74160d10 0x7f0b74163eb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00b970 tx=0x7f0b6c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:27.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b6aa00640 1 -- 172.21.15.161:0/530840084 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c00cb40 con 0x7f0b74160d10 2024-05-22T21:15:27.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b6aa00640 1 -- 172.21.15.161:0/530840084 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6c00cca0 con 0x7f0b74160d10 2024-05-22T21:15:27.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b6aa00640 1 -- 172.21.15.161:0/530840084 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c014680 con 0x7f0b74160d10 2024-05-22T21:15:27.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b79c00640 1 -- 172.21.15.161:0/530840084 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b74160230 con 0x7f0b74160d10 2024-05-22T21:15:27.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b79c00640 1 -- 172.21.15.161:0/530840084 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b741607f0 con 0x7f0b74160d10 2024-05-22T21:15:27.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b79c00640 1 -- 172.21.15.161:0/530840084 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b74160a10 con 0x7f0b74160d10 2024-05-22T21:15:27.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b6aa00640 1 -- 172.21.15.161:0/530840084 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0b6c01d3b0 con 0x7f0b74160d10 2024-05-22T21:15:27.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.602+0000 7f0b6aa00640 1 --2- 172.21.15.161:0/530840084 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b50066690 0x7f0b50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.606+0000 7f0b6aa00640 1 -- 172.21.15.161:0/530840084 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0b6c083fe0 con 0x7f0b74160d10 2024-05-22T21:15:27.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.606+0000 7f0b72a00640 1 --2- 172.21.15.161:0/530840084 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b50066690 0x7f0b50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.606+0000 7f0b72a00640 1 --2- 172.21.15.161:0/530840084 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b50066690 0x7f0b50068b50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0b640ae000 tx=0x7f0b64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:27.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.610+0000 7f0b6aa00640 1 -- 172.21.15.161:0/530840084 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0b6c051340 con 0x7f0b74160d10 2024-05-22T21:15:27.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.794+0000 7f0b79c00640 1 -- 172.21.15.161:0/530840084 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b74067220 con 0x7f0b74160d10 2024-05-22T21:15:27.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.794+0000 7f0b6aa00640 1 -- 172.21.15.161:0/530840084 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f0b6c054960 con 0x7f0b74160d10 2024-05-22T21:15:27.802 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:27.802 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/1767934947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1767934947},{"type":"v1","addr":"172.21.15.161:6836","nonce":1767934947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1873337287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1873337287},{"type":"v1","addr":"172.21.15.179:6834","nonce":1873337287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1815838236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":1815838236},{"type":"v1","addr":"172.21.15.161:6837","nonce":1815838236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:50.052292+0000","modified":"2024-05-22T21:14:53.058881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5374":{"gid":5374,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/389418050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":389418050},{"type":"v1","addr":"172.21.15.179:6835","nonce":389418050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-22T21:15:27.802 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 23 2024-05-22T21:15:27.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.798+0000 7f0b79c00640 1 -- 172.21.15.161:0/530840084 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b50066690 msgr2=0x7f0b50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:27.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.798+0000 7f0b79c00640 1 --2- 172.21.15.161:0/530840084 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b50066690 0x7f0b50068b50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0b640ae000 tx=0x7f0b64008040 comp rx=0 tx=0).stop 2024-05-22T21:15:27.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.798+0000 7f0b79c00640 1 -- 172.21.15.161:0/530840084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b74160d10 msgr2=0x7f0b74163eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:27.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.798+0000 7f0b79c00640 1 --2- 172.21.15.161:0/530840084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b74160d10 0x7f0b74163eb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00b970 tx=0x7f0b6c00be30 comp rx=0 tx=0).stop 2024-05-22T21:15:27.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.798+0000 7f0b79c00640 1 -- 172.21.15.161:0/530840084 shutdown_connections 2024-05-22T21:15:27.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.798+0000 7f0b79c00640 1 --2- 172.21.15.161:0/530840084 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b50066690 0x7f0b50068b50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.798+0000 7f0b79c00640 1 --2- 172.21.15.161:0/530840084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b74160d10 0x7f0b74163eb0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.798+0000 7f0b79c00640 1 --2- 172.21.15.161:0/530840084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b7415ec90 0x7f0b741f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.798+0000 7f0b79c00640 1 --2- 172.21.15.161:0/530840084 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0b74143ec0 0x7f0b741f9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.798+0000 7f0b79c00640 1 -- 172.21.15.161:0/530840084 >> 172.21.15.161:0/530840084 conn(0x7f0b740b0bc0 msgr2=0x7f0b740b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:27.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.798+0000 7f0b79c00640 1 -- 172.21.15.161:0/530840084 shutdown_connections 2024-05-22T21:15:27.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.798+0000 7f0b79c00640 1 -- 172.21.15.161:0/530840084 wait complete. 2024-05-22T21:15:27.823 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:15:27.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f32a5000640 1 Processor -- start 2024-05-22T21:15:27.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f32a5000640 1 -- start start 2024-05-22T21:15:27.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f32a5000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f32a0159f80 0x7f32a006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f32a5000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f32a00cf450 0x7f32a00b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f32a5000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a015db40 con 0x7f32a00b3710 2024-05-22T21:15:27.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f32a5000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a0070eb0 con 0x7f32a0159f80 2024-05-22T21:15:27.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f32a5000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a0071190 con 0x7f32a00cf450 2024-05-22T21:15:27.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f329fe00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f32a00cf450 0x7f32a00b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f329fe00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f32a00cf450 0x7f32a00b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:60218/0 (socket says 172.21.15.161:60218) 2024-05-22T21:15:27.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f329fe00640 1 -- 172.21.15.161:0/2590938522 learned_addr learned my addr 172.21.15.161:0/2590938522 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:27.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f329f400640 1 --2- 172.21.15.161:0/2590938522 >> v2:172.21.15.161:3300/0 conn(0x7f32a0159f80 0x7f32a006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f329fe00640 1 -- 172.21.15.161:0/2590938522 >> v2:172.21.15.161:3300/0 conn(0x7f32a0159f80 msgr2=0x7f32a006cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:27.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f329fe00640 1 --2- 172.21.15.161:0/2590938522 >> v2:172.21.15.161:3300/0 conn(0x7f32a0159f80 0x7f32a006cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f329fe00640 1 -- 172.21.15.161:0/2590938522 >> v1:172.21.15.179:6789/0 conn(0x7f32a00b3710 legacy=0x7f32a00ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:27.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f329fe00640 1 -- 172.21.15.161:0/2590938522 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a0071470 con 0x7f32a00cf450 2024-05-22T21:15:27.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f329ea00640 1 -- 172.21.15.161:0/2590938522 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509876006 0 0) 0x7f32a015db40 con 0x7f32a00b3710 2024-05-22T21:15:27.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.954+0000 7f329f400640 1 --2- 172.21.15.161:0/2590938522 >> v2:172.21.15.161:3300/0 conn(0x7f32a0159f80 0x7f32a006cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:15:27.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.958+0000 7f329fe00640 1 --2- 172.21.15.161:0/2590938522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32a00cf450 0x7f32a00b3b60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f32a00b35c0 tx=0x7f329801f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f348d3a5975eaa06 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:27.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.958+0000 7f329ea00640 1 -- 172.21.15.161:0/2590938522 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32980af400 con 0x7f32a00cf450 2024-05-22T21:15:27.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.958+0000 7f329ea00640 1 -- 172.21.15.161:0/2590938522 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32980ab050 con 0x7f32a00cf450 2024-05-22T21:15:27.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.958+0000 7f329ea00640 1 -- 172.21.15.161:0/2590938522 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32980af780 con 0x7f32a00cf450 2024-05-22T21:15:27.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.958+0000 7f32a5000640 1 -- 172.21.15.161:0/2590938522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32a00cf450 msgr2=0x7f32a00b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:27.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.958+0000 7f32a5000640 1 --2- 172.21.15.161:0/2590938522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32a00cf450 0x7f32a00b3b60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f32a00b35c0 tx=0x7f329801f1e0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.958+0000 7f32a5000640 1 -- 172.21.15.161:0/2590938522 shutdown_connections 2024-05-22T21:15:27.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.958+0000 7f32a5000640 1 --2- 172.21.15.161:0/2590938522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32a00cf450 0x7f32a00b3b60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.958+0000 7f32a5000640 1 --2- 172.21.15.161:0/2590938522 >> v2:172.21.15.161:3300/0 conn(0x7f32a0159f80 0x7f32a006cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.958+0000 7f32a5000640 1 -- 172.21.15.161:0/2590938522 >> 172.21.15.161:0/2590938522 conn(0x7f32a00cfc20 msgr2=0x7f32a00d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:27.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.958+0000 7f32a5000640 1 -- 172.21.15.161:0/2590938522 shutdown_connections 2024-05-22T21:15:27.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.958+0000 7f32a5000640 1 -- 172.21.15.161:0/2590938522 wait complete. 2024-05-22T21:15:27.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f32a5000640 1 Processor -- start 2024-05-22T21:15:27.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f32a5000640 1 -- start start 2024-05-22T21:15:27.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f32a5000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32a00b3710 0x7f32a006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f32a5000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32a00cf450 0x7f32a014d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f32a5000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a0159f80 0x7f32a0150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f32a5000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a0150750 con 0x7f32a0159f80 2024-05-22T21:15:27.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f32a5000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a006df30 con 0x7f32a00b3710 2024-05-22T21:15:27.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f32a5000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a006d600 con 0x7f32a00cf450 2024-05-22T21:15:27.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f329fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a0159f80 0x7f32a0150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f329fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a0159f80 0x7f32a0150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42948/0 (socket says 172.21.15.161:42948) 2024-05-22T21:15:27.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f329fe00640 1 -- 172.21.15.161:0/2463764970 learned_addr learned my addr 172.21.15.161:0/2463764970 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:27.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f329f400640 1 --2- 172.21.15.161:0/2463764970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32a00b3710 0x7f32a006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f3296a00640 1 --2- 172.21.15.161:0/2463764970 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32a00cf450 0x7f32a014d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f329fe00640 1 -- 172.21.15.161:0/2463764970 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32a00cf450 msgr2=0x7f32a014d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:27.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f329fe00640 1 --2- 172.21.15.161:0/2463764970 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32a00cf450 0x7f32a014d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f329fe00640 1 -- 172.21.15.161:0/2463764970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32a00b3710 msgr2=0x7f32a006e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:27.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f329fe00640 1 --2- 172.21.15.161:0/2463764970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32a00b3710 0x7f32a006e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:27.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f329fe00640 1 -- 172.21.15.161:0/2463764970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3290009420 con 0x7f32a0159f80 2024-05-22T21:15:27.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f329fe00640 1 --2- 172.21.15.161:0/2463764970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a0159f80 0x7f32a0150210 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f32980b2040 tx=0x7f329801bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:27.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f3297e00640 1 -- 172.21.15.161:0/2463764970 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32980afd30 con 0x7f32a0159f80 2024-05-22T21:15:27.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f3297e00640 1 -- 172.21.15.161:0/2463764970 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3298009510 con 0x7f32a0159f80 2024-05-22T21:15:27.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f3297e00640 1 -- 172.21.15.161:0/2463764970 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32980ba660 con 0x7f32a0159f80 2024-05-22T21:15:27.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f32a5000640 1 -- 172.21.15.161:0/2463764970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3298009b30 con 0x7f32a0159f80 2024-05-22T21:15:27.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f32a5000640 1 -- 172.21.15.161:0/2463764970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a006dbc0 con 0x7f32a0159f80 2024-05-22T21:15:27.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.962+0000 7f32a5000640 1 -- 172.21.15.161:0/2463764970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a00746c0 con 0x7f32a0159f80 2024-05-22T21:15:27.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.966+0000 7f3297e00640 1 -- 172.21.15.161:0/2463764970 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f32980b6040 con 0x7f32a0159f80 2024-05-22T21:15:27.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.966+0000 7f3297e00640 1 --2- 172.21.15.161:0/2463764970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f327c066690 0x7f327c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:27.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.966+0000 7f3297e00640 1 -- 172.21.15.161:0/2463764970 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3298122440 con 0x7f32a0159f80 2024-05-22T21:15:27.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.966+0000 7f329f400640 1 --2- 172.21.15.161:0/2463764970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f327c066690 0x7f327c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:27.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.966+0000 7f329f400640 1 --2- 172.21.15.161:0/2463764970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f327c066690 0x7f327c068b50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3290009c80 tx=0x7f3290008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:27.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:27.970+0000 7f3297e00640 1 -- 172.21.15.161:0/2463764970 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f32980ef7f0 con 0x7f32a0159f80 2024-05-22T21:15:28.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.122+0000 7f32a5000640 1 -- 172.21.15.161:0/2463764970 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32a006e0a0 con 0x7f32a0159f80 2024-05-22T21:15:28.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.122+0000 7f3297e00640 1 -- 172.21.15.161:0/2463764970 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16183 (secure 0 0 0) 0x7f32980f2e10 con 0x7f32a0159f80 2024-05-22T21:15:28.128 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:28.128 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":55,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:15:17.769605+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:48.612726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:49.053022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.9","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:15:28.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.126+0000 7f32a5000640 1 -- 172.21.15.161:0/2463764970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f327c066690 msgr2=0x7f327c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.126+0000 7f32a5000640 1 --2- 172.21.15.161:0/2463764970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f327c066690 0x7f327c068b50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3290009c80 tx=0x7f3290008040 comp rx=0 tx=0).stop 2024-05-22T21:15:28.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.126+0000 7f32a5000640 1 -- 172.21.15.161:0/2463764970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a0159f80 msgr2=0x7f32a0150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.126+0000 7f32a5000640 1 --2- 172.21.15.161:0/2463764970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a0159f80 0x7f32a0150210 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f32980b2040 tx=0x7f329801bf80 comp rx=0 tx=0).stop 2024-05-22T21:15:28.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.126+0000 7f32a5000640 1 -- 172.21.15.161:0/2463764970 shutdown_connections 2024-05-22T21:15:28.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.126+0000 7f32a5000640 1 --2- 172.21.15.161:0/2463764970 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f327c066690 0x7f327c068b50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.126+0000 7f32a5000640 1 --2- 172.21.15.161:0/2463764970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32a0159f80 0x7f32a0150210 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.126+0000 7f32a5000640 1 --2- 172.21.15.161:0/2463764970 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32a00cf450 0x7f32a014d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.126+0000 7f32a5000640 1 --2- 172.21.15.161:0/2463764970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32a00b3710 0x7f32a006e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.126+0000 7f32a5000640 1 -- 172.21.15.161:0/2463764970 >> 172.21.15.161:0/2463764970 conn(0x7f32a00cfc20 msgr2=0x7f32a0157a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:28.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.126+0000 7f32a5000640 1 -- 172.21.15.161:0/2463764970 shutdown_connections 2024-05-22T21:15:28.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.126+0000 7f32a5000640 1 -- 172.21.15.161:0/2463764970 wait complete. 2024-05-22T21:15:28.150 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:14:48.612726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:14:49.053022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:15:28.150 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:15:28.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.282+0000 7f1c47a00640 1 Processor -- start 2024-05-22T21:15:28.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.282+0000 7f1c47a00640 1 -- start start 2024-05-22T21:15:28.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.282+0000 7f1c47a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1c4013cfc0 0x7f1c401402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.282+0000 7f1c47a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c4013d3a0 con 0x7f1c4013fe50 2024-05-22T21:15:28.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.282+0000 7f1c47a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c400cd920 con 0x7f1c4015af60 2024-05-22T21:15:28.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.282+0000 7f1c47a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c400cdc00 con 0x7f1c4013cfc0 2024-05-22T21:15:28.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.286+0000 7f1c45a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1c4013fe50 0x7f1c4006fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60458/0 (socket says 172.21.15.161:60458) 2024-05-22T21:15:28.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.286+0000 7f1c45a00640 1 -- 172.21.15.161:0/2222405716 learned_addr learned my addr 172.21.15.161:0/2222405716 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:28.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.286+0000 7f1c45000640 1 --2- 172.21.15.161:0/2222405716 >> v2:172.21.15.179:3300/0 conn(0x7f1c4013cfc0 0x7f1c401402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.286+0000 7f1c3fe00640 1 -- 172.21.15.161:0/2222405716 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824995245 0 0) 0x7f1c4013d3a0 con 0x7f1c4013fe50 2024-05-22T21:15:28.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.286+0000 7f1c3fe00640 1 -- 172.21.15.161:0/2222405716 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c24003530 con 0x7f1c4013fe50 2024-05-22T21:15:28.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.286+0000 7f1c45000640 1 -- 172.21.15.161:0/2222405716 >> v1:172.21.15.179:6789/0 conn(0x7f1c4015af60 legacy=0x7f1c4013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.286+0000 7f1c45000640 1 -- 172.21.15.161:0/2222405716 >> v1:172.21.15.161:6789/0 conn(0x7f1c4013fe50 legacy=0x7f1c4006fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.286+0000 7f1c45000640 1 -- 172.21.15.161:0/2222405716 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c400cdee0 con 0x7f1c4013cfc0 2024-05-22T21:15:28.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.286+0000 7f1c45000640 1 --2- 172.21.15.161:0/2222405716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c4013cfc0 0x7f1c401402e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1c300097c0 tx=0x7f1c3001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51fcc60c0c6abe2a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:28.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c3fe00640 1 -- 172.21.15.161:0/2222405716 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c300ac440 con 0x7f1c4013cfc0 2024-05-22T21:15:28.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c3fe00640 1 -- 172.21.15.161:0/2222405716 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c300ac5a0 con 0x7f1c4013cfc0 2024-05-22T21:15:28.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c3fe00640 1 -- 172.21.15.161:0/2222405716 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c300ac930 con 0x7f1c4013cfc0 2024-05-22T21:15:28.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c47a00640 1 -- 172.21.15.161:0/2222405716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c4013cfc0 msgr2=0x7f1c401402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c47a00640 1 --2- 172.21.15.161:0/2222405716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c4013cfc0 0x7f1c401402e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1c300097c0 tx=0x7f1c3001c3e0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c47a00640 1 -- 172.21.15.161:0/2222405716 shutdown_connections 2024-05-22T21:15:28.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c47a00640 1 --2- 172.21.15.161:0/2222405716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c4013cfc0 0x7f1c401402e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c47a00640 1 -- 172.21.15.161:0/2222405716 >> 172.21.15.161:0/2222405716 conn(0x7f1c4006a2d0 msgr2=0x7f1c4006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:28.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c47a00640 1 -- 172.21.15.161:0/2222405716 shutdown_connections 2024-05-22T21:15:28.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c47a00640 1 -- 172.21.15.161:0/2222405716 wait complete. 2024-05-22T21:15:28.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c47a00640 1 Processor -- start 2024-05-22T21:15:28.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c47a00640 1 -- start start 2024-05-22T21:15:28.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c47a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c4013cfc0 0x7f1c4015c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c47a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c4013fe50 0x7f1c401388c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c47a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c4015af60 0x7f1c4013a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c47a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c40139320 con 0x7f1c4015af60 2024-05-22T21:15:28.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c47a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c40158f80 con 0x7f1c4013cfc0 2024-05-22T21:15:28.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c47a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c40159240 con 0x7f1c4013fe50 2024-05-22T21:15:28.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c45a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c4015af60 0x7f1c4013a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c45a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c4015af60 0x7f1c4013a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42954/0 (socket says 172.21.15.161:42954) 2024-05-22T21:15:28.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c45a00640 1 -- 172.21.15.161:0/3334250939 learned_addr learned my addr 172.21.15.161:0/3334250939 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:28.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c37e00640 1 --2- 172.21.15.161:0/3334250939 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c4013fe50 0x7f1c401388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.290+0000 7f1c45000640 1 --2- 172.21.15.161:0/3334250939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c4013cfc0 0x7f1c4015c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.294+0000 7f1c45a00640 1 -- 172.21.15.161:0/3334250939 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c4013fe50 msgr2=0x7f1c401388c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.294+0000 7f1c45a00640 1 --2- 172.21.15.161:0/3334250939 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c4013fe50 0x7f1c401388c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.294+0000 7f1c45a00640 1 -- 172.21.15.161:0/3334250939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c4013cfc0 msgr2=0x7f1c4015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.294+0000 7f1c45a00640 1 --2- 172.21.15.161:0/3334250939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c4013cfc0 0x7f1c4015c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.294+0000 7f1c45a00640 1 -- 172.21.15.161:0/3334250939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c30009420 con 0x7f1c4015af60 2024-05-22T21:15:28.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.294+0000 7f1c45000640 1 --2- 172.21.15.161:0/3334250939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c4013cfc0 0x7f1c4015c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:28.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.294+0000 7f1c45a00640 1 --2- 172.21.15.161:0/3334250939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c4015af60 0x7f1c4013a840 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800cb40 tx=0x7f1c3800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:28.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.294+0000 7f1c3d600640 1 -- 172.21.15.161:0/3334250939 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3800aca0 con 0x7f1c4015af60 2024-05-22T21:15:28.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.294+0000 7f1c3d600640 1 -- 172.21.15.161:0/3334250939 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3800ae00 con 0x7f1c4015af60 2024-05-22T21:15:28.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.294+0000 7f1c3d600640 1 -- 172.21.15.161:0/3334250939 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38012620 con 0x7f1c4015af60 2024-05-22T21:15:28.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.294+0000 7f1c47a00640 1 -- 172.21.15.161:0/3334250939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c40159520 con 0x7f1c4015af60 2024-05-22T21:15:28.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.294+0000 7f1c47a00640 1 -- 172.21.15.161:0/3334250939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c40157900 con 0x7f1c4015af60 2024-05-22T21:15:28.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.294+0000 7f1c47a00640 1 -- 172.21.15.161:0/3334250939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4013d5b0 con 0x7f1c4015af60 2024-05-22T21:15:28.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.294+0000 7f1c3d600640 1 -- 172.21.15.161:0/3334250939 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1c38007070 con 0x7f1c4015af60 2024-05-22T21:15:28.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.298+0000 7f1c3d600640 1 --2- 172.21.15.161:0/3334250939 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1c1c066690 0x7f1c1c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.298+0000 7f1c3d600640 1 -- 172.21.15.161:0/3334250939 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1c380842a0 con 0x7f1c4015af60 2024-05-22T21:15:28.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.298+0000 7f1c45000640 1 --2- 172.21.15.161:0/3334250939 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1c1c066690 0x7f1c1c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.298+0000 7f1c45000640 1 --2- 172.21.15.161:0/3334250939 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1c1c066690 0x7f1c1c068b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1c300ae000 tx=0x7f1c30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:28.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.302+0000 7f1c3d600640 1 -- 172.21.15.161:0/3334250939 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1c4013d5b0 con 0x7f1c4015af60 2024-05-22T21:15:28.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.454+0000 7f1c47a00640 1 -- 172.21.15.161:0/3334250939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c4013d5b0 con 0x7f1c4015af60 2024-05-22T21:15:28.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.454+0000 7f1c3d600640 1 -- 172.21.15.161:0/3334250939 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c4013d5b0 con 0x7f1c4015af60 2024-05-22T21:15:28.461 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:15:28.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.458+0000 7f1c47a00640 1 -- 172.21.15.161:0/3334250939 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1c1c066690 msgr2=0x7f1c1c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.458+0000 7f1c47a00640 1 --2- 172.21.15.161:0/3334250939 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1c1c066690 0x7f1c1c068b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1c300ae000 tx=0x7f1c30008040 comp rx=0 tx=0).stop 2024-05-22T21:15:28.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.458+0000 7f1c47a00640 1 -- 172.21.15.161:0/3334250939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c4015af60 msgr2=0x7f1c4013a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.458+0000 7f1c47a00640 1 --2- 172.21.15.161:0/3334250939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c4015af60 0x7f1c4013a840 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800cb40 tx=0x7f1c3800a060 comp rx=0 tx=0).stop 2024-05-22T21:15:28.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.458+0000 7f1c47a00640 1 -- 172.21.15.161:0/3334250939 shutdown_connections 2024-05-22T21:15:28.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.458+0000 7f1c47a00640 1 --2- 172.21.15.161:0/3334250939 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1c1c066690 0x7f1c1c068b50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.458+0000 7f1c47a00640 1 --2- 172.21.15.161:0/3334250939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c4015af60 0x7f1c4013a840 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.458+0000 7f1c47a00640 1 --2- 172.21.15.161:0/3334250939 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c4013fe50 0x7f1c401388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.458+0000 7f1c47a00640 1 --2- 172.21.15.161:0/3334250939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c4013cfc0 0x7f1c4015c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.458+0000 7f1c47a00640 1 -- 172.21.15.161:0/3334250939 >> 172.21.15.161:0/3334250939 conn(0x7f1c4006a2d0 msgr2=0x7f1c4006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:28.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.458+0000 7f1c47a00640 1 -- 172.21.15.161:0/3334250939 shutdown_connections 2024-05-22T21:15:28.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.458+0000 7f1c47a00640 1 -- 172.21.15.161:0/3334250939 wait complete. 2024-05-22T21:15:28.482 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:15:28.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.614+0000 7f9594c00640 1 Processor -- start 2024-05-22T21:15:28.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.614+0000 7f9594c00640 1 -- start start 2024-05-22T21:15:28.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.618+0000 7f9594c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f959013cfc0 0x7f95901402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.618+0000 7f9594c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f959015af60 0x7f959013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.618+0000 7f9594c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9590140090 con 0x7f959013fc80 2024-05-22T21:15:28.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.618+0000 7f9594c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f959013d3a0 con 0x7f959013cfc0 2024-05-22T21:15:28.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.618+0000 7f9594c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95900cd920 con 0x7f959015af60 2024-05-22T21:15:28.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.618+0000 7f958e800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f959013fc80 0x7f959015dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60460/0 (socket says 172.21.15.161:60460) 2024-05-22T21:15:28.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.618+0000 7f958e800640 1 -- 172.21.15.161:0/4286742572 learned_addr learned my addr 172.21.15.161:0/4286742572 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:28.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.618+0000 7f958de00640 1 --2- 172.21.15.161:0/4286742572 >> v2:172.21.15.161:3300/0 conn(0x7f959013cfc0 0x7f95901402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.618+0000 7f9585400640 1 --2- 172.21.15.161:0/4286742572 >> v2:172.21.15.179:3300/0 conn(0x7f959015af60 0x7f959013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.618+0000 7f958d400640 1 -- 172.21.15.161:0/4286742572 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937654724 0 0) 0x7f9590140090 con 0x7f959013fc80 2024-05-22T21:15:28.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.618+0000 7f958d400640 1 -- 172.21.15.161:0/4286742572 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9574003530 con 0x7f959013fc80 2024-05-22T21:15:28.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.618+0000 7f958de00640 1 -- 172.21.15.161:0/4286742572 >> v2:172.21.15.179:3300/0 conn(0x7f959015af60 msgr2=0x7f959013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:28.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.618+0000 7f958de00640 1 --2- 172.21.15.161:0/4286742572 >> v2:172.21.15.179:3300/0 conn(0x7f959015af60 0x7f959013f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.618+0000 7f958de00640 1 -- 172.21.15.161:0/4286742572 >> v1:172.21.15.161:6789/0 conn(0x7f959013fc80 legacy=0x7f959015dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.618+0000 7f958de00640 1 -- 172.21.15.161:0/4286742572 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95900cdc00 con 0x7f959013cfc0 2024-05-22T21:15:28.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.622+0000 7f958de00640 1 --2- 172.21.15.161:0/4286742572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959013cfc0 0x7f95901402e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f959006e1d0 tx=0x7f958001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17d163b308c577c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:28.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.622+0000 7f958d400640 1 -- 172.21.15.161:0/4286742572 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95800ab600 con 0x7f959013cfc0 2024-05-22T21:15:28.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.622+0000 7f958d400640 1 -- 172.21.15.161:0/4286742572 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95800a7050 con 0x7f959013cfc0 2024-05-22T21:15:28.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.622+0000 7f958d400640 1 -- 172.21.15.161:0/4286742572 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95800ab960 con 0x7f959013cfc0 2024-05-22T21:15:28.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.622+0000 7f9594c00640 1 -- 172.21.15.161:0/4286742572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959013cfc0 msgr2=0x7f95901402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.622+0000 7f9594c00640 1 --2- 172.21.15.161:0/4286742572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959013cfc0 0x7f95901402e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f959006e1d0 tx=0x7f958001c3f0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.622+0000 7f9594c00640 1 -- 172.21.15.161:0/4286742572 shutdown_connections 2024-05-22T21:15:28.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.622+0000 7f9594c00640 1 --2- 172.21.15.161:0/4286742572 >> v2:172.21.15.179:3300/0 conn(0x7f959015af60 0x7f959013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.622+0000 7f9594c00640 1 --2- 172.21.15.161:0/4286742572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959013cfc0 0x7f95901402e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.622+0000 7f9594c00640 1 -- 172.21.15.161:0/4286742572 >> 172.21.15.161:0/4286742572 conn(0x7f959006a2d0 msgr2=0x7f959006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:28.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.622+0000 7f9594c00640 1 -- 172.21.15.161:0/4286742572 shutdown_connections 2024-05-22T21:15:28.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.622+0000 7f9594c00640 1 -- 172.21.15.161:0/4286742572 wait complete. 2024-05-22T21:15:28.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.622+0000 7f9594c00640 1 Processor -- start 2024-05-22T21:15:28.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.622+0000 7f9594c00640 1 -- start start 2024-05-22T21:15:28.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f9594c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f959013cfc0 0x7f959013c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f9594c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f959013fc80 0x7f95901646b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f9594c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959015af60 0x7f9590140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f9594c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959013cc90 con 0x7f959015af60 2024-05-22T21:15:28.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f9594c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9590164bf0 con 0x7f959013fc80 2024-05-22T21:15:28.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f9594c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95901404d0 con 0x7f959013cfc0 2024-05-22T21:15:28.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f958e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959015af60 0x7f9590140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f958e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959015af60 0x7f9590140970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42974/0 (socket says 172.21.15.161:42974) 2024-05-22T21:15:28.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f958e800640 1 -- 172.21.15.161:0/35239038 learned_addr learned my addr 172.21.15.161:0/35239038 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:28.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f958de00640 1 --2- 172.21.15.161:0/35239038 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f959013cfc0 0x7f959013c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f958e800640 1 -- 172.21.15.161:0/35239038 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f959013cfc0 msgr2=0x7f959013c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f958e800640 1 --2- 172.21.15.161:0/35239038 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f959013cfc0 0x7f959013c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f958e800640 1 -- 172.21.15.161:0/35239038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f959013fc80 msgr2=0x7f95901646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f9585400640 1 --2- 172.21.15.161:0/35239038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f959013fc80 0x7f95901646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f958e800640 1 --2- 172.21.15.161:0/35239038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f959013fc80 0x7f95901646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f958e800640 1 -- 172.21.15.161:0/35239038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9580009420 con 0x7f959015af60 2024-05-22T21:15:28.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f958de00640 1 --2- 172.21.15.161:0/35239038 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f959013cfc0 0x7f959013c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:28.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f958e800640 1 --2- 172.21.15.161:0/35239038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959015af60 0x7f9590140970 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f958800cb40 tx=0x7f958800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:28.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f9586a00640 1 -- 172.21.15.161:0/35239038 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958800aca0 con 0x7f959015af60 2024-05-22T21:15:28.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f9586a00640 1 -- 172.21.15.161:0/35239038 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958800ae00 con 0x7f959015af60 2024-05-22T21:15:28.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f9594c00640 1 -- 172.21.15.161:0/35239038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959015b4f0 con 0x7f959015af60 2024-05-22T21:15:28.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f9586a00640 1 -- 172.21.15.161:0/35239038 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588012620 con 0x7f959015af60 2024-05-22T21:15:28.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f9594c00640 1 -- 172.21.15.161:0/35239038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9590141640 con 0x7f959015af60 2024-05-22T21:15:28.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f9594c00640 1 -- 172.21.15.161:0/35239038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9590067220 con 0x7f959015af60 2024-05-22T21:15:28.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.626+0000 7f9586a00640 1 -- 172.21.15.161:0/35239038 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9588007070 con 0x7f959015af60 2024-05-22T21:15:28.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.630+0000 7f9586a00640 1 --2- 172.21.15.161:0/35239038 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f956c066690 0x7f956c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.630+0000 7f9586a00640 1 -- 172.21.15.161:0/35239038 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9588084330 con 0x7f959015af60 2024-05-22T21:15:28.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.630+0000 7f958de00640 1 --2- 172.21.15.161:0/35239038 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f956c066690 0x7f956c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.630+0000 7f958de00640 1 --2- 172.21.15.161:0/35239038 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f956c066690 0x7f956c068b50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f95800ad000 tx=0x7f95800a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:28.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.634+0000 7f9586a00640 1 -- 172.21.15.161:0/35239038 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9588051530 con 0x7f959015af60 2024-05-22T21:15:28.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.786+0000 7f9594c00640 1 -- 172.21.15.161:0/35239038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9590071670 con 0x7f959015af60 2024-05-22T21:15:28.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.786+0000 7f9586a00640 1 -- 172.21.15.161:0/35239038 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9588054b50 con 0x7f959015af60 2024-05-22T21:15:28.791 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:15:28.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.786+0000 7f9594c00640 1 -- 172.21.15.161:0/35239038 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f956c066690 msgr2=0x7f956c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.786+0000 7f9594c00640 1 --2- 172.21.15.161:0/35239038 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f956c066690 0x7f956c068b50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f95800ad000 tx=0x7f95800a6000 comp rx=0 tx=0).stop 2024-05-22T21:15:28.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.786+0000 7f9594c00640 1 -- 172.21.15.161:0/35239038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959015af60 msgr2=0x7f9590140970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.786+0000 7f9594c00640 1 --2- 172.21.15.161:0/35239038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959015af60 0x7f9590140970 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f958800cb40 tx=0x7f958800a060 comp rx=0 tx=0).stop 2024-05-22T21:15:28.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.786+0000 7f9594c00640 1 -- 172.21.15.161:0/35239038 shutdown_connections 2024-05-22T21:15:28.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.786+0000 7f9594c00640 1 --2- 172.21.15.161:0/35239038 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f956c066690 0x7f956c068b50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.786+0000 7f9594c00640 1 --2- 172.21.15.161:0/35239038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959015af60 0x7f9590140970 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.786+0000 7f9594c00640 1 --2- 172.21.15.161:0/35239038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f959013fc80 0x7f95901646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.786+0000 7f9594c00640 1 --2- 172.21.15.161:0/35239038 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f959013cfc0 0x7f959013c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.786+0000 7f9594c00640 1 -- 172.21.15.161:0/35239038 >> 172.21.15.161:0/35239038 conn(0x7f959006a2d0 msgr2=0x7f959006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:28.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.786+0000 7f9594c00640 1 -- 172.21.15.161:0/35239038 shutdown_connections 2024-05-22T21:15:28.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.790+0000 7f9594c00640 1 -- 172.21.15.161:0/35239038 wait complete. 2024-05-22T21:15:28.813 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:15:28.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.946+0000 7ffb66c00640 1 Processor -- start 2024-05-22T21:15:28.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.946+0000 7ffb66c00640 1 -- start start 2024-05-22T21:15:28.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.946+0000 7ffb66c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffb601596b0 0x7ffb60071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.946+0000 7ffb66c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb6006d240 0x7ffb600b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.946+0000 7ffb66c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb6015d250 con 0x7ffb60071db0 2024-05-22T21:15:28.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.946+0000 7ffb66c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb60159a90 con 0x7ffb601596b0 2024-05-22T21:15:28.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.946+0000 7ffb66c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6015d510 con 0x7ffb6006d240 2024-05-22T21:15:28.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.946+0000 7ffb64c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb6006d240 0x7ffb600b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.946+0000 7ffb5fe00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffb601596b0 0x7ffb60071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.946+0000 7ffb64c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb6006d240 0x7ffb600b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:59938/0 (socket says 172.21.15.161:59938) 2024-05-22T21:15:28.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.946+0000 7ffb64c00640 1 -- 172.21.15.161:0/4260337965 learned_addr learned my addr 172.21.15.161:0/4260337965 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:28.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.946+0000 7ffb5fe00640 1 -- 172.21.15.161:0/4260337965 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb6006d240 msgr2=0x7ffb600b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:28.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.946+0000 7ffb5fe00640 1 --2- 172.21.15.161:0/4260337965 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb6006d240 0x7ffb600b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.946+0000 7ffb5fe00640 1 -- 172.21.15.161:0/4260337965 >> v1:172.21.15.179:6789/0 conn(0x7ffb60071db0 legacy=0x7ffb6006cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.946+0000 7ffb5fe00640 1 -- 172.21.15.161:0/4260337965 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6015d7f0 con 0x7ffb601596b0 2024-05-22T21:15:28.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.946+0000 7ffb5ea00640 1 -- 172.21.15.161:0/4260337965 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3798477855 0 0) 0x7ffb6015d250 con 0x7ffb60071db0 2024-05-22T21:15:28.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.950+0000 7ffb5fe00640 1 --2- 172.21.15.161:0/4260337965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb601596b0 0x7ffb60071870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000a960 tx=0x7ffb5001d870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e7aaea016d50a00 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:28.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.950+0000 7ffb5ea00640 1 -- 172.21.15.161:0/4260337965 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb500a67d0 con 0x7ffb601596b0 2024-05-22T21:15:28.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.950+0000 7ffb5ea00640 1 -- 172.21.15.161:0/4260337965 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb500a6930 con 0x7ffb601596b0 2024-05-22T21:15:28.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.950+0000 7ffb5ea00640 1 -- 172.21.15.161:0/4260337965 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb500a6cc0 con 0x7ffb601596b0 2024-05-22T21:15:28.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.950+0000 7ffb66c00640 1 -- 172.21.15.161:0/4260337965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb601596b0 msgr2=0x7ffb60071870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.950+0000 7ffb66c00640 1 --2- 172.21.15.161:0/4260337965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb601596b0 0x7ffb60071870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000a960 tx=0x7ffb5001d870 comp rx=0 tx=0).stop 2024-05-22T21:15:28.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.950+0000 7ffb66c00640 1 -- 172.21.15.161:0/4260337965 shutdown_connections 2024-05-22T21:15:28.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.950+0000 7ffb66c00640 1 --2- 172.21.15.161:0/4260337965 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb6006d240 0x7ffb600b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.950+0000 7ffb66c00640 1 --2- 172.21.15.161:0/4260337965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb601596b0 0x7ffb60071870 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.950+0000 7ffb66c00640 1 -- 172.21.15.161:0/4260337965 >> 172.21.15.161:0/4260337965 conn(0x7ffb6006a2d0 msgr2=0x7ffb6006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:28.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.950+0000 7ffb66c00640 1 -- 172.21.15.161:0/4260337965 shutdown_connections 2024-05-22T21:15:28.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.950+0000 7ffb66c00640 1 -- 172.21.15.161:0/4260337965 wait complete. 2024-05-22T21:15:28.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb66c00640 1 Processor -- start 2024-05-22T21:15:28.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb66c00640 1 -- start start 2024-05-22T21:15:28.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb66c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6006d240 0x7ffb60149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb66c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb60071db0 0x7ffb6015a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb66c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb601596b0 0x7ffb60144740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb66c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6015b480 con 0x7ffb601596b0 2024-05-22T21:15:28.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb66c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6015a5c0 con 0x7ffb6006d240 2024-05-22T21:15:28.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb66c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6015a800 con 0x7ffb60071db0 2024-05-22T21:15:28.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb64c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb601596b0 0x7ffb60144740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb64c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb601596b0 0x7ffb60144740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42990/0 (socket says 172.21.15.161:42990) 2024-05-22T21:15:28.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb64c00640 1 -- 172.21.15.161:0/3085378302 learned_addr learned my addr 172.21.15.161:0/3085378302 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:28.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb5fe00640 1 --2- 172.21.15.161:0/3085378302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6006d240 0x7ffb60149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb5f400640 1 --2- 172.21.15.161:0/3085378302 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb60071db0 0x7ffb6015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb64c00640 1 -- 172.21.15.161:0/3085378302 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb60071db0 msgr2=0x7ffb6015a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb64c00640 1 --2- 172.21.15.161:0/3085378302 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb60071db0 0x7ffb6015a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb64c00640 1 -- 172.21.15.161:0/3085378302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6006d240 msgr2=0x7ffb60149700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:28.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb64c00640 1 --2- 172.21.15.161:0/3085378302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6006d240 0x7ffb60149700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:28.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb64c00640 1 -- 172.21.15.161:0/3085378302 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5000a5c0 con 0x7ffb601596b0 2024-05-22T21:15:28.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb64c00640 1 --2- 172.21.15.161:0/3085378302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb601596b0 0x7ffb60144740 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800efc0 tx=0x7ffb5800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:28.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb57e00640 1 -- 172.21.15.161:0/3085378302 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5800ccc0 con 0x7ffb601596b0 2024-05-22T21:15:28.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb57e00640 1 -- 172.21.15.161:0/3085378302 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5800f040 con 0x7ffb601596b0 2024-05-22T21:15:28.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb57e00640 1 -- 172.21.15.161:0/3085378302 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58003be0 con 0x7ffb601596b0 2024-05-22T21:15:28.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb66c00640 1 -- 172.21.15.161:0/3085378302 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6015aae0 con 0x7ffb601596b0 2024-05-22T21:15:28.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.954+0000 7ffb66c00640 1 -- 172.21.15.161:0/3085378302 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6015b0a0 con 0x7ffb601596b0 2024-05-22T21:15:28.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.958+0000 7ffb66c00640 1 -- 172.21.15.161:0/3085378302 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6015b5c0 con 0x7ffb601596b0 2024-05-22T21:15:28.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.958+0000 7ffb57e00640 1 -- 172.21.15.161:0/3085378302 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffb58003d40 con 0x7ffb601596b0 2024-05-22T21:15:28.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.958+0000 7ffb57e00640 1 --2- 172.21.15.161:0/3085378302 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb40066690 0x7ffb40068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:28.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.958+0000 7ffb57e00640 1 -- 172.21.15.161:0/3085378302 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ffb58084640 con 0x7ffb601596b0 2024-05-22T21:15:28.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.958+0000 7ffb5fe00640 1 --2- 172.21.15.161:0/3085378302 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb40066690 0x7ffb40068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:28.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.958+0000 7ffb5fe00640 1 --2- 172.21.15.161:0/3085378302 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb40066690 0x7ffb40068b50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffb500b0000 tx=0x7ffb50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:28.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:28.962+0000 7ffb57e00640 1 -- 172.21.15.161:0/3085378302 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffb6015b5c0 con 0x7ffb601596b0 2024-05-22T21:15:29.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.118+0000 7ffb66c00640 1 -- 172.21.15.161:0/3085378302 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb6015b5c0 con 0x7ffb601596b0 2024-05-22T21:15:29.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.118+0000 7ffb57e00640 1 -- 172.21.15.161:0/3085378302 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb6015b5c0 con 0x7ffb601596b0 2024-05-22T21:15:29.123 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:15:29.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.118+0000 7ffb66c00640 1 -- 172.21.15.161:0/3085378302 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb40066690 msgr2=0x7ffb40068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.118+0000 7ffb66c00640 1 --2- 172.21.15.161:0/3085378302 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb40066690 0x7ffb40068b50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffb500b0000 tx=0x7ffb50008040 comp rx=0 tx=0).stop 2024-05-22T21:15:29.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.118+0000 7ffb66c00640 1 -- 172.21.15.161:0/3085378302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb601596b0 msgr2=0x7ffb60144740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.118+0000 7ffb66c00640 1 --2- 172.21.15.161:0/3085378302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb601596b0 0x7ffb60144740 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800efc0 tx=0x7ffb5800c490 comp rx=0 tx=0).stop 2024-05-22T21:15:29.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.118+0000 7ffb66c00640 1 -- 172.21.15.161:0/3085378302 shutdown_connections 2024-05-22T21:15:29.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.118+0000 7ffb66c00640 1 --2- 172.21.15.161:0/3085378302 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb40066690 0x7ffb40068b50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.118+0000 7ffb66c00640 1 --2- 172.21.15.161:0/3085378302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb601596b0 0x7ffb60144740 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.118+0000 7ffb66c00640 1 --2- 172.21.15.161:0/3085378302 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb60071db0 0x7ffb6015a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.118+0000 7ffb66c00640 1 --2- 172.21.15.161:0/3085378302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6006d240 0x7ffb60149700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.118+0000 7ffb66c00640 1 -- 172.21.15.161:0/3085378302 >> 172.21.15.161:0/3085378302 conn(0x7ffb6006a2d0 msgr2=0x7ffb600cd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:29.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.118+0000 7ffb66c00640 1 -- 172.21.15.161:0/3085378302 shutdown_connections 2024-05-22T21:15:29.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.118+0000 7ffb66c00640 1 -- 172.21.15.161:0/3085378302 wait complete. 2024-05-22T21:15:29.145 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:15:29.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.274+0000 7f4131000640 1 Processor -- start 2024-05-22T21:15:29.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.274+0000 7f4131000640 1 -- start start 2024-05-22T21:15:29.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.274+0000 7f4131000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f412c0715a0 0x7f412c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.274+0000 7f4131000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f412c15ec90 0x7f412c15f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.274+0000 7f4131000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412c1442a0 con 0x7f412c143ec0 2024-05-22T21:15:29.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.274+0000 7f4131000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f412c15f660 con 0x7f412c0715a0 2024-05-22T21:15:29.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.274+0000 7f4131000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412c070500 con 0x7f412c15ec90 2024-05-22T21:15:29.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.274+0000 7f4129e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f412c143ec0 0x7f412c06fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60468/0 (socket says 172.21.15.161:60468) 2024-05-22T21:15:29.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.274+0000 7f4129e00640 1 -- 172.21.15.161:0/3538892320 learned_addr learned my addr 172.21.15.161:0/3538892320 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:29.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4129400640 1 --2- 172.21.15.161:0/3538892320 >> v2:172.21.15.179:3300/0 conn(0x7f412c0715a0 0x7f412c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f412a800640 1 --2- 172.21.15.161:0/3538892320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f412c15ec90 0x7f412c15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4128a00640 1 -- 172.21.15.161:0/3538892320 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528754574 0 0) 0x7f412c1442a0 con 0x7f412c143ec0 2024-05-22T21:15:29.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4128a00640 1 -- 172.21.15.161:0/3538892320 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4110003530 con 0x7f412c143ec0 2024-05-22T21:15:29.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4128a00640 1 -- 172.21.15.161:0/3538892320 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3079228077 0 0) 0x7f4110003530 con 0x7f412c143ec0 2024-05-22T21:15:29.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4128a00640 1 -- 172.21.15.161:0/3538892320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f412c15ec90 msgr2=0x7f412c15f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:29.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4128a00640 1 --2- 172.21.15.161:0/3538892320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f412c15ec90 0x7f412c15f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4128a00640 1 -- 172.21.15.161:0/3538892320 >> v2:172.21.15.179:3300/0 conn(0x7f412c0715a0 msgr2=0x7f412c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:29.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4128a00640 1 --2- 172.21.15.161:0/3538892320 >> v2:172.21.15.179:3300/0 conn(0x7f412c0715a0 0x7f412c15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4128a00640 1 -- 172.21.15.161:0/3538892320 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c0707e0 con 0x7f412c143ec0 2024-05-22T21:15:29.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4128a00640 1 -- 172.21.15.161:0/3538892320 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f411c001a00 con 0x7f412c143ec0 2024-05-22T21:15:29.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4128a00640 1 -- 172.21.15.161:0/3538892320 >> v1:172.21.15.161:6789/0 conn(0x7f412c143ec0 legacy=0x7f412c06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4128a00640 1 --2- 172.21.15.161:0/3538892320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4110003980 0x7f4110003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4128a00640 1 --2- 172.21.15.161:0/3538892320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41100047d0 0x7f4110004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4128a00640 1 --2- 172.21.15.161:0/3538892320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f412c15ec90 0x7f4110005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4128a00640 1 -- 172.21.15.161:0/3538892320 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c15f660 con 0x7f4110003980 2024-05-22T21:15:29.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4128a00640 1 -- 172.21.15.161:0/3538892320 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c070500 con 0x7f41100047d0 2024-05-22T21:15:29.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4128a00640 1 -- 172.21.15.161:0/3538892320 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411000b990 con 0x7f412c15ec90 2024-05-22T21:15:29.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4129e00640 1 --2- 172.21.15.161:0/3538892320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4110003980 0x7f4110003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4129400640 1 --2- 172.21.15.161:0/3538892320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41100047d0 0x7f4110004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f412a800640 1 --2- 172.21.15.161:0/3538892320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f412c15ec90 0x7f4110005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4129e00640 1 -- 172.21.15.161:0/3538892320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f412c15ec90 msgr2=0x7f4110005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:29.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4129e00640 1 --2- 172.21.15.161:0/3538892320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f412c15ec90 0x7f4110005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4129e00640 1 -- 172.21.15.161:0/3538892320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41100047d0 msgr2=0x7f4110004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:29.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4129e00640 1 --2- 172.21.15.161:0/3538892320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41100047d0 0x7f4110004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.278+0000 7f4129e00640 1 -- 172.21.15.161:0/3538892320 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4110003530 con 0x7f4110003980 2024-05-22T21:15:29.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4129e00640 1 --2- 172.21.15.161:0/3538892320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4110003980 0x7f4110003da0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f411c00a860 tx=0x7f411c01d350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb1959b60e701b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:29.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4128a00640 1 -- 172.21.15.161:0/3538892320 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c0ad630 con 0x7f4110003980 2024-05-22T21:15:29.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4128a00640 1 -- 172.21.15.161:0/3538892320 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c0a8070 con 0x7f4110003980 2024-05-22T21:15:29.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4128a00640 1 -- 172.21.15.161:0/3538892320 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c0ad9f0 con 0x7f4110003980 2024-05-22T21:15:29.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4131000640 1 -- 172.21.15.161:0/3538892320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4110003980 msgr2=0x7f4110003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4131000640 1 --2- 172.21.15.161:0/3538892320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4110003980 0x7f4110003da0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f411c00a860 tx=0x7f411c01d350 comp rx=0 tx=0).stop 2024-05-22T21:15:29.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f412a800640 1 -- 172.21.15.161:0/3538892320 reap_dead start 2024-05-22T21:15:29.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4131000640 1 -- 172.21.15.161:0/3538892320 shutdown_connections 2024-05-22T21:15:29.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4131000640 1 -- 172.21.15.161:0/3538892320 >> 172.21.15.161:0/3538892320 conn(0x7f412c0b0bc0 msgr2=0x7f412c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:29.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4131000640 1 -- 172.21.15.161:0/3538892320 shutdown_connections 2024-05-22T21:15:29.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4131000640 1 -- 172.21.15.161:0/3538892320 wait complete. 2024-05-22T21:15:29.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4131000640 1 Processor -- start 2024-05-22T21:15:29.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4131000640 1 -- start start 2024-05-22T21:15:29.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4131000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4110003980 0x7f412c0a9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4131000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f412c13fa70 0x7f412c15d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4131000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f412c1f62c0 0x7f412c1f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4131000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c0aab70 con 0x7f412c1f62c0 2024-05-22T21:15:29.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4131000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c0a9c70 con 0x7f412c13fa70 2024-05-22T21:15:29.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.282+0000 7f4131000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412c0a9ef0 con 0x7f4110003980 2024-05-22T21:15:29.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f412a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f412c1f62c0 0x7f412c1f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f412a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f412c1f62c0 0x7f412c1f6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43020/0 (socket says 172.21.15.161:43020) 2024-05-22T21:15:29.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f412a800640 1 -- 172.21.15.161:0/2635479158 learned_addr learned my addr 172.21.15.161:0/2635479158 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:29.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f4129400640 1 --2- 172.21.15.161:0/2635479158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f412c13fa70 0x7f412c15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f4129e00640 1 --2- 172.21.15.161:0/2635479158 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4110003980 0x7f412c0a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f412a800640 1 -- 172.21.15.161:0/2635479158 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4110003980 msgr2=0x7f412c0a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f412a800640 1 --2- 172.21.15.161:0/2635479158 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4110003980 0x7f412c0a9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f412a800640 1 -- 172.21.15.161:0/2635479158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f412c13fa70 msgr2=0x7f412c15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f412a800640 1 --2- 172.21.15.161:0/2635479158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f412c13fa70 0x7f412c15d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f412a800640 1 -- 172.21.15.161:0/2635479158 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c0aa1d0 con 0x7f412c1f62c0 2024-05-22T21:15:29.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f412a800640 1 --2- 172.21.15.161:0/2635479158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f412c1f62c0 0x7f412c1f6740 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f412c0cedc0 tx=0x7f4124011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:29.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f4122000640 1 -- 172.21.15.161:0/2635479158 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4124011ee0 con 0x7f412c1f62c0 2024-05-22T21:15:29.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f4122000640 1 -- 172.21.15.161:0/2635479158 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4124004240 con 0x7f412c1f62c0 2024-05-22T21:15:29.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f4122000640 1 -- 172.21.15.161:0/2635479158 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412400f770 con 0x7f412c1f62c0 2024-05-22T21:15:29.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f4131000640 1 -- 172.21.15.161:0/2635479158 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412c0aa4b0 con 0x7f412c1f62c0 2024-05-22T21:15:29.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f4131000640 1 -- 172.21.15.161:0/2635479158 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412c15dec0 con 0x7f412c1f62c0 2024-05-22T21:15:29.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f4131000640 1 -- 172.21.15.161:0/2635479158 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412c0aacb0 con 0x7f412c1f62c0 2024-05-22T21:15:29.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f4122000640 1 -- 172.21.15.161:0/2635479158 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f412400f8d0 con 0x7f412c1f62c0 2024-05-22T21:15:29.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f4122000640 1 --2- 172.21.15.161:0/2635479158 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f410c0666c0 0x7f410c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.286+0000 7f4122000640 1 -- 172.21.15.161:0/2635479158 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f41240845e0 con 0x7f412c1f62c0 2024-05-22T21:15:29.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.290+0000 7f4129e00640 1 --2- 172.21.15.161:0/2635479158 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f410c0666c0 0x7f410c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.290+0000 7f4129e00640 1 --2- 172.21.15.161:0/2635479158 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f410c0666c0 0x7f410c068b80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f411c0ad000 tx=0x7f411c001e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:29.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.290+0000 7f4122000640 1 -- 172.21.15.161:0/2635479158 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f412c0aacb0 con 0x7f412c1f62c0 2024-05-22T21:15:29.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.478+0000 7f4131000640 1 -- 172.21.15.161:0/2635479158 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f412c0aacb0 con 0x7f412c1f62c0 2024-05-22T21:15:29.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.478+0000 7f4122000640 1 -- 172.21.15.161:0/2635479158 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f412c0aacb0 con 0x7f412c1f62c0 2024-05-22T21:15:29.482 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:29.483 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/1767934947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1767934947},{"type":"v1","addr":"172.21.15.161:6836","nonce":1767934947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1873337287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1873337287},{"type":"v1","addr":"172.21.15.179:6834","nonce":1873337287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1815838236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":1815838236},{"type":"v1","addr":"172.21.15.161:6837","nonce":1815838236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:50.052292+0000","modified":"2024-05-22T21:14:53.058881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5374":{"gid":5374,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/389418050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":389418050},{"type":"v1","addr":"172.21.15.179:6835","nonce":389418050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-22T21:15:29.483 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 23 2024-05-22T21:15:29.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.478+0000 7f4131000640 1 -- 172.21.15.161:0/2635479158 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f410c0666c0 msgr2=0x7f410c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.478+0000 7f4131000640 1 --2- 172.21.15.161:0/2635479158 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f410c0666c0 0x7f410c068b80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f411c0ad000 tx=0x7f411c001e00 comp rx=0 tx=0).stop 2024-05-22T21:15:29.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.478+0000 7f4131000640 1 -- 172.21.15.161:0/2635479158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f412c1f62c0 msgr2=0x7f412c1f6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.478+0000 7f4131000640 1 --2- 172.21.15.161:0/2635479158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f412c1f62c0 0x7f412c1f6740 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f412c0cedc0 tx=0x7f4124011c30 comp rx=0 tx=0).stop 2024-05-22T21:15:29.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.478+0000 7f4131000640 1 -- 172.21.15.161:0/2635479158 shutdown_connections 2024-05-22T21:15:29.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.478+0000 7f4131000640 1 --2- 172.21.15.161:0/2635479158 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f410c0666c0 0x7f410c068b80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.478+0000 7f4131000640 1 --2- 172.21.15.161:0/2635479158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f412c1f62c0 0x7f412c1f6740 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.478+0000 7f4131000640 1 --2- 172.21.15.161:0/2635479158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f412c13fa70 0x7f412c15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.478+0000 7f4131000640 1 --2- 172.21.15.161:0/2635479158 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4110003980 0x7f412c0a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.478+0000 7f4131000640 1 -- 172.21.15.161:0/2635479158 >> 172.21.15.161:0/2635479158 conn(0x7f412c0b0bc0 msgr2=0x7f412c0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:29.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.478+0000 7f4131000640 1 -- 172.21.15.161:0/2635479158 shutdown_connections 2024-05-22T21:15:29.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.478+0000 7f4131000640 1 -- 172.21.15.161:0/2635479158 wait complete. 2024-05-22T21:15:29.503 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:15:29.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.634+0000 7fd817600640 1 Processor -- start 2024-05-22T21:15:29.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.634+0000 7fd817600640 1 -- start start 2024-05-22T21:15:29.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.634+0000 7fd817600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd81006afe0 0x7fd8100c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.634+0000 7fd817600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81006b440 0x7fd8100c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.634+0000 7fd817600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8100d0930 con 0x7fd8100c3f70 2024-05-22T21:15:29.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.634+0000 7fd817600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8100cfd20 con 0x7fd81006afe0 2024-05-22T21:15:29.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.634+0000 7fd817600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8100d0000 con 0x7fd81006b440 2024-05-22T21:15:29.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.634+0000 7fd80fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd81006afe0 0x7fd8100c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.634+0000 7fd80fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd81006afe0 0x7fd8100c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43026/0 (socket says 172.21.15.161:43026) 2024-05-22T21:15:29.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.634+0000 7fd80fe00640 1 -- 172.21.15.161:0/2324817597 learned_addr learned my addr 172.21.15.161:0/2324817597 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:29.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.634+0000 7fd815600640 1 --2- 172.21.15.161:0/2324817597 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81006b440 0x7fd8100c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.634+0000 7fd80fe00640 1 -- 172.21.15.161:0/2324817597 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81006b440 msgr2=0x7fd8100c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:29.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.634+0000 7fd80fe00640 1 --2- 172.21.15.161:0/2324817597 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81006b440 0x7fd8100c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.634+0000 7fd80fe00640 1 -- 172.21.15.161:0/2324817597 >> v1:172.21.15.179:6789/0 conn(0x7fd8100c3f70 legacy=0x7fd8100c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.634+0000 7fd80fe00640 1 -- 172.21.15.161:0/2324817597 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8100d02e0 con 0x7fd81006afe0 2024-05-22T21:15:29.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.638+0000 7fd80fe00640 1 --2- 172.21.15.161:0/2324817597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81006afe0 0x7fd8100c8270 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd800009870 tx=0x7fd80001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d3b75064048d42 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:29.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.638+0000 7fd80f400640 1 -- 172.21.15.161:0/2324817597 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8000ab680 con 0x7fd81006afe0 2024-05-22T21:15:29.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.638+0000 7fd80f400640 1 -- 172.21.15.161:0/2324817597 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8000ab7e0 con 0x7fd81006afe0 2024-05-22T21:15:29.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.638+0000 7fd80f400640 1 -- 172.21.15.161:0/2324817597 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8000abb70 con 0x7fd81006afe0 2024-05-22T21:15:29.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.638+0000 7fd817600640 1 -- 172.21.15.161:0/2324817597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81006afe0 msgr2=0x7fd8100c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.638+0000 7fd817600640 1 --2- 172.21.15.161:0/2324817597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81006afe0 0x7fd8100c8270 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd800009870 tx=0x7fd80001c4a0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.638+0000 7fd817600640 1 -- 172.21.15.161:0/2324817597 shutdown_connections 2024-05-22T21:15:29.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.638+0000 7fd817600640 1 --2- 172.21.15.161:0/2324817597 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81006b440 0x7fd8100c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.638+0000 7fd817600640 1 --2- 172.21.15.161:0/2324817597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81006afe0 0x7fd8100c8270 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.638+0000 7fd817600640 1 -- 172.21.15.161:0/2324817597 >> 172.21.15.161:0/2324817597 conn(0x7fd810099a40 msgr2=0x7fd810099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:29.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.638+0000 7fd817600640 1 -- 172.21.15.161:0/2324817597 shutdown_connections 2024-05-22T21:15:29.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.638+0000 7fd817600640 1 -- 172.21.15.161:0/2324817597 wait complete. 2024-05-22T21:15:29.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd817600640 1 Processor -- start 2024-05-22T21:15:29.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd817600640 1 -- start start 2024-05-22T21:15:29.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd817600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81006afe0 0x7fd810153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd817600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81006b440 0x7fd810153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd817600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8100c3f70 0x7fd810160560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd817600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8100adcb0 con 0x7fd81006afe0 2024-05-22T21:15:29.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd817600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8101545f0 con 0x7fd8100c3f70 2024-05-22T21:15:29.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd817600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8101548d0 con 0x7fd81006b440 2024-05-22T21:15:29.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd814c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81006afe0 0x7fd810153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd814c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81006afe0 0x7fd810153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43032/0 (socket says 172.21.15.161:43032) 2024-05-22T21:15:29.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd814c00640 1 -- 172.21.15.161:0/1916062186 learned_addr learned my addr 172.21.15.161:0/1916062186 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:29.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd815600640 1 --2- 172.21.15.161:0/1916062186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8100c3f70 0x7fd810160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd80fe00640 1 --2- 172.21.15.161:0/1916062186 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81006b440 0x7fd810153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd814c00640 1 -- 172.21.15.161:0/1916062186 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81006b440 msgr2=0x7fd810153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd814c00640 1 --2- 172.21.15.161:0/1916062186 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81006b440 0x7fd810153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd814c00640 1 -- 172.21.15.161:0/1916062186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8100c3f70 msgr2=0x7fd810160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd814c00640 1 --2- 172.21.15.161:0/1916062186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8100c3f70 0x7fd810160560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd814c00640 1 -- 172.21.15.161:0/1916062186 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8000094d0 con 0x7fd81006afe0 2024-05-22T21:15:29.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd814c00640 1 --2- 172.21.15.161:0/1916062186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81006afe0 0x7fd810153670 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800c500 tx=0x7fd7f800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:29.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd80cc00640 1 -- 172.21.15.161:0/1916062186 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8005220 con 0x7fd81006afe0 2024-05-22T21:15:29.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd80cc00640 1 -- 172.21.15.161:0/1916062186 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f8005380 con 0x7fd81006afe0 2024-05-22T21:15:29.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd817600640 1 -- 172.21.15.161:0/1916062186 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd810154bb0 con 0x7fd81006afe0 2024-05-22T21:15:29.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd80cc00640 1 -- 172.21.15.161:0/1916062186 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8007270 con 0x7fd81006afe0 2024-05-22T21:15:29.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.642+0000 7fd817600640 1 -- 172.21.15.161:0/1916062186 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd810154230 con 0x7fd81006afe0 2024-05-22T21:15:29.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.646+0000 7fd817600640 1 -- 172.21.15.161:0/1916062186 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd810067220 con 0x7fd81006afe0 2024-05-22T21:15:29.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.646+0000 7fd80cc00640 1 -- 172.21.15.161:0/1916062186 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd7f80073d0 con 0x7fd81006afe0 2024-05-22T21:15:29.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.646+0000 7fd80cc00640 1 --2- 172.21.15.161:0/1916062186 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7ec066690 0x7fd7ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.646+0000 7fd80cc00640 1 -- 172.21.15.161:0/1916062186 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd7f8083a20 con 0x7fd81006afe0 2024-05-22T21:15:29.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.646+0000 7fd80fe00640 1 --2- 172.21.15.161:0/1916062186 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7ec066690 0x7fd7ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.646+0000 7fd80fe00640 1 --2- 172.21.15.161:0/1916062186 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7ec066690 0x7fd7ec068b50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd8000af000 tx=0x7fd8000b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:29.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.650+0000 7fd80cc00640 1 -- 172.21.15.161:0/1916062186 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd7f8050d70 con 0x7fd81006afe0 2024-05-22T21:15:29.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.806+0000 7fd817600640 1 -- 172.21.15.161:0/1916062186 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8100c5410 con 0x7fd81006afe0 2024-05-22T21:15:29.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.806+0000 7fd80cc00640 1 -- 172.21.15.161:0/1916062186 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16183 (secure 0 0 0) 0x7fd7f8054390 con 0x7fd81006afe0 2024-05-22T21:15:29.813 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:29.813 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":55,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:15:17.769605+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:48.612726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:49.053022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.9","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:15:29.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.810+0000 7fd817600640 1 -- 172.21.15.161:0/1916062186 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7ec066690 msgr2=0x7fd7ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.810+0000 7fd817600640 1 --2- 172.21.15.161:0/1916062186 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7ec066690 0x7fd7ec068b50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd8000af000 tx=0x7fd8000b6040 comp rx=0 tx=0).stop 2024-05-22T21:15:29.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.810+0000 7fd817600640 1 -- 172.21.15.161:0/1916062186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81006afe0 msgr2=0x7fd810153670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.810+0000 7fd817600640 1 --2- 172.21.15.161:0/1916062186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81006afe0 0x7fd810153670 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800c500 tx=0x7fd7f800c9c0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.810+0000 7fd817600640 1 -- 172.21.15.161:0/1916062186 shutdown_connections 2024-05-22T21:15:29.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.810+0000 7fd817600640 1 --2- 172.21.15.161:0/1916062186 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd7ec066690 0x7fd7ec068b50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.810+0000 7fd817600640 1 --2- 172.21.15.161:0/1916062186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8100c3f70 0x7fd810160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.810+0000 7fd817600640 1 --2- 172.21.15.161:0/1916062186 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd81006b440 0x7fd810153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.810+0000 7fd817600640 1 --2- 172.21.15.161:0/1916062186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd81006afe0 0x7fd810153670 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.810+0000 7fd817600640 1 -- 172.21.15.161:0/1916062186 >> 172.21.15.161:0/1916062186 conn(0x7fd810099a40 msgr2=0x7fd8100c5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:29.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.810+0000 7fd817600640 1 -- 172.21.15.161:0/1916062186 shutdown_connections 2024-05-22T21:15:29.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.810+0000 7fd817600640 1 -- 172.21.15.161:0/1916062186 wait complete. 2024-05-22T21:15:29.835 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-22T21:15:29.835 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:15:29.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.966+0000 7f0f9aa00640 1 Processor -- start 2024-05-22T21:15:29.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.966+0000 7f0f9aa00640 1 -- start start 2024-05-22T21:15:29.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.966+0000 7f0f9aa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0f940715a0 0x7f0f9415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.966+0000 7f0f9aa00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f941442a0 con 0x7f0f94143ec0 2024-05-22T21:15:29.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.966+0000 7f0f9aa00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f94070500 con 0x7f0f9415ec90 2024-05-22T21:15:29.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.966+0000 7f0f9aa00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f940707e0 con 0x7f0f940715a0 2024-05-22T21:15:29.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.966+0000 7f0f93e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0f94143ec0 0x7f0f9406fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60474/0 (socket says 172.21.15.161:60474) 2024-05-22T21:15:29.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.966+0000 7f0f93e00640 1 -- 172.21.15.161:0/2914388323 learned_addr learned my addr 172.21.15.161:0/2914388323 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:29.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.966+0000 7f0f8b400640 1 --2- 172.21.15.161:0/2914388323 >> v2:172.21.15.161:3300/0 conn(0x7f0f940715a0 0x7f0f9415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.966+0000 7f0f93400640 1 -- 172.21.15.161:0/2914388323 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333704367 0 0) 0x7f0f941442a0 con 0x7f0f94143ec0 2024-05-22T21:15:29.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.966+0000 7f0f93400640 1 -- 172.21.15.161:0/2914388323 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f78003530 con 0x7f0f94143ec0 2024-05-22T21:15:29.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.966+0000 7f0f8b400640 1 -- 172.21.15.161:0/2914388323 >> v1:172.21.15.179:6789/0 conn(0x7f0f9415ec90 legacy=0x7f0f940cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:29.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.966+0000 7f0f8b400640 1 -- 172.21.15.161:0/2914388323 >> v1:172.21.15.161:6789/0 conn(0x7f0f94143ec0 legacy=0x7f0f9406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.966+0000 7f0f8b400640 1 -- 172.21.15.161:0/2914388323 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f94070ac0 con 0x7f0f940715a0 2024-05-22T21:15:29.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.970+0000 7f0f8b400640 1 --2- 172.21.15.161:0/2914388323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f940715a0 0x7f0f9415f830 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0097c0 tx=0x7f0f7c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671df2e221f558f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:29.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.970+0000 7f0f93400640 1 -- 172.21.15.161:0/2914388323 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0ab670 con 0x7f0f940715a0 2024-05-22T21:15:29.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.970+0000 7f0f93400640 1 -- 172.21.15.161:0/2914388323 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7c0ab7d0 con 0x7f0f940715a0 2024-05-22T21:15:29.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.970+0000 7f0f93400640 1 -- 172.21.15.161:0/2914388323 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0abb60 con 0x7f0f940715a0 2024-05-22T21:15:29.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.970+0000 7f0f9aa00640 1 -- 172.21.15.161:0/2914388323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f940715a0 msgr2=0x7f0f9415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.970+0000 7f0f9aa00640 1 --2- 172.21.15.161:0/2914388323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f940715a0 0x7f0f9415f830 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0097c0 tx=0x7f0f7c01c470 comp rx=0 tx=0).stop 2024-05-22T21:15:29.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.970+0000 7f0f9aa00640 1 -- 172.21.15.161:0/2914388323 shutdown_connections 2024-05-22T21:15:29.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.970+0000 7f0f9aa00640 1 --2- 172.21.15.161:0/2914388323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f940715a0 0x7f0f9415f830 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.970+0000 7f0f9aa00640 1 -- 172.21.15.161:0/2914388323 >> 172.21.15.161:0/2914388323 conn(0x7f0f940b0bc0 msgr2=0x7f0f940b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:29.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.970+0000 7f0f9aa00640 1 -- 172.21.15.161:0/2914388323 shutdown_connections 2024-05-22T21:15:29.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.970+0000 7f0f9aa00640 1 -- 172.21.15.161:0/2914388323 wait complete. 2024-05-22T21:15:29.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f9aa00640 1 Processor -- start 2024-05-22T21:15:29.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f9aa00640 1 -- start start 2024-05-22T21:15:29.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f9aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f94143ec0 0x7f0f9415a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f9aa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f9415ec90 0x7f0f940b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f9aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f940ac8b0 0x7f0f940acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f9aa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9415a550 con 0x7f0f94143ec0 2024-05-22T21:15:29.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f9aa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f941f6ee0 con 0x7f0f940ac8b0 2024-05-22T21:15:29.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f9aa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f94140530 con 0x7f0f9415ec90 2024-05-22T21:15:29.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f93e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f94143ec0 0x7f0f9415a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f93e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f94143ec0 0x7f0f9415a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43056/0 (socket says 172.21.15.161:43056) 2024-05-22T21:15:29.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f93e00640 1 -- 172.21.15.161:0/1052694385 learned_addr learned my addr 172.21.15.161:0/1052694385 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:29.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f8b400640 1 --2- 172.21.15.161:0/1052694385 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f9415ec90 0x7f0f940b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f98a00640 1 --2- 172.21.15.161:0/1052694385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f940ac8b0 0x7f0f940acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f93e00640 1 -- 172.21.15.161:0/1052694385 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f9415ec90 msgr2=0x7f0f940b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f93e00640 1 --2- 172.21.15.161:0/1052694385 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f9415ec90 0x7f0f940b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f93e00640 1 -- 172.21.15.161:0/1052694385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f940ac8b0 msgr2=0x7f0f940acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:29.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f93e00640 1 --2- 172.21.15.161:0/1052694385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f940ac8b0 0x7f0f940acd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:29.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f93e00640 1 -- 172.21.15.161:0/1052694385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c009420 con 0x7f0f94143ec0 2024-05-22T21:15:29.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f93e00640 1 --2- 172.21.15.161:0/1052694385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f94143ec0 0x7f0f9415a010 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400c500 tx=0x7f0f8400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:29.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f90c00640 1 -- 172.21.15.161:0/1052694385 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84005cf0 con 0x7f0f94143ec0 2024-05-22T21:15:29.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f90c00640 1 -- 172.21.15.161:0/1052694385 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f84005e50 con 0x7f0f94143ec0 2024-05-22T21:15:29.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f9aa00640 1 -- 172.21.15.161:0/1052694385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f941f7130 con 0x7f0f94143ec0 2024-05-22T21:15:29.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f90c00640 1 -- 172.21.15.161:0/1052694385 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f840072f0 con 0x7f0f94143ec0 2024-05-22T21:15:29.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.974+0000 7f0f9aa00640 1 -- 172.21.15.161:0/1052694385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f941401c0 con 0x7f0f94143ec0 2024-05-22T21:15:29.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.978+0000 7f0f9aa00640 1 -- 172.21.15.161:0/1052694385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f94067220 con 0x7f0f94143ec0 2024-05-22T21:15:29.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.978+0000 7f0f90c00640 1 -- 172.21.15.161:0/1052694385 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0f84007450 con 0x7f0f94143ec0 2024-05-22T21:15:29.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.978+0000 7f0f90c00640 1 --2- 172.21.15.161:0/1052694385 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f700666c0 0x7f0f70068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:29.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.978+0000 7f0f90c00640 1 -- 172.21.15.161:0/1052694385 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0f84083700 con 0x7f0f94143ec0 2024-05-22T21:15:29.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.978+0000 7f0f8b400640 1 --2- 172.21.15.161:0/1052694385 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f700666c0 0x7f0f70068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:29.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.978+0000 7f0f8b400640 1 --2- 172.21.15.161:0/1052694385 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f700666c0 0x7f0f70068b80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0af000 tx=0x7f0f7c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:29.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:29.982+0000 7f0f90c00640 1 -- 172.21.15.161:0/1052694385 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0f84050ad0 con 0x7f0f94143ec0 2024-05-22T21:15:30.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.178+0000 7f0f9aa00640 1 -- 172.21.15.161:0/1052694385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f9406dcc0 con 0x7f0f94143ec0 2024-05-22T21:15:30.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.178+0000 7f0f90c00640 1 -- 172.21.15.161:0/1052694385 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f0f840540f0 con 0x7f0f94143ec0 2024-05-22T21:15:30.183 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:30.183 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/1767934947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1767934947},{"type":"v1","addr":"172.21.15.161:6836","nonce":1767934947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1873337287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1873337287},{"type":"v1","addr":"172.21.15.179:6834","nonce":1873337287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1815838236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":1815838236},{"type":"v1","addr":"172.21.15.161:6837","nonce":1815838236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:50.052292+0000","modified":"2024-05-22T21:14:53.058881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5374":{"gid":5374,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/389418050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":389418050},{"type":"v1","addr":"172.21.15.179:6835","nonce":389418050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-22T21:15:30.183 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 23 2024-05-22T21:15:30.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.178+0000 7f0f9aa00640 1 -- 172.21.15.161:0/1052694385 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f700666c0 msgr2=0x7f0f70068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:30.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.178+0000 7f0f9aa00640 1 --2- 172.21.15.161:0/1052694385 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f700666c0 0x7f0f70068b80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0af000 tx=0x7f0f7c0a7000 comp rx=0 tx=0).stop 2024-05-22T21:15:30.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.178+0000 7f0f9aa00640 1 -- 172.21.15.161:0/1052694385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f94143ec0 msgr2=0x7f0f9415a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:30.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.178+0000 7f0f9aa00640 1 --2- 172.21.15.161:0/1052694385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f94143ec0 0x7f0f9415a010 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400c500 tx=0x7f0f8400c9c0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.178+0000 7f0f9aa00640 1 -- 172.21.15.161:0/1052694385 shutdown_connections 2024-05-22T21:15:30.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.178+0000 7f0f9aa00640 1 --2- 172.21.15.161:0/1052694385 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f700666c0 0x7f0f70068b80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.178+0000 7f0f9aa00640 1 --2- 172.21.15.161:0/1052694385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f940ac8b0 0x7f0f940acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.178+0000 7f0f9aa00640 1 --2- 172.21.15.161:0/1052694385 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f9415ec90 0x7f0f940b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.178+0000 7f0f9aa00640 1 --2- 172.21.15.161:0/1052694385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f94143ec0 0x7f0f9415a010 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.178+0000 7f0f9aa00640 1 -- 172.21.15.161:0/1052694385 >> 172.21.15.161:0/1052694385 conn(0x7f0f940b0bc0 msgr2=0x7f0f940aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:30.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.178+0000 7f0f9aa00640 1 -- 172.21.15.161:0/1052694385 shutdown_connections 2024-05-22T21:15:30.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.178+0000 7f0f9aa00640 1 -- 172.21.15.161:0/1052694385 wait complete. 2024-05-22T21:15:30.204 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:15:30.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.334+0000 7f864ea00640 1 Processor -- start 2024-05-22T21:15:30.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.334+0000 7f864ea00640 1 -- start start 2024-05-22T21:15:30.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.334+0000 7f864ea00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f864815e130 0x7f864815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:30.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.334+0000 7f864ea00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f864815d410 0x7f864815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:30.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.334+0000 7f864ea00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8648071670 con 0x7f864815dd50 2024-05-22T21:15:30.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.334+0000 7f864ea00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864815d870 con 0x7f864815e130 2024-05-22T21:15:30.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.334+0000 7f864ea00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8648070a60 con 0x7f864815d410 2024-05-22T21:15:30.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.334+0000 7f863f400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f864815e130 0x7f864815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:30.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.334+0000 7f863f400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f864815e130 0x7f864815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43072/0 (socket says 172.21.15.161:43072) 2024-05-22T21:15:30.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.334+0000 7f863f400640 1 -- 172.21.15.161:0/456482727 learned_addr learned my addr 172.21.15.161:0/456482727 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:30.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.334+0000 7f864ca00640 1 --2- 172.21.15.161:0/456482727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f864815d410 0x7f864815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:30.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.334+0000 7f863f400640 1 -- 172.21.15.161:0/456482727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f864815d410 msgr2=0x7f864815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:30.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.334+0000 7f863f400640 1 --2- 172.21.15.161:0/456482727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f864815d410 0x7f864815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.334+0000 7f863f400640 1 -- 172.21.15.161:0/456482727 >> v1:172.21.15.179:6789/0 conn(0x7f864815dd50 legacy=0x7f864806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:30.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.334+0000 7f863f400640 1 -- 172.21.15.161:0/456482727 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8648070d40 con 0x7f864815e130 2024-05-22T21:15:30.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.338+0000 7f863f400640 1 --2- 172.21.15.161:0/456482727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f864815e130 0x7f864815ced0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8630009e50 tx=0x7f863001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacf3282e6c05f44 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:30.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.338+0000 7f8647400640 1 -- 172.21.15.161:0/456482727 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86300a58d0 con 0x7f864815e130 2024-05-22T21:15:30.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.338+0000 7f8647400640 1 -- 172.21.15.161:0/456482727 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86300a5a30 con 0x7f864815e130 2024-05-22T21:15:30.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.338+0000 7f8647400640 1 -- 172.21.15.161:0/456482727 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86300a5dc0 con 0x7f864815e130 2024-05-22T21:15:30.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.338+0000 7f864ea00640 1 -- 172.21.15.161:0/456482727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f864815e130 msgr2=0x7f864815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:30.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.338+0000 7f864ea00640 1 --2- 172.21.15.161:0/456482727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f864815e130 0x7f864815ced0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8630009e50 tx=0x7f863001ca00 comp rx=0 tx=0).stop 2024-05-22T21:15:30.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.338+0000 7f864ea00640 1 -- 172.21.15.161:0/456482727 shutdown_connections 2024-05-22T21:15:30.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.338+0000 7f864ea00640 1 --2- 172.21.15.161:0/456482727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f864815d410 0x7f864815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.338+0000 7f864ea00640 1 --2- 172.21.15.161:0/456482727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f864815e130 0x7f864815ced0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.338+0000 7f864ea00640 1 -- 172.21.15.161:0/456482727 >> 172.21.15.161:0/456482727 conn(0x7f86480b0bc0 msgr2=0x7f86480b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:30.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.338+0000 7f864ea00640 1 -- 172.21.15.161:0/456482727 shutdown_connections 2024-05-22T21:15:30.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.338+0000 7f864ea00640 1 -- 172.21.15.161:0/456482727 wait complete. 2024-05-22T21:15:30.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f864ea00640 1 Processor -- start 2024-05-22T21:15:30.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f864ea00640 1 -- start start 2024-05-22T21:15:30.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f864ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f864815d410 0x7f86480b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:30.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f864ea00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f864815dd50 0x7f8648164080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:30.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f864ea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f864815e130 0x7f86480b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:30.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f864ea00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8648157a80 con 0x7f864815d410 2024-05-22T21:15:30.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f864ea00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86480b1630 con 0x7f864815e130 2024-05-22T21:15:30.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f864ea00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8648156ab0 con 0x7f864815dd50 2024-05-22T21:15:30.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f8647e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f864815d410 0x7f86480b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:30.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f8647e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f864815d410 0x7f86480b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43078/0 (socket says 172.21.15.161:43078) 2024-05-22T21:15:30.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f8647e00640 1 -- 172.21.15.161:0/500955938 learned_addr learned my addr 172.21.15.161:0/500955938 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:30.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f863f400640 1 --2- 172.21.15.161:0/500955938 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f864815dd50 0x7f8648164080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:30.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f864ca00640 1 --2- 172.21.15.161:0/500955938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f864815e130 0x7f86480b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:30.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f8647e00640 1 -- 172.21.15.161:0/500955938 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f864815dd50 msgr2=0x7f8648164080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:30.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f8647e00640 1 --2- 172.21.15.161:0/500955938 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f864815dd50 0x7f8648164080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f8647e00640 1 -- 172.21.15.161:0/500955938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f864815e130 msgr2=0x7f86480b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:30.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f8647e00640 1 --2- 172.21.15.161:0/500955938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f864815e130 0x7f86480b2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f8647e00640 1 -- 172.21.15.161:0/500955938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8630009b30 con 0x7f864815d410 2024-05-22T21:15:30.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f8647e00640 1 --2- 172.21.15.161:0/500955938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f864815d410 0x7f86480b0fa0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f863800c500 tx=0x7f863800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:30.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f8644c00640 1 -- 172.21.15.161:0/500955938 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638005220 con 0x7f864815d410 2024-05-22T21:15:30.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f8644c00640 1 -- 172.21.15.161:0/500955938 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8638005380 con 0x7f864815d410 2024-05-22T21:15:30.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f8644c00640 1 -- 172.21.15.161:0/500955938 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638007270 con 0x7f864815d410 2024-05-22T21:15:30.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f864ea00640 1 -- 172.21.15.161:0/500955938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8648156cc0 con 0x7f864815d410 2024-05-22T21:15:30.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.342+0000 7f864ea00640 1 -- 172.21.15.161:0/500955938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8648157280 con 0x7f864815d410 2024-05-22T21:15:30.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.346+0000 7f864ea00640 1 -- 172.21.15.161:0/500955938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8648067220 con 0x7f864815d410 2024-05-22T21:15:30.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.346+0000 7f8644c00640 1 -- 172.21.15.161:0/500955938 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f86380073d0 con 0x7f864815d410 2024-05-22T21:15:30.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.346+0000 7f8644c00640 1 --2- 172.21.15.161:0/500955938 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8624066690 0x7f8624068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:30.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.346+0000 7f8644c00640 1 -- 172.21.15.161:0/500955938 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f86380837b0 con 0x7f864815d410 2024-05-22T21:15:30.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.346+0000 7f863f400640 1 --2- 172.21.15.161:0/500955938 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8624066690 0x7f8624068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:30.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.346+0000 7f863f400640 1 --2- 172.21.15.161:0/500955938 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8624066690 0x7f8624068b50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f86300ae000 tx=0x7f86300a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:30.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.350+0000 7f8644c00640 1 -- 172.21.15.161:0/500955938 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8638050af0 con 0x7f864815d410 2024-05-22T21:15:30.542 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.538+0000 7f864ea00640 1 -- 172.21.15.161:0/500955938 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f864806c310 con 0x7f864815d410 2024-05-22T21:15:30.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.538+0000 7f8644c00640 1 -- 172.21.15.161:0/500955938 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f8638054110 con 0x7f864815d410 2024-05-22T21:15:30.543 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:30.543 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/1767934947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1767934947},{"type":"v1","addr":"172.21.15.161:6836","nonce":1767934947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1873337287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1873337287},{"type":"v1","addr":"172.21.15.179:6834","nonce":1873337287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1815838236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":1815838236},{"type":"v1","addr":"172.21.15.161:6837","nonce":1815838236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:14:50.052292+0000","modified":"2024-05-22T21:14:53.058881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5374":{"gid":5374,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/389418050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":389418050},{"type":"v1","addr":"172.21.15.179:6835","nonce":389418050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-22T21:15:30.544 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 23 2024-05-22T21:15:30.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.538+0000 7f864ea00640 1 -- 172.21.15.161:0/500955938 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8624066690 msgr2=0x7f8624068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:30.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.538+0000 7f864ea00640 1 --2- 172.21.15.161:0/500955938 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8624066690 0x7f8624068b50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f86300ae000 tx=0x7f86300a7070 comp rx=0 tx=0).stop 2024-05-22T21:15:30.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.538+0000 7f864ea00640 1 -- 172.21.15.161:0/500955938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f864815d410 msgr2=0x7f86480b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:30.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.538+0000 7f864ea00640 1 --2- 172.21.15.161:0/500955938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f864815d410 0x7f86480b0fa0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f863800c500 tx=0x7f863800c9c0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.538+0000 7f864ea00640 1 -- 172.21.15.161:0/500955938 shutdown_connections 2024-05-22T21:15:30.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.538+0000 7f864ea00640 1 --2- 172.21.15.161:0/500955938 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8624066690 0x7f8624068b50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.538+0000 7f864ea00640 1 --2- 172.21.15.161:0/500955938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f864815e130 0x7f86480b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.538+0000 7f864ea00640 1 --2- 172.21.15.161:0/500955938 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f864815dd50 0x7f8648164080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.538+0000 7f864ea00640 1 --2- 172.21.15.161:0/500955938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f864815d410 0x7f86480b0fa0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.538+0000 7f864ea00640 1 -- 172.21.15.161:0/500955938 >> 172.21.15.161:0/500955938 conn(0x7f86480b0bc0 msgr2=0x7f8648152390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:30.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.538+0000 7f864ea00640 1 -- 172.21.15.161:0/500955938 shutdown_connections 2024-05-22T21:15:30.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.538+0000 7f864ea00640 1 -- 172.21.15.161:0/500955938 wait complete. 2024-05-22T21:15:30.565 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:15:30.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.694+0000 7f4d79a00640 1 Processor -- start 2024-05-22T21:15:30.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.694+0000 7f4d79a00640 1 -- start start 2024-05-22T21:15:30.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.694+0000 7f4d79a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4d74143ec0 0x7f4d7415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:30.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.694+0000 7f4d79a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d741442a0 con 0x7f4d740715a0 2024-05-22T21:15:30.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.694+0000 7f4d79a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d74070500 con 0x7f4d7415ec90 2024-05-22T21:15:30.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.694+0000 7f4d79a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d740707e0 con 0x7f4d74143ec0 2024-05-22T21:15:30.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.698+0000 7f4d72000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f4d740715a0 0x7f4d7406fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60486/0 (socket says 172.21.15.161:60486) 2024-05-22T21:15:30.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.698+0000 7f4d72000640 1 -- 172.21.15.161:0/2648779138 learned_addr learned my addr 172.21.15.161:0/2648779138 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:30.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.698+0000 7f4d72a00640 1 --2- 172.21.15.161:0/2648779138 >> v2:172.21.15.179:3300/0 conn(0x7f4d74143ec0 0x7f4d7415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:30.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.698+0000 7f4d71600640 1 -- 172.21.15.161:0/2648779138 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611695756 0 0) 0x7f4d741442a0 con 0x7f4d740715a0 2024-05-22T21:15:30.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.698+0000 7f4d71600640 1 -- 172.21.15.161:0/2648779138 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d58003530 con 0x7f4d740715a0 2024-05-22T21:15:30.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.698+0000 7f4d72a00640 1 -- 172.21.15.161:0/2648779138 >> v1:172.21.15.179:6789/0 conn(0x7f4d7415ec90 legacy=0x7f4d740cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:30.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.698+0000 7f4d72a00640 1 -- 172.21.15.161:0/2648779138 >> v1:172.21.15.161:6789/0 conn(0x7f4d740715a0 legacy=0x7f4d7406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:30.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.698+0000 7f4d72a00640 1 -- 172.21.15.161:0/2648779138 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d74070ac0 con 0x7f4d74143ec0 2024-05-22T21:15:30.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.698+0000 7f4d72a00640 1 --2- 172.21.15.161:0/2648779138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d74143ec0 0x7f4d7415f830 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4d640097c0 tx=0x7f4d6401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb0d35c0d45619d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:30.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d71600640 1 -- 172.21.15.161:0/2648779138 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d640ab670 con 0x7f4d74143ec0 2024-05-22T21:15:30.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d71600640 1 -- 172.21.15.161:0/2648779138 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d640ab7d0 con 0x7f4d74143ec0 2024-05-22T21:15:30.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d71600640 1 -- 172.21.15.161:0/2648779138 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d640abb60 con 0x7f4d74143ec0 2024-05-22T21:15:30.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d79a00640 1 -- 172.21.15.161:0/2648779138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d74143ec0 msgr2=0x7f4d7415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:30.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d79a00640 1 --2- 172.21.15.161:0/2648779138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d74143ec0 0x7f4d7415f830 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4d640097c0 tx=0x7f4d6401c470 comp rx=0 tx=0).stop 2024-05-22T21:15:30.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d79a00640 1 -- 172.21.15.161:0/2648779138 shutdown_connections 2024-05-22T21:15:30.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d79a00640 1 --2- 172.21.15.161:0/2648779138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d74143ec0 0x7f4d7415f830 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d79a00640 1 -- 172.21.15.161:0/2648779138 >> 172.21.15.161:0/2648779138 conn(0x7f4d740b0bc0 msgr2=0x7f4d740b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:30.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d79a00640 1 -- 172.21.15.161:0/2648779138 shutdown_connections 2024-05-22T21:15:30.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d79a00640 1 -- 172.21.15.161:0/2648779138 wait complete. 2024-05-22T21:15:30.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d79a00640 1 Processor -- start 2024-05-22T21:15:30.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d79a00640 1 -- start start 2024-05-22T21:15:30.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d79a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d74143ec0 0x7f4d740adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:30.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d79a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d7415ec90 0x7f4d740cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:30.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d79a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d74159b40 0x7f4d74158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:30.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d79a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740ac8b0 con 0x7f4d7415ec90 2024-05-22T21:15:30.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d79a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740acb90 con 0x7f4d74143ec0 2024-05-22T21:15:30.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d79a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740ace70 con 0x7f4d74159b40 2024-05-22T21:15:30.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d72000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d7415ec90 0x7f4d740cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:30.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d72000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d7415ec90 0x7f4d740cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43086/0 (socket says 172.21.15.161:43086) 2024-05-22T21:15:30.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.702+0000 7f4d72000640 1 -- 172.21.15.161:0/2082095127 learned_addr learned my addr 172.21.15.161:0/2082095127 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:30.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d73400640 1 --2- 172.21.15.161:0/2082095127 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d74159b40 0x7f4d74158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:30.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d72a00640 1 --2- 172.21.15.161:0/2082095127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d74143ec0 0x7f4d740adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:30.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d72000640 1 -- 172.21.15.161:0/2082095127 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d74159b40 msgr2=0x7f4d74158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:30.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d72000640 1 --2- 172.21.15.161:0/2082095127 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d74159b40 0x7f4d74158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d72000640 1 -- 172.21.15.161:0/2082095127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d74143ec0 msgr2=0x7f4d740adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:30.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d72000640 1 --2- 172.21.15.161:0/2082095127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d74143ec0 0x7f4d740adc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d72000640 1 -- 172.21.15.161:0/2082095127 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d64009420 con 0x7f4d7415ec90 2024-05-22T21:15:30.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d73400640 1 --2- 172.21.15.161:0/2082095127 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d74159b40 0x7f4d74158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:30.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d72a00640 1 --2- 172.21.15.161:0/2082095127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d74143ec0 0x7f4d740adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:30.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d72000640 1 --2- 172.21.15.161:0/2082095127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d7415ec90 0x7f4d740cef50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4d60001ec0 tx=0x7f4d6000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:30.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d6aa00640 1 -- 172.21.15.161:0/2082095127 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60005d50 con 0x7f4d7415ec90 2024-05-22T21:15:30.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d6aa00640 1 -- 172.21.15.161:0/2082095127 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d60007070 con 0x7f4d7415ec90 2024-05-22T21:15:30.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d6aa00640 1 -- 172.21.15.161:0/2082095127 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60007420 con 0x7f4d7415ec90 2024-05-22T21:15:30.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d79a00640 1 -- 172.21.15.161:0/2082095127 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d740ad150 con 0x7f4d7415ec90 2024-05-22T21:15:30.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d79a00640 1 -- 172.21.15.161:0/2082095127 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d740ad710 con 0x7f4d7415ec90 2024-05-22T21:15:30.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d79a00640 1 -- 172.21.15.161:0/2082095127 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d740ad930 con 0x7f4d7415ec90 2024-05-22T21:15:30.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d6aa00640 1 -- 172.21.15.161:0/2082095127 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4d60007580 con 0x7f4d7415ec90 2024-05-22T21:15:30.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d6aa00640 1 --2- 172.21.15.161:0/2082095127 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4d54066690 0x7f4d54068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:30.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d6aa00640 1 -- 172.21.15.161:0/2082095127 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4d600833c0 con 0x7f4d7415ec90 2024-05-22T21:15:30.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.706+0000 7f4d72a00640 1 --2- 172.21.15.161:0/2082095127 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4d54066690 0x7f4d54068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:30.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.710+0000 7f4d72a00640 1 --2- 172.21.15.161:0/2082095127 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4d54066690 0x7f4d54068b50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4d640af000 tx=0x7f4d64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:30.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.710+0000 7f4d6aa00640 1 -- 172.21.15.161:0/2082095127 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4d600507c0 con 0x7f4d7415ec90 2024-05-22T21:15:30.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.866+0000 7f4d79a00640 1 -- 172.21.15.161:0/2082095127 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d74067220 con 0x7f4d7415ec90 2024-05-22T21:15:30.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.866+0000 7f4d6aa00640 1 -- 172.21.15.161:0/2082095127 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16183 (secure 0 0 0) 0x7f4d60053de0 con 0x7f4d7415ec90 2024-05-22T21:15:30.871 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:30.871 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":55,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:15:17.769605+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:14:48.612726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-22T21:14:49.053022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.9","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:15:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.866+0000 7f4d79a00640 1 -- 172.21.15.161:0/2082095127 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4d54066690 msgr2=0x7f4d54068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.866+0000 7f4d79a00640 1 --2- 172.21.15.161:0/2082095127 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4d54066690 0x7f4d54068b50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4d640af000 tx=0x7f4d64008040 comp rx=0 tx=0).stop 2024-05-22T21:15:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.866+0000 7f4d79a00640 1 -- 172.21.15.161:0/2082095127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d7415ec90 msgr2=0x7f4d740cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.866+0000 7f4d79a00640 1 --2- 172.21.15.161:0/2082095127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d7415ec90 0x7f4d740cef50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4d60001ec0 tx=0x7f4d6000c960 comp rx=0 tx=0).stop 2024-05-22T21:15:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.866+0000 7f4d79a00640 1 -- 172.21.15.161:0/2082095127 shutdown_connections 2024-05-22T21:15:30.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.866+0000 7f4d79a00640 1 --2- 172.21.15.161:0/2082095127 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4d54066690 0x7f4d54068b50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.866+0000 7f4d79a00640 1 --2- 172.21.15.161:0/2082095127 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d74159b40 0x7f4d74158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.866+0000 7f4d79a00640 1 --2- 172.21.15.161:0/2082095127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d7415ec90 0x7f4d740cef50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.866+0000 7f4d79a00640 1 --2- 172.21.15.161:0/2082095127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d74143ec0 0x7f4d740adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:30.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.866+0000 7f4d79a00640 1 -- 172.21.15.161:0/2082095127 >> 172.21.15.161:0/2082095127 conn(0x7f4d740b0bc0 msgr2=0x7f4d7415d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:30.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.866+0000 7f4d79a00640 1 -- 172.21.15.161:0/2082095127 shutdown_connections 2024-05-22T21:15:30.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:30.866+0000 7f4d79a00640 1 -- 172.21.15.161:0/2082095127 wait complete. 2024-05-22T21:15:30.893 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-22T21:15:31.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef79c00640 1 Processor -- start 2024-05-22T21:15:31.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef79c00640 1 -- start start 2024-05-22T21:15:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef79c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef7413cfc0 0x7fef741402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef79c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fef7413fe50 0x7fef7406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef79c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7413d3a0 con 0x7fef7415af60 2024-05-22T21:15:31.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef79c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef740cd920 con 0x7fef7413fe50 2024-05-22T21:15:31.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef79c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef740cdc00 con 0x7fef7413cfc0 2024-05-22T21:15:31.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef73400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fef7413fe50 0x7fef7406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:31.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef72a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef7413cfc0 0x7fef741402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:31.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef6a000640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fef7415af60 0x7fef7413f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:36032/0 (socket says 172.21.15.161:36032) 2024-05-22T21:15:31.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef6a000640 1 -- 172.21.15.161:0/950529875 learned_addr learned my addr 172.21.15.161:0/950529875 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:31.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef72000640 1 -- 172.21.15.161:0/950529875 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1726687942 0 0) 0x7fef7413d3a0 con 0x7fef7415af60 2024-05-22T21:15:31.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef72000640 1 -- 172.21.15.161:0/950529875 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef58005740 con 0x7fef7415af60 2024-05-22T21:15:31.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef72a00640 1 -- 172.21.15.161:0/950529875 >> v2:172.21.15.179:3300/0 conn(0x7fef7413fe50 msgr2=0x7fef7406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:31.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef72a00640 1 --2- 172.21.15.161:0/950529875 >> v2:172.21.15.179:3300/0 conn(0x7fef7413fe50 0x7fef7406fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:31.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef72a00640 1 -- 172.21.15.161:0/950529875 >> v1:172.21.15.179:6789/0 conn(0x7fef7415af60 legacy=0x7fef7413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:31.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef72a00640 1 -- 172.21.15.161:0/950529875 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef740cdee0 con 0x7fef7413cfc0 2024-05-22T21:15:31.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.026+0000 7fef73400640 1 --2- 172.21.15.161:0/950529875 >> v2:172.21.15.179:3300/0 conn(0x7fef7413fe50 0x7fef7406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:15:31.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.030+0000 7fef72a00640 1 --2- 172.21.15.161:0/950529875 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef7413cfc0 0x7fef741402e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fef640097c0 tx=0x7fef6401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae0e90c4444ce462 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:31.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.030+0000 7fef72000640 1 -- 172.21.15.161:0/950529875 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef640ab600 con 0x7fef7413cfc0 2024-05-22T21:15:31.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.030+0000 7fef72000640 1 -- 172.21.15.161:0/950529875 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef640a7050 con 0x7fef7413cfc0 2024-05-22T21:15:31.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.030+0000 7fef72000640 1 -- 172.21.15.161:0/950529875 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef640ab980 con 0x7fef7413cfc0 2024-05-22T21:15:31.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.030+0000 7fef79c00640 1 -- 172.21.15.161:0/950529875 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef7413cfc0 msgr2=0x7fef741402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:31.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.030+0000 7fef79c00640 1 --2- 172.21.15.161:0/950529875 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef7413cfc0 0x7fef741402e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fef640097c0 tx=0x7fef6401c3e0 comp rx=0 tx=0).stop 2024-05-22T21:15:31.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.030+0000 7fef79c00640 1 -- 172.21.15.161:0/950529875 shutdown_connections 2024-05-22T21:15:31.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.030+0000 7fef79c00640 1 --2- 172.21.15.161:0/950529875 >> v2:172.21.15.179:3300/0 conn(0x7fef7413fe50 0x7fef7406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:31.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.030+0000 7fef79c00640 1 --2- 172.21.15.161:0/950529875 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef7413cfc0 0x7fef741402e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:31.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.030+0000 7fef79c00640 1 -- 172.21.15.161:0/950529875 >> 172.21.15.161:0/950529875 conn(0x7fef7406a2d0 msgr2=0x7fef7406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:31.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.030+0000 7fef79c00640 1 -- 172.21.15.161:0/950529875 shutdown_connections 2024-05-22T21:15:31.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.030+0000 7fef79c00640 1 -- 172.21.15.161:0/950529875 wait complete. 2024-05-22T21:15:31.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef79c00640 1 Processor -- start 2024-05-22T21:15:31.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef79c00640 1 -- start start 2024-05-22T21:15:31.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef79c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef7413cfc0 0x7fef741356c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef79c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef7413fe50 0x7fef740adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef79c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef7415af60 0x7fef74140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef79c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef74142360 con 0x7fef7415af60 2024-05-22T21:15:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef79c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef740ac8b0 con 0x7fef7413cfc0 2024-05-22T21:15:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef79c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef740acb00 con 0x7fef7413fe50 2024-05-22T21:15:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef73400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef7415af60 0x7fef74140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef73400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef7415af60 0x7fef74140270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43102/0 (socket says 172.21.15.161:43102) 2024-05-22T21:15:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef73400640 1 -- 172.21.15.161:0/1979438174 learned_addr learned my addr 172.21.15.161:0/1979438174 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef72a00640 1 --2- 172.21.15.161:0/1979438174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef7413cfc0 0x7fef741356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef6a000640 1 --2- 172.21.15.161:0/1979438174 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef7413fe50 0x7fef740adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef73400640 1 -- 172.21.15.161:0/1979438174 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef7413fe50 msgr2=0x7fef740adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef73400640 1 --2- 172.21.15.161:0/1979438174 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef7413fe50 0x7fef740adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:31.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef73400640 1 -- 172.21.15.161:0/1979438174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef7413cfc0 msgr2=0x7fef741356c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:31.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef73400640 1 --2- 172.21.15.161:0/1979438174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef7413cfc0 0x7fef741356c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:31.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef73400640 1 -- 172.21.15.161:0/1979438174 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c009a70 con 0x7fef7415af60 2024-05-22T21:15:31.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef73400640 1 --2- 172.21.15.161:0/1979438174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef7415af60 0x7fef74140270 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00efb0 tx=0x7fef6c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:31.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef6b400640 1 -- 172.21.15.161:0/1979438174 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c009d50 con 0x7fef7415af60 2024-05-22T21:15:31.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef6b400640 1 -- 172.21.15.161:0/1979438174 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6c00f040 con 0x7fef7415af60 2024-05-22T21:15:31.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef6b400640 1 -- 172.21.15.161:0/1979438174 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c0146a0 con 0x7fef7415af60 2024-05-22T21:15:31.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef79c00640 1 -- 172.21.15.161:0/1979438174 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef64009420 con 0x7fef7415af60 2024-05-22T21:15:31.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef79c00640 1 -- 172.21.15.161:0/1979438174 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef740ad0c0 con 0x7fef7415af60 2024-05-22T21:15:31.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.034+0000 7fef79c00640 1 -- 172.21.15.161:0/1979438174 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef74067220 con 0x7fef7415af60 2024-05-22T21:15:31.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.038+0000 7fef6b400640 1 -- 172.21.15.161:0/1979438174 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fef6c014800 con 0x7fef7415af60 2024-05-22T21:15:31.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.038+0000 7fef6b400640 1 --2- 172.21.15.161:0/1979438174 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fef50066690 0x7fef50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:31.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.038+0000 7fef6b400640 1 -- 172.21.15.161:0/1979438174 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fef6c085140 con 0x7fef7415af60 2024-05-22T21:15:31.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.038+0000 7fef72a00640 1 --2- 172.21.15.161:0/1979438174 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fef50066690 0x7fef50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:31.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.038+0000 7fef72a00640 1 --2- 172.21.15.161:0/1979438174 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fef50066690 0x7fef50068b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fef640ad000 tx=0x7fef640a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:31.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.042+0000 7fef6b400640 1 -- 172.21.15.161:0/1979438174 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fef6c052400 con 0x7fef7415af60 2024-05-22T21:15:31.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.226+0000 7fef79c00640 1 -- 172.21.15.161:0/1979438174 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fef7406cab0 con 0x7fef7415af60 2024-05-22T21:15:31.232 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:15:31.226+0000 7f7d32800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-22T21:15:31.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.234+0000 7fef6b400640 1 -- 172.21.15.161:0/1979438174 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7fef6c055a20 con 0x7fef7415af60 2024-05-22T21:15:31.241 INFO:teuthology.orchestra.run.smithi161.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-22T21:15:31.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.238+0000 7fef79c00640 1 -- 172.21.15.161:0/1979438174 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fef50066690 msgr2=0x7fef50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:31.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.238+0000 7fef79c00640 1 --2- 172.21.15.161:0/1979438174 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fef50066690 0x7fef50068b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fef640ad000 tx=0x7fef640a6000 comp rx=0 tx=0).stop 2024-05-22T21:15:31.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.238+0000 7fef79c00640 1 -- 172.21.15.161:0/1979438174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef7415af60 msgr2=0x7fef74140270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:31.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.238+0000 7fef79c00640 1 --2- 172.21.15.161:0/1979438174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef7415af60 0x7fef74140270 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00efb0 tx=0x7fef6c00c490 comp rx=0 tx=0).stop 2024-05-22T21:15:31.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.238+0000 7fef79c00640 1 -- 172.21.15.161:0/1979438174 shutdown_connections 2024-05-22T21:15:31.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.238+0000 7fef79c00640 1 --2- 172.21.15.161:0/1979438174 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fef50066690 0x7fef50068b50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:31.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.238+0000 7fef79c00640 1 --2- 172.21.15.161:0/1979438174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef7415af60 0x7fef74140270 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:31.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.238+0000 7fef79c00640 1 --2- 172.21.15.161:0/1979438174 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef7413fe50 0x7fef740adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:31.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.238+0000 7fef79c00640 1 --2- 172.21.15.161:0/1979438174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef7413cfc0 0x7fef741356c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:31.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.238+0000 7fef79c00640 1 -- 172.21.15.161:0/1979438174 >> 172.21.15.161:0/1979438174 conn(0x7fef7406a2d0 msgr2=0x7fef740af540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:31.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.238+0000 7fef79c00640 1 -- 172.21.15.161:0/1979438174 shutdown_connections 2024-05-22T21:15:31.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.238+0000 7fef79c00640 1 -- 172.21.15.161:0/1979438174 wait complete. 2024-05-22T21:15:31.262 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-22T21:15:31.307 INFO:tasks.ceph.mds.b.smithi179.stdout:starting mds.b at 2024-05-22T21:15:31.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.394+0000 7fbeba600640 1 Processor -- start 2024-05-22T21:15:31.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.394+0000 7fbeba600640 1 -- start start 2024-05-22T21:15:31.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.394+0000 7fbeba600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbeb4143c40 0x7fbeb415b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:31.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.394+0000 7fbeba600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbeb406c9b0 0x7fbeb40b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:31.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.394+0000 7fbeba600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb415efb0 con 0x7fbeb41444a0 2024-05-22T21:15:31.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.394+0000 7fbeba600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb415f290 con 0x7fbeb4143c40 2024-05-22T21:15:31.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.394+0000 7fbeba600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb415f570 con 0x7fbeb406c9b0 2024-05-22T21:15:31.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.394+0000 7fbeb3400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbeb4143c40 0x7fbeb415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:31.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.394+0000 7fbeaaa00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fbeb41444a0 0x7fbeb406c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60492/0 (socket says 172.21.15.161:60492) 2024-05-22T21:15:31.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.394+0000 7fbeaaa00640 1 -- 172.21.15.161:0/4192880471 learned_addr learned my addr 172.21.15.161:0/4192880471 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:31.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.394+0000 7fbeb3e00640 1 --2- 172.21.15.161:0/4192880471 >> v2:172.21.15.179:3300/0 conn(0x7fbeb406c9b0 0x7fbeb40b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:31.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.394+0000 7fbeb3400640 1 -- 172.21.15.161:0/4192880471 >> v2:172.21.15.179:3300/0 conn(0x7fbeb406c9b0 msgr2=0x7fbeb40b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:31.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.394+0000 7fbeb3400640 1 --2- 172.21.15.161:0/4192880471 >> v2:172.21.15.179:3300/0 conn(0x7fbeb406c9b0 0x7fbeb40b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:31.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.394+0000 7fbeb3400640 1 -- 172.21.15.161:0/4192880471 >> v1:172.21.15.161:6789/0 conn(0x7fbeb41444a0 legacy=0x7fbeb406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:31.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.394+0000 7fbeb2a00640 1 -- 172.21.15.161:0/4192880471 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084762793 0 0) 0x7fbeb415efb0 con 0x7fbeb41444a0 2024-05-22T21:15:31.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.394+0000 7fbeb3400640 1 -- 172.21.15.161:0/4192880471 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb415f850 con 0x7fbeb4143c40 2024-05-22T21:15:31.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.398+0000 7fbeb3400640 1 --2- 172.21.15.161:0/4192880471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb4143c40 0x7fbeb415b600 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40aead0 tx=0x7fbea401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39038fa5cd8a35c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:31.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.398+0000 7fbeb2a00640 1 -- 172.21.15.161:0/4192880471 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea40ac640 con 0x7fbeb4143c40 2024-05-22T21:15:31.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.398+0000 7fbeb2a00640 1 -- 172.21.15.161:0/4192880471 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea40ac7a0 con 0x7fbeb4143c40 2024-05-22T21:15:31.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.398+0000 7fbeb2a00640 1 -- 172.21.15.161:0/4192880471 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea40acb30 con 0x7fbeb4143c40 2024-05-22T21:15:31.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.398+0000 7fbeba600640 1 -- 172.21.15.161:0/4192880471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb4143c40 msgr2=0x7fbeb415b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:31.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.398+0000 7fbeba600640 1 --2- 172.21.15.161:0/4192880471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb4143c40 0x7fbeb415b600 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40aead0 tx=0x7fbea401c480 comp rx=0 tx=0).stop 2024-05-22T21:15:31.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.398+0000 7fbeba600640 1 -- 172.21.15.161:0/4192880471 shutdown_connections 2024-05-22T21:15:31.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.398+0000 7fbeba600640 1 --2- 172.21.15.161:0/4192880471 >> v2:172.21.15.179:3300/0 conn(0x7fbeb406c9b0 0x7fbeb40b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:31.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.398+0000 7fbeba600640 1 --2- 172.21.15.161:0/4192880471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb4143c40 0x7fbeb415b600 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:31.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.398+0000 7fbeba600640 1 -- 172.21.15.161:0/4192880471 >> 172.21.15.161:0/4192880471 conn(0x7fbeb406a2d0 msgr2=0x7fbeb406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:31.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.398+0000 7fbeba600640 1 -- 172.21.15.161:0/4192880471 shutdown_connections 2024-05-22T21:15:31.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.398+0000 7fbeba600640 1 -- 172.21.15.161:0/4192880471 wait complete. 2024-05-22T21:15:31.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeba600640 1 Processor -- start 2024-05-22T21:15:31.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeba600640 1 -- start start 2024-05-22T21:15:31.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeba600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb406c9b0 0x7fbeb4133fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:31.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeba600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb4143c40 0x7fbeb40cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:31.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeba600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbeb41444a0 0x7fbeb40ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:31.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeba600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb415b0e0 con 0x7fbeb406c9b0 2024-05-22T21:15:31.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeba600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb415a560 con 0x7fbeb41444a0 2024-05-22T21:15:31.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeba600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb415a840 con 0x7fbeb4143c40 2024-05-22T21:15:31.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeb3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb406c9b0 0x7fbeb4133fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:31.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeb3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb406c9b0 0x7fbeb4133fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43126/0 (socket says 172.21.15.161:43126) 2024-05-22T21:15:31.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeb3400640 1 -- 172.21.15.161:0/4039712756 learned_addr learned my addr 172.21.15.161:0/4039712756 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:31.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeb3e00640 1 --2- 172.21.15.161:0/4039712756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbeb41444a0 0x7fbeb40ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:31.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeaaa00640 1 --2- 172.21.15.161:0/4039712756 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb4143c40 0x7fbeb40cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:31.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeb3400640 1 -- 172.21.15.161:0/4039712756 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb4143c40 msgr2=0x7fbeb40cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:31.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeb3400640 1 --2- 172.21.15.161:0/4039712756 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb4143c40 0x7fbeb40cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:31.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeb3400640 1 -- 172.21.15.161:0/4039712756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbeb41444a0 msgr2=0x7fbeb40ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:31.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeb3400640 1 --2- 172.21.15.161:0/4039712756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbeb41444a0 0x7fbeb40ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:31.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeb3400640 1 -- 172.21.15.161:0/4039712756 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea4009420 con 0x7fbeb406c9b0 2024-05-22T21:15:31.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeaaa00640 1 --2- 172.21.15.161:0/4039712756 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb4143c40 0x7fbeb40cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:31.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeb3400640 1 --2- 172.21.15.161:0/4039712756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb406c9b0 0x7fbeb4133fc0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbea40af040 tx=0x7fbea401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:31.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeabe00640 1 -- 172.21.15.161:0/4039712756 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea40b8650 con 0x7fbeb406c9b0 2024-05-22T21:15:31.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeabe00640 1 -- 172.21.15.161:0/4039712756 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea40b87d0 con 0x7fbeb406c9b0 2024-05-22T21:15:31.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeabe00640 1 -- 172.21.15.161:0/4039712756 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea40b8b80 con 0x7fbeb406c9b0 2024-05-22T21:15:31.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeba600640 1 -- 172.21.15.161:0/4039712756 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb415ab20 con 0x7fbeb406c9b0 2024-05-22T21:15:31.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.402+0000 7fbeba600640 1 -- 172.21.15.161:0/4039712756 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb4134500 con 0x7fbeb406c9b0 2024-05-22T21:15:31.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.406+0000 7fbeba600640 1 -- 172.21.15.161:0/4039712756 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb4067220 con 0x7fbeb406c9b0 2024-05-22T21:15:31.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.406+0000 7fbeabe00640 1 -- 172.21.15.161:0/4039712756 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbea40b8ce0 con 0x7fbeb406c9b0 2024-05-22T21:15:31.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.406+0000 7fbeabe00640 1 --2- 172.21.15.161:0/4039712756 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbe90066690 0x7fbe90068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:31.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.406+0000 7fbeabe00640 1 -- 172.21.15.161:0/4039712756 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fbea411ec90 con 0x7fbeb406c9b0 2024-05-22T21:15:31.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.406+0000 7fbeaaa00640 1 --2- 172.21.15.161:0/4039712756 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbe90066690 0x7fbe90068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:31.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.406+0000 7fbeaaa00640 1 --2- 172.21.15.161:0/4039712756 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbe90066690 0x7fbe90068b50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c007fd0 tx=0x7fbe9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:31.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.410+0000 7fbeabe00640 1 -- 172.21.15.161:0/4039712756 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbea40ec000 con 0x7fbeb406c9b0 2024-05-22T21:15:31.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:31.598+0000 7fbeba600640 1 -- 172.21.15.161:0/4039712756 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbeb40ceaa0 con 0x7fbeb406c9b0 2024-05-22T21:15:32.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.238+0000 7fbeabe00640 1 -- 172.21.15.161:0/4039712756 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbea40ef620 con 0x7fbeb406c9b0 2024-05-22T21:15:32.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.242+0000 7fbeba600640 1 -- 172.21.15.161:0/4039712756 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbe90066690 msgr2=0x7fbe90068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:32.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.242+0000 7fbeba600640 1 --2- 172.21.15.161:0/4039712756 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbe90066690 0x7fbe90068b50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c007fd0 tx=0x7fbe9c009040 comp rx=0 tx=0).stop 2024-05-22T21:15:32.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.242+0000 7fbeba600640 1 -- 172.21.15.161:0/4039712756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb406c9b0 msgr2=0x7fbeb4133fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:32.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.242+0000 7fbeba600640 1 --2- 172.21.15.161:0/4039712756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb406c9b0 0x7fbeb4133fc0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbea40af040 tx=0x7fbea401acc0 comp rx=0 tx=0).stop 2024-05-22T21:15:32.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.242+0000 7fbeba600640 1 -- 172.21.15.161:0/4039712756 shutdown_connections 2024-05-22T21:15:32.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.242+0000 7fbeba600640 1 --2- 172.21.15.161:0/4039712756 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbe90066690 0x7fbe90068b50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:32.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.242+0000 7fbeba600640 1 --2- 172.21.15.161:0/4039712756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbeb41444a0 0x7fbeb40ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:32.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.242+0000 7fbeba600640 1 --2- 172.21.15.161:0/4039712756 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb4143c40 0x7fbeb40cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:32.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.242+0000 7fbeba600640 1 --2- 172.21.15.161:0/4039712756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbeb406c9b0 0x7fbeb4133fc0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:32.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.242+0000 7fbeba600640 1 -- 172.21.15.161:0/4039712756 >> 172.21.15.161:0/4039712756 conn(0x7fbeb406a2d0 msgr2=0x7fbeb4142c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:32.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.242+0000 7fbeba600640 1 -- 172.21.15.161:0/4039712756 shutdown_connections 2024-05-22T21:15:32.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.242+0000 7fbeba600640 1 -- 172.21.15.161:0/4039712756 wait complete. 2024-05-22T21:15:32.268 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-22T21:15:32.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.402+0000 7f40a9c00640 1 Processor -- start 2024-05-22T21:15:32.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.402+0000 7f40a9c00640 1 -- start start 2024-05-22T21:15:32.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a9c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f40a415e1d0 0x7f40a40ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:32.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a9c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a40ce830 con 0x7f40a413d560 2024-05-22T21:15:32.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a9c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a40ceb10 con 0x7f40a413cfe0 2024-05-22T21:15:32.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a9c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a4140560 con 0x7f40a415e1d0 2024-05-22T21:15:32.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f409a000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f40a413d560 0x7f40a40cdb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60504/0 (socket says 172.21.15.161:60504) 2024-05-22T21:15:32.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f409a000640 1 -- 172.21.15.161:0/323029942 learned_addr learned my addr 172.21.15.161:0/323029942 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:32.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a3400640 1 --2- 172.21.15.161:0/323029942 >> v2:172.21.15.179:3300/0 conn(0x7f40a415e1d0 0x7f40a40ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:32.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 -- 172.21.15.161:0/323029942 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854805490 0 0) 0x7f40a40ce830 con 0x7f40a413d560 2024-05-22T21:15:32.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 -- 172.21.15.161:0/323029942 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4088003530 con 0x7f40a413d560 2024-05-22T21:15:32.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 -- 172.21.15.161:0/323029942 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822135363 0 0) 0x7f40a40ceb10 con 0x7f40a413cfe0 2024-05-22T21:15:32.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 -- 172.21.15.161:0/323029942 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40a40ce830 con 0x7f40a413cfe0 2024-05-22T21:15:32.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 -- 172.21.15.161:0/323029942 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2079326191 0 0) 0x7f4088003530 con 0x7f40a413d560 2024-05-22T21:15:32.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 -- 172.21.15.161:0/323029942 >> v2:172.21.15.179:3300/0 conn(0x7f40a415e1d0 msgr2=0x7f40a40ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:32.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 --2- 172.21.15.161:0/323029942 >> v2:172.21.15.179:3300/0 conn(0x7f40a415e1d0 0x7f40a40ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:32.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 -- 172.21.15.161:0/323029942 >> v1:172.21.15.179:6789/0 conn(0x7f40a413cfe0 legacy=0x7f40a415dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:32.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 -- 172.21.15.161:0/323029942 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a406fd80 con 0x7f40a413d560 2024-05-22T21:15:32.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a3400640 1 --2- 172.21.15.161:0/323029942 >> v2:172.21.15.179:3300/0 conn(0x7f40a415e1d0 0x7f40a40ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:15:32.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 -- 172.21.15.161:0/323029942 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f408c001680 con 0x7f40a413d560 2024-05-22T21:15:32.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 -- 172.21.15.161:0/323029942 >> v1:172.21.15.161:6789/0 conn(0x7f40a413d560 legacy=0x7f40a40cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:32.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 --2- 172.21.15.161:0/323029942 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4088003b10 0x7f4088003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:32.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 --2- 172.21.15.161:0/323029942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40880047d0 0x7f4088004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:32.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 --2- 172.21.15.161:0/323029942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4088005190 0x7f408800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:32.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 -- 172.21.15.161:0/323029942 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a4140560 con 0x7f4088005190 2024-05-22T21:15:32.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 -- 172.21.15.161:0/323029942 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408800c0c0 con 0x7f40880047d0 2024-05-22T21:15:32.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2000640 1 -- 172.21.15.161:0/323029942 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408800c280 con 0x7f4088003b10 2024-05-22T21:15:32.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a3400640 1 --2- 172.21.15.161:0/323029942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4088005190 0x7f408800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:32.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f409a000640 1 --2- 172.21.15.161:0/323029942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40880047d0 0x7f4088004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:32.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a2a00640 1 --2- 172.21.15.161:0/323029942 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4088003b10 0x7f4088003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:32.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a3400640 1 -- 172.21.15.161:0/323029942 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4088003b10 msgr2=0x7f4088003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:32.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a3400640 1 --2- 172.21.15.161:0/323029942 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4088003b10 0x7f4088003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:32.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a3400640 1 -- 172.21.15.161:0/323029942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40880047d0 msgr2=0x7f4088004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:32.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a3400640 1 --2- 172.21.15.161:0/323029942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40880047d0 0x7f4088004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:32.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.406+0000 7f40a3400640 1 -- 172.21.15.161:0/323029942 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4088003530 con 0x7f4088005190 2024-05-22T21:15:32.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.410+0000 7f40a3400640 1 -- 172.21.15.161:0/323029942 reap_dead start 2024-05-22T21:15:32.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.410+0000 7f40a3400640 1 --2- 172.21.15.161:0/323029942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4088005190 0x7f408800b990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f409c00e350 tx=0x7f409c01ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971b82bc41dca688 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:32.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.410+0000 7f40a2000640 1 -- 172.21.15.161:0/323029942 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c0a5e90 con 0x7f4088005190 2024-05-22T21:15:32.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.410+0000 7f40a2000640 1 -- 172.21.15.161:0/323029942 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c00f040 con 0x7f4088005190 2024-05-22T21:15:32.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.410+0000 7f40a2000640 1 -- 172.21.15.161:0/323029942 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c0ad660 con 0x7f4088005190 2024-05-22T21:15:32.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.410+0000 7f40a9c00640 1 -- 172.21.15.161:0/323029942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4088005190 msgr2=0x7f408800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:32.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.410+0000 7f40a9c00640 1 --2- 172.21.15.161:0/323029942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4088005190 0x7f408800b990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f409c00e350 tx=0x7f409c01ce80 comp rx=0 tx=0).stop 2024-05-22T21:15:32.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.410+0000 7f40a9c00640 1 -- 172.21.15.161:0/323029942 shutdown_connections 2024-05-22T21:15:32.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.410+0000 7f40a9c00640 1 --2- 172.21.15.161:0/323029942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4088005190 0x7f408800b990 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:32.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.410+0000 7f40a9c00640 1 -- 172.21.15.161:0/323029942 >> 172.21.15.161:0/323029942 conn(0x7f40a406a0a0 msgr2=0x7f40a406a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:32.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.410+0000 7f40a9c00640 1 -- 172.21.15.161:0/323029942 shutdown_connections 2024-05-22T21:15:32.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.410+0000 7f40a9c00640 1 -- 172.21.15.161:0/323029942 wait complete. 2024-05-22T21:15:32.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a9c00640 1 Processor -- start 2024-05-22T21:15:32.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a9c00640 1 -- start start 2024-05-22T21:15:32.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a9c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4088005190 0x7f40a41406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:32.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a9c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40a413aca0 0x7f40a4141f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:32.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a9c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40a40b3370 0x7f40a4132ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:32.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a9c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a4140ef0 con 0x7f40a40b3370 2024-05-22T21:15:32.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a9c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a41411d0 con 0x7f4088005190 2024-05-22T21:15:32.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a9c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a41414b0 con 0x7f40a413aca0 2024-05-22T21:15:32.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40a40b3370 0x7f40a4132ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:32.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40a40b3370 0x7f40a4132ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43142/0 (socket says 172.21.15.161:43142) 2024-05-22T21:15:32.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a3400640 1 -- 172.21.15.161:0/3490791123 learned_addr learned my addr 172.21.15.161:0/3490791123 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:32.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a2a00640 1 --2- 172.21.15.161:0/3490791123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4088005190 0x7f40a41406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:32.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f409a000640 1 --2- 172.21.15.161:0/3490791123 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40a413aca0 0x7f40a4141f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:32.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a3400640 1 -- 172.21.15.161:0/3490791123 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40a413aca0 msgr2=0x7f40a4141f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:32.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a3400640 1 --2- 172.21.15.161:0/3490791123 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40a413aca0 0x7f40a4141f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:32.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a3400640 1 -- 172.21.15.161:0/3490791123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4088005190 msgr2=0x7f40a41406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:32.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a3400640 1 --2- 172.21.15.161:0/3490791123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4088005190 0x7f40a41406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:32.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a3400640 1 -- 172.21.15.161:0/3490791123 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a4141790 con 0x7f40a40b3370 2024-05-22T21:15:32.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a3400640 1 --2- 172.21.15.161:0/3490791123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40a40b3370 0x7f40a4132ef0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f409c01d720 tx=0x7f409c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:32.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f409b400640 1 -- 172.21.15.161:0/3490791123 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c0b63f0 con 0x7f40a40b3370 2024-05-22T21:15:32.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f409b400640 1 -- 172.21.15.161:0/3490791123 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c0b6550 con 0x7f40a40b3370 2024-05-22T21:15:32.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a9c00640 1 -- 172.21.15.161:0/3490791123 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a4141a70 con 0x7f40a40b3370 2024-05-22T21:15:32.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f409b400640 1 -- 172.21.15.161:0/3490791123 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c0b68e0 con 0x7f40a40b3370 2024-05-22T21:15:32.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a9c00640 1 -- 172.21.15.161:0/3490791123 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a41424d0 con 0x7f40a40b3370 2024-05-22T21:15:32.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.414+0000 7f40a9c00640 1 -- 172.21.15.161:0/3490791123 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a4067220 con 0x7f40a40b3370 2024-05-22T21:15:32.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.418+0000 7f409b400640 1 -- 172.21.15.161:0/3490791123 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f409c0b4040 con 0x7f40a40b3370 2024-05-22T21:15:32.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.418+0000 7f409b400640 1 --2- 172.21.15.161:0/3490791123 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4080066680 0x7f4080068b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:32.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.418+0000 7f409b400640 1 -- 172.21.15.161:0/3490791123 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f409c11e610 con 0x7f40a40b3370 2024-05-22T21:15:32.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.418+0000 7f40a2a00640 1 --2- 172.21.15.161:0/3490791123 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4080066680 0x7f4080068b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:32.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.418+0000 7f40a2a00640 1 --2- 172.21.15.161:0/3490791123 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4080066680 0x7f4080068b40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4094005630 tx=0x7f4094009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:32.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.422+0000 7f409b400640 1 -- 172.21.15.161:0/3490791123 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f409c0eb900 con 0x7f40a40b3370 2024-05-22T21:15:32.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:32.578+0000 7f40a9c00640 1 -- 172.21.15.161:0/3490791123 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f40a4071d50 con 0x7f40a40b3370 2024-05-22T21:15:33.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.514+0000 7f409b400640 1 -- 172.21.15.161:0/3490791123 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) v1 ==== 181+0+0 (secure 0 0 0) 0x7f409c009de0 con 0x7f40a40b3370 2024-05-22T21:15:33.522 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' removed 2024-05-22T21:15:33.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.518+0000 7f40a9c00640 1 -- 172.21.15.161:0/3490791123 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4080066680 msgr2=0x7f4080068b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:33.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.518+0000 7f40a9c00640 1 --2- 172.21.15.161:0/3490791123 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4080066680 0x7f4080068b40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4094005630 tx=0x7f4094009040 comp rx=0 tx=0).stop 2024-05-22T21:15:33.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.518+0000 7f40a9c00640 1 -- 172.21.15.161:0/3490791123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40a40b3370 msgr2=0x7f40a4132ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:33.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.518+0000 7f40a9c00640 1 --2- 172.21.15.161:0/3490791123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40a40b3370 0x7f40a4132ef0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f409c01d720 tx=0x7f409c009750 comp rx=0 tx=0).stop 2024-05-22T21:15:33.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.518+0000 7f40a9c00640 1 -- 172.21.15.161:0/3490791123 shutdown_connections 2024-05-22T21:15:33.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.518+0000 7f40a9c00640 1 --2- 172.21.15.161:0/3490791123 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4080066680 0x7f4080068b40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:33.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.518+0000 7f40a9c00640 1 --2- 172.21.15.161:0/3490791123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40a40b3370 0x7f40a4132ef0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:33.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.518+0000 7f40a9c00640 1 --2- 172.21.15.161:0/3490791123 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40a413aca0 0x7f40a4141f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:33.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.518+0000 7f40a9c00640 1 --2- 172.21.15.161:0/3490791123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4088005190 0x7f40a41406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:33.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.518+0000 7f40a9c00640 1 -- 172.21.15.161:0/3490791123 >> 172.21.15.161:0/3490791123 conn(0x7f40a406a0a0 msgr2=0x7f40a415bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:33.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.518+0000 7f40a9c00640 1 -- 172.21.15.161:0/3490791123 shutdown_connections 2024-05-22T21:15:33.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.518+0000 7f40a9c00640 1 -- 172.21.15.161:0/3490791123 wait complete. 2024-05-22T21:15:33.548 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-22T21:15:33.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad29c00640 1 Processor -- start 2024-05-22T21:15:33.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad29c00640 1 -- start start 2024-05-22T21:15:33.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad29c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fad2413cfe0 0x7fad2415dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:33.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad29c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fad2413d560 0x7fad2415e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:33.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad29c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad240cdb00 0x7fad240cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:33.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad29c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad24140560 con 0x7fad2413cfe0 2024-05-22T21:15:33.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad29c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad240ce830 con 0x7fad2413d560 2024-05-22T21:15:33.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad29c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad240ceb10 con 0x7fad240cdb00 2024-05-22T21:15:33.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad22a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fad2413cfe0 0x7fad2415dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:33.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad22a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fad2413cfe0 0x7fad2415dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43148/0 (socket says 172.21.15.161:43148) 2024-05-22T21:15:33.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad22a00640 1 -- 172.21.15.161:0/582322529 learned_addr learned my addr 172.21.15.161:0/582322529 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:33.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad1a000640 1 --2- 172.21.15.161:0/582322529 >> v2:172.21.15.179:3300/0 conn(0x7fad2413d560 0x7fad2415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:33.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad23400640 1 --2- 172.21.15.161:0/582322529 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad240cdb00 0x7fad240cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:33.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad22a00640 1 -- 172.21.15.161:0/582322529 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad240cdb00 msgr2=0x7fad240cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:33.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad22a00640 1 --2- 172.21.15.161:0/582322529 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad240cdb00 0x7fad240cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:33.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad22a00640 1 -- 172.21.15.161:0/582322529 >> v2:172.21.15.179:3300/0 conn(0x7fad2413d560 msgr2=0x7fad2415e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:33.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad22a00640 1 --2- 172.21.15.161:0/582322529 >> v2:172.21.15.179:3300/0 conn(0x7fad2413d560 0x7fad2415e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:33.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad22a00640 1 -- 172.21.15.161:0/582322529 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2406fd80 con 0x7fad2413cfe0 2024-05-22T21:15:33.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.686+0000 7fad23400640 1 --2- 172.21.15.161:0/582322529 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad240cdb00 0x7fad240cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:33.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.690+0000 7fad22a00640 1 --2- 172.21.15.161:0/582322529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2413cfe0 0x7fad2415dac0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fad2406de40 tx=0x7fad1401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55863b14e90a3165 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:33.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.690+0000 7fad22000640 1 -- 172.21.15.161:0/582322529 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad140ab620 con 0x7fad2413cfe0 2024-05-22T21:15:33.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.690+0000 7fad22000640 1 -- 172.21.15.161:0/582322529 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad140ab780 con 0x7fad2413cfe0 2024-05-22T21:15:33.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.690+0000 7fad22000640 1 -- 172.21.15.161:0/582322529 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad140abb10 con 0x7fad2413cfe0 2024-05-22T21:15:33.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.690+0000 7fad29c00640 1 -- 172.21.15.161:0/582322529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2413cfe0 msgr2=0x7fad2415dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:33.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.690+0000 7fad29c00640 1 --2- 172.21.15.161:0/582322529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2413cfe0 0x7fad2415dac0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fad2406de40 tx=0x7fad1401c460 comp rx=0 tx=0).stop 2024-05-22T21:15:33.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.690+0000 7fad29c00640 1 -- 172.21.15.161:0/582322529 shutdown_connections 2024-05-22T21:15:33.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.690+0000 7fad29c00640 1 --2- 172.21.15.161:0/582322529 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad240cdb00 0x7fad240cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:33.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.690+0000 7fad29c00640 1 --2- 172.21.15.161:0/582322529 >> v2:172.21.15.179:3300/0 conn(0x7fad2413d560 0x7fad2415e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:33.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.690+0000 7fad29c00640 1 --2- 172.21.15.161:0/582322529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2413cfe0 0x7fad2415dac0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:33.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.690+0000 7fad29c00640 1 -- 172.21.15.161:0/582322529 >> 172.21.15.161:0/582322529 conn(0x7fad2406a0a0 msgr2=0x7fad2406a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:33.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.690+0000 7fad29c00640 1 -- 172.21.15.161:0/582322529 shutdown_connections 2024-05-22T21:15:33.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.690+0000 7fad29c00640 1 -- 172.21.15.161:0/582322529 wait complete. 2024-05-22T21:15:33.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad29c00640 1 Processor -- start 2024-05-22T21:15:33.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad29c00640 1 -- start start 2024-05-22T21:15:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad29c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad240cdb00 0x7fad24142270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad29c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad2413cfe0 0x7fad24154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad29c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2413d560 0x7fad2406ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad29c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad24156c90 con 0x7fad2413d560 2024-05-22T21:15:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad29c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad24140bb0 con 0x7fad2413cfe0 2024-05-22T21:15:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad29c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad241566d0 con 0x7fad240cdb00 2024-05-22T21:15:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad23400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2413d560 0x7fad2406ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad23400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2413d560 0x7fad2406ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43164/0 (socket says 172.21.15.161:43164) 2024-05-22T21:15:33.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad23400640 1 -- 172.21.15.161:0/3367248076 learned_addr learned my addr 172.21.15.161:0/3367248076 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:33.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad22a00640 1 --2- 172.21.15.161:0/3367248076 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad240cdb00 0x7fad24142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:33.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad1a000640 1 --2- 172.21.15.161:0/3367248076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad2413cfe0 0x7fad24154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:33.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad23400640 1 -- 172.21.15.161:0/3367248076 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad240cdb00 msgr2=0x7fad24142270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:33.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad23400640 1 --2- 172.21.15.161:0/3367248076 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad240cdb00 0x7fad24142270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:33.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad23400640 1 -- 172.21.15.161:0/3367248076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad2413cfe0 msgr2=0x7fad24154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:33.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad23400640 1 --2- 172.21.15.161:0/3367248076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad2413cfe0 0x7fad24154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:33.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad23400640 1 -- 172.21.15.161:0/3367248076 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad140094d0 con 0x7fad2413d560 2024-05-22T21:15:33.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad23400640 1 --2- 172.21.15.161:0/3367248076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2413d560 0x7fad2406ca20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00cef0 tx=0x7fad1c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:33.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad1b400640 1 -- 172.21.15.161:0/3367248076 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c00d840 con 0x7fad2413d560 2024-05-22T21:15:33.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad1b400640 1 -- 172.21.15.161:0/3367248076 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1c00d9a0 con 0x7fad2413d560 2024-05-22T21:15:33.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad1b400640 1 -- 172.21.15.161:0/3367248076 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c00dd50 con 0x7fad2413d560 2024-05-22T21:15:33.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad29c00640 1 -- 172.21.15.161:0/3367248076 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad241569b0 con 0x7fad2413d560 2024-05-22T21:15:33.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad29c00640 1 -- 172.21.15.161:0/3367248076 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad241427b0 con 0x7fad2413d560 2024-05-22T21:15:33.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.694+0000 7fad29c00640 1 -- 172.21.15.161:0/3367248076 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad24067220 con 0x7fad2413d560 2024-05-22T21:15:33.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.698+0000 7fad1b400640 1 -- 172.21.15.161:0/3367248076 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fad1c01e3b0 con 0x7fad2413d560 2024-05-22T21:15:33.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.698+0000 7fad1b400640 1 --2- 172.21.15.161:0/3367248076 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fad00066690 0x7fad00068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:33.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.698+0000 7fad1b400640 1 -- 172.21.15.161:0/3367248076 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5415+0+0 (secure 0 0 0) 0x7fad1c084ab0 con 0x7fad2413d560 2024-05-22T21:15:33.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.698+0000 7fad22a00640 1 --2- 172.21.15.161:0/3367248076 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fad00066690 0x7fad00068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:33.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.698+0000 7fad22a00640 1 --2- 172.21.15.161:0/3367248076 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fad00066690 0x7fad00068b50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fad140af000 tx=0x7fad140b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:33.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.702+0000 7fad1b400640 1 -- 172.21.15.161:0/3367248076 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fad1c051f50 con 0x7fad2413d560 2024-05-22T21:15:33.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:33.854+0000 7fad29c00640 1 -- 172.21.15.161:0/3367248076 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fad24071d50 con 0x7fad2413d560 2024-05-22T21:15:34.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.530+0000 7fad1b400640 1 -- 172.21.15.161:0/3367248076 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) v1 ==== 169+0+0 (secure 0 0 0) 0x7fad1c01a020 con 0x7fad2413d560 2024-05-22T21:15:34.535 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' removed 2024-05-22T21:15:34.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.530+0000 7fad29c00640 1 -- 172.21.15.161:0/3367248076 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fad00066690 msgr2=0x7fad00068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:34.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.530+0000 7fad29c00640 1 --2- 172.21.15.161:0/3367248076 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fad00066690 0x7fad00068b50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fad140af000 tx=0x7fad140b6040 comp rx=0 tx=0).stop 2024-05-22T21:15:34.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.530+0000 7fad29c00640 1 -- 172.21.15.161:0/3367248076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2413d560 msgr2=0x7fad2406ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:34.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.530+0000 7fad29c00640 1 --2- 172.21.15.161:0/3367248076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2413d560 0x7fad2406ca20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00cef0 tx=0x7fad1c00a710 comp rx=0 tx=0).stop 2024-05-22T21:15:34.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.530+0000 7fad29c00640 1 -- 172.21.15.161:0/3367248076 shutdown_connections 2024-05-22T21:15:34.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.530+0000 7fad29c00640 1 --2- 172.21.15.161:0/3367248076 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fad00066690 0x7fad00068b50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:34.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.530+0000 7fad29c00640 1 --2- 172.21.15.161:0/3367248076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad2413d560 0x7fad2406ca20 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:34.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.530+0000 7fad29c00640 1 --2- 172.21.15.161:0/3367248076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad2413cfe0 0x7fad24154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:34.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.530+0000 7fad29c00640 1 --2- 172.21.15.161:0/3367248076 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad240cdb00 0x7fad24142270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:34.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.530+0000 7fad29c00640 1 -- 172.21.15.161:0/3367248076 >> 172.21.15.161:0/3367248076 conn(0x7fad2406a0a0 msgr2=0x7fad241574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:34.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.530+0000 7fad29c00640 1 -- 172.21.15.161:0/3367248076 shutdown_connections 2024-05-22T21:15:34.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.530+0000 7fad29c00640 1 -- 172.21.15.161:0/3367248076 wait complete. 2024-05-22T21:15:34.558 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-22T21:15:34.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.690+0000 7f83f1a00640 1 Processor -- start 2024-05-22T21:15:34.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.690+0000 7f83f1a00640 1 -- start start 2024-05-22T21:15:34.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.690+0000 7f83f1a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83ec06d780 0x7f83ec164db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:34.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.690+0000 7f83f1a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ec0b3c60 con 0x7f83ec1438c0 2024-05-22T21:15:34.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.690+0000 7f83f1a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ec143ca0 con 0x7f83ec0cd920 2024-05-22T21:15:34.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.690+0000 7f83f1a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec0b41d0 con 0x7f83ec06d780 2024-05-22T21:15:34.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.690+0000 7f83eaa00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f83ec1438c0 0x7f83ec0b3550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60514/0 (socket says 172.21.15.161:60514) 2024-05-22T21:15:34.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.690+0000 7f83eaa00640 1 -- 172.21.15.161:0/813353669 learned_addr learned my addr 172.21.15.161:0/813353669 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:34.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.690+0000 7f83eb400640 1 --2- 172.21.15.161:0/813353669 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83ec06d780 0x7f83ec164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:34.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.690+0000 7f83ea000640 1 -- 172.21.15.161:0/813353669 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122212142 0 0) 0x7f83ec0b3c60 con 0x7f83ec1438c0 2024-05-22T21:15:34.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.690+0000 7f83ea000640 1 -- 172.21.15.161:0/813353669 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83d0005740 con 0x7f83ec1438c0 2024-05-22T21:15:34.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.694+0000 7f83ea000640 1 -- 172.21.15.161:0/813353669 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478173553 0 0) 0x7f83ec143ca0 con 0x7f83ec0cd920 2024-05-22T21:15:34.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.694+0000 7f83ea000640 1 -- 172.21.15.161:0/813353669 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83ec0b3c60 con 0x7f83ec0cd920 2024-05-22T21:15:34.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.694+0000 7f83ea000640 1 -- 172.21.15.161:0/813353669 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1652477124 0 0) 0x7f83d0005740 con 0x7f83ec1438c0 2024-05-22T21:15:34.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.694+0000 7f83eb400640 1 -- 172.21.15.161:0/813353669 >> v1:172.21.15.179:6789/0 conn(0x7f83ec0cd920 legacy=0x7f83ec06d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:34.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.694+0000 7f83eb400640 1 -- 172.21.15.161:0/813353669 >> v1:172.21.15.161:6789/0 conn(0x7f83ec1438c0 legacy=0x7f83ec0b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:34.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.694+0000 7f83eb400640 1 -- 172.21.15.161:0/813353669 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec0b3ef0 con 0x7f83ec06d780 2024-05-22T21:15:34.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.694+0000 7f83eb400640 1 --2- 172.21.15.161:0/813353669 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83ec06d780 0x7f83ec164db0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009dc0 tx=0x7f83e401c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60aa0853f36ace08 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:34.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.694+0000 7f83ea000640 1 -- 172.21.15.161:0/813353669 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc001680 con 0x7f83ec06d780 2024-05-22T21:15:34.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.694+0000 7f83ea000640 1 -- 172.21.15.161:0/813353669 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e40a55c0 con 0x7f83ec06d780 2024-05-22T21:15:34.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.694+0000 7f83ea000640 1 -- 172.21.15.161:0/813353669 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40a5950 con 0x7f83ec06d780 2024-05-22T21:15:34.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.694+0000 7f83f1a00640 1 -- 172.21.15.161:0/813353669 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83ec06d780 msgr2=0x7f83ec164db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:34.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.694+0000 7f83f1a00640 1 --2- 172.21.15.161:0/813353669 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83ec06d780 0x7f83ec164db0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009dc0 tx=0x7f83e401c880 comp rx=0 tx=0).stop 2024-05-22T21:15:34.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83f1a00640 1 -- 172.21.15.161:0/813353669 shutdown_connections 2024-05-22T21:15:34.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83f1a00640 1 --2- 172.21.15.161:0/813353669 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83ec06d780 0x7f83ec164db0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:34.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83f1a00640 1 -- 172.21.15.161:0/813353669 >> 172.21.15.161:0/813353669 conn(0x7f83ec06a2d0 msgr2=0x7f83ec06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:34.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83f1a00640 1 -- 172.21.15.161:0/813353669 shutdown_connections 2024-05-22T21:15:34.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83f1a00640 1 -- 172.21.15.161:0/813353669 wait complete. 2024-05-22T21:15:34.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83f1a00640 1 Processor -- start 2024-05-22T21:15:34.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83f1a00640 1 -- start start 2024-05-22T21:15:34.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83f1a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83ec06d780 0x7f83ec140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:34.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83f1a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83ec0cd920 0x7f83ec1f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:34.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83f1a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83ec1438c0 0x7f83ec1f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:34.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83f1a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec1f6840 con 0x7f83ec1438c0 2024-05-22T21:15:34.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83f1a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec1f6b20 con 0x7f83ec06d780 2024-05-22T21:15:34.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83f1a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec1f6e00 con 0x7f83ec0cd920 2024-05-22T21:15:34.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83eb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83ec1438c0 0x7f83ec1f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:34.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83eb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83ec1438c0 0x7f83ec1f6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43168/0 (socket says 172.21.15.161:43168) 2024-05-22T21:15:34.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83eb400640 1 -- 172.21.15.161:0/1169912371 learned_addr learned my addr 172.21.15.161:0/1169912371 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:34.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83eaa00640 1 --2- 172.21.15.161:0/1169912371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83ec06d780 0x7f83ec140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:34.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83e2000640 1 --2- 172.21.15.161:0/1169912371 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83ec0cd920 0x7f83ec1f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:34.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83eb400640 1 -- 172.21.15.161:0/1169912371 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83ec0cd920 msgr2=0x7f83ec1f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:34.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83eb400640 1 --2- 172.21.15.161:0/1169912371 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83ec0cd920 0x7f83ec1f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:34.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83eb400640 1 -- 172.21.15.161:0/1169912371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83ec06d780 msgr2=0x7f83ec140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:34.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83eb400640 1 --2- 172.21.15.161:0/1169912371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83ec06d780 0x7f83ec140480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:34.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83eb400640 1 -- 172.21.15.161:0/1169912371 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e4009a70 con 0x7f83ec1438c0 2024-05-22T21:15:34.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.698+0000 7f83eb400640 1 --2- 172.21.15.161:0/1169912371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83ec1438c0 0x7f83ec1f6240 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f83e40b0040 tx=0x7f83e4019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:34.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.702+0000 7f83e3400640 1 -- 172.21.15.161:0/1169912371 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4003ba0 con 0x7f83ec1438c0 2024-05-22T21:15:34.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.702+0000 7f83e3400640 1 -- 172.21.15.161:0/1169912371 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e4003d20 con 0x7f83ec1438c0 2024-05-22T21:15:34.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.702+0000 7f83e3400640 1 -- 172.21.15.161:0/1169912371 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4004270 con 0x7f83ec1438c0 2024-05-22T21:15:34.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.702+0000 7f83f1a00640 1 -- 172.21.15.161:0/1169912371 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ec1f70e0 con 0x7f83ec1438c0 2024-05-22T21:15:34.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.702+0000 7f83f1a00640 1 -- 172.21.15.161:0/1169912371 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ec13f340 con 0x7f83ec1438c0 2024-05-22T21:15:34.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.702+0000 7f83f1a00640 1 -- 172.21.15.161:0/1169912371 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ec0ce8a0 con 0x7f83ec1438c0 2024-05-22T21:15:34.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.702+0000 7f83e3400640 1 -- 172.21.15.161:0/1169912371 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f83e40b4050 con 0x7f83ec1438c0 2024-05-22T21:15:34.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.702+0000 7f83e3400640 1 --2- 172.21.15.161:0/1169912371 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f83c8066690 0x7f83c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:34.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.702+0000 7f83e3400640 1 -- 172.21.15.161:0/1169912371 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f83e411eca0 con 0x7f83ec1438c0 2024-05-22T21:15:34.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.702+0000 7f83eaa00640 1 --2- 172.21.15.161:0/1169912371 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f83c8066690 0x7f83c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:34.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.702+0000 7f83eaa00640 1 --2- 172.21.15.161:0/1169912371 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f83c8066690 0x7f83c8068b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f83dc000f70 tx=0x7f83dc0016c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:34.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.706+0000 7f83e3400640 1 -- 172.21.15.161:0/1169912371 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f83e40ec3b0 con 0x7f83ec1438c0 2024-05-22T21:15:34.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.862+0000 7f83f1a00640 1 -- 172.21.15.161:0/1169912371 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f83ec067220 con 0x7f83ec1438c0 2024-05-22T21:15:34.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.866+0000 7f83e3400640 1 -- 172.21.15.161:0/1169912371 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7f83e4009450 con 0x7f83ec1438c0 2024-05-22T21:15:34.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.870+0000 7f83f1a00640 1 -- 172.21.15.161:0/1169912371 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f83c8066690 msgr2=0x7f83c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:34.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.870+0000 7f83f1a00640 1 --2- 172.21.15.161:0/1169912371 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f83c8066690 0x7f83c8068b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f83dc000f70 tx=0x7f83dc0016c0 comp rx=0 tx=0).stop 2024-05-22T21:15:34.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.870+0000 7f83f1a00640 1 -- 172.21.15.161:0/1169912371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83ec1438c0 msgr2=0x7f83ec1f6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:34.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.870+0000 7f83f1a00640 1 --2- 172.21.15.161:0/1169912371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83ec1438c0 0x7f83ec1f6240 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f83e40b0040 tx=0x7f83e4019620 comp rx=0 tx=0).stop 2024-05-22T21:15:34.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.870+0000 7f83f1a00640 1 -- 172.21.15.161:0/1169912371 shutdown_connections 2024-05-22T21:15:34.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.870+0000 7f83f1a00640 1 --2- 172.21.15.161:0/1169912371 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f83c8066690 0x7f83c8068b50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:34.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.870+0000 7f83f1a00640 1 --2- 172.21.15.161:0/1169912371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83ec1438c0 0x7f83ec1f6240 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:34.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.870+0000 7f83f1a00640 1 --2- 172.21.15.161:0/1169912371 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f83ec0cd920 0x7f83ec1f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:34.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.870+0000 7f83f1a00640 1 --2- 172.21.15.161:0/1169912371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83ec06d780 0x7f83ec140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:34.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.870+0000 7f83f1a00640 1 -- 172.21.15.161:0/1169912371 >> 172.21.15.161:0/1169912371 conn(0x7f83ec06a2d0 msgr2=0x7f83ec0b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:34.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.870+0000 7f83f1a00640 1 -- 172.21.15.161:0/1169912371 shutdown_connections 2024-05-22T21:15:34.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:34.870+0000 7f83f1a00640 1 -- 172.21.15.161:0/1169912371 wait complete. 2024-05-22T21:15:34.895 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2024-05-22T21:15:35.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.026+0000 7f07d6200640 1 Processor -- start 2024-05-22T21:15:35.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.026+0000 7f07d6200640 1 -- start start 2024-05-22T21:15:35.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.026+0000 7f07d6200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f07d00b3560 0x7f07d015e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:35.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.026+0000 7f07d6200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f07d009a080 0x7f07d00cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:35.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.026+0000 7f07d6200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d006df70 con 0x7f07d0099ca0 2024-05-22T21:15:35.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.026+0000 7f07d6200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d00c7ea0 con 0x7f07d009a080 2024-05-22T21:15:35.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.026+0000 7f07d6200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d0161c60 con 0x7f07d00b3560 2024-05-22T21:15:35.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.026+0000 7f07cfe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f07d009a080 0x7f07d00cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:35.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.026+0000 7f07cfe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f07d009a080 0x7f07d00cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43170/0 (socket says 172.21.15.161:43170) 2024-05-22T21:15:35.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.026+0000 7f07cfe00640 1 -- 172.21.15.161:0/3882612959 learned_addr learned my addr 172.21.15.161:0/3882612959 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:35.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.026+0000 7f07c6a00640 1 --2- 172.21.15.161:0/3882612959 >> v2:172.21.15.179:3300/0 conn(0x7f07d00b3560 0x7f07d015e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:35.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.026+0000 7f07cfe00640 1 -- 172.21.15.161:0/3882612959 >> v2:172.21.15.179:3300/0 conn(0x7f07d00b3560 msgr2=0x7f07d015e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:35.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.026+0000 7f07cfe00640 1 --2- 172.21.15.161:0/3882612959 >> v2:172.21.15.179:3300/0 conn(0x7f07d00b3560 0x7f07d015e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.026+0000 7f07cfe00640 1 -- 172.21.15.161:0/3882612959 >> v1:172.21.15.179:6789/0 conn(0x7f07d0099ca0 legacy=0x7f07d006eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:35.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.026+0000 7f07cfe00640 1 -- 172.21.15.161:0/3882612959 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d006e1c0 con 0x7f07d009a080 2024-05-22T21:15:35.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.030+0000 7f07cfe00640 1 --2- 172.21.15.161:0/3882612959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d009a080 0x7f07d00cf0e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f07d00cede0 tx=0x7f07c801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248d30ac50b8be82 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:35.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.030+0000 7f07cea00640 1 -- 172.21.15.161:0/3882612959 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c80a5880 con 0x7f07d009a080 2024-05-22T21:15:35.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.030+0000 7f07cea00640 1 -- 172.21.15.161:0/3882612959 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c80a59e0 con 0x7f07d009a080 2024-05-22T21:15:35.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.030+0000 7f07cea00640 1 -- 172.21.15.161:0/3882612959 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c80a5d70 con 0x7f07d009a080 2024-05-22T21:15:35.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.030+0000 7f07d6200640 1 -- 172.21.15.161:0/3882612959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d009a080 msgr2=0x7f07d00cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:35.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.030+0000 7f07d6200640 1 --2- 172.21.15.161:0/3882612959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d009a080 0x7f07d00cf0e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f07d00cede0 tx=0x7f07c801c9c0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.030+0000 7f07d6200640 1 -- 172.21.15.161:0/3882612959 shutdown_connections 2024-05-22T21:15:35.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.030+0000 7f07d6200640 1 --2- 172.21.15.161:0/3882612959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d009a080 0x7f07d00cf0e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.030+0000 7f07d6200640 1 --2- 172.21.15.161:0/3882612959 >> v2:172.21.15.179:3300/0 conn(0x7f07d00b3560 0x7f07d015e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.030+0000 7f07d6200640 1 -- 172.21.15.161:0/3882612959 >> 172.21.15.161:0/3882612959 conn(0x7f07d00b2a40 msgr2=0x7f07d00b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:35.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.030+0000 7f07d6200640 1 -- 172.21.15.161:0/3882612959 shutdown_connections 2024-05-22T21:15:35.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.030+0000 7f07d6200640 1 -- 172.21.15.161:0/3882612959 wait complete. 2024-05-22T21:15:35.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07d6200640 1 Processor -- start 2024-05-22T21:15:35.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07d6200640 1 -- start start 2024-05-22T21:15:35.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07d6200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07d0099ca0 0x7f07d00cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:35.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07d6200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f07d009a080 0x7f07d015b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:35.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07d6200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d00b3560 0x7f07d0152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:35.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07d6200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d0158c60 con 0x7f07d00b3560 2024-05-22T21:15:35.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07d6200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d0158050 con 0x7f07d0099ca0 2024-05-22T21:15:35.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07d6200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d0158330 con 0x7f07d009a080 2024-05-22T21:15:35.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07cfe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d00b3560 0x7f07d0152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:35.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07cfe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d00b3560 0x7f07d0152dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43174/0 (socket says 172.21.15.161:43174) 2024-05-22T21:15:35.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07cfe00640 1 -- 172.21.15.161:0/529737463 learned_addr learned my addr 172.21.15.161:0/529737463 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:35.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07cf400640 1 --2- 172.21.15.161:0/529737463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07d0099ca0 0x7f07d00cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:35.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07c6a00640 1 --2- 172.21.15.161:0/529737463 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f07d009a080 0x7f07d015b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:35.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07cfe00640 1 -- 172.21.15.161:0/529737463 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f07d009a080 msgr2=0x7f07d015b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:35.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07cfe00640 1 --2- 172.21.15.161:0/529737463 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f07d009a080 0x7f07d015b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07cfe00640 1 -- 172.21.15.161:0/529737463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07d0099ca0 msgr2=0x7f07d00cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:35.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07cfe00640 1 --2- 172.21.15.161:0/529737463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07d0099ca0 0x7f07d00cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07cfe00640 1 -- 172.21.15.161:0/529737463 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c8009b30 con 0x7f07d00b3560 2024-05-22T21:15:35.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07cfe00640 1 --2- 172.21.15.161:0/529737463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d00b3560 0x7f07d0152dc0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f07c80af040 tx=0x7f07c8019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:35.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07c7e00640 1 -- 172.21.15.161:0/529737463 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c80b8670 con 0x7f07d00b3560 2024-05-22T21:15:35.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07c7e00640 1 -- 172.21.15.161:0/529737463 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c80b87d0 con 0x7f07d00b3560 2024-05-22T21:15:35.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07d6200640 1 -- 172.21.15.161:0/529737463 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d0158610 con 0x7f07d00b3560 2024-05-22T21:15:35.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07c7e00640 1 -- 172.21.15.161:0/529737463 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c80b8b80 con 0x7f07d00b3560 2024-05-22T21:15:35.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.034+0000 7f07d6200640 1 -- 172.21.15.161:0/529737463 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d0157c70 con 0x7f07d00b3560 2024-05-22T21:15:35.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.038+0000 7f07d6200640 1 -- 172.21.15.161:0/529737463 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d0158dd0 con 0x7f07d00b3560 2024-05-22T21:15:35.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.038+0000 7f07c7e00640 1 -- 172.21.15.161:0/529737463 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f07c80b8ce0 con 0x7f07d00b3560 2024-05-22T21:15:35.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.038+0000 7f07c7e00640 1 --2- 172.21.15.161:0/529737463 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f07ac0666c0 0x7f07ac068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:35.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.038+0000 7f07c7e00640 1 -- 172.21.15.161:0/529737463 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f07c811f480 con 0x7f07d00b3560 2024-05-22T21:15:35.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.038+0000 7f07cf400640 1 --2- 172.21.15.161:0/529737463 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f07ac0666c0 0x7f07ac068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:35.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.038+0000 7f07cf400640 1 --2- 172.21.15.161:0/529737463 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f07ac0666c0 0x7f07ac068b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f07c0007fd0 tx=0x7f07c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:35.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.042+0000 7f07c7e00640 1 -- 172.21.15.161:0/529737463 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f07c80eccf0 con 0x7f07d00b3560 2024-05-22T21:15:35.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.250+0000 7f07d6200640 1 -- 172.21.15.161:0/529737463 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f07d0158470 con 0x7f07d00b3560 2024-05-22T21:15:35.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.526+0000 7f07c7e00640 1 -- 172.21.15.161:0/529737463 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v142) v1 ==== 142+0+0 (secure 0 0 0) 0x7f07d0158470 con 0x7f07d00b3560 2024-05-22T21:15:35.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.530+0000 7f07d6200640 1 -- 172.21.15.161:0/529737463 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f07ac0666c0 msgr2=0x7f07ac068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:35.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.530+0000 7f07d6200640 1 --2- 172.21.15.161:0/529737463 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f07ac0666c0 0x7f07ac068b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f07c0007fd0 tx=0x7f07c0009040 comp rx=0 tx=0).stop 2024-05-22T21:15:35.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.530+0000 7f07d6200640 1 -- 172.21.15.161:0/529737463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d00b3560 msgr2=0x7f07d0152dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:35.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.530+0000 7f07d6200640 1 --2- 172.21.15.161:0/529737463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d00b3560 0x7f07d0152dc0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f07c80af040 tx=0x7f07c8019760 comp rx=0 tx=0).stop 2024-05-22T21:15:35.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.530+0000 7f07d6200640 1 -- 172.21.15.161:0/529737463 shutdown_connections 2024-05-22T21:15:35.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.530+0000 7f07d6200640 1 --2- 172.21.15.161:0/529737463 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f07ac0666c0 0x7f07ac068b80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.530+0000 7f07d6200640 1 --2- 172.21.15.161:0/529737463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d00b3560 0x7f07d0152dc0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.530+0000 7f07d6200640 1 --2- 172.21.15.161:0/529737463 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f07d009a080 0x7f07d015b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.530+0000 7f07d6200640 1 --2- 172.21.15.161:0/529737463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07d0099ca0 0x7f07d00cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.530+0000 7f07d6200640 1 -- 172.21.15.161:0/529737463 >> 172.21.15.161:0/529737463 conn(0x7f07d00b2a40 msgr2=0x7f07d0156130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:35.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.530+0000 7f07d6200640 1 -- 172.21.15.161:0/529737463 shutdown_connections 2024-05-22T21:15:35.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.530+0000 7f07d6200640 1 -- 172.21.15.161:0/529737463 wait complete. 2024-05-22T21:15:35.555 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-05-22T21:15:35.555 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) 2024-05-22T21:15:35.555 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2024-05-22T21:15:35.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e59800640 1 Processor -- start 2024-05-22T21:15:35.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e59800640 1 -- start start 2024-05-22T21:15:35.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e59800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9e540b3440 0x7f9e540b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:35.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e59800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9e54161b30 0x7f9e540b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:35.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e59800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e540b1de0 0x7f9e540b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:35.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e59800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e54162b90 con 0x7f9e54161b30 2024-05-22T21:15:35.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e59800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e54162010 con 0x7f9e540b3440 2024-05-22T21:15:35.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e59800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e541622f0 con 0x7f9e540b1de0 2024-05-22T21:15:35.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e4a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9e54161b30 0x7f9e540b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:35.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e4a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9e54161b30 0x7f9e540b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43188/0 (socket says 172.21.15.161:43188) 2024-05-22T21:15:35.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e4a000640 1 -- 172.21.15.161:0/1292843273 learned_addr learned my addr 172.21.15.161:0/1292843273 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:35.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e53400640 1 --2- 172.21.15.161:0/1292843273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e540b1de0 0x7f9e540b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:35.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e52a00640 1 --2- 172.21.15.161:0/1292843273 >> v2:172.21.15.179:3300/0 conn(0x7f9e540b3440 0x7f9e540b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:35.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e4a000640 1 -- 172.21.15.161:0/1292843273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e540b1de0 msgr2=0x7f9e540b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:35.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e4a000640 1 --2- 172.21.15.161:0/1292843273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e540b1de0 0x7f9e540b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e4a000640 1 -- 172.21.15.161:0/1292843273 >> v2:172.21.15.179:3300/0 conn(0x7f9e540b3440 msgr2=0x7f9e540b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:35.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e4a000640 1 --2- 172.21.15.161:0/1292843273 >> v2:172.21.15.179:3300/0 conn(0x7f9e540b3440 0x7f9e540b24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e4a000640 1 -- 172.21.15.161:0/1292843273 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e541625d0 con 0x7f9e54161b30 2024-05-22T21:15:35.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e53400640 1 --2- 172.21.15.161:0/1292843273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e540b1de0 0x7f9e540b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:35.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.686+0000 7f9e52a00640 1 --2- 172.21.15.161:0/1292843273 >> v2:172.21.15.179:3300/0 conn(0x7f9e540b3440 0x7f9e540b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:35.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.690+0000 7f9e4a000640 1 --2- 172.21.15.161:0/1292843273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e54161b30 0x7f9e540b0d10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9e5406dd60 tx=0x7f9e3c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f23a02bbb4438e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:35.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.690+0000 7f9e52000640 1 -- 172.21.15.161:0/1292843273 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0aa610 con 0x7f9e54161b30 2024-05-22T21:15:35.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.690+0000 7f9e52000640 1 -- 172.21.15.161:0/1292843273 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c0a6040 con 0x7f9e54161b30 2024-05-22T21:15:35.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.690+0000 7f9e52000640 1 -- 172.21.15.161:0/1292843273 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0aa990 con 0x7f9e54161b30 2024-05-22T21:15:35.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.690+0000 7f9e59800640 1 -- 172.21.15.161:0/1292843273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e54161b30 msgr2=0x7f9e540b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:35.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.690+0000 7f9e59800640 1 --2- 172.21.15.161:0/1292843273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e54161b30 0x7f9e540b0d10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9e5406dd60 tx=0x7f9e3c01c4f0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.690+0000 7f9e59800640 1 -- 172.21.15.161:0/1292843273 shutdown_connections 2024-05-22T21:15:35.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.690+0000 7f9e59800640 1 --2- 172.21.15.161:0/1292843273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e540b1de0 0x7f9e540b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.690+0000 7f9e59800640 1 --2- 172.21.15.161:0/1292843273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e54161b30 0x7f9e540b0d10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.690+0000 7f9e59800640 1 --2- 172.21.15.161:0/1292843273 >> v2:172.21.15.179:3300/0 conn(0x7f9e540b3440 0x7f9e540b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.690+0000 7f9e59800640 1 -- 172.21.15.161:0/1292843273 >> 172.21.15.161:0/1292843273 conn(0x7f9e5406a0a0 msgr2=0x7f9e5406a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:35.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.690+0000 7f9e59800640 1 -- 172.21.15.161:0/1292843273 shutdown_connections 2024-05-22T21:15:35.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.690+0000 7f9e59800640 1 -- 172.21.15.161:0/1292843273 wait complete. 2024-05-22T21:15:35.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e59800640 1 Processor -- start 2024-05-22T21:15:35.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e59800640 1 -- start start 2024-05-22T21:15:35.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e59800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e540b1de0 0x7f9e54157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:35.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e59800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e540b3440 0x7f9e54153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:35.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e59800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e54161b30 0x7f9e541474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:35.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e59800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e541586f0 con 0x7f9e54161b30 2024-05-22T21:15:35.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e59800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e54157b70 con 0x7f9e540b3440 2024-05-22T21:15:35.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e59800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e54157e50 con 0x7f9e540b1de0 2024-05-22T21:15:35.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e53400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e54161b30 0x7f9e541474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:35.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e53400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e54161b30 0x7f9e541474b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43202/0 (socket says 172.21.15.161:43202) 2024-05-22T21:15:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e53400640 1 -- 172.21.15.161:0/2763367098 learned_addr learned my addr 172.21.15.161:0/2763367098 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e52a00640 1 --2- 172.21.15.161:0/2763367098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e540b1de0 0x7f9e54157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e4a000640 1 --2- 172.21.15.161:0/2763367098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e540b3440 0x7f9e54153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e53400640 1 -- 172.21.15.161:0/2763367098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e540b1de0 msgr2=0x7f9e54157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e53400640 1 --2- 172.21.15.161:0/2763367098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e540b1de0 0x7f9e54157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e53400640 1 -- 172.21.15.161:0/2763367098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e540b3440 msgr2=0x7f9e54153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e53400640 1 --2- 172.21.15.161:0/2763367098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e540b3440 0x7f9e54153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e53400640 1 -- 172.21.15.161:0/2763367098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0094d0 con 0x7f9e54161b30 2024-05-22T21:15:35.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e53400640 1 --2- 172.21.15.161:0/2763367098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e54161b30 0x7f9e541474b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c007e10 tx=0x7f9e4c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:35.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e4b400640 1 -- 172.21.15.161:0/2763367098 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c003ba0 con 0x7f9e54161b30 2024-05-22T21:15:35.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e4b400640 1 -- 172.21.15.161:0/2763367098 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4c00f040 con 0x7f9e54161b30 2024-05-22T21:15:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e4b400640 1 -- 172.21.15.161:0/2763367098 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c00de80 con 0x7f9e54161b30 2024-05-22T21:15:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e59800640 1 -- 172.21.15.161:0/2763367098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e54158130 con 0x7f9e54161b30 2024-05-22T21:15:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.694+0000 7f9e59800640 1 -- 172.21.15.161:0/2763367098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e54157780 con 0x7f9e54161b30 2024-05-22T21:15:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.698+0000 7f9e59800640 1 -- 172.21.15.161:0/2763367098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5414d1c0 con 0x7f9e54161b30 2024-05-22T21:15:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.698+0000 7f9e4b400640 1 -- 172.21.15.161:0/2763367098 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9e4c004030 con 0x7f9e54161b30 2024-05-22T21:15:35.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.698+0000 7f9e4b400640 1 --2- 172.21.15.161:0/2763367098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e30066690 0x7f9e30068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:35.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.698+0000 7f9e4b400640 1 -- 172.21.15.161:0/2763367098 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9e4c084110 con 0x7f9e54161b30 2024-05-22T21:15:35.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.698+0000 7f9e52a00640 1 --2- 172.21.15.161:0/2763367098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e30066690 0x7f9e30068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:35.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.698+0000 7f9e52a00640 1 --2- 172.21.15.161:0/2763367098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e30066690 0x7f9e30068b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009950 tx=0x7f9e44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:35.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.702+0000 7f9e4b400640 1 -- 172.21.15.161:0/2763367098 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9e5414d1c0 con 0x7f9e54161b30 2024-05-22T21:15:35.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:35.918+0000 7f9e59800640 1 -- 172.21.15.161:0/2763367098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f9e5414d1c0 con 0x7f9e54161b30 2024-05-22T21:15:36.542 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.538+0000 7f9e4b400640 1 -- 172.21.15.161:0/2763367098 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v143) v1 ==== 137+0+0 (secure 0 0 0) 0x7f9e5414d1c0 con 0x7f9e54161b30 2024-05-22T21:15:36.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.538+0000 7f9e59800640 1 -- 172.21.15.161:0/2763367098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e30066690 msgr2=0x7f9e30068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:36.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.538+0000 7f9e59800640 1 --2- 172.21.15.161:0/2763367098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e30066690 0x7f9e30068b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009950 tx=0x7f9e44008040 comp rx=0 tx=0).stop 2024-05-22T21:15:36.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.538+0000 7f9e59800640 1 -- 172.21.15.161:0/2763367098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e54161b30 msgr2=0x7f9e541474b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:36.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.538+0000 7f9e59800640 1 --2- 172.21.15.161:0/2763367098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e54161b30 0x7f9e541474b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c007e10 tx=0x7f9e4c00d4a0 comp rx=0 tx=0).stop 2024-05-22T21:15:36.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.538+0000 7f9e59800640 1 -- 172.21.15.161:0/2763367098 shutdown_connections 2024-05-22T21:15:36.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.538+0000 7f9e59800640 1 --2- 172.21.15.161:0/2763367098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e30066690 0x7f9e30068b50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:36.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.538+0000 7f9e59800640 1 --2- 172.21.15.161:0/2763367098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e54161b30 0x7f9e541474b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:36.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.538+0000 7f9e59800640 1 --2- 172.21.15.161:0/2763367098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e540b3440 0x7f9e54153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:36.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.538+0000 7f9e59800640 1 --2- 172.21.15.161:0/2763367098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e540b1de0 0x7f9e54157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:36.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.538+0000 7f9e59800640 1 -- 172.21.15.161:0/2763367098 >> 172.21.15.161:0/2763367098 conn(0x7f9e5406a0a0 msgr2=0x7f9e540ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:36.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.538+0000 7f9e59800640 1 -- 172.21.15.161:0/2763367098 shutdown_connections 2024-05-22T21:15:36.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.538+0000 7f9e59800640 1 -- 172.21.15.161:0/2763367098 wait complete. 2024-05-22T21:15:36.564 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-05-22T21:15:36.715 INFO:teuthology.orchestra.run.smithi161.stdout:{"osd_objectstore":"memstore"} 2024-05-22T21:15:36.715 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-05-22T21:15:36.716 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-22T21:15:36.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.846+0000 7fc229e00640 1 Processor -- start 2024-05-22T21:15:36.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.846+0000 7fc229e00640 1 -- start start 2024-05-22T21:15:36.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.850+0000 7fc229e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc22406ccd0 0x7fc2240afe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:36.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.850+0000 7fc229e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2240b34c0 con 0x7fc2240cdec0 2024-05-22T21:15:36.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.850+0000 7fc229e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2240b1f70 con 0x7fc22406bfe0 2024-05-22T21:15:36.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.850+0000 7fc229e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2240ce2a0 con 0x7fc22406ccd0 2024-05-22T21:15:36.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.850+0000 7fc223e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc22406ccd0 0x7fc2240afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:36.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.850+0000 7fc228a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc2240cdec0 0x7fc2240b0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:49488/0 (socket says 172.21.15.161:49488) 2024-05-22T21:15:36.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.850+0000 7fc228a00640 1 -- 172.21.15.161:0/1501347409 learned_addr learned my addr 172.21.15.161:0/1501347409 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:36.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.850+0000 7fc223e00640 1 -- 172.21.15.161:0/1501347409 >> v1:172.21.15.179:6789/0 conn(0x7fc22406bfe0 legacy=0x7fc224160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:36.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.850+0000 7fc223e00640 1 -- 172.21.15.161:0/1501347409 >> v1:172.21.15.161:6789/0 conn(0x7fc2240cdec0 legacy=0x7fc2240b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:36.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.850+0000 7fc223e00640 1 -- 172.21.15.161:0/1501347409 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2240b26e0 con 0x7fc22406ccd0 2024-05-22T21:15:36.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.850+0000 7fc223400640 1 -- 172.21.15.161:0/1501347409 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 940535783 0 0) 0x7fc2240b34c0 con 0x7fc2240cdec0 2024-05-22T21:15:36.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.850+0000 7fc223e00640 1 --2- 172.21.15.161:0/1501347409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc22406ccd0 0x7fc2240afe10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc2100097c0 tx=0x7fc21001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418a81c75273a259 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:36.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc223400640 1 -- 172.21.15.161:0/1501347409 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2100ab600 con 0x7fc22406ccd0 2024-05-22T21:15:36.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc223400640 1 -- 172.21.15.161:0/1501347409 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2100a7050 con 0x7fc22406ccd0 2024-05-22T21:15:36.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc223400640 1 -- 172.21.15.161:0/1501347409 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2100ab980 con 0x7fc22406ccd0 2024-05-22T21:15:36.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc229e00640 1 -- 172.21.15.161:0/1501347409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc22406ccd0 msgr2=0x7fc2240afe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:36.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc229e00640 1 --2- 172.21.15.161:0/1501347409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc22406ccd0 0x7fc2240afe10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc2100097c0 tx=0x7fc21001c410 comp rx=0 tx=0).stop 2024-05-22T21:15:36.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc229e00640 1 -- 172.21.15.161:0/1501347409 shutdown_connections 2024-05-22T21:15:36.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc229e00640 1 --2- 172.21.15.161:0/1501347409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc22406ccd0 0x7fc2240afe10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:36.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc229e00640 1 -- 172.21.15.161:0/1501347409 >> 172.21.15.161:0/1501347409 conn(0x7fc2240b29a0 msgr2=0x7fc2240b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:36.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc229e00640 1 -- 172.21.15.161:0/1501347409 shutdown_connections 2024-05-22T21:15:36.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc229e00640 1 -- 172.21.15.161:0/1501347409 wait complete. 2024-05-22T21:15:36.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc229e00640 1 Processor -- start 2024-05-22T21:15:36.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc229e00640 1 -- start start 2024-05-22T21:15:36.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc229e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc22406bfe0 0x7fc2240b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:36.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc229e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc22406ccd0 0x7fc22406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:36.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc229e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2240cdec0 0x7fc224158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:36.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc229e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240b4e40 con 0x7fc22406ccd0 2024-05-22T21:15:36.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc229e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240c9090 con 0x7fc2240cdec0 2024-05-22T21:15:36.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc229e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240c9370 con 0x7fc22406bfe0 2024-05-22T21:15:36.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc223e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc22406ccd0 0x7fc22406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:36.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc223e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc22406ccd0 0x7fc22406f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50000/0 (socket says 172.21.15.161:50000) 2024-05-22T21:15:36.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc223e00640 1 -- 172.21.15.161:0/3637714868 learned_addr learned my addr 172.21.15.161:0/3637714868 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:36.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc228a00640 1 --2- 172.21.15.161:0/3637714868 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc22406bfe0 0x7fc2240b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:36.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.854+0000 7fc229400640 1 --2- 172.21.15.161:0/3637714868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2240cdec0 0x7fc224158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:36.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc223e00640 1 -- 172.21.15.161:0/3637714868 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc22406bfe0 msgr2=0x7fc2240b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:36.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc223e00640 1 --2- 172.21.15.161:0/3637714868 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc22406bfe0 0x7fc2240b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:36.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc223e00640 1 -- 172.21.15.161:0/3637714868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2240cdec0 msgr2=0x7fc224158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:36.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc223e00640 1 --2- 172.21.15.161:0/3637714868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2240cdec0 0x7fc224158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:36.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc223e00640 1 -- 172.21.15.161:0/3637714868 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc210009420 con 0x7fc22406ccd0 2024-05-22T21:15:36.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc229400640 1 --2- 172.21.15.161:0/3637714868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2240cdec0 0x7fc224158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:36.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc223e00640 1 --2- 172.21.15.161:0/3637714868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc22406ccd0 0x7fc22406f1a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc2100ae040 tx=0x7fc2100191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:36.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc220c00640 1 -- 172.21.15.161:0/3637714868 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2100b4900 con 0x7fc22406ccd0 2024-05-22T21:15:36.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc220c00640 1 -- 172.21.15.161:0/3637714868 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2100b4a80 con 0x7fc22406ccd0 2024-05-22T21:15:36.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc229e00640 1 -- 172.21.15.161:0/3637714868 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2240c9650 con 0x7fc22406ccd0 2024-05-22T21:15:36.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc220c00640 1 -- 172.21.15.161:0/3637714868 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2100b4e30 con 0x7fc22406ccd0 2024-05-22T21:15:36.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc229e00640 1 -- 172.21.15.161:0/3637714868 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2240b6990 con 0x7fc22406ccd0 2024-05-22T21:15:36.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc229e00640 1 -- 172.21.15.161:0/3637714868 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc224067070 con 0x7fc22406ccd0 2024-05-22T21:15:36.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc220c00640 1 -- 172.21.15.161:0/3637714868 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc2100b2040 con 0x7fc22406ccd0 2024-05-22T21:15:36.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc220c00640 1 --2- 172.21.15.161:0/3637714868 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc204066690 0x7fc204068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:36.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc220c00640 1 -- 172.21.15.161:0/3637714868 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc21011e1b0 con 0x7fc22406ccd0 2024-05-22T21:15:36.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.858+0000 7fc228a00640 1 --2- 172.21.15.161:0/3637714868 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc204066690 0x7fc204068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:36.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.862+0000 7fc228a00640 1 --2- 172.21.15.161:0/3637714868 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc204066690 0x7fc204068b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc214007bc0 tx=0x7fc214007b10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:36.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:36.862+0000 7fc220c00640 1 -- 172.21.15.161:0/3637714868 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc2100eb780 con 0x7fc22406ccd0 2024-05-22T21:15:37.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.018+0000 7fc229e00640 1 -- 172.21.15.161:0/3637714868 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc2240b4f80 con 0x7fc22406ccd0 2024-05-22T21:15:37.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.026+0000 7fc220c00640 1 -- 172.21.15.161:0/3637714868 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc2100eeda0 con 0x7fc22406ccd0 2024-05-22T21:15:37.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.026+0000 7fc229e00640 1 -- 172.21.15.161:0/3637714868 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc204066690 msgr2=0x7fc204068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:37.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.026+0000 7fc229e00640 1 --2- 172.21.15.161:0/3637714868 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc204066690 0x7fc204068b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc214007bc0 tx=0x7fc214007b10 comp rx=0 tx=0).stop 2024-05-22T21:15:37.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.026+0000 7fc229e00640 1 -- 172.21.15.161:0/3637714868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc22406ccd0 msgr2=0x7fc22406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:37.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.026+0000 7fc229e00640 1 --2- 172.21.15.161:0/3637714868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc22406ccd0 0x7fc22406f1a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc2100ae040 tx=0x7fc2100191b0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.026+0000 7fc229e00640 1 -- 172.21.15.161:0/3637714868 shutdown_connections 2024-05-22T21:15:37.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.026+0000 7fc229e00640 1 --2- 172.21.15.161:0/3637714868 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc204066690 0x7fc204068b50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.026+0000 7fc229e00640 1 --2- 172.21.15.161:0/3637714868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2240cdec0 0x7fc224158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.026+0000 7fc229e00640 1 --2- 172.21.15.161:0/3637714868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc22406ccd0 0x7fc22406f1a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.026+0000 7fc229e00640 1 --2- 172.21.15.161:0/3637714868 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc22406bfe0 0x7fc2240b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.026+0000 7fc229e00640 1 -- 172.21.15.161:0/3637714868 >> 172.21.15.161:0/3637714868 conn(0x7fc2240b29a0 msgr2=0x7fc2240b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:37.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.026+0000 7fc229e00640 1 -- 172.21.15.161:0/3637714868 shutdown_connections 2024-05-22T21:15:37.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.030+0000 7fc229e00640 1 -- 172.21.15.161:0/3637714868 wait complete. 2024-05-22T21:15:37.053 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:15:37.053 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:15:37.117 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:15:37.211 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:15:37.211 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:15:37.211 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:15:37.221 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:15:37.221 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:15:37.221 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:15:37.231 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:15:37.231 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:15:37.231 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:15:37.269 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:15:37.269 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:15:37.269 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:15:37.283 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:15:37.283 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-22T21:15:37.284 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:15:37.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.442+0000 7f8a6fa00640 1 Processor -- start 2024-05-22T21:15:37.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.442+0000 7f8a6fa00640 1 -- start start 2024-05-22T21:15:37.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.446+0000 7f8a6fa00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8a680cf6e0 0x7f8a680b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:37.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.446+0000 7f8a6fa00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a680cbe40 con 0x7f8a6809a420 2024-05-22T21:15:37.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.446+0000 7f8a6fa00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a680cc120 con 0x7f8a6815b250 2024-05-22T21:15:37.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.446+0000 7f8a6fa00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a680cc400 con 0x7f8a680cf6e0 2024-05-22T21:15:37.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.446+0000 7f8a6da00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8a6809a420 0x7f8a681650f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:49494/0 (socket says 172.21.15.161:49494) 2024-05-22T21:15:37.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.446+0000 7f8a6da00640 1 -- 172.21.15.161:0/4144506550 learned_addr learned my addr 172.21.15.161:0/4144506550 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:37.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.446+0000 7f8a67e00640 1 --2- 172.21.15.161:0/4144506550 >> v2:172.21.15.179:3300/0 conn(0x7f8a680cf6e0 0x7f8a680b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:37.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.446+0000 7f8a67400640 1 -- 172.21.15.161:0/4144506550 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562752741 0 0) 0x7f8a680cbe40 con 0x7f8a6809a420 2024-05-22T21:15:37.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.446+0000 7f8a67400640 1 -- 172.21.15.161:0/4144506550 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a50003530 con 0x7f8a6809a420 2024-05-22T21:15:37.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.446+0000 7f8a67e00640 1 -- 172.21.15.161:0/4144506550 >> v1:172.21.15.179:6789/0 conn(0x7f8a6815b250 legacy=0x7f8a680cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:37.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.446+0000 7f8a67e00640 1 -- 172.21.15.161:0/4144506550 >> v1:172.21.15.161:6789/0 conn(0x7f8a6809a420 legacy=0x7f8a681650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:37.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.446+0000 7f8a67e00640 1 -- 172.21.15.161:0/4144506550 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a680cc6e0 con 0x7f8a680cf6e0 2024-05-22T21:15:37.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a67e00640 1 --2- 172.21.15.161:0/4144506550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a680cf6e0 0x7f8a680b3560 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8a540097c0 tx=0x7f8a5401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=786dff00194a79c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:37.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a67400640 1 -- 172.21.15.161:0/4144506550 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a540ab640 con 0x7f8a680cf6e0 2024-05-22T21:15:37.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a67400640 1 -- 172.21.15.161:0/4144506550 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a540a7070 con 0x7f8a680cf6e0 2024-05-22T21:15:37.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a67400640 1 -- 172.21.15.161:0/4144506550 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a540ab9a0 con 0x7f8a680cf6e0 2024-05-22T21:15:37.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a6fa00640 1 -- 172.21.15.161:0/4144506550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a680cf6e0 msgr2=0x7f8a680b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:37.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a6fa00640 1 --2- 172.21.15.161:0/4144506550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a680cf6e0 0x7f8a680b3560 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8a540097c0 tx=0x7f8a5401c4a0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a6fa00640 1 -- 172.21.15.161:0/4144506550 shutdown_connections 2024-05-22T21:15:37.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a6fa00640 1 --2- 172.21.15.161:0/4144506550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a680cf6e0 0x7f8a680b3560 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a6fa00640 1 -- 172.21.15.161:0/4144506550 >> 172.21.15.161:0/4144506550 conn(0x7f8a680b2a40 msgr2=0x7f8a680b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:37.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a6fa00640 1 -- 172.21.15.161:0/4144506550 shutdown_connections 2024-05-22T21:15:37.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a6fa00640 1 -- 172.21.15.161:0/4144506550 wait complete. 2024-05-22T21:15:37.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a6fa00640 1 Processor -- start 2024-05-22T21:15:37.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a6fa00640 1 -- start start 2024-05-22T21:15:37.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a6fa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a6809a420 0x7f8a680b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:37.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a6fa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a680cf6e0 0x7f8a681f7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:37.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a6fa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6815b250 0x7f8a681f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:37.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a6fa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a680b6880 con 0x7f8a6815b250 2024-05-22T21:15:37.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a6fa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a680b5c70 con 0x7f8a6809a420 2024-05-22T21:15:37.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.450+0000 7f8a6fa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a680b5f50 con 0x7f8a680cf6e0 2024-05-22T21:15:37.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a6da00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6815b250 0x7f8a681f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:37.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a6da00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6815b250 0x7f8a681f7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50004/0 (socket says 172.21.15.161:50004) 2024-05-22T21:15:37.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a6da00640 1 -- 172.21.15.161:0/3622662192 learned_addr learned my addr 172.21.15.161:0/3622662192 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:37.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a6d000640 1 --2- 172.21.15.161:0/3622662192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a6809a420 0x7f8a680b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:37.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a67e00640 1 --2- 172.21.15.161:0/3622662192 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a680cf6e0 0x7f8a681f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:37.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a6da00640 1 -- 172.21.15.161:0/3622662192 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a680cf6e0 msgr2=0x7f8a681f7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:37.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a6da00640 1 --2- 172.21.15.161:0/3622662192 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a680cf6e0 0x7f8a681f7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a6da00640 1 -- 172.21.15.161:0/3622662192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a6809a420 msgr2=0x7f8a680b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:37.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a6da00640 1 --2- 172.21.15.161:0/3622662192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a6809a420 0x7f8a680b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a6da00640 1 -- 172.21.15.161:0/3622662192 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a54009420 con 0x7f8a6815b250 2024-05-22T21:15:37.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a67e00640 1 --2- 172.21.15.161:0/3622662192 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a680cf6e0 0x7f8a681f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:37.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a6da00640 1 --2- 172.21.15.161:0/3622662192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6815b250 0x7f8a681f7d70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000cb40 tx=0x7f8a6000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:37.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a64c00640 1 -- 172.21.15.161:0/3622662192 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000aca0 con 0x7f8a6815b250 2024-05-22T21:15:37.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a64c00640 1 -- 172.21.15.161:0/3622662192 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6000ae00 con 0x7f8a6815b250 2024-05-22T21:15:37.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a6fa00640 1 -- 172.21.15.161:0/3622662192 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a680b6230 con 0x7f8a6815b250 2024-05-22T21:15:37.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a64c00640 1 -- 172.21.15.161:0/3622662192 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60012620 con 0x7f8a6815b250 2024-05-22T21:15:37.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a6fa00640 1 -- 172.21.15.161:0/3622662192 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a680b5900 con 0x7f8a6815b250 2024-05-22T21:15:37.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a6fa00640 1 -- 172.21.15.161:0/3622662192 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a68067220 con 0x7f8a6815b250 2024-05-22T21:15:37.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a64c00640 1 -- 172.21.15.161:0/3622662192 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8a60007070 con 0x7f8a6815b250 2024-05-22T21:15:37.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.454+0000 7f8a64c00640 1 --2- 172.21.15.161:0/3622662192 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a48066690 0x7f8a48068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:37.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.458+0000 7f8a64c00640 1 -- 172.21.15.161:0/3622662192 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8a60083e20 con 0x7f8a6815b250 2024-05-22T21:15:37.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.458+0000 7f8a6d000640 1 --2- 172.21.15.161:0/3622662192 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a48066690 0x7f8a48068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:37.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.458+0000 7f8a6d000640 1 --2- 172.21.15.161:0/3622662192 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a48066690 0x7f8a48068b50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c007fd0 tx=0x7f8a5c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:37.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.458+0000 7f8a64c00640 1 -- 172.21.15.161:0/3622662192 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8a60051530 con 0x7f8a6815b250 2024-05-22T21:15:37.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.646+0000 7f8a6fa00640 1 -- 172.21.15.161:0/3622662192 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a681588a0 con 0x7f8a6815b250 2024-05-22T21:15:37.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.646+0000 7f8a64c00640 1 -- 172.21.15.161:0/3622662192 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f8a60017df0 con 0x7f8a6815b250 2024-05-22T21:15:37.652 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:37.653 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/1767934947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1767934947},{"type":"v1","addr":"172.21.15.161:6836","nonce":1767934947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5036,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/1873337287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1873337287},{"type":"v1","addr":"172.21.15.179:6834","nonce":1873337287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1815838236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":1815838236},{"type":"v1","addr":"172.21.15.161:6837","nonce":1815838236}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5701,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2963712329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2963712329},{"type":"v1","addr":"172.21.15.179:6835","nonce":2963712329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[]} 2024-05-22T21:15:37.653 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 25 2024-05-22T21:15:37.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.650+0000 7f8a6fa00640 1 -- 172.21.15.161:0/3622662192 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a48066690 msgr2=0x7f8a48068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:37.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.650+0000 7f8a6fa00640 1 --2- 172.21.15.161:0/3622662192 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a48066690 0x7f8a48068b50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c007fd0 tx=0x7f8a5c009040 comp rx=0 tx=0).stop 2024-05-22T21:15:37.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.650+0000 7f8a6fa00640 1 -- 172.21.15.161:0/3622662192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6815b250 msgr2=0x7f8a681f7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:37.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.650+0000 7f8a6fa00640 1 --2- 172.21.15.161:0/3622662192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6815b250 0x7f8a681f7d70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000cb40 tx=0x7f8a6000a060 comp rx=0 tx=0).stop 2024-05-22T21:15:37.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.650+0000 7f8a6fa00640 1 -- 172.21.15.161:0/3622662192 shutdown_connections 2024-05-22T21:15:37.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.650+0000 7f8a6fa00640 1 --2- 172.21.15.161:0/3622662192 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a48066690 0x7f8a48068b50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.650+0000 7f8a6fa00640 1 --2- 172.21.15.161:0/3622662192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6815b250 0x7f8a681f7d70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.650+0000 7f8a6fa00640 1 --2- 172.21.15.161:0/3622662192 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a680cf6e0 0x7f8a681f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.650+0000 7f8a6fa00640 1 --2- 172.21.15.161:0/3622662192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a6809a420 0x7f8a680b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.650+0000 7f8a6fa00640 1 -- 172.21.15.161:0/3622662192 >> 172.21.15.161:0/3622662192 conn(0x7f8a680b2a40 msgr2=0x7f8a680cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:37.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.650+0000 7f8a6fa00640 1 -- 172.21.15.161:0/3622662192 shutdown_connections 2024-05-22T21:15:37.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.650+0000 7f8a6fa00640 1 -- 172.21.15.161:0/3622662192 wait complete. 2024-05-22T21:15:37.673 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-22T21:15:37.674 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-22T21:15:37.674 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-22T21:15:37.674 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:15:37.674 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-22T21:15:37.674 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-22T21:15:37.674 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-22T21:15:37.675 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:15:37.675 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-22T21:15:37.675 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-22T21:15:37.675 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-22T21:15:37.675 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:15:37.675 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-22T21:15:37.675 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-22T21:15:37.676 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-22T21:15:37.676 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:15:37.687 INFO:tasks.ceph.mds.a:Stopped 2024-05-22T21:15:37.688 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-22T21:15:37.689 INFO:tasks.ceph.mds.c:Stopped 2024-05-22T21:15:37.689 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-22T21:15:37.690 INFO:tasks.ceph.mds.b:Stopped 2024-05-22T21:15:37.690 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-22T21:15:37.691 INFO:tasks.ceph.mds.d:Stopped 2024-05-22T21:15:37.691 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-22T21:15:37.734 INFO:tasks.ceph.mds.a:Started 2024-05-22T21:15:37.734 INFO:tasks.ceph.mds.c:Started 2024-05-22T21:15:37.735 DEBUG:teuthology.parallel:result is None 2024-05-22T21:15:37.735 DEBUG:teuthology.parallel:result is None 2024-05-22T21:15:37.736 INFO:tasks.ceph.mds.b:Started 2024-05-22T21:15:37.736 INFO:tasks.ceph.mds.d:Started 2024-05-22T21:15:37.736 DEBUG:teuthology.parallel:result is None 2024-05-22T21:15:37.737 DEBUG:teuthology.parallel:result is None 2024-05-22T21:15:37.737 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-22T21:15:37.818 INFO:tasks.ceph.mds.c.smithi161.stdout:starting mds.c at 2024-05-22T21:15:37.818 INFO:tasks.ceph.mds.a.smithi161.stdout:starting mds.a at 2024-05-22T21:15:37.826 INFO:tasks.ceph.mds.d.smithi179.stdout:starting mds.d at 2024-05-22T21:15:37.826 INFO:tasks.ceph.mds.b.smithi179.stdout:starting mds.b at 2024-05-22T21:15:37.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.878+0000 7f3197200640 1 Processor -- start 2024-05-22T21:15:37.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.882+0000 7f3197200640 1 -- start start 2024-05-22T21:15:37.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.882+0000 7f3197200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319013cfc0 0x7f31901402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:37.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.882+0000 7f3197200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f319015af60 0x7f319013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:37.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.882+0000 7f3197200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31901400d0 con 0x7f319013fc80 2024-05-22T21:15:37.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.882+0000 7f3197200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f319013d3a0 con 0x7f319015af60 2024-05-22T21:15:37.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.882+0000 7f3197200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31900cd920 con 0x7f319013cfc0 2024-05-22T21:15:37.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.882+0000 7f318f400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f319015af60 0x7f319013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:37.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.882+0000 7f318f400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f319015af60 0x7f319013f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50068/0 (socket says 172.21.15.161:50068) 2024-05-22T21:15:37.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.882+0000 7f318f400640 1 -- 172.21.15.161:0/4062847437 learned_addr learned my addr 172.21.15.161:0/4062847437 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:37.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.882+0000 7f318fe00640 1 --2- 172.21.15.161:0/4062847437 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319013cfc0 0x7f31901402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:37.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.882+0000 7f318f400640 1 -- 172.21.15.161:0/4062847437 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319013cfc0 msgr2=0x7f31901402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:37.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.882+0000 7f318f400640 1 --2- 172.21.15.161:0/4062847437 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319013cfc0 0x7f31901402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.882+0000 7f318f400640 1 -- 172.21.15.161:0/4062847437 >> v1:172.21.15.179:6789/0 conn(0x7f319013fc80 legacy=0x7f319015dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:37.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.882+0000 7f318f400640 1 -- 172.21.15.161:0/4062847437 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31900cdc00 con 0x7f319015af60 2024-05-22T21:15:37.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.882+0000 7f318ea00640 1 -- 172.21.15.161:0/4062847437 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 904488717 0 0) 0x7f31880014a0 con 0x7f319013fc80 2024-05-22T21:15:37.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.882+0000 7f318fe00640 1 --2- 172.21.15.161:0/4062847437 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319013cfc0 0x7f31901402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:15:37.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.886+0000 7f318f400640 1 --2- 172.21.15.161:0/4062847437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319015af60 0x7f319013f740 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f317c009870 tx=0x7f317c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8a103d8638db43 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:37.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.886+0000 7f318ea00640 1 -- 172.21.15.161:0/4062847437 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c0ab600 con 0x7f319015af60 2024-05-22T21:15:37.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.886+0000 7f318ea00640 1 -- 172.21.15.161:0/4062847437 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c0a7050 con 0x7f319015af60 2024-05-22T21:15:37.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.886+0000 7f318ea00640 1 -- 172.21.15.161:0/4062847437 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c0ab980 con 0x7f319015af60 2024-05-22T21:15:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.886+0000 7f3197200640 1 -- 172.21.15.161:0/4062847437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319015af60 msgr2=0x7f319013f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.886+0000 7f3197200640 1 --2- 172.21.15.161:0/4062847437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319015af60 0x7f319013f740 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f317c009870 tx=0x7f317c01c4d0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.886+0000 7f3197200640 1 -- 172.21.15.161:0/4062847437 shutdown_connections 2024-05-22T21:15:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.886+0000 7f3197200640 1 --2- 172.21.15.161:0/4062847437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319015af60 0x7f319013f740 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.886+0000 7f3197200640 1 --2- 172.21.15.161:0/4062847437 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319013cfc0 0x7f31901402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.886+0000 7f3197200640 1 -- 172.21.15.161:0/4062847437 >> 172.21.15.161:0/4062847437 conn(0x7f319006a2d0 msgr2=0x7f319006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:37.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.886+0000 7f3197200640 1 -- 172.21.15.161:0/4062847437 shutdown_connections 2024-05-22T21:15:37.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.886+0000 7f3197200640 1 -- 172.21.15.161:0/4062847437 wait complete. 2024-05-22T21:15:37.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.886+0000 7f3197200640 1 Processor -- start 2024-05-22T21:15:37.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.886+0000 7f3197200640 1 -- start start 2024-05-22T21:15:37.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3197200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319013cfc0 0x7f3190157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:37.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3197200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f319013fc80 0x7f319015b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:37.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3197200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319015af60 0x7f319015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:37.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3197200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31900ae010 con 0x7f319015af60 2024-05-22T21:15:37.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3197200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3190157da0 con 0x7f319013fc80 2024-05-22T21:15:37.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3197200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31900ac9c0 con 0x7f319013cfc0 2024-05-22T21:15:37.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3195200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319015af60 0x7f319015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:37.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3195200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319015af60 0x7f319015ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50078/0 (socket says 172.21.15.161:50078) 2024-05-22T21:15:37.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3195200640 1 -- 172.21.15.161:0/3524511747 learned_addr learned my addr 172.21.15.161:0/3524511747 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:37.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f318fe00640 1 --2- 172.21.15.161:0/3524511747 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319013cfc0 0x7f3190157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:37.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f318f400640 1 --2- 172.21.15.161:0/3524511747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f319013fc80 0x7f319015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:37.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3195200640 1 -- 172.21.15.161:0/3524511747 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319013cfc0 msgr2=0x7f3190157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:37.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3195200640 1 --2- 172.21.15.161:0/3524511747 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319013cfc0 0x7f3190157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3195200640 1 -- 172.21.15.161:0/3524511747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f319013fc80 msgr2=0x7f319015b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:37.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3195200640 1 --2- 172.21.15.161:0/3524511747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f319013fc80 0x7f319015b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:37.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3195200640 1 -- 172.21.15.161:0/3524511747 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3180009420 con 0x7f319015af60 2024-05-22T21:15:37.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3195200640 1 --2- 172.21.15.161:0/3524511747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319015af60 0x7f319015ba80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f318800c7c0 tx=0x7f318800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:37.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3187e00640 1 -- 172.21.15.161:0/3524511747 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188003db0 con 0x7f319015af60 2024-05-22T21:15:37.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3187e00640 1 -- 172.21.15.161:0/3524511747 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3188005070 con 0x7f319015af60 2024-05-22T21:15:37.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3197200640 1 -- 172.21.15.161:0/3524511747 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317c0094d0 con 0x7f319015af60 2024-05-22T21:15:37.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3187e00640 1 -- 172.21.15.161:0/3524511747 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188005460 con 0x7f319015af60 2024-05-22T21:15:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3197200640 1 -- 172.21.15.161:0/3524511747 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31900adc30 con 0x7f319015af60 2024-05-22T21:15:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.890+0000 7f3197200640 1 -- 172.21.15.161:0/3524511747 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31900ae150 con 0x7f319015af60 2024-05-22T21:15:37.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.894+0000 7f3187e00640 1 -- 172.21.15.161:0/3524511747 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f31880055c0 con 0x7f319015af60 2024-05-22T21:15:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.894+0000 7f3187e00640 1 --2- 172.21.15.161:0/3524511747 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3170066690 0x7f3170068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.894+0000 7f3187e00640 1 -- 172.21.15.161:0/3524511747 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3188083600 con 0x7f319015af60 2024-05-22T21:15:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.894+0000 7f318fe00640 1 --2- 172.21.15.161:0/3524511747 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3170066690 0x7f3170068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:37.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.894+0000 7f318fe00640 1 --2- 172.21.15.161:0/3524511747 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3170066690 0x7f3170068b50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f319013a550 tx=0x7f3180008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:37.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:37.898+0000 7f3187e00640 1 -- 172.21.15.161:0/3524511747 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f31900ae150 con 0x7f319015af60 2024-05-22T21:15:38.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:38.062+0000 7f3197200640 1 -- 172.21.15.161:0/3524511747 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f31901400d0 con 0x7f319015af60 2024-05-22T21:15:38.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:38.854+0000 7f3187e00640 1 -- 172.21.15.161:0/3524511747 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v63) v1 ==== 97+0+0 (secure 0 0 0) 0x7f31901400d0 con 0x7f319015af60 2024-05-22T21:15:38.859 INFO:teuthology.orchestra.run.smithi161.stderr: removed all blocklist entries 2024-05-22T21:15:38.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:38.854+0000 7f3197200640 1 -- 172.21.15.161:0/3524511747 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3170066690 msgr2=0x7f3170068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:38.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:38.854+0000 7f3197200640 1 --2- 172.21.15.161:0/3524511747 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3170066690 0x7f3170068b50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f319013a550 tx=0x7f3180008040 comp rx=0 tx=0).stop 2024-05-22T21:15:38.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:38.854+0000 7f3197200640 1 -- 172.21.15.161:0/3524511747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319015af60 msgr2=0x7f319015ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:38.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:38.854+0000 7f3197200640 1 --2- 172.21.15.161:0/3524511747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319015af60 0x7f319015ba80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f318800c7c0 tx=0x7f318800cc80 comp rx=0 tx=0).stop 2024-05-22T21:15:38.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:38.854+0000 7f3197200640 1 -- 172.21.15.161:0/3524511747 shutdown_connections 2024-05-22T21:15:38.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:38.854+0000 7f3197200640 1 --2- 172.21.15.161:0/3524511747 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3170066690 0x7f3170068b50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:38.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:38.854+0000 7f3197200640 1 --2- 172.21.15.161:0/3524511747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319015af60 0x7f319015ba80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:38.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:38.854+0000 7f3197200640 1 --2- 172.21.15.161:0/3524511747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f319013fc80 0x7f319015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:38.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:38.854+0000 7f3197200640 1 --2- 172.21.15.161:0/3524511747 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319013cfc0 0x7f3190157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:38.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:38.854+0000 7f3197200640 1 -- 172.21.15.161:0/3524511747 >> 172.21.15.161:0/3524511747 conn(0x7f319006a2d0 msgr2=0x7f319006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:38.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:38.854+0000 7f3197200640 1 -- 172.21.15.161:0/3524511747 shutdown_connections 2024-05-22T21:15:38.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:38.858+0000 7f3197200640 1 -- 172.21.15.161:0/3524511747 wait complete. 2024-05-22T21:15:38.884 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-22T21:15:39.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.018+0000 7f0f20c00640 1 Processor -- start 2024-05-22T21:15:39.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.018+0000 7f0f20c00640 1 -- start start 2024-05-22T21:15:39.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.018+0000 7f0f20c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0f1c15ec00 0x7f0f1c06fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.018+0000 7f0f20c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f1c0cd4a0 con 0x7f0f1c1447d0 2024-05-22T21:15:39.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.018+0000 7f0f20c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f1c15f030 con 0x7f0f1c0b1420 2024-05-22T21:15:39.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.018+0000 7f0f20c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f1c15fb60 con 0x7f0f1c15ec00 2024-05-22T21:15:39.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.018+0000 7f0f19e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0f1c1447d0 0x7f0f1c0b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:49526/0 (socket says 172.21.15.161:49526) 2024-05-22T21:15:39.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.018+0000 7f0f19e00640 1 -- 172.21.15.161:0/630513051 learned_addr learned my addr 172.21.15.161:0/630513051 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:39.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.018+0000 7f0f1a800640 1 --2- 172.21.15.161:0/630513051 >> v2:172.21.15.179:3300/0 conn(0x7f0f1c15ec00 0x7f0f1c06fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.018+0000 7f0f19400640 1 -- 172.21.15.161:0/630513051 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408814155 0 0) 0x7f0f1c0cd4a0 con 0x7f0f1c1447d0 2024-05-22T21:15:39.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.018+0000 7f0f19400640 1 -- 172.21.15.161:0/630513051 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f00003530 con 0x7f0f1c1447d0 2024-05-22T21:15:39.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.018+0000 7f0f1a800640 1 -- 172.21.15.161:0/630513051 >> v1:172.21.15.179:6789/0 conn(0x7f0f1c0b1420 legacy=0x7f0f1c0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.018+0000 7f0f1a800640 1 -- 172.21.15.161:0/630513051 >> v1:172.21.15.161:6789/0 conn(0x7f0f1c1447d0 legacy=0x7f0f1c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.018+0000 7f0f19400640 1 -- 172.21.15.161:0/630513051 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1468223666 0 0) 0x7f0f00003530 con 0x7f0f1c1447d0 2024-05-22T21:15:39.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.018+0000 7f0f1a800640 1 -- 172.21.15.161:0/630513051 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c15f2c0 con 0x7f0f1c15ec00 2024-05-22T21:15:39.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.022+0000 7f0f1a800640 1 --2- 172.21.15.161:0/630513051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f1c15ec00 0x7f0f1c06fc50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0f140097c0 tx=0x7f0f1401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99a7c1b2429ebe41 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:39.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.022+0000 7f0f19400640 1 -- 172.21.15.161:0/630513051 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c001a00 con 0x7f0f1c15ec00 2024-05-22T21:15:39.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.022+0000 7f0f19400640 1 -- 172.21.15.161:0/630513051 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f140ac440 con 0x7f0f1c15ec00 2024-05-22T21:15:39.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.022+0000 7f0f19400640 1 -- 172.21.15.161:0/630513051 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f140ac7d0 con 0x7f0f1c15ec00 2024-05-22T21:15:39.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.022+0000 7f0f20c00640 1 -- 172.21.15.161:0/630513051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f1c15ec00 msgr2=0x7f0f1c06fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.022+0000 7f0f20c00640 1 --2- 172.21.15.161:0/630513051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f1c15ec00 0x7f0f1c06fc50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0f140097c0 tx=0x7f0f1401c390 comp rx=0 tx=0).stop 2024-05-22T21:15:39.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.022+0000 7f0f20c00640 1 -- 172.21.15.161:0/630513051 shutdown_connections 2024-05-22T21:15:39.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.022+0000 7f0f20c00640 1 --2- 172.21.15.161:0/630513051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f1c15ec00 0x7f0f1c06fc50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.022+0000 7f0f20c00640 1 -- 172.21.15.161:0/630513051 >> 172.21.15.161:0/630513051 conn(0x7f0f1c06a0a0 msgr2=0x7f0f1c06a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:39.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.022+0000 7f0f20c00640 1 -- 172.21.15.161:0/630513051 shutdown_connections 2024-05-22T21:15:39.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.022+0000 7f0f20c00640 1 -- 172.21.15.161:0/630513051 wait complete. 2024-05-22T21:15:39.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f20c00640 1 Processor -- start 2024-05-22T21:15:39.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f20c00640 1 -- start start 2024-05-22T21:15:39.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f20c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f1c0b1420 0x7f0f1c0b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f20c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f1c1447d0 0x7f0f1c1434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f20c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f1c15ec00 0x7f0f1c13a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f20c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c0accc0 con 0x7f0f1c0b1420 2024-05-22T21:15:39.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f20c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c0b3640 con 0x7f0f1c1447d0 2024-05-22T21:15:39.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f20c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c0b38c0 con 0x7f0f1c15ec00 2024-05-22T21:15:39.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f19e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f1c0b1420 0x7f0f1c0b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f19e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f1c0b1420 0x7f0f1c0b3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50086/0 (socket says 172.21.15.161:50086) 2024-05-22T21:15:39.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f19e00640 1 -- 172.21.15.161:0/1535340395 learned_addr learned my addr 172.21.15.161:0/1535340395 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:39.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f1a800640 1 --2- 172.21.15.161:0/1535340395 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f1c15ec00 0x7f0f1c13a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f11400640 1 --2- 172.21.15.161:0/1535340395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f1c1447d0 0x7f0f1c1434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f19e00640 1 -- 172.21.15.161:0/1535340395 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f1c15ec00 msgr2=0x7f0f1c13a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f19e00640 1 --2- 172.21.15.161:0/1535340395 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f1c15ec00 0x7f0f1c13a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f19e00640 1 -- 172.21.15.161:0/1535340395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f1c1447d0 msgr2=0x7f0f1c1434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f19e00640 1 --2- 172.21.15.161:0/1535340395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f1c1447d0 0x7f0f1c1434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f19e00640 1 -- 172.21.15.161:0/1535340395 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f14009420 con 0x7f0f1c0b1420 2024-05-22T21:15:39.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f19e00640 1 --2- 172.21.15.161:0/1535340395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f1c0b1420 0x7f0f1c0b3100 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c002000 tx=0x7f0f0c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:39.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f12a00640 1 -- 172.21.15.161:0/1535340395 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c012400 con 0x7f0f1c0b1420 2024-05-22T21:15:39.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f12a00640 1 -- 172.21.15.161:0/1535340395 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0c012560 con 0x7f0f1c0b1420 2024-05-22T21:15:39.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f20c00640 1 -- 172.21.15.161:0/1535340395 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f1c0b3ba0 con 0x7f0f1c0b1420 2024-05-22T21:15:39.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f12a00640 1 -- 172.21.15.161:0/1535340395 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c012930 con 0x7f0f1c0b1420 2024-05-22T21:15:39.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.026+0000 7f0f20c00640 1 -- 172.21.15.161:0/1535340395 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1c0ac950 con 0x7f0f1c0b1420 2024-05-22T21:15:39.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.030+0000 7f0f20c00640 1 -- 172.21.15.161:0/1535340395 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f1c067220 con 0x7f0f1c0b1420 2024-05-22T21:15:39.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.030+0000 7f0f12a00640 1 -- 172.21.15.161:0/1535340395 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0f0c012a90 con 0x7f0f1c0b1420 2024-05-22T21:15:39.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.030+0000 7f0f12a00640 1 --2- 172.21.15.161:0/1535340395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0ef8066690 0x7f0ef8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.030+0000 7f0f12a00640 1 -- 172.21.15.161:0/1535340395 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0f0c084610 con 0x7f0f1c0b1420 2024-05-22T21:15:39.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.030+0000 7f0f11400640 1 --2- 172.21.15.161:0/1535340395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0ef8066690 0x7f0ef8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.030+0000 7f0f11400640 1 --2- 172.21.15.161:0/1535340395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0ef8066690 0x7f0ef8068b50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c157b00 tx=0x7f0f04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:39.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.034+0000 7f0f12a00640 1 -- 172.21.15.161:0/1535340395 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0f0c051d60 con 0x7f0f1c0b1420 2024-05-22T21:15:39.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.246+0000 7f0f20c00640 1 -- 172.21.15.161:0/1535340395 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0f1c0cdda0 con 0x7f0f1c0b1420 2024-05-22T21:15:39.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.246+0000 7f0f12a00640 1 -- 172.21.15.161:0/1535340395 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0f0c055380 con 0x7f0f1c0b1420 2024-05-22T21:15:39.251 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:39.251 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:15:39.251 INFO:teuthology.orchestra.run.smithi161.stdout: "auth_dump": [ 2024-05-22T21:15:39.251 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.251 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.a", 2024-05-22T21:15:39.251 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m00QTFRAAPSRrpBRY75FlcWf3QxOOfg==", 2024-05-22T21:15:39.251 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.251 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:15:39.251 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:15:39.251 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:15:39.252 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:15:39.252 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.252 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.252 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.252 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.b", 2024-05-22T21:15:39.252 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m/ipAIRAA4gj5jGqTpNCr1K2OU/bsAQ==", 2024-05-22T21:15:39.252 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.252 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:15:39.252 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:15:39.252 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:15:39.252 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:15:39.252 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.252 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.253 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.c", 2024-05-22T21:15:39.253 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mWlM2HBAAEmJJWMDlTG9E7Qzld2gd5w==", 2024-05-22T21:15:39.253 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.253 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:15:39.253 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:15:39.253 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:15:39.253 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:15:39.253 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.253 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.253 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.253 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.d", 2024-05-22T21:15:39.253 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mmsuaKBAAqpCYRZZUGK5ZuxflUBcPgg==", 2024-05-22T21:15:39.254 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.254 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:15:39.254 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:15:39.254 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:15:39.254 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:15:39.254 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.254 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.254 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.0", 2024-05-22T21:15:39.254 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05mJZokLhAAh2vbQnRX25iYUIbvVmURuQ==", 2024-05-22T21:15:39.254 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.254 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:15:39.255 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:15:39.255 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:15:39.255 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.255 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.1", 2024-05-22T21:15:39.255 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05m8kWNNhAA6aaVT2xshFkqnk4G4gOttA==", 2024-05-22T21:15:39.255 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.255 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:15:39.255 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:15:39.255 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:15:39.255 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.255 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.255 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.256 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.2", 2024-05-22T21:15:39.256 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05ml1k9AxAAtL3XaJJI6cilpmvurh18Pw==", 2024-05-22T21:15:39.256 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.256 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:15:39.256 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:15:39.256 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:15:39.256 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.256 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.256 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.256 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.3", 2024-05-22T21:15:39.256 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05mbNBgCxAAas1L+QR4Aez+H1G0EVltfg==", 2024-05-22T21:15:39.256 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.256 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:15:39.257 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:15:39.257 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:15:39.257 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.257 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.257 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.257 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.4", 2024-05-22T21:15:39.257 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mOPLPABAAZBGIMF/FasNB27n/7di0AA==", 2024-05-22T21:15:39.257 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.257 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:15:39.257 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:15:39.257 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:15:39.257 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.257 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.257 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.258 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.5", 2024-05-22T21:15:39.258 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mT1poCRAASXm4tdP01pA1VOviqeoLVw==", 2024-05-22T21:15:39.258 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.258 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:15:39.258 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:15:39.258 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:15:39.258 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.258 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.258 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.258 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.6", 2024-05-22T21:15:39.258 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05m1ZPRERAA2S7WeTwysLCZm1OB/qCrAA==", 2024-05-22T21:15:39.258 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.259 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:15:39.259 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:15:39.259 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:15:39.259 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.259 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.259 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.7", 2024-05-22T21:15:39.259 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mCrUhGhAAvXoFWV7X3303rfXaxmfWqA==", 2024-05-22T21:15:39.259 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.259 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:15:39.259 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:15:39.259 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:15:39.259 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.259 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.260 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.260 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.0", 2024-05-22T21:15:39.260 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mh9B7LBAAevhvj5KpwP9pEIpJcxUvFA==", 2024-05-22T21:15:39.260 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.260 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:15:39.260 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow r", 2024-05-22T21:15:39.260 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-22T21:15:39.260 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.260 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.260 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.260 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.1", 2024-05-22T21:15:39.260 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mMnHtNRAAg/a/Mb1oRqx73d9efIHCfA==", 2024-05-22T21:15:39.260 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.261 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:15:39.261 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:15:39.261 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:15:39.261 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:15:39.261 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.261 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.261 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.261 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.2", 2024-05-22T21:15:39.261 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mOzdkMhAAbTiNdvyxqIyrluMiZW0cOw==", 2024-05-22T21:15:39.261 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.261 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:15:39.261 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:15:39.261 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:15:39.261 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:15:39.262 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.262 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.262 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.262 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.3", 2024-05-22T21:15:39.262 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDYX05mr4oxABAA1+bQQbxhoeU+kb0OCa+Wyw==", 2024-05-22T21:15:39.262 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.262 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:15:39.262 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:15:39.262 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:15:39.263 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:15:39.263 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.263 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.263 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.263 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.admin", 2024-05-22T21:15:39.263 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDWX05mHZY0NRAAiBvMDkrCZ+MgG2iwBH+bcg==", 2024-05-22T21:15:39.263 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.263 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:15:39.263 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:15:39.263 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:15:39.263 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:15:39.263 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.263 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.263 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.264 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.x", 2024-05-22T21:15:39.264 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mH2kuERAAgigMUWK3M3WaMXlqGUcNrw==", 2024-05-22T21:15:39.264 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.264 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:15:39.264 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:15:39.264 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:15:39.264 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.264 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:15:39.264 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:39.264 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.y", 2024-05-22T21:15:39.264 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mZRdiDRAA0vUwD+rPSfyhjgA+9XPp5w==", 2024-05-22T21:15:39.264 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:15:39.264 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:15:39.265 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:15:39.265 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:15:39.265 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.265 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:39.265 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:15:39.265 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:15:39.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.246+0000 7f0f20c00640 1 -- 172.21.15.161:0/1535340395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0ef8066690 msgr2=0x7f0ef8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.246+0000 7f0f20c00640 1 --2- 172.21.15.161:0/1535340395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0ef8066690 0x7f0ef8068b50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c157b00 tx=0x7f0f04009040 comp rx=0 tx=0).stop 2024-05-22T21:15:39.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.246+0000 7f0f20c00640 1 -- 172.21.15.161:0/1535340395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f1c0b1420 msgr2=0x7f0f1c0b3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.246+0000 7f0f20c00640 1 --2- 172.21.15.161:0/1535340395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f1c0b1420 0x7f0f1c0b3100 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c002000 tx=0x7f0f0c00a430 comp rx=0 tx=0).stop 2024-05-22T21:15:39.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.246+0000 7f0f20c00640 1 -- 172.21.15.161:0/1535340395 shutdown_connections 2024-05-22T21:15:39.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.246+0000 7f0f20c00640 1 --2- 172.21.15.161:0/1535340395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0ef8066690 0x7f0ef8068b50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.246+0000 7f0f20c00640 1 --2- 172.21.15.161:0/1535340395 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f1c15ec00 0x7f0f1c13a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.246+0000 7f0f20c00640 1 --2- 172.21.15.161:0/1535340395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f1c1447d0 0x7f0f1c1434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.246+0000 7f0f20c00640 1 --2- 172.21.15.161:0/1535340395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f1c0b1420 0x7f0f1c0b3100 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.246+0000 7f0f20c00640 1 -- 172.21.15.161:0/1535340395 >> 172.21.15.161:0/1535340395 conn(0x7f0f1c06a0a0 msgr2=0x7f0f1c06bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:39.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.246+0000 7f0f20c00640 1 -- 172.21.15.161:0/1535340395 shutdown_connections 2024-05-22T21:15:39.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.246+0000 7f0f20c00640 1 -- 172.21.15.161:0/1535340395 wait complete. 2024-05-22T21:15:39.272 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:15:39.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcc2600640 1 Processor -- start 2024-05-22T21:15:39.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcc2600640 1 -- start start 2024-05-22T21:15:39.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcc2600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbcbc15d430 0x7fbcbc0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcc2600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcbc0ac8b0 0x7fbcbc06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcc2600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcbc071fd0 con 0x7fbcbc0accf0 2024-05-22T21:15:39.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcc2600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcbc070e80 con 0x7fbcbc15d430 2024-05-22T21:15:39.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcc2600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc06c510 con 0x7fbcbc0ac8b0 2024-05-22T21:15:39.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcbbe00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fbcbc0accf0 0x7fbcbc0b1ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:49536/0 (socket says 172.21.15.161:49536) 2024-05-22T21:15:39.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcbbe00640 1 -- 172.21.15.161:0/3658830584 learned_addr learned my addr 172.21.15.161:0/3658830584 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:39.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcbb400640 1 --2- 172.21.15.161:0/3658830584 >> v2:172.21.15.179:3300/0 conn(0x7fbcbc15d430 0x7fbcbc0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcb2a00640 1 --2- 172.21.15.161:0/3658830584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcbc0ac8b0 0x7fbcbc06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcbaa00640 1 -- 172.21.15.161:0/3658830584 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491309341 0 0) 0x7fbcbc071fd0 con 0x7fbcbc0accf0 2024-05-22T21:15:39.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcbaa00640 1 -- 172.21.15.161:0/3658830584 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbca0003530 con 0x7fbcbc0accf0 2024-05-22T21:15:39.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcbb400640 1 -- 172.21.15.161:0/3658830584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcbc0ac8b0 msgr2=0x7fbcbc06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:39.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcbb400640 1 --2- 172.21.15.161:0/3658830584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcbc0ac8b0 0x7fbcbc06cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcbb400640 1 -- 172.21.15.161:0/3658830584 >> v1:172.21.15.161:6789/0 conn(0x7fbcbc0accf0 legacy=0x7fbcbc0b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcbb400640 1 -- 172.21.15.161:0/3658830584 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc06c7f0 con 0x7fbcbc15d430 2024-05-22T21:15:39.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.402+0000 7fbcb2a00640 1 --2- 172.21.15.161:0/3658830584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcbc0ac8b0 0x7fbcbc06cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:15:39.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.406+0000 7fbcbb400640 1 --2- 172.21.15.161:0/3658830584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcbc15d430 0x7fbcbc0707c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009d90 tx=0x7fbcac01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75ce83a73b128601 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:39.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.406+0000 7fbcbaa00640 1 -- 172.21.15.161:0/3658830584 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac0a58d0 con 0x7fbcbc15d430 2024-05-22T21:15:39.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.406+0000 7fbcbaa00640 1 -- 172.21.15.161:0/3658830584 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcac0a5a30 con 0x7fbcbc15d430 2024-05-22T21:15:39.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.406+0000 7fbcbaa00640 1 -- 172.21.15.161:0/3658830584 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac0a5dc0 con 0x7fbcbc15d430 2024-05-22T21:15:39.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.406+0000 7fbcc2600640 1 -- 172.21.15.161:0/3658830584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcbc15d430 msgr2=0x7fbcbc0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.406+0000 7fbcc2600640 1 --2- 172.21.15.161:0/3658830584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcbc15d430 0x7fbcbc0707c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009d90 tx=0x7fbcac01c8b0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.406+0000 7fbcc2600640 1 -- 172.21.15.161:0/3658830584 shutdown_connections 2024-05-22T21:15:39.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.406+0000 7fbcc2600640 1 --2- 172.21.15.161:0/3658830584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcbc0ac8b0 0x7fbcbc06cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.406+0000 7fbcc2600640 1 --2- 172.21.15.161:0/3658830584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcbc15d430 0x7fbcbc0707c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.406+0000 7fbcc2600640 1 -- 172.21.15.161:0/3658830584 >> 172.21.15.161:0/3658830584 conn(0x7fbcbc0b0bc0 msgr2=0x7fbcbc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:39.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.406+0000 7fbcc2600640 1 -- 172.21.15.161:0/3658830584 shutdown_connections 2024-05-22T21:15:39.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.406+0000 7fbcc2600640 1 -- 172.21.15.161:0/3658830584 wait complete. 2024-05-22T21:15:39.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcc2600640 1 Processor -- start 2024-05-22T21:15:39.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcc2600640 1 -- start start 2024-05-22T21:15:39.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcc2600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcbc0ac8b0 0x7fbcbc151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcc2600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcbc0accf0 0x7fbcbc153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcc2600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcbc15d430 0x7fbcbc1487f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcc2600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc152e20 con 0x7fbcbc15d430 2024-05-22T21:15:39.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcc2600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc146810 con 0x7fbcbc0accf0 2024-05-22T21:15:39.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcc2600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc155150 con 0x7fbcbc0ac8b0 2024-05-22T21:15:39.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcbbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcbc15d430 0x7fbcbc1487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcbbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcbc15d430 0x7fbcbc1487f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50094/0 (socket says 172.21.15.161:50094) 2024-05-22T21:15:39.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcbbe00640 1 -- 172.21.15.161:0/3751073378 learned_addr learned my addr 172.21.15.161:0/3751073378 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:39.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcb2a00640 1 --2- 172.21.15.161:0/3751073378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcbc0accf0 0x7fbcbc153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcbb400640 1 --2- 172.21.15.161:0/3751073378 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcbc0ac8b0 0x7fbcbc151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcbbe00640 1 -- 172.21.15.161:0/3751073378 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcbc0ac8b0 msgr2=0x7fbcbc151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcbbe00640 1 --2- 172.21.15.161:0/3751073378 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcbc0ac8b0 0x7fbcbc151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcbbe00640 1 -- 172.21.15.161:0/3751073378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcbc0accf0 msgr2=0x7fbcbc153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcbbe00640 1 --2- 172.21.15.161:0/3751073378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcbc0accf0 0x7fbcbc153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcbbe00640 1 -- 172.21.15.161:0/3751073378 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca4009420 con 0x7fbcbc15d430 2024-05-22T21:15:39.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcbbe00640 1 --2- 172.21.15.161:0/3751073378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcbc15d430 0x7fbcbc1487f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400c390 tx=0x7fbcb400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:39.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcb3e00640 1 -- 172.21.15.161:0/3751073378 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4005cf0 con 0x7fbcbc15d430 2024-05-22T21:15:39.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcb3e00640 1 -- 172.21.15.161:0/3751073378 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb4005e50 con 0x7fbcbc15d430 2024-05-22T21:15:39.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcc2600640 1 -- 172.21.15.161:0/3751073378 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcac009a70 con 0x7fbcbc15d430 2024-05-22T21:15:39.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcb3e00640 1 -- 172.21.15.161:0/3751073378 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb40072d0 con 0x7fbcbc15d430 2024-05-22T21:15:39.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcc2600640 1 -- 172.21.15.161:0/3751073378 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcbc152760 con 0x7fbcbc15d430 2024-05-22T21:15:39.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.410+0000 7fbcc2600640 1 -- 172.21.15.161:0/3751073378 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcbc067220 con 0x7fbcbc15d430 2024-05-22T21:15:39.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.414+0000 7fbcb3e00640 1 -- 172.21.15.161:0/3751073378 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbcb4007430 con 0x7fbcbc15d430 2024-05-22T21:15:39.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.414+0000 7fbcb3e00640 1 --2- 172.21.15.161:0/3751073378 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc98066690 0x7fbc98068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.414+0000 7fbcb3e00640 1 -- 172.21.15.161:0/3751073378 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbcb4083250 con 0x7fbcbc15d430 2024-05-22T21:15:39.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.414+0000 7fbcbb400640 1 --2- 172.21.15.161:0/3751073378 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc98066690 0x7fbc98068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.414+0000 7fbcbb400640 1 --2- 172.21.15.161:0/3751073378 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc98066690 0x7fbc98068b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0af000 tx=0x7fbcac0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:39.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.418+0000 7fbcb3e00640 1 -- 172.21.15.161:0/3751073378 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbcb40509b0 con 0x7fbcbc15d430 2024-05-22T21:15:39.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.574+0000 7fbcc2600640 1 -- 172.21.15.161:0/3751073378 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcbc0ae7a0 con 0x7fbcbc15d430 2024-05-22T21:15:39.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.574+0000 7fbcb3e00640 1 -- 172.21.15.161:0/3751073378 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+12437 (secure 0 0 0) 0x7fbcb4053fd0 con 0x7fbcbc15d430 2024-05-22T21:15:39.579 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:39.579 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":63,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:15:38.850870+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:15:39.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.574+0000 7fbcc2600640 1 -- 172.21.15.161:0/3751073378 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc98066690 msgr2=0x7fbc98068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.574+0000 7fbcc2600640 1 --2- 172.21.15.161:0/3751073378 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc98066690 0x7fbc98068b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0af000 tx=0x7fbcac0a7070 comp rx=0 tx=0).stop 2024-05-22T21:15:39.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.574+0000 7fbcc2600640 1 -- 172.21.15.161:0/3751073378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcbc15d430 msgr2=0x7fbcbc1487f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.574+0000 7fbcc2600640 1 --2- 172.21.15.161:0/3751073378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcbc15d430 0x7fbcbc1487f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400c390 tx=0x7fbcb400c850 comp rx=0 tx=0).stop 2024-05-22T21:15:39.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.574+0000 7fbcc2600640 1 -- 172.21.15.161:0/3751073378 shutdown_connections 2024-05-22T21:15:39.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.574+0000 7fbcc2600640 1 --2- 172.21.15.161:0/3751073378 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc98066690 0x7fbc98068b50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.574+0000 7fbcc2600640 1 --2- 172.21.15.161:0/3751073378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcbc15d430 0x7fbcbc1487f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.574+0000 7fbcc2600640 1 --2- 172.21.15.161:0/3751073378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcbc0accf0 0x7fbcbc153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.574+0000 7fbcc2600640 1 --2- 172.21.15.161:0/3751073378 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcbc0ac8b0 0x7fbcbc151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.574+0000 7fbcc2600640 1 -- 172.21.15.161:0/3751073378 >> 172.21.15.161:0/3751073378 conn(0x7fbcbc0b0bc0 msgr2=0x7fbcbc0ad120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:39.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.574+0000 7fbcc2600640 1 -- 172.21.15.161:0/3751073378 shutdown_connections 2024-05-22T21:15:39.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.574+0000 7fbcc2600640 1 -- 172.21.15.161:0/3751073378 wait complete. 2024-05-22T21:15:39.600 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:15:39.600 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:15:39.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.730+0000 7fa7fc400640 1 Processor -- start 2024-05-22T21:15:39.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.730+0000 7fa7fc400640 1 -- start start 2024-05-22T21:15:39.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7fc400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa7f4143c40 0x7fa7f415b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7fc400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7f41444a0 0x7fa7f406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7fc400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f4144900 0x7fa7f406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7fc400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f415efb0 con 0x7fa7f41444a0 2024-05-22T21:15:39.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7fc400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f415f290 con 0x7fa7f4143c40 2024-05-22T21:15:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7fc400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f415f570 con 0x7fa7f4144900 2024-05-22T21:15:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7f9000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7f41444a0 0x7fa7f406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7f9000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7f41444a0 0x7fa7f406c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50108/0 (socket says 172.21.15.161:50108) 2024-05-22T21:15:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7f9000640 1 -- 172.21.15.161:0/1866465184 learned_addr learned my addr 172.21.15.161:0/1866465184 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7f9a00640 1 --2- 172.21.15.161:0/1866465184 >> v2:172.21.15.179:3300/0 conn(0x7fa7f4143c40 0x7fa7f415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7fa400640 1 --2- 172.21.15.161:0/1866465184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f4144900 0x7fa7f406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7f9000640 1 -- 172.21.15.161:0/1866465184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f4144900 msgr2=0x7fa7f406c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7f9000640 1 --2- 172.21.15.161:0/1866465184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f4144900 0x7fa7f406c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7f9000640 1 -- 172.21.15.161:0/1866465184 >> v2:172.21.15.179:3300/0 conn(0x7fa7f4143c40 msgr2=0x7fa7f415b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7f9000640 1 --2- 172.21.15.161:0/1866465184 >> v2:172.21.15.179:3300/0 conn(0x7fa7f4143c40 0x7fa7f415b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7f9000640 1 -- 172.21.15.161:0/1866465184 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f415f850 con 0x7fa7f41444a0 2024-05-22T21:15:39.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.734+0000 7fa7fa400640 1 --2- 172.21.15.161:0/1866465184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f4144900 0x7fa7f406c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:39.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.738+0000 7fa7f9000640 1 --2- 172.21.15.161:0/1866465184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f41444a0 0x7fa7f406c2a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40724b0 tx=0x7fa7e001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e941b4941026fa server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:39.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.738+0000 7fa7f3e00640 1 -- 172.21.15.161:0/1866465184 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00ab640 con 0x7fa7f41444a0 2024-05-22T21:15:39.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.738+0000 7fa7f3e00640 1 -- 172.21.15.161:0/1866465184 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e00a7070 con 0x7fa7f41444a0 2024-05-22T21:15:39.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.738+0000 7fa7f3e00640 1 -- 172.21.15.161:0/1866465184 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00ab9c0 con 0x7fa7f41444a0 2024-05-22T21:15:39.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.738+0000 7fa7fc400640 1 -- 172.21.15.161:0/1866465184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f41444a0 msgr2=0x7fa7f406c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.738+0000 7fa7fc400640 1 --2- 172.21.15.161:0/1866465184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f41444a0 0x7fa7f406c2a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40724b0 tx=0x7fa7e001c520 comp rx=0 tx=0).stop 2024-05-22T21:15:39.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.738+0000 7fa7fc400640 1 -- 172.21.15.161:0/1866465184 shutdown_connections 2024-05-22T21:15:39.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.738+0000 7fa7fc400640 1 --2- 172.21.15.161:0/1866465184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f4144900 0x7fa7f406c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.738+0000 7fa7fc400640 1 --2- 172.21.15.161:0/1866465184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f41444a0 0x7fa7f406c2a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.738+0000 7fa7fc400640 1 --2- 172.21.15.161:0/1866465184 >> v2:172.21.15.179:3300/0 conn(0x7fa7f4143c40 0x7fa7f415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.738+0000 7fa7fc400640 1 -- 172.21.15.161:0/1866465184 >> 172.21.15.161:0/1866465184 conn(0x7fa7f406a2d0 msgr2=0x7fa7f406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:39.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.738+0000 7fa7fc400640 1 -- 172.21.15.161:0/1866465184 shutdown_connections 2024-05-22T21:15:39.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.738+0000 7fa7fc400640 1 -- 172.21.15.161:0/1866465184 wait complete. 2024-05-22T21:15:39.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.738+0000 7fa7fc400640 1 Processor -- start 2024-05-22T21:15:39.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.738+0000 7fa7fc400640 1 -- start start 2024-05-22T21:15:39.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.738+0000 7fa7fc400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f4143c40 0x7fa7f4135fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7fc400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f41444a0 0x7fa7f4155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7fc400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7f4144900 0x7fa7f413ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7fc400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f4136520 con 0x7fa7f41444a0 2024-05-22T21:15:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7fc400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f4152000 con 0x7fa7f4144900 2024-05-22T21:15:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7fc400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f41522e0 con 0x7fa7f4143c40 2024-05-22T21:15:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7f9000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f41444a0 0x7fa7f4155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7f9000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f41444a0 0x7fa7f4155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50118/0 (socket says 172.21.15.161:50118) 2024-05-22T21:15:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7f9000640 1 -- 172.21.15.161:0/1589442060 learned_addr learned my addr 172.21.15.161:0/1589442060 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7f9a00640 1 --2- 172.21.15.161:0/1589442060 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f4143c40 0x7fa7f4135fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7fa400640 1 --2- 172.21.15.161:0/1589442060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7f4144900 0x7fa7f413ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7f9000640 1 -- 172.21.15.161:0/1589442060 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f4143c40 msgr2=0x7fa7f4135fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7f9000640 1 --2- 172.21.15.161:0/1589442060 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f4143c40 0x7fa7f4135fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7f9000640 1 -- 172.21.15.161:0/1589442060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7f4144900 msgr2=0x7fa7f413ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7f9000640 1 --2- 172.21.15.161:0/1589442060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7f4144900 0x7fa7f413ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7f9000640 1 -- 172.21.15.161:0/1589442060 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e00094d0 con 0x7fa7f41444a0 2024-05-22T21:15:39.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7fa400640 1 --2- 172.21.15.161:0/1589442060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7f4144900 0x7fa7f413ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:39.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7f9000640 1 --2- 172.21.15.161:0/1589442060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f41444a0 0x7fa7f4155450 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa7e001abd0 tx=0x7fa7e001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:39.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7f1600640 1 -- 172.21.15.161:0/1589442060 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00be3f0 con 0x7fa7f41444a0 2024-05-22T21:15:39.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7f1600640 1 -- 172.21.15.161:0/1589442060 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e00be550 con 0x7fa7f41444a0 2024-05-22T21:15:39.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7fc400640 1 -- 172.21.15.161:0/1589442060 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f41525c0 con 0x7fa7f41444a0 2024-05-22T21:15:39.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7f1600640 1 -- 172.21.15.161:0/1589442060 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00be900 con 0x7fa7f41444a0 2024-05-22T21:15:39.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7fc400640 1 -- 172.21.15.161:0/1589442060 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f4152b80 con 0x7fa7f41444a0 2024-05-22T21:15:39.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7fc400640 1 -- 172.21.15.161:0/1589442060 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f4067220 con 0x7fa7f41444a0 2024-05-22T21:15:39.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7f1600640 1 -- 172.21.15.161:0/1589442060 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa7e00b43f0 con 0x7fa7f41444a0 2024-05-22T21:15:39.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.742+0000 7fa7f1600640 1 --2- 172.21.15.161:0/1589442060 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7d40666c0 0x7fa7d4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:39.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.746+0000 7fa7f1600640 1 -- 172.21.15.161:0/1589442060 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa7e011e4f0 con 0x7fa7f41444a0 2024-05-22T21:15:39.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.746+0000 7fa7f9a00640 1 --2- 172.21.15.161:0/1589442060 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7d40666c0 0x7fa7d4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:39.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.746+0000 7fa7f9a00640 1 --2- 172.21.15.161:0/1589442060 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7d40666c0 0x7fa7d4068b80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4140040 tx=0x7fa7e8009830 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:39.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.746+0000 7fa7f1600640 1 -- 172.21.15.161:0/1589442060 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa7e00ebd70 con 0x7fa7f41444a0 2024-05-22T21:15:39.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.906+0000 7fa7fc400640 1 -- 172.21.15.161:0/1589442060 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa7f40ceaa0 con 0x7fa7f41444a0 2024-05-22T21:15:39.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.906+0000 7fa7f1600640 1 -- 172.21.15.161:0/1589442060 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7e00ef390 con 0x7fa7f41444a0 2024-05-22T21:15:39.911 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:15:39.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.906+0000 7fa7fc400640 1 -- 172.21.15.161:0/1589442060 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7d40666c0 msgr2=0x7fa7d4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.906+0000 7fa7fc400640 1 --2- 172.21.15.161:0/1589442060 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7d40666c0 0x7fa7d4068b80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4140040 tx=0x7fa7e8009830 comp rx=0 tx=0).stop 2024-05-22T21:15:39.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.906+0000 7fa7fc400640 1 -- 172.21.15.161:0/1589442060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f41444a0 msgr2=0x7fa7f4155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:39.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.906+0000 7fa7fc400640 1 --2- 172.21.15.161:0/1589442060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f41444a0 0x7fa7f4155450 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa7e001abd0 tx=0x7fa7e001ad60 comp rx=0 tx=0).stop 2024-05-22T21:15:39.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.906+0000 7fa7fc400640 1 -- 172.21.15.161:0/1589442060 shutdown_connections 2024-05-22T21:15:39.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.906+0000 7fa7fc400640 1 --2- 172.21.15.161:0/1589442060 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7d40666c0 0x7fa7d4068b80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.906+0000 7fa7fc400640 1 --2- 172.21.15.161:0/1589442060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7f4144900 0x7fa7f413ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.906+0000 7fa7fc400640 1 --2- 172.21.15.161:0/1589442060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f41444a0 0x7fa7f4155450 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.906+0000 7fa7fc400640 1 --2- 172.21.15.161:0/1589442060 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f4143c40 0x7fa7f4135fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:39.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.906+0000 7fa7fc400640 1 -- 172.21.15.161:0/1589442060 >> 172.21.15.161:0/1589442060 conn(0x7fa7f406a2d0 msgr2=0x7fa7f415ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:39.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.906+0000 7fa7fc400640 1 -- 172.21.15.161:0/1589442060 shutdown_connections 2024-05-22T21:15:39.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:39.906+0000 7fa7fc400640 1 -- 172.21.15.161:0/1589442060 wait complete. 2024-05-22T21:15:39.932 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-22T21:15:40.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f2721c00640 1 Processor -- start 2024-05-22T21:15:40.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f2721c00640 1 -- start start 2024-05-22T21:15:40.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f2721c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f271c15af00 0x7f271c15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:40.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f2721c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f271c15f040 0x7f271c0d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:40.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f2721c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f271c0d0f80 0x7f271c0cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:40.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f2721c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271c0cb8a0 con 0x7f271c15f040 2024-05-22T21:15:40.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f2721c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271c15f530 con 0x7f271c15af00 2024-05-22T21:15:40.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f2721c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c15f810 con 0x7f271c0d0f80 2024-05-22T21:15:40.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f271b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f271c15f040 0x7f271c0d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:40.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f271b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f271c15f040 0x7f271c0d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50130/0 (socket says 172.21.15.161:50130) 2024-05-22T21:15:40.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f271b400640 1 -- 172.21.15.161:0/3680227655 learned_addr learned my addr 172.21.15.161:0/3680227655 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:40.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f271be00640 1 --2- 172.21.15.161:0/3680227655 >> v2:172.21.15.179:3300/0 conn(0x7f271c15af00 0x7f271c15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:40.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f2721200640 1 --2- 172.21.15.161:0/3680227655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f271c0d0f80 0x7f271c0cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:40.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f271b400640 1 -- 172.21.15.161:0/3680227655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f271c0d0f80 msgr2=0x7f271c0cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:40.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f271b400640 1 --2- 172.21.15.161:0/3680227655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f271c0d0f80 0x7f271c0cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f271b400640 1 -- 172.21.15.161:0/3680227655 >> v2:172.21.15.179:3300/0 conn(0x7f271c15af00 msgr2=0x7f271c15eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:40.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f271b400640 1 --2- 172.21.15.161:0/3680227655 >> v2:172.21.15.179:3300/0 conn(0x7f271c15af00 0x7f271c15eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.062+0000 7f271b400640 1 -- 172.21.15.161:0/3680227655 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c0cbb20 con 0x7f271c15f040 2024-05-22T21:15:40.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.066+0000 7f271b400640 1 --2- 172.21.15.161:0/3680227655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271c15f040 0x7f271c0d0a40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f271c09a880 tx=0x7f270c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108cc0a9cf4701c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:40.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.066+0000 7f271aa00640 1 -- 172.21.15.161:0/3680227655 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c0ab600 con 0x7f271c15f040 2024-05-22T21:15:40.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.066+0000 7f271aa00640 1 -- 172.21.15.161:0/3680227655 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270c0a7050 con 0x7f271c15f040 2024-05-22T21:15:40.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.066+0000 7f271aa00640 1 -- 172.21.15.161:0/3680227655 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c0ab980 con 0x7f271c15f040 2024-05-22T21:15:40.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.066+0000 7f2721c00640 1 -- 172.21.15.161:0/3680227655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271c15f040 msgr2=0x7f271c0d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:40.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.066+0000 7f2721c00640 1 --2- 172.21.15.161:0/3680227655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271c15f040 0x7f271c0d0a40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f271c09a880 tx=0x7f270c01ccc0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.066+0000 7f2721c00640 1 -- 172.21.15.161:0/3680227655 shutdown_connections 2024-05-22T21:15:40.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.066+0000 7f2721c00640 1 --2- 172.21.15.161:0/3680227655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f271c0d0f80 0x7f271c0cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.066+0000 7f2721c00640 1 --2- 172.21.15.161:0/3680227655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271c15f040 0x7f271c0d0a40 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.066+0000 7f2721c00640 1 --2- 172.21.15.161:0/3680227655 >> v2:172.21.15.179:3300/0 conn(0x7f271c15af00 0x7f271c15eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.066+0000 7f2721c00640 1 -- 172.21.15.161:0/3680227655 >> 172.21.15.161:0/3680227655 conn(0x7f271c099a40 msgr2=0x7f271c099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:40.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.066+0000 7f2721c00640 1 -- 172.21.15.161:0/3680227655 shutdown_connections 2024-05-22T21:15:40.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.066+0000 7f2721c00640 1 -- 172.21.15.161:0/3680227655 wait complete. 2024-05-22T21:15:40.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f2721c00640 1 Processor -- start 2024-05-22T21:15:40.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f2721c00640 1 -- start start 2024-05-22T21:15:40.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f2721c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f271c0d0f80 0x7f271c151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:40.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f2721c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271c15af00 0x7f271c06dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:40.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f2721c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f271c15f040 0x7f271c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:40.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f2721c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0c8d90 con 0x7f271c15af00 2024-05-22T21:15:40.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f2721c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0c9070 con 0x7f271c0d0f80 2024-05-22T21:15:40.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f2721c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0c9350 con 0x7f271c15f040 2024-05-22T21:15:40.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f271b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271c15af00 0x7f271c06dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:40.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f271b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271c15af00 0x7f271c06dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50136/0 (socket says 172.21.15.161:50136) 2024-05-22T21:15:40.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f271b400640 1 -- 172.21.15.161:0/1445829346 learned_addr learned my addr 172.21.15.161:0/1445829346 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:40.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f2721200640 1 --2- 172.21.15.161:0/1445829346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f271c15f040 0x7f271c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:40.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f271be00640 1 --2- 172.21.15.161:0/1445829346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f271c0d0f80 0x7f271c151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:40.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f271b400640 1 -- 172.21.15.161:0/1445829346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f271c15f040 msgr2=0x7f271c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:40.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f271b400640 1 --2- 172.21.15.161:0/1445829346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f271c15f040 0x7f271c06e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f271b400640 1 -- 172.21.15.161:0/1445829346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f271c0d0f80 msgr2=0x7f271c151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:40.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f271b400640 1 --2- 172.21.15.161:0/1445829346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f271c0d0f80 0x7f271c151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f271b400640 1 -- 172.21.15.161:0/1445829346 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c0094d0 con 0x7f271c15af00 2024-05-22T21:15:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f271b400640 1 --2- 172.21.15.161:0/1445829346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271c15af00 0x7f271c06dd50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f270c01a910 tx=0x7f270c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f2713e00640 1 -- 172.21.15.161:0/1445829346 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c0b4420 con 0x7f271c15af00 2024-05-22T21:15:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f2713e00640 1 -- 172.21.15.161:0/1445829346 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270c0b4580 con 0x7f271c15af00 2024-05-22T21:15:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f2721c00640 1 -- 172.21.15.161:0/1445829346 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271c0c9630 con 0x7f271c15af00 2024-05-22T21:15:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f2713e00640 1 -- 172.21.15.161:0/1445829346 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c0b4930 con 0x7f271c15af00 2024-05-22T21:15:40.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f2721c00640 1 -- 172.21.15.161:0/1445829346 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271c151740 con 0x7f271c15af00 2024-05-22T21:15:40.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.070+0000 7f2721c00640 1 -- 172.21.15.161:0/1445829346 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271c0cf180 con 0x7f271c15af00 2024-05-22T21:15:40.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.074+0000 7f2713e00640 1 -- 172.21.15.161:0/1445829346 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f270c0b2070 con 0x7f271c15af00 2024-05-22T21:15:40.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.074+0000 7f2713e00640 1 --2- 172.21.15.161:0/1445829346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f26f8066690 0x7f26f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:40.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.074+0000 7f2713e00640 1 -- 172.21.15.161:0/1445829346 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f270c11cf70 con 0x7f271c15af00 2024-05-22T21:15:40.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.074+0000 7f271be00640 1 --2- 172.21.15.161:0/1445829346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f26f8066690 0x7f26f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:40.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.074+0000 7f271be00640 1 --2- 172.21.15.161:0/1445829346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f26f8066690 0x7f26f8068b50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f271c06e8f0 tx=0x7f2704008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:40.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.078+0000 7f2713e00640 1 -- 172.21.15.161:0/1445829346 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f270c0ea740 con 0x7f271c15af00 2024-05-22T21:15:40.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.234+0000 7f2721c00640 1 -- 172.21.15.161:0/1445829346 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f271c067070 con 0x7f271c15af00 2024-05-22T21:15:40.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.234+0000 7f2713e00640 1 -- 172.21.15.161:0/1445829346 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v63) v1 ==== 84+0+65 (secure 0 0 0) 0x7f270c0b0060 con 0x7f271c15af00 2024-05-22T21:15:40.239 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:40.239 INFO:teuthology.orchestra.run.smithi161.stdout:[ 2024-05-22T21:15:40.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:15:40.239 INFO:teuthology.orchestra.run.smithi161.stdout: "poolnum": 1, 2024-05-22T21:15:40.239 INFO:teuthology.orchestra.run.smithi161.stdout: "poolname": ".mgr" 2024-05-22T21:15:40.239 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:15:40.239 INFO:teuthology.orchestra.run.smithi161.stdout:] 2024-05-22T21:15:40.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.234+0000 7f2721c00640 1 -- 172.21.15.161:0/1445829346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f26f8066690 msgr2=0x7f26f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:40.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.234+0000 7f2721c00640 1 --2- 172.21.15.161:0/1445829346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f26f8066690 0x7f26f8068b50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f271c06e8f0 tx=0x7f2704008040 comp rx=0 tx=0).stop 2024-05-22T21:15:40.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.234+0000 7f2721c00640 1 -- 172.21.15.161:0/1445829346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271c15af00 msgr2=0x7f271c06dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:40.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.234+0000 7f2721c00640 1 --2- 172.21.15.161:0/1445829346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271c15af00 0x7f271c06dd50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f270c01a910 tx=0x7f270c019000 comp rx=0 tx=0).stop 2024-05-22T21:15:40.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.234+0000 7f2721c00640 1 -- 172.21.15.161:0/1445829346 shutdown_connections 2024-05-22T21:15:40.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.234+0000 7f2721c00640 1 --2- 172.21.15.161:0/1445829346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f26f8066690 0x7f26f8068b50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.234+0000 7f2721c00640 1 --2- 172.21.15.161:0/1445829346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f271c15f040 0x7f271c06e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.234+0000 7f2721c00640 1 --2- 172.21.15.161:0/1445829346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f271c15af00 0x7f271c06dd50 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.234+0000 7f2721c00640 1 --2- 172.21.15.161:0/1445829346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f271c0d0f80 0x7f271c151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.234+0000 7f2721c00640 1 -- 172.21.15.161:0/1445829346 >> 172.21.15.161:0/1445829346 conn(0x7f271c099a40 msgr2=0x7f271c0cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:40.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.234+0000 7f2721c00640 1 -- 172.21.15.161:0/1445829346 shutdown_connections 2024-05-22T21:15:40.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.234+0000 7f2721c00640 1 -- 172.21.15.161:0/1445829346 wait complete. 2024-05-22T21:15:40.259 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-22T21:15:40.259 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-22T21:15:40.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7ebb200640 1 Processor -- start 2024-05-22T21:15:40.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7ebb200640 1 -- start start 2024-05-22T21:15:40.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7ebb200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7ebc15e700 0x7f7ebc0cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:40.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7ebb200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ebc0b33b0 0x7f7ebc0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:40.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7ebb200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ebc0b0bb0 con 0x7f7ebc0b1040 2024-05-22T21:15:40.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7ebb200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ebc15eae0 con 0x7f7ebc15e700 2024-05-22T21:15:40.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7ebb200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0cf6a0 con 0x7f7ebc0b33b0 2024-05-22T21:15:40.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eba800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f7ebc0b1040 0x7f7ebc0b14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:49538/0 (socket says 172.21.15.161:49538) 2024-05-22T21:15:40.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eba800640 1 -- 172.21.15.161:0/853383823 learned_addr learned my addr 172.21.15.161:0/853383823 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:40.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9e00640 1 --2- 172.21.15.161:0/853383823 >> v2:172.21.15.179:3300/0 conn(0x7f7ebc15e700 0x7f7ebc0cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:40.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb1400640 1 --2- 172.21.15.161:0/853383823 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ebc0b33b0 0x7f7ebc0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:40.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9400640 1 -- 172.21.15.161:0/853383823 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406583442 0 0) 0x7f7ebc0b0bb0 con 0x7f7ebc0b1040 2024-05-22T21:15:40.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9400640 1 -- 172.21.15.161:0/853383823 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ea0003530 con 0x7f7ebc0b1040 2024-05-22T21:15:40.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9400640 1 -- 172.21.15.161:0/853383823 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2515817853 0 0) 0x7f7ea0003530 con 0x7f7ebc0b1040 2024-05-22T21:15:40.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9400640 1 -- 172.21.15.161:0/853383823 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ebc0b33b0 msgr2=0x7f7ebc0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:40.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9400640 1 --2- 172.21.15.161:0/853383823 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ebc0b33b0 0x7f7ebc0cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9400640 1 -- 172.21.15.161:0/853383823 >> v2:172.21.15.179:3300/0 conn(0x7f7ebc15e700 msgr2=0x7f7ebc0cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:40.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9400640 1 --2- 172.21.15.161:0/853383823 >> v2:172.21.15.179:3300/0 conn(0x7f7ebc15e700 0x7f7ebc0cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9400640 1 -- 172.21.15.161:0/853383823 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc0cf980 con 0x7f7ebc0b1040 2024-05-22T21:15:40.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9400640 1 -- 172.21.15.161:0/853383823 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f7eb4001680 con 0x7f7ebc0b1040 2024-05-22T21:15:40.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9e00640 1 --2- 172.21.15.161:0/853383823 >> v2:172.21.15.179:3300/0 conn(0x7f7ebc15e700 0x7f7ebc0cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:15:40.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9400640 1 -- 172.21.15.161:0/853383823 >> v1:172.21.15.161:6789/0 conn(0x7f7ebc0b1040 legacy=0x7f7ebc0b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:40.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9400640 1 --2- 172.21.15.161:0/853383823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea0003980 0x7f7ea0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:40.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9400640 1 --2- 172.21.15.161:0/853383823 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ebc0b33b0 0x7f7ea00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9400640 1 --2- 172.21.15.161:0/853383823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ea0004d10 0x7f7ea0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9400640 1 -- 172.21.15.161:0/853383823 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc15eae0 con 0x7f7ea0003980 2024-05-22T21:15:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9400640 1 -- 172.21.15.161:0/853383823 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0cf6a0 con 0x7f7ea0004d10 2024-05-22T21:15:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9400640 1 -- 172.21.15.161:0/853383823 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea000b990 con 0x7f7ebc0b33b0 2024-05-22T21:15:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9e00640 1 --2- 172.21.15.161:0/853383823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea0003980 0x7f7ea0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb1400640 1 --2- 172.21.15.161:0/853383823 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ebc0b33b0 0x7f7ea00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eba800640 1 --2- 172.21.15.161:0/853383823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ea0004d10 0x7f7ea0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9e00640 1 -- 172.21.15.161:0/853383823 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ebc0b33b0 msgr2=0x7f7ea00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9e00640 1 --2- 172.21.15.161:0/853383823 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ebc0b33b0 0x7f7ea00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9e00640 1 -- 172.21.15.161:0/853383823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ea0004d10 msgr2=0x7f7ea0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9e00640 1 --2- 172.21.15.161:0/853383823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ea0004d10 0x7f7ea0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eb9e00640 1 -- 172.21.15.161:0/853383823 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea0003530 con 0x7f7ea0003980 2024-05-22T21:15:40.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.394+0000 7f7eba800640 1 --2- 172.21.15.161:0/853383823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ea0004d10 0x7f7ea0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.398+0000 7f7eb9e00640 1 --2- 172.21.15.161:0/853383823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea0003980 0x7f7ea0003da0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00e380 tx=0x7f7eac01ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471eaea9c240890b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.398+0000 7f7eb9400640 1 -- 172.21.15.161:0/853383823 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0a5c50 con 0x7f7ea0003980 2024-05-22T21:15:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.398+0000 7f7eb9400640 1 -- 172.21.15.161:0/853383823 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eac00f040 con 0x7f7ea0003980 2024-05-22T21:15:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.398+0000 7f7eb9400640 1 -- 172.21.15.161:0/853383823 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0ad440 con 0x7f7ea0003980 2024-05-22T21:15:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.398+0000 7f7ebb200640 1 -- 172.21.15.161:0/853383823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea0003980 msgr2=0x7f7ea0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:40.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.398+0000 7f7ebb200640 1 --2- 172.21.15.161:0/853383823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea0003980 0x7f7ea0003da0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00e380 tx=0x7f7eac01ce60 comp rx=0 tx=0).stop 2024-05-22T21:15:40.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.398+0000 7f7eba800640 1 -- 172.21.15.161:0/853383823 reap_dead start 2024-05-22T21:15:40.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.398+0000 7f7ebb200640 1 -- 172.21.15.161:0/853383823 shutdown_connections 2024-05-22T21:15:40.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.398+0000 7f7ebb200640 1 -- 172.21.15.161:0/853383823 >> 172.21.15.161:0/853383823 conn(0x7f7ebc0b28c0 msgr2=0x7f7ebc0b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:40.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.398+0000 7f7ebb200640 1 -- 172.21.15.161:0/853383823 shutdown_connections 2024-05-22T21:15:40.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.398+0000 7f7ebb200640 1 -- 172.21.15.161:0/853383823 wait complete. 2024-05-22T21:15:40.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7ebb200640 1 Processor -- start 2024-05-22T21:15:40.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7ebb200640 1 -- start start 2024-05-22T21:15:40.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7ebb200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea0003980 0x7f7ebc0af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:40.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7ebb200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ebc0adba0 0x7f7ebc0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:40.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7ebb200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ebc0b6670 0x7f7ebc0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:40.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7ebb200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc148cd0 con 0x7f7ea0003980 2024-05-22T21:15:40.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7ebb200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0ae360 con 0x7f7ebc0adba0 2024-05-22T21:15:40.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7ebb200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0ae640 con 0x7f7ebc0b6670 2024-05-22T21:15:40.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7eb9e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea0003980 0x7f7ebc0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:40.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7eb9e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea0003980 0x7f7ebc0af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50156/0 (socket says 172.21.15.161:50156) 2024-05-22T21:15:40.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7eb9e00640 1 -- 172.21.15.161:0/525504187 learned_addr learned my addr 172.21.15.161:0/525504187 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:40.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7eb1400640 1 --2- 172.21.15.161:0/525504187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ebc0adba0 0x7f7ebc0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:40.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7eba800640 1 --2- 172.21.15.161:0/525504187 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ebc0b6670 0x7f7ebc0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:40.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7eb9e00640 1 -- 172.21.15.161:0/525504187 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ebc0b6670 msgr2=0x7f7ebc0b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:40.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7eb9e00640 1 --2- 172.21.15.161:0/525504187 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ebc0b6670 0x7f7ebc0b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7eb9e00640 1 -- 172.21.15.161:0/525504187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ebc0adba0 msgr2=0x7f7ebc0b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:40.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7eb9e00640 1 --2- 172.21.15.161:0/525504187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ebc0adba0 0x7f7ebc0b6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:40.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7eb9e00640 1 -- 172.21.15.161:0/525504187 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc0ae920 con 0x7f7ea0003980 2024-05-22T21:15:40.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7eba800640 1 --2- 172.21.15.161:0/525504187 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ebc0b6670 0x7f7ebc0b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:40.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7eb1400640 1 --2- 172.21.15.161:0/525504187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ebc0adba0 0x7f7ebc0b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:40.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7eb9e00640 1 --2- 172.21.15.161:0/525504187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea0003980 0x7f7ebc0af400 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7eac01cc90 tx=0x7f7eac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:40.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7eb2a00640 1 -- 172.21.15.161:0/525504187 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0ad800 con 0x7f7ea0003980 2024-05-22T21:15:40.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7eb2a00640 1 -- 172.21.15.161:0/525504187 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eac0ad980 con 0x7f7ea0003980 2024-05-22T21:15:40.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7ebb200640 1 -- 172.21.15.161:0/525504187 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ebc0aec00 con 0x7f7ea0003980 2024-05-22T21:15:40.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7eb2a00640 1 -- 172.21.15.161:0/525504187 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0add30 con 0x7f7ea0003980 2024-05-22T21:15:40.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7ebb200640 1 -- 172.21.15.161:0/525504187 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ebc0ae020 con 0x7f7ea0003980 2024-05-22T21:15:40.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.402+0000 7f7ebb200640 1 -- 172.21.15.161:0/525504187 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ebc067110 con 0x7f7ea0003980 2024-05-22T21:15:40.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.406+0000 7f7eb2a00640 1 -- 172.21.15.161:0/525504187 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7eac0b4040 con 0x7f7ea0003980 2024-05-22T21:15:40.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.406+0000 7f7eb2a00640 1 --2- 172.21.15.161:0/525504187 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7e980666c0 0x7f7e98068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:40.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.406+0000 7f7eb2a00640 1 -- 172.21.15.161:0/525504187 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7eac11df80 con 0x7f7ea0003980 2024-05-22T21:15:40.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.406+0000 7f7eb1400640 1 --2- 172.21.15.161:0/525504187 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7e980666c0 0x7f7e98068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:40.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.406+0000 7f7eb1400640 1 --2- 172.21.15.161:0/525504187 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7e980666c0 0x7f7e98068b80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc09eef0 tx=0x7f7ea4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:40.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.410+0000 7f7eb2a00640 1 -- 172.21.15.161:0/525504187 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7eac0eb5a0 con 0x7f7ea0003980 2024-05-22T21:15:40.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:40.566+0000 7f7ebb200640 1 -- 172.21.15.161:0/525504187 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7ebc0cd280 con 0x7f7ea0003980 2024-05-22T21:15:41.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.557+0000 7f7eb2a00640 1 -- 172.21.15.161:0/525504187 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v64) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7eac0b20b0 con 0x7f7ea0003980 2024-05-22T21:15:41.565 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' created 2024-05-22T21:15:41.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.561+0000 7f7ebb200640 1 -- 172.21.15.161:0/525504187 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7e980666c0 msgr2=0x7f7e98068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:41.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.561+0000 7f7ebb200640 1 --2- 172.21.15.161:0/525504187 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7e980666c0 0x7f7e98068b80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc09eef0 tx=0x7f7ea4008040 comp rx=0 tx=0).stop 2024-05-22T21:15:41.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.561+0000 7f7ebb200640 1 -- 172.21.15.161:0/525504187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea0003980 msgr2=0x7f7ebc0af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:41.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.561+0000 7f7ebb200640 1 --2- 172.21.15.161:0/525504187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea0003980 0x7f7ebc0af400 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7eac01cc90 tx=0x7f7eac009750 comp rx=0 tx=0).stop 2024-05-22T21:15:41.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.561+0000 7f7ebb200640 1 -- 172.21.15.161:0/525504187 shutdown_connections 2024-05-22T21:15:41.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.561+0000 7f7ebb200640 1 --2- 172.21.15.161:0/525504187 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7e980666c0 0x7f7e98068b80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:41.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.561+0000 7f7ebb200640 1 --2- 172.21.15.161:0/525504187 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ebc0b6670 0x7f7ebc0b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:41.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.561+0000 7f7ebb200640 1 --2- 172.21.15.161:0/525504187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ebc0adba0 0x7f7ebc0b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:41.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.561+0000 7f7ebb200640 1 --2- 172.21.15.161:0/525504187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea0003980 0x7f7ebc0af400 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:41.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.561+0000 7f7ebb200640 1 -- 172.21.15.161:0/525504187 >> 172.21.15.161:0/525504187 conn(0x7f7ebc0b28c0 msgr2=0x7f7ebc06ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:41.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.561+0000 7f7ebb200640 1 -- 172.21.15.161:0/525504187 shutdown_connections 2024-05-22T21:15:41.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.561+0000 7f7ebb200640 1 -- 172.21.15.161:0/525504187 wait complete. 2024-05-22T21:15:41.593 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-22T21:15:41.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f0ae00640 1 Processor -- start 2024-05-22T21:15:41.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f0ae00640 1 -- start start 2024-05-22T21:15:41.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f0ae00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f04160840 0x7f6f040cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:41.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f0ae00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6f040b1ff0 0x7f6f04099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:41.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f0ae00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f040ccca0 con 0x7f6f04160c20 2024-05-22T21:15:41.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f0ae00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f040ceb60 con 0x7f6f040b1ff0 2024-05-22T21:15:41.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f0ae00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f040cced0 con 0x7f6f04160840 2024-05-22T21:15:41.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f08e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6f040b1ff0 0x7f6f04099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:41.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f03e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f04160840 0x7f6f040cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:41.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f08e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6f040b1ff0 0x7f6f04099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:47342/0 (socket says 172.21.15.161:47342) 2024-05-22T21:15:41.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6efb400640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f6f04160c20 0x7f6f040cf870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:48942/0 (socket says 172.21.15.161:48942) 2024-05-22T21:15:41.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f08e00640 1 -- 172.21.15.161:0/2804529490 learned_addr learned my addr 172.21.15.161:0/2804529490 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:41.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f08e00640 1 -- 172.21.15.161:0/2804529490 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f04160840 msgr2=0x7f6f040cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:41.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f08e00640 1 --2- 172.21.15.161:0/2804529490 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f04160840 0x7f6f040cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:41.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f08e00640 1 -- 172.21.15.161:0/2804529490 >> v1:172.21.15.179:6789/0 conn(0x7f6f04160c20 legacy=0x7f6f040cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:41.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f08e00640 1 -- 172.21.15.161:0/2804529490 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f040cedb0 con 0x7f6f040b1ff0 2024-05-22T21:15:41.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f03400640 1 -- 172.21.15.161:0/2804529490 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3959768753 0 0) 0x7f6f040ccca0 con 0x7f6f04160c20 2024-05-22T21:15:41.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.733+0000 7f6f03e00640 1 --2- 172.21.15.161:0/2804529490 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f04160840 0x7f6f040cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:15:41.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.737+0000 7f6f08e00640 1 --2- 172.21.15.161:0/2804529490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f040b1ff0 0x7f6f04099a40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009e50 tx=0x7f6efc01c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94529b31254f1e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:41.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.737+0000 7f6f03400640 1 -- 172.21.15.161:0/2804529490 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc0a5770 con 0x7f6f040b1ff0 2024-05-22T21:15:41.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.737+0000 7f6f03400640 1 -- 172.21.15.161:0/2804529490 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6efc0a58d0 con 0x7f6f040b1ff0 2024-05-22T21:15:41.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.737+0000 7f6f03400640 1 -- 172.21.15.161:0/2804529490 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc0a5c60 con 0x7f6f040b1ff0 2024-05-22T21:15:41.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.737+0000 7f6f0ae00640 1 -- 172.21.15.161:0/2804529490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f040b1ff0 msgr2=0x7f6f04099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:41.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.737+0000 7f6f0ae00640 1 --2- 172.21.15.161:0/2804529490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f040b1ff0 0x7f6f04099a40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009e50 tx=0x7f6efc01c9a0 comp rx=0 tx=0).stop 2024-05-22T21:15:41.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.737+0000 7f6f0ae00640 1 -- 172.21.15.161:0/2804529490 shutdown_connections 2024-05-22T21:15:41.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.737+0000 7f6f0ae00640 1 --2- 172.21.15.161:0/2804529490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f040b1ff0 0x7f6f04099a40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:41.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.737+0000 7f6f0ae00640 1 --2- 172.21.15.161:0/2804529490 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f04160840 0x7f6f040cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:41.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.737+0000 7f6f0ae00640 1 -- 172.21.15.161:0/2804529490 >> 172.21.15.161:0/2804529490 conn(0x7f6f040b2b20 msgr2=0x7f6f040b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:41.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.737+0000 7f6f0ae00640 1 -- 172.21.15.161:0/2804529490 shutdown_connections 2024-05-22T21:15:41.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.737+0000 7f6f0ae00640 1 -- 172.21.15.161:0/2804529490 wait complete. 2024-05-22T21:15:41.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6f0ae00640 1 Processor -- start 2024-05-22T21:15:41.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6f0ae00640 1 -- start start 2024-05-22T21:15:41.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6f0ae00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f040b1ff0 0x7f6f04153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:41.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6f0ae00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f04160840 0x7f6f040b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:41.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6f0ae00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f04160c20 0x7f6f0415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:41.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6f0ae00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f04155620 con 0x7f6f04160840 2024-05-22T21:15:41.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6f0ae00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f040c8b50 con 0x7f6f040b1ff0 2024-05-22T21:15:41.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6f0ae00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f04154d80 con 0x7f6f04160c20 2024-05-22T21:15:41.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6efb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f04160840 0x7f6f040b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:41.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6efb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f04160840 0x7f6f040b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50166/0 (socket says 172.21.15.161:50166) 2024-05-22T21:15:41.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6efb400640 1 -- 172.21.15.161:0/1304993520 learned_addr learned my addr 172.21.15.161:0/1304993520 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:41.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6f03e00640 1 --2- 172.21.15.161:0/1304993520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f040b1ff0 0x7f6f04153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:41.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6f08e00640 1 --2- 172.21.15.161:0/1304993520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f04160c20 0x7f6f0415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:41.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6efb400640 1 -- 172.21.15.161:0/1304993520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f04160c20 msgr2=0x7f6f0415aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:41.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6efb400640 1 --2- 172.21.15.161:0/1304993520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f04160c20 0x7f6f0415aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:41.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6efb400640 1 -- 172.21.15.161:0/1304993520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f040b1ff0 msgr2=0x7f6f04153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:41.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6efb400640 1 --2- 172.21.15.161:0/1304993520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f040b1ff0 0x7f6f04153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:41.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6efb400640 1 -- 172.21.15.161:0/1304993520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef4009420 con 0x7f6f04160840 2024-05-22T21:15:41.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6efb400640 1 --2- 172.21.15.161:0/1304993520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f04160840 0x7f6f040b5e30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00c4b0 tx=0x7f6eec00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:41.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6f00c00640 1 -- 172.21.15.161:0/1304993520 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec0074e0 con 0x7f6f04160840 2024-05-22T21:15:41.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6f00c00640 1 -- 172.21.15.161:0/1304993520 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eec007660 con 0x7f6f04160840 2024-05-22T21:15:41.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6f0ae00640 1 -- 172.21.15.161:0/1304993520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6efc009b30 con 0x7f6f04160840 2024-05-22T21:15:41.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6f00c00640 1 -- 172.21.15.161:0/1304993520 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec004a30 con 0x7f6f04160840 2024-05-22T21:15:41.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.741+0000 7f6f0ae00640 1 -- 172.21.15.161:0/1304993520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f04155340 con 0x7f6f04160840 2024-05-22T21:15:41.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.745+0000 7f6f0ae00640 1 -- 172.21.15.161:0/1304993520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f040b6fa0 con 0x7f6f04160840 2024-05-22T21:15:41.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.745+0000 7f6f00c00640 1 -- 172.21.15.161:0/1304993520 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6eec004b90 con 0x7f6f04160840 2024-05-22T21:15:41.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.745+0000 7f6f00c00640 1 --2- 172.21.15.161:0/1304993520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ee0066690 0x7f6ee0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:41.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.745+0000 7f6f00c00640 1 -- 172.21.15.161:0/1304993520 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6eec0830e0 con 0x7f6f04160840 2024-05-22T21:15:41.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.745+0000 7f6f03e00640 1 --2- 172.21.15.161:0/1304993520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ee0066690 0x7f6ee0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:41.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.745+0000 7f6f03e00640 1 --2- 172.21.15.161:0/1304993520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ee0066690 0x7f6ee0068b50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009c50 tx=0x7f6ef4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:41.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.749+0000 7f6f00c00640 1 -- 172.21.15.161:0/1304993520 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6eec050830 con 0x7f6f04160840 2024-05-22T21:15:41.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:41.905+0000 7f6f0ae00640 1 -- 172.21.15.161:0/1304993520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6f04067220 con 0x7f6f04160840 2024-05-22T21:15:42.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.569+0000 7f6f00c00640 1 -- 172.21.15.161:0/1304993520 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v65) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6eec0180d0 con 0x7f6f04160840 2024-05-22T21:15:42.579 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' created 2024-05-22T21:15:42.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.581+0000 7f6f0ae00640 1 -- 172.21.15.161:0/1304993520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ee0066690 msgr2=0x7f6ee0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:42.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.581+0000 7f6f0ae00640 1 --2- 172.21.15.161:0/1304993520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ee0066690 0x7f6ee0068b50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009c50 tx=0x7f6ef4009190 comp rx=0 tx=0).stop 2024-05-22T21:15:42.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.581+0000 7f6f0ae00640 1 -- 172.21.15.161:0/1304993520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f04160840 msgr2=0x7f6f040b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:42.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.581+0000 7f6f0ae00640 1 --2- 172.21.15.161:0/1304993520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f04160840 0x7f6f040b5e30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00c4b0 tx=0x7f6eec00c970 comp rx=0 tx=0).stop 2024-05-22T21:15:42.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.581+0000 7f6f0ae00640 1 -- 172.21.15.161:0/1304993520 shutdown_connections 2024-05-22T21:15:42.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.581+0000 7f6f0ae00640 1 --2- 172.21.15.161:0/1304993520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ee0066690 0x7f6ee0068b50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:42.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.581+0000 7f6f0ae00640 1 --2- 172.21.15.161:0/1304993520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f04160c20 0x7f6f0415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:42.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.581+0000 7f6f0ae00640 1 --2- 172.21.15.161:0/1304993520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f04160840 0x7f6f040b5e30 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:42.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.581+0000 7f6f0ae00640 1 --2- 172.21.15.161:0/1304993520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f040b1ff0 0x7f6f04153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:42.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.581+0000 7f6f0ae00640 1 -- 172.21.15.161:0/1304993520 >> 172.21.15.161:0/1304993520 conn(0x7f6f040b2b20 msgr2=0x7f6f04158300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:42.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.581+0000 7f6f0ae00640 1 -- 172.21.15.161:0/1304993520 shutdown_connections 2024-05-22T21:15:42.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.581+0000 7f6f0ae00640 1 -- 172.21.15.161:0/1304993520 wait complete. 2024-05-22T21:15:42.610 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-22T21:15:42.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.745+0000 7f168c600640 1 Processor -- start 2024-05-22T21:15:42.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.745+0000 7f168c600640 1 -- start start 2024-05-22T21:15:42.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.745+0000 7f168c600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f16841402e0 0x7f168413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:42.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.745+0000 7f168c600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f168413fc80 0x7f168406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:42.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.745+0000 7f168c600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16841400b0 con 0x7f168413cfc0 2024-05-22T21:15:42.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.745+0000 7f168c600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f168413d3a0 con 0x7f16841402e0 2024-05-22T21:15:42.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.745+0000 7f168c600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16840cd920 con 0x7f168413fc80 2024-05-22T21:15:42.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.745+0000 7f1689200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f16841402e0 0x7f168413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:42.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.745+0000 7f1689200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f16841402e0 0x7f168413f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50170/0 (socket says 172.21.15.161:50170) 2024-05-22T21:15:42.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.745+0000 7f1689200640 1 -- 172.21.15.161:0/1544237427 learned_addr learned my addr 172.21.15.161:0/1544237427 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:42.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.745+0000 7f1689200640 1 -- 172.21.15.161:0/1544237427 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f168413fc80 msgr2=0x7f168406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:42.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.745+0000 7f1689200640 1 --2- 172.21.15.161:0/1544237427 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f168413fc80 0x7f168406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:42.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.745+0000 7f1689200640 1 -- 172.21.15.161:0/1544237427 >> v1:172.21.15.179:6789/0 conn(0x7f168413cfc0 legacy=0x7f168415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:42.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.745+0000 7f1689200640 1 -- 172.21.15.161:0/1544237427 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16840cdc00 con 0x7f16841402e0 2024-05-22T21:15:42.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.749+0000 7f1689200640 1 --2- 172.21.15.161:0/1544237427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16841402e0 0x7f168413f740 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f167000bb10 tx=0x7f167001eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd4eeac21b2c5c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:42.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.749+0000 7f1683e00640 1 -- 172.21.15.161:0/1544237427 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16700a77f0 con 0x7f16841402e0 2024-05-22T21:15:42.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.749+0000 7f1683e00640 1 -- 172.21.15.161:0/1544237427 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16700a7950 con 0x7f16841402e0 2024-05-22T21:15:42.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.749+0000 7f1683e00640 1 -- 172.21.15.161:0/1544237427 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16700a7ce0 con 0x7f16841402e0 2024-05-22T21:15:42.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.749+0000 7f168c600640 1 -- 172.21.15.161:0/1544237427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16841402e0 msgr2=0x7f168413f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:42.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.749+0000 7f168c600640 1 --2- 172.21.15.161:0/1544237427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16841402e0 0x7f168413f740 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f167000bb10 tx=0x7f167001eb30 comp rx=0 tx=0).stop 2024-05-22T21:15:42.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.749+0000 7f168c600640 1 -- 172.21.15.161:0/1544237427 shutdown_connections 2024-05-22T21:15:42.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.749+0000 7f168c600640 1 --2- 172.21.15.161:0/1544237427 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f168413fc80 0x7f168406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:42.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.749+0000 7f168c600640 1 --2- 172.21.15.161:0/1544237427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16841402e0 0x7f168413f740 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:42.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.749+0000 7f168c600640 1 -- 172.21.15.161:0/1544237427 >> 172.21.15.161:0/1544237427 conn(0x7f168406a2d0 msgr2=0x7f168406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:42.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.749+0000 7f168c600640 1 -- 172.21.15.161:0/1544237427 shutdown_connections 2024-05-22T21:15:42.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.749+0000 7f168c600640 1 -- 172.21.15.161:0/1544237427 wait complete. 2024-05-22T21:15:42.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f168c600640 1 Processor -- start 2024-05-22T21:15:42.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f168c600640 1 -- start start 2024-05-22T21:15:42.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f168c600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f168413cfc0 0x7f16841376e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:42.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f168c600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f168413fc80 0x7f16840af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:42.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f168c600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16841402e0 0x7f16840afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:42.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f168c600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1684137c20 con 0x7f168413fc80 2024-05-22T21:15:42.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f168c600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168413d4e0 con 0x7f16841402e0 2024-05-22T21:15:42.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f168c600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16840b00b0 con 0x7f168413cfc0 2024-05-22T21:15:42.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f1689200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f168413fc80 0x7f16840af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:42.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f1689200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f168413fc80 0x7f16840af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50180/0 (socket says 172.21.15.161:50180) 2024-05-22T21:15:42.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f1689200640 1 -- 172.21.15.161:0/563296900 learned_addr learned my addr 172.21.15.161:0/563296900 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:42.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f1689c00640 1 --2- 172.21.15.161:0/563296900 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f168413cfc0 0x7f16841376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:42.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f168a600640 1 --2- 172.21.15.161:0/563296900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16841402e0 0x7f16840afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:42.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f1689200640 1 -- 172.21.15.161:0/563296900 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f168413cfc0 msgr2=0x7f16841376e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f1689200640 1 --2- 172.21.15.161:0/563296900 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f168413cfc0 0x7f16841376e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f1689200640 1 -- 172.21.15.161:0/563296900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16841402e0 msgr2=0x7f16840afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f1689200640 1 --2- 172.21.15.161:0/563296900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16841402e0 0x7f16840afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f1689200640 1 -- 172.21.15.161:0/563296900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167000b770 con 0x7f168413fc80 2024-05-22T21:15:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f1689c00640 1 --2- 172.21.15.161:0/563296900 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f168413cfc0 0x7f16841376e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f168a600640 1 --2- 172.21.15.161:0/563296900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16841402e0 0x7f16840afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f1689200640 1 --2- 172.21.15.161:0/563296900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f168413fc80 0x7f16840af540 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f167000b2c0 tx=0x7f167001b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f1681600640 1 -- 172.21.15.161:0/563296900 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16700af850 con 0x7f168413fc80 2024-05-22T21:15:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f1681600640 1 -- 172.21.15.161:0/563296900 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16700af9d0 con 0x7f168413fc80 2024-05-22T21:15:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f1681600640 1 -- 172.21.15.161:0/563296900 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16700afd80 con 0x7f168413fc80 2024-05-22T21:15:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f168c600640 1 -- 172.21.15.161:0/563296900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16840b0390 con 0x7f168413fc80 2024-05-22T21:15:42.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.753+0000 7f168c600640 1 -- 172.21.15.161:0/563296900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16841409f0 con 0x7f168413fc80 2024-05-22T21:15:42.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.757+0000 7f168c600640 1 -- 172.21.15.161:0/563296900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16840b0520 con 0x7f168413fc80 2024-05-22T21:15:42.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.757+0000 7f1681600640 1 -- 172.21.15.161:0/563296900 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f16700b6040 con 0x7f168413fc80 2024-05-22T21:15:42.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.757+0000 7f1681600640 1 --2- 172.21.15.161:0/563296900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1664066690 0x7f1664068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:42.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.757+0000 7f1681600640 1 -- 172.21.15.161:0/563296900 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f16701210d0 con 0x7f168413fc80 2024-05-22T21:15:42.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.757+0000 7f1689c00640 1 --2- 172.21.15.161:0/563296900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1664066690 0x7f1664068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:42.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.757+0000 7f1689c00640 1 --2- 172.21.15.161:0/563296900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1664066690 0x7f1664068b50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1678007fd0 tx=0x7f1678009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:42.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.761+0000 7f1681600640 1 -- 172.21.15.161:0/563296900 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f16700ee480 con 0x7f168413fc80 2024-05-22T21:15:42.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.949+0000 7f168c600640 1 -- 172.21.15.161:0/563296900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1684067220 con 0x7f168413fc80 2024-05-22T21:15:42.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.957+0000 7f1681600640 1 -- 172.21.15.161:0/563296900 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v27) v1 ==== 172+0+0 (secure 0 0 0) 0x7f16700bc0c0 con 0x7f168413fc80 2024-05-22T21:15:42.964 INFO:teuthology.orchestra.run.smithi161.stderr:new fs with metadata pool 8 and data pool 9 2024-05-22T21:15:42.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.961+0000 7f168c600640 1 -- 172.21.15.161:0/563296900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1664066690 msgr2=0x7f1664068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:42.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.961+0000 7f168c600640 1 --2- 172.21.15.161:0/563296900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1664066690 0x7f1664068b50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1678007fd0 tx=0x7f1678009040 comp rx=0 tx=0).stop 2024-05-22T21:15:42.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.961+0000 7f168c600640 1 -- 172.21.15.161:0/563296900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f168413fc80 msgr2=0x7f16840af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:42.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.961+0000 7f168c600640 1 --2- 172.21.15.161:0/563296900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f168413fc80 0x7f16840af540 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f167000b2c0 tx=0x7f167001b8d0 comp rx=0 tx=0).stop 2024-05-22T21:15:42.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.961+0000 7f168c600640 1 -- 172.21.15.161:0/563296900 shutdown_connections 2024-05-22T21:15:42.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.961+0000 7f168c600640 1 --2- 172.21.15.161:0/563296900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1664066690 0x7f1664068b50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:42.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.961+0000 7f168c600640 1 --2- 172.21.15.161:0/563296900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16841402e0 0x7f16840afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:42.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.961+0000 7f168c600640 1 --2- 172.21.15.161:0/563296900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f168413fc80 0x7f16840af540 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:42.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.961+0000 7f168c600640 1 --2- 172.21.15.161:0/563296900 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f168413cfc0 0x7f16841376e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:42.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.961+0000 7f168c600640 1 -- 172.21.15.161:0/563296900 >> 172.21.15.161:0/563296900 conn(0x7f168406a2d0 msgr2=0x7f168415da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:42.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.961+0000 7f168c600640 1 -- 172.21.15.161:0/563296900 shutdown_connections 2024-05-22T21:15:42.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:42.961+0000 7f168c600640 1 -- 172.21.15.161:0/563296900 wait complete. 2024-05-22T21:15:42.997 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:15:43.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.125+0000 7f808c000640 1 Processor -- start 2024-05-22T21:15:43.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.125+0000 7f808c000640 1 -- start start 2024-05-22T21:15:43.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f808c000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f80841597f0 0x7f80840706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f808c000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8084071040 con 0x7f80840b3070 2024-05-22T21:15:43.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f808c000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80840b34e0 con 0x7f8084070be0 2024-05-22T21:15:43.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f808c000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8084159bd0 con 0x7f80841597f0 2024-05-22T21:15:43.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f808a000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f80840b3070 0x7f80840cc550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:49542/0 (socket says 172.21.15.161:49542) 2024-05-22T21:15:43.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f808a000640 1 -- 172.21.15.161:0/2515168654 learned_addr learned my addr 172.21.15.161:0/2515168654 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:43.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8089600640 1 --2- 172.21.15.161:0/2515168654 >> v2:172.21.15.179:3300/0 conn(0x7f80841597f0 0x7f80840706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8083e00640 1 -- 172.21.15.161:0/2515168654 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707981178 0 0) 0x7f8084071040 con 0x7f80840b3070 2024-05-22T21:15:43.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8083e00640 1 -- 172.21.15.161:0/2515168654 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f806c003450 con 0x7f80840b3070 2024-05-22T21:15:43.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8083e00640 1 -- 172.21.15.161:0/2515168654 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3153810596 0 0) 0x7f806c003450 con 0x7f80840b3070 2024-05-22T21:15:43.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8083e00640 1 -- 172.21.15.161:0/2515168654 >> v2:172.21.15.179:3300/0 conn(0x7f80841597f0 msgr2=0x7f80840706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:43.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8083e00640 1 --2- 172.21.15.161:0/2515168654 >> v2:172.21.15.179:3300/0 conn(0x7f80841597f0 0x7f80840706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8083e00640 1 -- 172.21.15.161:0/2515168654 >> v1:172.21.15.179:6789/0 conn(0x7f8084070be0 legacy=0x7f80840b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8083e00640 1 -- 172.21.15.161:0/2515168654 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80840cbdd0 con 0x7f80840b3070 2024-05-22T21:15:43.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8083e00640 1 -- 172.21.15.161:0/2515168654 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f807c001680 con 0x7f80840b3070 2024-05-22T21:15:43.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8083e00640 1 -- 172.21.15.161:0/2515168654 >> v1:172.21.15.161:6789/0 conn(0x7f80840b3070 legacy=0x7f80840cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8083e00640 1 --2- 172.21.15.161:0/2515168654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f806c003a70 0x7f806c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8083e00640 1 --2- 172.21.15.161:0/2515168654 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f806c008860 0x7f806c00ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8083e00640 1 --2- 172.21.15.161:0/2515168654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c00b490 0x7f806c00d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8083e00640 1 -- 172.21.15.161:0/2515168654 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8084159bd0 con 0x7f806c003a70 2024-05-22T21:15:43.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8083e00640 1 -- 172.21.15.161:0/2515168654 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c00e230 con 0x7f806c00b490 2024-05-22T21:15:43.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8083e00640 1 -- 172.21.15.161:0/2515168654 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806c00e3f0 con 0x7f806c008860 2024-05-22T21:15:43.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8089600640 1 --2- 172.21.15.161:0/2515168654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f806c003a70 0x7f806c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8088c00640 1 --2- 172.21.15.161:0/2515168654 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f806c008860 0x7f806c00ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f808a000640 1 --2- 172.21.15.161:0/2515168654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c00b490 0x7f806c00d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8089600640 1 -- 172.21.15.161:0/2515168654 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f806c008860 msgr2=0x7f806c00ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:43.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8089600640 1 --2- 172.21.15.161:0/2515168654 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f806c008860 0x7f806c00ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8089600640 1 -- 172.21.15.161:0/2515168654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c00b490 msgr2=0x7f806c00d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:43.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8089600640 1 --2- 172.21.15.161:0/2515168654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c00b490 0x7f806c00d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8089600640 1 -- 172.21.15.161:0/2515168654 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806c003450 con 0x7f806c003a70 2024-05-22T21:15:43.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f808a000640 1 -- 172.21.15.161:0/2515168654 reap_dead start 2024-05-22T21:15:43.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.129+0000 7f8088c00640 1 --2- 172.21.15.161:0/2515168654 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f806c008860 0x7f806c00ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:43.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.133+0000 7f8089600640 1 --2- 172.21.15.161:0/2515168654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f806c003a70 0x7f806c003e90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8078009fc0 tx=0x7f807801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee5a7cfb344c321 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:43.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.133+0000 7f8083e00640 1 -- 172.21.15.161:0/2515168654 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80780a5b80 con 0x7f806c003a70 2024-05-22T21:15:43.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.133+0000 7f8083e00640 1 -- 172.21.15.161:0/2515168654 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807800f040 con 0x7f806c003a70 2024-05-22T21:15:43.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.133+0000 7f8083e00640 1 -- 172.21.15.161:0/2515168654 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80780ad440 con 0x7f806c003a70 2024-05-22T21:15:43.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.133+0000 7f808c000640 1 -- 172.21.15.161:0/2515168654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f806c003a70 msgr2=0x7f806c003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.133+0000 7f808c000640 1 --2- 172.21.15.161:0/2515168654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f806c003a70 0x7f806c003e90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8078009fc0 tx=0x7f807801cc80 comp rx=0 tx=0).stop 2024-05-22T21:15:43.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.133+0000 7f808c000640 1 -- 172.21.15.161:0/2515168654 shutdown_connections 2024-05-22T21:15:43.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.133+0000 7f808c000640 1 --2- 172.21.15.161:0/2515168654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f806c003a70 0x7f806c003e90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.133+0000 7f808c000640 1 -- 172.21.15.161:0/2515168654 >> 172.21.15.161:0/2515168654 conn(0x7f80840b0bc0 msgr2=0x7f80840b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:43.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.133+0000 7f808c000640 1 -- 172.21.15.161:0/2515168654 shutdown_connections 2024-05-22T21:15:43.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.133+0000 7f808c000640 1 -- 172.21.15.161:0/2515168654 wait complete. 2024-05-22T21:15:43.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.133+0000 7f808c000640 1 Processor -- start 2024-05-22T21:15:43.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.133+0000 7f808c000640 1 -- start start 2024-05-22T21:15:43.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f808c000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c003a70 0x7f80841426c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f808c000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f808414bbc0 0x7f808414f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f808c000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80841f24d0 0x7f8084154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f808c000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808415ba80 con 0x7f808414bbc0 2024-05-22T21:15:43.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f808c000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8084142c00 con 0x7f806c003a70 2024-05-22T21:15:43.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f808c000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808415b1c0 con 0x7f80841f24d0 2024-05-22T21:15:43.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f8088c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f808414bbc0 0x7f808414f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f8088c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f808414bbc0 0x7f808414f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50186/0 (socket says 172.21.15.161:50186) 2024-05-22T21:15:43.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f8088c00640 1 -- 172.21.15.161:0/1865861589 learned_addr learned my addr 172.21.15.161:0/1865861589 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:43.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f8089600640 1 --2- 172.21.15.161:0/1865861589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c003a70 0x7f80841426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f808a000640 1 --2- 172.21.15.161:0/1865861589 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80841f24d0 0x7f8084154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f8088c00640 1 -- 172.21.15.161:0/1865861589 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80841f24d0 msgr2=0x7f8084154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f8088c00640 1 --2- 172.21.15.161:0/1865861589 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80841f24d0 0x7f8084154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f8088c00640 1 -- 172.21.15.161:0/1865861589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c003a70 msgr2=0x7f80841426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f8088c00640 1 --2- 172.21.15.161:0/1865861589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c003a70 0x7f80841426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f8088c00640 1 -- 172.21.15.161:0/1865861589 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808415b4a0 con 0x7f808414bbc0 2024-05-22T21:15:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f8088c00640 1 --2- 172.21.15.161:0/1865861589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f808414bbc0 0x7f808414f960 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f807000e480 tx=0x7f807000e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f8081600640 1 -- 172.21.15.161:0/1865861589 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070017070 con 0x7f808414bbc0 2024-05-22T21:15:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f8081600640 1 -- 172.21.15.161:0/1865861589 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80700052e0 con 0x7f808414bbc0 2024-05-22T21:15:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f8081600640 1 -- 172.21.15.161:0/1865861589 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070005490 con 0x7f808414bbc0 2024-05-22T21:15:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f808c000640 1 -- 172.21.15.161:0/1865861589 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808415b780 con 0x7f808414bbc0 2024-05-22T21:15:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f808c000640 1 -- 172.21.15.161:0/1865861589 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808414fea0 con 0x7f808414bbc0 2024-05-22T21:15:43.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.137+0000 7f808c000640 1 -- 172.21.15.161:0/1865861589 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8084067220 con 0x7f808414bbc0 2024-05-22T21:15:43.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.141+0000 7f8081600640 1 -- 172.21.15.161:0/1865861589 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8070007020 con 0x7f808414bbc0 2024-05-22T21:15:43.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.141+0000 7f8081600640 1 --2- 172.21.15.161:0/1865861589 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f80640666c0 0x7f8064068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.141+0000 7f8081600640 1 -- 172.21.15.161:0/1865861589 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8070083480 con 0x7f808414bbc0 2024-05-22T21:15:43.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.141+0000 7f8089600640 1 --2- 172.21.15.161:0/1865861589 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f80640666c0 0x7f8064068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.141+0000 7f8089600640 1 --2- 172.21.15.161:0/1865861589 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f80640666c0 0x7f8064068b80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f80780af000 tx=0x7f8078004ac0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:43.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.145+0000 7f8081600640 1 -- 172.21.15.161:0/1865861589 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8070050930 con 0x7f808414bbc0 2024-05-22T21:15:43.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.297+0000 7f808c000640 1 -- 172.21.15.161:0/1865861589 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80840b34e0 con 0x7f808414bbc0 2024-05-22T21:15:43.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.301+0000 7f8081600640 1 -- 172.21.15.161:0/1865861589 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16158 (secure 0 0 0) 0x7f80840b34e0 con 0x7f808414bbc0 2024-05-22T21:15:43.307 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:43.307 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":66,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:15:42.954895+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:15:40.613692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-22T21:15:41.954314+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:15:43.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.301+0000 7f808c000640 1 -- 172.21.15.161:0/1865861589 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f80640666c0 msgr2=0x7f8064068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.301+0000 7f808c000640 1 --2- 172.21.15.161:0/1865861589 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f80640666c0 0x7f8064068b80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f80780af000 tx=0x7f8078004ac0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.301+0000 7f808c000640 1 -- 172.21.15.161:0/1865861589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f808414bbc0 msgr2=0x7f808414f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.301+0000 7f808c000640 1 --2- 172.21.15.161:0/1865861589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f808414bbc0 0x7f808414f960 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f807000e480 tx=0x7f807000e940 comp rx=0 tx=0).stop 2024-05-22T21:15:43.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.301+0000 7f808c000640 1 -- 172.21.15.161:0/1865861589 shutdown_connections 2024-05-22T21:15:43.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.301+0000 7f808c000640 1 --2- 172.21.15.161:0/1865861589 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f80640666c0 0x7f8064068b80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.301+0000 7f808c000640 1 --2- 172.21.15.161:0/1865861589 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80841f24d0 0x7f8084154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.301+0000 7f808c000640 1 --2- 172.21.15.161:0/1865861589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f808414bbc0 0x7f808414f960 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.301+0000 7f808c000640 1 --2- 172.21.15.161:0/1865861589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c003a70 0x7f80841426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.301+0000 7f808c000640 1 -- 172.21.15.161:0/1865861589 >> 172.21.15.161:0/1865861589 conn(0x7f80840b0bc0 msgr2=0x7f80840afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:43.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.301+0000 7f808c000640 1 -- 172.21.15.161:0/1865861589 shutdown_connections 2024-05-22T21:15:43.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.301+0000 7f808c000640 1 -- 172.21.15.161:0/1865861589 wait complete. 2024-05-22T21:15:43.329 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:15:43.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.457+0000 7f00c3200640 1 Processor -- start 2024-05-22T21:15:43.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.457+0000 7f00c3200640 1 -- start start 2024-05-22T21:15:43.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.457+0000 7f00c3200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f00bc1597f0 0x7f00bc0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.457+0000 7f00c3200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f00bc0b3070 0x7f00bc0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.457+0000 7f00c3200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00bc071040 con 0x7f00bc070be0 2024-05-22T21:15:43.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.457+0000 7f00c3200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00bc0b34e0 con 0x7f00bc1597f0 2024-05-22T21:15:43.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.457+0000 7f00c3200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00bc159bd0 con 0x7f00bc0b3070 2024-05-22T21:15:43.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.457+0000 7f00bbe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f00bc1597f0 0x7f00bc0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.457+0000 7f00bbe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f00bc1597f0 0x7f00bc0706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50198/0 (socket says 172.21.15.161:50198) 2024-05-22T21:15:43.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.457+0000 7f00bbe00640 1 -- 172.21.15.161:0/1941562232 learned_addr learned my addr 172.21.15.161:0/1941562232 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:43.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.457+0000 7f00c1200640 1 --2- 172.21.15.161:0/1941562232 >> v2:172.21.15.179:3300/0 conn(0x7f00bc0b3070 0x7f00bc0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00bbe00640 1 -- 172.21.15.161:0/1941562232 >> v2:172.21.15.179:3300/0 conn(0x7f00bc0b3070 msgr2=0x7f00bc0cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:43.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00bbe00640 1 --2- 172.21.15.161:0/1941562232 >> v2:172.21.15.179:3300/0 conn(0x7f00bc0b3070 0x7f00bc0cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00bbe00640 1 -- 172.21.15.161:0/1941562232 >> v1:172.21.15.179:6789/0 conn(0x7f00bc070be0 legacy=0x7f00bc0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00bbe00640 1 -- 172.21.15.161:0/1941562232 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc0cbdd0 con 0x7f00bc1597f0 2024-05-22T21:15:43.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00c1200640 1 --2- 172.21.15.161:0/1941562232 >> v2:172.21.15.179:3300/0 conn(0x7f00bc0b3070 0x7f00bc0cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:15:43.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00bbe00640 1 --2- 172.21.15.161:0/1941562232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00bc1597f0 0x7f00bc0706a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0b0b50 tx=0x7f00ac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53dce976c954be05 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:43.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00bb400640 1 -- 172.21.15.161:0/1941562232 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac0a6040 con 0x7f00bc1597f0 2024-05-22T21:15:43.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00bb400640 1 -- 172.21.15.161:0/1941562232 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac0aa600 con 0x7f00bc1597f0 2024-05-22T21:15:43.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00bb400640 1 -- 172.21.15.161:0/1941562232 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac0aa990 con 0x7f00bc1597f0 2024-05-22T21:15:43.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00c3200640 1 -- 172.21.15.161:0/1941562232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00bc1597f0 msgr2=0x7f00bc0706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00c3200640 1 --2- 172.21.15.161:0/1941562232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00bc1597f0 0x7f00bc0706a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0b0b50 tx=0x7f00ac01c520 comp rx=0 tx=0).stop 2024-05-22T21:15:43.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00c3200640 1 -- 172.21.15.161:0/1941562232 shutdown_connections 2024-05-22T21:15:43.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00c3200640 1 --2- 172.21.15.161:0/1941562232 >> v2:172.21.15.179:3300/0 conn(0x7f00bc0b3070 0x7f00bc0cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00c3200640 1 --2- 172.21.15.161:0/1941562232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00bc1597f0 0x7f00bc0706a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00c3200640 1 -- 172.21.15.161:0/1941562232 >> 172.21.15.161:0/1941562232 conn(0x7f00bc0b0bc0 msgr2=0x7f00bc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:43.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00c3200640 1 -- 172.21.15.161:0/1941562232 shutdown_connections 2024-05-22T21:15:43.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.461+0000 7f00c3200640 1 -- 172.21.15.161:0/1941562232 wait complete. 2024-05-22T21:15:43.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c3200640 1 Processor -- start 2024-05-22T21:15:43.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c3200640 1 -- start start 2024-05-22T21:15:43.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c3200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f00bc070be0 0x7f00bc147530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c3200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00bc0b3070 0x7f00bc14e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c3200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00bc1597f0 0x7f00bc14e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c3200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc148ab0 con 0x7f00bc1597f0 2024-05-22T21:15:43.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c3200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc13e220 con 0x7f00bc0b3070 2024-05-22T21:15:43.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c3200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc147ea0 con 0x7f00bc070be0 2024-05-22T21:15:43.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c1200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00bc1597f0 0x7f00bc14e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c1200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00bc1597f0 0x7f00bc14e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50204/0 (socket says 172.21.15.161:50204) 2024-05-22T21:15:43.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c1200640 1 -- 172.21.15.161:0/17487960 learned_addr learned my addr 172.21.15.161:0/17487960 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:43.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00b3400640 1 --2- 172.21.15.161:0/17487960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00bc0b3070 0x7f00bc14e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00bbe00640 1 --2- 172.21.15.161:0/17487960 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f00bc070be0 0x7f00bc147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c1200640 1 -- 172.21.15.161:0/17487960 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f00bc070be0 msgr2=0x7f00bc147530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c1200640 1 --2- 172.21.15.161:0/17487960 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f00bc070be0 0x7f00bc147530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c1200640 1 -- 172.21.15.161:0/17487960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00bc0b3070 msgr2=0x7f00bc14e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c1200640 1 --2- 172.21.15.161:0/17487960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00bc0b3070 0x7f00bc14e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c1200640 1 -- 172.21.15.161:0/17487960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b4009a70 con 0x7f00bc1597f0 2024-05-22T21:15:43.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c1200640 1 --2- 172.21.15.161:0/17487960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00bc1597f0 0x7f00bc14e570 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f00b400efc0 tx=0x7f00b4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:43.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00b8c00640 1 -- 172.21.15.161:0/17487960 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4009d70 con 0x7f00bc1597f0 2024-05-22T21:15:43.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00b8c00640 1 -- 172.21.15.161:0/17487960 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b400f040 con 0x7f00bc1597f0 2024-05-22T21:15:43.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c3200640 1 -- 172.21.15.161:0/17487960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ac0094d0 con 0x7f00bc1597f0 2024-05-22T21:15:43.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00b8c00640 1 -- 172.21.15.161:0/17487960 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b40039a0 con 0x7f00bc1597f0 2024-05-22T21:15:43.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.465+0000 7f00c3200640 1 -- 172.21.15.161:0/17487960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00bc148460 con 0x7f00bc1597f0 2024-05-22T21:15:43.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.469+0000 7f00c3200640 1 -- 172.21.15.161:0/17487960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00bc148c60 con 0x7f00bc1597f0 2024-05-22T21:15:43.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.469+0000 7f00b8c00640 1 -- 172.21.15.161:0/17487960 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f00b40144a0 con 0x7f00bc1597f0 2024-05-22T21:15:43.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.469+0000 7f00b8c00640 1 --2- 172.21.15.161:0/17487960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0098066690 0x7f0098068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.469+0000 7f00b8c00640 1 -- 172.21.15.161:0/17487960 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00b4084d50 con 0x7f00bc1597f0 2024-05-22T21:15:43.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.469+0000 7f00bbe00640 1 --2- 172.21.15.161:0/17487960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0098066690 0x7f0098068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.469+0000 7f00bbe00640 1 --2- 172.21.15.161:0/17487960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0098066690 0x7f0098068b50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0ae000 tx=0x7f00ac0b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:43.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.473+0000 7f00b8c00640 1 -- 172.21.15.161:0/17487960 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f00bc148c60 con 0x7f00bc1597f0 2024-05-22T21:15:43.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.629+0000 7f00c3200640 1 -- 172.21.15.161:0/17487960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00bc148c60 con 0x7f00bc1597f0 2024-05-22T21:15:43.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.629+0000 7f00b8c00640 1 -- 172.21.15.161:0/17487960 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16158 (secure 0 0 0) 0x7f00bc148c60 con 0x7f00bc1597f0 2024-05-22T21:15:43.637 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:43.637 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":66,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:15:42.954895+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:15:40.613692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-22T21:15:41.954314+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:15:43.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.633+0000 7f00c3200640 1 -- 172.21.15.161:0/17487960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0098066690 msgr2=0x7f0098068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.633+0000 7f00c3200640 1 --2- 172.21.15.161:0/17487960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0098066690 0x7f0098068b50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0ae000 tx=0x7f00ac0b5040 comp rx=0 tx=0).stop 2024-05-22T21:15:43.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.633+0000 7f00c3200640 1 -- 172.21.15.161:0/17487960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00bc1597f0 msgr2=0x7f00bc14e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.633+0000 7f00c3200640 1 --2- 172.21.15.161:0/17487960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00bc1597f0 0x7f00bc14e570 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f00b400efc0 tx=0x7f00b4009f90 comp rx=0 tx=0).stop 2024-05-22T21:15:43.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.633+0000 7f00c3200640 1 -- 172.21.15.161:0/17487960 shutdown_connections 2024-05-22T21:15:43.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.633+0000 7f00c3200640 1 --2- 172.21.15.161:0/17487960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0098066690 0x7f0098068b50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.633+0000 7f00c3200640 1 --2- 172.21.15.161:0/17487960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00bc1597f0 0x7f00bc14e570 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.633+0000 7f00c3200640 1 --2- 172.21.15.161:0/17487960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00bc0b3070 0x7f00bc14e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.633+0000 7f00c3200640 1 --2- 172.21.15.161:0/17487960 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f00bc070be0 0x7f00bc147530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.633+0000 7f00c3200640 1 -- 172.21.15.161:0/17487960 >> 172.21.15.161:0/17487960 conn(0x7f00bc0b0bc0 msgr2=0x7f00bc14a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:43.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.633+0000 7f00c3200640 1 -- 172.21.15.161:0/17487960 shutdown_connections 2024-05-22T21:15:43.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.633+0000 7f00c3200640 1 -- 172.21.15.161:0/17487960 wait complete. 2024-05-22T21:15:43.659 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-22T21:15:43.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.789+0000 7f9b71a00640 1 Processor -- start 2024-05-22T21:15:43.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.789+0000 7f9b71a00640 1 -- start start 2024-05-22T21:15:43.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.793+0000 7f9b71a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b6c13cfc0 0x7f9b6c1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.793+0000 7f9b71a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9b6c15af60 0x7f9b6c13f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.793+0000 7f9b71a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b6c1400d0 con 0x7f9b6c13fc80 2024-05-22T21:15:43.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.793+0000 7f9b71a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b6c13d3a0 con 0x7f9b6c15af60 2024-05-22T21:15:43.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.793+0000 7f9b71a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c0cd920 con 0x7f9b6c13cfc0 2024-05-22T21:15:43.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.793+0000 7f9b62000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9b6c15af60 0x7f9b6c13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.793+0000 7f9b6aa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b6c13cfc0 0x7f9b6c1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.793+0000 7f9b6b400640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f9b6c13fc80 0x7f9b6c15dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:48976/0 (socket says 172.21.15.161:48976) 2024-05-22T21:15:43.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.793+0000 7f9b6b400640 1 -- 172.21.15.161:0/343774711 learned_addr learned my addr 172.21.15.161:0/343774711 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:43.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.793+0000 7f9b6a000640 1 -- 172.21.15.161:0/343774711 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1059875735 0 0) 0x7f9b640014a0 con 0x7f9b6c13fc80 2024-05-22T21:15:43.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.793+0000 7f9b6a000640 1 -- 172.21.15.161:0/343774711 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b50005740 con 0x7f9b6c13fc80 2024-05-22T21:15:43.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.793+0000 7f9b62000640 1 -- 172.21.15.161:0/343774711 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b6c13cfc0 msgr2=0x7f9b6c1402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:43.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.793+0000 7f9b62000640 1 --2- 172.21.15.161:0/343774711 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b6c13cfc0 0x7f9b6c1402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.793+0000 7f9b62000640 1 -- 172.21.15.161:0/343774711 >> v1:172.21.15.179:6789/0 conn(0x7f9b6c13fc80 legacy=0x7f9b6c15dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.793+0000 7f9b62000640 1 -- 172.21.15.161:0/343774711 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c0cdc00 con 0x7f9b6c15af60 2024-05-22T21:15:43.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.797+0000 7f9b62000640 1 --2- 172.21.15.161:0/343774711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b6c15af60 0x7f9b6c13f740 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9b540097c0 tx=0x7f9b5401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a99043bfd6614ef9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:43.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.797+0000 7f9b6a000640 1 -- 172.21.15.161:0/343774711 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b540ab600 con 0x7f9b6c15af60 2024-05-22T21:15:43.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.797+0000 7f9b6a000640 1 -- 172.21.15.161:0/343774711 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b540a7050 con 0x7f9b6c15af60 2024-05-22T21:15:43.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.797+0000 7f9b6a000640 1 -- 172.21.15.161:0/343774711 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b540ab980 con 0x7f9b6c15af60 2024-05-22T21:15:43.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.797+0000 7f9b71a00640 1 -- 172.21.15.161:0/343774711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b6c15af60 msgr2=0x7f9b6c13f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.797+0000 7f9b71a00640 1 --2- 172.21.15.161:0/343774711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b6c15af60 0x7f9b6c13f740 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9b540097c0 tx=0x7f9b5401c410 comp rx=0 tx=0).stop 2024-05-22T21:15:43.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.797+0000 7f9b71a00640 1 -- 172.21.15.161:0/343774711 shutdown_connections 2024-05-22T21:15:43.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.797+0000 7f9b71a00640 1 --2- 172.21.15.161:0/343774711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b6c15af60 0x7f9b6c13f740 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.797+0000 7f9b71a00640 1 --2- 172.21.15.161:0/343774711 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b6c13cfc0 0x7f9b6c1402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.797+0000 7f9b71a00640 1 -- 172.21.15.161:0/343774711 >> 172.21.15.161:0/343774711 conn(0x7f9b6c06a2d0 msgr2=0x7f9b6c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:43.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.797+0000 7f9b71a00640 1 -- 172.21.15.161:0/343774711 shutdown_connections 2024-05-22T21:15:43.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.797+0000 7f9b71a00640 1 -- 172.21.15.161:0/343774711 wait complete. 2024-05-22T21:15:43.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.797+0000 7f9b71a00640 1 Processor -- start 2024-05-22T21:15:43.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.797+0000 7f9b71a00640 1 -- start start 2024-05-22T21:15:43.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b71a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b6c13cfc0 0x7f9b6c13e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b71a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b6c13fc80 0x7f9b6c13b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b71a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b6c15af60 0x7f9b6c0b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b71a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c13b850 con 0x7f9b6c13fc80 2024-05-22T21:15:43.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b71a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c13e670 con 0x7f9b6c13cfc0 2024-05-22T21:15:43.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b71a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c13e950 con 0x7f9b6c15af60 2024-05-22T21:15:43.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b62000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b6c13fc80 0x7f9b6c13b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b62000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b6c13fc80 0x7f9b6c13b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50218/0 (socket says 172.21.15.161:50218) 2024-05-22T21:15:43.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b62000640 1 -- 172.21.15.161:0/312808430 learned_addr learned my addr 172.21.15.161:0/312808430 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:43.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b6b400640 1 --2- 172.21.15.161:0/312808430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b6c15af60 0x7f9b6c0b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b6aa00640 1 --2- 172.21.15.161:0/312808430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b6c13cfc0 0x7f9b6c13e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b62000640 1 -- 172.21.15.161:0/312808430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b6c15af60 msgr2=0x7f9b6c0b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b62000640 1 --2- 172.21.15.161:0/312808430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b6c15af60 0x7f9b6c0b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b62000640 1 -- 172.21.15.161:0/312808430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b6c13cfc0 msgr2=0x7f9b6c13e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:43.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b62000640 1 --2- 172.21.15.161:0/312808430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b6c13cfc0 0x7f9b6c13e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:43.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b62000640 1 -- 172.21.15.161:0/312808430 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b54009420 con 0x7f9b6c13fc80 2024-05-22T21:15:43.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b6b400640 1 --2- 172.21.15.161:0/312808430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b6c15af60 0x7f9b6c0b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:43.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b6aa00640 1 --2- 172.21.15.161:0/312808430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b6c13cfc0 0x7f9b6c13e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:43.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b62000640 1 --2- 172.21.15.161:0/312808430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b6c13fc80 0x7f9b6c13b310 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9b5401a040 tx=0x7f9b540191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:43.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b63400640 1 -- 172.21.15.161:0/312808430 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b540b5410 con 0x7f9b6c13fc80 2024-05-22T21:15:43.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b63400640 1 -- 172.21.15.161:0/312808430 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b540b5570 con 0x7f9b6c13fc80 2024-05-22T21:15:43.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b71a00640 1 -- 172.21.15.161:0/312808430 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6c13ec30 con 0x7f9b6c13fc80 2024-05-22T21:15:43.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b63400640 1 -- 172.21.15.161:0/312808430 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b540b5900 con 0x7f9b6c13fc80 2024-05-22T21:15:43.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b71a00640 1 -- 172.21.15.161:0/312808430 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6c13f1f0 con 0x7f9b6c13fc80 2024-05-22T21:15:43.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b71a00640 1 -- 172.21.15.161:0/312808430 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b6c13f3e0 con 0x7f9b6c13fc80 2024-05-22T21:15:43.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.801+0000 7f9b63400640 1 -- 172.21.15.161:0/312808430 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9b540b2070 con 0x7f9b6c13fc80 2024-05-22T21:15:43.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.805+0000 7f9b63400640 1 --2- 172.21.15.161:0/312808430 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9b48066690 0x7f9b48068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:43.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.805+0000 7f9b63400640 1 -- 172.21.15.161:0/312808430 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b5411e640 con 0x7f9b6c13fc80 2024-05-22T21:15:43.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.805+0000 7f9b6aa00640 1 --2- 172.21.15.161:0/312808430 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9b48066690 0x7f9b48068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:43.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.805+0000 7f9b6aa00640 1 --2- 172.21.15.161:0/312808430 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9b48066690 0x7f9b48068b50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0b00e0 tx=0x7f9b5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:43.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:43.809+0000 7f9b63400640 1 -- 172.21.15.161:0/312808430 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9b540eb910 con 0x7f9b6c13fc80 2024-05-22T21:15:44.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:44.001+0000 7f9b71a00640 1 -- 172.21.15.161:0/312808430 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9b6c067220 con 0x7f9b6c13fc80 2024-05-22T21:15:44.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:44.973+0000 7f9b63400640 1 -- 172.21.15.161:0/312808430 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9b540eef30 con 0x7f9b6c13fc80 2024-05-22T21:15:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:44.977+0000 7f9b71a00640 1 -- 172.21.15.161:0/312808430 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9b48066690 msgr2=0x7f9b48068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:44.977+0000 7f9b71a00640 1 --2- 172.21.15.161:0/312808430 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9b48066690 0x7f9b48068b50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0b00e0 tx=0x7f9b5c008040 comp rx=0 tx=0).stop 2024-05-22T21:15:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:44.977+0000 7f9b71a00640 1 -- 172.21.15.161:0/312808430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b6c13fc80 msgr2=0x7f9b6c13b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:44.977+0000 7f9b71a00640 1 --2- 172.21.15.161:0/312808430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b6c13fc80 0x7f9b6c13b310 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9b5401a040 tx=0x7f9b540191b0 comp rx=0 tx=0).stop 2024-05-22T21:15:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:44.977+0000 7f9b71a00640 1 -- 172.21.15.161:0/312808430 shutdown_connections 2024-05-22T21:15:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:44.977+0000 7f9b71a00640 1 --2- 172.21.15.161:0/312808430 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9b48066690 0x7f9b48068b50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:44.977+0000 7f9b71a00640 1 --2- 172.21.15.161:0/312808430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b6c15af60 0x7f9b6c0b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:44.977+0000 7f9b71a00640 1 --2- 172.21.15.161:0/312808430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b6c13fc80 0x7f9b6c13b310 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:44.977+0000 7f9b71a00640 1 --2- 172.21.15.161:0/312808430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b6c13cfc0 0x7f9b6c13e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:44.977+0000 7f9b71a00640 1 -- 172.21.15.161:0/312808430 >> 172.21.15.161:0/312808430 conn(0x7f9b6c06a2d0 msgr2=0x7f9b6c0ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:44.977+0000 7f9b71a00640 1 -- 172.21.15.161:0/312808430 shutdown_connections 2024-05-22T21:15:44.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:44.977+0000 7f9b71a00640 1 -- 172.21.15.161:0/312808430 wait complete. 2024-05-22T21:15:45.006 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-22T21:15:45.006 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-22T21:15:45.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.141+0000 7fee3d000640 1 Processor -- start 2024-05-22T21:15:45.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.141+0000 7fee3d000640 1 -- start start 2024-05-22T21:15:45.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.145+0000 7fee3d000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee38143c40 0x7fee3815b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:45.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.145+0000 7fee3d000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fee381444a0 0x7fee3806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:45.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.145+0000 7fee3d000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3815efb0 con 0x7fee38144900 2024-05-22T21:15:45.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.145+0000 7fee3d000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3815f290 con 0x7fee381444a0 2024-05-22T21:15:45.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.145+0000 7fee3d000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3815f570 con 0x7fee38143c40 2024-05-22T21:15:45.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.145+0000 7fee2d400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fee381444a0 0x7fee3806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:45.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.145+0000 7fee36800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fee38144900 0x7fee380b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:49544/0 (socket says 172.21.15.161:49544) 2024-05-22T21:15:45.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.145+0000 7fee36800640 1 -- 172.21.15.161:0/1804164057 learned_addr learned my addr 172.21.15.161:0/1804164057 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:45.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.145+0000 7fee35e00640 1 --2- 172.21.15.161:0/1804164057 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee38143c40 0x7fee3815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:45.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.145+0000 7fee2d400640 1 -- 172.21.15.161:0/1804164057 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee38143c40 msgr2=0x7fee3815b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:45.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.145+0000 7fee2d400640 1 --2- 172.21.15.161:0/1804164057 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee38143c40 0x7fee3815b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:45.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.145+0000 7fee2d400640 1 -- 172.21.15.161:0/1804164057 >> v1:172.21.15.161:6789/0 conn(0x7fee38144900 legacy=0x7fee380b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:45.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.145+0000 7fee2d400640 1 -- 172.21.15.161:0/1804164057 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3815f850 con 0x7fee381444a0 2024-05-22T21:15:45.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.145+0000 7fee35400640 1 -- 172.21.15.161:0/1804164057 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930427980 0 0) 0x7fee3815efb0 con 0x7fee38144900 2024-05-22T21:15:45.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee2d400640 1 --2- 172.21.15.161:0/1804164057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee381444a0 0x7fee3806c2a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fee200097c0 tx=0x7fee2001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3105f1b336672c53 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:45.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee35400640 1 -- 172.21.15.161:0/1804164057 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee200ac410 con 0x7fee381444a0 2024-05-22T21:15:45.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee35400640 1 -- 172.21.15.161:0/1804164057 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee200ac570 con 0x7fee381444a0 2024-05-22T21:15:45.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee35400640 1 -- 172.21.15.161:0/1804164057 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee200ac900 con 0x7fee381444a0 2024-05-22T21:15:45.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee3d000640 1 -- 172.21.15.161:0/1804164057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee381444a0 msgr2=0x7fee3806c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:45.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee3d000640 1 --2- 172.21.15.161:0/1804164057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee381444a0 0x7fee3806c2a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fee200097c0 tx=0x7fee2001c4a0 comp rx=0 tx=0).stop 2024-05-22T21:15:45.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee3d000640 1 -- 172.21.15.161:0/1804164057 shutdown_connections 2024-05-22T21:15:45.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee3d000640 1 --2- 172.21.15.161:0/1804164057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee381444a0 0x7fee3806c2a0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:45.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee3d000640 1 --2- 172.21.15.161:0/1804164057 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee38143c40 0x7fee3815b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:45.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee3d000640 1 -- 172.21.15.161:0/1804164057 >> 172.21.15.161:0/1804164057 conn(0x7fee3806a2d0 msgr2=0x7fee3806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:45.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee3d000640 1 -- 172.21.15.161:0/1804164057 shutdown_connections 2024-05-22T21:15:45.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee3d000640 1 -- 172.21.15.161:0/1804164057 wait complete. 2024-05-22T21:15:45.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee3d000640 1 Processor -- start 2024-05-22T21:15:45.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee3d000640 1 -- start start 2024-05-22T21:15:45.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee3d000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee38143c40 0x7fee38136760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:45.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee3d000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee381444a0 0x7fee38151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:45.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee3d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee38144900 0x7fee3815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:45.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee3d000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee38136ca0 con 0x7fee38144900 2024-05-22T21:15:45.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee3d000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee381599e0 con 0x7fee38143c40 2024-05-22T21:15:45.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee3d000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3815a560 con 0x7fee381444a0 2024-05-22T21:15:45.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.149+0000 7fee36800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee38144900 0x7fee3815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:45.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee36800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee38144900 0x7fee3815a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50234/0 (socket says 172.21.15.161:50234) 2024-05-22T21:15:45.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee36800640 1 -- 172.21.15.161:0/2728431135 learned_addr learned my addr 172.21.15.161:0/2728431135 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:45.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee2d400640 1 --2- 172.21.15.161:0/2728431135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee381444a0 0x7fee38151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:45.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee35e00640 1 --2- 172.21.15.161:0/2728431135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee38143c40 0x7fee38136760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:45.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee36800640 1 -- 172.21.15.161:0/2728431135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee381444a0 msgr2=0x7fee38151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:45.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee36800640 1 --2- 172.21.15.161:0/2728431135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee381444a0 0x7fee38151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:45.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee36800640 1 -- 172.21.15.161:0/2728431135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee38143c40 msgr2=0x7fee38136760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:45.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee36800640 1 --2- 172.21.15.161:0/2728431135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee38143c40 0x7fee38136760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:45.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee36800640 1 -- 172.21.15.161:0/2728431135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee20009420 con 0x7fee38144900 2024-05-22T21:15:45.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee36800640 1 --2- 172.21.15.161:0/2728431135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee38144900 0x7fee3815a9f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fee3000c3f0 tx=0x7fee3000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:45.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee2ea00640 1 -- 172.21.15.161:0/2728431135 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee300074e0 con 0x7fee38144900 2024-05-22T21:15:45.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee2ea00640 1 -- 172.21.15.161:0/2728431135 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee30007640 con 0x7fee38144900 2024-05-22T21:15:45.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee3d000640 1 -- 172.21.15.161:0/2728431135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee38159c30 con 0x7fee38144900 2024-05-22T21:15:45.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee2ea00640 1 -- 172.21.15.161:0/2728431135 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30004a30 con 0x7fee38144900 2024-05-22T21:15:45.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee3d000640 1 -- 172.21.15.161:0/2728431135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3815a1f0 con 0x7fee38144900 2024-05-22T21:15:45.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee3d000640 1 -- 172.21.15.161:0/2728431135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3815a6f0 con 0x7fee38144900 2024-05-22T21:15:45.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee2ea00640 1 -- 172.21.15.161:0/2728431135 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fee30004b90 con 0x7fee38144900 2024-05-22T21:15:45.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee2ea00640 1 --2- 172.21.15.161:0/2728431135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fee14066690 0x7fee14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:45.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.153+0000 7fee2ea00640 1 -- 172.21.15.161:0/2728431135 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee30083f10 con 0x7fee38144900 2024-05-22T21:15:45.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.157+0000 7fee35e00640 1 --2- 172.21.15.161:0/2728431135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fee14066690 0x7fee14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:45.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.157+0000 7fee35e00640 1 --2- 172.21.15.161:0/2728431135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fee14066690 0x7fee14068b50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fee280074a0 tx=0x7fee2800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:45.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.157+0000 7fee2ea00640 1 -- 172.21.15.161:0/2728431135 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fee30054d00 con 0x7fee38144900 2024-05-22T21:15:45.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.345+0000 7fee3d000640 1 -- 172.21.15.161:0/2728431135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fee38067220 con 0x7fee38144900 2024-05-22T21:15:45.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.977+0000 7fee2ea00640 1 -- 172.21.15.161:0/2728431135 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7fee30054890 con 0x7fee38144900 2024-05-22T21:15:45.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.977+0000 7fee3d000640 1 -- 172.21.15.161:0/2728431135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fee14066690 msgr2=0x7fee14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:45.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.977+0000 7fee3d000640 1 --2- 172.21.15.161:0/2728431135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fee14066690 0x7fee14068b50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fee280074a0 tx=0x7fee2800b040 comp rx=0 tx=0).stop 2024-05-22T21:15:45.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.977+0000 7fee3d000640 1 -- 172.21.15.161:0/2728431135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee38144900 msgr2=0x7fee3815a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:45.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.977+0000 7fee3d000640 1 --2- 172.21.15.161:0/2728431135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee38144900 0x7fee3815a9f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fee3000c3f0 tx=0x7fee3000c8b0 comp rx=0 tx=0).stop 2024-05-22T21:15:45.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.977+0000 7fee3d000640 1 -- 172.21.15.161:0/2728431135 shutdown_connections 2024-05-22T21:15:45.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.977+0000 7fee3d000640 1 --2- 172.21.15.161:0/2728431135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fee14066690 0x7fee14068b50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:45.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.977+0000 7fee3d000640 1 --2- 172.21.15.161:0/2728431135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee38144900 0x7fee3815a9f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:45.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.977+0000 7fee3d000640 1 --2- 172.21.15.161:0/2728431135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee381444a0 0x7fee38151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:45.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.977+0000 7fee3d000640 1 --2- 172.21.15.161:0/2728431135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee38143c40 0x7fee38136760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:45.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.977+0000 7fee3d000640 1 -- 172.21.15.161:0/2728431135 >> 172.21.15.161:0/2728431135 conn(0x7fee3806a2d0 msgr2=0x7fee38157ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:45.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.977+0000 7fee3d000640 1 -- 172.21.15.161:0/2728431135 shutdown_connections 2024-05-22T21:15:45.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:45.981+0000 7fee3d000640 1 -- 172.21.15.161:0/2728431135 wait complete. 2024-05-22T21:15:46.007 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-22T21:15:46.007 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:15:46.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.137+0000 7ff73de00640 1 Processor -- start 2024-05-22T21:15:46.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.137+0000 7ff73de00640 1 -- start start 2024-05-22T21:15:46.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.137+0000 7ff73de00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff73815dd50 0x7ff73806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.137+0000 7ff73de00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff73815d5e0 0x7ff73815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.137+0000 7ff73de00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff738071670 con 0x7ff73815e130 2024-05-22T21:15:46.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.137+0000 7ff73de00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73806de40 con 0x7ff73815d5e0 2024-05-22T21:15:46.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.137+0000 7ff73de00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff738070a60 con 0x7ff73815dd50 2024-05-22T21:15:46.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.141+0000 7ff737400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff73815d5e0 0x7ff73815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.141+0000 7ff737400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff73815d5e0 0x7ff73815b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55660/0 (socket says 172.21.15.161:55660) 2024-05-22T21:15:46.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.141+0000 7ff737400640 1 -- 172.21.15.161:0/2470358897 learned_addr learned my addr 172.21.15.161:0/2470358897 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:46.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.141+0000 7ff736a00640 1 --2- 172.21.15.161:0/2470358897 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff73815dd50 0x7ff73806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.141+0000 7ff737400640 1 -- 172.21.15.161:0/2470358897 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff73815dd50 msgr2=0x7ff73806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:46.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.141+0000 7ff737400640 1 --2- 172.21.15.161:0/2470358897 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff73815dd50 0x7ff73806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.141+0000 7ff737400640 1 -- 172.21.15.161:0/2470358897 >> v1:172.21.15.179:6789/0 conn(0x7ff73815e130 legacy=0x7ff73815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.141+0000 7ff737400640 1 -- 172.21.15.161:0/2470358897 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff738070d40 con 0x7ff73815d5e0 2024-05-22T21:15:46.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.141+0000 7ff737400640 1 --2- 172.21.15.161:0/2470358897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73815d5e0 0x7ff73815b6c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff730009e50 tx=0x7ff73001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c016b6012815b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:46.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.141+0000 7ff736000640 1 -- 172.21.15.161:0/2470358897 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7300a5830 con 0x7ff73815d5e0 2024-05-22T21:15:46.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.141+0000 7ff736000640 1 -- 172.21.15.161:0/2470358897 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7300a5990 con 0x7ff73815d5e0 2024-05-22T21:15:46.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.141+0000 7ff736000640 1 -- 172.21.15.161:0/2470358897 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7300a5d20 con 0x7ff73815d5e0 2024-05-22T21:15:46.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff73de00640 1 -- 172.21.15.161:0/2470358897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73815d5e0 msgr2=0x7ff73815b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff73de00640 1 --2- 172.21.15.161:0/2470358897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73815d5e0 0x7ff73815b6c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff730009e50 tx=0x7ff73001c970 comp rx=0 tx=0).stop 2024-05-22T21:15:46.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff73de00640 1 -- 172.21.15.161:0/2470358897 shutdown_connections 2024-05-22T21:15:46.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff73de00640 1 --2- 172.21.15.161:0/2470358897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73815d5e0 0x7ff73815b6c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff73de00640 1 --2- 172.21.15.161:0/2470358897 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff73815dd50 0x7ff73806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff73de00640 1 -- 172.21.15.161:0/2470358897 >> 172.21.15.161:0/2470358897 conn(0x7ff7380b0bc0 msgr2=0x7ff7380b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:46.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff73de00640 1 -- 172.21.15.161:0/2470358897 shutdown_connections 2024-05-22T21:15:46.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff73de00640 1 -- 172.21.15.161:0/2470358897 wait complete. 2024-05-22T21:15:46.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff73de00640 1 Processor -- start 2024-05-22T21:15:46.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff73de00640 1 -- start start 2024-05-22T21:15:46.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff73de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73815d5e0 0x7ff73814da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff73de00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff73815dd50 0x7ff73814ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff73de00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff73815e130 0x7ff738150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff73de00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7381555a0 con 0x7ff73815d5e0 2024-05-22T21:15:46.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff73de00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73806d8d0 con 0x7ff73815e130 2024-05-22T21:15:46.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff73de00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7381557f0 con 0x7ff73815dd50 2024-05-22T21:15:46.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff736a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73815d5e0 0x7ff73814da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff736a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73815d5e0 0x7ff73814da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55666/0 (socket says 172.21.15.161:55666) 2024-05-22T21:15:46.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff736a00640 1 -- 172.21.15.161:0/1704761121 learned_addr learned my addr 172.21.15.161:0/1704761121 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:46.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff72e000640 1 --2- 172.21.15.161:0/1704761121 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff73815dd50 0x7ff73814ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff737400640 1 --2- 172.21.15.161:0/1704761121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff73815e130 0x7ff738150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff736a00640 1 -- 172.21.15.161:0/1704761121 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff73815dd50 msgr2=0x7ff73814ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff736a00640 1 --2- 172.21.15.161:0/1704761121 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff73815dd50 0x7ff73814ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff736a00640 1 -- 172.21.15.161:0/1704761121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff73815e130 msgr2=0x7ff738150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.145+0000 7ff736a00640 1 --2- 172.21.15.161:0/1704761121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff73815e130 0x7ff738150460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.149+0000 7ff736a00640 1 -- 172.21.15.161:0/1704761121 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff730009b30 con 0x7ff73815d5e0 2024-05-22T21:15:46.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.149+0000 7ff737400640 1 --2- 172.21.15.161:0/1704761121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff73815e130 0x7ff738150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:46.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.149+0000 7ff736a00640 1 --2- 172.21.15.161:0/1704761121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73815d5e0 0x7ff73814da80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff72800e960 tx=0x7ff72800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:46.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.149+0000 7ff72fe00640 1 -- 172.21.15.161:0/1704761121 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728009690 con 0x7ff73815d5e0 2024-05-22T21:15:46.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.149+0000 7ff72fe00640 1 -- 172.21.15.161:0/1704761121 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72800ca00 con 0x7ff73815d5e0 2024-05-22T21:15:46.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.149+0000 7ff73de00640 1 -- 172.21.15.161:0/1704761121 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff738156b50 con 0x7ff73815d5e0 2024-05-22T21:15:46.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.149+0000 7ff72fe00640 1 -- 172.21.15.161:0/1704761121 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72800cd70 con 0x7ff73815d5e0 2024-05-22T21:15:46.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.149+0000 7ff73de00640 1 -- 172.21.15.161:0/1704761121 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73814e400 con 0x7ff73815d5e0 2024-05-22T21:15:46.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.149+0000 7ff73de00640 1 -- 172.21.15.161:0/1704761121 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff738067220 con 0x7ff73815d5e0 2024-05-22T21:15:46.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.149+0000 7ff72fe00640 1 -- 172.21.15.161:0/1704761121 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff728013470 con 0x7ff73815d5e0 2024-05-22T21:15:46.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.149+0000 7ff72fe00640 1 --2- 172.21.15.161:0/1704761121 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff714066690 0x7ff714068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.149+0000 7ff72fe00640 1 -- 172.21.15.161:0/1704761121 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff728084380 con 0x7ff73815d5e0 2024-05-22T21:15:46.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.149+0000 7ff72e000640 1 --2- 172.21.15.161:0/1704761121 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff714066690 0x7ff714068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.153+0000 7ff72e000640 1 --2- 172.21.15.161:0/1704761121 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff714066690 0x7ff714068b50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff7380caf50 tx=0x7ff720009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:46.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.153+0000 7ff72fe00640 1 -- 172.21.15.161:0/1704761121 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff728051790 con 0x7ff73815d5e0 2024-05-22T21:15:46.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.341+0000 7ff73de00640 1 -- 172.21.15.161:0/1704761121 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7380af540 con 0x7ff73815d5e0 2024-05-22T21:15:46.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.341+0000 7ff72fe00640 1 -- 172.21.15.161:0/1704761121 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4743 (secure 0 0 0) 0x7ff728054db0 con 0x7ff73815d5e0 2024-05-22T21:15:46.347 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:46.347 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1620021722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":1620021722},{"type":"v1","addr":"172.21.15.179:6835","nonce":1620021722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/3548738614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3548738614},{"type":"v1","addr":"172.21.15.161:6836","nonce":3548738614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/400898801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":400898801},{"type":"v1","addr":"172.21.15.161:6837","nonce":400898801}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:15:42.955245+0000","modified":"2024-05-22T21:15:45.977718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6834/2239627963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2239627963},{"type":"v1","addr":"172.21.15.179:6834","nonce":2239627963}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-22T21:15:46.347 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 31 2024-05-22T21:15:46.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.341+0000 7ff73de00640 1 -- 172.21.15.161:0/1704761121 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff714066690 msgr2=0x7ff714068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.341+0000 7ff73de00640 1 --2- 172.21.15.161:0/1704761121 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff714066690 0x7ff714068b50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff7380caf50 tx=0x7ff720009040 comp rx=0 tx=0).stop 2024-05-22T21:15:46.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.341+0000 7ff73de00640 1 -- 172.21.15.161:0/1704761121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73815d5e0 msgr2=0x7ff73814da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.341+0000 7ff73de00640 1 --2- 172.21.15.161:0/1704761121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73815d5e0 0x7ff73814da80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff72800e960 tx=0x7ff72800ee20 comp rx=0 tx=0).stop 2024-05-22T21:15:46.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.341+0000 7ff73de00640 1 -- 172.21.15.161:0/1704761121 shutdown_connections 2024-05-22T21:15:46.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.341+0000 7ff73de00640 1 --2- 172.21.15.161:0/1704761121 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff714066690 0x7ff714068b50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.341+0000 7ff73de00640 1 --2- 172.21.15.161:0/1704761121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff73815e130 0x7ff738150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.341+0000 7ff73de00640 1 --2- 172.21.15.161:0/1704761121 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff73815dd50 0x7ff73814ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.341+0000 7ff73de00640 1 --2- 172.21.15.161:0/1704761121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73815d5e0 0x7ff73814da80 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.341+0000 7ff73de00640 1 -- 172.21.15.161:0/1704761121 >> 172.21.15.161:0/1704761121 conn(0x7ff7380b0bc0 msgr2=0x7ff7380b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:46.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.341+0000 7ff73de00640 1 -- 172.21.15.161:0/1704761121 shutdown_connections 2024-05-22T21:15:46.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.341+0000 7ff73de00640 1 -- 172.21.15.161:0/1704761121 wait complete. 2024-05-22T21:15:46.369 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:15:46.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.501+0000 7fe91a000640 1 Processor -- start 2024-05-22T21:15:46.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.501+0000 7fe91a000640 1 -- start start 2024-05-22T21:15:46.507 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.501+0000 7fe91a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe914071140 0x7fe914071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.507 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.501+0000 7fe91a000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe914071ab0 con 0x7fe91415b730 2024-05-22T21:15:46.507 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.501+0000 7fe91a000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe91415bdb0 con 0x7fe9141448b0 2024-05-22T21:15:46.507 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.501+0000 7fe91a000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91415bbb0 con 0x7fe914071140 2024-05-22T21:15:46.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.501+0000 7fe913e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe914071140 0x7fe914071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.501+0000 7fe912a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe91415b730 0x7fe91406e440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51848/0 (socket says 172.21.15.161:51848) 2024-05-22T21:15:46.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.501+0000 7fe912a00640 1 -- 172.21.15.161:0/3454469135 learned_addr learned my addr 172.21.15.161:0/3454469135 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:46.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.501+0000 7fe913e00640 1 -- 172.21.15.161:0/3454469135 >> v1:172.21.15.179:6789/0 conn(0x7fe9141448b0 legacy=0x7fe91406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.501+0000 7fe913e00640 1 -- 172.21.15.161:0/3454469135 >> v1:172.21.15.161:6789/0 conn(0x7fe91415b730 legacy=0x7fe91406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.501+0000 7fe913e00640 1 -- 172.21.15.161:0/3454469135 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91415c090 con 0x7fe914071140 2024-05-22T21:15:46.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.501+0000 7fe912000640 1 -- 172.21.15.161:0/3454469135 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583381154 0 0) 0x7fe914071ab0 con 0x7fe91415b730 2024-05-22T21:15:46.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.505+0000 7fe913e00640 1 --2- 172.21.15.161:0/3454469135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe914071140 0x7fe914071570 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009d90 tx=0x7fe90c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a17305f68a1521 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:46.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.505+0000 7fe912000640 1 -- 172.21.15.161:0/3454469135 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0a5a00 con 0x7fe914071140 2024-05-22T21:15:46.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.505+0000 7fe912000640 1 -- 172.21.15.161:0/3454469135 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90c0a5b60 con 0x7fe914071140 2024-05-22T21:15:46.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.505+0000 7fe912000640 1 -- 172.21.15.161:0/3454469135 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0ad440 con 0x7fe914071140 2024-05-22T21:15:46.512 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.505+0000 7fe91a000640 1 -- 172.21.15.161:0/3454469135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe914071140 msgr2=0x7fe914071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.512 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.505+0000 7fe91a000640 1 --2- 172.21.15.161:0/3454469135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe914071140 0x7fe914071570 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009d90 tx=0x7fe90c01c940 comp rx=0 tx=0).stop 2024-05-22T21:15:46.512 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.505+0000 7fe91a000640 1 -- 172.21.15.161:0/3454469135 shutdown_connections 2024-05-22T21:15:46.512 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.505+0000 7fe91a000640 1 --2- 172.21.15.161:0/3454469135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe914071140 0x7fe914071570 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.512 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.505+0000 7fe91a000640 1 -- 172.21.15.161:0/3454469135 >> 172.21.15.161:0/3454469135 conn(0x7fe9140b1c40 msgr2=0x7fe9140b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:46.512 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.505+0000 7fe91a000640 1 -- 172.21.15.161:0/3454469135 shutdown_connections 2024-05-22T21:15:46.512 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.505+0000 7fe91a000640 1 -- 172.21.15.161:0/3454469135 wait complete. 2024-05-22T21:15:46.513 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.505+0000 7fe91a000640 1 Processor -- start 2024-05-22T21:15:46.513 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.505+0000 7fe91a000640 1 -- start start 2024-05-22T21:15:46.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe91a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe914071140 0x7fe91415aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe91a000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe9141448b0 0x7fe91415dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe91a000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe91415b730 0x7fe914159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe91a000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe914142790 con 0x7fe914071140 2024-05-22T21:15:46.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe91a000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91413fd70 con 0x7fe91415b730 2024-05-22T21:15:46.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe91a000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe914142140 con 0x7fe9141448b0 2024-05-22T21:15:46.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe913400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe914071140 0x7fe91415aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe913400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe914071140 0x7fe91415aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55690/0 (socket says 172.21.15.161:55690) 2024-05-22T21:15:46.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe913400640 1 -- 172.21.15.161:0/3071745373 learned_addr learned my addr 172.21.15.161:0/3071745373 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:46.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe912a00640 1 --2- 172.21.15.161:0/3071745373 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe9141448b0 0x7fe91415dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe913e00640 1 --2- 172.21.15.161:0/3071745373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe91415b730 0x7fe914159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe913400640 1 -- 172.21.15.161:0/3071745373 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe9141448b0 msgr2=0x7fe91415dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe913400640 1 --2- 172.21.15.161:0/3071745373 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe9141448b0 0x7fe91415dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe913400640 1 -- 172.21.15.161:0/3071745373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe91415b730 msgr2=0x7fe914159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe913400640 1 --2- 172.21.15.161:0/3071745373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe91415b730 0x7fe914159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe913400640 1 -- 172.21.15.161:0/3071745373 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c009a70 con 0x7fe914071140 2024-05-22T21:15:46.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe913400640 1 --2- 172.21.15.161:0/3071745373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe914071140 0x7fe91415aac0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe90400c450 tx=0x7fe90400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:46.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe90b400640 1 -- 172.21.15.161:0/3071745373 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9040052d0 con 0x7fe914071140 2024-05-22T21:15:46.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe90b400640 1 -- 172.21.15.161:0/3071745373 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe904005450 con 0x7fe914071140 2024-05-22T21:15:46.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe91a000640 1 -- 172.21.15.161:0/3071745373 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe914142420 con 0x7fe914071140 2024-05-22T21:15:46.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe90b400640 1 -- 172.21.15.161:0/3071745373 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe904012870 con 0x7fe914071140 2024-05-22T21:15:46.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe91a000640 1 -- 172.21.15.161:0/3071745373 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91415a740 con 0x7fe914071140 2024-05-22T21:15:46.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.509+0000 7fe91a000640 1 -- 172.21.15.161:0/3071745373 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9140b2ba0 con 0x7fe914071140 2024-05-22T21:15:46.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.513+0000 7fe90b400640 1 -- 172.21.15.161:0/3071745373 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe9040129d0 con 0x7fe914071140 2024-05-22T21:15:46.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.513+0000 7fe90b400640 1 --2- 172.21.15.161:0/3071745373 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe8f4066690 0x7fe8f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.513+0000 7fe90b400640 1 -- 172.21.15.161:0/3071745373 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe904083c10 con 0x7fe914071140 2024-05-22T21:15:46.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.513+0000 7fe912a00640 1 --2- 172.21.15.161:0/3071745373 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe8f4066690 0x7fe8f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.513+0000 7fe912a00640 1 --2- 172.21.15.161:0/3071745373 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe8f4066690 0x7fe8f4068b50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe9140b3810 tx=0x7fe900007240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:46.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.517+0000 7fe90b400640 1 -- 172.21.15.161:0/3071745373 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe904050f90 con 0x7fe914071140 2024-05-22T21:15:46.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.673+0000 7fe91a000640 1 -- 172.21.15.161:0/3071745373 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe914067220 con 0x7fe914071140 2024-05-22T21:15:46.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.673+0000 7fe90b400640 1 -- 172.21.15.161:0/3071745373 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16145 (secure 0 0 0) 0x7fe9040545b0 con 0x7fe914071140 2024-05-22T21:15:46.681 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:46.681 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":67,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:15:43.958368+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:15:40.613692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-22T21:15:41.954314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:15:46.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.677+0000 7fe91a000640 1 -- 172.21.15.161:0/3071745373 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe8f4066690 msgr2=0x7fe8f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.677+0000 7fe91a000640 1 --2- 172.21.15.161:0/3071745373 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe8f4066690 0x7fe8f4068b50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe9140b3810 tx=0x7fe900007240 comp rx=0 tx=0).stop 2024-05-22T21:15:46.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.677+0000 7fe91a000640 1 -- 172.21.15.161:0/3071745373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe914071140 msgr2=0x7fe91415aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.677+0000 7fe91a000640 1 --2- 172.21.15.161:0/3071745373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe914071140 0x7fe91415aac0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe90400c450 tx=0x7fe90400c910 comp rx=0 tx=0).stop 2024-05-22T21:15:46.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.677+0000 7fe91a000640 1 -- 172.21.15.161:0/3071745373 shutdown_connections 2024-05-22T21:15:46.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.677+0000 7fe91a000640 1 --2- 172.21.15.161:0/3071745373 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe8f4066690 0x7fe8f4068b50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.677+0000 7fe91a000640 1 --2- 172.21.15.161:0/3071745373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe91415b730 0x7fe914159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.677+0000 7fe91a000640 1 --2- 172.21.15.161:0/3071745373 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe9141448b0 0x7fe91415dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.677+0000 7fe91a000640 1 --2- 172.21.15.161:0/3071745373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe914071140 0x7fe91415aac0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.677+0000 7fe91a000640 1 -- 172.21.15.161:0/3071745373 >> 172.21.15.161:0/3071745373 conn(0x7fe9140b1c40 msgr2=0x7fe9140cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:46.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.677+0000 7fe91a000640 1 -- 172.21.15.161:0/3071745373 shutdown_connections 2024-05-22T21:15:46.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.677+0000 7fe91a000640 1 -- 172.21.15.161:0/3071745373 wait complete. 2024-05-22T21:15:46.704 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-22T21:15:46.704 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:15:46.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.833+0000 7efc53000640 1 Processor -- start 2024-05-22T21:15:46.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.833+0000 7efc53000640 1 -- start start 2024-05-22T21:15:46.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.833+0000 7efc53000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efc4c1597f0 0x7efc4c0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.833+0000 7efc53000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc4c0b3070 0x7efc4c0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.833+0000 7efc53000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4c071040 con 0x7efc4c070be0 2024-05-22T21:15:46.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.833+0000 7efc53000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4c0b3500 con 0x7efc4c1597f0 2024-05-22T21:15:46.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.833+0000 7efc53000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c159bd0 con 0x7efc4c0b3070 2024-05-22T21:15:46.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.833+0000 7efc4b400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7efc4c070be0 0x7efc4c0b2960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51858/0 (socket says 172.21.15.161:51858) 2024-05-22T21:15:46.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.833+0000 7efc4b400640 1 -- 172.21.15.161:0/2204420911 learned_addr learned my addr 172.21.15.161:0/2204420911 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:46.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.833+0000 7efc4be00640 1 --2- 172.21.15.161:0/2204420911 >> v2:172.21.15.179:3300/0 conn(0x7efc4c1597f0 0x7efc4c0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.833+0000 7efc51000640 1 --2- 172.21.15.161:0/2204420911 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc4c0b3070 0x7efc4c0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.833+0000 7efc4aa00640 1 -- 172.21.15.161:0/2204420911 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 738702425 0 0) 0x7efc4c071040 con 0x7efc4c070be0 2024-05-22T21:15:46.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.833+0000 7efc4aa00640 1 -- 172.21.15.161:0/2204420911 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc30003530 con 0x7efc4c070be0 2024-05-22T21:15:46.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.837+0000 7efc4be00640 1 -- 172.21.15.161:0/2204420911 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc4c0b3070 msgr2=0x7efc4c0cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:46.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.837+0000 7efc4be00640 1 --2- 172.21.15.161:0/2204420911 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc4c0b3070 0x7efc4c0cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.837+0000 7efc4be00640 1 -- 172.21.15.161:0/2204420911 >> v1:172.21.15.161:6789/0 conn(0x7efc4c070be0 legacy=0x7efc4c0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.837+0000 7efc4be00640 1 -- 172.21.15.161:0/2204420911 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c0cbdd0 con 0x7efc4c1597f0 2024-05-22T21:15:46.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.837+0000 7efc4be00640 1 --2- 172.21.15.161:0/2204420911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc4c1597f0 0x7efc4c0706a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0097f0 tx=0x7efc3c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8bd66e8726c171b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:46.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.837+0000 7efc4aa00640 1 -- 172.21.15.161:0/2204420911 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c0ab620 con 0x7efc4c1597f0 2024-05-22T21:15:46.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.837+0000 7efc4aa00640 1 -- 172.21.15.161:0/2204420911 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3c0ab780 con 0x7efc4c1597f0 2024-05-22T21:15:46.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.837+0000 7efc4aa00640 1 -- 172.21.15.161:0/2204420911 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c0abaf0 con 0x7efc4c1597f0 2024-05-22T21:15:46.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.837+0000 7efc53000640 1 -- 172.21.15.161:0/2204420911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc4c1597f0 msgr2=0x7efc4c0706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.837+0000 7efc53000640 1 --2- 172.21.15.161:0/2204420911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc4c1597f0 0x7efc4c0706a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0097f0 tx=0x7efc3c01c4a0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.837+0000 7efc53000640 1 -- 172.21.15.161:0/2204420911 shutdown_connections 2024-05-22T21:15:46.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.837+0000 7efc53000640 1 --2- 172.21.15.161:0/2204420911 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc4c0b3070 0x7efc4c0cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.837+0000 7efc53000640 1 --2- 172.21.15.161:0/2204420911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc4c1597f0 0x7efc4c0706a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.837+0000 7efc53000640 1 -- 172.21.15.161:0/2204420911 >> 172.21.15.161:0/2204420911 conn(0x7efc4c0b0bc0 msgr2=0x7efc4c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:46.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.837+0000 7efc53000640 1 -- 172.21.15.161:0/2204420911 shutdown_connections 2024-05-22T21:15:46.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.837+0000 7efc53000640 1 -- 172.21.15.161:0/2204420911 wait complete. 2024-05-22T21:15:46.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc53000640 1 Processor -- start 2024-05-22T21:15:46.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc53000640 1 -- start start 2024-05-22T21:15:46.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc53000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc4c070be0 0x7efc4c146e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc53000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc4c0b3070 0x7efc4c158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc53000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc4c1597f0 0x7efc4c06ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc53000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c1473e0 con 0x7efc4c1597f0 2024-05-22T21:15:46.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc53000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c157da0 con 0x7efc4c0b3070 2024-05-22T21:15:46.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc53000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c158080 con 0x7efc4c070be0 2024-05-22T21:15:46.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc51000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc4c1597f0 0x7efc4c06ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc51000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc4c1597f0 0x7efc4c06ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55702/0 (socket says 172.21.15.161:55702) 2024-05-22T21:15:46.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc51000640 1 -- 172.21.15.161:0/950922296 learned_addr learned my addr 172.21.15.161:0/950922296 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:46.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc4b400640 1 --2- 172.21.15.161:0/950922296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc4c0b3070 0x7efc4c158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc4be00640 1 --2- 172.21.15.161:0/950922296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc4c070be0 0x7efc4c146e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc51000640 1 -- 172.21.15.161:0/950922296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc4c070be0 msgr2=0x7efc4c146e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc51000640 1 --2- 172.21.15.161:0/950922296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc4c070be0 0x7efc4c146e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc51000640 1 -- 172.21.15.161:0/950922296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc4c0b3070 msgr2=0x7efc4c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:46.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc51000640 1 --2- 172.21.15.161:0/950922296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc4c0b3070 0x7efc4c158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:46.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc51000640 1 -- 172.21.15.161:0/950922296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c009420 con 0x7efc4c1597f0 2024-05-22T21:15:46.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc4b400640 1 --2- 172.21.15.161:0/950922296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc4c0b3070 0x7efc4c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:46.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc51000640 1 --2- 172.21.15.161:0/950922296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc4c1597f0 0x7efc4c06ce70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7efc4400efc0 tx=0x7efc4400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:46.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc43e00640 1 -- 172.21.15.161:0/950922296 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44014480 con 0x7efc4c1597f0 2024-05-22T21:15:46.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc43e00640 1 -- 172.21.15.161:0/950922296 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4400f040 con 0x7efc4c1597f0 2024-05-22T21:15:46.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc53000640 1 -- 172.21.15.161:0/950922296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4c158360 con 0x7efc4c1597f0 2024-05-22T21:15:46.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc43e00640 1 -- 172.21.15.161:0/950922296 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44014820 con 0x7efc4c1597f0 2024-05-22T21:15:46.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.841+0000 7efc53000640 1 -- 172.21.15.161:0/950922296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4c13ed50 con 0x7efc4c1597f0 2024-05-22T21:15:46.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.845+0000 7efc53000640 1 -- 172.21.15.161:0/950922296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4c158880 con 0x7efc4c1597f0 2024-05-22T21:15:46.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.845+0000 7efc43e00640 1 -- 172.21.15.161:0/950922296 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efc44014980 con 0x7efc4c1597f0 2024-05-22T21:15:46.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.845+0000 7efc43e00640 1 --2- 172.21.15.161:0/950922296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc2c066690 0x7efc2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:46.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.845+0000 7efc43e00640 1 -- 172.21.15.161:0/950922296 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc44085120 con 0x7efc4c1597f0 2024-05-22T21:15:46.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.845+0000 7efc4be00640 1 --2- 172.21.15.161:0/950922296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc2c066690 0x7efc2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:46.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.845+0000 7efc4be00640 1 --2- 172.21.15.161:0/950922296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc2c066690 0x7efc2c068b50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0af000 tx=0x7efc3c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:46.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:46.849+0000 7efc43e00640 1 -- 172.21.15.161:0/950922296 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efc44052360 con 0x7efc4c1597f0 2024-05-22T21:15:47.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.005+0000 7efc53000640 1 -- 172.21.15.161:0/950922296 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc4c157f10 con 0x7efc2c066690 2024-05-22T21:15:47.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.013+0000 7efc43e00640 1 -- 172.21.15.161:0/950922296 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284866 (secure 0 0 0) 0x7efc4c157f10 con 0x7efc2c066690 2024-05-22T21:15:47.024 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:47.026 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:15:47.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.017+0000 7efc53000640 1 -- 172.21.15.161:0/950922296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc2c066690 msgr2=0x7efc2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:47.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.017+0000 7efc53000640 1 --2- 172.21.15.161:0/950922296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc2c066690 0x7efc2c068b50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0af000 tx=0x7efc3c0a7000 comp rx=0 tx=0).stop 2024-05-22T21:15:47.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.017+0000 7efc53000640 1 -- 172.21.15.161:0/950922296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc4c1597f0 msgr2=0x7efc4c06ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:47.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.017+0000 7efc53000640 1 --2- 172.21.15.161:0/950922296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc4c1597f0 0x7efc4c06ce70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7efc4400efc0 tx=0x7efc4400c490 comp rx=0 tx=0).stop 2024-05-22T21:15:47.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.017+0000 7efc53000640 1 -- 172.21.15.161:0/950922296 shutdown_connections 2024-05-22T21:15:47.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.017+0000 7efc53000640 1 --2- 172.21.15.161:0/950922296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc2c066690 0x7efc2c068b50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.017+0000 7efc53000640 1 --2- 172.21.15.161:0/950922296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc4c1597f0 0x7efc4c06ce70 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.017+0000 7efc53000640 1 --2- 172.21.15.161:0/950922296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc4c0b3070 0x7efc4c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.017+0000 7efc53000640 1 --2- 172.21.15.161:0/950922296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc4c070be0 0x7efc4c146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.017+0000 7efc53000640 1 -- 172.21.15.161:0/950922296 >> 172.21.15.161:0/950922296 conn(0x7efc4c0b0bc0 msgr2=0x7efc4c0cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:47.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.017+0000 7efc53000640 1 -- 172.21.15.161:0/950922296 shutdown_connections 2024-05-22T21:15:47.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.017+0000 7efc53000640 1 -- 172.21.15.161:0/950922296 wait complete. 2024-05-22T21:15:47.043 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-05-22T21:15:46.423363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3848,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558651,"statfs":{"total":1600000000,"available":1596059399,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.860703"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.400825+0000","last_change":"2024-05-22T21:15:42.970594+0000","last_active":"2024-05-22T21:15:44.400825+0000","last_peered":"2024-05-22T21:15:44.400825+0000","last_clean":"2024-05-22T21:15:44.400825+0000","last_became_active":"2024-05-22T21:15:42.970393+0000","last_became_peered":"2024-05-22T21:15:42.970393+0000","last_unstale":"2024-05-22T21:15:44.400825+0000","last_undegraded":"2024-05-22T21:15:44.400825+0000","last_fullsized":"2024-05-22T21:15:44.400825+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:22.960323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064119+0000","last_change":"2024-05-22T21:15:42.971240+0000","last_active":"2024-05-22T21:15:44.064119+0000","last_peered":"2024-05-22T21:15:44.064119+0000","last_clean":"2024-05-22T21:15:44.064119+0000","last_became_active":"2024-05-22T21:15:42.970353+0000","last_became_peered":"2024-05-22T21:15:42.970353+0000","last_unstale":"2024-05-22T21:15:44.064119+0000","last_undegraded":"2024-05-22T21:15:44.064119+0000","last_fullsized":"2024-05-22T21:15:44.064119+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:03:46.835560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.962729+0000","last_change":"2024-05-22T21:15:42.977075+0000","last_active":"2024-05-22T21:15:43.962729+0000","last_peered":"2024-05-22T21:15:43.962729+0000","last_clean":"2024-05-22T21:15:43.962729+0000","last_became_active":"2024-05-22T21:15:42.976413+0000","last_became_peered":"2024-05-22T21:15:42.976413+0000","last_unstale":"2024-05-22T21:15:43.962729+0000","last_undegraded":"2024-05-22T21:15:43.962729+0000","last_fullsized":"2024-05-22T21:15:43.962729+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:46:40.357385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401358+0000","last_change":"2024-05-22T21:15:42.978184+0000","last_active":"2024-05-22T21:15:44.401358+0000","last_peered":"2024-05-22T21:15:44.401358+0000","last_clean":"2024-05-22T21:15:44.401358+0000","last_became_active":"2024-05-22T21:15:42.977823+0000","last_became_peered":"2024-05-22T21:15:42.977823+0000","last_unstale":"2024-05-22T21:15:44.401358+0000","last_undegraded":"2024-05-22T21:15:44.401358+0000","last_fullsized":"2024-05-22T21:15:44.401358+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:55.539734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.400778+0000","last_change":"2024-05-22T21:15:42.976106+0000","last_active":"2024-05-22T21:15:44.400778+0000","last_peered":"2024-05-22T21:15:44.400778+0000","last_clean":"2024-05-22T21:15:44.400778+0000","last_became_active":"2024-05-22T21:15:42.975911+0000","last_became_peered":"2024-05-22T21:15:42.975911+0000","last_unstale":"2024-05-22T21:15:44.400778+0000","last_undegraded":"2024-05-22T21:15:44.400778+0000","last_fullsized":"2024-05-22T21:15:44.400778+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:19:28.350013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948839+0000","last_change":"2024-05-22T21:15:42.969930+0000","last_active":"2024-05-22T21:15:44.948839+0000","last_peered":"2024-05-22T21:15:44.948839+0000","last_clean":"2024-05-22T21:15:44.948839+0000","last_became_active":"2024-05-22T21:15:42.969715+0000","last_became_peered":"2024-05-22T21:15:42.969715+0000","last_unstale":"2024-05-22T21:15:44.948839+0000","last_undegraded":"2024-05-22T21:15:44.948839+0000","last_fullsized":"2024-05-22T21:15:44.948839+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:05:51.541715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.962894+0000","last_change":"2024-05-22T21:15:42.978863+0000","last_active":"2024-05-22T21:15:43.962894+0000","last_peered":"2024-05-22T21:15:43.962894+0000","last_clean":"2024-05-22T21:15:43.962894+0000","last_became_active":"2024-05-22T21:15:42.978481+0000","last_became_peered":"2024-05-22T21:15:42.978481+0000","last_unstale":"2024-05-22T21:15:43.962894+0000","last_undegraded":"2024-05-22T21:15:43.962894+0000","last_fullsized":"2024-05-22T21:15:43.962894+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:08.621948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064089+0000","last_change":"2024-05-22T21:15:42.973722+0000","last_active":"2024-05-22T21:15:44.064089+0000","last_peered":"2024-05-22T21:15:44.064089+0000","last_clean":"2024-05-22T21:15:44.064089+0000","last_became_active":"2024-05-22T21:15:42.973142+0000","last_became_peered":"2024-05-22T21:15:42.973142+0000","last_unstale":"2024-05-22T21:15:44.064089+0000","last_undegraded":"2024-05-22T21:15:44.064089+0000","last_fullsized":"2024-05-22T21:15:44.064089+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:31:41.440098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.962948+0000","last_change":"2024-05-22T21:15:42.971446+0000","last_active":"2024-05-22T21:15:43.962948+0000","last_peered":"2024-05-22T21:15:43.962948+0000","last_clean":"2024-05-22T21:15:43.962948+0000","last_became_active":"2024-05-22T21:15:42.971205+0000","last_became_peered":"2024-05-22T21:15:42.971205+0000","last_unstale":"2024-05-22T21:15:43.962948+0000","last_undegraded":"2024-05-22T21:15:43.962948+0000","last_fullsized":"2024-05-22T21:15:43.962948+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:53:24.792090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961665+0000","last_change":"2024-05-22T21:15:42.974961+0000","last_active":"2024-05-22T21:15:43.961665+0000","last_peered":"2024-05-22T21:15:43.961665+0000","last_clean":"2024-05-22T21:15:43.961665+0000","last_became_active":"2024-05-22T21:15:42.974656+0000","last_became_peered":"2024-05-22T21:15:42.974656+0000","last_unstale":"2024-05-22T21:15:43.961665+0000","last_undegraded":"2024-05-22T21:15:43.961665+0000","last_fullsized":"2024-05-22T21:15:43.961665+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:57.417032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961760+0000","last_change":"2024-05-22T21:15:42.971637+0000","last_active":"2024-05-22T21:15:43.961760+0000","last_peered":"2024-05-22T21:15:43.961760+0000","last_clean":"2024-05-22T21:15:43.961760+0000","last_became_active":"2024-05-22T21:15:42.971367+0000","last_became_peered":"2024-05-22T21:15:42.971367+0000","last_unstale":"2024-05-22T21:15:43.961760+0000","last_undegraded":"2024-05-22T21:15:43.961760+0000","last_fullsized":"2024-05-22T21:15:43.961760+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:42:29.381771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401300+0000","last_change":"2024-05-22T21:15:42.979177+0000","last_active":"2024-05-22T21:15:44.401300+0000","last_peered":"2024-05-22T21:15:44.401300+0000","last_clean":"2024-05-22T21:15:44.401300+0000","last_became_active":"2024-05-22T21:15:42.975992+0000","last_became_peered":"2024-05-22T21:15:42.975992+0000","last_unstale":"2024-05-22T21:15:44.401300+0000","last_undegraded":"2024-05-22T21:15:44.401300+0000","last_fullsized":"2024-05-22T21:15:44.401300+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:05:51.541715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.063832+0000","last_change":"2024-05-22T21:15:42.972885+0000","last_active":"2024-05-22T21:15:44.063832+0000","last_peered":"2024-05-22T21:15:44.063832+0000","last_clean":"2024-05-22T21:15:44.063832+0000","last_became_active":"2024-05-22T21:15:42.972674+0000","last_became_peered":"2024-05-22T21:15:42.972674+0000","last_unstale":"2024-05-22T21:15:44.063832+0000","last_undegraded":"2024-05-22T21:15:44.063832+0000","last_fullsized":"2024-05-22T21:15:44.063832+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:58:53.055471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963069+0000","last_change":"2024-05-22T21:15:42.975350+0000","last_active":"2024-05-22T21:15:43.963069+0000","last_peered":"2024-05-22T21:15:43.963069+0000","last_clean":"2024-05-22T21:15:43.963069+0000","last_became_active":"2024-05-22T21:15:42.975046+0000","last_became_peered":"2024-05-22T21:15:42.975046+0000","last_unstale":"2024-05-22T21:15:43.963069+0000","last_undegraded":"2024-05-22T21:15:43.963069+0000","last_fullsized":"2024-05-22T21:15:43.963069+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:29:47.821900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963408+0000","last_change":"2024-05-22T21:15:42.971633+0000","last_active":"2024-05-22T21:15:43.963408+0000","last_peered":"2024-05-22T21:15:43.963408+0000","last_clean":"2024-05-22T21:15:43.963408+0000","last_became_active":"2024-05-22T21:15:42.971356+0000","last_became_peered":"2024-05-22T21:15:42.971356+0000","last_unstale":"2024-05-22T21:15:43.963408+0000","last_undegraded":"2024-05-22T21:15:43.963408+0000","last_fullsized":"2024-05-22T21:15:43.963408+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:06:28.343465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.949089+0000","last_change":"2024-05-22T21:15:42.972704+0000","last_active":"2024-05-22T21:15:44.949089+0000","last_peered":"2024-05-22T21:15:44.949089+0000","last_clean":"2024-05-22T21:15:44.949089+0000","last_became_active":"2024-05-22T21:15:42.972499+0000","last_became_peered":"2024-05-22T21:15:42.972499+0000","last_unstale":"2024-05-22T21:15:44.949089+0000","last_undegraded":"2024-05-22T21:15:44.949089+0000","last_fullsized":"2024-05-22T21:15:44.949089+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:51:38.084205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064209+0000","last_change":"2024-05-22T21:15:42.976478+0000","last_active":"2024-05-22T21:15:44.064209+0000","last_peered":"2024-05-22T21:15:44.064209+0000","last_clean":"2024-05-22T21:15:44.064209+0000","last_became_active":"2024-05-22T21:15:42.976030+0000","last_became_peered":"2024-05-22T21:15:42.976030+0000","last_unstale":"2024-05-22T21:15:44.064209+0000","last_undegraded":"2024-05-22T21:15:44.064209+0000","last_fullsized":"2024-05-22T21:15:44.064209+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:04:10.450613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.962834+0000","last_change":"2024-05-22T21:15:42.976942+0000","last_active":"2024-05-22T21:15:43.962834+0000","last_peered":"2024-05-22T21:15:43.962834+0000","last_clean":"2024-05-22T21:15:43.962834+0000","last_became_active":"2024-05-22T21:15:42.974044+0000","last_became_peered":"2024-05-22T21:15:42.974044+0000","last_unstale":"2024-05-22T21:15:43.962834+0000","last_undegraded":"2024-05-22T21:15:43.962834+0000","last_fullsized":"2024-05-22T21:15:43.962834+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:51:38.084205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.400815+0000","last_change":"2024-05-22T21:15:42.974605+0000","last_active":"2024-05-22T21:15:44.400815+0000","last_peered":"2024-05-22T21:15:44.400815+0000","last_clean":"2024-05-22T21:15:44.400815+0000","last_became_active":"2024-05-22T21:15:42.974219+0000","last_became_peered":"2024-05-22T21:15:42.974219+0000","last_unstale":"2024-05-22T21:15:44.400815+0000","last_undegraded":"2024-05-22T21:15:44.400815+0000","last_fullsized":"2024-05-22T21:15:44.400815+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:02:06.403145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064233+0000","last_change":"2024-05-22T21:15:42.968120+0000","last_active":"2024-05-22T21:15:44.064233+0000","last_peered":"2024-05-22T21:15:44.064233+0000","last_clean":"2024-05-22T21:15:44.064233+0000","last_became_active":"2024-05-22T21:15:42.967910+0000","last_became_peered":"2024-05-22T21:15:42.967910+0000","last_unstale":"2024-05-22T21:15:44.064233+0000","last_undegraded":"2024-05-22T21:15:44.064233+0000","last_fullsized":"2024-05-22T21:15:44.064233+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:22:27.221021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963372+0000","last_change":"2024-05-22T21:15:42.975526+0000","last_active":"2024-05-22T21:15:43.963372+0000","last_peered":"2024-05-22T21:15:43.963372+0000","last_clean":"2024-05-22T21:15:43.963372+0000","last_became_active":"2024-05-22T21:15:42.975125+0000","last_became_peered":"2024-05-22T21:15:42.975125+0000","last_unstale":"2024-05-22T21:15:43.963372+0000","last_undegraded":"2024-05-22T21:15:43.963372+0000","last_fullsized":"2024-05-22T21:15:43.963372+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:05:51.541715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064314+0000","last_change":"2024-05-22T21:15:42.975533+0000","last_active":"2024-05-22T21:15:44.064314+0000","last_peered":"2024-05-22T21:15:44.064314+0000","last_clean":"2024-05-22T21:15:44.064314+0000","last_became_active":"2024-05-22T21:15:42.971033+0000","last_became_peered":"2024-05-22T21:15:42.971033+0000","last_unstale":"2024-05-22T21:15:44.064314+0000","last_undegraded":"2024-05-22T21:15:44.064314+0000","last_fullsized":"2024-05-22T21:15:44.064314+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:50:08.534446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.962930+0000","last_change":"2024-05-22T21:15:42.976645+0000","last_active":"2024-05-22T21:15:43.962930+0000","last_peered":"2024-05-22T21:15:43.962930+0000","last_clean":"2024-05-22T21:15:43.962930+0000","last_became_active":"2024-05-22T21:15:42.973393+0000","last_became_peered":"2024-05-22T21:15:42.973393+0000","last_unstale":"2024-05-22T21:15:43.962930+0000","last_undegraded":"2024-05-22T21:15:43.962930+0000","last_fullsized":"2024-05-22T21:15:43.962930+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:38:55.292112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.063897+0000","last_change":"2024-05-22T21:15:42.971878+0000","last_active":"2024-05-22T21:15:44.063897+0000","last_peered":"2024-05-22T21:15:44.063897+0000","last_clean":"2024-05-22T21:15:44.063897+0000","last_became_active":"2024-05-22T21:15:42.971633+0000","last_became_peered":"2024-05-22T21:15:42.971633+0000","last_unstale":"2024-05-22T21:15:44.063897+0000","last_undegraded":"2024-05-22T21:15:44.063897+0000","last_fullsized":"2024-05-22T21:15:44.063897+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:59:38.564543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.400891+0000","last_change":"2024-05-22T21:15:42.971830+0000","last_active":"2024-05-22T21:15:44.400891+0000","last_peered":"2024-05-22T21:15:44.400891+0000","last_clean":"2024-05-22T21:15:44.400891+0000","last_became_active":"2024-05-22T21:15:42.971545+0000","last_became_peered":"2024-05-22T21:15:42.971545+0000","last_unstale":"2024-05-22T21:15:44.400891+0000","last_undegraded":"2024-05-22T21:15:44.400891+0000","last_fullsized":"2024-05-22T21:15:44.400891+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:40:46.992971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401459+0000","last_change":"2024-05-22T21:15:42.972108+0000","last_active":"2024-05-22T21:15:44.401459+0000","last_peered":"2024-05-22T21:15:44.401459+0000","last_clean":"2024-05-22T21:15:44.401459+0000","last_became_active":"2024-05-22T21:15:42.971866+0000","last_became_peered":"2024-05-22T21:15:42.971866+0000","last_unstale":"2024-05-22T21:15:44.401459+0000","last_undegraded":"2024-05-22T21:15:44.401459+0000","last_fullsized":"2024-05-22T21:15:44.401459+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:49:33.826508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948696+0000","last_change":"2024-05-22T21:15:42.972272+0000","last_active":"2024-05-22T21:15:44.948696+0000","last_peered":"2024-05-22T21:15:44.948696+0000","last_clean":"2024-05-22T21:15:44.948696+0000","last_became_active":"2024-05-22T21:15:42.972050+0000","last_became_peered":"2024-05-22T21:15:42.972050+0000","last_unstale":"2024-05-22T21:15:44.948696+0000","last_undegraded":"2024-05-22T21:15:44.948696+0000","last_fullsized":"2024-05-22T21:15:44.948696+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:12:02.773432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.400796+0000","last_change":"2024-05-22T21:15:42.976992+0000","last_active":"2024-05-22T21:15:44.400796+0000","last_peered":"2024-05-22T21:15:44.400796+0000","last_clean":"2024-05-22T21:15:44.400796+0000","last_became_active":"2024-05-22T21:15:42.974819+0000","last_became_peered":"2024-05-22T21:15:42.974819+0000","last_unstale":"2024-05-22T21:15:44.400796+0000","last_undegraded":"2024-05-22T21:15:44.400796+0000","last_fullsized":"2024-05-22T21:15:44.400796+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:47:26.596903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064135+0000","last_change":"2024-05-22T21:15:42.979255+0000","last_active":"2024-05-22T21:15:44.064135+0000","last_peered":"2024-05-22T21:15:44.064135+0000","last_clean":"2024-05-22T21:15:44.064135+0000","last_became_active":"2024-05-22T21:15:42.978671+0000","last_became_peered":"2024-05-22T21:15:42.978671+0000","last_unstale":"2024-05-22T21:15:44.064135+0000","last_undegraded":"2024-05-22T21:15:44.064135+0000","last_fullsized":"2024-05-22T21:15:44.064135+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:01:30.502315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963508+0000","last_change":"2024-05-22T21:15:42.970791+0000","last_active":"2024-05-22T21:15:43.963508+0000","last_peered":"2024-05-22T21:15:43.963508+0000","last_clean":"2024-05-22T21:15:43.963508+0000","last_became_active":"2024-05-22T21:15:42.970565+0000","last_became_peered":"2024-05-22T21:15:42.970565+0000","last_unstale":"2024-05-22T21:15:43.963508+0000","last_undegraded":"2024-05-22T21:15:43.963508+0000","last_fullsized":"2024-05-22T21:15:43.963508+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:26:00.521024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.400917+0000","last_change":"2024-05-22T21:15:42.975667+0000","last_active":"2024-05-22T21:15:44.400917+0000","last_peered":"2024-05-22T21:15:44.400917+0000","last_clean":"2024-05-22T21:15:44.400917+0000","last_became_active":"2024-05-22T21:15:42.975439+0000","last_became_peered":"2024-05-22T21:15:42.975439+0000","last_unstale":"2024-05-22T21:15:44.400917+0000","last_undegraded":"2024-05-22T21:15:44.400917+0000","last_fullsized":"2024-05-22T21:15:44.400917+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:48:42.519585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948792+0000","last_change":"2024-05-22T21:15:42.968273+0000","last_active":"2024-05-22T21:15:44.948792+0000","last_peered":"2024-05-22T21:15:44.948792+0000","last_clean":"2024-05-22T21:15:44.948792+0000","last_became_active":"2024-05-22T21:15:42.968069+0000","last_became_peered":"2024-05-22T21:15:42.968069+0000","last_unstale":"2024-05-22T21:15:44.948792+0000","last_undegraded":"2024-05-22T21:15:44.948792+0000","last_fullsized":"2024-05-22T21:15:44.948792+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:20.645717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948837+0000","last_change":"2024-05-22T21:15:42.967224+0000","last_active":"2024-05-22T21:15:44.948837+0000","last_peered":"2024-05-22T21:15:44.948837+0000","last_clean":"2024-05-22T21:15:44.948837+0000","last_became_active":"2024-05-22T21:15:42.967014+0000","last_became_peered":"2024-05-22T21:15:42.967014+0000","last_unstale":"2024-05-22T21:15:44.948837+0000","last_undegraded":"2024-05-22T21:15:44.948837+0000","last_fullsized":"2024-05-22T21:15:44.948837+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:52:50.421947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"66'4","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.949079+0000","last_change":"2024-05-22T21:15:42.579540+0000","last_active":"2024-05-22T21:15:44.949079+0000","last_peered":"2024-05-22T21:15:44.949079+0000","last_clean":"2024-05-22T21:15:44.949079+0000","last_became_active":"2024-05-22T21:15:42.579314+0000","last_became_peered":"2024-05-22T21:15:42.579314+0000","last_unstale":"2024-05-22T21:15:44.949079+0000","last_undegraded":"2024-05-22T21:15:44.949079+0000","last_fullsized":"2024-05-22T21:15:44.949079+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:41:22.905870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961480+0000","last_change":"2024-05-22T21:15:42.976269+0000","last_active":"2024-05-22T21:15:43.961480+0000","last_peered":"2024-05-22T21:15:43.961480+0000","last_clean":"2024-05-22T21:15:43.961480+0000","last_became_active":"2024-05-22T21:15:42.975974+0000","last_became_peered":"2024-05-22T21:15:42.975974+0000","last_unstale":"2024-05-22T21:15:43.961480+0000","last_undegraded":"2024-05-22T21:15:43.961480+0000","last_fullsized":"2024-05-22T21:15:43.961480+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:20:24.411848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"66'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064265+0000","last_change":"2024-05-22T21:15:42.575027+0000","last_active":"2024-05-22T21:15:44.064265+0000","last_peered":"2024-05-22T21:15:44.064265+0000","last_clean":"2024-05-22T21:15:44.064265+0000","last_became_active":"2024-05-22T21:15:42.574819+0000","last_became_peered":"2024-05-22T21:15:42.574819+0000","last_unstale":"2024-05-22T21:15:44.064265+0000","last_undegraded":"2024-05-22T21:15:44.064265+0000","last_fullsized":"2024-05-22T21:15:44.064265+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:14:59.935734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948991+0000","last_change":"2024-05-22T21:15:42.965607+0000","last_active":"2024-05-22T21:15:44.948991+0000","last_peered":"2024-05-22T21:15:44.948991+0000","last_clean":"2024-05-22T21:15:44.948991+0000","last_became_active":"2024-05-22T21:15:42.965336+0000","last_became_peered":"2024-05-22T21:15:42.965336+0000","last_unstale":"2024-05-22T21:15:44.948991+0000","last_undegraded":"2024-05-22T21:15:44.948991+0000","last_fullsized":"2024-05-22T21:15:44.948991+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:07:20.924146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"66'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.949089+0000","last_change":"2024-05-22T21:15:42.584140+0000","last_active":"2024-05-22T21:15:44.949089+0000","last_peered":"2024-05-22T21:15:44.949089+0000","last_clean":"2024-05-22T21:15:44.949089+0000","last_became_active":"2024-05-22T21:15:42.581462+0000","last_became_peered":"2024-05-22T21:15:42.581462+0000","last_unstale":"2024-05-22T21:15:44.949089+0000","last_undegraded":"2024-05-22T21:15:44.949089+0000","last_fullsized":"2024-05-22T21:15:44.949089+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:07.614742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948875+0000","last_change":"2024-05-22T21:15:42.977758+0000","last_active":"2024-05-22T21:15:44.948875+0000","last_peered":"2024-05-22T21:15:44.948875+0000","last_clean":"2024-05-22T21:15:44.948875+0000","last_became_active":"2024-05-22T21:15:42.977305+0000","last_became_peered":"2024-05-22T21:15:42.977305+0000","last_unstale":"2024-05-22T21:15:44.948875+0000","last_undegraded":"2024-05-22T21:15:44.948875+0000","last_fullsized":"2024-05-22T21:15:44.948875+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:59:05.276396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"66'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401639+0000","last_change":"2024-05-22T21:15:42.584129+0000","last_active":"2024-05-22T21:15:44.401639+0000","last_peered":"2024-05-22T21:15:44.401639+0000","last_clean":"2024-05-22T21:15:44.401639+0000","last_became_active":"2024-05-22T21:15:42.583742+0000","last_became_peered":"2024-05-22T21:15:42.583742+0000","last_unstale":"2024-05-22T21:15:44.401639+0000","last_undegraded":"2024-05-22T21:15:44.401639+0000","last_fullsized":"2024-05-22T21:15:44.401639+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:53:23.784884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064341+0000","last_change":"2024-05-22T21:15:42.979443+0000","last_active":"2024-05-22T21:15:44.064341+0000","last_peered":"2024-05-22T21:15:44.064341+0000","last_clean":"2024-05-22T21:15:44.064341+0000","last_became_active":"2024-05-22T21:15:42.978956+0000","last_became_peered":"2024-05-22T21:15:42.978956+0000","last_unstale":"2024-05-22T21:15:44.064341+0000","last_undegraded":"2024-05-22T21:15:44.064341+0000","last_fullsized":"2024-05-22T21:15:44.064341+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:40:34.106010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"66'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963171+0000","last_change":"2024-05-22T21:15:42.578770+0000","last_active":"2024-05-22T21:15:43.963171+0000","last_peered":"2024-05-22T21:15:43.963171+0000","last_clean":"2024-05-22T21:15:43.963171+0000","last_became_active":"2024-05-22T21:15:42.578561+0000","last_became_peered":"2024-05-22T21:15:42.578561+0000","last_unstale":"2024-05-22T21:15:43.963171+0000","last_undegraded":"2024-05-22T21:15:43.963171+0000","last_fullsized":"2024-05-22T21:15:43.963171+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:16:53.893078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064418+0000","last_change":"2024-05-22T21:15:42.976356+0000","last_active":"2024-05-22T21:15:44.064418+0000","last_peered":"2024-05-22T21:15:44.064418+0000","last_clean":"2024-05-22T21:15:44.064418+0000","last_became_active":"2024-05-22T21:15:42.975730+0000","last_became_peered":"2024-05-22T21:15:42.975730+0000","last_unstale":"2024-05-22T21:15:44.064418+0000","last_undegraded":"2024-05-22T21:15:44.064418+0000","last_fullsized":"2024-05-22T21:15:44.064418+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:28:33.939764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"66'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.949175+0000","last_change":"2024-05-22T21:15:42.583201+0000","last_active":"2024-05-22T21:15:44.949175+0000","last_peered":"2024-05-22T21:15:44.949175+0000","last_clean":"2024-05-22T21:15:44.949175+0000","last_became_active":"2024-05-22T21:15:42.582592+0000","last_became_peered":"2024-05-22T21:15:42.582592+0000","last_unstale":"2024-05-22T21:15:44.949175+0000","last_undegraded":"2024-05-22T21:15:44.949175+0000","last_fullsized":"2024-05-22T21:15:44.949175+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:53:23.784884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.063905+0000","last_change":"2024-05-22T21:15:42.976275+0000","last_active":"2024-05-22T21:15:44.063905+0000","last_peered":"2024-05-22T21:15:44.063905+0000","last_clean":"2024-05-22T21:15:44.063905+0000","last_became_active":"2024-05-22T21:15:42.976060+0000","last_became_peered":"2024-05-22T21:15:42.976060+0000","last_unstale":"2024-05-22T21:15:44.063905+0000","last_undegraded":"2024-05-22T21:15:44.063905+0000","last_fullsized":"2024-05-22T21:15:44.063905+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:52:19.703848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401754+0000","last_change":"2024-05-22T21:15:42.585086+0000","last_active":"2024-05-22T21:15:44.401754+0000","last_peered":"2024-05-22T21:15:44.401754+0000","last_clean":"2024-05-22T21:15:44.401754+0000","last_became_active":"2024-05-22T21:15:42.584866+0000","last_became_peered":"2024-05-22T21:15:42.584866+0000","last_unstale":"2024-05-22T21:15:44.401754+0000","last_undegraded":"2024-05-22T21:15:44.401754+0000","last_fullsized":"2024-05-22T21:15:44.401754+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:41:22.905870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064202+0000","last_change":"2024-05-22T21:15:42.979117+0000","last_active":"2024-05-22T21:15:44.064202+0000","last_peered":"2024-05-22T21:15:44.064202+0000","last_clean":"2024-05-22T21:15:44.064202+0000","last_became_active":"2024-05-22T21:15:42.978338+0000","last_became_peered":"2024-05-22T21:15:42.978338+0000","last_unstale":"2024-05-22T21:15:44.064202+0000","last_undegraded":"2024-05-22T21:15:44.064202+0000","last_fullsized":"2024-05-22T21:15:44.064202+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:00:49.522445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963501+0000","last_change":"2024-05-22T21:15:42.574542+0000","last_active":"2024-05-22T21:15:43.963501+0000","last_peered":"2024-05-22T21:15:43.963501+0000","last_clean":"2024-05-22T21:15:43.963501+0000","last_became_active":"2024-05-22T21:15:42.574265+0000","last_became_peered":"2024-05-22T21:15:42.574265+0000","last_unstale":"2024-05-22T21:15:43.963501+0000","last_undegraded":"2024-05-22T21:15:43.963501+0000","last_fullsized":"2024-05-22T21:15:43.963501+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:59:37.557337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963086+0000","last_change":"2024-05-22T21:15:42.976007+0000","last_active":"2024-05-22T21:15:43.963086+0000","last_peered":"2024-05-22T21:15:43.963086+0000","last_clean":"2024-05-22T21:15:43.963086+0000","last_became_active":"2024-05-22T21:15:42.975720+0000","last_became_peered":"2024-05-22T21:15:42.975720+0000","last_unstale":"2024-05-22T21:15:43.963086+0000","last_undegraded":"2024-05-22T21:15:43.963086+0000","last_fullsized":"2024-05-22T21:15:43.963086+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:20:33.620075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963765+0000","last_change":"2024-05-22T21:15:42.574156+0000","last_active":"2024-05-22T21:15:43.963765+0000","last_peered":"2024-05-22T21:15:43.963765+0000","last_clean":"2024-05-22T21:15:43.963765+0000","last_became_active":"2024-05-22T21:15:42.573379+0000","last_became_peered":"2024-05-22T21:15:42.573379+0000","last_unstale":"2024-05-22T21:15:43.963765+0000","last_undegraded":"2024-05-22T21:15:43.963765+0000","last_fullsized":"2024-05-22T21:15:43.963765+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:53:01.070200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963452+0000","last_change":"2024-05-22T21:15:42.970401+0000","last_active":"2024-05-22T21:15:43.963452+0000","last_peered":"2024-05-22T21:15:43.963452+0000","last_clean":"2024-05-22T21:15:43.963452+0000","last_became_active":"2024-05-22T21:15:42.969670+0000","last_became_peered":"2024-05-22T21:15:42.969670+0000","last_unstale":"2024-05-22T21:15:43.963452+0000","last_undegraded":"2024-05-22T21:15:43.963452+0000","last_fullsized":"2024-05-22T21:15:43.963452+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:07:20.924146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064681+0000","last_change":"2024-05-22T21:15:42.587422+0000","last_active":"2024-05-22T21:15:44.064681+0000","last_peered":"2024-05-22T21:15:44.064681+0000","last_clean":"2024-05-22T21:15:44.064681+0000","last_became_active":"2024-05-22T21:15:42.583942+0000","last_became_peered":"2024-05-22T21:15:42.583942+0000","last_unstale":"2024-05-22T21:15:44.064681+0000","last_undegraded":"2024-05-22T21:15:44.064681+0000","last_fullsized":"2024-05-22T21:15:44.064681+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:40:00.827380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961655+0000","last_change":"2024-05-22T21:15:42.977404+0000","last_active":"2024-05-22T21:15:43.961655+0000","last_peered":"2024-05-22T21:15:43.961655+0000","last_clean":"2024-05-22T21:15:43.961655+0000","last_became_active":"2024-05-22T21:15:42.976868+0000","last_became_peered":"2024-05-22T21:15:42.976868+0000","last_unstale":"2024-05-22T21:15:43.961655+0000","last_undegraded":"2024-05-22T21:15:43.961655+0000","last_fullsized":"2024-05-22T21:15:43.961655+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:53:12.636434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"66'3","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963113+0000","last_change":"2024-05-22T21:15:42.582185+0000","last_active":"2024-05-22T21:15:43.963113+0000","last_peered":"2024-05-22T21:15:43.963113+0000","last_clean":"2024-05-22T21:15:43.963113+0000","last_became_active":"2024-05-22T21:15:42.581935+0000","last_became_peered":"2024-05-22T21:15:42.581935+0000","last_unstale":"2024-05-22T21:15:43.963113+0000","last_undegraded":"2024-05-22T21:15:43.963113+0000","last_fullsized":"2024-05-22T21:15:43.963113+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:09:38.871529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963030+0000","last_change":"2024-05-22T21:15:42.976794+0000","last_active":"2024-05-22T21:15:43.963030+0000","last_peered":"2024-05-22T21:15:43.963030+0000","last_clean":"2024-05-22T21:15:43.963030+0000","last_became_active":"2024-05-22T21:15:42.973696+0000","last_became_peered":"2024-05-22T21:15:42.973696+0000","last_unstale":"2024-05-22T21:15:43.963030+0000","last_undegraded":"2024-05-22T21:15:43.963030+0000","last_fullsized":"2024-05-22T21:15:43.963030+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:29:50.850273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"66'6","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963168+0000","last_change":"2024-05-22T21:15:42.585370+0000","last_active":"2024-05-22T21:15:43.963168+0000","last_peered":"2024-05-22T21:15:43.963168+0000","last_clean":"2024-05-22T21:15:43.963168+0000","last_became_active":"2024-05-22T21:15:42.584788+0000","last_became_peered":"2024-05-22T21:15:42.584788+0000","last_unstale":"2024-05-22T21:15:43.963168+0000","last_undegraded":"2024-05-22T21:15:43.963168+0000","last_fullsized":"2024-05-22T21:15:43.963168+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:52:18.696642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401560+0000","last_change":"2024-05-22T21:15:42.974952+0000","last_active":"2024-05-22T21:15:44.401560+0000","last_peered":"2024-05-22T21:15:44.401560+0000","last_clean":"2024-05-22T21:15:44.401560+0000","last_became_active":"2024-05-22T21:15:42.974712+0000","last_became_peered":"2024-05-22T21:15:42.974712+0000","last_unstale":"2024-05-22T21:15:44.401560+0000","last_undegraded":"2024-05-22T21:15:44.401560+0000","last_fullsized":"2024-05-22T21:15:44.401560+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:29:50.850273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"66'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064386+0000","last_change":"2024-05-22T21:15:42.581241+0000","last_active":"2024-05-22T21:15:44.064386+0000","last_peered":"2024-05-22T21:15:44.064386+0000","last_clean":"2024-05-22T21:15:44.064386+0000","last_became_active":"2024-05-22T21:15:42.581020+0000","last_became_peered":"2024-05-22T21:15:42.581020+0000","last_unstale":"2024-05-22T21:15:44.064386+0000","last_undegraded":"2024-05-22T21:15:44.064386+0000","last_fullsized":"2024-05-22T21:15:44.064386+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:46:39.350179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963668+0000","last_change":"2024-05-22T21:15:42.975321+0000","last_active":"2024-05-22T21:15:43.963668+0000","last_peered":"2024-05-22T21:15:43.963668+0000","last_clean":"2024-05-22T21:15:43.963668+0000","last_became_active":"2024-05-22T21:15:42.974830+0000","last_became_peered":"2024-05-22T21:15:42.974830+0000","last_unstale":"2024-05-22T21:15:43.963668+0000","last_undegraded":"2024-05-22T21:15:43.963668+0000","last_fullsized":"2024-05-22T21:15:43.963668+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:55.539734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.063802+0000","last_change":"2024-05-22T21:15:42.583656+0000","last_active":"2024-05-22T21:15:44.063802+0000","last_peered":"2024-05-22T21:15:44.063802+0000","last_clean":"2024-05-22T21:15:44.063802+0000","last_became_active":"2024-05-22T21:15:42.583439+0000","last_became_peered":"2024-05-22T21:15:42.583439+0000","last_unstale":"2024-05-22T21:15:44.063802+0000","last_undegraded":"2024-05-22T21:15:44.063802+0000","last_fullsized":"2024-05-22T21:15:44.063802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:39.462116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963582+0000","last_change":"2024-05-22T21:15:42.977201+0000","last_active":"2024-05-22T21:15:43.963582+0000","last_peered":"2024-05-22T21:15:43.963582+0000","last_clean":"2024-05-22T21:15:43.963582+0000","last_became_active":"2024-05-22T21:15:42.975235+0000","last_became_peered":"2024-05-22T21:15:42.975235+0000","last_unstale":"2024-05-22T21:15:43.963582+0000","last_undegraded":"2024-05-22T21:15:43.963582+0000","last_fullsized":"2024-05-22T21:15:43.963582+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:41:23.913076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401489+0000","last_change":"2024-05-22T21:15:42.580711+0000","last_active":"2024-05-22T21:15:44.401489+0000","last_peered":"2024-05-22T21:15:44.401489+0000","last_clean":"2024-05-22T21:15:44.401489+0000","last_became_active":"2024-05-22T21:15:42.579788+0000","last_became_peered":"2024-05-22T21:15:42.579788+0000","last_unstale":"2024-05-22T21:15:44.401489+0000","last_undegraded":"2024-05-22T21:15:44.401489+0000","last_fullsized":"2024-05-22T21:15:44.401489+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:29:46.814694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963149+0000","last_change":"2024-05-22T21:15:42.973979+0000","last_active":"2024-05-22T21:15:43.963149+0000","last_peered":"2024-05-22T21:15:43.963149+0000","last_clean":"2024-05-22T21:15:43.963149+0000","last_became_active":"2024-05-22T21:15:42.973733+0000","last_became_peered":"2024-05-22T21:15:42.973733+0000","last_unstale":"2024-05-22T21:15:43.963149+0000","last_undegraded":"2024-05-22T21:15:43.963149+0000","last_fullsized":"2024-05-22T21:15:43.963149+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:17:12.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"66'8","reported_seq":19,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963199+0000","last_change":"2024-05-22T21:15:42.579084+0000","last_active":"2024-05-22T21:15:43.963199+0000","last_peered":"2024-05-22T21:15:43.963199+0000","last_clean":"2024-05-22T21:15:43.963199+0000","last_became_active":"2024-05-22T21:15:42.578629+0000","last_became_peered":"2024-05-22T21:15:42.578629+0000","last_unstale":"2024-05-22T21:15:43.963199+0000","last_undegraded":"2024-05-22T21:15:43.963199+0000","last_fullsized":"2024-05-22T21:15:43.963199+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:42:28.374565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963755+0000","last_change":"2024-05-22T21:15:42.975149+0000","last_active":"2024-05-22T21:15:43.963755+0000","last_peered":"2024-05-22T21:15:43.963755+0000","last_clean":"2024-05-22T21:15:43.963755+0000","last_became_active":"2024-05-22T21:15:42.974473+0000","last_became_peered":"2024-05-22T21:15:42.974473+0000","last_unstale":"2024-05-22T21:15:43.963755+0000","last_undegraded":"2024-05-22T21:15:43.963755+0000","last_fullsized":"2024-05-22T21:15:43.963755+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:02.614285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"66'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064320+0000","last_change":"2024-05-22T21:15:42.582766+0000","last_active":"2024-05-22T21:15:44.064320+0000","last_peered":"2024-05-22T21:15:44.064320+0000","last_clean":"2024-05-22T21:15:44.064320+0000","last_became_active":"2024-05-22T21:15:42.582036+0000","last_became_peered":"2024-05-22T21:15:42.582036+0000","last_unstale":"2024-05-22T21:15:44.064320+0000","last_undegraded":"2024-05-22T21:15:44.064320+0000","last_fullsized":"2024-05-22T21:15:44.064320+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:25:59.513818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401494+0000","last_change":"2024-05-22T21:15:42.974415+0000","last_active":"2024-05-22T21:15:44.401494+0000","last_peered":"2024-05-22T21:15:44.401494+0000","last_clean":"2024-05-22T21:15:44.401494+0000","last_became_active":"2024-05-22T21:15:42.973742+0000","last_became_peered":"2024-05-22T21:15:42.973742+0000","last_unstale":"2024-05-22T21:15:44.401494+0000","last_undegraded":"2024-05-22T21:15:44.401494+0000","last_fullsized":"2024-05-22T21:15:44.401494+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:52:50.421947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"66'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.949180+0000","last_change":"2024-05-22T21:15:42.584480+0000","last_active":"2024-05-22T21:15:44.949180+0000","last_peered":"2024-05-22T21:15:44.949180+0000","last_clean":"2024-05-22T21:15:44.949180+0000","last_became_active":"2024-05-22T21:15:42.581781+0000","last_became_peered":"2024-05-22T21:15:42.581781+0000","last_unstale":"2024-05-22T21:15:44.949180+0000","last_undegraded":"2024-05-22T21:15:44.949180+0000","last_fullsized":"2024-05-22T21:15:44.949180+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:20:32.612869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064479+0000","last_change":"2024-05-22T21:15:42.976906+0000","last_active":"2024-05-22T21:15:44.064479+0000","last_peered":"2024-05-22T21:15:44.064479+0000","last_clean":"2024-05-22T21:15:44.064479+0000","last_became_active":"2024-05-22T21:15:42.976700+0000","last_became_peered":"2024-05-22T21:15:42.976700+0000","last_unstale":"2024-05-22T21:15:44.064479+0000","last_undegraded":"2024-05-22T21:15:44.064479+0000","last_fullsized":"2024-05-22T21:15:44.064479+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:33:48.422387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"66'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401690+0000","last_change":"2024-05-22T21:15:42.583988+0000","last_active":"2024-05-22T21:15:44.401690+0000","last_peered":"2024-05-22T21:15:44.401690+0000","last_clean":"2024-05-22T21:15:44.401690+0000","last_became_active":"2024-05-22T21:15:42.583426+0000","last_became_peered":"2024-05-22T21:15:42.583426+0000","last_unstale":"2024-05-22T21:15:44.401690+0000","last_undegraded":"2024-05-22T21:15:44.401690+0000","last_fullsized":"2024-05-22T21:15:44.401690+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:07.614742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963537+0000","last_change":"2024-05-22T21:15:42.970557+0000","last_active":"2024-05-22T21:15:43.963537+0000","last_peered":"2024-05-22T21:15:43.963537+0000","last_clean":"2024-05-22T21:15:43.963537+0000","last_became_active":"2024-05-22T21:15:42.969968+0000","last_became_peered":"2024-05-22T21:15:42.969968+0000","last_unstale":"2024-05-22T21:15:43.963537+0000","last_undegraded":"2024-05-22T21:15:43.963537+0000","last_fullsized":"2024-05-22T21:15:43.963537+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:52:50.421947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948920+0000","last_change":"2024-05-22T21:15:42.574470+0000","last_active":"2024-05-22T21:15:44.948920+0000","last_peered":"2024-05-22T21:15:44.948920+0000","last_clean":"2024-05-22T21:15:44.948920+0000","last_became_active":"2024-05-22T21:15:42.574189+0000","last_became_peered":"2024-05-22T21:15:42.574189+0000","last_unstale":"2024-05-22T21:15:44.948920+0000","last_undegraded":"2024-05-22T21:15:44.948920+0000","last_fullsized":"2024-05-22T21:15:44.948920+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:29:46.814694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401672+0000","last_change":"2024-05-22T21:15:42.974200+0000","last_active":"2024-05-22T21:15:44.401672+0000","last_peered":"2024-05-22T21:15:44.401672+0000","last_clean":"2024-05-22T21:15:44.401672+0000","last_became_active":"2024-05-22T21:15:42.973452+0000","last_became_peered":"2024-05-22T21:15:42.973452+0000","last_unstale":"2024-05-22T21:15:44.401672+0000","last_undegraded":"2024-05-22T21:15:44.401672+0000","last_fullsized":"2024-05-22T21:15:44.401672+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:51:38.084205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"66'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064427+0000","last_change":"2024-05-22T21:15:42.580675+0000","last_active":"2024-05-22T21:15:44.064427+0000","last_peered":"2024-05-22T21:15:44.064427+0000","last_clean":"2024-05-22T21:15:44.064427+0000","last_became_active":"2024-05-22T21:15:42.578358+0000","last_became_peered":"2024-05-22T21:15:42.578358+0000","last_unstale":"2024-05-22T21:15:44.064427+0000","last_undegraded":"2024-05-22T21:15:44.064427+0000","last_fullsized":"2024-05-22T21:15:44.064427+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:38:54.284906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948970+0000","last_change":"2024-05-22T21:15:42.971558+0000","last_active":"2024-05-22T21:15:44.948970+0000","last_peered":"2024-05-22T21:15:44.948970+0000","last_clean":"2024-05-22T21:15:44.948970+0000","last_became_active":"2024-05-22T21:15:42.971346+0000","last_became_peered":"2024-05-22T21:15:42.971346+0000","last_unstale":"2024-05-22T21:15:44.948970+0000","last_undegraded":"2024-05-22T21:15:44.948970+0000","last_fullsized":"2024-05-22T21:15:44.948970+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:55.539734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961809+0000","last_change":"2024-05-22T21:15:42.581482+0000","last_active":"2024-05-22T21:15:43.961809+0000","last_peered":"2024-05-22T21:15:43.961809+0000","last_clean":"2024-05-22T21:15:43.961809+0000","last_became_active":"2024-05-22T21:15:42.581106+0000","last_became_peered":"2024-05-22T21:15:42.581106+0000","last_unstale":"2024-05-22T21:15:43.961809+0000","last_undegraded":"2024-05-22T21:15:43.961809+0000","last_fullsized":"2024-05-22T21:15:43.961809+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:48:28.713342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948998+0000","last_change":"2024-05-22T21:15:42.966903+0000","last_active":"2024-05-22T21:15:44.948998+0000","last_peered":"2024-05-22T21:15:44.948998+0000","last_clean":"2024-05-22T21:15:44.948998+0000","last_became_active":"2024-05-22T21:15:42.966665+0000","last_became_peered":"2024-05-22T21:15:42.966665+0000","last_unstale":"2024-05-22T21:15:44.948998+0000","last_undegraded":"2024-05-22T21:15:44.948998+0000","last_fullsized":"2024-05-22T21:15:44.948998+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:49:33.826508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401740+0000","last_change":"2024-05-22T21:15:42.582430+0000","last_active":"2024-05-22T21:15:44.401740+0000","last_peered":"2024-05-22T21:15:44.401740+0000","last_clean":"2024-05-22T21:15:44.401740+0000","last_became_active":"2024-05-22T21:15:42.582175+0000","last_became_peered":"2024-05-22T21:15:42.582175+0000","last_unstale":"2024-05-22T21:15:44.401740+0000","last_undegraded":"2024-05-22T21:15:44.401740+0000","last_fullsized":"2024-05-22T21:15:44.401740+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:43:22.868781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401190+0000","last_change":"2024-05-22T21:15:42.976891+0000","last_active":"2024-05-22T21:15:44.401190+0000","last_peered":"2024-05-22T21:15:44.401190+0000","last_clean":"2024-05-22T21:15:44.401190+0000","last_became_active":"2024-05-22T21:15:42.976514+0000","last_became_peered":"2024-05-22T21:15:42.976514+0000","last_unstale":"2024-05-22T21:15:44.401190+0000","last_undegraded":"2024-05-22T21:15:44.401190+0000","last_fullsized":"2024-05-22T21:15:44.401190+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:07:20.924146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"66'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961604+0000","last_change":"2024-05-22T21:15:42.583164+0000","last_active":"2024-05-22T21:15:43.961604+0000","last_peered":"2024-05-22T21:15:43.961604+0000","last_clean":"2024-05-22T21:15:43.961604+0000","last_became_active":"2024-05-22T21:15:42.582561+0000","last_became_peered":"2024-05-22T21:15:42.582561+0000","last_unstale":"2024-05-22T21:15:43.961604+0000","last_undegraded":"2024-05-22T21:15:43.961604+0000","last_fullsized":"2024-05-22T21:15:43.961604+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:02:42.940946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401388+0000","last_change":"2024-05-22T21:15:42.979017+0000","last_active":"2024-05-22T21:15:44.401388+0000","last_peered":"2024-05-22T21:15:44.401388+0000","last_clean":"2024-05-22T21:15:44.401388+0000","last_became_active":"2024-05-22T21:15:42.975692+0000","last_became_peered":"2024-05-22T21:15:42.975692+0000","last_unstale":"2024-05-22T21:15:44.401388+0000","last_undegraded":"2024-05-22T21:15:44.401388+0000","last_fullsized":"2024-05-22T21:15:44.401388+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:46:40.357385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.949261+0000","last_change":"2024-05-22T21:15:42.582854+0000","last_active":"2024-05-22T21:15:44.949261+0000","last_peered":"2024-05-22T21:15:44.949261+0000","last_clean":"2024-05-22T21:15:44.949261+0000","last_became_active":"2024-05-22T21:15:42.581206+0000","last_became_peered":"2024-05-22T21:15:42.581206+0000","last_unstale":"2024-05-22T21:15:44.949261+0000","last_undegraded":"2024-05-22T21:15:44.949261+0000","last_fullsized":"2024-05-22T21:15:44.949261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:06:27.336259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961739+0000","last_change":"2024-05-22T21:15:42.974845+0000","last_active":"2024-05-22T21:15:43.961739+0000","last_peered":"2024-05-22T21:15:43.961739+0000","last_clean":"2024-05-22T21:15:43.961739+0000","last_became_active":"2024-05-22T21:15:42.974275+0000","last_became_peered":"2024-05-22T21:15:42.974275+0000","last_unstale":"2024-05-22T21:15:43.961739+0000","last_undegraded":"2024-05-22T21:15:43.961739+0000","last_fullsized":"2024-05-22T21:15:43.961739+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:59:24.847376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064016+0000","last_change":"2024-05-22T21:15:42.585556+0000","last_active":"2024-05-22T21:15:44.064016+0000","last_peered":"2024-05-22T21:15:44.064016+0000","last_clean":"2024-05-22T21:15:44.064016+0000","last_became_active":"2024-05-22T21:15:42.585336+0000","last_became_peered":"2024-05-22T21:15:42.585336+0000","last_unstale":"2024-05-22T21:15:44.064016+0000","last_undegraded":"2024-05-22T21:15:44.064016+0000","last_fullsized":"2024-05-22T21:15:44.064016+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:02:42.940946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948674+0000","last_change":"2024-05-22T21:15:42.975205+0000","last_active":"2024-05-22T21:15:44.948674+0000","last_peered":"2024-05-22T21:15:44.948674+0000","last_clean":"2024-05-22T21:15:44.948674+0000","last_became_active":"2024-05-22T21:15:42.974999+0000","last_became_peered":"2024-05-22T21:15:42.974999+0000","last_unstale":"2024-05-22T21:15:44.948674+0000","last_undegraded":"2024-05-22T21:15:44.948674+0000","last_fullsized":"2024-05-22T21:15:44.948674+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:29:50.850273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401583+0000","last_change":"2024-05-22T21:15:42.583555+0000","last_active":"2024-05-22T21:15:44.401583+0000","last_peered":"2024-05-22T21:15:44.401583+0000","last_clean":"2024-05-22T21:15:44.401583+0000","last_became_active":"2024-05-22T21:15:42.583165+0000","last_became_peered":"2024-05-22T21:15:42.583165+0000","last_unstale":"2024-05-22T21:15:44.401583+0000","last_undegraded":"2024-05-22T21:15:44.401583+0000","last_fullsized":"2024-05-22T21:15:44.401583+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:06:27.336259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064610+0000","last_change":"2024-05-22T21:15:42.975377+0000","last_active":"2024-05-22T21:15:44.064610+0000","last_peered":"2024-05-22T21:15:44.064610+0000","last_clean":"2024-05-22T21:15:44.064610+0000","last_became_active":"2024-05-22T21:15:42.970618+0000","last_became_peered":"2024-05-22T21:15:42.970618+0000","last_unstale":"2024-05-22T21:15:44.064610+0000","last_undegraded":"2024-05-22T21:15:44.064610+0000","last_fullsized":"2024-05-22T21:15:44.064610+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:40:02.153083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961836+0000","last_change":"2024-05-22T21:15:42.582957+0000","last_active":"2024-05-22T21:15:43.961836+0000","last_peered":"2024-05-22T21:15:43.961836+0000","last_clean":"2024-05-22T21:15:43.961836+0000","last_became_active":"2024-05-22T21:15:42.582489+0000","last_became_peered":"2024-05-22T21:15:42.582489+0000","last_unstale":"2024-05-22T21:15:43.961836+0000","last_undegraded":"2024-05-22T21:15:43.961836+0000","last_fullsized":"2024-05-22T21:15:43.961836+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:39.462116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963623+0000","last_change":"2024-05-22T21:15:42.970227+0000","last_active":"2024-05-22T21:15:43.963623+0000","last_peered":"2024-05-22T21:15:43.963623+0000","last_clean":"2024-05-22T21:15:43.963623+0000","last_became_active":"2024-05-22T21:15:42.969366+0000","last_became_peered":"2024-05-22T21:15:42.969366+0000","last_unstale":"2024-05-22T21:15:43.963623+0000","last_undegraded":"2024-05-22T21:15:43.963623+0000","last_fullsized":"2024-05-22T21:15:43.963623+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:04:14.349263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963257+0000","last_change":"2024-05-22T21:15:42.583052+0000","last_active":"2024-05-22T21:15:43.963257+0000","last_peered":"2024-05-22T21:15:43.963257+0000","last_clean":"2024-05-22T21:15:43.963257+0000","last_became_active":"2024-05-22T21:15:42.582824+0000","last_became_peered":"2024-05-22T21:15:42.582824+0000","last_unstale":"2024-05-22T21:15:43.963257+0000","last_undegraded":"2024-05-22T21:15:43.963257+0000","last_fullsized":"2024-05-22T21:15:43.963257+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:58:52.048265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961737+0000","last_change":"2024-05-22T21:15:42.972158+0000","last_active":"2024-05-22T21:15:43.961737+0000","last_peered":"2024-05-22T21:15:43.961737+0000","last_clean":"2024-05-22T21:15:43.961737+0000","last_became_active":"2024-05-22T21:15:42.971929+0000","last_became_peered":"2024-05-22T21:15:42.971929+0000","last_unstale":"2024-05-22T21:15:43.961737+0000","last_undegraded":"2024-05-22T21:15:43.961737+0000","last_fullsized":"2024-05-22T21:15:43.961737+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:16:54.900284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961579+0000","last_change":"2024-05-22T21:15:42.585366+0000","last_active":"2024-05-22T21:15:43.961579+0000","last_peered":"2024-05-22T21:15:43.961579+0000","last_clean":"2024-05-22T21:15:43.961579+0000","last_became_active":"2024-05-22T21:15:42.585137+0000","last_became_peered":"2024-05-22T21:15:42.585137+0000","last_unstale":"2024-05-22T21:15:43.961579+0000","last_undegraded":"2024-05-22T21:15:43.961579+0000","last_fullsized":"2024-05-22T21:15:43.961579+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:42:39.578833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064178+0000","last_change":"2024-05-22T21:15:42.978290+0000","last_active":"2024-05-22T21:15:44.064178+0000","last_peered":"2024-05-22T21:15:44.064178+0000","last_clean":"2024-05-22T21:15:44.064178+0000","last_became_active":"2024-05-22T21:15:42.973526+0000","last_became_peered":"2024-05-22T21:15:42.973526+0000","last_unstale":"2024-05-22T21:15:44.064178+0000","last_undegraded":"2024-05-22T21:15:44.064178+0000","last_fullsized":"2024-05-22T21:15:44.064178+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:34:22.363001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064374+0000","last_change":"2024-05-22T21:15:42.587015+0000","last_active":"2024-05-22T21:15:44.064374+0000","last_peered":"2024-05-22T21:15:44.064374+0000","last_clean":"2024-05-22T21:15:44.064374+0000","last_became_active":"2024-05-22T21:15:42.582539+0000","last_became_peered":"2024-05-22T21:15:42.582539+0000","last_unstale":"2024-05-22T21:15:44.064374+0000","last_undegraded":"2024-05-22T21:15:44.064374+0000","last_fullsized":"2024-05-22T21:15:44.064374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:04:13.342057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064471+0000","last_change":"2024-05-22T21:15:42.973023+0000","last_active":"2024-05-22T21:15:44.064471+0000","last_peered":"2024-05-22T21:15:44.064471+0000","last_clean":"2024-05-22T21:15:44.064471+0000","last_became_active":"2024-05-22T21:15:42.972799+0000","last_became_peered":"2024-05-22T21:15:42.972799+0000","last_unstale":"2024-05-22T21:15:44.064471+0000","last_undegraded":"2024-05-22T21:15:44.064471+0000","last_fullsized":"2024-05-22T21:15:44.064471+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:38:31.751953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.063972+0000","last_change":"2024-05-22T21:15:42.582878+0000","last_active":"2024-05-22T21:15:44.063972+0000","last_peered":"2024-05-22T21:15:44.063972+0000","last_clean":"2024-05-22T21:15:44.063972+0000","last_became_active":"2024-05-22T21:15:42.582664+0000","last_became_peered":"2024-05-22T21:15:42.582664+0000","last_unstale":"2024-05-22T21:15:44.063972+0000","last_undegraded":"2024-05-22T21:15:44.063972+0000","last_fullsized":"2024-05-22T21:15:44.063972+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:48:28.713342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-05-22T21:15:42.962476+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:15:42.962476+0000","last_peered":"2024-05-22T21:15:42.962476+0000","last_clean":"2024-05-22T21:15:42.962476+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:15:42.962476+0000","last_undegraded":"2024-05-22T21:15:42.962476+0000","last_fullsized":"2024-05-22T21:15:42.962476+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":913,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194399,"statfs":{"total":200000000,"available":199065002,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.751,"15min":0.77000000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.998,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68799999999999994,"15min":0.68700000000000006},"min":{"1min":0.44700000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86899999999999999,"5min":1.889,"15min":1.889},"last":0.48599999999999999}]},{"osd":1,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.48099999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93000000000000005,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.753,"15min":0.77300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86499999999999999,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.40300000000000002}]},{"osd":2,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.51100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90600000000000003,"5min":1.923,"15min":1.923},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71199999999999997,"15min":0.71599999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93600000000000005,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.74399999999999999}]},{"osd":3,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.83799999999999997,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.443},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.78700000000000003,"15min":0.80300000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.95499999999999996,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.78000000000000003}]},{"osd":4,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61199999999999999,"15min":0.60899999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.71999999999999997}]},{"osd":5,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.43099999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.71099999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64200000000000002,"15min":0.63400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655530,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:15:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.48099999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.88600000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.68100000000000005},"min":{"1min":0.47399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90300000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.90500000000000003}]},{"osd":1,"last update":"Wed May 22 21:15:11 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70699999999999996,"15min":0.70599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0029999999999999,"5min":1.081,"15min":1.081},"last":0.92600000000000005}]},{"osd":2,"last update":"Wed May 22 21:15:11 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.75900000000000001,"15min":0.751},"min":{"1min":0.61899999999999999,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.73799999999999999,"15min":0.73599999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.94199999999999995,"5min":0.997,"15min":0.997},"last":0.95999999999999996}]},{"osd":3,"last update":"Wed May 22 21:15:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.499,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96599999999999997,"5min":1.415,"15min":1.415},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73599999999999999,"15min":0.73099999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.94499999999999995}]},{"osd":4,"last update":"Wed May 22 21:15:11 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64900000000000002,"15min":0.65500000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81000000000000005,"5min":1.006,"15min":1.006},"last":0.627},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.718,"5min":2.718,"15min":2.718},"last":0.871}]},{"osd":5,"last update":"Wed May 22 21:15:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":0.79000000000000004,"15min":0.71799999999999997},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":8.5749999999999993,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.68400000000000005}]},{"osd":7,"last update":"Wed May 22 21:15:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.745,"15min":0.749},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90000000000000002,"5min":1.095,"15min":1.095},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77900000000000003,"15min":0.77400000000000002},"min":{"1min":0.52800000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2450000000000001,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.80500000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":337,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194975,"statfs":{"total":200000000,"available":199654878,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69899999999999995,"15min":0.70699999999999996},"min":{"1min":0.53400000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.83399999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.53300000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.82099999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.71599999999999997}]},{"osd":1,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70499999999999996,"15min":0.70899999999999996},"min":{"1min":0.51000000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.92200000000000004,"5min":1.216,"15min":1.216},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.51700000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.84099999999999997,"5min":1.048,"15min":1.048},"last":0.90700000000000003}]},{"osd":2,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69699999999999995,"15min":0.69199999999999995},"min":{"1min":0.56899999999999995,"5min":0.436,"15min":0.436},"max":{"1min":0.86199999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66400000000000003,"15min":0.66500000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.81299999999999994,"5min":1.252,"15min":1.252},"last":0.73499999999999999}]},{"osd":3,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70599999999999996,"15min":0.70999999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.84699999999999998,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72599999999999998,"15min":0.72699999999999998},"min":{"1min":0.55600000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.876,"5min":1.111,"15min":1.111},"last":0.84499999999999997}]},{"osd":4,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.624,"15min":0.63800000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69799999999999995,"5min":1.272,"15min":1.272},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.621,"15min":0.629},"min":{"1min":0.443,"5min":0.434,"15min":0.434},"max":{"1min":0.754,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61899999999999999}]},{"osd":6,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.624,"15min":0.626},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.65600000000000003,"15min":0.66400000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81299999999999994,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.64200000000000002}]},{"osd":7,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.71299999999999997,"15min":0.71699999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.755},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.79100000000000004,"15min":0.80100000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94599999999999995,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.58899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":337,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194974,"statfs":{"total":200000000,"available":199653966,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:15:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90300000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.70799999999999996,"15min":0.71499999999999997},"min":{"1min":0.52900000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.89100000000000001,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.61099999999999999}]},{"osd":1,"last update":"Wed May 22 21:15:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.71299999999999997,"15min":0.72699999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.85499999999999998,"5min":1.5,"15min":1.5},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.56899999999999995}]},{"osd":2,"last update":"Wed May 22 21:15:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67800000000000005,"15min":0.68100000000000005},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.90700000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.873,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.624}]},{"osd":3,"last update":"Wed May 22 21:15:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68200000000000005,"15min":0.68799999999999994},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.78500000000000003,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93300000000000005,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.59499999999999997}]},{"osd":5,"last update":"Wed May 22 21:15:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.40600000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":1.012,"15min":1.012},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.48799999999999999}]},{"osd":6,"last update":"Wed May 22 21:15:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69899999999999995,"15min":0.70599999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.89300000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.51000000000000001}]},{"osd":7,"last update":"Wed May 22 21:15:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.52600000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96799999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78600000000000003,"15min":0.79500000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98699999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.55100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655474,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.53200000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65400000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.49399999999999999}]},{"osd":1,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.442,"15min":0.442},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48999999999999999,"15min":0.49399999999999999},"min":{"1min":0.35399999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65900000000000003,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.51600000000000001}]},{"osd":2,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48999999999999999,"15min":0.48899999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69999999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48099999999999998,"15min":0.47999999999999998},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71499999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.504,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95099999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.71199999999999997,"15min":0.70599999999999996},"min":{"1min":0.51900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.022,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.73199999999999998}]},{"osd":5,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68400000000000005,"15min":0.67400000000000004},"min":{"1min":0.48099999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.03,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.52800000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.88800000000000001,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.77900000000000003}]},{"osd":6,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.53700000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0329999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.69599999999999995,"15min":0.68700000000000006},"min":{"1min":0.59899999999999998,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91800000000000004,"5min":1.25,"15min":1.25},"last":0.69099999999999995}]},{"osd":7,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.751,"15min":0.74299999999999999},"min":{"1min":0.55800000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.0429999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.746},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.76700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655474,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39500000000000002},"min":{"1min":0.33000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.621,"5min":0.624,"15min":0.624},"last":0.35799999999999998}]},{"osd":1,"last update":"Wed May 22 21:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.443,"15min":0.442},"min":{"1min":0.316,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.41599999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.48199999999999998}]},{"osd":3,"last update":"Wed May 22 21:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.49199999999999999,"15min":0.495},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.61899999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.44900000000000001,"15min":0.45100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.59399999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.46100000000000002}]},{"osd":4,"last update":"Wed May 22 21:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67600000000000005,"15min":0.67100000000000004},"min":{"1min":0.54300000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73699999999999999,"15min":0.73799999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85099999999999998,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.73999999999999999}]},{"osd":5,"last update":"Wed May 22 21:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68600000000000005,"15min":0.69099999999999995},"min":{"1min":0.50900000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.58299999999999996}]},{"osd":6,"last update":"Wed May 22 21:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.52800000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.76700000000000002}]},{"osd":7,"last update":"Wed May 22 21:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.94399999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76500000000000001,"15min":0.76400000000000001},"min":{"1min":0.628,"5min":0.621,"15min":0.621},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.66500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655440,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.43099999999999999,"15min":0.435},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59399999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.42799999999999999,"15min":0.42599999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63200000000000001,"5min":0.876,"15min":0.876},"last":0.59499999999999997}]},{"osd":2,"last update":"Wed May 22 21:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.61299999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52900000000000003,"15min":0.54200000000000004},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58899999999999997,"5min":1.923,"15min":1.923},"last":0.621}]},{"osd":3,"last update":"Wed May 22 21:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.751,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.63300000000000001,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.505}]},{"osd":4,"last update":"Wed May 22 21:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.753,"15min":0.76400000000000001},"min":{"1min":0.49299999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.95399999999999996,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.67900000000000005,"15min":0.67400000000000004},"min":{"1min":0.50600000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.03,"5min":1.3,"15min":1.3},"last":1.069}]},{"osd":5,"last update":"Wed May 22 21:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72799999999999998,"15min":0.72299999999999998},"min":{"1min":0.61199999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0009999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.028},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.752},"min":{"1min":0.52200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92300000000000004,"5min":1.97,"15min":1.97},"last":0.99099999999999999}]},{"osd":6,"last update":"Wed May 22 21:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73699999999999999,"15min":0.73599999999999999},"min":{"1min":0.53800000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0820000000000001,"5min":1.272,"15min":1.272},"last":1.085},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.59699999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0680000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.96299999999999997}]},{"osd":7,"last update":"Wed May 22 21:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96499999999999997,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0449999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.008}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":914,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194398,"statfs":{"total":200000000,"available":199063635,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.39600000000000002},"min":{"1min":0.25800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.41199999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.438}]},{"osd":2,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.45400000000000001,"15min":0.443},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.42199999999999999,"15min":0.41799999999999998},"min":{"1min":0.32400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59599999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.433,"15min":0.42599999999999999},"min":{"1min":0.33100000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.39500000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.496}]},{"osd":4,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.012,"5min":1.244,"15min":1.244},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0640000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55900000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.57899999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.056,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.84699999999999998}]},{"osd":6,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.60399999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.028,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.998,"5min":1.5,"15min":1.5},"last":0.81000000000000005}]},{"osd":7,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.55000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.91900000000000004,"5min":1.085,"15min":1.085},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.753,"15min":0.754},"min":{"1min":0.54400000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.115,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.71699999999999997}]}]}],"pool_statfs":[]}} 2024-05-22T21:15:47.050 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:15:47.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.217+0000 7f0ed8800640 1 Processor -- start 2024-05-22T21:15:47.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.217+0000 7f0ed8800640 1 -- start start 2024-05-22T21:15:47.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.217+0000 7f0ed8800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0ed0143ec0 0x7f0ed015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:47.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.217+0000 7f0ed8800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0ed015ec90 0x7f0ed015f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:47.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.217+0000 7f0ed8800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed01442a0 con 0x7f0ed00715a0 2024-05-22T21:15:47.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.217+0000 7f0ed8800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed015f680 con 0x7f0ed015ec90 2024-05-22T21:15:47.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.217+0000 7f0ed8800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed0070500 con 0x7f0ed0143ec0 2024-05-22T21:15:47.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.221+0000 7f0ed5400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0ed00715a0 0x7f0ed006fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51864/0 (socket says 172.21.15.161:51864) 2024-05-22T21:15:47.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.221+0000 7f0ed5400640 1 -- 172.21.15.161:0/4275718599 learned_addr learned my addr 172.21.15.161:0/4275718599 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:47.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.221+0000 7f0ed6800640 1 --2- 172.21.15.161:0/4275718599 >> v2:172.21.15.179:3300/0 conn(0x7f0ed015ec90 0x7f0ed015f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:47.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.221+0000 7f0ed5e00640 1 --2- 172.21.15.161:0/4275718599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0ed0143ec0 0x7f0ed015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:47.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.221+0000 7f0ed4a00640 1 -- 172.21.15.161:0/4275718599 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869125799 0 0) 0x7f0ed01442a0 con 0x7f0ed00715a0 2024-05-22T21:15:47.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.221+0000 7f0ed4a00640 1 -- 172.21.15.161:0/4275718599 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eb8003530 con 0x7f0ed00715a0 2024-05-22T21:15:47.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.221+0000 7f0ed6800640 1 -- 172.21.15.161:0/4275718599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0ed0143ec0 msgr2=0x7f0ed015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:47.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.221+0000 7f0ed6800640 1 --2- 172.21.15.161:0/4275718599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0ed0143ec0 0x7f0ed015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.221+0000 7f0ed6800640 1 -- 172.21.15.161:0/4275718599 >> v1:172.21.15.161:6789/0 conn(0x7f0ed00715a0 legacy=0x7f0ed006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:47.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.221+0000 7f0ed6800640 1 -- 172.21.15.161:0/4275718599 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed00707e0 con 0x7f0ed015ec90 2024-05-22T21:15:47.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.221+0000 7f0ed6800640 1 --2- 172.21.15.161:0/4275718599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ed015ec90 0x7f0ed015f140 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009d90 tx=0x7f0ecc01c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43bcdeafba8d794f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:47.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed4a00640 1 -- 172.21.15.161:0/4275718599 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc0a5900 con 0x7f0ed015ec90 2024-05-22T21:15:47.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed4a00640 1 -- 172.21.15.161:0/4275718599 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ecc0a5a60 con 0x7f0ed015ec90 2024-05-22T21:15:47.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed4a00640 1 -- 172.21.15.161:0/4275718599 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc0a5df0 con 0x7f0ed015ec90 2024-05-22T21:15:47.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed8800640 1 -- 172.21.15.161:0/4275718599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ed015ec90 msgr2=0x7f0ed015f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:47.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed8800640 1 --2- 172.21.15.161:0/4275718599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ed015ec90 0x7f0ed015f140 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009d90 tx=0x7f0ecc01c940 comp rx=0 tx=0).stop 2024-05-22T21:15:47.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed8800640 1 -- 172.21.15.161:0/4275718599 shutdown_connections 2024-05-22T21:15:47.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed8800640 1 --2- 172.21.15.161:0/4275718599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ed015ec90 0x7f0ed015f140 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed8800640 1 --2- 172.21.15.161:0/4275718599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0ed0143ec0 0x7f0ed015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed8800640 1 -- 172.21.15.161:0/4275718599 >> 172.21.15.161:0/4275718599 conn(0x7f0ed00b0bc0 msgr2=0x7f0ed00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:47.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed8800640 1 -- 172.21.15.161:0/4275718599 shutdown_connections 2024-05-22T21:15:47.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed8800640 1 -- 172.21.15.161:0/4275718599 wait complete. 2024-05-22T21:15:47.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed8800640 1 Processor -- start 2024-05-22T21:15:47.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed8800640 1 -- start start 2024-05-22T21:15:47.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed8800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ed0143ec0 0x7f0ed00b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:47.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed8800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0ed015ec90 0x7f0ed01f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:47.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed8800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ed01f5300 0x7f0ed0141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed8800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed00ad5a0 con 0x7f0ed0143ec0 2024-05-22T21:15:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed8800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed00ad880 con 0x7f0ed01f5300 2024-05-22T21:15:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed8800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed00adb60 con 0x7f0ed015ec90 2024-05-22T21:15:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed5e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ed0143ec0 0x7f0ed00b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed5e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ed0143ec0 0x7f0ed00b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55710/0 (socket says 172.21.15.161:55710) 2024-05-22T21:15:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.225+0000 7f0ed5e00640 1 -- 172.21.15.161:0/2883552694 learned_addr learned my addr 172.21.15.161:0/2883552694 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.229+0000 7f0ed6800640 1 --2- 172.21.15.161:0/2883552694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ed01f5300 0x7f0ed0141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.229+0000 7f0ed5400640 1 --2- 172.21.15.161:0/2883552694 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0ed015ec90 0x7f0ed01f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.229+0000 7f0ed5e00640 1 -- 172.21.15.161:0/2883552694 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0ed015ec90 msgr2=0x7f0ed01f4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.229+0000 7f0ed5e00640 1 --2- 172.21.15.161:0/2883552694 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0ed015ec90 0x7f0ed01f4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.229+0000 7f0ed5e00640 1 -- 172.21.15.161:0/2883552694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ed01f5300 msgr2=0x7f0ed0141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.229+0000 7f0ed5e00640 1 --2- 172.21.15.161:0/2883552694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ed01f5300 0x7f0ed0141b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.229+0000 7f0ed5e00640 1 -- 172.21.15.161:0/2883552694 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc009a70 con 0x7f0ed0143ec0 2024-05-22T21:15:47.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.229+0000 7f0ed5e00640 1 --2- 172.21.15.161:0/2883552694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ed0143ec0 0x7f0ed00b3990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400e930 tx=0x7f0ec400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:47.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.229+0000 7f0eca000640 1 -- 172.21.15.161:0/2883552694 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4009690 con 0x7f0ed0143ec0 2024-05-22T21:15:47.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.229+0000 7f0eca000640 1 -- 172.21.15.161:0/2883552694 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec400ca00 con 0x7f0ed0143ec0 2024-05-22T21:15:47.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.229+0000 7f0eca000640 1 -- 172.21.15.161:0/2883552694 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400cd70 con 0x7f0ed0143ec0 2024-05-22T21:15:47.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.229+0000 7f0ed8800640 1 -- 172.21.15.161:0/2883552694 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed00ade40 con 0x7f0ed0143ec0 2024-05-22T21:15:47.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.229+0000 7f0ed8800640 1 -- 172.21.15.161:0/2883552694 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed00b3ed0 con 0x7f0ed0143ec0 2024-05-22T21:15:47.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.229+0000 7f0ed8800640 1 -- 172.21.15.161:0/2883552694 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed01f5710 con 0x7f0ed0143ec0 2024-05-22T21:15:47.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.229+0000 7f0eca000640 1 -- 172.21.15.161:0/2883552694 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0ec4013470 con 0x7f0ed0143ec0 2024-05-22T21:15:47.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.229+0000 7f0eca000640 1 --2- 172.21.15.161:0/2883552694 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0eb4066690 0x7f0eb4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:47.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.233+0000 7f0eca000640 1 -- 172.21.15.161:0/2883552694 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ec40844e0 con 0x7f0ed0143ec0 2024-05-22T21:15:47.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.233+0000 7f0ed5400640 1 --2- 172.21.15.161:0/2883552694 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0eb4066690 0x7f0eb4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:47.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.233+0000 7f0ed5400640 1 --2- 172.21.15.161:0/2883552694 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0eb4066690 0x7f0eb4068b50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0142f30 tx=0x7f0ec0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:47.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.237+0000 7f0eca000640 1 -- 172.21.15.161:0/2883552694 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0ec4051870 con 0x7f0ed0143ec0 2024-05-22T21:15:47.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.389+0000 7f0ed8800640 1 -- 172.21.15.161:0/2883552694 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ed0067220 con 0x7f0eb4066690 2024-05-22T21:15:47.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.397+0000 7f0eca000640 1 -- 172.21.15.161:0/2883552694 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284866 (secure 0 0 0) 0x7f0ed0067220 con 0x7f0eb4066690 2024-05-22T21:15:47.405 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:47.406 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:15:47.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.401+0000 7f0ed8800640 1 -- 172.21.15.161:0/2883552694 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0eb4066690 msgr2=0x7f0eb4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:47.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.401+0000 7f0ed8800640 1 --2- 172.21.15.161:0/2883552694 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0eb4066690 0x7f0eb4068b50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0142f30 tx=0x7f0ec0009040 comp rx=0 tx=0).stop 2024-05-22T21:15:47.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.401+0000 7f0ed8800640 1 -- 172.21.15.161:0/2883552694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ed0143ec0 msgr2=0x7f0ed00b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:47.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.401+0000 7f0ed8800640 1 --2- 172.21.15.161:0/2883552694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ed0143ec0 0x7f0ed00b3990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400e930 tx=0x7f0ec400edf0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.401+0000 7f0ed8800640 1 -- 172.21.15.161:0/2883552694 shutdown_connections 2024-05-22T21:15:47.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.401+0000 7f0ed8800640 1 --2- 172.21.15.161:0/2883552694 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0eb4066690 0x7f0eb4068b50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.401+0000 7f0ed8800640 1 --2- 172.21.15.161:0/2883552694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ed01f5300 0x7f0ed0141b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.401+0000 7f0ed8800640 1 --2- 172.21.15.161:0/2883552694 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0ed015ec90 0x7f0ed01f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.401+0000 7f0ed8800640 1 --2- 172.21.15.161:0/2883552694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ed0143ec0 0x7f0ed00b3990 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.401+0000 7f0ed8800640 1 -- 172.21.15.161:0/2883552694 >> 172.21.15.161:0/2883552694 conn(0x7f0ed00b0bc0 msgr2=0x7f0ed0140f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:47.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.401+0000 7f0ed8800640 1 -- 172.21.15.161:0/2883552694 shutdown_connections 2024-05-22T21:15:47.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.401+0000 7f0ed8800640 1 -- 172.21.15.161:0/2883552694 wait complete. 2024-05-22T21:15:47.425 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-05-22T21:15:46.423363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3848,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558651,"statfs":{"total":1600000000,"available":1596059399,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.860703"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.400825+0000","last_change":"2024-05-22T21:15:42.970594+0000","last_active":"2024-05-22T21:15:44.400825+0000","last_peered":"2024-05-22T21:15:44.400825+0000","last_clean":"2024-05-22T21:15:44.400825+0000","last_became_active":"2024-05-22T21:15:42.970393+0000","last_became_peered":"2024-05-22T21:15:42.970393+0000","last_unstale":"2024-05-22T21:15:44.400825+0000","last_undegraded":"2024-05-22T21:15:44.400825+0000","last_fullsized":"2024-05-22T21:15:44.400825+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:22.960323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064119+0000","last_change":"2024-05-22T21:15:42.971240+0000","last_active":"2024-05-22T21:15:44.064119+0000","last_peered":"2024-05-22T21:15:44.064119+0000","last_clean":"2024-05-22T21:15:44.064119+0000","last_became_active":"2024-05-22T21:15:42.970353+0000","last_became_peered":"2024-05-22T21:15:42.970353+0000","last_unstale":"2024-05-22T21:15:44.064119+0000","last_undegraded":"2024-05-22T21:15:44.064119+0000","last_fullsized":"2024-05-22T21:15:44.064119+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:03:46.835560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.962729+0000","last_change":"2024-05-22T21:15:42.977075+0000","last_active":"2024-05-22T21:15:43.962729+0000","last_peered":"2024-05-22T21:15:43.962729+0000","last_clean":"2024-05-22T21:15:43.962729+0000","last_became_active":"2024-05-22T21:15:42.976413+0000","last_became_peered":"2024-05-22T21:15:42.976413+0000","last_unstale":"2024-05-22T21:15:43.962729+0000","last_undegraded":"2024-05-22T21:15:43.962729+0000","last_fullsized":"2024-05-22T21:15:43.962729+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:46:40.357385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401358+0000","last_change":"2024-05-22T21:15:42.978184+0000","last_active":"2024-05-22T21:15:44.401358+0000","last_peered":"2024-05-22T21:15:44.401358+0000","last_clean":"2024-05-22T21:15:44.401358+0000","last_became_active":"2024-05-22T21:15:42.977823+0000","last_became_peered":"2024-05-22T21:15:42.977823+0000","last_unstale":"2024-05-22T21:15:44.401358+0000","last_undegraded":"2024-05-22T21:15:44.401358+0000","last_fullsized":"2024-05-22T21:15:44.401358+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:55.539734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.400778+0000","last_change":"2024-05-22T21:15:42.976106+0000","last_active":"2024-05-22T21:15:44.400778+0000","last_peered":"2024-05-22T21:15:44.400778+0000","last_clean":"2024-05-22T21:15:44.400778+0000","last_became_active":"2024-05-22T21:15:42.975911+0000","last_became_peered":"2024-05-22T21:15:42.975911+0000","last_unstale":"2024-05-22T21:15:44.400778+0000","last_undegraded":"2024-05-22T21:15:44.400778+0000","last_fullsized":"2024-05-22T21:15:44.400778+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:19:28.350013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948839+0000","last_change":"2024-05-22T21:15:42.969930+0000","last_active":"2024-05-22T21:15:44.948839+0000","last_peered":"2024-05-22T21:15:44.948839+0000","last_clean":"2024-05-22T21:15:44.948839+0000","last_became_active":"2024-05-22T21:15:42.969715+0000","last_became_peered":"2024-05-22T21:15:42.969715+0000","last_unstale":"2024-05-22T21:15:44.948839+0000","last_undegraded":"2024-05-22T21:15:44.948839+0000","last_fullsized":"2024-05-22T21:15:44.948839+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:05:51.541715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.962894+0000","last_change":"2024-05-22T21:15:42.978863+0000","last_active":"2024-05-22T21:15:43.962894+0000","last_peered":"2024-05-22T21:15:43.962894+0000","last_clean":"2024-05-22T21:15:43.962894+0000","last_became_active":"2024-05-22T21:15:42.978481+0000","last_became_peered":"2024-05-22T21:15:42.978481+0000","last_unstale":"2024-05-22T21:15:43.962894+0000","last_undegraded":"2024-05-22T21:15:43.962894+0000","last_fullsized":"2024-05-22T21:15:43.962894+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:08.621948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064089+0000","last_change":"2024-05-22T21:15:42.973722+0000","last_active":"2024-05-22T21:15:44.064089+0000","last_peered":"2024-05-22T21:15:44.064089+0000","last_clean":"2024-05-22T21:15:44.064089+0000","last_became_active":"2024-05-22T21:15:42.973142+0000","last_became_peered":"2024-05-22T21:15:42.973142+0000","last_unstale":"2024-05-22T21:15:44.064089+0000","last_undegraded":"2024-05-22T21:15:44.064089+0000","last_fullsized":"2024-05-22T21:15:44.064089+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:31:41.440098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.962948+0000","last_change":"2024-05-22T21:15:42.971446+0000","last_active":"2024-05-22T21:15:43.962948+0000","last_peered":"2024-05-22T21:15:43.962948+0000","last_clean":"2024-05-22T21:15:43.962948+0000","last_became_active":"2024-05-22T21:15:42.971205+0000","last_became_peered":"2024-05-22T21:15:42.971205+0000","last_unstale":"2024-05-22T21:15:43.962948+0000","last_undegraded":"2024-05-22T21:15:43.962948+0000","last_fullsized":"2024-05-22T21:15:43.962948+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:53:24.792090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961665+0000","last_change":"2024-05-22T21:15:42.974961+0000","last_active":"2024-05-22T21:15:43.961665+0000","last_peered":"2024-05-22T21:15:43.961665+0000","last_clean":"2024-05-22T21:15:43.961665+0000","last_became_active":"2024-05-22T21:15:42.974656+0000","last_became_peered":"2024-05-22T21:15:42.974656+0000","last_unstale":"2024-05-22T21:15:43.961665+0000","last_undegraded":"2024-05-22T21:15:43.961665+0000","last_fullsized":"2024-05-22T21:15:43.961665+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:57.417032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961760+0000","last_change":"2024-05-22T21:15:42.971637+0000","last_active":"2024-05-22T21:15:43.961760+0000","last_peered":"2024-05-22T21:15:43.961760+0000","last_clean":"2024-05-22T21:15:43.961760+0000","last_became_active":"2024-05-22T21:15:42.971367+0000","last_became_peered":"2024-05-22T21:15:42.971367+0000","last_unstale":"2024-05-22T21:15:43.961760+0000","last_undegraded":"2024-05-22T21:15:43.961760+0000","last_fullsized":"2024-05-22T21:15:43.961760+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:42:29.381771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401300+0000","last_change":"2024-05-22T21:15:42.979177+0000","last_active":"2024-05-22T21:15:44.401300+0000","last_peered":"2024-05-22T21:15:44.401300+0000","last_clean":"2024-05-22T21:15:44.401300+0000","last_became_active":"2024-05-22T21:15:42.975992+0000","last_became_peered":"2024-05-22T21:15:42.975992+0000","last_unstale":"2024-05-22T21:15:44.401300+0000","last_undegraded":"2024-05-22T21:15:44.401300+0000","last_fullsized":"2024-05-22T21:15:44.401300+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:05:51.541715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.063832+0000","last_change":"2024-05-22T21:15:42.972885+0000","last_active":"2024-05-22T21:15:44.063832+0000","last_peered":"2024-05-22T21:15:44.063832+0000","last_clean":"2024-05-22T21:15:44.063832+0000","last_became_active":"2024-05-22T21:15:42.972674+0000","last_became_peered":"2024-05-22T21:15:42.972674+0000","last_unstale":"2024-05-22T21:15:44.063832+0000","last_undegraded":"2024-05-22T21:15:44.063832+0000","last_fullsized":"2024-05-22T21:15:44.063832+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:58:53.055471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963069+0000","last_change":"2024-05-22T21:15:42.975350+0000","last_active":"2024-05-22T21:15:43.963069+0000","last_peered":"2024-05-22T21:15:43.963069+0000","last_clean":"2024-05-22T21:15:43.963069+0000","last_became_active":"2024-05-22T21:15:42.975046+0000","last_became_peered":"2024-05-22T21:15:42.975046+0000","last_unstale":"2024-05-22T21:15:43.963069+0000","last_undegraded":"2024-05-22T21:15:43.963069+0000","last_fullsized":"2024-05-22T21:15:43.963069+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:29:47.821900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963408+0000","last_change":"2024-05-22T21:15:42.971633+0000","last_active":"2024-05-22T21:15:43.963408+0000","last_peered":"2024-05-22T21:15:43.963408+0000","last_clean":"2024-05-22T21:15:43.963408+0000","last_became_active":"2024-05-22T21:15:42.971356+0000","last_became_peered":"2024-05-22T21:15:42.971356+0000","last_unstale":"2024-05-22T21:15:43.963408+0000","last_undegraded":"2024-05-22T21:15:43.963408+0000","last_fullsized":"2024-05-22T21:15:43.963408+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:06:28.343465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.949089+0000","last_change":"2024-05-22T21:15:42.972704+0000","last_active":"2024-05-22T21:15:44.949089+0000","last_peered":"2024-05-22T21:15:44.949089+0000","last_clean":"2024-05-22T21:15:44.949089+0000","last_became_active":"2024-05-22T21:15:42.972499+0000","last_became_peered":"2024-05-22T21:15:42.972499+0000","last_unstale":"2024-05-22T21:15:44.949089+0000","last_undegraded":"2024-05-22T21:15:44.949089+0000","last_fullsized":"2024-05-22T21:15:44.949089+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:51:38.084205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064209+0000","last_change":"2024-05-22T21:15:42.976478+0000","last_active":"2024-05-22T21:15:44.064209+0000","last_peered":"2024-05-22T21:15:44.064209+0000","last_clean":"2024-05-22T21:15:44.064209+0000","last_became_active":"2024-05-22T21:15:42.976030+0000","last_became_peered":"2024-05-22T21:15:42.976030+0000","last_unstale":"2024-05-22T21:15:44.064209+0000","last_undegraded":"2024-05-22T21:15:44.064209+0000","last_fullsized":"2024-05-22T21:15:44.064209+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:04:10.450613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.962834+0000","last_change":"2024-05-22T21:15:42.976942+0000","last_active":"2024-05-22T21:15:43.962834+0000","last_peered":"2024-05-22T21:15:43.962834+0000","last_clean":"2024-05-22T21:15:43.962834+0000","last_became_active":"2024-05-22T21:15:42.974044+0000","last_became_peered":"2024-05-22T21:15:42.974044+0000","last_unstale":"2024-05-22T21:15:43.962834+0000","last_undegraded":"2024-05-22T21:15:43.962834+0000","last_fullsized":"2024-05-22T21:15:43.962834+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:51:38.084205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.400815+0000","last_change":"2024-05-22T21:15:42.974605+0000","last_active":"2024-05-22T21:15:44.400815+0000","last_peered":"2024-05-22T21:15:44.400815+0000","last_clean":"2024-05-22T21:15:44.400815+0000","last_became_active":"2024-05-22T21:15:42.974219+0000","last_became_peered":"2024-05-22T21:15:42.974219+0000","last_unstale":"2024-05-22T21:15:44.400815+0000","last_undegraded":"2024-05-22T21:15:44.400815+0000","last_fullsized":"2024-05-22T21:15:44.400815+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:02:06.403145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064233+0000","last_change":"2024-05-22T21:15:42.968120+0000","last_active":"2024-05-22T21:15:44.064233+0000","last_peered":"2024-05-22T21:15:44.064233+0000","last_clean":"2024-05-22T21:15:44.064233+0000","last_became_active":"2024-05-22T21:15:42.967910+0000","last_became_peered":"2024-05-22T21:15:42.967910+0000","last_unstale":"2024-05-22T21:15:44.064233+0000","last_undegraded":"2024-05-22T21:15:44.064233+0000","last_fullsized":"2024-05-22T21:15:44.064233+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:22:27.221021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963372+0000","last_change":"2024-05-22T21:15:42.975526+0000","last_active":"2024-05-22T21:15:43.963372+0000","last_peered":"2024-05-22T21:15:43.963372+0000","last_clean":"2024-05-22T21:15:43.963372+0000","last_became_active":"2024-05-22T21:15:42.975125+0000","last_became_peered":"2024-05-22T21:15:42.975125+0000","last_unstale":"2024-05-22T21:15:43.963372+0000","last_undegraded":"2024-05-22T21:15:43.963372+0000","last_fullsized":"2024-05-22T21:15:43.963372+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:05:51.541715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064314+0000","last_change":"2024-05-22T21:15:42.975533+0000","last_active":"2024-05-22T21:15:44.064314+0000","last_peered":"2024-05-22T21:15:44.064314+0000","last_clean":"2024-05-22T21:15:44.064314+0000","last_became_active":"2024-05-22T21:15:42.971033+0000","last_became_peered":"2024-05-22T21:15:42.971033+0000","last_unstale":"2024-05-22T21:15:44.064314+0000","last_undegraded":"2024-05-22T21:15:44.064314+0000","last_fullsized":"2024-05-22T21:15:44.064314+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:50:08.534446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.962930+0000","last_change":"2024-05-22T21:15:42.976645+0000","last_active":"2024-05-22T21:15:43.962930+0000","last_peered":"2024-05-22T21:15:43.962930+0000","last_clean":"2024-05-22T21:15:43.962930+0000","last_became_active":"2024-05-22T21:15:42.973393+0000","last_became_peered":"2024-05-22T21:15:42.973393+0000","last_unstale":"2024-05-22T21:15:43.962930+0000","last_undegraded":"2024-05-22T21:15:43.962930+0000","last_fullsized":"2024-05-22T21:15:43.962930+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:38:55.292112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.063897+0000","last_change":"2024-05-22T21:15:42.971878+0000","last_active":"2024-05-22T21:15:44.063897+0000","last_peered":"2024-05-22T21:15:44.063897+0000","last_clean":"2024-05-22T21:15:44.063897+0000","last_became_active":"2024-05-22T21:15:42.971633+0000","last_became_peered":"2024-05-22T21:15:42.971633+0000","last_unstale":"2024-05-22T21:15:44.063897+0000","last_undegraded":"2024-05-22T21:15:44.063897+0000","last_fullsized":"2024-05-22T21:15:44.063897+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:59:38.564543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.400891+0000","last_change":"2024-05-22T21:15:42.971830+0000","last_active":"2024-05-22T21:15:44.400891+0000","last_peered":"2024-05-22T21:15:44.400891+0000","last_clean":"2024-05-22T21:15:44.400891+0000","last_became_active":"2024-05-22T21:15:42.971545+0000","last_became_peered":"2024-05-22T21:15:42.971545+0000","last_unstale":"2024-05-22T21:15:44.400891+0000","last_undegraded":"2024-05-22T21:15:44.400891+0000","last_fullsized":"2024-05-22T21:15:44.400891+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:40:46.992971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401459+0000","last_change":"2024-05-22T21:15:42.972108+0000","last_active":"2024-05-22T21:15:44.401459+0000","last_peered":"2024-05-22T21:15:44.401459+0000","last_clean":"2024-05-22T21:15:44.401459+0000","last_became_active":"2024-05-22T21:15:42.971866+0000","last_became_peered":"2024-05-22T21:15:42.971866+0000","last_unstale":"2024-05-22T21:15:44.401459+0000","last_undegraded":"2024-05-22T21:15:44.401459+0000","last_fullsized":"2024-05-22T21:15:44.401459+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:49:33.826508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948696+0000","last_change":"2024-05-22T21:15:42.972272+0000","last_active":"2024-05-22T21:15:44.948696+0000","last_peered":"2024-05-22T21:15:44.948696+0000","last_clean":"2024-05-22T21:15:44.948696+0000","last_became_active":"2024-05-22T21:15:42.972050+0000","last_became_peered":"2024-05-22T21:15:42.972050+0000","last_unstale":"2024-05-22T21:15:44.948696+0000","last_undegraded":"2024-05-22T21:15:44.948696+0000","last_fullsized":"2024-05-22T21:15:44.948696+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:12:02.773432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.400796+0000","last_change":"2024-05-22T21:15:42.976992+0000","last_active":"2024-05-22T21:15:44.400796+0000","last_peered":"2024-05-22T21:15:44.400796+0000","last_clean":"2024-05-22T21:15:44.400796+0000","last_became_active":"2024-05-22T21:15:42.974819+0000","last_became_peered":"2024-05-22T21:15:42.974819+0000","last_unstale":"2024-05-22T21:15:44.400796+0000","last_undegraded":"2024-05-22T21:15:44.400796+0000","last_fullsized":"2024-05-22T21:15:44.400796+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:47:26.596903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064135+0000","last_change":"2024-05-22T21:15:42.979255+0000","last_active":"2024-05-22T21:15:44.064135+0000","last_peered":"2024-05-22T21:15:44.064135+0000","last_clean":"2024-05-22T21:15:44.064135+0000","last_became_active":"2024-05-22T21:15:42.978671+0000","last_became_peered":"2024-05-22T21:15:42.978671+0000","last_unstale":"2024-05-22T21:15:44.064135+0000","last_undegraded":"2024-05-22T21:15:44.064135+0000","last_fullsized":"2024-05-22T21:15:44.064135+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:01:30.502315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963508+0000","last_change":"2024-05-22T21:15:42.970791+0000","last_active":"2024-05-22T21:15:43.963508+0000","last_peered":"2024-05-22T21:15:43.963508+0000","last_clean":"2024-05-22T21:15:43.963508+0000","last_became_active":"2024-05-22T21:15:42.970565+0000","last_became_peered":"2024-05-22T21:15:42.970565+0000","last_unstale":"2024-05-22T21:15:43.963508+0000","last_undegraded":"2024-05-22T21:15:43.963508+0000","last_fullsized":"2024-05-22T21:15:43.963508+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:26:00.521024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.400917+0000","last_change":"2024-05-22T21:15:42.975667+0000","last_active":"2024-05-22T21:15:44.400917+0000","last_peered":"2024-05-22T21:15:44.400917+0000","last_clean":"2024-05-22T21:15:44.400917+0000","last_became_active":"2024-05-22T21:15:42.975439+0000","last_became_peered":"2024-05-22T21:15:42.975439+0000","last_unstale":"2024-05-22T21:15:44.400917+0000","last_undegraded":"2024-05-22T21:15:44.400917+0000","last_fullsized":"2024-05-22T21:15:44.400917+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:48:42.519585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948792+0000","last_change":"2024-05-22T21:15:42.968273+0000","last_active":"2024-05-22T21:15:44.948792+0000","last_peered":"2024-05-22T21:15:44.948792+0000","last_clean":"2024-05-22T21:15:44.948792+0000","last_became_active":"2024-05-22T21:15:42.968069+0000","last_became_peered":"2024-05-22T21:15:42.968069+0000","last_unstale":"2024-05-22T21:15:44.948792+0000","last_undegraded":"2024-05-22T21:15:44.948792+0000","last_fullsized":"2024-05-22T21:15:44.948792+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:20.645717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948837+0000","last_change":"2024-05-22T21:15:42.967224+0000","last_active":"2024-05-22T21:15:44.948837+0000","last_peered":"2024-05-22T21:15:44.948837+0000","last_clean":"2024-05-22T21:15:44.948837+0000","last_became_active":"2024-05-22T21:15:42.967014+0000","last_became_peered":"2024-05-22T21:15:42.967014+0000","last_unstale":"2024-05-22T21:15:44.948837+0000","last_undegraded":"2024-05-22T21:15:44.948837+0000","last_fullsized":"2024-05-22T21:15:44.948837+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:52:50.421947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"66'4","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.949079+0000","last_change":"2024-05-22T21:15:42.579540+0000","last_active":"2024-05-22T21:15:44.949079+0000","last_peered":"2024-05-22T21:15:44.949079+0000","last_clean":"2024-05-22T21:15:44.949079+0000","last_became_active":"2024-05-22T21:15:42.579314+0000","last_became_peered":"2024-05-22T21:15:42.579314+0000","last_unstale":"2024-05-22T21:15:44.949079+0000","last_undegraded":"2024-05-22T21:15:44.949079+0000","last_fullsized":"2024-05-22T21:15:44.949079+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:41:22.905870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961480+0000","last_change":"2024-05-22T21:15:42.976269+0000","last_active":"2024-05-22T21:15:43.961480+0000","last_peered":"2024-05-22T21:15:43.961480+0000","last_clean":"2024-05-22T21:15:43.961480+0000","last_became_active":"2024-05-22T21:15:42.975974+0000","last_became_peered":"2024-05-22T21:15:42.975974+0000","last_unstale":"2024-05-22T21:15:43.961480+0000","last_undegraded":"2024-05-22T21:15:43.961480+0000","last_fullsized":"2024-05-22T21:15:43.961480+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:20:24.411848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"66'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064265+0000","last_change":"2024-05-22T21:15:42.575027+0000","last_active":"2024-05-22T21:15:44.064265+0000","last_peered":"2024-05-22T21:15:44.064265+0000","last_clean":"2024-05-22T21:15:44.064265+0000","last_became_active":"2024-05-22T21:15:42.574819+0000","last_became_peered":"2024-05-22T21:15:42.574819+0000","last_unstale":"2024-05-22T21:15:44.064265+0000","last_undegraded":"2024-05-22T21:15:44.064265+0000","last_fullsized":"2024-05-22T21:15:44.064265+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:14:59.935734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948991+0000","last_change":"2024-05-22T21:15:42.965607+0000","last_active":"2024-05-22T21:15:44.948991+0000","last_peered":"2024-05-22T21:15:44.948991+0000","last_clean":"2024-05-22T21:15:44.948991+0000","last_became_active":"2024-05-22T21:15:42.965336+0000","last_became_peered":"2024-05-22T21:15:42.965336+0000","last_unstale":"2024-05-22T21:15:44.948991+0000","last_undegraded":"2024-05-22T21:15:44.948991+0000","last_fullsized":"2024-05-22T21:15:44.948991+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:07:20.924146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"66'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.949089+0000","last_change":"2024-05-22T21:15:42.584140+0000","last_active":"2024-05-22T21:15:44.949089+0000","last_peered":"2024-05-22T21:15:44.949089+0000","last_clean":"2024-05-22T21:15:44.949089+0000","last_became_active":"2024-05-22T21:15:42.581462+0000","last_became_peered":"2024-05-22T21:15:42.581462+0000","last_unstale":"2024-05-22T21:15:44.949089+0000","last_undegraded":"2024-05-22T21:15:44.949089+0000","last_fullsized":"2024-05-22T21:15:44.949089+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:07.614742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948875+0000","last_change":"2024-05-22T21:15:42.977758+0000","last_active":"2024-05-22T21:15:44.948875+0000","last_peered":"2024-05-22T21:15:44.948875+0000","last_clean":"2024-05-22T21:15:44.948875+0000","last_became_active":"2024-05-22T21:15:42.977305+0000","last_became_peered":"2024-05-22T21:15:42.977305+0000","last_unstale":"2024-05-22T21:15:44.948875+0000","last_undegraded":"2024-05-22T21:15:44.948875+0000","last_fullsized":"2024-05-22T21:15:44.948875+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:59:05.276396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"66'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401639+0000","last_change":"2024-05-22T21:15:42.584129+0000","last_active":"2024-05-22T21:15:44.401639+0000","last_peered":"2024-05-22T21:15:44.401639+0000","last_clean":"2024-05-22T21:15:44.401639+0000","last_became_active":"2024-05-22T21:15:42.583742+0000","last_became_peered":"2024-05-22T21:15:42.583742+0000","last_unstale":"2024-05-22T21:15:44.401639+0000","last_undegraded":"2024-05-22T21:15:44.401639+0000","last_fullsized":"2024-05-22T21:15:44.401639+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:53:23.784884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064341+0000","last_change":"2024-05-22T21:15:42.979443+0000","last_active":"2024-05-22T21:15:44.064341+0000","last_peered":"2024-05-22T21:15:44.064341+0000","last_clean":"2024-05-22T21:15:44.064341+0000","last_became_active":"2024-05-22T21:15:42.978956+0000","last_became_peered":"2024-05-22T21:15:42.978956+0000","last_unstale":"2024-05-22T21:15:44.064341+0000","last_undegraded":"2024-05-22T21:15:44.064341+0000","last_fullsized":"2024-05-22T21:15:44.064341+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:40:34.106010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"66'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963171+0000","last_change":"2024-05-22T21:15:42.578770+0000","last_active":"2024-05-22T21:15:43.963171+0000","last_peered":"2024-05-22T21:15:43.963171+0000","last_clean":"2024-05-22T21:15:43.963171+0000","last_became_active":"2024-05-22T21:15:42.578561+0000","last_became_peered":"2024-05-22T21:15:42.578561+0000","last_unstale":"2024-05-22T21:15:43.963171+0000","last_undegraded":"2024-05-22T21:15:43.963171+0000","last_fullsized":"2024-05-22T21:15:43.963171+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:16:53.893078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064418+0000","last_change":"2024-05-22T21:15:42.976356+0000","last_active":"2024-05-22T21:15:44.064418+0000","last_peered":"2024-05-22T21:15:44.064418+0000","last_clean":"2024-05-22T21:15:44.064418+0000","last_became_active":"2024-05-22T21:15:42.975730+0000","last_became_peered":"2024-05-22T21:15:42.975730+0000","last_unstale":"2024-05-22T21:15:44.064418+0000","last_undegraded":"2024-05-22T21:15:44.064418+0000","last_fullsized":"2024-05-22T21:15:44.064418+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:28:33.939764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"66'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.949175+0000","last_change":"2024-05-22T21:15:42.583201+0000","last_active":"2024-05-22T21:15:44.949175+0000","last_peered":"2024-05-22T21:15:44.949175+0000","last_clean":"2024-05-22T21:15:44.949175+0000","last_became_active":"2024-05-22T21:15:42.582592+0000","last_became_peered":"2024-05-22T21:15:42.582592+0000","last_unstale":"2024-05-22T21:15:44.949175+0000","last_undegraded":"2024-05-22T21:15:44.949175+0000","last_fullsized":"2024-05-22T21:15:44.949175+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:53:23.784884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.063905+0000","last_change":"2024-05-22T21:15:42.976275+0000","last_active":"2024-05-22T21:15:44.063905+0000","last_peered":"2024-05-22T21:15:44.063905+0000","last_clean":"2024-05-22T21:15:44.063905+0000","last_became_active":"2024-05-22T21:15:42.976060+0000","last_became_peered":"2024-05-22T21:15:42.976060+0000","last_unstale":"2024-05-22T21:15:44.063905+0000","last_undegraded":"2024-05-22T21:15:44.063905+0000","last_fullsized":"2024-05-22T21:15:44.063905+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:52:19.703848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401754+0000","last_change":"2024-05-22T21:15:42.585086+0000","last_active":"2024-05-22T21:15:44.401754+0000","last_peered":"2024-05-22T21:15:44.401754+0000","last_clean":"2024-05-22T21:15:44.401754+0000","last_became_active":"2024-05-22T21:15:42.584866+0000","last_became_peered":"2024-05-22T21:15:42.584866+0000","last_unstale":"2024-05-22T21:15:44.401754+0000","last_undegraded":"2024-05-22T21:15:44.401754+0000","last_fullsized":"2024-05-22T21:15:44.401754+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:41:22.905870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064202+0000","last_change":"2024-05-22T21:15:42.979117+0000","last_active":"2024-05-22T21:15:44.064202+0000","last_peered":"2024-05-22T21:15:44.064202+0000","last_clean":"2024-05-22T21:15:44.064202+0000","last_became_active":"2024-05-22T21:15:42.978338+0000","last_became_peered":"2024-05-22T21:15:42.978338+0000","last_unstale":"2024-05-22T21:15:44.064202+0000","last_undegraded":"2024-05-22T21:15:44.064202+0000","last_fullsized":"2024-05-22T21:15:44.064202+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:00:49.522445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963501+0000","last_change":"2024-05-22T21:15:42.574542+0000","last_active":"2024-05-22T21:15:43.963501+0000","last_peered":"2024-05-22T21:15:43.963501+0000","last_clean":"2024-05-22T21:15:43.963501+0000","last_became_active":"2024-05-22T21:15:42.574265+0000","last_became_peered":"2024-05-22T21:15:42.574265+0000","last_unstale":"2024-05-22T21:15:43.963501+0000","last_undegraded":"2024-05-22T21:15:43.963501+0000","last_fullsized":"2024-05-22T21:15:43.963501+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:59:37.557337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963086+0000","last_change":"2024-05-22T21:15:42.976007+0000","last_active":"2024-05-22T21:15:43.963086+0000","last_peered":"2024-05-22T21:15:43.963086+0000","last_clean":"2024-05-22T21:15:43.963086+0000","last_became_active":"2024-05-22T21:15:42.975720+0000","last_became_peered":"2024-05-22T21:15:42.975720+0000","last_unstale":"2024-05-22T21:15:43.963086+0000","last_undegraded":"2024-05-22T21:15:43.963086+0000","last_fullsized":"2024-05-22T21:15:43.963086+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:20:33.620075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963765+0000","last_change":"2024-05-22T21:15:42.574156+0000","last_active":"2024-05-22T21:15:43.963765+0000","last_peered":"2024-05-22T21:15:43.963765+0000","last_clean":"2024-05-22T21:15:43.963765+0000","last_became_active":"2024-05-22T21:15:42.573379+0000","last_became_peered":"2024-05-22T21:15:42.573379+0000","last_unstale":"2024-05-22T21:15:43.963765+0000","last_undegraded":"2024-05-22T21:15:43.963765+0000","last_fullsized":"2024-05-22T21:15:43.963765+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:53:01.070200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963452+0000","last_change":"2024-05-22T21:15:42.970401+0000","last_active":"2024-05-22T21:15:43.963452+0000","last_peered":"2024-05-22T21:15:43.963452+0000","last_clean":"2024-05-22T21:15:43.963452+0000","last_became_active":"2024-05-22T21:15:42.969670+0000","last_became_peered":"2024-05-22T21:15:42.969670+0000","last_unstale":"2024-05-22T21:15:43.963452+0000","last_undegraded":"2024-05-22T21:15:43.963452+0000","last_fullsized":"2024-05-22T21:15:43.963452+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:07:20.924146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064681+0000","last_change":"2024-05-22T21:15:42.587422+0000","last_active":"2024-05-22T21:15:44.064681+0000","last_peered":"2024-05-22T21:15:44.064681+0000","last_clean":"2024-05-22T21:15:44.064681+0000","last_became_active":"2024-05-22T21:15:42.583942+0000","last_became_peered":"2024-05-22T21:15:42.583942+0000","last_unstale":"2024-05-22T21:15:44.064681+0000","last_undegraded":"2024-05-22T21:15:44.064681+0000","last_fullsized":"2024-05-22T21:15:44.064681+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:40:00.827380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961655+0000","last_change":"2024-05-22T21:15:42.977404+0000","last_active":"2024-05-22T21:15:43.961655+0000","last_peered":"2024-05-22T21:15:43.961655+0000","last_clean":"2024-05-22T21:15:43.961655+0000","last_became_active":"2024-05-22T21:15:42.976868+0000","last_became_peered":"2024-05-22T21:15:42.976868+0000","last_unstale":"2024-05-22T21:15:43.961655+0000","last_undegraded":"2024-05-22T21:15:43.961655+0000","last_fullsized":"2024-05-22T21:15:43.961655+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:53:12.636434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"66'3","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963113+0000","last_change":"2024-05-22T21:15:42.582185+0000","last_active":"2024-05-22T21:15:43.963113+0000","last_peered":"2024-05-22T21:15:43.963113+0000","last_clean":"2024-05-22T21:15:43.963113+0000","last_became_active":"2024-05-22T21:15:42.581935+0000","last_became_peered":"2024-05-22T21:15:42.581935+0000","last_unstale":"2024-05-22T21:15:43.963113+0000","last_undegraded":"2024-05-22T21:15:43.963113+0000","last_fullsized":"2024-05-22T21:15:43.963113+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:09:38.871529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963030+0000","last_change":"2024-05-22T21:15:42.976794+0000","last_active":"2024-05-22T21:15:43.963030+0000","last_peered":"2024-05-22T21:15:43.963030+0000","last_clean":"2024-05-22T21:15:43.963030+0000","last_became_active":"2024-05-22T21:15:42.973696+0000","last_became_peered":"2024-05-22T21:15:42.973696+0000","last_unstale":"2024-05-22T21:15:43.963030+0000","last_undegraded":"2024-05-22T21:15:43.963030+0000","last_fullsized":"2024-05-22T21:15:43.963030+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:29:50.850273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"66'6","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963168+0000","last_change":"2024-05-22T21:15:42.585370+0000","last_active":"2024-05-22T21:15:43.963168+0000","last_peered":"2024-05-22T21:15:43.963168+0000","last_clean":"2024-05-22T21:15:43.963168+0000","last_became_active":"2024-05-22T21:15:42.584788+0000","last_became_peered":"2024-05-22T21:15:42.584788+0000","last_unstale":"2024-05-22T21:15:43.963168+0000","last_undegraded":"2024-05-22T21:15:43.963168+0000","last_fullsized":"2024-05-22T21:15:43.963168+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:52:18.696642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401560+0000","last_change":"2024-05-22T21:15:42.974952+0000","last_active":"2024-05-22T21:15:44.401560+0000","last_peered":"2024-05-22T21:15:44.401560+0000","last_clean":"2024-05-22T21:15:44.401560+0000","last_became_active":"2024-05-22T21:15:42.974712+0000","last_became_peered":"2024-05-22T21:15:42.974712+0000","last_unstale":"2024-05-22T21:15:44.401560+0000","last_undegraded":"2024-05-22T21:15:44.401560+0000","last_fullsized":"2024-05-22T21:15:44.401560+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:29:50.850273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"66'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064386+0000","last_change":"2024-05-22T21:15:42.581241+0000","last_active":"2024-05-22T21:15:44.064386+0000","last_peered":"2024-05-22T21:15:44.064386+0000","last_clean":"2024-05-22T21:15:44.064386+0000","last_became_active":"2024-05-22T21:15:42.581020+0000","last_became_peered":"2024-05-22T21:15:42.581020+0000","last_unstale":"2024-05-22T21:15:44.064386+0000","last_undegraded":"2024-05-22T21:15:44.064386+0000","last_fullsized":"2024-05-22T21:15:44.064386+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:46:39.350179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963668+0000","last_change":"2024-05-22T21:15:42.975321+0000","last_active":"2024-05-22T21:15:43.963668+0000","last_peered":"2024-05-22T21:15:43.963668+0000","last_clean":"2024-05-22T21:15:43.963668+0000","last_became_active":"2024-05-22T21:15:42.974830+0000","last_became_peered":"2024-05-22T21:15:42.974830+0000","last_unstale":"2024-05-22T21:15:43.963668+0000","last_undegraded":"2024-05-22T21:15:43.963668+0000","last_fullsized":"2024-05-22T21:15:43.963668+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:55.539734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.063802+0000","last_change":"2024-05-22T21:15:42.583656+0000","last_active":"2024-05-22T21:15:44.063802+0000","last_peered":"2024-05-22T21:15:44.063802+0000","last_clean":"2024-05-22T21:15:44.063802+0000","last_became_active":"2024-05-22T21:15:42.583439+0000","last_became_peered":"2024-05-22T21:15:42.583439+0000","last_unstale":"2024-05-22T21:15:44.063802+0000","last_undegraded":"2024-05-22T21:15:44.063802+0000","last_fullsized":"2024-05-22T21:15:44.063802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:39.462116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963582+0000","last_change":"2024-05-22T21:15:42.977201+0000","last_active":"2024-05-22T21:15:43.963582+0000","last_peered":"2024-05-22T21:15:43.963582+0000","last_clean":"2024-05-22T21:15:43.963582+0000","last_became_active":"2024-05-22T21:15:42.975235+0000","last_became_peered":"2024-05-22T21:15:42.975235+0000","last_unstale":"2024-05-22T21:15:43.963582+0000","last_undegraded":"2024-05-22T21:15:43.963582+0000","last_fullsized":"2024-05-22T21:15:43.963582+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:41:23.913076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401489+0000","last_change":"2024-05-22T21:15:42.580711+0000","last_active":"2024-05-22T21:15:44.401489+0000","last_peered":"2024-05-22T21:15:44.401489+0000","last_clean":"2024-05-22T21:15:44.401489+0000","last_became_active":"2024-05-22T21:15:42.579788+0000","last_became_peered":"2024-05-22T21:15:42.579788+0000","last_unstale":"2024-05-22T21:15:44.401489+0000","last_undegraded":"2024-05-22T21:15:44.401489+0000","last_fullsized":"2024-05-22T21:15:44.401489+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:29:46.814694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963149+0000","last_change":"2024-05-22T21:15:42.973979+0000","last_active":"2024-05-22T21:15:43.963149+0000","last_peered":"2024-05-22T21:15:43.963149+0000","last_clean":"2024-05-22T21:15:43.963149+0000","last_became_active":"2024-05-22T21:15:42.973733+0000","last_became_peered":"2024-05-22T21:15:42.973733+0000","last_unstale":"2024-05-22T21:15:43.963149+0000","last_undegraded":"2024-05-22T21:15:43.963149+0000","last_fullsized":"2024-05-22T21:15:43.963149+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:17:12.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"66'8","reported_seq":19,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963199+0000","last_change":"2024-05-22T21:15:42.579084+0000","last_active":"2024-05-22T21:15:43.963199+0000","last_peered":"2024-05-22T21:15:43.963199+0000","last_clean":"2024-05-22T21:15:43.963199+0000","last_became_active":"2024-05-22T21:15:42.578629+0000","last_became_peered":"2024-05-22T21:15:42.578629+0000","last_unstale":"2024-05-22T21:15:43.963199+0000","last_undegraded":"2024-05-22T21:15:43.963199+0000","last_fullsized":"2024-05-22T21:15:43.963199+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:42:28.374565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963755+0000","last_change":"2024-05-22T21:15:42.975149+0000","last_active":"2024-05-22T21:15:43.963755+0000","last_peered":"2024-05-22T21:15:43.963755+0000","last_clean":"2024-05-22T21:15:43.963755+0000","last_became_active":"2024-05-22T21:15:42.974473+0000","last_became_peered":"2024-05-22T21:15:42.974473+0000","last_unstale":"2024-05-22T21:15:43.963755+0000","last_undegraded":"2024-05-22T21:15:43.963755+0000","last_fullsized":"2024-05-22T21:15:43.963755+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:02.614285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"66'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064320+0000","last_change":"2024-05-22T21:15:42.582766+0000","last_active":"2024-05-22T21:15:44.064320+0000","last_peered":"2024-05-22T21:15:44.064320+0000","last_clean":"2024-05-22T21:15:44.064320+0000","last_became_active":"2024-05-22T21:15:42.582036+0000","last_became_peered":"2024-05-22T21:15:42.582036+0000","last_unstale":"2024-05-22T21:15:44.064320+0000","last_undegraded":"2024-05-22T21:15:44.064320+0000","last_fullsized":"2024-05-22T21:15:44.064320+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:25:59.513818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401494+0000","last_change":"2024-05-22T21:15:42.974415+0000","last_active":"2024-05-22T21:15:44.401494+0000","last_peered":"2024-05-22T21:15:44.401494+0000","last_clean":"2024-05-22T21:15:44.401494+0000","last_became_active":"2024-05-22T21:15:42.973742+0000","last_became_peered":"2024-05-22T21:15:42.973742+0000","last_unstale":"2024-05-22T21:15:44.401494+0000","last_undegraded":"2024-05-22T21:15:44.401494+0000","last_fullsized":"2024-05-22T21:15:44.401494+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:52:50.421947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"66'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.949180+0000","last_change":"2024-05-22T21:15:42.584480+0000","last_active":"2024-05-22T21:15:44.949180+0000","last_peered":"2024-05-22T21:15:44.949180+0000","last_clean":"2024-05-22T21:15:44.949180+0000","last_became_active":"2024-05-22T21:15:42.581781+0000","last_became_peered":"2024-05-22T21:15:42.581781+0000","last_unstale":"2024-05-22T21:15:44.949180+0000","last_undegraded":"2024-05-22T21:15:44.949180+0000","last_fullsized":"2024-05-22T21:15:44.949180+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:20:32.612869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064479+0000","last_change":"2024-05-22T21:15:42.976906+0000","last_active":"2024-05-22T21:15:44.064479+0000","last_peered":"2024-05-22T21:15:44.064479+0000","last_clean":"2024-05-22T21:15:44.064479+0000","last_became_active":"2024-05-22T21:15:42.976700+0000","last_became_peered":"2024-05-22T21:15:42.976700+0000","last_unstale":"2024-05-22T21:15:44.064479+0000","last_undegraded":"2024-05-22T21:15:44.064479+0000","last_fullsized":"2024-05-22T21:15:44.064479+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:33:48.422387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"66'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401690+0000","last_change":"2024-05-22T21:15:42.583988+0000","last_active":"2024-05-22T21:15:44.401690+0000","last_peered":"2024-05-22T21:15:44.401690+0000","last_clean":"2024-05-22T21:15:44.401690+0000","last_became_active":"2024-05-22T21:15:42.583426+0000","last_became_peered":"2024-05-22T21:15:42.583426+0000","last_unstale":"2024-05-22T21:15:44.401690+0000","last_undegraded":"2024-05-22T21:15:44.401690+0000","last_fullsized":"2024-05-22T21:15:44.401690+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:07.614742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963537+0000","last_change":"2024-05-22T21:15:42.970557+0000","last_active":"2024-05-22T21:15:43.963537+0000","last_peered":"2024-05-22T21:15:43.963537+0000","last_clean":"2024-05-22T21:15:43.963537+0000","last_became_active":"2024-05-22T21:15:42.969968+0000","last_became_peered":"2024-05-22T21:15:42.969968+0000","last_unstale":"2024-05-22T21:15:43.963537+0000","last_undegraded":"2024-05-22T21:15:43.963537+0000","last_fullsized":"2024-05-22T21:15:43.963537+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:52:50.421947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948920+0000","last_change":"2024-05-22T21:15:42.574470+0000","last_active":"2024-05-22T21:15:44.948920+0000","last_peered":"2024-05-22T21:15:44.948920+0000","last_clean":"2024-05-22T21:15:44.948920+0000","last_became_active":"2024-05-22T21:15:42.574189+0000","last_became_peered":"2024-05-22T21:15:42.574189+0000","last_unstale":"2024-05-22T21:15:44.948920+0000","last_undegraded":"2024-05-22T21:15:44.948920+0000","last_fullsized":"2024-05-22T21:15:44.948920+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:29:46.814694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401672+0000","last_change":"2024-05-22T21:15:42.974200+0000","last_active":"2024-05-22T21:15:44.401672+0000","last_peered":"2024-05-22T21:15:44.401672+0000","last_clean":"2024-05-22T21:15:44.401672+0000","last_became_active":"2024-05-22T21:15:42.973452+0000","last_became_peered":"2024-05-22T21:15:42.973452+0000","last_unstale":"2024-05-22T21:15:44.401672+0000","last_undegraded":"2024-05-22T21:15:44.401672+0000","last_fullsized":"2024-05-22T21:15:44.401672+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:51:38.084205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"66'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064427+0000","last_change":"2024-05-22T21:15:42.580675+0000","last_active":"2024-05-22T21:15:44.064427+0000","last_peered":"2024-05-22T21:15:44.064427+0000","last_clean":"2024-05-22T21:15:44.064427+0000","last_became_active":"2024-05-22T21:15:42.578358+0000","last_became_peered":"2024-05-22T21:15:42.578358+0000","last_unstale":"2024-05-22T21:15:44.064427+0000","last_undegraded":"2024-05-22T21:15:44.064427+0000","last_fullsized":"2024-05-22T21:15:44.064427+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:38:54.284906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948970+0000","last_change":"2024-05-22T21:15:42.971558+0000","last_active":"2024-05-22T21:15:44.948970+0000","last_peered":"2024-05-22T21:15:44.948970+0000","last_clean":"2024-05-22T21:15:44.948970+0000","last_became_active":"2024-05-22T21:15:42.971346+0000","last_became_peered":"2024-05-22T21:15:42.971346+0000","last_unstale":"2024-05-22T21:15:44.948970+0000","last_undegraded":"2024-05-22T21:15:44.948970+0000","last_fullsized":"2024-05-22T21:15:44.948970+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:55.539734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961809+0000","last_change":"2024-05-22T21:15:42.581482+0000","last_active":"2024-05-22T21:15:43.961809+0000","last_peered":"2024-05-22T21:15:43.961809+0000","last_clean":"2024-05-22T21:15:43.961809+0000","last_became_active":"2024-05-22T21:15:42.581106+0000","last_became_peered":"2024-05-22T21:15:42.581106+0000","last_unstale":"2024-05-22T21:15:43.961809+0000","last_undegraded":"2024-05-22T21:15:43.961809+0000","last_fullsized":"2024-05-22T21:15:43.961809+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:48:28.713342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948998+0000","last_change":"2024-05-22T21:15:42.966903+0000","last_active":"2024-05-22T21:15:44.948998+0000","last_peered":"2024-05-22T21:15:44.948998+0000","last_clean":"2024-05-22T21:15:44.948998+0000","last_became_active":"2024-05-22T21:15:42.966665+0000","last_became_peered":"2024-05-22T21:15:42.966665+0000","last_unstale":"2024-05-22T21:15:44.948998+0000","last_undegraded":"2024-05-22T21:15:44.948998+0000","last_fullsized":"2024-05-22T21:15:44.948998+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:49:33.826508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401740+0000","last_change":"2024-05-22T21:15:42.582430+0000","last_active":"2024-05-22T21:15:44.401740+0000","last_peered":"2024-05-22T21:15:44.401740+0000","last_clean":"2024-05-22T21:15:44.401740+0000","last_became_active":"2024-05-22T21:15:42.582175+0000","last_became_peered":"2024-05-22T21:15:42.582175+0000","last_unstale":"2024-05-22T21:15:44.401740+0000","last_undegraded":"2024-05-22T21:15:44.401740+0000","last_fullsized":"2024-05-22T21:15:44.401740+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:43:22.868781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401190+0000","last_change":"2024-05-22T21:15:42.976891+0000","last_active":"2024-05-22T21:15:44.401190+0000","last_peered":"2024-05-22T21:15:44.401190+0000","last_clean":"2024-05-22T21:15:44.401190+0000","last_became_active":"2024-05-22T21:15:42.976514+0000","last_became_peered":"2024-05-22T21:15:42.976514+0000","last_unstale":"2024-05-22T21:15:44.401190+0000","last_undegraded":"2024-05-22T21:15:44.401190+0000","last_fullsized":"2024-05-22T21:15:44.401190+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:07:20.924146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"66'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961604+0000","last_change":"2024-05-22T21:15:42.583164+0000","last_active":"2024-05-22T21:15:43.961604+0000","last_peered":"2024-05-22T21:15:43.961604+0000","last_clean":"2024-05-22T21:15:43.961604+0000","last_became_active":"2024-05-22T21:15:42.582561+0000","last_became_peered":"2024-05-22T21:15:42.582561+0000","last_unstale":"2024-05-22T21:15:43.961604+0000","last_undegraded":"2024-05-22T21:15:43.961604+0000","last_fullsized":"2024-05-22T21:15:43.961604+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:02:42.940946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401388+0000","last_change":"2024-05-22T21:15:42.979017+0000","last_active":"2024-05-22T21:15:44.401388+0000","last_peered":"2024-05-22T21:15:44.401388+0000","last_clean":"2024-05-22T21:15:44.401388+0000","last_became_active":"2024-05-22T21:15:42.975692+0000","last_became_peered":"2024-05-22T21:15:42.975692+0000","last_unstale":"2024-05-22T21:15:44.401388+0000","last_undegraded":"2024-05-22T21:15:44.401388+0000","last_fullsized":"2024-05-22T21:15:44.401388+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:46:40.357385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.949261+0000","last_change":"2024-05-22T21:15:42.582854+0000","last_active":"2024-05-22T21:15:44.949261+0000","last_peered":"2024-05-22T21:15:44.949261+0000","last_clean":"2024-05-22T21:15:44.949261+0000","last_became_active":"2024-05-22T21:15:42.581206+0000","last_became_peered":"2024-05-22T21:15:42.581206+0000","last_unstale":"2024-05-22T21:15:44.949261+0000","last_undegraded":"2024-05-22T21:15:44.949261+0000","last_fullsized":"2024-05-22T21:15:44.949261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:06:27.336259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961739+0000","last_change":"2024-05-22T21:15:42.974845+0000","last_active":"2024-05-22T21:15:43.961739+0000","last_peered":"2024-05-22T21:15:43.961739+0000","last_clean":"2024-05-22T21:15:43.961739+0000","last_became_active":"2024-05-22T21:15:42.974275+0000","last_became_peered":"2024-05-22T21:15:42.974275+0000","last_unstale":"2024-05-22T21:15:43.961739+0000","last_undegraded":"2024-05-22T21:15:43.961739+0000","last_fullsized":"2024-05-22T21:15:43.961739+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:59:24.847376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064016+0000","last_change":"2024-05-22T21:15:42.585556+0000","last_active":"2024-05-22T21:15:44.064016+0000","last_peered":"2024-05-22T21:15:44.064016+0000","last_clean":"2024-05-22T21:15:44.064016+0000","last_became_active":"2024-05-22T21:15:42.585336+0000","last_became_peered":"2024-05-22T21:15:42.585336+0000","last_unstale":"2024-05-22T21:15:44.064016+0000","last_undegraded":"2024-05-22T21:15:44.064016+0000","last_fullsized":"2024-05-22T21:15:44.064016+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:02:42.940946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.948674+0000","last_change":"2024-05-22T21:15:42.975205+0000","last_active":"2024-05-22T21:15:44.948674+0000","last_peered":"2024-05-22T21:15:44.948674+0000","last_clean":"2024-05-22T21:15:44.948674+0000","last_became_active":"2024-05-22T21:15:42.974999+0000","last_became_peered":"2024-05-22T21:15:42.974999+0000","last_unstale":"2024-05-22T21:15:44.948674+0000","last_undegraded":"2024-05-22T21:15:44.948674+0000","last_fullsized":"2024-05-22T21:15:44.948674+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:29:50.850273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.401583+0000","last_change":"2024-05-22T21:15:42.583555+0000","last_active":"2024-05-22T21:15:44.401583+0000","last_peered":"2024-05-22T21:15:44.401583+0000","last_clean":"2024-05-22T21:15:44.401583+0000","last_became_active":"2024-05-22T21:15:42.583165+0000","last_became_peered":"2024-05-22T21:15:42.583165+0000","last_unstale":"2024-05-22T21:15:44.401583+0000","last_undegraded":"2024-05-22T21:15:44.401583+0000","last_fullsized":"2024-05-22T21:15:44.401583+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:06:27.336259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064610+0000","last_change":"2024-05-22T21:15:42.975377+0000","last_active":"2024-05-22T21:15:44.064610+0000","last_peered":"2024-05-22T21:15:44.064610+0000","last_clean":"2024-05-22T21:15:44.064610+0000","last_became_active":"2024-05-22T21:15:42.970618+0000","last_became_peered":"2024-05-22T21:15:42.970618+0000","last_unstale":"2024-05-22T21:15:44.064610+0000","last_undegraded":"2024-05-22T21:15:44.064610+0000","last_fullsized":"2024-05-22T21:15:44.064610+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:40:02.153083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961836+0000","last_change":"2024-05-22T21:15:42.582957+0000","last_active":"2024-05-22T21:15:43.961836+0000","last_peered":"2024-05-22T21:15:43.961836+0000","last_clean":"2024-05-22T21:15:43.961836+0000","last_became_active":"2024-05-22T21:15:42.582489+0000","last_became_peered":"2024-05-22T21:15:42.582489+0000","last_unstale":"2024-05-22T21:15:43.961836+0000","last_undegraded":"2024-05-22T21:15:43.961836+0000","last_fullsized":"2024-05-22T21:15:43.961836+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:39.462116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963623+0000","last_change":"2024-05-22T21:15:42.970227+0000","last_active":"2024-05-22T21:15:43.963623+0000","last_peered":"2024-05-22T21:15:43.963623+0000","last_clean":"2024-05-22T21:15:43.963623+0000","last_became_active":"2024-05-22T21:15:42.969366+0000","last_became_peered":"2024-05-22T21:15:42.969366+0000","last_unstale":"2024-05-22T21:15:43.963623+0000","last_undegraded":"2024-05-22T21:15:43.963623+0000","last_fullsized":"2024-05-22T21:15:43.963623+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:04:14.349263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.963257+0000","last_change":"2024-05-22T21:15:42.583052+0000","last_active":"2024-05-22T21:15:43.963257+0000","last_peered":"2024-05-22T21:15:43.963257+0000","last_clean":"2024-05-22T21:15:43.963257+0000","last_became_active":"2024-05-22T21:15:42.582824+0000","last_became_peered":"2024-05-22T21:15:42.582824+0000","last_unstale":"2024-05-22T21:15:43.963257+0000","last_undegraded":"2024-05-22T21:15:43.963257+0000","last_fullsized":"2024-05-22T21:15:43.963257+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:58:52.048265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961737+0000","last_change":"2024-05-22T21:15:42.972158+0000","last_active":"2024-05-22T21:15:43.961737+0000","last_peered":"2024-05-22T21:15:43.961737+0000","last_clean":"2024-05-22T21:15:43.961737+0000","last_became_active":"2024-05-22T21:15:42.971929+0000","last_became_peered":"2024-05-22T21:15:42.971929+0000","last_unstale":"2024-05-22T21:15:43.961737+0000","last_undegraded":"2024-05-22T21:15:43.961737+0000","last_fullsized":"2024-05-22T21:15:43.961737+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:16:54.900284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:43.961579+0000","last_change":"2024-05-22T21:15:42.585366+0000","last_active":"2024-05-22T21:15:43.961579+0000","last_peered":"2024-05-22T21:15:43.961579+0000","last_clean":"2024-05-22T21:15:43.961579+0000","last_became_active":"2024-05-22T21:15:42.585137+0000","last_became_peered":"2024-05-22T21:15:42.585137+0000","last_unstale":"2024-05-22T21:15:43.961579+0000","last_undegraded":"2024-05-22T21:15:43.961579+0000","last_fullsized":"2024-05-22T21:15:43.961579+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:42:39.578833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064178+0000","last_change":"2024-05-22T21:15:42.978290+0000","last_active":"2024-05-22T21:15:44.064178+0000","last_peered":"2024-05-22T21:15:44.064178+0000","last_clean":"2024-05-22T21:15:44.064178+0000","last_became_active":"2024-05-22T21:15:42.973526+0000","last_became_peered":"2024-05-22T21:15:42.973526+0000","last_unstale":"2024-05-22T21:15:44.064178+0000","last_undegraded":"2024-05-22T21:15:44.064178+0000","last_fullsized":"2024-05-22T21:15:44.064178+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:34:22.363001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064374+0000","last_change":"2024-05-22T21:15:42.587015+0000","last_active":"2024-05-22T21:15:44.064374+0000","last_peered":"2024-05-22T21:15:44.064374+0000","last_clean":"2024-05-22T21:15:44.064374+0000","last_became_active":"2024-05-22T21:15:42.582539+0000","last_became_peered":"2024-05-22T21:15:42.582539+0000","last_unstale":"2024-05-22T21:15:44.064374+0000","last_undegraded":"2024-05-22T21:15:44.064374+0000","last_fullsized":"2024-05-22T21:15:44.064374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:04:13.342057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.064471+0000","last_change":"2024-05-22T21:15:42.973023+0000","last_active":"2024-05-22T21:15:44.064471+0000","last_peered":"2024-05-22T21:15:44.064471+0000","last_clean":"2024-05-22T21:15:44.064471+0000","last_became_active":"2024-05-22T21:15:42.972799+0000","last_became_peered":"2024-05-22T21:15:42.972799+0000","last_unstale":"2024-05-22T21:15:44.064471+0000","last_undegraded":"2024-05-22T21:15:44.064471+0000","last_fullsized":"2024-05-22T21:15:44.064471+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:42.562562+0000","last_clean_scrub_stamp":"2024-05-22T21:15:42.562562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:38:31.751953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-22T21:15:44.063972+0000","last_change":"2024-05-22T21:15:42.582878+0000","last_active":"2024-05-22T21:15:44.063972+0000","last_peered":"2024-05-22T21:15:44.063972+0000","last_clean":"2024-05-22T21:15:44.063972+0000","last_became_active":"2024-05-22T21:15:42.582664+0000","last_became_peered":"2024-05-22T21:15:42.582664+0000","last_unstale":"2024-05-22T21:15:44.063972+0000","last_undegraded":"2024-05-22T21:15:44.063972+0000","last_fullsized":"2024-05-22T21:15:44.063972+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:15:41.555356+0000","last_clean_scrub_stamp":"2024-05-22T21:15:41.555356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:48:28.713342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-05-22T21:15:42.962476+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:15:42.962476+0000","last_peered":"2024-05-22T21:15:42.962476+0000","last_clean":"2024-05-22T21:15:42.962476+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:15:42.962476+0000","last_undegraded":"2024-05-22T21:15:42.962476+0000","last_fullsized":"2024-05-22T21:15:42.962476+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":913,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194399,"statfs":{"total":200000000,"available":199065002,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.751,"15min":0.77000000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.998,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68799999999999994,"15min":0.68700000000000006},"min":{"1min":0.44700000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.86899999999999999,"5min":1.889,"15min":1.889},"last":0.48599999999999999}]},{"osd":1,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.48099999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93000000000000005,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.753,"15min":0.77300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86499999999999999,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.40300000000000002}]},{"osd":2,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.51100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90600000000000003,"5min":1.923,"15min":1.923},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71199999999999997,"15min":0.71599999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93600000000000005,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.74399999999999999}]},{"osd":3,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.83799999999999997,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.443},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.78700000000000003,"15min":0.80300000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.95499999999999996,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.78000000000000003}]},{"osd":4,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61199999999999999,"15min":0.60899999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.71999999999999997}]},{"osd":5,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64200000000000002,"15min":0.64000000000000001},"min":{"1min":0.43099999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.71099999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64200000000000002,"15min":0.63400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655530,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:15:11 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.48099999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.88600000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.68100000000000005},"min":{"1min":0.47399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90300000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.90500000000000003}]},{"osd":1,"last update":"Wed May 22 21:15:11 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68500000000000005,"15min":0.67500000000000004},"min":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70699999999999996,"15min":0.70599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0029999999999999,"5min":1.081,"15min":1.081},"last":0.92600000000000005}]},{"osd":2,"last update":"Wed May 22 21:15:11 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.75900000000000001,"15min":0.751},"min":{"1min":0.61899999999999999,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.73799999999999999,"15min":0.73599999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.94199999999999995,"5min":0.997,"15min":0.997},"last":0.95999999999999996}]},{"osd":3,"last update":"Wed May 22 21:15:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.499,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96599999999999997,"5min":1.415,"15min":1.415},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73599999999999999,"15min":0.73099999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.94499999999999995}]},{"osd":4,"last update":"Wed May 22 21:15:11 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64900000000000002,"15min":0.65500000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.81000000000000005,"5min":1.006,"15min":1.006},"last":0.627},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.718,"5min":2.718,"15min":2.718},"last":0.871}]},{"osd":5,"last update":"Wed May 22 21:15:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":0.79000000000000004,"15min":0.71799999999999997},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":8.5749999999999993,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.68400000000000005}]},{"osd":7,"last update":"Wed May 22 21:15:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.745,"15min":0.749},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90000000000000002,"5min":1.095,"15min":1.095},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77900000000000003,"15min":0.77400000000000002},"min":{"1min":0.52800000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2450000000000001,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.80500000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":337,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194975,"statfs":{"total":200000000,"available":199654878,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69899999999999995,"15min":0.70699999999999996},"min":{"1min":0.53400000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.83399999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.53300000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.82099999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.71599999999999997}]},{"osd":1,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70499999999999996,"15min":0.70899999999999996},"min":{"1min":0.51000000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.92200000000000004,"5min":1.216,"15min":1.216},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.51700000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.84099999999999997,"5min":1.048,"15min":1.048},"last":0.90700000000000003}]},{"osd":2,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69699999999999995,"15min":0.69199999999999995},"min":{"1min":0.56899999999999995,"5min":0.436,"15min":0.436},"max":{"1min":0.86199999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66400000000000003,"15min":0.66500000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.81299999999999994,"5min":1.252,"15min":1.252},"last":0.73499999999999999}]},{"osd":3,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70599999999999996,"15min":0.70999999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.84699999999999998,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72599999999999998,"15min":0.72699999999999998},"min":{"1min":0.55600000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.876,"5min":1.111,"15min":1.111},"last":0.84499999999999997}]},{"osd":4,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.624,"15min":0.63800000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69799999999999995,"5min":1.272,"15min":1.272},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.621,"15min":0.629},"min":{"1min":0.443,"5min":0.434,"15min":0.434},"max":{"1min":0.754,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61899999999999999}]},{"osd":6,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.624,"15min":0.626},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.65600000000000003,"15min":0.66400000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81299999999999994,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.64200000000000002}]},{"osd":7,"last update":"Wed May 22 21:15:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.71299999999999997,"15min":0.71699999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.755},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.79100000000000004,"15min":0.80100000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94599999999999995,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.58899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":337,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194974,"statfs":{"total":200000000,"available":199653966,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:15:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90300000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.70799999999999996,"15min":0.71499999999999997},"min":{"1min":0.52900000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.89100000000000001,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.61099999999999999}]},{"osd":1,"last update":"Wed May 22 21:15:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.71299999999999997,"15min":0.72699999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.85499999999999998,"5min":1.5,"15min":1.5},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.56899999999999995}]},{"osd":2,"last update":"Wed May 22 21:15:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67800000000000005,"15min":0.68100000000000005},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.90700000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.873,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.624}]},{"osd":3,"last update":"Wed May 22 21:15:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68200000000000005,"15min":0.68799999999999994},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.78500000000000003,"15min":0.79600000000000004},"min":{"1min":0.57499999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93300000000000005,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.59499999999999997}]},{"osd":5,"last update":"Wed May 22 21:15:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64200000000000002,"15min":0.64900000000000002},"min":{"1min":0.40600000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":1.012,"15min":1.012},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.48799999999999999}]},{"osd":6,"last update":"Wed May 22 21:15:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69899999999999995,"15min":0.70599999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.89300000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.51000000000000001}]},{"osd":7,"last update":"Wed May 22 21:15:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.52600000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96799999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.78600000000000003,"15min":0.79500000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98699999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.55100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655474,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.53200000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39800000000000002,"15min":0.39400000000000002},"min":{"1min":0.309,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65400000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.49399999999999999}]},{"osd":1,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.442,"15min":0.442},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48999999999999999,"15min":0.49399999999999999},"min":{"1min":0.35399999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65900000000000003,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.51600000000000001}]},{"osd":2,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48999999999999999,"15min":0.48899999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69999999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48099999999999998,"15min":0.47999999999999998},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71499999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.504,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95099999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.71199999999999997,"15min":0.70599999999999996},"min":{"1min":0.51900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.022,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.73199999999999998}]},{"osd":5,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68400000000000005,"15min":0.67400000000000004},"min":{"1min":0.48099999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.03,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.52800000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.88800000000000001,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.77900000000000003}]},{"osd":6,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.53700000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0329999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.69599999999999995,"15min":0.68700000000000006},"min":{"1min":0.59899999999999998,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91800000000000004,"5min":1.25,"15min":1.25},"last":0.69099999999999995}]},{"osd":7,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.751,"15min":0.74299999999999999},"min":{"1min":0.55800000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.0429999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.746},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.76700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655474,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39500000000000002},"min":{"1min":0.33000000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.621,"5min":0.624,"15min":0.624},"last":0.35799999999999998}]},{"osd":1,"last update":"Wed May 22 21:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.443,"15min":0.442},"min":{"1min":0.316,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.41599999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.48199999999999998}]},{"osd":3,"last update":"Wed May 22 21:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.49199999999999999,"15min":0.495},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.61899999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.44900000000000001,"15min":0.45100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.59399999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.46100000000000002}]},{"osd":4,"last update":"Wed May 22 21:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67600000000000005,"15min":0.67100000000000004},"min":{"1min":0.54300000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73699999999999999,"15min":0.73799999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85099999999999998,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.73999999999999999}]},{"osd":5,"last update":"Wed May 22 21:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68600000000000005,"15min":0.69099999999999995},"min":{"1min":0.50900000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.58299999999999996}]},{"osd":6,"last update":"Wed May 22 21:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.52800000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.76700000000000002}]},{"osd":7,"last update":"Wed May 22 21:15:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.94399999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76500000000000001,"15min":0.76400000000000001},"min":{"1min":0.628,"5min":0.621,"15min":0.621},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.66500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194976,"statfs":{"total":200000000,"available":199655440,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.43099999999999999,"15min":0.435},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59399999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.42799999999999999,"15min":0.42599999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63200000000000001,"5min":0.876,"15min":0.876},"last":0.59499999999999997}]},{"osd":2,"last update":"Wed May 22 21:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.61299999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52900000000000003,"15min":0.54200000000000004},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58899999999999997,"5min":1.923,"15min":1.923},"last":0.621}]},{"osd":3,"last update":"Wed May 22 21:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.751,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.63300000000000001,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.505}]},{"osd":4,"last update":"Wed May 22 21:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.753,"15min":0.76400000000000001},"min":{"1min":0.49299999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.95399999999999996,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.67900000000000005,"15min":0.67400000000000004},"min":{"1min":0.50600000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.03,"5min":1.3,"15min":1.3},"last":1.069}]},{"osd":5,"last update":"Wed May 22 21:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72799999999999998,"15min":0.72299999999999998},"min":{"1min":0.61199999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0009999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.028},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.752},"min":{"1min":0.52200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92300000000000004,"5min":1.97,"15min":1.97},"last":0.99099999999999999}]},{"osd":6,"last update":"Wed May 22 21:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73699999999999999,"15min":0.73599999999999999},"min":{"1min":0.53800000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0820000000000001,"5min":1.272,"15min":1.272},"last":1.085},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.59699999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0680000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.96299999999999997}]},{"osd":7,"last update":"Wed May 22 21:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96499999999999997,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0449999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.008}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":914,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194398,"statfs":{"total":200000000,"available":199063635,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.39600000000000002},"min":{"1min":0.25800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.41199999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.438}]},{"osd":2,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.45400000000000001,"15min":0.443},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.42199999999999999,"15min":0.41799999999999998},"min":{"1min":0.32400000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59599999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.433,"15min":0.42599999999999999},"min":{"1min":0.33100000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.39500000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.496}]},{"osd":4,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.012,"5min":1.244,"15min":1.244},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0640000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.55900000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.57899999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.056,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.84699999999999998}]},{"osd":6,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.60399999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.028,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.998,"5min":1.5,"15min":1.5},"last":0.81000000000000005}]},{"osd":7,"last update":"Wed May 22 21:15:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.55000000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.91900000000000004,"5min":1.085,"15min":1.085},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.753,"15min":0.754},"min":{"1min":0.54400000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.115,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.71699999999999997}]}]}],"pool_statfs":[]}} 2024-05-22T21:15:47.430 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-22T21:15:47.431 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-22T21:15:47.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3176c00640 1 Processor -- start 2024-05-22T21:15:47.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3176c00640 1 -- start start 2024-05-22T21:15:47.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3176c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f317013cfc0 0x7f31701402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:47.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3176c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f317015af60 0x7f317013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:47.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3176c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f317013fc80 0x7f317015dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:47.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3176c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31701400d0 con 0x7f317013fc80 2024-05-22T21:15:47.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3176c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f317013d3a0 con 0x7f317015af60 2024-05-22T21:15:47.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3176c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31700cd920 con 0x7f317013cfc0 2024-05-22T21:15:47.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3174c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f317013fc80 0x7f317015dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:47.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3174c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f317013fc80 0x7f317015dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55726/0 (socket says 172.21.15.161:55726) 2024-05-22T21:15:47.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3174c00640 1 -- 172.21.15.161:0/775443389 learned_addr learned my addr 172.21.15.161:0/775443389 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:47.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3167400640 1 --2- 172.21.15.161:0/775443389 >> v2:172.21.15.179:3300/0 conn(0x7f317015af60 0x7f317013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:47.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f316fe00640 1 --2- 172.21.15.161:0/775443389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f317013cfc0 0x7f31701402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:47.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3174c00640 1 -- 172.21.15.161:0/775443389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f317013cfc0 msgr2=0x7f31701402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:47.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3174c00640 1 --2- 172.21.15.161:0/775443389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f317013cfc0 0x7f31701402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3174c00640 1 -- 172.21.15.161:0/775443389 >> v2:172.21.15.179:3300/0 conn(0x7f317015af60 msgr2=0x7f317013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:47.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3174c00640 1 --2- 172.21.15.161:0/775443389 >> v2:172.21.15.179:3300/0 conn(0x7f317015af60 0x7f317013f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f3174c00640 1 -- 172.21.15.161:0/775443389 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31700cdc00 con 0x7f317013fc80 2024-05-22T21:15:47.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.609+0000 7f316fe00640 1 --2- 172.21.15.161:0/775443389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f317013cfc0 0x7f31701402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:47.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.613+0000 7f3174c00640 1 --2- 172.21.15.161:0/775443389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317013fc80 0x7f317015dad0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f31700723a0 tx=0x7f316801c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4b63a53fcc0449 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:47.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.613+0000 7f316f400640 1 -- 172.21.15.161:0/775443389 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31680a6500 con 0x7f317013fc80 2024-05-22T21:15:47.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.613+0000 7f316f400640 1 -- 172.21.15.161:0/775443389 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31680a6660 con 0x7f317013fc80 2024-05-22T21:15:47.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.613+0000 7f316f400640 1 -- 172.21.15.161:0/775443389 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31680a69f0 con 0x7f317013fc80 2024-05-22T21:15:47.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.613+0000 7f3176c00640 1 -- 172.21.15.161:0/775443389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317013fc80 msgr2=0x7f317015dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:47.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.613+0000 7f3176c00640 1 --2- 172.21.15.161:0/775443389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317013fc80 0x7f317015dad0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f31700723a0 tx=0x7f316801c670 comp rx=0 tx=0).stop 2024-05-22T21:15:47.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.613+0000 7f3176c00640 1 -- 172.21.15.161:0/775443389 shutdown_connections 2024-05-22T21:15:47.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.613+0000 7f3176c00640 1 --2- 172.21.15.161:0/775443389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317013fc80 0x7f317015dad0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.613+0000 7f3176c00640 1 --2- 172.21.15.161:0/775443389 >> v2:172.21.15.179:3300/0 conn(0x7f317015af60 0x7f317013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.613+0000 7f3176c00640 1 --2- 172.21.15.161:0/775443389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f317013cfc0 0x7f31701402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.613+0000 7f3176c00640 1 -- 172.21.15.161:0/775443389 >> 172.21.15.161:0/775443389 conn(0x7f317006a2d0 msgr2=0x7f317006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:47.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.613+0000 7f3176c00640 1 -- 172.21.15.161:0/775443389 shutdown_connections 2024-05-22T21:15:47.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.613+0000 7f3176c00640 1 -- 172.21.15.161:0/775443389 wait complete. 2024-05-22T21:15:47.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3176c00640 1 Processor -- start 2024-05-22T21:15:47.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3176c00640 1 -- start start 2024-05-22T21:15:47.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3176c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f317013cfc0 0x7f31701369f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:47.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3176c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f317013fc80 0x7f31700b3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:47.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3176c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317015af60 0x7f317015c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:47.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3176c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31700b3bd0 con 0x7f317015af60 2024-05-22T21:15:47.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3176c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31701f4020 con 0x7f317013fc80 2024-05-22T21:15:47.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3176c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31700b38f0 con 0x7f317013cfc0 2024-05-22T21:15:47.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3174c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317015af60 0x7f317015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:47.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3174c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317015af60 0x7f317015c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55740/0 (socket says 172.21.15.161:55740) 2024-05-22T21:15:47.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3174c00640 1 -- 172.21.15.161:0/431828910 learned_addr learned my addr 172.21.15.161:0/431828910 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:47.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f316fe00640 1 --2- 172.21.15.161:0/431828910 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f317013cfc0 0x7f31701369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:47.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3167400640 1 --2- 172.21.15.161:0/431828910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f317013fc80 0x7f31700b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:47.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3174c00640 1 -- 172.21.15.161:0/431828910 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f317013cfc0 msgr2=0x7f31701369f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:47.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3174c00640 1 --2- 172.21.15.161:0/431828910 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f317013cfc0 0x7f31701369f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3174c00640 1 -- 172.21.15.161:0/431828910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f317013fc80 msgr2=0x7f31700b3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:47.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3174c00640 1 --2- 172.21.15.161:0/431828910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f317013fc80 0x7f31700b3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3174c00640 1 -- 172.21.15.161:0/431828910 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3168009b30 con 0x7f317015af60 2024-05-22T21:15:47.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3174c00640 1 --2- 172.21.15.161:0/431828910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317015af60 0x7f317015c160 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f316801ad20 tx=0x7f3168019410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:47.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f316cc00640 1 -- 172.21.15.161:0/431828910 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31680a6be0 con 0x7f317015af60 2024-05-22T21:15:47.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f316cc00640 1 -- 172.21.15.161:0/431828910 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31680a6d40 con 0x7f317015af60 2024-05-22T21:15:47.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f316cc00640 1 -- 172.21.15.161:0/431828910 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31680af6c0 con 0x7f317015af60 2024-05-22T21:15:47.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3176c00640 1 -- 172.21.15.161:0/431828910 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3170140410 con 0x7f317015af60 2024-05-22T21:15:47.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.617+0000 7f3176c00640 1 -- 172.21.15.161:0/431828910 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31701409d0 con 0x7f317015af60 2024-05-22T21:15:47.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.621+0000 7f3176c00640 1 -- 172.21.15.161:0/431828910 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3170140270 con 0x7f317015af60 2024-05-22T21:15:47.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.621+0000 7f316cc00640 1 -- 172.21.15.161:0/431828910 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f31680af820 con 0x7f317015af60 2024-05-22T21:15:47.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.621+0000 7f316cc00640 1 --2- 172.21.15.161:0/431828910 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f314c066690 0x7f314c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:47.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.621+0000 7f316cc00640 1 -- 172.21.15.161:0/431828910 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31680aa070 con 0x7f317015af60 2024-05-22T21:15:47.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.621+0000 7f316fe00640 1 --2- 172.21.15.161:0/431828910 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f314c066690 0x7f314c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:47.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.621+0000 7f316fe00640 1 --2- 172.21.15.161:0/431828910 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f314c066690 0x7f314c068b50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3160006fd0 tx=0x7f3160008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:47.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.625+0000 7f316cc00640 1 -- 172.21.15.161:0/431828910 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3170140270 con 0x7f317015af60 2024-05-22T21:15:47.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.833+0000 7f3176c00640 1 -- 172.21.15.161:0/431828910 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3170140270 con 0x7f317015af60 2024-05-22T21:15:47.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.837+0000 7f316cc00640 1 -- 172.21.15.161:0/431828910 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f3170140270 con 0x7f317015af60 2024-05-22T21:15:47.844 INFO:teuthology.orchestra.run.smithi161.stderr:updated caps for client.0 2024-05-22T21:15:47.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.841+0000 7f3176c00640 1 -- 172.21.15.161:0/431828910 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f314c066690 msgr2=0x7f314c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:47.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.841+0000 7f3176c00640 1 --2- 172.21.15.161:0/431828910 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f314c066690 0x7f314c068b50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3160006fd0 tx=0x7f3160008040 comp rx=0 tx=0).stop 2024-05-22T21:15:47.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.841+0000 7f3176c00640 1 -- 172.21.15.161:0/431828910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317015af60 msgr2=0x7f317015c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:47.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.841+0000 7f3176c00640 1 --2- 172.21.15.161:0/431828910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317015af60 0x7f317015c160 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f316801ad20 tx=0x7f3168019410 comp rx=0 tx=0).stop 2024-05-22T21:15:47.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.841+0000 7f3176c00640 1 -- 172.21.15.161:0/431828910 shutdown_connections 2024-05-22T21:15:47.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.841+0000 7f3176c00640 1 --2- 172.21.15.161:0/431828910 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f314c066690 0x7f314c068b50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.841+0000 7f3176c00640 1 --2- 172.21.15.161:0/431828910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317015af60 0x7f317015c160 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.841+0000 7f3176c00640 1 --2- 172.21.15.161:0/431828910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f317013fc80 0x7f31700b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.841+0000 7f3176c00640 1 --2- 172.21.15.161:0/431828910 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f317013cfc0 0x7f31701369f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:47.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.841+0000 7f3176c00640 1 -- 172.21.15.161:0/431828910 >> 172.21.15.161:0/431828910 conn(0x7f317006a2d0 msgr2=0x7f317013b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:47.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.841+0000 7f3176c00640 1 -- 172.21.15.161:0/431828910 shutdown_connections 2024-05-22T21:15:47.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.841+0000 7f3176c00640 1 -- 172.21.15.161:0/431828910 wait complete. 2024-05-22T21:15:47.866 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:15:48.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab77e00640 1 Processor -- start 2024-05-22T21:15:48.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab77e00640 1 -- start start 2024-05-22T21:15:48.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab77e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fab7015dd50 0x7fab7006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab77e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fab7015d5e0 0x7fab7015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab77e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab70071670 con 0x7fab7015e130 2024-05-22T21:15:48.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab77e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab7006de40 con 0x7fab7015d5e0 2024-05-22T21:15:48.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab77e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab70070a60 con 0x7fab7015dd50 2024-05-22T21:15:48.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab74a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fab7015e130 0x7fab7015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51874/0 (socket says 172.21.15.161:51874) 2024-05-22T21:15:48.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab74a00640 1 -- 172.21.15.161:0/1871151198 learned_addr learned my addr 172.21.15.161:0/1871151198 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:48.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab75e00640 1 --2- 172.21.15.161:0/1871151198 >> v2:172.21.15.179:3300/0 conn(0x7fab7015d5e0 0x7fab7015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab75400640 1 --2- 172.21.15.161:0/1871151198 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fab7015dd50 0x7fab7006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab6fe00640 1 -- 172.21.15.161:0/1871151198 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923009713 0 0) 0x7fab70071670 con 0x7fab7015e130 2024-05-22T21:15:48.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab6fe00640 1 -- 172.21.15.161:0/1871151198 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab58003530 con 0x7fab7015e130 2024-05-22T21:15:48.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab75e00640 1 -- 172.21.15.161:0/1871151198 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fab7015dd50 msgr2=0x7fab7006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:48.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab75e00640 1 --2- 172.21.15.161:0/1871151198 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fab7015dd50 0x7fab7006d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab75e00640 1 -- 172.21.15.161:0/1871151198 >> v1:172.21.15.161:6789/0 conn(0x7fab7015e130 legacy=0x7fab7015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:47.997+0000 7fab75e00640 1 -- 172.21.15.161:0/1871151198 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab70070d40 con 0x7fab7015d5e0 2024-05-22T21:15:48.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.001+0000 7fab75e00640 1 --2- 172.21.15.161:0/1871151198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab7015d5e0 0x7fab7015b6c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fab68009d90 tx=0x7fab6801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=462ba984a05fe16 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:48.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.001+0000 7fab6fe00640 1 -- 172.21.15.161:0/1871151198 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab680a5970 con 0x7fab7015d5e0 2024-05-22T21:15:48.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.001+0000 7fab6fe00640 1 -- 172.21.15.161:0/1871151198 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab680a5ad0 con 0x7fab7015d5e0 2024-05-22T21:15:48.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.001+0000 7fab6fe00640 1 -- 172.21.15.161:0/1871151198 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab680a5e60 con 0x7fab7015d5e0 2024-05-22T21:15:48.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.001+0000 7fab77e00640 1 -- 172.21.15.161:0/1871151198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab7015d5e0 msgr2=0x7fab7015b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.001+0000 7fab77e00640 1 --2- 172.21.15.161:0/1871151198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab7015d5e0 0x7fab7015b6c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fab68009d90 tx=0x7fab6801c8b0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.001+0000 7fab77e00640 1 -- 172.21.15.161:0/1871151198 shutdown_connections 2024-05-22T21:15:48.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.001+0000 7fab77e00640 1 --2- 172.21.15.161:0/1871151198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab7015d5e0 0x7fab7015b6c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.001+0000 7fab77e00640 1 --2- 172.21.15.161:0/1871151198 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fab7015dd50 0x7fab7006d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.001+0000 7fab77e00640 1 -- 172.21.15.161:0/1871151198 >> 172.21.15.161:0/1871151198 conn(0x7fab700b0bc0 msgr2=0x7fab700b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:48.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.001+0000 7fab77e00640 1 -- 172.21.15.161:0/1871151198 shutdown_connections 2024-05-22T21:15:48.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.001+0000 7fab77e00640 1 -- 172.21.15.161:0/1871151198 wait complete. 2024-05-22T21:15:48.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab77e00640 1 Processor -- start 2024-05-22T21:15:48.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab77e00640 1 -- start start 2024-05-22T21:15:48.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab77e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fab7015d5e0 0x7fab700cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab77e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab7015dd50 0x7fab7014ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab77e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab7015e130 0x7fab7014d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab77e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab701f9f10 con 0x7fab7015dd50 2024-05-22T21:15:48.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab77e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab70158480 con 0x7fab7015e130 2024-05-22T21:15:48.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab77e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab70158760 con 0x7fab7015d5e0 2024-05-22T21:15:48.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab74a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab7015dd50 0x7fab7014ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab74a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab7015dd50 0x7fab7014ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55754/0 (socket says 172.21.15.161:55754) 2024-05-22T21:15:48.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab74a00640 1 -- 172.21.15.161:0/3234824586 learned_addr learned my addr 172.21.15.161:0/3234824586 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:48.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab75400640 1 --2- 172.21.15.161:0/3234824586 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fab7015d5e0 0x7fab700cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab75e00640 1 --2- 172.21.15.161:0/3234824586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab7015e130 0x7fab7014d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab74a00640 1 -- 172.21.15.161:0/3234824586 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fab7015d5e0 msgr2=0x7fab700cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab74a00640 1 --2- 172.21.15.161:0/3234824586 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fab7015d5e0 0x7fab700cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab74a00640 1 -- 172.21.15.161:0/3234824586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab7015e130 msgr2=0x7fab7014d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab74a00640 1 --2- 172.21.15.161:0/3234824586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab7015e130 0x7fab7014d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab74a00640 1 -- 172.21.15.161:0/3234824586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab68009a70 con 0x7fab7015dd50 2024-05-22T21:15:48.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab74a00640 1 --2- 172.21.15.161:0/3234824586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab7015dd50 0x7fab7014ce90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00c3c0 tx=0x7fab5c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:48.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab6d600640 1 -- 172.21.15.161:0/3234824586 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c005cf0 con 0x7fab7015dd50 2024-05-22T21:15:48.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab6d600640 1 -- 172.21.15.161:0/3234824586 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c005e50 con 0x7fab7015dd50 2024-05-22T21:15:48.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab77e00640 1 -- 172.21.15.161:0/3234824586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab70158a40 con 0x7fab7015dd50 2024-05-22T21:15:48.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab6d600640 1 -- 172.21.15.161:0/3234824586 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0072f0 con 0x7fab7015dd50 2024-05-22T21:15:48.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab77e00640 1 -- 172.21.15.161:0/3234824586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab700ce490 con 0x7fab7015dd50 2024-05-22T21:15:48.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.005+0000 7fab77e00640 1 -- 172.21.15.161:0/3234824586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab70067220 con 0x7fab7015dd50 2024-05-22T21:15:48.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.009+0000 7fab6d600640 1 -- 172.21.15.161:0/3234824586 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fab5c007450 con 0x7fab7015dd50 2024-05-22T21:15:48.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.009+0000 7fab6d600640 1 --2- 172.21.15.161:0/3234824586 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fab50066690 0x7fab50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.009+0000 7fab6d600640 1 -- 172.21.15.161:0/3234824586 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab5c083a80 con 0x7fab7015dd50 2024-05-22T21:15:48.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.009+0000 7fab75400640 1 --2- 172.21.15.161:0/3234824586 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fab50066690 0x7fab50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.009+0000 7fab75400640 1 --2- 172.21.15.161:0/3234824586 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fab50066690 0x7fab50068b50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fab64009b60 tx=0x7fab64009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:48.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.013+0000 7fab6d600640 1 -- 172.21.15.161:0/3234824586 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fab5c050f80 con 0x7fab7015dd50 2024-05-22T21:15:48.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.197+0000 7fab77e00640 1 -- 172.21.15.161:0/3234824586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab7006d8a0 con 0x7fab7015dd50 2024-05-22T21:15:48.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.197+0000 7fab6d600640 1 -- 172.21.15.161:0/3234824586 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4743 (secure 0 0 0) 0x7fab5c017070 con 0x7fab7015dd50 2024-05-22T21:15:48.204 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:48.204 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1620021722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":1620021722},{"type":"v1","addr":"172.21.15.179:6835","nonce":1620021722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/3548738614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3548738614},{"type":"v1","addr":"172.21.15.161:6836","nonce":3548738614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/400898801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":400898801},{"type":"v1","addr":"172.21.15.161:6837","nonce":400898801}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:15:42.955245+0000","modified":"2024-05-22T21:15:45.977718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6834/2239627963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2239627963},{"type":"v1","addr":"172.21.15.179:6834","nonce":2239627963}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-22T21:15:48.204 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 31 2024-05-22T21:15:48.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.197+0000 7fab77e00640 1 -- 172.21.15.161:0/3234824586 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fab50066690 msgr2=0x7fab50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.197+0000 7fab77e00640 1 --2- 172.21.15.161:0/3234824586 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fab50066690 0x7fab50068b50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fab64009b60 tx=0x7fab64009190 comp rx=0 tx=0).stop 2024-05-22T21:15:48.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.197+0000 7fab77e00640 1 -- 172.21.15.161:0/3234824586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab7015dd50 msgr2=0x7fab7014ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.197+0000 7fab77e00640 1 --2- 172.21.15.161:0/3234824586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab7015dd50 0x7fab7014ce90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00c3c0 tx=0x7fab5c00c880 comp rx=0 tx=0).stop 2024-05-22T21:15:48.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.197+0000 7fab77e00640 1 -- 172.21.15.161:0/3234824586 shutdown_connections 2024-05-22T21:15:48.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.197+0000 7fab77e00640 1 --2- 172.21.15.161:0/3234824586 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fab50066690 0x7fab50068b50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.197+0000 7fab77e00640 1 --2- 172.21.15.161:0/3234824586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab7015e130 0x7fab7014d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.197+0000 7fab77e00640 1 --2- 172.21.15.161:0/3234824586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab7015dd50 0x7fab7014ce90 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.197+0000 7fab77e00640 1 --2- 172.21.15.161:0/3234824586 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fab7015d5e0 0x7fab700cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.197+0000 7fab77e00640 1 -- 172.21.15.161:0/3234824586 >> 172.21.15.161:0/3234824586 conn(0x7fab700b0bc0 msgr2=0x7fab700af010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:48.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.201+0000 7fab77e00640 1 -- 172.21.15.161:0/3234824586 shutdown_connections 2024-05-22T21:15:48.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.201+0000 7fab77e00640 1 -- 172.21.15.161:0/3234824586 wait complete. 2024-05-22T21:15:48.556 INFO:tasks.ceph.mds.d.smithi179.stderr:2024-05-22T21:15:48.554+0000 7fd769a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T21:15:48.627 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-22T21:15:42.955245+0000', 'modified': '2024-05-22T21:15:45.977718+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5946}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5946': {'gid': 5946, 'name': 'd', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.179:6834/2239627963', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6832', 'nonce': 2239627963}, {'type': 'v1', 'addr': '172.21.15.179:6834', 'nonce': 2239627963}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-22T21:15:48.627 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-22T21:15:48.628 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-22T21:15:48.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f770d800640 1 Processor -- start 2024-05-22T21:15:48.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f770d800640 1 -- start start 2024-05-22T21:15:48.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f770d800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f77081596b0 0x7f7708071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f770d800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770815d250 con 0x7f770806d240 2024-05-22T21:15:48.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f770d800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7708159a90 con 0x7f7708071db0 2024-05-22T21:15:48.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f770d800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770815d510 con 0x7f77081596b0 2024-05-22T21:15:48.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7707400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f770806d240 0x7f77080b2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51876/0 (socket says 172.21.15.161:51876) 2024-05-22T21:15:48.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7707400640 1 -- 172.21.15.161:0/593414184 learned_addr learned my addr 172.21.15.161:0/593414184 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:48.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7706a00640 1 --2- 172.21.15.161:0/593414184 >> v2:172.21.15.179:3300/0 conn(0x7f77081596b0 0x7f7708071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7705600640 1 -- 172.21.15.161:0/593414184 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851172123 0 0) 0x7f770815d250 con 0x7f770806d240 2024-05-22T21:15:48.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7705600640 1 -- 172.21.15.161:0/593414184 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76ec003530 con 0x7f770806d240 2024-05-22T21:15:48.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7705600640 1 -- 172.21.15.161:0/593414184 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1205272376 0 0) 0x7f76ec003530 con 0x7f770806d240 2024-05-22T21:15:48.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7705600640 1 -- 172.21.15.161:0/593414184 >> v2:172.21.15.179:3300/0 conn(0x7f77081596b0 msgr2=0x7f7708071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:48.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7705600640 1 --2- 172.21.15.161:0/593414184 >> v2:172.21.15.179:3300/0 conn(0x7f77081596b0 0x7f7708071870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7705600640 1 -- 172.21.15.161:0/593414184 >> v1:172.21.15.179:6789/0 conn(0x7f7708071db0 legacy=0x7f770806cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7705600640 1 -- 172.21.15.161:0/593414184 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770815d7f0 con 0x7f770806d240 2024-05-22T21:15:48.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7705600640 1 -- 172.21.15.161:0/593414184 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f7700001680 con 0x7f770806d240 2024-05-22T21:15:48.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7706a00640 1 --2- 172.21.15.161:0/593414184 >> v2:172.21.15.179:3300/0 conn(0x7f77081596b0 0x7f7708071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:15:48.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7705600640 1 -- 172.21.15.161:0/593414184 >> v1:172.21.15.161:6789/0 conn(0x7f770806d240 legacy=0x7f77080b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7705600640 1 --2- 172.21.15.161:0/593414184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76ec003980 0x7f76ec003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7705600640 1 --2- 172.21.15.161:0/593414184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76ec0047d0 0x7f76ec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7705600640 1 --2- 172.21.15.161:0/593414184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76ec005190 0x7f76ec00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7705600640 1 -- 172.21.15.161:0/593414184 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770815d510 con 0x7f76ec005190 2024-05-22T21:15:48.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7705600640 1 -- 172.21.15.161:0/593414184 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec00c030 con 0x7f76ec003980 2024-05-22T21:15:48.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7705600640 1 -- 172.21.15.161:0/593414184 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec00e400 con 0x7f76ec0047d0 2024-05-22T21:15:48.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7707400640 1 --2- 172.21.15.161:0/593414184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76ec005190 0x7f76ec00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7706a00640 1 --2- 172.21.15.161:0/593414184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76ec003980 0x7f76ec003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7706000640 1 --2- 172.21.15.161:0/593414184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76ec0047d0 0x7f76ec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7707400640 1 -- 172.21.15.161:0/593414184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76ec0047d0 msgr2=0x7f76ec004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:48.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7707400640 1 --2- 172.21.15.161:0/593414184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76ec0047d0 0x7f76ec004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7707400640 1 -- 172.21.15.161:0/593414184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76ec003980 msgr2=0x7f76ec003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:48.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7707400640 1 --2- 172.21.15.161:0/593414184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76ec003980 0x7f76ec003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.761+0000 7f7707400640 1 -- 172.21.15.161:0/593414184 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec003530 con 0x7f76ec005190 2024-05-22T21:15:48.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.765+0000 7f7707400640 1 -- 172.21.15.161:0/593414184 reap_dead start 2024-05-22T21:15:48.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.765+0000 7f7707400640 1 --2- 172.21.15.161:0/593414184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76ec005190 0x7f76ec00b990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f770000c1c0 tx=0x7f770001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584ee22f648d1169 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:48.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.765+0000 7f7705600640 1 -- 172.21.15.161:0/593414184 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77000a5b40 con 0x7f76ec005190 2024-05-22T21:15:48.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.765+0000 7f7705600640 1 -- 172.21.15.161:0/593414184 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77000a5ce0 con 0x7f76ec005190 2024-05-22T21:15:48.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.765+0000 7f7705600640 1 -- 172.21.15.161:0/593414184 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7700005b20 con 0x7f76ec005190 2024-05-22T21:15:48.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.765+0000 7f770d800640 1 -- 172.21.15.161:0/593414184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76ec005190 msgr2=0x7f76ec00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.765+0000 7f770d800640 1 --2- 172.21.15.161:0/593414184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76ec005190 0x7f76ec00b990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f770000c1c0 tx=0x7f770001cc40 comp rx=0 tx=0).stop 2024-05-22T21:15:48.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.765+0000 7f770d800640 1 -- 172.21.15.161:0/593414184 shutdown_connections 2024-05-22T21:15:48.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.765+0000 7f770d800640 1 --2- 172.21.15.161:0/593414184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76ec005190 0x7f76ec00b990 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.765+0000 7f770d800640 1 -- 172.21.15.161:0/593414184 >> 172.21.15.161:0/593414184 conn(0x7f770806a2d0 msgr2=0x7f770806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:48.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.765+0000 7f770d800640 1 -- 172.21.15.161:0/593414184 shutdown_connections 2024-05-22T21:15:48.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.765+0000 7f770d800640 1 -- 172.21.15.161:0/593414184 wait complete. 2024-05-22T21:15:48.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f770d800640 1 Processor -- start 2024-05-22T21:15:48.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f770d800640 1 -- start start 2024-05-22T21:15:48.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f770d800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76ec005190 0x7f770814da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f770d800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77081498b0 0x7f77081490f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f770d800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f770814f620 0x7f770814faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f770d800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770814e230 con 0x7f77081498b0 2024-05-22T21:15:48.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f770d800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7708150e00 con 0x7f76ec005190 2024-05-22T21:15:48.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f770d800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770814df60 con 0x7f770814f620 2024-05-22T21:15:48.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f7706000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77081498b0 0x7f77081490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f7706000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77081498b0 0x7f77081490f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55766/0 (socket says 172.21.15.161:55766) 2024-05-22T21:15:48.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f7706000640 1 -- 172.21.15.161:0/3070057161 learned_addr learned my addr 172.21.15.161:0/3070057161 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:48.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f7706a00640 1 --2- 172.21.15.161:0/3070057161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76ec005190 0x7f770814da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f7707400640 1 --2- 172.21.15.161:0/3070057161 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f770814f620 0x7f770814faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f7706000640 1 -- 172.21.15.161:0/3070057161 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f770814f620 msgr2=0x7f770814faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f7706000640 1 --2- 172.21.15.161:0/3070057161 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f770814f620 0x7f770814faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f7706000640 1 -- 172.21.15.161:0/3070057161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76ec005190 msgr2=0x7f770814da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f7706000640 1 --2- 172.21.15.161:0/3070057161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76ec005190 0x7f770814da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f7706000640 1 -- 172.21.15.161:0/3070057161 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7708150460 con 0x7f77081498b0 2024-05-22T21:15:48.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f7707400640 1 --2- 172.21.15.161:0/3070057161 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f770814f620 0x7f770814faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:48.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f7706a00640 1 --2- 172.21.15.161:0/3070057161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76ec005190 0x7f770814da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:48.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f7706000640 1 --2- 172.21.15.161:0/3070057161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77081498b0 0x7f77081490f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f76f400e500 tx=0x7f76f400e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:48.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f76fea00640 1 -- 172.21.15.161:0/3070057161 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f40052e0 con 0x7f77081498b0 2024-05-22T21:15:48.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f76fea00640 1 -- 172.21.15.161:0/3070057161 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f4005440 con 0x7f77081498b0 2024-05-22T21:15:48.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f76fea00640 1 -- 172.21.15.161:0/3070057161 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4003820 con 0x7f77081498b0 2024-05-22T21:15:48.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f770d800640 1 -- 172.21.15.161:0/3070057161 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7708150740 con 0x7f77081498b0 2024-05-22T21:15:48.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.769+0000 7f770d800640 1 -- 172.21.15.161:0/3070057161 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7708149630 con 0x7f77081498b0 2024-05-22T21:15:48.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.773+0000 7f76fea00640 1 -- 172.21.15.161:0/3070057161 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f76f4003980 con 0x7f77081498b0 2024-05-22T21:15:48.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.773+0000 7f76fea00640 1 --2- 172.21.15.161:0/3070057161 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76e8066690 0x7f76e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.773+0000 7f76fea00640 1 -- 172.21.15.161:0/3070057161 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76f4083ef0 con 0x7f77081498b0 2024-05-22T21:15:48.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.773+0000 7f7706a00640 1 --2- 172.21.15.161:0/3070057161 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76e8066690 0x7f76e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.773+0000 7f7710cea000 1 Processor -- start 2024-05-22T21:15:48.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.773+0000 7f7710cea000 1 -- start start 2024-05-22T21:15:48.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.773+0000 7f7706a00640 1 --2- 172.21.15.161:0/3070057161 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76e8066690 0x7f76e8068b50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f76f800a3e0 tx=0x7f76f800a040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:48.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.773+0000 7f7710cea000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb9919c0 0x55cbcb991da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.773+0000 7f7710cea000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb9922e0 0x55cbcb947980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.773+0000 7f7710cea000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb947ec0 0x55cbcb9482a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.773+0000 7f7710cea000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbcb80fe80 con 0x55cbcb9919c0 2024-05-22T21:15:48.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.773+0000 7f7710cea000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbcb79daf0 con 0x55cbcb947ec0 2024-05-22T21:15:48.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.773+0000 7f7710cea000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbcb79b0c0 con 0x55cbcb9922e0 2024-05-22T21:15:48.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.773+0000 7f7706a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb9919c0 0x55cbcb991da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.773+0000 7f7706a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb9919c0 0x55cbcb991da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55778/0 (socket says 172.21.15.161:55778) 2024-05-22T21:15:48.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.773+0000 7f7706a00640 1 -- 172.21.15.161:0/386213244 learned_addr learned my addr 172.21.15.161:0/386213244 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:48.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7706000640 1 --2- 172.21.15.161:0/386213244 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb9922e0 0x55cbcb947980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7707400640 1 --2- 172.21.15.161:0/386213244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb947ec0 0x55cbcb9482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7706a00640 1 -- 172.21.15.161:0/386213244 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb9922e0 msgr2=0x55cbcb947980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:48.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7706a00640 1 --2- 172.21.15.161:0/386213244 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb9922e0 0x55cbcb947980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7706a00640 1 -- 172.21.15.161:0/386213244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb947ec0 msgr2=0x55cbcb9482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:48.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7706a00640 1 --2- 172.21.15.161:0/386213244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb947ec0 0x55cbcb9482a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7706a00640 1 -- 172.21.15.161:0/386213244 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbcb492890 con 0x55cbcb9919c0 2024-05-22T21:15:48.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7706a00640 1 --2- 172.21.15.161:0/386213244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb9919c0 0x55cbcb991da0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f76f801b820 tx=0x7f76f800e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21e998a256dca61 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:48.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f76f3400640 1 -- 172.21.15.161:0/386213244 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f801be60 con 0x55cbcb9919c0 2024-05-22T21:15:48.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f76f3400640 1 -- 172.21.15.161:0/386213244 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f8012040 con 0x55cbcb9919c0 2024-05-22T21:15:48.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f76f3400640 1 -- 172.21.15.161:0/386213244 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8019b40 con 0x55cbcb9919c0 2024-05-22T21:15:48.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 -- 172.21.15.161:0/386213244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb9919c0 msgr2=0x55cbcb991da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 --2- 172.21.15.161:0/386213244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb9919c0 0x55cbcb991da0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f76f801b820 tx=0x7f76f800e050 comp rx=0 tx=0).stop 2024-05-22T21:15:48.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 -- 172.21.15.161:0/386213244 shutdown_connections 2024-05-22T21:15:48.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 --2- 172.21.15.161:0/386213244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb947ec0 0x55cbcb9482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 --2- 172.21.15.161:0/386213244 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb9922e0 0x55cbcb947980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 --2- 172.21.15.161:0/386213244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb9919c0 0x55cbcb991da0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 -- 172.21.15.161:0/386213244 >> 172.21.15.161:0/386213244 conn(0x55cbcb94e440 msgr2=0x55cbcb946260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:48.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 -- 172.21.15.161:0/386213244 shutdown_connections 2024-05-22T21:15:48.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 -- 172.21.15.161:0/386213244 wait complete. 2024-05-22T21:15:48.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 -- start start 2024-05-22T21:15:48.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 Processor -- start 2024-05-22T21:15:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb947ec0 0x55cbcb9533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9919c0 0x55cbcb9538e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb9922e0 0x55cbcb953e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbcb7969b0 con 0x55cbcb947ec0 2024-05-22T21:15:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbcb735c00 con 0x55cbcb9919c0 2024-05-22T21:15:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbcb754070 con 0x55cbcb9922e0 2024-05-22T21:15:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7710cea000 10 client.0 _ll_register_callbacks cb 0x55cbcb933ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:15:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7706a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb947ec0 0x55cbcb9533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7706a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb947ec0 0x55cbcb9533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55782/0 (socket says 172.21.15.161:55782) 2024-05-22T21:15:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.777+0000 7f7706a00640 1 -- 172.21.15.161:0/338010369 learned_addr learned my addr 172.21.15.161:0/338010369 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7707400640 1 --2- 172.21.15.161:0/338010369 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb9922e0 0x55cbcb953e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7706000640 1 --2- 172.21.15.161:0/338010369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9919c0 0x55cbcb9538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7706a00640 1 -- 172.21.15.161:0/338010369 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb9922e0 msgr2=0x55cbcb953e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:48.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7706a00640 1 --2- 172.21.15.161:0/338010369 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb9922e0 0x55cbcb953e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7706a00640 1 -- 172.21.15.161:0/338010369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9919c0 msgr2=0x55cbcb9538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:48.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7706a00640 1 --2- 172.21.15.161:0/338010369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9919c0 0x55cbcb9538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7706a00640 1 -- 172.21.15.161:0/338010369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f801b500 con 0x55cbcb947ec0 2024-05-22T21:15:48.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7707400640 1 --2- 172.21.15.161:0/338010369 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb9922e0 0x55cbcb953e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:48.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7706000640 1 --2- 172.21.15.161:0/338010369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9919c0 0x55cbcb9538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:48.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7706a00640 1 --2- 172.21.15.161:0/338010369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb947ec0 0x55cbcb9533a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f76f800a1c0 tx=0x7f76f8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770cd3f5fa2de978 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:48.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f76e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:15:48.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f76e7e00640 1 -- 172.21.15.161:0/338010369 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f801b320 con 0x55cbcb947ec0 2024-05-22T21:15:48.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f76e7e00640 1 -- 172.21.15.161:0/338010369 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76f802d660 con 0x55cbcb947ec0 2024-05-22T21:15:48.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 1 -- 172.21.15.161:0/338010369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbcb7535f0 con 0x55cbcb947ec0 2024-05-22T21:15:48.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f76e7e00640 1 -- 172.21.15.161:0/338010369 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76f802d820 con 0x55cbcb947ec0 2024-05-22T21:15:48.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 10 client.6069 fetch_fsmap learned FSMap version 31 2024-05-22T21:15:48.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 1 -- 172.21.15.161:0/338010369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55cbcb75bab0 con 0x55cbcb947ec0 2024-05-22T21:15:48.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f76e7e00640 1 -- 172.21.15.161:0/338010369 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cbcb7535f0 con 0x55cbcb947ec0 2024-05-22T21:15:48.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f76e7e00640 1 -- 172.21.15.161:0/338010369 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f76f802b070 con 0x55cbcb947ec0 2024-05-22T21:15:48.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 10 client.6069 fetch_fsmap finished waiting for FSMap version 31 2024-05-22T21:15:48.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 10 client.6069 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-22T21:15:48.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 20 client.6069 populate_metadata read hostname 'smithi161' 2024-05-22T21:15:48.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 1 --2- 172.21.15.161:0/338010369 >> [v2:172.21.15.179:6832/2239627963,v1:172.21.15.179:6834/2239627963] conn(0x55cbcb9546f0 0x55cbcb9a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 4 client.6069 mds_command: new command op to 5946 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T21:15:48.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 1 -- 172.21.15.161:0/338010369 --> [v2:172.21.15.179:6832/2239627963,v1:172.21.15.179:6834/2239627963] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbcb793790 con 0x55cbcb9546f0 2024-05-22T21:15:48.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7706000640 1 --2- 172.21.15.161:0/338010369 >> [v2:172.21.15.179:6832/2239627963,v1:172.21.15.179:6834/2239627963] conn(0x55cbcb9546f0 0x55cbcb9a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7706000640 1 --2- 172.21.15.161:0/338010369 >> [v2:172.21.15.179:6832/2239627963,v1:172.21.15.179:6834/2239627963] conn(0x55cbcb9546f0 0x55cbcb9a2f30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dadadead433d69d server_cookie=5df2b90e5cdd50a8 in_seq=0 out_seq=0 2024-05-22T21:15:48.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f76e7e00640 10 client.6069 ms_handle_connect on v2:172.21.15.179:6832/2239627963 2024-05-22T21:15:48.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f76e7e00640 1 -- 172.21.15.161:0/338010369 <== mds.0 v2:172.21.15.179:6832/2239627963 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cbcb793790 con 0x55cbcb9546f0 2024-05-22T21:15:48.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f76e7e00640 10 client.6069 handle_command_reply: tid=0 2024-05-22T21:15:48.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 1 client.6069 shutdown 2024-05-22T21:15:48.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 1 -- 172.21.15.161:0/338010369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb947ec0 msgr2=0x55cbcb9533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 1 --2- 172.21.15.161:0/338010369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb947ec0 0x55cbcb9533a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f76f800a1c0 tx=0x7f76f8004ac0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 1 -- 172.21.15.161:0/338010369 shutdown_connections 2024-05-22T21:15:48.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 1 --2- 172.21.15.161:0/338010369 >> [v2:172.21.15.179:6832/2239627963,v1:172.21.15.179:6834/2239627963] conn(0x55cbcb9546f0 0x55cbcb9a2f30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 1 --2- 172.21.15.161:0/338010369 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb9922e0 0x55cbcb953e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 1 --2- 172.21.15.161:0/338010369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9919c0 0x55cbcb9538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f76e7e00640 0 client.6069 ms_handle_reset on v2:172.21.15.179:6832/2239627963 2024-05-22T21:15:48.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 1 --2- 172.21.15.161:0/338010369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb947ec0 0x55cbcb9533a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 1 -- 172.21.15.161:0/338010369 >> 172.21.15.161:0/338010369 conn(0x55cbcb94e440 msgr2=0x55cbcb946960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:48.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 1 -- 172.21.15.161:0/338010369 shutdown_connections 2024-05-22T21:15:48.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 1 -- 172.21.15.161:0/338010369 wait complete. 2024-05-22T21:15:48.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.781+0000 7f7710cea000 20 client.6069 trim_cache size 0 max 16384 2024-05-22T21:15:48.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.797+0000 7f7710cea000 1 Processor -- start 2024-05-22T21:15:48.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.797+0000 7f7710cea000 1 -- start start 2024-05-22T21:15:48.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb947ec0 0x55cbcb9a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb9919c0 0x55cbcb9a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9922e0 0x55cbcb9a2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbcb453940 con 0x55cbcb9919c0 2024-05-22T21:15:48.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbcb79a9b0 con 0x55cbcb9922e0 2024-05-22T21:15:48.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbcb79ed80 con 0x55cbcb947ec0 2024-05-22T21:15:48.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7706000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb9919c0 0x55cbcb9a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7706000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb9919c0 0x55cbcb9a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55790/0 (socket says 172.21.15.161:55790) 2024-05-22T21:15:48.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7706000640 1 -- 172.21.15.161:0/466978591 learned_addr learned my addr 172.21.15.161:0/466978591 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:48.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7706a00640 1 --2- 172.21.15.161:0/466978591 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb947ec0 0x55cbcb9a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7706000640 1 -- 172.21.15.161:0/466978591 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb947ec0 msgr2=0x55cbcb9a52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:48.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7706000640 1 --2- 172.21.15.161:0/466978591 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb947ec0 0x55cbcb9a52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7707400640 1 --2- 172.21.15.161:0/466978591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9922e0 0x55cbcb9a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7706000640 1 -- 172.21.15.161:0/466978591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9922e0 msgr2=0x55cbcb9a2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:48.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7706000640 1 --2- 172.21.15.161:0/466978591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9922e0 0x55cbcb9a2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7706000640 1 -- 172.21.15.161:0/466978591 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbcb741c00 con 0x55cbcb9919c0 2024-05-22T21:15:48.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7706a00640 1 --2- 172.21.15.161:0/466978591 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb947ec0 0x55cbcb9a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:48.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7707400640 1 --2- 172.21.15.161:0/466978591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9922e0 0x55cbcb9a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:15:48.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7706000640 1 --2- 172.21.15.161:0/466978591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb9919c0 0x55cbcb9a1f30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f76f4054c40 tx=0x7f76f4051340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276281cccae5d64f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:48.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f76f3e00640 1 -- 172.21.15.161:0/466978591 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4055ea0 con 0x55cbcb9919c0 2024-05-22T21:15:48.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f76f3e00640 1 -- 172.21.15.161:0/466978591 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f40532e0 con 0x55cbcb9919c0 2024-05-22T21:15:48.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f76f3e00640 1 -- 172.21.15.161:0/466978591 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f40551a0 con 0x55cbcb9919c0 2024-05-22T21:15:48.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 -- 172.21.15.161:0/466978591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb9919c0 msgr2=0x55cbcb9a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 --2- 172.21.15.161:0/466978591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb9919c0 0x55cbcb9a1f30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f76f4054c40 tx=0x7f76f4051340 comp rx=0 tx=0).stop 2024-05-22T21:15:48.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 -- 172.21.15.161:0/466978591 shutdown_connections 2024-05-22T21:15:48.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 --2- 172.21.15.161:0/466978591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9922e0 0x55cbcb9a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 --2- 172.21.15.161:0/466978591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb9919c0 0x55cbcb9a1f30 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 --2- 172.21.15.161:0/466978591 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb947ec0 0x55cbcb9a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 -- 172.21.15.161:0/466978591 >> 172.21.15.161:0/466978591 conn(0x55cbcb94e440 msgr2=0x55cbcb94d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:48.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 -- 172.21.15.161:0/466978591 shutdown_connections 2024-05-22T21:15:48.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 -- 172.21.15.161:0/466978591 wait complete. 2024-05-22T21:15:48.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 -- start start 2024-05-22T21:15:48.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 Processor -- start 2024-05-22T21:15:48.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb947ec0 0x55cbcb9d6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9919c0 0x55cbcb9d72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb9922e0 0x55cbcb9d7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbcb71d820 con 0x55cbcb947ec0 2024-05-22T21:15:48.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbcb799510 con 0x55cbcb9919c0 2024-05-22T21:15:48.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.801+0000 7f7710cea000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbcb953fd0 con 0x55cbcb9922e0 2024-05-22T21:15:48.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 10 client.0 _ll_register_callbacks cb 0x7f76f800a0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:15:48.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7706a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb947ec0 0x55cbcb9d6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7706a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb947ec0 0x55cbcb9d6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55796/0 (socket says 172.21.15.161:55796) 2024-05-22T21:15:48.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7706a00640 1 -- 172.21.15.161:0/380331631 learned_addr learned my addr 172.21.15.161:0/380331631 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:48.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7706000640 1 --2- 172.21.15.161:0/380331631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9919c0 0x55cbcb9d72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7707400640 1 --2- 172.21.15.161:0/380331631 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb9922e0 0x55cbcb9d7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7706a00640 1 -- 172.21.15.161:0/380331631 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb9922e0 msgr2=0x55cbcb9d7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:48.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7706a00640 1 --2- 172.21.15.161:0/380331631 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb9922e0 0x55cbcb9d7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7706a00640 1 -- 172.21.15.161:0/380331631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9919c0 msgr2=0x55cbcb9d72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:48.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7706a00640 1 --2- 172.21.15.161:0/380331631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9919c0 0x55cbcb9d72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7706a00640 1 -- 172.21.15.161:0/380331631 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7708149630 con 0x55cbcb947ec0 2024-05-22T21:15:48.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7706a00640 1 --2- 172.21.15.161:0/380331631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb947ec0 0x55cbcb9d6da0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f76f8004a30 tx=0x7f76f8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4a31b51ae405ad server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:48.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f76e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:15:48.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f76e7e00640 1 -- 172.21.15.161:0/380331631 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f801b320 con 0x55cbcb947ec0 2024-05-22T21:15:48.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f76e7e00640 1 -- 172.21.15.161:0/380331631 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76f8023540 con 0x55cbcb947ec0 2024-05-22T21:15:48.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 1 -- 172.21.15.161:0/380331631 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbcb766460 con 0x55cbcb947ec0 2024-05-22T21:15:48.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f76e7e00640 1 -- 172.21.15.161:0/380331631 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76f8023700 con 0x55cbcb947ec0 2024-05-22T21:15:48.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 10 client.6075 fetch_fsmap learned FSMap version 31 2024-05-22T21:15:48.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 1 -- 172.21.15.161:0/380331631 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55cbcb492cc0 con 0x55cbcb947ec0 2024-05-22T21:15:48.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f76e7e00640 1 -- 172.21.15.161:0/380331631 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cbcb766460 con 0x55cbcb947ec0 2024-05-22T21:15:48.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f76e7e00640 1 -- 172.21.15.161:0/380331631 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f76f8005d20 con 0x55cbcb947ec0 2024-05-22T21:15:48.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 10 client.6075 fetch_fsmap finished waiting for FSMap version 31 2024-05-22T21:15:48.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 10 client.6075 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-22T21:15:48.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 20 client.6075 populate_metadata read hostname 'smithi161' 2024-05-22T21:15:48.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 1 --2- 172.21.15.161:0/380331631 >> [v2:172.21.15.179:6832/2239627963,v1:172.21.15.179:6834/2239627963] conn(0x55cbcb9d7d60 0x55cbcb9d8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 4 client.6075 mds_command: new command op to 5946 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-22T21:15:48.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 1 -- 172.21.15.161:0/380331631 --> [v2:172.21.15.179:6832/2239627963,v1:172.21.15.179:6834/2239627963] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cbcb75f9c0 con 0x55cbcb9d7d60 2024-05-22T21:15:48.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7706000640 1 --2- 172.21.15.161:0/380331631 >> [v2:172.21.15.179:6832/2239627963,v1:172.21.15.179:6834/2239627963] conn(0x55cbcb9d7d60 0x55cbcb9d8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7706000640 1 --2- 172.21.15.161:0/380331631 >> [v2:172.21.15.179:6832/2239627963,v1:172.21.15.179:6834/2239627963] conn(0x55cbcb9d7d60 0x55cbcb9d8140 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4ef0fd9223a6d1a server_cookie=37c2b5cbc150ad77 in_seq=0 out_seq=0 2024-05-22T21:15:48.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f76e7e00640 10 client.6075 ms_handle_connect on v2:172.21.15.179:6832/2239627963 2024-05-22T21:15:48.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f76e7e00640 1 -- 172.21.15.161:0/380331631 <== mds.0 v2:172.21.15.179:6832/2239627963 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x55cbcb75f9c0 con 0x55cbcb9d7d60 2024-05-22T21:15:48.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f76e7e00640 10 client.6075 handle_command_reply: tid=0 2024-05-22T21:15:48.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 1 client.6075 shutdown 2024-05-22T21:15:48.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 1 -- 172.21.15.161:0/380331631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb947ec0 msgr2=0x55cbcb9d6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 1 --2- 172.21.15.161:0/380331631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb947ec0 0x55cbcb9d6da0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f76f8004a30 tx=0x7f76f8013000 comp rx=0 tx=0).stop 2024-05-22T21:15:48.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 1 -- 172.21.15.161:0/380331631 shutdown_connections 2024-05-22T21:15:48.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 1 --2- 172.21.15.161:0/380331631 >> [v2:172.21.15.179:6832/2239627963,v1:172.21.15.179:6834/2239627963] conn(0x55cbcb9d7d60 0x55cbcb9d8140 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 1 --2- 172.21.15.161:0/380331631 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cbcb9922e0 0x55cbcb9d7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 1 --2- 172.21.15.161:0/380331631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cbcb9919c0 0x55cbcb9d72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 1 --2- 172.21.15.161:0/380331631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cbcb947ec0 0x55cbcb9d6da0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f76e7e00640 0 client.6075 ms_handle_reset on v2:172.21.15.179:6832/2239627963 2024-05-22T21:15:48.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 1 -- 172.21.15.161:0/380331631 >> 172.21.15.161:0/380331631 conn(0x55cbcb94e440 msgr2=0x55cbcb94d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:48.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 1 -- 172.21.15.161:0/380331631 shutdown_connections 2024-05-22T21:15:48.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 1 -- 172.21.15.161:0/380331631 wait complete. 2024-05-22T21:15:48.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.805+0000 7f7710cea000 20 client.6075 trim_cache size 0 max 16384 2024-05-22T21:15:48.818 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:15:48.818 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:15:48.818 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 0, 2024-05-22T21:15:48.818 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 5946, 2024-05-22T21:15:48.818 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2024-05-22T21:15:48.818 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2024-05-22T21:15:48.818 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2024-05-22T21:15:48.818 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 5.85319257, 2024-05-22T21:15:48.818 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 31, 2024-05-22T21:15:48.818 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 67, 2024-05-22T21:15:48.819 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 67, 2024-05-22T21:15:48.819 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 10.983350550000001 2024-05-22T21:15:48.819 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:15:48.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.809+0000 7f770d800640 1 -- 172.21.15.161:0/3070057161 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76e8066690 msgr2=0x7f76e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.809+0000 7f770d800640 1 --2- 172.21.15.161:0/3070057161 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76e8066690 0x7f76e8068b50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f76f800a3e0 tx=0x7f76f800a040 comp rx=0 tx=0).stop 2024-05-22T21:15:48.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.809+0000 7f770d800640 1 -- 172.21.15.161:0/3070057161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77081498b0 msgr2=0x7f77081490f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.809+0000 7f770d800640 1 --2- 172.21.15.161:0/3070057161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77081498b0 0x7f77081490f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f76f400e500 tx=0x7f76f400e9c0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.809+0000 7f770d800640 1 -- 172.21.15.161:0/3070057161 shutdown_connections 2024-05-22T21:15:48.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.809+0000 7f770d800640 1 --2- 172.21.15.161:0/3070057161 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76e8066690 0x7f76e8068b50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.809+0000 7f770d800640 1 --2- 172.21.15.161:0/3070057161 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f770814f620 0x7f770814faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.809+0000 7f770d800640 1 --2- 172.21.15.161:0/3070057161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77081498b0 0x7f77081490f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.809+0000 7f770d800640 1 --2- 172.21.15.161:0/3070057161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76ec005190 0x7f770814da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.809+0000 7f770d800640 1 -- 172.21.15.161:0/3070057161 >> 172.21.15.161:0/3070057161 conn(0x7f770806a2d0 msgr2=0x7f77080adae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:48.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.809+0000 7f770d800640 1 -- 172.21.15.161:0/3070057161 shutdown_connections 2024-05-22T21:15:48.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.809+0000 7f770d800640 1 -- 172.21.15.161:0/3070057161 wait complete. 2024-05-22T21:15:48.833 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:15:48.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.961+0000 7f1e6f600640 1 Processor -- start 2024-05-22T21:15:48.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.961+0000 7f1e6f600640 1 -- start start 2024-05-22T21:15:48.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.961+0000 7f1e6f600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e6815dd50 0x7f1e6806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.961+0000 7f1e6f600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e68071670 con 0x7f1e6815d5e0 2024-05-22T21:15:48.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.961+0000 7f1e6f600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e6806de40 con 0x7f1e6815e130 2024-05-22T21:15:48.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.961+0000 7f1e6f600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e68070a60 con 0x7f1e6815dd50 2024-05-22T21:15:48.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.961+0000 7f1e6d600640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1e6815d5e0 0x7f1e6815b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51884/0 (socket says 172.21.15.161:51884) 2024-05-22T21:15:48.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.961+0000 7f1e6d600640 1 -- 172.21.15.161:0/2408683328 learned_addr learned my addr 172.21.15.161:0/2408683328 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:48.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.961+0000 7f1e6cc00640 1 --2- 172.21.15.161:0/2408683328 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e6815dd50 0x7f1e6806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.961+0000 7f1e67400640 1 -- 172.21.15.161:0/2408683328 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816277637 0 0) 0x7f1e6806de40 con 0x7f1e6815e130 2024-05-22T21:15:48.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.961+0000 7f1e67400640 1 -- 172.21.15.161:0/2408683328 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e4c003530 con 0x7f1e6815e130 2024-05-22T21:15:48.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.961+0000 7f1e67400640 1 -- 172.21.15.161:0/2408683328 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925808254 0 0) 0x7f1e68071670 con 0x7f1e6815d5e0 2024-05-22T21:15:48.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.965+0000 7f1e67400640 1 -- 172.21.15.161:0/2408683328 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e6806de40 con 0x7f1e6815d5e0 2024-05-22T21:15:48.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.965+0000 7f1e6cc00640 1 -- 172.21.15.161:0/2408683328 >> v1:172.21.15.179:6789/0 conn(0x7f1e6815e130 legacy=0x7f1e6815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.965+0000 7f1e6cc00640 1 -- 172.21.15.161:0/2408683328 >> v1:172.21.15.161:6789/0 conn(0x7f1e6815d5e0 legacy=0x7f1e6815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.965+0000 7f1e6cc00640 1 -- 172.21.15.161:0/2408683328 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e68070d40 con 0x7f1e6815dd50 2024-05-22T21:15:48.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.965+0000 7f1e6cc00640 1 --2- 172.21.15.161:0/2408683328 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e6815dd50 0x7f1e6806d900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1e580097f0 tx=0x7f1e5801c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91ea53d216a311eb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:48.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.965+0000 7f1e67400640 1 -- 172.21.15.161:0/2408683328 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e580ab620 con 0x7f1e6815dd50 2024-05-22T21:15:48.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.965+0000 7f1e67400640 1 -- 172.21.15.161:0/2408683328 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e580ab780 con 0x7f1e6815dd50 2024-05-22T21:15:48.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.965+0000 7f1e67400640 1 -- 172.21.15.161:0/2408683328 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e580abb10 con 0x7f1e6815dd50 2024-05-22T21:15:48.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.965+0000 7f1e6f600640 1 -- 172.21.15.161:0/2408683328 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e6815dd50 msgr2=0x7f1e6806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.965+0000 7f1e6f600640 1 --2- 172.21.15.161:0/2408683328 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e6815dd50 0x7f1e6806d900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1e580097f0 tx=0x7f1e5801c420 comp rx=0 tx=0).stop 2024-05-22T21:15:48.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.965+0000 7f1e6f600640 1 -- 172.21.15.161:0/2408683328 shutdown_connections 2024-05-22T21:15:48.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.965+0000 7f1e6f600640 1 --2- 172.21.15.161:0/2408683328 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e6815dd50 0x7f1e6806d900 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.965+0000 7f1e6f600640 1 -- 172.21.15.161:0/2408683328 >> 172.21.15.161:0/2408683328 conn(0x7f1e680b0bc0 msgr2=0x7f1e680b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:48.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.965+0000 7f1e6f600640 1 -- 172.21.15.161:0/2408683328 shutdown_connections 2024-05-22T21:15:48.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.965+0000 7f1e6f600640 1 -- 172.21.15.161:0/2408683328 wait complete. 2024-05-22T21:15:48.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6f600640 1 Processor -- start 2024-05-22T21:15:48.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6f600640 1 -- start start 2024-05-22T21:15:48.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6f600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e6815d5e0 0x7f1e6814cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6f600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e6815dd50 0x7f1e6814d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6f600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e6815e130 0x7f1e6814d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6f600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e680b2aa0 con 0x7f1e6815e130 2024-05-22T21:15:48.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6f600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e680b24e0 con 0x7f1e6815dd50 2024-05-22T21:15:48.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6f600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e680b27c0 con 0x7f1e6815d5e0 2024-05-22T21:15:48.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6d600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e6815e130 0x7f1e6814d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6d600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e6815e130 0x7f1e6814d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55802/0 (socket says 172.21.15.161:55802) 2024-05-22T21:15:48.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6d600640 1 -- 172.21.15.161:0/3996484584 learned_addr learned my addr 172.21.15.161:0/3996484584 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:48.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6cc00640 1 --2- 172.21.15.161:0/3996484584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e6815d5e0 0x7f1e6814cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e67e00640 1 --2- 172.21.15.161:0/3996484584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e6815dd50 0x7f1e6814d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6d600640 1 -- 172.21.15.161:0/3996484584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e6815d5e0 msgr2=0x7f1e6814cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6d600640 1 --2- 172.21.15.161:0/3996484584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e6815d5e0 0x7f1e6814cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6d600640 1 -- 172.21.15.161:0/3996484584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e6815dd50 msgr2=0x7f1e6814d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:48.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6d600640 1 --2- 172.21.15.161:0/3996484584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e6815dd50 0x7f1e6814d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:48.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6d600640 1 -- 172.21.15.161:0/3996484584 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e58009420 con 0x7f1e6815e130 2024-05-22T21:15:48.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e67e00640 1 --2- 172.21.15.161:0/3996484584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e6815dd50 0x7f1e6814d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:15:48.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.969+0000 7f1e6d600640 1 --2- 172.21.15.161:0/3996484584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e6815e130 0x7f1e6814d770 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000ca70 tx=0x7f1e6000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:48.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.973+0000 7f1e64c00640 1 -- 172.21.15.161:0/3996484584 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6000abd0 con 0x7f1e6815e130 2024-05-22T21:15:48.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.973+0000 7f1e64c00640 1 -- 172.21.15.161:0/3996484584 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6000ad30 con 0x7f1e6815e130 2024-05-22T21:15:48.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.973+0000 7f1e64c00640 1 -- 172.21.15.161:0/3996484584 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60012600 con 0x7f1e6815e130 2024-05-22T21:15:48.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.973+0000 7f1e6f600640 1 -- 172.21.15.161:0/3996484584 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e681518f0 con 0x7f1e6815e130 2024-05-22T21:15:48.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.973+0000 7f1e6f600640 1 -- 172.21.15.161:0/3996484584 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e680ce1e0 con 0x7f1e6815e130 2024-05-22T21:15:48.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.973+0000 7f1e6f600640 1 -- 172.21.15.161:0/3996484584 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e68151750 con 0x7f1e6815e130 2024-05-22T21:15:48.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.973+0000 7f1e64c00640 1 -- 172.21.15.161:0/3996484584 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1e60007070 con 0x7f1e6815e130 2024-05-22T21:15:48.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.973+0000 7f1e64c00640 1 --2- 172.21.15.161:0/3996484584 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e48066690 0x7f1e48068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:48.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.973+0000 7f1e64c00640 1 -- 172.21.15.161:0/3996484584 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e60083fb0 con 0x7f1e6815e130 2024-05-22T21:15:48.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.973+0000 7f1e6cc00640 1 --2- 172.21.15.161:0/3996484584 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e48066690 0x7f1e48068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:48.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.973+0000 7f1e6cc00640 1 --2- 172.21.15.161:0/3996484584 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e48066690 0x7f1e48068b50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1e580af000 tx=0x7f1e580a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:48.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:48.977+0000 7f1e64c00640 1 -- 172.21.15.161:0/3996484584 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1e68151750 con 0x7f1e6815e130 2024-05-22T21:15:49.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:49.169+0000 7f1e6f600640 1 -- 172.21.15.161:0/3996484584 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1e68151750 con 0x7f1e6815e130 2024-05-22T21:15:49.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:49.169+0000 7f1e64c00640 1 -- 172.21.15.161:0/3996484584 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1e68151750 con 0x7f1e6815e130 2024-05-22T21:15:49.175 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:15:49.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:49.169+0000 7f1e6f600640 1 -- 172.21.15.161:0/3996484584 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e48066690 msgr2=0x7f1e48068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:49.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:49.169+0000 7f1e6f600640 1 --2- 172.21.15.161:0/3996484584 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e48066690 0x7f1e48068b50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1e580af000 tx=0x7f1e580a7000 comp rx=0 tx=0).stop 2024-05-22T21:15:49.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:49.169+0000 7f1e6f600640 1 -- 172.21.15.161:0/3996484584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e6815e130 msgr2=0x7f1e6814d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:49.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:49.169+0000 7f1e6f600640 1 --2- 172.21.15.161:0/3996484584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e6815e130 0x7f1e6814d770 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000ca70 tx=0x7f1e6000cf30 comp rx=0 tx=0).stop 2024-05-22T21:15:49.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:49.169+0000 7f1e6f600640 1 -- 172.21.15.161:0/3996484584 shutdown_connections 2024-05-22T21:15:49.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:49.169+0000 7f1e6f600640 1 --2- 172.21.15.161:0/3996484584 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e48066690 0x7f1e48068b50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:49.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:49.169+0000 7f1e6f600640 1 --2- 172.21.15.161:0/3996484584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e6815e130 0x7f1e6814d770 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:49.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:49.169+0000 7f1e6f600640 1 --2- 172.21.15.161:0/3996484584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e6815dd50 0x7f1e6814d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:49.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:49.169+0000 7f1e6f600640 1 --2- 172.21.15.161:0/3996484584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e6815d5e0 0x7f1e6814cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:49.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:49.169+0000 7f1e6f600640 1 -- 172.21.15.161:0/3996484584 >> 172.21.15.161:0/3996484584 conn(0x7f1e680b0bc0 msgr2=0x7f1e6815a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:49.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:49.169+0000 7f1e6f600640 1 -- 172.21.15.161:0/3996484584 shutdown_connections 2024-05-22T21:15:49.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:49.169+0000 7f1e6f600640 1 -- 172.21.15.161:0/3996484584 wait complete. 2024-05-22T21:15:49.194 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:15:49.194 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:15:49.194 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-05-22T21:15:49.194 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-22T21:15:49.194 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-22T21:15:49.194 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:15:49.194 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:15:49.195 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:15:49.195 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:15:49.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:49.195 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-05-22T21:15:49.207 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:15:49.207 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:15:49.207 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:15:49.207 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:15:49.207 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-05-22T21:15:49.207 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:15:49.207 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:15:49.207 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-05-22T21:15:49.207 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2024-05-22T21:15:49.207 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:15:49.207 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:15:49.208 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2024-05-22T21:15:49.208 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:15:49.208 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:15:49.208 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:15:49.208 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:15:49.208 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:15:49.209 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42343sec preferred_lft 42343sec 2024-05-22T21:15:49.209 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-05-22T21:15:49.209 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:15:49.209 INFO:teuthology.orchestra.run.smithi161.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:15:49.210 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 02:42:b9:0f:57:df brd ff:ff:ff:ff:ff:ff 2024-05-22T21:15:49.210 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:15:49.210 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:15:49.210 INFO:teuthology.orchestra.run.smithi161.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:15:49.210 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:15:49.210 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:15:49.211 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:15:49.211 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:15:49.211 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:15:49.211 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:15:49.211 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-22T21:15:49.211 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:15:49.211 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:15:49.211 INFO:teuthology.orchestra.run.smithi161.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:15:49.212 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-22T21:15:49.212 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:15:49.212 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:15:49.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:49.212 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:15:49.266 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-05-22T21:15:49.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:49.269 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-05-22T21:15:49.321 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-05-22T21:15:49.321 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:15:49.321 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:15:49.321 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:15:49.321 INFO:teuthology.orchestra.run.smithi161.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:15:49.322 INFO:teuthology.orchestra.run.smithi161.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:15:49.322 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:15:49.322 INFO:teuthology.orchestra.run.smithi161.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:15:49.322 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:15:49.322 INFO:teuthology.orchestra.run.smithi161.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:15:49.322 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:15:49.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:49.323 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:15:49.323 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:15:49.323 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:15:49.323 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:15:49.323 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:15:49.323 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:15:49.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:49.414 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-05-22T21:15:49.469 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-22T21:15:49.469 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-22T21:15:49.470 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:15:49.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:49.471 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2024-05-22T21:15:49.537 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:15:49.537 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-22T21:15:49.589 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:15:49.589 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:15:49.590 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:15:49.641 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:15:49.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:49.641 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:15:49.729 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:15:49.729 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:15:49.730 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:15:49.745 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:15:49.745 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:15:49.745 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:15:49.745 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:15:49.745 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:15:49.753 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:15:49.754 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:15:49.754 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:15:49.754 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:15:49.754 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:15:49.754 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:15:49.755 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:15:49.755 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:15:49.758 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:15:49.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:49.759 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:15:49.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:49.781 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:15:49.844 INFO:teuthology.orchestra.run.smithi161.stdout:client6084 2024-05-22T21:15:49.844 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:15:49.844 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:15:49.893 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:15:50.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.065+0000 7fcb26800640 1 Processor -- start 2024-05-22T21:15:50.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.065+0000 7fcb26800640 1 -- start start 2024-05-22T21:15:50.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb26800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb200b1df0 0x7fcb2015f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:50.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb26800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb200b3560 con 0x7fcb2015e7d0 2024-05-22T21:15:50.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb26800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb200cf530 con 0x7fcb200cbda0 2024-05-22T21:15:50.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb26800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb200b37b0 con 0x7fcb200b1df0 2024-05-22T21:15:50.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1f400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fcb2015e7d0 0x7fcb2015eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51896/0 (socket says 172.21.15.161:51896) 2024-05-22T21:15:50.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1f400640 1 -- 172.21.15.161:0/2827156622 learned_addr learned my addr 172.21.15.161:0/2827156622 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:15:50.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1ea00640 1 --2- 172.21.15.161:0/2827156622 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb200b1df0 0x7fcb2015f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:50.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1e000640 1 -- 172.21.15.161:0/2827156622 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394047960 0 0) 0x7fcb200b3560 con 0x7fcb2015e7d0 2024-05-22T21:15:50.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1e000640 1 -- 172.21.15.161:0/2827156622 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb04003530 con 0x7fcb2015e7d0 2024-05-22T21:15:50.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1e000640 1 -- 172.21.15.161:0/2827156622 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 355398514 0 0) 0x7fcb04003530 con 0x7fcb2015e7d0 2024-05-22T21:15:50.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1e000640 1 -- 172.21.15.161:0/2827156622 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb200b1df0 msgr2=0x7fcb2015f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:50.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1e000640 1 --2- 172.21.15.161:0/2827156622 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb200b1df0 0x7fcb2015f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:50.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1e000640 1 -- 172.21.15.161:0/2827156622 >> v1:172.21.15.179:6789/0 conn(0x7fcb200cbda0 legacy=0x7fcb200cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:50.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1e000640 1 -- 172.21.15.161:0/2827156622 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb200cf780 con 0x7fcb2015e7d0 2024-05-22T21:15:50.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1e000640 1 -- 172.21.15.161:0/2827156622 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fcb10001680 con 0x7fcb2015e7d0 2024-05-22T21:15:50.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1e000640 1 -- 172.21.15.161:0/2827156622 >> v1:172.21.15.161:6789/0 conn(0x7fcb2015e7d0 legacy=0x7fcb2015eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:50.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1e000640 1 --2- 172.21.15.161:0/2827156622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb040039b0 0x7fcb04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:50.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1e000640 1 --2- 172.21.15.161:0/2827156622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb040047d0 0x7fcb04004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:50.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1e000640 1 --2- 172.21.15.161:0/2827156622 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb200b1df0 0x7fcb04005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:50.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1e000640 1 -- 172.21.15.161:0/2827156622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb200b37b0 con 0x7fcb040039b0 2024-05-22T21:15:50.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1e000640 1 -- 172.21.15.161:0/2827156622 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0400ba40 con 0x7fcb040047d0 2024-05-22T21:15:50.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1e000640 1 -- 172.21.15.161:0/2827156622 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0400bc00 con 0x7fcb200b1df0 2024-05-22T21:15:50.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1f400640 1 --2- 172.21.15.161:0/2827156622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb040039b0 0x7fcb04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:50.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1ea00640 1 --2- 172.21.15.161:0/2827156622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb040047d0 0x7fcb04004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:50.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1fe00640 1 --2- 172.21.15.161:0/2827156622 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb200b1df0 0x7fcb04005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:50.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1f400640 1 -- 172.21.15.161:0/2827156622 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb200b1df0 msgr2=0x7fcb04005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:50.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1f400640 1 --2- 172.21.15.161:0/2827156622 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb200b1df0 0x7fcb04005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:50.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1f400640 1 -- 172.21.15.161:0/2827156622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb040047d0 msgr2=0x7fcb04004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:15:50.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1f400640 1 --2- 172.21.15.161:0/2827156622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb040047d0 0x7fcb04004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:50.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.069+0000 7fcb1f400640 1 -- 172.21.15.161:0/2827156622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb04003530 con 0x7fcb040039b0 2024-05-22T21:15:50.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.073+0000 7fcb1f400640 1 --2- 172.21.15.161:0/2827156622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb040039b0 0x7fcb04003dd0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c210 tx=0x7fcb1001cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b860d582dafc22 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:50.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.073+0000 7fcb1e000640 1 -- 172.21.15.161:0/2827156622 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10005ae0 con 0x7fcb040039b0 2024-05-22T21:15:50.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.073+0000 7fcb1e000640 1 -- 172.21.15.161:0/2827156622 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb10005c40 con 0x7fcb040039b0 2024-05-22T21:15:50.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.073+0000 7fcb1e000640 1 -- 172.21.15.161:0/2827156622 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb100a57e0 con 0x7fcb040039b0 2024-05-22T21:15:50.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.073+0000 7fcb26800640 1 -- 172.21.15.161:0/2827156622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb040039b0 msgr2=0x7fcb04003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:50.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.073+0000 7fcb26800640 1 --2- 172.21.15.161:0/2827156622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb040039b0 0x7fcb04003dd0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c210 tx=0x7fcb1001cbf0 comp rx=0 tx=0).stop 2024-05-22T21:15:50.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.073+0000 7fcb1fe00640 1 -- 172.21.15.161:0/2827156622 reap_dead start 2024-05-22T21:15:50.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.073+0000 7fcb26800640 1 -- 172.21.15.161:0/2827156622 shutdown_connections 2024-05-22T21:15:50.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.073+0000 7fcb26800640 1 -- 172.21.15.161:0/2827156622 >> 172.21.15.161:0/2827156622 conn(0x7fcb200b2a40 msgr2=0x7fcb200b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:50.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.073+0000 7fcb26800640 1 -- 172.21.15.161:0/2827156622 shutdown_connections 2024-05-22T21:15:50.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.073+0000 7fcb26800640 1 -- 172.21.15.161:0/2827156622 wait complete. 2024-05-22T21:15:50.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb26800640 1 Processor -- start 2024-05-22T21:15:50.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb26800640 1 -- start start 2024-05-22T21:15:50.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb26800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb20155400 0x7fcb200c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:50.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb26800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb200c5130 0x7fcb2006df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:50.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb26800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb201557e0 0x7fcb2006e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:50.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb26800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb200c9c70 con 0x7fcb200c5130 2024-05-22T21:15:50.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb26800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb200c9f50 con 0x7fcb20155400 2024-05-22T21:15:50.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb26800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb200ca230 con 0x7fcb201557e0 2024-05-22T21:15:50.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb1ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb200c5130 0x7fcb2006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:50.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb1ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb200c5130 0x7fcb2006df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55824/0 (socket says 172.21.15.161:55824) 2024-05-22T21:15:50.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb1ea00640 1 -- 172.21.15.161:0/2041125698 learned_addr learned my addr 172.21.15.161:0/2041125698 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:50.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb1f400640 1 --2- 172.21.15.161:0/2041125698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb20155400 0x7fcb200c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:50.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb1fe00640 1 --2- 172.21.15.161:0/2041125698 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb201557e0 0x7fcb2006e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:50.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb1fe00640 1 -- 172.21.15.161:0/2041125698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb20155400 msgr2=0x7fcb200c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:50.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb1fe00640 1 --2- 172.21.15.161:0/2041125698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb20155400 0x7fcb200c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:50.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb1fe00640 1 -- 172.21.15.161:0/2041125698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb200c5130 msgr2=0x7fcb2006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:50.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb1fe00640 1 --2- 172.21.15.161:0/2041125698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb200c5130 0x7fcb2006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:50.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb1fe00640 1 -- 172.21.15.161:0/2041125698 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb200ca510 con 0x7fcb201557e0 2024-05-22T21:15:50.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb1ea00640 1 --2- 172.21.15.161:0/2041125698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb200c5130 0x7fcb2006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:15:50.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.077+0000 7fcb1fe00640 1 --2- 172.21.15.161:0/2041125698 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb201557e0 0x7fcb2006e4d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcb200711e0 tx=0x7fcb1800ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:50.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.081+0000 7fcb17400640 1 -- 172.21.15.161:0/2041125698 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18003980 con 0x7fcb201557e0 2024-05-22T21:15:50.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.081+0000 7fcb17400640 1 -- 172.21.15.161:0/2041125698 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb18003ae0 con 0x7fcb201557e0 2024-05-22T21:15:50.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.081+0000 7fcb26800640 1 -- 172.21.15.161:0/2041125698 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb200ca7f0 con 0x7fcb201557e0 2024-05-22T21:15:50.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.081+0000 7fcb17400640 1 -- 172.21.15.161:0/2041125698 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18003e90 con 0x7fcb201557e0 2024-05-22T21:15:50.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.081+0000 7fcb26800640 1 -- 172.21.15.161:0/2041125698 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb200c8020 con 0x7fcb201557e0 2024-05-22T21:15:50.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.081+0000 7fcb26800640 1 -- 172.21.15.161:0/2041125698 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb200ca9e0 con 0x7fcb201557e0 2024-05-22T21:15:50.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.081+0000 7fcb17400640 1 -- 172.21.15.161:0/2041125698 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcb180050d0 con 0x7fcb201557e0 2024-05-22T21:15:50.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.081+0000 7fcb17400640 1 --2- 172.21.15.161:0/2041125698 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcb00066710 0x7fcb00068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:50.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.081+0000 7fcb17400640 1 -- 172.21.15.161:0/2041125698 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb18083e60 con 0x7fcb201557e0 2024-05-22T21:15:50.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.081+0000 7fcb1f400640 1 --2- 172.21.15.161:0/2041125698 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcb00066710 0x7fcb00068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:50.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.085+0000 7fcb1f400640 1 --2- 172.21.15.161:0/2041125698 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcb00066710 0x7fcb00068bd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcb100ab000 tx=0x7fcb100b1040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:50.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.089+0000 7fcb17400640 1 -- 172.21.15.161:0/2041125698 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcb18051260 con 0x7fcb201557e0 2024-05-22T21:15:50.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.273+0000 7fcb26800640 1 -- 172.21.15.161:0/2041125698 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcb20067220 con 0x7fcb201557e0 2024-05-22T21:15:50.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.273+0000 7fcb17400640 1 -- 172.21.15.161:0/2041125698 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcb18054880 con 0x7fcb201557e0 2024-05-22T21:15:50.281 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:15:50.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.277+0000 7fcb26800640 1 -- 172.21.15.161:0/2041125698 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcb00066710 msgr2=0x7fcb00068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:50.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.277+0000 7fcb26800640 1 --2- 172.21.15.161:0/2041125698 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcb00066710 0x7fcb00068bd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcb100ab000 tx=0x7fcb100b1040 comp rx=0 tx=0).stop 2024-05-22T21:15:50.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.277+0000 7fcb26800640 1 -- 172.21.15.161:0/2041125698 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb201557e0 msgr2=0x7fcb2006e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:50.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.277+0000 7fcb26800640 1 --2- 172.21.15.161:0/2041125698 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb201557e0 0x7fcb2006e4d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcb200711e0 tx=0x7fcb1800ec10 comp rx=0 tx=0).stop 2024-05-22T21:15:50.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.277+0000 7fcb26800640 1 -- 172.21.15.161:0/2041125698 shutdown_connections 2024-05-22T21:15:50.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.277+0000 7fcb26800640 1 --2- 172.21.15.161:0/2041125698 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcb00066710 0x7fcb00068bd0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:50.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.277+0000 7fcb26800640 1 --2- 172.21.15.161:0/2041125698 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb201557e0 0x7fcb2006e4d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:50.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.277+0000 7fcb26800640 1 --2- 172.21.15.161:0/2041125698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb200c5130 0x7fcb2006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:50.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.277+0000 7fcb26800640 1 --2- 172.21.15.161:0/2041125698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb20155400 0x7fcb200c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:50.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.277+0000 7fcb26800640 1 -- 172.21.15.161:0/2041125698 >> 172.21.15.161:0/2041125698 conn(0x7fcb200b2a40 msgr2=0x7fcb200cc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:50.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.277+0000 7fcb26800640 1 -- 172.21.15.161:0/2041125698 shutdown_connections 2024-05-22T21:15:50.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:50.277+0000 7fcb26800640 1 -- 172.21.15.161:0/2041125698 wait complete. 2024-05-22T21:15:50.301 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:15:50.301 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:15:50.302 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-22T21:15:50.302 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-22T21:15:50.302 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-22T21:15:50.302 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:15:50.302 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:15:50.302 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:15:50.302 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:15:50.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:50.303 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-22T21:15:50.313 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:15:50.313 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:15:50.313 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:15:50.313 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:15:50.313 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-22T21:15:50.313 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:15:50.313 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:15:50.313 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:15:50.314 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f0 2024-05-22T21:15:50.314 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:15:50.314 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:15:50.314 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f1 2024-05-22T21:15:50.314 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:15:50.314 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-22T21:15:50.314 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:15:50.314 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-22T21:15:50.314 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:15:50.314 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42334sec preferred_lft 42334sec 2024-05-22T21:15:50.314 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-22T21:15:50.314 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:15:50.315 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:15:50.315 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:bd:4e:25:3d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:15:50.315 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:15:50.315 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:15:50.315 INFO:teuthology.orchestra.run.smithi179.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:15:50.315 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:15:50.315 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:15:50.315 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:15:50.315 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:15:50.315 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:15:50.315 INFO:teuthology.orchestra.run.smithi179.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:15:50.316 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-22T21:15:50.316 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:15:50.316 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:15:50.316 INFO:teuthology.orchestra.run.smithi179.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:15:50.316 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-22T21:15:50.316 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:15:50.316 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:15:50.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:50.317 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:15:50.379 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-22T21:15:50.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:50.382 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-22T21:15:50.435 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-22T21:15:50.435 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:15:50.435 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:15:50.435 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:15:50.435 INFO:teuthology.orchestra.run.smithi179.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:15:50.435 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:15:50.435 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:15:50.436 INFO:teuthology.orchestra.run.smithi179.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:15:50.436 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:15:50.436 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:15:50.436 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:15:50.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:50.436 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:15:50.436 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:15:50.436 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:15:50.436 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:15:50.436 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:15:50.437 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:15:50.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:50.533 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-22T21:15:50.542 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-22T21:15:50.542 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-22T21:15:50.543 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:15:50.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:50.543 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-22T21:15:50.603 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:15:50.603 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-22T21:15:50.655 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:15:50.655 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:15:50.655 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:15:50.706 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:15:50.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:50.707 DEBUG:teuthology.orchestra.run.smithi179:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:15:50.794 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:15:50.794 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:15:50.795 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:15:50.809 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:15:50.809 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:15:50.810 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:15:50.810 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:15:50.810 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:15:50.818 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:15:50.818 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:15:50.818 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:15:50.818 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:15:50.819 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:15:50.819 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:15:50.819 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:15:50.819 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:15:50.822 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:15:50.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:50.822 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:15:50.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:15:50.886 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:15:50.937 INFO:teuthology.orchestra.run.smithi179.stdout:client5420 2024-05-22T21:15:50.937 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:15:50.937 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:15:50.987 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:15:51.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.117+0000 7f2063a00640 1 Processor -- start 2024-05-22T21:15:51.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.117+0000 7f2063a00640 1 -- start start 2024-05-22T21:15:51.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.117+0000 7f2063a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f205c15f3a0 0x7f205c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:51.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.117+0000 7f2063a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205c1442a0 con 0x7f205c143ec0 2024-05-22T21:15:51.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.117+0000 7f2063a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205c070500 con 0x7f205c0715a0 2024-05-22T21:15:51.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.117+0000 7f2063a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205c0707e0 con 0x7f205c15f3a0 2024-05-22T21:15:51.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.117+0000 7f2061a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f205c15f3a0 0x7f205c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:51.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.121+0000 7f2061a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f205c15f3a0 0x7f205c15f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55834/0 (socket says 172.21.15.161:55834) 2024-05-22T21:15:51.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.121+0000 7f2061a00640 1 -- 172.21.15.161:0/2499658781 learned_addr learned my addr 172.21.15.161:0/2499658781 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:51.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.121+0000 7f2061000640 1 --1- 172.21.15.161:0/2499658781 >> v1:172.21.15.161:6789/0 conn(0x7f205c143ec0 0x7f205c06fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51912/0 (socket says 172.21.15.161:51912) 2024-05-22T21:15:51.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.121+0000 7f2061a00640 1 -- 172.21.15.161:0/2499658781 >> v1:172.21.15.179:6789/0 conn(0x7f205c0715a0 legacy=0x7f205c15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:51.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.121+0000 7f2061a00640 1 -- 172.21.15.161:0/2499658781 >> v1:172.21.15.161:6789/0 conn(0x7f205c143ec0 legacy=0x7f205c06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:51.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.121+0000 7f2061a00640 1 -- 172.21.15.161:0/2499658781 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c070ac0 con 0x7f205c15f3a0 2024-05-22T21:15:51.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.121+0000 7f2061a00640 1 --2- 172.21.15.161:0/2499658781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c15f3a0 0x7f205c15f830 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2054009e50 tx=0x7f205401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97fec65f4f64922 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:51.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.121+0000 7f205be00640 1 -- 172.21.15.161:0/2499658781 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20540a58d0 con 0x7f205c15f3a0 2024-05-22T21:15:51.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.121+0000 7f205be00640 1 -- 172.21.15.161:0/2499658781 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20540a5a30 con 0x7f205c15f3a0 2024-05-22T21:15:51.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.121+0000 7f205be00640 1 -- 172.21.15.161:0/2499658781 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20540a5dc0 con 0x7f205c15f3a0 2024-05-22T21:15:51.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2063a00640 1 -- 172.21.15.161:0/2499658781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c15f3a0 msgr2=0x7f205c15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:51.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2063a00640 1 --2- 172.21.15.161:0/2499658781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c15f3a0 0x7f205c15f830 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2054009e50 tx=0x7f205401ca00 comp rx=0 tx=0).stop 2024-05-22T21:15:51.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2063a00640 1 -- 172.21.15.161:0/2499658781 shutdown_connections 2024-05-22T21:15:51.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2063a00640 1 --2- 172.21.15.161:0/2499658781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c15f3a0 0x7f205c15f830 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:51.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2063a00640 1 -- 172.21.15.161:0/2499658781 >> 172.21.15.161:0/2499658781 conn(0x7f205c0b0bc0 msgr2=0x7f205c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:51.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2063a00640 1 -- 172.21.15.161:0/2499658781 shutdown_connections 2024-05-22T21:15:51.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2063a00640 1 -- 172.21.15.161:0/2499658781 wait complete. 2024-05-22T21:15:51.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2063a00640 1 Processor -- start 2024-05-22T21:15:51.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2063a00640 1 -- start start 2024-05-22T21:15:51.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2063a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c143ec0 0x7f205c15ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:51.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2063a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f205c15f3a0 0x7f205c15e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:51.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2063a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f205c13f5c0 0x7f205c13ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:51.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2063a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c15dc20 con 0x7f205c143ec0 2024-05-22T21:15:51.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2063a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c15d010 con 0x7f205c13f5c0 2024-05-22T21:15:51.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2063a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c15d2f0 con 0x7f205c15f3a0 2024-05-22T21:15:51.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2061000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c143ec0 0x7f205c15ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2061000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c143ec0 0x7f205c15ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55846/0 (socket says 172.21.15.161:55846) 2024-05-22T21:15:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2061000640 1 -- 172.21.15.161:0/2211558308 learned_addr learned my addr 172.21.15.161:0/2211558308 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:15:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2061a00640 1 --2- 172.21.15.161:0/2211558308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f205c13f5c0 0x7f205c13ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2053e00640 1 --2- 172.21.15.161:0/2211558308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f205c15f3a0 0x7f205c15e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2061000640 1 -- 172.21.15.161:0/2211558308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f205c15f3a0 msgr2=0x7f205c15e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2061000640 1 --2- 172.21.15.161:0/2211558308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f205c15f3a0 0x7f205c15e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2061000640 1 -- 172.21.15.161:0/2211558308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f205c13f5c0 msgr2=0x7f205c13ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2061000640 1 --2- 172.21.15.161:0/2211558308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f205c13f5c0 0x7f205c13ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2061000640 1 -- 172.21.15.161:0/2211558308 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2054009b30 con 0x7f205c143ec0 2024-05-22T21:15:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.125+0000 7f2061000640 1 --2- 172.21.15.161:0/2211558308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c143ec0 0x7f205c15ca30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f204c00c490 tx=0x7f204c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.129+0000 7f2059600640 1 -- 172.21.15.161:0/2211558308 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c0074e0 con 0x7f205c143ec0 2024-05-22T21:15:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.129+0000 7f2059600640 1 -- 172.21.15.161:0/2211558308 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c007640 con 0x7f205c143ec0 2024-05-22T21:15:51.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.129+0000 7f2059600640 1 -- 172.21.15.161:0/2211558308 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c004a10 con 0x7f205c143ec0 2024-05-22T21:15:51.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.129+0000 7f2063a00640 1 -- 172.21.15.161:0/2211558308 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205c15d5d0 con 0x7f205c143ec0 2024-05-22T21:15:51.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.129+0000 7f2063a00640 1 -- 172.21.15.161:0/2211558308 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c15e5f0 con 0x7f205c143ec0 2024-05-22T21:15:51.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.129+0000 7f2063a00640 1 -- 172.21.15.161:0/2211558308 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205c067220 con 0x7f205c143ec0 2024-05-22T21:15:51.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.129+0000 7f2059600640 1 -- 172.21.15.161:0/2211558308 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f204c004b70 con 0x7f205c143ec0 2024-05-22T21:15:51.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.129+0000 7f2059600640 1 --2- 172.21.15.161:0/2211558308 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2038066690 0x7f2038068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:15:51.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.129+0000 7f2059600640 1 -- 172.21.15.161:0/2211558308 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f204c083ae0 con 0x7f205c143ec0 2024-05-22T21:15:51.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.129+0000 7f2053e00640 1 --2- 172.21.15.161:0/2211558308 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2038066690 0x7f2038068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:15:51.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.133+0000 7f2053e00640 1 --2- 172.21.15.161:0/2211558308 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2038066690 0x7f2038068b50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f205c15c960 tx=0x7f2044009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:15:51.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.133+0000 7f2059600640 1 -- 172.21.15.161:0/2211558308 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f204c050e60 con 0x7f205c143ec0 2024-05-22T21:15:51.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.317+0000 7f2063a00640 1 -- 172.21.15.161:0/2211558308 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f205c0b18c0 con 0x7f205c143ec0 2024-05-22T21:15:51.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.321+0000 7f2059600640 1 -- 172.21.15.161:0/2211558308 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f204c054480 con 0x7f205c143ec0 2024-05-22T21:15:51.328 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:15:51.328 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1620021722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":1620021722},{"type":"v1","addr":"172.21.15.179:6835","nonce":1620021722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/3548738614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3548738614},{"type":"v1","addr":"172.21.15.161:6836","nonce":3548738614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/400898801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":400898801},{"type":"v1","addr":"172.21.15.161:6837","nonce":400898801}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:15:42.955245+0000","modified":"2024-05-22T21:15:45.977718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6834/2239627963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2239627963},{"type":"v1","addr":"172.21.15.179:6834","nonce":2239627963}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-22T21:15:51.329 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 31 2024-05-22T21:15:51.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.321+0000 7f2063a00640 1 -- 172.21.15.161:0/2211558308 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2038066690 msgr2=0x7f2038068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:51.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.321+0000 7f2063a00640 1 --2- 172.21.15.161:0/2211558308 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2038066690 0x7f2038068b50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f205c15c960 tx=0x7f2044009040 comp rx=0 tx=0).stop 2024-05-22T21:15:51.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.321+0000 7f2063a00640 1 -- 172.21.15.161:0/2211558308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c143ec0 msgr2=0x7f205c15ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:15:51.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.321+0000 7f2063a00640 1 --2- 172.21.15.161:0/2211558308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c143ec0 0x7f205c15ca30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f204c00c490 tx=0x7f204c00c950 comp rx=0 tx=0).stop 2024-05-22T21:15:51.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.321+0000 7f2063a00640 1 -- 172.21.15.161:0/2211558308 shutdown_connections 2024-05-22T21:15:51.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.321+0000 7f2063a00640 1 --2- 172.21.15.161:0/2211558308 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2038066690 0x7f2038068b50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:51.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.321+0000 7f2063a00640 1 --2- 172.21.15.161:0/2211558308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f205c13f5c0 0x7f205c13ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:51.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.321+0000 7f2063a00640 1 --2- 172.21.15.161:0/2211558308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f205c15f3a0 0x7f205c15e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:51.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.321+0000 7f2063a00640 1 --2- 172.21.15.161:0/2211558308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f205c143ec0 0x7f205c15ca30 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:15:51.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.321+0000 7f2063a00640 1 -- 172.21.15.161:0/2211558308 >> 172.21.15.161:0/2211558308 conn(0x7f205c0b0bc0 msgr2=0x7f205c0aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:15:51.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.321+0000 7f2063a00640 1 -- 172.21.15.161:0/2211558308 shutdown_connections 2024-05-22T21:15:51.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:15:51.321+0000 7f2063a00640 1 -- 172.21.15.161:0/2211558308 wait complete. 2024-05-22T21:15:51.347 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2024-05-22T21:15:51.503 INFO:teuthology.orchestra.run.smithi179.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":5946,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.5287248899999994,"mdsmap_epoch":31,"osdmap_epoch":67,"osdmap_epoch_barrier":67,"uptime":13.658883927} 2024-05-22T21:15:51.504 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 5946, "mdsmap_epoch": 31, "osdmap_epoch": 67, "osdmap_epoch_barrier": 67, "rank_uptime": 8.52872489, "state": "up:active", "uptime": 13.658883927, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:15:51.504 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-05-22T21:15:51.656 INFO:teuthology.orchestra.run.smithi161.stdout:{"osd_mon_report_interval":"5"} 2024-05-22T21:15:51.657 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-05-22T21:15:51.657 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-05-22T21:15:51.803 INFO:teuthology.orchestra.run.smithi161.stdout:{"mon_tick_interval":"5"} 2024-05-22T21:15:51.803 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-05-22T21:15:51.803 INFO:tasks.cephfs.test_full:pool capacity 557061696, 531MB should be enough to fill it 2024-05-22T21:15:51.803 DEBUG:teuthology.orchestra.run.smithi161:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-22T21:15:51.804 DEBUG:teuthology.orchestra.run.smithi161:> import time 2024-05-22T21:15:51.804 DEBUG:teuthology.orchestra.run.smithi161:> import datetime 2024-05-22T21:15:51.804 DEBUG:teuthology.orchestra.run.smithi161:> import subprocess 2024-05-22T21:15:51.804 DEBUG:teuthology.orchestra.run.smithi161:> import os 2024-05-22T21:15:51.804 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:15:51.804 DEBUG:teuthology.orchestra.run.smithi161:> # Write some buffered data through before going full, all should be well 2024-05-22T21:15:51.804 DEBUG:teuthology.orchestra.run.smithi161:> print("writing some data through which we expect to succeed") 2024-05-22T21:15:51.804 DEBUG:teuthology.orchestra.run.smithi161:> bytes = 0 2024-05-22T21:15:51.804 DEBUG:teuthology.orchestra.run.smithi161:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-05-22T21:15:51.804 DEBUG:teuthology.orchestra.run.smithi161:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-05-22T21:15:51.804 DEBUG:teuthology.orchestra.run.smithi161:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2024-05-22T21:15:51.804 DEBUG:teuthology.orchestra.run.smithi161:> os.fsync(f) 2024-05-22T21:15:51.805 DEBUG:teuthology.orchestra.run.smithi161:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-05-22T21:15:51.805 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:15:51.805 DEBUG:teuthology.orchestra.run.smithi161:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-05-22T21:15:51.805 DEBUG:teuthology.orchestra.run.smithi161:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-05-22T21:15:51.805 DEBUG:teuthology.orchestra.run.smithi161:> # buffered IO, the error should always only appear from fsync and not 2024-05-22T21:15:51.805 DEBUG:teuthology.orchestra.run.smithi161:> # from write 2024-05-22T21:15:51.805 DEBUG:teuthology.orchestra.run.smithi161:> full = False 2024-05-22T21:15:51.805 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:15:51.805 DEBUG:teuthology.orchestra.run.smithi161:> for n in range(0, int(531 * 0.9)): 2024-05-22T21:15:51.805 DEBUG:teuthology.orchestra.run.smithi161:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-05-22T21:15:51.805 DEBUG:teuthology.orchestra.run.smithi161:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2024-05-22T21:15:51.805 DEBUG:teuthology.orchestra.run.smithi161:> print("done writing {0} bytes".format(bytes)) 2024-05-22T21:15:51.805 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:15:51.805 DEBUG:teuthology.orchestra.run.smithi161:> # OK, now we should sneak in under the full condition 2024-05-22T21:15:51.806 DEBUG:teuthology.orchestra.run.smithi161:> # due to the time it takes the OSDs to report to the 2024-05-22T21:15:51.806 DEBUG:teuthology.orchestra.run.smithi161:> # mons, and get a successful fsync on our full-making data 2024-05-22T21:15:51.806 DEBUG:teuthology.orchestra.run.smithi161:> os.fsync(f) 2024-05-22T21:15:51.806 DEBUG:teuthology.orchestra.run.smithi161:> print("successfully fsync'"'"'ed prior to getting full state reported") 2024-05-22T21:15:51.806 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:15:51.806 DEBUG:teuthology.orchestra.run.smithi161:> # buffered write, add more dirty data to the buffer 2024-05-22T21:15:51.806 DEBUG:teuthology.orchestra.run.smithi161:> print("starting buffered write") 2024-05-22T21:15:51.806 DEBUG:teuthology.orchestra.run.smithi161:> try: 2024-05-22T21:15:51.806 DEBUG:teuthology.orchestra.run.smithi161:> for n in range(0, int(531 * 0.2)): 2024-05-22T21:15:51.806 DEBUG:teuthology.orchestra.run.smithi161:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-05-22T21:15:51.806 DEBUG:teuthology.orchestra.run.smithi161:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-05-22T21:15:51.806 DEBUG:teuthology.orchestra.run.smithi161:> time.sleep(15.0) 2024-05-22T21:15:51.806 DEBUG:teuthology.orchestra.run.smithi161:> except OSError: 2024-05-22T21:15:51.806 DEBUG:teuthology.orchestra.run.smithi161:> pass; 2024-05-22T21:15:51.807 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:15:51.807 DEBUG:teuthology.orchestra.run.smithi161:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2024-05-22T21:15:51.807 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:15:51.807 DEBUG:teuthology.orchestra.run.smithi161:> # Wait long enough for a background flush that should fail 2024-05-22T21:15:51.807 DEBUG:teuthology.orchestra.run.smithi161:> time.sleep(30) 2024-05-22T21:15:51.807 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:15:51.807 DEBUG:teuthology.orchestra.run.smithi161:> if False: 2024-05-22T21:15:51.807 DEBUG:teuthology.orchestra.run.smithi161:> # ...and check that the failed background flush is reflected in fclose 2024-05-22T21:15:51.807 DEBUG:teuthology.orchestra.run.smithi161:> try: 2024-05-22T21:15:51.807 DEBUG:teuthology.orchestra.run.smithi161:> os.close(f) 2024-05-22T21:15:51.807 DEBUG:teuthology.orchestra.run.smithi161:> except OSError: 2024-05-22T21:15:51.807 DEBUG:teuthology.orchestra.run.smithi161:> print("close() returned an error as expected") 2024-05-22T21:15:51.807 DEBUG:teuthology.orchestra.run.smithi161:> else: 2024-05-22T21:15:51.808 DEBUG:teuthology.orchestra.run.smithi161:> raise RuntimeError("close() failed to raise error") 2024-05-22T21:15:51.808 DEBUG:teuthology.orchestra.run.smithi161:> else: 2024-05-22T21:15:51.808 DEBUG:teuthology.orchestra.run.smithi161:> # The kernel cephfs client does not raise errors on fclose 2024-05-22T21:15:51.808 DEBUG:teuthology.orchestra.run.smithi161:> os.close(f) 2024-05-22T21:15:51.808 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:15:51.808 DEBUG:teuthology.orchestra.run.smithi161:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-05-22T21:15:51.808 DEBUG:teuthology.orchestra.run.smithi161:> ' 2024-05-22T21:15:53.028 INFO:teuthology.orchestra.run.smithi161.stdout:writing some data through which we expect to succeed 2024-05-22T21:15:53.028 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-05-22T21:15:53.028 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 1572864 bytes via buffered write, may repeat 2024-05-22T21:15:53.028 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 2621440 bytes via buffered write, may repeat 2024-05-22T21:15:53.029 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 3670016 bytes via buffered write, may repeat 2024-05-22T21:15:53.029 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 4718592 bytes via buffered write, may repeat 2024-05-22T21:15:53.029 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 5767168 bytes via buffered write, may repeat 2024-05-22T21:15:53.029 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 6815744 bytes via buffered write, may repeat 2024-05-22T21:15:53.029 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 7864320 bytes via buffered write, may repeat 2024-05-22T21:15:53.029 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 8912896 bytes via buffered write, may repeat 2024-05-22T21:15:53.030 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 9961472 bytes via buffered write, may repeat 2024-05-22T21:15:53.030 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 11010048 bytes via buffered write, may repeat 2024-05-22T21:15:53.030 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 12058624 bytes via buffered write, may repeat 2024-05-22T21:15:53.030 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 13107200 bytes via buffered write, may repeat 2024-05-22T21:15:53.030 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 14155776 bytes via buffered write, may repeat 2024-05-22T21:15:53.031 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 15204352 bytes via buffered write, may repeat 2024-05-22T21:15:53.031 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 16252928 bytes via buffered write, may repeat 2024-05-22T21:15:53.031 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 17301504 bytes via buffered write, may repeat 2024-05-22T21:15:53.031 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 18350080 bytes via buffered write, may repeat 2024-05-22T21:15:53.031 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 19398656 bytes via buffered write, may repeat 2024-05-22T21:15:53.031 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 20447232 bytes via buffered write, may repeat 2024-05-22T21:15:53.032 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 21495808 bytes via buffered write, may repeat 2024-05-22T21:15:53.032 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 22544384 bytes via buffered write, may repeat 2024-05-22T21:15:53.032 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 23592960 bytes via buffered write, may repeat 2024-05-22T21:15:53.032 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 24641536 bytes via buffered write, may repeat 2024-05-22T21:15:53.032 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 25690112 bytes via buffered write, may repeat 2024-05-22T21:15:53.032 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 26738688 bytes via buffered write, may repeat 2024-05-22T21:15:53.032 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 27787264 bytes via buffered write, may repeat 2024-05-22T21:15:53.033 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 28835840 bytes via buffered write, may repeat 2024-05-22T21:15:53.033 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 29884416 bytes via buffered write, may repeat 2024-05-22T21:15:53.033 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 30932992 bytes via buffered write, may repeat 2024-05-22T21:15:53.033 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 31981568 bytes via buffered write, may repeat 2024-05-22T21:15:53.033 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 33030144 bytes via buffered write, may repeat 2024-05-22T21:15:53.033 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 34078720 bytes via buffered write, may repeat 2024-05-22T21:15:53.033 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 35127296 bytes via buffered write, may repeat 2024-05-22T21:15:53.034 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 36175872 bytes via buffered write, may repeat 2024-05-22T21:15:53.034 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 37224448 bytes via buffered write, may repeat 2024-05-22T21:15:53.034 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 38273024 bytes via buffered write, may repeat 2024-05-22T21:15:53.034 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 39321600 bytes via buffered write, may repeat 2024-05-22T21:15:53.034 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 40370176 bytes via buffered write, may repeat 2024-05-22T21:15:53.034 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 41418752 bytes via buffered write, may repeat 2024-05-22T21:15:53.035 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 42467328 bytes via buffered write, may repeat 2024-05-22T21:15:53.035 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 43515904 bytes via buffered write, may repeat 2024-05-22T21:15:53.035 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 44564480 bytes via buffered write, may repeat 2024-05-22T21:15:53.035 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 45613056 bytes via buffered write, may repeat 2024-05-22T21:15:53.035 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 46661632 bytes via buffered write, may repeat 2024-05-22T21:15:53.035 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 47710208 bytes via buffered write, may repeat 2024-05-22T21:15:53.035 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 48758784 bytes via buffered write, may repeat 2024-05-22T21:15:53.036 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 49807360 bytes via buffered write, may repeat 2024-05-22T21:15:53.036 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 50855936 bytes via buffered write, may repeat 2024-05-22T21:15:53.036 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 51904512 bytes via buffered write, may repeat 2024-05-22T21:15:53.036 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 52953088 bytes via buffered write, may repeat 2024-05-22T21:15:53.036 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 54001664 bytes via buffered write, may repeat 2024-05-22T21:15:53.036 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 55050240 bytes via buffered write, may repeat 2024-05-22T21:15:53.037 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 56098816 bytes via buffered write, may repeat 2024-05-22T21:15:53.037 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 57147392 bytes via buffered write, may repeat 2024-05-22T21:15:53.037 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 58195968 bytes via buffered write, may repeat 2024-05-22T21:15:53.037 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 59244544 bytes via buffered write, may repeat 2024-05-22T21:15:53.037 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 60293120 bytes via buffered write, may repeat 2024-05-22T21:15:53.037 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 61341696 bytes via buffered write, may repeat 2024-05-22T21:15:53.037 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 62390272 bytes via buffered write, may repeat 2024-05-22T21:15:53.037 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 63438848 bytes via buffered write, may repeat 2024-05-22T21:15:53.038 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 64487424 bytes via buffered write, may repeat 2024-05-22T21:15:53.038 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 65536000 bytes via buffered write, may repeat 2024-05-22T21:15:53.038 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 66584576 bytes via buffered write, may repeat 2024-05-22T21:15:53.038 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 67633152 bytes via buffered write, may repeat 2024-05-22T21:15:53.038 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 68681728 bytes via buffered write, may repeat 2024-05-22T21:15:53.038 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 69730304 bytes via buffered write, may repeat 2024-05-22T21:15:53.038 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 70778880 bytes via buffered write, may repeat 2024-05-22T21:15:53.038 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 71827456 bytes via buffered write, may repeat 2024-05-22T21:15:53.038 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 72876032 bytes via buffered write, may repeat 2024-05-22T21:15:53.038 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 73924608 bytes via buffered write, may repeat 2024-05-22T21:15:53.038 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 74973184 bytes via buffered write, may repeat 2024-05-22T21:15:53.039 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 76021760 bytes via buffered write, may repeat 2024-05-22T21:15:53.039 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 77070336 bytes via buffered write, may repeat 2024-05-22T21:15:53.039 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 78118912 bytes via buffered write, may repeat 2024-05-22T21:15:53.039 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 79167488 bytes via buffered write, may repeat 2024-05-22T21:15:53.039 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 80216064 bytes via buffered write, may repeat 2024-05-22T21:15:53.039 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 81264640 bytes via buffered write, may repeat 2024-05-22T21:15:53.039 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 82313216 bytes via buffered write, may repeat 2024-05-22T21:15:53.039 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 83361792 bytes via buffered write, may repeat 2024-05-22T21:15:53.039 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 84410368 bytes via buffered write, may repeat 2024-05-22T21:15:53.039 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 85458944 bytes via buffered write, may repeat 2024-05-22T21:15:53.039 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 86507520 bytes via buffered write, may repeat 2024-05-22T21:15:53.039 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 87556096 bytes via buffered write, may repeat 2024-05-22T21:15:53.039 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 88604672 bytes via buffered write, may repeat 2024-05-22T21:15:53.039 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 89653248 bytes via buffered write, may repeat 2024-05-22T21:15:53.040 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 90701824 bytes via buffered write, may repeat 2024-05-22T21:15:53.040 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 91750400 bytes via buffered write, may repeat 2024-05-22T21:15:53.040 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 92798976 bytes via buffered write, may repeat 2024-05-22T21:15:53.040 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 93847552 bytes via buffered write, may repeat 2024-05-22T21:15:53.040 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 94896128 bytes via buffered write, may repeat 2024-05-22T21:15:53.040 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 95944704 bytes via buffered write, may repeat 2024-05-22T21:15:53.040 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 96993280 bytes via buffered write, may repeat 2024-05-22T21:15:53.040 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 98041856 bytes via buffered write, may repeat 2024-05-22T21:15:53.040 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 99090432 bytes via buffered write, may repeat 2024-05-22T21:15:53.040 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 100139008 bytes via buffered write, may repeat 2024-05-22T21:15:53.040 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 101187584 bytes via buffered write, may repeat 2024-05-22T21:15:53.040 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 102236160 bytes via buffered write, may repeat 2024-05-22T21:15:53.040 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 103284736 bytes via buffered write, may repeat 2024-05-22T21:15:53.041 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 104333312 bytes via buffered write, may repeat 2024-05-22T21:15:53.041 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 105381888 bytes via buffered write, may repeat 2024-05-22T21:15:53.041 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 106430464 bytes via buffered write, may repeat 2024-05-22T21:15:53.041 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 107479040 bytes via buffered write, may repeat 2024-05-22T21:15:53.041 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 108527616 bytes via buffered write, may repeat 2024-05-22T21:15:53.041 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 109576192 bytes via buffered write, may repeat 2024-05-22T21:15:53.041 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 110624768 bytes via buffered write, may repeat 2024-05-22T21:15:53.041 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 111673344 bytes via buffered write, may repeat 2024-05-22T21:15:53.041 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 112721920 bytes via buffered write, may repeat 2024-05-22T21:15:53.041 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 113770496 bytes via buffered write, may repeat 2024-05-22T21:15:53.041 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 114819072 bytes via buffered write, may repeat 2024-05-22T21:15:53.041 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 115867648 bytes via buffered write, may repeat 2024-05-22T21:15:53.042 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 116916224 bytes via buffered write, may repeat 2024-05-22T21:15:53.042 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 117964800 bytes via buffered write, may repeat 2024-05-22T21:15:53.042 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 119013376 bytes via buffered write, may repeat 2024-05-22T21:15:53.042 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 120061952 bytes via buffered write, may repeat 2024-05-22T21:15:53.042 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 121110528 bytes via buffered write, may repeat 2024-05-22T21:15:53.042 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 122159104 bytes via buffered write, may repeat 2024-05-22T21:15:53.042 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 123207680 bytes via buffered write, may repeat 2024-05-22T21:15:53.042 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 124256256 bytes via buffered write, may repeat 2024-05-22T21:15:53.042 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 125304832 bytes via buffered write, may repeat 2024-05-22T21:15:53.042 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 126353408 bytes via buffered write, may repeat 2024-05-22T21:15:53.042 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 127401984 bytes via buffered write, may repeat 2024-05-22T21:15:53.042 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 128450560 bytes via buffered write, may repeat 2024-05-22T21:15:53.043 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 129499136 bytes via buffered write, may repeat 2024-05-22T21:15:53.043 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 130547712 bytes via buffered write, may repeat 2024-05-22T21:15:53.043 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 131596288 bytes via buffered write, may repeat 2024-05-22T21:15:53.043 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 132644864 bytes via buffered write, may repeat 2024-05-22T21:15:53.043 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 133693440 bytes via buffered write, may repeat 2024-05-22T21:15:53.043 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 134742016 bytes via buffered write, may repeat 2024-05-22T21:15:53.043 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 135790592 bytes via buffered write, may repeat 2024-05-22T21:15:53.043 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 136839168 bytes via buffered write, may repeat 2024-05-22T21:15:53.043 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 137887744 bytes via buffered write, may repeat 2024-05-22T21:15:53.043 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 138936320 bytes via buffered write, may repeat 2024-05-22T21:15:53.043 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 139984896 bytes via buffered write, may repeat 2024-05-22T21:15:53.043 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 141033472 bytes via buffered write, may repeat 2024-05-22T21:15:53.043 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 142082048 bytes via buffered write, may repeat 2024-05-22T21:15:53.044 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 143130624 bytes via buffered write, may repeat 2024-05-22T21:15:53.044 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 144179200 bytes via buffered write, may repeat 2024-05-22T21:15:53.044 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 145227776 bytes via buffered write, may repeat 2024-05-22T21:15:53.044 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 146276352 bytes via buffered write, may repeat 2024-05-22T21:15:53.044 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 147324928 bytes via buffered write, may repeat 2024-05-22T21:15:53.044 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 148373504 bytes via buffered write, may repeat 2024-05-22T21:15:53.044 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 149422080 bytes via buffered write, may repeat 2024-05-22T21:15:53.044 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 150470656 bytes via buffered write, may repeat 2024-05-22T21:15:53.044 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 151519232 bytes via buffered write, may repeat 2024-05-22T21:15:53.044 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 152567808 bytes via buffered write, may repeat 2024-05-22T21:15:53.044 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 153616384 bytes via buffered write, may repeat 2024-05-22T21:15:53.044 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 154664960 bytes via buffered write, may repeat 2024-05-22T21:15:53.044 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 155713536 bytes via buffered write, may repeat 2024-05-22T21:15:53.045 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 156762112 bytes via buffered write, may repeat 2024-05-22T21:15:53.045 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 157810688 bytes via buffered write, may repeat 2024-05-22T21:15:53.045 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 158859264 bytes via buffered write, may repeat 2024-05-22T21:15:53.045 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 159907840 bytes via buffered write, may repeat 2024-05-22T21:15:53.045 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 160956416 bytes via buffered write, may repeat 2024-05-22T21:15:53.045 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 162004992 bytes via buffered write, may repeat 2024-05-22T21:15:53.191 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 163053568 bytes via buffered write, may repeat 2024-05-22T21:15:53.192 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 164102144 bytes via buffered write, may repeat 2024-05-22T21:15:53.192 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 165150720 bytes via buffered write, may repeat 2024-05-22T21:15:53.192 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 166199296 bytes via buffered write, may repeat 2024-05-22T21:15:53.192 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 167247872 bytes via buffered write, may repeat 2024-05-22T21:15:53.192 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 168296448 bytes via buffered write, may repeat 2024-05-22T21:15:53.192 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 169345024 bytes via buffered write, may repeat 2024-05-22T21:15:53.192 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 170393600 bytes via buffered write, may repeat 2024-05-22T21:15:53.192 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 171442176 bytes via buffered write, may repeat 2024-05-22T21:15:53.192 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 172490752 bytes via buffered write, may repeat 2024-05-22T21:15:53.192 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 173539328 bytes via buffered write, may repeat 2024-05-22T21:15:53.192 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 174587904 bytes via buffered write, may repeat 2024-05-22T21:15:53.193 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 175636480 bytes via buffered write, may repeat 2024-05-22T21:15:53.193 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 176685056 bytes via buffered write, may repeat 2024-05-22T21:15:53.193 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 177733632 bytes via buffered write, may repeat 2024-05-22T21:15:53.193 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 178782208 bytes via buffered write, may repeat 2024-05-22T21:15:53.193 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 179830784 bytes via buffered write, may repeat 2024-05-22T21:15:53.193 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 180879360 bytes via buffered write, may repeat 2024-05-22T21:15:53.193 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 181927936 bytes via buffered write, may repeat 2024-05-22T21:15:53.193 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 182976512 bytes via buffered write, may repeat 2024-05-22T21:15:53.193 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 184025088 bytes via buffered write, may repeat 2024-05-22T21:15:53.193 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 185073664 bytes via buffered write, may repeat 2024-05-22T21:15:53.193 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 186122240 bytes via buffered write, may repeat 2024-05-22T21:15:53.193 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 187170816 bytes via buffered write, may repeat 2024-05-22T21:15:53.193 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 188219392 bytes via buffered write, may repeat 2024-05-22T21:15:53.194 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 189267968 bytes via buffered write, may repeat 2024-05-22T21:15:53.194 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 190316544 bytes via buffered write, may repeat 2024-05-22T21:15:53.194 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 191365120 bytes via buffered write, may repeat 2024-05-22T21:15:53.194 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 192413696 bytes via buffered write, may repeat 2024-05-22T21:15:53.194 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 193462272 bytes via buffered write, may repeat 2024-05-22T21:15:53.194 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 194510848 bytes via buffered write, may repeat 2024-05-22T21:15:53.194 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 195559424 bytes via buffered write, may repeat 2024-05-22T21:15:53.194 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 196608000 bytes via buffered write, may repeat 2024-05-22T21:15:53.194 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 197656576 bytes via buffered write, may repeat 2024-05-22T21:15:53.194 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 198705152 bytes via buffered write, may repeat 2024-05-22T21:15:53.194 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 199753728 bytes via buffered write, may repeat 2024-05-22T21:15:53.194 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 200802304 bytes via buffered write, may repeat 2024-05-22T21:15:53.195 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 201850880 bytes via buffered write, may repeat 2024-05-22T21:15:53.195 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 202899456 bytes via buffered write, may repeat 2024-05-22T21:15:53.195 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 203948032 bytes via buffered write, may repeat 2024-05-22T21:15:53.195 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 204996608 bytes via buffered write, may repeat 2024-05-22T21:15:53.195 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 206045184 bytes via buffered write, may repeat 2024-05-22T21:15:53.195 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 207093760 bytes via buffered write, may repeat 2024-05-22T21:15:53.195 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 208142336 bytes via buffered write, may repeat 2024-05-22T21:15:53.195 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 209190912 bytes via buffered write, may repeat 2024-05-22T21:15:53.195 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 210239488 bytes via buffered write, may repeat 2024-05-22T21:15:53.195 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 211288064 bytes via buffered write, may repeat 2024-05-22T21:15:53.195 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 212336640 bytes via buffered write, may repeat 2024-05-22T21:15:53.195 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 213385216 bytes via buffered write, may repeat 2024-05-22T21:15:53.195 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 214433792 bytes via buffered write, may repeat 2024-05-22T21:15:53.195 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 215482368 bytes via buffered write, may repeat 2024-05-22T21:15:53.196 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 216530944 bytes via buffered write, may repeat 2024-05-22T21:15:53.196 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 217579520 bytes via buffered write, may repeat 2024-05-22T21:15:53.196 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 218628096 bytes via buffered write, may repeat 2024-05-22T21:15:53.196 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 219676672 bytes via buffered write, may repeat 2024-05-22T21:15:53.196 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 220725248 bytes via buffered write, may repeat 2024-05-22T21:15:53.196 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 221773824 bytes via buffered write, may repeat 2024-05-22T21:15:53.196 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 222822400 bytes via buffered write, may repeat 2024-05-22T21:15:53.196 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 223870976 bytes via buffered write, may repeat 2024-05-22T21:15:53.196 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 224919552 bytes via buffered write, may repeat 2024-05-22T21:15:53.196 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 225968128 bytes via buffered write, may repeat 2024-05-22T21:15:53.196 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 227016704 bytes via buffered write, may repeat 2024-05-22T21:15:53.196 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 228065280 bytes via buffered write, may repeat 2024-05-22T21:15:53.196 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 229113856 bytes via buffered write, may repeat 2024-05-22T21:15:53.197 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 230162432 bytes via buffered write, may repeat 2024-05-22T21:15:53.197 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 231211008 bytes via buffered write, may repeat 2024-05-22T21:15:53.197 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 232259584 bytes via buffered write, may repeat 2024-05-22T21:15:53.197 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 233308160 bytes via buffered write, may repeat 2024-05-22T21:15:53.197 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 234356736 bytes via buffered write, may repeat 2024-05-22T21:15:53.197 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 235405312 bytes via buffered write, may repeat 2024-05-22T21:15:53.197 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 236453888 bytes via buffered write, may repeat 2024-05-22T21:15:53.197 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 237502464 bytes via buffered write, may repeat 2024-05-22T21:15:53.197 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 238551040 bytes via buffered write, may repeat 2024-05-22T21:15:53.197 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 239599616 bytes via buffered write, may repeat 2024-05-22T21:15:53.197 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 240648192 bytes via buffered write, may repeat 2024-05-22T21:15:53.197 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 241696768 bytes via buffered write, may repeat 2024-05-22T21:15:53.197 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 242745344 bytes via buffered write, may repeat 2024-05-22T21:15:53.198 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 243793920 bytes via buffered write, may repeat 2024-05-22T21:15:53.198 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 244842496 bytes via buffered write, may repeat 2024-05-22T21:15:53.198 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 245891072 bytes via buffered write, may repeat 2024-05-22T21:15:53.198 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 246939648 bytes via buffered write, may repeat 2024-05-22T21:15:53.198 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 247988224 bytes via buffered write, may repeat 2024-05-22T21:15:53.198 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 249036800 bytes via buffered write, may repeat 2024-05-22T21:15:53.198 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 250085376 bytes via buffered write, may repeat 2024-05-22T21:15:53.198 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 251133952 bytes via buffered write, may repeat 2024-05-22T21:15:53.198 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 252182528 bytes via buffered write, may repeat 2024-05-22T21:15:53.198 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 253231104 bytes via buffered write, may repeat 2024-05-22T21:15:53.198 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 254279680 bytes via buffered write, may repeat 2024-05-22T21:15:53.198 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 255328256 bytes via buffered write, may repeat 2024-05-22T21:15:53.198 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 256376832 bytes via buffered write, may repeat 2024-05-22T21:15:53.199 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 257425408 bytes via buffered write, may repeat 2024-05-22T21:15:53.199 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 258473984 bytes via buffered write, may repeat 2024-05-22T21:15:53.199 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 259522560 bytes via buffered write, may repeat 2024-05-22T21:15:53.199 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 260571136 bytes via buffered write, may repeat 2024-05-22T21:15:53.199 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 261619712 bytes via buffered write, may repeat 2024-05-22T21:15:53.199 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 262668288 bytes via buffered write, may repeat 2024-05-22T21:15:53.199 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 263716864 bytes via buffered write, may repeat 2024-05-22T21:15:53.199 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 264765440 bytes via buffered write, may repeat 2024-05-22T21:15:53.199 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 265814016 bytes via buffered write, may repeat 2024-05-22T21:15:53.199 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 266862592 bytes via buffered write, may repeat 2024-05-22T21:15:53.199 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 267911168 bytes via buffered write, may repeat 2024-05-22T21:15:53.200 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 268959744 bytes via buffered write, may repeat 2024-05-22T21:15:53.200 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 270008320 bytes via buffered write, may repeat 2024-05-22T21:15:53.200 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 271056896 bytes via buffered write, may repeat 2024-05-22T21:15:53.200 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 272105472 bytes via buffered write, may repeat 2024-05-22T21:15:53.200 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 273154048 bytes via buffered write, may repeat 2024-05-22T21:15:53.200 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 274202624 bytes via buffered write, may repeat 2024-05-22T21:15:53.200 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 275251200 bytes via buffered write, may repeat 2024-05-22T21:15:53.200 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 276299776 bytes via buffered write, may repeat 2024-05-22T21:15:53.200 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 277348352 bytes via buffered write, may repeat 2024-05-22T21:15:53.200 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 278396928 bytes via buffered write, may repeat 2024-05-22T21:15:53.200 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 279445504 bytes via buffered write, may repeat 2024-05-22T21:15:53.200 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 280494080 bytes via buffered write, may repeat 2024-05-22T21:15:53.200 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 281542656 bytes via buffered write, may repeat 2024-05-22T21:15:53.201 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 282591232 bytes via buffered write, may repeat 2024-05-22T21:15:53.201 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 283639808 bytes via buffered write, may repeat 2024-05-22T21:15:53.201 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 284688384 bytes via buffered write, may repeat 2024-05-22T21:15:53.201 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 285736960 bytes via buffered write, may repeat 2024-05-22T21:15:53.201 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 286785536 bytes via buffered write, may repeat 2024-05-22T21:15:53.201 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 287834112 bytes via buffered write, may repeat 2024-05-22T21:15:53.201 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 288882688 bytes via buffered write, may repeat 2024-05-22T21:15:53.201 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 289931264 bytes via buffered write, may repeat 2024-05-22T21:15:53.201 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 290979840 bytes via buffered write, may repeat 2024-05-22T21:15:53.201 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 292028416 bytes via buffered write, may repeat 2024-05-22T21:15:53.201 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 293076992 bytes via buffered write, may repeat 2024-05-22T21:15:53.202 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 294125568 bytes via buffered write, may repeat 2024-05-22T21:15:53.202 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 295174144 bytes via buffered write, may repeat 2024-05-22T21:15:53.202 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 296222720 bytes via buffered write, may repeat 2024-05-22T21:15:53.202 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 297271296 bytes via buffered write, may repeat 2024-05-22T21:15:53.202 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 298319872 bytes via buffered write, may repeat 2024-05-22T21:15:53.202 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 299368448 bytes via buffered write, may repeat 2024-05-22T21:15:53.202 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 300417024 bytes via buffered write, may repeat 2024-05-22T21:15:53.202 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 301465600 bytes via buffered write, may repeat 2024-05-22T21:15:53.202 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 302514176 bytes via buffered write, may repeat 2024-05-22T21:15:53.202 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 303562752 bytes via buffered write, may repeat 2024-05-22T21:15:53.202 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 304611328 bytes via buffered write, may repeat 2024-05-22T21:15:53.203 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 305659904 bytes via buffered write, may repeat 2024-05-22T21:15:53.203 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 306708480 bytes via buffered write, may repeat 2024-05-22T21:15:53.203 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 307757056 bytes via buffered write, may repeat 2024-05-22T21:15:53.203 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 308805632 bytes via buffered write, may repeat 2024-05-22T21:15:53.203 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 309854208 bytes via buffered write, may repeat 2024-05-22T21:15:53.203 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 310902784 bytes via buffered write, may repeat 2024-05-22T21:15:53.204 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 311951360 bytes via buffered write, may repeat 2024-05-22T21:15:53.204 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 312999936 bytes via buffered write, may repeat 2024-05-22T21:15:53.204 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 314048512 bytes via buffered write, may repeat 2024-05-22T21:15:53.204 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 315097088 bytes via buffered write, may repeat 2024-05-22T21:15:53.204 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 316145664 bytes via buffered write, may repeat 2024-05-22T21:15:53.204 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 317194240 bytes via buffered write, may repeat 2024-05-22T21:15:53.205 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 318242816 bytes via buffered write, may repeat 2024-05-22T21:15:53.205 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 319291392 bytes via buffered write, may repeat 2024-05-22T21:15:53.205 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 320339968 bytes via buffered write, may repeat 2024-05-22T21:15:53.205 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 321388544 bytes via buffered write, may repeat 2024-05-22T21:15:53.205 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 322437120 bytes via buffered write, may repeat 2024-05-22T21:15:53.205 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 323485696 bytes via buffered write, may repeat 2024-05-22T21:15:53.266 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:15:53.269 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:15:53.355 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 324534272 bytes via buffered write, may repeat 2024-05-22T21:15:53.355 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 325582848 bytes via buffered write, may repeat 2024-05-22T21:15:53.355 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 326631424 bytes via buffered write, may repeat 2024-05-22T21:15:53.356 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 327680000 bytes via buffered write, may repeat 2024-05-22T21:15:53.356 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 328728576 bytes via buffered write, may repeat 2024-05-22T21:15:53.356 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 329777152 bytes via buffered write, may repeat 2024-05-22T21:15:53.356 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 330825728 bytes via buffered write, may repeat 2024-05-22T21:15:53.356 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 331874304 bytes via buffered write, may repeat 2024-05-22T21:15:53.356 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 332922880 bytes via buffered write, may repeat 2024-05-22T21:15:53.356 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 333971456 bytes via buffered write, may repeat 2024-05-22T21:15:53.356 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 335020032 bytes via buffered write, may repeat 2024-05-22T21:15:53.356 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 336068608 bytes via buffered write, may repeat 2024-05-22T21:15:53.357 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 337117184 bytes via buffered write, may repeat 2024-05-22T21:15:53.357 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 338165760 bytes via buffered write, may repeat 2024-05-22T21:15:53.357 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 339214336 bytes via buffered write, may repeat 2024-05-22T21:15:53.357 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 340262912 bytes via buffered write, may repeat 2024-05-22T21:15:53.357 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 341311488 bytes via buffered write, may repeat 2024-05-22T21:15:53.357 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 342360064 bytes via buffered write, may repeat 2024-05-22T21:15:53.357 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 343408640 bytes via buffered write, may repeat 2024-05-22T21:15:53.357 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 344457216 bytes via buffered write, may repeat 2024-05-22T21:15:53.357 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 345505792 bytes via buffered write, may repeat 2024-05-22T21:15:53.357 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 346554368 bytes via buffered write, may repeat 2024-05-22T21:15:53.357 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 347602944 bytes via buffered write, may repeat 2024-05-22T21:15:53.357 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 348651520 bytes via buffered write, may repeat 2024-05-22T21:15:53.357 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 349700096 bytes via buffered write, may repeat 2024-05-22T21:15:53.358 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 350748672 bytes via buffered write, may repeat 2024-05-22T21:15:53.358 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 351797248 bytes via buffered write, may repeat 2024-05-22T21:15:53.358 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 352845824 bytes via buffered write, may repeat 2024-05-22T21:15:53.358 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 353894400 bytes via buffered write, may repeat 2024-05-22T21:15:53.358 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 354942976 bytes via buffered write, may repeat 2024-05-22T21:15:53.358 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 355991552 bytes via buffered write, may repeat 2024-05-22T21:15:53.358 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 357040128 bytes via buffered write, may repeat 2024-05-22T21:15:53.358 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 358088704 bytes via buffered write, may repeat 2024-05-22T21:15:53.358 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 359137280 bytes via buffered write, may repeat 2024-05-22T21:15:53.358 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 360185856 bytes via buffered write, may repeat 2024-05-22T21:15:53.358 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 361234432 bytes via buffered write, may repeat 2024-05-22T21:15:53.358 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 362283008 bytes via buffered write, may repeat 2024-05-22T21:15:53.358 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 363331584 bytes via buffered write, may repeat 2024-05-22T21:15:53.359 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 364380160 bytes via buffered write, may repeat 2024-05-22T21:15:53.359 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 365428736 bytes via buffered write, may repeat 2024-05-22T21:15:53.359 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 366477312 bytes via buffered write, may repeat 2024-05-22T21:15:53.359 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 367525888 bytes via buffered write, may repeat 2024-05-22T21:15:53.359 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 368574464 bytes via buffered write, may repeat 2024-05-22T21:15:53.359 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 369623040 bytes via buffered write, may repeat 2024-05-22T21:15:53.359 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 370671616 bytes via buffered write, may repeat 2024-05-22T21:15:53.359 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 371720192 bytes via buffered write, may repeat 2024-05-22T21:15:53.359 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 372768768 bytes via buffered write, may repeat 2024-05-22T21:15:53.359 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 373817344 bytes via buffered write, may repeat 2024-05-22T21:15:53.359 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 374865920 bytes via buffered write, may repeat 2024-05-22T21:15:53.359 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 375914496 bytes via buffered write, may repeat 2024-05-22T21:15:53.359 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 376963072 bytes via buffered write, may repeat 2024-05-22T21:15:53.360 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 378011648 bytes via buffered write, may repeat 2024-05-22T21:15:53.360 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 379060224 bytes via buffered write, may repeat 2024-05-22T21:15:53.360 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 380108800 bytes via buffered write, may repeat 2024-05-22T21:15:53.360 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 381157376 bytes via buffered write, may repeat 2024-05-22T21:15:53.360 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 382205952 bytes via buffered write, may repeat 2024-05-22T21:15:53.360 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 383254528 bytes via buffered write, may repeat 2024-05-22T21:15:53.360 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 384303104 bytes via buffered write, may repeat 2024-05-22T21:15:53.360 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 385351680 bytes via buffered write, may repeat 2024-05-22T21:15:53.360 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 386400256 bytes via buffered write, may repeat 2024-05-22T21:15:53.360 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 387448832 bytes via buffered write, may repeat 2024-05-22T21:15:53.360 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 388497408 bytes via buffered write, may repeat 2024-05-22T21:15:53.360 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 389545984 bytes via buffered write, may repeat 2024-05-22T21:15:53.360 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 390594560 bytes via buffered write, may repeat 2024-05-22T21:15:53.361 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 391643136 bytes via buffered write, may repeat 2024-05-22T21:15:53.361 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 392691712 bytes via buffered write, may repeat 2024-05-22T21:15:53.361 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 393740288 bytes via buffered write, may repeat 2024-05-22T21:15:53.361 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 394788864 bytes via buffered write, may repeat 2024-05-22T21:15:53.361 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 395837440 bytes via buffered write, may repeat 2024-05-22T21:15:53.361 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 396886016 bytes via buffered write, may repeat 2024-05-22T21:15:53.361 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 397934592 bytes via buffered write, may repeat 2024-05-22T21:15:53.361 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 398983168 bytes via buffered write, may repeat 2024-05-22T21:15:53.361 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 400031744 bytes via buffered write, may repeat 2024-05-22T21:15:53.361 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 401080320 bytes via buffered write, may repeat 2024-05-22T21:15:53.361 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 402128896 bytes via buffered write, may repeat 2024-05-22T21:15:53.361 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 403177472 bytes via buffered write, may repeat 2024-05-22T21:15:53.361 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 404226048 bytes via buffered write, may repeat 2024-05-22T21:15:53.362 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 405274624 bytes via buffered write, may repeat 2024-05-22T21:15:53.362 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 406323200 bytes via buffered write, may repeat 2024-05-22T21:15:53.362 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 407371776 bytes via buffered write, may repeat 2024-05-22T21:15:53.362 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 408420352 bytes via buffered write, may repeat 2024-05-22T21:15:53.362 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 409468928 bytes via buffered write, may repeat 2024-05-22T21:15:53.362 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 410517504 bytes via buffered write, may repeat 2024-05-22T21:15:53.362 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 411566080 bytes via buffered write, may repeat 2024-05-22T21:15:53.362 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 412614656 bytes via buffered write, may repeat 2024-05-22T21:15:53.362 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 413663232 bytes via buffered write, may repeat 2024-05-22T21:15:53.362 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 414711808 bytes via buffered write, may repeat 2024-05-22T21:15:53.362 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 415760384 bytes via buffered write, may repeat 2024-05-22T21:15:53.362 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 416808960 bytes via buffered write, may repeat 2024-05-22T21:15:53.362 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 417857536 bytes via buffered write, may repeat 2024-05-22T21:15:53.363 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 418906112 bytes via buffered write, may repeat 2024-05-22T21:15:53.363 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 419954688 bytes via buffered write, may repeat 2024-05-22T21:15:53.363 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 421003264 bytes via buffered write, may repeat 2024-05-22T21:15:53.363 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 422051840 bytes via buffered write, may repeat 2024-05-22T21:15:53.363 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 423100416 bytes via buffered write, may repeat 2024-05-22T21:15:53.363 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 424148992 bytes via buffered write, may repeat 2024-05-22T21:15:53.363 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 425197568 bytes via buffered write, may repeat 2024-05-22T21:15:53.363 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 426246144 bytes via buffered write, may repeat 2024-05-22T21:15:53.363 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 427294720 bytes via buffered write, may repeat 2024-05-22T21:15:53.363 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 428343296 bytes via buffered write, may repeat 2024-05-22T21:15:53.363 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 429391872 bytes via buffered write, may repeat 2024-05-22T21:15:53.363 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 430440448 bytes via buffered write, may repeat 2024-05-22T21:15:53.363 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 431489024 bytes via buffered write, may repeat 2024-05-22T21:15:53.364 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 432537600 bytes via buffered write, may repeat 2024-05-22T21:15:53.364 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 433586176 bytes via buffered write, may repeat 2024-05-22T21:15:53.364 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 434634752 bytes via buffered write, may repeat 2024-05-22T21:15:53.364 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 435683328 bytes via buffered write, may repeat 2024-05-22T21:15:53.364 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 436731904 bytes via buffered write, may repeat 2024-05-22T21:15:53.364 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 437780480 bytes via buffered write, may repeat 2024-05-22T21:15:53.364 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 438829056 bytes via buffered write, may repeat 2024-05-22T21:15:53.364 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 439877632 bytes via buffered write, may repeat 2024-05-22T21:15:53.364 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 440926208 bytes via buffered write, may repeat 2024-05-22T21:15:53.365 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 441974784 bytes via buffered write, may repeat 2024-05-22T21:15:53.365 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 443023360 bytes via buffered write, may repeat 2024-05-22T21:15:53.365 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 444071936 bytes via buffered write, may repeat 2024-05-22T21:15:53.365 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 445120512 bytes via buffered write, may repeat 2024-05-22T21:15:53.365 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 446169088 bytes via buffered write, may repeat 2024-05-22T21:15:53.365 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 447217664 bytes via buffered write, may repeat 2024-05-22T21:15:53.365 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 448266240 bytes via buffered write, may repeat 2024-05-22T21:15:53.365 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 449314816 bytes via buffered write, may repeat 2024-05-22T21:15:53.365 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 450363392 bytes via buffered write, may repeat 2024-05-22T21:15:53.365 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 451411968 bytes via buffered write, may repeat 2024-05-22T21:15:53.365 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 452460544 bytes via buffered write, may repeat 2024-05-22T21:15:53.365 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 453509120 bytes via buffered write, may repeat 2024-05-22T21:15:53.365 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 454557696 bytes via buffered write, may repeat 2024-05-22T21:15:53.366 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 455606272 bytes via buffered write, may repeat 2024-05-22T21:15:53.366 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 456654848 bytes via buffered write, may repeat 2024-05-22T21:15:53.366 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 457703424 bytes via buffered write, may repeat 2024-05-22T21:15:53.366 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 458752000 bytes via buffered write, may repeat 2024-05-22T21:15:53.366 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 459800576 bytes via buffered write, may repeat 2024-05-22T21:15:53.366 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 460849152 bytes via buffered write, may repeat 2024-05-22T21:15:53.366 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 461897728 bytes via buffered write, may repeat 2024-05-22T21:15:53.366 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 462946304 bytes via buffered write, may repeat 2024-05-22T21:15:53.366 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 463994880 bytes via buffered write, may repeat 2024-05-22T21:15:53.366 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 465043456 bytes via buffered write, may repeat 2024-05-22T21:15:53.366 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 466092032 bytes via buffered write, may repeat 2024-05-22T21:15:53.366 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 467140608 bytes via buffered write, may repeat 2024-05-22T21:15:53.366 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 468189184 bytes via buffered write, may repeat 2024-05-22T21:15:53.367 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 469237760 bytes via buffered write, may repeat 2024-05-22T21:15:53.367 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 470286336 bytes via buffered write, may repeat 2024-05-22T21:15:53.367 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 471334912 bytes via buffered write, may repeat 2024-05-22T21:15:53.367 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 472383488 bytes via buffered write, may repeat 2024-05-22T21:15:53.367 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 473432064 bytes via buffered write, may repeat 2024-05-22T21:15:53.367 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 474480640 bytes via buffered write, may repeat 2024-05-22T21:15:53.367 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 475529216 bytes via buffered write, may repeat 2024-05-22T21:15:53.367 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 476577792 bytes via buffered write, may repeat 2024-05-22T21:15:53.367 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 477626368 bytes via buffered write, may repeat 2024-05-22T21:15:53.367 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 478674944 bytes via buffered write, may repeat 2024-05-22T21:15:53.367 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 479723520 bytes via buffered write, may repeat 2024-05-22T21:15:53.367 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 480772096 bytes via buffered write, may repeat 2024-05-22T21:15:53.367 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 481820672 bytes via buffered write, may repeat 2024-05-22T21:15:53.367 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 482869248 bytes via buffered write, may repeat 2024-05-22T21:15:53.368 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 483917824 bytes via buffered write, may repeat 2024-05-22T21:15:53.368 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 484966400 bytes via buffered write, may repeat 2024-05-22T21:15:56.480 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:15:56.477+0000 7f7d35a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-05-22T21:16:05.218 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:16:05.213+0000 7f7d35a00640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-05-22T21:16:23.560 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:16:23.565 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:16:53.622 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:16:53.626 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:16:55.609 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 486014976 bytes via buffered write, may repeat 2024-05-22T21:16:55.609 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 487063552 bytes via buffered write, may repeat 2024-05-22T21:16:55.609 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 488112128 bytes via buffered write, may repeat 2024-05-22T21:16:55.609 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 489160704 bytes via buffered write, may repeat 2024-05-22T21:16:55.609 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 490209280 bytes via buffered write, may repeat 2024-05-22T21:16:55.609 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 491257856 bytes via buffered write, may repeat 2024-05-22T21:16:55.609 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 492306432 bytes via buffered write, may repeat 2024-05-22T21:16:55.609 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 493355008 bytes via buffered write, may repeat 2024-05-22T21:16:55.609 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 494403584 bytes via buffered write, may repeat 2024-05-22T21:16:55.609 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 495452160 bytes via buffered write, may repeat 2024-05-22T21:16:55.609 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 496500736 bytes via buffered write, may repeat 2024-05-22T21:16:55.610 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 497549312 bytes via buffered write, may repeat 2024-05-22T21:16:55.610 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 498597888 bytes via buffered write, may repeat 2024-05-22T21:16:55.610 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 499646464 bytes via buffered write, may repeat 2024-05-22T21:16:55.610 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 500695040 bytes via buffered write, may repeat 2024-05-22T21:16:55.610 INFO:teuthology.orchestra.run.smithi161.stdout:done writing 500695040 bytes 2024-05-22T21:16:55.610 INFO:teuthology.orchestra.run.smithi161.stdout:successfully fsync'ed prior to getting full state reported 2024-05-22T21:16:55.610 INFO:teuthology.orchestra.run.smithi161.stdout:starting buffered write 2024-05-22T21:16:55.610 INFO:teuthology.orchestra.run.smithi161.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-05-22T21:16:55.610 INFO:teuthology.orchestra.run.smithi161.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-05-22T21:16:55.610 INFO:teuthology.orchestra.run.smithi161.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2024-05-22T21:16:55.757 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:16:55.758 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:16:55.816 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:16:55.893 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:16:55.893 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:16:55.893 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:16:55.902 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:16:55.902 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:16:55.953 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:16:56.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.132+0000 7f9a6ec00640 1 Processor -- start 2024-05-22T21:16:56.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.132+0000 7f9a6ec00640 1 -- start start 2024-05-22T21:16:56.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a6ec00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9a68143760 0x7f9a68070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a6ec00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a68143b40 con 0x7f9a6806bfc0 2024-05-22T21:16:56.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a6ec00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a6806c4e0 con 0x7f9a6815e640 2024-05-22T21:16:56.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a6ec00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a6806c7c0 con 0x7f9a68143760 2024-05-22T21:16:56.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a6e200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f9a6806bfc0 0x7f9a6815a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58544/0 (socket says 172.21.15.161:58544) 2024-05-22T21:16:56.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a6e200640 1 -- 172.21.15.161:0/1707603764 learned_addr learned my addr 172.21.15.161:0/1707603764 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:16:56.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a6d800640 1 --2- 172.21.15.161:0/1707603764 >> v2:172.21.15.179:3300/0 conn(0x7f9a68143760 0x7f9a68070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a67e00640 1 -- 172.21.15.161:0/1707603764 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986362593 0 0) 0x7f9a68143b40 con 0x7f9a6806bfc0 2024-05-22T21:16:56.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a67e00640 1 -- 172.21.15.161:0/1707603764 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a50003530 con 0x7f9a6806bfc0 2024-05-22T21:16:56.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a67e00640 1 -- 172.21.15.161:0/1707603764 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3885255980 0 0) 0x7f9a50003530 con 0x7f9a6806bfc0 2024-05-22T21:16:56.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a67e00640 1 -- 172.21.15.161:0/1707603764 >> v2:172.21.15.179:3300/0 conn(0x7f9a68143760 msgr2=0x7f9a68070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:56.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a67e00640 1 --2- 172.21.15.161:0/1707603764 >> v2:172.21.15.179:3300/0 conn(0x7f9a68143760 0x7f9a68070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a67e00640 1 -- 172.21.15.161:0/1707603764 >> v1:172.21.15.179:6789/0 conn(0x7f9a6815e640 legacy=0x7f9a6806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a67e00640 1 -- 172.21.15.161:0/1707603764 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6815db60 con 0x7f9a6806bfc0 2024-05-22T21:16:56.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a67e00640 1 -- 172.21.15.161:0/1707603764 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f9a60001680 con 0x7f9a6806bfc0 2024-05-22T21:16:56.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a67e00640 1 -- 172.21.15.161:0/1707603764 >> v1:172.21.15.161:6789/0 conn(0x7f9a6806bfc0 legacy=0x7f9a6815a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a67e00640 1 --2- 172.21.15.161:0/1707603764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a50003980 0x7f9a50003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a67e00640 1 --2- 172.21.15.161:0/1707603764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a500047d0 0x7f9a50004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a67e00640 1 --2- 172.21.15.161:0/1707603764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a50005190 0x7f9a5000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a67e00640 1 -- 172.21.15.161:0/1707603764 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6806c7c0 con 0x7f9a50005190 2024-05-22T21:16:56.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a67e00640 1 -- 172.21.15.161:0/1707603764 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5000c030 con 0x7f9a50003980 2024-05-22T21:16:56.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a67e00640 1 -- 172.21.15.161:0/1707603764 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5000e400 con 0x7f9a500047d0 2024-05-22T21:16:56.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a6e200640 1 --2- 172.21.15.161:0/1707603764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a50005190 0x7f9a5000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a6d800640 1 --2- 172.21.15.161:0/1707603764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a50003980 0x7f9a50003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a6ce00640 1 --2- 172.21.15.161:0/1707603764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a500047d0 0x7f9a50004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a6e200640 1 -- 172.21.15.161:0/1707603764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a500047d0 msgr2=0x7f9a50004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:56.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a6e200640 1 --2- 172.21.15.161:0/1707603764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a500047d0 0x7f9a50004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a6e200640 1 -- 172.21.15.161:0/1707603764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a50003980 msgr2=0x7f9a50003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:56.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a6e200640 1 --2- 172.21.15.161:0/1707603764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a50003980 0x7f9a50003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.136+0000 7f9a6e200640 1 -- 172.21.15.161:0/1707603764 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a50003530 con 0x7f9a50005190 2024-05-22T21:16:56.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.140+0000 7f9a6e200640 1 -- 172.21.15.161:0/1707603764 reap_dead start 2024-05-22T21:16:56.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.140+0000 7f9a6e200640 1 --2- 172.21.15.161:0/1707603764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a50005190 0x7f9a5000b990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000c210 tx=0x7f9a6001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcaa687cfcf2f4e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:56.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.140+0000 7f9a67e00640 1 -- 172.21.15.161:0/1707603764 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a600a5b40 con 0x7f9a50005190 2024-05-22T21:16:56.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.140+0000 7f9a67e00640 1 -- 172.21.15.161:0/1707603764 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a600a5ca0 con 0x7f9a50005190 2024-05-22T21:16:56.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.140+0000 7f9a67e00640 1 -- 172.21.15.161:0/1707603764 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60005b20 con 0x7f9a50005190 2024-05-22T21:16:56.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.140+0000 7f9a6ec00640 1 -- 172.21.15.161:0/1707603764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a50005190 msgr2=0x7f9a5000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.140+0000 7f9a6ec00640 1 --2- 172.21.15.161:0/1707603764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a50005190 0x7f9a5000b990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000c210 tx=0x7f9a6001cc40 comp rx=0 tx=0).stop 2024-05-22T21:16:56.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.140+0000 7f9a6ec00640 1 -- 172.21.15.161:0/1707603764 shutdown_connections 2024-05-22T21:16:56.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.140+0000 7f9a6ec00640 1 --2- 172.21.15.161:0/1707603764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a50005190 0x7f9a5000b990 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.140+0000 7f9a6ec00640 1 -- 172.21.15.161:0/1707603764 >> 172.21.15.161:0/1707603764 conn(0x7f9a6806a4d0 msgr2=0x7f9a680b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:56.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.140+0000 7f9a6ec00640 1 -- 172.21.15.161:0/1707603764 shutdown_connections 2024-05-22T21:16:56.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.140+0000 7f9a6ec00640 1 -- 172.21.15.161:0/1707603764 wait complete. 2024-05-22T21:16:56.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.140+0000 7f9a6ec00640 1 Processor -- start 2024-05-22T21:16:56.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.140+0000 7f9a6ec00640 1 -- start start 2024-05-22T21:16:56.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6ec00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a50005190 0x7f9a680a8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6ec00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a680a9cb0 0x7f9a6813fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6ec00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a68140320 0x7f9a6815bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6ec00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a68140120 con 0x7f9a50005190 2024-05-22T21:16:56.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6ec00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a680a8c10 con 0x7f9a68140320 2024-05-22T21:16:56.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6ec00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a680a8ef0 con 0x7f9a680a9cb0 2024-05-22T21:16:56.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6d800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a50005190 0x7f9a680a8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6d800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a50005190 0x7f9a680a8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53524/0 (socket says 172.21.15.161:53524) 2024-05-22T21:16:56.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6d800640 1 -- 172.21.15.161:0/4272277477 learned_addr learned my addr 172.21.15.161:0/4272277477 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:16:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6ce00640 1 --2- 172.21.15.161:0/4272277477 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a680a9cb0 0x7f9a6813fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6e200640 1 --2- 172.21.15.161:0/4272277477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a68140320 0x7f9a6815bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6d800640 1 -- 172.21.15.161:0/4272277477 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a680a9cb0 msgr2=0x7f9a6813fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6d800640 1 --2- 172.21.15.161:0/4272277477 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a680a9cb0 0x7f9a6813fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6d800640 1 -- 172.21.15.161:0/4272277477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a68140320 msgr2=0x7f9a6815bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6d800640 1 --2- 172.21.15.161:0/4272277477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a68140320 0x7f9a6815bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6d800640 1 -- 172.21.15.161:0/4272277477 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a680a91d0 con 0x7f9a50005190 2024-05-22T21:16:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6d800640 1 --2- 172.21.15.161:0/4272277477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a50005190 0x7f9a680a8630 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009f90 tx=0x7f9a5c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a65600640 1 -- 172.21.15.161:0/4272277477 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c00e040 con 0x7f9a50005190 2024-05-22T21:16:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a65600640 1 -- 172.21.15.161:0/4272277477 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5c00f040 con 0x7f9a50005190 2024-05-22T21:16:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a65600640 1 -- 172.21.15.161:0/4272277477 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c015870 con 0x7f9a50005190 2024-05-22T21:16:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6ec00640 1 -- 172.21.15.161:0/4272277477 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a680a94b0 con 0x7f9a50005190 2024-05-22T21:16:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6ec00640 1 -- 172.21.15.161:0/4272277477 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a68140a20 con 0x7f9a50005190 2024-05-22T21:16:56.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a6ec00640 1 -- 172.21.15.161:0/4272277477 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a68067070 con 0x7f9a50005190 2024-05-22T21:16:56.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.144+0000 7f9a65600640 1 -- 172.21.15.161:0/4272277477 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9a5c0159d0 con 0x7f9a50005190 2024-05-22T21:16:56.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.148+0000 7f9a65600640 1 --2- 172.21.15.161:0/4272277477 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a480666c0 0x7f9a48068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.148+0000 7f9a65600640 1 -- 172.21.15.161:0/4272277477 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a5c084d70 con 0x7f9a50005190 2024-05-22T21:16:56.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.148+0000 7f9a6ce00640 1 --2- 172.21.15.161:0/4272277477 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a480666c0 0x7f9a48068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.148+0000 7f9a65600640 1 -- 172.21.15.161:0/4272277477 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f9a4806e1f0 con 0x7f9a50005190 2024-05-22T21:16:56.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.148+0000 7f9a6ce00640 1 --2- 172.21.15.161:0/4272277477 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a480666c0 0x7f9a48068b80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9a680a85c0 tx=0x7f9a54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:56.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.152+0000 7f9a65600640 1 -- 172.21.15.161:0/4272277477 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9a5c052180 con 0x7f9a50005190 2024-05-22T21:16:56.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.308+0000 7f9a6ec00640 1 -- 172.21.15.161:0/4272277477 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a6815cff0 con 0x7f9a50005190 2024-05-22T21:16:56.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.308+0000 7f9a65600640 1 -- 172.21.15.161:0/4272277477 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16203 (secure 0 0 0) 0x7f9a5c022030 con 0x7f9a50005190 2024-05-22T21:16:56.312 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:16:56.312 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":69,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:15:57.531374+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:15:40.613692+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-22T21:15:41.954314+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:16:56.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.312+0000 7f9a6ec00640 1 -- 172.21.15.161:0/4272277477 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a480666c0 msgr2=0x7f9a48068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.312+0000 7f9a6ec00640 1 --2- 172.21.15.161:0/4272277477 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a480666c0 0x7f9a48068b80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9a680a85c0 tx=0x7f9a54009040 comp rx=0 tx=0).stop 2024-05-22T21:16:56.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.312+0000 7f9a6ec00640 1 -- 172.21.15.161:0/4272277477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a50005190 msgr2=0x7f9a680a8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.312+0000 7f9a6ec00640 1 --2- 172.21.15.161:0/4272277477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a50005190 0x7f9a680a8630 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009f90 tx=0x7f9a5c00f580 comp rx=0 tx=0).stop 2024-05-22T21:16:56.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.312+0000 7f9a6ec00640 1 -- 172.21.15.161:0/4272277477 shutdown_connections 2024-05-22T21:16:56.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.312+0000 7f9a6ec00640 1 --2- 172.21.15.161:0/4272277477 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9a480666c0 0x7f9a48068b80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.312+0000 7f9a6ec00640 1 --2- 172.21.15.161:0/4272277477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a68140320 0x7f9a6815bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.312+0000 7f9a6ec00640 1 --2- 172.21.15.161:0/4272277477 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a680a9cb0 0x7f9a6813fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.312+0000 7f9a6ec00640 1 --2- 172.21.15.161:0/4272277477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a50005190 0x7f9a680a8630 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.312+0000 7f9a6ec00640 1 -- 172.21.15.161:0/4272277477 >> 172.21.15.161:0/4272277477 conn(0x7f9a6806a4d0 msgr2=0x7f9a6815b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:56.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.312+0000 7f9a6ec00640 1 -- 172.21.15.161:0/4272277477 shutdown_connections 2024-05-22T21:16:56.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.312+0000 7f9a6ec00640 1 -- 172.21.15.161:0/4272277477 wait complete. 2024-05-22T21:16:56.333 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:15:40.613692+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:15:41.954314+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:16:56.334 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:16:56.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.468+0000 7fcfbf800640 1 Processor -- start 2024-05-22T21:16:56.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.468+0000 7fcfbf800640 1 -- start start 2024-05-22T21:16:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.472+0000 7fcfbf800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcfb813cfc0 0x7fcfb81402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.472+0000 7fcfbf800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcfb815af60 0x7fcfb813f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.472+0000 7fcfbf800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcfb813fc80 0x7fcfb815dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.472+0000 7fcfbf800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb81400b0 con 0x7fcfb815af60 2024-05-22T21:16:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.472+0000 7fcfbf800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb813d3a0 con 0x7fcfb813cfc0 2024-05-22T21:16:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.472+0000 7fcfbf800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb80cd920 con 0x7fcfb813fc80 2024-05-22T21:16:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.472+0000 7fcfafe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcfb815af60 0x7fcfb813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.472+0000 7fcfafe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcfb815af60 0x7fcfb813f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53538/0 (socket says 172.21.15.161:53538) 2024-05-22T21:16:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.472+0000 7fcfafe00640 1 -- 172.21.15.161:0/1051241746 learned_addr learned my addr 172.21.15.161:0/1051241746 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:16:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.472+0000 7fcfbce00640 1 --2- 172.21.15.161:0/1051241746 >> v2:172.21.15.179:3300/0 conn(0x7fcfb813cfc0 0x7fcfb81402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.472+0000 7fcfbd800640 1 --2- 172.21.15.161:0/1051241746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcfb813fc80 0x7fcfb815dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.472+0000 7fcfafe00640 1 -- 172.21.15.161:0/1051241746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcfb813fc80 msgr2=0x7fcfb815dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.472+0000 7fcfafe00640 1 --2- 172.21.15.161:0/1051241746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcfb813fc80 0x7fcfb815dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.472+0000 7fcfafe00640 1 -- 172.21.15.161:0/1051241746 >> v2:172.21.15.179:3300/0 conn(0x7fcfb813cfc0 msgr2=0x7fcfb81402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.472+0000 7fcfafe00640 1 --2- 172.21.15.161:0/1051241746 >> v2:172.21.15.179:3300/0 conn(0x7fcfb813cfc0 0x7fcfb81402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.472+0000 7fcfafe00640 1 -- 172.21.15.161:0/1051241746 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb80cdc00 con 0x7fcfb815af60 2024-05-22T21:16:56.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.476+0000 7fcfafe00640 1 --2- 172.21.15.161:0/1051241746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb815af60 0x7fcfb813f740 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fcfb806de90 tx=0x7fcfa001e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa158ede329419b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:56.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.476+0000 7fcfb7e00640 1 -- 172.21.15.161:0/1051241746 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00ad600 con 0x7fcfb815af60 2024-05-22T21:16:56.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.476+0000 7fcfb7e00640 1 -- 172.21.15.161:0/1051241746 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa00a9050 con 0x7fcfb815af60 2024-05-22T21:16:56.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.476+0000 7fcfb7e00640 1 -- 172.21.15.161:0/1051241746 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00ad980 con 0x7fcfb815af60 2024-05-22T21:16:56.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.476+0000 7fcfbf800640 1 -- 172.21.15.161:0/1051241746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb815af60 msgr2=0x7fcfb813f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.476+0000 7fcfbf800640 1 --2- 172.21.15.161:0/1051241746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb815af60 0x7fcfb813f740 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fcfb806de90 tx=0x7fcfa001e3b0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.476+0000 7fcfbf800640 1 -- 172.21.15.161:0/1051241746 shutdown_connections 2024-05-22T21:16:56.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.476+0000 7fcfbf800640 1 --2- 172.21.15.161:0/1051241746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcfb813fc80 0x7fcfb815dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.476+0000 7fcfbf800640 1 --2- 172.21.15.161:0/1051241746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb815af60 0x7fcfb813f740 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.476+0000 7fcfbf800640 1 --2- 172.21.15.161:0/1051241746 >> v2:172.21.15.179:3300/0 conn(0x7fcfb813cfc0 0x7fcfb81402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.476+0000 7fcfbf800640 1 -- 172.21.15.161:0/1051241746 >> 172.21.15.161:0/1051241746 conn(0x7fcfb806a2d0 msgr2=0x7fcfb806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:56.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.476+0000 7fcfbf800640 1 -- 172.21.15.161:0/1051241746 shutdown_connections 2024-05-22T21:16:56.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.476+0000 7fcfbf800640 1 -- 172.21.15.161:0/1051241746 wait complete. 2024-05-22T21:16:56.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.476+0000 7fcfbf800640 1 Processor -- start 2024-05-22T21:16:56.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.476+0000 7fcfbf800640 1 -- start start 2024-05-22T21:16:56.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbf800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcfb813cfc0 0x7fcfb8140250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbf800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcfb813fc80 0x7fcfb8154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbf800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb815af60 0x7fcfb80b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbf800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb8138480 con 0x7fcfb815af60 2024-05-22T21:16:56.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbf800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb8138d20 con 0x7fcfb813fc80 2024-05-22T21:16:56.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbf800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb81420d0 con 0x7fcfb813cfc0 2024-05-22T21:16:56.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbd800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb815af60 0x7fcfb80b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbce00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcfb813cfc0 0x7fcfb8140250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbd800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb815af60 0x7fcfb80b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53548/0 (socket says 172.21.15.161:53548) 2024-05-22T21:16:56.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbd800640 1 -- 172.21.15.161:0/2728299779 learned_addr learned my addr 172.21.15.161:0/2728299779 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:16:56.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfafe00640 1 --2- 172.21.15.161:0/2728299779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcfb813fc80 0x7fcfb8154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbd800640 1 -- 172.21.15.161:0/2728299779 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcfb813cfc0 msgr2=0x7fcfb8140250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbd800640 1 --2- 172.21.15.161:0/2728299779 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcfb813cfc0 0x7fcfb8140250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbd800640 1 -- 172.21.15.161:0/2728299779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcfb813fc80 msgr2=0x7fcfb8154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbd800640 1 --2- 172.21.15.161:0/2728299779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcfb813fc80 0x7fcfb8154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbd800640 1 -- 172.21.15.161:0/2728299779 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa00094d0 con 0x7fcfb815af60 2024-05-22T21:16:56.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbd800640 1 --2- 172.21.15.161:0/2728299779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb815af60 0x7fcfb80b32f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0007e90 tx=0x7fcfb000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:56.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfb5600640 1 -- 172.21.15.161:0/2728299779 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0014420 con 0x7fcfb815af60 2024-05-22T21:16:56.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfb5600640 1 -- 172.21.15.161:0/2728299779 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb0014580 con 0x7fcfb815af60 2024-05-22T21:16:56.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbf800640 1 -- 172.21.15.161:0/2728299779 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb81386d0 con 0x7fcfb815af60 2024-05-22T21:16:56.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfb5600640 1 -- 172.21.15.161:0/2728299779 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0014930 con 0x7fcfb815af60 2024-05-22T21:16:56.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbf800640 1 -- 172.21.15.161:0/2728299779 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb8155070 con 0x7fcfb815af60 2024-05-22T21:16:56.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfbf800640 1 -- 172.21.15.161:0/2728299779 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb8067220 con 0x7fcfb815af60 2024-05-22T21:16:56.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.480+0000 7fcfb5600640 1 -- 172.21.15.161:0/2728299779 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcfb0014a90 con 0x7fcfb815af60 2024-05-22T21:16:56.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.484+0000 7fcfb5600640 1 --2- 172.21.15.161:0/2728299779 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcf94066690 0x7fcf94068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.484+0000 7fcfb5600640 1 -- 172.21.15.161:0/2728299779 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfb00849f0 con 0x7fcfb815af60 2024-05-22T21:16:56.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.484+0000 7fcfb5600640 1 -- 172.21.15.161:0/2728299779 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7fcf9406e1c0 con 0x7fcfb815af60 2024-05-22T21:16:56.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.484+0000 7fcfbce00640 1 --2- 172.21.15.161:0/2728299779 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcf94066690 0x7fcf94068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.484+0000 7fcfbce00640 1 --2- 172.21.15.161:0/2728299779 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcf94066690 0x7fcf94068b50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80079c0 tx=0x7fcfa8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:56.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.488+0000 7fcfb5600640 1 -- 172.21.15.161:0/2728299779 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcfb0051ce0 con 0x7fcfb815af60 2024-05-22T21:16:56.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.640+0000 7fcfbf800640 1 -- 172.21.15.161:0/2728299779 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcfb8138e60 con 0x7fcfb815af60 2024-05-22T21:16:56.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.640+0000 7fcfb5600640 1 -- 172.21.15.161:0/2728299779 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcfb0055300 con 0x7fcfb815af60 2024-05-22T21:16:56.645 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:16:56.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.644+0000 7fcfbf800640 1 -- 172.21.15.161:0/2728299779 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcf94066690 msgr2=0x7fcf94068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.644+0000 7fcfbf800640 1 --2- 172.21.15.161:0/2728299779 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcf94066690 0x7fcf94068b50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80079c0 tx=0x7fcfa8008040 comp rx=0 tx=0).stop 2024-05-22T21:16:56.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.644+0000 7fcfbf800640 1 -- 172.21.15.161:0/2728299779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb815af60 msgr2=0x7fcfb80b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.644+0000 7fcfbf800640 1 --2- 172.21.15.161:0/2728299779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb815af60 0x7fcfb80b32f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0007e90 tx=0x7fcfb000e4b0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.644+0000 7fcfbf800640 1 -- 172.21.15.161:0/2728299779 shutdown_connections 2024-05-22T21:16:56.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.644+0000 7fcfbf800640 1 --2- 172.21.15.161:0/2728299779 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcf94066690 0x7fcf94068b50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.644+0000 7fcfbf800640 1 --2- 172.21.15.161:0/2728299779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb815af60 0x7fcfb80b32f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.644+0000 7fcfbf800640 1 --2- 172.21.15.161:0/2728299779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcfb813fc80 0x7fcfb8154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.644+0000 7fcfbf800640 1 --2- 172.21.15.161:0/2728299779 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcfb813cfc0 0x7fcfb8140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.644+0000 7fcfbf800640 1 -- 172.21.15.161:0/2728299779 >> 172.21.15.161:0/2728299779 conn(0x7fcfb806a2d0 msgr2=0x7fcfb806a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:56.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.644+0000 7fcfbf800640 1 -- 172.21.15.161:0/2728299779 shutdown_connections 2024-05-22T21:16:56.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.644+0000 7fcfbf800640 1 -- 172.21.15.161:0/2728299779 wait complete. 2024-05-22T21:16:56.666 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:16:56.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8079400640 1 Processor -- start 2024-05-22T21:16:56.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8079400640 1 -- start start 2024-05-22T21:16:56.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8079400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f80741402e0 0x7f807413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8079400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f807413fc80 0x7f807406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8079400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80741400b0 con 0x7f807413cfc0 2024-05-22T21:16:56.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8079400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807413d3a0 con 0x7f80741402e0 2024-05-22T21:16:56.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8079400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80740cd920 con 0x7f807413fc80 2024-05-22T21:16:56.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8072800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f807413cfc0 0x7f807415af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58554/0 (socket says 172.21.15.161:58554) 2024-05-22T21:16:56.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8072800640 1 -- 172.21.15.161:0/3698712224 learned_addr learned my addr 172.21.15.161:0/3698712224 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:16:56.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8071e00640 1 --2- 172.21.15.161:0/3698712224 >> v2:172.21.15.179:3300/0 conn(0x7f80741402e0 0x7f807413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8073200640 1 --2- 172.21.15.161:0/3698712224 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f807413fc80 0x7f807406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8071400640 1 -- 172.21.15.161:0/3698712224 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916713528 0 0) 0x7f80741400b0 con 0x7f807413cfc0 2024-05-22T21:16:56.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8071400640 1 -- 172.21.15.161:0/3698712224 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8058003530 con 0x7f807413cfc0 2024-05-22T21:16:56.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8073200640 1 -- 172.21.15.161:0/3698712224 >> v2:172.21.15.179:3300/0 conn(0x7f80741402e0 msgr2=0x7f807413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:56.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8073200640 1 --2- 172.21.15.161:0/3698712224 >> v2:172.21.15.179:3300/0 conn(0x7f80741402e0 0x7f807413f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8073200640 1 -- 172.21.15.161:0/3698712224 >> v1:172.21.15.161:6789/0 conn(0x7f807413cfc0 legacy=0x7f807415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8071400640 1 -- 172.21.15.161:0/3698712224 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2519844301 0 0) 0x7f8058003530 con 0x7f807413cfc0 2024-05-22T21:16:56.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8073200640 1 -- 172.21.15.161:0/3698712224 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80740cdc00 con 0x7f807413fc80 2024-05-22T21:16:56.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.800+0000 7f8071e00640 1 --2- 172.21.15.161:0/3698712224 >> v2:172.21.15.179:3300/0 conn(0x7f80741402e0 0x7f807413f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:16:56.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.804+0000 7f8073200640 1 --2- 172.21.15.161:0/3698712224 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f807413fc80 0x7f807406fce0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f806c0097c0 tx=0x7f806c01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6a89b8a3457f027 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:56.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.804+0000 7f8071400640 1 -- 172.21.15.161:0/3698712224 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c0ac440 con 0x7f807413fc80 2024-05-22T21:16:56.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.804+0000 7f8071400640 1 -- 172.21.15.161:0/3698712224 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806c0ac5a0 con 0x7f807413fc80 2024-05-22T21:16:56.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.804+0000 7f8071400640 1 -- 172.21.15.161:0/3698712224 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c0ac930 con 0x7f807413fc80 2024-05-22T21:16:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.804+0000 7f8079400640 1 -- 172.21.15.161:0/3698712224 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f807413fc80 msgr2=0x7f807406fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.804+0000 7f8079400640 1 --2- 172.21.15.161:0/3698712224 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f807413fc80 0x7f807406fce0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f806c0097c0 tx=0x7f806c01c3e0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.804+0000 7f8079400640 1 -- 172.21.15.161:0/3698712224 shutdown_connections 2024-05-22T21:16:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.804+0000 7f8079400640 1 --2- 172.21.15.161:0/3698712224 >> v2:172.21.15.179:3300/0 conn(0x7f80741402e0 0x7f807413f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.804+0000 7f8079400640 1 --2- 172.21.15.161:0/3698712224 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f807413fc80 0x7f807406fce0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.804+0000 7f8079400640 1 -- 172.21.15.161:0/3698712224 >> 172.21.15.161:0/3698712224 conn(0x7f807406a2d0 msgr2=0x7f807406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:56.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.804+0000 7f8079400640 1 -- 172.21.15.161:0/3698712224 shutdown_connections 2024-05-22T21:16:56.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.804+0000 7f8079400640 1 -- 172.21.15.161:0/3698712224 wait complete. 2024-05-22T21:16:56.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8079400640 1 Processor -- start 2024-05-22T21:16:56.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8079400640 1 -- start start 2024-05-22T21:16:56.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8079400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f807413cfc0 0x7f807415ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8079400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f807413fc80 0x7f807413b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8079400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80741402e0 0x7f807415c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8079400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80741424a0 con 0x7f807413cfc0 2024-05-22T21:16:56.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8079400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807413bdf0 con 0x7f807413fc80 2024-05-22T21:16:56.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8079400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80741586c0 con 0x7f80741402e0 2024-05-22T21:16:56.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8072800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f807413cfc0 0x7f807415ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8072800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f807413cfc0 0x7f807415ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53556/0 (socket says 172.21.15.161:53556) 2024-05-22T21:16:56.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8072800640 1 -- 172.21.15.161:0/1358375232 learned_addr learned my addr 172.21.15.161:0/1358375232 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:16:56.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8073200640 1 --2- 172.21.15.161:0/1358375232 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80741402e0 0x7f807415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8071e00640 1 --2- 172.21.15.161:0/1358375232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f807413fc80 0x7f807413b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8072800640 1 -- 172.21.15.161:0/1358375232 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80741402e0 msgr2=0x7f807415c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8072800640 1 --2- 172.21.15.161:0/1358375232 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80741402e0 0x7f807415c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8072800640 1 -- 172.21.15.161:0/1358375232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f807413fc80 msgr2=0x7f807413b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8072800640 1 --2- 172.21.15.161:0/1358375232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f807413fc80 0x7f807413b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8072800640 1 -- 172.21.15.161:0/1358375232 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8060009420 con 0x7f807413cfc0 2024-05-22T21:16:56.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8071e00640 1 --2- 172.21.15.161:0/1358375232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f807413fc80 0x7f807413b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:16:56.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8072800640 1 --2- 172.21.15.161:0/1358375232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f807413cfc0 0x7f807415ad30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f806400cc40 tx=0x7f806400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:56.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f806aa00640 1 -- 172.21.15.161:0/1358375232 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806400ad40 con 0x7f807413cfc0 2024-05-22T21:16:56.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f806aa00640 1 -- 172.21.15.161:0/1358375232 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8064012400 con 0x7f807413cfc0 2024-05-22T21:16:56.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f806aa00640 1 -- 172.21.15.161:0/1358375232 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806400aea0 con 0x7f807413cfc0 2024-05-22T21:16:56.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8079400640 1 -- 172.21.15.161:0/1358375232 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f806c009420 con 0x7f807413cfc0 2024-05-22T21:16:56.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.808+0000 7f8079400640 1 -- 172.21.15.161:0/1358375232 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8074158c80 con 0x7f807413cfc0 2024-05-22T21:16:56.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.812+0000 7f8079400640 1 -- 172.21.15.161:0/1358375232 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8074067220 con 0x7f807413cfc0 2024-05-22T21:16:56.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.812+0000 7f806aa00640 1 -- 172.21.15.161:0/1358375232 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8064012790 con 0x7f807413cfc0 2024-05-22T21:16:56.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.812+0000 7f806aa00640 1 --2- 172.21.15.161:0/1358375232 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8054066690 0x7f8054068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:56.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.812+0000 7f806aa00640 1 -- 172.21.15.161:0/1358375232 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8064084890 con 0x7f807413cfc0 2024-05-22T21:16:56.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.812+0000 7f806aa00640 1 -- 172.21.15.161:0/1358375232 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f805406e1c0 con 0x7f807413cfc0 2024-05-22T21:16:56.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.812+0000 7f8071e00640 1 --2- 172.21.15.161:0/1358375232 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8054066690 0x7f8054068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:56.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.812+0000 7f8071e00640 1 --2- 172.21.15.161:0/1358375232 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8054066690 0x7f8054068b50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f807413e7c0 tx=0x7f8060008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:56.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.816+0000 7f806aa00640 1 -- 172.21.15.161:0/1358375232 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8064051b60 con 0x7f807413cfc0 2024-05-22T21:16:56.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.968+0000 7f8079400640 1 -- 172.21.15.161:0/1358375232 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8074071670 con 0x7f807413cfc0 2024-05-22T21:16:56.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.968+0000 7f806aa00640 1 -- 172.21.15.161:0/1358375232 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8064055180 con 0x7f807413cfc0 2024-05-22T21:16:56.973 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:16:56.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.972+0000 7f8079400640 1 -- 172.21.15.161:0/1358375232 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8054066690 msgr2=0x7f8054068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.972+0000 7f8079400640 1 --2- 172.21.15.161:0/1358375232 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8054066690 0x7f8054068b50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f807413e7c0 tx=0x7f8060008040 comp rx=0 tx=0).stop 2024-05-22T21:16:56.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.972+0000 7f8079400640 1 -- 172.21.15.161:0/1358375232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f807413cfc0 msgr2=0x7f807415ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:56.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.972+0000 7f8079400640 1 --2- 172.21.15.161:0/1358375232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f807413cfc0 0x7f807415ad30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f806400cc40 tx=0x7f806400a430 comp rx=0 tx=0).stop 2024-05-22T21:16:56.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.972+0000 7f8079400640 1 -- 172.21.15.161:0/1358375232 shutdown_connections 2024-05-22T21:16:56.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.972+0000 7f8079400640 1 --2- 172.21.15.161:0/1358375232 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8054066690 0x7f8054068b50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.972+0000 7f8079400640 1 --2- 172.21.15.161:0/1358375232 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80741402e0 0x7f807415c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.972+0000 7f8079400640 1 --2- 172.21.15.161:0/1358375232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f807413fc80 0x7f807413b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.972+0000 7f8079400640 1 --2- 172.21.15.161:0/1358375232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f807413cfc0 0x7f807415ad30 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:56.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.972+0000 7f8079400640 1 -- 172.21.15.161:0/1358375232 >> 172.21.15.161:0/1358375232 conn(0x7f807406a2d0 msgr2=0x7f807406a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:56.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.972+0000 7f8079400640 1 -- 172.21.15.161:0/1358375232 shutdown_connections 2024-05-22T21:16:56.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:56.972+0000 7f8079400640 1 -- 172.21.15.161:0/1358375232 wait complete. 2024-05-22T21:16:56.993 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:16:57.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.128+0000 7ff882000640 1 Processor -- start 2024-05-22T21:16:57.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.128+0000 7ff882000640 1 -- start start 2024-05-22T21:16:57.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.128+0000 7ff882000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff87c13cfc0 0x7ff87c1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.128+0000 7ff882000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff87c13fe50 0x7ff87c06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.128+0000 7ff882000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87c13d3a0 con 0x7ff87c15af60 2024-05-22T21:16:57.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.128+0000 7ff882000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff87c0cd920 con 0x7ff87c13cfc0 2024-05-22T21:16:57.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.128+0000 7ff882000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0cdc00 con 0x7ff87c13fe50 2024-05-22T21:16:57.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.128+0000 7ff872a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff87c15af60 0x7ff87c13f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58568/0 (socket says 172.21.15.161:58568) 2024-05-22T21:16:57.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.128+0000 7ff872a00640 1 -- 172.21.15.161:0/700008780 learned_addr learned my addr 172.21.15.161:0/700008780 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:16:57.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.128+0000 7ff87b400640 1 --2- 172.21.15.161:0/700008780 >> v2:172.21.15.179:3300/0 conn(0x7ff87c13cfc0 0x7ff87c1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.128+0000 7ff87be00640 1 --2- 172.21.15.161:0/700008780 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff87c13fe50 0x7ff87c06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.128+0000 7ff87aa00640 1 -- 172.21.15.161:0/700008780 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056882337 0 0) 0x7ff87c13d3a0 con 0x7ff87c15af60 2024-05-22T21:16:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.128+0000 7ff87aa00640 1 -- 172.21.15.161:0/700008780 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff860005740 con 0x7ff87c15af60 2024-05-22T21:16:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 -- 172.21.15.161:0/700008780 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1553680887 0 0) 0x7ff860005740 con 0x7ff87c15af60 2024-05-22T21:16:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 -- 172.21.15.161:0/700008780 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff87c13fe50 msgr2=0x7ff87c06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 --2- 172.21.15.161:0/700008780 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff87c13fe50 0x7ff87c06fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 -- 172.21.15.161:0/700008780 >> v2:172.21.15.179:3300/0 conn(0x7ff87c13cfc0 msgr2=0x7ff87c1402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 --2- 172.21.15.161:0/700008780 >> v2:172.21.15.179:3300/0 conn(0x7ff87c13cfc0 0x7ff87c1402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 -- 172.21.15.161:0/700008780 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c0cdee0 con 0x7ff87c15af60 2024-05-22T21:16:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 -- 172.21.15.161:0/700008780 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7ff864001680 con 0x7ff87c15af60 2024-05-22T21:16:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 -- 172.21.15.161:0/700008780 >> v1:172.21.15.161:6789/0 conn(0x7ff87c15af60 legacy=0x7ff87c13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 --2- 172.21.15.161:0/700008780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff860005b90 0x7ff860005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 --2- 172.21.15.161:0/700008780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8600069e0 0x7ff860006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 --2- 172.21.15.161:0/700008780 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff87c13fe50 0x7ff8600073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 -- 172.21.15.161:0/700008780 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0cd920 con 0x7ff860005b90 2024-05-22T21:16:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 -- 172.21.15.161:0/700008780 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0cdc00 con 0x7ff8600069e0 2024-05-22T21:16:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 -- 172.21.15.161:0/700008780 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86000dbd0 con 0x7ff87c13fe50 2024-05-22T21:16:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87b400640 1 --2- 172.21.15.161:0/700008780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff860005b90 0x7ff860005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff872a00640 1 --2- 172.21.15.161:0/700008780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8600069e0 0x7ff860006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87be00640 1 --2- 172.21.15.161:0/700008780 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff87c13fe50 0x7ff8600073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87b400640 1 -- 172.21.15.161:0/700008780 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff87c13fe50 msgr2=0x7ff8600073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87b400640 1 --2- 172.21.15.161:0/700008780 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff87c13fe50 0x7ff8600073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87b400640 1 -- 172.21.15.161:0/700008780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8600069e0 msgr2=0x7ff860006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87b400640 1 --2- 172.21.15.161:0/700008780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8600069e0 0x7ff860006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87b400640 1 -- 172.21.15.161:0/700008780 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff860005740 con 0x7ff860005b90 2024-05-22T21:16:57.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff872a00640 1 --2- 172.21.15.161:0/700008780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8600069e0 0x7ff860006e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:16:57.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87b400640 1 --2- 172.21.15.161:0/700008780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff860005b90 0x7ff860005fb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009ae0 tx=0x7ff86c01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbfafb40654a46b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:57.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 -- 172.21.15.161:0/700008780 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0a6470 con 0x7ff860005b90 2024-05-22T21:16:57.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 -- 172.21.15.161:0/700008780 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c0a65d0 con 0x7ff860005b90 2024-05-22T21:16:57.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.132+0000 7ff87aa00640 1 -- 172.21.15.161:0/700008780 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0a6980 con 0x7ff860005b90 2024-05-22T21:16:57.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff882000640 1 -- 172.21.15.161:0/700008780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff860005b90 msgr2=0x7ff860005fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff882000640 1 --2- 172.21.15.161:0/700008780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff860005b90 0x7ff860005fb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009ae0 tx=0x7ff86c01ca20 comp rx=0 tx=0).stop 2024-05-22T21:16:57.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff87be00640 1 -- 172.21.15.161:0/700008780 reap_dead start 2024-05-22T21:16:57.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff882000640 1 -- 172.21.15.161:0/700008780 shutdown_connections 2024-05-22T21:16:57.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff882000640 1 -- 172.21.15.161:0/700008780 >> 172.21.15.161:0/700008780 conn(0x7ff87c06a2d0 msgr2=0x7ff87c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:57.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff882000640 1 -- 172.21.15.161:0/700008780 shutdown_connections 2024-05-22T21:16:57.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff882000640 1 -- 172.21.15.161:0/700008780 wait complete. 2024-05-22T21:16:57.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff882000640 1 Processor -- start 2024-05-22T21:16:57.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff882000640 1 -- start start 2024-05-22T21:16:57.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff882000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff860005b90 0x7ff87c1578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff882000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff87c15bf90 0x7ff87c0ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff882000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff87c15c3f0 0x7ff87c0aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff882000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c157e10 con 0x7ff860005b90 2024-05-22T21:16:57.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff882000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c15c950 con 0x7ff87c15c3f0 2024-05-22T21:16:57.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff882000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c15cc30 con 0x7ff87c15bf90 2024-05-22T21:16:57.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff87b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff860005b90 0x7ff87c1578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff87b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff860005b90 0x7ff87c1578d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53582/0 (socket says 172.21.15.161:53582) 2024-05-22T21:16:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff87b400640 1 -- 172.21.15.161:0/4141170850 learned_addr learned my addr 172.21.15.161:0/4141170850 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:16:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff872a00640 1 --2- 172.21.15.161:0/4141170850 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff87c15bf90 0x7ff87c0ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff87be00640 1 --2- 172.21.15.161:0/4141170850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff87c15c3f0 0x7ff87c0aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff87b400640 1 -- 172.21.15.161:0/4141170850 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff87c15bf90 msgr2=0x7ff87c0ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff87b400640 1 --2- 172.21.15.161:0/4141170850 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff87c15bf90 0x7ff87c0ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff87b400640 1 -- 172.21.15.161:0/4141170850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff87c15c3f0 msgr2=0x7ff87c0aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff87b400640 1 --2- 172.21.15.161:0/4141170850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff87c15c3f0 0x7ff87c0aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.136+0000 7ff87b400640 1 -- 172.21.15.161:0/4141170850 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c15cf10 con 0x7ff860005b90 2024-05-22T21:16:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.140+0000 7ff87b400640 1 --2- 172.21.15.161:0/4141170850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff860005b90 0x7ff87c1578d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009ae0 tx=0x7ff860007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.140+0000 7ff873e00640 1 -- 172.21.15.161:0/4141170850 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0a6cc0 con 0x7ff860005b90 2024-05-22T21:16:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.140+0000 7ff873e00640 1 -- 172.21.15.161:0/4141170850 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c0a6e20 con 0x7ff860005b90 2024-05-22T21:16:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.140+0000 7ff882000640 1 -- 172.21.15.161:0/4141170850 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87c15d1f0 con 0x7ff860005b90 2024-05-22T21:16:57.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.140+0000 7ff873e00640 1 -- 172.21.15.161:0/4141170850 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0bc670 con 0x7ff860005b90 2024-05-22T21:16:57.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.140+0000 7ff882000640 1 -- 172.21.15.161:0/4141170850 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87c13b5d0 con 0x7ff860005b90 2024-05-22T21:16:57.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.140+0000 7ff882000640 1 -- 172.21.15.161:0/4141170850 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87c067220 con 0x7ff860005b90 2024-05-22T21:16:57.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.140+0000 7ff873e00640 1 -- 172.21.15.161:0/4141170850 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff86c0a7450 con 0x7ff860005b90 2024-05-22T21:16:57.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.140+0000 7ff873e00640 1 --2- 172.21.15.161:0/4141170850 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8580666c0 0x7ff858068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.140+0000 7ff873e00640 1 -- 172.21.15.161:0/4141170850 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff86c11dea0 con 0x7ff860005b90 2024-05-22T21:16:57.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.140+0000 7ff873e00640 1 -- 172.21.15.161:0/4141170850 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7ff85806e1f0 con 0x7ff860005b90 2024-05-22T21:16:57.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.140+0000 7ff872a00640 1 --2- 172.21.15.161:0/4141170850 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8580666c0 0x7ff858068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.144+0000 7ff872a00640 1 --2- 172.21.15.161:0/4141170850 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8580666c0 0x7ff858068b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff864001ec0 tx=0x7ff864007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:57.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.144+0000 7ff873e00640 1 -- 172.21.15.161:0/4141170850 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff86c0eb190 con 0x7ff860005b90 2024-05-22T21:16:57.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.296+0000 7ff882000640 1 -- 172.21.15.161:0/4141170850 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff87c140230 con 0x7ff860005b90 2024-05-22T21:16:57.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.300+0000 7ff873e00640 1 -- 172.21.15.161:0/4141170850 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff86c0ee7b0 con 0x7ff860005b90 2024-05-22T21:16:57.302 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:16:57.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.300+0000 7ff882000640 1 -- 172.21.15.161:0/4141170850 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8580666c0 msgr2=0x7ff858068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.300+0000 7ff882000640 1 --2- 172.21.15.161:0/4141170850 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8580666c0 0x7ff858068b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff864001ec0 tx=0x7ff864007040 comp rx=0 tx=0).stop 2024-05-22T21:16:57.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.300+0000 7ff882000640 1 -- 172.21.15.161:0/4141170850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff860005b90 msgr2=0x7ff87c1578d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.300+0000 7ff882000640 1 --2- 172.21.15.161:0/4141170850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff860005b90 0x7ff87c1578d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009ae0 tx=0x7ff860007a10 comp rx=0 tx=0).stop 2024-05-22T21:16:57.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.300+0000 7ff882000640 1 -- 172.21.15.161:0/4141170850 shutdown_connections 2024-05-22T21:16:57.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.300+0000 7ff882000640 1 --2- 172.21.15.161:0/4141170850 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8580666c0 0x7ff858068b80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.300+0000 7ff882000640 1 --2- 172.21.15.161:0/4141170850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff87c15c3f0 0x7ff87c0aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.300+0000 7ff882000640 1 --2- 172.21.15.161:0/4141170850 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff87c15bf90 0x7ff87c0ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.300+0000 7ff882000640 1 --2- 172.21.15.161:0/4141170850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff860005b90 0x7ff87c1578d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.300+0000 7ff882000640 1 -- 172.21.15.161:0/4141170850 >> 172.21.15.161:0/4141170850 conn(0x7ff87c06a2d0 msgr2=0x7ff87c06cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:57.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.300+0000 7ff882000640 1 -- 172.21.15.161:0/4141170850 shutdown_connections 2024-05-22T21:16:57.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.300+0000 7ff882000640 1 -- 172.21.15.161:0/4141170850 wait complete. 2024-05-22T21:16:57.323 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:16:57.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.456+0000 7f23cae00640 1 Processor -- start 2024-05-22T21:16:57.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.456+0000 7f23cae00640 1 -- start start 2024-05-22T21:16:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.456+0000 7f23cae00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f23c41402e0 0x7f23c413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.456+0000 7f23cae00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c41400d0 con 0x7f23c413fc80 2024-05-22T21:16:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.456+0000 7f23cae00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c413d3a0 con 0x7f23c413cfc0 2024-05-22T21:16:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.456+0000 7f23cae00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c40cd920 con 0x7f23c41402e0 2024-05-22T21:16:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.456+0000 7f23c8e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f23c413fc80 0x7f23c406fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58580/0 (socket says 172.21.15.161:58580) 2024-05-22T21:16:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.456+0000 7f23c8e00640 1 -- 172.21.15.161:0/3037856200 learned_addr learned my addr 172.21.15.161:0/3037856200 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:16:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.456+0000 7f23bb400640 1 --2- 172.21.15.161:0/3037856200 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f23c41402e0 0x7f23c413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.460+0000 7f23c3400640 1 -- 172.21.15.161:0/3037856200 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680738633 0 0) 0x7f23bc001120 con 0x7f23c413fc80 2024-05-22T21:16:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.460+0000 7f23c3400640 1 -- 172.21.15.161:0/3037856200 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23a8003530 con 0x7f23c413fc80 2024-05-22T21:16:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.460+0000 7f23c3400640 1 -- 172.21.15.161:0/3037856200 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665621288 0 0) 0x7f23c413d3a0 con 0x7f23c413cfc0 2024-05-22T21:16:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.460+0000 7f23c3400640 1 -- 172.21.15.161:0/3037856200 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23bc001120 con 0x7f23c413cfc0 2024-05-22T21:16:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.460+0000 7f23bb400640 1 -- 172.21.15.161:0/3037856200 >> v1:172.21.15.179:6789/0 conn(0x7f23c413cfc0 legacy=0x7f23c415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.460+0000 7f23bb400640 1 -- 172.21.15.161:0/3037856200 >> v1:172.21.15.161:6789/0 conn(0x7f23c413fc80 legacy=0x7f23c406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.460+0000 7f23c3400640 1 -- 172.21.15.161:0/3037856200 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4078456074 0 0) 0x7f23a8003530 con 0x7f23c413fc80 2024-05-22T21:16:57.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.460+0000 7f23bb400640 1 -- 172.21.15.161:0/3037856200 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c40cdc00 con 0x7f23c41402e0 2024-05-22T21:16:57.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.460+0000 7f23bb400640 1 --2- 172.21.15.161:0/3037856200 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f23c41402e0 0x7f23c413f740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0097f0 tx=0x7f23ac01c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b941e12a4347660 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:57.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.460+0000 7f23c3400640 1 -- 172.21.15.161:0/3037856200 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc001810 con 0x7f23c41402e0 2024-05-22T21:16:57.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.460+0000 7f23c3400640 1 -- 172.21.15.161:0/3037856200 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac0a7050 con 0x7f23c41402e0 2024-05-22T21:16:57.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.460+0000 7f23c3400640 1 -- 172.21.15.161:0/3037856200 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0ab400 con 0x7f23c41402e0 2024-05-22T21:16:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23cae00640 1 -- 172.21.15.161:0/3037856200 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f23c41402e0 msgr2=0x7f23c413f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23cae00640 1 --2- 172.21.15.161:0/3037856200 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f23c41402e0 0x7f23c413f740 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0097f0 tx=0x7f23ac01c340 comp rx=0 tx=0).stop 2024-05-22T21:16:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23cae00640 1 -- 172.21.15.161:0/3037856200 shutdown_connections 2024-05-22T21:16:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23cae00640 1 --2- 172.21.15.161:0/3037856200 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f23c41402e0 0x7f23c413f740 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23cae00640 1 -- 172.21.15.161:0/3037856200 >> 172.21.15.161:0/3037856200 conn(0x7f23c406a2d0 msgr2=0x7f23c406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23cae00640 1 -- 172.21.15.161:0/3037856200 shutdown_connections 2024-05-22T21:16:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23cae00640 1 -- 172.21.15.161:0/3037856200 wait complete. 2024-05-22T21:16:57.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23cae00640 1 Processor -- start 2024-05-22T21:16:57.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23cae00640 1 -- start start 2024-05-22T21:16:57.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23cae00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f23c413cfc0 0x7f23c4159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23cae00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23c413fc80 0x7f23c4139840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23cae00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23c41402e0 0x7f23c413b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23cae00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c4157890 con 0x7f23c413fc80 2024-05-22T21:16:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23cae00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c40aa230 con 0x7f23c41402e0 2024-05-22T21:16:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23cae00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c413d810 con 0x7f23c413cfc0 2024-05-22T21:16:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23bb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23c413fc80 0x7f23c4139840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23bb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23c413fc80 0x7f23c4139840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53590/0 (socket says 172.21.15.161:53590) 2024-05-22T21:16:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23bb400640 1 -- 172.21.15.161:0/3111917420 learned_addr learned my addr 172.21.15.161:0/3111917420 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:16:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23c3e00640 1 --2- 172.21.15.161:0/3111917420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f23c413cfc0 0x7f23c4159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23bb400640 1 -- 172.21.15.161:0/3111917420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f23c413cfc0 msgr2=0x7f23c4159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23c8e00640 1 --2- 172.21.15.161:0/3111917420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23c41402e0 0x7f23c413b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23bb400640 1 --2- 172.21.15.161:0/3111917420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f23c413cfc0 0x7f23c4159680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23bb400640 1 -- 172.21.15.161:0/3111917420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23c41402e0 msgr2=0x7f23c413b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23bb400640 1 --2- 172.21.15.161:0/3111917420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23c41402e0 0x7f23c413b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23bb400640 1 -- 172.21.15.161:0/3111917420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac009420 con 0x7f23c413fc80 2024-05-22T21:16:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23c3e00640 1 --2- 172.21.15.161:0/3111917420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f23c413cfc0 0x7f23c4159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:16:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.464+0000 7f23c8e00640 1 --2- 172.21.15.161:0/3111917420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23c41402e0 0x7f23c413b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:16:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.468+0000 7f23bb400640 1 --2- 172.21.15.161:0/3111917420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23c413fc80 0x7f23c4139840 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0097f0 tx=0x7f23ac0190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.468+0000 7f23c0c00640 1 -- 172.21.15.161:0/3111917420 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac006e20 con 0x7f23c413fc80 2024-05-22T21:16:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.468+0000 7f23c0c00640 1 -- 172.21.15.161:0/3111917420 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac0b5410 con 0x7f23c413fc80 2024-05-22T21:16:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.468+0000 7f23c0c00640 1 -- 172.21.15.161:0/3111917420 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0b5780 con 0x7f23c413fc80 2024-05-22T21:16:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.468+0000 7f23cae00640 1 -- 172.21.15.161:0/3111917420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c413daf0 con 0x7f23c413fc80 2024-05-22T21:16:57.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.468+0000 7f23cae00640 1 -- 172.21.15.161:0/3111917420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c413e0b0 con 0x7f23c413fc80 2024-05-22T21:16:57.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.468+0000 7f23cae00640 1 -- 172.21.15.161:0/3111917420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c406dad0 con 0x7f23c413fc80 2024-05-22T21:16:57.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.468+0000 7f23c0c00640 1 -- 172.21.15.161:0/3111917420 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f23ac0b2050 con 0x7f23c413fc80 2024-05-22T21:16:57.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.468+0000 7f23c0c00640 1 --2- 172.21.15.161:0/3111917420 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f23a0066690 0x7f23a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.468+0000 7f23c0c00640 1 -- 172.21.15.161:0/3111917420 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23ac11e3f0 con 0x7f23c413fc80 2024-05-22T21:16:57.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.468+0000 7f23c3e00640 1 --2- 172.21.15.161:0/3111917420 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f23a0066690 0x7f23a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.468+0000 7f23c0c00640 1 -- 172.21.15.161:0/3111917420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f23a006e1c0 con 0x7f23c413fc80 2024-05-22T21:16:57.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.472+0000 7f23c3e00640 1 --2- 172.21.15.161:0/3111917420 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f23a0066690 0x7f23a0068b50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f23b4001570 tx=0x7f23b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:57.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.472+0000 7f23c0c00640 1 -- 172.21.15.161:0/3111917420 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f23ac0c5060 con 0x7f23c413fc80 2024-05-22T21:16:57.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.640+0000 7f23cae00640 1 -- 172.21.15.161:0/3111917420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f23c4067220 con 0x7f23c413fc80 2024-05-22T21:16:57.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.640+0000 7f23c0c00640 1 -- 172.21.15.161:0/3111917420 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) v1 ==== 80+0+0 (secure 0 0 0) 0x7f23ac0b0020 con 0x7f23c413fc80 2024-05-22T21:16:57.644 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:16:57.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.644+0000 7f23cae00640 1 -- 172.21.15.161:0/3111917420 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f23a0066690 msgr2=0x7f23a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.644+0000 7f23cae00640 1 --2- 172.21.15.161:0/3111917420 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f23a0066690 0x7f23a0068b50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f23b4001570 tx=0x7f23b4009040 comp rx=0 tx=0).stop 2024-05-22T21:16:57.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.644+0000 7f23cae00640 1 -- 172.21.15.161:0/3111917420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23c413fc80 msgr2=0x7f23c4139840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.644+0000 7f23cae00640 1 --2- 172.21.15.161:0/3111917420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23c413fc80 0x7f23c4139840 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0097f0 tx=0x7f23ac0190e0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.644+0000 7f23cae00640 1 -- 172.21.15.161:0/3111917420 shutdown_connections 2024-05-22T21:16:57.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.644+0000 7f23cae00640 1 --2- 172.21.15.161:0/3111917420 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f23a0066690 0x7f23a0068b50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.644+0000 7f23cae00640 1 --2- 172.21.15.161:0/3111917420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23c41402e0 0x7f23c413b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.644+0000 7f23cae00640 1 --2- 172.21.15.161:0/3111917420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23c413fc80 0x7f23c4139840 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.644+0000 7f23cae00640 1 --2- 172.21.15.161:0/3111917420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f23c413cfc0 0x7f23c4159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.644+0000 7f23cae00640 1 -- 172.21.15.161:0/3111917420 >> 172.21.15.161:0/3111917420 conn(0x7f23c406a2d0 msgr2=0x7f23c406d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:57.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.644+0000 7f23cae00640 1 -- 172.21.15.161:0/3111917420 shutdown_connections 2024-05-22T21:16:57.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.644+0000 7f23cae00640 1 -- 172.21.15.161:0/3111917420 wait complete. 2024-05-22T21:16:57.665 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-22T21:16:57.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:16:57.665 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-22T21:16:57.692 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:16:57.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:16:57.693 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-22T21:16:57.741 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:16:57.741 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:16:57.751 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:16:57.751 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:16:57.802 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:16:57.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.936+0000 7fdc67e00640 1 Processor -- start 2024-05-22T21:16:57.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.936+0000 7fdc67e00640 1 -- start start 2024-05-22T21:16:57.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.936+0000 7fdc67e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdc6815e430 0x7fdc680b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.936+0000 7fdc67e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdc680d1400 0x7fdc680cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.936+0000 7fdc67e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc680b13d0 con 0x7fdc6815d040 2024-05-22T21:16:57.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.936+0000 7fdc67e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc680b0aa0 con 0x7fdc680d1400 2024-05-22T21:16:57.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.936+0000 7fdc67e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc680b0d80 con 0x7fdc6815e430 2024-05-22T21:16:57.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.936+0000 7fdc67400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdc680d1400 0x7fdc680cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.936+0000 7fdc67400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdc680d1400 0x7fdc680cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53598/0 (socket says 172.21.15.161:53598) 2024-05-22T21:16:57.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.936+0000 7fdc67400640 1 -- 172.21.15.161:0/2596673905 learned_addr learned my addr 172.21.15.161:0/2596673905 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:16:57.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.936+0000 7fdc66000640 1 --2- 172.21.15.161:0/2596673905 >> v2:172.21.15.179:3300/0 conn(0x7fdc6815e430 0x7fdc680b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.936+0000 7fdc67400640 1 -- 172.21.15.161:0/2596673905 >> v2:172.21.15.179:3300/0 conn(0x7fdc6815e430 msgr2=0x7fdc680b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:57.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.936+0000 7fdc67400640 1 --2- 172.21.15.161:0/2596673905 >> v2:172.21.15.179:3300/0 conn(0x7fdc6815e430 0x7fdc680b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.936+0000 7fdc67400640 1 -- 172.21.15.161:0/2596673905 >> v1:172.21.15.179:6789/0 conn(0x7fdc6815d040 legacy=0x7fdc680d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.936+0000 7fdc67400640 1 -- 172.21.15.161:0/2596673905 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc680b1060 con 0x7fdc680d1400 2024-05-22T21:16:57.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.940+0000 7fdc67400640 1 --2- 172.21.15.161:0/2596673905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680d1400 0x7fdc680cd840 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009e50 tx=0x7fdc6001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc43961ba161b7d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:57.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.940+0000 7fdc65600640 1 -- 172.21.15.161:0/2596673905 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc600a57e0 con 0x7fdc680d1400 2024-05-22T21:16:57.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.940+0000 7fdc65600640 1 -- 172.21.15.161:0/2596673905 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc680ce3a0 con 0x7fdc680d1400 2024-05-22T21:16:57.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.940+0000 7fdc65600640 1 -- 172.21.15.161:0/2596673905 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc600a5b60 con 0x7fdc680d1400 2024-05-22T21:16:57.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.940+0000 7fdc67e00640 1 -- 172.21.15.161:0/2596673905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680d1400 msgr2=0x7fdc680cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.940+0000 7fdc67e00640 1 --2- 172.21.15.161:0/2596673905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680d1400 0x7fdc680cd840 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009e50 tx=0x7fdc6001c9f0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.940+0000 7fdc67e00640 1 -- 172.21.15.161:0/2596673905 shutdown_connections 2024-05-22T21:16:57.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.940+0000 7fdc67e00640 1 --2- 172.21.15.161:0/2596673905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680d1400 0x7fdc680cd840 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.940+0000 7fdc67e00640 1 --2- 172.21.15.161:0/2596673905 >> v2:172.21.15.179:3300/0 conn(0x7fdc6815e430 0x7fdc680b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.940+0000 7fdc67e00640 1 -- 172.21.15.161:0/2596673905 >> 172.21.15.161:0/2596673905 conn(0x7fdc68099a40 msgr2=0x7fdc68099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:57.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.940+0000 7fdc67e00640 1 -- 172.21.15.161:0/2596673905 shutdown_connections 2024-05-22T21:16:57.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.940+0000 7fdc67e00640 1 -- 172.21.15.161:0/2596673905 wait complete. 2024-05-22T21:16:57.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.940+0000 7fdc67e00640 1 Processor -- start 2024-05-22T21:16:57.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.940+0000 7fdc67e00640 1 -- start start 2024-05-22T21:16:57.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc67e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680d1400 0x7fdc68153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc67e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc6815d040 0x7fdc680cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc67e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc6815e430 0x7fdc680cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc67e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc68154580 con 0x7fdc680d1400 2024-05-22T21:16:57.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc67e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc680cbc60 con 0x7fdc6815d040 2024-05-22T21:16:57.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc67e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6815bc80 con 0x7fdc6815e430 2024-05-22T21:16:57.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc66a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680d1400 0x7fdc68153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc66000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc6815d040 0x7fdc680cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc66a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680d1400 0x7fdc68153bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53612/0 (socket says 172.21.15.161:53612) 2024-05-22T21:16:57.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc66000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc6815d040 0x7fdc680cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:48766/0 (socket says 172.21.15.161:48766) 2024-05-22T21:16:57.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc66a00640 1 -- 172.21.15.161:0/2912297369 learned_addr learned my addr 172.21.15.161:0/2912297369 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:16:57.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc67400640 1 --2- 172.21.15.161:0/2912297369 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc6815e430 0x7fdc680cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc66a00640 1 -- 172.21.15.161:0/2912297369 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc6815e430 msgr2=0x7fdc680cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc66a00640 1 --2- 172.21.15.161:0/2912297369 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc6815e430 0x7fdc680cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc66a00640 1 -- 172.21.15.161:0/2912297369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc6815d040 msgr2=0x7fdc680cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:57.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc66a00640 1 --2- 172.21.15.161:0/2912297369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc6815d040 0x7fdc680cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:57.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc66a00640 1 -- 172.21.15.161:0/2912297369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc60009b30 con 0x7fdc680d1400 2024-05-22T21:16:57.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc66a00640 1 --2- 172.21.15.161:0/2912297369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680d1400 0x7fdc68153bb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800c530 tx=0x7fdc5800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:57.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc5f400640 1 -- 172.21.15.161:0/2912297369 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58005220 con 0x7fdc680d1400 2024-05-22T21:16:57.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc5f400640 1 -- 172.21.15.161:0/2912297369 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc58005380 con 0x7fdc680d1400 2024-05-22T21:16:57.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc5f400640 1 -- 172.21.15.161:0/2912297369 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58007270 con 0x7fdc680d1400 2024-05-22T21:16:57.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc67e00640 1 -- 172.21.15.161:0/2912297369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6806bd00 con 0x7fdc680d1400 2024-05-22T21:16:57.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc67e00640 1 -- 172.21.15.161:0/2912297369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6815c240 con 0x7fdc680d1400 2024-05-22T21:16:57.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.944+0000 7fdc67e00640 1 -- 172.21.15.161:0/2912297369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc68067070 con 0x7fdc680d1400 2024-05-22T21:16:57.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.948+0000 7fdc5f400640 1 -- 172.21.15.161:0/2912297369 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdc580073d0 con 0x7fdc680d1400 2024-05-22T21:16:57.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.948+0000 7fdc5f400640 1 --2- 172.21.15.161:0/2912297369 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc48066690 0x7fdc48068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:57.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.948+0000 7fdc5f400640 1 -- 172.21.15.161:0/2912297369 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc580839a0 con 0x7fdc680d1400 2024-05-22T21:16:57.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.948+0000 7fdc66000640 1 --2- 172.21.15.161:0/2912297369 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc48066690 0x7fdc48068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:57.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.948+0000 7fdc5f400640 1 -- 172.21.15.161:0/2912297369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7fdc4806e1c0 con 0x7fdc680d1400 2024-05-22T21:16:57.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.948+0000 7fdc66000640 1 --2- 172.21.15.161:0/2912297369 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc48066690 0x7fdc48068b50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdc680b0200 tx=0x7fdc54008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:57.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:57.952+0000 7fdc5f400640 1 -- 172.21.15.161:0/2912297369 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdc58050db0 con 0x7fdc680d1400 2024-05-22T21:16:58.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.184+0000 7fdc67e00640 1 -- 172.21.15.161:0/2912297369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc680cbe90 con 0x7fdc680d1400 2024-05-22T21:16:58.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.188+0000 7fdc5f400640 1 -- 172.21.15.161:0/2912297369 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16203 (secure 0 0 0) 0x7fdc580543d0 con 0x7fdc680d1400 2024-05-22T21:16:58.190 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:16:58.190 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":69,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:15:57.531374+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:15:40.613692+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-22T21:15:41.954314+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:16:58.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.188+0000 7fdc67e00640 1 -- 172.21.15.161:0/2912297369 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc48066690 msgr2=0x7fdc48068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:58.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.188+0000 7fdc67e00640 1 --2- 172.21.15.161:0/2912297369 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc48066690 0x7fdc48068b50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdc680b0200 tx=0x7fdc54008040 comp rx=0 tx=0).stop 2024-05-22T21:16:58.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.188+0000 7fdc67e00640 1 -- 172.21.15.161:0/2912297369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680d1400 msgr2=0x7fdc68153bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:58.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.188+0000 7fdc67e00640 1 --2- 172.21.15.161:0/2912297369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680d1400 0x7fdc68153bb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800c530 tx=0x7fdc5800c9f0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.188+0000 7fdc67e00640 1 -- 172.21.15.161:0/2912297369 shutdown_connections 2024-05-22T21:16:58.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.188+0000 7fdc67e00640 1 --2- 172.21.15.161:0/2912297369 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc48066690 0x7fdc48068b50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.188+0000 7fdc67e00640 1 --2- 172.21.15.161:0/2912297369 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc6815e430 0x7fdc680cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.188+0000 7fdc67e00640 1 --2- 172.21.15.161:0/2912297369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc6815d040 0x7fdc680cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.188+0000 7fdc67e00640 1 --2- 172.21.15.161:0/2912297369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc680d1400 0x7fdc68153bb0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.188+0000 7fdc67e00640 1 -- 172.21.15.161:0/2912297369 >> 172.21.15.161:0/2912297369 conn(0x7fdc68099a40 msgr2=0x7fdc6806dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:58.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.188+0000 7fdc67e00640 1 -- 172.21.15.161:0/2912297369 shutdown_connections 2024-05-22T21:16:58.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.188+0000 7fdc67e00640 1 -- 172.21.15.161:0/2912297369 wait complete. 2024-05-22T21:16:58.211 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:15:40.613692+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:15:41.954314+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:16:58.212 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:16:58.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bcd200640 1 Processor -- start 2024-05-22T21:16:58.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bcd200640 1 -- start start 2024-05-22T21:16:58.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bcd200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4bc80b35f0 0x7f4bc80cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:58.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bcd200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc80cfd30 con 0x7f4bc815b300 2024-05-22T21:16:58.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bcd200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc80cbb60 con 0x7f4bc815ece0 2024-05-22T21:16:58.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bcd200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc80b3a50 con 0x7f4bc80b35f0 2024-05-22T21:16:58.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc5e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f4bc815b300 0x7f4bc80cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58596/0 (socket says 172.21.15.161:58596) 2024-05-22T21:16:58.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc5e00640 1 -- 172.21.15.161:0/1051020453 learned_addr learned my addr 172.21.15.161:0/1051020453 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:16:58.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc5400640 1 --2- 172.21.15.161:0/1051020453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4bc80b35f0 0x7f4bc80cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:58.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc4a00640 1 -- 172.21.15.161:0/1051020453 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067091902 0 0) 0x7f4bc80cfd30 con 0x7f4bc815b300 2024-05-22T21:16:58.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc4a00640 1 -- 172.21.15.161:0/1051020453 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bac003530 con 0x7f4bc815b300 2024-05-22T21:16:58.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc4a00640 1 -- 172.21.15.161:0/1051020453 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4177763229 0 0) 0x7f4bac003530 con 0x7f4bc815b300 2024-05-22T21:16:58.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc4a00640 1 -- 172.21.15.161:0/1051020453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4bc80b35f0 msgr2=0x7f4bc80cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:58.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc4a00640 1 --2- 172.21.15.161:0/1051020453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4bc80b35f0 0x7f4bc80cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc4a00640 1 -- 172.21.15.161:0/1051020453 >> v1:172.21.15.179:6789/0 conn(0x7f4bc815ece0 legacy=0x7f4bc806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:58.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc4a00640 1 -- 172.21.15.161:0/1051020453 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc80cbe40 con 0x7f4bc815b300 2024-05-22T21:16:58.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc4a00640 1 -- 172.21.15.161:0/1051020453 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f4bb8001680 con 0x7f4bc815b300 2024-05-22T21:16:58.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc4a00640 1 -- 172.21.15.161:0/1051020453 >> v1:172.21.15.161:6789/0 conn(0x7f4bc815b300 legacy=0x7f4bc80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:58.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc4a00640 1 --2- 172.21.15.161:0/1051020453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4bc80b35f0 0x7f4bac0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:58.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc4a00640 1 --2- 172.21.15.161:0/1051020453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bac003ef0 0x7f4bac0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:58.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc4a00640 1 --2- 172.21.15.161:0/1051020453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bac004d10 0x7f4bac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:58.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc4a00640 1 -- 172.21.15.161:0/1051020453 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc80b3a50 con 0x7f4bac003ef0 2024-05-22T21:16:58.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc4a00640 1 -- 172.21.15.161:0/1051020453 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac00b9f0 con 0x7f4bac004d10 2024-05-22T21:16:58.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc4a00640 1 -- 172.21.15.161:0/1051020453 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac00bbb0 con 0x7f4bc80b35f0 2024-05-22T21:16:58.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc5e00640 1 --2- 172.21.15.161:0/1051020453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4bc80b35f0 0x7f4bac0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:58.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc5400640 1 --2- 172.21.15.161:0/1051020453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bac003ef0 0x7f4bac0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:58.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc6800640 1 --2- 172.21.15.161:0/1051020453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bac004d10 0x7f4bac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:58.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc5e00640 1 -- 172.21.15.161:0/1051020453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bac004d10 msgr2=0x7f4bac005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:58.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc5e00640 1 --2- 172.21.15.161:0/1051020453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bac004d10 0x7f4bac005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc5e00640 1 -- 172.21.15.161:0/1051020453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bac003ef0 msgr2=0x7f4bac0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:58.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc5e00640 1 --2- 172.21.15.161:0/1051020453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bac003ef0 0x7f4bac0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc5e00640 1 -- 172.21.15.161:0/1051020453 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac003530 con 0x7f4bc80b35f0 2024-05-22T21:16:58.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.348+0000 7f4bc5400640 1 --2- 172.21.15.161:0/1051020453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bac003ef0 0x7f4bac0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:16:58.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.352+0000 7f4bc5e00640 1 --2- 172.21.15.161:0/1051020453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4bc80b35f0 0x7f4bac0039b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800c190 tx=0x7f4bb801cb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d082ea5bc3c838f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:58.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.352+0000 7f4bc4a00640 1 -- 172.21.15.161:0/1051020453 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8005ae0 con 0x7f4bc80b35f0 2024-05-22T21:16:58.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.352+0000 7f4bc4a00640 1 -- 172.21.15.161:0/1051020453 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb8005c60 con 0x7f4bc80b35f0 2024-05-22T21:16:58.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.352+0000 7f4bc4a00640 1 -- 172.21.15.161:0/1051020453 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80a5a80 con 0x7f4bc80b35f0 2024-05-22T21:16:58.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.352+0000 7f4bcd200640 1 -- 172.21.15.161:0/1051020453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4bc80b35f0 msgr2=0x7f4bac0039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:58.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.352+0000 7f4bcd200640 1 --2- 172.21.15.161:0/1051020453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4bc80b35f0 0x7f4bac0039b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800c190 tx=0x7f4bb801cb40 comp rx=0 tx=0).stop 2024-05-22T21:16:58.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.352+0000 7f4bc6800640 1 -- 172.21.15.161:0/1051020453 reap_dead start 2024-05-22T21:16:58.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.352+0000 7f4bcd200640 1 -- 172.21.15.161:0/1051020453 shutdown_connections 2024-05-22T21:16:58.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.352+0000 7f4bcd200640 1 -- 172.21.15.161:0/1051020453 >> 172.21.15.161:0/1051020453 conn(0x7f4bc80b2a60 msgr2=0x7f4bc80b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:58.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.352+0000 7f4bcd200640 1 -- 172.21.15.161:0/1051020453 shutdown_connections 2024-05-22T21:16:58.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.352+0000 7f4bcd200640 1 -- 172.21.15.161:0/1051020453 wait complete. 2024-05-22T21:16:58.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bcd200640 1 Processor -- start 2024-05-22T21:16:58.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bcd200640 1 -- start start 2024-05-22T21:16:58.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bcd200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4bc80b35f0 0x7f4bc8155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:58.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bcd200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bc81554b0 0x7f4bc8158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:58.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bcd200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bc8155930 0x7f4bc8159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:58.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bcd200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc8157d40 con 0x7f4bc81554b0 2024-05-22T21:16:58.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bcd200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc8158000 con 0x7f4bc8155930 2024-05-22T21:16:58.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bcd200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc81582e0 con 0x7f4bc80b35f0 2024-05-22T21:16:58.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bc5400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bc81554b0 0x7f4bc8158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:58.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bc5400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bc81554b0 0x7f4bc8158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53630/0 (socket says 172.21.15.161:53630) 2024-05-22T21:16:58.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bc5400640 1 -- 172.21.15.161:0/1610087296 learned_addr learned my addr 172.21.15.161:0/1610087296 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:16:58.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bc6800640 1 --2- 172.21.15.161:0/1610087296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bc8155930 0x7f4bc8159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:58.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bc5e00640 1 --2- 172.21.15.161:0/1610087296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4bc80b35f0 0x7f4bc8155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:58.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bc5400640 1 -- 172.21.15.161:0/1610087296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4bc80b35f0 msgr2=0x7f4bc8155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:58.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bc5400640 1 --2- 172.21.15.161:0/1610087296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4bc80b35f0 0x7f4bc8155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bc5400640 1 -- 172.21.15.161:0/1610087296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bc8155930 msgr2=0x7f4bc8159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:58.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bc5400640 1 --2- 172.21.15.161:0/1610087296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bc8155930 0x7f4bc8159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bc5400640 1 -- 172.21.15.161:0/1610087296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc81585c0 con 0x7f4bc81554b0 2024-05-22T21:16:58.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bc5400640 1 --2- 172.21.15.161:0/1610087296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bc81554b0 0x7f4bc8158dc0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4011cb0 tx=0x7f4bb400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:58.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bbe000640 1 -- 172.21.15.161:0/1610087296 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb400fd20 con 0x7f4bc81554b0 2024-05-22T21:16:58.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bbe000640 1 -- 172.21.15.161:0/1610087296 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb400fe80 con 0x7f4bc81554b0 2024-05-22T21:16:58.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bbe000640 1 -- 172.21.15.161:0/1610087296 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4004220 con 0x7f4bc81554b0 2024-05-22T21:16:58.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bcd200640 1 -- 172.21.15.161:0/1610087296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc81588a0 con 0x7f4bc81554b0 2024-05-22T21:16:58.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.356+0000 7f4bcd200640 1 -- 172.21.15.161:0/1610087296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc80b4a40 con 0x7f4bc81554b0 2024-05-22T21:16:58.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.360+0000 7f4bcd200640 1 -- 172.21.15.161:0/1610087296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc8073ce0 con 0x7f4bc81554b0 2024-05-22T21:16:58.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.360+0000 7f4bbe000640 1 -- 172.21.15.161:0/1610087296 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4bb4004380 con 0x7f4bc81554b0 2024-05-22T21:16:58.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.360+0000 7f4bbe000640 1 --2- 172.21.15.161:0/1610087296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ba8066690 0x7f4ba8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:58.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.360+0000 7f4bbe000640 1 -- 172.21.15.161:0/1610087296 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bb4084bf0 con 0x7f4bc81554b0 2024-05-22T21:16:58.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.360+0000 7f4bbe000640 1 -- 172.21.15.161:0/1610087296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f4ba806e1c0 con 0x7f4bc81554b0 2024-05-22T21:16:58.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.360+0000 7f4bc5e00640 1 --2- 172.21.15.161:0/1610087296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ba8066690 0x7f4ba8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:58.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.360+0000 7f4bc5e00640 1 --2- 172.21.15.161:0/1610087296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ba8066690 0x7f4ba8068b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80ab000 tx=0x7f4bb80a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:58.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.364+0000 7f4bbe000640 1 -- 172.21.15.161:0/1610087296 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4bb4022020 con 0x7f4bc81554b0 2024-05-22T21:16:58.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.524+0000 7f4bcd200640 1 -- 172.21.15.161:0/1610087296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4bc8158140 con 0x7f4bc81554b0 2024-05-22T21:16:58.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.524+0000 7f4bbe000640 1 -- 172.21.15.161:0/1610087296 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4bc8158140 con 0x7f4bc81554b0 2024-05-22T21:16:58.526 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:16:58.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.524+0000 7f4bcd200640 1 -- 172.21.15.161:0/1610087296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ba8066690 msgr2=0x7f4ba8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:58.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.524+0000 7f4bcd200640 1 --2- 172.21.15.161:0/1610087296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ba8066690 0x7f4ba8068b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80ab000 tx=0x7f4bb80a7070 comp rx=0 tx=0).stop 2024-05-22T21:16:58.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.524+0000 7f4bcd200640 1 -- 172.21.15.161:0/1610087296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bc81554b0 msgr2=0x7f4bc8158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:58.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.524+0000 7f4bcd200640 1 --2- 172.21.15.161:0/1610087296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bc81554b0 0x7f4bc8158dc0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4011cb0 tx=0x7f4bb400f610 comp rx=0 tx=0).stop 2024-05-22T21:16:58.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.524+0000 7f4bcd200640 1 -- 172.21.15.161:0/1610087296 shutdown_connections 2024-05-22T21:16:58.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.524+0000 7f4bcd200640 1 --2- 172.21.15.161:0/1610087296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4ba8066690 0x7f4ba8068b50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.524+0000 7f4bcd200640 1 --2- 172.21.15.161:0/1610087296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4bc8155930 0x7f4bc8159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.524+0000 7f4bcd200640 1 --2- 172.21.15.161:0/1610087296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bc81554b0 0x7f4bc8158dc0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.524+0000 7f4bcd200640 1 --2- 172.21.15.161:0/1610087296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4bc80b35f0 0x7f4bc8155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.524+0000 7f4bcd200640 1 -- 172.21.15.161:0/1610087296 >> 172.21.15.161:0/1610087296 conn(0x7f4bc80b2a60 msgr2=0x7f4bc806f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:58.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.524+0000 7f4bcd200640 1 -- 172.21.15.161:0/1610087296 shutdown_connections 2024-05-22T21:16:58.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.524+0000 7f4bcd200640 1 -- 172.21.15.161:0/1610087296 wait complete. 2024-05-22T21:16:58.547 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:16:58.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.680+0000 7efc68400640 1 Processor -- start 2024-05-22T21:16:58.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.680+0000 7efc68400640 1 -- start start 2024-05-22T21:16:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.684+0000 7efc68400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efc6013cfc0 0x7efc601402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.684+0000 7efc68400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc6013fe50 0x7efc6006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.684+0000 7efc68400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc6013d3a0 con 0x7efc6015af60 2024-05-22T21:16:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.684+0000 7efc68400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc600cd920 con 0x7efc6013cfc0 2024-05-22T21:16:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.684+0000 7efc68400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc600cdc00 con 0x7efc6013fe50 2024-05-22T21:16:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.684+0000 7efc65000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7efc6015af60 0x7efc6013f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58600/0 (socket says 172.21.15.161:58600) 2024-05-22T21:16:58.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.684+0000 7efc65000640 1 -- 172.21.15.161:0/1465606790 learned_addr learned my addr 172.21.15.161:0/1465606790 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:16:58.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.684+0000 7efc65a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efc6013cfc0 0x7efc601402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:58.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.684+0000 7efc66400640 1 --2- 172.21.15.161:0/1465606790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc6013fe50 0x7efc6006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:58.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.684+0000 7efc5fe00640 1 -- 172.21.15.161:0/1465606790 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498838942 0 0) 0x7efc6013d3a0 con 0x7efc6015af60 2024-05-22T21:16:58.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.684+0000 7efc5fe00640 1 -- 172.21.15.161:0/1465606790 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc48003530 con 0x7efc6015af60 2024-05-22T21:16:58.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.684+0000 7efc65a00640 1 -- 172.21.15.161:0/1465606790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc6013fe50 msgr2=0x7efc6006fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:58.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.684+0000 7efc65a00640 1 --2- 172.21.15.161:0/1465606790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc6013fe50 0x7efc6006fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.684+0000 7efc65a00640 1 -- 172.21.15.161:0/1465606790 >> v1:172.21.15.161:6789/0 conn(0x7efc6015af60 legacy=0x7efc6013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:58.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.684+0000 7efc65a00640 1 -- 172.21.15.161:0/1465606790 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc600cdee0 con 0x7efc6013cfc0 2024-05-22T21:16:58.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.688+0000 7efc65a00640 1 --2- 172.21.15.161:0/1465606790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc6013cfc0 0x7efc601402e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7efc540097c0 tx=0x7efc5401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbbda11e4e4ffc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:58.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.688+0000 7efc5fe00640 1 -- 172.21.15.161:0/1465606790 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc540ab600 con 0x7efc6013cfc0 2024-05-22T21:16:58.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.688+0000 7efc5fe00640 1 -- 172.21.15.161:0/1465606790 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc540a7050 con 0x7efc6013cfc0 2024-05-22T21:16:58.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.688+0000 7efc5fe00640 1 -- 172.21.15.161:0/1465606790 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc540ab980 con 0x7efc6013cfc0 2024-05-22T21:16:58.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.688+0000 7efc68400640 1 -- 172.21.15.161:0/1465606790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc6013cfc0 msgr2=0x7efc601402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:58.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.688+0000 7efc68400640 1 --2- 172.21.15.161:0/1465606790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc6013cfc0 0x7efc601402e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7efc540097c0 tx=0x7efc5401c3e0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.688+0000 7efc68400640 1 -- 172.21.15.161:0/1465606790 shutdown_connections 2024-05-22T21:16:58.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.688+0000 7efc68400640 1 --2- 172.21.15.161:0/1465606790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc6013fe50 0x7efc6006fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.688+0000 7efc68400640 1 --2- 172.21.15.161:0/1465606790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc6013cfc0 0x7efc601402e0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.688+0000 7efc68400640 1 -- 172.21.15.161:0/1465606790 >> 172.21.15.161:0/1465606790 conn(0x7efc6006a2d0 msgr2=0x7efc6006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:58.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.688+0000 7efc68400640 1 -- 172.21.15.161:0/1465606790 shutdown_connections 2024-05-22T21:16:58.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.688+0000 7efc68400640 1 -- 172.21.15.161:0/1465606790 wait complete. 2024-05-22T21:16:58.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.688+0000 7efc68400640 1 Processor -- start 2024-05-22T21:16:58.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.688+0000 7efc68400640 1 -- start start 2024-05-22T21:16:58.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc68400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc6013cfc0 0x7efc60156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:58.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc68400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc6013fe50 0x7efc60158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:58.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc68400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc6015af60 0x7efc6013ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:58.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc68400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc60159f10 con 0x7efc6015af60 2024-05-22T21:16:58.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc68400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc60159390 con 0x7efc6013fe50 2024-05-22T21:16:58.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc68400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc60159670 con 0x7efc6013cfc0 2024-05-22T21:16:58.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc66400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc6015af60 0x7efc6013ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:58.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc66400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc6015af60 0x7efc6013ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53658/0 (socket says 172.21.15.161:53658) 2024-05-22T21:16:58.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc66400640 1 -- 172.21.15.161:0/2379519264 learned_addr learned my addr 172.21.15.161:0/2379519264 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:16:58.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc65a00640 1 --2- 172.21.15.161:0/2379519264 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc6013cfc0 0x7efc60156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:58.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc65000640 1 --2- 172.21.15.161:0/2379519264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc6013fe50 0x7efc60158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:58.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc66400640 1 -- 172.21.15.161:0/2379519264 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc6013cfc0 msgr2=0x7efc60156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:58.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc66400640 1 --2- 172.21.15.161:0/2379519264 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc6013cfc0 0x7efc60156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc66400640 1 -- 172.21.15.161:0/2379519264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc6013fe50 msgr2=0x7efc60158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:58.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc66400640 1 --2- 172.21.15.161:0/2379519264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc6013fe50 0x7efc60158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc66400640 1 -- 172.21.15.161:0/2379519264 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc54009420 con 0x7efc6015af60 2024-05-22T21:16:58.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc66400640 1 --2- 172.21.15.161:0/2379519264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc6015af60 0x7efc6013ad10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7efc5800efc0 tx=0x7efc5800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:58.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc5d600640 1 -- 172.21.15.161:0/2379519264 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc58009ca0 con 0x7efc6015af60 2024-05-22T21:16:58.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc5d600640 1 -- 172.21.15.161:0/2379519264 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5800f040 con 0x7efc6015af60 2024-05-22T21:16:58.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc5d600640 1 -- 172.21.15.161:0/2379519264 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc580146c0 con 0x7efc6015af60 2024-05-22T21:16:58.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc68400640 1 -- 172.21.15.161:0/2379519264 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc60159950 con 0x7efc6015af60 2024-05-22T21:16:58.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc68400640 1 -- 172.21.15.161:0/2379519264 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc60157320 con 0x7efc6015af60 2024-05-22T21:16:58.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc68400640 1 -- 172.21.15.161:0/2379519264 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc60164c90 con 0x7efc6015af60 2024-05-22T21:16:58.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.692+0000 7efc5d600640 1 -- 172.21.15.161:0/2379519264 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efc58014820 con 0x7efc6015af60 2024-05-22T21:16:58.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.696+0000 7efc5d600640 1 --2- 172.21.15.161:0/2379519264 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc40066690 0x7efc40068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:58.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.696+0000 7efc5d600640 1 -- 172.21.15.161:0/2379519264 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc58084e50 con 0x7efc6015af60 2024-05-22T21:16:58.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.696+0000 7efc65a00640 1 --2- 172.21.15.161:0/2379519264 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc40066690 0x7efc40068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:58.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.696+0000 7efc5d600640 1 -- 172.21.15.161:0/2379519264 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7efc4006e1c0 con 0x7efc6015af60 2024-05-22T21:16:58.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.696+0000 7efc65a00640 1 --2- 172.21.15.161:0/2379519264 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc40066690 0x7efc40068b50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efc540ad000 tx=0x7efc54008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:58.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.700+0000 7efc5d600640 1 -- 172.21.15.161:0/2379519264 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efc58052300 con 0x7efc6015af60 2024-05-22T21:16:58.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.856+0000 7efc68400640 1 -- 172.21.15.161:0/2379519264 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efc60067220 con 0x7efc6015af60 2024-05-22T21:16:58.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.856+0000 7efc5d600640 1 -- 172.21.15.161:0/2379519264 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7efc58018070 con 0x7efc6015af60 2024-05-22T21:16:58.859 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:16:58.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.856+0000 7efc68400640 1 -- 172.21.15.161:0/2379519264 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc40066690 msgr2=0x7efc40068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:58.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.856+0000 7efc68400640 1 --2- 172.21.15.161:0/2379519264 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc40066690 0x7efc40068b50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efc540ad000 tx=0x7efc54008040 comp rx=0 tx=0).stop 2024-05-22T21:16:58.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.856+0000 7efc68400640 1 -- 172.21.15.161:0/2379519264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc6015af60 msgr2=0x7efc6013ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:58.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.856+0000 7efc68400640 1 --2- 172.21.15.161:0/2379519264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc6015af60 0x7efc6013ad10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7efc5800efc0 tx=0x7efc5800c490 comp rx=0 tx=0).stop 2024-05-22T21:16:58.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.856+0000 7efc68400640 1 -- 172.21.15.161:0/2379519264 shutdown_connections 2024-05-22T21:16:58.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.856+0000 7efc68400640 1 --2- 172.21.15.161:0/2379519264 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc40066690 0x7efc40068b50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.856+0000 7efc68400640 1 --2- 172.21.15.161:0/2379519264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc6015af60 0x7efc6013ad10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.856+0000 7efc68400640 1 --2- 172.21.15.161:0/2379519264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc6013fe50 0x7efc60158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.856+0000 7efc68400640 1 --2- 172.21.15.161:0/2379519264 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc6013cfc0 0x7efc60156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:58.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.856+0000 7efc68400640 1 -- 172.21.15.161:0/2379519264 >> 172.21.15.161:0/2379519264 conn(0x7efc6006a2d0 msgr2=0x7efc6006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:58.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.856+0000 7efc68400640 1 -- 172.21.15.161:0/2379519264 shutdown_connections 2024-05-22T21:16:58.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:58.856+0000 7efc68400640 1 -- 172.21.15.161:0/2379519264 wait complete. 2024-05-22T21:16:58.879 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:16:59.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f947a000640 1 Processor -- start 2024-05-22T21:16:59.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f947a000640 1 -- start start 2024-05-22T21:16:59.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f947a000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947413cfc0 0x7f94741402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f947a000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f947413fe50 0x7f947406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f947a000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947413d3a0 con 0x7f947415af60 2024-05-22T21:16:59.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f947a000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94740cd920 con 0x7f947413fe50 2024-05-22T21:16:59.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f947a000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94740cdc00 con 0x7f947413cfc0 2024-05-22T21:16:59.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f946aa00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f947415af60 0x7f947413f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58604/0 (socket says 172.21.15.161:58604) 2024-05-22T21:16:59.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f946aa00640 1 -- 172.21.15.161:0/951242509 learned_addr learned my addr 172.21.15.161:0/951242509 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:16:59.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f9473400640 1 --2- 172.21.15.161:0/951242509 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947413cfc0 0x7f94741402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f9473e00640 1 --2- 172.21.15.161:0/951242509 >> v2:172.21.15.179:3300/0 conn(0x7f947413fe50 0x7f947406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f9472a00640 1 -- 172.21.15.161:0/951242509 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2647594955 0 0) 0x7f947413d3a0 con 0x7f947415af60 2024-05-22T21:16:59.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f9472a00640 1 -- 172.21.15.161:0/951242509 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9458003530 con 0x7f947415af60 2024-05-22T21:16:59.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f9473400640 1 -- 172.21.15.161:0/951242509 >> v2:172.21.15.179:3300/0 conn(0x7f947413fe50 msgr2=0x7f947406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:59.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f9473400640 1 --2- 172.21.15.161:0/951242509 >> v2:172.21.15.179:3300/0 conn(0x7f947413fe50 0x7f947406fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f9473400640 1 -- 172.21.15.161:0/951242509 >> v1:172.21.15.161:6789/0 conn(0x7f947415af60 legacy=0x7f947413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f9473400640 1 -- 172.21.15.161:0/951242509 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94740cdee0 con 0x7f947413cfc0 2024-05-22T21:16:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.016+0000 7f9473e00640 1 --2- 172.21.15.161:0/951242509 >> v2:172.21.15.179:3300/0 conn(0x7f947413fe50 0x7f947406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:16:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.020+0000 7f9473400640 1 --2- 172.21.15.161:0/951242509 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947413cfc0 0x7f94741402e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f94640097c0 tx=0x7f946401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e9b9eeebb51019b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:59.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.020+0000 7f9472a00640 1 -- 172.21.15.161:0/951242509 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94640ab600 con 0x7f947413cfc0 2024-05-22T21:16:59.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.020+0000 7f9472a00640 1 -- 172.21.15.161:0/951242509 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94640a7050 con 0x7f947413cfc0 2024-05-22T21:16:59.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.020+0000 7f9472a00640 1 -- 172.21.15.161:0/951242509 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94640ab980 con 0x7f947413cfc0 2024-05-22T21:16:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.020+0000 7f947a000640 1 -- 172.21.15.161:0/951242509 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947413cfc0 msgr2=0x7f94741402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.020+0000 7f947a000640 1 --2- 172.21.15.161:0/951242509 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947413cfc0 0x7f94741402e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f94640097c0 tx=0x7f946401c3e0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.020+0000 7f947a000640 1 -- 172.21.15.161:0/951242509 shutdown_connections 2024-05-22T21:16:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.020+0000 7f947a000640 1 --2- 172.21.15.161:0/951242509 >> v2:172.21.15.179:3300/0 conn(0x7f947413fe50 0x7f947406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.020+0000 7f947a000640 1 --2- 172.21.15.161:0/951242509 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947413cfc0 0x7f94741402e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.020+0000 7f947a000640 1 -- 172.21.15.161:0/951242509 >> 172.21.15.161:0/951242509 conn(0x7f947406a2d0 msgr2=0x7f947406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:59.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.020+0000 7f947a000640 1 -- 172.21.15.161:0/951242509 shutdown_connections 2024-05-22T21:16:59.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.020+0000 7f947a000640 1 -- 172.21.15.161:0/951242509 wait complete. 2024-05-22T21:16:59.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f947a000640 1 Processor -- start 2024-05-22T21:16:59.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f947a000640 1 -- start start 2024-05-22T21:16:59.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f947a000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947413cfc0 0x7f947413b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f947a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f947413fe50 0x7f9474136660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f947a000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f947415af60 0x7f9474136ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f947a000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947413c960 con 0x7f947413fe50 2024-05-22T21:16:59.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f947a000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947413bad0 con 0x7f947415af60 2024-05-22T21:16:59.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f947a000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947413bd50 con 0x7f947413cfc0 2024-05-22T21:16:59.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f946aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f947413fe50 0x7f9474136660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f946aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f947413fe50 0x7f9474136660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53672/0 (socket says 172.21.15.161:53672) 2024-05-22T21:16:59.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f946aa00640 1 -- 172.21.15.161:0/928458320 learned_addr learned my addr 172.21.15.161:0/928458320 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:16:59.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f9473400640 1 --2- 172.21.15.161:0/928458320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947413cfc0 0x7f947413b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f9473e00640 1 --2- 172.21.15.161:0/928458320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f947415af60 0x7f9474136ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f946aa00640 1 -- 172.21.15.161:0/928458320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947413cfc0 msgr2=0x7f947413b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f946aa00640 1 --2- 172.21.15.161:0/928458320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947413cfc0 0x7f947413b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f946aa00640 1 -- 172.21.15.161:0/928458320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f947415af60 msgr2=0x7f9474136ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f946aa00640 1 --2- 172.21.15.161:0/928458320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f947415af60 0x7f9474136ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f946aa00640 1 -- 172.21.15.161:0/928458320 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c009a70 con 0x7f947413fe50 2024-05-22T21:16:59.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f946aa00640 1 --2- 172.21.15.161:0/928458320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f947413fe50 0x7f9474136660 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f945c00c3f0 tx=0x7f945c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:59.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f946be00640 1 -- 172.21.15.161:0/928458320 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c005cf0 con 0x7f947413fe50 2024-05-22T21:16:59.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f946be00640 1 -- 172.21.15.161:0/928458320 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c005e50 con 0x7f947413fe50 2024-05-22T21:16:59.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f947a000640 1 -- 172.21.15.161:0/928458320 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9464009420 con 0x7f947413fe50 2024-05-22T21:16:59.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f946be00640 1 -- 172.21.15.161:0/928458320 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c0072f0 con 0x7f947413fe50 2024-05-22T21:16:59.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f947a000640 1 -- 172.21.15.161:0/928458320 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947413c310 con 0x7f947413fe50 2024-05-22T21:16:59.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.024+0000 7f947a000640 1 -- 172.21.15.161:0/928458320 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947413cb10 con 0x7f947413fe50 2024-05-22T21:16:59.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.028+0000 7f946be00640 1 -- 172.21.15.161:0/928458320 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f945c007450 con 0x7f947413fe50 2024-05-22T21:16:59.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.028+0000 7f946be00640 1 --2- 172.21.15.161:0/928458320 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9450066690 0x7f9450068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.028+0000 7f946be00640 1 -- 172.21.15.161:0/928458320 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f945c0836f0 con 0x7f947413fe50 2024-05-22T21:16:59.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.028+0000 7f946be00640 1 -- 172.21.15.161:0/928458320 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f945006e1c0 con 0x7f947413fe50 2024-05-22T21:16:59.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.028+0000 7f9473400640 1 --2- 172.21.15.161:0/928458320 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9450066690 0x7f9450068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.028+0000 7f9473400640 1 --2- 172.21.15.161:0/928458320 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9450066690 0x7f9450068b50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f94640ad000 tx=0x7f94640a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:59.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.032+0000 7f946be00640 1 -- 172.21.15.161:0/928458320 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f945c017060 con 0x7f947413fe50 2024-05-22T21:16:59.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.184+0000 7f947a000640 1 -- 172.21.15.161:0/928458320 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9474067220 con 0x7f947413fe50 2024-05-22T21:16:59.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.184+0000 7f946be00640 1 -- 172.21.15.161:0/928458320 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7f947413cb10 con 0x7f947413fe50 2024-05-22T21:16:59.188 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:16:59.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.188+0000 7f947a000640 1 -- 172.21.15.161:0/928458320 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9450066690 msgr2=0x7f9450068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.188+0000 7f947a000640 1 --2- 172.21.15.161:0/928458320 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9450066690 0x7f9450068b50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f94640ad000 tx=0x7f94640a6000 comp rx=0 tx=0).stop 2024-05-22T21:16:59.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.188+0000 7f947a000640 1 -- 172.21.15.161:0/928458320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f947413fe50 msgr2=0x7f9474136660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.188+0000 7f947a000640 1 --2- 172.21.15.161:0/928458320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f947413fe50 0x7f9474136660 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f945c00c3f0 tx=0x7f945c00c7f0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.188+0000 7f947a000640 1 -- 172.21.15.161:0/928458320 shutdown_connections 2024-05-22T21:16:59.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.188+0000 7f947a000640 1 --2- 172.21.15.161:0/928458320 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9450066690 0x7f9450068b50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.188+0000 7f947a000640 1 --2- 172.21.15.161:0/928458320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f947415af60 0x7f9474136ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.188+0000 7f947a000640 1 --2- 172.21.15.161:0/928458320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f947413fe50 0x7f9474136660 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.188+0000 7f947a000640 1 --2- 172.21.15.161:0/928458320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947413cfc0 0x7f947413b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.188+0000 7f947a000640 1 -- 172.21.15.161:0/928458320 >> 172.21.15.161:0/928458320 conn(0x7f947406a2d0 msgr2=0x7f94740aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:59.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.188+0000 7f947a000640 1 -- 172.21.15.161:0/928458320 shutdown_connections 2024-05-22T21:16:59.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.188+0000 7f947a000640 1 -- 172.21.15.161:0/928458320 wait complete. 2024-05-22T21:16:59.209 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:16:59.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb84d400640 1 Processor -- start 2024-05-22T21:16:59.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb84d400640 1 -- start start 2024-05-22T21:16:59.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb84d400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb8481402e0 0x7fb84813f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb84d400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb84813fc80 0x7fb84806fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb84d400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8481400b0 con 0x7fb84813cfc0 2024-05-22T21:16:59.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb84d400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb84813d3a0 con 0x7fb8481402e0 2024-05-22T21:16:59.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb84d400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8480cd920 con 0x7fb84813fc80 2024-05-22T21:16:59.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb846800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb84813cfc0 0x7fb84815af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58616/0 (socket says 172.21.15.161:58616) 2024-05-22T21:16:59.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb846800640 1 -- 172.21.15.161:0/557210207 learned_addr learned my addr 172.21.15.161:0/557210207 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:16:59.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb847200640 1 --2- 172.21.15.161:0/557210207 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb84813fc80 0x7fb84806fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb845e00640 1 --2- 172.21.15.161:0/557210207 >> v2:172.21.15.179:3300/0 conn(0x7fb8481402e0 0x7fb84813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb845400640 1 -- 172.21.15.161:0/557210207 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 576603327 0 0) 0x7fb8481400b0 con 0x7fb84813cfc0 2024-05-22T21:16:59.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb845400640 1 -- 172.21.15.161:0/557210207 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb82c003530 con 0x7fb84813cfc0 2024-05-22T21:16:59.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb845e00640 1 -- 172.21.15.161:0/557210207 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb84813fc80 msgr2=0x7fb84806fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:59.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb845e00640 1 --2- 172.21.15.161:0/557210207 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb84813fc80 0x7fb84806fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb845400640 1 -- 172.21.15.161:0/557210207 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2365892159 0 0) 0x7fb82c003530 con 0x7fb84813cfc0 2024-05-22T21:16:59.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb845e00640 1 -- 172.21.15.161:0/557210207 >> v1:172.21.15.161:6789/0 conn(0x7fb84813cfc0 legacy=0x7fb84815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.344+0000 7fb845e00640 1 -- 172.21.15.161:0/557210207 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8480cdc00 con 0x7fb8481402e0 2024-05-22T21:16:59.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.348+0000 7fb845e00640 1 --2- 172.21.15.161:0/557210207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8481402e0 0x7fb84813f740 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb8340097c0 tx=0x7fb83401c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebc934de9e3ae7cf server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:59.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.348+0000 7fb845400640 1 -- 172.21.15.161:0/557210207 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb838001680 con 0x7fb8481402e0 2024-05-22T21:16:59.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.348+0000 7fb845400640 1 -- 172.21.15.161:0/557210207 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8340a4c20 con 0x7fb8481402e0 2024-05-22T21:16:59.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.348+0000 7fb845400640 1 -- 172.21.15.161:0/557210207 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8340ac440 con 0x7fb8481402e0 2024-05-22T21:16:59.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.348+0000 7fb84d400640 1 -- 172.21.15.161:0/557210207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8481402e0 msgr2=0x7fb84813f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.348+0000 7fb84d400640 1 --2- 172.21.15.161:0/557210207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8481402e0 0x7fb84813f740 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb8340097c0 tx=0x7fb83401c300 comp rx=0 tx=0).stop 2024-05-22T21:16:59.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.348+0000 7fb84d400640 1 -- 172.21.15.161:0/557210207 shutdown_connections 2024-05-22T21:16:59.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.348+0000 7fb84d400640 1 --2- 172.21.15.161:0/557210207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8481402e0 0x7fb84813f740 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.348+0000 7fb84d400640 1 --2- 172.21.15.161:0/557210207 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb84813fc80 0x7fb84806fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.348+0000 7fb84d400640 1 -- 172.21.15.161:0/557210207 >> 172.21.15.161:0/557210207 conn(0x7fb84806a2d0 msgr2=0x7fb84806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:59.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.348+0000 7fb84d400640 1 -- 172.21.15.161:0/557210207 shutdown_connections 2024-05-22T21:16:59.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.348+0000 7fb84d400640 1 -- 172.21.15.161:0/557210207 wait complete. 2024-05-22T21:16:59.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb84d400640 1 Processor -- start 2024-05-22T21:16:59.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb84d400640 1 -- start start 2024-05-22T21:16:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb84d400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb84813cfc0 0x7fb8481579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb84d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb84813fc80 0x7fb84813c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb84d400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8481402e0 0x7fb8481571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb84d400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8481f6480 con 0x7fb84813fc80 2024-05-22T21:16:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb84d400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8481f6720 con 0x7fb84813cfc0 2024-05-22T21:16:59.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb84d400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8481f6a00 con 0x7fb8481402e0 2024-05-22T21:16:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb845e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb84813fc80 0x7fb84813c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb845e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb84813fc80 0x7fb84813c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53674/0 (socket says 172.21.15.161:53674) 2024-05-22T21:16:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb845e00640 1 -- 172.21.15.161:0/1029472564 learned_addr learned my addr 172.21.15.161:0/1029472564 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:16:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb846800640 1 --2- 172.21.15.161:0/1029472564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb84813cfc0 0x7fb8481579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb847200640 1 --2- 172.21.15.161:0/1029472564 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8481402e0 0x7fb8481571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb845e00640 1 -- 172.21.15.161:0/1029472564 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8481402e0 msgr2=0x7fb8481571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb845e00640 1 --2- 172.21.15.161:0/1029472564 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8481402e0 0x7fb8481571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb845e00640 1 -- 172.21.15.161:0/1029472564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb84813cfc0 msgr2=0x7fb8481579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb845e00640 1 --2- 172.21.15.161:0/1029472564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb84813cfc0 0x7fb8481579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb845e00640 1 -- 172.21.15.161:0/1029472564 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb834009420 con 0x7fb84813fc80 2024-05-22T21:16:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb847200640 1 --2- 172.21.15.161:0/1029472564 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8481402e0 0x7fb8481571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:16:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb845e00640 1 --2- 172.21.15.161:0/1029472564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb84813fc80 0x7fb84813c300 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb83401a9b0 tx=0x7fb8340190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:59.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb83ea00640 1 -- 172.21.15.161:0/1029472564 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8340b5410 con 0x7fb84813fc80 2024-05-22T21:16:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb83ea00640 1 -- 172.21.15.161:0/1029472564 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8340b5590 con 0x7fb84813fc80 2024-05-22T21:16:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb83ea00640 1 -- 172.21.15.161:0/1029472564 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8340b5940 con 0x7fb84813fc80 2024-05-22T21:16:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb84d400640 1 -- 172.21.15.161:0/1029472564 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8481f6ce0 con 0x7fb84813fc80 2024-05-22T21:16:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb84d400640 1 -- 172.21.15.161:0/1029472564 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84813c840 con 0x7fb84813fc80 2024-05-22T21:16:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.352+0000 7fb84d400640 1 -- 172.21.15.161:0/1029472564 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8481f6e70 con 0x7fb84813fc80 2024-05-22T21:16:59.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.356+0000 7fb83ea00640 1 -- 172.21.15.161:0/1029472564 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb8340b3040 con 0x7fb84813fc80 2024-05-22T21:16:59.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.356+0000 7fb83ea00640 1 --2- 172.21.15.161:0/1029472564 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb828066690 0x7fb828068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.356+0000 7fb83ea00640 1 -- 172.21.15.161:0/1029472564 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb83411e200 con 0x7fb84813fc80 2024-05-22T21:16:59.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.356+0000 7fb846800640 1 --2- 172.21.15.161:0/1029472564 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb828066690 0x7fb828068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.356+0000 7fb83ea00640 1 -- 172.21.15.161:0/1029472564 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7fb82806e1c0 con 0x7fb84813fc80 2024-05-22T21:16:59.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.356+0000 7fb846800640 1 --2- 172.21.15.161:0/1029472564 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb828066690 0x7fb828068b50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb838001ba0 tx=0x7fb838009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:59.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.360+0000 7fb83ea00640 1 -- 172.21.15.161:0/1029472564 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb8340eb4d0 con 0x7fb84813fc80 2024-05-22T21:16:59.508 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:16:59.504+0000 7f7d35a00640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2024-05-22T21:16:59.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.512+0000 7fb83ea00640 1 -- 172.21.15.161:0/1029472564 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(70..70 src has 1..70) v4 ==== 296+0+0 (secure 0 0 0) 0x7fb8340ef890 con 0x7fb84813fc80 2024-05-22T21:16:59.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.512+0000 7fb83ea00640 1 -- 172.21.15.161:0/1029472564 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=71}) v3 -- 0x7fb82806ea70 con 0x7fb84813fc80 2024-05-22T21:16:59.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.528+0000 7fb84d400640 1 -- 172.21.15.161:0/1029472564 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb848067220 con 0x7fb84813fc80 2024-05-22T21:16:59.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.528+0000 7fb83ea00640 1 -- 172.21.15.161:0/1029472564 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb8340b1020 con 0x7fb84813fc80 2024-05-22T21:16:59.532 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:16:59.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.532+0000 7fb84d400640 1 -- 172.21.15.161:0/1029472564 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb828066690 msgr2=0x7fb828068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.532+0000 7fb84d400640 1 --2- 172.21.15.161:0/1029472564 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb828066690 0x7fb828068b50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb838001ba0 tx=0x7fb838009040 comp rx=0 tx=0).stop 2024-05-22T21:16:59.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.532+0000 7fb84d400640 1 -- 172.21.15.161:0/1029472564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb84813fc80 msgr2=0x7fb84813c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.532+0000 7fb84d400640 1 --2- 172.21.15.161:0/1029472564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb84813fc80 0x7fb84813c300 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb83401a9b0 tx=0x7fb8340190a0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.532+0000 7fb84d400640 1 -- 172.21.15.161:0/1029472564 shutdown_connections 2024-05-22T21:16:59.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.532+0000 7fb84d400640 1 --2- 172.21.15.161:0/1029472564 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb828066690 0x7fb828068b50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.532+0000 7fb84d400640 1 --2- 172.21.15.161:0/1029472564 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8481402e0 0x7fb8481571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.532+0000 7fb84d400640 1 --2- 172.21.15.161:0/1029472564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb84813fc80 0x7fb84813c300 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.532+0000 7fb84d400640 1 --2- 172.21.15.161:0/1029472564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb84813cfc0 0x7fb8481579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.532+0000 7fb84d400640 1 -- 172.21.15.161:0/1029472564 >> 172.21.15.161:0/1029472564 conn(0x7fb84806a2d0 msgr2=0x7fb848158a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:59.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.532+0000 7fb84d400640 1 -- 172.21.15.161:0/1029472564 shutdown_connections 2024-05-22T21:16:59.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.532+0000 7fb84d400640 1 -- 172.21.15.161:0/1029472564 wait complete. 2024-05-22T21:16:59.553 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-22T21:16:59.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:16:59.553 DEBUG:teuthology.orchestra.run.smithi179:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-22T21:16:59.581 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:16:59.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:16:59.581 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-22T21:16:59.635 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:16:59.635 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:16:59.644 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:16:59.644 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:16:59.644 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:16:59.686 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:16:59.687 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:16:59.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.816+0000 7f3452600640 1 Processor -- start 2024-05-22T21:16:59.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.816+0000 7f3452600640 1 -- start start 2024-05-22T21:16:59.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f3452600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f344c0715a0 0x7f344c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f3452600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f344c1442a0 con 0x7f344c143ec0 2024-05-22T21:16:59.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f3452600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f344c070500 con 0x7f344c15ec90 2024-05-22T21:16:59.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f3452600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0707e0 con 0x7f344c0715a0 2024-05-22T21:16:59.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344b400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f344c143ec0 0x7f344c06fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58622/0 (socket says 172.21.15.161:58622) 2024-05-22T21:16:59.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344b400640 1 -- 172.21.15.161:0/3325750365 learned_addr learned my addr 172.21.15.161:0/3325750365 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:16:59.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344aa00640 1 --2- 172.21.15.161:0/3325750365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f344c0715a0 0x7f344c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344a000640 1 -- 172.21.15.161:0/3325750365 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296725798 0 0) 0x7f344c1442a0 con 0x7f344c143ec0 2024-05-22T21:16:59.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344a000640 1 -- 172.21.15.161:0/3325750365 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3430003530 con 0x7f344c143ec0 2024-05-22T21:16:59.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344a000640 1 -- 172.21.15.161:0/3325750365 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3758498527 0 0) 0x7f3430003530 con 0x7f344c143ec0 2024-05-22T21:16:59.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344a000640 1 -- 172.21.15.161:0/3325750365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f344c0715a0 msgr2=0x7f344c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:59.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344a000640 1 --2- 172.21.15.161:0/3325750365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f344c0715a0 0x7f344c15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344a000640 1 -- 172.21.15.161:0/3325750365 >> v1:172.21.15.179:6789/0 conn(0x7f344c15ec90 legacy=0x7f344c0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344a000640 1 -- 172.21.15.161:0/3325750365 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c070ac0 con 0x7f344c143ec0 2024-05-22T21:16:59.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344a000640 1 -- 172.21.15.161:0/3325750365 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f343c001680 con 0x7f344c143ec0 2024-05-22T21:16:59.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344aa00640 1 --2- 172.21.15.161:0/3325750365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f344c0715a0 0x7f344c15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:16:59.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344a000640 1 -- 172.21.15.161:0/3325750365 >> v1:172.21.15.161:6789/0 conn(0x7f344c143ec0 legacy=0x7f344c06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344a000640 1 --2- 172.21.15.161:0/3325750365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34300039b0 0x7f3430003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344a000640 1 --2- 172.21.15.161:0/3325750365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34300047d0 0x7f3430004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344a000640 1 --2- 172.21.15.161:0/3325750365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3430005190 0x7f343000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344a000640 1 -- 172.21.15.161:0/3325750365 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0707e0 con 0x7f3430005190 2024-05-22T21:16:59.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344a000640 1 -- 172.21.15.161:0/3325750365 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343000c060 con 0x7f34300047d0 2024-05-22T21:16:59.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344a000640 1 -- 172.21.15.161:0/3325750365 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343000c220 con 0x7f34300039b0 2024-05-22T21:16:59.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344be00640 1 --2- 172.21.15.161:0/3325750365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3430005190 0x7f343000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344aa00640 1 --2- 172.21.15.161:0/3325750365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34300047d0 0x7f3430004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344b400640 1 --2- 172.21.15.161:0/3325750365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34300039b0 0x7f3430003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344be00640 1 -- 172.21.15.161:0/3325750365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34300039b0 msgr2=0x7f3430003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:59.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344be00640 1 --2- 172.21.15.161:0/3325750365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34300039b0 0x7f3430003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344be00640 1 -- 172.21.15.161:0/3325750365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34300047d0 msgr2=0x7f3430004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:16:59.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344be00640 1 --2- 172.21.15.161:0/3325750365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34300047d0 0x7f3430004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.820+0000 7f344be00640 1 -- 172.21.15.161:0/3325750365 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3430003530 con 0x7f3430005190 2024-05-22T21:16:59.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.824+0000 7f344be00640 1 --2- 172.21.15.161:0/3325750365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3430005190 0x7f343000b990 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3444001e70 tx=0x7f344401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0d8e901fe29b88 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:59.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.824+0000 7f344a000640 1 -- 172.21.15.161:0/3325750365 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444003b80 con 0x7f3430005190 2024-05-22T21:16:59.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.824+0000 7f344a000640 1 -- 172.21.15.161:0/3325750365 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3444003ce0 con 0x7f3430005190 2024-05-22T21:16:59.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.824+0000 7f344a000640 1 -- 172.21.15.161:0/3325750365 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34440052c0 con 0x7f3430005190 2024-05-22T21:16:59.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.824+0000 7f3452600640 1 -- 172.21.15.161:0/3325750365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3430005190 msgr2=0x7f343000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.824+0000 7f3452600640 1 --2- 172.21.15.161:0/3325750365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3430005190 0x7f343000b990 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3444001e70 tx=0x7f344401c780 comp rx=0 tx=0).stop 2024-05-22T21:16:59.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.824+0000 7f344be00640 1 -- 172.21.15.161:0/3325750365 reap_dead start 2024-05-22T21:16:59.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.824+0000 7f3452600640 1 -- 172.21.15.161:0/3325750365 shutdown_connections 2024-05-22T21:16:59.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.824+0000 7f3452600640 1 -- 172.21.15.161:0/3325750365 >> 172.21.15.161:0/3325750365 conn(0x7f344c0b0bc0 msgr2=0x7f344c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:16:59.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.824+0000 7f3452600640 1 -- 172.21.15.161:0/3325750365 shutdown_connections 2024-05-22T21:16:59.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.824+0000 7f3452600640 1 -- 172.21.15.161:0/3325750365 wait complete. 2024-05-22T21:16:59.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.824+0000 7f3452600640 1 Processor -- start 2024-05-22T21:16:59.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f3452600640 1 -- start start 2024-05-22T21:16:59.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f3452600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3430005190 0x7f344c0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f3452600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f344c15e0b0 0x7f344c15e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f3452600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f344c0af120 0x7f344c0af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f3452600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0b00c0 con 0x7f344c0af120 2024-05-22T21:16:59.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f3452600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c15ea70 con 0x7f344c15e0b0 2024-05-22T21:16:59.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f3452600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0afde0 con 0x7f3430005190 2024-05-22T21:16:59.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f344be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f344c0af120 0x7f344c0af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f344aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f344c15e0b0 0x7f344c15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f344b400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3430005190 0x7f344c0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f344aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f344c15e0b0 0x7f344c15e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:48852/0 (socket says 172.21.15.161:48852) 2024-05-22T21:16:59.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f344aa00640 1 -- 172.21.15.161:0/3778041450 learned_addr learned my addr 172.21.15.161:0/3778041450 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:16:59.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f344be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f344c0af120 0x7f344c0af5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53702/0 (socket says 172.21.15.161:53702) 2024-05-22T21:16:59.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f344be00640 1 -- 172.21.15.161:0/3778041450 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3430005190 msgr2=0x7f344c0cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f344be00640 1 --2- 172.21.15.161:0/3778041450 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3430005190 0x7f344c0cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f344be00640 1 -- 172.21.15.161:0/3778041450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f344c15e0b0 msgr2=0x7f344c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:16:59.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f344be00640 1 --2- 172.21.15.161:0/3778041450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f344c15e0b0 0x7f344c15e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:16:59.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f344be00640 1 -- 172.21.15.161:0/3778041450 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c14fc20 con 0x7f344c0af120 2024-05-22T21:16:59.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f344be00640 1 --2- 172.21.15.161:0/3778041450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f344c0af120 0x7f344c0af5a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3444009000 tx=0x7f3430005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:59.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f3443400640 1 -- 172.21.15.161:0/3778041450 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444003b80 con 0x7f344c0af120 2024-05-22T21:16:59.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f3443400640 1 -- 172.21.15.161:0/3778041450 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3444005820 con 0x7f344c0af120 2024-05-22T21:16:59.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f3443400640 1 -- 172.21.15.161:0/3778041450 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444006da0 con 0x7f344c0af120 2024-05-22T21:16:59.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f3452600640 1 -- 172.21.15.161:0/3778041450 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344c0b00c0 con 0x7f344c0af120 2024-05-22T21:16:59.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f3452600640 1 -- 172.21.15.161:0/3778041450 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c1501e0 con 0x7f344c0af120 2024-05-22T21:16:59.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.828+0000 7f3452600640 1 -- 172.21.15.161:0/3778041450 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344c1503d0 con 0x7f344c0af120 2024-05-22T21:16:59.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.832+0000 7f3443400640 1 -- 172.21.15.161:0/3778041450 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f34440a6480 con 0x7f344c0af120 2024-05-22T21:16:59.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.832+0000 7f3443400640 1 --2- 172.21.15.161:0/3778041450 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f342c066690 0x7f342c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:16:59.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.832+0000 7f3443400640 1 -- 172.21.15.161:0/3778041450 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f344411d930 con 0x7f344c0af120 2024-05-22T21:16:59.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.832+0000 7f344b400640 1 --2- 172.21.15.161:0/3778041450 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f342c066690 0x7f342c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:16:59.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.832+0000 7f3443400640 1 -- 172.21.15.161:0/3778041450 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=71}) v3 -- 0x7f342c06e1c0 con 0x7f344c0af120 2024-05-22T21:16:59.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.832+0000 7f344b400640 1 --2- 172.21.15.161:0/3778041450 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f342c066690 0x7f342c068b50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f343c001ca0 tx=0x7f343c007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:16:59.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:16:59.836+0000 7f3443400640 1 -- 172.21.15.161:0/3778041450 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f34440ba020 con 0x7f344c0af120 2024-05-22T21:17:00.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.024+0000 7f3452600640 1 -- 172.21.15.161:0/3778041450 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f344c067220 con 0x7f344c0af120 2024-05-22T21:17:00.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.024+0000 7f3443400640 1 -- 172.21.15.161:0/3778041450 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f34440eac20 con 0x7f344c0af120 2024-05-22T21:17:00.026 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:00.027 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1620021722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":1620021722},{"type":"v1","addr":"172.21.15.179:6835","nonce":1620021722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/3548738614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3548738614},{"type":"v1","addr":"172.21.15.161:6836","nonce":3548738614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/400898801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":400898801},{"type":"v1","addr":"172.21.15.161:6837","nonce":400898801}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:15:42.955245+0000","modified":"2024-05-22T21:15:45.977718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6834/2239627963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2239627963},{"type":"v1","addr":"172.21.15.179:6834","nonce":2239627963}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-22T21:17:00.027 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 31 2024-05-22T21:17:00.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.024+0000 7f3452600640 1 -- 172.21.15.161:0/3778041450 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f342c066690 msgr2=0x7f342c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.024+0000 7f3452600640 1 --2- 172.21.15.161:0/3778041450 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f342c066690 0x7f342c068b50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f343c001ca0 tx=0x7f343c007040 comp rx=0 tx=0).stop 2024-05-22T21:17:00.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.024+0000 7f3452600640 1 -- 172.21.15.161:0/3778041450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f344c0af120 msgr2=0x7f344c0af5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.024+0000 7f3452600640 1 --2- 172.21.15.161:0/3778041450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f344c0af120 0x7f344c0af5a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3444009000 tx=0x7f3430005800 comp rx=0 tx=0).stop 2024-05-22T21:17:00.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.024+0000 7f3452600640 1 -- 172.21.15.161:0/3778041450 shutdown_connections 2024-05-22T21:17:00.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.024+0000 7f3452600640 1 --2- 172.21.15.161:0/3778041450 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f342c066690 0x7f342c068b50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.024+0000 7f3452600640 1 --2- 172.21.15.161:0/3778041450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f344c0af120 0x7f344c0af5a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.024+0000 7f3452600640 1 --2- 172.21.15.161:0/3778041450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f344c15e0b0 0x7f344c15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.024+0000 7f3452600640 1 --2- 172.21.15.161:0/3778041450 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3430005190 0x7f344c0cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.024+0000 7f3452600640 1 -- 172.21.15.161:0/3778041450 >> 172.21.15.161:0/3778041450 conn(0x7f344c0b0bc0 msgr2=0x7f344c06bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:00.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.024+0000 7f3452600640 1 -- 172.21.15.161:0/3778041450 shutdown_connections 2024-05-22T21:17:00.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.024+0000 7f3452600640 1 -- 172.21.15.161:0/3778041450 wait complete. 2024-05-22T21:17:00.048 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:17:00.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.180+0000 7fbb70400640 1 Processor -- start 2024-05-22T21:17:00.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.180+0000 7fbb70400640 1 -- start start 2024-05-22T21:17:00.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.180+0000 7fbb70400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb680b4fa0 0x7fbb6815e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.180+0000 7fbb70400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbb6806a5e0 0x7fbb681613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.180+0000 7fbb70400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6815e000 con 0x7fbb68160f70 2024-05-22T21:17:00.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.180+0000 7fbb70400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6815e9d0 con 0x7fbb6806a5e0 2024-05-22T21:17:00.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.180+0000 7fbb70400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb68161a30 con 0x7fbb680b4fa0 2024-05-22T21:17:00.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.180+0000 7fbb6e400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbb6806a5e0 0x7fbb681613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.180+0000 7fbb6e400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbb6806a5e0 0x7fbb681613f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53706/0 (socket says 172.21.15.161:53706) 2024-05-22T21:17:00.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.180+0000 7fbb6e400640 1 -- 172.21.15.161:0/1278481549 learned_addr learned my addr 172.21.15.161:0/1278481549 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:00.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.180+0000 7fbb6da00640 1 --2- 172.21.15.161:0/1278481549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb680b4fa0 0x7fbb6815e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.180+0000 7fbb6e400640 1 -- 172.21.15.161:0/1278481549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb680b4fa0 msgr2=0x7fbb6815e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:00.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.180+0000 7fbb6e400640 1 --2- 172.21.15.161:0/1278481549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb680b4fa0 0x7fbb6815e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.180+0000 7fbb6e400640 1 -- 172.21.15.161:0/1278481549 >> v1:172.21.15.179:6789/0 conn(0x7fbb68160f70 legacy=0x7fbb68069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.180+0000 7fbb6e400640 1 -- 172.21.15.161:0/1278481549 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb68161d10 con 0x7fbb6806a5e0 2024-05-22T21:17:00.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.180+0000 7fbb6da00640 1 --2- 172.21.15.161:0/1278481549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb680b4fa0 0x7fbb6815e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:00.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.184+0000 7fbb6e400640 1 --2- 172.21.15.161:0/1278481549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb6806a5e0 0x7fbb681613f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009e50 tx=0x7fbb6001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd1f901672183fd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:00.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.184+0000 7fbb67e00640 1 -- 172.21.15.161:0/1278481549 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb600a5870 con 0x7fbb6806a5e0 2024-05-22T21:17:00.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.184+0000 7fbb67e00640 1 -- 172.21.15.161:0/1278481549 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb600a59d0 con 0x7fbb6806a5e0 2024-05-22T21:17:00.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.184+0000 7fbb67e00640 1 -- 172.21.15.161:0/1278481549 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb600a5d60 con 0x7fbb6806a5e0 2024-05-22T21:17:00.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.184+0000 7fbb70400640 1 -- 172.21.15.161:0/1278481549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb6806a5e0 msgr2=0x7fbb681613f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.184+0000 7fbb70400640 1 --2- 172.21.15.161:0/1278481549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb6806a5e0 0x7fbb681613f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009e50 tx=0x7fbb6001ca00 comp rx=0 tx=0).stop 2024-05-22T21:17:00.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.184+0000 7fbb70400640 1 -- 172.21.15.161:0/1278481549 shutdown_connections 2024-05-22T21:17:00.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.184+0000 7fbb70400640 1 --2- 172.21.15.161:0/1278481549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb6806a5e0 0x7fbb681613f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.184+0000 7fbb70400640 1 --2- 172.21.15.161:0/1278481549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb680b4fa0 0x7fbb6815e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.184+0000 7fbb70400640 1 -- 172.21.15.161:0/1278481549 >> 172.21.15.161:0/1278481549 conn(0x7fbb680b1c40 msgr2=0x7fbb680b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:00.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.184+0000 7fbb70400640 1 -- 172.21.15.161:0/1278481549 shutdown_connections 2024-05-22T21:17:00.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.184+0000 7fbb70400640 1 -- 172.21.15.161:0/1278481549 wait complete. 2024-05-22T21:17:00.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb70400640 1 Processor -- start 2024-05-22T21:17:00.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb70400640 1 -- start start 2024-05-22T21:17:00.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb70400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb6806a5e0 0x7fbb6806e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb70400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb680b4fa0 0x7fbb6815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb70400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb68160f70 0x7fbb680adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb70400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb680ada30 con 0x7fbb68160f70 2024-05-22T21:17:00.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb70400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb680ace20 con 0x7fbb6806a5e0 2024-05-22T21:17:00.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb70400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb680ad100 con 0x7fbb680b4fa0 2024-05-22T21:17:00.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb6e400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb68160f70 0x7fbb680adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb6da00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb6806a5e0 0x7fbb6806e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb6e400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb68160f70 0x7fbb680adec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53716/0 (socket says 172.21.15.161:53716) 2024-05-22T21:17:00.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb6da00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb6806a5e0 0x7fbb6806e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:48864/0 (socket says 172.21.15.161:48864) 2024-05-22T21:17:00.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb6e400640 1 -- 172.21.15.161:0/421973963 learned_addr learned my addr 172.21.15.161:0/421973963 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:00.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb6d000640 1 --2- 172.21.15.161:0/421973963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb680b4fa0 0x7fbb6815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb6e400640 1 -- 172.21.15.161:0/421973963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb680b4fa0 msgr2=0x7fbb6815ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb6e400640 1 --2- 172.21.15.161:0/421973963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb680b4fa0 0x7fbb6815ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb6e400640 1 -- 172.21.15.161:0/421973963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb6806a5e0 msgr2=0x7fbb6806e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb6e400640 1 --2- 172.21.15.161:0/421973963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb6806a5e0 0x7fbb6806e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb6e400640 1 -- 172.21.15.161:0/421973963 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb60009b30 con 0x7fbb68160f70 2024-05-22T21:17:00.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb6e400640 1 --2- 172.21.15.161:0/421973963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb68160f70 0x7fbb680adec0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbb600af040 tx=0x7fbb600197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:00.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb65600640 1 -- 172.21.15.161:0/421973963 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb600b8670 con 0x7fbb68160f70 2024-05-22T21:17:00.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb65600640 1 -- 172.21.15.161:0/421973963 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb600b87f0 con 0x7fbb68160f70 2024-05-22T21:17:00.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb70400640 1 -- 172.21.15.161:0/421973963 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb680ad3e0 con 0x7fbb68160f70 2024-05-22T21:17:00.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb65600640 1 -- 172.21.15.161:0/421973963 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb600b8ba0 con 0x7fbb68160f70 2024-05-22T21:17:00.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb70400640 1 -- 172.21.15.161:0/421973963 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6806ec30 con 0x7fbb68160f70 2024-05-22T21:17:00.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.188+0000 7fbb70400640 1 -- 172.21.15.161:0/421973963 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb680adba0 con 0x7fbb68160f70 2024-05-22T21:17:00.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.192+0000 7fbb65600640 1 -- 172.21.15.161:0/421973963 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbb600b8d00 con 0x7fbb68160f70 2024-05-22T21:17:00.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.192+0000 7fbb65600640 1 --2- 172.21.15.161:0/421973963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb48066690 0x7fbb48068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.192+0000 7fbb65600640 1 -- 172.21.15.161:0/421973963 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb6011f730 con 0x7fbb68160f70 2024-05-22T21:17:00.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.192+0000 7fbb6da00640 1 --2- 172.21.15.161:0/421973963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb48066690 0x7fbb48068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.192+0000 7fbb65600640 1 -- 172.21.15.161:0/421973963 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=71}) v3 -- 0x7fbb4806e1c0 con 0x7fbb68160f70 2024-05-22T21:17:00.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.192+0000 7fbb6da00640 1 --2- 172.21.15.161:0/421973963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb48066690 0x7fbb48068b50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009950 tx=0x7fbb5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:00.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.196+0000 7fbb65600640 1 -- 172.21.15.161:0/421973963 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbb600ecbd0 con 0x7fbb68160f70 2024-05-22T21:17:00.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.352+0000 7fbb70400640 1 -- 172.21.15.161:0/421973963 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb680ad240 con 0x7fbb68160f70 2024-05-22T21:17:00.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.352+0000 7fbb65600640 1 -- 172.21.15.161:0/421973963 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16196 (secure 0 0 0) 0x7fbb680ad240 con 0x7fbb68160f70 2024-05-22T21:17:00.355 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:00.355 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":70,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:16:59.508643+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:15:40.613692+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-22T21:15:41.954314+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:17:00.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.352+0000 7fbb70400640 1 -- 172.21.15.161:0/421973963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb48066690 msgr2=0x7fbb48068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.352+0000 7fbb70400640 1 --2- 172.21.15.161:0/421973963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb48066690 0x7fbb48068b50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009950 tx=0x7fbb5c008040 comp rx=0 tx=0).stop 2024-05-22T21:17:00.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.352+0000 7fbb70400640 1 -- 172.21.15.161:0/421973963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb68160f70 msgr2=0x7fbb680adec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.352+0000 7fbb70400640 1 --2- 172.21.15.161:0/421973963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb68160f70 0x7fbb680adec0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbb600af040 tx=0x7fbb600197a0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.352+0000 7fbb70400640 1 -- 172.21.15.161:0/421973963 shutdown_connections 2024-05-22T21:17:00.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.352+0000 7fbb70400640 1 --2- 172.21.15.161:0/421973963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb48066690 0x7fbb48068b50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.352+0000 7fbb70400640 1 --2- 172.21.15.161:0/421973963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb68160f70 0x7fbb680adec0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.352+0000 7fbb70400640 1 --2- 172.21.15.161:0/421973963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb680b4fa0 0x7fbb6815ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.352+0000 7fbb70400640 1 --2- 172.21.15.161:0/421973963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb6806a5e0 0x7fbb6806e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.352+0000 7fbb70400640 1 -- 172.21.15.161:0/421973963 >> 172.21.15.161:0/421973963 conn(0x7fbb680b1c40 msgr2=0x7fbb68069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:00.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.352+0000 7fbb70400640 1 -- 172.21.15.161:0/421973963 shutdown_connections 2024-05-22T21:17:00.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.352+0000 7fbb70400640 1 -- 172.21.15.161:0/421973963 wait complete. 2024-05-22T21:17:00.376 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:15:40.613692+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:15:41.954314+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:17:00.376 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:17:00.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.512+0000 7f25b2800640 1 Processor -- start 2024-05-22T21:17:00.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.512+0000 7f25b2800640 1 -- start start 2024-05-22T21:17:00.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.516+0000 7f25b2800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f25ac0b3450 0x7f25ac0ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.516+0000 7f25b2800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ac0ce9b0 con 0x7f25ac0cc1b0 2024-05-22T21:17:00.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.516+0000 7f25b2800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ac0cba20 con 0x7f25ac15b150 2024-05-22T21:17:00.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.516+0000 7f25b2800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ac06e5a0 con 0x7f25ac0b3450 2024-05-22T21:17:00.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.516+0000 7f25b0a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f25ac0b3450 0x7f25ac0ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.516+0000 7f25b0a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f25ac0b3450 0x7f25ac0ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53722/0 (socket says 172.21.15.161:53722) 2024-05-22T21:17:00.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.516+0000 7f25b0a00640 1 -- 172.21.15.161:0/345875972 learned_addr learned my addr 172.21.15.161:0/345875972 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:00.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.520+0000 7f25b0a00640 1 -- 172.21.15.161:0/345875972 >> v1:172.21.15.179:6789/0 conn(0x7f25ac15b150 legacy=0x7f25ac06de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.520+0000 7f25b0a00640 1 -- 172.21.15.161:0/345875972 >> v1:172.21.15.161:6789/0 conn(0x7f25ac0cc1b0 legacy=0x7f25ac0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.520+0000 7f25b0a00640 1 -- 172.21.15.161:0/345875972 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac15b530 con 0x7f25ac0b3450 2024-05-22T21:17:00.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.520+0000 7f25b0a00640 1 --2- 172.21.15.161:0/345875972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25ac0b3450 0x7f25ac0ce200 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2598009870 tx=0x7f259801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30da93dff910ee6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:00.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.520+0000 7f25abe00640 1 -- 172.21.15.161:0/345875972 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980aa430 con 0x7f25ac0b3450 2024-05-22T21:17:00.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.520+0000 7f25abe00640 1 -- 172.21.15.161:0/345875972 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25980aa590 con 0x7f25ac0b3450 2024-05-22T21:17:00.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.520+0000 7f25abe00640 1 -- 172.21.15.161:0/345875972 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980aa920 con 0x7f25ac0b3450 2024-05-22T21:17:00.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.520+0000 7f25b2800640 1 -- 172.21.15.161:0/345875972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25ac0b3450 msgr2=0x7f25ac0ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.520+0000 7f25b2800640 1 --2- 172.21.15.161:0/345875972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25ac0b3450 0x7f25ac0ce200 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2598009870 tx=0x7f259801c4d0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b2800640 1 -- 172.21.15.161:0/345875972 shutdown_connections 2024-05-22T21:17:00.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b2800640 1 --2- 172.21.15.161:0/345875972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25ac0b3450 0x7f25ac0ce200 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b2800640 1 -- 172.21.15.161:0/345875972 >> 172.21.15.161:0/345875972 conn(0x7f25ac0b2930 msgr2=0x7f25ac0b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:00.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b2800640 1 -- 172.21.15.161:0/345875972 shutdown_connections 2024-05-22T21:17:00.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b2800640 1 -- 172.21.15.161:0/345875972 wait complete. 2024-05-22T21:17:00.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b2800640 1 Processor -- start 2024-05-22T21:17:00.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b2800640 1 -- start start 2024-05-22T21:17:00.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b2800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac0b3450 0x7f25ac148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b2800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25ac0cc1b0 0x7f25ac149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b2800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25ac15b150 0x7f25ac14c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b2800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0c5a80 con 0x7f25ac0cc1b0 2024-05-22T21:17:00.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b2800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac148b80 con 0x7f25ac0b3450 2024-05-22T21:17:00.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b2800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0c5d20 con 0x7f25ac15b150 2024-05-22T21:17:00.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b0a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25ac0cc1b0 0x7f25ac149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b0a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25ac0cc1b0 0x7f25ac149360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53728/0 (socket says 172.21.15.161:53728) 2024-05-22T21:17:00.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b0a00640 1 -- 172.21.15.161:0/3664738981 learned_addr learned my addr 172.21.15.161:0/3664738981 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:00.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b1400640 1 --2- 172.21.15.161:0/3664738981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac0b3450 0x7f25ac148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b1e00640 1 --2- 172.21.15.161:0/3664738981 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25ac15b150 0x7f25ac14c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b0a00640 1 -- 172.21.15.161:0/3664738981 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25ac15b150 msgr2=0x7f25ac14c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b0a00640 1 --2- 172.21.15.161:0/3664738981 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25ac15b150 0x7f25ac14c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b0a00640 1 -- 172.21.15.161:0/3664738981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac0b3450 msgr2=0x7f25ac148640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b0a00640 1 --2- 172.21.15.161:0/3664738981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac0b3450 0x7f25ac148640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b0a00640 1 -- 172.21.15.161:0/3664738981 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25980094d0 con 0x7f25ac0cc1b0 2024-05-22T21:17:00.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.524+0000 7f25b0a00640 1 --2- 172.21.15.161:0/3664738981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25ac0cc1b0 0x7f25ac149360 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f25980ac040 tx=0x7f259801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:00.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.528+0000 7f25a9600640 1 -- 172.21.15.161:0/3664738981 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980a9960 con 0x7f25ac0cc1b0 2024-05-22T21:17:00.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.528+0000 7f25a9600640 1 -- 172.21.15.161:0/3664738981 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25980a9ae0 con 0x7f25ac0cc1b0 2024-05-22T21:17:00.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.528+0000 7f25b2800640 1 -- 172.21.15.161:0/3664738981 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ac0c6000 con 0x7f25ac0cc1b0 2024-05-22T21:17:00.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.528+0000 7f25a9600640 1 -- 172.21.15.161:0/3664738981 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980a9e90 con 0x7f25ac0cc1b0 2024-05-22T21:17:00.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.528+0000 7f25b2800640 1 -- 172.21.15.161:0/3664738981 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ac14a680 con 0x7f25ac0cc1b0 2024-05-22T21:17:00.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.528+0000 7f25b2800640 1 -- 172.21.15.161:0/3664738981 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ac14a870 con 0x7f25ac0cc1b0 2024-05-22T21:17:00.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.528+0000 7f25a9600640 1 -- 172.21.15.161:0/3664738981 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f25980c6440 con 0x7f25ac0cc1b0 2024-05-22T21:17:00.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.528+0000 7f25a9600640 1 --2- 172.21.15.161:0/3664738981 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f258c066690 0x7f258c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.528+0000 7f25a9600640 1 -- 172.21.15.161:0/3664738981 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f259811eda0 con 0x7f25ac0cc1b0 2024-05-22T21:17:00.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.528+0000 7f25b1400640 1 --2- 172.21.15.161:0/3664738981 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f258c066690 0x7f258c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.528+0000 7f25a9600640 1 -- 172.21.15.161:0/3664738981 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7f258c06e1c0 con 0x7f25ac0cc1b0 2024-05-22T21:17:00.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.532+0000 7f25b1400640 1 --2- 172.21.15.161:0/3664738981 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f258c066690 0x7f258c068b50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f25a0007fd0 tx=0x7f25a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:00.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.532+0000 7f25a9600640 1 -- 172.21.15.161:0/3664738981 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f25980ec250 con 0x7f25ac0cc1b0 2024-05-22T21:17:00.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.688+0000 7f25b2800640 1 -- 172.21.15.161:0/3664738981 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25ac0c64b0 con 0x7f25ac0cc1b0 2024-05-22T21:17:00.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.688+0000 7f25a9600640 1 -- 172.21.15.161:0/3664738981 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25980ef870 con 0x7f25ac0cc1b0 2024-05-22T21:17:00.690 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:17:00.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.688+0000 7f25b2800640 1 -- 172.21.15.161:0/3664738981 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f258c066690 msgr2=0x7f258c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.688+0000 7f25b2800640 1 --2- 172.21.15.161:0/3664738981 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f258c066690 0x7f258c068b50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f25a0007fd0 tx=0x7f25a0009040 comp rx=0 tx=0).stop 2024-05-22T21:17:00.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.688+0000 7f25b2800640 1 -- 172.21.15.161:0/3664738981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25ac0cc1b0 msgr2=0x7f25ac149360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.688+0000 7f25b2800640 1 --2- 172.21.15.161:0/3664738981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25ac0cc1b0 0x7f25ac149360 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f25980ac040 tx=0x7f259801ad10 comp rx=0 tx=0).stop 2024-05-22T21:17:00.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.688+0000 7f25b2800640 1 -- 172.21.15.161:0/3664738981 shutdown_connections 2024-05-22T21:17:00.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.688+0000 7f25b2800640 1 --2- 172.21.15.161:0/3664738981 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f258c066690 0x7f258c068b50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.688+0000 7f25b2800640 1 --2- 172.21.15.161:0/3664738981 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25ac15b150 0x7f25ac14c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.688+0000 7f25b2800640 1 --2- 172.21.15.161:0/3664738981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25ac0cc1b0 0x7f25ac149360 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.688+0000 7f25b2800640 1 --2- 172.21.15.161:0/3664738981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac0b3450 0x7f25ac148640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.688+0000 7f25b2800640 1 -- 172.21.15.161:0/3664738981 >> 172.21.15.161:0/3664738981 conn(0x7f25ac0b2930 msgr2=0x7f25ac0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:00.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.688+0000 7f25b2800640 1 -- 172.21.15.161:0/3664738981 shutdown_connections 2024-05-22T21:17:00.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.688+0000 7f25b2800640 1 -- 172.21.15.161:0/3664738981 wait complete. 2024-05-22T21:17:00.710 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:17:00.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.844+0000 7fe62aa00640 1 Processor -- start 2024-05-22T21:17:00.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.844+0000 7fe62aa00640 1 -- start start 2024-05-22T21:17:00.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe62aa00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe624143ed0 0x7fe6241442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe62aa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6241448e0 0x7fe6240b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe62aa00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6240ce830 con 0x7fe6240b2620 2024-05-22T21:17:00.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe62aa00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6240b2e60 con 0x7fe624143ed0 2024-05-22T21:17:00.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe62aa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6240b3140 con 0x7fe6241448e0 2024-05-22T21:17:00.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe62a000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe6240b2620 0x7fe6240cd360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58636/0 (socket says 172.21.15.161:58636) 2024-05-22T21:17:00.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe62a000640 1 -- 172.21.15.161:0/1260114267 learned_addr learned my addr 172.21.15.161:0/1260114267 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:00.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe629600640 1 --2- 172.21.15.161:0/1260114267 >> v2:172.21.15.179:3300/0 conn(0x7fe624143ed0 0x7fe6241442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe628c00640 1 --2- 172.21.15.161:0/1260114267 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6241448e0 0x7fe6240b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe623e00640 1 -- 172.21.15.161:0/1260114267 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26364324 0 0) 0x7fe6240ce830 con 0x7fe6240b2620 2024-05-22T21:17:00.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe623e00640 1 -- 172.21.15.161:0/1260114267 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe60c003530 con 0x7fe6240b2620 2024-05-22T21:17:00.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe623e00640 1 -- 172.21.15.161:0/1260114267 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1346841609 0 0) 0x7fe60c003530 con 0x7fe6240b2620 2024-05-22T21:17:00.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe623e00640 1 -- 172.21.15.161:0/1260114267 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6241448e0 msgr2=0x7fe6240b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:00.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe623e00640 1 --2- 172.21.15.161:0/1260114267 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6241448e0 0x7fe6240b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe623e00640 1 -- 172.21.15.161:0/1260114267 >> v2:172.21.15.179:3300/0 conn(0x7fe624143ed0 msgr2=0x7fe6241442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:00.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe623e00640 1 --2- 172.21.15.161:0/1260114267 >> v2:172.21.15.179:3300/0 conn(0x7fe624143ed0 0x7fe6241442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe623e00640 1 -- 172.21.15.161:0/1260114267 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6240b3420 con 0x7fe6240b2620 2024-05-22T21:17:00.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe623e00640 1 -- 172.21.15.161:0/1260114267 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fe61c001680 con 0x7fe6240b2620 2024-05-22T21:17:00.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe629600640 1 --2- 172.21.15.161:0/1260114267 >> v2:172.21.15.179:3300/0 conn(0x7fe624143ed0 0x7fe6241442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:17:00.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe628c00640 1 --2- 172.21.15.161:0/1260114267 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6241448e0 0x7fe6240b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:17:00.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe623e00640 1 -- 172.21.15.161:0/1260114267 >> v1:172.21.15.161:6789/0 conn(0x7fe6240b2620 legacy=0x7fe6240cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe623e00640 1 --2- 172.21.15.161:0/1260114267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe60c003980 0x7fe60c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe623e00640 1 --2- 172.21.15.161:0/1260114267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe60c0047d0 0x7fe60c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe623e00640 1 --2- 172.21.15.161:0/1260114267 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6241448e0 0x7fe60c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe623e00640 1 -- 172.21.15.161:0/1260114267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6240b2e60 con 0x7fe60c0047d0 2024-05-22T21:17:00.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe623e00640 1 -- 172.21.15.161:0/1260114267 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6240b3140 con 0x7fe60c003980 2024-05-22T21:17:00.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe623e00640 1 -- 172.21.15.161:0/1260114267 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c00b990 con 0x7fe6241448e0 2024-05-22T21:17:00.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe628c00640 1 --2- 172.21.15.161:0/1260114267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe60c0047d0 0x7fe60c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe629600640 1 --2- 172.21.15.161:0/1260114267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe60c003980 0x7fe60c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe62a000640 1 --2- 172.21.15.161:0/1260114267 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6241448e0 0x7fe60c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe628c00640 1 -- 172.21.15.161:0/1260114267 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6241448e0 msgr2=0x7fe60c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:00.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe628c00640 1 --2- 172.21.15.161:0/1260114267 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6241448e0 0x7fe60c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe628c00640 1 -- 172.21.15.161:0/1260114267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe60c003980 msgr2=0x7fe60c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:00.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe628c00640 1 --2- 172.21.15.161:0/1260114267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe60c003980 0x7fe60c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.848+0000 7fe628c00640 1 -- 172.21.15.161:0/1260114267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe610009420 con 0x7fe60c0047d0 2024-05-22T21:17:00.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.852+0000 7fe628c00640 1 --2- 172.21.15.161:0/1260114267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe60c0047d0 0x7fe60c004c50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe610009cb0 tx=0x7fe61001cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9160d61a95dda4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:00.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.852+0000 7fe623e00640 1 -- 172.21.15.161:0/1260114267 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6100a56f0 con 0x7fe60c0047d0 2024-05-22T21:17:00.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.852+0000 7fe623e00640 1 -- 172.21.15.161:0/1260114267 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6100a5850 con 0x7fe60c0047d0 2024-05-22T21:17:00.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.852+0000 7fe623e00640 1 -- 172.21.15.161:0/1260114267 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6100a5c00 con 0x7fe60c0047d0 2024-05-22T21:17:00.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.852+0000 7fe62aa00640 1 -- 172.21.15.161:0/1260114267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe60c0047d0 msgr2=0x7fe60c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.852+0000 7fe62aa00640 1 --2- 172.21.15.161:0/1260114267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe60c0047d0 0x7fe60c004c50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe610009cb0 tx=0x7fe61001cc10 comp rx=0 tx=0).stop 2024-05-22T21:17:00.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.852+0000 7fe62a000640 1 -- 172.21.15.161:0/1260114267 reap_dead start 2024-05-22T21:17:00.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.852+0000 7fe62aa00640 1 -- 172.21.15.161:0/1260114267 shutdown_connections 2024-05-22T21:17:00.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.852+0000 7fe62aa00640 1 -- 172.21.15.161:0/1260114267 >> 172.21.15.161:0/1260114267 conn(0x7fe6240cec00 msgr2=0x7fe6240cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:00.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.852+0000 7fe62aa00640 1 -- 172.21.15.161:0/1260114267 shutdown_connections 2024-05-22T21:17:00.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.852+0000 7fe62aa00640 1 -- 172.21.15.161:0/1260114267 wait complete. 2024-05-22T21:17:00.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.852+0000 7fe62aa00640 1 Processor -- start 2024-05-22T21:17:00.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.852+0000 7fe62aa00640 1 -- start start 2024-05-22T21:17:00.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe62aa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe60c0047d0 0x7fe624159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe62aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe62415a3e0 0x7fe624160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe62aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6241f3ca0 0x7fe62415ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe62aa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62415a250 con 0x7fe62415a3e0 2024-05-22T21:17:00.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe62aa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe624160db0 con 0x7fe6241f3ca0 2024-05-22T21:17:00.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe62aa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe624161030 con 0x7fe60c0047d0 2024-05-22T21:17:00.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe628c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe62415a3e0 0x7fe624160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe628c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe62415a3e0 0x7fe624160870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53746/0 (socket says 172.21.15.161:53746) 2024-05-22T21:17:00.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe628c00640 1 -- 172.21.15.161:0/2974293404 learned_addr learned my addr 172.21.15.161:0/2974293404 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:00.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe629600640 1 --2- 172.21.15.161:0/2974293404 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe60c0047d0 0x7fe624159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe62a000640 1 --2- 172.21.15.161:0/2974293404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6241f3ca0 0x7fe62415ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe628c00640 1 -- 172.21.15.161:0/2974293404 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe60c0047d0 msgr2=0x7fe624159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe628c00640 1 --2- 172.21.15.161:0/2974293404 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe60c0047d0 0x7fe624159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe628c00640 1 -- 172.21.15.161:0/2974293404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6241f3ca0 msgr2=0x7fe62415ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:00.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe628c00640 1 --2- 172.21.15.161:0/2974293404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6241f3ca0 0x7fe62415ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:00.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe628c00640 1 -- 172.21.15.161:0/2974293404 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62415c0f0 con 0x7fe62415a3e0 2024-05-22T21:17:00.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe62a000640 1 --2- 172.21.15.161:0/2974293404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6241f3ca0 0x7fe62415ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:00.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe628c00640 1 --2- 172.21.15.161:0/2974293404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe62415a3e0 0x7fe624160870 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe610009cb0 tx=0x7fe60c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:00.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe621600640 1 -- 172.21.15.161:0/2974293404 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6100ad440 con 0x7fe62415a3e0 2024-05-22T21:17:00.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe621600640 1 -- 172.21.15.161:0/2974293404 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6100ad5c0 con 0x7fe62415a3e0 2024-05-22T21:17:00.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe62aa00640 1 -- 172.21.15.161:0/2974293404 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62415c3d0 con 0x7fe62415a3e0 2024-05-22T21:17:00.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe621600640 1 -- 172.21.15.161:0/2974293404 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6100ad970 con 0x7fe62415a3e0 2024-05-22T21:17:00.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe62aa00640 1 -- 172.21.15.161:0/2974293404 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62415c990 con 0x7fe62415a3e0 2024-05-22T21:17:00.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.856+0000 7fe62aa00640 1 -- 172.21.15.161:0/2974293404 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe624067070 con 0x7fe62415a3e0 2024-05-22T21:17:00.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.860+0000 7fe621600640 1 -- 172.21.15.161:0/2974293404 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe6100b4040 con 0x7fe62415a3e0 2024-05-22T21:17:00.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.860+0000 7fe621600640 1 --2- 172.21.15.161:0/2974293404 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe604066710 0x7fe604068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:00.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.860+0000 7fe621600640 1 -- 172.21.15.161:0/2974293404 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe61011e5e0 con 0x7fe62415a3e0 2024-05-22T21:17:00.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.860+0000 7fe621600640 1 -- 172.21.15.161:0/2974293404 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7fe60406e240 con 0x7fe62415a3e0 2024-05-22T21:17:00.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.860+0000 7fe629600640 1 --2- 172.21.15.161:0/2974293404 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe604066710 0x7fe604068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:00.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.860+0000 7fe629600640 1 --2- 172.21.15.161:0/2974293404 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe604066710 0x7fe604068bd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe618009a40 tx=0x7fe61800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:00.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:00.864+0000 7fe621600640 1 -- 172.21.15.161:0/2974293404 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe6100eba70 con 0x7fe62415a3e0 2024-05-22T21:17:01.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.020+0000 7fe62aa00640 1 -- 172.21.15.161:0/2974293404 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe62413acd0 con 0x7fe62415a3e0 2024-05-22T21:17:01.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.020+0000 7fe621600640 1 -- 172.21.15.161:0/2974293404 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe6100b20a0 con 0x7fe62415a3e0 2024-05-22T21:17:01.024 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:17:01.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.024+0000 7fe62aa00640 1 -- 172.21.15.161:0/2974293404 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe604066710 msgr2=0x7fe604068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.024+0000 7fe62aa00640 1 --2- 172.21.15.161:0/2974293404 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe604066710 0x7fe604068bd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe618009a40 tx=0x7fe61800b040 comp rx=0 tx=0).stop 2024-05-22T21:17:01.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.024+0000 7fe62aa00640 1 -- 172.21.15.161:0/2974293404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe62415a3e0 msgr2=0x7fe624160870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.024+0000 7fe62aa00640 1 --2- 172.21.15.161:0/2974293404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe62415a3e0 0x7fe624160870 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe610009cb0 tx=0x7fe60c005800 comp rx=0 tx=0).stop 2024-05-22T21:17:01.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.024+0000 7fe62aa00640 1 -- 172.21.15.161:0/2974293404 shutdown_connections 2024-05-22T21:17:01.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.024+0000 7fe62aa00640 1 --2- 172.21.15.161:0/2974293404 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe604066710 0x7fe604068bd0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.024+0000 7fe62aa00640 1 --2- 172.21.15.161:0/2974293404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6241f3ca0 0x7fe62415ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.024+0000 7fe62aa00640 1 --2- 172.21.15.161:0/2974293404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe62415a3e0 0x7fe624160870 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.024+0000 7fe62aa00640 1 --2- 172.21.15.161:0/2974293404 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe60c0047d0 0x7fe624159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.024+0000 7fe62aa00640 1 -- 172.21.15.161:0/2974293404 >> 172.21.15.161:0/2974293404 conn(0x7fe6240cec00 msgr2=0x7fe62413e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:01.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.024+0000 7fe62aa00640 1 -- 172.21.15.161:0/2974293404 shutdown_connections 2024-05-22T21:17:01.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.024+0000 7fe62aa00640 1 -- 172.21.15.161:0/2974293404 wait complete. 2024-05-22T21:17:01.045 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:17:01.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20b8600640 1 Processor -- start 2024-05-22T21:17:01.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20b8600640 1 -- start start 2024-05-22T21:17:01.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20b8600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20b00cf160 0x7f20b00cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20b8600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f20b00cdce0 0x7f20b00cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20b8600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b00d1c00 con 0x7f20b015d400 2024-05-22T21:17:01.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20b8600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b00ce170 con 0x7f20b00cdce0 2024-05-22T21:17:01.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20b8600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b006e920 con 0x7f20b00cf160 2024-05-22T21:17:01.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20b5c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f20b015d400 0x7f20b006edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58640/0 (socket says 172.21.15.161:58640) 2024-05-22T21:17:01.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20b5c00640 1 -- 172.21.15.161:0/103153226 learned_addr learned my addr 172.21.15.161:0/103153226 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:01.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20b6600640 1 --2- 172.21.15.161:0/103153226 >> v2:172.21.15.179:3300/0 conn(0x7f20b00cdce0 0x7f20b00cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20b5200640 1 --2- 172.21.15.161:0/103153226 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20b00cf160 0x7f20b00cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20afe00640 1 -- 172.21.15.161:0/103153226 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199699624 0 0) 0x7f20b00d1c00 con 0x7f20b015d400 2024-05-22T21:17:01.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20afe00640 1 -- 172.21.15.161:0/103153226 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2098003530 con 0x7f20b015d400 2024-05-22T21:17:01.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20afe00640 1 -- 172.21.15.161:0/103153226 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2239131584 0 0) 0x7f2098003530 con 0x7f20b015d400 2024-05-22T21:17:01.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20afe00640 1 -- 172.21.15.161:0/103153226 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20b00cf160 msgr2=0x7f20b00cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:01.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20afe00640 1 --2- 172.21.15.161:0/103153226 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20b00cf160 0x7f20b00cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20afe00640 1 -- 172.21.15.161:0/103153226 >> v2:172.21.15.179:3300/0 conn(0x7f20b00cdce0 msgr2=0x7f20b00cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:01.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20afe00640 1 --2- 172.21.15.161:0/103153226 >> v2:172.21.15.179:3300/0 conn(0x7f20b00cdce0 0x7f20b00cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20afe00640 1 -- 172.21.15.161:0/103153226 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b00cf5a0 con 0x7f20b015d400 2024-05-22T21:17:01.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20afe00640 1 -- 172.21.15.161:0/103153226 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f20a4001a00 con 0x7f20b015d400 2024-05-22T21:17:01.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20afe00640 1 -- 172.21.15.161:0/103153226 >> v1:172.21.15.161:6789/0 conn(0x7f20b015d400 legacy=0x7f20b006edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20afe00640 1 --2- 172.21.15.161:0/103153226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2098003980 0x7f2098003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20afe00640 1 --2- 172.21.15.161:0/103153226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20980047d0 0x7f2098004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20afe00640 1 --2- 172.21.15.161:0/103153226 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20b00cf160 0x7f2098005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20afe00640 1 -- 172.21.15.161:0/103153226 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b00ce170 con 0x7f20980047d0 2024-05-22T21:17:01.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.180+0000 7f20afe00640 1 -- 172.21.15.161:0/103153226 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b006e920 con 0x7f2098003980 2024-05-22T21:17:01.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.184+0000 7f20afe00640 1 -- 172.21.15.161:0/103153226 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209800b990 con 0x7f20b00cf160 2024-05-22T21:17:01.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.184+0000 7f20b5200640 1 --2- 172.21.15.161:0/103153226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20980047d0 0x7f2098004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.184+0000 7f20b5c00640 1 --2- 172.21.15.161:0/103153226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2098003980 0x7f2098003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.184+0000 7f20b6600640 1 --2- 172.21.15.161:0/103153226 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20b00cf160 0x7f2098005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.184+0000 7f20b5200640 1 -- 172.21.15.161:0/103153226 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20b00cf160 msgr2=0x7f2098005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:01.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.184+0000 7f20b5200640 1 --2- 172.21.15.161:0/103153226 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20b00cf160 0x7f2098005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.184+0000 7f20b5200640 1 -- 172.21.15.161:0/103153226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2098003980 msgr2=0x7f2098003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:01.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.184+0000 7f20b5200640 1 --2- 172.21.15.161:0/103153226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2098003980 0x7f2098003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.184+0000 7f20b5200640 1 -- 172.21.15.161:0/103153226 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2098003530 con 0x7f20980047d0 2024-05-22T21:17:01.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.184+0000 7f20b6600640 1 --2- 172.21.15.161:0/103153226 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20b00cf160 0x7f2098005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:01.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.184+0000 7f20b5200640 1 --2- 172.21.15.161:0/103153226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20980047d0 0x7f2098004c50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f209c009b10 tx=0x7f209c01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4bebe3d1aa48cc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:01.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.184+0000 7f20afe00640 1 -- 172.21.15.161:0/103153226 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c0a6450 con 0x7f20980047d0 2024-05-22T21:17:01.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.184+0000 7f20afe00640 1 -- 172.21.15.161:0/103153226 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209c0a65b0 con 0x7f20980047d0 2024-05-22T21:17:01.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.184+0000 7f20afe00640 1 -- 172.21.15.161:0/103153226 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c0a6960 con 0x7f20980047d0 2024-05-22T21:17:01.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b8600640 1 -- 172.21.15.161:0/103153226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20980047d0 msgr2=0x7f2098004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b8600640 1 --2- 172.21.15.161:0/103153226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20980047d0 0x7f2098004c50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f209c009b10 tx=0x7f209c01ca40 comp rx=0 tx=0).stop 2024-05-22T21:17:01.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b6600640 1 -- 172.21.15.161:0/103153226 reap_dead start 2024-05-22T21:17:01.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b8600640 1 -- 172.21.15.161:0/103153226 shutdown_connections 2024-05-22T21:17:01.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b8600640 1 -- 172.21.15.161:0/103153226 >> 172.21.15.161:0/103153226 conn(0x7f20b00d0f20 msgr2=0x7f20b00d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:01.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b8600640 1 -- 172.21.15.161:0/103153226 shutdown_connections 2024-05-22T21:17:01.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b8600640 1 -- 172.21.15.161:0/103153226 wait complete. 2024-05-22T21:17:01.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b8600640 1 Processor -- start 2024-05-22T21:17:01.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b8600640 1 -- start start 2024-05-22T21:17:01.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b8600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20980047d0 0x7f20b0158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b8600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20b0156830 0x7f20b0157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b8600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20b01559b0 0x7f20b0166290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b8600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b01591a0 con 0x7f20b0156830 2024-05-22T21:17:01.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b8600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b00c50d0 con 0x7f20980047d0 2024-05-22T21:17:01.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b8600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b01580b0 con 0x7f20b01559b0 2024-05-22T21:17:01.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b5200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20b0156830 0x7f20b0157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b5200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20b0156830 0x7f20b0157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53754/0 (socket says 172.21.15.161:53754) 2024-05-22T21:17:01.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b5200640 1 -- 172.21.15.161:0/2696227003 learned_addr learned my addr 172.21.15.161:0/2696227003 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:01.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b5c00640 1 --2- 172.21.15.161:0/2696227003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20980047d0 0x7f20b0158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b5200640 1 -- 172.21.15.161:0/2696227003 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20b01559b0 msgr2=0x7f20b0166290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b5200640 1 --2- 172.21.15.161:0/2696227003 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20b01559b0 0x7f20b0166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b5200640 1 -- 172.21.15.161:0/2696227003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20980047d0 msgr2=0x7f20b0158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b5200640 1 --2- 172.21.15.161:0/2696227003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20980047d0 0x7f20b0158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b5200640 1 -- 172.21.15.161:0/2696227003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b00c4df0 con 0x7f20b0156830 2024-05-22T21:17:01.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b5c00640 1 --2- 172.21.15.161:0/2696227003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20980047d0 0x7f20b0158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:17:01.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.188+0000 7f20b5200640 1 --2- 172.21.15.161:0/2696227003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20b0156830 0x7f20b0157b70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f209c01c870 tx=0x7f2098005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:01.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.192+0000 7f20ad600640 1 -- 172.21.15.161:0/2696227003 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c00f000 con 0x7f20b0156830 2024-05-22T21:17:01.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.192+0000 7f20ad600640 1 -- 172.21.15.161:0/2696227003 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209c0a6ca0 con 0x7f20b0156830 2024-05-22T21:17:01.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.192+0000 7f20b8600640 1 -- 172.21.15.161:0/2696227003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b0157090 con 0x7f20b0156830 2024-05-22T21:17:01.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.192+0000 7f20ad600640 1 -- 172.21.15.161:0/2696227003 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c0a7670 con 0x7f20b0156830 2024-05-22T21:17:01.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.192+0000 7f20b8600640 1 -- 172.21.15.161:0/2696227003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b0157650 con 0x7f20b0156830 2024-05-22T21:17:01.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.192+0000 7f20b8600640 1 -- 172.21.15.161:0/2696227003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b0067220 con 0x7f20b0156830 2024-05-22T21:17:01.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.192+0000 7f20ad600640 1 -- 172.21.15.161:0/2696227003 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f209c0a77d0 con 0x7f20b0156830 2024-05-22T21:17:01.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.192+0000 7f20ad600640 1 --2- 172.21.15.161:0/2696227003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2090066690 0x7f2090068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.192+0000 7f20ad600640 1 -- 172.21.15.161:0/2696227003 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f209c11de60 con 0x7f20b0156830 2024-05-22T21:17:01.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.192+0000 7f20b5c00640 1 --2- 172.21.15.161:0/2696227003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2090066690 0x7f2090068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.192+0000 7f20ad600640 1 -- 172.21.15.161:0/2696227003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7f209006e1c0 con 0x7f20b0156830 2024-05-22T21:17:01.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.192+0000 7f20b5c00640 1 --2- 172.21.15.161:0/2696227003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2090066690 0x7f2090068b50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f20a4002240 tx=0x7f20a4006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:01.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.196+0000 7f20ad600640 1 -- 172.21.15.161:0/2696227003 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f209c0be020 con 0x7f20b0156830 2024-05-22T21:17:01.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.348+0000 7f20b8600640 1 -- 172.21.15.161:0/2696227003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20b00b6ea0 con 0x7f20b0156830 2024-05-22T21:17:01.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.348+0000 7f20ad600640 1 -- 172.21.15.161:0/2696227003 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f209c0eb130 con 0x7f20b0156830 2024-05-22T21:17:01.353 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:17:01.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.352+0000 7f20b8600640 1 -- 172.21.15.161:0/2696227003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2090066690 msgr2=0x7f2090068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.352+0000 7f20b8600640 1 --2- 172.21.15.161:0/2696227003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2090066690 0x7f2090068b50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f20a4002240 tx=0x7f20a4006040 comp rx=0 tx=0).stop 2024-05-22T21:17:01.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.352+0000 7f20b8600640 1 -- 172.21.15.161:0/2696227003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20b0156830 msgr2=0x7f20b0157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.352+0000 7f20b8600640 1 --2- 172.21.15.161:0/2696227003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20b0156830 0x7f20b0157b70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f209c01c870 tx=0x7f2098005800 comp rx=0 tx=0).stop 2024-05-22T21:17:01.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.352+0000 7f20b8600640 1 -- 172.21.15.161:0/2696227003 shutdown_connections 2024-05-22T21:17:01.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.352+0000 7f20b8600640 1 --2- 172.21.15.161:0/2696227003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2090066690 0x7f2090068b50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.352+0000 7f20b8600640 1 --2- 172.21.15.161:0/2696227003 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20b01559b0 0x7f20b0166290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.352+0000 7f20b8600640 1 --2- 172.21.15.161:0/2696227003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20b0156830 0x7f20b0157b70 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.352+0000 7f20b8600640 1 --2- 172.21.15.161:0/2696227003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20980047d0 0x7f20b0158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.352+0000 7f20b8600640 1 -- 172.21.15.161:0/2696227003 >> 172.21.15.161:0/2696227003 conn(0x7f20b00d0f20 msgr2=0x7f20b00cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:01.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.352+0000 7f20b8600640 1 -- 172.21.15.161:0/2696227003 shutdown_connections 2024-05-22T21:17:01.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.352+0000 7f20b8600640 1 -- 172.21.15.161:0/2696227003 wait complete. 2024-05-22T21:17:01.374 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:17:01.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.508+0000 7fb116600640 1 Processor -- start 2024-05-22T21:17:01.512 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.508+0000 7fb116600640 1 -- start start 2024-05-22T21:17:01.513 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.508+0000 7fb116600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb11015f3a0 0x7fb11015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.513 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.508+0000 7fb116600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1101442a0 con 0x7fb110143ec0 2024-05-22T21:17:01.513 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.508+0000 7fb116600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb110070500 con 0x7fb1100715a0 2024-05-22T21:17:01.513 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.508+0000 7fb116600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1100707e0 con 0x7fb11015f3a0 2024-05-22T21:17:01.513 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.508+0000 7fb10f400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb110143ec0 0x7fb11006fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58656/0 (socket says 172.21.15.161:58656) 2024-05-22T21:17:01.513 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.508+0000 7fb10f400640 1 -- 172.21.15.161:0/1443771624 learned_addr learned my addr 172.21.15.161:0/1443771624 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:01.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10fe00640 1 --2- 172.21.15.161:0/1443771624 >> v2:172.21.15.179:3300/0 conn(0x7fb11015f3a0 0x7fb11015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10e000640 1 -- 172.21.15.161:0/1443771624 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620761395 0 0) 0x7fb1101442a0 con 0x7fb110143ec0 2024-05-22T21:17:01.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10e000640 1 -- 172.21.15.161:0/1443771624 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0f4003530 con 0x7fb110143ec0 2024-05-22T21:17:01.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10e000640 1 -- 172.21.15.161:0/1443771624 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 371517421 0 0) 0x7fb0f4003530 con 0x7fb110143ec0 2024-05-22T21:17:01.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10e000640 1 -- 172.21.15.161:0/1443771624 >> v2:172.21.15.179:3300/0 conn(0x7fb11015f3a0 msgr2=0x7fb11015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:01.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10e000640 1 --2- 172.21.15.161:0/1443771624 >> v2:172.21.15.179:3300/0 conn(0x7fb11015f3a0 0x7fb11015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10e000640 1 -- 172.21.15.161:0/1443771624 >> v1:172.21.15.179:6789/0 conn(0x7fb1100715a0 legacy=0x7fb11015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10e000640 1 -- 172.21.15.161:0/1443771624 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb110070ac0 con 0x7fb110143ec0 2024-05-22T21:17:01.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10e000640 1 -- 172.21.15.161:0/1443771624 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fb100001680 con 0x7fb110143ec0 2024-05-22T21:17:01.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10fe00640 1 --2- 172.21.15.161:0/1443771624 >> v2:172.21.15.179:3300/0 conn(0x7fb11015f3a0 0x7fb11015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:17:01.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10e000640 1 -- 172.21.15.161:0/1443771624 >> v1:172.21.15.161:6789/0 conn(0x7fb110143ec0 legacy=0x7fb11006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10e000640 1 --2- 172.21.15.161:0/1443771624 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f4003980 0x7fb0f4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10e000640 1 --2- 172.21.15.161:0/1443771624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f40047d0 0x7fb0f4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10e000640 1 --2- 172.21.15.161:0/1443771624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f4005190 0x7fb0f400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10e000640 1 -- 172.21.15.161:0/1443771624 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1100707e0 con 0x7fb0f4005190 2024-05-22T21:17:01.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10e000640 1 -- 172.21.15.161:0/1443771624 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f400c030 con 0x7fb0f40047d0 2024-05-22T21:17:01.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10e000640 1 -- 172.21.15.161:0/1443771624 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f400c1f0 con 0x7fb0f4003980 2024-05-22T21:17:01.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10fe00640 1 --2- 172.21.15.161:0/1443771624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f4005190 0x7fb0f400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10f400640 1 --2- 172.21.15.161:0/1443771624 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f4003980 0x7fb0f4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10ea00640 1 --2- 172.21.15.161:0/1443771624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f40047d0 0x7fb0f4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10fe00640 1 -- 172.21.15.161:0/1443771624 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f4003980 msgr2=0x7fb0f4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:01.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10fe00640 1 --2- 172.21.15.161:0/1443771624 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f4003980 0x7fb0f4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10fe00640 1 -- 172.21.15.161:0/1443771624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f40047d0 msgr2=0x7fb0f4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:01.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10fe00640 1 --2- 172.21.15.161:0/1443771624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f40047d0 0x7fb0f4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.512+0000 7fb10fe00640 1 -- 172.21.15.161:0/1443771624 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f4003530 con 0x7fb0f4005190 2024-05-22T21:17:01.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.516+0000 7fb10fe00640 1 -- 172.21.15.161:0/1443771624 reap_dead start 2024-05-22T21:17:01.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.516+0000 7fb10fe00640 1 --2- 172.21.15.161:0/1443771624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f4005190 0x7fb0f400b990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb1100b1f90 tx=0x7fb10801ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860fe2bee1a15b9f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:01.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.516+0000 7fb10e000640 1 -- 172.21.15.161:0/1443771624 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1080a5e30 con 0x7fb0f4005190 2024-05-22T21:17:01.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.516+0000 7fb10e000640 1 -- 172.21.15.161:0/1443771624 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10800f040 con 0x7fb0f4005190 2024-05-22T21:17:01.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.516+0000 7fb10e000640 1 -- 172.21.15.161:0/1443771624 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108003ba0 con 0x7fb0f4005190 2024-05-22T21:17:01.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.516+0000 7fb116600640 1 -- 172.21.15.161:0/1443771624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f4005190 msgr2=0x7fb0f400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.516+0000 7fb116600640 1 --2- 172.21.15.161:0/1443771624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f4005190 0x7fb0f400b990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb1100b1f90 tx=0x7fb10801ce20 comp rx=0 tx=0).stop 2024-05-22T21:17:01.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.516+0000 7fb116600640 1 -- 172.21.15.161:0/1443771624 shutdown_connections 2024-05-22T21:17:01.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.516+0000 7fb116600640 1 --2- 172.21.15.161:0/1443771624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f4005190 0x7fb0f400b990 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.516+0000 7fb116600640 1 -- 172.21.15.161:0/1443771624 >> 172.21.15.161:0/1443771624 conn(0x7fb1100b0bc0 msgr2=0x7fb1100b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:01.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.516+0000 7fb116600640 1 -- 172.21.15.161:0/1443771624 shutdown_connections 2024-05-22T21:17:01.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.516+0000 7fb116600640 1 -- 172.21.15.161:0/1443771624 wait complete. 2024-05-22T21:17:01.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.516+0000 7fb116600640 1 Processor -- start 2024-05-22T21:17:01.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.516+0000 7fb116600640 1 -- start start 2024-05-22T21:17:01.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb116600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f4005190 0x7fb1100adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb116600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1100ac8b0 0x7fb11015d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb116600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb11013db30 0x7fb11015dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb116600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1100ad9c0 con 0x7fb1100ac8b0 2024-05-22T21:17:01.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb116600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1100ae320 con 0x7fb11013db30 2024-05-22T21:17:01.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb116600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1100acd40 con 0x7fb0f4005190 2024-05-22T21:17:01.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb10ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1100ac8b0 0x7fb11015d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb10ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1100ac8b0 0x7fb11015d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53774/0 (socket says 172.21.15.161:53774) 2024-05-22T21:17:01.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb10ea00640 1 -- 172.21.15.161:0/2232940354 learned_addr learned my addr 172.21.15.161:0/2232940354 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:01.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb10f400640 1 --2- 172.21.15.161:0/2232940354 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f4005190 0x7fb1100adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb10fe00640 1 --2- 172.21.15.161:0/2232940354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb11013db30 0x7fb11015dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb10ea00640 1 -- 172.21.15.161:0/2232940354 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f4005190 msgr2=0x7fb1100adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb10ea00640 1 --2- 172.21.15.161:0/2232940354 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f4005190 0x7fb1100adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb10ea00640 1 -- 172.21.15.161:0/2232940354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb11013db30 msgr2=0x7fb11015dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb10ea00640 1 --2- 172.21.15.161:0/2232940354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb11013db30 0x7fb11015dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb10ea00640 1 -- 172.21.15.161:0/2232940354 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1100ad020 con 0x7fb1100ac8b0 2024-05-22T21:17:01.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb10fe00640 1 --2- 172.21.15.161:0/2232940354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb11013db30 0x7fb11015dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:01.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb10ea00640 1 --2- 172.21.15.161:0/2232940354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1100ac8b0 0x7fb11015d040 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00f510 tx=0x7fb0fc00f910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:01.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb107400640 1 -- 172.21.15.161:0/2232940354 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0052e0 con 0x7fb1100ac8b0 2024-05-22T21:17:01.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb107400640 1 -- 172.21.15.161:0/2232940354 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc005440 con 0x7fb1100ac8b0 2024-05-22T21:17:01.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb107400640 1 -- 172.21.15.161:0/2232940354 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc003800 con 0x7fb1100ac8b0 2024-05-22T21:17:01.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb116600640 1 -- 172.21.15.161:0/2232940354 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1100ad300 con 0x7fb1100ac8b0 2024-05-22T21:17:01.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb116600640 1 -- 172.21.15.161:0/2232940354 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11015d580 con 0x7fb1100ac8b0 2024-05-22T21:17:01.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb116600640 1 -- 172.21.15.161:0/2232940354 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11015d7a0 con 0x7fb1100ac8b0 2024-05-22T21:17:01.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.520+0000 7fb107400640 1 -- 172.21.15.161:0/2232940354 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb0fc003960 con 0x7fb1100ac8b0 2024-05-22T21:17:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.524+0000 7fb107400640 1 --2- 172.21.15.161:0/2232940354 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0f0066690 0x7fb0f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.524+0000 7fb107400640 1 -- 172.21.15.161:0/2232940354 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0fc083fd0 con 0x7fb1100ac8b0 2024-05-22T21:17:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.524+0000 7fb107400640 1 -- 172.21.15.161:0/2232940354 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7fb0f006e1c0 con 0x7fb1100ac8b0 2024-05-22T21:17:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.524+0000 7fb10f400640 1 --2- 172.21.15.161:0/2232940354 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0f0066690 0x7fb0f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.524+0000 7fb10f400640 1 --2- 172.21.15.161:0/2232940354 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0f0066690 0x7fb0f0068b50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb100007d30 tx=0x7fb100000ee0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:01.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.528+0000 7fb107400640 1 -- 172.21.15.161:0/2232940354 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb11015d7a0 con 0x7fb1100ac8b0 2024-05-22T21:17:01.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.712+0000 7fb116600640 1 -- 172.21.15.161:0/2232940354 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb11015d7a0 con 0x7fb1100ac8b0 2024-05-22T21:17:01.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.716+0000 7fb107400640 1 -- 172.21.15.161:0/2232940354 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4743 (secure 0 0 0) 0x7fb11015d7a0 con 0x7fb1100ac8b0 2024-05-22T21:17:01.718 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:01.718 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1620021722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":1620021722},{"type":"v1","addr":"172.21.15.179:6835","nonce":1620021722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/3548738614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3548738614},{"type":"v1","addr":"172.21.15.161:6836","nonce":3548738614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/400898801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":400898801},{"type":"v1","addr":"172.21.15.161:6837","nonce":400898801}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:15:42.955245+0000","modified":"2024-05-22T21:15:45.977718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6834/2239627963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2239627963},{"type":"v1","addr":"172.21.15.179:6834","nonce":2239627963}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-22T21:17:01.718 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 31 2024-05-22T21:17:01.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.716+0000 7fb116600640 1 -- 172.21.15.161:0/2232940354 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0f0066690 msgr2=0x7fb0f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.716+0000 7fb116600640 1 --2- 172.21.15.161:0/2232940354 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0f0066690 0x7fb0f0068b50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb100007d30 tx=0x7fb100000ee0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.716+0000 7fb116600640 1 -- 172.21.15.161:0/2232940354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1100ac8b0 msgr2=0x7fb11015d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.716+0000 7fb116600640 1 --2- 172.21.15.161:0/2232940354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1100ac8b0 0x7fb11015d040 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00f510 tx=0x7fb0fc00f910 comp rx=0 tx=0).stop 2024-05-22T21:17:01.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.716+0000 7fb116600640 1 -- 172.21.15.161:0/2232940354 shutdown_connections 2024-05-22T21:17:01.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.716+0000 7fb116600640 1 --2- 172.21.15.161:0/2232940354 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0f0066690 0x7fb0f0068b50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.716+0000 7fb116600640 1 --2- 172.21.15.161:0/2232940354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb11013db30 0x7fb11015dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.716+0000 7fb116600640 1 --2- 172.21.15.161:0/2232940354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1100ac8b0 0x7fb11015d040 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.716+0000 7fb116600640 1 --2- 172.21.15.161:0/2232940354 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f4005190 0x7fb1100adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.716+0000 7fb116600640 1 -- 172.21.15.161:0/2232940354 >> 172.21.15.161:0/2232940354 conn(0x7fb1100b0bc0 msgr2=0x7fb110141930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:01.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.716+0000 7fb116600640 1 -- 172.21.15.161:0/2232940354 shutdown_connections 2024-05-22T21:17:01.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.716+0000 7fb116600640 1 -- 172.21.15.161:0/2232940354 wait complete. 2024-05-22T21:17:01.740 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:17:01.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.876+0000 7fb9ab200640 1 Processor -- start 2024-05-22T21:17:01.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.876+0000 7fb9ab200640 1 -- start start 2024-05-22T21:17:01.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.876+0000 7fb9ab200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb9ac15af00 0x7fb9ac15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.876+0000 7fb9ab200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb9ac0d1150 0x7fb9ac0cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.876+0000 7fb9ab200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ac0cb8a0 con 0x7fb9ac15f040 2024-05-22T21:17:01.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.876+0000 7fb9ab200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ac15f530 con 0x7fb9ac0d1150 2024-05-22T21:17:01.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.876+0000 7fb9ab200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac15f810 con 0x7fb9ac15af00 2024-05-22T21:17:01.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.876+0000 7fb9a9e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb9ac15af00 0x7fb9ac15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.876+0000 7fb9aa800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb9ac0d1150 0x7fb9ac0cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.876+0000 7fb9a9e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb9ac15af00 0x7fb9ac15eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:38386/0 (socket says 172.21.15.161:38386) 2024-05-22T21:17:01.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.876+0000 7fb9a9e00640 1 -- 172.21.15.161:0/3921513670 learned_addr learned my addr 172.21.15.161:0/3921513670 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:01.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.876+0000 7fb9a9e00640 1 -- 172.21.15.161:0/3921513670 >> v2:172.21.15.179:3300/0 conn(0x7fb9ac0d1150 msgr2=0x7fb9ac0cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:01.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.876+0000 7fb9a9e00640 1 --2- 172.21.15.161:0/3921513670 >> v2:172.21.15.179:3300/0 conn(0x7fb9ac0d1150 0x7fb9ac0cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.876+0000 7fb9a9e00640 1 -- 172.21.15.161:0/3921513670 >> v1:172.21.15.179:6789/0 conn(0x7fb9ac15f040 legacy=0x7fb9ac0d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.876+0000 7fb9a9e00640 1 -- 172.21.15.161:0/3921513670 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac0cbb20 con 0x7fb9ac15af00 2024-05-22T21:17:01.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.876+0000 7fb9aa800640 1 --2- 172.21.15.161:0/3921513670 >> v2:172.21.15.179:3300/0 conn(0x7fb9ac0d1150 0x7fb9ac0cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:17:01.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.880+0000 7fb9a9e00640 1 --2- 172.21.15.161:0/3921513670 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb9ac15af00 0x7fb9ac15eb00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009870 tx=0x7fb99c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5305e51c83f32b7f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:01.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.880+0000 7fb9a8a00640 1 -- 172.21.15.161:0/3921513670 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0ac640 con 0x7fb9ac15af00 2024-05-22T21:17:01.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.880+0000 7fb9a8a00640 1 -- 172.21.15.161:0/3921513670 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c0ac7a0 con 0x7fb9ac15af00 2024-05-22T21:17:01.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.880+0000 7fb9a8a00640 1 -- 172.21.15.161:0/3921513670 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0acb30 con 0x7fb9ac15af00 2024-05-22T21:17:01.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.880+0000 7fb9ab200640 1 -- 172.21.15.161:0/3921513670 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb9ac15af00 msgr2=0x7fb9ac15eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.880+0000 7fb9ab200640 1 --2- 172.21.15.161:0/3921513670 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb9ac15af00 0x7fb9ac15eb00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009870 tx=0x7fb99c01c4d0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.880+0000 7fb9ab200640 1 -- 172.21.15.161:0/3921513670 shutdown_connections 2024-05-22T21:17:01.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.880+0000 7fb9ab200640 1 --2- 172.21.15.161:0/3921513670 >> v2:172.21.15.179:3300/0 conn(0x7fb9ac0d1150 0x7fb9ac0cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.880+0000 7fb9ab200640 1 --2- 172.21.15.161:0/3921513670 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb9ac15af00 0x7fb9ac15eb00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.880+0000 7fb9ab200640 1 -- 172.21.15.161:0/3921513670 >> 172.21.15.161:0/3921513670 conn(0x7fb9ac099a40 msgr2=0x7fb9ac099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:01.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.880+0000 7fb9ab200640 1 -- 172.21.15.161:0/3921513670 shutdown_connections 2024-05-22T21:17:01.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.880+0000 7fb9ab200640 1 -- 172.21.15.161:0/3921513670 wait complete. 2024-05-22T21:17:01.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9ab200640 1 Processor -- start 2024-05-22T21:17:01.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9ab200640 1 -- start start 2024-05-22T21:17:01.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9ab200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb9ac0d1150 0x7fb9ac0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9ab200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9ac15af00 0x7fb9ac14dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9ab200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9ac15f040 0x7fb9ac14e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9ab200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac14e630 con 0x7fb9ac15f040 2024-05-22T21:17:01.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9ab200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac0c9880 con 0x7fb9ac15af00 2024-05-22T21:17:01.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9ab200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac0b32e0 con 0x7fb9ac0d1150 2024-05-22T21:17:01.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9aa800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9ac15f040 0x7fb9ac14e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9aa800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9ac15f040 0x7fb9ac14e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53780/0 (socket says 172.21.15.161:53780) 2024-05-22T21:17:01.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9aa800640 1 -- 172.21.15.161:0/2244793318 learned_addr learned my addr 172.21.15.161:0/2244793318 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:01.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9a9e00640 1 --2- 172.21.15.161:0/2244793318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb9ac0d1150 0x7fb9ac0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9a9400640 1 --2- 172.21.15.161:0/2244793318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9ac15af00 0x7fb9ac14dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9aa800640 1 -- 172.21.15.161:0/2244793318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb9ac0d1150 msgr2=0x7fb9ac0b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9aa800640 1 --2- 172.21.15.161:0/2244793318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb9ac0d1150 0x7fb9ac0b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9aa800640 1 -- 172.21.15.161:0/2244793318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9ac15af00 msgr2=0x7fb9ac14dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:01.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9aa800640 1 --2- 172.21.15.161:0/2244793318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9ac15af00 0x7fb9ac14dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:01.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9aa800640 1 -- 172.21.15.161:0/2244793318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a4009a70 con 0x7fb9ac15f040 2024-05-22T21:17:01.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9a9400640 1 --2- 172.21.15.161:0/2244793318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9ac15af00 0x7fb9ac14dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:01.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9aa800640 1 --2- 172.21.15.161:0/2244793318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9ac15f040 0x7fb9ac14e0f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400efc0 tx=0x7fb9a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:01.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9a2000640 1 -- 172.21.15.161:0/2244793318 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4009cd0 con 0x7fb9ac15f040 2024-05-22T21:17:01.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9a2000640 1 -- 172.21.15.161:0/2244793318 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a400f040 con 0x7fb9ac15f040 2024-05-22T21:17:01.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9ab200640 1 -- 172.21.15.161:0/2244793318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb99c0094d0 con 0x7fb9ac15f040 2024-05-22T21:17:01.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9a2000640 1 -- 172.21.15.161:0/2244793318 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a40039a0 con 0x7fb9ac15f040 2024-05-22T21:17:01.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9ab200640 1 -- 172.21.15.161:0/2244793318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ac0b4de0 con 0x7fb9ac15f040 2024-05-22T21:17:01.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.884+0000 7fb9ab200640 1 -- 172.21.15.161:0/2244793318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ac067070 con 0x7fb9ac15f040 2024-05-22T21:17:01.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.888+0000 7fb9a2000640 1 -- 172.21.15.161:0/2244793318 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb9a4014480 con 0x7fb9ac15f040 2024-05-22T21:17:01.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.888+0000 7fb9a2000640 1 --2- 172.21.15.161:0/2244793318 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb98c0666c0 0x7fb98c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:01.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.888+0000 7fb9a2000640 1 -- 172.21.15.161:0/2244793318 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9a4084d20 con 0x7fb9ac15f040 2024-05-22T21:17:01.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.888+0000 7fb9a9e00640 1 --2- 172.21.15.161:0/2244793318 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb98c0666c0 0x7fb98c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:01.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.888+0000 7fb9a2000640 1 -- 172.21.15.161:0/2244793318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7fb98c06e1f0 con 0x7fb9ac15f040 2024-05-22T21:17:01.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.888+0000 7fb9a9e00640 1 --2- 172.21.15.161:0/2244793318 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb98c0666c0 0x7fb98c068b80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0ae000 tx=0x7fb99c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:01.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:01.892+0000 7fb9a2000640 1 -- 172.21.15.161:0/2244793318 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb9a4018060 con 0x7fb9ac15f040 2024-05-22T21:17:02.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.052+0000 7fb9ab200640 1 -- 172.21.15.161:0/2244793318 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9ac0b6d10 con 0x7fb9ac15f040 2024-05-22T21:17:02.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.052+0000 7fb9a2000640 1 -- 172.21.15.161:0/2244793318 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16189 (secure 0 0 0) 0x7fb9a4018070 con 0x7fb9ac15f040 2024-05-22T21:17:02.056 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:02.056 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":71,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:17:00.514525+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:15:40.613692+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-22T21:15:41.954314+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:17:02.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.056+0000 7fb9ab200640 1 -- 172.21.15.161:0/2244793318 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb98c0666c0 msgr2=0x7fb98c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.056+0000 7fb9ab200640 1 --2- 172.21.15.161:0/2244793318 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb98c0666c0 0x7fb98c068b80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0ae000 tx=0x7fb99c0a7000 comp rx=0 tx=0).stop 2024-05-22T21:17:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.056+0000 7fb9ab200640 1 -- 172.21.15.161:0/2244793318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9ac15f040 msgr2=0x7fb9ac14e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.056+0000 7fb9ab200640 1 --2- 172.21.15.161:0/2244793318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9ac15f040 0x7fb9ac14e0f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400efc0 tx=0x7fb9a400c490 comp rx=0 tx=0).stop 2024-05-22T21:17:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.056+0000 7fb9ab200640 1 -- 172.21.15.161:0/2244793318 shutdown_connections 2024-05-22T21:17:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.056+0000 7fb9ab200640 1 --2- 172.21.15.161:0/2244793318 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb98c0666c0 0x7fb98c068b80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.056+0000 7fb9ab200640 1 --2- 172.21.15.161:0/2244793318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9ac15f040 0x7fb9ac14e0f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.056+0000 7fb9ab200640 1 --2- 172.21.15.161:0/2244793318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9ac15af00 0x7fb9ac14dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.056+0000 7fb9ab200640 1 --2- 172.21.15.161:0/2244793318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb9ac0d1150 0x7fb9ac0b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.056+0000 7fb9ab200640 1 -- 172.21.15.161:0/2244793318 >> 172.21.15.161:0/2244793318 conn(0x7fb9ac099a40 msgr2=0x7fb9ac155350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.056+0000 7fb9ab200640 1 -- 172.21.15.161:0/2244793318 shutdown_connections 2024-05-22T21:17:02.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.056+0000 7fb9ab200640 1 -- 172.21.15.161:0/2244793318 wait complete. 2024-05-22T21:17:02.078 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-22T21:17:02.078 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:17:02.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f8b000640 1 Processor -- start 2024-05-22T21:17:02.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f8b000640 1 -- start start 2024-05-22T21:17:02.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f8b000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2f84142ad0 0x7f2f840cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f8b000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2f840cd550 0x7f2f8406a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f8b000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f84071110 con 0x7f2f8415db70 2024-05-22T21:17:02.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f8b000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f841430e0 con 0x7f2f840cd550 2024-05-22T21:17:02.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f8b000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f841433c0 con 0x7f2f84142ad0 2024-05-22T21:17:02.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f83e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f2f8415db70 0x7f2f84140a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58662/0 (socket says 172.21.15.161:58662) 2024-05-22T21:17:02.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f83e00640 1 -- 172.21.15.161:0/4098058760 learned_addr learned my addr 172.21.15.161:0/4098058760 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:02.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f89000640 1 --2- 172.21.15.161:0/4098058760 >> v2:172.21.15.179:3300/0 conn(0x7f2f840cd550 0x7f2f8406a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f7be00640 1 --2- 172.21.15.161:0/4098058760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2f84142ad0 0x7f2f840cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f83400640 1 -- 172.21.15.161:0/4098058760 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073238193 0 0) 0x7f2f84071110 con 0x7f2f8415db70 2024-05-22T21:17:02.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f83400640 1 -- 172.21.15.161:0/4098058760 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f68003530 con 0x7f2f8415db70 2024-05-22T21:17:02.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f89000640 1 -- 172.21.15.161:0/4098058760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2f84142ad0 msgr2=0x7f2f840cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:02.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f89000640 1 --2- 172.21.15.161:0/4098058760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2f84142ad0 0x7f2f840cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f89000640 1 -- 172.21.15.161:0/4098058760 >> v1:172.21.15.161:6789/0 conn(0x7f2f8415db70 legacy=0x7f2f84140a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f83400640 1 -- 172.21.15.161:0/4098058760 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 45849744 0 0) 0x7f2f68003530 con 0x7f2f8415db70 2024-05-22T21:17:02.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.212+0000 7f2f89000640 1 -- 172.21.15.161:0/4098058760 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f841436a0 con 0x7f2f840cd550 2024-05-22T21:17:02.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.216+0000 7f2f89000640 1 --2- 172.21.15.161:0/4098058760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f840cd550 0x7f2f8406a490 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009d90 tx=0x7f2f7c01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4caefa5ab4b202ea server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:02.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.216+0000 7f2f83400640 1 -- 172.21.15.161:0/4098058760 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f74001680 con 0x7f2f840cd550 2024-05-22T21:17:02.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.216+0000 7f2f83400640 1 -- 172.21.15.161:0/4098058760 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c0a64a0 con 0x7f2f840cd550 2024-05-22T21:17:02.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.216+0000 7f2f83400640 1 -- 172.21.15.161:0/4098058760 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c0a6830 con 0x7f2f840cd550 2024-05-22T21:17:02.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.216+0000 7f2f8b000640 1 -- 172.21.15.161:0/4098058760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f840cd550 msgr2=0x7f2f8406a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.216+0000 7f2f8b000640 1 --2- 172.21.15.161:0/4098058760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f840cd550 0x7f2f8406a490 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009d90 tx=0x7f2f7c01c890 comp rx=0 tx=0).stop 2024-05-22T21:17:02.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.216+0000 7f2f8b000640 1 -- 172.21.15.161:0/4098058760 shutdown_connections 2024-05-22T21:17:02.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.216+0000 7f2f8b000640 1 --2- 172.21.15.161:0/4098058760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f840cd550 0x7f2f8406a490 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.216+0000 7f2f8b000640 1 --2- 172.21.15.161:0/4098058760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2f84142ad0 0x7f2f840cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.216+0000 7f2f8b000640 1 -- 172.21.15.161:0/4098058760 >> 172.21.15.161:0/4098058760 conn(0x7f2f840b5540 msgr2=0x7f2f840b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:02.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.216+0000 7f2f8b000640 1 -- 172.21.15.161:0/4098058760 shutdown_connections 2024-05-22T21:17:02.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.216+0000 7f2f8b000640 1 -- 172.21.15.161:0/4098058760 wait complete. 2024-05-22T21:17:02.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f8b000640 1 Processor -- start 2024-05-22T21:17:02.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f8b000640 1 -- start start 2024-05-22T21:17:02.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f8b000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f840cd550 0x7f2f8415e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f8b000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f84142ad0 0x7f2f8415fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f8b000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2f8415db70 0x7f2f840b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f8b000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f84154b20 con 0x7f2f84142ad0 2024-05-22T21:17:02.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f8b000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8415eb10 con 0x7f2f840cd550 2024-05-22T21:17:02.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f8b000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8415edf0 con 0x7f2f8415db70 2024-05-22T21:17:02.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f7be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f84142ad0 0x7f2f8415fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f7be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f84142ad0 0x7f2f8415fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53784/0 (socket says 172.21.15.161:53784) 2024-05-22T21:17:02.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f7be00640 1 -- 172.21.15.161:0/3081083038 learned_addr learned my addr 172.21.15.161:0/3081083038 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:02.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f83e00640 1 --2- 172.21.15.161:0/3081083038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f840cd550 0x7f2f8415e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f89000640 1 --2- 172.21.15.161:0/3081083038 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2f8415db70 0x7f2f840b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f7be00640 1 -- 172.21.15.161:0/3081083038 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2f8415db70 msgr2=0x7f2f840b0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f7be00640 1 --2- 172.21.15.161:0/3081083038 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2f8415db70 0x7f2f840b0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f7be00640 1 -- 172.21.15.161:0/3081083038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f840cd550 msgr2=0x7f2f8415e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f7be00640 1 --2- 172.21.15.161:0/3081083038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f840cd550 0x7f2f8415e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f7be00640 1 -- 172.21.15.161:0/3081083038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c009a70 con 0x7f2f84142ad0 2024-05-22T21:17:02.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f83e00640 1 --2- 172.21.15.161:0/3081083038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f840cd550 0x7f2f8415e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:02.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f7be00640 1 --2- 172.21.15.161:0/3081083038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f84142ad0 0x7f2f8415fbb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00e930 tx=0x7f2f6c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:02.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f80c00640 1 -- 172.21.15.161:0/3081083038 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c009690 con 0x7f2f84142ad0 2024-05-22T21:17:02.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f80c00640 1 -- 172.21.15.161:0/3081083038 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c00ca00 con 0x7f2f84142ad0 2024-05-22T21:17:02.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f80c00640 1 -- 172.21.15.161:0/3081083038 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c00cd70 con 0x7f2f84142ad0 2024-05-22T21:17:02.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f8b000640 1 -- 172.21.15.161:0/3081083038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8415f0d0 con 0x7f2f84142ad0 2024-05-22T21:17:02.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f8b000640 1 -- 172.21.15.161:0/3081083038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8415f690 con 0x7f2f84142ad0 2024-05-22T21:17:02.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.220+0000 7f2f8b000640 1 -- 172.21.15.161:0/3081083038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8415f850 con 0x7f2f84142ad0 2024-05-22T21:17:02.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.224+0000 7f2f80c00640 1 -- 172.21.15.161:0/3081083038 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2f6c013470 con 0x7f2f84142ad0 2024-05-22T21:17:02.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.224+0000 7f2f80c00640 1 --2- 172.21.15.161:0/3081083038 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f60066690 0x7f2f60068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.224+0000 7f2f80c00640 1 -- 172.21.15.161:0/3081083038 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f6c0844d0 con 0x7f2f84142ad0 2024-05-22T21:17:02.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.224+0000 7f2f83e00640 1 --2- 172.21.15.161:0/3081083038 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f60066690 0x7f2f60068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.224+0000 7f2f80c00640 1 -- 172.21.15.161:0/3081083038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7f2f6006e1c0 con 0x7f2f84142ad0 2024-05-22T21:17:02.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.224+0000 7f2f83e00640 1 --2- 172.21.15.161:0/3081083038 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f60066690 0x7f2f60068b50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2f74005890 tx=0x7f2f74009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:02.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.228+0000 7f2f80c00640 1 -- 172.21.15.161:0/3081083038 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2f6c017060 con 0x7f2f84142ad0 2024-05-22T21:17:02.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.412+0000 7f2f8b000640 1 -- 172.21.15.161:0/3081083038 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f84070cb0 con 0x7f2f84142ad0 2024-05-22T21:17:02.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.412+0000 7f2f80c00640 1 -- 172.21.15.161:0/3081083038 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4743 (secure 0 0 0) 0x7f2f6c021070 con 0x7f2f84142ad0 2024-05-22T21:17:02.416 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:02.417 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1620021722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":1620021722},{"type":"v1","addr":"172.21.15.179:6835","nonce":1620021722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/3548738614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3548738614},{"type":"v1","addr":"172.21.15.161:6836","nonce":3548738614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/400898801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":400898801},{"type":"v1","addr":"172.21.15.161:6837","nonce":400898801}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:15:42.955245+0000","modified":"2024-05-22T21:15:45.977718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6834/2239627963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2239627963},{"type":"v1","addr":"172.21.15.179:6834","nonce":2239627963}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-22T21:17:02.417 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 31 2024-05-22T21:17:02.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.416+0000 7f2f8b000640 1 -- 172.21.15.161:0/3081083038 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f60066690 msgr2=0x7f2f60068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.416+0000 7f2f8b000640 1 --2- 172.21.15.161:0/3081083038 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f60066690 0x7f2f60068b50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2f74005890 tx=0x7f2f74009040 comp rx=0 tx=0).stop 2024-05-22T21:17:02.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.416+0000 7f2f8b000640 1 -- 172.21.15.161:0/3081083038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f84142ad0 msgr2=0x7f2f8415fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.416+0000 7f2f8b000640 1 --2- 172.21.15.161:0/3081083038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f84142ad0 0x7f2f8415fbb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00e930 tx=0x7f2f6c00edf0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.416+0000 7f2f8b000640 1 -- 172.21.15.161:0/3081083038 shutdown_connections 2024-05-22T21:17:02.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.416+0000 7f2f8b000640 1 --2- 172.21.15.161:0/3081083038 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f60066690 0x7f2f60068b50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.416+0000 7f2f8b000640 1 --2- 172.21.15.161:0/3081083038 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2f8415db70 0x7f2f840b0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.416+0000 7f2f8b000640 1 --2- 172.21.15.161:0/3081083038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f84142ad0 0x7f2f8415fbb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.416+0000 7f2f8b000640 1 --2- 172.21.15.161:0/3081083038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f840cd550 0x7f2f8415e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.416+0000 7f2f8b000640 1 -- 172.21.15.161:0/3081083038 >> 172.21.15.161:0/3081083038 conn(0x7f2f840b5540 msgr2=0x7f2f840b5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:02.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.416+0000 7f2f8b000640 1 -- 172.21.15.161:0/3081083038 shutdown_connections 2024-05-22T21:17:02.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.416+0000 7f2f8b000640 1 -- 172.21.15.161:0/3081083038 wait complete. 2024-05-22T21:17:02.438 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:17:02.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff532a00640 1 Processor -- start 2024-05-22T21:17:02.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff532a00640 1 -- start start 2024-05-22T21:17:02.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff532a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff52c099ca0 0x7ff52c0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff532a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52c0cba60 0x7ff52c09a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff532a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c070050 con 0x7ff52c09a5c0 2024-05-22T21:17:02.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff532a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c06e760 con 0x7ff52c099ca0 2024-05-22T21:17:02.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff532a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c06fab0 con 0x7ff52c0cba60 2024-05-22T21:17:02.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff530a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff52c09a5c0 0x7ff52c06ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58672/0 (socket says 172.21.15.161:58672) 2024-05-22T21:17:02.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff530a00640 1 -- 172.21.15.161:0/4141708274 learned_addr learned my addr 172.21.15.161:0/4141708274 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:02.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff523400640 1 --2- 172.21.15.161:0/4141708274 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52c0cba60 0x7ff52c09a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff52be00640 1 --2- 172.21.15.161:0/4141708274 >> v2:172.21.15.179:3300/0 conn(0x7ff52c099ca0 0x7ff52c0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff52b400640 1 -- 172.21.15.161:0/4141708274 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2234108895 0 0) 0x7ff5240014a0 con 0x7ff52c09a5c0 2024-05-22T21:17:02.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff52b400640 1 -- 172.21.15.161:0/4141708274 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff510003530 con 0x7ff52c09a5c0 2024-05-22T21:17:02.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff52b400640 1 -- 172.21.15.161:0/4141708274 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1247399008 0 0) 0x7ff510003530 con 0x7ff52c09a5c0 2024-05-22T21:17:02.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff523400640 1 -- 172.21.15.161:0/4141708274 >> v2:172.21.15.179:3300/0 conn(0x7ff52c099ca0 msgr2=0x7ff52c0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:02.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff523400640 1 --2- 172.21.15.161:0/4141708274 >> v2:172.21.15.179:3300/0 conn(0x7ff52c099ca0 0x7ff52c0b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff523400640 1 -- 172.21.15.161:0/4141708274 >> v1:172.21.15.161:6789/0 conn(0x7ff52c09a5c0 legacy=0x7ff52c06ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff523400640 1 -- 172.21.15.161:0/4141708274 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c06fd70 con 0x7ff52c0cba60 2024-05-22T21:17:02.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.576+0000 7ff52be00640 1 --2- 172.21.15.161:0/4141708274 >> v2:172.21.15.179:3300/0 conn(0x7ff52c099ca0 0x7ff52c0b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:17:02.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.580+0000 7ff523400640 1 --2- 172.21.15.161:0/4141708274 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52c0cba60 0x7ff52c09a080 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5140097c0 tx=0x7ff51401c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=653099e09ecc0c6f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:02.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.580+0000 7ff52b400640 1 -- 172.21.15.161:0/4141708274 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524001b90 con 0x7ff52c0cba60 2024-05-22T21:17:02.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.580+0000 7ff52b400640 1 -- 172.21.15.161:0/4141708274 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5140a7050 con 0x7ff52c0cba60 2024-05-22T21:17:02.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.580+0000 7ff52b400640 1 -- 172.21.15.161:0/4141708274 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5140ab600 con 0x7ff52c0cba60 2024-05-22T21:17:02.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.580+0000 7ff532a00640 1 -- 172.21.15.161:0/4141708274 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52c0cba60 msgr2=0x7ff52c09a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.580+0000 7ff532a00640 1 --2- 172.21.15.161:0/4141708274 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52c0cba60 0x7ff52c09a080 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5140097c0 tx=0x7ff51401c390 comp rx=0 tx=0).stop 2024-05-22T21:17:02.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.580+0000 7ff532a00640 1 -- 172.21.15.161:0/4141708274 shutdown_connections 2024-05-22T21:17:02.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff532a00640 1 --2- 172.21.15.161:0/4141708274 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52c0cba60 0x7ff52c09a080 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff532a00640 1 --2- 172.21.15.161:0/4141708274 >> v2:172.21.15.179:3300/0 conn(0x7ff52c099ca0 0x7ff52c0b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff532a00640 1 -- 172.21.15.161:0/4141708274 >> 172.21.15.161:0/4141708274 conn(0x7ff52c0b2a40 msgr2=0x7ff52c0b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:02.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff532a00640 1 -- 172.21.15.161:0/4141708274 shutdown_connections 2024-05-22T21:17:02.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff532a00640 1 -- 172.21.15.161:0/4141708274 wait complete. 2024-05-22T21:17:02.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff532a00640 1 Processor -- start 2024-05-22T21:17:02.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff532a00640 1 -- start start 2024-05-22T21:17:02.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff532a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff52c099ca0 0x7ff52c0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff532a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52c09a5c0 0x7ff52c0c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff532a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff52c0cba60 0x7ff52c154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff532a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c1f7d60 con 0x7ff52c0cba60 2024-05-22T21:17:02.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff532a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0c4fc0 con 0x7ff52c099ca0 2024-05-22T21:17:02.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff532a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0c51f0 con 0x7ff52c09a5c0 2024-05-22T21:17:02.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff530a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff52c0cba60 0x7ff52c154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff530a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff52c0cba60 0x7ff52c154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53796/0 (socket says 172.21.15.161:53796) 2024-05-22T21:17:02.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff530a00640 1 -- 172.21.15.161:0/1238524574 learned_addr learned my addr 172.21.15.161:0/1238524574 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:02.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff523400640 1 --2- 172.21.15.161:0/1238524574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52c09a5c0 0x7ff52c0c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff52be00640 1 --2- 172.21.15.161:0/1238524574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff52c099ca0 0x7ff52c0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff530a00640 1 -- 172.21.15.161:0/1238524574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52c09a5c0 msgr2=0x7ff52c0c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff530a00640 1 --2- 172.21.15.161:0/1238524574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52c09a5c0 0x7ff52c0c4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff530a00640 1 -- 172.21.15.161:0/1238524574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff52c099ca0 msgr2=0x7ff52c0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff530a00640 1 --2- 172.21.15.161:0/1238524574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff52c099ca0 0x7ff52c0c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff530a00640 1 -- 172.21.15.161:0/1238524574 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c009420 con 0x7ff52c0cba60 2024-05-22T21:17:02.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.584+0000 7ff530a00640 1 --2- 172.21.15.161:0/1238524574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff52c0cba60 0x7ff52c154510 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff52400cc10 tx=0x7ff52400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:02.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.588+0000 7ff528c00640 1 -- 172.21.15.161:0/1238524574 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52400ce20 con 0x7ff52c0cba60 2024-05-22T21:17:02.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.588+0000 7ff528c00640 1 -- 172.21.15.161:0/1238524574 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff524012400 con 0x7ff52c0cba60 2024-05-22T21:17:02.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.588+0000 7ff528c00640 1 -- 172.21.15.161:0/1238524574 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5240127b0 con 0x7ff52c0cba60 2024-05-22T21:17:02.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.588+0000 7ff532a00640 1 -- 172.21.15.161:0/1238524574 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff514009420 con 0x7ff52c0cba60 2024-05-22T21:17:02.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.588+0000 7ff532a00640 1 -- 172.21.15.161:0/1238524574 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c0c57b0 con 0x7ff52c0cba60 2024-05-22T21:17:02.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.588+0000 7ff532a00640 1 -- 172.21.15.161:0/1238524574 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52c1f7ea0 con 0x7ff52c0cba60 2024-05-22T21:17:02.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.588+0000 7ff528c00640 1 -- 172.21.15.161:0/1238524574 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff524012910 con 0x7ff52c0cba60 2024-05-22T21:17:02.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.588+0000 7ff528c00640 1 --2- 172.21.15.161:0/1238524574 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff508066690 0x7ff508068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.588+0000 7ff528c00640 1 -- 172.21.15.161:0/1238524574 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff524084580 con 0x7ff52c0cba60 2024-05-22T21:17:02.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.588+0000 7ff52be00640 1 --2- 172.21.15.161:0/1238524574 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff508066690 0x7ff508068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.588+0000 7ff528c00640 1 -- 172.21.15.161:0/1238524574 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7ff50806e1c0 con 0x7ff52c0cba60 2024-05-22T21:17:02.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.588+0000 7ff52be00640 1 --2- 172.21.15.161:0/1238524574 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff508066690 0x7ff508068b50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009c50 tx=0x7ff51c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:02.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.592+0000 7ff528c00640 1 -- 172.21.15.161:0/1238524574 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff52c1f7ea0 con 0x7ff52c0cba60 2024-05-22T21:17:02.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.780+0000 7ff532a00640 1 -- 172.21.15.161:0/1238524574 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff52c070050 con 0x7ff52c0cba60 2024-05-22T21:17:02.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.780+0000 7ff528c00640 1 -- 172.21.15.161:0/1238524574 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4743 (secure 0 0 0) 0x7ff52c070050 con 0x7ff52c0cba60 2024-05-22T21:17:02.784 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:02.784 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1620021722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":1620021722},{"type":"v1","addr":"172.21.15.179:6835","nonce":1620021722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/3548738614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3548738614},{"type":"v1","addr":"172.21.15.161:6836","nonce":3548738614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/400898801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":400898801},{"type":"v1","addr":"172.21.15.161:6837","nonce":400898801}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:15:42.955245+0000","modified":"2024-05-22T21:15:45.977718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6834/2239627963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2239627963},{"type":"v1","addr":"172.21.15.179:6834","nonce":2239627963}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-22T21:17:02.784 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 31 2024-05-22T21:17:02.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.780+0000 7ff532a00640 1 -- 172.21.15.161:0/1238524574 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff508066690 msgr2=0x7ff508068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.780+0000 7ff532a00640 1 --2- 172.21.15.161:0/1238524574 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff508066690 0x7ff508068b50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009c50 tx=0x7ff51c009190 comp rx=0 tx=0).stop 2024-05-22T21:17:02.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.780+0000 7ff532a00640 1 -- 172.21.15.161:0/1238524574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff52c0cba60 msgr2=0x7ff52c154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.784+0000 7ff532a00640 1 --2- 172.21.15.161:0/1238524574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff52c0cba60 0x7ff52c154510 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff52400cc10 tx=0x7ff52400a430 comp rx=0 tx=0).stop 2024-05-22T21:17:02.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.784+0000 7ff532a00640 1 -- 172.21.15.161:0/1238524574 shutdown_connections 2024-05-22T21:17:02.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.784+0000 7ff532a00640 1 --2- 172.21.15.161:0/1238524574 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff508066690 0x7ff508068b50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.784+0000 7ff532a00640 1 --2- 172.21.15.161:0/1238524574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff52c0cba60 0x7ff52c154510 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.784+0000 7ff532a00640 1 --2- 172.21.15.161:0/1238524574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff52c09a5c0 0x7ff52c0c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.784+0000 7ff532a00640 1 --2- 172.21.15.161:0/1238524574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff52c099ca0 0x7ff52c0c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.784+0000 7ff532a00640 1 -- 172.21.15.161:0/1238524574 >> 172.21.15.161:0/1238524574 conn(0x7ff52c0b2a40 msgr2=0x7ff52c06afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:02.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.784+0000 7ff532a00640 1 -- 172.21.15.161:0/1238524574 shutdown_connections 2024-05-22T21:17:02.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.784+0000 7ff532a00640 1 -- 172.21.15.161:0/1238524574 wait complete. 2024-05-22T21:17:02.805 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:17:02.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.936+0000 7f9104800640 1 Processor -- start 2024-05-22T21:17:02.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.936+0000 7f9104800640 1 -- start start 2024-05-22T21:17:02.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.936+0000 7f9104800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f90fc0715a0 0x7f90fc15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.940+0000 7f9104800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f90fc15ec90 0x7f90fc15f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.940+0000 7f9104800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90fc1442a0 con 0x7f90fc143ec0 2024-05-22T21:17:02.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.940+0000 7f9104800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90fc15f640 con 0x7f90fc0715a0 2024-05-22T21:17:02.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.940+0000 7f9104800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90fc070500 con 0x7f90fc15ec90 2024-05-22T21:17:02.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.940+0000 7f9101400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f90fc0715a0 0x7f90fc15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.940+0000 7f9101400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f90fc0715a0 0x7f90fc15f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53808/0 (socket says 172.21.15.161:53808) 2024-05-22T21:17:02.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.940+0000 7f9101400640 1 -- 172.21.15.161:0/266100867 learned_addr learned my addr 172.21.15.161:0/266100867 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:02.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.940+0000 7f9102800640 1 --2- 172.21.15.161:0/266100867 >> v2:172.21.15.179:3300/0 conn(0x7f90fc15ec90 0x7f90fc15f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.940+0000 7f9101400640 1 -- 172.21.15.161:0/266100867 >> v2:172.21.15.179:3300/0 conn(0x7f90fc15ec90 msgr2=0x7f90fc15f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:02.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.940+0000 7f9101400640 1 --2- 172.21.15.161:0/266100867 >> v2:172.21.15.179:3300/0 conn(0x7f90fc15ec90 0x7f90fc15f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.940+0000 7f9101400640 1 -- 172.21.15.161:0/266100867 >> v1:172.21.15.179:6789/0 conn(0x7f90fc143ec0 legacy=0x7f90fc06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.940+0000 7f9101400640 1 -- 172.21.15.161:0/266100867 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90fc0707e0 con 0x7f90fc0715a0 2024-05-22T21:17:02.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.940+0000 7f9101400640 1 --2- 172.21.15.161:0/266100867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90fc0715a0 0x7f90fc15f830 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0cedc0 tx=0x7f90ec01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a58bbd205fcd05 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:02.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.940+0000 7f9100a00640 1 -- 172.21.15.161:0/266100867 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec0ab5f0 con 0x7f90fc0715a0 2024-05-22T21:17:02.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9100a00640 1 -- 172.21.15.161:0/266100867 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ec0ab750 con 0x7f90fc0715a0 2024-05-22T21:17:02.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9100a00640 1 -- 172.21.15.161:0/266100867 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec0abae0 con 0x7f90fc0715a0 2024-05-22T21:17:02.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9104800640 1 -- 172.21.15.161:0/266100867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90fc0715a0 msgr2=0x7f90fc15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9104800640 1 --2- 172.21.15.161:0/266100867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90fc0715a0 0x7f90fc15f830 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0cedc0 tx=0x7f90ec01c4f0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9104800640 1 -- 172.21.15.161:0/266100867 shutdown_connections 2024-05-22T21:17:02.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9104800640 1 --2- 172.21.15.161:0/266100867 >> v2:172.21.15.179:3300/0 conn(0x7f90fc15ec90 0x7f90fc15f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9104800640 1 --2- 172.21.15.161:0/266100867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90fc0715a0 0x7f90fc15f830 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9104800640 1 -- 172.21.15.161:0/266100867 >> 172.21.15.161:0/266100867 conn(0x7f90fc0b0bc0 msgr2=0x7f90fc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:02.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9104800640 1 -- 172.21.15.161:0/266100867 shutdown_connections 2024-05-22T21:17:02.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9104800640 1 -- 172.21.15.161:0/266100867 wait complete. 2024-05-22T21:17:02.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9104800640 1 Processor -- start 2024-05-22T21:17:02.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9104800640 1 -- start start 2024-05-22T21:17:02.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9104800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90fc143ec0 0x7f90fc15e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9104800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90fc15ec90 0x7f90fc15ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9104800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90fc15db20 0x7f90fc13f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9104800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc15c120 con 0x7f90fc15db20 2024-05-22T21:17:02.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9104800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc15c9c0 con 0x7f90fc15ec90 2024-05-22T21:17:02.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9104800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc13fde0 con 0x7f90fc143ec0 2024-05-22T21:17:02.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9102800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90fc15db20 0x7f90fc13f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9102800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90fc15db20 0x7f90fc13f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53810/0 (socket says 172.21.15.161:53810) 2024-05-22T21:17:02.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9102800640 1 -- 172.21.15.161:0/1171376715 learned_addr learned my addr 172.21.15.161:0/1171376715 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:02.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9101e00640 1 --2- 172.21.15.161:0/1171376715 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90fc143ec0 0x7f90fc15e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9101400640 1 --2- 172.21.15.161:0/1171376715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90fc15ec90 0x7f90fc15ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9102800640 1 -- 172.21.15.161:0/1171376715 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90fc143ec0 msgr2=0x7f90fc15e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9102800640 1 --2- 172.21.15.161:0/1171376715 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90fc143ec0 0x7f90fc15e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9102800640 1 -- 172.21.15.161:0/1171376715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90fc15ec90 msgr2=0x7f90fc15ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:02.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9102800640 1 --2- 172.21.15.161:0/1171376715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90fc15ec90 0x7f90fc15ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:02.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.944+0000 7f9102800640 1 -- 172.21.15.161:0/1171376715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec0094d0 con 0x7f90fc15db20 2024-05-22T21:17:02.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.948+0000 7f9102800640 1 --2- 172.21.15.161:0/1171376715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90fc15db20 0x7f90fc13f8a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009d30 tx=0x7f90f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:02.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.948+0000 7f90f6000640 1 -- 172.21.15.161:0/1171376715 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8009ba0 con 0x7f90fc15db20 2024-05-22T21:17:02.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.948+0000 7f90f6000640 1 -- 172.21.15.161:0/1171376715 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f800f040 con 0x7f90fc15db20 2024-05-22T21:17:02.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.948+0000 7f90f6000640 1 -- 172.21.15.161:0/1171376715 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f80146c0 con 0x7f90fc15db20 2024-05-22T21:17:02.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.948+0000 7f9104800640 1 -- 172.21.15.161:0/1171376715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90fc15e880 con 0x7f90fc15db20 2024-05-22T21:17:02.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.948+0000 7f9104800640 1 -- 172.21.15.161:0/1171376715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90fc15c650 con 0x7f90fc15db20 2024-05-22T21:17:02.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.948+0000 7f9104800640 1 -- 172.21.15.161:0/1171376715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90fc067220 con 0x7f90fc15db20 2024-05-22T21:17:02.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.948+0000 7f90f6000640 1 -- 172.21.15.161:0/1171376715 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f90f8014820 con 0x7f90fc15db20 2024-05-22T21:17:02.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.948+0000 7f90f6000640 1 --2- 172.21.15.161:0/1171376715 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90e0066690 0x7f90e0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:02.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.948+0000 7f90f6000640 1 -- 172.21.15.161:0/1171376715 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90f8084a70 con 0x7f90fc15db20 2024-05-22T21:17:02.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.948+0000 7f9101e00640 1 --2- 172.21.15.161:0/1171376715 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90e0066690 0x7f90e0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:02.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.948+0000 7f90f6000640 1 -- 172.21.15.161:0/1171376715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7f90e006e1c0 con 0x7f90fc15db20 2024-05-22T21:17:02.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.952+0000 7f9101e00640 1 --2- 172.21.15.161:0/1171376715 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90e0066690 0x7f90e0068b50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f90f0007fd0 tx=0x7f90f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:02.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:02.952+0000 7f90f6000640 1 -- 172.21.15.161:0/1171376715 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f90f8018060 con 0x7f90fc15db20 2024-05-22T21:17:03.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.112+0000 7f9104800640 1 -- 172.21.15.161:0/1171376715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90fc15cb00 con 0x7f90fc15db20 2024-05-22T21:17:03.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.112+0000 7f90f6000640 1 -- 172.21.15.161:0/1171376715 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16167 (secure 0 0 0) 0x7f90f8051f20 con 0x7f90fc15db20 2024-05-22T21:17:03.115 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:03.115 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":72,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:17:02.520076+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:15:40.613692+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-22T21:15:41.954314+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:17:03.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.112+0000 7f9104800640 1 -- 172.21.15.161:0/1171376715 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90e0066690 msgr2=0x7f90e0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:03.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.112+0000 7f9104800640 1 --2- 172.21.15.161:0/1171376715 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90e0066690 0x7f90e0068b50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f90f0007fd0 tx=0x7f90f0009040 comp rx=0 tx=0).stop 2024-05-22T21:17:03.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.112+0000 7f9104800640 1 -- 172.21.15.161:0/1171376715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90fc15db20 msgr2=0x7f90fc13f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:03.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.112+0000 7f9104800640 1 --2- 172.21.15.161:0/1171376715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90fc15db20 0x7f90fc13f8a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009d30 tx=0x7f90f800c490 comp rx=0 tx=0).stop 2024-05-22T21:17:03.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.112+0000 7f9104800640 1 -- 172.21.15.161:0/1171376715 shutdown_connections 2024-05-22T21:17:03.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.112+0000 7f9104800640 1 --2- 172.21.15.161:0/1171376715 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90e0066690 0x7f90e0068b50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.112+0000 7f9104800640 1 --2- 172.21.15.161:0/1171376715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90fc15db20 0x7f90fc13f8a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.112+0000 7f9104800640 1 --2- 172.21.15.161:0/1171376715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90fc15ec90 0x7f90fc15ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.112+0000 7f9104800640 1 --2- 172.21.15.161:0/1171376715 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90fc143ec0 0x7f90fc15e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.112+0000 7f9104800640 1 -- 172.21.15.161:0/1171376715 >> 172.21.15.161:0/1171376715 conn(0x7f90fc0b0bc0 msgr2=0x7f90fc0b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:03.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.116+0000 7f9104800640 1 -- 172.21.15.161:0/1171376715 shutdown_connections 2024-05-22T21:17:03.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.116+0000 7f9104800640 1 -- 172.21.15.161:0/1171376715 wait complete. 2024-05-22T21:17:03.137 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-22T21:17:03.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.268+0000 7f1343a00640 1 Processor -- start 2024-05-22T21:17:03.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.268+0000 7f1343a00640 1 -- start start 2024-05-22T21:17:03.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.272+0000 7f1343a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f133c13cfc0 0x7f133c1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:03.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.272+0000 7f1343a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f133c15af60 0x7f133c13f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:03.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.272+0000 7f1343a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133c1400d0 con 0x7f133c13fc80 2024-05-22T21:17:03.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.272+0000 7f1343a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f133c13d3a0 con 0x7f133c15af60 2024-05-22T21:17:03.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.272+0000 7f1343a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0cd920 con 0x7f133c13cfc0 2024-05-22T21:17:03.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.272+0000 7f1333e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f133c15af60 0x7f133c13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:03.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.272+0000 7f1341000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f133c13cfc0 0x7f133c1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:03.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.272+0000 7f1333e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f133c15af60 0x7f133c13f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:49058/0 (socket says 172.21.15.161:49058) 2024-05-22T21:17:03.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.272+0000 7f1333e00640 1 -- 172.21.15.161:0/4075250686 learned_addr learned my addr 172.21.15.161:0/4075250686 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:03.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.272+0000 7f1333e00640 1 -- 172.21.15.161:0/4075250686 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f133c13cfc0 msgr2=0x7f133c1402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:03.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.272+0000 7f1333e00640 1 --2- 172.21.15.161:0/4075250686 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f133c13cfc0 0x7f133c1402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.272+0000 7f1333e00640 1 -- 172.21.15.161:0/4075250686 >> v1:172.21.15.179:6789/0 conn(0x7f133c13fc80 legacy=0x7f133c15dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:03.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.272+0000 7f1333e00640 1 -- 172.21.15.161:0/4075250686 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c0cdc00 con 0x7f133c15af60 2024-05-22T21:17:03.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.272+0000 7f133be00640 1 -- 172.21.15.161:0/4075250686 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167878435 0 0) 0x7f13340014a0 con 0x7f133c13fc80 2024-05-22T21:17:03.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.276+0000 7f1333e00640 1 --2- 172.21.15.161:0/4075250686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f133c15af60 0x7f133c13f740 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1324009870 tx=0x7f132401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e713772bc1a411f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:03.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.276+0000 7f133be00640 1 -- 172.21.15.161:0/4075250686 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13240ab600 con 0x7f133c15af60 2024-05-22T21:17:03.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.276+0000 7f133be00640 1 -- 172.21.15.161:0/4075250686 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13240a7050 con 0x7f133c15af60 2024-05-22T21:17:03.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.276+0000 7f133be00640 1 -- 172.21.15.161:0/4075250686 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13240ab980 con 0x7f133c15af60 2024-05-22T21:17:03.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.276+0000 7f1343a00640 1 -- 172.21.15.161:0/4075250686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f133c15af60 msgr2=0x7f133c13f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:03.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.276+0000 7f1343a00640 1 --2- 172.21.15.161:0/4075250686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f133c15af60 0x7f133c13f740 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1324009870 tx=0x7f132401c4d0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.276+0000 7f1343a00640 1 -- 172.21.15.161:0/4075250686 shutdown_connections 2024-05-22T21:17:03.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.276+0000 7f1343a00640 1 --2- 172.21.15.161:0/4075250686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f133c15af60 0x7f133c13f740 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.276+0000 7f1343a00640 1 --2- 172.21.15.161:0/4075250686 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f133c13cfc0 0x7f133c1402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.276+0000 7f1343a00640 1 -- 172.21.15.161:0/4075250686 >> 172.21.15.161:0/4075250686 conn(0x7f133c06a2d0 msgr2=0x7f133c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:03.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.276+0000 7f1343a00640 1 -- 172.21.15.161:0/4075250686 shutdown_connections 2024-05-22T21:17:03.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.276+0000 7f1343a00640 1 -- 172.21.15.161:0/4075250686 wait complete. 2024-05-22T21:17:03.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.276+0000 7f1343a00640 1 Processor -- start 2024-05-22T21:17:03.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.276+0000 7f1343a00640 1 -- start start 2024-05-22T21:17:03.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1343a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133c13cfc0 0x7f133c13e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:03.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1343a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f133c13fc80 0x7f133c13c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:03.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1343a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f133c15af60 0x7f133c13c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:03.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1343a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0b3bd0 con 0x7f133c13cfc0 2024-05-22T21:17:03.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1343a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0b3050 con 0x7f133c15af60 2024-05-22T21:17:03.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1343a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0b3330 con 0x7f133c13fc80 2024-05-22T21:17:03.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1341000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133c13cfc0 0x7f133c13e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:03.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1341000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133c13cfc0 0x7f133c13e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53822/0 (socket says 172.21.15.161:53822) 2024-05-22T21:17:03.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1341000640 1 -- 172.21.15.161:0/1284493499 learned_addr learned my addr 172.21.15.161:0/1284493499 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:03.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1333e00640 1 --2- 172.21.15.161:0/1284493499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f133c13fc80 0x7f133c13c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:03.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1341a00640 1 --2- 172.21.15.161:0/1284493499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f133c15af60 0x7f133c13c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:03.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1341000640 1 -- 172.21.15.161:0/1284493499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f133c13fc80 msgr2=0x7f133c13c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:03.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1341000640 1 --2- 172.21.15.161:0/1284493499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f133c13fc80 0x7f133c13c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1341000640 1 -- 172.21.15.161:0/1284493499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f133c15af60 msgr2=0x7f133c13c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:03.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1341000640 1 --2- 172.21.15.161:0/1284493499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f133c15af60 0x7f133c13c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1341000640 1 -- 172.21.15.161:0/1284493499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13240094d0 con 0x7f133c13cfc0 2024-05-22T21:17:03.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1341000640 1 --2- 172.21.15.161:0/1284493499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133c13cfc0 0x7f133c13e5f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f132c00e870 tx=0x7f132c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:03.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1339600640 1 -- 172.21.15.161:0/1284493499 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c009510 con 0x7f133c13cfc0 2024-05-22T21:17:03.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1339600640 1 -- 172.21.15.161:0/1284493499 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c00eec0 con 0x7f133c13cfc0 2024-05-22T21:17:03.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1343a00640 1 -- 172.21.15.161:0/1284493499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133c0b3610 con 0x7f133c13cfc0 2024-05-22T21:17:03.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1339600640 1 -- 172.21.15.161:0/1284493499 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c00cb40 con 0x7f133c13cfc0 2024-05-22T21:17:03.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1343a00640 1 -- 172.21.15.161:0/1284493499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133c13d610 con 0x7f133c13cfc0 2024-05-22T21:17:03.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.280+0000 7f1343a00640 1 -- 172.21.15.161:0/1284493499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133c067220 con 0x7f133c13cfc0 2024-05-22T21:17:03.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.284+0000 7f1339600640 1 -- 172.21.15.161:0/1284493499 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f132c00cca0 con 0x7f133c13cfc0 2024-05-22T21:17:03.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.284+0000 7f1339600640 1 --2- 172.21.15.161:0/1284493499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1318066690 0x7f1318068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:03.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.284+0000 7f1339600640 1 -- 172.21.15.161:0/1284493499 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f132c0841f0 con 0x7f133c13cfc0 2024-05-22T21:17:03.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.284+0000 7f1333e00640 1 --2- 172.21.15.161:0/1284493499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1318066690 0x7f1318068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:03.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.284+0000 7f1339600640 1 -- 172.21.15.161:0/1284493499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7f131806e1c0 con 0x7f133c13cfc0 2024-05-22T21:17:03.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.284+0000 7f1333e00640 1 --2- 172.21.15.161:0/1284493499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1318066690 0x7f1318068b50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f13240ad000 tx=0x7f13240a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:03.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.288+0000 7f1339600640 1 -- 172.21.15.161:0/1284493499 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f132c051600 con 0x7f133c13cfc0 2024-05-22T21:17:03.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.476+0000 7f1343a00640 1 -- 172.21.15.161:0/1284493499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f133c071670 con 0x7f133c13cfc0 2024-05-22T21:17:03.480 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:17:03.476+0000 7f7d32800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-22T21:17:03.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.480+0000 7f1339600640 1 -- 172.21.15.161:0/1284493499 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(73..73 src has 1..73) v4 ==== 1554+0+0 (secure 0 0 0) 0x7f132c0552c0 con 0x7f133c13cfc0 2024-05-22T21:17:03.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.480+0000 7f1339600640 1 -- 172.21.15.161:0/1284493499 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7f131806f4f0 con 0x7f133c13cfc0 2024-05-22T21:17:03.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.488+0000 7f1339600640 1 -- 172.21.15.161:0/1284493499 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) v1 ==== 161+0+0 (secure 0 0 0) 0x7f132c021090 con 0x7f133c13cfc0 2024-05-22T21:17:03.492 INFO:teuthology.orchestra.run.smithi161.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-22T21:17:03.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.492+0000 7f1343a00640 1 -- 172.21.15.161:0/1284493499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1318066690 msgr2=0x7f1318068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:03.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.492+0000 7f1343a00640 1 --2- 172.21.15.161:0/1284493499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1318066690 0x7f1318068b50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f13240ad000 tx=0x7f13240a6000 comp rx=0 tx=0).stop 2024-05-22T21:17:03.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.492+0000 7f1343a00640 1 -- 172.21.15.161:0/1284493499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133c13cfc0 msgr2=0x7f133c13e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:03.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.492+0000 7f1343a00640 1 --2- 172.21.15.161:0/1284493499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133c13cfc0 0x7f133c13e5f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f132c00e870 tx=0x7f132c00ec70 comp rx=0 tx=0).stop 2024-05-22T21:17:03.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.492+0000 7f1343a00640 1 -- 172.21.15.161:0/1284493499 shutdown_connections 2024-05-22T21:17:03.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.492+0000 7f1343a00640 1 --2- 172.21.15.161:0/1284493499 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1318066690 0x7f1318068b50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.492+0000 7f1343a00640 1 --2- 172.21.15.161:0/1284493499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f133c15af60 0x7f133c13c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.492+0000 7f1343a00640 1 --2- 172.21.15.161:0/1284493499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f133c13fc80 0x7f133c13c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.492+0000 7f1343a00640 1 --2- 172.21.15.161:0/1284493499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f133c13cfc0 0x7f133c13e5f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.492+0000 7f1343a00640 1 -- 172.21.15.161:0/1284493499 >> 172.21.15.161:0/1284493499 conn(0x7f133c06a2d0 msgr2=0x7f133c06a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:03.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.492+0000 7f1343a00640 1 -- 172.21.15.161:0/1284493499 shutdown_connections 2024-05-22T21:17:03.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.492+0000 7f1343a00640 1 -- 172.21.15.161:0/1284493499 wait complete. 2024-05-22T21:17:03.514 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-22T21:17:03.557 INFO:tasks.ceph.mds.d.smithi179.stdout:starting mds.d at 2024-05-22T21:17:03.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.648+0000 7f6f14c00640 1 Processor -- start 2024-05-22T21:17:03.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.648+0000 7f6f14c00640 1 -- start start 2024-05-22T21:17:03.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.652+0000 7f6f14c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f1006c9b0 0x7f6f100b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:03.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.652+0000 7f6f14c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f10144bb0 con 0x7f6f10143c40 2024-05-22T21:17:03.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.652+0000 7f6f14c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1015efb0 con 0x7f6f1015b600 2024-05-22T21:17:03.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.652+0000 7f6f14c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1015ba60 con 0x7f6f1006c9b0 2024-05-22T21:17:03.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.652+0000 7f6f0de00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f6f10143c40 0x7f6f101444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58674/0 (socket says 172.21.15.161:58674) 2024-05-22T21:17:03.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.652+0000 7f6f0de00640 1 -- 172.21.15.161:0/3822036623 learned_addr learned my addr 172.21.15.161:0/3822036623 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:03.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.652+0000 7f6f0e800640 1 --2- 172.21.15.161:0/3822036623 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f1006c9b0 0x7f6f100b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:03.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.652+0000 7f6f0ca00640 1 -- 172.21.15.161:0/3822036623 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1872802108 0 0) 0x7f6f10144bb0 con 0x7f6f10143c40 2024-05-22T21:17:03.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.652+0000 7f6f0ca00640 1 -- 172.21.15.161:0/3822036623 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ef4003530 con 0x7f6f10143c40 2024-05-22T21:17:03.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.652+0000 7f6f0e800640 1 -- 172.21.15.161:0/3822036623 >> v1:172.21.15.179:6789/0 conn(0x7f6f1015b600 legacy=0x7f6f1006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:03.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.652+0000 7f6f0ca00640 1 -- 172.21.15.161:0/3822036623 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746172272 0 0) 0x7f6f1015efb0 con 0x7f6f1015b600 2024-05-22T21:17:03.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.652+0000 7f6f0e800640 1 -- 172.21.15.161:0/3822036623 >> v1:172.21.15.161:6789/0 conn(0x7f6f10143c40 legacy=0x7f6f101444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:03.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.652+0000 7f6f0e800640 1 -- 172.21.15.161:0/3822036623 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1015f290 con 0x7f6f1006c9b0 2024-05-22T21:17:03.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f0e800640 1 --2- 172.21.15.161:0/3822036623 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f1006c9b0 0x7f6f100b2160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6f080097f0 tx=0x7f6f0801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8aef1ff486a2aed server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:03.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f0ca00640 1 -- 172.21.15.161:0/3822036623 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f080aa640 con 0x7f6f1006c9b0 2024-05-22T21:17:03.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f0ca00640 1 -- 172.21.15.161:0/3822036623 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f080aa7a0 con 0x7f6f1006c9b0 2024-05-22T21:17:03.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f0ca00640 1 -- 172.21.15.161:0/3822036623 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f080aab30 con 0x7f6f1006c9b0 2024-05-22T21:17:03.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f14c00640 1 -- 172.21.15.161:0/3822036623 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f1006c9b0 msgr2=0x7f6f100b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:03.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f14c00640 1 --2- 172.21.15.161:0/3822036623 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f1006c9b0 0x7f6f100b2160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6f080097f0 tx=0x7f6f0801c4e0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f14c00640 1 -- 172.21.15.161:0/3822036623 shutdown_connections 2024-05-22T21:17:03.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f14c00640 1 --2- 172.21.15.161:0/3822036623 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f1006c9b0 0x7f6f100b2160 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f14c00640 1 -- 172.21.15.161:0/3822036623 >> 172.21.15.161:0/3822036623 conn(0x7f6f1006a2d0 msgr2=0x7f6f1006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:03.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f14c00640 1 -- 172.21.15.161:0/3822036623 shutdown_connections 2024-05-22T21:17:03.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f14c00640 1 -- 172.21.15.161:0/3822036623 wait complete. 2024-05-22T21:17:03.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f14c00640 1 Processor -- start 2024-05-22T21:17:03.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f14c00640 1 -- start start 2024-05-22T21:17:03.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f14c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f1006c9b0 0x7f6f1013a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:03.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f14c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f10143c40 0x7f6f10135da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:03.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f14c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f1015b600 0x7f6f1013c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:03.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f14c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f10139f00 con 0x7f6f1006c9b0 2024-05-22T21:17:03.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f14c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1013ceb0 con 0x7f6f1015b600 2024-05-22T21:17:03.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.656+0000 7f6f14c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1013d190 con 0x7f6f10143c40 2024-05-22T21:17:03.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f0de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f1006c9b0 0x7f6f1013a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:03.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f0de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f1006c9b0 0x7f6f1013a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53836/0 (socket says 172.21.15.161:53836) 2024-05-22T21:17:03.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f0de00640 1 -- 172.21.15.161:0/1399028043 learned_addr learned my addr 172.21.15.161:0/1399028043 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:03.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f0e800640 1 --2- 172.21.15.161:0/1399028043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f1015b600 0x7f6f1013c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:03.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f0d400640 1 --2- 172.21.15.161:0/1399028043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f10143c40 0x7f6f10135da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:03.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f0de00640 1 -- 172.21.15.161:0/1399028043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f10143c40 msgr2=0x7f6f10135da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:03.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f0de00640 1 --2- 172.21.15.161:0/1399028043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f10143c40 0x7f6f10135da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f0de00640 1 -- 172.21.15.161:0/1399028043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f1015b600 msgr2=0x7f6f1013c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:03.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f0de00640 1 --2- 172.21.15.161:0/1399028043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f1015b600 0x7f6f1013c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:03.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f0de00640 1 -- 172.21.15.161:0/1399028043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f08009420 con 0x7f6f1006c9b0 2024-05-22T21:17:03.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f0d400640 1 --2- 172.21.15.161:0/1399028043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f10143c40 0x7f6f10135da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:03.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f0de00640 1 --2- 172.21.15.161:0/1399028043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f1006c9b0 0x7f6f1013a740 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000cb40 tx=0x7f6f0000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:03.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f06000640 1 -- 172.21.15.161:0/1399028043 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0000aca0 con 0x7f6f1006c9b0 2024-05-22T21:17:03.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f06000640 1 -- 172.21.15.161:0/1399028043 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0000ae00 con 0x7f6f1006c9b0 2024-05-22T21:17:03.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f14c00640 1 -- 172.21.15.161:0/1399028043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1013d470 con 0x7f6f1006c9b0 2024-05-22T21:17:03.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f06000640 1 -- 172.21.15.161:0/1399028043 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f00012620 con 0x7f6f1006c9b0 2024-05-22T21:17:03.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f14c00640 1 -- 172.21.15.161:0/1399028043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f101362e0 con 0x7f6f1006c9b0 2024-05-22T21:17:03.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f14c00640 1 -- 172.21.15.161:0/1399028043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f10067220 con 0x7f6f1006c9b0 2024-05-22T21:17:03.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f06000640 1 -- 172.21.15.161:0/1399028043 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6f00007070 con 0x7f6f1006c9b0 2024-05-22T21:17:03.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.660+0000 7f6f06000640 1 --2- 172.21.15.161:0/1399028043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ef0066690 0x7f6ef0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:03.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.664+0000 7f6f06000640 1 -- 172.21.15.161:0/1399028043 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6f00084170 con 0x7f6f1006c9b0 2024-05-22T21:17:03.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.664+0000 7f6f0d400640 1 --2- 172.21.15.161:0/1399028043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ef0066690 0x7f6ef0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:03.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.664+0000 7f6f0d400640 1 --2- 172.21.15.161:0/1399028043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ef0066690 0x7f6ef0068b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6f10067b00 tx=0x7f6efc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:03.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.664+0000 7f6f06000640 1 -- 172.21.15.161:0/1399028043 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6f00051530 con 0x7f6f1006c9b0 2024-05-22T21:17:03.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:03.852+0000 7f6f14c00640 1 -- 172.21.15.161:0/1399028043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6f100ceaa0 con 0x7f6f1006c9b0 2024-05-22T21:17:04.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.532+0000 7f6f06000640 1 -- 172.21.15.161:0/1399028043 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6f00054b50 con 0x7f6f1006c9b0 2024-05-22T21:17:04.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.536+0000 7f6f14c00640 1 -- 172.21.15.161:0/1399028043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ef0066690 msgr2=0x7f6ef0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:04.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.536+0000 7f6f14c00640 1 --2- 172.21.15.161:0/1399028043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ef0066690 0x7f6ef0068b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6f10067b00 tx=0x7f6efc009040 comp rx=0 tx=0).stop 2024-05-22T21:17:04.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.536+0000 7f6f14c00640 1 -- 172.21.15.161:0/1399028043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f1006c9b0 msgr2=0x7f6f1013a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:04.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.536+0000 7f6f14c00640 1 --2- 172.21.15.161:0/1399028043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f1006c9b0 0x7f6f1013a740 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000cb40 tx=0x7f6f0000a060 comp rx=0 tx=0).stop 2024-05-22T21:17:04.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.536+0000 7f6f14c00640 1 -- 172.21.15.161:0/1399028043 shutdown_connections 2024-05-22T21:17:04.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.536+0000 7f6f14c00640 1 --2- 172.21.15.161:0/1399028043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ef0066690 0x7f6ef0068b50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:04.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.536+0000 7f6f14c00640 1 --2- 172.21.15.161:0/1399028043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f1015b600 0x7f6f1013c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:04.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.536+0000 7f6f14c00640 1 --2- 172.21.15.161:0/1399028043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f10143c40 0x7f6f10135da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:04.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.536+0000 7f6f14c00640 1 --2- 172.21.15.161:0/1399028043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f1006c9b0 0x7f6f1013a740 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:04.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.536+0000 7f6f14c00640 1 -- 172.21.15.161:0/1399028043 >> 172.21.15.161:0/1399028043 conn(0x7f6f1006a2d0 msgr2=0x7f6f101530d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:04.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.536+0000 7f6f14c00640 1 -- 172.21.15.161:0/1399028043 shutdown_connections 2024-05-22T21:17:04.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.536+0000 7f6f14c00640 1 -- 172.21.15.161:0/1399028043 wait complete. 2024-05-22T21:17:04.561 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-22T21:17:04.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.700+0000 7fb0f3600640 1 Processor -- start 2024-05-22T21:17:04.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.700+0000 7fb0f3600640 1 -- start start 2024-05-22T21:17:04.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.700+0000 7fb0f3600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb0ec143630 0x7fb0ec06d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:04.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.700+0000 7fb0f3600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb0ec143a10 0x7fb0ec1619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:04.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.700+0000 7fb0f3600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0ec06d290 con 0x7fb0ec0ad560 2024-05-22T21:17:04.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.700+0000 7fb0f3600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ec0ad990 con 0x7fb0ec143630 2024-05-22T21:17:04.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.700+0000 7fb0f3600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ec0adf60 con 0x7fb0ec143a10 2024-05-22T21:17:04.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.700+0000 7fb0f0c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb0ec143630 0x7fb0ec06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:04.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.704+0000 7fb0f0c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb0ec143630 0x7fb0ec06d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53842/0 (socket says 172.21.15.161:53842) 2024-05-22T21:17:04.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.704+0000 7fb0f0c00640 1 -- 172.21.15.161:0/3158074421 learned_addr learned my addr 172.21.15.161:0/3158074421 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:04.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.704+0000 7fb0f1600640 1 --1- 172.21.15.161:0/3158074421 >> v1:172.21.15.161:6789/0 conn(0x7fb0ec0ad560 0x7fb0ec160e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58684/0 (socket says 172.21.15.161:58684) 2024-05-22T21:17:04.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.704+0000 7fb0e3e00640 1 --2- 172.21.15.161:0/3158074421 >> v2:172.21.15.179:3300/0 conn(0x7fb0ec143a10 0x7fb0ec1619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:04.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.704+0000 7fb0f0c00640 1 -- 172.21.15.161:0/3158074421 >> v2:172.21.15.179:3300/0 conn(0x7fb0ec143a10 msgr2=0x7fb0ec1619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:04.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.704+0000 7fb0f0c00640 1 --2- 172.21.15.161:0/3158074421 >> v2:172.21.15.179:3300/0 conn(0x7fb0ec143a10 0x7fb0ec1619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:04.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.704+0000 7fb0f0c00640 1 -- 172.21.15.161:0/3158074421 >> v1:172.21.15.161:6789/0 conn(0x7fb0ec0ad560 legacy=0x7fb0ec160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:04.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.704+0000 7fb0f0c00640 1 -- 172.21.15.161:0/3158074421 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec06cc40 con 0x7fb0ec143630 2024-05-22T21:17:04.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.704+0000 7fb0f0c00640 1 --2- 172.21.15.161:0/3158074421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ec143630 0x7fb0ec06d720 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec1470b0 tx=0x7fb0dc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c076f3be8120467a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:04.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.704+0000 7fb0ebe00640 1 -- 172.21.15.161:0/3158074421 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc0ab650 con 0x7fb0ec143630 2024-05-22T21:17:04.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.704+0000 7fb0ebe00640 1 -- 172.21.15.161:0/3158074421 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0dc0ab7b0 con 0x7fb0ec143630 2024-05-22T21:17:04.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.704+0000 7fb0ebe00640 1 -- 172.21.15.161:0/3158074421 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc0abb40 con 0x7fb0ec143630 2024-05-22T21:17:04.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f3600640 1 -- 172.21.15.161:0/3158074421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ec143630 msgr2=0x7fb0ec06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:04.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f3600640 1 --2- 172.21.15.161:0/3158074421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ec143630 0x7fb0ec06d720 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec1470b0 tx=0x7fb0dc01c4f0 comp rx=0 tx=0).stop 2024-05-22T21:17:04.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f3600640 1 -- 172.21.15.161:0/3158074421 shutdown_connections 2024-05-22T21:17:04.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f3600640 1 --2- 172.21.15.161:0/3158074421 >> v2:172.21.15.179:3300/0 conn(0x7fb0ec143a10 0x7fb0ec1619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:04.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f3600640 1 --2- 172.21.15.161:0/3158074421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ec143630 0x7fb0ec06d720 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:04.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f3600640 1 -- 172.21.15.161:0/3158074421 >> 172.21.15.161:0/3158074421 conn(0x7fb0ec06a0a0 msgr2=0x7fb0ec06a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:04.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f3600640 1 -- 172.21.15.161:0/3158074421 shutdown_connections 2024-05-22T21:17:04.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f3600640 1 -- 172.21.15.161:0/3158074421 wait complete. 2024-05-22T21:17:04.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f3600640 1 Processor -- start 2024-05-22T21:17:04.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f3600640 1 -- start start 2024-05-22T21:17:04.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f3600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0ec0ad560 0x7fb0ec13b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:04.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f3600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0ec143630 0x7fb0ec15a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:04.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f3600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ec143a10 0x7fb0ec13a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f3600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec15ee40 con 0x7fb0ec143a10 2024-05-22T21:17:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f3600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec13a880 con 0x7fb0ec143630 2024-05-22T21:17:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f3600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec13ab60 con 0x7fb0ec0ad560 2024-05-22T21:17:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f1600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ec143a10 0x7fb0ec13a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f1600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ec143a10 0x7fb0ec13a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53850/0 (socket says 172.21.15.161:53850) 2024-05-22T21:17:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f1600640 1 -- 172.21.15.161:0/2804426503 learned_addr learned my addr 172.21.15.161:0/2804426503 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f1600640 1 -- 172.21.15.161:0/2804426503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0ec0ad560 msgr2=0x7fb0ec13b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f0c00640 1 --2- 172.21.15.161:0/2804426503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0ec0ad560 0x7fb0ec13b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f1600640 1 --2- 172.21.15.161:0/2804426503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0ec0ad560 0x7fb0ec13b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0e3e00640 1 --2- 172.21.15.161:0/2804426503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0ec143630 0x7fb0ec15a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f1600640 1 -- 172.21.15.161:0/2804426503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0ec143630 msgr2=0x7fb0ec15a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f1600640 1 --2- 172.21.15.161:0/2804426503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0ec143630 0x7fb0ec15a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:04.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f1600640 1 -- 172.21.15.161:0/2804426503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc0094d0 con 0x7fb0ec143a10 2024-05-22T21:17:04.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0e3e00640 1 --2- 172.21.15.161:0/2804426503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0ec143630 0x7fb0ec15a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:17:04.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f0c00640 1 --2- 172.21.15.161:0/2804426503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0ec0ad560 0x7fb0ec13b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:17:04.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0f1600640 1 --2- 172.21.15.161:0/2804426503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ec143a10 0x7fb0ec13a050 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400c7a0 tx=0x7fb0e400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:04.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.708+0000 7fb0e9600640 1 -- 172.21.15.161:0/2804426503 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400a640 con 0x7fb0ec143a10 2024-05-22T21:17:04.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.712+0000 7fb0e9600640 1 -- 172.21.15.161:0/2804426503 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e400a7a0 con 0x7fb0ec143a10 2024-05-22T21:17:04.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.712+0000 7fb0f3600640 1 -- 172.21.15.161:0/2804426503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ec13ae40 con 0x7fb0ec143a10 2024-05-22T21:17:04.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.712+0000 7fb0e9600640 1 -- 172.21.15.161:0/2804426503 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400ab10 con 0x7fb0ec143a10 2024-05-22T21:17:04.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.712+0000 7fb0f3600640 1 -- 172.21.15.161:0/2804426503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ec13a590 con 0x7fb0ec143a10 2024-05-22T21:17:04.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.712+0000 7fb0f3600640 1 -- 172.21.15.161:0/2804426503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ec15efe0 con 0x7fb0ec143a10 2024-05-22T21:17:04.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.712+0000 7fb0e9600640 1 -- 172.21.15.161:0/2804426503 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb0e400ac70 con 0x7fb0ec143a10 2024-05-22T21:17:04.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.712+0000 7fb0e9600640 1 --2- 172.21.15.161:0/2804426503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0c8066690 0x7fb0c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:04.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.712+0000 7fb0e9600640 1 -- 172.21.15.161:0/2804426503 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb0e4083a60 con 0x7fb0ec143a10 2024-05-22T21:17:04.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.712+0000 7fb0f0c00640 1 --2- 172.21.15.161:0/2804426503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0c8066690 0x7fb0c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:04.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.716+0000 7fb0f0c00640 1 --2- 172.21.15.161:0/2804426503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0c8066690 0x7fb0c8068b50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0af000 tx=0x7fb0dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:04.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.716+0000 7fb0e9600640 1 -- 172.21.15.161:0/2804426503 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb0e4050d80 con 0x7fb0ec143a10 2024-05-22T21:17:04.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:04.872+0000 7fb0f3600640 1 -- 172.21.15.161:0/2804426503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb0ec067220 con 0x7fb0ec143a10 2024-05-22T21:17:05.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.544+0000 7fb0e9600640 1 -- 172.21.15.161:0/2804426503 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb0e4017020 con 0x7fb0ec143a10 2024-05-22T21:17:05.548 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' removed 2024-05-22T21:17:05.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.548+0000 7fb0f3600640 1 -- 172.21.15.161:0/2804426503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0c8066690 msgr2=0x7fb0c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:05.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.548+0000 7fb0f3600640 1 --2- 172.21.15.161:0/2804426503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0c8066690 0x7fb0c8068b50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0af000 tx=0x7fb0dc008040 comp rx=0 tx=0).stop 2024-05-22T21:17:05.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.548+0000 7fb0f3600640 1 -- 172.21.15.161:0/2804426503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ec143a10 msgr2=0x7fb0ec13a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:05.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.548+0000 7fb0f3600640 1 --2- 172.21.15.161:0/2804426503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ec143a10 0x7fb0ec13a050 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400c7a0 tx=0x7fb0e400cc60 comp rx=0 tx=0).stop 2024-05-22T21:17:05.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.548+0000 7fb0f3600640 1 -- 172.21.15.161:0/2804426503 shutdown_connections 2024-05-22T21:17:05.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.548+0000 7fb0f3600640 1 --2- 172.21.15.161:0/2804426503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0c8066690 0x7fb0c8068b50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:05.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.548+0000 7fb0f3600640 1 --2- 172.21.15.161:0/2804426503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ec143a10 0x7fb0ec13a050 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:05.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.548+0000 7fb0f3600640 1 --2- 172.21.15.161:0/2804426503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0ec143630 0x7fb0ec15a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:05.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.548+0000 7fb0f3600640 1 --2- 172.21.15.161:0/2804426503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0ec0ad560 0x7fb0ec13b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:05.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.548+0000 7fb0f3600640 1 -- 172.21.15.161:0/2804426503 >> 172.21.15.161:0/2804426503 conn(0x7fb0ec06a0a0 msgr2=0x7fb0ec0b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:05.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.548+0000 7fb0f3600640 1 -- 172.21.15.161:0/2804426503 shutdown_connections 2024-05-22T21:17:05.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.548+0000 7fb0f3600640 1 -- 172.21.15.161:0/2804426503 wait complete. 2024-05-22T21:17:05.572 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-22T21:17:05.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.708+0000 7fa080c00640 1 Processor -- start 2024-05-22T21:17:05.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.708+0000 7fa080c00640 1 -- start start 2024-05-22T21:17:05.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa080c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa07c13cfe0 0x7fa07c15dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:05.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa080c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa07c0ce830 con 0x7fa07c13d560 2024-05-22T21:17:05.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa080c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa07c0ceb10 con 0x7fa07c0ce210 2024-05-22T21:17:05.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa080c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa07c140560 con 0x7fa07c13cfe0 2024-05-22T21:17:05.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa071400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa07c13d560 0x7fa07c0cdb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58690/0 (socket says 172.21.15.161:58690) 2024-05-22T21:17:05.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa071400640 1 -- 172.21.15.161:0/368925946 learned_addr learned my addr 172.21.15.161:0/368925946 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:05.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079400640 1 -- 172.21.15.161:0/368925946 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050548862 0 0) 0x7fa07c0ce830 con 0x7fa07c13d560 2024-05-22T21:17:05.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079400640 1 -- 172.21.15.161:0/368925946 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa060005740 con 0x7fa07c13d560 2024-05-22T21:17:05.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079e00640 1 --2- 172.21.15.161:0/368925946 >> v2:172.21.15.179:3300/0 conn(0x7fa07c13cfe0 0x7fa07c15dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:05.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079400640 1 -- 172.21.15.161:0/368925946 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3068600335 0 0) 0x7fa060005740 con 0x7fa07c13d560 2024-05-22T21:17:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079400640 1 -- 172.21.15.161:0/368925946 >> v2:172.21.15.179:3300/0 conn(0x7fa07c13cfe0 msgr2=0x7fa07c15dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079400640 1 --2- 172.21.15.161:0/368925946 >> v2:172.21.15.179:3300/0 conn(0x7fa07c13cfe0 0x7fa07c15dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079400640 1 -- 172.21.15.161:0/368925946 >> v1:172.21.15.179:6789/0 conn(0x7fa07c0ce210 legacy=0x7fa07c070860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079400640 1 -- 172.21.15.161:0/368925946 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c06fd80 con 0x7fa07c13d560 2024-05-22T21:17:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079400640 1 -- 172.21.15.161:0/368925946 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fa064001680 con 0x7fa07c13d560 2024-05-22T21:17:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079400640 1 -- 172.21.15.161:0/368925946 >> v1:172.21.15.161:6789/0 conn(0x7fa07c13d560 legacy=0x7fa07c0cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079400640 1 --2- 172.21.15.161:0/368925946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa060005b90 0x7fa060005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079400640 1 --2- 172.21.15.161:0/368925946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0600069e0 0x7fa060006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079400640 1 --2- 172.21.15.161:0/368925946 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa0600074c0 0x7fa06000dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079400640 1 -- 172.21.15.161:0/368925946 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c140560 con 0x7fa060005b90 2024-05-22T21:17:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079400640 1 -- 172.21.15.161:0/368925946 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06000e390 con 0x7fa0600069e0 2024-05-22T21:17:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079400640 1 -- 172.21.15.161:0/368925946 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06000e550 con 0x7fa0600074c0 2024-05-22T21:17:05.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079e00640 1 --2- 172.21.15.161:0/368925946 >> v2:172.21.15.179:3300/0 conn(0x7fa07c13cfe0 0x7fa07c15dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:05.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa071400640 1 --2- 172.21.15.161:0/368925946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0600069e0 0x7fa060006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:05.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079e00640 1 --2- 172.21.15.161:0/368925946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa060005b90 0x7fa060005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:05.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa07a800640 1 --2- 172.21.15.161:0/368925946 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa0600074c0 0x7fa06000dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:05.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079e00640 1 -- 172.21.15.161:0/368925946 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa0600074c0 msgr2=0x7fa06000dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:05.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079e00640 1 --2- 172.21.15.161:0/368925946 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa0600074c0 0x7fa06000dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:05.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079e00640 1 -- 172.21.15.161:0/368925946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0600069e0 msgr2=0x7fa060006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:05.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079e00640 1 --2- 172.21.15.161:0/368925946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0600069e0 0x7fa060006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:05.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa079e00640 1 -- 172.21.15.161:0/368925946 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa060005740 con 0x7fa060005b90 2024-05-22T21:17:05.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.712+0000 7fa07a800640 1 -- 172.21.15.161:0/368925946 reap_dead start 2024-05-22T21:17:05.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.716+0000 7fa079e00640 1 --2- 172.21.15.161:0/368925946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa060005b90 0x7fa060005fb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009880 tx=0x7fa06c01c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c919faf26d3e38 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:05.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.716+0000 7fa079400640 1 -- 172.21.15.161:0/368925946 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c0ae070 con 0x7fa060005b90 2024-05-22T21:17:05.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.716+0000 7fa079400640 1 -- 172.21.15.161:0/368925946 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06c004020 con 0x7fa060005b90 2024-05-22T21:17:05.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.716+0000 7fa079400640 1 -- 172.21.15.161:0/368925946 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c0043b0 con 0x7fa060005b90 2024-05-22T21:17:05.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.716+0000 7fa080c00640 1 -- 172.21.15.161:0/368925946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa060005b90 msgr2=0x7fa060005fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:05.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.716+0000 7fa080c00640 1 --2- 172.21.15.161:0/368925946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa060005b90 0x7fa060005fb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009880 tx=0x7fa06c01c550 comp rx=0 tx=0).stop 2024-05-22T21:17:05.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.716+0000 7fa080c00640 1 -- 172.21.15.161:0/368925946 shutdown_connections 2024-05-22T21:17:05.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.716+0000 7fa080c00640 1 --2- 172.21.15.161:0/368925946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa060005b90 0x7fa060005fb0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:05.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.716+0000 7fa080c00640 1 -- 172.21.15.161:0/368925946 >> 172.21.15.161:0/368925946 conn(0x7fa07c06a0a0 msgr2=0x7fa07c06a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:05.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.716+0000 7fa080c00640 1 -- 172.21.15.161:0/368925946 shutdown_connections 2024-05-22T21:17:05.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.716+0000 7fa080c00640 1 -- 172.21.15.161:0/368925946 wait complete. 2024-05-22T21:17:05.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.716+0000 7fa080c00640 1 Processor -- start 2024-05-22T21:17:05.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.716+0000 7fa080c00640 1 -- start start 2024-05-22T21:17:05.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa080c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa060005b90 0x7fa07c1f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:05.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa080c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa07c0ac8b0 0x7fa07c1e6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:05.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa080c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa07c0acd30 0x7fa07c1e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:05.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa080c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c0adf10 con 0x7fa07c0acd30 2024-05-22T21:17:05.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa080c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c1f6c80 con 0x7fa07c0ac8b0 2024-05-22T21:17:05.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa080c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c0ad300 con 0x7fa060005b90 2024-05-22T21:17:05.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa07a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa07c0acd30 0x7fa07c1e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:05.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa07a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa07c0acd30 0x7fa07c1e6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53878/0 (socket says 172.21.15.161:53878) 2024-05-22T21:17:05.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa07a800640 1 -- 172.21.15.161:0/3107509778 learned_addr learned my addr 172.21.15.161:0/3107509778 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:05.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa079e00640 1 --2- 172.21.15.161:0/3107509778 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa060005b90 0x7fa07c1f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:05.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa071400640 1 --2- 172.21.15.161:0/3107509778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa07c0ac8b0 0x7fa07c1e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:05.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa07a800640 1 -- 172.21.15.161:0/3107509778 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa060005b90 msgr2=0x7fa07c1f6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:05.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa07a800640 1 --2- 172.21.15.161:0/3107509778 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa060005b90 0x7fa07c1f6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:05.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa07a800640 1 -- 172.21.15.161:0/3107509778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa07c0ac8b0 msgr2=0x7fa07c1e6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:05.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa07a800640 1 --2- 172.21.15.161:0/3107509778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa07c0ac8b0 0x7fa07c1e6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:05.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa07a800640 1 -- 172.21.15.161:0/3107509778 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c0ad5e0 con 0x7fa07c0acd30 2024-05-22T21:17:05.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa07a800640 1 --2- 172.21.15.161:0/3107509778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa07c0acd30 0x7fa07c1e6cc0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa07c06dc10 tx=0x7fa07400fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:05.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa072a00640 1 -- 172.21.15.161:0/3107509778 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07400fec0 con 0x7fa07c0acd30 2024-05-22T21:17:05.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa072a00640 1 -- 172.21.15.161:0/3107509778 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa074003b90 con 0x7fa07c0acd30 2024-05-22T21:17:05.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa072a00640 1 -- 172.21.15.161:0/3107509778 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0740050d0 con 0x7fa07c0acd30 2024-05-22T21:17:05.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa080c00640 1 -- 172.21.15.161:0/3107509778 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07c0ad8c0 con 0x7fa07c0acd30 2024-05-22T21:17:05.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa080c00640 1 -- 172.21.15.161:0/3107509778 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07c1f5f80 con 0x7fa07c0acd30 2024-05-22T21:17:05.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.720+0000 7fa080c00640 1 -- 172.21.15.161:0/3107509778 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07c0ae0c0 con 0x7fa07c0acd30 2024-05-22T21:17:05.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.724+0000 7fa072a00640 1 -- 172.21.15.161:0/3107509778 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa074005230 con 0x7fa07c0acd30 2024-05-22T21:17:05.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.724+0000 7fa072a00640 1 --2- 172.21.15.161:0/3107509778 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0580666c0 0x7fa058068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:05.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.724+0000 7fa072a00640 1 -- 172.21.15.161:0/3107509778 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fa074083e30 con 0x7fa07c0acd30 2024-05-22T21:17:05.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.724+0000 7fa079e00640 1 --2- 172.21.15.161:0/3107509778 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0580666c0 0x7fa058068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:05.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.724+0000 7fa079e00640 1 --2- 172.21.15.161:0/3107509778 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0580666c0 0x7fa058068b80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0ad000 tx=0x7fa06c0046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:05.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.728+0000 7fa072a00640 1 -- 172.21.15.161:0/3107509778 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa07c0ae0c0 con 0x7fa07c0acd30 2024-05-22T21:17:05.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:05.884+0000 7fa080c00640 1 -- 172.21.15.161:0/3107509778 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa07c0ae0c0 con 0x7fa07c0acd30 2024-05-22T21:17:06.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.548+0000 7fa072a00640 1 -- 172.21.15.161:0/3107509778 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa07c0ae0c0 con 0x7fa07c0acd30 2024-05-22T21:17:06.553 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' removed 2024-05-22T21:17:06.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.552+0000 7fa080c00640 1 -- 172.21.15.161:0/3107509778 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0580666c0 msgr2=0x7fa058068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:06.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.552+0000 7fa080c00640 1 --2- 172.21.15.161:0/3107509778 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0580666c0 0x7fa058068b80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0ad000 tx=0x7fa06c0046d0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.552+0000 7fa080c00640 1 -- 172.21.15.161:0/3107509778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa07c0acd30 msgr2=0x7fa07c1e6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:06.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.552+0000 7fa080c00640 1 --2- 172.21.15.161:0/3107509778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa07c0acd30 0x7fa07c1e6cc0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa07c06dc10 tx=0x7fa07400fbb0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.552+0000 7fa080c00640 1 -- 172.21.15.161:0/3107509778 shutdown_connections 2024-05-22T21:17:06.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.552+0000 7fa080c00640 1 --2- 172.21.15.161:0/3107509778 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0580666c0 0x7fa058068b80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.552+0000 7fa080c00640 1 --2- 172.21.15.161:0/3107509778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa07c0acd30 0x7fa07c1e6cc0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.552+0000 7fa080c00640 1 --2- 172.21.15.161:0/3107509778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa07c0ac8b0 0x7fa07c1e6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.552+0000 7fa080c00640 1 --2- 172.21.15.161:0/3107509778 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa060005b90 0x7fa07c1f6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.552+0000 7fa080c00640 1 -- 172.21.15.161:0/3107509778 >> 172.21.15.161:0/3107509778 conn(0x7fa07c06a0a0 msgr2=0x7fa07c06c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:06.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.552+0000 7fa080c00640 1 -- 172.21.15.161:0/3107509778 shutdown_connections 2024-05-22T21:17:06.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.552+0000 7fa080c00640 1 -- 172.21.15.161:0/3107509778 wait complete. 2024-05-22T21:17:06.575 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-22T21:17:06.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f7022a00640 1 Processor -- start 2024-05-22T21:17:06.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f7022a00640 1 -- start start 2024-05-22T21:17:06.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f7022a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f701c143c40 0x7f701c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:06.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f7022a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f701c06c9b0 0x7f701c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:06.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f7022a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701c15efb0 con 0x7f701c1444a0 2024-05-22T21:17:06.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f7022a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f701c15f290 con 0x7f701c06c9b0 2024-05-22T21:17:06.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f7022a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701c15f570 con 0x7f701c143c40 2024-05-22T21:17:06.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f7013400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f701c1444a0 0x7f701c06c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54400/0 (socket says 172.21.15.161:54400) 2024-05-22T21:17:06.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f7013400640 1 -- 172.21.15.161:0/556711498 learned_addr learned my addr 172.21.15.161:0/556711498 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:06.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701be00640 1 --2- 172.21.15.161:0/556711498 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f701c143c40 0x7f701c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:06.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f7020a00640 1 --2- 172.21.15.161:0/556711498 >> v2:172.21.15.179:3300/0 conn(0x7f701c06c9b0 0x7f701c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:06.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701b400640 1 -- 172.21.15.161:0/556711498 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142114948 0 0) 0x7f701c15efb0 con 0x7f701c1444a0 2024-05-22T21:17:06.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701b400640 1 -- 172.21.15.161:0/556711498 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7000003530 con 0x7f701c1444a0 2024-05-22T21:17:06.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701b400640 1 -- 172.21.15.161:0/556711498 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2217044517 0 0) 0x7f7000003530 con 0x7f701c1444a0 2024-05-22T21:17:06.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701b400640 1 -- 172.21.15.161:0/556711498 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f701c143c40 msgr2=0x7f701c15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:06.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701b400640 1 --2- 172.21.15.161:0/556711498 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f701c143c40 0x7f701c15b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701b400640 1 -- 172.21.15.161:0/556711498 >> v2:172.21.15.179:3300/0 conn(0x7f701c06c9b0 msgr2=0x7f701c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:06.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701b400640 1 --2- 172.21.15.161:0/556711498 >> v2:172.21.15.179:3300/0 conn(0x7f701c06c9b0 0x7f701c0b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701b400640 1 -- 172.21.15.161:0/556711498 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c15f850 con 0x7f701c1444a0 2024-05-22T21:17:06.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701b400640 1 -- 172.21.15.161:0/556711498 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f7004001680 con 0x7f701c1444a0 2024-05-22T21:17:06.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701b400640 1 -- 172.21.15.161:0/556711498 >> v1:172.21.15.161:6789/0 conn(0x7f701c1444a0 legacy=0x7f701c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:06.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701b400640 1 --2- 172.21.15.161:0/556711498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7000003980 0x7f7000003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:06.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701b400640 1 --2- 172.21.15.161:0/556711498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70000047d0 0x7f7000004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:06.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701b400640 1 --2- 172.21.15.161:0/556711498 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f701c143c40 0x7f7000005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:06.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701b400640 1 -- 172.21.15.161:0/556711498 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c15f290 con 0x7f7000003980 2024-05-22T21:17:06.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701b400640 1 -- 172.21.15.161:0/556711498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c15f570 con 0x7f70000047d0 2024-05-22T21:17:06.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701b400640 1 -- 172.21.15.161:0/556711498 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700000b990 con 0x7f701c143c40 2024-05-22T21:17:06.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701be00640 1 --2- 172.21.15.161:0/556711498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7000003980 0x7f7000003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:06.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f7013400640 1 --2- 172.21.15.161:0/556711498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70000047d0 0x7f7000004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:06.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f7020a00640 1 --2- 172.21.15.161:0/556711498 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f701c143c40 0x7f7000005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:06.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701be00640 1 -- 172.21.15.161:0/556711498 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f701c143c40 msgr2=0x7f7000005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:06.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701be00640 1 --2- 172.21.15.161:0/556711498 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f701c143c40 0x7f7000005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701be00640 1 -- 172.21.15.161:0/556711498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70000047d0 msgr2=0x7f7000004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:06.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701be00640 1 --2- 172.21.15.161:0/556711498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70000047d0 0x7f7000004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f701be00640 1 -- 172.21.15.161:0/556711498 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7000003530 con 0x7f7000003980 2024-05-22T21:17:06.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.712+0000 7f7013400640 1 --2- 172.21.15.161:0/556711498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70000047d0 0x7f7000004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:06.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.716+0000 7f701be00640 1 --2- 172.21.15.161:0/556711498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7000003980 0x7f7000003da0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f700c009fc0 tx=0x7f700c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c855307180745fd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:06.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.716+0000 7f701b400640 1 -- 172.21.15.161:0/556711498 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c0a5af0 con 0x7f7000003980 2024-05-22T21:17:06.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.716+0000 7f701b400640 1 -- 172.21.15.161:0/556711498 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700c00f040 con 0x7f7000003980 2024-05-22T21:17:06.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.716+0000 7f701b400640 1 -- 172.21.15.161:0/556711498 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c0a5e90 con 0x7f7000003980 2024-05-22T21:17:06.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.716+0000 7f7022a00640 1 -- 172.21.15.161:0/556711498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7000003980 msgr2=0x7f7000003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:06.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.716+0000 7f7022a00640 1 --2- 172.21.15.161:0/556711498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7000003980 0x7f7000003da0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f700c009fc0 tx=0x7f700c01cc80 comp rx=0 tx=0).stop 2024-05-22T21:17:06.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.716+0000 7f7020a00640 1 -- 172.21.15.161:0/556711498 reap_dead start 2024-05-22T21:17:06.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.716+0000 7f7022a00640 1 -- 172.21.15.161:0/556711498 shutdown_connections 2024-05-22T21:17:06.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.716+0000 7f7022a00640 1 -- 172.21.15.161:0/556711498 >> 172.21.15.161:0/556711498 conn(0x7f701c06a2d0 msgr2=0x7f701c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:06.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.716+0000 7f7022a00640 1 -- 172.21.15.161:0/556711498 shutdown_connections 2024-05-22T21:17:06.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.716+0000 7f7022a00640 1 -- 172.21.15.161:0/556711498 wait complete. 2024-05-22T21:17:06.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7022a00640 1 Processor -- start 2024-05-22T21:17:06.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7022a00640 1 -- start start 2024-05-22T21:17:06.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7022a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7000003980 0x7f701c15e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:06.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7022a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f701c15a270 0x7f701c164410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:06.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7022a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f701c15dc80 0x7f701c159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:06.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7022a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c15d860 con 0x7f701c15a270 2024-05-22T21:17:06.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7022a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c15cbe0 con 0x7f7000003980 2024-05-22T21:17:06.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7022a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701c15cec0 con 0x7f701c15dc80 2024-05-22T21:17:06.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7013400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f701c15a270 0x7f701c164410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:06.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7013400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f701c15a270 0x7f701c164410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33438/0 (socket says 172.21.15.161:33438) 2024-05-22T21:17:06.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7013400640 1 -- 172.21.15.161:0/81481954 learned_addr learned my addr 172.21.15.161:0/81481954 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:06.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f701be00640 1 --2- 172.21.15.161:0/81481954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7000003980 0x7f701c15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:06.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7020a00640 1 --2- 172.21.15.161:0/81481954 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f701c15dc80 0x7f701c159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:06.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7013400640 1 -- 172.21.15.161:0/81481954 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f701c15dc80 msgr2=0x7f701c159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:06.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7013400640 1 --2- 172.21.15.161:0/81481954 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f701c15dc80 0x7f701c159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7013400640 1 -- 172.21.15.161:0/81481954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7000003980 msgr2=0x7f701c15e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:06.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7013400640 1 --2- 172.21.15.161:0/81481954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7000003980 0x7f701c15e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7013400640 1 -- 172.21.15.161:0/81481954 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c15d1a0 con 0x7f701c15a270 2024-05-22T21:17:06.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7013400640 1 --2- 172.21.15.161:0/81481954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f701c15a270 0x7f701c164410 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7004001000 tx=0x7f700400eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:06.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7018c00640 1 -- 172.21.15.161:0/81481954 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700400c690 con 0x7f701c15a270 2024-05-22T21:17:06.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7018c00640 1 -- 172.21.15.161:0/81481954 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700400c7f0 con 0x7f701c15a270 2024-05-22T21:17:06.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7022a00640 1 -- 172.21.15.161:0/81481954 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701c15d480 con 0x7f701c15a270 2024-05-22T21:17:06.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7018c00640 1 -- 172.21.15.161:0/81481954 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700400cb60 con 0x7f701c15a270 2024-05-22T21:17:06.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.720+0000 7f7022a00640 1 -- 172.21.15.161:0/81481954 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c164b20 con 0x7f701c15a270 2024-05-22T21:17:06.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.724+0000 7f7022a00640 1 -- 172.21.15.161:0/81481954 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f701c15ec20 con 0x7f701c15a270 2024-05-22T21:17:06.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.724+0000 7f7018c00640 1 -- 172.21.15.161:0/81481954 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f700400ccc0 con 0x7f701c15a270 2024-05-22T21:17:06.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.724+0000 7f7018c00640 1 --2- 172.21.15.161:0/81481954 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ff8066690 0x7f6ff8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:06.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.724+0000 7f7018c00640 1 -- 172.21.15.161:0/81481954 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7004083c70 con 0x7f701c15a270 2024-05-22T21:17:06.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.724+0000 7f701be00640 1 --2- 172.21.15.161:0/81481954 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ff8066690 0x7f6ff8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:06.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.724+0000 7f701be00640 1 --2- 172.21.15.161:0/81481954 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ff8066690 0x7f6ff8068b50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f700c0af000 tx=0x7f700c004ac0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:06.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.728+0000 7f7018c00640 1 -- 172.21.15.161:0/81481954 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f701c15ec20 con 0x7f701c15a270 2024-05-22T21:17:06.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.884+0000 7f7022a00640 1 -- 172.21.15.161:0/81481954 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f701c15ec20 con 0x7f701c15a270 2024-05-22T21:17:06.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.892+0000 7f7018c00640 1 -- 172.21.15.161:0/81481954 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f701c15ec20 con 0x7f701c15a270 2024-05-22T21:17:06.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.892+0000 7f7022a00640 1 -- 172.21.15.161:0/81481954 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ff8066690 msgr2=0x7f6ff8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:06.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.892+0000 7f7022a00640 1 --2- 172.21.15.161:0/81481954 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ff8066690 0x7f6ff8068b50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f700c0af000 tx=0x7f700c004ac0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.892+0000 7f7022a00640 1 -- 172.21.15.161:0/81481954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f701c15a270 msgr2=0x7f701c164410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:06.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.892+0000 7f7022a00640 1 --2- 172.21.15.161:0/81481954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f701c15a270 0x7f701c164410 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7004001000 tx=0x7f700400eba0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.892+0000 7f7022a00640 1 -- 172.21.15.161:0/81481954 shutdown_connections 2024-05-22T21:17:06.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.892+0000 7f7022a00640 1 --2- 172.21.15.161:0/81481954 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6ff8066690 0x7f6ff8068b50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.892+0000 7f7022a00640 1 --2- 172.21.15.161:0/81481954 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f701c15dc80 0x7f701c159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.892+0000 7f7022a00640 1 --2- 172.21.15.161:0/81481954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f701c15a270 0x7f701c164410 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.892+0000 7f7022a00640 1 --2- 172.21.15.161:0/81481954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7000003980 0x7f701c15e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:06.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.892+0000 7f7022a00640 1 -- 172.21.15.161:0/81481954 >> 172.21.15.161:0/81481954 conn(0x7f701c06a2d0 msgr2=0x7f701c06c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:06.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.892+0000 7f7022a00640 1 -- 172.21.15.161:0/81481954 shutdown_connections 2024-05-22T21:17:06.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:06.892+0000 7f7022a00640 1 -- 172.21.15.161:0/81481954 wait complete. 2024-05-22T21:17:06.916 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2024-05-22T21:17:07.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.048+0000 7fda42600640 1 Processor -- start 2024-05-22T21:17:07.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.048+0000 7fda42600640 1 -- start start 2024-05-22T21:17:07.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.048+0000 7fda42600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fda3c15e130 0x7fda3c15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:07.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.048+0000 7fda42600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda3c15d410 0x7fda3c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:07.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.048+0000 7fda42600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda3c071670 con 0x7fda3c15dd50 2024-05-22T21:17:07.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.048+0000 7fda42600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda3c15d870 con 0x7fda3c15e130 2024-05-22T21:17:07.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.048+0000 7fda42600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c070a60 con 0x7fda3c15d410 2024-05-22T21:17:07.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.052+0000 7fda3aa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fda3c15e130 0x7fda3c15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:07.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.052+0000 7fda3aa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fda3c15e130 0x7fda3c15ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33448/0 (socket says 172.21.15.161:33448) 2024-05-22T21:17:07.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.052+0000 7fda3aa00640 1 -- 172.21.15.161:0/1574163565 learned_addr learned my addr 172.21.15.161:0/1574163565 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:07.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.052+0000 7fda3be00640 1 --2- 172.21.15.161:0/1574163565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda3c15d410 0x7fda3c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:07.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.052+0000 7fda3aa00640 1 -- 172.21.15.161:0/1574163565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda3c15d410 msgr2=0x7fda3c15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:07.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.052+0000 7fda3aa00640 1 --2- 172.21.15.161:0/1574163565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda3c15d410 0x7fda3c15b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:07.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.052+0000 7fda3aa00640 1 -- 172.21.15.161:0/1574163565 >> v1:172.21.15.179:6789/0 conn(0x7fda3c15dd50 legacy=0x7fda3c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:07.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.052+0000 7fda3aa00640 1 -- 172.21.15.161:0/1574163565 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c070d40 con 0x7fda3c15e130 2024-05-22T21:17:07.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.052+0000 7fda3be00640 1 --2- 172.21.15.161:0/1574163565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda3c15d410 0x7fda3c15b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:07.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.052+0000 7fda3aa00640 1 --2- 172.21.15.161:0/1574163565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda3c15e130 0x7fda3c15ced0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fda28009870 tx=0x7fda2801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d416d0cf7b75f485 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:07.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.052+0000 7fda3a000640 1 -- 172.21.15.161:0/1574163565 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda280a95f0 con 0x7fda3c15e130 2024-05-22T21:17:07.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.052+0000 7fda3a000640 1 -- 172.21.15.161:0/1574163565 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda280a9750 con 0x7fda3c15e130 2024-05-22T21:17:07.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.052+0000 7fda3a000640 1 -- 172.21.15.161:0/1574163565 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda280a9ae0 con 0x7fda3c15e130 2024-05-22T21:17:07.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda42600640 1 -- 172.21.15.161:0/1574163565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda3c15e130 msgr2=0x7fda3c15ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:07.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda42600640 1 --2- 172.21.15.161:0/1574163565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda3c15e130 0x7fda3c15ced0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fda28009870 tx=0x7fda2801c530 comp rx=0 tx=0).stop 2024-05-22T21:17:07.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda42600640 1 -- 172.21.15.161:0/1574163565 shutdown_connections 2024-05-22T21:17:07.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda42600640 1 --2- 172.21.15.161:0/1574163565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda3c15d410 0x7fda3c15b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:07.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda42600640 1 --2- 172.21.15.161:0/1574163565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda3c15e130 0x7fda3c15ced0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:07.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda42600640 1 -- 172.21.15.161:0/1574163565 >> 172.21.15.161:0/1574163565 conn(0x7fda3c0b0bc0 msgr2=0x7fda3c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:07.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda42600640 1 -- 172.21.15.161:0/1574163565 shutdown_connections 2024-05-22T21:17:07.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda42600640 1 -- 172.21.15.161:0/1574163565 wait complete. 2024-05-22T21:17:07.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda42600640 1 Processor -- start 2024-05-22T21:17:07.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda42600640 1 -- start start 2024-05-22T21:17:07.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda42600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda3c15d410 0x7fda3c0ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:07.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda42600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda3c15dd50 0x7fda3c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda42600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda3c15e130 0x7fda3c0b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda42600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c159b90 con 0x7fda3c15d410 2024-05-22T21:17:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda42600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c0acdb0 con 0x7fda3c15dd50 2024-05-22T21:17:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda42600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c0ad090 con 0x7fda3c15e130 2024-05-22T21:17:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda3b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda3c15d410 0x7fda3c0ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda3b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda3c15d410 0x7fda3c0ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33458/0 (socket says 172.21.15.161:33458) 2024-05-22T21:17:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda3b400640 1 -- 172.21.15.161:0/2799902334 learned_addr learned my addr 172.21.15.161:0/2799902334 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda3be00640 1 --2- 172.21.15.161:0/2799902334 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda3c15e130 0x7fda3c0b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda3aa00640 1 --2- 172.21.15.161:0/2799902334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda3c15dd50 0x7fda3c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda3b400640 1 -- 172.21.15.161:0/2799902334 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda3c15e130 msgr2=0x7fda3c0b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda3b400640 1 --2- 172.21.15.161:0/2799902334 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda3c15e130 0x7fda3c0b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda3b400640 1 -- 172.21.15.161:0/2799902334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda3c15dd50 msgr2=0x7fda3c0b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda3b400640 1 --2- 172.21.15.161:0/2799902334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda3c15dd50 0x7fda3c0b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:07.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda3b400640 1 -- 172.21.15.161:0/2799902334 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda280094d0 con 0x7fda3c15d410 2024-05-22T21:17:07.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.056+0000 7fda3b400640 1 --2- 172.21.15.161:0/2799902334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda3c15d410 0x7fda3c0ade50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00c7f0 tx=0x7fda2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:07.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.060+0000 7fda33400640 1 -- 172.21.15.161:0/2799902334 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c003b80 con 0x7fda3c15d410 2024-05-22T21:17:07.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.060+0000 7fda33400640 1 -- 172.21.15.161:0/2799902334 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c003ce0 con 0x7fda3c15d410 2024-05-22T21:17:07.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.060+0000 7fda42600640 1 -- 172.21.15.161:0/2799902334 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3c0ad370 con 0x7fda3c15d410 2024-05-22T21:17:07.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.060+0000 7fda33400640 1 -- 172.21.15.161:0/2799902334 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c00aa90 con 0x7fda3c15d410 2024-05-22T21:17:07.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.060+0000 7fda42600640 1 -- 172.21.15.161:0/2799902334 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3c0ad930 con 0x7fda3c15d410 2024-05-22T21:17:07.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.060+0000 7fda42600640 1 -- 172.21.15.161:0/2799902334 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3c067220 con 0x7fda3c15d410 2024-05-22T21:17:07.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.060+0000 7fda33400640 1 -- 172.21.15.161:0/2799902334 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fda2c00abf0 con 0x7fda3c15d410 2024-05-22T21:17:07.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.060+0000 7fda33400640 1 --2- 172.21.15.161:0/2799902334 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda1c066690 0x7fda1c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:07.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.060+0000 7fda33400640 1 -- 172.21.15.161:0/2799902334 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fda2c083c30 con 0x7fda3c15d410 2024-05-22T21:17:07.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.060+0000 7fda3aa00640 1 --2- 172.21.15.161:0/2799902334 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda1c066690 0x7fda1c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:07.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.060+0000 7fda3aa00640 1 --2- 172.21.15.161:0/2799902334 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda1c066690 0x7fda1c068b50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fda280ae000 tx=0x7fda280a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:07.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.064+0000 7fda33400640 1 -- 172.21.15.161:0/2799902334 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fda2c051350 con 0x7fda3c15d410 2024-05-22T21:17:07.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.280+0000 7fda42600640 1 -- 172.21.15.161:0/2799902334 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7fda3c06c310 con 0x7fda3c15d410 2024-05-22T21:17:07.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.548+0000 7fda33400640 1 -- 172.21.15.161:0/2799902334 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v203) v1 ==== 134+0+0 (secure 0 0 0) 0x7fda2c054970 con 0x7fda3c15d410 2024-05-22T21:17:07.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.552+0000 7fda42600640 1 -- 172.21.15.161:0/2799902334 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda1c066690 msgr2=0x7fda1c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:07.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.552+0000 7fda42600640 1 --2- 172.21.15.161:0/2799902334 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda1c066690 0x7fda1c068b50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fda280ae000 tx=0x7fda280a7000 comp rx=0 tx=0).stop 2024-05-22T21:17:07.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.552+0000 7fda42600640 1 -- 172.21.15.161:0/2799902334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda3c15d410 msgr2=0x7fda3c0ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:07.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.552+0000 7fda42600640 1 --2- 172.21.15.161:0/2799902334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda3c15d410 0x7fda3c0ade50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00c7f0 tx=0x7fda2c00ccb0 comp rx=0 tx=0).stop 2024-05-22T21:17:07.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.552+0000 7fda42600640 1 -- 172.21.15.161:0/2799902334 shutdown_connections 2024-05-22T21:17:07.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.552+0000 7fda42600640 1 --2- 172.21.15.161:0/2799902334 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda1c066690 0x7fda1c068b50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:07.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.552+0000 7fda42600640 1 --2- 172.21.15.161:0/2799902334 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda3c15e130 0x7fda3c0b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:07.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.552+0000 7fda42600640 1 --2- 172.21.15.161:0/2799902334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda3c15dd50 0x7fda3c0b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:07.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.552+0000 7fda42600640 1 --2- 172.21.15.161:0/2799902334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda3c15d410 0x7fda3c0ade50 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:07.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.552+0000 7fda42600640 1 -- 172.21.15.161:0/2799902334 >> 172.21.15.161:0/2799902334 conn(0x7fda3c0b0bc0 msgr2=0x7fda3c154c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:07.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.552+0000 7fda42600640 1 -- 172.21.15.161:0/2799902334 shutdown_connections 2024-05-22T21:17:07.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.552+0000 7fda42600640 1 -- 172.21.15.161:0/2799902334 wait complete. 2024-05-22T21:17:07.577 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-05-22T21:17:07.577 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) That when the full flag is encountered during asynchronous 2024-05-22T21:17:07.578 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2024-05-22T21:17:07.578 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2024-05-22T21:17:07.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea4200640 1 Processor -- start 2024-05-22T21:17:07.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea4200640 1 -- start start 2024-05-22T21:17:07.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea4200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae9c099ca0 0x7fae9c06eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:07.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea4200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fae9c0b3560 0x7fae9c09a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:07.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea4200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9c06df70 con 0x7fae9c0cf0e0 2024-05-22T21:17:07.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea4200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9c0c7ea0 con 0x7fae9c0b3560 2024-05-22T21:17:07.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea4200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c161c60 con 0x7fae9c099ca0 2024-05-22T21:17:07.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea1800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae9c099ca0 0x7fae9c06eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:07.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea0e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fae9c0b3560 0x7fae9c09a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:07.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea0e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fae9c0b3560 0x7fae9c09a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:52644/0 (socket says 172.21.15.161:52644) 2024-05-22T21:17:07.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea1800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae9c099ca0 0x7fae9c06eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:48030/0 (socket says 172.21.15.161:48030) 2024-05-22T21:17:07.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea0e00640 1 -- 172.21.15.161:0/103185274 learned_addr learned my addr 172.21.15.161:0/103185274 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:07.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea2200640 1 --1- 172.21.15.161:0/103185274 >> v1:172.21.15.179:6789/0 conn(0x7fae9c0cf0e0 0x7fae9c0cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:49744/0 (socket says 172.21.15.161:49744) 2024-05-22T21:17:07.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea0e00640 1 -- 172.21.15.161:0/103185274 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae9c099ca0 msgr2=0x7fae9c06eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:07.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea0e00640 1 --2- 172.21.15.161:0/103185274 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae9c099ca0 0x7fae9c06eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:07.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea0e00640 1 -- 172.21.15.161:0/103185274 >> v1:172.21.15.179:6789/0 conn(0x7fae9c0cf0e0 legacy=0x7fae9c0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:07.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea0e00640 1 -- 172.21.15.161:0/103185274 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c06e1c0 con 0x7fae9c0b3560 2024-05-22T21:17:07.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7faea1800640 1 --2- 172.21.15.161:0/103185274 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae9c099ca0 0x7fae9c06eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:17:07.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.712+0000 7fae9be00640 1 -- 172.21.15.161:0/103185274 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743919057 0 0) 0x7fae9c06df70 con 0x7fae9c0cf0e0 2024-05-22T21:17:07.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.716+0000 7faea0e00640 1 --2- 172.21.15.161:0/103185274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c0b3560 0x7fae9c09a080 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fae88009870 tx=0x7fae8801c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=858d2c6a588b692c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:07.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.716+0000 7fae9be00640 1 -- 172.21.15.161:0/103185274 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae880ab640 con 0x7fae9c0b3560 2024-05-22T21:17:07.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.716+0000 7fae9be00640 1 -- 172.21.15.161:0/103185274 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae880a7070 con 0x7fae9c0b3560 2024-05-22T21:17:07.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.716+0000 7fae9be00640 1 -- 172.21.15.161:0/103185274 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae880ab9c0 con 0x7fae9c0b3560 2024-05-22T21:17:07.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.716+0000 7faea4200640 1 -- 172.21.15.161:0/103185274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c0b3560 msgr2=0x7fae9c09a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:07.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.716+0000 7faea4200640 1 --2- 172.21.15.161:0/103185274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c0b3560 0x7fae9c09a080 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fae88009870 tx=0x7fae8801c530 comp rx=0 tx=0).stop 2024-05-22T21:17:07.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.716+0000 7faea4200640 1 -- 172.21.15.161:0/103185274 shutdown_connections 2024-05-22T21:17:07.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.716+0000 7faea4200640 1 --2- 172.21.15.161:0/103185274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c0b3560 0x7fae9c09a080 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:07.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.716+0000 7faea4200640 1 --2- 172.21.15.161:0/103185274 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae9c099ca0 0x7fae9c06eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:07.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.716+0000 7faea4200640 1 -- 172.21.15.161:0/103185274 >> 172.21.15.161:0/103185274 conn(0x7fae9c0b2a40 msgr2=0x7fae9c0b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:07.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.716+0000 7faea4200640 1 -- 172.21.15.161:0/103185274 shutdown_connections 2024-05-22T21:17:07.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea4200640 1 -- 172.21.15.161:0/103185274 wait complete. 2024-05-22T21:17:07.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea4200640 1 Processor -- start 2024-05-22T21:17:07.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea4200640 1 -- start start 2024-05-22T21:17:07.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea4200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c099ca0 0x7fae9c14f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:07.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea4200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae9c0b3560 0x7fae9c0afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:07.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea4200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae9c0cf0e0 0x7fae9c15a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:07.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea4200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0b4f30 con 0x7fae9c0cf0e0 2024-05-22T21:17:07.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea4200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c15c4f0 con 0x7fae9c099ca0 2024-05-22T21:17:07.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea4200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c15c7d0 con 0x7fae9c0b3560 2024-05-22T21:17:07.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea2200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae9c0cf0e0 0x7fae9c15a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:07.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea2200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae9c0cf0e0 0x7fae9c15a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33468/0 (socket says 172.21.15.161:33468) 2024-05-22T21:17:07.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea2200640 1 -- 172.21.15.161:0/3067736601 learned_addr learned my addr 172.21.15.161:0/3067736601 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:07.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea1800640 1 --2- 172.21.15.161:0/3067736601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c099ca0 0x7fae9c14f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:07.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea0e00640 1 --2- 172.21.15.161:0/3067736601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae9c0b3560 0x7fae9c0afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:07.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea2200640 1 -- 172.21.15.161:0/3067736601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae9c0b3560 msgr2=0x7fae9c0afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:07.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea2200640 1 --2- 172.21.15.161:0/3067736601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae9c0b3560 0x7fae9c0afbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:07.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea2200640 1 -- 172.21.15.161:0/3067736601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c099ca0 msgr2=0x7fae9c14f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:07.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea2200640 1 --2- 172.21.15.161:0/3067736601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c099ca0 0x7fae9c14f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:07.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea2200640 1 -- 172.21.15.161:0/3067736601 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae90009420 con 0x7fae9c0cf0e0 2024-05-22T21:17:07.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7faea2200640 1 --2- 172.21.15.161:0/3067736601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae9c0cf0e0 0x7fae9c15a050 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fae94001570 tx=0x7fae9400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:07.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7fae99600640 1 -- 172.21.15.161:0/3067736601 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9400a640 con 0x7fae9c0cf0e0 2024-05-22T21:17:07.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.720+0000 7fae99600640 1 -- 172.21.15.161:0/3067736601 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9400a7c0 con 0x7fae9c0cf0e0 2024-05-22T21:17:07.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.724+0000 7fae99600640 1 -- 172.21.15.161:0/3067736601 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9400ab70 con 0x7fae9c0cf0e0 2024-05-22T21:17:07.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.724+0000 7faea4200640 1 -- 172.21.15.161:0/3067736601 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae880094d0 con 0x7fae9c0cf0e0 2024-05-22T21:17:07.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.724+0000 7faea4200640 1 -- 172.21.15.161:0/3067736601 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9c0b4870 con 0x7fae9c0cf0e0 2024-05-22T21:17:07.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.724+0000 7faea4200640 1 -- 172.21.15.161:0/3067736601 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9c0b50a0 con 0x7fae9c0cf0e0 2024-05-22T21:17:07.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.724+0000 7fae99600640 1 -- 172.21.15.161:0/3067736601 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fae9400acd0 con 0x7fae9c0cf0e0 2024-05-22T21:17:07.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.724+0000 7fae99600640 1 --2- 172.21.15.161:0/3067736601 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae7c0666c0 0x7fae7c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:07.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.724+0000 7fae99600640 1 -- 172.21.15.161:0/3067736601 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fae940833b0 con 0x7fae9c0cf0e0 2024-05-22T21:17:07.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.724+0000 7faea1800640 1 --2- 172.21.15.161:0/3067736601 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae7c0666c0 0x7fae7c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:07.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.724+0000 7faea1800640 1 --2- 172.21.15.161:0/3067736601 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae7c0666c0 0x7fae7c068b80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fae90009c50 tx=0x7fae90009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:07.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.728+0000 7fae99600640 1 -- 172.21.15.161:0/3067736601 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fae94050a30 con 0x7fae9c0cf0e0 2024-05-22T21:17:07.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:07.936+0000 7faea4200640 1 -- 172.21.15.161:0/3067736601 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7fae9c067220 con 0x7fae9c0cf0e0 2024-05-22T21:17:08.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.556+0000 7fae99600640 1 -- 172.21.15.161:0/3067736601 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v204) v1 ==== 136+0+0 (secure 0 0 0) 0x7fae94054050 con 0x7fae9c0cf0e0 2024-05-22T21:17:08.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.556+0000 7faea4200640 1 -- 172.21.15.161:0/3067736601 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae7c0666c0 msgr2=0x7fae7c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:08.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.556+0000 7faea4200640 1 --2- 172.21.15.161:0/3067736601 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae7c0666c0 0x7fae7c068b80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fae90009c50 tx=0x7fae90009190 comp rx=0 tx=0).stop 2024-05-22T21:17:08.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.556+0000 7faea4200640 1 -- 172.21.15.161:0/3067736601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae9c0cf0e0 msgr2=0x7fae9c15a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:08.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.556+0000 7faea4200640 1 --2- 172.21.15.161:0/3067736601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae9c0cf0e0 0x7fae9c15a050 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fae94001570 tx=0x7fae9400cca0 comp rx=0 tx=0).stop 2024-05-22T21:17:08.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.556+0000 7faea4200640 1 -- 172.21.15.161:0/3067736601 shutdown_connections 2024-05-22T21:17:08.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.556+0000 7faea4200640 1 --2- 172.21.15.161:0/3067736601 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae7c0666c0 0x7fae7c068b80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:08.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.556+0000 7faea4200640 1 --2- 172.21.15.161:0/3067736601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae9c0cf0e0 0x7fae9c15a050 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:08.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.556+0000 7faea4200640 1 --2- 172.21.15.161:0/3067736601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae9c0b3560 0x7fae9c0afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:08.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.556+0000 7faea4200640 1 --2- 172.21.15.161:0/3067736601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c099ca0 0x7fae9c14f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:08.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.556+0000 7faea4200640 1 -- 172.21.15.161:0/3067736601 >> 172.21.15.161:0/3067736601 conn(0x7fae9c0b2a40 msgr2=0x7fae9c0b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:08.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.556+0000 7faea4200640 1 -- 172.21.15.161:0/3067736601 shutdown_connections 2024-05-22T21:17:08.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.556+0000 7faea4200640 1 -- 172.21.15.161:0/3067736601 wait complete. 2024-05-22T21:17:08.580 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-05-22T21:17:08.734 INFO:teuthology.orchestra.run.smithi161.stdout:{"osd_objectstore":"memstore"} 2024-05-22T21:17:08.734 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-05-22T21:17:08.734 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-22T21:17:08.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.868+0000 7fb98f200640 1 Processor -- start 2024-05-22T21:17:08.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.868+0000 7fb98f200640 1 -- start start 2024-05-22T21:17:08.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98f200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb9880ce370 0x7fb9881442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:08.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98f200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9880ce830 con 0x7fb9881448e0 2024-05-22T21:17:08.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98f200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9880b2e60 con 0x7fb988143ed0 2024-05-22T21:17:08.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98f200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9880b3140 con 0x7fb9880ce370 2024-05-22T21:17:08.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98e800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb9881448e0 0x7fb98806bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54412/0 (socket says 172.21.15.161:54412) 2024-05-22T21:17:08.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98e800640 1 -- 172.21.15.161:0/3298943882 learned_addr learned my addr 172.21.15.161:0/3298943882 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:08.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98d400640 1 --2- 172.21.15.161:0/3298943882 >> v2:172.21.15.179:3300/0 conn(0x7fb9880ce370 0x7fb9881442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:08.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98ca00640 1 -- 172.21.15.161:0/3298943882 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1700107693 0 0) 0x7fb9880ce830 con 0x7fb9881448e0 2024-05-22T21:17:08.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98ca00640 1 -- 172.21.15.161:0/3298943882 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb970003530 con 0x7fb9881448e0 2024-05-22T21:17:08.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98ca00640 1 -- 172.21.15.161:0/3298943882 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 70421339 0 0) 0x7fb970003530 con 0x7fb9881448e0 2024-05-22T21:17:08.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98ca00640 1 -- 172.21.15.161:0/3298943882 >> v2:172.21.15.179:3300/0 conn(0x7fb9880ce370 msgr2=0x7fb9881442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:08.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98ca00640 1 --2- 172.21.15.161:0/3298943882 >> v2:172.21.15.179:3300/0 conn(0x7fb9880ce370 0x7fb9881442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:08.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98ca00640 1 -- 172.21.15.161:0/3298943882 >> v1:172.21.15.179:6789/0 conn(0x7fb988143ed0 legacy=0x7fb9880b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:08.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98ca00640 1 -- 172.21.15.161:0/3298943882 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9880b3420 con 0x7fb9881448e0 2024-05-22T21:17:08.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98ca00640 1 -- 172.21.15.161:0/3298943882 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fb984001a00 con 0x7fb9881448e0 2024-05-22T21:17:08.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98ca00640 1 -- 172.21.15.161:0/3298943882 >> v1:172.21.15.161:6789/0 conn(0x7fb9881448e0 legacy=0x7fb98806bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:08.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98ca00640 1 --2- 172.21.15.161:0/3298943882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb970003980 0x7fb970003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:08.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98ca00640 1 --2- 172.21.15.161:0/3298943882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9700047d0 0x7fb970004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:08.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98ca00640 1 --2- 172.21.15.161:0/3298943882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb970005190 0x7fb97000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:08.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98ca00640 1 -- 172.21.15.161:0/3298943882 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880b3140 con 0x7fb970003980 2024-05-22T21:17:08.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98ca00640 1 -- 172.21.15.161:0/3298943882 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97000c130 con 0x7fb9700047d0 2024-05-22T21:17:08.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98ca00640 1 -- 172.21.15.161:0/3298943882 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97000c2f0 con 0x7fb970005190 2024-05-22T21:17:08.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98de00640 1 --2- 172.21.15.161:0/3298943882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb970003980 0x7fb970003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:08.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98d400640 1 --2- 172.21.15.161:0/3298943882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9700047d0 0x7fb970004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:08.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98e800640 1 --2- 172.21.15.161:0/3298943882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb970005190 0x7fb97000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:08.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98de00640 1 -- 172.21.15.161:0/3298943882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb970005190 msgr2=0x7fb97000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:08.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98de00640 1 --2- 172.21.15.161:0/3298943882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb970005190 0x7fb97000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:08.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98de00640 1 -- 172.21.15.161:0/3298943882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9700047d0 msgr2=0x7fb970004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:08.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98de00640 1 --2- 172.21.15.161:0/3298943882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9700047d0 0x7fb970004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:08.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98de00640 1 -- 172.21.15.161:0/3298943882 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb970003530 con 0x7fb970003980 2024-05-22T21:17:08.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.872+0000 7fb98e800640 1 -- 172.21.15.161:0/3298943882 reap_dead start 2024-05-22T21:17:08.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.876+0000 7fb98de00640 1 --2- 172.21.15.161:0/3298943882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb970003980 0x7fb970003da0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb97c001780 tx=0x7fb97c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51dacfc5a1937a26 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:08.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.876+0000 7fb98ca00640 1 -- 172.21.15.161:0/3298943882 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c0052d0 con 0x7fb970003980 2024-05-22T21:17:08.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.876+0000 7fb98ca00640 1 -- 172.21.15.161:0/3298943882 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97c0a7050 con 0x7fb970003980 2024-05-22T21:17:08.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.876+0000 7fb98ca00640 1 -- 172.21.15.161:0/3298943882 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c003600 con 0x7fb970003980 2024-05-22T21:17:08.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.876+0000 7fb98f200640 1 -- 172.21.15.161:0/3298943882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb970003980 msgr2=0x7fb970003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:08.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.876+0000 7fb98f200640 1 --2- 172.21.15.161:0/3298943882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb970003980 0x7fb970003da0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb97c001780 tx=0x7fb97c01c400 comp rx=0 tx=0).stop 2024-05-22T21:17:08.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.876+0000 7fb98f200640 1 -- 172.21.15.161:0/3298943882 shutdown_connections 2024-05-22T21:17:08.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.876+0000 7fb98f200640 1 --2- 172.21.15.161:0/3298943882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb970003980 0x7fb970003da0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:08.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.876+0000 7fb98f200640 1 -- 172.21.15.161:0/3298943882 >> 172.21.15.161:0/3298943882 conn(0x7fb9880cec00 msgr2=0x7fb9880cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:08.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.876+0000 7fb98f200640 1 -- 172.21.15.161:0/3298943882 shutdown_connections 2024-05-22T21:17:08.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.876+0000 7fb98f200640 1 -- 172.21.15.161:0/3298943882 wait complete. 2024-05-22T21:17:08.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.876+0000 7fb98f200640 1 Processor -- start 2024-05-22T21:17:08.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.876+0000 7fb98f200640 1 -- start start 2024-05-22T21:17:08.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98f200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb970003980 0x7fb98813b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:08.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb988070a70 0x7fb98813d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:08.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98f200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb988136000 0x7fb988136480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:08.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98f200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98813ce20 con 0x7fb988070a70 2024-05-22T21:17:08.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98f200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98813c210 con 0x7fb970003980 2024-05-22T21:17:08.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98f200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98813c4f0 con 0x7fb988136000 2024-05-22T21:17:08.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb988070a70 0x7fb98813d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:08.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb988070a70 0x7fb98813d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33488/0 (socket says 172.21.15.161:33488) 2024-05-22T21:17:08.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98d400640 1 -- 172.21.15.161:0/4149316028 learned_addr learned my addr 172.21.15.161:0/4149316028 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:08.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98e800640 1 --2- 172.21.15.161:0/4149316028 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb988136000 0x7fb988136480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:08.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98de00640 1 --2- 172.21.15.161:0/4149316028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb970003980 0x7fb98813b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:08.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98d400640 1 -- 172.21.15.161:0/4149316028 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb988136000 msgr2=0x7fb988136480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:08.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98d400640 1 --2- 172.21.15.161:0/4149316028 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb988136000 0x7fb988136480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:08.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98d400640 1 -- 172.21.15.161:0/4149316028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb970003980 msgr2=0x7fb98813b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:08.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98d400640 1 --2- 172.21.15.161:0/4149316028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb970003980 0x7fb98813b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:08.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98d400640 1 -- 172.21.15.161:0/4149316028 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98813c7d0 con 0x7fb988070a70 2024-05-22T21:17:08.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98d400640 1 --2- 172.21.15.161:0/4149316028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb988070a70 0x7fb98813d2b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb978011500 tx=0x7fb9780119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:08.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb982000640 1 -- 172.21.15.161:0/4149316028 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978015870 con 0x7fb988070a70 2024-05-22T21:17:08.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb982000640 1 -- 172.21.15.161:0/4149316028 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9780159d0 con 0x7fb988070a70 2024-05-22T21:17:08.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb982000640 1 -- 172.21.15.161:0/4149316028 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978015d40 con 0x7fb988070a70 2024-05-22T21:17:08.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98f200640 1 -- 172.21.15.161:0/4149316028 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98813cab0 con 0x7fb988070a70 2024-05-22T21:17:08.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98f200640 1 -- 172.21.15.161:0/4149316028 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98813d7f0 con 0x7fb988070a70 2024-05-22T21:17:08.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb98f200640 1 -- 172.21.15.161:0/4149316028 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98813cfd0 con 0x7fb988070a70 2024-05-22T21:17:08.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.880+0000 7fb982000640 1 -- 172.21.15.161:0/4149316028 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb978013400 con 0x7fb988070a70 2024-05-22T21:17:08.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.884+0000 7fb982000640 1 --2- 172.21.15.161:0/4149316028 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb96c0666c0 0x7fb96c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:08.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.884+0000 7fb982000640 1 -- 172.21.15.161:0/4149316028 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb978084130 con 0x7fb988070a70 2024-05-22T21:17:08.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.884+0000 7fb98de00640 1 --2- 172.21.15.161:0/4149316028 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb96c0666c0 0x7fb96c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:08.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.884+0000 7fb98de00640 1 --2- 172.21.15.161:0/4149316028 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb96c0666c0 0x7fb96c068b80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb97c004000 tx=0x7fb97c0011a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:08.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:08.888+0000 7fb982000640 1 -- 172.21.15.161:0/4149316028 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb98813cfd0 con 0x7fb988070a70 2024-05-22T21:17:09.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.040+0000 7fb98f200640 1 -- 172.21.15.161:0/4149316028 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb988067070 con 0x7fb988070a70 2024-05-22T21:17:09.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.044+0000 7fb982000640 1 -- 172.21.15.161:0/4149316028 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb9780517c0 con 0x7fb988070a70 2024-05-22T21:17:09.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.048+0000 7fb98f200640 1 -- 172.21.15.161:0/4149316028 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb96c0666c0 msgr2=0x7fb96c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:09.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.048+0000 7fb98f200640 1 --2- 172.21.15.161:0/4149316028 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb96c0666c0 0x7fb96c068b80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb97c004000 tx=0x7fb97c0011a0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.048+0000 7fb98f200640 1 -- 172.21.15.161:0/4149316028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb988070a70 msgr2=0x7fb98813d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:09.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.048+0000 7fb98f200640 1 --2- 172.21.15.161:0/4149316028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb988070a70 0x7fb98813d2b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb978011500 tx=0x7fb9780119c0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.048+0000 7fb98f200640 1 -- 172.21.15.161:0/4149316028 shutdown_connections 2024-05-22T21:17:09.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.048+0000 7fb98f200640 1 --2- 172.21.15.161:0/4149316028 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb96c0666c0 0x7fb96c068b80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.048+0000 7fb98f200640 1 --2- 172.21.15.161:0/4149316028 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb988136000 0x7fb988136480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.048+0000 7fb98f200640 1 --2- 172.21.15.161:0/4149316028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb988070a70 0x7fb98813d2b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.048+0000 7fb98f200640 1 --2- 172.21.15.161:0/4149316028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb970003980 0x7fb98813b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.048+0000 7fb98f200640 1 -- 172.21.15.161:0/4149316028 >> 172.21.15.161:0/4149316028 conn(0x7fb9880cec00 msgr2=0x7fb9880af290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:09.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.048+0000 7fb98f200640 1 -- 172.21.15.161:0/4149316028 shutdown_connections 2024-05-22T21:17:09.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.048+0000 7fb98f200640 1 -- 172.21.15.161:0/4149316028 wait complete. 2024-05-22T21:17:09.069 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:17:09.070 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:17:09.136 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:17:09.189 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:17:09.190 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:17:09.190 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:17:09.198 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:17:09.198 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:17:09.198 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:17:09.207 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:17:09.207 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:17:09.208 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:17:09.248 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:17:09.248 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:17:09.248 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:17:09.262 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:17:09.262 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-22T21:17:09.262 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:17:09.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.420+0000 7f7999200640 1 Processor -- start 2024-05-22T21:17:09.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.420+0000 7f7999200640 1 -- start start 2024-05-22T21:17:09.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.420+0000 7f7999200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f799415dd50 0x7f799406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:09.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.420+0000 7f7999200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f799415d5e0 0x7f799415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:09.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.420+0000 7f7999200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7994071670 con 0x7f799415e130 2024-05-22T21:17:09.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.420+0000 7f7999200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f799406de40 con 0x7f799415d5e0 2024-05-22T21:17:09.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.420+0000 7f7999200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7994070a60 con 0x7f799415dd50 2024-05-22T21:17:09.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.420+0000 7f7992800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f799415d5e0 0x7f799415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:09.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.420+0000 7f7992800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f799415d5e0 0x7f799415b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33492/0 (socket says 172.21.15.161:33492) 2024-05-22T21:17:09.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.420+0000 7f7992800640 1 -- 172.21.15.161:0/1036958990 learned_addr learned my addr 172.21.15.161:0/1036958990 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:09.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.420+0000 7f7991e00640 1 --2- 172.21.15.161:0/1036958990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f799415dd50 0x7f799406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:09.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.420+0000 7f7992800640 1 -- 172.21.15.161:0/1036958990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f799415dd50 msgr2=0x7f799406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:09.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.420+0000 7f7992800640 1 --2- 172.21.15.161:0/1036958990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f799415dd50 0x7f799406d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.420+0000 7f7992800640 1 -- 172.21.15.161:0/1036958990 >> v1:172.21.15.161:6789/0 conn(0x7f799415e130 legacy=0x7f799415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:09.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.420+0000 7f7992800640 1 -- 172.21.15.161:0/1036958990 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7994070d40 con 0x7f799415d5e0 2024-05-22T21:17:09.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.420+0000 7f7991e00640 1 --2- 172.21.15.161:0/1036958990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f799415dd50 0x7f799406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:17:09.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.424+0000 7f7992800640 1 --2- 172.21.15.161:0/1036958990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799415d5e0 0x7f799415b6c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f798c009e50 tx=0x7f798c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6b19517aebcce6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:09.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.424+0000 7f7991400640 1 -- 172.21.15.161:0/1036958990 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c0a5830 con 0x7f799415d5e0 2024-05-22T21:17:09.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.424+0000 7f7991400640 1 -- 172.21.15.161:0/1036958990 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798c0a5990 con 0x7f799415d5e0 2024-05-22T21:17:09.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.424+0000 7f7991400640 1 -- 172.21.15.161:0/1036958990 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c0a5d20 con 0x7f799415d5e0 2024-05-22T21:17:09.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.424+0000 7f7999200640 1 -- 172.21.15.161:0/1036958990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799415d5e0 msgr2=0x7f799415b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:09.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.424+0000 7f7999200640 1 --2- 172.21.15.161:0/1036958990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799415d5e0 0x7f799415b6c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f798c009e50 tx=0x7f798c01c970 comp rx=0 tx=0).stop 2024-05-22T21:17:09.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.424+0000 7f7999200640 1 -- 172.21.15.161:0/1036958990 shutdown_connections 2024-05-22T21:17:09.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.424+0000 7f7999200640 1 --2- 172.21.15.161:0/1036958990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799415d5e0 0x7f799415b6c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.424+0000 7f7999200640 1 --2- 172.21.15.161:0/1036958990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f799415dd50 0x7f799406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.424+0000 7f7999200640 1 -- 172.21.15.161:0/1036958990 >> 172.21.15.161:0/1036958990 conn(0x7f79940b0bc0 msgr2=0x7f79940b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:09.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.424+0000 7f7999200640 1 -- 172.21.15.161:0/1036958990 shutdown_connections 2024-05-22T21:17:09.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.424+0000 7f7999200640 1 -- 172.21.15.161:0/1036958990 wait complete. 2024-05-22T21:17:09.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7999200640 1 Processor -- start 2024-05-22T21:17:09.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7999200640 1 -- start start 2024-05-22T21:17:09.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7999200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799415d5e0 0x7f79940b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:09.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7999200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f799415dd50 0x7f79941493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:09.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7999200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f799415e130 0x7f799414c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:09.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7999200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799414ce00 con 0x7f799415d5e0 2024-05-22T21:17:09.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7999200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799414cb20 con 0x7f799415dd50 2024-05-22T21:17:09.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7999200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79940ac9e0 con 0x7f799415e130 2024-05-22T21:17:09.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7991e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799415d5e0 0x7f79940b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:09.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7991e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799415d5e0 0x7f79940b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33502/0 (socket says 172.21.15.161:33502) 2024-05-22T21:17:09.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7991e00640 1 -- 172.21.15.161:0/4035649995 learned_addr learned my addr 172.21.15.161:0/4035649995 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:09.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7991e00640 1 -- 172.21.15.161:0/4035649995 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f799415e130 msgr2=0x7f799414c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:09.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7992800640 1 --2- 172.21.15.161:0/4035649995 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f799415e130 0x7f799414c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:09.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7989400640 1 --2- 172.21.15.161:0/4035649995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f799415dd50 0x7f79941493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:09.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7991e00640 1 --2- 172.21.15.161:0/4035649995 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f799415e130 0x7f799414c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7991e00640 1 -- 172.21.15.161:0/4035649995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f799415dd50 msgr2=0x7f79941493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:09.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7991e00640 1 --2- 172.21.15.161:0/4035649995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f799415dd50 0x7f79941493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7991e00640 1 -- 172.21.15.161:0/4035649995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798c009b30 con 0x7f799415d5e0 2024-05-22T21:17:09.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7992800640 1 --2- 172.21.15.161:0/4035649995 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f799415e130 0x7f799414c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:17:09.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7989400640 1 --2- 172.21.15.161:0/4035649995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f799415dd50 0x7f79941493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:17:09.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7991e00640 1 --2- 172.21.15.161:0/4035649995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799415d5e0 0x7f79940b0fa0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f798400c7c0 tx=0x7f798400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:09.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f798aa00640 1 -- 172.21.15.161:0/4035649995 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984007a30 con 0x7f799415d5e0 2024-05-22T21:17:09.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f798aa00640 1 -- 172.21.15.161:0/4035649995 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7984007b90 con 0x7f799415d5e0 2024-05-22T21:17:09.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f798aa00640 1 -- 172.21.15.161:0/4035649995 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798400ce70 con 0x7f799415d5e0 2024-05-22T21:17:09.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7999200640 1 -- 172.21.15.161:0/4035649995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79940accc0 con 0x7f799415d5e0 2024-05-22T21:17:09.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.428+0000 7f7999200640 1 -- 172.21.15.161:0/4035649995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799414b370 con 0x7f799415d5e0 2024-05-22T21:17:09.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.432+0000 7f7999200640 1 -- 172.21.15.161:0/4035649995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7994067220 con 0x7f799415d5e0 2024-05-22T21:17:09.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.432+0000 7f798aa00640 1 -- 172.21.15.161:0/4035649995 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f79840113f0 con 0x7f799415d5e0 2024-05-22T21:17:09.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.432+0000 7f798aa00640 1 --2- 172.21.15.161:0/4035649995 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f79700666c0 0x7f7970068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:09.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.432+0000 7f798aa00640 1 -- 172.21.15.161:0/4035649995 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7984084370 con 0x7f799415d5e0 2024-05-22T21:17:09.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.432+0000 7f7989400640 1 --2- 172.21.15.161:0/4035649995 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f79700666c0 0x7f7970068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:09.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.432+0000 7f7989400640 1 --2- 172.21.15.161:0/4035649995 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f79700666c0 0x7f7970068b80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7994153840 tx=0x7f797c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:09.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.436+0000 7f798aa00640 1 -- 172.21.15.161:0/4035649995 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f79840519d0 con 0x7f799415d5e0 2024-05-22T21:17:09.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.620+0000 7f7999200640 1 -- 172.21.15.161:0/4035649995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7994070200 con 0x7f799415d5e0 2024-05-22T21:17:09.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.624+0000 7f798aa00640 1 -- 172.21.15.161:0/4035649995 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f798401ede0 con 0x7f799415d5e0 2024-05-22T21:17:09.626 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:09.626 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":33,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1620021722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":1620021722},{"type":"v1","addr":"172.21.15.179:6835","nonce":1620021722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5528,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/638590357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":638590357},{"type":"v1","addr":"172.21.15.179:6834","nonce":638590357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/3548738614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3548738614},{"type":"v1","addr":"172.21.15.161:6836","nonce":3548738614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/400898801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":400898801},{"type":"v1","addr":"172.21.15.161:6837","nonce":400898801}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2024-05-22T21:17:09.626 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 33 2024-05-22T21:17:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.624+0000 7f7999200640 1 -- 172.21.15.161:0/4035649995 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f79700666c0 msgr2=0x7f7970068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.624+0000 7f7999200640 1 --2- 172.21.15.161:0/4035649995 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f79700666c0 0x7f7970068b80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7994153840 tx=0x7f797c009040 comp rx=0 tx=0).stop 2024-05-22T21:17:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.624+0000 7f7999200640 1 -- 172.21.15.161:0/4035649995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799415d5e0 msgr2=0x7f79940b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.624+0000 7f7999200640 1 --2- 172.21.15.161:0/4035649995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799415d5e0 0x7f79940b0fa0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f798400c7c0 tx=0x7f798400cc80 comp rx=0 tx=0).stop 2024-05-22T21:17:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.624+0000 7f7999200640 1 -- 172.21.15.161:0/4035649995 shutdown_connections 2024-05-22T21:17:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.624+0000 7f7999200640 1 --2- 172.21.15.161:0/4035649995 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f79700666c0 0x7f7970068b80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.624+0000 7f7999200640 1 --2- 172.21.15.161:0/4035649995 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f799415e130 0x7f799414c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.624+0000 7f7999200640 1 --2- 172.21.15.161:0/4035649995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f799415dd50 0x7f79941493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.624+0000 7f7999200640 1 --2- 172.21.15.161:0/4035649995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799415d5e0 0x7f79940b0fa0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.624+0000 7f7999200640 1 -- 172.21.15.161:0/4035649995 >> 172.21.15.161:0/4035649995 conn(0x7f79940b0bc0 msgr2=0x7f79940afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.624+0000 7f7999200640 1 -- 172.21.15.161:0/4035649995 shutdown_connections 2024-05-22T21:17:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.624+0000 7f7999200640 1 -- 172.21.15.161:0/4035649995 wait complete. 2024-05-22T21:17:09.647 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-22T21:17:09.647 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-22T21:17:09.647 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-22T21:17:09.648 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:17:09.648 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-22T21:17:09.648 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-22T21:17:09.648 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-22T21:17:09.648 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:17:09.648 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-22T21:17:09.648 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-22T21:17:09.649 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-22T21:17:09.649 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:17:09.649 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-22T21:17:09.649 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-22T21:17:09.649 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-22T21:17:09.649 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:17:09.660 INFO:tasks.ceph.mds.c:Stopped 2024-05-22T21:17:09.660 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-22T21:17:09.661 INFO:tasks.ceph.mds.a:Stopped 2024-05-22T21:17:09.662 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-22T21:17:09.663 INFO:tasks.ceph.mds.b:Stopped 2024-05-22T21:17:09.663 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-22T21:17:09.664 INFO:tasks.ceph.mds.d:Stopped 2024-05-22T21:17:09.664 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-22T21:17:09.708 INFO:tasks.ceph.mds.c:Started 2024-05-22T21:17:09.708 DEBUG:teuthology.parallel:result is None 2024-05-22T21:17:09.709 INFO:tasks.ceph.mds.a:Started 2024-05-22T21:17:09.709 DEBUG:teuthology.parallel:result is None 2024-05-22T21:17:09.713 INFO:tasks.ceph.mds.b:Started 2024-05-22T21:17:09.713 DEBUG:teuthology.parallel:result is None 2024-05-22T21:17:09.714 INFO:tasks.ceph.mds.d:Started 2024-05-22T21:17:09.715 DEBUG:teuthology.parallel:result is None 2024-05-22T21:17:09.715 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-22T21:17:09.793 INFO:tasks.ceph.mds.a.smithi161.stdout:starting mds.a at 2024-05-22T21:17:09.793 INFO:tasks.ceph.mds.c.smithi161.stdout:starting mds.c at 2024-05-22T21:17:09.801 INFO:tasks.ceph.mds.d.smithi179.stdout:starting mds.d at 2024-05-22T21:17:09.802 INFO:tasks.ceph.mds.b.smithi179.stdout:starting mds.b at 2024-05-22T21:17:09.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.856+0000 7f6ab5800640 1 Processor -- start 2024-05-22T21:17:09.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.856+0000 7f6ab5800640 1 -- start start 2024-05-22T21:17:09.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.856+0000 7f6ab5800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab015d400 0x7f6ab00cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:09.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.856+0000 7f6ab5800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6ab006edb0 0x7f6ab00cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:09.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.856+0000 7f6ab5800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6ab00cdce0 0x7f6ab00cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:09.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.856+0000 7f6ab5800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab00ce170 con 0x7f6ab006edb0 2024-05-22T21:17:09.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.860+0000 7f6ab5800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab00d1c00 con 0x7f6ab00cdce0 2024-05-22T21:17:09.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.860+0000 7f6ab5800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab006e920 con 0x7f6ab015d400 2024-05-22T21:17:09.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.860+0000 7f6aae000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6ab006edb0 0x7f6ab00cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:09.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.860+0000 7f6aae000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6ab006edb0 0x7f6ab00cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33528/0 (socket says 172.21.15.161:33528) 2024-05-22T21:17:09.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.860+0000 7f6aae000640 1 -- 172.21.15.161:0/763873723 learned_addr learned my addr 172.21.15.161:0/763873723 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:09.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.860+0000 7f6aaea00640 1 --2- 172.21.15.161:0/763873723 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab015d400 0x7f6ab00cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:09.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.860+0000 7f6aaf400640 1 --2- 172.21.15.161:0/763873723 >> v2:172.21.15.179:3300/0 conn(0x7f6ab00cdce0 0x7f6ab00cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:09.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.860+0000 7f6aae000640 1 -- 172.21.15.161:0/763873723 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab015d400 msgr2=0x7f6ab00cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:09.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.860+0000 7f6aae000640 1 --2- 172.21.15.161:0/763873723 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab015d400 0x7f6ab00cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.860+0000 7f6aae000640 1 -- 172.21.15.161:0/763873723 >> v2:172.21.15.179:3300/0 conn(0x7f6ab00cdce0 msgr2=0x7f6ab00cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:09.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.860+0000 7f6aae000640 1 --2- 172.21.15.161:0/763873723 >> v2:172.21.15.179:3300/0 conn(0x7f6ab00cdce0 0x7f6ab00cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.860+0000 7f6aae000640 1 -- 172.21.15.161:0/763873723 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab00d0180 con 0x7f6ab006edb0 2024-05-22T21:17:09.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.860+0000 7f6aaf400640 1 --2- 172.21.15.161:0/763873723 >> v2:172.21.15.179:3300/0 conn(0x7f6ab00cdce0 0x7f6ab00cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:17:09.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.860+0000 7f6aaea00640 1 --2- 172.21.15.161:0/763873723 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab015d400 0x7f6ab00cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:17:09.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.860+0000 7f6aae000640 1 --2- 172.21.15.161:0/763873723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab006edb0 0x7f6ab00cd7a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00710f0 tx=0x7f6a9c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d0104f1c7dcce5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:09.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6aad600640 1 -- 172.21.15.161:0/763873723 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c0ab620 con 0x7f6ab006edb0 2024-05-22T21:17:09.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6aad600640 1 -- 172.21.15.161:0/763873723 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9c0ab780 con 0x7f6ab006edb0 2024-05-22T21:17:09.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6aad600640 1 -- 172.21.15.161:0/763873723 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c0abb10 con 0x7f6ab006edb0 2024-05-22T21:17:09.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 -- 172.21.15.161:0/763873723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab006edb0 msgr2=0x7f6ab00cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:09.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 --2- 172.21.15.161:0/763873723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab006edb0 0x7f6ab00cd7a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00710f0 tx=0x7f6a9c01c460 comp rx=0 tx=0).stop 2024-05-22T21:17:09.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 -- 172.21.15.161:0/763873723 shutdown_connections 2024-05-22T21:17:09.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 --2- 172.21.15.161:0/763873723 >> v2:172.21.15.179:3300/0 conn(0x7f6ab00cdce0 0x7f6ab00cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 --2- 172.21.15.161:0/763873723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab006edb0 0x7f6ab00cd7a0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 --2- 172.21.15.161:0/763873723 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab015d400 0x7f6ab00cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 -- 172.21.15.161:0/763873723 >> 172.21.15.161:0/763873723 conn(0x7f6ab00d0f20 msgr2=0x7f6ab00d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:09.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 -- 172.21.15.161:0/763873723 shutdown_connections 2024-05-22T21:17:09.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 -- 172.21.15.161:0/763873723 wait complete. 2024-05-22T21:17:09.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 Processor -- start 2024-05-22T21:17:09.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 -- start start 2024-05-22T21:17:09.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab006edb0 0x7f6ab00c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:09.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab00cdce0 0x7f6ab01550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:09.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab015d400 0x7f6ab015f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:09.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab00c53b0 con 0x7f6ab006edb0 2024-05-22T21:17:09.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab0155600 con 0x7f6ab00cdce0 2024-05-22T21:17:09.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6ab5800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab00b6b20 con 0x7f6ab015d400 2024-05-22T21:17:09.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6aaea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab006edb0 0x7f6ab00c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:09.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6aaea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab006edb0 0x7f6ab00c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33530/0 (socket says 172.21.15.161:33530) 2024-05-22T21:17:09.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.864+0000 7f6aaea00640 1 -- 172.21.15.161:0/536129457 learned_addr learned my addr 172.21.15.161:0/536129457 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:09.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.868+0000 7f6aaea00640 1 -- 172.21.15.161:0/536129457 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab015d400 msgr2=0x7f6ab015f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:09.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.868+0000 7f6aaea00640 1 --2- 172.21.15.161:0/536129457 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab015d400 0x7f6ab015f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.868+0000 7f6aaea00640 1 -- 172.21.15.161:0/536129457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab00cdce0 msgr2=0x7f6ab01550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:09.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.868+0000 7f6aaea00640 1 --2- 172.21.15.161:0/536129457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab00cdce0 0x7f6ab01550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:09.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.868+0000 7f6aaea00640 1 -- 172.21.15.161:0/536129457 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c0094d0 con 0x7f6ab006edb0 2024-05-22T21:17:09.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.868+0000 7f6aaea00640 1 --2- 172.21.15.161:0/536129457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab006edb0 0x7f6ab00c57d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000c7c0 tx=0x7f6aa000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:09.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.868+0000 7f6aa6a00640 1 -- 172.21.15.161:0/536129457 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0007a30 con 0x7f6ab006edb0 2024-05-22T21:17:09.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.868+0000 7f6aa6a00640 1 -- 172.21.15.161:0/536129457 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa0007b90 con 0x7f6ab006edb0 2024-05-22T21:17:09.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.868+0000 7f6ab5800640 1 -- 172.21.15.161:0/536129457 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab00c4a10 con 0x7f6ab006edb0 2024-05-22T21:17:09.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.868+0000 7f6aa6a00640 1 -- 172.21.15.161:0/536129457 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa000ce70 con 0x7f6ab006edb0 2024-05-22T21:17:09.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.868+0000 7f6ab5800640 1 -- 172.21.15.161:0/536129457 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab00c4fd0 con 0x7f6ab006edb0 2024-05-22T21:17:09.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.868+0000 7f6ab5800640 1 -- 172.21.15.161:0/536129457 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab0067220 con 0x7f6ab006edb0 2024-05-22T21:17:09.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.868+0000 7f6aa6a00640 1 -- 172.21.15.161:0/536129457 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6aa00113f0 con 0x7f6ab006edb0 2024-05-22T21:17:09.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.868+0000 7f6aa6a00640 1 --2- 172.21.15.161:0/536129457 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a90066690 0x7f6a90068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:09.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.868+0000 7f6aa6a00640 1 -- 172.21.15.161:0/536129457 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6aa00842f0 con 0x7f6ab006edb0 2024-05-22T21:17:09.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.868+0000 7f6aae000640 1 --2- 172.21.15.161:0/536129457 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a90066690 0x7f6a90068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:09.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.872+0000 7f6aae000640 1 --2- 172.21.15.161:0/536129457 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a90066690 0x7f6a90068b50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0af000 tx=0x7f6a9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:09.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:09.872+0000 7f6aa6a00640 1 -- 172.21.15.161:0/536129457 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6aa0051940 con 0x7f6ab006edb0 2024-05-22T21:17:10.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.036+0000 7f6ab5800640 1 -- 172.21.15.161:0/536129457 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6ab0160990 con 0x7f6ab006edb0 2024-05-22T21:17:10.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.824+0000 7f6aa6a00640 1 -- 172.21.15.161:0/536129457 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v80) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6aa0054f60 con 0x7f6ab006edb0 2024-05-22T21:17:10.828 INFO:teuthology.orchestra.run.smithi161.stderr: removed all blocklist entries 2024-05-22T21:17:10.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.828+0000 7f6ab5800640 1 -- 172.21.15.161:0/536129457 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a90066690 msgr2=0x7f6a90068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:10.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.828+0000 7f6ab5800640 1 --2- 172.21.15.161:0/536129457 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a90066690 0x7f6a90068b50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0af000 tx=0x7f6a9c008040 comp rx=0 tx=0).stop 2024-05-22T21:17:10.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.828+0000 7f6ab5800640 1 -- 172.21.15.161:0/536129457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab006edb0 msgr2=0x7f6ab00c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:10.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.828+0000 7f6ab5800640 1 --2- 172.21.15.161:0/536129457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab006edb0 0x7f6ab00c57d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000c7c0 tx=0x7f6aa000cc80 comp rx=0 tx=0).stop 2024-05-22T21:17:10.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.828+0000 7f6ab5800640 1 -- 172.21.15.161:0/536129457 shutdown_connections 2024-05-22T21:17:10.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.828+0000 7f6ab5800640 1 --2- 172.21.15.161:0/536129457 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a90066690 0x7f6a90068b50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:10.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.828+0000 7f6ab5800640 1 --2- 172.21.15.161:0/536129457 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab015d400 0x7f6ab015f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:10.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.828+0000 7f6ab5800640 1 --2- 172.21.15.161:0/536129457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab00cdce0 0x7f6ab01550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:10.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.828+0000 7f6ab5800640 1 --2- 172.21.15.161:0/536129457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ab006edb0 0x7f6ab00c57d0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:10.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.828+0000 7f6ab5800640 1 -- 172.21.15.161:0/536129457 >> 172.21.15.161:0/536129457 conn(0x7f6ab00d0f20 msgr2=0x7f6ab00ae7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:10.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.828+0000 7f6ab5800640 1 -- 172.21.15.161:0/536129457 shutdown_connections 2024-05-22T21:17:10.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.828+0000 7f6ab5800640 1 -- 172.21.15.161:0/536129457 wait complete. 2024-05-22T21:17:10.851 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-22T21:17:10.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.984+0000 7f3021200640 1 Processor -- start 2024-05-22T21:17:10.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.984+0000 7f3021200640 1 -- start start 2024-05-22T21:17:10.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.984+0000 7f3021200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f301c143ec0 0x7f301c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:10.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.984+0000 7f3021200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f301c0715a0 0x7f301c06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:10.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.984+0000 7f3021200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f301c1442a0 con 0x7f301c15ec90 2024-05-22T21:17:10.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.984+0000 7f3021200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f301c070260 con 0x7f301c0715a0 2024-05-22T21:17:10.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.984+0000 7f3021200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f301c070500 con 0x7f301c143ec0 2024-05-22T21:17:10.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.984+0000 7f3019400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f301c0715a0 0x7f301c06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:10.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.984+0000 7f301a800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f301c15ec90 0x7f301c06bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54448/0 (socket says 172.21.15.161:54448) 2024-05-22T21:17:10.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.984+0000 7f301a800640 1 -- 172.21.15.161:0/819925002 learned_addr learned my addr 172.21.15.161:0/819925002 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:10.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.988+0000 7f3019e00640 1 --2- 172.21.15.161:0/819925002 >> v2:172.21.15.179:3300/0 conn(0x7f301c143ec0 0x7f301c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:10.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.988+0000 7f3019400640 1 -- 172.21.15.161:0/819925002 >> v2:172.21.15.179:3300/0 conn(0x7f301c143ec0 msgr2=0x7f301c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:10.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.988+0000 7f3019400640 1 --2- 172.21.15.161:0/819925002 >> v2:172.21.15.179:3300/0 conn(0x7f301c143ec0 0x7f301c15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:10.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.988+0000 7f3019400640 1 -- 172.21.15.161:0/819925002 >> v1:172.21.15.161:6789/0 conn(0x7f301c15ec90 legacy=0x7f301c06bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:10.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.988+0000 7f3019400640 1 -- 172.21.15.161:0/819925002 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c0707e0 con 0x7f301c0715a0 2024-05-22T21:17:10.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.988+0000 7f3018a00640 1 -- 172.21.15.161:0/819925002 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3018508764 0 0) 0x7f301c1442a0 con 0x7f301c15ec90 2024-05-22T21:17:10.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.988+0000 7f3019400640 1 --2- 172.21.15.161:0/819925002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f301c0715a0 0x7f301c06fd20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f301c0724e0 tx=0x7f300c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbdbbc10fee65b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:10.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.988+0000 7f3018a00640 1 -- 172.21.15.161:0/819925002 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c0ab650 con 0x7f301c0715a0 2024-05-22T21:17:10.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.988+0000 7f3018a00640 1 -- 172.21.15.161:0/819925002 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300c0ab7b0 con 0x7f301c0715a0 2024-05-22T21:17:10.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.988+0000 7f3018a00640 1 -- 172.21.15.161:0/819925002 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c0abb40 con 0x7f301c0715a0 2024-05-22T21:17:10.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 -- 172.21.15.161:0/819925002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f301c0715a0 msgr2=0x7f301c06fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:10.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 --2- 172.21.15.161:0/819925002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f301c0715a0 0x7f301c06fd20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f301c0724e0 tx=0x7f300c01c450 comp rx=0 tx=0).stop 2024-05-22T21:17:10.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 -- 172.21.15.161:0/819925002 shutdown_connections 2024-05-22T21:17:10.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 --2- 172.21.15.161:0/819925002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f301c0715a0 0x7f301c06fd20 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:10.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 --2- 172.21.15.161:0/819925002 >> v2:172.21.15.179:3300/0 conn(0x7f301c143ec0 0x7f301c15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:10.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 -- 172.21.15.161:0/819925002 >> 172.21.15.161:0/819925002 conn(0x7f301c0b0bc0 msgr2=0x7f301c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:10.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 -- 172.21.15.161:0/819925002 shutdown_connections 2024-05-22T21:17:10.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 -- 172.21.15.161:0/819925002 wait complete. 2024-05-22T21:17:10.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 Processor -- start 2024-05-22T21:17:10.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 -- start start 2024-05-22T21:17:10.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f301c143ec0 0x7f301c158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:10.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f301c15ec90 0x7f301c159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:10.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f301c159e50 0x7f301c15b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:10.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c159ba0 con 0x7f301c15ec90 2024-05-22T21:17:10.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c139150 con 0x7f301c143ec0 2024-05-22T21:17:10.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f301c139430 con 0x7f301c159e50 2024-05-22T21:17:10.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3019400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f301c15ec90 0x7f301c159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:10.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3019400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f301c15ec90 0x7f301c159250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33546/0 (socket says 172.21.15.161:33546) 2024-05-22T21:17:10.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3019400640 1 -- 172.21.15.161:0/1152921994 learned_addr learned my addr 172.21.15.161:0/1152921994 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:10.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3019e00640 1 --2- 172.21.15.161:0/1152921994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f301c143ec0 0x7f301c158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:10.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f301a800640 1 --2- 172.21.15.161:0/1152921994 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f301c159e50 0x7f301c15b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:10.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3019400640 1 -- 172.21.15.161:0/1152921994 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f301c159e50 msgr2=0x7f301c15b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:10.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3019400640 1 --2- 172.21.15.161:0/1152921994 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f301c159e50 0x7f301c15b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:10.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3019400640 1 -- 172.21.15.161:0/1152921994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f301c143ec0 msgr2=0x7f301c158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:10.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3019400640 1 --2- 172.21.15.161:0/1152921994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f301c143ec0 0x7f301c158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:10.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3019400640 1 -- 172.21.15.161:0/1152921994 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300c009420 con 0x7f301c15ec90 2024-05-22T21:17:10.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3019e00640 1 --2- 172.21.15.161:0/1152921994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f301c143ec0 0x7f301c158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:10.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3019400640 1 --2- 172.21.15.161:0/1152921994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f301c15ec90 0x7f301c159250 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f300c019060 tx=0x7f300c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:10.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3012000640 1 -- 172.21.15.161:0/1152921994 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c0bf610 con 0x7f301c15ec90 2024-05-22T21:17:10.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3012000640 1 -- 172.21.15.161:0/1152921994 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300c0bf790 con 0x7f301c15ec90 2024-05-22T21:17:10.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3012000640 1 -- 172.21.15.161:0/1152921994 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c0bfb40 con 0x7f301c15ec90 2024-05-22T21:17:10.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.992+0000 7f3021200640 1 -- 172.21.15.161:0/1152921994 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f301c15f630 con 0x7f301c15ec90 2024-05-22T21:17:10.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.996+0000 7f3021200640 1 -- 172.21.15.161:0/1152921994 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f301c1399f0 con 0x7f301c15ec90 2024-05-22T21:17:11.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.996+0000 7f3021200640 1 -- 172.21.15.161:0/1152921994 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301c139be0 con 0x7f301c15ec90 2024-05-22T21:17:11.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.996+0000 7f3012000640 1 -- 172.21.15.161:0/1152921994 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f300c0bfca0 con 0x7f301c15ec90 2024-05-22T21:17:11.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.996+0000 7f3012000640 1 --2- 172.21.15.161:0/1152921994 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2ff8066690 0x7f2ff8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:11.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.996+0000 7f3012000640 1 -- 172.21.15.161:0/1152921994 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f301c06ce80 con 0x7f301c15ec90 2024-05-22T21:17:11.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.996+0000 7f3019e00640 1 --2- 172.21.15.161:0/1152921994 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2ff8066690 0x7f2ff8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:11.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:10.996+0000 7f3019e00640 1 --2- 172.21.15.161:0/1152921994 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2ff8066690 0x7f2ff8068b50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f301c1f7390 tx=0x7f3004008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:11.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.000+0000 7f3012000640 1 -- 172.21.15.161:0/1152921994 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f300c0ec490 con 0x7f301c15ec90 2024-05-22T21:17:11.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.204+0000 7f3021200640 1 -- 172.21.15.161:0/1152921994 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f301c139570 con 0x7f301c15ec90 2024-05-22T21:17:11.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.204+0000 7f3012000640 1 -- 172.21.15.161:0/1152921994 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f301c139570 con 0x7f301c15ec90 2024-05-22T21:17:11.207 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:11.207 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:17:11.207 INFO:teuthology.orchestra.run.smithi161.stdout: "auth_dump": [ 2024-05-22T21:17:11.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.207 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.a", 2024-05-22T21:17:11.207 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m00QTFRAAPSRrpBRY75FlcWf3QxOOfg==", 2024-05-22T21:17:11.208 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.208 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:17:11.208 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:17:11.208 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:17:11.208 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:17:11.208 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.208 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.b", 2024-05-22T21:17:11.208 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m/ipAIRAA4gj5jGqTpNCr1K2OU/bsAQ==", 2024-05-22T21:17:11.208 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.208 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:17:11.209 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:17:11.209 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:17:11.209 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:17:11.209 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.209 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.c", 2024-05-22T21:17:11.209 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mWlM2HBAAEmJJWMDlTG9E7Qzld2gd5w==", 2024-05-22T21:17:11.209 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.209 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:17:11.209 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:17:11.209 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:17:11.209 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:17:11.209 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.210 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.d", 2024-05-22T21:17:11.210 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mmsuaKBAAqpCYRZZUGK5ZuxflUBcPgg==", 2024-05-22T21:17:11.210 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.210 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:17:11.210 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:17:11.210 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:17:11.210 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:17:11.210 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.211 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.0", 2024-05-22T21:17:11.211 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05mJZokLhAAh2vbQnRX25iYUIbvVmURuQ==", 2024-05-22T21:17:11.211 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.211 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:17:11.211 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:17:11.211 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:17:11.211 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.211 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.1", 2024-05-22T21:17:11.211 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05m8kWNNhAA6aaVT2xshFkqnk4G4gOttA==", 2024-05-22T21:17:11.211 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.211 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:17:11.212 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:17:11.212 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:17:11.212 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.212 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.2", 2024-05-22T21:17:11.212 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05ml1k9AxAAtL3XaJJI6cilpmvurh18Pw==", 2024-05-22T21:17:11.212 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.212 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:17:11.212 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:17:11.212 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:17:11.212 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.213 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.3", 2024-05-22T21:17:11.213 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05mbNBgCxAAas1L+QR4Aez+H1G0EVltfg==", 2024-05-22T21:17:11.213 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.213 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:17:11.213 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:17:11.213 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:17:11.213 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.214 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.4", 2024-05-22T21:17:11.214 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mOPLPABAAZBGIMF/FasNB27n/7di0AA==", 2024-05-22T21:17:11.214 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.214 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:17:11.214 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:17:11.214 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:17:11.214 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.214 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.5", 2024-05-22T21:17:11.214 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mT1poCRAASXm4tdP01pA1VOviqeoLVw==", 2024-05-22T21:17:11.214 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.214 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:17:11.215 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:17:11.215 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:17:11.215 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.215 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.6", 2024-05-22T21:17:11.215 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05m1ZPRERAA2S7WeTwysLCZm1OB/qCrAA==", 2024-05-22T21:17:11.215 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.215 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:17:11.215 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:17:11.215 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:17:11.215 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.216 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.7", 2024-05-22T21:17:11.216 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mCrUhGhAAvXoFWV7X3303rfXaxmfWqA==", 2024-05-22T21:17:11.216 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.216 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:17:11.216 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:17:11.216 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:17:11.216 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.216 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.0", 2024-05-22T21:17:11.216 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mh9B7LBAAevhvj5KpwP9pEIpJcxUvFA==", 2024-05-22T21:17:11.216 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.216 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:17:11.217 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow r", 2024-05-22T21:17:11.217 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-22T21:17:11.217 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.217 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.1", 2024-05-22T21:17:11.217 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mMnHtNRAAg/a/Mb1oRqx73d9efIHCfA==", 2024-05-22T21:17:11.217 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.217 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:17:11.217 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:17:11.217 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:17:11.217 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:17:11.217 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.218 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.2", 2024-05-22T21:17:11.218 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mOzdkMhAAbTiNdvyxqIyrluMiZW0cOw==", 2024-05-22T21:17:11.218 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.218 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:17:11.218 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:17:11.218 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:17:11.218 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:17:11.218 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.218 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.3", 2024-05-22T21:17:11.218 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDYX05mr4oxABAA1+bQQbxhoeU+kb0OCa+Wyw==", 2024-05-22T21:17:11.218 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.219 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:17:11.219 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:17:11.219 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:17:11.219 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:17:11.219 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.219 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.admin", 2024-05-22T21:17:11.219 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDWX05mHZY0NRAAiBvMDkrCZ+MgG2iwBH+bcg==", 2024-05-22T21:17:11.219 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.219 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:17:11.219 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:17:11.219 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:17:11.219 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:17:11.220 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.220 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.x", 2024-05-22T21:17:11.220 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mH2kuERAAgigMUWK3M3WaMXlqGUcNrw==", 2024-05-22T21:17:11.220 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.220 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:17:11.220 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:17:11.220 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:17:11.220 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:17:11.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:11.220 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.y", 2024-05-22T21:17:11.220 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mZRdiDRAA0vUwD+rPSfyhjgA+9XPp5w==", 2024-05-22T21:17:11.220 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:17:11.221 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:17:11.221 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:17:11.221 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:17:11.221 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.221 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:11.221 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:17:11.221 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:17:11.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.204+0000 7f3021200640 1 -- 172.21.15.161:0/1152921994 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2ff8066690 msgr2=0x7f2ff8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:11.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.204+0000 7f3021200640 1 --2- 172.21.15.161:0/1152921994 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2ff8066690 0x7f2ff8068b50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f301c1f7390 tx=0x7f3004008040 comp rx=0 tx=0).stop 2024-05-22T21:17:11.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.204+0000 7f3021200640 1 -- 172.21.15.161:0/1152921994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f301c15ec90 msgr2=0x7f301c159250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:11.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.204+0000 7f3021200640 1 --2- 172.21.15.161:0/1152921994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f301c15ec90 0x7f301c159250 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f300c019060 tx=0x7f300c01ac90 comp rx=0 tx=0).stop 2024-05-22T21:17:11.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.204+0000 7f3021200640 1 -- 172.21.15.161:0/1152921994 shutdown_connections 2024-05-22T21:17:11.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.204+0000 7f3021200640 1 --2- 172.21.15.161:0/1152921994 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2ff8066690 0x7f2ff8068b50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.204+0000 7f3021200640 1 --2- 172.21.15.161:0/1152921994 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f301c159e50 0x7f301c15b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.204+0000 7f3021200640 1 --2- 172.21.15.161:0/1152921994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f301c15ec90 0x7f301c159250 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.204+0000 7f3021200640 1 --2- 172.21.15.161:0/1152921994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f301c143ec0 0x7f301c158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.204+0000 7f3021200640 1 -- 172.21.15.161:0/1152921994 >> 172.21.15.161:0/1152921994 conn(0x7f301c0b0bc0 msgr2=0x7f301c0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:11.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.204+0000 7f3021200640 1 -- 172.21.15.161:0/1152921994 shutdown_connections 2024-05-22T21:17:11.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.204+0000 7f3021200640 1 -- 172.21.15.161:0/1152921994 wait complete. 2024-05-22T21:17:11.229 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:17:11.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.360+0000 7fb6b7c00640 1 Processor -- start 2024-05-22T21:17:11.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.360+0000 7fb6b7c00640 1 -- start start 2024-05-22T21:17:11.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.364+0000 7fb6b7c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb6b015dd50 0x7fb6b006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:11.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.364+0000 7fb6b7c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6b015d5e0 0x7fb6b015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:11.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.364+0000 7fb6b7c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b0071670 con 0x7fb6b015e130 2024-05-22T21:17:11.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.364+0000 7fb6b7c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b006de40 con 0x7fb6b015dd50 2024-05-22T21:17:11.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.364+0000 7fb6b7c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b0070a60 con 0x7fb6b015d5e0 2024-05-22T21:17:11.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.364+0000 7fb6b5200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb6b015dd50 0x7fb6b006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:11.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.364+0000 7fb6b5200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb6b015dd50 0x7fb6b006d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33550/0 (socket says 172.21.15.161:33550) 2024-05-22T21:17:11.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.364+0000 7fb6b5200640 1 -- 172.21.15.161:0/3574957497 learned_addr learned my addr 172.21.15.161:0/3574957497 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:11.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.364+0000 7fb6b5c00640 1 --2- 172.21.15.161:0/3574957497 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6b015d5e0 0x7fb6b015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:11.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.364+0000 7fb6b5200640 1 -- 172.21.15.161:0/3574957497 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6b015d5e0 msgr2=0x7fb6b015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:11.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.364+0000 7fb6b5200640 1 --2- 172.21.15.161:0/3574957497 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6b015d5e0 0x7fb6b015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.364+0000 7fb6b5200640 1 -- 172.21.15.161:0/3574957497 >> v1:172.21.15.179:6789/0 conn(0x7fb6b015e130 legacy=0x7fb6b015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:11.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.364+0000 7fb6b5200640 1 -- 172.21.15.161:0/3574957497 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b0070d40 con 0x7fb6b015dd50 2024-05-22T21:17:11.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b5200640 1 --2- 172.21.15.161:0/3574957497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b015dd50 0x7fb6b006d900 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009870 tx=0x7fb6a001e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb192c2e56e6543b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:11.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6afe00640 1 -- 172.21.15.161:0/3574957497 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00a8710 con 0x7fb6b015dd50 2024-05-22T21:17:11.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6afe00640 1 -- 172.21.15.161:0/3574957497 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a00a8870 con 0x7fb6b015dd50 2024-05-22T21:17:11.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6afe00640 1 -- 172.21.15.161:0/3574957497 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00a8c00 con 0x7fb6b015dd50 2024-05-22T21:17:11.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b7c00640 1 -- 172.21.15.161:0/3574957497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b015dd50 msgr2=0x7fb6b006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:11.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b7c00640 1 --2- 172.21.15.161:0/3574957497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b015dd50 0x7fb6b006d900 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009870 tx=0x7fb6a001e740 comp rx=0 tx=0).stop 2024-05-22T21:17:11.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b7c00640 1 -- 172.21.15.161:0/3574957497 shutdown_connections 2024-05-22T21:17:11.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b7c00640 1 --2- 172.21.15.161:0/3574957497 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6b015d5e0 0x7fb6b015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b7c00640 1 --2- 172.21.15.161:0/3574957497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b015dd50 0x7fb6b006d900 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b7c00640 1 -- 172.21.15.161:0/3574957497 >> 172.21.15.161:0/3574957497 conn(0x7fb6b00b0bc0 msgr2=0x7fb6b00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:11.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b7c00640 1 -- 172.21.15.161:0/3574957497 shutdown_connections 2024-05-22T21:17:11.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b7c00640 1 -- 172.21.15.161:0/3574957497 wait complete. 2024-05-22T21:17:11.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b7c00640 1 Processor -- start 2024-05-22T21:17:11.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b7c00640 1 -- start start 2024-05-22T21:17:11.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b7c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b015d5e0 0x7fb6b0150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:11.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b7c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6b015dd50 0x7fb6b006c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:11.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b7c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b015e130 0x7fb6b006cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:11.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b7c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b00cd800 con 0x7fb6b015e130 2024-05-22T21:17:11.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b7c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b00b27a0 con 0x7fb6b015d5e0 2024-05-22T21:17:11.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b7c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b01f5cb0 con 0x7fb6b015dd50 2024-05-22T21:17:11.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.368+0000 7fb6b5c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b015e130 0x7fb6b006cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:11.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6b5c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b015e130 0x7fb6b006cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33564/0 (socket says 172.21.15.161:33564) 2024-05-22T21:17:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6b5c00640 1 -- 172.21.15.161:0/2860047175 learned_addr learned my addr 172.21.15.161:0/2860047175 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6b5200640 1 --2- 172.21.15.161:0/2860047175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b015d5e0 0x7fb6b0150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6a7e00640 1 --2- 172.21.15.161:0/2860047175 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6b015dd50 0x7fb6b006c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6b5c00640 1 -- 172.21.15.161:0/2860047175 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6b015dd50 msgr2=0x7fb6b006c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6b5c00640 1 --2- 172.21.15.161:0/2860047175 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6b015dd50 0x7fb6b006c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6b5c00640 1 -- 172.21.15.161:0/2860047175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b015d5e0 msgr2=0x7fb6b0150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6b5c00640 1 --2- 172.21.15.161:0/2860047175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b015d5e0 0x7fb6b0150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6b5c00640 1 -- 172.21.15.161:0/2860047175 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a00094d0 con 0x7fb6b015e130 2024-05-22T21:17:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6b5200640 1 --2- 172.21.15.161:0/2860047175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b015d5e0 0x7fb6b0150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6b5c00640 1 --2- 172.21.15.161:0/2860047175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b015e130 0x7fb6b006cd30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009e30 tx=0x7fb6a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6ad600640 1 -- 172.21.15.161:0/2860047175 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8009ca0 con 0x7fb6b015e130 2024-05-22T21:17:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6ad600640 1 -- 172.21.15.161:0/2860047175 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a800f040 con 0x7fb6b015e130 2024-05-22T21:17:11.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6b7c00640 1 -- 172.21.15.161:0/2860047175 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b00cd520 con 0x7fb6b015e130 2024-05-22T21:17:11.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6ad600640 1 -- 172.21.15.161:0/2860047175 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a80146c0 con 0x7fb6b015e130 2024-05-22T21:17:11.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6b7c00640 1 -- 172.21.15.161:0/2860047175 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b00b2430 con 0x7fb6b015e130 2024-05-22T21:17:11.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6b7c00640 1 -- 172.21.15.161:0/2860047175 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b00b2930 con 0x7fb6b015e130 2024-05-22T21:17:11.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.372+0000 7fb6ad600640 1 -- 172.21.15.161:0/2860047175 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb6a8014820 con 0x7fb6b015e130 2024-05-22T21:17:11.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.376+0000 7fb6ad600640 1 --2- 172.21.15.161:0/2860047175 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb68c066690 0x7fb68c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:11.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.376+0000 7fb6ad600640 1 -- 172.21.15.161:0/2860047175 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb6a8084df0 con 0x7fb6b015e130 2024-05-22T21:17:11.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.376+0000 7fb6b5200640 1 --2- 172.21.15.161:0/2860047175 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb68c066690 0x7fb68c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:11.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.376+0000 7fb6b5200640 1 --2- 172.21.15.161:0/2860047175 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb68c066690 0x7fb68c068b50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00b1000 tx=0x7fb6a00a9000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:11.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.380+0000 7fb6ad600640 1 -- 172.21.15.161:0/2860047175 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb6a8052410 con 0x7fb6b015e130 2024-05-22T21:17:11.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.536+0000 7fb6b7c00640 1 -- 172.21.15.161:0/2860047175 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6b0067220 con 0x7fb6b015e130 2024-05-22T21:17:11.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.536+0000 7fb6ad600640 1 -- 172.21.15.161:0/2860047175 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+12437 (secure 0 0 0) 0x7fb6a8018090 con 0x7fb6b015e130 2024-05-22T21:17:11.539 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:11.539 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":80,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:17:10.820772+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:17:11.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.536+0000 7fb6b7c00640 1 -- 172.21.15.161:0/2860047175 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb68c066690 msgr2=0x7fb68c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:11.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.536+0000 7fb6b7c00640 1 --2- 172.21.15.161:0/2860047175 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb68c066690 0x7fb68c068b50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00b1000 tx=0x7fb6a00a9000 comp rx=0 tx=0).stop 2024-05-22T21:17:11.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.536+0000 7fb6b7c00640 1 -- 172.21.15.161:0/2860047175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b015e130 msgr2=0x7fb6b006cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:11.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.536+0000 7fb6b7c00640 1 --2- 172.21.15.161:0/2860047175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b015e130 0x7fb6b006cd30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009e30 tx=0x7fb6a800c490 comp rx=0 tx=0).stop 2024-05-22T21:17:11.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.536+0000 7fb6b7c00640 1 -- 172.21.15.161:0/2860047175 shutdown_connections 2024-05-22T21:17:11.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.536+0000 7fb6b7c00640 1 --2- 172.21.15.161:0/2860047175 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb68c066690 0x7fb68c068b50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.536+0000 7fb6b7c00640 1 --2- 172.21.15.161:0/2860047175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6b015e130 0x7fb6b006cd30 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.536+0000 7fb6b7c00640 1 --2- 172.21.15.161:0/2860047175 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6b015dd50 0x7fb6b006c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.536+0000 7fb6b7c00640 1 --2- 172.21.15.161:0/2860047175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b015d5e0 0x7fb6b0150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.536+0000 7fb6b7c00640 1 -- 172.21.15.161:0/2860047175 >> 172.21.15.161:0/2860047175 conn(0x7fb6b00b0bc0 msgr2=0x7fb6b00afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:11.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.536+0000 7fb6b7c00640 1 -- 172.21.15.161:0/2860047175 shutdown_connections 2024-05-22T21:17:11.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.536+0000 7fb6b7c00640 1 -- 172.21.15.161:0/2860047175 wait complete. 2024-05-22T21:17:11.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:17:11.560 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:17:11.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.696+0000 7f5b13e00640 1 Processor -- start 2024-05-22T21:17:11.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.696+0000 7f5b13e00640 1 -- start start 2024-05-22T21:17:11.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.696+0000 7f5b13e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5b1406ca80 0x7f5b1415a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:11.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.696+0000 7f5b13e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5b1415aa40 0x7f5b1415ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:11.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.696+0000 7f5b13e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b140b2b90 con 0x7f5b1415a110 2024-05-22T21:17:11.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.696+0000 7f5b13e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b140b3020 con 0x7f5b1415aa40 2024-05-22T21:17:11.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.696+0000 7f5b13e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140b1f80 con 0x7f5b1406ca80 2024-05-22T21:17:11.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.696+0000 7f5b13400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5b1415aa40 0x7f5b1415ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:11.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.696+0000 7f5b13400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5b1415aa40 0x7f5b1415ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33570/0 (socket says 172.21.15.161:33570) 2024-05-22T21:17:11.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.696+0000 7f5b13400640 1 -- 172.21.15.161:0/3678745352 learned_addr learned my addr 172.21.15.161:0/3678745352 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:11.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.696+0000 7f5b12000640 1 --2- 172.21.15.161:0/3678745352 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5b1406ca80 0x7f5b1415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:11.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.700+0000 7f5b13400640 1 -- 172.21.15.161:0/3678745352 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5b1406ca80 msgr2=0x7f5b1415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:11.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.700+0000 7f5b13400640 1 --2- 172.21.15.161:0/3678745352 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5b1406ca80 0x7f5b1415a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.700+0000 7f5b13400640 1 -- 172.21.15.161:0/3678745352 >> v1:172.21.15.179:6789/0 conn(0x7f5b1415a110 legacy=0x7f5b1406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:11.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.700+0000 7f5b13400640 1 -- 172.21.15.161:0/3678745352 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b140b2260 con 0x7f5b1415aa40 2024-05-22T21:17:11.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.700+0000 7f5b13400640 1 --2- 172.21.15.161:0/3678745352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b1415aa40 0x7f5b1415ae90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009e50 tx=0x7f5b0c01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69200a3fc8ae3de server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:11.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.700+0000 7f5b11600640 1 -- 172.21.15.161:0/3678745352 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0a7990 con 0x7f5b1415aa40 2024-05-22T21:17:11.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.700+0000 7f5b11600640 1 -- 172.21.15.161:0/3678745352 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0a7af0 con 0x7f5b1415aa40 2024-05-22T21:17:11.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.700+0000 7f5b11600640 1 -- 172.21.15.161:0/3678745352 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0a7e80 con 0x7f5b1415aa40 2024-05-22T21:17:11.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.700+0000 7f5b13e00640 1 -- 172.21.15.161:0/3678745352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b1415aa40 msgr2=0x7f5b1415ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:11.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.700+0000 7f5b13e00640 1 --2- 172.21.15.161:0/3678745352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b1415aa40 0x7f5b1415ae90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009e50 tx=0x7f5b0c01ec50 comp rx=0 tx=0).stop 2024-05-22T21:17:11.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.700+0000 7f5b13e00640 1 -- 172.21.15.161:0/3678745352 shutdown_connections 2024-05-22T21:17:11.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.700+0000 7f5b13e00640 1 --2- 172.21.15.161:0/3678745352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b1415aa40 0x7f5b1415ae90 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.700+0000 7f5b13e00640 1 --2- 172.21.15.161:0/3678745352 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5b1406ca80 0x7f5b1415a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.700+0000 7f5b13e00640 1 -- 172.21.15.161:0/3678745352 >> 172.21.15.161:0/3678745352 conn(0x7f5b140cec00 msgr2=0x7f5b140cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:11.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.700+0000 7f5b13e00640 1 -- 172.21.15.161:0/3678745352 shutdown_connections 2024-05-22T21:17:11.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.700+0000 7f5b13e00640 1 -- 172.21.15.161:0/3678745352 wait complete. 2024-05-22T21:17:11.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b13e00640 1 Processor -- start 2024-05-22T21:17:11.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b13e00640 1 -- start start 2024-05-22T21:17:11.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b13e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b1406ca80 0x7f5b140af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:11.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b13e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b1415aa40 0x7f5b14147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:11.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b13e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5b140af620 0x7f5b14147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:11.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b13e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140cb990 con 0x7f5b1406ca80 2024-05-22T21:17:11.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b13e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140cc7b0 con 0x7f5b1415aa40 2024-05-22T21:17:11.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b13e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b14147ab0 con 0x7f5b140af620 2024-05-22T21:17:11.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b12a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b1406ca80 0x7f5b140af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:11.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b12a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b1406ca80 0x7f5b140af0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33574/0 (socket says 172.21.15.161:33574) 2024-05-22T21:17:11.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b12a00640 1 -- 172.21.15.161:0/3135820650 learned_addr learned my addr 172.21.15.161:0/3135820650 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:11.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b13400640 1 --2- 172.21.15.161:0/3135820650 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5b140af620 0x7f5b14147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:11.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b12000640 1 --2- 172.21.15.161:0/3135820650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b1415aa40 0x7f5b14147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:11.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b12a00640 1 -- 172.21.15.161:0/3135820650 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5b140af620 msgr2=0x7f5b14147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:11.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b12a00640 1 --2- 172.21.15.161:0/3135820650 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5b140af620 0x7f5b14147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b12a00640 1 -- 172.21.15.161:0/3135820650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b1415aa40 msgr2=0x7f5b14147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:11.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b12a00640 1 --2- 172.21.15.161:0/3135820650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b1415aa40 0x7f5b14147030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b12a00640 1 -- 172.21.15.161:0/3135820650 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c009b30 con 0x7f5b1406ca80 2024-05-22T21:17:11.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b12000640 1 --2- 172.21.15.161:0/3135820650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b1415aa40 0x7f5b14147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:11.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b12a00640 1 --2- 172.21.15.161:0/3135820650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b1406ca80 0x7f5b140af0e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400c500 tx=0x7f5b0400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:11.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b0b400640 1 -- 172.21.15.161:0/3135820650 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04005220 con 0x7f5b1406ca80 2024-05-22T21:17:11.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b0b400640 1 -- 172.21.15.161:0/3135820650 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b04005380 con 0x7f5b1406ca80 2024-05-22T21:17:11.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b0b400640 1 -- 172.21.15.161:0/3135820650 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04007270 con 0x7f5b1406ca80 2024-05-22T21:17:11.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b13e00640 1 -- 172.21.15.161:0/3135820650 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b14147f80 con 0x7f5b1406ca80 2024-05-22T21:17:11.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.704+0000 7f5b13e00640 1 -- 172.21.15.161:0/3135820650 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b140cbe80 con 0x7f5b1406ca80 2024-05-22T21:17:11.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.708+0000 7f5b13e00640 1 -- 172.21.15.161:0/3135820650 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b14067070 con 0x7f5b1406ca80 2024-05-22T21:17:11.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.708+0000 7f5b0b400640 1 -- 172.21.15.161:0/3135820650 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5b040073d0 con 0x7f5b1406ca80 2024-05-22T21:17:11.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.708+0000 7f5b0b400640 1 --2- 172.21.15.161:0/3135820650 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5af4066690 0x7f5af4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:11.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.708+0000 7f5b0b400640 1 -- 172.21.15.161:0/3135820650 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5b040836e0 con 0x7f5b1406ca80 2024-05-22T21:17:11.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.708+0000 7f5b12000640 1 --2- 172.21.15.161:0/3135820650 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5af4066690 0x7f5af4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:11.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.708+0000 7f5b12000640 1 --2- 172.21.15.161:0/3135820650 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5af4066690 0x7f5af4068b50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5b14164370 tx=0x7f5b00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:11.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.712+0000 7f5b0b400640 1 -- 172.21.15.161:0/3135820650 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5b04050db0 con 0x7f5b1406ca80 2024-05-22T21:17:11.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.868+0000 7f5b13e00640 1 -- 172.21.15.161:0/3135820650 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b140cb5d0 con 0x7f5b1406ca80 2024-05-22T21:17:11.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.868+0000 7f5b0b400640 1 -- 172.21.15.161:0/3135820650 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b040543d0 con 0x7f5b1406ca80 2024-05-22T21:17:11.870 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:17:11.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.868+0000 7f5b13e00640 1 -- 172.21.15.161:0/3135820650 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5af4066690 msgr2=0x7f5af4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:11.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.868+0000 7f5b13e00640 1 --2- 172.21.15.161:0/3135820650 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5af4066690 0x7f5af4068b50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5b14164370 tx=0x7f5b00008040 comp rx=0 tx=0).stop 2024-05-22T21:17:11.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.868+0000 7f5b13e00640 1 -- 172.21.15.161:0/3135820650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b1406ca80 msgr2=0x7f5b140af0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:11.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.868+0000 7f5b13e00640 1 --2- 172.21.15.161:0/3135820650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b1406ca80 0x7f5b140af0e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400c500 tx=0x7f5b0400c9c0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.868+0000 7f5b13e00640 1 -- 172.21.15.161:0/3135820650 shutdown_connections 2024-05-22T21:17:11.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.868+0000 7f5b13e00640 1 --2- 172.21.15.161:0/3135820650 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5af4066690 0x7f5af4068b50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.868+0000 7f5b13e00640 1 --2- 172.21.15.161:0/3135820650 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5b140af620 0x7f5b14147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.868+0000 7f5b13e00640 1 --2- 172.21.15.161:0/3135820650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b1415aa40 0x7f5b14147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.868+0000 7f5b13e00640 1 --2- 172.21.15.161:0/3135820650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b1406ca80 0x7f5b140af0e0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:11.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.868+0000 7f5b13e00640 1 -- 172.21.15.161:0/3135820650 >> 172.21.15.161:0/3135820650 conn(0x7f5b140cec00 msgr2=0x7f5b1415d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:11.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.868+0000 7f5b13e00640 1 -- 172.21.15.161:0/3135820650 shutdown_connections 2024-05-22T21:17:11.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:11.868+0000 7f5b13e00640 1 -- 172.21.15.161:0/3135820650 wait complete. 2024-05-22T21:17:11.892 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-22T21:17:12.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabdf800640 1 Processor -- start 2024-05-22T21:17:12.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabdf800640 1 -- start start 2024-05-22T21:17:12.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabdf800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fabd80707c0 0x7fabd806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabdf800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fabd806d530 0x7fabd8160e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabdf800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd8071fd0 con 0x7fabd815d430 2024-05-22T21:17:12.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabdf800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd80acfc0 con 0x7fabd806d530 2024-05-22T21:17:12.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabdf800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd8070c00 con 0x7fabd80707c0 2024-05-22T21:17:12.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabdce00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fabd815d430 0x7fabd80ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54452/0 (socket says 172.21.15.161:54452) 2024-05-22T21:17:12.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabdce00640 1 -- 172.21.15.161:0/651414987 learned_addr learned my addr 172.21.15.161:0/651414987 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:12.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabdd800640 1 --2- 172.21.15.161:0/651414987 >> v2:172.21.15.161:3300/0 conn(0x7fabd806d530 0x7fabd8160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabd7400640 1 -- 172.21.15.161:0/651414987 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 673641055 0 0) 0x7fabd8071fd0 con 0x7fabd815d430 2024-05-22T21:17:12.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabd7400640 1 -- 172.21.15.161:0/651414987 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabbc003530 con 0x7fabd815d430 2024-05-22T21:17:12.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabd7e00640 1 --2- 172.21.15.161:0/651414987 >> v2:172.21.15.179:3300/0 conn(0x7fabd80707c0 0x7fabd806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabdd800640 1 -- 172.21.15.161:0/651414987 >> v2:172.21.15.179:3300/0 conn(0x7fabd80707c0 msgr2=0x7fabd806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:12.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabdd800640 1 --2- 172.21.15.161:0/651414987 >> v2:172.21.15.179:3300/0 conn(0x7fabd80707c0 0x7fabd806cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabdd800640 1 -- 172.21.15.161:0/651414987 >> v1:172.21.15.161:6789/0 conn(0x7fabd815d430 legacy=0x7fabd80ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.024+0000 7fabdd800640 1 -- 172.21.15.161:0/651414987 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd8070e80 con 0x7fabd806d530 2024-05-22T21:17:12.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.028+0000 7fabdd800640 1 --2- 172.21.15.161:0/651414987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabd806d530 0x7fabd8160e80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fabd8072330 tx=0x7fabd001c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9b5d6023b59a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:12.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.028+0000 7fabd7400640 1 -- 172.21.15.161:0/651414987 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd00ab600 con 0x7fabd806d530 2024-05-22T21:17:12.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.028+0000 7fabd7400640 1 -- 172.21.15.161:0/651414987 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd00a7050 con 0x7fabd806d530 2024-05-22T21:17:12.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.028+0000 7fabd7400640 1 -- 172.21.15.161:0/651414987 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd00ab980 con 0x7fabd806d530 2024-05-22T21:17:12.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.028+0000 7fabdf800640 1 -- 172.21.15.161:0/651414987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabd806d530 msgr2=0x7fabd8160e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.028+0000 7fabdf800640 1 --2- 172.21.15.161:0/651414987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabd806d530 0x7fabd8160e80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fabd8072330 tx=0x7fabd001c100 comp rx=0 tx=0).stop 2024-05-22T21:17:12.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.028+0000 7fabdf800640 1 -- 172.21.15.161:0/651414987 shutdown_connections 2024-05-22T21:17:12.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.028+0000 7fabdf800640 1 --2- 172.21.15.161:0/651414987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabd806d530 0x7fabd8160e80 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.028+0000 7fabdf800640 1 --2- 172.21.15.161:0/651414987 >> v2:172.21.15.179:3300/0 conn(0x7fabd80707c0 0x7fabd806cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.028+0000 7fabdf800640 1 -- 172.21.15.161:0/651414987 >> 172.21.15.161:0/651414987 conn(0x7fabd80b0bc0 msgr2=0x7fabd80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:12.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.028+0000 7fabdf800640 1 -- 172.21.15.161:0/651414987 shutdown_connections 2024-05-22T21:17:12.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.028+0000 7fabdf800640 1 -- 172.21.15.161:0/651414987 wait complete. 2024-05-22T21:17:12.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdf800640 1 Processor -- start 2024-05-22T21:17:12.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdf800640 1 -- start start 2024-05-22T21:17:12.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdf800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabd806d530 0x7fabd8150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdf800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabd80707c0 0x7fabd81559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdf800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabd815d430 0x7fabd815ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdf800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd8158a90 con 0x7fabd815d430 2024-05-22T21:17:12.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdf800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd815e7b0 con 0x7fabd806d530 2024-05-22T21:17:12.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdf800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd81581f0 con 0x7fabd80707c0 2024-05-22T21:17:12.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdd800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabd815d430 0x7fabd815ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdd800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabd815d430 0x7fabd815ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33590/0 (socket says 172.21.15.161:33590) 2024-05-22T21:17:12.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdd800640 1 -- 172.21.15.161:0/766183026 learned_addr learned my addr 172.21.15.161:0/766183026 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:12.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabd7e00640 1 --2- 172.21.15.161:0/766183026 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabd80707c0 0x7fabd81559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdce00640 1 --2- 172.21.15.161:0/766183026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabd806d530 0x7fabd8150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdd800640 1 -- 172.21.15.161:0/766183026 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabd80707c0 msgr2=0x7fabd81559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdd800640 1 --2- 172.21.15.161:0/766183026 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabd80707c0 0x7fabd81559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdd800640 1 -- 172.21.15.161:0/766183026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabd806d530 msgr2=0x7fabd8150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdd800640 1 --2- 172.21.15.161:0/766183026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabd806d530 0x7fabd8150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdd800640 1 -- 172.21.15.161:0/766183026 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd0009420 con 0x7fabd815d430 2024-05-22T21:17:12.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabd7e00640 1 --2- 172.21.15.161:0/766183026 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabd80707c0 0x7fabd81559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:12.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdd800640 1 --2- 172.21.15.161:0/766183026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabd815d430 0x7fabd815ef70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fabd00ae040 tx=0x7fabd0018ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:12.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabd4c00640 1 -- 172.21.15.161:0/766183026 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd00b4410 con 0x7fabd815d430 2024-05-22T21:17:12.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabd4c00640 1 -- 172.21.15.161:0/766183026 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd00b4590 con 0x7fabd815d430 2024-05-22T21:17:12.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabd4c00640 1 -- 172.21.15.161:0/766183026 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd00b4940 con 0x7fabd815d430 2024-05-22T21:17:12.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdf800640 1 -- 172.21.15.161:0/766183026 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd81584d0 con 0x7fabd815d430 2024-05-22T21:17:12.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdf800640 1 -- 172.21.15.161:0/766183026 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd815ea50 con 0x7fabd815d430 2024-05-22T21:17:12.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.032+0000 7fabdf800640 1 -- 172.21.15.161:0/766183026 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd815ec40 con 0x7fabd815d430 2024-05-22T21:17:12.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.036+0000 7fabd4c00640 1 -- 172.21.15.161:0/766183026 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fabd00b2040 con 0x7fabd815d430 2024-05-22T21:17:12.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.036+0000 7fabd4c00640 1 --2- 172.21.15.161:0/766183026 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fabb8066690 0x7fabb8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.036+0000 7fabd4c00640 1 -- 172.21.15.161:0/766183026 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fabd011df00 con 0x7fabd815d430 2024-05-22T21:17:12.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.036+0000 7fabdce00640 1 --2- 172.21.15.161:0/766183026 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fabb8066690 0x7fabb8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.036+0000 7fabdce00640 1 --2- 172.21.15.161:0/766183026 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fabb8066690 0x7fabb8068b50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fabc8001570 tx=0x7fabc8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:12.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.040+0000 7fabd4c00640 1 -- 172.21.15.161:0/766183026 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fabd00eb520 con 0x7fabd815d430 2024-05-22T21:17:12.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.196+0000 7fabdf800640 1 -- 172.21.15.161:0/766183026 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fabd8067220 con 0x7fabd815d430 2024-05-22T21:17:12.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.196+0000 7fabd4c00640 1 -- 172.21.15.161:0/766183026 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7fabd00eeb40 con 0x7fabd815d430 2024-05-22T21:17:12.200 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:12.200 INFO:teuthology.orchestra.run.smithi161.stdout:[ 2024-05-22T21:17:12.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:17:12.200 INFO:teuthology.orchestra.run.smithi161.stdout: "poolnum": 1, 2024-05-22T21:17:12.200 INFO:teuthology.orchestra.run.smithi161.stdout: "poolname": ".mgr" 2024-05-22T21:17:12.200 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:17:12.200 INFO:teuthology.orchestra.run.smithi161.stdout:] 2024-05-22T21:17:12.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.196+0000 7fabdf800640 1 -- 172.21.15.161:0/766183026 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fabb8066690 msgr2=0x7fabb8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.196+0000 7fabdf800640 1 --2- 172.21.15.161:0/766183026 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fabb8066690 0x7fabb8068b50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fabc8001570 tx=0x7fabc8009040 comp rx=0 tx=0).stop 2024-05-22T21:17:12.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.196+0000 7fabdf800640 1 -- 172.21.15.161:0/766183026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabd815d430 msgr2=0x7fabd815ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.196+0000 7fabdf800640 1 --2- 172.21.15.161:0/766183026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabd815d430 0x7fabd815ef70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fabd00ae040 tx=0x7fabd0018ea0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.196+0000 7fabdf800640 1 -- 172.21.15.161:0/766183026 shutdown_connections 2024-05-22T21:17:12.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.196+0000 7fabdf800640 1 --2- 172.21.15.161:0/766183026 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fabb8066690 0x7fabb8068b50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.196+0000 7fabdf800640 1 --2- 172.21.15.161:0/766183026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabd815d430 0x7fabd815ef70 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.196+0000 7fabdf800640 1 --2- 172.21.15.161:0/766183026 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabd80707c0 0x7fabd81559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.196+0000 7fabdf800640 1 --2- 172.21.15.161:0/766183026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabd806d530 0x7fabd8150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.196+0000 7fabdf800640 1 -- 172.21.15.161:0/766183026 >> 172.21.15.161:0/766183026 conn(0x7fabd80b0bc0 msgr2=0x7fabd80ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:12.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.196+0000 7fabdf800640 1 -- 172.21.15.161:0/766183026 shutdown_connections 2024-05-22T21:17:12.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.196+0000 7fabdf800640 1 -- 172.21.15.161:0/766183026 wait complete. 2024-05-22T21:17:12.219 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-22T21:17:12.220 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-22T21:17:12.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f822fc00640 1 Processor -- start 2024-05-22T21:17:12.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f822fc00640 1 -- start start 2024-05-22T21:17:12.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f822fc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f822815e1d0 0x7f82280ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f822fc00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82280ce830 con 0x7f822813cfe0 2024-05-22T21:17:12.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f822fc00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82280ceb10 con 0x7f822813d560 2024-05-22T21:17:12.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f822fc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8228140560 con 0x7f822815e1d0 2024-05-22T21:17:12.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f822d200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f822813cfe0 0x7f822815dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54462/0 (socket says 172.21.15.161:54462) 2024-05-22T21:17:12.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f822d200640 1 -- 172.21.15.161:0/1944780962 learned_addr learned my addr 172.21.15.161:0/1944780962 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:12.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f822dc00640 1 --2- 172.21.15.161:0/1944780962 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f822815e1d0 0x7f82280ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 -- 172.21.15.161:0/1944780962 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79432290 0 0) 0x7f82280ce830 con 0x7f822813cfe0 2024-05-22T21:17:12.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 -- 172.21.15.161:0/1944780962 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f820c005740 con 0x7f822813cfe0 2024-05-22T21:17:12.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 -- 172.21.15.161:0/1944780962 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 783577987 0 0) 0x7f82280ceb10 con 0x7f822813d560 2024-05-22T21:17:12.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 -- 172.21.15.161:0/1944780962 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82280ce830 con 0x7f822813d560 2024-05-22T21:17:12.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 -- 172.21.15.161:0/1944780962 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3140000452 0 0) 0x7f820c005740 con 0x7f822813cfe0 2024-05-22T21:17:12.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 -- 172.21.15.161:0/1944780962 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f822815e1d0 msgr2=0x7f82280ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:12.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 --2- 172.21.15.161:0/1944780962 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f822815e1d0 0x7f82280ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 -- 172.21.15.161:0/1944780962 >> v1:172.21.15.179:6789/0 conn(0x7f822813d560 legacy=0x7f82280cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 -- 172.21.15.161:0/1944780962 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822806fd80 con 0x7f822813cfe0 2024-05-22T21:17:12.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 -- 172.21.15.161:0/1944780962 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f82180016a0 con 0x7f822813cfe0 2024-05-22T21:17:12.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 -- 172.21.15.161:0/1944780962 >> v1:172.21.15.161:6789/0 conn(0x7f822813cfe0 legacy=0x7f822815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 --2- 172.21.15.161:0/1944780962 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f822815e1d0 0x7f820c005d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 --2- 172.21.15.161:0/1944780962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f820c0062b0 0x7f820c0069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 --2- 172.21.15.161:0/1944780962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f820c006f20 0x7f820c0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 -- 172.21.15.161:0/1944780962 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8228140560 con 0x7f820c006f20 2024-05-22T21:17:12.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 -- 172.21.15.161:0/1944780962 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c00dd30 con 0x7f820c0062b0 2024-05-22T21:17:12.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f8227e00640 1 -- 172.21.15.161:0/1944780962 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c00dec0 con 0x7f822815e1d0 2024-05-22T21:17:12.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.356+0000 7f822dc00640 1 --2- 172.21.15.161:0/1944780962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f820c006f20 0x7f820c0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f821fe00640 1 --2- 172.21.15.161:0/1944780962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f820c0062b0 0x7f820c0069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f822d200640 1 --2- 172.21.15.161:0/1944780962 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f822815e1d0 0x7f820c005d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f822dc00640 1 -- 172.21.15.161:0/1944780962 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f822815e1d0 msgr2=0x7f820c005d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:12.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f822dc00640 1 --2- 172.21.15.161:0/1944780962 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f822815e1d0 0x7f820c005d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f822dc00640 1 -- 172.21.15.161:0/1944780962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f820c0062b0 msgr2=0x7f820c0069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:12.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f822dc00640 1 --2- 172.21.15.161:0/1944780962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f820c0062b0 0x7f820c0069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f822dc00640 1 -- 172.21.15.161:0/1944780962 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c005740 con 0x7f820c006f20 2024-05-22T21:17:12.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f821fe00640 1 --2- 172.21.15.161:0/1944780962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f820c0062b0 0x7f820c0069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:12.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f822dc00640 1 --2- 172.21.15.161:0/1944780962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f820c006f20 0x7f820c0073a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8220009fc0 tx=0x7f822001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af753837b3c247a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:12.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f8227e00640 1 -- 172.21.15.161:0/1944780962 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82200a5c90 con 0x7f820c006f20 2024-05-22T21:17:12.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f8227e00640 1 -- 172.21.15.161:0/1944780962 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822000f040 con 0x7f820c006f20 2024-05-22T21:17:12.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f8227e00640 1 -- 172.21.15.161:0/1944780962 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82200ad440 con 0x7f820c006f20 2024-05-22T21:17:12.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f822fc00640 1 -- 172.21.15.161:0/1944780962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f820c006f20 msgr2=0x7f820c0073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f822fc00640 1 --2- 172.21.15.161:0/1944780962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f820c006f20 0x7f820c0073a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8220009fc0 tx=0x7f822001cc80 comp rx=0 tx=0).stop 2024-05-22T21:17:12.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f822dc00640 1 -- 172.21.15.161:0/1944780962 reap_dead start 2024-05-22T21:17:12.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f822fc00640 1 -- 172.21.15.161:0/1944780962 shutdown_connections 2024-05-22T21:17:12.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.360+0000 7f822fc00640 1 -- 172.21.15.161:0/1944780962 >> 172.21.15.161:0/1944780962 conn(0x7f822806a0a0 msgr2=0x7f822806a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:12.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f822fc00640 1 -- 172.21.15.161:0/1944780962 shutdown_connections 2024-05-22T21:17:12.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f822fc00640 1 -- 172.21.15.161:0/1944780962 wait complete. 2024-05-22T21:17:12.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f822fc00640 1 Processor -- start 2024-05-22T21:17:12.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f822fc00640 1 -- start start 2024-05-22T21:17:12.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f822fc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f820c006f20 0x7f82281541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f822fc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8228153250 0x7f822813bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f822fc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8228156490 0x7f8228156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f822fc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8228153a40 con 0x7f8228153250 2024-05-22T21:17:12.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f822fc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82281556d0 con 0x7f8228156490 2024-05-22T21:17:12.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f822fc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82281559b0 con 0x7f820c006f20 2024-05-22T21:17:12.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f821fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8228153250 0x7f822813bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f821fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8228153250 0x7f822813bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33618/0 (socket says 172.21.15.161:33618) 2024-05-22T21:17:12.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f821fe00640 1 -- 172.21.15.161:0/2413687247 learned_addr learned my addr 172.21.15.161:0/2413687247 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:12.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f822d200640 1 --2- 172.21.15.161:0/2413687247 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f820c006f20 0x7f82281541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f822dc00640 1 --2- 172.21.15.161:0/2413687247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8228156490 0x7f8228156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f821fe00640 1 -- 172.21.15.161:0/2413687247 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f820c006f20 msgr2=0x7f82281541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f821fe00640 1 --2- 172.21.15.161:0/2413687247 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f820c006f20 0x7f82281541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f821fe00640 1 -- 172.21.15.161:0/2413687247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8228156490 msgr2=0x7f8228156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f821fe00640 1 --2- 172.21.15.161:0/2413687247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8228156490 0x7f8228156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f821fe00640 1 -- 172.21.15.161:0/2413687247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8228155c90 con 0x7f8228153250 2024-05-22T21:17:12.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f822dc00640 1 --2- 172.21.15.161:0/2413687247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8228156490 0x7f8228156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:12.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f821fe00640 1 --2- 172.21.15.161:0/2413687247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8228153250 0x7f822813bb90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8210007790 tx=0x7f821000f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:12.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f8225600640 1 -- 172.21.15.161:0/2413687247 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210015610 con 0x7f8228153250 2024-05-22T21:17:12.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f8225600640 1 -- 172.21.15.161:0/2413687247 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8210015770 con 0x7f8228153250 2024-05-22T21:17:12.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f8225600640 1 -- 172.21.15.161:0/2413687247 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210015ae0 con 0x7f8228153250 2024-05-22T21:17:12.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.364+0000 7f822fc00640 1 -- 172.21.15.161:0/2413687247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8228155f70 con 0x7f8228153250 2024-05-22T21:17:12.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.368+0000 7f822fc00640 1 -- 172.21.15.161:0/2413687247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8228153690 con 0x7f8228153250 2024-05-22T21:17:12.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.368+0000 7f822fc00640 1 -- 172.21.15.161:0/2413687247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8228067220 con 0x7f8228153250 2024-05-22T21:17:12.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.368+0000 7f8225600640 1 -- 172.21.15.161:0/2413687247 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8210015c40 con 0x7f8228153250 2024-05-22T21:17:12.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.368+0000 7f8225600640 1 --2- 172.21.15.161:0/2413687247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f82040666c0 0x7f8204068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.368+0000 7f8225600640 1 -- 172.21.15.161:0/2413687247 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f82100877b0 con 0x7f8228153250 2024-05-22T21:17:12.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.368+0000 7f822d200640 1 --2- 172.21.15.161:0/2413687247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f82040666c0 0x7f8204068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.368+0000 7f822d200640 1 --2- 172.21.15.161:0/2413687247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f82040666c0 0x7f8204068b80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8218001230 tx=0x7f8218009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:12.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.372+0000 7f8225600640 1 -- 172.21.15.161:0/2413687247 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8210054e60 con 0x7f8228153250 2024-05-22T21:17:12.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.528+0000 7f822fc00640 1 -- 172.21.15.161:0/2413687247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8228159f60 con 0x7f8228153250 2024-05-22T21:17:12.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.624+0000 7f8225600640 1 -- 172.21.15.161:0/2413687247 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8210058480 con 0x7f8228153250 2024-05-22T21:17:12.629 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' created 2024-05-22T21:17:12.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.628+0000 7f822fc00640 1 -- 172.21.15.161:0/2413687247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f82040666c0 msgr2=0x7f8204068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.628+0000 7f822fc00640 1 --2- 172.21.15.161:0/2413687247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f82040666c0 0x7f8204068b80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8218001230 tx=0x7f8218009040 comp rx=0 tx=0).stop 2024-05-22T21:17:12.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.628+0000 7f822fc00640 1 -- 172.21.15.161:0/2413687247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8228153250 msgr2=0x7f822813bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.628+0000 7f822fc00640 1 --2- 172.21.15.161:0/2413687247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8228153250 0x7f822813bb90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8210007790 tx=0x7f821000f540 comp rx=0 tx=0).stop 2024-05-22T21:17:12.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.628+0000 7f822fc00640 1 -- 172.21.15.161:0/2413687247 shutdown_connections 2024-05-22T21:17:12.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.628+0000 7f822fc00640 1 --2- 172.21.15.161:0/2413687247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f82040666c0 0x7f8204068b80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.628+0000 7f822fc00640 1 --2- 172.21.15.161:0/2413687247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8228156490 0x7f8228156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.628+0000 7f822fc00640 1 --2- 172.21.15.161:0/2413687247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8228153250 0x7f822813bb90 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.628+0000 7f822fc00640 1 --2- 172.21.15.161:0/2413687247 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f820c006f20 0x7f82281541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.628+0000 7f822fc00640 1 -- 172.21.15.161:0/2413687247 >> 172.21.15.161:0/2413687247 conn(0x7f822806a0a0 msgr2=0x7f82280cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:12.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.628+0000 7f822fc00640 1 -- 172.21.15.161:0/2413687247 shutdown_connections 2024-05-22T21:17:12.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.628+0000 7f822fc00640 1 -- 172.21.15.161:0/2413687247 wait complete. 2024-05-22T21:17:12.652 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-22T21:17:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.792+0000 7fc285800640 1 Processor -- start 2024-05-22T21:17:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.792+0000 7fc285800640 1 -- start start 2024-05-22T21:17:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.792+0000 7fc285800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc2800ce370 0x7fc2801442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.792+0000 7fc285800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2800ce830 con 0x7fc2801448e0 2024-05-22T21:17:12.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.792+0000 7fc285800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2800b2e60 con 0x7fc280143ed0 2024-05-22T21:17:12.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.792+0000 7fc285800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2800b3140 con 0x7fc2800ce370 2024-05-22T21:17:12.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.792+0000 7fc284e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc2801448e0 0x7fc28006bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54478/0 (socket says 172.21.15.161:54478) 2024-05-22T21:17:12.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.792+0000 7fc277400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc2800ce370 0x7fc2801442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.792+0000 7fc284e00640 1 -- 172.21.15.161:0/3433909035 learned_addr learned my addr 172.21.15.161:0/3433909035 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:12.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.796+0000 7fc277400640 1 -- 172.21.15.161:0/3433909035 >> v1:172.21.15.179:6789/0 conn(0x7fc280143ed0 legacy=0x7fc2800b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.796+0000 7fc277400640 1 -- 172.21.15.161:0/3433909035 >> v1:172.21.15.161:6789/0 conn(0x7fc2801448e0 legacy=0x7fc28006bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.796+0000 7fc277400640 1 -- 172.21.15.161:0/3433909035 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2800b3420 con 0x7fc2800ce370 2024-05-22T21:17:12.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.796+0000 7fc27f400640 1 -- 172.21.15.161:0/3433909035 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215183119 0 0) 0x7fc2800ce830 con 0x7fc2801448e0 2024-05-22T21:17:12.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.796+0000 7fc277400640 1 --2- 172.21.15.161:0/3433909035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2800ce370 0x7fc2801442b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc2680097c0 tx=0x7fc26801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f017d4159c2a9c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:12.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.796+0000 7fc27f400640 1 -- 172.21.15.161:0/3433909035 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2680a7070 con 0x7fc2800ce370 2024-05-22T21:17:12.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.796+0000 7fc27f400640 1 -- 172.21.15.161:0/3433909035 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2680ab430 con 0x7fc2800ce370 2024-05-22T21:17:12.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.796+0000 7fc27f400640 1 -- 172.21.15.161:0/3433909035 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2680ab7c0 con 0x7fc2800ce370 2024-05-22T21:17:12.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.796+0000 7fc285800640 1 -- 172.21.15.161:0/3433909035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2800ce370 msgr2=0x7fc2801442b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.796+0000 7fc285800640 1 --2- 172.21.15.161:0/3433909035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2800ce370 0x7fc2801442b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc2680097c0 tx=0x7fc26801c410 comp rx=0 tx=0).stop 2024-05-22T21:17:12.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.796+0000 7fc285800640 1 -- 172.21.15.161:0/3433909035 shutdown_connections 2024-05-22T21:17:12.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.796+0000 7fc285800640 1 --2- 172.21.15.161:0/3433909035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2800ce370 0x7fc2801442b0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.796+0000 7fc285800640 1 -- 172.21.15.161:0/3433909035 >> 172.21.15.161:0/3433909035 conn(0x7fc2800cec00 msgr2=0x7fc2800cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:12.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.796+0000 7fc285800640 1 -- 172.21.15.161:0/3433909035 shutdown_connections 2024-05-22T21:17:12.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.796+0000 7fc285800640 1 -- 172.21.15.161:0/3433909035 wait complete. 2024-05-22T21:17:12.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc285800640 1 Processor -- start 2024-05-22T21:17:12.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc285800640 1 -- start start 2024-05-22T21:17:12.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc285800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2800ce370 0x7fc2800ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc285800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc280143ed0 0x7fc280140760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc285800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2801448e0 0x7fc2801598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc285800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2800c8010 con 0x7fc280143ed0 2024-05-22T21:17:12.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc285800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2800c82f0 con 0x7fc2801448e0 2024-05-22T21:17:12.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc285800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2800c85d0 con 0x7fc2800ce370 2024-05-22T21:17:12.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc277400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc280143ed0 0x7fc280140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc277400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc280143ed0 0x7fc280140760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33630/0 (socket says 172.21.15.161:33630) 2024-05-22T21:17:12.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc277400640 1 -- 172.21.15.161:0/3629087339 learned_addr learned my addr 172.21.15.161:0/3629087339 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:12.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc27fe00640 1 --2- 172.21.15.161:0/3629087339 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2800ce370 0x7fc2800ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc284e00640 1 --2- 172.21.15.161:0/3629087339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2801448e0 0x7fc2801598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc277400640 1 -- 172.21.15.161:0/3629087339 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2800ce370 msgr2=0x7fc2800ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc277400640 1 --2- 172.21.15.161:0/3629087339 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2800ce370 0x7fc2800ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc277400640 1 -- 172.21.15.161:0/3629087339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2801448e0 msgr2=0x7fc2801598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:12.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc277400640 1 --2- 172.21.15.161:0/3629087339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2801448e0 0x7fc2801598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:12.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc277400640 1 -- 172.21.15.161:0/3629087339 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc268009420 con 0x7fc280143ed0 2024-05-22T21:17:12.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc277400640 1 --2- 172.21.15.161:0/3629087339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc280143ed0 0x7fc280140760 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc2680ad040 tx=0x7fc26801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:12.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc27cc00640 1 -- 172.21.15.161:0/3629087339 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268004220 con 0x7fc280143ed0 2024-05-22T21:17:12.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc27cc00640 1 -- 172.21.15.161:0/3629087339 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2680b9070 con 0x7fc280143ed0 2024-05-22T21:17:12.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc27cc00640 1 -- 172.21.15.161:0/3629087339 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268003620 con 0x7fc280143ed0 2024-05-22T21:17:12.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc285800640 1 -- 172.21.15.161:0/3629087339 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2800c88b0 con 0x7fc280143ed0 2024-05-22T21:17:12.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.800+0000 7fc285800640 1 -- 172.21.15.161:0/3629087339 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2800cada0 con 0x7fc280143ed0 2024-05-22T21:17:12.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.804+0000 7fc285800640 1 -- 172.21.15.161:0/3629087339 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc280067070 con 0x7fc280143ed0 2024-05-22T21:17:12.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.804+0000 7fc27cc00640 1 -- 172.21.15.161:0/3629087339 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc268003780 con 0x7fc280143ed0 2024-05-22T21:17:12.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.804+0000 7fc27cc00640 1 --2- 172.21.15.161:0/3629087339 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc25c066690 0x7fc25c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:12.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.804+0000 7fc27cc00640 1 -- 172.21.15.161:0/3629087339 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fc2680bb070 con 0x7fc280143ed0 2024-05-22T21:17:12.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.804+0000 7fc27fe00640 1 --2- 172.21.15.161:0/3629087339 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc25c066690 0x7fc25c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:12.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.804+0000 7fc27fe00640 1 --2- 172.21.15.161:0/3629087339 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc25c066690 0x7fc25c068b50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc2801f8eb0 tx=0x7fc270009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:12.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.808+0000 7fc27cc00640 1 -- 172.21.15.161:0/3629087339 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc2680ec530 con 0x7fc280143ed0 2024-05-22T21:17:12.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:12.968+0000 7fc285800640 1 -- 172.21.15.161:0/3629087339 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc2800c8710 con 0x7fc280143ed0 2024-05-22T21:17:13.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.628+0000 7fc27cc00640 1 -- 172.21.15.161:0/3629087339 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc2800c8710 con 0x7fc280143ed0 2024-05-22T21:17:13.633 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' created 2024-05-22T21:17:13.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.632+0000 7fc285800640 1 -- 172.21.15.161:0/3629087339 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc25c066690 msgr2=0x7fc25c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:13.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.632+0000 7fc285800640 1 --2- 172.21.15.161:0/3629087339 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc25c066690 0x7fc25c068b50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc2801f8eb0 tx=0x7fc270009040 comp rx=0 tx=0).stop 2024-05-22T21:17:13.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.632+0000 7fc285800640 1 -- 172.21.15.161:0/3629087339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc280143ed0 msgr2=0x7fc280140760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:13.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.632+0000 7fc285800640 1 --2- 172.21.15.161:0/3629087339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc280143ed0 0x7fc280140760 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc2680ad040 tx=0x7fc26801ac50 comp rx=0 tx=0).stop 2024-05-22T21:17:13.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.636+0000 7fc285800640 1 -- 172.21.15.161:0/3629087339 shutdown_connections 2024-05-22T21:17:13.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.636+0000 7fc285800640 1 --2- 172.21.15.161:0/3629087339 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc25c066690 0x7fc25c068b50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:13.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.636+0000 7fc285800640 1 --2- 172.21.15.161:0/3629087339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2801448e0 0x7fc2801598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:13.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.636+0000 7fc285800640 1 --2- 172.21.15.161:0/3629087339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc280143ed0 0x7fc280140760 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:13.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.636+0000 7fc285800640 1 --2- 172.21.15.161:0/3629087339 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2800ce370 0x7fc2800ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:13.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.636+0000 7fc285800640 1 -- 172.21.15.161:0/3629087339 >> 172.21.15.161:0/3629087339 conn(0x7fc2800cec00 msgr2=0x7fc280142a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:13.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.640+0000 7fc285800640 1 -- 172.21.15.161:0/3629087339 shutdown_connections 2024-05-22T21:17:13.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.640+0000 7fc285800640 1 -- 172.21.15.161:0/3629087339 wait complete. 2024-05-22T21:17:13.667 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-22T21:17:13.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.804+0000 7fda80600640 1 Processor -- start 2024-05-22T21:17:13.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.804+0000 7fda80600640 1 -- start start 2024-05-22T21:17:13.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.808+0000 7fda80600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fda7813cfc0 0x7fda781402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:13.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.808+0000 7fda80600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fda7815af60 0x7fda7813f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:13.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.808+0000 7fda80600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda78140090 con 0x7fda7813fc80 2024-05-22T21:17:13.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.808+0000 7fda80600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda7813d3a0 con 0x7fda7815af60 2024-05-22T21:17:13.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.808+0000 7fda80600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda780cd920 con 0x7fda7813cfc0 2024-05-22T21:17:13.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.808+0000 7fda7d200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fda7815af60 0x7fda7813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:13.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.808+0000 7fda7d200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fda7815af60 0x7fda7813f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33634/0 (socket says 172.21.15.161:33634) 2024-05-22T21:17:13.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.808+0000 7fda7d200640 1 -- 172.21.15.161:0/3146805483 learned_addr learned my addr 172.21.15.161:0/3146805483 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:13.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.808+0000 7fda7dc00640 1 --2- 172.21.15.161:0/3146805483 >> v2:172.21.15.179:3300/0 conn(0x7fda7813cfc0 0x7fda781402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:13.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.808+0000 7fda7d200640 1 -- 172.21.15.161:0/3146805483 >> v2:172.21.15.179:3300/0 conn(0x7fda7813cfc0 msgr2=0x7fda781402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:13.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.808+0000 7fda7d200640 1 --2- 172.21.15.161:0/3146805483 >> v2:172.21.15.179:3300/0 conn(0x7fda7813cfc0 0x7fda781402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:13.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.808+0000 7fda7d200640 1 -- 172.21.15.161:0/3146805483 >> v1:172.21.15.179:6789/0 conn(0x7fda7813fc80 legacy=0x7fda7815dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:13.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.808+0000 7fda7d200640 1 -- 172.21.15.161:0/3146805483 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda780cdc00 con 0x7fda7815af60 2024-05-22T21:17:13.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.808+0000 7fda7dc00640 1 --2- 172.21.15.161:0/3146805483 >> v2:172.21.15.179:3300/0 conn(0x7fda7813cfc0 0x7fda781402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:17:13.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda7d200640 1 --2- 172.21.15.161:0/3146805483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda7815af60 0x7fda7813f740 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fda7806de90 tx=0x7fda6401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b55acc0be2b107b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:13.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda77e00640 1 -- 172.21.15.161:0/3146805483 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda640ab600 con 0x7fda7815af60 2024-05-22T21:17:13.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda77e00640 1 -- 172.21.15.161:0/3146805483 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda640a7050 con 0x7fda7815af60 2024-05-22T21:17:13.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda77e00640 1 -- 172.21.15.161:0/3146805483 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda640ab980 con 0x7fda7815af60 2024-05-22T21:17:13.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda80600640 1 -- 172.21.15.161:0/3146805483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda7815af60 msgr2=0x7fda7813f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:13.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda80600640 1 --2- 172.21.15.161:0/3146805483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda7815af60 0x7fda7813f740 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fda7806de90 tx=0x7fda6401c450 comp rx=0 tx=0).stop 2024-05-22T21:17:13.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda80600640 1 -- 172.21.15.161:0/3146805483 shutdown_connections 2024-05-22T21:17:13.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda80600640 1 --2- 172.21.15.161:0/3146805483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda7815af60 0x7fda7813f740 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:13.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda80600640 1 --2- 172.21.15.161:0/3146805483 >> v2:172.21.15.179:3300/0 conn(0x7fda7813cfc0 0x7fda781402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:13.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda80600640 1 -- 172.21.15.161:0/3146805483 >> 172.21.15.161:0/3146805483 conn(0x7fda7806a2d0 msgr2=0x7fda7806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:13.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda80600640 1 -- 172.21.15.161:0/3146805483 shutdown_connections 2024-05-22T21:17:13.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda80600640 1 -- 172.21.15.161:0/3146805483 wait complete. 2024-05-22T21:17:13.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda80600640 1 Processor -- start 2024-05-22T21:17:13.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda80600640 1 -- start start 2024-05-22T21:17:13.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda80600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda7813cfc0 0x7fda781572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:13.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda80600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda7813fc80 0x7fda7813ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:13.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda80600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda7815af60 0x7fda78140e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:13.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda80600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda780ad010 con 0x7fda7813cfc0 2024-05-22T21:17:13.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda80600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda78157810 con 0x7fda7815af60 2024-05-22T21:17:13.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.812+0000 7fda80600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7813f8c0 con 0x7fda7813fc80 2024-05-22T21:17:13.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda7dc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda7813cfc0 0x7fda781572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:13.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda7dc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda7813cfc0 0x7fda781572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33646/0 (socket says 172.21.15.161:33646) 2024-05-22T21:17:13.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda7dc00640 1 -- 172.21.15.161:0/1241406395 learned_addr learned my addr 172.21.15.161:0/1241406395 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:13.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda7d200640 1 --2- 172.21.15.161:0/1241406395 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda7813fc80 0x7fda7813ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:13.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda7e600640 1 --2- 172.21.15.161:0/1241406395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda7815af60 0x7fda78140e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:13.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda7dc00640 1 -- 172.21.15.161:0/1241406395 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda7813fc80 msgr2=0x7fda7813ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:13.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda7dc00640 1 --2- 172.21.15.161:0/1241406395 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda7813fc80 0x7fda7813ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:13.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda7dc00640 1 -- 172.21.15.161:0/1241406395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda7815af60 msgr2=0x7fda78140e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:13.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda7dc00640 1 --2- 172.21.15.161:0/1241406395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda7815af60 0x7fda78140e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:13.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda7dc00640 1 -- 172.21.15.161:0/1241406395 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda640094d0 con 0x7fda7813cfc0 2024-05-22T21:17:13.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda7dc00640 1 --2- 172.21.15.161:0/1241406395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda7813cfc0 0x7fda781572d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00c820 tx=0x7fda6c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:13.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda75600640 1 -- 172.21.15.161:0/1241406395 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c007a30 con 0x7fda7813cfc0 2024-05-22T21:17:13.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda75600640 1 -- 172.21.15.161:0/1241406395 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c007b90 con 0x7fda7813cfc0 2024-05-22T21:17:13.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda75600640 1 -- 172.21.15.161:0/1241406395 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c00ce10 con 0x7fda7813cfc0 2024-05-22T21:17:13.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda80600640 1 -- 172.21.15.161:0/1241406395 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7813f5e0 con 0x7fda7813cfc0 2024-05-22T21:17:13.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda80600640 1 -- 172.21.15.161:0/1241406395 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda780acc30 con 0x7fda7813cfc0 2024-05-22T21:17:13.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda80600640 1 -- 172.21.15.161:0/1241406395 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda780ad150 con 0x7fda7813cfc0 2024-05-22T21:17:13.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.816+0000 7fda75600640 1 -- 172.21.15.161:0/1241406395 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fda6c024480 con 0x7fda7813cfc0 2024-05-22T21:17:13.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.820+0000 7fda75600640 1 --2- 172.21.15.161:0/1241406395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda58066690 0x7fda58068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:13.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.820+0000 7fda75600640 1 -- 172.21.15.161:0/1241406395 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fda6c084500 con 0x7fda7813cfc0 2024-05-22T21:17:13.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.820+0000 7fda7d200640 1 --2- 172.21.15.161:0/1241406395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda58066690 0x7fda58068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:13.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.820+0000 7fda7d200640 1 --2- 172.21.15.161:0/1241406395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda58066690 0x7fda58068b50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fda640ad000 tx=0x7fda64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:13.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:13.824+0000 7fda75600640 1 -- 172.21.15.161:0/1241406395 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fda6c051940 con 0x7fda7813cfc0 2024-05-22T21:17:14.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.008+0000 7fda80600640 1 -- 172.21.15.161:0/1241406395 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fda780ad150 con 0x7fda7813cfc0 2024-05-22T21:17:14.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.016+0000 7fda75600640 1 -- 172.21.15.161:0/1241406395 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v35) v1 ==== 174+0+0 (secure 0 0 0) 0x7fda780ad150 con 0x7fda7813cfc0 2024-05-22T21:17:14.017 INFO:teuthology.orchestra.run.smithi161.stderr:new fs with metadata pool 10 and data pool 11 2024-05-22T21:17:14.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.020+0000 7fda80600640 1 -- 172.21.15.161:0/1241406395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda58066690 msgr2=0x7fda58068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.020+0000 7fda80600640 1 --2- 172.21.15.161:0/1241406395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda58066690 0x7fda58068b50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fda640ad000 tx=0x7fda64008040 comp rx=0 tx=0).stop 2024-05-22T21:17:14.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.020+0000 7fda80600640 1 -- 172.21.15.161:0/1241406395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda7813cfc0 msgr2=0x7fda781572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.020+0000 7fda80600640 1 --2- 172.21.15.161:0/1241406395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda7813cfc0 0x7fda781572d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00c820 tx=0x7fda6c00cce0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.020+0000 7fda80600640 1 -- 172.21.15.161:0/1241406395 shutdown_connections 2024-05-22T21:17:14.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.020+0000 7fda80600640 1 --2- 172.21.15.161:0/1241406395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda58066690 0x7fda58068b50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.020+0000 7fda80600640 1 --2- 172.21.15.161:0/1241406395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda7815af60 0x7fda78140e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.020+0000 7fda80600640 1 --2- 172.21.15.161:0/1241406395 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda7813fc80 0x7fda7813ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.020+0000 7fda80600640 1 --2- 172.21.15.161:0/1241406395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda7813cfc0 0x7fda781572d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.020+0000 7fda80600640 1 -- 172.21.15.161:0/1241406395 >> 172.21.15.161:0/1241406395 conn(0x7fda7806a2d0 msgr2=0x7fda7815a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:14.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.020+0000 7fda80600640 1 -- 172.21.15.161:0/1241406395 shutdown_connections 2024-05-22T21:17:14.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.020+0000 7fda80600640 1 -- 172.21.15.161:0/1241406395 wait complete. 2024-05-22T21:17:14.054 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:17:14.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a88e00640 1 Processor -- start 2024-05-22T21:17:14.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a88e00640 1 -- start start 2024-05-22T21:17:14.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a88e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a8415f3a0 0x7f3a8415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a88e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a841442a0 con 0x7f3a840715a0 2024-05-22T21:17:14.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a88e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a84070500 con 0x7f3a84143ec0 2024-05-22T21:17:14.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a88e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a840707e0 con 0x7f3a8415f3a0 2024-05-22T21:17:14.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a81400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3a840715a0 0x7f3a8415ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54492/0 (socket says 172.21.15.161:54492) 2024-05-22T21:17:14.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a81400640 1 -- 172.21.15.161:0/1979480698 learned_addr learned my addr 172.21.15.161:0/1979480698 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:14.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a82800640 1 --2- 172.21.15.161:0/1979480698 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a8415f3a0 0x7f3a8415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a80a00640 1 -- 172.21.15.161:0/1979480698 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987276392 0 0) 0x7f3a841442a0 con 0x7f3a840715a0 2024-05-22T21:17:14.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a80a00640 1 -- 172.21.15.161:0/1979480698 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a68003530 con 0x7f3a840715a0 2024-05-22T21:17:14.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a80a00640 1 -- 172.21.15.161:0/1979480698 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4189203330 0 0) 0x7f3a68003530 con 0x7f3a840715a0 2024-05-22T21:17:14.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a80a00640 1 -- 172.21.15.161:0/1979480698 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a8415f3a0 msgr2=0x7f3a8415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:14.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a80a00640 1 --2- 172.21.15.161:0/1979480698 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a8415f3a0 0x7f3a8415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a80a00640 1 -- 172.21.15.161:0/1979480698 >> v1:172.21.15.179:6789/0 conn(0x7f3a84143ec0 legacy=0x7f3a8406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a80a00640 1 -- 172.21.15.161:0/1979480698 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a84070ac0 con 0x7f3a840715a0 2024-05-22T21:17:14.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a80a00640 1 -- 172.21.15.161:0/1979480698 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f3a70001680 con 0x7f3a840715a0 2024-05-22T21:17:14.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a80a00640 1 -- 172.21.15.161:0/1979480698 >> v1:172.21.15.161:6789/0 conn(0x7f3a840715a0 legacy=0x7f3a8415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a80a00640 1 --2- 172.21.15.161:0/1979480698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a680039b0 0x7f3a68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a80a00640 1 --2- 172.21.15.161:0/1979480698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a680047d0 0x7f3a68004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a80a00640 1 --2- 172.21.15.161:0/1979480698 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a8415f3a0 0x7f3a68005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a80a00640 1 -- 172.21.15.161:0/1979480698 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a840707e0 con 0x7f3a680039b0 2024-05-22T21:17:14.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a80a00640 1 -- 172.21.15.161:0/1979480698 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6800ba70 con 0x7f3a680047d0 2024-05-22T21:17:14.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a80a00640 1 -- 172.21.15.161:0/1979480698 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6800bc30 con 0x7f3a8415f3a0 2024-05-22T21:17:14.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a81e00640 1 --2- 172.21.15.161:0/1979480698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a680039b0 0x7f3a68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a82800640 1 --2- 172.21.15.161:0/1979480698 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a8415f3a0 0x7f3a68005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a81400640 1 --2- 172.21.15.161:0/1979480698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a680047d0 0x7f3a68004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a81e00640 1 -- 172.21.15.161:0/1979480698 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a8415f3a0 msgr2=0x7f3a68005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:14.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a81e00640 1 --2- 172.21.15.161:0/1979480698 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a8415f3a0 0x7f3a68005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a81e00640 1 -- 172.21.15.161:0/1979480698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a680047d0 msgr2=0x7f3a68004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:14.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a81e00640 1 --2- 172.21.15.161:0/1979480698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a680047d0 0x7f3a68004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a81e00640 1 -- 172.21.15.161:0/1979480698 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a68003530 con 0x7f3a680039b0 2024-05-22T21:17:14.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.188+0000 7f3a81400640 1 --2- 172.21.15.161:0/1979480698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a680047d0 0x7f3a68004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:14.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.192+0000 7f3a81e00640 1 --2- 172.21.15.161:0/1979480698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a680039b0 0x7f3a68003dd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3a74001e40 tx=0x7f3a7401c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced7a01aa1453bd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:14.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.192+0000 7f3a80a00640 1 -- 172.21.15.161:0/1979480698 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a74003980 con 0x7f3a680039b0 2024-05-22T21:17:14.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.192+0000 7f3a80a00640 1 -- 172.21.15.161:0/1979480698 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a74003ae0 con 0x7f3a680039b0 2024-05-22T21:17:14.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.192+0000 7f3a80a00640 1 -- 172.21.15.161:0/1979480698 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a74003e90 con 0x7f3a680039b0 2024-05-22T21:17:14.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.192+0000 7f3a88e00640 1 -- 172.21.15.161:0/1979480698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a680039b0 msgr2=0x7f3a68003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.192+0000 7f3a88e00640 1 --2- 172.21.15.161:0/1979480698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a680039b0 0x7f3a68003dd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3a74001e40 tx=0x7f3a7401c710 comp rx=0 tx=0).stop 2024-05-22T21:17:14.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.192+0000 7f3a82800640 1 -- 172.21.15.161:0/1979480698 reap_dead start 2024-05-22T21:17:14.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.192+0000 7f3a88e00640 1 -- 172.21.15.161:0/1979480698 shutdown_connections 2024-05-22T21:17:14.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.192+0000 7f3a88e00640 1 -- 172.21.15.161:0/1979480698 >> 172.21.15.161:0/1979480698 conn(0x7f3a840b0bc0 msgr2=0x7f3a840b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:14.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.192+0000 7f3a88e00640 1 -- 172.21.15.161:0/1979480698 shutdown_connections 2024-05-22T21:17:14.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.192+0000 7f3a88e00640 1 -- 172.21.15.161:0/1979480698 wait complete. 2024-05-22T21:17:14.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a88e00640 1 Processor -- start 2024-05-22T21:17:14.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a88e00640 1 -- start start 2024-05-22T21:17:14.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a88e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a680039b0 0x7f3a84160f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a88e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a841428a0 0x7f3a84159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a88e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a84142d00 0x7f3a8415a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a88e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a84160a90 con 0x7f3a84142d00 2024-05-22T21:17:14.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a88e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8415a950 con 0x7f3a841428a0 2024-05-22T21:17:14.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a88e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8415fe80 con 0x7f3a680039b0 2024-05-22T21:17:14.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a82800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a84142d00 0x7f3a8415a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a82800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a84142d00 0x7f3a8415a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33670/0 (socket says 172.21.15.161:33670) 2024-05-22T21:17:14.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a82800640 1 -- 172.21.15.161:0/2413197893 learned_addr learned my addr 172.21.15.161:0/2413197893 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:14.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a81e00640 1 --2- 172.21.15.161:0/2413197893 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a680039b0 0x7f3a84160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a81400640 1 --2- 172.21.15.161:0/2413197893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a841428a0 0x7f3a84159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a82800640 1 -- 172.21.15.161:0/2413197893 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a680039b0 msgr2=0x7f3a84160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a82800640 1 --2- 172.21.15.161:0/2413197893 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a680039b0 0x7f3a84160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a82800640 1 -- 172.21.15.161:0/2413197893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a841428a0 msgr2=0x7f3a84159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a82800640 1 --2- 172.21.15.161:0/2413197893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a841428a0 0x7f3a84159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a82800640 1 -- 172.21.15.161:0/2413197893 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a84160160 con 0x7f3a84142d00 2024-05-22T21:17:14.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a81e00640 1 --2- 172.21.15.161:0/2413197893 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a680039b0 0x7f3a84160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:14.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a81400640 1 --2- 172.21.15.161:0/2413197893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a841428a0 0x7f3a84159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:14.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a82800640 1 --2- 172.21.15.161:0/2413197893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a84142d00 0x7f3a8415a410 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3a840cede0 tx=0x7f3a7c011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:14.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a7a000640 1 -- 172.21.15.161:0/2413197893 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c004240 con 0x7f3a84142d00 2024-05-22T21:17:14.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a7a000640 1 -- 172.21.15.161:0/2413197893 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7c0043a0 con 0x7f3a84142d00 2024-05-22T21:17:14.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a7a000640 1 -- 172.21.15.161:0/2413197893 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c00f9e0 con 0x7f3a84142d00 2024-05-22T21:17:14.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a88e00640 1 -- 172.21.15.161:0/2413197893 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a84160440 con 0x7f3a84142d00 2024-05-22T21:17:14.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.196+0000 7f3a88e00640 1 -- 172.21.15.161:0/2413197893 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a84161460 con 0x7f3a84142d00 2024-05-22T21:17:14.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.200+0000 7f3a88e00640 1 -- 172.21.15.161:0/2413197893 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a84160c40 con 0x7f3a84142d00 2024-05-22T21:17:14.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.200+0000 7f3a7a000640 1 -- 172.21.15.161:0/2413197893 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3a7c00fb40 con 0x7f3a84142d00 2024-05-22T21:17:14.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.200+0000 7f3a7a000640 1 --2- 172.21.15.161:0/2413197893 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a64066690 0x7f3a64068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.200+0000 7f3a7a000640 1 -- 172.21.15.161:0/2413197893 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a7c084420 con 0x7f3a84142d00 2024-05-22T21:17:14.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.200+0000 7f3a81e00640 1 --2- 172.21.15.161:0/2413197893 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a64066690 0x7f3a64068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.200+0000 7f3a81e00640 1 --2- 172.21.15.161:0/2413197893 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a64066690 0x7f3a64068b50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3a74005000 tx=0x7f3a74001520 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:14.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.204+0000 7f3a7a000640 1 -- 172.21.15.161:0/2413197893 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3a84160c40 con 0x7f3a84142d00 2024-05-22T21:17:14.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.360+0000 7f3a88e00640 1 -- 172.21.15.161:0/2413197893 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a84160c40 con 0x7f3a84142d00 2024-05-22T21:17:14.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.360+0000 7f3a7a000640 1 -- 172.21.15.161:0/2413197893 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16161 (secure 0 0 0) 0x7f3a84160c40 con 0x7f3a84142d00 2024-05-22T21:17:14.365 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:14.365 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":83,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:17:14.011920+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:17:12.574579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-22T21:17:13.013467+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:17:14.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.364+0000 7f3a88e00640 1 -- 172.21.15.161:0/2413197893 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a64066690 msgr2=0x7f3a64068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.364+0000 7f3a88e00640 1 --2- 172.21.15.161:0/2413197893 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a64066690 0x7f3a64068b50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3a74005000 tx=0x7f3a74001520 comp rx=0 tx=0).stop 2024-05-22T21:17:14.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.364+0000 7f3a88e00640 1 -- 172.21.15.161:0/2413197893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a84142d00 msgr2=0x7f3a8415a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.364+0000 7f3a88e00640 1 --2- 172.21.15.161:0/2413197893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a84142d00 0x7f3a8415a410 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3a840cede0 tx=0x7f3a7c011cf0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.364+0000 7f3a88e00640 1 -- 172.21.15.161:0/2413197893 shutdown_connections 2024-05-22T21:17:14.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.364+0000 7f3a88e00640 1 --2- 172.21.15.161:0/2413197893 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a64066690 0x7f3a64068b50 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.364+0000 7f3a88e00640 1 --2- 172.21.15.161:0/2413197893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a84142d00 0x7f3a8415a410 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.364+0000 7f3a88e00640 1 --2- 172.21.15.161:0/2413197893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a841428a0 0x7f3a84159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.364+0000 7f3a88e00640 1 --2- 172.21.15.161:0/2413197893 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a680039b0 0x7f3a84160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.364+0000 7f3a88e00640 1 -- 172.21.15.161:0/2413197893 >> 172.21.15.161:0/2413197893 conn(0x7f3a840b0bc0 msgr2=0x7f3a840ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:14.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.364+0000 7f3a88e00640 1 -- 172.21.15.161:0/2413197893 shutdown_connections 2024-05-22T21:17:14.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.364+0000 7f3a88e00640 1 -- 172.21.15.161:0/2413197893 wait complete. 2024-05-22T21:17:14.386 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:17:14.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa71e000640 1 Processor -- start 2024-05-22T21:17:14.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa71e000640 1 -- start start 2024-05-22T21:17:14.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa71e000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa71815d430 0x7fa7180707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa71e000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7180ac8b0 0x7fa71806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa71e000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7180accf0 0x7fa71806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa71e000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa71806da70 con 0x7fa7180accf0 2024-05-22T21:17:14.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa71e000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa718071fd0 con 0x7fa71815d430 2024-05-22T21:17:14.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa71e000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa718070e80 con 0x7fa7180ac8b0 2024-05-22T21:17:14.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa717e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7180accf0 0x7fa71806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa717e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7180accf0 0x7fa71806d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33672/0 (socket says 172.21.15.161:33672) 2024-05-22T21:17:14.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa717e00640 1 -- 172.21.15.161:0/4143663237 learned_addr learned my addr 172.21.15.161:0/4143663237 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:14.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa716a00640 1 --2- 172.21.15.161:0/4143663237 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7180ac8b0 0x7fa71806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa717400640 1 --2- 172.21.15.161:0/4143663237 >> v2:172.21.15.179:3300/0 conn(0x7fa71815d430 0x7fa7180707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa717e00640 1 -- 172.21.15.161:0/4143663237 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7180ac8b0 msgr2=0x7fa71806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:14.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa717e00640 1 --2- 172.21.15.161:0/4143663237 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7180ac8b0 0x7fa71806cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa717e00640 1 -- 172.21.15.161:0/4143663237 >> v2:172.21.15.179:3300/0 conn(0x7fa71815d430 msgr2=0x7fa7180707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:14.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa717e00640 1 --2- 172.21.15.161:0/4143663237 >> v2:172.21.15.179:3300/0 conn(0x7fa71815d430 0x7fa7180707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.520+0000 7fa717e00640 1 -- 172.21.15.161:0/4143663237 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71806c510 con 0x7fa7180accf0 2024-05-22T21:17:14.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.524+0000 7fa717e00640 1 --2- 172.21.15.161:0/4143663237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7180accf0 0x7fa71806d530 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa7180aea00 tx=0x7fa71001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ccf1113e999e3a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:14.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.524+0000 7fa716000640 1 -- 172.21.15.161:0/4143663237 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7100ab620 con 0x7fa7180accf0 2024-05-22T21:17:14.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.524+0000 7fa716000640 1 -- 172.21.15.161:0/4143663237 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7100ab780 con 0x7fa7180accf0 2024-05-22T21:17:14.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.524+0000 7fa716000640 1 -- 172.21.15.161:0/4143663237 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7100abb10 con 0x7fa7180accf0 2024-05-22T21:17:14.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.524+0000 7fa71e000640 1 -- 172.21.15.161:0/4143663237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7180accf0 msgr2=0x7fa71806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.524+0000 7fa71e000640 1 --2- 172.21.15.161:0/4143663237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7180accf0 0x7fa71806d530 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa7180aea00 tx=0x7fa71001c460 comp rx=0 tx=0).stop 2024-05-22T21:17:14.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.524+0000 7fa71e000640 1 -- 172.21.15.161:0/4143663237 shutdown_connections 2024-05-22T21:17:14.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.524+0000 7fa71e000640 1 --2- 172.21.15.161:0/4143663237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7180accf0 0x7fa71806d530 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.524+0000 7fa71e000640 1 --2- 172.21.15.161:0/4143663237 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7180ac8b0 0x7fa71806cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.524+0000 7fa71e000640 1 --2- 172.21.15.161:0/4143663237 >> v2:172.21.15.179:3300/0 conn(0x7fa71815d430 0x7fa7180707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.524+0000 7fa71e000640 1 -- 172.21.15.161:0/4143663237 >> 172.21.15.161:0/4143663237 conn(0x7fa7180b0bc0 msgr2=0x7fa7180b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:14.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.524+0000 7fa71e000640 1 -- 172.21.15.161:0/4143663237 shutdown_connections 2024-05-22T21:17:14.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.524+0000 7fa71e000640 1 -- 172.21.15.161:0/4143663237 wait complete. 2024-05-22T21:17:14.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa71e000640 1 Processor -- start 2024-05-22T21:17:14.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa71e000640 1 -- start start 2024-05-22T21:17:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa71e000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7180ac8b0 0x7fa71815be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa71e000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7180accf0 0x7fa718145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa71e000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa71815d430 0x7fa7181446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa71e000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71815fb70 con 0x7fa71815d430 2024-05-22T21:17:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa71e000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71815fe50 con 0x7fa7180ac8b0 2024-05-22T21:17:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa71e000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7181452e0 con 0x7fa7180accf0 2024-05-22T21:17:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa717e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa71815d430 0x7fa7181446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa717e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa71815d430 0x7fa7181446b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33678/0 (socket says 172.21.15.161:33678) 2024-05-22T21:17:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa717e00640 1 -- 172.21.15.161:0/4234541565 learned_addr learned my addr 172.21.15.161:0/4234541565 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa717400640 1 --2- 172.21.15.161:0/4234541565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7180ac8b0 0x7fa71815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa716a00640 1 --2- 172.21.15.161:0/4234541565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7180accf0 0x7fa718145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa717e00640 1 -- 172.21.15.161:0/4234541565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7180accf0 msgr2=0x7fa718145dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa717e00640 1 --2- 172.21.15.161:0/4234541565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7180accf0 0x7fa718145dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa717e00640 1 -- 172.21.15.161:0/4234541565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7180ac8b0 msgr2=0x7fa71815be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa717e00640 1 --2- 172.21.15.161:0/4234541565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7180ac8b0 0x7fa71815be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa717e00640 1 -- 172.21.15.161:0/4234541565 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7100094d0 con 0x7fa71815d430 2024-05-22T21:17:14.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa717e00640 1 --2- 172.21.15.161:0/4234541565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa71815d430 0x7fa7181446b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa71001ab10 tx=0x7fa71001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:14.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa70f400640 1 -- 172.21.15.161:0/4234541565 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7100abe80 con 0x7fa71815d430 2024-05-22T21:17:14.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa70f400640 1 -- 172.21.15.161:0/4234541565 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7100be3f0 con 0x7fa71815d430 2024-05-22T21:17:14.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa70f400640 1 -- 172.21.15.161:0/4234541565 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7100be760 con 0x7fa71815d430 2024-05-22T21:17:14.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa71e000640 1 -- 172.21.15.161:0/4234541565 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7181455c0 con 0x7fa71815d430 2024-05-22T21:17:14.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.528+0000 7fa71e000640 1 -- 172.21.15.161:0/4234541565 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7181469c0 con 0x7fa71815d430 2024-05-22T21:17:14.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.532+0000 7fa71e000640 1 -- 172.21.15.161:0/4234541565 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7180ce7e0 con 0x7fa71815d430 2024-05-22T21:17:14.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.532+0000 7fa70f400640 1 -- 172.21.15.161:0/4234541565 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa7100b43f0 con 0x7fa71815d430 2024-05-22T21:17:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.532+0000 7fa70f400640 1 --2- 172.21.15.161:0/4234541565 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6f80666c0 0x7fa6f8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.532+0000 7fa70f400640 1 -- 172.21.15.161:0/4234541565 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa71011e580 con 0x7fa71815d430 2024-05-22T21:17:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.532+0000 7fa717400640 1 --2- 172.21.15.161:0/4234541565 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6f80666c0 0x7fa6f8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.532+0000 7fa717400640 1 --2- 172.21.15.161:0/4234541565 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6f80666c0 0x7fa6f8068b80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa708009fd0 tx=0x7fa708009800 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:14.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.536+0000 7fa70f400640 1 -- 172.21.15.161:0/4234541565 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa7100eb900 con 0x7fa71815d430 2024-05-22T21:17:14.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.692+0000 7fa71e000640 1 -- 172.21.15.161:0/4234541565 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa718067220 con 0x7fa71815d430 2024-05-22T21:17:14.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.696+0000 7fa70f400640 1 -- 172.21.15.161:0/4234541565 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16161 (secure 0 0 0) 0x7fa7100eef20 con 0x7fa71815d430 2024-05-22T21:17:14.698 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:14.698 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":83,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:17:14.011920+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:17:12.574579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-22T21:17:13.013467+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:17:14.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.696+0000 7fa71e000640 1 -- 172.21.15.161:0/4234541565 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6f80666c0 msgr2=0x7fa6f8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.696+0000 7fa71e000640 1 --2- 172.21.15.161:0/4234541565 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6f80666c0 0x7fa6f8068b80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa708009fd0 tx=0x7fa708009800 comp rx=0 tx=0).stop 2024-05-22T21:17:14.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.696+0000 7fa71e000640 1 -- 172.21.15.161:0/4234541565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa71815d430 msgr2=0x7fa7181446b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.696+0000 7fa71e000640 1 --2- 172.21.15.161:0/4234541565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa71815d430 0x7fa7181446b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa71001ab10 tx=0x7fa71001aca0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.696+0000 7fa71e000640 1 -- 172.21.15.161:0/4234541565 shutdown_connections 2024-05-22T21:17:14.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.696+0000 7fa71e000640 1 --2- 172.21.15.161:0/4234541565 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6f80666c0 0x7fa6f8068b80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.696+0000 7fa71e000640 1 --2- 172.21.15.161:0/4234541565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa71815d430 0x7fa7181446b0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.696+0000 7fa71e000640 1 --2- 172.21.15.161:0/4234541565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7180accf0 0x7fa718145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.696+0000 7fa71e000640 1 --2- 172.21.15.161:0/4234541565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7180ac8b0 0x7fa71815be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.696+0000 7fa71e000640 1 -- 172.21.15.161:0/4234541565 >> 172.21.15.161:0/4234541565 conn(0x7fa7180b0bc0 msgr2=0x7fa7180afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:14.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.696+0000 7fa71e000640 1 -- 172.21.15.161:0/4234541565 shutdown_connections 2024-05-22T21:17:14.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.696+0000 7fa71e000640 1 -- 172.21.15.161:0/4234541565 wait complete. 2024-05-22T21:17:14.718 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-22T21:17:14.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.852+0000 7f2e1c600640 1 Processor -- start 2024-05-22T21:17:14.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.856+0000 7f2e1c600640 1 -- start start 2024-05-22T21:17:14.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.856+0000 7f2e1c600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2e14143c40 0x7f2e1415b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.856+0000 7f2e1c600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2e1406c9b0 0x7f2e140b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.856+0000 7f2e1c600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1415efb0 con 0x7f2e141444a0 2024-05-22T21:17:14.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.856+0000 7f2e1c600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e1415f290 con 0x7f2e14143c40 2024-05-22T21:17:14.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.856+0000 7f2e1c600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e1415f570 con 0x7f2e1406c9b0 2024-05-22T21:17:14.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.856+0000 7f2e19200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f2e141444a0 0x7f2e1406c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54508/0 (socket says 172.21.15.161:54508) 2024-05-22T21:17:14.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.856+0000 7f2e19c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2e14143c40 0x7f2e1415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.856+0000 7f2e19200640 1 -- 172.21.15.161:0/3706082654 learned_addr learned my addr 172.21.15.161:0/3706082654 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:14.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.856+0000 7f2e1a600640 1 --2- 172.21.15.161:0/3706082654 >> v2:172.21.15.179:3300/0 conn(0x7f2e1406c9b0 0x7f2e140b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.856+0000 7f2e13e00640 1 -- 172.21.15.161:0/3706082654 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303792258 0 0) 0x7f2e1415efb0 con 0x7f2e141444a0 2024-05-22T21:17:14.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.856+0000 7f2e19c00640 1 -- 172.21.15.161:0/3706082654 >> v2:172.21.15.179:3300/0 conn(0x7f2e1406c9b0 msgr2=0x7f2e140b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:14.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.856+0000 7f2e19c00640 1 --2- 172.21.15.161:0/3706082654 >> v2:172.21.15.179:3300/0 conn(0x7f2e1406c9b0 0x7f2e140b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.856+0000 7f2e19c00640 1 -- 172.21.15.161:0/3706082654 >> v1:172.21.15.161:6789/0 conn(0x7f2e141444a0 legacy=0x7f2e1406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.856+0000 7f2e19c00640 1 -- 172.21.15.161:0/3706082654 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1415f850 con 0x7f2e14143c40 2024-05-22T21:17:14.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.860+0000 7f2e19c00640 1 --2- 172.21.15.161:0/3706082654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e14143c40 0x7f2e1415b600 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2e140aead0 tx=0x7f2e0801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a4da0eb53d29e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:14.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.860+0000 7f2e13e00640 1 -- 172.21.15.161:0/3706082654 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e080ac640 con 0x7f2e14143c40 2024-05-22T21:17:14.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.860+0000 7f2e13e00640 1 -- 172.21.15.161:0/3706082654 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e080ac7a0 con 0x7f2e14143c40 2024-05-22T21:17:14.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.860+0000 7f2e13e00640 1 -- 172.21.15.161:0/3706082654 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e080acb30 con 0x7f2e14143c40 2024-05-22T21:17:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.860+0000 7f2e1c600640 1 -- 172.21.15.161:0/3706082654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e14143c40 msgr2=0x7f2e1415b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.860+0000 7f2e1c600640 1 --2- 172.21.15.161:0/3706082654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e14143c40 0x7f2e1415b600 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2e140aead0 tx=0x7f2e0801c480 comp rx=0 tx=0).stop 2024-05-22T21:17:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.860+0000 7f2e1c600640 1 -- 172.21.15.161:0/3706082654 shutdown_connections 2024-05-22T21:17:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.860+0000 7f2e1c600640 1 --2- 172.21.15.161:0/3706082654 >> v2:172.21.15.179:3300/0 conn(0x7f2e1406c9b0 0x7f2e140b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.860+0000 7f2e1c600640 1 --2- 172.21.15.161:0/3706082654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e14143c40 0x7f2e1415b600 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.860+0000 7f2e1c600640 1 -- 172.21.15.161:0/3706082654 >> 172.21.15.161:0/3706082654 conn(0x7f2e1406a2d0 msgr2=0x7f2e1406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:14.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.860+0000 7f2e1c600640 1 -- 172.21.15.161:0/3706082654 shutdown_connections 2024-05-22T21:17:14.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.860+0000 7f2e1c600640 1 -- 172.21.15.161:0/3706082654 wait complete. 2024-05-22T21:17:14.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.860+0000 7f2e1c600640 1 Processor -- start 2024-05-22T21:17:14.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.860+0000 7f2e1c600640 1 -- start start 2024-05-22T21:17:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1c600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e1406c9b0 0x7f2e14139be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1c600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e14143c40 0x7f2e140ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1c600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e141444a0 0x7f2e14070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1c600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1413a120 con 0x7f2e141444a0 2024-05-22T21:17:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1c600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e14151ef0 con 0x7f2e14143c40 2024-05-22T21:17:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1c600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e141521d0 con 0x7f2e1406c9b0 2024-05-22T21:17:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1a600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e141444a0 0x7f2e14070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1a600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e141444a0 0x7f2e14070aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33698/0 (socket says 172.21.15.161:33698) 2024-05-22T21:17:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1a600640 1 -- 172.21.15.161:0/2861093730 learned_addr learned my addr 172.21.15.161:0/2861093730 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e19c00640 1 --2- 172.21.15.161:0/2861093730 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e1406c9b0 0x7f2e14139be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e19200640 1 --2- 172.21.15.161:0/2861093730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e14143c40 0x7f2e140ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1a600640 1 -- 172.21.15.161:0/2861093730 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e1406c9b0 msgr2=0x7f2e14139be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1a600640 1 --2- 172.21.15.161:0/2861093730 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e1406c9b0 0x7f2e14139be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1a600640 1 -- 172.21.15.161:0/2861093730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e14143c40 msgr2=0x7f2e140ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1a600640 1 --2- 172.21.15.161:0/2861093730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e14143c40 0x7f2e140ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1a600640 1 -- 172.21.15.161:0/2861093730 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e08009420 con 0x7f2e141444a0 2024-05-22T21:17:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1a600640 1 --2- 172.21.15.161:0/2861093730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e141444a0 0x7f2e14070aa0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009dd0 tx=0x7f2e0c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e11600640 1 -- 172.21.15.161:0/2861093730 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c014480 con 0x7f2e141444a0 2024-05-22T21:17:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e11600640 1 -- 172.21.15.161:0/2861093730 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c00f040 con 0x7f2e141444a0 2024-05-22T21:17:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1c600640 1 -- 172.21.15.161:0/2861093730 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e140ad7d0 con 0x7f2e141444a0 2024-05-22T21:17:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e11600640 1 -- 172.21.15.161:0/2861093730 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c014840 con 0x7f2e141444a0 2024-05-22T21:17:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1c600640 1 -- 172.21.15.161:0/2861093730 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e140add40 con 0x7f2e141444a0 2024-05-22T21:17:14.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e1c600640 1 -- 172.21.15.161:0/2861093730 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e14152340 con 0x7f2e141444a0 2024-05-22T21:17:14.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.864+0000 7f2e11600640 1 -- 172.21.15.161:0/2861093730 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2e0c0149a0 con 0x7f2e141444a0 2024-05-22T21:17:14.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.868+0000 7f2e11600640 1 --2- 172.21.15.161:0/2861093730 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2df40666c0 0x7f2df4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:14.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.868+0000 7f2e11600640 1 -- 172.21.15.161:0/2861093730 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e0c084d70 con 0x7f2e141444a0 2024-05-22T21:17:14.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.868+0000 7f2e19c00640 1 --2- 172.21.15.161:0/2861093730 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2df40666c0 0x7f2df4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:14.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.868+0000 7f2e19c00640 1 --2- 172.21.15.161:0/2861093730 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2df40666c0 0x7f2df4068b80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e080ae000 tx=0x7f2e080a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:14.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:14.872+0000 7f2e11600640 1 -- 172.21.15.161:0/2861093730 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2e0c052040 con 0x7f2e141444a0 2024-05-22T21:17:15.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:15.060+0000 7f2e1c600640 1 -- 172.21.15.161:0/2861093730 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2e140ceaa0 con 0x7f2e141444a0 2024-05-22T21:17:15.568 INFO:tasks.ceph.mds.b.smithi179.stderr:2024-05-22T21:17:15.565+0000 7f3534a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T21:17:16.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.028+0000 7f2e11600640 1 -- 172.21.15.161:0/2861093730 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v38) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2e0c055660 con 0x7f2e141444a0 2024-05-22T21:17:16.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.028+0000 7f2e1c600640 1 -- 172.21.15.161:0/2861093730 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2df40666c0 msgr2=0x7f2df4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:16.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.028+0000 7f2e1c600640 1 --2- 172.21.15.161:0/2861093730 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2df40666c0 0x7f2df4068b80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e080ae000 tx=0x7f2e080a7000 comp rx=0 tx=0).stop 2024-05-22T21:17:16.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.028+0000 7f2e1c600640 1 -- 172.21.15.161:0/2861093730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e141444a0 msgr2=0x7f2e14070aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:16.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.028+0000 7f2e1c600640 1 --2- 172.21.15.161:0/2861093730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e141444a0 0x7f2e14070aa0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009dd0 tx=0x7f2e0c00c490 comp rx=0 tx=0).stop 2024-05-22T21:17:16.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.028+0000 7f2e1c600640 1 -- 172.21.15.161:0/2861093730 shutdown_connections 2024-05-22T21:17:16.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.028+0000 7f2e1c600640 1 --2- 172.21.15.161:0/2861093730 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2df40666c0 0x7f2df4068b80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:16.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.028+0000 7f2e1c600640 1 --2- 172.21.15.161:0/2861093730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e141444a0 0x7f2e14070aa0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:16.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.028+0000 7f2e1c600640 1 --2- 172.21.15.161:0/2861093730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e14143c40 0x7f2e140ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:16.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.028+0000 7f2e1c600640 1 --2- 172.21.15.161:0/2861093730 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e1406c9b0 0x7f2e14139be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:16.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.028+0000 7f2e1c600640 1 -- 172.21.15.161:0/2861093730 >> 172.21.15.161:0/2861093730 conn(0x7f2e1406a2d0 msgr2=0x7f2e1415c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:16.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.028+0000 7f2e1c600640 1 -- 172.21.15.161:0/2861093730 shutdown_connections 2024-05-22T21:17:16.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.028+0000 7f2e1c600640 1 -- 172.21.15.161:0/2861093730 wait complete. 2024-05-22T21:17:16.054 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-22T21:17:16.054 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-22T21:17:16.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d91e00640 1 Processor -- start 2024-05-22T21:17:16.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d91e00640 1 -- start start 2024-05-22T21:17:16.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d91e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2d8c13fe50 0x7f2d8c164e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:16.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d91e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8c13d3a0 con 0x7f2d8c13cfc0 2024-05-22T21:17:16.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d91e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8c0cd920 con 0x7f2d8c1402e0 2024-05-22T21:17:16.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d91e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8c0cdc00 con 0x7f2d8c13fe50 2024-05-22T21:17:16.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d8aa00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f2d8c13cfc0 0x7f2d8c15af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39006/0 (socket says 172.21.15.161:39006) 2024-05-22T21:17:16.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d8aa00640 1 -- 172.21.15.161:0/3599454778 learned_addr learned my addr 172.21.15.161:0/3599454778 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:16.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d8b400640 1 --2- 172.21.15.161:0/3599454778 >> v2:172.21.15.179:3300/0 conn(0x7f2d8c13fe50 0x7f2d8c164e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:16.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d89600640 1 -- 172.21.15.161:0/3599454778 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561242562 0 0) 0x7f2d8c13d3a0 con 0x7f2d8c13cfc0 2024-05-22T21:17:16.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d89600640 1 -- 172.21.15.161:0/3599454778 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d70003530 con 0x7f2d8c13cfc0 2024-05-22T21:17:16.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d89600640 1 -- 172.21.15.161:0/3599454778 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 21692152 0 0) 0x7f2d70003530 con 0x7f2d8c13cfc0 2024-05-22T21:17:16.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d89600640 1 -- 172.21.15.161:0/3599454778 >> v2:172.21.15.179:3300/0 conn(0x7f2d8c13fe50 msgr2=0x7f2d8c164e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:16.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d89600640 1 --2- 172.21.15.161:0/3599454778 >> v2:172.21.15.179:3300/0 conn(0x7f2d8c13fe50 0x7f2d8c164e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:16.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d89600640 1 -- 172.21.15.161:0/3599454778 >> v1:172.21.15.179:6789/0 conn(0x7f2d8c1402e0 legacy=0x7f2d8c13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:16.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d89600640 1 -- 172.21.15.161:0/3599454778 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c0cdee0 con 0x7f2d8c13cfc0 2024-05-22T21:17:16.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d89600640 1 -- 172.21.15.161:0/3599454778 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f2d7c001680 con 0x7f2d8c13cfc0 2024-05-22T21:17:16.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d89600640 1 -- 172.21.15.161:0/3599454778 >> v1:172.21.15.161:6789/0 conn(0x7f2d8c13cfc0 legacy=0x7f2d8c15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:16.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d89600640 1 --2- 172.21.15.161:0/3599454778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d70003980 0x7f2d70003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:16.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d89600640 1 --2- 172.21.15.161:0/3599454778 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d700047d0 0x7f2d70004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:16.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d89600640 1 --2- 172.21.15.161:0/3599454778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d70005190 0x7f2d7000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:16.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d89600640 1 -- 172.21.15.161:0/3599454778 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0cdc00 con 0x7f2d70005190 2024-05-22T21:17:16.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d89600640 1 -- 172.21.15.161:0/3599454778 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7000c130 con 0x7f2d70003980 2024-05-22T21:17:16.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d89600640 1 -- 172.21.15.161:0/3599454778 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7000c2f0 con 0x7f2d700047d0 2024-05-22T21:17:16.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.192+0000 7f2d8b400640 1 --2- 172.21.15.161:0/3599454778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d70005190 0x7f2d7000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:16.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d8aa00640 1 --2- 172.21.15.161:0/3599454778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d70003980 0x7f2d70003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:16.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d8a000640 1 --2- 172.21.15.161:0/3599454778 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d700047d0 0x7f2d70004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:16.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d8b400640 1 -- 172.21.15.161:0/3599454778 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d700047d0 msgr2=0x7f2d70004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:16.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d8b400640 1 --2- 172.21.15.161:0/3599454778 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d700047d0 0x7f2d70004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:16.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d8b400640 1 -- 172.21.15.161:0/3599454778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d70003980 msgr2=0x7f2d70003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:16.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d8b400640 1 --2- 172.21.15.161:0/3599454778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d70003980 0x7f2d70003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:16.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d8b400640 1 -- 172.21.15.161:0/3599454778 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d70003530 con 0x7f2d70005190 2024-05-22T21:17:16.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d8b400640 1 -- 172.21.15.161:0/3599454778 reap_dead start 2024-05-22T21:17:16.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d8b400640 1 --2- 172.21.15.161:0/3599454778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d70005190 0x7f2d7000b990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009fc0 tx=0x7f2d8401eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfde9688dd62709a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:16.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d89600640 1 -- 172.21.15.161:0/3599454778 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d840a7dd0 con 0x7f2d70005190 2024-05-22T21:17:16.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d89600640 1 -- 172.21.15.161:0/3599454778 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8400f040 con 0x7f2d70005190 2024-05-22T21:17:16.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d89600640 1 -- 172.21.15.161:0/3599454778 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84003be0 con 0x7f2d70005190 2024-05-22T21:17:16.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d91e00640 1 -- 172.21.15.161:0/3599454778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d70005190 msgr2=0x7f2d7000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:16.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d91e00640 1 --2- 172.21.15.161:0/3599454778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d70005190 0x7f2d7000b990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009fc0 tx=0x7f2d8401eed0 comp rx=0 tx=0).stop 2024-05-22T21:17:16.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d91e00640 1 -- 172.21.15.161:0/3599454778 shutdown_connections 2024-05-22T21:17:16.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d91e00640 1 --2- 172.21.15.161:0/3599454778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d70005190 0x7f2d7000b990 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:16.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d91e00640 1 -- 172.21.15.161:0/3599454778 >> 172.21.15.161:0/3599454778 conn(0x7f2d8c06a2d0 msgr2=0x7f2d8c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:16.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d91e00640 1 -- 172.21.15.161:0/3599454778 shutdown_connections 2024-05-22T21:17:16.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.196+0000 7f2d91e00640 1 -- 172.21.15.161:0/3599454778 wait complete. 2024-05-22T21:17:16.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d91e00640 1 Processor -- start 2024-05-22T21:17:16.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d91e00640 1 -- start start 2024-05-22T21:17:16.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d91e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d70005190 0x7f2d8c157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:16.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d91e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d8c139f30 0x7f2d8c13a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:16.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d91e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d8c13f0c0 0x7f2d8c13f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:16.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d91e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c13a8f0 con 0x7f2d8c139f30 2024-05-22T21:17:16.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d91e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c13d590 con 0x7f2d8c13f0c0 2024-05-22T21:17:16.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d91e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c13d850 con 0x7f2d70005190 2024-05-22T21:17:16.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d8a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d8c139f30 0x7f2d8c13a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:16.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d8a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d8c139f30 0x7f2d8c13a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34176/0 (socket says 172.21.15.161:34176) 2024-05-22T21:17:16.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d8a000640 1 -- 172.21.15.161:0/359610407 learned_addr learned my addr 172.21.15.161:0/359610407 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:16.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d8b400640 1 --2- 172.21.15.161:0/359610407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d8c13f0c0 0x7f2d8c13f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:16.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d8aa00640 1 --2- 172.21.15.161:0/359610407 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d70005190 0x7f2d8c157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:16.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d8a000640 1 -- 172.21.15.161:0/359610407 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d70005190 msgr2=0x7f2d8c157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:16.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d8a000640 1 --2- 172.21.15.161:0/359610407 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d70005190 0x7f2d8c157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:16.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d8a000640 1 -- 172.21.15.161:0/359610407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d8c13f0c0 msgr2=0x7f2d8c13f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:16.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d8a000640 1 --2- 172.21.15.161:0/359610407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d8c13f0c0 0x7f2d8c13f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:16.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d8a000640 1 -- 172.21.15.161:0/359610407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c13db30 con 0x7f2d8c139f30 2024-05-22T21:17:16.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d8aa00640 1 --2- 172.21.15.161:0/359610407 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d70005190 0x7f2d8c157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:16.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d8a000640 1 --2- 172.21.15.161:0/359610407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d8c139f30 0x7f2d8c13a3b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800e4b0 tx=0x7f2d7800e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:16.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d83400640 1 -- 172.21.15.161:0/359610407 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d780052e0 con 0x7f2d8c139f30 2024-05-22T21:17:16.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d83400640 1 -- 172.21.15.161:0/359610407 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d78005440 con 0x7f2d8c139f30 2024-05-22T21:17:16.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d83400640 1 -- 172.21.15.161:0/359610407 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78003840 con 0x7f2d8c139f30 2024-05-22T21:17:16.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d91e00640 1 -- 172.21.15.161:0/359610407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8c13de10 con 0x7f2d8c139f30 2024-05-22T21:17:16.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.200+0000 7f2d91e00640 1 -- 172.21.15.161:0/359610407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c157570 con 0x7f2d8c139f30 2024-05-22T21:17:16.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.204+0000 7f2d91e00640 1 -- 172.21.15.161:0/359610407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8c067220 con 0x7f2d8c139f30 2024-05-22T21:17:16.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.204+0000 7f2d83400640 1 -- 172.21.15.161:0/359610407 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2d780039a0 con 0x7f2d8c139f30 2024-05-22T21:17:16.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.204+0000 7f2d83400640 1 --2- 172.21.15.161:0/359610407 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d6c066690 0x7f2d6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:16.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.204+0000 7f2d83400640 1 -- 172.21.15.161:0/359610407 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d78083f00 con 0x7f2d8c139f30 2024-05-22T21:17:16.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.204+0000 7f2d8aa00640 1 --2- 172.21.15.161:0/359610407 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d6c066690 0x7f2d6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:16.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.204+0000 7f2d8aa00640 1 --2- 172.21.15.161:0/359610407 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d6c066690 0x7f2d6c068b50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c001ec0 tx=0x7f2d7c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:16.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.208+0000 7f2d83400640 1 -- 172.21.15.161:0/359610407 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2d780511d0 con 0x7f2d8c139f30 2024-05-22T21:17:16.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:16.396+0000 7f2d91e00640 1 -- 172.21.15.161:0/359610407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2d8c071670 con 0x7f2d8c139f30 2024-05-22T21:17:17.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.036+0000 7f2d83400640 1 -- 172.21.15.161:0/359610407 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v39) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2d780547f0 con 0x7f2d8c139f30 2024-05-22T21:17:17.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.040+0000 7f2d91e00640 1 -- 172.21.15.161:0/359610407 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d6c066690 msgr2=0x7f2d6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.040+0000 7f2d91e00640 1 --2- 172.21.15.161:0/359610407 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d6c066690 0x7f2d6c068b50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c001ec0 tx=0x7f2d7c009040 comp rx=0 tx=0).stop 2024-05-22T21:17:17.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.040+0000 7f2d91e00640 1 -- 172.21.15.161:0/359610407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d8c139f30 msgr2=0x7f2d8c13a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.040+0000 7f2d91e00640 1 --2- 172.21.15.161:0/359610407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d8c139f30 0x7f2d8c13a3b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800e4b0 tx=0x7f2d7800e970 comp rx=0 tx=0).stop 2024-05-22T21:17:17.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.040+0000 7f2d91e00640 1 -- 172.21.15.161:0/359610407 shutdown_connections 2024-05-22T21:17:17.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.040+0000 7f2d91e00640 1 --2- 172.21.15.161:0/359610407 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d6c066690 0x7f2d6c068b50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.040+0000 7f2d91e00640 1 --2- 172.21.15.161:0/359610407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d8c13f0c0 0x7f2d8c13f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.040+0000 7f2d91e00640 1 --2- 172.21.15.161:0/359610407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d8c139f30 0x7f2d8c13a3b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.040+0000 7f2d91e00640 1 --2- 172.21.15.161:0/359610407 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d70005190 0x7f2d8c157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.040+0000 7f2d91e00640 1 -- 172.21.15.161:0/359610407 >> 172.21.15.161:0/359610407 conn(0x7f2d8c06a2d0 msgr2=0x7f2d8c13b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:17.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.040+0000 7f2d91e00640 1 -- 172.21.15.161:0/359610407 shutdown_connections 2024-05-22T21:17:17.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.040+0000 7f2d91e00640 1 -- 172.21.15.161:0/359610407 wait complete. 2024-05-22T21:17:17.064 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-22T21:17:17.064 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:17:17.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e20800640 1 Processor -- start 2024-05-22T21:17:17.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e20800640 1 -- start start 2024-05-22T21:17:17.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e20800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e181597f0 0x7f1e180706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e20800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1e18070be0 0x7f1e180b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e20800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1e180b2ea0 0x7f1e1806c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e20800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e18071040 con 0x7f1e18070be0 2024-05-22T21:17:17.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e20800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e18159bd0 con 0x7f1e180b2ea0 2024-05-22T21:17:17.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e20800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180cbdd0 con 0x7f1e181597f0 2024-05-22T21:17:17.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e1d400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1e18070be0 0x7f1e180b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e1d400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1e18070be0 0x7f1e180b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34180/0 (socket says 172.21.15.161:34180) 2024-05-22T21:17:17.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e1d400640 1 -- 172.21.15.161:0/3331061425 learned_addr learned my addr 172.21.15.161:0/3331061425 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:17.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e1e800640 1 --2- 172.21.15.161:0/3331061425 >> v2:172.21.15.179:3300/0 conn(0x7f1e180b2ea0 0x7f1e1806c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e1de00640 1 --2- 172.21.15.161:0/3331061425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e181597f0 0x7f1e180706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e1d400640 1 -- 172.21.15.161:0/3331061425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e181597f0 msgr2=0x7f1e180706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:17.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e1d400640 1 --2- 172.21.15.161:0/3331061425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e181597f0 0x7f1e180706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e1d400640 1 -- 172.21.15.161:0/3331061425 >> v2:172.21.15.179:3300/0 conn(0x7f1e180b2ea0 msgr2=0x7f1e1806c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:17.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e1d400640 1 --2- 172.21.15.161:0/3331061425 >> v2:172.21.15.179:3300/0 conn(0x7f1e180b2ea0 0x7f1e1806c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e1d400640 1 -- 172.21.15.161:0/3331061425 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e180cc050 con 0x7f1e18070be0 2024-05-22T21:17:17.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.200+0000 7f1e1de00640 1 --2- 172.21.15.161:0/3331061425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e181597f0 0x7f1e180706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:17.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.204+0000 7f1e1d400640 1 --2- 172.21.15.161:0/3331061425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e18070be0 0x7f1e180b2960 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1e180aeae0 tx=0x7f1e0801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa4eae6b7e2965b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:17.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.204+0000 7f1e1ca00640 1 -- 172.21.15.161:0/3331061425 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e080ab660 con 0x7f1e18070be0 2024-05-22T21:17:17.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.204+0000 7f1e1ca00640 1 -- 172.21.15.161:0/3331061425 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e080ab7c0 con 0x7f1e18070be0 2024-05-22T21:17:17.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.204+0000 7f1e1ca00640 1 -- 172.21.15.161:0/3331061425 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e080abb50 con 0x7f1e18070be0 2024-05-22T21:17:17.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.204+0000 7f1e20800640 1 -- 172.21.15.161:0/3331061425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e18070be0 msgr2=0x7f1e180b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.204+0000 7f1e20800640 1 --2- 172.21.15.161:0/3331061425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e18070be0 0x7f1e180b2960 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1e180aeae0 tx=0x7f1e0801c520 comp rx=0 tx=0).stop 2024-05-22T21:17:17.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.204+0000 7f1e20800640 1 -- 172.21.15.161:0/3331061425 shutdown_connections 2024-05-22T21:17:17.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.204+0000 7f1e20800640 1 --2- 172.21.15.161:0/3331061425 >> v2:172.21.15.179:3300/0 conn(0x7f1e180b2ea0 0x7f1e1806c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.204+0000 7f1e20800640 1 --2- 172.21.15.161:0/3331061425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e18070be0 0x7f1e180b2960 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.204+0000 7f1e20800640 1 --2- 172.21.15.161:0/3331061425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e181597f0 0x7f1e180706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.204+0000 7f1e20800640 1 -- 172.21.15.161:0/3331061425 >> 172.21.15.161:0/3331061425 conn(0x7f1e180b0bc0 msgr2=0x7f1e180b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.204+0000 7f1e20800640 1 -- 172.21.15.161:0/3331061425 shutdown_connections 2024-05-22T21:17:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.204+0000 7f1e20800640 1 -- 172.21.15.161:0/3331061425 wait complete. 2024-05-22T21:17:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e20800640 1 Processor -- start 2024-05-22T21:17:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e20800640 1 -- start start 2024-05-22T21:17:17.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e20800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e18070be0 0x7f1e181f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e20800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e180b2ea0 0x7f1e18151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e20800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e181597f0 0x7f1e181525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e20800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e181f6a70 con 0x7f1e181597f0 2024-05-22T21:17:17.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e20800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1815b7a0 con 0x7f1e180b2ea0 2024-05-22T21:17:17.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e20800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e18154100 con 0x7f1e18070be0 2024-05-22T21:17:17.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e1e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e181597f0 0x7f1e181525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e1e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e181597f0 0x7f1e181525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34194/0 (socket says 172.21.15.161:34194) 2024-05-22T21:17:17.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e1e800640 1 -- 172.21.15.161:0/46719683 learned_addr learned my addr 172.21.15.161:0/46719683 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:17.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e1de00640 1 --2- 172.21.15.161:0/46719683 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e18070be0 0x7f1e181f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e1d400640 1 --2- 172.21.15.161:0/46719683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e180b2ea0 0x7f1e18151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e1e800640 1 -- 172.21.15.161:0/46719683 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e18070be0 msgr2=0x7f1e181f6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e1e800640 1 --2- 172.21.15.161:0/46719683 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e18070be0 0x7f1e181f6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e1e800640 1 -- 172.21.15.161:0/46719683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e180b2ea0 msgr2=0x7f1e18151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e1e800640 1 --2- 172.21.15.161:0/46719683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e180b2ea0 0x7f1e18151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e1e800640 1 -- 172.21.15.161:0/46719683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e080094d0 con 0x7f1e181597f0 2024-05-22T21:17:17.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e1d400640 1 --2- 172.21.15.161:0/46719683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e180b2ea0 0x7f1e18151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:17.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e1e800640 1 --2- 172.21.15.161:0/46719683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e181597f0 0x7f1e181525d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400ef00 tx=0x7f1e1400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:17.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e12000640 1 -- 172.21.15.161:0/46719683 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14010040 con 0x7f1e181597f0 2024-05-22T21:17:17.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e12000640 1 -- 172.21.15.161:0/46719683 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1400bd20 con 0x7f1e181597f0 2024-05-22T21:17:17.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e12000640 1 -- 172.21.15.161:0/46719683 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14014640 con 0x7f1e181597f0 2024-05-22T21:17:17.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e20800640 1 -- 172.21.15.161:0/46719683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e181f95b0 con 0x7f1e181597f0 2024-05-22T21:17:17.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e20800640 1 -- 172.21.15.161:0/46719683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e181f9b70 con 0x7f1e181597f0 2024-05-22T21:17:17.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.208+0000 7f1e20800640 1 -- 172.21.15.161:0/46719683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1815b950 con 0x7f1e181597f0 2024-05-22T21:17:17.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.212+0000 7f1e12000640 1 -- 172.21.15.161:0/46719683 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1e140147a0 con 0x7f1e181597f0 2024-05-22T21:17:17.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.212+0000 7f1e12000640 1 --2- 172.21.15.161:0/46719683 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1dfc0666c0 0x7f1dfc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.212+0000 7f1e12000640 1 -- 172.21.15.161:0/46719683 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e14084830 con 0x7f1e181597f0 2024-05-22T21:17:17.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.212+0000 7f1e1de00640 1 --2- 172.21.15.161:0/46719683 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1dfc0666c0 0x7f1dfc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.212+0000 7f1e1de00640 1 --2- 172.21.15.161:0/46719683 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1dfc0666c0 0x7f1dfc068b80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0098b0 tx=0x7f1e0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:17.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.216+0000 7f1e12000640 1 -- 172.21.15.161:0/46719683 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1e1815b950 con 0x7f1e181597f0 2024-05-22T21:17:17.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.404+0000 7f1e20800640 1 -- 172.21.15.161:0/46719683 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e1815b950 con 0x7f1e181597f0 2024-05-22T21:17:17.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.404+0000 7f1e12000640 1 -- 172.21.15.161:0/46719683 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4745 (secure 0 0 0) 0x7f1e1815b950 con 0x7f1e181597f0 2024-05-22T21:17:17.407 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:17.407 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5567,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196093187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196093187},{"type":"v1","addr":"172.21.15.161:6836","nonce":196093187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6136,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4029369301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4029369301},{"type":"v1","addr":"172.21.15.161:6837","nonce":4029369301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6139,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3212560487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3212560487},{"type":"v1","addr":"172.21.15.179:6834","nonce":3212560487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:17:14.012254+0000","modified":"2024-05-22T21:17:17.029283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6306":{"gid":6306,"name":"b","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/3065973138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3065973138},{"type":"v1","addr":"172.21.15.179:6835","nonce":3065973138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-22T21:17:17.407 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 39 2024-05-22T21:17:17.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.404+0000 7f1e20800640 1 -- 172.21.15.161:0/46719683 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1dfc0666c0 msgr2=0x7f1dfc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.404+0000 7f1e20800640 1 --2- 172.21.15.161:0/46719683 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1dfc0666c0 0x7f1dfc068b80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0098b0 tx=0x7f1e0c008040 comp rx=0 tx=0).stop 2024-05-22T21:17:17.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.404+0000 7f1e20800640 1 -- 172.21.15.161:0/46719683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e181597f0 msgr2=0x7f1e181525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.404+0000 7f1e20800640 1 --2- 172.21.15.161:0/46719683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e181597f0 0x7f1e181525d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400ef00 tx=0x7f1e1400b480 comp rx=0 tx=0).stop 2024-05-22T21:17:17.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.404+0000 7f1e20800640 1 -- 172.21.15.161:0/46719683 shutdown_connections 2024-05-22T21:17:17.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.404+0000 7f1e20800640 1 --2- 172.21.15.161:0/46719683 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1dfc0666c0 0x7f1dfc068b80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.404+0000 7f1e20800640 1 --2- 172.21.15.161:0/46719683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e181597f0 0x7f1e181525d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.404+0000 7f1e20800640 1 --2- 172.21.15.161:0/46719683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e180b2ea0 0x7f1e18151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.404+0000 7f1e20800640 1 --2- 172.21.15.161:0/46719683 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e18070be0 0x7f1e181f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.404+0000 7f1e20800640 1 -- 172.21.15.161:0/46719683 >> 172.21.15.161:0/46719683 conn(0x7f1e180b0bc0 msgr2=0x7f1e180cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:17.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.404+0000 7f1e20800640 1 -- 172.21.15.161:0/46719683 shutdown_connections 2024-05-22T21:17:17.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.404+0000 7f1e20800640 1 -- 172.21.15.161:0/46719683 wait complete. 2024-05-22T21:17:17.427 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:17:17.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb9fe00640 1 Processor -- start 2024-05-22T21:17:17.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb9fe00640 1 -- start start 2024-05-22T21:17:17.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb9fe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb980b3560 0x7ffb9815e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb9fe00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9806df70 con 0x7ffb9809a080 2024-05-22T21:17:17.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb9fe00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb98161c60 con 0x7ffb98099ca0 2024-05-22T21:17:17.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb9fe00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb980c7ea0 con 0x7ffb980b3560 2024-05-22T21:17:17.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb9de00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ffb9809a080 0x7ffb980cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39008/0 (socket says 172.21.15.161:39008) 2024-05-22T21:17:17.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb9de00640 1 -- 172.21.15.161:0/2797499499 learned_addr learned my addr 172.21.15.161:0/2797499499 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:17.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb9ca00640 1 --2- 172.21.15.161:0/2797499499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb980b3560 0x7ffb9815e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb97e00640 1 -- 172.21.15.161:0/2797499499 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570095735 0 0) 0x7ffb9806df70 con 0x7ffb9809a080 2024-05-22T21:17:17.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb97e00640 1 -- 172.21.15.161:0/2797499499 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb80003530 con 0x7ffb9809a080 2024-05-22T21:17:17.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb97e00640 1 -- 172.21.15.161:0/2797499499 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1180433450 0 0) 0x7ffb98161c60 con 0x7ffb98099ca0 2024-05-22T21:17:17.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb97e00640 1 -- 172.21.15.161:0/2797499499 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb9806df70 con 0x7ffb98099ca0 2024-05-22T21:17:17.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb9ca00640 1 -- 172.21.15.161:0/2797499499 >> v1:172.21.15.179:6789/0 conn(0x7ffb98099ca0 legacy=0x7ffb9806eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb9ca00640 1 -- 172.21.15.161:0/2797499499 >> v1:172.21.15.161:6789/0 conn(0x7ffb9809a080 legacy=0x7ffb980cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb97e00640 1 -- 172.21.15.161:0/2797499499 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3314063881 0 0) 0x7ffb80003530 con 0x7ffb9809a080 2024-05-22T21:17:17.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.560+0000 7ffb9ca00640 1 -- 172.21.15.161:0/2797499499 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9806e1c0 con 0x7ffb980b3560 2024-05-22T21:17:17.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.564+0000 7ffb9ca00640 1 --2- 172.21.15.161:0/2797499499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb980b3560 0x7ffb9815e350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffb840097f0 tx=0x7ffb8401c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69dd767d6da82d9d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:17.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.564+0000 7ffb97e00640 1 -- 172.21.15.161:0/2797499499 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90001680 con 0x7ffb980b3560 2024-05-22T21:17:17.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.564+0000 7ffb97e00640 1 -- 172.21.15.161:0/2797499499 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb840a7050 con 0x7ffb980b3560 2024-05-22T21:17:17.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.564+0000 7ffb97e00640 1 -- 172.21.15.161:0/2797499499 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb84004250 con 0x7ffb980b3560 2024-05-22T21:17:17.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.564+0000 7ffb9fe00640 1 -- 172.21.15.161:0/2797499499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb980b3560 msgr2=0x7ffb9815e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.564+0000 7ffb9fe00640 1 --2- 172.21.15.161:0/2797499499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb980b3560 0x7ffb9815e350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffb840097f0 tx=0x7ffb8401c3d0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.564+0000 7ffb9fe00640 1 -- 172.21.15.161:0/2797499499 shutdown_connections 2024-05-22T21:17:17.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.564+0000 7ffb9fe00640 1 --2- 172.21.15.161:0/2797499499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb980b3560 0x7ffb9815e350 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.564+0000 7ffb9fe00640 1 -- 172.21.15.161:0/2797499499 >> 172.21.15.161:0/2797499499 conn(0x7ffb980b2a40 msgr2=0x7ffb980b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:17.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.564+0000 7ffb9fe00640 1 -- 172.21.15.161:0/2797499499 shutdown_connections 2024-05-22T21:17:17.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.564+0000 7ffb9fe00640 1 -- 172.21.15.161:0/2797499499 wait complete. 2024-05-22T21:17:17.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9fe00640 1 Processor -- start 2024-05-22T21:17:17.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9fe00640 1 -- start start 2024-05-22T21:17:17.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9fe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb98099ca0 0x7ffb98151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9809a080 0x7ffb9806c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9fe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb980b3560 0x7ffb9806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9fe00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9815c320 con 0x7ffb9809a080 2024-05-22T21:17:17.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9fe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb980c4f20 con 0x7ffb98099ca0 2024-05-22T21:17:17.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9fe00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9815bd60 con 0x7ffb980b3560 2024-05-22T21:17:17.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9ca00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9809a080 0x7ffb9806c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9ca00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9809a080 0x7ffb9806c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34198/0 (socket says 172.21.15.161:34198) 2024-05-22T21:17:17.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9ca00640 1 -- 172.21.15.161:0/894394905 learned_addr learned my addr 172.21.15.161:0/894394905 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:17.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9d400640 1 --2- 172.21.15.161:0/894394905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb98099ca0 0x7ffb98151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9de00640 1 --2- 172.21.15.161:0/894394905 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb980b3560 0x7ffb9806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9ca00640 1 -- 172.21.15.161:0/894394905 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb980b3560 msgr2=0x7ffb9806c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9ca00640 1 --2- 172.21.15.161:0/894394905 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb980b3560 0x7ffb9806c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9ca00640 1 -- 172.21.15.161:0/894394905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb98099ca0 msgr2=0x7ffb98151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9ca00640 1 --2- 172.21.15.161:0/894394905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb98099ca0 0x7ffb98151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9ca00640 1 -- 172.21.15.161:0/894394905 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb84009420 con 0x7ffb9809a080 2024-05-22T21:17:17.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9ca00640 1 --2- 172.21.15.161:0/894394905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9809a080 0x7ffb9806c120 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ffb840097f0 tx=0x7ffb84019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:17.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb95600640 1 -- 172.21.15.161:0/894394905 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb84003af0 con 0x7ffb9809a080 2024-05-22T21:17:17.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb95600640 1 -- 172.21.15.161:0/894394905 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb84003c50 con 0x7ffb9809a080 2024-05-22T21:17:17.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb95600640 1 -- 172.21.15.161:0/894394905 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb840ab440 con 0x7ffb9809a080 2024-05-22T21:17:17.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9fe00640 1 -- 172.21.15.161:0/894394905 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9815c040 con 0x7ffb9809a080 2024-05-22T21:17:17.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.568+0000 7ffb9fe00640 1 -- 172.21.15.161:0/894394905 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb98151b40 con 0x7ffb9809a080 2024-05-22T21:17:17.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.572+0000 7ffb9fe00640 1 -- 172.21.15.161:0/894394905 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb98067220 con 0x7ffb9809a080 2024-05-22T21:17:17.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.572+0000 7ffb95600640 1 -- 172.21.15.161:0/894394905 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffb840b2040 con 0x7ffb9809a080 2024-05-22T21:17:17.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.572+0000 7ffb95600640 1 --2- 172.21.15.161:0/894394905 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb780666c0 0x7ffb78068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.572+0000 7ffb95600640 1 -- 172.21.15.161:0/894394905 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb8411df40 con 0x7ffb9809a080 2024-05-22T21:17:17.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.572+0000 7ffb9d400640 1 --2- 172.21.15.161:0/894394905 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb780666c0 0x7ffb78068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.572+0000 7ffb9d400640 1 --2- 172.21.15.161:0/894394905 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb780666c0 0x7ffb78068b80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c007fd0 tx=0x7ffb8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:17.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.576+0000 7ffb95600640 1 -- 172.21.15.161:0/894394905 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffb840eb2b0 con 0x7ffb9809a080 2024-05-22T21:17:17.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.728+0000 7ffb9fe00640 1 -- 172.21.15.161:0/894394905 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb980d0970 con 0x7ffb9809a080 2024-05-22T21:17:17.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.732+0000 7ffb95600640 1 -- 172.21.15.161:0/894394905 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16148 (secure 0 0 0) 0x7ffb840ee8d0 con 0x7ffb9809a080 2024-05-22T21:17:17.734 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:17.734 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":84,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:17:15.014662+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:17:12.574579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-22T21:17:13.013467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:17:17.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.732+0000 7ffb9fe00640 1 -- 172.21.15.161:0/894394905 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb780666c0 msgr2=0x7ffb78068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.732+0000 7ffb9fe00640 1 --2- 172.21.15.161:0/894394905 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb780666c0 0x7ffb78068b80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c007fd0 tx=0x7ffb8c009040 comp rx=0 tx=0).stop 2024-05-22T21:17:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.732+0000 7ffb9fe00640 1 -- 172.21.15.161:0/894394905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9809a080 msgr2=0x7ffb9806c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.732+0000 7ffb9fe00640 1 --2- 172.21.15.161:0/894394905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9809a080 0x7ffb9806c120 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ffb840097f0 tx=0x7ffb84019170 comp rx=0 tx=0).stop 2024-05-22T21:17:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.732+0000 7ffb9fe00640 1 -- 172.21.15.161:0/894394905 shutdown_connections 2024-05-22T21:17:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.732+0000 7ffb9fe00640 1 --2- 172.21.15.161:0/894394905 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb780666c0 0x7ffb78068b80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.732+0000 7ffb9fe00640 1 --2- 172.21.15.161:0/894394905 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb980b3560 0x7ffb9806c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.732+0000 7ffb9fe00640 1 --2- 172.21.15.161:0/894394905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9809a080 0x7ffb9806c120 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.732+0000 7ffb9fe00640 1 --2- 172.21.15.161:0/894394905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb98099ca0 0x7ffb98151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.732+0000 7ffb9fe00640 1 -- 172.21.15.161:0/894394905 >> 172.21.15.161:0/894394905 conn(0x7ffb980b2a40 msgr2=0x7ffb980af240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.732+0000 7ffb9fe00640 1 -- 172.21.15.161:0/894394905 shutdown_connections 2024-05-22T21:17:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.732+0000 7ffb9fe00640 1 -- 172.21.15.161:0/894394905 wait complete. 2024-05-22T21:17:17.755 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-22T21:17:17.755 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:17:17.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.888+0000 7f473f800640 1 Processor -- start 2024-05-22T21:17:17.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.888+0000 7f473f800640 1 -- start start 2024-05-22T21:17:17.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.888+0000 7f473f800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f47380b0490 0x7f47380cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.888+0000 7f473f800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f473806bfb0 0x7f47380cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.888+0000 7f473f800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473806c440 con 0x7f473815d4a0 2024-05-22T21:17:17.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.888+0000 7f473f800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473815d880 con 0x7f473806bfb0 2024-05-22T21:17:17.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.888+0000 7f473f800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473806b440 con 0x7f47380b0490 2024-05-22T21:17:17.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.888+0000 7f473d800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f473806bfb0 0x7f47380cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.888+0000 7f473d800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f473806bfb0 0x7f47380cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34206/0 (socket says 172.21.15.161:34206) 2024-05-22T21:17:17.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.888+0000 7f473d800640 1 -- 172.21.15.161:0/2863713770 learned_addr learned my addr 172.21.15.161:0/2863713770 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:17.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.888+0000 7f472fe00640 1 --2- 172.21.15.161:0/2863713770 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f47380b0490 0x7f47380cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.888+0000 7f473d800640 1 -- 172.21.15.161:0/2863713770 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f47380b0490 msgr2=0x7f47380cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:17.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.888+0000 7f473d800640 1 --2- 172.21.15.161:0/2863713770 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f47380b0490 0x7f47380cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.888+0000 7f473d800640 1 -- 172.21.15.161:0/2863713770 >> v1:172.21.15.179:6789/0 conn(0x7f473815d4a0 legacy=0x7f4738099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.888+0000 7f473d800640 1 -- 172.21.15.161:0/2863713770 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47380cf190 con 0x7f473806bfb0 2024-05-22T21:17:17.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.892+0000 7f473d800640 1 --2- 172.21.15.161:0/2863713770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473806bfb0 0x7f47380cfbb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4730009e50 tx=0x7f473001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d222a3347a1610 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:17.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.892+0000 7f4737e00640 1 -- 172.21.15.161:0/2863713770 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47300a58d0 con 0x7f473806bfb0 2024-05-22T21:17:17.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.892+0000 7f4737e00640 1 -- 172.21.15.161:0/2863713770 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47300a5a30 con 0x7f473806bfb0 2024-05-22T21:17:17.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.892+0000 7f4737e00640 1 -- 172.21.15.161:0/2863713770 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47300a5dc0 con 0x7f473806bfb0 2024-05-22T21:17:17.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.892+0000 7f473f800640 1 -- 172.21.15.161:0/2863713770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473806bfb0 msgr2=0x7f47380cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.892+0000 7f473f800640 1 --2- 172.21.15.161:0/2863713770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473806bfb0 0x7f47380cfbb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4730009e50 tx=0x7f473001ca00 comp rx=0 tx=0).stop 2024-05-22T21:17:17.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.892+0000 7f473f800640 1 -- 172.21.15.161:0/2863713770 shutdown_connections 2024-05-22T21:17:17.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.892+0000 7f473f800640 1 --2- 172.21.15.161:0/2863713770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473806bfb0 0x7f47380cfbb0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.892+0000 7f473f800640 1 --2- 172.21.15.161:0/2863713770 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f47380b0490 0x7f47380cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.892+0000 7f473f800640 1 -- 172.21.15.161:0/2863713770 >> 172.21.15.161:0/2863713770 conn(0x7f47380b2a40 msgr2=0x7f47380b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:17.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.892+0000 7f473f800640 1 -- 172.21.15.161:0/2863713770 shutdown_connections 2024-05-22T21:17:17.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.892+0000 7f473f800640 1 -- 172.21.15.161:0/2863713770 wait complete. 2024-05-22T21:17:17.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f473f800640 1 Processor -- start 2024-05-22T21:17:17.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f473f800640 1 -- start start 2024-05-22T21:17:17.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f473f800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47380b0490 0x7f4738149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f473f800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473815d4a0 0x7f4738159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f473f800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f473815a460 0x7f47380b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f473f800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473815a8f0 con 0x7f473815d4a0 2024-05-22T21:17:17.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f473f800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47381498a0 con 0x7f47380b0490 2024-05-22T21:17:17.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f473f800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47380b5990 con 0x7f473815a460 2024-05-22T21:17:17.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f472fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473815d4a0 0x7f4738159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f472fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473815d4a0 0x7f4738159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34218/0 (socket says 172.21.15.161:34218) 2024-05-22T21:17:17.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f472fe00640 1 -- 172.21.15.161:0/4274737157 learned_addr learned my addr 172.21.15.161:0/4274737157 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:17.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f473ce00640 1 --2- 172.21.15.161:0/4274737157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47380b0490 0x7f4738149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f473d800640 1 --2- 172.21.15.161:0/4274737157 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f473815a460 0x7f47380b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f472fe00640 1 -- 172.21.15.161:0/4274737157 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f473815a460 msgr2=0x7f47380b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f472fe00640 1 --2- 172.21.15.161:0/4274737157 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f473815a460 0x7f47380b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f472fe00640 1 -- 172.21.15.161:0/4274737157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47380b0490 msgr2=0x7f4738149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:17.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f472fe00640 1 --2- 172.21.15.161:0/4274737157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47380b0490 0x7f4738149360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:17.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f472fe00640 1 -- 172.21.15.161:0/4274737157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4730009b30 con 0x7f473815d4a0 2024-05-22T21:17:17.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f472fe00640 1 --2- 172.21.15.161:0/4274737157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473815d4a0 0x7f4738159f20 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f472000e930 tx=0x7f472000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:17.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f4735600640 1 -- 172.21.15.161:0/4274737157 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720009690 con 0x7f473815d4a0 2024-05-22T21:17:17.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f4735600640 1 -- 172.21.15.161:0/4274737157 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472000ca00 con 0x7f473815d4a0 2024-05-22T21:17:17.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f4735600640 1 -- 172.21.15.161:0/4274737157 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472000cd70 con 0x7f473815d4a0 2024-05-22T21:17:17.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f473f800640 1 -- 172.21.15.161:0/4274737157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47380cb7b0 con 0x7f473815d4a0 2024-05-22T21:17:17.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.896+0000 7f473f800640 1 -- 172.21.15.161:0/4274737157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47380b4bd0 con 0x7f473815d4a0 2024-05-22T21:17:17.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.900+0000 7f473f800640 1 -- 172.21.15.161:0/4274737157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4738067220 con 0x7f473815d4a0 2024-05-22T21:17:17.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.900+0000 7f4735600640 1 -- 172.21.15.161:0/4274737157 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4720013470 con 0x7f473815d4a0 2024-05-22T21:17:17.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.900+0000 7f4735600640 1 --2- 172.21.15.161:0/4274737157 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4714066690 0x7f4714068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:17.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.900+0000 7f4735600640 1 -- 172.21.15.161:0/4274737157 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47200842b0 con 0x7f473815d4a0 2024-05-22T21:17:17.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.900+0000 7f473ce00640 1 --2- 172.21.15.161:0/4274737157 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4714066690 0x7f4714068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:17.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.900+0000 7f473ce00640 1 --2- 172.21.15.161:0/4274737157 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4714066690 0x7f4714068b50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f473815cec0 tx=0x7f4728009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:17.904+0000 7f4735600640 1 -- 172.21.15.161:0/4274737157 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4720051820 con 0x7f473815d4a0 2024-05-22T21:17:18.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.056+0000 7f473f800640 1 -- 172.21.15.161:0/4274737157 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47380afe40 con 0x7f4714066690 2024-05-22T21:17:18.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.064+0000 7f4735600640 1 -- 172.21.15.161:0/4274737157 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284665 (secure 0 0 0) 0x7f4728001470 con 0x7f4714066690 2024-05-22T21:17:18.069 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:18.071 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:17:18.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.068+0000 7f473f800640 1 -- 172.21.15.161:0/4274737157 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4714066690 msgr2=0x7f4714068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:18.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.068+0000 7f473f800640 1 --2- 172.21.15.161:0/4274737157 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4714066690 0x7f4714068b50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f473815cec0 tx=0x7f4728009040 comp rx=0 tx=0).stop 2024-05-22T21:17:18.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.068+0000 7f473f800640 1 -- 172.21.15.161:0/4274737157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473815d4a0 msgr2=0x7f4738159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:18.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.068+0000 7f473f800640 1 --2- 172.21.15.161:0/4274737157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473815d4a0 0x7f4738159f20 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f472000e930 tx=0x7f472000edf0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.068+0000 7f473f800640 1 -- 172.21.15.161:0/4274737157 shutdown_connections 2024-05-22T21:17:18.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.068+0000 7f473f800640 1 --2- 172.21.15.161:0/4274737157 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4714066690 0x7f4714068b50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.068+0000 7f473f800640 1 --2- 172.21.15.161:0/4274737157 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f473815a460 0x7f47380b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.068+0000 7f473f800640 1 --2- 172.21.15.161:0/4274737157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473815d4a0 0x7f4738159f20 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.068+0000 7f473f800640 1 --2- 172.21.15.161:0/4274737157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47380b0490 0x7f4738149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.068+0000 7f473f800640 1 -- 172.21.15.161:0/4274737157 >> 172.21.15.161:0/4274737157 conn(0x7f47380b2a40 msgr2=0x7f473806ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:18.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.068+0000 7f473f800640 1 -- 172.21.15.161:0/4274737157 shutdown_connections 2024-05-22T21:17:18.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.068+0000 7f473f800640 1 -- 172.21.15.161:0/4274737157 wait complete. 2024-05-22T21:17:18.090 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-05-22T21:17:16.445220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557866,"statfs":{"total":1600000000,"available":1595255527,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000701"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019844+0000","last_change":"2024-05-22T21:17:14.031361+0000","last_active":"2024-05-22T21:17:15.019844+0000","last_peered":"2024-05-22T21:17:15.019844+0000","last_clean":"2024-05-22T21:17:15.019844+0000","last_became_active":"2024-05-22T21:17:14.030636+0000","last_became_peered":"2024-05-22T21:17:14.030636+0000","last_unstale":"2024-05-22T21:17:15.019844+0000","last_undegraded":"2024-05-22T21:17:15.019844+0000","last_fullsized":"2024-05-22T21:17:15.019844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:12:02.268314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.261796+0000","last_change":"2024-05-22T21:17:14.038198+0000","last_active":"2024-05-22T21:17:15.261796+0000","last_peered":"2024-05-22T21:17:15.261796+0000","last_clean":"2024-05-22T21:17:15.261796+0000","last_became_active":"2024-05-22T21:17:14.035177+0000","last_became_peered":"2024-05-22T21:17:14.035177+0000","last_unstale":"2024-05-22T21:17:15.261796+0000","last_undegraded":"2024-05-22T21:17:15.261796+0000","last_fullsized":"2024-05-22T21:17:15.261796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:15.861947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313022+0000","last_change":"2024-05-22T21:17:14.023221+0000","last_active":"2024-05-22T21:17:15.313022+0000","last_peered":"2024-05-22T21:17:15.313022+0000","last_clean":"2024-05-22T21:17:15.313022+0000","last_became_active":"2024-05-22T21:17:14.022713+0000","last_became_peered":"2024-05-22T21:17:14.022713+0000","last_unstale":"2024-05-22T21:17:15.313022+0000","last_undegraded":"2024-05-22T21:17:15.313022+0000","last_fullsized":"2024-05-22T21:17:15.313022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:33:32.760757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260549+0000","last_change":"2024-05-22T21:17:14.042007+0000","last_active":"2024-05-22T21:17:15.260549+0000","last_peered":"2024-05-22T21:17:15.260549+0000","last_clean":"2024-05-22T21:17:15.260549+0000","last_became_active":"2024-05-22T21:17:14.041701+0000","last_became_peered":"2024-05-22T21:17:14.041701+0000","last_unstale":"2024-05-22T21:17:15.260549+0000","last_undegraded":"2024-05-22T21:17:15.260549+0000","last_fullsized":"2024-05-22T21:17:15.260549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:27:30.284397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018100+0000","last_change":"2024-05-22T21:17:14.027629+0000","last_active":"2024-05-22T21:17:15.018100+0000","last_peered":"2024-05-22T21:17:15.018100+0000","last_clean":"2024-05-22T21:17:15.018100+0000","last_became_active":"2024-05-22T21:17:14.027137+0000","last_became_peered":"2024-05-22T21:17:14.027137+0000","last_unstale":"2024-05-22T21:17:15.018100+0000","last_undegraded":"2024-05-22T21:17:15.018100+0000","last_fullsized":"2024-05-22T21:17:15.018100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:10:24.706988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262126+0000","last_change":"2024-05-22T21:17:14.038784+0000","last_active":"2024-05-22T21:17:15.262126+0000","last_peered":"2024-05-22T21:17:15.262126+0000","last_clean":"2024-05-22T21:17:15.262126+0000","last_became_active":"2024-05-22T21:17:14.037998+0000","last_became_peered":"2024-05-22T21:17:14.037998+0000","last_unstale":"2024-05-22T21:17:15.262126+0000","last_undegraded":"2024-05-22T21:17:15.262126+0000","last_fullsized":"2024-05-22T21:17:15.262126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:55:53.278600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.261640+0000","last_change":"2024-05-22T21:17:14.032031+0000","last_active":"2024-05-22T21:17:15.261640+0000","last_peered":"2024-05-22T21:17:15.261640+0000","last_clean":"2024-05-22T21:17:15.261640+0000","last_became_active":"2024-05-22T21:17:14.031612+0000","last_became_peered":"2024-05-22T21:17:14.031612+0000","last_unstale":"2024-05-22T21:17:15.261640+0000","last_undegraded":"2024-05-22T21:17:15.261640+0000","last_fullsized":"2024-05-22T21:17:15.261640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:22:47.130444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262016+0000","last_change":"2024-05-22T21:17:14.043432+0000","last_active":"2024-05-22T21:17:15.262016+0000","last_peered":"2024-05-22T21:17:15.262016+0000","last_clean":"2024-05-22T21:17:15.262016+0000","last_became_active":"2024-05-22T21:17:14.039368+0000","last_became_peered":"2024-05-22T21:17:14.039368+0000","last_unstale":"2024-05-22T21:17:15.262016+0000","last_undegraded":"2024-05-22T21:17:15.262016+0000","last_fullsized":"2024-05-22T21:17:15.262016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:24:35.728412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.017780+0000","last_change":"2024-05-22T21:17:14.027471+0000","last_active":"2024-05-22T21:17:15.017780+0000","last_peered":"2024-05-22T21:17:15.017780+0000","last_clean":"2024-05-22T21:17:15.017780+0000","last_became_active":"2024-05-22T21:17:14.027252+0000","last_became_peered":"2024-05-22T21:17:14.027252+0000","last_unstale":"2024-05-22T21:17:15.017780+0000","last_undegraded":"2024-05-22T21:17:15.017780+0000","last_fullsized":"2024-05-22T21:17:15.017780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:33:27.486334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019636+0000","last_change":"2024-05-22T21:17:14.029693+0000","last_active":"2024-05-22T21:17:15.019636+0000","last_peered":"2024-05-22T21:17:15.019636+0000","last_clean":"2024-05-22T21:17:15.019636+0000","last_became_active":"2024-05-22T21:17:14.029379+0000","last_became_peered":"2024-05-22T21:17:14.029379+0000","last_unstale":"2024-05-22T21:17:15.019636+0000","last_undegraded":"2024-05-22T21:17:15.019636+0000","last_fullsized":"2024-05-22T21:17:15.019636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:02:43.741069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.020075+0000","last_change":"2024-05-22T21:17:14.031218+0000","last_active":"2024-05-22T21:17:15.020075+0000","last_peered":"2024-05-22T21:17:15.020075+0000","last_clean":"2024-05-22T21:17:15.020075+0000","last_became_active":"2024-05-22T21:17:14.025707+0000","last_became_peered":"2024-05-22T21:17:14.025707+0000","last_unstale":"2024-05-22T21:17:15.020075+0000","last_undegraded":"2024-05-22T21:17:15.020075+0000","last_fullsized":"2024-05-22T21:17:15.020075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:57:36.092872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260616+0000","last_change":"2024-05-22T21:17:14.026892+0000","last_active":"2024-05-22T21:17:15.260616+0000","last_peered":"2024-05-22T21:17:15.260616+0000","last_clean":"2024-05-22T21:17:15.260616+0000","last_became_active":"2024-05-22T21:17:14.026399+0000","last_became_peered":"2024-05-22T21:17:14.026399+0000","last_unstale":"2024-05-22T21:17:15.260616+0000","last_undegraded":"2024-05-22T21:17:15.260616+0000","last_fullsized":"2024-05-22T21:17:15.260616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:22:05.079534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313022+0000","last_change":"2024-05-22T21:17:14.040709+0000","last_active":"2024-05-22T21:17:15.313022+0000","last_peered":"2024-05-22T21:17:15.313022+0000","last_clean":"2024-05-22T21:17:15.313022+0000","last_became_active":"2024-05-22T21:17:14.040466+0000","last_became_peered":"2024-05-22T21:17:14.040466+0000","last_unstale":"2024-05-22T21:17:15.313022+0000","last_undegraded":"2024-05-22T21:17:15.313022+0000","last_fullsized":"2024-05-22T21:17:15.313022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:12:02.268314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.017896+0000","last_change":"2024-05-22T21:17:14.030550+0000","last_active":"2024-05-22T21:17:15.017896+0000","last_peered":"2024-05-22T21:17:15.017896+0000","last_clean":"2024-05-22T21:17:15.017896+0000","last_became_active":"2024-05-22T21:17:14.030309+0000","last_became_peered":"2024-05-22T21:17:14.030309+0000","last_unstale":"2024-05-22T21:17:15.017896+0000","last_undegraded":"2024-05-22T21:17:15.017896+0000","last_fullsized":"2024-05-22T21:17:15.017896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:58:25.347111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313177+0000","last_change":"2024-05-22T21:17:14.028116+0000","last_active":"2024-05-22T21:17:15.313177+0000","last_peered":"2024-05-22T21:17:15.313177+0000","last_clean":"2024-05-22T21:17:15.313177+0000","last_became_active":"2024-05-22T21:17:14.027899+0000","last_became_peered":"2024-05-22T21:17:14.027899+0000","last_unstale":"2024-05-22T21:17:15.313177+0000","last_undegraded":"2024-05-22T21:17:15.313177+0000","last_fullsized":"2024-05-22T21:17:15.313177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:25:04.470055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.020001+0000","last_change":"2024-05-22T21:17:14.041498+0000","last_active":"2024-05-22T21:17:15.020001+0000","last_peered":"2024-05-22T21:17:15.020001+0000","last_clean":"2024-05-22T21:17:15.020001+0000","last_became_active":"2024-05-22T21:17:14.041126+0000","last_became_peered":"2024-05-22T21:17:14.041126+0000","last_unstale":"2024-05-22T21:17:15.020001+0000","last_undegraded":"2024-05-22T21:17:15.020001+0000","last_fullsized":"2024-05-22T21:17:15.020001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:10:56.731234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018042+0000","last_change":"2024-05-22T21:17:14.026057+0000","last_active":"2024-05-22T21:17:15.018042+0000","last_peered":"2024-05-22T21:17:15.018042+0000","last_clean":"2024-05-22T21:17:15.018042+0000","last_became_active":"2024-05-22T21:17:14.025199+0000","last_became_peered":"2024-05-22T21:17:14.025199+0000","last_unstale":"2024-05-22T21:17:15.018042+0000","last_undegraded":"2024-05-22T21:17:15.018042+0000","last_fullsized":"2024-05-22T21:17:15.018042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:20:37.809604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260715+0000","last_change":"2024-05-22T21:17:14.041999+0000","last_active":"2024-05-22T21:17:15.260715+0000","last_peered":"2024-05-22T21:17:15.260715+0000","last_clean":"2024-05-22T21:17:15.260715+0000","last_became_active":"2024-05-22T21:17:14.041698+0000","last_became_peered":"2024-05-22T21:17:14.041698+0000","last_unstale":"2024-05-22T21:17:15.260715+0000","last_undegraded":"2024-05-22T21:17:15.260715+0000","last_fullsized":"2024-05-22T21:17:15.260715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:36:33.104399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.261897+0000","last_change":"2024-05-22T21:17:14.042534+0000","last_active":"2024-05-22T21:17:15.261897+0000","last_peered":"2024-05-22T21:17:15.261897+0000","last_clean":"2024-05-22T21:17:15.261897+0000","last_became_active":"2024-05-22T21:17:14.040925+0000","last_became_peered":"2024-05-22T21:17:14.040925+0000","last_unstale":"2024-05-22T21:17:15.261897+0000","last_undegraded":"2024-05-22T21:17:15.261897+0000","last_fullsized":"2024-05-22T21:17:15.261897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:26:53.813491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313226+0000","last_change":"2024-05-22T21:17:14.028883+0000","last_active":"2024-05-22T21:17:15.313226+0000","last_peered":"2024-05-22T21:17:15.313226+0000","last_clean":"2024-05-22T21:17:15.313226+0000","last_became_active":"2024-05-22T21:17:14.028677+0000","last_became_peered":"2024-05-22T21:17:14.028677+0000","last_unstale":"2024-05-22T21:17:15.313226+0000","last_undegraded":"2024-05-22T21:17:15.313226+0000","last_fullsized":"2024-05-22T21:17:15.313226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:34:55.708745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019896+0000","last_change":"2024-05-22T21:17:14.024298+0000","last_active":"2024-05-22T21:17:15.019896+0000","last_peered":"2024-05-22T21:17:15.019896+0000","last_clean":"2024-05-22T21:17:15.019896+0000","last_became_active":"2024-05-22T21:17:14.024069+0000","last_became_peered":"2024-05-22T21:17:14.024069+0000","last_unstale":"2024-05-22T21:17:15.019896+0000","last_undegraded":"2024-05-22T21:17:15.019896+0000","last_fullsized":"2024-05-22T21:17:15.019896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:26:53.813491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019571+0000","last_change":"2024-05-22T21:17:14.029490+0000","last_active":"2024-05-22T21:17:15.019571+0000","last_peered":"2024-05-22T21:17:15.019571+0000","last_clean":"2024-05-22T21:17:15.019571+0000","last_became_active":"2024-05-22T21:17:14.029252+0000","last_became_peered":"2024-05-22T21:17:14.029252+0000","last_unstale":"2024-05-22T21:17:15.019571+0000","last_undegraded":"2024-05-22T21:17:15.019571+0000","last_fullsized":"2024-05-22T21:17:15.019571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:24:35.728412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019441+0000","last_change":"2024-05-22T21:17:14.027378+0000","last_active":"2024-05-22T21:17:15.019441+0000","last_peered":"2024-05-22T21:17:15.019441+0000","last_clean":"2024-05-22T21:17:15.019441+0000","last_became_active":"2024-05-22T21:17:14.027133+0000","last_became_peered":"2024-05-22T21:17:14.027133+0000","last_unstale":"2024-05-22T21:17:15.019441+0000","last_undegraded":"2024-05-22T21:17:15.019441+0000","last_fullsized":"2024-05-22T21:17:15.019441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:00:38.089359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019495+0000","last_change":"2024-05-22T21:17:14.044367+0000","last_active":"2024-05-22T21:17:15.019495+0000","last_peered":"2024-05-22T21:17:15.019495+0000","last_clean":"2024-05-22T21:17:15.019495+0000","last_became_active":"2024-05-22T21:17:14.043986+0000","last_became_peered":"2024-05-22T21:17:14.043986+0000","last_unstale":"2024-05-22T21:17:15.019495+0000","last_undegraded":"2024-05-22T21:17:15.019495+0000","last_fullsized":"2024-05-22T21:17:15.019495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:35:21.127439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019731+0000","last_change":"2024-05-22T21:17:14.027691+0000","last_active":"2024-05-22T21:17:15.019731+0000","last_peered":"2024-05-22T21:17:15.019731+0000","last_clean":"2024-05-22T21:17:15.019731+0000","last_became_active":"2024-05-22T21:17:14.026527+0000","last_became_peered":"2024-05-22T21:17:14.026527+0000","last_unstale":"2024-05-22T21:17:15.019731+0000","last_undegraded":"2024-05-22T21:17:15.019731+0000","last_fullsized":"2024-05-22T21:17:15.019731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:20:04.893466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262297+0000","last_change":"2024-05-22T21:17:14.044043+0000","last_active":"2024-05-22T21:17:15.262297+0000","last_peered":"2024-05-22T21:17:15.262297+0000","last_clean":"2024-05-22T21:17:15.262297+0000","last_became_active":"2024-05-22T21:17:14.043738+0000","last_became_peered":"2024-05-22T21:17:14.043738+0000","last_unstale":"2024-05-22T21:17:15.262297+0000","last_undegraded":"2024-05-22T21:17:15.262297+0000","last_fullsized":"2024-05-22T21:17:15.262297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:35:21.127439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313445+0000","last_change":"2024-05-22T21:17:14.022101+0000","last_active":"2024-05-22T21:17:15.313445+0000","last_peered":"2024-05-22T21:17:15.313445+0000","last_clean":"2024-05-22T21:17:15.313445+0000","last_became_active":"2024-05-22T21:17:14.021740+0000","last_became_peered":"2024-05-22T21:17:14.021740+0000","last_unstale":"2024-05-22T21:17:15.313445+0000","last_undegraded":"2024-05-22T21:17:15.313445+0000","last_fullsized":"2024-05-22T21:17:15.313445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:54:46.702287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019195+0000","last_change":"2024-05-22T21:17:14.028752+0000","last_active":"2024-05-22T21:17:15.019195+0000","last_peered":"2024-05-22T21:17:15.019195+0000","last_clean":"2024-05-22T21:17:15.019195+0000","last_became_active":"2024-05-22T21:17:14.028504+0000","last_became_peered":"2024-05-22T21:17:14.028504+0000","last_unstale":"2024-05-22T21:17:15.019195+0000","last_undegraded":"2024-05-22T21:17:15.019195+0000","last_fullsized":"2024-05-22T21:17:15.019195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:55:53.278600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313005+0000","last_change":"2024-05-22T21:17:14.029495+0000","last_active":"2024-05-22T21:17:15.313005+0000","last_peered":"2024-05-22T21:17:15.313005+0000","last_clean":"2024-05-22T21:17:15.313005+0000","last_became_active":"2024-05-22T21:17:14.029251+0000","last_became_peered":"2024-05-22T21:17:14.029251+0000","last_unstale":"2024-05-22T21:17:15.313005+0000","last_undegraded":"2024-05-22T21:17:15.313005+0000","last_fullsized":"2024-05-22T21:17:15.313005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:04:48.892044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262269+0000","last_change":"2024-05-22T21:17:14.041069+0000","last_active":"2024-05-22T21:17:15.262269+0000","last_peered":"2024-05-22T21:17:15.262269+0000","last_clean":"2024-05-22T21:17:15.262269+0000","last_became_active":"2024-05-22T21:17:14.040849+0000","last_became_peered":"2024-05-22T21:17:14.040849+0000","last_unstale":"2024-05-22T21:17:15.262269+0000","last_undegraded":"2024-05-22T21:17:15.262269+0000","last_fullsized":"2024-05-22T21:17:15.262269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:43:31.001610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262403+0000","last_change":"2024-05-22T21:17:14.030554+0000","last_active":"2024-05-22T21:17:15.262403+0000","last_peered":"2024-05-22T21:17:15.262403+0000","last_clean":"2024-05-22T21:17:15.262403+0000","last_became_active":"2024-05-22T21:17:14.030330+0000","last_became_peered":"2024-05-22T21:17:14.030330+0000","last_unstale":"2024-05-22T21:17:15.262403+0000","last_undegraded":"2024-05-22T21:17:15.262403+0000","last_fullsized":"2024-05-22T21:17:15.262403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:12:02.268314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.261924+0000","last_change":"2024-05-22T21:17:14.034879+0000","last_active":"2024-05-22T21:17:15.261924+0000","last_peered":"2024-05-22T21:17:15.261924+0000","last_clean":"2024-05-22T21:17:15.261924+0000","last_became_active":"2024-05-22T21:17:14.034587+0000","last_became_peered":"2024-05-22T21:17:14.034587+0000","last_unstale":"2024-05-22T21:17:15.261924+0000","last_undegraded":"2024-05-22T21:17:15.261924+0000","last_fullsized":"2024-05-22T21:17:15.261924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:02:43.741069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019618+0000","last_change":"2024-05-22T21:17:14.044210+0000","last_active":"2024-05-22T21:17:15.019618+0000","last_peered":"2024-05-22T21:17:15.019618+0000","last_clean":"2024-05-22T21:17:15.019618+0000","last_became_active":"2024-05-22T21:17:14.043559+0000","last_became_peered":"2024-05-22T21:17:14.043559+0000","last_unstale":"2024-05-22T21:17:15.019618+0000","last_undegraded":"2024-05-22T21:17:15.019618+0000","last_fullsized":"2024-05-22T21:17:15.019618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:43:31.001610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"83'4","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019689+0000","last_change":"2024-05-22T21:17:13.647773+0000","last_active":"2024-05-22T21:17:15.019689+0000","last_peered":"2024-05-22T21:17:15.019689+0000","last_clean":"2024-05-22T21:17:15.019689+0000","last_became_active":"2024-05-22T21:17:13.646391+0000","last_became_peered":"2024-05-22T21:17:13.646391+0000","last_unstale":"2024-05-22T21:17:15.019689+0000","last_undegraded":"2024-05-22T21:17:15.019689+0000","last_fullsized":"2024-05-22T21:17:15.019689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:26:52.811237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.017939+0000","last_change":"2024-05-22T21:17:14.024552+0000","last_active":"2024-05-22T21:17:15.017939+0000","last_peered":"2024-05-22T21:17:15.017939+0000","last_clean":"2024-05-22T21:17:15.017939+0000","last_became_active":"2024-05-22T21:17:14.024318+0000","last_became_peered":"2024-05-22T21:17:14.024318+0000","last_unstale":"2024-05-22T21:17:15.017939+0000","last_undegraded":"2024-05-22T21:17:15.017939+0000","last_fullsized":"2024-05-22T21:17:15.017939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:48:14.161903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"83'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.312998+0000","last_change":"2024-05-22T21:17:13.639194+0000","last_active":"2024-05-22T21:17:15.312998+0000","last_peered":"2024-05-22T21:17:15.312998+0000","last_clean":"2024-05-22T21:17:15.312998+0000","last_became_active":"2024-05-22T21:17:13.638897+0000","last_became_peered":"2024-05-22T21:17:13.638897+0000","last_unstale":"2024-05-22T21:17:15.312998+0000","last_undegraded":"2024-05-22T21:17:15.312998+0000","last_fullsized":"2024-05-22T21:17:15.312998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:26:16.724198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262251+0000","last_change":"2024-05-22T21:17:14.039069+0000","last_active":"2024-05-22T21:17:15.262251+0000","last_peered":"2024-05-22T21:17:15.262251+0000","last_clean":"2024-05-22T21:17:15.262251+0000","last_became_active":"2024-05-22T21:17:14.038576+0000","last_became_peered":"2024-05-22T21:17:14.038576+0000","last_unstale":"2024-05-22T21:17:15.262251+0000","last_undegraded":"2024-05-22T21:17:15.262251+0000","last_fullsized":"2024-05-22T21:17:15.262251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:20:04.893466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"83'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019975+0000","last_change":"2024-05-22T21:17:13.649420+0000","last_active":"2024-05-22T21:17:15.019975+0000","last_peered":"2024-05-22T21:17:15.019975+0000","last_clean":"2024-05-22T21:17:15.019975+0000","last_became_active":"2024-05-22T21:17:13.648844+0000","last_became_peered":"2024-05-22T21:17:13.648844+0000","last_unstale":"2024-05-22T21:17:15.019975+0000","last_undegraded":"2024-05-22T21:17:15.019975+0000","last_fullsized":"2024-05-22T21:17:15.019975+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:10:59.730979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.020293+0000","last_change":"2024-05-22T21:17:14.024385+0000","last_active":"2024-05-22T21:17:15.020293+0000","last_peered":"2024-05-22T21:17:15.020293+0000","last_clean":"2024-05-22T21:17:15.020293+0000","last_became_active":"2024-05-22T21:17:14.024084+0000","last_became_peered":"2024-05-22T21:17:14.024084+0000","last_unstale":"2024-05-22T21:17:15.020293+0000","last_undegraded":"2024-05-22T21:17:15.020293+0000","last_fullsized":"2024-05-22T21:17:15.020293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:06:06.207443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"83'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262484+0000","last_change":"2024-05-22T21:17:13.649399+0000","last_active":"2024-05-22T21:17:15.262484+0000","last_peered":"2024-05-22T21:17:15.262484+0000","last_clean":"2024-05-22T21:17:15.262484+0000","last_became_active":"2024-05-22T21:17:13.646416+0000","last_became_peered":"2024-05-22T21:17:13.646416+0000","last_unstale":"2024-05-22T21:17:15.262484+0000","last_undegraded":"2024-05-22T21:17:15.262484+0000","last_fullsized":"2024-05-22T21:17:15.262484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:05:13.084939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.261610+0000","last_change":"2024-05-22T21:17:14.030211+0000","last_active":"2024-05-22T21:17:15.261610+0000","last_peered":"2024-05-22T21:17:15.261610+0000","last_clean":"2024-05-22T21:17:15.261610+0000","last_became_active":"2024-05-22T21:17:14.029934+0000","last_became_peered":"2024-05-22T21:17:14.029934+0000","last_unstale":"2024-05-22T21:17:15.261610+0000","last_undegraded":"2024-05-22T21:17:15.261610+0000","last_fullsized":"2024-05-22T21:17:15.261610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:25:04.470055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"83'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313599+0000","last_change":"2024-05-22T21:17:13.644930+0000","last_active":"2024-05-22T21:17:15.313599+0000","last_peered":"2024-05-22T21:17:15.313599+0000","last_clean":"2024-05-22T21:17:15.313599+0000","last_became_active":"2024-05-22T21:17:13.644704+0000","last_became_peered":"2024-05-22T21:17:13.644704+0000","last_unstale":"2024-05-22T21:17:15.313599+0000","last_undegraded":"2024-05-22T21:17:15.313599+0000","last_fullsized":"2024-05-22T21:17:15.313599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:54:14.183878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019678+0000","last_change":"2024-05-22T21:17:14.028054+0000","last_active":"2024-05-22T21:17:15.019678+0000","last_peered":"2024-05-22T21:17:15.019678+0000","last_clean":"2024-05-22T21:17:15.019678+0000","last_became_active":"2024-05-22T21:17:14.027822+0000","last_became_peered":"2024-05-22T21:17:14.027822+0000","last_unstale":"2024-05-22T21:17:15.019678+0000","last_undegraded":"2024-05-22T21:17:15.019678+0000","last_fullsized":"2024-05-22T21:17:15.019678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:18:40.044538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"83'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262257+0000","last_change":"2024-05-22T21:17:13.643736+0000","last_active":"2024-05-22T21:17:15.262257+0000","last_peered":"2024-05-22T21:17:15.262257+0000","last_clean":"2024-05-22T21:17:15.262257+0000","last_became_active":"2024-05-22T21:17:13.643247+0000","last_became_peered":"2024-05-22T21:17:13.643247+0000","last_unstale":"2024-05-22T21:17:15.262257+0000","last_undegraded":"2024-05-22T21:17:15.262257+0000","last_fullsized":"2024-05-22T21:17:15.262257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:32:43.878695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018254+0000","last_change":"2024-05-22T21:17:14.027758+0000","last_active":"2024-05-22T21:17:15.018254+0000","last_peered":"2024-05-22T21:17:15.018254+0000","last_clean":"2024-05-22T21:17:15.018254+0000","last_became_active":"2024-05-22T21:17:14.027423+0000","last_became_peered":"2024-05-22T21:17:14.027423+0000","last_unstale":"2024-05-22T21:17:15.018254+0000","last_undegraded":"2024-05-22T21:17:15.018254+0000","last_fullsized":"2024-05-22T21:17:15.018254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:33:19.956193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260658+0000","last_change":"2024-05-22T21:17:13.643443+0000","last_active":"2024-05-22T21:17:15.260658+0000","last_peered":"2024-05-22T21:17:15.260658+0000","last_clean":"2024-05-22T21:17:15.260658+0000","last_became_active":"2024-05-22T21:17:13.643218+0000","last_became_peered":"2024-05-22T21:17:13.643218+0000","last_unstale":"2024-05-22T21:17:15.260658+0000","last_undegraded":"2024-05-22T21:17:15.260658+0000","last_fullsized":"2024-05-22T21:17:15.260658+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:58:34.189664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.312912+0000","last_change":"2024-05-22T21:17:14.024363+0000","last_active":"2024-05-22T21:17:15.312912+0000","last_peered":"2024-05-22T21:17:15.312912+0000","last_clean":"2024-05-22T21:17:15.312912+0000","last_became_active":"2024-05-22T21:17:14.024159+0000","last_became_peered":"2024-05-22T21:17:14.024159+0000","last_unstale":"2024-05-22T21:17:15.312912+0000","last_undegraded":"2024-05-22T21:17:15.312912+0000","last_fullsized":"2024-05-22T21:17:15.312912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:35:21.127439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018322+0000","last_change":"2024-05-22T21:17:13.642883+0000","last_active":"2024-05-22T21:17:15.018322+0000","last_peered":"2024-05-22T21:17:15.018322+0000","last_clean":"2024-05-22T21:17:15.018322+0000","last_became_active":"2024-05-22T21:17:13.642639+0000","last_became_peered":"2024-05-22T21:17:13.642639+0000","last_unstale":"2024-05-22T21:17:15.018322+0000","last_undegraded":"2024-05-22T21:17:15.018322+0000","last_fullsized":"2024-05-22T21:17:15.018322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:21:45.294614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262517+0000","last_change":"2024-05-22T21:17:14.033375+0000","last_active":"2024-05-22T21:17:15.262517+0000","last_peered":"2024-05-22T21:17:15.262517+0000","last_clean":"2024-05-22T21:17:15.262517+0000","last_became_active":"2024-05-22T21:17:14.033147+0000","last_became_peered":"2024-05-22T21:17:14.033147+0000","last_unstale":"2024-05-22T21:17:15.262517+0000","last_undegraded":"2024-05-22T21:17:15.262517+0000","last_fullsized":"2024-05-22T21:17:15.262517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:04.256203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.020161+0000","last_change":"2024-05-22T21:17:13.644117+0000","last_active":"2024-05-22T21:17:15.020161+0000","last_peered":"2024-05-22T21:17:15.020161+0000","last_clean":"2024-05-22T21:17:15.020161+0000","last_became_active":"2024-05-22T21:17:13.643843+0000","last_became_peered":"2024-05-22T21:17:13.643843+0000","last_unstale":"2024-05-22T21:17:15.020161+0000","last_undegraded":"2024-05-22T21:17:15.020161+0000","last_fullsized":"2024-05-22T21:17:15.020161+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:22:04.077280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.020178+0000","last_change":"2024-05-22T21:17:14.039306+0000","last_active":"2024-05-22T21:17:15.020178+0000","last_peered":"2024-05-22T21:17:15.020178+0000","last_clean":"2024-05-22T21:17:15.020178+0000","last_became_active":"2024-05-22T21:17:14.039084+0000","last_became_peered":"2024-05-22T21:17:14.039084+0000","last_unstale":"2024-05-22T21:17:15.020178+0000","last_undegraded":"2024-05-22T21:17:15.020178+0000","last_fullsized":"2024-05-22T21:17:15.020178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:22:47.130444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262067+0000","last_change":"2024-05-22T21:17:13.646634+0000","last_active":"2024-05-22T21:17:15.262067+0000","last_peered":"2024-05-22T21:17:15.262067+0000","last_clean":"2024-05-22T21:17:15.262067+0000","last_became_active":"2024-05-22T21:17:13.645876+0000","last_became_peered":"2024-05-22T21:17:13.645876+0000","last_unstale":"2024-05-22T21:17:15.262067+0000","last_undegraded":"2024-05-22T21:17:15.262067+0000","last_fullsized":"2024-05-22T21:17:15.262067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:37:06.289852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262021+0000","last_change":"2024-05-22T21:17:14.041272+0000","last_active":"2024-05-22T21:17:15.262021+0000","last_peered":"2024-05-22T21:17:15.262021+0000","last_clean":"2024-05-22T21:17:15.262021+0000","last_became_active":"2024-05-22T21:17:14.040666+0000","last_became_peered":"2024-05-22T21:17:14.040666+0000","last_unstale":"2024-05-22T21:17:15.262021+0000","last_undegraded":"2024-05-22T21:17:15.262021+0000","last_fullsized":"2024-05-22T21:17:15.262021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:21:32.020991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"83'3","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313195+0000","last_change":"2024-05-22T21:17:13.645672+0000","last_active":"2024-05-22T21:17:15.313195+0000","last_peered":"2024-05-22T21:17:15.313195+0000","last_clean":"2024-05-22T21:17:15.313195+0000","last_became_active":"2024-05-22T21:17:13.645438+0000","last_became_peered":"2024-05-22T21:17:13.645438+0000","last_unstale":"2024-05-22T21:17:15.313195+0000","last_undegraded":"2024-05-22T21:17:15.313195+0000","last_fullsized":"2024-05-22T21:17:15.313195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:12:01.266061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018036+0000","last_change":"2024-05-22T21:17:14.026726+0000","last_active":"2024-05-22T21:17:15.018036+0000","last_peered":"2024-05-22T21:17:15.018036+0000","last_clean":"2024-05-22T21:17:15.018036+0000","last_became_active":"2024-05-22T21:17:14.026328+0000","last_became_peered":"2024-05-22T21:17:14.026328+0000","last_unstale":"2024-05-22T21:17:15.018036+0000","last_undegraded":"2024-05-22T21:17:15.018036+0000","last_fullsized":"2024-05-22T21:17:15.018036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:52:13.416295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"83'6","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260747+0000","last_change":"2024-05-22T21:17:13.636869+0000","last_active":"2024-05-22T21:17:15.260747+0000","last_peered":"2024-05-22T21:17:15.260747+0000","last_clean":"2024-05-22T21:17:15.260747+0000","last_became_active":"2024-05-22T21:17:13.636611+0000","last_became_peered":"2024-05-22T21:17:13.636611+0000","last_unstale":"2024-05-22T21:17:15.260747+0000","last_undegraded":"2024-05-22T21:17:15.260747+0000","last_fullsized":"2024-05-22T21:17:15.260747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:32:39.395686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.020271+0000","last_change":"2024-05-22T21:17:14.031490+0000","last_active":"2024-05-22T21:17:15.020271+0000","last_peered":"2024-05-22T21:17:15.020271+0000","last_clean":"2024-05-22T21:17:15.020271+0000","last_became_active":"2024-05-22T21:17:14.030957+0000","last_became_peered":"2024-05-22T21:17:14.030957+0000","last_unstale":"2024-05-22T21:17:15.020271+0000","last_undegraded":"2024-05-22T21:17:15.020271+0000","last_fullsized":"2024-05-22T21:17:15.020271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:54:15.186131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"83'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260673+0000","last_change":"2024-05-22T21:17:13.639375+0000","last_active":"2024-05-22T21:17:15.260673+0000","last_peered":"2024-05-22T21:17:15.260673+0000","last_clean":"2024-05-22T21:17:15.260673+0000","last_became_active":"2024-05-22T21:17:13.636907+0000","last_became_peered":"2024-05-22T21:17:13.636907+0000","last_unstale":"2024-05-22T21:17:15.260673+0000","last_undegraded":"2024-05-22T21:17:15.260673+0000","last_fullsized":"2024-05-22T21:17:15.260673+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:40:36.202431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262368+0000","last_change":"2024-05-22T21:17:14.028881+0000","last_active":"2024-05-22T21:17:15.262368+0000","last_peered":"2024-05-22T21:17:15.262368+0000","last_clean":"2024-05-22T21:17:15.262368+0000","last_became_active":"2024-05-22T21:17:14.028648+0000","last_became_peered":"2024-05-22T21:17:14.028648+0000","last_unstale":"2024-05-22T21:17:15.262368+0000","last_undegraded":"2024-05-22T21:17:15.262368+0000","last_fullsized":"2024-05-22T21:17:15.262368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:34:55.708745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262571+0000","last_change":"2024-05-22T21:17:13.649073+0000","last_active":"2024-05-22T21:17:15.262571+0000","last_peered":"2024-05-22T21:17:15.262571+0000","last_clean":"2024-05-22T21:17:15.262571+0000","last_became_active":"2024-05-22T21:17:13.645401+0000","last_became_peered":"2024-05-22T21:17:13.645401+0000","last_unstale":"2024-05-22T21:17:15.262571+0000","last_undegraded":"2024-05-22T21:17:15.262571+0000","last_fullsized":"2024-05-22T21:17:15.262571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:58:34.189664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313114+0000","last_change":"2024-05-22T21:17:14.037570+0000","last_active":"2024-05-22T21:17:15.313114+0000","last_peered":"2024-05-22T21:17:15.313114+0000","last_clean":"2024-05-22T21:17:15.313114+0000","last_became_active":"2024-05-22T21:17:14.037346+0000","last_became_peered":"2024-05-22T21:17:14.037346+0000","last_unstale":"2024-05-22T21:17:15.313114+0000","last_undegraded":"2024-05-22T21:17:15.313114+0000","last_fullsized":"2024-05-22T21:17:15.313114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:54:15.186131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262625+0000","last_change":"2024-05-22T21:17:13.642468+0000","last_active":"2024-05-22T21:17:15.262625+0000","last_peered":"2024-05-22T21:17:15.262625+0000","last_clean":"2024-05-22T21:17:15.262625+0000","last_became_active":"2024-05-22T21:17:13.642235+0000","last_became_peered":"2024-05-22T21:17:13.642235+0000","last_unstale":"2024-05-22T21:17:15.262625+0000","last_undegraded":"2024-05-22T21:17:15.262625+0000","last_fullsized":"2024-05-22T21:17:15.262625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:20:03.891212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018146+0000","last_change":"2024-05-22T21:17:14.026299+0000","last_active":"2024-05-22T21:17:15.018146+0000","last_peered":"2024-05-22T21:17:15.018146+0000","last_clean":"2024-05-22T21:17:15.018146+0000","last_became_active":"2024-05-22T21:17:14.025759+0000","last_became_peered":"2024-05-22T21:17:14.025759+0000","last_unstale":"2024-05-22T21:17:15.018146+0000","last_undegraded":"2024-05-22T21:17:15.018146+0000","last_fullsized":"2024-05-22T21:17:15.018146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:39:05.614355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"83'8","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019292+0000","last_change":"2024-05-22T21:17:13.648532+0000","last_active":"2024-05-22T21:17:15.019292+0000","last_peered":"2024-05-22T21:17:15.019292+0000","last_clean":"2024-05-22T21:17:15.019292+0000","last_became_active":"2024-05-22T21:17:13.648237+0000","last_became_peered":"2024-05-22T21:17:13.648237+0000","last_unstale":"2024-05-22T21:17:15.019292+0000","last_undegraded":"2024-05-22T21:17:15.019292+0000","last_fullsized":"2024-05-22T21:17:15.019292+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:49:05.183946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019826+0000","last_change":"2024-05-22T21:17:14.028174+0000","last_active":"2024-05-22T21:17:15.019826+0000","last_peered":"2024-05-22T21:17:15.019826+0000","last_clean":"2024-05-22T21:17:15.019826+0000","last_became_active":"2024-05-22T21:17:14.027936+0000","last_became_peered":"2024-05-22T21:17:14.027936+0000","last_unstale":"2024-05-22T21:17:15.019826+0000","last_undegraded":"2024-05-22T21:17:15.019826+0000","last_fullsized":"2024-05-22T21:17:15.019826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:15.861947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"83'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262163+0000","last_change":"2024-05-22T21:17:13.647165+0000","last_active":"2024-05-22T21:17:15.262163+0000","last_peered":"2024-05-22T21:17:15.262163+0000","last_clean":"2024-05-22T21:17:15.262163+0000","last_became_active":"2024-05-22T21:17:13.645068+0000","last_became_peered":"2024-05-22T21:17:13.645068+0000","last_unstale":"2024-05-22T21:17:15.262163+0000","last_undegraded":"2024-05-22T21:17:15.262163+0000","last_fullsized":"2024-05-22T21:17:15.262163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:49:05.183946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019713+0000","last_change":"2024-05-22T21:17:14.026751+0000","last_active":"2024-05-22T21:17:15.019713+0000","last_peered":"2024-05-22T21:17:15.019713+0000","last_clean":"2024-05-22T21:17:15.019713+0000","last_became_active":"2024-05-22T21:17:14.026512+0000","last_became_peered":"2024-05-22T21:17:14.026512+0000","last_unstale":"2024-05-22T21:17:15.019713+0000","last_undegraded":"2024-05-22T21:17:15.019713+0000","last_fullsized":"2024-05-22T21:17:15.019713+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:25:04.470055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"83'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.020066+0000","last_change":"2024-05-22T21:17:13.649244+0000","last_active":"2024-05-22T21:17:15.020066+0000","last_peered":"2024-05-22T21:17:15.020066+0000","last_clean":"2024-05-22T21:17:15.020066+0000","last_became_active":"2024-05-22T21:17:13.648443+0000","last_became_peered":"2024-05-22T21:17:13.648443+0000","last_unstale":"2024-05-22T21:17:15.020066+0000","last_undegraded":"2024-05-22T21:17:15.020066+0000","last_fullsized":"2024-05-22T21:17:15.020066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:36:23.129996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260721+0000","last_change":"2024-05-22T21:17:14.028212+0000","last_active":"2024-05-22T21:17:15.260721+0000","last_peered":"2024-05-22T21:17:15.260721+0000","last_clean":"2024-05-22T21:17:15.260721+0000","last_became_active":"2024-05-22T21:17:14.026676+0000","last_became_peered":"2024-05-22T21:17:14.026676+0000","last_unstale":"2024-05-22T21:17:15.260721+0000","last_undegraded":"2024-05-22T21:17:15.260721+0000","last_fullsized":"2024-05-22T21:17:15.260721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:11:00.733233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"83'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262730+0000","last_change":"2024-05-22T21:17:13.649234+0000","last_active":"2024-05-22T21:17:15.262730+0000","last_peered":"2024-05-22T21:17:15.262730+0000","last_clean":"2024-05-22T21:17:15.262730+0000","last_became_active":"2024-05-22T21:17:13.646019+0000","last_became_peered":"2024-05-22T21:17:13.646019+0000","last_unstale":"2024-05-22T21:17:15.262730+0000","last_undegraded":"2024-05-22T21:17:15.262730+0000","last_fullsized":"2024-05-22T21:17:15.262730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:02:18.597049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018112+0000","last_change":"2024-05-22T21:17:14.023539+0000","last_active":"2024-05-22T21:17:15.018112+0000","last_peered":"2024-05-22T21:17:15.018112+0000","last_clean":"2024-05-22T21:17:15.018112+0000","last_became_active":"2024-05-22T21:17:14.023314+0000","last_became_peered":"2024-05-22T21:17:14.023314+0000","last_unstale":"2024-05-22T21:17:15.018112+0000","last_undegraded":"2024-05-22T21:17:15.018112+0000","last_fullsized":"2024-05-22T21:17:15.018112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:18:11.114230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019510+0000","last_change":"2024-05-22T21:17:13.644389+0000","last_active":"2024-05-22T21:17:15.019510+0000","last_peered":"2024-05-22T21:17:15.019510+0000","last_clean":"2024-05-22T21:17:15.019510+0000","last_became_active":"2024-05-22T21:17:13.644121+0000","last_became_peered":"2024-05-22T21:17:13.644121+0000","last_unstale":"2024-05-22T21:17:15.019510+0000","last_undegraded":"2024-05-22T21:17:15.019510+0000","last_fullsized":"2024-05-22T21:17:15.019510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:24:34.726158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313106+0000","last_change":"2024-05-22T21:17:14.023365+0000","last_active":"2024-05-22T21:17:15.313106+0000","last_peered":"2024-05-22T21:17:15.313106+0000","last_clean":"2024-05-22T21:17:15.313106+0000","last_became_active":"2024-05-22T21:17:14.023004+0000","last_became_peered":"2024-05-22T21:17:14.023004+0000","last_unstale":"2024-05-22T21:17:15.313106+0000","last_undegraded":"2024-05-22T21:17:15.313106+0000","last_fullsized":"2024-05-22T21:17:15.313106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:48:54.143205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262266+0000","last_change":"2024-05-22T21:17:13.645322+0000","last_active":"2024-05-22T21:17:15.262266+0000","last_peered":"2024-05-22T21:17:15.262266+0000","last_clean":"2024-05-22T21:17:15.262266+0000","last_became_active":"2024-05-22T21:17:13.644703+0000","last_became_peered":"2024-05-22T21:17:13.644703+0000","last_unstale":"2024-05-22T21:17:15.262266+0000","last_undegraded":"2024-05-22T21:17:15.262266+0000","last_fullsized":"2024-05-22T21:17:15.262266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:36:23.129996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313092+0000","last_change":"2024-05-22T21:17:14.039243+0000","last_active":"2024-05-22T21:17:15.313092+0000","last_peered":"2024-05-22T21:17:15.313092+0000","last_clean":"2024-05-22T21:17:15.313092+0000","last_became_active":"2024-05-22T21:17:14.039060+0000","last_became_peered":"2024-05-22T21:17:14.039060+0000","last_unstale":"2024-05-22T21:17:15.313092+0000","last_undegraded":"2024-05-22T21:17:15.313092+0000","last_fullsized":"2024-05-22T21:17:15.313092+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:48:29.806509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262456+0000","last_change":"2024-05-22T21:17:13.644094+0000","last_active":"2024-05-22T21:17:15.262456+0000","last_peered":"2024-05-22T21:17:15.262456+0000","last_clean":"2024-05-22T21:17:15.262456+0000","last_became_active":"2024-05-22T21:17:13.643877+0000","last_became_peered":"2024-05-22T21:17:13.643877+0000","last_unstale":"2024-05-22T21:17:15.262456+0000","last_undegraded":"2024-05-22T21:17:15.262456+0000","last_fullsized":"2024-05-22T21:17:15.262456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:18:39.042285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.312946+0000","last_change":"2024-05-22T21:17:14.040825+0000","last_active":"2024-05-22T21:17:15.312946+0000","last_peered":"2024-05-22T21:17:15.312946+0000","last_clean":"2024-05-22T21:17:15.312946+0000","last_became_active":"2024-05-22T21:17:14.040552+0000","last_became_peered":"2024-05-22T21:17:14.040552+0000","last_unstale":"2024-05-22T21:17:15.312946+0000","last_undegraded":"2024-05-22T21:17:15.312946+0000","last_fullsized":"2024-05-22T21:17:15.312946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:02:38.871316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313168+0000","last_change":"2024-05-22T21:17:13.638951+0000","last_active":"2024-05-22T21:17:15.313168+0000","last_peered":"2024-05-22T21:17:15.313168+0000","last_clean":"2024-05-22T21:17:15.313168+0000","last_became_active":"2024-05-22T21:17:13.638709+0000","last_became_peered":"2024-05-22T21:17:13.638709+0000","last_unstale":"2024-05-22T21:17:15.313168+0000","last_undegraded":"2024-05-22T21:17:15.313168+0000","last_fullsized":"2024-05-22T21:17:15.313168+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:42:39.440826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313179+0000","last_change":"2024-05-22T21:17:14.025279+0000","last_active":"2024-05-22T21:17:15.313179+0000","last_peered":"2024-05-22T21:17:15.313179+0000","last_clean":"2024-05-22T21:17:15.313179+0000","last_became_active":"2024-05-22T21:17:14.025009+0000","last_became_peered":"2024-05-22T21:17:14.025009+0000","last_unstale":"2024-05-22T21:17:15.313179+0000","last_undegraded":"2024-05-22T21:17:15.313179+0000","last_fullsized":"2024-05-22T21:17:15.313179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:43:31.001610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"83'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262172+0000","last_change":"2024-05-22T21:17:13.646475+0000","last_active":"2024-05-22T21:17:15.262172+0000","last_peered":"2024-05-22T21:17:15.262172+0000","last_clean":"2024-05-22T21:17:15.262172+0000","last_became_active":"2024-05-22T21:17:13.645525+0000","last_became_peered":"2024-05-22T21:17:13.645525+0000","last_unstale":"2024-05-22T21:17:15.262172+0000","last_undegraded":"2024-05-22T21:17:15.262172+0000","last_fullsized":"2024-05-22T21:17:15.262172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:32:33.619997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262582+0000","last_change":"2024-05-22T21:17:14.030583+0000","last_active":"2024-05-22T21:17:15.262582+0000","last_peered":"2024-05-22T21:17:15.262582+0000","last_clean":"2024-05-22T21:17:15.262582+0000","last_became_active":"2024-05-22T21:17:14.030372+0000","last_became_peered":"2024-05-22T21:17:14.030372+0000","last_unstale":"2024-05-22T21:17:15.262582+0000","last_undegraded":"2024-05-22T21:17:15.262582+0000","last_fullsized":"2024-05-22T21:17:15.262582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:54:15.186131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262201+0000","last_change":"2024-05-22T21:17:13.643297+0000","last_active":"2024-05-22T21:17:15.262201+0000","last_peered":"2024-05-22T21:17:15.262201+0000","last_clean":"2024-05-22T21:17:15.262201+0000","last_became_active":"2024-05-22T21:17:13.643070+0000","last_became_peered":"2024-05-22T21:17:13.643070+0000","last_unstale":"2024-05-22T21:17:15.262201+0000","last_undegraded":"2024-05-22T21:17:15.262201+0000","last_fullsized":"2024-05-22T21:17:15.262201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:10:55.728981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.020362+0000","last_change":"2024-05-22T21:17:14.041268+0000","last_active":"2024-05-22T21:17:15.020362+0000","last_peered":"2024-05-22T21:17:15.020362+0000","last_clean":"2024-05-22T21:17:15.020362+0000","last_became_active":"2024-05-22T21:17:14.040272+0000","last_became_peered":"2024-05-22T21:17:14.040272+0000","last_unstale":"2024-05-22T21:17:15.020362+0000","last_undegraded":"2024-05-22T21:17:15.020362+0000","last_fullsized":"2024-05-22T21:17:15.020362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:34:55.708745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018147+0000","last_change":"2024-05-22T21:17:13.642573+0000","last_active":"2024-05-22T21:17:15.018147+0000","last_peered":"2024-05-22T21:17:15.018147+0000","last_clean":"2024-05-22T21:17:15.018147+0000","last_became_active":"2024-05-22T21:17:13.642199+0000","last_became_peered":"2024-05-22T21:17:13.642199+0000","last_unstale":"2024-05-22T21:17:15.018147+0000","last_undegraded":"2024-05-22T21:17:15.018147+0000","last_fullsized":"2024-05-22T21:17:15.018147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:51:22.313810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019803+0000","last_change":"2024-05-22T21:17:14.029380+0000","last_active":"2024-05-22T21:17:15.019803+0000","last_peered":"2024-05-22T21:17:15.019803+0000","last_clean":"2024-05-22T21:17:15.019803+0000","last_became_active":"2024-05-22T21:17:14.029161+0000","last_became_peered":"2024-05-22T21:17:14.029161+0000","last_unstale":"2024-05-22T21:17:15.019803+0000","last_undegraded":"2024-05-22T21:17:15.019803+0000","last_fullsized":"2024-05-22T21:17:15.019803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:04.256203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313265+0000","last_change":"2024-05-22T21:17:13.643665+0000","last_active":"2024-05-22T21:17:15.313265+0000","last_peered":"2024-05-22T21:17:15.313265+0000","last_clean":"2024-05-22T21:17:15.313265+0000","last_became_active":"2024-05-22T21:17:13.643027+0000","last_became_peered":"2024-05-22T21:17:13.643027+0000","last_unstale":"2024-05-22T21:17:15.313265+0000","last_undegraded":"2024-05-22T21:17:15.313265+0000","last_fullsized":"2024-05-22T21:17:15.313265+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:43:27.468865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018229+0000","last_change":"2024-05-22T21:17:14.026178+0000","last_active":"2024-05-22T21:17:15.018229+0000","last_peered":"2024-05-22T21:17:15.018229+0000","last_clean":"2024-05-22T21:17:15.018229+0000","last_became_active":"2024-05-22T21:17:14.025471+0000","last_became_peered":"2024-05-22T21:17:14.025471+0000","last_unstale":"2024-05-22T21:17:15.018229+0000","last_undegraded":"2024-05-22T21:17:15.018229+0000","last_fullsized":"2024-05-22T21:17:15.018229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:10:33.711742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018189+0000","last_change":"2024-05-22T21:17:13.641901+0000","last_active":"2024-05-22T21:17:15.018189+0000","last_peered":"2024-05-22T21:17:15.018189+0000","last_clean":"2024-05-22T21:17:15.018189+0000","last_became_active":"2024-05-22T21:17:13.640915+0000","last_became_peered":"2024-05-22T21:17:13.640915+0000","last_unstale":"2024-05-22T21:17:15.018189+0000","last_undegraded":"2024-05-22T21:17:15.018189+0000","last_fullsized":"2024-05-22T21:17:15.018189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:49:46.314012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262376+0000","last_change":"2024-05-22T21:17:14.038923+0000","last_active":"2024-05-22T21:17:15.262376+0000","last_peered":"2024-05-22T21:17:15.262376+0000","last_clean":"2024-05-22T21:17:15.262376+0000","last_became_active":"2024-05-22T21:17:14.038297+0000","last_became_peered":"2024-05-22T21:17:14.038297+0000","last_unstale":"2024-05-22T21:17:15.262376+0000","last_undegraded":"2024-05-22T21:17:15.262376+0000","last_fullsized":"2024-05-22T21:17:15.262376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:18:40.044538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260765+0000","last_change":"2024-05-22T21:17:13.644060+0000","last_active":"2024-05-22T21:17:15.260765+0000","last_peered":"2024-05-22T21:17:15.260765+0000","last_clean":"2024-05-22T21:17:15.260765+0000","last_became_active":"2024-05-22T21:17:13.643827+0000","last_became_peered":"2024-05-22T21:17:13.643827+0000","last_unstale":"2024-05-22T21:17:15.260765+0000","last_undegraded":"2024-05-22T21:17:15.260765+0000","last_fullsized":"2024-05-22T21:17:15.260765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:32:33.619997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260612+0000","last_change":"2024-05-22T21:17:14.025635+0000","last_active":"2024-05-22T21:17:15.260612+0000","last_peered":"2024-05-22T21:17:15.260612+0000","last_clean":"2024-05-22T21:17:15.260612+0000","last_became_active":"2024-05-22T21:17:14.025421+0000","last_became_peered":"2024-05-22T21:17:14.025421+0000","last_unstale":"2024-05-22T21:17:15.260612+0000","last_undegraded":"2024-05-22T21:17:15.260612+0000","last_fullsized":"2024-05-22T21:17:15.260612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:54:38.421179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019376+0000","last_change":"2024-05-22T21:17:13.647412+0000","last_active":"2024-05-22T21:17:15.019376+0000","last_peered":"2024-05-22T21:17:15.019376+0000","last_clean":"2024-05-22T21:17:15.019376+0000","last_became_active":"2024-05-22T21:17:13.647139+0000","last_became_peered":"2024-05-22T21:17:13.647139+0000","last_unstale":"2024-05-22T21:17:15.019376+0000","last_undegraded":"2024-05-22T21:17:15.019376+0000","last_fullsized":"2024-05-22T21:17:15.019376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:02:18.597049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262147+0000","last_change":"2024-05-22T21:17:14.041414+0000","last_active":"2024-05-22T21:17:15.262147+0000","last_peered":"2024-05-22T21:17:15.262147+0000","last_clean":"2024-05-22T21:17:15.262147+0000","last_became_active":"2024-05-22T21:17:14.039111+0000","last_became_peered":"2024-05-22T21:17:14.039111+0000","last_unstale":"2024-05-22T21:17:15.262147+0000","last_undegraded":"2024-05-22T21:17:15.262147+0000","last_fullsized":"2024-05-22T21:17:15.262147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:55.777237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262362+0000","last_change":"2024-05-22T21:17:13.647319+0000","last_active":"2024-05-22T21:17:15.262362+0000","last_peered":"2024-05-22T21:17:15.262362+0000","last_clean":"2024-05-22T21:17:15.262362+0000","last_became_active":"2024-05-22T21:17:13.645880+0000","last_became_peered":"2024-05-22T21:17:13.645880+0000","last_unstale":"2024-05-22T21:17:15.262362+0000","last_undegraded":"2024-05-22T21:17:15.262362+0000","last_fullsized":"2024-05-22T21:17:15.262362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:40:36.202431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313105+0000","last_change":"2024-05-22T21:17:14.041401+0000","last_active":"2024-05-22T21:17:15.313105+0000","last_peered":"2024-05-22T21:17:15.313105+0000","last_clean":"2024-05-22T21:17:15.313105+0000","last_became_active":"2024-05-22T21:17:14.041237+0000","last_became_peered":"2024-05-22T21:17:14.041237+0000","last_unstale":"2024-05-22T21:17:15.313105+0000","last_undegraded":"2024-05-22T21:17:15.313105+0000","last_fullsized":"2024-05-22T21:17:15.313105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:36:24.027210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"83'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262463+0000","last_change":"2024-05-22T21:17:13.646851+0000","last_active":"2024-05-22T21:17:15.262463+0000","last_peered":"2024-05-22T21:17:15.262463+0000","last_clean":"2024-05-22T21:17:15.262463+0000","last_became_active":"2024-05-22T21:17:13.646363+0000","last_became_peered":"2024-05-22T21:17:13.646363+0000","last_unstale":"2024-05-22T21:17:15.262463+0000","last_undegraded":"2024-05-22T21:17:15.262463+0000","last_fullsized":"2024-05-22T21:17:15.262463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:32:39.395686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":229,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019893+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:17:15.019893+0000","last_peered":"2024-05-22T21:17:15.019893+0000","last_clean":"2024-05-22T21:17:15.019893+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:17:15.019893+0000","last_undegraded":"2024-05-22T21:17:15.019893+0000","last_fullsized":"2024-05-22T21:17:15.019893+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1011,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194300,"statfs":{"total":200000000,"available":198964062,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.73499999999999999,"15min":0.76500000000000001},"min":{"1min":0.46100000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85599999999999998,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.48599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93300000000000005,"5min":1.889,"15min":1.889},"last":0.82699999999999996}]},{"osd":1,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.76200000000000001,"15min":0.78200000000000003},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0840000000000001,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73599999999999999,"15min":0.76700000000000002},"min":{"1min":0.40300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88400000000000001,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.56299999999999994,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94999999999999996,"5min":1.923,"15min":1.923},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72599999999999998,"15min":0.72099999999999997},"min":{"1min":0.61599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0409999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.90400000000000003}]},{"osd":3,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69399999999999995,"15min":0.68999999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88600000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.626},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77800000000000002,"15min":0.80000000000000004},"min":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.95399999999999996,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.86799999999999999}]},{"osd":4,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.443,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.45200000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90300000000000002,"5min":1.018,"15min":1.018},"last":0.88900000000000001}]},{"osd":5,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64500000000000002,"15min":0.65700000000000003},"min":{"1min":0.42099999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76900000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.67200000000000004}]},{"osd":6,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.63700000000000001},"min":{"1min":0.51000000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93200000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.76600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":434,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194877,"statfs":{"total":200000000,"available":199554954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:16:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.68799999999999994,"15min":0.68999999999999995},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.68300000000000005},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90500000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.72199999999999998}]},{"osd":1,"last update":"Wed May 22 21:16:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.91300000000000003,"5min":1.107,"15min":1.107},"last":0.497},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70899999999999996,"15min":0.70699999999999996},"min":{"1min":0.53900000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92600000000000005,"5min":1.081,"15min":1.081},"last":0.70299999999999996}]},{"osd":2,"last update":"Wed May 22 21:16:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.63900000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0189999999999999,"5min":1.04,"15min":1.04},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.74099999999999999,"15min":0.73699999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96499999999999997,"5min":0.997,"15min":0.997},"last":0.67500000000000004}]},{"osd":3,"last update":"Wed May 22 21:16:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.628,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0069999999999999,"5min":1.415,"15min":1.415},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.55100000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.087,"5min":1.788,"15min":1.788},"last":0.63100000000000001}]},{"osd":4,"last update":"Wed May 22 21:16:15 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64200000000000002,"15min":0.65300000000000002},"min":{"1min":0.42999999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.87,"5min":1.006,"15min":1.006},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.871,"5min":2.718,"15min":2.718},"last":0.56399999999999995}]},{"osd":5,"last update":"Wed May 22 21:16:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.77200000000000002,"15min":0.71199999999999997},"min":{"1min":0.46500000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80000000000000004,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.432},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.73599999999999999}]},{"osd":7,"last update":"Wed May 22 21:16:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.45100000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.095},"last":0.749},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.76100000000000001,"15min":0.76800000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.86299999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.54800000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":434,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194877,"statfs":{"total":200000000,"available":199554956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.70499999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98699999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67500000000000004,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70799999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94899999999999995,"5min":1.216,"15min":1.216},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68799999999999994,"15min":0.68300000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.98299999999999998,"5min":1.048,"15min":1.048},"last":0.77800000000000002}]},{"osd":2,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.61299999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.96699999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.66900000000000004},"min":{"1min":0.53000000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.995,"5min":1.252,"15min":1.252},"last":0.75900000000000001}]},{"osd":3,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.53700000000000003,"5min":0.502,"15min":0.502},"max":{"1min":0.97199999999999998,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.746},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73499999999999999,"15min":0.72999999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.03,"5min":1.111,"15min":1.111},"last":0.83099999999999996}]},{"osd":4,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":0.63500000000000001},"min":{"1min":0.40999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86099999999999999,"5min":1.272,"15min":1.272},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.626,"15min":0.63100000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.70499999999999996}]},{"osd":6,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63200000000000001,"15min":0.629},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89400000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.85099999999999998}]},{"osd":7,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.874},{"interface":"front","average":{"1min":0.745,"5min":0.77900000000000003,"15min":0.79700000000000004},"min":{"1min":0.56499999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.008,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.90600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":434,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194877,"statfs":{"total":200000000,"available":199554976,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:16:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.72499999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92000000000000004,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69499999999999995,"15min":0.71099999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86299999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79800000000000004}]},{"osd":1,"last update":"Wed May 22 21:16:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69799999999999995,"15min":0.72199999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88900000000000001,"5min":1.5,"15min":1.5},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.45900000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.75900000000000001}]},{"osd":2,"last update":"Wed May 22 21:16:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82699999999999996,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94499999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60399999999999998}]},{"osd":3,"last update":"Wed May 22 21:16:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.66700000000000004,"15min":0.68300000000000005},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.73799999999999999,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.51500000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88700000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.67600000000000005}]},{"osd":5,"last update":"Wed May 22 21:16:14 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63600000000000001,"15min":0.64700000000000002},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":0.84299999999999997,"5min":1.012,"15min":1.012},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.875,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64000000000000001}]},{"osd":6,"last update":"Wed May 22 21:16:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68000000000000005,"15min":0.69999999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80300000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.68400000000000005,"15min":0.69199999999999995},"min":{"1min":0.47599999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66300000000000003}]},{"osd":7,"last update":"Wed May 22 21:16:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69099999999999995,"15min":0.69799999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82599999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.751},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76900000000000002,"15min":0.78900000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91100000000000003,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.77400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":434,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194877,"statfs":{"total":200000000,"available":199554934,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:16:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.40799999999999997,"15min":0.40000000000000002},"min":{"1min":0.34200000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.41799999999999998,"15min":0.40000000000000002},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.49399999999999999}]},{"osd":1,"last update":"Wed May 22 21:16:16 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45300000000000001,"15min":0.44500000000000001},"min":{"1min":0.34300000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.433,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83099999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.55100000000000005}]},{"osd":2,"last update":"Wed May 22 21:16:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.40600000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70699999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.441},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.49299999999999999,"15min":0.48399999999999999},"min":{"1min":0.38400000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Wed May 22 21:16:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71299999999999997,"15min":0.69699999999999995},"min":{"1min":0.59799999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71799999999999997,"15min":0.70799999999999996},"min":{"1min":0.56100000000000005,"5min":0.435,"15min":0.435},"max":{"1min":1.0549999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.66700000000000004}]},{"osd":5,"last update":"Wed May 22 21:16:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.60899999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1000000000000001,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.70899999999999996,"15min":0.68999999999999995},"min":{"1min":0.58499999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0189999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.59799999999999998}]},{"osd":6,"last update":"Wed May 22 21:16:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.72499999999999998,"15min":0.70899999999999996},"min":{"1min":0.53400000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.71699999999999997,"15min":0.69399999999999995},"min":{"1min":0.56799999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.999,"5min":1.25,"15min":1.25},"last":0.57999999999999996}]},{"osd":7,"last update":"Wed May 22 21:16:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.76700000000000002,"15min":0.749},"min":{"1min":0.61799999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.073,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.748,"15min":0.72799999999999998},"min":{"1min":0.63500000000000001,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.016,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":436,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194876,"statfs":{"total":200000000,"available":199553482,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.436,"15min":0.434},"min":{"1min":0.34300000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.308}]},{"osd":1,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.443,"15min":0.442},"min":{"1min":0.33300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68100000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.46400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.67200000000000004,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.41999999999999998}]},{"osd":3,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.495,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73899999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.45300000000000001},"min":{"1min":0.375,"5min":0.315,"15min":0.315},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.38100000000000001}]},{"osd":4,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.54200000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.57599999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.78200000000000003}]},{"osd":5,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.502,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.58899999999999997}]},{"osd":6,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.68200000000000005,"15min":0.67200000000000004},"min":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.628}]},{"osd":7,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.53700000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.77500000000000002,"15min":0.76700000000000002},"min":{"1min":0.622,"5min":0.621,"15min":0.621},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.76100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":436,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194875,"statfs":{"total":200000000,"available":199552866,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42799999999999999,"15min":0.434},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54100000000000004,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.438,"15min":0.42899999999999999},"min":{"1min":0.26900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69299999999999995,"5min":0.876,"15min":0.876},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52200000000000002,"15min":0.54000000000000004},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70699999999999996,"5min":1.923,"15min":1.923},"last":0.61499999999999999}]},{"osd":3,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56000000000000005,"15min":0.57399999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.755,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.64500000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.52600000000000002}]},{"osd":4,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.74399999999999999,"15min":0.76100000000000001},"min":{"1min":0.46500000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.98599999999999999,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68700000000000006,"15min":0.67700000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.069,"5min":1.3,"15min":1.3},"last":1.0389999999999999}]},{"osd":5,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.59499999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0609999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.749,"15min":0.752},"min":{"1min":0.54600000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0469999999999999,"5min":1.97,"15min":1.97},"last":1.1000000000000001}]},{"osd":6,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.746,"15min":0.73899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.085,"5min":1.272,"15min":1.272},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75,"15min":0.73799999999999999},"min":{"1min":0.60499999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0309999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.98799999999999999}]},{"osd":7,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96199999999999997,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.76900000000000002,"15min":0.78100000000000003},"min":{"1min":0.57699999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.008,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.88200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1010,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194302,"statfs":{"total":200000000,"available":198965297,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42599999999999999,"15min":0.40400000000000003},"min":{"1min":0.32200000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.73499999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.378,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67300000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.623}]},{"osd":2,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47699999999999998,"15min":0.45100000000000001},"min":{"1min":0.33200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70499999999999996,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.439,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76500000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.53400000000000003}]},{"osd":3,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.45000000000000001,"15min":0.432},"min":{"1min":0.30199999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68899999999999995,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51900000000000002,"15min":0.498},"min":{"1min":0.42499999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79300000000000004,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.54900000000000004}]},{"osd":4,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72699999999999998,"15min":0.70899999999999996},"min":{"1min":0.56000000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.754},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.72699999999999998},"min":{"1min":0.52600000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0049999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.88}]},{"osd":5,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97199999999999998,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.75600000000000001,"15min":0.73499999999999999},"min":{"1min":0.58699999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1220000000000001,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.78000000000000003}]},{"osd":6,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78800000000000003,"15min":0.77100000000000002},"min":{"1min":0.65400000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0820000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.77500000000000002,"15min":0.77000000000000002},"min":{"1min":0.58499999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0640000000000001,"5min":1.5,"15min":1.5},"last":0.73299999999999998}]},{"osd":7,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.74099999999999999,"15min":0.72199999999999998},"min":{"1min":0.61199999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.0589999999999999,"5min":1.085,"15min":1.085},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.65000000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.1990000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.80600000000000005}]}]}],"pool_statfs":[]}} 2024-05-22T21:17:18.096 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:17:18.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fcee1200640 1 Processor -- start 2024-05-22T21:17:18.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fcee1200640 1 -- start start 2024-05-22T21:17:18.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fcee1200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcedc070db0 0x7fcedc0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:18.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fcee1200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcedc0b2ea0 0x7fcedc0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:18.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fcee1200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcedc159bd0 con 0x7fcedc1597f0 2024-05-22T21:17:18.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fcee1200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcedc0cbdd0 con 0x7fcedc0b2ea0 2024-05-22T21:17:18.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fcee1200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcedc0cc050 con 0x7fcedc070db0 2024-05-22T21:17:18.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fced9e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fcedc1597f0 0x7fcedc0706a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39018/0 (socket says 172.21.15.161:39018) 2024-05-22T21:17:18.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fced9e00640 1 -- 172.21.15.161:0/62161647 learned_addr learned my addr 172.21.15.161:0/62161647 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:18.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fceda800640 1 --2- 172.21.15.161:0/62161647 >> v2:172.21.15.161:3300/0 conn(0x7fcedc0b2ea0 0x7fcedc0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:18.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fced1400640 1 --2- 172.21.15.161:0/62161647 >> v2:172.21.15.179:3300/0 conn(0x7fcedc070db0 0x7fcedc0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:18.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fceda800640 1 -- 172.21.15.161:0/62161647 >> v2:172.21.15.179:3300/0 conn(0x7fcedc070db0 msgr2=0x7fcedc0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:18.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fceda800640 1 --2- 172.21.15.161:0/62161647 >> v2:172.21.15.179:3300/0 conn(0x7fcedc070db0 0x7fcedc0b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fceda800640 1 -- 172.21.15.161:0/62161647 >> v1:172.21.15.161:6789/0 conn(0x7fcedc1597f0 legacy=0x7fcedc0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:18.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fced9400640 1 -- 172.21.15.161:0/62161647 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2996491979 0 0) 0x7fcedc159bd0 con 0x7fcedc1597f0 2024-05-22T21:17:18.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fceda800640 1 -- 172.21.15.161:0/62161647 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc0b3720 con 0x7fcedc0b2ea0 2024-05-22T21:17:18.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.264+0000 7fced1400640 1 --2- 172.21.15.161:0/62161647 >> v2:172.21.15.179:3300/0 conn(0x7fcedc070db0 0x7fcedc0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:17:18.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.268+0000 7fceda800640 1 --2- 172.21.15.161:0/62161647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcedc0b2ea0 0x7fcedc0cc550 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fcedc073ce0 tx=0x7fced401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7f0a774914ef6c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:18.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.268+0000 7fced9400640 1 -- 172.21.15.161:0/62161647 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced40ac640 con 0x7fcedc0b2ea0 2024-05-22T21:17:18.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.268+0000 7fced9400640 1 -- 172.21.15.161:0/62161647 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced40ac7a0 con 0x7fcedc0b2ea0 2024-05-22T21:17:18.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.268+0000 7fced9400640 1 -- 172.21.15.161:0/62161647 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced40acb30 con 0x7fcedc0b2ea0 2024-05-22T21:17:18.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.268+0000 7fcee1200640 1 -- 172.21.15.161:0/62161647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcedc0b2ea0 msgr2=0x7fcedc0cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:18.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.268+0000 7fcee1200640 1 --2- 172.21.15.161:0/62161647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcedc0b2ea0 0x7fcedc0cc550 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fcedc073ce0 tx=0x7fced401c460 comp rx=0 tx=0).stop 2024-05-22T21:17:18.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.268+0000 7fcee1200640 1 -- 172.21.15.161:0/62161647 shutdown_connections 2024-05-22T21:17:18.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.268+0000 7fcee1200640 1 --2- 172.21.15.161:0/62161647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcedc0b2ea0 0x7fcedc0cc550 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.268+0000 7fcee1200640 1 --2- 172.21.15.161:0/62161647 >> v2:172.21.15.179:3300/0 conn(0x7fcedc070db0 0x7fcedc0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.268+0000 7fcee1200640 1 -- 172.21.15.161:0/62161647 >> 172.21.15.161:0/62161647 conn(0x7fcedc0b0bc0 msgr2=0x7fcedc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:18.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.268+0000 7fcee1200640 1 -- 172.21.15.161:0/62161647 shutdown_connections 2024-05-22T21:17:18.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.268+0000 7fcee1200640 1 -- 172.21.15.161:0/62161647 wait complete. 2024-05-22T21:17:18.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fcee1200640 1 Processor -- start 2024-05-22T21:17:18.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fcee1200640 1 -- start start 2024-05-22T21:17:18.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fcee1200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcedc070db0 0x7fcedc153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:18.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fcee1200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcedc0b2ea0 0x7fcedc1f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:18.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fcee1200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcedc1597f0 0x7fcedc13d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:18.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fcee1200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc1f7e90 con 0x7fcedc1597f0 2024-05-22T21:17:18.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fcee1200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc15a8b0 con 0x7fcedc0b2ea0 2024-05-22T21:17:18.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fcee1200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc153850 con 0x7fcedc070db0 2024-05-22T21:17:18.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fceda800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcedc1597f0 0x7fcedc13d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:18.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fceda800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcedc1597f0 0x7fcedc13d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34242/0 (socket says 172.21.15.161:34242) 2024-05-22T21:17:18.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fceda800640 1 -- 172.21.15.161:0/1714547247 learned_addr learned my addr 172.21.15.161:0/1714547247 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:18.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fced9e00640 1 --2- 172.21.15.161:0/1714547247 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcedc070db0 0x7fcedc153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:18.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fceda800640 1 -- 172.21.15.161:0/1714547247 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcedc070db0 msgr2=0x7fcedc153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:18.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fceda800640 1 --2- 172.21.15.161:0/1714547247 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcedc070db0 0x7fcedc153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fceda800640 1 -- 172.21.15.161:0/1714547247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcedc0b2ea0 msgr2=0x7fcedc1f7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:18.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fced1400640 1 --2- 172.21.15.161:0/1714547247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcedc0b2ea0 0x7fcedc1f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:18.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fceda800640 1 --2- 172.21.15.161:0/1714547247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcedc0b2ea0 0x7fcedc1f7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fceda800640 1 -- 172.21.15.161:0/1714547247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced4009420 con 0x7fcedc1597f0 2024-05-22T21:17:18.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fced9e00640 1 --2- 172.21.15.161:0/1714547247 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcedc070db0 0x7fcedc153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:17:18.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fced1400640 1 --2- 172.21.15.161:0/1714547247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcedc0b2ea0 0x7fcedc1f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:17:18.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fceda800640 1 --2- 172.21.15.161:0/1714547247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcedc1597f0 0x7fcedc13d1c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fced4019070 tx=0x7fced401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:18.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fced2a00640 1 -- 172.21.15.161:0/1714547247 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced40b8660 con 0x7fcedc1597f0 2024-05-22T21:17:18.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fced2a00640 1 -- 172.21.15.161:0/1714547247 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced40b87c0 con 0x7fcedc1597f0 2024-05-22T21:17:18.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fcee1200640 1 -- 172.21.15.161:0/1714547247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc153b30 con 0x7fcedc1597f0 2024-05-22T21:17:18.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.272+0000 7fced2a00640 1 -- 172.21.15.161:0/1714547247 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced40b8b50 con 0x7fcedc1597f0 2024-05-22T21:17:18.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.276+0000 7fcee1200640 1 -- 172.21.15.161:0/1714547247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc1540f0 con 0x7fcedc1597f0 2024-05-22T21:17:18.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.276+0000 7fcee1200640 1 -- 172.21.15.161:0/1714547247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc067220 con 0x7fcedc1597f0 2024-05-22T21:17:18.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.276+0000 7fced2a00640 1 -- 172.21.15.161:0/1714547247 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fced40b8cb0 con 0x7fcedc1597f0 2024-05-22T21:17:18.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.276+0000 7fced2a00640 1 --2- 172.21.15.161:0/1714547247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fceb8066690 0x7fceb8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:18.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.276+0000 7fced2a00640 1 -- 172.21.15.161:0/1714547247 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fced411eea0 con 0x7fcedc1597f0 2024-05-22T21:17:18.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.276+0000 7fced9e00640 1 --2- 172.21.15.161:0/1714547247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fceb8066690 0x7fceb8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:18.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.276+0000 7fced9e00640 1 --2- 172.21.15.161:0/1714547247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fceb8066690 0x7fceb8068b50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0012f0 tx=0x7fcecc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:18.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.280+0000 7fced2a00640 1 -- 172.21.15.161:0/1714547247 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fced40ec180 con 0x7fcedc1597f0 2024-05-22T21:17:18.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.432+0000 7fcee1200640 1 -- 172.21.15.161:0/1714547247 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcedc0b3500 con 0x7fceb8066690 2024-05-22T21:17:18.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.440+0000 7fced2a00640 1 -- 172.21.15.161:0/1714547247 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284665 (secure 0 0 0) 0x7fcecc00dcf0 con 0x7fceb8066690 2024-05-22T21:17:18.445 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:18.448 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:17:18.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.440+0000 7fcee1200640 1 -- 172.21.15.161:0/1714547247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fceb8066690 msgr2=0x7fceb8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:18.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.440+0000 7fcee1200640 1 --2- 172.21.15.161:0/1714547247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fceb8066690 0x7fceb8068b50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0012f0 tx=0x7fcecc009040 comp rx=0 tx=0).stop 2024-05-22T21:17:18.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.440+0000 7fcee1200640 1 -- 172.21.15.161:0/1714547247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcedc1597f0 msgr2=0x7fcedc13d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:18.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.440+0000 7fcee1200640 1 --2- 172.21.15.161:0/1714547247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcedc1597f0 0x7fcedc13d1c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fced4019070 tx=0x7fced401aca0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.440+0000 7fcee1200640 1 -- 172.21.15.161:0/1714547247 shutdown_connections 2024-05-22T21:17:18.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.440+0000 7fcee1200640 1 --2- 172.21.15.161:0/1714547247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fceb8066690 0x7fceb8068b50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.440+0000 7fcee1200640 1 --2- 172.21.15.161:0/1714547247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcedc1597f0 0x7fcedc13d1c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.440+0000 7fcee1200640 1 --2- 172.21.15.161:0/1714547247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcedc0b2ea0 0x7fcedc1f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.440+0000 7fcee1200640 1 --2- 172.21.15.161:0/1714547247 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcedc070db0 0x7fcedc153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.440+0000 7fcee1200640 1 -- 172.21.15.161:0/1714547247 >> 172.21.15.161:0/1714547247 conn(0x7fcedc0b0bc0 msgr2=0x7fcedc06d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:18.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.440+0000 7fcee1200640 1 -- 172.21.15.161:0/1714547247 shutdown_connections 2024-05-22T21:17:18.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.440+0000 7fcee1200640 1 -- 172.21.15.161:0/1714547247 wait complete. 2024-05-22T21:17:18.465 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-05-22T21:17:16.445220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557866,"statfs":{"total":1600000000,"available":1595255527,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000701"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019844+0000","last_change":"2024-05-22T21:17:14.031361+0000","last_active":"2024-05-22T21:17:15.019844+0000","last_peered":"2024-05-22T21:17:15.019844+0000","last_clean":"2024-05-22T21:17:15.019844+0000","last_became_active":"2024-05-22T21:17:14.030636+0000","last_became_peered":"2024-05-22T21:17:14.030636+0000","last_unstale":"2024-05-22T21:17:15.019844+0000","last_undegraded":"2024-05-22T21:17:15.019844+0000","last_fullsized":"2024-05-22T21:17:15.019844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:12:02.268314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.261796+0000","last_change":"2024-05-22T21:17:14.038198+0000","last_active":"2024-05-22T21:17:15.261796+0000","last_peered":"2024-05-22T21:17:15.261796+0000","last_clean":"2024-05-22T21:17:15.261796+0000","last_became_active":"2024-05-22T21:17:14.035177+0000","last_became_peered":"2024-05-22T21:17:14.035177+0000","last_unstale":"2024-05-22T21:17:15.261796+0000","last_undegraded":"2024-05-22T21:17:15.261796+0000","last_fullsized":"2024-05-22T21:17:15.261796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:15.861947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313022+0000","last_change":"2024-05-22T21:17:14.023221+0000","last_active":"2024-05-22T21:17:15.313022+0000","last_peered":"2024-05-22T21:17:15.313022+0000","last_clean":"2024-05-22T21:17:15.313022+0000","last_became_active":"2024-05-22T21:17:14.022713+0000","last_became_peered":"2024-05-22T21:17:14.022713+0000","last_unstale":"2024-05-22T21:17:15.313022+0000","last_undegraded":"2024-05-22T21:17:15.313022+0000","last_fullsized":"2024-05-22T21:17:15.313022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:33:32.760757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260549+0000","last_change":"2024-05-22T21:17:14.042007+0000","last_active":"2024-05-22T21:17:15.260549+0000","last_peered":"2024-05-22T21:17:15.260549+0000","last_clean":"2024-05-22T21:17:15.260549+0000","last_became_active":"2024-05-22T21:17:14.041701+0000","last_became_peered":"2024-05-22T21:17:14.041701+0000","last_unstale":"2024-05-22T21:17:15.260549+0000","last_undegraded":"2024-05-22T21:17:15.260549+0000","last_fullsized":"2024-05-22T21:17:15.260549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:27:30.284397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018100+0000","last_change":"2024-05-22T21:17:14.027629+0000","last_active":"2024-05-22T21:17:15.018100+0000","last_peered":"2024-05-22T21:17:15.018100+0000","last_clean":"2024-05-22T21:17:15.018100+0000","last_became_active":"2024-05-22T21:17:14.027137+0000","last_became_peered":"2024-05-22T21:17:14.027137+0000","last_unstale":"2024-05-22T21:17:15.018100+0000","last_undegraded":"2024-05-22T21:17:15.018100+0000","last_fullsized":"2024-05-22T21:17:15.018100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:10:24.706988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262126+0000","last_change":"2024-05-22T21:17:14.038784+0000","last_active":"2024-05-22T21:17:15.262126+0000","last_peered":"2024-05-22T21:17:15.262126+0000","last_clean":"2024-05-22T21:17:15.262126+0000","last_became_active":"2024-05-22T21:17:14.037998+0000","last_became_peered":"2024-05-22T21:17:14.037998+0000","last_unstale":"2024-05-22T21:17:15.262126+0000","last_undegraded":"2024-05-22T21:17:15.262126+0000","last_fullsized":"2024-05-22T21:17:15.262126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:55:53.278600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.261640+0000","last_change":"2024-05-22T21:17:14.032031+0000","last_active":"2024-05-22T21:17:15.261640+0000","last_peered":"2024-05-22T21:17:15.261640+0000","last_clean":"2024-05-22T21:17:15.261640+0000","last_became_active":"2024-05-22T21:17:14.031612+0000","last_became_peered":"2024-05-22T21:17:14.031612+0000","last_unstale":"2024-05-22T21:17:15.261640+0000","last_undegraded":"2024-05-22T21:17:15.261640+0000","last_fullsized":"2024-05-22T21:17:15.261640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:22:47.130444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262016+0000","last_change":"2024-05-22T21:17:14.043432+0000","last_active":"2024-05-22T21:17:15.262016+0000","last_peered":"2024-05-22T21:17:15.262016+0000","last_clean":"2024-05-22T21:17:15.262016+0000","last_became_active":"2024-05-22T21:17:14.039368+0000","last_became_peered":"2024-05-22T21:17:14.039368+0000","last_unstale":"2024-05-22T21:17:15.262016+0000","last_undegraded":"2024-05-22T21:17:15.262016+0000","last_fullsized":"2024-05-22T21:17:15.262016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:24:35.728412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.017780+0000","last_change":"2024-05-22T21:17:14.027471+0000","last_active":"2024-05-22T21:17:15.017780+0000","last_peered":"2024-05-22T21:17:15.017780+0000","last_clean":"2024-05-22T21:17:15.017780+0000","last_became_active":"2024-05-22T21:17:14.027252+0000","last_became_peered":"2024-05-22T21:17:14.027252+0000","last_unstale":"2024-05-22T21:17:15.017780+0000","last_undegraded":"2024-05-22T21:17:15.017780+0000","last_fullsized":"2024-05-22T21:17:15.017780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:33:27.486334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019636+0000","last_change":"2024-05-22T21:17:14.029693+0000","last_active":"2024-05-22T21:17:15.019636+0000","last_peered":"2024-05-22T21:17:15.019636+0000","last_clean":"2024-05-22T21:17:15.019636+0000","last_became_active":"2024-05-22T21:17:14.029379+0000","last_became_peered":"2024-05-22T21:17:14.029379+0000","last_unstale":"2024-05-22T21:17:15.019636+0000","last_undegraded":"2024-05-22T21:17:15.019636+0000","last_fullsized":"2024-05-22T21:17:15.019636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:02:43.741069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.020075+0000","last_change":"2024-05-22T21:17:14.031218+0000","last_active":"2024-05-22T21:17:15.020075+0000","last_peered":"2024-05-22T21:17:15.020075+0000","last_clean":"2024-05-22T21:17:15.020075+0000","last_became_active":"2024-05-22T21:17:14.025707+0000","last_became_peered":"2024-05-22T21:17:14.025707+0000","last_unstale":"2024-05-22T21:17:15.020075+0000","last_undegraded":"2024-05-22T21:17:15.020075+0000","last_fullsized":"2024-05-22T21:17:15.020075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:57:36.092872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260616+0000","last_change":"2024-05-22T21:17:14.026892+0000","last_active":"2024-05-22T21:17:15.260616+0000","last_peered":"2024-05-22T21:17:15.260616+0000","last_clean":"2024-05-22T21:17:15.260616+0000","last_became_active":"2024-05-22T21:17:14.026399+0000","last_became_peered":"2024-05-22T21:17:14.026399+0000","last_unstale":"2024-05-22T21:17:15.260616+0000","last_undegraded":"2024-05-22T21:17:15.260616+0000","last_fullsized":"2024-05-22T21:17:15.260616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:22:05.079534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313022+0000","last_change":"2024-05-22T21:17:14.040709+0000","last_active":"2024-05-22T21:17:15.313022+0000","last_peered":"2024-05-22T21:17:15.313022+0000","last_clean":"2024-05-22T21:17:15.313022+0000","last_became_active":"2024-05-22T21:17:14.040466+0000","last_became_peered":"2024-05-22T21:17:14.040466+0000","last_unstale":"2024-05-22T21:17:15.313022+0000","last_undegraded":"2024-05-22T21:17:15.313022+0000","last_fullsized":"2024-05-22T21:17:15.313022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:12:02.268314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.017896+0000","last_change":"2024-05-22T21:17:14.030550+0000","last_active":"2024-05-22T21:17:15.017896+0000","last_peered":"2024-05-22T21:17:15.017896+0000","last_clean":"2024-05-22T21:17:15.017896+0000","last_became_active":"2024-05-22T21:17:14.030309+0000","last_became_peered":"2024-05-22T21:17:14.030309+0000","last_unstale":"2024-05-22T21:17:15.017896+0000","last_undegraded":"2024-05-22T21:17:15.017896+0000","last_fullsized":"2024-05-22T21:17:15.017896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:58:25.347111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313177+0000","last_change":"2024-05-22T21:17:14.028116+0000","last_active":"2024-05-22T21:17:15.313177+0000","last_peered":"2024-05-22T21:17:15.313177+0000","last_clean":"2024-05-22T21:17:15.313177+0000","last_became_active":"2024-05-22T21:17:14.027899+0000","last_became_peered":"2024-05-22T21:17:14.027899+0000","last_unstale":"2024-05-22T21:17:15.313177+0000","last_undegraded":"2024-05-22T21:17:15.313177+0000","last_fullsized":"2024-05-22T21:17:15.313177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:25:04.470055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.020001+0000","last_change":"2024-05-22T21:17:14.041498+0000","last_active":"2024-05-22T21:17:15.020001+0000","last_peered":"2024-05-22T21:17:15.020001+0000","last_clean":"2024-05-22T21:17:15.020001+0000","last_became_active":"2024-05-22T21:17:14.041126+0000","last_became_peered":"2024-05-22T21:17:14.041126+0000","last_unstale":"2024-05-22T21:17:15.020001+0000","last_undegraded":"2024-05-22T21:17:15.020001+0000","last_fullsized":"2024-05-22T21:17:15.020001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:10:56.731234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018042+0000","last_change":"2024-05-22T21:17:14.026057+0000","last_active":"2024-05-22T21:17:15.018042+0000","last_peered":"2024-05-22T21:17:15.018042+0000","last_clean":"2024-05-22T21:17:15.018042+0000","last_became_active":"2024-05-22T21:17:14.025199+0000","last_became_peered":"2024-05-22T21:17:14.025199+0000","last_unstale":"2024-05-22T21:17:15.018042+0000","last_undegraded":"2024-05-22T21:17:15.018042+0000","last_fullsized":"2024-05-22T21:17:15.018042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:20:37.809604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260715+0000","last_change":"2024-05-22T21:17:14.041999+0000","last_active":"2024-05-22T21:17:15.260715+0000","last_peered":"2024-05-22T21:17:15.260715+0000","last_clean":"2024-05-22T21:17:15.260715+0000","last_became_active":"2024-05-22T21:17:14.041698+0000","last_became_peered":"2024-05-22T21:17:14.041698+0000","last_unstale":"2024-05-22T21:17:15.260715+0000","last_undegraded":"2024-05-22T21:17:15.260715+0000","last_fullsized":"2024-05-22T21:17:15.260715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:36:33.104399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.261897+0000","last_change":"2024-05-22T21:17:14.042534+0000","last_active":"2024-05-22T21:17:15.261897+0000","last_peered":"2024-05-22T21:17:15.261897+0000","last_clean":"2024-05-22T21:17:15.261897+0000","last_became_active":"2024-05-22T21:17:14.040925+0000","last_became_peered":"2024-05-22T21:17:14.040925+0000","last_unstale":"2024-05-22T21:17:15.261897+0000","last_undegraded":"2024-05-22T21:17:15.261897+0000","last_fullsized":"2024-05-22T21:17:15.261897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:26:53.813491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313226+0000","last_change":"2024-05-22T21:17:14.028883+0000","last_active":"2024-05-22T21:17:15.313226+0000","last_peered":"2024-05-22T21:17:15.313226+0000","last_clean":"2024-05-22T21:17:15.313226+0000","last_became_active":"2024-05-22T21:17:14.028677+0000","last_became_peered":"2024-05-22T21:17:14.028677+0000","last_unstale":"2024-05-22T21:17:15.313226+0000","last_undegraded":"2024-05-22T21:17:15.313226+0000","last_fullsized":"2024-05-22T21:17:15.313226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:34:55.708745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019896+0000","last_change":"2024-05-22T21:17:14.024298+0000","last_active":"2024-05-22T21:17:15.019896+0000","last_peered":"2024-05-22T21:17:15.019896+0000","last_clean":"2024-05-22T21:17:15.019896+0000","last_became_active":"2024-05-22T21:17:14.024069+0000","last_became_peered":"2024-05-22T21:17:14.024069+0000","last_unstale":"2024-05-22T21:17:15.019896+0000","last_undegraded":"2024-05-22T21:17:15.019896+0000","last_fullsized":"2024-05-22T21:17:15.019896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:26:53.813491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019571+0000","last_change":"2024-05-22T21:17:14.029490+0000","last_active":"2024-05-22T21:17:15.019571+0000","last_peered":"2024-05-22T21:17:15.019571+0000","last_clean":"2024-05-22T21:17:15.019571+0000","last_became_active":"2024-05-22T21:17:14.029252+0000","last_became_peered":"2024-05-22T21:17:14.029252+0000","last_unstale":"2024-05-22T21:17:15.019571+0000","last_undegraded":"2024-05-22T21:17:15.019571+0000","last_fullsized":"2024-05-22T21:17:15.019571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:24:35.728412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019441+0000","last_change":"2024-05-22T21:17:14.027378+0000","last_active":"2024-05-22T21:17:15.019441+0000","last_peered":"2024-05-22T21:17:15.019441+0000","last_clean":"2024-05-22T21:17:15.019441+0000","last_became_active":"2024-05-22T21:17:14.027133+0000","last_became_peered":"2024-05-22T21:17:14.027133+0000","last_unstale":"2024-05-22T21:17:15.019441+0000","last_undegraded":"2024-05-22T21:17:15.019441+0000","last_fullsized":"2024-05-22T21:17:15.019441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:00:38.089359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019495+0000","last_change":"2024-05-22T21:17:14.044367+0000","last_active":"2024-05-22T21:17:15.019495+0000","last_peered":"2024-05-22T21:17:15.019495+0000","last_clean":"2024-05-22T21:17:15.019495+0000","last_became_active":"2024-05-22T21:17:14.043986+0000","last_became_peered":"2024-05-22T21:17:14.043986+0000","last_unstale":"2024-05-22T21:17:15.019495+0000","last_undegraded":"2024-05-22T21:17:15.019495+0000","last_fullsized":"2024-05-22T21:17:15.019495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:35:21.127439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019731+0000","last_change":"2024-05-22T21:17:14.027691+0000","last_active":"2024-05-22T21:17:15.019731+0000","last_peered":"2024-05-22T21:17:15.019731+0000","last_clean":"2024-05-22T21:17:15.019731+0000","last_became_active":"2024-05-22T21:17:14.026527+0000","last_became_peered":"2024-05-22T21:17:14.026527+0000","last_unstale":"2024-05-22T21:17:15.019731+0000","last_undegraded":"2024-05-22T21:17:15.019731+0000","last_fullsized":"2024-05-22T21:17:15.019731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:20:04.893466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262297+0000","last_change":"2024-05-22T21:17:14.044043+0000","last_active":"2024-05-22T21:17:15.262297+0000","last_peered":"2024-05-22T21:17:15.262297+0000","last_clean":"2024-05-22T21:17:15.262297+0000","last_became_active":"2024-05-22T21:17:14.043738+0000","last_became_peered":"2024-05-22T21:17:14.043738+0000","last_unstale":"2024-05-22T21:17:15.262297+0000","last_undegraded":"2024-05-22T21:17:15.262297+0000","last_fullsized":"2024-05-22T21:17:15.262297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:35:21.127439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313445+0000","last_change":"2024-05-22T21:17:14.022101+0000","last_active":"2024-05-22T21:17:15.313445+0000","last_peered":"2024-05-22T21:17:15.313445+0000","last_clean":"2024-05-22T21:17:15.313445+0000","last_became_active":"2024-05-22T21:17:14.021740+0000","last_became_peered":"2024-05-22T21:17:14.021740+0000","last_unstale":"2024-05-22T21:17:15.313445+0000","last_undegraded":"2024-05-22T21:17:15.313445+0000","last_fullsized":"2024-05-22T21:17:15.313445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:54:46.702287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019195+0000","last_change":"2024-05-22T21:17:14.028752+0000","last_active":"2024-05-22T21:17:15.019195+0000","last_peered":"2024-05-22T21:17:15.019195+0000","last_clean":"2024-05-22T21:17:15.019195+0000","last_became_active":"2024-05-22T21:17:14.028504+0000","last_became_peered":"2024-05-22T21:17:14.028504+0000","last_unstale":"2024-05-22T21:17:15.019195+0000","last_undegraded":"2024-05-22T21:17:15.019195+0000","last_fullsized":"2024-05-22T21:17:15.019195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:55:53.278600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313005+0000","last_change":"2024-05-22T21:17:14.029495+0000","last_active":"2024-05-22T21:17:15.313005+0000","last_peered":"2024-05-22T21:17:15.313005+0000","last_clean":"2024-05-22T21:17:15.313005+0000","last_became_active":"2024-05-22T21:17:14.029251+0000","last_became_peered":"2024-05-22T21:17:14.029251+0000","last_unstale":"2024-05-22T21:17:15.313005+0000","last_undegraded":"2024-05-22T21:17:15.313005+0000","last_fullsized":"2024-05-22T21:17:15.313005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:04:48.892044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262269+0000","last_change":"2024-05-22T21:17:14.041069+0000","last_active":"2024-05-22T21:17:15.262269+0000","last_peered":"2024-05-22T21:17:15.262269+0000","last_clean":"2024-05-22T21:17:15.262269+0000","last_became_active":"2024-05-22T21:17:14.040849+0000","last_became_peered":"2024-05-22T21:17:14.040849+0000","last_unstale":"2024-05-22T21:17:15.262269+0000","last_undegraded":"2024-05-22T21:17:15.262269+0000","last_fullsized":"2024-05-22T21:17:15.262269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:43:31.001610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262403+0000","last_change":"2024-05-22T21:17:14.030554+0000","last_active":"2024-05-22T21:17:15.262403+0000","last_peered":"2024-05-22T21:17:15.262403+0000","last_clean":"2024-05-22T21:17:15.262403+0000","last_became_active":"2024-05-22T21:17:14.030330+0000","last_became_peered":"2024-05-22T21:17:14.030330+0000","last_unstale":"2024-05-22T21:17:15.262403+0000","last_undegraded":"2024-05-22T21:17:15.262403+0000","last_fullsized":"2024-05-22T21:17:15.262403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:12:02.268314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.261924+0000","last_change":"2024-05-22T21:17:14.034879+0000","last_active":"2024-05-22T21:17:15.261924+0000","last_peered":"2024-05-22T21:17:15.261924+0000","last_clean":"2024-05-22T21:17:15.261924+0000","last_became_active":"2024-05-22T21:17:14.034587+0000","last_became_peered":"2024-05-22T21:17:14.034587+0000","last_unstale":"2024-05-22T21:17:15.261924+0000","last_undegraded":"2024-05-22T21:17:15.261924+0000","last_fullsized":"2024-05-22T21:17:15.261924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:02:43.741069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019618+0000","last_change":"2024-05-22T21:17:14.044210+0000","last_active":"2024-05-22T21:17:15.019618+0000","last_peered":"2024-05-22T21:17:15.019618+0000","last_clean":"2024-05-22T21:17:15.019618+0000","last_became_active":"2024-05-22T21:17:14.043559+0000","last_became_peered":"2024-05-22T21:17:14.043559+0000","last_unstale":"2024-05-22T21:17:15.019618+0000","last_undegraded":"2024-05-22T21:17:15.019618+0000","last_fullsized":"2024-05-22T21:17:15.019618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:43:31.001610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"83'4","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019689+0000","last_change":"2024-05-22T21:17:13.647773+0000","last_active":"2024-05-22T21:17:15.019689+0000","last_peered":"2024-05-22T21:17:15.019689+0000","last_clean":"2024-05-22T21:17:15.019689+0000","last_became_active":"2024-05-22T21:17:13.646391+0000","last_became_peered":"2024-05-22T21:17:13.646391+0000","last_unstale":"2024-05-22T21:17:15.019689+0000","last_undegraded":"2024-05-22T21:17:15.019689+0000","last_fullsized":"2024-05-22T21:17:15.019689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:26:52.811237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.017939+0000","last_change":"2024-05-22T21:17:14.024552+0000","last_active":"2024-05-22T21:17:15.017939+0000","last_peered":"2024-05-22T21:17:15.017939+0000","last_clean":"2024-05-22T21:17:15.017939+0000","last_became_active":"2024-05-22T21:17:14.024318+0000","last_became_peered":"2024-05-22T21:17:14.024318+0000","last_unstale":"2024-05-22T21:17:15.017939+0000","last_undegraded":"2024-05-22T21:17:15.017939+0000","last_fullsized":"2024-05-22T21:17:15.017939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:48:14.161903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"83'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.312998+0000","last_change":"2024-05-22T21:17:13.639194+0000","last_active":"2024-05-22T21:17:15.312998+0000","last_peered":"2024-05-22T21:17:15.312998+0000","last_clean":"2024-05-22T21:17:15.312998+0000","last_became_active":"2024-05-22T21:17:13.638897+0000","last_became_peered":"2024-05-22T21:17:13.638897+0000","last_unstale":"2024-05-22T21:17:15.312998+0000","last_undegraded":"2024-05-22T21:17:15.312998+0000","last_fullsized":"2024-05-22T21:17:15.312998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:26:16.724198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262251+0000","last_change":"2024-05-22T21:17:14.039069+0000","last_active":"2024-05-22T21:17:15.262251+0000","last_peered":"2024-05-22T21:17:15.262251+0000","last_clean":"2024-05-22T21:17:15.262251+0000","last_became_active":"2024-05-22T21:17:14.038576+0000","last_became_peered":"2024-05-22T21:17:14.038576+0000","last_unstale":"2024-05-22T21:17:15.262251+0000","last_undegraded":"2024-05-22T21:17:15.262251+0000","last_fullsized":"2024-05-22T21:17:15.262251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:20:04.893466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"83'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019975+0000","last_change":"2024-05-22T21:17:13.649420+0000","last_active":"2024-05-22T21:17:15.019975+0000","last_peered":"2024-05-22T21:17:15.019975+0000","last_clean":"2024-05-22T21:17:15.019975+0000","last_became_active":"2024-05-22T21:17:13.648844+0000","last_became_peered":"2024-05-22T21:17:13.648844+0000","last_unstale":"2024-05-22T21:17:15.019975+0000","last_undegraded":"2024-05-22T21:17:15.019975+0000","last_fullsized":"2024-05-22T21:17:15.019975+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:10:59.730979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.020293+0000","last_change":"2024-05-22T21:17:14.024385+0000","last_active":"2024-05-22T21:17:15.020293+0000","last_peered":"2024-05-22T21:17:15.020293+0000","last_clean":"2024-05-22T21:17:15.020293+0000","last_became_active":"2024-05-22T21:17:14.024084+0000","last_became_peered":"2024-05-22T21:17:14.024084+0000","last_unstale":"2024-05-22T21:17:15.020293+0000","last_undegraded":"2024-05-22T21:17:15.020293+0000","last_fullsized":"2024-05-22T21:17:15.020293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:06:06.207443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"83'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262484+0000","last_change":"2024-05-22T21:17:13.649399+0000","last_active":"2024-05-22T21:17:15.262484+0000","last_peered":"2024-05-22T21:17:15.262484+0000","last_clean":"2024-05-22T21:17:15.262484+0000","last_became_active":"2024-05-22T21:17:13.646416+0000","last_became_peered":"2024-05-22T21:17:13.646416+0000","last_unstale":"2024-05-22T21:17:15.262484+0000","last_undegraded":"2024-05-22T21:17:15.262484+0000","last_fullsized":"2024-05-22T21:17:15.262484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:05:13.084939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.261610+0000","last_change":"2024-05-22T21:17:14.030211+0000","last_active":"2024-05-22T21:17:15.261610+0000","last_peered":"2024-05-22T21:17:15.261610+0000","last_clean":"2024-05-22T21:17:15.261610+0000","last_became_active":"2024-05-22T21:17:14.029934+0000","last_became_peered":"2024-05-22T21:17:14.029934+0000","last_unstale":"2024-05-22T21:17:15.261610+0000","last_undegraded":"2024-05-22T21:17:15.261610+0000","last_fullsized":"2024-05-22T21:17:15.261610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:25:04.470055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"83'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313599+0000","last_change":"2024-05-22T21:17:13.644930+0000","last_active":"2024-05-22T21:17:15.313599+0000","last_peered":"2024-05-22T21:17:15.313599+0000","last_clean":"2024-05-22T21:17:15.313599+0000","last_became_active":"2024-05-22T21:17:13.644704+0000","last_became_peered":"2024-05-22T21:17:13.644704+0000","last_unstale":"2024-05-22T21:17:15.313599+0000","last_undegraded":"2024-05-22T21:17:15.313599+0000","last_fullsized":"2024-05-22T21:17:15.313599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:54:14.183878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019678+0000","last_change":"2024-05-22T21:17:14.028054+0000","last_active":"2024-05-22T21:17:15.019678+0000","last_peered":"2024-05-22T21:17:15.019678+0000","last_clean":"2024-05-22T21:17:15.019678+0000","last_became_active":"2024-05-22T21:17:14.027822+0000","last_became_peered":"2024-05-22T21:17:14.027822+0000","last_unstale":"2024-05-22T21:17:15.019678+0000","last_undegraded":"2024-05-22T21:17:15.019678+0000","last_fullsized":"2024-05-22T21:17:15.019678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:18:40.044538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"83'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262257+0000","last_change":"2024-05-22T21:17:13.643736+0000","last_active":"2024-05-22T21:17:15.262257+0000","last_peered":"2024-05-22T21:17:15.262257+0000","last_clean":"2024-05-22T21:17:15.262257+0000","last_became_active":"2024-05-22T21:17:13.643247+0000","last_became_peered":"2024-05-22T21:17:13.643247+0000","last_unstale":"2024-05-22T21:17:15.262257+0000","last_undegraded":"2024-05-22T21:17:15.262257+0000","last_fullsized":"2024-05-22T21:17:15.262257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:32:43.878695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018254+0000","last_change":"2024-05-22T21:17:14.027758+0000","last_active":"2024-05-22T21:17:15.018254+0000","last_peered":"2024-05-22T21:17:15.018254+0000","last_clean":"2024-05-22T21:17:15.018254+0000","last_became_active":"2024-05-22T21:17:14.027423+0000","last_became_peered":"2024-05-22T21:17:14.027423+0000","last_unstale":"2024-05-22T21:17:15.018254+0000","last_undegraded":"2024-05-22T21:17:15.018254+0000","last_fullsized":"2024-05-22T21:17:15.018254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:33:19.956193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260658+0000","last_change":"2024-05-22T21:17:13.643443+0000","last_active":"2024-05-22T21:17:15.260658+0000","last_peered":"2024-05-22T21:17:15.260658+0000","last_clean":"2024-05-22T21:17:15.260658+0000","last_became_active":"2024-05-22T21:17:13.643218+0000","last_became_peered":"2024-05-22T21:17:13.643218+0000","last_unstale":"2024-05-22T21:17:15.260658+0000","last_undegraded":"2024-05-22T21:17:15.260658+0000","last_fullsized":"2024-05-22T21:17:15.260658+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:58:34.189664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.312912+0000","last_change":"2024-05-22T21:17:14.024363+0000","last_active":"2024-05-22T21:17:15.312912+0000","last_peered":"2024-05-22T21:17:15.312912+0000","last_clean":"2024-05-22T21:17:15.312912+0000","last_became_active":"2024-05-22T21:17:14.024159+0000","last_became_peered":"2024-05-22T21:17:14.024159+0000","last_unstale":"2024-05-22T21:17:15.312912+0000","last_undegraded":"2024-05-22T21:17:15.312912+0000","last_fullsized":"2024-05-22T21:17:15.312912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:35:21.127439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018322+0000","last_change":"2024-05-22T21:17:13.642883+0000","last_active":"2024-05-22T21:17:15.018322+0000","last_peered":"2024-05-22T21:17:15.018322+0000","last_clean":"2024-05-22T21:17:15.018322+0000","last_became_active":"2024-05-22T21:17:13.642639+0000","last_became_peered":"2024-05-22T21:17:13.642639+0000","last_unstale":"2024-05-22T21:17:15.018322+0000","last_undegraded":"2024-05-22T21:17:15.018322+0000","last_fullsized":"2024-05-22T21:17:15.018322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:21:45.294614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262517+0000","last_change":"2024-05-22T21:17:14.033375+0000","last_active":"2024-05-22T21:17:15.262517+0000","last_peered":"2024-05-22T21:17:15.262517+0000","last_clean":"2024-05-22T21:17:15.262517+0000","last_became_active":"2024-05-22T21:17:14.033147+0000","last_became_peered":"2024-05-22T21:17:14.033147+0000","last_unstale":"2024-05-22T21:17:15.262517+0000","last_undegraded":"2024-05-22T21:17:15.262517+0000","last_fullsized":"2024-05-22T21:17:15.262517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:04.256203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.020161+0000","last_change":"2024-05-22T21:17:13.644117+0000","last_active":"2024-05-22T21:17:15.020161+0000","last_peered":"2024-05-22T21:17:15.020161+0000","last_clean":"2024-05-22T21:17:15.020161+0000","last_became_active":"2024-05-22T21:17:13.643843+0000","last_became_peered":"2024-05-22T21:17:13.643843+0000","last_unstale":"2024-05-22T21:17:15.020161+0000","last_undegraded":"2024-05-22T21:17:15.020161+0000","last_fullsized":"2024-05-22T21:17:15.020161+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:22:04.077280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.020178+0000","last_change":"2024-05-22T21:17:14.039306+0000","last_active":"2024-05-22T21:17:15.020178+0000","last_peered":"2024-05-22T21:17:15.020178+0000","last_clean":"2024-05-22T21:17:15.020178+0000","last_became_active":"2024-05-22T21:17:14.039084+0000","last_became_peered":"2024-05-22T21:17:14.039084+0000","last_unstale":"2024-05-22T21:17:15.020178+0000","last_undegraded":"2024-05-22T21:17:15.020178+0000","last_fullsized":"2024-05-22T21:17:15.020178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:22:47.130444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262067+0000","last_change":"2024-05-22T21:17:13.646634+0000","last_active":"2024-05-22T21:17:15.262067+0000","last_peered":"2024-05-22T21:17:15.262067+0000","last_clean":"2024-05-22T21:17:15.262067+0000","last_became_active":"2024-05-22T21:17:13.645876+0000","last_became_peered":"2024-05-22T21:17:13.645876+0000","last_unstale":"2024-05-22T21:17:15.262067+0000","last_undegraded":"2024-05-22T21:17:15.262067+0000","last_fullsized":"2024-05-22T21:17:15.262067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:37:06.289852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262021+0000","last_change":"2024-05-22T21:17:14.041272+0000","last_active":"2024-05-22T21:17:15.262021+0000","last_peered":"2024-05-22T21:17:15.262021+0000","last_clean":"2024-05-22T21:17:15.262021+0000","last_became_active":"2024-05-22T21:17:14.040666+0000","last_became_peered":"2024-05-22T21:17:14.040666+0000","last_unstale":"2024-05-22T21:17:15.262021+0000","last_undegraded":"2024-05-22T21:17:15.262021+0000","last_fullsized":"2024-05-22T21:17:15.262021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:21:32.020991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"83'3","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313195+0000","last_change":"2024-05-22T21:17:13.645672+0000","last_active":"2024-05-22T21:17:15.313195+0000","last_peered":"2024-05-22T21:17:15.313195+0000","last_clean":"2024-05-22T21:17:15.313195+0000","last_became_active":"2024-05-22T21:17:13.645438+0000","last_became_peered":"2024-05-22T21:17:13.645438+0000","last_unstale":"2024-05-22T21:17:15.313195+0000","last_undegraded":"2024-05-22T21:17:15.313195+0000","last_fullsized":"2024-05-22T21:17:15.313195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:12:01.266061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018036+0000","last_change":"2024-05-22T21:17:14.026726+0000","last_active":"2024-05-22T21:17:15.018036+0000","last_peered":"2024-05-22T21:17:15.018036+0000","last_clean":"2024-05-22T21:17:15.018036+0000","last_became_active":"2024-05-22T21:17:14.026328+0000","last_became_peered":"2024-05-22T21:17:14.026328+0000","last_unstale":"2024-05-22T21:17:15.018036+0000","last_undegraded":"2024-05-22T21:17:15.018036+0000","last_fullsized":"2024-05-22T21:17:15.018036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:52:13.416295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"83'6","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260747+0000","last_change":"2024-05-22T21:17:13.636869+0000","last_active":"2024-05-22T21:17:15.260747+0000","last_peered":"2024-05-22T21:17:15.260747+0000","last_clean":"2024-05-22T21:17:15.260747+0000","last_became_active":"2024-05-22T21:17:13.636611+0000","last_became_peered":"2024-05-22T21:17:13.636611+0000","last_unstale":"2024-05-22T21:17:15.260747+0000","last_undegraded":"2024-05-22T21:17:15.260747+0000","last_fullsized":"2024-05-22T21:17:15.260747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:32:39.395686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.020271+0000","last_change":"2024-05-22T21:17:14.031490+0000","last_active":"2024-05-22T21:17:15.020271+0000","last_peered":"2024-05-22T21:17:15.020271+0000","last_clean":"2024-05-22T21:17:15.020271+0000","last_became_active":"2024-05-22T21:17:14.030957+0000","last_became_peered":"2024-05-22T21:17:14.030957+0000","last_unstale":"2024-05-22T21:17:15.020271+0000","last_undegraded":"2024-05-22T21:17:15.020271+0000","last_fullsized":"2024-05-22T21:17:15.020271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:54:15.186131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"83'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260673+0000","last_change":"2024-05-22T21:17:13.639375+0000","last_active":"2024-05-22T21:17:15.260673+0000","last_peered":"2024-05-22T21:17:15.260673+0000","last_clean":"2024-05-22T21:17:15.260673+0000","last_became_active":"2024-05-22T21:17:13.636907+0000","last_became_peered":"2024-05-22T21:17:13.636907+0000","last_unstale":"2024-05-22T21:17:15.260673+0000","last_undegraded":"2024-05-22T21:17:15.260673+0000","last_fullsized":"2024-05-22T21:17:15.260673+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:40:36.202431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262368+0000","last_change":"2024-05-22T21:17:14.028881+0000","last_active":"2024-05-22T21:17:15.262368+0000","last_peered":"2024-05-22T21:17:15.262368+0000","last_clean":"2024-05-22T21:17:15.262368+0000","last_became_active":"2024-05-22T21:17:14.028648+0000","last_became_peered":"2024-05-22T21:17:14.028648+0000","last_unstale":"2024-05-22T21:17:15.262368+0000","last_undegraded":"2024-05-22T21:17:15.262368+0000","last_fullsized":"2024-05-22T21:17:15.262368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:34:55.708745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262571+0000","last_change":"2024-05-22T21:17:13.649073+0000","last_active":"2024-05-22T21:17:15.262571+0000","last_peered":"2024-05-22T21:17:15.262571+0000","last_clean":"2024-05-22T21:17:15.262571+0000","last_became_active":"2024-05-22T21:17:13.645401+0000","last_became_peered":"2024-05-22T21:17:13.645401+0000","last_unstale":"2024-05-22T21:17:15.262571+0000","last_undegraded":"2024-05-22T21:17:15.262571+0000","last_fullsized":"2024-05-22T21:17:15.262571+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:58:34.189664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313114+0000","last_change":"2024-05-22T21:17:14.037570+0000","last_active":"2024-05-22T21:17:15.313114+0000","last_peered":"2024-05-22T21:17:15.313114+0000","last_clean":"2024-05-22T21:17:15.313114+0000","last_became_active":"2024-05-22T21:17:14.037346+0000","last_became_peered":"2024-05-22T21:17:14.037346+0000","last_unstale":"2024-05-22T21:17:15.313114+0000","last_undegraded":"2024-05-22T21:17:15.313114+0000","last_fullsized":"2024-05-22T21:17:15.313114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:54:15.186131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262625+0000","last_change":"2024-05-22T21:17:13.642468+0000","last_active":"2024-05-22T21:17:15.262625+0000","last_peered":"2024-05-22T21:17:15.262625+0000","last_clean":"2024-05-22T21:17:15.262625+0000","last_became_active":"2024-05-22T21:17:13.642235+0000","last_became_peered":"2024-05-22T21:17:13.642235+0000","last_unstale":"2024-05-22T21:17:15.262625+0000","last_undegraded":"2024-05-22T21:17:15.262625+0000","last_fullsized":"2024-05-22T21:17:15.262625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:20:03.891212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018146+0000","last_change":"2024-05-22T21:17:14.026299+0000","last_active":"2024-05-22T21:17:15.018146+0000","last_peered":"2024-05-22T21:17:15.018146+0000","last_clean":"2024-05-22T21:17:15.018146+0000","last_became_active":"2024-05-22T21:17:14.025759+0000","last_became_peered":"2024-05-22T21:17:14.025759+0000","last_unstale":"2024-05-22T21:17:15.018146+0000","last_undegraded":"2024-05-22T21:17:15.018146+0000","last_fullsized":"2024-05-22T21:17:15.018146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:39:05.614355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"83'8","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019292+0000","last_change":"2024-05-22T21:17:13.648532+0000","last_active":"2024-05-22T21:17:15.019292+0000","last_peered":"2024-05-22T21:17:15.019292+0000","last_clean":"2024-05-22T21:17:15.019292+0000","last_became_active":"2024-05-22T21:17:13.648237+0000","last_became_peered":"2024-05-22T21:17:13.648237+0000","last_unstale":"2024-05-22T21:17:15.019292+0000","last_undegraded":"2024-05-22T21:17:15.019292+0000","last_fullsized":"2024-05-22T21:17:15.019292+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:49:05.183946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019826+0000","last_change":"2024-05-22T21:17:14.028174+0000","last_active":"2024-05-22T21:17:15.019826+0000","last_peered":"2024-05-22T21:17:15.019826+0000","last_clean":"2024-05-22T21:17:15.019826+0000","last_became_active":"2024-05-22T21:17:14.027936+0000","last_became_peered":"2024-05-22T21:17:14.027936+0000","last_unstale":"2024-05-22T21:17:15.019826+0000","last_undegraded":"2024-05-22T21:17:15.019826+0000","last_fullsized":"2024-05-22T21:17:15.019826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:15.861947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"83'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262163+0000","last_change":"2024-05-22T21:17:13.647165+0000","last_active":"2024-05-22T21:17:15.262163+0000","last_peered":"2024-05-22T21:17:15.262163+0000","last_clean":"2024-05-22T21:17:15.262163+0000","last_became_active":"2024-05-22T21:17:13.645068+0000","last_became_peered":"2024-05-22T21:17:13.645068+0000","last_unstale":"2024-05-22T21:17:15.262163+0000","last_undegraded":"2024-05-22T21:17:15.262163+0000","last_fullsized":"2024-05-22T21:17:15.262163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:49:05.183946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019713+0000","last_change":"2024-05-22T21:17:14.026751+0000","last_active":"2024-05-22T21:17:15.019713+0000","last_peered":"2024-05-22T21:17:15.019713+0000","last_clean":"2024-05-22T21:17:15.019713+0000","last_became_active":"2024-05-22T21:17:14.026512+0000","last_became_peered":"2024-05-22T21:17:14.026512+0000","last_unstale":"2024-05-22T21:17:15.019713+0000","last_undegraded":"2024-05-22T21:17:15.019713+0000","last_fullsized":"2024-05-22T21:17:15.019713+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:25:04.470055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"83'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.020066+0000","last_change":"2024-05-22T21:17:13.649244+0000","last_active":"2024-05-22T21:17:15.020066+0000","last_peered":"2024-05-22T21:17:15.020066+0000","last_clean":"2024-05-22T21:17:15.020066+0000","last_became_active":"2024-05-22T21:17:13.648443+0000","last_became_peered":"2024-05-22T21:17:13.648443+0000","last_unstale":"2024-05-22T21:17:15.020066+0000","last_undegraded":"2024-05-22T21:17:15.020066+0000","last_fullsized":"2024-05-22T21:17:15.020066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:36:23.129996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260721+0000","last_change":"2024-05-22T21:17:14.028212+0000","last_active":"2024-05-22T21:17:15.260721+0000","last_peered":"2024-05-22T21:17:15.260721+0000","last_clean":"2024-05-22T21:17:15.260721+0000","last_became_active":"2024-05-22T21:17:14.026676+0000","last_became_peered":"2024-05-22T21:17:14.026676+0000","last_unstale":"2024-05-22T21:17:15.260721+0000","last_undegraded":"2024-05-22T21:17:15.260721+0000","last_fullsized":"2024-05-22T21:17:15.260721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:11:00.733233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"83'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262730+0000","last_change":"2024-05-22T21:17:13.649234+0000","last_active":"2024-05-22T21:17:15.262730+0000","last_peered":"2024-05-22T21:17:15.262730+0000","last_clean":"2024-05-22T21:17:15.262730+0000","last_became_active":"2024-05-22T21:17:13.646019+0000","last_became_peered":"2024-05-22T21:17:13.646019+0000","last_unstale":"2024-05-22T21:17:15.262730+0000","last_undegraded":"2024-05-22T21:17:15.262730+0000","last_fullsized":"2024-05-22T21:17:15.262730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:02:18.597049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018112+0000","last_change":"2024-05-22T21:17:14.023539+0000","last_active":"2024-05-22T21:17:15.018112+0000","last_peered":"2024-05-22T21:17:15.018112+0000","last_clean":"2024-05-22T21:17:15.018112+0000","last_became_active":"2024-05-22T21:17:14.023314+0000","last_became_peered":"2024-05-22T21:17:14.023314+0000","last_unstale":"2024-05-22T21:17:15.018112+0000","last_undegraded":"2024-05-22T21:17:15.018112+0000","last_fullsized":"2024-05-22T21:17:15.018112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:18:11.114230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019510+0000","last_change":"2024-05-22T21:17:13.644389+0000","last_active":"2024-05-22T21:17:15.019510+0000","last_peered":"2024-05-22T21:17:15.019510+0000","last_clean":"2024-05-22T21:17:15.019510+0000","last_became_active":"2024-05-22T21:17:13.644121+0000","last_became_peered":"2024-05-22T21:17:13.644121+0000","last_unstale":"2024-05-22T21:17:15.019510+0000","last_undegraded":"2024-05-22T21:17:15.019510+0000","last_fullsized":"2024-05-22T21:17:15.019510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:24:34.726158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313106+0000","last_change":"2024-05-22T21:17:14.023365+0000","last_active":"2024-05-22T21:17:15.313106+0000","last_peered":"2024-05-22T21:17:15.313106+0000","last_clean":"2024-05-22T21:17:15.313106+0000","last_became_active":"2024-05-22T21:17:14.023004+0000","last_became_peered":"2024-05-22T21:17:14.023004+0000","last_unstale":"2024-05-22T21:17:15.313106+0000","last_undegraded":"2024-05-22T21:17:15.313106+0000","last_fullsized":"2024-05-22T21:17:15.313106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:48:54.143205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262266+0000","last_change":"2024-05-22T21:17:13.645322+0000","last_active":"2024-05-22T21:17:15.262266+0000","last_peered":"2024-05-22T21:17:15.262266+0000","last_clean":"2024-05-22T21:17:15.262266+0000","last_became_active":"2024-05-22T21:17:13.644703+0000","last_became_peered":"2024-05-22T21:17:13.644703+0000","last_unstale":"2024-05-22T21:17:15.262266+0000","last_undegraded":"2024-05-22T21:17:15.262266+0000","last_fullsized":"2024-05-22T21:17:15.262266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:36:23.129996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313092+0000","last_change":"2024-05-22T21:17:14.039243+0000","last_active":"2024-05-22T21:17:15.313092+0000","last_peered":"2024-05-22T21:17:15.313092+0000","last_clean":"2024-05-22T21:17:15.313092+0000","last_became_active":"2024-05-22T21:17:14.039060+0000","last_became_peered":"2024-05-22T21:17:14.039060+0000","last_unstale":"2024-05-22T21:17:15.313092+0000","last_undegraded":"2024-05-22T21:17:15.313092+0000","last_fullsized":"2024-05-22T21:17:15.313092+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:48:29.806509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262456+0000","last_change":"2024-05-22T21:17:13.644094+0000","last_active":"2024-05-22T21:17:15.262456+0000","last_peered":"2024-05-22T21:17:15.262456+0000","last_clean":"2024-05-22T21:17:15.262456+0000","last_became_active":"2024-05-22T21:17:13.643877+0000","last_became_peered":"2024-05-22T21:17:13.643877+0000","last_unstale":"2024-05-22T21:17:15.262456+0000","last_undegraded":"2024-05-22T21:17:15.262456+0000","last_fullsized":"2024-05-22T21:17:15.262456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:18:39.042285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.312946+0000","last_change":"2024-05-22T21:17:14.040825+0000","last_active":"2024-05-22T21:17:15.312946+0000","last_peered":"2024-05-22T21:17:15.312946+0000","last_clean":"2024-05-22T21:17:15.312946+0000","last_became_active":"2024-05-22T21:17:14.040552+0000","last_became_peered":"2024-05-22T21:17:14.040552+0000","last_unstale":"2024-05-22T21:17:15.312946+0000","last_undegraded":"2024-05-22T21:17:15.312946+0000","last_fullsized":"2024-05-22T21:17:15.312946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:02:38.871316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313168+0000","last_change":"2024-05-22T21:17:13.638951+0000","last_active":"2024-05-22T21:17:15.313168+0000","last_peered":"2024-05-22T21:17:15.313168+0000","last_clean":"2024-05-22T21:17:15.313168+0000","last_became_active":"2024-05-22T21:17:13.638709+0000","last_became_peered":"2024-05-22T21:17:13.638709+0000","last_unstale":"2024-05-22T21:17:15.313168+0000","last_undegraded":"2024-05-22T21:17:15.313168+0000","last_fullsized":"2024-05-22T21:17:15.313168+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:42:39.440826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313179+0000","last_change":"2024-05-22T21:17:14.025279+0000","last_active":"2024-05-22T21:17:15.313179+0000","last_peered":"2024-05-22T21:17:15.313179+0000","last_clean":"2024-05-22T21:17:15.313179+0000","last_became_active":"2024-05-22T21:17:14.025009+0000","last_became_peered":"2024-05-22T21:17:14.025009+0000","last_unstale":"2024-05-22T21:17:15.313179+0000","last_undegraded":"2024-05-22T21:17:15.313179+0000","last_fullsized":"2024-05-22T21:17:15.313179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:43:31.001610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"83'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262172+0000","last_change":"2024-05-22T21:17:13.646475+0000","last_active":"2024-05-22T21:17:15.262172+0000","last_peered":"2024-05-22T21:17:15.262172+0000","last_clean":"2024-05-22T21:17:15.262172+0000","last_became_active":"2024-05-22T21:17:13.645525+0000","last_became_peered":"2024-05-22T21:17:13.645525+0000","last_unstale":"2024-05-22T21:17:15.262172+0000","last_undegraded":"2024-05-22T21:17:15.262172+0000","last_fullsized":"2024-05-22T21:17:15.262172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:32:33.619997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262582+0000","last_change":"2024-05-22T21:17:14.030583+0000","last_active":"2024-05-22T21:17:15.262582+0000","last_peered":"2024-05-22T21:17:15.262582+0000","last_clean":"2024-05-22T21:17:15.262582+0000","last_became_active":"2024-05-22T21:17:14.030372+0000","last_became_peered":"2024-05-22T21:17:14.030372+0000","last_unstale":"2024-05-22T21:17:15.262582+0000","last_undegraded":"2024-05-22T21:17:15.262582+0000","last_fullsized":"2024-05-22T21:17:15.262582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:54:15.186131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262201+0000","last_change":"2024-05-22T21:17:13.643297+0000","last_active":"2024-05-22T21:17:15.262201+0000","last_peered":"2024-05-22T21:17:15.262201+0000","last_clean":"2024-05-22T21:17:15.262201+0000","last_became_active":"2024-05-22T21:17:13.643070+0000","last_became_peered":"2024-05-22T21:17:13.643070+0000","last_unstale":"2024-05-22T21:17:15.262201+0000","last_undegraded":"2024-05-22T21:17:15.262201+0000","last_fullsized":"2024-05-22T21:17:15.262201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:10:55.728981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.020362+0000","last_change":"2024-05-22T21:17:14.041268+0000","last_active":"2024-05-22T21:17:15.020362+0000","last_peered":"2024-05-22T21:17:15.020362+0000","last_clean":"2024-05-22T21:17:15.020362+0000","last_became_active":"2024-05-22T21:17:14.040272+0000","last_became_peered":"2024-05-22T21:17:14.040272+0000","last_unstale":"2024-05-22T21:17:15.020362+0000","last_undegraded":"2024-05-22T21:17:15.020362+0000","last_fullsized":"2024-05-22T21:17:15.020362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:34:55.708745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018147+0000","last_change":"2024-05-22T21:17:13.642573+0000","last_active":"2024-05-22T21:17:15.018147+0000","last_peered":"2024-05-22T21:17:15.018147+0000","last_clean":"2024-05-22T21:17:15.018147+0000","last_became_active":"2024-05-22T21:17:13.642199+0000","last_became_peered":"2024-05-22T21:17:13.642199+0000","last_unstale":"2024-05-22T21:17:15.018147+0000","last_undegraded":"2024-05-22T21:17:15.018147+0000","last_fullsized":"2024-05-22T21:17:15.018147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:51:22.313810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019803+0000","last_change":"2024-05-22T21:17:14.029380+0000","last_active":"2024-05-22T21:17:15.019803+0000","last_peered":"2024-05-22T21:17:15.019803+0000","last_clean":"2024-05-22T21:17:15.019803+0000","last_became_active":"2024-05-22T21:17:14.029161+0000","last_became_peered":"2024-05-22T21:17:14.029161+0000","last_unstale":"2024-05-22T21:17:15.019803+0000","last_undegraded":"2024-05-22T21:17:15.019803+0000","last_fullsized":"2024-05-22T21:17:15.019803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:04.256203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313265+0000","last_change":"2024-05-22T21:17:13.643665+0000","last_active":"2024-05-22T21:17:15.313265+0000","last_peered":"2024-05-22T21:17:15.313265+0000","last_clean":"2024-05-22T21:17:15.313265+0000","last_became_active":"2024-05-22T21:17:13.643027+0000","last_became_peered":"2024-05-22T21:17:13.643027+0000","last_unstale":"2024-05-22T21:17:15.313265+0000","last_undegraded":"2024-05-22T21:17:15.313265+0000","last_fullsized":"2024-05-22T21:17:15.313265+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:43:27.468865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018229+0000","last_change":"2024-05-22T21:17:14.026178+0000","last_active":"2024-05-22T21:17:15.018229+0000","last_peered":"2024-05-22T21:17:15.018229+0000","last_clean":"2024-05-22T21:17:15.018229+0000","last_became_active":"2024-05-22T21:17:14.025471+0000","last_became_peered":"2024-05-22T21:17:14.025471+0000","last_unstale":"2024-05-22T21:17:15.018229+0000","last_undegraded":"2024-05-22T21:17:15.018229+0000","last_fullsized":"2024-05-22T21:17:15.018229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:10:33.711742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.018189+0000","last_change":"2024-05-22T21:17:13.641901+0000","last_active":"2024-05-22T21:17:15.018189+0000","last_peered":"2024-05-22T21:17:15.018189+0000","last_clean":"2024-05-22T21:17:15.018189+0000","last_became_active":"2024-05-22T21:17:13.640915+0000","last_became_peered":"2024-05-22T21:17:13.640915+0000","last_unstale":"2024-05-22T21:17:15.018189+0000","last_undegraded":"2024-05-22T21:17:15.018189+0000","last_fullsized":"2024-05-22T21:17:15.018189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:49:46.314012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262376+0000","last_change":"2024-05-22T21:17:14.038923+0000","last_active":"2024-05-22T21:17:15.262376+0000","last_peered":"2024-05-22T21:17:15.262376+0000","last_clean":"2024-05-22T21:17:15.262376+0000","last_became_active":"2024-05-22T21:17:14.038297+0000","last_became_peered":"2024-05-22T21:17:14.038297+0000","last_unstale":"2024-05-22T21:17:15.262376+0000","last_undegraded":"2024-05-22T21:17:15.262376+0000","last_fullsized":"2024-05-22T21:17:15.262376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:18:40.044538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260765+0000","last_change":"2024-05-22T21:17:13.644060+0000","last_active":"2024-05-22T21:17:15.260765+0000","last_peered":"2024-05-22T21:17:15.260765+0000","last_clean":"2024-05-22T21:17:15.260765+0000","last_became_active":"2024-05-22T21:17:13.643827+0000","last_became_peered":"2024-05-22T21:17:13.643827+0000","last_unstale":"2024-05-22T21:17:15.260765+0000","last_undegraded":"2024-05-22T21:17:15.260765+0000","last_fullsized":"2024-05-22T21:17:15.260765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:32:33.619997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.260612+0000","last_change":"2024-05-22T21:17:14.025635+0000","last_active":"2024-05-22T21:17:15.260612+0000","last_peered":"2024-05-22T21:17:15.260612+0000","last_clean":"2024-05-22T21:17:15.260612+0000","last_became_active":"2024-05-22T21:17:14.025421+0000","last_became_peered":"2024-05-22T21:17:14.025421+0000","last_unstale":"2024-05-22T21:17:15.260612+0000","last_undegraded":"2024-05-22T21:17:15.260612+0000","last_fullsized":"2024-05-22T21:17:15.260612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:54:38.421179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019376+0000","last_change":"2024-05-22T21:17:13.647412+0000","last_active":"2024-05-22T21:17:15.019376+0000","last_peered":"2024-05-22T21:17:15.019376+0000","last_clean":"2024-05-22T21:17:15.019376+0000","last_became_active":"2024-05-22T21:17:13.647139+0000","last_became_peered":"2024-05-22T21:17:13.647139+0000","last_unstale":"2024-05-22T21:17:15.019376+0000","last_undegraded":"2024-05-22T21:17:15.019376+0000","last_fullsized":"2024-05-22T21:17:15.019376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:02:18.597049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262147+0000","last_change":"2024-05-22T21:17:14.041414+0000","last_active":"2024-05-22T21:17:15.262147+0000","last_peered":"2024-05-22T21:17:15.262147+0000","last_clean":"2024-05-22T21:17:15.262147+0000","last_became_active":"2024-05-22T21:17:14.039111+0000","last_became_peered":"2024-05-22T21:17:14.039111+0000","last_unstale":"2024-05-22T21:17:15.262147+0000","last_undegraded":"2024-05-22T21:17:15.262147+0000","last_fullsized":"2024-05-22T21:17:15.262147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:55.777237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262362+0000","last_change":"2024-05-22T21:17:13.647319+0000","last_active":"2024-05-22T21:17:15.262362+0000","last_peered":"2024-05-22T21:17:15.262362+0000","last_clean":"2024-05-22T21:17:15.262362+0000","last_became_active":"2024-05-22T21:17:13.645880+0000","last_became_peered":"2024-05-22T21:17:13.645880+0000","last_unstale":"2024-05-22T21:17:15.262362+0000","last_undegraded":"2024-05-22T21:17:15.262362+0000","last_fullsized":"2024-05-22T21:17:15.262362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:40:36.202431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.313105+0000","last_change":"2024-05-22T21:17:14.041401+0000","last_active":"2024-05-22T21:17:15.313105+0000","last_peered":"2024-05-22T21:17:15.313105+0000","last_clean":"2024-05-22T21:17:15.313105+0000","last_became_active":"2024-05-22T21:17:14.041237+0000","last_became_peered":"2024-05-22T21:17:14.041237+0000","last_unstale":"2024-05-22T21:17:15.313105+0000","last_undegraded":"2024-05-22T21:17:15.313105+0000","last_fullsized":"2024-05-22T21:17:15.313105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:13.627012+0000","last_clean_scrub_stamp":"2024-05-22T21:17:13.627012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:36:24.027210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"83'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.262463+0000","last_change":"2024-05-22T21:17:13.646851+0000","last_active":"2024-05-22T21:17:15.262463+0000","last_peered":"2024-05-22T21:17:15.262463+0000","last_clean":"2024-05-22T21:17:15.262463+0000","last_became_active":"2024-05-22T21:17:13.646363+0000","last_became_peered":"2024-05-22T21:17:13.646363+0000","last_unstale":"2024-05-22T21:17:15.262463+0000","last_undegraded":"2024-05-22T21:17:15.262463+0000","last_fullsized":"2024-05-22T21:17:15.262463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:17:12.624759+0000","last_clean_scrub_stamp":"2024-05-22T21:17:12.624759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:32:39.395686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":229,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-22T21:17:15.019893+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:17:15.019893+0000","last_peered":"2024-05-22T21:17:15.019893+0000","last_clean":"2024-05-22T21:17:15.019893+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:17:15.019893+0000","last_undegraded":"2024-05-22T21:17:15.019893+0000","last_fullsized":"2024-05-22T21:17:15.019893+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1011,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194300,"statfs":{"total":200000000,"available":198964062,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.73499999999999999,"15min":0.76500000000000001},"min":{"1min":0.46100000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85599999999999998,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.48599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93300000000000005,"5min":1.889,"15min":1.889},"last":0.82699999999999996}]},{"osd":1,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.76200000000000001,"15min":0.78200000000000003},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0840000000000001,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73599999999999999,"15min":0.76700000000000002},"min":{"1min":0.40300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88400000000000001,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.56299999999999994,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94999999999999996,"5min":1.923,"15min":1.923},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72599999999999998,"15min":0.72099999999999997},"min":{"1min":0.61599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0409999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.90400000000000003}]},{"osd":3,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69399999999999995,"15min":0.68999999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88600000000000001,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.626},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77800000000000002,"15min":0.80000000000000004},"min":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.95399999999999996,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.86799999999999999}]},{"osd":4,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.443,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.45200000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90300000000000002,"5min":1.018,"15min":1.018},"last":0.88900000000000001}]},{"osd":5,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64500000000000002,"15min":0.65700000000000003},"min":{"1min":0.42099999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76900000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.67200000000000004}]},{"osd":6,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.63700000000000001},"min":{"1min":0.51000000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93200000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.76600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":434,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194877,"statfs":{"total":200000000,"available":199554954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:16:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.68799999999999994,"15min":0.68999999999999995},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.68300000000000005},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90500000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.72199999999999998}]},{"osd":1,"last update":"Wed May 22 21:16:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.91300000000000003,"5min":1.107,"15min":1.107},"last":0.497},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70899999999999996,"15min":0.70699999999999996},"min":{"1min":0.53900000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92600000000000005,"5min":1.081,"15min":1.081},"last":0.70299999999999996}]},{"osd":2,"last update":"Wed May 22 21:16:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.63900000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0189999999999999,"5min":1.04,"15min":1.04},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.74099999999999999,"15min":0.73699999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96499999999999997,"5min":0.997,"15min":0.997},"last":0.67500000000000004}]},{"osd":3,"last update":"Wed May 22 21:16:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.746,"15min":0.74199999999999999},"min":{"1min":0.628,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0069999999999999,"5min":1.415,"15min":1.415},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.55100000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.087,"5min":1.788,"15min":1.788},"last":0.63100000000000001}]},{"osd":4,"last update":"Wed May 22 21:16:15 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64200000000000002,"15min":0.65300000000000002},"min":{"1min":0.42999999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.87,"5min":1.006,"15min":1.006},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.871,"5min":2.718,"15min":2.718},"last":0.56399999999999995}]},{"osd":5,"last update":"Wed May 22 21:16:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.77200000000000002,"15min":0.71199999999999997},"min":{"1min":0.46500000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80000000000000004,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.432},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.93100000000000005,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.73599999999999999}]},{"osd":7,"last update":"Wed May 22 21:16:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.45100000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.095},"last":0.749},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.76100000000000001,"15min":0.76800000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.86299999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.54800000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":434,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194877,"statfs":{"total":200000000,"available":199554956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.70499999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98699999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67500000000000004,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70799999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94899999999999995,"5min":1.216,"15min":1.216},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68799999999999994,"15min":0.68300000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.98299999999999998,"5min":1.048,"15min":1.048},"last":0.77800000000000002}]},{"osd":2,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.61299999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.96699999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.66900000000000004},"min":{"1min":0.53000000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.995,"5min":1.252,"15min":1.252},"last":0.75900000000000001}]},{"osd":3,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.53700000000000003,"5min":0.502,"15min":0.502},"max":{"1min":0.97199999999999998,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.746},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73499999999999999,"15min":0.72999999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.03,"5min":1.111,"15min":1.111},"last":0.83099999999999996}]},{"osd":4,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":0.63500000000000001},"min":{"1min":0.40999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86099999999999999,"5min":1.272,"15min":1.272},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.626,"15min":0.63100000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.70499999999999996}]},{"osd":6,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63200000000000001,"15min":0.629},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89400000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.85099999999999998}]},{"osd":7,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.874},{"interface":"front","average":{"1min":0.745,"5min":0.77900000000000003,"15min":0.79700000000000004},"min":{"1min":0.56499999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.008,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.90600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":434,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194877,"statfs":{"total":200000000,"available":199554976,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:16:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.72499999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92000000000000004,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69499999999999995,"15min":0.71099999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86299999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79800000000000004}]},{"osd":1,"last update":"Wed May 22 21:16:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69799999999999995,"15min":0.72199999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88900000000000001,"5min":1.5,"15min":1.5},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.45900000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.75900000000000001}]},{"osd":2,"last update":"Wed May 22 21:16:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82699999999999996,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94499999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60399999999999998}]},{"osd":3,"last update":"Wed May 22 21:16:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.66700000000000004,"15min":0.68300000000000005},"min":{"1min":0.504,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.73799999999999999,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.51500000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88700000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.67600000000000005}]},{"osd":5,"last update":"Wed May 22 21:16:14 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63600000000000001,"15min":0.64700000000000002},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":0.84299999999999997,"5min":1.012,"15min":1.012},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.875,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64000000000000001}]},{"osd":6,"last update":"Wed May 22 21:16:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68000000000000005,"15min":0.69999999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80300000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.68400000000000005,"15min":0.69199999999999995},"min":{"1min":0.47599999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66300000000000003}]},{"osd":7,"last update":"Wed May 22 21:16:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69099999999999995,"15min":0.69799999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82599999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.751},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76900000000000002,"15min":0.78900000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91100000000000003,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.77400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":434,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194877,"statfs":{"total":200000000,"available":199554934,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:16:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.40799999999999997,"15min":0.40000000000000002},"min":{"1min":0.34200000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.41799999999999998,"15min":0.40000000000000002},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.49399999999999999}]},{"osd":1,"last update":"Wed May 22 21:16:16 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45300000000000001,"15min":0.44500000000000001},"min":{"1min":0.34300000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.433,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83099999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.55100000000000005}]},{"osd":2,"last update":"Wed May 22 21:16:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.40600000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70699999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.441},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.49299999999999999,"15min":0.48399999999999999},"min":{"1min":0.38400000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Wed May 22 21:16:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71299999999999997,"15min":0.69699999999999995},"min":{"1min":0.59799999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71799999999999997,"15min":0.70799999999999996},"min":{"1min":0.56100000000000005,"5min":0.435,"15min":0.435},"max":{"1min":1.0549999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.66700000000000004}]},{"osd":5,"last update":"Wed May 22 21:16:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.60899999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1000000000000001,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.70899999999999996,"15min":0.68999999999999995},"min":{"1min":0.58499999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0189999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.59799999999999998}]},{"osd":6,"last update":"Wed May 22 21:16:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.72499999999999998,"15min":0.70899999999999996},"min":{"1min":0.53400000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.71699999999999997,"15min":0.69399999999999995},"min":{"1min":0.56799999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.999,"5min":1.25,"15min":1.25},"last":0.57999999999999996}]},{"osd":7,"last update":"Wed May 22 21:16:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.76700000000000002,"15min":0.749},"min":{"1min":0.61799999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.073,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.748,"15min":0.72799999999999998},"min":{"1min":0.63500000000000001,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.016,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":436,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194876,"statfs":{"total":200000000,"available":199553482,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.436,"15min":0.434},"min":{"1min":0.34300000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.308}]},{"osd":1,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.443,"15min":0.442},"min":{"1min":0.33300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68100000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.46400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.67200000000000004,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.41999999999999998}]},{"osd":3,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.495,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73899999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.45300000000000001},"min":{"1min":0.375,"5min":0.315,"15min":0.315},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.38100000000000001}]},{"osd":4,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.54200000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.57599999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.78200000000000003}]},{"osd":5,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.502,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.58899999999999997}]},{"osd":6,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.68200000000000005,"15min":0.67200000000000004},"min":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.628}]},{"osd":7,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.53700000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.77500000000000002,"15min":0.76700000000000002},"min":{"1min":0.622,"5min":0.621,"15min":0.621},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.76100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":436,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194875,"statfs":{"total":200000000,"available":199552866,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42799999999999999,"15min":0.434},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54100000000000004,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.438,"15min":0.42899999999999999},"min":{"1min":0.26900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69299999999999995,"5min":0.876,"15min":0.876},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52200000000000002,"15min":0.54000000000000004},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70699999999999996,"5min":1.923,"15min":1.923},"last":0.61499999999999999}]},{"osd":3,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56000000000000005,"15min":0.57399999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.755,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.64500000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.52600000000000002}]},{"osd":4,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.74399999999999999,"15min":0.76100000000000001},"min":{"1min":0.46500000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.98599999999999999,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68700000000000006,"15min":0.67700000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.069,"5min":1.3,"15min":1.3},"last":1.0389999999999999}]},{"osd":5,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.59499999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0609999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.749,"15min":0.752},"min":{"1min":0.54600000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0469999999999999,"5min":1.97,"15min":1.97},"last":1.1000000000000001}]},{"osd":6,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.746,"15min":0.73899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.085,"5min":1.272,"15min":1.272},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75,"15min":0.73799999999999999},"min":{"1min":0.60499999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0309999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.98799999999999999}]},{"osd":7,"last update":"Wed May 22 21:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96199999999999997,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.76900000000000002,"15min":0.78100000000000003},"min":{"1min":0.57699999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.008,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.88200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1010,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194302,"statfs":{"total":200000000,"available":198965297,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42599999999999999,"15min":0.40400000000000003},"min":{"1min":0.32200000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.73499999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.378,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67300000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.623}]},{"osd":2,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47699999999999998,"15min":0.45100000000000001},"min":{"1min":0.33200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70499999999999996,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.439,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76500000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.53400000000000003}]},{"osd":3,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.45000000000000001,"15min":0.432},"min":{"1min":0.30199999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68899999999999995,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51900000000000002,"15min":0.498},"min":{"1min":0.42499999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79300000000000004,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.54900000000000004}]},{"osd":4,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72699999999999998,"15min":0.70899999999999996},"min":{"1min":0.56000000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.754},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.72699999999999998},"min":{"1min":0.52600000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0049999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.88}]},{"osd":5,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97199999999999998,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.75600000000000001,"15min":0.73499999999999999},"min":{"1min":0.58699999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1220000000000001,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.78000000000000003}]},{"osd":6,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78800000000000003,"15min":0.77100000000000002},"min":{"1min":0.65400000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0820000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.77500000000000002,"15min":0.77000000000000002},"min":{"1min":0.58499999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0640000000000001,"5min":1.5,"15min":1.5},"last":0.73299999999999998}]},{"osd":7,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.74099999999999999,"15min":0.72199999999999998},"min":{"1min":0.61199999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.0589999999999999,"5min":1.085,"15min":1.085},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.65000000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.1990000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.80600000000000005}]}]}],"pool_statfs":[]}} 2024-05-22T21:17:18.470 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-22T21:17:18.470 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-22T21:17:18.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.648+0000 7f17a0c00640 1 Processor -- start 2024-05-22T21:17:18.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.648+0000 7f17a0c00640 1 -- start start 2024-05-22T21:17:18.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.648+0000 7f17a0c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f179c13cfc0 0x7f179c1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:18.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.648+0000 7f17a0c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f179c15af60 0x7f179c13f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:18.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.648+0000 7f17a0c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f179c140090 con 0x7f179c13fc80 2024-05-22T21:17:18.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.648+0000 7f17a0c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f179c13d3a0 con 0x7f179c15af60 2024-05-22T21:17:18.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.648+0000 7f17a0c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f179c0cd920 con 0x7f179c13cfc0 2024-05-22T21:17:18.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.648+0000 7f1791400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f179c15af60 0x7f179c13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:18.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.648+0000 7f1791400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f179c15af60 0x7f179c13f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34244/0 (socket says 172.21.15.161:34244) 2024-05-22T21:17:18.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.648+0000 7f1791400640 1 -- 172.21.15.161:0/1570940588 learned_addr learned my addr 172.21.15.161:0/1570940588 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:18.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.648+0000 7f1799e00640 1 --2- 172.21.15.161:0/1570940588 >> v2:172.21.15.179:3300/0 conn(0x7f179c13cfc0 0x7f179c1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:18.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.648+0000 7f1791400640 1 -- 172.21.15.161:0/1570940588 >> v2:172.21.15.179:3300/0 conn(0x7f179c13cfc0 msgr2=0x7f179c1402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:18.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.648+0000 7f1791400640 1 --2- 172.21.15.161:0/1570940588 >> v2:172.21.15.179:3300/0 conn(0x7f179c13cfc0 0x7f179c1402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.648+0000 7f1791400640 1 -- 172.21.15.161:0/1570940588 >> v1:172.21.15.179:6789/0 conn(0x7f179c13fc80 legacy=0x7f179c15dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:18.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.648+0000 7f1791400640 1 -- 172.21.15.161:0/1570940588 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c0cdc00 con 0x7f179c15af60 2024-05-22T21:17:18.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.648+0000 7f1799e00640 1 --2- 172.21.15.161:0/1570940588 >> v2:172.21.15.179:3300/0 conn(0x7f179c13cfc0 0x7f179c1402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:17:18.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.652+0000 7f1791400640 1 --2- 172.21.15.161:0/1570940588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f179c15af60 0x7f179c13f740 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f179c06de90 tx=0x7f178401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e9cf0658868696 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:18.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.652+0000 7f1799400640 1 -- 172.21.15.161:0/1570940588 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17840ab600 con 0x7f179c15af60 2024-05-22T21:17:18.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.652+0000 7f1799400640 1 -- 172.21.15.161:0/1570940588 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17840a7050 con 0x7f179c15af60 2024-05-22T21:17:18.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.652+0000 7f1799400640 1 -- 172.21.15.161:0/1570940588 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17840ab980 con 0x7f179c15af60 2024-05-22T21:17:18.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.652+0000 7f17a0c00640 1 -- 172.21.15.161:0/1570940588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f179c15af60 msgr2=0x7f179c13f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:18.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.652+0000 7f17a0c00640 1 --2- 172.21.15.161:0/1570940588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f179c15af60 0x7f179c13f740 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f179c06de90 tx=0x7f178401c450 comp rx=0 tx=0).stop 2024-05-22T21:17:18.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.652+0000 7f17a0c00640 1 -- 172.21.15.161:0/1570940588 shutdown_connections 2024-05-22T21:17:18.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.652+0000 7f17a0c00640 1 --2- 172.21.15.161:0/1570940588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f179c15af60 0x7f179c13f740 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.652+0000 7f17a0c00640 1 --2- 172.21.15.161:0/1570940588 >> v2:172.21.15.179:3300/0 conn(0x7f179c13cfc0 0x7f179c1402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.652+0000 7f17a0c00640 1 -- 172.21.15.161:0/1570940588 >> 172.21.15.161:0/1570940588 conn(0x7f179c06a2d0 msgr2=0x7f179c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:18.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.652+0000 7f17a0c00640 1 -- 172.21.15.161:0/1570940588 shutdown_connections 2024-05-22T21:17:18.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.652+0000 7f17a0c00640 1 -- 172.21.15.161:0/1570940588 wait complete. 2024-05-22T21:17:18.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f17a0c00640 1 Processor -- start 2024-05-22T21:17:18.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f17a0c00640 1 -- start start 2024-05-22T21:17:18.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f17a0c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f179c13cfc0 0x7f179c157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:18.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f17a0c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f179c13fc80 0x7f179c138b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:18.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f17a0c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f179c15af60 0x7f179c139060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:18.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f17a0c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c139fa0 con 0x7f179c13cfc0 2024-05-22T21:17:18.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f17a0c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c0b2d30 con 0x7f179c15af60 2024-05-22T21:17:18.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f17a0c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179c0b3010 con 0x7f179c13fc80 2024-05-22T21:17:18.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f1799e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f179c13cfc0 0x7f179c157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:18.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f1799e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f179c13cfc0 0x7f179c157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34252/0 (socket says 172.21.15.161:34252) 2024-05-22T21:17:18.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f1799e00640 1 -- 172.21.15.161:0/1789576304 learned_addr learned my addr 172.21.15.161:0/1789576304 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:18.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f1791400640 1 --2- 172.21.15.161:0/1789576304 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f179c13fc80 0x7f179c138b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:18.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f179a800640 1 --2- 172.21.15.161:0/1789576304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f179c15af60 0x7f179c139060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:18.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f1799e00640 1 -- 172.21.15.161:0/1789576304 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f179c13fc80 msgr2=0x7f179c138b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:18.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f1799e00640 1 --2- 172.21.15.161:0/1789576304 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f179c13fc80 0x7f179c138b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f1799e00640 1 -- 172.21.15.161:0/1789576304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f179c15af60 msgr2=0x7f179c139060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:18.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f1799e00640 1 --2- 172.21.15.161:0/1789576304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f179c15af60 0x7f179c139060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f1799e00640 1 -- 172.21.15.161:0/1789576304 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17840094d0 con 0x7f179c13cfc0 2024-05-22T21:17:18.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f179a800640 1 --2- 172.21.15.161:0/1789576304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f179c15af60 0x7f179c139060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:18.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f1799e00640 1 --2- 172.21.15.161:0/1789576304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f179c13cfc0 0x7f179c157350 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f178c00c7f0 tx=0x7f178c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:18.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f1792a00640 1 -- 172.21.15.161:0/1789576304 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c007820 con 0x7f179c13cfc0 2024-05-22T21:17:18.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f1792a00640 1 -- 172.21.15.161:0/1789576304 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178c007980 con 0x7f179c13cfc0 2024-05-22T21:17:18.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f1792a00640 1 -- 172.21.15.161:0/1789576304 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c007d30 con 0x7f179c13cfc0 2024-05-22T21:17:18.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f17a0c00640 1 -- 172.21.15.161:0/1789576304 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f179c0b32f0 con 0x7f179c13cfc0 2024-05-22T21:17:18.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f17a0c00640 1 -- 172.21.15.161:0/1789576304 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179c157890 con 0x7f179c13cfc0 2024-05-22T21:17:18.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.656+0000 7f17a0c00640 1 -- 172.21.15.161:0/1789576304 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f179c067220 con 0x7f179c13cfc0 2024-05-22T21:17:18.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.660+0000 7f1792a00640 1 -- 172.21.15.161:0/1789576304 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f178c0113f0 con 0x7f179c13cfc0 2024-05-22T21:17:18.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.660+0000 7f1792a00640 1 --2- 172.21.15.161:0/1789576304 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1778066690 0x7f1778068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:18.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.660+0000 7f1792a00640 1 -- 172.21.15.161:0/1789576304 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f178c084430 con 0x7f179c13cfc0 2024-05-22T21:17:18.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.660+0000 7f1791400640 1 --2- 172.21.15.161:0/1789576304 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1778066690 0x7f1778068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:18.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.660+0000 7f1791400640 1 --2- 172.21.15.161:0/1789576304 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1778066690 0x7f1778068b50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f17840ad000 tx=0x7f1784008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:18.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.664+0000 7f1792a00640 1 -- 172.21.15.161:0/1789576304 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f178c051850 con 0x7f179c13cfc0 2024-05-22T21:17:18.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.872+0000 7f17a0c00640 1 -- 172.21.15.161:0/1789576304 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f179c0b37a0 con 0x7f179c13cfc0 2024-05-22T21:17:18.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.876+0000 7f1792a00640 1 -- 172.21.15.161:0/1789576304 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f178c054e70 con 0x7f179c13cfc0 2024-05-22T21:17:18.879 INFO:teuthology.orchestra.run.smithi161.stderr:updated caps for client.0 2024-05-22T21:17:18.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.876+0000 7f17a0c00640 1 -- 172.21.15.161:0/1789576304 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1778066690 msgr2=0x7f1778068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:18.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.876+0000 7f17a0c00640 1 --2- 172.21.15.161:0/1789576304 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1778066690 0x7f1778068b50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f17840ad000 tx=0x7f1784008040 comp rx=0 tx=0).stop 2024-05-22T21:17:18.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.876+0000 7f17a0c00640 1 -- 172.21.15.161:0/1789576304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f179c13cfc0 msgr2=0x7f179c157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:18.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.876+0000 7f17a0c00640 1 --2- 172.21.15.161:0/1789576304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f179c13cfc0 0x7f179c157350 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f178c00c7f0 tx=0x7f178c00ccb0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.880+0000 7f17a0c00640 1 -- 172.21.15.161:0/1789576304 shutdown_connections 2024-05-22T21:17:18.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.880+0000 7f17a0c00640 1 --2- 172.21.15.161:0/1789576304 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1778066690 0x7f1778068b50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.880+0000 7f17a0c00640 1 --2- 172.21.15.161:0/1789576304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f179c15af60 0x7f179c139060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.880+0000 7f17a0c00640 1 --2- 172.21.15.161:0/1789576304 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f179c13fc80 0x7f179c138b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.880+0000 7f17a0c00640 1 --2- 172.21.15.161:0/1789576304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f179c13cfc0 0x7f179c157350 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:18.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.880+0000 7f17a0c00640 1 -- 172.21.15.161:0/1789576304 >> 172.21.15.161:0/1789576304 conn(0x7f179c06a2d0 msgr2=0x7f179c06a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:18.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.880+0000 7f17a0c00640 1 -- 172.21.15.161:0/1789576304 shutdown_connections 2024-05-22T21:17:18.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:18.880+0000 7f17a0c00640 1 -- 172.21.15.161:0/1789576304 wait complete. 2024-05-22T21:17:18.901 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:17:19.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4f0000640 1 Processor -- start 2024-05-22T21:17:19.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4f0000640 1 -- start start 2024-05-22T21:17:19.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4f0000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff4e8142ad0 0x7ff4e80cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4f0000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4e80cd550 0x7ff4e806a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4f0000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e8071110 con 0x7ff4e815db70 2024-05-22T21:17:19.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4f0000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e81430e0 con 0x7ff4e8142ad0 2024-05-22T21:17:19.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4f0000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e81433c0 con 0x7ff4e80cd550 2024-05-22T21:17:19.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4ed600640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff4e815db70 0x7ff4e8140a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39022/0 (socket says 172.21.15.161:39022) 2024-05-22T21:17:19.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4ed600640 1 -- 172.21.15.161:0/1784149946 learned_addr learned my addr 172.21.15.161:0/1784149946 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:19.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4ecc00640 1 --2- 172.21.15.161:0/1784149946 >> v2:172.21.15.161:3300/0 conn(0x7ff4e8142ad0 0x7ff4e80cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4ee000640 1 --2- 172.21.15.161:0/1784149946 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4e80cd550 0x7ff4e806a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4e7e00640 1 -- 172.21.15.161:0/1784149946 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657962252 0 0) 0x7ff4e8071110 con 0x7ff4e815db70 2024-05-22T21:17:19.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4e7e00640 1 -- 172.21.15.161:0/1784149946 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4d0003530 con 0x7ff4e815db70 2024-05-22T21:17:19.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4ecc00640 1 -- 172.21.15.161:0/1784149946 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4e80cd550 msgr2=0x7ff4e806a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:19.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4ecc00640 1 --2- 172.21.15.161:0/1784149946 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4e80cd550 0x7ff4e806a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4ecc00640 1 -- 172.21.15.161:0/1784149946 >> v1:172.21.15.161:6789/0 conn(0x7ff4e815db70 legacy=0x7ff4e8140a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:19.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4ecc00640 1 -- 172.21.15.161:0/1784149946 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e81436a0 con 0x7ff4e8142ad0 2024-05-22T21:17:19.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.032+0000 7ff4ee000640 1 --2- 172.21.15.161:0/1784149946 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4e80cd550 0x7ff4e806a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:19.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.036+0000 7ff4ecc00640 1 --2- 172.21.15.161:0/1784149946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e8142ad0 0x7ff4e80cd010 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40097c0 tx=0x7ff4d401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a027ac8b98c5f44b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:19.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.036+0000 7ff4e7e00640 1 -- 172.21.15.161:0/1784149946 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40ac410 con 0x7ff4e8142ad0 2024-05-22T21:17:19.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.036+0000 7ff4e7e00640 1 -- 172.21.15.161:0/1784149946 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d40ac570 con 0x7ff4e8142ad0 2024-05-22T21:17:19.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.036+0000 7ff4e7e00640 1 -- 172.21.15.161:0/1784149946 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40ac900 con 0x7ff4e8142ad0 2024-05-22T21:17:19.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.036+0000 7ff4f0000640 1 -- 172.21.15.161:0/1784149946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e8142ad0 msgr2=0x7ff4e80cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:19.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.036+0000 7ff4f0000640 1 --2- 172.21.15.161:0/1784149946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e8142ad0 0x7ff4e80cd010 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40097c0 tx=0x7ff4d401c4a0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.036+0000 7ff4f0000640 1 -- 172.21.15.161:0/1784149946 shutdown_connections 2024-05-22T21:17:19.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.036+0000 7ff4f0000640 1 --2- 172.21.15.161:0/1784149946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e8142ad0 0x7ff4e80cd010 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.036+0000 7ff4f0000640 1 --2- 172.21.15.161:0/1784149946 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4e80cd550 0x7ff4e806a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.036+0000 7ff4f0000640 1 -- 172.21.15.161:0/1784149946 >> 172.21.15.161:0/1784149946 conn(0x7ff4e80b5540 msgr2=0x7ff4e80b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:19.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.036+0000 7ff4f0000640 1 -- 172.21.15.161:0/1784149946 shutdown_connections 2024-05-22T21:17:19.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.036+0000 7ff4f0000640 1 -- 172.21.15.161:0/1784149946 wait complete. 2024-05-22T21:17:19.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4f0000640 1 Processor -- start 2024-05-22T21:17:19.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4f0000640 1 -- start start 2024-05-22T21:17:19.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4f0000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e80cd550 0x7ff4e80ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4f0000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4e8142ad0 0x7ff4e80b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4f0000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4e815db70 0x7ff4e813e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4f0000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80ad800 con 0x7ff4e80cd550 2024-05-22T21:17:19.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4f0000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80ada30 con 0x7ff4e8142ad0 2024-05-22T21:17:19.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4f0000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80add10 con 0x7ff4e815db70 2024-05-22T21:17:19.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4ed600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e80cd550 0x7ff4e80ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4ed600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e80cd550 0x7ff4e80ad2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34264/0 (socket says 172.21.15.161:34264) 2024-05-22T21:17:19.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4ed600640 1 -- 172.21.15.161:0/3129960213 learned_addr learned my addr 172.21.15.161:0/3129960213 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:19.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4ecc00640 1 --2- 172.21.15.161:0/3129960213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4e8142ad0 0x7ff4e80b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4ee000640 1 --2- 172.21.15.161:0/3129960213 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4e815db70 0x7ff4e813e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4ed600640 1 -- 172.21.15.161:0/3129960213 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4e815db70 msgr2=0x7ff4e813e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:19.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4ed600640 1 --2- 172.21.15.161:0/3129960213 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4e815db70 0x7ff4e813e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4ed600640 1 -- 172.21.15.161:0/3129960213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4e8142ad0 msgr2=0x7ff4e80b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:19.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4ed600640 1 --2- 172.21.15.161:0/3129960213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4e8142ad0 0x7ff4e80b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4ed600640 1 -- 172.21.15.161:0/3129960213 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d4009420 con 0x7ff4e80cd550 2024-05-22T21:17:19.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4ee000640 1 --2- 172.21.15.161:0/3129960213 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4e815db70 0x7ff4e813e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:19.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4ecc00640 1 --2- 172.21.15.161:0/3129960213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4e8142ad0 0x7ff4e80b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:19.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4ed600640 1 --2- 172.21.15.161:0/3129960213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e80cd550 0x7ff4e80ad2c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c420 tx=0x7ff4dc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:19.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4e5600640 1 -- 172.21.15.161:0/3129960213 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc005cf0 con 0x7ff4e80cd550 2024-05-22T21:17:19.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4e5600640 1 -- 172.21.15.161:0/3129960213 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4dc005e50 con 0x7ff4e80cd550 2024-05-22T21:17:19.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4e5600640 1 -- 172.21.15.161:0/3129960213 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc0072d0 con 0x7ff4e80cd550 2024-05-22T21:17:19.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4f0000640 1 -- 172.21.15.161:0/3129960213 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e80adff0 con 0x7ff4e80cd550 2024-05-22T21:17:19.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.040+0000 7ff4f0000640 1 -- 172.21.15.161:0/3129960213 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e80ae5b0 con 0x7ff4e80cd550 2024-05-22T21:17:19.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.044+0000 7ff4f0000640 1 -- 172.21.15.161:0/3129960213 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e80ae7a0 con 0x7ff4e80cd550 2024-05-22T21:17:19.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.044+0000 7ff4e5600640 1 -- 172.21.15.161:0/3129960213 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff4dc007430 con 0x7ff4e80cd550 2024-05-22T21:17:19.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.044+0000 7ff4e5600640 1 --2- 172.21.15.161:0/3129960213 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff4c80666c0 0x7ff4c8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.044+0000 7ff4e5600640 1 -- 172.21.15.161:0/3129960213 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4dc083720 con 0x7ff4e80cd550 2024-05-22T21:17:19.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.044+0000 7ff4ecc00640 1 --2- 172.21.15.161:0/3129960213 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff4c80666c0 0x7ff4c8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.044+0000 7ff4ecc00640 1 --2- 172.21.15.161:0/3129960213 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff4c80666c0 0x7ff4c8068b80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40ae000 tx=0x7ff4d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:19.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.052+0000 7ff4e5600640 1 -- 172.21.15.161:0/3129960213 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff4dc050aa0 con 0x7ff4e80cd550 2024-05-22T21:17:19.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.240+0000 7ff4f0000640 1 -- 172.21.15.161:0/3129960213 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4e8067220 con 0x7ff4e80cd550 2024-05-22T21:17:19.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.240+0000 7ff4e5600640 1 -- 172.21.15.161:0/3129960213 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4745 (secure 0 0 0) 0x7ff4dc0540c0 con 0x7ff4e80cd550 2024-05-22T21:17:19.243 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:19.243 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5567,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196093187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196093187},{"type":"v1","addr":"172.21.15.161:6836","nonce":196093187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6136,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4029369301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4029369301},{"type":"v1","addr":"172.21.15.161:6837","nonce":4029369301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6139,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3212560487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3212560487},{"type":"v1","addr":"172.21.15.179:6834","nonce":3212560487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:17:14.012254+0000","modified":"2024-05-22T21:17:17.029283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6306":{"gid":6306,"name":"b","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/3065973138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3065973138},{"type":"v1","addr":"172.21.15.179:6835","nonce":3065973138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-22T21:17:19.243 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 39 2024-05-22T21:17:19.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.240+0000 7ff4f0000640 1 -- 172.21.15.161:0/3129960213 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff4c80666c0 msgr2=0x7ff4c8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:19.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.240+0000 7ff4f0000640 1 --2- 172.21.15.161:0/3129960213 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff4c80666c0 0x7ff4c8068b80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40ae000 tx=0x7ff4d4008040 comp rx=0 tx=0).stop 2024-05-22T21:17:19.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.240+0000 7ff4f0000640 1 -- 172.21.15.161:0/3129960213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e80cd550 msgr2=0x7ff4e80ad2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:19.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.240+0000 7ff4f0000640 1 --2- 172.21.15.161:0/3129960213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e80cd550 0x7ff4e80ad2c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c420 tx=0x7ff4dc00c8e0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.240+0000 7ff4f0000640 1 -- 172.21.15.161:0/3129960213 shutdown_connections 2024-05-22T21:17:19.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.240+0000 7ff4f0000640 1 --2- 172.21.15.161:0/3129960213 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff4c80666c0 0x7ff4c8068b80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.240+0000 7ff4f0000640 1 --2- 172.21.15.161:0/3129960213 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4e815db70 0x7ff4e813e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.240+0000 7ff4f0000640 1 --2- 172.21.15.161:0/3129960213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4e8142ad0 0x7ff4e80b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.240+0000 7ff4f0000640 1 --2- 172.21.15.161:0/3129960213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4e80cd550 0x7ff4e80ad2c0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.240+0000 7ff4f0000640 1 -- 172.21.15.161:0/3129960213 >> 172.21.15.161:0/3129960213 conn(0x7ff4e80b5540 msgr2=0x7ff4e80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:19.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.240+0000 7ff4f0000640 1 -- 172.21.15.161:0/3129960213 shutdown_connections 2024-05-22T21:17:19.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.240+0000 7ff4f0000640 1 -- 172.21.15.161:0/3129960213 wait complete. 2024-05-22T21:17:19.667 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-22T21:17:14.012254+0000', 'modified': '2024-05-22T21:17:17.029283+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6306}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6306': {'gid': 6306, 'name': 'b', 'rank': 0, 'incarnation': 35, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.179:6835/3065973138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6833', 'nonce': 3065973138}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 3065973138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-22T21:17:19.667 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-22T21:17:19.667 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-22T21:17:19.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f1169c00640 1 Processor -- start 2024-05-22T21:17:19.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f1169c00640 1 -- start start 2024-05-22T21:17:19.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f1169c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f11641612d0 0x7f11640b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f1169c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11641616b0 0x7f116406b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f1169c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11640ce570 con 0x7f116406bf30 2024-05-22T21:17:19.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f1169c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116406c3c0 con 0x7f11641612d0 2024-05-22T21:17:19.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f1169c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11640b0820 con 0x7f11641616b0 2024-05-22T21:17:19.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f1169200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f116406bf30 0x7f116415ca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39028/0 (socket says 172.21.15.161:39028) 2024-05-22T21:17:19.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f1169200640 1 -- 172.21.15.161:0/2374667242 learned_addr learned my addr 172.21.15.161:0/2374667242 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:19.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f1163e00640 1 --2- 172.21.15.161:0/2374667242 >> v2:172.21.15.161:3300/0 conn(0x7f11641612d0 0x7f11640b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f115b400640 1 --2- 172.21.15.161:0/2374667242 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11641616b0 0x7f116406b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f1163400640 1 -- 172.21.15.161:0/2374667242 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414693938 0 0) 0x7f11640ce570 con 0x7f116406bf30 2024-05-22T21:17:19.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f1163400640 1 -- 172.21.15.161:0/2374667242 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1148003530 con 0x7f116406bf30 2024-05-22T21:17:19.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f1163e00640 1 -- 172.21.15.161:0/2374667242 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11641616b0 msgr2=0x7f116406b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:19.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f1163e00640 1 --2- 172.21.15.161:0/2374667242 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11641616b0 0x7f116406b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f1163e00640 1 -- 172.21.15.161:0/2374667242 >> v1:172.21.15.161:6789/0 conn(0x7f116406bf30 legacy=0x7f116415ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:19.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.804+0000 7f1163e00640 1 -- 172.21.15.161:0/2374667242 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11640cd6c0 con 0x7f11641612d0 2024-05-22T21:17:19.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.808+0000 7f1163e00640 1 --2- 172.21.15.161:0/2374667242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11641612d0 0x7f11640b02e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f11540097c0 tx=0x7f115401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a988ecf81638cd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:19.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.808+0000 7f1163400640 1 -- 172.21.15.161:0/2374667242 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11540ab670 con 0x7f11641612d0 2024-05-22T21:17:19.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.808+0000 7f1163400640 1 -- 172.21.15.161:0/2374667242 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11540ab7d0 con 0x7f11641612d0 2024-05-22T21:17:19.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.808+0000 7f1163400640 1 -- 172.21.15.161:0/2374667242 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11540abb60 con 0x7f11641612d0 2024-05-22T21:17:19.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.808+0000 7f1169c00640 1 -- 172.21.15.161:0/2374667242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11641612d0 msgr2=0x7f11640b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:19.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.808+0000 7f1169c00640 1 --2- 172.21.15.161:0/2374667242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11641612d0 0x7f11640b02e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f11540097c0 tx=0x7f115401c470 comp rx=0 tx=0).stop 2024-05-22T21:17:19.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.808+0000 7f1169c00640 1 -- 172.21.15.161:0/2374667242 shutdown_connections 2024-05-22T21:17:19.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.808+0000 7f1169c00640 1 --2- 172.21.15.161:0/2374667242 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11641616b0 0x7f116406b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.808+0000 7f1169c00640 1 --2- 172.21.15.161:0/2374667242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11641612d0 0x7f11640b02e0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.808+0000 7f1169c00640 1 -- 172.21.15.161:0/2374667242 >> 172.21.15.161:0/2374667242 conn(0x7f1164099a40 msgr2=0x7f1164099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:19.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.808+0000 7f1169c00640 1 -- 172.21.15.161:0/2374667242 shutdown_connections 2024-05-22T21:17:19.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.808+0000 7f1169c00640 1 -- 172.21.15.161:0/2374667242 wait complete. 2024-05-22T21:17:19.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169c00640 1 Processor -- start 2024-05-22T21:17:19.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169c00640 1 -- start start 2024-05-22T21:17:19.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f116406bf30 0x7f1164152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11641612d0 0x7f11641530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11641616b0 0x7f11641535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11640b58f0 con 0x7f11641616b0 2024-05-22T21:17:19.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1164152560 con 0x7f116406bf30 2024-05-22T21:17:19.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11640b5ae0 con 0x7f11641612d0 2024-05-22T21:17:19.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11641616b0 0x7f11641535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11641616b0 0x7f11641535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34272/0 (socket says 172.21.15.161:34272) 2024-05-22T21:17:19.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169200640 1 -- 172.21.15.161:0/2498213059 learned_addr learned my addr 172.21.15.161:0/2498213059 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:19.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169200640 1 -- 172.21.15.161:0/2498213059 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11641612d0 msgr2=0x7f11641530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:19.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169200640 1 --2- 172.21.15.161:0/2498213059 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11641612d0 0x7f11641530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169200640 1 -- 172.21.15.161:0/2498213059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f116406bf30 msgr2=0x7f1164152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:19.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169200640 1 --2- 172.21.15.161:0/2498213059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f116406bf30 0x7f1164152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169200640 1 -- 172.21.15.161:0/2498213059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1154009420 con 0x7f11641616b0 2024-05-22T21:17:19.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169200640 1 --2- 172.21.15.161:0/2498213059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11641616b0 0x7f11641535f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f115c00caa0 tx=0x7f115c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:19.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1160c00640 1 -- 172.21.15.161:0/2498213059 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c00ac00 con 0x7f11641616b0 2024-05-22T21:17:19.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1160c00640 1 -- 172.21.15.161:0/2498213059 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115c00ad60 con 0x7f11641616b0 2024-05-22T21:17:19.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169c00640 1 -- 172.21.15.161:0/2498213059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11640b5dc0 con 0x7f11641616b0 2024-05-22T21:17:19.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1160c00640 1 -- 172.21.15.161:0/2498213059 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c012620 con 0x7f11641616b0 2024-05-22T21:17:19.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.812+0000 7f1169c00640 1 -- 172.21.15.161:0/2498213059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11640b6380 con 0x7f11641616b0 2024-05-22T21:17:19.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f1160c00640 1 -- 172.21.15.161:0/2498213059 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f115c007070 con 0x7f11641616b0 2024-05-22T21:17:19.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f1160c00640 1 --2- 172.21.15.161:0/2498213059 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1140066690 0x7f1140068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f1160c00640 1 -- 172.21.15.161:0/2498213059 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f115c084180 con 0x7f11641616b0 2024-05-22T21:17:19.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f1163e00640 1 --2- 172.21.15.161:0/2498213059 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1140066690 0x7f1140068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f116e711000 1 Processor -- start 2024-05-22T21:17:19.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f116e711000 1 -- start start 2024-05-22T21:17:19.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f1163e00640 1 --2- 172.21.15.161:0/2498213059 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1140066690 0x7f1140068b50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f11540af000 tx=0x7f1154008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:19.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f116e711000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a38c49c0 0x55d4a38c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f116e711000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c52e0 0x55d4a387a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f116e711000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a387aec0 0x55d4a387b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f116e711000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4a3742e80 con 0x55d4a38c52e0 2024-05-22T21:17:19.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f116e711000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4a36d0af0 con 0x55d4a387aec0 2024-05-22T21:17:19.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f116e711000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4a36ce0c0 con 0x55d4a38c49c0 2024-05-22T21:17:19.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f115b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c52e0 0x55d4a387a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f115b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c52e0 0x55d4a387a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34288/0 (socket says 172.21.15.161:34288) 2024-05-22T21:17:19.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f115b400640 1 -- 172.21.15.161:0/2155505783 learned_addr learned my addr 172.21.15.161:0/2155505783 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:19.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f1163e00640 1 --2- 172.21.15.161:0/2155505783 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a38c49c0 0x55d4a38c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.816+0000 7f1169200640 1 --2- 172.21.15.161:0/2155505783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a387aec0 0x55d4a387b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f115b400640 1 -- 172.21.15.161:0/2155505783 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a38c49c0 msgr2=0x55d4a38c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:19.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f115b400640 1 --2- 172.21.15.161:0/2155505783 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a38c49c0 0x55d4a38c4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f115b400640 1 -- 172.21.15.161:0/2155505783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a387aec0 msgr2=0x55d4a387b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:19.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f115b400640 1 --2- 172.21.15.161:0/2155505783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a387aec0 0x55d4a387b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f115b400640 1 -- 172.21.15.161:0/2155505783 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4a33c5890 con 0x55d4a38c52e0 2024-05-22T21:17:19.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f1169200640 1 --2- 172.21.15.161:0/2155505783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a387aec0 0x55d4a387b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:19.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f115b400640 1 --2- 172.21.15.161:0/2155505783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c52e0 0x55d4a387a980 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f114c009b40 tx=0x7f114c00e1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae9cb32ea924030 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:19.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f1158c00640 1 -- 172.21.15.161:0/2155505783 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c0096c0 con 0x55d4a38c52e0 2024-05-22T21:17:19.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f1158c00640 1 -- 172.21.15.161:0/2155505783 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114c00ec10 con 0x55d4a38c52e0 2024-05-22T21:17:19.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f1158c00640 1 -- 172.21.15.161:0/2155505783 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c013470 con 0x55d4a38c52e0 2024-05-22T21:17:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 -- 172.21.15.161:0/2155505783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c52e0 msgr2=0x55d4a387a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 --2- 172.21.15.161:0/2155505783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c52e0 0x55d4a387a980 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f114c009b40 tx=0x7f114c00e1d0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 -- 172.21.15.161:0/2155505783 shutdown_connections 2024-05-22T21:17:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 --2- 172.21.15.161:0/2155505783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a387aec0 0x55d4a387b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 --2- 172.21.15.161:0/2155505783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c52e0 0x55d4a387a980 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 --2- 172.21.15.161:0/2155505783 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a38c49c0 0x55d4a38c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 -- 172.21.15.161:0/2155505783 >> 172.21.15.161:0/2155505783 conn(0x55d4a3881440 msgr2=0x55d4a3879260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 -- 172.21.15.161:0/2155505783 shutdown_connections 2024-05-22T21:17:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 -- 172.21.15.161:0/2155505783 wait complete. 2024-05-22T21:17:19.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 -- start start 2024-05-22T21:17:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 Processor -- start 2024-05-22T21:17:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a387aec0 0x55d4a38cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c49c0 0x55d4a38cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a38c52e0 0x55d4a3885fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4a36c99b0 con 0x55d4a38c49c0 2024-05-22T21:17:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4a3668c00 con 0x55d4a387aec0 2024-05-22T21:17:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4a3687070 con 0x55d4a38c52e0 2024-05-22T21:17:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f116e711000 10 client.0 _ll_register_callbacks cb 0x55d4a3866ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:17:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f115b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c49c0 0x55d4a38cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f115b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c49c0 0x55d4a38cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34294/0 (socket says 172.21.15.161:34294) 2024-05-22T21:17:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f115b400640 1 -- 172.21.15.161:0/2937188805 learned_addr learned my addr 172.21.15.161:0/2937188805 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f1169200640 1 --2- 172.21.15.161:0/2937188805 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a38c52e0 0x55d4a3885fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f1163e00640 1 --2- 172.21.15.161:0/2937188805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a387aec0 0x55d4a38cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f115b400640 1 -- 172.21.15.161:0/2937188805 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a38c52e0 msgr2=0x55d4a3885fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:19.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f115b400640 1 --2- 172.21.15.161:0/2937188805 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a38c52e0 0x55d4a3885fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f115b400640 1 -- 172.21.15.161:0/2937188805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a387aec0 msgr2=0x55d4a38cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:19.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f115b400640 1 --2- 172.21.15.161:0/2937188805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a387aec0 0x55d4a38cbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.820+0000 7f115b400640 1 -- 172.21.15.161:0/2937188805 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114c009820 con 0x55d4a38c49c0 2024-05-22T21:17:19.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f115b400640 1 --2- 172.21.15.161:0/2937188805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c49c0 0x55d4a38cc1a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f114c004630 tx=0x7f114c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec07517278cee6d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:19.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f1151600640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:17:19.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f1151600640 1 -- 172.21.15.161:0/2937188805 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c0096c0 con 0x55d4a38c49c0 2024-05-22T21:17:19.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f1151600640 1 -- 172.21.15.161:0/2937188805 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f114c013a10 con 0x55d4a38c49c0 2024-05-22T21:17:19.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 1 -- 172.21.15.161:0/2937188805 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4a36865f0 con 0x55d4a38c49c0 2024-05-22T21:17:19.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f1151600640 1 -- 172.21.15.161:0/2937188805 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f114c013c10 con 0x55d4a38c49c0 2024-05-22T21:17:19.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 10 client.6447 fetch_fsmap learned FSMap version 39 2024-05-22T21:17:19.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 1 -- 172.21.15.161:0/2937188805 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55d4a368eab0 con 0x55d4a38c49c0 2024-05-22T21:17:19.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f1151600640 1 -- 172.21.15.161:0/2937188805 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d4a36865f0 con 0x55d4a38c49c0 2024-05-22T21:17:19.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f1151600640 1 -- 172.21.15.161:0/2937188805 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f114c011070 con 0x55d4a38c49c0 2024-05-22T21:17:19.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 10 client.6447 fetch_fsmap finished waiting for FSMap version 39 2024-05-22T21:17:19.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 10 client.6447 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-22T21:17:19.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 20 client.6447 populate_metadata read hostname 'smithi161' 2024-05-22T21:17:19.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 1 --2- 172.21.15.161:0/2937188805 >> [v2:172.21.15.179:6833/3065973138,v1:172.21.15.179:6835/3065973138] conn(0x55d4a3886d20 0x55d4a3887100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 4 client.6447 mds_command: new command op to 6306 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T21:17:19.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 1 -- 172.21.15.161:0/2937188805 --> [v2:172.21.15.179:6833/3065973138,v1:172.21.15.179:6835/3065973138] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4a36c6790 con 0x55d4a3886d20 2024-05-22T21:17:19.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f1163e00640 1 --2- 172.21.15.161:0/2937188805 >> [v2:172.21.15.179:6833/3065973138,v1:172.21.15.179:6835/3065973138] conn(0x55d4a3886d20 0x55d4a3887100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f1163e00640 1 --2- 172.21.15.161:0/2937188805 >> [v2:172.21.15.179:6833/3065973138,v1:172.21.15.179:6835/3065973138] conn(0x55d4a3886d20 0x55d4a3887100 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d95725406b443592 server_cookie=b72bc587d8be7a64 in_seq=0 out_seq=0 2024-05-22T21:17:19.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f1151600640 10 client.6447 ms_handle_connect on v2:172.21.15.179:6833/3065973138 2024-05-22T21:17:19.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f1151600640 1 -- 172.21.15.161:0/2937188805 <== mds.0 v2:172.21.15.179:6833/3065973138 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d4a36c6790 con 0x55d4a3886d20 2024-05-22T21:17:19.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f1151600640 10 client.6447 handle_command_reply: tid=0 2024-05-22T21:17:19.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 1 client.6447 shutdown 2024-05-22T21:17:19.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 1 -- 172.21.15.161:0/2937188805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c49c0 msgr2=0x55d4a38cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:19.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 1 --2- 172.21.15.161:0/2937188805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c49c0 0x55d4a38cc1a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f114c004630 tx=0x7f114c009190 comp rx=0 tx=0).stop 2024-05-22T21:17:19.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 1 -- 172.21.15.161:0/2937188805 shutdown_connections 2024-05-22T21:17:19.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 1 --2- 172.21.15.161:0/2937188805 >> [v2:172.21.15.179:6833/3065973138,v1:172.21.15.179:6835/3065973138] conn(0x55d4a3886d20 0x55d4a3887100 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 1 --2- 172.21.15.161:0/2937188805 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a38c52e0 0x55d4a3885fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 1 --2- 172.21.15.161:0/2937188805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c49c0 0x55d4a38cc1a0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f1151600640 0 client.6447 ms_handle_reset on v2:172.21.15.179:6833/3065973138 2024-05-22T21:17:19.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 1 --2- 172.21.15.161:0/2937188805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a387aec0 0x55d4a38cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 1 -- 172.21.15.161:0/2937188805 >> 172.21.15.161:0/2937188805 conn(0x55d4a3881440 msgr2=0x55d4a3879960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:19.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 1 -- 172.21.15.161:0/2937188805 shutdown_connections 2024-05-22T21:17:19.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 1 -- 172.21.15.161:0/2937188805 wait complete. 2024-05-22T21:17:19.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.824+0000 7f116e711000 20 client.6447 trim_cache size 0 max 16384 2024-05-22T21:17:19.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.840+0000 7f116e711000 1 Processor -- start 2024-05-22T21:17:19.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.840+0000 7f116e711000 1 -- start start 2024-05-22T21:17:19.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.840+0000 7f116e711000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a387aec0 0x55d4a38c4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.840+0000 7f116e711000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c49c0 0x55d4a3880140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.840+0000 7f116e711000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a38c52e0 0x55d4a3880680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.840+0000 7f116e711000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4a3386940 con 0x55d4a38c49c0 2024-05-22T21:17:19.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.840+0000 7f116e711000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4a36cbfa0 con 0x55d4a38c52e0 2024-05-22T21:17:19.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.840+0000 7f116e711000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4a36d1d80 con 0x55d4a387aec0 2024-05-22T21:17:19.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f115b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c49c0 0x55d4a3880140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f115b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c49c0 0x55d4a3880140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34310/0 (socket says 172.21.15.161:34310) 2024-05-22T21:17:19.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f115b400640 1 -- 172.21.15.161:0/3948060592 learned_addr learned my addr 172.21.15.161:0/3948060592 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:19.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f115b400640 1 -- 172.21.15.161:0/3948060592 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a387aec0 msgr2=0x55d4a38c4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:19.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f1163e00640 1 --2- 172.21.15.161:0/3948060592 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a387aec0 0x55d4a38c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f1169200640 1 --2- 172.21.15.161:0/3948060592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a38c52e0 0x55d4a3880680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f115b400640 1 --2- 172.21.15.161:0/3948060592 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a387aec0 0x55d4a38c4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f115b400640 1 -- 172.21.15.161:0/3948060592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a38c52e0 msgr2=0x55d4a3880680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:19.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f115b400640 1 --2- 172.21.15.161:0/3948060592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a38c52e0 0x55d4a3880680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f115b400640 1 -- 172.21.15.161:0/3948060592 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4a3674c00 con 0x55d4a38c49c0 2024-05-22T21:17:19.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f1163e00640 1 --2- 172.21.15.161:0/3948060592 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a387aec0 0x55d4a38c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:17:19.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f115b400640 1 --2- 172.21.15.161:0/3948060592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c49c0 0x55d4a3880140 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f114c000c10 tx=0x7f114c01f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d961356d9477c26 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:19.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f1159600640 1 -- 172.21.15.161:0/3948060592 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c0096c0 con 0x55d4a38c49c0 2024-05-22T21:17:19.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f1159600640 1 -- 172.21.15.161:0/3948060592 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114c013e10 con 0x55d4a38c49c0 2024-05-22T21:17:19.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f1159600640 1 -- 172.21.15.161:0/3948060592 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c013680 con 0x55d4a38c49c0 2024-05-22T21:17:19.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 -- 172.21.15.161:0/3948060592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c49c0 msgr2=0x55d4a3880140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:19.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 --2- 172.21.15.161:0/3948060592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c49c0 0x55d4a3880140 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f114c000c10 tx=0x7f114c01f040 comp rx=0 tx=0).stop 2024-05-22T21:17:19.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 -- 172.21.15.161:0/3948060592 shutdown_connections 2024-05-22T21:17:19.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 --2- 172.21.15.161:0/3948060592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a38c52e0 0x55d4a3880680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 --2- 172.21.15.161:0/3948060592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a38c49c0 0x55d4a3880140 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 --2- 172.21.15.161:0/3948060592 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a387aec0 0x55d4a38c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 -- 172.21.15.161:0/3948060592 >> 172.21.15.161:0/3948060592 conn(0x55d4a3881440 msgr2=0x55d4a38c32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:19.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 -- 172.21.15.161:0/3948060592 shutdown_connections 2024-05-22T21:17:19.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 -- 172.21.15.161:0/3948060592 wait complete. 2024-05-22T21:17:19.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 -- start start 2024-05-22T21:17:19.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 Processor -- start 2024-05-22T21:17:19.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a387aec0 0x55d4a3925eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a38c49c0 0x55d4a39218a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a38c52e0 0x55d4a3921de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4a38cac60 con 0x55d4a387aec0 2024-05-22T21:17:19.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4a3650820 con 0x55d4a38c49c0 2024-05-22T21:17:19.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4a36c71c0 con 0x55d4a38c52e0 2024-05-22T21:17:19.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f116e711000 10 client.0 _ll_register_callbacks cb 0x7f114c0095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:17:19.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f1163e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a387aec0 0x55d4a3925eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f1163e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a387aec0 0x55d4a3925eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34324/0 (socket says 172.21.15.161:34324) 2024-05-22T21:17:19.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f1163e00640 1 -- 172.21.15.161:0/2442529978 learned_addr learned my addr 172.21.15.161:0/2442529978 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:19.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f115b400640 1 --2- 172.21.15.161:0/2442529978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a38c49c0 0x55d4a39218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.844+0000 7f1169200640 1 --2- 172.21.15.161:0/2442529978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a38c52e0 0x55d4a3921de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1163e00640 1 -- 172.21.15.161:0/2442529978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a38c52e0 msgr2=0x55d4a3921de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:19.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1163e00640 1 --2- 172.21.15.161:0/2442529978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a38c52e0 0x55d4a3921de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1163e00640 1 -- 172.21.15.161:0/2442529978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a38c49c0 msgr2=0x55d4a39218a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:19.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1163e00640 1 --2- 172.21.15.161:0/2442529978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a38c49c0 0x55d4a39218a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1163e00640 1 -- 172.21.15.161:0/2442529978 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4a368eab0 con 0x55d4a387aec0 2024-05-22T21:17:19.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1163e00640 1 --2- 172.21.15.161:0/2442529978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a387aec0 0x55d4a3925eb0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f115401c950 tx=0x7f11540046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f6cf7d65890136 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:19.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1151600640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:17:19.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1151600640 1 -- 172.21.15.161:0/2442529978 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11540c7040 con 0x55d4a387aec0 2024-05-22T21:17:19.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 1 -- 172.21.15.161:0/2442529978 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4a3699460 con 0x55d4a387aec0 2024-05-22T21:17:19.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1151600640 1 -- 172.21.15.161:0/2442529978 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11540c2b00 con 0x55d4a387aec0 2024-05-22T21:17:19.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1151600640 1 -- 172.21.15.161:0/2442529978 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11540c2cc0 con 0x55d4a387aec0 2024-05-22T21:17:19.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 10 client.6453 fetch_fsmap learned FSMap version 39 2024-05-22T21:17:19.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 1 -- 172.21.15.161:0/2442529978 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55d4a33c5cc0 con 0x55d4a387aec0 2024-05-22T21:17:19.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1151600640 1 -- 172.21.15.161:0/2442529978 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d4a3699460 con 0x55d4a387aec0 2024-05-22T21:17:19.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1151600640 1 -- 172.21.15.161:0/2442529978 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f11540ca070 con 0x55d4a387aec0 2024-05-22T21:17:19.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 10 client.6453 fetch_fsmap finished waiting for FSMap version 39 2024-05-22T21:17:19.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 10 client.6453 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-22T21:17:19.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 20 client.6453 populate_metadata read hostname 'smithi161' 2024-05-22T21:17:19.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 1 --2- 172.21.15.161:0/2442529978 >> [v2:172.21.15.179:6833/3065973138,v1:172.21.15.179:6835/3065973138] conn(0x55d4a3922320 0x55d4a3922700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:19.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 4 client.6453 mds_command: new command op to 6306 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-22T21:17:19.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 1 -- 172.21.15.161:0/2442529978 --> [v2:172.21.15.179:6833/3065973138,v1:172.21.15.179:6835/3065973138] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d4a36929c0 con 0x55d4a3922320 2024-05-22T21:17:19.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f115b400640 1 --2- 172.21.15.161:0/2442529978 >> [v2:172.21.15.179:6833/3065973138,v1:172.21.15.179:6835/3065973138] conn(0x55d4a3922320 0x55d4a3922700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:19.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1151600640 10 client.6453 ms_handle_connect on v2:172.21.15.179:6833/3065973138 2024-05-22T21:17:19.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f115b400640 1 --2- 172.21.15.161:0/2442529978 >> [v2:172.21.15.179:6833/3065973138,v1:172.21.15.179:6835/3065973138] conn(0x55d4a3922320 0x55d4a3922700 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=763f092e6fed347a server_cookie=6e502fc06b9f088b in_seq=0 out_seq=0 2024-05-22T21:17:19.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1151600640 1 -- 172.21.15.161:0/2442529978 <== mds.0 v2:172.21.15.179:6833/3065973138 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55d4a36929c0 con 0x55d4a3922320 2024-05-22T21:17:19.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1151600640 10 client.6453 handle_command_reply: tid=0 2024-05-22T21:17:19.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 1 client.6453 shutdown 2024-05-22T21:17:19.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 1 -- 172.21.15.161:0/2442529978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a387aec0 msgr2=0x55d4a3925eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:19.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 1 --2- 172.21.15.161:0/2442529978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a387aec0 0x55d4a3925eb0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f115401c950 tx=0x7f11540046d0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 1 -- 172.21.15.161:0/2442529978 shutdown_connections 2024-05-22T21:17:19.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 1 --2- 172.21.15.161:0/2442529978 >> [v2:172.21.15.179:6833/3065973138,v1:172.21.15.179:6835/3065973138] conn(0x55d4a3922320 0x55d4a3922700 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 1 --2- 172.21.15.161:0/2442529978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55d4a38c52e0 0x55d4a3921de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 1 --2- 172.21.15.161:0/2442529978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4a38c49c0 0x55d4a39218a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 1 --2- 172.21.15.161:0/2442529978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4a387aec0 0x55d4a3925eb0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1151600640 0 client.6453 ms_handle_reset on v2:172.21.15.179:6833/3065973138 2024-05-22T21:17:19.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 1 -- 172.21.15.161:0/2442529978 >> 172.21.15.161:0/2442529978 conn(0x55d4a3881440 msgr2=0x55d4a38c3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:19.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 1 -- 172.21.15.161:0/2442529978 shutdown_connections 2024-05-22T21:17:19.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 1 -- 172.21.15.161:0/2442529978 wait complete. 2024-05-22T21:17:19.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f116e711000 20 client.6453 trim_cache size 0 max 16384 2024-05-22T21:17:19.857 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:17:19.857 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:17:19.857 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 0, 2024-05-22T21:17:19.857 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 6306, 2024-05-22T21:17:19.857 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2024-05-22T21:17:19.857 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2024-05-22T21:17:19.858 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2024-05-22T21:17:19.858 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 5.8362564299999997, 2024-05-22T21:17:19.858 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 39, 2024-05-22T21:17:19.858 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 84, 2024-05-22T21:17:19.858 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 84, 2024-05-22T21:17:19.858 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 10.046088384999999 2024-05-22T21:17:19.858 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:17:19.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1169c00640 1 -- 172.21.15.161:0/2498213059 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1140066690 msgr2=0x7f1140068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:19.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1169c00640 1 --2- 172.21.15.161:0/2498213059 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1140066690 0x7f1140068b50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f11540af000 tx=0x7f1154008040 comp rx=0 tx=0).stop 2024-05-22T21:17:19.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1169c00640 1 -- 172.21.15.161:0/2498213059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11641616b0 msgr2=0x7f11641535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:19.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1169c00640 1 --2- 172.21.15.161:0/2498213059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11641616b0 0x7f11641535f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f115c00caa0 tx=0x7f115c00cf60 comp rx=0 tx=0).stop 2024-05-22T21:17:19.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1169c00640 1 -- 172.21.15.161:0/2498213059 shutdown_connections 2024-05-22T21:17:19.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1169c00640 1 --2- 172.21.15.161:0/2498213059 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1140066690 0x7f1140068b50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1169c00640 1 --2- 172.21.15.161:0/2498213059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11641616b0 0x7f11641535f0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1169c00640 1 --2- 172.21.15.161:0/2498213059 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11641612d0 0x7f11641530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1169c00640 1 --2- 172.21.15.161:0/2498213059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f116406bf30 0x7f1164152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:19.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1169c00640 1 -- 172.21.15.161:0/2498213059 >> 172.21.15.161:0/2498213059 conn(0x7f1164099a40 msgr2=0x7f11640add10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:19.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.848+0000 7f1169c00640 1 -- 172.21.15.161:0/2498213059 shutdown_connections 2024-05-22T21:17:19.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:19.852+0000 7f1169c00640 1 -- 172.21.15.161:0/2498213059 wait complete. 2024-05-22T21:17:19.871 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:17:20.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b15000640 1 Processor -- start 2024-05-22T21:17:20.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b15000640 1 -- start start 2024-05-22T21:17:20.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b15000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b1006bfb0 0x7f8b100cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:20.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b15000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b1006c440 con 0x7f8b100b0490 2024-05-22T21:17:20.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b15000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b100cf190 con 0x7f8b1015d4a0 2024-05-22T21:17:20.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b15000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1015d880 con 0x7f8b1006bfb0 2024-05-22T21:17:20.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b05400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8b100b0490 0x7f8b100cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39040/0 (socket says 172.21.15.161:39040) 2024-05-22T21:17:20.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b05400640 1 -- 172.21.15.161:0/141183243 learned_addr learned my addr 172.21.15.161:0/141183243 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:20.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b0e800640 1 --2- 172.21.15.161:0/141183243 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b1006bfb0 0x7f8b100cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:20.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b0d400640 1 -- 172.21.15.161:0/141183243 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360707339 0 0) 0x7f8af8001120 con 0x7f8b100b0490 2024-05-22T21:17:20.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b0d400640 1 -- 172.21.15.161:0/141183243 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8af4003530 con 0x7f8b100b0490 2024-05-22T21:17:20.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b0d400640 1 -- 172.21.15.161:0/141183243 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966672591 0 0) 0x7f8b100cf190 con 0x7f8b1015d4a0 2024-05-22T21:17:20.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b0d400640 1 -- 172.21.15.161:0/141183243 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8af8001120 con 0x7f8b1015d4a0 2024-05-22T21:17:20.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b0e800640 1 -- 172.21.15.161:0/141183243 >> v1:172.21.15.179:6789/0 conn(0x7f8b1015d4a0 legacy=0x7f8b10099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:20.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b0d400640 1 -- 172.21.15.161:0/141183243 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1723760794 0 0) 0x7f8af4003530 con 0x7f8b100b0490 2024-05-22T21:17:20.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b0e800640 1 -- 172.21.15.161:0/141183243 >> v1:172.21.15.161:6789/0 conn(0x7f8b100b0490 legacy=0x7f8b100cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:20.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.008+0000 7f8b0e800640 1 -- 172.21.15.161:0/141183243 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1006b160 con 0x7f8b1006bfb0 2024-05-22T21:17:20.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.012+0000 7f8b0e800640 1 --2- 172.21.15.161:0/141183243 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b1006bfb0 0x7f8b100cfd80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8b08009dc0 tx=0x7f8b0801c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15099d2480317148 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:20.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.012+0000 7f8b0d400640 1 -- 172.21.15.161:0/141183243 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8001810 con 0x7f8b1006bfb0 2024-05-22T21:17:20.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.012+0000 7f8b0d400640 1 -- 172.21.15.161:0/141183243 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b080a5600 con 0x7f8b1006bfb0 2024-05-22T21:17:20.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.012+0000 7f8b0d400640 1 -- 172.21.15.161:0/141183243 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b080a5970 con 0x7f8b1006bfb0 2024-05-22T21:17:20.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.012+0000 7f8b15000640 1 -- 172.21.15.161:0/141183243 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b1006bfb0 msgr2=0x7f8b100cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:20.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.012+0000 7f8b15000640 1 --2- 172.21.15.161:0/141183243 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b1006bfb0 0x7f8b100cfd80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8b08009dc0 tx=0x7f8b0801c8b0 comp rx=0 tx=0).stop 2024-05-22T21:17:20.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.012+0000 7f8b15000640 1 -- 172.21.15.161:0/141183243 shutdown_connections 2024-05-22T21:17:20.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.012+0000 7f8b15000640 1 --2- 172.21.15.161:0/141183243 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b1006bfb0 0x7f8b100cfd80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:20.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.012+0000 7f8b15000640 1 -- 172.21.15.161:0/141183243 >> 172.21.15.161:0/141183243 conn(0x7f8b100b2a40 msgr2=0x7f8b100b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:20.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.012+0000 7f8b15000640 1 -- 172.21.15.161:0/141183243 shutdown_connections 2024-05-22T21:17:20.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.012+0000 7f8b15000640 1 -- 172.21.15.161:0/141183243 wait complete. 2024-05-22T21:17:20.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b15000640 1 Processor -- start 2024-05-22T21:17:20.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b15000640 1 -- start start 2024-05-22T21:17:20.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b15000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b100b0490 0x7f8b100b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:20.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b15000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b1015d4a0 0x7f8b10160740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:20.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b15000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b1015e840 0x7f8b101560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:20.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b15000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b10160c80 con 0x7f8b1015e840 2024-05-22T21:17:20.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b15000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b10158a10 con 0x7f8b100b0490 2024-05-22T21:17:20.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b15000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b10157b10 con 0x7f8b1015d4a0 2024-05-22T21:17:20.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b0e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b1015e840 0x7f8b101560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:20.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b0e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b1015e840 0x7f8b101560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34340/0 (socket says 172.21.15.161:34340) 2024-05-22T21:17:20.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b0e800640 1 -- 172.21.15.161:0/30770445 learned_addr learned my addr 172.21.15.161:0/30770445 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:20.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b05400640 1 --2- 172.21.15.161:0/30770445 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b1015d4a0 0x7f8b10160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:20.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b0de00640 1 --2- 172.21.15.161:0/30770445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b100b0490 0x7f8b100b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:20.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b0e800640 1 -- 172.21.15.161:0/30770445 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b1015d4a0 msgr2=0x7f8b10160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:20.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b0e800640 1 --2- 172.21.15.161:0/30770445 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b1015d4a0 0x7f8b10160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:20.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b0e800640 1 -- 172.21.15.161:0/30770445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b100b0490 msgr2=0x7f8b100b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:20.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b0e800640 1 --2- 172.21.15.161:0/30770445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b100b0490 0x7f8b100b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:20.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b0e800640 1 -- 172.21.15.161:0/30770445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b08009a70 con 0x7f8b1015e840 2024-05-22T21:17:20.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b05400640 1 --2- 172.21.15.161:0/30770445 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b1015d4a0 0x7f8b10160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:17:20.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b0e800640 1 --2- 172.21.15.161:0/30770445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b1015e840 0x7f8b101560f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8b0801a4e0 tx=0x7f8b08019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:20.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b06a00640 1 -- 172.21.15.161:0/30770445 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b080bd070 con 0x7f8b1015e840 2024-05-22T21:17:20.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b06a00640 1 -- 172.21.15.161:0/30770445 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b08009450 con 0x7f8b1015e840 2024-05-22T21:17:20.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b06a00640 1 -- 172.21.15.161:0/30770445 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b080ad870 con 0x7f8b1015e840 2024-05-22T21:17:20.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b15000640 1 -- 172.21.15.161:0/30770445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1009a3e0 con 0x7f8b1015e840 2024-05-22T21:17:20.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.016+0000 7f8b15000640 1 -- 172.21.15.161:0/30770445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b10158f80 con 0x7f8b1015e840 2024-05-22T21:17:20.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.020+0000 7f8b15000640 1 -- 172.21.15.161:0/30770445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b10067220 con 0x7f8b1015e840 2024-05-22T21:17:20.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.020+0000 7f8b06a00640 1 -- 172.21.15.161:0/30770445 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b080b4070 con 0x7f8b1015e840 2024-05-22T21:17:20.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.020+0000 7f8b06a00640 1 --2- 172.21.15.161:0/30770445 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8aec066690 0x7f8aec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:20.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.020+0000 7f8b06a00640 1 -- 172.21.15.161:0/30770445 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b0811e2f0 con 0x7f8b1015e840 2024-05-22T21:17:20.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.020+0000 7f8b0de00640 1 --2- 172.21.15.161:0/30770445 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8aec066690 0x7f8aec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:20.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.020+0000 7f8b0de00640 1 --2- 172.21.15.161:0/30770445 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8aec066690 0x7f8aec068b50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8b000011f0 tx=0x7f8b00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:20.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.024+0000 7f8b06a00640 1 -- 172.21.15.161:0/30770445 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8b080eb5c0 con 0x7f8b1015e840 2024-05-22T21:17:20.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.212+0000 7f8b15000640 1 -- 172.21.15.161:0/30770445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8b1006f170 con 0x7f8b1015e840 2024-05-22T21:17:20.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.212+0000 7f8b06a00640 1 -- 172.21.15.161:0/30770445 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8b080eebe0 con 0x7f8b1015e840 2024-05-22T21:17:20.215 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:17:20.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.212+0000 7f8b15000640 1 -- 172.21.15.161:0/30770445 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8aec066690 msgr2=0x7f8aec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:20.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.212+0000 7f8b15000640 1 --2- 172.21.15.161:0/30770445 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8aec066690 0x7f8aec068b50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8b000011f0 tx=0x7f8b00009040 comp rx=0 tx=0).stop 2024-05-22T21:17:20.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.212+0000 7f8b15000640 1 -- 172.21.15.161:0/30770445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b1015e840 msgr2=0x7f8b101560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:20.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.212+0000 7f8b15000640 1 --2- 172.21.15.161:0/30770445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b1015e840 0x7f8b101560f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8b0801a4e0 tx=0x7f8b08019650 comp rx=0 tx=0).stop 2024-05-22T21:17:20.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.212+0000 7f8b15000640 1 -- 172.21.15.161:0/30770445 shutdown_connections 2024-05-22T21:17:20.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.212+0000 7f8b15000640 1 --2- 172.21.15.161:0/30770445 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8aec066690 0x7f8aec068b50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:20.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.212+0000 7f8b15000640 1 --2- 172.21.15.161:0/30770445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b1015e840 0x7f8b101560f0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:20.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.212+0000 7f8b15000640 1 --2- 172.21.15.161:0/30770445 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b1015d4a0 0x7f8b10160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:20.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.212+0000 7f8b15000640 1 --2- 172.21.15.161:0/30770445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b100b0490 0x7f8b100b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:20.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.212+0000 7f8b15000640 1 -- 172.21.15.161:0/30770445 >> 172.21.15.161:0/30770445 conn(0x7f8b100b2a40 msgr2=0x7f8b1015c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:20.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.212+0000 7f8b15000640 1 -- 172.21.15.161:0/30770445 shutdown_connections 2024-05-22T21:17:20.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:20.212+0000 7f8b15000640 1 -- 172.21.15.161:0/30770445 wait complete. 2024-05-22T21:17:20.238 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:17:20.238 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:17:20.238 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-05-22T21:17:20.238 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-22T21:17:20.238 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-22T21:17:20.238 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:17:20.238 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:17:20.238 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:17:20.238 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:17:20.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:20.239 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-05-22T21:17:20.248 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:17:20.248 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:17:20.248 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:17:20.248 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:17:20.248 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-05-22T21:17:20.248 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:17:20.248 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:17:20.249 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-05-22T21:17:20.249 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2024-05-22T21:17:20.249 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:17:20.249 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:17:20.249 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2024-05-22T21:17:20.249 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:17:20.249 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:17:20.249 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:17:20.249 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:17:20.249 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:17:20.249 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42252sec preferred_lft 42252sec 2024-05-22T21:17:20.249 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-05-22T21:17:20.249 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:17:20.250 INFO:teuthology.orchestra.run.smithi161.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:17:20.250 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 02:42:b9:0f:57:df brd ff:ff:ff:ff:ff:ff 2024-05-22T21:17:20.250 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:17:20.250 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:17:20.250 INFO:teuthology.orchestra.run.smithi161.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:17:20.250 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:17:20.250 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:17:20.251 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:17:20.251 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:17:20.251 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:17:20.251 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:17:20.251 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-22T21:17:20.251 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:17:20.251 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:17:20.252 INFO:teuthology.orchestra.run.smithi161.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:17:20.252 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-22T21:17:20.252 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:17:20.252 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:17:20.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:20.253 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:17:20.305 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-05-22T21:17:20.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:20.308 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-05-22T21:17:20.359 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-05-22T21:17:20.359 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:17:20.359 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:17:20.360 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:17:20.360 INFO:teuthology.orchestra.run.smithi161.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:17:20.360 INFO:teuthology.orchestra.run.smithi161.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:17:20.360 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:17:20.360 INFO:teuthology.orchestra.run.smithi161.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:17:20.360 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:17:20.360 INFO:teuthology.orchestra.run.smithi161.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:17:20.360 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:17:20.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:20.360 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:17:20.360 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:17:20.361 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:17:20.361 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:17:20.361 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:17:20.361 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:17:20.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:20.449 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-05-22T21:17:20.457 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-22T21:17:20.457 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-22T21:17:20.458 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:17:20.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:20.458 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2024-05-22T21:17:20.520 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:17:20.520 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-22T21:17:20.568 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:17:20.568 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:17:20.568 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:17:20.616 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:17:20.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:20.616 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:17:20.705 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:17:20.705 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:17:20.705 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:17:20.720 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:17:20.721 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:17:20.721 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:17:20.721 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:17:20.721 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:17:20.730 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:17:20.730 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:17:20.730 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:17:20.730 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:17:20.730 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:17:20.731 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:17:20.731 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:17:20.731 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:17:20.734 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:17:20.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:20.735 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:17:20.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:20.754 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:17:20.817 INFO:teuthology.orchestra.run.smithi161.stdout:client6250 2024-05-22T21:17:20.817 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:17:20.817 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:17:20.868 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:17:21.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.040+0000 7f332a000640 1 Processor -- start 2024-05-22T21:17:21.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.040+0000 7f332a000640 1 -- start start 2024-05-22T21:17:21.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.040+0000 7f332a000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f332415d040 0x7f33240b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:21.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.040+0000 7f332a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f33240d0cf0 0x7f33240d1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:21.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.040+0000 7f332a000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33240b13d0 con 0x7f33240cd840 2024-05-22T21:17:21.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.040+0000 7f332a000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33240d1690 con 0x7f33240d0cf0 2024-05-22T21:17:21.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.040+0000 7f332a000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33240b0aa0 con 0x7f332415d040 2024-05-22T21:17:21.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.040+0000 7f331be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f33240d0cf0 0x7f33240d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:21.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.040+0000 7f331be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f33240d0cf0 0x7f33240d1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34352/0 (socket says 172.21.15.161:34352) 2024-05-22T21:17:21.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.040+0000 7f331be00640 1 -- 172.21.15.161:0/2630967215 learned_addr learned my addr 172.21.15.161:0/2630967215 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:21.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.040+0000 7f3328c00640 1 --2- 172.21.15.161:0/2630967215 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f332415d040 0x7f33240b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:21.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.040+0000 7f331be00640 1 -- 172.21.15.161:0/2630967215 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f332415d040 msgr2=0x7f33240b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:17:21.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.040+0000 7f331be00640 1 --2- 172.21.15.161:0/2630967215 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f332415d040 0x7f33240b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:21.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.040+0000 7f331be00640 1 -- 172.21.15.161:0/2630967215 >> v1:172.21.15.179:6789/0 conn(0x7f33240cd840 legacy=0x7f33240cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:21.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.040+0000 7f331be00640 1 -- 172.21.15.161:0/2630967215 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33240b0d80 con 0x7f33240d0cf0 2024-05-22T21:17:21.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.044+0000 7f331be00640 1 --2- 172.21.15.161:0/2630967215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33240d0cf0 0x7f33240d1150 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f330c009870 tx=0x7f330c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94240c53c4bcefc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:21.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.044+0000 7f3323e00640 1 -- 172.21.15.161:0/2630967215 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c0ab600 con 0x7f33240d0cf0 2024-05-22T21:17:21.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.044+0000 7f3323e00640 1 -- 172.21.15.161:0/2630967215 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330c0a7050 con 0x7f33240d0cf0 2024-05-22T21:17:21.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.044+0000 7f3323e00640 1 -- 172.21.15.161:0/2630967215 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c0ab980 con 0x7f33240d0cf0 2024-05-22T21:17:21.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.044+0000 7f332a000640 1 -- 172.21.15.161:0/2630967215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33240d0cf0 msgr2=0x7f33240d1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:21.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.044+0000 7f332a000640 1 --2- 172.21.15.161:0/2630967215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33240d0cf0 0x7f33240d1150 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f330c009870 tx=0x7f330c01c560 comp rx=0 tx=0).stop 2024-05-22T21:17:21.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.044+0000 7f332a000640 1 -- 172.21.15.161:0/2630967215 shutdown_connections 2024-05-22T21:17:21.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.044+0000 7f332a000640 1 --2- 172.21.15.161:0/2630967215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33240d0cf0 0x7f33240d1150 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:21.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.044+0000 7f332a000640 1 --2- 172.21.15.161:0/2630967215 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f332415d040 0x7f33240b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:21.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.044+0000 7f332a000640 1 -- 172.21.15.161:0/2630967215 >> 172.21.15.161:0/2630967215 conn(0x7f3324099a40 msgr2=0x7f3324099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:21.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.044+0000 7f332a000640 1 -- 172.21.15.161:0/2630967215 shutdown_connections 2024-05-22T21:17:21.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.044+0000 7f332a000640 1 -- 172.21.15.161:0/2630967215 wait complete. 2024-05-22T21:17:21.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.048+0000 7f332a000640 1 Processor -- start 2024-05-22T21:17:21.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.048+0000 7f332a000640 1 -- start start 2024-05-22T21:17:21.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.048+0000 7f332a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33240cd840 0x7f3324155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:21.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.048+0000 7f332a000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33240d0cf0 0x7f3324154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:21.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.048+0000 7f332a000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f332415d040 0x7f3324158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:21.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.048+0000 7f332a000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3324155a90 con 0x7f33240cd840 2024-05-22T21:17:21.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.048+0000 7f332a000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3324154e80 con 0x7f33240d0cf0 2024-05-22T21:17:21.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.048+0000 7f332a000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3324155160 con 0x7f332415d040 2024-05-22T21:17:21.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.048+0000 7f3328c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33240cd840 0x7f3324155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:21.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.048+0000 7f3328c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33240cd840 0x7f3324155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34364/0 (socket says 172.21.15.161:34364) 2024-05-22T21:17:21.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.048+0000 7f3328c00640 1 -- 172.21.15.161:0/1328414140 learned_addr learned my addr 172.21.15.161:0/1328414140 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:21.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.048+0000 7f331be00640 1 --2- 172.21.15.161:0/1328414140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33240d0cf0 0x7f3324154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:21.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f3329600640 1 --2- 172.21.15.161:0/1328414140 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f332415d040 0x7f3324158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:21.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f3328c00640 1 -- 172.21.15.161:0/1328414140 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f332415d040 msgr2=0x7f3324158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:21.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f3328c00640 1 --2- 172.21.15.161:0/1328414140 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f332415d040 0x7f3324158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:21.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f3328c00640 1 -- 172.21.15.161:0/1328414140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33240d0cf0 msgr2=0x7f3324154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:21.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f3328c00640 1 --2- 172.21.15.161:0/1328414140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33240d0cf0 0x7f3324154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:21.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f3328c00640 1 -- 172.21.15.161:0/1328414140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c0094d0 con 0x7f33240cd840 2024-05-22T21:17:21.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f3328c00640 1 --2- 172.21.15.161:0/1328414140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33240cd840 0x7f3324155f30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f331400d5f0 tx=0x7f331400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:21.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f3321600640 1 -- 172.21.15.161:0/1328414140 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33140125f0 con 0x7f33240cd840 2024-05-22T21:17:21.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f3321600640 1 -- 172.21.15.161:0/1328414140 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3314012750 con 0x7f33240cd840 2024-05-22T21:17:21.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f332a000640 1 -- 172.21.15.161:0/1328414140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3324155440 con 0x7f33240cd840 2024-05-22T21:17:21.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f3321600640 1 -- 172.21.15.161:0/1328414140 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3314012b00 con 0x7f33240cd840 2024-05-22T21:17:21.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f332a000640 1 -- 172.21.15.161:0/1328414140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3324154660 con 0x7f33240cd840 2024-05-22T21:17:21.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f332a000640 1 -- 172.21.15.161:0/1328414140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3324067070 con 0x7f33240cd840 2024-05-22T21:17:21.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f3321600640 1 -- 172.21.15.161:0/1328414140 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3314012c60 con 0x7f33240cd840 2024-05-22T21:17:21.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f3321600640 1 --2- 172.21.15.161:0/1328414140 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3300066690 0x7f3300068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:21.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f3321600640 1 -- 172.21.15.161:0/1328414140 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33140847a0 con 0x7f33240cd840 2024-05-22T21:17:21.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.052+0000 7f331be00640 1 --2- 172.21.15.161:0/1328414140 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3300066690 0x7f3300068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:21.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.056+0000 7f331be00640 1 --2- 172.21.15.161:0/1328414140 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3300066690 0x7f3300068b50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f330c0ad000 tx=0x7f330c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:21.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.060+0000 7f3321600640 1 -- 172.21.15.161:0/1328414140 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3314051a70 con 0x7f33240cd840 2024-05-22T21:17:21.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.252+0000 7f332a000640 1 -- 172.21.15.161:0/1328414140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f33240b6d10 con 0x7f33240cd840 2024-05-22T21:17:21.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.256+0000 7f3321600640 1 -- 172.21.15.161:0/1328414140 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3314055090 con 0x7f33240cd840 2024-05-22T21:17:21.259 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:17:21.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.256+0000 7f332a000640 1 -- 172.21.15.161:0/1328414140 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3300066690 msgr2=0x7f3300068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:21.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.256+0000 7f332a000640 1 --2- 172.21.15.161:0/1328414140 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3300066690 0x7f3300068b50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f330c0ad000 tx=0x7f330c008040 comp rx=0 tx=0).stop 2024-05-22T21:17:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.256+0000 7f332a000640 1 -- 172.21.15.161:0/1328414140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33240cd840 msgr2=0x7f3324155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.256+0000 7f332a000640 1 --2- 172.21.15.161:0/1328414140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33240cd840 0x7f3324155f30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f331400d5f0 tx=0x7f331400dab0 comp rx=0 tx=0).stop 2024-05-22T21:17:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.256+0000 7f332a000640 1 -- 172.21.15.161:0/1328414140 shutdown_connections 2024-05-22T21:17:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.256+0000 7f332a000640 1 --2- 172.21.15.161:0/1328414140 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3300066690 0x7f3300068b50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.256+0000 7f332a000640 1 --2- 172.21.15.161:0/1328414140 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f332415d040 0x7f3324158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.256+0000 7f332a000640 1 --2- 172.21.15.161:0/1328414140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33240d0cf0 0x7f3324154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.256+0000 7f332a000640 1 --2- 172.21.15.161:0/1328414140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33240cd840 0x7f3324155f30 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.256+0000 7f332a000640 1 -- 172.21.15.161:0/1328414140 >> 172.21.15.161:0/1328414140 conn(0x7f3324099a40 msgr2=0x7f33240cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.256+0000 7f332a000640 1 -- 172.21.15.161:0/1328414140 shutdown_connections 2024-05-22T21:17:21.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:21.256+0000 7f332a000640 1 -- 172.21.15.161:0/1328414140 wait complete. 2024-05-22T21:17:21.278 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:17:21.278 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:17:21.278 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-22T21:17:21.278 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-22T21:17:21.278 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-22T21:17:21.278 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:17:21.278 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:17:21.278 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:17:21.278 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:17:21.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:21.278 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-22T21:17:21.289 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:17:21.289 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:17:21.289 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:17:21.289 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:17:21.289 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-22T21:17:21.289 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:17:21.290 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:17:21.290 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:17:21.290 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f0 2024-05-22T21:17:21.290 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:17:21.290 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:17:21.290 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f1 2024-05-22T21:17:21.290 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:17:21.290 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-22T21:17:21.290 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:17:21.290 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-22T21:17:21.290 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:17:21.291 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42243sec preferred_lft 42243sec 2024-05-22T21:17:21.291 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-22T21:17:21.291 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:17:21.291 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:17:21.291 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:bd:4e:25:3d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:17:21.291 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:17:21.291 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:17:21.291 INFO:teuthology.orchestra.run.smithi179.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:17:21.291 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:17:21.291 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:17:21.291 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:17:21.291 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:17:21.291 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:17:21.292 INFO:teuthology.orchestra.run.smithi179.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:17:21.292 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-22T21:17:21.292 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:17:21.292 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:17:21.292 INFO:teuthology.orchestra.run.smithi179.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:17:21.292 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-22T21:17:21.292 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:17:21.292 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:17:21.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:21.293 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:17:21.353 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-22T21:17:21.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:21.356 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-22T21:17:21.411 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-22T21:17:21.411 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:17:21.411 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:17:21.411 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:17:21.411 INFO:teuthology.orchestra.run.smithi179.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:17:21.411 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:17:21.411 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:17:21.411 INFO:teuthology.orchestra.run.smithi179.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:17:21.411 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:17:21.412 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:17:21.412 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:17:21.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:21.412 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:17:21.412 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:17:21.412 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:17:21.412 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:17:21.412 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:17:21.412 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:17:21.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:21.508 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-22T21:17:21.517 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-22T21:17:21.517 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-22T21:17:21.518 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:17:21.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:21.519 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-22T21:17:21.585 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:17:21.586 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-22T21:17:21.638 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:17:21.638 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:17:21.638 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:17:21.692 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:17:21.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:21.693 DEBUG:teuthology.orchestra.run.smithi179:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:17:21.784 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:17:21.784 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:17:21.784 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:17:21.799 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:17:21.799 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:17:21.799 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:17:21.799 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:17:21.799 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:17:21.807 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:17:21.807 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:17:21.808 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:17:21.808 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:17:21.808 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:17:21.808 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:17:21.808 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:17:21.808 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:17:21.811 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:17:21.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:21.812 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:17:21.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:17:21.831 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:17:21.884 INFO:teuthology.orchestra.run.smithi179.stdout:client5678 2024-05-22T21:17:21.885 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:17:21.885 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:17:21.933 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:17:22.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.072+0000 7f59d2a00640 1 Processor -- start 2024-05-22T21:17:22.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.072+0000 7f59d2a00640 1 -- start start 2024-05-22T21:17:22.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.072+0000 7f59d2a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f59cc15c6d0 0x7f59cc164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:22.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.072+0000 7f59d2a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59cc071670 con 0x7f59cc15e130 2024-05-22T21:17:22.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.072+0000 7f59d2a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59cc070a60 con 0x7f59cc15dd50 2024-05-22T21:17:22.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.072+0000 7f59d2a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59cc070d40 con 0x7f59cc15c6d0 2024-05-22T21:17:22.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.072+0000 7f59c3400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f59cc15e130 0x7f59cc15ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39042/0 (socket says 172.21.15.161:39042) 2024-05-22T21:17:22.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.072+0000 7f59c3400640 1 -- 172.21.15.161:0/4045487105 learned_addr learned my addr 172.21.15.161:0/4045487105 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:17:22.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.072+0000 7f59d0a00640 1 --2- 172.21.15.161:0/4045487105 >> v2:172.21.15.179:3300/0 conn(0x7f59cc15c6d0 0x7f59cc164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:22.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.076+0000 7f59cb400640 1 -- 172.21.15.161:0/4045487105 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552399029 0 0) 0x7f59cc071670 con 0x7f59cc15e130 2024-05-22T21:17:22.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.076+0000 7f59cb400640 1 -- 172.21.15.161:0/4045487105 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b0003530 con 0x7f59cc15e130 2024-05-22T21:17:22.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.076+0000 7f59cb400640 1 -- 172.21.15.161:0/4045487105 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261227890 0 0) 0x7f59cc070a60 con 0x7f59cc15dd50 2024-05-22T21:17:22.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.076+0000 7f59cb400640 1 -- 172.21.15.161:0/4045487105 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59cc071670 con 0x7f59cc15dd50 2024-05-22T21:17:22.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.076+0000 7f59d0a00640 1 -- 172.21.15.161:0/4045487105 >> v1:172.21.15.179:6789/0 conn(0x7f59cc15dd50 legacy=0x7f59cc06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:22.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.076+0000 7f59d0a00640 1 -- 172.21.15.161:0/4045487105 >> v1:172.21.15.161:6789/0 conn(0x7f59cc15e130 legacy=0x7f59cc15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:22.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.076+0000 7f59d0a00640 1 -- 172.21.15.161:0/4045487105 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc071020 con 0x7f59cc15c6d0 2024-05-22T21:17:22.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.076+0000 7f59d0a00640 1 --2- 172.21.15.161:0/4045487105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc15c6d0 0x7f59cc164e80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009d90 tx=0x7f59c401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87a124c5aa680841 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:22.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.076+0000 7f59cb400640 1 -- 172.21.15.161:0/4045487105 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40a59c0 con 0x7f59cc15c6d0 2024-05-22T21:17:22.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.076+0000 7f59cb400640 1 -- 172.21.15.161:0/4045487105 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c40a5b20 con 0x7f59cc15c6d0 2024-05-22T21:17:22.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.076+0000 7f59cb400640 1 -- 172.21.15.161:0/4045487105 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40ad440 con 0x7f59cc15c6d0 2024-05-22T21:17:22.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d2a00640 1 -- 172.21.15.161:0/4045487105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc15c6d0 msgr2=0x7f59cc164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:22.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d2a00640 1 --2- 172.21.15.161:0/4045487105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc15c6d0 0x7f59cc164e80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009d90 tx=0x7f59c401c940 comp rx=0 tx=0).stop 2024-05-22T21:17:22.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d2a00640 1 -- 172.21.15.161:0/4045487105 shutdown_connections 2024-05-22T21:17:22.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d2a00640 1 --2- 172.21.15.161:0/4045487105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc15c6d0 0x7f59cc164e80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:22.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d2a00640 1 -- 172.21.15.161:0/4045487105 >> 172.21.15.161:0/4045487105 conn(0x7f59cc0b0bc0 msgr2=0x7f59cc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:22.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d2a00640 1 -- 172.21.15.161:0/4045487105 shutdown_connections 2024-05-22T21:17:22.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d2a00640 1 -- 172.21.15.161:0/4045487105 wait complete. 2024-05-22T21:17:22.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d2a00640 1 Processor -- start 2024-05-22T21:17:22.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d2a00640 1 -- start start 2024-05-22T21:17:22.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d2a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc15c6d0 0x7f59cc157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:22.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d2a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59cc15dd50 0x7f59cc14afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:22.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15e130 0x7f59cc152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:22.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d2a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc14ecc0 con 0x7f59cc15e130 2024-05-22T21:17:22.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d2a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc14b500 con 0x7f59cc15c6d0 2024-05-22T21:17:22.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d2a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc14f0e0 con 0x7f59cc15dd50 2024-05-22T21:17:22.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d0a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15e130 0x7f59cc152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:22.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d0a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15e130 0x7f59cc152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34378/0 (socket says 172.21.15.161:34378) 2024-05-22T21:17:22.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d0a00640 1 -- 172.21.15.161:0/1180489577 learned_addr learned my addr 172.21.15.161:0/1180489577 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:17:22.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59cbe00640 1 --2- 172.21.15.161:0/1180489577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc15c6d0 0x7f59cc157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:22.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59c3400640 1 --2- 172.21.15.161:0/1180489577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59cc15dd50 0x7f59cc14afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:22.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d0a00640 1 -- 172.21.15.161:0/1180489577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59cc15dd50 msgr2=0x7f59cc14afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:22.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d0a00640 1 --2- 172.21.15.161:0/1180489577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59cc15dd50 0x7f59cc14afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:22.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d0a00640 1 -- 172.21.15.161:0/1180489577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc15c6d0 msgr2=0x7f59cc157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:22.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d0a00640 1 --2- 172.21.15.161:0/1180489577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc15c6d0 0x7f59cc157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:22.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d0a00640 1 -- 172.21.15.161:0/1180489577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c4009a70 con 0x7f59cc15e130 2024-05-22T21:17:22.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.080+0000 7f59d0a00640 1 --2- 172.21.15.161:0/1180489577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15e130 0x7f59cc152e40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f59c401a570 tx=0x7f59c40196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:22.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.084+0000 7f59c8c00640 1 -- 172.21.15.161:0/1180489577 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40adc60 con 0x7f59cc15e130 2024-05-22T21:17:22.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.084+0000 7f59c8c00640 1 -- 172.21.15.161:0/1180489577 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c40addc0 con 0x7f59cc15e130 2024-05-22T21:17:22.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.084+0000 7f59c8c00640 1 -- 172.21.15.161:0/1180489577 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40b9650 con 0x7f59cc15e130 2024-05-22T21:17:22.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.084+0000 7f59d2a00640 1 -- 172.21.15.161:0/1180489577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59cc14e0b0 con 0x7f59cc15e130 2024-05-22T21:17:22.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.084+0000 7f59d2a00640 1 -- 172.21.15.161:0/1180489577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc14e320 con 0x7f59cc15e130 2024-05-22T21:17:22.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.084+0000 7f59d2a00640 1 -- 172.21.15.161:0/1180489577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59cc14ee00 con 0x7f59cc15e130 2024-05-22T21:17:22.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.084+0000 7f59c8c00640 1 -- 172.21.15.161:0/1180489577 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59c40b97b0 con 0x7f59cc15e130 2024-05-22T21:17:22.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.084+0000 7f59c8c00640 1 --2- 172.21.15.161:0/1180489577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59a8066690 0x7f59a8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:17:22.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.084+0000 7f59c8c00640 1 -- 172.21.15.161:0/1180489577 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59c411f6e0 con 0x7f59cc15e130 2024-05-22T21:17:22.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.084+0000 7f59cbe00640 1 --2- 172.21.15.161:0/1180489577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59a8066690 0x7f59a8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:17:22.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.088+0000 7f59cbe00640 1 --2- 172.21.15.161:0/1180489577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59a8066690 0x7f59a8068b50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0011f0 tx=0x7f59bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:17:22.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.088+0000 7f59c8c00640 1 -- 172.21.15.161:0/1180489577 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f59cc14ee00 con 0x7f59cc15e130 2024-05-22T21:17:22.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.276+0000 7f59d2a00640 1 -- 172.21.15.161:0/1180489577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59cc14ee00 con 0x7f59cc15e130 2024-05-22T21:17:22.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.276+0000 7f59c8c00640 1 -- 172.21.15.161:0/1180489577 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4745 (secure 0 0 0) 0x7f59cc14ee00 con 0x7f59cc15e130 2024-05-22T21:17:22.278 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:17:22.278 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5567,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196093187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196093187},{"type":"v1","addr":"172.21.15.161:6836","nonce":196093187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6136,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4029369301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4029369301},{"type":"v1","addr":"172.21.15.161:6837","nonce":4029369301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6139,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3212560487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3212560487},{"type":"v1","addr":"172.21.15.179:6834","nonce":3212560487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:17:14.012254+0000","modified":"2024-05-22T21:17:17.029283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6306":{"gid":6306,"name":"b","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/3065973138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3065973138},{"type":"v1","addr":"172.21.15.179:6835","nonce":3065973138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-22T21:17:22.278 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 39 2024-05-22T21:17:22.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.276+0000 7f59d2a00640 1 -- 172.21.15.161:0/1180489577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59a8066690 msgr2=0x7f59a8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:22.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.276+0000 7f59d2a00640 1 --2- 172.21.15.161:0/1180489577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59a8066690 0x7f59a8068b50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0011f0 tx=0x7f59bc009040 comp rx=0 tx=0).stop 2024-05-22T21:17:22.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.276+0000 7f59d2a00640 1 -- 172.21.15.161:0/1180489577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15e130 msgr2=0x7f59cc152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:17:22.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.276+0000 7f59d2a00640 1 --2- 172.21.15.161:0/1180489577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15e130 0x7f59cc152e40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f59c401a570 tx=0x7f59c40196e0 comp rx=0 tx=0).stop 2024-05-22T21:17:22.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.276+0000 7f59d2a00640 1 -- 172.21.15.161:0/1180489577 shutdown_connections 2024-05-22T21:17:22.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.276+0000 7f59d2a00640 1 --2- 172.21.15.161:0/1180489577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59a8066690 0x7f59a8068b50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:22.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.276+0000 7f59d2a00640 1 --2- 172.21.15.161:0/1180489577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15e130 0x7f59cc152e40 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:22.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.276+0000 7f59d2a00640 1 --2- 172.21.15.161:0/1180489577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59cc15dd50 0x7f59cc14afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:22.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.276+0000 7f59d2a00640 1 --2- 172.21.15.161:0/1180489577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc15c6d0 0x7f59cc157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:17:22.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.276+0000 7f59d2a00640 1 -- 172.21.15.161:0/1180489577 >> 172.21.15.161:0/1180489577 conn(0x7f59cc0b0bc0 msgr2=0x7f59cc06d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:17:22.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.276+0000 7f59d2a00640 1 -- 172.21.15.161:0/1180489577 shutdown_connections 2024-05-22T21:17:22.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:17:22.276+0000 7f59d2a00640 1 -- 172.21.15.161:0/1180489577 wait complete. 2024-05-22T21:17:22.300 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-05-22T21:17:22.455 INFO:teuthology.orchestra.run.smithi179.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":6306,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.4246984030000007,"mdsmap_epoch":39,"osdmap_epoch":84,"osdmap_epoch_barrier":84,"uptime":12.634532460999999} 2024-05-22T21:17:22.456 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 6306, "mdsmap_epoch": 39, "osdmap_epoch": 84, "osdmap_epoch_barrier": 84, "rank_uptime": 8.424698403, "state": "up:active", "uptime": 12.634532461, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:17:22.456 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-05-22T21:17:22.608 INFO:teuthology.orchestra.run.smithi161.stdout:{"osd_mon_report_interval":"5"} 2024-05-22T21:17:22.608 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-05-22T21:17:22.608 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-05-22T21:17:22.754 INFO:teuthology.orchestra.run.smithi161.stdout:{"mon_tick_interval":"5"} 2024-05-22T21:17:22.754 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-05-22T21:17:22.754 INFO:tasks.cephfs.test_full:pool capacity 557061696, 531MB should be enough to fill it 2024-05-22T21:17:22.755 DEBUG:teuthology.orchestra.run.smithi161:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-22T21:17:22.755 DEBUG:teuthology.orchestra.run.smithi161:> import time 2024-05-22T21:17:22.755 DEBUG:teuthology.orchestra.run.smithi161:> import datetime 2024-05-22T21:17:22.755 DEBUG:teuthology.orchestra.run.smithi161:> import subprocess 2024-05-22T21:17:22.755 DEBUG:teuthology.orchestra.run.smithi161:> import os 2024-05-22T21:17:22.755 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:17:22.755 DEBUG:teuthology.orchestra.run.smithi161:> # Write some buffered data through before going full, all should be well 2024-05-22T21:17:22.755 DEBUG:teuthology.orchestra.run.smithi161:> print("writing some data through which we expect to succeed") 2024-05-22T21:17:22.755 DEBUG:teuthology.orchestra.run.smithi161:> bytes = 0 2024-05-22T21:17:22.755 DEBUG:teuthology.orchestra.run.smithi161:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-05-22T21:17:22.755 DEBUG:teuthology.orchestra.run.smithi161:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-05-22T21:17:22.755 DEBUG:teuthology.orchestra.run.smithi161:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2024-05-22T21:17:22.756 DEBUG:teuthology.orchestra.run.smithi161:> os.fsync(f) 2024-05-22T21:17:22.756 DEBUG:teuthology.orchestra.run.smithi161:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-05-22T21:17:22.756 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:17:22.756 DEBUG:teuthology.orchestra.run.smithi161:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-05-22T21:17:22.756 DEBUG:teuthology.orchestra.run.smithi161:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-05-22T21:17:22.756 DEBUG:teuthology.orchestra.run.smithi161:> # buffered IO, the error should always only appear from fsync and not 2024-05-22T21:17:22.756 DEBUG:teuthology.orchestra.run.smithi161:> # from write 2024-05-22T21:17:22.756 DEBUG:teuthology.orchestra.run.smithi161:> full = False 2024-05-22T21:17:22.756 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:17:22.756 DEBUG:teuthology.orchestra.run.smithi161:> for n in range(0, int(531 * 1.1)): 2024-05-22T21:17:22.756 DEBUG:teuthology.orchestra.run.smithi161:> try: 2024-05-22T21:17:22.756 DEBUG:teuthology.orchestra.run.smithi161:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-05-22T21:17:22.756 DEBUG:teuthology.orchestra.run.smithi161:> print("wrote bytes via buffered write, moving on to fsync") 2024-05-22T21:17:22.756 DEBUG:teuthology.orchestra.run.smithi161:> except OSError as e: 2024-05-22T21:17:22.757 DEBUG:teuthology.orchestra.run.smithi161:> if False: 2024-05-22T21:17:22.757 DEBUG:teuthology.orchestra.run.smithi161:> print("Unexpected error %s from write() instead of fsync()" % e) 2024-05-22T21:17:22.757 DEBUG:teuthology.orchestra.run.smithi161:> raise 2024-05-22T21:17:22.757 DEBUG:teuthology.orchestra.run.smithi161:> else: 2024-05-22T21:17:22.757 DEBUG:teuthology.orchestra.run.smithi161:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-05-22T21:17:22.757 DEBUG:teuthology.orchestra.run.smithi161:> full = True 2024-05-22T21:17:22.757 DEBUG:teuthology.orchestra.run.smithi161:> break 2024-05-22T21:17:22.757 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:17:22.757 DEBUG:teuthology.orchestra.run.smithi161:> try: 2024-05-22T21:17:22.757 DEBUG:teuthology.orchestra.run.smithi161:> os.fsync(f) 2024-05-22T21:17:22.757 DEBUG:teuthology.orchestra.run.smithi161:> print("fsync'"'"'ed successfully") 2024-05-22T21:17:22.757 DEBUG:teuthology.orchestra.run.smithi161:> except OSError as e: 2024-05-22T21:17:22.757 DEBUG:teuthology.orchestra.run.smithi161:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-05-22T21:17:22.757 DEBUG:teuthology.orchestra.run.smithi161:> full = True 2024-05-22T21:17:22.757 DEBUG:teuthology.orchestra.run.smithi161:> break 2024-05-22T21:17:22.757 DEBUG:teuthology.orchestra.run.smithi161:> else: 2024-05-22T21:17:22.758 DEBUG:teuthology.orchestra.run.smithi161:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-05-22T21:17:22.758 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:17:22.758 DEBUG:teuthology.orchestra.run.smithi161:> if n > 531 * 0.9: 2024-05-22T21:17:22.758 DEBUG:teuthology.orchestra.run.smithi161:> # Be cautious in the last region where we expect to hit 2024-05-22T21:17:22.758 DEBUG:teuthology.orchestra.run.smithi161:> # the full condition, so that we don'"'"'t overshoot too dramatically 2024-05-22T21:17:22.758 DEBUG:teuthology.orchestra.run.smithi161:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-05-22T21:17:22.758 DEBUG:teuthology.orchestra.run.smithi161:> time.sleep(15.0) 2024-05-22T21:17:22.758 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:17:22.758 DEBUG:teuthology.orchestra.run.smithi161:> if not full: 2024-05-22T21:17:22.758 DEBUG:teuthology.orchestra.run.smithi161:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2024-05-22T21:17:22.758 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:17:22.758 DEBUG:teuthology.orchestra.run.smithi161:> # close() should not raise an error because we already caught it in 2024-05-22T21:17:22.758 DEBUG:teuthology.orchestra.run.smithi161:> # fsync. There shouldn'"'"'t have been any more writeback errors 2024-05-22T21:17:22.758 DEBUG:teuthology.orchestra.run.smithi161:> # since then because all IOs got cancelled on the full flag. 2024-05-22T21:17:22.759 DEBUG:teuthology.orchestra.run.smithi161:> print("calling close") 2024-05-22T21:17:22.759 DEBUG:teuthology.orchestra.run.smithi161:> os.close(f) 2024-05-22T21:17:22.759 DEBUG:teuthology.orchestra.run.smithi161:> print("close() did not raise error") 2024-05-22T21:17:22.759 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:17:22.759 DEBUG:teuthology.orchestra.run.smithi161:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-05-22T21:17:22.759 DEBUG:teuthology.orchestra.run.smithi161:> ' 2024-05-22T21:17:23.799 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:17:23.802 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:17:25.626 INFO:teuthology.orchestra.run.smithi161.stdout:writing some data through which we expect to succeed 2024-05-22T21:17:25.627 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-05-22T21:17:25.627 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.627 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.627 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 1.00 MB 2024-05-22T21:17:25.627 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.627 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.627 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 2.00 MB 2024-05-22T21:17:25.627 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.627 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.627 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 3.00 MB 2024-05-22T21:17:25.628 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.628 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.628 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 4.00 MB 2024-05-22T21:17:25.628 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.628 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.628 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 5.00 MB 2024-05-22T21:17:25.628 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.628 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.628 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 6.00 MB 2024-05-22T21:17:25.628 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.629 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.629 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 7.00 MB 2024-05-22T21:17:25.629 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.629 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.629 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 8.00 MB 2024-05-22T21:17:25.629 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.629 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.629 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 9.00 MB 2024-05-22T21:17:25.629 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.629 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.629 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 10.00 MB 2024-05-22T21:17:25.630 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.630 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.630 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 11.00 MB 2024-05-22T21:17:25.630 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.630 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.630 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 12.00 MB 2024-05-22T21:17:25.630 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.630 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.630 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 13.00 MB 2024-05-22T21:17:25.630 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.630 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.631 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 14.00 MB 2024-05-22T21:17:25.631 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.631 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.631 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 15.00 MB 2024-05-22T21:17:25.631 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.631 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.631 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 16.00 MB 2024-05-22T21:17:25.631 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.631 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.631 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 17.00 MB 2024-05-22T21:17:25.632 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.632 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.632 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 18.00 MB 2024-05-22T21:17:25.632 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.632 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.632 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 19.00 MB 2024-05-22T21:17:25.632 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.632 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.632 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 20.00 MB 2024-05-22T21:17:25.632 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.632 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.633 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 21.00 MB 2024-05-22T21:17:25.633 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.633 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.633 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 22.00 MB 2024-05-22T21:17:25.633 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.633 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.633 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 23.00 MB 2024-05-22T21:17:25.633 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.633 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.633 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 24.00 MB 2024-05-22T21:17:25.633 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.634 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.634 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 25.00 MB 2024-05-22T21:17:25.634 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.634 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.634 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 26.00 MB 2024-05-22T21:17:25.634 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.634 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.634 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 27.00 MB 2024-05-22T21:17:25.634 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.634 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.635 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 28.00 MB 2024-05-22T21:17:25.635 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.635 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.635 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 29.00 MB 2024-05-22T21:17:25.635 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.635 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.635 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 30.00 MB 2024-05-22T21:17:25.635 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.635 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.635 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 31.00 MB 2024-05-22T21:17:25.635 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.635 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.636 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 32.00 MB 2024-05-22T21:17:25.636 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.636 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.636 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 33.00 MB 2024-05-22T21:17:25.636 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.636 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.636 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 34.00 MB 2024-05-22T21:17:25.636 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.636 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.636 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 35.00 MB 2024-05-22T21:17:25.636 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.637 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.637 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 36.00 MB 2024-05-22T21:17:25.637 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.637 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.637 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 37.00 MB 2024-05-22T21:17:25.637 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.637 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.637 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 38.00 MB 2024-05-22T21:17:25.637 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.637 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.637 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 39.00 MB 2024-05-22T21:17:25.637 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.638 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.638 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 40.00 MB 2024-05-22T21:17:25.638 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.638 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.638 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 41.00 MB 2024-05-22T21:17:25.638 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.638 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.638 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 42.00 MB 2024-05-22T21:17:25.638 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.638 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.639 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 43.00 MB 2024-05-22T21:17:25.639 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.639 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.639 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 44.00 MB 2024-05-22T21:17:25.639 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.639 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.639 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 45.00 MB 2024-05-22T21:17:25.639 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.639 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.639 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 46.00 MB 2024-05-22T21:17:25.639 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.639 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.640 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 47.00 MB 2024-05-22T21:17:25.640 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.640 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.640 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 48.00 MB 2024-05-22T21:17:25.640 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.640 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.640 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 49.00 MB 2024-05-22T21:17:25.640 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.640 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.640 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 50.00 MB 2024-05-22T21:17:25.640 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.640 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.641 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 51.00 MB 2024-05-22T21:17:25.641 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.641 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.641 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 52.00 MB 2024-05-22T21:17:25.641 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.641 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.641 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 53.00 MB 2024-05-22T21:17:25.641 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.641 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.641 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 54.00 MB 2024-05-22T21:17:25.641 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.641 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.641 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 55.00 MB 2024-05-22T21:17:25.642 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.642 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.642 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 56.00 MB 2024-05-22T21:17:25.642 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.642 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.642 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 57.00 MB 2024-05-22T21:17:25.642 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.642 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.642 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 58.00 MB 2024-05-22T21:17:25.642 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.642 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.643 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 59.00 MB 2024-05-22T21:17:25.643 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.643 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.643 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 60.00 MB 2024-05-22T21:17:25.643 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.643 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.643 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 61.00 MB 2024-05-22T21:17:25.643 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.643 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.643 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 62.00 MB 2024-05-22T21:17:25.643 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.643 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.644 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 63.00 MB 2024-05-22T21:17:25.644 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.644 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.644 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 64.00 MB 2024-05-22T21:17:25.644 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.644 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.644 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 65.00 MB 2024-05-22T21:17:25.644 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.644 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.644 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 66.00 MB 2024-05-22T21:17:25.644 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.644 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.644 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 67.00 MB 2024-05-22T21:17:25.645 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.645 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.645 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 68.00 MB 2024-05-22T21:17:25.645 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.645 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.645 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 69.00 MB 2024-05-22T21:17:25.645 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.645 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.645 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 70.00 MB 2024-05-22T21:17:25.645 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.645 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.645 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 71.00 MB 2024-05-22T21:17:25.646 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.646 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.646 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 72.00 MB 2024-05-22T21:17:25.646 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.646 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.646 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 73.00 MB 2024-05-22T21:17:25.646 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.646 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.646 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 74.00 MB 2024-05-22T21:17:25.646 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.646 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.647 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 75.00 MB 2024-05-22T21:17:25.647 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.647 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.647 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 76.00 MB 2024-05-22T21:17:25.647 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.647 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.647 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 77.00 MB 2024-05-22T21:17:25.647 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.647 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.647 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 78.00 MB 2024-05-22T21:17:25.647 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.647 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.647 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 79.00 MB 2024-05-22T21:17:25.648 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:25.648 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:25.648 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 80.00 MB 2024-05-22T21:17:26.387 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.387 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.387 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 81.00 MB 2024-05-22T21:17:26.387 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.387 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.387 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 82.00 MB 2024-05-22T21:17:26.387 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.387 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.388 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 83.00 MB 2024-05-22T21:17:26.388 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.388 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.388 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 84.00 MB 2024-05-22T21:17:26.388 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.388 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.388 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 85.00 MB 2024-05-22T21:17:26.388 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.388 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.388 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 86.00 MB 2024-05-22T21:17:26.388 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.388 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.388 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 87.00 MB 2024-05-22T21:17:26.389 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.389 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.389 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 88.00 MB 2024-05-22T21:17:26.389 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.389 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.389 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 89.00 MB 2024-05-22T21:17:26.389 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.389 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.389 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 90.00 MB 2024-05-22T21:17:26.389 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.389 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.389 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 91.00 MB 2024-05-22T21:17:26.389 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.390 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.390 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 92.00 MB 2024-05-22T21:17:26.390 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.390 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.390 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 93.00 MB 2024-05-22T21:17:26.390 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.390 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.390 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 94.00 MB 2024-05-22T21:17:26.390 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.390 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.390 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 95.00 MB 2024-05-22T21:17:26.390 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.390 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.391 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 96.00 MB 2024-05-22T21:17:26.391 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.391 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.391 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 97.00 MB 2024-05-22T21:17:26.391 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.391 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.391 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 98.00 MB 2024-05-22T21:17:26.391 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.391 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.391 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 99.00 MB 2024-05-22T21:17:26.391 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.391 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.391 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 100.00 MB 2024-05-22T21:17:26.392 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.392 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.392 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 101.00 MB 2024-05-22T21:17:26.392 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.392 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.392 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 102.00 MB 2024-05-22T21:17:26.392 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.392 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.392 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 103.00 MB 2024-05-22T21:17:26.392 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.392 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.392 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 104.00 MB 2024-05-22T21:17:26.392 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.393 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.393 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 105.00 MB 2024-05-22T21:17:26.393 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.393 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.393 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 106.00 MB 2024-05-22T21:17:26.393 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.393 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.393 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 107.00 MB 2024-05-22T21:17:26.393 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.393 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.393 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 108.00 MB 2024-05-22T21:17:26.393 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.394 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.394 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 109.00 MB 2024-05-22T21:17:26.394 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.394 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.394 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 110.00 MB 2024-05-22T21:17:26.394 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.394 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.394 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 111.00 MB 2024-05-22T21:17:26.394 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.395 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.395 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 112.00 MB 2024-05-22T21:17:26.395 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.395 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.395 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 113.00 MB 2024-05-22T21:17:26.395 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.395 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.395 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 114.00 MB 2024-05-22T21:17:26.395 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.395 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.395 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 115.00 MB 2024-05-22T21:17:26.395 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.396 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.396 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 116.00 MB 2024-05-22T21:17:26.396 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.396 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.396 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 117.00 MB 2024-05-22T21:17:26.396 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.396 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.396 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 118.00 MB 2024-05-22T21:17:26.396 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.396 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.396 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 119.00 MB 2024-05-22T21:17:26.396 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.396 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.397 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 120.00 MB 2024-05-22T21:17:26.397 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.397 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.397 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 121.00 MB 2024-05-22T21:17:26.397 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.397 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.397 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 122.00 MB 2024-05-22T21:17:26.397 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.397 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.397 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 123.00 MB 2024-05-22T21:17:26.397 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.398 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.398 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 124.00 MB 2024-05-22T21:17:26.398 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.398 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.398 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 125.00 MB 2024-05-22T21:17:26.398 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.398 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.398 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 126.00 MB 2024-05-22T21:17:26.398 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.398 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.402 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 127.00 MB 2024-05-22T21:17:26.402 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.402 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.402 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 128.00 MB 2024-05-22T21:17:26.402 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.402 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.402 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 129.00 MB 2024-05-22T21:17:26.403 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.403 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.403 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 130.00 MB 2024-05-22T21:17:26.403 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.403 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.403 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 131.00 MB 2024-05-22T21:17:26.403 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.403 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.403 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 132.00 MB 2024-05-22T21:17:26.403 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.403 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.403 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 133.00 MB 2024-05-22T21:17:26.403 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.404 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.404 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 134.00 MB 2024-05-22T21:17:26.404 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.404 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.404 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 135.00 MB 2024-05-22T21:17:26.404 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.404 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.404 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 136.00 MB 2024-05-22T21:17:26.404 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.404 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.404 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 137.00 MB 2024-05-22T21:17:26.404 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.404 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.404 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 138.00 MB 2024-05-22T21:17:26.404 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.405 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.405 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 139.00 MB 2024-05-22T21:17:26.405 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.405 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.405 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 140.00 MB 2024-05-22T21:17:26.405 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.405 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.405 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 141.00 MB 2024-05-22T21:17:26.405 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.405 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.405 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 142.00 MB 2024-05-22T21:17:26.405 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.405 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.405 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 143.00 MB 2024-05-22T21:17:26.406 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.406 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.406 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 144.00 MB 2024-05-22T21:17:26.406 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.406 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.406 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 145.00 MB 2024-05-22T21:17:26.406 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.406 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.406 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 146.00 MB 2024-05-22T21:17:26.406 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.406 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.406 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 147.00 MB 2024-05-22T21:17:26.406 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.407 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.407 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 148.00 MB 2024-05-22T21:17:26.407 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.407 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.407 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 149.00 MB 2024-05-22T21:17:26.407 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.407 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.407 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 150.00 MB 2024-05-22T21:17:26.407 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.407 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.407 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 151.00 MB 2024-05-22T21:17:26.407 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.407 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.408 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 152.00 MB 2024-05-22T21:17:26.408 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.408 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.408 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 153.00 MB 2024-05-22T21:17:26.408 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.408 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.408 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 154.00 MB 2024-05-22T21:17:26.408 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.408 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.408 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 155.00 MB 2024-05-22T21:17:26.408 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.408 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.408 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 156.00 MB 2024-05-22T21:17:26.408 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.409 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.409 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 157.00 MB 2024-05-22T21:17:26.409 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.409 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.409 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 158.00 MB 2024-05-22T21:17:26.409 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.409 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.409 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 159.00 MB 2024-05-22T21:17:26.409 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:26.409 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:26.409 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 160.00 MB 2024-05-22T21:17:26.409 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.148 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.148 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 161.00 MB 2024-05-22T21:17:27.149 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.149 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.149 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 162.00 MB 2024-05-22T21:17:27.149 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.149 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.149 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 163.00 MB 2024-05-22T21:17:27.149 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.149 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.149 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 164.00 MB 2024-05-22T21:17:27.149 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.149 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.149 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 165.00 MB 2024-05-22T21:17:27.150 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.150 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.150 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 166.00 MB 2024-05-22T21:17:27.150 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.150 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.150 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 167.00 MB 2024-05-22T21:17:27.150 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.150 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.150 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 168.00 MB 2024-05-22T21:17:27.150 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.150 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.150 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 169.00 MB 2024-05-22T21:17:27.150 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.151 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.151 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 170.00 MB 2024-05-22T21:17:27.151 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.151 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.151 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 171.00 MB 2024-05-22T21:17:27.151 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.151 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.151 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 172.00 MB 2024-05-22T21:17:27.151 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.151 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.151 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 173.00 MB 2024-05-22T21:17:27.151 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.151 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.152 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 174.00 MB 2024-05-22T21:17:27.152 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.152 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.152 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 175.00 MB 2024-05-22T21:17:27.152 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.152 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.152 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 176.00 MB 2024-05-22T21:17:27.152 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.152 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.152 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 177.00 MB 2024-05-22T21:17:27.152 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.152 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.152 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 178.00 MB 2024-05-22T21:17:27.152 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.153 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.153 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 179.00 MB 2024-05-22T21:17:27.153 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.153 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.153 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 180.00 MB 2024-05-22T21:17:27.153 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.153 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.153 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 181.00 MB 2024-05-22T21:17:27.153 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.153 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.153 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 182.00 MB 2024-05-22T21:17:27.153 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.153 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.154 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 183.00 MB 2024-05-22T21:17:27.154 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.154 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.154 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 184.00 MB 2024-05-22T21:17:27.154 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.154 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.154 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 185.00 MB 2024-05-22T21:17:27.154 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.154 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.154 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 186.00 MB 2024-05-22T21:17:27.154 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.154 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.154 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 187.00 MB 2024-05-22T21:17:27.154 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.155 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.155 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 188.00 MB 2024-05-22T21:17:27.155 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.155 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.155 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 189.00 MB 2024-05-22T21:17:27.155 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.155 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.155 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 190.00 MB 2024-05-22T21:17:27.155 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.155 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.155 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 191.00 MB 2024-05-22T21:17:27.155 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.155 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.156 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 192.00 MB 2024-05-22T21:17:27.156 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.156 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.156 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 193.00 MB 2024-05-22T21:17:27.156 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.156 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.156 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 194.00 MB 2024-05-22T21:17:27.156 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.156 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.156 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 195.00 MB 2024-05-22T21:17:27.156 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.156 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.156 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 196.00 MB 2024-05-22T21:17:27.157 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.157 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.157 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 197.00 MB 2024-05-22T21:17:27.157 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.157 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.157 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 198.00 MB 2024-05-22T21:17:27.157 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.157 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.157 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 199.00 MB 2024-05-22T21:17:27.157 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.157 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.157 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 200.00 MB 2024-05-22T21:17:27.157 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.158 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.158 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 201.00 MB 2024-05-22T21:17:27.158 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.158 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.158 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 202.00 MB 2024-05-22T21:17:27.158 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.158 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.158 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 203.00 MB 2024-05-22T21:17:27.158 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.158 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.158 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 204.00 MB 2024-05-22T21:17:27.158 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.158 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.159 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 205.00 MB 2024-05-22T21:17:27.159 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.159 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.159 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 206.00 MB 2024-05-22T21:17:27.159 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.159 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.159 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 207.00 MB 2024-05-22T21:17:27.159 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.159 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.159 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 208.00 MB 2024-05-22T21:17:27.159 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.159 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.159 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 209.00 MB 2024-05-22T21:17:27.159 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.160 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.160 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 210.00 MB 2024-05-22T21:17:27.160 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.160 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.160 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 211.00 MB 2024-05-22T21:17:27.160 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.160 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.160 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 212.00 MB 2024-05-22T21:17:27.160 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.160 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.160 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 213.00 MB 2024-05-22T21:17:27.160 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.160 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.161 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 214.00 MB 2024-05-22T21:17:27.161 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.161 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.161 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 215.00 MB 2024-05-22T21:17:27.161 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.161 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.161 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 216.00 MB 2024-05-22T21:17:27.161 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.161 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.161 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 217.00 MB 2024-05-22T21:17:27.161 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.161 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.161 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 218.00 MB 2024-05-22T21:17:27.162 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.162 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.162 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 219.00 MB 2024-05-22T21:17:27.162 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.162 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.162 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 220.00 MB 2024-05-22T21:17:27.162 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.162 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.162 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 221.00 MB 2024-05-22T21:17:27.162 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.162 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.162 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 222.00 MB 2024-05-22T21:17:27.162 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.163 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.163 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 223.00 MB 2024-05-22T21:17:27.163 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.163 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.163 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 224.00 MB 2024-05-22T21:17:27.163 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.163 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.163 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 225.00 MB 2024-05-22T21:17:27.163 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.163 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.163 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 226.00 MB 2024-05-22T21:17:27.163 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.163 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.163 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 227.00 MB 2024-05-22T21:17:27.164 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.164 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.164 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 228.00 MB 2024-05-22T21:17:27.164 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.164 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.164 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 229.00 MB 2024-05-22T21:17:27.164 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.164 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.164 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 230.00 MB 2024-05-22T21:17:27.164 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.164 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.164 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 231.00 MB 2024-05-22T21:17:27.164 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.165 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.165 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 232.00 MB 2024-05-22T21:17:27.165 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.165 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.165 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 233.00 MB 2024-05-22T21:17:27.165 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.165 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.165 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 234.00 MB 2024-05-22T21:17:27.165 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.165 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.165 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 235.00 MB 2024-05-22T21:17:27.165 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.165 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.165 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 236.00 MB 2024-05-22T21:17:27.166 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.166 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.166 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 237.00 MB 2024-05-22T21:17:27.166 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.166 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.166 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 238.00 MB 2024-05-22T21:17:27.166 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.166 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.166 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 239.00 MB 2024-05-22T21:17:27.166 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.166 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.166 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 240.00 MB 2024-05-22T21:17:27.166 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.167 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.890 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 241.00 MB 2024-05-22T21:17:27.890 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.890 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.890 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 242.00 MB 2024-05-22T21:17:27.890 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.890 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.890 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 243.00 MB 2024-05-22T21:17:27.890 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.891 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.891 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 244.00 MB 2024-05-22T21:17:27.891 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.891 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.891 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 245.00 MB 2024-05-22T21:17:27.891 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.891 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.891 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 246.00 MB 2024-05-22T21:17:27.891 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.891 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.891 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 247.00 MB 2024-05-22T21:17:27.892 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.892 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.892 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 248.00 MB 2024-05-22T21:17:27.892 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.892 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.892 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 249.00 MB 2024-05-22T21:17:27.892 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.892 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.892 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 250.00 MB 2024-05-22T21:17:27.892 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.892 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.892 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 251.00 MB 2024-05-22T21:17:27.893 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.893 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.893 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 252.00 MB 2024-05-22T21:17:27.893 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.893 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.893 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 253.00 MB 2024-05-22T21:17:27.893 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.893 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.893 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 254.00 MB 2024-05-22T21:17:27.893 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.893 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.893 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 255.00 MB 2024-05-22T21:17:27.893 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.894 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.894 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 256.00 MB 2024-05-22T21:17:27.894 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.894 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.894 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 257.00 MB 2024-05-22T21:17:27.894 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.894 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.894 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 258.00 MB 2024-05-22T21:17:27.894 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.894 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.894 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 259.00 MB 2024-05-22T21:17:27.895 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.895 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.895 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 260.00 MB 2024-05-22T21:17:27.895 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.895 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.895 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 261.00 MB 2024-05-22T21:17:27.895 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.895 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.895 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 262.00 MB 2024-05-22T21:17:27.895 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.895 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.895 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 263.00 MB 2024-05-22T21:17:27.895 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.896 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.896 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 264.00 MB 2024-05-22T21:17:27.896 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.896 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.896 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 265.00 MB 2024-05-22T21:17:27.896 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.896 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.896 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 266.00 MB 2024-05-22T21:17:27.896 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.897 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.897 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 267.00 MB 2024-05-22T21:17:27.897 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.897 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.897 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 268.00 MB 2024-05-22T21:17:27.897 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.897 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.897 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 269.00 MB 2024-05-22T21:17:27.897 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.897 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.897 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 270.00 MB 2024-05-22T21:17:27.897 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.897 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.898 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 271.00 MB 2024-05-22T21:17:27.898 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.898 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.898 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 272.00 MB 2024-05-22T21:17:27.898 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.898 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.898 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 273.00 MB 2024-05-22T21:17:27.898 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.898 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.898 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 274.00 MB 2024-05-22T21:17:27.898 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.898 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.898 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 275.00 MB 2024-05-22T21:17:27.899 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.899 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.899 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 276.00 MB 2024-05-22T21:17:27.899 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.899 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.899 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 277.00 MB 2024-05-22T21:17:27.899 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.899 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.899 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 278.00 MB 2024-05-22T21:17:27.899 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.899 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.899 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 279.00 MB 2024-05-22T21:17:27.899 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.900 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.900 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 280.00 MB 2024-05-22T21:17:27.900 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.900 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.900 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 281.00 MB 2024-05-22T21:17:27.900 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.900 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.900 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 282.00 MB 2024-05-22T21:17:27.900 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.900 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.900 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 283.00 MB 2024-05-22T21:17:27.900 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.900 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.901 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 284.00 MB 2024-05-22T21:17:27.901 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.901 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.901 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 285.00 MB 2024-05-22T21:17:27.901 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.901 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.901 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 286.00 MB 2024-05-22T21:17:27.901 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.901 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.901 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 287.00 MB 2024-05-22T21:17:27.901 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.901 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.901 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 288.00 MB 2024-05-22T21:17:27.902 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.902 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.902 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 289.00 MB 2024-05-22T21:17:27.902 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.902 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.902 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 290.00 MB 2024-05-22T21:17:27.902 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.902 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.902 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 291.00 MB 2024-05-22T21:17:27.902 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.902 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.902 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 292.00 MB 2024-05-22T21:17:27.902 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.903 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.903 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 293.00 MB 2024-05-22T21:17:27.903 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.903 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.903 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 294.00 MB 2024-05-22T21:17:27.903 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.903 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.903 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 295.00 MB 2024-05-22T21:17:27.903 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.903 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.903 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 296.00 MB 2024-05-22T21:17:27.903 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.903 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.904 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 297.00 MB 2024-05-22T21:17:27.904 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.904 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.904 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 298.00 MB 2024-05-22T21:17:27.904 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.904 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.904 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 299.00 MB 2024-05-22T21:17:27.904 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.904 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.904 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 300.00 MB 2024-05-22T21:17:27.904 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.904 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.904 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 301.00 MB 2024-05-22T21:17:27.905 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.905 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.905 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 302.00 MB 2024-05-22T21:17:27.905 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.905 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.905 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 303.00 MB 2024-05-22T21:17:27.905 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.905 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.905 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 304.00 MB 2024-05-22T21:17:27.905 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.906 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.906 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 305.00 MB 2024-05-22T21:17:27.906 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.906 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.906 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 306.00 MB 2024-05-22T21:17:27.906 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.906 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.906 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 307.00 MB 2024-05-22T21:17:27.906 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.906 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.906 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 308.00 MB 2024-05-22T21:17:27.906 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.907 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.907 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 309.00 MB 2024-05-22T21:17:27.907 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.907 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.907 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 310.00 MB 2024-05-22T21:17:27.907 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.907 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.907 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 311.00 MB 2024-05-22T21:17:27.907 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.907 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.907 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 312.00 MB 2024-05-22T21:17:27.907 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.907 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.908 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 313.00 MB 2024-05-22T21:17:27.908 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.908 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.908 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 314.00 MB 2024-05-22T21:17:27.908 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.908 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.908 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 315.00 MB 2024-05-22T21:17:27.908 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.908 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.908 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 316.00 MB 2024-05-22T21:17:27.908 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.909 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.909 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 317.00 MB 2024-05-22T21:17:27.909 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.909 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.909 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 318.00 MB 2024-05-22T21:17:27.909 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.909 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.909 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 319.00 MB 2024-05-22T21:17:27.909 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.909 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.909 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 320.00 MB 2024-05-22T21:17:27.909 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:27.909 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:27.910 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 321.00 MB 2024-05-22T21:17:28.732 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.732 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.732 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 322.00 MB 2024-05-22T21:17:28.732 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.732 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.732 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 323.00 MB 2024-05-22T21:17:28.732 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.732 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.732 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 324.00 MB 2024-05-22T21:17:28.732 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.733 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.733 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 325.00 MB 2024-05-22T21:17:28.733 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.733 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.733 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 326.00 MB 2024-05-22T21:17:28.733 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.733 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.733 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 327.00 MB 2024-05-22T21:17:28.733 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.733 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.733 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 328.00 MB 2024-05-22T21:17:28.733 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.733 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.734 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 329.00 MB 2024-05-22T21:17:28.734 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.734 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.734 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 330.00 MB 2024-05-22T21:17:28.734 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.734 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.734 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 331.00 MB 2024-05-22T21:17:28.734 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.734 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.734 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 332.00 MB 2024-05-22T21:17:28.734 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.734 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.735 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 333.00 MB 2024-05-22T21:17:28.735 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.735 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.735 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 334.00 MB 2024-05-22T21:17:28.735 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.735 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.735 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 335.00 MB 2024-05-22T21:17:28.735 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.735 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.735 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 336.00 MB 2024-05-22T21:17:28.735 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.735 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.735 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 337.00 MB 2024-05-22T21:17:28.736 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.736 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.736 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 338.00 MB 2024-05-22T21:17:28.736 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.736 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.736 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 339.00 MB 2024-05-22T21:17:28.736 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.736 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.736 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 340.00 MB 2024-05-22T21:17:28.736 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.736 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.736 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 341.00 MB 2024-05-22T21:17:28.736 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.737 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.737 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 342.00 MB 2024-05-22T21:17:28.737 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.737 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.737 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 343.00 MB 2024-05-22T21:17:28.737 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.737 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.737 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 344.00 MB 2024-05-22T21:17:28.737 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.737 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.737 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 345.00 MB 2024-05-22T21:17:28.737 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.738 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.738 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 346.00 MB 2024-05-22T21:17:28.738 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.738 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.738 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 347.00 MB 2024-05-22T21:17:28.738 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.738 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.738 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 348.00 MB 2024-05-22T21:17:28.738 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.738 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.738 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 349.00 MB 2024-05-22T21:17:28.738 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.739 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.739 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 350.00 MB 2024-05-22T21:17:28.739 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.739 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.739 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 351.00 MB 2024-05-22T21:17:28.739 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.739 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.739 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 352.00 MB 2024-05-22T21:17:28.739 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.739 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.739 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 353.00 MB 2024-05-22T21:17:28.739 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.739 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.740 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 354.00 MB 2024-05-22T21:17:28.740 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.740 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.740 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 355.00 MB 2024-05-22T21:17:28.740 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.740 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.740 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 356.00 MB 2024-05-22T21:17:28.740 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.740 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.740 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 357.00 MB 2024-05-22T21:17:28.740 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.740 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.740 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 358.00 MB 2024-05-22T21:17:28.741 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.741 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.741 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 359.00 MB 2024-05-22T21:17:28.741 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.741 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.741 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 360.00 MB 2024-05-22T21:17:28.741 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.741 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.741 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 361.00 MB 2024-05-22T21:17:28.741 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.741 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.741 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 362.00 MB 2024-05-22T21:17:28.741 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.741 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.742 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 363.00 MB 2024-05-22T21:17:28.742 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.742 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.742 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 364.00 MB 2024-05-22T21:17:28.742 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.742 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.742 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 365.00 MB 2024-05-22T21:17:28.742 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.742 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.742 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 366.00 MB 2024-05-22T21:17:28.742 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.742 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.743 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 367.00 MB 2024-05-22T21:17:28.743 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.743 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.743 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 368.00 MB 2024-05-22T21:17:28.743 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.743 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.743 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 369.00 MB 2024-05-22T21:17:28.743 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.743 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.743 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 370.00 MB 2024-05-22T21:17:28.743 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.743 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.743 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 371.00 MB 2024-05-22T21:17:28.743 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.743 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.744 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 372.00 MB 2024-05-22T21:17:28.744 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.744 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.744 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 373.00 MB 2024-05-22T21:17:28.744 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.744 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.744 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 374.00 MB 2024-05-22T21:17:28.744 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.744 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.744 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 375.00 MB 2024-05-22T21:17:28.744 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.744 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.744 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 376.00 MB 2024-05-22T21:17:28.744 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.745 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.745 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 377.00 MB 2024-05-22T21:17:28.745 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.745 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.745 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 378.00 MB 2024-05-22T21:17:28.745 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.745 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.745 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 379.00 MB 2024-05-22T21:17:28.745 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.745 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.745 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 380.00 MB 2024-05-22T21:17:28.745 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.745 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.746 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 381.00 MB 2024-05-22T21:17:28.746 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.746 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.746 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 382.00 MB 2024-05-22T21:17:28.746 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.746 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.746 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 383.00 MB 2024-05-22T21:17:28.746 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.746 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.746 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 384.00 MB 2024-05-22T21:17:28.746 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.746 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.747 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 385.00 MB 2024-05-22T21:17:28.747 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.747 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.747 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 386.00 MB 2024-05-22T21:17:28.747 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.747 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.747 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 387.00 MB 2024-05-22T21:17:28.747 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.747 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.747 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 388.00 MB 2024-05-22T21:17:28.747 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.747 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.747 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 389.00 MB 2024-05-22T21:17:28.747 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.748 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.748 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 390.00 MB 2024-05-22T21:17:28.748 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.748 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.748 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 391.00 MB 2024-05-22T21:17:28.748 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.748 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.748 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 392.00 MB 2024-05-22T21:17:28.748 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.748 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.748 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 393.00 MB 2024-05-22T21:17:28.748 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.749 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.749 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 394.00 MB 2024-05-22T21:17:28.749 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.749 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.749 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 395.00 MB 2024-05-22T21:17:28.749 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.749 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.749 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 396.00 MB 2024-05-22T21:17:28.749 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.749 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.749 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 397.00 MB 2024-05-22T21:17:28.749 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.749 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.750 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 398.00 MB 2024-05-22T21:17:28.750 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.750 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.750 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 399.00 MB 2024-05-22T21:17:28.750 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.750 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.750 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 400.00 MB 2024-05-22T21:17:28.750 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:28.750 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:28.750 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 401.00 MB 2024-05-22T21:17:31.554 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:17:31.552+0000 7f7d35a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-05-22T21:17:40.231 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:17:40.228+0000 7f7d35a00640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-05-22T21:17:53.858 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:17:53.861 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:17:59.549 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.549 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.549 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 402.00 MB 2024-05-22T21:17:59.550 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.550 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.550 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 403.00 MB 2024-05-22T21:17:59.550 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.550 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.550 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 404.00 MB 2024-05-22T21:17:59.550 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.550 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.550 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 405.00 MB 2024-05-22T21:17:59.551 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.551 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.551 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 406.00 MB 2024-05-22T21:17:59.551 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.551 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.551 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 407.00 MB 2024-05-22T21:17:59.551 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.551 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.551 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 408.00 MB 2024-05-22T21:17:59.551 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.551 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.551 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 409.00 MB 2024-05-22T21:17:59.552 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.552 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.552 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 410.00 MB 2024-05-22T21:17:59.552 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.552 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.552 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 411.00 MB 2024-05-22T21:17:59.552 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.552 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.552 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 412.00 MB 2024-05-22T21:17:59.552 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.553 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.553 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 413.00 MB 2024-05-22T21:17:59.553 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.553 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.553 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 414.00 MB 2024-05-22T21:17:59.553 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.553 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.553 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 415.00 MB 2024-05-22T21:17:59.553 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.553 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.553 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 416.00 MB 2024-05-22T21:17:59.553 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.554 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.554 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 417.00 MB 2024-05-22T21:17:59.554 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.554 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.554 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 418.00 MB 2024-05-22T21:17:59.554 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.554 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.554 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 419.00 MB 2024-05-22T21:17:59.554 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.554 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.554 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 420.00 MB 2024-05-22T21:17:59.555 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.555 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.555 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 421.00 MB 2024-05-22T21:17:59.555 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.555 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.555 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 422.00 MB 2024-05-22T21:17:59.555 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.555 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.555 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 423.00 MB 2024-05-22T21:17:59.555 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.555 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.555 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 424.00 MB 2024-05-22T21:17:59.556 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.556 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.556 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 425.00 MB 2024-05-22T21:17:59.556 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.556 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.556 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 426.00 MB 2024-05-22T21:17:59.556 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.556 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.556 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 427.00 MB 2024-05-22T21:17:59.556 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.556 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.557 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 428.00 MB 2024-05-22T21:17:59.557 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.557 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.557 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 429.00 MB 2024-05-22T21:17:59.557 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.557 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.557 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 430.00 MB 2024-05-22T21:17:59.557 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.557 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.557 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 431.00 MB 2024-05-22T21:17:59.557 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.558 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.558 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 432.00 MB 2024-05-22T21:17:59.558 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.558 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.558 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 433.00 MB 2024-05-22T21:17:59.558 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.558 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.558 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 434.00 MB 2024-05-22T21:17:59.558 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.558 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.558 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 435.00 MB 2024-05-22T21:17:59.558 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.558 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.559 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 436.00 MB 2024-05-22T21:17:59.559 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.559 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.559 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 437.00 MB 2024-05-22T21:17:59.559 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.559 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.559 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 438.00 MB 2024-05-22T21:17:59.559 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.559 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.559 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 439.00 MB 2024-05-22T21:17:59.559 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.559 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.560 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 440.00 MB 2024-05-22T21:17:59.560 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.560 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.560 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 441.00 MB 2024-05-22T21:17:59.560 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.560 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.560 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 442.00 MB 2024-05-22T21:17:59.560 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.560 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.560 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 443.00 MB 2024-05-22T21:17:59.560 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.560 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.560 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 444.00 MB 2024-05-22T21:17:59.561 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.561 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.561 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 445.00 MB 2024-05-22T21:17:59.561 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.561 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.561 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 446.00 MB 2024-05-22T21:17:59.561 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.561 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.561 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 447.00 MB 2024-05-22T21:17:59.561 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.561 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.562 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 448.00 MB 2024-05-22T21:17:59.562 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.562 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.562 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 449.00 MB 2024-05-22T21:17:59.562 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.562 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.562 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 450.00 MB 2024-05-22T21:17:59.562 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.562 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.562 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 451.00 MB 2024-05-22T21:17:59.562 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.563 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.563 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 452.00 MB 2024-05-22T21:17:59.563 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.563 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.563 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 453.00 MB 2024-05-22T21:17:59.563 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.563 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.563 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 454.00 MB 2024-05-22T21:17:59.563 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.563 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.563 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 455.00 MB 2024-05-22T21:17:59.564 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.564 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.564 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 456.00 MB 2024-05-22T21:17:59.564 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.564 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.564 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 457.00 MB 2024-05-22T21:17:59.564 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.564 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.564 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 458.00 MB 2024-05-22T21:17:59.564 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.565 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.565 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 459.00 MB 2024-05-22T21:17:59.565 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.565 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.565 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 460.00 MB 2024-05-22T21:17:59.565 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.565 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.565 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 461.00 MB 2024-05-22T21:17:59.565 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.565 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.566 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 462.00 MB 2024-05-22T21:17:59.566 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.566 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.566 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 463.00 MB 2024-05-22T21:17:59.566 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.566 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.566 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 464.00 MB 2024-05-22T21:17:59.566 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.566 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.566 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 465.00 MB 2024-05-22T21:17:59.566 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.567 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.567 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 466.00 MB 2024-05-22T21:17:59.567 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.567 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.567 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 467.00 MB 2024-05-22T21:17:59.567 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.567 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.567 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 468.00 MB 2024-05-22T21:17:59.567 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.567 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.568 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 469.00 MB 2024-05-22T21:17:59.568 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.568 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.568 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 470.00 MB 2024-05-22T21:17:59.568 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.568 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.568 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 471.00 MB 2024-05-22T21:17:59.568 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.568 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.568 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 472.00 MB 2024-05-22T21:17:59.568 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.569 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.569 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 473.00 MB 2024-05-22T21:17:59.569 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.569 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.569 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 474.00 MB 2024-05-22T21:17:59.569 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.569 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.569 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 475.00 MB 2024-05-22T21:17:59.569 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.569 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.569 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 476.00 MB 2024-05-22T21:17:59.569 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.570 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.570 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 477.00 MB 2024-05-22T21:17:59.570 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.570 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.570 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 478.00 MB 2024-05-22T21:17:59.570 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.570 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.570 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 479.00 MB 2024-05-22T21:17:59.570 INFO:teuthology.orchestra.run.smithi161.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-05-22T21:17:59.570 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:17:59.570 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:17:59.570 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 480.00 MB 2024-05-22T21:17:59.571 INFO:teuthology.orchestra.run.smithi161.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-05-22T21:17:59.624 INFO:teuthology.orchestra.run.smithi161.stdout:Reached fullness after 480.00 MB 2024-05-22T21:17:59.625 INFO:teuthology.orchestra.run.smithi161.stdout:calling close 2024-05-22T21:17:59.625 INFO:teuthology.orchestra.run.smithi161.stdout:close() did not raise error 2024-05-22T21:17:59.665 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:17:59.666 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:17:59.759 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:17:59.856 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:17:59.857 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:17:59.857 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:17:59.867 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:17:59.867 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:17:59.922 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:18:00.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.100+0000 7f7f7f400640 1 Processor -- start 2024-05-22T21:18:00.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.100+0000 7f7f7f400640 1 -- start start 2024-05-22T21:18:00.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.100+0000 7f7f7f400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7f7815e130 0x7f7f7815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.100+0000 7f7f7f400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7f7815d410 0x7f7f7815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.100+0000 7f7f7f400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f78071670 con 0x7f7f7815dd50 2024-05-22T21:18:00.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.100+0000 7f7f7f400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f7815d870 con 0x7f7f7815e130 2024-05-22T21:18:00.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.100+0000 7f7f7f400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f78070a60 con 0x7f7f7815d410 2024-05-22T21:18:00.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.100+0000 7f7f6fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7f7815e130 0x7f7f7815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.100+0000 7f7f6fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7f7815e130 0x7f7f7815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44926/0 (socket says 172.21.15.161:44926) 2024-05-22T21:18:00.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.100+0000 7f7f6fe00640 1 -- 172.21.15.161:0/510946578 learned_addr learned my addr 172.21.15.161:0/510946578 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:00.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.100+0000 7f7f7d400640 1 --2- 172.21.15.161:0/510946578 >> v2:172.21.15.179:3300/0 conn(0x7f7f7815d410 0x7f7f7815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.100+0000 7f7f6fe00640 1 -- 172.21.15.161:0/510946578 >> v2:172.21.15.179:3300/0 conn(0x7f7f7815d410 msgr2=0x7f7f7815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:00.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.100+0000 7f7f6fe00640 1 --2- 172.21.15.161:0/510946578 >> v2:172.21.15.179:3300/0 conn(0x7f7f7815d410 0x7f7f7815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.100+0000 7f7f6fe00640 1 -- 172.21.15.161:0/510946578 >> v1:172.21.15.179:6789/0 conn(0x7f7f7815dd50 legacy=0x7f7f7806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:00.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.100+0000 7f7f6fe00640 1 -- 172.21.15.161:0/510946578 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f78070d40 con 0x7f7f7815e130 2024-05-22T21:18:00.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.100+0000 7f7f7d400640 1 --2- 172.21.15.161:0/510946578 >> v2:172.21.15.179:3300/0 conn(0x7f7f7815d410 0x7f7f7815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:00.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.104+0000 7f7f6fe00640 1 --2- 172.21.15.161:0/510946578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f7815e130 0x7f7f7815ced0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f7f7815e5c0 tx=0x7f7f6001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18aac9ab86916907 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:00.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.104+0000 7f7f77e00640 1 -- 172.21.15.161:0/510946578 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f600aa650 con 0x7f7f7815e130 2024-05-22T21:18:00.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.104+0000 7f7f77e00640 1 -- 172.21.15.161:0/510946578 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f600aa7b0 con 0x7f7f7815e130 2024-05-22T21:18:00.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.104+0000 7f7f77e00640 1 -- 172.21.15.161:0/510946578 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f600aab40 con 0x7f7f7815e130 2024-05-22T21:18:00.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.104+0000 7f7f7f400640 1 -- 172.21.15.161:0/510946578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f7815e130 msgr2=0x7f7f7815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.104+0000 7f7f7f400640 1 --2- 172.21.15.161:0/510946578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f7815e130 0x7f7f7815ced0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f7f7815e5c0 tx=0x7f7f6001c4f0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.104+0000 7f7f7f400640 1 -- 172.21.15.161:0/510946578 shutdown_connections 2024-05-22T21:18:00.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.104+0000 7f7f7f400640 1 --2- 172.21.15.161:0/510946578 >> v2:172.21.15.179:3300/0 conn(0x7f7f7815d410 0x7f7f7815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.104+0000 7f7f7f400640 1 --2- 172.21.15.161:0/510946578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f7815e130 0x7f7f7815ced0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.104+0000 7f7f7f400640 1 -- 172.21.15.161:0/510946578 >> 172.21.15.161:0/510946578 conn(0x7f7f780b0bc0 msgr2=0x7f7f780b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:00.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.104+0000 7f7f7f400640 1 -- 172.21.15.161:0/510946578 shutdown_connections 2024-05-22T21:18:00.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.104+0000 7f7f7f400640 1 -- 172.21.15.161:0/510946578 wait complete. 2024-05-22T21:18:00.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7f400640 1 Processor -- start 2024-05-22T21:18:00.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7f400640 1 -- start start 2024-05-22T21:18:00.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7f400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f7815d410 0x7f7f7814cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7f400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7f7815dd50 0x7f7f78140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f7815e130 0x7f7f78150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7f400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f78158910 con 0x7f7f7815e130 2024-05-22T21:18:00.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7f400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7814cd60 con 0x7f7f7815d410 2024-05-22T21:18:00.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7f400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f78157d00 con 0x7f7f7815dd50 2024-05-22T21:18:00.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f7815e130 0x7f7f78150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f7815e130 0x7f7f78150160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44940/0 (socket says 172.21.15.161:44940) 2024-05-22T21:18:00.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7d400640 1 -- 172.21.15.161:0/2015487183 learned_addr learned my addr 172.21.15.161:0/2015487183 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:00.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f6fe00640 1 --2- 172.21.15.161:0/2015487183 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7f7815dd50 0x7f7f78140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7ca00640 1 --2- 172.21.15.161:0/2015487183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f7815d410 0x7f7f7814cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7d400640 1 -- 172.21.15.161:0/2015487183 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7f7815dd50 msgr2=0x7f7f78140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7d400640 1 --2- 172.21.15.161:0/2015487183 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7f7815dd50 0x7f7f78140940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7d400640 1 -- 172.21.15.161:0/2015487183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f7815d410 msgr2=0x7f7f7814cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7d400640 1 --2- 172.21.15.161:0/2015487183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f7815d410 0x7f7f7814cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7d400640 1 -- 172.21.15.161:0/2015487183 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f600094d0 con 0x7f7f7815e130 2024-05-22T21:18:00.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f6fe00640 1 --2- 172.21.15.161:0/2015487183 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7f7815dd50 0x7f7f78140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:00.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7ca00640 1 --2- 172.21.15.161:0/2015487183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f7815d410 0x7f7f7814cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:00.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7d400640 1 --2- 172.21.15.161:0/2015487183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f7815e130 0x7f7f78150160 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f7f7000cef0 tx=0x7f7f7000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:00.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f75600640 1 -- 172.21.15.161:0/2015487183 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7000d840 con 0x7f7f7815e130 2024-05-22T21:18:00.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f75600640 1 -- 172.21.15.161:0/2015487183 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7000d9c0 con 0x7f7f7815e130 2024-05-22T21:18:00.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f75600640 1 -- 172.21.15.161:0/2015487183 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7000dd70 con 0x7f7f7815e130 2024-05-22T21:18:00.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7f400640 1 -- 172.21.15.161:0/2015487183 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f78157fe0 con 0x7f7f7815e130 2024-05-22T21:18:00.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7f400640 1 -- 172.21.15.161:0/2015487183 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f781585a0 con 0x7f7f7815e130 2024-05-22T21:18:00.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.108+0000 7f7f7f400640 1 -- 172.21.15.161:0/2015487183 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f78158a80 con 0x7f7f7815e130 2024-05-22T21:18:00.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.112+0000 7f7f75600640 1 -- 172.21.15.161:0/2015487183 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7f7001e3b0 con 0x7f7f7815e130 2024-05-22T21:18:00.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.112+0000 7f7f75600640 1 --2- 172.21.15.161:0/2015487183 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7f54066690 0x7f7f54068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.112+0000 7f7f75600640 1 -- 172.21.15.161:0/2015487183 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f70084e00 con 0x7f7f7815e130 2024-05-22T21:18:00.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.112+0000 7f7f7ca00640 1 --2- 172.21.15.161:0/2015487183 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7f54066690 0x7f7f54068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.112+0000 7f7f75600640 1 -- 172.21.15.161:0/2015487183 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f7f5406e1c0 con 0x7f7f7815e130 2024-05-22T21:18:00.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.112+0000 7f7f7ca00640 1 --2- 172.21.15.161:0/2015487183 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7f54066690 0x7f7f54068b50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7f68000f70 tx=0x7f7f68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:00.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.116+0000 7f7f75600640 1 -- 172.21.15.161:0/2015487183 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7f7001a020 con 0x7f7f7815e130 2024-05-22T21:18:00.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.272+0000 7f7f7f400640 1 -- 172.21.15.161:0/2015487183 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f78067220 con 0x7f7f7815e130 2024-05-22T21:18:00.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.272+0000 7f7f75600640 1 -- 172.21.15.161:0/2015487183 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16191 (secure 0 0 0) 0x7f7f70052080 con 0x7f7f7815e130 2024-05-22T21:18:00.276 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:00.276 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":88,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:17:34.615226+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:17:12.574579+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-22T21:17:13.013467+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:18:00.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.276+0000 7f7f7f400640 1 -- 172.21.15.161:0/2015487183 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7f54066690 msgr2=0x7f7f54068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.276+0000 7f7f7f400640 1 --2- 172.21.15.161:0/2015487183 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7f54066690 0x7f7f54068b50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7f68000f70 tx=0x7f7f68009040 comp rx=0 tx=0).stop 2024-05-22T21:18:00.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.276+0000 7f7f7f400640 1 -- 172.21.15.161:0/2015487183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f7815e130 msgr2=0x7f7f78150160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.276+0000 7f7f7f400640 1 --2- 172.21.15.161:0/2015487183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f7815e130 0x7f7f78150160 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f7f7000cef0 tx=0x7f7f7000a710 comp rx=0 tx=0).stop 2024-05-22T21:18:00.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.276+0000 7f7f7f400640 1 -- 172.21.15.161:0/2015487183 shutdown_connections 2024-05-22T21:18:00.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.276+0000 7f7f7f400640 1 --2- 172.21.15.161:0/2015487183 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7f54066690 0x7f7f54068b50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.276+0000 7f7f7f400640 1 --2- 172.21.15.161:0/2015487183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f7815e130 0x7f7f78150160 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.276+0000 7f7f7f400640 1 --2- 172.21.15.161:0/2015487183 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7f7815dd50 0x7f7f78140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.276+0000 7f7f7f400640 1 --2- 172.21.15.161:0/2015487183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f7815d410 0x7f7f7814cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.276+0000 7f7f7f400640 1 -- 172.21.15.161:0/2015487183 >> 172.21.15.161:0/2015487183 conn(0x7f7f780b0bc0 msgr2=0x7f7f780af4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:00.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.276+0000 7f7f7f400640 1 -- 172.21.15.161:0/2015487183 shutdown_connections 2024-05-22T21:18:00.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.276+0000 7f7f7f400640 1 -- 172.21.15.161:0/2015487183 wait complete. 2024-05-22T21:18:00.297 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:17:12.574579+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:17:13.013467+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:18:00.298 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:18:00.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b8c00640 1 Processor -- start 2024-05-22T21:18:00.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b8c00640 1 -- start start 2024-05-22T21:18:00.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b8c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b4143c40 0x7f51b415b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b8c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b415efb0 con 0x7f51b406c9b0 2024-05-22T21:18:00.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b8c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b415f290 con 0x7f51b41444a0 2024-05-22T21:18:00.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b8c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b415f570 con 0x7f51b4143c40 2024-05-22T21:18:00.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b2800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f51b406c9b0 0x7f51b40b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39060/0 (socket says 172.21.15.161:39060) 2024-05-22T21:18:00.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b2800640 1 -- 172.21.15.161:0/1892850115 learned_addr learned my addr 172.21.15.161:0/1892850115 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:00.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1e00640 1 --2- 172.21.15.161:0/1892850115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b4143c40 0x7f51b415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 -- 172.21.15.161:0/1892850115 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1716588492 0 0) 0x7f51b415efb0 con 0x7f51b406c9b0 2024-05-22T21:18:00.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 -- 172.21.15.161:0/1892850115 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5198003530 con 0x7f51b406c9b0 2024-05-22T21:18:00.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 -- 172.21.15.161:0/1892850115 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458372192 0 0) 0x7f51b415f290 con 0x7f51b41444a0 2024-05-22T21:18:00.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 -- 172.21.15.161:0/1892850115 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51b415efb0 con 0x7f51b41444a0 2024-05-22T21:18:00.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 -- 172.21.15.161:0/1892850115 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1735503707 0 0) 0x7f5198003530 con 0x7f51b406c9b0 2024-05-22T21:18:00.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 -- 172.21.15.161:0/1892850115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b4143c40 msgr2=0x7f51b415b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:00.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 --2- 172.21.15.161:0/1892850115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b4143c40 0x7f51b415b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 -- 172.21.15.161:0/1892850115 >> v1:172.21.15.179:6789/0 conn(0x7f51b41444a0 legacy=0x7f51b406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 -- 172.21.15.161:0/1892850115 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b415f850 con 0x7f51b406c9b0 2024-05-22T21:18:00.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 -- 172.21.15.161:0/1892850115 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f51ac001600 con 0x7f51b406c9b0 2024-05-22T21:18:00.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 -- 172.21.15.161:0/1892850115 >> v1:172.21.15.161:6789/0 conn(0x7f51b406c9b0 legacy=0x7f51b40b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 --2- 172.21.15.161:0/1892850115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5198003b40 0x7f5198003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 --2- 172.21.15.161:0/1892850115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b4143c40 0x7f51980047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 --2- 172.21.15.161:0/1892850115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5198004d10 0x7f5198005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 -- 172.21.15.161:0/1892850115 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b415f570 con 0x7f5198004d10 2024-05-22T21:18:00.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 -- 172.21.15.161:0/1892850115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519800ba40 con 0x7f5198003b40 2024-05-22T21:18:00.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1400640 1 -- 172.21.15.161:0/1892850115 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519800bbb0 con 0x7f51b4143c40 2024-05-22T21:18:00.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51a9400640 1 --2- 172.21.15.161:0/1892850115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b4143c40 0x7f51980047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1e00640 1 --2- 172.21.15.161:0/1892850115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5198003b40 0x7f5198003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b2800640 1 --2- 172.21.15.161:0/1892850115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5198004d10 0x7f5198005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51a9400640 1 -- 172.21.15.161:0/1892850115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5198003b40 msgr2=0x7f5198003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:00.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51a9400640 1 --2- 172.21.15.161:0/1892850115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5198003b40 0x7f5198003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51a9400640 1 -- 172.21.15.161:0/1892850115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5198004d10 msgr2=0x7f5198005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:00.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51a9400640 1 --2- 172.21.15.161:0/1892850115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5198004d10 0x7f5198005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51a9400640 1 -- 172.21.15.161:0/1892850115 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5198003530 con 0x7f51b4143c40 2024-05-22T21:18:00.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b1e00640 1 --2- 172.21.15.161:0/1892850115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5198003b40 0x7f5198003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:00.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.436+0000 7f51b2800640 1 --2- 172.21.15.161:0/1892850115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5198004d10 0x7f5198005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:00.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.440+0000 7f51a9400640 1 --2- 172.21.15.161:0/1892850115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b4143c40 0x7f51980047d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f519c0018d0 tx=0x7f519c01c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad4a180301b738cf server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:00.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.440+0000 7f51b1400640 1 -- 172.21.15.161:0/1892850115 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c0052d0 con 0x7f51b4143c40 2024-05-22T21:18:00.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.440+0000 7f51b1400640 1 -- 172.21.15.161:0/1892850115 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519c005450 con 0x7f51b4143c40 2024-05-22T21:18:00.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.440+0000 7f51b1400640 1 -- 172.21.15.161:0/1892850115 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c003840 con 0x7f51b4143c40 2024-05-22T21:18:00.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.440+0000 7f51b8c00640 1 -- 172.21.15.161:0/1892850115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b4143c40 msgr2=0x7f51980047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.440+0000 7f51b8c00640 1 --2- 172.21.15.161:0/1892850115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b4143c40 0x7f51980047d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f519c0018d0 tx=0x7f519c01c540 comp rx=0 tx=0).stop 2024-05-22T21:18:00.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.440+0000 7f51b2800640 1 -- 172.21.15.161:0/1892850115 reap_dead start 2024-05-22T21:18:00.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.440+0000 7f51b8c00640 1 -- 172.21.15.161:0/1892850115 shutdown_connections 2024-05-22T21:18:00.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.440+0000 7f51b8c00640 1 -- 172.21.15.161:0/1892850115 >> 172.21.15.161:0/1892850115 conn(0x7f51b406a2d0 msgr2=0x7f51b406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:00.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.440+0000 7f51b8c00640 1 -- 172.21.15.161:0/1892850115 shutdown_connections 2024-05-22T21:18:00.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.440+0000 7f51b8c00640 1 -- 172.21.15.161:0/1892850115 wait complete. 2024-05-22T21:18:00.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51b8c00640 1 Processor -- start 2024-05-22T21:18:00.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51b8c00640 1 -- start start 2024-05-22T21:18:00.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51b8c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51b4143c40 0x7f51b41f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51b8c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b40ac8b0 0x7f51b41f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51b8c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b413d0f0 0x7f51b413d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51b8c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b413dae0 con 0x7f51b40ac8b0 2024-05-22T21:18:00.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51b8c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b40ad8f0 con 0x7f51b4143c40 2024-05-22T21:18:00.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51b8c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b4159180 con 0x7f51b413d0f0 2024-05-22T21:18:00.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51a9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b40ac8b0 0x7f51b41f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51b1e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51b4143c40 0x7f51b41f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51a9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b40ac8b0 0x7f51b41f02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44966/0 (socket says 172.21.15.161:44966) 2024-05-22T21:18:00.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51a9400640 1 -- 172.21.15.161:0/1877944709 learned_addr learned my addr 172.21.15.161:0/1877944709 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:00.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51b2800640 1 --2- 172.21.15.161:0/1877944709 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b413d0f0 0x7f51b413d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51a9400640 1 -- 172.21.15.161:0/1877944709 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b413d0f0 msgr2=0x7f51b413d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51a9400640 1 --2- 172.21.15.161:0/1877944709 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b413d0f0 0x7f51b413d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51a9400640 1 -- 172.21.15.161:0/1877944709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51b4143c40 msgr2=0x7f51b41f38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51a9400640 1 --2- 172.21.15.161:0/1877944709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51b4143c40 0x7f51b41f38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51a9400640 1 -- 172.21.15.161:0/1877944709 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b41f3e00 con 0x7f51b40ac8b0 2024-05-22T21:18:00.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51b1e00640 1 --2- 172.21.15.161:0/1877944709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51b4143c40 0x7f51b41f38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:00.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51a9400640 1 --2- 172.21.15.161:0/1877944709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b40ac8b0 0x7f51b41f02a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f519c005040 tx=0x7f5198005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:00.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51aaa00640 1 -- 172.21.15.161:0/1877944709 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c003810 con 0x7f51b40ac8b0 2024-05-22T21:18:00.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51aaa00640 1 -- 172.21.15.161:0/1877944709 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519c003970 con 0x7f51b40ac8b0 2024-05-22T21:18:00.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51aaa00640 1 -- 172.21.15.161:0/1877944709 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c003d00 con 0x7f51b40ac8b0 2024-05-22T21:18:00.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.444+0000 7f51b8c00640 1 -- 172.21.15.161:0/1877944709 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b40acd70 con 0x7f51b40ac8b0 2024-05-22T21:18:00.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.448+0000 7f51b8c00640 1 -- 172.21.15.161:0/1877944709 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b40ad330 con 0x7f51b40ac8b0 2024-05-22T21:18:00.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.448+0000 7f51b8c00640 1 -- 172.21.15.161:0/1877944709 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b4067220 con 0x7f51b40ac8b0 2024-05-22T21:18:00.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.448+0000 7f51aaa00640 1 -- 172.21.15.161:0/1877944709 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f519c0b1040 con 0x7f51b40ac8b0 2024-05-22T21:18:00.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.448+0000 7f51aaa00640 1 --2- 172.21.15.161:0/1877944709 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5190066690 0x7f5190068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.448+0000 7f51aaa00640 1 -- 172.21.15.161:0/1877944709 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f519c11d5b0 con 0x7f51b40ac8b0 2024-05-22T21:18:00.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.448+0000 7f51aaa00640 1 -- 172.21.15.161:0/1877944709 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f519006e1c0 con 0x7f51b40ac8b0 2024-05-22T21:18:00.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.448+0000 7f51b1e00640 1 --2- 172.21.15.161:0/1877944709 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5190066690 0x7f5190068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.448+0000 7f51b1e00640 1 --2- 172.21.15.161:0/1877944709 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5190066690 0x7f5190068b50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f51a40118f0 tx=0x7f51a400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:00.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.452+0000 7f51aaa00640 1 -- 172.21.15.161:0/1877944709 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f519c0be060 con 0x7f51b40ac8b0 2024-05-22T21:18:00.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.612+0000 7f51b8c00640 1 -- 172.21.15.161:0/1877944709 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51b4144020 con 0x7f51b40ac8b0 2024-05-22T21:18:00.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.612+0000 7f51aaa00640 1 -- 172.21.15.161:0/1877944709 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f519c0af020 con 0x7f51b40ac8b0 2024-05-22T21:18:00.614 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:18:00.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.612+0000 7f51b8c00640 1 -- 172.21.15.161:0/1877944709 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5190066690 msgr2=0x7f5190068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.612+0000 7f51b8c00640 1 --2- 172.21.15.161:0/1877944709 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5190066690 0x7f5190068b50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f51a40118f0 tx=0x7f51a400b040 comp rx=0 tx=0).stop 2024-05-22T21:18:00.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.612+0000 7f51b8c00640 1 -- 172.21.15.161:0/1877944709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b40ac8b0 msgr2=0x7f51b41f02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.612+0000 7f51b8c00640 1 --2- 172.21.15.161:0/1877944709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b40ac8b0 0x7f51b41f02a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f519c005040 tx=0x7f5198005800 comp rx=0 tx=0).stop 2024-05-22T21:18:00.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.612+0000 7f51b8c00640 1 -- 172.21.15.161:0/1877944709 shutdown_connections 2024-05-22T21:18:00.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.612+0000 7f51b8c00640 1 --2- 172.21.15.161:0/1877944709 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5190066690 0x7f5190068b50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.612+0000 7f51b8c00640 1 --2- 172.21.15.161:0/1877944709 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b413d0f0 0x7f51b413d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.612+0000 7f51b8c00640 1 --2- 172.21.15.161:0/1877944709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51b40ac8b0 0x7f51b41f02a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.612+0000 7f51b8c00640 1 --2- 172.21.15.161:0/1877944709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51b4143c40 0x7f51b41f38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.612+0000 7f51b8c00640 1 -- 172.21.15.161:0/1877944709 >> 172.21.15.161:0/1877944709 conn(0x7f51b406a2d0 msgr2=0x7f51b413e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:00.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.612+0000 7f51b8c00640 1 -- 172.21.15.161:0/1877944709 shutdown_connections 2024-05-22T21:18:00.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.612+0000 7f51b8c00640 1 -- 172.21.15.161:0/1877944709 wait complete. 2024-05-22T21:18:00.635 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:18:00.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.768+0000 7f829ae00640 1 Processor -- start 2024-05-22T21:18:00.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.768+0000 7f829ae00640 1 -- start start 2024-05-22T21:18:00.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.772+0000 7f829ae00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8294143c40 0x7f829415b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.772+0000 7f829ae00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f82941444a0 0x7f829406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.772+0000 7f829ae00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8294144900 0x7f829406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.772+0000 7f829ae00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f829415efb0 con 0x7f8294144900 2024-05-22T21:18:00.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.772+0000 7f829ae00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f829415f290 con 0x7f8294143c40 2024-05-22T21:18:00.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.772+0000 7f829ae00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829415f570 con 0x7f82941444a0 2024-05-22T21:18:00.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.772+0000 7f8298e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8294144900 0x7f829406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.772+0000 7f8298e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8294144900 0x7f829406c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44978/0 (socket says 172.21.15.161:44978) 2024-05-22T21:18:00.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.772+0000 7f8298e00640 1 -- 172.21.15.161:0/4155704385 learned_addr learned my addr 172.21.15.161:0/4155704385 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:00.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.772+0000 7f828b400640 1 --2- 172.21.15.161:0/4155704385 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f82941444a0 0x7f829406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.772+0000 7f8293e00640 1 --2- 172.21.15.161:0/4155704385 >> v2:172.21.15.179:3300/0 conn(0x7f8294143c40 0x7f829415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.772+0000 7f8298e00640 1 -- 172.21.15.161:0/4155704385 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f82941444a0 msgr2=0x7f829406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:00.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.772+0000 7f8298e00640 1 --2- 172.21.15.161:0/4155704385 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f82941444a0 0x7f829406c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.772+0000 7f8298e00640 1 -- 172.21.15.161:0/4155704385 >> v2:172.21.15.179:3300/0 conn(0x7f8294143c40 msgr2=0x7f829415b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:00.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.772+0000 7f8298e00640 1 --2- 172.21.15.161:0/4155704385 >> v2:172.21.15.179:3300/0 conn(0x7f8294143c40 0x7f829415b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.772+0000 7f8298e00640 1 -- 172.21.15.161:0/4155704385 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829415f850 con 0x7f8294144900 2024-05-22T21:18:00.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f8298e00640 1 --2- 172.21.15.161:0/4155704385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8294144900 0x7f829406c7e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f82940724b0 tx=0x7f828c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744cf048fd348fa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:00.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f8293400640 1 -- 172.21.15.161:0/4155704385 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c0a57d0 con 0x7f8294144900 2024-05-22T21:18:00.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f8293400640 1 -- 172.21.15.161:0/4155704385 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c0a5930 con 0x7f8294144900 2024-05-22T21:18:00.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f8293400640 1 -- 172.21.15.161:0/4155704385 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c0a5cc0 con 0x7f8294144900 2024-05-22T21:18:00.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 -- 172.21.15.161:0/4155704385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8294144900 msgr2=0x7f829406c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 --2- 172.21.15.161:0/4155704385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8294144900 0x7f829406c7e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f82940724b0 tx=0x7f828c01c9f0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 -- 172.21.15.161:0/4155704385 shutdown_connections 2024-05-22T21:18:00.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 --2- 172.21.15.161:0/4155704385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8294144900 0x7f829406c7e0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 --2- 172.21.15.161:0/4155704385 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f82941444a0 0x7f829406c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 --2- 172.21.15.161:0/4155704385 >> v2:172.21.15.179:3300/0 conn(0x7f8294143c40 0x7f829415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 -- 172.21.15.161:0/4155704385 >> 172.21.15.161:0/4155704385 conn(0x7f829406a2d0 msgr2=0x7f829406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:00.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 -- 172.21.15.161:0/4155704385 shutdown_connections 2024-05-22T21:18:00.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 -- 172.21.15.161:0/4155704385 wait complete. 2024-05-22T21:18:00.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 Processor -- start 2024-05-22T21:18:00.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 -- start start 2024-05-22T21:18:00.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8294143c40 0x7f82940adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f82941444a0 0x7f8294152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8294144900 0x7f8294140050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8294135fa0 con 0x7f8294143c40 2024-05-22T21:18:00.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82941592e0 con 0x7f8294144900 2024-05-22T21:18:00.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.776+0000 7f829ae00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82941589b0 con 0x7f82941444a0 2024-05-22T21:18:00.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f8293e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8294143c40 0x7f82940adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f8293e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8294143c40 0x7f82940adf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44990/0 (socket says 172.21.15.161:44990) 2024-05-22T21:18:00.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f8293e00640 1 -- 172.21.15.161:0/1520361617 learned_addr learned my addr 172.21.15.161:0/1520361617 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:00.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f828b400640 1 --2- 172.21.15.161:0/1520361617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f82941444a0 0x7f8294152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f8298e00640 1 --2- 172.21.15.161:0/1520361617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8294144900 0x7f8294140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f8293e00640 1 -- 172.21.15.161:0/1520361617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f82941444a0 msgr2=0x7f8294152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f8293e00640 1 --2- 172.21.15.161:0/1520361617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f82941444a0 0x7f8294152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f8293e00640 1 -- 172.21.15.161:0/1520361617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8294144900 msgr2=0x7f8294140050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f8293e00640 1 --2- 172.21.15.161:0/1520361617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8294144900 0x7f8294140050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f8293e00640 1 -- 172.21.15.161:0/1520361617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c009b30 con 0x7f8294143c40 2024-05-22T21:18:00.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f8293e00640 1 --2- 172.21.15.161:0/1520361617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8294143c40 0x7f82940adf90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f828400d830 tx=0x7f828400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:00.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f8290c00640 1 -- 172.21.15.161:0/1520361617 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82840096a0 con 0x7f8294143c40 2024-05-22T21:18:00.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f8290c00640 1 -- 172.21.15.161:0/1520361617 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8284009800 con 0x7f8294143c40 2024-05-22T21:18:00.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f8290c00640 1 -- 172.21.15.161:0/1520361617 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8284009b70 con 0x7f8294143c40 2024-05-22T21:18:00.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f829ae00640 1 -- 172.21.15.161:0/1520361617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8294158c90 con 0x7f8294143c40 2024-05-22T21:18:00.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f829ae00640 1 -- 172.21.15.161:0/1520361617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8294136920 con 0x7f8294143c40 2024-05-22T21:18:00.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f829ae00640 1 -- 172.21.15.161:0/1520361617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8294067220 con 0x7f8294143c40 2024-05-22T21:18:00.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f8290c00640 1 -- 172.21.15.161:0/1520361617 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8284014410 con 0x7f8294143c40 2024-05-22T21:18:00.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f8290c00640 1 --2- 172.21.15.161:0/1520361617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8270066690 0x7f8270068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:00.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.780+0000 7f8290c00640 1 -- 172.21.15.161:0/1520361617 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8284084ab0 con 0x7f8294143c40 2024-05-22T21:18:00.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.784+0000 7f828b400640 1 --2- 172.21.15.161:0/1520361617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8270066690 0x7f8270068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:00.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.784+0000 7f8290c00640 1 -- 172.21.15.161:0/1520361617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f827006e1c0 con 0x7f8294143c40 2024-05-22T21:18:00.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.784+0000 7f828b400640 1 --2- 172.21.15.161:0/1520361617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8270066690 0x7f8270068b50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f82940ae4d0 tx=0x7f827c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:00.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.784+0000 7f8290c00640 1 -- 172.21.15.161:0/1520361617 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8284051e30 con 0x7f8294143c40 2024-05-22T21:18:00.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.940+0000 7f829ae00640 1 -- 172.21.15.161:0/1520361617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8294136be0 con 0x7f8294143c40 2024-05-22T21:18:00.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.940+0000 7f8290c00640 1 -- 172.21.15.161:0/1520361617 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8284055450 con 0x7f8294143c40 2024-05-22T21:18:00.942 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:18:00.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.940+0000 7f829ae00640 1 -- 172.21.15.161:0/1520361617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8270066690 msgr2=0x7f8270068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.940+0000 7f829ae00640 1 --2- 172.21.15.161:0/1520361617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8270066690 0x7f8270068b50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f82940ae4d0 tx=0x7f827c008040 comp rx=0 tx=0).stop 2024-05-22T21:18:00.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.940+0000 7f829ae00640 1 -- 172.21.15.161:0/1520361617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8294143c40 msgr2=0x7f82940adf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:00.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.940+0000 7f829ae00640 1 --2- 172.21.15.161:0/1520361617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8294143c40 0x7f82940adf90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f828400d830 tx=0x7f828400dcf0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.940+0000 7f829ae00640 1 -- 172.21.15.161:0/1520361617 shutdown_connections 2024-05-22T21:18:00.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.940+0000 7f829ae00640 1 --2- 172.21.15.161:0/1520361617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8270066690 0x7f8270068b50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.940+0000 7f829ae00640 1 --2- 172.21.15.161:0/1520361617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8294144900 0x7f8294140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.940+0000 7f829ae00640 1 --2- 172.21.15.161:0/1520361617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f82941444a0 0x7f8294152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.940+0000 7f829ae00640 1 --2- 172.21.15.161:0/1520361617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8294143c40 0x7f82940adf90 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:00.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.940+0000 7f829ae00640 1 -- 172.21.15.161:0/1520361617 >> 172.21.15.161:0/1520361617 conn(0x7f829406a2d0 msgr2=0x7f82940af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:00.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.940+0000 7f829ae00640 1 -- 172.21.15.161:0/1520361617 shutdown_connections 2024-05-22T21:18:00.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:00.940+0000 7f829ae00640 1 -- 172.21.15.161:0/1520361617 wait complete. 2024-05-22T21:18:00.963 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:18:01.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.120+0000 7f9c96000640 1 Processor -- start 2024-05-22T21:18:01.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.120+0000 7f9c96000640 1 -- start start 2024-05-22T21:18:01.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.124+0000 7f9c96000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9c900cc530 0x7f9c900d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.124+0000 7f9c96000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c900cf130 0x7f9c90071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.124+0000 7f9c96000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c900cfeb0 con 0x7f9c9015d330 2024-05-22T21:18:01.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.124+0000 7f9c96000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9015d710 con 0x7f9c900cc530 2024-05-22T21:18:01.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.124+0000 7f9c96000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900cf610 con 0x7f9c900cf130 2024-05-22T21:18:01.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.124+0000 7f9c8ea00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9c900cc530 0x7f9c900d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:01.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.124+0000 7f9c8f400640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f9c9015d330 0x7f9c9006cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:33548/0 (socket says 172.21.15.161:33548) 2024-05-22T21:18:01.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.124+0000 7f9c8f400640 1 -- 172.21.15.161:0/2161553466 learned_addr learned my addr 172.21.15.161:0/2161553466 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:01.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.124+0000 7f9c8fe00640 1 --2- 172.21.15.161:0/2161553466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c900cf130 0x7f9c90071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:01.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.124+0000 7f9c8ea00640 1 -- 172.21.15.161:0/2161553466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c900cf130 msgr2=0x7f9c90071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:01.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.124+0000 7f9c8ea00640 1 --2- 172.21.15.161:0/2161553466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c900cf130 0x7f9c90071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.124+0000 7f9c8ea00640 1 -- 172.21.15.161:0/2161553466 >> v1:172.21.15.179:6789/0 conn(0x7f9c9015d330 legacy=0x7f9c9006cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.124+0000 7f9c8ea00640 1 -- 172.21.15.161:0/2161553466 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c900cf8f0 con 0x7f9c900cc530 2024-05-22T21:18:01.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.128+0000 7f9c8ea00640 1 --2- 172.21.15.161:0/2161553466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c900cc530 0x7f9c900d0bb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0097c0 tx=0x7f9c7c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1d9d6e26217c33 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:01.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.128+0000 7f9c8e000640 1 -- 172.21.15.161:0/2161553466 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0ab600 con 0x7f9c900cc530 2024-05-22T21:18:01.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.128+0000 7f9c8e000640 1 -- 172.21.15.161:0/2161553466 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c0a7050 con 0x7f9c900cc530 2024-05-22T21:18:01.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.128+0000 7f9c8e000640 1 -- 172.21.15.161:0/2161553466 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0ab980 con 0x7f9c900cc530 2024-05-22T21:18:01.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.132+0000 7f9c96000640 1 -- 172.21.15.161:0/2161553466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c900cc530 msgr2=0x7f9c900d0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.132+0000 7f9c96000640 1 --2- 172.21.15.161:0/2161553466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c900cc530 0x7f9c900d0bb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0097c0 tx=0x7f9c7c01c3e0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.132+0000 7f9c96000640 1 -- 172.21.15.161:0/2161553466 shutdown_connections 2024-05-22T21:18:01.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.132+0000 7f9c96000640 1 --2- 172.21.15.161:0/2161553466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c900cf130 0x7f9c90071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.132+0000 7f9c96000640 1 --2- 172.21.15.161:0/2161553466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c900cc530 0x7f9c900d0bb0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.132+0000 7f9c96000640 1 -- 172.21.15.161:0/2161553466 >> 172.21.15.161:0/2161553466 conn(0x7f9c900d0090 msgr2=0x7f9c900d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:01.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.132+0000 7f9c96000640 1 -- 172.21.15.161:0/2161553466 shutdown_connections 2024-05-22T21:18:01.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.132+0000 7f9c96000640 1 -- 172.21.15.161:0/2161553466 wait complete. 2024-05-22T21:18:01.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.132+0000 7f9c96000640 1 Processor -- start 2024-05-22T21:18:01.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.132+0000 7f9c96000640 1 -- start start 2024-05-22T21:18:01.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c96000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c900cc530 0x7f9c9014abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c96000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c900cf130 0x7f9c90157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c96000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c9015d330 0x7f9c900d0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c96000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c90156340 con 0x7f9c9015d330 2024-05-22T21:18:01.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c96000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c90154700 con 0x7f9c900cf130 2024-05-22T21:18:01.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c96000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900ae930 con 0x7f9c900cc530 2024-05-22T21:18:01.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c8fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c9015d330 0x7f9c900d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:01.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c8fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c9015d330 0x7f9c900d0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45008/0 (socket says 172.21.15.161:45008) 2024-05-22T21:18:01.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c8fe00640 1 -- 172.21.15.161:0/1733153000 learned_addr learned my addr 172.21.15.161:0/1733153000 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:01.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c8ea00640 1 --2- 172.21.15.161:0/1733153000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c900cf130 0x7f9c90157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:01.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c8fe00640 1 -- 172.21.15.161:0/1733153000 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c900cc530 msgr2=0x7f9c9014abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c8fe00640 1 --2- 172.21.15.161:0/1733153000 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c900cc530 0x7f9c9014abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c8fe00640 1 -- 172.21.15.161:0/1733153000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c900cf130 msgr2=0x7f9c90157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c8fe00640 1 --2- 172.21.15.161:0/1733153000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c900cf130 0x7f9c90157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c8fe00640 1 -- 172.21.15.161:0/1733153000 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c7c009420 con 0x7f9c9015d330 2024-05-22T21:18:01.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c8fe00640 1 --2- 172.21.15.161:0/1733153000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c9015d330 0x7f9c900d0470 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800ef90 tx=0x7f9c8800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:01.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c87400640 1 -- 172.21.15.161:0/1733153000 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88009ca0 con 0x7f9c9015d330 2024-05-22T21:18:01.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c87400640 1 -- 172.21.15.161:0/1733153000 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8800f040 con 0x7f9c9015d330 2024-05-22T21:18:01.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c87400640 1 -- 172.21.15.161:0/1733153000 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c880146c0 con 0x7f9c9015d330 2024-05-22T21:18:01.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c96000640 1 -- 172.21.15.161:0/1733153000 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c900aec10 con 0x7f9c9015d330 2024-05-22T21:18:01.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c96000640 1 -- 172.21.15.161:0/1733153000 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c900c48f0 con 0x7f9c9015d330 2024-05-22T21:18:01.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.136+0000 7f9c96000640 1 -- 172.21.15.161:0/1733153000 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c90067220 con 0x7f9c9015d330 2024-05-22T21:18:01.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.140+0000 7f9c87400640 1 -- 172.21.15.161:0/1733153000 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9c88014820 con 0x7f9c9015d330 2024-05-22T21:18:01.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.140+0000 7f9c87400640 1 --2- 172.21.15.161:0/1733153000 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9c700666e0 0x7f9c70068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.140+0000 7f9c87400640 1 -- 172.21.15.161:0/1733153000 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c88085070 con 0x7f9c9015d330 2024-05-22T21:18:01.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.140+0000 7f9c8f400640 1 --2- 172.21.15.161:0/1733153000 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9c700666e0 0x7f9c70068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:01.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.140+0000 7f9c87400640 1 -- 172.21.15.161:0/1733153000 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f9c7006e210 con 0x7f9c9015d330 2024-05-22T21:18:01.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.144+0000 7f9c8f400640 1 --2- 172.21.15.161:0/1733153000 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9c700666e0 0x7f9c70068ba0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9c80001560 tx=0x7f9c80007a50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:01.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.152+0000 7f9c87400640 1 -- 172.21.15.161:0/1733153000 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9c880b6cf0 con 0x7f9c9015d330 2024-05-22T21:18:01.337 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.336+0000 7f9c96000640 1 -- 172.21.15.161:0/1733153000 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9c900c6350 con 0x7f9c9015d330 2024-05-22T21:18:01.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.336+0000 7f9c87400640 1 -- 172.21.15.161:0/1733153000 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9c88018060 con 0x7f9c9015d330 2024-05-22T21:18:01.339 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:18:01.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.336+0000 7f9c96000640 1 -- 172.21.15.161:0/1733153000 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9c700666e0 msgr2=0x7f9c70068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.336+0000 7f9c96000640 1 --2- 172.21.15.161:0/1733153000 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9c700666e0 0x7f9c70068ba0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9c80001560 tx=0x7f9c80007a50 comp rx=0 tx=0).stop 2024-05-22T21:18:01.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.336+0000 7f9c96000640 1 -- 172.21.15.161:0/1733153000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c9015d330 msgr2=0x7f9c900d0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.336+0000 7f9c96000640 1 --2- 172.21.15.161:0/1733153000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c9015d330 0x7f9c900d0470 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800ef90 tx=0x7f9c8800c490 comp rx=0 tx=0).stop 2024-05-22T21:18:01.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.340+0000 7f9c96000640 1 -- 172.21.15.161:0/1733153000 shutdown_connections 2024-05-22T21:18:01.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.340+0000 7f9c96000640 1 --2- 172.21.15.161:0/1733153000 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9c700666e0 0x7f9c70068ba0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.340+0000 7f9c96000640 1 --2- 172.21.15.161:0/1733153000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c9015d330 0x7f9c900d0470 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.340+0000 7f9c96000640 1 --2- 172.21.15.161:0/1733153000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c900cf130 0x7f9c90157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.340+0000 7f9c96000640 1 --2- 172.21.15.161:0/1733153000 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c900cc530 0x7f9c9014abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.340+0000 7f9c96000640 1 -- 172.21.15.161:0/1733153000 >> 172.21.15.161:0/1733153000 conn(0x7f9c900d0090 msgr2=0x7f9c9006d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:01.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.340+0000 7f9c96000640 1 -- 172.21.15.161:0/1733153000 shutdown_connections 2024-05-22T21:18:01.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.340+0000 7f9c96000640 1 -- 172.21.15.161:0/1733153000 wait complete. 2024-05-22T21:18:01.360 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:18:01.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.496+0000 7f4a9b000640 1 Processor -- start 2024-05-22T21:18:01.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.496+0000 7f4a9b000640 1 -- start start 2024-05-22T21:18:01.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.496+0000 7f4a9b000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4a9413fe50 0x7f4a94164e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.496+0000 7f4a9b000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a9413d3a0 con 0x7f4a941402e0 2024-05-22T21:18:01.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.496+0000 7f4a9b000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a940cd920 con 0x7f4a9413cfc0 2024-05-22T21:18:01.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.496+0000 7f4a9b000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a940cdc00 con 0x7f4a9413fe50 2024-05-22T21:18:01.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.496+0000 7f4a8b400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f4a941402e0 0x7f4a9413f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39074/0 (socket says 172.21.15.161:39074) 2024-05-22T21:18:01.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.496+0000 7f4a8b400640 1 -- 172.21.15.161:0/559879434 learned_addr learned my addr 172.21.15.161:0/559879434 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:01.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.496+0000 7f4a99000640 1 --2- 172.21.15.161:0/559879434 >> v2:172.21.15.161:3300/0 conn(0x7f4a9413fe50 0x7f4a94164e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:01.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.496+0000 7f4a93400640 1 -- 172.21.15.161:0/559879434 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856229290 0 0) 0x7f4a9413d3a0 con 0x7f4a941402e0 2024-05-22T21:18:01.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.496+0000 7f4a93400640 1 -- 172.21.15.161:0/559879434 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a78005740 con 0x7f4a941402e0 2024-05-22T21:18:01.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.496+0000 7f4a99000640 1 -- 172.21.15.161:0/559879434 >> v1:172.21.15.179:6789/0 conn(0x7f4a9413cfc0 legacy=0x7f4a9415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.496+0000 7f4a93400640 1 -- 172.21.15.161:0/559879434 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792724895 0 0) 0x7f4a940cd920 con 0x7f4a9413cfc0 2024-05-22T21:18:01.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.496+0000 7f4a99000640 1 -- 172.21.15.161:0/559879434 >> v1:172.21.15.161:6789/0 conn(0x7f4a941402e0 legacy=0x7f4a9413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.496+0000 7f4a99000640 1 -- 172.21.15.161:0/559879434 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a940cdee0 con 0x7f4a9413fe50 2024-05-22T21:18:01.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.500+0000 7f4a99000640 1 --2- 172.21.15.161:0/559879434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a9413fe50 0x7f4a94164e60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009d90 tx=0x7f4a8c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda19ba6836107ed server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:01.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.500+0000 7f4a93400640 1 -- 172.21.15.161:0/559879434 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0a66b0 con 0x7f4a9413fe50 2024-05-22T21:18:01.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.500+0000 7f4a93400640 1 -- 172.21.15.161:0/559879434 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8c0a6810 con 0x7f4a9413fe50 2024-05-22T21:18:01.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.500+0000 7f4a93400640 1 -- 172.21.15.161:0/559879434 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0a6ba0 con 0x7f4a9413fe50 2024-05-22T21:18:01.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.500+0000 7f4a9b000640 1 -- 172.21.15.161:0/559879434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a9413fe50 msgr2=0x7f4a94164e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.500+0000 7f4a9b000640 1 --2- 172.21.15.161:0/559879434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a9413fe50 0x7f4a94164e60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009d90 tx=0x7f4a8c01c8e0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.500+0000 7f4a9b000640 1 -- 172.21.15.161:0/559879434 shutdown_connections 2024-05-22T21:18:01.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.500+0000 7f4a9b000640 1 --2- 172.21.15.161:0/559879434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a9413fe50 0x7f4a94164e60 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.500+0000 7f4a9b000640 1 -- 172.21.15.161:0/559879434 >> 172.21.15.161:0/559879434 conn(0x7f4a9406a2d0 msgr2=0x7f4a9406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:01.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.500+0000 7f4a9b000640 1 -- 172.21.15.161:0/559879434 shutdown_connections 2024-05-22T21:18:01.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.500+0000 7f4a9b000640 1 -- 172.21.15.161:0/559879434 wait complete. 2024-05-22T21:18:01.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.500+0000 7f4a9b000640 1 Processor -- start 2024-05-22T21:18:01.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.500+0000 7f4a9b000640 1 -- start start 2024-05-22T21:18:01.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a9b000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a9413cfc0 0x7f4a9415a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a9b000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a9413fe50 0x7f4a94159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a9b000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4a941402e0 0x7f4a940b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a9b000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a940b1c50 con 0x7f4a9413cfc0 2024-05-22T21:18:01.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a9b000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9413f020 con 0x7f4a9413fe50 2024-05-22T21:18:01.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a9b000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9413f300 con 0x7f4a941402e0 2024-05-22T21:18:01.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a93e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a9413cfc0 0x7f4a9415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:01.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a93e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a9413cfc0 0x7f4a9415a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45014/0 (socket says 172.21.15.161:45014) 2024-05-22T21:18:01.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a93e00640 1 -- 172.21.15.161:0/3354132425 learned_addr learned my addr 172.21.15.161:0/3354132425 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:01.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a99000640 1 --2- 172.21.15.161:0/3354132425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4a941402e0 0x7f4a940b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:01.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a93e00640 1 -- 172.21.15.161:0/3354132425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4a941402e0 msgr2=0x7f4a940b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a8b400640 1 --2- 172.21.15.161:0/3354132425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a9413fe50 0x7f4a94159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:01.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a93e00640 1 --2- 172.21.15.161:0/3354132425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4a941402e0 0x7f4a940b3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a93e00640 1 -- 172.21.15.161:0/3354132425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a9413fe50 msgr2=0x7f4a94159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a93e00640 1 --2- 172.21.15.161:0/3354132425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a9413fe50 0x7f4a94159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a93e00640 1 -- 172.21.15.161:0/3354132425 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c009a70 con 0x7f4a9413cfc0 2024-05-22T21:18:01.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a8b400640 1 --2- 172.21.15.161:0/3354132425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a9413fe50 0x7f4a94159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:01.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a93e00640 1 --2- 172.21.15.161:0/3354132425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a9413cfc0 0x7f4a9415a370 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400c4d0 tx=0x7f4a8400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:01.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a90c00640 1 -- 172.21.15.161:0/3354132425 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84005480 con 0x7f4a9413cfc0 2024-05-22T21:18:01.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a90c00640 1 -- 172.21.15.161:0/3354132425 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a84003600 con 0x7f4a9413cfc0 2024-05-22T21:18:01.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a9b000640 1 -- 172.21.15.161:0/3354132425 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a940b1ea0 con 0x7f4a9413cfc0 2024-05-22T21:18:01.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a90c00640 1 -- 172.21.15.161:0/3354132425 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a840039b0 con 0x7f4a9413cfc0 2024-05-22T21:18:01.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a9b000640 1 -- 172.21.15.161:0/3354132425 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a94157bf0 con 0x7f4a9413cfc0 2024-05-22T21:18:01.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.504+0000 7f4a9b000640 1 -- 172.21.15.161:0/3354132425 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a9413f4d0 con 0x7f4a9413cfc0 2024-05-22T21:18:01.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.508+0000 7f4a90c00640 1 -- 172.21.15.161:0/3354132425 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4a84003b10 con 0x7f4a9413cfc0 2024-05-22T21:18:01.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.508+0000 7f4a90c00640 1 --2- 172.21.15.161:0/3354132425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4a70066690 0x7f4a70068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.508+0000 7f4a90c00640 1 -- 172.21.15.161:0/3354132425 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a84083770 con 0x7f4a9413cfc0 2024-05-22T21:18:01.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.508+0000 7f4a8b400640 1 --2- 172.21.15.161:0/3354132425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4a70066690 0x7f4a70068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:01.512 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.508+0000 7f4a90c00640 1 -- 172.21.15.161:0/3354132425 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f4a7006e1c0 con 0x7f4a9413cfc0 2024-05-22T21:18:01.512 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.508+0000 7f4a8b400640 1 --2- 172.21.15.161:0/3354132425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4a70066690 0x7f4a70068b50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4a94072120 tx=0x7f4a7c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:01.513 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.512+0000 7f4a90c00640 1 -- 172.21.15.161:0/3354132425 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4a84017070 con 0x7f4a9413cfc0 2024-05-22T21:18:01.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.676+0000 7f4a9b000640 1 -- 172.21.15.161:0/3354132425 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4a94067220 con 0x7f4a9413cfc0 2024-05-22T21:18:01.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.680+0000 7f4a90c00640 1 -- 172.21.15.161:0/3354132425 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v88) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4a84054620 con 0x7f4a9413cfc0 2024-05-22T21:18:01.681 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:18:01.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.680+0000 7f4a9b000640 1 -- 172.21.15.161:0/3354132425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4a70066690 msgr2=0x7f4a70068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.680+0000 7f4a9b000640 1 --2- 172.21.15.161:0/3354132425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4a70066690 0x7f4a70068b50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4a94072120 tx=0x7f4a7c009040 comp rx=0 tx=0).stop 2024-05-22T21:18:01.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.680+0000 7f4a9b000640 1 -- 172.21.15.161:0/3354132425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a9413cfc0 msgr2=0x7f4a9415a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.680+0000 7f4a9b000640 1 --2- 172.21.15.161:0/3354132425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a9413cfc0 0x7f4a9415a370 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400c4d0 tx=0x7f4a8400c990 comp rx=0 tx=0).stop 2024-05-22T21:18:01.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.680+0000 7f4a9b000640 1 -- 172.21.15.161:0/3354132425 shutdown_connections 2024-05-22T21:18:01.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.680+0000 7f4a9b000640 1 --2- 172.21.15.161:0/3354132425 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4a70066690 0x7f4a70068b50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.680+0000 7f4a9b000640 1 --2- 172.21.15.161:0/3354132425 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4a941402e0 0x7f4a940b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.680+0000 7f4a9b000640 1 --2- 172.21.15.161:0/3354132425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a9413fe50 0x7f4a94159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.680+0000 7f4a9b000640 1 --2- 172.21.15.161:0/3354132425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a9413cfc0 0x7f4a9415a370 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.680+0000 7f4a9b000640 1 -- 172.21.15.161:0/3354132425 >> 172.21.15.161:0/3354132425 conn(0x7f4a9406a2d0 msgr2=0x7f4a941581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:01.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.680+0000 7f4a9b000640 1 -- 172.21.15.161:0/3354132425 shutdown_connections 2024-05-22T21:18:01.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.680+0000 7f4a9b000640 1 -- 172.21.15.161:0/3354132425 wait complete. 2024-05-22T21:18:01.702 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-22T21:18:01.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:01.703 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-22T21:18:01.729 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:18:01.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:01.729 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-22T21:18:01.781 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:18:01.781 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:01.790 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:18:01.790 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:01.845 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:18:01.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.976+0000 7f2fc2000640 1 Processor -- start 2024-05-22T21:18:01.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.976+0000 7f2fc2000640 1 -- start start 2024-05-22T21:18:01.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.980+0000 7f2fc2000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2fbc15e130 0x7f2fbc15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.980+0000 7f2fc2000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2fbc15d410 0x7f2fbc15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.980+0000 7f2fc2000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fbc071670 con 0x7f2fbc15dd50 2024-05-22T21:18:01.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.980+0000 7f2fc2000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc15d870 con 0x7f2fbc15d410 2024-05-22T21:18:01.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.980+0000 7f2fc2000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc070a60 con 0x7f2fbc15e130 2024-05-22T21:18:01.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.980+0000 7f2fbb400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f2fbc15dd50 0x7f2fbc06d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39088/0 (socket says 172.21.15.161:39088) 2024-05-22T21:18:01.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.980+0000 7f2fbb400640 1 -- 172.21.15.161:0/3868100392 learned_addr learned my addr 172.21.15.161:0/3868100392 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:01.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.980+0000 7f2fbbe00640 1 --2- 172.21.15.161:0/3868100392 >> v2:172.21.15.161:3300/0 conn(0x7f2fbc15d410 0x7f2fbc15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:01.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.980+0000 7f2fb2a00640 1 --2- 172.21.15.161:0/3868100392 >> v2:172.21.15.179:3300/0 conn(0x7f2fbc15e130 0x7f2fbc15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:01.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.980+0000 7f2fbaa00640 1 -- 172.21.15.161:0/3868100392 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942749337 0 0) 0x7f2fbc071670 con 0x7f2fbc15dd50 2024-05-22T21:18:01.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.980+0000 7f2fbaa00640 1 -- 172.21.15.161:0/3868100392 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fa0005740 con 0x7f2fbc15dd50 2024-05-22T21:18:01.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.980+0000 7f2fbbe00640 1 -- 172.21.15.161:0/3868100392 >> v2:172.21.15.179:3300/0 conn(0x7f2fbc15e130 msgr2=0x7f2fbc15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:01.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.980+0000 7f2fbbe00640 1 --2- 172.21.15.161:0/3868100392 >> v2:172.21.15.179:3300/0 conn(0x7f2fbc15e130 0x7f2fbc15ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.980+0000 7f2fbbe00640 1 -- 172.21.15.161:0/3868100392 >> v1:172.21.15.161:6789/0 conn(0x7f2fbc15dd50 legacy=0x7f2fbc06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.980+0000 7f2fbbe00640 1 -- 172.21.15.161:0/3868100392 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc070d40 con 0x7f2fbc15d410 2024-05-22T21:18:01.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.984+0000 7f2fbbe00640 1 --2- 172.21.15.161:0/3868100392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fbc15d410 0x7f2fbc15b6c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc06c7c0 tx=0x7f2fb401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f580fc8df8d8b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:01.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.984+0000 7f2fbaa00640 1 -- 172.21.15.161:0/3868100392 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb40ab650 con 0x7f2fbc15d410 2024-05-22T21:18:01.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.984+0000 7f2fbaa00640 1 -- 172.21.15.161:0/3868100392 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb40ab7b0 con 0x7f2fbc15d410 2024-05-22T21:18:01.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.984+0000 7f2fbaa00640 1 -- 172.21.15.161:0/3868100392 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb40abb40 con 0x7f2fbc15d410 2024-05-22T21:18:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.984+0000 7f2fc2000640 1 -- 172.21.15.161:0/3868100392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fbc15d410 msgr2=0x7f2fbc15b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.984+0000 7f2fc2000640 1 --2- 172.21.15.161:0/3868100392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fbc15d410 0x7f2fbc15b6c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc06c7c0 tx=0x7f2fb401c450 comp rx=0 tx=0).stop 2024-05-22T21:18:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.984+0000 7f2fc2000640 1 -- 172.21.15.161:0/3868100392 shutdown_connections 2024-05-22T21:18:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.984+0000 7f2fc2000640 1 --2- 172.21.15.161:0/3868100392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fbc15d410 0x7f2fbc15b6c0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.984+0000 7f2fc2000640 1 --2- 172.21.15.161:0/3868100392 >> v2:172.21.15.179:3300/0 conn(0x7f2fbc15e130 0x7f2fbc15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.984+0000 7f2fc2000640 1 -- 172.21.15.161:0/3868100392 >> 172.21.15.161:0/3868100392 conn(0x7f2fbc0b0bc0 msgr2=0x7f2fbc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:01.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.984+0000 7f2fc2000640 1 -- 172.21.15.161:0/3868100392 shutdown_connections 2024-05-22T21:18:01.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.984+0000 7f2fc2000640 1 -- 172.21.15.161:0/3868100392 wait complete. 2024-05-22T21:18:01.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.984+0000 7f2fc2000640 1 Processor -- start 2024-05-22T21:18:01.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.984+0000 7f2fc2000640 1 -- start start 2024-05-22T21:18:01.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.984+0000 7f2fc2000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2fbc15d410 0x7f2fbc14bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.984+0000 7f2fc2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fbc15dd50 0x7f2fbc153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fc2000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fbc15e130 0x7f2fbc0b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fc2000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc0b1420 con 0x7f2fbc15dd50 2024-05-22T21:18:01.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fc2000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc14c260 con 0x7f2fbc15e130 2024-05-22T21:18:01.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fc2000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc14c560 con 0x7f2fbc15d410 2024-05-22T21:18:01.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fb2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fbc15dd50 0x7f2fbc153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:01.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fb2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fbc15dd50 0x7f2fbc153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45042/0 (socket says 172.21.15.161:45042) 2024-05-22T21:18:01.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fb2a00640 1 -- 172.21.15.161:0/3057675492 learned_addr learned my addr 172.21.15.161:0/3057675492 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:01.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fbbe00640 1 --2- 172.21.15.161:0/3057675492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fbc15e130 0x7f2fbc0b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:01.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fbb400640 1 --2- 172.21.15.161:0/3057675492 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2fbc15d410 0x7f2fbc14bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:01.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fb2a00640 1 -- 172.21.15.161:0/3057675492 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2fbc15d410 msgr2=0x7f2fbc14bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fb2a00640 1 --2- 172.21.15.161:0/3057675492 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2fbc15d410 0x7f2fbc14bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fb2a00640 1 -- 172.21.15.161:0/3057675492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fbc15e130 msgr2=0x7f2fbc0b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:01.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fb2a00640 1 --2- 172.21.15.161:0/3057675492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fbc15e130 0x7f2fbc0b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:01.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fb2a00640 1 -- 172.21.15.161:0/3057675492 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb4009420 con 0x7f2fbc15dd50 2024-05-22T21:18:01.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fb2a00640 1 --2- 172.21.15.161:0/3057675492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fbc15dd50 0x7f2fbc153180 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400e8a0 tx=0x7f2fa400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:01.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fb3e00640 1 -- 172.21.15.161:0/3057675492 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4009510 con 0x7f2fbc15dd50 2024-05-22T21:18:01.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fb3e00640 1 -- 172.21.15.161:0/3057675492 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa400c940 con 0x7f2fbc15dd50 2024-05-22T21:18:01.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fb3e00640 1 -- 172.21.15.161:0/3057675492 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400ccb0 con 0x7f2fbc15dd50 2024-05-22T21:18:01.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fc2000640 1 -- 172.21.15.161:0/3057675492 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fbc14c840 con 0x7f2fbc15dd50 2024-05-22T21:18:01.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fc2000640 1 -- 172.21.15.161:0/3057675492 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fbc14cb20 con 0x7f2fbc15dd50 2024-05-22T21:18:01.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fc2000640 1 -- 172.21.15.161:0/3057675492 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fbc067220 con 0x7f2fbc15dd50 2024-05-22T21:18:01.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.988+0000 7f2fb3e00640 1 -- 172.21.15.161:0/3057675492 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2fa4013470 con 0x7f2fbc15dd50 2024-05-22T21:18:01.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.992+0000 7f2fb3e00640 1 --2- 172.21.15.161:0/3057675492 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f98066690 0x7f2f98068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:01.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.992+0000 7f2fb3e00640 1 -- 172.21.15.161:0/3057675492 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fa40841e0 con 0x7f2fbc15dd50 2024-05-22T21:18:01.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.992+0000 7f2fbb400640 1 --2- 172.21.15.161:0/3057675492 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f98066690 0x7f2f98068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:01.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.992+0000 7f2fb3e00640 1 -- 172.21.15.161:0/3057675492 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f2f9806e1c0 con 0x7f2fbc15dd50 2024-05-22T21:18:01.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.992+0000 7f2fbb400640 1 --2- 172.21.15.161:0/3057675492 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f98066690 0x7f2f98068b50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2fac001570 tx=0x7f2fac001aa0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:01.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:01.996+0000 7f2fb3e00640 1 -- 172.21.15.161:0/3057675492 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2fa40515f0 con 0x7f2fbc15dd50 2024-05-22T21:18:02.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.152+0000 7f2fc2000640 1 -- 172.21.15.161:0/3057675492 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fbc159fd0 con 0x7f2fbc15dd50 2024-05-22T21:18:02.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.156+0000 7f2fb3e00640 1 -- 172.21.15.161:0/3057675492 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16191 (secure 0 0 0) 0x7f2fa4021090 con 0x7f2fbc15dd50 2024-05-22T21:18:02.157 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:02.157 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":88,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:17:34.615226+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:17:12.574579+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-22T21:17:13.013467+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:18:02.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.156+0000 7f2fc2000640 1 -- 172.21.15.161:0/3057675492 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f98066690 msgr2=0x7f2f98068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.156+0000 7f2fc2000640 1 --2- 172.21.15.161:0/3057675492 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f98066690 0x7f2f98068b50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2fac001570 tx=0x7f2fac001aa0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.156+0000 7f2fc2000640 1 -- 172.21.15.161:0/3057675492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fbc15dd50 msgr2=0x7f2fbc153180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.156+0000 7f2fc2000640 1 --2- 172.21.15.161:0/3057675492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fbc15dd50 0x7f2fbc153180 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400e8a0 tx=0x7f2fa400eca0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.156+0000 7f2fc2000640 1 -- 172.21.15.161:0/3057675492 shutdown_connections 2024-05-22T21:18:02.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.156+0000 7f2fc2000640 1 --2- 172.21.15.161:0/3057675492 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f98066690 0x7f2f98068b50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.156+0000 7f2fc2000640 1 --2- 172.21.15.161:0/3057675492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fbc15e130 0x7f2fbc0b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.156+0000 7f2fc2000640 1 --2- 172.21.15.161:0/3057675492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fbc15dd50 0x7f2fbc153180 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.156+0000 7f2fc2000640 1 --2- 172.21.15.161:0/3057675492 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2fbc15d410 0x7f2fbc14bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.156+0000 7f2fc2000640 1 -- 172.21.15.161:0/3057675492 >> 172.21.15.161:0/3057675492 conn(0x7f2fbc0b0bc0 msgr2=0x7f2fbc0ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:02.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.156+0000 7f2fc2000640 1 -- 172.21.15.161:0/3057675492 shutdown_connections 2024-05-22T21:18:02.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.156+0000 7f2fc2000640 1 -- 172.21.15.161:0/3057675492 wait complete. 2024-05-22T21:18:02.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:17:12.574579+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:17:13.013467+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:18:02.179 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:18:02.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8e8200640 1 Processor -- start 2024-05-22T21:18:02.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8e8200640 1 -- start start 2024-05-22T21:18:02.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8e8200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8e01596b0 0x7fb8e0071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8e8200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb8e0071db0 0x7fb8e006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8e8200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e00b3110 con 0x7fb8e006d070 2024-05-22T21:18:02.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8e8200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e006d4e0 con 0x7fb8e0071db0 2024-05-22T21:18:02.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8e8200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e015d250 con 0x7fb8e01596b0 2024-05-22T21:18:02.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8e4e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb8e0071db0 0x7fb8e006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8e6200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb8e006d070 0x7fb8e006fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39102/0 (socket says 172.21.15.161:39102) 2024-05-22T21:18:02.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8e6200640 1 -- 172.21.15.161:0/3502370690 learned_addr learned my addr 172.21.15.161:0/3502370690 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:02.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8e5800640 1 --2- 172.21.15.161:0/3502370690 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8e01596b0 0x7fb8e0071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8dfe00640 1 -- 172.21.15.161:0/3502370690 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609044904 0 0) 0x7fb8e00b3110 con 0x7fb8e006d070 2024-05-22T21:18:02.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8dfe00640 1 -- 172.21.15.161:0/3502370690 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8c8003530 con 0x7fb8e006d070 2024-05-22T21:18:02.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8e4e00640 1 -- 172.21.15.161:0/3502370690 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8e01596b0 msgr2=0x7fb8e0071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:02.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8e4e00640 1 --2- 172.21.15.161:0/3502370690 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8e01596b0 0x7fb8e0071870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8e4e00640 1 -- 172.21.15.161:0/3502370690 >> v1:172.21.15.161:6789/0 conn(0x7fb8e006d070 legacy=0x7fb8e006fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8e4e00640 1 -- 172.21.15.161:0/3502370690 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e0159a90 con 0x7fb8e0071db0 2024-05-22T21:18:02.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.316+0000 7fb8e5800640 1 --2- 172.21.15.161:0/3502370690 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8e01596b0 0x7fb8e0071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:02.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.320+0000 7fb8e4e00640 1 --2- 172.21.15.161:0/3502370690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8e0071db0 0x7fb8e006cb30 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00a960 tx=0x7fb8cc01d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1330fc4e9d456f87 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:02.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.320+0000 7fb8dfe00640 1 -- 172.21.15.161:0/3502370690 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc0a67d0 con 0x7fb8e0071db0 2024-05-22T21:18:02.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.320+0000 7fb8dfe00640 1 -- 172.21.15.161:0/3502370690 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8cc0a6930 con 0x7fb8e0071db0 2024-05-22T21:18:02.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.320+0000 7fb8dfe00640 1 -- 172.21.15.161:0/3502370690 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc0a6cc0 con 0x7fb8e0071db0 2024-05-22T21:18:02.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.320+0000 7fb8e8200640 1 -- 172.21.15.161:0/3502370690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8e0071db0 msgr2=0x7fb8e006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.320+0000 7fb8e8200640 1 --2- 172.21.15.161:0/3502370690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8e0071db0 0x7fb8e006cb30 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00a960 tx=0x7fb8cc01d870 comp rx=0 tx=0).stop 2024-05-22T21:18:02.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.320+0000 7fb8e8200640 1 -- 172.21.15.161:0/3502370690 shutdown_connections 2024-05-22T21:18:02.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.320+0000 7fb8e8200640 1 --2- 172.21.15.161:0/3502370690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8e0071db0 0x7fb8e006cb30 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.320+0000 7fb8e8200640 1 --2- 172.21.15.161:0/3502370690 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8e01596b0 0x7fb8e0071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.320+0000 7fb8e8200640 1 -- 172.21.15.161:0/3502370690 >> 172.21.15.161:0/3502370690 conn(0x7fb8e006a2d0 msgr2=0x7fb8e006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:02.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.320+0000 7fb8e8200640 1 -- 172.21.15.161:0/3502370690 shutdown_connections 2024-05-22T21:18:02.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.320+0000 7fb8e8200640 1 -- 172.21.15.161:0/3502370690 wait complete. 2024-05-22T21:18:02.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e8200640 1 Processor -- start 2024-05-22T21:18:02.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e8200640 1 -- start start 2024-05-22T21:18:02.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e8200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8e006d070 0x7fb8e015c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e8200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e0071db0 0x7fb8e015a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e8200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8e01596b0 0x7fb8e014f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e8200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e015c0a0 con 0x7fb8e01596b0 2024-05-22T21:18:02.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e8200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e015ac20 con 0x7fb8e0071db0 2024-05-22T21:18:02.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e8200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e015b490 con 0x7fb8e006d070 2024-05-22T21:18:02.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e6200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8e01596b0 0x7fb8e014f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e6200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8e01596b0 0x7fb8e014f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45074/0 (socket says 172.21.15.161:45074) 2024-05-22T21:18:02.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e6200640 1 -- 172.21.15.161:0/3428730324 learned_addr learned my addr 172.21.15.161:0/3428730324 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:02.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e5800640 1 --2- 172.21.15.161:0/3428730324 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8e006d070 0x7fb8e015c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e4e00640 1 --2- 172.21.15.161:0/3428730324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e0071db0 0x7fb8e015a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e6200640 1 -- 172.21.15.161:0/3428730324 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8e006d070 msgr2=0x7fb8e015c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e6200640 1 --2- 172.21.15.161:0/3428730324 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8e006d070 0x7fb8e015c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e6200640 1 -- 172.21.15.161:0/3428730324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e0071db0 msgr2=0x7fb8e015a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e6200640 1 --2- 172.21.15.161:0/3428730324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e0071db0 0x7fb8e015a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e6200640 1 -- 172.21.15.161:0/3428730324 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8cc00a5c0 con 0x7fb8e01596b0 2024-05-22T21:18:02.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e5800640 1 --2- 172.21.15.161:0/3428730324 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8e006d070 0x7fb8e015c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:02.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e6200640 1 --2- 172.21.15.161:0/3428730324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8e01596b0 0x7fb8e014f320 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800caa0 tx=0x7fb8d800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:02.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8dd600640 1 -- 172.21.15.161:0/3428730324 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d800ac00 con 0x7fb8e01596b0 2024-05-22T21:18:02.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8dd600640 1 -- 172.21.15.161:0/3428730324 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d800ad60 con 0x7fb8e01596b0 2024-05-22T21:18:02.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8dd600640 1 -- 172.21.15.161:0/3428730324 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8012620 con 0x7fb8e01596b0 2024-05-22T21:18:02.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e8200640 1 -- 172.21.15.161:0/3428730324 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e015b770 con 0x7fb8e01596b0 2024-05-22T21:18:02.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e8200640 1 -- 172.21.15.161:0/3428730324 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e015bd30 con 0x7fb8e01596b0 2024-05-22T21:18:02.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.324+0000 7fb8e8200640 1 -- 172.21.15.161:0/3428730324 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e015c210 con 0x7fb8e01596b0 2024-05-22T21:18:02.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.328+0000 7fb8dd600640 1 -- 172.21.15.161:0/3428730324 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb8d8007070 con 0x7fb8e01596b0 2024-05-22T21:18:02.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.328+0000 7fb8dd600640 1 --2- 172.21.15.161:0/3428730324 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb8c0066690 0x7fb8c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.328+0000 7fb8dd600640 1 -- 172.21.15.161:0/3428730324 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8d8084180 con 0x7fb8e01596b0 2024-05-22T21:18:02.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.328+0000 7fb8e5800640 1 --2- 172.21.15.161:0/3428730324 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb8c0066690 0x7fb8c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.328+0000 7fb8dd600640 1 -- 172.21.15.161:0/3428730324 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7fb8c006e1c0 con 0x7fb8e01596b0 2024-05-22T21:18:02.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.328+0000 7fb8e5800640 1 --2- 172.21.15.161:0/3428730324 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb8c0066690 0x7fb8c0068b50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009970 tx=0x7fb8d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:02.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.332+0000 7fb8dd600640 1 -- 172.21.15.161:0/3428730324 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb8d80514e0 con 0x7fb8e01596b0 2024-05-22T21:18:02.335 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:18:02.332+0000 7f7d35a00640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2024-05-22T21:18:02.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.344+0000 7fb8dd600640 1 -- 172.21.15.161:0/3428730324 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(89..89 src has 1..89) v4 ==== 296+0+0 (secure 0 0 0) 0x7fb8d80558a0 con 0x7fb8e01596b0 2024-05-22T21:18:02.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.344+0000 7fb8dd600640 1 -- 172.21.15.161:0/3428730324 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7fb8c006ea00 con 0x7fb8e01596b0 2024-05-22T21:18:02.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.488+0000 7fb8e8200640 1 -- 172.21.15.161:0/3428730324 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb8e0067220 con 0x7fb8e01596b0 2024-05-22T21:18:02.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.488+0000 7fb8dd600640 1 -- 172.21.15.161:0/3428730324 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb8d8017020 con 0x7fb8e01596b0 2024-05-22T21:18:02.493 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:18:02.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.492+0000 7fb8e8200640 1 -- 172.21.15.161:0/3428730324 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb8c0066690 msgr2=0x7fb8c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.492+0000 7fb8e8200640 1 --2- 172.21.15.161:0/3428730324 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb8c0066690 0x7fb8c0068b50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009970 tx=0x7fb8d4008040 comp rx=0 tx=0).stop 2024-05-22T21:18:02.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.492+0000 7fb8e8200640 1 -- 172.21.15.161:0/3428730324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8e01596b0 msgr2=0x7fb8e014f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.492+0000 7fb8e8200640 1 --2- 172.21.15.161:0/3428730324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8e01596b0 0x7fb8e014f320 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800caa0 tx=0x7fb8d800cf60 comp rx=0 tx=0).stop 2024-05-22T21:18:02.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.492+0000 7fb8e8200640 1 -- 172.21.15.161:0/3428730324 shutdown_connections 2024-05-22T21:18:02.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.492+0000 7fb8e8200640 1 --2- 172.21.15.161:0/3428730324 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb8c0066690 0x7fb8c0068b50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.492+0000 7fb8e8200640 1 --2- 172.21.15.161:0/3428730324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8e01596b0 0x7fb8e014f320 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.492+0000 7fb8e8200640 1 --2- 172.21.15.161:0/3428730324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e0071db0 0x7fb8e015a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.492+0000 7fb8e8200640 1 --2- 172.21.15.161:0/3428730324 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8e006d070 0x7fb8e015c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.492+0000 7fb8e8200640 1 -- 172.21.15.161:0/3428730324 >> 172.21.15.161:0/3428730324 conn(0x7fb8e006a2d0 msgr2=0x7fb8e00cd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:02.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.492+0000 7fb8e8200640 1 -- 172.21.15.161:0/3428730324 shutdown_connections 2024-05-22T21:18:02.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.492+0000 7fb8e8200640 1 -- 172.21.15.161:0/3428730324 wait complete. 2024-05-22T21:18:02.513 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:18:02.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.648+0000 7fe131e00640 1 Processor -- start 2024-05-22T21:18:02.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.648+0000 7fe131e00640 1 -- start start 2024-05-22T21:18:02.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.648+0000 7fe131e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe12c143c40 0x7fe12c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.648+0000 7fe131e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12c15efb0 con 0x7fe12c1444a0 2024-05-22T21:18:02.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.648+0000 7fe131e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12c15f290 con 0x7fe12c06c9b0 2024-05-22T21:18:02.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.648+0000 7fe131e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c15f570 con 0x7fe12c143c40 2024-05-22T21:18:02.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.652+0000 7fe122000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe12c1444a0 0x7fe12c06c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39106/0 (socket says 172.21.15.161:39106) 2024-05-22T21:18:02.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.652+0000 7fe122000640 1 -- 172.21.15.161:0/253301054 learned_addr learned my addr 172.21.15.161:0/253301054 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:02.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.652+0000 7fe12aa00640 1 --2- 172.21.15.161:0/253301054 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe12c143c40 0x7fe12c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.652+0000 7fe12a000640 1 -- 172.21.15.161:0/253301054 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232794585 0 0) 0x7fe12c15efb0 con 0x7fe12c1444a0 2024-05-22T21:18:02.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.652+0000 7fe12a000640 1 -- 172.21.15.161:0/253301054 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe110003530 con 0x7fe12c1444a0 2024-05-22T21:18:02.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.652+0000 7fe12a000640 1 -- 172.21.15.161:0/253301054 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792055233 0 0) 0x7fe12c15f290 con 0x7fe12c06c9b0 2024-05-22T21:18:02.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.652+0000 7fe12a000640 1 -- 172.21.15.161:0/253301054 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe12c15efb0 con 0x7fe12c06c9b0 2024-05-22T21:18:02.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.652+0000 7fe12aa00640 1 -- 172.21.15.161:0/253301054 >> v1:172.21.15.179:6789/0 conn(0x7fe12c06c9b0 legacy=0x7fe12c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.652+0000 7fe12aa00640 1 -- 172.21.15.161:0/253301054 >> v1:172.21.15.161:6789/0 conn(0x7fe12c1444a0 legacy=0x7fe12c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.652+0000 7fe12aa00640 1 -- 172.21.15.161:0/253301054 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12c15f850 con 0x7fe12c143c40 2024-05-22T21:18:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.652+0000 7fe12aa00640 1 --2- 172.21.15.161:0/253301054 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe12c143c40 0x7fe12c15b600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0097f0 tx=0x7fe11c01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0191b760052b80e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.652+0000 7fe12a000640 1 -- 172.21.15.161:0/253301054 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c0ab640 con 0x7fe12c143c40 2024-05-22T21:18:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.652+0000 7fe12a000640 1 -- 172.21.15.161:0/253301054 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11c0a7070 con 0x7fe12c143c40 2024-05-22T21:18:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.652+0000 7fe12a000640 1 -- 172.21.15.161:0/253301054 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c0ab9c0 con 0x7fe12c143c40 2024-05-22T21:18:02.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe131e00640 1 -- 172.21.15.161:0/253301054 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe12c143c40 msgr2=0x7fe12c15b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe131e00640 1 --2- 172.21.15.161:0/253301054 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe12c143c40 0x7fe12c15b600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0097f0 tx=0x7fe11c01c4e0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe131e00640 1 -- 172.21.15.161:0/253301054 shutdown_connections 2024-05-22T21:18:02.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe131e00640 1 --2- 172.21.15.161:0/253301054 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe12c143c40 0x7fe12c15b600 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe131e00640 1 -- 172.21.15.161:0/253301054 >> 172.21.15.161:0/253301054 conn(0x7fe12c06a2d0 msgr2=0x7fe12c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:02.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe131e00640 1 -- 172.21.15.161:0/253301054 shutdown_connections 2024-05-22T21:18:02.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe131e00640 1 -- 172.21.15.161:0/253301054 wait complete. 2024-05-22T21:18:02.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe131e00640 1 Processor -- start 2024-05-22T21:18:02.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe131e00640 1 -- start start 2024-05-22T21:18:02.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe131e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe12c06c9b0 0x7fe12c070960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe131e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe12c143c40 0x7fe12c139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe131e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe12c1444a0 0x7fe12c15a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe131e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c15b200 con 0x7fe12c143c40 2024-05-22T21:18:02.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe131e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c15af20 con 0x7fe12c1444a0 2024-05-22T21:18:02.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe131e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c06fe80 con 0x7fe12c06c9b0 2024-05-22T21:18:02.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe122000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe12c143c40 0x7fe12c139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe122000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe12c143c40 0x7fe12c139920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45082/0 (socket says 172.21.15.161:45082) 2024-05-22T21:18:02.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe122000640 1 -- 172.21.15.161:0/1238168798 learned_addr learned my addr 172.21.15.161:0/1238168798 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:02.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe12b400640 1 --2- 172.21.15.161:0/1238168798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe12c1444a0 0x7fe12c15a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe12aa00640 1 --2- 172.21.15.161:0/1238168798 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe12c06c9b0 0x7fe12c070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe122000640 1 -- 172.21.15.161:0/1238168798 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe12c06c9b0 msgr2=0x7fe12c070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe122000640 1 --2- 172.21.15.161:0/1238168798 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe12c06c9b0 0x7fe12c070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe122000640 1 -- 172.21.15.161:0/1238168798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe12c1444a0 msgr2=0x7fe12c15a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe122000640 1 --2- 172.21.15.161:0/1238168798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe12c1444a0 0x7fe12c15a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.656+0000 7fe122000640 1 -- 172.21.15.161:0/1238168798 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11c009420 con 0x7fe12c143c40 2024-05-22T21:18:02.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.660+0000 7fe122000640 1 --2- 172.21.15.161:0/1238168798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe12c143c40 0x7fe12c139920 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe11400c420 tx=0x7fe11400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:02.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.660+0000 7fe123e00640 1 -- 172.21.15.161:0/1238168798 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114005cf0 con 0x7fe12c143c40 2024-05-22T21:18:02.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.660+0000 7fe123e00640 1 -- 172.21.15.161:0/1238168798 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe114005e50 con 0x7fe12c143c40 2024-05-22T21:18:02.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.660+0000 7fe131e00640 1 -- 172.21.15.161:0/1238168798 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12c070160 con 0x7fe12c143c40 2024-05-22T21:18:02.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.660+0000 7fe123e00640 1 -- 172.21.15.161:0/1238168798 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114007310 con 0x7fe12c143c40 2024-05-22T21:18:02.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.660+0000 7fe131e00640 1 -- 172.21.15.161:0/1238168798 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12c139e60 con 0x7fe12c143c40 2024-05-22T21:18:02.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.660+0000 7fe131e00640 1 -- 172.21.15.161:0/1238168798 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12c06fcc0 con 0x7fe12c143c40 2024-05-22T21:18:02.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.660+0000 7fe123e00640 1 -- 172.21.15.161:0/1238168798 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe114007470 con 0x7fe12c143c40 2024-05-22T21:18:02.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.660+0000 7fe123e00640 1 --2- 172.21.15.161:0/1238168798 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe108066690 0x7fe108068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.660+0000 7fe123e00640 1 -- 172.21.15.161:0/1238168798 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1140837b0 con 0x7fe12c143c40 2024-05-22T21:18:02.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.660+0000 7fe12aa00640 1 --2- 172.21.15.161:0/1238168798 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe108066690 0x7fe108068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.660+0000 7fe123e00640 1 -- 172.21.15.161:0/1238168798 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7fe10806e1c0 con 0x7fe12c143c40 2024-05-22T21:18:02.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.664+0000 7fe12aa00640 1 --2- 172.21.15.161:0/1238168798 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe108066690 0x7fe108068b50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0af000 tx=0x7fe11c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:02.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.664+0000 7fe123e00640 1 -- 172.21.15.161:0/1238168798 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe114017020 con 0x7fe12c143c40 2024-05-22T21:18:02.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.816+0000 7fe131e00640 1 -- 172.21.15.161:0/1238168798 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe12c067220 con 0x7fe12c143c40 2024-05-22T21:18:02.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.816+0000 7fe123e00640 1 -- 172.21.15.161:0/1238168798 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe1140545c0 con 0x7fe12c143c40 2024-05-22T21:18:02.820 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:18:02.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.820+0000 7fe131e00640 1 -- 172.21.15.161:0/1238168798 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe108066690 msgr2=0x7fe108068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.820+0000 7fe131e00640 1 --2- 172.21.15.161:0/1238168798 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe108066690 0x7fe108068b50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0af000 tx=0x7fe11c0a7000 comp rx=0 tx=0).stop 2024-05-22T21:18:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.820+0000 7fe131e00640 1 -- 172.21.15.161:0/1238168798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe12c143c40 msgr2=0x7fe12c139920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.820+0000 7fe131e00640 1 --2- 172.21.15.161:0/1238168798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe12c143c40 0x7fe12c139920 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe11400c420 tx=0x7fe11400c8e0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.820+0000 7fe131e00640 1 -- 172.21.15.161:0/1238168798 shutdown_connections 2024-05-22T21:18:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.820+0000 7fe131e00640 1 --2- 172.21.15.161:0/1238168798 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe108066690 0x7fe108068b50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.820+0000 7fe131e00640 1 --2- 172.21.15.161:0/1238168798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe12c1444a0 0x7fe12c15a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.820+0000 7fe131e00640 1 --2- 172.21.15.161:0/1238168798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe12c143c40 0x7fe12c139920 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.820+0000 7fe131e00640 1 --2- 172.21.15.161:0/1238168798 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe12c06c9b0 0x7fe12c070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.820+0000 7fe131e00640 1 -- 172.21.15.161:0/1238168798 >> 172.21.15.161:0/1238168798 conn(0x7fe12c06a2d0 msgr2=0x7fe12c15bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.820+0000 7fe131e00640 1 -- 172.21.15.161:0/1238168798 shutdown_connections 2024-05-22T21:18:02.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.820+0000 7fe131e00640 1 -- 172.21.15.161:0/1238168798 wait complete. 2024-05-22T21:18:02.841 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:18:02.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.976+0000 7f9d95e00640 1 Processor -- start 2024-05-22T21:18:02.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.976+0000 7f9d95e00640 1 -- start start 2024-05-22T21:18:02.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.976+0000 7f9d95e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9d9015d180 0x7f9d900b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.976+0000 7f9d95e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d9006b5f0 0x7f9d900ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.976+0000 7f9d95e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d90161c20 con 0x7f9d9006d080 2024-05-22T21:18:02.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.976+0000 7f9d95e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d9006c390 con 0x7f9d9015d180 2024-05-22T21:18:02.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.976+0000 7f9d95e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9006c5a0 con 0x7f9d9006b5f0 2024-05-22T21:18:02.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.976+0000 7f9d8ea00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9d9015d180 0x7f9d900b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.976+0000 7f9d8f400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d9006b5f0 0x7f9d900ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.976+0000 7f9d8f400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d9006b5f0 0x7f9d900ad570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:34544/0 (socket says 172.21.15.161:34544) 2024-05-22T21:18:02.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.976+0000 7f9d8f400640 1 -- 172.21.15.161:0/4190574044 learned_addr learned my addr 172.21.15.161:0/4190574044 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:02.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.976+0000 7f9d8f400640 1 -- 172.21.15.161:0/4190574044 >> v2:172.21.15.179:3300/0 conn(0x7f9d9015d180 msgr2=0x7f9d900b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:02.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.976+0000 7f9d8f400640 1 --2- 172.21.15.161:0/4190574044 >> v2:172.21.15.179:3300/0 conn(0x7f9d9015d180 0x7f9d900b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.976+0000 7f9d8f400640 1 -- 172.21.15.161:0/4190574044 >> v1:172.21.15.179:6789/0 conn(0x7f9d9006d080 legacy=0x7f9d9006aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.976+0000 7f9d8f400640 1 -- 172.21.15.161:0/4190574044 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9006c880 con 0x7f9d9006b5f0 2024-05-22T21:18:02.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.980+0000 7f9d8f400640 1 --2- 172.21.15.161:0/4190574044 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d9006b5f0 0x7f9d900ad570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009870 tx=0x7f9d8801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=909dc6d98a5f2741 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:02.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.980+0000 7f9d8d600640 1 -- 172.21.15.161:0/4190574044 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d880a4ea0 con 0x7f9d9006b5f0 2024-05-22T21:18:02.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.980+0000 7f9d8d600640 1 -- 172.21.15.161:0/4190574044 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d880a7050 con 0x7f9d9006b5f0 2024-05-22T21:18:02.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.980+0000 7f9d8d600640 1 -- 172.21.15.161:0/4190574044 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d880ab820 con 0x7f9d9006b5f0 2024-05-22T21:18:02.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.980+0000 7f9d95e00640 1 -- 172.21.15.161:0/4190574044 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d9006b5f0 msgr2=0x7f9d900ad570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.980+0000 7f9d95e00640 1 --2- 172.21.15.161:0/4190574044 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d9006b5f0 0x7f9d900ad570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009870 tx=0x7f9d8801c4d0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.980+0000 7f9d95e00640 1 -- 172.21.15.161:0/4190574044 shutdown_connections 2024-05-22T21:18:02.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.980+0000 7f9d95e00640 1 --2- 172.21.15.161:0/4190574044 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d9006b5f0 0x7f9d900ad570 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.980+0000 7f9d95e00640 1 --2- 172.21.15.161:0/4190574044 >> v2:172.21.15.179:3300/0 conn(0x7f9d9015d180 0x7f9d900b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.980+0000 7f9d95e00640 1 -- 172.21.15.161:0/4190574044 >> 172.21.15.161:0/4190574044 conn(0x7f9d90070610 msgr2=0x7f9d90070a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:02.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.980+0000 7f9d95e00640 1 -- 172.21.15.161:0/4190574044 shutdown_connections 2024-05-22T21:18:02.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.980+0000 7f9d95e00640 1 -- 172.21.15.161:0/4190574044 wait complete. 2024-05-22T21:18:02.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d95e00640 1 Processor -- start 2024-05-22T21:18:02.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d95e00640 1 -- start start 2024-05-22T21:18:02.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d95e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d9006b5f0 0x7f9d90158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d95e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d9006d080 0x7f9d90154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d95e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d9015d180 0x7f9d9015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d95e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d90150040 con 0x7f9d9006b5f0 2024-05-22T21:18:02.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d95e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d90155110 con 0x7f9d9015d180 2024-05-22T21:18:02.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d95e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d90150290 con 0x7f9d9006d080 2024-05-22T21:18:02.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d8ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d9006b5f0 0x7f9d90158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d8ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d9006b5f0 0x7f9d90158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45096/0 (socket says 172.21.15.161:45096) 2024-05-22T21:18:02.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d8ea00640 1 -- 172.21.15.161:0/3026934240 learned_addr learned my addr 172.21.15.161:0/3026934240 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:02.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d8f400640 1 --2- 172.21.15.161:0/3026934240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d9015d180 0x7f9d9015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d8e000640 1 --2- 172.21.15.161:0/3026934240 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d9006d080 0x7f9d90154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d8ea00640 1 -- 172.21.15.161:0/3026934240 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d9006d080 msgr2=0x7f9d90154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d8ea00640 1 --2- 172.21.15.161:0/3026934240 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d9006d080 0x7f9d90154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d8ea00640 1 -- 172.21.15.161:0/3026934240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d9015d180 msgr2=0x7f9d9015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:02.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d8ea00640 1 --2- 172.21.15.161:0/3026934240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d9015d180 0x7f9d9015b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:02.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d8ea00640 1 -- 172.21.15.161:0/3026934240 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d880094d0 con 0x7f9d9006b5f0 2024-05-22T21:18:02.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d8f400640 1 --2- 172.21.15.161:0/3026934240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d9015d180 0x7f9d9015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:02.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d8ea00640 1 --2- 172.21.15.161:0/3026934240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d9006b5f0 0x7f9d90158240 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000efc0 tx=0x7f9d8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:02.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d87400640 1 -- 172.21.15.161:0/3026934240 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d80009b80 con 0x7f9d9006b5f0 2024-05-22T21:18:02.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d87400640 1 -- 172.21.15.161:0/3026934240 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8000f040 con 0x7f9d9006b5f0 2024-05-22T21:18:02.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d87400640 1 -- 172.21.15.161:0/3026934240 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d800146a0 con 0x7f9d9006b5f0 2024-05-22T21:18:02.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d95e00640 1 -- 172.21.15.161:0/3026934240 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d90150570 con 0x7f9d9006b5f0 2024-05-22T21:18:02.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.984+0000 7f9d95e00640 1 -- 172.21.15.161:0/3026934240 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d90150b30 con 0x7f9d9006b5f0 2024-05-22T21:18:02.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.988+0000 7f9d95e00640 1 -- 172.21.15.161:0/3026934240 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d90150cc0 con 0x7f9d9006b5f0 2024-05-22T21:18:02.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.988+0000 7f9d87400640 1 -- 172.21.15.161:0/3026934240 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9d80014800 con 0x7f9d9006b5f0 2024-05-22T21:18:02.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.988+0000 7f9d87400640 1 --2- 172.21.15.161:0/3026934240 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d70066690 0x7f9d70068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:02.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.988+0000 7f9d87400640 1 -- 172.21.15.161:0/3026934240 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d80084b60 con 0x7f9d9006b5f0 2024-05-22T21:18:02.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.988+0000 7f9d8e000640 1 --2- 172.21.15.161:0/3026934240 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d70066690 0x7f9d70068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:02.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.988+0000 7f9d87400640 1 -- 172.21.15.161:0/3026934240 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7f9d7006e1c0 con 0x7f9d9006b5f0 2024-05-22T21:18:02.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.988+0000 7f9d8e000640 1 --2- 172.21.15.161:0/3026934240 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d70066690 0x7f9d70068b50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9d90156ac0 tx=0x7f9d7c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:02.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:02.992+0000 7f9d87400640 1 -- 172.21.15.161:0/3026934240 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9d80051f70 con 0x7f9d9006b5f0 2024-05-22T21:18:03.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.144+0000 7f9d95e00640 1 -- 172.21.15.161:0/3026934240 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9d90067220 con 0x7f9d9006b5f0 2024-05-22T21:18:03.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.144+0000 7f9d87400640 1 -- 172.21.15.161:0/3026934240 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9d80055590 con 0x7f9d9006b5f0 2024-05-22T21:18:03.147 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:18:03.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.148+0000 7f9d95e00640 1 -- 172.21.15.161:0/3026934240 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d70066690 msgr2=0x7f9d70068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.148+0000 7f9d95e00640 1 --2- 172.21.15.161:0/3026934240 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d70066690 0x7f9d70068b50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9d90156ac0 tx=0x7f9d7c009040 comp rx=0 tx=0).stop 2024-05-22T21:18:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.148+0000 7f9d95e00640 1 -- 172.21.15.161:0/3026934240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d9006b5f0 msgr2=0x7f9d90158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.148+0000 7f9d95e00640 1 --2- 172.21.15.161:0/3026934240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d9006b5f0 0x7f9d90158240 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000efc0 tx=0x7f9d8000c490 comp rx=0 tx=0).stop 2024-05-22T21:18:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.148+0000 7f9d95e00640 1 -- 172.21.15.161:0/3026934240 shutdown_connections 2024-05-22T21:18:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.148+0000 7f9d95e00640 1 --2- 172.21.15.161:0/3026934240 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d70066690 0x7f9d70068b50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.148+0000 7f9d95e00640 1 --2- 172.21.15.161:0/3026934240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d9015d180 0x7f9d9015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.148+0000 7f9d95e00640 1 --2- 172.21.15.161:0/3026934240 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d9006d080 0x7f9d90154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.148+0000 7f9d95e00640 1 --2- 172.21.15.161:0/3026934240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d9006b5f0 0x7f9d90158240 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.148+0000 7f9d95e00640 1 -- 172.21.15.161:0/3026934240 >> 172.21.15.161:0/3026934240 conn(0x7f9d90070610 msgr2=0x7f9d900af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.148+0000 7f9d95e00640 1 -- 172.21.15.161:0/3026934240 shutdown_connections 2024-05-22T21:18:03.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.148+0000 7f9d95e00640 1 -- 172.21.15.161:0/3026934240 wait complete. 2024-05-22T21:18:03.169 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:18:03.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.300+0000 7f1d17200640 1 Processor -- start 2024-05-22T21:18:03.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d17200640 1 -- start start 2024-05-22T21:18:03.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d17200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1d10143c40 0x7f1d1015b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:03.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d17200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1d101444a0 0x7f1d1006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:03.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d17200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1d10144900 0x7f1d1006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:03.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d17200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1015efb0 con 0x7f1d10143c40 2024-05-22T21:18:03.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d17200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1015f290 con 0x7f1d101444a0 2024-05-22T21:18:03.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d17200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1015f570 con 0x7f1d10144900 2024-05-22T21:18:03.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d0fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1d10143c40 0x7f1d1015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:03.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d0fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1d10143c40 0x7f1d1015b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45102/0 (socket says 172.21.15.161:45102) 2024-05-22T21:18:03.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d0fe00640 1 -- 172.21.15.161:0/3712968767 learned_addr learned my addr 172.21.15.161:0/3712968767 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:03.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d15200640 1 --2- 172.21.15.161:0/3712968767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1d10144900 0x7f1d1006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:03.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d07400640 1 --2- 172.21.15.161:0/3712968767 >> v2:172.21.15.179:3300/0 conn(0x7f1d101444a0 0x7f1d1006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:03.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d0fe00640 1 -- 172.21.15.161:0/3712968767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1d10144900 msgr2=0x7f1d1006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:03.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d0fe00640 1 --2- 172.21.15.161:0/3712968767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1d10144900 0x7f1d1006c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d0fe00640 1 -- 172.21.15.161:0/3712968767 >> v2:172.21.15.179:3300/0 conn(0x7f1d101444a0 msgr2=0x7f1d1006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:03.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d0fe00640 1 --2- 172.21.15.161:0/3712968767 >> v2:172.21.15.179:3300/0 conn(0x7f1d101444a0 0x7f1d1006c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d0fe00640 1 -- 172.21.15.161:0/3712968767 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1015f850 con 0x7f1d10143c40 2024-05-22T21:18:03.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.304+0000 7f1d15200640 1 --2- 172.21.15.161:0/3712968767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1d10144900 0x7f1d1006c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:03.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.308+0000 7f1d0fe00640 1 --2- 172.21.15.161:0/3712968767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d10143c40 0x7f1d1015b600 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1d100b1c80 tx=0x7f1d0001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e284d29bf22e2def server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:03.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.308+0000 7f1d0f400640 1 -- 172.21.15.161:0/3712968767 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d000ab640 con 0x7f1d10143c40 2024-05-22T21:18:03.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.308+0000 7f1d0f400640 1 -- 172.21.15.161:0/3712968767 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d000a7070 con 0x7f1d10143c40 2024-05-22T21:18:03.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.308+0000 7f1d0f400640 1 -- 172.21.15.161:0/3712968767 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d000ab9c0 con 0x7f1d10143c40 2024-05-22T21:18:03.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.308+0000 7f1d17200640 1 -- 172.21.15.161:0/3712968767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d10143c40 msgr2=0x7f1d1015b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:03.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.308+0000 7f1d17200640 1 --2- 172.21.15.161:0/3712968767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d10143c40 0x7f1d1015b600 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1d100b1c80 tx=0x7f1d0001c520 comp rx=0 tx=0).stop 2024-05-22T21:18:03.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.308+0000 7f1d17200640 1 -- 172.21.15.161:0/3712968767 shutdown_connections 2024-05-22T21:18:03.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.308+0000 7f1d17200640 1 --2- 172.21.15.161:0/3712968767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1d10144900 0x7f1d1006c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.308+0000 7f1d17200640 1 --2- 172.21.15.161:0/3712968767 >> v2:172.21.15.179:3300/0 conn(0x7f1d101444a0 0x7f1d1006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.308+0000 7f1d17200640 1 --2- 172.21.15.161:0/3712968767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d10143c40 0x7f1d1015b600 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.308+0000 7f1d17200640 1 -- 172.21.15.161:0/3712968767 >> 172.21.15.161:0/3712968767 conn(0x7f1d1006a2d0 msgr2=0x7f1d1006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:03.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.308+0000 7f1d17200640 1 -- 172.21.15.161:0/3712968767 shutdown_connections 2024-05-22T21:18:03.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.308+0000 7f1d17200640 1 -- 172.21.15.161:0/3712968767 wait complete. 2024-05-22T21:18:03.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.308+0000 7f1d17200640 1 Processor -- start 2024-05-22T21:18:03.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d17200640 1 -- start start 2024-05-22T21:18:03.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d17200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d10143c40 0x7f1d10136520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:03.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d17200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d101444a0 0x7f1d1015bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:03.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d17200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1d10144900 0x7f1d10156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:03.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d17200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d101f6510 con 0x7f1d10143c40 2024-05-22T21:18:03.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d17200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d10136a60 con 0x7f1d101444a0 2024-05-22T21:18:03.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d17200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d101f6720 con 0x7f1d10144900 2024-05-22T21:18:03.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d0fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d10143c40 0x7f1d10136520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:03.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d0fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d10143c40 0x7f1d10136520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45108/0 (socket says 172.21.15.161:45108) 2024-05-22T21:18:03.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d0fe00640 1 -- 172.21.15.161:0/1002249883 learned_addr learned my addr 172.21.15.161:0/1002249883 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:03.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d0fe00640 1 -- 172.21.15.161:0/1002249883 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1d10144900 msgr2=0x7f1d10156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:03.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d0fe00640 1 --2- 172.21.15.161:0/1002249883 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1d10144900 0x7f1d10156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d0fe00640 1 -- 172.21.15.161:0/1002249883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d101444a0 msgr2=0x7f1d1015bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:03.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d07400640 1 --2- 172.21.15.161:0/1002249883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d101444a0 0x7f1d1015bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:03.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d0fe00640 1 --2- 172.21.15.161:0/1002249883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d101444a0 0x7f1d1015bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d0fe00640 1 -- 172.21.15.161:0/1002249883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d000094d0 con 0x7f1d10143c40 2024-05-22T21:18:03.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d07400640 1 --2- 172.21.15.161:0/1002249883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d101444a0 0x7f1d1015bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:03.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d0fe00640 1 --2- 172.21.15.161:0/1002249883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d10143c40 0x7f1d10136520 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1d0001abd0 tx=0x7f1d0001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:03.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d0cc00640 1 -- 172.21.15.161:0/1002249883 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d000be3f0 con 0x7f1d10143c40 2024-05-22T21:18:03.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d0cc00640 1 -- 172.21.15.161:0/1002249883 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d000be550 con 0x7f1d10143c40 2024-05-22T21:18:03.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d0cc00640 1 -- 172.21.15.161:0/1002249883 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d000be8e0 con 0x7f1d10143c40 2024-05-22T21:18:03.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d17200640 1 -- 172.21.15.161:0/1002249883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d101f6a00 con 0x7f1d10143c40 2024-05-22T21:18:03.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d17200640 1 -- 172.21.15.161:0/1002249883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d101f6fc0 con 0x7f1d10143c40 2024-05-22T21:18:03.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.312+0000 7f1d17200640 1 -- 172.21.15.161:0/1002249883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d10067220 con 0x7f1d10143c40 2024-05-22T21:18:03.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.316+0000 7f1d0cc00640 1 -- 172.21.15.161:0/1002249883 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1d000b43f0 con 0x7f1d10143c40 2024-05-22T21:18:03.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.316+0000 7f1d0cc00640 1 --2- 172.21.15.161:0/1002249883 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1cec0666c0 0x7f1cec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:03.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.316+0000 7f1d0cc00640 1 -- 172.21.15.161:0/1002249883 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d0011e5b0 con 0x7f1d10143c40 2024-05-22T21:18:03.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.316+0000 7f1d07400640 1 --2- 172.21.15.161:0/1002249883 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1cec0666c0 0x7f1cec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:03.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.316+0000 7f1d0cc00640 1 -- 172.21.15.161:0/1002249883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7f1cec06e1f0 con 0x7f1d10143c40 2024-05-22T21:18:03.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.316+0000 7f1d07400640 1 --2- 172.21.15.161:0/1002249883 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1cec0666c0 0x7f1cec068b80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1d101f7680 tx=0x7f1cf8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:03.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.320+0000 7f1d0cc00640 1 -- 172.21.15.161:0/1002249883 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1d000c4060 con 0x7f1d10143c40 2024-05-22T21:18:03.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.348+0000 7f1d0cc00640 1 -- 172.21.15.161:0/1002249883 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(90..90 src has 1..90) v4 ==== 304+0+0 (secure 0 0 0) 0x7f1d000ef7a0 con 0x7f1d10143c40 2024-05-22T21:18:03.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.348+0000 7f1d0cc00640 1 -- 172.21.15.161:0/1002249883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f1cec06eaa0 con 0x7f1d10143c40 2024-05-22T21:18:03.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.488+0000 7f1d17200640 1 -- 172.21.15.161:0/1002249883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1d100ceaa0 con 0x7f1d10143c40 2024-05-22T21:18:03.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.488+0000 7f1d0cc00640 1 -- 172.21.15.161:0/1002249883 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v90) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1d0011ce60 con 0x7f1d10143c40 2024-05-22T21:18:03.492 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:18:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.492+0000 7f1d17200640 1 -- 172.21.15.161:0/1002249883 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1cec0666c0 msgr2=0x7f1cec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.492+0000 7f1d17200640 1 --2- 172.21.15.161:0/1002249883 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1cec0666c0 0x7f1cec068b80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1d101f7680 tx=0x7f1cf8008040 comp rx=0 tx=0).stop 2024-05-22T21:18:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.492+0000 7f1d17200640 1 -- 172.21.15.161:0/1002249883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d10143c40 msgr2=0x7f1d10136520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.492+0000 7f1d17200640 1 --2- 172.21.15.161:0/1002249883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d10143c40 0x7f1d10136520 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1d0001abd0 tx=0x7f1d0001ad60 comp rx=0 tx=0).stop 2024-05-22T21:18:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.492+0000 7f1d17200640 1 -- 172.21.15.161:0/1002249883 shutdown_connections 2024-05-22T21:18:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.492+0000 7f1d17200640 1 --2- 172.21.15.161:0/1002249883 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1cec0666c0 0x7f1cec068b80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.492+0000 7f1d17200640 1 --2- 172.21.15.161:0/1002249883 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1d10144900 0x7f1d10156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.492+0000 7f1d17200640 1 --2- 172.21.15.161:0/1002249883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d101444a0 0x7f1d1015bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.492+0000 7f1d17200640 1 --2- 172.21.15.161:0/1002249883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d10143c40 0x7f1d10136520 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.492+0000 7f1d17200640 1 -- 172.21.15.161:0/1002249883 >> 172.21.15.161:0/1002249883 conn(0x7f1d1006a2d0 msgr2=0x7f1d100afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.492+0000 7f1d17200640 1 -- 172.21.15.161:0/1002249883 shutdown_connections 2024-05-22T21:18:03.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.492+0000 7f1d17200640 1 -- 172.21.15.161:0/1002249883 wait complete. 2024-05-22T21:18:03.513 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-22T21:18:03.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:03.513 DEBUG:teuthology.orchestra.run.smithi179:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-22T21:18:03.543 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:18:03.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:03.543 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-22T21:18:03.595 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:18:03.595 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:03.606 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:18:03.606 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:18:03.606 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:03.646 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:18:03.646 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:18:03.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.780+0000 7f4150c00640 1 Processor -- start 2024-05-22T21:18:03.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.780+0000 7f4150c00640 1 -- start start 2024-05-22T21:18:03.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.780+0000 7f4150c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f414c0715a0 0x7f414c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:03.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.780+0000 7f4150c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f414c15ec90 0x7f414c15f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:03.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.780+0000 7f4150c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f414c1442a0 con 0x7f414c143ec0 2024-05-22T21:18:03.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.780+0000 7f4150c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f414c15f660 con 0x7f414c0715a0 2024-05-22T21:18:03.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.780+0000 7f4150c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414c070500 con 0x7f414c15ec90 2024-05-22T21:18:03.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.780+0000 7f4141400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f414c0715a0 0x7f414c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:03.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.780+0000 7f4141400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f414c0715a0 0x7f414c15f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45118/0 (socket says 172.21.15.161:45118) 2024-05-22T21:18:03.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.780+0000 7f4141400640 1 -- 172.21.15.161:0/68901423 learned_addr learned my addr 172.21.15.161:0/68901423 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:03.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.780+0000 7f414a800640 1 --2- 172.21.15.161:0/68901423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f414c15ec90 0x7f414c15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:03.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.780+0000 7f4141400640 1 -- 172.21.15.161:0/68901423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f414c15ec90 msgr2=0x7f414c15f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:03.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.780+0000 7f4141400640 1 --2- 172.21.15.161:0/68901423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f414c15ec90 0x7f414c15f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.780+0000 7f4141400640 1 -- 172.21.15.161:0/68901423 >> v1:172.21.15.179:6789/0 conn(0x7f414c143ec0 legacy=0x7f414c06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:03.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.780+0000 7f4141400640 1 -- 172.21.15.161:0/68901423 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414c0707e0 con 0x7f414c0715a0 2024-05-22T21:18:03.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.784+0000 7f4141400640 1 --2- 172.21.15.161:0/68901423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f414c0715a0 0x7f414c15f830 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4134009870 tx=0x7f413401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b7381f30afba3b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:03.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.784+0000 7f4149400640 1 -- 172.21.15.161:0/68901423 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41340ac410 con 0x7f414c0715a0 2024-05-22T21:18:03.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.784+0000 7f4149400640 1 -- 172.21.15.161:0/68901423 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41340ac570 con 0x7f414c0715a0 2024-05-22T21:18:03.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.784+0000 7f4149400640 1 -- 172.21.15.161:0/68901423 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41340ac900 con 0x7f414c0715a0 2024-05-22T21:18:03.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.784+0000 7f4150c00640 1 -- 172.21.15.161:0/68901423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f414c0715a0 msgr2=0x7f414c15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:03.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.784+0000 7f4150c00640 1 --2- 172.21.15.161:0/68901423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f414c0715a0 0x7f414c15f830 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4134009870 tx=0x7f413401c4f0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.784+0000 7f4150c00640 1 -- 172.21.15.161:0/68901423 shutdown_connections 2024-05-22T21:18:03.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.784+0000 7f4150c00640 1 --2- 172.21.15.161:0/68901423 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f414c15ec90 0x7f414c15f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.784+0000 7f4150c00640 1 --2- 172.21.15.161:0/68901423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f414c0715a0 0x7f414c15f830 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.784+0000 7f4150c00640 1 -- 172.21.15.161:0/68901423 >> 172.21.15.161:0/68901423 conn(0x7f414c0b0bc0 msgr2=0x7f414c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:03.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.784+0000 7f4150c00640 1 -- 172.21.15.161:0/68901423 shutdown_connections 2024-05-22T21:18:03.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.784+0000 7f4150c00640 1 -- 172.21.15.161:0/68901423 wait complete. 2024-05-22T21:18:03.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4150c00640 1 Processor -- start 2024-05-22T21:18:03.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4150c00640 1 -- start start 2024-05-22T21:18:03.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4150c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f414c143ec0 0x7f414c13ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:03.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4150c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f414c15ec90 0x7f414c13f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:03.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4150c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f414c15cd30 0x7f414c13fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:03.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4150c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c163480 con 0x7f414c143ec0 2024-05-22T21:18:03.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4150c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c15c8a0 con 0x7f414c15ec90 2024-05-22T21:18:03.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4150c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414c13f340 con 0x7f414c15cd30 2024-05-22T21:18:03.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4149e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f414c143ec0 0x7f414c13ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:03.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4149e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f414c143ec0 0x7f414c13ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45128/0 (socket says 172.21.15.161:45128) 2024-05-22T21:18:03.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4149e00640 1 -- 172.21.15.161:0/1546065440 learned_addr learned my addr 172.21.15.161:0/1546065440 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:03.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4141400640 1 --2- 172.21.15.161:0/1546065440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f414c15ec90 0x7f414c13f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:03.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f414a800640 1 --2- 172.21.15.161:0/1546065440 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f414c15cd30 0x7f414c13fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:03.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4149e00640 1 -- 172.21.15.161:0/1546065440 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f414c15cd30 msgr2=0x7f414c13fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:03.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4149e00640 1 --2- 172.21.15.161:0/1546065440 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f414c15cd30 0x7f414c13fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4149e00640 1 -- 172.21.15.161:0/1546065440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f414c15ec90 msgr2=0x7f414c13f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:03.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4149e00640 1 --2- 172.21.15.161:0/1546065440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f414c15ec90 0x7f414c13f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4149e00640 1 -- 172.21.15.161:0/1546065440 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41340094d0 con 0x7f414c143ec0 2024-05-22T21:18:03.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4149e00640 1 --2- 172.21.15.161:0/1546065440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f414c143ec0 0x7f414c13ee00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f413c00c450 tx=0x7f413c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:03.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4142a00640 1 -- 172.21.15.161:0/1546065440 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c012830 con 0x7f414c143ec0 2024-05-22T21:18:03.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4142a00640 1 -- 172.21.15.161:0/1546065440 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413c0129b0 con 0x7f414c143ec0 2024-05-22T21:18:03.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4142a00640 1 -- 172.21.15.161:0/1546065440 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c012d60 con 0x7f414c143ec0 2024-05-22T21:18:03.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4150c00640 1 -- 172.21.15.161:0/1546065440 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f414c15e660 con 0x7f414c143ec0 2024-05-22T21:18:03.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.788+0000 7f4150c00640 1 -- 172.21.15.161:0/1546065440 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414c15c250 con 0x7f414c143ec0 2024-05-22T21:18:03.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.792+0000 7f4150c00640 1 -- 172.21.15.161:0/1546065440 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f414c067220 con 0x7f414c143ec0 2024-05-22T21:18:03.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.792+0000 7f4142a00640 1 -- 172.21.15.161:0/1546065440 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f413c01c460 con 0x7f414c143ec0 2024-05-22T21:18:03.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.792+0000 7f4142a00640 1 --2- 172.21.15.161:0/1546065440 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41280666c0 0x7f4128068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:03.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.792+0000 7f4142a00640 1 -- 172.21.15.161:0/1546065440 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f413c083da0 con 0x7f414c143ec0 2024-05-22T21:18:03.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.792+0000 7f4142a00640 1 -- 172.21.15.161:0/1546065440 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f412806e1f0 con 0x7f414c143ec0 2024-05-22T21:18:03.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.792+0000 7f4141400640 1 --2- 172.21.15.161:0/1546065440 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41280666c0 0x7f4128068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:03.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.792+0000 7f4141400640 1 --2- 172.21.15.161:0/1546065440 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41280666c0 0x7f4128068b80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f41340ae000 tx=0x7f4134009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:03.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.796+0000 7f4142a00640 1 -- 172.21.15.161:0/1546065440 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f413c051250 con 0x7f414c143ec0 2024-05-22T21:18:03.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.980+0000 7f4150c00640 1 -- 172.21.15.161:0/1546065440 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f414c06dcc0 con 0x7f414c143ec0 2024-05-22T21:18:03.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.980+0000 7f4142a00640 1 -- 172.21.15.161:0/1546065440 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4745 (secure 0 0 0) 0x7f413c054870 con 0x7f414c143ec0 2024-05-22T21:18:03.983 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:03.983 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5567,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196093187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196093187},{"type":"v1","addr":"172.21.15.161:6836","nonce":196093187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6136,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4029369301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4029369301},{"type":"v1","addr":"172.21.15.161:6837","nonce":4029369301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6139,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3212560487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3212560487},{"type":"v1","addr":"172.21.15.179:6834","nonce":3212560487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:17:14.012254+0000","modified":"2024-05-22T21:17:17.029283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6306":{"gid":6306,"name":"b","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/3065973138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3065973138},{"type":"v1","addr":"172.21.15.179:6835","nonce":3065973138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-22T21:18:03.983 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 39 2024-05-22T21:18:03.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.980+0000 7f4150c00640 1 -- 172.21.15.161:0/1546065440 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41280666c0 msgr2=0x7f4128068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:03.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.980+0000 7f4150c00640 1 --2- 172.21.15.161:0/1546065440 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41280666c0 0x7f4128068b80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f41340ae000 tx=0x7f4134009040 comp rx=0 tx=0).stop 2024-05-22T21:18:03.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.980+0000 7f4150c00640 1 -- 172.21.15.161:0/1546065440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f414c143ec0 msgr2=0x7f414c13ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:03.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.980+0000 7f4150c00640 1 --2- 172.21.15.161:0/1546065440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f414c143ec0 0x7f414c13ee00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f413c00c450 tx=0x7f413c00c910 comp rx=0 tx=0).stop 2024-05-22T21:18:03.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.980+0000 7f4150c00640 1 -- 172.21.15.161:0/1546065440 shutdown_connections 2024-05-22T21:18:03.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.980+0000 7f4150c00640 1 --2- 172.21.15.161:0/1546065440 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41280666c0 0x7f4128068b80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.980+0000 7f4150c00640 1 --2- 172.21.15.161:0/1546065440 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f414c15cd30 0x7f414c13fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.980+0000 7f4150c00640 1 --2- 172.21.15.161:0/1546065440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f414c15ec90 0x7f414c13f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.980+0000 7f4150c00640 1 --2- 172.21.15.161:0/1546065440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f414c143ec0 0x7f414c13ee00 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:03.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.980+0000 7f4150c00640 1 -- 172.21.15.161:0/1546065440 >> 172.21.15.161:0/1546065440 conn(0x7f414c0b0bc0 msgr2=0x7f414c140990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:03.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.980+0000 7f4150c00640 1 -- 172.21.15.161:0/1546065440 shutdown_connections 2024-05-22T21:18:03.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:03.980+0000 7f4150c00640 1 -- 172.21.15.161:0/1546065440 wait complete. 2024-05-22T21:18:04.004 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:18:04.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.136+0000 7f7bb9200640 1 Processor -- start 2024-05-22T21:18:04.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.136+0000 7f7bb9200640 1 -- start start 2024-05-22T21:18:04.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.136+0000 7f7bb9200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7bb415dd50 0x7f7bb406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.136+0000 7f7bb9200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb4071670 con 0x7f7bb415e130 2024-05-22T21:18:04.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.136+0000 7f7bb9200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb406de40 con 0x7f7bb415d5e0 2024-05-22T21:18:04.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.136+0000 7f7bb9200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb4070a60 con 0x7f7bb415dd50 2024-05-22T21:18:04.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.136+0000 7f7ba9400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f7bb415e130 0x7f7bb415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39110/0 (socket says 172.21.15.161:39110) 2024-05-22T21:18:04.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.136+0000 7f7ba9400640 1 -- 172.21.15.161:0/1935575157 learned_addr learned my addr 172.21.15.161:0/1935575157 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:04.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.136+0000 7f7bb1e00640 1 --2- 172.21.15.161:0/1935575157 >> v2:172.21.15.179:3300/0 conn(0x7f7bb415dd50 0x7f7bb406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.140+0000 7f7bb1400640 1 -- 172.21.15.161:0/1935575157 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158197471 0 0) 0x7f7bb4071670 con 0x7f7bb415e130 2024-05-22T21:18:04.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.140+0000 7f7bb1400640 1 -- 172.21.15.161:0/1935575157 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b98003530 con 0x7f7bb415e130 2024-05-22T21:18:04.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.140+0000 7f7bb1400640 1 -- 172.21.15.161:0/1935575157 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987423844 0 0) 0x7f7bb406de40 con 0x7f7bb415d5e0 2024-05-22T21:18:04.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.140+0000 7f7bb1400640 1 -- 172.21.15.161:0/1935575157 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb4071670 con 0x7f7bb415d5e0 2024-05-22T21:18:04.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.140+0000 7f7bb1400640 1 -- 172.21.15.161:0/1935575157 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 57749955 0 0) 0x7f7b98003530 con 0x7f7bb415e130 2024-05-22T21:18:04.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.140+0000 7f7bb1e00640 1 -- 172.21.15.161:0/1935575157 >> v1:172.21.15.179:6789/0 conn(0x7f7bb415d5e0 legacy=0x7f7bb415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.140+0000 7f7bb1e00640 1 -- 172.21.15.161:0/1935575157 >> v1:172.21.15.161:6789/0 conn(0x7f7bb415e130 legacy=0x7f7bb415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.140+0000 7f7bb1e00640 1 -- 172.21.15.161:0/1935575157 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb4070d40 con 0x7f7bb415dd50 2024-05-22T21:18:04.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.140+0000 7f7bb1e00640 1 --2- 172.21.15.161:0/1935575157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb415dd50 0x7f7bb406d900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40097c0 tx=0x7f7ba401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7ba2fc6c24822b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:04.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.140+0000 7f7bb1400640 1 -- 172.21.15.161:0/1935575157 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c001680 con 0x7f7bb415dd50 2024-05-22T21:18:04.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.140+0000 7f7bb1400640 1 -- 172.21.15.161:0/1935575157 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba40ac440 con 0x7f7bb415dd50 2024-05-22T21:18:04.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.140+0000 7f7bb1400640 1 -- 172.21.15.161:0/1935575157 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40ac7b0 con 0x7f7bb415dd50 2024-05-22T21:18:04.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb9200640 1 -- 172.21.15.161:0/1935575157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb415dd50 msgr2=0x7f7bb406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb9200640 1 --2- 172.21.15.161:0/1935575157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb415dd50 0x7f7bb406d900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40097c0 tx=0x7f7ba401c390 comp rx=0 tx=0).stop 2024-05-22T21:18:04.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb9200640 1 -- 172.21.15.161:0/1935575157 shutdown_connections 2024-05-22T21:18:04.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb9200640 1 --2- 172.21.15.161:0/1935575157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb415dd50 0x7f7bb406d900 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb9200640 1 -- 172.21.15.161:0/1935575157 >> 172.21.15.161:0/1935575157 conn(0x7f7bb40b0bc0 msgr2=0x7f7bb40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:04.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb9200640 1 -- 172.21.15.161:0/1935575157 shutdown_connections 2024-05-22T21:18:04.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb9200640 1 -- 172.21.15.161:0/1935575157 wait complete. 2024-05-22T21:18:04.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb9200640 1 Processor -- start 2024-05-22T21:18:04.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb9200640 1 -- start start 2024-05-22T21:18:04.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb9200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bb415d5e0 0x7f7bb4150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb9200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7bb415dd50 0x7f7bb40ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb9200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb415e130 0x7f7bb412e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb9200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb4145d80 con 0x7f7bb415d5e0 2024-05-22T21:18:04.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb9200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40ce330 con 0x7f7bb415e130 2024-05-22T21:18:04.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb9200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40cd720 con 0x7f7bb415dd50 2024-05-22T21:18:04.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bb415d5e0 0x7f7bb4150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bb415d5e0 0x7f7bb4150d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45144/0 (socket says 172.21.15.161:45144) 2024-05-22T21:18:04.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb1e00640 1 -- 172.21.15.161:0/1200899728 learned_addr learned my addr 172.21.15.161:0/1200899728 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:04.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7ba9400640 1 --2- 172.21.15.161:0/1200899728 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7bb415dd50 0x7f7bb40ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb2800640 1 --2- 172.21.15.161:0/1200899728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb415e130 0x7f7bb412e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb1e00640 1 -- 172.21.15.161:0/1200899728 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7bb415dd50 msgr2=0x7f7bb40ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb1e00640 1 --2- 172.21.15.161:0/1200899728 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7bb415dd50 0x7f7bb40ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb1e00640 1 -- 172.21.15.161:0/1200899728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb415e130 msgr2=0x7f7bb412e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb1e00640 1 --2- 172.21.15.161:0/1200899728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb415e130 0x7f7bb412e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb1e00640 1 -- 172.21.15.161:0/1200899728 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba4009420 con 0x7f7bb415d5e0 2024-05-22T21:18:04.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.144+0000 7f7bb1e00640 1 --2- 172.21.15.161:0/1200899728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bb415d5e0 0x7f7bb4150d80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4000c10 tx=0x7f7ba4019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:04.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.148+0000 7f7baaa00640 1 -- 172.21.15.161:0/1200899728 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40b5600 con 0x7f7bb415d5e0 2024-05-22T21:18:04.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.148+0000 7f7baaa00640 1 -- 172.21.15.161:0/1200899728 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba40b5760 con 0x7f7bb415d5e0 2024-05-22T21:18:04.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.148+0000 7f7baaa00640 1 -- 172.21.15.161:0/1200899728 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40b5ad0 con 0x7f7bb415d5e0 2024-05-22T21:18:04.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.148+0000 7f7bb9200640 1 -- 172.21.15.161:0/1200899728 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb40cda00 con 0x7f7bb415d5e0 2024-05-22T21:18:04.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.148+0000 7f7bb9200640 1 -- 172.21.15.161:0/1200899728 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb40cdfc0 con 0x7f7bb415d5e0 2024-05-22T21:18:04.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.148+0000 7f7bb9200640 1 -- 172.21.15.161:0/1200899728 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb4067220 con 0x7f7bb415d5e0 2024-05-22T21:18:04.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.148+0000 7f7baaa00640 1 -- 172.21.15.161:0/1200899728 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7ba40b3040 con 0x7f7bb415d5e0 2024-05-22T21:18:04.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.148+0000 7f7baaa00640 1 --2- 172.21.15.161:0/1200899728 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7b900666c0 0x7f7b90068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.148+0000 7f7baaa00640 1 -- 172.21.15.161:0/1200899728 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ba411dd00 con 0x7f7bb415d5e0 2024-05-22T21:18:04.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.148+0000 7f7baaa00640 1 -- 172.21.15.161:0/1200899728 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f7b9006e1f0 con 0x7f7bb415d5e0 2024-05-22T21:18:04.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.148+0000 7f7ba9400640 1 --2- 172.21.15.161:0/1200899728 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7b900666c0 0x7f7b90068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.148+0000 7f7ba9400640 1 --2- 172.21.15.161:0/1200899728 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7b900666c0 0x7f7b90068b80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40cf320 tx=0x7f7b9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:04.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.152+0000 7f7baaa00640 1 -- 172.21.15.161:0/1200899728 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7ba40b1060 con 0x7f7bb415d5e0 2024-05-22T21:18:04.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.308+0000 7f7bb9200640 1 -- 172.21.15.161:0/1200899728 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bb4165c70 con 0x7f7bb415d5e0 2024-05-22T21:18:04.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.312+0000 7f7baaa00640 1 -- 172.21.15.161:0/1200899728 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16170 (secure 0 0 0) 0x7f7ba40eafd0 con 0x7f7bb415d5e0 2024-05-22T21:18:04.314 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:04.314 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":90,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:18:03.344552+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:17:12.574579+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-22T21:17:13.013467+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:18:04.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.312+0000 7f7bb9200640 1 -- 172.21.15.161:0/1200899728 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7b900666c0 msgr2=0x7f7b90068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.312+0000 7f7bb9200640 1 --2- 172.21.15.161:0/1200899728 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7b900666c0 0x7f7b90068b80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40cf320 tx=0x7f7b9c009040 comp rx=0 tx=0).stop 2024-05-22T21:18:04.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.312+0000 7f7bb9200640 1 -- 172.21.15.161:0/1200899728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bb415d5e0 msgr2=0x7f7bb4150d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.312+0000 7f7bb9200640 1 --2- 172.21.15.161:0/1200899728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bb415d5e0 0x7f7bb4150d80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4000c10 tx=0x7f7ba4019130 comp rx=0 tx=0).stop 2024-05-22T21:18:04.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.312+0000 7f7bb9200640 1 -- 172.21.15.161:0/1200899728 shutdown_connections 2024-05-22T21:18:04.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.312+0000 7f7bb9200640 1 --2- 172.21.15.161:0/1200899728 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7b900666c0 0x7f7b90068b80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.312+0000 7f7bb9200640 1 --2- 172.21.15.161:0/1200899728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb415e130 0x7f7bb412e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.312+0000 7f7bb9200640 1 --2- 172.21.15.161:0/1200899728 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7bb415dd50 0x7f7bb40ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.312+0000 7f7bb9200640 1 --2- 172.21.15.161:0/1200899728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bb415d5e0 0x7f7bb4150d80 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.312+0000 7f7bb9200640 1 -- 172.21.15.161:0/1200899728 >> 172.21.15.161:0/1200899728 conn(0x7f7bb40b0bc0 msgr2=0x7f7bb40aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:04.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.312+0000 7f7bb9200640 1 -- 172.21.15.161:0/1200899728 shutdown_connections 2024-05-22T21:18:04.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.312+0000 7f7bb9200640 1 -- 172.21.15.161:0/1200899728 wait complete. 2024-05-22T21:18:04.334 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:17:12.574579+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:17:13.013467+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:18:04.335 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:18:04.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.472+0000 7fc170200640 1 Processor -- start 2024-05-22T21:18:04.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.472+0000 7fc170200640 1 -- start start 2024-05-22T21:18:04.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.472+0000 7fc170200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1680cc530 0x7fc1680d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.472+0000 7fc170200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1680cfeb0 con 0x7fc16815d330 2024-05-22T21:18:04.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.472+0000 7fc170200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc16815d710 con 0x7fc1680cf130 2024-05-22T21:18:04.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.472+0000 7fc170200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1680cf610 con 0x7fc1680cc530 2024-05-22T21:18:04.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.472+0000 7fc16d800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc16815d330 0x7fc16806cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39122/0 (socket says 172.21.15.161:39122) 2024-05-22T21:18:04.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.472+0000 7fc16d800640 1 -- 172.21.15.161:0/158299752 learned_addr learned my addr 172.21.15.161:0/158299752 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:04.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc167e00640 1 -- 172.21.15.161:0/158299752 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2433608341 0 0) 0x7fc1680cfeb0 con 0x7fc16815d330 2024-05-22T21:18:04.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc16ce00640 1 --2- 172.21.15.161:0/158299752 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1680cc530 0x7fc1680d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc167e00640 1 -- 172.21.15.161:0/158299752 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc150003530 con 0x7fc16815d330 2024-05-22T21:18:04.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc167e00640 1 -- 172.21.15.161:0/158299752 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4207533918 0 0) 0x7fc150003530 con 0x7fc16815d330 2024-05-22T21:18:04.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc167e00640 1 -- 172.21.15.161:0/158299752 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1680cc530 msgr2=0x7fc1680d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:04.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc167e00640 1 --2- 172.21.15.161:0/158299752 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1680cc530 0x7fc1680d0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc167e00640 1 -- 172.21.15.161:0/158299752 >> v1:172.21.15.179:6789/0 conn(0x7fc1680cf130 legacy=0x7fc168071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc167e00640 1 -- 172.21.15.161:0/158299752 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1680cf8f0 con 0x7fc16815d330 2024-05-22T21:18:04.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc167e00640 1 -- 172.21.15.161:0/158299752 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fc15c001680 con 0x7fc16815d330 2024-05-22T21:18:04.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc167e00640 1 -- 172.21.15.161:0/158299752 >> v1:172.21.15.161:6789/0 conn(0x7fc16815d330 legacy=0x7fc16806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc167e00640 1 --2- 172.21.15.161:0/158299752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1500039b0 0x7fc150003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc167e00640 1 --2- 172.21.15.161:0/158299752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1500047d0 0x7fc150004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc167e00640 1 --2- 172.21.15.161:0/158299752 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1680cc530 0x7fc1500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc167e00640 1 -- 172.21.15.161:0/158299752 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1680cf610 con 0x7fc1500047d0 2024-05-22T21:18:04.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc167e00640 1 -- 172.21.15.161:0/158299752 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15000bac0 con 0x7fc1500039b0 2024-05-22T21:18:04.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc167e00640 1 -- 172.21.15.161:0/158299752 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15000bc80 con 0x7fc1680cc530 2024-05-22T21:18:04.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc16ce00640 1 --2- 172.21.15.161:0/158299752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1500047d0 0x7fc150004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc16d800640 1 --2- 172.21.15.161:0/158299752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1500039b0 0x7fc150003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc16e200640 1 --2- 172.21.15.161:0/158299752 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1680cc530 0x7fc1500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc16ce00640 1 -- 172.21.15.161:0/158299752 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1680cc530 msgr2=0x7fc1500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:04.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc16ce00640 1 --2- 172.21.15.161:0/158299752 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1680cc530 0x7fc1500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc16ce00640 1 -- 172.21.15.161:0/158299752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1500039b0 msgr2=0x7fc150003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:04.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc16ce00640 1 --2- 172.21.15.161:0/158299752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1500039b0 0x7fc150003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.476+0000 7fc16ce00640 1 -- 172.21.15.161:0/158299752 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc150003530 con 0x7fc1500047d0 2024-05-22T21:18:04.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.480+0000 7fc16ce00640 1 --2- 172.21.15.161:0/158299752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1500047d0 0x7fc150004c50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc154009ae0 tx=0x7fc15401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492c9a5af9e280f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:04.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.480+0000 7fc167e00640 1 -- 172.21.15.161:0/158299752 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1540a5520 con 0x7fc1500047d0 2024-05-22T21:18:04.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.480+0000 7fc167e00640 1 -- 172.21.15.161:0/158299752 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1540a5680 con 0x7fc1500047d0 2024-05-22T21:18:04.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.480+0000 7fc167e00640 1 -- 172.21.15.161:0/158299752 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1540a5a30 con 0x7fc1500047d0 2024-05-22T21:18:04.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.480+0000 7fc170200640 1 -- 172.21.15.161:0/158299752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1500047d0 msgr2=0x7fc150004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.480+0000 7fc170200640 1 --2- 172.21.15.161:0/158299752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1500047d0 0x7fc150004c50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc154009ae0 tx=0x7fc15401ca40 comp rx=0 tx=0).stop 2024-05-22T21:18:04.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.480+0000 7fc16e200640 1 -- 172.21.15.161:0/158299752 reap_dead start 2024-05-22T21:18:04.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.480+0000 7fc170200640 1 -- 172.21.15.161:0/158299752 shutdown_connections 2024-05-22T21:18:04.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.480+0000 7fc170200640 1 -- 172.21.15.161:0/158299752 >> 172.21.15.161:0/158299752 conn(0x7fc1680d0090 msgr2=0x7fc1680d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:04.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.480+0000 7fc170200640 1 -- 172.21.15.161:0/158299752 shutdown_connections 2024-05-22T21:18:04.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.480+0000 7fc170200640 1 -- 172.21.15.161:0/158299752 wait complete. 2024-05-22T21:18:04.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.480+0000 7fc170200640 1 Processor -- start 2024-05-22T21:18:04.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.480+0000 7fc170200640 1 -- start start 2024-05-22T21:18:04.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.480+0000 7fc170200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1500047d0 0x7fc16814c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc170200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc16815ed30 0x7fc16815c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc170200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc16815f1b0 0x7fc16815c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc170200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16814ca20 con 0x7fc16815ed30 2024-05-22T21:18:04.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc170200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1680b3000 con 0x7fc16815f1b0 2024-05-22T21:18:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc170200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1680b32e0 con 0x7fc1500047d0 2024-05-22T21:18:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc16d800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1500047d0 0x7fc16814c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc16ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc16815ed30 0x7fc16815c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc16d800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1500047d0 0x7fc16814c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:34660/0 (socket says 172.21.15.161:34660) 2024-05-22T21:18:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc16d800640 1 -- 172.21.15.161:0/3440576349 learned_addr learned my addr 172.21.15.161:0/3440576349 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc16e200640 1 --2- 172.21.15.161:0/3440576349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc16815f1b0 0x7fc16815c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc16ce00640 1 -- 172.21.15.161:0/3440576349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1500047d0 msgr2=0x7fc16814c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc16ce00640 1 --2- 172.21.15.161:0/3440576349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1500047d0 0x7fc16814c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc16ce00640 1 -- 172.21.15.161:0/3440576349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc16815f1b0 msgr2=0x7fc16815c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc16ce00640 1 --2- 172.21.15.161:0/3440576349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc16815f1b0 0x7fc16815c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc16ce00640 1 -- 172.21.15.161:0/3440576349 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1680b35c0 con 0x7fc16815ed30 2024-05-22T21:18:04.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc16d800640 1 --2- 172.21.15.161:0/3440576349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1500047d0 0x7fc16814c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:04.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc16ce00640 1 --2- 172.21.15.161:0/3440576349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc16815ed30 0x7fc16815c150 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc1540ac000 tx=0x7fc150005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:04.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc165600640 1 -- 172.21.15.161:0/3440576349 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15400f000 con 0x7fc16815ed30 2024-05-22T21:18:04.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc165600640 1 -- 172.21.15.161:0/3440576349 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc154004020 con 0x7fc16815ed30 2024-05-22T21:18:04.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc170200640 1 -- 172.21.15.161:0/3440576349 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1680b38a0 con 0x7fc16815ed30 2024-05-22T21:18:04.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc165600640 1 -- 172.21.15.161:0/3440576349 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1540043d0 con 0x7fc16815ed30 2024-05-22T21:18:04.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc170200640 1 -- 172.21.15.161:0/3440576349 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1680b2c60 con 0x7fc16815ed30 2024-05-22T21:18:04.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc170200640 1 -- 172.21.15.161:0/3440576349 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1680d09b0 con 0x7fc16815ed30 2024-05-22T21:18:04.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.484+0000 7fc165600640 1 -- 172.21.15.161:0/3440576349 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc1540b4040 con 0x7fc16815ed30 2024-05-22T21:18:04.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.488+0000 7fc165600640 1 --2- 172.21.15.161:0/3440576349 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc148066690 0x7fc148068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.488+0000 7fc165600640 1 -- 172.21.15.161:0/3440576349 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc15411e230 con 0x7fc16815ed30 2024-05-22T21:18:04.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.488+0000 7fc16d800640 1 --2- 172.21.15.161:0/3440576349 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc148066690 0x7fc148068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.488+0000 7fc165600640 1 -- 172.21.15.161:0/3440576349 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7fc14806e1c0 con 0x7fc16815ed30 2024-05-22T21:18:04.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.488+0000 7fc16d800640 1 --2- 172.21.15.161:0/3440576349 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc148066690 0x7fc148068b50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0017b0 tx=0x7fc15c001580 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:04.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.492+0000 7fc165600640 1 -- 172.21.15.161:0/3440576349 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc1540eb760 con 0x7fc16815ed30 2024-05-22T21:18:04.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.644+0000 7fc170200640 1 -- 172.21.15.161:0/3440576349 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc16814ca20 con 0x7fc16815ed30 2024-05-22T21:18:04.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.644+0000 7fc165600640 1 -- 172.21.15.161:0/3440576349 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc1540b20a0 con 0x7fc16815ed30 2024-05-22T21:18:04.646 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:18:04.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.644+0000 7fc170200640 1 -- 172.21.15.161:0/3440576349 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc148066690 msgr2=0x7fc148068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.644+0000 7fc170200640 1 --2- 172.21.15.161:0/3440576349 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc148066690 0x7fc148068b50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0017b0 tx=0x7fc15c001580 comp rx=0 tx=0).stop 2024-05-22T21:18:04.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.644+0000 7fc170200640 1 -- 172.21.15.161:0/3440576349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc16815ed30 msgr2=0x7fc16815c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.644+0000 7fc170200640 1 --2- 172.21.15.161:0/3440576349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc16815ed30 0x7fc16815c150 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc1540ac000 tx=0x7fc150005800 comp rx=0 tx=0).stop 2024-05-22T21:18:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.644+0000 7fc170200640 1 -- 172.21.15.161:0/3440576349 shutdown_connections 2024-05-22T21:18:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.644+0000 7fc170200640 1 --2- 172.21.15.161:0/3440576349 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc148066690 0x7fc148068b50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.644+0000 7fc170200640 1 --2- 172.21.15.161:0/3440576349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc16815f1b0 0x7fc16815c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.644+0000 7fc170200640 1 --2- 172.21.15.161:0/3440576349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc16815ed30 0x7fc16815c150 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.644+0000 7fc170200640 1 --2- 172.21.15.161:0/3440576349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1500047d0 0x7fc16814c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.644+0000 7fc170200640 1 -- 172.21.15.161:0/3440576349 >> 172.21.15.161:0/3440576349 conn(0x7fc1680d0090 msgr2=0x7fc1680cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.644+0000 7fc170200640 1 -- 172.21.15.161:0/3440576349 shutdown_connections 2024-05-22T21:18:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.644+0000 7fc170200640 1 -- 172.21.15.161:0/3440576349 wait complete. 2024-05-22T21:18:04.666 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:18:04.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.800+0000 7f8129200640 1 Processor -- start 2024-05-22T21:18:04.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.800+0000 7f8129200640 1 -- start start 2024-05-22T21:18:04.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.804+0000 7f8129200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f812413fe50 0x7f8124164e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.804+0000 7f8129200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812413d3a0 con 0x7f812413cfc0 2024-05-22T21:18:04.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.804+0000 7f8129200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81240cd920 con 0x7f81241402e0 2024-05-22T21:18:04.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.804+0000 7f8129200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81240cdc00 con 0x7f812413fe50 2024-05-22T21:18:04.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.804+0000 7f8121e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f812413cfc0 0x7f812415af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39126/0 (socket says 172.21.15.161:39126) 2024-05-22T21:18:04.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.804+0000 7f8121e00640 1 -- 172.21.15.161:0/2919507244 learned_addr learned my addr 172.21.15.161:0/2919507244 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:04.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.804+0000 7f8122800640 1 --2- 172.21.15.161:0/2919507244 >> v2:172.21.15.161:3300/0 conn(0x7f812413fe50 0x7f8124164e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.804+0000 7f8120a00640 1 -- 172.21.15.161:0/2919507244 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327678028 0 0) 0x7f812413d3a0 con 0x7f812413cfc0 2024-05-22T21:18:04.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.804+0000 7f8120a00640 1 -- 172.21.15.161:0/2919507244 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8108005740 con 0x7f812413cfc0 2024-05-22T21:18:04.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.804+0000 7f8122800640 1 -- 172.21.15.161:0/2919507244 >> v1:172.21.15.179:6789/0 conn(0x7f81241402e0 legacy=0x7f812413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:04.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.804+0000 7f8122800640 1 -- 172.21.15.161:0/2919507244 >> v1:172.21.15.161:6789/0 conn(0x7f812413cfc0 legacy=0x7f812415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.804+0000 7f8122800640 1 -- 172.21.15.161:0/2919507244 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81240cdee0 con 0x7f812413fe50 2024-05-22T21:18:04.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.808+0000 7f8122800640 1 --2- 172.21.15.161:0/2919507244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f812413fe50 0x7f8124164e60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f811c009d90 tx=0x7f811c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a400c6374daa2650 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:04.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.808+0000 7f8120a00640 1 -- 172.21.15.161:0/2919507244 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c0a66b0 con 0x7f812413fe50 2024-05-22T21:18:04.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.808+0000 7f8120a00640 1 -- 172.21.15.161:0/2919507244 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c0a6810 con 0x7f812413fe50 2024-05-22T21:18:04.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.808+0000 7f8120a00640 1 -- 172.21.15.161:0/2919507244 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c0a6ba0 con 0x7f812413fe50 2024-05-22T21:18:04.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.808+0000 7f8129200640 1 -- 172.21.15.161:0/2919507244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f812413fe50 msgr2=0x7f8124164e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.808+0000 7f8129200640 1 --2- 172.21.15.161:0/2919507244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f812413fe50 0x7f8124164e60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f811c009d90 tx=0x7f811c01c8e0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.808+0000 7f8129200640 1 -- 172.21.15.161:0/2919507244 shutdown_connections 2024-05-22T21:18:04.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.808+0000 7f8129200640 1 --2- 172.21.15.161:0/2919507244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f812413fe50 0x7f8124164e60 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.808+0000 7f8129200640 1 -- 172.21.15.161:0/2919507244 >> 172.21.15.161:0/2919507244 conn(0x7f812406a2d0 msgr2=0x7f812406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:04.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.808+0000 7f8129200640 1 -- 172.21.15.161:0/2919507244 shutdown_connections 2024-05-22T21:18:04.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.808+0000 7f8129200640 1 -- 172.21.15.161:0/2919507244 wait complete. 2024-05-22T21:18:04.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.808+0000 7f8129200640 1 Processor -- start 2024-05-22T21:18:04.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.808+0000 7f8129200640 1 -- start start 2024-05-22T21:18:04.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8129200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f812413cfc0 0x7f8124141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8129200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f812413fe50 0x7f8124157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8129200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81241402e0 0x7f8124138fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8129200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812415bdd0 con 0x7f81241402e0 2024-05-22T21:18:04.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8129200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812415b1c0 con 0x7f812413fe50 2024-05-22T21:18:04.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8129200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812415b4a0 con 0x7f812413cfc0 2024-05-22T21:18:04.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8122800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81241402e0 0x7f8124138fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8122800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81241402e0 0x7f8124138fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45176/0 (socket says 172.21.15.161:45176) 2024-05-22T21:18:04.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8122800640 1 -- 172.21.15.161:0/900383332 learned_addr learned my addr 172.21.15.161:0/900383332 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:04.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8121400640 1 --2- 172.21.15.161:0/900383332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f812413fe50 0x7f8124157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8121e00640 1 --2- 172.21.15.161:0/900383332 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f812413cfc0 0x7f8124141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8122800640 1 -- 172.21.15.161:0/900383332 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f812413cfc0 msgr2=0x7f8124141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8122800640 1 --2- 172.21.15.161:0/900383332 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f812413cfc0 0x7f8124141ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8122800640 1 -- 172.21.15.161:0/900383332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f812413fe50 msgr2=0x7f8124157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8122800640 1 --2- 172.21.15.161:0/900383332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f812413fe50 0x7f8124157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8122800640 1 -- 172.21.15.161:0/900383332 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c009a70 con 0x7f81241402e0 2024-05-22T21:18:04.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8121400640 1 --2- 172.21.15.161:0/900383332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f812413fe50 0x7f8124157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:04.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8122800640 1 --2- 172.21.15.161:0/900383332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81241402e0 0x7f8124138fd0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f811c01a510 tx=0x7f811c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:04.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f811a000640 1 -- 172.21.15.161:0/900383332 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c0a6d90 con 0x7f81241402e0 2024-05-22T21:18:04.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f811a000640 1 -- 172.21.15.161:0/900383332 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c0b8430 con 0x7f81241402e0 2024-05-22T21:18:04.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f811a000640 1 -- 172.21.15.161:0/900383332 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c0b87c0 con 0x7f81241402e0 2024-05-22T21:18:04.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8129200640 1 -- 172.21.15.161:0/900383332 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812415b780 con 0x7f81241402e0 2024-05-22T21:18:04.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8129200640 1 -- 172.21.15.161:0/900383332 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812415a530 con 0x7f81241402e0 2024-05-22T21:18:04.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f8129200640 1 -- 172.21.15.161:0/900383332 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812413a5d0 con 0x7f81241402e0 2024-05-22T21:18:04.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.812+0000 7f811a000640 1 -- 172.21.15.161:0/900383332 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f811c0b8920 con 0x7f81241402e0 2024-05-22T21:18:04.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.816+0000 7f811a000640 1 --2- 172.21.15.161:0/900383332 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8104066690 0x7f8104068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:04.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.816+0000 7f811a000640 1 -- 172.21.15.161:0/900383332 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f811c0aa070 con 0x7f81241402e0 2024-05-22T21:18:04.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.816+0000 7f811a000640 1 -- 172.21.15.161:0/900383332 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f810406e1c0 con 0x7f81241402e0 2024-05-22T21:18:04.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.816+0000 7f8121e00640 1 --2- 172.21.15.161:0/900383332 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8104066690 0x7f8104068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:04.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.816+0000 7f8121e00640 1 --2- 172.21.15.161:0/900383332 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8104066690 0x7f8104068b50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f81140011f0 tx=0x7f8114009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:04.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.820+0000 7f811a000640 1 -- 172.21.15.161:0/900383332 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f811c0c8020 con 0x7f81241402e0 2024-05-22T21:18:04.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.976+0000 7f8129200640 1 -- 172.21.15.161:0/900383332 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8124067220 con 0x7f81241402e0 2024-05-22T21:18:04.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.976+0000 7f811a000640 1 -- 172.21.15.161:0/900383332 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f812413a5d0 con 0x7f81241402e0 2024-05-22T21:18:04.977 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:18:04.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.976+0000 7f8129200640 1 -- 172.21.15.161:0/900383332 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8104066690 msgr2=0x7f8104068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.976+0000 7f8129200640 1 --2- 172.21.15.161:0/900383332 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8104066690 0x7f8104068b50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f81140011f0 tx=0x7f8114009040 comp rx=0 tx=0).stop 2024-05-22T21:18:04.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.976+0000 7f8129200640 1 -- 172.21.15.161:0/900383332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81241402e0 msgr2=0x7f8124138fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:04.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.976+0000 7f8129200640 1 --2- 172.21.15.161:0/900383332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81241402e0 0x7f8124138fd0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f811c01a510 tx=0x7f811c019680 comp rx=0 tx=0).stop 2024-05-22T21:18:04.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.976+0000 7f8129200640 1 -- 172.21.15.161:0/900383332 shutdown_connections 2024-05-22T21:18:04.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.976+0000 7f8129200640 1 --2- 172.21.15.161:0/900383332 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8104066690 0x7f8104068b50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.976+0000 7f8129200640 1 --2- 172.21.15.161:0/900383332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81241402e0 0x7f8124138fd0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.976+0000 7f8129200640 1 --2- 172.21.15.161:0/900383332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f812413fe50 0x7f8124157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.976+0000 7f8129200640 1 --2- 172.21.15.161:0/900383332 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f812413cfc0 0x7f8124141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:04.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.976+0000 7f8129200640 1 -- 172.21.15.161:0/900383332 >> 172.21.15.161:0/900383332 conn(0x7f812406a2d0 msgr2=0x7f8124159200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:04.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.976+0000 7f8129200640 1 -- 172.21.15.161:0/900383332 shutdown_connections 2024-05-22T21:18:04.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:04.976+0000 7f8129200640 1 -- 172.21.15.161:0/900383332 wait complete. 2024-05-22T21:18:04.997 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:18:05.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.132+0000 7f6a5ae00640 1 Processor -- start 2024-05-22T21:18:05.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.132+0000 7f6a5ae00640 1 -- start start 2024-05-22T21:18:05.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a5ae00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6a5413cfc0 0x7f6a541402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a5ae00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a5413fe50 0x7f6a5406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a5ae00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5413d3a0 con 0x7f6a5415af60 2024-05-22T21:18:05.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a5ae00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a540cd920 con 0x7f6a5413cfc0 2024-05-22T21:18:05.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a5ae00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540cdc00 con 0x7f6a5413fe50 2024-05-22T21:18:05.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a53400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f6a5415af60 0x7f6a5413f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39138/0 (socket says 172.21.15.161:39138) 2024-05-22T21:18:05.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a53400640 1 -- 172.21.15.161:0/2816565047 learned_addr learned my addr 172.21.15.161:0/2816565047 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:05.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a58e00640 1 --2- 172.21.15.161:0/2816565047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a5413fe50 0x7f6a5406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a52a00640 1 -- 172.21.15.161:0/2816565047 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573958179 0 0) 0x7f6a5413d3a0 con 0x7f6a5415af60 2024-05-22T21:18:05.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a53e00640 1 --2- 172.21.15.161:0/2816565047 >> v2:172.21.15.179:3300/0 conn(0x7f6a5413cfc0 0x7f6a541402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a52a00640 1 -- 172.21.15.161:0/2816565047 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a38003530 con 0x7f6a5415af60 2024-05-22T21:18:05.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a52a00640 1 -- 172.21.15.161:0/2816565047 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4290327399 0 0) 0x7f6a38003530 con 0x7f6a5415af60 2024-05-22T21:18:05.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a52a00640 1 -- 172.21.15.161:0/2816565047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a5413fe50 msgr2=0x7f6a5406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:05.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a52a00640 1 --2- 172.21.15.161:0/2816565047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a5413fe50 0x7f6a5406fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a52a00640 1 -- 172.21.15.161:0/2816565047 >> v2:172.21.15.179:3300/0 conn(0x7f6a5413cfc0 msgr2=0x7f6a541402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:05.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a52a00640 1 --2- 172.21.15.161:0/2816565047 >> v2:172.21.15.179:3300/0 conn(0x7f6a5413cfc0 0x7f6a541402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a52a00640 1 -- 172.21.15.161:0/2816565047 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a540cdee0 con 0x7f6a5415af60 2024-05-22T21:18:05.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a52a00640 1 -- 172.21.15.161:0/2816565047 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f6a40001680 con 0x7f6a5415af60 2024-05-22T21:18:05.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a58e00640 1 --2- 172.21.15.161:0/2816565047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a5413fe50 0x7f6a5406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:05.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a52a00640 1 -- 172.21.15.161:0/2816565047 >> v1:172.21.15.161:6789/0 conn(0x7f6a5415af60 legacy=0x7f6a5413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:05.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a52a00640 1 --2- 172.21.15.161:0/2816565047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a38003980 0x7f6a38003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a52a00640 1 --2- 172.21.15.161:0/2816565047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a5413fe50 0x7f6a380047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a52a00640 1 --2- 172.21.15.161:0/2816565047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a38004d10 0x7f6a38005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a52a00640 1 -- 172.21.15.161:0/2816565047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540cd920 con 0x7f6a38003980 2024-05-22T21:18:05.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a52a00640 1 -- 172.21.15.161:0/2816565047 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540cdc00 con 0x7f6a38004d10 2024-05-22T21:18:05.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a52a00640 1 -- 172.21.15.161:0/2816565047 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3800ba10 con 0x7f6a5413fe50 2024-05-22T21:18:05.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a53e00640 1 --2- 172.21.15.161:0/2816565047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a38003980 0x7f6a38003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a58e00640 1 --2- 172.21.15.161:0/2816565047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a38004d10 0x7f6a38005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a53400640 1 --2- 172.21.15.161:0/2816565047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a5413fe50 0x7f6a380047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a53e00640 1 -- 172.21.15.161:0/2816565047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a5413fe50 msgr2=0x7f6a380047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:05.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a53e00640 1 --2- 172.21.15.161:0/2816565047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a5413fe50 0x7f6a380047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a53e00640 1 -- 172.21.15.161:0/2816565047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a38004d10 msgr2=0x7f6a38005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:05.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a53e00640 1 --2- 172.21.15.161:0/2816565047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a38004d10 0x7f6a38005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.136+0000 7f6a53e00640 1 -- 172.21.15.161:0/2816565047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a38003530 con 0x7f6a38003980 2024-05-22T21:18:05.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.140+0000 7f6a53e00640 1 --2- 172.21.15.161:0/2816565047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a38003980 0x7f6a38003da0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009fc0 tx=0x7f6a4401cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ec3658941860e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:05.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.140+0000 7f6a52a00640 1 -- 172.21.15.161:0/2816565047 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a440a5b70 con 0x7f6a38003980 2024-05-22T21:18:05.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.140+0000 7f6a52a00640 1 -- 172.21.15.161:0/2816565047 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4400f040 con 0x7f6a38003980 2024-05-22T21:18:05.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.140+0000 7f6a52a00640 1 -- 172.21.15.161:0/2816565047 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a440ad440 con 0x7f6a38003980 2024-05-22T21:18:05.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.140+0000 7f6a5ae00640 1 -- 172.21.15.161:0/2816565047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a38003980 msgr2=0x7f6a38003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:05.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.140+0000 7f6a5ae00640 1 --2- 172.21.15.161:0/2816565047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a38003980 0x7f6a38003da0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009fc0 tx=0x7f6a4401cc90 comp rx=0 tx=0).stop 2024-05-22T21:18:05.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.140+0000 7f6a58e00640 1 -- 172.21.15.161:0/2816565047 reap_dead start 2024-05-22T21:18:05.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.140+0000 7f6a5ae00640 1 -- 172.21.15.161:0/2816565047 shutdown_connections 2024-05-22T21:18:05.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.140+0000 7f6a5ae00640 1 -- 172.21.15.161:0/2816565047 >> 172.21.15.161:0/2816565047 conn(0x7f6a5406a2d0 msgr2=0x7f6a5406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:05.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.140+0000 7f6a5ae00640 1 -- 172.21.15.161:0/2816565047 shutdown_connections 2024-05-22T21:18:05.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.140+0000 7f6a5ae00640 1 -- 172.21.15.161:0/2816565047 wait complete. 2024-05-22T21:18:05.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.140+0000 7f6a5ae00640 1 Processor -- start 2024-05-22T21:18:05.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.140+0000 7f6a5ae00640 1 -- start start 2024-05-22T21:18:05.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a5ae00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a38003980 0x7f6a54155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a5ae00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a540ae110 0x7f6a540ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a5ae00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a540aec50 0x7f6a540ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a5ae00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a54156090 con 0x7f6a540ae110 2024-05-22T21:18:05.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a5ae00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540af0f0 con 0x7f6a38003980 2024-05-22T21:18:05.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a5ae00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a540af3d0 con 0x7f6a540aec50 2024-05-22T21:18:05.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a53400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a540ae110 0x7f6a540ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a53400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a540ae110 0x7f6a540ae710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45196/0 (socket says 172.21.15.161:45196) 2024-05-22T21:18:05.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a53400640 1 -- 172.21.15.161:0/570480503 learned_addr learned my addr 172.21.15.161:0/570480503 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:05.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a53e00640 1 --2- 172.21.15.161:0/570480503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a38003980 0x7f6a54155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a58e00640 1 --2- 172.21.15.161:0/570480503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a540aec50 0x7f6a540ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a53400640 1 -- 172.21.15.161:0/570480503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a540aec50 msgr2=0x7f6a540ad0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:05.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a53400640 1 --2- 172.21.15.161:0/570480503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a540aec50 0x7f6a540ad0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a53400640 1 -- 172.21.15.161:0/570480503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a38003980 msgr2=0x7f6a54155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:05.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a53400640 1 --2- 172.21.15.161:0/570480503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a38003980 0x7f6a54155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a53400640 1 -- 172.21.15.161:0/570480503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a540af6b0 con 0x7f6a540ae110 2024-05-22T21:18:05.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a53400640 1 --2- 172.21.15.161:0/570480503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a540ae110 0x7f6a540ae710 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6a40001ec0 tx=0x7f6a4000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:05.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a4be00640 1 -- 172.21.15.161:0/570480503 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4000c8d0 con 0x7f6a540ae110 2024-05-22T21:18:05.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a4be00640 1 -- 172.21.15.161:0/570480503 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4000ca50 con 0x7f6a540ae110 2024-05-22T21:18:05.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a4be00640 1 -- 172.21.15.161:0/570480503 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4000ce00 con 0x7f6a540ae110 2024-05-22T21:18:05.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a5ae00640 1 -- 172.21.15.161:0/570480503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a540af990 con 0x7f6a540ae110 2024-05-22T21:18:05.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a5ae00640 1 -- 172.21.15.161:0/570480503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a540ad5e0 con 0x7f6a540ae110 2024-05-22T21:18:05.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a5ae00640 1 -- 172.21.15.161:0/570480503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a54067220 con 0x7f6a540ae110 2024-05-22T21:18:05.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.144+0000 7f6a4be00640 1 -- 172.21.15.161:0/570480503 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6a4001d460 con 0x7f6a540ae110 2024-05-22T21:18:05.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.148+0000 7f6a4be00640 1 --2- 172.21.15.161:0/570480503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a34066690 0x7f6a34068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.148+0000 7f6a4be00640 1 -- 172.21.15.161:0/570480503 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a40084190 con 0x7f6a540ae110 2024-05-22T21:18:05.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.148+0000 7f6a53e00640 1 --2- 172.21.15.161:0/570480503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a34066690 0x7f6a34068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.148+0000 7f6a4be00640 1 -- 172.21.15.161:0/570480503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f6a3406e1c0 con 0x7f6a540ae110 2024-05-22T21:18:05.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.148+0000 7f6a53e00640 1 --2- 172.21.15.161:0/570480503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a34066690 0x7f6a34068b50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6a440af000 tx=0x7f6a440a7040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:05.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.152+0000 7f6a4be00640 1 -- 172.21.15.161:0/570480503 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6a40051510 con 0x7f6a540ae110 2024-05-22T21:18:05.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.304+0000 7f6a5ae00640 1 -- 172.21.15.161:0/570480503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6a540afe40 con 0x7f6a540ae110 2024-05-22T21:18:05.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.304+0000 7f6a4be00640 1 -- 172.21.15.161:0/570480503 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a40054b30 con 0x7f6a540ae110 2024-05-22T21:18:05.306 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:18:05.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.304+0000 7f6a5ae00640 1 -- 172.21.15.161:0/570480503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a34066690 msgr2=0x7f6a34068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:05.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.304+0000 7f6a5ae00640 1 --2- 172.21.15.161:0/570480503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a34066690 0x7f6a34068b50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6a440af000 tx=0x7f6a440a7040 comp rx=0 tx=0).stop 2024-05-22T21:18:05.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.304+0000 7f6a5ae00640 1 -- 172.21.15.161:0/570480503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a540ae110 msgr2=0x7f6a540ae710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:05.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.304+0000 7f6a5ae00640 1 --2- 172.21.15.161:0/570480503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a540ae110 0x7f6a540ae710 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6a40001ec0 tx=0x7f6a4000ed50 comp rx=0 tx=0).stop 2024-05-22T21:18:05.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.304+0000 7f6a5ae00640 1 -- 172.21.15.161:0/570480503 shutdown_connections 2024-05-22T21:18:05.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.304+0000 7f6a5ae00640 1 --2- 172.21.15.161:0/570480503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a34066690 0x7f6a34068b50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.304+0000 7f6a5ae00640 1 --2- 172.21.15.161:0/570480503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a540aec50 0x7f6a540ad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.304+0000 7f6a5ae00640 1 --2- 172.21.15.161:0/570480503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a540ae110 0x7f6a540ae710 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.304+0000 7f6a5ae00640 1 --2- 172.21.15.161:0/570480503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a38003980 0x7f6a54155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.304+0000 7f6a5ae00640 1 -- 172.21.15.161:0/570480503 >> 172.21.15.161:0/570480503 conn(0x7f6a5406a2d0 msgr2=0x7f6a5406cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:05.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.304+0000 7f6a5ae00640 1 -- 172.21.15.161:0/570480503 shutdown_connections 2024-05-22T21:18:05.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.304+0000 7f6a5ae00640 1 -- 172.21.15.161:0/570480503 wait complete. 2024-05-22T21:18:05.327 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:18:05.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.464+0000 7fad23400640 1 Processor -- start 2024-05-22T21:18:05.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.464+0000 7fad23400640 1 -- start start 2024-05-22T21:18:05.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.464+0000 7fad23400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fad1c15d430 0x7fad1c0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.464+0000 7fad23400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fad1c0accf0 0x7fad1c160e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.464+0000 7fad23400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1c071fd0 con 0x7fad1c0ac8b0 2024-05-22T21:18:05.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.464+0000 7fad23400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1c070e80 con 0x7fad1c15d430 2024-05-22T21:18:05.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.464+0000 7fad23400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1c06c510 con 0x7fad1c0accf0 2024-05-22T21:18:05.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.464+0000 7fad20a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fad1c15d430 0x7fad1c0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.464+0000 7fad20a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fad1c15d430 0x7fad1c0707c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45202/0 (socket says 172.21.15.161:45202) 2024-05-22T21:18:05.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.464+0000 7fad20a00640 1 -- 172.21.15.161:0/3438611946 learned_addr learned my addr 172.21.15.161:0/3438611946 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:05.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.464+0000 7fad21400640 1 --2- 172.21.15.161:0/3438611946 >> v2:172.21.15.179:3300/0 conn(0x7fad1c0accf0 0x7fad1c160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.464+0000 7fad20a00640 1 -- 172.21.15.161:0/3438611946 >> v2:172.21.15.179:3300/0 conn(0x7fad1c0accf0 msgr2=0x7fad1c160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:05.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.464+0000 7fad20a00640 1 --2- 172.21.15.161:0/3438611946 >> v2:172.21.15.179:3300/0 conn(0x7fad1c0accf0 0x7fad1c160e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.464+0000 7fad20a00640 1 -- 172.21.15.161:0/3438611946 >> v1:172.21.15.179:6789/0 conn(0x7fad1c0ac8b0 legacy=0x7fad1c06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:05.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.464+0000 7fad20a00640 1 -- 172.21.15.161:0/3438611946 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c06c7f0 con 0x7fad1c15d430 2024-05-22T21:18:05.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.468+0000 7fad20a00640 1 --2- 172.21.15.161:0/3438611946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c15d430 0x7fad1c0707c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fad1c0aeac0 tx=0x7fad0c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a288bca2f55966 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:05.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.468+0000 7fad1be00640 1 -- 172.21.15.161:0/3438611946 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0ab600 con 0x7fad1c15d430 2024-05-22T21:18:05.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.468+0000 7fad1be00640 1 -- 172.21.15.161:0/3438611946 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c0a7050 con 0x7fad1c15d430 2024-05-22T21:18:05.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.468+0000 7fad1be00640 1 -- 172.21.15.161:0/3438611946 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0ab980 con 0x7fad1c15d430 2024-05-22T21:18:05.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.468+0000 7fad23400640 1 -- 172.21.15.161:0/3438611946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c15d430 msgr2=0x7fad1c0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:05.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.468+0000 7fad23400640 1 --2- 172.21.15.161:0/3438611946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c15d430 0x7fad1c0707c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fad1c0aeac0 tx=0x7fad0c01c420 comp rx=0 tx=0).stop 2024-05-22T21:18:05.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.468+0000 7fad23400640 1 -- 172.21.15.161:0/3438611946 shutdown_connections 2024-05-22T21:18:05.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.468+0000 7fad23400640 1 --2- 172.21.15.161:0/3438611946 >> v2:172.21.15.179:3300/0 conn(0x7fad1c0accf0 0x7fad1c160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.468+0000 7fad23400640 1 --2- 172.21.15.161:0/3438611946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c15d430 0x7fad1c0707c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.468+0000 7fad23400640 1 -- 172.21.15.161:0/3438611946 >> 172.21.15.161:0/3438611946 conn(0x7fad1c0b0bc0 msgr2=0x7fad1c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:05.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.468+0000 7fad23400640 1 -- 172.21.15.161:0/3438611946 shutdown_connections 2024-05-22T21:18:05.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.468+0000 7fad23400640 1 -- 172.21.15.161:0/3438611946 wait complete. 2024-05-22T21:18:05.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad23400640 1 Processor -- start 2024-05-22T21:18:05.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad23400640 1 -- start start 2024-05-22T21:18:05.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad23400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c0ac8b0 0x7fad1c0ad330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad23400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c0accf0 0x7fad1c15edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad23400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c15d430 0x7fad1c14f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad23400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c15f750 con 0x7fad1c0ac8b0 2024-05-22T21:18:05.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad23400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c15fa30 con 0x7fad1c0accf0 2024-05-22T21:18:05.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad23400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c15fd10 con 0x7fad1c15d430 2024-05-22T21:18:05.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad20a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c0ac8b0 0x7fad1c0ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad20a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c0ac8b0 0x7fad1c0ad330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45206/0 (socket says 172.21.15.161:45206) 2024-05-22T21:18:05.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad20a00640 1 -- 172.21.15.161:0/106003356 learned_addr learned my addr 172.21.15.161:0/106003356 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:05.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad21400640 1 --2- 172.21.15.161:0/106003356 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c15d430 0x7fad1c14f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad13e00640 1 --2- 172.21.15.161:0/106003356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c0accf0 0x7fad1c15edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad20a00640 1 -- 172.21.15.161:0/106003356 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c15d430 msgr2=0x7fad1c14f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:05.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad20a00640 1 --2- 172.21.15.161:0/106003356 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c15d430 0x7fad1c14f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad20a00640 1 -- 172.21.15.161:0/106003356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c0accf0 msgr2=0x7fad1c15edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:05.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad20a00640 1 --2- 172.21.15.161:0/106003356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c0accf0 0x7fad1c15edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad20a00640 1 -- 172.21.15.161:0/106003356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c0094d0 con 0x7fad1c0ac8b0 2024-05-22T21:18:05.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad20a00640 1 --2- 172.21.15.161:0/106003356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c0ac8b0 0x7fad1c0ad330 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fad0c01a050 tx=0x7fad0c0191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:05.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad19600640 1 -- 172.21.15.161:0/106003356 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0b4440 con 0x7fad1c0ac8b0 2024-05-22T21:18:05.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad19600640 1 -- 172.21.15.161:0/106003356 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c0b45a0 con 0x7fad1c0ac8b0 2024-05-22T21:18:05.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad23400640 1 -- 172.21.15.161:0/106003356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1c15fff0 con 0x7fad1c0ac8b0 2024-05-22T21:18:05.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad19600640 1 -- 172.21.15.161:0/106003356 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0b4950 con 0x7fad1c0ac8b0 2024-05-22T21:18:05.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad23400640 1 -- 172.21.15.161:0/106003356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1c1607f0 con 0x7fad1c0ac8b0 2024-05-22T21:18:05.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.472+0000 7fad23400640 1 -- 172.21.15.161:0/106003356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1c067220 con 0x7fad1c0ac8b0 2024-05-22T21:18:05.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.476+0000 7fad19600640 1 -- 172.21.15.161:0/106003356 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fad0c0b2070 con 0x7fad1c0ac8b0 2024-05-22T21:18:05.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.476+0000 7fad19600640 1 --2- 172.21.15.161:0/106003356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7facf8066690 0x7facf8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.476+0000 7fad19600640 1 -- 172.21.15.161:0/106003356 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad0c11e130 con 0x7fad1c0ac8b0 2024-05-22T21:18:05.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.476+0000 7fad19600640 1 -- 172.21.15.161:0/106003356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7facf806e1c0 con 0x7fad1c0ac8b0 2024-05-22T21:18:05.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.476+0000 7fad13e00640 1 --2- 172.21.15.161:0/106003356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7facf8066690 0x7facf8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.476+0000 7fad13e00640 1 --2- 172.21.15.161:0/106003356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7facf8066690 0x7facf8068b50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fad1c06e170 tx=0x7fad04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:05.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.480+0000 7fad19600640 1 -- 172.21.15.161:0/106003356 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fad0c0eb400 con 0x7fad1c0ac8b0 2024-05-22T21:18:05.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.668+0000 7fad23400640 1 -- 172.21.15.161:0/106003356 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad1c0ae7a0 con 0x7fad1c0ac8b0 2024-05-22T21:18:05.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.672+0000 7fad19600640 1 -- 172.21.15.161:0/106003356 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4745 (secure 0 0 0) 0x7fad0c0b0090 con 0x7fad1c0ac8b0 2024-05-22T21:18:05.673 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:05.673 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5567,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196093187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196093187},{"type":"v1","addr":"172.21.15.161:6836","nonce":196093187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6136,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4029369301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4029369301},{"type":"v1","addr":"172.21.15.161:6837","nonce":4029369301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6139,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3212560487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3212560487},{"type":"v1","addr":"172.21.15.179:6834","nonce":3212560487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:17:14.012254+0000","modified":"2024-05-22T21:17:17.029283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6306":{"gid":6306,"name":"b","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/3065973138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3065973138},{"type":"v1","addr":"172.21.15.179:6835","nonce":3065973138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-22T21:18:05.673 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 39 2024-05-22T21:18:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.672+0000 7fad23400640 1 -- 172.21.15.161:0/106003356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7facf8066690 msgr2=0x7facf8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.672+0000 7fad23400640 1 --2- 172.21.15.161:0/106003356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7facf8066690 0x7facf8068b50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fad1c06e170 tx=0x7fad04008040 comp rx=0 tx=0).stop 2024-05-22T21:18:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.672+0000 7fad23400640 1 -- 172.21.15.161:0/106003356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c0ac8b0 msgr2=0x7fad1c0ad330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:05.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.672+0000 7fad23400640 1 --2- 172.21.15.161:0/106003356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c0ac8b0 0x7fad1c0ad330 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fad0c01a050 tx=0x7fad0c0191c0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.672+0000 7fad23400640 1 -- 172.21.15.161:0/106003356 shutdown_connections 2024-05-22T21:18:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.672+0000 7fad23400640 1 --2- 172.21.15.161:0/106003356 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7facf8066690 0x7facf8068b50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.672+0000 7fad23400640 1 --2- 172.21.15.161:0/106003356 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad1c15d430 0x7fad1c14f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.672+0000 7fad23400640 1 --2- 172.21.15.161:0/106003356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c0accf0 0x7fad1c15edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.672+0000 7fad23400640 1 --2- 172.21.15.161:0/106003356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad1c0ac8b0 0x7fad1c0ad330 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.672+0000 7fad23400640 1 -- 172.21.15.161:0/106003356 >> 172.21.15.161:0/106003356 conn(0x7fad1c0b0bc0 msgr2=0x7fad1c0af760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.672+0000 7fad23400640 1 -- 172.21.15.161:0/106003356 shutdown_connections 2024-05-22T21:18:05.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.672+0000 7fad23400640 1 -- 172.21.15.161:0/106003356 wait complete. 2024-05-22T21:18:05.694 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:18:05.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.828+0000 7f52d9600640 1 Processor -- start 2024-05-22T21:18:05.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.828+0000 7f52d9600640 1 -- start start 2024-05-22T21:18:05.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.828+0000 7f52d9600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f52d415e130 0x7f52d415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.828+0000 7f52d9600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d415d410 0x7f52d415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.828+0000 7f52d9600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4071670 con 0x7f52d415dd50 2024-05-22T21:18:05.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.828+0000 7f52d9600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d415d870 con 0x7f52d415e130 2024-05-22T21:18:05.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.828+0000 7f52d9600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d4070a60 con 0x7f52d415d410 2024-05-22T21:18:05.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.828+0000 7f52d2a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f52d415dd50 0x7f52d406d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39150/0 (socket says 172.21.15.161:39150) 2024-05-22T21:18:05.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.828+0000 7f52d2a00640 1 -- 172.21.15.161:0/1181523990 learned_addr learned my addr 172.21.15.161:0/1181523990 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:05.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.828+0000 7f52d2000640 1 --2- 172.21.15.161:0/1181523990 >> v2:172.21.15.161:3300/0 conn(0x7f52d415e130 0x7f52d415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.828+0000 7f52d3400640 1 --2- 172.21.15.161:0/1181523990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d415d410 0x7f52d415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.832+0000 7f52d1600640 1 -- 172.21.15.161:0/1181523990 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840218208 0 0) 0x7f52d4071670 con 0x7f52d415dd50 2024-05-22T21:18:05.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.832+0000 7f52d1600640 1 -- 172.21.15.161:0/1181523990 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52b8003530 con 0x7f52d415dd50 2024-05-22T21:18:05.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.832+0000 7f52d2000640 1 -- 172.21.15.161:0/1181523990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d415d410 msgr2=0x7f52d415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:05.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.832+0000 7f52d2000640 1 --2- 172.21.15.161:0/1181523990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d415d410 0x7f52d415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.832+0000 7f52d2000640 1 -- 172.21.15.161:0/1181523990 >> v1:172.21.15.161:6789/0 conn(0x7f52d415dd50 legacy=0x7f52d406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:05.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.832+0000 7f52d2000640 1 -- 172.21.15.161:0/1181523990 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d4070d40 con 0x7f52d415e130 2024-05-22T21:18:05.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.832+0000 7f52d3400640 1 --2- 172.21.15.161:0/1181523990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d415d410 0x7f52d415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:05.838 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.832+0000 7f52d2000640 1 --2- 172.21.15.161:0/1181523990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d415e130 0x7f52d415ced0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f52c00097c0 tx=0x7f52c001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52df25c7a4d75dc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:05.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.832+0000 7f52d1600640 1 -- 172.21.15.161:0/1181523990 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c00ab670 con 0x7f52d415e130 2024-05-22T21:18:05.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.832+0000 7f52d1600640 1 -- 172.21.15.161:0/1181523990 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c00ab7d0 con 0x7f52d415e130 2024-05-22T21:18:05.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.832+0000 7f52d1600640 1 -- 172.21.15.161:0/1181523990 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c00abb60 con 0x7f52d415e130 2024-05-22T21:18:05.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.832+0000 7f52d9600640 1 -- 172.21.15.161:0/1181523990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d415e130 msgr2=0x7f52d415ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:05.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.832+0000 7f52d9600640 1 --2- 172.21.15.161:0/1181523990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d415e130 0x7f52d415ced0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f52c00097c0 tx=0x7f52c001c470 comp rx=0 tx=0).stop 2024-05-22T21:18:05.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.832+0000 7f52d9600640 1 -- 172.21.15.161:0/1181523990 shutdown_connections 2024-05-22T21:18:05.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d9600640 1 --2- 172.21.15.161:0/1181523990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d415e130 0x7f52d415ced0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d9600640 1 --2- 172.21.15.161:0/1181523990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d415d410 0x7f52d415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d9600640 1 -- 172.21.15.161:0/1181523990 >> 172.21.15.161:0/1181523990 conn(0x7f52d40b0bc0 msgr2=0x7f52d40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:05.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d9600640 1 -- 172.21.15.161:0/1181523990 shutdown_connections 2024-05-22T21:18:05.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d9600640 1 -- 172.21.15.161:0/1181523990 wait complete. 2024-05-22T21:18:05.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d9600640 1 Processor -- start 2024-05-22T21:18:05.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d9600640 1 -- start start 2024-05-22T21:18:05.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d9600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d415d410 0x7f52d4151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d9600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d415dd50 0x7f52d40b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d9600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d415e130 0x7f52d4153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d9600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41535b0 con 0x7f52d415dd50 2024-05-22T21:18:05.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d9600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d4145ce0 con 0x7f52d415d410 2024-05-22T21:18:05.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d9600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d4151fd0 con 0x7f52d415e130 2024-05-22T21:18:05.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d415dd50 0x7f52d40b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d415dd50 0x7f52d40b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45236/0 (socket says 172.21.15.161:45236) 2024-05-22T21:18:05.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d2000640 1 -- 172.21.15.161:0/3412476061 learned_addr learned my addr 172.21.15.161:0/3412476061 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:05.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d2000640 1 -- 172.21.15.161:0/3412476061 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d415e130 msgr2=0x7f52d4153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:05.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d2a00640 1 --2- 172.21.15.161:0/3412476061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d415d410 0x7f52d4151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d2000640 1 --2- 172.21.15.161:0/3412476061 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d415e130 0x7f52d4153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d2000640 1 -- 172.21.15.161:0/3412476061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d415d410 msgr2=0x7f52d4151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:05.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d2000640 1 --2- 172.21.15.161:0/3412476061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d415d410 0x7f52d4151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:05.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d2000640 1 -- 172.21.15.161:0/3412476061 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c0009420 con 0x7f52d415dd50 2024-05-22T21:18:05.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d2a00640 1 --2- 172.21.15.161:0/3412476061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d415d410 0x7f52d4151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:05.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d2000640 1 --2- 172.21.15.161:0/3412476061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d415dd50 0x7f52d40b2140 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f52c001a0a0 tx=0x7f52c0019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:05.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52caa00640 1 -- 172.21.15.161:0/3412476061 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c00abe10 con 0x7f52d415dd50 2024-05-22T21:18:05.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52caa00640 1 -- 172.21.15.161:0/3412476061 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c00bf450 con 0x7f52d415dd50 2024-05-22T21:18:05.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52d9600640 1 -- 172.21.15.161:0/3412476061 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d41522b0 con 0x7f52d415dd50 2024-05-22T21:18:05.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.836+0000 7f52caa00640 1 -- 172.21.15.161:0/3412476061 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c00bf7e0 con 0x7f52d415dd50 2024-05-22T21:18:05.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.840+0000 7f52d9600640 1 -- 172.21.15.161:0/3412476061 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d4152870 con 0x7f52d415dd50 2024-05-22T21:18:05.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.840+0000 7f52d9600640 1 -- 172.21.15.161:0/3412476061 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d40706b0 con 0x7f52d415dd50 2024-05-22T21:18:05.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.840+0000 7f52caa00640 1 -- 172.21.15.161:0/3412476061 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f52c00bf940 con 0x7f52d415dd50 2024-05-22T21:18:05.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.840+0000 7f52caa00640 1 --2- 172.21.15.161:0/3412476061 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52b4066690 0x7f52b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:05.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.840+0000 7f52caa00640 1 -- 172.21.15.161:0/3412476061 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52c011ed80 con 0x7f52d415dd50 2024-05-22T21:18:05.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.840+0000 7f52d2a00640 1 --2- 172.21.15.161:0/3412476061 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52b4066690 0x7f52b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:05.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.840+0000 7f52caa00640 1 -- 172.21.15.161:0/3412476061 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f52b406e1c0 con 0x7f52d415dd50 2024-05-22T21:18:05.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.844+0000 7f52d2a00640 1 --2- 172.21.15.161:0/3412476061 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52b4066690 0x7f52b4068b50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f52c40011f0 tx=0x7f52c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:05.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:05.844+0000 7f52caa00640 1 -- 172.21.15.161:0/3412476061 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f52c00ec050 con 0x7f52d415dd50 2024-05-22T21:18:06.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.000+0000 7f52d9600640 1 -- 172.21.15.161:0/3412476061 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52d4067220 con 0x7f52d415dd50 2024-05-22T21:18:06.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.000+0000 7f52caa00640 1 -- 172.21.15.161:0/3412476061 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16170 (secure 0 0 0) 0x7f52c00ef670 con 0x7f52d415dd50 2024-05-22T21:18:06.004 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:06.004 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":90,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:18:03.344552+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:17:12.574579+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-22T21:17:13.013467+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:18:06.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.004+0000 7f52d9600640 1 -- 172.21.15.161:0/3412476061 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52b4066690 msgr2=0x7f52b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.004+0000 7f52d9600640 1 --2- 172.21.15.161:0/3412476061 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52b4066690 0x7f52b4068b50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f52c40011f0 tx=0x7f52c4009040 comp rx=0 tx=0).stop 2024-05-22T21:18:06.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.004+0000 7f52d9600640 1 -- 172.21.15.161:0/3412476061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d415dd50 msgr2=0x7f52d40b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.004+0000 7f52d9600640 1 --2- 172.21.15.161:0/3412476061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d415dd50 0x7f52d40b2140 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f52c001a0a0 tx=0x7f52c0019210 comp rx=0 tx=0).stop 2024-05-22T21:18:06.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.004+0000 7f52d9600640 1 -- 172.21.15.161:0/3412476061 shutdown_connections 2024-05-22T21:18:06.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.004+0000 7f52d9600640 1 --2- 172.21.15.161:0/3412476061 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52b4066690 0x7f52b4068b50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.004+0000 7f52d9600640 1 --2- 172.21.15.161:0/3412476061 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52d415e130 0x7f52d4153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.004+0000 7f52d9600640 1 --2- 172.21.15.161:0/3412476061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d415dd50 0x7f52d40b2140 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.004+0000 7f52d9600640 1 --2- 172.21.15.161:0/3412476061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52d415d410 0x7f52d4151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.004+0000 7f52d9600640 1 -- 172.21.15.161:0/3412476061 >> 172.21.15.161:0/3412476061 conn(0x7f52d40b0bc0 msgr2=0x7f52d40cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:06.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.004+0000 7f52d9600640 1 -- 172.21.15.161:0/3412476061 shutdown_connections 2024-05-22T21:18:06.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.004+0000 7f52d9600640 1 -- 172.21.15.161:0/3412476061 wait complete. 2024-05-22T21:18:06.025 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-22T21:18:06.025 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:18:06.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.156+0000 7fb1caa00640 1 Processor -- start 2024-05-22T21:18:06.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.156+0000 7fb1caa00640 1 -- start start 2024-05-22T21:18:06.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.156+0000 7fb1caa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb1c4070db0 0x7fb1c40b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:06.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.156+0000 7fb1caa00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c4159bd0 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.156+0000 7fb1caa00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c40cbdd0 con 0x7fb1c41597f0 2024-05-22T21:18:06.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.156+0000 7fb1caa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40cc050 con 0x7fb1c4070db0 2024-05-22T21:18:06.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.160+0000 7fb1c8a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb1c40b2ea0 0x7fb1c40cc550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:50638/0 (socket says 172.21.15.161:50638) 2024-05-22T21:18:06.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.160+0000 7fb1c8a00640 1 -- 172.21.15.161:0/937842484 learned_addr learned my addr 172.21.15.161:0/937842484 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:06.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.160+0000 7fb1bb400640 1 --2- 172.21.15.161:0/937842484 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb1c4070db0 0x7fb1c40b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:06.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.160+0000 7fb1c3400640 1 -- 172.21.15.161:0/937842484 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440946478 0 0) 0x7fb1c4159bd0 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.160+0000 7fb1c3400640 1 -- 172.21.15.161:0/937842484 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1a8003450 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.160+0000 7fb1c3400640 1 -- 172.21.15.161:0/937842484 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964163079 0 0) 0x7fb1c40cbdd0 con 0x7fb1c41597f0 2024-05-22T21:18:06.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.160+0000 7fb1c3400640 1 -- 172.21.15.161:0/937842484 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1c4159bd0 con 0x7fb1c41597f0 2024-05-22T21:18:06.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.160+0000 7fb1bb400640 1 -- 172.21.15.161:0/937842484 >> v1:172.21.15.179:6789/0 conn(0x7fb1c41597f0 legacy=0x7fb1c40706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.160+0000 7fb1bb400640 1 -- 172.21.15.161:0/937842484 >> v1:172.21.15.161:6789/0 conn(0x7fb1c40b2ea0 legacy=0x7fb1c40cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.160+0000 7fb1bb400640 1 -- 172.21.15.161:0/937842484 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c40b3720 con 0x7fb1c4070db0 2024-05-22T21:18:06.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1bb400640 1 --2- 172.21.15.161:0/937842484 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb1c4070db0 0x7fb1c40b2960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0097f0 tx=0x7fb1ac01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6046b3b4029ee21 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:06.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1c3400640 1 -- 172.21.15.161:0/937842484 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac0ab660 con 0x7fb1c4070db0 2024-05-22T21:18:06.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1c3400640 1 -- 172.21.15.161:0/937842484 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac0ab7c0 con 0x7fb1c4070db0 2024-05-22T21:18:06.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1c3400640 1 -- 172.21.15.161:0/937842484 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac0abb50 con 0x7fb1c4070db0 2024-05-22T21:18:06.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1caa00640 1 -- 172.21.15.161:0/937842484 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb1c4070db0 msgr2=0x7fb1c40b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1caa00640 1 --2- 172.21.15.161:0/937842484 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb1c4070db0 0x7fb1c40b2960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0097f0 tx=0x7fb1ac01c4e0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1caa00640 1 -- 172.21.15.161:0/937842484 shutdown_connections 2024-05-22T21:18:06.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1caa00640 1 --2- 172.21.15.161:0/937842484 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb1c4070db0 0x7fb1c40b2960 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1caa00640 1 -- 172.21.15.161:0/937842484 >> 172.21.15.161:0/937842484 conn(0x7fb1c40b0bc0 msgr2=0x7fb1c40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:06.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1caa00640 1 -- 172.21.15.161:0/937842484 shutdown_connections 2024-05-22T21:18:06.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1caa00640 1 -- 172.21.15.161:0/937842484 wait complete. 2024-05-22T21:18:06.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1caa00640 1 Processor -- start 2024-05-22T21:18:06.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1caa00640 1 -- start start 2024-05-22T21:18:06.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1caa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c4070db0 0x7fb1c4138090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:06.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1caa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1c40b2ea0 0x7fb1c41f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:06.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1caa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb1c41597f0 0x7fb1c4135650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:06.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1caa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c415b500 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1caa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c413ba30 con 0x7fb1c4070db0 2024-05-22T21:18:06.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1caa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c414b880 con 0x7fb1c41597f0 2024-05-22T21:18:06.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1bb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1c40b2ea0 0x7fb1c41f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:06.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1bb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1c40b2ea0 0x7fb1c41f5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41558/0 (socket says 172.21.15.161:41558) 2024-05-22T21:18:06.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.164+0000 7fb1bb400640 1 -- 172.21.15.161:0/3293255043 learned_addr learned my addr 172.21.15.161:0/3293255043 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:06.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1c8a00640 1 --2- 172.21.15.161:0/3293255043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb1c41597f0 0x7fb1c4135650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:06.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1c3e00640 1 --2- 172.21.15.161:0/3293255043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c4070db0 0x7fb1c4138090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:06.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1bb400640 1 -- 172.21.15.161:0/3293255043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb1c41597f0 msgr2=0x7fb1c4135650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1bb400640 1 --2- 172.21.15.161:0/3293255043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb1c41597f0 0x7fb1c4135650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1bb400640 1 -- 172.21.15.161:0/3293255043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c4070db0 msgr2=0x7fb1c4138090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1bb400640 1 --2- 172.21.15.161:0/3293255043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c4070db0 0x7fb1c4138090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1bb400640 1 -- 172.21.15.161:0/3293255043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac009420 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1c3e00640 1 --2- 172.21.15.161:0/3293255043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c4070db0 0x7fb1c4138090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:06.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1bb400640 1 --2- 172.21.15.161:0/3293255043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1c40b2ea0 0x7fb1c41f5560 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0190f0 tx=0x7fb1ac01ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:06.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1c0c00640 1 -- 172.21.15.161:0/3293255043 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac0bf610 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1c0c00640 1 -- 172.21.15.161:0/3293255043 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac0bf790 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1c0c00640 1 -- 172.21.15.161:0/3293255043 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac0bfb40 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1caa00640 1 -- 172.21.15.161:0/3293255043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c415b730 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1caa00640 1 -- 172.21.15.161:0/3293255043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c415b9d0 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1caa00640 1 -- 172.21.15.161:0/3293255043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c4067220 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1c0c00640 1 -- 172.21.15.161:0/3293255043 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb1ac0a9070 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1c0c00640 1 --2- 172.21.15.161:0/3293255043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb1a0066690 0x7fb1a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:06.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1c0c00640 1 -- 172.21.15.161:0/3293255043 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1ac11e980 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1c3e00640 1 --2- 172.21.15.161:0/3293255043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb1a0066690 0x7fb1a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:06.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.168+0000 7fb1c0c00640 1 -- 172.21.15.161:0/3293255043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7fb1a006e1c0 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.172+0000 7fb1c3e00640 1 --2- 172.21.15.161:0/3293255043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb1a0066690 0x7fb1a0068b50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40011f0 tx=0x7fb1b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:06.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.172+0000 7fb1c0c00640 1 -- 172.21.15.161:0/3293255043 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb1ac0ebf50 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.356+0000 7fb1c0c00640 1 -- 172.21.15.161:0/3293255043 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(91..91 src has 1..91) v4 ==== 1468+0+0 (secure 0 0 0) 0x7fb1ac0efc10 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.356+0000 7fb1c0c00640 1 -- 172.21.15.161:0/3293255043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=92}) v3 -- 0x7fb1a006f3d0 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.364+0000 7fb1caa00640 1 -- 172.21.15.161:0/3293255043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1c41385d0 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.368+0000 7fb1c0c00640 1 -- 172.21.15.161:0/3293255043 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4745 (secure 0 0 0) 0x7fb1ac0ef570 con 0x7fb1c40b2ea0 2024-05-22T21:18:06.369 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:06.369 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5567,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196093187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196093187},{"type":"v1","addr":"172.21.15.161:6836","nonce":196093187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6136,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4029369301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4029369301},{"type":"v1","addr":"172.21.15.161:6837","nonce":4029369301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6139,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3212560487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3212560487},{"type":"v1","addr":"172.21.15.179:6834","nonce":3212560487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:17:14.012254+0000","modified":"2024-05-22T21:17:17.029283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6306":{"gid":6306,"name":"b","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/3065973138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3065973138},{"type":"v1","addr":"172.21.15.179:6835","nonce":3065973138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-22T21:18:06.369 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 39 2024-05-22T21:18:06.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.368+0000 7fb1caa00640 1 -- 172.21.15.161:0/3293255043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb1a0066690 msgr2=0x7fb1a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.368+0000 7fb1caa00640 1 --2- 172.21.15.161:0/3293255043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb1a0066690 0x7fb1a0068b50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40011f0 tx=0x7fb1b4009040 comp rx=0 tx=0).stop 2024-05-22T21:18:06.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.368+0000 7fb1caa00640 1 -- 172.21.15.161:0/3293255043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1c40b2ea0 msgr2=0x7fb1c41f5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.368+0000 7fb1caa00640 1 --2- 172.21.15.161:0/3293255043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1c40b2ea0 0x7fb1c41f5560 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0190f0 tx=0x7fb1ac01ad20 comp rx=0 tx=0).stop 2024-05-22T21:18:06.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.368+0000 7fb1caa00640 1 -- 172.21.15.161:0/3293255043 shutdown_connections 2024-05-22T21:18:06.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.368+0000 7fb1caa00640 1 --2- 172.21.15.161:0/3293255043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb1a0066690 0x7fb1a0068b50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.368+0000 7fb1caa00640 1 --2- 172.21.15.161:0/3293255043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb1c41597f0 0x7fb1c4135650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.368+0000 7fb1caa00640 1 --2- 172.21.15.161:0/3293255043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1c40b2ea0 0x7fb1c41f5560 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.368+0000 7fb1caa00640 1 --2- 172.21.15.161:0/3293255043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c4070db0 0x7fb1c4138090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.368+0000 7fb1caa00640 1 -- 172.21.15.161:0/3293255043 >> 172.21.15.161:0/3293255043 conn(0x7fb1c40b0bc0 msgr2=0x7fb1c41569a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:06.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.368+0000 7fb1caa00640 1 -- 172.21.15.161:0/3293255043 shutdown_connections 2024-05-22T21:18:06.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.368+0000 7fb1caa00640 1 -- 172.21.15.161:0/3293255043 wait complete. 2024-05-22T21:18:06.390 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:18:06.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.524+0000 7fc7e2200640 1 Processor -- start 2024-05-22T21:18:06.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.524+0000 7fc7e2200640 1 -- start start 2024-05-22T21:18:06.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.528+0000 7fc7e2200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc7dc15dd50 0x7fc7dc06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:06.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.528+0000 7fc7e2200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7dc071670 con 0x7fc7dc15d5e0 2024-05-22T21:18:06.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.528+0000 7fc7e2200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7dc06de40 con 0x7fc7dc15e130 2024-05-22T21:18:06.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.528+0000 7fc7e2200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7dc070a60 con 0x7fc7dc15dd50 2024-05-22T21:18:06.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.528+0000 7fc7dbe00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc7dc15d5e0 0x7fc7dc15b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:50646/0 (socket says 172.21.15.161:50646) 2024-05-22T21:18:06.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.528+0000 7fc7dbe00640 1 -- 172.21.15.161:0/1196182514 learned_addr learned my addr 172.21.15.161:0/1196182514 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:06.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.528+0000 7fc7db400640 1 --2- 172.21.15.161:0/1196182514 >> v2:172.21.15.161:3300/0 conn(0x7fc7dc15dd50 0x7fc7dc06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:06.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.528+0000 7fc7db400640 1 -- 172.21.15.161:0/1196182514 >> v1:172.21.15.179:6789/0 conn(0x7fc7dc15e130 legacy=0x7fc7dc15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.528+0000 7fc7db400640 1 -- 172.21.15.161:0/1196182514 >> v1:172.21.15.161:6789/0 conn(0x7fc7dc15d5e0 legacy=0x7fc7dc15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.528+0000 7fc7db400640 1 -- 172.21.15.161:0/1196182514 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7dc070d40 con 0x7fc7dc15dd50 2024-05-22T21:18:06.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.528+0000 7fc7daa00640 1 -- 172.21.15.161:0/1196182514 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634102815 0 0) 0x7fc7dc071670 con 0x7fc7dc15d5e0 2024-05-22T21:18:06.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.528+0000 7fc7db400640 1 --2- 172.21.15.161:0/1196182514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7dc15dd50 0x7fc7dc06d900 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0097c0 tx=0x7fc7cc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9478f02ece14f3b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:06.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7daa00640 1 -- 172.21.15.161:0/1196182514 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0ab670 con 0x7fc7dc15dd50 2024-05-22T21:18:06.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7daa00640 1 -- 172.21.15.161:0/1196182514 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc0ab7d0 con 0x7fc7dc15dd50 2024-05-22T21:18:06.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7daa00640 1 -- 172.21.15.161:0/1196182514 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0abb60 con 0x7fc7dc15dd50 2024-05-22T21:18:06.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7e2200640 1 -- 172.21.15.161:0/1196182514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7dc15dd50 msgr2=0x7fc7dc06d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7e2200640 1 --2- 172.21.15.161:0/1196182514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7dc15dd50 0x7fc7dc06d900 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0097c0 tx=0x7fc7cc01c470 comp rx=0 tx=0).stop 2024-05-22T21:18:06.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7e2200640 1 -- 172.21.15.161:0/1196182514 shutdown_connections 2024-05-22T21:18:06.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7e2200640 1 --2- 172.21.15.161:0/1196182514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7dc15dd50 0x7fc7dc06d900 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7e2200640 1 -- 172.21.15.161:0/1196182514 >> 172.21.15.161:0/1196182514 conn(0x7fc7dc0b0bc0 msgr2=0x7fc7dc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:06.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7e2200640 1 -- 172.21.15.161:0/1196182514 shutdown_connections 2024-05-22T21:18:06.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7e2200640 1 -- 172.21.15.161:0/1196182514 wait complete. 2024-05-22T21:18:06.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7e2200640 1 Processor -- start 2024-05-22T21:18:06.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7e2200640 1 -- start start 2024-05-22T21:18:06.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7e2200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7dc15d5e0 0x7fc7dc13a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:06.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7e2200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7dc15dd50 0x7fc7dc0b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:06.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7e2200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc7dc15e130 0x7fc7dc14f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:06.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7e2200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc0b2da0 con 0x7fc7dc15dd50 2024-05-22T21:18:06.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7e2200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc0acdb0 con 0x7fc7dc15d5e0 2024-05-22T21:18:06.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7e2200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc0ad090 con 0x7fc7dc15e130 2024-05-22T21:18:06.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7d2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7dc15dd50 0x7fc7dc0b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:06.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7d2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7dc15dd50 0x7fc7dc0b1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41584/0 (socket says 172.21.15.161:41584) 2024-05-22T21:18:06.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7d2a00640 1 -- 172.21.15.161:0/1204641727 learned_addr learned my addr 172.21.15.161:0/1204641727 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:06.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7db400640 1 --2- 172.21.15.161:0/1204641727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7dc15d5e0 0x7fc7dc13a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:06.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.532+0000 7fc7dbe00640 1 --2- 172.21.15.161:0/1204641727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc7dc15e130 0x7fc7dc14f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:06.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7d2a00640 1 -- 172.21.15.161:0/1204641727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc7dc15e130 msgr2=0x7fc7dc14f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7d2a00640 1 --2- 172.21.15.161:0/1204641727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc7dc15e130 0x7fc7dc14f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7d2a00640 1 -- 172.21.15.161:0/1204641727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7dc15d5e0 msgr2=0x7fc7dc13a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7d2a00640 1 --2- 172.21.15.161:0/1204641727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7dc15d5e0 0x7fc7dc13a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7d2a00640 1 -- 172.21.15.161:0/1204641727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc009420 con 0x7fc7dc15dd50 2024-05-22T21:18:06.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7db400640 1 --2- 172.21.15.161:0/1204641727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7dc15d5e0 0x7fc7dc13a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:06.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7d2a00640 1 --2- 172.21.15.161:0/1204641727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7dc15dd50 0x7fc7dc0b1240 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400c3c0 tx=0x7fc7c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:06.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7d3e00640 1 -- 172.21.15.161:0/1204641727 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40052d0 con 0x7fc7dc15dd50 2024-05-22T21:18:06.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7d3e00640 1 -- 172.21.15.161:0/1204641727 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c4005430 con 0x7fc7dc15dd50 2024-05-22T21:18:06.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7d3e00640 1 -- 172.21.15.161:0/1204641727 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4003840 con 0x7fc7dc15dd50 2024-05-22T21:18:06.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7e2200640 1 -- 172.21.15.161:0/1204641727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7dc0ad370 con 0x7fc7dc15dd50 2024-05-22T21:18:06.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7e2200640 1 -- 172.21.15.161:0/1204641727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7dc0b1780 con 0x7fc7dc15dd50 2024-05-22T21:18:06.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7e2200640 1 -- 172.21.15.161:0/1204641727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7dc067940 con 0x7fc7dc15dd50 2024-05-22T21:18:06.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7d3e00640 1 -- 172.21.15.161:0/1204641727 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc7c40039a0 con 0x7fc7dc15dd50 2024-05-22T21:18:06.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7d3e00640 1 --2- 172.21.15.161:0/1204641727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7b8066690 0x7fc7b8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:06.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7d3e00640 1 -- 172.21.15.161:0/1204641727 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7c4083770 con 0x7fc7dc15dd50 2024-05-22T21:18:06.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.536+0000 7fc7db400640 1 --2- 172.21.15.161:0/1204641727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7b8066690 0x7fc7b8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:06.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.540+0000 7fc7db400640 1 --2- 172.21.15.161:0/1204641727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7b8066690 0x7fc7b8068b50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0af000 tx=0x7fc7cc0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:06.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.540+0000 7fc7d3e00640 1 -- 172.21.15.161:0/1204641727 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc7c4050a40 con 0x7fc7dc15dd50 2024-05-22T21:18:06.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.732+0000 7fc7e2200640 1 -- 172.21.15.161:0/1204641727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7dc067220 con 0x7fc7dc15dd50 2024-05-22T21:18:06.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.732+0000 7fc7d3e00640 1 -- 172.21.15.161:0/1204641727 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4745 (secure 0 0 0) 0x7fc7c4054060 con 0x7fc7dc15dd50 2024-05-22T21:18:06.736 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:06.736 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5567,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196093187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196093187},{"type":"v1","addr":"172.21.15.161:6836","nonce":196093187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6136,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4029369301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4029369301},{"type":"v1","addr":"172.21.15.161:6837","nonce":4029369301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6139,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3212560487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3212560487},{"type":"v1","addr":"172.21.15.179:6834","nonce":3212560487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:17:14.012254+0000","modified":"2024-05-22T21:17:17.029283+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6306":{"gid":6306,"name":"b","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/3065973138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3065973138},{"type":"v1","addr":"172.21.15.179:6835","nonce":3065973138}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-22T21:18:06.736 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 39 2024-05-22T21:18:06.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.736+0000 7fc7e2200640 1 -- 172.21.15.161:0/1204641727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7b8066690 msgr2=0x7fc7b8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.736+0000 7fc7e2200640 1 --2- 172.21.15.161:0/1204641727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7b8066690 0x7fc7b8068b50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0af000 tx=0x7fc7cc0a7000 comp rx=0 tx=0).stop 2024-05-22T21:18:06.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.736+0000 7fc7e2200640 1 -- 172.21.15.161:0/1204641727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7dc15dd50 msgr2=0x7fc7dc0b1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.736+0000 7fc7e2200640 1 --2- 172.21.15.161:0/1204641727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7dc15dd50 0x7fc7dc0b1240 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400c3c0 tx=0x7fc7c400c880 comp rx=0 tx=0).stop 2024-05-22T21:18:06.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.736+0000 7fc7e2200640 1 -- 172.21.15.161:0/1204641727 shutdown_connections 2024-05-22T21:18:06.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.736+0000 7fc7e2200640 1 --2- 172.21.15.161:0/1204641727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7b8066690 0x7fc7b8068b50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.736+0000 7fc7e2200640 1 --2- 172.21.15.161:0/1204641727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc7dc15e130 0x7fc7dc14f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.736+0000 7fc7e2200640 1 --2- 172.21.15.161:0/1204641727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7dc15dd50 0x7fc7dc0b1240 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.736+0000 7fc7e2200640 1 --2- 172.21.15.161:0/1204641727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7dc15d5e0 0x7fc7dc13a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.736+0000 7fc7e2200640 1 -- 172.21.15.161:0/1204641727 >> 172.21.15.161:0/1204641727 conn(0x7fc7dc0b0bc0 msgr2=0x7fc7dc0b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:06.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.736+0000 7fc7e2200640 1 -- 172.21.15.161:0/1204641727 shutdown_connections 2024-05-22T21:18:06.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.736+0000 7fc7e2200640 1 -- 172.21.15.161:0/1204641727 wait complete. 2024-05-22T21:18:06.758 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:18:06.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f4867000640 1 Processor -- start 2024-05-22T21:18:06.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f4867000640 1 -- start start 2024-05-22T21:18:06.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f4867000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f486006f430 0x7f486006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:06.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f4867000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4860071350 0x7f486006db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:06.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f4867000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48600ad9f0 con 0x7f4860159e80 2024-05-22T21:18:06.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f4867000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48600ace20 con 0x7f486006f430 2024-05-22T21:18:06.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f4867000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48600ad0c0 con 0x7f4860071350 2024-05-22T21:18:06.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f4865200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f486006f430 0x7f486006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:06.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f4865c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f4860159e80 0x7f4860070c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:50658/0 (socket says 172.21.15.161:50658) 2024-05-22T21:18:06.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f4865c00640 1 -- 172.21.15.161:0/2928040483 learned_addr learned my addr 172.21.15.161:0/2928040483 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:06.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f4866600640 1 --2- 172.21.15.161:0/2928040483 >> v2:172.21.15.179:3300/0 conn(0x7f4860071350 0x7f486006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:06.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f4865200640 1 -- 172.21.15.161:0/2928040483 >> v2:172.21.15.179:3300/0 conn(0x7f4860071350 msgr2=0x7f486006db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:06.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f4865200640 1 --2- 172.21.15.161:0/2928040483 >> v2:172.21.15.179:3300/0 conn(0x7f4860071350 0x7f486006db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f4865200640 1 -- 172.21.15.161:0/2928040483 >> v1:172.21.15.161:6789/0 conn(0x7f4860159e80 legacy=0x7f4860070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f4865200640 1 -- 172.21.15.161:0/2928040483 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48600ad3a0 con 0x7f486006f430 2024-05-22T21:18:06.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f4866600640 1 --2- 172.21.15.161:0/2928040483 >> v2:172.21.15.179:3300/0 conn(0x7f4860071350 0x7f486006db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:06.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.892+0000 7f485fe00640 1 -- 172.21.15.161:0/2928040483 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951885811 0 0) 0x7f48600ad9f0 con 0x7f4860159e80 2024-05-22T21:18:06.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.896+0000 7f4865200640 1 --2- 172.21.15.161:0/2928040483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f486006f430 0x7f486006d5f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f48600cf6d0 tx=0x7f484c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43194d975831965 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:06.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.896+0000 7f485fe00640 1 -- 172.21.15.161:0/2928040483 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c0ab670 con 0x7f486006f430 2024-05-22T21:18:06.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.896+0000 7f485fe00640 1 -- 172.21.15.161:0/2928040483 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484c0ab7d0 con 0x7f486006f430 2024-05-22T21:18:06.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.896+0000 7f485fe00640 1 -- 172.21.15.161:0/2928040483 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c0abb60 con 0x7f486006f430 2024-05-22T21:18:06.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.896+0000 7f4867000640 1 -- 172.21.15.161:0/2928040483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f486006f430 msgr2=0x7f486006d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.896+0000 7f4867000640 1 --2- 172.21.15.161:0/2928040483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f486006f430 0x7f486006d5f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f48600cf6d0 tx=0x7f484c01c450 comp rx=0 tx=0).stop 2024-05-22T21:18:06.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.896+0000 7f4867000640 1 -- 172.21.15.161:0/2928040483 shutdown_connections 2024-05-22T21:18:06.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.896+0000 7f4867000640 1 --2- 172.21.15.161:0/2928040483 >> v2:172.21.15.179:3300/0 conn(0x7f4860071350 0x7f486006db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.896+0000 7f4867000640 1 --2- 172.21.15.161:0/2928040483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f486006f430 0x7f486006d5f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.896+0000 7f4867000640 1 -- 172.21.15.161:0/2928040483 >> 172.21.15.161:0/2928040483 conn(0x7f486006ba00 msgr2=0x7f48600b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:06.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.896+0000 7f4867000640 1 -- 172.21.15.161:0/2928040483 shutdown_connections 2024-05-22T21:18:06.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.896+0000 7f4867000640 1 -- 172.21.15.161:0/2928040483 wait complete. 2024-05-22T21:18:06.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.896+0000 7f4867000640 1 Processor -- start 2024-05-22T21:18:06.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.896+0000 7f4867000640 1 -- start start 2024-05-22T21:18:06.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4867000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4860071350 0x7f4860140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:06.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4867000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4860159e80 0x7f48600ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:06.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4867000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4860150aa0 0x7f486013d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:06.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4867000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48600cde30 con 0x7f4860071350 2024-05-22T21:18:06.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4867000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48601413d0 con 0x7f4860150aa0 2024-05-22T21:18:06.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4867000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486015d180 con 0x7f4860159e80 2024-05-22T21:18:06.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4865c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4860071350 0x7f4860140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:06.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4865c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4860071350 0x7f4860140e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41594/0 (socket says 172.21.15.161:41594) 2024-05-22T21:18:06.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4865c00640 1 -- 172.21.15.161:0/4249529125 learned_addr learned my addr 172.21.15.161:0/4249529125 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:06.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4865200640 1 --2- 172.21.15.161:0/4249529125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4860159e80 0x7f48600ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:06.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4866600640 1 --2- 172.21.15.161:0/4249529125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4860150aa0 0x7f486013d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:06.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4865c00640 1 -- 172.21.15.161:0/4249529125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4860159e80 msgr2=0x7f48600ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4865c00640 1 --2- 172.21.15.161:0/4249529125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4860159e80 0x7f48600ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4865c00640 1 -- 172.21.15.161:0/4249529125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4860150aa0 msgr2=0x7f486013d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:06.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4865c00640 1 --2- 172.21.15.161:0/4249529125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4860150aa0 0x7f486013d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:06.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4865c00640 1 -- 172.21.15.161:0/4249529125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c009420 con 0x7f4860071350 2024-05-22T21:18:06.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4865c00640 1 --2- 172.21.15.161:0/4249529125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4860071350 0x7f4860140e90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f485400c4b0 tx=0x7f485400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:06.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f485d600640 1 -- 172.21.15.161:0/4249529125 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48540074e0 con 0x7f4860071350 2024-05-22T21:18:06.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f485d600640 1 -- 172.21.15.161:0/4249529125 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4854007660 con 0x7f4860071350 2024-05-22T21:18:06.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f485d600640 1 -- 172.21.15.161:0/4249529125 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854004a30 con 0x7f4860071350 2024-05-22T21:18:06.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4867000640 1 -- 172.21.15.161:0/4249529125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f486015cbc0 con 0x7f4860071350 2024-05-22T21:18:06.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4867000640 1 -- 172.21.15.161:0/4249529125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48600cdac0 con 0x7f4860071350 2024-05-22T21:18:06.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f4867000640 1 -- 172.21.15.161:0/4249529125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48600cd920 con 0x7f4860071350 2024-05-22T21:18:06.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.900+0000 7f485d600640 1 -- 172.21.15.161:0/4249529125 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4854004b90 con 0x7f4860071350 2024-05-22T21:18:06.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.904+0000 7f485d600640 1 --2- 172.21.15.161:0/4249529125 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4840066690 0x7f4840068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:06.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.904+0000 7f485d600640 1 -- 172.21.15.161:0/4249529125 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4854083220 con 0x7f4860071350 2024-05-22T21:18:06.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.904+0000 7f4865200640 1 --2- 172.21.15.161:0/4249529125 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4840066690 0x7f4840068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:06.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.904+0000 7f4865200640 1 --2- 172.21.15.161:0/4249529125 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4840066690 0x7f4840068b50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f484c0af000 tx=0x7f484c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:06.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:06.908+0000 7f485d600640 1 -- 172.21.15.161:0/4249529125 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f48600cd920 con 0x7f4860071350 2024-05-22T21:18:07.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.060+0000 7f4867000640 1 -- 172.21.15.161:0/4249529125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48600cd920 con 0x7f4860071350 2024-05-22T21:18:07.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.064+0000 7f485d600640 1 -- 172.21.15.161:0/4249529125 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f48600cd920 con 0x7f4860071350 2024-05-22T21:18:07.065 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:07.065 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":91,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:18:06.357415+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:17:12.574579+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-22T21:17:13.013467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:18:07.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.064+0000 7f4867000640 1 -- 172.21.15.161:0/4249529125 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4840066690 msgr2=0x7f4840068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:07.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.064+0000 7f4867000640 1 --2- 172.21.15.161:0/4249529125 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4840066690 0x7f4840068b50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f484c0af000 tx=0x7f484c0a7000 comp rx=0 tx=0).stop 2024-05-22T21:18:07.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.064+0000 7f4867000640 1 -- 172.21.15.161:0/4249529125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4860071350 msgr2=0x7f4860140e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:07.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.064+0000 7f4867000640 1 --2- 172.21.15.161:0/4249529125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4860071350 0x7f4860140e90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f485400c4b0 tx=0x7f485400c970 comp rx=0 tx=0).stop 2024-05-22T21:18:07.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.064+0000 7f4867000640 1 -- 172.21.15.161:0/4249529125 shutdown_connections 2024-05-22T21:18:07.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.064+0000 7f4867000640 1 --2- 172.21.15.161:0/4249529125 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4840066690 0x7f4840068b50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.064+0000 7f4867000640 1 --2- 172.21.15.161:0/4249529125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4860150aa0 0x7f486013d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.064+0000 7f4867000640 1 --2- 172.21.15.161:0/4249529125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4860159e80 0x7f48600ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.064+0000 7f4867000640 1 --2- 172.21.15.161:0/4249529125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4860071350 0x7f4860140e90 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.064+0000 7f4867000640 1 -- 172.21.15.161:0/4249529125 >> 172.21.15.161:0/4249529125 conn(0x7f486006ba00 msgr2=0x7f48600afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:07.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.064+0000 7f4867000640 1 -- 172.21.15.161:0/4249529125 shutdown_connections 2024-05-22T21:18:07.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.064+0000 7f4867000640 1 -- 172.21.15.161:0/4249529125 wait complete. 2024-05-22T21:18:07.086 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-22T21:18:07.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.220+0000 7f7972e00640 1 Processor -- start 2024-05-22T21:18:07.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.220+0000 7f7972e00640 1 -- start start 2024-05-22T21:18:07.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.220+0000 7f7972e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f796c143c40 0x7f796c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:07.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.220+0000 7f7972e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f796c1444a0 0x7f796c06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:07.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.220+0000 7f7972e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796c15efb0 con 0x7f796c144900 2024-05-22T21:18:07.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.220+0000 7f7972e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f796c15f290 con 0x7f796c1444a0 2024-05-22T21:18:07.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.220+0000 7f7972e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796c15f570 con 0x7f796c143c40 2024-05-22T21:18:07.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.220+0000 7f7963400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f796c1444a0 0x7f796c06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:07.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.220+0000 7f7970e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f796c144900 0x7f796c0b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:50664/0 (socket says 172.21.15.161:50664) 2024-05-22T21:18:07.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.220+0000 7f7970e00640 1 -- 172.21.15.161:0/545560651 learned_addr learned my addr 172.21.15.161:0/545560651 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:07.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.220+0000 7f796be00640 1 --2- 172.21.15.161:0/545560651 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f796c143c40 0x7f796c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:07.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.220+0000 7f7963400640 1 -- 172.21.15.161:0/545560651 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f796c143c40 msgr2=0x7f796c15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:07.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.220+0000 7f7963400640 1 --2- 172.21.15.161:0/545560651 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f796c143c40 0x7f796c15b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.220+0000 7f7963400640 1 -- 172.21.15.161:0/545560651 >> v1:172.21.15.161:6789/0 conn(0x7f796c144900 legacy=0x7f796c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:07.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.220+0000 7f7963400640 1 -- 172.21.15.161:0/545560651 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c15f850 con 0x7f796c1444a0 2024-05-22T21:18:07.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.220+0000 7f796b400640 1 -- 172.21.15.161:0/545560651 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220611501 0 0) 0x7f796c15efb0 con 0x7f796c144900 2024-05-22T21:18:07.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.224+0000 7f7963400640 1 --2- 172.21.15.161:0/545560651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f796c1444a0 0x7f796c06c2a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f79540097c0 tx=0x7f795401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf92b30df20abdbe server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:07.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.224+0000 7f796b400640 1 -- 172.21.15.161:0/545560651 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79540ac410 con 0x7f796c1444a0 2024-05-22T21:18:07.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.224+0000 7f796b400640 1 -- 172.21.15.161:0/545560651 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79540ac570 con 0x7f796c1444a0 2024-05-22T21:18:07.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.224+0000 7f796b400640 1 -- 172.21.15.161:0/545560651 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79540ac900 con 0x7f796c1444a0 2024-05-22T21:18:07.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.224+0000 7f7972e00640 1 -- 172.21.15.161:0/545560651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f796c1444a0 msgr2=0x7f796c06c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:07.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.224+0000 7f7972e00640 1 --2- 172.21.15.161:0/545560651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f796c1444a0 0x7f796c06c2a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f79540097c0 tx=0x7f795401c4a0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.224+0000 7f7972e00640 1 -- 172.21.15.161:0/545560651 shutdown_connections 2024-05-22T21:18:07.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.224+0000 7f7972e00640 1 --2- 172.21.15.161:0/545560651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f796c1444a0 0x7f796c06c2a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.224+0000 7f7972e00640 1 --2- 172.21.15.161:0/545560651 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f796c143c40 0x7f796c15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.224+0000 7f7972e00640 1 -- 172.21.15.161:0/545560651 >> 172.21.15.161:0/545560651 conn(0x7f796c06a2d0 msgr2=0x7f796c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:07.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.224+0000 7f7972e00640 1 -- 172.21.15.161:0/545560651 shutdown_connections 2024-05-22T21:18:07.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.224+0000 7f7972e00640 1 -- 172.21.15.161:0/545560651 wait complete. 2024-05-22T21:18:07.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7972e00640 1 Processor -- start 2024-05-22T21:18:07.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7972e00640 1 -- start start 2024-05-22T21:18:07.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7972e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796c143c40 0x7f796c136760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:07.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7972e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f796c1444a0 0x7f796c151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:07.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7972e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f796c144900 0x7f796c15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:07.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7972e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c136ca0 con 0x7f796c144900 2024-05-22T21:18:07.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7972e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c1599e0 con 0x7f796c143c40 2024-05-22T21:18:07.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7972e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796c15a560 con 0x7f796c1444a0 2024-05-22T21:18:07.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7970e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f796c144900 0x7f796c15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:07.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7970e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f796c144900 0x7f796c15a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41608/0 (socket says 172.21.15.161:41608) 2024-05-22T21:18:07.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7970e00640 1 -- 172.21.15.161:0/3549614150 learned_addr learned my addr 172.21.15.161:0/3549614150 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:07.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f796be00640 1 --2- 172.21.15.161:0/3549614150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796c143c40 0x7f796c136760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:07.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7963400640 1 --2- 172.21.15.161:0/3549614150 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f796c1444a0 0x7f796c151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:07.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7970e00640 1 -- 172.21.15.161:0/3549614150 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f796c1444a0 msgr2=0x7f796c151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:07.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7970e00640 1 --2- 172.21.15.161:0/3549614150 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f796c1444a0 0x7f796c151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7970e00640 1 -- 172.21.15.161:0/3549614150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796c143c40 msgr2=0x7f796c136760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:07.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7970e00640 1 --2- 172.21.15.161:0/3549614150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796c143c40 0x7f796c136760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7970e00640 1 -- 172.21.15.161:0/3549614150 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7954009420 con 0x7f796c144900 2024-05-22T21:18:07.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7970e00640 1 --2- 172.21.15.161:0/3549614150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f796c144900 0x7f796c15a9f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f796400c7b0 tx=0x7f796400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:07.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7968c00640 1 -- 172.21.15.161:0/3549614150 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796400a640 con 0x7f796c144900 2024-05-22T21:18:07.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7968c00640 1 -- 172.21.15.161:0/3549614150 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796400a7c0 con 0x7f796c144900 2024-05-22T21:18:07.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7968c00640 1 -- 172.21.15.161:0/3549614150 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796400ab70 con 0x7f796c144900 2024-05-22T21:18:07.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7972e00640 1 -- 172.21.15.161:0/3549614150 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796c159c30 con 0x7f796c144900 2024-05-22T21:18:07.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.228+0000 7f7972e00640 1 -- 172.21.15.161:0/3549614150 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796c15a1f0 con 0x7f796c144900 2024-05-22T21:18:07.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.232+0000 7f7972e00640 1 -- 172.21.15.161:0/3549614150 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796c15a6f0 con 0x7f796c144900 2024-05-22T21:18:07.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.232+0000 7f7968c00640 1 -- 172.21.15.161:0/3549614150 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f796400acd0 con 0x7f796c144900 2024-05-22T21:18:07.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.232+0000 7f7968c00640 1 --2- 172.21.15.161:0/3549614150 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7948066690 0x7f7948068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:07.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.232+0000 7f7968c00640 1 -- 172.21.15.161:0/3549614150 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79640835d0 con 0x7f796c144900 2024-05-22T21:18:07.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.232+0000 7f796be00640 1 --2- 172.21.15.161:0/3549614150 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7948066690 0x7f7948068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:07.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.232+0000 7f796be00640 1 --2- 172.21.15.161:0/3549614150 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7948066690 0x7f7948068b50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f795c009af0 tx=0x7f795c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:07.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.236+0000 7f7968c00640 1 -- 172.21.15.161:0/3549614150 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7964050930 con 0x7f796c144900 2024-05-22T21:18:07.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.428+0000 7f7972e00640 1 -- 172.21.15.161:0/3549614150 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f796c067220 con 0x7f796c144900 2024-05-22T21:18:07.429 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:18:07.428+0000 7f7d32800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-22T21:18:07.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.436+0000 7f7968c00640 1 -- 172.21.15.161:0/3549614150 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7964053f50 con 0x7f796c144900 2024-05-22T21:18:07.439 INFO:teuthology.orchestra.run.smithi161.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-22T21:18:07.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.440+0000 7f7972e00640 1 -- 172.21.15.161:0/3549614150 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7948066690 msgr2=0x7f7948068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:07.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.440+0000 7f7972e00640 1 --2- 172.21.15.161:0/3549614150 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7948066690 0x7f7948068b50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f795c009af0 tx=0x7f795c008040 comp rx=0 tx=0).stop 2024-05-22T21:18:07.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.440+0000 7f7972e00640 1 -- 172.21.15.161:0/3549614150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f796c144900 msgr2=0x7f796c15a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:07.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.440+0000 7f7972e00640 1 --2- 172.21.15.161:0/3549614150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f796c144900 0x7f796c15a9f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f796400c7b0 tx=0x7f796400cc70 comp rx=0 tx=0).stop 2024-05-22T21:18:07.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.440+0000 7f7972e00640 1 -- 172.21.15.161:0/3549614150 shutdown_connections 2024-05-22T21:18:07.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.440+0000 7f7972e00640 1 --2- 172.21.15.161:0/3549614150 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7948066690 0x7f7948068b50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.440+0000 7f7972e00640 1 --2- 172.21.15.161:0/3549614150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f796c144900 0x7f796c15a9f0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.440+0000 7f7972e00640 1 --2- 172.21.15.161:0/3549614150 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f796c1444a0 0x7f796c151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.440+0000 7f7972e00640 1 --2- 172.21.15.161:0/3549614150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f796c143c40 0x7f796c136760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.440+0000 7f7972e00640 1 -- 172.21.15.161:0/3549614150 >> 172.21.15.161:0/3549614150 conn(0x7f796c06a2d0 msgr2=0x7f796c157ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:07.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.440+0000 7f7972e00640 1 -- 172.21.15.161:0/3549614150 shutdown_connections 2024-05-22T21:18:07.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.440+0000 7f7972e00640 1 -- 172.21.15.161:0/3549614150 wait complete. 2024-05-22T21:18:07.460 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-22T21:18:07.503 INFO:tasks.ceph.mds.b.smithi179.stdout:starting mds.b at 2024-05-22T21:18:07.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.596+0000 7f5494600640 1 Processor -- start 2024-05-22T21:18:07.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.596+0000 7f5494600640 1 -- start start 2024-05-22T21:18:07.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.596+0000 7f5494600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f548c13fe50 0x7f548c164e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:07.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.596+0000 7f5494600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548c13d3a0 con 0x7f548c13cfc0 2024-05-22T21:18:07.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.596+0000 7f5494600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548c0cd920 con 0x7f548c1402e0 2024-05-22T21:18:07.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.596+0000 7f5494600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f548c0cdc00 con 0x7f548c13fe50 2024-05-22T21:18:07.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.596+0000 7f5491c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f548c13cfc0 0x7f548c15af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:50670/0 (socket says 172.21.15.161:50670) 2024-05-22T21:18:07.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.596+0000 7f5491c00640 1 -- 172.21.15.161:0/2140947897 learned_addr learned my addr 172.21.15.161:0/2140947897 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:07.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.596+0000 7f5492600640 1 --2- 172.21.15.161:0/2140947897 >> v2:172.21.15.161:3300/0 conn(0x7f548c13fe50 0x7f548c164e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:07.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.596+0000 7f548be00640 1 -- 172.21.15.161:0/2140947897 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2061108015 0 0) 0x7f548c13d3a0 con 0x7f548c13cfc0 2024-05-22T21:18:07.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.596+0000 7f548be00640 1 -- 172.21.15.161:0/2140947897 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5474003530 con 0x7f548c13cfc0 2024-05-22T21:18:07.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.596+0000 7f5492600640 1 -- 172.21.15.161:0/2140947897 >> v1:172.21.15.179:6789/0 conn(0x7f548c1402e0 legacy=0x7f548c13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:07.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.596+0000 7f5492600640 1 -- 172.21.15.161:0/2140947897 >> v1:172.21.15.161:6789/0 conn(0x7f548c13cfc0 legacy=0x7f548c15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:07.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.596+0000 7f5492600640 1 -- 172.21.15.161:0/2140947897 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c0cdee0 con 0x7f548c13fe50 2024-05-22T21:18:07.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.600+0000 7f5492600640 1 --2- 172.21.15.161:0/2140947897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c13fe50 0x7f548c164e60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5484009d90 tx=0x7f548401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0d4caa3415d56d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:07.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.600+0000 7f548be00640 1 -- 172.21.15.161:0/2140947897 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54840a57a0 con 0x7f548c13fe50 2024-05-22T21:18:07.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.600+0000 7f548be00640 1 -- 172.21.15.161:0/2140947897 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54840a5900 con 0x7f548c13fe50 2024-05-22T21:18:07.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.600+0000 7f548be00640 1 -- 172.21.15.161:0/2140947897 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54840a5c70 con 0x7f548c13fe50 2024-05-22T21:18:07.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.600+0000 7f5494600640 1 -- 172.21.15.161:0/2140947897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c13fe50 msgr2=0x7f548c164e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:07.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.600+0000 7f5494600640 1 --2- 172.21.15.161:0/2140947897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c13fe50 0x7f548c164e60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5484009d90 tx=0x7f548401c8e0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.600+0000 7f5494600640 1 -- 172.21.15.161:0/2140947897 shutdown_connections 2024-05-22T21:18:07.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.600+0000 7f5494600640 1 --2- 172.21.15.161:0/2140947897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c13fe50 0x7f548c164e60 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.600+0000 7f5494600640 1 -- 172.21.15.161:0/2140947897 >> 172.21.15.161:0/2140947897 conn(0x7f548c06a2d0 msgr2=0x7f548c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:07.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.600+0000 7f5494600640 1 -- 172.21.15.161:0/2140947897 shutdown_connections 2024-05-22T21:18:07.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.600+0000 7f5494600640 1 -- 172.21.15.161:0/2140947897 wait complete. 2024-05-22T21:18:07.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.600+0000 7f5494600640 1 Processor -- start 2024-05-22T21:18:07.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5494600640 1 -- start start 2024-05-22T21:18:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5494600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c13cfc0 0x7f548c13b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5494600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f548c13fe50 0x7f548c157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5494600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f548c1402e0 0x7f548c13e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5494600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c13fba0 con 0x7f548c13cfc0 2024-05-22T21:18:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5494600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c13eb00 con 0x7f548c13fe50 2024-05-22T21:18:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5494600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548c13ede0 con 0x7f548c1402e0 2024-05-22T21:18:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5491c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c13cfc0 0x7f548c13b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5491c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c13cfc0 0x7f548c13b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41616/0 (socket says 172.21.15.161:41616) 2024-05-22T21:18:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5491c00640 1 -- 172.21.15.161:0/9133617 learned_addr learned my addr 172.21.15.161:0/9133617 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5491200640 1 --2- 172.21.15.161:0/9133617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f548c13fe50 0x7f548c157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5492600640 1 --2- 172.21.15.161:0/9133617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f548c1402e0 0x7f548c13e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:07.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5491c00640 1 -- 172.21.15.161:0/9133617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f548c1402e0 msgr2=0x7f548c13e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:07.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5491c00640 1 --2- 172.21.15.161:0/9133617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f548c1402e0 0x7f548c13e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5491c00640 1 -- 172.21.15.161:0/9133617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f548c13fe50 msgr2=0x7f548c157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:07.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5491c00640 1 --2- 172.21.15.161:0/9133617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f548c13fe50 0x7f548c157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:07.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5491c00640 1 -- 172.21.15.161:0/9133617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5484009a70 con 0x7f548c13cfc0 2024-05-22T21:18:07.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5491c00640 1 --2- 172.21.15.161:0/9133617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c13cfc0 0x7f548c13b820 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f548000c420 tx=0x7f548000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:07.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5489600640 1 -- 172.21.15.161:0/9133617 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480005cf0 con 0x7f548c13cfc0 2024-05-22T21:18:07.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5489600640 1 -- 172.21.15.161:0/9133617 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5480005e50 con 0x7f548c13cfc0 2024-05-22T21:18:07.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5494600640 1 -- 172.21.15.161:0/9133617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548c13f0c0 con 0x7f548c13cfc0 2024-05-22T21:18:07.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5489600640 1 -- 172.21.15.161:0/9133617 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54800072d0 con 0x7f548c13cfc0 2024-05-22T21:18:07.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5494600640 1 -- 172.21.15.161:0/9133617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548c13f680 con 0x7f548c13cfc0 2024-05-22T21:18:07.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.604+0000 7f5494600640 1 -- 172.21.15.161:0/9133617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f548c067220 con 0x7f548c13cfc0 2024-05-22T21:18:07.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.608+0000 7f5489600640 1 -- 172.21.15.161:0/9133617 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5480007430 con 0x7f548c13cfc0 2024-05-22T21:18:07.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.608+0000 7f5489600640 1 --2- 172.21.15.161:0/9133617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f546c066690 0x7f546c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:07.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.608+0000 7f5489600640 1 -- 172.21.15.161:0/9133617 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5480083770 con 0x7f548c13cfc0 2024-05-22T21:18:07.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.608+0000 7f5491200640 1 --2- 172.21.15.161:0/9133617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f546c066690 0x7f546c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:07.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.608+0000 7f5491200640 1 --2- 172.21.15.161:0/9133617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f546c066690 0x7f546c068b50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f548c159ff0 tx=0x7f5478009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:07.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.612+0000 7f5489600640 1 -- 172.21.15.161:0/9133617 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5480050a90 con 0x7f548c13cfc0 2024-05-22T21:18:07.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:07.796+0000 7f5494600640 1 -- 172.21.15.161:0/9133617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f548c070430 con 0x7f548c13cfc0 2024-05-22T21:18:08.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.440+0000 7f5489600640 1 -- 172.21.15.161:0/9133617 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7f54800540b0 con 0x7f548c13cfc0 2024-05-22T21:18:08.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.440+0000 7f5494600640 1 -- 172.21.15.161:0/9133617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f546c066690 msgr2=0x7f546c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:08.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.440+0000 7f5494600640 1 --2- 172.21.15.161:0/9133617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f546c066690 0x7f546c068b50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f548c159ff0 tx=0x7f5478009040 comp rx=0 tx=0).stop 2024-05-22T21:18:08.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.440+0000 7f5494600640 1 -- 172.21.15.161:0/9133617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c13cfc0 msgr2=0x7f548c13b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:08.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.440+0000 7f5494600640 1 --2- 172.21.15.161:0/9133617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c13cfc0 0x7f548c13b820 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f548000c420 tx=0x7f548000c8e0 comp rx=0 tx=0).stop 2024-05-22T21:18:08.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.440+0000 7f5494600640 1 -- 172.21.15.161:0/9133617 shutdown_connections 2024-05-22T21:18:08.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.440+0000 7f5494600640 1 --2- 172.21.15.161:0/9133617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f546c066690 0x7f546c068b50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:08.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.440+0000 7f5494600640 1 --2- 172.21.15.161:0/9133617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f548c1402e0 0x7f548c13e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:08.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.440+0000 7f5494600640 1 --2- 172.21.15.161:0/9133617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f548c13fe50 0x7f548c157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:08.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.440+0000 7f5494600640 1 --2- 172.21.15.161:0/9133617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c13cfc0 0x7f548c13b820 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:08.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.440+0000 7f5494600640 1 -- 172.21.15.161:0/9133617 >> 172.21.15.161:0/9133617 conn(0x7f548c06a2d0 msgr2=0x7f548c06a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:08.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.440+0000 7f5494600640 1 -- 172.21.15.161:0/9133617 shutdown_connections 2024-05-22T21:18:08.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.440+0000 7f5494600640 1 -- 172.21.15.161:0/9133617 wait complete. 2024-05-22T21:18:08.466 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-22T21:18:08.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.604+0000 7f3952800640 1 Processor -- start 2024-05-22T21:18:08.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.604+0000 7f3952800640 1 -- start start 2024-05-22T21:18:08.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.608+0000 7f3952800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f394c0ccc10 0x7f394c15c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:08.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.608+0000 7f3952800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394c06d0d0 con 0x7f394c144980 2024-05-22T21:18:08.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.608+0000 7f3952800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394c06c4c0 con 0x7f394c06d560 2024-05-22T21:18:08.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.608+0000 7f3952800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394c06c7a0 con 0x7f394c0ccc10 2024-05-22T21:18:08.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.608+0000 7f394b400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f394c144980 0x7f394c15c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:50672/0 (socket says 172.21.15.161:50672) 2024-05-22T21:18:08.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.608+0000 7f394b400640 1 -- 172.21.15.161:0/3005609140 learned_addr learned my addr 172.21.15.161:0/3005609140 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:08.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.608+0000 7f394be00640 1 --2- 172.21.15.161:0/3005609140 >> v2:172.21.15.161:3300/0 conn(0x7f394c0ccc10 0x7f394c15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:08.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.608+0000 7f394aa00640 1 -- 172.21.15.161:0/3005609140 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165491691 0 0) 0x7f394c06d0d0 con 0x7f394c144980 2024-05-22T21:18:08.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.608+0000 7f394aa00640 1 -- 172.21.15.161:0/3005609140 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3930003530 con 0x7f394c144980 2024-05-22T21:18:08.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.608+0000 7f394be00640 1 -- 172.21.15.161:0/3005609140 >> v1:172.21.15.179:6789/0 conn(0x7f394c06d560 legacy=0x7f394c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:08.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.608+0000 7f394be00640 1 -- 172.21.15.161:0/3005609140 >> v1:172.21.15.161:6789/0 conn(0x7f394c144980 legacy=0x7f394c15c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:08.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.608+0000 7f394be00640 1 -- 172.21.15.161:0/3005609140 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c06ca80 con 0x7f394c0ccc10 2024-05-22T21:18:08.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f394be00640 1 --2- 172.21.15.161:0/3005609140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f394c0ccc10 0x7f394c15c880 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f39440097c0 tx=0x7f394401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97133cca83ae9f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:08.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f394aa00640 1 -- 172.21.15.161:0/3005609140 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39440ab600 con 0x7f394c0ccc10 2024-05-22T21:18:08.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f394aa00640 1 -- 172.21.15.161:0/3005609140 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39440a7050 con 0x7f394c0ccc10 2024-05-22T21:18:08.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f394aa00640 1 -- 172.21.15.161:0/3005609140 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39440ab980 con 0x7f394c0ccc10 2024-05-22T21:18:08.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f3952800640 1 -- 172.21.15.161:0/3005609140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f394c0ccc10 msgr2=0x7f394c15c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:08.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f3952800640 1 --2- 172.21.15.161:0/3005609140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f394c0ccc10 0x7f394c15c880 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f39440097c0 tx=0x7f394401c4a0 comp rx=0 tx=0).stop 2024-05-22T21:18:08.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f3952800640 1 -- 172.21.15.161:0/3005609140 shutdown_connections 2024-05-22T21:18:08.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f3952800640 1 --2- 172.21.15.161:0/3005609140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f394c0ccc10 0x7f394c15c880 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:08.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f3952800640 1 -- 172.21.15.161:0/3005609140 >> 172.21.15.161:0/3005609140 conn(0x7f394c06a0a0 msgr2=0x7f394c06a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:08.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f3952800640 1 -- 172.21.15.161:0/3005609140 shutdown_connections 2024-05-22T21:18:08.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f3952800640 1 -- 172.21.15.161:0/3005609140 wait complete. 2024-05-22T21:18:08.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f3952800640 1 Processor -- start 2024-05-22T21:18:08.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f3952800640 1 -- start start 2024-05-22T21:18:08.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f3952800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f394c06d560 0x7f394c13c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:08.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f3952800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f394c0ccc10 0x7f394c143130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:08.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f3952800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f394c144980 0x7f394c1401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:08.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f3952800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c140720 con 0x7f394c06d560 2024-05-22T21:18:08.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f3952800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c13b150 con 0x7f394c0ccc10 2024-05-22T21:18:08.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f3952800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394c13b430 con 0x7f394c144980 2024-05-22T21:18:08.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.612+0000 7f394b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f394c06d560 0x7f394c13c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:08.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f394b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f394c06d560 0x7f394c13c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41642/0 (socket says 172.21.15.161:41642) 2024-05-22T21:18:08.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f394b400640 1 -- 172.21.15.161:0/2241032812 learned_addr learned my addr 172.21.15.161:0/2241032812 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:08.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f394be00640 1 --2- 172.21.15.161:0/2241032812 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f394c144980 0x7f394c1401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:08.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f3942a00640 1 --2- 172.21.15.161:0/2241032812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f394c0ccc10 0x7f394c143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:08.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f394b400640 1 -- 172.21.15.161:0/2241032812 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f394c144980 msgr2=0x7f394c1401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:08.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f394b400640 1 --2- 172.21.15.161:0/2241032812 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f394c144980 0x7f394c1401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:08.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f394b400640 1 -- 172.21.15.161:0/2241032812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f394c0ccc10 msgr2=0x7f394c143130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:08.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f394b400640 1 --2- 172.21.15.161:0/2241032812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f394c0ccc10 0x7f394c143130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:08.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f394b400640 1 -- 172.21.15.161:0/2241032812 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3944009420 con 0x7f394c06d560 2024-05-22T21:18:08.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f394b400640 1 --2- 172.21.15.161:0/2241032812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f394c06d560 0x7f394c13c1f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f393c00cbc0 tx=0x7f393c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:08.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f3943e00640 1 -- 172.21.15.161:0/2241032812 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c00ad40 con 0x7f394c06d560 2024-05-22T21:18:08.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f3943e00640 1 -- 172.21.15.161:0/2241032812 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c012400 con 0x7f394c06d560 2024-05-22T21:18:08.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f3952800640 1 -- 172.21.15.161:0/2241032812 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394c13b710 con 0x7f394c06d560 2024-05-22T21:18:08.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f3943e00640 1 -- 172.21.15.161:0/2241032812 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c00aea0 con 0x7f394c06d560 2024-05-22T21:18:08.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f3952800640 1 -- 172.21.15.161:0/2241032812 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394c13bcd0 con 0x7f394c06d560 2024-05-22T21:18:08.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f3952800640 1 -- 172.21.15.161:0/2241032812 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394c067220 con 0x7f394c06d560 2024-05-22T21:18:08.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f3943e00640 1 -- 172.21.15.161:0/2241032812 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f393c007070 con 0x7f394c06d560 2024-05-22T21:18:08.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f3943e00640 1 --2- 172.21.15.161:0/2241032812 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3928066690 0x7f3928068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:08.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f3943e00640 1 -- 172.21.15.161:0/2241032812 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f393c084250 con 0x7f394c06d560 2024-05-22T21:18:08.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.616+0000 7f3942a00640 1 --2- 172.21.15.161:0/2241032812 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3928066690 0x7f3928068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:08.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.620+0000 7f3942a00640 1 --2- 172.21.15.161:0/2241032812 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3928066690 0x7f3928068b50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f394c155500 tx=0x7f3934009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:08.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.620+0000 7f3943e00640 1 -- 172.21.15.161:0/2241032812 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f393c0514e0 con 0x7f394c06d560 2024-05-22T21:18:08.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:08.776+0000 7f3952800640 1 -- 172.21.15.161:0/2241032812 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f394c158b90 con 0x7f394c06d560 2024-05-22T21:18:09.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.448+0000 7f3943e00640 1 -- 172.21.15.161:0/2241032812 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) v1 ==== 181+0+0 (secure 0 0 0) 0x7f393c0170d0 con 0x7f394c06d560 2024-05-22T21:18:09.449 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' removed 2024-05-22T21:18:09.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.448+0000 7f3952800640 1 -- 172.21.15.161:0/2241032812 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3928066690 msgr2=0x7f3928068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.448+0000 7f3952800640 1 --2- 172.21.15.161:0/2241032812 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3928066690 0x7f3928068b50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f394c155500 tx=0x7f3934009040 comp rx=0 tx=0).stop 2024-05-22T21:18:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.448+0000 7f3952800640 1 -- 172.21.15.161:0/2241032812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f394c06d560 msgr2=0x7f394c13c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.448+0000 7f3952800640 1 --2- 172.21.15.161:0/2241032812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f394c06d560 0x7f394c13c1f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f393c00cbc0 tx=0x7f393c00a430 comp rx=0 tx=0).stop 2024-05-22T21:18:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.448+0000 7f3952800640 1 -- 172.21.15.161:0/2241032812 shutdown_connections 2024-05-22T21:18:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.448+0000 7f3952800640 1 --2- 172.21.15.161:0/2241032812 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3928066690 0x7f3928068b50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.448+0000 7f3952800640 1 --2- 172.21.15.161:0/2241032812 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f394c144980 0x7f394c1401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.448+0000 7f3952800640 1 --2- 172.21.15.161:0/2241032812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f394c0ccc10 0x7f394c143130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.448+0000 7f3952800640 1 --2- 172.21.15.161:0/2241032812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f394c06d560 0x7f394c13c1f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.448+0000 7f3952800640 1 -- 172.21.15.161:0/2241032812 >> 172.21.15.161:0/2241032812 conn(0x7f394c06a0a0 msgr2=0x7f394c0b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.448+0000 7f3952800640 1 -- 172.21.15.161:0/2241032812 shutdown_connections 2024-05-22T21:18:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.448+0000 7f3952800640 1 -- 172.21.15.161:0/2241032812 wait complete. 2024-05-22T21:18:09.473 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-22T21:18:09.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.612+0000 7f0336600640 1 Processor -- start 2024-05-22T21:18:09.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.612+0000 7f0336600640 1 -- start start 2024-05-22T21:18:09.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.612+0000 7f0336600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f033013cfe0 0x7f033015dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:09.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.612+0000 7f0336600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f033013d560 0x7f033015e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:09.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.612+0000 7f0336600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03300ce830 con 0x7f03300cdb00 2024-05-22T21:18:09.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.612+0000 7f0336600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03300ce650 con 0x7f033013d560 2024-05-22T21:18:09.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.612+0000 7f0336600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0330140560 con 0x7f033013cfe0 2024-05-22T21:18:09.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.612+0000 7f032fe00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f03300cdb00 0x7f03300cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:50682/0 (socket says 172.21.15.161:50682) 2024-05-22T21:18:09.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.612+0000 7f032fe00640 1 -- 172.21.15.161:0/4218730097 learned_addr learned my addr 172.21.15.161:0/4218730097 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:09.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032f400640 1 --2- 172.21.15.161:0/4218730097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f033013cfe0 0x7f033015dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:09.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032ea00640 1 -- 172.21.15.161:0/4218730097 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529820360 0 0) 0x7f03300ce830 con 0x7f03300cdb00 2024-05-22T21:18:09.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f0326a00640 1 --2- 172.21.15.161:0/4218730097 >> v2:172.21.15.179:3300/0 conn(0x7f033013d560 0x7f033015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:09.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032ea00640 1 -- 172.21.15.161:0/4218730097 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0314003530 con 0x7f03300cdb00 2024-05-22T21:18:09.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032ea00640 1 -- 172.21.15.161:0/4218730097 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3886474555 0 0) 0x7f0314003530 con 0x7f03300cdb00 2024-05-22T21:18:09.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032ea00640 1 -- 172.21.15.161:0/4218730097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f033013cfe0 msgr2=0x7f033015dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:09.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032ea00640 1 --2- 172.21.15.161:0/4218730097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f033013cfe0 0x7f033015dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:09.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032ea00640 1 -- 172.21.15.161:0/4218730097 >> v2:172.21.15.179:3300/0 conn(0x7f033013d560 msgr2=0x7f033015e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:09.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032ea00640 1 --2- 172.21.15.161:0/4218730097 >> v2:172.21.15.179:3300/0 conn(0x7f033013d560 0x7f033015e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:09.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032ea00640 1 -- 172.21.15.161:0/4218730097 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03300ceb10 con 0x7f03300cdb00 2024-05-22T21:18:09.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032ea00640 1 -- 172.21.15.161:0/4218730097 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f0328001680 con 0x7f03300cdb00 2024-05-22T21:18:09.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032ea00640 1 -- 172.21.15.161:0/4218730097 >> v1:172.21.15.161:6789/0 conn(0x7f03300cdb00 legacy=0x7f03300cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:09.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032ea00640 1 --2- 172.21.15.161:0/4218730097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0314003980 0x7f0314003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:09.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032ea00640 1 --2- 172.21.15.161:0/4218730097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03140047d0 0x7f0314004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:09.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032ea00640 1 --2- 172.21.15.161:0/4218730097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0314005190 0x7f031400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:09.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032ea00640 1 -- 172.21.15.161:0/4218730097 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03300ce650 con 0x7f0314003980 2024-05-22T21:18:09.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032ea00640 1 -- 172.21.15.161:0/4218730097 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0330140560 con 0x7f03140047d0 2024-05-22T21:18:09.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032ea00640 1 -- 172.21.15.161:0/4218730097 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031400bf00 con 0x7f0314005190 2024-05-22T21:18:09.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032f400640 1 --2- 172.21.15.161:0/4218730097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f033013cfe0 0x7f033015dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:09.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f0326a00640 1 --2- 172.21.15.161:0/4218730097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03140047d0 0x7f0314004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:09.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032f400640 1 --2- 172.21.15.161:0/4218730097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0314003980 0x7f0314003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:09.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f032fe00640 1 --2- 172.21.15.161:0/4218730097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0314005190 0x7f031400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:09.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f0326a00640 1 -- 172.21.15.161:0/4218730097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0314005190 msgr2=0x7f031400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:09.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f0326a00640 1 --2- 172.21.15.161:0/4218730097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0314005190 0x7f031400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:09.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f0326a00640 1 -- 172.21.15.161:0/4218730097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0314003980 msgr2=0x7f0314003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:09.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f0326a00640 1 --2- 172.21.15.161:0/4218730097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0314003980 0x7f0314003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:09.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.616+0000 7f0326a00640 1 -- 172.21.15.161:0/4218730097 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0314003530 con 0x7f03140047d0 2024-05-22T21:18:09.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.620+0000 7f0326a00640 1 --2- 172.21.15.161:0/4218730097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03140047d0 0x7f0314004c50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f033006c240 tx=0x7f031801ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa60c684c591ebc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:09.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.620+0000 7f032ea00640 1 -- 172.21.15.161:0/4218730097 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03180a6510 con 0x7f03140047d0 2024-05-22T21:18:09.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.620+0000 7f032ea00640 1 -- 172.21.15.161:0/4218730097 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03180a6670 con 0x7f03140047d0 2024-05-22T21:18:09.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.620+0000 7f032ea00640 1 -- 172.21.15.161:0/4218730097 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03180a69e0 con 0x7f03140047d0 2024-05-22T21:18:09.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.620+0000 7f0336600640 1 -- 172.21.15.161:0/4218730097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03140047d0 msgr2=0x7f0314004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:09.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.620+0000 7f0336600640 1 --2- 172.21.15.161:0/4218730097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03140047d0 0x7f0314004c50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f033006c240 tx=0x7f031801ca20 comp rx=0 tx=0).stop 2024-05-22T21:18:09.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.620+0000 7f032fe00640 1 -- 172.21.15.161:0/4218730097 reap_dead start 2024-05-22T21:18:09.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.620+0000 7f0336600640 1 -- 172.21.15.161:0/4218730097 shutdown_connections 2024-05-22T21:18:09.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.620+0000 7f0336600640 1 -- 172.21.15.161:0/4218730097 >> 172.21.15.161:0/4218730097 conn(0x7f033006a0a0 msgr2=0x7f033006a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:09.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.620+0000 7f0336600640 1 -- 172.21.15.161:0/4218730097 shutdown_connections 2024-05-22T21:18:09.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.620+0000 7f0336600640 1 -- 172.21.15.161:0/4218730097 wait complete. 2024-05-22T21:18:09.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.620+0000 7f0336600640 1 Processor -- start 2024-05-22T21:18:09.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.620+0000 7f0336600640 1 -- start start 2024-05-22T21:18:09.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0336600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03140047d0 0x7f03301397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:09.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0336600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03301f9520 0x7f03301408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:09.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0336600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03300aef10 0x7f03300af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:09.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0336600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0330142180 con 0x7f03301f9520 2024-05-22T21:18:09.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0336600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03300af8e0 con 0x7f03140047d0 2024-05-22T21:18:09.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0336600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0330141220 con 0x7f03300aef10 2024-05-22T21:18:09.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0326a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03301f9520 0x7f03301408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:09.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0326a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03301f9520 0x7f03301408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41658/0 (socket says 172.21.15.161:41658) 2024-05-22T21:18:09.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0326a00640 1 -- 172.21.15.161:0/2523610034 learned_addr learned my addr 172.21.15.161:0/2523610034 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:09.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f032fe00640 1 --2- 172.21.15.161:0/2523610034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03300aef10 0x7f03300af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:09.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f032f400640 1 --2- 172.21.15.161:0/2523610034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03140047d0 0x7f03301397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:09.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0326a00640 1 -- 172.21.15.161:0/2523610034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03300aef10 msgr2=0x7f03300af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:09.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0326a00640 1 --2- 172.21.15.161:0/2523610034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03300aef10 0x7f03300af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:09.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0326a00640 1 -- 172.21.15.161:0/2523610034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03140047d0 msgr2=0x7f03301397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:09.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0326a00640 1 --2- 172.21.15.161:0/2523610034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03140047d0 0x7f03301397b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0326a00640 1 -- 172.21.15.161:0/2523610034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0330141500 con 0x7f03301f9520 2024-05-22T21:18:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0326a00640 1 --2- 172.21.15.161:0/2523610034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03301f9520 0x7f03301408b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f03180b0040 tx=0x7f0314005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0327e00640 1 -- 172.21.15.161:0/2523610034 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03180a6510 con 0x7f03301f9520 2024-05-22T21:18:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0327e00640 1 -- 172.21.15.161:0/2523610034 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03180a6e60 con 0x7f03301f9520 2024-05-22T21:18:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0327e00640 1 -- 172.21.15.161:0/2523610034 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03180b9680 con 0x7f03301f9520 2024-05-22T21:18:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0336600640 1 -- 172.21.15.161:0/2523610034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03301417e0 con 0x7f03301f9520 2024-05-22T21:18:09.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0336600640 1 -- 172.21.15.161:0/2523610034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0330141da0 con 0x7f03301f9520 2024-05-22T21:18:09.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0336600640 1 -- 172.21.15.161:0/2523610034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0330067220 con 0x7f03301f9520 2024-05-22T21:18:09.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.624+0000 7f0327e00640 1 -- 172.21.15.161:0/2523610034 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f03180b97e0 con 0x7f03301f9520 2024-05-22T21:18:09.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.628+0000 7f0327e00640 1 --2- 172.21.15.161:0/2523610034 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f030c066690 0x7f030c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:09.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.628+0000 7f0327e00640 1 -- 172.21.15.161:0/2523610034 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f03180aa070 con 0x7f03301f9520 2024-05-22T21:18:09.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.628+0000 7f032f400640 1 --2- 172.21.15.161:0/2523610034 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f030c066690 0x7f030c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:09.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.628+0000 7f032f400640 1 --2- 172.21.15.161:0/2523610034 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f030c066690 0x7f030c068b50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f033006e070 tx=0x7f032000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:09.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.632+0000 7f0327e00640 1 -- 172.21.15.161:0/2523610034 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f03180efc60 con 0x7f03301f9520 2024-05-22T21:18:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:09.784+0000 7f0336600640 1 -- 172.21.15.161:0/2523610034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0330071d50 con 0x7f03301f9520 2024-05-22T21:18:10.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.456+0000 7f0327e00640 1 -- 172.21.15.161:0/2523610034 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) v1 ==== 169+0+0 (secure 0 0 0) 0x7f03180ef7f0 con 0x7f03301f9520 2024-05-22T21:18:10.457 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' removed 2024-05-22T21:18:10.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.456+0000 7f0336600640 1 -- 172.21.15.161:0/2523610034 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f030c066690 msgr2=0x7f030c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:10.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.456+0000 7f0336600640 1 --2- 172.21.15.161:0/2523610034 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f030c066690 0x7f030c068b50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f033006e070 tx=0x7f032000b040 comp rx=0 tx=0).stop 2024-05-22T21:18:10.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.456+0000 7f0336600640 1 -- 172.21.15.161:0/2523610034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03301f9520 msgr2=0x7f03301408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:10.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.456+0000 7f0336600640 1 --2- 172.21.15.161:0/2523610034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03301f9520 0x7f03301408b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f03180b0040 tx=0x7f0314005800 comp rx=0 tx=0).stop 2024-05-22T21:18:10.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.456+0000 7f0336600640 1 -- 172.21.15.161:0/2523610034 shutdown_connections 2024-05-22T21:18:10.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.456+0000 7f0336600640 1 --2- 172.21.15.161:0/2523610034 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f030c066690 0x7f030c068b50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.456+0000 7f0336600640 1 --2- 172.21.15.161:0/2523610034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03300aef10 0x7f03300af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.456+0000 7f0336600640 1 --2- 172.21.15.161:0/2523610034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03301f9520 0x7f03301408b0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.456+0000 7f0336600640 1 --2- 172.21.15.161:0/2523610034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03140047d0 0x7f03301397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.456+0000 7f0336600640 1 -- 172.21.15.161:0/2523610034 >> 172.21.15.161:0/2523610034 conn(0x7f033006a0a0 msgr2=0x7f033006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:10.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.456+0000 7f0336600640 1 -- 172.21.15.161:0/2523610034 shutdown_connections 2024-05-22T21:18:10.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.456+0000 7f0336600640 1 -- 172.21.15.161:0/2523610034 wait complete. 2024-05-22T21:18:10.482 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-22T21:18:10.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc44c800640 1 Processor -- start 2024-05-22T21:18:10.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc44c800640 1 -- start start 2024-05-22T21:18:10.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc44c800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4441402e0 0x7fc44413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:10.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc44c800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc44413fc80 0x7fc44406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:10.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc44c800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4441400d0 con 0x7fc44413cfc0 2024-05-22T21:18:10.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc44c800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44413d3a0 con 0x7fc44413fc80 2024-05-22T21:18:10.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc44c800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4440cd920 con 0x7fc4441402e0 2024-05-22T21:18:10.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc44a800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc44413fc80 0x7fc44406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:10.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc449e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc44413cfc0 0x7fc44415af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:50688/0 (socket says 172.21.15.161:50688) 2024-05-22T21:18:10.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc449e00640 1 -- 172.21.15.161:0/2617528623 learned_addr learned my addr 172.21.15.161:0/2617528623 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:10.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc449400640 1 --2- 172.21.15.161:0/2617528623 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4441402e0 0x7fc44413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:10.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc44a800640 1 -- 172.21.15.161:0/2617528623 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4441402e0 msgr2=0x7fc44413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:10.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc44a800640 1 --2- 172.21.15.161:0/2617528623 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4441402e0 0x7fc44413f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc44a800640 1 -- 172.21.15.161:0/2617528623 >> v1:172.21.15.161:6789/0 conn(0x7fc44413cfc0 legacy=0x7fc44415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:10.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc44a800640 1 -- 172.21.15.161:0/2617528623 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4440cdc00 con 0x7fc44413fc80 2024-05-22T21:18:10.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc448a00640 1 -- 172.21.15.161:0/2617528623 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 942460082 0 0) 0x7fc438001120 con 0x7fc44413cfc0 2024-05-22T21:18:10.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.620+0000 7fc449400640 1 --2- 172.21.15.161:0/2617528623 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4441402e0 0x7fc44413f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:10.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.624+0000 7fc44a800640 1 --2- 172.21.15.161:0/2617528623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc44413fc80 0x7fc44406fce0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc440009d90 tx=0x7fc44001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7ffd573ba67476 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:10.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.624+0000 7fc448a00640 1 -- 172.21.15.161:0/2617528623 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4400aa660 con 0x7fc44413fc80 2024-05-22T21:18:10.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.624+0000 7fc448a00640 1 -- 172.21.15.161:0/2617528623 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4400aa7c0 con 0x7fc44413fc80 2024-05-22T21:18:10.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.624+0000 7fc448a00640 1 -- 172.21.15.161:0/2617528623 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4400aab50 con 0x7fc44413fc80 2024-05-22T21:18:10.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.624+0000 7fc44c800640 1 -- 172.21.15.161:0/2617528623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc44413fc80 msgr2=0x7fc44406fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:10.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.624+0000 7fc44c800640 1 --2- 172.21.15.161:0/2617528623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc44413fc80 0x7fc44406fce0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc440009d90 tx=0x7fc44001c8b0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.624+0000 7fc44c800640 1 -- 172.21.15.161:0/2617528623 shutdown_connections 2024-05-22T21:18:10.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.624+0000 7fc44c800640 1 --2- 172.21.15.161:0/2617528623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc44413fc80 0x7fc44406fce0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.624+0000 7fc44c800640 1 --2- 172.21.15.161:0/2617528623 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4441402e0 0x7fc44413f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.624+0000 7fc44c800640 1 -- 172.21.15.161:0/2617528623 >> 172.21.15.161:0/2617528623 conn(0x7fc44406a2d0 msgr2=0x7fc44406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:10.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.624+0000 7fc44c800640 1 -- 172.21.15.161:0/2617528623 shutdown_connections 2024-05-22T21:18:10.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.624+0000 7fc44c800640 1 -- 172.21.15.161:0/2617528623 wait complete. 2024-05-22T21:18:10.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.624+0000 7fc44c800640 1 Processor -- start 2024-05-22T21:18:10.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.624+0000 7fc44c800640 1 -- start start 2024-05-22T21:18:10.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44c800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc44413cfc0 0x7fc444137790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:10.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44c800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc44413fc80 0x7fc444138e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44c800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4441402e0 0x7fc444156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44c800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc444139350 con 0x7fc4441402e0 2024-05-22T21:18:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44c800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc444137d70 con 0x7fc44413fc80 2024-05-22T21:18:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44c800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc444138050 con 0x7fc44413cfc0 2024-05-22T21:18:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4441402e0 0x7fc444156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4441402e0 0x7fc444156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41664/0 (socket says 172.21.15.161:41664) 2024-05-22T21:18:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44a800640 1 -- 172.21.15.161:0/2916998971 learned_addr learned my addr 172.21.15.161:0/2916998971 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc449e00640 1 --2- 172.21.15.161:0/2916998971 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc44413cfc0 0x7fc444137790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc449400640 1 --2- 172.21.15.161:0/2916998971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc44413fc80 0x7fc444138e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44a800640 1 -- 172.21.15.161:0/2916998971 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc44413cfc0 msgr2=0x7fc444137790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44a800640 1 --2- 172.21.15.161:0/2916998971 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc44413cfc0 0x7fc444137790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44a800640 1 -- 172.21.15.161:0/2916998971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc44413fc80 msgr2=0x7fc444138e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:10.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44a800640 1 --2- 172.21.15.161:0/2916998971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc44413fc80 0x7fc444138e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44a800640 1 -- 172.21.15.161:0/2916998971 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc440009a70 con 0x7fc4441402e0 2024-05-22T21:18:10.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc449400640 1 --2- 172.21.15.161:0/2916998971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc44413fc80 0x7fc444138e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:10.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44a800640 1 --2- 172.21.15.161:0/2916998971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4441402e0 0x7fc444156fa0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc4400ac040 tx=0x7fc440019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:10.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc43e000640 1 -- 172.21.15.161:0/2916998971 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4400aae40 con 0x7fc4441402e0 2024-05-22T21:18:10.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc43e000640 1 -- 172.21.15.161:0/2916998971 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4400bf450 con 0x7fc4441402e0 2024-05-22T21:18:10.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc43e000640 1 -- 172.21.15.161:0/2916998971 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4400bf800 con 0x7fc4441402e0 2024-05-22T21:18:10.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44c800640 1 -- 172.21.15.161:0/2916998971 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc444138330 con 0x7fc4441402e0 2024-05-22T21:18:10.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44c800640 1 -- 172.21.15.161:0/2916998971 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4441388f0 con 0x7fc4441402e0 2024-05-22T21:18:10.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.628+0000 7fc44c800640 1 -- 172.21.15.161:0/2916998971 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc444067220 con 0x7fc4441402e0 2024-05-22T21:18:10.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.632+0000 7fc43e000640 1 -- 172.21.15.161:0/2916998971 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc4400bf960 con 0x7fc4441402e0 2024-05-22T21:18:10.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.632+0000 7fc43e000640 1 --2- 172.21.15.161:0/2916998971 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc428066690 0x7fc428068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:10.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.632+0000 7fc43e000640 1 -- 172.21.15.161:0/2916998971 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc4400a6040 con 0x7fc4441402e0 2024-05-22T21:18:10.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.632+0000 7fc449e00640 1 --2- 172.21.15.161:0/2916998971 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc428066690 0x7fc428068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:10.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.632+0000 7fc449e00640 1 --2- 172.21.15.161:0/2916998971 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc428066690 0x7fc428068b50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc438000f70 tx=0x7fc438007ba0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:10.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.636+0000 7fc43e000640 1 -- 172.21.15.161:0/2916998971 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc4400ec770 con 0x7fc4441402e0 2024-05-22T21:18:10.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.792+0000 7fc44c800640 1 -- 172.21.15.161:0/2916998971 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc444138ae0 con 0x7fc4441402e0 2024-05-22T21:18:10.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.796+0000 7fc43e000640 1 -- 172.21.15.161:0/2916998971 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc4400efd90 con 0x7fc4441402e0 2024-05-22T21:18:10.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.800+0000 7fc44c800640 1 -- 172.21.15.161:0/2916998971 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc428066690 msgr2=0x7fc428068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:10.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.800+0000 7fc44c800640 1 --2- 172.21.15.161:0/2916998971 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc428066690 0x7fc428068b50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc438000f70 tx=0x7fc438007ba0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.800+0000 7fc44c800640 1 -- 172.21.15.161:0/2916998971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4441402e0 msgr2=0x7fc444156fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:10.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.800+0000 7fc44c800640 1 --2- 172.21.15.161:0/2916998971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4441402e0 0x7fc444156fa0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc4400ac040 tx=0x7fc440019650 comp rx=0 tx=0).stop 2024-05-22T21:18:10.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.800+0000 7fc44c800640 1 -- 172.21.15.161:0/2916998971 shutdown_connections 2024-05-22T21:18:10.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.800+0000 7fc44c800640 1 --2- 172.21.15.161:0/2916998971 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc428066690 0x7fc428068b50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.800+0000 7fc44c800640 1 --2- 172.21.15.161:0/2916998971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4441402e0 0x7fc444156fa0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.800+0000 7fc44c800640 1 --2- 172.21.15.161:0/2916998971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc44413fc80 0x7fc444138e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.800+0000 7fc44c800640 1 --2- 172.21.15.161:0/2916998971 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc44413cfc0 0x7fc444137790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.800+0000 7fc44c800640 1 -- 172.21.15.161:0/2916998971 >> 172.21.15.161:0/2916998971 conn(0x7fc44406a2d0 msgr2=0x7fc4440cc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:10.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.800+0000 7fc44c800640 1 -- 172.21.15.161:0/2916998971 shutdown_connections 2024-05-22T21:18:10.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.800+0000 7fc44c800640 1 -- 172.21.15.161:0/2916998971 wait complete. 2024-05-22T21:18:10.821 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2024-05-22T21:18:10.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.952+0000 7f5788600640 1 Processor -- start 2024-05-22T21:18:10.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.952+0000 7f5788600640 1 -- start start 2024-05-22T21:18:10.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.956+0000 7f5788600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f578015dd50 0x7f578006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:10.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.956+0000 7f5788600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f578015d5e0 0x7f578015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:10.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.956+0000 7f5788600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5780071670 con 0x7f578015e130 2024-05-22T21:18:10.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.956+0000 7f5788600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f578006de40 con 0x7f578015dd50 2024-05-22T21:18:10.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.956+0000 7f5788600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5780070a60 con 0x7f578015d5e0 2024-05-22T21:18:10.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.956+0000 7f5785c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f578015dd50 0x7f578006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:10.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.956+0000 7f5785200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f578015e130 0x7f578015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:50700/0 (socket says 172.21.15.161:50700) 2024-05-22T21:18:10.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.956+0000 7f5785200640 1 -- 172.21.15.161:0/164177089 learned_addr learned my addr 172.21.15.161:0/164177089 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:10.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.956+0000 7f5786600640 1 --2- 172.21.15.161:0/164177089 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f578015d5e0 0x7f578015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:10.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.956+0000 7f5785c00640 1 -- 172.21.15.161:0/164177089 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f578015d5e0 msgr2=0x7f578015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:10.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.956+0000 7f5785c00640 1 --2- 172.21.15.161:0/164177089 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f578015d5e0 0x7f578015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.956+0000 7f5785c00640 1 -- 172.21.15.161:0/164177089 >> v1:172.21.15.161:6789/0 conn(0x7f578015e130 legacy=0x7f578015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:10.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.956+0000 7f5785c00640 1 -- 172.21.15.161:0/164177089 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5780070d40 con 0x7f578015dd50 2024-05-22T21:18:10.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.960+0000 7f5785c00640 1 --2- 172.21.15.161:0/164177089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f578015dd50 0x7f578006d900 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f57740097c0 tx=0x7f577401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0d60bf286273ef server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:10.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.960+0000 7f577fe00640 1 -- 172.21.15.161:0/164177089 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57740ab670 con 0x7f578015dd50 2024-05-22T21:18:10.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.960+0000 7f577fe00640 1 -- 172.21.15.161:0/164177089 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57740ab7d0 con 0x7f578015dd50 2024-05-22T21:18:10.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.960+0000 7f577fe00640 1 -- 172.21.15.161:0/164177089 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57740abb60 con 0x7f578015dd50 2024-05-22T21:18:10.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.960+0000 7f5788600640 1 -- 172.21.15.161:0/164177089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f578015dd50 msgr2=0x7f578006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:10.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.960+0000 7f5788600640 1 --2- 172.21.15.161:0/164177089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f578015dd50 0x7f578006d900 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f57740097c0 tx=0x7f577401c470 comp rx=0 tx=0).stop 2024-05-22T21:18:10.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.960+0000 7f5788600640 1 -- 172.21.15.161:0/164177089 shutdown_connections 2024-05-22T21:18:10.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.960+0000 7f5788600640 1 --2- 172.21.15.161:0/164177089 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f578015d5e0 0x7f578015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.960+0000 7f5788600640 1 --2- 172.21.15.161:0/164177089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f578015dd50 0x7f578006d900 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.960+0000 7f5788600640 1 -- 172.21.15.161:0/164177089 >> 172.21.15.161:0/164177089 conn(0x7f57800b0bc0 msgr2=0x7f57800b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:10.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.960+0000 7f5788600640 1 -- 172.21.15.161:0/164177089 shutdown_connections 2024-05-22T21:18:10.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.960+0000 7f5788600640 1 -- 172.21.15.161:0/164177089 wait complete. 2024-05-22T21:18:10.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.960+0000 7f5788600640 1 Processor -- start 2024-05-22T21:18:10.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.960+0000 7f5788600640 1 -- start start 2024-05-22T21:18:10.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5788600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f578015d5e0 0x7f578014ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:10.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5788600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f578015dd50 0x7f578014c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:10.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5788600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f578015e130 0x7f578014b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:10.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5788600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5780158470 con 0x7f578015d5e0 2024-05-22T21:18:10.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5788600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5780157c60 con 0x7f578015dd50 2024-05-22T21:18:10.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5788600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5780154a30 con 0x7f578015e130 2024-05-22T21:18:10.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5785c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f578015d5e0 0x7f578014ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:10.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5785c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f578015d5e0 0x7f578014ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41680/0 (socket says 172.21.15.161:41680) 2024-05-22T21:18:10.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5785c00640 1 -- 172.21.15.161:0/924362321 learned_addr learned my addr 172.21.15.161:0/924362321 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:10.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5785200640 1 --2- 172.21.15.161:0/924362321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f578015dd50 0x7f578014c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:10.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5786600640 1 --2- 172.21.15.161:0/924362321 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f578015e130 0x7f578014b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:10.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5785c00640 1 -- 172.21.15.161:0/924362321 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f578015e130 msgr2=0x7f578014b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:10.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5785c00640 1 --2- 172.21.15.161:0/924362321 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f578015e130 0x7f578014b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5785c00640 1 -- 172.21.15.161:0/924362321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f578015dd50 msgr2=0x7f578014c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:10.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5785c00640 1 --2- 172.21.15.161:0/924362321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f578015dd50 0x7f578014c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:10.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5785c00640 1 -- 172.21.15.161:0/924362321 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5774009420 con 0x7f578015d5e0 2024-05-22T21:18:10.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5785200640 1 --2- 172.21.15.161:0/924362321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f578015dd50 0x7f578014c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:10.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5785c00640 1 --2- 172.21.15.161:0/924362321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f578015d5e0 0x7f578014ae30 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f57740ad040 tx=0x7f577401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:10.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f577d600640 1 -- 172.21.15.161:0/924362321 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57740b5810 con 0x7f578015d5e0 2024-05-22T21:18:10.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f577d600640 1 -- 172.21.15.161:0/924362321 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57740b5990 con 0x7f578015d5e0 2024-05-22T21:18:10.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f577d600640 1 -- 172.21.15.161:0/924362321 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57740b5d40 con 0x7f578015d5e0 2024-05-22T21:18:10.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5788600640 1 -- 172.21.15.161:0/924362321 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5780157eb0 con 0x7f578015d5e0 2024-05-22T21:18:10.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5788600640 1 -- 172.21.15.161:0/924362321 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57801546e0 con 0x7f578015d5e0 2024-05-22T21:18:10.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f5788600640 1 -- 172.21.15.161:0/924362321 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5780070710 con 0x7f578015d5e0 2024-05-22T21:18:10.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f577d600640 1 -- 172.21.15.161:0/924362321 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f57740b4390 con 0x7f578015d5e0 2024-05-22T21:18:10.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.964+0000 7f577d600640 1 --2- 172.21.15.161:0/924362321 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5760066690 0x7f5760068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:10.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.968+0000 7f577d600640 1 -- 172.21.15.161:0/924362321 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f577411ec30 con 0x7f578015d5e0 2024-05-22T21:18:10.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.968+0000 7f5785200640 1 --2- 172.21.15.161:0/924362321 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5760066690 0x7f5760068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:10.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.968+0000 7f5785200640 1 --2- 172.21.15.161:0/924362321 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5760066690 0x7f5760068b50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f578014f4f0 tx=0x7f576c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:10.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:10.972+0000 7f577d600640 1 -- 172.21.15.161:0/924362321 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5780070710 con 0x7f578015d5e0 2024-05-22T21:18:11.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.180+0000 7f5788600640 1 -- 172.21.15.161:0/924362321 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f5780070710 con 0x7f578015d5e0 2024-05-22T21:18:11.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.460+0000 7f577d600640 1 -- 172.21.15.161:0/924362321 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v251) v1 ==== 133+0+0 (secure 0 0 0) 0x7f5780070710 con 0x7f578015d5e0 2024-05-22T21:18:11.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.460+0000 7f5788600640 1 -- 172.21.15.161:0/924362321 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5760066690 msgr2=0x7f5760068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:11.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.460+0000 7f5788600640 1 --2- 172.21.15.161:0/924362321 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5760066690 0x7f5760068b50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f578014f4f0 tx=0x7f576c009040 comp rx=0 tx=0).stop 2024-05-22T21:18:11.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.460+0000 7f5788600640 1 -- 172.21.15.161:0/924362321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f578015d5e0 msgr2=0x7f578014ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:11.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.460+0000 7f5788600640 1 --2- 172.21.15.161:0/924362321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f578015d5e0 0x7f578014ae30 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f57740ad040 tx=0x7f577401acb0 comp rx=0 tx=0).stop 2024-05-22T21:18:11.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.460+0000 7f5788600640 1 -- 172.21.15.161:0/924362321 shutdown_connections 2024-05-22T21:18:11.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.460+0000 7f5788600640 1 --2- 172.21.15.161:0/924362321 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5760066690 0x7f5760068b50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:11.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.460+0000 7f5788600640 1 --2- 172.21.15.161:0/924362321 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f578015e130 0x7f578014b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:11.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.460+0000 7f5788600640 1 --2- 172.21.15.161:0/924362321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f578015dd50 0x7f578014c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:11.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.460+0000 7f5788600640 1 --2- 172.21.15.161:0/924362321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f578015d5e0 0x7f578014ae30 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:11.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.460+0000 7f5788600640 1 -- 172.21.15.161:0/924362321 >> 172.21.15.161:0/924362321 conn(0x7f57800b0bc0 msgr2=0x7f57800b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:11.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.460+0000 7f5788600640 1 -- 172.21.15.161:0/924362321 shutdown_connections 2024-05-22T21:18:11.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.464+0000 7f5788600640 1 -- 172.21.15.161:0/924362321 wait complete. 2024-05-22T21:18:11.486 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2024-05-22T21:18:11.486 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestClusterFull) 2024-05-22T21:18:11.486 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2024-05-22T21:18:11.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58ef800640 1 Processor -- start 2024-05-22T21:18:11.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58ef800640 1 -- start start 2024-05-22T21:18:11.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58ef800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58e815d4a0 0x7f58e8099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:11.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58ef800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f58e80b0490 0x7f58e8099f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:11.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58ef800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58e806bfb0 0x7f58e8160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:11.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58ef800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e815d880 con 0x7f58e80b0490 2024-05-22T21:18:11.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58ef800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e806b440 con 0x7f58e815d4a0 2024-05-22T21:18:11.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58ef800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e806c440 con 0x7f58e806bfb0 2024-05-22T21:18:11.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58e7e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f58e80b0490 0x7f58e8099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:11.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58e7e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f58e80b0490 0x7f58e8099f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41686/0 (socket says 172.21.15.161:41686) 2024-05-22T21:18:11.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58e7e00640 1 -- 172.21.15.161:0/2963787600 learned_addr learned my addr 172.21.15.161:0/2963787600 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:11.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58ed800640 1 --2- 172.21.15.161:0/2963787600 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58e806bfb0 0x7f58e8160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:11.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58ece00640 1 --2- 172.21.15.161:0/2963787600 >> v2:172.21.15.179:3300/0 conn(0x7f58e815d4a0 0x7f58e8099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:11.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58e7e00640 1 -- 172.21.15.161:0/2963787600 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58e806bfb0 msgr2=0x7f58e8160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:11.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58e7e00640 1 --2- 172.21.15.161:0/2963787600 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58e806bfb0 0x7f58e8160ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:11.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58e7e00640 1 -- 172.21.15.161:0/2963787600 >> v2:172.21.15.179:3300/0 conn(0x7f58e815d4a0 msgr2=0x7f58e8099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:11.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58e7e00640 1 --2- 172.21.15.161:0/2963787600 >> v2:172.21.15.179:3300/0 conn(0x7f58e815d4a0 0x7f58e8099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:11.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58e7e00640 1 -- 172.21.15.161:0/2963787600 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e80cf190 con 0x7f58e80b0490 2024-05-22T21:18:11.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58ed800640 1 --2- 172.21.15.161:0/2963787600 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58e806bfb0 0x7f58e8160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:11.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.620+0000 7f58ece00640 1 --2- 172.21.15.161:0/2963787600 >> v2:172.21.15.179:3300/0 conn(0x7f58e815d4a0 0x7f58e8099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:11.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.624+0000 7f58e7e00640 1 --2- 172.21.15.161:0/2963787600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e80b0490 0x7f58e8099f80 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f58e80708c0 tx=0x7f58d401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc09171441ae56 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:11.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.624+0000 7f58e7400640 1 -- 172.21.15.161:0/2963787600 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d40aa610 con 0x7f58e80b0490 2024-05-22T21:18:11.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.624+0000 7f58e7400640 1 -- 172.21.15.161:0/2963787600 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d40a6040 con 0x7f58e80b0490 2024-05-22T21:18:11.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.624+0000 7f58e7400640 1 -- 172.21.15.161:0/2963787600 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d40aa990 con 0x7f58e80b0490 2024-05-22T21:18:11.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.624+0000 7f58ef800640 1 -- 172.21.15.161:0/2963787600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e80b0490 msgr2=0x7f58e8099f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:11.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.624+0000 7f58ef800640 1 --2- 172.21.15.161:0/2963787600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e80b0490 0x7f58e8099f80 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f58e80708c0 tx=0x7f58d401c4f0 comp rx=0 tx=0).stop 2024-05-22T21:18:11.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.624+0000 7f58ef800640 1 -- 172.21.15.161:0/2963787600 shutdown_connections 2024-05-22T21:18:11.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.624+0000 7f58ef800640 1 --2- 172.21.15.161:0/2963787600 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58e806bfb0 0x7f58e8160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:11.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.624+0000 7f58ef800640 1 --2- 172.21.15.161:0/2963787600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e80b0490 0x7f58e8099f80 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:11.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.624+0000 7f58ef800640 1 --2- 172.21.15.161:0/2963787600 >> v2:172.21.15.179:3300/0 conn(0x7f58e815d4a0 0x7f58e8099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:11.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.624+0000 7f58ef800640 1 -- 172.21.15.161:0/2963787600 >> 172.21.15.161:0/2963787600 conn(0x7f58e80b2a40 msgr2=0x7f58e80b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:11.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.624+0000 7f58ef800640 1 -- 172.21.15.161:0/2963787600 shutdown_connections 2024-05-22T21:18:11.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.624+0000 7f58ef800640 1 -- 172.21.15.161:0/2963787600 wait complete. 2024-05-22T21:18:11.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ef800640 1 Processor -- start 2024-05-22T21:18:11.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ef800640 1 -- start start 2024-05-22T21:18:11.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ef800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e80b0490 0x7f58e8155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:11.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ef800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58e815d4a0 0x7f58e814ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:11.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ef800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e8159eb0 0x7f58e815a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:11.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ef800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e815a8a0 con 0x7f58e80b0490 2024-05-22T21:18:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ef800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e81512a0 con 0x7f58e8159eb0 2024-05-22T21:18:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ef800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e8151580 con 0x7f58e815d4a0 2024-05-22T21:18:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ece00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e80b0490 0x7f58e8155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ece00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e80b0490 0x7f58e8155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41688/0 (socket says 172.21.15.161:41688) 2024-05-22T21:18:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ece00640 1 -- 172.21.15.161:0/271456990 learned_addr learned my addr 172.21.15.161:0/271456990 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58e7e00640 1 --2- 172.21.15.161:0/271456990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58e815d4a0 0x7f58e814ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ed800640 1 --2- 172.21.15.161:0/271456990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e8159eb0 0x7f58e815a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ece00640 1 -- 172.21.15.161:0/271456990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58e815d4a0 msgr2=0x7f58e814ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ece00640 1 --2- 172.21.15.161:0/271456990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58e815d4a0 0x7f58e814ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ece00640 1 -- 172.21.15.161:0/271456990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e8159eb0 msgr2=0x7f58e815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ece00640 1 --2- 172.21.15.161:0/271456990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e8159eb0 0x7f58e815a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ece00640 1 -- 172.21.15.161:0/271456990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d40094d0 con 0x7f58e80b0490 2024-05-22T21:18:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58e7e00640 1 --2- 172.21.15.161:0/271456990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58e815d4a0 0x7f58e814ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:11.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ece00640 1 --2- 172.21.15.161:0/271456990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e80b0490 0x7f58e8155450 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f58d800c7f0 tx=0x7f58d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:11.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58e4c00640 1 -- 172.21.15.161:0/271456990 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d8007820 con 0x7f58e80b0490 2024-05-22T21:18:11.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58e4c00640 1 -- 172.21.15.161:0/271456990 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d8007980 con 0x7f58e80b0490 2024-05-22T21:18:11.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58e4c00640 1 -- 172.21.15.161:0/271456990 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d8007d30 con 0x7f58e80b0490 2024-05-22T21:18:11.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ef800640 1 -- 172.21.15.161:0/271456990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e80c98f0 con 0x7f58e80b0490 2024-05-22T21:18:11.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ef800640 1 -- 172.21.15.161:0/271456990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e8151860 con 0x7f58e80b0490 2024-05-22T21:18:11.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.628+0000 7f58ef800640 1 -- 172.21.15.161:0/271456990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e80c9750 con 0x7f58e80b0490 2024-05-22T21:18:11.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.632+0000 7f58e4c00640 1 -- 172.21.15.161:0/271456990 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f58d80113f0 con 0x7f58e80b0490 2024-05-22T21:18:11.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.632+0000 7f58e4c00640 1 --2- 172.21.15.161:0/271456990 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f58c8066690 0x7f58c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:11.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.632+0000 7f58e4c00640 1 -- 172.21.15.161:0/271456990 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f58d8084190 con 0x7f58e80b0490 2024-05-22T21:18:11.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.632+0000 7f58e7e00640 1 --2- 172.21.15.161:0/271456990 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f58c8066690 0x7f58c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:11.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.632+0000 7f58e7e00640 1 --2- 172.21.15.161:0/271456990 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f58c8066690 0x7f58c8068b50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f58d40ae000 tx=0x7f58d40a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:11.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.636+0000 7f58e4c00640 1 -- 172.21.15.161:0/271456990 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f58e80c9750 con 0x7f58e80b0490 2024-05-22T21:18:11.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:11.848+0000 7f58ef800640 1 -- 172.21.15.161:0/271456990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7f58e80c9750 con 0x7f58e80b0490 2024-05-22T21:18:12.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.464+0000 7f58e4c00640 1 -- 172.21.15.161:0/271456990 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v252) v1 ==== 140+0+0 (secure 0 0 0) 0x7f58e80c9750 con 0x7f58e80b0490 2024-05-22T21:18:12.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.468+0000 7f58ef800640 1 -- 172.21.15.161:0/271456990 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f58c8066690 msgr2=0x7f58c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:12.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.468+0000 7f58ef800640 1 --2- 172.21.15.161:0/271456990 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f58c8066690 0x7f58c8068b50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f58d40ae000 tx=0x7f58d40a7000 comp rx=0 tx=0).stop 2024-05-22T21:18:12.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.468+0000 7f58ef800640 1 -- 172.21.15.161:0/271456990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e80b0490 msgr2=0x7f58e8155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:12.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.468+0000 7f58ef800640 1 --2- 172.21.15.161:0/271456990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e80b0490 0x7f58e8155450 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f58d800c7f0 tx=0x7f58d800ccb0 comp rx=0 tx=0).stop 2024-05-22T21:18:12.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.468+0000 7f58ef800640 1 -- 172.21.15.161:0/271456990 shutdown_connections 2024-05-22T21:18:12.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.468+0000 7f58ef800640 1 --2- 172.21.15.161:0/271456990 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f58c8066690 0x7f58c8068b50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:12.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.468+0000 7f58ef800640 1 --2- 172.21.15.161:0/271456990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e8159eb0 0x7f58e815a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:12.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.468+0000 7f58ef800640 1 --2- 172.21.15.161:0/271456990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58e815d4a0 0x7f58e814ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:12.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.468+0000 7f58ef800640 1 --2- 172.21.15.161:0/271456990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58e80b0490 0x7f58e8155450 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:12.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.468+0000 7f58ef800640 1 -- 172.21.15.161:0/271456990 >> 172.21.15.161:0/271456990 conn(0x7f58e80b2a40 msgr2=0x7f58e80cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:12.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.468+0000 7f58ef800640 1 -- 172.21.15.161:0/271456990 shutdown_connections 2024-05-22T21:18:12.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.468+0000 7f58ef800640 1 -- 172.21.15.161:0/271456990 wait complete. 2024-05-22T21:18:12.491 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-05-22T21:18:12.646 INFO:teuthology.orchestra.run.smithi161.stdout:{"osd_objectstore":"memstore"} 2024-05-22T21:18:12.646 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-05-22T21:18:12.646 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-22T21:18:12.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.784+0000 7f55b6000640 1 Processor -- start 2024-05-22T21:18:12.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.784+0000 7f55b6000640 1 -- start start 2024-05-22T21:18:12.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.784+0000 7f55b6000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f55b015b600 0x7f55b006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:12.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.784+0000 7f55b6000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f55b006c7e0 0x7f55b00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:12.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.784+0000 7f55b6000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b006cc40 con 0x7f55b0143c40 2024-05-22T21:18:12.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.784+0000 7f55b6000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b015ba60 con 0x7f55b006c7e0 2024-05-22T21:18:12.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.784+0000 7f55b6000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b0144bb0 con 0x7f55b015b600 2024-05-22T21:18:12.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.784+0000 7f55afe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f55b006c7e0 0x7f55b00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:12.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.784+0000 7f55afe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f55b006c7e0 0x7f55b00b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41702/0 (socket says 172.21.15.161:41702) 2024-05-22T21:18:12.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.784+0000 7f55afe00640 1 -- 172.21.15.161:0/1787332010 learned_addr learned my addr 172.21.15.161:0/1787332010 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:12.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.784+0000 7f55afe00640 1 -- 172.21.15.161:0/1787332010 >> v2:172.21.15.179:3300/0 conn(0x7f55b015b600 msgr2=0x7f55b006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:12.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.784+0000 7f55a6a00640 1 --2- 172.21.15.161:0/1787332010 >> v2:172.21.15.179:3300/0 conn(0x7f55b015b600 0x7f55b006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:12.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.784+0000 7f55afe00640 1 --2- 172.21.15.161:0/1787332010 >> v2:172.21.15.179:3300/0 conn(0x7f55b015b600 0x7f55b006c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:12.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.784+0000 7f55afe00640 1 -- 172.21.15.161:0/1787332010 >> v1:172.21.15.179:6789/0 conn(0x7f55b0143c40 legacy=0x7f55b01444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:12.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.784+0000 7f55afe00640 1 -- 172.21.15.161:0/1787332010 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b015efb0 con 0x7f55b006c7e0 2024-05-22T21:18:12.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.784+0000 7f55a6a00640 1 --2- 172.21.15.161:0/1787332010 >> v2:172.21.15.179:3300/0 conn(0x7f55b015b600 0x7f55b006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:12.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.788+0000 7f55afe00640 1 --2- 172.21.15.161:0/1787332010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b006c7e0 0x7f55b00b2160 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f55b00b1c80 tx=0x7f55a801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e355dd24a71a00d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:12.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.788+0000 7f55aea00640 1 -- 172.21.15.161:0/1787332010 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a80aa640 con 0x7f55b006c7e0 2024-05-22T21:18:12.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.788+0000 7f55aea00640 1 -- 172.21.15.161:0/1787332010 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a80aa7a0 con 0x7f55b006c7e0 2024-05-22T21:18:12.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.788+0000 7f55aea00640 1 -- 172.21.15.161:0/1787332010 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a80aab30 con 0x7f55b006c7e0 2024-05-22T21:18:12.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.788+0000 7f55b6000640 1 -- 172.21.15.161:0/1787332010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b006c7e0 msgr2=0x7f55b00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:12.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.788+0000 7f55b6000640 1 --2- 172.21.15.161:0/1787332010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b006c7e0 0x7f55b00b2160 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f55b00b1c80 tx=0x7f55a801c4e0 comp rx=0 tx=0).stop 2024-05-22T21:18:12.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.788+0000 7f55b6000640 1 -- 172.21.15.161:0/1787332010 shutdown_connections 2024-05-22T21:18:12.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.788+0000 7f55b6000640 1 --2- 172.21.15.161:0/1787332010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b006c7e0 0x7f55b00b2160 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:12.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.788+0000 7f55b6000640 1 --2- 172.21.15.161:0/1787332010 >> v2:172.21.15.179:3300/0 conn(0x7f55b015b600 0x7f55b006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:12.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.788+0000 7f55b6000640 1 -- 172.21.15.161:0/1787332010 >> 172.21.15.161:0/1787332010 conn(0x7f55b006a2d0 msgr2=0x7f55b006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:12.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.788+0000 7f55b6000640 1 -- 172.21.15.161:0/1787332010 shutdown_connections 2024-05-22T21:18:12.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.788+0000 7f55b6000640 1 -- 172.21.15.161:0/1787332010 wait complete. 2024-05-22T21:18:12.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.788+0000 7f55b6000640 1 Processor -- start 2024-05-22T21:18:12.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.788+0000 7f55b6000640 1 -- start start 2024-05-22T21:18:12.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55b6000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b006c7e0 0x7f55b015a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55b6000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55b0143c40 0x7f55b0157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55b6000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f55b015b600 0x7f55b01569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55b6000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b0157d40 con 0x7f55b006c7e0 2024-05-22T21:18:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55b6000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b00ac8b0 con 0x7f55b0143c40 2024-05-22T21:18:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55b6000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b015a7f0 con 0x7f55b015b600 2024-05-22T21:18:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55af400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b006c7e0 0x7f55b015a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55af400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b006c7e0 0x7f55b015a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41718/0 (socket says 172.21.15.161:41718) 2024-05-22T21:18:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55af400640 1 -- 172.21.15.161:0/2139557498 learned_addr learned my addr 172.21.15.161:0/2139557498 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55a6a00640 1 --2- 172.21.15.161:0/2139557498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55b0143c40 0x7f55b0157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55afe00640 1 --2- 172.21.15.161:0/2139557498 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f55b015b600 0x7f55b01569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:12.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55af400640 1 -- 172.21.15.161:0/2139557498 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f55b015b600 msgr2=0x7f55b01569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55af400640 1 --2- 172.21.15.161:0/2139557498 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f55b015b600 0x7f55b01569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55af400640 1 -- 172.21.15.161:0/2139557498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55b0143c40 msgr2=0x7f55b0157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55af400640 1 --2- 172.21.15.161:0/2139557498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55b0143c40 0x7f55b0157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55af400640 1 -- 172.21.15.161:0/2139557498 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a80094d0 con 0x7f55b006c7e0 2024-05-22T21:18:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55a6a00640 1 --2- 172.21.15.161:0/2139557498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55b0143c40 0x7f55b0157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55af400640 1 --2- 172.21.15.161:0/2139557498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b006c7e0 0x7f55b015a1f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f55a000c7c0 tx=0x7f55a000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55a7e00640 1 -- 172.21.15.161:0/2139557498 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a000aa50 con 0x7f55b006c7e0 2024-05-22T21:18:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55a7e00640 1 -- 172.21.15.161:0/2139557498 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a000abb0 con 0x7f55b006c7e0 2024-05-22T21:18:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55a7e00640 1 -- 172.21.15.161:0/2139557498 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a0012420 con 0x7f55b006c7e0 2024-05-22T21:18:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55b6000640 1 -- 172.21.15.161:0/2139557498 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b015aad0 con 0x7f55b006c7e0 2024-05-22T21:18:12.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55b6000640 1 -- 172.21.15.161:0/2139557498 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b015b090 con 0x7f55b006c7e0 2024-05-22T21:18:12.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55b6000640 1 -- 172.21.15.161:0/2139557498 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b0067220 con 0x7f55b006c7e0 2024-05-22T21:18:12.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.792+0000 7f55a7e00640 1 -- 172.21.15.161:0/2139557498 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f55a0012580 con 0x7f55b006c7e0 2024-05-22T21:18:12.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.796+0000 7f55a7e00640 1 --2- 172.21.15.161:0/2139557498 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f558c0666c0 0x7f558c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:12.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.796+0000 7f55a7e00640 1 -- 172.21.15.161:0/2139557498 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f55a0084060 con 0x7f55b006c7e0 2024-05-22T21:18:12.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.796+0000 7f55a6a00640 1 --2- 172.21.15.161:0/2139557498 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f558c0666c0 0x7f558c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:12.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.796+0000 7f55a6a00640 1 --2- 172.21.15.161:0/2139557498 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f558c0666c0 0x7f558c068b80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5598006fd0 tx=0x7f5598008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:12.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.800+0000 7f55a7e00640 1 -- 172.21.15.161:0/2139557498 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f55a00b5cf0 con 0x7f55b006c7e0 2024-05-22T21:18:12.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.952+0000 7f55b6000640 1 -- 172.21.15.161:0/2139557498 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f55b00ceaa0 con 0x7f55b006c7e0 2024-05-22T21:18:12.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.956+0000 7f55a7e00640 1 -- 172.21.15.161:0/2139557498 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f55a00160c0 con 0x7f55b006c7e0 2024-05-22T21:18:12.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.960+0000 7f55b6000640 1 -- 172.21.15.161:0/2139557498 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f558c0666c0 msgr2=0x7f558c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:12.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.960+0000 7f55b6000640 1 --2- 172.21.15.161:0/2139557498 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f558c0666c0 0x7f558c068b80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5598006fd0 tx=0x7f5598008040 comp rx=0 tx=0).stop 2024-05-22T21:18:12.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.960+0000 7f55b6000640 1 -- 172.21.15.161:0/2139557498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b006c7e0 msgr2=0x7f55b015a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:12.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.960+0000 7f55b6000640 1 --2- 172.21.15.161:0/2139557498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b006c7e0 0x7f55b015a1f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f55a000c7c0 tx=0x7f55a000cc80 comp rx=0 tx=0).stop 2024-05-22T21:18:12.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.960+0000 7f55b6000640 1 -- 172.21.15.161:0/2139557498 shutdown_connections 2024-05-22T21:18:12.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.960+0000 7f55b6000640 1 --2- 172.21.15.161:0/2139557498 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f558c0666c0 0x7f558c068b80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:12.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.960+0000 7f55b6000640 1 --2- 172.21.15.161:0/2139557498 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f55b015b600 0x7f55b01569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:12.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.960+0000 7f55b6000640 1 --2- 172.21.15.161:0/2139557498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55b0143c40 0x7f55b0157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:12.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.960+0000 7f55b6000640 1 --2- 172.21.15.161:0/2139557498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b006c7e0 0x7f55b015a1f0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:12.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.960+0000 7f55b6000640 1 -- 172.21.15.161:0/2139557498 >> 172.21.15.161:0/2139557498 conn(0x7f55b006a2d0 msgr2=0x7f55b0154dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:12.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.960+0000 7f55b6000640 1 -- 172.21.15.161:0/2139557498 shutdown_connections 2024-05-22T21:18:12.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:12.960+0000 7f55b6000640 1 -- 172.21.15.161:0/2139557498 wait complete. 2024-05-22T21:18:12.981 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:18:12.981 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:18:13.032 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:18:13.093 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:18:13.093 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:18:13.093 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:13.101 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:18:13.102 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:18:13.102 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:13.111 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:18:13.111 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:18:13.111 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:13.152 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:18:13.152 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:18:13.152 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:13.162 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:18:13.162 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-22T21:18:13.162 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:18:13.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.328+0000 7f65c2c00640 1 Processor -- start 2024-05-22T21:18:13.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.328+0000 7f65c2c00640 1 -- start start 2024-05-22T21:18:13.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.328+0000 7f65c2c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65bc15dd50 0x7f65bc06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:13.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.328+0000 7f65c2c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f65bc15e130 0x7f65bc15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:13.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.328+0000 7f65c2c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f65bc15d410 0x7f65bc15a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:13.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.328+0000 7f65c2c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65bc071670 con 0x7f65bc15d410 2024-05-22T21:18:13.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.328+0000 7f65c2c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65bc15d8a0 con 0x7f65bc15e130 2024-05-22T21:18:13.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.328+0000 7f65c2c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc06de40 con 0x7f65bc15dd50 2024-05-22T21:18:13.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.328+0000 7f65c0c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f65bc15d410 0x7f65bc15a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:13.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.328+0000 7f65c0c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f65bc15d410 0x7f65bc15a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41722/0 (socket says 172.21.15.161:41722) 2024-05-22T21:18:13.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.328+0000 7f65c0c00640 1 -- 172.21.15.161:0/1513700183 learned_addr learned my addr 172.21.15.161:0/1513700183 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:13.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.328+0000 7f65bbe00640 1 --2- 172.21.15.161:0/1513700183 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65bc15dd50 0x7f65bc06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:13.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.328+0000 7f65b3400640 1 --2- 172.21.15.161:0/1513700183 >> v2:172.21.15.179:3300/0 conn(0x7f65bc15e130 0x7f65bc15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:13.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65c0c00640 1 -- 172.21.15.161:0/1513700183 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65bc15dd50 msgr2=0x7f65bc06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:13.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65c0c00640 1 --2- 172.21.15.161:0/1513700183 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65bc15dd50 0x7f65bc06d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65c0c00640 1 -- 172.21.15.161:0/1513700183 >> v2:172.21.15.179:3300/0 conn(0x7f65bc15e130 msgr2=0x7f65bc15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:13.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65c0c00640 1 --2- 172.21.15.161:0/1513700183 >> v2:172.21.15.179:3300/0 conn(0x7f65bc15e130 0x7f65bc15ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65c0c00640 1 -- 172.21.15.161:0/1513700183 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc070a60 con 0x7f65bc15d410 2024-05-22T21:18:13.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65bbe00640 1 --2- 172.21.15.161:0/1513700183 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65bc15dd50 0x7f65bc06d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:13.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65b3400640 1 --2- 172.21.15.161:0/1513700183 >> v2:172.21.15.179:3300/0 conn(0x7f65bc15e130 0x7f65bc15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:13.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65c0c00640 1 --2- 172.21.15.161:0/1513700183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65bc15d410 0x7f65bc15a500 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f65bc072510 tx=0x7f65b401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a64930af8527516 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:13.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65bb400640 1 -- 172.21.15.161:0/1513700183 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b40a57b0 con 0x7f65bc15d410 2024-05-22T21:18:13.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65bb400640 1 -- 172.21.15.161:0/1513700183 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b40a5910 con 0x7f65bc15d410 2024-05-22T21:18:13.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65bb400640 1 -- 172.21.15.161:0/1513700183 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b40a5ca0 con 0x7f65bc15d410 2024-05-22T21:18:13.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65c2c00640 1 -- 172.21.15.161:0/1513700183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65bc15d410 msgr2=0x7f65bc15a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:13.337 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65c2c00640 1 --2- 172.21.15.161:0/1513700183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65bc15d410 0x7f65bc15a500 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f65bc072510 tx=0x7f65b401c930 comp rx=0 tx=0).stop 2024-05-22T21:18:13.337 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65c2c00640 1 -- 172.21.15.161:0/1513700183 shutdown_connections 2024-05-22T21:18:13.337 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65c2c00640 1 --2- 172.21.15.161:0/1513700183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65bc15d410 0x7f65bc15a500 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.337 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65c2c00640 1 --2- 172.21.15.161:0/1513700183 >> v2:172.21.15.179:3300/0 conn(0x7f65bc15e130 0x7f65bc15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.337 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65c2c00640 1 --2- 172.21.15.161:0/1513700183 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65bc15dd50 0x7f65bc06d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.337 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65c2c00640 1 -- 172.21.15.161:0/1513700183 >> 172.21.15.161:0/1513700183 conn(0x7f65bc0b0bc0 msgr2=0x7f65bc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:13.337 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.332+0000 7f65c2c00640 1 -- 172.21.15.161:0/1513700183 shutdown_connections 2024-05-22T21:18:13.337 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c2c00640 1 -- 172.21.15.161:0/1513700183 wait complete. 2024-05-22T21:18:13.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c2c00640 1 Processor -- start 2024-05-22T21:18:13.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c2c00640 1 -- start start 2024-05-22T21:18:13.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c2c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65bc15d410 0x7f65bc140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:13.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c2c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65bc15dd50 0x7f65bc157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:13.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c2c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65bc15e130 0x7f65bc1582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:13.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c2c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc158950 con 0x7f65bc15e130 2024-05-22T21:18:13.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c2c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc158c30 con 0x7f65bc15d410 2024-05-22T21:18:13.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c2c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc158f10 con 0x7f65bc15dd50 2024-05-22T21:18:13.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c0c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65bc15e130 0x7f65bc1582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:13.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c0c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65bc15e130 0x7f65bc1582d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41732/0 (socket says 172.21.15.161:41732) 2024-05-22T21:18:13.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c0c00640 1 -- 172.21.15.161:0/414804726 learned_addr learned my addr 172.21.15.161:0/414804726 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:13.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65bbe00640 1 --2- 172.21.15.161:0/414804726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65bc15d410 0x7f65bc140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:13.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65b3400640 1 --2- 172.21.15.161:0/414804726 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65bc15dd50 0x7f65bc157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:13.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c0c00640 1 -- 172.21.15.161:0/414804726 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65bc15dd50 msgr2=0x7f65bc157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:13.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c0c00640 1 --2- 172.21.15.161:0/414804726 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65bc15dd50 0x7f65bc157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c0c00640 1 -- 172.21.15.161:0/414804726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65bc15d410 msgr2=0x7f65bc140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:13.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c0c00640 1 --2- 172.21.15.161:0/414804726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65bc15d410 0x7f65bc140940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c0c00640 1 -- 172.21.15.161:0/414804726 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b4009b30 con 0x7f65bc15e130 2024-05-22T21:18:13.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.336+0000 7f65c0c00640 1 --2- 172.21.15.161:0/414804726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65bc15e130 0x7f65bc1582d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f65b40af040 tx=0x7f65b40196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:13.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.340+0000 7f65b8c00640 1 -- 172.21.15.161:0/414804726 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b40b8670 con 0x7f65bc15e130 2024-05-22T21:18:13.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.340+0000 7f65b8c00640 1 -- 172.21.15.161:0/414804726 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b40b87d0 con 0x7f65bc15e130 2024-05-22T21:18:13.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.340+0000 7f65c2c00640 1 -- 172.21.15.161:0/414804726 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65bc1591f0 con 0x7f65bc15e130 2024-05-22T21:18:13.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.340+0000 7f65b8c00640 1 -- 172.21.15.161:0/414804726 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b40b8b80 con 0x7f65bc15e130 2024-05-22T21:18:13.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.340+0000 7f65c2c00640 1 -- 172.21.15.161:0/414804726 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65bc0b2ef0 con 0x7f65bc15e130 2024-05-22T21:18:13.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.340+0000 7f65c2c00640 1 -- 172.21.15.161:0/414804726 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65bc1550a0 con 0x7f65bc15e130 2024-05-22T21:18:13.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.340+0000 7f65b8c00640 1 -- 172.21.15.161:0/414804726 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f65b40b8ce0 con 0x7f65bc15e130 2024-05-22T21:18:13.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.340+0000 7f65b8c00640 1 --2- 172.21.15.161:0/414804726 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65980666c0 0x7f6598068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:13.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.340+0000 7f65b8c00640 1 -- 172.21.15.161:0/414804726 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f65b411f430 con 0x7f65bc15e130 2024-05-22T21:18:13.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.340+0000 7f65bbe00640 1 --2- 172.21.15.161:0/414804726 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65980666c0 0x7f6598068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:13.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.340+0000 7f65bbe00640 1 --2- 172.21.15.161:0/414804726 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65980666c0 0x7f6598068b80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f65ac006e90 tx=0x7f65ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:13.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.344+0000 7f65b8c00640 1 -- 172.21.15.161:0/414804726 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f65bc1550a0 con 0x7f65bc15e130 2024-05-22T21:18:13.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.532+0000 7f65c2c00640 1 -- 172.21.15.161:0/414804726 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65bc1550a0 con 0x7f65bc15e130 2024-05-22T21:18:13.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.532+0000 7f65b8c00640 1 -- 172.21.15.161:0/414804726 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f65bc1550a0 con 0x7f65bc15e130 2024-05-22T21:18:13.536 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:13.536 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":41,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5567,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196093187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196093187},{"type":"v1","addr":"172.21.15.161:6836","nonce":196093187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2079803867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2079803867},{"type":"v1","addr":"172.21.15.179:6835","nonce":2079803867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6136,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/4029369301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4029369301},{"type":"v1","addr":"172.21.15.161:6837","nonce":4029369301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6139,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3212560487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3212560487},{"type":"v1","addr":"172.21.15.179:6834","nonce":3212560487}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[]} 2024-05-22T21:18:13.536 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 41 2024-05-22T21:18:13.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.536+0000 7f65c2c00640 1 -- 172.21.15.161:0/414804726 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65980666c0 msgr2=0x7f6598068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:13.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.536+0000 7f65c2c00640 1 --2- 172.21.15.161:0/414804726 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65980666c0 0x7f6598068b80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f65ac006e90 tx=0x7f65ac008040 comp rx=0 tx=0).stop 2024-05-22T21:18:13.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.536+0000 7f65c2c00640 1 -- 172.21.15.161:0/414804726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65bc15e130 msgr2=0x7f65bc1582d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:13.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.536+0000 7f65c2c00640 1 --2- 172.21.15.161:0/414804726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65bc15e130 0x7f65bc1582d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f65b40af040 tx=0x7f65b40196d0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.536+0000 7f65c2c00640 1 -- 172.21.15.161:0/414804726 shutdown_connections 2024-05-22T21:18:13.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.536+0000 7f65c2c00640 1 --2- 172.21.15.161:0/414804726 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65980666c0 0x7f6598068b80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.536+0000 7f65c2c00640 1 --2- 172.21.15.161:0/414804726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65bc15e130 0x7f65bc1582d0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.536+0000 7f65c2c00640 1 --2- 172.21.15.161:0/414804726 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65bc15dd50 0x7f65bc157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.536+0000 7f65c2c00640 1 --2- 172.21.15.161:0/414804726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65bc15d410 0x7f65bc140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.536+0000 7f65c2c00640 1 -- 172.21.15.161:0/414804726 >> 172.21.15.161:0/414804726 conn(0x7f65bc0b0bc0 msgr2=0x7f65bc0ada70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:13.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.536+0000 7f65c2c00640 1 -- 172.21.15.161:0/414804726 shutdown_connections 2024-05-22T21:18:13.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.536+0000 7f65c2c00640 1 -- 172.21.15.161:0/414804726 wait complete. 2024-05-22T21:18:13.557 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-22T21:18:13.558 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-22T21:18:13.558 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-22T21:18:13.558 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:18:13.558 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-22T21:18:13.558 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-22T21:18:13.558 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-22T21:18:13.558 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:18:13.559 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-22T21:18:13.559 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-22T21:18:13.559 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-22T21:18:13.559 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:18:13.559 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-22T21:18:13.559 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-22T21:18:13.559 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-22T21:18:13.559 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:18:13.570 INFO:tasks.ceph.mds.a:Stopped 2024-05-22T21:18:13.570 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-22T21:18:13.571 INFO:tasks.ceph.mds.c:Stopped 2024-05-22T21:18:13.571 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-22T21:18:13.572 INFO:tasks.ceph.mds.b:Stopped 2024-05-22T21:18:13.573 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-22T21:18:13.573 INFO:tasks.ceph.mds.d:Stopped 2024-05-22T21:18:13.573 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-22T21:18:13.614 INFO:tasks.ceph.mds.a:Started 2024-05-22T21:18:13.615 DEBUG:teuthology.parallel:result is None 2024-05-22T21:18:13.616 INFO:tasks.ceph.mds.b:Started 2024-05-22T21:18:13.616 DEBUG:teuthology.parallel:result is None 2024-05-22T21:18:13.617 INFO:tasks.ceph.mds.c:Started 2024-05-22T21:18:13.617 DEBUG:teuthology.parallel:result is None 2024-05-22T21:18:13.618 INFO:tasks.ceph.mds.d:Started 2024-05-22T21:18:13.618 DEBUG:teuthology.parallel:result is None 2024-05-22T21:18:13.618 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-22T21:18:13.701 INFO:tasks.ceph.mds.a.smithi161.stdout:starting mds.a at 2024-05-22T21:18:13.702 INFO:tasks.ceph.mds.c.smithi161.stdout:starting mds.c at 2024-05-22T21:18:13.704 INFO:tasks.ceph.mds.d.smithi179.stdout:starting mds.d at 2024-05-22T21:18:13.705 INFO:tasks.ceph.mds.b.smithi179.stdout:starting mds.b at 2024-05-22T21:18:13.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.760+0000 7ffaa3200640 1 Processor -- start 2024-05-22T21:18:13.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.760+0000 7ffaa3200640 1 -- start start 2024-05-22T21:18:13.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.764+0000 7ffaa3200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ffa9c1402e0 0x7ffa9c13f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:13.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.764+0000 7ffaa3200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffa9c13fc80 0x7ffa9c06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:13.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.764+0000 7ffaa3200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9c140090 con 0x7ffa9c13cfc0 2024-05-22T21:18:13.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.764+0000 7ffaa3200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9c13d3a0 con 0x7ffa9c1402e0 2024-05-22T21:18:13.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.764+0000 7ffaa3200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9c0cd920 con 0x7ffa9c13fc80 2024-05-22T21:18:13.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.764+0000 7ffa93400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ffa9c1402e0 0x7ffa9c13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:13.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.764+0000 7ffa93400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ffa9c1402e0 0x7ffa9c13f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41770/0 (socket says 172.21.15.161:41770) 2024-05-22T21:18:13.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.764+0000 7ffa93400640 1 -- 172.21.15.161:0/2062742005 learned_addr learned my addr 172.21.15.161:0/2062742005 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:13.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.764+0000 7ffaa1200640 1 --2- 172.21.15.161:0/2062742005 >> v2:172.21.15.179:3300/0 conn(0x7ffa9c13fc80 0x7ffa9c06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:13.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.764+0000 7ffa93400640 1 -- 172.21.15.161:0/2062742005 >> v2:172.21.15.179:3300/0 conn(0x7ffa9c13fc80 msgr2=0x7ffa9c06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:13.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.764+0000 7ffa93400640 1 --2- 172.21.15.161:0/2062742005 >> v2:172.21.15.179:3300/0 conn(0x7ffa9c13fc80 0x7ffa9c06fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.764+0000 7ffa93400640 1 -- 172.21.15.161:0/2062742005 >> v1:172.21.15.179:6789/0 conn(0x7ffa9c13cfc0 legacy=0x7ffa9c15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:13.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.764+0000 7ffa93400640 1 -- 172.21.15.161:0/2062742005 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c0cdc00 con 0x7ffa9c1402e0 2024-05-22T21:18:13.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.768+0000 7ffa93400640 1 --2- 172.21.15.161:0/2062742005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c1402e0 0x7ffa9c13f740 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c06c5e0 tx=0x7ffa8401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfb926712a4093f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:13.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.768+0000 7ffa9b400640 1 -- 172.21.15.161:0/2062742005 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa840ab600 con 0x7ffa9c1402e0 2024-05-22T21:18:13.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.768+0000 7ffa9b400640 1 -- 172.21.15.161:0/2062742005 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa840a7050 con 0x7ffa9c1402e0 2024-05-22T21:18:13.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.768+0000 7ffa9b400640 1 -- 172.21.15.161:0/2062742005 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa840ab980 con 0x7ffa9c1402e0 2024-05-22T21:18:13.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.768+0000 7ffaa3200640 1 -- 172.21.15.161:0/2062742005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c1402e0 msgr2=0x7ffa9c13f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:13.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.768+0000 7ffaa3200640 1 --2- 172.21.15.161:0/2062742005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c1402e0 0x7ffa9c13f740 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c06c5e0 tx=0x7ffa8401c460 comp rx=0 tx=0).stop 2024-05-22T21:18:13.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.768+0000 7ffaa3200640 1 -- 172.21.15.161:0/2062742005 shutdown_connections 2024-05-22T21:18:13.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.768+0000 7ffaa3200640 1 --2- 172.21.15.161:0/2062742005 >> v2:172.21.15.179:3300/0 conn(0x7ffa9c13fc80 0x7ffa9c06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.768+0000 7ffaa3200640 1 --2- 172.21.15.161:0/2062742005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c1402e0 0x7ffa9c13f740 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.768+0000 7ffaa3200640 1 -- 172.21.15.161:0/2062742005 >> 172.21.15.161:0/2062742005 conn(0x7ffa9c06a2d0 msgr2=0x7ffa9c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:13.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.768+0000 7ffaa3200640 1 -- 172.21.15.161:0/2062742005 shutdown_connections 2024-05-22T21:18:13.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.768+0000 7ffaa3200640 1 -- 172.21.15.161:0/2062742005 wait complete. 2024-05-22T21:18:13.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.768+0000 7ffaa3200640 1 Processor -- start 2024-05-22T21:18:13.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.768+0000 7ffaa3200640 1 -- start start 2024-05-22T21:18:13.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffaa3200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffa9c13cfc0 0x7ffa9c0ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:13.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffaa3200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa9c13fc80 0x7ffa9c156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:13.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffaa3200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c1402e0 0x7ffa9c0b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:13.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffaa3200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c13df10 con 0x7ffa9c1402e0 2024-05-22T21:18:13.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffaa3200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c13e1d0 con 0x7ffa9c13fc80 2024-05-22T21:18:13.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffaa3200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c13d610 con 0x7ffa9c13cfc0 2024-05-22T21:18:13.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffaa1200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c1402e0 0x7ffa9c0b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:13.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffa93400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa9c13fc80 0x7ffa9c156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:13.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffa93400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa9c13fc80 0x7ffa9c156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:57212/0 (socket says 172.21.15.161:57212) 2024-05-22T21:18:13.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffaa1200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c1402e0 0x7ffa9c0b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41772/0 (socket says 172.21.15.161:41772) 2024-05-22T21:18:13.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffa93400640 1 -- 172.21.15.161:0/3778759286 learned_addr learned my addr 172.21.15.161:0/3778759286 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:13.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffa9be00640 1 --2- 172.21.15.161:0/3778759286 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffa9c13cfc0 0x7ffa9c0ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:13.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffa93400640 1 -- 172.21.15.161:0/3778759286 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffa9c13cfc0 msgr2=0x7ffa9c0ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:13.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffa93400640 1 --2- 172.21.15.161:0/3778759286 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffa9c13cfc0 0x7ffa9c0ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffa93400640 1 -- 172.21.15.161:0/3778759286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c1402e0 msgr2=0x7ffa9c0b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:13.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffa93400640 1 --2- 172.21.15.161:0/3778759286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c1402e0 0x7ffa9c0b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:13.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffa93400640 1 -- 172.21.15.161:0/3778759286 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa840094d0 con 0x7ffa9c13fc80 2024-05-22T21:18:13.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffaa1200640 1 --2- 172.21.15.161:0/3778759286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c1402e0 0x7ffa9c0b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:13.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffa93400640 1 --2- 172.21.15.161:0/3778759286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa9c13fc80 0x7ffa9c156ea0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ffa84019070 tx=0x7ffa84019200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:13.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffa98c00640 1 -- 172.21.15.161:0/3778759286 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa840b4420 con 0x7ffa9c13fc80 2024-05-22T21:18:13.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffa98c00640 1 -- 172.21.15.161:0/3778759286 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa840b4580 con 0x7ffa9c13fc80 2024-05-22T21:18:13.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffa98c00640 1 -- 172.21.15.161:0/3778759286 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa840b4930 con 0x7ffa9c13fc80 2024-05-22T21:18:13.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffaa3200640 1 -- 172.21.15.161:0/3778759286 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9c13d8f0 con 0x7ffa9c13fc80 2024-05-22T21:18:13.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffaa3200640 1 -- 172.21.15.161:0/3778759286 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9c0ad800 con 0x7ffa9c13fc80 2024-05-22T21:18:13.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffaa3200640 1 -- 172.21.15.161:0/3778759286 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa9c140800 con 0x7ffa9c13fc80 2024-05-22T21:18:13.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.772+0000 7ffa98c00640 1 -- 172.21.15.161:0/3778759286 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffa840b2070 con 0x7ffa9c13fc80 2024-05-22T21:18:13.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.776+0000 7ffa98c00640 1 --2- 172.21.15.161:0/3778759286 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffa78066690 0x7ffa78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:13.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.776+0000 7ffa98c00640 1 -- 172.21.15.161:0/3778759286 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffa8411d8d0 con 0x7ffa9c13fc80 2024-05-22T21:18:13.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.776+0000 7ffa9be00640 1 --2- 172.21.15.161:0/3778759286 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffa78066690 0x7ffa78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:13.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.776+0000 7ffa9be00640 1 --2- 172.21.15.161:0/3778759286 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffa78066690 0x7ffa78068b50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c007fd0 tx=0x7ffa8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:13.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.780+0000 7ffa98c00640 1 -- 172.21.15.161:0/3778759286 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffa9c140800 con 0x7ffa9c13fc80 2024-05-22T21:18:13.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:13.948+0000 7ffaa3200640 1 -- 172.21.15.161:0/3778759286 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ffa9c140800 con 0x7ffa9c13fc80 2024-05-22T21:18:14.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.736+0000 7ffa98c00640 1 -- 172.21.15.161:0/3778759286 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v99) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffa9c140800 con 0x7ffa9c13fc80 2024-05-22T21:18:14.739 INFO:teuthology.orchestra.run.smithi161.stderr: removed all blocklist entries 2024-05-22T21:18:14.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.740+0000 7ffaa3200640 1 -- 172.21.15.161:0/3778759286 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffa78066690 msgr2=0x7ffa78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:14.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.740+0000 7ffaa3200640 1 --2- 172.21.15.161:0/3778759286 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffa78066690 0x7ffa78068b50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c007fd0 tx=0x7ffa8c009040 comp rx=0 tx=0).stop 2024-05-22T21:18:14.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.740+0000 7ffaa3200640 1 -- 172.21.15.161:0/3778759286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa9c13fc80 msgr2=0x7ffa9c156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:14.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.740+0000 7ffaa3200640 1 --2- 172.21.15.161:0/3778759286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa9c13fc80 0x7ffa9c156ea0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ffa84019070 tx=0x7ffa84019200 comp rx=0 tx=0).stop 2024-05-22T21:18:14.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.740+0000 7ffaa3200640 1 -- 172.21.15.161:0/3778759286 shutdown_connections 2024-05-22T21:18:14.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.740+0000 7ffaa3200640 1 --2- 172.21.15.161:0/3778759286 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffa78066690 0x7ffa78068b50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:14.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.740+0000 7ffaa3200640 1 --2- 172.21.15.161:0/3778759286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c1402e0 0x7ffa9c0b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:14.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.740+0000 7ffaa3200640 1 --2- 172.21.15.161:0/3778759286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa9c13fc80 0x7ffa9c156ea0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:14.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.740+0000 7ffaa3200640 1 --2- 172.21.15.161:0/3778759286 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffa9c13cfc0 0x7ffa9c0ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:14.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.740+0000 7ffaa3200640 1 -- 172.21.15.161:0/3778759286 >> 172.21.15.161:0/3778759286 conn(0x7ffa9c06a2d0 msgr2=0x7ffa9c15d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:14.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.740+0000 7ffaa3200640 1 -- 172.21.15.161:0/3778759286 shutdown_connections 2024-05-22T21:18:14.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.740+0000 7ffaa3200640 1 -- 172.21.15.161:0/3778759286 wait complete. 2024-05-22T21:18:14.762 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-22T21:18:14.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.896+0000 7fb5cb800640 1 Processor -- start 2024-05-22T21:18:14.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.896+0000 7fb5cb800640 1 -- start start 2024-05-22T21:18:14.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.896+0000 7fb5cb800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb5c415dd50 0x7fb5c406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:14.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.896+0000 7fb5cb800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c415d5e0 0x7fb5c415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:14.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.896+0000 7fb5cb800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c4071670 con 0x7fb5c415e130 2024-05-22T21:18:14.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5cb800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c406de40 con 0x7fb5c415dd50 2024-05-22T21:18:14.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5cb800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c4070a60 con 0x7fb5c415d5e0 2024-05-22T21:18:14.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb5c415e130 0x7fb5c415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:50720/0 (socket says 172.21.15.161:50720) 2024-05-22T21:18:14.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3e00640 1 -- 172.21.15.161:0/3019464927 learned_addr learned my addr 172.21.15.161:0/3019464927 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:14.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c9800640 1 --2- 172.21.15.161:0/3019464927 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c415d5e0 0x7fb5c415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:14.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c8e00640 1 --2- 172.21.15.161:0/3019464927 >> v2:172.21.15.179:3300/0 conn(0x7fb5c415dd50 0x7fb5c406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:14.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3400640 1 -- 172.21.15.161:0/3019464927 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083800206 0 0) 0x7fb5c4071670 con 0x7fb5c415e130 2024-05-22T21:18:14.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3400640 1 -- 172.21.15.161:0/3019464927 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5a8003530 con 0x7fb5c415e130 2024-05-22T21:18:14.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3400640 1 -- 172.21.15.161:0/3019464927 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4025019614 0 0) 0x7fb5a8003530 con 0x7fb5c415e130 2024-05-22T21:18:14.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3400640 1 -- 172.21.15.161:0/3019464927 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c415d5e0 msgr2=0x7fb5c415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:14.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3400640 1 --2- 172.21.15.161:0/3019464927 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c415d5e0 0x7fb5c415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:14.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3400640 1 -- 172.21.15.161:0/3019464927 >> v2:172.21.15.179:3300/0 conn(0x7fb5c415dd50 msgr2=0x7fb5c406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:14.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3400640 1 --2- 172.21.15.161:0/3019464927 >> v2:172.21.15.179:3300/0 conn(0x7fb5c415dd50 0x7fb5c406d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:14.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3400640 1 -- 172.21.15.161:0/3019464927 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c4070d40 con 0x7fb5c415e130 2024-05-22T21:18:14.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3400640 1 -- 172.21.15.161:0/3019464927 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fb5ac001680 con 0x7fb5c415e130 2024-05-22T21:18:14.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3400640 1 -- 172.21.15.161:0/3019464927 >> v1:172.21.15.161:6789/0 conn(0x7fb5c415e130 legacy=0x7fb5c415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:14.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3400640 1 --2- 172.21.15.161:0/3019464927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5a8003980 0x7fb5a8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:14.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3400640 1 --2- 172.21.15.161:0/3019464927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a80047d0 0x7fb5a8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:14.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3400640 1 --2- 172.21.15.161:0/3019464927 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c415d5e0 0x7fb5a8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:14.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3400640 1 -- 172.21.15.161:0/3019464927 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c406de40 con 0x7fb5a80047d0 2024-05-22T21:18:14.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3400640 1 -- 172.21.15.161:0/3019464927 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c4070a60 con 0x7fb5a8003980 2024-05-22T21:18:14.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3400640 1 -- 172.21.15.161:0/3019464927 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a800b990 con 0x7fb5c415d5e0 2024-05-22T21:18:14.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3e00640 1 --2- 172.21.15.161:0/3019464927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a80047d0 0x7fb5a8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:14.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c8e00640 1 --2- 172.21.15.161:0/3019464927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5a8003980 0x7fb5a8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:14.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c9800640 1 --2- 172.21.15.161:0/3019464927 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c415d5e0 0x7fb5a8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:14.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3e00640 1 -- 172.21.15.161:0/3019464927 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c415d5e0 msgr2=0x7fb5a8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:14.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3e00640 1 --2- 172.21.15.161:0/3019464927 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c415d5e0 0x7fb5a8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:14.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3e00640 1 -- 172.21.15.161:0/3019464927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5a8003980 msgr2=0x7fb5a8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:14.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3e00640 1 --2- 172.21.15.161:0/3019464927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5a8003980 0x7fb5a8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:14.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.900+0000 7fb5c3e00640 1 -- 172.21.15.161:0/3019464927 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a8003530 con 0x7fb5a80047d0 2024-05-22T21:18:14.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.904+0000 7fb5c3e00640 1 --2- 172.21.15.161:0/3019464927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a80047d0 0x7fb5a8004c50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00e480 tx=0x7fb5ac021ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e02bc7b133e0ea3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:14.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.904+0000 7fb5c3400640 1 -- 172.21.15.161:0/3019464927 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac005ae0 con 0x7fb5a80047d0 2024-05-22T21:18:14.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.904+0000 7fb5c3400640 1 -- 172.21.15.161:0/3019464927 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ac005c40 con 0x7fb5a80047d0 2024-05-22T21:18:14.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.904+0000 7fb5c3400640 1 -- 172.21.15.161:0/3019464927 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac007070 con 0x7fb5a80047d0 2024-05-22T21:18:14.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.904+0000 7fb5cb800640 1 -- 172.21.15.161:0/3019464927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a80047d0 msgr2=0x7fb5a8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:14.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.904+0000 7fb5cb800640 1 --2- 172.21.15.161:0/3019464927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a80047d0 0x7fb5a8004c50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00e480 tx=0x7fb5ac021ab0 comp rx=0 tx=0).stop 2024-05-22T21:18:14.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.904+0000 7fb5c9800640 1 -- 172.21.15.161:0/3019464927 reap_dead start 2024-05-22T21:18:14.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.904+0000 7fb5cb800640 1 -- 172.21.15.161:0/3019464927 shutdown_connections 2024-05-22T21:18:14.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.904+0000 7fb5cb800640 1 -- 172.21.15.161:0/3019464927 >> 172.21.15.161:0/3019464927 conn(0x7fb5c40b0bc0 msgr2=0x7fb5c40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:14.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.904+0000 7fb5cb800640 1 -- 172.21.15.161:0/3019464927 shutdown_connections 2024-05-22T21:18:14.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.904+0000 7fb5cb800640 1 -- 172.21.15.161:0/3019464927 wait complete. 2024-05-22T21:18:14.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.904+0000 7fb5cb800640 1 Processor -- start 2024-05-22T21:18:14.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.904+0000 7fb5cb800640 1 -- start start 2024-05-22T21:18:14.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5cb800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a80047d0 0x7fb5c4159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:14.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5cb800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c40ce1c0 0x7fb5c414e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:14.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5cb800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5c4156940 0x7fb5c40b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:14.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5cb800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c41564f0 con 0x7fb5a80047d0 2024-05-22T21:18:14.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5cb800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c4157af0 con 0x7fb5c4156940 2024-05-22T21:18:14.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5cb800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c4157810 con 0x7fb5c40ce1c0 2024-05-22T21:18:14.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5c8e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a80047d0 0x7fb5c4159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:14.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5c8e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a80047d0 0x7fb5c4159a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41780/0 (socket says 172.21.15.161:41780) 2024-05-22T21:18:14.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5c8e00640 1 -- 172.21.15.161:0/3625720456 learned_addr learned my addr 172.21.15.161:0/3625720456 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:14.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5c3e00640 1 --2- 172.21.15.161:0/3625720456 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c40ce1c0 0x7fb5c414e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:14.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5c9800640 1 --2- 172.21.15.161:0/3625720456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5c4156940 0x7fb5c40b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:14.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5c8e00640 1 -- 172.21.15.161:0/3625720456 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c40ce1c0 msgr2=0x7fb5c414e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:14.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5c8e00640 1 --2- 172.21.15.161:0/3625720456 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c40ce1c0 0x7fb5c414e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:14.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5c8e00640 1 -- 172.21.15.161:0/3625720456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5c4156940 msgr2=0x7fb5c40b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:14.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5c8e00640 1 --2- 172.21.15.161:0/3625720456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5c4156940 0x7fb5c40b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:14.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5c8e00640 1 -- 172.21.15.161:0/3625720456 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c4152300 con 0x7fb5a80047d0 2024-05-22T21:18:14.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5c9800640 1 --2- 172.21.15.161:0/3625720456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5c4156940 0x7fb5c40b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:14.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5c8e00640 1 --2- 172.21.15.161:0/3625720456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a80047d0 0x7fb5c4159a40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009f90 tx=0x7fb5b400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:14.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5c0c00640 1 -- 172.21.15.161:0/3625720456 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b400e040 con 0x7fb5a80047d0 2024-05-22T21:18:14.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5c0c00640 1 -- 172.21.15.161:0/3625720456 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b400f040 con 0x7fb5a80047d0 2024-05-22T21:18:14.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5c0c00640 1 -- 172.21.15.161:0/3625720456 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4015870 con 0x7fb5a80047d0 2024-05-22T21:18:14.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5cb800640 1 -- 172.21.15.161:0/3625720456 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c41525e0 con 0x7fb5a80047d0 2024-05-22T21:18:14.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5cb800640 1 -- 172.21.15.161:0/3625720456 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c40b3200 con 0x7fb5a80047d0 2024-05-22T21:18:14.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5cb800640 1 -- 172.21.15.161:0/3625720456 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c4152160 con 0x7fb5a80047d0 2024-05-22T21:18:14.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.908+0000 7fb5c0c00640 1 -- 172.21.15.161:0/3625720456 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb5b40159d0 con 0x7fb5a80047d0 2024-05-22T21:18:14.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.912+0000 7fb5c0c00640 1 --2- 172.21.15.161:0/3625720456 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb5a0066690 0x7fb5a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:14.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.912+0000 7fb5c0c00640 1 -- 172.21.15.161:0/3625720456 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb5b4084c30 con 0x7fb5a80047d0 2024-05-22T21:18:14.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.912+0000 7fb5c3e00640 1 --2- 172.21.15.161:0/3625720456 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb5a0066690 0x7fb5a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:14.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.912+0000 7fb5c3e00640 1 --2- 172.21.15.161:0/3625720456 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb5a0066690 0x7fb5a0068b50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0b2000 tx=0x7fb5ac001a80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:14.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:14.916+0000 7fb5c0c00640 1 -- 172.21.15.161:0/3625720456 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb5c4152160 con 0x7fb5a80047d0 2024-05-22T21:18:15.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.124+0000 7fb5cb800640 1 -- 172.21.15.161:0/3625720456 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb5c4152160 con 0x7fb5a80047d0 2024-05-22T21:18:15.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.124+0000 7fb5c0c00640 1 -- 172.21.15.161:0/3625720456 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb5c4152160 con 0x7fb5a80047d0 2024-05-22T21:18:15.127 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:15.127 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:18:15.128 INFO:teuthology.orchestra.run.smithi161.stdout: "auth_dump": [ 2024-05-22T21:18:15.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.128 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.a", 2024-05-22T21:18:15.128 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m00QTFRAAPSRrpBRY75FlcWf3QxOOfg==", 2024-05-22T21:18:15.128 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.128 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:18:15.128 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:18:15.128 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:18:15.128 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:18:15.128 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.129 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.129 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.b", 2024-05-22T21:18:15.129 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m/ipAIRAA4gj5jGqTpNCr1K2OU/bsAQ==", 2024-05-22T21:18:15.129 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.129 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:18:15.129 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:18:15.129 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:18:15.129 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:18:15.129 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.130 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.c", 2024-05-22T21:18:15.130 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mWlM2HBAAEmJJWMDlTG9E7Qzld2gd5w==", 2024-05-22T21:18:15.130 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.130 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:18:15.130 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:18:15.130 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:18:15.130 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:18:15.130 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.131 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.d", 2024-05-22T21:18:15.131 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mmsuaKBAAqpCYRZZUGK5ZuxflUBcPgg==", 2024-05-22T21:18:15.131 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.131 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:18:15.131 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:18:15.131 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:18:15.131 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:18:15.131 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.132 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.0", 2024-05-22T21:18:15.132 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05mJZokLhAAh2vbQnRX25iYUIbvVmURuQ==", 2024-05-22T21:18:15.132 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.132 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:18:15.132 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:18:15.132 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:18:15.132 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.132 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.1", 2024-05-22T21:18:15.132 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05m8kWNNhAA6aaVT2xshFkqnk4G4gOttA==", 2024-05-22T21:18:15.133 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.133 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:18:15.133 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:18:15.133 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:18:15.133 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.133 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.2", 2024-05-22T21:18:15.133 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05ml1k9AxAAtL3XaJJI6cilpmvurh18Pw==", 2024-05-22T21:18:15.133 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.133 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:18:15.134 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:18:15.134 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:18:15.134 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.134 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.3", 2024-05-22T21:18:15.134 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05mbNBgCxAAas1L+QR4Aez+H1G0EVltfg==", 2024-05-22T21:18:15.134 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.134 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:18:15.135 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:18:15.135 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:18:15.135 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.135 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.135 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.4", 2024-05-22T21:18:15.135 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mOPLPABAAZBGIMF/FasNB27n/7di0AA==", 2024-05-22T21:18:15.135 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.135 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:18:15.135 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:18:15.135 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:18:15.136 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.136 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.5", 2024-05-22T21:18:15.136 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mT1poCRAASXm4tdP01pA1VOviqeoLVw==", 2024-05-22T21:18:15.136 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.136 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:18:15.136 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:18:15.136 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:18:15.136 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.137 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.6", 2024-05-22T21:18:15.137 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05m1ZPRERAA2S7WeTwysLCZm1OB/qCrAA==", 2024-05-22T21:18:15.137 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.137 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:18:15.137 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:18:15.137 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:18:15.137 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.137 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.7", 2024-05-22T21:18:15.138 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mCrUhGhAAvXoFWV7X3303rfXaxmfWqA==", 2024-05-22T21:18:15.138 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.138 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:18:15.138 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:18:15.138 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:18:15.138 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.138 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.0", 2024-05-22T21:18:15.138 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mh9B7LBAAevhvj5KpwP9pEIpJcxUvFA==", 2024-05-22T21:18:15.138 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.139 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:18:15.139 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow r", 2024-05-22T21:18:15.139 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-22T21:18:15.139 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.139 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.1", 2024-05-22T21:18:15.139 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mMnHtNRAAg/a/Mb1oRqx73d9efIHCfA==", 2024-05-22T21:18:15.139 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.139 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:18:15.139 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:18:15.140 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:18:15.140 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:18:15.140 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.140 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.2", 2024-05-22T21:18:15.140 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mOzdkMhAAbTiNdvyxqIyrluMiZW0cOw==", 2024-05-22T21:18:15.140 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.140 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:18:15.140 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:18:15.141 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:18:15.141 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:18:15.141 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.141 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.141 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.3", 2024-05-22T21:18:15.141 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDYX05mr4oxABAA1+bQQbxhoeU+kb0OCa+Wyw==", 2024-05-22T21:18:15.141 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.141 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:18:15.141 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:18:15.141 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:18:15.142 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:18:15.142 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.142 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.admin", 2024-05-22T21:18:15.142 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDWX05mHZY0NRAAiBvMDkrCZ+MgG2iwBH+bcg==", 2024-05-22T21:18:15.142 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.142 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:18:15.142 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:18:15.142 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:18:15.142 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:18:15.143 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.143 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.x", 2024-05-22T21:18:15.143 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mH2kuERAAgigMUWK3M3WaMXlqGUcNrw==", 2024-05-22T21:18:15.143 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.144 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:18:15.144 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:18:15.144 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:18:15.144 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:15.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:15.144 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.y", 2024-05-22T21:18:15.144 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mZRdiDRAA0vUwD+rPSfyhjgA+9XPp5w==", 2024-05-22T21:18:15.144 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:18:15.145 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:18:15.145 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:18:15.145 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:18:15.145 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.145 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:15.145 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:15.145 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:18:15.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.128+0000 7fb5cb800640 1 -- 172.21.15.161:0/3625720456 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb5a0066690 msgr2=0x7fb5a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.128+0000 7fb5cb800640 1 --2- 172.21.15.161:0/3625720456 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb5a0066690 0x7fb5a0068b50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0b2000 tx=0x7fb5ac001a80 comp rx=0 tx=0).stop 2024-05-22T21:18:15.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.128+0000 7fb5cb800640 1 -- 172.21.15.161:0/3625720456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a80047d0 msgr2=0x7fb5c4159a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.128+0000 7fb5cb800640 1 --2- 172.21.15.161:0/3625720456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a80047d0 0x7fb5c4159a40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009f90 tx=0x7fb5b400f580 comp rx=0 tx=0).stop 2024-05-22T21:18:15.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.128+0000 7fb5cb800640 1 -- 172.21.15.161:0/3625720456 shutdown_connections 2024-05-22T21:18:15.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.128+0000 7fb5cb800640 1 --2- 172.21.15.161:0/3625720456 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb5a0066690 0x7fb5a0068b50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.128+0000 7fb5cb800640 1 --2- 172.21.15.161:0/3625720456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5c4156940 0x7fb5c40b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.128+0000 7fb5cb800640 1 --2- 172.21.15.161:0/3625720456 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c40ce1c0 0x7fb5c414e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.128+0000 7fb5cb800640 1 --2- 172.21.15.161:0/3625720456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a80047d0 0x7fb5c4159a40 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.128+0000 7fb5cb800640 1 -- 172.21.15.161:0/3625720456 >> 172.21.15.161:0/3625720456 conn(0x7fb5c40b0bc0 msgr2=0x7fb5c4154de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:15.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.128+0000 7fb5cb800640 1 -- 172.21.15.161:0/3625720456 shutdown_connections 2024-05-22T21:18:15.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.128+0000 7fb5cb800640 1 -- 172.21.15.161:0/3625720456 wait complete. 2024-05-22T21:18:15.149 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:18:15.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.280+0000 7f7d03c00640 1 Processor -- start 2024-05-22T21:18:15.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.280+0000 7f7d03c00640 1 -- start start 2024-05-22T21:18:15.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.284+0000 7f7d03c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7cfc15e130 0x7f7cfc15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.284+0000 7f7d03c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc071670 con 0x7f7cfc15d410 2024-05-22T21:18:15.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.284+0000 7f7d03c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc15d870 con 0x7f7cfc15dd50 2024-05-22T21:18:15.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.284+0000 7f7d03c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cfc070a60 con 0x7f7cfc15e130 2024-05-22T21:18:15.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.284+0000 7f7d01c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f7cfc15d410 0x7f7cfc15b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:50722/0 (socket says 172.21.15.161:50722) 2024-05-22T21:18:15.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.284+0000 7f7d01c00640 1 -- 172.21.15.161:0/3915210720 learned_addr learned my addr 172.21.15.161:0/3915210720 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:15.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.284+0000 7f7cf3e00640 1 --2- 172.21.15.161:0/3915210720 >> v2:172.21.15.179:3300/0 conn(0x7f7cfc15e130 0x7f7cfc15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.284+0000 7f7cfbe00640 1 -- 172.21.15.161:0/3915210720 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530566339 0 0) 0x7f7cfc071670 con 0x7f7cfc15d410 2024-05-22T21:18:15.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.284+0000 7f7cfbe00640 1 -- 172.21.15.161:0/3915210720 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ce0003530 con 0x7f7cfc15d410 2024-05-22T21:18:15.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.284+0000 7f7cf3e00640 1 -- 172.21.15.161:0/3915210720 >> v1:172.21.15.179:6789/0 conn(0x7f7cfc15dd50 legacy=0x7f7cfc06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.284+0000 7f7cf3e00640 1 -- 172.21.15.161:0/3915210720 >> v1:172.21.15.161:6789/0 conn(0x7f7cfc15d410 legacy=0x7f7cfc15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.284+0000 7f7cf3e00640 1 -- 172.21.15.161:0/3915210720 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc070d40 con 0x7f7cfc15e130 2024-05-22T21:18:15.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.288+0000 7f7cf3e00640 1 --2- 172.21.15.161:0/3915210720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cfc15e130 0x7f7cfc15ced0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40097c0 tx=0x7f7ce401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5781aa28a58f8b36 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:15.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.288+0000 7f7cfbe00640 1 -- 172.21.15.161:0/3915210720 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce40ab670 con 0x7f7cfc15e130 2024-05-22T21:18:15.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.288+0000 7f7cfbe00640 1 -- 172.21.15.161:0/3915210720 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce40ab7d0 con 0x7f7cfc15e130 2024-05-22T21:18:15.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.288+0000 7f7cfbe00640 1 -- 172.21.15.161:0/3915210720 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce40abb60 con 0x7f7cfc15e130 2024-05-22T21:18:15.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.288+0000 7f7d03c00640 1 -- 172.21.15.161:0/3915210720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cfc15e130 msgr2=0x7f7cfc15ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.288+0000 7f7d03c00640 1 --2- 172.21.15.161:0/3915210720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cfc15e130 0x7f7cfc15ced0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40097c0 tx=0x7f7ce401c470 comp rx=0 tx=0).stop 2024-05-22T21:18:15.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.288+0000 7f7d03c00640 1 -- 172.21.15.161:0/3915210720 shutdown_connections 2024-05-22T21:18:15.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.288+0000 7f7d03c00640 1 --2- 172.21.15.161:0/3915210720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cfc15e130 0x7f7cfc15ced0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.288+0000 7f7d03c00640 1 -- 172.21.15.161:0/3915210720 >> 172.21.15.161:0/3915210720 conn(0x7f7cfc0b0bc0 msgr2=0x7f7cfc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:15.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.288+0000 7f7d03c00640 1 -- 172.21.15.161:0/3915210720 shutdown_connections 2024-05-22T21:18:15.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.288+0000 7f7d03c00640 1 -- 172.21.15.161:0/3915210720 wait complete. 2024-05-22T21:18:15.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.288+0000 7f7d03c00640 1 Processor -- start 2024-05-22T21:18:15.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.288+0000 7f7d03c00640 1 -- start start 2024-05-22T21:18:15.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7d03c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cfc15d410 0x7f7cfc13b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7d03c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cfc15dd50 0x7f7cfc0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7d03c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cfc15e130 0x7f7cfc0b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7d03c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0cdd70 con 0x7f7cfc15dd50 2024-05-22T21:18:15.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7d03c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0cd0f0 con 0x7f7cfc15e130 2024-05-22T21:18:15.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7d03c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0cd3d0 con 0x7f7cfc15d410 2024-05-22T21:18:15.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7cf3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cfc15dd50 0x7f7cfc0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7cf3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cfc15dd50 0x7f7cfc0cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41782/0 (socket says 172.21.15.161:41782) 2024-05-22T21:18:15.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7cf3e00640 1 -- 172.21.15.161:0/1030328633 learned_addr learned my addr 172.21.15.161:0/1030328633 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:15.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7d01c00640 1 --2- 172.21.15.161:0/1030328633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cfc15e130 0x7f7cfc0b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7d01200640 1 --2- 172.21.15.161:0/1030328633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cfc15d410 0x7f7cfc13b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7cf3e00640 1 -- 172.21.15.161:0/1030328633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cfc15d410 msgr2=0x7f7cfc13b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7cf3e00640 1 --2- 172.21.15.161:0/1030328633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cfc15d410 0x7f7cfc13b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7cf3e00640 1 -- 172.21.15.161:0/1030328633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cfc15e130 msgr2=0x7f7cfc0b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7cf3e00640 1 --2- 172.21.15.161:0/1030328633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cfc15e130 0x7f7cfc0b2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7cf3e00640 1 -- 172.21.15.161:0/1030328633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce4009420 con 0x7f7cfc15dd50 2024-05-22T21:18:15.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7cf3e00640 1 --2- 172.21.15.161:0/1030328633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cfc15dd50 0x7f7cfc0cc630 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4019080 tx=0x7f7ce4019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:15.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7cf9600640 1 -- 172.21.15.161:0/1030328633 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce40abe10 con 0x7f7cfc15dd50 2024-05-22T21:18:15.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7cf9600640 1 -- 172.21.15.161:0/1030328633 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce40bf450 con 0x7f7cfc15dd50 2024-05-22T21:18:15.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7cf9600640 1 -- 172.21.15.161:0/1030328633 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce40bf7e0 con 0x7f7cfc15dd50 2024-05-22T21:18:15.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7d03c00640 1 -- 172.21.15.161:0/1030328633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc0cd6b0 con 0x7f7cfc15dd50 2024-05-22T21:18:15.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7d03c00640 1 -- 172.21.15.161:0/1030328633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc0ce5c0 con 0x7f7cfc15dd50 2024-05-22T21:18:15.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7d03c00640 1 -- 172.21.15.161:0/1030328633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc067220 con 0x7f7cfc15dd50 2024-05-22T21:18:15.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.292+0000 7f7cf9600640 1 -- 172.21.15.161:0/1030328633 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7ce40b43f0 con 0x7f7cfc15dd50 2024-05-22T21:18:15.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.296+0000 7f7cf9600640 1 --2- 172.21.15.161:0/1030328633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7cd8066690 0x7f7cd8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.296+0000 7f7cf9600640 1 -- 172.21.15.161:0/1030328633 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7ce411e8a0 con 0x7f7cfc15dd50 2024-05-22T21:18:15.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.296+0000 7f7d01200640 1 --2- 172.21.15.161:0/1030328633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7cd8066690 0x7f7cd8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.296+0000 7f7d01200640 1 --2- 172.21.15.161:0/1030328633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7cd8066690 0x7f7cd8068b50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7cec007fd0 tx=0x7f7cec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:15.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.300+0000 7f7cf9600640 1 -- 172.21.15.161:0/1030328633 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7ce40ec000 con 0x7f7cfc15dd50 2024-05-22T21:18:15.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.456+0000 7f7d03c00640 1 -- 172.21.15.161:0/1030328633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cfc0cd510 con 0x7f7cfc15dd50 2024-05-22T21:18:15.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.456+0000 7f7cf9600640 1 -- 172.21.15.161:0/1030328633 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+12438 (secure 0 0 0) 0x7f7cfc0cd510 con 0x7f7cfc15dd50 2024-05-22T21:18:15.462 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:15.462 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":99,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:18:14.730089+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:18:15.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.460+0000 7f7d03c00640 1 -- 172.21.15.161:0/1030328633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7cd8066690 msgr2=0x7f7cd8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.460+0000 7f7d03c00640 1 --2- 172.21.15.161:0/1030328633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7cd8066690 0x7f7cd8068b50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7cec007fd0 tx=0x7f7cec009040 comp rx=0 tx=0).stop 2024-05-22T21:18:15.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.460+0000 7f7d03c00640 1 -- 172.21.15.161:0/1030328633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cfc15dd50 msgr2=0x7f7cfc0cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.460+0000 7f7d03c00640 1 --2- 172.21.15.161:0/1030328633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cfc15dd50 0x7f7cfc0cc630 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4019080 tx=0x7f7ce4019210 comp rx=0 tx=0).stop 2024-05-22T21:18:15.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.460+0000 7f7d03c00640 1 -- 172.21.15.161:0/1030328633 shutdown_connections 2024-05-22T21:18:15.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.460+0000 7f7d03c00640 1 --2- 172.21.15.161:0/1030328633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7cd8066690 0x7f7cd8068b50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.460+0000 7f7d03c00640 1 --2- 172.21.15.161:0/1030328633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cfc15e130 0x7f7cfc0b2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.460+0000 7f7d03c00640 1 --2- 172.21.15.161:0/1030328633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cfc15dd50 0x7f7cfc0cc630 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.460+0000 7f7d03c00640 1 --2- 172.21.15.161:0/1030328633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cfc15d410 0x7f7cfc13b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.460+0000 7f7d03c00640 1 -- 172.21.15.161:0/1030328633 >> 172.21.15.161:0/1030328633 conn(0x7f7cfc0b0bc0 msgr2=0x7f7cfc0af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:15.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.460+0000 7f7d03c00640 1 -- 172.21.15.161:0/1030328633 shutdown_connections 2024-05-22T21:18:15.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.460+0000 7f7d03c00640 1 -- 172.21.15.161:0/1030328633 wait complete. 2024-05-22T21:18:15.482 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:18:15.482 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:18:15.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.616+0000 7f7727c00640 1 Processor -- start 2024-05-22T21:18:15.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.616+0000 7f7727c00640 1 -- start start 2024-05-22T21:18:15.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.620+0000 7f7727c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f772015b600 0x7f772006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.620+0000 7f7727c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f772006c7e0 0x7f77200b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.620+0000 7f7727c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7720144bb0 con 0x7f7720143c40 2024-05-22T21:18:15.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.620+0000 7f7727c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772015ba60 con 0x7f772006c7e0 2024-05-22T21:18:15.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.620+0000 7f7727c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772006cc70 con 0x7f772015b600 2024-05-22T21:18:15.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.620+0000 7f7717e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f772015b600 0x7f772006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.620+0000 7f7725c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f772006c7e0 0x7f77200b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.620+0000 7f7717e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f772015b600 0x7f772006c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:45654/0 (socket says 172.21.15.161:45654) 2024-05-22T21:18:15.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.620+0000 7f7717e00640 1 -- 172.21.15.161:0/1188439236 learned_addr learned my addr 172.21.15.161:0/1188439236 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:15.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.620+0000 7f7717e00640 1 -- 172.21.15.161:0/1188439236 >> v2:172.21.15.179:3300/0 conn(0x7f772006c7e0 msgr2=0x7f77200b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:15.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.620+0000 7f7717e00640 1 --2- 172.21.15.161:0/1188439236 >> v2:172.21.15.179:3300/0 conn(0x7f772006c7e0 0x7f77200b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.620+0000 7f7717e00640 1 -- 172.21.15.161:0/1188439236 >> v1:172.21.15.179:6789/0 conn(0x7f7720143c40 legacy=0x7f77201444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.620+0000 7f7717e00640 1 -- 172.21.15.161:0/1188439236 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772015efb0 con 0x7f772015b600 2024-05-22T21:18:15.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.620+0000 7f7725c00640 1 --2- 172.21.15.161:0/1188439236 >> v2:172.21.15.179:3300/0 conn(0x7f772006c7e0 0x7f77200b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:15.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.624+0000 7f7717e00640 1 --2- 172.21.15.161:0/1188439236 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f772015b600 0x7f772006c2a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7708009870 tx=0x7f770801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54347685afd231b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:15.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.624+0000 7f771fe00640 1 -- 172.21.15.161:0/1188439236 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080a9620 con 0x7f772015b600 2024-05-22T21:18:15.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.624+0000 7f771fe00640 1 -- 172.21.15.161:0/1188439236 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77080a9780 con 0x7f772015b600 2024-05-22T21:18:15.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.624+0000 7f771fe00640 1 -- 172.21.15.161:0/1188439236 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080a9b10 con 0x7f772015b600 2024-05-22T21:18:15.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.624+0000 7f7727c00640 1 -- 172.21.15.161:0/1188439236 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f772015b600 msgr2=0x7f772006c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.624+0000 7f7727c00640 1 --2- 172.21.15.161:0/1188439236 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f772015b600 0x7f772006c2a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7708009870 tx=0x7f770801c560 comp rx=0 tx=0).stop 2024-05-22T21:18:15.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.624+0000 7f7727c00640 1 -- 172.21.15.161:0/1188439236 shutdown_connections 2024-05-22T21:18:15.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.624+0000 7f7727c00640 1 --2- 172.21.15.161:0/1188439236 >> v2:172.21.15.179:3300/0 conn(0x7f772006c7e0 0x7f77200b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.624+0000 7f7727c00640 1 --2- 172.21.15.161:0/1188439236 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f772015b600 0x7f772006c2a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.624+0000 7f7727c00640 1 -- 172.21.15.161:0/1188439236 >> 172.21.15.161:0/1188439236 conn(0x7f772006a2d0 msgr2=0x7f772006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:15.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.624+0000 7f7727c00640 1 -- 172.21.15.161:0/1188439236 shutdown_connections 2024-05-22T21:18:15.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.624+0000 7f7727c00640 1 -- 172.21.15.161:0/1188439236 wait complete. 2024-05-22T21:18:15.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.624+0000 7f7727c00640 1 Processor -- start 2024-05-22T21:18:15.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.624+0000 7f7727c00640 1 -- start start 2024-05-22T21:18:15.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7727c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f772006c7e0 0x7f7720136660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7727c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7720143c40 0x7f77200ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7727c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f772015b600 0x7f77200ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7727c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772013a0e0 con 0x7f7720143c40 2024-05-22T21:18:15.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7727c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77200b3fc0 con 0x7f772006c7e0 2024-05-22T21:18:15.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7727c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7720153a60 con 0x7f772015b600 2024-05-22T21:18:15.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7717e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7720143c40 0x7f77200ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7717e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7720143c40 0x7f77200ad900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41796/0 (socket says 172.21.15.161:41796) 2024-05-22T21:18:15.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7717e00640 1 -- 172.21.15.161:0/3206019786 learned_addr learned my addr 172.21.15.161:0/3206019786 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:15.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7725c00640 1 --2- 172.21.15.161:0/3206019786 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f772015b600 0x7f77200ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7717e00640 1 -- 172.21.15.161:0/3206019786 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f772015b600 msgr2=0x7f77200ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7717e00640 1 --2- 172.21.15.161:0/3206019786 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f772015b600 0x7f77200ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7717e00640 1 -- 172.21.15.161:0/3206019786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f772006c7e0 msgr2=0x7f7720136660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7725200640 1 --2- 172.21.15.161:0/3206019786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f772006c7e0 0x7f7720136660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7717e00640 1 --2- 172.21.15.161:0/3206019786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f772006c7e0 0x7f7720136660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7717e00640 1 -- 172.21.15.161:0/3206019786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7718009a70 con 0x7f7720143c40 2024-05-22T21:18:15.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7725200640 1 --2- 172.21.15.161:0/3206019786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f772006c7e0 0x7f7720136660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:15.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7717e00640 1 --2- 172.21.15.161:0/3206019786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7720143c40 0x7f77200ad900 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f770801ac10 tx=0x7f7708019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:15.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f771d600640 1 -- 172.21.15.161:0/3206019786 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080a9e80 con 0x7f7720143c40 2024-05-22T21:18:15.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f771d600640 1 -- 172.21.15.161:0/3206019786 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77080b9430 con 0x7f7720143c40 2024-05-22T21:18:15.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f771d600640 1 -- 172.21.15.161:0/3206019786 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080b97a0 con 0x7f7720143c40 2024-05-22T21:18:15.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7727c00640 1 -- 172.21.15.161:0/3206019786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77080094d0 con 0x7f7720143c40 2024-05-22T21:18:15.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7727c00640 1 -- 172.21.15.161:0/3206019786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7720153130 con 0x7f7720143c40 2024-05-22T21:18:15.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.628+0000 7f7727c00640 1 -- 172.21.15.161:0/3206019786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7720153bd0 con 0x7f7720143c40 2024-05-22T21:18:15.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.632+0000 7f771d600640 1 -- 172.21.15.161:0/3206019786 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f77080b9900 con 0x7f7720143c40 2024-05-22T21:18:15.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.632+0000 7f771d600640 1 --2- 172.21.15.161:0/3206019786 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76fc066690 0x7f76fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.632+0000 7f771d600640 1 -- 172.21.15.161:0/3206019786 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f770811dcf0 con 0x7f7720143c40 2024-05-22T21:18:15.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.632+0000 7f7725200640 1 --2- 172.21.15.161:0/3206019786 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76fc066690 0x7f76fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.632+0000 7f7725200640 1 --2- 172.21.15.161:0/3206019786 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76fc066690 0x7f76fc068b50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7710000f70 tx=0x7f7710009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:15.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.636+0000 7f771d600640 1 -- 172.21.15.161:0/3206019786 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f77080eb310 con 0x7f7720143c40 2024-05-22T21:18:15.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.792+0000 7f7727c00640 1 -- 172.21.15.161:0/3206019786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7720067220 con 0x7f7720143c40 2024-05-22T21:18:15.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.792+0000 7f771d600640 1 -- 172.21.15.161:0/3206019786 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77080ee930 con 0x7f7720143c40 2024-05-22T21:18:15.794 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:18:15.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.792+0000 7f7727c00640 1 -- 172.21.15.161:0/3206019786 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76fc066690 msgr2=0x7f76fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.792+0000 7f7727c00640 1 --2- 172.21.15.161:0/3206019786 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76fc066690 0x7f76fc068b50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7710000f70 tx=0x7f7710009040 comp rx=0 tx=0).stop 2024-05-22T21:18:15.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.792+0000 7f7727c00640 1 -- 172.21.15.161:0/3206019786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7720143c40 msgr2=0x7f77200ad900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.792+0000 7f7727c00640 1 --2- 172.21.15.161:0/3206019786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7720143c40 0x7f77200ad900 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f770801ac10 tx=0x7f7708019300 comp rx=0 tx=0).stop 2024-05-22T21:18:15.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.792+0000 7f7727c00640 1 -- 172.21.15.161:0/3206019786 shutdown_connections 2024-05-22T21:18:15.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.792+0000 7f7727c00640 1 --2- 172.21.15.161:0/3206019786 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f76fc066690 0x7f76fc068b50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.792+0000 7f7727c00640 1 --2- 172.21.15.161:0/3206019786 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f772015b600 0x7f77200ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.792+0000 7f7727c00640 1 --2- 172.21.15.161:0/3206019786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7720143c40 0x7f77200ad900 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.792+0000 7f7727c00640 1 --2- 172.21.15.161:0/3206019786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f772006c7e0 0x7f7720136660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.792+0000 7f7727c00640 1 -- 172.21.15.161:0/3206019786 >> 172.21.15.161:0/3206019786 conn(0x7f772006a2d0 msgr2=0x7f77200afee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:15.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.792+0000 7f7727c00640 1 -- 172.21.15.161:0/3206019786 shutdown_connections 2024-05-22T21:18:15.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.792+0000 7f7727c00640 1 -- 172.21.15.161:0/3206019786 wait complete. 2024-05-22T21:18:15.815 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-22T21:18:15.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.948+0000 7fddfd600640 1 Processor -- start 2024-05-22T21:18:15.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.948+0000 7fddfd600640 1 -- start start 2024-05-22T21:18:15.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.948+0000 7fddfd600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddf8143ec0 0x7fddf815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.948+0000 7fddfd600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fddf815ec90 0x7fddf815f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.948+0000 7fddfd600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf81442a0 con 0x7fddf80715a0 2024-05-22T21:18:15.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.948+0000 7fddfd600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf815f680 con 0x7fddf815ec90 2024-05-22T21:18:15.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.948+0000 7fddfd600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf8070500 con 0x7fddf8143ec0 2024-05-22T21:18:15.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.948+0000 7fddf7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fddf815ec90 0x7fddf815f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.948+0000 7fddf7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fddf815ec90 0x7fddf815f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38408/0 (socket says 172.21.15.161:38408) 2024-05-22T21:18:15.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.948+0000 7fddf7400640 1 -- 172.21.15.161:0/2025721053 learned_addr learned my addr 172.21.15.161:0/2025721053 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:15.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.948+0000 7fddf6a00640 1 --2- 172.21.15.161:0/2025721053 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddf8143ec0 0x7fddf815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.952+0000 7fddf7400640 1 -- 172.21.15.161:0/2025721053 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddf8143ec0 msgr2=0x7fddf815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:15.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.952+0000 7fddf7400640 1 --2- 172.21.15.161:0/2025721053 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddf8143ec0 0x7fddf815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.952+0000 7fddf7400640 1 -- 172.21.15.161:0/2025721053 >> v1:172.21.15.161:6789/0 conn(0x7fddf80715a0 legacy=0x7fddf806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.952+0000 7fddf7400640 1 -- 172.21.15.161:0/2025721053 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf80707e0 con 0x7fddf815ec90 2024-05-22T21:18:15.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.952+0000 7fddf7400640 1 --2- 172.21.15.161:0/2025721053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddf815ec90 0x7fddf815f140 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009e50 tx=0x7fddf001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e731c8454de2f23a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:15.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.952+0000 7fddf5600640 1 -- 172.21.15.161:0/2025721053 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf00a57d0 con 0x7fddf815ec90 2024-05-22T21:18:15.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.952+0000 7fddf5600640 1 -- 172.21.15.161:0/2025721053 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf00a5930 con 0x7fddf815ec90 2024-05-22T21:18:15.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.952+0000 7fddf5600640 1 -- 172.21.15.161:0/2025721053 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf00a5cc0 con 0x7fddf815ec90 2024-05-22T21:18:15.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.952+0000 7fddfd600640 1 -- 172.21.15.161:0/2025721053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddf815ec90 msgr2=0x7fddf815f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.952+0000 7fddfd600640 1 --2- 172.21.15.161:0/2025721053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddf815ec90 0x7fddf815f140 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009e50 tx=0x7fddf001ca00 comp rx=0 tx=0).stop 2024-05-22T21:18:15.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.952+0000 7fddfd600640 1 -- 172.21.15.161:0/2025721053 shutdown_connections 2024-05-22T21:18:15.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.952+0000 7fddfd600640 1 --2- 172.21.15.161:0/2025721053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddf815ec90 0x7fddf815f140 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.952+0000 7fddfd600640 1 --2- 172.21.15.161:0/2025721053 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddf8143ec0 0x7fddf815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.952+0000 7fddfd600640 1 -- 172.21.15.161:0/2025721053 >> 172.21.15.161:0/2025721053 conn(0x7fddf80b0bc0 msgr2=0x7fddf80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:15.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.952+0000 7fddfd600640 1 -- 172.21.15.161:0/2025721053 shutdown_connections 2024-05-22T21:18:15.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.952+0000 7fddfd600640 1 -- 172.21.15.161:0/2025721053 wait complete. 2024-05-22T21:18:15.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddfd600640 1 Processor -- start 2024-05-22T21:18:15.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddfd600640 1 -- start start 2024-05-22T21:18:15.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddfd600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddf8143ec0 0x7fddf8138a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddfd600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddf815ec90 0x7fddf813cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddfd600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddf813d1e0 0x7fddf81455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddfd600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf813d670 con 0x7fddf8143ec0 2024-05-22T21:18:15.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddfd600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf81489e0 con 0x7fddf815ec90 2024-05-22T21:18:15.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddfd600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf8146340 con 0x7fddf813d1e0 2024-05-22T21:18:15.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddf6a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddf8143ec0 0x7fddf8138a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddf6a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddf8143ec0 0x7fddf8138a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38412/0 (socket says 172.21.15.161:38412) 2024-05-22T21:18:15.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddf6a00640 1 -- 172.21.15.161:0/3425767176 learned_addr learned my addr 172.21.15.161:0/3425767176 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:15.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddf7400640 1 --2- 172.21.15.161:0/3425767176 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddf813d1e0 0x7fddf81455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddf6000640 1 --2- 172.21.15.161:0/3425767176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddf815ec90 0x7fddf813cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddf6a00640 1 -- 172.21.15.161:0/3425767176 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddf813d1e0 msgr2=0x7fddf81455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddf6a00640 1 --2- 172.21.15.161:0/3425767176 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddf813d1e0 0x7fddf81455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddf6a00640 1 -- 172.21.15.161:0/3425767176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddf815ec90 msgr2=0x7fddf813cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:15.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddf6a00640 1 --2- 172.21.15.161:0/3425767176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddf815ec90 0x7fddf813cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:15.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddf6a00640 1 -- 172.21.15.161:0/3425767176 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf0009b30 con 0x7fddf8143ec0 2024-05-22T21:18:15.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddf6000640 1 --2- 172.21.15.161:0/3425767176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddf815ec90 0x7fddf813cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:15.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddf6a00640 1 --2- 172.21.15.161:0/3425767176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddf8143ec0 0x7fddf8138a50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fdde800d800 tx=0x7fdde800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:15.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddeea00640 1 -- 172.21.15.161:0/3425767176 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde800b6c0 con 0x7fddf8143ec0 2024-05-22T21:18:15.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddeea00640 1 -- 172.21.15.161:0/3425767176 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde800b820 con 0x7fddf8143ec0 2024-05-22T21:18:15.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddfd600640 1 -- 172.21.15.161:0/3425767176 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf8146620 con 0x7fddf8143ec0 2024-05-22T21:18:15.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddeea00640 1 -- 172.21.15.161:0/3425767176 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde800bb90 con 0x7fddf8143ec0 2024-05-22T21:18:15.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.956+0000 7fddfd600640 1 -- 172.21.15.161:0/3425767176 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf8146be0 con 0x7fddf8143ec0 2024-05-22T21:18:15.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.960+0000 7fddfd600640 1 -- 172.21.15.161:0/3425767176 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf80ccfb0 con 0x7fddf8143ec0 2024-05-22T21:18:15.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.960+0000 7fddeea00640 1 -- 172.21.15.161:0/3425767176 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdde800bcf0 con 0x7fddf8143ec0 2024-05-22T21:18:15.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.960+0000 7fddeea00640 1 --2- 172.21.15.161:0/3425767176 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fddd8066690 0x7fddd8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:15.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.960+0000 7fddeea00640 1 -- 172.21.15.161:0/3425767176 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdde80843b0 con 0x7fddf8143ec0 2024-05-22T21:18:15.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.960+0000 7fddf6000640 1 --2- 172.21.15.161:0/3425767176 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fddd8066690 0x7fddd8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:15.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.960+0000 7fddf6000640 1 --2- 172.21.15.161:0/3425767176 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fddd8066690 0x7fddd8068b50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fddf815ebb0 tx=0x7fdde4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:15.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:15.964+0000 7fddeea00640 1 -- 172.21.15.161:0/3425767176 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdde80519d0 con 0x7fddf8143ec0 2024-05-22T21:18:16.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.120+0000 7fddfd600640 1 -- 172.21.15.161:0/3425767176 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fddf8067220 con 0x7fddf8143ec0 2024-05-22T21:18:16.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.120+0000 7fddeea00640 1 -- 172.21.15.161:0/3425767176 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdde8054ff0 con 0x7fddf8143ec0 2024-05-22T21:18:16.122 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:16.122 INFO:teuthology.orchestra.run.smithi161.stdout:[ 2024-05-22T21:18:16.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:16.123 INFO:teuthology.orchestra.run.smithi161.stdout: "poolnum": 1, 2024-05-22T21:18:16.123 INFO:teuthology.orchestra.run.smithi161.stdout: "poolname": ".mgr" 2024-05-22T21:18:16.123 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:16.123 INFO:teuthology.orchestra.run.smithi161.stdout:] 2024-05-22T21:18:16.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.120+0000 7fddfd600640 1 -- 172.21.15.161:0/3425767176 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fddd8066690 msgr2=0x7fddd8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:16.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.120+0000 7fddfd600640 1 --2- 172.21.15.161:0/3425767176 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fddd8066690 0x7fddd8068b50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fddf815ebb0 tx=0x7fdde4009040 comp rx=0 tx=0).stop 2024-05-22T21:18:16.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.120+0000 7fddfd600640 1 -- 172.21.15.161:0/3425767176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddf8143ec0 msgr2=0x7fddf8138a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:16.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.120+0000 7fddfd600640 1 --2- 172.21.15.161:0/3425767176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddf8143ec0 0x7fddf8138a50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fdde800d800 tx=0x7fdde800dcc0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.120+0000 7fddfd600640 1 -- 172.21.15.161:0/3425767176 shutdown_connections 2024-05-22T21:18:16.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.120+0000 7fddfd600640 1 --2- 172.21.15.161:0/3425767176 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fddd8066690 0x7fddd8068b50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.120+0000 7fddfd600640 1 --2- 172.21.15.161:0/3425767176 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddf813d1e0 0x7fddf81455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.120+0000 7fddfd600640 1 --2- 172.21.15.161:0/3425767176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddf815ec90 0x7fddf813cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.120+0000 7fddfd600640 1 --2- 172.21.15.161:0/3425767176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddf8143ec0 0x7fddf8138a50 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.120+0000 7fddfd600640 1 -- 172.21.15.161:0/3425767176 >> 172.21.15.161:0/3425767176 conn(0x7fddf80b0bc0 msgr2=0x7fddf81430f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:16.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.120+0000 7fddfd600640 1 -- 172.21.15.161:0/3425767176 shutdown_connections 2024-05-22T21:18:16.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.120+0000 7fddfd600640 1 -- 172.21.15.161:0/3425767176 wait complete. 2024-05-22T21:18:16.143 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-22T21:18:16.143 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-22T21:18:16.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.280+0000 7effcdc00640 1 Processor -- start 2024-05-22T21:18:16.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.280+0000 7effcdc00640 1 -- start start 2024-05-22T21:18:16.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.280+0000 7effcdc00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7effc8144980 0x7effc815c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:16.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.280+0000 7effcdc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7effc80ccc10 0x7effc815c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:16.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.280+0000 7effcdc00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc806d0d0 con 0x7effc806d560 2024-05-22T21:18:16.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.280+0000 7effcdc00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc815c680 con 0x7effc80ccc10 2024-05-22T21:18:16.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.280+0000 7effcdc00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc806c4c0 con 0x7effc8144980 2024-05-22T21:18:16.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.280+0000 7effc7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7effc80ccc10 0x7effc815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:16.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.280+0000 7effc7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7effc80ccc10 0x7effc815c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38420/0 (socket says 172.21.15.161:38420) 2024-05-22T21:18:16.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.280+0000 7effc7400640 1 -- 172.21.15.161:0/228343313 learned_addr learned my addr 172.21.15.161:0/228343313 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:16.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.280+0000 7effc6a00640 1 --2- 172.21.15.161:0/228343313 >> v2:172.21.15.179:3300/0 conn(0x7effc8144980 0x7effc815c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:16.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.280+0000 7effc7400640 1 -- 172.21.15.161:0/228343313 >> v2:172.21.15.179:3300/0 conn(0x7effc8144980 msgr2=0x7effc815c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:16.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.280+0000 7effc7400640 1 --2- 172.21.15.161:0/228343313 >> v2:172.21.15.179:3300/0 conn(0x7effc8144980 0x7effc815c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.280+0000 7effc7400640 1 -- 172.21.15.161:0/228343313 >> v1:172.21.15.179:6789/0 conn(0x7effc806d560 legacy=0x7effc80cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:16.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.284+0000 7effc7400640 1 -- 172.21.15.161:0/228343313 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc806c7a0 con 0x7effc80ccc10 2024-05-22T21:18:16.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.284+0000 7effc7400640 1 --2- 172.21.15.161:0/228343313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effc80ccc10 0x7effc815c880 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7effc8071fd0 tx=0x7effc001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b41c75b2a207f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:16.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.284+0000 7effc6000640 1 -- 172.21.15.161:0/228343313 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc00a57a0 con 0x7effc80ccc10 2024-05-22T21:18:16.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.284+0000 7effc6000640 1 -- 172.21.15.161:0/228343313 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc00a5900 con 0x7effc80ccc10 2024-05-22T21:18:16.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.284+0000 7effc6000640 1 -- 172.21.15.161:0/228343313 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc00a5c90 con 0x7effc80ccc10 2024-05-22T21:18:16.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.284+0000 7effcdc00640 1 -- 172.21.15.161:0/228343313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effc80ccc10 msgr2=0x7effc815c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:16.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.284+0000 7effcdc00640 1 --2- 172.21.15.161:0/228343313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effc80ccc10 0x7effc815c880 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7effc8071fd0 tx=0x7effc001c9f0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.284+0000 7effcdc00640 1 -- 172.21.15.161:0/228343313 shutdown_connections 2024-05-22T21:18:16.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.284+0000 7effcdc00640 1 --2- 172.21.15.161:0/228343313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effc80ccc10 0x7effc815c880 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.284+0000 7effcdc00640 1 --2- 172.21.15.161:0/228343313 >> v2:172.21.15.179:3300/0 conn(0x7effc8144980 0x7effc815c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.284+0000 7effcdc00640 1 -- 172.21.15.161:0/228343313 >> 172.21.15.161:0/228343313 conn(0x7effc806a0a0 msgr2=0x7effc806a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:16.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.284+0000 7effcdc00640 1 -- 172.21.15.161:0/228343313 shutdown_connections 2024-05-22T21:18:16.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.284+0000 7effcdc00640 1 -- 172.21.15.161:0/228343313 wait complete. 2024-05-22T21:18:16.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effcdc00640 1 Processor -- start 2024-05-22T21:18:16.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effcdc00640 1 -- start start 2024-05-22T21:18:16.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effcdc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effc806d560 0x7effc813b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:16.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effcdc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effc80ccc10 0x7effc8157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:16.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effcdc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effc8144980 0x7effc8158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:16.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effcdc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc813bb20 con 0x7effc806d560 2024-05-22T21:18:16.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effcdc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc813be00 con 0x7effc80ccc10 2024-05-22T21:18:16.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effcdc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc813c0e0 con 0x7effc8144980 2024-05-22T21:18:16.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effc6a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effc806d560 0x7effc813b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:16.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effc6a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effc806d560 0x7effc813b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38436/0 (socket says 172.21.15.161:38436) 2024-05-22T21:18:16.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effc6a00640 1 -- 172.21.15.161:0/2317640977 learned_addr learned my addr 172.21.15.161:0/2317640977 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:16.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effc7400640 1 --2- 172.21.15.161:0/2317640977 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effc8144980 0x7effc8158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:16.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effbe000640 1 --2- 172.21.15.161:0/2317640977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effc80ccc10 0x7effc8157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:16.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effc6a00640 1 -- 172.21.15.161:0/2317640977 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effc8144980 msgr2=0x7effc8158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:16.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effc6a00640 1 --2- 172.21.15.161:0/2317640977 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effc8144980 0x7effc8158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effc6a00640 1 -- 172.21.15.161:0/2317640977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effc80ccc10 msgr2=0x7effc8157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:16.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effc6a00640 1 --2- 172.21.15.161:0/2317640977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effc80ccc10 0x7effc8157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effc6a00640 1 -- 172.21.15.161:0/2317640977 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc0009b30 con 0x7effc806d560 2024-05-22T21:18:16.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effc6a00640 1 --2- 172.21.15.161:0/2317640977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effc806d560 0x7effc813b0f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7effb800d5f0 tx=0x7effb800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:16.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effbf400640 1 -- 172.21.15.161:0/2317640977 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb80125f0 con 0x7effc806d560 2024-05-22T21:18:16.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effbf400640 1 -- 172.21.15.161:0/2317640977 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb8012750 con 0x7effc806d560 2024-05-22T21:18:16.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effcdc00640 1 -- 172.21.15.161:0/2317640977 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc813c3c0 con 0x7effc806d560 2024-05-22T21:18:16.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effbf400640 1 -- 172.21.15.161:0/2317640977 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8012ae0 con 0x7effc806d560 2024-05-22T21:18:16.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.288+0000 7effcdc00640 1 -- 172.21.15.161:0/2317640977 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc813c810 con 0x7effc806d560 2024-05-22T21:18:16.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.292+0000 7effcdc00640 1 -- 172.21.15.161:0/2317640977 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc806d970 con 0x7effc806d560 2024-05-22T21:18:16.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.292+0000 7effbf400640 1 -- 172.21.15.161:0/2317640977 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7effb8012c40 con 0x7effc806d560 2024-05-22T21:18:16.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.292+0000 7effbf400640 1 --2- 172.21.15.161:0/2317640977 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7effa40666c0 0x7effa4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:16.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.292+0000 7effbf400640 1 -- 172.21.15.161:0/2317640977 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7effb8084230 con 0x7effc806d560 2024-05-22T21:18:16.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.292+0000 7effbe000640 1 --2- 172.21.15.161:0/2317640977 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7effa40666c0 0x7effa4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:16.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.292+0000 7effbe000640 1 --2- 172.21.15.161:0/2317640977 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7effa40666c0 0x7effa4068b80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7effc8154280 tx=0x7effb0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:16.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.296+0000 7effbf400640 1 -- 172.21.15.161:0/2317640977 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7effc806d970 con 0x7effc806d560 2024-05-22T21:18:16.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.451+0000 7effcdc00640 1 -- 172.21.15.161:0/2317640977 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7effc806d970 con 0x7effc806d560 2024-05-22T21:18:16.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.551+0000 7effbf400640 1 -- 172.21.15.161:0/2317640977 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) v1 ==== 138+0+0 (secure 0 0 0) 0x7effc806d970 con 0x7effc806d560 2024-05-22T21:18:16.554 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' created 2024-05-22T21:18:16.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.555+0000 7effcdc00640 1 -- 172.21.15.161:0/2317640977 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7effa40666c0 msgr2=0x7effa4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:16.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.555+0000 7effcdc00640 1 --2- 172.21.15.161:0/2317640977 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7effa40666c0 0x7effa4068b80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7effc8154280 tx=0x7effb0009040 comp rx=0 tx=0).stop 2024-05-22T21:18:16.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.555+0000 7effcdc00640 1 -- 172.21.15.161:0/2317640977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effc806d560 msgr2=0x7effc813b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:16.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.555+0000 7effcdc00640 1 --2- 172.21.15.161:0/2317640977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effc806d560 0x7effc813b0f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7effb800d5f0 tx=0x7effb800dab0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.555+0000 7effcdc00640 1 -- 172.21.15.161:0/2317640977 shutdown_connections 2024-05-22T21:18:16.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.555+0000 7effcdc00640 1 --2- 172.21.15.161:0/2317640977 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7effa40666c0 0x7effa4068b80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.555+0000 7effcdc00640 1 --2- 172.21.15.161:0/2317640977 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effc8144980 0x7effc8158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.555+0000 7effcdc00640 1 --2- 172.21.15.161:0/2317640977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effc80ccc10 0x7effc8157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.555+0000 7effcdc00640 1 --2- 172.21.15.161:0/2317640977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effc806d560 0x7effc813b0f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.555+0000 7effcdc00640 1 -- 172.21.15.161:0/2317640977 >> 172.21.15.161:0/2317640977 conn(0x7effc806a0a0 msgr2=0x7effc8143400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:16.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.555+0000 7effcdc00640 1 -- 172.21.15.161:0/2317640977 shutdown_connections 2024-05-22T21:18:16.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.555+0000 7effcdc00640 1 -- 172.21.15.161:0/2317640977 wait complete. 2024-05-22T21:18:16.578 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-22T21:18:16.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f9045400640 1 Processor -- start 2024-05-22T21:18:16.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f9045400640 1 -- start start 2024-05-22T21:18:16.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f9045400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f904013cfc0 0x7f90401402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:16.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f9045400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f904015af60 0x7f904013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:16.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f9045400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90401400d0 con 0x7f904013fc80 2024-05-22T21:18:16.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f9045400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f904013d3a0 con 0x7f904013cfc0 2024-05-22T21:18:16.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f9045400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90400cd920 con 0x7f904015af60 2024-05-22T21:18:16.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f903e800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f904013cfc0 0x7f90401402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:16.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f903de00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f904015af60 0x7f904013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:16.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f903f200640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f904013fc80 0x7f904015dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:56600/0 (socket says 172.21.15.161:56600) 2024-05-22T21:18:16.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f903e800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f904013cfc0 0x7f90401402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:54764/0 (socket says 172.21.15.161:54764) 2024-05-22T21:18:16.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f903f200640 1 -- 172.21.15.161:0/3175368849 learned_addr learned my addr 172.21.15.161:0/3175368849 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:16.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f903d400640 1 -- 172.21.15.161:0/3175368849 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770067715 0 0) 0x7f90380014a0 con 0x7f904013fc80 2024-05-22T21:18:16.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f903d400640 1 -- 172.21.15.161:0/3175368849 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9024005740 con 0x7f904013fc80 2024-05-22T21:18:16.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f903e800640 1 -- 172.21.15.161:0/3175368849 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f904015af60 msgr2=0x7f904013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:16.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f903e800640 1 --2- 172.21.15.161:0/3175368849 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f904015af60 0x7f904013f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f903e800640 1 -- 172.21.15.161:0/3175368849 >> v1:172.21.15.179:6789/0 conn(0x7f904013fc80 legacy=0x7f904015dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:16.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f903e800640 1 -- 172.21.15.161:0/3175368849 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90400cdc00 con 0x7f904013cfc0 2024-05-22T21:18:16.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.719+0000 7f903de00640 1 --2- 172.21.15.161:0/3175368849 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f904015af60 0x7f904013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:16.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.723+0000 7f903e800640 1 --2- 172.21.15.161:0/3175368849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f904013cfc0 0x7f90401402e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f90300097c0 tx=0x7f903001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94609736514efcff server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:16.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.723+0000 7f903d400640 1 -- 172.21.15.161:0/3175368849 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90300ab600 con 0x7f904013cfc0 2024-05-22T21:18:16.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.723+0000 7f903d400640 1 -- 172.21.15.161:0/3175368849 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90300a7050 con 0x7f904013cfc0 2024-05-22T21:18:16.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.723+0000 7f903d400640 1 -- 172.21.15.161:0/3175368849 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90300ab980 con 0x7f904013cfc0 2024-05-22T21:18:16.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.723+0000 7f9045400640 1 -- 172.21.15.161:0/3175368849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f904013cfc0 msgr2=0x7f90401402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:16.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.723+0000 7f9045400640 1 --2- 172.21.15.161:0/3175368849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f904013cfc0 0x7f90401402e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f90300097c0 tx=0x7f903001c410 comp rx=0 tx=0).stop 2024-05-22T21:18:16.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.723+0000 7f9045400640 1 -- 172.21.15.161:0/3175368849 shutdown_connections 2024-05-22T21:18:16.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.723+0000 7f9045400640 1 --2- 172.21.15.161:0/3175368849 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f904015af60 0x7f904013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.723+0000 7f9045400640 1 --2- 172.21.15.161:0/3175368849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f904013cfc0 0x7f90401402e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.723+0000 7f9045400640 1 -- 172.21.15.161:0/3175368849 >> 172.21.15.161:0/3175368849 conn(0x7f904006a2d0 msgr2=0x7f904006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:16.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.723+0000 7f9045400640 1 -- 172.21.15.161:0/3175368849 shutdown_connections 2024-05-22T21:18:16.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.723+0000 7f9045400640 1 -- 172.21.15.161:0/3175368849 wait complete. 2024-05-22T21:18:16.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f9045400640 1 Processor -- start 2024-05-22T21:18:16.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f9045400640 1 -- start start 2024-05-22T21:18:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f9045400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f904013cfc0 0x7f9040140c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f9045400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f904013fc80 0x7f904013ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f9045400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f904015af60 0x7f9040134760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f9045400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9040141e50 con 0x7f904015af60 2024-05-22T21:18:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f9045400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904013f120 con 0x7f904013fc80 2024-05-22T21:18:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f9045400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9040157e20 con 0x7f904013cfc0 2024-05-22T21:18:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f903f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f904015af60 0x7f9040134760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f903f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f904015af60 0x7f9040134760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38440/0 (socket says 172.21.15.161:38440) 2024-05-22T21:18:16.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f903f200640 1 -- 172.21.15.161:0/1558012634 learned_addr learned my addr 172.21.15.161:0/1558012634 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:16.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f903de00640 1 --2- 172.21.15.161:0/1558012634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f904013fc80 0x7f904013ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:16.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f903e800640 1 --2- 172.21.15.161:0/1558012634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f904013cfc0 0x7f9040140c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:16.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f903f200640 1 -- 172.21.15.161:0/1558012634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f904013cfc0 msgr2=0x7f9040140c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:16.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f903f200640 1 --2- 172.21.15.161:0/1558012634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f904013cfc0 0x7f9040140c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f903f200640 1 -- 172.21.15.161:0/1558012634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f904013fc80 msgr2=0x7f904013ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:16.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f903f200640 1 --2- 172.21.15.161:0/1558012634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f904013fc80 0x7f904013ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:16.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f903f200640 1 -- 172.21.15.161:0/1558012634 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c009420 con 0x7f904015af60 2024-05-22T21:18:16.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f903f200640 1 --2- 172.21.15.161:0/1558012634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f904015af60 0x7f9040134760 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f903800cac0 tx=0x7f903800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:16.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f9036a00640 1 -- 172.21.15.161:0/1558012634 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903800ac30 con 0x7f904015af60 2024-05-22T21:18:16.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f9036a00640 1 -- 172.21.15.161:0/1558012634 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903800adb0 con 0x7f904015af60 2024-05-22T21:18:16.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f9045400640 1 -- 172.21.15.161:0/1558012634 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9030009420 con 0x7f904015af60 2024-05-22T21:18:16.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f9036a00640 1 -- 172.21.15.161:0/1558012634 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038012640 con 0x7f904015af60 2024-05-22T21:18:16.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f9045400640 1 -- 172.21.15.161:0/1558012634 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9040141520 con 0x7f904015af60 2024-05-22T21:18:16.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.727+0000 7f9045400640 1 -- 172.21.15.161:0/1558012634 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9040142000 con 0x7f904015af60 2024-05-22T21:18:16.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.731+0000 7f9036a00640 1 -- 172.21.15.161:0/1558012634 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9038007070 con 0x7f904015af60 2024-05-22T21:18:16.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.731+0000 7f9036a00640 1 --2- 172.21.15.161:0/1558012634 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90200666c0 0x7f9020068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:16.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.731+0000 7f9036a00640 1 -- 172.21.15.161:0/1558012634 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9038083da0 con 0x7f904015af60 2024-05-22T21:18:16.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.731+0000 7f903e800640 1 --2- 172.21.15.161:0/1558012634 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90200666c0 0x7f9020068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:16.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.731+0000 7f903e800640 1 --2- 172.21.15.161:0/1558012634 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90200666c0 0x7f9020068b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f90300ad000 tx=0x7f9030008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:16.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.735+0000 7f9036a00640 1 -- 172.21.15.161:0/1558012634 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9040142000 con 0x7f904015af60 2024-05-22T21:18:16.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:16.891+0000 7f9045400640 1 -- 172.21.15.161:0/1558012634 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f90401400d0 con 0x7f904015af60 2024-05-22T21:18:17.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.555+0000 7f9036a00640 1 -- 172.21.15.161:0/1558012634 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) v1 ==== 170+0+0 (secure 0 0 0) 0x7f90401400d0 con 0x7f904015af60 2024-05-22T21:18:17.560 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' created 2024-05-22T21:18:17.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.563+0000 7f9045400640 1 -- 172.21.15.161:0/1558012634 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90200666c0 msgr2=0x7f9020068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:17.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.563+0000 7f9045400640 1 --2- 172.21.15.161:0/1558012634 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90200666c0 0x7f9020068b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f90300ad000 tx=0x7f9030008040 comp rx=0 tx=0).stop 2024-05-22T21:18:17.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.563+0000 7f9045400640 1 -- 172.21.15.161:0/1558012634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f904015af60 msgr2=0x7f9040134760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:17.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.563+0000 7f9045400640 1 --2- 172.21.15.161:0/1558012634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f904015af60 0x7f9040134760 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f903800cac0 tx=0x7f903800cf80 comp rx=0 tx=0).stop 2024-05-22T21:18:17.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.563+0000 7f9045400640 1 -- 172.21.15.161:0/1558012634 shutdown_connections 2024-05-22T21:18:17.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.563+0000 7f9045400640 1 --2- 172.21.15.161:0/1558012634 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90200666c0 0x7f9020068b80 secure :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f90300ad000 tx=0x7f9030008040 comp rx=0 tx=0).stop 2024-05-22T21:18:17.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.563+0000 7f9045400640 1 --2- 172.21.15.161:0/1558012634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f904015af60 0x7f9040134760 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:17.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.563+0000 7f9045400640 1 --2- 172.21.15.161:0/1558012634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f904013fc80 0x7f904013ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:17.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.563+0000 7f9045400640 1 --2- 172.21.15.161:0/1558012634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f904013cfc0 0x7f9040140c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:17.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.563+0000 7f9045400640 1 -- 172.21.15.161:0/1558012634 >> 172.21.15.161:0/1558012634 conn(0x7f904006a2d0 msgr2=0x7f90400cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:17.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.567+0000 7f9045400640 1 -- 172.21.15.161:0/1558012634 shutdown_connections 2024-05-22T21:18:17.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.571+0000 7f9045400640 1 -- 172.21.15.161:0/1558012634 wait complete. 2024-05-22T21:18:17.595 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-22T21:18:17.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.731+0000 7f0911000640 1 Processor -- start 2024-05-22T21:18:17.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.731+0000 7f0911000640 1 -- start start 2024-05-22T21:18:17.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.731+0000 7f0911000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f090c143c40 0x7f090c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:17.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.731+0000 7f0911000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f090c1444a0 0x7f090c06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.731+0000 7f0911000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090c15efb0 con 0x7f090c144900 2024-05-22T21:18:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.731+0000 7f0911000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090c15f290 con 0x7f090c1444a0 2024-05-22T21:18:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.731+0000 7f0911000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090c15f570 con 0x7f090c143c40 2024-05-22T21:18:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.731+0000 7f0909400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f090c1444a0 0x7f090c06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.731+0000 7f0909400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f090c1444a0 0x7f090c06c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38454/0 (socket says 172.21.15.161:38454) 2024-05-22T21:18:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.731+0000 7f0909400640 1 -- 172.21.15.161:0/3570432690 learned_addr learned my addr 172.21.15.161:0/3570432690 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.735+0000 7f0909e00640 1 --2- 172.21.15.161:0/3570432690 >> v2:172.21.15.179:3300/0 conn(0x7f090c143c40 0x7f090c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.735+0000 7f0909400640 1 -- 172.21.15.161:0/3570432690 >> v2:172.21.15.179:3300/0 conn(0x7f090c143c40 msgr2=0x7f090c15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.735+0000 7f0909400640 1 --2- 172.21.15.161:0/3570432690 >> v2:172.21.15.179:3300/0 conn(0x7f090c143c40 0x7f090c15b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.735+0000 7f0909400640 1 -- 172.21.15.161:0/3570432690 >> v1:172.21.15.179:6789/0 conn(0x7f090c144900 legacy=0x7f090c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:17.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.735+0000 7f0909400640 1 -- 172.21.15.161:0/3570432690 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c15f850 con 0x7f090c1444a0 2024-05-22T21:18:17.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.735+0000 7f0909400640 1 --2- 172.21.15.161:0/3570432690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c1444a0 0x7f090c06c2a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f090c0724b0 tx=0x7f08f801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0df62ccd04a9fc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:17.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.735+0000 7f0908a00640 1 -- 172.21.15.161:0/3570432690 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f80a7070 con 0x7f090c1444a0 2024-05-22T21:18:17.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.735+0000 7f0908a00640 1 -- 172.21.15.161:0/3570432690 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f80ab430 con 0x7f090c1444a0 2024-05-22T21:18:17.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.735+0000 7f0908a00640 1 -- 172.21.15.161:0/3570432690 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f80ab7c0 con 0x7f090c1444a0 2024-05-22T21:18:17.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0911000640 1 -- 172.21.15.161:0/3570432690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c1444a0 msgr2=0x7f090c06c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:17.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0911000640 1 --2- 172.21.15.161:0/3570432690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c1444a0 0x7f090c06c2a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f090c0724b0 tx=0x7f08f801c520 comp rx=0 tx=0).stop 2024-05-22T21:18:17.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0911000640 1 -- 172.21.15.161:0/3570432690 shutdown_connections 2024-05-22T21:18:17.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0911000640 1 --2- 172.21.15.161:0/3570432690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c1444a0 0x7f090c06c2a0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:17.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0911000640 1 --2- 172.21.15.161:0/3570432690 >> v2:172.21.15.179:3300/0 conn(0x7f090c143c40 0x7f090c15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:17.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0911000640 1 -- 172.21.15.161:0/3570432690 >> 172.21.15.161:0/3570432690 conn(0x7f090c06a2d0 msgr2=0x7f090c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:17.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0911000640 1 -- 172.21.15.161:0/3570432690 shutdown_connections 2024-05-22T21:18:17.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0911000640 1 -- 172.21.15.161:0/3570432690 wait complete. 2024-05-22T21:18:17.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0911000640 1 Processor -- start 2024-05-22T21:18:17.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0911000640 1 -- start start 2024-05-22T21:18:17.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0911000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f090c143c40 0x7f090c15ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:17.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0911000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f090c1444a0 0x7f090c156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:17.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0911000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c144900 0x7f090c135e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:17.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0911000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c155450 con 0x7f090c144900 2024-05-22T21:18:17.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0911000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c1547d0 con 0x7f090c1444a0 2024-05-22T21:18:17.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0911000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090c154ab0 con 0x7f090c143c40 2024-05-22T21:18:17.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f090a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c144900 0x7f090c135e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:17.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f090a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c144900 0x7f090c135e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38470/0 (socket says 172.21.15.161:38470) 2024-05-22T21:18:17.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f090a800640 1 -- 172.21.15.161:0/3418619055 learned_addr learned my addr 172.21.15.161:0/3418619055 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:17.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0909e00640 1 --2- 172.21.15.161:0/3418619055 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f090c143c40 0x7f090c15ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:17.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f0909400640 1 --2- 172.21.15.161:0/3418619055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f090c1444a0 0x7f090c156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:17.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f090a800640 1 -- 172.21.15.161:0/3418619055 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f090c143c40 msgr2=0x7f090c15ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:17.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f090a800640 1 --2- 172.21.15.161:0/3418619055 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f090c143c40 0x7f090c15ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:17.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f090a800640 1 -- 172.21.15.161:0/3418619055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f090c1444a0 msgr2=0x7f090c156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:17.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f090a800640 1 --2- 172.21.15.161:0/3418619055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f090c1444a0 0x7f090c156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:17.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.739+0000 7f090a800640 1 -- 172.21.15.161:0/3418619055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f80094d0 con 0x7f090c144900 2024-05-22T21:18:17.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.743+0000 7f090a800640 1 --2- 172.21.15.161:0/3418619055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c144900 0x7f090c135e20 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f090400c310 tx=0x7f090400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:17.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.743+0000 7f0902000640 1 -- 172.21.15.161:0/3418619055 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09040052d0 con 0x7f090c144900 2024-05-22T21:18:17.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.743+0000 7f0902000640 1 -- 172.21.15.161:0/3418619055 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0904005430 con 0x7f090c144900 2024-05-22T21:18:17.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.743+0000 7f0902000640 1 -- 172.21.15.161:0/3418619055 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904003800 con 0x7f090c144900 2024-05-22T21:18:17.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.743+0000 7f0911000640 1 -- 172.21.15.161:0/3418619055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090c154d90 con 0x7f090c144900 2024-05-22T21:18:17.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.743+0000 7f0911000640 1 -- 172.21.15.161:0/3418619055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c1574f0 con 0x7f090c144900 2024-05-22T21:18:17.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.743+0000 7f0911000640 1 -- 172.21.15.161:0/3418619055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090c1576e0 con 0x7f090c144900 2024-05-22T21:18:17.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.743+0000 7f0902000640 1 -- 172.21.15.161:0/3418619055 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0904003960 con 0x7f090c144900 2024-05-22T21:18:17.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.747+0000 7f0902000640 1 --2- 172.21.15.161:0/3418619055 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f08ec066690 0x7f08ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:17.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.747+0000 7f0902000640 1 -- 172.21.15.161:0/3418619055 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f09040837e0 con 0x7f090c144900 2024-05-22T21:18:17.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.747+0000 7f0909e00640 1 --2- 172.21.15.161:0/3418619055 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f08ec066690 0x7f08ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:17.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.747+0000 7f0909e00640 1 --2- 172.21.15.161:0/3418619055 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f08ec066690 0x7f08ec068b50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f08fc003b90 tx=0x7f08fc0098a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:17.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.747+0000 7f0902000640 1 -- 172.21.15.161:0/3418619055 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0904050cd0 con 0x7f090c144900 2024-05-22T21:18:17.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.935+0000 7f0911000640 1 -- 172.21.15.161:0/3418619055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f090c155590 con 0x7f090c144900 2024-05-22T21:18:17.950 INFO:teuthology.orchestra.run.smithi161.stderr:new fs with metadata pool 12 and data pool 13 2024-05-22T21:18:17.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.943+0000 7f0902000640 1 -- 172.21.15.161:0/3418619055 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v43) v1 ==== 174+0+0 (secure 0 0 0) 0x7f090c155590 con 0x7f090c144900 2024-05-22T21:18:17.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.951+0000 7f0911000640 1 -- 172.21.15.161:0/3418619055 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f08ec066690 msgr2=0x7f08ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:17.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.951+0000 7f0911000640 1 --2- 172.21.15.161:0/3418619055 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f08ec066690 0x7f08ec068b50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f08fc003b90 tx=0x7f08fc0098a0 comp rx=0 tx=0).stop 2024-05-22T21:18:17.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.951+0000 7f0911000640 1 -- 172.21.15.161:0/3418619055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c144900 msgr2=0x7f090c135e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:17.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.951+0000 7f0911000640 1 --2- 172.21.15.161:0/3418619055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c144900 0x7f090c135e20 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f090400c310 tx=0x7f090400c7d0 comp rx=0 tx=0).stop 2024-05-22T21:18:17.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.951+0000 7f0911000640 1 -- 172.21.15.161:0/3418619055 shutdown_connections 2024-05-22T21:18:17.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.951+0000 7f0911000640 1 --2- 172.21.15.161:0/3418619055 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f08ec066690 0x7f08ec068b50 secure :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f08fc003b90 tx=0x7f08fc0098a0 comp rx=0 tx=0).stop 2024-05-22T21:18:17.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.951+0000 7f0911000640 1 --2- 172.21.15.161:0/3418619055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c144900 0x7f090c135e20 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:17.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.951+0000 7f0911000640 1 --2- 172.21.15.161:0/3418619055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f090c1444a0 0x7f090c156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:17.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.951+0000 7f0911000640 1 --2- 172.21.15.161:0/3418619055 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f090c143c40 0x7f090c15ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:17.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.951+0000 7f0911000640 1 -- 172.21.15.161:0/3418619055 >> 172.21.15.161:0/3418619055 conn(0x7f090c06a2d0 msgr2=0x7f090c0cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:17.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.951+0000 7f0911000640 1 -- 172.21.15.161:0/3418619055 shutdown_connections 2024-05-22T21:18:17.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:17.951+0000 7f0911000640 1 -- 172.21.15.161:0/3418619055 wait complete. 2024-05-22T21:18:17.985 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:18:18.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f763a800640 1 Processor -- start 2024-05-22T21:18:18.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f763a800640 1 -- start start 2024-05-22T21:18:18.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f763a800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7634151d00 0x7f76340ae930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f763a800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7634070e00 0x7f763415bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f763a800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763406a050 con 0x7f76341520e0 2024-05-22T21:18:18.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f763a800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763406d440 con 0x7f7634070e00 2024-05-22T21:18:18.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f763a800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763406a330 con 0x7f7634151d00 2024-05-22T21:18:18.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7638a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f76341520e0 0x7f76340706f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:40240/0 (socket says 172.21.15.161:40240) 2024-05-22T21:18:18.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7638a00640 1 -- 172.21.15.161:0/1065205189 learned_addr learned my addr 172.21.15.161:0/1065205189 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:18.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7639e00640 1 --2- 172.21.15.161:0/1065205189 >> v2:172.21.15.179:3300/0 conn(0x7f7634070e00 0x7f763415bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7639400640 1 --2- 172.21.15.161:0/1065205189 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7634151d00 0x7f76340ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7633e00640 1 -- 172.21.15.161:0/1065205189 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965595593 0 0) 0x7f763406a050 con 0x7f76341520e0 2024-05-22T21:18:18.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7633e00640 1 -- 172.21.15.161:0/1065205189 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f761c003530 con 0x7f76341520e0 2024-05-22T21:18:18.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7633e00640 1 -- 172.21.15.161:0/1065205189 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1058283667 0 0) 0x7f761c003530 con 0x7f76341520e0 2024-05-22T21:18:18.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7633e00640 1 -- 172.21.15.161:0/1065205189 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7634151d00 msgr2=0x7f76340ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:18.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7633e00640 1 --2- 172.21.15.161:0/1065205189 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7634151d00 0x7f76340ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7633e00640 1 -- 172.21.15.161:0/1065205189 >> v2:172.21.15.179:3300/0 conn(0x7f7634070e00 msgr2=0x7f763415bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:18.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7633e00640 1 --2- 172.21.15.161:0/1065205189 >> v2:172.21.15.179:3300/0 conn(0x7f7634070e00 0x7f763415bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7633e00640 1 -- 172.21.15.161:0/1065205189 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763406a610 con 0x7f76341520e0 2024-05-22T21:18:18.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7633e00640 1 -- 172.21.15.161:0/1065205189 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f7620001680 con 0x7f76341520e0 2024-05-22T21:18:18.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7639e00640 1 --2- 172.21.15.161:0/1065205189 >> v2:172.21.15.179:3300/0 conn(0x7f7634070e00 0x7f763415bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:18.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7633e00640 1 -- 172.21.15.161:0/1065205189 >> v1:172.21.15.161:6789/0 conn(0x7f76341520e0 legacy=0x7f76340706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:18.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7633e00640 1 --2- 172.21.15.161:0/1065205189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f761c003980 0x7f761c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7633e00640 1 --2- 172.21.15.161:0/1065205189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c0047d0 0x7f761c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7633e00640 1 --2- 172.21.15.161:0/1065205189 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7634151d00 0x7f761c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7633e00640 1 -- 172.21.15.161:0/1065205189 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763406d440 con 0x7f761c0047d0 2024-05-22T21:18:18.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7633e00640 1 -- 172.21.15.161:0/1065205189 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763406a330 con 0x7f761c003980 2024-05-22T21:18:18.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7633e00640 1 -- 172.21.15.161:0/1065205189 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761c00b990 con 0x7f7634151d00 2024-05-22T21:18:18.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7638a00640 1 --2- 172.21.15.161:0/1065205189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c0047d0 0x7f761c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7639400640 1 --2- 172.21.15.161:0/1065205189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f761c003980 0x7f761c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7639e00640 1 --2- 172.21.15.161:0/1065205189 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7634151d00 0x7f761c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7638a00640 1 -- 172.21.15.161:0/1065205189 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7634151d00 msgr2=0x7f761c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:18.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7638a00640 1 --2- 172.21.15.161:0/1065205189 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7634151d00 0x7f761c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7638a00640 1 -- 172.21.15.161:0/1065205189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f761c003980 msgr2=0x7f761c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:18.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7638a00640 1 --2- 172.21.15.161:0/1065205189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f761c003980 0x7f761c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.119+0000 7f7638a00640 1 -- 172.21.15.161:0/1065205189 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761c003530 con 0x7f761c0047d0 2024-05-22T21:18:18.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.123+0000 7f7638a00640 1 --2- 172.21.15.161:0/1065205189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c0047d0 0x7f761c004c50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f762000c240 tx=0x7f762001f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26db92a497833c35 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:18.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.123+0000 7f7633e00640 1 -- 172.21.15.161:0/1065205189 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7620005ae0 con 0x7f761c0047d0 2024-05-22T21:18:18.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.123+0000 7f7633e00640 1 -- 172.21.15.161:0/1065205189 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76200ab070 con 0x7f761c0047d0 2024-05-22T21:18:18.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.123+0000 7f7633e00640 1 -- 172.21.15.161:0/1065205189 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7620005ea0 con 0x7f761c0047d0 2024-05-22T21:18:18.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.123+0000 7f763a800640 1 -- 172.21.15.161:0/1065205189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c0047d0 msgr2=0x7f761c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:18.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.123+0000 7f763a800640 1 --2- 172.21.15.161:0/1065205189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c0047d0 0x7f761c004c50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f762000c240 tx=0x7f762001f3f0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.123+0000 7f7639e00640 1 -- 172.21.15.161:0/1065205189 reap_dead start 2024-05-22T21:18:18.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.123+0000 7f763a800640 1 -- 172.21.15.161:0/1065205189 shutdown_connections 2024-05-22T21:18:18.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.123+0000 7f763a800640 1 -- 172.21.15.161:0/1065205189 >> 172.21.15.161:0/1065205189 conn(0x7f76340b1570 msgr2=0x7f76340b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:18.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.123+0000 7f763a800640 1 -- 172.21.15.161:0/1065205189 shutdown_connections 2024-05-22T21:18:18.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.123+0000 7f763a800640 1 -- 172.21.15.161:0/1065205189 wait complete. 2024-05-22T21:18:18.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f763a800640 1 Processor -- start 2024-05-22T21:18:18.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f763a800640 1 -- start start 2024-05-22T21:18:18.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f763a800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f761c0047d0 0x7f76341f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f763a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763414f510 0x7f76340ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f763a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76340cad60 0x7f76340cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f763a800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76341f2730 con 0x7f763414f510 2024-05-22T21:18:18.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f763a800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763414f080 con 0x7f76340cad60 2024-05-22T21:18:18.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f763a800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763414e7c0 con 0x7f761c0047d0 2024-05-22T21:18:18.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f7638a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763414f510 0x7f76340ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f7638a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763414f510 0x7f76340ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38480/0 (socket says 172.21.15.161:38480) 2024-05-22T21:18:18.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f7638a00640 1 -- 172.21.15.161:0/462346950 learned_addr learned my addr 172.21.15.161:0/462346950 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:18.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f7639e00640 1 --2- 172.21.15.161:0/462346950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76340cad60 0x7f76340cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f7639400640 1 --2- 172.21.15.161:0/462346950 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f761c0047d0 0x7f76341f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f7638a00640 1 -- 172.21.15.161:0/462346950 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f761c0047d0 msgr2=0x7f76341f21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:18.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f7638a00640 1 --2- 172.21.15.161:0/462346950 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f761c0047d0 0x7f76341f21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f7638a00640 1 -- 172.21.15.161:0/462346950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76340cad60 msgr2=0x7f76340cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:18.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f7638a00640 1 --2- 172.21.15.161:0/462346950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76340cad60 0x7f76340cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f7638a00640 1 -- 172.21.15.161:0/462346950 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763414ea30 con 0x7f763414f510 2024-05-22T21:18:18.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f7639e00640 1 --2- 172.21.15.161:0/462346950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76340cad60 0x7f76340cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:18.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f7638a00640 1 --2- 172.21.15.161:0/462346950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763414f510 0x7f76340ca820 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f762000c240 tx=0x7f761c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:18.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f7631600640 1 -- 172.21.15.161:0/462346950 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76200b7070 con 0x7f763414f510 2024-05-22T21:18:18.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f7631600640 1 -- 172.21.15.161:0/462346950 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7620005880 con 0x7f763414f510 2024-05-22T21:18:18.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f763a800640 1 -- 172.21.15.161:0/462346950 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763414ed10 con 0x7f763414f510 2024-05-22T21:18:18.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f7631600640 1 -- 172.21.15.161:0/462346950 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7620005c10 con 0x7f763414f510 2024-05-22T21:18:18.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.127+0000 7f763a800640 1 -- 172.21.15.161:0/462346950 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763414f990 con 0x7f763414f510 2024-05-22T21:18:18.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.131+0000 7f763a800640 1 -- 172.21.15.161:0/462346950 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76340cb140 con 0x7f763414f510 2024-05-22T21:18:18.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.131+0000 7f7631600640 1 -- 172.21.15.161:0/462346950 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f76200ad020 con 0x7f763414f510 2024-05-22T21:18:18.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.131+0000 7f7631600640 1 --2- 172.21.15.161:0/462346950 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7614066690 0x7f7614068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.131+0000 7f7631600640 1 -- 172.21.15.161:0/462346950 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7620120b60 con 0x7f763414f510 2024-05-22T21:18:18.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.131+0000 7f7639400640 1 --2- 172.21.15.161:0/462346950 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7614066690 0x7f7614068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.131+0000 7f7639400640 1 --2- 172.21.15.161:0/462346950 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7614066690 0x7f7614068b50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7628009a80 tx=0x7f7628009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:18.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.135+0000 7f7631600640 1 -- 172.21.15.161:0/462346950 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f76200f1aa0 con 0x7f763414f510 2024-05-22T21:18:18.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.287+0000 7f763a800640 1 -- 172.21.15.161:0/462346950 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76340676a0 con 0x7f763414f510 2024-05-22T21:18:18.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.291+0000 7f7631600640 1 -- 172.21.15.161:0/462346950 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16164 (secure 0 0 0) 0x7f76200c0020 con 0x7f763414f510 2024-05-22T21:18:18.293 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:18.293 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":102,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:18:17.938537+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:18:16.500534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-22T21:18:16.937445+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:18:18.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.291+0000 7f763a800640 1 -- 172.21.15.161:0/462346950 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7614066690 msgr2=0x7f7614068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:18.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.291+0000 7f763a800640 1 --2- 172.21.15.161:0/462346950 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7614066690 0x7f7614068b50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7628009a80 tx=0x7f7628009190 comp rx=0 tx=0).stop 2024-05-22T21:18:18.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.291+0000 7f763a800640 1 -- 172.21.15.161:0/462346950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763414f510 msgr2=0x7f76340ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:18.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.291+0000 7f763a800640 1 --2- 172.21.15.161:0/462346950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763414f510 0x7f76340ca820 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f762000c240 tx=0x7f761c005800 comp rx=0 tx=0).stop 2024-05-22T21:18:18.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.291+0000 7f763a800640 1 -- 172.21.15.161:0/462346950 shutdown_connections 2024-05-22T21:18:18.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.291+0000 7f763a800640 1 --2- 172.21.15.161:0/462346950 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7614066690 0x7f7614068b50 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.291+0000 7f763a800640 1 --2- 172.21.15.161:0/462346950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76340cad60 0x7f76340cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.291+0000 7f763a800640 1 --2- 172.21.15.161:0/462346950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763414f510 0x7f76340ca820 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.291+0000 7f763a800640 1 --2- 172.21.15.161:0/462346950 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f761c0047d0 0x7f76341f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.291+0000 7f763a800640 1 -- 172.21.15.161:0/462346950 >> 172.21.15.161:0/462346950 conn(0x7f76340b1570 msgr2=0x7f76341501d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:18.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.291+0000 7f763a800640 1 -- 172.21.15.161:0/462346950 shutdown_connections 2024-05-22T21:18:18.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.291+0000 7f763a800640 1 -- 172.21.15.161:0/462346950 wait complete. 2024-05-22T21:18:18.314 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:18:18.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c53a00640 1 Processor -- start 2024-05-22T21:18:18.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c53a00640 1 -- start start 2024-05-22T21:18:18.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c53a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2c4c160ae0 0x7f2c4c0cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c53a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c4c0ced60 0x7f2c4c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c53a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4c0b2380 con 0x7f2c4c0d0090 2024-05-22T21:18:18.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c53a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4c0d04d0 con 0x7f2c4c160ae0 2024-05-22T21:18:18.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c53a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c0d1970 con 0x7f2c4c0ced60 2024-05-22T21:18:18.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c4be00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f2c4c0d0090 0x7f2c4c0b3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:40246/0 (socket says 172.21.15.161:40246) 2024-05-22T21:18:18.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c4be00640 1 -- 172.21.15.161:0/277996100 learned_addr learned my addr 172.21.15.161:0/277996100 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:18.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c51000640 1 --2- 172.21.15.161:0/277996100 >> v2:172.21.15.179:3300/0 conn(0x7f2c4c160ae0 0x7f2c4c0cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c51a00640 1 --2- 172.21.15.161:0/277996100 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c4c0ced60 0x7f2c4c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c4b400640 1 -- 172.21.15.161:0/277996100 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028643713 0 0) 0x7f2c4c0b2380 con 0x7f2c4c0d0090 2024-05-22T21:18:18.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c4b400640 1 -- 172.21.15.161:0/277996100 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c30003530 con 0x7f2c4c0d0090 2024-05-22T21:18:18.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c51000640 1 -- 172.21.15.161:0/277996100 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c4c0ced60 msgr2=0x7f2c4c0b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:18.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c51000640 1 --2- 172.21.15.161:0/277996100 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c4c0ced60 0x7f2c4c0b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c4b400640 1 -- 172.21.15.161:0/277996100 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4001818206 0 0) 0x7f2c30003530 con 0x7f2c4c0d0090 2024-05-22T21:18:18.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c51000640 1 -- 172.21.15.161:0/277996100 >> v1:172.21.15.161:6789/0 conn(0x7f2c4c0d0090 legacy=0x7f2c4c0b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:18.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.447+0000 7f2c51000640 1 -- 172.21.15.161:0/277996100 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c0d1c50 con 0x7f2c4c160ae0 2024-05-22T21:18:18.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.451+0000 7f2c51000640 1 --2- 172.21.15.161:0/277996100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c4c160ae0 0x7f2c4c0cfb50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0097c0 tx=0x7f2c4c0b4570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ddda947efb41d48 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:18.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.451+0000 7f2c4b400640 1 -- 172.21.15.161:0/277996100 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c340016a0 con 0x7f2c4c160ae0 2024-05-22T21:18:18.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.451+0000 7f2c4b400640 1 -- 172.21.15.161:0/277996100 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c0a7050 con 0x7f2c4c160ae0 2024-05-22T21:18:18.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.451+0000 7f2c4b400640 1 -- 172.21.15.161:0/277996100 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c0ab600 con 0x7f2c4c160ae0 2024-05-22T21:18:18.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.451+0000 7f2c53a00640 1 -- 172.21.15.161:0/277996100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c4c160ae0 msgr2=0x7f2c4c0cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:18.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.451+0000 7f2c53a00640 1 --2- 172.21.15.161:0/277996100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c4c160ae0 0x7f2c4c0cfb50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0097c0 tx=0x7f2c4c0b4570 comp rx=0 tx=0).stop 2024-05-22T21:18:18.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.451+0000 7f2c53a00640 1 -- 172.21.15.161:0/277996100 shutdown_connections 2024-05-22T21:18:18.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.451+0000 7f2c53a00640 1 --2- 172.21.15.161:0/277996100 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c4c0ced60 0x7f2c4c0b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.451+0000 7f2c53a00640 1 --2- 172.21.15.161:0/277996100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c4c160ae0 0x7f2c4c0cfb50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.451+0000 7f2c53a00640 1 -- 172.21.15.161:0/277996100 >> 172.21.15.161:0/277996100 conn(0x7f2c4c09c490 msgr2=0x7f2c4c0d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:18.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.451+0000 7f2c53a00640 1 -- 172.21.15.161:0/277996100 shutdown_connections 2024-05-22T21:18:18.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.451+0000 7f2c53a00640 1 -- 172.21.15.161:0/277996100 wait complete. 2024-05-22T21:18:18.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c53a00640 1 Processor -- start 2024-05-22T21:18:18.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c53a00640 1 -- start start 2024-05-22T21:18:18.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c53a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c4c0ced60 0x7f2c4c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c53a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c4c0d0090 0x7f2c4c153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c53a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c4c160ae0 0x7f2c4c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c53a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c09cf00 con 0x7f2c4c0ced60 2024-05-22T21:18:18.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c53a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c154f50 con 0x7f2c4c160ae0 2024-05-22T21:18:18.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c53a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c155230 con 0x7f2c4c0d0090 2024-05-22T21:18:18.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c51000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c4c0ced60 0x7f2c4c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c51000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c4c0ced60 0x7f2c4c0b89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38492/0 (socket says 172.21.15.161:38492) 2024-05-22T21:18:18.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c51000640 1 -- 172.21.15.161:0/2573226646 learned_addr learned my addr 172.21.15.161:0/2573226646 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:18.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c51a00640 1 --2- 172.21.15.161:0/2573226646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c4c160ae0 0x7f2c4c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c4be00640 1 --2- 172.21.15.161:0/2573226646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c4c0d0090 0x7f2c4c153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c51000640 1 -- 172.21.15.161:0/2573226646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c4c0d0090 msgr2=0x7f2c4c153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:18.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c51000640 1 --2- 172.21.15.161:0/2573226646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c4c0d0090 0x7f2c4c153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c51000640 1 -- 172.21.15.161:0/2573226646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c4c160ae0 msgr2=0x7f2c4c0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:18.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c51000640 1 --2- 172.21.15.161:0/2573226646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c4c160ae0 0x7f2c4c0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c51000640 1 -- 172.21.15.161:0/2573226646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c009420 con 0x7f2c4c0ced60 2024-05-22T21:18:18.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c4be00640 1 --2- 172.21.15.161:0/2573226646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c4c0d0090 0x7f2c4c153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:18.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c51000640 1 --2- 172.21.15.161:0/2573226646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c4c0ced60 0x7f2c4c0b89e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c000c10 tx=0x7f2c3c0190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:18.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c48c00640 1 -- 172.21.15.161:0/2573226646 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c0b4400 con 0x7f2c4c0ced60 2024-05-22T21:18:18.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c48c00640 1 -- 172.21.15.161:0/2573226646 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c0b4560 con 0x7f2c4c0ced60 2024-05-22T21:18:18.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.455+0000 7f2c48c00640 1 -- 172.21.15.161:0/2573226646 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c0b48f0 con 0x7f2c4c0ced60 2024-05-22T21:18:18.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.459+0000 7f2c53a00640 1 -- 172.21.15.161:0/2573226646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4c09d150 con 0x7f2c4c0ced60 2024-05-22T21:18:18.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.459+0000 7f2c53a00640 1 -- 172.21.15.161:0/2573226646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4c153dc0 con 0x7f2c4c0ced60 2024-05-22T21:18:18.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.459+0000 7f2c53a00640 1 -- 172.21.15.161:0/2573226646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4c067220 con 0x7f2c4c0ced60 2024-05-22T21:18:18.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.459+0000 7f2c48c00640 1 -- 172.21.15.161:0/2573226646 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2c3c0b2040 con 0x7f2c4c0ced60 2024-05-22T21:18:18.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.459+0000 7f2c48c00640 1 --2- 172.21.15.161:0/2573226646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c28066690 0x7f2c28068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.459+0000 7f2c48c00640 1 -- 172.21.15.161:0/2573226646 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c3c11e1c0 con 0x7f2c4c0ced60 2024-05-22T21:18:18.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.459+0000 7f2c4be00640 1 --2- 172.21.15.161:0/2573226646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c28066690 0x7f2c28068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.459+0000 7f2c4be00640 1 --2- 172.21.15.161:0/2573226646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c28066690 0x7f2c28068b50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2c34001ec0 tx=0x7f2c34001cd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:18.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.463+0000 7f2c48c00640 1 -- 172.21.15.161:0/2573226646 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2c3c0eb5d0 con 0x7f2c4c0ced60 2024-05-22T21:18:18.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.623+0000 7f2c53a00640 1 -- 172.21.15.161:0/2573226646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c4c0d4070 con 0x7f2c4c0ced60 2024-05-22T21:18:18.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.623+0000 7f2c48c00640 1 -- 172.21.15.161:0/2573226646 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16164 (secure 0 0 0) 0x7f2c3c0eebf0 con 0x7f2c4c0ced60 2024-05-22T21:18:18.626 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:18.626 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":102,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:18:17.938537+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:18:16.500534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-22T21:18:16.937445+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:18:18.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.623+0000 7f2c53a00640 1 -- 172.21.15.161:0/2573226646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c28066690 msgr2=0x7f2c28068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:18.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.623+0000 7f2c53a00640 1 --2- 172.21.15.161:0/2573226646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c28066690 0x7f2c28068b50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2c34001ec0 tx=0x7f2c34001cd0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.623+0000 7f2c53a00640 1 -- 172.21.15.161:0/2573226646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c4c0ced60 msgr2=0x7f2c4c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:18.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.623+0000 7f2c53a00640 1 --2- 172.21.15.161:0/2573226646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c4c0ced60 0x7f2c4c0b89e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c000c10 tx=0x7f2c3c0190a0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.623+0000 7f2c53a00640 1 -- 172.21.15.161:0/2573226646 shutdown_connections 2024-05-22T21:18:18.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.623+0000 7f2c53a00640 1 --2- 172.21.15.161:0/2573226646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c28066690 0x7f2c28068b50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.623+0000 7f2c53a00640 1 --2- 172.21.15.161:0/2573226646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c4c160ae0 0x7f2c4c0cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.623+0000 7f2c53a00640 1 --2- 172.21.15.161:0/2573226646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c4c0d0090 0x7f2c4c153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.623+0000 7f2c53a00640 1 --2- 172.21.15.161:0/2573226646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c4c0ced60 0x7f2c4c0b89e0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.623+0000 7f2c53a00640 1 -- 172.21.15.161:0/2573226646 >> 172.21.15.161:0/2573226646 conn(0x7f2c4c09c490 msgr2=0x7f2c4c0b4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:18.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.623+0000 7f2c53a00640 1 -- 172.21.15.161:0/2573226646 shutdown_connections 2024-05-22T21:18:18.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.623+0000 7f2c53a00640 1 -- 172.21.15.161:0/2573226646 wait complete. 2024-05-22T21:18:18.648 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-22T21:18:18.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.783+0000 7f73fe200640 1 Processor -- start 2024-05-22T21:18:18.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.783+0000 7f73fe200640 1 -- start start 2024-05-22T21:18:18.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.783+0000 7f73fe200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73f813cfc0 0x7f73f81402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.783+0000 7f73fe200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f73f815af60 0x7f73f813f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.783+0000 7f73fe200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f81400d0 con 0x7f73f813fc80 2024-05-22T21:18:18.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.783+0000 7f73fe200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f813d3a0 con 0x7f73f815af60 2024-05-22T21:18:18.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.783+0000 7f73fe200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f80cd920 con 0x7f73f813cfc0 2024-05-22T21:18:18.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.787+0000 7f73f6a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f73f815af60 0x7f73f813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.787+0000 7f73f7e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f73f813fc80 0x7f73f815dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:40260/0 (socket says 172.21.15.161:40260) 2024-05-22T21:18:18.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.787+0000 7f73f7e00640 1 -- 172.21.15.161:0/2049264170 learned_addr learned my addr 172.21.15.161:0/2049264170 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:18.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.787+0000 7f73f7400640 1 --2- 172.21.15.161:0/2049264170 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73f813cfc0 0x7f73f81402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.787+0000 7f73f6a00640 1 -- 172.21.15.161:0/2049264170 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73f813cfc0 msgr2=0x7f73f81402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:18.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.787+0000 7f73f6a00640 1 --2- 172.21.15.161:0/2049264170 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73f813cfc0 0x7f73f81402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.787+0000 7f73f6a00640 1 -- 172.21.15.161:0/2049264170 >> v1:172.21.15.161:6789/0 conn(0x7f73f813fc80 legacy=0x7f73f815dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:18.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.787+0000 7f73f6a00640 1 -- 172.21.15.161:0/2049264170 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f80cdc00 con 0x7f73f815af60 2024-05-22T21:18:18.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.787+0000 7f73f6000640 1 -- 172.21.15.161:0/2049264170 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990675840 0 0) 0x7f73f0001120 con 0x7f73f813fc80 2024-05-22T21:18:18.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.787+0000 7f73f6a00640 1 --2- 172.21.15.161:0/2049264170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73f815af60 0x7f73f813f740 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f73e40097c0 tx=0x7f73e401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43182a1342852de9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:18.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.787+0000 7f73f6000640 1 -- 172.21.15.161:0/2049264170 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e40ab600 con 0x7f73f815af60 2024-05-22T21:18:18.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.787+0000 7f73f6000640 1 -- 172.21.15.161:0/2049264170 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e40a7050 con 0x7f73f815af60 2024-05-22T21:18:18.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.787+0000 7f73f6000640 1 -- 172.21.15.161:0/2049264170 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e40ab980 con 0x7f73f815af60 2024-05-22T21:18:18.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73fe200640 1 -- 172.21.15.161:0/2049264170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73f815af60 msgr2=0x7f73f813f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:18.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73fe200640 1 --2- 172.21.15.161:0/2049264170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73f815af60 0x7f73f813f740 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f73e40097c0 tx=0x7f73e401c410 comp rx=0 tx=0).stop 2024-05-22T21:18:18.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73fe200640 1 -- 172.21.15.161:0/2049264170 shutdown_connections 2024-05-22T21:18:18.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73fe200640 1 --2- 172.21.15.161:0/2049264170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73f815af60 0x7f73f813f740 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73fe200640 1 --2- 172.21.15.161:0/2049264170 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73f813cfc0 0x7f73f81402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73fe200640 1 -- 172.21.15.161:0/2049264170 >> 172.21.15.161:0/2049264170 conn(0x7f73f806a2d0 msgr2=0x7f73f806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:18.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73fe200640 1 -- 172.21.15.161:0/2049264170 shutdown_connections 2024-05-22T21:18:18.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73fe200640 1 -- 172.21.15.161:0/2049264170 wait complete. 2024-05-22T21:18:18.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73fe200640 1 Processor -- start 2024-05-22T21:18:18.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73fe200640 1 -- start start 2024-05-22T21:18:18.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73fe200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73f813cfc0 0x7f73f8157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73fe200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73f813fc80 0x7f73f8140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73fe200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73f815af60 0x7f73f815b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73fe200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f80b31e0 con 0x7f73f813cfc0 2024-05-22T21:18:18.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73fe200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f80b25d0 con 0x7f73f813fc80 2024-05-22T21:18:18.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73fe200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f80b28b0 con 0x7f73f815af60 2024-05-22T21:18:18.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73f7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73f813cfc0 0x7f73f8157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73f7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73f813cfc0 0x7f73f8157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38508/0 (socket says 172.21.15.161:38508) 2024-05-22T21:18:18.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73f7400640 1 -- 172.21.15.161:0/362433775 learned_addr learned my addr 172.21.15.161:0/362433775 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:18.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73f6a00640 1 --2- 172.21.15.161:0/362433775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73f813fc80 0x7f73f8140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73f7e00640 1 --2- 172.21.15.161:0/362433775 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73f815af60 0x7f73f815b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73f7400640 1 -- 172.21.15.161:0/362433775 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73f815af60 msgr2=0x7f73f815b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:18.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73f7400640 1 --2- 172.21.15.161:0/362433775 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73f815af60 0x7f73f815b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73f7400640 1 -- 172.21.15.161:0/362433775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73f813fc80 msgr2=0x7f73f8140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:18.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73f7400640 1 --2- 172.21.15.161:0/362433775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73f813fc80 0x7f73f8140270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:18.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73f7400640 1 -- 172.21.15.161:0/362433775 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e4009420 con 0x7f73f813cfc0 2024-05-22T21:18:18.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.791+0000 7f73f7e00640 1 --2- 172.21.15.161:0/362433775 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73f815af60 0x7f73f815b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:18.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.795+0000 7f73f7400640 1 --2- 172.21.15.161:0/362433775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73f813cfc0 0x7f73f8157840 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f73e8007e40 tx=0x7f73e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:18.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.795+0000 7f73ef400640 1 -- 172.21.15.161:0/362433775 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e8014620 con 0x7f73f813cfc0 2024-05-22T21:18:18.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.795+0000 7f73ef400640 1 -- 172.21.15.161:0/362433775 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e800ee90 con 0x7f73f813cfc0 2024-05-22T21:18:18.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.795+0000 7f73fe200640 1 -- 172.21.15.161:0/362433775 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f80b2b90 con 0x7f73f813cfc0 2024-05-22T21:18:18.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.795+0000 7f73ef400640 1 -- 172.21.15.161:0/362433775 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e80149c0 con 0x7f73f813cfc0 2024-05-22T21:18:18.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.795+0000 7f73fe200640 1 -- 172.21.15.161:0/362433775 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f8157d80 con 0x7f73f813cfc0 2024-05-22T21:18:18.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.795+0000 7f73fe200640 1 -- 172.21.15.161:0/362433775 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f8067220 con 0x7f73f813cfc0 2024-05-22T21:18:18.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.795+0000 7f73ef400640 1 -- 172.21.15.161:0/362433775 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f73e8014b20 con 0x7f73f813cfc0 2024-05-22T21:18:18.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.795+0000 7f73ef400640 1 --2- 172.21.15.161:0/362433775 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73d8066690 0x7f73d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:18.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.795+0000 7f73ef400640 1 -- 172.21.15.161:0/362433775 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73e8084bb0 con 0x7f73f813cfc0 2024-05-22T21:18:18.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.795+0000 7f73f6a00640 1 --2- 172.21.15.161:0/362433775 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73d8066690 0x7f73d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:18.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.799+0000 7f73f6a00640 1 --2- 172.21.15.161:0/362433775 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73d8066690 0x7f73d8068b50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f73e40ad000 tx=0x7f73e40a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:18.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.799+0000 7f73ef400640 1 -- 172.21.15.161:0/362433775 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f73e8051e80 con 0x7f73f813cfc0 2024-05-22T21:18:18.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:18.991+0000 7f73fe200640 1 -- 172.21.15.161:0/362433775 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f73f80ad5d0 con 0x7f73f813cfc0 2024-05-22T21:18:19.485 INFO:tasks.ceph.mds.c.smithi161.stderr:2024-05-22T21:18:19.483+0000 7f0fed200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T21:18:19.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:19.959+0000 7f73ef400640 1 -- 172.21.15.161:0/362433775 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) v1 ==== 118+0+0 (secure 0 0 0) 0x7f73e80554a0 con 0x7f73f813cfc0 2024-05-22T21:18:19.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:19.963+0000 7f73fe200640 1 -- 172.21.15.161:0/362433775 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73d8066690 msgr2=0x7f73d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:19.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:19.963+0000 7f73fe200640 1 --2- 172.21.15.161:0/362433775 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73d8066690 0x7f73d8068b50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f73e40ad000 tx=0x7f73e40a6000 comp rx=0 tx=0).stop 2024-05-22T21:18:19.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:19.963+0000 7f73fe200640 1 -- 172.21.15.161:0/362433775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73f813cfc0 msgr2=0x7f73f8157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:19.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:19.963+0000 7f73fe200640 1 --2- 172.21.15.161:0/362433775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73f813cfc0 0x7f73f8157840 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f73e8007e40 tx=0x7f73e800e4b0 comp rx=0 tx=0).stop 2024-05-22T21:18:19.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:19.963+0000 7f73fe200640 1 -- 172.21.15.161:0/362433775 shutdown_connections 2024-05-22T21:18:19.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:19.963+0000 7f73fe200640 1 --2- 172.21.15.161:0/362433775 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73d8066690 0x7f73d8068b50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:19.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:19.963+0000 7f73fe200640 1 --2- 172.21.15.161:0/362433775 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73f815af60 0x7f73f815b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:19.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:19.963+0000 7f73fe200640 1 --2- 172.21.15.161:0/362433775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73f813fc80 0x7f73f8140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:19.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:19.963+0000 7f73fe200640 1 --2- 172.21.15.161:0/362433775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73f813cfc0 0x7f73f8157840 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:19.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:19.963+0000 7f73fe200640 1 -- 172.21.15.161:0/362433775 >> 172.21.15.161:0/362433775 conn(0x7f73f806a2d0 msgr2=0x7f73f806a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:19.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:19.963+0000 7f73fe200640 1 -- 172.21.15.161:0/362433775 shutdown_connections 2024-05-22T21:18:19.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:19.963+0000 7f73fe200640 1 -- 172.21.15.161:0/362433775 wait complete. 2024-05-22T21:18:19.987 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-22T21:18:19.987 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-22T21:18:20.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.123+0000 7f1c01800640 1 Processor -- start 2024-05-22T21:18:20.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.123+0000 7f1c01800640 1 -- start start 2024-05-22T21:18:20.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.123+0000 7f1c01800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1bfc0c3ef0 0x7f1bfc0c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:20.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.123+0000 7f1c01800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bfc0add70 con 0x7f1bfc0ae200 2024-05-22T21:18:20.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.123+0000 7f1c01800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bfc0c4810 con 0x7f1bfc0b02c0 2024-05-22T21:18:20.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.123+0000 7f1c01800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bfc161f50 con 0x7f1bfc0c3ef0 2024-05-22T21:18:20.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.123+0000 7f1bfb400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1bfc0ae200 0x7f1bfc0ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:40266/0 (socket says 172.21.15.161:40266) 2024-05-22T21:18:20.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.123+0000 7f1bfb400640 1 -- 172.21.15.161:0/3972501622 learned_addr learned my addr 172.21.15.161:0/3972501622 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:20.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.123+0000 7f1bfaa00640 1 --2- 172.21.15.161:0/3972501622 >> v2:172.21.15.179:3300/0 conn(0x7f1bfc0c3ef0 0x7f1bfc0c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:20.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.123+0000 7f1bf9600640 1 -- 172.21.15.161:0/3972501622 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4095134438 0 0) 0x7f1bfc0add70 con 0x7f1bfc0ae200 2024-05-22T21:18:20.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.123+0000 7f1bf9600640 1 -- 172.21.15.161:0/3972501622 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1be0003530 con 0x7f1bfc0ae200 2024-05-22T21:18:20.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.123+0000 7f1bf9600640 1 -- 172.21.15.161:0/3972501622 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 279800493 0 0) 0x7f1be0003530 con 0x7f1bfc0ae200 2024-05-22T21:18:20.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.123+0000 7f1bfaa00640 1 -- 172.21.15.161:0/3972501622 >> v1:172.21.15.179:6789/0 conn(0x7f1bfc0b02c0 legacy=0x7f1bfc161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:20.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.123+0000 7f1bfaa00640 1 -- 172.21.15.161:0/3972501622 >> v1:172.21.15.161:6789/0 conn(0x7f1bfc0ae200 legacy=0x7f1bfc0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:20.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.123+0000 7f1bfaa00640 1 -- 172.21.15.161:0/3972501622 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc162170 con 0x7f1bfc0c3ef0 2024-05-22T21:18:20.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.127+0000 7f1bfaa00640 1 --2- 172.21.15.161:0/3972501622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bfc0c3ef0 0x7f1bfc0c42d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1bec009d90 tx=0x7f1bec01c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95ea29b42909d0ff server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:20.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.127+0000 7f1bf9600640 1 -- 172.21.15.161:0/3972501622 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4001680 con 0x7f1bfc0c3ef0 2024-05-22T21:18:20.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.127+0000 7f1bf9600640 1 -- 172.21.15.161:0/3972501622 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bec0a56a0 con 0x7f1bfc0c3ef0 2024-05-22T21:18:20.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.127+0000 7f1bf9600640 1 -- 172.21.15.161:0/3972501622 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec0a5a30 con 0x7f1bfc0c3ef0 2024-05-22T21:18:20.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.127+0000 7f1c01800640 1 -- 172.21.15.161:0/3972501622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bfc0c3ef0 msgr2=0x7f1bfc0c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:20.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.127+0000 7f1c01800640 1 --2- 172.21.15.161:0/3972501622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bfc0c3ef0 0x7f1bfc0c42d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1bec009d90 tx=0x7f1bec01c860 comp rx=0 tx=0).stop 2024-05-22T21:18:20.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.127+0000 7f1c01800640 1 -- 172.21.15.161:0/3972501622 shutdown_connections 2024-05-22T21:18:20.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.127+0000 7f1c01800640 1 --2- 172.21.15.161:0/3972501622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bfc0c3ef0 0x7f1bfc0c42d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:20.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.127+0000 7f1c01800640 1 -- 172.21.15.161:0/3972501622 >> 172.21.15.161:0/3972501622 conn(0x7f1bfc0d0f20 msgr2=0x7f1bfc0d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:20.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.127+0000 7f1c01800640 1 -- 172.21.15.161:0/3972501622 shutdown_connections 2024-05-22T21:18:20.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.127+0000 7f1c01800640 1 -- 172.21.15.161:0/3972501622 wait complete. 2024-05-22T21:18:20.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1c01800640 1 Processor -- start 2024-05-22T21:18:20.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1c01800640 1 -- start start 2024-05-22T21:18:20.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1c01800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bfc0ae200 0x7f1bfc154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:20.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1c01800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1bfc0b02c0 0x7f1bfc0b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:20.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1c01800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bfc0c3ef0 0x7f1bfc0b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:20.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1c01800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc1548b0 con 0x7f1bfc0ae200 2024-05-22T21:18:20.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1c01800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc0b4c10 con 0x7f1bfc0c3ef0 2024-05-22T21:18:20.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1c01800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc1fba20 con 0x7f1bfc0b02c0 2024-05-22T21:18:20.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1bfaa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bfc0ae200 0x7f1bfc154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:20.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1bfaa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bfc0ae200 0x7f1bfc154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38512/0 (socket says 172.21.15.161:38512) 2024-05-22T21:18:20.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1bfaa00640 1 -- 172.21.15.161:0/4223720054 learned_addr learned my addr 172.21.15.161:0/4223720054 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:20.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1bfaa00640 1 -- 172.21.15.161:0/4223720054 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1bfc0b02c0 msgr2=0x7f1bfc0b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:20.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1bfaa00640 1 --2- 172.21.15.161:0/4223720054 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1bfc0b02c0 0x7f1bfc0b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:20.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1bfaa00640 1 -- 172.21.15.161:0/4223720054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bfc0c3ef0 msgr2=0x7f1bfc0b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:20.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1bfaa00640 1 --2- 172.21.15.161:0/4223720054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bfc0c3ef0 0x7f1bfc0b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:20.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1bfaa00640 1 -- 172.21.15.161:0/4223720054 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec009a70 con 0x7f1bfc0ae200 2024-05-22T21:18:20.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1bfaa00640 1 --2- 172.21.15.161:0/4223720054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bfc0ae200 0x7f1bfc154370 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0b0040 tx=0x7f1bec019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:20.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1bf2a00640 1 -- 172.21.15.161:0/4223720054 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec003ba0 con 0x7f1bfc0ae200 2024-05-22T21:18:20.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1bf2a00640 1 -- 172.21.15.161:0/4223720054 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bec003d20 con 0x7f1bfc0ae200 2024-05-22T21:18:20.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1bf2a00640 1 -- 172.21.15.161:0/4223720054 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec0ad890 con 0x7f1bfc0ae200 2024-05-22T21:18:20.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1c01800640 1 -- 172.21.15.161:0/4223720054 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bfc0c55b0 con 0x7f1bfc0ae200 2024-05-22T21:18:20.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.131+0000 7f1c01800640 1 -- 172.21.15.161:0/4223720054 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bfc0c5870 con 0x7f1bfc0ae200 2024-05-22T21:18:20.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.135+0000 7f1c01800640 1 -- 172.21.15.161:0/4223720054 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bfc067220 con 0x7f1bfc0ae200 2024-05-22T21:18:20.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.135+0000 7f1bf2a00640 1 -- 172.21.15.161:0/4223720054 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1bec0b4050 con 0x7f1bfc0ae200 2024-05-22T21:18:20.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.135+0000 7f1bf2a00640 1 --2- 172.21.15.161:0/4223720054 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bdc0666c0 0x7f1bdc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:20.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.135+0000 7f1bf2a00640 1 -- 172.21.15.161:0/4223720054 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bec11ed80 con 0x7f1bfc0ae200 2024-05-22T21:18:20.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.135+0000 7f1bfa000640 1 --2- 172.21.15.161:0/4223720054 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bdc0666c0 0x7f1bdc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:20.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.135+0000 7f1bfa000640 1 --2- 172.21.15.161:0/4223720054 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bdc0666c0 0x7f1bdc068b80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1be8000f70 tx=0x7f1be80079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:20.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.139+0000 7f1bf2a00640 1 -- 172.21.15.161:0/4223720054 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1bec0ec0e0 con 0x7f1bfc0ae200 2024-05-22T21:18:20.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.327+0000 7f1c01800640 1 -- 172.21.15.161:0/4223720054 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1bfc1549f0 con 0x7f1bfc0ae200 2024-05-22T21:18:20.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.967+0000 7f1bf2a00640 1 -- 172.21.15.161:0/4223720054 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v47) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1bec0b2020 con 0x7f1bfc0ae200 2024-05-22T21:18:20.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.967+0000 7f1c01800640 1 -- 172.21.15.161:0/4223720054 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bdc0666c0 msgr2=0x7f1bdc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:20.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.967+0000 7f1c01800640 1 --2- 172.21.15.161:0/4223720054 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bdc0666c0 0x7f1bdc068b80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1be8000f70 tx=0x7f1be80079c0 comp rx=0 tx=0).stop 2024-05-22T21:18:20.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.967+0000 7f1c01800640 1 -- 172.21.15.161:0/4223720054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bfc0ae200 msgr2=0x7f1bfc154370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:20.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.967+0000 7f1c01800640 1 --2- 172.21.15.161:0/4223720054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bfc0ae200 0x7f1bfc154370 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0b0040 tx=0x7f1bec019600 comp rx=0 tx=0).stop 2024-05-22T21:18:20.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.967+0000 7f1c01800640 1 -- 172.21.15.161:0/4223720054 shutdown_connections 2024-05-22T21:18:20.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.967+0000 7f1c01800640 1 --2- 172.21.15.161:0/4223720054 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1bdc0666c0 0x7f1bdc068b80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:20.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.967+0000 7f1c01800640 1 --2- 172.21.15.161:0/4223720054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1bfc0c3ef0 0x7f1bfc0b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:20.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.967+0000 7f1c01800640 1 --2- 172.21.15.161:0/4223720054 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1bfc0b02c0 0x7f1bfc0b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:20.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.967+0000 7f1c01800640 1 --2- 172.21.15.161:0/4223720054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bfc0ae200 0x7f1bfc154370 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:20.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.967+0000 7f1c01800640 1 -- 172.21.15.161:0/4223720054 >> 172.21.15.161:0/4223720054 conn(0x7f1bfc0d0f20 msgr2=0x7f1bfc06afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:20.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.967+0000 7f1c01800640 1 -- 172.21.15.161:0/4223720054 shutdown_connections 2024-05-22T21:18:20.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:20.967+0000 7f1c01800640 1 -- 172.21.15.161:0/4223720054 wait complete. 2024-05-22T21:18:20.991 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-22T21:18:20.991 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:18:21.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb91400640 1 Processor -- start 2024-05-22T21:18:21.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb91400640 1 -- start start 2024-05-22T21:18:21.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb91400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ffb8c0715a0 0x7ffb8c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb91400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffb8c15ec90 0x7ffb8c15f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb91400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c1442a0 con 0x7ffb8c143ec0 2024-05-22T21:18:21.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb91400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8c15f640 con 0x7ffb8c0715a0 2024-05-22T21:18:21.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb91400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8c070500 con 0x7ffb8c15ec90 2024-05-22T21:18:21.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb89e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ffb8c0715a0 0x7ffb8c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb8a800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ffb8c143ec0 0x7ffb8c06fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:40276/0 (socket says 172.21.15.161:40276) 2024-05-22T21:18:21.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb8a800640 1 -- 172.21.15.161:0/551280104 learned_addr learned my addr 172.21.15.161:0/551280104 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:21.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb8b200640 1 --2- 172.21.15.161:0/551280104 >> v2:172.21.15.179:3300/0 conn(0x7ffb8c15ec90 0x7ffb8c15f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb89e00640 1 -- 172.21.15.161:0/551280104 >> v2:172.21.15.179:3300/0 conn(0x7ffb8c15ec90 msgr2=0x7ffb8c15f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:21.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb89e00640 1 --2- 172.21.15.161:0/551280104 >> v2:172.21.15.179:3300/0 conn(0x7ffb8c15ec90 0x7ffb8c15f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb89e00640 1 -- 172.21.15.161:0/551280104 >> v1:172.21.15.161:6789/0 conn(0x7ffb8c143ec0 legacy=0x7ffb8c06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:21.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb89e00640 1 -- 172.21.15.161:0/551280104 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0707e0 con 0x7ffb8c0715a0 2024-05-22T21:18:21.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb89400640 1 -- 172.21.15.161:0/551280104 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136030861 0 0) 0x7ffb8c1442a0 con 0x7ffb8c143ec0 2024-05-22T21:18:21.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.123+0000 7ffb8b200640 1 --2- 172.21.15.161:0/551280104 >> v2:172.21.15.179:3300/0 conn(0x7ffb8c15ec90 0x7ffb8c15f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:21.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.127+0000 7ffb89e00640 1 --2- 172.21.15.161:0/551280104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb8c0715a0 0x7ffb8c15f830 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0cedc0 tx=0x7ffb7801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed674c527c14010 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:21.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.127+0000 7ffb89400640 1 -- 172.21.15.161:0/551280104 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78004230 con 0x7ffb8c0715a0 2024-05-22T21:18:21.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.127+0000 7ffb89400640 1 -- 172.21.15.161:0/551280104 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb780a7050 con 0x7ffb8c0715a0 2024-05-22T21:18:21.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.127+0000 7ffb89400640 1 -- 172.21.15.161:0/551280104 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78003620 con 0x7ffb8c0715a0 2024-05-22T21:18:21.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.127+0000 7ffb91400640 1 -- 172.21.15.161:0/551280104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb8c0715a0 msgr2=0x7ffb8c15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:21.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.127+0000 7ffb91400640 1 --2- 172.21.15.161:0/551280104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb8c0715a0 0x7ffb8c15f830 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0cedc0 tx=0x7ffb7801c450 comp rx=0 tx=0).stop 2024-05-22T21:18:21.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.127+0000 7ffb91400640 1 -- 172.21.15.161:0/551280104 shutdown_connections 2024-05-22T21:18:21.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.127+0000 7ffb91400640 1 --2- 172.21.15.161:0/551280104 >> v2:172.21.15.179:3300/0 conn(0x7ffb8c15ec90 0x7ffb8c15f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.127+0000 7ffb91400640 1 --2- 172.21.15.161:0/551280104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb8c0715a0 0x7ffb8c15f830 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.127+0000 7ffb91400640 1 -- 172.21.15.161:0/551280104 >> 172.21.15.161:0/551280104 conn(0x7ffb8c0b0bc0 msgr2=0x7ffb8c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:21.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.127+0000 7ffb91400640 1 -- 172.21.15.161:0/551280104 shutdown_connections 2024-05-22T21:18:21.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.127+0000 7ffb91400640 1 -- 172.21.15.161:0/551280104 wait complete. 2024-05-22T21:18:21.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb91400640 1 Processor -- start 2024-05-22T21:18:21.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb91400640 1 -- start start 2024-05-22T21:18:21.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb91400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c143ec0 0x7ffb8c13df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb91400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb8c15ec90 0x7ffb8c13ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb91400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c15a2f0 0x7ffb8c1f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb91400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c13e440 con 0x7ffb8c15ec90 2024-05-22T21:18:21.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb91400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c13bf50 con 0x7ffb8c15a2f0 2024-05-22T21:18:21.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb91400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0b2060 con 0x7ffb8c143ec0 2024-05-22T21:18:21.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb89e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb8c15ec90 0x7ffb8c13ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb89e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb8c15ec90 0x7ffb8c13ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38540/0 (socket says 172.21.15.161:38540) 2024-05-22T21:18:21.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb89e00640 1 -- 172.21.15.161:0/3966458361 learned_addr learned my addr 172.21.15.161:0/3966458361 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:21.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb8a800640 1 --2- 172.21.15.161:0/3966458361 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c143ec0 0x7ffb8c13df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb8b200640 1 --2- 172.21.15.161:0/3966458361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c15a2f0 0x7ffb8c1f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb89e00640 1 -- 172.21.15.161:0/3966458361 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c143ec0 msgr2=0x7ffb8c13df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:21.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb89e00640 1 --2- 172.21.15.161:0/3966458361 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c143ec0 0x7ffb8c13df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb89e00640 1 -- 172.21.15.161:0/3966458361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c15a2f0 msgr2=0x7ffb8c1f71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:21.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb89e00640 1 --2- 172.21.15.161:0/3966458361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c15a2f0 0x7ffb8c1f71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb89e00640 1 -- 172.21.15.161:0/3966458361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb78009420 con 0x7ffb8c15ec90 2024-05-22T21:18:21.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb89e00640 1 --2- 172.21.15.161:0/3966458361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb8c15ec90 0x7ffb8c13ba10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ffb78019060 tx=0x7ffb780191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:21.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb82a00640 1 -- 172.21.15.161:0/3966458361 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78003cf0 con 0x7ffb8c15ec90 2024-05-22T21:18:21.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb82a00640 1 -- 172.21.15.161:0/3966458361 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb78003e50 con 0x7ffb8c15ec90 2024-05-22T21:18:21.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb91400640 1 -- 172.21.15.161:0/3966458361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8c0b2340 con 0x7ffb8c15ec90 2024-05-22T21:18:21.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb82a00640 1 -- 172.21.15.161:0/3966458361 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb780ab660 con 0x7ffb8c15ec90 2024-05-22T21:18:21.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.131+0000 7ffb91400640 1 -- 172.21.15.161:0/3966458361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8c0b2900 con 0x7ffb8c15ec90 2024-05-22T21:18:21.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.135+0000 7ffb91400640 1 -- 172.21.15.161:0/3966458361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c067220 con 0x7ffb8c15ec90 2024-05-22T21:18:21.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.135+0000 7ffb82a00640 1 -- 172.21.15.161:0/3966458361 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffb780b2040 con 0x7ffb8c15ec90 2024-05-22T21:18:21.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.135+0000 7ffb82a00640 1 --2- 172.21.15.161:0/3966458361 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb6c066690 0x7ffb6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.135+0000 7ffb82a00640 1 -- 172.21.15.161:0/3966458361 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb7811e350 con 0x7ffb8c15ec90 2024-05-22T21:18:21.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.135+0000 7ffb8a800640 1 --2- 172.21.15.161:0/3966458361 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb6c066690 0x7ffb6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.135+0000 7ffb8a800640 1 --2- 172.21.15.161:0/3966458361 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb6c066690 0x7ffb6c068b50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c007fd0 tx=0x7ffb7c007ee0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:21.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.139+0000 7ffb82a00640 1 -- 172.21.15.161:0/3966458361 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ffb780eb590 con 0x7ffb8c15ec90 2024-05-22T21:18:21.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.327+0000 7ffb91400640 1 -- 172.21.15.161:0/3966458361 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb8c06dcc0 con 0x7ffb8c15ec90 2024-05-22T21:18:21.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.327+0000 7ffb82a00640 1 -- 172.21.15.161:0/3966458361 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4748 (secure 0 0 0) 0x7ffb780b0090 con 0x7ffb8c15ec90 2024-05-22T21:18:21.331 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:21.331 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2771409351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2771409351},{"type":"v1","addr":"172.21.15.179:6834","nonce":2771409351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6442,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2520119431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2520119431},{"type":"v1","addr":"172.21.15.179:6835","nonce":2520119431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1415362547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1415362547},{"type":"v1","addr":"172.21.15.161:6835","nonce":1415362547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:18:17.938858+0000","modified":"2024-05-22T21:18:20.963065+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/4127130864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4127130864},{"type":"v1","addr":"172.21.15.161:6837","nonce":4127130864}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-22T21:18:21.332 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 47 2024-05-22T21:18:21.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.331+0000 7ffb91400640 1 -- 172.21.15.161:0/3966458361 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb6c066690 msgr2=0x7ffb6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:21.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.331+0000 7ffb91400640 1 --2- 172.21.15.161:0/3966458361 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb6c066690 0x7ffb6c068b50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c007fd0 tx=0x7ffb7c007ee0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.331+0000 7ffb91400640 1 -- 172.21.15.161:0/3966458361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb8c15ec90 msgr2=0x7ffb8c13ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:21.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.331+0000 7ffb91400640 1 --2- 172.21.15.161:0/3966458361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb8c15ec90 0x7ffb8c13ba10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ffb78019060 tx=0x7ffb780191f0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.331+0000 7ffb91400640 1 -- 172.21.15.161:0/3966458361 shutdown_connections 2024-05-22T21:18:21.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.331+0000 7ffb91400640 1 --2- 172.21.15.161:0/3966458361 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ffb6c066690 0x7ffb6c068b50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.331+0000 7ffb91400640 1 --2- 172.21.15.161:0/3966458361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c15a2f0 0x7ffb8c1f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.331+0000 7ffb91400640 1 --2- 172.21.15.161:0/3966458361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb8c15ec90 0x7ffb8c13ba10 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.331+0000 7ffb91400640 1 --2- 172.21.15.161:0/3966458361 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c143ec0 0x7ffb8c13df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.331+0000 7ffb91400640 1 -- 172.21.15.161:0/3966458361 >> 172.21.15.161:0/3966458361 conn(0x7ffb8c0b0bc0 msgr2=0x7ffb8c06c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:21.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.331+0000 7ffb91400640 1 -- 172.21.15.161:0/3966458361 shutdown_connections 2024-05-22T21:18:21.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.331+0000 7ffb91400640 1 -- 172.21.15.161:0/3966458361 wait complete. 2024-05-22T21:18:21.352 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:18:21.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.487+0000 7efd87200640 1 Processor -- start 2024-05-22T21:18:21.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.487+0000 7efd87200640 1 -- start start 2024-05-22T21:18:21.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.487+0000 7efd87200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd8806f1b0 0x7efd88070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.487+0000 7efd87200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efd880af1b0 0x7efd88071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.487+0000 7efd87200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd880b2d90 con 0x7efd8815d2f0 2024-05-22T21:18:21.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.487+0000 7efd87200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd880cf330 con 0x7efd880af1b0 2024-05-22T21:18:21.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.487+0000 7efd87200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd880cf610 con 0x7efd8806f1b0 2024-05-22T21:18:21.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.487+0000 7efd86800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efd880af1b0 0x7efd88071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.487+0000 7efd85e00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7efd8815d2f0 0x7efd8806eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:56646/0 (socket says 172.21.15.161:56646) 2024-05-22T21:18:21.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.487+0000 7efd85e00640 1 -- 172.21.15.161:0/3036823585 learned_addr learned my addr 172.21.15.161:0/3036823585 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:21.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.487+0000 7efd7d400640 1 --2- 172.21.15.161:0/3036823585 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd8806f1b0 0x7efd88070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.487+0000 7efd86800640 1 -- 172.21.15.161:0/3036823585 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd8806f1b0 msgr2=0x7efd88070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:21.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.487+0000 7efd86800640 1 --2- 172.21.15.161:0/3036823585 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd8806f1b0 0x7efd88070af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.487+0000 7efd86800640 1 -- 172.21.15.161:0/3036823585 >> v1:172.21.15.179:6789/0 conn(0x7efd8815d2f0 legacy=0x7efd8806eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:21.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.487+0000 7efd86800640 1 -- 172.21.15.161:0/3036823585 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd880cf8f0 con 0x7efd880af1b0 2024-05-22T21:18:21.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.487+0000 7efd85400640 1 -- 172.21.15.161:0/3036823585 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673919974 0 0) 0x7efd880b2d90 con 0x7efd8815d2f0 2024-05-22T21:18:21.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.491+0000 7efd86800640 1 --2- 172.21.15.161:0/3036823585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd880af1b0 0x7efd88071030 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efd80009d90 tx=0x7efd8001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df8cf2ccbf008c0e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:21.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.491+0000 7efd85400640 1 -- 172.21.15.161:0/3036823585 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd800a5a30 con 0x7efd880af1b0 2024-05-22T21:18:21.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.491+0000 7efd85400640 1 -- 172.21.15.161:0/3036823585 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd800a5b90 con 0x7efd880af1b0 2024-05-22T21:18:21.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.491+0000 7efd85400640 1 -- 172.21.15.161:0/3036823585 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd800ad440 con 0x7efd880af1b0 2024-05-22T21:18:21.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.491+0000 7efd87200640 1 -- 172.21.15.161:0/3036823585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd880af1b0 msgr2=0x7efd88071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:21.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.491+0000 7efd87200640 1 --2- 172.21.15.161:0/3036823585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd880af1b0 0x7efd88071030 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efd80009d90 tx=0x7efd8001c970 comp rx=0 tx=0).stop 2024-05-22T21:18:21.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.491+0000 7efd87200640 1 -- 172.21.15.161:0/3036823585 shutdown_connections 2024-05-22T21:18:21.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.491+0000 7efd87200640 1 --2- 172.21.15.161:0/3036823585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd880af1b0 0x7efd88071030 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.491+0000 7efd87200640 1 --2- 172.21.15.161:0/3036823585 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd8806f1b0 0x7efd88070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.491+0000 7efd87200640 1 -- 172.21.15.161:0/3036823585 >> 172.21.15.161:0/3036823585 conn(0x7efd880cfc20 msgr2=0x7efd880d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:21.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.491+0000 7efd87200640 1 -- 172.21.15.161:0/3036823585 shutdown_connections 2024-05-22T21:18:21.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.491+0000 7efd87200640 1 -- 172.21.15.161:0/3036823585 wait complete. 2024-05-22T21:18:21.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd87200640 1 Processor -- start 2024-05-22T21:18:21.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd87200640 1 -- start start 2024-05-22T21:18:21.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd87200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd8806f1b0 0x7efd880ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd87200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd880af1b0 0x7efd88156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd87200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd8815d2f0 0x7efd8814d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd87200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd88155f30 con 0x7efd880af1b0 2024-05-22T21:18:21.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd87200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8814fc10 con 0x7efd8806f1b0 2024-05-22T21:18:21.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd87200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd880aeaa0 con 0x7efd8815d2f0 2024-05-22T21:18:21.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd7d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd880af1b0 0x7efd88156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd7d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd880af1b0 0x7efd88156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38550/0 (socket says 172.21.15.161:38550) 2024-05-22T21:18:21.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd7d400640 1 -- 172.21.15.161:0/2274874616 learned_addr learned my addr 172.21.15.161:0/2274874616 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:21.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd85e00640 1 --2- 172.21.15.161:0/2274874616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd8806f1b0 0x7efd880ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd86800640 1 --2- 172.21.15.161:0/2274874616 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd8815d2f0 0x7efd8814d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd7d400640 1 -- 172.21.15.161:0/2274874616 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd8815d2f0 msgr2=0x7efd8814d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:21.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd7d400640 1 --2- 172.21.15.161:0/2274874616 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd8815d2f0 0x7efd8814d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd7d400640 1 -- 172.21.15.161:0/2274874616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd8806f1b0 msgr2=0x7efd880ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:21.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd7d400640 1 --2- 172.21.15.161:0/2274874616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd8806f1b0 0x7efd880ae560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd7d400640 1 -- 172.21.15.161:0/2274874616 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd80009a70 con 0x7efd880af1b0 2024-05-22T21:18:21.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd7d400640 1 --2- 172.21.15.161:0/2274874616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd880af1b0 0x7efd88156350 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7efd7000e960 tx=0x7efd7000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:21.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd7ea00640 1 -- 172.21.15.161:0/2274874616 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd70009690 con 0x7efd880af1b0 2024-05-22T21:18:21.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd7ea00640 1 -- 172.21.15.161:0/2274874616 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7000ca00 con 0x7efd880af1b0 2024-05-22T21:18:21.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd7ea00640 1 -- 172.21.15.161:0/2274874616 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7000cd70 con 0x7efd880af1b0 2024-05-22T21:18:21.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd87200640 1 -- 172.21.15.161:0/2274874616 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8815c430 con 0x7efd880af1b0 2024-05-22T21:18:21.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.495+0000 7efd87200640 1 -- 172.21.15.161:0/2274874616 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8815c960 con 0x7efd880af1b0 2024-05-22T21:18:21.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.499+0000 7efd87200640 1 -- 172.21.15.161:0/2274874616 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8815cb50 con 0x7efd880af1b0 2024-05-22T21:18:21.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.499+0000 7efd7ea00640 1 -- 172.21.15.161:0/2274874616 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efd70013470 con 0x7efd880af1b0 2024-05-22T21:18:21.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.499+0000 7efd7ea00640 1 --2- 172.21.15.161:0/2274874616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd64066690 0x7efd64068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.499+0000 7efd7ea00640 1 -- 172.21.15.161:0/2274874616 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd70084360 con 0x7efd880af1b0 2024-05-22T21:18:21.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.499+0000 7efd85e00640 1 --2- 172.21.15.161:0/2274874616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd64066690 0x7efd64068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.499+0000 7efd85e00640 1 --2- 172.21.15.161:0/2274874616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd64066690 0x7efd64068b50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efd78000f70 tx=0x7efd78007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:21.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.503+0000 7efd7ea00640 1 -- 172.21.15.161:0/2274874616 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efd70051770 con 0x7efd880af1b0 2024-05-22T21:18:21.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.655+0000 7efd87200640 1 -- 172.21.15.161:0/2274874616 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd88067070 con 0x7efd880af1b0 2024-05-22T21:18:21.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.659+0000 7efd7ea00640 1 -- 172.21.15.161:0/2274874616 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16151 (secure 0 0 0) 0x7efd70054d90 con 0x7efd880af1b0 2024-05-22T21:18:21.661 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:21.661 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":103,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:18:18.942765+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:18:16.500534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-22T21:18:16.937445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:18:21.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.659+0000 7efd87200640 1 -- 172.21.15.161:0/2274874616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd64066690 msgr2=0x7efd64068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:21.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.659+0000 7efd87200640 1 --2- 172.21.15.161:0/2274874616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd64066690 0x7efd64068b50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efd78000f70 tx=0x7efd78007df0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.659+0000 7efd87200640 1 -- 172.21.15.161:0/2274874616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd880af1b0 msgr2=0x7efd88156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:21.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.659+0000 7efd87200640 1 --2- 172.21.15.161:0/2274874616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd880af1b0 0x7efd88156350 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7efd7000e960 tx=0x7efd7000ee20 comp rx=0 tx=0).stop 2024-05-22T21:18:21.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.659+0000 7efd87200640 1 -- 172.21.15.161:0/2274874616 shutdown_connections 2024-05-22T21:18:21.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.659+0000 7efd87200640 1 --2- 172.21.15.161:0/2274874616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd64066690 0x7efd64068b50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.659+0000 7efd87200640 1 --2- 172.21.15.161:0/2274874616 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd8815d2f0 0x7efd8814d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.659+0000 7efd87200640 1 --2- 172.21.15.161:0/2274874616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd880af1b0 0x7efd88156350 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.659+0000 7efd87200640 1 --2- 172.21.15.161:0/2274874616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd8806f1b0 0x7efd880ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.659+0000 7efd87200640 1 -- 172.21.15.161:0/2274874616 >> 172.21.15.161:0/2274874616 conn(0x7efd880cfc20 msgr2=0x7efd880b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:21.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.659+0000 7efd87200640 1 -- 172.21.15.161:0/2274874616 shutdown_connections 2024-05-22T21:18:21.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.659+0000 7efd87200640 1 -- 172.21.15.161:0/2274874616 wait complete. 2024-05-22T21:18:21.682 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-22T21:18:21.682 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:18:21.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fe3600640 1 Processor -- start 2024-05-22T21:18:21.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fe3600640 1 -- start start 2024-05-22T21:18:21.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fe3600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2fdc1438e0 0x7f2fdc0714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fe3600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2fdc0b1510 0x7f2fdc163af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fe3600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fdc143cc0 con 0x7f2fdc070f30 2024-05-22T21:18:21.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fe3600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fdc070a50 con 0x7f2fdc0b1510 2024-05-22T21:18:21.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fe3600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fdc06fdd0 con 0x7f2fdc1438e0 2024-05-22T21:18:21.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fe1600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2fdc0b1510 0x7f2fdc163af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fd3e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f2fdc070f30 0x7f2fdc0b0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:40284/0 (socket says 172.21.15.161:40284) 2024-05-22T21:18:21.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fd3e00640 1 -- 172.21.15.161:0/424576488 learned_addr learned my addr 172.21.15.161:0/424576488 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:21.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fe1600640 1 -- 172.21.15.161:0/424576488 >> v2:172.21.15.179:3300/0 conn(0x7f2fdc1438e0 msgr2=0x7f2fdc0714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:21.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fe0c00640 1 --2- 172.21.15.161:0/424576488 >> v2:172.21.15.179:3300/0 conn(0x7f2fdc1438e0 0x7f2fdc0714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fe1600640 1 --2- 172.21.15.161:0/424576488 >> v2:172.21.15.179:3300/0 conn(0x7f2fdc1438e0 0x7f2fdc0714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fe1600640 1 -- 172.21.15.161:0/424576488 >> v1:172.21.15.161:6789/0 conn(0x7f2fdc070f30 legacy=0x7f2fdc0b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:21.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fdbe00640 1 -- 172.21.15.161:0/424576488 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638968285 0 0) 0x7f2fdc143cc0 con 0x7f2fdc070f30 2024-05-22T21:18:21.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fe1600640 1 -- 172.21.15.161:0/424576488 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc0700b0 con 0x7f2fdc0b1510 2024-05-22T21:18:21.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.815+0000 7f2fe0c00640 1 --2- 172.21.15.161:0/424576488 >> v2:172.21.15.179:3300/0 conn(0x7f2fdc1438e0 0x7f2fdc0714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:21.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.819+0000 7f2fe1600640 1 --2- 172.21.15.161:0/424576488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fdc0b1510 0x7f2fdc163af0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0b1980 tx=0x7f2fd401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d435524ca08a59c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:21.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.819+0000 7f2fdbe00640 1 -- 172.21.15.161:0/424576488 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd40a5850 con 0x7f2fdc0b1510 2024-05-22T21:18:21.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.819+0000 7f2fdbe00640 1 -- 172.21.15.161:0/424576488 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd40a59b0 con 0x7f2fdc0b1510 2024-05-22T21:18:21.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.819+0000 7f2fdbe00640 1 -- 172.21.15.161:0/424576488 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd40a5d40 con 0x7f2fdc0b1510 2024-05-22T21:18:21.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.819+0000 7f2fe3600640 1 -- 172.21.15.161:0/424576488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fdc0b1510 msgr2=0x7f2fdc163af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:21.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.819+0000 7f2fe3600640 1 --2- 172.21.15.161:0/424576488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fdc0b1510 0x7f2fdc163af0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0b1980 tx=0x7f2fd401c8a0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.819+0000 7f2fe3600640 1 -- 172.21.15.161:0/424576488 shutdown_connections 2024-05-22T21:18:21.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.819+0000 7f2fe3600640 1 --2- 172.21.15.161:0/424576488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fdc0b1510 0x7f2fdc163af0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.819+0000 7f2fe3600640 1 --2- 172.21.15.161:0/424576488 >> v2:172.21.15.179:3300/0 conn(0x7f2fdc1438e0 0x7f2fdc0714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.819+0000 7f2fe3600640 1 -- 172.21.15.161:0/424576488 >> 172.21.15.161:0/424576488 conn(0x7f2fdc06a0a0 msgr2=0x7f2fdc06a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:21.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.819+0000 7f2fe3600640 1 -- 172.21.15.161:0/424576488 shutdown_connections 2024-05-22T21:18:21.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.819+0000 7f2fe3600640 1 -- 172.21.15.161:0/424576488 wait complete. 2024-05-22T21:18:21.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fe3600640 1 Processor -- start 2024-05-22T21:18:21.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fe3600640 1 -- start start 2024-05-22T21:18:21.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fe3600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fdc070f30 0x7f2fdc140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fe3600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fdc0b1510 0x7f2fdc1638a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fe3600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2fdc1438e0 0x7f2fdc13f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fe3600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc163de0 con 0x7f2fdc0b1510 2024-05-22T21:18:21.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fe3600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc1421f0 con 0x7f2fdc070f30 2024-05-22T21:18:21.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fe3600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc1415e0 con 0x7f2fdc1438e0 2024-05-22T21:18:21.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fd3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fdc0b1510 0x7f2fdc1638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fd3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fdc0b1510 0x7f2fdc1638a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38570/0 (socket says 172.21.15.161:38570) 2024-05-22T21:18:21.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fd3e00640 1 -- 172.21.15.161:0/2693478464 learned_addr learned my addr 172.21.15.161:0/2693478464 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:21.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fe0c00640 1 --2- 172.21.15.161:0/2693478464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fdc070f30 0x7f2fdc140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fe1600640 1 --2- 172.21.15.161:0/2693478464 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2fdc1438e0 0x7f2fdc13f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fd3e00640 1 -- 172.21.15.161:0/2693478464 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2fdc1438e0 msgr2=0x7f2fdc13f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:21.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fd3e00640 1 --2- 172.21.15.161:0/2693478464 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2fdc1438e0 0x7f2fdc13f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fd3e00640 1 -- 172.21.15.161:0/2693478464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fdc070f30 msgr2=0x7f2fdc140ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:21.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fd3e00640 1 --2- 172.21.15.161:0/2693478464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fdc070f30 0x7f2fdc140ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:21.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fd3e00640 1 -- 172.21.15.161:0/2693478464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd4009a70 con 0x7f2fdc0b1510 2024-05-22T21:18:21.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fe1600640 1 --2- 172.21.15.161:0/2693478464 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2fdc1438e0 0x7f2fdc13f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:21.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fd3e00640 1 --2- 172.21.15.161:0/2693478464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fdc0b1510 0x7f2fdc1638a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400c4b0 tx=0x7f2fc400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:21.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fd9600640 1 -- 172.21.15.161:0/2693478464 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40074e0 con 0x7f2fdc0b1510 2024-05-22T21:18:21.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fd9600640 1 -- 172.21.15.161:0/2693478464 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc4007660 con 0x7f2fdc0b1510 2024-05-22T21:18:21.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fd9600640 1 -- 172.21.15.161:0/2693478464 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4004a30 con 0x7f2fdc0b1510 2024-05-22T21:18:21.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fe3600640 1 -- 172.21.15.161:0/2693478464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc1418c0 con 0x7f2fdc0b1510 2024-05-22T21:18:21.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.823+0000 7f2fe3600640 1 -- 172.21.15.161:0/2693478464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc141e80 con 0x7f2fdc0b1510 2024-05-22T21:18:21.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.827+0000 7f2fe3600640 1 -- 172.21.15.161:0/2693478464 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fdc142360 con 0x7f2fdc0b1510 2024-05-22T21:18:21.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.827+0000 7f2fd9600640 1 -- 172.21.15.161:0/2693478464 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2fc4004b90 con 0x7f2fdc0b1510 2024-05-22T21:18:21.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.827+0000 7f2fd9600640 1 --2- 172.21.15.161:0/2693478464 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2fb8066690 0x7f2fb8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:21.830 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.827+0000 7f2fd9600640 1 -- 172.21.15.161:0/2693478464 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fc4083400 con 0x7f2fdc0b1510 2024-05-22T21:18:21.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.827+0000 7f2fe0c00640 1 --2- 172.21.15.161:0/2693478464 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2fb8066690 0x7f2fb8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:21.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.827+0000 7f2fe0c00640 1 --2- 172.21.15.161:0/2693478464 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2fb8066690 0x7f2fb8068b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc006fd0 tx=0x7f2fcc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:21.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.831+0000 7f2fd9600640 1 -- 172.21.15.161:0/2693478464 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2fc4050830 con 0x7f2fdc0b1510 2024-05-22T21:18:21.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.983+0000 7f2fe3600640 1 -- 172.21.15.161:0/2693478464 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2fdc0b4270 con 0x7f2fb8066690 2024-05-22T21:18:21.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.991+0000 7f2fd9600640 1 -- 172.21.15.161:0/2693478464 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+273520 (secure 0 0 0) 0x7f2fdc0b4270 con 0x7f2fb8066690 2024-05-22T21:18:21.997 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:21.998 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:18:22.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.995+0000 7f2fe3600640 1 -- 172.21.15.161:0/2693478464 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2fb8066690 msgr2=0x7f2fb8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.995+0000 7f2fe3600640 1 --2- 172.21.15.161:0/2693478464 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2fb8066690 0x7f2fb8068b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc006fd0 tx=0x7f2fcc008040 comp rx=0 tx=0).stop 2024-05-22T21:18:22.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.995+0000 7f2fe3600640 1 -- 172.21.15.161:0/2693478464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fdc0b1510 msgr2=0x7f2fdc1638a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.995+0000 7f2fe3600640 1 --2- 172.21.15.161:0/2693478464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fdc0b1510 0x7f2fdc1638a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400c4b0 tx=0x7f2fc400c970 comp rx=0 tx=0).stop 2024-05-22T21:18:22.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.995+0000 7f2fe3600640 1 -- 172.21.15.161:0/2693478464 shutdown_connections 2024-05-22T21:18:22.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.995+0000 7f2fe3600640 1 --2- 172.21.15.161:0/2693478464 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2fb8066690 0x7f2fb8068b50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.995+0000 7f2fe3600640 1 --2- 172.21.15.161:0/2693478464 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2fdc1438e0 0x7f2fdc13f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.995+0000 7f2fe3600640 1 --2- 172.21.15.161:0/2693478464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fdc0b1510 0x7f2fdc1638a0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.995+0000 7f2fe3600640 1 --2- 172.21.15.161:0/2693478464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fdc070f30 0x7f2fdc140ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.995+0000 7f2fe3600640 1 -- 172.21.15.161:0/2693478464 >> 172.21.15.161:0/2693478464 conn(0x7f2fdc06a0a0 msgr2=0x7f2fdc0cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:22.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.995+0000 7f2fe3600640 1 -- 172.21.15.161:0/2693478464 shutdown_connections 2024-05-22T21:18:22.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:21.995+0000 7f2fe3600640 1 -- 172.21.15.161:0/2693478464 wait complete. 2024-05-22T21:18:22.017 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-05-22T21:18:20.459394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557181,"statfs":{"total":1600000000,"available":1594553795,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000707"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":244,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-22T21:18:13.725365+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:18:13.725365+0000","last_peered":"2024-05-22T21:18:13.725365+0000","last_clean":"2024-05-22T21:18:13.725365+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:18:13.725365+0000","last_undegraded":"2024-05-22T21:18:13.725365+0000","last_fullsized":"2024-05-22T21:18:13.725365+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574911,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1097,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194215,"statfs":{"total":200000000,"available":198876371,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72599999999999998,"15min":0.76200000000000001},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.98899999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70399999999999996,"15min":0.69199999999999995},"min":{"1min":0.48599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97699999999999998,"5min":1.889,"15min":1.889},"last":0.59699999999999998}]},{"osd":1,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75,"15min":0.77800000000000002},"min":{"1min":0.502,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95799999999999996,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.72999999999999998,"15min":0.76500000000000001},"min":{"1min":0.44900000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92700000000000005,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.64900000000000002}]},{"osd":2,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.51500000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94199999999999995,"5min":1.923,"15min":1.923},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.74399999999999999,"15min":0.72699999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.048,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.78700000000000003}]},{"osd":3,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.57299999999999995,"5min":0.443,"15min":0.443},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77500000000000002,"15min":0.79900000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.014,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.748}]},{"osd":4,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.74299999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88900000000000001,"5min":1.018,"15min":1.018},"last":0.629}]},{"osd":5,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63900000000000001,"15min":0.65500000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89600000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82999999999999996,"5min":0.996,"15min":0.996},"last":0.55000000000000004}]},{"osd":6,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96999999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66000000000000003,"15min":0.64000000000000001},"min":{"1min":0.51000000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83199999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.66400000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194791,"statfs":{"total":200000000,"available":199466739,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.58599999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.85299999999999998,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.69799999999999995,"15min":0.68700000000000006},"min":{"1min":0.57399999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86599999999999999,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.64200000000000002}]},{"osd":1,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69899999999999995,"15min":0.68000000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.88400000000000001,"5min":1.107,"15min":1.107},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71499999999999997,"15min":0.70899999999999996},"min":{"1min":0.55000000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88100000000000001,"5min":1.081,"15min":1.081},"last":0.71099999999999997}]},{"osd":2,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.77900000000000003,"15min":0.75700000000000001},"min":{"1min":0.64400000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.91500000000000004,"5min":1.04,"15min":1.04},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.59799999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90400000000000003,"5min":0.997,"15min":0.997},"last":0.56599999999999995}]},{"osd":3,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.75,"15min":0.74299999999999999},"min":{"1min":0.61499999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.86399999999999999,"5min":1.415,"15min":1.415},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73399999999999999,"15min":0.72999999999999998},"min":{"1min":0.54300000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84999999999999998,"5min":1.788,"15min":1.788},"last":0.54400000000000004}]},{"osd":4,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63700000000000001,"15min":0.65100000000000002},"min":{"1min":0.436,"5min":0.374,"15min":0.374},"max":{"1min":0.90400000000000003,"5min":1.006,"15min":1.006},"last":0.63},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.68100000000000005,"15min":0.68500000000000005},"min":{"1min":0.436,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81799999999999995,"5min":2.718,"15min":2.718},"last":0.76700000000000002}]},{"osd":5,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.75600000000000001,"15min":0.70699999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83399999999999996,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.879,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.59299999999999997}]},{"osd":7,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71999999999999997,"15min":0.74099999999999999},"min":{"1min":0.55100000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":1.095,"15min":1.095},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.745,"15min":0.76300000000000001},"min":{"1min":0.52300000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88200000000000001,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.68100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194791,"statfs":{"total":200000000,"available":199466739,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69699999999999995,"15min":0.70699999999999996},"min":{"1min":0.58199999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89300000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.69399999999999995,"15min":0.67400000000000004},"min":{"1min":0.63100000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.93600000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.497,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88600000000000001,"5min":1.216,"15min":1.216},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69599999999999995,"15min":0.68600000000000005},"min":{"1min":0.52900000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.84299999999999997,"5min":1.048,"15min":1.048},"last":0.56299999999999994}]},{"osd":2,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.72499999999999998,"15min":0.70099999999999996},"min":{"1min":0.56999999999999995,"5min":0.436,"15min":0.436},"max":{"1min":0.96499999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.69799999999999995,"15min":0.67600000000000005},"min":{"1min":0.61299999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92000000000000004,"5min":1.252,"15min":1.252},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.88600000000000001,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.746,"15min":0.73399999999999999},"min":{"1min":0.56299999999999994,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95299999999999996,"5min":1.111,"15min":1.111},"last":0.52800000000000002}]},{"osd":4,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60299999999999998,"15min":0.63100000000000001},"min":{"1min":0.42899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92800000000000005,"5min":1.272,"15min":1.272},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.44400000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.91700000000000004,"5min":1.038,"15min":1.038},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.49099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65100000000000002,"15min":0.66300000000000003},"min":{"1min":0.41499999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.877,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.442}]},{"osd":7,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71499999999999997,"15min":0.71699999999999997},"min":{"1min":0.53000000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77800000000000002,"15min":0.79600000000000004},"min":{"1min":0.57799999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95899999999999996,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.72899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194791,"statfs":{"total":200000000,"available":199466739,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.78000000000000003,"15min":0.747},"min":{"1min":0.42899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":6.6539999999999999,"5min":6.6539999999999999,"15min":6.6539999999999999},"last":0.755},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.745,"15min":0.72699999999999998},"min":{"1min":0.54600000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":5.4370000000000003,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":0.60799999999999998}]},{"osd":1,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69299999999999995,"15min":0.71999999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.71099999999999997,"15min":0.70299999999999996},"min":{"1min":0.51100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.877}]},{"osd":2,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.69399999999999995,"15min":0.68600000000000005},"min":{"1min":0.497,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.58899999999999997,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96599999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.85499999999999998}]},{"osd":3,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.68100000000000005,"15min":0.68799999999999994},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.54600000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2470000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.71399999999999997}]},{"osd":5,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63,"15min":0.64500000000000002},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.45400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.68200000000000005}]},{"osd":6,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67400000000000004,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.621},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.82999999999999996}]},{"osd":7,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69599999999999995,"15min":0.69999999999999996},"min":{"1min":0.499,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.751,"15min":0.78300000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97799999999999998,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.77900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194791,"statfs":{"total":200000000,"available":199466739,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.42099999999999999,"15min":0.40400000000000003},"min":{"1min":0.26100000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.432,"15min":0.40500000000000003},"min":{"1min":0.28799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65600000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.372}]},{"osd":1,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.45800000000000002,"15min":0.44700000000000001},"min":{"1min":0.29599999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.52300000000000002,"15min":0.505},"min":{"1min":0.44900000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.49399999999999999},"min":{"1min":0.40999999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.66100000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.505,"15min":0.48799999999999999},"min":{"1min":0.34999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.45400000000000001}]},{"osd":4,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72899999999999998,"15min":0.70199999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.73899999999999999,"15min":0.71499999999999997},"min":{"1min":0.60099999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.397,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.71899999999999997}]},{"osd":5,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73599999999999999,"15min":0.69199999999999995},"min":{"1min":0.59399999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.752},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.73499999999999999,"15min":0.69899999999999995},"min":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.292,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.78200000000000003}]},{"osd":6,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73699999999999999,"15min":0.71299999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.73799999999999999,"15min":0.70099999999999996},"min":{"1min":0.55300000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.81699999999999995}]},{"osd":7,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.78700000000000003,"15min":0.755},"min":{"1min":0.63,"5min":0.495,"15min":0.495},"max":{"1min":1.1000000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.76800000000000002,"15min":0.73499999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.79100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194791,"statfs":{"total":200000000,"available":199466739,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40799999999999997,"15min":0.39800000000000002},"min":{"1min":0.26000000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.44600000000000001,"15min":0.437},"min":{"1min":0.308,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51600000000000001}]},{"osd":1,"last update":"Wed May 22 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.32900000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52500000000000002,"15min":0.51500000000000001},"min":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55500000000000005}]},{"osd":3,"last update":"Wed May 22 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.502,"15min":0.498},"min":{"1min":0.34399999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.496,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.35999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.46899999999999997}]},{"osd":4,"last update":"Wed May 22 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71799999999999997,"15min":0.68500000000000005},"min":{"1min":0.56499999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.55000000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.113,"5min":1.173,"15min":1.173},"last":0.65400000000000003}]},{"osd":5,"last update":"Wed May 22 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.71199999999999997,"15min":0.69999999999999996},"min":{"1min":0.48099999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0509999999999999,"5min":1.143,"15min":1.143},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.72199999999999998,"15min":0.70099999999999996},"min":{"1min":0.55300000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.71699999999999997}]},{"osd":6,"last update":"Wed May 22 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72099999999999997,"15min":0.69999999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.98799999999999999,"5min":1.054,"15min":1.054},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed May 22 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.745,"15min":0.72499999999999998},"min":{"1min":0.61799999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.98799999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.78300000000000003,"15min":0.77000000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0680000000000001,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.75700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194791,"statfs":{"total":200000000,"available":199466739,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42199999999999999,"15min":0.432},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56999999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.44900000000000001,"15min":0.433},"min":{"1min":0.28399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.84799999999999998,"5min":0.876,"15min":0.876},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48899999999999999,"15min":0.48999999999999999},"min":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.748,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.51100000000000001,"15min":0.53600000000000003},"min":{"1min":0.35699999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64700000000000002,"5min":1.923,"15min":1.923},"last":0.42799999999999999}]},{"osd":3,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.57099999999999995},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.49099999999999999,"15min":0.48699999999999999},"min":{"1min":0.34599999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72099999999999997,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.45500000000000002}]},{"osd":4,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.74099999999999999,"15min":0.76000000000000001},"min":{"1min":0.52300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.999,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.70699999999999996,"15min":0.68300000000000005},"min":{"1min":0.504,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0389999999999999,"5min":1.3,"15min":1.3},"last":0.53200000000000003}]},{"osd":5,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.53500000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0089999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.751,"15min":0.753},"min":{"1min":0.56499999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1000000000000001,"5min":1.97,"15min":1.97},"last":0.621}]},{"osd":6,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.76300000000000001,"15min":0.745},"min":{"1min":0.54600000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0169999999999999,"5min":1.272,"15min":1.272},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.77000000000000002,"15min":0.745},"min":{"1min":0.64700000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.04,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.50600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0129999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76300000000000001,"15min":0.77900000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.91200000000000003,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.69299999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1096,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194215,"statfs":{"total":200000000,"available":198876990,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42599999999999999,"15min":0.40400000000000003},"min":{"1min":0.32200000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.73499999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.378,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67300000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.47699999999999998}]},{"osd":2,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47699999999999998,"15min":0.45100000000000001},"min":{"1min":0.33200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70499999999999996,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.439,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76500000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.54000000000000004}]},{"osd":3,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.45000000000000001,"15min":0.432},"min":{"1min":0.30199999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68899999999999995,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51900000000000002,"15min":0.498},"min":{"1min":0.42499999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79300000000000004,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.50900000000000001}]},{"osd":4,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72699999999999998,"15min":0.70899999999999996},"min":{"1min":0.56000000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.499},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.72699999999999998},"min":{"1min":0.52600000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0049999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.57099999999999995}]},{"osd":5,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97199999999999998,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.75600000000000001,"15min":0.73499999999999999},"min":{"1min":0.58699999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1220000000000001,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.59799999999999998}]},{"osd":6,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78800000000000003,"15min":0.77100000000000002},"min":{"1min":0.65400000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0820000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.77500000000000002,"15min":0.77000000000000002},"min":{"1min":0.58499999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0640000000000001,"5min":1.5,"15min":1.5},"last":0.65100000000000002}]},{"osd":7,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.74099999999999999,"15min":0.72199999999999998},"min":{"1min":0.61199999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.0589999999999999,"5min":1.085,"15min":1.085},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.65000000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.1990000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.55900000000000005}]}]}],"pool_statfs":[]}} 2024-05-22T21:18:22.023 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:18:22.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.195+0000 7f858e800640 1 Processor -- start 2024-05-22T21:18:22.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.195+0000 7f858e800640 1 -- start start 2024-05-22T21:18:22.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.195+0000 7f858e800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f858815f3a0 0x7f858815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.195+0000 7f858e800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85881442a0 con 0x7f8588143ec0 2024-05-22T21:18:22.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.195+0000 7f858e800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8588070500 con 0x7f85880715a0 2024-05-22T21:18:22.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.195+0000 7f858e800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85880707e0 con 0x7f858815f3a0 2024-05-22T21:18:22.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8587400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8588143ec0 0x7f858806fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:40300/0 (socket says 172.21.15.161:40300) 2024-05-22T21:18:22.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8587400640 1 -- 172.21.15.161:0/499045822 learned_addr learned my addr 172.21.15.161:0/499045822 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:22.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8587e00640 1 --2- 172.21.15.161:0/499045822 >> v2:172.21.15.179:3300/0 conn(0x7f858815f3a0 0x7f858815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 -- 172.21.15.161:0/499045822 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1183542568 0 0) 0x7f85881442a0 con 0x7f8588143ec0 2024-05-22T21:18:22.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 -- 172.21.15.161:0/499045822 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f856c003530 con 0x7f8588143ec0 2024-05-22T21:18:22.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 -- 172.21.15.161:0/499045822 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939939911 0 0) 0x7f8588070500 con 0x7f85880715a0 2024-05-22T21:18:22.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 -- 172.21.15.161:0/499045822 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85881442a0 con 0x7f85880715a0 2024-05-22T21:18:22.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 -- 172.21.15.161:0/499045822 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1109293004 0 0) 0x7f856c003530 con 0x7f8588143ec0 2024-05-22T21:18:22.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 -- 172.21.15.161:0/499045822 >> v2:172.21.15.179:3300/0 conn(0x7f858815f3a0 msgr2=0x7f858815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:22.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 --2- 172.21.15.161:0/499045822 >> v2:172.21.15.179:3300/0 conn(0x7f858815f3a0 0x7f858815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 -- 172.21.15.161:0/499045822 >> v1:172.21.15.179:6789/0 conn(0x7f85880715a0 legacy=0x7f858815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 -- 172.21.15.161:0/499045822 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8588070ac0 con 0x7f8588143ec0 2024-05-22T21:18:22.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 -- 172.21.15.161:0/499045822 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f8578001a00 con 0x7f8588143ec0 2024-05-22T21:18:22.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8587e00640 1 --2- 172.21.15.161:0/499045822 >> v2:172.21.15.179:3300/0 conn(0x7f858815f3a0 0x7f858815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:22.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 -- 172.21.15.161:0/499045822 >> v1:172.21.15.161:6789/0 conn(0x7f8588143ec0 legacy=0x7f858806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 --2- 172.21.15.161:0/499045822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f856c003b10 0x7f856c003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 --2- 172.21.15.161:0/499045822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f856c0047d0 0x7f856c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 --2- 172.21.15.161:0/499045822 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f856c005190 0x7f856c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 -- 172.21.15.161:0/499045822 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85880707e0 con 0x7f856c003b10 2024-05-22T21:18:22.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 -- 172.21.15.161:0/499045822 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856c00c010 con 0x7f856c0047d0 2024-05-22T21:18:22.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8586a00640 1 -- 172.21.15.161:0/499045822 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856c00c1d0 con 0x7f856c005190 2024-05-22T21:18:22.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8587400640 1 --2- 172.21.15.161:0/499045822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f856c003b10 0x7f856c003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f857ea00640 1 --2- 172.21.15.161:0/499045822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f856c0047d0 0x7f856c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8587e00640 1 --2- 172.21.15.161:0/499045822 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f856c005190 0x7f856c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8587400640 1 -- 172.21.15.161:0/499045822 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f856c005190 msgr2=0x7f856c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:22.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8587400640 1 --2- 172.21.15.161:0/499045822 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f856c005190 0x7f856c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8587400640 1 -- 172.21.15.161:0/499045822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f856c0047d0 msgr2=0x7f856c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:22.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8587400640 1 --2- 172.21.15.161:0/499045822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f856c0047d0 0x7f856c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8587400640 1 -- 172.21.15.161:0/499045822 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856c003530 con 0x7f856c003b10 2024-05-22T21:18:22.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f857ea00640 1 --2- 172.21.15.161:0/499045822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f856c0047d0 0x7f856c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:22.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.199+0000 7f8587e00640 1 -- 172.21.15.161:0/499045822 reap_dead start 2024-05-22T21:18:22.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f8587400640 1 --2- 172.21.15.161:0/499045822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f856c003b10 0x7f856c003f10 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f857800a7c0 tx=0x7f857801d2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e94a7d368f1f80 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:22.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f8586a00640 1 -- 172.21.15.161:0/499045822 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85780ad630 con 0x7f856c003b10 2024-05-22T21:18:22.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f8586a00640 1 -- 172.21.15.161:0/499045822 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85780a8070 con 0x7f856c003b10 2024-05-22T21:18:22.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f8586a00640 1 -- 172.21.15.161:0/499045822 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85780ada30 con 0x7f856c003b10 2024-05-22T21:18:22.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f858e800640 1 -- 172.21.15.161:0/499045822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f856c003b10 msgr2=0x7f856c003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f858e800640 1 --2- 172.21.15.161:0/499045822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f856c003b10 0x7f856c003f10 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f857800a7c0 tx=0x7f857801d2b0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f858e800640 1 -- 172.21.15.161:0/499045822 shutdown_connections 2024-05-22T21:18:22.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f858e800640 1 --2- 172.21.15.161:0/499045822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f856c003b10 0x7f856c003f10 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f858e800640 1 -- 172.21.15.161:0/499045822 >> 172.21.15.161:0/499045822 conn(0x7f85880b0bc0 msgr2=0x7f85880b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:22.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f858e800640 1 -- 172.21.15.161:0/499045822 shutdown_connections 2024-05-22T21:18:22.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f858e800640 1 -- 172.21.15.161:0/499045822 wait complete. 2024-05-22T21:18:22.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f858e800640 1 Processor -- start 2024-05-22T21:18:22.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f858e800640 1 -- start start 2024-05-22T21:18:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f858e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f856c003b10 0x7f85880c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f858e800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f858815d130 0x7f85881f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f858e800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85881f5890 0x7f85880af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f858e800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858815d8d0 con 0x7f856c003b10 2024-05-22T21:18:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f858e800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858815dbb0 con 0x7f85881f5890 2024-05-22T21:18:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.203+0000 7f858e800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858815de90 con 0x7f858815d130 2024-05-22T21:18:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f8587400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f856c003b10 0x7f85880c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f8587400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f856c003b10 0x7f85880c4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38594/0 (socket says 172.21.15.161:38594) 2024-05-22T21:18:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f8587400640 1 -- 172.21.15.161:0/1312508848 learned_addr learned my addr 172.21.15.161:0/1312508848 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f857ea00640 1 --2- 172.21.15.161:0/1312508848 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f858815d130 0x7f85881f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f8587e00640 1 --2- 172.21.15.161:0/1312508848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85881f5890 0x7f85880af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f8587400640 1 -- 172.21.15.161:0/1312508848 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f858815d130 msgr2=0x7f85881f5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f8587400640 1 --2- 172.21.15.161:0/1312508848 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f858815d130 0x7f85881f5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f8587400640 1 -- 172.21.15.161:0/1312508848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85881f5890 msgr2=0x7f85880af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f8587400640 1 --2- 172.21.15.161:0/1312508848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85881f5890 0x7f85880af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f8587400640 1 -- 172.21.15.161:0/1312508848 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858815e170 con 0x7f856c003b10 2024-05-22T21:18:22.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f8587400640 1 --2- 172.21.15.161:0/1312508848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f856c003b10 0x7f85880c4c00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f857801cd20 tx=0x7f856c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:22.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f857fe00640 1 -- 172.21.15.161:0/1312508848 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578001800 con 0x7f856c003b10 2024-05-22T21:18:22.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f857fe00640 1 -- 172.21.15.161:0/1312508848 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8578001960 con 0x7f856c003b10 2024-05-22T21:18:22.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f857fe00640 1 -- 172.21.15.161:0/1312508848 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578004d90 con 0x7f856c003b10 2024-05-22T21:18:22.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f858e800640 1 -- 172.21.15.161:0/1312508848 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858815e450 con 0x7f856c003b10 2024-05-22T21:18:22.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f858e800640 1 -- 172.21.15.161:0/1312508848 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85880afdd0 con 0x7f856c003b10 2024-05-22T21:18:22.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f858e800640 1 -- 172.21.15.161:0/1312508848 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8588067220 con 0x7f856c003b10 2024-05-22T21:18:22.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.207+0000 7f857fe00640 1 -- 172.21.15.161:0/1312508848 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f85780aa030 con 0x7f856c003b10 2024-05-22T21:18:22.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.211+0000 7f857fe00640 1 --2- 172.21.15.161:0/1312508848 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85640666c0 0x7f8564068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.211+0000 7f857fe00640 1 -- 172.21.15.161:0/1312508848 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f857811da40 con 0x7f856c003b10 2024-05-22T21:18:22.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.211+0000 7f857ea00640 1 --2- 172.21.15.161:0/1312508848 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85640666c0 0x7f8564068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.211+0000 7f857ea00640 1 --2- 172.21.15.161:0/1312508848 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85640666c0 0x7f8564068b80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f85880cbf90 tx=0x7f8570009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:22.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.215+0000 7f857fe00640 1 -- 172.21.15.161:0/1312508848 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f85780eadc0 con 0x7f856c003b10 2024-05-22T21:18:22.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.363+0000 7f858e800640 1 -- 172.21.15.161:0/1312508848 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f858815dd20 con 0x7f85640666c0 2024-05-22T21:18:22.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.371+0000 7f857fe00640 1 -- 172.21.15.161:0/1312508848 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+273520 (secure 0 0 0) 0x7f858815dd20 con 0x7f85640666c0 2024-05-22T21:18:22.376 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:22.377 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:18:22.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.375+0000 7f858e800640 1 -- 172.21.15.161:0/1312508848 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85640666c0 msgr2=0x7f8564068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.375+0000 7f858e800640 1 --2- 172.21.15.161:0/1312508848 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85640666c0 0x7f8564068b80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f85880cbf90 tx=0x7f8570009040 comp rx=0 tx=0).stop 2024-05-22T21:18:22.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.375+0000 7f858e800640 1 -- 172.21.15.161:0/1312508848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f856c003b10 msgr2=0x7f85880c4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.375+0000 7f858e800640 1 --2- 172.21.15.161:0/1312508848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f856c003b10 0x7f85880c4c00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f857801cd20 tx=0x7f856c005800 comp rx=0 tx=0).stop 2024-05-22T21:18:22.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.375+0000 7f858e800640 1 -- 172.21.15.161:0/1312508848 shutdown_connections 2024-05-22T21:18:22.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.375+0000 7f858e800640 1 --2- 172.21.15.161:0/1312508848 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85640666c0 0x7f8564068b80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.375+0000 7f858e800640 1 --2- 172.21.15.161:0/1312508848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85881f5890 0x7f85880af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.375+0000 7f858e800640 1 --2- 172.21.15.161:0/1312508848 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f858815d130 0x7f85881f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.375+0000 7f858e800640 1 --2- 172.21.15.161:0/1312508848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f856c003b10 0x7f85880c4c00 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.375+0000 7f858e800640 1 -- 172.21.15.161:0/1312508848 >> 172.21.15.161:0/1312508848 conn(0x7f85880b0bc0 msgr2=0x7f8588142e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:22.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.375+0000 7f858e800640 1 -- 172.21.15.161:0/1312508848 shutdown_connections 2024-05-22T21:18:22.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.375+0000 7f858e800640 1 -- 172.21.15.161:0/1312508848 wait complete. 2024-05-22T21:18:22.395 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-05-22T21:18:20.459394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557181,"statfs":{"total":1600000000,"available":1594553795,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000707"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:17.552828+0000","last_change":"2024-05-22T21:18:17.552828+0000","last_active":"2024-05-22T21:18:17.552828+0000","last_peered":"2024-05-22T21:18:17.552828+0000","last_clean":"2024-05-22T21:18:17.552828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:17.552828+0000","last_undegraded":"2024-05-22T21:18:17.552828+0000","last_fullsized":"2024-05-22T21:18:17.552828+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-22T21:18:16.550714+0000","last_change":"2024-05-22T21:18:16.550714+0000","last_active":"2024-05-22T21:18:16.550714+0000","last_peered":"2024-05-22T21:18:16.550714+0000","last_clean":"2024-05-22T21:18:16.550714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-22T21:18:16.550714+0000","last_undegraded":"2024-05-22T21:18:16.550714+0000","last_fullsized":"2024-05-22T21:18:16.550714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":244,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-05-22T21:18:13.725365+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:18:13.725365+0000","last_peered":"2024-05-22T21:18:13.725365+0000","last_clean":"2024-05-22T21:18:13.725365+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:18:13.725365+0000","last_undegraded":"2024-05-22T21:18:13.725365+0000","last_fullsized":"2024-05-22T21:18:13.725365+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574911,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1097,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194215,"statfs":{"total":200000000,"available":198876371,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72599999999999998,"15min":0.76200000000000001},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.98899999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70399999999999996,"15min":0.69199999999999995},"min":{"1min":0.48599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97699999999999998,"5min":1.889,"15min":1.889},"last":0.59699999999999998}]},{"osd":1,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75,"15min":0.77800000000000002},"min":{"1min":0.502,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95799999999999996,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.72999999999999998,"15min":0.76500000000000001},"min":{"1min":0.44900000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92700000000000005,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.64900000000000002}]},{"osd":2,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.51500000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94199999999999995,"5min":1.923,"15min":1.923},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.74399999999999999,"15min":0.72699999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.048,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.78700000000000003}]},{"osd":3,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.57299999999999995,"5min":0.443,"15min":0.443},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77500000000000002,"15min":0.79900000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.014,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.748}]},{"osd":4,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.74299999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88900000000000001,"5min":1.018,"15min":1.018},"last":0.629}]},{"osd":5,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63900000000000001,"15min":0.65500000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89600000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82999999999999996,"5min":0.996,"15min":0.996},"last":0.55000000000000004}]},{"osd":6,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96999999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66000000000000003,"15min":0.64000000000000001},"min":{"1min":0.51000000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83199999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.66400000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194791,"statfs":{"total":200000000,"available":199466739,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.58599999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.85299999999999998,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.69799999999999995,"15min":0.68700000000000006},"min":{"1min":0.57399999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86599999999999999,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.64200000000000002}]},{"osd":1,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69899999999999995,"15min":0.68000000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.88400000000000001,"5min":1.107,"15min":1.107},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71499999999999997,"15min":0.70899999999999996},"min":{"1min":0.55000000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88100000000000001,"5min":1.081,"15min":1.081},"last":0.71099999999999997}]},{"osd":2,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.77900000000000003,"15min":0.75700000000000001},"min":{"1min":0.64400000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.91500000000000004,"5min":1.04,"15min":1.04},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.59799999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90400000000000003,"5min":0.997,"15min":0.997},"last":0.56599999999999995}]},{"osd":3,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.75,"15min":0.74299999999999999},"min":{"1min":0.61499999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.86399999999999999,"5min":1.415,"15min":1.415},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73399999999999999,"15min":0.72999999999999998},"min":{"1min":0.54300000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84999999999999998,"5min":1.788,"15min":1.788},"last":0.54400000000000004}]},{"osd":4,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63700000000000001,"15min":0.65100000000000002},"min":{"1min":0.436,"5min":0.374,"15min":0.374},"max":{"1min":0.90400000000000003,"5min":1.006,"15min":1.006},"last":0.63},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.68100000000000005,"15min":0.68500000000000005},"min":{"1min":0.436,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81799999999999995,"5min":2.718,"15min":2.718},"last":0.76700000000000002}]},{"osd":5,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.75600000000000001,"15min":0.70699999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83399999999999996,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.879,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.59299999999999997}]},{"osd":7,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71999999999999997,"15min":0.74099999999999999},"min":{"1min":0.55100000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":1.095,"15min":1.095},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.745,"15min":0.76300000000000001},"min":{"1min":0.52300000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88200000000000001,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.68100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194791,"statfs":{"total":200000000,"available":199466739,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69699999999999995,"15min":0.70699999999999996},"min":{"1min":0.58199999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89300000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.69399999999999995,"15min":0.67400000000000004},"min":{"1min":0.63100000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.93600000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.497,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88600000000000001,"5min":1.216,"15min":1.216},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69599999999999995,"15min":0.68600000000000005},"min":{"1min":0.52900000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.84299999999999997,"5min":1.048,"15min":1.048},"last":0.56299999999999994}]},{"osd":2,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.72499999999999998,"15min":0.70099999999999996},"min":{"1min":0.56999999999999995,"5min":0.436,"15min":0.436},"max":{"1min":0.96499999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.69799999999999995,"15min":0.67600000000000005},"min":{"1min":0.61299999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92000000000000004,"5min":1.252,"15min":1.252},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.88600000000000001,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.746,"15min":0.73399999999999999},"min":{"1min":0.56299999999999994,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95299999999999996,"5min":1.111,"15min":1.111},"last":0.52800000000000002}]},{"osd":4,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60299999999999998,"15min":0.63100000000000001},"min":{"1min":0.42899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92800000000000005,"5min":1.272,"15min":1.272},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.44400000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.91700000000000004,"5min":1.038,"15min":1.038},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.49099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65100000000000002,"15min":0.66300000000000003},"min":{"1min":0.41499999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.877,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.442}]},{"osd":7,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71499999999999997,"15min":0.71699999999999997},"min":{"1min":0.53000000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77800000000000002,"15min":0.79600000000000004},"min":{"1min":0.57799999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95899999999999996,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.72899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194791,"statfs":{"total":200000000,"available":199466739,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.78000000000000003,"15min":0.747},"min":{"1min":0.42899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":6.6539999999999999,"5min":6.6539999999999999,"15min":6.6539999999999999},"last":0.755},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.745,"15min":0.72699999999999998},"min":{"1min":0.54600000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":5.4370000000000003,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":0.60799999999999998}]},{"osd":1,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69299999999999995,"15min":0.71999999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.71099999999999997,"15min":0.70299999999999996},"min":{"1min":0.51100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.877}]},{"osd":2,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.69399999999999995,"15min":0.68600000000000005},"min":{"1min":0.497,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.58899999999999997,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96599999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.85499999999999998}]},{"osd":3,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.68100000000000005,"15min":0.68799999999999994},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.54600000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2470000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.71399999999999997}]},{"osd":5,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63,"15min":0.64500000000000002},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.45400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.68200000000000005}]},{"osd":6,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67400000000000004,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.621},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.82999999999999996}]},{"osd":7,"last update":"Wed May 22 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69599999999999995,"15min":0.69999999999999996},"min":{"1min":0.499,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.751,"15min":0.78300000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97799999999999998,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.77900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194791,"statfs":{"total":200000000,"available":199466739,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.42099999999999999,"15min":0.40400000000000003},"min":{"1min":0.26100000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.432,"15min":0.40500000000000003},"min":{"1min":0.28799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65600000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.372}]},{"osd":1,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.45800000000000002,"15min":0.44700000000000001},"min":{"1min":0.29599999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.52300000000000002,"15min":0.505},"min":{"1min":0.44900000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.49399999999999999},"min":{"1min":0.40999999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.66100000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.505,"15min":0.48799999999999999},"min":{"1min":0.34999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.45400000000000001}]},{"osd":4,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72899999999999998,"15min":0.70199999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.73899999999999999,"15min":0.71499999999999997},"min":{"1min":0.60099999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.397,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.71899999999999997}]},{"osd":5,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73599999999999999,"15min":0.69199999999999995},"min":{"1min":0.59399999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.752},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.73499999999999999,"15min":0.69899999999999995},"min":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.292,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.78200000000000003}]},{"osd":6,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73699999999999999,"15min":0.71299999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.73799999999999999,"15min":0.70099999999999996},"min":{"1min":0.55300000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.81699999999999995}]},{"osd":7,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.78700000000000003,"15min":0.755},"min":{"1min":0.63,"5min":0.495,"15min":0.495},"max":{"1min":1.1000000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.76800000000000002,"15min":0.73499999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.79100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194791,"statfs":{"total":200000000,"available":199466739,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40799999999999997,"15min":0.39800000000000002},"min":{"1min":0.26000000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.44600000000000001,"15min":0.437},"min":{"1min":0.308,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51600000000000001}]},{"osd":1,"last update":"Wed May 22 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.32900000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52500000000000002,"15min":0.51500000000000001},"min":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55500000000000005}]},{"osd":3,"last update":"Wed May 22 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.502,"15min":0.498},"min":{"1min":0.34399999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.496,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.35999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.46899999999999997}]},{"osd":4,"last update":"Wed May 22 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71799999999999997,"15min":0.68500000000000005},"min":{"1min":0.56499999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.55000000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.113,"5min":1.173,"15min":1.173},"last":0.65400000000000003}]},{"osd":5,"last update":"Wed May 22 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.71199999999999997,"15min":0.69999999999999996},"min":{"1min":0.48099999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0509999999999999,"5min":1.143,"15min":1.143},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.72199999999999998,"15min":0.70099999999999996},"min":{"1min":0.55300000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.71699999999999997}]},{"osd":6,"last update":"Wed May 22 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72099999999999997,"15min":0.69999999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.98799999999999999,"5min":1.054,"15min":1.054},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed May 22 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.745,"15min":0.72499999999999998},"min":{"1min":0.61799999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.98799999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.78300000000000003,"15min":0.77000000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0680000000000001,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.75700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194791,"statfs":{"total":200000000,"available":199466739,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42199999999999999,"15min":0.432},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56999999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.44900000000000001,"15min":0.433},"min":{"1min":0.28399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.84799999999999998,"5min":0.876,"15min":0.876},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48899999999999999,"15min":0.48999999999999999},"min":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.748,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.51100000000000001,"15min":0.53600000000000003},"min":{"1min":0.35699999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64700000000000002,"5min":1.923,"15min":1.923},"last":0.42799999999999999}]},{"osd":3,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.57099999999999995},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.49099999999999999,"15min":0.48699999999999999},"min":{"1min":0.34599999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72099999999999997,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.45500000000000002}]},{"osd":4,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.74099999999999999,"15min":0.76000000000000001},"min":{"1min":0.52300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.999,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.70699999999999996,"15min":0.68300000000000005},"min":{"1min":0.504,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0389999999999999,"5min":1.3,"15min":1.3},"last":0.53200000000000003}]},{"osd":5,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.53500000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0089999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.751,"15min":0.753},"min":{"1min":0.56499999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1000000000000001,"5min":1.97,"15min":1.97},"last":0.621}]},{"osd":6,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.76300000000000001,"15min":0.745},"min":{"1min":0.54600000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0169999999999999,"5min":1.272,"15min":1.272},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.77000000000000002,"15min":0.745},"min":{"1min":0.64700000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.04,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.50600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0129999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76300000000000001,"15min":0.77900000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.91200000000000003,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.69299999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1096,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194215,"statfs":{"total":200000000,"available":198876990,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42599999999999999,"15min":0.40400000000000003},"min":{"1min":0.32200000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.73499999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.504,"15min":0.48199999999999998},"min":{"1min":0.378,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67300000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.47699999999999998}]},{"osd":2,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47699999999999998,"15min":0.45100000000000001},"min":{"1min":0.33200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70499999999999996,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.439,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76500000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.54000000000000004}]},{"osd":3,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.45000000000000001,"15min":0.432},"min":{"1min":0.30199999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68899999999999995,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51900000000000002,"15min":0.498},"min":{"1min":0.42499999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79300000000000004,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.50900000000000001}]},{"osd":4,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72699999999999998,"15min":0.70899999999999996},"min":{"1min":0.56000000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.499},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.72699999999999998},"min":{"1min":0.52600000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0049999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.57099999999999995}]},{"osd":5,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97199999999999998,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.75600000000000001,"15min":0.73499999999999999},"min":{"1min":0.58699999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1220000000000001,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.59799999999999998}]},{"osd":6,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78800000000000003,"15min":0.77100000000000002},"min":{"1min":0.65400000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0820000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.77500000000000002,"15min":0.77000000000000002},"min":{"1min":0.58499999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0640000000000001,"5min":1.5,"15min":1.5},"last":0.65100000000000002}]},{"osd":7,"last update":"Wed May 22 21:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.74099999999999999,"15min":0.72199999999999998},"min":{"1min":0.61199999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.0589999999999999,"5min":1.085,"15min":1.085},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.65000000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.1990000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.55900000000000005}]}]}],"pool_statfs":[]}} 2024-05-22T21:18:22.400 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:18:22.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.571+0000 7ff889600640 1 Processor -- start 2024-05-22T21:18:22.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.571+0000 7ff889600640 1 -- start start 2024-05-22T21:18:22.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.575+0000 7ff889600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff88415c6d0 0x7ff884164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.575+0000 7ff889600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff884071670 con 0x7ff88415dd50 2024-05-22T21:18:22.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.575+0000 7ff889600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff884070a60 con 0x7ff88415e130 2024-05-22T21:18:22.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.575+0000 7ff889600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff884070d40 con 0x7ff88415c6d0 2024-05-22T21:18:22.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.575+0000 7ff882a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff88415dd50 0x7ff88406d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:40302/0 (socket says 172.21.15.161:40302) 2024-05-22T21:18:22.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.575+0000 7ff882a00640 1 -- 172.21.15.161:0/713671374 learned_addr learned my addr 172.21.15.161:0/713671374 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:22.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.575+0000 7ff883400640 1 --2- 172.21.15.161:0/713671374 >> v2:172.21.15.179:3300/0 conn(0x7ff88415c6d0 0x7ff884164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.575+0000 7ff881600640 1 -- 172.21.15.161:0/713671374 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960676163 0 0) 0x7ff884071670 con 0x7ff88415dd50 2024-05-22T21:18:22.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.575+0000 7ff881600640 1 -- 172.21.15.161:0/713671374 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff868003530 con 0x7ff88415dd50 2024-05-22T21:18:22.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.575+0000 7ff883400640 1 -- 172.21.15.161:0/713671374 >> v1:172.21.15.179:6789/0 conn(0x7ff88415e130 legacy=0x7ff88415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.575+0000 7ff883400640 1 -- 172.21.15.161:0/713671374 >> v1:172.21.15.161:6789/0 conn(0x7ff88415dd50 legacy=0x7ff88406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.575+0000 7ff883400640 1 -- 172.21.15.161:0/713671374 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff884071020 con 0x7ff88415c6d0 2024-05-22T21:18:22.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.579+0000 7ff883400640 1 --2- 172.21.15.161:0/713671374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff88415c6d0 0x7ff884164e80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0097c0 tx=0x7ff87c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7a955945dc0d612 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:22.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.579+0000 7ff881600640 1 -- 172.21.15.161:0/713671374 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c0a7070 con 0x7ff88415c6d0 2024-05-22T21:18:22.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.579+0000 7ff881600640 1 -- 172.21.15.161:0/713671374 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c0ab430 con 0x7ff88415c6d0 2024-05-22T21:18:22.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.579+0000 7ff881600640 1 -- 172.21.15.161:0/713671374 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c0ab7c0 con 0x7ff88415c6d0 2024-05-22T21:18:22.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.579+0000 7ff889600640 1 -- 172.21.15.161:0/713671374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff88415c6d0 msgr2=0x7ff884164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.579+0000 7ff889600640 1 --2- 172.21.15.161:0/713671374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff88415c6d0 0x7ff884164e80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0097c0 tx=0x7ff87c01c470 comp rx=0 tx=0).stop 2024-05-22T21:18:22.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.579+0000 7ff889600640 1 -- 172.21.15.161:0/713671374 shutdown_connections 2024-05-22T21:18:22.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.579+0000 7ff889600640 1 --2- 172.21.15.161:0/713671374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff88415c6d0 0x7ff884164e80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.579+0000 7ff889600640 1 -- 172.21.15.161:0/713671374 >> 172.21.15.161:0/713671374 conn(0x7ff8840b0bc0 msgr2=0x7ff8840b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:22.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.579+0000 7ff889600640 1 -- 172.21.15.161:0/713671374 shutdown_connections 2024-05-22T21:18:22.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.579+0000 7ff889600640 1 -- 172.21.15.161:0/713671374 wait complete. 2024-05-22T21:18:22.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.579+0000 7ff889600640 1 Processor -- start 2024-05-22T21:18:22.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.579+0000 7ff889600640 1 -- start start 2024-05-22T21:18:22.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff889600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff88415c6d0 0x7ff88414be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff889600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff88415dd50 0x7ff8840b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff889600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff88415e130 0x7ff88414c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff889600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88414c380 con 0x7ff88415c6d0 2024-05-22T21:18:22.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff889600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88415b500 con 0x7ff88415e130 2024-05-22T21:18:22.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff889600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff884158c90 con 0x7ff88415dd50 2024-05-22T21:18:22.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff882a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff88415c6d0 0x7ff88414be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff882a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff88415c6d0 0x7ff88414be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38600/0 (socket says 172.21.15.161:38600) 2024-05-22T21:18:22.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff882a00640 1 -- 172.21.15.161:0/2750051546 learned_addr learned my addr 172.21.15.161:0/2750051546 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:22.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff883400640 1 --2- 172.21.15.161:0/2750051546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff88415e130 0x7ff88414c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff882000640 1 --2- 172.21.15.161:0/2750051546 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff88415dd50 0x7ff8840b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff882a00640 1 -- 172.21.15.161:0/2750051546 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff88415dd50 msgr2=0x7ff8840b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff882a00640 1 --2- 172.21.15.161:0/2750051546 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff88415dd50 0x7ff8840b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff882a00640 1 -- 172.21.15.161:0/2750051546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff88415e130 msgr2=0x7ff88414c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff882a00640 1 --2- 172.21.15.161:0/2750051546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff88415e130 0x7ff88414c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff882a00640 1 -- 172.21.15.161:0/2750051546 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c009420 con 0x7ff88415c6d0 2024-05-22T21:18:22.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff883400640 1 --2- 172.21.15.161:0/2750051546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff88415e130 0x7ff88414c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:22.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff882a00640 1 --2- 172.21.15.161:0/2750051546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff88415c6d0 0x7ff88414be40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff87400cbc0 tx=0x7ff87400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:22.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff87aa00640 1 -- 172.21.15.161:0/2750051546 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87400ace0 con 0x7ff88415c6d0 2024-05-22T21:18:22.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff87aa00640 1 -- 172.21.15.161:0/2750051546 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87400ae40 con 0x7ff88415c6d0 2024-05-22T21:18:22.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff87aa00640 1 -- 172.21.15.161:0/2750051546 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff874012620 con 0x7ff88415c6d0 2024-05-22T21:18:22.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff889600640 1 -- 172.21.15.161:0/2750051546 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff88415b220 con 0x7ff88415c6d0 2024-05-22T21:18:22.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff889600640 1 -- 172.21.15.161:0/2750051546 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff884158920 con 0x7ff88415c6d0 2024-05-22T21:18:22.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff889600640 1 -- 172.21.15.161:0/2750051546 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff884067220 con 0x7ff88415c6d0 2024-05-22T21:18:22.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.583+0000 7ff87aa00640 1 -- 172.21.15.161:0/2750051546 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff874007070 con 0x7ff88415c6d0 2024-05-22T21:18:22.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.587+0000 7ff87aa00640 1 --2- 172.21.15.161:0/2750051546 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff864066690 0x7ff864068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.587+0000 7ff87aa00640 1 -- 172.21.15.161:0/2750051546 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8740841f0 con 0x7ff88415c6d0 2024-05-22T21:18:22.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.587+0000 7ff882000640 1 --2- 172.21.15.161:0/2750051546 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff864066690 0x7ff864068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.587+0000 7ff882000640 1 --2- 172.21.15.161:0/2750051546 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff864066690 0x7ff864068b50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff8840b05f0 tx=0x7ff870009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:22.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.591+0000 7ff87aa00640 1 -- 172.21.15.161:0/2750051546 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff8740514c0 con 0x7ff88415c6d0 2024-05-22T21:18:22.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.739+0000 7ff889600640 1 -- 172.21.15.161:0/2750051546 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8840ad250 con 0x7ff864066690 2024-05-22T21:18:22.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.747+0000 7ff87aa00640 1 -- 172.21.15.161:0/2750051546 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284772 (secure 0 0 0) 0x7ff8840ad250 con 0x7ff864066690 2024-05-22T21:18:22.755 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:22.757 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:18:22.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.751+0000 7ff889600640 1 -- 172.21.15.161:0/2750051546 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff864066690 msgr2=0x7ff864068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.751+0000 7ff889600640 1 --2- 172.21.15.161:0/2750051546 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff864066690 0x7ff864068b50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff8840b05f0 tx=0x7ff870009040 comp rx=0 tx=0).stop 2024-05-22T21:18:22.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.751+0000 7ff889600640 1 -- 172.21.15.161:0/2750051546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff88415c6d0 msgr2=0x7ff88414be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.751+0000 7ff889600640 1 --2- 172.21.15.161:0/2750051546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff88415c6d0 0x7ff88414be40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff87400cbc0 tx=0x7ff87400a430 comp rx=0 tx=0).stop 2024-05-22T21:18:22.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.751+0000 7ff889600640 1 -- 172.21.15.161:0/2750051546 shutdown_connections 2024-05-22T21:18:22.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.751+0000 7ff889600640 1 --2- 172.21.15.161:0/2750051546 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff864066690 0x7ff864068b50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.751+0000 7ff889600640 1 --2- 172.21.15.161:0/2750051546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff88415e130 0x7ff88414c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.751+0000 7ff889600640 1 --2- 172.21.15.161:0/2750051546 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff88415dd50 0x7ff8840b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.751+0000 7ff889600640 1 --2- 172.21.15.161:0/2750051546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff88415c6d0 0x7ff88414be40 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.751+0000 7ff889600640 1 -- 172.21.15.161:0/2750051546 >> 172.21.15.161:0/2750051546 conn(0x7ff8840b0bc0 msgr2=0x7ff8841533f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:22.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.751+0000 7ff889600640 1 -- 172.21.15.161:0/2750051546 shutdown_connections 2024-05-22T21:18:22.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.751+0000 7ff889600640 1 -- 172.21.15.161:0/2750051546 wait complete. 2024-05-22T21:18:22.774 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-05-22T21:18:22.459831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5519,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556980,"statfs":{"total":1600000000,"available":1594347735,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.906544"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170375+0000","last_change":"2024-05-22T21:18:17.960599+0000","last_active":"2024-05-22T21:18:19.170375+0000","last_peered":"2024-05-22T21:18:19.170375+0000","last_clean":"2024-05-22T21:18:19.170375+0000","last_became_active":"2024-05-22T21:18:17.960246+0000","last_became_peered":"2024-05-22T21:18:17.960246+0000","last_unstale":"2024-05-22T21:18:19.170375+0000","last_undegraded":"2024-05-22T21:18:19.170375+0000","last_fullsized":"2024-05-22T21:18:19.170375+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:54:13.859086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169155+0000","last_change":"2024-05-22T21:18:17.963308+0000","last_active":"2024-05-22T21:18:19.169155+0000","last_peered":"2024-05-22T21:18:19.169155+0000","last_clean":"2024-05-22T21:18:19.169155+0000","last_became_active":"2024-05-22T21:18:17.962108+0000","last_became_peered":"2024-05-22T21:18:17.962108+0000","last_unstale":"2024-05-22T21:18:19.169155+0000","last_undegraded":"2024-05-22T21:18:19.169155+0000","last_fullsized":"2024-05-22T21:18:19.169155+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:25:51.935444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953613+0000","last_change":"2024-05-22T21:18:17.960645+0000","last_active":"2024-05-22T21:18:18.953613+0000","last_peered":"2024-05-22T21:18:18.953613+0000","last_clean":"2024-05-22T21:18:18.953613+0000","last_became_active":"2024-05-22T21:18:17.960427+0000","last_became_peered":"2024-05-22T21:18:17.960427+0000","last_unstale":"2024-05-22T21:18:18.953613+0000","last_undegraded":"2024-05-22T21:18:18.953613+0000","last_fullsized":"2024-05-22T21:18:18.953613+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:52:41.544599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.392835+0000","last_change":"2024-05-22T21:18:17.959284+0000","last_active":"2024-05-22T21:18:19.392835+0000","last_peered":"2024-05-22T21:18:19.392835+0000","last_clean":"2024-05-22T21:18:19.392835+0000","last_became_active":"2024-05-22T21:18:17.957127+0000","last_became_peered":"2024-05-22T21:18:17.957127+0000","last_unstale":"2024-05-22T21:18:19.392835+0000","last_undegraded":"2024-05-22T21:18:19.392835+0000","last_fullsized":"2024-05-22T21:18:19.392835+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:35:51.060507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954426+0000","last_change":"2024-05-22T21:18:17.965991+0000","last_active":"2024-05-22T21:18:18.954426+0000","last_peered":"2024-05-22T21:18:18.954426+0000","last_clean":"2024-05-22T21:18:18.954426+0000","last_became_active":"2024-05-22T21:18:17.965764+0000","last_became_peered":"2024-05-22T21:18:17.965764+0000","last_unstale":"2024-05-22T21:18:18.954426+0000","last_undegraded":"2024-05-22T21:18:18.954426+0000","last_fullsized":"2024-05-22T21:18:18.954426+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:31:06.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393978+0000","last_change":"2024-05-22T21:18:17.953557+0000","last_active":"2024-05-22T21:18:19.393978+0000","last_peered":"2024-05-22T21:18:19.393978+0000","last_clean":"2024-05-22T21:18:19.393978+0000","last_became_active":"2024-05-22T21:18:17.952902+0000","last_became_peered":"2024-05-22T21:18:17.952902+0000","last_unstale":"2024-05-22T21:18:19.393978+0000","last_undegraded":"2024-05-22T21:18:19.393978+0000","last_fullsized":"2024-05-22T21:18:19.393978+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:49.458817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393470+0000","last_change":"2024-05-22T21:18:17.953697+0000","last_active":"2024-05-22T21:18:19.393470+0000","last_peered":"2024-05-22T21:18:19.393470+0000","last_clean":"2024-05-22T21:18:19.393470+0000","last_became_active":"2024-05-22T21:18:17.953466+0000","last_became_peered":"2024-05-22T21:18:17.953466+0000","last_unstale":"2024-05-22T21:18:19.393470+0000","last_undegraded":"2024-05-22T21:18:19.393470+0000","last_fullsized":"2024-05-22T21:18:19.393470+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:37:13.521838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393857+0000","last_change":"2024-05-22T21:18:17.957649+0000","last_active":"2024-05-22T21:18:19.393857+0000","last_peered":"2024-05-22T21:18:19.393857+0000","last_clean":"2024-05-22T21:18:19.393857+0000","last_became_active":"2024-05-22T21:18:17.957409+0000","last_became_peered":"2024-05-22T21:18:17.957409+0000","last_unstale":"2024-05-22T21:18:19.393857+0000","last_undegraded":"2024-05-22T21:18:19.393857+0000","last_fullsized":"2024-05-22T21:18:19.393857+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:22:06.607488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393996+0000","last_change":"2024-05-22T21:18:17.952455+0000","last_active":"2024-05-22T21:18:19.393996+0000","last_peered":"2024-05-22T21:18:19.393996+0000","last_clean":"2024-05-22T21:18:19.393996+0000","last_became_active":"2024-05-22T21:18:17.952241+0000","last_became_peered":"2024-05-22T21:18:17.952241+0000","last_unstale":"2024-05-22T21:18:19.393996+0000","last_undegraded":"2024-05-22T21:18:19.393996+0000","last_fullsized":"2024-05-22T21:18:19.393996+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:09:17.217291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170339+0000","last_change":"2024-05-22T21:18:17.962139+0000","last_active":"2024-05-22T21:18:19.170339+0000","last_peered":"2024-05-22T21:18:19.170339+0000","last_clean":"2024-05-22T21:18:19.170339+0000","last_became_active":"2024-05-22T21:18:17.961250+0000","last_became_peered":"2024-05-22T21:18:17.961250+0000","last_unstale":"2024-05-22T21:18:19.170339+0000","last_undegraded":"2024-05-22T21:18:19.170339+0000","last_fullsized":"2024-05-22T21:18:19.170339+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:56:49.462456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169245+0000","last_change":"2024-05-22T21:18:17.957003+0000","last_active":"2024-05-22T21:18:19.169245+0000","last_peered":"2024-05-22T21:18:19.169245+0000","last_clean":"2024-05-22T21:18:19.169245+0000","last_became_active":"2024-05-22T21:18:17.956399+0000","last_became_peered":"2024-05-22T21:18:17.956399+0000","last_unstale":"2024-05-22T21:18:19.169245+0000","last_undegraded":"2024-05-22T21:18:19.169245+0000","last_fullsized":"2024-05-22T21:18:19.169245+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:20.225993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170530+0000","last_change":"2024-05-22T21:18:17.958838+0000","last_active":"2024-05-22T21:18:19.170530+0000","last_peered":"2024-05-22T21:18:19.170530+0000","last_clean":"2024-05-22T21:18:19.170530+0000","last_became_active":"2024-05-22T21:18:17.956822+0000","last_became_peered":"2024-05-22T21:18:17.956822+0000","last_unstale":"2024-05-22T21:18:19.170530+0000","last_undegraded":"2024-05-22T21:18:19.170530+0000","last_fullsized":"2024-05-22T21:18:19.170530+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:16.013875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953716+0000","last_change":"2024-05-22T21:18:17.959719+0000","last_active":"2024-05-22T21:18:18.953716+0000","last_peered":"2024-05-22T21:18:18.953716+0000","last_clean":"2024-05-22T21:18:18.953716+0000","last_became_active":"2024-05-22T21:18:17.959502+0000","last_became_peered":"2024-05-22T21:18:17.959502+0000","last_unstale":"2024-05-22T21:18:18.953716+0000","last_undegraded":"2024-05-22T21:18:18.953716+0000","last_fullsized":"2024-05-22T21:18:18.953716+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:41:21.219804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.392913+0000","last_change":"2024-05-22T21:18:17.959416+0000","last_active":"2024-05-22T21:18:19.392913+0000","last_peered":"2024-05-22T21:18:19.392913+0000","last_clean":"2024-05-22T21:18:19.392913+0000","last_became_active":"2024-05-22T21:18:17.957403+0000","last_became_peered":"2024-05-22T21:18:17.957403+0000","last_unstale":"2024-05-22T21:18:19.392913+0000","last_undegraded":"2024-05-22T21:18:19.392913+0000","last_fullsized":"2024-05-22T21:18:19.392913+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:59:08.119575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393041+0000","last_change":"2024-05-22T21:18:17.954286+0000","last_active":"2024-05-22T21:18:19.393041+0000","last_peered":"2024-05-22T21:18:19.393041+0000","last_clean":"2024-05-22T21:18:19.393041+0000","last_became_active":"2024-05-22T21:18:17.953720+0000","last_became_peered":"2024-05-22T21:18:17.953720+0000","last_unstale":"2024-05-22T21:18:19.393041+0000","last_undegraded":"2024-05-22T21:18:19.393041+0000","last_fullsized":"2024-05-22T21:18:19.393041+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:41:06.493942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953699+0000","last_change":"2024-05-22T21:18:17.964022+0000","last_active":"2024-05-22T21:18:18.953699+0000","last_peered":"2024-05-22T21:18:18.953699+0000","last_clean":"2024-05-22T21:18:18.953699+0000","last_became_active":"2024-05-22T21:18:17.963173+0000","last_became_peered":"2024-05-22T21:18:17.963173+0000","last_unstale":"2024-05-22T21:18:18.953699+0000","last_undegraded":"2024-05-22T21:18:18.953699+0000","last_fullsized":"2024-05-22T21:18:18.953699+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:46.836261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953976+0000","last_change":"2024-05-22T21:18:17.965200+0000","last_active":"2024-05-22T21:18:18.953976+0000","last_peered":"2024-05-22T21:18:18.953976+0000","last_clean":"2024-05-22T21:18:18.953976+0000","last_became_active":"2024-05-22T21:18:17.964346+0000","last_became_peered":"2024-05-22T21:18:17.964346+0000","last_unstale":"2024-05-22T21:18:18.953976+0000","last_undegraded":"2024-05-22T21:18:18.953976+0000","last_fullsized":"2024-05-22T21:18:18.953976+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:09:17.217291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393281+0000","last_change":"2024-05-22T21:18:17.963044+0000","last_active":"2024-05-22T21:18:19.393281+0000","last_peered":"2024-05-22T21:18:19.393281+0000","last_clean":"2024-05-22T21:18:19.393281+0000","last_became_active":"2024-05-22T21:18:17.961677+0000","last_became_peered":"2024-05-22T21:18:17.961677+0000","last_unstale":"2024-05-22T21:18:19.393281+0000","last_undegraded":"2024-05-22T21:18:19.393281+0000","last_fullsized":"2024-05-22T21:18:19.393281+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:28:09.553272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954454+0000","last_change":"2024-05-22T21:18:17.956830+0000","last_active":"2024-05-22T21:18:18.954454+0000","last_peered":"2024-05-22T21:18:18.954454+0000","last_clean":"2024-05-22T21:18:18.954454+0000","last_became_active":"2024-05-22T21:18:17.956077+0000","last_became_peered":"2024-05-22T21:18:17.956077+0000","last_unstale":"2024-05-22T21:18:18.954454+0000","last_undegraded":"2024-05-22T21:18:18.954454+0000","last_fullsized":"2024-05-22T21:18:18.954454+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:31:06.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393570+0000","last_change":"2024-05-22T21:18:17.962103+0000","last_active":"2024-05-22T21:18:19.393570+0000","last_peered":"2024-05-22T21:18:19.393570+0000","last_clean":"2024-05-22T21:18:19.393570+0000","last_became_active":"2024-05-22T21:18:17.960714+0000","last_became_peered":"2024-05-22T21:18:17.960714+0000","last_unstale":"2024-05-22T21:18:19.393570+0000","last_undegraded":"2024-05-22T21:18:19.393570+0000","last_fullsized":"2024-05-22T21:18:19.393570+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:08:44.201170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393427+0000","last_change":"2024-05-22T21:18:17.958661+0000","last_active":"2024-05-22T21:18:19.393427+0000","last_peered":"2024-05-22T21:18:19.393427+0000","last_clean":"2024-05-22T21:18:19.393427+0000","last_became_active":"2024-05-22T21:18:17.958456+0000","last_became_peered":"2024-05-22T21:18:17.958456+0000","last_unstale":"2024-05-22T21:18:19.393427+0000","last_undegraded":"2024-05-22T21:18:19.393427+0000","last_fullsized":"2024-05-22T21:18:19.393427+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:32:39.995038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169266+0000","last_change":"2024-05-22T21:18:17.963540+0000","last_active":"2024-05-22T21:18:19.169266+0000","last_peered":"2024-05-22T21:18:19.169266+0000","last_clean":"2024-05-22T21:18:19.169266+0000","last_became_active":"2024-05-22T21:18:17.962903+0000","last_became_peered":"2024-05-22T21:18:17.962903+0000","last_unstale":"2024-05-22T21:18:19.169266+0000","last_undegraded":"2024-05-22T21:18:19.169266+0000","last_fullsized":"2024-05-22T21:18:19.169266+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:10:11.105319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170376+0000","last_change":"2024-05-22T21:18:17.966436+0000","last_active":"2024-05-22T21:18:19.170376+0000","last_peered":"2024-05-22T21:18:19.170376+0000","last_clean":"2024-05-22T21:18:19.170376+0000","last_became_active":"2024-05-22T21:18:17.964774+0000","last_became_peered":"2024-05-22T21:18:17.964774+0000","last_unstale":"2024-05-22T21:18:19.170376+0000","last_undegraded":"2024-05-22T21:18:19.170376+0000","last_fullsized":"2024-05-22T21:18:19.170376+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:22.052183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.392989+0000","last_change":"2024-05-22T21:18:17.959671+0000","last_active":"2024-05-22T21:18:19.392989+0000","last_peered":"2024-05-22T21:18:19.392989+0000","last_clean":"2024-05-22T21:18:19.392989+0000","last_became_active":"2024-05-22T21:18:17.957995+0000","last_became_peered":"2024-05-22T21:18:17.957995+0000","last_unstale":"2024-05-22T21:18:19.392989+0000","last_undegraded":"2024-05-22T21:18:19.392989+0000","last_fullsized":"2024-05-22T21:18:19.392989+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:57:39.101800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169181+0000","last_change":"2024-05-22T21:18:17.957318+0000","last_active":"2024-05-22T21:18:19.169181+0000","last_peered":"2024-05-22T21:18:19.169181+0000","last_clean":"2024-05-22T21:18:19.169181+0000","last_became_active":"2024-05-22T21:18:17.956778+0000","last_became_peered":"2024-05-22T21:18:17.956778+0000","last_unstale":"2024-05-22T21:18:19.169181+0000","last_undegraded":"2024-05-22T21:18:19.169181+0000","last_fullsized":"2024-05-22T21:18:19.169181+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:48:16.720449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954340+0000","last_change":"2024-05-22T21:18:17.955832+0000","last_active":"2024-05-22T21:18:18.954340+0000","last_peered":"2024-05-22T21:18:18.954340+0000","last_clean":"2024-05-22T21:18:18.954340+0000","last_became_active":"2024-05-22T21:18:17.955610+0000","last_became_peered":"2024-05-22T21:18:17.955610+0000","last_unstale":"2024-05-22T21:18:18.954340+0000","last_undegraded":"2024-05-22T21:18:18.954340+0000","last_fullsized":"2024-05-22T21:18:18.954340+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:22.052183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954287+0000","last_change":"2024-05-22T21:18:17.966183+0000","last_active":"2024-05-22T21:18:18.954287+0000","last_peered":"2024-05-22T21:18:18.954287+0000","last_clean":"2024-05-22T21:18:18.954287+0000","last_became_active":"2024-05-22T21:18:17.964912+0000","last_became_peered":"2024-05-22T21:18:17.964912+0000","last_unstale":"2024-05-22T21:18:18.954287+0000","last_undegraded":"2024-05-22T21:18:18.954287+0000","last_fullsized":"2024-05-22T21:18:18.954287+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:49.458817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170513+0000","last_change":"2024-05-22T21:18:17.965157+0000","last_active":"2024-05-22T21:18:19.170513+0000","last_peered":"2024-05-22T21:18:19.170513+0000","last_clean":"2024-05-22T21:18:19.170513+0000","last_became_active":"2024-05-22T21:18:17.960663+0000","last_became_peered":"2024-05-22T21:18:17.960663+0000","last_unstale":"2024-05-22T21:18:19.170513+0000","last_undegraded":"2024-05-22T21:18:19.170513+0000","last_fullsized":"2024-05-22T21:18:19.170513+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:49.458817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169172+0000","last_change":"2024-05-22T21:18:17.955463+0000","last_active":"2024-05-22T21:18:19.169172+0000","last_peered":"2024-05-22T21:18:19.169172+0000","last_clean":"2024-05-22T21:18:19.169172+0000","last_became_active":"2024-05-22T21:18:17.955244+0000","last_became_peered":"2024-05-22T21:18:17.955244+0000","last_unstale":"2024-05-22T21:18:19.169172+0000","last_undegraded":"2024-05-22T21:18:19.169172+0000","last_fullsized":"2024-05-22T21:18:19.169172+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:30:05.982663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169273+0000","last_change":"2024-05-22T21:18:17.963537+0000","last_active":"2024-05-22T21:18:19.169273+0000","last_peered":"2024-05-22T21:18:19.169273+0000","last_clean":"2024-05-22T21:18:19.169273+0000","last_became_active":"2024-05-22T21:18:17.960629+0000","last_became_peered":"2024-05-22T21:18:17.960629+0000","last_unstale":"2024-05-22T21:18:19.169273+0000","last_undegraded":"2024-05-22T21:18:19.169273+0000","last_fullsized":"2024-05-22T21:18:19.169273+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:42:08.700957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953865+0000","last_change":"2024-05-22T21:18:17.964234+0000","last_active":"2024-05-22T21:18:18.953865+0000","last_peered":"2024-05-22T21:18:18.953865+0000","last_clean":"2024-05-22T21:18:18.953865+0000","last_became_active":"2024-05-22T21:18:17.963492+0000","last_became_peered":"2024-05-22T21:18:17.963492+0000","last_unstale":"2024-05-22T21:18:18.953865+0000","last_undegraded":"2024-05-22T21:18:18.953865+0000","last_fullsized":"2024-05-22T21:18:18.953865+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:22.295248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170669+0000","last_change":"2024-05-22T21:18:17.960444+0000","last_active":"2024-05-22T21:18:19.170669+0000","last_peered":"2024-05-22T21:18:19.170669+0000","last_clean":"2024-05-22T21:18:19.170669+0000","last_became_active":"2024-05-22T21:18:17.960207+0000","last_became_peered":"2024-05-22T21:18:17.960207+0000","last_unstale":"2024-05-22T21:18:19.170669+0000","last_undegraded":"2024-05-22T21:18:19.170669+0000","last_fullsized":"2024-05-22T21:18:19.170669+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:22.295248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953441+0000","last_change":"2024-05-22T21:18:17.958767+0000","last_active":"2024-05-22T21:18:18.953441+0000","last_peered":"2024-05-22T21:18:18.953441+0000","last_clean":"2024-05-22T21:18:18.953441+0000","last_became_active":"2024-05-22T21:18:17.958481+0000","last_became_peered":"2024-05-22T21:18:17.958481+0000","last_unstale":"2024-05-22T21:18:18.953441+0000","last_undegraded":"2024-05-22T21:18:18.953441+0000","last_fullsized":"2024-05-22T21:18:18.953441+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:23.958727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"102'4","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170722+0000","last_change":"2024-05-22T21:18:17.577456+0000","last_active":"2024-05-22T21:18:19.170722+0000","last_peered":"2024-05-22T21:18:19.170722+0000","last_clean":"2024-05-22T21:18:19.170722+0000","last_became_active":"2024-05-22T21:18:17.576784+0000","last_became_peered":"2024-05-22T21:18:17.576784+0000","last_unstale":"2024-05-22T21:18:19.170722+0000","last_undegraded":"2024-05-22T21:18:19.170722+0000","last_fullsized":"2024-05-22T21:18:19.170722+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:48:15.718335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170650+0000","last_change":"2024-05-22T21:18:17.965724+0000","last_active":"2024-05-22T21:18:19.170650+0000","last_peered":"2024-05-22T21:18:19.170650+0000","last_clean":"2024-05-22T21:18:19.170650+0000","last_became_active":"2024-05-22T21:18:17.962527+0000","last_became_peered":"2024-05-22T21:18:17.962527+0000","last_unstale":"2024-05-22T21:18:19.170650+0000","last_undegraded":"2024-05-22T21:18:19.170650+0000","last_fullsized":"2024-05-22T21:18:19.170650+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:09:17.217291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"102'1","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953680+0000","last_change":"2024-05-22T21:18:17.570132+0000","last_active":"2024-05-22T21:18:18.953680+0000","last_peered":"2024-05-22T21:18:18.953680+0000","last_clean":"2024-05-22T21:18:18.953680+0000","last_became_active":"2024-05-22T21:18:17.567690+0000","last_became_peered":"2024-05-22T21:18:17.567690+0000","last_unstale":"2024-05-22T21:18:18.953680+0000","last_undegraded":"2024-05-22T21:18:18.953680+0000","last_fullsized":"2024-05-22T21:18:18.953680+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:25:50.933329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170500+0000","last_change":"2024-05-22T21:18:17.963395+0000","last_active":"2024-05-22T21:18:19.170500+0000","last_peered":"2024-05-22T21:18:19.170500+0000","last_clean":"2024-05-22T21:18:19.170500+0000","last_became_active":"2024-05-22T21:18:17.963135+0000","last_became_peered":"2024-05-22T21:18:17.963135+0000","last_unstale":"2024-05-22T21:18:19.170500+0000","last_undegraded":"2024-05-22T21:18:19.170500+0000","last_fullsized":"2024-05-22T21:18:19.170500+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:31:06.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954515+0000","last_change":"2024-05-22T21:18:17.569334+0000","last_active":"2024-05-22T21:18:18.954515+0000","last_peered":"2024-05-22T21:18:18.954515+0000","last_clean":"2024-05-22T21:18:18.954515+0000","last_became_active":"2024-05-22T21:18:17.566887+0000","last_became_peered":"2024-05-22T21:18:17.566887+0000","last_unstale":"2024-05-22T21:18:18.954515+0000","last_undegraded":"2024-05-22T21:18:18.954515+0000","last_fullsized":"2024-05-22T21:18:18.954515+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:13:03.234580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954048+0000","last_change":"2024-05-22T21:18:17.959066+0000","last_active":"2024-05-22T21:18:18.954048+0000","last_peered":"2024-05-22T21:18:18.954048+0000","last_clean":"2024-05-22T21:18:18.954048+0000","last_became_active":"2024-05-22T21:18:17.956795+0000","last_became_peered":"2024-05-22T21:18:17.956795+0000","last_unstale":"2024-05-22T21:18:18.954048+0000","last_undegraded":"2024-05-22T21:18:18.954048+0000","last_fullsized":"2024-05-22T21:18:18.954048+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:23.958727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954584+0000","last_change":"2024-05-22T21:18:17.570719+0000","last_active":"2024-05-22T21:18:18.954584+0000","last_peered":"2024-05-22T21:18:18.954584+0000","last_clean":"2024-05-22T21:18:18.954584+0000","last_became_active":"2024-05-22T21:18:17.570489+0000","last_became_peered":"2024-05-22T21:18:17.570489+0000","last_unstale":"2024-05-22T21:18:18.954584+0000","last_undegraded":"2024-05-22T21:18:18.954584+0000","last_fullsized":"2024-05-22T21:18:18.954584+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:58:09.369054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169227+0000","last_change":"2024-05-22T21:18:17.960911+0000","last_active":"2024-05-22T21:18:19.169227+0000","last_peered":"2024-05-22T21:18:19.169227+0000","last_clean":"2024-05-22T21:18:19.169227+0000","last_became_active":"2024-05-22T21:18:17.957746+0000","last_became_peered":"2024-05-22T21:18:17.957746+0000","last_unstale":"2024-05-22T21:18:19.169227+0000","last_undegraded":"2024-05-22T21:18:19.169227+0000","last_fullsized":"2024-05-22T21:18:19.169227+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:58:33.493841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"102'1","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954180+0000","last_change":"2024-05-22T21:18:17.575282+0000","last_active":"2024-05-22T21:18:18.954180+0000","last_peered":"2024-05-22T21:18:18.954180+0000","last_clean":"2024-05-22T21:18:18.954180+0000","last_became_active":"2024-05-22T21:18:17.574110+0000","last_became_peered":"2024-05-22T21:18:17.574110+0000","last_unstale":"2024-05-22T21:18:18.954180+0000","last_undegraded":"2024-05-22T21:18:18.954180+0000","last_fullsized":"2024-05-22T21:18:18.954180+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:48:15.718335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954075+0000","last_change":"2024-05-22T21:18:17.964643+0000","last_active":"2024-05-22T21:18:18.954075+0000","last_peered":"2024-05-22T21:18:18.954075+0000","last_clean":"2024-05-22T21:18:18.954075+0000","last_became_active":"2024-05-22T21:18:17.960412+0000","last_became_peered":"2024-05-22T21:18:17.960412+0000","last_unstale":"2024-05-22T21:18:18.954075+0000","last_undegraded":"2024-05-22T21:18:18.954075+0000","last_fullsized":"2024-05-22T21:18:18.954075+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:56:49.462456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"102'1","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954423+0000","last_change":"2024-05-22T21:18:17.571413+0000","last_active":"2024-05-22T21:18:18.954423+0000","last_peered":"2024-05-22T21:18:18.954423+0000","last_clean":"2024-05-22T21:18:18.954423+0000","last_became_active":"2024-05-22T21:18:17.569088+0000","last_became_peered":"2024-05-22T21:18:17.569088+0000","last_unstale":"2024-05-22T21:18:18.954423+0000","last_undegraded":"2024-05-22T21:18:18.954423+0000","last_fullsized":"2024-05-22T21:18:18.954423+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:19.223879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953862+0000","last_change":"2024-05-22T21:18:17.961734+0000","last_active":"2024-05-22T21:18:18.953862+0000","last_peered":"2024-05-22T21:18:18.953862+0000","last_clean":"2024-05-22T21:18:18.953862+0000","last_became_active":"2024-05-22T21:18:17.961203+0000","last_became_peered":"2024-05-22T21:18:17.961203+0000","last_unstale":"2024-05-22T21:18:18.953862+0000","last_undegraded":"2024-05-22T21:18:18.953862+0000","last_fullsized":"2024-05-22T21:18:18.953862+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:49.458817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954654+0000","last_change":"2024-05-22T21:18:17.567816+0000","last_active":"2024-05-22T21:18:18.954654+0000","last_peered":"2024-05-22T21:18:18.954654+0000","last_clean":"2024-05-22T21:18:18.954654+0000","last_became_active":"2024-05-22T21:18:17.567601+0000","last_became_peered":"2024-05-22T21:18:17.567601+0000","last_unstale":"2024-05-22T21:18:18.954654+0000","last_undegraded":"2024-05-22T21:18:18.954654+0000","last_fullsized":"2024-05-22T21:18:18.954654+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:54:12.856971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393016+0000","last_change":"2024-05-22T21:18:17.954207+0000","last_active":"2024-05-22T21:18:19.393016+0000","last_peered":"2024-05-22T21:18:19.393016+0000","last_clean":"2024-05-22T21:18:19.393016+0000","last_became_active":"2024-05-22T21:18:17.953977+0000","last_became_peered":"2024-05-22T21:18:17.953977+0000","last_unstale":"2024-05-22T21:18:19.393016+0000","last_undegraded":"2024-05-22T21:18:19.393016+0000","last_fullsized":"2024-05-22T21:18:19.393016+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:41:23.892969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169341+0000","last_change":"2024-05-22T21:18:17.566911+0000","last_active":"2024-05-22T21:18:19.169341+0000","last_peered":"2024-05-22T21:18:19.169341+0000","last_clean":"2024-05-22T21:18:19.169341+0000","last_became_active":"2024-05-22T21:18:17.566696+0000","last_became_peered":"2024-05-22T21:18:17.566696+0000","last_unstale":"2024-05-22T21:18:19.169341+0000","last_undegraded":"2024-05-22T21:18:19.169341+0000","last_fullsized":"2024-05-22T21:18:19.169341+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:18:20.507884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393137+0000","last_change":"2024-05-22T21:18:17.955339+0000","last_active":"2024-05-22T21:18:19.393137+0000","last_peered":"2024-05-22T21:18:19.393137+0000","last_clean":"2024-05-22T21:18:19.393137+0000","last_became_active":"2024-05-22T21:18:17.955173+0000","last_became_peered":"2024-05-22T21:18:17.955173+0000","last_unstale":"2024-05-22T21:18:19.393137+0000","last_undegraded":"2024-05-22T21:18:19.393137+0000","last_fullsized":"2024-05-22T21:18:19.393137+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:06:36.975586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170634+0000","last_change":"2024-05-22T21:18:17.574214+0000","last_active":"2024-05-22T21:18:19.170634+0000","last_peered":"2024-05-22T21:18:19.170634+0000","last_clean":"2024-05-22T21:18:19.170634+0000","last_became_active":"2024-05-22T21:18:17.573769+0000","last_became_peered":"2024-05-22T21:18:17.573769+0000","last_unstale":"2024-05-22T21:18:19.170634+0000","last_undegraded":"2024-05-22T21:18:19.170634+0000","last_fullsized":"2024-05-22T21:18:19.170634+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:10:10.103204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.394075+0000","last_change":"2024-05-22T21:18:17.953706+0000","last_active":"2024-05-22T21:18:19.394075+0000","last_peered":"2024-05-22T21:18:19.394075+0000","last_clean":"2024-05-22T21:18:19.394075+0000","last_became_active":"2024-05-22T21:18:17.953296+0000","last_became_peered":"2024-05-22T21:18:17.953296+0000","last_unstale":"2024-05-22T21:18:19.394075+0000","last_undegraded":"2024-05-22T21:18:19.394075+0000","last_fullsized":"2024-05-22T21:18:19.394075+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:16.013875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170743+0000","last_change":"2024-05-22T21:18:17.572117+0000","last_active":"2024-05-22T21:18:19.170743+0000","last_peered":"2024-05-22T21:18:19.170743+0000","last_clean":"2024-05-22T21:18:19.170743+0000","last_became_active":"2024-05-22T21:18:17.571601+0000","last_became_peered":"2024-05-22T21:18:17.571601+0000","last_unstale":"2024-05-22T21:18:19.170743+0000","last_undegraded":"2024-05-22T21:18:19.170743+0000","last_fullsized":"2024-05-22T21:18:19.170743+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:25:50.933329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954263+0000","last_change":"2024-05-22T21:18:17.957084+0000","last_active":"2024-05-22T21:18:18.954263+0000","last_peered":"2024-05-22T21:18:18.954263+0000","last_clean":"2024-05-22T21:18:18.954263+0000","last_became_active":"2024-05-22T21:18:17.956592+0000","last_became_peered":"2024-05-22T21:18:17.956592+0000","last_unstale":"2024-05-22T21:18:18.954263+0000","last_undegraded":"2024-05-22T21:18:18.954263+0000","last_fullsized":"2024-05-22T21:18:18.954263+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:22.052183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"102'3","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170828+0000","last_change":"2024-05-22T21:18:17.577310+0000","last_active":"2024-05-22T21:18:19.170828+0000","last_peered":"2024-05-22T21:18:19.170828+0000","last_clean":"2024-05-22T21:18:19.170828+0000","last_became_active":"2024-05-22T21:18:17.576229+0000","last_became_peered":"2024-05-22T21:18:17.576229+0000","last_unstale":"2024-05-22T21:18:19.170828+0000","last_undegraded":"2024-05-22T21:18:19.170828+0000","last_fullsized":"2024-05-22T21:18:19.170828+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:27:43.610883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954225+0000","last_change":"2024-05-22T21:18:17.962890+0000","last_active":"2024-05-22T21:18:18.954225+0000","last_peered":"2024-05-22T21:18:18.954225+0000","last_clean":"2024-05-22T21:18:18.954225+0000","last_became_active":"2024-05-22T21:18:17.962655+0000","last_became_peered":"2024-05-22T21:18:17.962655+0000","last_unstale":"2024-05-22T21:18:18.954225+0000","last_undegraded":"2024-05-22T21:18:18.954225+0000","last_fullsized":"2024-05-22T21:18:18.954225+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:09:27.584040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"102'6","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.394098+0000","last_change":"2024-05-22T21:18:17.570233+0000","last_active":"2024-05-22T21:18:19.394098+0000","last_peered":"2024-05-22T21:18:19.394098+0000","last_clean":"2024-05-22T21:18:19.394098+0000","last_became_active":"2024-05-22T21:18:17.569591+0000","last_became_peered":"2024-05-22T21:18:17.569591+0000","last_unstale":"2024-05-22T21:18:19.394098+0000","last_undegraded":"2024-05-22T21:18:19.394098+0000","last_fullsized":"2024-05-22T21:18:19.394098+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:35:29.469510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954400+0000","last_change":"2024-05-22T21:18:17.960697+0000","last_active":"2024-05-22T21:18:18.954400+0000","last_peered":"2024-05-22T21:18:18.954400+0000","last_clean":"2024-05-22T21:18:18.954400+0000","last_became_active":"2024-05-22T21:18:17.960207+0000","last_became_peered":"2024-05-22T21:18:17.960207+0000","last_unstale":"2024-05-22T21:18:18.954400+0000","last_undegraded":"2024-05-22T21:18:18.954400+0000","last_fullsized":"2024-05-22T21:18:18.954400+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:49.458817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169388+0000","last_change":"2024-05-22T21:18:17.571909+0000","last_active":"2024-05-22T21:18:19.169388+0000","last_peered":"2024-05-22T21:18:19.169388+0000","last_clean":"2024-05-22T21:18:19.169388+0000","last_became_active":"2024-05-22T21:18:17.571512+0000","last_became_peered":"2024-05-22T21:18:17.571512+0000","last_unstale":"2024-05-22T21:18:19.169388+0000","last_undegraded":"2024-05-22T21:18:19.169388+0000","last_fullsized":"2024-05-22T21:18:19.169388+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:36:46.664343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953954+0000","last_change":"2024-05-22T21:18:17.961953+0000","last_active":"2024-05-22T21:18:18.953954+0000","last_peered":"2024-05-22T21:18:18.953954+0000","last_clean":"2024-05-22T21:18:18.953954+0000","last_became_active":"2024-05-22T21:18:17.961516+0000","last_became_peered":"2024-05-22T21:18:17.961516+0000","last_unstale":"2024-05-22T21:18:18.953954+0000","last_undegraded":"2024-05-22T21:18:18.953954+0000","last_fullsized":"2024-05-22T21:18:18.953954+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:16.013875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393111+0000","last_change":"2024-05-22T21:18:17.572880+0000","last_active":"2024-05-22T21:18:19.393111+0000","last_peered":"2024-05-22T21:18:19.393111+0000","last_clean":"2024-05-22T21:18:19.393111+0000","last_became_active":"2024-05-22T21:18:17.572655+0000","last_became_peered":"2024-05-22T21:18:17.572655+0000","last_unstale":"2024-05-22T21:18:19.393111+0000","last_undegraded":"2024-05-22T21:18:19.393111+0000","last_fullsized":"2024-05-22T21:18:19.393111+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:21.050069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169092+0000","last_change":"2024-05-22T21:18:17.957857+0000","last_active":"2024-05-22T21:18:19.169092+0000","last_peered":"2024-05-22T21:18:19.169092+0000","last_clean":"2024-05-22T21:18:19.169092+0000","last_became_active":"2024-05-22T21:18:17.957491+0000","last_became_peered":"2024-05-22T21:18:17.957491+0000","last_unstale":"2024-05-22T21:18:19.169092+0000","last_undegraded":"2024-05-22T21:18:19.169092+0000","last_fullsized":"2024-05-22T21:18:19.169092+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:14:43.052939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393981+0000","last_change":"2024-05-22T21:18:17.569729+0000","last_active":"2024-05-22T21:18:19.393981+0000","last_peered":"2024-05-22T21:18:19.393981+0000","last_clean":"2024-05-22T21:18:19.393981+0000","last_became_active":"2024-05-22T21:18:17.567831+0000","last_became_peered":"2024-05-22T21:18:17.567831+0000","last_unstale":"2024-05-22T21:18:19.393981+0000","last_undegraded":"2024-05-22T21:18:19.393981+0000","last_fullsized":"2024-05-22T21:18:19.393981+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:42:10.010243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169360+0000","last_change":"2024-05-22T21:18:17.963428+0000","last_active":"2024-05-22T21:18:19.169360+0000","last_peered":"2024-05-22T21:18:19.169360+0000","last_clean":"2024-05-22T21:18:19.169360+0000","last_became_active":"2024-05-22T21:18:17.962429+0000","last_became_peered":"2024-05-22T21:18:17.962429+0000","last_unstale":"2024-05-22T21:18:19.169360+0000","last_undegraded":"2024-05-22T21:18:19.169360+0000","last_fullsized":"2024-05-22T21:18:19.169360+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:19:44.456977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"102'8","reported_seq":19,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954520+0000","last_change":"2024-05-22T21:18:17.570044+0000","last_active":"2024-05-22T21:18:18.954520+0000","last_peered":"2024-05-22T21:18:18.954520+0000","last_clean":"2024-05-22T21:18:18.954520+0000","last_became_active":"2024-05-22T21:18:17.569796+0000","last_became_peered":"2024-05-22T21:18:17.569796+0000","last_unstale":"2024-05-22T21:18:18.954520+0000","last_undegraded":"2024-05-22T21:18:18.954520+0000","last_fullsized":"2024-05-22T21:18:18.954520+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:53:59.583140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954136+0000","last_change":"2024-05-22T21:18:17.957133+0000","last_active":"2024-05-22T21:18:18.954136+0000","last_peered":"2024-05-22T21:18:18.954136+0000","last_clean":"2024-05-22T21:18:18.954136+0000","last_became_active":"2024-05-22T21:18:17.954295+0000","last_became_peered":"2024-05-22T21:18:17.954295+0000","last_unstale":"2024-05-22T21:18:18.954136+0000","last_undegraded":"2024-05-22T21:18:18.954136+0000","last_fullsized":"2024-05-22T21:18:18.954136+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:45:07.161984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"102'1","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170585+0000","last_change":"2024-05-22T21:18:17.576505+0000","last_active":"2024-05-22T21:18:19.170585+0000","last_peered":"2024-05-22T21:18:19.170585+0000","last_clean":"2024-05-22T21:18:19.170585+0000","last_became_active":"2024-05-22T21:18:17.574374+0000","last_became_peered":"2024-05-22T21:18:17.574374+0000","last_unstale":"2024-05-22T21:18:19.170585+0000","last_undegraded":"2024-05-22T21:18:19.170585+0000","last_fullsized":"2024-05-22T21:18:19.170585+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:06:41.500056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170443+0000","last_change":"2024-05-22T21:18:17.962646+0000","last_active":"2024-05-22T21:18:19.170443+0000","last_peered":"2024-05-22T21:18:19.170443+0000","last_clean":"2024-05-22T21:18:19.170443+0000","last_became_active":"2024-05-22T21:18:17.962413+0000","last_became_peered":"2024-05-22T21:18:17.962413+0000","last_unstale":"2024-05-22T21:18:19.170443+0000","last_undegraded":"2024-05-22T21:18:19.170443+0000","last_fullsized":"2024-05-22T21:18:19.170443+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:09:27.584040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954565+0000","last_change":"2024-05-22T21:18:17.573846+0000","last_active":"2024-05-22T21:18:18.954565+0000","last_peered":"2024-05-22T21:18:18.954565+0000","last_clean":"2024-05-22T21:18:18.954565+0000","last_became_active":"2024-05-22T21:18:17.573596+0000","last_became_peered":"2024-05-22T21:18:17.573596+0000","last_unstale":"2024-05-22T21:18:18.954565+0000","last_undegraded":"2024-05-22T21:18:18.954565+0000","last_fullsized":"2024-05-22T21:18:18.954565+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:45.834147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953953+0000","last_change":"2024-05-22T21:18:17.966432+0000","last_active":"2024-05-22T21:18:18.953953+0000","last_peered":"2024-05-22T21:18:18.953953+0000","last_clean":"2024-05-22T21:18:18.953953+0000","last_became_active":"2024-05-22T21:18:17.966172+0000","last_became_peered":"2024-05-22T21:18:17.966172+0000","last_unstale":"2024-05-22T21:18:18.953953+0000","last_undegraded":"2024-05-22T21:18:18.953953+0000","last_fullsized":"2024-05-22T21:18:18.953953+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:16.013875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954299+0000","last_change":"2024-05-22T21:18:17.572905+0000","last_active":"2024-05-22T21:18:18.954299+0000","last_peered":"2024-05-22T21:18:18.954299+0000","last_clean":"2024-05-22T21:18:18.954299+0000","last_became_active":"2024-05-22T21:18:17.572686+0000","last_became_peered":"2024-05-22T21:18:17.572686+0000","last_unstale":"2024-05-22T21:18:18.954299+0000","last_undegraded":"2024-05-22T21:18:18.954299+0000","last_fullsized":"2024-05-22T21:18:18.954299+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:36.130802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393070+0000","last_change":"2024-05-22T21:18:17.959542+0000","last_active":"2024-05-22T21:18:19.393070+0000","last_peered":"2024-05-22T21:18:19.393070+0000","last_clean":"2024-05-22T21:18:19.393070+0000","last_became_active":"2024-05-22T21:18:17.957710+0000","last_became_peered":"2024-05-22T21:18:17.957710+0000","last_unstale":"2024-05-22T21:18:19.393070+0000","last_undegraded":"2024-05-22T21:18:19.393070+0000","last_fullsized":"2024-05-22T21:18:19.393070+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:48:21.058352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393023+0000","last_change":"2024-05-22T21:18:17.565636+0000","last_active":"2024-05-22T21:18:19.393023+0000","last_peered":"2024-05-22T21:18:19.393023+0000","last_clean":"2024-05-22T21:18:19.393023+0000","last_became_active":"2024-05-22T21:18:17.565314+0000","last_became_peered":"2024-05-22T21:18:17.565314+0000","last_unstale":"2024-05-22T21:18:19.393023+0000","last_undegraded":"2024-05-22T21:18:19.393023+0000","last_fullsized":"2024-05-22T21:18:19.393023+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:48.456703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393948+0000","last_change":"2024-05-22T21:18:17.955262+0000","last_active":"2024-05-22T21:18:19.393948+0000","last_peered":"2024-05-22T21:18:19.393948+0000","last_clean":"2024-05-22T21:18:19.393948+0000","last_became_active":"2024-05-22T21:18:17.955047+0000","last_became_peered":"2024-05-22T21:18:17.955047+0000","last_unstale":"2024-05-22T21:18:19.393948+0000","last_undegraded":"2024-05-22T21:18:19.393948+0000","last_fullsized":"2024-05-22T21:18:19.393948+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:22.295248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393142+0000","last_change":"2024-05-22T21:18:17.567966+0000","last_active":"2024-05-22T21:18:19.393142+0000","last_peered":"2024-05-22T21:18:19.393142+0000","last_clean":"2024-05-22T21:18:19.393142+0000","last_became_active":"2024-05-22T21:18:17.567024+0000","last_became_peered":"2024-05-22T21:18:17.567024+0000","last_unstale":"2024-05-22T21:18:19.393142+0000","last_undegraded":"2024-05-22T21:18:19.393142+0000","last_fullsized":"2024-05-22T21:18:19.393142+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:08:28.543920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393481+0000","last_change":"2024-05-22T21:18:17.953096+0000","last_active":"2024-05-22T21:18:19.393481+0000","last_peered":"2024-05-22T21:18:19.393481+0000","last_clean":"2024-05-22T21:18:19.393481+0000","last_became_active":"2024-05-22T21:18:17.952744+0000","last_became_peered":"2024-05-22T21:18:17.952744+0000","last_unstale":"2024-05-22T21:18:19.393481+0000","last_undegraded":"2024-05-22T21:18:19.393481+0000","last_fullsized":"2024-05-22T21:18:19.393481+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:21:00.882050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393609+0000","last_change":"2024-05-22T21:18:17.571043+0000","last_active":"2024-05-22T21:18:19.393609+0000","last_peered":"2024-05-22T21:18:19.393609+0000","last_clean":"2024-05-22T21:18:19.393609+0000","last_became_active":"2024-05-22T21:18:17.570750+0000","last_became_peered":"2024-05-22T21:18:17.570750+0000","last_unstale":"2024-05-22T21:18:19.393609+0000","last_undegraded":"2024-05-22T21:18:19.393609+0000","last_fullsized":"2024-05-22T21:18:19.393609+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:41:09.473523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393978+0000","last_change":"2024-05-22T21:18:17.961201+0000","last_active":"2024-05-22T21:18:19.393978+0000","last_peered":"2024-05-22T21:18:19.393978+0000","last_clean":"2024-05-22T21:18:19.393978+0000","last_became_active":"2024-05-22T21:18:17.960963+0000","last_became_peered":"2024-05-22T21:18:17.960963+0000","last_unstale":"2024-05-22T21:18:19.393978+0000","last_undegraded":"2024-05-22T21:18:19.393978+0000","last_fullsized":"2024-05-22T21:18:19.393978+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:56:49.462456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393183+0000","last_change":"2024-05-22T21:18:17.569485+0000","last_active":"2024-05-22T21:18:19.393183+0000","last_peered":"2024-05-22T21:18:19.393183+0000","last_clean":"2024-05-22T21:18:19.393183+0000","last_became_active":"2024-05-22T21:18:17.569276+0000","last_became_peered":"2024-05-22T21:18:17.569276+0000","last_unstale":"2024-05-22T21:18:19.393183+0000","last_undegraded":"2024-05-22T21:18:19.393183+0000","last_fullsized":"2024-05-22T21:18:19.393183+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:45:06.159869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954330+0000","last_change":"2024-05-22T21:18:17.957705+0000","last_active":"2024-05-22T21:18:18.954330+0000","last_peered":"2024-05-22T21:18:18.954330+0000","last_clean":"2024-05-22T21:18:18.954330+0000","last_became_active":"2024-05-22T21:18:17.957268+0000","last_became_peered":"2024-05-22T21:18:17.957268+0000","last_unstale":"2024-05-22T21:18:18.954330+0000","last_undegraded":"2024-05-22T21:18:18.954330+0000","last_fullsized":"2024-05-22T21:18:18.954330+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:13:04.236694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169584+0000","last_change":"2024-05-22T21:18:17.570186+0000","last_active":"2024-05-22T21:18:19.169584+0000","last_peered":"2024-05-22T21:18:19.169584+0000","last_clean":"2024-05-22T21:18:19.169584+0000","last_became_active":"2024-05-22T21:18:17.569973+0000","last_became_peered":"2024-05-22T21:18:17.569973+0000","last_unstale":"2024-05-22T21:18:19.169584+0000","last_undegraded":"2024-05-22T21:18:19.169584+0000","last_fullsized":"2024-05-22T21:18:19.169584+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:52:12.503228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954075+0000","last_change":"2024-05-22T21:18:17.958467+0000","last_active":"2024-05-22T21:18:18.954075+0000","last_peered":"2024-05-22T21:18:18.954075+0000","last_clean":"2024-05-22T21:18:18.954075+0000","last_became_active":"2024-05-22T21:18:17.958201+0000","last_became_peered":"2024-05-22T21:18:17.958201+0000","last_unstale":"2024-05-22T21:18:18.954075+0000","last_undegraded":"2024-05-22T21:18:18.954075+0000","last_fullsized":"2024-05-22T21:18:18.954075+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:05:16.362966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169309+0000","last_change":"2024-05-22T21:18:17.570492+0000","last_active":"2024-05-22T21:18:19.169309+0000","last_peered":"2024-05-22T21:18:19.169309+0000","last_clean":"2024-05-22T21:18:19.169309+0000","last_became_active":"2024-05-22T21:18:17.570091+0000","last_became_peered":"2024-05-22T21:18:17.570091+0000","last_unstale":"2024-05-22T21:18:19.169309+0000","last_undegraded":"2024-05-22T21:18:19.169309+0000","last_fullsized":"2024-05-22T21:18:19.169309+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:30:19.397462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170478+0000","last_change":"2024-05-22T21:18:17.964113+0000","last_active":"2024-05-22T21:18:19.170478+0000","last_peered":"2024-05-22T21:18:19.170478+0000","last_clean":"2024-05-22T21:18:19.170478+0000","last_became_active":"2024-05-22T21:18:17.961742+0000","last_became_peered":"2024-05-22T21:18:17.961742+0000","last_unstale":"2024-05-22T21:18:19.170478+0000","last_undegraded":"2024-05-22T21:18:19.170478+0000","last_fullsized":"2024-05-22T21:18:19.170478+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:46.836261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170677+0000","last_change":"2024-05-22T21:18:17.572676+0000","last_active":"2024-05-22T21:18:19.170677+0000","last_peered":"2024-05-22T21:18:19.170677+0000","last_clean":"2024-05-22T21:18:19.170677+0000","last_became_active":"2024-05-22T21:18:17.572431+0000","last_became_peered":"2024-05-22T21:18:17.572431+0000","last_unstale":"2024-05-22T21:18:19.170677+0000","last_undegraded":"2024-05-22T21:18:19.170677+0000","last_fullsized":"2024-05-22T21:18:19.170677+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:58:32.491726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953576+0000","last_change":"2024-05-22T21:18:17.961274+0000","last_active":"2024-05-22T21:18:18.953576+0000","last_peered":"2024-05-22T21:18:18.953576+0000","last_clean":"2024-05-22T21:18:18.953576+0000","last_became_active":"2024-05-22T21:18:17.961038+0000","last_became_peered":"2024-05-22T21:18:17.961038+0000","last_unstale":"2024-05-22T21:18:18.953576+0000","last_undegraded":"2024-05-22T21:18:18.953576+0000","last_fullsized":"2024-05-22T21:18:18.953576+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:22:06.607488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170614+0000","last_change":"2024-05-22T21:18:17.576592+0000","last_active":"2024-05-22T21:18:19.170614+0000","last_peered":"2024-05-22T21:18:19.170614+0000","last_clean":"2024-05-22T21:18:19.170614+0000","last_became_active":"2024-05-22T21:18:17.576366+0000","last_became_peered":"2024-05-22T21:18:17.576366+0000","last_unstale":"2024-05-22T21:18:19.170614+0000","last_undegraded":"2024-05-22T21:18:19.170614+0000","last_fullsized":"2024-05-22T21:18:19.170614+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:19.223879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393911+0000","last_change":"2024-05-22T21:18:17.956103+0000","last_active":"2024-05-22T21:18:19.393911+0000","last_peered":"2024-05-22T21:18:19.393911+0000","last_clean":"2024-05-22T21:18:19.393911+0000","last_became_active":"2024-05-22T21:18:17.955874+0000","last_became_peered":"2024-05-22T21:18:17.955874+0000","last_unstale":"2024-05-22T21:18:19.393911+0000","last_undegraded":"2024-05-22T21:18:19.393911+0000","last_fullsized":"2024-05-22T21:18:19.393911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:09:27.584040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954657+0000","last_change":"2024-05-22T21:18:17.573273+0000","last_active":"2024-05-22T21:18:18.954657+0000","last_peered":"2024-05-22T21:18:18.954657+0000","last_clean":"2024-05-22T21:18:18.954657+0000","last_became_active":"2024-05-22T21:18:17.572717+0000","last_became_peered":"2024-05-22T21:18:17.572717+0000","last_unstale":"2024-05-22T21:18:18.954657+0000","last_undegraded":"2024-05-22T21:18:18.954657+0000","last_fullsized":"2024-05-22T21:18:18.954657+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:21.293134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954495+0000","last_change":"2024-05-22T21:18:17.961113+0000","last_active":"2024-05-22T21:18:18.954495+0000","last_peered":"2024-05-22T21:18:18.954495+0000","last_clean":"2024-05-22T21:18:18.954495+0000","last_became_active":"2024-05-22T21:18:17.960493+0000","last_became_peered":"2024-05-22T21:18:17.960493+0000","last_unstale":"2024-05-22T21:18:18.954495+0000","last_undegraded":"2024-05-22T21:18:18.954495+0000","last_fullsized":"2024-05-22T21:18:18.954495+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:16.013875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954380+0000","last_change":"2024-05-22T21:18:17.575482+0000","last_active":"2024-05-22T21:18:18.954380+0000","last_peered":"2024-05-22T21:18:18.954380+0000","last_clean":"2024-05-22T21:18:18.954380+0000","last_became_active":"2024-05-22T21:18:17.574884+0000","last_became_peered":"2024-05-22T21:18:17.574884+0000","last_unstale":"2024-05-22T21:18:18.954380+0000","last_undegraded":"2024-05-22T21:18:18.954380+0000","last_fullsized":"2024-05-22T21:18:18.954380+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:06:41.500056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953592+0000","last_change":"2024-05-22T21:18:17.958011+0000","last_active":"2024-05-22T21:18:18.953592+0000","last_peered":"2024-05-22T21:18:18.953592+0000","last_clean":"2024-05-22T21:18:18.953592+0000","last_became_active":"2024-05-22T21:18:17.957625+0000","last_became_peered":"2024-05-22T21:18:17.957625+0000","last_unstale":"2024-05-22T21:18:18.953592+0000","last_undegraded":"2024-05-22T21:18:18.953592+0000","last_fullsized":"2024-05-22T21:18:18.953592+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:31:06.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170872+0000","last_change":"2024-05-22T21:18:17.577139+0000","last_active":"2024-05-22T21:18:19.170872+0000","last_peered":"2024-05-22T21:18:19.170872+0000","last_clean":"2024-05-22T21:18:19.170872+0000","last_became_active":"2024-05-22T21:18:17.573999+0000","last_became_peered":"2024-05-22T21:18:17.573999+0000","last_unstale":"2024-05-22T21:18:19.170872+0000","last_undegraded":"2024-05-22T21:18:19.170872+0000","last_fullsized":"2024-05-22T21:18:19.170872+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:42:07.698843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170780+0000","last_change":"2024-05-22T21:18:17.962821+0000","last_active":"2024-05-22T21:18:19.170780+0000","last_peered":"2024-05-22T21:18:19.170780+0000","last_clean":"2024-05-22T21:18:19.170780+0000","last_became_active":"2024-05-22T21:18:17.958435+0000","last_became_peered":"2024-05-22T21:18:17.958435+0000","last_unstale":"2024-05-22T21:18:19.170780+0000","last_undegraded":"2024-05-22T21:18:19.170780+0000","last_fullsized":"2024-05-22T21:18:19.170780+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:23.958727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953772+0000","last_change":"2024-05-22T21:18:17.572361+0000","last_active":"2024-05-22T21:18:18.953772+0000","last_peered":"2024-05-22T21:18:18.953772+0000","last_clean":"2024-05-22T21:18:18.953772+0000","last_became_active":"2024-05-22T21:18:17.572121+0000","last_became_peered":"2024-05-22T21:18:17.572121+0000","last_unstale":"2024-05-22T21:18:18.953772+0000","last_undegraded":"2024-05-22T21:18:18.953772+0000","last_fullsized":"2024-05-22T21:18:18.953772+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:10:10.103204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169493+0000","last_change":"2024-05-22T21:18:17.959934+0000","last_active":"2024-05-22T21:18:19.169493+0000","last_peered":"2024-05-22T21:18:19.169493+0000","last_clean":"2024-05-22T21:18:19.169493+0000","last_became_active":"2024-05-22T21:18:17.959654+0000","last_became_peered":"2024-05-22T21:18:17.959654+0000","last_unstale":"2024-05-22T21:18:19.169493+0000","last_undegraded":"2024-05-22T21:18:19.169493+0000","last_fullsized":"2024-05-22T21:18:19.169493+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:06:42.502171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"102'1","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954428+0000","last_change":"2024-05-22T21:18:17.575307+0000","last_active":"2024-05-22T21:18:18.954428+0000","last_peered":"2024-05-22T21:18:18.954428+0000","last_clean":"2024-05-22T21:18:18.954428+0000","last_became_active":"2024-05-22T21:18:17.574793+0000","last_became_peered":"2024-05-22T21:18:17.574793+0000","last_unstale":"2024-05-22T21:18:18.954428+0000","last_undegraded":"2024-05-22T21:18:18.954428+0000","last_fullsized":"2024-05-22T21:18:18.954428+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:34:20.054587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":248,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-22T21:18:17.947888+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:18:17.947888+0000","last_peered":"2024-05-22T21:18:17.947888+0000","last_clean":"2024-05-22T21:18:17.947888+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:18:17.947888+0000","last_undegraded":"2024-05-22T21:18:17.947888+0000","last_fullsized":"2024-05-22T21:18:17.947888+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1121,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194190,"statfs":{"total":200000000,"available":198851134,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72599999999999998,"15min":0.76200000000000001},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.98899999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70399999999999996,"15min":0.69199999999999995},"min":{"1min":0.48599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97699999999999998,"5min":1.889,"15min":1.889},"last":0.54900000000000004}]},{"osd":1,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75,"15min":0.77800000000000002},"min":{"1min":0.502,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95799999999999996,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.72999999999999998,"15min":0.76500000000000001},"min":{"1min":0.44900000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92700000000000005,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.51500000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94199999999999995,"5min":1.923,"15min":1.923},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.74399999999999999,"15min":0.72699999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.048,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.74199999999999999}]},{"osd":3,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.57299999999999995,"5min":0.443,"15min":0.443},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77500000000000002,"15min":0.79900000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.014,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.79100000000000004}]},{"osd":4,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.74299999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88900000000000001,"5min":1.018,"15min":1.018},"last":0.88700000000000001}]},{"osd":5,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63900000000000001,"15min":0.65500000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89600000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82999999999999996,"5min":0.996,"15min":0.996},"last":0.91000000000000003}]},{"osd":6,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96999999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66000000000000003,"15min":0.64000000000000001},"min":{"1min":0.51000000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83199999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.97699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194766,"statfs":{"total":200000000,"available":199440878,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.49299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70599999999999996,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95899999999999996,"5min":1.107,"15min":1.107},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.034,"5min":1.081,"15min":1.081},"last":0.56299999999999994}]},{"osd":2,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78300000000000003,"15min":0.75900000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.746,"15min":0.73899999999999999},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.53900000000000003}]},{"osd":3,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.748,"15min":0.74199999999999999},"min":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0249999999999999,"5min":1.415,"15min":1.415},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.004,"5min":1.788,"15min":1.788},"last":0.50800000000000001}]},{"osd":4,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.41699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.89900000000000002,"5min":1.006,"15min":1.006},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95899999999999996,"5min":2.718,"15min":2.718},"last":0.61899999999999999}]},{"osd":5,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.745,"15min":0.70299999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81799999999999995,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.45500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.67100000000000004}]},{"osd":7,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70999999999999996,"15min":0.73799999999999999},"min":{"1min":0.53600000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0469999999999999,"5min":1.095,"15min":1.095},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72699999999999998,"15min":0.75700000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92500000000000004,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.438}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194765,"statfs":{"total":200000000,"available":199439986,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69699999999999995,"15min":0.70699999999999996},"min":{"1min":0.58199999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89300000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.69399999999999995,"15min":0.67400000000000004},"min":{"1min":0.63100000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.93600000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.497,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88600000000000001,"5min":1.216,"15min":1.216},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69599999999999995,"15min":0.68600000000000005},"min":{"1min":0.52900000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.84299999999999997,"5min":1.048,"15min":1.048},"last":0.81100000000000005}]},{"osd":2,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.72499999999999998,"15min":0.70099999999999996},"min":{"1min":0.56999999999999995,"5min":0.436,"15min":0.436},"max":{"1min":0.96499999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.69799999999999995,"15min":0.67600000000000005},"min":{"1min":0.61299999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92000000000000004,"5min":1.252,"15min":1.252},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.88600000000000001,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.746,"15min":0.73399999999999999},"min":{"1min":0.56299999999999994,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95299999999999996,"5min":1.111,"15min":1.111},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60299999999999998,"15min":0.63100000000000001},"min":{"1min":0.42899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92800000000000005,"5min":1.272,"15min":1.272},"last":0.754},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.44400000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.91700000000000004,"5min":1.038,"15min":1.038},"last":0.52100000000000002}]},{"osd":6,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.49099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.498},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65100000000000002,"15min":0.66300000000000003},"min":{"1min":0.41499999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.877,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.83399999999999996}]},{"osd":7,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71499999999999997,"15min":0.71699999999999997},"min":{"1min":0.53000000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77800000000000002,"15min":0.79600000000000004},"min":{"1min":0.57799999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95899999999999996,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.71099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194767,"statfs":{"total":200000000,"available":199441572,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78200000000000003,"15min":0.748},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.89300000000000002,"5min":6.6539999999999999,"15min":6.6539999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.747,"15min":0.72799999999999998},"min":{"1min":0.49299999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93300000000000005,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":0.49299999999999999}]},{"osd":1,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68200000000000005,"15min":0.71599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88200000000000001,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92200000000000004,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.48399999999999999}]},{"osd":2,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70199999999999996,"15min":0.68899999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0940000000000001,"5min":1.335,"15min":1.335},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70099999999999996,"15min":0.70399999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95099999999999996,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.436}]},{"osd":3,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.54100000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.98999999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.78900000000000003},"min":{"1min":0.53200000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96799999999999997,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.68700000000000006}]},{"osd":5,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.39400000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.93500000000000005,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91900000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.91900000000000004}]},{"osd":6,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65900000000000003,"15min":0.69299999999999995},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91600000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.51800000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.91100000000000003,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.52400000000000002}]},{"osd":7,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.69599999999999995,"15min":0.69999999999999996},"min":{"1min":0.48099999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98199999999999998,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.74099999999999999,"15min":0.78000000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97999999999999998,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.73399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194765,"statfs":{"total":200000000,"available":199440008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.42999999999999999,"15min":0.40699999999999997},"min":{"1min":0.29699999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.71399999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.442,"15min":0.40799999999999997},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.33400000000000002}]},{"osd":1,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46000000000000002,"15min":0.44800000000000001},"min":{"1min":0.28399999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.624,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71799999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.46000000000000002}]},{"osd":2,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51000000000000001,"15min":0.496},"min":{"1min":0.376,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68999999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51100000000000001,"15min":0.48999999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69899999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.439}]},{"osd":4,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.74099999999999999,"15min":0.70599999999999996},"min":{"1min":0.50900000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0209999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.749,"15min":0.71899999999999997},"min":{"1min":0.57599999999999996,"5min":0.435,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.72399999999999998}]},{"osd":5,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.75800000000000001,"15min":0.69899999999999995},"min":{"1min":0.622,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.002,"5min":1.234,"15min":1.234},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.70299999999999996},"min":{"1min":0.53500000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0449999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.73199999999999998}]},{"osd":6,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.71599999999999997},"min":{"1min":0.48899999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97499999999999998,"5min":1.276,"15min":1.276},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75800000000000001,"15min":0.70799999999999996},"min":{"1min":0.54200000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0760000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.69099999999999995}]},{"osd":7,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.75800000000000001},"min":{"1min":0.52800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.0900000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78500000000000003,"15min":0.74099999999999999},"min":{"1min":0.63300000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0649999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.85799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194766,"statfs":{"total":200000000,"available":199440990,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.59399999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.46300000000000002,"15min":0.443},"min":{"1min":0.33200000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.36599999999999999}]},{"osd":1,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.32900000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.624,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54300000000000004,"15min":0.52100000000000002},"min":{"1min":0.442,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.442}]},{"osd":3,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83099999999999996,"5min":0.871,"15min":0.871},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.47399999999999998,"15min":0.45900000000000002},"min":{"1min":0.34699999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.68000000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.34699999999999998}]},{"osd":4,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73399999999999999,"15min":0.68999999999999995},"min":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95599999999999996,"5min":1.095,"15min":1.095},"last":0.88},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77900000000000003,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.77400000000000002}]},{"osd":5,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.73299999999999998,"15min":0.70699999999999996},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72599999999999998,"15min":0.70199999999999996},"min":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92900000000000005,"5min":1.127,"15min":1.127},"last":0.73099999999999998}]},{"osd":6,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73099999999999998,"15min":0.70299999999999996},"min":{"1min":0.52000000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.02,"5min":1.054,"15min":1.054},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69899999999999995,"15min":0.67800000000000005},"min":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.89600000000000002,"5min":1.002,"15min":1.002},"last":0.82899999999999996}]},{"osd":7,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.76800000000000002,"15min":0.73199999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.77200000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0249999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.89700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194766,"statfs":{"total":200000000,"available":199441378,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42199999999999999,"15min":0.432},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56999999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.432},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.44900000000000001,"15min":0.433},"min":{"1min":0.28399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.84799999999999998,"5min":0.876,"15min":0.876},"last":0.53600000000000003}]},{"osd":2,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48899999999999999,"15min":0.48999999999999999},"min":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.748,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.51100000000000001,"15min":0.53600000000000003},"min":{"1min":0.35699999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64700000000000002,"5min":1.923,"15min":1.923},"last":0.40999999999999998}]},{"osd":3,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.57099999999999995},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.49099999999999999,"15min":0.48699999999999999},"min":{"1min":0.34599999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72099999999999997,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.50600000000000001}]},{"osd":4,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.74099999999999999,"15min":0.76000000000000001},"min":{"1min":0.52300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.999,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.70699999999999996,"15min":0.68300000000000005},"min":{"1min":0.504,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0389999999999999,"5min":1.3,"15min":1.3},"last":0.89700000000000002}]},{"osd":5,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.53500000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0089999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.751,"15min":0.753},"min":{"1min":0.56499999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1000000000000001,"5min":1.97,"15min":1.97},"last":0.71499999999999997}]},{"osd":6,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.76300000000000001,"15min":0.745},"min":{"1min":0.54600000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0169999999999999,"5min":1.272,"15min":1.272},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.77000000000000002,"15min":0.745},"min":{"1min":0.64700000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.04,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.77400000000000002}]},{"osd":7,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.50600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0129999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.88},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76300000000000001,"15min":0.77900000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.91200000000000003,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.73299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1121,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194191,"statfs":{"total":200000000,"available":198851789,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42499999999999999,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58099999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.255},{"interface":"front","average":{"1min":0.504,"5min":0.51100000000000001,"15min":0.48499999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69199999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.501}]},{"osd":2,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.45400000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76200000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.42799999999999999},"min":{"1min":0.29399999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.623,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.47599999999999998}]},{"osd":3,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45600000000000002,"15min":0.434},"min":{"1min":0.33900000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61699999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53400000000000003,"15min":0.503},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.746,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.58399999999999996}]},{"osd":4,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.71099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1020000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.57099999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.113,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.872}]},{"osd":5,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71899999999999997,"15min":0.70699999999999996},"min":{"1min":0.54200000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99299999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.76900000000000002,"15min":0.73899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.143,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.93799999999999994}]},{"osd":6,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79300000000000004,"15min":0.77300000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1240000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78200000000000003,"15min":0.77200000000000002},"min":{"1min":0.55300000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2090000000000001,"5min":1.5,"15min":1.5},"last":0.85499999999999998}]},{"osd":7,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.75600000000000001,"15min":0.72699999999999998},"min":{"1min":0.57499999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.75},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76900000000000002,"15min":0.75900000000000001},"min":{"1min":0.55900000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.0589999999999999,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.83799999999999997}]}]}],"pool_statfs":[]}} 2024-05-22T21:18:22.779 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-22T21:18:22.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a3e00640 1 Processor -- start 2024-05-22T21:18:22.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a3e00640 1 -- start start 2024-05-22T21:18:22.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a3e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f25a40cc260 0x7f25a4162b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a3e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f25a406dd40 0x7f25a41650b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a3e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a40cf670 con 0x7f25a4069ed0 2024-05-22T21:18:22.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a3e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a40cf120 con 0x7f25a40cc260 2024-05-22T21:18:22.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a3e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a4071b00 con 0x7f25a406dd40 2024-05-22T21:18:22.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a2000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f25a40cc260 0x7f25a4162b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a2a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f25a4069ed0 0x7f25a406a2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:40304/0 (socket says 172.21.15.161:40304) 2024-05-22T21:18:22.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a2a00640 1 -- 172.21.15.161:0/1748178575 learned_addr learned my addr 172.21.15.161:0/1748178575 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:22.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a3400640 1 --2- 172.21.15.161:0/1748178575 >> v2:172.21.15.179:3300/0 conn(0x7f25a406dd40 0x7f25a41650b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a2000640 1 -- 172.21.15.161:0/1748178575 >> v2:172.21.15.179:3300/0 conn(0x7f25a406dd40 msgr2=0x7f25a41650b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:22.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a2000640 1 --2- 172.21.15.161:0/1748178575 >> v2:172.21.15.179:3300/0 conn(0x7f25a406dd40 0x7f25a41650b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a2000640 1 -- 172.21.15.161:0/1748178575 >> v1:172.21.15.161:6789/0 conn(0x7f25a4069ed0 legacy=0x7f25a406a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a2000640 1 -- 172.21.15.161:0/1748178575 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a415c070 con 0x7f25a40cc260 2024-05-22T21:18:22.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a1600640 1 -- 172.21.15.161:0/1748178575 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576192665 0 0) 0x7f25a40cf670 con 0x7f25a4069ed0 2024-05-22T21:18:22.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.947+0000 7f25a3400640 1 --2- 172.21.15.161:0/1748178575 >> v2:172.21.15.179:3300/0 conn(0x7f25a406dd40 0x7f25a41650b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:22.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.951+0000 7f25a2000640 1 --2- 172.21.15.161:0/1748178575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25a40cc260 0x7f25a4162b60 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f25a4071f70 tx=0x7f259001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d439d6fcf4246e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:22.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.951+0000 7f25a1600640 1 -- 172.21.15.161:0/1748178575 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25900ab650 con 0x7f25a40cc260 2024-05-22T21:18:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.951+0000 7f25a1600640 1 -- 172.21.15.161:0/1748178575 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25900ab7b0 con 0x7f25a40cc260 2024-05-22T21:18:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.951+0000 7f25a1600640 1 -- 172.21.15.161:0/1748178575 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25900abb40 con 0x7f25a40cc260 2024-05-22T21:18:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.951+0000 7f25a3e00640 1 -- 172.21.15.161:0/1748178575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25a40cc260 msgr2=0x7f25a4162b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.951+0000 7f25a3e00640 1 --2- 172.21.15.161:0/1748178575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25a40cc260 0x7f25a4162b60 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f25a4071f70 tx=0x7f259001c450 comp rx=0 tx=0).stop 2024-05-22T21:18:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.951+0000 7f25a3e00640 1 -- 172.21.15.161:0/1748178575 shutdown_connections 2024-05-22T21:18:22.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.951+0000 7f25a3e00640 1 --2- 172.21.15.161:0/1748178575 >> v2:172.21.15.179:3300/0 conn(0x7f25a406dd40 0x7f25a41650b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.951+0000 7f25a3e00640 1 --2- 172.21.15.161:0/1748178575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25a40cc260 0x7f25a4162b60 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.951+0000 7f25a3e00640 1 -- 172.21.15.161:0/1748178575 >> 172.21.15.161:0/1748178575 conn(0x7f25a40cfc20 msgr2=0x7f25a40d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:22.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.951+0000 7f25a3e00640 1 -- 172.21.15.161:0/1748178575 shutdown_connections 2024-05-22T21:18:22.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.951+0000 7f25a3e00640 1 -- 172.21.15.161:0/1748178575 wait complete. 2024-05-22T21:18:22.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a3e00640 1 Processor -- start 2024-05-22T21:18:22.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a3e00640 1 -- start start 2024-05-22T21:18:22.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a3e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25a4069ed0 0x7f25a4154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25a406dd40 0x7f25a4128e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a3e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a40cc260 0x7f25a4143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a3e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a4154f60 con 0x7f25a406dd40 2024-05-22T21:18:22.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a3e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a412a2d0 con 0x7f25a40cc260 2024-05-22T21:18:22.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a3e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41296c0 con 0x7f25a4069ed0 2024-05-22T21:18:22.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25a406dd40 0x7f25a4128e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25a406dd40 0x7f25a4128e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38628/0 (socket says 172.21.15.161:38628) 2024-05-22T21:18:22.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a2000640 1 -- 172.21.15.161:0/1324742211 learned_addr learned my addr 172.21.15.161:0/1324742211 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:22.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a3400640 1 --2- 172.21.15.161:0/1324742211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a40cc260 0x7f25a4143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a2a00640 1 --2- 172.21.15.161:0/1324742211 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25a4069ed0 0x7f25a4154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a2000640 1 -- 172.21.15.161:0/1324742211 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25a4069ed0 msgr2=0x7f25a4154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a2000640 1 --2- 172.21.15.161:0/1324742211 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25a4069ed0 0x7f25a4154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a2000640 1 -- 172.21.15.161:0/1324742211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a40cc260 msgr2=0x7f25a4143480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:22.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a2000640 1 --2- 172.21.15.161:0/1324742211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a40cc260 0x7f25a4143480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:22.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a2000640 1 -- 172.21.15.161:0/1324742211 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2590009420 con 0x7f25a406dd40 2024-05-22T21:18:22.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a2000640 1 --2- 172.21.15.161:0/1324742211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25a406dd40 0x7f25a4128e00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f25900ad040 tx=0x7f259001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:22.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f259aa00640 1 -- 172.21.15.161:0/1324742211 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25900b5810 con 0x7f25a406dd40 2024-05-22T21:18:22.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f259aa00640 1 -- 172.21.15.161:0/1324742211 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25900b5970 con 0x7f25a406dd40 2024-05-22T21:18:22.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f259aa00640 1 -- 172.21.15.161:0/1324742211 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25900b5d20 con 0x7f25a406dd40 2024-05-22T21:18:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a3e00640 1 -- 172.21.15.161:0/1324742211 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a4144e30 con 0x7f25a406dd40 2024-05-22T21:18:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a3e00640 1 -- 172.21.15.161:0/1324742211 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a4129c80 con 0x7f25a406dd40 2024-05-22T21:18:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.955+0000 7f25a3e00640 1 -- 172.21.15.161:0/1324742211 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a4067070 con 0x7f25a406dd40 2024-05-22T21:18:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.959+0000 7f259aa00640 1 -- 172.21.15.161:0/1324742211 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f25900bf410 con 0x7f25a406dd40 2024-05-22T21:18:22.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.959+0000 7f259aa00640 1 --2- 172.21.15.161:0/1324742211 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2584066690 0x7f2584068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:22.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.959+0000 7f259aa00640 1 -- 172.21.15.161:0/1324742211 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f259011eeb0 con 0x7f25a406dd40 2024-05-22T21:18:22.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.959+0000 7f25a2a00640 1 --2- 172.21.15.161:0/1324742211 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2584066690 0x7f2584068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:22.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.959+0000 7f25a2a00640 1 --2- 172.21.15.161:0/1324742211 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2584066690 0x7f2584068b50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2594000f70 tx=0x7f2594007b60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:22.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:22.963+0000 7f259aa00640 1 -- 172.21.15.161:0/1324742211 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f25900ec210 con 0x7f25a406dd40 2024-05-22T21:18:23.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.171+0000 7f25a3e00640 1 -- 172.21.15.161:0/1324742211 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f25a406a7c0 con 0x7f25a406dd40 2024-05-22T21:18:23.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.171+0000 7f259aa00640 1 -- 172.21.15.161:0/1324742211 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1168 (secure 0 0 0) 0x7f25900ef830 con 0x7f25a406dd40 2024-05-22T21:18:23.175 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:23.175 INFO:teuthology.orchestra.run.smithi161.stdout:{"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":317,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":103,"num_osds":8,"num_up_osds":8,"osd_up_since":1716412390,"num_in_osds":8,"osd_in_since":1716412387,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592730,"bytes_used":5652265,"bytes_avail":1594347735,"bytes_total":1600000000,"write_bytes_sec":2427,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":47,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":6699}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-05-22T21:18:12.457937+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-22T21:18:23.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.175+0000 7f25a3e00640 1 -- 172.21.15.161:0/1324742211 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2584066690 msgr2=0x7f2584068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:23.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.175+0000 7f25a3e00640 1 --2- 172.21.15.161:0/1324742211 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2584066690 0x7f2584068b50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2594000f70 tx=0x7f2594007b60 comp rx=0 tx=0).stop 2024-05-22T21:18:23.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.175+0000 7f25a3e00640 1 -- 172.21.15.161:0/1324742211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25a406dd40 msgr2=0x7f25a4128e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:23.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.175+0000 7f25a3e00640 1 --2- 172.21.15.161:0/1324742211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25a406dd40 0x7f25a4128e00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f25900ad040 tx=0x7f259001ac90 comp rx=0 tx=0).stop 2024-05-22T21:18:23.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.175+0000 7f25a3e00640 1 -- 172.21.15.161:0/1324742211 shutdown_connections 2024-05-22T21:18:23.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.175+0000 7f25a3e00640 1 --2- 172.21.15.161:0/1324742211 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2584066690 0x7f2584068b50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:23.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.175+0000 7f25a3e00640 1 --2- 172.21.15.161:0/1324742211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25a40cc260 0x7f25a4143480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:23.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.175+0000 7f25a3e00640 1 --2- 172.21.15.161:0/1324742211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25a406dd40 0x7f25a4128e00 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:23.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.175+0000 7f25a3e00640 1 --2- 172.21.15.161:0/1324742211 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f25a4069ed0 0x7f25a4154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:23.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.175+0000 7f25a3e00640 1 -- 172.21.15.161:0/1324742211 >> 172.21.15.161:0/1324742211 conn(0x7f25a40cfc20 msgr2=0x7f25a40cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:23.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.175+0000 7f25a3e00640 1 -- 172.21.15.161:0/1324742211 shutdown_connections 2024-05-22T21:18:23.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.175+0000 7f25a3e00640 1 -- 172.21.15.161:0/1324742211 wait complete. 2024-05-22T21:18:23.197 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-22T21:18:23.197 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:18:23.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.327+0000 7fec2fe00640 1 Processor -- start 2024-05-22T21:18:23.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.331+0000 7fec2fe00640 1 -- start start 2024-05-22T21:18:23.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.331+0000 7fec2fe00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fec3015e430 0x7fec300b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:23.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.331+0000 7fec2fe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec300d1400 0x7fec300cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:23.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.331+0000 7fec2fe00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec300b13d0 con 0x7fec3015d040 2024-05-22T21:18:23.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.331+0000 7fec2fe00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec300b0aa0 con 0x7fec3015e430 2024-05-22T21:18:23.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.331+0000 7fec2fe00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec300b0d80 con 0x7fec300d1400 2024-05-22T21:18:23.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.331+0000 7fec2e000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fec3015e430 0x7fec300b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:23.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.331+0000 7fec2f400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec300d1400 0x7fec300cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:23.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.331+0000 7fec2ea00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fec3015d040 0x7fec300d0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:56678/0 (socket says 172.21.15.161:56678) 2024-05-22T21:18:23.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.331+0000 7fec2ea00640 1 -- 172.21.15.161:0/974555998 learned_addr learned my addr 172.21.15.161:0/974555998 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:23.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.331+0000 7fec2e000640 1 -- 172.21.15.161:0/974555998 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec300d1400 msgr2=0x7fec300cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:23.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.331+0000 7fec2e000640 1 --2- 172.21.15.161:0/974555998 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec300d1400 0x7fec300cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:23.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.331+0000 7fec2e000640 1 -- 172.21.15.161:0/974555998 >> v1:172.21.15.179:6789/0 conn(0x7fec3015d040 legacy=0x7fec300d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:23.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.331+0000 7fec2e000640 1 -- 172.21.15.161:0/974555998 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec300b1060 con 0x7fec3015e430 2024-05-22T21:18:23.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.331+0000 7fec2d600640 1 -- 172.21.15.161:0/974555998 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4194848426 0 0) 0x7fec300b13d0 con 0x7fec3015d040 2024-05-22T21:18:23.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.331+0000 7fec2f400640 1 --2- 172.21.15.161:0/974555998 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec300d1400 0x7fec300cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:23.337 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.335+0000 7fec2e000640 1 --2- 172.21.15.161:0/974555998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3015e430 0x7fec300b1860 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0097c0 tx=0x7fec1c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6a784478520a24f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:23.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.335+0000 7fec2d600640 1 -- 172.21.15.161:0/974555998 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c0ac620 con 0x7fec3015e430 2024-05-22T21:18:23.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.335+0000 7fec2d600640 1 -- 172.21.15.161:0/974555998 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec300ce3a0 con 0x7fec3015e430 2024-05-22T21:18:23.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.335+0000 7fec2d600640 1 -- 172.21.15.161:0/974555998 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c0ac9a0 con 0x7fec3015e430 2024-05-22T21:18:23.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.335+0000 7fec2fe00640 1 -- 172.21.15.161:0/974555998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3015e430 msgr2=0x7fec300b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:23.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.335+0000 7fec2fe00640 1 --2- 172.21.15.161:0/974555998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3015e430 0x7fec300b1860 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0097c0 tx=0x7fec1c01c4a0 comp rx=0 tx=0).stop 2024-05-22T21:18:23.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.335+0000 7fec2fe00640 1 -- 172.21.15.161:0/974555998 shutdown_connections 2024-05-22T21:18:23.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.335+0000 7fec2fe00640 1 --2- 172.21.15.161:0/974555998 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec300d1400 0x7fec300cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:23.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.335+0000 7fec2fe00640 1 --2- 172.21.15.161:0/974555998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3015e430 0x7fec300b1860 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:23.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.335+0000 7fec2fe00640 1 -- 172.21.15.161:0/974555998 >> 172.21.15.161:0/974555998 conn(0x7fec30099a40 msgr2=0x7fec30099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:23.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.335+0000 7fec2fe00640 1 -- 172.21.15.161:0/974555998 shutdown_connections 2024-05-22T21:18:23.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.335+0000 7fec2fe00640 1 -- 172.21.15.161:0/974555998 wait complete. 2024-05-22T21:18:23.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.335+0000 7fec2fe00640 1 Processor -- start 2024-05-22T21:18:23.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2fe00640 1 -- start start 2024-05-22T21:18:23.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec300d1400 0x7fec30155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:23.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2fe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3015d040 0x7fec300d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:23.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2fe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3015e430 0x7fec300d0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:23.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2fe00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec30155ac0 con 0x7fec300d1400 2024-05-22T21:18:23.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2fe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec30155da0 con 0x7fec3015e430 2024-05-22T21:18:23.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2fe00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec30156080 con 0x7fec3015d040 2024-05-22T21:18:23.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec300d1400 0x7fec30155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:23.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec300d1400 0x7fec30155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38632/0 (socket says 172.21.15.161:38632) 2024-05-22T21:18:23.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2ea00640 1 -- 172.21.15.161:0/713523408 learned_addr learned my addr 172.21.15.161:0/713523408 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2e000640 1 --2- 172.21.15.161:0/713523408 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3015d040 0x7fec300d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2f400640 1 --2- 172.21.15.161:0/713523408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3015e430 0x7fec300d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2ea00640 1 -- 172.21.15.161:0/713523408 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3015d040 msgr2=0x7fec300d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2ea00640 1 --2- 172.21.15.161:0/713523408 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3015d040 0x7fec300d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2ea00640 1 -- 172.21.15.161:0/713523408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3015e430 msgr2=0x7fec300d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2ea00640 1 --2- 172.21.15.161:0/713523408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3015e430 0x7fec300d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2ea00640 1 -- 172.21.15.161:0/713523408 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec28009a70 con 0x7fec300d1400 2024-05-22T21:18:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2e000640 1 --2- 172.21.15.161:0/713523408 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3015d040 0x7fec300d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2f400640 1 --2- 172.21.15.161:0/713523408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3015e430 0x7fec300d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2ea00640 1 --2- 172.21.15.161:0/713523408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec300d1400 0x7fec30155150 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fec200011f0 tx=0x7fec2000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec26a00640 1 -- 172.21.15.161:0/713523408 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec200074e0 con 0x7fec300d1400 2024-05-22T21:18:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec26a00640 1 -- 172.21.15.161:0/713523408 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec20007640 con 0x7fec300d1400 2024-05-22T21:18:23.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec26a00640 1 -- 172.21.15.161:0/713523408 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20004a10 con 0x7fec300d1400 2024-05-22T21:18:23.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2fe00640 1 -- 172.21.15.161:0/713523408 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c009420 con 0x7fec300d1400 2024-05-22T21:18:23.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2fe00640 1 -- 172.21.15.161:0/713523408 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3006bd00 con 0x7fec300d1400 2024-05-22T21:18:23.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.339+0000 7fec2fe00640 1 -- 172.21.15.161:0/713523408 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec30067070 con 0x7fec300d1400 2024-05-22T21:18:23.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.343+0000 7fec26a00640 1 -- 172.21.15.161:0/713523408 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fec20004b70 con 0x7fec300d1400 2024-05-22T21:18:23.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.343+0000 7fec26a00640 1 --2- 172.21.15.161:0/713523408 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fec10066690 0x7fec10068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:23.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.343+0000 7fec26a00640 1 -- 172.21.15.161:0/713523408 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec20083480 con 0x7fec300d1400 2024-05-22T21:18:23.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.343+0000 7fec2e000640 1 --2- 172.21.15.161:0/713523408 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fec10066690 0x7fec10068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:23.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.343+0000 7fec2e000640 1 --2- 172.21.15.161:0/713523408 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fec10066690 0x7fec10068b50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0ae000 tx=0x7fec1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:23.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.347+0000 7fec26a00640 1 -- 172.21.15.161:0/713523408 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fec200507e0 con 0x7fec300d1400 2024-05-22T21:18:23.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.499+0000 7fec2fe00640 1 -- 172.21.15.161:0/713523408 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec300b6920 con 0x7fec10066690 2024-05-22T21:18:23.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.507+0000 7fec26a00640 1 -- 172.21.15.161:0/713523408 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284772 (secure 0 0 0) 0x7fec300b6920 con 0x7fec10066690 2024-05-22T21:18:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:23.514 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:18:23.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.511+0000 7fec2fe00640 1 -- 172.21.15.161:0/713523408 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fec10066690 msgr2=0x7fec10068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:23.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.511+0000 7fec2fe00640 1 --2- 172.21.15.161:0/713523408 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fec10066690 0x7fec10068b50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0ae000 tx=0x7fec1c008040 comp rx=0 tx=0).stop 2024-05-22T21:18:23.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.511+0000 7fec2fe00640 1 -- 172.21.15.161:0/713523408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec300d1400 msgr2=0x7fec30155150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:23.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.511+0000 7fec2fe00640 1 --2- 172.21.15.161:0/713523408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec300d1400 0x7fec30155150 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fec200011f0 tx=0x7fec2000c9b0 comp rx=0 tx=0).stop 2024-05-22T21:18:23.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.511+0000 7fec2fe00640 1 -- 172.21.15.161:0/713523408 shutdown_connections 2024-05-22T21:18:23.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.511+0000 7fec2fe00640 1 --2- 172.21.15.161:0/713523408 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fec10066690 0x7fec10068b50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:23.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.511+0000 7fec2fe00640 1 --2- 172.21.15.161:0/713523408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3015e430 0x7fec300d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:23.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.511+0000 7fec2fe00640 1 --2- 172.21.15.161:0/713523408 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3015d040 0x7fec300d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:23.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.511+0000 7fec2fe00640 1 --2- 172.21.15.161:0/713523408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec300d1400 0x7fec30155150 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:23.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.511+0000 7fec2fe00640 1 -- 172.21.15.161:0/713523408 >> 172.21.15.161:0/713523408 conn(0x7fec30099a40 msgr2=0x7fec300b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:23.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.511+0000 7fec2fe00640 1 -- 172.21.15.161:0/713523408 shutdown_connections 2024-05-22T21:18:23.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:23.511+0000 7fec2fe00640 1 -- 172.21.15.161:0/713523408 wait complete. 2024-05-22T21:18:23.532 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-05-22T21:18:22.459831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5519,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556980,"statfs":{"total":1600000000,"available":1594347735,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.906544"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170375+0000","last_change":"2024-05-22T21:18:17.960599+0000","last_active":"2024-05-22T21:18:19.170375+0000","last_peered":"2024-05-22T21:18:19.170375+0000","last_clean":"2024-05-22T21:18:19.170375+0000","last_became_active":"2024-05-22T21:18:17.960246+0000","last_became_peered":"2024-05-22T21:18:17.960246+0000","last_unstale":"2024-05-22T21:18:19.170375+0000","last_undegraded":"2024-05-22T21:18:19.170375+0000","last_fullsized":"2024-05-22T21:18:19.170375+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:54:13.859086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169155+0000","last_change":"2024-05-22T21:18:17.963308+0000","last_active":"2024-05-22T21:18:19.169155+0000","last_peered":"2024-05-22T21:18:19.169155+0000","last_clean":"2024-05-22T21:18:19.169155+0000","last_became_active":"2024-05-22T21:18:17.962108+0000","last_became_peered":"2024-05-22T21:18:17.962108+0000","last_unstale":"2024-05-22T21:18:19.169155+0000","last_undegraded":"2024-05-22T21:18:19.169155+0000","last_fullsized":"2024-05-22T21:18:19.169155+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:25:51.935444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953613+0000","last_change":"2024-05-22T21:18:17.960645+0000","last_active":"2024-05-22T21:18:18.953613+0000","last_peered":"2024-05-22T21:18:18.953613+0000","last_clean":"2024-05-22T21:18:18.953613+0000","last_became_active":"2024-05-22T21:18:17.960427+0000","last_became_peered":"2024-05-22T21:18:17.960427+0000","last_unstale":"2024-05-22T21:18:18.953613+0000","last_undegraded":"2024-05-22T21:18:18.953613+0000","last_fullsized":"2024-05-22T21:18:18.953613+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:52:41.544599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.392835+0000","last_change":"2024-05-22T21:18:17.959284+0000","last_active":"2024-05-22T21:18:19.392835+0000","last_peered":"2024-05-22T21:18:19.392835+0000","last_clean":"2024-05-22T21:18:19.392835+0000","last_became_active":"2024-05-22T21:18:17.957127+0000","last_became_peered":"2024-05-22T21:18:17.957127+0000","last_unstale":"2024-05-22T21:18:19.392835+0000","last_undegraded":"2024-05-22T21:18:19.392835+0000","last_fullsized":"2024-05-22T21:18:19.392835+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:35:51.060507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954426+0000","last_change":"2024-05-22T21:18:17.965991+0000","last_active":"2024-05-22T21:18:18.954426+0000","last_peered":"2024-05-22T21:18:18.954426+0000","last_clean":"2024-05-22T21:18:18.954426+0000","last_became_active":"2024-05-22T21:18:17.965764+0000","last_became_peered":"2024-05-22T21:18:17.965764+0000","last_unstale":"2024-05-22T21:18:18.954426+0000","last_undegraded":"2024-05-22T21:18:18.954426+0000","last_fullsized":"2024-05-22T21:18:18.954426+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:31:06.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393978+0000","last_change":"2024-05-22T21:18:17.953557+0000","last_active":"2024-05-22T21:18:19.393978+0000","last_peered":"2024-05-22T21:18:19.393978+0000","last_clean":"2024-05-22T21:18:19.393978+0000","last_became_active":"2024-05-22T21:18:17.952902+0000","last_became_peered":"2024-05-22T21:18:17.952902+0000","last_unstale":"2024-05-22T21:18:19.393978+0000","last_undegraded":"2024-05-22T21:18:19.393978+0000","last_fullsized":"2024-05-22T21:18:19.393978+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:49.458817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393470+0000","last_change":"2024-05-22T21:18:17.953697+0000","last_active":"2024-05-22T21:18:19.393470+0000","last_peered":"2024-05-22T21:18:19.393470+0000","last_clean":"2024-05-22T21:18:19.393470+0000","last_became_active":"2024-05-22T21:18:17.953466+0000","last_became_peered":"2024-05-22T21:18:17.953466+0000","last_unstale":"2024-05-22T21:18:19.393470+0000","last_undegraded":"2024-05-22T21:18:19.393470+0000","last_fullsized":"2024-05-22T21:18:19.393470+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:37:13.521838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393857+0000","last_change":"2024-05-22T21:18:17.957649+0000","last_active":"2024-05-22T21:18:19.393857+0000","last_peered":"2024-05-22T21:18:19.393857+0000","last_clean":"2024-05-22T21:18:19.393857+0000","last_became_active":"2024-05-22T21:18:17.957409+0000","last_became_peered":"2024-05-22T21:18:17.957409+0000","last_unstale":"2024-05-22T21:18:19.393857+0000","last_undegraded":"2024-05-22T21:18:19.393857+0000","last_fullsized":"2024-05-22T21:18:19.393857+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:22:06.607488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393996+0000","last_change":"2024-05-22T21:18:17.952455+0000","last_active":"2024-05-22T21:18:19.393996+0000","last_peered":"2024-05-22T21:18:19.393996+0000","last_clean":"2024-05-22T21:18:19.393996+0000","last_became_active":"2024-05-22T21:18:17.952241+0000","last_became_peered":"2024-05-22T21:18:17.952241+0000","last_unstale":"2024-05-22T21:18:19.393996+0000","last_undegraded":"2024-05-22T21:18:19.393996+0000","last_fullsized":"2024-05-22T21:18:19.393996+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:09:17.217291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170339+0000","last_change":"2024-05-22T21:18:17.962139+0000","last_active":"2024-05-22T21:18:19.170339+0000","last_peered":"2024-05-22T21:18:19.170339+0000","last_clean":"2024-05-22T21:18:19.170339+0000","last_became_active":"2024-05-22T21:18:17.961250+0000","last_became_peered":"2024-05-22T21:18:17.961250+0000","last_unstale":"2024-05-22T21:18:19.170339+0000","last_undegraded":"2024-05-22T21:18:19.170339+0000","last_fullsized":"2024-05-22T21:18:19.170339+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:56:49.462456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169245+0000","last_change":"2024-05-22T21:18:17.957003+0000","last_active":"2024-05-22T21:18:19.169245+0000","last_peered":"2024-05-22T21:18:19.169245+0000","last_clean":"2024-05-22T21:18:19.169245+0000","last_became_active":"2024-05-22T21:18:17.956399+0000","last_became_peered":"2024-05-22T21:18:17.956399+0000","last_unstale":"2024-05-22T21:18:19.169245+0000","last_undegraded":"2024-05-22T21:18:19.169245+0000","last_fullsized":"2024-05-22T21:18:19.169245+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:20.225993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170530+0000","last_change":"2024-05-22T21:18:17.958838+0000","last_active":"2024-05-22T21:18:19.170530+0000","last_peered":"2024-05-22T21:18:19.170530+0000","last_clean":"2024-05-22T21:18:19.170530+0000","last_became_active":"2024-05-22T21:18:17.956822+0000","last_became_peered":"2024-05-22T21:18:17.956822+0000","last_unstale":"2024-05-22T21:18:19.170530+0000","last_undegraded":"2024-05-22T21:18:19.170530+0000","last_fullsized":"2024-05-22T21:18:19.170530+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:16.013875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953716+0000","last_change":"2024-05-22T21:18:17.959719+0000","last_active":"2024-05-22T21:18:18.953716+0000","last_peered":"2024-05-22T21:18:18.953716+0000","last_clean":"2024-05-22T21:18:18.953716+0000","last_became_active":"2024-05-22T21:18:17.959502+0000","last_became_peered":"2024-05-22T21:18:17.959502+0000","last_unstale":"2024-05-22T21:18:18.953716+0000","last_undegraded":"2024-05-22T21:18:18.953716+0000","last_fullsized":"2024-05-22T21:18:18.953716+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:41:21.219804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.392913+0000","last_change":"2024-05-22T21:18:17.959416+0000","last_active":"2024-05-22T21:18:19.392913+0000","last_peered":"2024-05-22T21:18:19.392913+0000","last_clean":"2024-05-22T21:18:19.392913+0000","last_became_active":"2024-05-22T21:18:17.957403+0000","last_became_peered":"2024-05-22T21:18:17.957403+0000","last_unstale":"2024-05-22T21:18:19.392913+0000","last_undegraded":"2024-05-22T21:18:19.392913+0000","last_fullsized":"2024-05-22T21:18:19.392913+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:59:08.119575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393041+0000","last_change":"2024-05-22T21:18:17.954286+0000","last_active":"2024-05-22T21:18:19.393041+0000","last_peered":"2024-05-22T21:18:19.393041+0000","last_clean":"2024-05-22T21:18:19.393041+0000","last_became_active":"2024-05-22T21:18:17.953720+0000","last_became_peered":"2024-05-22T21:18:17.953720+0000","last_unstale":"2024-05-22T21:18:19.393041+0000","last_undegraded":"2024-05-22T21:18:19.393041+0000","last_fullsized":"2024-05-22T21:18:19.393041+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:41:06.493942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953699+0000","last_change":"2024-05-22T21:18:17.964022+0000","last_active":"2024-05-22T21:18:18.953699+0000","last_peered":"2024-05-22T21:18:18.953699+0000","last_clean":"2024-05-22T21:18:18.953699+0000","last_became_active":"2024-05-22T21:18:17.963173+0000","last_became_peered":"2024-05-22T21:18:17.963173+0000","last_unstale":"2024-05-22T21:18:18.953699+0000","last_undegraded":"2024-05-22T21:18:18.953699+0000","last_fullsized":"2024-05-22T21:18:18.953699+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:46.836261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953976+0000","last_change":"2024-05-22T21:18:17.965200+0000","last_active":"2024-05-22T21:18:18.953976+0000","last_peered":"2024-05-22T21:18:18.953976+0000","last_clean":"2024-05-22T21:18:18.953976+0000","last_became_active":"2024-05-22T21:18:17.964346+0000","last_became_peered":"2024-05-22T21:18:17.964346+0000","last_unstale":"2024-05-22T21:18:18.953976+0000","last_undegraded":"2024-05-22T21:18:18.953976+0000","last_fullsized":"2024-05-22T21:18:18.953976+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:09:17.217291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393281+0000","last_change":"2024-05-22T21:18:17.963044+0000","last_active":"2024-05-22T21:18:19.393281+0000","last_peered":"2024-05-22T21:18:19.393281+0000","last_clean":"2024-05-22T21:18:19.393281+0000","last_became_active":"2024-05-22T21:18:17.961677+0000","last_became_peered":"2024-05-22T21:18:17.961677+0000","last_unstale":"2024-05-22T21:18:19.393281+0000","last_undegraded":"2024-05-22T21:18:19.393281+0000","last_fullsized":"2024-05-22T21:18:19.393281+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:28:09.553272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954454+0000","last_change":"2024-05-22T21:18:17.956830+0000","last_active":"2024-05-22T21:18:18.954454+0000","last_peered":"2024-05-22T21:18:18.954454+0000","last_clean":"2024-05-22T21:18:18.954454+0000","last_became_active":"2024-05-22T21:18:17.956077+0000","last_became_peered":"2024-05-22T21:18:17.956077+0000","last_unstale":"2024-05-22T21:18:18.954454+0000","last_undegraded":"2024-05-22T21:18:18.954454+0000","last_fullsized":"2024-05-22T21:18:18.954454+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:31:06.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393570+0000","last_change":"2024-05-22T21:18:17.962103+0000","last_active":"2024-05-22T21:18:19.393570+0000","last_peered":"2024-05-22T21:18:19.393570+0000","last_clean":"2024-05-22T21:18:19.393570+0000","last_became_active":"2024-05-22T21:18:17.960714+0000","last_became_peered":"2024-05-22T21:18:17.960714+0000","last_unstale":"2024-05-22T21:18:19.393570+0000","last_undegraded":"2024-05-22T21:18:19.393570+0000","last_fullsized":"2024-05-22T21:18:19.393570+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:08:44.201170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393427+0000","last_change":"2024-05-22T21:18:17.958661+0000","last_active":"2024-05-22T21:18:19.393427+0000","last_peered":"2024-05-22T21:18:19.393427+0000","last_clean":"2024-05-22T21:18:19.393427+0000","last_became_active":"2024-05-22T21:18:17.958456+0000","last_became_peered":"2024-05-22T21:18:17.958456+0000","last_unstale":"2024-05-22T21:18:19.393427+0000","last_undegraded":"2024-05-22T21:18:19.393427+0000","last_fullsized":"2024-05-22T21:18:19.393427+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:32:39.995038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169266+0000","last_change":"2024-05-22T21:18:17.963540+0000","last_active":"2024-05-22T21:18:19.169266+0000","last_peered":"2024-05-22T21:18:19.169266+0000","last_clean":"2024-05-22T21:18:19.169266+0000","last_became_active":"2024-05-22T21:18:17.962903+0000","last_became_peered":"2024-05-22T21:18:17.962903+0000","last_unstale":"2024-05-22T21:18:19.169266+0000","last_undegraded":"2024-05-22T21:18:19.169266+0000","last_fullsized":"2024-05-22T21:18:19.169266+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:10:11.105319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170376+0000","last_change":"2024-05-22T21:18:17.966436+0000","last_active":"2024-05-22T21:18:19.170376+0000","last_peered":"2024-05-22T21:18:19.170376+0000","last_clean":"2024-05-22T21:18:19.170376+0000","last_became_active":"2024-05-22T21:18:17.964774+0000","last_became_peered":"2024-05-22T21:18:17.964774+0000","last_unstale":"2024-05-22T21:18:19.170376+0000","last_undegraded":"2024-05-22T21:18:19.170376+0000","last_fullsized":"2024-05-22T21:18:19.170376+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:22.052183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.392989+0000","last_change":"2024-05-22T21:18:17.959671+0000","last_active":"2024-05-22T21:18:19.392989+0000","last_peered":"2024-05-22T21:18:19.392989+0000","last_clean":"2024-05-22T21:18:19.392989+0000","last_became_active":"2024-05-22T21:18:17.957995+0000","last_became_peered":"2024-05-22T21:18:17.957995+0000","last_unstale":"2024-05-22T21:18:19.392989+0000","last_undegraded":"2024-05-22T21:18:19.392989+0000","last_fullsized":"2024-05-22T21:18:19.392989+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:57:39.101800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169181+0000","last_change":"2024-05-22T21:18:17.957318+0000","last_active":"2024-05-22T21:18:19.169181+0000","last_peered":"2024-05-22T21:18:19.169181+0000","last_clean":"2024-05-22T21:18:19.169181+0000","last_became_active":"2024-05-22T21:18:17.956778+0000","last_became_peered":"2024-05-22T21:18:17.956778+0000","last_unstale":"2024-05-22T21:18:19.169181+0000","last_undegraded":"2024-05-22T21:18:19.169181+0000","last_fullsized":"2024-05-22T21:18:19.169181+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:48:16.720449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954340+0000","last_change":"2024-05-22T21:18:17.955832+0000","last_active":"2024-05-22T21:18:18.954340+0000","last_peered":"2024-05-22T21:18:18.954340+0000","last_clean":"2024-05-22T21:18:18.954340+0000","last_became_active":"2024-05-22T21:18:17.955610+0000","last_became_peered":"2024-05-22T21:18:17.955610+0000","last_unstale":"2024-05-22T21:18:18.954340+0000","last_undegraded":"2024-05-22T21:18:18.954340+0000","last_fullsized":"2024-05-22T21:18:18.954340+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:22.052183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954287+0000","last_change":"2024-05-22T21:18:17.966183+0000","last_active":"2024-05-22T21:18:18.954287+0000","last_peered":"2024-05-22T21:18:18.954287+0000","last_clean":"2024-05-22T21:18:18.954287+0000","last_became_active":"2024-05-22T21:18:17.964912+0000","last_became_peered":"2024-05-22T21:18:17.964912+0000","last_unstale":"2024-05-22T21:18:18.954287+0000","last_undegraded":"2024-05-22T21:18:18.954287+0000","last_fullsized":"2024-05-22T21:18:18.954287+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:49.458817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170513+0000","last_change":"2024-05-22T21:18:17.965157+0000","last_active":"2024-05-22T21:18:19.170513+0000","last_peered":"2024-05-22T21:18:19.170513+0000","last_clean":"2024-05-22T21:18:19.170513+0000","last_became_active":"2024-05-22T21:18:17.960663+0000","last_became_peered":"2024-05-22T21:18:17.960663+0000","last_unstale":"2024-05-22T21:18:19.170513+0000","last_undegraded":"2024-05-22T21:18:19.170513+0000","last_fullsized":"2024-05-22T21:18:19.170513+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:49.458817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169172+0000","last_change":"2024-05-22T21:18:17.955463+0000","last_active":"2024-05-22T21:18:19.169172+0000","last_peered":"2024-05-22T21:18:19.169172+0000","last_clean":"2024-05-22T21:18:19.169172+0000","last_became_active":"2024-05-22T21:18:17.955244+0000","last_became_peered":"2024-05-22T21:18:17.955244+0000","last_unstale":"2024-05-22T21:18:19.169172+0000","last_undegraded":"2024-05-22T21:18:19.169172+0000","last_fullsized":"2024-05-22T21:18:19.169172+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:30:05.982663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169273+0000","last_change":"2024-05-22T21:18:17.963537+0000","last_active":"2024-05-22T21:18:19.169273+0000","last_peered":"2024-05-22T21:18:19.169273+0000","last_clean":"2024-05-22T21:18:19.169273+0000","last_became_active":"2024-05-22T21:18:17.960629+0000","last_became_peered":"2024-05-22T21:18:17.960629+0000","last_unstale":"2024-05-22T21:18:19.169273+0000","last_undegraded":"2024-05-22T21:18:19.169273+0000","last_fullsized":"2024-05-22T21:18:19.169273+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:42:08.700957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953865+0000","last_change":"2024-05-22T21:18:17.964234+0000","last_active":"2024-05-22T21:18:18.953865+0000","last_peered":"2024-05-22T21:18:18.953865+0000","last_clean":"2024-05-22T21:18:18.953865+0000","last_became_active":"2024-05-22T21:18:17.963492+0000","last_became_peered":"2024-05-22T21:18:17.963492+0000","last_unstale":"2024-05-22T21:18:18.953865+0000","last_undegraded":"2024-05-22T21:18:18.953865+0000","last_fullsized":"2024-05-22T21:18:18.953865+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:22.295248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170669+0000","last_change":"2024-05-22T21:18:17.960444+0000","last_active":"2024-05-22T21:18:19.170669+0000","last_peered":"2024-05-22T21:18:19.170669+0000","last_clean":"2024-05-22T21:18:19.170669+0000","last_became_active":"2024-05-22T21:18:17.960207+0000","last_became_peered":"2024-05-22T21:18:17.960207+0000","last_unstale":"2024-05-22T21:18:19.170669+0000","last_undegraded":"2024-05-22T21:18:19.170669+0000","last_fullsized":"2024-05-22T21:18:19.170669+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:22.295248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953441+0000","last_change":"2024-05-22T21:18:17.958767+0000","last_active":"2024-05-22T21:18:18.953441+0000","last_peered":"2024-05-22T21:18:18.953441+0000","last_clean":"2024-05-22T21:18:18.953441+0000","last_became_active":"2024-05-22T21:18:17.958481+0000","last_became_peered":"2024-05-22T21:18:17.958481+0000","last_unstale":"2024-05-22T21:18:18.953441+0000","last_undegraded":"2024-05-22T21:18:18.953441+0000","last_fullsized":"2024-05-22T21:18:18.953441+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:23.958727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"102'4","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170722+0000","last_change":"2024-05-22T21:18:17.577456+0000","last_active":"2024-05-22T21:18:19.170722+0000","last_peered":"2024-05-22T21:18:19.170722+0000","last_clean":"2024-05-22T21:18:19.170722+0000","last_became_active":"2024-05-22T21:18:17.576784+0000","last_became_peered":"2024-05-22T21:18:17.576784+0000","last_unstale":"2024-05-22T21:18:19.170722+0000","last_undegraded":"2024-05-22T21:18:19.170722+0000","last_fullsized":"2024-05-22T21:18:19.170722+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:48:15.718335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170650+0000","last_change":"2024-05-22T21:18:17.965724+0000","last_active":"2024-05-22T21:18:19.170650+0000","last_peered":"2024-05-22T21:18:19.170650+0000","last_clean":"2024-05-22T21:18:19.170650+0000","last_became_active":"2024-05-22T21:18:17.962527+0000","last_became_peered":"2024-05-22T21:18:17.962527+0000","last_unstale":"2024-05-22T21:18:19.170650+0000","last_undegraded":"2024-05-22T21:18:19.170650+0000","last_fullsized":"2024-05-22T21:18:19.170650+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:09:17.217291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"102'1","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953680+0000","last_change":"2024-05-22T21:18:17.570132+0000","last_active":"2024-05-22T21:18:18.953680+0000","last_peered":"2024-05-22T21:18:18.953680+0000","last_clean":"2024-05-22T21:18:18.953680+0000","last_became_active":"2024-05-22T21:18:17.567690+0000","last_became_peered":"2024-05-22T21:18:17.567690+0000","last_unstale":"2024-05-22T21:18:18.953680+0000","last_undegraded":"2024-05-22T21:18:18.953680+0000","last_fullsized":"2024-05-22T21:18:18.953680+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:25:50.933329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170500+0000","last_change":"2024-05-22T21:18:17.963395+0000","last_active":"2024-05-22T21:18:19.170500+0000","last_peered":"2024-05-22T21:18:19.170500+0000","last_clean":"2024-05-22T21:18:19.170500+0000","last_became_active":"2024-05-22T21:18:17.963135+0000","last_became_peered":"2024-05-22T21:18:17.963135+0000","last_unstale":"2024-05-22T21:18:19.170500+0000","last_undegraded":"2024-05-22T21:18:19.170500+0000","last_fullsized":"2024-05-22T21:18:19.170500+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:31:06.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954515+0000","last_change":"2024-05-22T21:18:17.569334+0000","last_active":"2024-05-22T21:18:18.954515+0000","last_peered":"2024-05-22T21:18:18.954515+0000","last_clean":"2024-05-22T21:18:18.954515+0000","last_became_active":"2024-05-22T21:18:17.566887+0000","last_became_peered":"2024-05-22T21:18:17.566887+0000","last_unstale":"2024-05-22T21:18:18.954515+0000","last_undegraded":"2024-05-22T21:18:18.954515+0000","last_fullsized":"2024-05-22T21:18:18.954515+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:13:03.234580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954048+0000","last_change":"2024-05-22T21:18:17.959066+0000","last_active":"2024-05-22T21:18:18.954048+0000","last_peered":"2024-05-22T21:18:18.954048+0000","last_clean":"2024-05-22T21:18:18.954048+0000","last_became_active":"2024-05-22T21:18:17.956795+0000","last_became_peered":"2024-05-22T21:18:17.956795+0000","last_unstale":"2024-05-22T21:18:18.954048+0000","last_undegraded":"2024-05-22T21:18:18.954048+0000","last_fullsized":"2024-05-22T21:18:18.954048+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:23.958727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954584+0000","last_change":"2024-05-22T21:18:17.570719+0000","last_active":"2024-05-22T21:18:18.954584+0000","last_peered":"2024-05-22T21:18:18.954584+0000","last_clean":"2024-05-22T21:18:18.954584+0000","last_became_active":"2024-05-22T21:18:17.570489+0000","last_became_peered":"2024-05-22T21:18:17.570489+0000","last_unstale":"2024-05-22T21:18:18.954584+0000","last_undegraded":"2024-05-22T21:18:18.954584+0000","last_fullsized":"2024-05-22T21:18:18.954584+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:58:09.369054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169227+0000","last_change":"2024-05-22T21:18:17.960911+0000","last_active":"2024-05-22T21:18:19.169227+0000","last_peered":"2024-05-22T21:18:19.169227+0000","last_clean":"2024-05-22T21:18:19.169227+0000","last_became_active":"2024-05-22T21:18:17.957746+0000","last_became_peered":"2024-05-22T21:18:17.957746+0000","last_unstale":"2024-05-22T21:18:19.169227+0000","last_undegraded":"2024-05-22T21:18:19.169227+0000","last_fullsized":"2024-05-22T21:18:19.169227+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:58:33.493841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"102'1","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954180+0000","last_change":"2024-05-22T21:18:17.575282+0000","last_active":"2024-05-22T21:18:18.954180+0000","last_peered":"2024-05-22T21:18:18.954180+0000","last_clean":"2024-05-22T21:18:18.954180+0000","last_became_active":"2024-05-22T21:18:17.574110+0000","last_became_peered":"2024-05-22T21:18:17.574110+0000","last_unstale":"2024-05-22T21:18:18.954180+0000","last_undegraded":"2024-05-22T21:18:18.954180+0000","last_fullsized":"2024-05-22T21:18:18.954180+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:48:15.718335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954075+0000","last_change":"2024-05-22T21:18:17.964643+0000","last_active":"2024-05-22T21:18:18.954075+0000","last_peered":"2024-05-22T21:18:18.954075+0000","last_clean":"2024-05-22T21:18:18.954075+0000","last_became_active":"2024-05-22T21:18:17.960412+0000","last_became_peered":"2024-05-22T21:18:17.960412+0000","last_unstale":"2024-05-22T21:18:18.954075+0000","last_undegraded":"2024-05-22T21:18:18.954075+0000","last_fullsized":"2024-05-22T21:18:18.954075+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:56:49.462456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"102'1","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954423+0000","last_change":"2024-05-22T21:18:17.571413+0000","last_active":"2024-05-22T21:18:18.954423+0000","last_peered":"2024-05-22T21:18:18.954423+0000","last_clean":"2024-05-22T21:18:18.954423+0000","last_became_active":"2024-05-22T21:18:17.569088+0000","last_became_peered":"2024-05-22T21:18:17.569088+0000","last_unstale":"2024-05-22T21:18:18.954423+0000","last_undegraded":"2024-05-22T21:18:18.954423+0000","last_fullsized":"2024-05-22T21:18:18.954423+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:19.223879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953862+0000","last_change":"2024-05-22T21:18:17.961734+0000","last_active":"2024-05-22T21:18:18.953862+0000","last_peered":"2024-05-22T21:18:18.953862+0000","last_clean":"2024-05-22T21:18:18.953862+0000","last_became_active":"2024-05-22T21:18:17.961203+0000","last_became_peered":"2024-05-22T21:18:17.961203+0000","last_unstale":"2024-05-22T21:18:18.953862+0000","last_undegraded":"2024-05-22T21:18:18.953862+0000","last_fullsized":"2024-05-22T21:18:18.953862+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:49.458817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954654+0000","last_change":"2024-05-22T21:18:17.567816+0000","last_active":"2024-05-22T21:18:18.954654+0000","last_peered":"2024-05-22T21:18:18.954654+0000","last_clean":"2024-05-22T21:18:18.954654+0000","last_became_active":"2024-05-22T21:18:17.567601+0000","last_became_peered":"2024-05-22T21:18:17.567601+0000","last_unstale":"2024-05-22T21:18:18.954654+0000","last_undegraded":"2024-05-22T21:18:18.954654+0000","last_fullsized":"2024-05-22T21:18:18.954654+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:54:12.856971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393016+0000","last_change":"2024-05-22T21:18:17.954207+0000","last_active":"2024-05-22T21:18:19.393016+0000","last_peered":"2024-05-22T21:18:19.393016+0000","last_clean":"2024-05-22T21:18:19.393016+0000","last_became_active":"2024-05-22T21:18:17.953977+0000","last_became_peered":"2024-05-22T21:18:17.953977+0000","last_unstale":"2024-05-22T21:18:19.393016+0000","last_undegraded":"2024-05-22T21:18:19.393016+0000","last_fullsized":"2024-05-22T21:18:19.393016+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:41:23.892969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169341+0000","last_change":"2024-05-22T21:18:17.566911+0000","last_active":"2024-05-22T21:18:19.169341+0000","last_peered":"2024-05-22T21:18:19.169341+0000","last_clean":"2024-05-22T21:18:19.169341+0000","last_became_active":"2024-05-22T21:18:17.566696+0000","last_became_peered":"2024-05-22T21:18:17.566696+0000","last_unstale":"2024-05-22T21:18:19.169341+0000","last_undegraded":"2024-05-22T21:18:19.169341+0000","last_fullsized":"2024-05-22T21:18:19.169341+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:18:20.507884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393137+0000","last_change":"2024-05-22T21:18:17.955339+0000","last_active":"2024-05-22T21:18:19.393137+0000","last_peered":"2024-05-22T21:18:19.393137+0000","last_clean":"2024-05-22T21:18:19.393137+0000","last_became_active":"2024-05-22T21:18:17.955173+0000","last_became_peered":"2024-05-22T21:18:17.955173+0000","last_unstale":"2024-05-22T21:18:19.393137+0000","last_undegraded":"2024-05-22T21:18:19.393137+0000","last_fullsized":"2024-05-22T21:18:19.393137+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:06:36.975586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170634+0000","last_change":"2024-05-22T21:18:17.574214+0000","last_active":"2024-05-22T21:18:19.170634+0000","last_peered":"2024-05-22T21:18:19.170634+0000","last_clean":"2024-05-22T21:18:19.170634+0000","last_became_active":"2024-05-22T21:18:17.573769+0000","last_became_peered":"2024-05-22T21:18:17.573769+0000","last_unstale":"2024-05-22T21:18:19.170634+0000","last_undegraded":"2024-05-22T21:18:19.170634+0000","last_fullsized":"2024-05-22T21:18:19.170634+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:10:10.103204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.394075+0000","last_change":"2024-05-22T21:18:17.953706+0000","last_active":"2024-05-22T21:18:19.394075+0000","last_peered":"2024-05-22T21:18:19.394075+0000","last_clean":"2024-05-22T21:18:19.394075+0000","last_became_active":"2024-05-22T21:18:17.953296+0000","last_became_peered":"2024-05-22T21:18:17.953296+0000","last_unstale":"2024-05-22T21:18:19.394075+0000","last_undegraded":"2024-05-22T21:18:19.394075+0000","last_fullsized":"2024-05-22T21:18:19.394075+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:16.013875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170743+0000","last_change":"2024-05-22T21:18:17.572117+0000","last_active":"2024-05-22T21:18:19.170743+0000","last_peered":"2024-05-22T21:18:19.170743+0000","last_clean":"2024-05-22T21:18:19.170743+0000","last_became_active":"2024-05-22T21:18:17.571601+0000","last_became_peered":"2024-05-22T21:18:17.571601+0000","last_unstale":"2024-05-22T21:18:19.170743+0000","last_undegraded":"2024-05-22T21:18:19.170743+0000","last_fullsized":"2024-05-22T21:18:19.170743+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:25:50.933329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954263+0000","last_change":"2024-05-22T21:18:17.957084+0000","last_active":"2024-05-22T21:18:18.954263+0000","last_peered":"2024-05-22T21:18:18.954263+0000","last_clean":"2024-05-22T21:18:18.954263+0000","last_became_active":"2024-05-22T21:18:17.956592+0000","last_became_peered":"2024-05-22T21:18:17.956592+0000","last_unstale":"2024-05-22T21:18:18.954263+0000","last_undegraded":"2024-05-22T21:18:18.954263+0000","last_fullsized":"2024-05-22T21:18:18.954263+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:22.052183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"102'3","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170828+0000","last_change":"2024-05-22T21:18:17.577310+0000","last_active":"2024-05-22T21:18:19.170828+0000","last_peered":"2024-05-22T21:18:19.170828+0000","last_clean":"2024-05-22T21:18:19.170828+0000","last_became_active":"2024-05-22T21:18:17.576229+0000","last_became_peered":"2024-05-22T21:18:17.576229+0000","last_unstale":"2024-05-22T21:18:19.170828+0000","last_undegraded":"2024-05-22T21:18:19.170828+0000","last_fullsized":"2024-05-22T21:18:19.170828+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:27:43.610883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954225+0000","last_change":"2024-05-22T21:18:17.962890+0000","last_active":"2024-05-22T21:18:18.954225+0000","last_peered":"2024-05-22T21:18:18.954225+0000","last_clean":"2024-05-22T21:18:18.954225+0000","last_became_active":"2024-05-22T21:18:17.962655+0000","last_became_peered":"2024-05-22T21:18:17.962655+0000","last_unstale":"2024-05-22T21:18:18.954225+0000","last_undegraded":"2024-05-22T21:18:18.954225+0000","last_fullsized":"2024-05-22T21:18:18.954225+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:09:27.584040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"102'6","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.394098+0000","last_change":"2024-05-22T21:18:17.570233+0000","last_active":"2024-05-22T21:18:19.394098+0000","last_peered":"2024-05-22T21:18:19.394098+0000","last_clean":"2024-05-22T21:18:19.394098+0000","last_became_active":"2024-05-22T21:18:17.569591+0000","last_became_peered":"2024-05-22T21:18:17.569591+0000","last_unstale":"2024-05-22T21:18:19.394098+0000","last_undegraded":"2024-05-22T21:18:19.394098+0000","last_fullsized":"2024-05-22T21:18:19.394098+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:35:29.469510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954400+0000","last_change":"2024-05-22T21:18:17.960697+0000","last_active":"2024-05-22T21:18:18.954400+0000","last_peered":"2024-05-22T21:18:18.954400+0000","last_clean":"2024-05-22T21:18:18.954400+0000","last_became_active":"2024-05-22T21:18:17.960207+0000","last_became_peered":"2024-05-22T21:18:17.960207+0000","last_unstale":"2024-05-22T21:18:18.954400+0000","last_undegraded":"2024-05-22T21:18:18.954400+0000","last_fullsized":"2024-05-22T21:18:18.954400+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:49.458817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169388+0000","last_change":"2024-05-22T21:18:17.571909+0000","last_active":"2024-05-22T21:18:19.169388+0000","last_peered":"2024-05-22T21:18:19.169388+0000","last_clean":"2024-05-22T21:18:19.169388+0000","last_became_active":"2024-05-22T21:18:17.571512+0000","last_became_peered":"2024-05-22T21:18:17.571512+0000","last_unstale":"2024-05-22T21:18:19.169388+0000","last_undegraded":"2024-05-22T21:18:19.169388+0000","last_fullsized":"2024-05-22T21:18:19.169388+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:36:46.664343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953954+0000","last_change":"2024-05-22T21:18:17.961953+0000","last_active":"2024-05-22T21:18:18.953954+0000","last_peered":"2024-05-22T21:18:18.953954+0000","last_clean":"2024-05-22T21:18:18.953954+0000","last_became_active":"2024-05-22T21:18:17.961516+0000","last_became_peered":"2024-05-22T21:18:17.961516+0000","last_unstale":"2024-05-22T21:18:18.953954+0000","last_undegraded":"2024-05-22T21:18:18.953954+0000","last_fullsized":"2024-05-22T21:18:18.953954+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:16.013875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393111+0000","last_change":"2024-05-22T21:18:17.572880+0000","last_active":"2024-05-22T21:18:19.393111+0000","last_peered":"2024-05-22T21:18:19.393111+0000","last_clean":"2024-05-22T21:18:19.393111+0000","last_became_active":"2024-05-22T21:18:17.572655+0000","last_became_peered":"2024-05-22T21:18:17.572655+0000","last_unstale":"2024-05-22T21:18:19.393111+0000","last_undegraded":"2024-05-22T21:18:19.393111+0000","last_fullsized":"2024-05-22T21:18:19.393111+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:21.050069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169092+0000","last_change":"2024-05-22T21:18:17.957857+0000","last_active":"2024-05-22T21:18:19.169092+0000","last_peered":"2024-05-22T21:18:19.169092+0000","last_clean":"2024-05-22T21:18:19.169092+0000","last_became_active":"2024-05-22T21:18:17.957491+0000","last_became_peered":"2024-05-22T21:18:17.957491+0000","last_unstale":"2024-05-22T21:18:19.169092+0000","last_undegraded":"2024-05-22T21:18:19.169092+0000","last_fullsized":"2024-05-22T21:18:19.169092+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:14:43.052939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393981+0000","last_change":"2024-05-22T21:18:17.569729+0000","last_active":"2024-05-22T21:18:19.393981+0000","last_peered":"2024-05-22T21:18:19.393981+0000","last_clean":"2024-05-22T21:18:19.393981+0000","last_became_active":"2024-05-22T21:18:17.567831+0000","last_became_peered":"2024-05-22T21:18:17.567831+0000","last_unstale":"2024-05-22T21:18:19.393981+0000","last_undegraded":"2024-05-22T21:18:19.393981+0000","last_fullsized":"2024-05-22T21:18:19.393981+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:42:10.010243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169360+0000","last_change":"2024-05-22T21:18:17.963428+0000","last_active":"2024-05-22T21:18:19.169360+0000","last_peered":"2024-05-22T21:18:19.169360+0000","last_clean":"2024-05-22T21:18:19.169360+0000","last_became_active":"2024-05-22T21:18:17.962429+0000","last_became_peered":"2024-05-22T21:18:17.962429+0000","last_unstale":"2024-05-22T21:18:19.169360+0000","last_undegraded":"2024-05-22T21:18:19.169360+0000","last_fullsized":"2024-05-22T21:18:19.169360+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:19:44.456977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"102'8","reported_seq":19,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954520+0000","last_change":"2024-05-22T21:18:17.570044+0000","last_active":"2024-05-22T21:18:18.954520+0000","last_peered":"2024-05-22T21:18:18.954520+0000","last_clean":"2024-05-22T21:18:18.954520+0000","last_became_active":"2024-05-22T21:18:17.569796+0000","last_became_peered":"2024-05-22T21:18:17.569796+0000","last_unstale":"2024-05-22T21:18:18.954520+0000","last_undegraded":"2024-05-22T21:18:18.954520+0000","last_fullsized":"2024-05-22T21:18:18.954520+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:53:59.583140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954136+0000","last_change":"2024-05-22T21:18:17.957133+0000","last_active":"2024-05-22T21:18:18.954136+0000","last_peered":"2024-05-22T21:18:18.954136+0000","last_clean":"2024-05-22T21:18:18.954136+0000","last_became_active":"2024-05-22T21:18:17.954295+0000","last_became_peered":"2024-05-22T21:18:17.954295+0000","last_unstale":"2024-05-22T21:18:18.954136+0000","last_undegraded":"2024-05-22T21:18:18.954136+0000","last_fullsized":"2024-05-22T21:18:18.954136+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:45:07.161984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"102'1","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170585+0000","last_change":"2024-05-22T21:18:17.576505+0000","last_active":"2024-05-22T21:18:19.170585+0000","last_peered":"2024-05-22T21:18:19.170585+0000","last_clean":"2024-05-22T21:18:19.170585+0000","last_became_active":"2024-05-22T21:18:17.574374+0000","last_became_peered":"2024-05-22T21:18:17.574374+0000","last_unstale":"2024-05-22T21:18:19.170585+0000","last_undegraded":"2024-05-22T21:18:19.170585+0000","last_fullsized":"2024-05-22T21:18:19.170585+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:06:41.500056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170443+0000","last_change":"2024-05-22T21:18:17.962646+0000","last_active":"2024-05-22T21:18:19.170443+0000","last_peered":"2024-05-22T21:18:19.170443+0000","last_clean":"2024-05-22T21:18:19.170443+0000","last_became_active":"2024-05-22T21:18:17.962413+0000","last_became_peered":"2024-05-22T21:18:17.962413+0000","last_unstale":"2024-05-22T21:18:19.170443+0000","last_undegraded":"2024-05-22T21:18:19.170443+0000","last_fullsized":"2024-05-22T21:18:19.170443+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:09:27.584040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954565+0000","last_change":"2024-05-22T21:18:17.573846+0000","last_active":"2024-05-22T21:18:18.954565+0000","last_peered":"2024-05-22T21:18:18.954565+0000","last_clean":"2024-05-22T21:18:18.954565+0000","last_became_active":"2024-05-22T21:18:17.573596+0000","last_became_peered":"2024-05-22T21:18:17.573596+0000","last_unstale":"2024-05-22T21:18:18.954565+0000","last_undegraded":"2024-05-22T21:18:18.954565+0000","last_fullsized":"2024-05-22T21:18:18.954565+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:45.834147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953953+0000","last_change":"2024-05-22T21:18:17.966432+0000","last_active":"2024-05-22T21:18:18.953953+0000","last_peered":"2024-05-22T21:18:18.953953+0000","last_clean":"2024-05-22T21:18:18.953953+0000","last_became_active":"2024-05-22T21:18:17.966172+0000","last_became_peered":"2024-05-22T21:18:17.966172+0000","last_unstale":"2024-05-22T21:18:18.953953+0000","last_undegraded":"2024-05-22T21:18:18.953953+0000","last_fullsized":"2024-05-22T21:18:18.953953+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:16.013875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954299+0000","last_change":"2024-05-22T21:18:17.572905+0000","last_active":"2024-05-22T21:18:18.954299+0000","last_peered":"2024-05-22T21:18:18.954299+0000","last_clean":"2024-05-22T21:18:18.954299+0000","last_became_active":"2024-05-22T21:18:17.572686+0000","last_became_peered":"2024-05-22T21:18:17.572686+0000","last_unstale":"2024-05-22T21:18:18.954299+0000","last_undegraded":"2024-05-22T21:18:18.954299+0000","last_fullsized":"2024-05-22T21:18:18.954299+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:36.130802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393070+0000","last_change":"2024-05-22T21:18:17.959542+0000","last_active":"2024-05-22T21:18:19.393070+0000","last_peered":"2024-05-22T21:18:19.393070+0000","last_clean":"2024-05-22T21:18:19.393070+0000","last_became_active":"2024-05-22T21:18:17.957710+0000","last_became_peered":"2024-05-22T21:18:17.957710+0000","last_unstale":"2024-05-22T21:18:19.393070+0000","last_undegraded":"2024-05-22T21:18:19.393070+0000","last_fullsized":"2024-05-22T21:18:19.393070+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:48:21.058352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393023+0000","last_change":"2024-05-22T21:18:17.565636+0000","last_active":"2024-05-22T21:18:19.393023+0000","last_peered":"2024-05-22T21:18:19.393023+0000","last_clean":"2024-05-22T21:18:19.393023+0000","last_became_active":"2024-05-22T21:18:17.565314+0000","last_became_peered":"2024-05-22T21:18:17.565314+0000","last_unstale":"2024-05-22T21:18:19.393023+0000","last_undegraded":"2024-05-22T21:18:19.393023+0000","last_fullsized":"2024-05-22T21:18:19.393023+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:48.456703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393948+0000","last_change":"2024-05-22T21:18:17.955262+0000","last_active":"2024-05-22T21:18:19.393948+0000","last_peered":"2024-05-22T21:18:19.393948+0000","last_clean":"2024-05-22T21:18:19.393948+0000","last_became_active":"2024-05-22T21:18:17.955047+0000","last_became_peered":"2024-05-22T21:18:17.955047+0000","last_unstale":"2024-05-22T21:18:19.393948+0000","last_undegraded":"2024-05-22T21:18:19.393948+0000","last_fullsized":"2024-05-22T21:18:19.393948+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:22.295248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393142+0000","last_change":"2024-05-22T21:18:17.567966+0000","last_active":"2024-05-22T21:18:19.393142+0000","last_peered":"2024-05-22T21:18:19.393142+0000","last_clean":"2024-05-22T21:18:19.393142+0000","last_became_active":"2024-05-22T21:18:17.567024+0000","last_became_peered":"2024-05-22T21:18:17.567024+0000","last_unstale":"2024-05-22T21:18:19.393142+0000","last_undegraded":"2024-05-22T21:18:19.393142+0000","last_fullsized":"2024-05-22T21:18:19.393142+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:08:28.543920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393481+0000","last_change":"2024-05-22T21:18:17.953096+0000","last_active":"2024-05-22T21:18:19.393481+0000","last_peered":"2024-05-22T21:18:19.393481+0000","last_clean":"2024-05-22T21:18:19.393481+0000","last_became_active":"2024-05-22T21:18:17.952744+0000","last_became_peered":"2024-05-22T21:18:17.952744+0000","last_unstale":"2024-05-22T21:18:19.393481+0000","last_undegraded":"2024-05-22T21:18:19.393481+0000","last_fullsized":"2024-05-22T21:18:19.393481+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:21:00.882050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393609+0000","last_change":"2024-05-22T21:18:17.571043+0000","last_active":"2024-05-22T21:18:19.393609+0000","last_peered":"2024-05-22T21:18:19.393609+0000","last_clean":"2024-05-22T21:18:19.393609+0000","last_became_active":"2024-05-22T21:18:17.570750+0000","last_became_peered":"2024-05-22T21:18:17.570750+0000","last_unstale":"2024-05-22T21:18:19.393609+0000","last_undegraded":"2024-05-22T21:18:19.393609+0000","last_fullsized":"2024-05-22T21:18:19.393609+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:41:09.473523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393978+0000","last_change":"2024-05-22T21:18:17.961201+0000","last_active":"2024-05-22T21:18:19.393978+0000","last_peered":"2024-05-22T21:18:19.393978+0000","last_clean":"2024-05-22T21:18:19.393978+0000","last_became_active":"2024-05-22T21:18:17.960963+0000","last_became_peered":"2024-05-22T21:18:17.960963+0000","last_unstale":"2024-05-22T21:18:19.393978+0000","last_undegraded":"2024-05-22T21:18:19.393978+0000","last_fullsized":"2024-05-22T21:18:19.393978+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:56:49.462456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393183+0000","last_change":"2024-05-22T21:18:17.569485+0000","last_active":"2024-05-22T21:18:19.393183+0000","last_peered":"2024-05-22T21:18:19.393183+0000","last_clean":"2024-05-22T21:18:19.393183+0000","last_became_active":"2024-05-22T21:18:17.569276+0000","last_became_peered":"2024-05-22T21:18:17.569276+0000","last_unstale":"2024-05-22T21:18:19.393183+0000","last_undegraded":"2024-05-22T21:18:19.393183+0000","last_fullsized":"2024-05-22T21:18:19.393183+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:45:06.159869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954330+0000","last_change":"2024-05-22T21:18:17.957705+0000","last_active":"2024-05-22T21:18:18.954330+0000","last_peered":"2024-05-22T21:18:18.954330+0000","last_clean":"2024-05-22T21:18:18.954330+0000","last_became_active":"2024-05-22T21:18:17.957268+0000","last_became_peered":"2024-05-22T21:18:17.957268+0000","last_unstale":"2024-05-22T21:18:18.954330+0000","last_undegraded":"2024-05-22T21:18:18.954330+0000","last_fullsized":"2024-05-22T21:18:18.954330+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:13:04.236694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169584+0000","last_change":"2024-05-22T21:18:17.570186+0000","last_active":"2024-05-22T21:18:19.169584+0000","last_peered":"2024-05-22T21:18:19.169584+0000","last_clean":"2024-05-22T21:18:19.169584+0000","last_became_active":"2024-05-22T21:18:17.569973+0000","last_became_peered":"2024-05-22T21:18:17.569973+0000","last_unstale":"2024-05-22T21:18:19.169584+0000","last_undegraded":"2024-05-22T21:18:19.169584+0000","last_fullsized":"2024-05-22T21:18:19.169584+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:52:12.503228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954075+0000","last_change":"2024-05-22T21:18:17.958467+0000","last_active":"2024-05-22T21:18:18.954075+0000","last_peered":"2024-05-22T21:18:18.954075+0000","last_clean":"2024-05-22T21:18:18.954075+0000","last_became_active":"2024-05-22T21:18:17.958201+0000","last_became_peered":"2024-05-22T21:18:17.958201+0000","last_unstale":"2024-05-22T21:18:18.954075+0000","last_undegraded":"2024-05-22T21:18:18.954075+0000","last_fullsized":"2024-05-22T21:18:18.954075+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:05:16.362966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169309+0000","last_change":"2024-05-22T21:18:17.570492+0000","last_active":"2024-05-22T21:18:19.169309+0000","last_peered":"2024-05-22T21:18:19.169309+0000","last_clean":"2024-05-22T21:18:19.169309+0000","last_became_active":"2024-05-22T21:18:17.570091+0000","last_became_peered":"2024-05-22T21:18:17.570091+0000","last_unstale":"2024-05-22T21:18:19.169309+0000","last_undegraded":"2024-05-22T21:18:19.169309+0000","last_fullsized":"2024-05-22T21:18:19.169309+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:30:19.397462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170478+0000","last_change":"2024-05-22T21:18:17.964113+0000","last_active":"2024-05-22T21:18:19.170478+0000","last_peered":"2024-05-22T21:18:19.170478+0000","last_clean":"2024-05-22T21:18:19.170478+0000","last_became_active":"2024-05-22T21:18:17.961742+0000","last_became_peered":"2024-05-22T21:18:17.961742+0000","last_unstale":"2024-05-22T21:18:19.170478+0000","last_undegraded":"2024-05-22T21:18:19.170478+0000","last_fullsized":"2024-05-22T21:18:19.170478+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:46.836261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170677+0000","last_change":"2024-05-22T21:18:17.572676+0000","last_active":"2024-05-22T21:18:19.170677+0000","last_peered":"2024-05-22T21:18:19.170677+0000","last_clean":"2024-05-22T21:18:19.170677+0000","last_became_active":"2024-05-22T21:18:17.572431+0000","last_became_peered":"2024-05-22T21:18:17.572431+0000","last_unstale":"2024-05-22T21:18:19.170677+0000","last_undegraded":"2024-05-22T21:18:19.170677+0000","last_fullsized":"2024-05-22T21:18:19.170677+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:58:32.491726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953576+0000","last_change":"2024-05-22T21:18:17.961274+0000","last_active":"2024-05-22T21:18:18.953576+0000","last_peered":"2024-05-22T21:18:18.953576+0000","last_clean":"2024-05-22T21:18:18.953576+0000","last_became_active":"2024-05-22T21:18:17.961038+0000","last_became_peered":"2024-05-22T21:18:17.961038+0000","last_unstale":"2024-05-22T21:18:18.953576+0000","last_undegraded":"2024-05-22T21:18:18.953576+0000","last_fullsized":"2024-05-22T21:18:18.953576+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:22:06.607488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170614+0000","last_change":"2024-05-22T21:18:17.576592+0000","last_active":"2024-05-22T21:18:19.170614+0000","last_peered":"2024-05-22T21:18:19.170614+0000","last_clean":"2024-05-22T21:18:19.170614+0000","last_became_active":"2024-05-22T21:18:17.576366+0000","last_became_peered":"2024-05-22T21:18:17.576366+0000","last_unstale":"2024-05-22T21:18:19.170614+0000","last_undegraded":"2024-05-22T21:18:19.170614+0000","last_fullsized":"2024-05-22T21:18:19.170614+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:19.223879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393911+0000","last_change":"2024-05-22T21:18:17.956103+0000","last_active":"2024-05-22T21:18:19.393911+0000","last_peered":"2024-05-22T21:18:19.393911+0000","last_clean":"2024-05-22T21:18:19.393911+0000","last_became_active":"2024-05-22T21:18:17.955874+0000","last_became_peered":"2024-05-22T21:18:17.955874+0000","last_unstale":"2024-05-22T21:18:19.393911+0000","last_undegraded":"2024-05-22T21:18:19.393911+0000","last_fullsized":"2024-05-22T21:18:19.393911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:09:27.584040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954657+0000","last_change":"2024-05-22T21:18:17.573273+0000","last_active":"2024-05-22T21:18:18.954657+0000","last_peered":"2024-05-22T21:18:18.954657+0000","last_clean":"2024-05-22T21:18:18.954657+0000","last_became_active":"2024-05-22T21:18:17.572717+0000","last_became_peered":"2024-05-22T21:18:17.572717+0000","last_unstale":"2024-05-22T21:18:18.954657+0000","last_undegraded":"2024-05-22T21:18:18.954657+0000","last_fullsized":"2024-05-22T21:18:18.954657+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:21.293134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954495+0000","last_change":"2024-05-22T21:18:17.961113+0000","last_active":"2024-05-22T21:18:18.954495+0000","last_peered":"2024-05-22T21:18:18.954495+0000","last_clean":"2024-05-22T21:18:18.954495+0000","last_became_active":"2024-05-22T21:18:17.960493+0000","last_became_peered":"2024-05-22T21:18:17.960493+0000","last_unstale":"2024-05-22T21:18:18.954495+0000","last_undegraded":"2024-05-22T21:18:18.954495+0000","last_fullsized":"2024-05-22T21:18:18.954495+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:16.013875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954380+0000","last_change":"2024-05-22T21:18:17.575482+0000","last_active":"2024-05-22T21:18:18.954380+0000","last_peered":"2024-05-22T21:18:18.954380+0000","last_clean":"2024-05-22T21:18:18.954380+0000","last_became_active":"2024-05-22T21:18:17.574884+0000","last_became_peered":"2024-05-22T21:18:17.574884+0000","last_unstale":"2024-05-22T21:18:18.954380+0000","last_undegraded":"2024-05-22T21:18:18.954380+0000","last_fullsized":"2024-05-22T21:18:18.954380+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:06:41.500056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953592+0000","last_change":"2024-05-22T21:18:17.958011+0000","last_active":"2024-05-22T21:18:18.953592+0000","last_peered":"2024-05-22T21:18:18.953592+0000","last_clean":"2024-05-22T21:18:18.953592+0000","last_became_active":"2024-05-22T21:18:17.957625+0000","last_became_peered":"2024-05-22T21:18:17.957625+0000","last_unstale":"2024-05-22T21:18:18.953592+0000","last_undegraded":"2024-05-22T21:18:18.953592+0000","last_fullsized":"2024-05-22T21:18:18.953592+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:31:06.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170872+0000","last_change":"2024-05-22T21:18:17.577139+0000","last_active":"2024-05-22T21:18:19.170872+0000","last_peered":"2024-05-22T21:18:19.170872+0000","last_clean":"2024-05-22T21:18:19.170872+0000","last_became_active":"2024-05-22T21:18:17.573999+0000","last_became_peered":"2024-05-22T21:18:17.573999+0000","last_unstale":"2024-05-22T21:18:19.170872+0000","last_undegraded":"2024-05-22T21:18:19.170872+0000","last_fullsized":"2024-05-22T21:18:19.170872+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:42:07.698843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170780+0000","last_change":"2024-05-22T21:18:17.962821+0000","last_active":"2024-05-22T21:18:19.170780+0000","last_peered":"2024-05-22T21:18:19.170780+0000","last_clean":"2024-05-22T21:18:19.170780+0000","last_became_active":"2024-05-22T21:18:17.958435+0000","last_became_peered":"2024-05-22T21:18:17.958435+0000","last_unstale":"2024-05-22T21:18:19.170780+0000","last_undegraded":"2024-05-22T21:18:19.170780+0000","last_fullsized":"2024-05-22T21:18:19.170780+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:23.958727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953772+0000","last_change":"2024-05-22T21:18:17.572361+0000","last_active":"2024-05-22T21:18:18.953772+0000","last_peered":"2024-05-22T21:18:18.953772+0000","last_clean":"2024-05-22T21:18:18.953772+0000","last_became_active":"2024-05-22T21:18:17.572121+0000","last_became_peered":"2024-05-22T21:18:17.572121+0000","last_unstale":"2024-05-22T21:18:18.953772+0000","last_undegraded":"2024-05-22T21:18:18.953772+0000","last_fullsized":"2024-05-22T21:18:18.953772+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:10:10.103204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169493+0000","last_change":"2024-05-22T21:18:17.959934+0000","last_active":"2024-05-22T21:18:19.169493+0000","last_peered":"2024-05-22T21:18:19.169493+0000","last_clean":"2024-05-22T21:18:19.169493+0000","last_became_active":"2024-05-22T21:18:17.959654+0000","last_became_peered":"2024-05-22T21:18:17.959654+0000","last_unstale":"2024-05-22T21:18:19.169493+0000","last_undegraded":"2024-05-22T21:18:19.169493+0000","last_fullsized":"2024-05-22T21:18:19.169493+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:06:42.502171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"102'1","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954428+0000","last_change":"2024-05-22T21:18:17.575307+0000","last_active":"2024-05-22T21:18:18.954428+0000","last_peered":"2024-05-22T21:18:18.954428+0000","last_clean":"2024-05-22T21:18:18.954428+0000","last_became_active":"2024-05-22T21:18:17.574793+0000","last_became_peered":"2024-05-22T21:18:17.574793+0000","last_unstale":"2024-05-22T21:18:18.954428+0000","last_undegraded":"2024-05-22T21:18:18.954428+0000","last_fullsized":"2024-05-22T21:18:18.954428+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:34:20.054587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":248,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-22T21:18:17.947888+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:18:17.947888+0000","last_peered":"2024-05-22T21:18:17.947888+0000","last_clean":"2024-05-22T21:18:17.947888+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:18:17.947888+0000","last_undegraded":"2024-05-22T21:18:17.947888+0000","last_fullsized":"2024-05-22T21:18:17.947888+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1121,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194190,"statfs":{"total":200000000,"available":198851134,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72599999999999998,"15min":0.76200000000000001},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.98899999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70399999999999996,"15min":0.69199999999999995},"min":{"1min":0.48599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97699999999999998,"5min":1.889,"15min":1.889},"last":0.54900000000000004}]},{"osd":1,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75,"15min":0.77800000000000002},"min":{"1min":0.502,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95799999999999996,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.72999999999999998,"15min":0.76500000000000001},"min":{"1min":0.44900000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92700000000000005,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.51500000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94199999999999995,"5min":1.923,"15min":1.923},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.74399999999999999,"15min":0.72699999999999998},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.048,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.74199999999999999}]},{"osd":3,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.57299999999999995,"5min":0.443,"15min":0.443},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77500000000000002,"15min":0.79900000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.014,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.79100000000000004}]},{"osd":4,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.74299999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88900000000000001,"5min":1.018,"15min":1.018},"last":0.88700000000000001}]},{"osd":5,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63900000000000001,"15min":0.65500000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89600000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82999999999999996,"5min":0.996,"15min":0.996},"last":0.91000000000000003}]},{"osd":6,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96999999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66000000000000003,"15min":0.64000000000000001},"min":{"1min":0.51000000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83199999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.97699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194766,"statfs":{"total":200000000,"available":199440878,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.49299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70599999999999996,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95899999999999996,"5min":1.107,"15min":1.107},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.034,"5min":1.081,"15min":1.081},"last":0.56299999999999994}]},{"osd":2,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78300000000000003,"15min":0.75900000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.746,"15min":0.73899999999999999},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.53900000000000003}]},{"osd":3,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.748,"15min":0.74199999999999999},"min":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0249999999999999,"5min":1.415,"15min":1.415},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.004,"5min":1.788,"15min":1.788},"last":0.50800000000000001}]},{"osd":4,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.41699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.89900000000000002,"5min":1.006,"15min":1.006},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95899999999999996,"5min":2.718,"15min":2.718},"last":0.61899999999999999}]},{"osd":5,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.745,"15min":0.70299999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81799999999999995,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.45500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.67100000000000004}]},{"osd":7,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70999999999999996,"15min":0.73799999999999999},"min":{"1min":0.53600000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0469999999999999,"5min":1.095,"15min":1.095},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72699999999999998,"15min":0.75700000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92500000000000004,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.438}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194765,"statfs":{"total":200000000,"available":199439986,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69699999999999995,"15min":0.70699999999999996},"min":{"1min":0.58199999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89300000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.69399999999999995,"15min":0.67400000000000004},"min":{"1min":0.63100000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.93600000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.497,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88600000000000001,"5min":1.216,"15min":1.216},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69599999999999995,"15min":0.68600000000000005},"min":{"1min":0.52900000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.84299999999999997,"5min":1.048,"15min":1.048},"last":0.81100000000000005}]},{"osd":2,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.72499999999999998,"15min":0.70099999999999996},"min":{"1min":0.56999999999999995,"5min":0.436,"15min":0.436},"max":{"1min":0.96499999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.69799999999999995,"15min":0.67600000000000005},"min":{"1min":0.61299999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92000000000000004,"5min":1.252,"15min":1.252},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.88600000000000001,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.746,"15min":0.73399999999999999},"min":{"1min":0.56299999999999994,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95299999999999996,"5min":1.111,"15min":1.111},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60299999999999998,"15min":0.63100000000000001},"min":{"1min":0.42899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92800000000000005,"5min":1.272,"15min":1.272},"last":0.754},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.44400000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.91700000000000004,"5min":1.038,"15min":1.038},"last":0.52100000000000002}]},{"osd":6,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63900000000000001,"15min":0.63100000000000001},"min":{"1min":0.49099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.498},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65100000000000002,"15min":0.66300000000000003},"min":{"1min":0.41499999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.877,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.83399999999999996}]},{"osd":7,"last update":"Wed May 22 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71499999999999997,"15min":0.71699999999999997},"min":{"1min":0.53000000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77800000000000002,"15min":0.79600000000000004},"min":{"1min":0.57799999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95899999999999996,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.71099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194767,"statfs":{"total":200000000,"available":199441572,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78200000000000003,"15min":0.748},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.89300000000000002,"5min":6.6539999999999999,"15min":6.6539999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.747,"15min":0.72799999999999998},"min":{"1min":0.49299999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93300000000000005,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":0.49299999999999999}]},{"osd":1,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68200000000000005,"15min":0.71599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88200000000000001,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92200000000000004,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.48399999999999999}]},{"osd":2,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70199999999999996,"15min":0.68899999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0940000000000001,"5min":1.335,"15min":1.335},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70099999999999996,"15min":0.70399999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95099999999999996,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.436}]},{"osd":3,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.54100000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.98999999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.78900000000000003},"min":{"1min":0.53200000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96799999999999997,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.68700000000000006}]},{"osd":5,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.39400000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.93500000000000005,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91900000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.91900000000000004}]},{"osd":6,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65900000000000003,"15min":0.69299999999999995},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91600000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.51800000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.91100000000000003,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.52400000000000002}]},{"osd":7,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.69599999999999995,"15min":0.69999999999999996},"min":{"1min":0.48099999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98199999999999998,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.74099999999999999,"15min":0.78000000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97999999999999998,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.73399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194765,"statfs":{"total":200000000,"available":199440008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.42999999999999999,"15min":0.40699999999999997},"min":{"1min":0.29699999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.71399999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.442,"15min":0.40799999999999997},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.33400000000000002}]},{"osd":1,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46000000000000002,"15min":0.44800000000000001},"min":{"1min":0.28399999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.624,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71799999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.46000000000000002}]},{"osd":2,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51000000000000001,"15min":0.496},"min":{"1min":0.376,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68999999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51100000000000001,"15min":0.48999999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69899999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.439}]},{"osd":4,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.74099999999999999,"15min":0.70599999999999996},"min":{"1min":0.50900000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0209999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.749,"15min":0.71899999999999997},"min":{"1min":0.57599999999999996,"5min":0.435,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.72399999999999998}]},{"osd":5,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.75800000000000001,"15min":0.69899999999999995},"min":{"1min":0.622,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.002,"5min":1.234,"15min":1.234},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.70299999999999996},"min":{"1min":0.53500000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0449999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.73199999999999998}]},{"osd":6,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.71599999999999997},"min":{"1min":0.48899999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97499999999999998,"5min":1.276,"15min":1.276},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75800000000000001,"15min":0.70799999999999996},"min":{"1min":0.54200000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0760000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.69099999999999995}]},{"osd":7,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.75800000000000001},"min":{"1min":0.52800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.0900000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78500000000000003,"15min":0.74099999999999999},"min":{"1min":0.63300000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0649999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.85799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194766,"statfs":{"total":200000000,"available":199440990,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.59399999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.46300000000000002,"15min":0.443},"min":{"1min":0.33200000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.36599999999999999}]},{"osd":1,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.32900000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.624,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54300000000000004,"15min":0.52100000000000002},"min":{"1min":0.442,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.442}]},{"osd":3,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83099999999999996,"5min":0.871,"15min":0.871},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.47399999999999998,"15min":0.45900000000000002},"min":{"1min":0.34699999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.68000000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.34699999999999998}]},{"osd":4,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73399999999999999,"15min":0.68999999999999995},"min":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95599999999999996,"5min":1.095,"15min":1.095},"last":0.88},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77900000000000003,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.77400000000000002}]},{"osd":5,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.73299999999999998,"15min":0.70699999999999996},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72599999999999998,"15min":0.70199999999999996},"min":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92900000000000005,"5min":1.127,"15min":1.127},"last":0.73099999999999998}]},{"osd":6,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73099999999999998,"15min":0.70299999999999996},"min":{"1min":0.52000000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.02,"5min":1.054,"15min":1.054},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69899999999999995,"15min":0.67800000000000005},"min":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.89600000000000002,"5min":1.002,"15min":1.002},"last":0.82899999999999996}]},{"osd":7,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.76800000000000002,"15min":0.73199999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.77200000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0249999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.89700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194766,"statfs":{"total":200000000,"available":199441378,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42199999999999999,"15min":0.432},"min":{"1min":0.30299999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56999999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.432},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.44900000000000001,"15min":0.433},"min":{"1min":0.28399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.84799999999999998,"5min":0.876,"15min":0.876},"last":0.53600000000000003}]},{"osd":2,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48899999999999999,"15min":0.48999999999999999},"min":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.748,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.51100000000000001,"15min":0.53600000000000003},"min":{"1min":0.35699999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64700000000000002,"5min":1.923,"15min":1.923},"last":0.40999999999999998}]},{"osd":3,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.57099999999999995},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.49099999999999999,"15min":0.48699999999999999},"min":{"1min":0.34599999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72099999999999997,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.50600000000000001}]},{"osd":4,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.74099999999999999,"15min":0.76000000000000001},"min":{"1min":0.52300000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.999,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.70699999999999996,"15min":0.68300000000000005},"min":{"1min":0.504,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0389999999999999,"5min":1.3,"15min":1.3},"last":0.89700000000000002}]},{"osd":5,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.53500000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0089999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.751,"15min":0.753},"min":{"1min":0.56499999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1000000000000001,"5min":1.97,"15min":1.97},"last":0.71499999999999997}]},{"osd":6,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.76300000000000001,"15min":0.745},"min":{"1min":0.54600000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0169999999999999,"5min":1.272,"15min":1.272},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.77000000000000002,"15min":0.745},"min":{"1min":0.64700000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.04,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.77400000000000002}]},{"osd":7,"last update":"Wed May 22 21:17:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.50600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0129999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.88},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76300000000000001,"15min":0.77900000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.91200000000000003,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.73299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1121,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194191,"statfs":{"total":200000000,"available":198851789,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42499999999999999,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58099999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.255},{"interface":"front","average":{"1min":0.504,"5min":0.51100000000000001,"15min":0.48499999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69199999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.501}]},{"osd":2,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.45400000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76200000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.42799999999999999},"min":{"1min":0.29399999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.623,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.47599999999999998}]},{"osd":3,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45600000000000002,"15min":0.434},"min":{"1min":0.33900000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61699999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53400000000000003,"15min":0.503},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.746,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.58399999999999996}]},{"osd":4,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.71099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1020000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.57099999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.113,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.872}]},{"osd":5,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71899999999999997,"15min":0.70699999999999996},"min":{"1min":0.54200000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99299999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.76900000000000002,"15min":0.73899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.143,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.93799999999999994}]},{"osd":6,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79300000000000004,"15min":0.77300000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1240000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78200000000000003,"15min":0.77200000000000002},"min":{"1min":0.55300000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2090000000000001,"5min":1.5,"15min":1.5},"last":0.85499999999999998}]},{"osd":7,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.75600000000000001,"15min":0.72699999999999998},"min":{"1min":0.57499999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.75},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76900000000000002,"15min":0.75900000000000001},"min":{"1min":0.55900000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.0589999999999999,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.83799999999999997}]}]}],"pool_statfs":[]}} 2024-05-22T21:18:23.946 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:18:23.949 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:18:26.538 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:18:26.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d9600640 1 Processor -- start 2024-05-22T21:18:26.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d9600640 1 -- start start 2024-05-22T21:18:26.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d9600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f48d4158c30 0x7f48d406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:26.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d9600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f48d406a5e0 0x7f48d415b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:26.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d9600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d40ad140 con 0x7f48d4158850 2024-05-22T21:18:26.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d9600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d40cfaa0 con 0x7f48d4158c30 2024-05-22T21:18:26.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d9600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d406f580 con 0x7f48d406a5e0 2024-05-22T21:18:26.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d3400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f48d4158c30 0x7f48d406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:26.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d8c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f48d406a5e0 0x7f48d415b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:26.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d3400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f48d4158c30 0x7f48d406c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:56386/0 (socket says 172.21.15.161:56386) 2024-05-22T21:18:26.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d3400640 1 -- 172.21.15.161:0/3097648396 learned_addr learned my addr 172.21.15.161:0/3097648396 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:26.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d3e00640 1 --1- 172.21.15.161:0/3097648396 >> v1:172.21.15.179:6789/0 conn(0x7f48d4158850 0x7f48d4069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:57682/0 (socket says 172.21.15.161:57682) 2024-05-22T21:18:26.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d3400640 1 -- 172.21.15.161:0/3097648396 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f48d406a5e0 msgr2=0x7f48d415b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:26.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d3400640 1 --2- 172.21.15.161:0/3097648396 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f48d406a5e0 0x7f48d415b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:26.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d3400640 1 -- 172.21.15.161:0/3097648396 >> v1:172.21.15.179:6789/0 conn(0x7f48d4158850 legacy=0x7f48d4069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:26.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d3400640 1 -- 172.21.15.161:0/3097648396 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d406d0d0 con 0x7f48d4158c30 2024-05-22T21:18:26.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.683+0000 7f48d2a00640 1 -- 172.21.15.161:0/3097648396 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4266411559 0 0) 0x7f48d40ad140 con 0x7f48d4158850 2024-05-22T21:18:26.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.687+0000 7f48d3400640 1 --2- 172.21.15.161:0/3097648396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48d4158c30 0x7f48d406c9c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009870 tx=0x7f48c001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82e00eccfc59800c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:26.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.687+0000 7f48d2a00640 1 -- 172.21.15.161:0/3097648396 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c00ac620 con 0x7f48d4158c30 2024-05-22T21:18:26.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.687+0000 7f48d2a00640 1 -- 172.21.15.161:0/3097648396 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c00ac780 con 0x7f48d4158c30 2024-05-22T21:18:26.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.687+0000 7f48d2a00640 1 -- 172.21.15.161:0/3097648396 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c00acb10 con 0x7f48d4158c30 2024-05-22T21:18:26.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.687+0000 7f48d9600640 1 -- 172.21.15.161:0/3097648396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48d4158c30 msgr2=0x7f48d406c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:26.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.687+0000 7f48d9600640 1 --2- 172.21.15.161:0/3097648396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48d4158c30 0x7f48d406c9c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009870 tx=0x7f48c001c4d0 comp rx=0 tx=0).stop 2024-05-22T21:18:26.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.687+0000 7f48d9600640 1 -- 172.21.15.161:0/3097648396 shutdown_connections 2024-05-22T21:18:26.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.687+0000 7f48d9600640 1 --2- 172.21.15.161:0/3097648396 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f48d406a5e0 0x7f48d415b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:26.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.687+0000 7f48d9600640 1 --2- 172.21.15.161:0/3097648396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48d4158c30 0x7f48d406c9c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:26.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.687+0000 7f48d9600640 1 -- 172.21.15.161:0/3097648396 >> 172.21.15.161:0/3097648396 conn(0x7f48d406ba00 msgr2=0x7f48d40b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:26.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.687+0000 7f48d9600640 1 -- 172.21.15.161:0/3097648396 shutdown_connections 2024-05-22T21:18:26.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.687+0000 7f48d9600640 1 -- 172.21.15.161:0/3097648396 wait complete. 2024-05-22T21:18:26.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d9600640 1 Processor -- start 2024-05-22T21:18:26.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d9600640 1 -- start start 2024-05-22T21:18:26.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d9600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d406a5e0 0x7f48d4162fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:26.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d9600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f48d4158850 0x7f48d4149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:26.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d9600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48d40af950 0x7f48d40afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:26.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d9600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40b0320 con 0x7f48d406a5e0 2024-05-22T21:18:26.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d9600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40ad720 con 0x7f48d40af950 2024-05-22T21:18:26.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d9600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40ace30 con 0x7f48d4158850 2024-05-22T21:18:26.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d406a5e0 0x7f48d4162fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:26.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d406a5e0 0x7f48d4162fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50388/0 (socket says 172.21.15.161:50388) 2024-05-22T21:18:26.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d3e00640 1 -- 172.21.15.161:0/3855371453 learned_addr learned my addr 172.21.15.161:0/3855371453 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:26.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d8c00640 1 --2- 172.21.15.161:0/3855371453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48d40af950 0x7f48d40afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:26.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d3400640 1 --2- 172.21.15.161:0/3855371453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f48d4158850 0x7f48d4149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:26.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d3e00640 1 -- 172.21.15.161:0/3855371453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f48d4158850 msgr2=0x7f48d4149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:26.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d3e00640 1 --2- 172.21.15.161:0/3855371453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f48d4158850 0x7f48d4149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:26.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d3e00640 1 -- 172.21.15.161:0/3855371453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48d40af950 msgr2=0x7f48d40afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:26.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d3e00640 1 --2- 172.21.15.161:0/3855371453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48d40af950 0x7f48d40afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:26.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d3e00640 1 -- 172.21.15.161:0/3855371453 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c00094d0 con 0x7f48d406a5e0 2024-05-22T21:18:26.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d8c00640 1 --2- 172.21.15.161:0/3855371453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48d40af950 0x7f48d40afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:26.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d3e00640 1 --2- 172.21.15.161:0/3855371453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d406a5e0 0x7f48d4162fb0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c480 tx=0x7f48c400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:26.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48cbe00640 1 -- 172.21.15.161:0/3855371453 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c40074e0 con 0x7f48d406a5e0 2024-05-22T21:18:26.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48cbe00640 1 -- 172.21.15.161:0/3855371453 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c4007660 con 0x7f48d406a5e0 2024-05-22T21:18:26.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48cbe00640 1 -- 172.21.15.161:0/3855371453 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4004a30 con 0x7f48d406a5e0 2024-05-22T21:18:26.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d9600640 1 -- 172.21.15.161:0/3855371453 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d40b4630 con 0x7f48d406a5e0 2024-05-22T21:18:26.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d9600640 1 -- 172.21.15.161:0/3855371453 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d40ad3b0 con 0x7f48d406a5e0 2024-05-22T21:18:26.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.691+0000 7f48d9600640 1 -- 172.21.15.161:0/3855371453 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d4067070 con 0x7f48d406a5e0 2024-05-22T21:18:26.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.695+0000 7f48cbe00640 1 -- 172.21.15.161:0/3855371453 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f48c4004b90 con 0x7f48d406a5e0 2024-05-22T21:18:26.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.695+0000 7f48cbe00640 1 --2- 172.21.15.161:0/3855371453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f48b40666c0 0x7f48b4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:26.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.695+0000 7f48cbe00640 1 -- 172.21.15.161:0/3855371453 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48c4083260 con 0x7f48d406a5e0 2024-05-22T21:18:26.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.695+0000 7f48d3400640 1 --2- 172.21.15.161:0/3855371453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f48b40666c0 0x7f48b4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:26.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.695+0000 7f48d3400640 1 --2- 172.21.15.161:0/3855371453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f48b40666c0 0x7f48b4068b80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f48c00ae000 tx=0x7f48c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:26.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.699+0000 7f48cbe00640 1 -- 172.21.15.161:0/3855371453 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f48c4050670 con 0x7f48d406a5e0 2024-05-22T21:18:26.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.851+0000 7f48d9600640 1 -- 172.21.15.161:0/3855371453 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f48d406e750 con 0x7f48b40666c0 2024-05-22T21:18:26.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.859+0000 7f48cbe00640 1 -- 172.21.15.161:0/3855371453 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284609 (secure 0 0 0) 0x7f48c00ac8f0 con 0x7f48b40666c0 2024-05-22T21:18:26.864 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:26.865 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:18:26.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.863+0000 7f48d9600640 1 -- 172.21.15.161:0/3855371453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f48b40666c0 msgr2=0x7f48b4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:26.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.863+0000 7f48d9600640 1 --2- 172.21.15.161:0/3855371453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f48b40666c0 0x7f48b4068b80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f48c00ae000 tx=0x7f48c0008040 comp rx=0 tx=0).stop 2024-05-22T21:18:26.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.863+0000 7f48d9600640 1 -- 172.21.15.161:0/3855371453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d406a5e0 msgr2=0x7f48d4162fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:26.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.863+0000 7f48d9600640 1 --2- 172.21.15.161:0/3855371453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d406a5e0 0x7f48d4162fb0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c480 tx=0x7f48c400c940 comp rx=0 tx=0).stop 2024-05-22T21:18:26.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.863+0000 7f48d9600640 1 -- 172.21.15.161:0/3855371453 shutdown_connections 2024-05-22T21:18:26.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.863+0000 7f48d9600640 1 --2- 172.21.15.161:0/3855371453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f48b40666c0 0x7f48b4068b80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:26.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.863+0000 7f48d9600640 1 --2- 172.21.15.161:0/3855371453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48d40af950 0x7f48d40afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:26.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.863+0000 7f48d9600640 1 --2- 172.21.15.161:0/3855371453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f48d4158850 0x7f48d4149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:26.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.863+0000 7f48d9600640 1 --2- 172.21.15.161:0/3855371453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d406a5e0 0x7f48d4162fb0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:26.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.863+0000 7f48d9600640 1 -- 172.21.15.161:0/3855371453 >> 172.21.15.161:0/3855371453 conn(0x7f48d406ba00 msgr2=0x7f48d415d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:26.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.863+0000 7f48d9600640 1 -- 172.21.15.161:0/3855371453 shutdown_connections 2024-05-22T21:18:26.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:26.863+0000 7f48d9600640 1 -- 172.21.15.161:0/3855371453 wait complete. 2024-05-22T21:18:26.884 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-05-22T21:18:26.460860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5519,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556980,"statfs":{"total":1600000000,"available":1594347735,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.516064"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170375+0000","last_change":"2024-05-22T21:18:17.960599+0000","last_active":"2024-05-22T21:18:19.170375+0000","last_peered":"2024-05-22T21:18:19.170375+0000","last_clean":"2024-05-22T21:18:19.170375+0000","last_became_active":"2024-05-22T21:18:17.960246+0000","last_became_peered":"2024-05-22T21:18:17.960246+0000","last_unstale":"2024-05-22T21:18:19.170375+0000","last_undegraded":"2024-05-22T21:18:19.170375+0000","last_fullsized":"2024-05-22T21:18:19.170375+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:54:13.859086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169155+0000","last_change":"2024-05-22T21:18:17.963308+0000","last_active":"2024-05-22T21:18:19.169155+0000","last_peered":"2024-05-22T21:18:19.169155+0000","last_clean":"2024-05-22T21:18:19.169155+0000","last_became_active":"2024-05-22T21:18:17.962108+0000","last_became_peered":"2024-05-22T21:18:17.962108+0000","last_unstale":"2024-05-22T21:18:19.169155+0000","last_undegraded":"2024-05-22T21:18:19.169155+0000","last_fullsized":"2024-05-22T21:18:19.169155+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:25:51.935444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953613+0000","last_change":"2024-05-22T21:18:17.960645+0000","last_active":"2024-05-22T21:18:18.953613+0000","last_peered":"2024-05-22T21:18:18.953613+0000","last_clean":"2024-05-22T21:18:18.953613+0000","last_became_active":"2024-05-22T21:18:17.960427+0000","last_became_peered":"2024-05-22T21:18:17.960427+0000","last_unstale":"2024-05-22T21:18:18.953613+0000","last_undegraded":"2024-05-22T21:18:18.953613+0000","last_fullsized":"2024-05-22T21:18:18.953613+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:52:41.544599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.392835+0000","last_change":"2024-05-22T21:18:17.959284+0000","last_active":"2024-05-22T21:18:19.392835+0000","last_peered":"2024-05-22T21:18:19.392835+0000","last_clean":"2024-05-22T21:18:19.392835+0000","last_became_active":"2024-05-22T21:18:17.957127+0000","last_became_peered":"2024-05-22T21:18:17.957127+0000","last_unstale":"2024-05-22T21:18:19.392835+0000","last_undegraded":"2024-05-22T21:18:19.392835+0000","last_fullsized":"2024-05-22T21:18:19.392835+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:35:51.060507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954426+0000","last_change":"2024-05-22T21:18:17.965991+0000","last_active":"2024-05-22T21:18:18.954426+0000","last_peered":"2024-05-22T21:18:18.954426+0000","last_clean":"2024-05-22T21:18:18.954426+0000","last_became_active":"2024-05-22T21:18:17.965764+0000","last_became_peered":"2024-05-22T21:18:17.965764+0000","last_unstale":"2024-05-22T21:18:18.954426+0000","last_undegraded":"2024-05-22T21:18:18.954426+0000","last_fullsized":"2024-05-22T21:18:18.954426+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:31:06.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393978+0000","last_change":"2024-05-22T21:18:17.953557+0000","last_active":"2024-05-22T21:18:19.393978+0000","last_peered":"2024-05-22T21:18:19.393978+0000","last_clean":"2024-05-22T21:18:19.393978+0000","last_became_active":"2024-05-22T21:18:17.952902+0000","last_became_peered":"2024-05-22T21:18:17.952902+0000","last_unstale":"2024-05-22T21:18:19.393978+0000","last_undegraded":"2024-05-22T21:18:19.393978+0000","last_fullsized":"2024-05-22T21:18:19.393978+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:49.458817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393470+0000","last_change":"2024-05-22T21:18:17.953697+0000","last_active":"2024-05-22T21:18:19.393470+0000","last_peered":"2024-05-22T21:18:19.393470+0000","last_clean":"2024-05-22T21:18:19.393470+0000","last_became_active":"2024-05-22T21:18:17.953466+0000","last_became_peered":"2024-05-22T21:18:17.953466+0000","last_unstale":"2024-05-22T21:18:19.393470+0000","last_undegraded":"2024-05-22T21:18:19.393470+0000","last_fullsized":"2024-05-22T21:18:19.393470+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:37:13.521838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393857+0000","last_change":"2024-05-22T21:18:17.957649+0000","last_active":"2024-05-22T21:18:19.393857+0000","last_peered":"2024-05-22T21:18:19.393857+0000","last_clean":"2024-05-22T21:18:19.393857+0000","last_became_active":"2024-05-22T21:18:17.957409+0000","last_became_peered":"2024-05-22T21:18:17.957409+0000","last_unstale":"2024-05-22T21:18:19.393857+0000","last_undegraded":"2024-05-22T21:18:19.393857+0000","last_fullsized":"2024-05-22T21:18:19.393857+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:22:06.607488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393996+0000","last_change":"2024-05-22T21:18:17.952455+0000","last_active":"2024-05-22T21:18:19.393996+0000","last_peered":"2024-05-22T21:18:19.393996+0000","last_clean":"2024-05-22T21:18:19.393996+0000","last_became_active":"2024-05-22T21:18:17.952241+0000","last_became_peered":"2024-05-22T21:18:17.952241+0000","last_unstale":"2024-05-22T21:18:19.393996+0000","last_undegraded":"2024-05-22T21:18:19.393996+0000","last_fullsized":"2024-05-22T21:18:19.393996+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:09:17.217291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170339+0000","last_change":"2024-05-22T21:18:17.962139+0000","last_active":"2024-05-22T21:18:19.170339+0000","last_peered":"2024-05-22T21:18:19.170339+0000","last_clean":"2024-05-22T21:18:19.170339+0000","last_became_active":"2024-05-22T21:18:17.961250+0000","last_became_peered":"2024-05-22T21:18:17.961250+0000","last_unstale":"2024-05-22T21:18:19.170339+0000","last_undegraded":"2024-05-22T21:18:19.170339+0000","last_fullsized":"2024-05-22T21:18:19.170339+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:56:49.462456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169245+0000","last_change":"2024-05-22T21:18:17.957003+0000","last_active":"2024-05-22T21:18:19.169245+0000","last_peered":"2024-05-22T21:18:19.169245+0000","last_clean":"2024-05-22T21:18:19.169245+0000","last_became_active":"2024-05-22T21:18:17.956399+0000","last_became_peered":"2024-05-22T21:18:17.956399+0000","last_unstale":"2024-05-22T21:18:19.169245+0000","last_undegraded":"2024-05-22T21:18:19.169245+0000","last_fullsized":"2024-05-22T21:18:19.169245+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:20.225993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170530+0000","last_change":"2024-05-22T21:18:17.958838+0000","last_active":"2024-05-22T21:18:19.170530+0000","last_peered":"2024-05-22T21:18:19.170530+0000","last_clean":"2024-05-22T21:18:19.170530+0000","last_became_active":"2024-05-22T21:18:17.956822+0000","last_became_peered":"2024-05-22T21:18:17.956822+0000","last_unstale":"2024-05-22T21:18:19.170530+0000","last_undegraded":"2024-05-22T21:18:19.170530+0000","last_fullsized":"2024-05-22T21:18:19.170530+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:16.013875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953716+0000","last_change":"2024-05-22T21:18:17.959719+0000","last_active":"2024-05-22T21:18:18.953716+0000","last_peered":"2024-05-22T21:18:18.953716+0000","last_clean":"2024-05-22T21:18:18.953716+0000","last_became_active":"2024-05-22T21:18:17.959502+0000","last_became_peered":"2024-05-22T21:18:17.959502+0000","last_unstale":"2024-05-22T21:18:18.953716+0000","last_undegraded":"2024-05-22T21:18:18.953716+0000","last_fullsized":"2024-05-22T21:18:18.953716+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:41:21.219804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.392913+0000","last_change":"2024-05-22T21:18:17.959416+0000","last_active":"2024-05-22T21:18:19.392913+0000","last_peered":"2024-05-22T21:18:19.392913+0000","last_clean":"2024-05-22T21:18:19.392913+0000","last_became_active":"2024-05-22T21:18:17.957403+0000","last_became_peered":"2024-05-22T21:18:17.957403+0000","last_unstale":"2024-05-22T21:18:19.392913+0000","last_undegraded":"2024-05-22T21:18:19.392913+0000","last_fullsized":"2024-05-22T21:18:19.392913+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:59:08.119575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393041+0000","last_change":"2024-05-22T21:18:17.954286+0000","last_active":"2024-05-22T21:18:19.393041+0000","last_peered":"2024-05-22T21:18:19.393041+0000","last_clean":"2024-05-22T21:18:19.393041+0000","last_became_active":"2024-05-22T21:18:17.953720+0000","last_became_peered":"2024-05-22T21:18:17.953720+0000","last_unstale":"2024-05-22T21:18:19.393041+0000","last_undegraded":"2024-05-22T21:18:19.393041+0000","last_fullsized":"2024-05-22T21:18:19.393041+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:41:06.493942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953699+0000","last_change":"2024-05-22T21:18:17.964022+0000","last_active":"2024-05-22T21:18:18.953699+0000","last_peered":"2024-05-22T21:18:18.953699+0000","last_clean":"2024-05-22T21:18:18.953699+0000","last_became_active":"2024-05-22T21:18:17.963173+0000","last_became_peered":"2024-05-22T21:18:17.963173+0000","last_unstale":"2024-05-22T21:18:18.953699+0000","last_undegraded":"2024-05-22T21:18:18.953699+0000","last_fullsized":"2024-05-22T21:18:18.953699+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:46.836261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953976+0000","last_change":"2024-05-22T21:18:17.965200+0000","last_active":"2024-05-22T21:18:18.953976+0000","last_peered":"2024-05-22T21:18:18.953976+0000","last_clean":"2024-05-22T21:18:18.953976+0000","last_became_active":"2024-05-22T21:18:17.964346+0000","last_became_peered":"2024-05-22T21:18:17.964346+0000","last_unstale":"2024-05-22T21:18:18.953976+0000","last_undegraded":"2024-05-22T21:18:18.953976+0000","last_fullsized":"2024-05-22T21:18:18.953976+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:09:17.217291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393281+0000","last_change":"2024-05-22T21:18:17.963044+0000","last_active":"2024-05-22T21:18:19.393281+0000","last_peered":"2024-05-22T21:18:19.393281+0000","last_clean":"2024-05-22T21:18:19.393281+0000","last_became_active":"2024-05-22T21:18:17.961677+0000","last_became_peered":"2024-05-22T21:18:17.961677+0000","last_unstale":"2024-05-22T21:18:19.393281+0000","last_undegraded":"2024-05-22T21:18:19.393281+0000","last_fullsized":"2024-05-22T21:18:19.393281+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:28:09.553272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954454+0000","last_change":"2024-05-22T21:18:17.956830+0000","last_active":"2024-05-22T21:18:18.954454+0000","last_peered":"2024-05-22T21:18:18.954454+0000","last_clean":"2024-05-22T21:18:18.954454+0000","last_became_active":"2024-05-22T21:18:17.956077+0000","last_became_peered":"2024-05-22T21:18:17.956077+0000","last_unstale":"2024-05-22T21:18:18.954454+0000","last_undegraded":"2024-05-22T21:18:18.954454+0000","last_fullsized":"2024-05-22T21:18:18.954454+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:31:06.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393570+0000","last_change":"2024-05-22T21:18:17.962103+0000","last_active":"2024-05-22T21:18:19.393570+0000","last_peered":"2024-05-22T21:18:19.393570+0000","last_clean":"2024-05-22T21:18:19.393570+0000","last_became_active":"2024-05-22T21:18:17.960714+0000","last_became_peered":"2024-05-22T21:18:17.960714+0000","last_unstale":"2024-05-22T21:18:19.393570+0000","last_undegraded":"2024-05-22T21:18:19.393570+0000","last_fullsized":"2024-05-22T21:18:19.393570+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:08:44.201170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393427+0000","last_change":"2024-05-22T21:18:17.958661+0000","last_active":"2024-05-22T21:18:19.393427+0000","last_peered":"2024-05-22T21:18:19.393427+0000","last_clean":"2024-05-22T21:18:19.393427+0000","last_became_active":"2024-05-22T21:18:17.958456+0000","last_became_peered":"2024-05-22T21:18:17.958456+0000","last_unstale":"2024-05-22T21:18:19.393427+0000","last_undegraded":"2024-05-22T21:18:19.393427+0000","last_fullsized":"2024-05-22T21:18:19.393427+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:32:39.995038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169266+0000","last_change":"2024-05-22T21:18:17.963540+0000","last_active":"2024-05-22T21:18:19.169266+0000","last_peered":"2024-05-22T21:18:19.169266+0000","last_clean":"2024-05-22T21:18:19.169266+0000","last_became_active":"2024-05-22T21:18:17.962903+0000","last_became_peered":"2024-05-22T21:18:17.962903+0000","last_unstale":"2024-05-22T21:18:19.169266+0000","last_undegraded":"2024-05-22T21:18:19.169266+0000","last_fullsized":"2024-05-22T21:18:19.169266+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:10:11.105319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170376+0000","last_change":"2024-05-22T21:18:17.966436+0000","last_active":"2024-05-22T21:18:19.170376+0000","last_peered":"2024-05-22T21:18:19.170376+0000","last_clean":"2024-05-22T21:18:19.170376+0000","last_became_active":"2024-05-22T21:18:17.964774+0000","last_became_peered":"2024-05-22T21:18:17.964774+0000","last_unstale":"2024-05-22T21:18:19.170376+0000","last_undegraded":"2024-05-22T21:18:19.170376+0000","last_fullsized":"2024-05-22T21:18:19.170376+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:22.052183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.392989+0000","last_change":"2024-05-22T21:18:17.959671+0000","last_active":"2024-05-22T21:18:19.392989+0000","last_peered":"2024-05-22T21:18:19.392989+0000","last_clean":"2024-05-22T21:18:19.392989+0000","last_became_active":"2024-05-22T21:18:17.957995+0000","last_became_peered":"2024-05-22T21:18:17.957995+0000","last_unstale":"2024-05-22T21:18:19.392989+0000","last_undegraded":"2024-05-22T21:18:19.392989+0000","last_fullsized":"2024-05-22T21:18:19.392989+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:57:39.101800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169181+0000","last_change":"2024-05-22T21:18:17.957318+0000","last_active":"2024-05-22T21:18:19.169181+0000","last_peered":"2024-05-22T21:18:19.169181+0000","last_clean":"2024-05-22T21:18:19.169181+0000","last_became_active":"2024-05-22T21:18:17.956778+0000","last_became_peered":"2024-05-22T21:18:17.956778+0000","last_unstale":"2024-05-22T21:18:19.169181+0000","last_undegraded":"2024-05-22T21:18:19.169181+0000","last_fullsized":"2024-05-22T21:18:19.169181+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:48:16.720449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954340+0000","last_change":"2024-05-22T21:18:17.955832+0000","last_active":"2024-05-22T21:18:18.954340+0000","last_peered":"2024-05-22T21:18:18.954340+0000","last_clean":"2024-05-22T21:18:18.954340+0000","last_became_active":"2024-05-22T21:18:17.955610+0000","last_became_peered":"2024-05-22T21:18:17.955610+0000","last_unstale":"2024-05-22T21:18:18.954340+0000","last_undegraded":"2024-05-22T21:18:18.954340+0000","last_fullsized":"2024-05-22T21:18:18.954340+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:22.052183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954287+0000","last_change":"2024-05-22T21:18:17.966183+0000","last_active":"2024-05-22T21:18:18.954287+0000","last_peered":"2024-05-22T21:18:18.954287+0000","last_clean":"2024-05-22T21:18:18.954287+0000","last_became_active":"2024-05-22T21:18:17.964912+0000","last_became_peered":"2024-05-22T21:18:17.964912+0000","last_unstale":"2024-05-22T21:18:18.954287+0000","last_undegraded":"2024-05-22T21:18:18.954287+0000","last_fullsized":"2024-05-22T21:18:18.954287+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:49.458817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170513+0000","last_change":"2024-05-22T21:18:17.965157+0000","last_active":"2024-05-22T21:18:19.170513+0000","last_peered":"2024-05-22T21:18:19.170513+0000","last_clean":"2024-05-22T21:18:19.170513+0000","last_became_active":"2024-05-22T21:18:17.960663+0000","last_became_peered":"2024-05-22T21:18:17.960663+0000","last_unstale":"2024-05-22T21:18:19.170513+0000","last_undegraded":"2024-05-22T21:18:19.170513+0000","last_fullsized":"2024-05-22T21:18:19.170513+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:49.458817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169172+0000","last_change":"2024-05-22T21:18:17.955463+0000","last_active":"2024-05-22T21:18:19.169172+0000","last_peered":"2024-05-22T21:18:19.169172+0000","last_clean":"2024-05-22T21:18:19.169172+0000","last_became_active":"2024-05-22T21:18:17.955244+0000","last_became_peered":"2024-05-22T21:18:17.955244+0000","last_unstale":"2024-05-22T21:18:19.169172+0000","last_undegraded":"2024-05-22T21:18:19.169172+0000","last_fullsized":"2024-05-22T21:18:19.169172+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:30:05.982663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169273+0000","last_change":"2024-05-22T21:18:17.963537+0000","last_active":"2024-05-22T21:18:19.169273+0000","last_peered":"2024-05-22T21:18:19.169273+0000","last_clean":"2024-05-22T21:18:19.169273+0000","last_became_active":"2024-05-22T21:18:17.960629+0000","last_became_peered":"2024-05-22T21:18:17.960629+0000","last_unstale":"2024-05-22T21:18:19.169273+0000","last_undegraded":"2024-05-22T21:18:19.169273+0000","last_fullsized":"2024-05-22T21:18:19.169273+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:42:08.700957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953865+0000","last_change":"2024-05-22T21:18:17.964234+0000","last_active":"2024-05-22T21:18:18.953865+0000","last_peered":"2024-05-22T21:18:18.953865+0000","last_clean":"2024-05-22T21:18:18.953865+0000","last_became_active":"2024-05-22T21:18:17.963492+0000","last_became_peered":"2024-05-22T21:18:17.963492+0000","last_unstale":"2024-05-22T21:18:18.953865+0000","last_undegraded":"2024-05-22T21:18:18.953865+0000","last_fullsized":"2024-05-22T21:18:18.953865+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:22.295248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170669+0000","last_change":"2024-05-22T21:18:17.960444+0000","last_active":"2024-05-22T21:18:19.170669+0000","last_peered":"2024-05-22T21:18:19.170669+0000","last_clean":"2024-05-22T21:18:19.170669+0000","last_became_active":"2024-05-22T21:18:17.960207+0000","last_became_peered":"2024-05-22T21:18:17.960207+0000","last_unstale":"2024-05-22T21:18:19.170669+0000","last_undegraded":"2024-05-22T21:18:19.170669+0000","last_fullsized":"2024-05-22T21:18:19.170669+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:22.295248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953441+0000","last_change":"2024-05-22T21:18:17.958767+0000","last_active":"2024-05-22T21:18:18.953441+0000","last_peered":"2024-05-22T21:18:18.953441+0000","last_clean":"2024-05-22T21:18:18.953441+0000","last_became_active":"2024-05-22T21:18:17.958481+0000","last_became_peered":"2024-05-22T21:18:17.958481+0000","last_unstale":"2024-05-22T21:18:18.953441+0000","last_undegraded":"2024-05-22T21:18:18.953441+0000","last_fullsized":"2024-05-22T21:18:18.953441+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:23.958727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"102'4","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170722+0000","last_change":"2024-05-22T21:18:17.577456+0000","last_active":"2024-05-22T21:18:19.170722+0000","last_peered":"2024-05-22T21:18:19.170722+0000","last_clean":"2024-05-22T21:18:19.170722+0000","last_became_active":"2024-05-22T21:18:17.576784+0000","last_became_peered":"2024-05-22T21:18:17.576784+0000","last_unstale":"2024-05-22T21:18:19.170722+0000","last_undegraded":"2024-05-22T21:18:19.170722+0000","last_fullsized":"2024-05-22T21:18:19.170722+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:48:15.718335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170650+0000","last_change":"2024-05-22T21:18:17.965724+0000","last_active":"2024-05-22T21:18:19.170650+0000","last_peered":"2024-05-22T21:18:19.170650+0000","last_clean":"2024-05-22T21:18:19.170650+0000","last_became_active":"2024-05-22T21:18:17.962527+0000","last_became_peered":"2024-05-22T21:18:17.962527+0000","last_unstale":"2024-05-22T21:18:19.170650+0000","last_undegraded":"2024-05-22T21:18:19.170650+0000","last_fullsized":"2024-05-22T21:18:19.170650+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:09:17.217291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"102'1","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953680+0000","last_change":"2024-05-22T21:18:17.570132+0000","last_active":"2024-05-22T21:18:18.953680+0000","last_peered":"2024-05-22T21:18:18.953680+0000","last_clean":"2024-05-22T21:18:18.953680+0000","last_became_active":"2024-05-22T21:18:17.567690+0000","last_became_peered":"2024-05-22T21:18:17.567690+0000","last_unstale":"2024-05-22T21:18:18.953680+0000","last_undegraded":"2024-05-22T21:18:18.953680+0000","last_fullsized":"2024-05-22T21:18:18.953680+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:25:50.933329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170500+0000","last_change":"2024-05-22T21:18:17.963395+0000","last_active":"2024-05-22T21:18:19.170500+0000","last_peered":"2024-05-22T21:18:19.170500+0000","last_clean":"2024-05-22T21:18:19.170500+0000","last_became_active":"2024-05-22T21:18:17.963135+0000","last_became_peered":"2024-05-22T21:18:17.963135+0000","last_unstale":"2024-05-22T21:18:19.170500+0000","last_undegraded":"2024-05-22T21:18:19.170500+0000","last_fullsized":"2024-05-22T21:18:19.170500+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:31:06.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954515+0000","last_change":"2024-05-22T21:18:17.569334+0000","last_active":"2024-05-22T21:18:18.954515+0000","last_peered":"2024-05-22T21:18:18.954515+0000","last_clean":"2024-05-22T21:18:18.954515+0000","last_became_active":"2024-05-22T21:18:17.566887+0000","last_became_peered":"2024-05-22T21:18:17.566887+0000","last_unstale":"2024-05-22T21:18:18.954515+0000","last_undegraded":"2024-05-22T21:18:18.954515+0000","last_fullsized":"2024-05-22T21:18:18.954515+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:13:03.234580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954048+0000","last_change":"2024-05-22T21:18:17.959066+0000","last_active":"2024-05-22T21:18:18.954048+0000","last_peered":"2024-05-22T21:18:18.954048+0000","last_clean":"2024-05-22T21:18:18.954048+0000","last_became_active":"2024-05-22T21:18:17.956795+0000","last_became_peered":"2024-05-22T21:18:17.956795+0000","last_unstale":"2024-05-22T21:18:18.954048+0000","last_undegraded":"2024-05-22T21:18:18.954048+0000","last_fullsized":"2024-05-22T21:18:18.954048+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:23.958727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954584+0000","last_change":"2024-05-22T21:18:17.570719+0000","last_active":"2024-05-22T21:18:18.954584+0000","last_peered":"2024-05-22T21:18:18.954584+0000","last_clean":"2024-05-22T21:18:18.954584+0000","last_became_active":"2024-05-22T21:18:17.570489+0000","last_became_peered":"2024-05-22T21:18:17.570489+0000","last_unstale":"2024-05-22T21:18:18.954584+0000","last_undegraded":"2024-05-22T21:18:18.954584+0000","last_fullsized":"2024-05-22T21:18:18.954584+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:58:09.369054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169227+0000","last_change":"2024-05-22T21:18:17.960911+0000","last_active":"2024-05-22T21:18:19.169227+0000","last_peered":"2024-05-22T21:18:19.169227+0000","last_clean":"2024-05-22T21:18:19.169227+0000","last_became_active":"2024-05-22T21:18:17.957746+0000","last_became_peered":"2024-05-22T21:18:17.957746+0000","last_unstale":"2024-05-22T21:18:19.169227+0000","last_undegraded":"2024-05-22T21:18:19.169227+0000","last_fullsized":"2024-05-22T21:18:19.169227+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:58:33.493841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"102'1","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954180+0000","last_change":"2024-05-22T21:18:17.575282+0000","last_active":"2024-05-22T21:18:18.954180+0000","last_peered":"2024-05-22T21:18:18.954180+0000","last_clean":"2024-05-22T21:18:18.954180+0000","last_became_active":"2024-05-22T21:18:17.574110+0000","last_became_peered":"2024-05-22T21:18:17.574110+0000","last_unstale":"2024-05-22T21:18:18.954180+0000","last_undegraded":"2024-05-22T21:18:18.954180+0000","last_fullsized":"2024-05-22T21:18:18.954180+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:48:15.718335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954075+0000","last_change":"2024-05-22T21:18:17.964643+0000","last_active":"2024-05-22T21:18:18.954075+0000","last_peered":"2024-05-22T21:18:18.954075+0000","last_clean":"2024-05-22T21:18:18.954075+0000","last_became_active":"2024-05-22T21:18:17.960412+0000","last_became_peered":"2024-05-22T21:18:17.960412+0000","last_unstale":"2024-05-22T21:18:18.954075+0000","last_undegraded":"2024-05-22T21:18:18.954075+0000","last_fullsized":"2024-05-22T21:18:18.954075+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:56:49.462456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"102'1","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954423+0000","last_change":"2024-05-22T21:18:17.571413+0000","last_active":"2024-05-22T21:18:18.954423+0000","last_peered":"2024-05-22T21:18:18.954423+0000","last_clean":"2024-05-22T21:18:18.954423+0000","last_became_active":"2024-05-22T21:18:17.569088+0000","last_became_peered":"2024-05-22T21:18:17.569088+0000","last_unstale":"2024-05-22T21:18:18.954423+0000","last_undegraded":"2024-05-22T21:18:18.954423+0000","last_fullsized":"2024-05-22T21:18:18.954423+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:19.223879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953862+0000","last_change":"2024-05-22T21:18:17.961734+0000","last_active":"2024-05-22T21:18:18.953862+0000","last_peered":"2024-05-22T21:18:18.953862+0000","last_clean":"2024-05-22T21:18:18.953862+0000","last_became_active":"2024-05-22T21:18:17.961203+0000","last_became_peered":"2024-05-22T21:18:17.961203+0000","last_unstale":"2024-05-22T21:18:18.953862+0000","last_undegraded":"2024-05-22T21:18:18.953862+0000","last_fullsized":"2024-05-22T21:18:18.953862+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:49.458817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954654+0000","last_change":"2024-05-22T21:18:17.567816+0000","last_active":"2024-05-22T21:18:18.954654+0000","last_peered":"2024-05-22T21:18:18.954654+0000","last_clean":"2024-05-22T21:18:18.954654+0000","last_became_active":"2024-05-22T21:18:17.567601+0000","last_became_peered":"2024-05-22T21:18:17.567601+0000","last_unstale":"2024-05-22T21:18:18.954654+0000","last_undegraded":"2024-05-22T21:18:18.954654+0000","last_fullsized":"2024-05-22T21:18:18.954654+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:54:12.856971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393016+0000","last_change":"2024-05-22T21:18:17.954207+0000","last_active":"2024-05-22T21:18:19.393016+0000","last_peered":"2024-05-22T21:18:19.393016+0000","last_clean":"2024-05-22T21:18:19.393016+0000","last_became_active":"2024-05-22T21:18:17.953977+0000","last_became_peered":"2024-05-22T21:18:17.953977+0000","last_unstale":"2024-05-22T21:18:19.393016+0000","last_undegraded":"2024-05-22T21:18:19.393016+0000","last_fullsized":"2024-05-22T21:18:19.393016+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:41:23.892969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169341+0000","last_change":"2024-05-22T21:18:17.566911+0000","last_active":"2024-05-22T21:18:19.169341+0000","last_peered":"2024-05-22T21:18:19.169341+0000","last_clean":"2024-05-22T21:18:19.169341+0000","last_became_active":"2024-05-22T21:18:17.566696+0000","last_became_peered":"2024-05-22T21:18:17.566696+0000","last_unstale":"2024-05-22T21:18:19.169341+0000","last_undegraded":"2024-05-22T21:18:19.169341+0000","last_fullsized":"2024-05-22T21:18:19.169341+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:18:20.507884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393137+0000","last_change":"2024-05-22T21:18:17.955339+0000","last_active":"2024-05-22T21:18:19.393137+0000","last_peered":"2024-05-22T21:18:19.393137+0000","last_clean":"2024-05-22T21:18:19.393137+0000","last_became_active":"2024-05-22T21:18:17.955173+0000","last_became_peered":"2024-05-22T21:18:17.955173+0000","last_unstale":"2024-05-22T21:18:19.393137+0000","last_undegraded":"2024-05-22T21:18:19.393137+0000","last_fullsized":"2024-05-22T21:18:19.393137+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:06:36.975586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170634+0000","last_change":"2024-05-22T21:18:17.574214+0000","last_active":"2024-05-22T21:18:19.170634+0000","last_peered":"2024-05-22T21:18:19.170634+0000","last_clean":"2024-05-22T21:18:19.170634+0000","last_became_active":"2024-05-22T21:18:17.573769+0000","last_became_peered":"2024-05-22T21:18:17.573769+0000","last_unstale":"2024-05-22T21:18:19.170634+0000","last_undegraded":"2024-05-22T21:18:19.170634+0000","last_fullsized":"2024-05-22T21:18:19.170634+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:10:10.103204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.394075+0000","last_change":"2024-05-22T21:18:17.953706+0000","last_active":"2024-05-22T21:18:19.394075+0000","last_peered":"2024-05-22T21:18:19.394075+0000","last_clean":"2024-05-22T21:18:19.394075+0000","last_became_active":"2024-05-22T21:18:17.953296+0000","last_became_peered":"2024-05-22T21:18:17.953296+0000","last_unstale":"2024-05-22T21:18:19.394075+0000","last_undegraded":"2024-05-22T21:18:19.394075+0000","last_fullsized":"2024-05-22T21:18:19.394075+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:16.013875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170743+0000","last_change":"2024-05-22T21:18:17.572117+0000","last_active":"2024-05-22T21:18:19.170743+0000","last_peered":"2024-05-22T21:18:19.170743+0000","last_clean":"2024-05-22T21:18:19.170743+0000","last_became_active":"2024-05-22T21:18:17.571601+0000","last_became_peered":"2024-05-22T21:18:17.571601+0000","last_unstale":"2024-05-22T21:18:19.170743+0000","last_undegraded":"2024-05-22T21:18:19.170743+0000","last_fullsized":"2024-05-22T21:18:19.170743+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:25:50.933329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954263+0000","last_change":"2024-05-22T21:18:17.957084+0000","last_active":"2024-05-22T21:18:18.954263+0000","last_peered":"2024-05-22T21:18:18.954263+0000","last_clean":"2024-05-22T21:18:18.954263+0000","last_became_active":"2024-05-22T21:18:17.956592+0000","last_became_peered":"2024-05-22T21:18:17.956592+0000","last_unstale":"2024-05-22T21:18:18.954263+0000","last_undegraded":"2024-05-22T21:18:18.954263+0000","last_fullsized":"2024-05-22T21:18:18.954263+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:22.052183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"102'3","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170828+0000","last_change":"2024-05-22T21:18:17.577310+0000","last_active":"2024-05-22T21:18:19.170828+0000","last_peered":"2024-05-22T21:18:19.170828+0000","last_clean":"2024-05-22T21:18:19.170828+0000","last_became_active":"2024-05-22T21:18:17.576229+0000","last_became_peered":"2024-05-22T21:18:17.576229+0000","last_unstale":"2024-05-22T21:18:19.170828+0000","last_undegraded":"2024-05-22T21:18:19.170828+0000","last_fullsized":"2024-05-22T21:18:19.170828+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:27:43.610883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954225+0000","last_change":"2024-05-22T21:18:17.962890+0000","last_active":"2024-05-22T21:18:18.954225+0000","last_peered":"2024-05-22T21:18:18.954225+0000","last_clean":"2024-05-22T21:18:18.954225+0000","last_became_active":"2024-05-22T21:18:17.962655+0000","last_became_peered":"2024-05-22T21:18:17.962655+0000","last_unstale":"2024-05-22T21:18:18.954225+0000","last_undegraded":"2024-05-22T21:18:18.954225+0000","last_fullsized":"2024-05-22T21:18:18.954225+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:09:27.584040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"102'6","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.394098+0000","last_change":"2024-05-22T21:18:17.570233+0000","last_active":"2024-05-22T21:18:19.394098+0000","last_peered":"2024-05-22T21:18:19.394098+0000","last_clean":"2024-05-22T21:18:19.394098+0000","last_became_active":"2024-05-22T21:18:17.569591+0000","last_became_peered":"2024-05-22T21:18:17.569591+0000","last_unstale":"2024-05-22T21:18:19.394098+0000","last_undegraded":"2024-05-22T21:18:19.394098+0000","last_fullsized":"2024-05-22T21:18:19.394098+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:35:29.469510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954400+0000","last_change":"2024-05-22T21:18:17.960697+0000","last_active":"2024-05-22T21:18:18.954400+0000","last_peered":"2024-05-22T21:18:18.954400+0000","last_clean":"2024-05-22T21:18:18.954400+0000","last_became_active":"2024-05-22T21:18:17.960207+0000","last_became_peered":"2024-05-22T21:18:17.960207+0000","last_unstale":"2024-05-22T21:18:18.954400+0000","last_undegraded":"2024-05-22T21:18:18.954400+0000","last_fullsized":"2024-05-22T21:18:18.954400+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:49.458817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169388+0000","last_change":"2024-05-22T21:18:17.571909+0000","last_active":"2024-05-22T21:18:19.169388+0000","last_peered":"2024-05-22T21:18:19.169388+0000","last_clean":"2024-05-22T21:18:19.169388+0000","last_became_active":"2024-05-22T21:18:17.571512+0000","last_became_peered":"2024-05-22T21:18:17.571512+0000","last_unstale":"2024-05-22T21:18:19.169388+0000","last_undegraded":"2024-05-22T21:18:19.169388+0000","last_fullsized":"2024-05-22T21:18:19.169388+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:36:46.664343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953954+0000","last_change":"2024-05-22T21:18:17.961953+0000","last_active":"2024-05-22T21:18:18.953954+0000","last_peered":"2024-05-22T21:18:18.953954+0000","last_clean":"2024-05-22T21:18:18.953954+0000","last_became_active":"2024-05-22T21:18:17.961516+0000","last_became_peered":"2024-05-22T21:18:17.961516+0000","last_unstale":"2024-05-22T21:18:18.953954+0000","last_undegraded":"2024-05-22T21:18:18.953954+0000","last_fullsized":"2024-05-22T21:18:18.953954+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:16.013875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393111+0000","last_change":"2024-05-22T21:18:17.572880+0000","last_active":"2024-05-22T21:18:19.393111+0000","last_peered":"2024-05-22T21:18:19.393111+0000","last_clean":"2024-05-22T21:18:19.393111+0000","last_became_active":"2024-05-22T21:18:17.572655+0000","last_became_peered":"2024-05-22T21:18:17.572655+0000","last_unstale":"2024-05-22T21:18:19.393111+0000","last_undegraded":"2024-05-22T21:18:19.393111+0000","last_fullsized":"2024-05-22T21:18:19.393111+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:21.050069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169092+0000","last_change":"2024-05-22T21:18:17.957857+0000","last_active":"2024-05-22T21:18:19.169092+0000","last_peered":"2024-05-22T21:18:19.169092+0000","last_clean":"2024-05-22T21:18:19.169092+0000","last_became_active":"2024-05-22T21:18:17.957491+0000","last_became_peered":"2024-05-22T21:18:17.957491+0000","last_unstale":"2024-05-22T21:18:19.169092+0000","last_undegraded":"2024-05-22T21:18:19.169092+0000","last_fullsized":"2024-05-22T21:18:19.169092+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:14:43.052939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393981+0000","last_change":"2024-05-22T21:18:17.569729+0000","last_active":"2024-05-22T21:18:19.393981+0000","last_peered":"2024-05-22T21:18:19.393981+0000","last_clean":"2024-05-22T21:18:19.393981+0000","last_became_active":"2024-05-22T21:18:17.567831+0000","last_became_peered":"2024-05-22T21:18:17.567831+0000","last_unstale":"2024-05-22T21:18:19.393981+0000","last_undegraded":"2024-05-22T21:18:19.393981+0000","last_fullsized":"2024-05-22T21:18:19.393981+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:42:10.010243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169360+0000","last_change":"2024-05-22T21:18:17.963428+0000","last_active":"2024-05-22T21:18:19.169360+0000","last_peered":"2024-05-22T21:18:19.169360+0000","last_clean":"2024-05-22T21:18:19.169360+0000","last_became_active":"2024-05-22T21:18:17.962429+0000","last_became_peered":"2024-05-22T21:18:17.962429+0000","last_unstale":"2024-05-22T21:18:19.169360+0000","last_undegraded":"2024-05-22T21:18:19.169360+0000","last_fullsized":"2024-05-22T21:18:19.169360+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:19:44.456977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"102'8","reported_seq":19,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954520+0000","last_change":"2024-05-22T21:18:17.570044+0000","last_active":"2024-05-22T21:18:18.954520+0000","last_peered":"2024-05-22T21:18:18.954520+0000","last_clean":"2024-05-22T21:18:18.954520+0000","last_became_active":"2024-05-22T21:18:17.569796+0000","last_became_peered":"2024-05-22T21:18:17.569796+0000","last_unstale":"2024-05-22T21:18:18.954520+0000","last_undegraded":"2024-05-22T21:18:18.954520+0000","last_fullsized":"2024-05-22T21:18:18.954520+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:53:59.583140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954136+0000","last_change":"2024-05-22T21:18:17.957133+0000","last_active":"2024-05-22T21:18:18.954136+0000","last_peered":"2024-05-22T21:18:18.954136+0000","last_clean":"2024-05-22T21:18:18.954136+0000","last_became_active":"2024-05-22T21:18:17.954295+0000","last_became_peered":"2024-05-22T21:18:17.954295+0000","last_unstale":"2024-05-22T21:18:18.954136+0000","last_undegraded":"2024-05-22T21:18:18.954136+0000","last_fullsized":"2024-05-22T21:18:18.954136+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:45:07.161984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"102'1","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170585+0000","last_change":"2024-05-22T21:18:17.576505+0000","last_active":"2024-05-22T21:18:19.170585+0000","last_peered":"2024-05-22T21:18:19.170585+0000","last_clean":"2024-05-22T21:18:19.170585+0000","last_became_active":"2024-05-22T21:18:17.574374+0000","last_became_peered":"2024-05-22T21:18:17.574374+0000","last_unstale":"2024-05-22T21:18:19.170585+0000","last_undegraded":"2024-05-22T21:18:19.170585+0000","last_fullsized":"2024-05-22T21:18:19.170585+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:06:41.500056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170443+0000","last_change":"2024-05-22T21:18:17.962646+0000","last_active":"2024-05-22T21:18:19.170443+0000","last_peered":"2024-05-22T21:18:19.170443+0000","last_clean":"2024-05-22T21:18:19.170443+0000","last_became_active":"2024-05-22T21:18:17.962413+0000","last_became_peered":"2024-05-22T21:18:17.962413+0000","last_unstale":"2024-05-22T21:18:19.170443+0000","last_undegraded":"2024-05-22T21:18:19.170443+0000","last_fullsized":"2024-05-22T21:18:19.170443+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:09:27.584040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954565+0000","last_change":"2024-05-22T21:18:17.573846+0000","last_active":"2024-05-22T21:18:18.954565+0000","last_peered":"2024-05-22T21:18:18.954565+0000","last_clean":"2024-05-22T21:18:18.954565+0000","last_became_active":"2024-05-22T21:18:17.573596+0000","last_became_peered":"2024-05-22T21:18:17.573596+0000","last_unstale":"2024-05-22T21:18:18.954565+0000","last_undegraded":"2024-05-22T21:18:18.954565+0000","last_fullsized":"2024-05-22T21:18:18.954565+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:45.834147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953953+0000","last_change":"2024-05-22T21:18:17.966432+0000","last_active":"2024-05-22T21:18:18.953953+0000","last_peered":"2024-05-22T21:18:18.953953+0000","last_clean":"2024-05-22T21:18:18.953953+0000","last_became_active":"2024-05-22T21:18:17.966172+0000","last_became_peered":"2024-05-22T21:18:17.966172+0000","last_unstale":"2024-05-22T21:18:18.953953+0000","last_undegraded":"2024-05-22T21:18:18.953953+0000","last_fullsized":"2024-05-22T21:18:18.953953+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:16.013875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954299+0000","last_change":"2024-05-22T21:18:17.572905+0000","last_active":"2024-05-22T21:18:18.954299+0000","last_peered":"2024-05-22T21:18:18.954299+0000","last_clean":"2024-05-22T21:18:18.954299+0000","last_became_active":"2024-05-22T21:18:17.572686+0000","last_became_peered":"2024-05-22T21:18:17.572686+0000","last_unstale":"2024-05-22T21:18:18.954299+0000","last_undegraded":"2024-05-22T21:18:18.954299+0000","last_fullsized":"2024-05-22T21:18:18.954299+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:36.130802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393070+0000","last_change":"2024-05-22T21:18:17.959542+0000","last_active":"2024-05-22T21:18:19.393070+0000","last_peered":"2024-05-22T21:18:19.393070+0000","last_clean":"2024-05-22T21:18:19.393070+0000","last_became_active":"2024-05-22T21:18:17.957710+0000","last_became_peered":"2024-05-22T21:18:17.957710+0000","last_unstale":"2024-05-22T21:18:19.393070+0000","last_undegraded":"2024-05-22T21:18:19.393070+0000","last_fullsized":"2024-05-22T21:18:19.393070+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:48:21.058352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393023+0000","last_change":"2024-05-22T21:18:17.565636+0000","last_active":"2024-05-22T21:18:19.393023+0000","last_peered":"2024-05-22T21:18:19.393023+0000","last_clean":"2024-05-22T21:18:19.393023+0000","last_became_active":"2024-05-22T21:18:17.565314+0000","last_became_peered":"2024-05-22T21:18:17.565314+0000","last_unstale":"2024-05-22T21:18:19.393023+0000","last_undegraded":"2024-05-22T21:18:19.393023+0000","last_fullsized":"2024-05-22T21:18:19.393023+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:03:48.456703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393948+0000","last_change":"2024-05-22T21:18:17.955262+0000","last_active":"2024-05-22T21:18:19.393948+0000","last_peered":"2024-05-22T21:18:19.393948+0000","last_clean":"2024-05-22T21:18:19.393948+0000","last_became_active":"2024-05-22T21:18:17.955047+0000","last_became_peered":"2024-05-22T21:18:17.955047+0000","last_unstale":"2024-05-22T21:18:19.393948+0000","last_undegraded":"2024-05-22T21:18:19.393948+0000","last_fullsized":"2024-05-22T21:18:19.393948+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:22.295248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393142+0000","last_change":"2024-05-22T21:18:17.567966+0000","last_active":"2024-05-22T21:18:19.393142+0000","last_peered":"2024-05-22T21:18:19.393142+0000","last_clean":"2024-05-22T21:18:19.393142+0000","last_became_active":"2024-05-22T21:18:17.567024+0000","last_became_peered":"2024-05-22T21:18:17.567024+0000","last_unstale":"2024-05-22T21:18:19.393142+0000","last_undegraded":"2024-05-22T21:18:19.393142+0000","last_fullsized":"2024-05-22T21:18:19.393142+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:08:28.543920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393481+0000","last_change":"2024-05-22T21:18:17.953096+0000","last_active":"2024-05-22T21:18:19.393481+0000","last_peered":"2024-05-22T21:18:19.393481+0000","last_clean":"2024-05-22T21:18:19.393481+0000","last_became_active":"2024-05-22T21:18:17.952744+0000","last_became_peered":"2024-05-22T21:18:17.952744+0000","last_unstale":"2024-05-22T21:18:19.393481+0000","last_undegraded":"2024-05-22T21:18:19.393481+0000","last_fullsized":"2024-05-22T21:18:19.393481+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:21:00.882050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393609+0000","last_change":"2024-05-22T21:18:17.571043+0000","last_active":"2024-05-22T21:18:19.393609+0000","last_peered":"2024-05-22T21:18:19.393609+0000","last_clean":"2024-05-22T21:18:19.393609+0000","last_became_active":"2024-05-22T21:18:17.570750+0000","last_became_peered":"2024-05-22T21:18:17.570750+0000","last_unstale":"2024-05-22T21:18:19.393609+0000","last_undegraded":"2024-05-22T21:18:19.393609+0000","last_fullsized":"2024-05-22T21:18:19.393609+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:41:09.473523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393978+0000","last_change":"2024-05-22T21:18:17.961201+0000","last_active":"2024-05-22T21:18:19.393978+0000","last_peered":"2024-05-22T21:18:19.393978+0000","last_clean":"2024-05-22T21:18:19.393978+0000","last_became_active":"2024-05-22T21:18:17.960963+0000","last_became_peered":"2024-05-22T21:18:17.960963+0000","last_unstale":"2024-05-22T21:18:19.393978+0000","last_undegraded":"2024-05-22T21:18:19.393978+0000","last_fullsized":"2024-05-22T21:18:19.393978+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:56:49.462456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393183+0000","last_change":"2024-05-22T21:18:17.569485+0000","last_active":"2024-05-22T21:18:19.393183+0000","last_peered":"2024-05-22T21:18:19.393183+0000","last_clean":"2024-05-22T21:18:19.393183+0000","last_became_active":"2024-05-22T21:18:17.569276+0000","last_became_peered":"2024-05-22T21:18:17.569276+0000","last_unstale":"2024-05-22T21:18:19.393183+0000","last_undegraded":"2024-05-22T21:18:19.393183+0000","last_fullsized":"2024-05-22T21:18:19.393183+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:45:06.159869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954330+0000","last_change":"2024-05-22T21:18:17.957705+0000","last_active":"2024-05-22T21:18:18.954330+0000","last_peered":"2024-05-22T21:18:18.954330+0000","last_clean":"2024-05-22T21:18:18.954330+0000","last_became_active":"2024-05-22T21:18:17.957268+0000","last_became_peered":"2024-05-22T21:18:17.957268+0000","last_unstale":"2024-05-22T21:18:18.954330+0000","last_undegraded":"2024-05-22T21:18:18.954330+0000","last_fullsized":"2024-05-22T21:18:18.954330+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:13:04.236694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"102'2","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169584+0000","last_change":"2024-05-22T21:18:17.570186+0000","last_active":"2024-05-22T21:18:19.169584+0000","last_peered":"2024-05-22T21:18:19.169584+0000","last_clean":"2024-05-22T21:18:19.169584+0000","last_became_active":"2024-05-22T21:18:17.569973+0000","last_became_peered":"2024-05-22T21:18:17.569973+0000","last_unstale":"2024-05-22T21:18:19.169584+0000","last_undegraded":"2024-05-22T21:18:19.169584+0000","last_fullsized":"2024-05-22T21:18:19.169584+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:52:12.503228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954075+0000","last_change":"2024-05-22T21:18:17.958467+0000","last_active":"2024-05-22T21:18:18.954075+0000","last_peered":"2024-05-22T21:18:18.954075+0000","last_clean":"2024-05-22T21:18:18.954075+0000","last_became_active":"2024-05-22T21:18:17.958201+0000","last_became_peered":"2024-05-22T21:18:17.958201+0000","last_unstale":"2024-05-22T21:18:18.954075+0000","last_undegraded":"2024-05-22T21:18:18.954075+0000","last_fullsized":"2024-05-22T21:18:18.954075+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:05:16.362966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"103'1","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:21.421325+0000","last_change":"2024-05-22T21:18:17.570492+0000","last_active":"2024-05-22T21:18:21.421325+0000","last_peered":"2024-05-22T21:18:21.421325+0000","last_clean":"2024-05-22T21:18:21.421325+0000","last_became_active":"2024-05-22T21:18:17.570091+0000","last_became_peered":"2024-05-22T21:18:17.570091+0000","last_unstale":"2024-05-22T21:18:21.421325+0000","last_undegraded":"2024-05-22T21:18:21.421325+0000","last_fullsized":"2024-05-22T21:18:21.421325+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:30:19.397462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170478+0000","last_change":"2024-05-22T21:18:17.964113+0000","last_active":"2024-05-22T21:18:19.170478+0000","last_peered":"2024-05-22T21:18:19.170478+0000","last_clean":"2024-05-22T21:18:19.170478+0000","last_became_active":"2024-05-22T21:18:17.961742+0000","last_became_peered":"2024-05-22T21:18:17.961742+0000","last_unstale":"2024-05-22T21:18:19.170478+0000","last_undegraded":"2024-05-22T21:18:19.170478+0000","last_fullsized":"2024-05-22T21:18:19.170478+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:46.836261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170677+0000","last_change":"2024-05-22T21:18:17.572676+0000","last_active":"2024-05-22T21:18:19.170677+0000","last_peered":"2024-05-22T21:18:19.170677+0000","last_clean":"2024-05-22T21:18:19.170677+0000","last_became_active":"2024-05-22T21:18:17.572431+0000","last_became_peered":"2024-05-22T21:18:17.572431+0000","last_unstale":"2024-05-22T21:18:19.170677+0000","last_undegraded":"2024-05-22T21:18:19.170677+0000","last_fullsized":"2024-05-22T21:18:19.170677+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:58:32.491726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953576+0000","last_change":"2024-05-22T21:18:17.961274+0000","last_active":"2024-05-22T21:18:18.953576+0000","last_peered":"2024-05-22T21:18:18.953576+0000","last_clean":"2024-05-22T21:18:18.953576+0000","last_became_active":"2024-05-22T21:18:17.961038+0000","last_became_peered":"2024-05-22T21:18:17.961038+0000","last_unstale":"2024-05-22T21:18:18.953576+0000","last_undegraded":"2024-05-22T21:18:18.953576+0000","last_fullsized":"2024-05-22T21:18:18.953576+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:22:06.607488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170614+0000","last_change":"2024-05-22T21:18:17.576592+0000","last_active":"2024-05-22T21:18:19.170614+0000","last_peered":"2024-05-22T21:18:19.170614+0000","last_clean":"2024-05-22T21:18:19.170614+0000","last_became_active":"2024-05-22T21:18:17.576366+0000","last_became_peered":"2024-05-22T21:18:17.576366+0000","last_unstale":"2024-05-22T21:18:19.170614+0000","last_undegraded":"2024-05-22T21:18:19.170614+0000","last_fullsized":"2024-05-22T21:18:19.170614+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:19.223879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.393911+0000","last_change":"2024-05-22T21:18:17.956103+0000","last_active":"2024-05-22T21:18:19.393911+0000","last_peered":"2024-05-22T21:18:19.393911+0000","last_clean":"2024-05-22T21:18:19.393911+0000","last_became_active":"2024-05-22T21:18:17.955874+0000","last_became_peered":"2024-05-22T21:18:17.955874+0000","last_unstale":"2024-05-22T21:18:19.393911+0000","last_undegraded":"2024-05-22T21:18:19.393911+0000","last_fullsized":"2024-05-22T21:18:19.393911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:09:27.584040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954657+0000","last_change":"2024-05-22T21:18:17.573273+0000","last_active":"2024-05-22T21:18:18.954657+0000","last_peered":"2024-05-22T21:18:18.954657+0000","last_clean":"2024-05-22T21:18:18.954657+0000","last_became_active":"2024-05-22T21:18:17.572717+0000","last_became_peered":"2024-05-22T21:18:17.572717+0000","last_unstale":"2024-05-22T21:18:18.954657+0000","last_undegraded":"2024-05-22T21:18:18.954657+0000","last_fullsized":"2024-05-22T21:18:18.954657+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:21.293134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954495+0000","last_change":"2024-05-22T21:18:17.961113+0000","last_active":"2024-05-22T21:18:18.954495+0000","last_peered":"2024-05-22T21:18:18.954495+0000","last_clean":"2024-05-22T21:18:18.954495+0000","last_became_active":"2024-05-22T21:18:17.960493+0000","last_became_peered":"2024-05-22T21:18:17.960493+0000","last_unstale":"2024-05-22T21:18:18.954495+0000","last_undegraded":"2024-05-22T21:18:18.954495+0000","last_fullsized":"2024-05-22T21:18:18.954495+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:21:16.013875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954380+0000","last_change":"2024-05-22T21:18:17.575482+0000","last_active":"2024-05-22T21:18:18.954380+0000","last_peered":"2024-05-22T21:18:18.954380+0000","last_clean":"2024-05-22T21:18:18.954380+0000","last_became_active":"2024-05-22T21:18:17.574884+0000","last_became_peered":"2024-05-22T21:18:17.574884+0000","last_unstale":"2024-05-22T21:18:18.954380+0000","last_undegraded":"2024-05-22T21:18:18.954380+0000","last_fullsized":"2024-05-22T21:18:18.954380+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:06:41.500056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953592+0000","last_change":"2024-05-22T21:18:17.958011+0000","last_active":"2024-05-22T21:18:18.953592+0000","last_peered":"2024-05-22T21:18:18.953592+0000","last_clean":"2024-05-22T21:18:18.953592+0000","last_became_active":"2024-05-22T21:18:17.957625+0000","last_became_peered":"2024-05-22T21:18:17.957625+0000","last_unstale":"2024-05-22T21:18:18.953592+0000","last_undegraded":"2024-05-22T21:18:18.953592+0000","last_fullsized":"2024-05-22T21:18:18.953592+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:31:06.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170872+0000","last_change":"2024-05-22T21:18:17.577139+0000","last_active":"2024-05-22T21:18:19.170872+0000","last_peered":"2024-05-22T21:18:19.170872+0000","last_clean":"2024-05-22T21:18:19.170872+0000","last_became_active":"2024-05-22T21:18:17.573999+0000","last_became_peered":"2024-05-22T21:18:17.573999+0000","last_unstale":"2024-05-22T21:18:19.170872+0000","last_undegraded":"2024-05-22T21:18:19.170872+0000","last_fullsized":"2024-05-22T21:18:19.170872+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:42:07.698843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.170780+0000","last_change":"2024-05-22T21:18:17.962821+0000","last_active":"2024-05-22T21:18:19.170780+0000","last_peered":"2024-05-22T21:18:19.170780+0000","last_clean":"2024-05-22T21:18:19.170780+0000","last_became_active":"2024-05-22T21:18:17.958435+0000","last_became_peered":"2024-05-22T21:18:17.958435+0000","last_unstale":"2024-05-22T21:18:19.170780+0000","last_undegraded":"2024-05-22T21:18:19.170780+0000","last_fullsized":"2024-05-22T21:18:19.170780+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:07:23.958727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.953772+0000","last_change":"2024-05-22T21:18:17.572361+0000","last_active":"2024-05-22T21:18:18.953772+0000","last_peered":"2024-05-22T21:18:18.953772+0000","last_clean":"2024-05-22T21:18:18.953772+0000","last_became_active":"2024-05-22T21:18:17.572121+0000","last_became_peered":"2024-05-22T21:18:17.572121+0000","last_unstale":"2024-05-22T21:18:18.953772+0000","last_undegraded":"2024-05-22T21:18:18.953772+0000","last_fullsized":"2024-05-22T21:18:18.953772+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:10:10.103204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:19.169493+0000","last_change":"2024-05-22T21:18:17.959934+0000","last_active":"2024-05-22T21:18:19.169493+0000","last_peered":"2024-05-22T21:18:19.169493+0000","last_clean":"2024-05-22T21:18:19.169493+0000","last_became_active":"2024-05-22T21:18:17.959654+0000","last_became_peered":"2024-05-22T21:18:17.959654+0000","last_unstale":"2024-05-22T21:18:19.169493+0000","last_undegraded":"2024-05-22T21:18:19.169493+0000","last_fullsized":"2024-05-22T21:18:19.169493+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:17.552828+0000","last_clean_scrub_stamp":"2024-05-22T21:18:17.552828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:06:42.502171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"102'1","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-22T21:18:18.954428+0000","last_change":"2024-05-22T21:18:17.575307+0000","last_active":"2024-05-22T21:18:18.954428+0000","last_peered":"2024-05-22T21:18:18.954428+0000","last_clean":"2024-05-22T21:18:18.954428+0000","last_became_active":"2024-05-22T21:18:17.574793+0000","last_became_peered":"2024-05-22T21:18:17.574793+0000","last_unstale":"2024-05-22T21:18:18.954428+0000","last_undegraded":"2024-05-22T21:18:18.954428+0000","last_fullsized":"2024-05-22T21:18:18.954428+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:18:16.550714+0000","last_clean_scrub_stamp":"2024-05-22T21:18:16.550714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:34:20.054587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":248,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-22T21:18:17.947888+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:18:17.947888+0000","last_peered":"2024-05-22T21:18:17.947888+0000","last_clean":"2024-05-22T21:18:17.947888+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:18:17.947888+0000","last_undegraded":"2024-05-22T21:18:17.947888+0000","last_fullsized":"2024-05-22T21:18:17.947888+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1121,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194190,"statfs":{"total":200000000,"available":198851134,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70299999999999996,"15min":0.754},"min":{"1min":0.40000000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93700000000000006,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88300000000000001,"5min":1.889,"15min":1.889},"last":0.51000000000000001}]},{"osd":1,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72699999999999998,"15min":0.77000000000000002},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89500000000000002,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71399999999999997,"15min":0.76000000000000001},"min":{"1min":0.47899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88600000000000001,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.57199999999999995}]},{"osd":2,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69299999999999995,"15min":0.69999999999999996},"min":{"1min":0.503,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81899999999999995,"5min":1.923,"15min":1.923},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.74399999999999999,"15min":0.72699999999999998},"min":{"1min":0.47399999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86099999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.72599999999999998}]},{"osd":3,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.69599999999999995},"min":{"1min":0.53900000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.90600000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.752,"15min":0.79100000000000004},"min":{"1min":0.505,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88600000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.55200000000000005}]},{"osd":4,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.437,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.92000000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.621,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88700000000000001,"5min":1.018,"15min":1.018},"last":0.64900000000000002}]},{"osd":5,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63900000000000001,"15min":0.65500000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84899999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64800000000000002,"15min":0.64200000000000002},"min":{"1min":0.47899999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91000000000000003,"5min":0.996,"15min":0.996},"last":0.69899999999999995}]},{"osd":6,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.71199999999999997},"min":{"1min":0.54700000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85799999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67400000000000004,"15min":0.64400000000000002},"min":{"1min":0.504,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.68700000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194766,"statfs":{"total":200000000,"available":199440878,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.49299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70599999999999996,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56100000000000005}]},{"osd":1,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95899999999999996,"5min":1.107,"15min":1.107},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.034,"5min":1.081,"15min":1.081},"last":0.71099999999999997}]},{"osd":2,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78300000000000003,"15min":0.75900000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.746,"15min":0.73899999999999999},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.748,"15min":0.74199999999999999},"min":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0249999999999999,"5min":1.415,"15min":1.415},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.004,"5min":1.788,"15min":1.788},"last":0.62}]},{"osd":4,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.41699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.89900000000000002,"5min":1.006,"15min":1.006},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95899999999999996,"5min":2.718,"15min":2.718},"last":0.5}]},{"osd":5,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.745,"15min":0.70299999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81799999999999995,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.45500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.67300000000000004}]},{"osd":7,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70999999999999996,"15min":0.73799999999999999},"min":{"1min":0.53600000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0469999999999999,"5min":1.095,"15min":1.095},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72699999999999998,"15min":0.75700000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92500000000000004,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.57399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194765,"statfs":{"total":200000000,"available":199439986,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68999999999999995,"15min":0.70399999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.91000000000000003,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69699999999999995,"15min":0.67500000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.96499999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.74199999999999999}]},{"osd":1,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.46500000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.028,"5min":1.216,"15min":1.216},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.44900000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.875,"5min":1.048,"15min":1.048},"last":0.66200000000000003}]},{"osd":2,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.71899999999999997,"15min":0.69899999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91100000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.69799999999999995,"15min":0.67600000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86699999999999999,"5min":1.252,"15min":1.252},"last":0.67300000000000004}]},{"osd":3,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.52800000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91500000000000004,"5min":1.111,"15min":1.111},"last":0.64400000000000002}]},{"osd":4,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.627},"min":{"1min":0.38400000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88800000000000001,"5min":1.272,"15min":1.272},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.629,"15min":0.63200000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.014,"5min":1.038,"15min":1.038},"last":0.61899999999999999}]},{"osd":6,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.63},"min":{"1min":0.41299999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.877,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.442,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.60099999999999998}]},{"osd":7,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93899999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.75900000000000001,"15min":0.79000000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.99399999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.58199999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194767,"statfs":{"total":200000000,"available":199441572,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78200000000000003,"15min":0.748},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.89300000000000002,"5min":6.6539999999999999,"15min":6.6539999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.747,"15min":0.72799999999999998},"min":{"1min":0.49299999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93300000000000005,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":0.64000000000000001}]},{"osd":1,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68200000000000005,"15min":0.71599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88200000000000001,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.504},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92200000000000004,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70199999999999996,"15min":0.68899999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0940000000000001,"5min":1.335,"15min":1.335},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70099999999999996,"15min":0.70399999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95099999999999996,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.54500000000000004}]},{"osd":3,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.54100000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.98999999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.745},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.78900000000000003},"min":{"1min":0.53200000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96799999999999997,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.70499999999999996}]},{"osd":5,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.39400000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.93500000000000005,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91900000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.57299999999999995}]},{"osd":6,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65900000000000003,"15min":0.69299999999999995},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91600000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.623},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.51800000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.91100000000000003,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.69599999999999995,"15min":0.69999999999999996},"min":{"1min":0.48099999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98199999999999998,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.74099999999999999,"15min":0.78000000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97999999999999998,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.73499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194765,"statfs":{"total":200000000,"available":199440008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.42999999999999999,"15min":0.40699999999999997},"min":{"1min":0.29699999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.71399999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.442,"15min":0.40799999999999997},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.437}]},{"osd":1,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46000000000000002,"15min":0.44800000000000001},"min":{"1min":0.28399999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.624,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71799999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.47299999999999998}]},{"osd":2,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51000000000000001,"15min":0.496},"min":{"1min":0.376,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68999999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51100000000000001,"15min":0.48999999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69899999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.41499999999999998}]},{"osd":4,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.74099999999999999,"15min":0.70599999999999996},"min":{"1min":0.50900000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0209999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.749,"15min":0.71899999999999997},"min":{"1min":0.57599999999999996,"5min":0.435,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.66900000000000004}]},{"osd":5,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.75800000000000001,"15min":0.69899999999999995},"min":{"1min":0.622,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.002,"5min":1.234,"15min":1.234},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.70299999999999996},"min":{"1min":0.53500000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0449999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.629}]},{"osd":6,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.71599999999999997},"min":{"1min":0.48899999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97499999999999998,"5min":1.276,"15min":1.276},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75800000000000001,"15min":0.70799999999999996},"min":{"1min":0.54200000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0760000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.75800000000000001},"min":{"1min":0.52800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.0900000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78500000000000003,"15min":0.74099999999999999},"min":{"1min":0.63300000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0649999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.60799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194766,"statfs":{"total":200000000,"available":199440990,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.59399999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.46300000000000002,"15min":0.443},"min":{"1min":0.33200000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.36799999999999999}]},{"osd":1,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.32900000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.624,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.432},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54300000000000004,"15min":0.52100000000000002},"min":{"1min":0.442,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.50800000000000001}]},{"osd":3,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83099999999999996,"5min":0.871,"15min":0.871},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.47399999999999998,"15min":0.45900000000000002},"min":{"1min":0.34699999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.68000000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45900000000000002}]},{"osd":4,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73399999999999999,"15min":0.68999999999999995},"min":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95599999999999996,"5min":1.095,"15min":1.095},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77900000000000003,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.0760000000000001}]},{"osd":5,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.73299999999999998,"15min":0.70699999999999996},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72599999999999998,"15min":0.70199999999999996},"min":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92900000000000005,"5min":1.127,"15min":1.127},"last":1.1220000000000001}]},{"osd":6,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73099999999999998,"15min":0.70299999999999996},"min":{"1min":0.52000000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.02,"5min":1.054,"15min":1.054},"last":1.028},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69899999999999995,"15min":0.67800000000000005},"min":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.89600000000000002,"5min":1.002,"15min":1.002},"last":0.89600000000000002}]},{"osd":7,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.76800000000000002,"15min":0.73199999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.77200000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0249999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":1.1799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194766,"statfs":{"total":200000000,"available":199441378,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41599999999999998,"15min":0.42999999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58999999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.499,"5min":0.46400000000000002,"15min":0.438},"min":{"1min":0.35799999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.67800000000000005,"5min":0.876,"15min":0.876},"last":0.60199999999999998}]},{"osd":2,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.36099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80500000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.495,"15min":0.53100000000000003},"min":{"1min":0.35199999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64100000000000001,"5min":1.923,"15min":1.923},"last":0.44}]},{"osd":3,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53600000000000003,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.497,"15min":0.48899999999999999},"min":{"1min":0.34699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.62,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.53700000000000003}]},{"osd":4,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.75900000000000001},"min":{"1min":0.496,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.109,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.72299999999999998,"15min":0.68899999999999995},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0389999999999999,"5min":1.3,"15min":1.3},"last":0.85599999999999998}]},{"osd":5,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.76700000000000002,"15min":0.73599999999999999},"min":{"1min":0.59499999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0169999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.751,"15min":0.753},"min":{"1min":0.621,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97099999999999997,"5min":1.97,"15min":1.97},"last":0.92900000000000005}]},{"osd":6,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.748},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0009999999999999,"5min":1.272,"15min":1.272},"last":1.028},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78600000000000003,"15min":0.75},"min":{"1min":0.57899999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0620000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.91500000000000004}]},{"osd":7,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.76400000000000001,"15min":0.77600000000000002},"min":{"1min":0.63300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98799999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.997},{"interface":"front","average":{"1min":0.754,"5min":0.75600000000000001,"15min":0.77600000000000002},"min":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98899999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.95399999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1121,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194191,"statfs":{"total":200000000,"available":198851789,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42499999999999999,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58099999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.51100000000000001,"15min":0.48499999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69199999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.52800000000000002}]},{"osd":2,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.45400000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76200000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.42799999999999999},"min":{"1min":0.29399999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.623,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.437}]},{"osd":3,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45600000000000002,"15min":0.434},"min":{"1min":0.33900000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61699999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53400000000000003,"15min":0.503},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.746,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.49299999999999999}]},{"osd":4,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.71099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1020000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.748},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.57099999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.113,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.71799999999999997}]},{"osd":5,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71899999999999997,"15min":0.70699999999999996},"min":{"1min":0.54200000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99299999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.76900000000000002,"15min":0.73899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.143,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.59699999999999998}]},{"osd":6,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79300000000000004,"15min":0.77300000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1240000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78200000000000003,"15min":0.77200000000000002},"min":{"1min":0.55300000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2090000000000001,"5min":1.5,"15min":1.5},"last":0.61299999999999999}]},{"osd":7,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.75600000000000001,"15min":0.72699999999999998},"min":{"1min":0.57499999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76900000000000002,"15min":0.75900000000000001},"min":{"1min":0.55900000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.0589999999999999,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.63100000000000001}]}]}],"pool_statfs":[]}} 2024-05-22T21:18:26.888 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-22T21:18:26.888 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-22T21:18:27.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.067+0000 7f65d4a00640 1 Processor -- start 2024-05-22T21:18:27.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.067+0000 7f65d4a00640 1 -- start start 2024-05-22T21:18:27.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.067+0000 7f65d4a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65d013cfc0 0x7f65d01402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:27.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.067+0000 7f65d4a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f65d013fe50 0x7f65d006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:27.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.067+0000 7f65d4a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d013d3a0 con 0x7f65d015af60 2024-05-22T21:18:27.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.067+0000 7f65d4a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d00cd920 con 0x7f65d013fe50 2024-05-22T21:18:27.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.067+0000 7f65d4a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00cdc00 con 0x7f65d013cfc0 2024-05-22T21:18:27.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.067+0000 7f65ce800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f65d013fe50 0x7f65d006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:27.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.067+0000 7f65ce800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f65d013fe50 0x7f65d006fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50398/0 (socket says 172.21.15.161:50398) 2024-05-22T21:18:27.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.067+0000 7f65ce800640 1 -- 172.21.15.161:0/1803838849 learned_addr learned my addr 172.21.15.161:0/1803838849 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:27.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.067+0000 7f65cde00640 1 --2- 172.21.15.161:0/1803838849 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65d013cfc0 0x7f65d01402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:27.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.067+0000 7f65ce800640 1 -- 172.21.15.161:0/1803838849 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65d013cfc0 msgr2=0x7f65d01402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:27.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.067+0000 7f65ce800640 1 --2- 172.21.15.161:0/1803838849 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65d013cfc0 0x7f65d01402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.067+0000 7f65ce800640 1 -- 172.21.15.161:0/1803838849 >> v1:172.21.15.179:6789/0 conn(0x7f65d015af60 legacy=0x7f65d013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:27.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.067+0000 7f65ce800640 1 -- 172.21.15.161:0/1803838849 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d00cdee0 con 0x7f65d013fe50 2024-05-22T21:18:27.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.071+0000 7f65ce800640 1 --2- 172.21.15.161:0/1803838849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d013fe50 0x7f65d006fce0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009e50 tx=0x7f65c801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3bf595780234fb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:27.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.071+0000 7f65cd400640 1 -- 172.21.15.161:0/1803838849 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80a6470 con 0x7f65d013fe50 2024-05-22T21:18:27.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.071+0000 7f65cd400640 1 -- 172.21.15.161:0/1803838849 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80a65d0 con 0x7f65d013fe50 2024-05-22T21:18:27.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.071+0000 7f65cd400640 1 -- 172.21.15.161:0/1803838849 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80a6960 con 0x7f65d013fe50 2024-05-22T21:18:27.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.071+0000 7f65d4a00640 1 -- 172.21.15.161:0/1803838849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d013fe50 msgr2=0x7f65d006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:27.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.071+0000 7f65d4a00640 1 --2- 172.21.15.161:0/1803838849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d013fe50 0x7f65d006fce0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009e50 tx=0x7f65c801c970 comp rx=0 tx=0).stop 2024-05-22T21:18:27.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.071+0000 7f65d4a00640 1 -- 172.21.15.161:0/1803838849 shutdown_connections 2024-05-22T21:18:27.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.071+0000 7f65d4a00640 1 --2- 172.21.15.161:0/1803838849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d013fe50 0x7f65d006fce0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.071+0000 7f65d4a00640 1 --2- 172.21.15.161:0/1803838849 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65d013cfc0 0x7f65d01402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.071+0000 7f65d4a00640 1 -- 172.21.15.161:0/1803838849 >> 172.21.15.161:0/1803838849 conn(0x7f65d006a2d0 msgr2=0x7f65d006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:27.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.071+0000 7f65d4a00640 1 -- 172.21.15.161:0/1803838849 shutdown_connections 2024-05-22T21:18:27.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.071+0000 7f65d4a00640 1 -- 172.21.15.161:0/1803838849 wait complete. 2024-05-22T21:18:27.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65d4a00640 1 Processor -- start 2024-05-22T21:18:27.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65d4a00640 1 -- start start 2024-05-22T21:18:27.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65d4a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d013cfc0 0x7f65d01f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:27.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65d4a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d013fe50 0x7f65d0140b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:27.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65d4a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65d015af60 0x7f65d01f9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:27.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65d4a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d01f9c50 con 0x7f65d013fe50 2024-05-22T21:18:27.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65d4a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d013e290 con 0x7f65d013cfc0 2024-05-22T21:18:27.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65d4a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d013d6c0 con 0x7f65d015af60 2024-05-22T21:18:27.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65c5400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d013fe50 0x7f65d0140b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:27.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65c5400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d013fe50 0x7f65d0140b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50406/0 (socket says 172.21.15.161:50406) 2024-05-22T21:18:27.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65c5400640 1 -- 172.21.15.161:0/2475900387 learned_addr learned my addr 172.21.15.161:0/2475900387 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:27.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65cde00640 1 --2- 172.21.15.161:0/2475900387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d013cfc0 0x7f65d01f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65c5400640 1 -- 172.21.15.161:0/2475900387 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65d015af60 msgr2=0x7f65d01f9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65c5400640 1 --2- 172.21.15.161:0/2475900387 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65d015af60 0x7f65d01f9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65c5400640 1 -- 172.21.15.161:0/2475900387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d013cfc0 msgr2=0x7f65d01f6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65c5400640 1 --2- 172.21.15.161:0/2475900387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d013cfc0 0x7f65d01f6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65c5400640 1 -- 172.21.15.161:0/2475900387 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c8009b30 con 0x7f65d013fe50 2024-05-22T21:18:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65cde00640 1 --2- 172.21.15.161:0/2475900387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d013cfc0 0x7f65d01f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65c5400640 1 --2- 172.21.15.161:0/2475900387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d013fe50 0x7f65d0140b70 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f65b800c3c0 tx=0x7f65b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65c6a00640 1 -- 172.21.15.161:0/2475900387 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b80052d0 con 0x7f65d013fe50 2024-05-22T21:18:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65c6a00640 1 -- 172.21.15.161:0/2475900387 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b8005430 con 0x7f65d013fe50 2024-05-22T21:18:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65c6a00640 1 -- 172.21.15.161:0/2475900387 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b8003840 con 0x7f65d013fe50 2024-05-22T21:18:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65d4a00640 1 -- 172.21.15.161:0/2475900387 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d013d960 con 0x7f65d013fe50 2024-05-22T21:18:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65d4a00640 1 -- 172.21.15.161:0/2475900387 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d013df20 con 0x7f65d013fe50 2024-05-22T21:18:27.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.075+0000 7f65d4a00640 1 -- 172.21.15.161:0/2475900387 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d013e440 con 0x7f65d013fe50 2024-05-22T21:18:27.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.079+0000 7f65c6a00640 1 -- 172.21.15.161:0/2475900387 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f65b80039a0 con 0x7f65d013fe50 2024-05-22T21:18:27.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.079+0000 7f65c6a00640 1 --2- 172.21.15.161:0/2475900387 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65ac0666c0 0x7f65ac068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:27.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.079+0000 7f65c6a00640 1 -- 172.21.15.161:0/2475900387 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65b8083680 con 0x7f65d013fe50 2024-05-22T21:18:27.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.079+0000 7f65cde00640 1 --2- 172.21.15.161:0/2475900387 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65ac0666c0 0x7f65ac068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:27.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.079+0000 7f65cde00640 1 --2- 172.21.15.161:0/2475900387 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65ac0666c0 0x7f65ac068b80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009ae0 tx=0x7f65c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:27.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.083+0000 7f65c6a00640 1 -- 172.21.15.161:0/2475900387 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f65d013e440 con 0x7f65d013fe50 2024-05-22T21:18:27.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.291+0000 7f65d4a00640 1 -- 172.21.15.161:0/2475900387 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f65d013e400 con 0x7f65d013fe50 2024-05-22T21:18:27.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.295+0000 7f65c6a00640 1 -- 172.21.15.161:0/2475900387 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f65b80509e0 con 0x7f65d013fe50 2024-05-22T21:18:27.298 INFO:teuthology.orchestra.run.smithi161.stderr:updated caps for client.0 2024-05-22T21:18:27.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.299+0000 7f65d4a00640 1 -- 172.21.15.161:0/2475900387 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65ac0666c0 msgr2=0x7f65ac068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:27.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.299+0000 7f65d4a00640 1 --2- 172.21.15.161:0/2475900387 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65ac0666c0 0x7f65ac068b80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009ae0 tx=0x7f65c0008040 comp rx=0 tx=0).stop 2024-05-22T21:18:27.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.299+0000 7f65d4a00640 1 -- 172.21.15.161:0/2475900387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d013fe50 msgr2=0x7f65d0140b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:27.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.299+0000 7f65d4a00640 1 --2- 172.21.15.161:0/2475900387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d013fe50 0x7f65d0140b70 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f65b800c3c0 tx=0x7f65b800c880 comp rx=0 tx=0).stop 2024-05-22T21:18:27.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.299+0000 7f65d4a00640 1 -- 172.21.15.161:0/2475900387 shutdown_connections 2024-05-22T21:18:27.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.299+0000 7f65d4a00640 1 --2- 172.21.15.161:0/2475900387 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f65ac0666c0 0x7f65ac068b80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.299+0000 7f65d4a00640 1 --2- 172.21.15.161:0/2475900387 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f65d015af60 0x7f65d01f9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.299+0000 7f65d4a00640 1 --2- 172.21.15.161:0/2475900387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d013fe50 0x7f65d0140b70 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.299+0000 7f65d4a00640 1 --2- 172.21.15.161:0/2475900387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f65d013cfc0 0x7f65d01f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.299+0000 7f65d4a00640 1 -- 172.21.15.161:0/2475900387 >> 172.21.15.161:0/2475900387 conn(0x7f65d006a2d0 msgr2=0x7f65d006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:27.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.299+0000 7f65d4a00640 1 -- 172.21.15.161:0/2475900387 shutdown_connections 2024-05-22T21:18:27.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.299+0000 7f65d4a00640 1 -- 172.21.15.161:0/2475900387 wait complete. 2024-05-22T21:18:27.320 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:18:27.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.451+0000 7f622c000640 1 Processor -- start 2024-05-22T21:18:27.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.451+0000 7f622c000640 1 -- start start 2024-05-22T21:18:27.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.451+0000 7f622c000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f622415dd50 0x7f622406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:27.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.451+0000 7f622c000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f622415d5e0 0x7f622415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:27.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.451+0000 7f622c000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6224071670 con 0x7f622415e130 2024-05-22T21:18:27.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.455+0000 7f622c000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622406de40 con 0x7f622415d5e0 2024-05-22T21:18:27.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.455+0000 7f622c000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6224070a60 con 0x7f622415dd50 2024-05-22T21:18:27.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.455+0000 7f622a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f622415d5e0 0x7f622415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:27.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.455+0000 7f6228c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f622415e130 0x7f622415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56972/0 (socket says 172.21.15.161:56972) 2024-05-22T21:18:27.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.455+0000 7f6228c00640 1 -- 172.21.15.161:0/2743067019 learned_addr learned my addr 172.21.15.161:0/2743067019 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:27.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.455+0000 7f6229600640 1 --2- 172.21.15.161:0/2743067019 >> v2:172.21.15.179:3300/0 conn(0x7f622415dd50 0x7f622406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:27.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.455+0000 7f622a000640 1 -- 172.21.15.161:0/2743067019 >> v2:172.21.15.179:3300/0 conn(0x7f622415dd50 msgr2=0x7f622406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:27.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.455+0000 7f622a000640 1 --2- 172.21.15.161:0/2743067019 >> v2:172.21.15.179:3300/0 conn(0x7f622415dd50 0x7f622406d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.455+0000 7f622a000640 1 -- 172.21.15.161:0/2743067019 >> v1:172.21.15.161:6789/0 conn(0x7f622415e130 legacy=0x7f622415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:27.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.455+0000 7f622a000640 1 -- 172.21.15.161:0/2743067019 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6224070d40 con 0x7f622415d5e0 2024-05-22T21:18:27.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.455+0000 7f6223e00640 1 -- 172.21.15.161:0/2743067019 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594480604 0 0) 0x7f6224071670 con 0x7f622415e130 2024-05-22T21:18:27.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.455+0000 7f622a000640 1 --2- 172.21.15.161:0/2743067019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622415d5e0 0x7f622415b6c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6224072330 tx=0x7f621c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1508c60591de6551 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:27.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.455+0000 7f6223e00640 1 -- 172.21.15.161:0/2743067019 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0a59c0 con 0x7f622415d5e0 2024-05-22T21:18:27.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.455+0000 7f6223e00640 1 -- 172.21.15.161:0/2743067019 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c0a5b20 con 0x7f622415d5e0 2024-05-22T21:18:27.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.455+0000 7f6223e00640 1 -- 172.21.15.161:0/2743067019 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0ad440 con 0x7f622415d5e0 2024-05-22T21:18:27.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622c000640 1 -- 172.21.15.161:0/2743067019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622415d5e0 msgr2=0x7f622415b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:27.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622c000640 1 --2- 172.21.15.161:0/2743067019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622415d5e0 0x7f622415b6c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6224072330 tx=0x7f621c01c900 comp rx=0 tx=0).stop 2024-05-22T21:18:27.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622c000640 1 -- 172.21.15.161:0/2743067019 shutdown_connections 2024-05-22T21:18:27.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622c000640 1 --2- 172.21.15.161:0/2743067019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622415d5e0 0x7f622415b6c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622c000640 1 --2- 172.21.15.161:0/2743067019 >> v2:172.21.15.179:3300/0 conn(0x7f622415dd50 0x7f622406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622c000640 1 -- 172.21.15.161:0/2743067019 >> 172.21.15.161:0/2743067019 conn(0x7f62240b0bc0 msgr2=0x7f62240b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:27.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622c000640 1 -- 172.21.15.161:0/2743067019 shutdown_connections 2024-05-22T21:18:27.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622c000640 1 -- 172.21.15.161:0/2743067019 wait complete. 2024-05-22T21:18:27.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622c000640 1 Processor -- start 2024-05-22T21:18:27.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622c000640 1 -- start start 2024-05-22T21:18:27.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622c000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f622415d5e0 0x7f6224155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:27.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622c000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f622415dd50 0x7f622406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:27.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622c000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622415e130 0x7f62240ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:27.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622c000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62240ad5f0 con 0x7f622415e130 2024-05-22T21:18:27.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622c000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62240ac9e0 con 0x7f622415dd50 2024-05-22T21:18:27.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622c000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62240accc0 con 0x7f622415d5e0 2024-05-22T21:18:27.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622415e130 0x7f62240ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:27.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622415e130 0x7f62240ada80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50436/0 (socket says 172.21.15.161:50436) 2024-05-22T21:18:27.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f622a000640 1 -- 172.21.15.161:0/821210175 learned_addr learned my addr 172.21.15.161:0/821210175 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:27.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f6229600640 1 --2- 172.21.15.161:0/821210175 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f622415d5e0 0x7f6224155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:27.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.459+0000 7f6228c00640 1 --2- 172.21.15.161:0/821210175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f622415dd50 0x7f622406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:27.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f622a000640 1 -- 172.21.15.161:0/821210175 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f622415d5e0 msgr2=0x7f6224155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:27.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f622a000640 1 --2- 172.21.15.161:0/821210175 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f622415d5e0 0x7f6224155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f622a000640 1 -- 172.21.15.161:0/821210175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f622415dd50 msgr2=0x7f622406d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:27.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f622a000640 1 --2- 172.21.15.161:0/821210175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f622415dd50 0x7f622406d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f622a000640 1 -- 172.21.15.161:0/821210175 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c009a70 con 0x7f622415e130 2024-05-22T21:18:27.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f6228c00640 1 --2- 172.21.15.161:0/821210175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f622415dd50 0x7f622406d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:27.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f622a000640 1 --2- 172.21.15.161:0/821210175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622415e130 0x7f62240ada80 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f621c019510 tx=0x7f621c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:27.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f6221600640 1 -- 172.21.15.161:0/821210175 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0ade20 con 0x7f622415e130 2024-05-22T21:18:27.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f6221600640 1 -- 172.21.15.161:0/821210175 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c009450 con 0x7f622415e130 2024-05-22T21:18:27.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f6221600640 1 -- 172.21.15.161:0/821210175 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0b9690 con 0x7f622415e130 2024-05-22T21:18:27.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f622c000640 1 -- 172.21.15.161:0/821210175 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62240acfa0 con 0x7f622415e130 2024-05-22T21:18:27.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f622c000640 1 -- 172.21.15.161:0/821210175 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622406d9f0 con 0x7f622415e130 2024-05-22T21:18:27.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f622c000640 1 -- 172.21.15.161:0/821210175 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62240ad760 con 0x7f622415e130 2024-05-22T21:18:27.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f6221600640 1 -- 172.21.15.161:0/821210175 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f621c0b97f0 con 0x7f622415e130 2024-05-22T21:18:27.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f6221600640 1 --2- 172.21.15.161:0/821210175 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6204066690 0x7f6204068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:27.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f6221600640 1 -- 172.21.15.161:0/821210175 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f621c11f9f0 con 0x7f622415e130 2024-05-22T21:18:27.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.463+0000 7f6229600640 1 --2- 172.21.15.161:0/821210175 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6204066690 0x7f6204068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:27.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.467+0000 7f6229600640 1 --2- 172.21.15.161:0/821210175 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6204066690 0x7f6204068b50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6218007a60 tx=0x7f6218008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:27.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.467+0000 7f6221600640 1 -- 172.21.15.161:0/821210175 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f621c0ecd50 con 0x7f622415e130 2024-05-22T21:18:27.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.655+0000 7f622c000640 1 -- 172.21.15.161:0/821210175 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62240ace00 con 0x7f622415e130 2024-05-22T21:18:27.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.659+0000 7f6221600640 1 -- 172.21.15.161:0/821210175 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f62240ace00 con 0x7f622415e130 2024-05-22T21:18:27.660 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:27.661 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2771409351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2771409351},{"type":"v1","addr":"172.21.15.179:6834","nonce":2771409351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6442,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2520119431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2520119431},{"type":"v1","addr":"172.21.15.179:6835","nonce":2520119431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1415362547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1415362547},{"type":"v1","addr":"172.21.15.161:6835","nonce":1415362547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:18:17.938858+0000","modified":"2024-05-22T21:18:20.963065+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/4127130864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4127130864},{"type":"v1","addr":"172.21.15.161:6837","nonce":4127130864}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-22T21:18:27.661 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 47 2024-05-22T21:18:27.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.659+0000 7f622c000640 1 -- 172.21.15.161:0/821210175 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6204066690 msgr2=0x7f6204068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:27.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.659+0000 7f622c000640 1 --2- 172.21.15.161:0/821210175 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6204066690 0x7f6204068b50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6218007a60 tx=0x7f6218008040 comp rx=0 tx=0).stop 2024-05-22T21:18:27.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.659+0000 7f622c000640 1 -- 172.21.15.161:0/821210175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622415e130 msgr2=0x7f62240ada80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:27.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.659+0000 7f622c000640 1 --2- 172.21.15.161:0/821210175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622415e130 0x7f62240ada80 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f621c019510 tx=0x7f621c0196a0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.659+0000 7f622c000640 1 -- 172.21.15.161:0/821210175 shutdown_connections 2024-05-22T21:18:27.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.659+0000 7f622c000640 1 --2- 172.21.15.161:0/821210175 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6204066690 0x7f6204068b50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.659+0000 7f622c000640 1 --2- 172.21.15.161:0/821210175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f622415e130 0x7f62240ada80 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.659+0000 7f622c000640 1 --2- 172.21.15.161:0/821210175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f622415dd50 0x7f622406d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.659+0000 7f622c000640 1 --2- 172.21.15.161:0/821210175 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f622415d5e0 0x7f6224155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:27.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.659+0000 7f622c000640 1 -- 172.21.15.161:0/821210175 >> 172.21.15.161:0/821210175 conn(0x7f62240b0bc0 msgr2=0x7f62240aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:27.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.659+0000 7f622c000640 1 -- 172.21.15.161:0/821210175 shutdown_connections 2024-05-22T21:18:27.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:27.659+0000 7f622c000640 1 -- 172.21.15.161:0/821210175 wait complete. 2024-05-22T21:18:28.083 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-22T21:18:17.938858+0000', 'modified': '2024-05-22T21:18:20.963065+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6699}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6699': {'gid': 6699, 'name': 'c', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6837/4127130864', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 4127130864}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 4127130864}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-22T21:18:28.084 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-22T21:18:28.084 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-22T21:18:28.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.219+0000 7ff9b7600640 1 Processor -- start 2024-05-22T21:18:28.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.219+0000 7ff9b7600640 1 -- start start 2024-05-22T21:18:28.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.219+0000 7ff9b7600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff9b013cfc0 0x7ff9b01402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.223+0000 7ff9b7600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9b015af60 0x7ff9b013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.223+0000 7ff9b7600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b01400d0 con 0x7ff9b013fc80 2024-05-22T21:18:28.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.223+0000 7ff9b7600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b013d3a0 con 0x7ff9b013cfc0 2024-05-22T21:18:28.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.223+0000 7ff9b7600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00cd920 con 0x7ff9b015af60 2024-05-22T21:18:28.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.223+0000 7ff9b5600640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff9b013fc80 0x7ff9b015dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56982/0 (socket says 172.21.15.161:56982) 2024-05-22T21:18:28.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.223+0000 7ff9b5600640 1 -- 172.21.15.161:0/1651531584 learned_addr learned my addr 172.21.15.161:0/1651531584 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:28.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.223+0000 7ff9a7e00640 1 --2- 172.21.15.161:0/1651531584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9b015af60 0x7ff9b013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.223+0000 7ff9b4c00640 1 --2- 172.21.15.161:0/1651531584 >> v2:172.21.15.179:3300/0 conn(0x7ff9b013cfc0 0x7ff9b01402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.223+0000 7ff9afe00640 1 -- 172.21.15.161:0/1651531584 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356897689 0 0) 0x7ff9a80014a0 con 0x7ff9b013fc80 2024-05-22T21:18:28.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.223+0000 7ff9afe00640 1 -- 172.21.15.161:0/1651531584 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff994003530 con 0x7ff9b013fc80 2024-05-22T21:18:28.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.223+0000 7ff9b4c00640 1 -- 172.21.15.161:0/1651531584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9b015af60 msgr2=0x7ff9b013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:28.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.223+0000 7ff9b4c00640 1 --2- 172.21.15.161:0/1651531584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9b015af60 0x7ff9b013f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.223+0000 7ff9b4c00640 1 -- 172.21.15.161:0/1651531584 >> v1:172.21.15.161:6789/0 conn(0x7ff9b013fc80 legacy=0x7ff9b015dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.223+0000 7ff9b4c00640 1 -- 172.21.15.161:0/1651531584 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b00cdc00 con 0x7ff9b013cfc0 2024-05-22T21:18:28.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.223+0000 7ff9a7e00640 1 --2- 172.21.15.161:0/1651531584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9b015af60 0x7ff9b013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:28.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b4c00640 1 --2- 172.21.15.161:0/1651531584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b013cfc0 0x7ff9b01402e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00097c0 tx=0x7ff9a001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdc09f12eb18495 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:28.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9afe00640 1 -- 172.21.15.161:0/1651531584 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a00ab600 con 0x7ff9b013cfc0 2024-05-22T21:18:28.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9afe00640 1 -- 172.21.15.161:0/1651531584 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a00a7050 con 0x7ff9b013cfc0 2024-05-22T21:18:28.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9afe00640 1 -- 172.21.15.161:0/1651531584 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a00ab980 con 0x7ff9b013cfc0 2024-05-22T21:18:28.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b7600640 1 -- 172.21.15.161:0/1651531584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b013cfc0 msgr2=0x7ff9b01402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b7600640 1 --2- 172.21.15.161:0/1651531584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b013cfc0 0x7ff9b01402e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00097c0 tx=0x7ff9a001c410 comp rx=0 tx=0).stop 2024-05-22T21:18:28.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b7600640 1 -- 172.21.15.161:0/1651531584 shutdown_connections 2024-05-22T21:18:28.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b7600640 1 --2- 172.21.15.161:0/1651531584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9b015af60 0x7ff9b013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b7600640 1 --2- 172.21.15.161:0/1651531584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b013cfc0 0x7ff9b01402e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b7600640 1 -- 172.21.15.161:0/1651531584 >> 172.21.15.161:0/1651531584 conn(0x7ff9b006a2d0 msgr2=0x7ff9b006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:28.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b7600640 1 -- 172.21.15.161:0/1651531584 shutdown_connections 2024-05-22T21:18:28.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b7600640 1 -- 172.21.15.161:0/1651531584 wait complete. 2024-05-22T21:18:28.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b7600640 1 Processor -- start 2024-05-22T21:18:28.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b7600640 1 -- start start 2024-05-22T21:18:28.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b7600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9b013cfc0 0x7ff9b0140c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b7600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b013fc80 0x7ff9b0156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b7600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b015af60 0x7ff9b00b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b7600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b0156f90 con 0x7ff9b013fc80 2024-05-22T21:18:28.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b7600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b015ce30 con 0x7ff9b015af60 2024-05-22T21:18:28.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9b7600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b015d110 con 0x7ff9b013cfc0 2024-05-22T21:18:28.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9a7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b013fc80 0x7ff9b0156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9a7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b013fc80 0x7ff9b0156a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50446/0 (socket says 172.21.15.161:50446) 2024-05-22T21:18:28.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.227+0000 7ff9a7e00640 1 -- 172.21.15.161:0/284290209 learned_addr learned my addr 172.21.15.161:0/284290209 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:28.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.231+0000 7ff9b4c00640 1 --2- 172.21.15.161:0/284290209 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9b013cfc0 0x7ff9b0140c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.231+0000 7ff9b5600640 1 --2- 172.21.15.161:0/284290209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b015af60 0x7ff9b00b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.231+0000 7ff9a7e00640 1 -- 172.21.15.161:0/284290209 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9b013cfc0 msgr2=0x7ff9b0140c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.231+0000 7ff9a7e00640 1 --2- 172.21.15.161:0/284290209 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9b013cfc0 0x7ff9b0140c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.231+0000 7ff9a7e00640 1 -- 172.21.15.161:0/284290209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b015af60 msgr2=0x7ff9b00b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.231+0000 7ff9a7e00640 1 --2- 172.21.15.161:0/284290209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b015af60 0x7ff9b00b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.231+0000 7ff9a7e00640 1 -- 172.21.15.161:0/284290209 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff998009420 con 0x7ff9b013fc80 2024-05-22T21:18:28.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.231+0000 7ff9a7e00640 1 --2- 172.21.15.161:0/284290209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b013fc80 0x7ff9b0156a50 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff99800ebe0 tx=0x7ff99800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:28.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.231+0000 7ff9ad600640 1 -- 172.21.15.161:0/284290209 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99800eee0 con 0x7ff9b013fc80 2024-05-22T21:18:28.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.231+0000 7ff9ad600640 1 -- 172.21.15.161:0/284290209 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99800cc30 con 0x7ff9b013fc80 2024-05-22T21:18:28.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.231+0000 7ff9ad600640 1 -- 172.21.15.161:0/284290209 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998013470 con 0x7ff9b013fc80 2024-05-22T21:18:28.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.231+0000 7ff9b7600640 1 -- 172.21.15.161:0/284290209 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a0009420 con 0x7ff9b013fc80 2024-05-22T21:18:28.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.231+0000 7ff9b7600640 1 -- 172.21.15.161:0/284290209 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b013f430 con 0x7ff9b013fc80 2024-05-22T21:18:28.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.231+0000 7ff9ad600640 1 -- 172.21.15.161:0/284290209 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff9980135d0 con 0x7ff9b013fc80 2024-05-22T21:18:28.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9ad600640 1 --2- 172.21.15.161:0/284290209 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff98c066690 0x7ff98c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9ad600640 1 -- 172.21.15.161:0/284290209 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9980844e0 con 0x7ff9b013fc80 2024-05-22T21:18:28.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9b4c00640 1 --2- 172.21.15.161:0/284290209 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff98c066690 0x7ff98c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 Processor -- start 2024-05-22T21:18:28.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 -- start start 2024-05-22T21:18:28.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9b4c00640 1 --2- 172.21.15.161:0/284290209 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff98c066690 0x7ff98c068b50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00ad000 tx=0x7ff9a00a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:28.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275b99c0 0x55aa275b9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa275ba2e0 0x55aa2756f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa2756fec0 0x55aa275702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa27437e80 con 0x55aa275b99c0 2024-05-22T21:18:28.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa273c5af0 con 0x55aa2756fec0 2024-05-22T21:18:28.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa273c30c0 con 0x55aa275ba2e0 2024-05-22T21:18:28.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9b4c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275b99c0 0x55aa275b9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9b4c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275b99c0 0x55aa275b9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50456/0 (socket says 172.21.15.161:50456) 2024-05-22T21:18:28.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9b4c00640 1 -- 172.21.15.161:0/2442616311 learned_addr learned my addr 172.21.15.161:0/2442616311 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:28.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9b5600640 1 --2- 172.21.15.161:0/2442616311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa2756fec0 0x55aa275702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9a7e00640 1 --2- 172.21.15.161:0/2442616311 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa275ba2e0 0x55aa2756f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9b4c00640 1 -- 172.21.15.161:0/2442616311 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa275ba2e0 msgr2=0x55aa2756f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:28.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9b4c00640 1 --2- 172.21.15.161:0/2442616311 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa275ba2e0 0x55aa2756f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9b4c00640 1 -- 172.21.15.161:0/2442616311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa2756fec0 msgr2=0x55aa275702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:28.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9b4c00640 1 --2- 172.21.15.161:0/2442616311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa2756fec0 0x55aa275702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9b4c00640 1 -- 172.21.15.161:0/2442616311 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa270ba890 con 0x55aa275b99c0 2024-05-22T21:18:28.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9b4c00640 1 --2- 172.21.15.161:0/2442616311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275b99c0 0x55aa275b9da0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00ae040 tx=0x7ff9a0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286c68f7a90daace server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:28.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9a5600640 1 -- 172.21.15.161:0/2442616311 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a00cb660 con 0x55aa275b99c0 2024-05-22T21:18:28.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9a5600640 1 -- 172.21.15.161:0/2442616311 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a00cb7c0 con 0x55aa275b99c0 2024-05-22T21:18:28.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9a5600640 1 -- 172.21.15.161:0/2442616311 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a00cbb30 con 0x55aa275b99c0 2024-05-22T21:18:28.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 -- 172.21.15.161:0/2442616311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275b99c0 msgr2=0x55aa275b9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/2442616311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275b99c0 0x55aa275b9da0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00ae040 tx=0x7ff9a0004740 comp rx=0 tx=0).stop 2024-05-22T21:18:28.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 -- 172.21.15.161:0/2442616311 shutdown_connections 2024-05-22T21:18:28.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/2442616311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa2756fec0 0x55aa275702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/2442616311 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa275ba2e0 0x55aa2756f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/2442616311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275b99c0 0x55aa275b9da0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 -- 172.21.15.161:0/2442616311 >> 172.21.15.161:0/2442616311 conn(0x55aa27576440 msgr2=0x55aa2756e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:28.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 -- 172.21.15.161:0/2442616311 shutdown_connections 2024-05-22T21:18:28.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.235+0000 7ff9baa1a000 1 -- 172.21.15.161:0/2442616311 wait complete. 2024-05-22T21:18:28.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 1 -- start start 2024-05-22T21:18:28.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 1 Processor -- start 2024-05-22T21:18:28.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa2756fec0 0x55aa275c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275b99c0 0x55aa275c1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa275ba2e0 0x55aa2757afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa273be9b0 con 0x55aa275b99c0 2024-05-22T21:18:28.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2735dc00 con 0x55aa2756fec0 2024-05-22T21:18:28.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa2737c070 con 0x55aa275ba2e0 2024-05-22T21:18:28.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 10 client.0 _ll_register_callbacks cb 0x55aa2755bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:18:28.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9a7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275b99c0 0x55aa275c1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9a7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275b99c0 0x55aa275c1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50470/0 (socket says 172.21.15.161:50470) 2024-05-22T21:18:28.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9a7e00640 1 -- 172.21.15.161:0/2492772906 learned_addr learned my addr 172.21.15.161:0/2492772906 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:28.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9b4c00640 1 --2- 172.21.15.161:0/2492772906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa2756fec0 0x55aa275c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9b5600640 1 --2- 172.21.15.161:0/2492772906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa275ba2e0 0x55aa2757afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9a7e00640 1 -- 172.21.15.161:0/2492772906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa275ba2e0 msgr2=0x55aa2757afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:28.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9a7e00640 1 --2- 172.21.15.161:0/2492772906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa275ba2e0 0x55aa2757afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9a7e00640 1 -- 172.21.15.161:0/2492772906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa2756fec0 msgr2=0x55aa275c0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:28.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9a7e00640 1 --2- 172.21.15.161:0/2492772906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa2756fec0 0x55aa275c0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9a7e00640 1 -- 172.21.15.161:0/2492772906 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b00cdc00 con 0x55aa275b99c0 2024-05-22T21:18:28.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9b5600640 1 --2- 172.21.15.161:0/2492772906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa275ba2e0 0x55aa2757afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:28.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9a7e00640 1 --2- 172.21.15.161:0/2492772906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275b99c0 0x55aa275c1260 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff99800ca20 tx=0x7ff998055660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0dce3b82864c463 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:28.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff99e000640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:18:28.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff99e000640 1 -- 172.21.15.161:0/2492772906 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998053980 con 0x55aa275b99c0 2024-05-22T21:18:28.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff99e000640 1 -- 172.21.15.161:0/2492772906 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff99805d3d0 con 0x55aa275b99c0 2024-05-22T21:18:28.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 1 -- 172.21.15.161:0/2492772906 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa2737b5f0 con 0x55aa275b99c0 2024-05-22T21:18:28.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff99e000640 1 -- 172.21.15.161:0/2492772906 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff99805d590 con 0x55aa275b99c0 2024-05-22T21:18:28.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 10 client.6852 fetch_fsmap learned FSMap version 47 2024-05-22T21:18:28.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 1 -- 172.21.15.161:0/2492772906 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55aa27383ab0 con 0x55aa275b99c0 2024-05-22T21:18:28.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff99e000640 1 -- 172.21.15.161:0/2492772906 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aa2737b5f0 con 0x55aa275b99c0 2024-05-22T21:18:28.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff99e000640 1 -- 172.21.15.161:0/2492772906 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 47) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff99806a030 con 0x55aa275b99c0 2024-05-22T21:18:28.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 10 client.6852 fetch_fsmap finished waiting for FSMap version 47 2024-05-22T21:18:28.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 10 client.6852 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-22T21:18:28.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 20 client.6852 populate_metadata read hostname 'smithi161' 2024-05-22T21:18:28.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/2492772906 >> [v2:172.21.15.161:6836/4127130864,v1:172.21.15.161:6837/4127130864] conn(0x55aa2757bd20 0x55aa2757c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 4 client.6852 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T21:18:28.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9baa1a000 1 -- 172.21.15.161:0/2492772906 --> [v2:172.21.15.161:6836/4127130864,v1:172.21.15.161:6837/4127130864] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa273bb790 con 0x55aa2757bd20 2024-05-22T21:18:28.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9b4c00640 1 --2- 172.21.15.161:0/2492772906 >> [v2:172.21.15.161:6836/4127130864,v1:172.21.15.161:6837/4127130864] conn(0x55aa2757bd20 0x55aa2757c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff99e000640 10 client.6852 ms_handle_connect on v2:172.21.15.161:6836/4127130864 2024-05-22T21:18:28.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.239+0000 7ff9b4c00640 1 --2- 172.21.15.161:0/2492772906 >> [v2:172.21.15.161:6836/4127130864,v1:172.21.15.161:6837/4127130864] conn(0x55aa2757bd20 0x55aa2757c100 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7fc8eaccc063cdd server_cookie=5cc7214337968dc7 in_seq=0 out_seq=0 2024-05-22T21:18:28.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.243+0000 7ff99e000640 1 -- 172.21.15.161:0/2492772906 <== mds.0 v2:172.21.15.161:6836/4127130864 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aa273bb790 con 0x55aa2757bd20 2024-05-22T21:18:28.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.243+0000 7ff99e000640 10 client.6852 handle_command_reply: tid=0 2024-05-22T21:18:28.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.243+0000 7ff9baa1a000 1 client.6852 shutdown 2024-05-22T21:18:28.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.243+0000 7ff9baa1a000 1 -- 172.21.15.161:0/2492772906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275b99c0 msgr2=0x55aa275c1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.243+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/2492772906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275b99c0 0x55aa275c1260 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff99800ca20 tx=0x7ff998055660 comp rx=0 tx=0).stop 2024-05-22T21:18:28.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.243+0000 7ff9baa1a000 1 -- 172.21.15.161:0/2492772906 shutdown_connections 2024-05-22T21:18:28.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.243+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/2492772906 >> [v2:172.21.15.161:6836/4127130864,v1:172.21.15.161:6837/4127130864] conn(0x55aa2757bd20 0x55aa2757c100 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.243+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/2492772906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa275ba2e0 0x55aa2757afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.243+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/2492772906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275b99c0 0x55aa275c1260 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.243+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/2492772906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa2756fec0 0x55aa275c0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.243+0000 7ff99e000640 0 client.6852 ms_handle_reset on v2:172.21.15.161:6836/4127130864 2024-05-22T21:18:28.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.243+0000 7ff9baa1a000 1 -- 172.21.15.161:0/2492772906 >> 172.21.15.161:0/2492772906 conn(0x55aa27576440 msgr2=0x55aa2756e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:28.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.243+0000 7ff9baa1a000 1 -- 172.21.15.161:0/2492772906 shutdown_connections 2024-05-22T21:18:28.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.243+0000 7ff9baa1a000 1 -- 172.21.15.161:0/2492772906 wait complete. 2024-05-22T21:18:28.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.243+0000 7ff9baa1a000 20 client.6852 trim_cache size 0 max 16384 2024-05-22T21:18:28.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 Processor -- start 2024-05-22T21:18:28.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 -- start start 2024-05-22T21:18:28.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa2756fec0 0x55aa275b9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa275b99c0 0x55aa27575130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275ba2e0 0x55aa27575670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2707b940 con 0x55aa275ba2e0 2024-05-22T21:18:28.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa275bfc60 con 0x55aa275b99c0 2024-05-22T21:18:28.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa27454860 con 0x55aa2756fec0 2024-05-22T21:18:28.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9b5600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275ba2e0 0x55aa27575670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9b5600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275ba2e0 0x55aa27575670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50472/0 (socket says 172.21.15.161:50472) 2024-05-22T21:18:28.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9b5600640 1 -- 172.21.15.161:0/3288882922 learned_addr learned my addr 172.21.15.161:0/3288882922 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:28.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9a7e00640 1 --2- 172.21.15.161:0/3288882922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa275b99c0 0x55aa27575130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9b4c00640 1 --2- 172.21.15.161:0/3288882922 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa2756fec0 0x55aa275b9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9b5600640 1 -- 172.21.15.161:0/3288882922 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa2756fec0 msgr2=0x55aa275b9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:28.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9b5600640 1 --2- 172.21.15.161:0/3288882922 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa2756fec0 0x55aa275b9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9b5600640 1 -- 172.21.15.161:0/3288882922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa275b99c0 msgr2=0x55aa27575130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:28.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9b5600640 1 --2- 172.21.15.161:0/3288882922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa275b99c0 0x55aa27575130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9b5600640 1 -- 172.21.15.161:0/3288882922 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa275c07a0 con 0x55aa275ba2e0 2024-05-22T21:18:28.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9b5600640 1 --2- 172.21.15.161:0/3288882922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275ba2e0 0x55aa27575670 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80071b0 tx=0x7ff9a8001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7917ce103302e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:28.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9a6000640 1 -- 172.21.15.161:0/3288882922 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a80166f0 con 0x55aa275ba2e0 2024-05-22T21:18:28.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9a6000640 1 -- 172.21.15.161:0/3288882922 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a8016850 con 0x55aa275ba2e0 2024-05-22T21:18:28.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9a6000640 1 -- 172.21.15.161:0/3288882922 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8016c00 con 0x55aa275ba2e0 2024-05-22T21:18:28.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 -- 172.21.15.161:0/3288882922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275ba2e0 msgr2=0x55aa27575670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/3288882922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275ba2e0 0x55aa27575670 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80071b0 tx=0x7ff9a8001a30 comp rx=0 tx=0).stop 2024-05-22T21:18:28.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 -- 172.21.15.161:0/3288882922 shutdown_connections 2024-05-22T21:18:28.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/3288882922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275ba2e0 0x55aa27575670 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/3288882922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa275b99c0 0x55aa27575130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/3288882922 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa2756fec0 0x55aa275b9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 -- 172.21.15.161:0/3288882922 >> 172.21.15.161:0/3288882922 conn(0x55aa27576440 msgr2=0x55aa275b82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:28.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 -- 172.21.15.161:0/3288882922 shutdown_connections 2024-05-22T21:18:28.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 -- 172.21.15.161:0/3288882922 wait complete. 2024-05-22T21:18:28.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.259+0000 7ff9baa1a000 1 -- start start 2024-05-22T21:18:28.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 1 Processor -- start 2024-05-22T21:18:28.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa2756fec0 0x55aa275ff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa275b99c0 0x55aa275ff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275ba2e0 0x55aa275ffe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa273bbdb0 con 0x55aa275ba2e0 2024-05-22T21:18:28.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa273ba1a0 con 0x55aa275b99c0 2024-05-22T21:18:28.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa273c6d80 con 0x55aa2756fec0 2024-05-22T21:18:28.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 10 client.0 _ll_register_callbacks cb 0x7ff9a00ba050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:18:28.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9b5600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275ba2e0 0x55aa275ffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9b4c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa2756fec0 0x55aa275ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9b5600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275ba2e0 0x55aa275ffe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50480/0 (socket says 172.21.15.161:50480) 2024-05-22T21:18:28.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9b5600640 1 -- 172.21.15.161:0/1348391657 learned_addr learned my addr 172.21.15.161:0/1348391657 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:28.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9a7e00640 1 --2- 172.21.15.161:0/1348391657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa275b99c0 0x55aa275ff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9b5600640 1 -- 172.21.15.161:0/1348391657 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa2756fec0 msgr2=0x55aa275ff3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:28.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9b5600640 1 --2- 172.21.15.161:0/1348391657 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa2756fec0 0x55aa275ff3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9b5600640 1 -- 172.21.15.161:0/1348391657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa275b99c0 msgr2=0x55aa275ff8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:28.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9b5600640 1 --2- 172.21.15.161:0/1348391657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa275b99c0 0x55aa275ff8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9b5600640 1 -- 172.21.15.161:0/1348391657 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff994003530 con 0x55aa275ba2e0 2024-05-22T21:18:28.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9b5600640 1 --2- 172.21.15.161:0/1348391657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275ba2e0 0x55aa275ffe20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8001720 tx=0x7ff9a8001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c82d1684efe506 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:28.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff99e000640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:18:28.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff99e000640 1 -- 172.21.15.161:0/1348391657 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a80166f0 con 0x55aa275ba2e0 2024-05-22T21:18:28.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff99e000640 1 -- 172.21.15.161:0/1348391657 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9a8029aa0 con 0x55aa275ba2e0 2024-05-22T21:18:28.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 1 -- 172.21.15.161:0/1348391657 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa2738e460 con 0x55aa275ba2e0 2024-05-22T21:18:28.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff99e000640 1 -- 172.21.15.161:0/1348391657 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9a8029c80 con 0x55aa275ba2e0 2024-05-22T21:18:28.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 10 client.6858 fetch_fsmap learned FSMap version 47 2024-05-22T21:18:28.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 1 -- 172.21.15.161:0/1348391657 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55aa27369c00 con 0x55aa275ba2e0 2024-05-22T21:18:28.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff99e000640 1 -- 172.21.15.161:0/1348391657 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aa2738e460 con 0x55aa275ba2e0 2024-05-22T21:18:28.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff99e000640 1 -- 172.21.15.161:0/1348391657 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 47) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff9a801c020 con 0x55aa275ba2e0 2024-05-22T21:18:28.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 10 client.6858 fetch_fsmap finished waiting for FSMap version 47 2024-05-22T21:18:28.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 10 client.6858 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-22T21:18:28.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 20 client.6858 populate_metadata read hostname 'smithi161' 2024-05-22T21:18:28.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/1348391657 >> [v2:172.21.15.161:6836/4127130864,v1:172.21.15.161:6837/4127130864] conn(0x55aa275fed00 0x55aa27600360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 4 client.6858 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-22T21:18:28.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 1 -- 172.21.15.161:0/1348391657 --> [v2:172.21.15.161:6836/4127130864,v1:172.21.15.161:6837/4127130864] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55aa273879c0 con 0x55aa275fed00 2024-05-22T21:18:28.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9b4c00640 1 --2- 172.21.15.161:0/1348391657 >> [v2:172.21.15.161:6836/4127130864,v1:172.21.15.161:6837/4127130864] conn(0x55aa275fed00 0x55aa27600360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9b4c00640 1 --2- 172.21.15.161:0/1348391657 >> [v2:172.21.15.161:6836/4127130864,v1:172.21.15.161:6837/4127130864] conn(0x55aa275fed00 0x55aa27600360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99bd050d2a3bc7a5 server_cookie=cdeee0d2441bd66d in_seq=0 out_seq=0 2024-05-22T21:18:28.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff99e000640 10 client.6858 ms_handle_connect on v2:172.21.15.161:6836/4127130864 2024-05-22T21:18:28.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff99e000640 1 -- 172.21.15.161:0/1348391657 <== mds.0 v2:172.21.15.161:6836/4127130864 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55aa273879c0 con 0x55aa275fed00 2024-05-22T21:18:28.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff99e000640 10 client.6858 handle_command_reply: tid=0 2024-05-22T21:18:28.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.263+0000 7ff9baa1a000 1 client.6858 shutdown 2024-05-22T21:18:28.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9baa1a000 1 -- 172.21.15.161:0/1348391657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275ba2e0 msgr2=0x55aa275ffe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/1348391657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275ba2e0 0x55aa275ffe20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8001720 tx=0x7ff9a8001260 comp rx=0 tx=0).stop 2024-05-22T21:18:28.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9baa1a000 1 -- 172.21.15.161:0/1348391657 shutdown_connections 2024-05-22T21:18:28.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/1348391657 >> [v2:172.21.15.161:6836/4127130864,v1:172.21.15.161:6837/4127130864] conn(0x55aa275fed00 0x55aa27600360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/1348391657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa275ba2e0 0x55aa275ffe20 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/1348391657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55aa275b99c0 0x55aa275ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff99e000640 0 client.6858 ms_handle_reset on v2:172.21.15.161:6836/4127130864 2024-05-22T21:18:28.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9baa1a000 1 --2- 172.21.15.161:0/1348391657 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55aa2756fec0 0x55aa275ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9baa1a000 1 -- 172.21.15.161:0/1348391657 >> 172.21.15.161:0/1348391657 conn(0x55aa27576440 msgr2=0x55aa275b8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:28.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9baa1a000 1 -- 172.21.15.161:0/1348391657 shutdown_connections 2024-05-22T21:18:28.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9baa1a000 1 -- 172.21.15.161:0/1348391657 wait complete. 2024-05-22T21:18:28.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9baa1a000 20 client.6858 trim_cache size 0 max 16384 2024-05-22T21:18:28.273 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:18:28.273 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:18:28.273 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 0, 2024-05-22T21:18:28.273 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 6699, 2024-05-22T21:18:28.273 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2024-05-22T21:18:28.273 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2024-05-22T21:18:28.273 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2024-05-22T21:18:28.273 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 10.326342024000001, 2024-05-22T21:18:28.273 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 47, 2024-05-22T21:18:28.273 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 103, 2024-05-22T21:18:28.273 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 103, 2024-05-22T21:18:28.273 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 14.560626981 2024-05-22T21:18:28.273 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:18:28.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9b7600640 1 -- 172.21.15.161:0/284290209 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff98c066690 msgr2=0x7ff98c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9b7600640 1 --2- 172.21.15.161:0/284290209 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff98c066690 0x7ff98c068b50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00ad000 tx=0x7ff9a00a6000 comp rx=0 tx=0).stop 2024-05-22T21:18:28.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9b7600640 1 -- 172.21.15.161:0/284290209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b013fc80 msgr2=0x7ff9b0156a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9b7600640 1 --2- 172.21.15.161:0/284290209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b013fc80 0x7ff9b0156a50 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff99800ebe0 tx=0x7ff99800c520 comp rx=0 tx=0).stop 2024-05-22T21:18:28.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9b7600640 1 -- 172.21.15.161:0/284290209 shutdown_connections 2024-05-22T21:18:28.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9b7600640 1 --2- 172.21.15.161:0/284290209 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff98c066690 0x7ff98c068b50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9b7600640 1 --2- 172.21.15.161:0/284290209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b015af60 0x7ff9b00b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9b7600640 1 --2- 172.21.15.161:0/284290209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b013fc80 0x7ff9b0156a50 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9b7600640 1 --2- 172.21.15.161:0/284290209 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9b013cfc0 0x7ff9b0140c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9b7600640 1 -- 172.21.15.161:0/284290209 >> 172.21.15.161:0/284290209 conn(0x7ff9b006a2d0 msgr2=0x7ff9b00aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:28.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9b7600640 1 -- 172.21.15.161:0/284290209 shutdown_connections 2024-05-22T21:18:28.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.267+0000 7ff9b7600640 1 -- 172.21.15.161:0/284290209 wait complete. 2024-05-22T21:18:28.288 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:18:28.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.419+0000 7fdc3d200640 1 Processor -- start 2024-05-22T21:18:28.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.419+0000 7fdc3d200640 1 -- start start 2024-05-22T21:18:28.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.419+0000 7fdc3d200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdc3815c6d0 0x7fdc38164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.419+0000 7fdc3d200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc38071670 con 0x7fdc3815dd50 2024-05-22T21:18:28.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.419+0000 7fdc3d200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc38070a60 con 0x7fdc3815e130 2024-05-22T21:18:28.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.419+0000 7fdc3d200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc38070d40 con 0x7fdc3815c6d0 2024-05-22T21:18:28.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.419+0000 7fdc36800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdc3815c6d0 0x7fdc38164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.419+0000 7fdc36800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdc3815c6d0 0x7fdc38164e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50488/0 (socket says 172.21.15.161:50488) 2024-05-22T21:18:28.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.419+0000 7fdc36800640 1 -- 172.21.15.161:0/3852829372 learned_addr learned my addr 172.21.15.161:0/3852829372 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:28.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.419+0000 7fdc35e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fdc3815dd50 0x7fdc3806d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56996/0 (socket says 172.21.15.161:56996) 2024-05-22T21:18:28.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.419+0000 7fdc36800640 1 -- 172.21.15.161:0/3852829372 >> v1:172.21.15.179:6789/0 conn(0x7fdc3815e130 legacy=0x7fdc3815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.419+0000 7fdc36800640 1 -- 172.21.15.161:0/3852829372 >> v1:172.21.15.161:6789/0 conn(0x7fdc3815dd50 legacy=0x7fdc3806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.419+0000 7fdc36800640 1 -- 172.21.15.161:0/3852829372 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc38071020 con 0x7fdc3815c6d0 2024-05-22T21:18:28.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.423+0000 7fdc36800640 1 --2- 172.21.15.161:0/3852829372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc3815c6d0 0x7fdc38164e80 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009e50 tx=0x7fdc3001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9568506191a85b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:28.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.423+0000 7fdc35400640 1 -- 172.21.15.161:0/3852829372 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc300a5890 con 0x7fdc3815c6d0 2024-05-22T21:18:28.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.423+0000 7fdc35400640 1 -- 172.21.15.161:0/3852829372 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc300a59f0 con 0x7fdc3815c6d0 2024-05-22T21:18:28.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.423+0000 7fdc35400640 1 -- 172.21.15.161:0/3852829372 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc300a5d80 con 0x7fdc3815c6d0 2024-05-22T21:18:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.423+0000 7fdc3d200640 1 -- 172.21.15.161:0/3852829372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc3815c6d0 msgr2=0x7fdc38164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.423+0000 7fdc3d200640 1 --2- 172.21.15.161:0/3852829372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc3815c6d0 0x7fdc38164e80 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009e50 tx=0x7fdc3001ca00 comp rx=0 tx=0).stop 2024-05-22T21:18:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.423+0000 7fdc3d200640 1 -- 172.21.15.161:0/3852829372 shutdown_connections 2024-05-22T21:18:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.423+0000 7fdc3d200640 1 --2- 172.21.15.161:0/3852829372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc3815c6d0 0x7fdc38164e80 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.423+0000 7fdc3d200640 1 -- 172.21.15.161:0/3852829372 >> 172.21.15.161:0/3852829372 conn(0x7fdc380b0bc0 msgr2=0x7fdc380b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.423+0000 7fdc3d200640 1 -- 172.21.15.161:0/3852829372 shutdown_connections 2024-05-22T21:18:28.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.423+0000 7fdc3d200640 1 -- 172.21.15.161:0/3852829372 wait complete. 2024-05-22T21:18:28.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc3d200640 1 Processor -- start 2024-05-22T21:18:28.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc3d200640 1 -- start start 2024-05-22T21:18:28.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc3d200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc3815c6d0 0x7fdc380b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc3d200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc3815dd50 0x7fdc381531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc3d200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc3815e130 0x7fdc381536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc3d200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc38147e50 con 0x7fdc3815c6d0 2024-05-22T21:18:28.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc3d200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc381555a0 con 0x7fdc3815e130 2024-05-22T21:18:28.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc3d200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc38155840 con 0x7fdc3815dd50 2024-05-22T21:18:28.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc35e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc3815c6d0 0x7fdc380b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc35e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc3815c6d0 0x7fdc380b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50492/0 (socket says 172.21.15.161:50492) 2024-05-22T21:18:28.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc35e00640 1 -- 172.21.15.161:0/4182664542 learned_addr learned my addr 172.21.15.161:0/4182664542 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:28.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc35e00640 1 -- 172.21.15.161:0/4182664542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc3815dd50 msgr2=0x7fdc381531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc35e00640 1 --2- 172.21.15.161:0/4182664542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc3815dd50 0x7fdc381531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc35e00640 1 -- 172.21.15.161:0/4182664542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc3815e130 msgr2=0x7fdc381536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc35e00640 1 --2- 172.21.15.161:0/4182664542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc3815e130 0x7fdc381536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc35e00640 1 -- 172.21.15.161:0/4182664542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc30009b30 con 0x7fdc3815c6d0 2024-05-22T21:18:28.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc35e00640 1 --2- 172.21.15.161:0/4182664542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc3815c6d0 0x7fdc380b35d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800c430 tx=0x7fdc2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:28.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc2ea00640 1 -- 172.21.15.161:0/4182664542 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc280074e0 con 0x7fdc3815c6d0 2024-05-22T21:18:28.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc2ea00640 1 -- 172.21.15.161:0/4182664542 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc28007640 con 0x7fdc3815c6d0 2024-05-22T21:18:28.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc2ea00640 1 -- 172.21.15.161:0/4182664542 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc280049f0 con 0x7fdc3815c6d0 2024-05-22T21:18:28.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc3d200640 1 -- 172.21.15.161:0/4182664542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc38155b20 con 0x7fdc3815c6d0 2024-05-22T21:18:28.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.427+0000 7fdc3d200640 1 -- 172.21.15.161:0/4182664542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc381560e0 con 0x7fdc3815c6d0 2024-05-22T21:18:28.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.431+0000 7fdc3d200640 1 -- 172.21.15.161:0/4182664542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc38067220 con 0x7fdc3815c6d0 2024-05-22T21:18:28.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.431+0000 7fdc2ea00640 1 -- 172.21.15.161:0/4182664542 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdc28004b50 con 0x7fdc3815c6d0 2024-05-22T21:18:28.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.431+0000 7fdc2ea00640 1 --2- 172.21.15.161:0/4182664542 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc14066690 0x7fdc14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:28.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.431+0000 7fdc2ea00640 1 -- 172.21.15.161:0/4182664542 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc28083b90 con 0x7fdc3815c6d0 2024-05-22T21:18:28.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.431+0000 7fdc2d400640 1 --2- 172.21.15.161:0/4182664542 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc14066690 0x7fdc14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:28.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.431+0000 7fdc2d400640 1 --2- 172.21.15.161:0/4182664542 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc14066690 0x7fdc14068b50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdc20000f70 tx=0x7fdc200079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:28.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.435+0000 7fdc2ea00640 1 -- 172.21.15.161:0/4182664542 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdc28050f70 con 0x7fdc3815c6d0 2024-05-22T21:18:28.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.623+0000 7fdc3d200640 1 -- 172.21.15.161:0/4182664542 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdc3806c310 con 0x7fdc3815c6d0 2024-05-22T21:18:28.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.623+0000 7fdc2ea00640 1 -- 172.21.15.161:0/4182664542 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdc28054590 con 0x7fdc3815c6d0 2024-05-22T21:18:28.628 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:18:28.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.627+0000 7fdc3d200640 1 -- 172.21.15.161:0/4182664542 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc14066690 msgr2=0x7fdc14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.627+0000 7fdc3d200640 1 --2- 172.21.15.161:0/4182664542 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc14066690 0x7fdc14068b50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdc20000f70 tx=0x7fdc200079c0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.627+0000 7fdc3d200640 1 -- 172.21.15.161:0/4182664542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc3815c6d0 msgr2=0x7fdc380b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:28.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.627+0000 7fdc3d200640 1 --2- 172.21.15.161:0/4182664542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc3815c6d0 0x7fdc380b35d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800c430 tx=0x7fdc2800c8f0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.627+0000 7fdc3d200640 1 -- 172.21.15.161:0/4182664542 shutdown_connections 2024-05-22T21:18:28.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.627+0000 7fdc3d200640 1 --2- 172.21.15.161:0/4182664542 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc14066690 0x7fdc14068b50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.627+0000 7fdc3d200640 1 --2- 172.21.15.161:0/4182664542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc3815e130 0x7fdc381536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.627+0000 7fdc3d200640 1 --2- 172.21.15.161:0/4182664542 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc3815dd50 0x7fdc381531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.627+0000 7fdc3d200640 1 --2- 172.21.15.161:0/4182664542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc3815c6d0 0x7fdc380b35d0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:28.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.627+0000 7fdc3d200640 1 -- 172.21.15.161:0/4182664542 >> 172.21.15.161:0/4182664542 conn(0x7fdc380b0bc0 msgr2=0x7fdc38154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:28.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.627+0000 7fdc3d200640 1 -- 172.21.15.161:0/4182664542 shutdown_connections 2024-05-22T21:18:28.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:28.627+0000 7fdc3d200640 1 -- 172.21.15.161:0/4182664542 wait complete. 2024-05-22T21:18:28.649 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:18:28.871 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:18:28.871 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-05-22T21:18:28.871 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-22T21:18:28.872 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-22T21:18:28.872 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:18:28.872 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:18:28.872 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:18:28.872 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:18:28.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:28.872 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-05-22T21:18:28.884 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:18:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:18:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:18:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:18:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-05-22T21:18:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:18:28.884 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:18:28.884 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-05-22T21:18:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2024-05-22T21:18:28.885 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:18:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:18:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2024-05-22T21:18:28.885 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:18:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:18:28.885 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:18:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:18:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:18:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42183sec preferred_lft 42183sec 2024-05-22T21:18:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-05-22T21:18:28.885 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:18:28.886 INFO:teuthology.orchestra.run.smithi161.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:18:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 02:42:b9:0f:57:df brd ff:ff:ff:ff:ff:ff 2024-05-22T21:18:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:18:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:18:28.886 INFO:teuthology.orchestra.run.smithi161.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:18:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:18:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:18:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:18:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:18:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:18:28.886 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:18:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-22T21:18:28.886 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:18:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:18:28.887 INFO:teuthology.orchestra.run.smithi161.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:18:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-22T21:18:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:18:28.887 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:18:28.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:28.887 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:18:28.946 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-05-22T21:18:28.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:28.949 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-05-22T21:18:29.008 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-05-22T21:18:29.008 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:18:29.008 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:18:29.008 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:18:29.008 INFO:teuthology.orchestra.run.smithi161.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:18:29.008 INFO:teuthology.orchestra.run.smithi161.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:18:29.008 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:18:29.008 INFO:teuthology.orchestra.run.smithi161.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:18:29.008 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:18:29.008 INFO:teuthology.orchestra.run.smithi161.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:18:29.009 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:18:29.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:29.009 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:18:29.009 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:18:29.009 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:18:29.009 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:18:29.009 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:18:29.009 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:18:29.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:29.100 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-05-22T21:18:29.109 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-22T21:18:29.109 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-22T21:18:29.109 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:18:29.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:29.110 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2024-05-22T21:18:29.171 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:18:29.171 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-22T21:18:29.220 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:18:29.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:18:29.220 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:18:29.271 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:18:29.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:29.271 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:18:29.359 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:18:29.360 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:18:29.360 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:18:29.378 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:18:29.378 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:18:29.378 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:18:29.378 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:18:29.378 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:18:29.387 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:18:29.387 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:18:29.387 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:18:29.387 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:18:29.387 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:18:29.387 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:18:29.387 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:18:29.387 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:18:29.391 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:18:29.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:29.392 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:18:29.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:29.411 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:18:29.472 INFO:teuthology.orchestra.run.smithi161.stdout:client6610 2024-05-22T21:18:29.473 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:18:29.473 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:29.524 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:18:29.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.699+0000 7fb685400640 1 Processor -- start 2024-05-22T21:18:29.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.699+0000 7fb685400640 1 -- start start 2024-05-22T21:18:29.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.699+0000 7fb685400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb68015c6d0 0x7fb680164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:29.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.699+0000 7fb685400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb680071670 con 0x7fb68015dd50 2024-05-22T21:18:29.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.699+0000 7fb685400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb680070a60 con 0x7fb68015e130 2024-05-22T21:18:29.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.699+0000 7fb685400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb680070d40 con 0x7fb68015c6d0 2024-05-22T21:18:29.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.699+0000 7fb67e800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb68015dd50 0x7fb68006d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:57010/0 (socket says 172.21.15.161:57010) 2024-05-22T21:18:29.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.699+0000 7fb67e800640 1 -- 172.21.15.161:0/1028623686 learned_addr learned my addr 172.21.15.161:0/1028623686 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:29.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.699+0000 7fb67f200640 1 --2- 172.21.15.161:0/1028623686 >> v2:172.21.15.179:3300/0 conn(0x7fb68015c6d0 0x7fb680164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:29.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67de00640 1 -- 172.21.15.161:0/1028623686 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603277945 0 0) 0x7fb680071670 con 0x7fb68015dd50 2024-05-22T21:18:29.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67de00640 1 -- 172.21.15.161:0/1028623686 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb664003530 con 0x7fb68015dd50 2024-05-22T21:18:29.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67de00640 1 -- 172.21.15.161:0/1028623686 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 190702097 0 0) 0x7fb664003530 con 0x7fb68015dd50 2024-05-22T21:18:29.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67de00640 1 -- 172.21.15.161:0/1028623686 >> v2:172.21.15.179:3300/0 conn(0x7fb68015c6d0 msgr2=0x7fb680164e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:29.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67de00640 1 --2- 172.21.15.161:0/1028623686 >> v2:172.21.15.179:3300/0 conn(0x7fb68015c6d0 0x7fb680164e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:29.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67de00640 1 -- 172.21.15.161:0/1028623686 >> v1:172.21.15.179:6789/0 conn(0x7fb68015e130 legacy=0x7fb68015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:29.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67de00640 1 -- 172.21.15.161:0/1028623686 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb680071020 con 0x7fb68015dd50 2024-05-22T21:18:29.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67de00640 1 -- 172.21.15.161:0/1028623686 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fb670001a00 con 0x7fb68015dd50 2024-05-22T21:18:29.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67de00640 1 -- 172.21.15.161:0/1028623686 >> v1:172.21.15.161:6789/0 conn(0x7fb68015dd50 legacy=0x7fb68006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:29.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67de00640 1 --2- 172.21.15.161:0/1028623686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb664003980 0x7fb664003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:29.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67de00640 1 --2- 172.21.15.161:0/1028623686 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6640047d0 0x7fb664004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:29.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67de00640 1 --2- 172.21.15.161:0/1028623686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb664005190 0x7fb66400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:29.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67de00640 1 -- 172.21.15.161:0/1028623686 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb680070d40 con 0x7fb664003980 2024-05-22T21:18:29.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67de00640 1 -- 172.21.15.161:0/1028623686 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66400c030 con 0x7fb664005190 2024-05-22T21:18:29.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67de00640 1 -- 172.21.15.161:0/1028623686 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66400e400 con 0x7fb6640047d0 2024-05-22T21:18:29.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67e800640 1 --2- 172.21.15.161:0/1028623686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb664003980 0x7fb664003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:29.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67f200640 1 --2- 172.21.15.161:0/1028623686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb664005190 0x7fb66400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:29.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb675e00640 1 --2- 172.21.15.161:0/1028623686 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6640047d0 0x7fb664004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:29.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67e800640 1 -- 172.21.15.161:0/1028623686 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6640047d0 msgr2=0x7fb664004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:29.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67e800640 1 --2- 172.21.15.161:0/1028623686 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6640047d0 0x7fb664004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:29.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67e800640 1 -- 172.21.15.161:0/1028623686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb664005190 msgr2=0x7fb66400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:29.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67e800640 1 --2- 172.21.15.161:0/1028623686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb664005190 0x7fb66400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:29.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67e800640 1 -- 172.21.15.161:0/1028623686 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb664003530 con 0x7fb664003980 2024-05-22T21:18:29.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67f200640 1 --2- 172.21.15.161:0/1028623686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb664005190 0x7fb66400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:29.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67f200640 1 -- 172.21.15.161:0/1028623686 reap_dead start 2024-05-22T21:18:29.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.703+0000 7fb67e800640 1 --2- 172.21.15.161:0/1028623686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb664003980 0x7fb664003da0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb67000a830 tx=0x7fb67001d2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7080e918fc8ce7df server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:29.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb67de00640 1 -- 172.21.15.161:0/1028623686 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6700072d0 con 0x7fb664003980 2024-05-22T21:18:29.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb67de00640 1 -- 172.21.15.161:0/1028623686 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6700a8070 con 0x7fb664003980 2024-05-22T21:18:29.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb67de00640 1 -- 172.21.15.161:0/1028623686 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6700076f0 con 0x7fb664003980 2024-05-22T21:18:29.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb685400640 1 -- 172.21.15.161:0/1028623686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb664003980 msgr2=0x7fb664003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:29.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb685400640 1 --2- 172.21.15.161:0/1028623686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb664003980 0x7fb664003da0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb67000a830 tx=0x7fb67001d2f0 comp rx=0 tx=0).stop 2024-05-22T21:18:29.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb685400640 1 -- 172.21.15.161:0/1028623686 shutdown_connections 2024-05-22T21:18:29.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb685400640 1 --2- 172.21.15.161:0/1028623686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb664003980 0x7fb664003da0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:29.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb685400640 1 -- 172.21.15.161:0/1028623686 >> 172.21.15.161:0/1028623686 conn(0x7fb6800b0bc0 msgr2=0x7fb6800b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:29.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb685400640 1 -- 172.21.15.161:0/1028623686 shutdown_connections 2024-05-22T21:18:29.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb685400640 1 -- 172.21.15.161:0/1028623686 wait complete. 2024-05-22T21:18:29.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb685400640 1 Processor -- start 2024-05-22T21:18:29.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb685400640 1 -- start start 2024-05-22T21:18:29.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb685400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb664003980 0x7fb68013e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:29.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb685400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6800b1600 0x7fb680150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:29.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb685400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6800b1a80 0x7fb680150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:29.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb685400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800ce070 con 0x7fb664003980 2024-05-22T21:18:29.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb685400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68014f340 con 0x7fb6800b1600 2024-05-22T21:18:29.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb685400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800cd180 con 0x7fb6800b1a80 2024-05-22T21:18:29.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb67e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb664003980 0x7fb68013e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:29.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb67e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb664003980 0x7fb68013e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50502/0 (socket says 172.21.15.161:50502) 2024-05-22T21:18:29.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb67e800640 1 -- 172.21.15.161:0/2408229977 learned_addr learned my addr 172.21.15.161:0/2408229977 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:29.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb675e00640 1 --2- 172.21.15.161:0/2408229977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6800b1600 0x7fb680150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:29.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.707+0000 7fb67f200640 1 --2- 172.21.15.161:0/2408229977 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6800b1a80 0x7fb680150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:29.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.711+0000 7fb67e800640 1 -- 172.21.15.161:0/2408229977 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6800b1a80 msgr2=0x7fb680150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:29.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.711+0000 7fb67e800640 1 --2- 172.21.15.161:0/2408229977 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6800b1a80 0x7fb680150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:29.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.711+0000 7fb67e800640 1 -- 172.21.15.161:0/2408229977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6800b1600 msgr2=0x7fb680150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:29.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.711+0000 7fb67e800640 1 --2- 172.21.15.161:0/2408229977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6800b1600 0x7fb680150420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:29.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.711+0000 7fb67e800640 1 -- 172.21.15.161:0/2408229977 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6800cd460 con 0x7fb664003980 2024-05-22T21:18:29.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.711+0000 7fb67e800640 1 --2- 172.21.15.161:0/2408229977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb664003980 0x7fb68013e6f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb67001d330 tx=0x7fb664005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:29.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.711+0000 7fb677400640 1 -- 172.21.15.161:0/2408229977 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6700070d0 con 0x7fb664003980 2024-05-22T21:18:29.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.711+0000 7fb677400640 1 -- 172.21.15.161:0/2408229977 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb670007250 con 0x7fb664003980 2024-05-22T21:18:29.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.711+0000 7fb685400640 1 -- 172.21.15.161:0/2408229977 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6800cd740 con 0x7fb664003980 2024-05-22T21:18:29.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.711+0000 7fb677400640 1 -- 172.21.15.161:0/2408229977 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670007600 con 0x7fb664003980 2024-05-22T21:18:29.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.711+0000 7fb685400640 1 -- 172.21.15.161:0/2408229977 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6800cdd00 con 0x7fb664003980 2024-05-22T21:18:29.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.711+0000 7fb685400640 1 -- 172.21.15.161:0/2408229977 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb680067220 con 0x7fb664003980 2024-05-22T21:18:29.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.711+0000 7fb677400640 1 -- 172.21.15.161:0/2408229977 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb6700aa030 con 0x7fb664003980 2024-05-22T21:18:29.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.711+0000 7fb677400640 1 --2- 172.21.15.161:0/2408229977 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb65c066690 0x7fb65c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:29.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.711+0000 7fb677400640 1 -- 172.21.15.161:0/2408229977 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb67011e5e0 con 0x7fb664003980 2024-05-22T21:18:29.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.715+0000 7fb675e00640 1 --2- 172.21.15.161:0/2408229977 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb65c066690 0x7fb65c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:29.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.715+0000 7fb675e00640 1 --2- 172.21.15.161:0/2408229977 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb65c066690 0x7fb65c068b50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb668007fd0 tx=0x7fb668009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:29.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.715+0000 7fb677400640 1 -- 172.21.15.161:0/2408229977 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb6700eb960 con 0x7fb664003980 2024-05-22T21:18:29.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.903+0000 7fb685400640 1 -- 172.21.15.161:0/2408229977 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb68006c310 con 0x7fb664003980 2024-05-22T21:18:29.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.903+0000 7fb677400640 1 -- 172.21.15.161:0/2408229977 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb6700eef80 con 0x7fb664003980 2024-05-22T21:18:29.908 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:18:29.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.907+0000 7fb685400640 1 -- 172.21.15.161:0/2408229977 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb65c066690 msgr2=0x7fb65c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:29.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.907+0000 7fb685400640 1 --2- 172.21.15.161:0/2408229977 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb65c066690 0x7fb65c068b50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb668007fd0 tx=0x7fb668009040 comp rx=0 tx=0).stop 2024-05-22T21:18:29.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.907+0000 7fb685400640 1 -- 172.21.15.161:0/2408229977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb664003980 msgr2=0x7fb68013e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:29.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.907+0000 7fb685400640 1 --2- 172.21.15.161:0/2408229977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb664003980 0x7fb68013e6f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb67001d330 tx=0x7fb664005800 comp rx=0 tx=0).stop 2024-05-22T21:18:29.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.907+0000 7fb685400640 1 -- 172.21.15.161:0/2408229977 shutdown_connections 2024-05-22T21:18:29.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.907+0000 7fb685400640 1 --2- 172.21.15.161:0/2408229977 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb65c066690 0x7fb65c068b50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:29.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.907+0000 7fb685400640 1 --2- 172.21.15.161:0/2408229977 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6800b1a80 0x7fb680150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:29.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.907+0000 7fb685400640 1 --2- 172.21.15.161:0/2408229977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6800b1600 0x7fb680150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:29.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.907+0000 7fb685400640 1 --2- 172.21.15.161:0/2408229977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb664003980 0x7fb68013e6f0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:29.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.907+0000 7fb685400640 1 -- 172.21.15.161:0/2408229977 >> 172.21.15.161:0/2408229977 conn(0x7fb6800b0bc0 msgr2=0x7fb680158e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:29.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.907+0000 7fb685400640 1 -- 172.21.15.161:0/2408229977 shutdown_connections 2024-05-22T21:18:29.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:29.907+0000 7fb685400640 1 -- 172.21.15.161:0/2408229977 wait complete. 2024-05-22T21:18:29.929 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:18:29.929 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:18:29.929 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-22T21:18:29.929 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-22T21:18:29.929 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-22T21:18:29.929 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:18:29.929 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:18:29.929 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:18:29.929 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:18:29.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:29.930 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-22T21:18:29.939 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:18:29.940 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:18:29.940 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:18:29.940 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:18:29.940 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-22T21:18:29.940 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:18:29.940 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:18:29.940 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:18:29.940 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f0 2024-05-22T21:18:29.940 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:18:29.940 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:18:29.940 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f1 2024-05-22T21:18:29.940 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:18:29.941 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-22T21:18:29.941 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:18:29.941 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-22T21:18:29.941 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:18:29.941 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42175sec preferred_lft 42175sec 2024-05-22T21:18:29.941 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-22T21:18:29.941 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:18:29.941 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:18:29.941 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:bd:4e:25:3d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:18:29.941 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:18:29.941 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:18:29.941 INFO:teuthology.orchestra.run.smithi179.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:18:29.942 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:18:29.942 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:18:29.942 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:18:29.942 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:18:29.942 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:18:29.942 INFO:teuthology.orchestra.run.smithi179.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:18:29.942 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-22T21:18:29.942 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:18:29.942 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:18:29.942 INFO:teuthology.orchestra.run.smithi179.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:18:29.942 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-22T21:18:29.942 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:18:29.942 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:18:29.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:29.944 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:18:30.009 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-22T21:18:30.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:30.012 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-22T21:18:30.064 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-22T21:18:30.064 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:18:30.064 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:18:30.065 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:18:30.065 INFO:teuthology.orchestra.run.smithi179.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:18:30.065 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:18:30.065 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:18:30.065 INFO:teuthology.orchestra.run.smithi179.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:18:30.065 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:18:30.065 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:18:30.065 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:18:30.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:30.065 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:18:30.065 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:18:30.066 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:18:30.066 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:18:30.066 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:18:30.066 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:18:30.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:30.158 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-22T21:18:30.167 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-22T21:18:30.167 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-22T21:18:30.168 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:18:30.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:30.168 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-22T21:18:30.228 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:18:30.229 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-22T21:18:30.281 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:18:30.281 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:18:30.281 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:18:30.333 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:18:30.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:30.333 DEBUG:teuthology.orchestra.run.smithi179:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:18:30.427 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:18:30.427 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:18:30.427 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:18:30.443 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:18:30.443 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:18:30.443 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:18:30.443 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:18:30.443 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:18:30.451 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:18:30.451 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:18:30.451 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:18:30.451 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:18:30.451 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:18:30.451 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:18:30.452 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:18:30.452 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:18:30.455 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:18:30.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:30.456 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:18:30.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:30.475 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:18:30.532 INFO:teuthology.orchestra.run.smithi179.stdout:client6622 2024-05-22T21:18:30.533 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:18:30.533 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:30.585 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:18:30.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.723+0000 7f29f5800640 1 Processor -- start 2024-05-22T21:18:30.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.723+0000 7f29f5800640 1 -- start start 2024-05-22T21:18:30.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.723+0000 7f29f5800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f29f00d0a40 0x7f29f00d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:30.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.723+0000 7f29f5800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29f00cc320 0x7f29f00cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:30.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.723+0000 7f29f5800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f00cb8a0 con 0x7f29f015af00 2024-05-22T21:18:30.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.723+0000 7f29f5800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29f00d13f0 con 0x7f29f00d0a40 2024-05-22T21:18:30.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.723+0000 7f29f5800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f015f530 con 0x7f29f00cc320 2024-05-22T21:18:30.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.723+0000 7f29efe00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f29f015af00 0x7f29f015eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:57024/0 (socket says 172.21.15.161:57024) 2024-05-22T21:18:30.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.723+0000 7f29efe00640 1 -- 172.21.15.161:0/3381036034 learned_addr learned my addr 172.21.15.161:0/3381036034 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:30.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.723+0000 7f29ef400640 1 --2- 172.21.15.161:0/3381036034 >> v2:172.21.15.161:3300/0 conn(0x7f29f00d0a40 0x7f29f00d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:30.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.723+0000 7f29f4e00640 1 --2- 172.21.15.161:0/3381036034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29f00cc320 0x7f29f00cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:30.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.723+0000 7f29ef400640 1 -- 172.21.15.161:0/3381036034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29f00cc320 msgr2=0x7f29f00cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:30.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.723+0000 7f29ef400640 1 --2- 172.21.15.161:0/3381036034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29f00cc320 0x7f29f00cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:30.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.723+0000 7f29ef400640 1 -- 172.21.15.161:0/3381036034 >> v1:172.21.15.161:6789/0 conn(0x7f29f015af00 legacy=0x7f29f015eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:30.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.723+0000 7f29ef400640 1 -- 172.21.15.161:0/3381036034 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f015f810 con 0x7f29f00d0a40 2024-05-22T21:18:30.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.727+0000 7f29ef400640 1 --2- 172.21.15.161:0/3381036034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29f00d0a40 0x7f29f00d0eb0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f29dc0097c0 tx=0x7f29dc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9532aade8e17a466 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:30.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.727+0000 7f29eea00640 1 -- 172.21.15.161:0/3381036034 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0ab660 con 0x7f29f00d0a40 2024-05-22T21:18:30.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.727+0000 7f29eea00640 1 -- 172.21.15.161:0/3381036034 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc0ab7c0 con 0x7f29f00d0a40 2024-05-22T21:18:30.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.727+0000 7f29eea00640 1 -- 172.21.15.161:0/3381036034 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0abb50 con 0x7f29f00d0a40 2024-05-22T21:18:30.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.727+0000 7f29f5800640 1 -- 172.21.15.161:0/3381036034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29f00d0a40 msgr2=0x7f29f00d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:30.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.727+0000 7f29f5800640 1 --2- 172.21.15.161:0/3381036034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29f00d0a40 0x7f29f00d0eb0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f29dc0097c0 tx=0x7f29dc01c460 comp rx=0 tx=0).stop 2024-05-22T21:18:30.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.727+0000 7f29f5800640 1 -- 172.21.15.161:0/3381036034 shutdown_connections 2024-05-22T21:18:30.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.727+0000 7f29f5800640 1 --2- 172.21.15.161:0/3381036034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29f00d0a40 0x7f29f00d0eb0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:30.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.727+0000 7f29f5800640 1 --2- 172.21.15.161:0/3381036034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29f00cc320 0x7f29f00cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:30.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.727+0000 7f29f5800640 1 -- 172.21.15.161:0/3381036034 >> 172.21.15.161:0/3381036034 conn(0x7f29f0099a40 msgr2=0x7f29f0099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:30.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.727+0000 7f29f5800640 1 -- 172.21.15.161:0/3381036034 shutdown_connections 2024-05-22T21:18:30.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.727+0000 7f29f5800640 1 -- 172.21.15.161:0/3381036034 wait complete. 2024-05-22T21:18:30.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.727+0000 7f29f5800640 1 Processor -- start 2024-05-22T21:18:30.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.727+0000 7f29f5800640 1 -- start start 2024-05-22T21:18:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29f5800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29f00cc320 0x7f29f00c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29f5800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29f00d0a40 0x7f29f00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29f5800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29f015af00 0x7f29f0156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29f5800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f015c630 con 0x7f29f00d0a40 2024-05-22T21:18:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29f5800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f00b5420 con 0x7f29f00cc320 2024-05-22T21:18:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29f5800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f00b3c20 con 0x7f29f015af00 2024-05-22T21:18:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29ef400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29f00d0a40 0x7f29f00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29ef400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29f00d0a40 0x7f29f00b4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50522/0 (socket says 172.21.15.161:50522) 2024-05-22T21:18:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29ef400640 1 -- 172.21.15.161:0/2356873344 learned_addr learned my addr 172.21.15.161:0/2356873344 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29f4e00640 1 --2- 172.21.15.161:0/2356873344 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29f015af00 0x7f29f0156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:30.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29efe00640 1 --2- 172.21.15.161:0/2356873344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29f00cc320 0x7f29f00c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:30.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29ef400640 1 -- 172.21.15.161:0/2356873344 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29f015af00 msgr2=0x7f29f0156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:30.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29ef400640 1 --2- 172.21.15.161:0/2356873344 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29f015af00 0x7f29f0156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:30.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29ef400640 1 -- 172.21.15.161:0/2356873344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29f00cc320 msgr2=0x7f29f00c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:30.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29ef400640 1 --2- 172.21.15.161:0/2356873344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29f00cc320 0x7f29f00c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:30.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29ef400640 1 -- 172.21.15.161:0/2356873344 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc009420 con 0x7f29f00d0a40 2024-05-22T21:18:30.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29ef400640 1 --2- 172.21.15.161:0/2356873344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29f00d0a40 0x7f29f00b4ee0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f29dc0ad040 tx=0x7f29dc01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:30.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29e7e00640 1 -- 172.21.15.161:0/2356873344 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0bf610 con 0x7f29f00d0a40 2024-05-22T21:18:30.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29e7e00640 1 -- 172.21.15.161:0/2356873344 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc0bf7b0 con 0x7f29f00d0a40 2024-05-22T21:18:30.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29e7e00640 1 -- 172.21.15.161:0/2356873344 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0bfb60 con 0x7f29f00d0a40 2024-05-22T21:18:30.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29f5800640 1 -- 172.21.15.161:0/2356873344 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f00b3f00 con 0x7f29f00d0a40 2024-05-22T21:18:30.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29f5800640 1 -- 172.21.15.161:0/2356873344 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f0156660 con 0x7f29f00d0a40 2024-05-22T21:18:30.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.731+0000 7f29f5800640 1 -- 172.21.15.161:0/2356873344 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f00b3d60 con 0x7f29f00d0a40 2024-05-22T21:18:30.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.735+0000 7f29e7e00640 1 -- 172.21.15.161:0/2356873344 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f29dc0a9070 con 0x7f29f00d0a40 2024-05-22T21:18:30.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.735+0000 7f29e7e00640 1 --2- 172.21.15.161:0/2356873344 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f29d0066690 0x7f29d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:30.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.735+0000 7f29e7e00640 1 -- 172.21.15.161:0/2356873344 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29dc11eaf0 con 0x7f29f00d0a40 2024-05-22T21:18:30.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.735+0000 7f29efe00640 1 --2- 172.21.15.161:0/2356873344 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f29d0066690 0x7f29d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:30.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.735+0000 7f29efe00640 1 --2- 172.21.15.161:0/2356873344 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f29d0066690 0x7f29d0068b50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f29e0005890 tx=0x7f29e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:30.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.739+0000 7f29e7e00640 1 -- 172.21.15.161:0/2356873344 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f29f00b3d60 con 0x7f29f00d0a40 2024-05-22T21:18:30.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.927+0000 7f29f5800640 1 -- 172.21.15.161:0/2356873344 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29f00b3d60 con 0x7f29f00d0a40 2024-05-22T21:18:30.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.927+0000 7f29e7e00640 1 -- 172.21.15.161:0/2356873344 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f29f00b3d60 con 0x7f29f00d0a40 2024-05-22T21:18:30.930 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:30.930 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2771409351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2771409351},{"type":"v1","addr":"172.21.15.179:6834","nonce":2771409351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6442,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2520119431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2520119431},{"type":"v1","addr":"172.21.15.179:6835","nonce":2520119431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1415362547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1415362547},{"type":"v1","addr":"172.21.15.161:6835","nonce":1415362547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:18:17.938858+0000","modified":"2024-05-22T21:18:20.963065+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/4127130864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4127130864},{"type":"v1","addr":"172.21.15.161:6837","nonce":4127130864}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-22T21:18:30.930 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 47 2024-05-22T21:18:30.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.927+0000 7f29f5800640 1 -- 172.21.15.161:0/2356873344 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f29d0066690 msgr2=0x7f29d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.927+0000 7f29f5800640 1 --2- 172.21.15.161:0/2356873344 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f29d0066690 0x7f29d0068b50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f29e0005890 tx=0x7f29e0009040 comp rx=0 tx=0).stop 2024-05-22T21:18:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.927+0000 7f29f5800640 1 -- 172.21.15.161:0/2356873344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29f00d0a40 msgr2=0x7f29f00b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.927+0000 7f29f5800640 1 --2- 172.21.15.161:0/2356873344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29f00d0a40 0x7f29f00b4ee0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f29dc0ad040 tx=0x7f29dc01aca0 comp rx=0 tx=0).stop 2024-05-22T21:18:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.927+0000 7f29f5800640 1 -- 172.21.15.161:0/2356873344 shutdown_connections 2024-05-22T21:18:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.927+0000 7f29f5800640 1 --2- 172.21.15.161:0/2356873344 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f29d0066690 0x7f29d0068b50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.927+0000 7f29f5800640 1 --2- 172.21.15.161:0/2356873344 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29f015af00 0x7f29f0156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.927+0000 7f29f5800640 1 --2- 172.21.15.161:0/2356873344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29f00d0a40 0x7f29f00b4ee0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.927+0000 7f29f5800640 1 --2- 172.21.15.161:0/2356873344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29f00cc320 0x7f29f00c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.927+0000 7f29f5800640 1 -- 172.21.15.161:0/2356873344 >> 172.21.15.161:0/2356873344 conn(0x7f29f0099a40 msgr2=0x7f29f0165580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.927+0000 7f29f5800640 1 -- 172.21.15.161:0/2356873344 shutdown_connections 2024-05-22T21:18:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:30.927+0000 7f29f5800640 1 -- 172.21.15.161:0/2356873344 wait complete. 2024-05-22T21:18:30.952 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-05-22T21:18:31.106 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":6699,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":13.148649041000001,"mdsmap_epoch":47,"osdmap_epoch":103,"osdmap_epoch_barrier":103,"uptime":17.382936535999999} 2024-05-22T21:18:31.106 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 6699, "mdsmap_epoch": 47, "osdmap_epoch": 103, "osdmap_epoch_barrier": 103, "rank_uptime": 13.148649041, "state": "up:active", "uptime": 17.382936536, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:18:31.106 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-05-22T21:18:31.259 INFO:teuthology.orchestra.run.smithi161.stdout:{"osd_mon_report_interval":"5"} 2024-05-22T21:18:31.259 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-05-22T21:18:31.259 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2024-05-22T21:18:31.259 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:18:31.259 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:31.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:31.268 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2024-05-22T21:18:33.749 INFO:teuthology.orchestra.run.smithi161.stderr:265+0 records in 2024-05-22T21:18:33.750 INFO:teuthology.orchestra.run.smithi161.stderr:265+0 records out 2024-05-22T21:18:33.750 INFO:teuthology.orchestra.run.smithi161.stderr:277872640 bytes (278 MB, 265 MiB) copied, 2.42624 s, 115 MB/s 2024-05-22T21:18:33.751 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:18:33.751 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:33.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:33.759 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2024-05-22T21:18:36.329 INFO:teuthology.orchestra.run.smithi161.stderr:292+0 records in 2024-05-22T21:18:36.330 INFO:teuthology.orchestra.run.smithi161.stderr:292+0 records out 2024-05-22T21:18:36.330 INFO:teuthology.orchestra.run.smithi161.stderr:306184192 bytes (306 MB, 292 MiB) copied, 2.51633 s, 122 MB/s 2024-05-22T21:18:36.331 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-05-22T21:18:36.331 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:18:36.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.463+0000 7f8d77200640 1 Processor -- start 2024-05-22T21:18:36.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.463+0000 7f8d77200640 1 -- start start 2024-05-22T21:18:36.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.463+0000 7f8d77200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d780d0a40 0x7f8d780d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:36.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.463+0000 7f8d77200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8d780cc320 0x7f8d780cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:36.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.463+0000 7f8d77200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d780cb8a0 con 0x7f8d7815af00 2024-05-22T21:18:36.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.463+0000 7f8d77200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d780d13f0 con 0x7f8d780cc320 2024-05-22T21:18:36.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.463+0000 7f8d77200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7815f530 con 0x7f8d780d0a40 2024-05-22T21:18:36.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.463+0000 7f8d76800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8d780cc320 0x7f8d780cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:36.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.463+0000 7f8d76800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8d780cc320 0x7f8d780cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38070/0 (socket says 172.21.15.161:38070) 2024-05-22T21:18:36.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.463+0000 7f8d75e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8d7815af00 0x7f8d7815eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:38128/0 (socket says 172.21.15.161:38128) 2024-05-22T21:18:36.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.463+0000 7f8d76800640 1 -- 172.21.15.161:0/1662690300 learned_addr learned my addr 172.21.15.161:0/1662690300 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:36.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.463+0000 7f8d75400640 1 --2- 172.21.15.161:0/1662690300 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d780d0a40 0x7f8d780d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:36.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d76800640 1 -- 172.21.15.161:0/1662690300 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d780d0a40 msgr2=0x7f8d780d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:36.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d76800640 1 --2- 172.21.15.161:0/1662690300 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d780d0a40 0x7f8d780d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:36.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d76800640 1 -- 172.21.15.161:0/1662690300 >> v1:172.21.15.161:6789/0 conn(0x7f8d7815af00 legacy=0x7f8d7815eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:36.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d76800640 1 -- 172.21.15.161:0/1662690300 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7815f810 con 0x7f8d780cc320 2024-05-22T21:18:36.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d75400640 1 --2- 172.21.15.161:0/1662690300 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d780d0a40 0x7f8d780d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:36.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d74a00640 1 -- 172.21.15.161:0/1662690300 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235254940 0 0) 0x7f8d780cb8a0 con 0x7f8d7815af00 2024-05-22T21:18:36.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d76800640 1 --2- 172.21.15.161:0/1662690300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d780cc320 0x7f8d780cc770 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009e50 tx=0x7f8d7001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c1cb2834fd3ad6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:36.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d74a00640 1 -- 172.21.15.161:0/1662690300 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700a5900 con 0x7f8d780cc320 2024-05-22T21:18:36.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d74a00640 1 -- 172.21.15.161:0/1662690300 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d700a5a60 con 0x7f8d780cc320 2024-05-22T21:18:36.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d74a00640 1 -- 172.21.15.161:0/1662690300 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700a5df0 con 0x7f8d780cc320 2024-05-22T21:18:36.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d77200640 1 -- 172.21.15.161:0/1662690300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d780cc320 msgr2=0x7f8d780cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:36.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d77200640 1 --2- 172.21.15.161:0/1662690300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d780cc320 0x7f8d780cc770 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009e50 tx=0x7f8d7001ca30 comp rx=0 tx=0).stop 2024-05-22T21:18:36.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d77200640 1 -- 172.21.15.161:0/1662690300 shutdown_connections 2024-05-22T21:18:36.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d77200640 1 --2- 172.21.15.161:0/1662690300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d780cc320 0x7f8d780cc770 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:36.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d77200640 1 --2- 172.21.15.161:0/1662690300 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d780d0a40 0x7f8d780d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:36.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d77200640 1 -- 172.21.15.161:0/1662690300 >> 172.21.15.161:0/1662690300 conn(0x7f8d78099a40 msgr2=0x7f8d78099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:36.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d77200640 1 -- 172.21.15.161:0/1662690300 shutdown_connections 2024-05-22T21:18:36.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.467+0000 7f8d77200640 1 -- 172.21.15.161:0/1662690300 wait complete. 2024-05-22T21:18:36.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d77200640 1 Processor -- start 2024-05-22T21:18:36.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d77200640 1 -- start start 2024-05-22T21:18:36.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d77200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d780cc320 0x7f8d781f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:36.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d77200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d780d0a40 0x7f8d780c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:36.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d77200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d7815af00 0x7f8d78158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:36.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d77200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d78159090 con 0x7f8d780d0a40 2024-05-22T21:18:36.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d77200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d781f9070 con 0x7f8d7815af00 2024-05-22T21:18:36.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d77200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d781f9350 con 0x7f8d780cc320 2024-05-22T21:18:36.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d75400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d780d0a40 0x7f8d780c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:36.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d75400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d780d0a40 0x7f8d780c5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38084/0 (socket says 172.21.15.161:38084) 2024-05-22T21:18:36.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d75400640 1 -- 172.21.15.161:0/2857969630 learned_addr learned my addr 172.21.15.161:0/2857969630 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:36.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d75e00640 1 --2- 172.21.15.161:0/2857969630 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d780cc320 0x7f8d781f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:36.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d76800640 1 --2- 172.21.15.161:0/2857969630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d7815af00 0x7f8d78158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:36.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d75400640 1 -- 172.21.15.161:0/2857969630 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d780cc320 msgr2=0x7f8d781f8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:36.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d75400640 1 --2- 172.21.15.161:0/2857969630 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d780cc320 0x7f8d781f8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:36.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d75400640 1 -- 172.21.15.161:0/2857969630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d7815af00 msgr2=0x7f8d78158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:36.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d75400640 1 --2- 172.21.15.161:0/2857969630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d7815af00 0x7f8d78158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:36.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d75400640 1 -- 172.21.15.161:0/2857969630 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d70009b30 con 0x7f8d780d0a40 2024-05-22T21:18:36.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d75400640 1 --2- 172.21.15.161:0/2857969630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d780d0a40 0x7f8d780c5be0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400e810 tx=0x7f8d6400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:36.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d6e000640 1 -- 172.21.15.161:0/2857969630 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6400c8d0 con 0x7f8d780d0a40 2024-05-22T21:18:36.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d6e000640 1 -- 172.21.15.161:0/2857969630 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6400ca30 con 0x7f8d780d0a40 2024-05-22T21:18:36.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d6e000640 1 -- 172.21.15.161:0/2857969630 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6400cde0 con 0x7f8d780d0a40 2024-05-22T21:18:36.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.471+0000 7f8d77200640 1 -- 172.21.15.161:0/2857969630 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d781f9630 con 0x7f8d780d0a40 2024-05-22T21:18:36.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.475+0000 7f8d77200640 1 -- 172.21.15.161:0/2857969630 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d78156590 con 0x7f8d780d0a40 2024-05-22T21:18:36.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.475+0000 7f8d77200640 1 -- 172.21.15.161:0/2857969630 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d78067070 con 0x7f8d780d0a40 2024-05-22T21:18:36.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.475+0000 7f8d6e000640 1 -- 172.21.15.161:0/2857969630 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8d64013410 con 0x7f8d780d0a40 2024-05-22T21:18:36.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.475+0000 7f8d6e000640 1 --2- 172.21.15.161:0/2857969630 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8d58066690 0x7f8d58068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:36.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.475+0000 7f8d6e000640 1 -- 172.21.15.161:0/2857969630 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d64083fb0 con 0x7f8d780d0a40 2024-05-22T21:18:36.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.475+0000 7f8d75e00640 1 --2- 172.21.15.161:0/2857969630 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8d58066690 0x7f8d58068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:36.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.475+0000 7f8d75e00640 1 --2- 172.21.15.161:0/2857969630 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8d58066690 0x7f8d58068b50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8d68000f70 tx=0x7f8d68007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:36.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.479+0000 7f8d6e000640 1 -- 172.21.15.161:0/2857969630 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8d64051330 con 0x7f8d780d0a40 2024-05-22T21:18:36.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.635+0000 7f8d77200640 1 -- 172.21.15.161:0/2857969630 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8d780b6d10 con 0x7f8d780d0a40 2024-05-22T21:18:36.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.639+0000 7f8d6e000640 1 -- 172.21.15.161:0/2857969630 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v103) v1 ==== 81+0+33586 (secure 0 0 0) 0x7f8d64054950 con 0x7f8d780d0a40 2024-05-22T21:18:36.642 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:36.642 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:18:36.643 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 103, 2024-05-22T21:18:36.643 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:18:36.643 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:18:36.643 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:18:18.942765+0000", 2024-05-22T21:18:36.643 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:18:36.643 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:18:36.643 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:18:36.643 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:18:36.643 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:18:36.643 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:18:36.643 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:18:36.644 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:18:36.644 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:18:36.644 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:36.644 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:18:36.644 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:18:36.644 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:18:36.644 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:18:36.644 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:18:36.644 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 13, 2024-05-22T21:18:36.644 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:18:36.644 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:18:36.644 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:18:36.644 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:18:36.644 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:18:36.645 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:18:36.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.645 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:18:36.645 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:18:36.645 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:18:36.645 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:18:36.645 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:18:36.645 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:36.645 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:36.645 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:36.645 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:36.645 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:36.645 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:36.646 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:36.646 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:36.646 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:36.646 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:36.646 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:18:36.646 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:18:36.646 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:18:36.646 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:18:36.646 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:18:36.646 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:36.646 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:36.646 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:36.647 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:36.647 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:36.647 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:36.647 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:36.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.647 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "91", 2024-05-22T21:18:36.647 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:36.647 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:36.647 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:36.647 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:36.647 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:36.647 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:36.647 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:36.647 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:36.647 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:36.648 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:36.648 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:36.648 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:36.648 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:36.648 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:36.648 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:36.648 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:36.648 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:36.648 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:36.648 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:36.648 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:36.648 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:36.648 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:36.648 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:36.649 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:36.649 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:36.649 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:36.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.649 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:36.649 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:36.649 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:36.649 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:36.649 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:36.649 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:36.649 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:36.649 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:36.649 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:36.650 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:36.650 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:36.650 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:36.650 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:18:36.650 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:18:36.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.650 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:36.650 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:18:36.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.650 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:36.650 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:18:36.650 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:18:36.650 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:18:36.651 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:18:36.651 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:18:36.651 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:36.651 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:36.651 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:36.651 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.651 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 12, 2024-05-22T21:18:36.651 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:18:36.651 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:18:16.500534+0000", 2024-05-22T21:18:36.651 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:18:36.652 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:18:36.652 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:36.652 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:36.652 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:36.652 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:36.652 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:36.652 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:36.652 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:36.652 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:36.652 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:36.652 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:36.652 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:18:36.652 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:18:36.653 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:18:36.653 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:18:36.653 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:18:36.653 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:36.653 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:36.653 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:36.653 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:36.653 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:36.653 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:36.653 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:36.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.653 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "102", 2024-05-22T21:18:36.654 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:36.654 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:36.654 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:36.654 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:36.654 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:36.654 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:36.654 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:36.654 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:36.654 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:36.654 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:36.654 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:36.654 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:36.654 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:36.654 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:36.655 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:36.655 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:36.655 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:36.655 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:36.655 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:36.655 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:36.655 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:36.655 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:36.655 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:36.655 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:36.655 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:36.655 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:36.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.656 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:36.656 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:36.656 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:36.656 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:36.656 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:36.656 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:36.656 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:36.656 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:36.656 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:36.656 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:36.656 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:36.656 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:36.656 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:18:36.656 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:18:36.657 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:18:36.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.657 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:36.657 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:18:36.657 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:18:36.657 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.657 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.657 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:36.657 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 2, 2024-05-22T21:18:36.657 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 2, 2024-05-22T21:18:36.657 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:18:36.657 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:18:36.657 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:18:36.657 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:36.658 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:36.658 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:36.658 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.658 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.658 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 13, 2024-05-22T21:18:36.658 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:18:36.658 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:18:16.937445+0000", 2024-05-22T21:18:36.658 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:18:36.658 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:18:36.658 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:36.658 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:36.658 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:36.658 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:36.658 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:36.659 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:36.659 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:36.659 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:36.659 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:36.659 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:36.659 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:18:36.659 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:18:36.659 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:18:36.659 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:18:36.659 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:18:36.659 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:36.659 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:36.659 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:36.659 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:36.660 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:36.660 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:36.660 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:36.660 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.660 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "103", 2024-05-22T21:18:36.660 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:36.660 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:36.660 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:36.660 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:36.660 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:36.660 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:36.660 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:36.660 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:36.660 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:36.661 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:36.661 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:36.661 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:36.661 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:36.661 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:36.661 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:36.661 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:36.661 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:36.661 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:36.661 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:36.661 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:36.661 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:36.661 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:36.662 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:36.662 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:36.662 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:36.662 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:36.662 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.662 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:36.662 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:36.662 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:36.662 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:36.662 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:36.662 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:36.662 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:36.662 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:36.662 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:36.663 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:36.663 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:36.663 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:36.663 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:18:36.663 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:18:36.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.663 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:36.663 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:18:36.663 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:18:36.663 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.663 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.663 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:36.663 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.3799999952316284, 2024-05-22T21:18:36.663 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.3799999952316284, 2024-05-22T21:18:36.663 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:18:36.664 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.3799999952316284, 2024-05-22T21:18:36.664 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.3799999952316284, 2024-05-22T21:18:36.664 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:36.664 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:36.664 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:36.664 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.664 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.664 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:36.664 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:18:36.664 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.664 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:18:36.664 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:36.664 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:36.664 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:36.664 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:36.665 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:36.665 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:36.665 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:36.665 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:36.665 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:36.665 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:36.665 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:36.665 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:36.665 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.665 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.665 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.665 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:18:36.665 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:36.665 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.666 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.666 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:18:36.666 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:36.666 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.666 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.666 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:36.666 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.666 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.666 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:18:36.666 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:36.666 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.666 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.667 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.667 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:18:36.667 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:36.667 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.667 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.667 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:36.667 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.667 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.667 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:18:36.667 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:36.667 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.667 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.667 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.668 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:18:36.668 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:36.668 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.668 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.668 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:36.668 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.668 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.668 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:18:36.668 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:36.668 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.668 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.668 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.669 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:18:36.669 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:36.669 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.669 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.669 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:18:36.669 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:18:36.669 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:18:36.669 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:18:36.669 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:36.669 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:36.669 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:36.669 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.669 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.669 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.670 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:18:36.670 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:36.670 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:36.670 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:36.670 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:36.670 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:36.670 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:36.670 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:36.670 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:36.670 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:36.670 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:36.670 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:36.670 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:36.671 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.671 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.671 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:18:36.671 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:36.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.671 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.671 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.671 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:18:36.671 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:36.671 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.671 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.671 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.671 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:36.672 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.672 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.672 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:18:36.672 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:36.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.672 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.672 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.672 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:18:36.672 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:36.672 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.672 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.672 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.672 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:36.672 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.673 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.673 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:18:36.673 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:36.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.673 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.673 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.673 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:18:36.673 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:36.673 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.673 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.673 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.673 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:36.673 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.674 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.674 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:18:36.674 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:36.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.674 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.674 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.674 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:18:36.674 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:36.674 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.674 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.674 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.674 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:18:36.675 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:18:36.675 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:18:36.675 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:18:36.675 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:36.675 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:36.675 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:36.675 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.675 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.675 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.675 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:18:36.675 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:36.675 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:36.675 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:36.675 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:36.675 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:36.676 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:36.676 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:36.676 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:36.676 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:36.676 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:36.676 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:36.676 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:36.676 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.676 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.676 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:18:36.676 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:36.676 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.676 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.676 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.677 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:18:36.677 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:36.677 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.677 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.677 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:36.677 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.677 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.677 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:18:36.677 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:36.677 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.677 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.677 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.678 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:18:36.678 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:36.678 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.678 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.678 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:36.678 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.678 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.678 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:18:36.678 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:36.678 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.678 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.678 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.679 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:18:36.679 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:36.679 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.679 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.679 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:36.679 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.679 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.679 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:18:36.679 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:36.679 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.679 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.679 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.680 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:18:36.680 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:36.680 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.680 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.680 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:18:36.680 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:18:36.680 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:18:36.680 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:18:36.680 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:36.680 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:36.680 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:36.680 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.680 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.681 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.681 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:18:36.681 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:36.681 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:36.681 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:36.681 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:36.681 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:36.681 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:36.681 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:36.681 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:36.681 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:36.681 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:36.681 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:36.682 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:36.682 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.682 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.682 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:18:36.682 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:36.682 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.682 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.682 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.682 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:18:36.682 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:36.682 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.683 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.683 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:36.683 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.683 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.683 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:18:36.683 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:36.683 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.683 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.683 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.683 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:18:36.684 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:36.684 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.684 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.684 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:36.684 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.684 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.684 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:18:36.684 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:36.684 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.684 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.684 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.684 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:18:36.685 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:36.685 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.685 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.685 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:36.685 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.685 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.685 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:18:36.685 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:36.685 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.685 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.685 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.685 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:18:36.685 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:36.686 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.686 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.686 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:18:36.686 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:18:36.686 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:18:36.686 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:18:36.686 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:36.686 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:36.686 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:36.686 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.686 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.687 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.687 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:18:36.687 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:36.687 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:36.687 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:36.687 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:36.687 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:36.687 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:36.687 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:36.687 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:36.687 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:36.687 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:36.687 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:36.687 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:36.688 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.688 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.688 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:18:36.688 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:36.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.688 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.688 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.688 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:18:36.688 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:36.688 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.688 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.688 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.688 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:36.689 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.689 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.689 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:18:36.689 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:36.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.689 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.689 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.689 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:18:36.689 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:36.689 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.689 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.689 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.689 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:36.690 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.690 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.690 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:18:36.690 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:36.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.690 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.690 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.690 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:18:36.690 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:36.690 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.690 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.690 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.690 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:36.690 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.691 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.691 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:18:36.691 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:36.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.691 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.691 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.691 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:18:36.691 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:36.691 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.691 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.691 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.691 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:18:36.691 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:18:36.691 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:18:36.692 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:18:36.692 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:36.692 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:36.692 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:36.692 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.692 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.692 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.692 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:18:36.692 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:36.692 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:36.692 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:36.692 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:36.692 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:36.692 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:36.693 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:36.693 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:36.693 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:36.693 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:36.693 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:36.693 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:36.693 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.693 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.693 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:18:36.693 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:36.693 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.693 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.693 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.693 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:18:36.694 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:36.694 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.694 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.694 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:36.694 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.694 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.694 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:18:36.694 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:36.694 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.694 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.694 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.694 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:18:36.695 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:36.695 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.695 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.695 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:36.695 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.695 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.695 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:18:36.695 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:36.695 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.695 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.695 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.695 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:18:36.695 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:36.696 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.696 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.696 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:36.696 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.696 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.696 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:18:36.696 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:36.696 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.696 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.696 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.696 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:18:36.696 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:36.696 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.697 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.697 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:18:36.697 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:18:36.697 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:18:36.697 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:18:36.697 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:36.697 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:36.697 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:36.697 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.697 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.697 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.697 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:18:36.697 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:36.697 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:36.698 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:36.698 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:36.698 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:36.698 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:36.698 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:36.698 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:36.698 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:36.698 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:36.698 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:36.698 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:36.698 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.698 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.698 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.698 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:18:36.698 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:36.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.699 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.699 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:18:36.699 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:36.699 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.699 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.699 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:36.699 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.699 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.699 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.699 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:18:36.699 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:36.699 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.700 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.700 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:18:36.700 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:36.700 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.700 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.700 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:36.700 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.700 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.700 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:18:36.700 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:36.700 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.700 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.701 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.701 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:18:36.701 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:36.701 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.701 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.701 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:36.701 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.701 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.701 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:18:36.701 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:36.701 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.701 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.702 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.702 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:18:36.702 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:36.702 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.702 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.702 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.702 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:18:36.702 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:18:36.702 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:18:36.702 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:18:36.702 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:36.702 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:36.703 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:36.703 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.703 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.703 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:18:36.703 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:36.703 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:36.703 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:36.703 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:36.703 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:36.703 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:36.703 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:36.703 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:36.704 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:36.704 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:36.704 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:36.704 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:36.704 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.704 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.704 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:18:36.704 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:36.704 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.704 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.704 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.704 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:18:36.704 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:36.704 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.705 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.705 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:36.705 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.705 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.705 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:18:36.705 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:36.705 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.705 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.705 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.705 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:18:36.705 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:36.705 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.706 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.706 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:36.706 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.706 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.706 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:18:36.706 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:36.706 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.706 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.706 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.706 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:18:36.706 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:36.707 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.707 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.707 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:36.707 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:36.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.707 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:36.707 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:18:36.707 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:36.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.707 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.707 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:36.707 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:18:36.707 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:36.708 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.708 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.708 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:18:36.708 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:18:36.708 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:18:36.708 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:18:36.708 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:36.708 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:36.708 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:36.708 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:36.708 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.708 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:36.708 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:18:36.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.709 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:18:36.709 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:36.709 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:36.709 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:36.709 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:36.709 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:36.709 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:18:36.709 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:36.709 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.709 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.709 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:18:36.709 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:36.709 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:36.710 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:36.710 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:36.710 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:36.710 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:18:36.710 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:36.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.710 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.710 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:18:36.710 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:36.710 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:36.710 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:36.710 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:36.710 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:36.711 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:18:36.711 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:36.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.711 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.711 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:18:36.711 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:36.711 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:36.711 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:36.711 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:36.711 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:36.711 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:18:36.711 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:36.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.712 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:18:36.712 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:36.712 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:36.712 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:36.712 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:36.712 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:36.712 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:18:36.712 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:36.712 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.712 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.712 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:18:36.712 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:36.713 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:36.713 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:36.713 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:36.713 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:36.713 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:18:36.713 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:36.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.713 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.713 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:18:36.713 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:36.713 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:36.713 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:36.713 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:36.713 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:36.714 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:18:36.714 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:36.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.714 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:36.714 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:18:36.714 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:36.714 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:36.714 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:36.714 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:36.714 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:36.714 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:18:36.714 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:36.714 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.715 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:36.715 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:18:36.715 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:18:36.715 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:18:36.715 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:18:36.715 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:18:36.715 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:18:36.715 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:18:36.715 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:18:36.715 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:18:36.715 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:18:36.715 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:18:36.715 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:18:36.715 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:18:36.716 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:18:36.716 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:36.716 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:18:36.716 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:18:36.716 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:18:36.716 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:18:36.716 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:18:36.716 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:18:36.716 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:18:36.716 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:18:36.716 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:18:36.716 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:18:36.716 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:18:36.717 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:36.717 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:18:36.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.639+0000 7f8d77200640 1 -- 172.21.15.161:0/2857969630 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8d58066690 msgr2=0x7f8d58068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:36.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.639+0000 7f8d77200640 1 --2- 172.21.15.161:0/2857969630 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8d58066690 0x7f8d58068b50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8d68000f70 tx=0x7f8d68007df0 comp rx=0 tx=0).stop 2024-05-22T21:18:36.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.639+0000 7f8d77200640 1 -- 172.21.15.161:0/2857969630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d780d0a40 msgr2=0x7f8d780c5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:36.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.639+0000 7f8d77200640 1 --2- 172.21.15.161:0/2857969630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d780d0a40 0x7f8d780c5be0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400e810 tx=0x7f8d6400ecd0 comp rx=0 tx=0).stop 2024-05-22T21:18:36.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.639+0000 7f8d77200640 1 -- 172.21.15.161:0/2857969630 shutdown_connections 2024-05-22T21:18:36.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.639+0000 7f8d77200640 1 --2- 172.21.15.161:0/2857969630 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8d58066690 0x7f8d58068b50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:36.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.639+0000 7f8d77200640 1 --2- 172.21.15.161:0/2857969630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d7815af00 0x7f8d78158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:36.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.639+0000 7f8d77200640 1 --2- 172.21.15.161:0/2857969630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d780d0a40 0x7f8d780c5be0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:36.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.639+0000 7f8d77200640 1 --2- 172.21.15.161:0/2857969630 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d780cc320 0x7f8d781f8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:36.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.639+0000 7f8d77200640 1 -- 172.21.15.161:0/2857969630 >> 172.21.15.161:0/2857969630 conn(0x7f8d78099a40 msgr2=0x7f8d781575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:36.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.639+0000 7f8d77200640 1 -- 172.21.15.161:0/2857969630 shutdown_connections 2024-05-22T21:18:36.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:36.639+0000 7f8d77200640 1 -- 172.21.15.161:0/2857969630 wait complete. 2024-05-22T21:18:36.719 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-05-22T21:18:39.572 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:18:39.571+0000 7f7d35a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-05-22T21:18:41.721 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:18:41.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.863+0000 7f552ba00640 1 Processor -- start 2024-05-22T21:18:41.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.863+0000 7f552ba00640 1 -- start start 2024-05-22T21:18:41.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.863+0000 7f552ba00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f552415dd50 0x7f552406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:41.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.863+0000 7f552ba00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f552415d5e0 0x7f552415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:41.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.863+0000 7f552ba00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5524071670 con 0x7f552415e130 2024-05-22T21:18:41.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.863+0000 7f552ba00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552406de40 con 0x7f552415dd50 2024-05-22T21:18:41.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.863+0000 7f552ba00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5524070a60 con 0x7f552415d5e0 2024-05-22T21:18:41.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.863+0000 7f5523e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f552415e130 0x7f552415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:38136/0 (socket says 172.21.15.161:38136) 2024-05-22T21:18:41.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.863+0000 7f5523e00640 1 -- 172.21.15.161:0/2652134384 learned_addr learned my addr 172.21.15.161:0/2652134384 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:41.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.863+0000 7f5529a00640 1 --2- 172.21.15.161:0/2652134384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f552415d5e0 0x7f552415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:41.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.863+0000 7f5529000640 1 --2- 172.21.15.161:0/2652134384 >> v2:172.21.15.179:3300/0 conn(0x7f552415dd50 0x7f552406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:41.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.863+0000 7f5523400640 1 -- 172.21.15.161:0/2652134384 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848447850 0 0) 0x7f5524071670 con 0x7f552415e130 2024-05-22T21:18:41.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.867+0000 7f5523400640 1 -- 172.21.15.161:0/2652134384 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5508003530 con 0x7f552415e130 2024-05-22T21:18:41.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.867+0000 7f5529000640 1 -- 172.21.15.161:0/2652134384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f552415d5e0 msgr2=0x7f552415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:41.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.867+0000 7f5529000640 1 --2- 172.21.15.161:0/2652134384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f552415d5e0 0x7f552415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:41.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.867+0000 7f5529000640 1 -- 172.21.15.161:0/2652134384 >> v1:172.21.15.161:6789/0 conn(0x7f552415e130 legacy=0x7f552415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:41.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.867+0000 7f5523400640 1 -- 172.21.15.161:0/2652134384 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1626079836 0 0) 0x7f5508003530 con 0x7f552415e130 2024-05-22T21:18:41.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.867+0000 7f5529000640 1 -- 172.21.15.161:0/2652134384 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5524070d40 con 0x7f552415dd50 2024-05-22T21:18:41.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.867+0000 7f5529a00640 1 --2- 172.21.15.161:0/2652134384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f552415d5e0 0x7f552415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:41.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.867+0000 7f5529000640 1 --2- 172.21.15.161:0/2652134384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415dd50 0x7f552406d900 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f55140097c0 tx=0x7f551401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6de0e779d9781038 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:41.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.867+0000 7f5523400640 1 -- 172.21.15.161:0/2652134384 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5510001680 con 0x7f552415dd50 2024-05-22T21:18:41.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.867+0000 7f5523400640 1 -- 172.21.15.161:0/2652134384 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55140a7050 con 0x7f552415dd50 2024-05-22T21:18:41.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.867+0000 7f5523400640 1 -- 172.21.15.161:0/2652134384 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55140ab620 con 0x7f552415dd50 2024-05-22T21:18:41.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f552ba00640 1 -- 172.21.15.161:0/2652134384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415dd50 msgr2=0x7f552406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:41.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f552ba00640 1 --2- 172.21.15.161:0/2652134384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415dd50 0x7f552406d900 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f55140097c0 tx=0x7f551401c390 comp rx=0 tx=0).stop 2024-05-22T21:18:41.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f552ba00640 1 -- 172.21.15.161:0/2652134384 shutdown_connections 2024-05-22T21:18:41.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f552ba00640 1 --2- 172.21.15.161:0/2652134384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f552415d5e0 0x7f552415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:41.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f552ba00640 1 --2- 172.21.15.161:0/2652134384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415dd50 0x7f552406d900 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:41.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f552ba00640 1 -- 172.21.15.161:0/2652134384 >> 172.21.15.161:0/2652134384 conn(0x7f55240b0bc0 msgr2=0x7f55240b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:41.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f552ba00640 1 -- 172.21.15.161:0/2652134384 shutdown_connections 2024-05-22T21:18:41.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f552ba00640 1 -- 172.21.15.161:0/2652134384 wait complete. 2024-05-22T21:18:41.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f552ba00640 1 Processor -- start 2024-05-22T21:18:41.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f552ba00640 1 -- start start 2024-05-22T21:18:41.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f552ba00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415d5e0 0x7f55241475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:41.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f552ba00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f552415dd50 0x7f552413f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:41.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f552ba00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f552415e130 0x7f5524149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:41.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f552ba00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5524143dd0 con 0x7f552415dd50 2024-05-22T21:18:41.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f552ba00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5524147c40 con 0x7f552415d5e0 2024-05-22T21:18:41.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f552ba00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5524147f20 con 0x7f552415e130 2024-05-22T21:18:41.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f5523e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f552415dd50 0x7f552413f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:41.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f5523e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f552415dd50 0x7f552413f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38088/0 (socket says 172.21.15.161:38088) 2024-05-22T21:18:41.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f5523e00640 1 -- 172.21.15.161:0/414924003 learned_addr learned my addr 172.21.15.161:0/414924003 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:41.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f5523e00640 1 -- 172.21.15.161:0/414924003 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f552415e130 msgr2=0x7f5524149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:41.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f5523e00640 1 --2- 172.21.15.161:0/414924003 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f552415e130 0x7f5524149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:41.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f5523e00640 1 -- 172.21.15.161:0/414924003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415d5e0 msgr2=0x7f55241475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:41.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f5523e00640 1 --2- 172.21.15.161:0/414924003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415d5e0 0x7f55241475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:41.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.871+0000 7f5523e00640 1 -- 172.21.15.161:0/414924003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c009a70 con 0x7f552415dd50 2024-05-22T21:18:41.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.875+0000 7f5523e00640 1 --2- 172.21.15.161:0/414924003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f552415dd50 0x7f552413f930 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f551000c750 tx=0x7f551000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:41.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.875+0000 7f5520c00640 1 -- 172.21.15.161:0/414924003 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5510005ae0 con 0x7f552415dd50 2024-05-22T21:18:41.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.875+0000 7f5520c00640 1 -- 172.21.15.161:0/414924003 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5510005c40 con 0x7f552415dd50 2024-05-22T21:18:41.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.875+0000 7f552ba00640 1 -- 172.21.15.161:0/414924003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5514009420 con 0x7f552415dd50 2024-05-22T21:18:41.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.875+0000 7f5520c00640 1 -- 172.21.15.161:0/414924003 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5510007070 con 0x7f552415dd50 2024-05-22T21:18:41.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.875+0000 7f552ba00640 1 -- 172.21.15.161:0/414924003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55241484e0 con 0x7f552415dd50 2024-05-22T21:18:41.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.875+0000 7f552ba00640 1 -- 172.21.15.161:0/414924003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5524148700 con 0x7f552415dd50 2024-05-22T21:18:41.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.875+0000 7f5520c00640 1 -- 172.21.15.161:0/414924003 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f55100071f0 con 0x7f552415dd50 2024-05-22T21:18:41.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.875+0000 7f5520c00640 1 --2- 172.21.15.161:0/414924003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5504066690 0x7f5504068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:41.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.875+0000 7f5520c00640 1 -- 172.21.15.161:0/414924003 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5510083ff0 con 0x7f552415dd50 2024-05-22T21:18:41.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.875+0000 7f5529000640 1 --2- 172.21.15.161:0/414924003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5504066690 0x7f5504068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:41.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.875+0000 7f5520c00640 1 -- 172.21.15.161:0/414924003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=108}) v3 -- 0x7f550406e1c0 con 0x7f552415dd50 2024-05-22T21:18:41.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.879+0000 7f5529000640 1 --2- 172.21.15.161:0/414924003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5504066690 0x7f5504068b50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f55140ad000 tx=0x7f5514004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:41.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:41.879+0000 7f5520c00640 1 -- 172.21.15.161:0/414924003 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f55100512c0 con 0x7f552415dd50 2024-05-22T21:18:42.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.039+0000 7f552ba00640 1 -- 172.21.15.161:0/414924003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f55240706b0 con 0x7f552415dd50 2024-05-22T21:18:42.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.039+0000 7f5520c00640 1 -- 172.21.15.161:0/414924003 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v107) v1 ==== 81+0+33730 (secure 0 0 0) 0x7f55100548e0 con 0x7f552415dd50 2024-05-22T21:18:42.044 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:42.044 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:18:42.044 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 107, 2024-05-22T21:18:42.044 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:18:42.044 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:18:42.044 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:18:41.584848+0000", 2024-05-22T21:18:42.044 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:18:42.044 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:18:42.044 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:18:42.044 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:18:42.044 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:18:42.044 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:18:42.045 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:18:42.045 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:18:42.045 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:18:42.045 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:42.045 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:18:42.045 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:18:42.045 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:18:42.045 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:18:42.045 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:18:42.045 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 13, 2024-05-22T21:18:42.045 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:18:42.045 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:18:42.045 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:18:42.045 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:18:42.046 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:18:42.046 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:18:42.046 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.046 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:18:42.046 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:18:42.046 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:18:42.046 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:18:42.046 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:18:42.046 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:42.046 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:42.046 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:42.046 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:42.046 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:42.047 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:42.047 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:42.047 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:42.047 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:42.047 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:42.047 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:18:42.047 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:18:42.047 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:18:42.047 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:18:42.047 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:18:42.047 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:42.047 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:42.047 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:42.047 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:42.047 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:42.048 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:42.048 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:42.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.048 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "105", 2024-05-22T21:18:42.048 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:42.048 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:42.048 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:42.048 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:42.048 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:42.048 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:42.048 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:42.048 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:42.048 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:42.048 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:42.049 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:42.049 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:42.049 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:42.049 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:42.049 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:42.049 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:42.049 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:42.049 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:42.049 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:42.049 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:42.049 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:42.049 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:42.049 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:42.049 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:42.050 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:42.050 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:42.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.050 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:42.050 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:42.050 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:42.050 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:42.050 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:42.050 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:42.050 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:42.050 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:42.051 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:42.051 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:42.051 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:42.051 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:42.051 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:18:42.051 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:18:42.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.051 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:42.051 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:18:42.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.051 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:42.051 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:18:42.051 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:18:42.052 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:18:42.052 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:18:42.052 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:18:42.052 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:42.052 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:42.052 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:42.052 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.052 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.052 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 12, 2024-05-22T21:18:42.052 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:18:42.052 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:18:16.500534+0000", 2024-05-22T21:18:42.052 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:18:42.052 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:18:42.053 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:42.053 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:42.053 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:42.053 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:42.053 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:42.053 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:42.053 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:42.053 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:42.053 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:42.053 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:42.053 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:18:42.053 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:18:42.053 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:18:42.054 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:18:42.054 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:18:42.054 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:42.054 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:42.054 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:42.054 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:42.054 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:42.054 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:42.054 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:42.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.054 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "105", 2024-05-22T21:18:42.054 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:42.054 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:42.054 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:42.055 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:42.055 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:42.055 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:42.055 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:42.055 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:42.055 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:42.055 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:42.055 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:42.055 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:42.055 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:42.055 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:42.055 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:42.055 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:42.055 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:42.055 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:42.056 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:42.056 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:42.056 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:42.056 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:42.056 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:42.056 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:42.056 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:42.056 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:42.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.056 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:42.056 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:42.056 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:42.056 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:42.057 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:42.057 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:42.057 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:42.057 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:42.057 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:42.057 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:42.057 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:42.057 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:42.057 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:18:42.057 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:18:42.057 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:18:42.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.057 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:42.057 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:18:42.057 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:18:42.058 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.058 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.058 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:42.058 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 2, 2024-05-22T21:18:42.058 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 2, 2024-05-22T21:18:42.058 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:18:42.058 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:18:42.058 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:18:42.058 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:42.058 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:42.058 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:42.058 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.058 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.058 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.059 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 13, 2024-05-22T21:18:42.059 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:18:42.059 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:18:16.937445+0000", 2024-05-22T21:18:42.059 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:18:42.059 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:18:42.059 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:42.059 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:42.059 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:42.059 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:42.059 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:42.059 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:42.059 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:42.059 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:42.059 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:42.059 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:18:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:18:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:18:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:18:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:18:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "105", 2024-05-22T21:18:42.060 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:42.061 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:42.061 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:42.061 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:42.061 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:42.061 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:42.061 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:42.061 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:42.061 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:42.061 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:42.061 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:42.061 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:42.061 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:42.061 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:42.062 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:42.062 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:42.062 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:42.062 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:42.062 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:42.062 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:42.062 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:42.062 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:42.062 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:42.062 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:42.062 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:42.062 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:42.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.062 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:42.063 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:42.063 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:42.063 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:42.063 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:42.063 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:42.063 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:42.063 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:42.063 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:42.063 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:42.063 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:42.063 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:42.063 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:18:42.063 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:18:42.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.064 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:42.064 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:18:42.064 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:18:42.064 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.064 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:42.064 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.3799999952316284, 2024-05-22T21:18:42.064 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.3799999952316284, 2024-05-22T21:18:42.064 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:18:42.064 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.3799999952316284, 2024-05-22T21:18:42.064 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.3799999952316284, 2024-05-22T21:18:42.064 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:42.064 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:42.064 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:42.064 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.065 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.065 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:42.065 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:18:42.065 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.065 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:18:42.065 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:42.065 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:42.065 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:42.065 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:42.065 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:42.065 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:42.065 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:42.065 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:42.065 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:42.065 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:42.066 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:42.066 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:42.066 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.066 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.066 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.066 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:18:42.066 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:42.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.066 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.066 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.066 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:18:42.066 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:42.066 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.066 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.067 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:42.067 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.067 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.067 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:18:42.067 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:42.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.067 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.067 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.067 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:18:42.067 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:42.067 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.067 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.068 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:42.068 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.068 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.068 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:18:42.068 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:42.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.068 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.068 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.068 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:18:42.068 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:42.068 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.068 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.068 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:42.069 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.069 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.069 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:18:42.069 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:42.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.069 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.069 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.069 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:18:42.069 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:42.069 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.069 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.069 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:18:42.069 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:18:42.070 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:18:42.070 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:18:42.070 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:42.070 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:42.070 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:18:42.070 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:42.070 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.070 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.070 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:18:42.070 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:42.070 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:42.070 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:42.070 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:42.070 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:42.071 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:42.071 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:42.071 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:42.071 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:42.071 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:42.071 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:42.071 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:42.071 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.071 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.071 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:18:42.071 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:42.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.071 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.072 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.072 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:18:42.072 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:42.072 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.072 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.072 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:42.072 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.072 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.072 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:18:42.072 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:42.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.072 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.072 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.073 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:18:42.073 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:42.073 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.073 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.073 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:42.073 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.073 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.073 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:18:42.073 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:42.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.073 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.073 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.073 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:18:42.074 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:42.074 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.074 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.074 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:42.074 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.074 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.074 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:18:42.074 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:42.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.074 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.074 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.074 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:18:42.074 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:42.075 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.075 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.075 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:18:42.075 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:18:42.075 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:18:42.075 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:18:42.075 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:42.075 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:42.075 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:18:42.075 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:42.075 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.075 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.075 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:18:42.076 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:42.076 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:42.076 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:42.076 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:42.076 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:42.076 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:42.076 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:42.076 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:42.076 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:42.076 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:42.076 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:42.076 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:42.076 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.076 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.076 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.076 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:18:42.077 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:42.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.077 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.077 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:18:42.077 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:42.077 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.077 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.077 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:42.077 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.077 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.077 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.077 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:18:42.077 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:42.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.078 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.078 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:18:42.078 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:42.078 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.078 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.078 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:42.078 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.078 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.078 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.078 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:18:42.078 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:42.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.079 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.079 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.079 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:18:42.079 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:42.079 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.079 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.079 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:42.079 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.079 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.079 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.079 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:18:42.079 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:42.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.080 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.080 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.080 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:18:42.080 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:42.080 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.080 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.080 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:18:42.080 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:18:42.080 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:18:42.080 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:18:42.080 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:42.081 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:42.081 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:18:42.081 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:42.081 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.081 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.081 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:18:42.081 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:42.081 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:42.081 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:42.081 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:42.081 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:42.082 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:42.082 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:42.082 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:42.082 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:42.082 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:42.082 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:42.082 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:42.082 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.082 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.082 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:18:42.082 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:42.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.083 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.083 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:18:42.083 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:42.083 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.083 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.083 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:42.083 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.083 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.083 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.083 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:18:42.083 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:42.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.084 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.084 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:18:42.084 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:42.084 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.084 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.084 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:42.084 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.084 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.084 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.084 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:18:42.084 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:42.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.085 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.085 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:18:42.085 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:42.085 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.085 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.085 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:42.085 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.085 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.085 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:18:42.085 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:42.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.086 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.086 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.086 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:18:42.086 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:42.086 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.086 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.086 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:18:42.086 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:18:42.086 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:18:42.086 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:18:42.086 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:42.086 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:42.087 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:42.087 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.087 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.087 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:18:42.087 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:42.087 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:42.087 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:42.087 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:42.087 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:42.087 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:42.087 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:42.087 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:42.087 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:42.087 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:42.088 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:42.088 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:42.088 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.088 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.088 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:18:42.088 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:42.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.088 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.088 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.088 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:18:42.088 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:42.088 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.088 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.089 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:42.089 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.089 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.089 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:18:42.089 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:42.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.089 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.089 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:18:42.089 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:42.089 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.089 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.090 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:42.090 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.090 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.090 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:18:42.090 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:42.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.090 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.090 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:18:42.090 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:42.090 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.090 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.090 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:42.091 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.091 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.091 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:18:42.091 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:42.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.091 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.091 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:18:42.091 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:42.091 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.091 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.091 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:18:42.092 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:18:42.092 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:18:42.092 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:18:42.092 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:42.092 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:42.092 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:42.092 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.092 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:18:42.092 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:42.092 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:42.092 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:42.092 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:42.093 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:42.093 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:42.093 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:42.093 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:42.093 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:42.093 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:42.093 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:42.093 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:42.093 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.093 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.093 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:18:42.093 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:42.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.094 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.094 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:18:42.094 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:42.094 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.094 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.094 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:42.094 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.094 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.094 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:18:42.094 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:42.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.095 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.095 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:18:42.095 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:42.095 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.095 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.095 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:42.095 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.095 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.095 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:18:42.095 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:42.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.095 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.096 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:18:42.096 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:42.096 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.096 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.096 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:42.096 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.096 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.096 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:18:42.096 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:42.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.096 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.096 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:18:42.097 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:42.097 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.097 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.097 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:18:42.097 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:18:42.097 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:18:42.097 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:18:42.097 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:42.097 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:42.097 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:42.097 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.097 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:18:42.098 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:42.098 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:42.098 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:42.098 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:42.098 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:42.098 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:42.098 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:42.098 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:42.098 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:42.098 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:42.098 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:42.098 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:42.098 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.099 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.099 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:18:42.099 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:42.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.099 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.099 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:18:42.099 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:42.099 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.099 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.099 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:42.101 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.101 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.101 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:18:42.101 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:42.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.101 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.101 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:18:42.101 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:42.102 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.102 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.102 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:42.102 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.102 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.102 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:18:42.102 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:42.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.102 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.102 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:18:42.102 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:42.102 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.103 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.103 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:42.103 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.103 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.103 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:18:42.103 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:42.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.103 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.103 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:18:42.103 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:42.103 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.104 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.104 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:18:42.104 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:18:42.104 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:18:42.104 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:18:42.104 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:42.104 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:42.104 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:18:42.104 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:42.104 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.105 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:18:42.105 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:42.105 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:42.105 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:42.105 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:42.105 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:42.105 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:42.105 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:42.105 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:42.105 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:42.105 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:42.105 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:42.105 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:42.105 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.106 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.106 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:18:42.106 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:42.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.106 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.106 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.106 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:18:42.106 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:42.106 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.106 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.107 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:42.107 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.107 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.107 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:18:42.107 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:42.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.107 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.107 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:18:42.107 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:42.107 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.107 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.107 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:42.108 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.108 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.108 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:18:42.108 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:42.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.108 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.108 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:18:42.108 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:42.108 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.108 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.108 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:42.109 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.109 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.109 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:18:42.109 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:42.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.109 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.109 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:18:42.109 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:42.109 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.109 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.110 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:18:42.110 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:18:42.110 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:18:42.110 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:18:42.110 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:42.110 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull", 2024-05-22T21:18:42.110 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:42.110 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:42.110 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.110 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.110 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:42.110 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:18:42.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.111 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:18:42.111 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:42.111 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:42.111 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:42.111 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:42.112 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:42.112 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:18:42.112 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:42.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.113 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:18:42.113 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:42.113 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:42.113 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:42.113 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:42.113 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:42.113 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:18:42.113 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:42.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.113 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:18:42.114 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:42.114 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:42.114 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:42.114 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:42.114 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:42.114 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:18:42.114 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:42.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.114 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:18:42.114 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:42.114 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:42.114 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:18:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:18:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:42.115 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:18:42.116 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:42.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.116 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:18:42.116 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:42.116 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:42.116 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:42.116 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:42.116 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:42.116 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:18:42.116 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:42.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.116 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:18:42.116 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:42.117 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:42.117 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:42.117 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:42.117 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:42.117 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:18:42.117 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:42.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.117 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:18:42.117 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:42.117 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:42.117 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:42.117 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:42.117 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:42.117 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:18:42.118 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:42.118 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.118 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:42.118 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:18:42.118 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:18:42.118 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:18:42.118 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:18:42.118 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:18:42.118 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:18:42.118 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:18:42.118 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:18:42.118 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:18:42.118 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:18:42.118 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:18:42.119 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:18:42.119 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:18:42.119 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:18:42.119 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.119 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:18:42.119 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:18:42.119 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:18:42.119 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:18:42.119 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:18:42.119 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:18:42.119 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:18:42.119 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:18:42.119 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:18:42.120 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:18:42.120 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:18:42.120 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.120 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:18:42.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.043+0000 7f552ba00640 1 -- 172.21.15.161:0/414924003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5504066690 msgr2=0x7f5504068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:42.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.043+0000 7f552ba00640 1 --2- 172.21.15.161:0/414924003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5504066690 0x7f5504068b50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f55140ad000 tx=0x7f5514004740 comp rx=0 tx=0).stop 2024-05-22T21:18:42.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.043+0000 7f552ba00640 1 -- 172.21.15.161:0/414924003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f552415dd50 msgr2=0x7f552413f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:42.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.043+0000 7f552ba00640 1 --2- 172.21.15.161:0/414924003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f552415dd50 0x7f552413f930 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f551000c750 tx=0x7f551000cc10 comp rx=0 tx=0).stop 2024-05-22T21:18:42.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.043+0000 7f552ba00640 1 -- 172.21.15.161:0/414924003 shutdown_connections 2024-05-22T21:18:42.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.043+0000 7f552ba00640 1 --2- 172.21.15.161:0/414924003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5504066690 0x7f5504068b50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:42.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.043+0000 7f552ba00640 1 --2- 172.21.15.161:0/414924003 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f552415e130 0x7f5524149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:42.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.043+0000 7f552ba00640 1 --2- 172.21.15.161:0/414924003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f552415dd50 0x7f552413f930 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:42.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.043+0000 7f552ba00640 1 --2- 172.21.15.161:0/414924003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f552415d5e0 0x7f55241475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:42.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.043+0000 7f552ba00640 1 -- 172.21.15.161:0/414924003 >> 172.21.15.161:0/414924003 conn(0x7f55240b0bc0 msgr2=0x7f5524159080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:42.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.043+0000 7f552ba00640 1 -- 172.21.15.161:0/414924003 shutdown_connections 2024-05-22T21:18:42.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.043+0000 7f552ba00640 1 -- 172.21.15.161:0/414924003 wait complete. 2024-05-22T21:18:42.123 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-05-22T21:18:42.123 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:18:42.123 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:42.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:42.131 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2024-05-22T21:18:42.520 INFO:teuthology.orchestra.run.smithi161.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-05-22T21:18:42.520 INFO:teuthology.orchestra.run.smithi161.stderr:50+0 records in 2024-05-22T21:18:42.520 INFO:teuthology.orchestra.run.smithi161.stderr:50+0 records out 2024-05-22T21:18:42.520 INFO:teuthology.orchestra.run.smithi161.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.331559 s, 158 MB/s 2024-05-22T21:18:42.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:18:42.522 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:18:42.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.659+0000 7f7581800640 1 Processor -- start 2024-05-22T21:18:42.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.659+0000 7f7581800640 1 -- start start 2024-05-22T21:18:42.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.659+0000 7f7581800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f757c144550 0x7f757c0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:42.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.659+0000 7f7581800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f757c06be70 0x7f757c06c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:42.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.659+0000 7f7581800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c06de20 con 0x7f757c15b420 2024-05-22T21:18:42.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.659+0000 7f7581800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757c06ba50 con 0x7f757c144550 2024-05-22T21:18:42.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.659+0000 7f7581800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0cdd20 con 0x7f757c06be70 2024-05-22T21:18:42.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.659+0000 7f757be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f757c144550 0x7f757c0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:42.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.659+0000 7f757be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f757c144550 0x7f757c0cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38102/0 (socket says 172.21.15.161:38102) 2024-05-22T21:18:42.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.659+0000 7f757be00640 1 -- 172.21.15.161:0/2326057330 learned_addr learned my addr 172.21.15.161:0/2326057330 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:42.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.659+0000 7f7580e00640 1 --2- 172.21.15.161:0/2326057330 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f757c06be70 0x7f757c06c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:42.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.659+0000 7f757be00640 1 -- 172.21.15.161:0/2326057330 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f757c06be70 msgr2=0x7f757c06c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:42.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.659+0000 7f757be00640 1 --2- 172.21.15.161:0/2326057330 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f757c06be70 0x7f757c06c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:42.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.659+0000 7f757be00640 1 -- 172.21.15.161:0/2326057330 >> v1:172.21.15.179:6789/0 conn(0x7f757c15b420 legacy=0x7f757c15b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:42.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.659+0000 7f757be00640 1 -- 172.21.15.161:0/2326057330 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c0ce000 con 0x7f757c144550 2024-05-22T21:18:42.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.663+0000 7f757be00640 1 --2- 172.21.15.161:0/2326057330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c144550 0x7f757c0cb5a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f756c009870 tx=0x7f756c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c2d612dde0b430 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:42.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.663+0000 7f757aa00640 1 -- 172.21.15.161:0/2326057330 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c0a6040 con 0x7f757c144550 2024-05-22T21:18:42.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.663+0000 7f757aa00640 1 -- 172.21.15.161:0/2326057330 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c0aa600 con 0x7f757c144550 2024-05-22T21:18:42.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.663+0000 7f757aa00640 1 -- 172.21.15.161:0/2326057330 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c0aa990 con 0x7f757c144550 2024-05-22T21:18:42.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.663+0000 7f7581800640 1 -- 172.21.15.161:0/2326057330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c144550 msgr2=0x7f757c0cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:42.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.663+0000 7f7581800640 1 --2- 172.21.15.161:0/2326057330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c144550 0x7f757c0cb5a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f756c009870 tx=0x7f756c01c4a0 comp rx=0 tx=0).stop 2024-05-22T21:18:42.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.663+0000 7f7581800640 1 -- 172.21.15.161:0/2326057330 shutdown_connections 2024-05-22T21:18:42.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.663+0000 7f7581800640 1 --2- 172.21.15.161:0/2326057330 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f757c06be70 0x7f757c06c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:42.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.663+0000 7f7581800640 1 --2- 172.21.15.161:0/2326057330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c144550 0x7f757c0cb5a0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:42.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.663+0000 7f7581800640 1 -- 172.21.15.161:0/2326057330 >> 172.21.15.161:0/2326057330 conn(0x7f757c06a4d0 msgr2=0x7f757c0b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:42.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.663+0000 7f7581800640 1 -- 172.21.15.161:0/2326057330 shutdown_connections 2024-05-22T21:18:42.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.663+0000 7f7581800640 1 -- 172.21.15.161:0/2326057330 wait complete. 2024-05-22T21:18:42.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.663+0000 7f7581800640 1 Processor -- start 2024-05-22T21:18:42.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.663+0000 7f7581800640 1 -- start start 2024-05-22T21:18:42.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f7581800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c06be70 0x7f757c159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:42.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f7581800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f757c144550 0x7f757c1598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:42.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f7581800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f757c15b420 0x7f757c159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:42.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f7581800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0713d0 con 0x7f757c06be70 2024-05-22T21:18:42.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f7581800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c134fd0 con 0x7f757c144550 2024-05-22T21:18:42.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f7581800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c1352b0 con 0x7f757c15b420 2024-05-22T21:18:42.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f757be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c06be70 0x7f757c159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:42.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f757be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c06be70 0x7f757c159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38118/0 (socket says 172.21.15.161:38118) 2024-05-22T21:18:42.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f757be00640 1 -- 172.21.15.161:0/25668364 learned_addr learned my addr 172.21.15.161:0/25668364 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:42.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f7580e00640 1 --2- 172.21.15.161:0/25668364 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f757c15b420 0x7f757c159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:42.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f757b400640 1 --2- 172.21.15.161:0/25668364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f757c144550 0x7f757c1598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:42.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f757be00640 1 -- 172.21.15.161:0/25668364 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f757c15b420 msgr2=0x7f757c159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:42.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f757be00640 1 --2- 172.21.15.161:0/25668364 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f757c15b420 0x7f757c159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:42.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f757be00640 1 -- 172.21.15.161:0/25668364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f757c144550 msgr2=0x7f757c1598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:42.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f757be00640 1 --2- 172.21.15.161:0/25668364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f757c144550 0x7f757c1598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:42.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f757be00640 1 -- 172.21.15.161:0/25668364 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c0094d0 con 0x7f757c06be70 2024-05-22T21:18:42.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f757be00640 1 --2- 172.21.15.161:0/25668364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c06be70 0x7f757c159120 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f756c01a0d0 tx=0x7f756c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:42.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f7573e00640 1 -- 172.21.15.161:0/25668364 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c0b3800 con 0x7f757c06be70 2024-05-22T21:18:42.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f7573e00640 1 -- 172.21.15.161:0/25668364 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c0b3960 con 0x7f757c06be70 2024-05-22T21:18:42.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f7573e00640 1 -- 172.21.15.161:0/25668364 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c0b3cd0 con 0x7f757c06be70 2024-05-22T21:18:42.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f7581800640 1 -- 172.21.15.161:0/25668364 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757c135590 con 0x7f757c06be70 2024-05-22T21:18:42.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f7581800640 1 -- 172.21.15.161:0/25668364 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757c159660 con 0x7f757c06be70 2024-05-22T21:18:42.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f7581800640 1 -- 172.21.15.161:0/25668364 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757c067070 con 0x7f757c06be70 2024-05-22T21:18:42.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.667+0000 7f7573e00640 1 -- 172.21.15.161:0/25668364 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f756c0c6430 con 0x7f757c06be70 2024-05-22T21:18:42.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.671+0000 7f7573e00640 1 --2- 172.21.15.161:0/25668364 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f755c0666c0 0x7f755c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:42.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.671+0000 7f7573e00640 1 -- 172.21.15.161:0/25668364 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f756c11ed20 con 0x7f757c06be70 2024-05-22T21:18:42.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.671+0000 7f7573e00640 1 -- 172.21.15.161:0/25668364 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=109}) v3 -- 0x7f755c06e1f0 con 0x7f757c06be70 2024-05-22T21:18:42.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.671+0000 7f757b400640 1 --2- 172.21.15.161:0/25668364 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f755c0666c0 0x7f755c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:42.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.671+0000 7f757b400640 1 --2- 172.21.15.161:0/25668364 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f755c0666c0 0x7f755c068b80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f757c1395f0 tx=0x7f7568009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:42.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.675+0000 7f7573e00640 1 -- 172.21.15.161:0/25668364 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f756c0ec080 con 0x7f757c06be70 2024-05-22T21:18:42.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.827+0000 7f7581800640 1 -- 172.21.15.161:0/25668364 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f757c0b13c0 con 0x7f757c06be70 2024-05-22T21:18:42.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.831+0000 7f7573e00640 1 -- 172.21.15.161:0/25668364 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v108) v1 ==== 81+0+33754 (secure 0 0 0) 0x7f756c0ef6a0 con 0x7f757c06be70 2024-05-22T21:18:42.833 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:42.834 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:18:42.834 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 108, 2024-05-22T21:18:42.834 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:18:42.834 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:18:42.834 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:18:42.591218+0000", 2024-05-22T21:18:42.834 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:18:42.834 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:18:42.834 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:18:42.834 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:18:42.834 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:18:42.834 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:18:42.834 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:18:42.835 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:18:42.835 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:18:42.835 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:42.835 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:18:42.835 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:18:42.835 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:18:42.835 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:18:42.835 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:18:42.835 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 13, 2024-05-22T21:18:42.835 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:18:42.835 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:18:42.835 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:18:42.835 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:18:42.836 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:18:42.836 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:18:42.836 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.836 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:18:42.836 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:18:42.836 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:18:42.836 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:18:42.836 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:18:42.836 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:42.836 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:42.836 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:42.836 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:42.836 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:42.836 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:42.837 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:42.837 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:42.837 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:42.837 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:42.837 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:18:42.837 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:18:42.837 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:18:42.837 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:18:42.837 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:18:42.837 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:42.837 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:42.837 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:42.837 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:42.837 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:42.837 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:42.838 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:42.838 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.838 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "105", 2024-05-22T21:18:42.838 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:42.838 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:42.838 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:42.838 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:42.838 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:42.838 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:42.838 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:42.838 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:42.838 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:42.838 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:42.839 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:42.839 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:42.839 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:42.839 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:42.839 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:42.839 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:42.839 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:42.839 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:42.839 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:42.839 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:42.839 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:42.839 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:42.839 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:42.839 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:42.840 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:42.840 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:42.840 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.840 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:42.840 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:42.840 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:42.840 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:42.840 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:42.840 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:42.840 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:42.840 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:42.840 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:42.840 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:42.841 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:42.841 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:42.841 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:18:42.841 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:18:42.841 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.841 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:42.841 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:18:42.841 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.841 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:42.841 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:18:42.841 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:18:42.841 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:18:42.841 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:18:42.841 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:18:42.841 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:42.841 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:42.842 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:42.842 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.842 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.842 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.842 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 12, 2024-05-22T21:18:42.842 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:18:42.842 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:18:16.500534+0000", 2024-05-22T21:18:42.842 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:18:42.842 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:18:42.842 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:42.842 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:42.842 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:42.842 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:42.842 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:42.843 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:42.843 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:42.843 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:42.843 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:42.843 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:42.843 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:18:42.843 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:18:42.843 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:18:42.843 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:18:42.843 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:18:42.843 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:42.843 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:42.843 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:42.844 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:42.844 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:42.844 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:42.844 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:42.844 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.844 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "105", 2024-05-22T21:18:42.844 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:42.844 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:42.844 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:42.844 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:42.844 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:42.844 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:42.844 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:42.844 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:42.844 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:42.845 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:42.845 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:42.845 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:42.845 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:42.845 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:42.845 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:42.845 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:42.845 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:42.845 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:42.845 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:42.845 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:42.845 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:42.845 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:42.845 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:42.845 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:42.846 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:42.846 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:42.846 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.846 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:42.846 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:42.846 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:42.846 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:42.846 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:42.846 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:42.846 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:42.846 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:42.846 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:42.846 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:42.847 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:42.847 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:42.847 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:18:42.847 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:18:42.847 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:18:42.847 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.847 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:42.847 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:18:42.847 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:18:42.847 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.847 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.847 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:42.847 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 2, 2024-05-22T21:18:42.847 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 2, 2024-05-22T21:18:42.847 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:18:42.848 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:18:42.848 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:18:42.848 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:42.848 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:42.848 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:42.848 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.848 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.848 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 13, 2024-05-22T21:18:42.848 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:18:42.848 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:18:16.937445+0000", 2024-05-22T21:18:42.848 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:18:42.848 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:18:42.848 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:42.848 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:42.848 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:42.849 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:42.849 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:42.849 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:42.849 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:42.849 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:42.849 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:42.849 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:42.849 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:18:42.849 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:18:42.849 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:18:42.849 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:18:42.849 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:18:42.849 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:42.849 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:42.849 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:42.850 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:42.850 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:42.850 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:42.850 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:42.850 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.850 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "105", 2024-05-22T21:18:42.850 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:42.850 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:42.850 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:42.850 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:42.850 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:42.850 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:42.850 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:42.851 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:42.851 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:42.851 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:42.851 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:42.851 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:42.851 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:42.851 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:42.851 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:42.851 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:42.851 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:42.851 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:42.851 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:42.851 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:42.851 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:42.852 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:42.852 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:42.852 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:42.852 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:42.852 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:42.852 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.852 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:42.852 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:42.852 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:42.852 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:42.852 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:42.852 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:42.852 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:42.852 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:42.852 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:42.853 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:42.853 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:42.853 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:42.853 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:18:42.853 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:18:42.853 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.853 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:42.853 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:18:42.853 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:18:42.853 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.853 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.853 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:42.853 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.3799999952316284, 2024-05-22T21:18:42.853 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.3799999952316284, 2024-05-22T21:18:42.853 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:18:42.854 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.3799999952316284, 2024-05-22T21:18:42.854 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.3799999952316284, 2024-05-22T21:18:42.854 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:42.854 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:42.854 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:42.854 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.854 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.854 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:42.854 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:18:42.854 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.854 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:18:42.854 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:42.854 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:42.855 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:42.855 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:42.855 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:42.855 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:42.855 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:42.855 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:42.855 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:42.855 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:42.855 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:42.855 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:42.855 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.855 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.855 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.855 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:18:42.856 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:42.856 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.856 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.856 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.856 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:18:42.856 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:42.856 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.856 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.856 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.856 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:42.856 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.856 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.856 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.856 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:18:42.856 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:42.856 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.857 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.857 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.857 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:18:42.857 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:42.857 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.857 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.857 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.857 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:42.857 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.857 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.857 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.857 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:18:42.857 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:42.857 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.857 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.858 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.858 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:18:42.858 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:42.858 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.858 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.858 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.858 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:42.858 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.858 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.858 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.858 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:18:42.858 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:42.859 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.859 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.859 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.859 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:18:42.859 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:42.859 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.859 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.859 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.859 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:18:42.859 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:18:42.859 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:18:42.859 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:18:42.859 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:42.859 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:42.859 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:18:42.860 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:42.860 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.860 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.860 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.860 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:18:42.860 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:42.860 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:42.860 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:42.860 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:42.860 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:42.860 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:42.860 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:42.860 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:42.860 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:42.860 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:42.861 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:42.861 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:42.861 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.861 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.861 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.861 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:18:42.861 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:42.861 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.861 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.861 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.861 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:18:42.861 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:42.861 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.861 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.861 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.862 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:42.862 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.862 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.862 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.862 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:18:42.862 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:42.862 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.862 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.862 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.862 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:18:42.862 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:42.862 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.862 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.863 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.863 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:42.863 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.863 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.863 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.863 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:18:42.863 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:42.863 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.863 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.863 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.863 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:18:42.863 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:42.863 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.864 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.864 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.864 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:42.864 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.864 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.864 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.864 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:18:42.864 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:42.864 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.864 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.864 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.864 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:18:42.864 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:42.864 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.864 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.865 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.865 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:18:42.865 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:18:42.865 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:18:42.865 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:18:42.865 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:42.865 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:42.865 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:18:42.865 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:42.865 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.865 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.865 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.865 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:18:42.865 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:42.866 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:42.866 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:42.866 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:42.866 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:42.866 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:42.866 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:42.866 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:42.866 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:42.866 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:42.866 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:42.866 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:42.866 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.866 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.867 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.867 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:18:42.867 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:42.867 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.867 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.867 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.867 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:18:42.867 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:42.867 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.867 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.867 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.867 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:42.867 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.867 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.868 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.868 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:18:42.868 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:42.868 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.868 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.868 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.868 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:18:42.868 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:42.868 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.868 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.868 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.868 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:42.868 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.868 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.869 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.869 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:18:42.869 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:42.869 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.869 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.869 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.869 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:18:42.869 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:42.869 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.869 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.869 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.869 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:42.869 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.869 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.869 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.870 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:18:42.870 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:42.870 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.870 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.870 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.870 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:18:42.870 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:42.870 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.870 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.870 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.870 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:18:42.870 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:18:42.870 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:18:42.871 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:18:42.871 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:42.871 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:42.871 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:18:42.871 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:42.871 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.871 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.871 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.871 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:18:42.871 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:42.871 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:42.871 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:42.871 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:42.872 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:42.872 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:42.872 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:42.872 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:42.872 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:42.872 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:42.872 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:42.872 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:42.872 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.872 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.872 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.872 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:18:42.872 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:42.872 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.873 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.873 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.873 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:18:42.873 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:42.873 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.873 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.873 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.873 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:42.873 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.873 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.873 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.873 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:18:42.873 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:42.874 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.874 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.874 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.874 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:18:42.874 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:42.874 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.874 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.874 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.874 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:42.874 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.874 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.874 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.875 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:18:42.875 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:42.875 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.875 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.875 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.875 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:18:42.875 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:42.875 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.875 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.875 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.875 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:42.875 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.875 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.875 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.875 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:18:42.876 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:42.876 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.876 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.876 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.876 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:18:42.876 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:42.876 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.876 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.876 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.876 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:18:42.876 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:18:42.876 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:18:42.876 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:18:42.876 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:42.876 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:18:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:42.877 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:42.878 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:42.878 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.878 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.878 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.878 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:18:42.878 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:42.878 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.878 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.878 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.878 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:18:42.878 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:42.878 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.878 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.879 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.879 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:42.879 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.879 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.879 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.879 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:18:42.879 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:42.879 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.879 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.879 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.879 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:18:42.879 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:42.879 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.879 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.879 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.880 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:42.880 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.880 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.880 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.880 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:18:42.880 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:42.880 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.880 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.880 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.880 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:18:42.880 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:42.880 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.880 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.880 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.880 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:42.881 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.881 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.881 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.881 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:18:42.881 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:42.881 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.881 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.881 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.881 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:18:42.881 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:42.881 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.881 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.881 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.881 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:18:42.882 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:18:42.882 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:18:42.882 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:18:42.882 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:42.882 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:42.882 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:18:42.882 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:42.882 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.882 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.882 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.882 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:18:42.882 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:42.883 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:42.883 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:42.883 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:42.883 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:42.883 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:42.883 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:42.883 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:42.883 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:42.883 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:42.883 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:42.883 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:42.883 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.884 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.884 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.884 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:18:42.884 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:42.884 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.884 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.884 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.884 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:18:42.884 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:42.884 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.884 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.884 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.885 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:42.885 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.885 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.885 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.885 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:18:42.885 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:42.885 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.885 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.885 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.885 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:18:42.885 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:42.885 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.886 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.886 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.886 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:42.886 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.886 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.886 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.886 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:18:42.886 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:42.886 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.886 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.886 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.886 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:18:42.886 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:42.887 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.887 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.887 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.887 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:42.887 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.887 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.887 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.887 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:18:42.887 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:42.887 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.887 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.887 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.887 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:18:42.887 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:42.888 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.888 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.888 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.888 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:18:42.888 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:18:42.888 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:18:42.888 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:18:42.888 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:42.888 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:42.888 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:42.888 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.888 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.888 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.888 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:18:42.889 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:42.889 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:42.889 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:42.889 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:42.889 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:42.889 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:42.889 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:42.889 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:42.889 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:42.889 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:42.889 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:42.889 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:42.889 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.889 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.889 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.890 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:18:42.890 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:42.890 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.890 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.890 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.890 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:18:42.890 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:42.890 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.890 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.890 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.890 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:42.890 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.891 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.891 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.891 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:18:42.891 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:42.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.891 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.891 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.891 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:18:42.891 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:42.891 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.891 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.891 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.891 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:42.892 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.892 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.892 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.892 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:18:42.892 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:42.892 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.892 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.892 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.892 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:18:42.892 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:42.892 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.892 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.892 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.892 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:42.892 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.893 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.893 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.893 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:18:42.893 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:42.893 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.893 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.893 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.893 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:18:42.893 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:42.893 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.893 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.893 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.893 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:18:42.893 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:18:42.893 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:18:42.894 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:18:42.894 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:42.894 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:42.894 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:18:42.894 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:42.894 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.894 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.894 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.894 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:18:42.894 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:42.894 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:42.894 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:42.894 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:42.895 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:42.895 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:42.895 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:42.895 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:42.895 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:42.895 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:42.895 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:42.895 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:42.895 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.895 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.895 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.895 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:18:42.895 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:42.895 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.896 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.896 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.896 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:18:42.896 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:42.896 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.896 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.896 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:42.896 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.896 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.896 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.896 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:18:42.896 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:42.896 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.897 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.897 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.897 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:18:42.897 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:42.897 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.897 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.897 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.897 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:42.897 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.897 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.897 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.897 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:18:42.897 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:42.897 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.897 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.898 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.898 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:18:42.898 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:42.898 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.898 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.898 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.898 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:42.898 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:42.898 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.898 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:42.898 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:18:42.898 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:42.898 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.899 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.899 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:42.899 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:18:42.899 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:42.899 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.899 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.899 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.899 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:18:42.899 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:18:42.899 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:18:42.899 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:18:42.899 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:42.899 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull", 2024-05-22T21:18:42.900 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:42.900 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:42.900 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:42.900 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.900 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:42.900 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:18:42.900 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.900 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:18:42.900 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:42.900 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:42.900 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:42.900 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:42.900 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:42.901 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:18:42.901 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:42.901 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.901 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.901 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:18:42.901 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:42.901 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:42.901 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:42.901 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:42.901 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:42.901 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:18:42.901 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:42.901 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.901 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.902 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:18:42.902 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:42.902 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:42.902 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:42.902 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:42.902 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:42.902 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:18:42.902 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:42.902 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.902 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.902 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:18:42.902 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:42.903 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:42.903 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:42.903 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:42.903 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:42.903 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:18:42.903 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:42.903 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.903 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.903 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:18:42.903 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:42.903 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:42.903 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:42.903 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:42.903 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:42.903 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:18:42.904 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:42.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.904 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.904 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:18:42.904 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:42.904 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:42.904 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:42.904 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:42.904 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:42.904 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:18:42.904 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:42.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.904 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.904 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:18:42.905 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:42.905 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:42.905 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:42.905 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:42.905 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:42.905 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:18:42.905 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:42.905 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.905 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:42.905 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:18:42.905 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:42.905 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:42.905 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:42.906 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:42.906 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:42.906 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:18:42.906 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:42.906 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.906 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:42.906 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:18:42.906 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:18:42.906 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:18:42.906 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:18:42.906 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:18:42.906 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:18:42.906 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:18:42.907 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:18:42.907 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:18:42.907 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:18:42.907 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:18:42.907 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:18:42.907 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:18:42.907 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:18:42.907 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:42.907 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:18:42.907 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:18:42.907 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:18:42.907 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:18:42.907 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:18:42.908 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:18:42.908 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:18:42.908 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:18:42.908 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:18:42.908 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:18:42.908 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:18:42.908 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:42.908 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:18:42.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.831+0000 7f7581800640 1 -- 172.21.15.161:0/25668364 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f755c0666c0 msgr2=0x7f755c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:42.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.831+0000 7f7581800640 1 --2- 172.21.15.161:0/25668364 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f755c0666c0 0x7f755c068b80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f757c1395f0 tx=0x7f7568009040 comp rx=0 tx=0).stop 2024-05-22T21:18:42.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.831+0000 7f7581800640 1 -- 172.21.15.161:0/25668364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c06be70 msgr2=0x7f757c159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:42.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.831+0000 7f7581800640 1 --2- 172.21.15.161:0/25668364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c06be70 0x7f757c159120 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f756c01a0d0 tx=0x7f756c01ace0 comp rx=0 tx=0).stop 2024-05-22T21:18:42.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.831+0000 7f7581800640 1 -- 172.21.15.161:0/25668364 shutdown_connections 2024-05-22T21:18:42.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.831+0000 7f7581800640 1 --2- 172.21.15.161:0/25668364 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f755c0666c0 0x7f755c068b80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:42.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.831+0000 7f7581800640 1 --2- 172.21.15.161:0/25668364 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f757c15b420 0x7f757c159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:42.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.831+0000 7f7581800640 1 --2- 172.21.15.161:0/25668364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f757c144550 0x7f757c1598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:42.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.831+0000 7f7581800640 1 --2- 172.21.15.161:0/25668364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c06be70 0x7f757c159120 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:42.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.831+0000 7f7581800640 1 -- 172.21.15.161:0/25668364 >> 172.21.15.161:0/25668364 conn(0x7f757c06a4d0 msgr2=0x7f757c0b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:42.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.831+0000 7f7581800640 1 -- 172.21.15.161:0/25668364 shutdown_connections 2024-05-22T21:18:42.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:42.831+0000 7f7581800640 1 -- 172.21.15.161:0/25668364 wait complete. 2024-05-22T21:18:42.911 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:18:43.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb63600640 1 Processor -- start 2024-05-22T21:18:43.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb63600640 1 -- start start 2024-05-22T21:18:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb63600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb5c0b3560 0x7feb5c06df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb63600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feb5c0cf510 0x7feb5c0cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb63600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb5c0cbe40 con 0x7feb5c15b1e0 2024-05-22T21:18:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb63600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5c0cfee0 con 0x7feb5c0cf510 2024-05-22T21:18:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb63600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c0cebc0 con 0x7feb5c0b3560 2024-05-22T21:18:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb61600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feb5c0cf510 0x7feb5c0cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb61600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feb5c0cf510 0x7feb5c0cf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38126/0 (socket says 172.21.15.161:38126) 2024-05-22T21:18:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb53e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb5c0b3560 0x7feb5c06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:43.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb60c00640 1 --1- 172.21.15.161:0/4115975678 >> v1:172.21.15.179:6789/0 conn(0x7feb5c15b1e0 0x7feb5c09a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:53824/0 (socket says 172.21.15.161:53824) 2024-05-22T21:18:43.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb61600640 1 -- 172.21.15.161:0/4115975678 learned_addr learned my addr 172.21.15.161:0/4115975678 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:43.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb5be00640 1 -- 172.21.15.161:0/4115975678 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1864855658 0 0) 0x7feb5c0cbe40 con 0x7feb5c15b1e0 2024-05-22T21:18:43.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb5be00640 1 -- 172.21.15.161:0/4115975678 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb40003530 con 0x7feb5c15b1e0 2024-05-22T21:18:43.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb53e00640 1 -- 172.21.15.161:0/4115975678 >> v2:172.21.15.161:3300/0 conn(0x7feb5c0cf510 msgr2=0x7feb5c0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:43.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb53e00640 1 --2- 172.21.15.161:0/4115975678 >> v2:172.21.15.161:3300/0 conn(0x7feb5c0cf510 0x7feb5c0cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:43.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb53e00640 1 -- 172.21.15.161:0/4115975678 >> v1:172.21.15.179:6789/0 conn(0x7feb5c15b1e0 legacy=0x7feb5c09a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:43.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb53e00640 1 -- 172.21.15.161:0/4115975678 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c0ceea0 con 0x7feb5c0b3560 2024-05-22T21:18:43.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.043+0000 7feb61600640 1 --2- 172.21.15.161:0/4115975678 >> v2:172.21.15.161:3300/0 conn(0x7feb5c0cf510 0x7feb5c0cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:43.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.047+0000 7feb53e00640 1 --2- 172.21.15.161:0/4115975678 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb5c0b3560 0x7feb5c06df70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feb440097c0 tx=0x7feb4401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a0322489f73a070 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:43.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.047+0000 7feb5be00640 1 -- 172.21.15.161:0/4115975678 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb440ab600 con 0x7feb5c0b3560 2024-05-22T21:18:43.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.047+0000 7feb5be00640 1 -- 172.21.15.161:0/4115975678 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb440a7050 con 0x7feb5c0b3560 2024-05-22T21:18:43.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.047+0000 7feb5be00640 1 -- 172.21.15.161:0/4115975678 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb440ab980 con 0x7feb5c0b3560 2024-05-22T21:18:43.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.047+0000 7feb63600640 1 -- 172.21.15.161:0/4115975678 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb5c0b3560 msgr2=0x7feb5c06df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:43.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.047+0000 7feb63600640 1 --2- 172.21.15.161:0/4115975678 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb5c0b3560 0x7feb5c06df70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feb440097c0 tx=0x7feb4401c410 comp rx=0 tx=0).stop 2024-05-22T21:18:43.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.047+0000 7feb63600640 1 -- 172.21.15.161:0/4115975678 shutdown_connections 2024-05-22T21:18:43.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.047+0000 7feb63600640 1 --2- 172.21.15.161:0/4115975678 >> v2:172.21.15.161:3300/0 conn(0x7feb5c0cf510 0x7feb5c0cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:43.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.047+0000 7feb63600640 1 --2- 172.21.15.161:0/4115975678 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb5c0b3560 0x7feb5c06df70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:43.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.047+0000 7feb63600640 1 -- 172.21.15.161:0/4115975678 >> 172.21.15.161:0/4115975678 conn(0x7feb5c0b2a40 msgr2=0x7feb5c0b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:43.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.047+0000 7feb63600640 1 -- 172.21.15.161:0/4115975678 shutdown_connections 2024-05-22T21:18:43.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.047+0000 7feb63600640 1 -- 172.21.15.161:0/4115975678 wait complete. 2024-05-22T21:18:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb63600640 1 Processor -- start 2024-05-22T21:18:43.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb63600640 1 -- start start 2024-05-22T21:18:43.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb63600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb5c0b3560 0x7feb5c06afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:43.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb63600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb5c0cf510 0x7feb5c152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:43.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb63600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb5c15b1e0 0x7feb5c0c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb63600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c155e80 con 0x7feb5c15b1e0 2024-05-22T21:18:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb63600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c06c9c0 con 0x7feb5c0b3560 2024-05-22T21:18:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb63600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c157240 con 0x7feb5c0cf510 2024-05-22T21:18:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb61600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb5c15b1e0 0x7feb5c0c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb60c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb5c0b3560 0x7feb5c06afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb61600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb5c15b1e0 0x7feb5c0c6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38142/0 (socket says 172.21.15.161:38142) 2024-05-22T21:18:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb61600640 1 -- 172.21.15.161:0/2169928713 learned_addr learned my addr 172.21.15.161:0/2169928713 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:43.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb53e00640 1 --2- 172.21.15.161:0/2169928713 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb5c0cf510 0x7feb5c152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb61600640 1 -- 172.21.15.161:0/2169928713 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb5c0cf510 msgr2=0x7feb5c152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb61600640 1 --2- 172.21.15.161:0/2169928713 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb5c0cf510 0x7feb5c152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb61600640 1 -- 172.21.15.161:0/2169928713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb5c0b3560 msgr2=0x7feb5c06afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb61600640 1 --2- 172.21.15.161:0/2169928713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb5c0b3560 0x7feb5c06afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb61600640 1 -- 172.21.15.161:0/2169928713 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb54009b30 con 0x7feb5c15b1e0 2024-05-22T21:18:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb61600640 1 --2- 172.21.15.161:0/2169928713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb5c15b1e0 0x7feb5c0c6a50 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7feb5400efc0 tx=0x7feb5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb59600640 1 -- 172.21.15.161:0/2169928713 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54018070 con 0x7feb5c15b1e0 2024-05-22T21:18:43.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb59600640 1 -- 172.21.15.161:0/2169928713 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5400f040 con 0x7feb5c15b1e0 2024-05-22T21:18:43.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb59600640 1 -- 172.21.15.161:0/2169928713 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5400ce00 con 0x7feb5c15b1e0 2024-05-22T21:18:43.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb63600640 1 -- 172.21.15.161:0/2169928713 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb44009420 con 0x7feb5c15b1e0 2024-05-22T21:18:43.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.051+0000 7feb63600640 1 -- 172.21.15.161:0/2169928713 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5c1574e0 con 0x7feb5c15b1e0 2024-05-22T21:18:43.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.055+0000 7feb63600640 1 -- 172.21.15.161:0/2169928713 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5c067220 con 0x7feb5c15b1e0 2024-05-22T21:18:43.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.055+0000 7feb59600640 1 -- 172.21.15.161:0/2169928713 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feb54014480 con 0x7feb5c15b1e0 2024-05-22T21:18:43.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.055+0000 7feb59600640 1 --2- 172.21.15.161:0/2169928713 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feb380666c0 0x7feb38068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:43.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.055+0000 7feb59600640 1 -- 172.21.15.161:0/2169928713 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb540843a0 con 0x7feb5c15b1e0 2024-05-22T21:18:43.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.055+0000 7feb60c00640 1 --2- 172.21.15.161:0/2169928713 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feb380666c0 0x7feb38068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:43.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.055+0000 7feb59600640 1 -- 172.21.15.161:0/2169928713 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=109}) v3 -- 0x7feb3806e1f0 con 0x7feb5c15b1e0 2024-05-22T21:18:43.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.055+0000 7feb60c00640 1 --2- 172.21.15.161:0/2169928713 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feb380666c0 0x7feb38068b80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0011f0 tx=0x7feb4c007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:43.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.059+0000 7feb59600640 1 -- 172.21.15.161:0/2169928713 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7feb540517b0 con 0x7feb5c15b1e0 2024-05-22T21:18:43.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.243+0000 7feb63600640 1 -- 172.21.15.161:0/2169928713 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb5c0d0400 con 0x7feb5c15b1e0 2024-05-22T21:18:43.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.243+0000 7feb59600640 1 -- 172.21.15.161:0/2169928713 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4748 (secure 0 0 0) 0x7feb54054dd0 con 0x7feb5c15b1e0 2024-05-22T21:18:43.248 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:43.248 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2771409351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2771409351},{"type":"v1","addr":"172.21.15.179:6834","nonce":2771409351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6442,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2520119431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2520119431},{"type":"v1","addr":"172.21.15.179:6835","nonce":2520119431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1415362547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1415362547},{"type":"v1","addr":"172.21.15.161:6835","nonce":1415362547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:18:17.938858+0000","modified":"2024-05-22T21:18:20.963065+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/4127130864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4127130864},{"type":"v1","addr":"172.21.15.161:6837","nonce":4127130864}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-22T21:18:43.248 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 47 2024-05-22T21:18:43.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.247+0000 7feb63600640 1 -- 172.21.15.161:0/2169928713 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feb380666c0 msgr2=0x7feb38068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:43.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.247+0000 7feb63600640 1 --2- 172.21.15.161:0/2169928713 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feb380666c0 0x7feb38068b80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0011f0 tx=0x7feb4c007d40 comp rx=0 tx=0).stop 2024-05-22T21:18:43.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.247+0000 7feb63600640 1 -- 172.21.15.161:0/2169928713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb5c15b1e0 msgr2=0x7feb5c0c6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:43.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.247+0000 7feb63600640 1 --2- 172.21.15.161:0/2169928713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb5c15b1e0 0x7feb5c0c6a50 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7feb5400efc0 tx=0x7feb5400c490 comp rx=0 tx=0).stop 2024-05-22T21:18:43.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.247+0000 7feb63600640 1 -- 172.21.15.161:0/2169928713 shutdown_connections 2024-05-22T21:18:43.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.247+0000 7feb63600640 1 --2- 172.21.15.161:0/2169928713 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feb380666c0 0x7feb38068b80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:43.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.247+0000 7feb63600640 1 --2- 172.21.15.161:0/2169928713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb5c15b1e0 0x7feb5c0c6a50 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:43.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.247+0000 7feb63600640 1 --2- 172.21.15.161:0/2169928713 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb5c0cf510 0x7feb5c152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:43.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.247+0000 7feb63600640 1 --2- 172.21.15.161:0/2169928713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb5c0b3560 0x7feb5c06afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:43.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.247+0000 7feb63600640 1 -- 172.21.15.161:0/2169928713 >> 172.21.15.161:0/2169928713 conn(0x7feb5c0b2a40 msgr2=0x7feb5c0ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:43.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.247+0000 7feb63600640 1 -- 172.21.15.161:0/2169928713 shutdown_connections 2024-05-22T21:18:43.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.247+0000 7feb63600640 1 -- 172.21.15.161:0/2169928713 wait complete. 2024-05-22T21:18:43.268 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-05-22T21:18:43.422 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":6699,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":25.465735988999999,"mdsmap_epoch":47,"osdmap_epoch":108,"osdmap_epoch_barrier":107,"uptime":29.700022891} 2024-05-22T21:18:43.423 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 6699, "mdsmap_epoch": 47, "osdmap_epoch": 108, "osdmap_epoch_barrier": 107, "rank_uptime": 25.465735989, "state": "up:active", "uptime": 29.700022891, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:18:43.423 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-22T21:18:43.423 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:18:43.423 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:43.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:43.432 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2024-05-22T21:18:43.483 INFO:teuthology.orchestra.run.smithi161.stderr:dd: failed to open 'small_file_1': Permission denied 2024-05-22T21:18:43.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:18:43.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:43.484 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-05-22T21:18:43.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:43.542 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-05-22T21:18:43.646 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:18:43.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.779+0000 7f9012a00640 1 Processor -- start 2024-05-22T21:18:43.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.779+0000 7f9012a00640 1 -- start start 2024-05-22T21:18:43.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.783+0000 7f9012a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f900c143ec0 0x7f900c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:43.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.783+0000 7f9012a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f900c0715a0 0x7f900c06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:43.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.783+0000 7f9012a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f900c1442a0 con 0x7f900c15ec90 2024-05-22T21:18:43.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.783+0000 7f9012a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900c070260 con 0x7f900c0715a0 2024-05-22T21:18:43.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.783+0000 7f9012a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900c070500 con 0x7f900c143ec0 2024-05-22T21:18:43.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.783+0000 7f9003400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f900c0715a0 0x7f900c06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:43.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.783+0000 7f9010a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f900c15ec90 0x7f900c06bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:38150/0 (socket says 172.21.15.161:38150) 2024-05-22T21:18:43.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.783+0000 7f9010a00640 1 -- 172.21.15.161:0/2218654156 learned_addr learned my addr 172.21.15.161:0/2218654156 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:43.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.783+0000 7f900be00640 1 --2- 172.21.15.161:0/2218654156 >> v2:172.21.15.179:3300/0 conn(0x7f900c143ec0 0x7f900c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:43.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.783+0000 7f9003400640 1 -- 172.21.15.161:0/2218654156 >> v2:172.21.15.179:3300/0 conn(0x7f900c143ec0 msgr2=0x7f900c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:43.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.783+0000 7f9003400640 1 --2- 172.21.15.161:0/2218654156 >> v2:172.21.15.179:3300/0 conn(0x7f900c143ec0 0x7f900c15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:43.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.783+0000 7f900b400640 1 -- 172.21.15.161:0/2218654156 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866875230 0 0) 0x7f900c1442a0 con 0x7f900c15ec90 2024-05-22T21:18:43.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.783+0000 7f9003400640 1 -- 172.21.15.161:0/2218654156 >> v1:172.21.15.161:6789/0 conn(0x7f900c15ec90 legacy=0x7f900c06bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:43.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.783+0000 7f9003400640 1 -- 172.21.15.161:0/2218654156 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c0707e0 con 0x7f900c0715a0 2024-05-22T21:18:43.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.783+0000 7f900be00640 1 --2- 172.21.15.161:0/2218654156 >> v2:172.21.15.179:3300/0 conn(0x7f900c143ec0 0x7f900c15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:43.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9003400640 1 --2- 172.21.15.161:0/2218654156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f900c0715a0 0x7f900c06fd20 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f900c0724e0 tx=0x7f8ff401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb0dde5c0aff957 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:43.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f900b400640 1 -- 172.21.15.161:0/2218654156 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff40ab650 con 0x7f900c0715a0 2024-05-22T21:18:43.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f900b400640 1 -- 172.21.15.161:0/2218654156 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff40ab7b0 con 0x7f900c0715a0 2024-05-22T21:18:43.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f900b400640 1 -- 172.21.15.161:0/2218654156 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff40abb40 con 0x7f900c0715a0 2024-05-22T21:18:43.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9012a00640 1 -- 172.21.15.161:0/2218654156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f900c0715a0 msgr2=0x7f900c06fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:43.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9012a00640 1 --2- 172.21.15.161:0/2218654156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f900c0715a0 0x7f900c06fd20 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f900c0724e0 tx=0x7f8ff401c450 comp rx=0 tx=0).stop 2024-05-22T21:18:43.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9012a00640 1 -- 172.21.15.161:0/2218654156 shutdown_connections 2024-05-22T21:18:43.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9012a00640 1 --2- 172.21.15.161:0/2218654156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f900c0715a0 0x7f900c06fd20 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:43.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9012a00640 1 --2- 172.21.15.161:0/2218654156 >> v2:172.21.15.179:3300/0 conn(0x7f900c143ec0 0x7f900c15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:43.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9012a00640 1 -- 172.21.15.161:0/2218654156 >> 172.21.15.161:0/2218654156 conn(0x7f900c0b0bc0 msgr2=0x7f900c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:43.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9012a00640 1 -- 172.21.15.161:0/2218654156 shutdown_connections 2024-05-22T21:18:43.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9012a00640 1 -- 172.21.15.161:0/2218654156 wait complete. 2024-05-22T21:18:43.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9012a00640 1 Processor -- start 2024-05-22T21:18:43.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9012a00640 1 -- start start 2024-05-22T21:18:43.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9012a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f900c143ec0 0x7f900c159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:43.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9012a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f900c15ec90 0x7f900c1594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:43.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9012a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c1f6420 0x7f900c1f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:43.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9012a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c1f75a0 con 0x7f900c143ec0 2024-05-22T21:18:43.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9012a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c142f00 con 0x7f900c1f6420 2024-05-22T21:18:43.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.787+0000 7f9012a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c1431a0 con 0x7f900c15ec90 2024-05-22T21:18:43.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f900be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f900c143ec0 0x7f900c159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:43.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f900be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f900c143ec0 0x7f900c159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:38156/0 (socket says 172.21.15.161:38156) 2024-05-22T21:18:43.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f900be00640 1 -- 172.21.15.161:0/2398553588 learned_addr learned my addr 172.21.15.161:0/2398553588 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:43.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f9010a00640 1 --2- 172.21.15.161:0/2398553588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c1f6420 0x7f900c1f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:43.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f9003400640 1 --2- 172.21.15.161:0/2398553588 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f900c15ec90 0x7f900c1594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:43.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f900be00640 1 -- 172.21.15.161:0/2398553588 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f900c15ec90 msgr2=0x7f900c1594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:43.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f900be00640 1 --2- 172.21.15.161:0/2398553588 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f900c15ec90 0x7f900c1594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:43.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f900be00640 1 -- 172.21.15.161:0/2398553588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c1f6420 msgr2=0x7f900c1f68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:43.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f900be00640 1 --2- 172.21.15.161:0/2398553588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c1f6420 0x7f900c1f68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:43.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f900be00640 1 -- 172.21.15.161:0/2398553588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff4009420 con 0x7f900c143ec0 2024-05-22T21:18:43.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f9003400640 1 --2- 172.21.15.161:0/2398553588 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f900c15ec90 0x7f900c1594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:43.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f900be00640 1 --2- 172.21.15.161:0/2398553588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f900c143ec0 0x7f900c159c80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00c7c0 tx=0x7f8ffc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:43.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f9008c00640 1 -- 172.21.15.161:0/2398553588 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc007a30 con 0x7f900c143ec0 2024-05-22T21:18:43.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f9008c00640 1 -- 172.21.15.161:0/2398553588 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc007b90 con 0x7f900c143ec0 2024-05-22T21:18:43.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f9008c00640 1 -- 172.21.15.161:0/2398553588 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00ce70 con 0x7f900c143ec0 2024-05-22T21:18:43.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f9012a00640 1 -- 172.21.15.161:0/2398553588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c15f630 con 0x7f900c143ec0 2024-05-22T21:18:43.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f9012a00640 1 -- 172.21.15.161:0/2398553588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c159a00 con 0x7f900c143ec0 2024-05-22T21:18:43.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f9012a00640 1 -- 172.21.15.161:0/2398553588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900c067220 con 0x7f900c143ec0 2024-05-22T21:18:43.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f9008c00640 1 -- 172.21.15.161:0/2398553588 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8ffc0113f0 con 0x7f900c143ec0 2024-05-22T21:18:43.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f9008c00640 1 --2- 172.21.15.161:0/2398553588 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8fe80666c0 0x7f8fe8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:43.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.791+0000 7f9008c00640 1 -- 172.21.15.161:0/2398553588 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ffc084550 con 0x7f900c143ec0 2024-05-22T21:18:43.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.795+0000 7f9003400640 1 --2- 172.21.15.161:0/2398553588 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8fe80666c0 0x7f8fe8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:43.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.795+0000 7f9008c00640 1 -- 172.21.15.161:0/2398553588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=109}) v3 -- 0x7f8fe806e1f0 con 0x7f900c143ec0 2024-05-22T21:18:43.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.795+0000 7f9003400640 1 --2- 172.21.15.161:0/2398553588 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8fe80666c0 0x7f8fe8068b80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40af000 tx=0x7f8ff40a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:43.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.795+0000 7f9008c00640 1 -- 172.21.15.161:0/2398553588 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8ffc051960 con 0x7f900c143ec0 2024-05-22T21:18:43.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.955+0000 7f9012a00640 1 -- 172.21.15.161:0/2398553588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f900c06dcc0 con 0x7f900c143ec0 2024-05-22T21:18:43.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.955+0000 7f9008c00640 1 -- 172.21.15.161:0/2398553588 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v108) v1 ==== 81+0+33754 (secure 0 0 0) 0x7f8ffc054f80 con 0x7f900c143ec0 2024-05-22T21:18:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:43.960 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:18:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 108, 2024-05-22T21:18:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:18:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:18:43.960 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:18:42.591218+0000", 2024-05-22T21:18:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:18:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:18:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:18:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:18:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:18:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:18:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:18:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:18:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:18:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:18:43.961 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:18:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:18:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:18:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:18:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 13, 2024-05-22T21:18:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:18:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:18:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:18:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:18:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:18:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:18:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.962 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:18:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:18:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:18:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:18:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:18:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:43.963 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:18:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:18:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:18:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:18:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:18:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.964 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "105", 2024-05-22T21:18:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:43.965 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.966 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:18:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:18:43.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:18:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:18:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:18:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:18:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:18:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:18:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 12, 2024-05-22T21:18:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:18:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:18:16.500534+0000", 2024-05-22T21:18:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:18:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:18:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:43.969 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:18:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:18:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:18:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:18:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:18:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:43.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "105", 2024-05-22T21:18:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:43.971 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:43.972 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:18:43.973 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:18:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:18:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:18:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:18:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 2, 2024-05-22T21:18:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 2, 2024-05-22T21:18:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:18:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:18:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:18:43.974 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 13, 2024-05-22T21:18:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:18:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:18:16.937445+0000", 2024-05-22T21:18:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:18:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:18:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:18:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:18:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:18:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:18:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:18:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:43.976 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "105", 2024-05-22T21:18:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:43.977 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:43.978 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:43.979 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:18:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:18:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:18:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:18:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.3799999952316284, 2024-05-22T21:18:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.3799999952316284, 2024-05-22T21:18:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:18:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.3799999952316284, 2024-05-22T21:18:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.3799999952316284, 2024-05-22T21:18:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:18:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:18:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:43.981 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:43.982 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:18:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:18:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:43.983 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:18:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:18:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:43.984 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:18:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:18:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:43.985 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:18:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:18:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:18:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:18:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:18:43.986 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:18:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:18:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:18:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:43.987 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:18:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.988 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:18:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:18:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.989 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:18:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:18:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.990 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:43.991 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:18:43.991 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:43.991 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.991 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:43.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.991 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:43.991 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:43.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.991 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:43.991 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:18:43.991 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:43.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.991 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:43.991 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:18:43.992 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:43.992 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.992 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:43.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.992 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:18:43.992 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:18:43.992 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:18:43.992 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:18:43.992 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:43.992 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:43.992 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:18:43.992 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:43.992 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:43.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.993 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:18:43.993 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:43.993 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:43.993 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:43.993 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:43.993 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:43.993 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:43.993 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:43.993 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:43.993 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:43.993 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:43.993 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:43.993 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:43.994 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:43.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.994 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:43.994 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:18:43.994 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:43.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.994 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:43.994 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:18:43.994 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:43.994 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.994 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:43.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.995 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:43.995 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:43.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.995 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:43.995 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:18:43.995 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:43.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.995 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:43.995 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:18:43.995 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:43.995 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.995 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:43.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.996 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:43.996 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:43.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.996 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:43.996 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:18:43.996 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:43.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.996 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:43.996 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:18:43.996 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:43.996 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.996 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:43.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.997 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:43.997 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:43.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.997 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:43.997 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:18:43.997 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:43.997 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.997 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:43.997 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:18:43.997 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:43.997 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:43.997 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:43.997 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.998 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:18:43.998 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:18:43.998 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:18:43.998 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:18:43.998 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:43.998 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:43.998 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:18:43.998 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:43.998 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:43.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:43.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.998 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:18:43.999 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:43.999 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:43.999 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:43.999 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:43.999 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:43.999 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:43.999 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:43.999 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:43.999 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:43.999 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:43.999 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:43.999 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:43.999 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:43.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:43.999 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.000 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:18:44.000 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:44.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.000 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.000 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:18:44.000 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:44.000 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.000 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.000 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:44.000 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.000 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.001 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:18:44.001 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:44.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.001 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.001 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.001 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:18:44.001 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:44.001 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.001 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.001 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:44.001 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.001 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.002 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.002 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:18:44.002 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:44.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.002 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.002 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:18:44.002 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:44.002 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.002 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.002 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:44.002 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.003 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.003 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:18:44.003 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:44.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.003 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.003 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:18:44.003 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:44.003 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.003 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.003 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:18:44.003 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:18:44.003 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:18:44.004 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:18:44.004 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:44.004 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:44.004 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:44.004 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.004 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:18:44.004 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:44.004 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:44.004 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:44.004 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:44.004 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:44.004 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:44.005 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:44.005 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:44.005 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:44.005 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:44.005 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:44.005 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:44.005 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.005 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.005 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:18:44.005 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:44.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.005 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.005 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:18:44.006 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:44.006 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.006 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.006 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:44.006 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.006 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.006 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:18:44.006 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:44.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.006 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.007 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:18:44.007 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:44.007 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.007 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.007 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:44.007 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.007 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.007 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:18:44.007 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:44.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.007 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.008 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:18:44.008 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:44.008 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.008 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.008 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:44.008 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.008 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.008 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:18:44.008 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:44.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.008 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:18:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:18:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:18:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:18:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:18:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:18:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.010 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:18:44.010 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:44.010 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:44.010 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:44.010 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:44.010 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:44.010 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:44.010 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:44.010 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:44.010 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:44.010 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:44.010 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:44.010 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:44.011 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.011 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.011 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:18:44.011 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:44.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.011 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.011 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:18:44.011 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:44.011 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.011 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.011 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:44.012 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.012 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.012 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:18:44.012 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:44.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.012 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.012 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:18:44.012 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:44.012 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.012 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.012 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:44.012 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.013 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.013 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:18:44.013 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:44.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.013 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.013 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:18:44.013 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:44.013 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.013 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.013 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:44.013 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.014 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.014 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:18:44.014 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:44.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.014 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.014 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:18:44.014 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:44.014 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.014 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.014 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:18:44.014 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:18:44.015 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:18:44.015 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:18:44.015 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:44.015 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:44.015 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:44.015 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.015 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:18:44.015 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:44.015 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:44.015 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:44.015 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:44.015 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:44.015 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:44.016 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:44.016 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:44.016 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:44.016 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:44.016 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:44.016 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:44.016 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.016 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.016 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:18:44.016 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:44.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.016 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.017 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:18:44.017 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:44.017 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.017 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.017 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:44.017 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.017 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.017 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:18:44.017 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:44.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.017 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.018 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:18:44.018 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:44.018 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.018 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.018 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:44.018 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.018 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.018 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:18:44.018 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:44.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.018 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.019 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:18:44.019 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:44.019 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.019 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.019 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:44.019 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.019 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.019 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:18:44.019 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:44.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.019 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.019 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:18:44.020 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:44.020 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.020 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.020 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:18:44.020 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:18:44.020 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:18:44.020 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:18:44.020 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:44.020 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:44.020 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:18:44.020 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:44.020 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.020 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.021 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:18:44.021 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:44.021 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:44.021 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:44.021 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:44.021 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:44.021 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:44.021 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:44.021 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:44.021 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:44.021 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:44.021 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:44.021 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:44.021 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.022 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.022 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:18:44.022 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:44.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.022 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.022 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:18:44.022 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:44.022 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.022 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.022 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:44.022 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.023 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.023 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:18:44.023 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:44.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.023 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.023 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.023 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:18:44.023 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:44.023 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.023 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.023 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:44.023 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.024 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.024 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:18:44.024 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:44.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.024 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.024 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:18:44.024 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:44.024 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.024 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.024 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:44.024 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:44.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.025 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:44.025 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:18:44.025 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:44.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.025 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:44.025 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:18:44.025 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:44.025 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.025 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.025 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:18:44.025 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:18:44.025 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:18:44.026 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:18:44.026 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:44.026 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull", 2024-05-22T21:18:44.026 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:44.026 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:44.026 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:44.026 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.026 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:44.026 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:18:44.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.026 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:18:44.026 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:44.027 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:44.027 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:44.027 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:44.027 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:44.027 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:18:44.027 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:44.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.027 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:18:44.027 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:44.027 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:44.027 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:44.027 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:44.027 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:44.028 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:18:44.028 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:44.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.028 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:18:44.028 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:44.028 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:44.028 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:44.028 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:44.028 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:44.028 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:18:44.028 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:44.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.029 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:18:44.029 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:44.029 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:44.029 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:44.029 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:44.029 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:44.029 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:18:44.029 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:44.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.029 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:18:44.029 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:44.029 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:44.029 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:44.029 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:44.030 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:44.030 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:18:44.030 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:44.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.030 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:18:44.030 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:44.030 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:44.030 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:44.030 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:44.030 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:44.030 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:18:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:18:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:18:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:18:44.032 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:44.032 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:44.032 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:44.032 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:44.032 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:44.032 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:18:44.032 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:44.032 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.032 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:44.032 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:18:44.032 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:18:44.032 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:18:44.032 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:18:44.032 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:18:44.032 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:18:44.033 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:18:44.033 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:18:44.033 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:18:44.033 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:18:44.033 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:18:44.033 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:18:44.033 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:18:44.033 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:18:44.033 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:44.033 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:18:44.033 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:18:44.033 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:18:44.033 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:18:44.034 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:18:44.034 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:18:44.034 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:18:44.034 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:18:44.034 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:18:44.034 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:18:44.034 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:18:44.034 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:44.034 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:18:44.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.959+0000 7f9012a00640 1 -- 172.21.15.161:0/2398553588 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8fe80666c0 msgr2=0x7f8fe8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:44.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.959+0000 7f9012a00640 1 --2- 172.21.15.161:0/2398553588 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8fe80666c0 0x7f8fe8068b80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40af000 tx=0x7f8ff40a7000 comp rx=0 tx=0).stop 2024-05-22T21:18:44.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.959+0000 7f9012a00640 1 -- 172.21.15.161:0/2398553588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f900c143ec0 msgr2=0x7f900c159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:44.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.959+0000 7f9012a00640 1 --2- 172.21.15.161:0/2398553588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f900c143ec0 0x7f900c159c80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00c7c0 tx=0x7f8ffc00cc80 comp rx=0 tx=0).stop 2024-05-22T21:18:44.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.959+0000 7f9012a00640 1 -- 172.21.15.161:0/2398553588 shutdown_connections 2024-05-22T21:18:44.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.959+0000 7f9012a00640 1 --2- 172.21.15.161:0/2398553588 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8fe80666c0 0x7f8fe8068b80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:44.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.959+0000 7f9012a00640 1 --2- 172.21.15.161:0/2398553588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c1f6420 0x7f900c1f68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:44.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.959+0000 7f9012a00640 1 --2- 172.21.15.161:0/2398553588 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f900c15ec90 0x7f900c1594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:44.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.959+0000 7f9012a00640 1 --2- 172.21.15.161:0/2398553588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f900c143ec0 0x7f900c159c80 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:44.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.959+0000 7f9012a00640 1 -- 172.21.15.161:0/2398553588 >> 172.21.15.161:0/2398553588 conn(0x7f900c0b0bc0 msgr2=0x7f900c0ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:44.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.959+0000 7f9012a00640 1 -- 172.21.15.161:0/2398553588 shutdown_connections 2024-05-22T21:18:44.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:43.959+0000 7f9012a00640 1 -- 172.21.15.161:0/2398553588 wait complete. 2024-05-22T21:18:44.037 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-05-22T21:18:45.240 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:18:45.239+0000 7f7d35a00640 -1 log_channel(cluster) log [ERR] : Health check update: 5 full osd(s) (OSD_FULL) 2024-05-22T21:18:49.038 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:18:49.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.179+0000 7fe6ea200640 1 Processor -- start 2024-05-22T21:18:49.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.179+0000 7fe6ea200640 1 -- start start 2024-05-22T21:18:49.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.183+0000 7fe6ea200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe6e40715a0 0x7fe6e415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:49.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.183+0000 7fe6ea200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6e415ec90 0x7fe6e415f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:49.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.183+0000 7fe6ea200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e41442a0 con 0x7fe6e4143ec0 2024-05-22T21:18:49.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.183+0000 7fe6ea200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e415f660 con 0x7fe6e40715a0 2024-05-22T21:18:49.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.183+0000 7fe6ea200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e4070500 con 0x7fe6e415ec90 2024-05-22T21:18:49.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.183+0000 7fe6daa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe6e40715a0 0x7fe6e415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:49.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.183+0000 7fe6daa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe6e40715a0 0x7fe6e415f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59896/0 (socket says 172.21.15.161:59896) 2024-05-22T21:18:49.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.183+0000 7fe6daa00640 1 -- 172.21.15.161:0/4070143778 learned_addr learned my addr 172.21.15.161:0/4070143778 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:49.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.183+0000 7fe6e3e00640 1 --2- 172.21.15.161:0/4070143778 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6e415ec90 0x7fe6e415f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:49.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.183+0000 7fe6daa00640 1 -- 172.21.15.161:0/4070143778 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6e415ec90 msgr2=0x7fe6e415f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:49.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.183+0000 7fe6daa00640 1 --2- 172.21.15.161:0/4070143778 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6e415ec90 0x7fe6e415f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:49.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.183+0000 7fe6daa00640 1 -- 172.21.15.161:0/4070143778 >> v1:172.21.15.179:6789/0 conn(0x7fe6e4143ec0 legacy=0x7fe6e406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:49.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.183+0000 7fe6daa00640 1 -- 172.21.15.161:0/4070143778 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e40707e0 con 0x7fe6e40715a0 2024-05-22T21:18:49.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.187+0000 7fe6daa00640 1 --2- 172.21.15.161:0/4070143778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e40715a0 0x7fe6e415f830 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009870 tx=0x7fe6cc01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7a7f80a8a9fb5c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:49.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.187+0000 7fe6e2a00640 1 -- 172.21.15.161:0/4070143778 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc0a7070 con 0x7fe6e40715a0 2024-05-22T21:18:49.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.187+0000 7fe6e2a00640 1 -- 172.21.15.161:0/4070143778 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6cc0ab430 con 0x7fe6e40715a0 2024-05-22T21:18:49.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.187+0000 7fe6e2a00640 1 -- 172.21.15.161:0/4070143778 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc0ab7c0 con 0x7fe6e40715a0 2024-05-22T21:18:49.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.187+0000 7fe6ea200640 1 -- 172.21.15.161:0/4070143778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e40715a0 msgr2=0x7fe6e415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:49.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.187+0000 7fe6ea200640 1 --2- 172.21.15.161:0/4070143778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e40715a0 0x7fe6e415f830 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009870 tx=0x7fe6cc01c530 comp rx=0 tx=0).stop 2024-05-22T21:18:49.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.187+0000 7fe6ea200640 1 -- 172.21.15.161:0/4070143778 shutdown_connections 2024-05-22T21:18:49.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.187+0000 7fe6ea200640 1 --2- 172.21.15.161:0/4070143778 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6e415ec90 0x7fe6e415f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:49.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.187+0000 7fe6ea200640 1 --2- 172.21.15.161:0/4070143778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e40715a0 0x7fe6e415f830 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:49.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.187+0000 7fe6ea200640 1 -- 172.21.15.161:0/4070143778 >> 172.21.15.161:0/4070143778 conn(0x7fe6e40b0bc0 msgr2=0x7fe6e40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:49.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.187+0000 7fe6ea200640 1 -- 172.21.15.161:0/4070143778 shutdown_connections 2024-05-22T21:18:49.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.187+0000 7fe6ea200640 1 -- 172.21.15.161:0/4070143778 wait complete. 2024-05-22T21:18:49.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.187+0000 7fe6ea200640 1 Processor -- start 2024-05-22T21:18:49.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.187+0000 7fe6ea200640 1 -- start start 2024-05-22T21:18:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6ea200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6e4143ec0 0x7fe6e4162910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6ea200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6e415ec90 0x7fe6e412eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6ea200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e413f400 0x7fe6e4160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6ea200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e4162e50 con 0x7fe6e413f400 2024-05-22T21:18:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6ea200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e412c3a0 con 0x7fe6e415ec90 2024-05-22T21:18:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6ea200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e412f2e0 con 0x7fe6e4143ec0 2024-05-22T21:18:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6e3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e413f400 0x7fe6e4160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6e3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e413f400 0x7fe6e4160b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59904/0 (socket says 172.21.15.161:59904) 2024-05-22T21:18:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6e3e00640 1 -- 172.21.15.161:0/1526768993 learned_addr learned my addr 172.21.15.161:0/1526768993 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6e3400640 1 --2- 172.21.15.161:0/1526768993 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6e4143ec0 0x7fe6e4162910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6daa00640 1 --2- 172.21.15.161:0/1526768993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6e415ec90 0x7fe6e412eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6e3e00640 1 -- 172.21.15.161:0/1526768993 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6e4143ec0 msgr2=0x7fe6e4162910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6e3e00640 1 --2- 172.21.15.161:0/1526768993 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6e4143ec0 0x7fe6e4162910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:49.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6e3e00640 1 -- 172.21.15.161:0/1526768993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6e415ec90 msgr2=0x7fe6e412eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:49.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6e3e00640 1 --2- 172.21.15.161:0/1526768993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6e415ec90 0x7fe6e412eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:49.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6e3e00640 1 -- 172.21.15.161:0/1526768993 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc0094d0 con 0x7fe6e413f400 2024-05-22T21:18:49.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6e3e00640 1 --2- 172.21.15.161:0/1526768993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e413f400 0x7fe6e4160b40 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc009d30 tx=0x7fe6dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:49.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6dbe00640 1 -- 172.21.15.161:0/1526768993 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc009ba0 con 0x7fe6e413f400 2024-05-22T21:18:49.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6dbe00640 1 -- 172.21.15.161:0/1526768993 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6dc00f040 con 0x7fe6e413f400 2024-05-22T21:18:49.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6ea200640 1 -- 172.21.15.161:0/1526768993 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e412c610 con 0x7fe6e413f400 2024-05-22T21:18:49.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6dbe00640 1 -- 172.21.15.161:0/1526768993 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc0146c0 con 0x7fe6e413f400 2024-05-22T21:18:49.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6ea200640 1 -- 172.21.15.161:0/1526768993 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e412cbd0 con 0x7fe6e413f400 2024-05-22T21:18:49.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6ea200640 1 -- 172.21.15.161:0/1526768993 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e4067220 con 0x7fe6e413f400 2024-05-22T21:18:49.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.191+0000 7fe6dbe00640 1 -- 172.21.15.161:0/1526768993 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe6dc014820 con 0x7fe6e413f400 2024-05-22T21:18:49.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.195+0000 7fe6dbe00640 1 --2- 172.21.15.161:0/1526768993 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe6c0066690 0x7fe6c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:49.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.195+0000 7fe6dbe00640 1 -- 172.21.15.161:0/1526768993 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6dc084b30 con 0x7fe6e413f400 2024-05-22T21:18:49.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.195+0000 7fe6e3400640 1 --2- 172.21.15.161:0/1526768993 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe6c0066690 0x7fe6c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:49.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.195+0000 7fe6dbe00640 1 -- 172.21.15.161:0/1526768993 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=113}) v3 -- 0x7fe6c006e1c0 con 0x7fe6e413f400 2024-05-22T21:18:49.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.195+0000 7fe6e3400640 1 --2- 172.21.15.161:0/1526768993 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe6c0066690 0x7fe6c0068b50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4007fd0 tx=0x7fe6d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:49.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.199+0000 7fe6dbe00640 1 -- 172.21.15.161:0/1526768993 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe6dc018060 con 0x7fe6e413f400 2024-05-22T21:18:49.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.351+0000 7fe6ea200640 1 -- 172.21.15.161:0/1526768993 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe6e406dcc0 con 0x7fe6e413f400 2024-05-22T21:18:49.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.355+0000 7fe6dbe00640 1 -- 172.21.15.161:0/1526768993 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v112) v1 ==== 81+0+33650 (secure 0 0 0) 0x7fe6dc051f50 con 0x7fe6e413f400 2024-05-22T21:18:49.357 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:49.357 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:18:49.357 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 112, 2024-05-22T21:18:49.357 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:18:49.357 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:18:49.357 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:18:48.625468+0000", 2024-05-22T21:18:49.357 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:18:49.357 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:18:49.357 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:18:49.357 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:18:49.357 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:18:49.358 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:18:49.358 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:18:49.358 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:18:49.358 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:18:49.358 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:49.358 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:18:49.358 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:18:49.358 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:18:49.358 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:18:49.358 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:18:49.358 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 13, 2024-05-22T21:18:49.358 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:18:49.358 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:18:49.359 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:18:49.359 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:18:49.359 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:18:49.359 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:18:49.359 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.359 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:18:49.359 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:18:49.359 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:18:49.359 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:18:49.359 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:18:49.359 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:49.359 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:49.359 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:49.360 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:49.360 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:49.360 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:49.360 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:49.360 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:49.360 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:49.360 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:49.360 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:18:49.360 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:18:49.360 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:18:49.360 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:18:49.360 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:18:49.360 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:49.361 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:49.361 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:49.361 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:49.361 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:49.361 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:49.361 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:49.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.361 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "105", 2024-05-22T21:18:49.361 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:49.361 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:49.361 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:49.361 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:49.361 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:49.362 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:49.362 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:49.362 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:49.362 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:49.362 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:49.362 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:49.362 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:49.362 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:49.362 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:49.362 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:49.362 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:49.362 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:49.362 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:49.363 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:49.363 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:49.363 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:49.363 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:49.363 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:49.363 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:49.363 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:49.363 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:49.363 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.363 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:49.363 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:49.363 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:49.363 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:49.363 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:49.364 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:49.364 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:49.364 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:49.364 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:49.364 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:49.364 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:49.364 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:49.364 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:18:49.364 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:18:49.364 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.364 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:49.364 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:18:49.364 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.365 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:49.365 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:18:49.365 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:18:49.365 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:18:49.365 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:18:49.365 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:18:49.365 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:49.365 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:49.365 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:49.365 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.365 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.365 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 12, 2024-05-22T21:18:49.365 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:18:49.366 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:18:16.500534+0000", 2024-05-22T21:18:49.366 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:18:49.366 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:18:49.366 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:49.366 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:49.366 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:49.366 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:49.366 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:49.366 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:49.366 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:49.366 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:49.366 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:49.366 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:49.367 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:18:49.367 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:18:49.367 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:18:49.367 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:18:49.367 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:18:49.367 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:49.367 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:49.367 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:49.367 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:49.367 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:49.367 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:49.367 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:49.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.368 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "105", 2024-05-22T21:18:49.368 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:49.368 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:49.368 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:49.368 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:49.368 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:49.368 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:49.368 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:49.368 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:49.368 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:49.368 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:49.368 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:49.368 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:49.368 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:49.369 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:49.369 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:49.369 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:49.369 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:49.369 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:49.369 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:49.369 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:49.369 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:49.369 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:49.369 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:49.369 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:49.369 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:49.369 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:49.370 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.370 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:49.370 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:49.370 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:49.370 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:49.370 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:49.370 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:49.370 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:49.370 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:49.370 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:49.370 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:49.370 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:49.370 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:49.371 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:18:49.371 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:18:49.371 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:18:49.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.371 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:49.371 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:18:49.371 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:18:49.371 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.371 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:49.371 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 2, 2024-05-22T21:18:49.371 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 2, 2024-05-22T21:18:49.371 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:18:49.372 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:18:49.372 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:18:49.372 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:49.372 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:49.372 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:49.372 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.372 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.372 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 13, 2024-05-22T21:18:49.372 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:18:49.372 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:18:16.937445+0000", 2024-05-22T21:18:49.372 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 3, 2024-05-22T21:18:49.372 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full", 2024-05-22T21:18:49.372 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:49.373 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:49.373 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:49.373 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:49.373 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:49.373 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:49.373 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:49.373 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:49.373 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:49.373 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:49.373 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:18:49.373 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:18:49.373 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:18:49.373 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:18:49.374 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:18:49.374 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:49.374 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:49.374 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:49.374 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:49.374 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:49.374 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:49.374 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:49.374 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.374 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "105", 2024-05-22T21:18:49.374 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:49.374 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:49.374 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:49.375 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:49.375 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:49.375 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:49.375 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:49.375 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:49.375 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:49.375 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:49.375 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:49.375 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:49.375 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:49.375 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:49.375 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:49.375 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:49.376 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:49.376 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:49.376 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:49.376 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:49.376 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:49.376 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:49.376 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:49.376 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:49.376 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:49.376 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:49.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.376 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:49.376 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:49.377 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:49.377 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:49.377 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:49.377 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:49.377 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:49.377 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:49.377 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:49.377 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:49.377 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:49.377 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:49.377 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:18:49.377 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:18:49.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.377 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:49.378 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:18:49.378 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:18:49.378 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.378 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:49.378 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.3799999952316284, 2024-05-22T21:18:49.378 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.3799999952316284, 2024-05-22T21:18:49.378 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:18:49.378 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.3799999952316284, 2024-05-22T21:18:49.378 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.3799999952316284, 2024-05-22T21:18:49.378 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:49.378 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:49.378 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:49.379 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.379 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.379 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:49.379 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:18:49.379 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.379 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:18:49.379 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:49.379 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:49.379 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:49.379 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:49.379 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:49.379 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:49.379 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:49.380 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:49.380 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:49.380 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:49.380 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:49.380 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:49.380 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.380 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.380 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.380 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:18:49.380 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:49.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.380 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.380 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.381 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:18:49.381 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:49.381 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.381 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.381 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.381 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:49.381 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.381 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.381 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.381 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:18:49.381 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:49.381 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.381 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.382 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.382 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:18:49.382 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:49.382 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.382 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.382 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.382 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:49.382 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.382 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.382 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.382 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:18:49.382 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:49.382 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.383 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.383 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.383 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:18:49.383 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:49.383 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.383 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.383 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.383 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:49.383 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.383 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.383 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.383 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:18:49.383 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:49.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.384 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.384 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.384 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:18:49.384 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:49.384 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.384 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.384 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:18:49.384 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:18:49.384 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:18:49.384 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:18:49.384 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:49.384 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:49.385 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:49.385 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.385 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.385 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:18:49.385 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:49.385 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:49.385 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:49.385 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:49.385 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:49.385 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:49.385 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:49.385 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:49.385 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:49.386 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:49.386 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:49.386 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:49.386 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.386 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.386 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.386 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:18:49.386 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:49.386 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.386 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.386 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.386 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:18:49.386 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:49.387 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.387 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.387 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:49.387 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.387 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.387 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.387 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:18:49.387 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:49.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.387 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.387 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.387 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:18:49.388 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:49.388 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.388 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.388 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:49.388 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.388 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.388 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.388 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:18:49.388 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:49.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.388 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.388 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.388 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:18:49.389 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:49.389 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.389 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.389 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:49.389 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.389 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.389 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.389 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:18:49.389 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:49.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.389 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.389 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.390 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:18:49.390 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:49.390 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.390 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.390 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:18:49.390 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:18:49.390 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:18:49.390 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:18:49.390 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:49.390 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:49.390 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:18:49.390 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:49.391 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.391 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.391 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:18:49.391 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:49.391 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:49.391 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:49.391 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:49.391 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:49.391 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:49.391 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:49.391 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:49.391 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:49.391 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:49.392 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:49.392 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:49.392 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.392 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.392 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.392 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:18:49.392 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:49.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.392 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.392 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.392 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:18:49.392 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:49.392 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.393 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.393 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:49.393 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.393 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.393 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.393 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:18:49.393 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:49.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.393 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.393 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.393 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:18:49.393 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:49.394 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.394 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.394 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:49.394 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.394 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.394 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.394 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:18:49.394 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:49.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.394 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.394 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.394 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:18:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:18:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:18:49.396 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:49.396 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.396 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.396 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:18:49.396 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:18:49.396 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:18:49.396 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:18:49.396 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:49.396 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:49.396 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:49.396 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.397 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.397 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:18:49.397 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:49.397 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:49.397 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:49.397 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:49.397 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:49.397 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:49.397 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:49.397 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:49.397 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:49.397 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:49.397 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:49.397 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:49.398 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.398 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.398 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.398 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:18:49.398 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:49.398 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.398 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.398 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.398 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:18:49.398 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:49.398 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.398 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.398 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.399 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:49.399 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.399 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.399 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.399 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:18:49.399 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:49.399 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.399 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.399 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.399 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:18:49.399 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:49.399 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.399 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.400 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:49.400 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.400 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.400 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.400 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:18:49.400 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:49.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.400 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.400 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.400 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:18:49.400 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:49.400 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.400 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.401 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:49.401 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.401 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.401 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.401 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:18:49.401 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:49.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.401 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.401 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.401 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:18:49.401 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:49.401 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.402 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.402 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:18:49.402 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:18:49.402 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:18:49.402 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:18:49.402 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:49.402 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:49.402 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:49.402 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.402 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.402 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:18:49.403 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:49.403 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:49.403 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:49.403 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:49.403 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:49.403 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:49.403 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:49.403 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:49.403 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:49.403 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:49.403 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:49.403 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:49.403 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.404 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.404 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.404 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:18:49.404 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:49.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.404 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.404 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.404 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:18:49.404 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:49.404 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.404 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.404 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:49.404 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.405 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.405 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.405 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:18:49.405 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:49.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.405 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.405 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.405 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:18:49.405 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:49.405 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.405 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.405 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:49.406 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.406 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.406 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.406 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:18:49.406 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:49.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.406 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.406 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.406 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:18:49.406 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:49.406 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.406 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.407 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:49.407 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.407 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.407 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:18:49.407 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:49.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.407 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.407 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:18:49.407 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:49.408 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.408 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.408 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:18:49.408 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:18:49.408 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:18:49.408 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:18:49.408 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:49.408 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:49.408 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:49.409 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.409 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.409 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:18:49.409 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:49.409 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:49.409 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:49.409 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:49.409 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:49.409 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:49.409 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:49.409 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:49.410 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:49.410 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:49.410 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:49.410 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:49.410 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.410 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.410 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.410 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:18:49.410 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:49.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.411 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.411 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:18:49.411 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:49.411 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.411 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.411 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:49.411 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.411 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.411 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:18:49.412 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:49.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.412 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.412 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:18:49.412 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:49.412 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.412 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.413 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:49.413 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.413 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.413 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:18:49.413 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:49.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.413 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.413 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:18:49.413 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:49.413 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.414 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.414 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:49.414 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.414 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.414 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.414 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:18:49.414 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:49.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.414 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.414 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.415 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:18:49.415 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:49.415 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.415 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.415 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:18:49.415 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:18:49.415 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:18:49.415 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:18:49.415 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:49.415 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:49.416 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:49.416 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.416 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:18:49.416 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:49.416 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:49.416 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:49.416 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:49.416 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:49.416 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:49.417 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:49.417 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:49.417 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:49.417 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:49.417 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:49.417 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:49.417 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.417 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.417 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:18:49.417 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:49.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.418 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.418 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:18:49.418 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:49.418 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.418 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.418 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:49.418 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.419 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.419 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:18:49.419 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:49.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.419 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.419 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:18:49.419 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:49.419 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.419 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.420 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:49.420 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.420 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.420 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:18:49.420 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:49.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.420 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.420 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:18:49.420 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:49.420 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.421 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.421 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:49.421 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.421 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.421 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:18:49.421 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:49.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.421 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.421 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:18:49.421 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:49.422 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.422 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.422 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:18:49.422 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:18:49.422 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:18:49.422 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:18:49.422 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:49.422 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:49.422 INFO:teuthology.orchestra.run.smithi161.stdout: "full", 2024-05-22T21:18:49.422 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:49.422 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.423 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.423 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:18:49.423 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:49.423 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:49.423 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:49.423 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:49.423 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:49.423 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:49.423 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:49.423 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:49.423 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:49.424 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:49.424 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:49.424 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:49.424 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.424 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.424 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:18:49.424 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:49.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.424 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.424 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:18:49.425 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:49.425 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.425 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.425 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:49.425 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.425 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.425 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:18:49.425 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:49.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.426 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.426 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:18:49.426 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:49.426 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.426 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.426 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:49.426 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.426 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.426 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:18:49.426 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:49.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.427 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.427 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:18:49.427 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:49.427 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.427 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.427 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:49.427 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:49.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.427 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:49.428 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:18:49.428 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:49.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.428 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:49.428 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:18:49.428 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:49.428 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.428 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.428 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:18:49.428 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:18:49.428 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:18:49.429 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:18:49.429 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:49.429 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:49.429 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:49.429 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:49.429 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.429 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:49.429 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:18:49.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.429 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:18:49.429 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:49.429 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:49.429 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:49.430 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:49.430 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:49.430 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:18:49.430 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:49.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.430 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.430 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:18:49.430 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:49.430 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:49.430 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:49.430 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:49.431 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:49.431 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:18:49.431 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:49.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.431 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:18:49.431 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:49.431 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:49.431 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:49.431 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:49.431 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:49.431 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:18:49.432 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:49.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.432 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:18:49.432 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:49.432 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:49.432 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:49.432 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:49.432 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:49.432 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:18:49.432 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:49.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.433 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:18:49.433 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:49.433 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:49.433 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:49.433 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:49.433 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:49.433 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:18:49.433 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:49.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.434 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.434 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:18:49.434 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:49.434 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:49.434 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:49.434 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:49.434 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:49.434 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:18:49.434 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:49.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.434 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.434 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:18:49.435 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:49.435 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:49.435 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:49.435 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:49.435 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:49.435 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:18:49.435 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:49.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.435 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:49.435 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:18:49.435 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:49.435 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:49.435 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:49.436 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:49.436 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:49.436 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:18:49.436 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:49.436 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.436 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:49.436 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:18:49.436 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:18:49.436 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:18:49.436 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:18:49.436 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:18:49.437 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:18:49.437 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:18:49.437 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:18:49.437 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:18:49.437 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:18:49.437 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:18:49.437 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:18:49.437 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:18:49.437 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:18:49.437 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:49.437 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:18:49.438 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:18:49.438 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:18:49.438 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:18:49.438 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:18:49.438 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:18:49.438 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:18:49.438 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:18:49.438 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:18:49.438 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:18:49.438 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:18:49.439 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:49.439 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:18:49.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.355+0000 7fe6ea200640 1 -- 172.21.15.161:0/1526768993 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe6c0066690 msgr2=0x7fe6c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:49.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.355+0000 7fe6ea200640 1 --2- 172.21.15.161:0/1526768993 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe6c0066690 0x7fe6c0068b50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4007fd0 tx=0x7fe6d4009040 comp rx=0 tx=0).stop 2024-05-22T21:18:49.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.355+0000 7fe6ea200640 1 -- 172.21.15.161:0/1526768993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e413f400 msgr2=0x7fe6e4160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:49.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.355+0000 7fe6ea200640 1 --2- 172.21.15.161:0/1526768993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e413f400 0x7fe6e4160b40 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc009d30 tx=0x7fe6dc00c490 comp rx=0 tx=0).stop 2024-05-22T21:18:49.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.355+0000 7fe6ea200640 1 -- 172.21.15.161:0/1526768993 shutdown_connections 2024-05-22T21:18:49.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.355+0000 7fe6ea200640 1 --2- 172.21.15.161:0/1526768993 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe6c0066690 0x7fe6c0068b50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:49.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.355+0000 7fe6ea200640 1 --2- 172.21.15.161:0/1526768993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e413f400 0x7fe6e4160b40 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:49.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.355+0000 7fe6ea200640 1 --2- 172.21.15.161:0/1526768993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6e415ec90 0x7fe6e412eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:49.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.355+0000 7fe6ea200640 1 --2- 172.21.15.161:0/1526768993 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6e4143ec0 0x7fe6e4162910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:49.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.355+0000 7fe6ea200640 1 -- 172.21.15.161:0/1526768993 >> 172.21.15.161:0/1526768993 conn(0x7fe6e40b0bc0 msgr2=0x7fe6e40b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:49.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.355+0000 7fe6ea200640 1 -- 172.21.15.161:0/1526768993 shutdown_connections 2024-05-22T21:18:49.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:49.355+0000 7fe6ea200640 1 -- 172.21.15.161:0/1526768993 wait complete. 2024-05-22T21:18:49.442 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-05-22T21:18:50.241 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:18:50.239+0000 7f7d35a00640 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2024-05-22T21:18:54.012 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:18:54.016 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:18:54.443 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:18:54.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.587+0000 7fb11a000640 1 Processor -- start 2024-05-22T21:18:54.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.587+0000 7fb11a000640 1 -- start start 2024-05-22T21:18:54.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb11a000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb1140d0a40 0x7fb1140d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:54.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb11a000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1140cb8a0 con 0x7fb11415af00 2024-05-22T21:18:54.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb11a000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1140d13f0 con 0x7fb1140cc320 2024-05-22T21:18:54.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb11a000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11415f530 con 0x7fb1140d0a40 2024-05-22T21:18:54.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb118c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb11415af00 0x7fb11415eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44298/0 (socket says 172.21.15.161:44298) 2024-05-22T21:18:54.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb118c00640 1 -- 172.21.15.161:0/2129264539 learned_addr learned my addr 172.21.15.161:0/2129264539 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:54.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb10be00640 1 --2- 172.21.15.161:0/2129264539 >> v2:172.21.15.179:3300/0 conn(0x7fb1140d0a40 0x7fb1140d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:54.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb113e00640 1 -- 172.21.15.161:0/2129264539 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930285402 0 0) 0x7fb1140cb8a0 con 0x7fb11415af00 2024-05-22T21:18:54.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb113e00640 1 -- 172.21.15.161:0/2129264539 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0f8003530 con 0x7fb11415af00 2024-05-22T21:18:54.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb113e00640 1 -- 172.21.15.161:0/2129264539 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 100027477 0 0) 0x7fb0f8003530 con 0x7fb11415af00 2024-05-22T21:18:54.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb113e00640 1 -- 172.21.15.161:0/2129264539 >> v2:172.21.15.179:3300/0 conn(0x7fb1140d0a40 msgr2=0x7fb1140d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:54.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb113e00640 1 --2- 172.21.15.161:0/2129264539 >> v2:172.21.15.179:3300/0 conn(0x7fb1140d0a40 0x7fb1140d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb113e00640 1 -- 172.21.15.161:0/2129264539 >> v1:172.21.15.179:6789/0 conn(0x7fb1140cc320 legacy=0x7fb1140cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:54.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb113e00640 1 -- 172.21.15.161:0/2129264539 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11415f810 con 0x7fb11415af00 2024-05-22T21:18:54.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb113e00640 1 -- 172.21.15.161:0/2129264539 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fb104001680 con 0x7fb11415af00 2024-05-22T21:18:54.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb113e00640 1 -- 172.21.15.161:0/2129264539 >> v1:172.21.15.161:6789/0 conn(0x7fb11415af00 legacy=0x7fb11415eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:54.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb113e00640 1 --2- 172.21.15.161:0/2129264539 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f8003980 0x7fb0f8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:54.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb113e00640 1 --2- 172.21.15.161:0/2129264539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f80047d0 0x7fb0f8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:54.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb113e00640 1 --2- 172.21.15.161:0/2129264539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f8005190 0x7fb0f800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:54.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb113e00640 1 -- 172.21.15.161:0/2129264539 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11415f530 con 0x7fb0f80047d0 2024-05-22T21:18:54.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb113e00640 1 -- 172.21.15.161:0/2129264539 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f800c030 con 0x7fb0f8005190 2024-05-22T21:18:54.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb113e00640 1 -- 172.21.15.161:0/2129264539 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f800c1f0 con 0x7fb0f8003980 2024-05-22T21:18:54.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb10be00640 1 --2- 172.21.15.161:0/2129264539 >> v2:172.21.15.179:3300/0 conn(0x7fb1140d0a40 0x7fb1140d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:54.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb10be00640 1 --2- 172.21.15.161:0/2129264539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f80047d0 0x7fb0f8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:54.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb119600640 1 --2- 172.21.15.161:0/2129264539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f8005190 0x7fb0f800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:54.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb118c00640 1 --2- 172.21.15.161:0/2129264539 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f8003980 0x7fb0f8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:54.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb10be00640 1 -- 172.21.15.161:0/2129264539 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f8003980 msgr2=0x7fb0f8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:54.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb10be00640 1 --2- 172.21.15.161:0/2129264539 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f8003980 0x7fb0f8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb10be00640 1 -- 172.21.15.161:0/2129264539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f8005190 msgr2=0x7fb0f800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:54.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb10be00640 1 --2- 172.21.15.161:0/2129264539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f8005190 0x7fb0f800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb119600640 1 -- 172.21.15.161:0/2129264539 reap_dead start 2024-05-22T21:18:54.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.591+0000 7fb10be00640 1 -- 172.21.15.161:0/2129264539 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f8003530 con 0x7fb0f80047d0 2024-05-22T21:18:54.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.595+0000 7fb10be00640 1 --2- 172.21.15.161:0/2129264539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f80047d0 0x7fb0f8004c50 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009c90 tx=0x7fb0fc01cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e175dd887c729c5c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:54.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.595+0000 7fb113e00640 1 -- 172.21.15.161:0/2129264539 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc004020 con 0x7fb0f80047d0 2024-05-22T21:18:54.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.595+0000 7fb113e00640 1 -- 172.21.15.161:0/2129264539 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc004180 con 0x7fb0f80047d0 2024-05-22T21:18:54.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.595+0000 7fb113e00640 1 -- 172.21.15.161:0/2129264539 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0a56a0 con 0x7fb0f80047d0 2024-05-22T21:18:54.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.595+0000 7fb11a000640 1 -- 172.21.15.161:0/2129264539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f80047d0 msgr2=0x7fb0f8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:54.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.595+0000 7fb11a000640 1 --2- 172.21.15.161:0/2129264539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f80047d0 0x7fb0f8004c50 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009c90 tx=0x7fb0fc01cbf0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.595+0000 7fb11a000640 1 -- 172.21.15.161:0/2129264539 shutdown_connections 2024-05-22T21:18:54.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.595+0000 7fb11a000640 1 --2- 172.21.15.161:0/2129264539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f80047d0 0x7fb0f8004c50 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.595+0000 7fb11a000640 1 -- 172.21.15.161:0/2129264539 >> 172.21.15.161:0/2129264539 conn(0x7fb114099a40 msgr2=0x7fb114099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:54.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.595+0000 7fb11a000640 1 -- 172.21.15.161:0/2129264539 shutdown_connections 2024-05-22T21:18:54.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.595+0000 7fb11a000640 1 -- 172.21.15.161:0/2129264539 wait complete. 2024-05-22T21:18:54.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb11a000640 1 Processor -- start 2024-05-22T21:18:54.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb11a000640 1 -- start start 2024-05-22T21:18:54.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb11a000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f80047d0 0x7fb114154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:54.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb11a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb114159200 0x7fb1140c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:54.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb11a000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1140c9710 0x7fb1141541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:54.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb11a000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb114159d40 con 0x7fb114159200 2024-05-22T21:18:54.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb11a000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1140c89f0 con 0x7fb1140c9710 2024-05-22T21:18:54.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb11a000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb114159fe0 con 0x7fb0f80047d0 2024-05-22T21:18:54.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb10be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb114159200 0x7fb1140c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:54.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb10be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb114159200 0x7fb1140c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59922/0 (socket says 172.21.15.161:59922) 2024-05-22T21:18:54.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb10be00640 1 -- 172.21.15.161:0/1493519173 learned_addr learned my addr 172.21.15.161:0/1493519173 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:54.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb118c00640 1 --2- 172.21.15.161:0/1493519173 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f80047d0 0x7fb114154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:54.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb119600640 1 --2- 172.21.15.161:0/1493519173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1140c9710 0x7fb1141541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:54.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb10be00640 1 -- 172.21.15.161:0/1493519173 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f80047d0 msgr2=0x7fb114154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:54.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb10be00640 1 --2- 172.21.15.161:0/1493519173 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f80047d0 0x7fb114154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb10be00640 1 -- 172.21.15.161:0/1493519173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1140c9710 msgr2=0x7fb1141541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:54.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb10be00640 1 --2- 172.21.15.161:0/1493519173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1140c9710 0x7fb1141541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb10be00640 1 -- 172.21.15.161:0/1493519173 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1140c8cb0 con 0x7fb114159200 2024-05-22T21:18:54.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb118c00640 1 --2- 172.21.15.161:0/1493519173 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f80047d0 0x7fb114154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:54.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb119600640 1 --2- 172.21.15.161:0/1493519173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1140c9710 0x7fb1141541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:54.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb10be00640 1 --2- 172.21.15.161:0/1493519173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb114159200 0x7fb1140c91d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc01ca20 tx=0x7fb0f8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:54.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb111600640 1 -- 172.21.15.161:0/1493519173 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0a58a0 con 0x7fb114159200 2024-05-22T21:18:54.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb111600640 1 -- 172.21.15.161:0/1493519173 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc004500 con 0x7fb114159200 2024-05-22T21:18:54.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb111600640 1 -- 172.21.15.161:0/1493519173 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0a5c60 con 0x7fb114159200 2024-05-22T21:18:54.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb11a000640 1 -- 172.21.15.161:0/1493519173 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb114159680 con 0x7fb114159200 2024-05-22T21:18:54.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb11a000640 1 -- 172.21.15.161:0/1493519173 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb114154900 con 0x7fb114159200 2024-05-22T21:18:54.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.599+0000 7fb11a000640 1 -- 172.21.15.161:0/1493519173 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1140c8e40 con 0x7fb114159200 2024-05-22T21:18:54.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.603+0000 7fb111600640 1 -- 172.21.15.161:0/1493519173 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb0fc006040 con 0x7fb114159200 2024-05-22T21:18:54.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.603+0000 7fb111600640 1 --2- 172.21.15.161:0/1493519173 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0f0066690 0x7fb0f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:54.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.603+0000 7fb111600640 1 -- 172.21.15.161:0/1493519173 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0fc11ec10 con 0x7fb114159200 2024-05-22T21:18:54.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.603+0000 7fb118c00640 1 --2- 172.21.15.161:0/1493519173 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0f0066690 0x7fb0f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:54.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.603+0000 7fb118c00640 1 --2- 172.21.15.161:0/1493519173 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0f0066690 0x7fb0f0068b50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb1040011f0 tx=0x7fb104001160 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:54.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.607+0000 7fb111600640 1 -- 172.21.15.161:0/1493519173 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb0fc0ec000 con 0x7fb114159200 2024-05-22T21:18:54.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.759+0000 7fb11a000640 1 -- 172.21.15.161:0/1493519173 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb1140d0ed0 con 0x7fb114159200 2024-05-22T21:18:54.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.759+0000 7fb111600640 1 -- 172.21.15.161:0/1493519173 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v113) v1 ==== 81+0+33587 (secure 0 0 0) 0x7fb0fc0ba070 con 0x7fb114159200 2024-05-22T21:18:54.764 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:54.764 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:18:54.764 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 113, 2024-05-22T21:18:54.764 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:18:54.764 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:18:54.764 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:18:50.632514+0000", 2024-05-22T21:18:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:18:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:18:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:18:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:18:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:18:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:18:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:18:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:18:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:18:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:18:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:18:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:18:54.765 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:18:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:18:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 13, 2024-05-22T21:18:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:18:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:18:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:18:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:18:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:18:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:18:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:18:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:18:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:18:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:18:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:18:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:18:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:18:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:18:54.767 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:18:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:18:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:18:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:54.768 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:54.769 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:54.770 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:18:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:18:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:18:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:54.771 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:18:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:18:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:18:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:18:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:18:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 12, 2024-05-22T21:18:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:18:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:18:16.500534+0000", 2024-05-22T21:18:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:18:54.772 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:18:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:18:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:18:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:18:54.773 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:18:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:18:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:18:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:54.774 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:54.775 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:54.776 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:54.777 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:54.777 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:54.777 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:54.777 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:54.777 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:54.777 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:54.777 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:18:54.777 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:18:54.777 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:18:54.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.777 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:54.777 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:18:54.777 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:18:54.777 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.778 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:54.778 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 2, 2024-05-22T21:18:54.778 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 2, 2024-05-22T21:18:54.778 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:18:54.778 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:18:54.778 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:18:54.778 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:54.778 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:54.778 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:54.778 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.778 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.778 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 13, 2024-05-22T21:18:54.779 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:18:54.779 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:18:16.937445+0000", 2024-05-22T21:18:54.779 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:18:54.779 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:18:54.779 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:54.779 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:54.779 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:54.779 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:54.779 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:54.779 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:54.779 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:54.779 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:54.779 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:54.779 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:54.779 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:18:54.780 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:18:54.780 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:18:54.780 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:18:54.780 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:18:54.780 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:54.780 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:54.780 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:54.780 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:54.780 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:54.780 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:54.780 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:54.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.780 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:18:54.780 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:54.780 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:54.781 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:54.781 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:54.781 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:54.781 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:54.781 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:54.781 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:54.781 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:54.781 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:54.781 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:54.781 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:54.781 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:54.781 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:54.781 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:54.781 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:54.782 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:54.782 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:54.782 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:54.782 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:54.782 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:54.782 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:54.782 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:54.782 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:54.782 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:54.782 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:54.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.782 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:54.782 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:54.783 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:54.783 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:54.783 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:54.783 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:54.783 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:54.783 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:54.783 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:54.783 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:54.783 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:54.783 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:54.783 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:18:54.783 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:18:54.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.783 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:54.784 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:18:54.784 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:18:54.784 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.784 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:54.784 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.3799999952316284, 2024-05-22T21:18:54.784 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.3799999952316284, 2024-05-22T21:18:54.784 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:18:54.784 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.3799999952316284, 2024-05-22T21:18:54.784 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.3799999952316284, 2024-05-22T21:18:54.784 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:54.784 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:54.784 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:54.784 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.785 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.785 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:54.785 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:18:54.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.785 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:18:54.785 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:54.785 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:54.785 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:54.785 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:54.785 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:54.785 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:54.785 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:54.785 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:54.785 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:54.785 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:54.786 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:54.786 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:54.786 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.786 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.786 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:18:54.786 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:54.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.786 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.786 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:18:54.786 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:54.787 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.787 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.787 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:54.787 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.787 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.787 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.787 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:18:54.787 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:54.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.787 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.787 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.787 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:18:54.787 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:54.788 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.788 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.788 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.788 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:54.788 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.788 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.788 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:18:54.788 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:54.788 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.788 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.788 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:18:54.789 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:54.789 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.789 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.789 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:54.789 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.789 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.789 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.789 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:18:54.789 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:54.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.789 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.789 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.789 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:18:54.790 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:54.790 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.790 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.790 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:18:54.790 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:18:54.790 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:18:54.790 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:18:54.790 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:54.790 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:54.790 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:54.790 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.791 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:18:54.791 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:54.791 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:54.791 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:54.791 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:54.791 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:54.791 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:54.791 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:54.791 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:54.791 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:54.791 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:54.791 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:54.791 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:54.792 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.792 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.792 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.792 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:18:54.792 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:54.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.792 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.792 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.792 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:18:54.792 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:54.792 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.792 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.793 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:54.793 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.793 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.793 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.793 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:18:54.793 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:54.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.793 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.793 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.793 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:18:54.793 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:54.793 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.793 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.794 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:54.794 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.794 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.794 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.794 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:18:54.794 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:54.794 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.794 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.794 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.794 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:18:54.794 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:54.794 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.794 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.795 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:54.795 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.795 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.795 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.795 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:18:54.795 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:54.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.795 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.795 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.795 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:18:54.795 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:54.795 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.796 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.796 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:18:54.796 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:18:54.796 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:18:54.796 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:18:54.796 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:54.796 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:54.796 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:54.796 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.796 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.796 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:18:54.796 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:54.797 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:54.797 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:54.797 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:54.797 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:54.797 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:54.797 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:54.797 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:54.797 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:54.797 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:54.797 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:54.797 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:54.797 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.797 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.797 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.798 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:18:54.798 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:54.798 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.798 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.798 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.798 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:18:54.798 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:54.798 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.798 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.798 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.798 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:54.798 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.798 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.799 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.799 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:18:54.799 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:54.799 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.799 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.799 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.799 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:18:54.799 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:54.799 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.799 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.799 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.799 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:54.799 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.799 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.800 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.800 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:18:54.800 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:54.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.800 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.800 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.800 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:18:54.800 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:54.800 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.800 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.800 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:54.800 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.800 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.801 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.801 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:18:54.801 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:54.801 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.801 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.801 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.801 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:18:54.801 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:54.801 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.801 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.801 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.801 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:18:54.801 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:18:54.801 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:18:54.802 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:18:54.802 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:54.802 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:54.802 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:54.802 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.802 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.802 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.802 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:18:54.802 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:54.802 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:54.802 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:54.802 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:54.803 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:54.803 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:54.803 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:54.803 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:54.803 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:54.803 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:54.803 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:54.803 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:54.803 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.803 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.803 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.803 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:18:54.803 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:54.803 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.804 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.804 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.804 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:18:54.804 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:54.804 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.804 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.804 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:54.804 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.804 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.804 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.804 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:18:54.804 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:54.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.804 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.805 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.805 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:18:54.805 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:54.805 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.805 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.805 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:54.805 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.805 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.805 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.805 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:18:54.805 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:54.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.805 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.806 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.806 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:18:54.806 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:54.806 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.806 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.806 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.806 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:54.806 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.806 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.806 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.806 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:18:54.806 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:54.807 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.807 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.807 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.807 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:18:54.807 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:54.807 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.807 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.807 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.807 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:18:54.807 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:18:54.807 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:18:54.807 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:18:54.807 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:54.807 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:54.808 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:54.808 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.808 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.808 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:18:54.808 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:54.808 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:54.808 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:54.808 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:54.808 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:54.808 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:54.808 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:54.808 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:54.808 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:54.809 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:54.809 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:54.809 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:54.809 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.809 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.809 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.809 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:18:54.809 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:54.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.809 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.809 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.809 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:18:54.809 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:54.810 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.810 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.810 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.810 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:54.810 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.810 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.810 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.810 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:18:54.810 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:54.810 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.810 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.810 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.810 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:18:54.811 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:54.811 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.811 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.811 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.811 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:54.811 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.811 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.811 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.811 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:18:54.811 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:54.811 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.811 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.811 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.811 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:18:54.812 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:54.812 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.812 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.812 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.812 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:54.812 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.812 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.812 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.812 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:18:54.812 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:54.813 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.813 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.813 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.813 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:18:54.813 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:54.813 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.813 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.813 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.813 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:18:54.813 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:18:54.813 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:18:54.813 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:18:54.813 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:54.814 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:54.814 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:54.814 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.814 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.814 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.814 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:18:54.814 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:54.814 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:54.814 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:54.814 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:54.814 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:54.814 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:54.814 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:54.815 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:54.815 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:54.815 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:54.815 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:54.815 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:54.815 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.815 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.815 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.815 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:18:54.815 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:54.815 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.815 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.815 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.816 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:18:54.816 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:54.816 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.816 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.816 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.816 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:54.816 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.816 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.816 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.816 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:18:54.816 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:54.816 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.816 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.816 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.817 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:18:54.817 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:54.817 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.817 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.817 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.817 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:54.817 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.817 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.817 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.817 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:18:54.817 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:54.817 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.817 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.817 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.818 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:18:54.818 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:54.818 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.818 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.818 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.818 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:54.818 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.818 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.818 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.818 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:18:54.818 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:54.818 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.819 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.819 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.819 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:18:54.819 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:54.819 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.819 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.819 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.819 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:18:54.819 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:18:54.819 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:18:54.819 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:18:54.819 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:54.820 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:54.820 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:54.820 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.820 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.820 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:18:54.820 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:54.820 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:54.820 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:54.820 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:54.820 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:54.820 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:54.820 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:54.821 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:54.821 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:54.821 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:54.821 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:54.821 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:54.821 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.821 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.821 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.821 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:18:54.821 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:54.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.821 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.821 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.821 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:18:54.822 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:54.822 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.822 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.822 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:54.822 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.822 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.822 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.822 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:18:54.822 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:54.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.822 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.823 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.823 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:18:54.823 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:54.823 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.823 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.823 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.823 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:54.823 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.823 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.823 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.823 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:18:54.823 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:54.823 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.823 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.824 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.824 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:18:54.824 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:54.824 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.824 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.824 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:54.824 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.824 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.824 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.824 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:18:54.824 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:54.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.825 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.825 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.825 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:18:54.825 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:54.825 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.825 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.825 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.825 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:18:54.825 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:18:54.825 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:18:54.825 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:18:54.825 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:54.825 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:54.825 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:54.826 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.826 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.826 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:18:54.826 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:54.826 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:54.826 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:54.826 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:54.826 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:54.826 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:54.826 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:54.826 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:54.827 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:54.827 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:54.827 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:54.827 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:54.827 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.827 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.827 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.827 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:18:54.827 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:54.827 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.827 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.827 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.827 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:18:54.827 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:54.828 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.828 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.828 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.828 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:54.828 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.828 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.828 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.828 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:18:54.828 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:54.828 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.828 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.828 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.828 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:18:54.829 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:54.829 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.829 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.829 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:54.829 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.829 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.829 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.829 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:18:54.829 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:54.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.829 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.829 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.829 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:18:54.829 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:54.830 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.830 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.830 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:54.830 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:54.830 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.830 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:54.830 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:18:54.830 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:54.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.830 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.830 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:54.831 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:18:54.831 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:54.831 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.831 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.831 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.831 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:18:54.831 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:18:54.831 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:18:54.831 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:18:54.831 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:54.831 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:54.831 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:54.831 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:54.831 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.832 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:54.832 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:18:54.832 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.832 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:18:54.832 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:54.832 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:54.832 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:54.832 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:54.832 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:54.832 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:18:54.832 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:54.832 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.832 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.832 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:18:54.832 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:54.833 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:54.833 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:54.833 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:54.833 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:54.833 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:18:54.833 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:54.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.833 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.833 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:18:54.833 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:54.833 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:54.833 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:54.833 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:54.833 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:54.834 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:18:54.834 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:54.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.834 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.834 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:18:54.834 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:54.834 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:54.834 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:54.834 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:54.834 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:54.834 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:18:54.834 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:54.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.835 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.835 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:18:54.835 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:54.835 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:54.835 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:54.835 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:54.835 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:54.835 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:18:54.835 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:54.835 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.835 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.835 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:18:54.835 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:54.835 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:54.836 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:54.836 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:54.836 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:54.836 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:18:54.836 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:54.836 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.836 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.836 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:18:54.836 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:54.836 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:54.836 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:54.836 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:54.836 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:54.836 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:18:54.837 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:54.837 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.837 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:54.837 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:18:54.837 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:54.837 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:54.837 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:54.837 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:54.837 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:54.837 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:18:54.837 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:54.837 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.837 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:54.837 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:18:54.838 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:18:54.838 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:18:54.838 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:18:54.838 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:18:54.838 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:18:54.838 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:18:54.838 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:18:54.838 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:18:54.838 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:18:54.838 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:18:54.838 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:18:54.838 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:18:54.838 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:18:54.839 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:54.839 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:18:54.839 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:18:54.839 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:18:54.839 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:18:54.839 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:18:54.839 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:18:54.839 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:18:54.839 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:18:54.839 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:18:54.839 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:18:54.839 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:18:54.839 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:54.840 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:18:54.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.763+0000 7fb11a000640 1 -- 172.21.15.161:0/1493519173 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0f0066690 msgr2=0x7fb0f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:54.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.763+0000 7fb11a000640 1 --2- 172.21.15.161:0/1493519173 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0f0066690 0x7fb0f0068b50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb1040011f0 tx=0x7fb104001160 comp rx=0 tx=0).stop 2024-05-22T21:18:54.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.763+0000 7fb11a000640 1 -- 172.21.15.161:0/1493519173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb114159200 msgr2=0x7fb1140c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:54.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.763+0000 7fb11a000640 1 --2- 172.21.15.161:0/1493519173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb114159200 0x7fb1140c91d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc01ca20 tx=0x7fb0f8005800 comp rx=0 tx=0).stop 2024-05-22T21:18:54.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.763+0000 7fb11a000640 1 -- 172.21.15.161:0/1493519173 shutdown_connections 2024-05-22T21:18:54.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.763+0000 7fb11a000640 1 --2- 172.21.15.161:0/1493519173 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0f0066690 0x7fb0f0068b50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.763+0000 7fb11a000640 1 --2- 172.21.15.161:0/1493519173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1140c9710 0x7fb1141541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.763+0000 7fb11a000640 1 --2- 172.21.15.161:0/1493519173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb114159200 0x7fb1140c91d0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.763+0000 7fb11a000640 1 --2- 172.21.15.161:0/1493519173 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0f80047d0 0x7fb114154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.763+0000 7fb11a000640 1 -- 172.21.15.161:0/1493519173 >> 172.21.15.161:0/1493519173 conn(0x7fb114099a40 msgr2=0x7fb1140cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:54.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.763+0000 7fb11a000640 1 -- 172.21.15.161:0/1493519173 shutdown_connections 2024-05-22T21:18:54.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.763+0000 7fb11a000640 1 -- 172.21.15.161:0/1493519173 wait complete. 2024-05-22T21:18:54.843 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-05-22T21:18:54.843 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:18:54.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.975+0000 7f6192400640 1 Processor -- start 2024-05-22T21:18:54.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.975+0000 7f6192400640 1 -- start start 2024-05-22T21:18:54.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.975+0000 7f6192400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f618c15dd50 0x7f618c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:54.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.975+0000 7f6192400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618c071670 con 0x7f618c15e130 2024-05-22T21:18:54.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.975+0000 7f6192400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618c06de40 con 0x7f618c15d5e0 2024-05-22T21:18:54.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.975+0000 7f6192400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c070a60 con 0x7f618c15dd50 2024-05-22T21:18:54.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.975+0000 7f6182a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f618c15e130 0x7f618c15ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44308/0 (socket says 172.21.15.161:44308) 2024-05-22T21:18:54.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.975+0000 7f6182a00640 1 -- 172.21.15.161:0/2844328451 learned_addr learned my addr 172.21.15.161:0/2844328451 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:54.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.975+0000 7f618b400640 1 --2- 172.21.15.161:0/2844328451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f618c15dd50 0x7f618c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:54.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.975+0000 7f618aa00640 1 -- 172.21.15.161:0/2844328451 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945968613 0 0) 0x7f618c071670 con 0x7f618c15e130 2024-05-22T21:18:54.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.975+0000 7f618aa00640 1 -- 172.21.15.161:0/2844328451 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6170005740 con 0x7f618c15e130 2024-05-22T21:18:54.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618aa00640 1 -- 172.21.15.161:0/2844328451 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1939019745 0 0) 0x7f6170005740 con 0x7f618c15e130 2024-05-22T21:18:54.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618aa00640 1 -- 172.21.15.161:0/2844328451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f618c15dd50 msgr2=0x7f618c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:54.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618aa00640 1 --2- 172.21.15.161:0/2844328451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f618c15dd50 0x7f618c06d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618aa00640 1 -- 172.21.15.161:0/2844328451 >> v1:172.21.15.179:6789/0 conn(0x7f618c15d5e0 legacy=0x7f618c15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:54.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618aa00640 1 -- 172.21.15.161:0/2844328451 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c070d40 con 0x7f618c15e130 2024-05-22T21:18:54.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618aa00640 1 -- 172.21.15.161:0/2844328451 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f6174001a00 con 0x7f618c15e130 2024-05-22T21:18:54.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618aa00640 1 -- 172.21.15.161:0/2844328451 >> v1:172.21.15.161:6789/0 conn(0x7f618c15e130 legacy=0x7f618c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:54.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618aa00640 1 --2- 172.21.15.161:0/2844328451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6170005bc0 0x7f6170005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:54.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618aa00640 1 --2- 172.21.15.161:0/2844328451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f618c15dd50 0x7f61700069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:54.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618aa00640 1 --2- 172.21.15.161:0/2844328451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6170006f20 0x7f61700073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:54.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618aa00640 1 -- 172.21.15.161:0/2844328451 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c070a60 con 0x7f6170006f20 2024-05-22T21:18:54.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618aa00640 1 -- 172.21.15.161:0/2844328451 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617000dcd0 con 0x7f6170005bc0 2024-05-22T21:18:54.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618aa00640 1 -- 172.21.15.161:0/2844328451 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617000de90 con 0x7f618c15dd50 2024-05-22T21:18:54.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618be00640 1 --2- 172.21.15.161:0/2844328451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6170006f20 0x7f61700073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:54.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618b400640 1 --2- 172.21.15.161:0/2844328451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6170005bc0 0x7f6170005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:54.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f6182a00640 1 --2- 172.21.15.161:0/2844328451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f618c15dd50 0x7f61700069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:54.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618be00640 1 -- 172.21.15.161:0/2844328451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f618c15dd50 msgr2=0x7f61700069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:54.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618be00640 1 --2- 172.21.15.161:0/2844328451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f618c15dd50 0x7f61700069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618be00640 1 -- 172.21.15.161:0/2844328451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6170005bc0 msgr2=0x7f6170005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:54.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618be00640 1 --2- 172.21.15.161:0/2844328451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6170005bc0 0x7f6170005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618be00640 1 -- 172.21.15.161:0/2844328451 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6170005740 con 0x7f6170006f20 2024-05-22T21:18:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.979+0000 7f618be00640 1 --2- 172.21.15.161:0/2844328451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6170006f20 0x7f61700073a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6184001750 tx=0x7f618401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791e9a63abdb9164 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f618aa00640 1 -- 172.21.15.161:0/2844328451 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840a4ea0 con 0x7f6170006f20 2024-05-22T21:18:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f618aa00640 1 -- 172.21.15.161:0/2844328451 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61840a7050 con 0x7f6170006f20 2024-05-22T21:18:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f618aa00640 1 -- 172.21.15.161:0/2844328451 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6184003600 con 0x7f6170006f20 2024-05-22T21:18:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f6192400640 1 -- 172.21.15.161:0/2844328451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6170006f20 msgr2=0x7f61700073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f6192400640 1 --2- 172.21.15.161:0/2844328451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6170006f20 0x7f61700073a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6184001750 tx=0x7f618401c3e0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f618be00640 1 -- 172.21.15.161:0/2844328451 reap_dead start 2024-05-22T21:18:54.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f6192400640 1 -- 172.21.15.161:0/2844328451 shutdown_connections 2024-05-22T21:18:54.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f6192400640 1 -- 172.21.15.161:0/2844328451 >> 172.21.15.161:0/2844328451 conn(0x7f618c0b0bc0 msgr2=0x7f618c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:54.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f6192400640 1 -- 172.21.15.161:0/2844328451 shutdown_connections 2024-05-22T21:18:54.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f6192400640 1 -- 172.21.15.161:0/2844328451 wait complete. 2024-05-22T21:18:54.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f6192400640 1 Processor -- start 2024-05-22T21:18:54.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f6192400640 1 -- start start 2024-05-22T21:18:54.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f6192400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f618c0cd610 0x7f618c155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:54.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f6192400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f618c0cd9f0 0x7f618c0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:54.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f6192400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f618c1460c0 0x7f618c146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:54.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f6192400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c155050 con 0x7f618c0cd610 2024-05-22T21:18:54.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f6192400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c14ff70 con 0x7f618c0cd9f0 2024-05-22T21:18:54.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f6192400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c1543d0 con 0x7f618c1460c0 2024-05-22T21:18:54.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f618b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f618c0cd610 0x7f618c155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:54.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f618b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f618c0cd610 0x7f618c155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59954/0 (socket says 172.21.15.161:59954) 2024-05-22T21:18:54.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f618b400640 1 -- 172.21.15.161:0/193921118 learned_addr learned my addr 172.21.15.161:0/193921118 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:54.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f618be00640 1 --2- 172.21.15.161:0/193921118 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f618c1460c0 0x7f618c146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:54.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f6182a00640 1 --2- 172.21.15.161:0/193921118 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f618c0cd9f0 0x7f618c0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:54.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f618b400640 1 -- 172.21.15.161:0/193921118 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f618c1460c0 msgr2=0x7f618c146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:54.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f618b400640 1 --2- 172.21.15.161:0/193921118 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f618c1460c0 0x7f618c146530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f618b400640 1 -- 172.21.15.161:0/193921118 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f618c0cd9f0 msgr2=0x7f618c0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:54.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f618b400640 1 --2- 172.21.15.161:0/193921118 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f618c0cd9f0 0x7f618c0cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:54.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f618b400640 1 -- 172.21.15.161:0/193921118 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c1546b0 con 0x7f618c0cd610 2024-05-22T21:18:54.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.983+0000 7f618be00640 1 --2- 172.21.15.161:0/193921118 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f618c1460c0 0x7f618c146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:54.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.987+0000 7f6182a00640 1 --2- 172.21.15.161:0/193921118 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f618c0cd9f0 0x7f618c0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:54.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.987+0000 7f618b400640 1 --2- 172.21.15.161:0/193921118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f618c0cd610 0x7f618c155450 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f617c0119d0 tx=0x7f617c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:54.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.987+0000 7f6183e00640 1 -- 172.21.15.161:0/193921118 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c00faf0 con 0x7f618c0cd610 2024-05-22T21:18:54.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.987+0000 7f6183e00640 1 -- 172.21.15.161:0/193921118 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c00fc50 con 0x7f618c0cd610 2024-05-22T21:18:54.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.987+0000 7f6183e00640 1 -- 172.21.15.161:0/193921118 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c014440 con 0x7f618c0cd610 2024-05-22T21:18:54.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.987+0000 7f6192400640 1 -- 172.21.15.161:0/193921118 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618c154990 con 0x7f618c0cd610 2024-05-22T21:18:54.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.987+0000 7f6192400640 1 -- 172.21.15.161:0/193921118 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618c155990 con 0x7f618c0cd610 2024-05-22T21:18:54.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.987+0000 7f6192400640 1 -- 172.21.15.161:0/193921118 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618c067220 con 0x7f618c0cd610 2024-05-22T21:18:54.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.987+0000 7f6183e00640 1 -- 172.21.15.161:0/193921118 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f617c0145a0 con 0x7f618c0cd610 2024-05-22T21:18:54.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.987+0000 7f6183e00640 1 --2- 172.21.15.161:0/193921118 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6168066690 0x7f6168068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:54.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.987+0000 7f6183e00640 1 -- 172.21.15.161:0/193921118 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f617c0849c0 con 0x7f618c0cd610 2024-05-22T21:18:54.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.987+0000 7f6182a00640 1 --2- 172.21.15.161:0/193921118 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6168066690 0x7f6168068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:54.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.991+0000 7f6182a00640 1 --2- 172.21.15.161:0/193921118 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6168066690 0x7f6168068b50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6174004bd0 tx=0x7f6174001360 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:54.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:54.991+0000 7f6183e00640 1 -- 172.21.15.161:0/193921118 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f617c051e80 con 0x7f618c0cd610 2024-05-22T21:18:55.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.147+0000 7f6192400640 1 -- 172.21.15.161:0/193921118 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f618c15e860 con 0x7f618c0cd610 2024-05-22T21:18:55.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.147+0000 7f6183e00640 1 -- 172.21.15.161:0/193921118 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v113) v1 ==== 81+0+33587 (secure 0 0 0) 0x7f617c0554a0 con 0x7f618c0cd610 2024-05-22T21:18:55.151 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:55.151 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:18:55.151 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 113, 2024-05-22T21:18:55.151 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:18:55.151 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:18:55.151 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:18:50.632514+0000", 2024-05-22T21:18:55.151 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:18:55.151 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:18:55.152 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:18:55.152 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:18:55.152 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:18:55.152 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:18:55.152 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:18:55.152 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:18:55.152 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:18:55.152 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:55.152 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:18:55.152 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:18:55.152 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:18:55.152 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:18:55.152 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:18:55.153 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 13, 2024-05-22T21:18:55.153 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:18:55.153 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:18:55.153 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:18:55.153 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:18:55.153 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:18:55.153 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:18:55.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.153 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:18:55.153 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:18:55.153 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:18:55.153 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:18:55.153 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:18:55.153 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:55.154 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:55.154 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:55.154 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:55.154 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:55.154 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:55.154 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:55.154 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:55.154 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:55.154 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:55.154 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:18:55.154 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:18:55.154 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:18:55.154 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:18:55.155 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:18:55.155 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:55.155 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:55.155 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:55.155 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:55.155 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:55.155 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:55.155 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:55.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.155 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:18:55.155 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:55.155 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:55.155 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:55.156 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:55.156 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:55.156 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:55.156 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:55.156 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:55.156 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:55.156 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:55.156 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:55.156 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:55.156 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:55.156 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:55.156 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:55.156 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:55.156 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:55.157 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:55.157 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:55.157 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:55.157 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:55.157 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:55.157 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:55.157 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:55.157 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:55.157 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:55.157 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.157 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:55.157 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:55.158 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:55.158 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:55.158 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:55.158 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:55.158 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:55.158 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:55.158 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:55.158 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:55.158 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:55.158 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:55.158 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:18:55.158 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:18:55.158 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.159 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:55.159 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:18:55.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.159 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:55.159 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:18:55.159 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:18:55.159 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:18:55.159 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:18:55.159 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:18:55.159 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:55.159 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:55.159 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:55.159 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.159 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.159 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.160 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 12, 2024-05-22T21:18:55.160 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:18:55.160 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:18:16.500534+0000", 2024-05-22T21:18:55.160 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:18:55.160 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:18:55.160 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:55.160 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:55.160 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:55.160 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:55.160 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:55.160 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:55.160 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:55.161 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:55.161 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:55.161 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:55.161 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:18:55.161 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:18:55.161 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:18:55.161 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:18:55.161 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:18:55.161 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:55.161 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:55.161 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:55.161 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:55.161 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:55.161 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:55.162 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:55.162 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.162 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:18:55.162 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:55.162 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:55.162 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:55.162 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:55.162 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:55.162 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:55.162 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:55.162 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:55.162 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:55.162 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:55.163 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:55.163 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:55.163 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:55.163 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:55.163 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:55.163 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:55.163 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:55.163 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:55.163 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:55.163 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:55.163 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:55.163 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:55.163 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:55.163 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:55.164 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:55.164 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:55.164 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.164 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:55.164 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:55.164 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:55.164 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:55.164 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:55.164 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:55.164 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:55.164 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:55.164 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:55.164 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:55.164 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:55.164 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:55.165 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:18:55.165 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:18:55.165 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:18:55.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.165 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:55.165 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:18:55.165 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:18:55.165 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.165 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.165 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:55.165 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 2, 2024-05-22T21:18:55.165 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 2, 2024-05-22T21:18:55.165 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:18:55.166 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:18:55.166 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:18:55.166 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:55.166 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:55.166 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:55.166 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.166 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.166 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.166 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 13, 2024-05-22T21:18:55.166 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:18:55.166 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:18:16.937445+0000", 2024-05-22T21:18:55.166 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:18:55.167 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:18:55.167 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:18:55.167 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:18:55.167 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:18:55.167 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:18:55.167 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:18:55.167 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:18:55.167 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:18:55.167 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:18:55.167 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:18:55.167 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:18:55.167 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:18:55.167 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:18:55.167 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:18:55.167 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:18:55.168 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:18:55.168 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:18:55.168 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:18:55.168 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:18:55.168 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:18:55.168 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:18:55.168 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:18:55.168 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:18:55.168 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.168 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:18:55.168 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:18:55.168 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:18:55.168 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:18:55.168 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:18:55.168 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:18:55.169 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:18:55.169 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:18:55.169 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:18:55.169 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:18:55.169 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:18:55.169 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:18:55.169 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:18:55.169 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:18:55.169 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:18:55.169 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:18:55.169 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:18:55.169 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:18:55.169 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:18:55.169 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:18:55.170 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:18:55.170 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:18:55.170 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:18:55.170 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:18:55.170 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:18:55.170 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:18:55.170 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:18:55.170 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.170 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:18:55.170 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:18:55.170 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:18:55.171 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:18:55.171 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:18:55.171 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:18:55.171 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:18:55.171 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:18:55.171 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:18:55.171 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:18:55.171 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:18:55.172 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:18:55.172 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:18:55.172 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:18:55.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.172 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:18:55.172 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:18:55.172 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:18:55.172 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.172 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:18:55.172 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.3799999952316284, 2024-05-22T21:18:55.172 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.3799999952316284, 2024-05-22T21:18:55.172 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:18:55.172 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.3799999952316284, 2024-05-22T21:18:55.173 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.3799999952316284, 2024-05-22T21:18:55.173 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:18:55.173 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:18:55.173 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:18:55.173 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.173 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.173 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:55.173 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:18:55.173 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.173 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:18:55.173 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:55.173 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:55.173 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:55.173 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:55.174 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:55.174 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:55.174 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:55.174 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:55.174 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:55.174 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:55.174 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:55.174 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:55.174 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.174 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.174 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.174 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:18:55.174 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:55.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.175 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.175 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:18:55.175 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:55.175 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.175 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.175 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:55.175 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.175 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.175 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:18:55.175 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:55.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.175 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.176 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.176 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:18:55.176 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:55.176 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.176 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.176 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:55.176 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.176 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.176 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:18:55.176 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:55.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.177 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.177 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:18:55.177 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:55.177 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.177 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.177 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:55.177 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.177 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.177 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:18:55.177 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:55.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.177 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.177 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.178 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:18:55.178 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:18:55.178 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.178 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.178 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.178 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:18:55.178 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:18:55.178 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:18:55.178 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:18:55.178 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:55.178 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:55.178 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:55.178 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.179 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.179 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:18:55.179 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:55.179 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:55.179 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:55.179 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:55.179 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:55.179 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:55.179 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:55.179 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:55.179 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:55.179 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:55.179 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:55.180 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:55.180 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.180 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.180 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:18:55.180 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:55.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.180 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.180 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.180 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:18:55.180 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:55.180 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.180 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.181 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:55.181 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.181 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.181 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:18:55.181 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:55.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.181 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.181 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.181 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:18:55.181 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:55.181 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.181 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.182 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:55.182 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.182 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.182 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:18:55.182 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:55.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.182 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.182 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.182 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:18:55.183 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:55.183 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.183 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.183 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:55.183 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.183 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.183 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:18:55.183 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:55.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.184 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.184 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.184 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:18:55.184 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:18:55.184 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.184 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.184 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.184 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:18:55.184 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:18:55.184 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:18:55.184 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:18:55.184 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:55.184 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:55.185 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:55.185 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.185 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.185 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:18:55.185 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:55.186 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:55.186 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:55.186 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:55.186 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:55.186 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:55.187 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:55.187 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:55.187 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:55.187 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:55.187 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:55.187 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:55.188 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.188 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.188 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:18:55.188 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:55.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.188 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.188 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.188 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:18:55.188 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:55.188 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.188 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.189 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:55.189 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.189 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.189 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.189 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:18:55.189 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:55.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.190 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.190 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:18:55.190 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:55.190 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.190 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.190 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:55.190 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.190 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.191 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:18:55.191 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:55.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.191 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.191 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:18:55.191 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:55.191 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.191 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.191 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:55.191 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.192 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.192 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:18:55.192 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:55.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.192 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.192 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:18:55.192 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:18:55.192 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.192 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.192 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:18:55.192 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:18:55.192 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:18:55.193 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:18:55.193 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:55.193 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:55.193 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:55.193 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.193 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:18:55.193 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:55.193 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:55.193 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:55.193 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:55.193 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:55.193 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:55.193 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:55.194 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:55.194 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:55.194 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:55.194 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:55.194 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:55.194 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.194 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.194 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:18:55.194 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:55.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.194 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.195 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:18:55.195 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:55.195 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.195 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.195 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:55.195 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.195 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.195 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:18:55.195 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:55.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.196 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.196 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:18:55.196 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:55.196 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.196 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.196 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:55.196 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.196 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.196 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:18:55.196 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:55.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.197 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.197 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:18:55.197 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:55.197 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.197 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.197 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:55.197 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.197 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.197 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:18:55.197 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:55.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.198 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.198 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:18:55.198 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:18:55.198 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.198 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.198 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:18:55.198 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:18:55.198 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:18:55.198 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:18:55.198 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:55.198 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:55.199 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:55.199 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.199 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:18:55.199 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:55.199 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:55.199 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:55.199 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:55.199 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:55.199 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:55.199 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:55.199 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:55.199 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:55.199 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:55.200 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:55.200 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:55.200 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.200 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.200 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:18:55.200 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:55.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.200 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.200 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:18:55.200 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:55.200 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.200 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.201 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:55.201 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.201 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.201 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:18:55.201 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:55.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.201 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.201 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:18:55.201 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:55.201 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.201 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.202 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:55.202 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.202 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.202 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:18:55.202 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:55.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.202 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.202 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:18:55.202 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:55.202 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.203 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.203 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:55.203 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.203 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.203 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:18:55.205 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:55.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.205 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.205 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:18:55.205 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:18:55.205 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.205 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.206 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:18:55.206 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:18:55.206 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:18:55.206 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:18:55.206 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:55.206 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:55.206 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:55.206 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.206 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:18:55.207 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:55.207 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:55.207 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:55.207 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:55.207 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:55.207 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:55.207 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:55.207 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:55.207 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:55.207 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:55.207 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:55.207 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:55.208 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.208 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.208 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:18:55.208 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:55.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.208 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.208 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:18:55.208 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:55.208 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.208 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.208 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:55.208 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.209 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.209 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:18:55.209 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:55.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.209 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.209 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:18:55.209 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:55.209 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.209 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.209 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:55.209 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.209 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.210 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.210 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:18:55.210 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:55.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.210 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.210 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:18:55.210 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:55.210 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.210 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.210 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:55.210 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.211 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.211 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:18:55.211 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:55.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.211 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.211 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:18:55.211 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:18:55.211 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.211 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.211 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:18:55.211 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:18:55.212 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:18:55.212 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:18:55.212 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:55.212 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:55.212 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:55.212 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.212 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:18:55.212 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:55.212 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:55.212 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:55.212 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:55.212 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:55.212 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:55.213 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:55.213 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:55.213 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:55.213 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:55.213 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:55.213 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:55.213 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.213 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.213 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:18:55.214 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:55.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.214 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.214 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:18:55.214 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:55.214 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.214 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.214 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:55.214 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.215 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.215 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:18:55.215 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:55.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.215 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.215 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:18:55.215 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:55.215 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.215 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.215 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:55.215 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.215 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.216 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:18:55.216 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:55.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.216 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.216 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:18:55.216 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:55.216 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.216 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.216 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:55.216 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.216 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.217 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:18:55.217 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:55.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.217 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.217 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:18:55.217 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:18:55.217 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.217 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.217 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:18:55.217 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:18:55.217 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:18:55.217 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:18:55.218 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:55.218 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:55.218 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:55.218 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.218 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:18:55.218 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:18:55.218 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:18:55.218 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:18:55.218 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:18:55.218 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:18:55.218 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:18:55.218 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:18:55.219 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:18:55.219 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 101, 2024-05-22T21:18:55.219 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:18:55.219 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:18:55.219 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:18:55.219 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.219 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.219 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:18:55.219 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:55.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.219 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.219 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:18:55.219 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:55.220 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.220 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.220 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:18:55.220 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.220 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.220 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:18:55.220 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:55.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.220 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.220 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:18:55.220 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:55.220 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.221 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.221 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:18:55.221 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.221 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.221 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:18:55.221 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:55.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.221 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.221 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:18:55.221 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:55.221 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.222 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.222 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:18:55.222 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:18:55.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.222 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:18:55.222 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:18:55.222 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:55.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.222 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:18:55.223 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:18:55.223 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:18:55.223 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.223 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.223 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:18:55.223 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:18:55.223 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:18:55.223 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:18:55.223 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:18:55.223 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:18:55.223 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:18:55.223 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:18:55.223 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.223 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:55.224 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:18:55.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.224 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:18:55.224 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:55.224 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:55.224 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:55.224 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:55.224 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:55.224 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:18:55.224 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:55.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.225 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:18:55.225 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:55.225 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:55.225 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:55.225 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:55.225 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:55.225 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:18:55.225 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:55.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.225 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:18:55.225 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:55.225 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:55.226 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:55.226 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:55.226 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:55.226 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:18:55.226 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:55.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.226 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.226 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:18:55.226 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:55.226 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:55.226 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:55.226 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:55.226 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:55.227 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:18:55.227 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:55.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.227 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:18:55.227 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:55.227 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:55.227 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:55.227 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:55.227 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:55.227 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:18:55.227 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:55.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.228 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:18:55.228 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:55.228 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:55.228 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:55.228 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:55.228 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:55.228 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:18:55.228 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:55.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.229 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:18:55.229 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:55.229 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:55.229 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:55.229 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:55.229 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:55.229 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:18:55.229 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:55.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:18:55.229 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:18:55.229 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:18:55.229 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:18:55.229 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:18:55.230 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:18:55.230 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:18:55.230 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:18:55.230 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:18:55.230 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.230 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:18:55.230 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:18:55.230 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:18:55.230 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:18:55.230 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:18:55.230 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:18:55.230 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:18:55.230 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:18:55.231 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:18:55.231 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:18:55.231 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:18:55.231 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:18:55.231 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:18:55.231 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:18:55.231 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:18:55.231 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:18:55.231 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:18:55.231 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:18:55.231 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:18:55.231 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:18:55.231 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:18:55.232 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:18:55.232 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:18:55.232 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:18:55.232 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:18:55.232 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:18:55.232 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:18:55.232 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:18:55.232 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:18:55.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.151+0000 7f6192400640 1 -- 172.21.15.161:0/193921118 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6168066690 msgr2=0x7f6168068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:55.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.151+0000 7f6192400640 1 --2- 172.21.15.161:0/193921118 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6168066690 0x7f6168068b50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6174004bd0 tx=0x7f6174001360 comp rx=0 tx=0).stop 2024-05-22T21:18:55.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.151+0000 7f6192400640 1 -- 172.21.15.161:0/193921118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f618c0cd610 msgr2=0x7f618c155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:55.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.151+0000 7f6192400640 1 --2- 172.21.15.161:0/193921118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f618c0cd610 0x7f618c155450 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f617c0119d0 tx=0x7f617c011e90 comp rx=0 tx=0).stop 2024-05-22T21:18:55.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.151+0000 7f6192400640 1 -- 172.21.15.161:0/193921118 shutdown_connections 2024-05-22T21:18:55.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.151+0000 7f6192400640 1 --2- 172.21.15.161:0/193921118 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6168066690 0x7f6168068b50 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:55.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.151+0000 7f6192400640 1 --2- 172.21.15.161:0/193921118 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f618c1460c0 0x7f618c146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:55.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.151+0000 7f6192400640 1 --2- 172.21.15.161:0/193921118 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f618c0cd9f0 0x7f618c0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:55.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.151+0000 7f6192400640 1 --2- 172.21.15.161:0/193921118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f618c0cd610 0x7f618c155450 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:55.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.151+0000 7f6192400640 1 -- 172.21.15.161:0/193921118 >> 172.21.15.161:0/193921118 conn(0x7f618c0b0bc0 msgr2=0x7f618c1593a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:55.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.151+0000 7f6192400640 1 -- 172.21.15.161:0/193921118 shutdown_connections 2024-05-22T21:18:55.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.151+0000 7f6192400640 1 -- 172.21.15.161:0/193921118 wait complete. 2024-05-22T21:18:55.235 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:18:55.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.367+0000 7fd148c00640 1 Processor -- start 2024-05-22T21:18:55.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.367+0000 7fd148c00640 1 -- start start 2024-05-22T21:18:55.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.367+0000 7fd148c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd14415d430 0x7fd1440707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:55.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.367+0000 7fd148c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd144071fd0 con 0x7fd1440ac8b0 2024-05-22T21:18:55.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.367+0000 7fd148c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd144070e80 con 0x7fd1440accf0 2024-05-22T21:18:55.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.367+0000 7fd148c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14406c510 con 0x7fd14415d430 2024-05-22T21:18:55.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.367+0000 7fd139400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd1440ac8b0 0x7fd14406cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44312/0 (socket says 172.21.15.161:44312) 2024-05-22T21:18:55.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.367+0000 7fd139400640 1 -- 172.21.15.161:0/934631543 learned_addr learned my addr 172.21.15.161:0/934631543 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:55.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.367+0000 7fd141e00640 1 --2- 172.21.15.161:0/934631543 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd14415d430 0x7fd1440707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:55.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.371+0000 7fd141400640 1 -- 172.21.15.161:0/934631543 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074142719 0 0) 0x7fd144071fd0 con 0x7fd1440ac8b0 2024-05-22T21:18:55.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.371+0000 7fd141400640 1 -- 172.21.15.161:0/934631543 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd128003530 con 0x7fd1440ac8b0 2024-05-22T21:18:55.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.371+0000 7fd141e00640 1 -- 172.21.15.161:0/934631543 >> v1:172.21.15.179:6789/0 conn(0x7fd1440accf0 legacy=0x7fd144160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:55.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.371+0000 7fd141400640 1 -- 172.21.15.161:0/934631543 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584803369 0 0) 0x7fd144070e80 con 0x7fd1440accf0 2024-05-22T21:18:55.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.371+0000 7fd141e00640 1 -- 172.21.15.161:0/934631543 >> v1:172.21.15.161:6789/0 conn(0x7fd1440ac8b0 legacy=0x7fd14406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:55.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.371+0000 7fd141e00640 1 -- 172.21.15.161:0/934631543 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14406c7f0 con 0x7fd14415d430 2024-05-22T21:18:55.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.371+0000 7fd141e00640 1 --2- 172.21.15.161:0/934631543 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd14415d430 0x7fd1440707c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd134009dc0 tx=0x7fd13401c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac9ccf5ea3740b10 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:55.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.371+0000 7fd141400640 1 -- 172.21.15.161:0/934631543 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1340a58a0 con 0x7fd14415d430 2024-05-22T21:18:55.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.371+0000 7fd141400640 1 -- 172.21.15.161:0/934631543 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1340a5a00 con 0x7fd14415d430 2024-05-22T21:18:55.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.371+0000 7fd141400640 1 -- 172.21.15.161:0/934631543 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1340a5d70 con 0x7fd14415d430 2024-05-22T21:18:55.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd148c00640 1 -- 172.21.15.161:0/934631543 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd14415d430 msgr2=0x7fd1440707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:55.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd148c00640 1 --2- 172.21.15.161:0/934631543 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd14415d430 0x7fd1440707c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd134009dc0 tx=0x7fd13401c8d0 comp rx=0 tx=0).stop 2024-05-22T21:18:55.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd148c00640 1 -- 172.21.15.161:0/934631543 shutdown_connections 2024-05-22T21:18:55.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd148c00640 1 --2- 172.21.15.161:0/934631543 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd14415d430 0x7fd1440707c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:55.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd148c00640 1 -- 172.21.15.161:0/934631543 >> 172.21.15.161:0/934631543 conn(0x7fd1440b0bc0 msgr2=0x7fd1440b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:55.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd148c00640 1 -- 172.21.15.161:0/934631543 shutdown_connections 2024-05-22T21:18:55.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd148c00640 1 -- 172.21.15.161:0/934631543 wait complete. 2024-05-22T21:18:55.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd148c00640 1 Processor -- start 2024-05-22T21:18:55.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd148c00640 1 -- start start 2024-05-22T21:18:55.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd148c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1440ac8b0 0x7fd14415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:55.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd148c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1440accf0 0x7fd144151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:55.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd148c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd14415d430 0x7fd1441462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:55.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd148c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd144150050 con 0x7fd1440ac8b0 2024-05-22T21:18:55.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd148c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14415f220 con 0x7fd1440accf0 2024-05-22T21:18:55.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd148c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14414bab0 con 0x7fd14415d430 2024-05-22T21:18:55.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd141e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1440ac8b0 0x7fd14415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:55.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd141e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1440ac8b0 0x7fd14415ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59958/0 (socket says 172.21.15.161:59958) 2024-05-22T21:18:55.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd141e00640 1 -- 172.21.15.161:0/1917528515 learned_addr learned my addr 172.21.15.161:0/1917528515 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:55.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd141e00640 1 -- 172.21.15.161:0/1917528515 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd14415d430 msgr2=0x7fd1441462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:55.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd141e00640 1 --2- 172.21.15.161:0/1917528515 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd14415d430 0x7fd1441462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:55.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd141e00640 1 -- 172.21.15.161:0/1917528515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1440accf0 msgr2=0x7fd144151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:55.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd139400640 1 --2- 172.21.15.161:0/1917528515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1440accf0 0x7fd144151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:55.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd141e00640 1 --2- 172.21.15.161:0/1917528515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1440accf0 0x7fd144151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:55.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd141e00640 1 -- 172.21.15.161:0/1917528515 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd134009a70 con 0x7fd1440ac8b0 2024-05-22T21:18:55.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd139400640 1 --2- 172.21.15.161:0/1917528515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1440accf0 0x7fd144151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:55.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.375+0000 7fd141e00640 1 --2- 172.21.15.161:0/1917528515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1440ac8b0 0x7fd14415ece0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd1340194e0 tx=0x7fd134019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:55.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.379+0000 7fd13aa00640 1 -- 172.21.15.161:0/1917528515 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1340a5820 con 0x7fd1440ac8b0 2024-05-22T21:18:55.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.379+0000 7fd13aa00640 1 -- 172.21.15.161:0/1917528515 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1340ad650 con 0x7fd1440ac8b0 2024-05-22T21:18:55.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.379+0000 7fd13aa00640 1 -- 172.21.15.161:0/1917528515 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1340ada00 con 0x7fd1440ac8b0 2024-05-22T21:18:55.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.379+0000 7fd148c00640 1 -- 172.21.15.161:0/1917528515 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14414bd90 con 0x7fd1440ac8b0 2024-05-22T21:18:55.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.379+0000 7fd148c00640 1 -- 172.21.15.161:0/1917528515 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14414c350 con 0x7fd1440ac8b0 2024-05-22T21:18:55.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.379+0000 7fd148c00640 1 -- 172.21.15.161:0/1917528515 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd144150190 con 0x7fd1440ac8b0 2024-05-22T21:18:55.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.379+0000 7fd13aa00640 1 -- 172.21.15.161:0/1917528515 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd1340adb60 con 0x7fd1440ac8b0 2024-05-22T21:18:55.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.379+0000 7fd13aa00640 1 --2- 172.21.15.161:0/1917528515 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd120066690 0x7fd120068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:55.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.379+0000 7fd13aa00640 1 -- 172.21.15.161:0/1917528515 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd13411f260 con 0x7fd1440ac8b0 2024-05-22T21:18:55.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.379+0000 7fd139400640 1 --2- 172.21.15.161:0/1917528515 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd120066690 0x7fd120068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:55.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.379+0000 7fd139400640 1 --2- 172.21.15.161:0/1917528515 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd120066690 0x7fd120068b50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd1440b1e20 tx=0x7fd12c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:55.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.383+0000 7fd13aa00640 1 -- 172.21.15.161:0/1917528515 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd144150190 con 0x7fd1440ac8b0 2024-05-22T21:18:55.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.571+0000 7fd148c00640 1 -- 172.21.15.161:0/1917528515 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd144150190 con 0x7fd1440ac8b0 2024-05-22T21:18:55.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.571+0000 7fd13aa00640 1 -- 172.21.15.161:0/1917528515 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4748 (secure 0 0 0) 0x7fd1340ec5c0 con 0x7fd1440ac8b0 2024-05-22T21:18:55.573 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:55.574 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2771409351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2771409351},{"type":"v1","addr":"172.21.15.179:6834","nonce":2771409351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6442,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2520119431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2520119431},{"type":"v1","addr":"172.21.15.179:6835","nonce":2520119431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1415362547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1415362547},{"type":"v1","addr":"172.21.15.161:6835","nonce":1415362547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:18:17.938858+0000","modified":"2024-05-22T21:18:20.963065+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/4127130864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4127130864},{"type":"v1","addr":"172.21.15.161:6837","nonce":4127130864}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-22T21:18:55.574 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 47 2024-05-22T21:18:55.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.571+0000 7fd148c00640 1 -- 172.21.15.161:0/1917528515 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd120066690 msgr2=0x7fd120068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.571+0000 7fd148c00640 1 --2- 172.21.15.161:0/1917528515 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd120066690 0x7fd120068b50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd1440b1e20 tx=0x7fd12c009040 comp rx=0 tx=0).stop 2024-05-22T21:18:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.571+0000 7fd148c00640 1 -- 172.21.15.161:0/1917528515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1440ac8b0 msgr2=0x7fd14415ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.571+0000 7fd148c00640 1 --2- 172.21.15.161:0/1917528515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1440ac8b0 0x7fd14415ece0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd1340194e0 tx=0x7fd134019670 comp rx=0 tx=0).stop 2024-05-22T21:18:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.571+0000 7fd148c00640 1 -- 172.21.15.161:0/1917528515 shutdown_connections 2024-05-22T21:18:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.571+0000 7fd148c00640 1 --2- 172.21.15.161:0/1917528515 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd120066690 0x7fd120068b50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.571+0000 7fd148c00640 1 --2- 172.21.15.161:0/1917528515 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd14415d430 0x7fd1441462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.571+0000 7fd148c00640 1 --2- 172.21.15.161:0/1917528515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1440accf0 0x7fd144151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.571+0000 7fd148c00640 1 --2- 172.21.15.161:0/1917528515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1440ac8b0 0x7fd14415ece0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.571+0000 7fd148c00640 1 -- 172.21.15.161:0/1917528515 >> 172.21.15.161:0/1917528515 conn(0x7fd1440b0bc0 msgr2=0x7fd1440b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.571+0000 7fd148c00640 1 -- 172.21.15.161:0/1917528515 shutdown_connections 2024-05-22T21:18:55.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:55.571+0000 7fd148c00640 1 -- 172.21.15.161:0/1917528515 wait complete. 2024-05-22T21:18:55.595 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-05-22T21:18:55.751 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":6699,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":37.793993675000003,"mdsmap_epoch":47,"osdmap_epoch":113,"osdmap_epoch_barrier":107,"uptime":42.028285189999998} 2024-05-22T21:18:55.751 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 6699, "mdsmap_epoch": 47, "osdmap_epoch": 113, "osdmap_epoch_barrier": 107, "rank_uptime": 37.793993675, "state": "up:active", "uptime": 42.02828519, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:18:55.751 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-22T21:18:55.752 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:18:55.752 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:55.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:55.761 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-05-22T21:18:56.293 INFO:teuthology.orchestra.run.smithi161.stderr:50+0 records in 2024-05-22T21:18:56.293 INFO:teuthology.orchestra.run.smithi161.stderr:50+0 records out 2024-05-22T21:18:56.293 INFO:teuthology.orchestra.run.smithi161.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.475816 s, 110 MB/s 2024-05-22T21:18:56.294 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:18:56.294 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:18:56.347 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:18:56.420 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:18:56.420 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:18:56.420 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:56.431 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:18:56.431 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:56.484 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:18:56.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.663+0000 7fdc27600640 1 Processor -- start 2024-05-22T21:18:56.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.663+0000 7fdc27600640 1 -- start start 2024-05-22T21:18:56.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.663+0000 7fdc27600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdc20143ec0 0x7fdc2015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:56.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.663+0000 7fdc27600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc2015ec90 0x7fdc2015f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:56.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.663+0000 7fdc27600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc201442a0 con 0x7fdc200715a0 2024-05-22T21:18:56.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.663+0000 7fdc27600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2015f680 con 0x7fdc20143ec0 2024-05-22T21:18:56.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.663+0000 7fdc27600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc20070500 con 0x7fdc2015ec90 2024-05-22T21:18:56.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.663+0000 7fdc24c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdc20143ec0 0x7fdc2015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:56.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.663+0000 7fdc24c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdc20143ec0 0x7fdc2015f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51748/0 (socket says 172.21.15.161:51748) 2024-05-22T21:18:56.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.663+0000 7fdc24c00640 1 -- 172.21.15.161:0/2494066644 learned_addr learned my addr 172.21.15.161:0/2494066644 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:56.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.663+0000 7fdc25600640 1 --2- 172.21.15.161:0/2494066644 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc2015ec90 0x7fdc2015f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:56.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.663+0000 7fdc24c00640 1 -- 172.21.15.161:0/2494066644 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc2015ec90 msgr2=0x7fdc2015f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:56.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.663+0000 7fdc24c00640 1 --2- 172.21.15.161:0/2494066644 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc2015ec90 0x7fdc2015f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:56.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.663+0000 7fdc24c00640 1 -- 172.21.15.161:0/2494066644 >> v1:172.21.15.179:6789/0 conn(0x7fdc200715a0 legacy=0x7fdc2006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:56.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.663+0000 7fdc24c00640 1 -- 172.21.15.161:0/2494066644 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc200707e0 con 0x7fdc20143ec0 2024-05-22T21:18:56.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.663+0000 7fdc1f400640 1 -- 172.21.15.161:0/2494066644 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2015337807 0 0) 0x7fdc201442a0 con 0x7fdc200715a0 2024-05-22T21:18:56.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.667+0000 7fdc24c00640 1 --2- 172.21.15.161:0/2494066644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc20143ec0 0x7fdc2015f830 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009870 tx=0x7fdc1001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b696163994e19be server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:56.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.667+0000 7fdc1f400640 1 -- 172.21.15.161:0/2494066644 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc100a7070 con 0x7fdc20143ec0 2024-05-22T21:18:56.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.667+0000 7fdc1f400640 1 -- 172.21.15.161:0/2494066644 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc100ab430 con 0x7fdc20143ec0 2024-05-22T21:18:56.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.667+0000 7fdc1f400640 1 -- 172.21.15.161:0/2494066644 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc100ab7c0 con 0x7fdc20143ec0 2024-05-22T21:18:56.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.667+0000 7fdc27600640 1 -- 172.21.15.161:0/2494066644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc20143ec0 msgr2=0x7fdc2015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:56.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.667+0000 7fdc27600640 1 --2- 172.21.15.161:0/2494066644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc20143ec0 0x7fdc2015f830 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009870 tx=0x7fdc1001c530 comp rx=0 tx=0).stop 2024-05-22T21:18:56.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.667+0000 7fdc27600640 1 -- 172.21.15.161:0/2494066644 shutdown_connections 2024-05-22T21:18:56.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.667+0000 7fdc27600640 1 --2- 172.21.15.161:0/2494066644 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc2015ec90 0x7fdc2015f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:56.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.667+0000 7fdc27600640 1 --2- 172.21.15.161:0/2494066644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc20143ec0 0x7fdc2015f830 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:56.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.667+0000 7fdc27600640 1 -- 172.21.15.161:0/2494066644 >> 172.21.15.161:0/2494066644 conn(0x7fdc200b0bc0 msgr2=0x7fdc200b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:56.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.667+0000 7fdc27600640 1 -- 172.21.15.161:0/2494066644 shutdown_connections 2024-05-22T21:18:56.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.667+0000 7fdc27600640 1 -- 172.21.15.161:0/2494066644 wait complete. 2024-05-22T21:18:56.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc27600640 1 Processor -- start 2024-05-22T21:18:56.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc27600640 1 -- start start 2024-05-22T21:18:56.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc27600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc2015ec90 0x7fdc200cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:56.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc27600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc201601f0 0x7fdc201f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:56.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc27600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc20160f00 0x7fdc201613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:56.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc27600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc20160c50 con 0x7fdc20160f00 2024-05-22T21:18:56.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc27600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc201618f0 con 0x7fdc201601f0 2024-05-22T21:18:56.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc27600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2015cb20 con 0x7fdc2015ec90 2024-05-22T21:18:56.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc25600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc20160f00 0x7fdc201613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:56.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc25600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc20160f00 0x7fdc201613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51754/0 (socket says 172.21.15.161:51754) 2024-05-22T21:18:56.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc25600640 1 -- 172.21.15.161:0/3300748346 learned_addr learned my addr 172.21.15.161:0/3300748346 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:56.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc24c00640 1 --2- 172.21.15.161:0/3300748346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc2015ec90 0x7fdc200cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:56.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc1fe00640 1 --2- 172.21.15.161:0/3300748346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc201601f0 0x7fdc201f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:56.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc25600640 1 -- 172.21.15.161:0/3300748346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc2015ec90 msgr2=0x7fdc200cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:56.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc25600640 1 --2- 172.21.15.161:0/3300748346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc2015ec90 0x7fdc200cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:56.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc25600640 1 -- 172.21.15.161:0/3300748346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc201601f0 msgr2=0x7fdc201f5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:56.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc25600640 1 --2- 172.21.15.161:0/3300748346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc201601f0 0x7fdc201f5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:56.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc25600640 1 -- 172.21.15.161:0/3300748346 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc100094d0 con 0x7fdc20160f00 2024-05-22T21:18:56.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc25600640 1 --2- 172.21.15.161:0/3300748346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc20160f00 0x7fdc201613b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800efc0 tx=0x7fdc1800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:56.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc1cc00640 1 -- 172.21.15.161:0/3300748346 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18009ba0 con 0x7fdc20160f00 2024-05-22T21:18:56.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc1cc00640 1 -- 172.21.15.161:0/3300748346 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1800f040 con 0x7fdc20160f00 2024-05-22T21:18:56.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc1cc00640 1 -- 172.21.15.161:0/3300748346 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc180146c0 con 0x7fdc20160f00 2024-05-22T21:18:56.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc27600640 1 -- 172.21.15.161:0/3300748346 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc201f59c0 con 0x7fdc20160f00 2024-05-22T21:18:56.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc27600640 1 -- 172.21.15.161:0/3300748346 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc20160970 con 0x7fdc20160f00 2024-05-22T21:18:56.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.671+0000 7fdc27600640 1 -- 172.21.15.161:0/3300748346 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2015cc60 con 0x7fdc20160f00 2024-05-22T21:18:56.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.675+0000 7fdc1cc00640 1 -- 172.21.15.161:0/3300748346 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdc18014820 con 0x7fdc20160f00 2024-05-22T21:18:56.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.675+0000 7fdc1cc00640 1 --2- 172.21.15.161:0/3300748346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc000666e0 0x7fdc00068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:56.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.675+0000 7fdc1cc00640 1 -- 172.21.15.161:0/3300748346 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc18084e00 con 0x7fdc20160f00 2024-05-22T21:18:56.678 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.675+0000 7fdc24c00640 1 --2- 172.21.15.161:0/3300748346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc000666e0 0x7fdc00068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:56.678 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.675+0000 7fdc24c00640 1 --2- 172.21.15.161:0/3300748346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc000666e0 0x7fdc00068ba0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdc100af000 tx=0x7fdc10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:56.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.679+0000 7fdc1cc00640 1 -- 172.21.15.161:0/3300748346 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdc18052200 con 0x7fdc20160f00 2024-05-22T21:18:56.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.831+0000 7fdc27600640 1 -- 172.21.15.161:0/3300748346 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc20067220 con 0x7fdc20160f00 2024-05-22T21:18:56.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.835+0000 7fdc1cc00640 1 -- 172.21.15.161:0/3300748346 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16152 (secure 0 0 0) 0x7fdc18055820 con 0x7fdc20160f00 2024-05-22T21:18:56.837 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:56.837 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":113,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:18:50.632514+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:18:16.500534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-22T21:18:16.937445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:18:56.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.835+0000 7fdc27600640 1 -- 172.21.15.161:0/3300748346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc000666e0 msgr2=0x7fdc00068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:56.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.835+0000 7fdc27600640 1 --2- 172.21.15.161:0/3300748346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc000666e0 0x7fdc00068ba0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdc100af000 tx=0x7fdc10008040 comp rx=0 tx=0).stop 2024-05-22T21:18:56.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.835+0000 7fdc27600640 1 -- 172.21.15.161:0/3300748346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc20160f00 msgr2=0x7fdc201613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:56.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.835+0000 7fdc27600640 1 --2- 172.21.15.161:0/3300748346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc20160f00 0x7fdc201613b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800efc0 tx=0x7fdc1800c490 comp rx=0 tx=0).stop 2024-05-22T21:18:56.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.835+0000 7fdc27600640 1 -- 172.21.15.161:0/3300748346 shutdown_connections 2024-05-22T21:18:56.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.835+0000 7fdc27600640 1 --2- 172.21.15.161:0/3300748346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdc000666e0 0x7fdc00068ba0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:56.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.835+0000 7fdc27600640 1 --2- 172.21.15.161:0/3300748346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc20160f00 0x7fdc201613b0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:56.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.835+0000 7fdc27600640 1 --2- 172.21.15.161:0/3300748346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc201601f0 0x7fdc201f5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:56.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.835+0000 7fdc27600640 1 --2- 172.21.15.161:0/3300748346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc2015ec90 0x7fdc200cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:56.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.835+0000 7fdc27600640 1 -- 172.21.15.161:0/3300748346 >> 172.21.15.161:0/3300748346 conn(0x7fdc200b0bc0 msgr2=0x7fdc2006bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:56.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.835+0000 7fdc27600640 1 -- 172.21.15.161:0/3300748346 shutdown_connections 2024-05-22T21:18:56.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.835+0000 7fdc27600640 1 -- 172.21.15.161:0/3300748346 wait complete. 2024-05-22T21:18:56.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:18:16.500534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:18:16.937445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:18:56.858 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:18:56.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.995+0000 7f7ddfa00640 1 Processor -- start 2024-05-22T21:18:56.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.995+0000 7f7ddfa00640 1 -- start start 2024-05-22T21:18:56.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.995+0000 7f7ddfa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7dd8143c40 0x7f7dd815b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:56.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.995+0000 7f7ddfa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7dd806c9b0 0x7f7dd80b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:56.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.995+0000 7f7ddfa00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd815efb0 con 0x7f7dd81444a0 2024-05-22T21:18:56.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.995+0000 7f7ddfa00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd815f290 con 0x7f7dd8143c40 2024-05-22T21:18:56.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.995+0000 7f7ddfa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd815f570 con 0x7f7dd806c9b0 2024-05-22T21:18:56.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.995+0000 7f7ddd000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7dd8143c40 0x7f7dd815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:56.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.995+0000 7f7ddd000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7dd8143c40 0x7f7dd815b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51768/0 (socket says 172.21.15.161:51768) 2024-05-22T21:18:56.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.995+0000 7f7ddd000640 1 -- 172.21.15.161:0/402102882 learned_addr learned my addr 172.21.15.161:0/402102882 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:56.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.995+0000 7f7ddda00640 1 --2- 172.21.15.161:0/402102882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7dd806c9b0 0x7f7dd80b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:56.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.995+0000 7f7ddd000640 1 -- 172.21.15.161:0/402102882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7dd806c9b0 msgr2=0x7f7dd80b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:56.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.995+0000 7f7ddd000640 1 --2- 172.21.15.161:0/402102882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7dd806c9b0 0x7f7dd80b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:56.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.995+0000 7f7ddd000640 1 -- 172.21.15.161:0/402102882 >> v1:172.21.15.179:6789/0 conn(0x7f7dd81444a0 legacy=0x7f7dd806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:56.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.995+0000 7f7ddd000640 1 -- 172.21.15.161:0/402102882 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd815f850 con 0x7f7dd8143c40 2024-05-22T21:18:57.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.995+0000 7f7ddda00640 1 --2- 172.21.15.161:0/402102882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7dd806c9b0 0x7f7dd80b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:57.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.999+0000 7f7ddd000640 1 --2- 172.21.15.161:0/402102882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd8143c40 0x7f7dd815b600 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009870 tx=0x7f7dcc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b060981969c3c5fb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:57.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.999+0000 7f7dd7400640 1 -- 172.21.15.161:0/402102882 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc0ab640 con 0x7f7dd8143c40 2024-05-22T21:18:57.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.999+0000 7f7dd7400640 1 -- 172.21.15.161:0/402102882 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc0a7070 con 0x7f7dd8143c40 2024-05-22T21:18:57.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.999+0000 7f7dd7400640 1 -- 172.21.15.161:0/402102882 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc0ab9c0 con 0x7f7dd8143c40 2024-05-22T21:18:57.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.999+0000 7f7ddfa00640 1 -- 172.21.15.161:0/402102882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd8143c40 msgr2=0x7f7dd815b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.999+0000 7f7ddfa00640 1 --2- 172.21.15.161:0/402102882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd8143c40 0x7f7dd815b600 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009870 tx=0x7f7dcc01c520 comp rx=0 tx=0).stop 2024-05-22T21:18:57.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.999+0000 7f7ddfa00640 1 -- 172.21.15.161:0/402102882 shutdown_connections 2024-05-22T21:18:57.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.999+0000 7f7ddfa00640 1 --2- 172.21.15.161:0/402102882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7dd806c9b0 0x7f7dd80b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.999+0000 7f7ddfa00640 1 --2- 172.21.15.161:0/402102882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd8143c40 0x7f7dd815b600 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.999+0000 7f7ddfa00640 1 -- 172.21.15.161:0/402102882 >> 172.21.15.161:0/402102882 conn(0x7f7dd806a2d0 msgr2=0x7f7dd806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:57.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.999+0000 7f7ddfa00640 1 -- 172.21.15.161:0/402102882 shutdown_connections 2024-05-22T21:18:57.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:56.999+0000 7f7ddfa00640 1 -- 172.21.15.161:0/402102882 wait complete. 2024-05-22T21:18:57.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddfa00640 1 Processor -- start 2024-05-22T21:18:57.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddfa00640 1 -- start start 2024-05-22T21:18:57.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddfa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7dd806c9b0 0x7f7dd815d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddfa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7dd8143c40 0x7f7dd80ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddfa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd81444a0 0x7f7dd80acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddfa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd8159cd0 con 0x7f7dd81444a0 2024-05-22T21:18:57.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddfa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd8157490 con 0x7f7dd8143c40 2024-05-22T21:18:57.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddfa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd815e0e0 con 0x7f7dd806c9b0 2024-05-22T21:18:57.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddda00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd81444a0 0x7f7dd80acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddda00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd81444a0 0x7f7dd80acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51784/0 (socket says 172.21.15.161:51784) 2024-05-22T21:18:57.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddda00640 1 -- 172.21.15.161:0/1903298102 learned_addr learned my addr 172.21.15.161:0/1903298102 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:57.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7dd7e00640 1 --2- 172.21.15.161:0/1903298102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7dd8143c40 0x7f7dd80ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddd000640 1 --2- 172.21.15.161:0/1903298102 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7dd806c9b0 0x7f7dd815d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddda00640 1 -- 172.21.15.161:0/1903298102 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7dd806c9b0 msgr2=0x7f7dd815d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddda00640 1 --2- 172.21.15.161:0/1903298102 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7dd806c9b0 0x7f7dd815d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddda00640 1 -- 172.21.15.161:0/1903298102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7dd8143c40 msgr2=0x7f7dd80ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddda00640 1 --2- 172.21.15.161:0/1903298102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7dd8143c40 0x7f7dd80ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddda00640 1 -- 172.21.15.161:0/1903298102 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc0094d0 con 0x7f7dd81444a0 2024-05-22T21:18:57.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddda00640 1 --2- 172.21.15.161:0/1903298102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd81444a0 0x7f7dd80acdf0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000cef0 tx=0x7f7dd000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:57.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7dd4c00640 1 -- 172.21.15.161:0/1903298102 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000d840 con 0x7f7dd81444a0 2024-05-22T21:18:57.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7dd4c00640 1 -- 172.21.15.161:0/1903298102 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd000d9c0 con 0x7f7dd81444a0 2024-05-22T21:18:57.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7dd4c00640 1 -- 172.21.15.161:0/1903298102 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000dd70 con 0x7f7dd81444a0 2024-05-22T21:18:57.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddfa00640 1 -- 172.21.15.161:0/1903298102 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd815e3c0 con 0x7f7dd81444a0 2024-05-22T21:18:57.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddfa00640 1 -- 172.21.15.161:0/1903298102 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd815ddd0 con 0x7f7dd81444a0 2024-05-22T21:18:57.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.003+0000 7f7ddfa00640 1 -- 172.21.15.161:0/1903298102 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd8159e40 con 0x7f7dd81444a0 2024-05-22T21:18:57.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.007+0000 7f7dd4c00640 1 -- 172.21.15.161:0/1903298102 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7dd001e3b0 con 0x7f7dd81444a0 2024-05-22T21:18:57.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.007+0000 7f7dd4c00640 1 --2- 172.21.15.161:0/1903298102 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7db8066690 0x7f7db8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.007+0000 7f7dd4c00640 1 -- 172.21.15.161:0/1903298102 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7dd0084cd0 con 0x7f7dd81444a0 2024-05-22T21:18:57.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.007+0000 7f7ddd000640 1 --2- 172.21.15.161:0/1903298102 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7db8066690 0x7f7db8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.007+0000 7f7ddd000640 1 --2- 172.21.15.161:0/1903298102 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7db8066690 0x7f7db8068b50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0af000 tx=0x7f7dcc0b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:57.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.011+0000 7f7dd4c00640 1 -- 172.21.15.161:0/1903298102 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7dd8159e40 con 0x7f7dd81444a0 2024-05-22T21:18:57.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.163+0000 7f7ddfa00640 1 -- 172.21.15.161:0/1903298102 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7dd8159e10 con 0x7f7dd81444a0 2024-05-22T21:18:57.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.163+0000 7f7dd4c00640 1 -- 172.21.15.161:0/1903298102 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7dd001a070 con 0x7f7dd81444a0 2024-05-22T21:18:57.167 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:18:57.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.167+0000 7f7ddfa00640 1 -- 172.21.15.161:0/1903298102 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7db8066690 msgr2=0x7f7db8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.167+0000 7f7ddfa00640 1 --2- 172.21.15.161:0/1903298102 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7db8066690 0x7f7db8068b50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0af000 tx=0x7f7dcc0b6040 comp rx=0 tx=0).stop 2024-05-22T21:18:57.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.167+0000 7f7ddfa00640 1 -- 172.21.15.161:0/1903298102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd81444a0 msgr2=0x7f7dd80acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.167+0000 7f7ddfa00640 1 --2- 172.21.15.161:0/1903298102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd81444a0 0x7f7dd80acdf0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000cef0 tx=0x7f7dd000a710 comp rx=0 tx=0).stop 2024-05-22T21:18:57.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.167+0000 7f7ddfa00640 1 -- 172.21.15.161:0/1903298102 shutdown_connections 2024-05-22T21:18:57.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.167+0000 7f7ddfa00640 1 --2- 172.21.15.161:0/1903298102 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7db8066690 0x7f7db8068b50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.167+0000 7f7ddfa00640 1 --2- 172.21.15.161:0/1903298102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7dd81444a0 0x7f7dd80acdf0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.167+0000 7f7ddfa00640 1 --2- 172.21.15.161:0/1903298102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7dd8143c40 0x7f7dd80ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.167+0000 7f7ddfa00640 1 --2- 172.21.15.161:0/1903298102 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7dd806c9b0 0x7f7dd815d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.167+0000 7f7ddfa00640 1 -- 172.21.15.161:0/1903298102 >> 172.21.15.161:0/1903298102 conn(0x7f7dd806a2d0 msgr2=0x7f7dd815bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:57.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.167+0000 7f7ddfa00640 1 -- 172.21.15.161:0/1903298102 shutdown_connections 2024-05-22T21:18:57.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.167+0000 7f7ddfa00640 1 -- 172.21.15.161:0/1903298102 wait complete. 2024-05-22T21:18:57.187 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:18:57.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f2585e00640 1 Processor -- start 2024-05-22T21:18:57.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f2585e00640 1 -- start start 2024-05-22T21:18:57.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f2585e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f258006c2a0 0x7f258006c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f2585e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258015d9d0 con 0x7f25801402b0 2024-05-22T21:18:57.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f2585e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258015dcb0 con 0x7f25800b3370 2024-05-22T21:18:57.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f2585e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258006cc90 con 0x7f258006c2a0 2024-05-22T21:18:57.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257ea00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f25801402b0 0x7f2580070590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51920/0 (socket says 172.21.15.161:51920) 2024-05-22T21:18:57.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257ea00640 1 -- 172.21.15.161:0/1059141003 learned_addr learned my addr 172.21.15.161:0/1059141003 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:57.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257d600640 1 -- 172.21.15.161:0/1059141003 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224863585 0 0) 0x7f258015d9d0 con 0x7f25801402b0 2024-05-22T21:18:57.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257e000640 1 --2- 172.21.15.161:0/1059141003 >> v2:172.21.15.179:3300/0 conn(0x7f258006c2a0 0x7f258006c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257d600640 1 -- 172.21.15.161:0/1059141003 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2564003530 con 0x7f25801402b0 2024-05-22T21:18:57.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257d600640 1 -- 172.21.15.161:0/1059141003 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2197422916 0 0) 0x7f2564003530 con 0x7f25801402b0 2024-05-22T21:18:57.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257d600640 1 -- 172.21.15.161:0/1059141003 >> v2:172.21.15.179:3300/0 conn(0x7f258006c2a0 msgr2=0x7f258006c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:57.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257d600640 1 --2- 172.21.15.161:0/1059141003 >> v2:172.21.15.179:3300/0 conn(0x7f258006c2a0 0x7f258006c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257d600640 1 -- 172.21.15.161:0/1059141003 >> v1:172.21.15.179:6789/0 conn(0x7f25800b3370 legacy=0x7f25800b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257d600640 1 -- 172.21.15.161:0/1059141003 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258015df90 con 0x7f25801402b0 2024-05-22T21:18:57.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257d600640 1 -- 172.21.15.161:0/1059141003 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f2570001680 con 0x7f25801402b0 2024-05-22T21:18:57.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257d600640 1 -- 172.21.15.161:0/1059141003 >> v1:172.21.15.161:6789/0 conn(0x7f25801402b0 legacy=0x7f2580070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257d600640 1 --2- 172.21.15.161:0/1059141003 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2564003980 0x7f2564003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257d600640 1 --2- 172.21.15.161:0/1059141003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25640047d0 0x7f2564004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257d600640 1 --2- 172.21.15.161:0/1059141003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2564005190 0x7f256400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257d600640 1 -- 172.21.15.161:0/1059141003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258006cc90 con 0x7f25640047d0 2024-05-22T21:18:57.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257d600640 1 -- 172.21.15.161:0/1059141003 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256400c030 con 0x7f2564005190 2024-05-22T21:18:57.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.323+0000 7f257d600640 1 -- 172.21.15.161:0/1059141003 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256400c1f0 con 0x7f2564003980 2024-05-22T21:18:57.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.327+0000 7f257e000640 1 --2- 172.21.15.161:0/1059141003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25640047d0 0x7f2564004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.327+0000 7f257ea00640 1 --2- 172.21.15.161:0/1059141003 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2564003980 0x7f2564003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.327+0000 7f257f400640 1 --2- 172.21.15.161:0/1059141003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2564005190 0x7f256400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.327+0000 7f257e000640 1 -- 172.21.15.161:0/1059141003 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2564003980 msgr2=0x7f2564003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:57.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.327+0000 7f257e000640 1 --2- 172.21.15.161:0/1059141003 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2564003980 0x7f2564003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.327+0000 7f257e000640 1 -- 172.21.15.161:0/1059141003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2564005190 msgr2=0x7f256400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:57.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.327+0000 7f257e000640 1 --2- 172.21.15.161:0/1059141003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2564005190 0x7f256400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.327+0000 7f257e000640 1 -- 172.21.15.161:0/1059141003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2564003530 con 0x7f25640047d0 2024-05-22T21:18:57.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.327+0000 7f257f400640 1 -- 172.21.15.161:0/1059141003 reap_dead start 2024-05-22T21:18:57.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.327+0000 7f257e000640 1 --2- 172.21.15.161:0/1059141003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25640047d0 0x7f2564004c50 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f256c009ae0 tx=0x7f256c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376977f5fce04b3f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:57.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.327+0000 7f257d600640 1 -- 172.21.15.161:0/1059141003 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c0a5580 con 0x7f25640047d0 2024-05-22T21:18:57.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.327+0000 7f257d600640 1 -- 172.21.15.161:0/1059141003 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256c0a56e0 con 0x7f25640047d0 2024-05-22T21:18:57.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.327+0000 7f257d600640 1 -- 172.21.15.161:0/1059141003 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c0a5a90 con 0x7f25640047d0 2024-05-22T21:18:57.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f2585e00640 1 -- 172.21.15.161:0/1059141003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25640047d0 msgr2=0x7f2564004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f2585e00640 1 --2- 172.21.15.161:0/1059141003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25640047d0 0x7f2564004c50 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f256c009ae0 tx=0x7f256c01ca70 comp rx=0 tx=0).stop 2024-05-22T21:18:57.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f2585e00640 1 -- 172.21.15.161:0/1059141003 shutdown_connections 2024-05-22T21:18:57.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f2585e00640 1 --2- 172.21.15.161:0/1059141003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25640047d0 0x7f2564004c50 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f2585e00640 1 -- 172.21.15.161:0/1059141003 >> 172.21.15.161:0/1059141003 conn(0x7f258006a2d0 msgr2=0x7f258006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:57.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f2585e00640 1 -- 172.21.15.161:0/1059141003 shutdown_connections 2024-05-22T21:18:57.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f2585e00640 1 -- 172.21.15.161:0/1059141003 wait complete. 2024-05-22T21:18:57.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f2585e00640 1 Processor -- start 2024-05-22T21:18:57.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f2585e00640 1 -- start start 2024-05-22T21:18:57.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f2585e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25640047d0 0x7f25801f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f2585e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2580158870 0x7f25801f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f2585e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2580158cf0 0x7f25801f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f2585e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25801f6900 con 0x7f25640047d0 2024-05-22T21:18:57.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f2585e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25801f6260 con 0x7f2580158cf0 2024-05-22T21:18:57.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f2585e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25801f37a0 con 0x7f2580158870 2024-05-22T21:18:57.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f257ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25640047d0 0x7f25801f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f257ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25640047d0 0x7f25801f6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51798/0 (socket says 172.21.15.161:51798) 2024-05-22T21:18:57.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f257ea00640 1 -- 172.21.15.161:0/444003997 learned_addr learned my addr 172.21.15.161:0/444003997 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:57.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f257e000640 1 --2- 172.21.15.161:0/444003997 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2580158870 0x7f25801f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f257f400640 1 --2- 172.21.15.161:0/444003997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2580158cf0 0x7f25801f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f257ea00640 1 -- 172.21.15.161:0/444003997 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2580158870 msgr2=0x7f25801f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f257ea00640 1 --2- 172.21.15.161:0/444003997 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2580158870 0x7f25801f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f257ea00640 1 -- 172.21.15.161:0/444003997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2580158cf0 msgr2=0x7f25801f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f257ea00640 1 --2- 172.21.15.161:0/444003997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2580158cf0 0x7f25801f31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f257ea00640 1 -- 172.21.15.161:0/444003997 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25801f3a80 con 0x7f25640047d0 2024-05-22T21:18:57.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.331+0000 7f257ea00640 1 --2- 172.21.15.161:0/444003997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25640047d0 0x7f25801f6d20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2570012490 tx=0x7f2570012950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:57.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.335+0000 7f2577400640 1 -- 172.21.15.161:0/444003997 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570014750 con 0x7f25640047d0 2024-05-22T21:18:57.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.335+0000 7f2577400640 1 -- 172.21.15.161:0/444003997 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25700148b0 con 0x7f25640047d0 2024-05-22T21:18:57.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.335+0000 7f2577400640 1 -- 172.21.15.161:0/444003997 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570014c60 con 0x7f25640047d0 2024-05-22T21:18:57.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.335+0000 7f2585e00640 1 -- 172.21.15.161:0/444003997 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25801f6520 con 0x7f25640047d0 2024-05-22T21:18:57.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.335+0000 7f2585e00640 1 -- 172.21.15.161:0/444003997 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2580156df0 con 0x7f25640047d0 2024-05-22T21:18:57.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.335+0000 7f2585e00640 1 -- 172.21.15.161:0/444003997 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25801f6a40 con 0x7f25640047d0 2024-05-22T21:18:57.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.335+0000 7f2577400640 1 -- 172.21.15.161:0/444003997 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f25700153e0 con 0x7f25640047d0 2024-05-22T21:18:57.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.335+0000 7f2577400640 1 --2- 172.21.15.161:0/444003997 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2560066690 0x7f2560068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.335+0000 7f2577400640 1 -- 172.21.15.161:0/444003997 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2570086980 con 0x7f25640047d0 2024-05-22T21:18:57.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.335+0000 7f257e000640 1 --2- 172.21.15.161:0/444003997 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2560066690 0x7f2560068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.335+0000 7f257e000640 1 --2- 172.21.15.161:0/444003997 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2560066690 0x7f2560068b50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f256c0af000 tx=0x7f256c0046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:57.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.339+0000 7f2577400640 1 -- 172.21.15.161:0/444003997 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f25801f6a40 con 0x7f25640047d0 2024-05-22T21:18:57.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.491+0000 7f2585e00640 1 -- 172.21.15.161:0/444003997 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25801f3c10 con 0x7f25640047d0 2024-05-22T21:18:57.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.495+0000 7f2577400640 1 -- 172.21.15.161:0/444003997 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7f257001c0a0 con 0x7f25640047d0 2024-05-22T21:18:57.497 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:18:57.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.495+0000 7f2585e00640 1 -- 172.21.15.161:0/444003997 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2560066690 msgr2=0x7f2560068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.495+0000 7f2585e00640 1 --2- 172.21.15.161:0/444003997 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2560066690 0x7f2560068b50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f256c0af000 tx=0x7f256c0046d0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.495+0000 7f2585e00640 1 -- 172.21.15.161:0/444003997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25640047d0 msgr2=0x7f25801f6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.495+0000 7f2585e00640 1 --2- 172.21.15.161:0/444003997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25640047d0 0x7f25801f6d20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2570012490 tx=0x7f2570012950 comp rx=0 tx=0).stop 2024-05-22T21:18:57.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.495+0000 7f2585e00640 1 -- 172.21.15.161:0/444003997 shutdown_connections 2024-05-22T21:18:57.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.495+0000 7f2585e00640 1 --2- 172.21.15.161:0/444003997 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2560066690 0x7f2560068b50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.495+0000 7f2585e00640 1 --2- 172.21.15.161:0/444003997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2580158cf0 0x7f25801f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.495+0000 7f2585e00640 1 --2- 172.21.15.161:0/444003997 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2580158870 0x7f25801f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.495+0000 7f2585e00640 1 --2- 172.21.15.161:0/444003997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25640047d0 0x7f25801f6d20 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.495+0000 7f2585e00640 1 -- 172.21.15.161:0/444003997 >> 172.21.15.161:0/444003997 conn(0x7f258006a2d0 msgr2=0x7f258015c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:57.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.495+0000 7f2585e00640 1 -- 172.21.15.161:0/444003997 shutdown_connections 2024-05-22T21:18:57.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.495+0000 7f2585e00640 1 -- 172.21.15.161:0/444003997 wait complete. 2024-05-22T21:18:57.516 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:18:57.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.655+0000 7fdf97000640 1 Processor -- start 2024-05-22T21:18:57.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.655+0000 7fdf97000640 1 -- start start 2024-05-22T21:18:57.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.655+0000 7fdf97000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdf901402e0 0x7fdf9013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.655+0000 7fdf97000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf90140090 con 0x7fdf9013fc80 2024-05-22T21:18:57.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.655+0000 7fdf97000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9013d3a0 con 0x7fdf9013cfc0 2024-05-22T21:18:57.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.655+0000 7fdf97000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf900cd920 con 0x7fdf901402e0 2024-05-22T21:18:57.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.655+0000 7fdf95000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fdf9013fc80 0x7fdf9006fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51926/0 (socket says 172.21.15.161:51926) 2024-05-22T21:18:57.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.655+0000 7fdf95000640 1 -- 172.21.15.161:0/2448838236 learned_addr learned my addr 172.21.15.161:0/2448838236 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:57.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.655+0000 7fdf87400640 1 --2- 172.21.15.161:0/2448838236 >> v2:172.21.15.161:3300/0 conn(0x7fdf901402e0 0x7fdf9013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.655+0000 7fdf87400640 1 -- 172.21.15.161:0/2448838236 >> v1:172.21.15.179:6789/0 conn(0x7fdf9013cfc0 legacy=0x7fdf9015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.655+0000 7fdf87400640 1 -- 172.21.15.161:0/2448838236 >> v1:172.21.15.161:6789/0 conn(0x7fdf9013fc80 legacy=0x7fdf9006fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.655+0000 7fdf87400640 1 -- 172.21.15.161:0/2448838236 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf900cdc00 con 0x7fdf901402e0 2024-05-22T21:18:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.655+0000 7fdf8f400640 1 -- 172.21.15.161:0/2448838236 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2112387283 0 0) 0x7fdf90140090 con 0x7fdf9013fc80 2024-05-22T21:18:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.659+0000 7fdf87400640 1 --2- 172.21.15.161:0/2448838236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf901402e0 0x7fdf9013f740 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fdf780097c0 tx=0x7fdf7801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b85cfcf4e0652e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:57.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.659+0000 7fdf8f400640 1 -- 172.21.15.161:0/2448838236 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf780ac440 con 0x7fdf901402e0 2024-05-22T21:18:57.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.659+0000 7fdf8f400640 1 -- 172.21.15.161:0/2448838236 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf780ac5a0 con 0x7fdf901402e0 2024-05-22T21:18:57.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.659+0000 7fdf8f400640 1 -- 172.21.15.161:0/2448838236 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf780ac930 con 0x7fdf901402e0 2024-05-22T21:18:57.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.659+0000 7fdf97000640 1 -- 172.21.15.161:0/2448838236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf901402e0 msgr2=0x7fdf9013f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.659+0000 7fdf97000640 1 --2- 172.21.15.161:0/2448838236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf901402e0 0x7fdf9013f740 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fdf780097c0 tx=0x7fdf7801c3e0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.659+0000 7fdf97000640 1 -- 172.21.15.161:0/2448838236 shutdown_connections 2024-05-22T21:18:57.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.659+0000 7fdf97000640 1 --2- 172.21.15.161:0/2448838236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf901402e0 0x7fdf9013f740 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.659+0000 7fdf97000640 1 -- 172.21.15.161:0/2448838236 >> 172.21.15.161:0/2448838236 conn(0x7fdf9006a2d0 msgr2=0x7fdf9006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:57.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.659+0000 7fdf97000640 1 -- 172.21.15.161:0/2448838236 shutdown_connections 2024-05-22T21:18:57.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.659+0000 7fdf97000640 1 -- 172.21.15.161:0/2448838236 wait complete. 2024-05-22T21:18:57.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf97000640 1 Processor -- start 2024-05-22T21:18:57.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf97000640 1 -- start start 2024-05-22T21:18:57.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf97000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf9013cfc0 0x7fdf9015cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf97000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf9013fc80 0x7fdf90140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf97000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf901402e0 0x7fdf901585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf97000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf90141e60 con 0x7fdf901402e0 2024-05-22T21:18:57.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf97000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf901411e0 con 0x7fdf9013cfc0 2024-05-22T21:18:57.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf97000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf901414c0 con 0x7fdf9013fc80 2024-05-22T21:18:57.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf95000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf901402e0 0x7fdf901585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf87400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf9013fc80 0x7fdf90140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf8fe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf9013cfc0 0x7fdf9015cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf87400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf9013fc80 0x7fdf90140890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:46560/0 (socket says 172.21.15.161:46560) 2024-05-22T21:18:57.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf95000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf901402e0 0x7fdf901585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51816/0 (socket says 172.21.15.161:51816) 2024-05-22T21:18:57.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf87400640 1 -- 172.21.15.161:0/2253742968 learned_addr learned my addr 172.21.15.161:0/2253742968 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:57.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf8fe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf9013cfc0 0x7fdf9015cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:52792/0 (socket says 172.21.15.161:52792) 2024-05-22T21:18:57.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf95000640 1 -- 172.21.15.161:0/2253742968 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf9013fc80 msgr2=0x7fdf90140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf95000640 1 --2- 172.21.15.161:0/2253742968 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf9013fc80 0x7fdf90140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf95000640 1 -- 172.21.15.161:0/2253742968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf9013cfc0 msgr2=0x7fdf9015cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf95000640 1 --2- 172.21.15.161:0/2253742968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf9013cfc0 0x7fdf9015cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf95000640 1 -- 172.21.15.161:0/2253742968 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf78009420 con 0x7fdf901402e0 2024-05-22T21:18:57.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf8fe00640 1 --2- 172.21.15.161:0/2253742968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf9013cfc0 0x7fdf9015cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:18:57.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf95000640 1 --2- 172.21.15.161:0/2253742968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf901402e0 0x7fdf901585f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fdf88002240 tx=0x7fdf8800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:57.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf8cc00640 1 -- 172.21.15.161:0/2253742968 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8800ce30 con 0x7fdf901402e0 2024-05-22T21:18:57.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf8cc00640 1 -- 172.21.15.161:0/2253742968 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8800a640 con 0x7fdf901402e0 2024-05-22T21:18:57.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf8cc00640 1 -- 172.21.15.161:0/2253742968 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8800a9f0 con 0x7fdf901402e0 2024-05-22T21:18:57.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf97000640 1 -- 172.21.15.161:0/2253742968 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf901417a0 con 0x7fdf901402e0 2024-05-22T21:18:57.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf97000640 1 -- 172.21.15.161:0/2253742968 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9015d530 con 0x7fdf901402e0 2024-05-22T21:18:57.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.663+0000 7fdf97000640 1 -- 172.21.15.161:0/2253742968 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf90067220 con 0x7fdf901402e0 2024-05-22T21:18:57.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.667+0000 7fdf8cc00640 1 -- 172.21.15.161:0/2253742968 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf8800ab50 con 0x7fdf901402e0 2024-05-22T21:18:57.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.667+0000 7fdf8cc00640 1 --2- 172.21.15.161:0/2253742968 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdf6c066690 0x7fdf6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.667+0000 7fdf8cc00640 1 -- 172.21.15.161:0/2253742968 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf880834f0 con 0x7fdf901402e0 2024-05-22T21:18:57.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.667+0000 7fdf8fe00640 1 --2- 172.21.15.161:0/2253742968 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdf6c066690 0x7fdf6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.667+0000 7fdf8fe00640 1 --2- 172.21.15.161:0/2253742968 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdf6c066690 0x7fdf6c068b50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdf80007fd0 tx=0x7fdf80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:57.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.671+0000 7fdf8cc00640 1 -- 172.21.15.161:0/2253742968 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdf88050900 con 0x7fdf901402e0 2024-05-22T21:18:57.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.823+0000 7fdf97000640 1 -- 172.21.15.161:0/2253742968 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf90141fa0 con 0x7fdf901402e0 2024-05-22T21:18:57.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.823+0000 7fdf8cc00640 1 -- 172.21.15.161:0/2253742968 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf88018020 con 0x7fdf901402e0 2024-05-22T21:18:57.826 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:18:57.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.823+0000 7fdf97000640 1 -- 172.21.15.161:0/2253742968 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdf6c066690 msgr2=0x7fdf6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.823+0000 7fdf97000640 1 --2- 172.21.15.161:0/2253742968 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdf6c066690 0x7fdf6c068b50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdf80007fd0 tx=0x7fdf80009040 comp rx=0 tx=0).stop 2024-05-22T21:18:57.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.823+0000 7fdf97000640 1 -- 172.21.15.161:0/2253742968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf901402e0 msgr2=0x7fdf901585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.823+0000 7fdf97000640 1 --2- 172.21.15.161:0/2253742968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf901402e0 0x7fdf901585f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fdf88002240 tx=0x7fdf8800cbe0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.823+0000 7fdf97000640 1 -- 172.21.15.161:0/2253742968 shutdown_connections 2024-05-22T21:18:57.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.823+0000 7fdf97000640 1 --2- 172.21.15.161:0/2253742968 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdf6c066690 0x7fdf6c068b50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.823+0000 7fdf97000640 1 --2- 172.21.15.161:0/2253742968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf901402e0 0x7fdf901585f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.823+0000 7fdf97000640 1 --2- 172.21.15.161:0/2253742968 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf9013fc80 0x7fdf90140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.823+0000 7fdf97000640 1 --2- 172.21.15.161:0/2253742968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf9013cfc0 0x7fdf9015cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.823+0000 7fdf97000640 1 -- 172.21.15.161:0/2253742968 >> 172.21.15.161:0/2253742968 conn(0x7fdf9006a2d0 msgr2=0x7fdf9006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:57.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.827+0000 7fdf97000640 1 -- 172.21.15.161:0/2253742968 shutdown_connections 2024-05-22T21:18:57.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.827+0000 7fdf97000640 1 -- 172.21.15.161:0/2253742968 wait complete. 2024-05-22T21:18:57.847 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:18:57.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.983+0000 7efeb4000640 1 Processor -- start 2024-05-22T21:18:57.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.983+0000 7efeb4000640 1 -- start start 2024-05-22T21:18:57.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.983+0000 7efeb4000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efeac13cfc0 0x7efeac1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.983+0000 7efeb4000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efeac15af60 0x7efeac13f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.983+0000 7efeb4000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeac140090 con 0x7efeac13fc80 2024-05-22T21:18:57.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.983+0000 7efeb4000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeac13d3a0 con 0x7efeac15af60 2024-05-22T21:18:57.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.983+0000 7efeb4000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeac0cd920 con 0x7efeac13cfc0 2024-05-22T21:18:57.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.987+0000 7efeb0c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efeac15af60 0x7efeac13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.987+0000 7efeb0c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efeac15af60 0x7efeac13f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51818/0 (socket says 172.21.15.161:51818) 2024-05-22T21:18:57.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.987+0000 7efeb0c00640 1 -- 172.21.15.161:0/3883280022 learned_addr learned my addr 172.21.15.161:0/3883280022 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:57.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.987+0000 7efeb1600640 1 --2- 172.21.15.161:0/3883280022 >> v2:172.21.15.179:3300/0 conn(0x7efeac13cfc0 0x7efeac1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.987+0000 7efeb0c00640 1 -- 172.21.15.161:0/3883280022 >> v2:172.21.15.179:3300/0 conn(0x7efeac13cfc0 msgr2=0x7efeac1402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:57.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.987+0000 7efeb0c00640 1 --2- 172.21.15.161:0/3883280022 >> v2:172.21.15.179:3300/0 conn(0x7efeac13cfc0 0x7efeac1402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.987+0000 7efeb0c00640 1 -- 172.21.15.161:0/3883280022 >> v1:172.21.15.179:6789/0 conn(0x7efeac13fc80 legacy=0x7efeac15dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.987+0000 7efeb0c00640 1 -- 172.21.15.161:0/3883280022 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeac0cdc00 con 0x7efeac15af60 2024-05-22T21:18:57.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.987+0000 7efeb0c00640 1 --2- 172.21.15.161:0/3883280022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeac15af60 0x7efeac13f740 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7efeac06de90 tx=0x7efe9801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a2d958a04cdf04 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:57.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.987+0000 7efeabe00640 1 -- 172.21.15.161:0/3883280022 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe980ab600 con 0x7efeac15af60 2024-05-22T21:18:57.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.987+0000 7efeabe00640 1 -- 172.21.15.161:0/3883280022 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe980a7050 con 0x7efeac15af60 2024-05-22T21:18:57.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.987+0000 7efeabe00640 1 -- 172.21.15.161:0/3883280022 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe980ab980 con 0x7efeac15af60 2024-05-22T21:18:57.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb4000640 1 -- 172.21.15.161:0/3883280022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeac15af60 msgr2=0x7efeac13f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb4000640 1 --2- 172.21.15.161:0/3883280022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeac15af60 0x7efeac13f740 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7efeac06de90 tx=0x7efe9801c490 comp rx=0 tx=0).stop 2024-05-22T21:18:57.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb4000640 1 -- 172.21.15.161:0/3883280022 shutdown_connections 2024-05-22T21:18:57.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb4000640 1 --2- 172.21.15.161:0/3883280022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeac15af60 0x7efeac13f740 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb4000640 1 --2- 172.21.15.161:0/3883280022 >> v2:172.21.15.179:3300/0 conn(0x7efeac13cfc0 0x7efeac1402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb4000640 1 -- 172.21.15.161:0/3883280022 >> 172.21.15.161:0/3883280022 conn(0x7efeac06a2d0 msgr2=0x7efeac06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:57.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb4000640 1 -- 172.21.15.161:0/3883280022 shutdown_connections 2024-05-22T21:18:57.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb4000640 1 -- 172.21.15.161:0/3883280022 wait complete. 2024-05-22T21:18:57.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb4000640 1 Processor -- start 2024-05-22T21:18:57.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb4000640 1 -- start start 2024-05-22T21:18:57.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb4000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efeac13cfc0 0x7efeac0b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb4000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeac13fc80 0x7efeac0b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb4000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeac15af60 0x7efeac159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:57.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb4000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac0b4020 con 0x7efeac13fc80 2024-05-22T21:18:57.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb4000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac138e70 con 0x7efeac15af60 2024-05-22T21:18:57.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb4000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeac138540 con 0x7efeac13cfc0 2024-05-22T21:18:57.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb0c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeac13fc80 0x7efeac0b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb0c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeac13fc80 0x7efeac0b2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51834/0 (socket says 172.21.15.161:51834) 2024-05-22T21:18:57.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb0c00640 1 -- 172.21.15.161:0/4048408640 learned_addr learned my addr 172.21.15.161:0/4048408640 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:57.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb1600640 1 --2- 172.21.15.161:0/4048408640 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efeac13cfc0 0x7efeac0b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb2000640 1 --2- 172.21.15.161:0/4048408640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeac15af60 0x7efeac159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:57.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb0c00640 1 -- 172.21.15.161:0/4048408640 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efeac13cfc0 msgr2=0x7efeac0b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb0c00640 1 --2- 172.21.15.161:0/4048408640 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efeac13cfc0 0x7efeac0b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb0c00640 1 -- 172.21.15.161:0/4048408640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeac15af60 msgr2=0x7efeac159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:57.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb0c00640 1 --2- 172.21.15.161:0/4048408640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeac15af60 0x7efeac159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:57.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.991+0000 7efeb0c00640 1 -- 172.21.15.161:0/4048408640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe980094d0 con 0x7efeac13fc80 2024-05-22T21:18:57.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.995+0000 7efeb0c00640 1 --2- 172.21.15.161:0/4048408640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeac13fc80 0x7efeac0b2180 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7efe980ae040 tx=0x7efe98019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:57.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.995+0000 7efea9600640 1 -- 172.21.15.161:0/4048408640 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe980b4420 con 0x7efeac13fc80 2024-05-22T21:18:57.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.995+0000 7efea9600640 1 -- 172.21.15.161:0/4048408640 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe980b4580 con 0x7efeac13fc80 2024-05-22T21:18:57.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.995+0000 7efea9600640 1 -- 172.21.15.161:0/4048408640 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe980b48f0 con 0x7efeac13fc80 2024-05-22T21:18:57.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.995+0000 7efeb4000640 1 -- 172.21.15.161:0/4048408640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeac138820 con 0x7efeac13fc80 2024-05-22T21:18:57.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.995+0000 7efeb4000640 1 -- 172.21.15.161:0/4048408640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeac156850 con 0x7efeac13fc80 2024-05-22T21:18:57.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.995+0000 7efeb4000640 1 -- 172.21.15.161:0/4048408640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeac067220 con 0x7efeac13fc80 2024-05-22T21:18:57.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.995+0000 7efea9600640 1 -- 172.21.15.161:0/4048408640 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efe980b2070 con 0x7efeac13fc80 2024-05-22T21:18:58.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.995+0000 7efea9600640 1 --2- 172.21.15.161:0/4048408640 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efe8c066690 0x7efe8c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:58.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.995+0000 7efea9600640 1 -- 172.21.15.161:0/4048408640 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe9811dce0 con 0x7efeac13fc80 2024-05-22T21:18:58.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.995+0000 7efeb1600640 1 --2- 172.21.15.161:0/4048408640 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efe8c066690 0x7efe8c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:58.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.999+0000 7efeb1600640 1 --2- 172.21.15.161:0/4048408640 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efe8c066690 0x7efe8c068b50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efeac0b4200 tx=0x7efea0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:58.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:57.999+0000 7efea9600640 1 -- 172.21.15.161:0/4048408640 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efe980eb0f0 con 0x7efeac13fc80 2024-05-22T21:18:58.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.163+0000 7efeb4000640 1 -- 172.21.15.161:0/4048408640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efeac138fb0 con 0x7efeac13fc80 2024-05-22T21:18:58.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.163+0000 7efea9600640 1 -- 172.21.15.161:0/4048408640 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) v1 ==== 80+0+0 (secure 0 0 0) 0x7efe980ee710 con 0x7efeac13fc80 2024-05-22T21:18:58.167 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:18:58.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.167+0000 7efeb4000640 1 -- 172.21.15.161:0/4048408640 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efe8c066690 msgr2=0x7efe8c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:58.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.167+0000 7efeb4000640 1 --2- 172.21.15.161:0/4048408640 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efe8c066690 0x7efe8c068b50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efeac0b4200 tx=0x7efea0009190 comp rx=0 tx=0).stop 2024-05-22T21:18:58.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.167+0000 7efeb4000640 1 -- 172.21.15.161:0/4048408640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeac13fc80 msgr2=0x7efeac0b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:58.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.167+0000 7efeb4000640 1 --2- 172.21.15.161:0/4048408640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeac13fc80 0x7efeac0b2180 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7efe980ae040 tx=0x7efe98019230 comp rx=0 tx=0).stop 2024-05-22T21:18:58.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.167+0000 7efeb4000640 1 -- 172.21.15.161:0/4048408640 shutdown_connections 2024-05-22T21:18:58.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.167+0000 7efeb4000640 1 --2- 172.21.15.161:0/4048408640 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efe8c066690 0x7efe8c068b50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.167+0000 7efeb4000640 1 --2- 172.21.15.161:0/4048408640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeac15af60 0x7efeac159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.167+0000 7efeb4000640 1 --2- 172.21.15.161:0/4048408640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeac13fc80 0x7efeac0b2180 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.167+0000 7efeb4000640 1 --2- 172.21.15.161:0/4048408640 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efeac13cfc0 0x7efeac0b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.167+0000 7efeb4000640 1 -- 172.21.15.161:0/4048408640 >> 172.21.15.161:0/4048408640 conn(0x7efeac06a2d0 msgr2=0x7efeac1372e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:58.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.167+0000 7efeb4000640 1 -- 172.21.15.161:0/4048408640 shutdown_connections 2024-05-22T21:18:58.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.167+0000 7efeb4000640 1 -- 172.21.15.161:0/4048408640 wait complete. 2024-05-22T21:18:58.187 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-22T21:18:58.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:58.187 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-22T21:18:58.237 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:18:58.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:58.238 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-22T21:18:58.256 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:18:58.256 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:58.266 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:18:58.266 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:18:58.317 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:18:58.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.451+0000 7f3ddb000640 1 Processor -- start 2024-05-22T21:18:58.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.451+0000 7f3ddb000640 1 -- start start 2024-05-22T21:18:58.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.451+0000 7f3ddb000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd40b3c10 0x7f3dd40cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:58.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.451+0000 7f3ddb000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3dd406afe0 0x7f3dd40cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:58.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.451+0000 7f3ddb000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd40cb620 con 0x7f3dd40cd7a0 2024-05-22T21:18:58.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.451+0000 7f3ddb000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd406c470 con 0x7f3dd406afe0 2024-05-22T21:18:58.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.451+0000 7f3ddb000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd40cb410 con 0x7f3dd40b3c10 2024-05-22T21:18:58.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.451+0000 7f3dda600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3dd406afe0 0x7f3dd40cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:58.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.451+0000 7f3dda600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3dd406afe0 0x7f3dd40cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51838/0 (socket says 172.21.15.161:51838) 2024-05-22T21:18:58.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.451+0000 7f3dda600640 1 -- 172.21.15.161:0/3098397672 learned_addr learned my addr 172.21.15.161:0/3098397672 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:58.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.451+0000 7f3dd9200640 1 --2- 172.21.15.161:0/3098397672 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd40b3c10 0x7f3dd40cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:58.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.451+0000 7f3dda600640 1 -- 172.21.15.161:0/3098397672 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd40b3c10 msgr2=0x7f3dd40cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:58.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.451+0000 7f3dda600640 1 --2- 172.21.15.161:0/3098397672 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd40b3c10 0x7f3dd40cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.451+0000 7f3dda600640 1 -- 172.21.15.161:0/3098397672 >> v1:172.21.15.179:6789/0 conn(0x7f3dd40cd7a0 legacy=0x7f3dd40cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:58.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.451+0000 7f3dda600640 1 -- 172.21.15.161:0/3098397672 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd40cb900 con 0x7f3dd406afe0 2024-05-22T21:18:58.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.455+0000 7f3dda600640 1 --2- 172.21.15.161:0/3098397672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd406afe0 0x7f3dd40cf5b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009e50 tx=0x7f3dcc01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2d363bfbd627ca server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:58.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.455+0000 7f3dd3e00640 1 -- 172.21.15.161:0/3098397672 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc0a57e0 con 0x7f3dd406afe0 2024-05-22T21:18:58.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.455+0000 7f3dd3e00640 1 -- 172.21.15.161:0/3098397672 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dcc0a5940 con 0x7f3dd406afe0 2024-05-22T21:18:58.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.455+0000 7f3dd3e00640 1 -- 172.21.15.161:0/3098397672 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc0a5cd0 con 0x7f3dd406afe0 2024-05-22T21:18:58.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.455+0000 7f3ddb000640 1 -- 172.21.15.161:0/3098397672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd406afe0 msgr2=0x7f3dd40cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:58.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.455+0000 7f3ddb000640 1 --2- 172.21.15.161:0/3098397672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd406afe0 0x7f3dd40cf5b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009e50 tx=0x7f3dcc01ca00 comp rx=0 tx=0).stop 2024-05-22T21:18:58.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.455+0000 7f3ddb000640 1 -- 172.21.15.161:0/3098397672 shutdown_connections 2024-05-22T21:18:58.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.455+0000 7f3ddb000640 1 --2- 172.21.15.161:0/3098397672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd406afe0 0x7f3dd40cf5b0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.455+0000 7f3ddb000640 1 --2- 172.21.15.161:0/3098397672 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd40b3c10 0x7f3dd40cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.455+0000 7f3ddb000640 1 -- 172.21.15.161:0/3098397672 >> 172.21.15.161:0/3098397672 conn(0x7f3dd40d0dc0 msgr2=0x7f3dd40d11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:58.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.455+0000 7f3ddb000640 1 -- 172.21.15.161:0/3098397672 shutdown_connections 2024-05-22T21:18:58.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.455+0000 7f3ddb000640 1 -- 172.21.15.161:0/3098397672 wait complete. 2024-05-22T21:18:58.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3ddb000640 1 Processor -- start 2024-05-22T21:18:58.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3ddb000640 1 -- start start 2024-05-22T21:18:58.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3ddb000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd406afe0 0x7f3dd4160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:58.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3ddb000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd40b3c10 0x7f3dd414f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:58.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3ddb000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd40cd7a0 0x7f3dd415e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:58.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3ddb000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd414fde0 con 0x7f3dd406afe0 2024-05-22T21:18:58.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3ddb000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd4155a40 con 0x7f3dd40cd7a0 2024-05-22T21:18:58.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3ddb000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd4155d20 con 0x7f3dd40b3c10 2024-05-22T21:18:58.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3dd9c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd406afe0 0x7f3dd4160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:58.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3dd9c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd406afe0 0x7f3dd4160540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51844/0 (socket says 172.21.15.161:51844) 2024-05-22T21:18:58.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3dd9c00640 1 -- 172.21.15.161:0/1945235894 learned_addr learned my addr 172.21.15.161:0/1945235894 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:58.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3dda600640 1 --2- 172.21.15.161:0/1945235894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd40cd7a0 0x7f3dd415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:58.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3dd9200640 1 --2- 172.21.15.161:0/1945235894 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd40b3c10 0x7f3dd414f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:58.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3dd9c00640 1 -- 172.21.15.161:0/1945235894 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd40b3c10 msgr2=0x7f3dd414f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:58.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3dd9c00640 1 --2- 172.21.15.161:0/1945235894 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd40b3c10 0x7f3dd414f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3dd9c00640 1 -- 172.21.15.161:0/1945235894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd40cd7a0 msgr2=0x7f3dd415e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:58.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3dd9c00640 1 --2- 172.21.15.161:0/1945235894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd40cd7a0 0x7f3dd415e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3dd9c00640 1 -- 172.21.15.161:0/1945235894 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc009b30 con 0x7f3dd406afe0 2024-05-22T21:18:58.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3dd9c00640 1 --2- 172.21.15.161:0/1945235894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd406afe0 0x7f3dd4160540 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800c3c0 tx=0x7f3dc800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:58.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3dd1600640 1 -- 172.21.15.161:0/1945235894 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc80052d0 con 0x7f3dd406afe0 2024-05-22T21:18:58.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3dd1600640 1 -- 172.21.15.161:0/1945235894 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc8005430 con 0x7f3dd406afe0 2024-05-22T21:18:58.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3ddb000640 1 -- 172.21.15.161:0/1945235894 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd4156000 con 0x7f3dd406afe0 2024-05-22T21:18:58.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3dd1600640 1 -- 172.21.15.161:0/1945235894 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8003840 con 0x7f3dd406afe0 2024-05-22T21:18:58.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3ddb000640 1 -- 172.21.15.161:0/1945235894 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd41565c0 con 0x7f3dd406afe0 2024-05-22T21:18:58.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.459+0000 7f3ddb000640 1 -- 172.21.15.161:0/1945235894 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd41567e0 con 0x7f3dd406afe0 2024-05-22T21:18:58.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.463+0000 7f3dd1600640 1 -- 172.21.15.161:0/1945235894 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3dc80039c0 con 0x7f3dd406afe0 2024-05-22T21:18:58.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.463+0000 7f3dd1600640 1 --2- 172.21.15.161:0/1945235894 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3db4066690 0x7f3db4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:58.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.463+0000 7f3dd1600640 1 -- 172.21.15.161:0/1945235894 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3dc8083750 con 0x7f3dd406afe0 2024-05-22T21:18:58.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.463+0000 7f3dd9200640 1 --2- 172.21.15.161:0/1945235894 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3db4066690 0x7f3db4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:58.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.463+0000 7f3dd9200640 1 --2- 172.21.15.161:0/1945235894 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3db4066690 0x7f3db4068b50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3dd41f9c40 tx=0x7f3dc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:58.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.467+0000 7f3dd1600640 1 -- 172.21.15.161:0/1945235894 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3dc8050a40 con 0x7f3dd406afe0 2024-05-22T21:18:58.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.627+0000 7f3ddb000640 1 -- 172.21.15.161:0/1945235894 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3dd4067070 con 0x7f3dd406afe0 2024-05-22T21:18:58.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.627+0000 7f3dd1600640 1 -- 172.21.15.161:0/1945235894 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16152 (secure 0 0 0) 0x7f3dc8054060 con 0x7f3dd406afe0 2024-05-22T21:18:58.631 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:18:58.631 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":113,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:18:50.632514+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:18:16.500534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-22T21:18:16.937445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:18:58.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.631+0000 7f3ddb000640 1 -- 172.21.15.161:0/1945235894 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3db4066690 msgr2=0x7f3db4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:58.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.631+0000 7f3ddb000640 1 --2- 172.21.15.161:0/1945235894 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3db4066690 0x7f3db4068b50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3dd41f9c40 tx=0x7f3dc0008040 comp rx=0 tx=0).stop 2024-05-22T21:18:58.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.631+0000 7f3ddb000640 1 -- 172.21.15.161:0/1945235894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd406afe0 msgr2=0x7f3dd4160540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:58.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.631+0000 7f3ddb000640 1 --2- 172.21.15.161:0/1945235894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd406afe0 0x7f3dd4160540 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800c3c0 tx=0x7f3dc800c880 comp rx=0 tx=0).stop 2024-05-22T21:18:58.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.631+0000 7f3ddb000640 1 -- 172.21.15.161:0/1945235894 shutdown_connections 2024-05-22T21:18:58.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.631+0000 7f3ddb000640 1 --2- 172.21.15.161:0/1945235894 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3db4066690 0x7f3db4068b50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.631+0000 7f3ddb000640 1 --2- 172.21.15.161:0/1945235894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd40cd7a0 0x7f3dd415e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.631+0000 7f3ddb000640 1 --2- 172.21.15.161:0/1945235894 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd40b3c10 0x7f3dd414f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.631+0000 7f3ddb000640 1 --2- 172.21.15.161:0/1945235894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3dd406afe0 0x7f3dd4160540 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.631+0000 7f3ddb000640 1 -- 172.21.15.161:0/1945235894 >> 172.21.15.161:0/1945235894 conn(0x7f3dd40d0dc0 msgr2=0x7f3dd40cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:58.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.631+0000 7f3ddb000640 1 -- 172.21.15.161:0/1945235894 shutdown_connections 2024-05-22T21:18:58.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.631+0000 7f3ddb000640 1 -- 172.21.15.161:0/1945235894 wait complete. 2024-05-22T21:18:58.652 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:18:16.500534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:18:16.937445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:18:58.653 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:18:58.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.787+0000 7f10ce000640 1 Processor -- start 2024-05-22T21:18:58.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.787+0000 7f10ce000640 1 -- start start 2024-05-22T21:18:58.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.787+0000 7f10ce000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f10c806c9b0 0x7f10c80b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:58.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.787+0000 7f10ce000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c8144bb0 con 0x7f10c8143c40 2024-05-22T21:18:58.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.787+0000 7f10ce000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c815efb0 con 0x7f10c815b600 2024-05-22T21:18:58.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.787+0000 7f10ce000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c815ba60 con 0x7f10c806c9b0 2024-05-22T21:18:58.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.787+0000 7f10c7400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f10c8143c40 0x7f10c81444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51936/0 (socket says 172.21.15.161:51936) 2024-05-22T21:18:58.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.787+0000 7f10c7400640 1 -- 172.21.15.161:0/2806825442 learned_addr learned my addr 172.21.15.161:0/2806825442 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:58.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c7e00640 1 --2- 172.21.15.161:0/2806825442 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f10c806c9b0 0x7f10c80b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:58.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 -- 172.21.15.161:0/2806825442 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241010904 0 0) 0x7f10c8144bb0 con 0x7f10c8143c40 2024-05-22T21:18:58.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 -- 172.21.15.161:0/2806825442 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10ac003530 con 0x7f10c8143c40 2024-05-22T21:18:58.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 -- 172.21.15.161:0/2806825442 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124428707 0 0) 0x7f10c815efb0 con 0x7f10c815b600 2024-05-22T21:18:58.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 -- 172.21.15.161:0/2806825442 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10c8144bb0 con 0x7f10c815b600 2024-05-22T21:18:58.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 -- 172.21.15.161:0/2806825442 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2169799666 0 0) 0x7f10ac003530 con 0x7f10c8143c40 2024-05-22T21:18:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 -- 172.21.15.161:0/2806825442 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f10c806c9b0 msgr2=0x7f10c80b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 --2- 172.21.15.161:0/2806825442 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f10c806c9b0 0x7f10c80b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 -- 172.21.15.161:0/2806825442 >> v1:172.21.15.179:6789/0 conn(0x7f10c815b600 legacy=0x7f10c806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 -- 172.21.15.161:0/2806825442 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c815f290 con 0x7f10c8143c40 2024-05-22T21:18:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 -- 172.21.15.161:0/2806825442 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f10b8001680 con 0x7f10c8143c40 2024-05-22T21:18:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 -- 172.21.15.161:0/2806825442 >> v1:172.21.15.161:6789/0 conn(0x7f10c8143c40 legacy=0x7f10c81444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 --2- 172.21.15.161:0/2806825442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10ac003b40 0x7f10ac003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 --2- 172.21.15.161:0/2806825442 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f10c806c9b0 0x7f10ac0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 --2- 172.21.15.161:0/2806825442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10ac004d10 0x7f10ac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 -- 172.21.15.161:0/2806825442 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c815ba60 con 0x7f10ac004d10 2024-05-22T21:18:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 -- 172.21.15.161:0/2806825442 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac00ba40 con 0x7f10ac003b40 2024-05-22T21:18:58.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c6a00640 1 -- 172.21.15.161:0/2806825442 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac00bbb0 con 0x7f10c806c9b0 2024-05-22T21:18:58.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c7e00640 1 --2- 172.21.15.161:0/2806825442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10ac004d10 0x7f10ac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:58.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c7400640 1 --2- 172.21.15.161:0/2806825442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10ac003b40 0x7f10ac003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:58.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10bea00640 1 --2- 172.21.15.161:0/2806825442 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f10c806c9b0 0x7f10ac0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:58.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c7e00640 1 -- 172.21.15.161:0/2806825442 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f10c806c9b0 msgr2=0x7f10ac0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:58.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c7e00640 1 --2- 172.21.15.161:0/2806825442 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f10c806c9b0 0x7f10ac0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c7e00640 1 -- 172.21.15.161:0/2806825442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10ac003b40 msgr2=0x7f10ac003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:58.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c7e00640 1 --2- 172.21.15.161:0/2806825442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10ac003b40 0x7f10ac003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10c7e00640 1 -- 172.21.15.161:0/2806825442 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac003530 con 0x7f10ac004d10 2024-05-22T21:18:58.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.791+0000 7f10bea00640 1 --2- 172.21.15.161:0/2806825442 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f10c806c9b0 0x7f10ac0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:18:58.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10c7e00640 1 --2- 172.21.15.161:0/2806825442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10ac004d10 0x7f10ac005190 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009fc0 tx=0x7f10c001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a321133c1469f7dd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:58.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10c6a00640 1 -- 172.21.15.161:0/2806825442 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b8007ea0 con 0x7f10ac004d10 2024-05-22T21:18:58.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10c6a00640 1 -- 172.21.15.161:0/2806825442 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c000f040 con 0x7f10ac004d10 2024-05-22T21:18:58.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10c6a00640 1 -- 172.21.15.161:0/2806825442 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c00a5dc0 con 0x7f10ac004d10 2024-05-22T21:18:58.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10ce000640 1 -- 172.21.15.161:0/2806825442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10ac004d10 msgr2=0x7f10ac005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:58.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10ce000640 1 --2- 172.21.15.161:0/2806825442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10ac004d10 0x7f10ac005190 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009fc0 tx=0x7f10c001cc80 comp rx=0 tx=0).stop 2024-05-22T21:18:58.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10c7e00640 1 -- 172.21.15.161:0/2806825442 reap_dead start 2024-05-22T21:18:58.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10ce000640 1 -- 172.21.15.161:0/2806825442 shutdown_connections 2024-05-22T21:18:58.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10ce000640 1 -- 172.21.15.161:0/2806825442 >> 172.21.15.161:0/2806825442 conn(0x7f10c806a2d0 msgr2=0x7f10c806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:58.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10ce000640 1 -- 172.21.15.161:0/2806825442 shutdown_connections 2024-05-22T21:18:58.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10ce000640 1 -- 172.21.15.161:0/2806825442 wait complete. 2024-05-22T21:18:58.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10ce000640 1 Processor -- start 2024-05-22T21:18:58.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10ce000640 1 -- start start 2024-05-22T21:18:58.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10ce000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10ac004d10 0x7f10c815e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:58.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10ce000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f10c8156a70 0x7f10c813eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:58.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10ce000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c813f090 0x7f10c813fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:58.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.795+0000 7f10ce000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c813f510 con 0x7f10c813f090 2024-05-22T21:18:58.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10ce000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c81f7570 con 0x7f10ac004d10 2024-05-22T21:18:58.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10ce000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c81f69f0 con 0x7f10c8156a70 2024-05-22T21:18:58.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10c7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c813f090 0x7f10c813fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:58.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10c7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c813f090 0x7f10c813fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51864/0 (socket says 172.21.15.161:51864) 2024-05-22T21:18:58.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10c7e00640 1 -- 172.21.15.161:0/3524068445 learned_addr learned my addr 172.21.15.161:0/3524068445 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:58.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10c7400640 1 --2- 172.21.15.161:0/3524068445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10ac004d10 0x7f10c815e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:58.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10bea00640 1 --2- 172.21.15.161:0/3524068445 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f10c8156a70 0x7f10c813eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:58.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10c7e00640 1 -- 172.21.15.161:0/3524068445 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f10c8156a70 msgr2=0x7f10c813eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:58.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10c7e00640 1 --2- 172.21.15.161:0/3524068445 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f10c8156a70 0x7f10c813eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10c7e00640 1 -- 172.21.15.161:0/3524068445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10ac004d10 msgr2=0x7f10c815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:58.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10c7e00640 1 --2- 172.21.15.161:0/3524068445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10ac004d10 0x7f10c815e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10c7e00640 1 -- 172.21.15.161:0/3524068445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c81f6cd0 con 0x7f10c813f090 2024-05-22T21:18:58.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10c7e00640 1 --2- 172.21.15.161:0/3524068445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c813f090 0x7f10c813fed0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f10c00b0040 tx=0x7f10c0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:58.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10bfe00640 1 -- 172.21.15.161:0/3524068445 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c00039a0 con 0x7f10c813f090 2024-05-22T21:18:58.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10bfe00640 1 -- 172.21.15.161:0/3524068445 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c0003b00 con 0x7f10c813f090 2024-05-22T21:18:58.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10ce000640 1 -- 172.21.15.161:0/3524068445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c81f6fb0 con 0x7f10c813f090 2024-05-22T21:18:58.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10bfe00640 1 -- 172.21.15.161:0/3524068445 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0003e70 con 0x7f10c813f090 2024-05-22T21:18:58.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10ce000640 1 -- 172.21.15.161:0/3524068445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c815e960 con 0x7f10c813f090 2024-05-22T21:18:58.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10ce000640 1 -- 172.21.15.161:0/3524068445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c8067220 con 0x7f10c813f090 2024-05-22T21:18:58.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.799+0000 7f10bfe00640 1 -- 172.21.15.161:0/3524068445 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f10c00b4040 con 0x7f10c813f090 2024-05-22T21:18:58.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.803+0000 7f10bfe00640 1 --2- 172.21.15.161:0/3524068445 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f10a40666c0 0x7f10a4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:58.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.803+0000 7f10bfe00640 1 -- 172.21.15.161:0/3524068445 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10c011df60 con 0x7f10c813f090 2024-05-22T21:18:58.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.803+0000 7f10c7400640 1 --2- 172.21.15.161:0/3524068445 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f10a40666c0 0x7f10a4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:58.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.803+0000 7f10c7400640 1 --2- 172.21.15.161:0/3524068445 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f10a40666c0 0x7f10a4068b80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f10b8001c50 tx=0x7f10b8001e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:58.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.807+0000 7f10bfe00640 1 -- 172.21.15.161:0/3524068445 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f10c00eb250 con 0x7f10c813f090 2024-05-22T21:18:58.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.959+0000 7f10ce000640 1 -- 172.21.15.161:0/3524068445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f10c8157600 con 0x7f10c813f090 2024-05-22T21:18:58.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.959+0000 7f10bfe00640 1 -- 172.21.15.161:0/3524068445 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f10c00bc070 con 0x7f10c813f090 2024-05-22T21:18:58.961 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:18:58.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.959+0000 7f10ce000640 1 -- 172.21.15.161:0/3524068445 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f10a40666c0 msgr2=0x7f10a4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:58.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.959+0000 7f10ce000640 1 --2- 172.21.15.161:0/3524068445 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f10a40666c0 0x7f10a4068b80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f10b8001c50 tx=0x7f10b8001e30 comp rx=0 tx=0).stop 2024-05-22T21:18:58.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.959+0000 7f10ce000640 1 -- 172.21.15.161:0/3524068445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c813f090 msgr2=0x7f10c813fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:58.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.959+0000 7f10ce000640 1 --2- 172.21.15.161:0/3524068445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c813f090 0x7f10c813fed0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f10c00b0040 tx=0x7f10c0009750 comp rx=0 tx=0).stop 2024-05-22T21:18:58.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.959+0000 7f10ce000640 1 -- 172.21.15.161:0/3524068445 shutdown_connections 2024-05-22T21:18:58.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.959+0000 7f10ce000640 1 --2- 172.21.15.161:0/3524068445 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f10a40666c0 0x7f10a4068b80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.959+0000 7f10ce000640 1 --2- 172.21.15.161:0/3524068445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10c813f090 0x7f10c813fed0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.959+0000 7f10ce000640 1 --2- 172.21.15.161:0/3524068445 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f10c8156a70 0x7f10c813eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.959+0000 7f10ce000640 1 --2- 172.21.15.161:0/3524068445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10ac004d10 0x7f10c815e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:58.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.959+0000 7f10ce000640 1 -- 172.21.15.161:0/3524068445 >> 172.21.15.161:0/3524068445 conn(0x7f10c806a2d0 msgr2=0x7f10c8141520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:58.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.959+0000 7f10ce000640 1 -- 172.21.15.161:0/3524068445 shutdown_connections 2024-05-22T21:18:58.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:58.959+0000 7f10ce000640 1 -- 172.21.15.161:0/3524068445 wait complete. 2024-05-22T21:18:58.983 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:18:59.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.119+0000 7f53e8000640 1 Processor -- start 2024-05-22T21:18:59.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.119+0000 7f53e8000640 1 -- start start 2024-05-22T21:18:59.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.119+0000 7f53e8000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f53e015b600 0x7f53e006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:59.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.119+0000 7f53e8000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e006cc40 con 0x7f53e006c7e0 2024-05-22T21:18:59.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.119+0000 7f53e8000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e0144bb0 con 0x7f53e0143c40 2024-05-22T21:18:59.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.119+0000 7f53e8000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e015ba60 con 0x7f53e015b600 2024-05-22T21:18:59.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.119+0000 7f53e4c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f53e015b600 0x7f53e006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:59.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.119+0000 7f53e6000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f53e006c7e0 0x7f53e00b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51942/0 (socket says 172.21.15.161:51942) 2024-05-22T21:18:59.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.119+0000 7f53e6000640 1 -- 172.21.15.161:0/2416002879 learned_addr learned my addr 172.21.15.161:0/2416002879 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:59.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.119+0000 7f53e4c00640 1 -- 172.21.15.161:0/2416002879 >> v1:172.21.15.179:6789/0 conn(0x7f53e0143c40 legacy=0x7f53e01444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.119+0000 7f53e4c00640 1 -- 172.21.15.161:0/2416002879 >> v1:172.21.15.161:6789/0 conn(0x7f53e006c7e0 legacy=0x7f53e00b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.119+0000 7f53e4c00640 1 -- 172.21.15.161:0/2416002879 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e015efb0 con 0x7f53e015b600 2024-05-22T21:18:59.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.123+0000 7f53e4c00640 1 --2- 172.21.15.161:0/2416002879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e015b600 0x7f53e006c2a0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0097c0 tx=0x7f53cc01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc92093f3cf60011 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:59.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.123+0000 7f53dfe00640 1 -- 172.21.15.161:0/2416002879 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0ab600 con 0x7f53e015b600 2024-05-22T21:18:59.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.123+0000 7f53dfe00640 1 -- 172.21.15.161:0/2416002879 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53cc0a7050 con 0x7f53e015b600 2024-05-22T21:18:59.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.123+0000 7f53dfe00640 1 -- 172.21.15.161:0/2416002879 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0ab980 con 0x7f53e015b600 2024-05-22T21:18:59.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.123+0000 7f53e8000640 1 -- 172.21.15.161:0/2416002879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e015b600 msgr2=0x7f53e006c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.123+0000 7f53e8000640 1 --2- 172.21.15.161:0/2416002879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e015b600 0x7f53e006c2a0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0097c0 tx=0x7f53cc01c330 comp rx=0 tx=0).stop 2024-05-22T21:18:59.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.123+0000 7f53e8000640 1 -- 172.21.15.161:0/2416002879 shutdown_connections 2024-05-22T21:18:59.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.123+0000 7f53e8000640 1 --2- 172.21.15.161:0/2416002879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e015b600 0x7f53e006c2a0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.123+0000 7f53e8000640 1 -- 172.21.15.161:0/2416002879 >> 172.21.15.161:0/2416002879 conn(0x7f53e006a2d0 msgr2=0x7f53e006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:59.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.123+0000 7f53e8000640 1 -- 172.21.15.161:0/2416002879 shutdown_connections 2024-05-22T21:18:59.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.123+0000 7f53e8000640 1 -- 172.21.15.161:0/2416002879 wait complete. 2024-05-22T21:18:59.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e8000640 1 Processor -- start 2024-05-22T21:18:59.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e8000640 1 -- start start 2024-05-22T21:18:59.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e8000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53e006c7e0 0x7f53e00717e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:59.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e8000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e0143c40 0x7f53e0138db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:59.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e8000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53e015b600 0x7f53e013ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:59.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e8000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e0071d20 con 0x7f53e0143c40 2024-05-22T21:18:59.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e8000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00714b0 con 0x7f53e006c7e0 2024-05-22T21:18:59.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e8000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e0141ff0 con 0x7f53e015b600 2024-05-22T21:18:59.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e4c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e0143c40 0x7f53e0138db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:59.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e4c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e0143c40 0x7f53e0138db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51882/0 (socket says 172.21.15.161:51882) 2024-05-22T21:18:59.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e4c00640 1 -- 172.21.15.161:0/3171496299 learned_addr learned my addr 172.21.15.161:0/3171496299 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:59.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e5600640 1 --2- 172.21.15.161:0/3171496299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53e006c7e0 0x7f53e00717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:59.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e6000640 1 --2- 172.21.15.161:0/3171496299 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53e015b600 0x7f53e013ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:59.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e4c00640 1 -- 172.21.15.161:0/3171496299 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53e015b600 msgr2=0x7f53e013ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e4c00640 1 --2- 172.21.15.161:0/3171496299 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53e015b600 0x7f53e013ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e4c00640 1 -- 172.21.15.161:0/3171496299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53e006c7e0 msgr2=0x7f53e00717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e4c00640 1 --2- 172.21.15.161:0/3171496299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53e006c7e0 0x7f53e00717e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e4c00640 1 -- 172.21.15.161:0/3171496299 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc009420 con 0x7f53e0143c40 2024-05-22T21:18:59.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e6000640 1 --2- 172.21.15.161:0/3171496299 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53e015b600 0x7f53e013ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:18:59.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e4c00640 1 --2- 172.21.15.161:0/3171496299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e0143c40 0x7f53e0138db0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f53cc018f40 tx=0x7f53cc0190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:59.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53dd600640 1 -- 172.21.15.161:0/3171496299 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0aa830 con 0x7f53e0143c40 2024-05-22T21:18:59.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53dd600640 1 -- 172.21.15.161:0/3171496299 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53cc0aa9b0 con 0x7f53e0143c40 2024-05-22T21:18:59.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e8000640 1 -- 172.21.15.161:0/3171496299 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e00706f0 con 0x7f53e0143c40 2024-05-22T21:18:59.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53dd600640 1 -- 172.21.15.161:0/3171496299 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0aad60 con 0x7f53e0143c40 2024-05-22T21:18:59.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.127+0000 7f53e8000640 1 -- 172.21.15.161:0/3171496299 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e0070cb0 con 0x7f53e0143c40 2024-05-22T21:18:59.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.131+0000 7f53e8000640 1 -- 172.21.15.161:0/3171496299 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e0067220 con 0x7f53e0143c40 2024-05-22T21:18:59.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.131+0000 7f53dd600640 1 -- 172.21.15.161:0/3171496299 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f53cc0b2040 con 0x7f53e0143c40 2024-05-22T21:18:59.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.131+0000 7f53dd600640 1 --2- 172.21.15.161:0/3171496299 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f53c0066690 0x7f53c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:59.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.131+0000 7f53dd600640 1 -- 172.21.15.161:0/3171496299 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53cc11e670 con 0x7f53e0143c40 2024-05-22T21:18:59.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.131+0000 7f53e5600640 1 --2- 172.21.15.161:0/3171496299 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f53c0066690 0x7f53c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:59.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.131+0000 7f53e5600640 1 --2- 172.21.15.161:0/3171496299 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f53c0066690 0x7f53c0068b50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f53d4000f70 tx=0x7f53d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:59.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.135+0000 7f53dd600640 1 -- 172.21.15.161:0/3171496299 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f53cc0eb940 con 0x7f53e0143c40 2024-05-22T21:18:59.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.291+0000 7f53e8000640 1 -- 172.21.15.161:0/3171496299 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f53e00ceaa0 con 0x7f53e0143c40 2024-05-22T21:18:59.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.291+0000 7f53dd600640 1 -- 172.21.15.161:0/3171496299 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7f53cc0eef60 con 0x7f53e0143c40 2024-05-22T21:18:59.293 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:18:59.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.291+0000 7f53e8000640 1 -- 172.21.15.161:0/3171496299 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f53c0066690 msgr2=0x7f53c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.291+0000 7f53e8000640 1 --2- 172.21.15.161:0/3171496299 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f53c0066690 0x7f53c0068b50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f53d4000f70 tx=0x7f53d4009040 comp rx=0 tx=0).stop 2024-05-22T21:18:59.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.291+0000 7f53e8000640 1 -- 172.21.15.161:0/3171496299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e0143c40 msgr2=0x7f53e0138db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.291+0000 7f53e8000640 1 --2- 172.21.15.161:0/3171496299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e0143c40 0x7f53e0138db0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f53cc018f40 tx=0x7f53cc0190d0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.291+0000 7f53e8000640 1 -- 172.21.15.161:0/3171496299 shutdown_connections 2024-05-22T21:18:59.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.291+0000 7f53e8000640 1 --2- 172.21.15.161:0/3171496299 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f53c0066690 0x7f53c0068b50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.291+0000 7f53e8000640 1 --2- 172.21.15.161:0/3171496299 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53e015b600 0x7f53e013ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.291+0000 7f53e8000640 1 --2- 172.21.15.161:0/3171496299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e0143c40 0x7f53e0138db0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.291+0000 7f53e8000640 1 --2- 172.21.15.161:0/3171496299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53e006c7e0 0x7f53e00717e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.291+0000 7f53e8000640 1 -- 172.21.15.161:0/3171496299 >> 172.21.15.161:0/3171496299 conn(0x7f53e006a2d0 msgr2=0x7f53e015a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:59.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.291+0000 7f53e8000640 1 -- 172.21.15.161:0/3171496299 shutdown_connections 2024-05-22T21:18:59.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.291+0000 7f53e8000640 1 -- 172.21.15.161:0/3171496299 wait complete. 2024-05-22T21:18:59.313 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:18:59.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.447+0000 7fa12fe00640 1 Processor -- start 2024-05-22T21:18:59.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.447+0000 7fa12fe00640 1 -- start start 2024-05-22T21:18:59.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.447+0000 7fa12fe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1281402e0 0x7fa12813f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:59.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.447+0000 7fa12fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa12813fc80 0x7fa12806fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:59.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.447+0000 7fa12fe00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1281400d0 con 0x7fa12813cfc0 2024-05-22T21:18:59.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.451+0000 7fa12fe00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12813d3a0 con 0x7fa12813fc80 2024-05-22T21:18:59.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.451+0000 7fa12fe00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280cd920 con 0x7fa1281402e0 2024-05-22T21:18:59.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.451+0000 7fa12de00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa12813fc80 0x7fa12806fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:59.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.451+0000 7fa12de00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa12813fc80 0x7fa12806fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51892/0 (socket says 172.21.15.161:51892) 2024-05-22T21:18:59.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.451+0000 7fa12de00640 1 -- 172.21.15.161:0/141537665 learned_addr learned my addr 172.21.15.161:0/141537665 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:59.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.451+0000 7fa12ca00640 1 --2- 172.21.15.161:0/141537665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1281402e0 0x7fa12813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:59.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.451+0000 7fa12de00640 1 -- 172.21.15.161:0/141537665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1281402e0 msgr2=0x7fa12813f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:18:59.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.451+0000 7fa12de00640 1 --2- 172.21.15.161:0/141537665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1281402e0 0x7fa12813f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.451+0000 7fa12de00640 1 -- 172.21.15.161:0/141537665 >> v1:172.21.15.161:6789/0 conn(0x7fa12813cfc0 legacy=0x7fa12815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.451+0000 7fa12de00640 1 -- 172.21.15.161:0/141537665 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1280cdc00 con 0x7fa12813fc80 2024-05-22T21:18:59.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.451+0000 7fa127e00640 1 -- 172.21.15.161:0/141537665 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685851570 0 0) 0x7fa11c001120 con 0x7fa12813cfc0 2024-05-22T21:18:59.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.451+0000 7fa12de00640 1 --2- 172.21.15.161:0/141537665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12813fc80 0x7fa12806fce0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa12000d150 tx=0x7fa12001eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3eb13f54d5153b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:59.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.451+0000 7fa127e00640 1 -- 172.21.15.161:0/141537665 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12000b2d0 con 0x7fa12813fc80 2024-05-22T21:18:59.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.451+0000 7fa127e00640 1 -- 172.21.15.161:0/141537665 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12000b430 con 0x7fa12813fc80 2024-05-22T21:18:59.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.451+0000 7fa127e00640 1 -- 172.21.15.161:0/141537665 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120009820 con 0x7fa12813fc80 2024-05-22T21:18:59.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12fe00640 1 -- 172.21.15.161:0/141537665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12813fc80 msgr2=0x7fa12806fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12fe00640 1 --2- 172.21.15.161:0/141537665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12813fc80 0x7fa12806fce0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa12000d150 tx=0x7fa12001eb70 comp rx=0 tx=0).stop 2024-05-22T21:18:59.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12fe00640 1 -- 172.21.15.161:0/141537665 shutdown_connections 2024-05-22T21:18:59.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12fe00640 1 --2- 172.21.15.161:0/141537665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12813fc80 0x7fa12806fce0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12fe00640 1 --2- 172.21.15.161:0/141537665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1281402e0 0x7fa12813f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12fe00640 1 -- 172.21.15.161:0/141537665 >> 172.21.15.161:0/141537665 conn(0x7fa12806a2d0 msgr2=0x7fa12806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:59.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12fe00640 1 -- 172.21.15.161:0/141537665 shutdown_connections 2024-05-22T21:18:59.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12fe00640 1 -- 172.21.15.161:0/141537665 wait complete. 2024-05-22T21:18:59.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12fe00640 1 Processor -- start 2024-05-22T21:18:59.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12fe00640 1 -- start start 2024-05-22T21:18:59.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12fe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa12813cfc0 0x7fa12812ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:59.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12813fc80 0x7fa1281570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:59.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12fe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1281402e0 0x7fa12815a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:59.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12fe00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12813faa0 con 0x7fa12813fc80 2024-05-22T21:18:59.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12fe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12813d4e0 con 0x7fa1281402e0 2024-05-22T21:18:59.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12fe00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12815cb80 con 0x7fa12813cfc0 2024-05-22T21:18:59.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12ca00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12813fc80 0x7fa1281570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:59.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12ca00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12813fc80 0x7fa1281570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51896/0 (socket says 172.21.15.161:51896) 2024-05-22T21:18:59.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12ca00640 1 -- 172.21.15.161:0/1112910841 learned_addr learned my addr 172.21.15.161:0/1112910841 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:59.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12d400640 1 --2- 172.21.15.161:0/1112910841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa12813cfc0 0x7fa12812ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:59.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12de00640 1 --2- 172.21.15.161:0/1112910841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1281402e0 0x7fa12815a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:59.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12ca00640 1 -- 172.21.15.161:0/1112910841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa12813cfc0 msgr2=0x7fa12812ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12ca00640 1 --2- 172.21.15.161:0/1112910841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa12813cfc0 0x7fa12812ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12ca00640 1 -- 172.21.15.161:0/1112910841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1281402e0 msgr2=0x7fa12815a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12ca00640 1 --2- 172.21.15.161:0/1112910841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1281402e0 0x7fa12815a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.455+0000 7fa12ca00640 1 -- 172.21.15.161:0/1112910841 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12000bd40 con 0x7fa12813fc80 2024-05-22T21:18:59.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.459+0000 7fa12ca00640 1 --2- 172.21.15.161:0/1112910841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12813fc80 0x7fa1281570c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa11400c7f0 tx=0x7fa11400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:59.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.459+0000 7fa125600640 1 -- 172.21.15.161:0/1112910841 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114007820 con 0x7fa12813fc80 2024-05-22T21:18:59.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.459+0000 7fa125600640 1 -- 172.21.15.161:0/1112910841 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa114007980 con 0x7fa12813fc80 2024-05-22T21:18:59.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.459+0000 7fa12fe00640 1 -- 172.21.15.161:0/1112910841 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12815ce20 con 0x7fa12813fc80 2024-05-22T21:18:59.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.459+0000 7fa125600640 1 -- 172.21.15.161:0/1112910841 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114007d30 con 0x7fa12813fc80 2024-05-22T21:18:59.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.459+0000 7fa12fe00640 1 -- 172.21.15.161:0/1112910841 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12815d3e0 con 0x7fa12813fc80 2024-05-22T21:18:59.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.459+0000 7fa12fe00640 1 -- 172.21.15.161:0/1112910841 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12815d5d0 con 0x7fa12813fc80 2024-05-22T21:18:59.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.459+0000 7fa125600640 1 -- 172.21.15.161:0/1112910841 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa1140113f0 con 0x7fa12813fc80 2024-05-22T21:18:59.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.459+0000 7fa125600640 1 --2- 172.21.15.161:0/1112910841 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1080666c0 0x7fa108068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:59.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.459+0000 7fa125600640 1 -- 172.21.15.161:0/1112910841 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1140843e0 con 0x7fa12813fc80 2024-05-22T21:18:59.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.459+0000 7fa12d400640 1 --2- 172.21.15.161:0/1112910841 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1080666c0 0x7fa108068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:59.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.459+0000 7fa12d400640 1 --2- 172.21.15.161:0/1112910841 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1080666c0 0x7fa108068b80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa11c007fd0 tx=0x7fa11c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:59.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.463+0000 7fa125600640 1 -- 172.21.15.161:0/1112910841 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa114051740 con 0x7fa12813fc80 2024-05-22T21:18:59.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.619+0000 7fa12fe00640 1 -- 172.21.15.161:0/1112910841 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa12813d650 con 0x7fa12813fc80 2024-05-22T21:18:59.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.619+0000 7fa125600640 1 -- 172.21.15.161:0/1112910841 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa114054d60 con 0x7fa12813fc80 2024-05-22T21:18:59.624 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:18:59.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.623+0000 7fa12fe00640 1 -- 172.21.15.161:0/1112910841 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1080666c0 msgr2=0x7fa108068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.623+0000 7fa12fe00640 1 --2- 172.21.15.161:0/1112910841 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1080666c0 0x7fa108068b80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa11c007fd0 tx=0x7fa11c009040 comp rx=0 tx=0).stop 2024-05-22T21:18:59.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.623+0000 7fa12fe00640 1 -- 172.21.15.161:0/1112910841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12813fc80 msgr2=0x7fa1281570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.623+0000 7fa12fe00640 1 --2- 172.21.15.161:0/1112910841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12813fc80 0x7fa1281570c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa11400c7f0 tx=0x7fa11400ccb0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.623+0000 7fa12fe00640 1 -- 172.21.15.161:0/1112910841 shutdown_connections 2024-05-22T21:18:59.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.623+0000 7fa12fe00640 1 --2- 172.21.15.161:0/1112910841 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa1080666c0 0x7fa108068b80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.623+0000 7fa12fe00640 1 --2- 172.21.15.161:0/1112910841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1281402e0 0x7fa12815a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.623+0000 7fa12fe00640 1 --2- 172.21.15.161:0/1112910841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa12813fc80 0x7fa1281570c0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.623+0000 7fa12fe00640 1 --2- 172.21.15.161:0/1112910841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa12813cfc0 0x7fa12812ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.623+0000 7fa12fe00640 1 -- 172.21.15.161:0/1112910841 >> 172.21.15.161:0/1112910841 conn(0x7fa12806a2d0 msgr2=0x7fa12815da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:59.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.623+0000 7fa12fe00640 1 -- 172.21.15.161:0/1112910841 shutdown_connections 2024-05-22T21:18:59.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.623+0000 7fa12fe00640 1 -- 172.21.15.161:0/1112910841 wait complete. 2024-05-22T21:18:59.645 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:18:59.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.775+0000 7fbc8ee00640 1 Processor -- start 2024-05-22T21:18:59.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.775+0000 7fbc8ee00640 1 -- start start 2024-05-22T21:18:59.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.779+0000 7fbc8ee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc8806c9b0 0x7fbc880b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:59.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.779+0000 7fbc8ee00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc88144bb0 con 0x7fbc88143c40 2024-05-22T21:18:59.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.779+0000 7fbc8ee00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8815efb0 con 0x7fbc8815b600 2024-05-22T21:18:59.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.779+0000 7fbc8ee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8815ba60 con 0x7fbc8806c9b0 2024-05-22T21:18:59.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.779+0000 7fbc87e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fbc88143c40 0x7fbc881444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51968/0 (socket says 172.21.15.161:51968) 2024-05-22T21:18:59.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.779+0000 7fbc87e00640 1 -- 172.21.15.161:0/574429152 learned_addr learned my addr 172.21.15.161:0/574429152 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:18:59.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.779+0000 7fbc8ce00640 1 --2- 172.21.15.161:0/574429152 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc8806c9b0 0x7fbc880b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:59.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.779+0000 7fbc87400640 1 -- 172.21.15.161:0/574429152 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459768606 0 0) 0x7fbc88144bb0 con 0x7fbc88143c40 2024-05-22T21:18:59.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.779+0000 7fbc87400640 1 -- 172.21.15.161:0/574429152 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc6c003530 con 0x7fbc88143c40 2024-05-22T21:18:59.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.779+0000 7fbc87400640 1 -- 172.21.15.161:0/574429152 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2222373438 0 0) 0x7fbc8815efb0 con 0x7fbc8815b600 2024-05-22T21:18:59.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.779+0000 7fbc87400640 1 -- 172.21.15.161:0/574429152 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc88144bb0 con 0x7fbc8815b600 2024-05-22T21:18:59.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.779+0000 7fbc8ce00640 1 -- 172.21.15.161:0/574429152 >> v1:172.21.15.179:6789/0 conn(0x7fbc8815b600 legacy=0x7fbc8806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.779+0000 7fbc8ce00640 1 -- 172.21.15.161:0/574429152 >> v1:172.21.15.161:6789/0 conn(0x7fbc88143c40 legacy=0x7fbc881444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.779+0000 7fbc8ce00640 1 -- 172.21.15.161:0/574429152 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8815f290 con 0x7fbc8806c9b0 2024-05-22T21:18:59.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.783+0000 7fbc8ce00640 1 --2- 172.21.15.161:0/574429152 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc8806c9b0 0x7fbc880b2160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009dc0 tx=0x7fbc8001c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3cebe33c51a5ca9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:59.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.783+0000 7fbc87400640 1 -- 172.21.15.161:0/574429152 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc800a5a50 con 0x7fbc8806c9b0 2024-05-22T21:18:59.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.783+0000 7fbc87400640 1 -- 172.21.15.161:0/574429152 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc800a5bb0 con 0x7fbc8806c9b0 2024-05-22T21:18:59.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.783+0000 7fbc87400640 1 -- 172.21.15.161:0/574429152 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc800ad440 con 0x7fbc8806c9b0 2024-05-22T21:18:59.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.783+0000 7fbc8ee00640 1 -- 172.21.15.161:0/574429152 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc8806c9b0 msgr2=0x7fbc880b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.783+0000 7fbc8ee00640 1 --2- 172.21.15.161:0/574429152 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc8806c9b0 0x7fbc880b2160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009dc0 tx=0x7fbc8001c990 comp rx=0 tx=0).stop 2024-05-22T21:18:59.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.783+0000 7fbc8ee00640 1 -- 172.21.15.161:0/574429152 shutdown_connections 2024-05-22T21:18:59.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.783+0000 7fbc8ee00640 1 --2- 172.21.15.161:0/574429152 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc8806c9b0 0x7fbc880b2160 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.783+0000 7fbc8ee00640 1 -- 172.21.15.161:0/574429152 >> 172.21.15.161:0/574429152 conn(0x7fbc8806a2d0 msgr2=0x7fbc8806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:59.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.783+0000 7fbc8ee00640 1 -- 172.21.15.161:0/574429152 shutdown_connections 2024-05-22T21:18:59.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.783+0000 7fbc8ee00640 1 -- 172.21.15.161:0/574429152 wait complete. 2024-05-22T21:18:59.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.783+0000 7fbc8ee00640 1 Processor -- start 2024-05-22T21:18:59.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.783+0000 7fbc8ee00640 1 -- start start 2024-05-22T21:18:59.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc8ee00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc8806c9b0 0x7fbc88136b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:59.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc8ee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc88143c40 0x7fbc8814bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:59.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc8ee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc8815b600 0x7fbc8814c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:59.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc8ee00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc88137040 con 0x7fbc8806c9b0 2024-05-22T21:18:59.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc8ee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8815c210 con 0x7fbc8815b600 2024-05-22T21:18:59.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc8ee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8815cab0 con 0x7fbc88143c40 2024-05-22T21:18:59.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc87e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc8806c9b0 0x7fbc88136b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:59.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc87e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc8806c9b0 0x7fbc88136b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51904/0 (socket says 172.21.15.161:51904) 2024-05-22T21:18:59.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc87e00640 1 -- 172.21.15.161:0/2021958472 learned_addr learned my addr 172.21.15.161:0/2021958472 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:18:59.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc7f400640 1 --2- 172.21.15.161:0/2021958472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc88143c40 0x7fbc8814bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:59.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc8ce00640 1 --2- 172.21.15.161:0/2021958472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc8815b600 0x7fbc8814c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:59.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc87e00640 1 -- 172.21.15.161:0/2021958472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc88143c40 msgr2=0x7fbc8814bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc87e00640 1 --2- 172.21.15.161:0/2021958472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc88143c40 0x7fbc8814bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc87e00640 1 -- 172.21.15.161:0/2021958472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc8815b600 msgr2=0x7fbc8814c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc87e00640 1 --2- 172.21.15.161:0/2021958472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc8815b600 0x7fbc8814c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc87e00640 1 -- 172.21.15.161:0/2021958472 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc80009a70 con 0x7fbc8806c9b0 2024-05-22T21:18:59.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc87e00640 1 --2- 172.21.15.161:0/2021958472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc8806c9b0 0x7fbc88136b00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000c500 tx=0x7fbc7000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:59.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc84c00640 1 -- 172.21.15.161:0/2021958472 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70005cf0 con 0x7fbc8806c9b0 2024-05-22T21:18:59.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc84c00640 1 -- 172.21.15.161:0/2021958472 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc70005e50 con 0x7fbc8806c9b0 2024-05-22T21:18:59.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc8ee00640 1 -- 172.21.15.161:0/2021958472 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc88142d10 con 0x7fbc8806c9b0 2024-05-22T21:18:59.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc84c00640 1 -- 172.21.15.161:0/2021958472 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70007330 con 0x7fbc8806c9b0 2024-05-22T21:18:59.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc8ee00640 1 -- 172.21.15.161:0/2021958472 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8815c740 con 0x7fbc8806c9b0 2024-05-22T21:18:59.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.787+0000 7fbc8ee00640 1 -- 172.21.15.161:0/2021958472 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc88067220 con 0x7fbc8806c9b0 2024-05-22T21:18:59.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.791+0000 7fbc84c00640 1 -- 172.21.15.161:0/2021958472 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbc70007490 con 0x7fbc8806c9b0 2024-05-22T21:18:59.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.791+0000 7fbc84c00640 1 --2- 172.21.15.161:0/2021958472 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc640666c0 0x7fbc64068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:18:59.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.791+0000 7fbc84c00640 1 -- 172.21.15.161:0/2021958472 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc70083760 con 0x7fbc8806c9b0 2024-05-22T21:18:59.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.791+0000 7fbc7f400640 1 --2- 172.21.15.161:0/2021958472 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc640666c0 0x7fbc64068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:18:59.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.791+0000 7fbc7f400640 1 --2- 172.21.15.161:0/2021958472 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc640666c0 0x7fbc64068b80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbc88154200 tx=0x7fbc78009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:18:59.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.795+0000 7fbc84c00640 1 -- 172.21.15.161:0/2021958472 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbc70050b70 con 0x7fbc8806c9b0 2024-05-22T21:18:59.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.959+0000 7fbc8ee00640 1 -- 172.21.15.161:0/2021958472 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbc8815ec00 con 0x7fbc8806c9b0 2024-05-22T21:18:59.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.963+0000 7fbc84c00640 1 -- 172.21.15.161:0/2021958472 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbc70017080 con 0x7fbc8806c9b0 2024-05-22T21:18:59.964 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:18:59.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.963+0000 7fbc8ee00640 1 -- 172.21.15.161:0/2021958472 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc640666c0 msgr2=0x7fbc64068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.963+0000 7fbc8ee00640 1 --2- 172.21.15.161:0/2021958472 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc640666c0 0x7fbc64068b80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbc88154200 tx=0x7fbc78009040 comp rx=0 tx=0).stop 2024-05-22T21:18:59.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.963+0000 7fbc8ee00640 1 -- 172.21.15.161:0/2021958472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc8806c9b0 msgr2=0x7fbc88136b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:18:59.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.963+0000 7fbc8ee00640 1 --2- 172.21.15.161:0/2021958472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc8806c9b0 0x7fbc88136b00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000c500 tx=0x7fbc7000c9c0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.963+0000 7fbc8ee00640 1 -- 172.21.15.161:0/2021958472 shutdown_connections 2024-05-22T21:18:59.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.963+0000 7fbc8ee00640 1 --2- 172.21.15.161:0/2021958472 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc640666c0 0x7fbc64068b80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.963+0000 7fbc8ee00640 1 --2- 172.21.15.161:0/2021958472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc8815b600 0x7fbc8814c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.963+0000 7fbc8ee00640 1 --2- 172.21.15.161:0/2021958472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc88143c40 0x7fbc8814bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.963+0000 7fbc8ee00640 1 --2- 172.21.15.161:0/2021958472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc8806c9b0 0x7fbc88136b00 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:18:59.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.963+0000 7fbc8ee00640 1 -- 172.21.15.161:0/2021958472 >> 172.21.15.161:0/2021958472 conn(0x7fbc8806a2d0 msgr2=0x7fbc881532f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:18:59.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.963+0000 7fbc8ee00640 1 -- 172.21.15.161:0/2021958472 shutdown_connections 2024-05-22T21:18:59.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:18:59.963+0000 7fbc8ee00640 1 -- 172.21.15.161:0/2021958472 wait complete. 2024-05-22T21:18:59.986 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-22T21:18:59.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:18:59.987 DEBUG:teuthology.orchestra.run.smithi179:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-22T21:19:00.015 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:19:00.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:00.015 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-22T21:19:00.070 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:19:00.070 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:00.079 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:19:00.079 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:19:00.079 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:00.121 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:19:00.121 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:00.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8534200640 1 Processor -- start 2024-05-22T21:19:00.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8534200640 1 -- start start 2024-05-22T21:19:00.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8534200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f852c1597f0 0x7f852c0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8534200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852c0b3070 0x7f852c0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8534200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852c071040 con 0x7f852c070be0 2024-05-22T21:19:00.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8534200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852c0b3500 con 0x7f852c1597f0 2024-05-22T21:19:00.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8534200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c159bd0 con 0x7f852c0b3070 2024-05-22T21:19:00.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8531800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f852c1597f0 0x7f852c0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8531800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f852c1597f0 0x7f852c0706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51910/0 (socket says 172.21.15.161:51910) 2024-05-22T21:19:00.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8531800640 1 -- 172.21.15.161:0/4260141428 learned_addr learned my addr 172.21.15.161:0/4260141428 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:00.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8530e00640 1 --1- 172.21.15.161:0/4260141428 >> v1:172.21.15.161:6789/0 conn(0x7f852c070be0 0x7f852c0b2960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51980/0 (socket says 172.21.15.161:51980) 2024-05-22T21:19:00.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8532200640 1 --2- 172.21.15.161:0/4260141428 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852c0b3070 0x7f852c0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8531800640 1 -- 172.21.15.161:0/4260141428 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852c0b3070 msgr2=0x7f852c0cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:00.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8531800640 1 --2- 172.21.15.161:0/4260141428 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852c0b3070 0x7f852c0cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8531800640 1 -- 172.21.15.161:0/4260141428 >> v1:172.21.15.161:6789/0 conn(0x7f852c070be0 legacy=0x7f852c0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8531800640 1 -- 172.21.15.161:0/4260141428 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c0cbdd0 con 0x7f852c1597f0 2024-05-22T21:19:00.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f852be00640 1 -- 172.21.15.161:0/4260141428 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3542177691 0 0) 0x7f852c071040 con 0x7f852c070be0 2024-05-22T21:19:00.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.255+0000 7f8532200640 1 --2- 172.21.15.161:0/4260141428 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852c0b3070 0x7f852c0cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:00.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.259+0000 7f8531800640 1 --2- 172.21.15.161:0/4260141428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c1597f0 0x7f852c0706a0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8520009e50 tx=0x7f852001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdacb74db5bdbce4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:00.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.259+0000 7f852be00640 1 -- 172.21.15.161:0/4260141428 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85200a5900 con 0x7f852c1597f0 2024-05-22T21:19:00.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.259+0000 7f852be00640 1 -- 172.21.15.161:0/4260141428 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85200a5a60 con 0x7f852c1597f0 2024-05-22T21:19:00.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.259+0000 7f852be00640 1 -- 172.21.15.161:0/4260141428 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85200a5df0 con 0x7f852c1597f0 2024-05-22T21:19:00.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.259+0000 7f8534200640 1 -- 172.21.15.161:0/4260141428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c1597f0 msgr2=0x7f852c0706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.259+0000 7f8534200640 1 --2- 172.21.15.161:0/4260141428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c1597f0 0x7f852c0706a0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8520009e50 tx=0x7f852001ca30 comp rx=0 tx=0).stop 2024-05-22T21:19:00.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.259+0000 7f8534200640 1 -- 172.21.15.161:0/4260141428 shutdown_connections 2024-05-22T21:19:00.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.259+0000 7f8534200640 1 --2- 172.21.15.161:0/4260141428 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852c0b3070 0x7f852c0cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.259+0000 7f8534200640 1 --2- 172.21.15.161:0/4260141428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c1597f0 0x7f852c0706a0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.259+0000 7f8534200640 1 -- 172.21.15.161:0/4260141428 >> 172.21.15.161:0/4260141428 conn(0x7f852c0b0bc0 msgr2=0x7f852c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:00.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.259+0000 7f8534200640 1 -- 172.21.15.161:0/4260141428 shutdown_connections 2024-05-22T21:19:00.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.259+0000 7f8534200640 1 -- 172.21.15.161:0/4260141428 wait complete. 2024-05-22T21:19:00.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8534200640 1 Processor -- start 2024-05-22T21:19:00.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8534200640 1 -- start start 2024-05-22T21:19:00.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8534200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c070be0 0x7f852c14a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8534200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c0b3070 0x7f852c15c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8534200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852c1597f0 0x7f852c159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8534200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c158e90 con 0x7f852c0b3070 2024-05-22T21:19:00.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8534200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c1496c0 con 0x7f852c070be0 2024-05-22T21:19:00.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8534200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c1499a0 con 0x7f852c1597f0 2024-05-22T21:19:00.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8530e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c0b3070 0x7f852c15c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8530e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c0b3070 0x7f852c15c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51918/0 (socket says 172.21.15.161:51918) 2024-05-22T21:19:00.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8530e00640 1 -- 172.21.15.161:0/2219495184 learned_addr learned my addr 172.21.15.161:0/2219495184 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:00.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8532200640 1 --2- 172.21.15.161:0/2219495184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852c1597f0 0x7f852c159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8531800640 1 --2- 172.21.15.161:0/2219495184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c070be0 0x7f852c14a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8530e00640 1 -- 172.21.15.161:0/2219495184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852c1597f0 msgr2=0x7f852c159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8530e00640 1 --2- 172.21.15.161:0/2219495184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852c1597f0 0x7f852c159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8530e00640 1 -- 172.21.15.161:0/2219495184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c070be0 msgr2=0x7f852c14a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8530e00640 1 --2- 172.21.15.161:0/2219495184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c070be0 0x7f852c14a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8530e00640 1 -- 172.21.15.161:0/2219495184 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8520009b30 con 0x7f852c0b3070 2024-05-22T21:19:00.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8530e00640 1 --2- 172.21.15.161:0/2219495184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c0b3070 0x7f852c15c500 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f85180011f0 tx=0x7f851800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:00.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8529600640 1 -- 172.21.15.161:0/2219495184 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85180074e0 con 0x7f852c0b3070 2024-05-22T21:19:00.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8529600640 1 -- 172.21.15.161:0/2219495184 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8518007640 con 0x7f852c0b3070 2024-05-22T21:19:00.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8534200640 1 -- 172.21.15.161:0/2219495184 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852c149c80 con 0x7f852c0b3070 2024-05-22T21:19:00.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8529600640 1 -- 172.21.15.161:0/2219495184 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518004a10 con 0x7f852c0b3070 2024-05-22T21:19:00.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.263+0000 7f8534200640 1 -- 172.21.15.161:0/2219495184 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c14a240 con 0x7f852c0b3070 2024-05-22T21:19:00.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.267+0000 7f8534200640 1 -- 172.21.15.161:0/2219495184 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852c14a430 con 0x7f852c0b3070 2024-05-22T21:19:00.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.267+0000 7f8529600640 1 -- 172.21.15.161:0/2219495184 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8518004b70 con 0x7f852c0b3070 2024-05-22T21:19:00.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.267+0000 7f8529600640 1 --2- 172.21.15.161:0/2219495184 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f850c0666c0 0x7f850c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.267+0000 7f8529600640 1 -- 172.21.15.161:0/2219495184 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8518083600 con 0x7f852c0b3070 2024-05-22T21:19:00.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.267+0000 7f8531800640 1 --2- 172.21.15.161:0/2219495184 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f850c0666c0 0x7f850c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.267+0000 7f8531800640 1 --2- 172.21.15.161:0/2219495184 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f850c0666c0 0x7f850c068b80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f85200ae000 tx=0x7f85200a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:00.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.271+0000 7f8529600640 1 -- 172.21.15.161:0/2219495184 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8518050a10 con 0x7f852c0b3070 2024-05-22T21:19:00.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.459+0000 7f8534200640 1 -- 172.21.15.161:0/2219495184 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f852c0cb1d0 con 0x7f852c0b3070 2024-05-22T21:19:00.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.459+0000 7f8529600640 1 -- 172.21.15.161:0/2219495184 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f8518054030 con 0x7f852c0b3070 2024-05-22T21:19:00.464 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:00.464 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2771409351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2771409351},{"type":"v1","addr":"172.21.15.179:6834","nonce":2771409351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6442,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2520119431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2520119431},{"type":"v1","addr":"172.21.15.179:6835","nonce":2520119431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1415362547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1415362547},{"type":"v1","addr":"172.21.15.161:6835","nonce":1415362547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:18:17.938858+0000","modified":"2024-05-22T21:18:20.963065+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/4127130864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4127130864},{"type":"v1","addr":"172.21.15.161:6837","nonce":4127130864}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-22T21:19:00.464 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 47 2024-05-22T21:19:00.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.463+0000 7f8534200640 1 -- 172.21.15.161:0/2219495184 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f850c0666c0 msgr2=0x7f850c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.463+0000 7f8534200640 1 --2- 172.21.15.161:0/2219495184 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f850c0666c0 0x7f850c068b80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f85200ae000 tx=0x7f85200a7070 comp rx=0 tx=0).stop 2024-05-22T21:19:00.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.463+0000 7f8534200640 1 -- 172.21.15.161:0/2219495184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c0b3070 msgr2=0x7f852c15c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.463+0000 7f8534200640 1 --2- 172.21.15.161:0/2219495184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c0b3070 0x7f852c15c500 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f85180011f0 tx=0x7f851800c9b0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.463+0000 7f8534200640 1 -- 172.21.15.161:0/2219495184 shutdown_connections 2024-05-22T21:19:00.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.463+0000 7f8534200640 1 --2- 172.21.15.161:0/2219495184 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f850c0666c0 0x7f850c068b80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.463+0000 7f8534200640 1 --2- 172.21.15.161:0/2219495184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852c1597f0 0x7f852c159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.463+0000 7f8534200640 1 --2- 172.21.15.161:0/2219495184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f852c0b3070 0x7f852c15c500 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.463+0000 7f8534200640 1 --2- 172.21.15.161:0/2219495184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c070be0 0x7f852c14a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.463+0000 7f8534200640 1 -- 172.21.15.161:0/2219495184 >> 172.21.15.161:0/2219495184 conn(0x7f852c0b0bc0 msgr2=0x7f852c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:00.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.463+0000 7f8534200640 1 -- 172.21.15.161:0/2219495184 shutdown_connections 2024-05-22T21:19:00.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.463+0000 7f8534200640 1 -- 172.21.15.161:0/2219495184 wait complete. 2024-05-22T21:19:00.484 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:00.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.619+0000 7fbea2600640 1 Processor -- start 2024-05-22T21:19:00.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.619+0000 7fbea2600640 1 -- start start 2024-05-22T21:19:00.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.619+0000 7fbea2600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbe9c1597f0 0x7fbe9c0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.619+0000 7fbea2600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbe9c0b3070 0x7fbe9c06cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.619+0000 7fbea2600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9c071040 con 0x7fbe9c070be0 2024-05-22T21:19:00.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.619+0000 7fbea2600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9c0b3500 con 0x7fbe9c1597f0 2024-05-22T21:19:00.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.619+0000 7fbea2600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c159bd0 con 0x7fbe9c0b3070 2024-05-22T21:19:00.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.619+0000 7fbe9b400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbe9c1597f0 0x7fbe9c0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.619+0000 7fbe9b400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbe9c1597f0 0x7fbe9c0706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:52898/0 (socket says 172.21.15.161:52898) 2024-05-22T21:19:00.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.619+0000 7fbe9b400640 1 -- 172.21.15.161:0/1896688034 learned_addr learned my addr 172.21.15.161:0/1896688034 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:00.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.619+0000 7fbe9be00640 1 --2- 172.21.15.161:0/1896688034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbe9c0b3070 0x7fbe9c06cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.619+0000 7fbe92a00640 1 --1- 172.21.15.161:0/1896688034 >> v1:172.21.15.179:6789/0 conn(0x7fbe9c070be0 0x7fbe9c0b2960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:59564/0 (socket says 172.21.15.161:59564) 2024-05-22T21:19:00.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.623+0000 7fbe9b400640 1 -- 172.21.15.161:0/1896688034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbe9c0b3070 msgr2=0x7fbe9c06cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:00.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.623+0000 7fbe9b400640 1 --2- 172.21.15.161:0/1896688034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbe9c0b3070 0x7fbe9c06cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.623+0000 7fbe9b400640 1 -- 172.21.15.161:0/1896688034 >> v1:172.21.15.179:6789/0 conn(0x7fbe9c070be0 legacy=0x7fbe9c0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.623+0000 7fbe9b400640 1 -- 172.21.15.161:0/1896688034 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c0cbdd0 con 0x7fbe9c1597f0 2024-05-22T21:19:00.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.623+0000 7fbe9be00640 1 --2- 172.21.15.161:0/1896688034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbe9c0b3070 0x7fbe9c06cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:00.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.623+0000 7fbe9b400640 1 --2- 172.21.15.161:0/1896688034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe9c1597f0 0x7fbe9c0706a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009870 tx=0x7fbe8c01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f7d0b1836d330f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:00.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.623+0000 7fbe9aa00640 1 -- 172.21.15.161:0/1896688034 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c0a9620 con 0x7fbe9c1597f0 2024-05-22T21:19:00.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.623+0000 7fbe9aa00640 1 -- 172.21.15.161:0/1896688034 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8c0a9780 con 0x7fbe9c1597f0 2024-05-22T21:19:00.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.623+0000 7fbe9aa00640 1 -- 172.21.15.161:0/1896688034 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c0a9b10 con 0x7fbe9c1597f0 2024-05-22T21:19:00.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.623+0000 7fbea2600640 1 -- 172.21.15.161:0/1896688034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe9c1597f0 msgr2=0x7fbe9c0706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.623+0000 7fbea2600640 1 --2- 172.21.15.161:0/1896688034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe9c1597f0 0x7fbe9c0706a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009870 tx=0x7fbe8c01c560 comp rx=0 tx=0).stop 2024-05-22T21:19:00.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbea2600640 1 -- 172.21.15.161:0/1896688034 shutdown_connections 2024-05-22T21:19:00.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbea2600640 1 --2- 172.21.15.161:0/1896688034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbe9c0b3070 0x7fbe9c06cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbea2600640 1 --2- 172.21.15.161:0/1896688034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe9c1597f0 0x7fbe9c0706a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbea2600640 1 -- 172.21.15.161:0/1896688034 >> 172.21.15.161:0/1896688034 conn(0x7fbe9c0b0bc0 msgr2=0x7fbe9c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:00.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbea2600640 1 -- 172.21.15.161:0/1896688034 shutdown_connections 2024-05-22T21:19:00.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbea2600640 1 -- 172.21.15.161:0/1896688034 wait complete. 2024-05-22T21:19:00.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbea2600640 1 Processor -- start 2024-05-22T21:19:00.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbea2600640 1 -- start start 2024-05-22T21:19:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbea2600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe9c070be0 0x7fbe9c1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbea2600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe9c0b3070 0x7fbe9c15c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbea2600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbe9c1597f0 0x7fbe9c146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbea2600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c158ed0 con 0x7fbe9c070be0 2024-05-22T21:19:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbea2600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c15b9c0 con 0x7fbe9c0b3070 2024-05-22T21:19:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbea2600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c15bca0 con 0x7fbe9c1597f0 2024-05-22T21:19:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbe9b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe9c070be0 0x7fbe9c1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbe9b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe9c070be0 0x7fbe9c1588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51930/0 (socket says 172.21.15.161:51930) 2024-05-22T21:19:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbe9b400640 1 -- 172.21.15.161:0/3650624079 learned_addr learned my addr 172.21.15.161:0/3650624079 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbe92a00640 1 --2- 172.21.15.161:0/3650624079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe9c0b3070 0x7fbe9c15c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbe9be00640 1 --2- 172.21.15.161:0/3650624079 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbe9c1597f0 0x7fbe9c146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbe9b400640 1 -- 172.21.15.161:0/3650624079 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbe9c1597f0 msgr2=0x7fbe9c146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbe9b400640 1 --2- 172.21.15.161:0/3650624079 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbe9c1597f0 0x7fbe9c146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbe9b400640 1 -- 172.21.15.161:0/3650624079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe9c0b3070 msgr2=0x7fbe9c15c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbe9b400640 1 --2- 172.21.15.161:0/3650624079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe9c0b3070 0x7fbe9c15c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbe9b400640 1 -- 172.21.15.161:0/3650624079 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9400ba70 con 0x7fbe9c070be0 2024-05-22T21:19:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.627+0000 7fbe9b400640 1 --2- 172.21.15.161:0/3650624079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe9c070be0 0x7fbe9c1588a0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0ab040 tx=0x7fbe8c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.631+0000 7fbe93e00640 1 -- 172.21.15.161:0/3650624079 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c0bc650 con 0x7fbe9c070be0 2024-05-22T21:19:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.631+0000 7fbe93e00640 1 -- 172.21.15.161:0/3650624079 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8c0bc7d0 con 0x7fbe9c070be0 2024-05-22T21:19:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.631+0000 7fbea2600640 1 -- 172.21.15.161:0/3650624079 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8c0094d0 con 0x7fbe9c070be0 2024-05-22T21:19:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.631+0000 7fbe93e00640 1 -- 172.21.15.161:0/3650624079 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c0bcb80 con 0x7fbe9c070be0 2024-05-22T21:19:00.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.631+0000 7fbea2600640 1 -- 172.21.15.161:0/3650624079 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9c15c9e0 con 0x7fbe9c070be0 2024-05-22T21:19:00.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.631+0000 7fbea2600640 1 -- 172.21.15.161:0/3650624079 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe9c067220 con 0x7fbe9c070be0 2024-05-22T21:19:00.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.631+0000 7fbe93e00640 1 -- 172.21.15.161:0/3650624079 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbe8c0bcce0 con 0x7fbe9c070be0 2024-05-22T21:19:00.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.631+0000 7fbe93e00640 1 --2- 172.21.15.161:0/3650624079 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbe78066690 0x7fbe78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.631+0000 7fbe93e00640 1 -- 172.21.15.161:0/3650624079 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe8c11e760 con 0x7fbe9c070be0 2024-05-22T21:19:00.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.631+0000 7fbe92a00640 1 --2- 172.21.15.161:0/3650624079 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbe78066690 0x7fbe78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.631+0000 7fbe92a00640 1 --2- 172.21.15.161:0/3650624079 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbe78066690 0x7fbe78068b50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c14cda0 tx=0x7fbe84007e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:00.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.635+0000 7fbe93e00640 1 -- 172.21.15.161:0/3650624079 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbe8c0ebc90 con 0x7fbe9c070be0 2024-05-22T21:19:00.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.791+0000 7fbea2600640 1 -- 172.21.15.161:0/3650624079 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe9c15bde0 con 0x7fbe9c070be0 2024-05-22T21:19:00.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.791+0000 7fbe93e00640 1 -- 172.21.15.161:0/3650624079 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16152 (secure 0 0 0) 0x7fbe9c15bde0 con 0x7fbe9c070be0 2024-05-22T21:19:00.795 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:00.795 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":113,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:18:50.632514+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:18:16.500534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-22T21:18:16.937445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.795+0000 7fbea2600640 1 -- 172.21.15.161:0/3650624079 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbe78066690 msgr2=0x7fbe78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.795+0000 7fbea2600640 1 --2- 172.21.15.161:0/3650624079 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbe78066690 0x7fbe78068b50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c14cda0 tx=0x7fbe84007e50 comp rx=0 tx=0).stop 2024-05-22T21:19:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.795+0000 7fbea2600640 1 -- 172.21.15.161:0/3650624079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe9c070be0 msgr2=0x7fbe9c1588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.795+0000 7fbea2600640 1 --2- 172.21.15.161:0/3650624079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe9c070be0 0x7fbe9c1588a0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0ab040 tx=0x7fbe8c01ada0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.795+0000 7fbea2600640 1 -- 172.21.15.161:0/3650624079 shutdown_connections 2024-05-22T21:19:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.795+0000 7fbea2600640 1 --2- 172.21.15.161:0/3650624079 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbe78066690 0x7fbe78068b50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.795+0000 7fbea2600640 1 --2- 172.21.15.161:0/3650624079 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbe9c1597f0 0x7fbe9c146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.795+0000 7fbea2600640 1 --2- 172.21.15.161:0/3650624079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe9c0b3070 0x7fbe9c15c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.795+0000 7fbea2600640 1 --2- 172.21.15.161:0/3650624079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe9c070be0 0x7fbe9c1588a0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.795+0000 7fbea2600640 1 -- 172.21.15.161:0/3650624079 >> 172.21.15.161:0/3650624079 conn(0x7fbe9c0b0bc0 msgr2=0x7fbe9c0ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:00.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.795+0000 7fbea2600640 1 -- 172.21.15.161:0/3650624079 shutdown_connections 2024-05-22T21:19:00.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.795+0000 7fbea2600640 1 -- 172.21.15.161:0/3650624079 wait complete. 2024-05-22T21:19:00.817 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:18:16.500534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:18:16.937445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:19:00.818 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:19:00.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.951+0000 7f737ac00640 1 Processor -- start 2024-05-22T21:19:00.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.951+0000 7f737ac00640 1 -- start start 2024-05-22T21:19:00.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f737ac00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f737406c9b0 0x7f73740b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f737ac00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7374144bb0 con 0x7f7374143c40 2024-05-22T21:19:00.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f737ac00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f737415efb0 con 0x7f737415b600 2024-05-22T21:19:00.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f737ac00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737415ba60 con 0x7f737406c9b0 2024-05-22T21:19:00.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f7374143c40 0x7f73741444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51986/0 (socket says 172.21.15.161:51986) 2024-05-22T21:19:00.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373e00640 1 -- 172.21.15.161:0/1278033449 learned_addr learned my addr 172.21.15.161:0/1278033449 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:00.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7378c00640 1 --2- 172.21.15.161:0/1278033449 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f737406c9b0 0x7f73740b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373400640 1 -- 172.21.15.161:0/1278033449 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 943238135 0 0) 0x7f7374144bb0 con 0x7f7374143c40 2024-05-22T21:19:00.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373400640 1 -- 172.21.15.161:0/1278033449 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7358003530 con 0x7f7374143c40 2024-05-22T21:19:00.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373400640 1 -- 172.21.15.161:0/1278033449 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1834350228 0 0) 0x7f7358003530 con 0x7f7374143c40 2024-05-22T21:19:00.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373400640 1 -- 172.21.15.161:0/1278033449 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f737406c9b0 msgr2=0x7f73740b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:00.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373400640 1 --2- 172.21.15.161:0/1278033449 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f737406c9b0 0x7f73740b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373400640 1 -- 172.21.15.161:0/1278033449 >> v1:172.21.15.179:6789/0 conn(0x7f737415b600 legacy=0x7f737406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373400640 1 -- 172.21.15.161:0/1278033449 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737415f290 con 0x7f7374143c40 2024-05-22T21:19:00.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373400640 1 -- 172.21.15.161:0/1278033449 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f7364001a00 con 0x7f7374143c40 2024-05-22T21:19:00.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373400640 1 -- 172.21.15.161:0/1278033449 >> v1:172.21.15.161:6789/0 conn(0x7f7374143c40 legacy=0x7f73741444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373400640 1 --2- 172.21.15.161:0/1278033449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73580039b0 0x7f7358003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373400640 1 --2- 172.21.15.161:0/1278033449 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f737406c9b0 0x7f73580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373400640 1 --2- 172.21.15.161:0/1278033449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7358004d10 0x7f7358005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373400640 1 -- 172.21.15.161:0/1278033449 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737415ba60 con 0x7f73580039b0 2024-05-22T21:19:00.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373400640 1 -- 172.21.15.161:0/1278033449 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735800ba40 con 0x7f7358004d10 2024-05-22T21:19:00.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373400640 1 -- 172.21.15.161:0/1278033449 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735800bc00 con 0x7f737406c9b0 2024-05-22T21:19:00.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373e00640 1 --2- 172.21.15.161:0/1278033449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73580039b0 0x7f7358003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f736b400640 1 --2- 172.21.15.161:0/1278033449 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f737406c9b0 0x7f73580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7378c00640 1 --2- 172.21.15.161:0/1278033449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7358004d10 0x7f7358005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373e00640 1 -- 172.21.15.161:0/1278033449 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f737406c9b0 msgr2=0x7f73580047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:00.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373e00640 1 --2- 172.21.15.161:0/1278033449 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f737406c9b0 0x7f73580047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373e00640 1 -- 172.21.15.161:0/1278033449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7358004d10 msgr2=0x7f7358005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:00.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373e00640 1 --2- 172.21.15.161:0/1278033449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7358004d10 0x7f7358005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.955+0000 7f7373e00640 1 -- 172.21.15.161:0/1278033449 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7358003530 con 0x7f73580039b0 2024-05-22T21:19:00.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.959+0000 7f7373e00640 1 --2- 172.21.15.161:0/1278033449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73580039b0 0x7f7358003dd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f736400a800 tx=0x7f736401d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c37e674e3183412 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:00.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.959+0000 7f7373400640 1 -- 172.21.15.161:0/1278033449 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73640072d0 con 0x7f73580039b0 2024-05-22T21:19:00.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.959+0000 7f7373400640 1 -- 172.21.15.161:0/1278033449 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73640a8070 con 0x7f73580039b0 2024-05-22T21:19:00.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.959+0000 7f7373400640 1 -- 172.21.15.161:0/1278033449 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73640076b0 con 0x7f73580039b0 2024-05-22T21:19:00.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.959+0000 7f737ac00640 1 -- 172.21.15.161:0/1278033449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73580039b0 msgr2=0x7f7358003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.959+0000 7f737ac00640 1 --2- 172.21.15.161:0/1278033449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73580039b0 0x7f7358003dd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f736400a800 tx=0x7f736401d320 comp rx=0 tx=0).stop 2024-05-22T21:19:00.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.959+0000 7f7378c00640 1 -- 172.21.15.161:0/1278033449 reap_dead start 2024-05-22T21:19:00.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.959+0000 7f737ac00640 1 -- 172.21.15.161:0/1278033449 shutdown_connections 2024-05-22T21:19:00.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.959+0000 7f737ac00640 1 -- 172.21.15.161:0/1278033449 >> 172.21.15.161:0/1278033449 conn(0x7f737406a2d0 msgr2=0x7f737406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:00.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.959+0000 7f737ac00640 1 -- 172.21.15.161:0/1278033449 shutdown_connections 2024-05-22T21:19:00.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.959+0000 7f737ac00640 1 -- 172.21.15.161:0/1278033449 wait complete. 2024-05-22T21:19:00.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.959+0000 7f737ac00640 1 Processor -- start 2024-05-22T21:19:00.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f737ac00640 1 -- start start 2024-05-22T21:19:00.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f737ac00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73580039b0 0x7f7374137440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f737ac00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73740af8f0 0x7f73740afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f737ac00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73740b0330 0x7f737413eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f737ac00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73740b0840 con 0x7f73740b0330 2024-05-22T21:19:00.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f737ac00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73740aeb30 con 0x7f73580039b0 2024-05-22T21:19:00.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f737ac00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73740aee10 con 0x7f73740af8f0 2024-05-22T21:19:00.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f7378c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73740b0330 0x7f737413eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f7378c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73740b0330 0x7f737413eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51958/0 (socket says 172.21.15.161:51958) 2024-05-22T21:19:00.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f7378c00640 1 -- 172.21.15.161:0/1446533684 learned_addr learned my addr 172.21.15.161:0/1446533684 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:00.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f7373e00640 1 --2- 172.21.15.161:0/1446533684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73580039b0 0x7f7374137440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f736b400640 1 --2- 172.21.15.161:0/1446533684 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73740af8f0 0x7f73740afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f7378c00640 1 -- 172.21.15.161:0/1446533684 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73740af8f0 msgr2=0x7f73740afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f7378c00640 1 --2- 172.21.15.161:0/1446533684 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73740af8f0 0x7f73740afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f7378c00640 1 -- 172.21.15.161:0/1446533684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73580039b0 msgr2=0x7f7374137440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:00.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f7378c00640 1 --2- 172.21.15.161:0/1446533684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73580039b0 0x7f7374137440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:00.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f7378c00640 1 -- 172.21.15.161:0/1446533684 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73740af0f0 con 0x7f73740b0330 2024-05-22T21:19:00.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f7373e00640 1 --2- 172.21.15.161:0/1446533684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73580039b0 0x7f7374137440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:00.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f7378c00640 1 --2- 172.21.15.161:0/1446533684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73740b0330 0x7f737413eb50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f73740b4560 tx=0x7f736c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:00.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f7370c00640 1 -- 172.21.15.161:0/1446533684 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c004500 con 0x7f73740b0330 2024-05-22T21:19:00.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f7370c00640 1 -- 172.21.15.161:0/1446533684 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736c003620 con 0x7f73740b0330 2024-05-22T21:19:00.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f7370c00640 1 -- 172.21.15.161:0/1446533684 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c003a10 con 0x7f73740b0330 2024-05-22T21:19:00.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f737ac00640 1 -- 172.21.15.161:0/1446533684 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73740af3d0 con 0x7f73740b0330 2024-05-22T21:19:00.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f737ac00640 1 -- 172.21.15.161:0/1446533684 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7374153010 con 0x7f73740b0330 2024-05-22T21:19:00.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.963+0000 7f737ac00640 1 -- 172.21.15.161:0/1446533684 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7374067220 con 0x7f73740b0330 2024-05-22T21:19:00.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.967+0000 7f7370c00640 1 -- 172.21.15.161:0/1446533684 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f736c003b70 con 0x7f73740b0330 2024-05-22T21:19:00.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.967+0000 7f7370c00640 1 --2- 172.21.15.161:0/1446533684 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73500666c0 0x7f7350068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:00.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.967+0000 7f7370c00640 1 -- 172.21.15.161:0/1446533684 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f736c084920 con 0x7f73740b0330 2024-05-22T21:19:00.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.967+0000 7f7373e00640 1 --2- 172.21.15.161:0/1446533684 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73500666c0 0x7f7350068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:00.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.967+0000 7f7373e00640 1 --2- 172.21.15.161:0/1446533684 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73500666c0 0x7f7350068b80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f73640ad000 tx=0x7f7364009050 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:00.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:00.971+0000 7f7370c00640 1 -- 172.21.15.161:0/1446533684 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f736c051c70 con 0x7f73740b0330 2024-05-22T21:19:01.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.123+0000 7f737ac00640 1 -- 172.21.15.161:0/1446533684 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73740ceaa0 con 0x7f73740b0330 2024-05-22T21:19:01.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.123+0000 7f7370c00640 1 -- 172.21.15.161:0/1446533684 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f736c055290 con 0x7f73740b0330 2024-05-22T21:19:01.128 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:19:01.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.127+0000 7f737ac00640 1 -- 172.21.15.161:0/1446533684 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73500666c0 msgr2=0x7f7350068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.127+0000 7f737ac00640 1 --2- 172.21.15.161:0/1446533684 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73500666c0 0x7f7350068b80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f73640ad000 tx=0x7f7364009050 comp rx=0 tx=0).stop 2024-05-22T21:19:01.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.127+0000 7f737ac00640 1 -- 172.21.15.161:0/1446533684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73740b0330 msgr2=0x7f737413eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.127+0000 7f737ac00640 1 --2- 172.21.15.161:0/1446533684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73740b0330 0x7f737413eb50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f73740b4560 tx=0x7f736c011920 comp rx=0 tx=0).stop 2024-05-22T21:19:01.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.127+0000 7f737ac00640 1 -- 172.21.15.161:0/1446533684 shutdown_connections 2024-05-22T21:19:01.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.127+0000 7f737ac00640 1 --2- 172.21.15.161:0/1446533684 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73500666c0 0x7f7350068b80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.127+0000 7f737ac00640 1 --2- 172.21.15.161:0/1446533684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73740b0330 0x7f737413eb50 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.127+0000 7f737ac00640 1 --2- 172.21.15.161:0/1446533684 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73740af8f0 0x7f73740afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.127+0000 7f737ac00640 1 --2- 172.21.15.161:0/1446533684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73580039b0 0x7f7374137440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.127+0000 7f737ac00640 1 -- 172.21.15.161:0/1446533684 >> 172.21.15.161:0/1446533684 conn(0x7f737406a2d0 msgr2=0x7f7374139f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:01.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.127+0000 7f737ac00640 1 -- 172.21.15.161:0/1446533684 shutdown_connections 2024-05-22T21:19:01.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.127+0000 7f737ac00640 1 -- 172.21.15.161:0/1446533684 wait complete. 2024-05-22T21:19:01.149 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:19:01.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.283+0000 7f756f800640 1 Processor -- start 2024-05-22T21:19:01.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.283+0000 7f756f800640 1 -- start start 2024-05-22T21:19:01.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.287+0000 7f756f800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7568143c40 0x7f756815b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.287+0000 7f756f800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f756806c9b0 0x7f75680b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.287+0000 7f756f800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756815efb0 con 0x7f75681444a0 2024-05-22T21:19:01.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.287+0000 7f756f800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756815f290 con 0x7f756806c9b0 2024-05-22T21:19:01.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.287+0000 7f756f800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756815f570 con 0x7f7568143c40 2024-05-22T21:19:01.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.287+0000 7f756ce00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7568143c40 0x7f756815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.287+0000 7f756d800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f756806c9b0 0x7f75680b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.287+0000 7f756ce00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7568143c40 0x7f756815b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:46730/0 (socket says 172.21.15.161:46730) 2024-05-22T21:19:01.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.287+0000 7f756ce00640 1 -- 172.21.15.161:0/4124469503 learned_addr learned my addr 172.21.15.161:0/4124469503 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:01.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.287+0000 7f756ce00640 1 -- 172.21.15.161:0/4124469503 >> v2:172.21.15.179:3300/0 conn(0x7f756806c9b0 msgr2=0x7f75680b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:01.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.287+0000 7f756ce00640 1 --2- 172.21.15.161:0/4124469503 >> v2:172.21.15.179:3300/0 conn(0x7f756806c9b0 0x7f75680b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.287+0000 7f756ce00640 1 -- 172.21.15.161:0/4124469503 >> v1:172.21.15.179:6789/0 conn(0x7f75681444a0 legacy=0x7f756806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.287+0000 7f756ce00640 1 -- 172.21.15.161:0/4124469503 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756815f850 con 0x7f7568143c40 2024-05-22T21:19:01.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.287+0000 7f756d800640 1 --2- 172.21.15.161:0/4124469503 >> v2:172.21.15.179:3300/0 conn(0x7f756806c9b0 0x7f75680b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:01.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.291+0000 7f756ce00640 1 --2- 172.21.15.161:0/4124469503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7568143c40 0x7f756815b600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7558009e50 tx=0x7f755801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88fa457929e75806 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:01.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.291+0000 7f7567400640 1 -- 172.21.15.161:0/4124469503 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75580a5860 con 0x7f7568143c40 2024-05-22T21:19:01.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.291+0000 7f7567400640 1 -- 172.21.15.161:0/4124469503 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75580a59c0 con 0x7f7568143c40 2024-05-22T21:19:01.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.291+0000 7f7567400640 1 -- 172.21.15.161:0/4124469503 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75580a5d50 con 0x7f7568143c40 2024-05-22T21:19:01.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.291+0000 7f756f800640 1 -- 172.21.15.161:0/4124469503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7568143c40 msgr2=0x7f756815b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.291+0000 7f756f800640 1 --2- 172.21.15.161:0/4124469503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7568143c40 0x7f756815b600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7558009e50 tx=0x7f755801ca30 comp rx=0 tx=0).stop 2024-05-22T21:19:01.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.291+0000 7f756f800640 1 -- 172.21.15.161:0/4124469503 shutdown_connections 2024-05-22T21:19:01.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.291+0000 7f756f800640 1 --2- 172.21.15.161:0/4124469503 >> v2:172.21.15.179:3300/0 conn(0x7f756806c9b0 0x7f75680b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.291+0000 7f756f800640 1 --2- 172.21.15.161:0/4124469503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7568143c40 0x7f756815b600 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.291+0000 7f756f800640 1 -- 172.21.15.161:0/4124469503 >> 172.21.15.161:0/4124469503 conn(0x7f756806a2d0 msgr2=0x7f756806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:01.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.291+0000 7f756f800640 1 -- 172.21.15.161:0/4124469503 shutdown_connections 2024-05-22T21:19:01.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.291+0000 7f756f800640 1 -- 172.21.15.161:0/4124469503 wait complete. 2024-05-22T21:19:01.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.291+0000 7f756f800640 1 Processor -- start 2024-05-22T21:19:01.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.291+0000 7f756f800640 1 -- start start 2024-05-22T21:19:01.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756f800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f756806c9b0 0x7f7568139390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756f800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7568143c40 0x7f7568157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756f800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75681444a0 0x7f7568138b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756f800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7568154ee0 con 0x7f75681444a0 2024-05-22T21:19:01.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756f800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7568157a90 con 0x7f7568143c40 2024-05-22T21:19:01.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756f800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7568070280 con 0x7f756806c9b0 2024-05-22T21:19:01.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756d800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75681444a0 0x7f7568138b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756d800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75681444a0 0x7f7568138b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51974/0 (socket says 172.21.15.161:51974) 2024-05-22T21:19:01.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756d800640 1 -- 172.21.15.161:0/352770650 learned_addr learned my addr 172.21.15.161:0/352770650 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:01.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f7567e00640 1 --2- 172.21.15.161:0/352770650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7568143c40 0x7f7568157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756ce00640 1 --2- 172.21.15.161:0/352770650 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f756806c9b0 0x7f7568139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756d800640 1 -- 172.21.15.161:0/352770650 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f756806c9b0 msgr2=0x7f7568139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756d800640 1 --2- 172.21.15.161:0/352770650 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f756806c9b0 0x7f7568139390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756d800640 1 -- 172.21.15.161:0/352770650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7568143c40 msgr2=0x7f7568157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756d800640 1 --2- 172.21.15.161:0/352770650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7568143c40 0x7f7568157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756d800640 1 -- 172.21.15.161:0/352770650 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7560009420 con 0x7f75681444a0 2024-05-22T21:19:01.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756d800640 1 --2- 172.21.15.161:0/352770650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75681444a0 0x7f7568138b40 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f756000e950 tx=0x7f756000ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:01.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f7564c00640 1 -- 172.21.15.161:0/352770650 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560009720 con 0x7f75681444a0 2024-05-22T21:19:01.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f7564c00640 1 -- 172.21.15.161:0/352770650 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756000ca00 con 0x7f75681444a0 2024-05-22T21:19:01.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f7564c00640 1 -- 172.21.15.161:0/352770650 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756000cd90 con 0x7f75681444a0 2024-05-22T21:19:01.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756f800640 1 -- 172.21.15.161:0/352770650 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7558009b30 con 0x7f75681444a0 2024-05-22T21:19:01.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756f800640 1 -- 172.21.15.161:0/352770650 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7568070840 con 0x7f75681444a0 2024-05-22T21:19:01.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f756f800640 1 -- 172.21.15.161:0/352770650 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7568067220 con 0x7f75681444a0 2024-05-22T21:19:01.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.295+0000 7f7564c00640 1 -- 172.21.15.161:0/352770650 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7560013470 con 0x7f75681444a0 2024-05-22T21:19:01.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.299+0000 7f7564c00640 1 --2- 172.21.15.161:0/352770650 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7548066690 0x7f7548068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.299+0000 7f7564c00640 1 -- 172.21.15.161:0/352770650 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7560084160 con 0x7f75681444a0 2024-05-22T21:19:01.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.299+0000 7f756ce00640 1 --2- 172.21.15.161:0/352770650 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7548066690 0x7f7548068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.299+0000 7f756ce00640 1 --2- 172.21.15.161:0/352770650 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7548066690 0x7f7548068b50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f75580ae000 tx=0x7f75580a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:01.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.303+0000 7f7564c00640 1 -- 172.21.15.161:0/352770650 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f75600514e0 con 0x7f75681444a0 2024-05-22T21:19:01.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.459+0000 7f756f800640 1 -- 172.21.15.161:0/352770650 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7568155020 con 0x7f75681444a0 2024-05-22T21:19:01.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.459+0000 7f7564c00640 1 -- 172.21.15.161:0/352770650 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7560054b00 con 0x7f75681444a0 2024-05-22T21:19:01.463 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:19:01.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.463+0000 7f756f800640 1 -- 172.21.15.161:0/352770650 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7548066690 msgr2=0x7f7548068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.463+0000 7f756f800640 1 --2- 172.21.15.161:0/352770650 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7548066690 0x7f7548068b50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f75580ae000 tx=0x7f75580a7070 comp rx=0 tx=0).stop 2024-05-22T21:19:01.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.463+0000 7f756f800640 1 -- 172.21.15.161:0/352770650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75681444a0 msgr2=0x7f7568138b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.463+0000 7f756f800640 1 --2- 172.21.15.161:0/352770650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75681444a0 0x7f7568138b40 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f756000e950 tx=0x7f756000ee10 comp rx=0 tx=0).stop 2024-05-22T21:19:01.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.463+0000 7f756f800640 1 -- 172.21.15.161:0/352770650 shutdown_connections 2024-05-22T21:19:01.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.463+0000 7f756f800640 1 --2- 172.21.15.161:0/352770650 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7548066690 0x7f7548068b50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.463+0000 7f756f800640 1 --2- 172.21.15.161:0/352770650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75681444a0 0x7f7568138b40 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.463+0000 7f756f800640 1 --2- 172.21.15.161:0/352770650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7568143c40 0x7f7568157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.463+0000 7f756f800640 1 --2- 172.21.15.161:0/352770650 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f756806c9b0 0x7f7568139390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.463+0000 7f756f800640 1 -- 172.21.15.161:0/352770650 >> 172.21.15.161:0/352770650 conn(0x7f756806a2d0 msgr2=0x7f75680b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:01.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.463+0000 7f756f800640 1 -- 172.21.15.161:0/352770650 shutdown_connections 2024-05-22T21:19:01.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.463+0000 7f756f800640 1 -- 172.21.15.161:0/352770650 wait complete. 2024-05-22T21:19:01.484 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:19:01.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.619+0000 7fb10f000640 1 Processor -- start 2024-05-22T21:19:01.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.619+0000 7fb10f000640 1 -- start start 2024-05-22T21:19:01.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.619+0000 7fb10f000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb10813cfc0 0x7fb1081402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.619+0000 7fb10f000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb10815af60 0x7fb10813f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.619+0000 7fb10f000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb10813fc80 0x7fb10815dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.619+0000 7fb10f000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1081400d0 con 0x7fb10813cfc0 2024-05-22T21:19:01.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.619+0000 7fb10f000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10813d3a0 con 0x7fb10813fc80 2024-05-22T21:19:01.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.619+0000 7fb10f000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1080cd920 con 0x7fb10815af60 2024-05-22T21:19:01.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.623+0000 7fb107e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb10813cfc0 0x7fb1081402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.623+0000 7fb107e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb10813cfc0 0x7fb1081402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51978/0 (socket says 172.21.15.161:51978) 2024-05-22T21:19:01.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.623+0000 7fb107e00640 1 -- 172.21.15.161:0/1308595912 learned_addr learned my addr 172.21.15.161:0/1308595912 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:01.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.623+0000 7fb0ff400640 1 --2- 172.21.15.161:0/1308595912 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb10815af60 0x7fb10813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.623+0000 7fb10d000640 1 --2- 172.21.15.161:0/1308595912 >> v2:172.21.15.179:3300/0 conn(0x7fb10813fc80 0x7fb10815dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.623+0000 7fb107e00640 1 -- 172.21.15.161:0/1308595912 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb10815af60 msgr2=0x7fb10813f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:01.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.623+0000 7fb107e00640 1 --2- 172.21.15.161:0/1308595912 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb10815af60 0x7fb10813f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.623+0000 7fb107e00640 1 -- 172.21.15.161:0/1308595912 >> v2:172.21.15.179:3300/0 conn(0x7fb10813fc80 msgr2=0x7fb10815dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:01.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.623+0000 7fb107e00640 1 --2- 172.21.15.161:0/1308595912 >> v2:172.21.15.179:3300/0 conn(0x7fb10813fc80 0x7fb10815dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.623+0000 7fb107e00640 1 -- 172.21.15.161:0/1308595912 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1080cdc00 con 0x7fb10813cfc0 2024-05-22T21:19:01.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.623+0000 7fb10d000640 1 --2- 172.21.15.161:0/1308595912 >> v2:172.21.15.179:3300/0 conn(0x7fb10813fc80 0x7fb10815dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:01.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.623+0000 7fb107e00640 1 --2- 172.21.15.161:0/1308595912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10813cfc0 0x7fb1081402e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb10806de40 tx=0x7fb0f801c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719b1be5361dfbf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:01.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.623+0000 7fb107400640 1 -- 172.21.15.161:0/1308595912 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80ab600 con 0x7fb10813cfc0 2024-05-22T21:19:01.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.623+0000 7fb107400640 1 -- 172.21.15.161:0/1308595912 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f80a7050 con 0x7fb10813cfc0 2024-05-22T21:19:01.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.623+0000 7fb107400640 1 -- 172.21.15.161:0/1308595912 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80ab980 con 0x7fb10813cfc0 2024-05-22T21:19:01.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 -- 172.21.15.161:0/1308595912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10813cfc0 msgr2=0x7fb1081402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 --2- 172.21.15.161:0/1308595912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10813cfc0 0x7fb1081402e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb10806de40 tx=0x7fb0f801c1a0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 -- 172.21.15.161:0/1308595912 shutdown_connections 2024-05-22T21:19:01.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 --2- 172.21.15.161:0/1308595912 >> v2:172.21.15.179:3300/0 conn(0x7fb10813fc80 0x7fb10815dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 --2- 172.21.15.161:0/1308595912 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb10815af60 0x7fb10813f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 --2- 172.21.15.161:0/1308595912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10813cfc0 0x7fb1081402e0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 -- 172.21.15.161:0/1308595912 >> 172.21.15.161:0/1308595912 conn(0x7fb10806a2d0 msgr2=0x7fb10806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:01.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 -- 172.21.15.161:0/1308595912 shutdown_connections 2024-05-22T21:19:01.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 -- 172.21.15.161:0/1308595912 wait complete. 2024-05-22T21:19:01.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 Processor -- start 2024-05-22T21:19:01.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 -- start start 2024-05-22T21:19:01.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb10813cfc0 0x7fb1081572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb10813fc80 0x7fb10815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10815af60 0x7fb108140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb108157830 con 0x7fb10815af60 2024-05-22T21:19:01.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10813d510 con 0x7fb10813fc80 2024-05-22T21:19:01.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10f000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10813e370 con 0x7fb10813cfc0 2024-05-22T21:19:01.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10815af60 0x7fb108140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10815af60 0x7fb108140270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51980/0 (socket says 172.21.15.161:51980) 2024-05-22T21:19:01.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10d000640 1 -- 172.21.15.161:0/4196737156 learned_addr learned my addr 172.21.15.161:0/4196737156 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:01.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb107e00640 1 --2- 172.21.15.161:0/4196737156 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb10813cfc0 0x7fb1081572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb0ff400640 1 --2- 172.21.15.161:0/4196737156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb10813fc80 0x7fb10815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10d000640 1 -- 172.21.15.161:0/4196737156 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb10813cfc0 msgr2=0x7fb1081572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10d000640 1 --2- 172.21.15.161:0/4196737156 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb10813cfc0 0x7fb1081572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10d000640 1 -- 172.21.15.161:0/4196737156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb10813fc80 msgr2=0x7fb10815cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10d000640 1 --2- 172.21.15.161:0/4196737156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb10813fc80 0x7fb10815cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.627+0000 7fb10d000640 1 -- 172.21.15.161:0/4196737156 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f80094d0 con 0x7fb10815af60 2024-05-22T21:19:01.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.631+0000 7fb10d000640 1 --2- 172.21.15.161:0/4196737156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10815af60 0x7fb108140270 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb100007ea0 tx=0x7fb100007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:01.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.631+0000 7fb104c00640 1 -- 172.21.15.161:0/4196737156 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb100003ba0 con 0x7fb10815af60 2024-05-22T21:19:01.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.631+0000 7fb104c00640 1 -- 172.21.15.161:0/4196737156 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10000f040 con 0x7fb10815af60 2024-05-22T21:19:01.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.631+0000 7fb10f000640 1 -- 172.21.15.161:0/4196737156 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10813d760 con 0x7fb10815af60 2024-05-22T21:19:01.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.631+0000 7fb104c00640 1 -- 172.21.15.161:0/4196737156 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10000de70 con 0x7fb10815af60 2024-05-22T21:19:01.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.631+0000 7fb10f000640 1 -- 172.21.15.161:0/4196737156 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10813dd20 con 0x7fb10815af60 2024-05-22T21:19:01.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.631+0000 7fb10f000640 1 -- 172.21.15.161:0/4196737156 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10815dde0 con 0x7fb10815af60 2024-05-22T21:19:01.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.631+0000 7fb104c00640 1 -- 172.21.15.161:0/4196737156 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb100004030 con 0x7fb10815af60 2024-05-22T21:19:01.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.631+0000 7fb104c00640 1 --2- 172.21.15.161:0/4196737156 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0e4066690 0x7fb0e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.631+0000 7fb104c00640 1 -- 172.21.15.161:0/4196737156 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb100084480 con 0x7fb10815af60 2024-05-22T21:19:01.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.631+0000 7fb107e00640 1 --2- 172.21.15.161:0/4196737156 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0e4066690 0x7fb0e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.635+0000 7fb107e00640 1 --2- 172.21.15.161:0/4196737156 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0e4066690 0x7fb0e4068b50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb0f80ad000 tx=0x7fb0f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:01.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.635+0000 7fb104c00640 1 -- 172.21.15.161:0/4196737156 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb100051800 con 0x7fb10815af60 2024-05-22T21:19:01.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.787+0000 7fb10f000640 1 -- 172.21.15.161:0/4196737156 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb108067220 con 0x7fb10815af60 2024-05-22T21:19:01.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.787+0000 7fb104c00640 1 -- 172.21.15.161:0/4196737156 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb100054e20 con 0x7fb10815af60 2024-05-22T21:19:01.791 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:19:01.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.791+0000 7fb10f000640 1 -- 172.21.15.161:0/4196737156 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0e4066690 msgr2=0x7fb0e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.791+0000 7fb10f000640 1 --2- 172.21.15.161:0/4196737156 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0e4066690 0x7fb0e4068b50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb0f80ad000 tx=0x7fb0f8008040 comp rx=0 tx=0).stop 2024-05-22T21:19:01.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.791+0000 7fb10f000640 1 -- 172.21.15.161:0/4196737156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10815af60 msgr2=0x7fb108140270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.791+0000 7fb10f000640 1 --2- 172.21.15.161:0/4196737156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10815af60 0x7fb108140270 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb100007ea0 tx=0x7fb100007f90 comp rx=0 tx=0).stop 2024-05-22T21:19:01.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.791+0000 7fb10f000640 1 -- 172.21.15.161:0/4196737156 shutdown_connections 2024-05-22T21:19:01.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.791+0000 7fb10f000640 1 --2- 172.21.15.161:0/4196737156 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb0e4066690 0x7fb0e4068b50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.791+0000 7fb10f000640 1 --2- 172.21.15.161:0/4196737156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10815af60 0x7fb108140270 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.791+0000 7fb10f000640 1 --2- 172.21.15.161:0/4196737156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb10813fc80 0x7fb10815cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.791+0000 7fb10f000640 1 --2- 172.21.15.161:0/4196737156 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb10813cfc0 0x7fb1081572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.791+0000 7fb10f000640 1 -- 172.21.15.161:0/4196737156 >> 172.21.15.161:0/4196737156 conn(0x7fb10806a2d0 msgr2=0x7fb1080ad260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:01.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.791+0000 7fb10f000640 1 -- 172.21.15.161:0/4196737156 shutdown_connections 2024-05-22T21:19:01.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.791+0000 7fb10f000640 1 -- 172.21.15.161:0/4196737156 wait complete. 2024-05-22T21:19:01.813 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:01.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.943+0000 7f3438c00640 1 Processor -- start 2024-05-22T21:19:01.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.943+0000 7f3438c00640 1 -- start start 2024-05-22T21:19:01.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3438c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f343415c6d0 0x7f3434164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3438c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3434071670 con 0x7f343415e130 2024-05-22T21:19:01.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3438c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3434070a60 con 0x7f343415dd50 2024-05-22T21:19:01.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3438c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3434070d40 con 0x7f343415c6d0 2024-05-22T21:19:01.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3429400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f343415e130 0x7f343415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51994/0 (socket says 172.21.15.161:51994) 2024-05-22T21:19:01.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3429400640 1 -- 172.21.15.161:0/2656173831 learned_addr learned my addr 172.21.15.161:0/2656173831 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:01.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3432800640 1 --2- 172.21.15.161:0/2656173831 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f343415c6d0 0x7f3434164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431400640 1 -- 172.21.15.161:0/2656173831 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443126841 0 0) 0x7f3434071670 con 0x7f343415e130 2024-05-22T21:19:01.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431400640 1 -- 172.21.15.161:0/2656173831 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3418003530 con 0x7f343415e130 2024-05-22T21:19:01.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431400640 1 -- 172.21.15.161:0/2656173831 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1624957288 0 0) 0x7f3418003530 con 0x7f343415e130 2024-05-22T21:19:01.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431400640 1 -- 172.21.15.161:0/2656173831 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f343415c6d0 msgr2=0x7f3434164e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:01.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431400640 1 --2- 172.21.15.161:0/2656173831 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f343415c6d0 0x7f3434164e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431400640 1 -- 172.21.15.161:0/2656173831 >> v1:172.21.15.179:6789/0 conn(0x7f343415dd50 legacy=0x7f343406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431400640 1 -- 172.21.15.161:0/2656173831 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3434071020 con 0x7f343415e130 2024-05-22T21:19:01.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431400640 1 -- 172.21.15.161:0/2656173831 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f341c001680 con 0x7f343415e130 2024-05-22T21:19:01.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3432800640 1 --2- 172.21.15.161:0/2656173831 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f343415c6d0 0x7f3434164e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:01.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431400640 1 -- 172.21.15.161:0/2656173831 >> v1:172.21.15.161:6789/0 conn(0x7f343415e130 legacy=0x7f343415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431400640 1 --2- 172.21.15.161:0/2656173831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34180039b0 0x7f3418003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431400640 1 --2- 172.21.15.161:0/2656173831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3418004360 0x7f3418004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431400640 1 --2- 172.21.15.161:0/2656173831 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f343415c6d0 0x7f3418005050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431400640 1 -- 172.21.15.161:0/2656173831 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3434070d40 con 0x7f34180039b0 2024-05-22T21:19:01.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431400640 1 -- 172.21.15.161:0/2656173831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341800dba0 con 0x7f3418004360 2024-05-22T21:19:01.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431400640 1 -- 172.21.15.161:0/2656173831 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341800dd10 con 0x7f343415c6d0 2024-05-22T21:19:01.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431e00640 1 --2- 172.21.15.161:0/2656173831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34180039b0 0x7f3418003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3432800640 1 --2- 172.21.15.161:0/2656173831 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f343415c6d0 0x7f3418005050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3429400640 1 --2- 172.21.15.161:0/2656173831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3418004360 0x7f3418004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431e00640 1 -- 172.21.15.161:0/2656173831 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f343415c6d0 msgr2=0x7f3418005050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:01.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431e00640 1 --2- 172.21.15.161:0/2656173831 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f343415c6d0 0x7f3418005050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431e00640 1 -- 172.21.15.161:0/2656173831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3418004360 msgr2=0x7f3418004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:01.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431e00640 1 --2- 172.21.15.161:0/2656173831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3418004360 0x7f3418004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.947+0000 7f3431e00640 1 -- 172.21.15.161:0/2656173831 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c009420 con 0x7f34180039b0 2024-05-22T21:19:01.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.951+0000 7f3431e00640 1 --2- 172.21.15.161:0/2656173831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34180039b0 0x7f3418003d90 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3424001f60 tx=0x7f342401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b6eb0e2129dd09 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:01.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.951+0000 7f3431400640 1 -- 172.21.15.161:0/2656173831 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34240a6610 con 0x7f34180039b0 2024-05-22T21:19:01.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.951+0000 7f3431400640 1 -- 172.21.15.161:0/2656173831 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34240a6770 con 0x7f34180039b0 2024-05-22T21:19:01.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.951+0000 7f3431400640 1 -- 172.21.15.161:0/2656173831 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34240a6b20 con 0x7f34180039b0 2024-05-22T21:19:01.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.951+0000 7f3438c00640 1 -- 172.21.15.161:0/2656173831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34180039b0 msgr2=0x7f3418003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.951+0000 7f3438c00640 1 --2- 172.21.15.161:0/2656173831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34180039b0 0x7f3418003d90 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3424001f60 tx=0x7f342401c870 comp rx=0 tx=0).stop 2024-05-22T21:19:01.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.951+0000 7f3432800640 1 -- 172.21.15.161:0/2656173831 reap_dead start 2024-05-22T21:19:01.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.951+0000 7f3438c00640 1 -- 172.21.15.161:0/2656173831 shutdown_connections 2024-05-22T21:19:01.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.951+0000 7f3438c00640 1 -- 172.21.15.161:0/2656173831 >> 172.21.15.161:0/2656173831 conn(0x7f34340b0bc0 msgr2=0x7f34340b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:01.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.951+0000 7f3438c00640 1 -- 172.21.15.161:0/2656173831 shutdown_connections 2024-05-22T21:19:01.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.951+0000 7f3438c00640 1 -- 172.21.15.161:0/2656173831 wait complete. 2024-05-22T21:19:01.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3438c00640 1 Processor -- start 2024-05-22T21:19:01.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3438c00640 1 -- start start 2024-05-22T21:19:01.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3438c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34180039b0 0x7f3434150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3438c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3434158a20 0x7f3434158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3438c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34341593e0 0x7f34340b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3438c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3434159860 con 0x7f3434158a20 2024-05-22T21:19:01.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3438c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34340cca10 con 0x7f34180039b0 2024-05-22T21:19:01.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3438c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34340cccf0 con 0x7f34341593e0 2024-05-22T21:19:01.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3429400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3434158a20 0x7f3434158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3429400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3434158a20 0x7f3434158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51988/0 (socket says 172.21.15.161:51988) 2024-05-22T21:19:01.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3429400640 1 -- 172.21.15.161:0/3914256063 learned_addr learned my addr 172.21.15.161:0/3914256063 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:01.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3431e00640 1 --2- 172.21.15.161:0/3914256063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34180039b0 0x7f3434150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3432800640 1 --2- 172.21.15.161:0/3914256063 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34341593e0 0x7f34340b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3429400640 1 -- 172.21.15.161:0/3914256063 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34341593e0 msgr2=0x7f34340b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3429400640 1 --2- 172.21.15.161:0/3914256063 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34341593e0 0x7f34340b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3429400640 1 -- 172.21.15.161:0/3914256063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34180039b0 msgr2=0x7f3434150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:01.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3429400640 1 --2- 172.21.15.161:0/3914256063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34180039b0 0x7f3434150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:01.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3429400640 1 -- 172.21.15.161:0/3914256063 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34340ccfd0 con 0x7f3434158a20 2024-05-22T21:19:01.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3429400640 1 --2- 172.21.15.161:0/3914256063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3434158a20 0x7f3434158ea0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f341c0011f0 tx=0x7f341c007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:01.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f342aa00640 1 -- 172.21.15.161:0/3914256063 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c00f9c0 con 0x7f3434158a20 2024-05-22T21:19:01.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f342aa00640 1 -- 172.21.15.161:0/3914256063 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c00fb60 con 0x7f3434158a20 2024-05-22T21:19:01.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f342aa00640 1 -- 172.21.15.161:0/3914256063 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c0133f0 con 0x7f3434158a20 2024-05-22T21:19:01.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3438c00640 1 -- 172.21.15.161:0/3914256063 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34340cd2b0 con 0x7f3434158a20 2024-05-22T21:19:01.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3438c00640 1 -- 172.21.15.161:0/3914256063 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34340cc630 con 0x7f3434158a20 2024-05-22T21:19:01.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.955+0000 7f3438c00640 1 -- 172.21.15.161:0/3914256063 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34340cd440 con 0x7f3434158a20 2024-05-22T21:19:01.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.959+0000 7f342aa00640 1 -- 172.21.15.161:0/3914256063 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f341c013550 con 0x7f3434158a20 2024-05-22T21:19:01.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.959+0000 7f342aa00640 1 --2- 172.21.15.161:0/3914256063 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3410066690 0x7f3410068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:01.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.959+0000 7f342aa00640 1 -- 172.21.15.161:0/3914256063 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f341c0841f0 con 0x7f3434158a20 2024-05-22T21:19:01.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.959+0000 7f3431e00640 1 --2- 172.21.15.161:0/3914256063 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3410066690 0x7f3410068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:01.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.959+0000 7f3431e00640 1 --2- 172.21.15.161:0/3914256063 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3410066690 0x7f3410068b50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f34240ae000 tx=0x7f34240b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:01.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:01.963+0000 7f342aa00640 1 -- 172.21.15.161:0/3914256063 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f341c051600 con 0x7f3434158a20 2024-05-22T21:19:02.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.147+0000 7f3438c00640 1 -- 172.21.15.161:0/3914256063 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3434067220 con 0x7f3434158a20 2024-05-22T21:19:02.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.151+0000 7f342aa00640 1 -- 172.21.15.161:0/3914256063 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f341c054c20 con 0x7f3434158a20 2024-05-22T21:19:02.152 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:02.152 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2771409351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2771409351},{"type":"v1","addr":"172.21.15.179:6834","nonce":2771409351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6442,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2520119431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2520119431},{"type":"v1","addr":"172.21.15.179:6835","nonce":2520119431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1415362547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1415362547},{"type":"v1","addr":"172.21.15.161:6835","nonce":1415362547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:18:17.938858+0000","modified":"2024-05-22T21:18:20.963065+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/4127130864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4127130864},{"type":"v1","addr":"172.21.15.161:6837","nonce":4127130864}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-22T21:19:02.153 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 47 2024-05-22T21:19:02.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.151+0000 7f3438c00640 1 -- 172.21.15.161:0/3914256063 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3410066690 msgr2=0x7f3410068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:02.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.151+0000 7f3438c00640 1 --2- 172.21.15.161:0/3914256063 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3410066690 0x7f3410068b50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f34240ae000 tx=0x7f34240b3040 comp rx=0 tx=0).stop 2024-05-22T21:19:02.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.151+0000 7f3438c00640 1 -- 172.21.15.161:0/3914256063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3434158a20 msgr2=0x7f3434158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:02.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.151+0000 7f3438c00640 1 --2- 172.21.15.161:0/3914256063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3434158a20 0x7f3434158ea0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f341c0011f0 tx=0x7f341c007720 comp rx=0 tx=0).stop 2024-05-22T21:19:02.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.151+0000 7f3438c00640 1 -- 172.21.15.161:0/3914256063 shutdown_connections 2024-05-22T21:19:02.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.151+0000 7f3438c00640 1 --2- 172.21.15.161:0/3914256063 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3410066690 0x7f3410068b50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.151+0000 7f3438c00640 1 --2- 172.21.15.161:0/3914256063 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34341593e0 0x7f34340b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.151+0000 7f3438c00640 1 --2- 172.21.15.161:0/3914256063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3434158a20 0x7f3434158ea0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.151+0000 7f3438c00640 1 --2- 172.21.15.161:0/3914256063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34180039b0 0x7f3434150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.151+0000 7f3438c00640 1 -- 172.21.15.161:0/3914256063 >> 172.21.15.161:0/3914256063 conn(0x7f34340b0bc0 msgr2=0x7f34340aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:02.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.151+0000 7f3438c00640 1 -- 172.21.15.161:0/3914256063 shutdown_connections 2024-05-22T21:19:02.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.151+0000 7f3438c00640 1 -- 172.21.15.161:0/3914256063 wait complete. 2024-05-22T21:19:02.174 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:02.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd392200640 1 Processor -- start 2024-05-22T21:19:02.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd392200640 1 -- start start 2024-05-22T21:19:02.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd392200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd38c070db0 0x7fd38c0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:02.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd392200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd38c0b2ea0 0x7fd38c0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:02.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd392200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c159bd0 con 0x7fd38c1597f0 2024-05-22T21:19:02.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd392200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c0cbdd0 con 0x7fd38c070db0 2024-05-22T21:19:02.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd392200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0cc050 con 0x7fd38c0b2ea0 2024-05-22T21:19:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd38b400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd38c1597f0 0x7fd38c0706a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51996/0 (socket says 172.21.15.161:51996) 2024-05-22T21:19:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd38b400640 1 -- 172.21.15.161:0/1531521456 learned_addr learned my addr 172.21.15.161:0/1531521456 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd38aa00640 1 --2- 172.21.15.161:0/1531521456 >> v2:172.21.15.179:3300/0 conn(0x7fd38c070db0 0x7fd38c0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd38be00640 1 --2- 172.21.15.161:0/1531521456 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd38c0b2ea0 0x7fd38c0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd38a000640 1 -- 172.21.15.161:0/1531521456 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466631101 0 0) 0x7fd38c159bd0 con 0x7fd38c1597f0 2024-05-22T21:19:02.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd38a000640 1 -- 172.21.15.161:0/1531521456 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd370005740 con 0x7fd38c1597f0 2024-05-22T21:19:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd38a000640 1 -- 172.21.15.161:0/1531521456 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2373920521 0 0) 0x7fd370005740 con 0x7fd38c1597f0 2024-05-22T21:19:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd38a000640 1 -- 172.21.15.161:0/1531521456 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd38c0b2ea0 msgr2=0x7fd38c0cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd38a000640 1 --2- 172.21.15.161:0/1531521456 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd38c0b2ea0 0x7fd38c0cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd38a000640 1 -- 172.21.15.161:0/1531521456 >> v2:172.21.15.179:3300/0 conn(0x7fd38c070db0 msgr2=0x7fd38c0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd38a000640 1 --2- 172.21.15.161:0/1531521456 >> v2:172.21.15.179:3300/0 conn(0x7fd38c070db0 0x7fd38c0b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd38a000640 1 -- 172.21.15.161:0/1531521456 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c0b3720 con 0x7fd38c1597f0 2024-05-22T21:19:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd38a000640 1 -- 172.21.15.161:0/1531521456 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fd37c001a00 con 0x7fd38c1597f0 2024-05-22T21:19:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.307+0000 7fd38a000640 1 -- 172.21.15.161:0/1531521456 >> v1:172.21.15.161:6789/0 conn(0x7fd38c1597f0 legacy=0x7fd38c0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38a000640 1 --2- 172.21.15.161:0/1531521456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370005b90 0x7fd370005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38a000640 1 --2- 172.21.15.161:0/1531521456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3700069e0 0x7fd370006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38a000640 1 --2- 172.21.15.161:0/1531521456 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd38c0b2ea0 0x7fd3700073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:02.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38a000640 1 -- 172.21.15.161:0/1531521456 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0cbdd0 con 0x7fd370005b90 2024-05-22T21:19:02.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38a000640 1 -- 172.21.15.161:0/1531521456 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0cc050 con 0x7fd3700069e0 2024-05-22T21:19:02.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38a000640 1 -- 172.21.15.161:0/1531521456 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37000dba0 con 0x7fd38c0b2ea0 2024-05-22T21:19:02.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38aa00640 1 --2- 172.21.15.161:0/1531521456 >> v2:172.21.15.179:3300/0 conn(0x7fd38c070db0 0x7fd38c0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:02.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38b400640 1 --2- 172.21.15.161:0/1531521456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370005b90 0x7fd370005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:02.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38be00640 1 --2- 172.21.15.161:0/1531521456 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd38c0b2ea0 0x7fd3700073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:02.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38aa00640 1 --2- 172.21.15.161:0/1531521456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3700069e0 0x7fd370006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:02.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38b400640 1 -- 172.21.15.161:0/1531521456 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd38c0b2ea0 msgr2=0x7fd3700073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:02.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38b400640 1 --2- 172.21.15.161:0/1531521456 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd38c0b2ea0 0x7fd3700073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38b400640 1 -- 172.21.15.161:0/1531521456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3700069e0 msgr2=0x7fd370006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:02.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38b400640 1 --2- 172.21.15.161:0/1531521456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3700069e0 0x7fd370006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38b400640 1 -- 172.21.15.161:0/1531521456 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd370005740 con 0x7fd370005b90 2024-05-22T21:19:02.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38be00640 1 --2- 172.21.15.161:0/1531521456 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd38c0b2ea0 0x7fd3700073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:02.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38b400640 1 --2- 172.21.15.161:0/1531521456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370005b90 0x7fd370005fb0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00a860 tx=0x7fd37c01d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850c3257b34ba7cf server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:02.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38a000640 1 -- 172.21.15.161:0/1531521456 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0ad630 con 0x7fd370005b90 2024-05-22T21:19:02.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38a000640 1 -- 172.21.15.161:0/1531521456 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c0a8070 con 0x7fd370005b90 2024-05-22T21:19:02.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.311+0000 7fd38a000640 1 -- 172.21.15.161:0/1531521456 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0ad9f0 con 0x7fd370005b90 2024-05-22T21:19:02.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd392200640 1 -- 172.21.15.161:0/1531521456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370005b90 msgr2=0x7fd370005fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:02.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd392200640 1 --2- 172.21.15.161:0/1531521456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370005b90 0x7fd370005fb0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00a860 tx=0x7fd37c01d380 comp rx=0 tx=0).stop 2024-05-22T21:19:02.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd38be00640 1 -- 172.21.15.161:0/1531521456 reap_dead start 2024-05-22T21:19:02.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd392200640 1 -- 172.21.15.161:0/1531521456 shutdown_connections 2024-05-22T21:19:02.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd392200640 1 -- 172.21.15.161:0/1531521456 >> 172.21.15.161:0/1531521456 conn(0x7fd38c0b0bc0 msgr2=0x7fd38c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:02.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd392200640 1 -- 172.21.15.161:0/1531521456 shutdown_connections 2024-05-22T21:19:02.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd392200640 1 -- 172.21.15.161:0/1531521456 wait complete. 2024-05-22T21:19:02.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd392200640 1 Processor -- start 2024-05-22T21:19:02.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd392200640 1 -- start start 2024-05-22T21:19:02.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd392200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370005b90 0x7fd38c15b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:02.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd392200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd38c1fa000 0x7fd38c1ebb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:02.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd392200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c1f9340 0x7fd38c1f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:02.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd392200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c1e9d90 con 0x7fd370005b90 2024-05-22T21:19:02.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd392200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c15bb10 con 0x7fd38c1f9340 2024-05-22T21:19:02.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd392200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c1f9d00 con 0x7fd38c1fa000 2024-05-22T21:19:02.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd38b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370005b90 0x7fd38c15b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:02.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd38b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370005b90 0x7fd38c15b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:51996/0 (socket says 172.21.15.161:51996) 2024-05-22T21:19:02.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd38b400640 1 -- 172.21.15.161:0/3502571372 learned_addr learned my addr 172.21.15.161:0/3502571372 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:02.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd38aa00640 1 --2- 172.21.15.161:0/3502571372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd38c1fa000 0x7fd38c1ebb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:02.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd38be00640 1 --2- 172.21.15.161:0/3502571372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c1f9340 0x7fd38c1f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:02.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd38b400640 1 -- 172.21.15.161:0/3502571372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd38c1fa000 msgr2=0x7fd38c1ebb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:02.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd38b400640 1 --2- 172.21.15.161:0/3502571372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd38c1fa000 0x7fd38c1ebb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd38b400640 1 -- 172.21.15.161:0/3502571372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c1f9340 msgr2=0x7fd38c1f97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:02.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd38b400640 1 --2- 172.21.15.161:0/3502571372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c1f9340 0x7fd38c1f97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd38b400640 1 -- 172.21.15.161:0/3502571372 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c1e9110 con 0x7fd370005b90 2024-05-22T21:19:02.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd38be00640 1 --2- 172.21.15.161:0/3502571372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c1f9340 0x7fd38c1f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:02.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.315+0000 7fd38b400640 1 --2- 172.21.15.161:0/3502571372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370005b90 0x7fd38c15b5d0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00a860 tx=0x7fd370007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:02.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.319+0000 7fd383400640 1 -- 172.21.15.161:0/3502571372 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c001980 con 0x7fd370005b90 2024-05-22T21:19:02.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.319+0000 7fd383400640 1 -- 172.21.15.161:0/3502571372 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c004b70 con 0x7fd370005b90 2024-05-22T21:19:02.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.319+0000 7fd383400640 1 -- 172.21.15.161:0/3502571372 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0b7460 con 0x7fd370005b90 2024-05-22T21:19:02.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.319+0000 7fd392200640 1 -- 172.21.15.161:0/3502571372 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c1e93f0 con 0x7fd370005b90 2024-05-22T21:19:02.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.319+0000 7fd392200640 1 -- 172.21.15.161:0/3502571372 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c1e99b0 con 0x7fd370005b90 2024-05-22T21:19:02.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.319+0000 7fd392200640 1 -- 172.21.15.161:0/3502571372 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c067220 con 0x7fd370005b90 2024-05-22T21:19:02.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.319+0000 7fd383400640 1 -- 172.21.15.161:0/3502571372 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd37c0aa030 con 0x7fd370005b90 2024-05-22T21:19:02.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.319+0000 7fd383400640 1 --2- 172.21.15.161:0/3502571372 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd36c066690 0x7fd36c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:02.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.319+0000 7fd383400640 1 -- 172.21.15.161:0/3502571372 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd37c11e120 con 0x7fd370005b90 2024-05-22T21:19:02.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.319+0000 7fd38aa00640 1 --2- 172.21.15.161:0/3502571372 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd36c066690 0x7fd36c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:02.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.323+0000 7fd38aa00640 1 --2- 172.21.15.161:0/3502571372 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd36c066690 0x7fd36c068b50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0b2b10 tx=0x7fd378004040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:02.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.323+0000 7fd383400640 1 -- 172.21.15.161:0/3502571372 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd37c0eb530 con 0x7fd370005b90 2024-05-22T21:19:02.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.483+0000 7fd392200640 1 -- 172.21.15.161:0/3502571372 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd38c06dc30 con 0x7fd370005b90 2024-05-22T21:19:02.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.483+0000 7fd383400640 1 -- 172.21.15.161:0/3502571372 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16152 (secure 0 0 0) 0x7fd37c0eeb50 con 0x7fd370005b90 2024-05-22T21:19:02.486 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:02.486 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":113,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:18:50.632514+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:18:16.500534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-22T21:18:16.937445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:02.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.483+0000 7fd392200640 1 -- 172.21.15.161:0/3502571372 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd36c066690 msgr2=0x7fd36c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:02.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.483+0000 7fd392200640 1 --2- 172.21.15.161:0/3502571372 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd36c066690 0x7fd36c068b50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0b2b10 tx=0x7fd378004040 comp rx=0 tx=0).stop 2024-05-22T21:19:02.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.483+0000 7fd392200640 1 -- 172.21.15.161:0/3502571372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370005b90 msgr2=0x7fd38c15b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:02.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.483+0000 7fd392200640 1 --2- 172.21.15.161:0/3502571372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370005b90 0x7fd38c15b5d0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00a860 tx=0x7fd370007a10 comp rx=0 tx=0).stop 2024-05-22T21:19:02.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.483+0000 7fd392200640 1 -- 172.21.15.161:0/3502571372 shutdown_connections 2024-05-22T21:19:02.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.483+0000 7fd392200640 1 --2- 172.21.15.161:0/3502571372 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd36c066690 0x7fd36c068b50 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.483+0000 7fd392200640 1 --2- 172.21.15.161:0/3502571372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd38c1f9340 0x7fd38c1f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.483+0000 7fd392200640 1 --2- 172.21.15.161:0/3502571372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd38c1fa000 0x7fd38c1ebb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.483+0000 7fd392200640 1 --2- 172.21.15.161:0/3502571372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd370005b90 0x7fd38c15b5d0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.483+0000 7fd392200640 1 -- 172.21.15.161:0/3502571372 >> 172.21.15.161:0/3502571372 conn(0x7fd38c0b0bc0 msgr2=0x7fd38c150200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:02.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.483+0000 7fd392200640 1 -- 172.21.15.161:0/3502571372 shutdown_connections 2024-05-22T21:19:02.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.483+0000 7fd392200640 1 -- 172.21.15.161:0/3502571372 wait complete. 2024-05-22T21:19:02.507 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-22T21:19:02.507 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:02.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.639+0000 7f2d57a00640 1 Processor -- start 2024-05-22T21:19:02.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.639+0000 7f2d57a00640 1 -- start start 2024-05-22T21:19:02.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.643+0000 7f2d57a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2d5015e130 0x7f2d5015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:02.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.643+0000 7f2d57a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d5015d410 0x7f2d5015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:02.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.643+0000 7f2d57a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d50071670 con 0x7f2d5015dd50 2024-05-22T21:19:02.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.643+0000 7f2d57a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d5015d870 con 0x7f2d5015e130 2024-05-22T21:19:02.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.643+0000 7f2d57a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d50070a60 con 0x7f2d5015d410 2024-05-22T21:19:02.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.643+0000 7f2d55a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d5015d410 0x7f2d5015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:02.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.643+0000 7f2d47e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2d5015e130 0x7f2d5015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:02.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.643+0000 7f2d55a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d5015d410 0x7f2d5015b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:46854/0 (socket says 172.21.15.161:46854) 2024-05-22T21:19:02.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.643+0000 7f2d55a00640 1 -- 172.21.15.161:0/1471928630 learned_addr learned my addr 172.21.15.161:0/1471928630 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:02.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.643+0000 7f2d55000640 1 --1- 172.21.15.161:0/1471928630 >> v1:172.21.15.179:6789/0 conn(0x7f2d5015dd50 0x7f2d5006d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:59590/0 (socket says 172.21.15.161:59590) 2024-05-22T21:19:02.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.643+0000 7f2d55a00640 1 -- 172.21.15.161:0/1471928630 >> v2:172.21.15.179:3300/0 conn(0x7f2d5015e130 msgr2=0x7f2d5015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:02.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.643+0000 7f2d55a00640 1 --2- 172.21.15.161:0/1471928630 >> v2:172.21.15.179:3300/0 conn(0x7f2d5015e130 0x7f2d5015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.643+0000 7f2d55a00640 1 -- 172.21.15.161:0/1471928630 >> v1:172.21.15.179:6789/0 conn(0x7f2d5015dd50 legacy=0x7f2d5006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:02.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.643+0000 7f2d4fe00640 1 -- 172.21.15.161:0/1471928630 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564668160 0 0) 0x7f2d50071670 con 0x7f2d5015dd50 2024-05-22T21:19:02.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.643+0000 7f2d55a00640 1 -- 172.21.15.161:0/1471928630 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d50070d40 con 0x7f2d5015d410 2024-05-22T21:19:02.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.647+0000 7f2d55a00640 1 --2- 172.21.15.161:0/1471928630 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d5015d410 0x7f2d5015b6c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800f370 tx=0x7f2d48020e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa4631cb23c80187 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:02.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.647+0000 7f2d4fe00640 1 -- 172.21.15.161:0/1471928630 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4800d0d0 con 0x7f2d5015d410 2024-05-22T21:19:02.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.647+0000 7f2d4fe00640 1 -- 172.21.15.161:0/1471928630 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4800d230 con 0x7f2d5015d410 2024-05-22T21:19:02.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.647+0000 7f2d4fe00640 1 -- 172.21.15.161:0/1471928630 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4800d5c0 con 0x7f2d5015d410 2024-05-22T21:19:02.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.647+0000 7f2d57a00640 1 -- 172.21.15.161:0/1471928630 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d5015d410 msgr2=0x7f2d5015b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:02.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.647+0000 7f2d57a00640 1 --2- 172.21.15.161:0/1471928630 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d5015d410 0x7f2d5015b6c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800f370 tx=0x7f2d48020e00 comp rx=0 tx=0).stop 2024-05-22T21:19:02.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.647+0000 7f2d57a00640 1 -- 172.21.15.161:0/1471928630 shutdown_connections 2024-05-22T21:19:02.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.647+0000 7f2d57a00640 1 --2- 172.21.15.161:0/1471928630 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d5015d410 0x7f2d5015b6c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.647+0000 7f2d57a00640 1 --2- 172.21.15.161:0/1471928630 >> v2:172.21.15.179:3300/0 conn(0x7f2d5015e130 0x7f2d5015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.647+0000 7f2d57a00640 1 -- 172.21.15.161:0/1471928630 >> 172.21.15.161:0/1471928630 conn(0x7f2d500b0bc0 msgr2=0x7f2d500b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:02.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.647+0000 7f2d57a00640 1 -- 172.21.15.161:0/1471928630 shutdown_connections 2024-05-22T21:19:02.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.647+0000 7f2d57a00640 1 -- 172.21.15.161:0/1471928630 wait complete. 2024-05-22T21:19:02.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.647+0000 7f2d57a00640 1 Processor -- start 2024-05-22T21:19:02.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d57a00640 1 -- start start 2024-05-22T21:19:02.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d57a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d5015d410 0x7f2d50158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:02.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d57a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d5015dd50 0x7f2d5006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:02.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d57a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d5015e130 0x7f2d500b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:02.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d57a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d501507d0 con 0x7f2d5015d410 2024-05-22T21:19:02.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d57a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d50150d90 con 0x7f2d5015e130 2024-05-22T21:19:02.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d57a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d50158dc0 con 0x7f2d5015dd50 2024-05-22T21:19:02.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d55000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d5015d410 0x7f2d50158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:02.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d55000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d5015d410 0x7f2d50158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52000/0 (socket says 172.21.15.161:52000) 2024-05-22T21:19:02.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d55000640 1 -- 172.21.15.161:0/350449221 learned_addr learned my addr 172.21.15.161:0/350449221 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:02.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d47e00640 1 --2- 172.21.15.161:0/350449221 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d5015dd50 0x7f2d5006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:02.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d55a00640 1 --2- 172.21.15.161:0/350449221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d5015e130 0x7f2d500b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d55000640 1 -- 172.21.15.161:0/350449221 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d5015dd50 msgr2=0x7f2d5006c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d55000640 1 --2- 172.21.15.161:0/350449221 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d5015dd50 0x7f2d5006c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d55000640 1 -- 172.21.15.161:0/350449221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d5015e130 msgr2=0x7f2d500b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d55000640 1 --2- 172.21.15.161:0/350449221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d5015e130 0x7f2d500b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d55000640 1 -- 172.21.15.161:0/350449221 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4800f050 con 0x7f2d5015d410 2024-05-22T21:19:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d47e00640 1 --2- 172.21.15.161:0/350449221 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d5015dd50 0x7f2d5006c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d55000640 1 --2- 172.21.15.161:0/350449221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d5015d410 0x7f2d50158880 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000c520 tx=0x7f2d4000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d4d600640 1 -- 172.21.15.161:0/350449221 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d40006480 con 0x7f2d5015d410 2024-05-22T21:19:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d4d600640 1 -- 172.21.15.161:0/350449221 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d400065e0 con 0x7f2d5015d410 2024-05-22T21:19:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d57a00640 1 -- 172.21.15.161:0/350449221 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d500b36d0 con 0x7f2d5015d410 2024-05-22T21:19:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d4d600640 1 -- 172.21.15.161:0/350449221 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d40004a10 con 0x7f2d5015d410 2024-05-22T21:19:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d57a00640 1 -- 172.21.15.161:0/350449221 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d500b3c50 con 0x7f2d5015d410 2024-05-22T21:19:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.651+0000 7f2d57a00640 1 -- 172.21.15.161:0/350449221 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d50067220 con 0x7f2d5015d410 2024-05-22T21:19:02.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.655+0000 7f2d4d600640 1 -- 172.21.15.161:0/350449221 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2d40004b70 con 0x7f2d5015d410 2024-05-22T21:19:02.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.655+0000 7f2d4d600640 1 --2- 172.21.15.161:0/350449221 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d2c066690 0x7f2d2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:02.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.655+0000 7f2d4d600640 1 -- 172.21.15.161:0/350449221 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d40083800 con 0x7f2d5015d410 2024-05-22T21:19:02.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.655+0000 7f2d47e00640 1 --2- 172.21.15.161:0/350449221 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d2c066690 0x7f2d2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:02.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.655+0000 7f2d47e00640 1 --2- 172.21.15.161:0/350449221 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d2c066690 0x7f2d2c068b50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2d5014cae0 tx=0x7f2d38009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:02.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.659+0000 7f2d4d600640 1 -- 172.21.15.161:0/350449221 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2d40050b80 con 0x7f2d5015d410 2024-05-22T21:19:02.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.843+0000 7f2d57a00640 1 -- 172.21.15.161:0/350449221 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d50150430 con 0x7f2d5015d410 2024-05-22T21:19:02.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.847+0000 7f2d4d600640 1 -- 172.21.15.161:0/350449221 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f2d40017020 con 0x7f2d5015d410 2024-05-22T21:19:02.848 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:02.848 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2771409351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2771409351},{"type":"v1","addr":"172.21.15.179:6834","nonce":2771409351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6442,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2520119431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2520119431},{"type":"v1","addr":"172.21.15.179:6835","nonce":2520119431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1415362547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1415362547},{"type":"v1","addr":"172.21.15.161:6835","nonce":1415362547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:18:17.938858+0000","modified":"2024-05-22T21:18:20.963065+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/4127130864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4127130864},{"type":"v1","addr":"172.21.15.161:6837","nonce":4127130864}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-22T21:19:02.848 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 47 2024-05-22T21:19:02.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.847+0000 7f2d57a00640 1 -- 172.21.15.161:0/350449221 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d2c066690 msgr2=0x7f2d2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:02.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.847+0000 7f2d57a00640 1 --2- 172.21.15.161:0/350449221 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d2c066690 0x7f2d2c068b50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2d5014cae0 tx=0x7f2d38009190 comp rx=0 tx=0).stop 2024-05-22T21:19:02.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.847+0000 7f2d57a00640 1 -- 172.21.15.161:0/350449221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d5015d410 msgr2=0x7f2d50158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:02.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.847+0000 7f2d57a00640 1 --2- 172.21.15.161:0/350449221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d5015d410 0x7f2d50158880 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000c520 tx=0x7f2d4000c9e0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.847+0000 7f2d57a00640 1 -- 172.21.15.161:0/350449221 shutdown_connections 2024-05-22T21:19:02.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.847+0000 7f2d57a00640 1 --2- 172.21.15.161:0/350449221 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d2c066690 0x7f2d2c068b50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.847+0000 7f2d57a00640 1 --2- 172.21.15.161:0/350449221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d5015e130 0x7f2d500b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.847+0000 7f2d57a00640 1 --2- 172.21.15.161:0/350449221 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d5015dd50 0x7f2d5006c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.847+0000 7f2d57a00640 1 --2- 172.21.15.161:0/350449221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d5015d410 0x7f2d50158880 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:02.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.847+0000 7f2d57a00640 1 -- 172.21.15.161:0/350449221 >> 172.21.15.161:0/350449221 conn(0x7f2d500b0bc0 msgr2=0x7f2d500b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:02.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.847+0000 7f2d57a00640 1 -- 172.21.15.161:0/350449221 shutdown_connections 2024-05-22T21:19:02.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:02.847+0000 7f2d57a00640 1 -- 172.21.15.161:0/350449221 wait complete. 2024-05-22T21:19:02.870 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:03.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.003+0000 7fd37d400640 1 Processor -- start 2024-05-22T21:19:03.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.003+0000 7fd37d400640 1 -- start start 2024-05-22T21:19:03.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.003+0000 7fd37d400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd378069ed0 0x7fd37806a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.003+0000 7fd37d400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd378071cb0 0x7fd37806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.003+0000 7fd37d400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3780cff20 con 0x7fd3780d0c10 2024-05-22T21:19:03.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.003+0000 7fd37d400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd37806a7f0 con 0x7fd378071cb0 2024-05-22T21:19:03.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.003+0000 7fd37d400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780cf970 con 0x7fd378069ed0 2024-05-22T21:19:03.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.003+0000 7fd377200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd378071cb0 0x7fd37806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.003+0000 7fd377200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd378071cb0 0x7fd37806cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52016/0 (socket says 172.21.15.161:52016) 2024-05-22T21:19:03.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.003+0000 7fd377200640 1 -- 172.21.15.161:0/121529347 learned_addr learned my addr 172.21.15.161:0/121529347 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:03.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.003+0000 7fd376800640 1 --2- 172.21.15.161:0/121529347 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd378069ed0 0x7fd37806a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.003+0000 7fd377200640 1 -- 172.21.15.161:0/121529347 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd378069ed0 msgr2=0x7fd37806a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:03.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.003+0000 7fd377200640 1 --2- 172.21.15.161:0/121529347 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd378069ed0 0x7fd37806a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.003+0000 7fd377200640 1 -- 172.21.15.161:0/121529347 >> v1:172.21.15.179:6789/0 conn(0x7fd3780d0c10 legacy=0x7fd3780cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.003+0000 7fd377200640 1 -- 172.21.15.161:0/121529347 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3780cfc40 con 0x7fd378071cb0 2024-05-22T21:19:03.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.007+0000 7fd377200640 1 --2- 172.21.15.161:0/121529347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378071cb0 0x7fd37806cff0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd370009e50 tx=0x7fd37001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b72f3c2c1f9aa5d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:03.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.007+0000 7fd375400640 1 -- 172.21.15.161:0/121529347 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3700a58d0 con 0x7fd378071cb0 2024-05-22T21:19:03.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.007+0000 7fd375400640 1 -- 172.21.15.161:0/121529347 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3700a5a30 con 0x7fd378071cb0 2024-05-22T21:19:03.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.007+0000 7fd375400640 1 -- 172.21.15.161:0/121529347 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3700a5dc0 con 0x7fd378071cb0 2024-05-22T21:19:03.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.007+0000 7fd37d400640 1 -- 172.21.15.161:0/121529347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378071cb0 msgr2=0x7fd37806cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.007+0000 7fd37d400640 1 --2- 172.21.15.161:0/121529347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378071cb0 0x7fd37806cff0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd370009e50 tx=0x7fd37001ca00 comp rx=0 tx=0).stop 2024-05-22T21:19:03.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.007+0000 7fd37d400640 1 -- 172.21.15.161:0/121529347 shutdown_connections 2024-05-22T21:19:03.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.007+0000 7fd37d400640 1 --2- 172.21.15.161:0/121529347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378071cb0 0x7fd37806cff0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.007+0000 7fd37d400640 1 --2- 172.21.15.161:0/121529347 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd378069ed0 0x7fd37806a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.007+0000 7fd37d400640 1 -- 172.21.15.161:0/121529347 >> 172.21.15.161:0/121529347 conn(0x7fd3780d00b0 msgr2=0x7fd3780d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:03.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.007+0000 7fd37d400640 1 -- 172.21.15.161:0/121529347 shutdown_connections 2024-05-22T21:19:03.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.007+0000 7fd37d400640 1 -- 172.21.15.161:0/121529347 wait complete. 2024-05-22T21:19:03.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd37d400640 1 Processor -- start 2024-05-22T21:19:03.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd37d400640 1 -- start start 2024-05-22T21:19:03.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd37d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378069ed0 0x7fd3780b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd37d400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd378071cb0 0x7fd3781667b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd37d400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3780d0c10 0x7fd3781566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd37d400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd378166cf0 con 0x7fd378069ed0 2024-05-22T21:19:03.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd37d400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780b4b60 con 0x7fd3780d0c10 2024-05-22T21:19:03.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd37d400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780b3f50 con 0x7fd378071cb0 2024-05-22T21:19:03.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd376800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378069ed0 0x7fd3780b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd376800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378069ed0 0x7fd3780b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52018/0 (socket says 172.21.15.161:52018) 2024-05-22T21:19:03.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd376800640 1 -- 172.21.15.161:0/4030466251 learned_addr learned my addr 172.21.15.161:0/4030466251 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:03.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd375e00640 1 --2- 172.21.15.161:0/4030466251 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd378071cb0 0x7fd3781667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd377200640 1 --2- 172.21.15.161:0/4030466251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3780d0c10 0x7fd3781566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd376800640 1 -- 172.21.15.161:0/4030466251 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd378071cb0 msgr2=0x7fd3781667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd376800640 1 --2- 172.21.15.161:0/4030466251 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd378071cb0 0x7fd3781667b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd376800640 1 -- 172.21.15.161:0/4030466251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3780d0c10 msgr2=0x7fd3781566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd376800640 1 --2- 172.21.15.161:0/4030466251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3780d0c10 0x7fd3781566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd376800640 1 -- 172.21.15.161:0/4030466251 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd370009b30 con 0x7fd378069ed0 2024-05-22T21:19:03.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd376800640 1 --2- 172.21.15.161:0/4030466251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378069ed0 0x7fd3780b4ff0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd36800d5f0 tx=0x7fd36800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:03.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd36ea00640 1 -- 172.21.15.161:0/4030466251 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3680125f0 con 0x7fd378069ed0 2024-05-22T21:19:03.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd36ea00640 1 -- 172.21.15.161:0/4030466251 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd368012750 con 0x7fd378069ed0 2024-05-22T21:19:03.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd36ea00640 1 -- 172.21.15.161:0/4030466251 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368012ae0 con 0x7fd378069ed0 2024-05-22T21:19:03.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd37d400640 1 -- 172.21.15.161:0/4030466251 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3780b4230 con 0x7fd378069ed0 2024-05-22T21:19:03.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd37d400640 1 -- 172.21.15.161:0/4030466251 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3780b47f0 con 0x7fd378069ed0 2024-05-22T21:19:03.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.011+0000 7fd37d400640 1 -- 172.21.15.161:0/4030466251 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd378067220 con 0x7fd378069ed0 2024-05-22T21:19:03.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.015+0000 7fd36ea00640 1 -- 172.21.15.161:0/4030466251 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd368012c40 con 0x7fd378069ed0 2024-05-22T21:19:03.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.015+0000 7fd36ea00640 1 --2- 172.21.15.161:0/4030466251 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd358066690 0x7fd358068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.015+0000 7fd36ea00640 1 -- 172.21.15.161:0/4030466251 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3680847d0 con 0x7fd378069ed0 2024-05-22T21:19:03.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.015+0000 7fd375e00640 1 --2- 172.21.15.161:0/4030466251 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd358066690 0x7fd358068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.015+0000 7fd375e00640 1 --2- 172.21.15.161:0/4030466251 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd358066690 0x7fd358068b50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd378167090 tx=0x7fd364009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:03.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.019+0000 7fd36ea00640 1 -- 172.21.15.161:0/4030466251 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd368051aa0 con 0x7fd378069ed0 2024-05-22T21:19:03.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.203+0000 7fd37d400640 1 -- 172.21.15.161:0/4030466251 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd378072180 con 0x7fd378069ed0 2024-05-22T21:19:03.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.203+0000 7fd36ea00640 1 -- 172.21.15.161:0/4030466251 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4748 (secure 0 0 0) 0x7fd368007380 con 0x7fd378069ed0 2024-05-22T21:19:03.206 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:03.206 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2771409351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2771409351},{"type":"v1","addr":"172.21.15.179:6834","nonce":2771409351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6442,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2520119431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2520119431},{"type":"v1","addr":"172.21.15.179:6835","nonce":2520119431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1415362547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1415362547},{"type":"v1","addr":"172.21.15.161:6835","nonce":1415362547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:18:17.938858+0000","modified":"2024-05-22T21:18:20.963065+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/4127130864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4127130864},{"type":"v1","addr":"172.21.15.161:6837","nonce":4127130864}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-22T21:19:03.206 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 47 2024-05-22T21:19:03.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.203+0000 7fd37d400640 1 -- 172.21.15.161:0/4030466251 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd358066690 msgr2=0x7fd358068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.203+0000 7fd37d400640 1 --2- 172.21.15.161:0/4030466251 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd358066690 0x7fd358068b50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd378167090 tx=0x7fd364009040 comp rx=0 tx=0).stop 2024-05-22T21:19:03.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.203+0000 7fd37d400640 1 -- 172.21.15.161:0/4030466251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378069ed0 msgr2=0x7fd3780b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.203+0000 7fd37d400640 1 --2- 172.21.15.161:0/4030466251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378069ed0 0x7fd3780b4ff0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd36800d5f0 tx=0x7fd36800dab0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.203+0000 7fd37d400640 1 -- 172.21.15.161:0/4030466251 shutdown_connections 2024-05-22T21:19:03.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.203+0000 7fd37d400640 1 --2- 172.21.15.161:0/4030466251 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd358066690 0x7fd358068b50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.203+0000 7fd37d400640 1 --2- 172.21.15.161:0/4030466251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3780d0c10 0x7fd3781566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.203+0000 7fd37d400640 1 --2- 172.21.15.161:0/4030466251 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd378071cb0 0x7fd3781667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.203+0000 7fd37d400640 1 --2- 172.21.15.161:0/4030466251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378069ed0 0x7fd3780b4ff0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.203+0000 7fd37d400640 1 -- 172.21.15.161:0/4030466251 >> 172.21.15.161:0/4030466251 conn(0x7fd3780d00b0 msgr2=0x7fd378072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:03.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.207+0000 7fd37d400640 1 -- 172.21.15.161:0/4030466251 shutdown_connections 2024-05-22T21:19:03.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.207+0000 7fd37d400640 1 -- 172.21.15.161:0/4030466251 wait complete. 2024-05-22T21:19:03.228 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:03.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a8ec00640 1 Processor -- start 2024-05-22T21:19:03.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a8ec00640 1 -- start start 2024-05-22T21:19:03.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a8ec00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a880715a0 0x7f1a8815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a8ec00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a881442a0 con 0x7f1a8815ec90 2024-05-22T21:19:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a8ec00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a88070500 con 0x7f1a88143ec0 2024-05-22T21:19:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a8ec00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a880707e0 con 0x7f1a880715a0 2024-05-22T21:19:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a8cc00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1a8815ec90 0x7f1a880cd710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:52012/0 (socket says 172.21.15.161:52012) 2024-05-22T21:19:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a8cc00640 1 -- 172.21.15.161:0/3494366601 learned_addr learned my addr 172.21.15.161:0/3494366601 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a87400640 1 --2- 172.21.15.161:0/3494366601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a880715a0 0x7f1a8815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a86a00640 1 -- 172.21.15.161:0/3494366601 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548545625 0 0) 0x7f1a881442a0 con 0x7f1a8815ec90 2024-05-22T21:19:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a86a00640 1 -- 172.21.15.161:0/3494366601 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a6c003530 con 0x7f1a8815ec90 2024-05-22T21:19:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a86a00640 1 -- 172.21.15.161:0/3494366601 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957257071 0 0) 0x7f1a88070500 con 0x7f1a88143ec0 2024-05-22T21:19:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a86a00640 1 -- 172.21.15.161:0/3494366601 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a881442a0 con 0x7f1a88143ec0 2024-05-22T21:19:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a87400640 1 -- 172.21.15.161:0/3494366601 >> v1:172.21.15.179:6789/0 conn(0x7f1a88143ec0 legacy=0x7f1a8806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a87400640 1 -- 172.21.15.161:0/3494366601 >> v1:172.21.15.161:6789/0 conn(0x7f1a8815ec90 legacy=0x7f1a880cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a86a00640 1 -- 172.21.15.161:0/3494366601 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 745794519 0 0) 0x7f1a6c003530 con 0x7f1a8815ec90 2024-05-22T21:19:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.363+0000 7f1a87400640 1 -- 172.21.15.161:0/3494366601 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a88070ac0 con 0x7f1a880715a0 2024-05-22T21:19:03.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.367+0000 7f1a87400640 1 --2- 172.21.15.161:0/3494366601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a880715a0 0x7f1a8815f830 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1a780097f0 tx=0x7f1a7801c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4ec6b6927507db6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:03.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.367+0000 7f1a86a00640 1 -- 172.21.15.161:0/3494366601 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80001a00 con 0x7f1a880715a0 2024-05-22T21:19:03.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.367+0000 7f1a86a00640 1 -- 172.21.15.161:0/3494366601 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a780a7050 con 0x7f1a880715a0 2024-05-22T21:19:03.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.367+0000 7f1a86a00640 1 -- 172.21.15.161:0/3494366601 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a78004230 con 0x7f1a880715a0 2024-05-22T21:19:03.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.367+0000 7f1a8ec00640 1 -- 172.21.15.161:0/3494366601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a880715a0 msgr2=0x7f1a8815f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.367+0000 7f1a8ec00640 1 --2- 172.21.15.161:0/3494366601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a880715a0 0x7f1a8815f830 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1a780097f0 tx=0x7f1a7801c3d0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.367+0000 7f1a8ec00640 1 -- 172.21.15.161:0/3494366601 shutdown_connections 2024-05-22T21:19:03.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.367+0000 7f1a8ec00640 1 --2- 172.21.15.161:0/3494366601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a880715a0 0x7f1a8815f830 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.367+0000 7f1a8ec00640 1 -- 172.21.15.161:0/3494366601 >> 172.21.15.161:0/3494366601 conn(0x7f1a880b0bc0 msgr2=0x7f1a880b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:03.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.367+0000 7f1a8ec00640 1 -- 172.21.15.161:0/3494366601 shutdown_connections 2024-05-22T21:19:03.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.367+0000 7f1a8ec00640 1 -- 172.21.15.161:0/3494366601 wait complete. 2024-05-22T21:19:03.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a8ec00640 1 Processor -- start 2024-05-22T21:19:03.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a8ec00640 1 -- start start 2024-05-22T21:19:03.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a8ec00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a88143ec0 0x7f1a881597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a8ec00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a8815ec90 0x7f1a880ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a8ec00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a88159e60 0x7f1a88138c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a8ec00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a880ad0a0 con 0x7f1a88143ec0 2024-05-22T21:19:03.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a8ec00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a880b3450 con 0x7f1a8815ec90 2024-05-22T21:19:03.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a8ec00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a880b3170 con 0x7f1a88159e60 2024-05-22T21:19:03.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a87e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a88143ec0 0x7f1a881597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a87e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a88143ec0 0x7f1a881597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52024/0 (socket says 172.21.15.161:52024) 2024-05-22T21:19:03.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a87e00640 1 -- 172.21.15.161:0/1416656712 learned_addr learned my addr 172.21.15.161:0/1416656712 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:03.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a87400640 1 --2- 172.21.15.161:0/1416656712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a8815ec90 0x7f1a880ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a8cc00640 1 --2- 172.21.15.161:0/1416656712 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a88159e60 0x7f1a88138c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a87e00640 1 -- 172.21.15.161:0/1416656712 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a88159e60 msgr2=0x7f1a88138c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a87e00640 1 --2- 172.21.15.161:0/1416656712 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a88159e60 0x7f1a88138c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a87e00640 1 -- 172.21.15.161:0/1416656712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a8815ec90 msgr2=0x7f1a880ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a87e00640 1 --2- 172.21.15.161:0/1416656712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a8815ec90 0x7f1a880ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a87e00640 1 -- 172.21.15.161:0/1416656712 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a78009420 con 0x7f1a88143ec0 2024-05-22T21:19:03.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a87e00640 1 --2- 172.21.15.161:0/1416656712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a88143ec0 0x7f1a881597c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000c4c0 tx=0x7f1a7000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:03.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a7fe00640 1 -- 172.21.15.161:0/1416656712 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a700052d0 con 0x7f1a88143ec0 2024-05-22T21:19:03.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a7fe00640 1 -- 172.21.15.161:0/1416656712 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a70005430 con 0x7f1a88143ec0 2024-05-22T21:19:03.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a7fe00640 1 -- 172.21.15.161:0/1416656712 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a70012870 con 0x7f1a88143ec0 2024-05-22T21:19:03.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a8ec00640 1 -- 172.21.15.161:0/1416656712 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a880aca50 con 0x7f1a88143ec0 2024-05-22T21:19:03.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.371+0000 7f1a8ec00640 1 -- 172.21.15.161:0/1416656712 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a880ada70 con 0x7f1a88143ec0 2024-05-22T21:19:03.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.375+0000 7f1a8ec00640 1 -- 172.21.15.161:0/1416656712 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a880ac8b0 con 0x7f1a88143ec0 2024-05-22T21:19:03.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.375+0000 7f1a7fe00640 1 -- 172.21.15.161:0/1416656712 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1a700129d0 con 0x7f1a88143ec0 2024-05-22T21:19:03.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.375+0000 7f1a7fe00640 1 --2- 172.21.15.161:0/1416656712 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1a64066690 0x7f1a64068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.375+0000 7f1a7fe00640 1 -- 172.21.15.161:0/1416656712 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a70083d30 con 0x7f1a88143ec0 2024-05-22T21:19:03.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.375+0000 7f1a87400640 1 --2- 172.21.15.161:0/1416656712 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1a64066690 0x7f1a64068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.375+0000 7f1a87400640 1 --2- 172.21.15.161:0/1416656712 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1a64066690 0x7f1a64068b50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1a780ad000 tx=0x7f1a780a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:03.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.379+0000 7f1a7fe00640 1 -- 172.21.15.161:0/1416656712 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1a880ac8b0 con 0x7f1a88143ec0 2024-05-22T21:19:03.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.539+0000 7f1a8ec00640 1 -- 172.21.15.161:0/1416656712 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a880ac8b0 con 0x7f1a88143ec0 2024-05-22T21:19:03.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.539+0000 7f1a7fe00640 1 -- 172.21.15.161:0/1416656712 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16152 (secure 0 0 0) 0x7f1a880ac8b0 con 0x7f1a88143ec0 2024-05-22T21:19:03.541 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:03.541 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":113,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:18:50.632514+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:18:16.500534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-22T21:18:16.937445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:03.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.539+0000 7f1a8ec00640 1 -- 172.21.15.161:0/1416656712 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1a64066690 msgr2=0x7f1a64068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.539+0000 7f1a8ec00640 1 --2- 172.21.15.161:0/1416656712 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1a64066690 0x7f1a64068b50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1a780ad000 tx=0x7f1a780a6000 comp rx=0 tx=0).stop 2024-05-22T21:19:03.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.539+0000 7f1a8ec00640 1 -- 172.21.15.161:0/1416656712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a88143ec0 msgr2=0x7f1a881597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.539+0000 7f1a8ec00640 1 --2- 172.21.15.161:0/1416656712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a88143ec0 0x7f1a881597c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000c4c0 tx=0x7f1a7000c980 comp rx=0 tx=0).stop 2024-05-22T21:19:03.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.539+0000 7f1a8ec00640 1 -- 172.21.15.161:0/1416656712 shutdown_connections 2024-05-22T21:19:03.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.539+0000 7f1a8ec00640 1 --2- 172.21.15.161:0/1416656712 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1a64066690 0x7f1a64068b50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.539+0000 7f1a8ec00640 1 --2- 172.21.15.161:0/1416656712 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a88159e60 0x7f1a88138c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.539+0000 7f1a8ec00640 1 --2- 172.21.15.161:0/1416656712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a8815ec90 0x7f1a880ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.539+0000 7f1a8ec00640 1 --2- 172.21.15.161:0/1416656712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a88143ec0 0x7f1a881597c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.539+0000 7f1a8ec00640 1 -- 172.21.15.161:0/1416656712 >> 172.21.15.161:0/1416656712 conn(0x7f1a880b0bc0 msgr2=0x7f1a8815ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:03.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.539+0000 7f1a8ec00640 1 -- 172.21.15.161:0/1416656712 shutdown_connections 2024-05-22T21:19:03.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.539+0000 7f1a8ec00640 1 -- 172.21.15.161:0/1416656712 wait complete. 2024-05-22T21:19:03.566 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-22T21:19:03.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.699+0000 7f98cec00640 1 Processor -- start 2024-05-22T21:19:03.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.699+0000 7f98cec00640 1 -- start start 2024-05-22T21:19:03.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.703+0000 7f98cec00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98c81596b0 0x7f98c8071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.703+0000 7f98cec00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98c8071db0 0x7f98c806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.703+0000 7f98cec00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c80b3110 con 0x7f98c806d070 2024-05-22T21:19:03.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.703+0000 7f98cec00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c806d4e0 con 0x7f98c8071db0 2024-05-22T21:19:03.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.703+0000 7f98cec00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c815d250 con 0x7f98c81596b0 2024-05-22T21:19:03.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.703+0000 7f98c7400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98c8071db0 0x7f98c806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.703+0000 7f98c7e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98c81596b0 0x7f98c8071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.703+0000 7f98c7400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98c8071db0 0x7f98c806cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:53022/0 (socket says 172.21.15.161:53022) 2024-05-22T21:19:03.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.703+0000 7f98c7400640 1 -- 172.21.15.161:0/1304772951 learned_addr learned my addr 172.21.15.161:0/1304772951 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:03.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.703+0000 7f98c7e00640 1 -- 172.21.15.161:0/1304772951 >> v2:172.21.15.179:3300/0 conn(0x7f98c8071db0 msgr2=0x7f98c806cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:03.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.703+0000 7f98c7e00640 1 --2- 172.21.15.161:0/1304772951 >> v2:172.21.15.179:3300/0 conn(0x7f98c8071db0 0x7f98c806cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.703+0000 7f98c7e00640 1 -- 172.21.15.161:0/1304772951 >> v1:172.21.15.179:6789/0 conn(0x7f98c806d070 legacy=0x7f98c806fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.703+0000 7f98c7e00640 1 -- 172.21.15.161:0/1304772951 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c8159a90 con 0x7f98c81596b0 2024-05-22T21:19:03.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.703+0000 7f98c7400640 1 --2- 172.21.15.161:0/1304772951 >> v2:172.21.15.179:3300/0 conn(0x7f98c8071db0 0x7f98c806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:03.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.707+0000 7f98c7e00640 1 --2- 172.21.15.161:0/1304772951 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98c81596b0 0x7f98c8071870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f98b80097c0 tx=0x7f98b801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef156843908e7563 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:03.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.707+0000 7f98c6a00640 1 -- 172.21.15.161:0/1304772951 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b80ab600 con 0x7f98c81596b0 2024-05-22T21:19:03.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.707+0000 7f98c6a00640 1 -- 172.21.15.161:0/1304772951 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b80a7050 con 0x7f98c81596b0 2024-05-22T21:19:03.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.707+0000 7f98c6a00640 1 -- 172.21.15.161:0/1304772951 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b80ab980 con 0x7f98c81596b0 2024-05-22T21:19:03.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.707+0000 7f98cec00640 1 -- 172.21.15.161:0/1304772951 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98c81596b0 msgr2=0x7f98c8071870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.707+0000 7f98cec00640 1 --2- 172.21.15.161:0/1304772951 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98c81596b0 0x7f98c8071870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f98b80097c0 tx=0x7f98b801c4a0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.707+0000 7f98cec00640 1 -- 172.21.15.161:0/1304772951 shutdown_connections 2024-05-22T21:19:03.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.707+0000 7f98cec00640 1 --2- 172.21.15.161:0/1304772951 >> v2:172.21.15.179:3300/0 conn(0x7f98c8071db0 0x7f98c806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.707+0000 7f98cec00640 1 --2- 172.21.15.161:0/1304772951 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98c81596b0 0x7f98c8071870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.707+0000 7f98cec00640 1 -- 172.21.15.161:0/1304772951 >> 172.21.15.161:0/1304772951 conn(0x7f98c806a2d0 msgr2=0x7f98c806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:03.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.707+0000 7f98cec00640 1 -- 172.21.15.161:0/1304772951 shutdown_connections 2024-05-22T21:19:03.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.707+0000 7f98cec00640 1 -- 172.21.15.161:0/1304772951 wait complete. 2024-05-22T21:19:03.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.707+0000 7f98cec00640 1 Processor -- start 2024-05-22T21:19:03.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.707+0000 7f98cec00640 1 -- start start 2024-05-22T21:19:03.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98cec00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98c806d070 0x7f98c813d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98cec00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98c8071db0 0x7f98c814b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98cec00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98c81596b0 0x7f98c80cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98cec00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c814bdd0 con 0x7f98c8071db0 2024-05-22T21:19:03.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98cec00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c814db30 con 0x7f98c81596b0 2024-05-22T21:19:03.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98cec00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c814d200 con 0x7f98c806d070 2024-05-22T21:19:03.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98c7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98c8071db0 0x7f98c814b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98c7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98c8071db0 0x7f98c814b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52034/0 (socket says 172.21.15.161:52034) 2024-05-22T21:19:03.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98c7400640 1 -- 172.21.15.161:0/486880219 learned_addr learned my addr 172.21.15.161:0/486880219 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:03.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98c7e00640 1 --2- 172.21.15.161:0/486880219 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98c806d070 0x7f98c813d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98ccc00640 1 --2- 172.21.15.161:0/486880219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98c81596b0 0x7f98c80cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98c7400640 1 -- 172.21.15.161:0/486880219 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98c806d070 msgr2=0x7f98c813d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98c7400640 1 --2- 172.21.15.161:0/486880219 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98c806d070 0x7f98c813d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98c7400640 1 -- 172.21.15.161:0/486880219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98c81596b0 msgr2=0x7f98c80cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98c7400640 1 --2- 172.21.15.161:0/486880219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98c81596b0 0x7f98c80cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98c7400640 1 -- 172.21.15.161:0/486880219 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b40094d0 con 0x7f98c8071db0 2024-05-22T21:19:03.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98c7400640 1 --2- 172.21.15.161:0/486880219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98c8071db0 0x7f98c814b3a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f98b400e9a0 tx=0x7f98b400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:03.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98bfe00640 1 -- 172.21.15.161:0/486880219 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b400ca00 con 0x7f98c8071db0 2024-05-22T21:19:03.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98bfe00640 1 -- 172.21.15.161:0/486880219 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b400cb60 con 0x7f98c8071db0 2024-05-22T21:19:03.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98cec00640 1 -- 172.21.15.161:0/486880219 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b8009420 con 0x7f98c8071db0 2024-05-22T21:19:03.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98bfe00640 1 -- 172.21.15.161:0/486880219 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b4013470 con 0x7f98c8071db0 2024-05-22T21:19:03.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98cec00640 1 -- 172.21.15.161:0/486880219 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c814d7c0 con 0x7f98c8071db0 2024-05-22T21:19:03.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98cec00640 1 -- 172.21.15.161:0/486880219 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c8067220 con 0x7f98c8071db0 2024-05-22T21:19:03.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.711+0000 7f98bfe00640 1 -- 172.21.15.161:0/486880219 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f98b40135d0 con 0x7f98c8071db0 2024-05-22T21:19:03.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.715+0000 7f98bfe00640 1 --2- 172.21.15.161:0/486880219 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f98a80666c0 0x7f98a8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:03.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.715+0000 7f98bfe00640 1 -- 172.21.15.161:0/486880219 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98b40844a0 con 0x7f98c8071db0 2024-05-22T21:19:03.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.715+0000 7f98c7e00640 1 --2- 172.21.15.161:0/486880219 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f98a80666c0 0x7f98a8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:03.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.715+0000 7f98c7e00640 1 --2- 172.21.15.161:0/486880219 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f98a80666c0 0x7f98a8068b80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f98b80ad000 tx=0x7f98b80a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:03.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.719+0000 7f98bfe00640 1 -- 172.21.15.161:0/486880219 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f98b4051770 con 0x7f98c8071db0 2024-05-22T21:19:03.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.903+0000 7f98cec00640 1 -- 172.21.15.161:0/486880219 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f98c80b1640 con 0x7f98c8071db0 2024-05-22T21:19:03.905 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:19:03.903+0000 7f7d32800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-22T21:19:03.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.911+0000 7f98bfe00640 1 -- 172.21.15.161:0/486880219 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7f98b4054d90 con 0x7f98c8071db0 2024-05-22T21:19:03.915 INFO:teuthology.orchestra.run.smithi161.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-22T21:19:03.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.915+0000 7f98cec00640 1 -- 172.21.15.161:0/486880219 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f98a80666c0 msgr2=0x7f98a8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.915+0000 7f98cec00640 1 --2- 172.21.15.161:0/486880219 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f98a80666c0 0x7f98a8068b80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f98b80ad000 tx=0x7f98b80a6000 comp rx=0 tx=0).stop 2024-05-22T21:19:03.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.915+0000 7f98cec00640 1 -- 172.21.15.161:0/486880219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98c8071db0 msgr2=0x7f98c814b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:03.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.915+0000 7f98cec00640 1 --2- 172.21.15.161:0/486880219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98c8071db0 0x7f98c814b3a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f98b400e9a0 tx=0x7f98b400ee60 comp rx=0 tx=0).stop 2024-05-22T21:19:03.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.915+0000 7f98cec00640 1 -- 172.21.15.161:0/486880219 shutdown_connections 2024-05-22T21:19:03.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.915+0000 7f98cec00640 1 --2- 172.21.15.161:0/486880219 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f98a80666c0 0x7f98a8068b80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.915+0000 7f98cec00640 1 --2- 172.21.15.161:0/486880219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98c81596b0 0x7f98c80cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.915+0000 7f98cec00640 1 --2- 172.21.15.161:0/486880219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98c8071db0 0x7f98c814b3a0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.915+0000 7f98cec00640 1 --2- 172.21.15.161:0/486880219 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98c806d070 0x7f98c813d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:03.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.915+0000 7f98cec00640 1 -- 172.21.15.161:0/486880219 >> 172.21.15.161:0/486880219 conn(0x7f98c806a2d0 msgr2=0x7f98c80b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:03.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.915+0000 7f98cec00640 1 -- 172.21.15.161:0/486880219 shutdown_connections 2024-05-22T21:19:03.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:03.915+0000 7f98cec00640 1 -- 172.21.15.161:0/486880219 wait complete. 2024-05-22T21:19:03.936 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-22T21:19:03.979 INFO:tasks.ceph.mds.c.smithi161.stdout:starting mds.c at 2024-05-22T21:19:04.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.071+0000 7f8c33400640 1 Processor -- start 2024-05-22T21:19:04.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.071+0000 7f8c33400640 1 -- start start 2024-05-22T21:19:04.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.075+0000 7f8c33400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c2c13cfc0 0x7f8c2c1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:04.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.075+0000 7f8c33400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8c2c15af60 0x7f8c2c13f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:04.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.075+0000 7f8c33400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c2c1400d0 con 0x7f8c2c13fc80 2024-05-22T21:19:04.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.075+0000 7f8c33400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c2c13d3a0 con 0x7f8c2c15af60 2024-05-22T21:19:04.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.075+0000 7f8c33400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c0cd920 con 0x7f8c2c13cfc0 2024-05-22T21:19:04.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.075+0000 7f8c23e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8c2c15af60 0x7f8c2c13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:04.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.075+0000 7f8c30a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c2c13cfc0 0x7f8c2c1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:04.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.075+0000 7f8c23e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8c2c15af60 0x7f8c2c13f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:53054/0 (socket says 172.21.15.161:53054) 2024-05-22T21:19:04.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.075+0000 7f8c23e00640 1 -- 172.21.15.161:0/1087116431 learned_addr learned my addr 172.21.15.161:0/1087116431 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:04.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.075+0000 7f8c30a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c2c13cfc0 0x7f8c2c1402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:46920/0 (socket says 172.21.15.161:46920) 2024-05-22T21:19:04.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.075+0000 7f8c31400640 1 --1- 172.21.15.161:0/1087116431 >> v1:172.21.15.179:6789/0 conn(0x7f8c2c13fc80 0x7f8c2c15dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:59646/0 (socket says 172.21.15.161:59646) 2024-05-22T21:19:04.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.075+0000 7f8c23e00640 1 -- 172.21.15.161:0/1087116431 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c2c13cfc0 msgr2=0x7f8c2c1402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:04.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.075+0000 7f8c23e00640 1 --2- 172.21.15.161:0/1087116431 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c2c13cfc0 0x7f8c2c1402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:04.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.075+0000 7f8c23e00640 1 -- 172.21.15.161:0/1087116431 >> v1:172.21.15.179:6789/0 conn(0x7f8c2c13fc80 legacy=0x7f8c2c15dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:04.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.075+0000 7f8c23e00640 1 -- 172.21.15.161:0/1087116431 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c0cdc00 con 0x7f8c2c15af60 2024-05-22T21:19:04.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.079+0000 7f8c23e00640 1 --2- 172.21.15.161:0/1087116431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c15af60 0x7f8c2c13f740 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400bb10 tx=0x7f8c1401eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76730c3e1c8705aa server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:04.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.079+0000 7f8c2be00640 1 -- 172.21.15.161:0/1087116431 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c140a7a80 con 0x7f8c2c15af60 2024-05-22T21:19:04.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.079+0000 7f8c2be00640 1 -- 172.21.15.161:0/1087116431 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c140a7be0 con 0x7f8c2c15af60 2024-05-22T21:19:04.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.079+0000 7f8c2be00640 1 -- 172.21.15.161:0/1087116431 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c140af440 con 0x7f8c2c15af60 2024-05-22T21:19:04.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.079+0000 7f8c33400640 1 -- 172.21.15.161:0/1087116431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c15af60 msgr2=0x7f8c2c13f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:04.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.079+0000 7f8c33400640 1 --2- 172.21.15.161:0/1087116431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c15af60 0x7f8c2c13f740 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400bb10 tx=0x7f8c1401eba0 comp rx=0 tx=0).stop 2024-05-22T21:19:04.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.079+0000 7f8c33400640 1 -- 172.21.15.161:0/1087116431 shutdown_connections 2024-05-22T21:19:04.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.079+0000 7f8c33400640 1 --2- 172.21.15.161:0/1087116431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c15af60 0x7f8c2c13f740 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:04.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.079+0000 7f8c33400640 1 --2- 172.21.15.161:0/1087116431 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c2c13cfc0 0x7f8c2c1402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:04.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.079+0000 7f8c33400640 1 -- 172.21.15.161:0/1087116431 >> 172.21.15.161:0/1087116431 conn(0x7f8c2c06a2d0 msgr2=0x7f8c2c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:04.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.079+0000 7f8c33400640 1 -- 172.21.15.161:0/1087116431 shutdown_connections 2024-05-22T21:19:04.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.079+0000 7f8c33400640 1 -- 172.21.15.161:0/1087116431 wait complete. 2024-05-22T21:19:04.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.079+0000 7f8c33400640 1 Processor -- start 2024-05-22T21:19:04.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c33400640 1 -- start start 2024-05-22T21:19:04.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c33400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c13cfc0 0x7f8c2c13f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c33400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c2c13fc80 0x7f8c2c154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c33400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c2c15af60 0x7f8c2c140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c33400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c13af80 con 0x7f8c2c13fc80 2024-05-22T21:19:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c33400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c13b260 con 0x7f8c2c13cfc0 2024-05-22T21:19:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c33400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c13b540 con 0x7f8c2c15af60 2024-05-22T21:19:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c23e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c2c13fc80 0x7f8c2c154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c30a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c13cfc0 0x7f8c2c13f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c30a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c13cfc0 0x7f8c2c13f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:53068/0 (socket says 172.21.15.161:53068) 2024-05-22T21:19:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c30a00640 1 -- 172.21.15.161:0/2642490999 learned_addr learned my addr 172.21.15.161:0/2642490999 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c23e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c2c13fc80 0x7f8c2c154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52046/0 (socket says 172.21.15.161:52046) 2024-05-22T21:19:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c31400640 1 --2- 172.21.15.161:0/2642490999 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c2c15af60 0x7f8c2c140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:04.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c30a00640 1 -- 172.21.15.161:0/2642490999 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c2c15af60 msgr2=0x7f8c2c140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:04.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c30a00640 1 --2- 172.21.15.161:0/2642490999 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c2c15af60 0x7f8c2c140270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:04.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c30a00640 1 -- 172.21.15.161:0/2642490999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c2c13fc80 msgr2=0x7f8c2c154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:04.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c30a00640 1 --2- 172.21.15.161:0/2642490999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c2c13fc80 0x7f8c2c154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:04.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c30a00640 1 -- 172.21.15.161:0/2642490999 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1400b770 con 0x7f8c2c13cfc0 2024-05-22T21:19:04.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c31400640 1 --2- 172.21.15.161:0/2642490999 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c2c15af60 0x7f8c2c140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:04.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c30a00640 1 --2- 172.21.15.161:0/2642490999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c13cfc0 0x7f8c2c13f2b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00e870 tx=0x7f8c1c00ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:04.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c29600640 1 -- 172.21.15.161:0/2642490999 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c009510 con 0x7f8c2c13cfc0 2024-05-22T21:19:04.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c29600640 1 -- 172.21.15.161:0/2642490999 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1c00eec0 con 0x7f8c2c13cfc0 2024-05-22T21:19:04.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c29600640 1 -- 172.21.15.161:0/2642490999 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c00cb40 con 0x7f8c2c13cfc0 2024-05-22T21:19:04.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c33400640 1 -- 172.21.15.161:0/2642490999 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c2c13b820 con 0x7f8c2c13cfc0 2024-05-22T21:19:04.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c33400640 1 -- 172.21.15.161:0/2642490999 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2c13f7f0 con 0x7f8c2c13cfc0 2024-05-22T21:19:04.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.083+0000 7f8c33400640 1 -- 172.21.15.161:0/2642490999 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2c13b9b0 con 0x7f8c2c13cfc0 2024-05-22T21:19:04.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.087+0000 7f8c29600640 1 -- 172.21.15.161:0/2642490999 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8c1c00cca0 con 0x7f8c2c13cfc0 2024-05-22T21:19:04.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.087+0000 7f8c29600640 1 --2- 172.21.15.161:0/2642490999 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c08066690 0x7f8c08068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:04.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.087+0000 7f8c29600640 1 -- 172.21.15.161:0/2642490999 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8c1c0847e0 con 0x7f8c2c13cfc0 2024-05-22T21:19:04.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.087+0000 7f8c23e00640 1 --2- 172.21.15.161:0/2642490999 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c08066690 0x7f8c08068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:04.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.087+0000 7f8c23e00640 1 --2- 172.21.15.161:0/2642490999 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c08066690 0x7f8c08068b50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8c140b1000 tx=0x7f8c14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:04.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.091+0000 7f8c29600640 1 -- 172.21.15.161:0/2642490999 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8c1c051af0 con 0x7f8c2c13cfc0 2024-05-22T21:19:04.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.275+0000 7f8c33400640 1 -- 172.21.15.161:0/2642490999 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8c2c15d5f0 con 0x7f8c2c13cfc0 2024-05-22T21:19:04.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.915+0000 7f8c29600640 1 -- 172.21.15.161:0/2642490999 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8c1c055110 con 0x7f8c2c13cfc0 2024-05-22T21:19:04.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.919+0000 7f8c33400640 1 -- 172.21.15.161:0/2642490999 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c08066690 msgr2=0x7f8c08068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:04.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.919+0000 7f8c33400640 1 --2- 172.21.15.161:0/2642490999 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c08066690 0x7f8c08068b50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8c140b1000 tx=0x7f8c14008040 comp rx=0 tx=0).stop 2024-05-22T21:19:04.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.919+0000 7f8c33400640 1 -- 172.21.15.161:0/2642490999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c13cfc0 msgr2=0x7f8c2c13f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:04.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.919+0000 7f8c33400640 1 --2- 172.21.15.161:0/2642490999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c13cfc0 0x7f8c2c13f2b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00e870 tx=0x7f8c1c00ec70 comp rx=0 tx=0).stop 2024-05-22T21:19:04.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.919+0000 7f8c33400640 1 -- 172.21.15.161:0/2642490999 shutdown_connections 2024-05-22T21:19:04.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.919+0000 7f8c33400640 1 --2- 172.21.15.161:0/2642490999 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c08066690 0x7f8c08068b50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:04.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.919+0000 7f8c33400640 1 --2- 172.21.15.161:0/2642490999 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c2c15af60 0x7f8c2c140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:04.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.919+0000 7f8c33400640 1 --2- 172.21.15.161:0/2642490999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c2c13fc80 0x7f8c2c154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:04.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.919+0000 7f8c33400640 1 --2- 172.21.15.161:0/2642490999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c13cfc0 0x7f8c2c13f2b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:04.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.919+0000 7f8c33400640 1 -- 172.21.15.161:0/2642490999 >> 172.21.15.161:0/2642490999 conn(0x7f8c2c06a2d0 msgr2=0x7f8c2c0af650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:04.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.919+0000 7f8c33400640 1 -- 172.21.15.161:0/2642490999 shutdown_connections 2024-05-22T21:19:04.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:04.919+0000 7f8c33400640 1 -- 172.21.15.161:0/2642490999 wait complete. 2024-05-22T21:19:04.942 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-22T21:19:05.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.119+0000 7fa779400640 1 Processor -- start 2024-05-22T21:19:05.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.119+0000 7fa779400640 1 -- start start 2024-05-22T21:19:05.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.119+0000 7fa779400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa77413d560 0x7fa7740cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:05.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.119+0000 7fa779400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa77415e1d0 0x7fa7740ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:05.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.119+0000 7fa779400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7740ce580 con 0x7fa77413cfe0 2024-05-22T21:19:05.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.119+0000 7fa779400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa774140560 con 0x7fa77413d560 2024-05-22T21:19:05.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.123+0000 7fa779400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740ce830 con 0x7fa77415e1d0 2024-05-22T21:19:05.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.123+0000 7fa769e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa77413d560 0x7fa7740cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:05.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.123+0000 7fa769e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa77413d560 0x7fa7740cdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52056/0 (socket says 172.21.15.161:52056) 2024-05-22T21:19:05.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.123+0000 7fa769e00640 1 -- 172.21.15.161:0/19009762 learned_addr learned my addr 172.21.15.161:0/19009762 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:05.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.123+0000 7fa772800640 1 --1- 172.21.15.161:0/19009762 >> v1:172.21.15.161:6789/0 conn(0x7fa77413cfe0 0x7fa77415dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:52032/0 (socket says 172.21.15.161:52032) 2024-05-22T21:19:05.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.123+0000 7fa773200640 1 --2- 172.21.15.161:0/19009762 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa77415e1d0 0x7fa7740ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:05.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.123+0000 7fa769e00640 1 -- 172.21.15.161:0/19009762 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa77415e1d0 msgr2=0x7fa7740ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:05.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.123+0000 7fa769e00640 1 --2- 172.21.15.161:0/19009762 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa77415e1d0 0x7fa7740ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:05.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.123+0000 7fa769e00640 1 -- 172.21.15.161:0/19009762 >> v1:172.21.15.161:6789/0 conn(0x7fa77413cfe0 legacy=0x7fa77415dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:05.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.123+0000 7fa771e00640 1 -- 172.21.15.161:0/19009762 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474089735 0 0) 0x7fa7740ce580 con 0x7fa77413cfe0 2024-05-22T21:19:05.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.123+0000 7fa769e00640 1 -- 172.21.15.161:0/19009762 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7740ceb10 con 0x7fa77413d560 2024-05-22T21:19:05.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.123+0000 7fa773200640 1 --2- 172.21.15.161:0/19009762 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa77415e1d0 0x7fa7740ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:05.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.123+0000 7fa769e00640 1 --2- 172.21.15.161:0/19009762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77413d560 0x7fa7740cdb00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00bb10 tx=0x7fa75c01eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45006a55a889ca97 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:05.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.123+0000 7fa771e00640 1 -- 172.21.15.161:0/19009762 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0a7c40 con 0x7fa77413d560 2024-05-22T21:19:05.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.123+0000 7fa771e00640 1 -- 172.21.15.161:0/19009762 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c0a7da0 con 0x7fa77413d560 2024-05-22T21:19:05.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.123+0000 7fa771e00640 1 -- 172.21.15.161:0/19009762 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0af660 con 0x7fa77413d560 2024-05-22T21:19:05.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa779400640 1 -- 172.21.15.161:0/19009762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77413d560 msgr2=0x7fa7740cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:05.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa779400640 1 --2- 172.21.15.161:0/19009762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77413d560 0x7fa7740cdb00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00bb10 tx=0x7fa75c01eb70 comp rx=0 tx=0).stop 2024-05-22T21:19:05.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa779400640 1 -- 172.21.15.161:0/19009762 shutdown_connections 2024-05-22T21:19:05.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa779400640 1 --2- 172.21.15.161:0/19009762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77413d560 0x7fa7740cdb00 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:05.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa779400640 1 --2- 172.21.15.161:0/19009762 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa77415e1d0 0x7fa7740ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:05.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa779400640 1 -- 172.21.15.161:0/19009762 >> 172.21.15.161:0/19009762 conn(0x7fa77406a0a0 msgr2=0x7fa77406a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:05.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa779400640 1 -- 172.21.15.161:0/19009762 shutdown_connections 2024-05-22T21:19:05.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa779400640 1 -- 172.21.15.161:0/19009762 wait complete. 2024-05-22T21:19:05.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa779400640 1 Processor -- start 2024-05-22T21:19:05.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa779400640 1 -- start start 2024-05-22T21:19:05.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa779400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77413cfe0 0x7fa7741526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:05.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa779400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa77413d560 0x7fa77413a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:05.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa779400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa77415e1d0 0x7fa77413ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:05.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa779400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa774153170 con 0x7fa77413cfe0 2024-05-22T21:19:05.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa779400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa774153450 con 0x7fa77415e1d0 2024-05-22T21:19:05.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa779400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa774153730 con 0x7fa77413d560 2024-05-22T21:19:05.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa772800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77413cfe0 0x7fa7741526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:05.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa772800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77413cfe0 0x7fa7741526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52060/0 (socket says 172.21.15.161:52060) 2024-05-22T21:19:05.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa772800640 1 -- 172.21.15.161:0/1685361667 learned_addr learned my addr 172.21.15.161:0/1685361667 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:05.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa772800640 1 -- 172.21.15.161:0/1685361667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa77413d560 msgr2=0x7fa77413a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:05.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa772800640 1 --2- 172.21.15.161:0/1685361667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa77413d560 0x7fa77413a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:05.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa772800640 1 -- 172.21.15.161:0/1685361667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa77415e1d0 msgr2=0x7fa77413ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:05.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa772800640 1 --2- 172.21.15.161:0/1685361667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa77415e1d0 0x7fa77413ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:05.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa772800640 1 -- 172.21.15.161:0/1685361667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c00b770 con 0x7fa77413cfe0 2024-05-22T21:19:05.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.127+0000 7fa772800640 1 --2- 172.21.15.161:0/1685361667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77413cfe0 0x7fa7741526d0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa7640011f0 tx=0x7fa76400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:05.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.131+0000 7fa76b400640 1 -- 172.21.15.161:0/1685361667 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7640074e0 con 0x7fa77413cfe0 2024-05-22T21:19:05.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.131+0000 7fa76b400640 1 -- 172.21.15.161:0/1685361667 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa764007640 con 0x7fa77413cfe0 2024-05-22T21:19:05.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.131+0000 7fa76b400640 1 -- 172.21.15.161:0/1685361667 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764004a10 con 0x7fa77413cfe0 2024-05-22T21:19:05.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.131+0000 7fa779400640 1 -- 172.21.15.161:0/1685361667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa774153a10 con 0x7fa77413cfe0 2024-05-22T21:19:05.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.131+0000 7fa779400640 1 -- 172.21.15.161:0/1685361667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa774135cc0 con 0x7fa77413cfe0 2024-05-22T21:19:05.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.131+0000 7fa779400640 1 -- 172.21.15.161:0/1685361667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa774067220 con 0x7fa77413cfe0 2024-05-22T21:19:05.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.131+0000 7fa76b400640 1 -- 172.21.15.161:0/1685361667 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa764004b70 con 0x7fa77413cfe0 2024-05-22T21:19:05.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.131+0000 7fa76b400640 1 --2- 172.21.15.161:0/1685361667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa750066690 0x7fa750068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:05.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.131+0000 7fa76b400640 1 -- 172.21.15.161:0/1685361667 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa7640834c0 con 0x7fa77413cfe0 2024-05-22T21:19:05.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.131+0000 7fa769e00640 1 --2- 172.21.15.161:0/1685361667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa750066690 0x7fa750068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:05.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.131+0000 7fa769e00640 1 --2- 172.21.15.161:0/1685361667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa750066690 0x7fa750068b50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0b1000 tx=0x7fa75c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:05.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.135+0000 7fa76b400640 1 -- 172.21.15.161:0/1685361667 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa7640507c0 con 0x7fa77413cfe0 2024-05-22T21:19:05.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.295+0000 7fa779400640 1 -- 172.21.15.161:0/1685361667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa774071d50 con 0x7fa77413cfe0 2024-05-22T21:19:05.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.923+0000 7fa76b400640 1 -- 172.21.15.161:0/1685361667 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v115) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa7640180d0 con 0x7fa77413cfe0 2024-05-22T21:19:05.926 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' removed 2024-05-22T21:19:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.923+0000 7fa779400640 1 -- 172.21.15.161:0/1685361667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa750066690 msgr2=0x7fa750068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.923+0000 7fa779400640 1 --2- 172.21.15.161:0/1685361667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa750066690 0x7fa750068b50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0b1000 tx=0x7fa75c008040 comp rx=0 tx=0).stop 2024-05-22T21:19:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.923+0000 7fa779400640 1 -- 172.21.15.161:0/1685361667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77413cfe0 msgr2=0x7fa7741526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.923+0000 7fa779400640 1 --2- 172.21.15.161:0/1685361667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77413cfe0 0x7fa7741526d0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa7640011f0 tx=0x7fa76400c9b0 comp rx=0 tx=0).stop 2024-05-22T21:19:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.923+0000 7fa779400640 1 -- 172.21.15.161:0/1685361667 shutdown_connections 2024-05-22T21:19:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.923+0000 7fa779400640 1 --2- 172.21.15.161:0/1685361667 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa750066690 0x7fa750068b50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.923+0000 7fa779400640 1 --2- 172.21.15.161:0/1685361667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa77415e1d0 0x7fa77413ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.923+0000 7fa779400640 1 --2- 172.21.15.161:0/1685361667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa77413d560 0x7fa77413a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:05.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.923+0000 7fa779400640 1 --2- 172.21.15.161:0/1685361667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa77413cfe0 0x7fa7741526d0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:05.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.923+0000 7fa779400640 1 -- 172.21.15.161:0/1685361667 >> 172.21.15.161:0/1685361667 conn(0x7fa77406a0a0 msgr2=0x7fa77415c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:05.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.927+0000 7fa779400640 1 -- 172.21.15.161:0/1685361667 shutdown_connections 2024-05-22T21:19:05.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:05.927+0000 7fa779400640 1 -- 172.21.15.161:0/1685361667 wait complete. 2024-05-22T21:19:05.948 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-22T21:19:06.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.087+0000 7ff813800640 1 Processor -- start 2024-05-22T21:19:06.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.087+0000 7ff813800640 1 -- start start 2024-05-22T21:19:06.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.087+0000 7ff813800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff80c144980 0x7ff80c15c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:06.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff813800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff80c06d560 0x7ff80c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:06.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff813800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80c0cce90 con 0x7ff80c0cca40 2024-05-22T21:19:06.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff813800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff80c06d0d0 con 0x7ff80c06d560 2024-05-22T21:19:06.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff813800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff80c15c680 con 0x7ff80c144980 2024-05-22T21:19:06.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff80be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff80c06d560 0x7ff80c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:06.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff80be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff80c06d560 0x7ff80c0cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48922/0 (socket says 172.21.15.161:48922) 2024-05-22T21:19:06.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff80be00640 1 -- 172.21.15.161:0/3012317884 learned_addr learned my addr 172.21.15.161:0/3012317884 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:06.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff811800640 1 --1- 172.21.15.161:0/3012317884 >> v1:172.21.15.161:6789/0 conn(0x7ff80c0cca40 0x7ff80c15c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41448/0 (socket says 172.21.15.161:41448) 2024-05-22T21:19:06.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff810e00640 1 --2- 172.21.15.161:0/3012317884 >> v2:172.21.15.179:3300/0 conn(0x7ff80c144980 0x7ff80c15c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:06.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff80be00640 1 -- 172.21.15.161:0/3012317884 >> v2:172.21.15.179:3300/0 conn(0x7ff80c144980 msgr2=0x7ff80c15c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:06.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff80be00640 1 --2- 172.21.15.161:0/3012317884 >> v2:172.21.15.179:3300/0 conn(0x7ff80c144980 0x7ff80c15c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:06.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff80be00640 1 -- 172.21.15.161:0/3012317884 >> v1:172.21.15.161:6789/0 conn(0x7ff80c0cca40 legacy=0x7ff80c15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:06.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff80be00640 1 -- 172.21.15.161:0/3012317884 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80c06c4c0 con 0x7ff80c06d560 2024-05-22T21:19:06.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff80b400640 1 -- 172.21.15.161:0/3012317884 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252575680 0 0) 0x7ff804001120 con 0x7ff80c0cca40 2024-05-22T21:19:06.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff80be00640 1 --2- 172.21.15.161:0/3012317884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c06d560 0x7ff80c0cc500 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff80c071fd0 tx=0x7ff7f801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ff69ae84e9919a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:06.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff80b400640 1 -- 172.21.15.161:0/3012317884 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f80ab610 con 0x7ff80c06d560 2024-05-22T21:19:06.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff80b400640 1 -- 172.21.15.161:0/3012317884 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f80ab770 con 0x7ff80c06d560 2024-05-22T21:19:06.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.091+0000 7ff80b400640 1 -- 172.21.15.161:0/3012317884 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f80abb00 con 0x7ff80c06d560 2024-05-22T21:19:06.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff813800640 1 -- 172.21.15.161:0/3012317884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c06d560 msgr2=0x7ff80c0cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:06.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff813800640 1 --2- 172.21.15.161:0/3012317884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c06d560 0x7ff80c0cc500 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff80c071fd0 tx=0x7ff7f801c520 comp rx=0 tx=0).stop 2024-05-22T21:19:06.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff813800640 1 -- 172.21.15.161:0/3012317884 shutdown_connections 2024-05-22T21:19:06.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff813800640 1 --2- 172.21.15.161:0/3012317884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c06d560 0x7ff80c0cc500 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:06.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff813800640 1 --2- 172.21.15.161:0/3012317884 >> v2:172.21.15.179:3300/0 conn(0x7ff80c144980 0x7ff80c15c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:06.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff813800640 1 -- 172.21.15.161:0/3012317884 >> 172.21.15.161:0/3012317884 conn(0x7ff80c06a0a0 msgr2=0x7ff80c06a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:06.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff813800640 1 -- 172.21.15.161:0/3012317884 shutdown_connections 2024-05-22T21:19:06.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff813800640 1 -- 172.21.15.161:0/3012317884 wait complete. 2024-05-22T21:19:06.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff813800640 1 Processor -- start 2024-05-22T21:19:06.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff813800640 1 -- start start 2024-05-22T21:19:06.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff813800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff80c06d560 0x7ff80c0ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:06.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff813800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff80c0cca40 0x7ff80c1549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:06.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff813800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c144980 0x7ff80c156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:06.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff813800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c154f10 con 0x7ff80c144980 2024-05-22T21:19:06.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff813800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c157400 con 0x7ff80c06d560 2024-05-22T21:19:06.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff813800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c1576c0 con 0x7ff80c0cca40 2024-05-22T21:19:06.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff811800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c144980 0x7ff80c156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:06.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff811800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c144980 0x7ff80c156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48926/0 (socket says 172.21.15.161:48926) 2024-05-22T21:19:06.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff811800640 1 -- 172.21.15.161:0/2108876584 learned_addr learned my addr 172.21.15.161:0/2108876584 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:06.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff810e00640 1 --2- 172.21.15.161:0/2108876584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff80c06d560 0x7ff80c0ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:06.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff80be00640 1 --2- 172.21.15.161:0/2108876584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff80c0cca40 0x7ff80c1549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:06.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff811800640 1 -- 172.21.15.161:0/2108876584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff80c0cca40 msgr2=0x7ff80c1549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:06.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff811800640 1 --2- 172.21.15.161:0/2108876584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff80c0cca40 0x7ff80c1549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:06.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff811800640 1 -- 172.21.15.161:0/2108876584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff80c06d560 msgr2=0x7ff80c0ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:06.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff811800640 1 --2- 172.21.15.161:0/2108876584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff80c06d560 0x7ff80c0ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:06.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff811800640 1 -- 172.21.15.161:0/2108876584 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f80094d0 con 0x7ff80c144980 2024-05-22T21:19:06.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff80be00640 1 --2- 172.21.15.161:0/2108876584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff80c0cca40 0x7ff80c1549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:06.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.095+0000 7ff810e00640 1 --2- 172.21.15.161:0/2108876584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff80c06d560 0x7ff80c0ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:06.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.099+0000 7ff811800640 1 --2- 172.21.15.161:0/2108876584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c144980 0x7ff80c156ec0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff80400c780 tx=0x7ff80400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:06.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.099+0000 7ff808c00640 1 -- 172.21.15.161:0/2108876584 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8040072d0 con 0x7ff80c144980 2024-05-22T21:19:06.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.099+0000 7ff808c00640 1 -- 172.21.15.161:0/2108876584 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff804007450 con 0x7ff80c144980 2024-05-22T21:19:06.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.099+0000 7ff808c00640 1 -- 172.21.15.161:0/2108876584 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804001d20 con 0x7ff80c144980 2024-05-22T21:19:06.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.099+0000 7ff813800640 1 -- 172.21.15.161:0/2108876584 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80c1579a0 con 0x7ff80c144980 2024-05-22T21:19:06.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.099+0000 7ff813800640 1 -- 172.21.15.161:0/2108876584 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80c157f60 con 0x7ff80c144980 2024-05-22T21:19:06.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.099+0000 7ff813800640 1 -- 172.21.15.161:0/2108876584 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80c067220 con 0x7ff80c144980 2024-05-22T21:19:06.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.099+0000 7ff808c00640 1 -- 172.21.15.161:0/2108876584 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff8040047f0 con 0x7ff80c144980 2024-05-22T21:19:06.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.099+0000 7ff808c00640 1 --2- 172.21.15.161:0/2108876584 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff7ec066690 0x7ff7ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:06.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.099+0000 7ff808c00640 1 -- 172.21.15.161:0/2108876584 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5357+0+0 (secure 0 0 0) 0x7ff804083110 con 0x7ff80c144980 2024-05-22T21:19:06.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.099+0000 7ff810e00640 1 --2- 172.21.15.161:0/2108876584 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff7ec066690 0x7ff7ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:06.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.103+0000 7ff810e00640 1 --2- 172.21.15.161:0/2108876584 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff7ec066690 0x7ff7ec068b50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff80c15e1f0 tx=0x7ff7fc00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:06.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.103+0000 7ff808c00640 1 -- 172.21.15.161:0/2108876584 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff8040505d0 con 0x7ff80c144980 2024-05-22T21:19:06.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.259+0000 7ff813800640 1 -- 172.21.15.161:0/2108876584 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff80c0cce50 con 0x7ff80c144980 2024-05-22T21:19:06.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.931+0000 7ff808c00640 1 -- 172.21.15.161:0/2108876584 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v116) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff804053bf0 con 0x7ff80c144980 2024-05-22T21:19:06.935 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' removed 2024-05-22T21:19:06.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.935+0000 7ff813800640 1 -- 172.21.15.161:0/2108876584 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff7ec066690 msgr2=0x7ff7ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:06.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.935+0000 7ff813800640 1 --2- 172.21.15.161:0/2108876584 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff7ec066690 0x7ff7ec068b50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff80c15e1f0 tx=0x7ff7fc00b040 comp rx=0 tx=0).stop 2024-05-22T21:19:06.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.935+0000 7ff813800640 1 -- 172.21.15.161:0/2108876584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c144980 msgr2=0x7ff80c156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:06.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.935+0000 7ff813800640 1 --2- 172.21.15.161:0/2108876584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c144980 0x7ff80c156ec0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff80400c780 tx=0x7ff80400cc40 comp rx=0 tx=0).stop 2024-05-22T21:19:06.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.935+0000 7ff813800640 1 -- 172.21.15.161:0/2108876584 shutdown_connections 2024-05-22T21:19:06.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.935+0000 7ff813800640 1 --2- 172.21.15.161:0/2108876584 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff7ec066690 0x7ff7ec068b50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:06.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.935+0000 7ff813800640 1 --2- 172.21.15.161:0/2108876584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c144980 0x7ff80c156ec0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:06.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.935+0000 7ff813800640 1 --2- 172.21.15.161:0/2108876584 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff80c0cca40 0x7ff80c1549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:06.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.935+0000 7ff813800640 1 --2- 172.21.15.161:0/2108876584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff80c06d560 0x7ff80c0ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:06.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.935+0000 7ff813800640 1 -- 172.21.15.161:0/2108876584 >> 172.21.15.161:0/2108876584 conn(0x7ff80c06a0a0 msgr2=0x7ff80c15e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:06.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.935+0000 7ff813800640 1 -- 172.21.15.161:0/2108876584 shutdown_connections 2024-05-22T21:19:06.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:06.935+0000 7ff813800640 1 -- 172.21.15.161:0/2108876584 wait complete. 2024-05-22T21:19:06.957 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-22T21:19:07.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.091+0000 7f5a61a00640 1 Processor -- start 2024-05-22T21:19:07.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.091+0000 7f5a61a00640 1 -- start start 2024-05-22T21:19:07.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.091+0000 7f5a61a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a5c0cdf70 0x7f5a5c0cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:07.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.091+0000 7f5a61a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a5c0b23f0 con 0x7f5a5c15e7b0 2024-05-22T21:19:07.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.091+0000 7f5a61a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a5c0b2690 con 0x7f5a5c0b01f0 2024-05-22T21:19:07.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.091+0000 7f5a61a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0cd230 con 0x7f5a5c0cdf70 2024-05-22T21:19:07.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5be00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f5a5c15e7b0 0x7f5a5c0cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41450/0 (socket says 172.21.15.161:41450) 2024-05-22T21:19:07.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5be00640 1 -- 172.21.15.161:0/3723060741 learned_addr learned my addr 172.21.15.161:0/3723060741 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:07.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a53400640 1 --2- 172.21.15.161:0/3723060741 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a5c0cdf70 0x7f5a5c0cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:07.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5b400640 1 -- 172.21.15.161:0/3723060741 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290619911 0 0) 0x7f5a5c0b23f0 con 0x7f5a5c15e7b0 2024-05-22T21:19:07.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5b400640 1 -- 172.21.15.161:0/3723060741 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a40003530 con 0x7f5a5c15e7b0 2024-05-22T21:19:07.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5b400640 1 -- 172.21.15.161:0/3723060741 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1404509787 0 0) 0x7f5a40003530 con 0x7f5a5c15e7b0 2024-05-22T21:19:07.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5b400640 1 -- 172.21.15.161:0/3723060741 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a5c0cdf70 msgr2=0x7f5a5c0cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:07.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5b400640 1 --2- 172.21.15.161:0/3723060741 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a5c0cdf70 0x7f5a5c0cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5b400640 1 -- 172.21.15.161:0/3723060741 >> v1:172.21.15.179:6789/0 conn(0x7f5a5c0b01f0 legacy=0x7f5a5c0cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:07.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5b400640 1 -- 172.21.15.161:0/3723060741 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c0cd510 con 0x7f5a5c15e7b0 2024-05-22T21:19:07.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5b400640 1 -- 172.21.15.161:0/3723060741 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f5a4c001a00 con 0x7f5a5c15e7b0 2024-05-22T21:19:07.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5b400640 1 -- 172.21.15.161:0/3723060741 >> v1:172.21.15.161:6789/0 conn(0x7f5a5c15e7b0 legacy=0x7f5a5c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:07.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5b400640 1 --2- 172.21.15.161:0/3723060741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a400039b0 0x7f5a40003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:07.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5b400640 1 --2- 172.21.15.161:0/3723060741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a40004360 0x7f5a40004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:07.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5b400640 1 --2- 172.21.15.161:0/3723060741 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a5c0cdf70 0x7f5a40005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:07.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5b400640 1 -- 172.21.15.161:0/3723060741 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0cd230 con 0x7f5a40004360 2024-05-22T21:19:07.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5b400640 1 -- 172.21.15.161:0/3723060741 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4000b990 con 0x7f5a400039b0 2024-05-22T21:19:07.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5b400640 1 -- 172.21.15.161:0/3723060741 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4000bb50 con 0x7f5a5c0cdf70 2024-05-22T21:19:07.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a53400640 1 --2- 172.21.15.161:0/3723060741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a40004360 0x7f5a40004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:07.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a5be00640 1 --2- 172.21.15.161:0/3723060741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a400039b0 0x7f5a40003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:07.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a61000640 1 --2- 172.21.15.161:0/3723060741 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a5c0cdf70 0x7f5a40005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:07.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a53400640 1 -- 172.21.15.161:0/3723060741 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a5c0cdf70 msgr2=0x7f5a40005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:07.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a53400640 1 --2- 172.21.15.161:0/3723060741 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a5c0cdf70 0x7f5a40005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a53400640 1 -- 172.21.15.161:0/3723060741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a400039b0 msgr2=0x7f5a40003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:07.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a53400640 1 --2- 172.21.15.161:0/3723060741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a400039b0 0x7f5a40003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.095+0000 7f5a53400640 1 -- 172.21.15.161:0/3723060741 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a40003530 con 0x7f5a40004360 2024-05-22T21:19:07.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.099+0000 7f5a53400640 1 --2- 172.21.15.161:0/3723060741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a40004360 0x7f5a40004830 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009ae0 tx=0x7f5a4401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0c73f4cef0a6c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:07.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.099+0000 7f5a5b400640 1 -- 172.21.15.161:0/3723060741 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a440a5660 con 0x7f5a40004360 2024-05-22T21:19:07.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.099+0000 7f5a5b400640 1 -- 172.21.15.161:0/3723060741 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a440a57c0 con 0x7f5a40004360 2024-05-22T21:19:07.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.099+0000 7f5a5b400640 1 -- 172.21.15.161:0/3723060741 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a440a5b70 con 0x7f5a40004360 2024-05-22T21:19:07.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.099+0000 7f5a61a00640 1 -- 172.21.15.161:0/3723060741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a40004360 msgr2=0x7f5a40004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:07.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.099+0000 7f5a61a00640 1 --2- 172.21.15.161:0/3723060741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a40004360 0x7f5a40004830 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009ae0 tx=0x7f5a4401ca70 comp rx=0 tx=0).stop 2024-05-22T21:19:07.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.099+0000 7f5a61000640 1 -- 172.21.15.161:0/3723060741 reap_dead start 2024-05-22T21:19:07.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.099+0000 7f5a61a00640 1 -- 172.21.15.161:0/3723060741 shutdown_connections 2024-05-22T21:19:07.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.099+0000 7f5a61a00640 1 -- 172.21.15.161:0/3723060741 >> 172.21.15.161:0/3723060741 conn(0x7f5a5c0b29a0 msgr2=0x7f5a5c0b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:07.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.099+0000 7f5a61a00640 1 -- 172.21.15.161:0/3723060741 shutdown_connections 2024-05-22T21:19:07.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.099+0000 7f5a61a00640 1 -- 172.21.15.161:0/3723060741 wait complete. 2024-05-22T21:19:07.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.099+0000 7f5a61a00640 1 Processor -- start 2024-05-22T21:19:07.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.099+0000 7f5a61a00640 1 -- start start 2024-05-22T21:19:07.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.099+0000 7f5a61a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a40004360 0x7f5a5c159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:07.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.099+0000 7f5a61a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a5c1599d0 0x7f5a5c1473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:07.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a61a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a5c0af3c0 0x7f5a5c1478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:07.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a61a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0aef30 con 0x7f5a5c0af3c0 2024-05-22T21:19:07.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a61a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0ae660 con 0x7f5a5c1599d0 2024-05-22T21:19:07.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a61a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c148240 con 0x7f5a40004360 2024-05-22T21:19:07.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a61000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a5c0af3c0 0x7f5a5c1478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:07.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a61000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a5c0af3c0 0x7f5a5c1478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48944/0 (socket says 172.21.15.161:48944) 2024-05-22T21:19:07.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a61000640 1 -- 172.21.15.161:0/3195816849 learned_addr learned my addr 172.21.15.161:0/3195816849 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:07.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a61000640 1 -- 172.21.15.161:0/3195816849 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a40004360 msgr2=0x7f5a5c159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:07.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a61000640 1 --2- 172.21.15.161:0/3195816849 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a40004360 0x7f5a5c159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a61000640 1 -- 172.21.15.161:0/3195816849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a5c1599d0 msgr2=0x7f5a5c1473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:07.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a61000640 1 --2- 172.21.15.161:0/3195816849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a5c1599d0 0x7f5a5c1473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a61000640 1 -- 172.21.15.161:0/3195816849 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c159f40 con 0x7f5a5c0af3c0 2024-05-22T21:19:07.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a61000640 1 --2- 172.21.15.161:0/3195816849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a5c0af3c0 0x7f5a5c1478e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f5a54007fd0 tx=0x7f5a5400ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:07.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a58c00640 1 -- 172.21.15.161:0/3195816849 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54005320 con 0x7f5a5c0af3c0 2024-05-22T21:19:07.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a58c00640 1 -- 172.21.15.161:0/3195816849 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a54005480 con 0x7f5a5c0af3c0 2024-05-22T21:19:07.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a61a00640 1 -- 172.21.15.161:0/3195816849 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5c0ae8e0 con 0x7f5a5c0af3c0 2024-05-22T21:19:07.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a58c00640 1 -- 172.21.15.161:0/3195816849 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54006a20 con 0x7f5a5c0af3c0 2024-05-22T21:19:07.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a61a00640 1 -- 172.21.15.161:0/3195816849 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5c159760 con 0x7f5a5c0af3c0 2024-05-22T21:19:07.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a61a00640 1 -- 172.21.15.161:0/3195816849 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5c073af0 con 0x7f5a5c0af3c0 2024-05-22T21:19:07.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.103+0000 7f5a58c00640 1 -- 172.21.15.161:0/3195816849 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5a54006b80 con 0x7f5a5c0af3c0 2024-05-22T21:19:07.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.107+0000 7f5a58c00640 1 --2- 172.21.15.161:0/3195816849 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5a38066690 0x7f5a38068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:07.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.107+0000 7f5a58c00640 1 -- 172.21.15.161:0/3195816849 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5a54086940 con 0x7f5a5c0af3c0 2024-05-22T21:19:07.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.107+0000 7f5a5be00640 1 --2- 172.21.15.161:0/3195816849 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5a38066690 0x7f5a38068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:07.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.107+0000 7f5a5be00640 1 --2- 172.21.15.161:0/3195816849 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5a38066690 0x7f5a38068b50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c001570 tx=0x7f5a4c0014e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:07.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.111+0000 7f5a58c00640 1 -- 172.21.15.161:0/3195816849 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5a54053fa0 con 0x7f5a5c0af3c0 2024-05-22T21:19:07.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.263+0000 7f5a61a00640 1 -- 172.21.15.161:0/3195816849 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5a5c067070 con 0x7f5a5c0af3c0 2024-05-22T21:19:07.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.271+0000 7f5a58c00640 1 -- 172.21.15.161:0/3195816849 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5a5401b0a0 con 0x7f5a5c0af3c0 2024-05-22T21:19:07.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.271+0000 7f5a61a00640 1 -- 172.21.15.161:0/3195816849 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5a38066690 msgr2=0x7f5a38068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:07.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.271+0000 7f5a61a00640 1 --2- 172.21.15.161:0/3195816849 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5a38066690 0x7f5a38068b50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c001570 tx=0x7f5a4c0014e0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.271+0000 7f5a61a00640 1 -- 172.21.15.161:0/3195816849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a5c0af3c0 msgr2=0x7f5a5c1478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:07.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.271+0000 7f5a61a00640 1 --2- 172.21.15.161:0/3195816849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a5c0af3c0 0x7f5a5c1478e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f5a54007fd0 tx=0x7f5a5400ae80 comp rx=0 tx=0).stop 2024-05-22T21:19:07.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.271+0000 7f5a61a00640 1 -- 172.21.15.161:0/3195816849 shutdown_connections 2024-05-22T21:19:07.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.271+0000 7f5a61a00640 1 --2- 172.21.15.161:0/3195816849 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5a38066690 0x7f5a38068b50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.271+0000 7f5a61a00640 1 --2- 172.21.15.161:0/3195816849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a5c0af3c0 0x7f5a5c1478e0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.271+0000 7f5a61a00640 1 --2- 172.21.15.161:0/3195816849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a5c1599d0 0x7f5a5c1473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.271+0000 7f5a61a00640 1 --2- 172.21.15.161:0/3195816849 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a40004360 0x7f5a5c159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.271+0000 7f5a61a00640 1 -- 172.21.15.161:0/3195816849 >> 172.21.15.161:0/3195816849 conn(0x7f5a5c0b29a0 msgr2=0x7f5a5c0b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:07.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.271+0000 7f5a61a00640 1 -- 172.21.15.161:0/3195816849 shutdown_connections 2024-05-22T21:19:07.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.271+0000 7f5a61a00640 1 -- 172.21.15.161:0/3195816849 wait complete. 2024-05-22T21:19:07.295 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2024-05-22T21:19:07.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.431+0000 7f9fdb200640 1 Processor -- start 2024-05-22T21:19:07.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.431+0000 7f9fdb200640 1 -- start start 2024-05-22T21:19:07.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.431+0000 7f9fdb200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9fd40b3560 0x7f9fd40cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:07.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.431+0000 7f9fdb200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd4070050 con 0x7f9fd415f720 2024-05-22T21:19:07.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.431+0000 7f9fdb200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd406e760 con 0x7f9fd4099ca0 2024-05-22T21:19:07.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.431+0000 7f9fdb200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd40cbfa0 con 0x7f9fd40b3560 2024-05-22T21:19:07.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.431+0000 7f9fcb400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9fd40b3560 0x7f9fd40cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:07.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.431+0000 7f9fd9200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f9fd415f720 0x7f9fd406ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41466/0 (socket says 172.21.15.161:41466) 2024-05-22T21:19:07.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.431+0000 7f9fd9200640 1 -- 172.21.15.161:0/723906509 learned_addr learned my addr 172.21.15.161:0/723906509 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:07.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.431+0000 7f9fcb400640 1 --2- 172.21.15.161:0/723906509 >> v2:172.21.15.161:3300/0 conn(0x7f9fd40b3560 0x7f9fd40cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48960/0 (socket says 172.21.15.161:48960) 2024-05-22T21:19:07.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.431+0000 7f9fcb400640 1 -- 172.21.15.161:0/723906509 >> v1:172.21.15.179:6789/0 conn(0x7f9fd4099ca0 legacy=0x7f9fd409a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:07.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.431+0000 7f9fcb400640 1 -- 172.21.15.161:0/723906509 >> v1:172.21.15.161:6789/0 conn(0x7f9fd415f720 legacy=0x7f9fd406ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:07.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.431+0000 7f9fcb400640 1 -- 172.21.15.161:0/723906509 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd406fab0 con 0x7f9fd40b3560 2024-05-22T21:19:07.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.431+0000 7f9fd3400640 1 -- 172.21.15.161:0/723906509 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771320843 0 0) 0x7f9fcc0014a0 con 0x7f9fd415f720 2024-05-22T21:19:07.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.435+0000 7f9fcb400640 1 --2- 172.21.15.161:0/723906509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fd40b3560 0x7f9fd40cba60 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0097c0 tx=0x7f9fbc01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcdd9cd97f55724a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:07.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.435+0000 7f9fd3400640 1 -- 172.21.15.161:0/723906509 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc0a8990 con 0x7f9fd40b3560 2024-05-22T21:19:07.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.435+0000 7f9fd3400640 1 -- 172.21.15.161:0/723906509 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fbc0a8af0 con 0x7f9fd40b3560 2024-05-22T21:19:07.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.435+0000 7f9fd3400640 1 -- 172.21.15.161:0/723906509 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc0a8e80 con 0x7f9fd40b3560 2024-05-22T21:19:07.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.435+0000 7f9fdb200640 1 -- 172.21.15.161:0/723906509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fd40b3560 msgr2=0x7f9fd40cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:07.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.435+0000 7f9fdb200640 1 --2- 172.21.15.161:0/723906509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fd40b3560 0x7f9fd40cba60 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0097c0 tx=0x7f9fbc01ec90 comp rx=0 tx=0).stop 2024-05-22T21:19:07.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.435+0000 7f9fdb200640 1 -- 172.21.15.161:0/723906509 shutdown_connections 2024-05-22T21:19:07.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.435+0000 7f9fdb200640 1 --2- 172.21.15.161:0/723906509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fd40b3560 0x7f9fd40cba60 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.435+0000 7f9fdb200640 1 -- 172.21.15.161:0/723906509 >> 172.21.15.161:0/723906509 conn(0x7f9fd40b2a40 msgr2=0x7f9fd40b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:07.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.435+0000 7f9fdb200640 1 -- 172.21.15.161:0/723906509 shutdown_connections 2024-05-22T21:19:07.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.435+0000 7f9fdb200640 1 -- 172.21.15.161:0/723906509 wait complete. 2024-05-22T21:19:07.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fdb200640 1 Processor -- start 2024-05-22T21:19:07.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fdb200640 1 -- start start 2024-05-22T21:19:07.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fdb200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fd4099ca0 0x7f9fd414f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:07.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fdb200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9fd40b3560 0x7f9fd406b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:07.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fdb200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fd415f720 0x7f9fd41e5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:07.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fdb200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd406be20 con 0x7f9fd4099ca0 2024-05-22T21:19:07.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fdb200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd406c100 con 0x7f9fd415f720 2024-05-22T21:19:07.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fdb200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd406c3e0 con 0x7f9fd40b3560 2024-05-22T21:19:07.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fd3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fd4099ca0 0x7f9fd414f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:07.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fd3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fd4099ca0 0x7f9fd414f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48976/0 (socket says 172.21.15.161:48976) 2024-05-22T21:19:07.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fd3e00640 1 -- 172.21.15.161:0/1418129873 learned_addr learned my addr 172.21.15.161:0/1418129873 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:07.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fd3e00640 1 -- 172.21.15.161:0/1418129873 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9fd40b3560 msgr2=0x7f9fd406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:07.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fd3e00640 1 --2- 172.21.15.161:0/1418129873 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9fd40b3560 0x7f9fd406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fd3e00640 1 -- 172.21.15.161:0/1418129873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fd415f720 msgr2=0x7f9fd41e5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:07.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fd3e00640 1 --2- 172.21.15.161:0/1418129873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fd415f720 0x7f9fd41e5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fd3e00640 1 -- 172.21.15.161:0/1418129873 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc009420 con 0x7f9fd4099ca0 2024-05-22T21:19:07.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fd3e00640 1 --2- 172.21.15.161:0/1418129873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fd4099ca0 0x7f9fd414f380 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400c3c0 tx=0x7f9fc400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:07.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fd0c00640 1 -- 172.21.15.161:0/1418129873 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40052d0 con 0x7f9fd4099ca0 2024-05-22T21:19:07.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fd0c00640 1 -- 172.21.15.161:0/1418129873 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc4005430 con 0x7f9fd4099ca0 2024-05-22T21:19:07.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fd0c00640 1 -- 172.21.15.161:0/1418129873 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4003840 con 0x7f9fd4099ca0 2024-05-22T21:19:07.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fdb200640 1 -- 172.21.15.161:0/1418129873 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd406c6c0 con 0x7f9fd4099ca0 2024-05-22T21:19:07.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.439+0000 7f9fdb200640 1 -- 172.21.15.161:0/1418129873 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd414f8c0 con 0x7f9fd4099ca0 2024-05-22T21:19:07.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.443+0000 7f9fdb200640 1 -- 172.21.15.161:0/1418129873 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd4067220 con 0x7f9fd4099ca0 2024-05-22T21:19:07.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.443+0000 7f9fd0c00640 1 -- 172.21.15.161:0/1418129873 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9fc40039a0 con 0x7f9fd4099ca0 2024-05-22T21:19:07.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.443+0000 7f9fd0c00640 1 --2- 172.21.15.161:0/1418129873 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9fb0066690 0x7f9fb0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:07.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.443+0000 7f9fd0c00640 1 -- 172.21.15.161:0/1418129873 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9fc40834c0 con 0x7f9fd4099ca0 2024-05-22T21:19:07.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.443+0000 7f9fcb400640 1 --2- 172.21.15.161:0/1418129873 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9fb0066690 0x7f9fb0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:07.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.443+0000 7f9fcb400640 1 --2- 172.21.15.161:0/1418129873 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9fb0066690 0x7f9fb0068b50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0b2000 tx=0x7f9fbc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:07.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.447+0000 7f9fd0c00640 1 -- 172.21.15.161:0/1418129873 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9fc4050a90 con 0x7f9fd4099ca0 2024-05-22T21:19:07.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.663+0000 7f9fdb200640 1 -- 172.21.15.161:0/1418129873 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7f9fd40cec20 con 0x7f9fd4099ca0 2024-05-22T21:19:07.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.931+0000 7f9fd0c00640 1 -- 172.21.15.161:0/1418129873 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v303) v1 ==== 137+0+0 (secure 0 0 0) 0x7f9fc40540b0 con 0x7f9fd4099ca0 2024-05-22T21:19:07.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.931+0000 7f9fdb200640 1 -- 172.21.15.161:0/1418129873 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9fb0066690 msgr2=0x7f9fb0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:07.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.931+0000 7f9fdb200640 1 --2- 172.21.15.161:0/1418129873 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9fb0066690 0x7f9fb0068b50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0b2000 tx=0x7f9fbc008040 comp rx=0 tx=0).stop 2024-05-22T21:19:07.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.931+0000 7f9fdb200640 1 -- 172.21.15.161:0/1418129873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fd4099ca0 msgr2=0x7f9fd414f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:07.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.931+0000 7f9fdb200640 1 --2- 172.21.15.161:0/1418129873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fd4099ca0 0x7f9fd414f380 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400c3c0 tx=0x7f9fc400c880 comp rx=0 tx=0).stop 2024-05-22T21:19:07.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.931+0000 7f9fdb200640 1 -- 172.21.15.161:0/1418129873 shutdown_connections 2024-05-22T21:19:07.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.931+0000 7f9fdb200640 1 --2- 172.21.15.161:0/1418129873 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9fb0066690 0x7f9fb0068b50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.931+0000 7f9fdb200640 1 --2- 172.21.15.161:0/1418129873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fd415f720 0x7f9fd41e5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.931+0000 7f9fdb200640 1 --2- 172.21.15.161:0/1418129873 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9fd40b3560 0x7f9fd406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.931+0000 7f9fdb200640 1 --2- 172.21.15.161:0/1418129873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fd4099ca0 0x7f9fd414f380 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:07.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.931+0000 7f9fdb200640 1 -- 172.21.15.161:0/1418129873 >> 172.21.15.161:0/1418129873 conn(0x7f9fd40b2a40 msgr2=0x7f9fd415e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:07.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.931+0000 7f9fdb200640 1 -- 172.21.15.161:0/1418129873 shutdown_connections 2024-05-22T21:19:07.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:07.931+0000 7f9fdb200640 1 -- 172.21.15.161:0/1418129873 wait complete. 2024-05-22T21:19:07.954 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-05-22T21:19:07.954 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestQuotaFull) That when an OSD epoch barrier is set on an MDS, subsequently 2024-05-22T21:19:07.955 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestQuotaFull) 2024-05-22T21:19:07.955 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2024-05-22T21:19:08.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.087+0000 7fb22f000640 1 Processor -- start 2024-05-22T21:19:08.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.087+0000 7fb22f000640 1 -- start start 2024-05-22T21:19:08.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.087+0000 7fb22f000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb228143ec0 0x7fb22815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:08.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.087+0000 7fb22f000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb22815ec90 0x7fb22815f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:08.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.087+0000 7fb22f000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2281442a0 con 0x7fb2280715a0 2024-05-22T21:19:08.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.087+0000 7fb22f000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22815f680 con 0x7fb228143ec0 2024-05-22T21:19:08.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.087+0000 7fb22f000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb228070500 con 0x7fb22815ec90 2024-05-22T21:19:08.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.087+0000 7fb227e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb228143ec0 0x7fb22815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:08.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.087+0000 7fb227e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb228143ec0 0x7fb22815f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48990/0 (socket says 172.21.15.161:48990) 2024-05-22T21:19:08.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.087+0000 7fb227e00640 1 -- 172.21.15.161:0/3572261292 learned_addr learned my addr 172.21.15.161:0/3572261292 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:08.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.091+0000 7fb22d000640 1 --2- 172.21.15.161:0/3572261292 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb22815ec90 0x7fb22815f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:08.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.091+0000 7fb227e00640 1 -- 172.21.15.161:0/3572261292 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb22815ec90 msgr2=0x7fb22815f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:08.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.091+0000 7fb227e00640 1 --2- 172.21.15.161:0/3572261292 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb22815ec90 0x7fb22815f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:08.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.091+0000 7fb227e00640 1 -- 172.21.15.161:0/3572261292 >> v1:172.21.15.179:6789/0 conn(0x7fb2280715a0 legacy=0x7fb22806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:08.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.091+0000 7fb227e00640 1 -- 172.21.15.161:0/3572261292 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2280707e0 con 0x7fb228143ec0 2024-05-22T21:19:08.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.091+0000 7fb22d000640 1 --2- 172.21.15.161:0/3572261292 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb22815ec90 0x7fb22815f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:08.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.091+0000 7fb227e00640 1 --2- 172.21.15.161:0/3572261292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb228143ec0 0x7fb22815f830 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb218009e50 tx=0x7fb21801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4d65ce737b7c45 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:08.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.091+0000 7fb227400640 1 -- 172.21.15.161:0/3572261292 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2180a57d0 con 0x7fb228143ec0 2024-05-22T21:19:08.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.091+0000 7fb227400640 1 -- 172.21.15.161:0/3572261292 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2180a5930 con 0x7fb228143ec0 2024-05-22T21:19:08.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.091+0000 7fb227400640 1 -- 172.21.15.161:0/3572261292 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2180a5cc0 con 0x7fb228143ec0 2024-05-22T21:19:08.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.091+0000 7fb22f000640 1 -- 172.21.15.161:0/3572261292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb228143ec0 msgr2=0x7fb22815f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:08.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22f000640 1 --2- 172.21.15.161:0/3572261292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb228143ec0 0x7fb22815f830 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb218009e50 tx=0x7fb21801ca00 comp rx=0 tx=0).stop 2024-05-22T21:19:08.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22f000640 1 -- 172.21.15.161:0/3572261292 shutdown_connections 2024-05-22T21:19:08.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22f000640 1 --2- 172.21.15.161:0/3572261292 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb22815ec90 0x7fb22815f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:08.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22f000640 1 --2- 172.21.15.161:0/3572261292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb228143ec0 0x7fb22815f830 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:08.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22f000640 1 -- 172.21.15.161:0/3572261292 >> 172.21.15.161:0/3572261292 conn(0x7fb2280b0bc0 msgr2=0x7fb2280b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:08.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22f000640 1 -- 172.21.15.161:0/3572261292 shutdown_connections 2024-05-22T21:19:08.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22f000640 1 -- 172.21.15.161:0/3572261292 wait complete. 2024-05-22T21:19:08.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22f000640 1 Processor -- start 2024-05-22T21:19:08.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22f000640 1 -- start start 2024-05-22T21:19:08.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22f000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb228143ec0 0x7fb228159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:08.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22f000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb22815ec90 0x7fb2281f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:08.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22f000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2281f6480 0x7fb2281357f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:08.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22f000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb228135360 con 0x7fb2281f6480 2024-05-22T21:19:08.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22f000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb228134750 con 0x7fb228143ec0 2024-05-22T21:19:08.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22f000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb228134a30 con 0x7fb22815ec90 2024-05-22T21:19:08.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2281f6480 0x7fb2281357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:08.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2281f6480 0x7fb2281357f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48992/0 (socket says 172.21.15.161:48992) 2024-05-22T21:19:08.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22d000640 1 -- 172.21.15.161:0/1550907135 learned_addr learned my addr 172.21.15.161:0/1550907135 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:08.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb21f400640 1 --2- 172.21.15.161:0/1550907135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb22815ec90 0x7fb2281f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:08.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb227e00640 1 --2- 172.21.15.161:0/1550907135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb228143ec0 0x7fb228159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:08.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22d000640 1 -- 172.21.15.161:0/1550907135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb22815ec90 msgr2=0x7fb2281f5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:08.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22d000640 1 --2- 172.21.15.161:0/1550907135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb22815ec90 0x7fb2281f5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:08.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22d000640 1 -- 172.21.15.161:0/1550907135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb228143ec0 msgr2=0x7fb228159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:08.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22d000640 1 --2- 172.21.15.161:0/1550907135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb228143ec0 0x7fb228159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:08.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22d000640 1 -- 172.21.15.161:0/1550907135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb218009b30 con 0x7fb2281f6480 2024-05-22T21:19:08.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.095+0000 7fb22d000640 1 --2- 172.21.15.161:0/1550907135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2281f6480 0x7fb2281357f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb22000d800 tx=0x7fb22000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:08.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.099+0000 7fb224c00640 1 -- 172.21.15.161:0/1550907135 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22000b6c0 con 0x7fb2281f6480 2024-05-22T21:19:08.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.099+0000 7fb224c00640 1 -- 172.21.15.161:0/1550907135 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22000b820 con 0x7fb2281f6480 2024-05-22T21:19:08.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.099+0000 7fb224c00640 1 -- 172.21.15.161:0/1550907135 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22000bbd0 con 0x7fb2281f6480 2024-05-22T21:19:08.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.099+0000 7fb22f000640 1 -- 172.21.15.161:0/1550907135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb228134d10 con 0x7fb2281f6480 2024-05-22T21:19:08.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.099+0000 7fb22f000640 1 -- 172.21.15.161:0/1550907135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22815a090 con 0x7fb2281f6480 2024-05-22T21:19:08.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.099+0000 7fb22f000640 1 -- 172.21.15.161:0/1550907135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb228067220 con 0x7fb2281f6480 2024-05-22T21:19:08.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.099+0000 7fb224c00640 1 -- 172.21.15.161:0/1550907135 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb22000bd30 con 0x7fb2281f6480 2024-05-22T21:19:08.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.099+0000 7fb224c00640 1 --2- 172.21.15.161:0/1550907135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb2040666c0 0x7fb204068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:08.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.099+0000 7fb224c00640 1 -- 172.21.15.161:0/1550907135 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb2200842b0 con 0x7fb2281f6480 2024-05-22T21:19:08.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.099+0000 7fb227e00640 1 --2- 172.21.15.161:0/1550907135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb2040666c0 0x7fb204068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:08.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.103+0000 7fb227e00640 1 --2- 172.21.15.161:0/1550907135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb2040666c0 0x7fb204068b80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb2180ae000 tx=0x7fb2180a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:08.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.103+0000 7fb224c00640 1 -- 172.21.15.161:0/1550907135 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb2200519d0 con 0x7fb2281f6480 2024-05-22T21:19:08.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.315+0000 7fb22f000640 1 -- 172.21.15.161:0/1550907135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7fb2280ccf50 con 0x7fb2281f6480 2024-05-22T21:19:08.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.939+0000 7fb224c00640 1 -- 172.21.15.161:0/1550907135 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v304) v1 ==== 131+0+0 (secure 0 0 0) 0x7fb220054ff0 con 0x7fb2281f6480 2024-05-22T21:19:08.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.939+0000 7fb22f000640 1 -- 172.21.15.161:0/1550907135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb2040666c0 msgr2=0x7fb204068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:08.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.939+0000 7fb22f000640 1 --2- 172.21.15.161:0/1550907135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb2040666c0 0x7fb204068b80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb2180ae000 tx=0x7fb2180a7070 comp rx=0 tx=0).stop 2024-05-22T21:19:08.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.939+0000 7fb22f000640 1 -- 172.21.15.161:0/1550907135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2281f6480 msgr2=0x7fb2281357f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:08.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.939+0000 7fb22f000640 1 --2- 172.21.15.161:0/1550907135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2281f6480 0x7fb2281357f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb22000d800 tx=0x7fb22000dcc0 comp rx=0 tx=0).stop 2024-05-22T21:19:08.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.939+0000 7fb22f000640 1 -- 172.21.15.161:0/1550907135 shutdown_connections 2024-05-22T21:19:08.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.939+0000 7fb22f000640 1 --2- 172.21.15.161:0/1550907135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb2040666c0 0x7fb204068b80 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:08.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.939+0000 7fb22f000640 1 --2- 172.21.15.161:0/1550907135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2281f6480 0x7fb2281357f0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:08.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.939+0000 7fb22f000640 1 --2- 172.21.15.161:0/1550907135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb22815ec90 0x7fb2281f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:08.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.939+0000 7fb22f000640 1 --2- 172.21.15.161:0/1550907135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb228143ec0 0x7fb228159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:08.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.939+0000 7fb22f000640 1 -- 172.21.15.161:0/1550907135 >> 172.21.15.161:0/1550907135 conn(0x7fb2280b0bc0 msgr2=0x7fb2280ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:08.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.939+0000 7fb22f000640 1 -- 172.21.15.161:0/1550907135 shutdown_connections 2024-05-22T21:19:08.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:08.939+0000 7fb22f000640 1 -- 172.21.15.161:0/1550907135 wait complete. 2024-05-22T21:19:08.964 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-22T21:19:09.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.099+0000 7fca00200640 1 Processor -- start 2024-05-22T21:19:09.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.099+0000 7fca00200640 1 -- start start 2024-05-22T21:19:09.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.099+0000 7fca00200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc9f8143c40 0x7fc9f815b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:09.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.099+0000 7fca00200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f815efb0 con 0x7fc9f81444a0 2024-05-22T21:19:09.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.099+0000 7fca00200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f815f290 con 0x7fc9f806c9b0 2024-05-22T21:19:09.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.099+0000 7fca00200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f815f570 con 0x7fc9f8143c40 2024-05-22T21:19:09.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.099+0000 7fc9fce00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc9f81444a0 0x7fc9f806c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41470/0 (socket says 172.21.15.161:41470) 2024-05-22T21:19:09.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.099+0000 7fc9fce00640 1 -- 172.21.15.161:0/789022776 learned_addr learned my addr 172.21.15.161:0/789022776 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:09.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.099+0000 7fc9fd800640 1 --2- 172.21.15.161:0/789022776 >> v2:172.21.15.179:3300/0 conn(0x7fc9f8143c40 0x7fc9f815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:09.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.099+0000 7fc9f7e00640 1 -- 172.21.15.161:0/789022776 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067393007 0 0) 0x7fc9f815efb0 con 0x7fc9f81444a0 2024-05-22T21:19:09.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.103+0000 7fc9f7e00640 1 -- 172.21.15.161:0/789022776 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9e0003530 con 0x7fc9f81444a0 2024-05-22T21:19:09.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.103+0000 7fc9f7e00640 1 -- 172.21.15.161:0/789022776 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 679382238 0 0) 0x7fc9e0003530 con 0x7fc9f81444a0 2024-05-22T21:19:09.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.103+0000 7fc9fd800640 1 -- 172.21.15.161:0/789022776 >> v1:172.21.15.179:6789/0 conn(0x7fc9f806c9b0 legacy=0x7fc9f80b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:09.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.103+0000 7fc9fd800640 1 -- 172.21.15.161:0/789022776 >> v1:172.21.15.161:6789/0 conn(0x7fc9f81444a0 legacy=0x7fc9f806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:09.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.103+0000 7fc9fd800640 1 -- 172.21.15.161:0/789022776 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f815f850 con 0x7fc9f8143c40 2024-05-22T21:19:09.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.103+0000 7fc9fd800640 1 --2- 172.21.15.161:0/789022776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9f8143c40 0x7fc9f815b600 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0097c0 tx=0x7fc9ec01c270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17bfebae133e52 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:09.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.103+0000 7fc9f7e00640 1 -- 172.21.15.161:0/789022776 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4001680 con 0x7fc9f8143c40 2024-05-22T21:19:09.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.103+0000 7fc9f7e00640 1 -- 172.21.15.161:0/789022776 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ec0a7050 con 0x7fc9f8143c40 2024-05-22T21:19:09.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.103+0000 7fc9f7e00640 1 -- 172.21.15.161:0/789022776 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec0ab620 con 0x7fc9f8143c40 2024-05-22T21:19:09.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fca00200640 1 -- 172.21.15.161:0/789022776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9f8143c40 msgr2=0x7fc9f815b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:09.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fca00200640 1 --2- 172.21.15.161:0/789022776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9f8143c40 0x7fc9f815b600 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0097c0 tx=0x7fc9ec01c270 comp rx=0 tx=0).stop 2024-05-22T21:19:09.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fca00200640 1 -- 172.21.15.161:0/789022776 shutdown_connections 2024-05-22T21:19:09.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fca00200640 1 --2- 172.21.15.161:0/789022776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9f8143c40 0x7fc9f815b600 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fca00200640 1 -- 172.21.15.161:0/789022776 >> 172.21.15.161:0/789022776 conn(0x7fc9f806a2d0 msgr2=0x7fc9f806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:09.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fca00200640 1 -- 172.21.15.161:0/789022776 shutdown_connections 2024-05-22T21:19:09.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fca00200640 1 -- 172.21.15.161:0/789022776 wait complete. 2024-05-22T21:19:09.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fca00200640 1 Processor -- start 2024-05-22T21:19:09.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fca00200640 1 -- start start 2024-05-22T21:19:09.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fca00200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9f806c9b0 0x7fc9f81400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:09.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fca00200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc9f8143c40 0x7fc9f8138aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:09.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fca00200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f81444a0 0x7fc9f81568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:09.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fca00200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f81429a0 con 0x7fc9f81444a0 2024-05-22T21:19:09.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fca00200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f8142c80 con 0x7fc9f806c9b0 2024-05-22T21:19:09.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fca00200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f8142f60 con 0x7fc9f8143c40 2024-05-22T21:19:09.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fc9fe200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f81444a0 0x7fc9f81568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:09.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fc9fe200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f81444a0 0x7fc9f81568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48996/0 (socket says 172.21.15.161:48996) 2024-05-22T21:19:09.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fc9fe200640 1 -- 172.21.15.161:0/2345124255 learned_addr learned my addr 172.21.15.161:0/2345124255 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:09.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fc9fd800640 1 --2- 172.21.15.161:0/2345124255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9f806c9b0 0x7fc9f81400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:09.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fc9fce00640 1 --2- 172.21.15.161:0/2345124255 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc9f8143c40 0x7fc9f8138aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:09.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fc9fe200640 1 -- 172.21.15.161:0/2345124255 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc9f8143c40 msgr2=0x7fc9f8138aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:09.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fc9fe200640 1 --2- 172.21.15.161:0/2345124255 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc9f8143c40 0x7fc9f8138aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fc9fe200640 1 -- 172.21.15.161:0/2345124255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9f806c9b0 msgr2=0x7fc9f81400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:09.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fc9fe200640 1 --2- 172.21.15.161:0/2345124255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9f806c9b0 0x7fc9f81400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fc9fe200640 1 -- 172.21.15.161:0/2345124255 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec009420 con 0x7fc9f81444a0 2024-05-22T21:19:09.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fc9fce00640 1 --2- 172.21.15.161:0/2345124255 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc9f8143c40 0x7fc9f8138aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:09.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.107+0000 7fc9fe200640 1 --2- 172.21.15.161:0/2345124255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f81444a0 0x7fc9f81568c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000c160 tx=0x7fc9f000c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:09.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.111+0000 7fc9f5600640 1 -- 172.21.15.161:0/2345124255 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0009050 con 0x7fc9f81444a0 2024-05-22T21:19:09.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.111+0000 7fc9f5600640 1 -- 172.21.15.161:0/2345124255 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f00091b0 con 0x7fc9f81444a0 2024-05-22T21:19:09.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.111+0000 7fc9f5600640 1 -- 172.21.15.161:0/2345124255 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f00095a0 con 0x7fc9f81444a0 2024-05-22T21:19:09.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.111+0000 7fca00200640 1 -- 172.21.15.161:0/2345124255 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f8143240 con 0x7fc9f81444a0 2024-05-22T21:19:09.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.111+0000 7fca00200640 1 -- 172.21.15.161:0/2345124255 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f8138fe0 con 0x7fc9f81444a0 2024-05-22T21:19:09.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.111+0000 7fca00200640 1 -- 172.21.15.161:0/2345124255 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f81448e0 con 0x7fc9f81444a0 2024-05-22T21:19:09.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.111+0000 7fc9f5600640 1 -- 172.21.15.161:0/2345124255 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc9f0009700 con 0x7fc9f81444a0 2024-05-22T21:19:09.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.111+0000 7fc9f5600640 1 --2- 172.21.15.161:0/2345124255 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc9d80666c0 0x7fc9d8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:09.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.111+0000 7fc9f5600640 1 -- 172.21.15.161:0/2345124255 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc9f0085e20 con 0x7fc9f81444a0 2024-05-22T21:19:09.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.111+0000 7fc9fd800640 1 --2- 172.21.15.161:0/2345124255 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc9d80666c0 0x7fc9d8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:09.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.111+0000 7fc9fd800640 1 --2- 172.21.15.161:0/2345124255 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc9d80666c0 0x7fc9d8068b80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0ad000 tx=0x7fc9ec0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:09.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.115+0000 7fc9f5600640 1 -- 172.21.15.161:0/2345124255 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc9f81448e0 con 0x7fc9f81444a0 2024-05-22T21:19:09.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.267+0000 7fca00200640 1 -- 172.21.15.161:0/2345124255 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc9f81448e0 con 0x7fc9f81444a0 2024-05-22T21:19:09.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.275+0000 7fc9f5600640 1 -- 172.21.15.161:0/2345124255 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc9f81448e0 con 0x7fc9f81444a0 2024-05-22T21:19:09.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.279+0000 7fca00200640 1 -- 172.21.15.161:0/2345124255 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc9d80666c0 msgr2=0x7fc9d8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:09.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.279+0000 7fca00200640 1 --2- 172.21.15.161:0/2345124255 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc9d80666c0 0x7fc9d8068b80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0ad000 tx=0x7fc9ec0a6000 comp rx=0 tx=0).stop 2024-05-22T21:19:09.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.279+0000 7fca00200640 1 -- 172.21.15.161:0/2345124255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f81444a0 msgr2=0x7fc9f81568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:09.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.279+0000 7fca00200640 1 --2- 172.21.15.161:0/2345124255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f81444a0 0x7fc9f81568c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000c160 tx=0x7fc9f000c620 comp rx=0 tx=0).stop 2024-05-22T21:19:09.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.279+0000 7fca00200640 1 -- 172.21.15.161:0/2345124255 shutdown_connections 2024-05-22T21:19:09.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.279+0000 7fca00200640 1 --2- 172.21.15.161:0/2345124255 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc9d80666c0 0x7fc9d8068b80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.279+0000 7fca00200640 1 --2- 172.21.15.161:0/2345124255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f81444a0 0x7fc9f81568c0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.279+0000 7fca00200640 1 --2- 172.21.15.161:0/2345124255 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc9f8143c40 0x7fc9f8138aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.279+0000 7fca00200640 1 --2- 172.21.15.161:0/2345124255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9f806c9b0 0x7fc9f81400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.279+0000 7fca00200640 1 -- 172.21.15.161:0/2345124255 >> 172.21.15.161:0/2345124255 conn(0x7fc9f806a2d0 msgr2=0x7fc9f8071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:09.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.279+0000 7fca00200640 1 -- 172.21.15.161:0/2345124255 shutdown_connections 2024-05-22T21:19:09.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.279+0000 7fca00200640 1 -- 172.21.15.161:0/2345124255 wait complete. 2024-05-22T21:19:09.300 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:19:09.300 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:19:09.359 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:19:09.455 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:19:09.455 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:19:09.455 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:09.464 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:19:09.464 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:19:09.464 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:09.473 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:19:09.473 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:19:09.473 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:09.515 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:19:09.515 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:19:09.515 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:09.528 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:19:09.528 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-22T21:19:09.528 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:09.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd62d200640 1 Processor -- start 2024-05-22T21:19:09.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd62d200640 1 -- start start 2024-05-22T21:19:09.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd62d200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd6280715a0 0x7fd62815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:09.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd62d200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6281442a0 con 0x7fd628143ec0 2024-05-22T21:19:09.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd62d200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd628070500 con 0x7fd62815ec90 2024-05-22T21:19:09.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd62d200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6280707e0 con 0x7fd6280715a0 2024-05-22T21:19:09.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd625e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd628143ec0 0x7fd62806fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41478/0 (socket says 172.21.15.161:41478) 2024-05-22T21:19:09.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd625e00640 1 -- 172.21.15.161:0/114110856 learned_addr learned my addr 172.21.15.161:0/114110856 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd61d400640 1 --2- 172.21.15.161:0/114110856 >> v2:172.21.15.179:3300/0 conn(0x7fd6280715a0 0x7fd62815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd625400640 1 -- 172.21.15.161:0/114110856 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834650114 0 0) 0x7fd6281442a0 con 0x7fd628143ec0 2024-05-22T21:19:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd625400640 1 -- 172.21.15.161:0/114110856 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd60c003530 con 0x7fd628143ec0 2024-05-22T21:19:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd625400640 1 -- 172.21.15.161:0/114110856 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 330425396 0 0) 0x7fd60c003530 con 0x7fd628143ec0 2024-05-22T21:19:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd625400640 1 -- 172.21.15.161:0/114110856 >> v2:172.21.15.179:3300/0 conn(0x7fd6280715a0 msgr2=0x7fd62815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd625400640 1 --2- 172.21.15.161:0/114110856 >> v2:172.21.15.179:3300/0 conn(0x7fd6280715a0 0x7fd62815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd625400640 1 -- 172.21.15.161:0/114110856 >> v1:172.21.15.179:6789/0 conn(0x7fd62815ec90 legacy=0x7fd6280cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd625400640 1 -- 172.21.15.161:0/114110856 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd628070ac0 con 0x7fd628143ec0 2024-05-22T21:19:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd625400640 1 -- 172.21.15.161:0/114110856 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fd618001680 con 0x7fd628143ec0 2024-05-22T21:19:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd625400640 1 -- 172.21.15.161:0/114110856 >> v1:172.21.15.161:6789/0 conn(0x7fd628143ec0 legacy=0x7fd62806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.687+0000 7fd625400640 1 --2- 172.21.15.161:0/114110856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd60c003980 0x7fd60c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd625400640 1 --2- 172.21.15.161:0/114110856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd60c0047d0 0x7fd60c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd625400640 1 --2- 172.21.15.161:0/114110856 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd60c005190 0x7fd60c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd625400640 1 -- 172.21.15.161:0/114110856 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280707e0 con 0x7fd60c0047d0 2024-05-22T21:19:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd625400640 1 -- 172.21.15.161:0/114110856 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c010440 con 0x7fd60c003980 2024-05-22T21:19:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd625400640 1 -- 172.21.15.161:0/114110856 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c010600 con 0x7fd60c005190 2024-05-22T21:19:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd61d400640 1 --2- 172.21.15.161:0/114110856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd60c0047d0 0x7fd60c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd625e00640 1 --2- 172.21.15.161:0/114110856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd60c003980 0x7fd60c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd626800640 1 --2- 172.21.15.161:0/114110856 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd60c005190 0x7fd60c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd61d400640 1 -- 172.21.15.161:0/114110856 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd60c005190 msgr2=0x7fd60c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd61d400640 1 --2- 172.21.15.161:0/114110856 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd60c005190 0x7fd60c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd61d400640 1 -- 172.21.15.161:0/114110856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd60c003980 msgr2=0x7fd60c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd61d400640 1 --2- 172.21.15.161:0/114110856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd60c003980 0x7fd60c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd61d400640 1 -- 172.21.15.161:0/114110856 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60c003530 con 0x7fd60c0047d0 2024-05-22T21:19:09.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd625e00640 1 --2- 172.21.15.161:0/114110856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd60c003980 0x7fd60c003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:09.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd626800640 1 -- 172.21.15.161:0/114110856 reap_dead start 2024-05-22T21:19:09.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd61d400640 1 --2- 172.21.15.161:0/114110856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd60c0047d0 0x7fd60c004c50 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fd610009ae0 tx=0x7fd61001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87df771ba08dd75d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:09.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd625400640 1 -- 172.21.15.161:0/114110856 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6100a5660 con 0x7fd60c0047d0 2024-05-22T21:19:09.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd625400640 1 -- 172.21.15.161:0/114110856 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6100a57c0 con 0x7fd60c0047d0 2024-05-22T21:19:09.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.691+0000 7fd625400640 1 -- 172.21.15.161:0/114110856 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6100a5b70 con 0x7fd60c0047d0 2024-05-22T21:19:09.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd62d200640 1 -- 172.21.15.161:0/114110856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd60c0047d0 msgr2=0x7fd60c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:09.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd62d200640 1 --2- 172.21.15.161:0/114110856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd60c0047d0 0x7fd60c004c50 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fd610009ae0 tx=0x7fd61001ca70 comp rx=0 tx=0).stop 2024-05-22T21:19:09.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd62d200640 1 -- 172.21.15.161:0/114110856 shutdown_connections 2024-05-22T21:19:09.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd62d200640 1 --2- 172.21.15.161:0/114110856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd60c0047d0 0x7fd60c004c50 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd62d200640 1 -- 172.21.15.161:0/114110856 >> 172.21.15.161:0/114110856 conn(0x7fd6280b0bc0 msgr2=0x7fd6280b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:09.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd62d200640 1 -- 172.21.15.161:0/114110856 shutdown_connections 2024-05-22T21:19:09.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd62d200640 1 -- 172.21.15.161:0/114110856 wait complete. 2024-05-22T21:19:09.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd62d200640 1 Processor -- start 2024-05-22T21:19:09.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd62d200640 1 -- start start 2024-05-22T21:19:09.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd62d200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd60c0047d0 0x7fd628158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:09.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd62d200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd62813ef80 0x7fd6281414c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:09.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd62d200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd628141a00 0x7fd628141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:09.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd62d200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd628159320 con 0x7fd62813ef80 2024-05-22T21:19:09.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd62d200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62813f6f0 con 0x7fd60c0047d0 2024-05-22T21:19:09.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd62d200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd628142770 con 0x7fd628141a00 2024-05-22T21:19:09.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd61d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd62813ef80 0x7fd6281414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:09.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd61d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd62813ef80 0x7fd6281414c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49022/0 (socket says 172.21.15.161:49022) 2024-05-22T21:19:09.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd61d400640 1 -- 172.21.15.161:0/1197818746 learned_addr learned my addr 172.21.15.161:0/1197818746 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:09.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd626800640 1 --2- 172.21.15.161:0/1197818746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd628141a00 0x7fd628141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:09.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd625e00640 1 --2- 172.21.15.161:0/1197818746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd60c0047d0 0x7fd628158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:09.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd61d400640 1 -- 172.21.15.161:0/1197818746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd628141a00 msgr2=0x7fd628141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:09.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd61d400640 1 --2- 172.21.15.161:0/1197818746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd628141a00 0x7fd628141e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd61d400640 1 -- 172.21.15.161:0/1197818746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd60c0047d0 msgr2=0x7fd628158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:09.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd61d400640 1 --2- 172.21.15.161:0/1197818746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd60c0047d0 0x7fd628158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd61d400640 1 -- 172.21.15.161:0/1197818746 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62813f410 con 0x7fd62813ef80 2024-05-22T21:19:09.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.695+0000 7fd61d400640 1 --2- 172.21.15.161:0/1197818746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd62813ef80 0x7fd6281414c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fd6100b0040 tx=0x7fd610019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.699+0000 7fd61ea00640 1 -- 172.21.15.161:0/1197818746 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610004220 con 0x7fd62813ef80 2024-05-22T21:19:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.699+0000 7fd61ea00640 1 -- 172.21.15.161:0/1197818746 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd610004380 con 0x7fd62813ef80 2024-05-22T21:19:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.699+0000 7fd61ea00640 1 -- 172.21.15.161:0/1197818746 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6100b68b0 con 0x7fd62813ef80 2024-05-22T21:19:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.699+0000 7fd62d200640 1 -- 172.21.15.161:0/1197818746 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6280ae840 con 0x7fd62813ef80 2024-05-22T21:19:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.699+0000 7fd62d200640 1 -- 172.21.15.161:0/1197818746 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6280aee00 con 0x7fd62813ef80 2024-05-22T21:19:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.699+0000 7fd62d200640 1 -- 172.21.15.161:0/1197818746 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd628067220 con 0x7fd62813ef80 2024-05-22T21:19:09.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.699+0000 7fd61ea00640 1 -- 172.21.15.161:0/1197818746 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd6100b4070 con 0x7fd62813ef80 2024-05-22T21:19:09.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.699+0000 7fd61ea00640 1 --2- 172.21.15.161:0/1197818746 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd6040666c0 0x7fd604068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:09.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.699+0000 7fd61ea00640 1 -- 172.21.15.161:0/1197818746 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd61011dff0 con 0x7fd62813ef80 2024-05-22T21:19:09.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.699+0000 7fd625e00640 1 --2- 172.21.15.161:0/1197818746 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd6040666c0 0x7fd604068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:09.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.699+0000 7fd625e00640 1 --2- 172.21.15.161:0/1197818746 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd6040666c0 0x7fd604068b80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd628063da0 tx=0x7fd618009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:09.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.703+0000 7fd61ea00640 1 -- 172.21.15.161:0/1197818746 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd6100eb650 con 0x7fd62813ef80 2024-05-22T21:19:09.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.891+0000 7fd62d200640 1 -- 172.21.15.161:0/1197818746 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd62806dcc0 con 0x7fd62813ef80 2024-05-22T21:19:09.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.891+0000 7fd61ea00640 1 -- 172.21.15.161:0/1197818746 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+3541 (secure 0 0 0) 0x7fd6100b2020 con 0x7fd62813ef80 2024-05-22T21:19:09.895 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:09.895 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2771409351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2771409351},{"type":"v1","addr":"172.21.15.179:6834","nonce":2771409351}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6442,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2520119431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2520119431},{"type":"v1","addr":"172.21.15.179:6835","nonce":2520119431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1415362547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1415362547},{"type":"v1","addr":"172.21.15.161:6835","nonce":1415362547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7074,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3031043017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3031043017},{"type":"v1","addr":"172.21.15.161:6837","nonce":3031043017}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[]} 2024-05-22T21:19:09.895 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 49 2024-05-22T21:19:09.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.895+0000 7fd62d200640 1 -- 172.21.15.161:0/1197818746 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd6040666c0 msgr2=0x7fd604068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:09.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.895+0000 7fd62d200640 1 --2- 172.21.15.161:0/1197818746 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd6040666c0 0x7fd604068b80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd628063da0 tx=0x7fd618009040 comp rx=0 tx=0).stop 2024-05-22T21:19:09.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.895+0000 7fd62d200640 1 -- 172.21.15.161:0/1197818746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd62813ef80 msgr2=0x7fd6281414c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:09.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.895+0000 7fd62d200640 1 --2- 172.21.15.161:0/1197818746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd62813ef80 0x7fd6281414c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fd6100b0040 tx=0x7fd610019810 comp rx=0 tx=0).stop 2024-05-22T21:19:09.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.895+0000 7fd62d200640 1 -- 172.21.15.161:0/1197818746 shutdown_connections 2024-05-22T21:19:09.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.895+0000 7fd62d200640 1 --2- 172.21.15.161:0/1197818746 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd6040666c0 0x7fd604068b80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.895+0000 7fd62d200640 1 --2- 172.21.15.161:0/1197818746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd628141a00 0x7fd628141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.895+0000 7fd62d200640 1 --2- 172.21.15.161:0/1197818746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd62813ef80 0x7fd6281414c0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.895+0000 7fd62d200640 1 --2- 172.21.15.161:0/1197818746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd60c0047d0 0x7fd628158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:09.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.895+0000 7fd62d200640 1 -- 172.21.15.161:0/1197818746 >> 172.21.15.161:0/1197818746 conn(0x7fd6280b0bc0 msgr2=0x7fd6280ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:09.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.895+0000 7fd62d200640 1 -- 172.21.15.161:0/1197818746 shutdown_connections 2024-05-22T21:19:09.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:09.895+0000 7fd62d200640 1 -- 172.21.15.161:0/1197818746 wait complete. 2024-05-22T21:19:09.916 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-22T21:19:09.917 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-22T21:19:09.917 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-22T21:19:09.917 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:19:09.917 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-22T21:19:09.917 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-22T21:19:09.917 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-22T21:19:09.917 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:19:09.918 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-22T21:19:09.918 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-22T21:19:09.918 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-22T21:19:09.918 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:19:09.918 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-22T21:19:09.918 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-22T21:19:09.919 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-22T21:19:09.919 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:19:09.929 INFO:tasks.ceph.mds.c:Stopped 2024-05-22T21:19:09.929 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-22T21:19:09.930 INFO:tasks.ceph.mds.a:Stopped 2024-05-22T21:19:09.930 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-22T21:19:09.933 INFO:tasks.ceph.mds.b:Stopped 2024-05-22T21:19:09.933 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-22T21:19:09.934 INFO:tasks.ceph.mds.d:Stopped 2024-05-22T21:19:09.934 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-22T21:19:09.978 INFO:tasks.ceph.mds.c:Started 2024-05-22T21:19:09.979 DEBUG:teuthology.parallel:result is None 2024-05-22T21:19:09.979 INFO:tasks.ceph.mds.b:Started 2024-05-22T21:19:09.979 DEBUG:teuthology.parallel:result is None 2024-05-22T21:19:09.980 INFO:tasks.ceph.mds.a:Started 2024-05-22T21:19:09.980 DEBUG:teuthology.parallel:result is None 2024-05-22T21:19:09.980 INFO:tasks.ceph.mds.d:Started 2024-05-22T21:19:09.981 DEBUG:teuthology.parallel:result is None 2024-05-22T21:19:09.981 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-22T21:19:10.066 INFO:tasks.ceph.mds.a.smithi161.stdout:starting mds.a at 2024-05-22T21:19:10.066 INFO:tasks.ceph.mds.b.smithi179.stdout:starting mds.b at 2024-05-22T21:19:10.067 INFO:tasks.ceph.mds.c.smithi161.stdout:starting mds.c at 2024-05-22T21:19:10.067 INFO:tasks.ceph.mds.d.smithi179.stdout:starting mds.d at 2024-05-22T21:19:10.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.123+0000 7f8865e00640 1 Processor -- start 2024-05-22T21:19:10.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.123+0000 7f8865e00640 1 -- start start 2024-05-22T21:19:10.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.127+0000 7f8865e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8860143c40 0x7f886015b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:10.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.127+0000 7f8865e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f886006c9b0 0x7f88600b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:10.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.127+0000 7f8865e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886015efb0 con 0x7f88601444a0 2024-05-22T21:19:10.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.127+0000 7f8865e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f886015f290 con 0x7f8860143c40 2024-05-22T21:19:10.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.127+0000 7f8865e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886015f570 con 0x7f886006c9b0 2024-05-22T21:19:10.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.127+0000 7f8856000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f88601444a0 0x7f886006c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41506/0 (socket says 172.21.15.161:41506) 2024-05-22T21:19:10.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.127+0000 7f8856000640 1 -- 172.21.15.161:0/3525406360 learned_addr learned my addr 172.21.15.161:0/3525406360 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:10.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.127+0000 7f885ea00640 1 --2- 172.21.15.161:0/3525406360 >> v2:172.21.15.161:3300/0 conn(0x7f8860143c40 0x7f886015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:10.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.127+0000 7f885f400640 1 --2- 172.21.15.161:0/3525406360 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f886006c9b0 0x7f88600b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:10.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.127+0000 7f885ea00640 1 -- 172.21.15.161:0/3525406360 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f886006c9b0 msgr2=0x7f88600b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:10.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.127+0000 7f885ea00640 1 --2- 172.21.15.161:0/3525406360 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f886006c9b0 0x7f88600b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:10.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.127+0000 7f885ea00640 1 -- 172.21.15.161:0/3525406360 >> v1:172.21.15.161:6789/0 conn(0x7f88601444a0 legacy=0x7f886006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:10.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.127+0000 7f885ea00640 1 -- 172.21.15.161:0/3525406360 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886015f850 con 0x7f8860143c40 2024-05-22T21:19:10.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f885ea00640 1 --2- 172.21.15.161:0/3525406360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8860143c40 0x7f886015b600 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f88500097c0 tx=0x7f885001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a18b89bd19d9ded server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:10.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f885e000640 1 -- 172.21.15.161:0/3525406360 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88500ab600 con 0x7f8860143c40 2024-05-22T21:19:10.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f885e000640 1 -- 172.21.15.161:0/3525406360 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88500ab760 con 0x7f8860143c40 2024-05-22T21:19:10.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f885e000640 1 -- 172.21.15.161:0/3525406360 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88500abaf0 con 0x7f8860143c40 2024-05-22T21:19:10.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f8865e00640 1 -- 172.21.15.161:0/3525406360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8860143c40 msgr2=0x7f886015b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:10.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f8865e00640 1 --2- 172.21.15.161:0/3525406360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8860143c40 0x7f886015b600 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f88500097c0 tx=0x7f885001c4a0 comp rx=0 tx=0).stop 2024-05-22T21:19:10.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f8865e00640 1 -- 172.21.15.161:0/3525406360 shutdown_connections 2024-05-22T21:19:10.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f8865e00640 1 --2- 172.21.15.161:0/3525406360 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f886006c9b0 0x7f88600b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:10.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f8865e00640 1 --2- 172.21.15.161:0/3525406360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8860143c40 0x7f886015b600 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:10.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f8865e00640 1 -- 172.21.15.161:0/3525406360 >> 172.21.15.161:0/3525406360 conn(0x7f886006a2d0 msgr2=0x7f886006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:10.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f8865e00640 1 -- 172.21.15.161:0/3525406360 shutdown_connections 2024-05-22T21:19:10.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f8865e00640 1 -- 172.21.15.161:0/3525406360 wait complete. 2024-05-22T21:19:10.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f8865e00640 1 Processor -- start 2024-05-22T21:19:10.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f8865e00640 1 -- start start 2024-05-22T21:19:10.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f8865e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886006c9b0 0x7f8860155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:10.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f8865e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8860143c40 0x7f8860139e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:10.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f8865e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88601444a0 0x7f886006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:10.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f8865e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886015ca90 con 0x7f886006c9b0 2024-05-22T21:19:10.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f8865e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886015d250 con 0x7f88601444a0 2024-05-22T21:19:10.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.131+0000 7f8865e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886015d530 con 0x7f8860143c40 2024-05-22T21:19:10.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f885ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886006c9b0 0x7f8860155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:10.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f885ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886006c9b0 0x7f8860155b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49068/0 (socket says 172.21.15.161:49068) 2024-05-22T21:19:10.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f885ea00640 1 -- 172.21.15.161:0/2158239663 learned_addr learned my addr 172.21.15.161:0/2158239663 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:10.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f885f400640 1 --2- 172.21.15.161:0/2158239663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88601444a0 0x7f886006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:10.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f8856000640 1 --2- 172.21.15.161:0/2158239663 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8860143c40 0x7f8860139e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:10.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f885ea00640 1 -- 172.21.15.161:0/2158239663 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8860143c40 msgr2=0x7f8860139e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:10.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f885ea00640 1 --2- 172.21.15.161:0/2158239663 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8860143c40 0x7f8860139e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:10.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f885ea00640 1 -- 172.21.15.161:0/2158239663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88601444a0 msgr2=0x7f886006fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:10.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f885ea00640 1 --2- 172.21.15.161:0/2158239663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88601444a0 0x7f886006fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:10.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f885ea00640 1 -- 172.21.15.161:0/2158239663 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8850009420 con 0x7f886006c9b0 2024-05-22T21:19:10.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f8856000640 1 --2- 172.21.15.161:0/2158239663 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8860143c40 0x7f8860139e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:10.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f885f400640 1 --2- 172.21.15.161:0/2158239663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88601444a0 0x7f886006fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:10.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f885ea00640 1 --2- 172.21.15.161:0/2158239663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886006c9b0 0x7f8860155b90 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f88500ad040 tx=0x7f885001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:10.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f8857e00640 1 -- 172.21.15.161:0/2158239663 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88500b5810 con 0x7f886006c9b0 2024-05-22T21:19:10.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f8857e00640 1 -- 172.21.15.161:0/2158239663 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88500b59b0 con 0x7f886006c9b0 2024-05-22T21:19:10.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f8865e00640 1 -- 172.21.15.161:0/2158239663 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f886015d810 con 0x7f886006c9b0 2024-05-22T21:19:10.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f8857e00640 1 -- 172.21.15.161:0/2158239663 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88500b5d60 con 0x7f886006c9b0 2024-05-22T21:19:10.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f8865e00640 1 -- 172.21.15.161:0/2158239663 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886015ddd0 con 0x7f886006c9b0 2024-05-22T21:19:10.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f8865e00640 1 -- 172.21.15.161:0/2158239663 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886015dfc0 con 0x7f886006c9b0 2024-05-22T21:19:10.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f8857e00640 1 -- 172.21.15.161:0/2158239663 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f88500c83f0 con 0x7f886006c9b0 2024-05-22T21:19:10.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f8857e00640 1 --2- 172.21.15.161:0/2158239663 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f883c066690 0x7f883c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:10.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.135+0000 7f8857e00640 1 -- 172.21.15.161:0/2158239663 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f885011eae0 con 0x7f886006c9b0 2024-05-22T21:19:10.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.139+0000 7f8856000640 1 --2- 172.21.15.161:0/2158239663 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f883c066690 0x7f883c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:10.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.139+0000 7f8856000640 1 --2- 172.21.15.161:0/2158239663 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f883c066690 0x7f883c068b50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8860155920 tx=0x7f8848009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:10.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.139+0000 7f8857e00640 1 -- 172.21.15.161:0/2158239663 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f88500ec140 con 0x7f886006c9b0 2024-05-22T21:19:10.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:10.307+0000 7f8865e00640 1 -- 172.21.15.161:0/2158239663 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8860067220 con 0x7f886006c9b0 2024-05-22T21:19:11.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.099+0000 7f8857e00640 1 -- 172.21.15.161:0/2158239663 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v121) v1 ==== 97+0+0 (secure 0 0 0) 0x7f88500ab5c0 con 0x7f886006c9b0 2024-05-22T21:19:11.101 INFO:teuthology.orchestra.run.smithi161.stderr: removed all blocklist entries 2024-05-22T21:19:11.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.099+0000 7f8865e00640 1 -- 172.21.15.161:0/2158239663 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f883c066690 msgr2=0x7f883c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.099+0000 7f8865e00640 1 --2- 172.21.15.161:0/2158239663 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f883c066690 0x7f883c068b50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8860155920 tx=0x7f8848009040 comp rx=0 tx=0).stop 2024-05-22T21:19:11.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.099+0000 7f8865e00640 1 -- 172.21.15.161:0/2158239663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886006c9b0 msgr2=0x7f8860155b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.099+0000 7f8865e00640 1 --2- 172.21.15.161:0/2158239663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886006c9b0 0x7f8860155b90 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f88500ad040 tx=0x7f885001ace0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.099+0000 7f8865e00640 1 -- 172.21.15.161:0/2158239663 shutdown_connections 2024-05-22T21:19:11.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.099+0000 7f8865e00640 1 --2- 172.21.15.161:0/2158239663 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f883c066690 0x7f883c068b50 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.099+0000 7f8865e00640 1 --2- 172.21.15.161:0/2158239663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88601444a0 0x7f886006fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.099+0000 7f8865e00640 1 --2- 172.21.15.161:0/2158239663 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8860143c40 0x7f8860139e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.099+0000 7f8865e00640 1 --2- 172.21.15.161:0/2158239663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886006c9b0 0x7f8860155b90 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.099+0000 7f8865e00640 1 -- 172.21.15.161:0/2158239663 >> 172.21.15.161:0/2158239663 conn(0x7f886006a2d0 msgr2=0x7f88600b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:11.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.099+0000 7f8865e00640 1 -- 172.21.15.161:0/2158239663 shutdown_connections 2024-05-22T21:19:11.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.099+0000 7f8865e00640 1 -- 172.21.15.161:0/2158239663 wait complete. 2024-05-22T21:19:11.123 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-22T21:19:11.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3ff5400640 1 Processor -- start 2024-05-22T21:19:11.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3ff5400640 1 -- start start 2024-05-22T21:19:11.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3ff5400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3ff0143ec0 0x7f3ff015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3ff5400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ff015ec90 0x7f3ff015f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3ff5400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff01442a0 con 0x7f3ff00715a0 2024-05-22T21:19:11.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3ff5400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff015f680 con 0x7f3ff0143ec0 2024-05-22T21:19:11.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3ff5400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff0070500 con 0x7f3ff015ec90 2024-05-22T21:19:11.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3fede00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3ff00715a0 0x7f3ff006fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41516/0 (socket says 172.21.15.161:41516) 2024-05-22T21:19:11.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3fede00640 1 -- 172.21.15.161:0/615054546 learned_addr learned my addr 172.21.15.161:0/615054546 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:11.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3fee800640 1 --2- 172.21.15.161:0/615054546 >> v2:172.21.15.179:3300/0 conn(0x7f3ff0143ec0 0x7f3ff015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3fef200640 1 --2- 172.21.15.161:0/615054546 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ff015ec90 0x7f3ff015f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3fed400640 1 -- 172.21.15.161:0/615054546 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19311497 0 0) 0x7f3ff01442a0 con 0x7f3ff00715a0 2024-05-22T21:19:11.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3fed400640 1 -- 172.21.15.161:0/615054546 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fd4003530 con 0x7f3ff00715a0 2024-05-22T21:19:11.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3fee800640 1 -- 172.21.15.161:0/615054546 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ff015ec90 msgr2=0x7f3ff015f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:11.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3fee800640 1 --2- 172.21.15.161:0/615054546 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ff015ec90 0x7f3ff015f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3fee800640 1 -- 172.21.15.161:0/615054546 >> v1:172.21.15.161:6789/0 conn(0x7f3ff00715a0 legacy=0x7f3ff006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3fed400640 1 -- 172.21.15.161:0/615054546 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 808690039 0 0) 0x7f3fd4003530 con 0x7f3ff00715a0 2024-05-22T21:19:11.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.259+0000 7f3fee800640 1 -- 172.21.15.161:0/615054546 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff00707e0 con 0x7f3ff0143ec0 2024-05-22T21:19:11.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.263+0000 7f3fee800640 1 --2- 172.21.15.161:0/615054546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff0143ec0 0x7f3ff015f830 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00097c0 tx=0x7f3fe001c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c62bbc5daa20f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:11.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.263+0000 7f3fed400640 1 -- 172.21.15.161:0/615054546 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc001680 con 0x7f3ff0143ec0 2024-05-22T21:19:11.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.263+0000 7f3fed400640 1 -- 172.21.15.161:0/615054546 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe00a7050 con 0x7f3ff0143ec0 2024-05-22T21:19:11.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.263+0000 7f3fed400640 1 -- 172.21.15.161:0/615054546 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00ab620 con 0x7f3ff0143ec0 2024-05-22T21:19:11.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.263+0000 7f3ff5400640 1 -- 172.21.15.161:0/615054546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff0143ec0 msgr2=0x7f3ff015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.263+0000 7f3ff5400640 1 --2- 172.21.15.161:0/615054546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff0143ec0 0x7f3ff015f830 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00097c0 tx=0x7f3fe001c390 comp rx=0 tx=0).stop 2024-05-22T21:19:11.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.263+0000 7f3ff5400640 1 -- 172.21.15.161:0/615054546 shutdown_connections 2024-05-22T21:19:11.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.263+0000 7f3ff5400640 1 --2- 172.21.15.161:0/615054546 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ff015ec90 0x7f3ff015f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.263+0000 7f3ff5400640 1 --2- 172.21.15.161:0/615054546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff0143ec0 0x7f3ff015f830 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.263+0000 7f3ff5400640 1 -- 172.21.15.161:0/615054546 >> 172.21.15.161:0/615054546 conn(0x7f3ff00b0bc0 msgr2=0x7f3ff00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:11.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.263+0000 7f3ff5400640 1 -- 172.21.15.161:0/615054546 shutdown_connections 2024-05-22T21:19:11.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.263+0000 7f3ff5400640 1 -- 172.21.15.161:0/615054546 wait complete. 2024-05-22T21:19:11.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3ff5400640 1 Processor -- start 2024-05-22T21:19:11.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3ff5400640 1 -- start start 2024-05-22T21:19:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3ff5400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff0143ec0 0x7f3ff013f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3ff5400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ff015ec90 0x7f3ff01651a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3ff5400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff015f830 0x7f3ff01641d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3ff5400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff0163d40 con 0x7f3ff0143ec0 2024-05-22T21:19:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3ff5400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff013fd00 con 0x7f3ff015f830 2024-05-22T21:19:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3ff5400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff015fd80 con 0x7f3ff015ec90 2024-05-22T21:19:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3fee800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff0143ec0 0x7f3ff013f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3fee800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff0143ec0 0x7f3ff013f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49070/0 (socket says 172.21.15.161:49070) 2024-05-22T21:19:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3fee800640 1 -- 172.21.15.161:0/4051526921 learned_addr learned my addr 172.21.15.161:0/4051526921 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3fede00640 1 --2- 172.21.15.161:0/4051526921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ff015ec90 0x7f3ff01651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3fef200640 1 --2- 172.21.15.161:0/4051526921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff015f830 0x7f3ff01641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3fee800640 1 -- 172.21.15.161:0/4051526921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ff015ec90 msgr2=0x7f3ff01651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3fee800640 1 --2- 172.21.15.161:0/4051526921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ff015ec90 0x7f3ff01651a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3fee800640 1 -- 172.21.15.161:0/4051526921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff015f830 msgr2=0x7f3ff01641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3fee800640 1 --2- 172.21.15.161:0/4051526921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff015f830 0x7f3ff01641d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3fee800640 1 -- 172.21.15.161:0/4051526921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe0009420 con 0x7f3ff0143ec0 2024-05-22T21:19:11.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3fede00640 1 --2- 172.21.15.161:0/4051526921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ff015ec90 0x7f3ff01651a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:11.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3fee800640 1 --2- 172.21.15.161:0/4051526921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff0143ec0 0x7f3ff013f7c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00097c0 tx=0x7f3fe001abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:11.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3fe6a00640 1 -- 172.21.15.161:0/4051526921 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00b7450 con 0x7f3ff0143ec0 2024-05-22T21:19:11.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3fe6a00640 1 -- 172.21.15.161:0/4051526921 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe00b75d0 con 0x7f3ff0143ec0 2024-05-22T21:19:11.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3fe6a00640 1 -- 172.21.15.161:0/4051526921 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00b7980 con 0x7f3ff0143ec0 2024-05-22T21:19:11.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3ff5400640 1 -- 172.21.15.161:0/4051526921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff01636f0 con 0x7f3ff0143ec0 2024-05-22T21:19:11.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3ff5400640 1 -- 172.21.15.161:0/4051526921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff01656e0 con 0x7f3ff0143ec0 2024-05-22T21:19:11.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.267+0000 7f3ff5400640 1 -- 172.21.15.161:0/4051526921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff0067220 con 0x7f3ff0143ec0 2024-05-22T21:19:11.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.271+0000 7f3fe6a00640 1 -- 172.21.15.161:0/4051526921 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3fe00b7ae0 con 0x7f3ff0143ec0 2024-05-22T21:19:11.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.271+0000 7f3fe6a00640 1 --2- 172.21.15.161:0/4051526921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3fd00666c0 0x7f3fd0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.271+0000 7f3fe6a00640 1 -- 172.21.15.161:0/4051526921 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3fe011de80 con 0x7f3ff0143ec0 2024-05-22T21:19:11.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.271+0000 7f3fede00640 1 --2- 172.21.15.161:0/4051526921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3fd00666c0 0x7f3fd0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.271+0000 7f3fede00640 1 --2- 172.21.15.161:0/4051526921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3fd00666c0 0x7f3fd0068b80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3ff013fff0 tx=0x7f3fdc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:11.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.275+0000 7f3fe6a00640 1 -- 172.21.15.161:0/4051526921 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3fe00eb4a0 con 0x7f3ff0143ec0 2024-05-22T21:19:11.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.483+0000 7f3ff5400640 1 -- 172.21.15.161:0/4051526921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3ff006dcc0 con 0x7f3ff0143ec0 2024-05-22T21:19:11.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.483+0000 7f3fe6a00640 1 -- 172.21.15.161:0/4051526921 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3fe00f24d0 con 0x7f3ff0143ec0 2024-05-22T21:19:11.487 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:11.487 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:19:11.487 INFO:teuthology.orchestra.run.smithi161.stdout: "auth_dump": [ 2024-05-22T21:19:11.487 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.487 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.a", 2024-05-22T21:19:11.487 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m00QTFRAAPSRrpBRY75FlcWf3QxOOfg==", 2024-05-22T21:19:11.487 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.487 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:19:11.487 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:19:11.488 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:19:11.488 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:11.488 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.488 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.488 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.b", 2024-05-22T21:19:11.488 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m/ipAIRAA4gj5jGqTpNCr1K2OU/bsAQ==", 2024-05-22T21:19:11.488 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.488 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:19:11.488 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:19:11.488 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:19:11.489 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:11.489 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.489 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.489 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.c", 2024-05-22T21:19:11.489 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mWlM2HBAAEmJJWMDlTG9E7Qzld2gd5w==", 2024-05-22T21:19:11.489 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.489 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:19:11.489 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:19:11.489 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:19:11.489 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:11.489 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.490 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.d", 2024-05-22T21:19:11.490 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mmsuaKBAAqpCYRZZUGK5ZuxflUBcPgg==", 2024-05-22T21:19:11.490 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.490 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:19:11.490 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:19:11.490 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:19:11.490 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:11.490 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.490 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.490 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.490 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.0", 2024-05-22T21:19:11.490 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05mJZokLhAAh2vbQnRX25iYUIbvVmURuQ==", 2024-05-22T21:19:11.491 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.491 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:19:11.491 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:19:11.491 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:11.491 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.491 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.491 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.1", 2024-05-22T21:19:11.491 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05m8kWNNhAA6aaVT2xshFkqnk4G4gOttA==", 2024-05-22T21:19:11.491 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.491 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:19:11.491 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:19:11.491 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:11.492 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.492 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.492 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.492 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.2", 2024-05-22T21:19:11.492 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05ml1k9AxAAtL3XaJJI6cilpmvurh18Pw==", 2024-05-22T21:19:11.492 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.492 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:19:11.492 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:19:11.492 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:11.492 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.492 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.492 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.492 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.3", 2024-05-22T21:19:11.493 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05mbNBgCxAAas1L+QR4Aez+H1G0EVltfg==", 2024-05-22T21:19:11.493 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.493 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:19:11.493 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:19:11.493 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:11.493 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.493 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.493 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.493 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.4", 2024-05-22T21:19:11.493 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mOPLPABAAZBGIMF/FasNB27n/7di0AA==", 2024-05-22T21:19:11.493 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.493 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:19:11.494 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:19:11.494 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:11.494 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.494 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.494 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.494 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.5", 2024-05-22T21:19:11.494 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mT1poCRAASXm4tdP01pA1VOviqeoLVw==", 2024-05-22T21:19:11.494 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.494 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:19:11.494 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:19:11.495 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:11.495 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.495 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.6", 2024-05-22T21:19:11.495 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05m1ZPRERAA2S7WeTwysLCZm1OB/qCrAA==", 2024-05-22T21:19:11.495 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.495 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:19:11.495 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:19:11.495 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:11.495 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.495 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.496 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.7", 2024-05-22T21:19:11.496 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mCrUhGhAAvXoFWV7X3303rfXaxmfWqA==", 2024-05-22T21:19:11.496 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.496 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:19:11.496 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:19:11.496 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:11.496 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.496 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.496 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.496 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.0", 2024-05-22T21:19:11.496 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mh9B7LBAAevhvj5KpwP9pEIpJcxUvFA==", 2024-05-22T21:19:11.496 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.497 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:19:11.497 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow r", 2024-05-22T21:19:11.497 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-22T21:19:11.497 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.497 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.497 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.497 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.1", 2024-05-22T21:19:11.497 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mMnHtNRAAg/a/Mb1oRqx73d9efIHCfA==", 2024-05-22T21:19:11.497 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.497 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:19:11.497 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:19:11.497 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:19:11.498 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:19:11.498 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.498 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.498 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.498 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.2", 2024-05-22T21:19:11.498 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mOzdkMhAAbTiNdvyxqIyrluMiZW0cOw==", 2024-05-22T21:19:11.498 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.498 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:19:11.498 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:19:11.498 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:19:11.499 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:19:11.499 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.499 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.499 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.499 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.3", 2024-05-22T21:19:11.499 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDYX05mr4oxABAA1+bQQbxhoeU+kb0OCa+Wyw==", 2024-05-22T21:19:11.499 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.499 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:19:11.499 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:19:11.499 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:19:11.499 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:19:11.499 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.500 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.500 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.500 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.admin", 2024-05-22T21:19:11.500 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDWX05mHZY0NRAAiBvMDkrCZ+MgG2iwBH+bcg==", 2024-05-22T21:19:11.500 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.500 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:19:11.500 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:19:11.500 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:19:11.500 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:11.500 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.500 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.500 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.501 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.x", 2024-05-22T21:19:11.501 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mH2kuERAAgigMUWK3M3WaMXlqGUcNrw==", 2024-05-22T21:19:11.501 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.501 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:19:11.501 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:19:11.501 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:11.501 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.501 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:11.501 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:11.501 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.y", 2024-05-22T21:19:11.501 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mZRdiDRAA0vUwD+rPSfyhjgA+9XPp5w==", 2024-05-22T21:19:11.501 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:11.501 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:19:11.501 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:19:11.502 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:11.502 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.502 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:11.502 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:11.502 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:19:11.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.487+0000 7f3ff5400640 1 -- 172.21.15.161:0/4051526921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3fd00666c0 msgr2=0x7f3fd0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.487+0000 7f3ff5400640 1 --2- 172.21.15.161:0/4051526921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3fd00666c0 0x7f3fd0068b80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3ff013fff0 tx=0x7f3fdc009040 comp rx=0 tx=0).stop 2024-05-22T21:19:11.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.487+0000 7f3ff5400640 1 -- 172.21.15.161:0/4051526921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff0143ec0 msgr2=0x7f3ff013f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.487+0000 7f3ff5400640 1 --2- 172.21.15.161:0/4051526921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff0143ec0 0x7f3ff013f7c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00097c0 tx=0x7f3fe001abd0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.487+0000 7f3ff5400640 1 -- 172.21.15.161:0/4051526921 shutdown_connections 2024-05-22T21:19:11.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.487+0000 7f3ff5400640 1 --2- 172.21.15.161:0/4051526921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3fd00666c0 0x7f3fd0068b80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.487+0000 7f3ff5400640 1 --2- 172.21.15.161:0/4051526921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ff015f830 0x7f3ff01641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.487+0000 7f3ff5400640 1 --2- 172.21.15.161:0/4051526921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ff015ec90 0x7f3ff01651a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.487+0000 7f3ff5400640 1 --2- 172.21.15.161:0/4051526921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff0143ec0 0x7f3ff013f7c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.487+0000 7f3ff5400640 1 -- 172.21.15.161:0/4051526921 >> 172.21.15.161:0/4051526921 conn(0x7f3ff00b0bc0 msgr2=0x7f3ff00b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:11.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.487+0000 7f3ff5400640 1 -- 172.21.15.161:0/4051526921 shutdown_connections 2024-05-22T21:19:11.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.487+0000 7f3ff5400640 1 -- 172.21.15.161:0/4051526921 wait complete. 2024-05-22T21:19:11.507 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:11.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.639+0000 7ff5b7600640 1 Processor -- start 2024-05-22T21:19:11.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.639+0000 7ff5b7600640 1 -- start start 2024-05-22T21:19:11.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.643+0000 7ff5b7600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff5b015c6d0 0x7ff5b0164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.643+0000 7ff5b7600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b0071670 con 0x7ff5b015dd50 2024-05-22T21:19:11.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.643+0000 7ff5b7600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b0070a60 con 0x7ff5b015e130 2024-05-22T21:19:11.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.643+0000 7ff5b7600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b0070d40 con 0x7ff5b015c6d0 2024-05-22T21:19:11.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.643+0000 7ff5b4c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff5b015dd50 0x7ff5b006d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41530/0 (socket says 172.21.15.161:41530) 2024-05-22T21:19:11.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.643+0000 7ff5b4c00640 1 -- 172.21.15.161:0/1471584087 learned_addr learned my addr 172.21.15.161:0/1471584087 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:11.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.643+0000 7ff5b5600640 1 --2- 172.21.15.161:0/1471584087 >> v2:172.21.15.161:3300/0 conn(0x7ff5b015c6d0 0x7ff5b0164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.643+0000 7ff5afe00640 1 -- 172.21.15.161:0/1471584087 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3254198695 0 0) 0x7ff5b0071670 con 0x7ff5b015dd50 2024-05-22T21:19:11.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.643+0000 7ff5afe00640 1 -- 172.21.15.161:0/1471584087 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff594003530 con 0x7ff5b015dd50 2024-05-22T21:19:11.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.643+0000 7ff5b5600640 1 -- 172.21.15.161:0/1471584087 >> v1:172.21.15.179:6789/0 conn(0x7ff5b015e130 legacy=0x7ff5b015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.643+0000 7ff5b5600640 1 -- 172.21.15.161:0/1471584087 >> v1:172.21.15.161:6789/0 conn(0x7ff5b015dd50 legacy=0x7ff5b006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.643+0000 7ff5b5600640 1 -- 172.21.15.161:0/1471584087 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b0071020 con 0x7ff5b015c6d0 2024-05-22T21:19:11.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.643+0000 7ff5b5600640 1 --2- 172.21.15.161:0/1471584087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b015c6d0 0x7ff5b0164e80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009d90 tx=0x7ff5a801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28619d2b1fa51c26 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:11.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5afe00640 1 -- 172.21.15.161:0/1471584087 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80a59c0 con 0x7ff5b015c6d0 2024-05-22T21:19:11.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5afe00640 1 -- 172.21.15.161:0/1471584087 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a80a5b20 con 0x7ff5b015c6d0 2024-05-22T21:19:11.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5afe00640 1 -- 172.21.15.161:0/1471584087 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80ad440 con 0x7ff5b015c6d0 2024-05-22T21:19:11.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b7600640 1 -- 172.21.15.161:0/1471584087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b015c6d0 msgr2=0x7ff5b0164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b7600640 1 --2- 172.21.15.161:0/1471584087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b015c6d0 0x7ff5b0164e80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009d90 tx=0x7ff5a801c940 comp rx=0 tx=0).stop 2024-05-22T21:19:11.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b7600640 1 -- 172.21.15.161:0/1471584087 shutdown_connections 2024-05-22T21:19:11.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b7600640 1 --2- 172.21.15.161:0/1471584087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b015c6d0 0x7ff5b0164e80 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b7600640 1 -- 172.21.15.161:0/1471584087 >> 172.21.15.161:0/1471584087 conn(0x7ff5b00b0bc0 msgr2=0x7ff5b00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:11.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b7600640 1 -- 172.21.15.161:0/1471584087 shutdown_connections 2024-05-22T21:19:11.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b7600640 1 -- 172.21.15.161:0/1471584087 wait complete. 2024-05-22T21:19:11.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b7600640 1 Processor -- start 2024-05-22T21:19:11.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b7600640 1 -- start start 2024-05-22T21:19:11.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b7600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5b015c6d0 0x7ff5b0154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b7600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5b015dd50 0x7ff5b0157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b7600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b015e130 0x7ff5b014d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b7600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b0153ac0 con 0x7ff5b015e130 2024-05-22T21:19:11.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b7600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b00ad3f0 con 0x7ff5b015c6d0 2024-05-22T21:19:11.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b7600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b00ad6d0 con 0x7ff5b015dd50 2024-05-22T21:19:11.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b5600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b015e130 0x7ff5b014d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b5600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b015e130 0x7ff5b014d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49098/0 (socket says 172.21.15.161:49098) 2024-05-22T21:19:11.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b5600640 1 -- 172.21.15.161:0/1697069737 learned_addr learned my addr 172.21.15.161:0/1697069737 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:11.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5b4c00640 1 --2- 172.21.15.161:0/1697069737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5b015c6d0 0x7ff5b0154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.647+0000 7ff5a7e00640 1 --2- 172.21.15.161:0/1697069737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5b015dd50 0x7ff5b0157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.651+0000 7ff5b5600640 1 -- 172.21.15.161:0/1697069737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5b015dd50 msgr2=0x7ff5b0157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.651+0000 7ff5b5600640 1 --2- 172.21.15.161:0/1697069737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5b015dd50 0x7ff5b0157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.651+0000 7ff5b5600640 1 -- 172.21.15.161:0/1697069737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5b015c6d0 msgr2=0x7ff5b0154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.651+0000 7ff5b5600640 1 --2- 172.21.15.161:0/1697069737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5b015c6d0 0x7ff5b0154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.651+0000 7ff5b5600640 1 -- 172.21.15.161:0/1697069737 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a8009a70 con 0x7ff5b015e130 2024-05-22T21:19:11.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.651+0000 7ff5b5600640 1 --2- 172.21.15.161:0/1697069737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b015e130 0x7ff5b014d910 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff5a801a570 tx=0x7ff5a80196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:11.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.651+0000 7ff5ad600640 1 -- 172.21.15.161:0/1697069737 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80add10 con 0x7ff5b015e130 2024-05-22T21:19:11.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.651+0000 7ff5ad600640 1 -- 172.21.15.161:0/1697069737 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a80ade70 con 0x7ff5b015e130 2024-05-22T21:19:11.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.651+0000 7ff5ad600640 1 -- 172.21.15.161:0/1697069737 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80b9650 con 0x7ff5b015e130 2024-05-22T21:19:11.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.651+0000 7ff5b7600640 1 -- 172.21.15.161:0/1697069737 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b0153750 con 0x7ff5b015e130 2024-05-22T21:19:11.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.651+0000 7ff5b7600640 1 -- 172.21.15.161:0/1697069737 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b014de50 con 0x7ff5b015e130 2024-05-22T21:19:11.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.651+0000 7ff5b7600640 1 -- 172.21.15.161:0/1697069737 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b0153c30 con 0x7ff5b015e130 2024-05-22T21:19:11.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.651+0000 7ff5ad600640 1 -- 172.21.15.161:0/1697069737 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff5a80b97b0 con 0x7ff5b015e130 2024-05-22T21:19:11.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.651+0000 7ff5ad600640 1 --2- 172.21.15.161:0/1697069737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff58c066690 0x7ff58c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.655+0000 7ff5ad600640 1 -- 172.21.15.161:0/1697069737 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff5a811f530 con 0x7ff5b015e130 2024-05-22T21:19:11.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.655+0000 7ff5b4c00640 1 --2- 172.21.15.161:0/1697069737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff58c066690 0x7ff58c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.655+0000 7ff5b4c00640 1 --2- 172.21.15.161:0/1697069737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff58c066690 0x7ff58c068b50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00011f0 tx=0x7ff5a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:11.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.659+0000 7ff5ad600640 1 -- 172.21.15.161:0/1697069737 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff5a80ecc60 con 0x7ff5b015e130 2024-05-22T21:19:11.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.811+0000 7ff5b7600640 1 -- 172.21.15.161:0/1697069737 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5b0153540 con 0x7ff5b015e130 2024-05-22T21:19:11.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.811+0000 7ff5ad600640 1 -- 172.21.15.161:0/1697069737 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+12448 (secure 0 0 0) 0x7ff5a8009450 con 0x7ff5b015e130 2024-05-22T21:19:11.815 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:11.815 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":121,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:19:11.096000+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:11.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.815+0000 7ff5b7600640 1 -- 172.21.15.161:0/1697069737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff58c066690 msgr2=0x7ff58c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.815+0000 7ff5b7600640 1 --2- 172.21.15.161:0/1697069737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff58c066690 0x7ff58c068b50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00011f0 tx=0x7ff5a0009040 comp rx=0 tx=0).stop 2024-05-22T21:19:11.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.815+0000 7ff5b7600640 1 -- 172.21.15.161:0/1697069737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b015e130 msgr2=0x7ff5b014d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.815+0000 7ff5b7600640 1 --2- 172.21.15.161:0/1697069737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b015e130 0x7ff5b014d910 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff5a801a570 tx=0x7ff5a80196e0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.815+0000 7ff5b7600640 1 -- 172.21.15.161:0/1697069737 shutdown_connections 2024-05-22T21:19:11.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.815+0000 7ff5b7600640 1 --2- 172.21.15.161:0/1697069737 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff58c066690 0x7ff58c068b50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.815+0000 7ff5b7600640 1 --2- 172.21.15.161:0/1697069737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5b015e130 0x7ff5b014d910 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.815+0000 7ff5b7600640 1 --2- 172.21.15.161:0/1697069737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5b015dd50 0x7ff5b0157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.815+0000 7ff5b7600640 1 --2- 172.21.15.161:0/1697069737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5b015c6d0 0x7ff5b0154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.815+0000 7ff5b7600640 1 -- 172.21.15.161:0/1697069737 >> 172.21.15.161:0/1697069737 conn(0x7ff5b00b0bc0 msgr2=0x7ff5b00b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:11.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.815+0000 7ff5b7600640 1 -- 172.21.15.161:0/1697069737 shutdown_connections 2024-05-22T21:19:11.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.815+0000 7ff5b7600640 1 -- 172.21.15.161:0/1697069737 wait complete. 2024-05-22T21:19:11.836 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:19:11.836 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:19:11.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.971+0000 7f1230e00640 1 Processor -- start 2024-05-22T21:19:11.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.971+0000 7f1230e00640 1 -- start start 2024-05-22T21:19:11.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.971+0000 7f1230e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f122c13cfc0 0x7f122c1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.971+0000 7f1230e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f122c15af60 0x7f122c13f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.971+0000 7f1230e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122c1400d0 con 0x7f122c13fc80 2024-05-22T21:19:11.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.971+0000 7f1230e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122c13d3a0 con 0x7f122c15af60 2024-05-22T21:19:11.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.971+0000 7f1230e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122c0cd920 con 0x7f122c13cfc0 2024-05-22T21:19:11.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.971+0000 7f122a800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f122c13fc80 0x7f122c15dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41546/0 (socket says 172.21.15.161:41546) 2024-05-22T21:19:11.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.971+0000 7f122a800640 1 -- 172.21.15.161:0/2286222785 learned_addr learned my addr 172.21.15.161:0/2286222785 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:11.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1221400640 1 --2- 172.21.15.161:0/2286222785 >> v2:172.21.15.179:3300/0 conn(0x7f122c15af60 0x7f122c13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229e00640 1 --2- 172.21.15.161:0/2286222785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f122c13cfc0 0x7f122c1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229400640 1 -- 172.21.15.161:0/2286222785 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804518172 0 0) 0x7f12240014a0 con 0x7f122c13fc80 2024-05-22T21:19:11.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229400640 1 -- 172.21.15.161:0/2286222785 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1210005740 con 0x7f122c13fc80 2024-05-22T21:19:11.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229400640 1 -- 172.21.15.161:0/2286222785 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1297181567 0 0) 0x7f1210005740 con 0x7f122c13fc80 2024-05-22T21:19:11.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229400640 1 -- 172.21.15.161:0/2286222785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f122c13cfc0 msgr2=0x7f122c1402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:11.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229400640 1 --2- 172.21.15.161:0/2286222785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f122c13cfc0 0x7f122c1402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229400640 1 -- 172.21.15.161:0/2286222785 >> v2:172.21.15.179:3300/0 conn(0x7f122c15af60 msgr2=0x7f122c13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:11.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229400640 1 --2- 172.21.15.161:0/2286222785 >> v2:172.21.15.179:3300/0 conn(0x7f122c15af60 0x7f122c13f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229400640 1 -- 172.21.15.161:0/2286222785 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c0cdc00 con 0x7f122c13fc80 2024-05-22T21:19:11.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229400640 1 -- 172.21.15.161:0/2286222785 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f1224001b90 con 0x7f122c13fc80 2024-05-22T21:19:11.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1221400640 1 --2- 172.21.15.161:0/2286222785 >> v2:172.21.15.179:3300/0 conn(0x7f122c15af60 0x7f122c13f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:11.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229400640 1 -- 172.21.15.161:0/2286222785 >> v1:172.21.15.161:6789/0 conn(0x7f122c13fc80 legacy=0x7f122c15dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229400640 1 --2- 172.21.15.161:0/2286222785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f122c13cfc0 0x7f1210005b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229400640 1 --2- 172.21.15.161:0/2286222785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1210006110 0x7f12100069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229400640 1 --2- 172.21.15.161:0/2286222785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1210006f20 0x7f12100073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229400640 1 -- 172.21.15.161:0/2286222785 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c13d3a0 con 0x7f1210006f20 2024-05-22T21:19:11.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229400640 1 -- 172.21.15.161:0/2286222785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c0cd920 con 0x7f1210006110 2024-05-22T21:19:11.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229400640 1 -- 172.21.15.161:0/2286222785 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121000dbf0 con 0x7f122c13cfc0 2024-05-22T21:19:11.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f122a800640 1 --2- 172.21.15.161:0/2286222785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1210006f20 0x7f12100073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1229e00640 1 --2- 172.21.15.161:0/2286222785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f122c13cfc0 0x7f1210005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f1221400640 1 --2- 172.21.15.161:0/2286222785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1210006110 0x7f12100069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f122a800640 1 -- 172.21.15.161:0/2286222785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f122c13cfc0 msgr2=0x7f1210005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:11.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f122a800640 1 --2- 172.21.15.161:0/2286222785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f122c13cfc0 0x7f1210005b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f122a800640 1 -- 172.21.15.161:0/2286222785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1210006110 msgr2=0x7f12100069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:11.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f122a800640 1 --2- 172.21.15.161:0/2286222785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1210006110 0x7f12100069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.975+0000 7f122a800640 1 -- 172.21.15.161:0/2286222785 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1210005740 con 0x7f1210006f20 2024-05-22T21:19:11.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.979+0000 7f122a800640 1 --2- 172.21.15.161:0/2286222785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1210006f20 0x7f12100073a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f122400a860 tx=0x7f122401fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab6ea0692a4ccaf server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:11.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.979+0000 7f1229400640 1 -- 172.21.15.161:0/2286222785 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12240aa6f0 con 0x7f1210006f20 2024-05-22T21:19:11.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.979+0000 7f1229400640 1 -- 172.21.15.161:0/2286222785 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12240aa870 con 0x7f1210006f20 2024-05-22T21:19:11.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.979+0000 7f1229400640 1 -- 172.21.15.161:0/2286222785 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12240aac60 con 0x7f1210006f20 2024-05-22T21:19:11.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.979+0000 7f1230e00640 1 -- 172.21.15.161:0/2286222785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1210006f20 msgr2=0x7f12100073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.979+0000 7f1230e00640 1 --2- 172.21.15.161:0/2286222785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1210006f20 0x7f12100073a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f122400a860 tx=0x7f122401fac0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.979+0000 7f122a800640 1 -- 172.21.15.161:0/2286222785 reap_dead start 2024-05-22T21:19:11.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.979+0000 7f1230e00640 1 -- 172.21.15.161:0/2286222785 shutdown_connections 2024-05-22T21:19:11.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.979+0000 7f1230e00640 1 -- 172.21.15.161:0/2286222785 >> 172.21.15.161:0/2286222785 conn(0x7f122c06a2d0 msgr2=0x7f122c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:11.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.979+0000 7f1230e00640 1 -- 172.21.15.161:0/2286222785 shutdown_connections 2024-05-22T21:19:11.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.979+0000 7f1230e00640 1 -- 172.21.15.161:0/2286222785 wait complete. 2024-05-22T21:19:11.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.979+0000 7f1230e00640 1 Processor -- start 2024-05-22T21:19:11.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.979+0000 7f1230e00640 1 -- start start 2024-05-22T21:19:11.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.979+0000 7f1230e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1210006f20 0x7f122c13bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.979+0000 7f1230e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f122c0b2f00 0x7f122c13a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1230e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f122c0b3380 0x7f122c13a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1230e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c142690 con 0x7f122c0b2f00 2024-05-22T21:19:11.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1230e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c142200 con 0x7f122c0b3380 2024-05-22T21:19:11.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1230e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122c13c1e0 con 0x7f1210006f20 2024-05-22T21:19:11.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1221400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f122c0b2f00 0x7f122c13a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1221400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f122c0b2f00 0x7f122c13a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49116/0 (socket says 172.21.15.161:49116) 2024-05-22T21:19:11.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1221400640 1 -- 172.21.15.161:0/368994215 learned_addr learned my addr 172.21.15.161:0/368994215 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:11.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1229e00640 1 --2- 172.21.15.161:0/368994215 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1210006f20 0x7f122c13bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f122a800640 1 --2- 172.21.15.161:0/368994215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f122c0b3380 0x7f122c13a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1221400640 1 -- 172.21.15.161:0/368994215 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1210006f20 msgr2=0x7f122c13bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1221400640 1 --2- 172.21.15.161:0/368994215 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1210006f20 0x7f122c13bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1221400640 1 -- 172.21.15.161:0/368994215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f122c0b3380 msgr2=0x7f122c13a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:11.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1221400640 1 --2- 172.21.15.161:0/368994215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f122c0b3380 0x7f122c13a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:11.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1221400640 1 -- 172.21.15.161:0/368994215 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c1415f0 con 0x7f122c0b2f00 2024-05-22T21:19:11.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1221400640 1 --2- 172.21.15.161:0/368994215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f122c0b2f00 0x7f122c13a0f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1214011d30 tx=0x7f121400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:11.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1222a00640 1 -- 172.21.15.161:0/368994215 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121400fd20 con 0x7f122c0b2f00 2024-05-22T21:19:11.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1222a00640 1 -- 172.21.15.161:0/368994215 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121400fe80 con 0x7f122c0b2f00 2024-05-22T21:19:11.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1230e00640 1 -- 172.21.15.161:0/368994215 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122c1418d0 con 0x7f122c0b2f00 2024-05-22T21:19:11.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1222a00640 1 -- 172.21.15.161:0/368994215 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214004270 con 0x7f122c0b2f00 2024-05-22T21:19:11.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1230e00640 1 -- 172.21.15.161:0/368994215 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122c141e90 con 0x7f122c0b2f00 2024-05-22T21:19:11.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1230e00640 1 -- 172.21.15.161:0/368994215 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122c1423b0 con 0x7f122c0b2f00 2024-05-22T21:19:11.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1222a00640 1 -- 172.21.15.161:0/368994215 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f12140043d0 con 0x7f122c0b2f00 2024-05-22T21:19:11.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.983+0000 7f1222a00640 1 --2- 172.21.15.161:0/368994215 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1208066690 0x7f1208068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:11.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.987+0000 7f1222a00640 1 -- 172.21.15.161:0/368994215 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1214084630 con 0x7f122c0b2f00 2024-05-22T21:19:11.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.987+0000 7f1229e00640 1 --2- 172.21.15.161:0/368994215 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1208066690 0x7f1208068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:11.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.987+0000 7f1229e00640 1 --2- 172.21.15.161:0/368994215 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1208066690 0x7f1208068b50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f121c009be0 tx=0x7f121c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:11.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:11.987+0000 7f1222a00640 1 -- 172.21.15.161:0/368994215 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f122c1423b0 con 0x7f122c0b2f00 2024-05-22T21:19:12.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.143+0000 7f1230e00640 1 -- 172.21.15.161:0/368994215 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f122c1423b0 con 0x7f122c0b2f00 2024-05-22T21:19:12.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.143+0000 7f1222a00640 1 -- 172.21.15.161:0/368994215 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f122c1423b0 con 0x7f122c0b2f00 2024-05-22T21:19:12.146 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:19:12.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.147+0000 7f1230e00640 1 -- 172.21.15.161:0/368994215 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1208066690 msgr2=0x7f1208068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:12.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.147+0000 7f1230e00640 1 --2- 172.21.15.161:0/368994215 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1208066690 0x7f1208068b50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f121c009be0 tx=0x7f121c009190 comp rx=0 tx=0).stop 2024-05-22T21:19:12.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.147+0000 7f1230e00640 1 -- 172.21.15.161:0/368994215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f122c0b2f00 msgr2=0x7f122c13a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.147+0000 7f1230e00640 1 --2- 172.21.15.161:0/368994215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f122c0b2f00 0x7f122c13a0f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1214011d30 tx=0x7f121400f610 comp rx=0 tx=0).stop 2024-05-22T21:19:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.147+0000 7f1230e00640 1 -- 172.21.15.161:0/368994215 shutdown_connections 2024-05-22T21:19:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.147+0000 7f1230e00640 1 --2- 172.21.15.161:0/368994215 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1208066690 0x7f1208068b50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.147+0000 7f1230e00640 1 --2- 172.21.15.161:0/368994215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f122c0b3380 0x7f122c13a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.147+0000 7f1230e00640 1 --2- 172.21.15.161:0/368994215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f122c0b2f00 0x7f122c13a0f0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.147+0000 7f1230e00640 1 --2- 172.21.15.161:0/368994215 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1210006f20 0x7f122c13bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.147+0000 7f1230e00640 1 -- 172.21.15.161:0/368994215 >> 172.21.15.161:0/368994215 conn(0x7f122c06a2d0 msgr2=0x7f122c15da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.147+0000 7f1230e00640 1 -- 172.21.15.161:0/368994215 shutdown_connections 2024-05-22T21:19:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.147+0000 7f1230e00640 1 -- 172.21.15.161:0/368994215 wait complete. 2024-05-22T21:19:12.167 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-22T21:19:12.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.299+0000 7fc944a00640 1 Processor -- start 2024-05-22T21:19:12.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.299+0000 7fc944a00640 1 -- start start 2024-05-22T21:19:12.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.299+0000 7fc944a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc9400715a0 0x7fc94015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:12.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.299+0000 7fc944a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc94015ec90 0x7fc94015f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:12.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.299+0000 7fc944a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9401442a0 con 0x7fc940143ec0 2024-05-22T21:19:12.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.299+0000 7fc944a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc94015f660 con 0x7fc9400715a0 2024-05-22T21:19:12.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.299+0000 7fc944a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc940070500 con 0x7fc94015ec90 2024-05-22T21:19:12.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.299+0000 7fc935400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc9400715a0 0x7fc94015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:12.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.299+0000 7fc93e800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc94015ec90 0x7fc94015f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:12.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.299+0000 7fc935400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc9400715a0 0x7fc94015f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:39832/0 (socket says 172.21.15.161:39832) 2024-05-22T21:19:12.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.299+0000 7fc935400640 1 -- 172.21.15.161:0/4044641065 learned_addr learned my addr 172.21.15.161:0/4044641065 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:12.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.299+0000 7fc93de00640 1 --1- 172.21.15.161:0/4044641065 >> v1:172.21.15.179:6789/0 conn(0x7fc940143ec0 0x7fc94006fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:46052/0 (socket says 172.21.15.161:46052) 2024-05-22T21:19:12.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.303+0000 7fc935400640 1 -- 172.21.15.161:0/4044641065 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc94015ec90 msgr2=0x7fc94015f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:12.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.303+0000 7fc935400640 1 --2- 172.21.15.161:0/4044641065 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc94015ec90 0x7fc94015f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.303+0000 7fc935400640 1 -- 172.21.15.161:0/4044641065 >> v1:172.21.15.179:6789/0 conn(0x7fc940143ec0 legacy=0x7fc94006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:12.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.303+0000 7fc935400640 1 -- 172.21.15.161:0/4044641065 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9400707e0 con 0x7fc9400715a0 2024-05-22T21:19:12.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.303+0000 7fc93d400640 1 -- 172.21.15.161:0/4044641065 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864086558 0 0) 0x7fc9401442a0 con 0x7fc940143ec0 2024-05-22T21:19:12.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.303+0000 7fc935400640 1 --2- 172.21.15.161:0/4044641065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9400715a0 0x7fc94015f830 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc928009e50 tx=0x7fc92801ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0733971f7b7b9bb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:12.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.303+0000 7fc93d400640 1 -- 172.21.15.161:0/4044641065 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9280a57d0 con 0x7fc9400715a0 2024-05-22T21:19:12.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.303+0000 7fc93d400640 1 -- 172.21.15.161:0/4044641065 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9280a5930 con 0x7fc9400715a0 2024-05-22T21:19:12.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.303+0000 7fc93d400640 1 -- 172.21.15.161:0/4044641065 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9280a5cc0 con 0x7fc9400715a0 2024-05-22T21:19:12.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.303+0000 7fc944a00640 1 -- 172.21.15.161:0/4044641065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9400715a0 msgr2=0x7fc94015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:12.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.303+0000 7fc944a00640 1 --2- 172.21.15.161:0/4044641065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9400715a0 0x7fc94015f830 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc928009e50 tx=0x7fc92801ca00 comp rx=0 tx=0).stop 2024-05-22T21:19:12.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc944a00640 1 -- 172.21.15.161:0/4044641065 shutdown_connections 2024-05-22T21:19:12.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc944a00640 1 --2- 172.21.15.161:0/4044641065 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc94015ec90 0x7fc94015f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc944a00640 1 --2- 172.21.15.161:0/4044641065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9400715a0 0x7fc94015f830 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc944a00640 1 -- 172.21.15.161:0/4044641065 >> 172.21.15.161:0/4044641065 conn(0x7fc9400b0bc0 msgr2=0x7fc9400b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:12.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc944a00640 1 -- 172.21.15.161:0/4044641065 shutdown_connections 2024-05-22T21:19:12.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc944a00640 1 -- 172.21.15.161:0/4044641065 wait complete. 2024-05-22T21:19:12.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc944a00640 1 Processor -- start 2024-05-22T21:19:12.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc944a00640 1 -- start start 2024-05-22T21:19:12.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc944a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc940143ec0 0x7fc94013a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:12.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc944a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc94015ec90 0x7fc9400a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:12.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc944a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc940159670 0x7fc94013fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:12.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc944a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94013af30 con 0x7fc940143ec0 2024-05-22T21:19:12.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc944a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9400a89e0 con 0x7fc94015ec90 2024-05-22T21:19:12.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc944a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94013a3b0 con 0x7fc940159670 2024-05-22T21:19:12.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc93de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc940143ec0 0x7fc94013a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:12.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc93de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc940143ec0 0x7fc94013a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49126/0 (socket says 172.21.15.161:49126) 2024-05-22T21:19:12.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc93de00640 1 -- 172.21.15.161:0/2132241185 learned_addr learned my addr 172.21.15.161:0/2132241185 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:12.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc935400640 1 --2- 172.21.15.161:0/2132241185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc94015ec90 0x7fc9400a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:12.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc93e800640 1 --2- 172.21.15.161:0/2132241185 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc940159670 0x7fc94013fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:12.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc93de00640 1 -- 172.21.15.161:0/2132241185 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc940159670 msgr2=0x7fc94013fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:12.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc93de00640 1 --2- 172.21.15.161:0/2132241185 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc940159670 0x7fc94013fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc93de00640 1 -- 172.21.15.161:0/2132241185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc94015ec90 msgr2=0x7fc9400a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:12.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc93de00640 1 --2- 172.21.15.161:0/2132241185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc94015ec90 0x7fc9400a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc93de00640 1 -- 172.21.15.161:0/2132241185 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc928009b30 con 0x7fc940143ec0 2024-05-22T21:19:12.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc935400640 1 --2- 172.21.15.161:0/2132241185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc94015ec90 0x7fc9400a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:12.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.307+0000 7fc93de00640 1 --2- 172.21.15.161:0/2132241185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc940143ec0 0x7fc94013a9f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc9300011f0 tx=0x7fc93000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:12.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.311+0000 7fc936a00640 1 -- 172.21.15.161:0/2132241185 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9300074e0 con 0x7fc940143ec0 2024-05-22T21:19:12.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.311+0000 7fc936a00640 1 -- 172.21.15.161:0/2132241185 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc930007640 con 0x7fc940143ec0 2024-05-22T21:19:12.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.311+0000 7fc944a00640 1 -- 172.21.15.161:0/2132241185 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94013a690 con 0x7fc940143ec0 2024-05-22T21:19:12.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.311+0000 7fc936a00640 1 -- 172.21.15.161:0/2132241185 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc930004a10 con 0x7fc940143ec0 2024-05-22T21:19:12.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.311+0000 7fc944a00640 1 -- 172.21.15.161:0/2132241185 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94015c240 con 0x7fc940143ec0 2024-05-22T21:19:12.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.311+0000 7fc944a00640 1 -- 172.21.15.161:0/2132241185 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc940067220 con 0x7fc940143ec0 2024-05-22T21:19:12.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.311+0000 7fc936a00640 1 -- 172.21.15.161:0/2132241185 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc930004b70 con 0x7fc940143ec0 2024-05-22T21:19:12.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.311+0000 7fc936a00640 1 --2- 172.21.15.161:0/2132241185 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc91c066690 0x7fc91c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:12.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.311+0000 7fc936a00640 1 -- 172.21.15.161:0/2132241185 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc9300833b0 con 0x7fc940143ec0 2024-05-22T21:19:12.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.311+0000 7fc935400640 1 --2- 172.21.15.161:0/2132241185 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc91c066690 0x7fc91c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:12.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.315+0000 7fc935400640 1 --2- 172.21.15.161:0/2132241185 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc91c066690 0x7fc91c068b50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc9280ae000 tx=0x7fc9280a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:12.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.315+0000 7fc936a00640 1 -- 172.21.15.161:0/2132241185 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc930050a80 con 0x7fc940143ec0 2024-05-22T21:19:12.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.471+0000 7fc944a00640 1 -- 172.21.15.161:0/2132241185 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc940072050 con 0x7fc940143ec0 2024-05-22T21:19:12.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.471+0000 7fc936a00640 1 -- 172.21.15.161:0/2132241185 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v121) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc930018060 con 0x7fc940143ec0 2024-05-22T21:19:12.475 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:12.475 INFO:teuthology.orchestra.run.smithi161.stdout:[ 2024-05-22T21:19:12.475 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:12.475 INFO:teuthology.orchestra.run.smithi161.stdout: "poolnum": 1, 2024-05-22T21:19:12.475 INFO:teuthology.orchestra.run.smithi161.stdout: "poolname": ".mgr" 2024-05-22T21:19:12.475 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:12.475 INFO:teuthology.orchestra.run.smithi161.stdout:] 2024-05-22T21:19:12.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.475+0000 7fc944a00640 1 -- 172.21.15.161:0/2132241185 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc91c066690 msgr2=0x7fc91c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:12.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.475+0000 7fc944a00640 1 --2- 172.21.15.161:0/2132241185 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc91c066690 0x7fc91c068b50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc9280ae000 tx=0x7fc9280a7070 comp rx=0 tx=0).stop 2024-05-22T21:19:12.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.475+0000 7fc944a00640 1 -- 172.21.15.161:0/2132241185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc940143ec0 msgr2=0x7fc94013a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.475+0000 7fc944a00640 1 --2- 172.21.15.161:0/2132241185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc940143ec0 0x7fc94013a9f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc9300011f0 tx=0x7fc93000c9b0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.475+0000 7fc944a00640 1 -- 172.21.15.161:0/2132241185 shutdown_connections 2024-05-22T21:19:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.475+0000 7fc944a00640 1 --2- 172.21.15.161:0/2132241185 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc91c066690 0x7fc91c068b50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.475+0000 7fc944a00640 1 --2- 172.21.15.161:0/2132241185 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc940159670 0x7fc94013fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.475+0000 7fc944a00640 1 --2- 172.21.15.161:0/2132241185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc94015ec90 0x7fc9400a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.475+0000 7fc944a00640 1 --2- 172.21.15.161:0/2132241185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc940143ec0 0x7fc94013a9f0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.475+0000 7fc944a00640 1 -- 172.21.15.161:0/2132241185 >> 172.21.15.161:0/2132241185 conn(0x7fc9400b0bc0 msgr2=0x7fc9400ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.475+0000 7fc944a00640 1 -- 172.21.15.161:0/2132241185 shutdown_connections 2024-05-22T21:19:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.475+0000 7fc944a00640 1 -- 172.21.15.161:0/2132241185 wait complete. 2024-05-22T21:19:12.495 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-22T21:19:12.495 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-22T21:19:12.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.631+0000 7fb800c00640 1 Processor -- start 2024-05-22T21:19:12.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.631+0000 7fb800c00640 1 -- start start 2024-05-22T21:19:12.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.631+0000 7fb800c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7fc06d560 0x7fb7fc0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:12.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.631+0000 7fb800c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb7fc0cca40 0x7fb7fc15c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:12.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.631+0000 7fb800c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7fc06d0d0 con 0x7fb7fc144980 2024-05-22T21:19:12.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.631+0000 7fb800c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7fc0cced0 con 0x7fb7fc0cca40 2024-05-22T21:19:12.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.631+0000 7fb800c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc06c4c0 con 0x7fb7fc06d560 2024-05-22T21:19:12.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.631+0000 7fb7fa800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb7fc0cca40 0x7fb7fc15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:12.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.631+0000 7fb7fa800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb7fc0cca40 0x7fb7fc15c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49138/0 (socket says 172.21.15.161:49138) 2024-05-22T21:19:12.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.631+0000 7fb7fa800640 1 -- 172.21.15.161:0/659320194 learned_addr learned my addr 172.21.15.161:0/659320194 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:12.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.635+0000 7fb7f9400640 1 --2- 172.21.15.161:0/659320194 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7fc06d560 0x7fb7fc0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:12.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.635+0000 7fb7fa800640 1 -- 172.21.15.161:0/659320194 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7fc06d560 msgr2=0x7fb7fc0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:12.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.635+0000 7fb7fa800640 1 --2- 172.21.15.161:0/659320194 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7fc06d560 0x7fb7fc0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.635+0000 7fb7fa800640 1 -- 172.21.15.161:0/659320194 >> v1:172.21.15.179:6789/0 conn(0x7fb7fc144980 legacy=0x7fb7fc15c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:12.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.635+0000 7fb7fa800640 1 -- 172.21.15.161:0/659320194 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc06c7a0 con 0x7fb7fc0cca40 2024-05-22T21:19:12.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.635+0000 7fb7f9400640 1 --2- 172.21.15.161:0/659320194 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7fc06d560 0x7fb7fc0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:12.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.635+0000 7fb7fa800640 1 --2- 172.21.15.161:0/659320194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc0cca40 0x7fb7fc15c880 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009870 tx=0x7fb7f401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c287074553d844e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:12.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.635+0000 7fb7f8a00640 1 -- 172.21.15.161:0/659320194 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f40ab610 con 0x7fb7fc0cca40 2024-05-22T21:19:12.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.635+0000 7fb7f8a00640 1 -- 172.21.15.161:0/659320194 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f40ab770 con 0x7fb7fc0cca40 2024-05-22T21:19:12.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.635+0000 7fb7f8a00640 1 -- 172.21.15.161:0/659320194 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f40abb00 con 0x7fb7fc0cca40 2024-05-22T21:19:12.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.635+0000 7fb800c00640 1 -- 172.21.15.161:0/659320194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc0cca40 msgr2=0x7fb7fc15c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:12.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.635+0000 7fb800c00640 1 --2- 172.21.15.161:0/659320194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc0cca40 0x7fb7fc15c880 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009870 tx=0x7fb7f401c560 comp rx=0 tx=0).stop 2024-05-22T21:19:12.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb800c00640 1 -- 172.21.15.161:0/659320194 shutdown_connections 2024-05-22T21:19:12.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb800c00640 1 --2- 172.21.15.161:0/659320194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc0cca40 0x7fb7fc15c880 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb800c00640 1 --2- 172.21.15.161:0/659320194 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7fc06d560 0x7fb7fc0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb800c00640 1 -- 172.21.15.161:0/659320194 >> 172.21.15.161:0/659320194 conn(0x7fb7fc06a0a0 msgr2=0x7fb7fc06a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:12.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb800c00640 1 -- 172.21.15.161:0/659320194 shutdown_connections 2024-05-22T21:19:12.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb800c00640 1 -- 172.21.15.161:0/659320194 wait complete. 2024-05-22T21:19:12.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb800c00640 1 Processor -- start 2024-05-22T21:19:12.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb800c00640 1 -- start start 2024-05-22T21:19:12.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb800c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7fc06d560 0x7fb7fc13f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:12.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb800c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7fc0cca40 0x7fb7fc138b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:12.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb800c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc144980 0x7fb7fc15b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:12.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb800c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc13a820 con 0x7fb7fc144980 2024-05-22T21:19:12.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb800c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc1390a0 con 0x7fb7fc0cca40 2024-05-22T21:19:12.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb800c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc13aa90 con 0x7fb7fc06d560 2024-05-22T21:19:12.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb7fa800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc144980 0x7fb7fc15b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:12.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb7fa800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc144980 0x7fb7fc15b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49150/0 (socket says 172.21.15.161:49150) 2024-05-22T21:19:12.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb7fa800640 1 -- 172.21.15.161:0/1705515688 learned_addr learned my addr 172.21.15.161:0/1705515688 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:12.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb7f9400640 1 --2- 172.21.15.161:0/1705515688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7fc0cca40 0x7fb7fc138b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:12.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb7f9e00640 1 --2- 172.21.15.161:0/1705515688 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7fc06d560 0x7fb7fc13f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:12.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb7fa800640 1 -- 172.21.15.161:0/1705515688 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7fc06d560 msgr2=0x7fb7fc13f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:12.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb7fa800640 1 --2- 172.21.15.161:0/1705515688 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7fc06d560 0x7fb7fc13f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb7fa800640 1 -- 172.21.15.161:0/1705515688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7fc0cca40 msgr2=0x7fb7fc138b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:12.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb7fa800640 1 --2- 172.21.15.161:0/1705515688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7fc0cca40 0x7fb7fc138b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb7fa800640 1 -- 172.21.15.161:0/1705515688 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f40094d0 con 0x7fb7fc144980 2024-05-22T21:19:12.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb7fa800640 1 --2- 172.21.15.161:0/1705515688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc144980 0x7fb7fc15b870 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40ad040 tx=0x7fb7f401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:12.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb7f2000640 1 -- 172.21.15.161:0/1705515688 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f40b4800 con 0x7fb7fc144980 2024-05-22T21:19:12.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb7f2000640 1 -- 172.21.15.161:0/1705515688 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f40b4980 con 0x7fb7fc144980 2024-05-22T21:19:12.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb7f2000640 1 -- 172.21.15.161:0/1705515688 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f40b4d30 con 0x7fb7fc144980 2024-05-22T21:19:12.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.639+0000 7fb800c00640 1 -- 172.21.15.161:0/1705515688 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7fc158560 con 0x7fb7fc144980 2024-05-22T21:19:12.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.643+0000 7fb800c00640 1 -- 172.21.15.161:0/1705515688 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc158b20 con 0x7fb7fc144980 2024-05-22T21:19:12.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.643+0000 7fb800c00640 1 -- 172.21.15.161:0/1705515688 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7fc15b590 con 0x7fb7fc144980 2024-05-22T21:19:12.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.643+0000 7fb7f2000640 1 -- 172.21.15.161:0/1705515688 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb7f40b3380 con 0x7fb7fc144980 2024-05-22T21:19:12.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.643+0000 7fb7f2000640 1 --2- 172.21.15.161:0/1705515688 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7dc066690 0x7fb7dc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:12.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.643+0000 7fb7f2000640 1 -- 172.21.15.161:0/1705515688 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb7f411e450 con 0x7fb7fc144980 2024-05-22T21:19:12.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.643+0000 7fb7f9e00640 1 --2- 172.21.15.161:0/1705515688 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7dc066690 0x7fb7dc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:12.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.643+0000 7fb7f9e00640 1 --2- 172.21.15.161:0/1705515688 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7dc066690 0x7fb7dc068b50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc13c020 tx=0x7fb7ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:12.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.647+0000 7fb7f2000640 1 -- 172.21.15.161:0/1705515688 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb7fc15b590 con 0x7fb7fc144980 2024-05-22T21:19:12.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.799+0000 7fb800c00640 1 -- 172.21.15.161:0/1705515688 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb7fc15b590 con 0x7fb7fc144980 2024-05-22T21:19:12.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.959+0000 7fb7f2000640 1 -- 172.21.15.161:0/1705515688 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v122) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb7fc15b590 con 0x7fb7fc144980 2024-05-22T21:19:12.961 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' created 2024-05-22T21:19:12.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.963+0000 7fb800c00640 1 -- 172.21.15.161:0/1705515688 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7dc066690 msgr2=0x7fb7dc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:12.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.963+0000 7fb800c00640 1 --2- 172.21.15.161:0/1705515688 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7dc066690 0x7fb7dc068b50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc13c020 tx=0x7fb7ec009040 comp rx=0 tx=0).stop 2024-05-22T21:19:12.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.963+0000 7fb800c00640 1 -- 172.21.15.161:0/1705515688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc144980 msgr2=0x7fb7fc15b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:12.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.963+0000 7fb800c00640 1 --2- 172.21.15.161:0/1705515688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc144980 0x7fb7fc15b870 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40ad040 tx=0x7fb7f401ada0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.963+0000 7fb800c00640 1 -- 172.21.15.161:0/1705515688 shutdown_connections 2024-05-22T21:19:12.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.963+0000 7fb800c00640 1 --2- 172.21.15.161:0/1705515688 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7dc066690 0x7fb7dc068b50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.963+0000 7fb800c00640 1 --2- 172.21.15.161:0/1705515688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7fc144980 0x7fb7fc15b870 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.963+0000 7fb800c00640 1 --2- 172.21.15.161:0/1705515688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7fc0cca40 0x7fb7fc138b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.963+0000 7fb800c00640 1 --2- 172.21.15.161:0/1705515688 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb7fc06d560 0x7fb7fc13f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:12.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.963+0000 7fb800c00640 1 -- 172.21.15.161:0/1705515688 >> 172.21.15.161:0/1705515688 conn(0x7fb7fc06a0a0 msgr2=0x7fb7fc143360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:12.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.963+0000 7fb800c00640 1 -- 172.21.15.161:0/1705515688 shutdown_connections 2024-05-22T21:19:12.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:12.963+0000 7fb800c00640 1 -- 172.21.15.161:0/1705515688 wait complete. 2024-05-22T21:19:12.984 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-22T21:19:13.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.123+0000 7ff8f8800640 1 Processor -- start 2024-05-22T21:19:13.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.123+0000 7ff8f8800640 1 -- start start 2024-05-22T21:19:13.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.123+0000 7ff8f8800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8f013cfc0 0x7ff8f01402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:13.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.123+0000 7ff8f8800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff8f013fe50 0x7ff8f006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:13.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.123+0000 7ff8f8800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f013d3a0 con 0x7ff8f015af60 2024-05-22T21:19:13.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.123+0000 7ff8f8800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f00cd920 con 0x7ff8f013fe50 2024-05-22T21:19:13.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.123+0000 7ff8f8800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f00cdc00 con 0x7ff8f013cfc0 2024-05-22T21:19:13.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.127+0000 7ff8f6800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff8f013fe50 0x7ff8f006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:13.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.127+0000 7ff8f5e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8f013cfc0 0x7ff8f01402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:13.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.127+0000 7ff8f6800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff8f013fe50 0x7ff8f006fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:39868/0 (socket says 172.21.15.161:39868) 2024-05-22T21:19:13.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.127+0000 7ff8f5400640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff8f015af60 0x7ff8f013f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:46072/0 (socket says 172.21.15.161:46072) 2024-05-22T21:19:13.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.127+0000 7ff8f6800640 1 -- 172.21.15.161:0/3223411830 learned_addr learned my addr 172.21.15.161:0/3223411830 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:13.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.127+0000 7ff8f6800640 1 -- 172.21.15.161:0/3223411830 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8f013cfc0 msgr2=0x7ff8f01402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:13.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.127+0000 7ff8f6800640 1 --2- 172.21.15.161:0/3223411830 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8f013cfc0 0x7ff8f01402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:13.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.127+0000 7ff8f6800640 1 -- 172.21.15.161:0/3223411830 >> v1:172.21.15.179:6789/0 conn(0x7ff8f015af60 legacy=0x7ff8f013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:13.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.127+0000 7ff8f6800640 1 -- 172.21.15.161:0/3223411830 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f00cdee0 con 0x7ff8f013fe50 2024-05-22T21:19:13.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.127+0000 7ff8f4a00640 1 -- 172.21.15.161:0/3223411830 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319690794 0 0) 0x7ff8f013d3a0 con 0x7ff8f015af60 2024-05-22T21:19:13.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.127+0000 7ff8f5e00640 1 --2- 172.21.15.161:0/3223411830 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8f013cfc0 0x7ff8f01402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:13.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.127+0000 7ff8f6800640 1 --2- 172.21.15.161:0/3223411830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8f013fe50 0x7ff8f006fce0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009e50 tx=0x7ff8ec01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a11de5185ee23364 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:13.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.127+0000 7ff8f4a00640 1 -- 172.21.15.161:0/3223411830 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec0a6470 con 0x7ff8f013fe50 2024-05-22T21:19:13.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.127+0000 7ff8f4a00640 1 -- 172.21.15.161:0/3223411830 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ec0a65d0 con 0x7ff8f013fe50 2024-05-22T21:19:13.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.127+0000 7ff8f4a00640 1 -- 172.21.15.161:0/3223411830 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec0a6960 con 0x7ff8f013fe50 2024-05-22T21:19:13.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f8800640 1 -- 172.21.15.161:0/3223411830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8f013fe50 msgr2=0x7ff8f006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:13.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f8800640 1 --2- 172.21.15.161:0/3223411830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8f013fe50 0x7ff8f006fce0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009e50 tx=0x7ff8ec01c970 comp rx=0 tx=0).stop 2024-05-22T21:19:13.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f8800640 1 -- 172.21.15.161:0/3223411830 shutdown_connections 2024-05-22T21:19:13.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f8800640 1 --2- 172.21.15.161:0/3223411830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8f013fe50 0x7ff8f006fce0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:13.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f8800640 1 --2- 172.21.15.161:0/3223411830 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8f013cfc0 0x7ff8f01402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:13.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f8800640 1 -- 172.21.15.161:0/3223411830 >> 172.21.15.161:0/3223411830 conn(0x7ff8f006a2d0 msgr2=0x7ff8f006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:13.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f8800640 1 -- 172.21.15.161:0/3223411830 shutdown_connections 2024-05-22T21:19:13.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f8800640 1 -- 172.21.15.161:0/3223411830 wait complete. 2024-05-22T21:19:13.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f8800640 1 Processor -- start 2024-05-22T21:19:13.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f8800640 1 -- start start 2024-05-22T21:19:13.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f8800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8f013cfc0 0x7ff8f013ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:13.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f8800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8f013fe50 0x7ff8f00b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:13.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f8800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8f015af60 0x7ff8f013bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:13.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f8800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f013efb0 con 0x7ff8f013fe50 2024-05-22T21:19:13.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f8800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f00b2cc0 con 0x7ff8f015af60 2024-05-22T21:19:13.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f8800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f00b2f40 con 0x7ff8f013cfc0 2024-05-22T21:19:13.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f5400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8f013fe50 0x7ff8f00b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:13.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f5400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8f013fe50 0x7ff8f00b2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49162/0 (socket says 172.21.15.161:49162) 2024-05-22T21:19:13.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.131+0000 7ff8f5400640 1 -- 172.21.15.161:0/2758972146 learned_addr learned my addr 172.21.15.161:0/2758972146 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:13.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8f5e00640 1 --2- 172.21.15.161:0/2758972146 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8f013cfc0 0x7ff8f013ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:13.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8f6800640 1 --2- 172.21.15.161:0/2758972146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8f015af60 0x7ff8f013bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:13.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8f5400640 1 -- 172.21.15.161:0/2758972146 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8f013cfc0 msgr2=0x7ff8f013ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:13.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8f5400640 1 --2- 172.21.15.161:0/2758972146 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8f013cfc0 0x7ff8f013ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:13.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8f5400640 1 -- 172.21.15.161:0/2758972146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8f015af60 msgr2=0x7ff8f013bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:13.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8f5400640 1 --2- 172.21.15.161:0/2758972146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8f015af60 0x7ff8f013bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:13.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8f5400640 1 -- 172.21.15.161:0/2758972146 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e4009420 con 0x7ff8f013fe50 2024-05-22T21:19:13.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8f5e00640 1 --2- 172.21.15.161:0/2758972146 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8f013cfc0 0x7ff8f013ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:13.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8f5400640 1 --2- 172.21.15.161:0/2758972146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8f013fe50 0x7ff8f00b2780 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000c480 tx=0x7ff8e000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:13.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8ea000640 1 -- 172.21.15.161:0/2758972146 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00074e0 con 0x7ff8f013fe50 2024-05-22T21:19:13.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8f8800640 1 -- 172.21.15.161:0/2758972146 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ec009b30 con 0x7ff8f013fe50 2024-05-22T21:19:13.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8ea000640 1 -- 172.21.15.161:0/2758972146 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e0007660 con 0x7ff8f013fe50 2024-05-22T21:19:13.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8ea000640 1 -- 172.21.15.161:0/2758972146 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0004a30 con 0x7ff8f013fe50 2024-05-22T21:19:13.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8f8800640 1 -- 172.21.15.161:0/2758972146 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f00b3500 con 0x7ff8f013fe50 2024-05-22T21:19:13.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8f8800640 1 -- 172.21.15.161:0/2758972146 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f0067220 con 0x7ff8f013fe50 2024-05-22T21:19:13.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8ea000640 1 -- 172.21.15.161:0/2758972146 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff8e0004b90 con 0x7ff8f013fe50 2024-05-22T21:19:13.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.135+0000 7ff8ea000640 1 --2- 172.21.15.161:0/2758972146 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8d4066690 0x7ff8d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:13.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.139+0000 7ff8ea000640 1 -- 172.21.15.161:0/2758972146 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff8e0083740 con 0x7ff8f013fe50 2024-05-22T21:19:13.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.139+0000 7ff8f5e00640 1 --2- 172.21.15.161:0/2758972146 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8d4066690 0x7ff8d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:13.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.139+0000 7ff8f5e00640 1 --2- 172.21.15.161:0/2758972146 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8d4066690 0x7ff8d4068b50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00b2550 tx=0x7ff8e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:13.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.139+0000 7ff8ea000640 1 -- 172.21.15.161:0/2758972146 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff8e0054710 con 0x7ff8f013fe50 2024-05-22T21:19:13.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.299+0000 7ff8f8800640 1 -- 172.21.15.161:0/2758972146 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff8f00cd040 con 0x7ff8f013fe50 2024-05-22T21:19:13.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.959+0000 7ff8ea000640 1 -- 172.21.15.161:0/2758972146 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v123) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff8e0018020 con 0x7ff8f013fe50 2024-05-22T21:19:13.968 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' created 2024-05-22T21:19:13.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.971+0000 7ff8f8800640 1 -- 172.21.15.161:0/2758972146 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8d4066690 msgr2=0x7ff8d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:13.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.971+0000 7ff8f8800640 1 --2- 172.21.15.161:0/2758972146 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8d4066690 0x7ff8d4068b50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00b2550 tx=0x7ff8e4008040 comp rx=0 tx=0).stop 2024-05-22T21:19:13.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.971+0000 7ff8f8800640 1 -- 172.21.15.161:0/2758972146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8f013fe50 msgr2=0x7ff8f00b2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:13.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.971+0000 7ff8f8800640 1 --2- 172.21.15.161:0/2758972146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8f013fe50 0x7ff8f00b2780 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000c480 tx=0x7ff8e000c940 comp rx=0 tx=0).stop 2024-05-22T21:19:13.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.971+0000 7ff8f8800640 1 -- 172.21.15.161:0/2758972146 shutdown_connections 2024-05-22T21:19:13.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.971+0000 7ff8f8800640 1 --2- 172.21.15.161:0/2758972146 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8d4066690 0x7ff8d4068b50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:13.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.971+0000 7ff8f8800640 1 --2- 172.21.15.161:0/2758972146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8f015af60 0x7ff8f013bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:13.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.971+0000 7ff8f8800640 1 --2- 172.21.15.161:0/2758972146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8f013fe50 0x7ff8f00b2780 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:13.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.971+0000 7ff8f8800640 1 --2- 172.21.15.161:0/2758972146 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8f013cfc0 0x7ff8f013ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:13.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.971+0000 7ff8f8800640 1 -- 172.21.15.161:0/2758972146 >> 172.21.15.161:0/2758972146 conn(0x7ff8f006a2d0 msgr2=0x7ff8f00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:13.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.975+0000 7ff8f8800640 1 -- 172.21.15.161:0/2758972146 shutdown_connections 2024-05-22T21:19:13.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:13.975+0000 7ff8f8800640 1 -- 172.21.15.161:0/2758972146 wait complete. 2024-05-22T21:19:13.998 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-22T21:19:14.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.135+0000 7fbbed000640 1 Processor -- start 2024-05-22T21:19:14.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.135+0000 7fbbed000640 1 -- start start 2024-05-22T21:19:14.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.135+0000 7fbbed000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbbe813cfc0 0x7fbbe81402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.135+0000 7fbbed000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbbe815af60 0x7fbbe813f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.135+0000 7fbbed000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe81400d0 con 0x7fbbe813fc80 2024-05-22T21:19:14.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.135+0000 7fbbed000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe813d3a0 con 0x7fbbe813cfc0 2024-05-22T21:19:14.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.135+0000 7fbbed000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80cd920 con 0x7fbbe815af60 2024-05-22T21:19:14.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.135+0000 7fbbe5e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbbe813cfc0 0x7fbbe81402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.135+0000 7fbbe5e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbbe813cfc0 0x7fbbe81402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49166/0 (socket says 172.21.15.161:49166) 2024-05-22T21:19:14.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.135+0000 7fbbe5e00640 1 -- 172.21.15.161:0/1188429945 learned_addr learned my addr 172.21.15.161:0/1188429945 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:14.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.135+0000 7fbbe5400640 1 --2- 172.21.15.161:0/1188429945 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbbe815af60 0x7fbbe813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.135+0000 7fbbe5e00640 1 -- 172.21.15.161:0/1188429945 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbbe815af60 msgr2=0x7fbbe813f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:14.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.135+0000 7fbbe5e00640 1 --2- 172.21.15.161:0/1188429945 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbbe815af60 0x7fbbe813f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.135+0000 7fbbe5e00640 1 -- 172.21.15.161:0/1188429945 >> v1:172.21.15.179:6789/0 conn(0x7fbbe813fc80 legacy=0x7fbbe815dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:14.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.135+0000 7fbbe5e00640 1 -- 172.21.15.161:0/1188429945 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe80cdc00 con 0x7fbbe813cfc0 2024-05-22T21:19:14.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.139+0000 7fbbe5e00640 1 --2- 172.21.15.161:0/1188429945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe813cfc0 0x7fbbe81402e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009870 tx=0x7fbbd801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb376f381c25f59 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:14.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.139+0000 7fbbe4a00640 1 -- 172.21.15.161:0/1188429945 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80ab600 con 0x7fbbe813cfc0 2024-05-22T21:19:14.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.139+0000 7fbbe4a00640 1 -- 172.21.15.161:0/1188429945 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd80a7050 con 0x7fbbe813cfc0 2024-05-22T21:19:14.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.139+0000 7fbbe4a00640 1 -- 172.21.15.161:0/1188429945 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80ab980 con 0x7fbbe813cfc0 2024-05-22T21:19:14.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.139+0000 7fbbed000640 1 -- 172.21.15.161:0/1188429945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe813cfc0 msgr2=0x7fbbe81402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:14.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.139+0000 7fbbed000640 1 --2- 172.21.15.161:0/1188429945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe813cfc0 0x7fbbe81402e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009870 tx=0x7fbbd801c4d0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.139+0000 7fbbed000640 1 -- 172.21.15.161:0/1188429945 shutdown_connections 2024-05-22T21:19:14.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.139+0000 7fbbed000640 1 --2- 172.21.15.161:0/1188429945 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbbe815af60 0x7fbbe813f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.139+0000 7fbbed000640 1 --2- 172.21.15.161:0/1188429945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe813cfc0 0x7fbbe81402e0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.139+0000 7fbbed000640 1 -- 172.21.15.161:0/1188429945 >> 172.21.15.161:0/1188429945 conn(0x7fbbe806a2d0 msgr2=0x7fbbe806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:14.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.139+0000 7fbbed000640 1 -- 172.21.15.161:0/1188429945 shutdown_connections 2024-05-22T21:19:14.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.139+0000 7fbbed000640 1 -- 172.21.15.161:0/1188429945 wait complete. 2024-05-22T21:19:14.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbed000640 1 Processor -- start 2024-05-22T21:19:14.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbed000640 1 -- start start 2024-05-22T21:19:14.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbed000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbbe813cfc0 0x7fbbe8157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbed000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbe813fc80 0x7fbbe813a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbed000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe815af60 0x7fbbe8141f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbed000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8140c50 con 0x7fbbe815af60 2024-05-22T21:19:14.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbed000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8140e80 con 0x7fbbe813fc80 2024-05-22T21:19:14.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbed000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8141160 con 0x7fbbe813cfc0 2024-05-22T21:19:14.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbe6800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe815af60 0x7fbbe8141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbe6800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe815af60 0x7fbbe8141f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49174/0 (socket says 172.21.15.161:49174) 2024-05-22T21:19:14.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbe6800640 1 -- 172.21.15.161:0/3645359330 learned_addr learned my addr 172.21.15.161:0/3645359330 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:14.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbe5e00640 1 --2- 172.21.15.161:0/3645359330 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbbe813cfc0 0x7fbbe8157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbe5400640 1 --2- 172.21.15.161:0/3645359330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbe813fc80 0x7fbbe813a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbe6800640 1 -- 172.21.15.161:0/3645359330 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbbe813cfc0 msgr2=0x7fbbe8157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:14.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbe6800640 1 --2- 172.21.15.161:0/3645359330 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbbe813cfc0 0x7fbbe8157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbe6800640 1 -- 172.21.15.161:0/3645359330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbe813fc80 msgr2=0x7fbbe813a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:14.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbe6800640 1 --2- 172.21.15.161:0/3645359330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbe813fc80 0x7fbbe813a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbe6800640 1 -- 172.21.15.161:0/3645359330 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd80094d0 con 0x7fbbe815af60 2024-05-22T21:19:14.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbe5400640 1 --2- 172.21.15.161:0/3645359330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbe813fc80 0x7fbbe813a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:14.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbe6800640 1 --2- 172.21.15.161:0/3645359330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe815af60 0x7fbbe8141f20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000c770 tx=0x7fbbe000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:14.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbde000640 1 -- 172.21.15.161:0/3645359330 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0003b80 con 0x7fbbe815af60 2024-05-22T21:19:14.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbde000640 1 -- 172.21.15.161:0/3645359330 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe0003ce0 con 0x7fbbe815af60 2024-05-22T21:19:14.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbed000640 1 -- 172.21.15.161:0/3645359330 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe8141440 con 0x7fbbe815af60 2024-05-22T21:19:14.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbde000640 1 -- 172.21.15.161:0/3645359330 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe000aa90 con 0x7fbbe815af60 2024-05-22T21:19:14.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.143+0000 7fbbed000640 1 -- 172.21.15.161:0/3645359330 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe8141a00 con 0x7fbbe815af60 2024-05-22T21:19:14.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.147+0000 7fbbed000640 1 -- 172.21.15.161:0/3645359330 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe8141c20 con 0x7fbbe815af60 2024-05-22T21:19:14.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.147+0000 7fbbde000640 1 -- 172.21.15.161:0/3645359330 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbbe000abf0 con 0x7fbbe815af60 2024-05-22T21:19:14.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.147+0000 7fbbde000640 1 --2- 172.21.15.161:0/3645359330 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbbc8066690 0x7fbbc8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.147+0000 7fbbde000640 1 -- 172.21.15.161:0/3645359330 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fbbe0083ec0 con 0x7fbbe815af60 2024-05-22T21:19:14.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.147+0000 7fbbe5e00640 1 --2- 172.21.15.161:0/3645359330 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbbc8066690 0x7fbbc8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.147+0000 7fbbe5e00640 1 --2- 172.21.15.161:0/3645359330 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbbc8066690 0x7fbbc8068b50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80ad000 tx=0x7fbbd80a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:14.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.151+0000 7fbbde000640 1 -- 172.21.15.161:0/3645359330 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbbe00511f0 con 0x7fbbe815af60 2024-05-22T21:19:14.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.339+0000 7fbbed000640 1 -- 172.21.15.161:0/3645359330 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbbe81400d0 con 0x7fbbe815af60 2024-05-22T21:19:14.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.351+0000 7fbbde000640 1 -- 172.21.15.161:0/3645359330 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v51) v1 ==== 174+0+0 (secure 0 0 0) 0x7fbbe81400d0 con 0x7fbbe815af60 2024-05-22T21:19:14.359 INFO:teuthology.orchestra.run.smithi161.stderr:new fs with metadata pool 14 and data pool 15 2024-05-22T21:19:14.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.355+0000 7fbbed000640 1 -- 172.21.15.161:0/3645359330 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbbc8066690 msgr2=0x7fbbc8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:14.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.355+0000 7fbbed000640 1 --2- 172.21.15.161:0/3645359330 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbbc8066690 0x7fbbc8068b50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80ad000 tx=0x7fbbd80a6000 comp rx=0 tx=0).stop 2024-05-22T21:19:14.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.355+0000 7fbbed000640 1 -- 172.21.15.161:0/3645359330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe815af60 msgr2=0x7fbbe8141f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:14.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.355+0000 7fbbed000640 1 --2- 172.21.15.161:0/3645359330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe815af60 0x7fbbe8141f20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000c770 tx=0x7fbbe000cc30 comp rx=0 tx=0).stop 2024-05-22T21:19:14.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.359+0000 7fbbed000640 1 -- 172.21.15.161:0/3645359330 shutdown_connections 2024-05-22T21:19:14.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.359+0000 7fbbed000640 1 --2- 172.21.15.161:0/3645359330 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbbc8066690 0x7fbbc8068b50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.359+0000 7fbbed000640 1 --2- 172.21.15.161:0/3645359330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe815af60 0x7fbbe8141f20 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.359+0000 7fbbed000640 1 --2- 172.21.15.161:0/3645359330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbe813fc80 0x7fbbe813a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.359+0000 7fbbed000640 1 --2- 172.21.15.161:0/3645359330 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbbe813cfc0 0x7fbbe8157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.359+0000 7fbbed000640 1 -- 172.21.15.161:0/3645359330 >> 172.21.15.161:0/3645359330 conn(0x7fbbe806a2d0 msgr2=0x7fbbe80ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:14.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.359+0000 7fbbed000640 1 -- 172.21.15.161:0/3645359330 shutdown_connections 2024-05-22T21:19:14.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.359+0000 7fbbed000640 1 -- 172.21.15.161:0/3645359330 wait complete. 2024-05-22T21:19:14.389 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:14.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff924000640 1 Processor -- start 2024-05-22T21:19:14.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff924000640 1 -- start start 2024-05-22T21:19:14.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff924000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff91c15e130 0x7ff91c15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff924000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff91c15d410 0x7ff91c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff924000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff91c071670 con 0x7ff91c15dd50 2024-05-22T21:19:14.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff924000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91c15d870 con 0x7ff91c15d410 2024-05-22T21:19:14.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff924000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c070a60 con 0x7ff91c15e130 2024-05-22T21:19:14.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff922000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff91c15d410 0x7ff91c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff921600640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff91c15dd50 0x7ff91c06d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41554/0 (socket says 172.21.15.161:41554) 2024-05-22T21:19:14.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff921600640 1 -- 172.21.15.161:0/1383206288 learned_addr learned my addr 172.21.15.161:0/1383206288 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:14.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff920c00640 1 --2- 172.21.15.161:0/1383206288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff91c15e130 0x7ff91c15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff922000640 1 -- 172.21.15.161:0/1383206288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff91c15e130 msgr2=0x7ff91c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:14.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff922000640 1 --2- 172.21.15.161:0/1383206288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff91c15e130 0x7ff91c15ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff922000640 1 -- 172.21.15.161:0/1383206288 >> v1:172.21.15.161:6789/0 conn(0x7ff91c15dd50 legacy=0x7ff91c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:14.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff922000640 1 -- 172.21.15.161:0/1383206288 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c070d40 con 0x7ff91c15d410 2024-05-22T21:19:14.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff91be00640 1 -- 172.21.15.161:0/1383206288 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375657438 0 0) 0x7ff91c071670 con 0x7ff91c15dd50 2024-05-22T21:19:14.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.523+0000 7ff920c00640 1 --2- 172.21.15.161:0/1383206288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff91c15e130 0x7ff91c15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:14.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.527+0000 7ff922000640 1 --2- 172.21.15.161:0/1383206288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff91c15d410 0x7ff91c15b6c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ff9140097c0 tx=0x7ff91401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd089dd34d440b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:14.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.527+0000 7ff91be00640 1 -- 172.21.15.161:0/1383206288 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9140ac640 con 0x7ff91c15d410 2024-05-22T21:19:14.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.527+0000 7ff91be00640 1 -- 172.21.15.161:0/1383206288 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9140ac7a0 con 0x7ff91c15d410 2024-05-22T21:19:14.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.527+0000 7ff91be00640 1 -- 172.21.15.161:0/1383206288 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9140acb30 con 0x7ff91c15d410 2024-05-22T21:19:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.527+0000 7ff924000640 1 -- 172.21.15.161:0/1383206288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff91c15d410 msgr2=0x7ff91c15b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.527+0000 7ff924000640 1 --2- 172.21.15.161:0/1383206288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff91c15d410 0x7ff91c15b6c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ff9140097c0 tx=0x7ff91401c3e0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.527+0000 7ff924000640 1 -- 172.21.15.161:0/1383206288 shutdown_connections 2024-05-22T21:19:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.527+0000 7ff924000640 1 --2- 172.21.15.161:0/1383206288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff91c15d410 0x7ff91c15b6c0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.527+0000 7ff924000640 1 --2- 172.21.15.161:0/1383206288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff91c15e130 0x7ff91c15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.527+0000 7ff924000640 1 -- 172.21.15.161:0/1383206288 >> 172.21.15.161:0/1383206288 conn(0x7ff91c0b0bc0 msgr2=0x7ff91c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.527+0000 7ff924000640 1 -- 172.21.15.161:0/1383206288 shutdown_connections 2024-05-22T21:19:14.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.527+0000 7ff924000640 1 -- 172.21.15.161:0/1383206288 wait complete. 2024-05-22T21:19:14.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff924000640 1 Processor -- start 2024-05-22T21:19:14.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff924000640 1 -- start start 2024-05-22T21:19:14.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff924000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91c15d410 0x7ff91c1592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff924000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff91c15dd50 0x7ff91c1516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff924000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff91c15e130 0x7ff91c14d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff924000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c151c20 con 0x7ff91c15dd50 2024-05-22T21:19:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff924000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c151e50 con 0x7ff91c15d410 2024-05-22T21:19:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff924000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c152130 con 0x7ff91c15e130 2024-05-22T21:19:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff920c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff91c15dd50 0x7ff91c1516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff920c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff91c15dd50 0x7ff91c1516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49200/0 (socket says 172.21.15.161:49200) 2024-05-22T21:19:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff920c00640 1 -- 172.21.15.161:0/704291774 learned_addr learned my addr 172.21.15.161:0/704291774 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff922000640 1 --2- 172.21.15.161:0/704291774 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff91c15e130 0x7ff91c14d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff921600640 1 --2- 172.21.15.161:0/704291774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91c15d410 0x7ff91c1592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff920c00640 1 -- 172.21.15.161:0/704291774 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff91c15e130 msgr2=0x7ff91c14d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:14.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff920c00640 1 --2- 172.21.15.161:0/704291774 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff91c15e130 0x7ff91c14d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff920c00640 1 -- 172.21.15.161:0/704291774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91c15d410 msgr2=0x7ff91c1592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff920c00640 1 --2- 172.21.15.161:0/704291774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91c15d410 0x7ff91c1592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff920c00640 1 -- 172.21.15.161:0/704291774 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff914009420 con 0x7ff91c15dd50 2024-05-22T21:19:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff921600640 1 --2- 172.21.15.161:0/704291774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91c15d410 0x7ff91c1592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff920c00640 1 --2- 172.21.15.161:0/704291774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff91c15dd50 0x7ff91c1516e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff90800d800 tx=0x7ff90800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff919600640 1 -- 172.21.15.161:0/704291774 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90800b6c0 con 0x7ff91c15dd50 2024-05-22T21:19:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff919600640 1 -- 172.21.15.161:0/704291774 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90800b820 con 0x7ff91c15dd50 2024-05-22T21:19:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff924000640 1 -- 172.21.15.161:0/704291774 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91c152410 con 0x7ff91c15dd50 2024-05-22T21:19:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.531+0000 7ff919600640 1 -- 172.21.15.161:0/704291774 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90800bbd0 con 0x7ff91c15dd50 2024-05-22T21:19:14.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.535+0000 7ff924000640 1 -- 172.21.15.161:0/704291774 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91c1526f0 con 0x7ff91c15dd50 2024-05-22T21:19:14.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.535+0000 7ff924000640 1 -- 172.21.15.161:0/704291774 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91c0b24e0 con 0x7ff91c15dd50 2024-05-22T21:19:14.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.535+0000 7ff919600640 1 -- 172.21.15.161:0/704291774 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff90800bd30 con 0x7ff91c15dd50 2024-05-22T21:19:14.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.535+0000 7ff919600640 1 --2- 172.21.15.161:0/704291774 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8fc066690 0x7ff8fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.535+0000 7ff919600640 1 -- 172.21.15.161:0/704291774 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff908084600 con 0x7ff91c15dd50 2024-05-22T21:19:14.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.535+0000 7ff921600640 1 --2- 172.21.15.161:0/704291774 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8fc066690 0x7ff8fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.535+0000 7ff921600640 1 --2- 172.21.15.161:0/704291774 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8fc066690 0x7ff8fc068b50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff910007fd0 tx=0x7ff910009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:14.542 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.539+0000 7ff919600640 1 -- 172.21.15.161:0/704291774 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff91c0b24e0 con 0x7ff91c15dd50 2024-05-22T21:19:14.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.695+0000 7ff924000640 1 -- 172.21.15.161:0/704291774 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff91c0b24e0 con 0x7ff91c15dd50 2024-05-22T21:19:14.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.695+0000 7ff919600640 1 -- 172.21.15.161:0/704291774 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16113 (secure 0 0 0) 0x7ff91c0b24e0 con 0x7ff91c15dd50 2024-05-22T21:19:14.699 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:14.699 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":124,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:19:14.345593+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:12.848545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:13.344629+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:14.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.699+0000 7ff924000640 1 -- 172.21.15.161:0/704291774 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8fc066690 msgr2=0x7ff8fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:14.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.699+0000 7ff924000640 1 --2- 172.21.15.161:0/704291774 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8fc066690 0x7ff8fc068b50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff910007fd0 tx=0x7ff910009040 comp rx=0 tx=0).stop 2024-05-22T21:19:14.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.699+0000 7ff924000640 1 -- 172.21.15.161:0/704291774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff91c15dd50 msgr2=0x7ff91c1516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:14.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.699+0000 7ff924000640 1 --2- 172.21.15.161:0/704291774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff91c15dd50 0x7ff91c1516e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff90800d800 tx=0x7ff90800dcc0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.699+0000 7ff924000640 1 -- 172.21.15.161:0/704291774 shutdown_connections 2024-05-22T21:19:14.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.699+0000 7ff924000640 1 --2- 172.21.15.161:0/704291774 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff8fc066690 0x7ff8fc068b50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.699+0000 7ff924000640 1 --2- 172.21.15.161:0/704291774 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff91c15e130 0x7ff91c14d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.699+0000 7ff924000640 1 --2- 172.21.15.161:0/704291774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff91c15dd50 0x7ff91c1516e0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.699+0000 7ff924000640 1 --2- 172.21.15.161:0/704291774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff91c15d410 0x7ff91c1592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.699+0000 7ff924000640 1 -- 172.21.15.161:0/704291774 >> 172.21.15.161:0/704291774 conn(0x7ff91c0b0bc0 msgr2=0x7ff91c0adc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:14.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.699+0000 7ff924000640 1 -- 172.21.15.161:0/704291774 shutdown_connections 2024-05-22T21:19:14.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.699+0000 7ff924000640 1 -- 172.21.15.161:0/704291774 wait complete. 2024-05-22T21:19:14.720 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:14.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.851+0000 7fdbe3000640 1 Processor -- start 2024-05-22T21:19:14.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.855+0000 7fdbe3000640 1 -- start start 2024-05-22T21:19:14.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.855+0000 7fdbe3000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdbdc0d0a40 0x7fdbdc0d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.855+0000 7fdbe3000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbdc0cc320 0x7fdbdc0cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.855+0000 7fdbe3000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbdc0cb8a0 con 0x7fdbdc15af00 2024-05-22T21:19:14.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.855+0000 7fdbe3000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbdc0d13f0 con 0x7fdbdc0d0a40 2024-05-22T21:19:14.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.855+0000 7fdbe3000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc15f530 con 0x7fdbdc0cc320 2024-05-22T21:19:14.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.855+0000 7fdbe1200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdbdc0d0a40 0x7fdbdc0d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.855+0000 7fdbe1c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fdbdc15af00 0x7fdbdc15eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41564/0 (socket says 172.21.15.161:41564) 2024-05-22T21:19:14.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.855+0000 7fdbe1c00640 1 -- 172.21.15.161:0/3154706668 learned_addr learned my addr 172.21.15.161:0/3154706668 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:14.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.855+0000 7fdbe2600640 1 --2- 172.21.15.161:0/3154706668 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbdc0cc320 0x7fdbdc0cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.855+0000 7fdbe1200640 1 -- 172.21.15.161:0/3154706668 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbdc0cc320 msgr2=0x7fdbdc0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:14.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.855+0000 7fdbe1200640 1 --2- 172.21.15.161:0/3154706668 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbdc0cc320 0x7fdbdc0cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.855+0000 7fdbe1200640 1 -- 172.21.15.161:0/3154706668 >> v1:172.21.15.161:6789/0 conn(0x7fdbdc15af00 legacy=0x7fdbdc15eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:14.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.855+0000 7fdbe1200640 1 -- 172.21.15.161:0/3154706668 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc15f810 con 0x7fdbdc0d0a40 2024-05-22T21:19:14.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.855+0000 7fdbdbe00640 1 -- 172.21.15.161:0/3154706668 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376867573 0 0) 0x7fdbdc0cb8a0 con 0x7fdbdc15af00 2024-05-22T21:19:14.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.859+0000 7fdbe1200640 1 --2- 172.21.15.161:0/3154706668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbdc0d0a40 0x7fdbdc0d0eb0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80097c0 tx=0x7fdbc801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733af43efa0895a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:14.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.859+0000 7fdbdbe00640 1 -- 172.21.15.161:0/3154706668 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80ab660 con 0x7fdbdc0d0a40 2024-05-22T21:19:14.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.859+0000 7fdbdbe00640 1 -- 172.21.15.161:0/3154706668 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc80ab7c0 con 0x7fdbdc0d0a40 2024-05-22T21:19:14.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.859+0000 7fdbdbe00640 1 -- 172.21.15.161:0/3154706668 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80abb50 con 0x7fdbdc0d0a40 2024-05-22T21:19:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.859+0000 7fdbe3000640 1 -- 172.21.15.161:0/3154706668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbdc0d0a40 msgr2=0x7fdbdc0d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.859+0000 7fdbe3000640 1 --2- 172.21.15.161:0/3154706668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbdc0d0a40 0x7fdbdc0d0eb0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80097c0 tx=0x7fdbc801c460 comp rx=0 tx=0).stop 2024-05-22T21:19:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.859+0000 7fdbe3000640 1 -- 172.21.15.161:0/3154706668 shutdown_connections 2024-05-22T21:19:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.859+0000 7fdbe3000640 1 --2- 172.21.15.161:0/3154706668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbdc0d0a40 0x7fdbdc0d0eb0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.859+0000 7fdbe3000640 1 --2- 172.21.15.161:0/3154706668 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbdc0cc320 0x7fdbdc0cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.859+0000 7fdbe3000640 1 -- 172.21.15.161:0/3154706668 >> 172.21.15.161:0/3154706668 conn(0x7fdbdc099a40 msgr2=0x7fdbdc099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.859+0000 7fdbe3000640 1 -- 172.21.15.161:0/3154706668 shutdown_connections 2024-05-22T21:19:14.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.859+0000 7fdbe3000640 1 -- 172.21.15.161:0/3154706668 wait complete. 2024-05-22T21:19:14.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.859+0000 7fdbe3000640 1 Processor -- start 2024-05-22T21:19:14.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.859+0000 7fdbe3000640 1 -- start start 2024-05-22T21:19:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe3000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbdc0cc320 0x7fdbdc0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe3000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbdc0d0a40 0x7fdbdc0c4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe3000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbdc15af00 0x7fdbdc154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe3000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc0cab40 con 0x7fdbdc15af00 2024-05-22T21:19:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe3000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc0b5e10 con 0x7fdbdc0d0a40 2024-05-22T21:19:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe3000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc0b60f0 con 0x7fdbdc0cc320 2024-05-22T21:19:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe2600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbdc15af00 0x7fdbdc154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe2600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbdc15af00 0x7fdbdc154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49220/0 (socket says 172.21.15.161:49220) 2024-05-22T21:19:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe2600640 1 -- 172.21.15.161:0/1656413816 learned_addr learned my addr 172.21.15.161:0/1656413816 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe1c00640 1 --2- 172.21.15.161:0/1656413816 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbdc0cc320 0x7fdbdc0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe1200640 1 --2- 172.21.15.161:0/1656413816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbdc0d0a40 0x7fdbdc0c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe2600640 1 -- 172.21.15.161:0/1656413816 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbdc0cc320 msgr2=0x7fdbdc0b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe2600640 1 --2- 172.21.15.161:0/1656413816 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbdc0cc320 0x7fdbdc0b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe2600640 1 -- 172.21.15.161:0/1656413816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbdc0d0a40 msgr2=0x7fdbdc0c4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe2600640 1 --2- 172.21.15.161:0/1656413816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbdc0d0a40 0x7fdbdc0c4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe2600640 1 -- 172.21.15.161:0/1656413816 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc8009420 con 0x7fdbdc15af00 2024-05-22T21:19:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe1200640 1 --2- 172.21.15.161:0/1656413816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbdc0d0a40 0x7fdbdc0c4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe2600640 1 --2- 172.21.15.161:0/1656413816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbdc15af00 0x7fdbdc154ca0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4007e60 tx=0x7fdbd400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbd9600640 1 -- 172.21.15.161:0/1656413816 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4014420 con 0x7fdbdc15af00 2024-05-22T21:19:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbd9600640 1 -- 172.21.15.161:0/1656413816 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd40145a0 con 0x7fdbdc15af00 2024-05-22T21:19:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbd9600640 1 -- 172.21.15.161:0/1656413816 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4014950 con 0x7fdbdc15af00 2024-05-22T21:19:14.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe3000640 1 -- 172.21.15.161:0/1656413816 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbdc0b63d0 con 0x7fdbdc15af00 2024-05-22T21:19:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe3000640 1 -- 172.21.15.161:0/1656413816 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbdc0c4540 con 0x7fdbdc15af00 2024-05-22T21:19:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbe3000640 1 -- 172.21.15.161:0/1656413816 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbdc0b65c0 con 0x7fdbdc15af00 2024-05-22T21:19:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.863+0000 7fdbd9600640 1 -- 172.21.15.161:0/1656413816 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdbd4014ab0 con 0x7fdbdc15af00 2024-05-22T21:19:14.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.867+0000 7fdbd9600640 1 --2- 172.21.15.161:0/1656413816 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdbbc066690 0x7fdbbc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:14.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.867+0000 7fdbd9600640 1 -- 172.21.15.161:0/1656413816 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbd4084b00 con 0x7fdbdc15af00 2024-05-22T21:19:14.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.867+0000 7fdbe1c00640 1 --2- 172.21.15.161:0/1656413816 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdbbc066690 0x7fdbbc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:14.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.867+0000 7fdbe1c00640 1 --2- 172.21.15.161:0/1656413816 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdbbc066690 0x7fdbbc068b50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0000f70 tx=0x7fdbd0007b60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:14.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:14.871+0000 7fdbd9600640 1 -- 172.21.15.161:0/1656413816 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdbd4086e60 con 0x7fdbdc15af00 2024-05-22T21:19:15.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.027+0000 7fdbe3000640 1 -- 172.21.15.161:0/1656413816 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbdc067070 con 0x7fdbdc15af00 2024-05-22T21:19:15.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.027+0000 7fdbd9600640 1 -- 172.21.15.161:0/1656413816 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16113 (secure 0 0 0) 0x7fdbd40558f0 con 0x7fdbdc15af00 2024-05-22T21:19:15.030 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:15.030 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":124,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:19:14.345593+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:12.848545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:13.344629+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:15.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.027+0000 7fdbe3000640 1 -- 172.21.15.161:0/1656413816 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdbbc066690 msgr2=0x7fdbbc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:15.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.027+0000 7fdbe3000640 1 --2- 172.21.15.161:0/1656413816 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdbbc066690 0x7fdbbc068b50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0000f70 tx=0x7fdbd0007b60 comp rx=0 tx=0).stop 2024-05-22T21:19:15.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.027+0000 7fdbe3000640 1 -- 172.21.15.161:0/1656413816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbdc15af00 msgr2=0x7fdbdc154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:15.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.027+0000 7fdbe3000640 1 --2- 172.21.15.161:0/1656413816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbdc15af00 0x7fdbdc154ca0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4007e60 tx=0x7fdbd400e4b0 comp rx=0 tx=0).stop 2024-05-22T21:19:15.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.031+0000 7fdbe3000640 1 -- 172.21.15.161:0/1656413816 shutdown_connections 2024-05-22T21:19:15.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.031+0000 7fdbe3000640 1 --2- 172.21.15.161:0/1656413816 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdbbc066690 0x7fdbbc068b50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:15.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.031+0000 7fdbe3000640 1 --2- 172.21.15.161:0/1656413816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbdc15af00 0x7fdbdc154ca0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:15.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.031+0000 7fdbe3000640 1 --2- 172.21.15.161:0/1656413816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbdc0d0a40 0x7fdbdc0c4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:15.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.031+0000 7fdbe3000640 1 --2- 172.21.15.161:0/1656413816 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdbdc0cc320 0x7fdbdc0b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:15.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.031+0000 7fdbe3000640 1 -- 172.21.15.161:0/1656413816 >> 172.21.15.161:0/1656413816 conn(0x7fdbdc099a40 msgr2=0x7fdbdc1576d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:15.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.031+0000 7fdbe3000640 1 -- 172.21.15.161:0/1656413816 shutdown_connections 2024-05-22T21:19:15.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.031+0000 7fdbe3000640 1 -- 172.21.15.161:0/1656413816 wait complete. 2024-05-22T21:19:15.052 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-22T21:19:15.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.187+0000 7f1ba1200640 1 Processor -- start 2024-05-22T21:19:15.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.187+0000 7f1ba1200640 1 -- start start 2024-05-22T21:19:15.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.187+0000 7f1ba1200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1b9c15b600 0x7f1b9c06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:15.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.187+0000 7f1ba1200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9c06cc40 con 0x7f1b9c06c7e0 2024-05-22T21:19:15.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.187+0000 7f1ba1200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9c144bb0 con 0x7f1b9c143c40 2024-05-22T21:19:15.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.187+0000 7f1ba1200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b9c15ba60 con 0x7f1b9c15b600 2024-05-22T21:19:15.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.187+0000 7f1b9a800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1b9c06c7e0 0x7f1b9c0b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41574/0 (socket says 172.21.15.161:41574) 2024-05-22T21:19:15.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.187+0000 7f1b9a800640 1 -- 172.21.15.161:0/640651273 learned_addr learned my addr 172.21.15.161:0/640651273 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:15.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.187+0000 7f1b99400640 1 --2- 172.21.15.161:0/640651273 >> v2:172.21.15.179:3300/0 conn(0x7f1b9c15b600 0x7f1b9c06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:15.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.191+0000 7f1b98a00640 1 -- 172.21.15.161:0/640651273 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783018455 0 0) 0x7f1b9c06cc40 con 0x7f1b9c06c7e0 2024-05-22T21:19:15.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.191+0000 7f1b98a00640 1 -- 172.21.15.161:0/640651273 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b80003530 con 0x7f1b9c06c7e0 2024-05-22T21:19:15.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.191+0000 7f1b99400640 1 -- 172.21.15.161:0/640651273 >> v1:172.21.15.179:6789/0 conn(0x7f1b9c143c40 legacy=0x7f1b9c1444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:15.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.191+0000 7f1b98a00640 1 -- 172.21.15.161:0/640651273 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642680280 0 0) 0x7f1b9c144bb0 con 0x7f1b9c143c40 2024-05-22T21:19:15.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.191+0000 7f1b99400640 1 -- 172.21.15.161:0/640651273 >> v1:172.21.15.161:6789/0 conn(0x7f1b9c06c7e0 legacy=0x7f1b9c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:15.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.191+0000 7f1b99400640 1 -- 172.21.15.161:0/640651273 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c15efb0 con 0x7f1b9c15b600 2024-05-22T21:19:15.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.191+0000 7f1b99400640 1 --2- 172.21.15.161:0/640651273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b9c15b600 0x7f1b9c06c2a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1b880097c0 tx=0x7f1b8801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1c7b70a1667ca70 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:15.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.191+0000 7f1b98a00640 1 -- 172.21.15.161:0/640651273 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b880ab600 con 0x7f1b9c15b600 2024-05-22T21:19:15.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.191+0000 7f1b98a00640 1 -- 172.21.15.161:0/640651273 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b880a7050 con 0x7f1b9c15b600 2024-05-22T21:19:15.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.191+0000 7f1b98a00640 1 -- 172.21.15.161:0/640651273 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b880ab960 con 0x7f1b9c15b600 2024-05-22T21:19:15.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1ba1200640 1 -- 172.21.15.161:0/640651273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b9c15b600 msgr2=0x7f1b9c06c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:15.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1ba1200640 1 --2- 172.21.15.161:0/640651273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b9c15b600 0x7f1b9c06c2a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1b880097c0 tx=0x7f1b8801c3c0 comp rx=0 tx=0).stop 2024-05-22T21:19:15.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1ba1200640 1 -- 172.21.15.161:0/640651273 shutdown_connections 2024-05-22T21:19:15.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1ba1200640 1 --2- 172.21.15.161:0/640651273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b9c15b600 0x7f1b9c06c2a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:15.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1ba1200640 1 -- 172.21.15.161:0/640651273 >> 172.21.15.161:0/640651273 conn(0x7f1b9c06a2d0 msgr2=0x7f1b9c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:15.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1ba1200640 1 -- 172.21.15.161:0/640651273 shutdown_connections 2024-05-22T21:19:15.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1ba1200640 1 -- 172.21.15.161:0/640651273 wait complete. 2024-05-22T21:19:15.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1ba1200640 1 Processor -- start 2024-05-22T21:19:15.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1ba1200640 1 -- start start 2024-05-22T21:19:15.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1ba1200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b9c06c7e0 0x7f1b9c15aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:15.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1ba1200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1b9c143c40 0x7f1b9c137500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:15.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1ba1200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b9c15b600 0x7f1b9c1361d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:15.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1ba1200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c15b410 con 0x7f1b9c06c7e0 2024-05-22T21:19:15.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1ba1200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c13a160 con 0x7f1b9c15b600 2024-05-22T21:19:15.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1ba1200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c139370 con 0x7f1b9c143c40 2024-05-22T21:19:15.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1b99e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b9c06c7e0 0x7f1b9c15aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:15.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1b99e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b9c06c7e0 0x7f1b9c15aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49234/0 (socket says 172.21.15.161:49234) 2024-05-22T21:19:15.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1b99e00640 1 -- 172.21.15.161:0/1657420536 learned_addr learned my addr 172.21.15.161:0/1657420536 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:15.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1b99400640 1 --2- 172.21.15.161:0/1657420536 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1b9c143c40 0x7f1b9c137500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:15.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1b9a800640 1 --2- 172.21.15.161:0/1657420536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b9c15b600 0x7f1b9c1361d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:15.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1b99e00640 1 -- 172.21.15.161:0/1657420536 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1b9c143c40 msgr2=0x7f1b9c137500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:15.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1b99e00640 1 --2- 172.21.15.161:0/1657420536 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1b9c143c40 0x7f1b9c137500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:15.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1b99e00640 1 -- 172.21.15.161:0/1657420536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b9c15b600 msgr2=0x7f1b9c1361d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:15.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1b99e00640 1 --2- 172.21.15.161:0/1657420536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b9c15b600 0x7f1b9c1361d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:15.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.195+0000 7f1b99e00640 1 -- 172.21.15.161:0/1657420536 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b88009420 con 0x7f1b9c06c7e0 2024-05-22T21:19:15.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.199+0000 7f1b99e00640 1 --2- 172.21.15.161:0/1657420536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b9c06c7e0 0x7f1b9c15aed0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00c4b0 tx=0x7f1b8c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:15.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.199+0000 7f1b92000640 1 -- 172.21.15.161:0/1657420536 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c005480 con 0x7f1b9c06c7e0 2024-05-22T21:19:15.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.199+0000 7f1b92000640 1 -- 172.21.15.161:0/1657420536 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c003600 con 0x7f1b9c06c7e0 2024-05-22T21:19:15.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.199+0000 7f1b92000640 1 -- 172.21.15.161:0/1657420536 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c0039b0 con 0x7f1b9c06c7e0 2024-05-22T21:19:15.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.199+0000 7f1ba1200640 1 -- 172.21.15.161:0/1657420536 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9c1395e0 con 0x7f1b9c06c7e0 2024-05-22T21:19:15.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.199+0000 7f1ba1200640 1 -- 172.21.15.161:0/1657420536 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9c139ba0 con 0x7f1b9c06c7e0 2024-05-22T21:19:15.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.199+0000 7f1ba1200640 1 -- 172.21.15.161:0/1657420536 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9c067220 con 0x7f1b9c06c7e0 2024-05-22T21:19:15.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.199+0000 7f1b92000640 1 -- 172.21.15.161:0/1657420536 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1b8c003b10 con 0x7f1b9c06c7e0 2024-05-22T21:19:15.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.199+0000 7f1b92000640 1 --2- 172.21.15.161:0/1657420536 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1b7c0666c0 0x7f1b7c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:15.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.199+0000 7f1b92000640 1 -- 172.21.15.161:0/1657420536 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b8c0846b0 con 0x7f1b9c06c7e0 2024-05-22T21:19:15.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.203+0000 7f1b99400640 1 --2- 172.21.15.161:0/1657420536 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1b7c0666c0 0x7f1b7c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:15.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.203+0000 7f1b99400640 1 --2- 172.21.15.161:0/1657420536 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1b7c0666c0 0x7f1b7c068b80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0aead0 tx=0x7f1b880a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:15.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.203+0000 7f1b92000640 1 -- 172.21.15.161:0/1657420536 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1b8c051ab0 con 0x7f1b9c06c7e0 2024-05-22T21:19:15.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:15.399+0000 7f1ba1200640 1 -- 172.21.15.161:0/1657420536 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1b9c0ceaa0 con 0x7f1b9c06c7e0 2024-05-22T21:19:15.953 INFO:tasks.ceph.mds.a.smithi161.stderr:2024-05-22T21:19:15.951+0000 7f5bd8600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T21:19:16.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.351+0000 7f1b92000640 1 -- 172.21.15.161:0/1657420536 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v53) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1b8c0550d0 con 0x7f1b9c06c7e0 2024-05-22T21:19:16.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.355+0000 7f1ba1200640 1 -- 172.21.15.161:0/1657420536 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1b7c0666c0 msgr2=0x7f1b7c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:16.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.355+0000 7f1ba1200640 1 --2- 172.21.15.161:0/1657420536 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1b7c0666c0 0x7f1b7c068b80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0aead0 tx=0x7f1b880a6000 comp rx=0 tx=0).stop 2024-05-22T21:19:16.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.355+0000 7f1ba1200640 1 -- 172.21.15.161:0/1657420536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b9c06c7e0 msgr2=0x7f1b9c15aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:16.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.355+0000 7f1ba1200640 1 --2- 172.21.15.161:0/1657420536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b9c06c7e0 0x7f1b9c15aed0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00c4b0 tx=0x7f1b8c00c970 comp rx=0 tx=0).stop 2024-05-22T21:19:16.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.355+0000 7f1ba1200640 1 -- 172.21.15.161:0/1657420536 shutdown_connections 2024-05-22T21:19:16.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.355+0000 7f1ba1200640 1 --2- 172.21.15.161:0/1657420536 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1b7c0666c0 0x7f1b7c068b80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:16.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.355+0000 7f1ba1200640 1 --2- 172.21.15.161:0/1657420536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b9c15b600 0x7f1b9c1361d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:16.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.355+0000 7f1ba1200640 1 --2- 172.21.15.161:0/1657420536 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1b9c143c40 0x7f1b9c137500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:16.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.355+0000 7f1ba1200640 1 --2- 172.21.15.161:0/1657420536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b9c06c7e0 0x7f1b9c15aed0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:16.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.355+0000 7f1ba1200640 1 -- 172.21.15.161:0/1657420536 >> 172.21.15.161:0/1657420536 conn(0x7f1b9c06a2d0 msgr2=0x7f1b9c06a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:16.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.355+0000 7f1ba1200640 1 -- 172.21.15.161:0/1657420536 shutdown_connections 2024-05-22T21:19:16.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.355+0000 7f1ba1200640 1 -- 172.21.15.161:0/1657420536 wait complete. 2024-05-22T21:19:16.380 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-22T21:19:16.380 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-22T21:19:16.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.519+0000 7f234ba00640 1 Processor -- start 2024-05-22T21:19:16.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.519+0000 7f234ba00640 1 -- start start 2024-05-22T21:19:16.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.523+0000 7f234ba00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f234415b600 0x7f234406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:16.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.523+0000 7f234ba00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f234406c7e0 0x7f23440b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:16.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.523+0000 7f234ba00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2344144bb0 con 0x7f2344143c40 2024-05-22T21:19:16.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.523+0000 7f234ba00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f234415ba60 con 0x7f234406c7e0 2024-05-22T21:19:16.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.523+0000 7f234ba00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234406cc70 con 0x7f234415b600 2024-05-22T21:19:16.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.523+0000 7f2349000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f2344143c40 0x7f23441444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48808/0 (socket says 172.21.15.161:48808) 2024-05-22T21:19:16.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.523+0000 7f2349000640 1 -- 172.21.15.161:0/342689696 learned_addr learned my addr 172.21.15.161:0/342689696 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:16.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.523+0000 7f2349a00640 1 --2- 172.21.15.161:0/342689696 >> v2:172.21.15.161:3300/0 conn(0x7f234406c7e0 0x7f23440b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:16.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.523+0000 7f233be00640 1 --2- 172.21.15.161:0/342689696 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f234415b600 0x7f234406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:16.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.523+0000 7f2343e00640 1 -- 172.21.15.161:0/342689696 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008577980 0 0) 0x7f2344144bb0 con 0x7f2344143c40 2024-05-22T21:19:16.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.523+0000 7f2343e00640 1 -- 172.21.15.161:0/342689696 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2328003530 con 0x7f2344143c40 2024-05-22T21:19:16.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.523+0000 7f2349a00640 1 -- 172.21.15.161:0/342689696 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f234415b600 msgr2=0x7f234406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:16.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.523+0000 7f2349a00640 1 --2- 172.21.15.161:0/342689696 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f234415b600 0x7f234406c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:16.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.523+0000 7f2349a00640 1 -- 172.21.15.161:0/342689696 >> v1:172.21.15.161:6789/0 conn(0x7f2344143c40 legacy=0x7f23441444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:16.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.523+0000 7f2349a00640 1 -- 172.21.15.161:0/342689696 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234415efb0 con 0x7f234406c7e0 2024-05-22T21:19:16.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.523+0000 7f233be00640 1 --2- 172.21.15.161:0/342689696 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f234415b600 0x7f234406c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:16.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.527+0000 7f2349a00640 1 --2- 172.21.15.161:0/342689696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f234406c7e0 0x7f23440b2160 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f233c009d90 tx=0x7f233c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a13cf440cd2781 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:16.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.527+0000 7f2343e00640 1 -- 172.21.15.161:0/342689696 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c0a5a30 con 0x7f234406c7e0 2024-05-22T21:19:16.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.527+0000 7f2343e00640 1 -- 172.21.15.161:0/342689696 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233c0a5b90 con 0x7f234406c7e0 2024-05-22T21:19:16.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.527+0000 7f2343e00640 1 -- 172.21.15.161:0/342689696 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c0ad440 con 0x7f234406c7e0 2024-05-22T21:19:16.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.527+0000 7f234ba00640 1 -- 172.21.15.161:0/342689696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f234406c7e0 msgr2=0x7f23440b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:16.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.527+0000 7f234ba00640 1 --2- 172.21.15.161:0/342689696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f234406c7e0 0x7f23440b2160 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f233c009d90 tx=0x7f233c01c970 comp rx=0 tx=0).stop 2024-05-22T21:19:16.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.527+0000 7f234ba00640 1 -- 172.21.15.161:0/342689696 shutdown_connections 2024-05-22T21:19:16.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.527+0000 7f234ba00640 1 --2- 172.21.15.161:0/342689696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f234406c7e0 0x7f23440b2160 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:16.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.527+0000 7f234ba00640 1 --2- 172.21.15.161:0/342689696 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f234415b600 0x7f234406c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:16.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.527+0000 7f234ba00640 1 -- 172.21.15.161:0/342689696 >> 172.21.15.161:0/342689696 conn(0x7f234406a2d0 msgr2=0x7f234406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:16.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.527+0000 7f234ba00640 1 -- 172.21.15.161:0/342689696 shutdown_connections 2024-05-22T21:19:16.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.527+0000 7f234ba00640 1 -- 172.21.15.161:0/342689696 wait complete. 2024-05-22T21:19:16.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.527+0000 7f234ba00640 1 Processor -- start 2024-05-22T21:19:16.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.527+0000 7f234ba00640 1 -- start start 2024-05-22T21:19:16.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f234ba00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f234406c7e0 0x7f2344156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:16.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f234ba00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2344143c40 0x7f2344159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:16.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f234ba00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f234415b600 0x7f2344070d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:16.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f234ba00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234415bba0 con 0x7f234406c7e0 2024-05-22T21:19:16.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f234ba00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23441573f0 con 0x7f2344143c40 2024-05-22T21:19:16.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f234ba00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2344158d00 con 0x7f234415b600 2024-05-22T21:19:16.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f2349000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f234406c7e0 0x7f2344156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:16.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f2349000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f234406c7e0 0x7f2344156eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44672/0 (socket says 172.21.15.161:44672) 2024-05-22T21:19:16.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f2349000640 1 -- 172.21.15.161:0/3442297691 learned_addr learned my addr 172.21.15.161:0/3442297691 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:16.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f2349000640 1 -- 172.21.15.161:0/3442297691 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f234415b600 msgr2=0x7f2344070d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:16.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f2349a00640 1 --2- 172.21.15.161:0/3442297691 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f234415b600 0x7f2344070d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:16.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f233be00640 1 --2- 172.21.15.161:0/3442297691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2344143c40 0x7f2344159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:16.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f2349000640 1 --2- 172.21.15.161:0/3442297691 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f234415b600 0x7f2344070d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:16.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f2349000640 1 -- 172.21.15.161:0/3442297691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2344143c40 msgr2=0x7f2344159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:16.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f2349000640 1 --2- 172.21.15.161:0/3442297691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2344143c40 0x7f2344159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:16.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f2349000640 1 -- 172.21.15.161:0/3442297691 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c009a70 con 0x7f234406c7e0 2024-05-22T21:19:16.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f2349a00640 1 --2- 172.21.15.161:0/3442297691 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f234415b600 0x7f2344070d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:16.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f233be00640 1 --2- 172.21.15.161:0/3442297691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2344143c40 0x7f2344159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:16.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f2349000640 1 --2- 172.21.15.161:0/3442297691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f234406c7e0 0x7f2344156eb0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f233400c4d0 tx=0x7f233400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:16.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f2341600640 1 -- 172.21.15.161:0/3442297691 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2334005cf0 con 0x7f234406c7e0 2024-05-22T21:19:16.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f2341600640 1 -- 172.21.15.161:0/3442297691 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2334005e50 con 0x7f234406c7e0 2024-05-22T21:19:16.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f234ba00640 1 -- 172.21.15.161:0/3442297691 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2344158fe0 con 0x7f234406c7e0 2024-05-22T21:19:16.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f2341600640 1 -- 172.21.15.161:0/3442297691 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2334007270 con 0x7f234406c7e0 2024-05-22T21:19:16.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f234ba00640 1 -- 172.21.15.161:0/3442297691 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23441595a0 con 0x7f234406c7e0 2024-05-22T21:19:16.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f234ba00640 1 -- 172.21.15.161:0/3442297691 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2344067220 con 0x7f234406c7e0 2024-05-22T21:19:16.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.531+0000 7f2341600640 1 -- 172.21.15.161:0/3442297691 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f23340073d0 con 0x7f234406c7e0 2024-05-22T21:19:16.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.535+0000 7f2341600640 1 --2- 172.21.15.161:0/3442297691 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2320066690 0x7f2320068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:16.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.535+0000 7f2341600640 1 -- 172.21.15.161:0/3442297691 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23340836f0 con 0x7f234406c7e0 2024-05-22T21:19:16.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.535+0000 7f233be00640 1 --2- 172.21.15.161:0/3442297691 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2320066690 0x7f2320068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:16.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.535+0000 7f233be00640 1 --2- 172.21.15.161:0/3442297691 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2320066690 0x7f2320068b50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f23440720d0 tx=0x7f232c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:16.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.539+0000 7f2341600640 1 -- 172.21.15.161:0/3442297691 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2334050a70 con 0x7f234406c7e0 2024-05-22T21:19:16.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:16.723+0000 7f234ba00640 1 -- 172.21.15.161:0/3442297691 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2344143260 con 0x7f234406c7e0 2024-05-22T21:19:17.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.363+0000 7f2341600640 1 -- 172.21.15.161:0/3442297691 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v54) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2334054090 con 0x7f234406c7e0 2024-05-22T21:19:17.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.363+0000 7f234ba00640 1 -- 172.21.15.161:0/3442297691 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2320066690 msgr2=0x7f2320068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:17.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.363+0000 7f234ba00640 1 --2- 172.21.15.161:0/3442297691 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2320066690 0x7f2320068b50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f23440720d0 tx=0x7f232c008040 comp rx=0 tx=0).stop 2024-05-22T21:19:17.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.363+0000 7f234ba00640 1 -- 172.21.15.161:0/3442297691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f234406c7e0 msgr2=0x7f2344156eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:17.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.363+0000 7f234ba00640 1 --2- 172.21.15.161:0/3442297691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f234406c7e0 0x7f2344156eb0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f233400c4d0 tx=0x7f233400c990 comp rx=0 tx=0).stop 2024-05-22T21:19:17.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.363+0000 7f234ba00640 1 -- 172.21.15.161:0/3442297691 shutdown_connections 2024-05-22T21:19:17.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.363+0000 7f234ba00640 1 --2- 172.21.15.161:0/3442297691 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2320066690 0x7f2320068b50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.363+0000 7f234ba00640 1 --2- 172.21.15.161:0/3442297691 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f234415b600 0x7f2344070d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.363+0000 7f234ba00640 1 --2- 172.21.15.161:0/3442297691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2344143c40 0x7f2344159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.363+0000 7f234ba00640 1 --2- 172.21.15.161:0/3442297691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f234406c7e0 0x7f2344156eb0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.363+0000 7f234ba00640 1 -- 172.21.15.161:0/3442297691 >> 172.21.15.161:0/3442297691 conn(0x7f234406a2d0 msgr2=0x7f23440cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:17.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.363+0000 7f234ba00640 1 -- 172.21.15.161:0/3442297691 shutdown_connections 2024-05-22T21:19:17.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.363+0000 7f234ba00640 1 -- 172.21.15.161:0/3442297691 wait complete. 2024-05-22T21:19:17.388 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-22T21:19:17.389 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:17.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.523+0000 7fe8dc400640 1 Processor -- start 2024-05-22T21:19:17.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.523+0000 7fe8dc400640 1 -- start start 2024-05-22T21:19:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.523+0000 7fe8dc400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe8d415dd50 0x7fe8d406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.523+0000 7fe8dc400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe8d415e130 0x7fe8d415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.523+0000 7fe8dc400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d4071670 con 0x7fe8d415d410 2024-05-22T21:19:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.523+0000 7fe8dc400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d415d8a0 con 0x7fe8d415e130 2024-05-22T21:19:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.523+0000 7fe8dc400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d406de40 con 0x7fe8d415dd50 2024-05-22T21:19:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.523+0000 7fe8d9000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe8d415e130 0x7fe8d415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.523+0000 7fe8d9000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe8d415e130 0x7fe8d415ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44674/0 (socket says 172.21.15.161:44674) 2024-05-22T21:19:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.523+0000 7fe8d9000640 1 -- 172.21.15.161:0/2103643945 learned_addr learned my addr 172.21.15.161:0/2103643945 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.523+0000 7fe8d9a00640 1 --2- 172.21.15.161:0/2103643945 >> v2:172.21.15.179:3300/0 conn(0x7fe8d415dd50 0x7fe8d406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.523+0000 7fe8d9000640 1 -- 172.21.15.161:0/2103643945 >> v2:172.21.15.179:3300/0 conn(0x7fe8d415dd50 msgr2=0x7fe8d406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.523+0000 7fe8d9000640 1 --2- 172.21.15.161:0/2103643945 >> v2:172.21.15.179:3300/0 conn(0x7fe8d415dd50 0x7fe8d406d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.523+0000 7fe8d9000640 1 -- 172.21.15.161:0/2103643945 >> v1:172.21.15.179:6789/0 conn(0x7fe8d415d410 legacy=0x7fe8d415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:17.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.523+0000 7fe8d9000640 1 -- 172.21.15.161:0/2103643945 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d4070a60 con 0x7fe8d415e130 2024-05-22T21:19:17.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.527+0000 7fe8d9000640 1 --2- 172.21.15.161:0/2103643945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8d415e130 0x7fe8d415ced0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4072330 tx=0x7fe8c001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec4af224a7d1ef0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:17.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.527+0000 7fe8d3e00640 1 -- 172.21.15.161:0/2103643945 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c00aa610 con 0x7fe8d415e130 2024-05-22T21:19:17.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.527+0000 7fe8d3e00640 1 -- 172.21.15.161:0/2103643945 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c00a6040 con 0x7fe8d415e130 2024-05-22T21:19:17.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.527+0000 7fe8d3e00640 1 -- 172.21.15.161:0/2103643945 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c00aa990 con 0x7fe8d415e130 2024-05-22T21:19:17.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.527+0000 7fe8dc400640 1 -- 172.21.15.161:0/2103643945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8d415e130 msgr2=0x7fe8d415ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:17.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.527+0000 7fe8dc400640 1 --2- 172.21.15.161:0/2103643945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8d415e130 0x7fe8d415ced0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4072330 tx=0x7fe8c001c4b0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.527+0000 7fe8dc400640 1 -- 172.21.15.161:0/2103643945 shutdown_connections 2024-05-22T21:19:17.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.527+0000 7fe8dc400640 1 --2- 172.21.15.161:0/2103643945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8d415e130 0x7fe8d415ced0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.527+0000 7fe8dc400640 1 --2- 172.21.15.161:0/2103643945 >> v2:172.21.15.179:3300/0 conn(0x7fe8d415dd50 0x7fe8d406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.527+0000 7fe8dc400640 1 -- 172.21.15.161:0/2103643945 >> 172.21.15.161:0/2103643945 conn(0x7fe8d40b0bc0 msgr2=0x7fe8d40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:17.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.527+0000 7fe8dc400640 1 -- 172.21.15.161:0/2103643945 shutdown_connections 2024-05-22T21:19:17.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.527+0000 7fe8dc400640 1 -- 172.21.15.161:0/2103643945 wait complete. 2024-05-22T21:19:17.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8dc400640 1 Processor -- start 2024-05-22T21:19:17.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8dc400640 1 -- start start 2024-05-22T21:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8dc400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8d415d410 0x7fe8d40b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8dc400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8d415dd50 0x7fe8d41536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8dc400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8d415e130 0x7fe8d40cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8dc400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d40cd370 con 0x7fe8d415d410 2024-05-22T21:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8dc400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d40cc760 con 0x7fe8d415dd50 2024-05-22T21:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8dc400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d40cca40 con 0x7fe8d415e130 2024-05-22T21:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8d9a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8d415d410 0x7fe8d40b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8d9a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8d415d410 0x7fe8d40b3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44682/0 (socket says 172.21.15.161:44682) 2024-05-22T21:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8d9a00640 1 -- 172.21.15.161:0/745427210 learned_addr learned my addr 172.21.15.161:0/745427210 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8d9a00640 1 -- 172.21.15.161:0/745427210 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8d415e130 msgr2=0x7fe8d40cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:17.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8d9a00640 1 --2- 172.21.15.161:0/745427210 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8d415e130 0x7fe8d40cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8d9000640 1 --2- 172.21.15.161:0/745427210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8d415dd50 0x7fe8d41536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:17.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8d9a00640 1 -- 172.21.15.161:0/745427210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8d415dd50 msgr2=0x7fe8d41536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:17.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8d9a00640 1 --2- 172.21.15.161:0/745427210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8d415dd50 0x7fe8d41536e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8d9a00640 1 -- 172.21.15.161:0/745427210 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c00094d0 con 0x7fe8d415d410 2024-05-22T21:19:17.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8d9a00640 1 --2- 172.21.15.161:0/745427210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8d415d410 0x7fe8d40b3670 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800d800 tx=0x7fe8c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:17.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8d1600640 1 -- 172.21.15.161:0/745427210 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c800b6c0 con 0x7fe8d415d410 2024-05-22T21:19:17.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8d1600640 1 -- 172.21.15.161:0/745427210 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c800b820 con 0x7fe8d415d410 2024-05-22T21:19:17.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8d1600640 1 -- 172.21.15.161:0/745427210 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c800bb90 con 0x7fe8d415d410 2024-05-22T21:19:17.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8dc400640 1 -- 172.21.15.161:0/745427210 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d40ccd20 con 0x7fe8d415d410 2024-05-22T21:19:17.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.531+0000 7fe8dc400640 1 -- 172.21.15.161:0/745427210 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d41529b0 con 0x7fe8d415d410 2024-05-22T21:19:17.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.535+0000 7fe8dc400640 1 -- 172.21.15.161:0/745427210 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d4067220 con 0x7fe8d415d410 2024-05-22T21:19:17.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.535+0000 7fe8d1600640 1 -- 172.21.15.161:0/745427210 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe8c800bcf0 con 0x7fe8d415d410 2024-05-22T21:19:17.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.535+0000 7fe8d1600640 1 --2- 172.21.15.161:0/745427210 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe8b4066690 0x7fe8b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:17.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.535+0000 7fe8d1600640 1 -- 172.21.15.161:0/745427210 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8c80847c0 con 0x7fe8d415d410 2024-05-22T21:19:17.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.535+0000 7fe8d9000640 1 --2- 172.21.15.161:0/745427210 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe8b4066690 0x7fe8b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:17.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.535+0000 7fe8d9000640 1 --2- 172.21.15.161:0/745427210 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe8b4066690 0x7fe8b4068b50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00ae000 tx=0x7fe8c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:17.542 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.539+0000 7fe8d1600640 1 -- 172.21.15.161:0/745427210 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe8c8051a90 con 0x7fe8d415d410 2024-05-22T21:19:17.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.723+0000 7fe8dc400640 1 -- 172.21.15.161:0/745427210 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8d40b05b0 con 0x7fe8d415d410 2024-05-22T21:19:17.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.723+0000 7fe8d1600640 1 -- 172.21.15.161:0/745427210 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4748 (secure 0 0 0) 0x7fe8c8011340 con 0x7fe8d415d410 2024-05-22T21:19:17.725 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:17.725 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3944033857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3944033857},{"type":"v1","addr":"172.21.15.179:6834","nonce":3944033857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6125,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/4216709187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4216709187},{"type":"v1","addr":"172.21.15.161:6836","nonce":4216709187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2364917653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2364917653},{"type":"v1","addr":"172.21.15.179:6835","nonce":2364917653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:14.345936+0000","modified":"2024-05-22T21:19:17.355628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/1424863663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1424863663},{"type":"v1","addr":"172.21.15.161:6837","nonce":1424863663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-22T21:19:17.726 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 54 2024-05-22T21:19:17.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.723+0000 7fe8dc400640 1 -- 172.21.15.161:0/745427210 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe8b4066690 msgr2=0x7fe8b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:17.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.723+0000 7fe8dc400640 1 --2- 172.21.15.161:0/745427210 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe8b4066690 0x7fe8b4068b50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00ae000 tx=0x7fe8c0008040 comp rx=0 tx=0).stop 2024-05-22T21:19:17.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.723+0000 7fe8dc400640 1 -- 172.21.15.161:0/745427210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8d415d410 msgr2=0x7fe8d40b3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:17.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.723+0000 7fe8dc400640 1 --2- 172.21.15.161:0/745427210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8d415d410 0x7fe8d40b3670 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800d800 tx=0x7fe8c800dcc0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.723+0000 7fe8dc400640 1 -- 172.21.15.161:0/745427210 shutdown_connections 2024-05-22T21:19:17.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.723+0000 7fe8dc400640 1 --2- 172.21.15.161:0/745427210 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe8b4066690 0x7fe8b4068b50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.723+0000 7fe8dc400640 1 --2- 172.21.15.161:0/745427210 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8d415e130 0x7fe8d40cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.723+0000 7fe8dc400640 1 --2- 172.21.15.161:0/745427210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8d415dd50 0x7fe8d41536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.723+0000 7fe8dc400640 1 --2- 172.21.15.161:0/745427210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8d415d410 0x7fe8d40b3670 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.723+0000 7fe8dc400640 1 -- 172.21.15.161:0/745427210 >> 172.21.15.161:0/745427210 conn(0x7fe8d40b0bc0 msgr2=0x7fe8d40ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:17.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.723+0000 7fe8dc400640 1 -- 172.21.15.161:0/745427210 shutdown_connections 2024-05-22T21:19:17.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.723+0000 7fe8dc400640 1 -- 172.21.15.161:0/745427210 wait complete. 2024-05-22T21:19:17.747 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:17.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.879+0000 7fa757800640 1 Processor -- start 2024-05-22T21:19:17.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.879+0000 7fa757800640 1 -- start start 2024-05-22T21:19:17.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.879+0000 7fa757800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa75015e130 0x7fa75015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:17.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.879+0000 7fa757800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa75015d410 0x7fa75015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:17.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.879+0000 7fa757800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa750071670 con 0x7fa75015dd50 2024-05-22T21:19:17.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.879+0000 7fa757800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa75015d870 con 0x7fa75015d410 2024-05-22T21:19:17.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.879+0000 7fa757800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa750070a60 con 0x7fa75015e130 2024-05-22T21:19:17.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.879+0000 7fa755800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa75015d410 0x7fa75015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:17.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.879+0000 7fa755800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa75015d410 0x7fa75015b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44684/0 (socket says 172.21.15.161:44684) 2024-05-22T21:19:17.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.879+0000 7fa755800640 1 -- 172.21.15.161:0/4136989597 learned_addr learned my addr 172.21.15.161:0/4136989597 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:17.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.879+0000 7fa747e00640 1 --2- 172.21.15.161:0/4136989597 >> v2:172.21.15.179:3300/0 conn(0x7fa75015e130 0x7fa75015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:17.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.879+0000 7fa755800640 1 -- 172.21.15.161:0/4136989597 >> v2:172.21.15.179:3300/0 conn(0x7fa75015e130 msgr2=0x7fa75015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:17.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.883+0000 7fa755800640 1 --2- 172.21.15.161:0/4136989597 >> v2:172.21.15.179:3300/0 conn(0x7fa75015e130 0x7fa75015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.883+0000 7fa755800640 1 -- 172.21.15.161:0/4136989597 >> v1:172.21.15.179:6789/0 conn(0x7fa75015dd50 legacy=0x7fa75006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:17.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.883+0000 7fa755800640 1 -- 172.21.15.161:0/4136989597 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa750070d40 con 0x7fa75015d410 2024-05-22T21:19:17.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.883+0000 7fa747e00640 1 --2- 172.21.15.161:0/4136989597 >> v2:172.21.15.179:3300/0 conn(0x7fa75015e130 0x7fa75015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:17.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.883+0000 7fa755800640 1 --2- 172.21.15.161:0/4136989597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75015d410 0x7fa75015b6c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa75006c7c0 tx=0x7fa74801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfea5c1c26e4abe9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:17.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.883+0000 7fa74fe00640 1 -- 172.21.15.161:0/4136989597 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7480a5880 con 0x7fa75015d410 2024-05-22T21:19:17.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.883+0000 7fa74fe00640 1 -- 172.21.15.161:0/4136989597 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7480a59e0 con 0x7fa75015d410 2024-05-22T21:19:17.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.883+0000 7fa74fe00640 1 -- 172.21.15.161:0/4136989597 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7480a5d70 con 0x7fa75015d410 2024-05-22T21:19:17.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.883+0000 7fa757800640 1 -- 172.21.15.161:0/4136989597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75015d410 msgr2=0x7fa75015b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:17.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.883+0000 7fa757800640 1 --2- 172.21.15.161:0/4136989597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75015d410 0x7fa75015b6c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa75006c7c0 tx=0x7fa74801c9c0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.883+0000 7fa757800640 1 -- 172.21.15.161:0/4136989597 shutdown_connections 2024-05-22T21:19:17.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.883+0000 7fa757800640 1 --2- 172.21.15.161:0/4136989597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75015d410 0x7fa75015b6c0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.883+0000 7fa757800640 1 --2- 172.21.15.161:0/4136989597 >> v2:172.21.15.179:3300/0 conn(0x7fa75015e130 0x7fa75015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.883+0000 7fa757800640 1 -- 172.21.15.161:0/4136989597 >> 172.21.15.161:0/4136989597 conn(0x7fa7500b0bc0 msgr2=0x7fa7500b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:17.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.883+0000 7fa757800640 1 -- 172.21.15.161:0/4136989597 shutdown_connections 2024-05-22T21:19:17.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.883+0000 7fa757800640 1 -- 172.21.15.161:0/4136989597 wait complete. 2024-05-22T21:19:17.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa757800640 1 Processor -- start 2024-05-22T21:19:17.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa757800640 1 -- start start 2024-05-22T21:19:17.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa757800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa75015d410 0x7fa7500cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:17.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa757800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa75015dd50 0x7fa7500cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:17.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa757800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75015e130 0x7fa750158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:17.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa757800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7500ccd20 con 0x7fa75015e130 2024-05-22T21:19:17.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa757800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7500ce2e0 con 0x7fa75015dd50 2024-05-22T21:19:17.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa757800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7500cdd30 con 0x7fa75015d410 2024-05-22T21:19:17.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa755800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75015e130 0x7fa750158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:17.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa755800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75015e130 0x7fa750158610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44688/0 (socket says 172.21.15.161:44688) 2024-05-22T21:19:17.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa747e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa75015dd50 0x7fa7500cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:17.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa755800640 1 -- 172.21.15.161:0/3295174870 learned_addr learned my addr 172.21.15.161:0/3295174870 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:17.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa754e00640 1 --2- 172.21.15.161:0/3295174870 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa75015d410 0x7fa7500cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:17.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa755800640 1 -- 172.21.15.161:0/3295174870 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa75015d410 msgr2=0x7fa7500cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:17.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa755800640 1 --2- 172.21.15.161:0/3295174870 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa75015d410 0x7fa7500cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa755800640 1 -- 172.21.15.161:0/3295174870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa75015dd50 msgr2=0x7fa7500cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:17.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa755800640 1 --2- 172.21.15.161:0/3295174870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa75015dd50 0x7fa7500cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:17.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa755800640 1 -- 172.21.15.161:0/3295174870 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa748009b30 con 0x7fa75015e130 2024-05-22T21:19:17.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa755800640 1 --2- 172.21.15.161:0/3295174870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75015e130 0x7fa750158610 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa7480af040 tx=0x7fa748019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:17.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa74d600640 1 -- 172.21.15.161:0/3295174870 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7480b8670 con 0x7fa75015e130 2024-05-22T21:19:17.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa74d600640 1 -- 172.21.15.161:0/3295174870 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7480b87f0 con 0x7fa75015e130 2024-05-22T21:19:17.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.887+0000 7fa757800640 1 -- 172.21.15.161:0/3295174870 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7500ce000 con 0x7fa75015e130 2024-05-22T21:19:17.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.891+0000 7fa74d600640 1 -- 172.21.15.161:0/3295174870 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7480b8ba0 con 0x7fa75015e130 2024-05-22T21:19:17.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.891+0000 7fa757800640 1 -- 172.21.15.161:0/3295174870 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7500cc9b0 con 0x7fa75015e130 2024-05-22T21:19:17.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.891+0000 7fa757800640 1 -- 172.21.15.161:0/3295174870 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7500cced0 con 0x7fa75015e130 2024-05-22T21:19:17.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.891+0000 7fa74d600640 1 -- 172.21.15.161:0/3295174870 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa7480b8d00 con 0x7fa75015e130 2024-05-22T21:19:17.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.891+0000 7fa74d600640 1 --2- 172.21.15.161:0/3295174870 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa72c066690 0x7fa72c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:17.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.891+0000 7fa74d600640 1 -- 172.21.15.161:0/3295174870 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa74811f8b0 con 0x7fa75015e130 2024-05-22T21:19:17.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.891+0000 7fa754e00640 1 --2- 172.21.15.161:0/3295174870 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa72c066690 0x7fa72c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:17.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.891+0000 7fa754e00640 1 --2- 172.21.15.161:0/3295174870 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa72c066690 0x7fa72c068b50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa740007fd0 tx=0x7fa740009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:17.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:17.895+0000 7fa74d600640 1 -- 172.21.15.161:0/3295174870 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa7500cced0 con 0x7fa75015e130 2024-05-22T21:19:18.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.051+0000 7fa757800640 1 -- 172.21.15.161:0/3295174870 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7500cced0 con 0x7fa75015e130 2024-05-22T21:19:18.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.051+0000 7fa74d600640 1 -- 172.21.15.161:0/3295174870 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16100 (secure 0 0 0) 0x7fa7500cced0 con 0x7fa75015e130 2024-05-22T21:19:18.055 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:18.056 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":125,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:19:15.348424+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:12.848545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:13.344629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.055+0000 7fa757800640 1 -- 172.21.15.161:0/3295174870 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa72c066690 msgr2=0x7fa72c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.055+0000 7fa757800640 1 --2- 172.21.15.161:0/3295174870 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa72c066690 0x7fa72c068b50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa740007fd0 tx=0x7fa740009040 comp rx=0 tx=0).stop 2024-05-22T21:19:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.055+0000 7fa757800640 1 -- 172.21.15.161:0/3295174870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75015e130 msgr2=0x7fa750158610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.055+0000 7fa757800640 1 --2- 172.21.15.161:0/3295174870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75015e130 0x7fa750158610 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa7480af040 tx=0x7fa748019760 comp rx=0 tx=0).stop 2024-05-22T21:19:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.055+0000 7fa757800640 1 -- 172.21.15.161:0/3295174870 shutdown_connections 2024-05-22T21:19:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.055+0000 7fa757800640 1 --2- 172.21.15.161:0/3295174870 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa72c066690 0x7fa72c068b50 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.055+0000 7fa757800640 1 --2- 172.21.15.161:0/3295174870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75015e130 0x7fa750158610 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.055+0000 7fa757800640 1 --2- 172.21.15.161:0/3295174870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa75015dd50 0x7fa7500cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.055+0000 7fa757800640 1 --2- 172.21.15.161:0/3295174870 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa75015d410 0x7fa7500cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.055+0000 7fa757800640 1 -- 172.21.15.161:0/3295174870 >> 172.21.15.161:0/3295174870 conn(0x7fa7500b0bc0 msgr2=0x7fa7500ad9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:18.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.055+0000 7fa757800640 1 -- 172.21.15.161:0/3295174870 shutdown_connections 2024-05-22T21:19:18.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.055+0000 7fa757800640 1 -- 172.21.15.161:0/3295174870 wait complete. 2024-05-22T21:19:18.077 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-22T21:19:18.078 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:19:18.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61ec600640 1 Processor -- start 2024-05-22T21:19:18.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61ec600640 1 -- start start 2024-05-22T21:19:18.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61ec600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61e40707c0 0x7f61e406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61ec600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f61e406d530 0x7f61e4160e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61ec600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e4071fd0 con 0x7f61e415d430 2024-05-22T21:19:18.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61ec600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e40acfc0 con 0x7f61e406d530 2024-05-22T21:19:18.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61ec600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e4070c00 con 0x7f61e40707c0 2024-05-22T21:19:18.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61e9200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61e40707c0 0x7f61e406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61ea600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f61e406d530 0x7f61e4160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61e9200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61e40707c0 0x7f61e406cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:39034/0 (socket says 172.21.15.161:39034) 2024-05-22T21:19:18.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61e9200640 1 -- 172.21.15.161:0/3555975253 learned_addr learned my addr 172.21.15.161:0/3555975253 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:18.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61e9c00640 1 --1- 172.21.15.161:0/3555975253 >> v1:172.21.15.179:6789/0 conn(0x7f61e415d430 0x7f61e40ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:45078/0 (socket says 172.21.15.161:45078) 2024-05-22T21:19:18.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61e9200640 1 -- 172.21.15.161:0/3555975253 >> v2:172.21.15.179:3300/0 conn(0x7f61e406d530 msgr2=0x7f61e4160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:18.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61e9200640 1 --2- 172.21.15.161:0/3555975253 >> v2:172.21.15.179:3300/0 conn(0x7f61e406d530 0x7f61e4160e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61e9200640 1 -- 172.21.15.161:0/3555975253 >> v1:172.21.15.179:6789/0 conn(0x7f61e415d430 legacy=0x7f61e40ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61e3e00640 1 -- 172.21.15.161:0/3555975253 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 41908654 0 0) 0x7f61e4071fd0 con 0x7f61e415d430 2024-05-22T21:19:18.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.211+0000 7f61e9200640 1 -- 172.21.15.161:0/3555975253 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e4070e80 con 0x7f61e40707c0 2024-05-22T21:19:18.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.215+0000 7f61e9200640 1 --2- 172.21.15.161:0/3555975253 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61e40707c0 0x7f61e406cff0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009870 tx=0x7f61d001c1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90ef51fcdc248a58 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:18.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.215+0000 7f61e3e00640 1 -- 172.21.15.161:0/3555975253 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d00ab600 con 0x7f61e40707c0 2024-05-22T21:19:18.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.215+0000 7f61e3e00640 1 -- 172.21.15.161:0/3555975253 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d00a7050 con 0x7f61e40707c0 2024-05-22T21:19:18.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.215+0000 7f61e3e00640 1 -- 172.21.15.161:0/3555975253 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d00ab980 con 0x7f61e40707c0 2024-05-22T21:19:18.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.215+0000 7f61ec600640 1 -- 172.21.15.161:0/3555975253 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61e40707c0 msgr2=0x7f61e406cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.215+0000 7f61ec600640 1 --2- 172.21.15.161:0/3555975253 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61e40707c0 0x7f61e406cff0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009870 tx=0x7f61d001c1e0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.215+0000 7f61ec600640 1 -- 172.21.15.161:0/3555975253 shutdown_connections 2024-05-22T21:19:18.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.215+0000 7f61ec600640 1 --2- 172.21.15.161:0/3555975253 >> v2:172.21.15.179:3300/0 conn(0x7f61e406d530 0x7f61e4160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.215+0000 7f61ec600640 1 --2- 172.21.15.161:0/3555975253 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61e40707c0 0x7f61e406cff0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.215+0000 7f61ec600640 1 -- 172.21.15.161:0/3555975253 >> 172.21.15.161:0/3555975253 conn(0x7f61e40b0bc0 msgr2=0x7f61e40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:18.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.215+0000 7f61ec600640 1 -- 172.21.15.161:0/3555975253 shutdown_connections 2024-05-22T21:19:18.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.215+0000 7f61ec600640 1 -- 172.21.15.161:0/3555975253 wait complete. 2024-05-22T21:19:18.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61ec600640 1 Processor -- start 2024-05-22T21:19:18.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61ec600640 1 -- start start 2024-05-22T21:19:18.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61ec600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61e406d530 0x7f61e40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61ec600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61e40707c0 0x7f61e4152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61ec600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61e415d430 0x7f61e415be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61ec600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e4153880 con 0x7f61e40707c0 2024-05-22T21:19:18.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61ec600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e4151780 con 0x7f61e415d430 2024-05-22T21:19:18.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61ec600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e4151a60 con 0x7f61e406d530 2024-05-22T21:19:18.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61e9200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61e40707c0 0x7f61e4152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61e9200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61e40707c0 0x7f61e4152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44694/0 (socket says 172.21.15.161:44694) 2024-05-22T21:19:18.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61e9200640 1 -- 172.21.15.161:0/916009075 learned_addr learned my addr 172.21.15.161:0/916009075 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:18.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61e9c00640 1 --2- 172.21.15.161:0/916009075 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61e406d530 0x7f61e40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61ea600640 1 --2- 172.21.15.161:0/916009075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61e415d430 0x7f61e415be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61e9200640 1 -- 172.21.15.161:0/916009075 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61e406d530 msgr2=0x7f61e40af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61e9200640 1 --2- 172.21.15.161:0/916009075 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61e406d530 0x7f61e40af8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61e9200640 1 -- 172.21.15.161:0/916009075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61e415d430 msgr2=0x7f61e415be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61e9200640 1 --2- 172.21.15.161:0/916009075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61e415d430 0x7f61e415be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61e9200640 1 -- 172.21.15.161:0/916009075 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d00094d0 con 0x7f61e40707c0 2024-05-22T21:19:18.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61e9200640 1 --2- 172.21.15.161:0/916009075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61e40707c0 0x7f61e4152540 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f61d00ae040 tx=0x7f61d0018f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:18.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61e1600640 1 -- 172.21.15.161:0/916009075 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d00b4420 con 0x7f61e40707c0 2024-05-22T21:19:18.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61e1600640 1 -- 172.21.15.161:0/916009075 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d00b45a0 con 0x7f61e40707c0 2024-05-22T21:19:18.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61e1600640 1 -- 172.21.15.161:0/916009075 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d00b4950 con 0x7f61e40707c0 2024-05-22T21:19:18.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61ec600640 1 -- 172.21.15.161:0/916009075 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e4151d40 con 0x7f61e40707c0 2024-05-22T21:19:18.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61ec600640 1 -- 172.21.15.161:0/916009075 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e41534a0 con 0x7f61e40707c0 2024-05-22T21:19:18.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.219+0000 7f61ec600640 1 -- 172.21.15.161:0/916009075 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e4067220 con 0x7f61e40707c0 2024-05-22T21:19:18.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.223+0000 7f61e1600640 1 -- 172.21.15.161:0/916009075 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f61d00b2070 con 0x7f61e40707c0 2024-05-22T21:19:18.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.223+0000 7f61e1600640 1 --2- 172.21.15.161:0/916009075 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61c4066690 0x7f61c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.223+0000 7f61e1600640 1 -- 172.21.15.161:0/916009075 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61d011db60 con 0x7f61e40707c0 2024-05-22T21:19:18.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.223+0000 7f61e9c00640 1 --2- 172.21.15.161:0/916009075 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61c4066690 0x7f61c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.223+0000 7f61e9c00640 1 --2- 172.21.15.161:0/916009075 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61c4066690 0x7f61c4068b50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f61d8000f70 tx=0x7f61d8007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:18.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.227+0000 7f61e1600640 1 -- 172.21.15.161:0/916009075 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f61d00eaf70 con 0x7f61e40707c0 2024-05-22T21:19:18.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.379+0000 7f61ec600640 1 -- 172.21.15.161:0/916009075 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f61e406d940 con 0x7f61c4066690 2024-05-22T21:19:18.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.387+0000 7f61e1600640 1 -- 172.21.15.161:0/916009075 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284645 (secure 0 0 0) 0x7f61d8004c80 con 0x7f61c4066690 2024-05-22T21:19:18.392 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:18.393 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:19:18.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.391+0000 7f61ec600640 1 -- 172.21.15.161:0/916009075 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61c4066690 msgr2=0x7f61c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.391+0000 7f61ec600640 1 --2- 172.21.15.161:0/916009075 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61c4066690 0x7f61c4068b50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f61d8000f70 tx=0x7f61d8007df0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.391+0000 7f61ec600640 1 -- 172.21.15.161:0/916009075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61e40707c0 msgr2=0x7f61e4152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.391+0000 7f61ec600640 1 --2- 172.21.15.161:0/916009075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61e40707c0 0x7f61e4152540 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f61d00ae040 tx=0x7f61d0018f80 comp rx=0 tx=0).stop 2024-05-22T21:19:18.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.391+0000 7f61ec600640 1 -- 172.21.15.161:0/916009075 shutdown_connections 2024-05-22T21:19:18.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.391+0000 7f61ec600640 1 --2- 172.21.15.161:0/916009075 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61c4066690 0x7f61c4068b50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.391+0000 7f61ec600640 1 --2- 172.21.15.161:0/916009075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61e415d430 0x7f61e415be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.391+0000 7f61ec600640 1 --2- 172.21.15.161:0/916009075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61e40707c0 0x7f61e4152540 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.391+0000 7f61ec600640 1 --2- 172.21.15.161:0/916009075 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61e406d530 0x7f61e40af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.391+0000 7f61ec600640 1 -- 172.21.15.161:0/916009075 >> 172.21.15.161:0/916009075 conn(0x7f61e40b0bc0 msgr2=0x7f61e40ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:18.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.391+0000 7f61ec600640 1 -- 172.21.15.161:0/916009075 shutdown_connections 2024-05-22T21:19:18.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.391+0000 7f61ec600640 1 -- 172.21.15.161:0/916009075 wait complete. 2024-05-22T21:19:18.412 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-05-22T21:19:16.471237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6532,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555967,"statfs":{"total":1600000000,"available":1593310599,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000645"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780663+0000","last_change":"2024-05-22T21:19:14.364488+0000","last_active":"2024-05-22T21:19:15.780663+0000","last_peered":"2024-05-22T21:19:15.780663+0000","last_clean":"2024-05-22T21:19:15.780663+0000","last_became_active":"2024-05-22T21:19:14.364275+0000","last_became_peered":"2024-05-22T21:19:14.364275+0000","last_unstale":"2024-05-22T21:19:15.780663+0000","last_undegraded":"2024-05-22T21:19:15.780663+0000","last_fullsized":"2024-05-22T21:19:15.780663+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:26:45.307076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354084+0000","last_change":"2024-05-22T21:19:14.361359+0000","last_active":"2024-05-22T21:19:15.354084+0000","last_peered":"2024-05-22T21:19:15.354084+0000","last_clean":"2024-05-22T21:19:15.354084+0000","last_became_active":"2024-05-22T21:19:14.361124+0000","last_became_peered":"2024-05-22T21:19:14.361124+0000","last_unstale":"2024-05-22T21:19:15.354084+0000","last_undegraded":"2024-05-22T21:19:15.354084+0000","last_fullsized":"2024-05-22T21:19:15.354084+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:19:19.207878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780778+0000","last_change":"2024-05-22T21:19:14.364059+0000","last_active":"2024-05-22T21:19:15.780778+0000","last_peered":"2024-05-22T21:19:15.780778+0000","last_clean":"2024-05-22T21:19:15.780778+0000","last_became_active":"2024-05-22T21:19:14.363842+0000","last_became_peered":"2024-05-22T21:19:14.363842+0000","last_unstale":"2024-05-22T21:19:15.780778+0000","last_undegraded":"2024-05-22T21:19:15.780778+0000","last_fullsized":"2024-05-22T21:19:15.780778+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:48:30.354035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781230+0000","last_change":"2024-05-22T21:19:14.367626+0000","last_active":"2024-05-22T21:19:15.781230+0000","last_peered":"2024-05-22T21:19:15.781230+0000","last_clean":"2024-05-22T21:19:15.781230+0000","last_became_active":"2024-05-22T21:19:14.367419+0000","last_became_peered":"2024-05-22T21:19:14.367419+0000","last_unstale":"2024-05-22T21:19:15.781230+0000","last_undegraded":"2024-05-22T21:19:15.781230+0000","last_fullsized":"2024-05-22T21:19:15.781230+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:33:27.771256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.782129+0000","last_change":"2024-05-22T21:19:14.364747+0000","last_active":"2024-05-22T21:19:15.782129+0000","last_peered":"2024-05-22T21:19:15.782129+0000","last_clean":"2024-05-22T21:19:15.782129+0000","last_became_active":"2024-05-22T21:19:14.364470+0000","last_became_peered":"2024-05-22T21:19:14.364470+0000","last_unstale":"2024-05-22T21:19:15.782129+0000","last_undegraded":"2024-05-22T21:19:15.782129+0000","last_fullsized":"2024-05-22T21:19:15.782129+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:42:32.241040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355941+0000","last_change":"2024-05-22T21:19:14.361806+0000","last_active":"2024-05-22T21:19:15.355941+0000","last_peered":"2024-05-22T21:19:15.355941+0000","last_clean":"2024-05-22T21:19:15.355941+0000","last_became_active":"2024-05-22T21:19:14.361596+0000","last_became_peered":"2024-05-22T21:19:14.361596+0000","last_unstale":"2024-05-22T21:19:15.355941+0000","last_undegraded":"2024-05-22T21:19:15.355941+0000","last_fullsized":"2024-05-22T21:19:15.355941+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:40:13.108550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355493+0000","last_change":"2024-05-22T21:19:14.362109+0000","last_active":"2024-05-22T21:19:15.355493+0000","last_peered":"2024-05-22T21:19:15.355493+0000","last_clean":"2024-05-22T21:19:15.355493+0000","last_became_active":"2024-05-22T21:19:14.361747+0000","last_became_peered":"2024-05-22T21:19:14.361747+0000","last_unstale":"2024-05-22T21:19:15.355493+0000","last_undegraded":"2024-05-22T21:19:15.355493+0000","last_fullsized":"2024-05-22T21:19:15.355493+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:29:22.441939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354022+0000","last_change":"2024-05-22T21:19:14.359960+0000","last_active":"2024-05-22T21:19:15.354022+0000","last_peered":"2024-05-22T21:19:15.354022+0000","last_clean":"2024-05-22T21:19:15.354022+0000","last_became_active":"2024-05-22T21:19:14.357742+0000","last_became_peered":"2024-05-22T21:19:14.357742+0000","last_unstale":"2024-05-22T21:19:15.354022+0000","last_undegraded":"2024-05-22T21:19:15.354022+0000","last_fullsized":"2024-05-22T21:19:15.354022+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:55:51.198941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355893+0000","last_change":"2024-05-22T21:19:14.362220+0000","last_active":"2024-05-22T21:19:15.355893+0000","last_peered":"2024-05-22T21:19:15.355893+0000","last_clean":"2024-05-22T21:19:15.355893+0000","last_became_active":"2024-05-22T21:19:14.361982+0000","last_became_peered":"2024-05-22T21:19:14.361982+0000","last_unstale":"2024-05-22T21:19:15.355893+0000","last_undegraded":"2024-05-22T21:19:15.355893+0000","last_fullsized":"2024-05-22T21:19:15.355893+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:42.559556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.782234+0000","last_change":"2024-05-22T21:19:14.360223+0000","last_active":"2024-05-22T21:19:15.782234+0000","last_peered":"2024-05-22T21:19:15.782234+0000","last_clean":"2024-05-22T21:19:15.782234+0000","last_became_active":"2024-05-22T21:19:14.359784+0000","last_became_peered":"2024-05-22T21:19:14.359784+0000","last_unstale":"2024-05-22T21:19:15.782234+0000","last_undegraded":"2024-05-22T21:19:15.782234+0000","last_fullsized":"2024-05-22T21:19:15.782234+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:20:21.114154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356030+0000","last_change":"2024-05-22T21:19:14.355883+0000","last_active":"2024-05-22T21:19:15.356030+0000","last_peered":"2024-05-22T21:19:15.356030+0000","last_clean":"2024-05-22T21:19:15.356030+0000","last_became_active":"2024-05-22T21:19:14.355274+0000","last_became_peered":"2024-05-22T21:19:14.355274+0000","last_unstale":"2024-05-22T21:19:15.356030+0000","last_undegraded":"2024-05-22T21:19:15.356030+0000","last_fullsized":"2024-05-22T21:19:15.356030+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:57:26.931482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356335+0000","last_change":"2024-05-22T21:19:14.358871+0000","last_active":"2024-05-22T21:19:15.356335+0000","last_peered":"2024-05-22T21:19:15.356335+0000","last_clean":"2024-05-22T21:19:15.356335+0000","last_became_active":"2024-05-22T21:19:14.358522+0000","last_became_peered":"2024-05-22T21:19:14.358522+0000","last_unstale":"2024-05-22T21:19:15.356335+0000","last_undegraded":"2024-05-22T21:19:15.356335+0000","last_fullsized":"2024-05-22T21:19:15.356335+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:56:11.698434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780685+0000","last_change":"2024-05-22T21:19:14.365985+0000","last_active":"2024-05-22T21:19:15.780685+0000","last_peered":"2024-05-22T21:19:15.780685+0000","last_clean":"2024-05-22T21:19:15.780685+0000","last_became_active":"2024-05-22T21:19:14.365495+0000","last_became_peered":"2024-05-22T21:19:14.365495+0000","last_unstale":"2024-05-22T21:19:15.780685+0000","last_undegraded":"2024-05-22T21:19:15.780685+0000","last_fullsized":"2024-05-22T21:19:15.780685+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:20:21.114154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430337+0000","last_change":"2024-05-22T21:19:14.366921+0000","last_active":"2024-05-22T21:19:15.430337+0000","last_peered":"2024-05-22T21:19:15.430337+0000","last_clean":"2024-05-22T21:19:15.430337+0000","last_became_active":"2024-05-22T21:19:14.365934+0000","last_became_peered":"2024-05-22T21:19:14.365934+0000","last_unstale":"2024-05-22T21:19:15.430337+0000","last_undegraded":"2024-05-22T21:19:15.430337+0000","last_fullsized":"2024-05-22T21:19:15.430337+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:12:16.565090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781007+0000","last_change":"2024-05-22T21:19:14.358187+0000","last_active":"2024-05-22T21:19:15.781007+0000","last_peered":"2024-05-22T21:19:15.781007+0000","last_clean":"2024-05-22T21:19:15.781007+0000","last_became_active":"2024-05-22T21:19:14.357593+0000","last_became_peered":"2024-05-22T21:19:14.357593+0000","last_unstale":"2024-05-22T21:19:15.781007+0000","last_undegraded":"2024-05-22T21:19:15.781007+0000","last_fullsized":"2024-05-22T21:19:15.781007+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:08:05.273893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356119+0000","last_change":"2024-05-22T21:19:14.359327+0000","last_active":"2024-05-22T21:19:15.356119+0000","last_peered":"2024-05-22T21:19:15.356119+0000","last_clean":"2024-05-22T21:19:15.356119+0000","last_became_active":"2024-05-22T21:19:14.358923+0000","last_became_peered":"2024-05-22T21:19:14.358923+0000","last_unstale":"2024-05-22T21:19:15.356119+0000","last_undegraded":"2024-05-22T21:19:15.356119+0000","last_fullsized":"2024-05-22T21:19:15.356119+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:55:24.116469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354182+0000","last_change":"2024-05-22T21:19:14.371433+0000","last_active":"2024-05-22T21:19:15.354182+0000","last_peered":"2024-05-22T21:19:15.354182+0000","last_clean":"2024-05-22T21:19:15.354182+0000","last_became_active":"2024-05-22T21:19:14.371146+0000","last_became_peered":"2024-05-22T21:19:14.371146+0000","last_unstale":"2024-05-22T21:19:15.354182+0000","last_undegraded":"2024-05-22T21:19:15.354182+0000","last_fullsized":"2024-05-22T21:19:15.354182+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:04:40.868247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780779+0000","last_change":"2024-05-22T21:19:14.367029+0000","last_active":"2024-05-22T21:19:15.780779+0000","last_peered":"2024-05-22T21:19:15.780779+0000","last_clean":"2024-05-22T21:19:15.780779+0000","last_became_active":"2024-05-22T21:19:14.366752+0000","last_became_peered":"2024-05-22T21:19:14.366752+0000","last_unstale":"2024-05-22T21:19:15.780779+0000","last_undegraded":"2024-05-22T21:19:15.780779+0000","last_fullsized":"2024-05-22T21:19:15.780779+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:27:50.460840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430416+0000","last_change":"2024-05-22T21:19:14.370443+0000","last_active":"2024-05-22T21:19:15.430416+0000","last_peered":"2024-05-22T21:19:15.430416+0000","last_clean":"2024-05-22T21:19:15.430416+0000","last_became_active":"2024-05-22T21:19:14.369808+0000","last_became_peered":"2024-05-22T21:19:14.369808+0000","last_unstale":"2024-05-22T21:19:15.430416+0000","last_undegraded":"2024-05-22T21:19:15.430416+0000","last_fullsized":"2024-05-22T21:19:15.430416+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:20:42.029795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356379+0000","last_change":"2024-05-22T21:19:14.364250+0000","last_active":"2024-05-22T21:19:15.356379+0000","last_peered":"2024-05-22T21:19:15.356379+0000","last_clean":"2024-05-22T21:19:15.356379+0000","last_became_active":"2024-05-22T21:19:14.363689+0000","last_became_peered":"2024-05-22T21:19:14.363689+0000","last_unstale":"2024-05-22T21:19:15.356379+0000","last_undegraded":"2024-05-22T21:19:15.356379+0000","last_fullsized":"2024-05-22T21:19:15.356379+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:42:32.241040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430362+0000","last_change":"2024-05-22T21:19:14.362961+0000","last_active":"2024-05-22T21:19:15.430362+0000","last_peered":"2024-05-22T21:19:15.430362+0000","last_clean":"2024-05-22T21:19:15.430362+0000","last_became_active":"2024-05-22T21:19:14.362698+0000","last_became_peered":"2024-05-22T21:19:14.362698+0000","last_unstale":"2024-05-22T21:19:15.430362+0000","last_undegraded":"2024-05-22T21:19:15.430362+0000","last_fullsized":"2024-05-22T21:19:15.430362+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:18:09.280531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354349+0000","last_change":"2024-05-22T21:19:14.366403+0000","last_active":"2024-05-22T21:19:15.354349+0000","last_peered":"2024-05-22T21:19:15.354349+0000","last_clean":"2024-05-22T21:19:15.354349+0000","last_became_active":"2024-05-22T21:19:14.366182+0000","last_became_peered":"2024-05-22T21:19:14.366182+0000","last_unstale":"2024-05-22T21:19:15.354349+0000","last_undegraded":"2024-05-22T21:19:15.354349+0000","last_fullsized":"2024-05-22T21:19:15.354349+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:32:01.457841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354112+0000","last_change":"2024-05-22T21:19:14.368291+0000","last_active":"2024-05-22T21:19:15.354112+0000","last_peered":"2024-05-22T21:19:15.354112+0000","last_clean":"2024-05-22T21:19:15.354112+0000","last_became_active":"2024-05-22T21:19:14.367265+0000","last_became_peered":"2024-05-22T21:19:14.367265+0000","last_unstale":"2024-05-22T21:19:15.354112+0000","last_undegraded":"2024-05-22T21:19:15.354112+0000","last_fullsized":"2024-05-22T21:19:15.354112+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:49:38.868866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355555+0000","last_change":"2024-05-22T21:19:14.362548+0000","last_active":"2024-05-22T21:19:15.355555+0000","last_peered":"2024-05-22T21:19:15.355555+0000","last_clean":"2024-05-22T21:19:15.355555+0000","last_became_active":"2024-05-22T21:19:14.362342+0000","last_became_peered":"2024-05-22T21:19:14.362342+0000","last_unstale":"2024-05-22T21:19:15.355555+0000","last_undegraded":"2024-05-22T21:19:15.355555+0000","last_fullsized":"2024-05-22T21:19:15.355555+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:20:21.114154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.782332+0000","last_change":"2024-05-22T21:19:14.365909+0000","last_active":"2024-05-22T21:19:15.782332+0000","last_peered":"2024-05-22T21:19:15.782332+0000","last_clean":"2024-05-22T21:19:15.782332+0000","last_became_active":"2024-05-22T21:19:14.365017+0000","last_became_peered":"2024-05-22T21:19:14.365017+0000","last_unstale":"2024-05-22T21:19:15.782332+0000","last_undegraded":"2024-05-22T21:19:15.782332+0000","last_fullsized":"2024-05-22T21:19:15.782332+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:20:19.111100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354113+0000","last_change":"2024-05-22T21:19:14.356082+0000","last_active":"2024-05-22T21:19:15.354113+0000","last_peered":"2024-05-22T21:19:15.354113+0000","last_clean":"2024-05-22T21:19:15.354113+0000","last_became_active":"2024-05-22T21:19:14.355865+0000","last_became_peered":"2024-05-22T21:19:14.355865+0000","last_unstale":"2024-05-22T21:19:15.354113+0000","last_undegraded":"2024-05-22T21:19:15.354113+0000","last_fullsized":"2024-05-22T21:19:15.354113+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:40:13.108550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355615+0000","last_change":"2024-05-22T21:19:14.356990+0000","last_active":"2024-05-22T21:19:15.355615+0000","last_peered":"2024-05-22T21:19:15.355615+0000","last_clean":"2024-05-22T21:19:15.355615+0000","last_became_active":"2024-05-22T21:19:14.356756+0000","last_became_peered":"2024-05-22T21:19:14.356756+0000","last_unstale":"2024-05-22T21:19:15.355615+0000","last_undegraded":"2024-05-22T21:19:15.355615+0000","last_fullsized":"2024-05-22T21:19:15.355615+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:09:35.979308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781851+0000","last_change":"2024-05-22T21:19:14.367805+0000","last_active":"2024-05-22T21:19:15.781851+0000","last_peered":"2024-05-22T21:19:15.781851+0000","last_clean":"2024-05-22T21:19:15.781851+0000","last_became_active":"2024-05-22T21:19:14.367093+0000","last_became_peered":"2024-05-22T21:19:14.367093+0000","last_unstale":"2024-05-22T21:19:15.781851+0000","last_undegraded":"2024-05-22T21:19:15.781851+0000","last_fullsized":"2024-05-22T21:19:15.781851+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:42.559556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.353998+0000","last_change":"2024-05-22T21:19:14.369196+0000","last_active":"2024-05-22T21:19:15.353998+0000","last_peered":"2024-05-22T21:19:15.353998+0000","last_clean":"2024-05-22T21:19:15.353998+0000","last_became_active":"2024-05-22T21:19:14.368984+0000","last_became_peered":"2024-05-22T21:19:14.368984+0000","last_unstale":"2024-05-22T21:19:15.353998+0000","last_undegraded":"2024-05-22T21:19:15.353998+0000","last_fullsized":"2024-05-22T21:19:15.353998+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:29:22.441939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430645+0000","last_change":"2024-05-22T21:19:14.359063+0000","last_active":"2024-05-22T21:19:15.430645+0000","last_peered":"2024-05-22T21:19:15.430645+0000","last_clean":"2024-05-22T21:19:15.430645+0000","last_became_active":"2024-05-22T21:19:14.358819+0000","last_became_peered":"2024-05-22T21:19:14.358819+0000","last_unstale":"2024-05-22T21:19:15.430645+0000","last_undegraded":"2024-05-22T21:19:15.430645+0000","last_fullsized":"2024-05-22T21:19:15.430645+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:44:39.869878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780601+0000","last_change":"2024-05-22T21:19:14.362184+0000","last_active":"2024-05-22T21:19:15.780601+0000","last_peered":"2024-05-22T21:19:15.780601+0000","last_clean":"2024-05-22T21:19:15.780601+0000","last_became_active":"2024-05-22T21:19:14.361968+0000","last_became_peered":"2024-05-22T21:19:14.361968+0000","last_unstale":"2024-05-22T21:19:15.780601+0000","last_undegraded":"2024-05-22T21:19:15.780601+0000","last_fullsized":"2024-05-22T21:19:15.780601+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:29:19.636553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781038+0000","last_change":"2024-05-22T21:19:14.363924+0000","last_active":"2024-05-22T21:19:15.781038+0000","last_peered":"2024-05-22T21:19:15.781038+0000","last_clean":"2024-05-22T21:19:15.781038+0000","last_became_active":"2024-05-22T21:19:14.363638+0000","last_became_peered":"2024-05-22T21:19:14.363638+0000","last_unstale":"2024-05-22T21:19:15.781038+0000","last_undegraded":"2024-05-22T21:19:15.781038+0000","last_fullsized":"2024-05-22T21:19:15.781038+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:10:13.431903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430466+0000","last_change":"2024-05-22T21:19:14.359239+0000","last_active":"2024-05-22T21:19:15.430466+0000","last_peered":"2024-05-22T21:19:15.430466+0000","last_clean":"2024-05-22T21:19:15.430466+0000","last_became_active":"2024-05-22T21:19:14.359032+0000","last_became_peered":"2024-05-22T21:19:14.359032+0000","last_unstale":"2024-05-22T21:19:15.430466+0000","last_undegraded":"2024-05-22T21:19:15.430466+0000","last_fullsized":"2024-05-22T21:19:15.430466+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:03:10.351285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"124'4","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354371+0000","last_change":"2024-05-22T21:19:13.975681+0000","last_active":"2024-05-22T21:19:15.354371+0000","last_peered":"2024-05-22T21:19:15.354371+0000","last_clean":"2024-05-22T21:19:15.354371+0000","last_became_active":"2024-05-22T21:19:13.975430+0000","last_became_peered":"2024-05-22T21:19:13.975430+0000","last_unstale":"2024-05-22T21:19:15.354371+0000","last_undegraded":"2024-05-22T21:19:15.354371+0000","last_fullsized":"2024-05-22T21:19:15.354371+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:20:47.503141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780862+0000","last_change":"2024-05-22T21:19:14.366128+0000","last_active":"2024-05-22T21:19:15.780862+0000","last_peered":"2024-05-22T21:19:15.780862+0000","last_clean":"2024-05-22T21:19:15.780862+0000","last_became_active":"2024-05-22T21:19:14.365777+0000","last_became_peered":"2024-05-22T21:19:14.365777+0000","last_unstale":"2024-05-22T21:19:15.780862+0000","last_undegraded":"2024-05-22T21:19:15.780862+0000","last_fullsized":"2024-05-22T21:19:15.780862+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:14:03.575567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"124'1","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355653+0000","last_change":"2024-05-22T21:19:13.975173+0000","last_active":"2024-05-22T21:19:15.355653+0000","last_peered":"2024-05-22T21:19:15.355653+0000","last_clean":"2024-05-22T21:19:15.355653+0000","last_became_active":"2024-05-22T21:19:13.974948+0000","last_became_peered":"2024-05-22T21:19:13.974948+0000","last_unstale":"2024-05-22T21:19:15.355653+0000","last_undegraded":"2024-05-22T21:19:15.355653+0000","last_fullsized":"2024-05-22T21:19:15.355653+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:35:48.944267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780885+0000","last_change":"2024-05-22T21:19:14.359636+0000","last_active":"2024-05-22T21:19:15.780885+0000","last_peered":"2024-05-22T21:19:15.780885+0000","last_clean":"2024-05-22T21:19:15.780885+0000","last_became_active":"2024-05-22T21:19:14.356830+0000","last_became_peered":"2024-05-22T21:19:14.356830+0000","last_unstale":"2024-05-22T21:19:15.780885+0000","last_undegraded":"2024-05-22T21:19:15.780885+0000","last_fullsized":"2024-05-22T21:19:15.780885+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:44:36.454680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"124'2","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354186+0000","last_change":"2024-05-22T21:19:13.970734+0000","last_active":"2024-05-22T21:19:15.354186+0000","last_peered":"2024-05-22T21:19:15.354186+0000","last_clean":"2024-05-22T21:19:15.354186+0000","last_became_active":"2024-05-22T21:19:13.970499+0000","last_became_peered":"2024-05-22T21:19:13.970499+0000","last_unstale":"2024-05-22T21:19:15.354186+0000","last_undegraded":"2024-05-22T21:19:15.354186+0000","last_fullsized":"2024-05-22T21:19:15.354186+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:53:19.407325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356481+0000","last_change":"2024-05-22T21:19:14.364396+0000","last_active":"2024-05-22T21:19:15.356481+0000","last_peered":"2024-05-22T21:19:15.356481+0000","last_clean":"2024-05-22T21:19:15.356481+0000","last_became_active":"2024-05-22T21:19:14.364006+0000","last_became_peered":"2024-05-22T21:19:14.364006+0000","last_unstale":"2024-05-22T21:19:15.356481+0000","last_undegraded":"2024-05-22T21:19:15.356481+0000","last_fullsized":"2024-05-22T21:19:15.356481+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:20:48.510850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"124'2","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354230+0000","last_change":"2024-05-22T21:19:13.972827+0000","last_active":"2024-05-22T21:19:15.354230+0000","last_peered":"2024-05-22T21:19:15.354230+0000","last_clean":"2024-05-22T21:19:15.354230+0000","last_became_active":"2024-05-22T21:19:13.972123+0000","last_became_peered":"2024-05-22T21:19:13.972123+0000","last_unstale":"2024-05-22T21:19:15.354230+0000","last_undegraded":"2024-05-22T21:19:15.354230+0000","last_fullsized":"2024-05-22T21:19:15.354230+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:56:10.690726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781781+0000","last_change":"2024-05-22T21:19:14.363924+0000","last_active":"2024-05-22T21:19:15.781781+0000","last_peered":"2024-05-22T21:19:15.781781+0000","last_clean":"2024-05-22T21:19:15.781781+0000","last_became_active":"2024-05-22T21:19:14.363477+0000","last_became_peered":"2024-05-22T21:19:14.363477+0000","last_unstale":"2024-05-22T21:19:15.781781+0000","last_undegraded":"2024-05-22T21:19:15.781781+0000","last_fullsized":"2024-05-22T21:19:15.781781+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:34:21.879517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"124'1","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781196+0000","last_change":"2024-05-22T21:19:13.976685+0000","last_active":"2024-05-22T21:19:15.781196+0000","last_peered":"2024-05-22T21:19:15.781196+0000","last_clean":"2024-05-22T21:19:15.781196+0000","last_became_active":"2024-05-22T21:19:13.976376+0000","last_became_peered":"2024-05-22T21:19:13.976376+0000","last_unstale":"2024-05-22T21:19:15.781196+0000","last_undegraded":"2024-05-22T21:19:15.781196+0000","last_fullsized":"2024-05-22T21:19:15.781196+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:36:43.412685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355724+0000","last_change":"2024-05-22T21:19:14.361960+0000","last_active":"2024-05-22T21:19:15.355724+0000","last_peered":"2024-05-22T21:19:15.355724+0000","last_clean":"2024-05-22T21:19:15.355724+0000","last_became_active":"2024-05-22T21:19:14.360475+0000","last_became_peered":"2024-05-22T21:19:14.360475+0000","last_unstale":"2024-05-22T21:19:15.355724+0000","last_undegraded":"2024-05-22T21:19:15.355724+0000","last_fullsized":"2024-05-22T21:19:15.355724+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:33:27.771256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"124'1","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354451+0000","last_change":"2024-05-22T21:19:13.972249+0000","last_active":"2024-05-22T21:19:15.354451+0000","last_peered":"2024-05-22T21:19:15.354451+0000","last_clean":"2024-05-22T21:19:15.354451+0000","last_became_active":"2024-05-22T21:19:13.972020+0000","last_became_peered":"2024-05-22T21:19:13.972020+0000","last_unstale":"2024-05-22T21:19:15.354451+0000","last_undegraded":"2024-05-22T21:19:15.354451+0000","last_fullsized":"2024-05-22T21:19:15.354451+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:42:31.233331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781077+0000","last_change":"2024-05-22T21:19:14.362662+0000","last_active":"2024-05-22T21:19:15.781077+0000","last_peered":"2024-05-22T21:19:15.781077+0000","last_clean":"2024-05-22T21:19:15.781077+0000","last_became_active":"2024-05-22T21:19:14.362027+0000","last_became_peered":"2024-05-22T21:19:14.362027+0000","last_unstale":"2024-05-22T21:19:15.781077+0000","last_undegraded":"2024-05-22T21:19:15.781077+0000","last_fullsized":"2024-05-22T21:19:15.781077+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:20:19.111100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781907+0000","last_change":"2024-05-22T21:19:13.976799+0000","last_active":"2024-05-22T21:19:15.781907+0000","last_peered":"2024-05-22T21:19:15.781907+0000","last_clean":"2024-05-22T21:19:15.781907+0000","last_became_active":"2024-05-22T21:19:13.976325+0000","last_became_peered":"2024-05-22T21:19:13.976325+0000","last_unstale":"2024-05-22T21:19:15.781907+0000","last_undegraded":"2024-05-22T21:19:15.781907+0000","last_fullsized":"2024-05-22T21:19:15.781907+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:10:23.280341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356005+0000","last_change":"2024-05-22T21:19:14.360884+0000","last_active":"2024-05-22T21:19:15.356005+0000","last_peered":"2024-05-22T21:19:15.356005+0000","last_clean":"2024-05-22T21:19:15.356005+0000","last_became_active":"2024-05-22T21:19:14.360370+0000","last_became_peered":"2024-05-22T21:19:14.360370+0000","last_unstale":"2024-05-22T21:19:15.356005+0000","last_undegraded":"2024-05-22T21:19:15.356005+0000","last_fullsized":"2024-05-22T21:19:15.356005+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:53:20.415034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781675+0000","last_change":"2024-05-22T21:19:13.973640+0000","last_active":"2024-05-22T21:19:15.781675+0000","last_peered":"2024-05-22T21:19:15.781675+0000","last_clean":"2024-05-22T21:19:15.781675+0000","last_became_active":"2024-05-22T21:19:13.973417+0000","last_became_peered":"2024-05-22T21:19:13.973417+0000","last_unstale":"2024-05-22T21:19:15.781675+0000","last_undegraded":"2024-05-22T21:19:15.781675+0000","last_fullsized":"2024-05-22T21:19:15.781675+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:41:34.052524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354435+0000","last_change":"2024-05-22T21:19:14.364098+0000","last_active":"2024-05-22T21:19:15.354435+0000","last_peered":"2024-05-22T21:19:15.354435+0000","last_clean":"2024-05-22T21:19:15.354435+0000","last_became_active":"2024-05-22T21:19:14.363890+0000","last_became_peered":"2024-05-22T21:19:14.363890+0000","last_unstale":"2024-05-22T21:19:15.354435+0000","last_undegraded":"2024-05-22T21:19:15.354435+0000","last_fullsized":"2024-05-22T21:19:15.354435+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:57:26.931482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781019+0000","last_change":"2024-05-22T21:19:13.975036+0000","last_active":"2024-05-22T21:19:15.781019+0000","last_peered":"2024-05-22T21:19:15.781019+0000","last_clean":"2024-05-22T21:19:15.781019+0000","last_became_active":"2024-05-22T21:19:13.972123+0000","last_became_peered":"2024-05-22T21:19:13.972123+0000","last_unstale":"2024-05-22T21:19:15.781019+0000","last_undegraded":"2024-05-22T21:19:15.781019+0000","last_fullsized":"2024-05-22T21:19:15.781019+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:20:41.022086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355678+0000","last_change":"2024-05-22T21:19:14.362315+0000","last_active":"2024-05-22T21:19:15.355678+0000","last_peered":"2024-05-22T21:19:15.355678+0000","last_clean":"2024-05-22T21:19:15.355678+0000","last_became_active":"2024-05-22T21:19:14.361708+0000","last_became_peered":"2024-05-22T21:19:14.361708+0000","last_unstale":"2024-05-22T21:19:15.355678+0000","last_undegraded":"2024-05-22T21:19:15.355678+0000","last_fullsized":"2024-05-22T21:19:15.355678+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:44:54.074369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780569+0000","last_change":"2024-05-22T21:19:13.976061+0000","last_active":"2024-05-22T21:19:15.780569+0000","last_peered":"2024-05-22T21:19:15.780569+0000","last_clean":"2024-05-22T21:19:15.780569+0000","last_became_active":"2024-05-22T21:19:13.975835+0000","last_became_peered":"2024-05-22T21:19:13.975835+0000","last_unstale":"2024-05-22T21:19:15.780569+0000","last_undegraded":"2024-05-22T21:19:15.780569+0000","last_fullsized":"2024-05-22T21:19:15.780569+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:01:15.829792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781559+0000","last_change":"2024-05-22T21:19:14.365677+0000","last_active":"2024-05-22T21:19:15.781559+0000","last_peered":"2024-05-22T21:19:15.781559+0000","last_clean":"2024-05-22T21:19:15.781559+0000","last_became_active":"2024-05-22T21:19:14.365466+0000","last_became_peered":"2024-05-22T21:19:14.365466+0000","last_unstale":"2024-05-22T21:19:15.781559+0000","last_undegraded":"2024-05-22T21:19:15.781559+0000","last_fullsized":"2024-05-22T21:19:15.781559+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:14:03.575567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"124'3","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355741+0000","last_change":"2024-05-22T21:19:13.974320+0000","last_active":"2024-05-22T21:19:15.355741+0000","last_peered":"2024-05-22T21:19:15.355741+0000","last_clean":"2024-05-22T21:19:15.355741+0000","last_became_active":"2024-05-22T21:19:13.974090+0000","last_became_peered":"2024-05-22T21:19:13.974090+0000","last_unstale":"2024-05-22T21:19:15.355741+0000","last_undegraded":"2024-05-22T21:19:15.355741+0000","last_fullsized":"2024-05-22T21:19:15.355741+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:26:41.742313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355627+0000","last_change":"2024-05-22T21:19:14.358867+0000","last_active":"2024-05-22T21:19:15.355627+0000","last_peered":"2024-05-22T21:19:15.355627+0000","last_clean":"2024-05-22T21:19:15.355627+0000","last_became_active":"2024-05-22T21:19:14.358651+0000","last_became_peered":"2024-05-22T21:19:14.358651+0000","last_unstale":"2024-05-22T21:19:15.355627+0000","last_undegraded":"2024-05-22T21:19:15.355627+0000","last_fullsized":"2024-05-22T21:19:15.355627+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:27:50.460840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"124'6","reported_seq":17,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356249+0000","last_change":"2024-05-22T21:19:13.978625+0000","last_active":"2024-05-22T21:19:15.356249+0000","last_peered":"2024-05-22T21:19:15.356249+0000","last_clean":"2024-05-22T21:19:15.356249+0000","last_became_active":"2024-05-22T21:19:13.978170+0000","last_became_peered":"2024-05-22T21:19:13.978170+0000","last_unstale":"2024-05-22T21:19:15.356249+0000","last_undegraded":"2024-05-22T21:19:15.356249+0000","last_fullsized":"2024-05-22T21:19:15.356249+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:53:11.275117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780702+0000","last_change":"2024-05-22T21:19:14.357855+0000","last_active":"2024-05-22T21:19:15.780702+0000","last_peered":"2024-05-22T21:19:15.780702+0000","last_clean":"2024-05-22T21:19:15.780702+0000","last_became_active":"2024-05-22T21:19:14.357643+0000","last_became_peered":"2024-05-22T21:19:14.357643+0000","last_unstale":"2024-05-22T21:19:15.780702+0000","last_undegraded":"2024-05-22T21:19:15.780702+0000","last_fullsized":"2024-05-22T21:19:15.780702+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:38:48.877194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"124'2","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430776+0000","last_change":"2024-05-22T21:19:13.980290+0000","last_active":"2024-05-22T21:19:15.430776+0000","last_peered":"2024-05-22T21:19:15.430776+0000","last_clean":"2024-05-22T21:19:15.430776+0000","last_became_active":"2024-05-22T21:19:13.979862+0000","last_became_peered":"2024-05-22T21:19:13.979862+0000","last_unstale":"2024-05-22T21:19:15.430776+0000","last_undegraded":"2024-05-22T21:19:15.430776+0000","last_fullsized":"2024-05-22T21:19:15.430776+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:20:57.410269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781590+0000","last_change":"2024-05-22T21:19:14.364846+0000","last_active":"2024-05-22T21:19:15.781590+0000","last_peered":"2024-05-22T21:19:15.781590+0000","last_clean":"2024-05-22T21:19:15.781590+0000","last_became_active":"2024-05-22T21:19:14.359728+0000","last_became_peered":"2024-05-22T21:19:14.359728+0000","last_unstale":"2024-05-22T21:19:15.781590+0000","last_undegraded":"2024-05-22T21:19:15.781590+0000","last_fullsized":"2024-05-22T21:19:15.781590+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:31:36.605047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430550+0000","last_change":"2024-05-22T21:19:13.979649+0000","last_active":"2024-05-22T21:19:15.430550+0000","last_peered":"2024-05-22T21:19:15.430550+0000","last_clean":"2024-05-22T21:19:15.430550+0000","last_became_active":"2024-05-22T21:19:13.979186+0000","last_became_peered":"2024-05-22T21:19:13.979186+0000","last_unstale":"2024-05-22T21:19:15.430550+0000","last_undegraded":"2024-05-22T21:19:15.430550+0000","last_fullsized":"2024-05-22T21:19:15.430550+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:36:04.543841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356070+0000","last_change":"2024-05-22T21:19:14.365472+0000","last_active":"2024-05-22T21:19:15.356070+0000","last_peered":"2024-05-22T21:19:15.356070+0000","last_clean":"2024-05-22T21:19:15.356070+0000","last_became_active":"2024-05-22T21:19:14.365098+0000","last_became_peered":"2024-05-22T21:19:14.365098+0000","last_unstale":"2024-05-22T21:19:15.356070+0000","last_undegraded":"2024-05-22T21:19:15.356070+0000","last_fullsized":"2024-05-22T21:19:15.356070+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:26:42.750022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356458+0000","last_change":"2024-05-22T21:19:13.975623+0000","last_active":"2024-05-22T21:19:15.356458+0000","last_peered":"2024-05-22T21:19:15.356458+0000","last_clean":"2024-05-22T21:19:15.356458+0000","last_became_active":"2024-05-22T21:19:13.975118+0000","last_became_peered":"2024-05-22T21:19:13.975118+0000","last_unstale":"2024-05-22T21:19:15.356458+0000","last_undegraded":"2024-05-22T21:19:15.356458+0000","last_fullsized":"2024-05-22T21:19:15.356458+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:26:26.886392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430466+0000","last_change":"2024-05-22T21:19:14.362154+0000","last_active":"2024-05-22T21:19:15.430466+0000","last_peered":"2024-05-22T21:19:15.430466+0000","last_clean":"2024-05-22T21:19:15.430466+0000","last_became_active":"2024-05-22T21:19:14.361441+0000","last_became_peered":"2024-05-22T21:19:14.361441+0000","last_unstale":"2024-05-22T21:19:15.430466+0000","last_undegraded":"2024-05-22T21:19:15.430466+0000","last_fullsized":"2024-05-22T21:19:15.430466+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:24:15.256014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"124'8","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781005+0000","last_change":"2024-05-22T21:19:13.972217+0000","last_active":"2024-05-22T21:19:15.781005+0000","last_peered":"2024-05-22T21:19:15.781005+0000","last_clean":"2024-05-22T21:19:15.781005+0000","last_became_active":"2024-05-22T21:19:13.971997+0000","last_became_peered":"2024-05-22T21:19:13.971997+0000","last_unstale":"2024-05-22T21:19:15.781005+0000","last_undegraded":"2024-05-22T21:19:15.781005+0000","last_fullsized":"2024-05-22T21:19:15.781005+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:44:38.862169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781683+0000","last_change":"2024-05-22T21:19:14.364973+0000","last_active":"2024-05-22T21:19:15.781683+0000","last_peered":"2024-05-22T21:19:15.781683+0000","last_clean":"2024-05-22T21:19:15.781683+0000","last_became_active":"2024-05-22T21:19:14.364704+0000","last_became_peered":"2024-05-22T21:19:14.364704+0000","last_unstale":"2024-05-22T21:19:15.781683+0000","last_undegraded":"2024-05-22T21:19:15.781683+0000","last_fullsized":"2024-05-22T21:19:15.781683+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:27:50.460840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"124'2","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356564+0000","last_change":"2024-05-22T21:19:13.979237+0000","last_active":"2024-05-22T21:19:15.356564+0000","last_peered":"2024-05-22T21:19:15.356564+0000","last_clean":"2024-05-22T21:19:15.356564+0000","last_became_active":"2024-05-22T21:19:13.979022+0000","last_became_peered":"2024-05-22T21:19:13.979022+0000","last_unstale":"2024-05-22T21:19:15.356564+0000","last_undegraded":"2024-05-22T21:19:15.356564+0000","last_fullsized":"2024-05-22T21:19:15.356564+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:48:35.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781680+0000","last_change":"2024-05-22T21:19:14.360706+0000","last_active":"2024-05-22T21:19:15.781680+0000","last_peered":"2024-05-22T21:19:15.781680+0000","last_clean":"2024-05-22T21:19:15.781680+0000","last_became_active":"2024-05-22T21:19:14.360488+0000","last_became_peered":"2024-05-22T21:19:14.360488+0000","last_unstale":"2024-05-22T21:19:15.781680+0000","last_undegraded":"2024-05-22T21:19:15.781680+0000","last_fullsized":"2024-05-22T21:19:15.781680+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:46:40.734526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"124'1","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.782119+0000","last_change":"2024-05-22T21:19:13.975276+0000","last_active":"2024-05-22T21:19:15.782119+0000","last_peered":"2024-05-22T21:19:15.782119+0000","last_clean":"2024-05-22T21:19:15.782119+0000","last_became_active":"2024-05-22T21:19:13.974985+0000","last_became_peered":"2024-05-22T21:19:13.974985+0000","last_unstale":"2024-05-22T21:19:15.782119+0000","last_undegraded":"2024-05-22T21:19:15.782119+0000","last_fullsized":"2024-05-22T21:19:15.782119+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:46:39.726817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356207+0000","last_change":"2024-05-22T21:19:14.357206+0000","last_active":"2024-05-22T21:19:15.356207+0000","last_peered":"2024-05-22T21:19:15.356207+0000","last_clean":"2024-05-22T21:19:15.356207+0000","last_became_active":"2024-05-22T21:19:14.356993+0000","last_became_peered":"2024-05-22T21:19:14.356993+0000","last_unstale":"2024-05-22T21:19:15.356207+0000","last_undegraded":"2024-05-22T21:19:15.356207+0000","last_fullsized":"2024-05-22T21:19:15.356207+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:08:05.273893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354511+0000","last_change":"2024-05-22T21:19:13.974166+0000","last_active":"2024-05-22T21:19:15.354511+0000","last_peered":"2024-05-22T21:19:15.354511+0000","last_clean":"2024-05-22T21:19:15.354511+0000","last_became_active":"2024-05-22T21:19:13.972043+0000","last_became_peered":"2024-05-22T21:19:13.972043+0000","last_unstale":"2024-05-22T21:19:15.354511+0000","last_undegraded":"2024-05-22T21:19:15.354511+0000","last_fullsized":"2024-05-22T21:19:15.354511+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:26:41.742313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430517+0000","last_change":"2024-05-22T21:19:14.370571+0000","last_active":"2024-05-22T21:19:15.430517+0000","last_peered":"2024-05-22T21:19:15.430517+0000","last_clean":"2024-05-22T21:19:15.430517+0000","last_became_active":"2024-05-22T21:19:14.370108+0000","last_became_peered":"2024-05-22T21:19:14.370108+0000","last_unstale":"2024-05-22T21:19:15.430517+0000","last_undegraded":"2024-05-22T21:19:15.430517+0000","last_fullsized":"2024-05-22T21:19:15.430517+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:05:32.058773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"124'2","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354531+0000","last_change":"2024-05-22T21:19:13.969131+0000","last_active":"2024-05-22T21:19:15.354531+0000","last_peered":"2024-05-22T21:19:15.354531+0000","last_clean":"2024-05-22T21:19:15.354531+0000","last_became_active":"2024-05-22T21:19:13.968790+0000","last_became_peered":"2024-05-22T21:19:13.968790+0000","last_unstale":"2024-05-22T21:19:15.354531+0000","last_undegraded":"2024-05-22T21:19:15.354531+0000","last_fullsized":"2024-05-22T21:19:15.354531+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:15:58.666861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430301+0000","last_change":"2024-05-22T21:19:14.363202+0000","last_active":"2024-05-22T21:19:15.430301+0000","last_peered":"2024-05-22T21:19:15.430301+0000","last_clean":"2024-05-22T21:19:15.430301+0000","last_became_active":"2024-05-22T21:19:14.363044+0000","last_became_peered":"2024-05-22T21:19:14.363044+0000","last_unstale":"2024-05-22T21:19:15.430301+0000","last_undegraded":"2024-05-22T21:19:15.430301+0000","last_fullsized":"2024-05-22T21:19:15.430301+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:41:35.060233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781468+0000","last_change":"2024-05-22T21:19:13.978007+0000","last_active":"2024-05-22T21:19:15.781468+0000","last_peered":"2024-05-22T21:19:15.781468+0000","last_clean":"2024-05-22T21:19:15.781468+0000","last_became_active":"2024-05-22T21:19:13.977780+0000","last_became_peered":"2024-05-22T21:19:13.977780+0000","last_unstale":"2024-05-22T21:19:15.781468+0000","last_undegraded":"2024-05-22T21:19:15.781468+0000","last_fullsized":"2024-05-22T21:19:15.781468+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:26:26.886392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430466+0000","last_change":"2024-05-22T21:19:14.365248+0000","last_active":"2024-05-22T21:19:15.430466+0000","last_peered":"2024-05-22T21:19:15.430466+0000","last_clean":"2024-05-22T21:19:15.430466+0000","last_became_active":"2024-05-22T21:19:14.364991+0000","last_became_peered":"2024-05-22T21:19:14.364991+0000","last_unstale":"2024-05-22T21:19:15.430466+0000","last_undegraded":"2024-05-22T21:19:15.430466+0000","last_fullsized":"2024-05-22T21:19:15.430466+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:31:36.498469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781830+0000","last_change":"2024-05-22T21:19:13.976779+0000","last_active":"2024-05-22T21:19:15.781830+0000","last_peered":"2024-05-22T21:19:15.781830+0000","last_clean":"2024-05-22T21:19:15.781830+0000","last_became_active":"2024-05-22T21:19:13.976541+0000","last_became_peered":"2024-05-22T21:19:13.976541+0000","last_unstale":"2024-05-22T21:19:15.781830+0000","last_undegraded":"2024-05-22T21:19:15.781830+0000","last_fullsized":"2024-05-22T21:19:15.781830+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:53:11.275117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.782435+0000","last_change":"2024-05-22T21:19:14.360039+0000","last_active":"2024-05-22T21:19:15.782435+0000","last_peered":"2024-05-22T21:19:15.782435+0000","last_clean":"2024-05-22T21:19:15.782435+0000","last_became_active":"2024-05-22T21:19:14.359393+0000","last_became_peered":"2024-05-22T21:19:14.359393+0000","last_unstale":"2024-05-22T21:19:15.782435+0000","last_undegraded":"2024-05-22T21:19:15.782435+0000","last_fullsized":"2024-05-22T21:19:15.782435+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:08:05.273893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354299+0000","last_change":"2024-05-22T21:19:13.967940+0000","last_active":"2024-05-22T21:19:15.354299+0000","last_peered":"2024-05-22T21:19:15.354299+0000","last_clean":"2024-05-22T21:19:15.354299+0000","last_became_active":"2024-05-22T21:19:13.967580+0000","last_became_peered":"2024-05-22T21:19:13.967580+0000","last_unstale":"2024-05-22T21:19:15.354299+0000","last_undegraded":"2024-05-22T21:19:15.354299+0000","last_fullsized":"2024-05-22T21:19:15.354299+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:44:53.066660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354217+0000","last_change":"2024-05-22T21:19:14.368425+0000","last_active":"2024-05-22T21:19:15.354217+0000","last_peered":"2024-05-22T21:19:15.354217+0000","last_clean":"2024-05-22T21:19:15.354217+0000","last_became_active":"2024-05-22T21:19:14.367534+0000","last_became_peered":"2024-05-22T21:19:14.367534+0000","last_unstale":"2024-05-22T21:19:15.354217+0000","last_undegraded":"2024-05-22T21:19:15.354217+0000","last_fullsized":"2024-05-22T21:19:15.354217+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:09:35.979308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"124'2","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781793+0000","last_change":"2024-05-22T21:19:13.975255+0000","last_active":"2024-05-22T21:19:15.781793+0000","last_peered":"2024-05-22T21:19:15.781793+0000","last_clean":"2024-05-22T21:19:15.781793+0000","last_became_active":"2024-05-22T21:19:13.974968+0000","last_became_peered":"2024-05-22T21:19:13.974968+0000","last_unstale":"2024-05-22T21:19:15.781793+0000","last_undegraded":"2024-05-22T21:19:15.781793+0000","last_fullsized":"2024-05-22T21:19:15.781793+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:48:35.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781119+0000","last_change":"2024-05-22T21:19:14.358327+0000","last_active":"2024-05-22T21:19:15.781119+0000","last_peered":"2024-05-22T21:19:15.781119+0000","last_clean":"2024-05-22T21:19:15.781119+0000","last_became_active":"2024-05-22T21:19:14.358034+0000","last_became_peered":"2024-05-22T21:19:14.358034+0000","last_unstale":"2024-05-22T21:19:15.781119+0000","last_undegraded":"2024-05-22T21:19:15.781119+0000","last_fullsized":"2024-05-22T21:19:15.781119+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:54:16.959652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354603+0000","last_change":"2024-05-22T21:19:13.968982+0000","last_active":"2024-05-22T21:19:15.354603+0000","last_peered":"2024-05-22T21:19:15.354603+0000","last_clean":"2024-05-22T21:19:15.354603+0000","last_became_active":"2024-05-22T21:19:13.968482+0000","last_became_peered":"2024-05-22T21:19:13.968482+0000","last_unstale":"2024-05-22T21:19:15.354603+0000","last_undegraded":"2024-05-22T21:19:15.354603+0000","last_fullsized":"2024-05-22T21:19:15.354603+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:58:26.400264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781634+0000","last_change":"2024-05-22T21:19:14.364686+0000","last_active":"2024-05-22T21:19:15.781634+0000","last_peered":"2024-05-22T21:19:15.781634+0000","last_clean":"2024-05-22T21:19:15.781634+0000","last_became_active":"2024-05-22T21:19:14.359433+0000","last_became_peered":"2024-05-22T21:19:14.359433+0000","last_unstale":"2024-05-22T21:19:15.781634+0000","last_undegraded":"2024-05-22T21:19:15.781634+0000","last_fullsized":"2024-05-22T21:19:15.781634+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:25:31.495943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354087+0000","last_change":"2024-05-22T21:19:13.969743+0000","last_active":"2024-05-22T21:19:15.354087+0000","last_peered":"2024-05-22T21:19:15.354087+0000","last_clean":"2024-05-22T21:19:15.354087+0000","last_became_active":"2024-05-22T21:19:13.969112+0000","last_became_peered":"2024-05-22T21:19:13.969112+0000","last_unstale":"2024-05-22T21:19:15.354087+0000","last_undegraded":"2024-05-22T21:19:15.354087+0000","last_fullsized":"2024-05-22T21:19:15.354087+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:35:48.944267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781790+0000","last_change":"2024-05-22T21:19:14.367850+0000","last_active":"2024-05-22T21:19:15.781790+0000","last_peered":"2024-05-22T21:19:15.781790+0000","last_clean":"2024-05-22T21:19:15.781790+0000","last_became_active":"2024-05-22T21:19:14.367360+0000","last_became_peered":"2024-05-22T21:19:14.367360+0000","last_unstale":"2024-05-22T21:19:15.781790+0000","last_undegraded":"2024-05-22T21:19:15.781790+0000","last_fullsized":"2024-05-22T21:19:15.781790+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:38:48.877194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.782525+0000","last_change":"2024-05-22T21:19:13.977791+0000","last_active":"2024-05-22T21:19:15.782525+0000","last_peered":"2024-05-22T21:19:15.782525+0000","last_clean":"2024-05-22T21:19:15.782525+0000","last_became_active":"2024-05-22T21:19:13.977568+0000","last_became_peered":"2024-05-22T21:19:13.977568+0000","last_unstale":"2024-05-22T21:19:15.782525+0000","last_undegraded":"2024-05-22T21:19:15.782525+0000","last_fullsized":"2024-05-22T21:19:15.782525+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:41:38.428159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430555+0000","last_change":"2024-05-22T21:19:14.362859+0000","last_active":"2024-05-22T21:19:15.430555+0000","last_peered":"2024-05-22T21:19:15.430555+0000","last_clean":"2024-05-22T21:19:15.430555+0000","last_became_active":"2024-05-22T21:19:14.361771+0000","last_became_peered":"2024-05-22T21:19:14.361771+0000","last_unstale":"2024-05-22T21:19:15.430555+0000","last_undegraded":"2024-05-22T21:19:15.430555+0000","last_fullsized":"2024-05-22T21:19:15.430555+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:37.869461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354355+0000","last_change":"2024-05-22T21:19:13.972676+0000","last_active":"2024-05-22T21:19:15.354355+0000","last_peered":"2024-05-22T21:19:15.354355+0000","last_clean":"2024-05-22T21:19:15.354355+0000","last_became_active":"2024-05-22T21:19:13.971741+0000","last_became_peered":"2024-05-22T21:19:13.971741+0000","last_unstale":"2024-05-22T21:19:15.354355+0000","last_undegraded":"2024-05-22T21:19:15.354355+0000","last_fullsized":"2024-05-22T21:19:15.354355+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:01:15.829792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355772+0000","last_change":"2024-05-22T21:19:14.362534+0000","last_active":"2024-05-22T21:19:15.355772+0000","last_peered":"2024-05-22T21:19:15.355772+0000","last_clean":"2024-05-22T21:19:15.355772+0000","last_became_active":"2024-05-22T21:19:14.362088+0000","last_became_peered":"2024-05-22T21:19:14.362088+0000","last_unstale":"2024-05-22T21:19:15.355772+0000","last_undegraded":"2024-05-22T21:19:15.355772+0000","last_fullsized":"2024-05-22T21:19:15.355772+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:15:59.674570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781913+0000","last_change":"2024-05-22T21:19:13.978831+0000","last_active":"2024-05-22T21:19:15.781913+0000","last_peered":"2024-05-22T21:19:15.781913+0000","last_clean":"2024-05-22T21:19:15.781913+0000","last_became_active":"2024-05-22T21:19:13.978462+0000","last_became_peered":"2024-05-22T21:19:13.978462+0000","last_unstale":"2024-05-22T21:19:15.781913+0000","last_undegraded":"2024-05-22T21:19:15.781913+0000","last_fullsized":"2024-05-22T21:19:15.781913+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:48:53.288659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430565+0000","last_change":"2024-05-22T21:19:14.366194+0000","last_active":"2024-05-22T21:19:15.430565+0000","last_peered":"2024-05-22T21:19:15.430565+0000","last_clean":"2024-05-22T21:19:15.430565+0000","last_became_active":"2024-05-22T21:19:14.365009+0000","last_became_peered":"2024-05-22T21:19:14.365009+0000","last_unstale":"2024-05-22T21:19:15.430565+0000","last_undegraded":"2024-05-22T21:19:15.430565+0000","last_fullsized":"2024-05-22T21:19:15.430565+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:43:11.548823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780914+0000","last_change":"2024-05-22T21:19:13.974434+0000","last_active":"2024-05-22T21:19:15.780914+0000","last_peered":"2024-05-22T21:19:15.780914+0000","last_clean":"2024-05-22T21:19:15.780914+0000","last_became_active":"2024-05-22T21:19:13.973566+0000","last_became_peered":"2024-05-22T21:19:13.973566+0000","last_unstale":"2024-05-22T21:19:15.780914+0000","last_undegraded":"2024-05-22T21:19:15.780914+0000","last_fullsized":"2024-05-22T21:19:15.780914+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:58:26.400264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781731+0000","last_change":"2024-05-22T21:19:14.367984+0000","last_active":"2024-05-22T21:19:15.781731+0000","last_peered":"2024-05-22T21:19:15.781731+0000","last_clean":"2024-05-22T21:19:15.781731+0000","last_became_active":"2024-05-22T21:19:14.367393+0000","last_became_peered":"2024-05-22T21:19:14.367393+0000","last_unstale":"2024-05-22T21:19:15.781731+0000","last_undegraded":"2024-05-22T21:19:15.781731+0000","last_fullsized":"2024-05-22T21:19:15.781731+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:29:19.636553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430614+0000","last_change":"2024-05-22T21:19:13.979249+0000","last_active":"2024-05-22T21:19:15.430614+0000","last_peered":"2024-05-22T21:19:15.430614+0000","last_clean":"2024-05-22T21:19:15.430614+0000","last_became_active":"2024-05-22T21:19:13.978885+0000","last_became_peered":"2024-05-22T21:19:13.978885+0000","last_unstale":"2024-05-22T21:19:15.430614+0000","last_undegraded":"2024-05-22T21:19:15.430614+0000","last_fullsized":"2024-05-22T21:19:15.430614+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:24:21.746105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430492+0000","last_change":"2024-05-22T21:19:14.370527+0000","last_active":"2024-05-22T21:19:15.430492+0000","last_peered":"2024-05-22T21:19:15.430492+0000","last_clean":"2024-05-22T21:19:15.430492+0000","last_became_active":"2024-05-22T21:19:14.370301+0000","last_became_peered":"2024-05-22T21:19:14.370301+0000","last_unstale":"2024-05-22T21:19:15.430492+0000","last_undegraded":"2024-05-22T21:19:15.430492+0000","last_fullsized":"2024-05-22T21:19:15.430492+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:31:00.630803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"124'1","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355757+0000","last_change":"2024-05-22T21:19:13.976882+0000","last_active":"2024-05-22T21:19:15.355757+0000","last_peered":"2024-05-22T21:19:15.355757+0000","last_clean":"2024-05-22T21:19:15.355757+0000","last_became_active":"2024-05-22T21:19:13.976502+0000","last_became_peered":"2024-05-22T21:19:13.976502+0000","last_unstale":"2024-05-22T21:19:15.355757+0000","last_undegraded":"2024-05-22T21:19:15.355757+0000","last_fullsized":"2024-05-22T21:19:15.355757+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:58:26.400264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":271,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355849+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:19:15.355849+0000","last_peered":"2024-05-22T21:19:15.355849+0000","last_clean":"2024-05-22T21:19:15.355849+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:19:15.355849+0000","last_undegraded":"2024-05-22T21:19:15.355849+0000","last_fullsized":"2024-05-22T21:19:15.355849+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1250,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194062,"statfs":{"total":200000000,"available":198719875,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70299999999999996,"15min":0.754},"min":{"1min":0.40000000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93700000000000006,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88300000000000001,"5min":1.889,"15min":1.889},"last":0.81799999999999995}]},{"osd":1,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72699999999999998,"15min":0.77000000000000002},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89500000000000002,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71399999999999997,"15min":0.76000000000000001},"min":{"1min":0.47899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88600000000000001,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.77600000000000002}]},{"osd":2,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69299999999999995,"15min":0.69999999999999996},"min":{"1min":0.503,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81899999999999995,"5min":1.923,"15min":1.923},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.74399999999999999,"15min":0.72699999999999998},"min":{"1min":0.47399999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86099999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.83699999999999997}]},{"osd":3,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.69599999999999995},"min":{"1min":0.53900000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.90600000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.752,"15min":0.79100000000000004},"min":{"1min":0.505,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88600000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.65700000000000003}]},{"osd":4,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.437,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.92000000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.621,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88700000000000001,"5min":1.018,"15min":1.018},"last":0.90900000000000003}]},{"osd":5,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63900000000000001,"15min":0.65500000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84899999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64800000000000002,"15min":0.64200000000000002},"min":{"1min":0.47899999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91000000000000003,"5min":0.996,"15min":0.996},"last":0.70499999999999996}]},{"osd":6,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.71199999999999997},"min":{"1min":0.54700000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85799999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67400000000000004,"15min":0.64400000000000002},"min":{"1min":0.504,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.95299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":674,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194638,"statfs":{"total":200000000,"available":199309703,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.49299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70599999999999996,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.58799999999999997}]},{"osd":1,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95899999999999996,"5min":1.107,"15min":1.107},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.034,"5min":1.081,"15min":1.081},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78300000000000003,"15min":0.75900000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.746,"15min":0.73899999999999999},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.748,"15min":0.74199999999999999},"min":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0249999999999999,"5min":1.415,"15min":1.415},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.004,"5min":1.788,"15min":1.788},"last":0.60399999999999998}]},{"osd":4,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.41699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.89900000000000002,"5min":1.006,"15min":1.006},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95899999999999996,"5min":2.718,"15min":2.718},"last":0.44400000000000001}]},{"osd":5,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.745,"15min":0.70299999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81799999999999995,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.45500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.42099999999999999}]},{"osd":7,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70999999999999996,"15min":0.73799999999999999},"min":{"1min":0.53600000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0469999999999999,"5min":1.095,"15min":1.095},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72699999999999998,"15min":0.75700000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92500000000000004,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.65100000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":673,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194638,"statfs":{"total":200000000,"available":199310285,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68999999999999995,"15min":0.70399999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.91000000000000003,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69699999999999995,"15min":0.67500000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.96499999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.81899999999999995}]},{"osd":1,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.46500000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.028,"5min":1.216,"15min":1.216},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.44900000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.875,"5min":1.048,"15min":1.048},"last":0.76200000000000001}]},{"osd":2,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.71899999999999997,"15min":0.69899999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91100000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.69799999999999995,"15min":0.67600000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86699999999999999,"5min":1.252,"15min":1.252},"last":0.86099999999999999}]},{"osd":3,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.52800000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91500000000000004,"5min":1.111,"15min":1.111},"last":0.79900000000000004}]},{"osd":4,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.627},"min":{"1min":0.38400000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88800000000000001,"5min":1.272,"15min":1.272},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.629,"15min":0.63200000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.014,"5min":1.038,"15min":1.038},"last":0.57899999999999996}]},{"osd":6,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.63},"min":{"1min":0.41299999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.877,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.442,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.68000000000000005}]},{"osd":7,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93899999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.75900000000000001,"15min":0.79000000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.99399999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.61599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":675,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194637,"statfs":{"total":200000000,"available":199308667,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78200000000000003,"15min":0.748},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.89300000000000002,"5min":6.6539999999999999,"15min":6.6539999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.747,"15min":0.72799999999999998},"min":{"1min":0.49299999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93300000000000005,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":0.84899999999999998}]},{"osd":1,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68200000000000005,"15min":0.71599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88200000000000001,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92200000000000004,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.77300000000000002}]},{"osd":2,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70199999999999996,"15min":0.68899999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0940000000000001,"5min":1.335,"15min":1.335},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70099999999999996,"15min":0.70399999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95099999999999996,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.54100000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.98999999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.78900000000000003},"min":{"1min":0.53200000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96799999999999997,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.97699999999999998}]},{"osd":5,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.39400000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.93500000000000005,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91900000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.0409999999999999}]},{"osd":6,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65900000000000003,"15min":0.69299999999999995},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91600000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.51800000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.91100000000000003,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.91600000000000004}]},{"osd":7,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.69599999999999995,"15min":0.69999999999999996},"min":{"1min":0.48099999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98199999999999998,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.74099999999999999,"15min":0.78000000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97999999999999998,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.1200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194643,"statfs":{"total":200000000,"available":199315069,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.42999999999999999,"15min":0.40699999999999997},"min":{"1min":0.29699999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.71399999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.442,"15min":0.40799999999999997},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.49199999999999999}]},{"osd":1,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46000000000000002,"15min":0.44800000000000001},"min":{"1min":0.28399999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.624,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.625},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71799999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.68600000000000005}]},{"osd":2,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51000000000000001,"15min":0.496},"min":{"1min":0.376,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68999999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51100000000000001,"15min":0.48999999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69899999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60099999999999998}]},{"osd":4,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.74099999999999999,"15min":0.70599999999999996},"min":{"1min":0.50900000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0209999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.749,"15min":0.71899999999999997},"min":{"1min":0.57599999999999996,"5min":0.435,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1}]},{"osd":5,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.75800000000000001,"15min":0.69899999999999995},"min":{"1min":0.622,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.002,"5min":1.234,"15min":1.234},"last":1.048},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.70299999999999996},"min":{"1min":0.53500000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0449999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.93799999999999994}]},{"osd":6,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.71599999999999997},"min":{"1min":0.48899999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97499999999999998,"5min":1.276,"15min":1.276},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75800000000000001,"15min":0.70799999999999996},"min":{"1min":0.54200000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0760000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.0820000000000001}]},{"osd":7,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.75800000000000001},"min":{"1min":0.52800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.0900000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78500000000000003,"15min":0.74099999999999999},"min":{"1min":0.63300000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0649999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.071}]}]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":673,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194638,"statfs":{"total":200000000,"available":199310321,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.59399999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.504},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.46300000000000002,"15min":0.443},"min":{"1min":0.33200000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.56399999999999995}]},{"osd":1,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.32900000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.624,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54300000000000004,"15min":0.52100000000000002},"min":{"1min":0.442,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.60599999999999998}]},{"osd":3,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83099999999999996,"5min":0.871,"15min":0.871},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.47399999999999998,"15min":0.45900000000000002},"min":{"1min":0.34699999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.68000000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73399999999999999,"15min":0.68999999999999995},"min":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95599999999999996,"5min":1.095,"15min":1.095},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77900000000000003,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.79000000000000004}]},{"osd":5,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.73299999999999998,"15min":0.70699999999999996},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.746},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72599999999999998,"15min":0.70199999999999996},"min":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92900000000000005,"5min":1.127,"15min":1.127},"last":0.80700000000000005}]},{"osd":6,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73099999999999998,"15min":0.70299999999999996},"min":{"1min":0.52000000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.02,"5min":1.054,"15min":1.054},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69899999999999995,"15min":0.67800000000000005},"min":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.89600000000000002,"5min":1.002,"15min":1.002},"last":0.70699999999999996}]},{"osd":7,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.76800000000000002,"15min":0.73199999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.77200000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0249999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.89300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":667,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194645,"statfs":{"total":200000000,"available":199316655,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41599999999999998,"15min":0.42999999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58999999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.46400000000000002,"15min":0.438},"min":{"1min":0.35799999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.67800000000000005,"5min":0.876,"15min":0.876},"last":0.372}]},{"osd":2,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.36099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80500000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.495,"15min":0.53100000000000003},"min":{"1min":0.35199999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64100000000000001,"5min":1.923,"15min":1.923},"last":0.51100000000000001}]},{"osd":3,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53600000000000003,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.497,"15min":0.48899999999999999},"min":{"1min":0.34699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.62,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44400000000000001}]},{"osd":4,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.75900000000000001},"min":{"1min":0.496,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.109,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.72299999999999998,"15min":0.68899999999999995},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0389999999999999,"5min":1.3,"15min":1.3},"last":0.52500000000000002}]},{"osd":5,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.76700000000000002,"15min":0.73599999999999999},"min":{"1min":0.59499999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0169999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.751,"15min":0.753},"min":{"1min":0.621,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97099999999999997,"5min":1.97,"15min":1.97},"last":0.54000000000000004}]},{"osd":6,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.748},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0009999999999999,"5min":1.272,"15min":1.272},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78600000000000003,"15min":0.75},"min":{"1min":0.57899999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0620000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.61599999999999999}]},{"osd":7,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.76400000000000001,"15min":0.77600000000000002},"min":{"1min":0.63300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98799999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.625},{"interface":"front","average":{"1min":0.754,"5min":0.75600000000000001,"15min":0.77600000000000002},"min":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98899999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.67100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1249,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194062,"statfs":{"total":200000000,"available":198720024,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42499999999999999,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58099999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.51100000000000001,"15min":0.48499999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69199999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48899999999999999}]},{"osd":2,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.45400000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76200000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.42799999999999999},"min":{"1min":0.29399999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.623,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.442}]},{"osd":3,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45600000000000002,"15min":0.434},"min":{"1min":0.33900000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61699999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53400000000000003,"15min":0.503},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.746,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.54600000000000004}]},{"osd":4,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.71099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1020000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.57099999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.113,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.86299999999999999}]},{"osd":5,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71899999999999997,"15min":0.70699999999999996},"min":{"1min":0.54200000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99299999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.76900000000000002,"15min":0.73899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.143,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.874}]},{"osd":6,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79300000000000004,"15min":0.77300000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1240000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78200000000000003,"15min":0.77200000000000002},"min":{"1min":0.55300000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2090000000000001,"5min":1.5,"15min":1.5},"last":0.84399999999999997}]},{"osd":7,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.75600000000000001,"15min":0.72699999999999998},"min":{"1min":0.57499999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76900000000000002,"15min":0.75900000000000001},"min":{"1min":0.55900000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.0589999999999999,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.72399999999999998}]}]}],"pool_statfs":[]}} 2024-05-22T21:19:18.462 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:19:18.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.595+0000 7f2d6b400640 1 Processor -- start 2024-05-22T21:19:18.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.595+0000 7f2d6b400640 1 -- start start 2024-05-22T21:19:18.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.595+0000 7f2d6b400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2d6415d430 0x7f2d640707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.595+0000 7f2d6b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2d640ac8b0 0x7f2d6406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.595+0000 7f2d6b400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d64071fd0 con 0x7f2d640accf0 2024-05-22T21:19:18.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.595+0000 7f2d6b400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d64070e80 con 0x7f2d640ac8b0 2024-05-22T21:19:18.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.595+0000 7f2d6b400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6406c510 con 0x7f2d6415d430 2024-05-22T21:19:18.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.595+0000 7f2d63e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2d640ac8b0 0x7f2d6406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.595+0000 7f2d63e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2d640ac8b0 0x7f2d6406cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44706/0 (socket says 172.21.15.161:44706) 2024-05-22T21:19:18.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.595+0000 7f2d63e00640 1 -- 172.21.15.161:0/3879145543 learned_addr learned my addr 172.21.15.161:0/3879145543 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:18.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.595+0000 7f2d68a00640 1 --2- 172.21.15.161:0/3879145543 >> v2:172.21.15.179:3300/0 conn(0x7f2d6415d430 0x7f2d640707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.595+0000 7f2d63e00640 1 -- 172.21.15.161:0/3879145543 >> v2:172.21.15.179:3300/0 conn(0x7f2d6415d430 msgr2=0x7f2d640707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:18.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.595+0000 7f2d63e00640 1 --2- 172.21.15.161:0/3879145543 >> v2:172.21.15.179:3300/0 conn(0x7f2d6415d430 0x7f2d640707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.595+0000 7f2d63e00640 1 -- 172.21.15.161:0/3879145543 >> v1:172.21.15.179:6789/0 conn(0x7f2d640accf0 legacy=0x7f2d640b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:18.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.595+0000 7f2d63e00640 1 -- 172.21.15.161:0/3879145543 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6406c7f0 con 0x7f2d640ac8b0 2024-05-22T21:19:18.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.595+0000 7f2d68a00640 1 --2- 172.21.15.161:0/3879145543 >> v2:172.21.15.179:3300/0 conn(0x7f2d6415d430 0x7f2d640707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:18.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.599+0000 7f2d63e00640 1 --2- 172.21.15.161:0/3879145543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d640ac8b0 0x7f2d6406cff0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f2d640aea00 tx=0x7f2d5401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce0d8f2e8036e84 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:18.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.599+0000 7f2d63400640 1 -- 172.21.15.161:0/3879145543 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d540ad680 con 0x7f2d640ac8b0 2024-05-22T21:19:18.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.599+0000 7f2d63400640 1 -- 172.21.15.161:0/3879145543 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d540ad7e0 con 0x7f2d640ac8b0 2024-05-22T21:19:18.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.599+0000 7f2d63400640 1 -- 172.21.15.161:0/3879145543 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d540adb70 con 0x7f2d640ac8b0 2024-05-22T21:19:18.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.599+0000 7f2d6b400640 1 -- 172.21.15.161:0/3879145543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d640ac8b0 msgr2=0x7f2d6406cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.599+0000 7f2d6b400640 1 --2- 172.21.15.161:0/3879145543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d640ac8b0 0x7f2d6406cff0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f2d640aea00 tx=0x7f2d5401c420 comp rx=0 tx=0).stop 2024-05-22T21:19:18.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.599+0000 7f2d6b400640 1 -- 172.21.15.161:0/3879145543 shutdown_connections 2024-05-22T21:19:18.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.599+0000 7f2d6b400640 1 --2- 172.21.15.161:0/3879145543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d640ac8b0 0x7f2d6406cff0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.599+0000 7f2d6b400640 1 --2- 172.21.15.161:0/3879145543 >> v2:172.21.15.179:3300/0 conn(0x7f2d6415d430 0x7f2d640707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.599+0000 7f2d6b400640 1 -- 172.21.15.161:0/3879145543 >> 172.21.15.161:0/3879145543 conn(0x7f2d640b0bc0 msgr2=0x7f2d640b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:18.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.599+0000 7f2d6b400640 1 -- 172.21.15.161:0/3879145543 shutdown_connections 2024-05-22T21:19:18.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.599+0000 7f2d6b400640 1 -- 172.21.15.161:0/3879145543 wait complete. 2024-05-22T21:19:18.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d6b400640 1 Processor -- start 2024-05-22T21:19:18.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d6b400640 1 -- start start 2024-05-22T21:19:18.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d6b400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d640ac8b0 0x7f2d64146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d6b400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d640accf0 0x7f2d640cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d6b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d6415d430 0x7f2d6414acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d6b400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6415be80 con 0x7f2d6415d430 2024-05-22T21:19:18.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d6b400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6415c160 con 0x7f2d640ac8b0 2024-05-22T21:19:18.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d6b400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6415c440 con 0x7f2d640accf0 2024-05-22T21:19:18.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d69400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d6415d430 0x7f2d6414acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d69400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d6415d430 0x7f2d6414acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44714/0 (socket says 172.21.15.161:44714) 2024-05-22T21:19:18.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d69400640 1 -- 172.21.15.161:0/1129280677 learned_addr learned my addr 172.21.15.161:0/1129280677 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:18.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d68a00640 1 --2- 172.21.15.161:0/1129280677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d640ac8b0 0x7f2d64146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d63e00640 1 --2- 172.21.15.161:0/1129280677 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d640accf0 0x7f2d640cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d69400640 1 -- 172.21.15.161:0/1129280677 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d640accf0 msgr2=0x7f2d640cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d69400640 1 --2- 172.21.15.161:0/1129280677 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d640accf0 0x7f2d640cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d69400640 1 -- 172.21.15.161:0/1129280677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d640ac8b0 msgr2=0x7f2d64146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d69400640 1 --2- 172.21.15.161:0/1129280677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d640ac8b0 0x7f2d64146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d69400640 1 -- 172.21.15.161:0/1129280677 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d540094d0 con 0x7f2d6415d430 2024-05-22T21:19:18.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d69400640 1 --2- 172.21.15.161:0/1129280677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d6415d430 0x7f2d6414acc0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00b9e0 tx=0x7f2d5c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:18.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d60c00640 1 -- 172.21.15.161:0/1129280677 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c003b80 con 0x7f2d6415d430 2024-05-22T21:19:18.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d60c00640 1 -- 172.21.15.161:0/1129280677 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c003ce0 con 0x7f2d6415d430 2024-05-22T21:19:18.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d60c00640 1 -- 172.21.15.161:0/1129280677 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c00cb60 con 0x7f2d6415d430 2024-05-22T21:19:18.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d6b400640 1 -- 172.21.15.161:0/1129280677 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d640cd0f0 con 0x7f2d6415d430 2024-05-22T21:19:18.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.603+0000 7f2d6b400640 1 -- 172.21.15.161:0/1129280677 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d64146cb0 con 0x7f2d6415d430 2024-05-22T21:19:18.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.607+0000 7f2d6b400640 1 -- 172.21.15.161:0/1129280677 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d64067220 con 0x7f2d6415d430 2024-05-22T21:19:18.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.607+0000 7f2d60c00640 1 -- 172.21.15.161:0/1129280677 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2d5c00ccc0 con 0x7f2d6415d430 2024-05-22T21:19:18.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.607+0000 7f2d60c00640 1 --2- 172.21.15.161:0/1129280677 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d40066690 0x7f2d40068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.607+0000 7f2d60c00640 1 -- 172.21.15.161:0/1129280677 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d5c083ea0 con 0x7f2d6415d430 2024-05-22T21:19:18.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.607+0000 7f2d68a00640 1 --2- 172.21.15.161:0/1129280677 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d40066690 0x7f2d40068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.607+0000 7f2d68a00640 1 --2- 172.21.15.161:0/1129280677 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d40066690 0x7f2d40068b50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2d641681e0 tx=0x7f2d4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:18.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.611+0000 7f2d60c00640 1 -- 172.21.15.161:0/1129280677 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2d5c051220 con 0x7f2d6415d430 2024-05-22T21:19:18.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.763+0000 7f2d6b400640 1 -- 172.21.15.161:0/1129280677 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d640cd5a0 con 0x7f2d40066690 2024-05-22T21:19:18.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.771+0000 7f2d60c00640 1 -- 172.21.15.161:0/1129280677 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284645 (secure 0 0 0) 0x7f2d640cd5a0 con 0x7f2d40066690 2024-05-22T21:19:18.776 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:18.778 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:19:18.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.775+0000 7f2d6b400640 1 -- 172.21.15.161:0/1129280677 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d40066690 msgr2=0x7f2d40068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.775+0000 7f2d6b400640 1 --2- 172.21.15.161:0/1129280677 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d40066690 0x7f2d40068b50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2d641681e0 tx=0x7f2d4c008040 comp rx=0 tx=0).stop 2024-05-22T21:19:18.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.775+0000 7f2d6b400640 1 -- 172.21.15.161:0/1129280677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d6415d430 msgr2=0x7f2d6414acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.775+0000 7f2d6b400640 1 --2- 172.21.15.161:0/1129280677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d6415d430 0x7f2d6414acc0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00b9e0 tx=0x7f2d5c00bea0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.775+0000 7f2d6b400640 1 -- 172.21.15.161:0/1129280677 shutdown_connections 2024-05-22T21:19:18.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.775+0000 7f2d6b400640 1 --2- 172.21.15.161:0/1129280677 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d40066690 0x7f2d40068b50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.775+0000 7f2d6b400640 1 --2- 172.21.15.161:0/1129280677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d6415d430 0x7f2d6414acc0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.775+0000 7f2d6b400640 1 --2- 172.21.15.161:0/1129280677 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d640accf0 0x7f2d640cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.775+0000 7f2d6b400640 1 --2- 172.21.15.161:0/1129280677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d640ac8b0 0x7f2d64146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.775+0000 7f2d6b400640 1 -- 172.21.15.161:0/1129280677 >> 172.21.15.161:0/1129280677 conn(0x7f2d640b0bc0 msgr2=0x7f2d640ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:18.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.775+0000 7f2d6b400640 1 -- 172.21.15.161:0/1129280677 shutdown_connections 2024-05-22T21:19:18.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.775+0000 7f2d6b400640 1 -- 172.21.15.161:0/1129280677 wait complete. 2024-05-22T21:19:18.797 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-05-22T21:19:18.471768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6532,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555967,"statfs":{"total":1600000000,"available":1593310599,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.511648"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780663+0000","last_change":"2024-05-22T21:19:14.364488+0000","last_active":"2024-05-22T21:19:15.780663+0000","last_peered":"2024-05-22T21:19:15.780663+0000","last_clean":"2024-05-22T21:19:15.780663+0000","last_became_active":"2024-05-22T21:19:14.364275+0000","last_became_peered":"2024-05-22T21:19:14.364275+0000","last_unstale":"2024-05-22T21:19:15.780663+0000","last_undegraded":"2024-05-22T21:19:15.780663+0000","last_fullsized":"2024-05-22T21:19:15.780663+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:26:45.307076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354084+0000","last_change":"2024-05-22T21:19:14.361359+0000","last_active":"2024-05-22T21:19:15.354084+0000","last_peered":"2024-05-22T21:19:15.354084+0000","last_clean":"2024-05-22T21:19:15.354084+0000","last_became_active":"2024-05-22T21:19:14.361124+0000","last_became_peered":"2024-05-22T21:19:14.361124+0000","last_unstale":"2024-05-22T21:19:15.354084+0000","last_undegraded":"2024-05-22T21:19:15.354084+0000","last_fullsized":"2024-05-22T21:19:15.354084+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:19:19.207878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780778+0000","last_change":"2024-05-22T21:19:14.364059+0000","last_active":"2024-05-22T21:19:15.780778+0000","last_peered":"2024-05-22T21:19:15.780778+0000","last_clean":"2024-05-22T21:19:15.780778+0000","last_became_active":"2024-05-22T21:19:14.363842+0000","last_became_peered":"2024-05-22T21:19:14.363842+0000","last_unstale":"2024-05-22T21:19:15.780778+0000","last_undegraded":"2024-05-22T21:19:15.780778+0000","last_fullsized":"2024-05-22T21:19:15.780778+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:48:30.354035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781230+0000","last_change":"2024-05-22T21:19:14.367626+0000","last_active":"2024-05-22T21:19:15.781230+0000","last_peered":"2024-05-22T21:19:15.781230+0000","last_clean":"2024-05-22T21:19:15.781230+0000","last_became_active":"2024-05-22T21:19:14.367419+0000","last_became_peered":"2024-05-22T21:19:14.367419+0000","last_unstale":"2024-05-22T21:19:15.781230+0000","last_undegraded":"2024-05-22T21:19:15.781230+0000","last_fullsized":"2024-05-22T21:19:15.781230+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:33:27.771256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.782129+0000","last_change":"2024-05-22T21:19:14.364747+0000","last_active":"2024-05-22T21:19:15.782129+0000","last_peered":"2024-05-22T21:19:15.782129+0000","last_clean":"2024-05-22T21:19:15.782129+0000","last_became_active":"2024-05-22T21:19:14.364470+0000","last_became_peered":"2024-05-22T21:19:14.364470+0000","last_unstale":"2024-05-22T21:19:15.782129+0000","last_undegraded":"2024-05-22T21:19:15.782129+0000","last_fullsized":"2024-05-22T21:19:15.782129+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:42:32.241040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355941+0000","last_change":"2024-05-22T21:19:14.361806+0000","last_active":"2024-05-22T21:19:15.355941+0000","last_peered":"2024-05-22T21:19:15.355941+0000","last_clean":"2024-05-22T21:19:15.355941+0000","last_became_active":"2024-05-22T21:19:14.361596+0000","last_became_peered":"2024-05-22T21:19:14.361596+0000","last_unstale":"2024-05-22T21:19:15.355941+0000","last_undegraded":"2024-05-22T21:19:15.355941+0000","last_fullsized":"2024-05-22T21:19:15.355941+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:40:13.108550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355493+0000","last_change":"2024-05-22T21:19:14.362109+0000","last_active":"2024-05-22T21:19:15.355493+0000","last_peered":"2024-05-22T21:19:15.355493+0000","last_clean":"2024-05-22T21:19:15.355493+0000","last_became_active":"2024-05-22T21:19:14.361747+0000","last_became_peered":"2024-05-22T21:19:14.361747+0000","last_unstale":"2024-05-22T21:19:15.355493+0000","last_undegraded":"2024-05-22T21:19:15.355493+0000","last_fullsized":"2024-05-22T21:19:15.355493+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:29:22.441939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354022+0000","last_change":"2024-05-22T21:19:14.359960+0000","last_active":"2024-05-22T21:19:15.354022+0000","last_peered":"2024-05-22T21:19:15.354022+0000","last_clean":"2024-05-22T21:19:15.354022+0000","last_became_active":"2024-05-22T21:19:14.357742+0000","last_became_peered":"2024-05-22T21:19:14.357742+0000","last_unstale":"2024-05-22T21:19:15.354022+0000","last_undegraded":"2024-05-22T21:19:15.354022+0000","last_fullsized":"2024-05-22T21:19:15.354022+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:55:51.198941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355893+0000","last_change":"2024-05-22T21:19:14.362220+0000","last_active":"2024-05-22T21:19:15.355893+0000","last_peered":"2024-05-22T21:19:15.355893+0000","last_clean":"2024-05-22T21:19:15.355893+0000","last_became_active":"2024-05-22T21:19:14.361982+0000","last_became_peered":"2024-05-22T21:19:14.361982+0000","last_unstale":"2024-05-22T21:19:15.355893+0000","last_undegraded":"2024-05-22T21:19:15.355893+0000","last_fullsized":"2024-05-22T21:19:15.355893+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:42.559556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.782234+0000","last_change":"2024-05-22T21:19:14.360223+0000","last_active":"2024-05-22T21:19:15.782234+0000","last_peered":"2024-05-22T21:19:15.782234+0000","last_clean":"2024-05-22T21:19:15.782234+0000","last_became_active":"2024-05-22T21:19:14.359784+0000","last_became_peered":"2024-05-22T21:19:14.359784+0000","last_unstale":"2024-05-22T21:19:15.782234+0000","last_undegraded":"2024-05-22T21:19:15.782234+0000","last_fullsized":"2024-05-22T21:19:15.782234+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:20:21.114154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356030+0000","last_change":"2024-05-22T21:19:14.355883+0000","last_active":"2024-05-22T21:19:15.356030+0000","last_peered":"2024-05-22T21:19:15.356030+0000","last_clean":"2024-05-22T21:19:15.356030+0000","last_became_active":"2024-05-22T21:19:14.355274+0000","last_became_peered":"2024-05-22T21:19:14.355274+0000","last_unstale":"2024-05-22T21:19:15.356030+0000","last_undegraded":"2024-05-22T21:19:15.356030+0000","last_fullsized":"2024-05-22T21:19:15.356030+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:57:26.931482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356335+0000","last_change":"2024-05-22T21:19:14.358871+0000","last_active":"2024-05-22T21:19:15.356335+0000","last_peered":"2024-05-22T21:19:15.356335+0000","last_clean":"2024-05-22T21:19:15.356335+0000","last_became_active":"2024-05-22T21:19:14.358522+0000","last_became_peered":"2024-05-22T21:19:14.358522+0000","last_unstale":"2024-05-22T21:19:15.356335+0000","last_undegraded":"2024-05-22T21:19:15.356335+0000","last_fullsized":"2024-05-22T21:19:15.356335+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:56:11.698434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780685+0000","last_change":"2024-05-22T21:19:14.365985+0000","last_active":"2024-05-22T21:19:15.780685+0000","last_peered":"2024-05-22T21:19:15.780685+0000","last_clean":"2024-05-22T21:19:15.780685+0000","last_became_active":"2024-05-22T21:19:14.365495+0000","last_became_peered":"2024-05-22T21:19:14.365495+0000","last_unstale":"2024-05-22T21:19:15.780685+0000","last_undegraded":"2024-05-22T21:19:15.780685+0000","last_fullsized":"2024-05-22T21:19:15.780685+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:20:21.114154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430337+0000","last_change":"2024-05-22T21:19:14.366921+0000","last_active":"2024-05-22T21:19:15.430337+0000","last_peered":"2024-05-22T21:19:15.430337+0000","last_clean":"2024-05-22T21:19:15.430337+0000","last_became_active":"2024-05-22T21:19:14.365934+0000","last_became_peered":"2024-05-22T21:19:14.365934+0000","last_unstale":"2024-05-22T21:19:15.430337+0000","last_undegraded":"2024-05-22T21:19:15.430337+0000","last_fullsized":"2024-05-22T21:19:15.430337+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:12:16.565090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781007+0000","last_change":"2024-05-22T21:19:14.358187+0000","last_active":"2024-05-22T21:19:15.781007+0000","last_peered":"2024-05-22T21:19:15.781007+0000","last_clean":"2024-05-22T21:19:15.781007+0000","last_became_active":"2024-05-22T21:19:14.357593+0000","last_became_peered":"2024-05-22T21:19:14.357593+0000","last_unstale":"2024-05-22T21:19:15.781007+0000","last_undegraded":"2024-05-22T21:19:15.781007+0000","last_fullsized":"2024-05-22T21:19:15.781007+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:08:05.273893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356119+0000","last_change":"2024-05-22T21:19:14.359327+0000","last_active":"2024-05-22T21:19:15.356119+0000","last_peered":"2024-05-22T21:19:15.356119+0000","last_clean":"2024-05-22T21:19:15.356119+0000","last_became_active":"2024-05-22T21:19:14.358923+0000","last_became_peered":"2024-05-22T21:19:14.358923+0000","last_unstale":"2024-05-22T21:19:15.356119+0000","last_undegraded":"2024-05-22T21:19:15.356119+0000","last_fullsized":"2024-05-22T21:19:15.356119+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:55:24.116469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354182+0000","last_change":"2024-05-22T21:19:14.371433+0000","last_active":"2024-05-22T21:19:15.354182+0000","last_peered":"2024-05-22T21:19:15.354182+0000","last_clean":"2024-05-22T21:19:15.354182+0000","last_became_active":"2024-05-22T21:19:14.371146+0000","last_became_peered":"2024-05-22T21:19:14.371146+0000","last_unstale":"2024-05-22T21:19:15.354182+0000","last_undegraded":"2024-05-22T21:19:15.354182+0000","last_fullsized":"2024-05-22T21:19:15.354182+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:04:40.868247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780779+0000","last_change":"2024-05-22T21:19:14.367029+0000","last_active":"2024-05-22T21:19:15.780779+0000","last_peered":"2024-05-22T21:19:15.780779+0000","last_clean":"2024-05-22T21:19:15.780779+0000","last_became_active":"2024-05-22T21:19:14.366752+0000","last_became_peered":"2024-05-22T21:19:14.366752+0000","last_unstale":"2024-05-22T21:19:15.780779+0000","last_undegraded":"2024-05-22T21:19:15.780779+0000","last_fullsized":"2024-05-22T21:19:15.780779+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:27:50.460840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430416+0000","last_change":"2024-05-22T21:19:14.370443+0000","last_active":"2024-05-22T21:19:15.430416+0000","last_peered":"2024-05-22T21:19:15.430416+0000","last_clean":"2024-05-22T21:19:15.430416+0000","last_became_active":"2024-05-22T21:19:14.369808+0000","last_became_peered":"2024-05-22T21:19:14.369808+0000","last_unstale":"2024-05-22T21:19:15.430416+0000","last_undegraded":"2024-05-22T21:19:15.430416+0000","last_fullsized":"2024-05-22T21:19:15.430416+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:20:42.029795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356379+0000","last_change":"2024-05-22T21:19:14.364250+0000","last_active":"2024-05-22T21:19:15.356379+0000","last_peered":"2024-05-22T21:19:15.356379+0000","last_clean":"2024-05-22T21:19:15.356379+0000","last_became_active":"2024-05-22T21:19:14.363689+0000","last_became_peered":"2024-05-22T21:19:14.363689+0000","last_unstale":"2024-05-22T21:19:15.356379+0000","last_undegraded":"2024-05-22T21:19:15.356379+0000","last_fullsized":"2024-05-22T21:19:15.356379+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:42:32.241040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430362+0000","last_change":"2024-05-22T21:19:14.362961+0000","last_active":"2024-05-22T21:19:15.430362+0000","last_peered":"2024-05-22T21:19:15.430362+0000","last_clean":"2024-05-22T21:19:15.430362+0000","last_became_active":"2024-05-22T21:19:14.362698+0000","last_became_peered":"2024-05-22T21:19:14.362698+0000","last_unstale":"2024-05-22T21:19:15.430362+0000","last_undegraded":"2024-05-22T21:19:15.430362+0000","last_fullsized":"2024-05-22T21:19:15.430362+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:18:09.280531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354349+0000","last_change":"2024-05-22T21:19:14.366403+0000","last_active":"2024-05-22T21:19:15.354349+0000","last_peered":"2024-05-22T21:19:15.354349+0000","last_clean":"2024-05-22T21:19:15.354349+0000","last_became_active":"2024-05-22T21:19:14.366182+0000","last_became_peered":"2024-05-22T21:19:14.366182+0000","last_unstale":"2024-05-22T21:19:15.354349+0000","last_undegraded":"2024-05-22T21:19:15.354349+0000","last_fullsized":"2024-05-22T21:19:15.354349+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:32:01.457841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354112+0000","last_change":"2024-05-22T21:19:14.368291+0000","last_active":"2024-05-22T21:19:15.354112+0000","last_peered":"2024-05-22T21:19:15.354112+0000","last_clean":"2024-05-22T21:19:15.354112+0000","last_became_active":"2024-05-22T21:19:14.367265+0000","last_became_peered":"2024-05-22T21:19:14.367265+0000","last_unstale":"2024-05-22T21:19:15.354112+0000","last_undegraded":"2024-05-22T21:19:15.354112+0000","last_fullsized":"2024-05-22T21:19:15.354112+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:49:38.868866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355555+0000","last_change":"2024-05-22T21:19:14.362548+0000","last_active":"2024-05-22T21:19:15.355555+0000","last_peered":"2024-05-22T21:19:15.355555+0000","last_clean":"2024-05-22T21:19:15.355555+0000","last_became_active":"2024-05-22T21:19:14.362342+0000","last_became_peered":"2024-05-22T21:19:14.362342+0000","last_unstale":"2024-05-22T21:19:15.355555+0000","last_undegraded":"2024-05-22T21:19:15.355555+0000","last_fullsized":"2024-05-22T21:19:15.355555+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:20:21.114154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.782332+0000","last_change":"2024-05-22T21:19:14.365909+0000","last_active":"2024-05-22T21:19:15.782332+0000","last_peered":"2024-05-22T21:19:15.782332+0000","last_clean":"2024-05-22T21:19:15.782332+0000","last_became_active":"2024-05-22T21:19:14.365017+0000","last_became_peered":"2024-05-22T21:19:14.365017+0000","last_unstale":"2024-05-22T21:19:15.782332+0000","last_undegraded":"2024-05-22T21:19:15.782332+0000","last_fullsized":"2024-05-22T21:19:15.782332+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:20:19.111100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354113+0000","last_change":"2024-05-22T21:19:14.356082+0000","last_active":"2024-05-22T21:19:15.354113+0000","last_peered":"2024-05-22T21:19:15.354113+0000","last_clean":"2024-05-22T21:19:15.354113+0000","last_became_active":"2024-05-22T21:19:14.355865+0000","last_became_peered":"2024-05-22T21:19:14.355865+0000","last_unstale":"2024-05-22T21:19:15.354113+0000","last_undegraded":"2024-05-22T21:19:15.354113+0000","last_fullsized":"2024-05-22T21:19:15.354113+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:40:13.108550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355615+0000","last_change":"2024-05-22T21:19:14.356990+0000","last_active":"2024-05-22T21:19:15.355615+0000","last_peered":"2024-05-22T21:19:15.355615+0000","last_clean":"2024-05-22T21:19:15.355615+0000","last_became_active":"2024-05-22T21:19:14.356756+0000","last_became_peered":"2024-05-22T21:19:14.356756+0000","last_unstale":"2024-05-22T21:19:15.355615+0000","last_undegraded":"2024-05-22T21:19:15.355615+0000","last_fullsized":"2024-05-22T21:19:15.355615+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:09:35.979308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781851+0000","last_change":"2024-05-22T21:19:14.367805+0000","last_active":"2024-05-22T21:19:15.781851+0000","last_peered":"2024-05-22T21:19:15.781851+0000","last_clean":"2024-05-22T21:19:15.781851+0000","last_became_active":"2024-05-22T21:19:14.367093+0000","last_became_peered":"2024-05-22T21:19:14.367093+0000","last_unstale":"2024-05-22T21:19:15.781851+0000","last_undegraded":"2024-05-22T21:19:15.781851+0000","last_fullsized":"2024-05-22T21:19:15.781851+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:42.559556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.353998+0000","last_change":"2024-05-22T21:19:14.369196+0000","last_active":"2024-05-22T21:19:15.353998+0000","last_peered":"2024-05-22T21:19:15.353998+0000","last_clean":"2024-05-22T21:19:15.353998+0000","last_became_active":"2024-05-22T21:19:14.368984+0000","last_became_peered":"2024-05-22T21:19:14.368984+0000","last_unstale":"2024-05-22T21:19:15.353998+0000","last_undegraded":"2024-05-22T21:19:15.353998+0000","last_fullsized":"2024-05-22T21:19:15.353998+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:29:22.441939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430645+0000","last_change":"2024-05-22T21:19:14.359063+0000","last_active":"2024-05-22T21:19:15.430645+0000","last_peered":"2024-05-22T21:19:15.430645+0000","last_clean":"2024-05-22T21:19:15.430645+0000","last_became_active":"2024-05-22T21:19:14.358819+0000","last_became_peered":"2024-05-22T21:19:14.358819+0000","last_unstale":"2024-05-22T21:19:15.430645+0000","last_undegraded":"2024-05-22T21:19:15.430645+0000","last_fullsized":"2024-05-22T21:19:15.430645+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:44:39.869878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780601+0000","last_change":"2024-05-22T21:19:14.362184+0000","last_active":"2024-05-22T21:19:15.780601+0000","last_peered":"2024-05-22T21:19:15.780601+0000","last_clean":"2024-05-22T21:19:15.780601+0000","last_became_active":"2024-05-22T21:19:14.361968+0000","last_became_peered":"2024-05-22T21:19:14.361968+0000","last_unstale":"2024-05-22T21:19:15.780601+0000","last_undegraded":"2024-05-22T21:19:15.780601+0000","last_fullsized":"2024-05-22T21:19:15.780601+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:29:19.636553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781038+0000","last_change":"2024-05-22T21:19:14.363924+0000","last_active":"2024-05-22T21:19:15.781038+0000","last_peered":"2024-05-22T21:19:15.781038+0000","last_clean":"2024-05-22T21:19:15.781038+0000","last_became_active":"2024-05-22T21:19:14.363638+0000","last_became_peered":"2024-05-22T21:19:14.363638+0000","last_unstale":"2024-05-22T21:19:15.781038+0000","last_undegraded":"2024-05-22T21:19:15.781038+0000","last_fullsized":"2024-05-22T21:19:15.781038+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:10:13.431903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430466+0000","last_change":"2024-05-22T21:19:14.359239+0000","last_active":"2024-05-22T21:19:15.430466+0000","last_peered":"2024-05-22T21:19:15.430466+0000","last_clean":"2024-05-22T21:19:15.430466+0000","last_became_active":"2024-05-22T21:19:14.359032+0000","last_became_peered":"2024-05-22T21:19:14.359032+0000","last_unstale":"2024-05-22T21:19:15.430466+0000","last_undegraded":"2024-05-22T21:19:15.430466+0000","last_fullsized":"2024-05-22T21:19:15.430466+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:03:10.351285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"124'4","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354371+0000","last_change":"2024-05-22T21:19:13.975681+0000","last_active":"2024-05-22T21:19:15.354371+0000","last_peered":"2024-05-22T21:19:15.354371+0000","last_clean":"2024-05-22T21:19:15.354371+0000","last_became_active":"2024-05-22T21:19:13.975430+0000","last_became_peered":"2024-05-22T21:19:13.975430+0000","last_unstale":"2024-05-22T21:19:15.354371+0000","last_undegraded":"2024-05-22T21:19:15.354371+0000","last_fullsized":"2024-05-22T21:19:15.354371+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:20:47.503141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780862+0000","last_change":"2024-05-22T21:19:14.366128+0000","last_active":"2024-05-22T21:19:15.780862+0000","last_peered":"2024-05-22T21:19:15.780862+0000","last_clean":"2024-05-22T21:19:15.780862+0000","last_became_active":"2024-05-22T21:19:14.365777+0000","last_became_peered":"2024-05-22T21:19:14.365777+0000","last_unstale":"2024-05-22T21:19:15.780862+0000","last_undegraded":"2024-05-22T21:19:15.780862+0000","last_fullsized":"2024-05-22T21:19:15.780862+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:14:03.575567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"124'1","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355653+0000","last_change":"2024-05-22T21:19:13.975173+0000","last_active":"2024-05-22T21:19:15.355653+0000","last_peered":"2024-05-22T21:19:15.355653+0000","last_clean":"2024-05-22T21:19:15.355653+0000","last_became_active":"2024-05-22T21:19:13.974948+0000","last_became_peered":"2024-05-22T21:19:13.974948+0000","last_unstale":"2024-05-22T21:19:15.355653+0000","last_undegraded":"2024-05-22T21:19:15.355653+0000","last_fullsized":"2024-05-22T21:19:15.355653+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:35:48.944267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780885+0000","last_change":"2024-05-22T21:19:14.359636+0000","last_active":"2024-05-22T21:19:15.780885+0000","last_peered":"2024-05-22T21:19:15.780885+0000","last_clean":"2024-05-22T21:19:15.780885+0000","last_became_active":"2024-05-22T21:19:14.356830+0000","last_became_peered":"2024-05-22T21:19:14.356830+0000","last_unstale":"2024-05-22T21:19:15.780885+0000","last_undegraded":"2024-05-22T21:19:15.780885+0000","last_fullsized":"2024-05-22T21:19:15.780885+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:44:36.454680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"124'2","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354186+0000","last_change":"2024-05-22T21:19:13.970734+0000","last_active":"2024-05-22T21:19:15.354186+0000","last_peered":"2024-05-22T21:19:15.354186+0000","last_clean":"2024-05-22T21:19:15.354186+0000","last_became_active":"2024-05-22T21:19:13.970499+0000","last_became_peered":"2024-05-22T21:19:13.970499+0000","last_unstale":"2024-05-22T21:19:15.354186+0000","last_undegraded":"2024-05-22T21:19:15.354186+0000","last_fullsized":"2024-05-22T21:19:15.354186+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:53:19.407325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356481+0000","last_change":"2024-05-22T21:19:14.364396+0000","last_active":"2024-05-22T21:19:15.356481+0000","last_peered":"2024-05-22T21:19:15.356481+0000","last_clean":"2024-05-22T21:19:15.356481+0000","last_became_active":"2024-05-22T21:19:14.364006+0000","last_became_peered":"2024-05-22T21:19:14.364006+0000","last_unstale":"2024-05-22T21:19:15.356481+0000","last_undegraded":"2024-05-22T21:19:15.356481+0000","last_fullsized":"2024-05-22T21:19:15.356481+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:20:48.510850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"124'2","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354230+0000","last_change":"2024-05-22T21:19:13.972827+0000","last_active":"2024-05-22T21:19:15.354230+0000","last_peered":"2024-05-22T21:19:15.354230+0000","last_clean":"2024-05-22T21:19:15.354230+0000","last_became_active":"2024-05-22T21:19:13.972123+0000","last_became_peered":"2024-05-22T21:19:13.972123+0000","last_unstale":"2024-05-22T21:19:15.354230+0000","last_undegraded":"2024-05-22T21:19:15.354230+0000","last_fullsized":"2024-05-22T21:19:15.354230+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:56:10.690726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781781+0000","last_change":"2024-05-22T21:19:14.363924+0000","last_active":"2024-05-22T21:19:15.781781+0000","last_peered":"2024-05-22T21:19:15.781781+0000","last_clean":"2024-05-22T21:19:15.781781+0000","last_became_active":"2024-05-22T21:19:14.363477+0000","last_became_peered":"2024-05-22T21:19:14.363477+0000","last_unstale":"2024-05-22T21:19:15.781781+0000","last_undegraded":"2024-05-22T21:19:15.781781+0000","last_fullsized":"2024-05-22T21:19:15.781781+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:34:21.879517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"124'1","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781196+0000","last_change":"2024-05-22T21:19:13.976685+0000","last_active":"2024-05-22T21:19:15.781196+0000","last_peered":"2024-05-22T21:19:15.781196+0000","last_clean":"2024-05-22T21:19:15.781196+0000","last_became_active":"2024-05-22T21:19:13.976376+0000","last_became_peered":"2024-05-22T21:19:13.976376+0000","last_unstale":"2024-05-22T21:19:15.781196+0000","last_undegraded":"2024-05-22T21:19:15.781196+0000","last_fullsized":"2024-05-22T21:19:15.781196+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:36:43.412685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355724+0000","last_change":"2024-05-22T21:19:14.361960+0000","last_active":"2024-05-22T21:19:15.355724+0000","last_peered":"2024-05-22T21:19:15.355724+0000","last_clean":"2024-05-22T21:19:15.355724+0000","last_became_active":"2024-05-22T21:19:14.360475+0000","last_became_peered":"2024-05-22T21:19:14.360475+0000","last_unstale":"2024-05-22T21:19:15.355724+0000","last_undegraded":"2024-05-22T21:19:15.355724+0000","last_fullsized":"2024-05-22T21:19:15.355724+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:33:27.771256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"124'1","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354451+0000","last_change":"2024-05-22T21:19:13.972249+0000","last_active":"2024-05-22T21:19:15.354451+0000","last_peered":"2024-05-22T21:19:15.354451+0000","last_clean":"2024-05-22T21:19:15.354451+0000","last_became_active":"2024-05-22T21:19:13.972020+0000","last_became_peered":"2024-05-22T21:19:13.972020+0000","last_unstale":"2024-05-22T21:19:15.354451+0000","last_undegraded":"2024-05-22T21:19:15.354451+0000","last_fullsized":"2024-05-22T21:19:15.354451+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:42:31.233331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781077+0000","last_change":"2024-05-22T21:19:14.362662+0000","last_active":"2024-05-22T21:19:15.781077+0000","last_peered":"2024-05-22T21:19:15.781077+0000","last_clean":"2024-05-22T21:19:15.781077+0000","last_became_active":"2024-05-22T21:19:14.362027+0000","last_became_peered":"2024-05-22T21:19:14.362027+0000","last_unstale":"2024-05-22T21:19:15.781077+0000","last_undegraded":"2024-05-22T21:19:15.781077+0000","last_fullsized":"2024-05-22T21:19:15.781077+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:20:19.111100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781907+0000","last_change":"2024-05-22T21:19:13.976799+0000","last_active":"2024-05-22T21:19:15.781907+0000","last_peered":"2024-05-22T21:19:15.781907+0000","last_clean":"2024-05-22T21:19:15.781907+0000","last_became_active":"2024-05-22T21:19:13.976325+0000","last_became_peered":"2024-05-22T21:19:13.976325+0000","last_unstale":"2024-05-22T21:19:15.781907+0000","last_undegraded":"2024-05-22T21:19:15.781907+0000","last_fullsized":"2024-05-22T21:19:15.781907+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:10:23.280341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356005+0000","last_change":"2024-05-22T21:19:14.360884+0000","last_active":"2024-05-22T21:19:15.356005+0000","last_peered":"2024-05-22T21:19:15.356005+0000","last_clean":"2024-05-22T21:19:15.356005+0000","last_became_active":"2024-05-22T21:19:14.360370+0000","last_became_peered":"2024-05-22T21:19:14.360370+0000","last_unstale":"2024-05-22T21:19:15.356005+0000","last_undegraded":"2024-05-22T21:19:15.356005+0000","last_fullsized":"2024-05-22T21:19:15.356005+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:53:20.415034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781675+0000","last_change":"2024-05-22T21:19:13.973640+0000","last_active":"2024-05-22T21:19:15.781675+0000","last_peered":"2024-05-22T21:19:15.781675+0000","last_clean":"2024-05-22T21:19:15.781675+0000","last_became_active":"2024-05-22T21:19:13.973417+0000","last_became_peered":"2024-05-22T21:19:13.973417+0000","last_unstale":"2024-05-22T21:19:15.781675+0000","last_undegraded":"2024-05-22T21:19:15.781675+0000","last_fullsized":"2024-05-22T21:19:15.781675+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:41:34.052524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354435+0000","last_change":"2024-05-22T21:19:14.364098+0000","last_active":"2024-05-22T21:19:15.354435+0000","last_peered":"2024-05-22T21:19:15.354435+0000","last_clean":"2024-05-22T21:19:15.354435+0000","last_became_active":"2024-05-22T21:19:14.363890+0000","last_became_peered":"2024-05-22T21:19:14.363890+0000","last_unstale":"2024-05-22T21:19:15.354435+0000","last_undegraded":"2024-05-22T21:19:15.354435+0000","last_fullsized":"2024-05-22T21:19:15.354435+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:57:26.931482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781019+0000","last_change":"2024-05-22T21:19:13.975036+0000","last_active":"2024-05-22T21:19:15.781019+0000","last_peered":"2024-05-22T21:19:15.781019+0000","last_clean":"2024-05-22T21:19:15.781019+0000","last_became_active":"2024-05-22T21:19:13.972123+0000","last_became_peered":"2024-05-22T21:19:13.972123+0000","last_unstale":"2024-05-22T21:19:15.781019+0000","last_undegraded":"2024-05-22T21:19:15.781019+0000","last_fullsized":"2024-05-22T21:19:15.781019+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:20:41.022086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355678+0000","last_change":"2024-05-22T21:19:14.362315+0000","last_active":"2024-05-22T21:19:15.355678+0000","last_peered":"2024-05-22T21:19:15.355678+0000","last_clean":"2024-05-22T21:19:15.355678+0000","last_became_active":"2024-05-22T21:19:14.361708+0000","last_became_peered":"2024-05-22T21:19:14.361708+0000","last_unstale":"2024-05-22T21:19:15.355678+0000","last_undegraded":"2024-05-22T21:19:15.355678+0000","last_fullsized":"2024-05-22T21:19:15.355678+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:44:54.074369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780569+0000","last_change":"2024-05-22T21:19:13.976061+0000","last_active":"2024-05-22T21:19:15.780569+0000","last_peered":"2024-05-22T21:19:15.780569+0000","last_clean":"2024-05-22T21:19:15.780569+0000","last_became_active":"2024-05-22T21:19:13.975835+0000","last_became_peered":"2024-05-22T21:19:13.975835+0000","last_unstale":"2024-05-22T21:19:15.780569+0000","last_undegraded":"2024-05-22T21:19:15.780569+0000","last_fullsized":"2024-05-22T21:19:15.780569+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:01:15.829792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781559+0000","last_change":"2024-05-22T21:19:14.365677+0000","last_active":"2024-05-22T21:19:15.781559+0000","last_peered":"2024-05-22T21:19:15.781559+0000","last_clean":"2024-05-22T21:19:15.781559+0000","last_became_active":"2024-05-22T21:19:14.365466+0000","last_became_peered":"2024-05-22T21:19:14.365466+0000","last_unstale":"2024-05-22T21:19:15.781559+0000","last_undegraded":"2024-05-22T21:19:15.781559+0000","last_fullsized":"2024-05-22T21:19:15.781559+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:14:03.575567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"124'3","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355741+0000","last_change":"2024-05-22T21:19:13.974320+0000","last_active":"2024-05-22T21:19:15.355741+0000","last_peered":"2024-05-22T21:19:15.355741+0000","last_clean":"2024-05-22T21:19:15.355741+0000","last_became_active":"2024-05-22T21:19:13.974090+0000","last_became_peered":"2024-05-22T21:19:13.974090+0000","last_unstale":"2024-05-22T21:19:15.355741+0000","last_undegraded":"2024-05-22T21:19:15.355741+0000","last_fullsized":"2024-05-22T21:19:15.355741+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:26:41.742313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355627+0000","last_change":"2024-05-22T21:19:14.358867+0000","last_active":"2024-05-22T21:19:15.355627+0000","last_peered":"2024-05-22T21:19:15.355627+0000","last_clean":"2024-05-22T21:19:15.355627+0000","last_became_active":"2024-05-22T21:19:14.358651+0000","last_became_peered":"2024-05-22T21:19:14.358651+0000","last_unstale":"2024-05-22T21:19:15.355627+0000","last_undegraded":"2024-05-22T21:19:15.355627+0000","last_fullsized":"2024-05-22T21:19:15.355627+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:27:50.460840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"124'6","reported_seq":17,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356249+0000","last_change":"2024-05-22T21:19:13.978625+0000","last_active":"2024-05-22T21:19:15.356249+0000","last_peered":"2024-05-22T21:19:15.356249+0000","last_clean":"2024-05-22T21:19:15.356249+0000","last_became_active":"2024-05-22T21:19:13.978170+0000","last_became_peered":"2024-05-22T21:19:13.978170+0000","last_unstale":"2024-05-22T21:19:15.356249+0000","last_undegraded":"2024-05-22T21:19:15.356249+0000","last_fullsized":"2024-05-22T21:19:15.356249+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:53:11.275117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780702+0000","last_change":"2024-05-22T21:19:14.357855+0000","last_active":"2024-05-22T21:19:15.780702+0000","last_peered":"2024-05-22T21:19:15.780702+0000","last_clean":"2024-05-22T21:19:15.780702+0000","last_became_active":"2024-05-22T21:19:14.357643+0000","last_became_peered":"2024-05-22T21:19:14.357643+0000","last_unstale":"2024-05-22T21:19:15.780702+0000","last_undegraded":"2024-05-22T21:19:15.780702+0000","last_fullsized":"2024-05-22T21:19:15.780702+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:38:48.877194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"124'2","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430776+0000","last_change":"2024-05-22T21:19:13.980290+0000","last_active":"2024-05-22T21:19:15.430776+0000","last_peered":"2024-05-22T21:19:15.430776+0000","last_clean":"2024-05-22T21:19:15.430776+0000","last_became_active":"2024-05-22T21:19:13.979862+0000","last_became_peered":"2024-05-22T21:19:13.979862+0000","last_unstale":"2024-05-22T21:19:15.430776+0000","last_undegraded":"2024-05-22T21:19:15.430776+0000","last_fullsized":"2024-05-22T21:19:15.430776+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:20:57.410269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781590+0000","last_change":"2024-05-22T21:19:14.364846+0000","last_active":"2024-05-22T21:19:15.781590+0000","last_peered":"2024-05-22T21:19:15.781590+0000","last_clean":"2024-05-22T21:19:15.781590+0000","last_became_active":"2024-05-22T21:19:14.359728+0000","last_became_peered":"2024-05-22T21:19:14.359728+0000","last_unstale":"2024-05-22T21:19:15.781590+0000","last_undegraded":"2024-05-22T21:19:15.781590+0000","last_fullsized":"2024-05-22T21:19:15.781590+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:31:36.605047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430550+0000","last_change":"2024-05-22T21:19:13.979649+0000","last_active":"2024-05-22T21:19:15.430550+0000","last_peered":"2024-05-22T21:19:15.430550+0000","last_clean":"2024-05-22T21:19:15.430550+0000","last_became_active":"2024-05-22T21:19:13.979186+0000","last_became_peered":"2024-05-22T21:19:13.979186+0000","last_unstale":"2024-05-22T21:19:15.430550+0000","last_undegraded":"2024-05-22T21:19:15.430550+0000","last_fullsized":"2024-05-22T21:19:15.430550+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:36:04.543841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356070+0000","last_change":"2024-05-22T21:19:14.365472+0000","last_active":"2024-05-22T21:19:15.356070+0000","last_peered":"2024-05-22T21:19:15.356070+0000","last_clean":"2024-05-22T21:19:15.356070+0000","last_became_active":"2024-05-22T21:19:14.365098+0000","last_became_peered":"2024-05-22T21:19:14.365098+0000","last_unstale":"2024-05-22T21:19:15.356070+0000","last_undegraded":"2024-05-22T21:19:15.356070+0000","last_fullsized":"2024-05-22T21:19:15.356070+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:26:42.750022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356458+0000","last_change":"2024-05-22T21:19:13.975623+0000","last_active":"2024-05-22T21:19:15.356458+0000","last_peered":"2024-05-22T21:19:15.356458+0000","last_clean":"2024-05-22T21:19:15.356458+0000","last_became_active":"2024-05-22T21:19:13.975118+0000","last_became_peered":"2024-05-22T21:19:13.975118+0000","last_unstale":"2024-05-22T21:19:15.356458+0000","last_undegraded":"2024-05-22T21:19:15.356458+0000","last_fullsized":"2024-05-22T21:19:15.356458+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:26:26.886392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430466+0000","last_change":"2024-05-22T21:19:14.362154+0000","last_active":"2024-05-22T21:19:15.430466+0000","last_peered":"2024-05-22T21:19:15.430466+0000","last_clean":"2024-05-22T21:19:15.430466+0000","last_became_active":"2024-05-22T21:19:14.361441+0000","last_became_peered":"2024-05-22T21:19:14.361441+0000","last_unstale":"2024-05-22T21:19:15.430466+0000","last_undegraded":"2024-05-22T21:19:15.430466+0000","last_fullsized":"2024-05-22T21:19:15.430466+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:24:15.256014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"124'8","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781005+0000","last_change":"2024-05-22T21:19:13.972217+0000","last_active":"2024-05-22T21:19:15.781005+0000","last_peered":"2024-05-22T21:19:15.781005+0000","last_clean":"2024-05-22T21:19:15.781005+0000","last_became_active":"2024-05-22T21:19:13.971997+0000","last_became_peered":"2024-05-22T21:19:13.971997+0000","last_unstale":"2024-05-22T21:19:15.781005+0000","last_undegraded":"2024-05-22T21:19:15.781005+0000","last_fullsized":"2024-05-22T21:19:15.781005+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:44:38.862169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781683+0000","last_change":"2024-05-22T21:19:14.364973+0000","last_active":"2024-05-22T21:19:15.781683+0000","last_peered":"2024-05-22T21:19:15.781683+0000","last_clean":"2024-05-22T21:19:15.781683+0000","last_became_active":"2024-05-22T21:19:14.364704+0000","last_became_peered":"2024-05-22T21:19:14.364704+0000","last_unstale":"2024-05-22T21:19:15.781683+0000","last_undegraded":"2024-05-22T21:19:15.781683+0000","last_fullsized":"2024-05-22T21:19:15.781683+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:27:50.460840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"124'2","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356564+0000","last_change":"2024-05-22T21:19:13.979237+0000","last_active":"2024-05-22T21:19:15.356564+0000","last_peered":"2024-05-22T21:19:15.356564+0000","last_clean":"2024-05-22T21:19:15.356564+0000","last_became_active":"2024-05-22T21:19:13.979022+0000","last_became_peered":"2024-05-22T21:19:13.979022+0000","last_unstale":"2024-05-22T21:19:15.356564+0000","last_undegraded":"2024-05-22T21:19:15.356564+0000","last_fullsized":"2024-05-22T21:19:15.356564+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:48:35.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781680+0000","last_change":"2024-05-22T21:19:14.360706+0000","last_active":"2024-05-22T21:19:15.781680+0000","last_peered":"2024-05-22T21:19:15.781680+0000","last_clean":"2024-05-22T21:19:15.781680+0000","last_became_active":"2024-05-22T21:19:14.360488+0000","last_became_peered":"2024-05-22T21:19:14.360488+0000","last_unstale":"2024-05-22T21:19:15.781680+0000","last_undegraded":"2024-05-22T21:19:15.781680+0000","last_fullsized":"2024-05-22T21:19:15.781680+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:46:40.734526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"124'1","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.782119+0000","last_change":"2024-05-22T21:19:13.975276+0000","last_active":"2024-05-22T21:19:15.782119+0000","last_peered":"2024-05-22T21:19:15.782119+0000","last_clean":"2024-05-22T21:19:15.782119+0000","last_became_active":"2024-05-22T21:19:13.974985+0000","last_became_peered":"2024-05-22T21:19:13.974985+0000","last_unstale":"2024-05-22T21:19:15.782119+0000","last_undegraded":"2024-05-22T21:19:15.782119+0000","last_fullsized":"2024-05-22T21:19:15.782119+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:46:39.726817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.356207+0000","last_change":"2024-05-22T21:19:14.357206+0000","last_active":"2024-05-22T21:19:15.356207+0000","last_peered":"2024-05-22T21:19:15.356207+0000","last_clean":"2024-05-22T21:19:15.356207+0000","last_became_active":"2024-05-22T21:19:14.356993+0000","last_became_peered":"2024-05-22T21:19:14.356993+0000","last_unstale":"2024-05-22T21:19:15.356207+0000","last_undegraded":"2024-05-22T21:19:15.356207+0000","last_fullsized":"2024-05-22T21:19:15.356207+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:08:05.273893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354511+0000","last_change":"2024-05-22T21:19:13.974166+0000","last_active":"2024-05-22T21:19:15.354511+0000","last_peered":"2024-05-22T21:19:15.354511+0000","last_clean":"2024-05-22T21:19:15.354511+0000","last_became_active":"2024-05-22T21:19:13.972043+0000","last_became_peered":"2024-05-22T21:19:13.972043+0000","last_unstale":"2024-05-22T21:19:15.354511+0000","last_undegraded":"2024-05-22T21:19:15.354511+0000","last_fullsized":"2024-05-22T21:19:15.354511+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:26:41.742313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430517+0000","last_change":"2024-05-22T21:19:14.370571+0000","last_active":"2024-05-22T21:19:15.430517+0000","last_peered":"2024-05-22T21:19:15.430517+0000","last_clean":"2024-05-22T21:19:15.430517+0000","last_became_active":"2024-05-22T21:19:14.370108+0000","last_became_peered":"2024-05-22T21:19:14.370108+0000","last_unstale":"2024-05-22T21:19:15.430517+0000","last_undegraded":"2024-05-22T21:19:15.430517+0000","last_fullsized":"2024-05-22T21:19:15.430517+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:05:32.058773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"124'2","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354531+0000","last_change":"2024-05-22T21:19:13.969131+0000","last_active":"2024-05-22T21:19:15.354531+0000","last_peered":"2024-05-22T21:19:15.354531+0000","last_clean":"2024-05-22T21:19:15.354531+0000","last_became_active":"2024-05-22T21:19:13.968790+0000","last_became_peered":"2024-05-22T21:19:13.968790+0000","last_unstale":"2024-05-22T21:19:15.354531+0000","last_undegraded":"2024-05-22T21:19:15.354531+0000","last_fullsized":"2024-05-22T21:19:15.354531+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:15:58.666861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430301+0000","last_change":"2024-05-22T21:19:14.363202+0000","last_active":"2024-05-22T21:19:15.430301+0000","last_peered":"2024-05-22T21:19:15.430301+0000","last_clean":"2024-05-22T21:19:15.430301+0000","last_became_active":"2024-05-22T21:19:14.363044+0000","last_became_peered":"2024-05-22T21:19:14.363044+0000","last_unstale":"2024-05-22T21:19:15.430301+0000","last_undegraded":"2024-05-22T21:19:15.430301+0000","last_fullsized":"2024-05-22T21:19:15.430301+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:41:35.060233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781468+0000","last_change":"2024-05-22T21:19:13.978007+0000","last_active":"2024-05-22T21:19:15.781468+0000","last_peered":"2024-05-22T21:19:15.781468+0000","last_clean":"2024-05-22T21:19:15.781468+0000","last_became_active":"2024-05-22T21:19:13.977780+0000","last_became_peered":"2024-05-22T21:19:13.977780+0000","last_unstale":"2024-05-22T21:19:15.781468+0000","last_undegraded":"2024-05-22T21:19:15.781468+0000","last_fullsized":"2024-05-22T21:19:15.781468+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:26:26.886392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430466+0000","last_change":"2024-05-22T21:19:14.365248+0000","last_active":"2024-05-22T21:19:15.430466+0000","last_peered":"2024-05-22T21:19:15.430466+0000","last_clean":"2024-05-22T21:19:15.430466+0000","last_became_active":"2024-05-22T21:19:14.364991+0000","last_became_peered":"2024-05-22T21:19:14.364991+0000","last_unstale":"2024-05-22T21:19:15.430466+0000","last_undegraded":"2024-05-22T21:19:15.430466+0000","last_fullsized":"2024-05-22T21:19:15.430466+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:31:36.498469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781830+0000","last_change":"2024-05-22T21:19:13.976779+0000","last_active":"2024-05-22T21:19:15.781830+0000","last_peered":"2024-05-22T21:19:15.781830+0000","last_clean":"2024-05-22T21:19:15.781830+0000","last_became_active":"2024-05-22T21:19:13.976541+0000","last_became_peered":"2024-05-22T21:19:13.976541+0000","last_unstale":"2024-05-22T21:19:15.781830+0000","last_undegraded":"2024-05-22T21:19:15.781830+0000","last_fullsized":"2024-05-22T21:19:15.781830+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:53:11.275117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.782435+0000","last_change":"2024-05-22T21:19:14.360039+0000","last_active":"2024-05-22T21:19:15.782435+0000","last_peered":"2024-05-22T21:19:15.782435+0000","last_clean":"2024-05-22T21:19:15.782435+0000","last_became_active":"2024-05-22T21:19:14.359393+0000","last_became_peered":"2024-05-22T21:19:14.359393+0000","last_unstale":"2024-05-22T21:19:15.782435+0000","last_undegraded":"2024-05-22T21:19:15.782435+0000","last_fullsized":"2024-05-22T21:19:15.782435+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:08:05.273893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354299+0000","last_change":"2024-05-22T21:19:13.967940+0000","last_active":"2024-05-22T21:19:15.354299+0000","last_peered":"2024-05-22T21:19:15.354299+0000","last_clean":"2024-05-22T21:19:15.354299+0000","last_became_active":"2024-05-22T21:19:13.967580+0000","last_became_peered":"2024-05-22T21:19:13.967580+0000","last_unstale":"2024-05-22T21:19:15.354299+0000","last_undegraded":"2024-05-22T21:19:15.354299+0000","last_fullsized":"2024-05-22T21:19:15.354299+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:44:53.066660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354217+0000","last_change":"2024-05-22T21:19:14.368425+0000","last_active":"2024-05-22T21:19:15.354217+0000","last_peered":"2024-05-22T21:19:15.354217+0000","last_clean":"2024-05-22T21:19:15.354217+0000","last_became_active":"2024-05-22T21:19:14.367534+0000","last_became_peered":"2024-05-22T21:19:14.367534+0000","last_unstale":"2024-05-22T21:19:15.354217+0000","last_undegraded":"2024-05-22T21:19:15.354217+0000","last_fullsized":"2024-05-22T21:19:15.354217+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:09:35.979308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"124'2","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781793+0000","last_change":"2024-05-22T21:19:13.975255+0000","last_active":"2024-05-22T21:19:15.781793+0000","last_peered":"2024-05-22T21:19:15.781793+0000","last_clean":"2024-05-22T21:19:15.781793+0000","last_became_active":"2024-05-22T21:19:13.974968+0000","last_became_peered":"2024-05-22T21:19:13.974968+0000","last_unstale":"2024-05-22T21:19:15.781793+0000","last_undegraded":"2024-05-22T21:19:15.781793+0000","last_fullsized":"2024-05-22T21:19:15.781793+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:48:35.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781119+0000","last_change":"2024-05-22T21:19:14.358327+0000","last_active":"2024-05-22T21:19:15.781119+0000","last_peered":"2024-05-22T21:19:15.781119+0000","last_clean":"2024-05-22T21:19:15.781119+0000","last_became_active":"2024-05-22T21:19:14.358034+0000","last_became_peered":"2024-05-22T21:19:14.358034+0000","last_unstale":"2024-05-22T21:19:15.781119+0000","last_undegraded":"2024-05-22T21:19:15.781119+0000","last_fullsized":"2024-05-22T21:19:15.781119+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:54:16.959652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354603+0000","last_change":"2024-05-22T21:19:13.968982+0000","last_active":"2024-05-22T21:19:15.354603+0000","last_peered":"2024-05-22T21:19:15.354603+0000","last_clean":"2024-05-22T21:19:15.354603+0000","last_became_active":"2024-05-22T21:19:13.968482+0000","last_became_peered":"2024-05-22T21:19:13.968482+0000","last_unstale":"2024-05-22T21:19:15.354603+0000","last_undegraded":"2024-05-22T21:19:15.354603+0000","last_fullsized":"2024-05-22T21:19:15.354603+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:58:26.400264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781634+0000","last_change":"2024-05-22T21:19:14.364686+0000","last_active":"2024-05-22T21:19:15.781634+0000","last_peered":"2024-05-22T21:19:15.781634+0000","last_clean":"2024-05-22T21:19:15.781634+0000","last_became_active":"2024-05-22T21:19:14.359433+0000","last_became_peered":"2024-05-22T21:19:14.359433+0000","last_unstale":"2024-05-22T21:19:15.781634+0000","last_undegraded":"2024-05-22T21:19:15.781634+0000","last_fullsized":"2024-05-22T21:19:15.781634+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:25:31.495943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354087+0000","last_change":"2024-05-22T21:19:13.969743+0000","last_active":"2024-05-22T21:19:15.354087+0000","last_peered":"2024-05-22T21:19:15.354087+0000","last_clean":"2024-05-22T21:19:15.354087+0000","last_became_active":"2024-05-22T21:19:13.969112+0000","last_became_peered":"2024-05-22T21:19:13.969112+0000","last_unstale":"2024-05-22T21:19:15.354087+0000","last_undegraded":"2024-05-22T21:19:15.354087+0000","last_fullsized":"2024-05-22T21:19:15.354087+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:35:48.944267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781790+0000","last_change":"2024-05-22T21:19:14.367850+0000","last_active":"2024-05-22T21:19:15.781790+0000","last_peered":"2024-05-22T21:19:15.781790+0000","last_clean":"2024-05-22T21:19:15.781790+0000","last_became_active":"2024-05-22T21:19:14.367360+0000","last_became_peered":"2024-05-22T21:19:14.367360+0000","last_unstale":"2024-05-22T21:19:15.781790+0000","last_undegraded":"2024-05-22T21:19:15.781790+0000","last_fullsized":"2024-05-22T21:19:15.781790+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:38:48.877194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.782525+0000","last_change":"2024-05-22T21:19:13.977791+0000","last_active":"2024-05-22T21:19:15.782525+0000","last_peered":"2024-05-22T21:19:15.782525+0000","last_clean":"2024-05-22T21:19:15.782525+0000","last_became_active":"2024-05-22T21:19:13.977568+0000","last_became_peered":"2024-05-22T21:19:13.977568+0000","last_unstale":"2024-05-22T21:19:15.782525+0000","last_undegraded":"2024-05-22T21:19:15.782525+0000","last_fullsized":"2024-05-22T21:19:15.782525+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:41:38.428159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430555+0000","last_change":"2024-05-22T21:19:14.362859+0000","last_active":"2024-05-22T21:19:15.430555+0000","last_peered":"2024-05-22T21:19:15.430555+0000","last_clean":"2024-05-22T21:19:15.430555+0000","last_became_active":"2024-05-22T21:19:14.361771+0000","last_became_peered":"2024-05-22T21:19:14.361771+0000","last_unstale":"2024-05-22T21:19:15.430555+0000","last_undegraded":"2024-05-22T21:19:15.430555+0000","last_fullsized":"2024-05-22T21:19:15.430555+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:28:37.869461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.354355+0000","last_change":"2024-05-22T21:19:13.972676+0000","last_active":"2024-05-22T21:19:15.354355+0000","last_peered":"2024-05-22T21:19:15.354355+0000","last_clean":"2024-05-22T21:19:15.354355+0000","last_became_active":"2024-05-22T21:19:13.971741+0000","last_became_peered":"2024-05-22T21:19:13.971741+0000","last_unstale":"2024-05-22T21:19:15.354355+0000","last_undegraded":"2024-05-22T21:19:15.354355+0000","last_fullsized":"2024-05-22T21:19:15.354355+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:01:15.829792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355772+0000","last_change":"2024-05-22T21:19:14.362534+0000","last_active":"2024-05-22T21:19:15.355772+0000","last_peered":"2024-05-22T21:19:15.355772+0000","last_clean":"2024-05-22T21:19:15.355772+0000","last_became_active":"2024-05-22T21:19:14.362088+0000","last_became_peered":"2024-05-22T21:19:14.362088+0000","last_unstale":"2024-05-22T21:19:15.355772+0000","last_undegraded":"2024-05-22T21:19:15.355772+0000","last_fullsized":"2024-05-22T21:19:15.355772+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:15:59.674570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781913+0000","last_change":"2024-05-22T21:19:13.978831+0000","last_active":"2024-05-22T21:19:15.781913+0000","last_peered":"2024-05-22T21:19:15.781913+0000","last_clean":"2024-05-22T21:19:15.781913+0000","last_became_active":"2024-05-22T21:19:13.978462+0000","last_became_peered":"2024-05-22T21:19:13.978462+0000","last_unstale":"2024-05-22T21:19:15.781913+0000","last_undegraded":"2024-05-22T21:19:15.781913+0000","last_fullsized":"2024-05-22T21:19:15.781913+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:48:53.288659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430565+0000","last_change":"2024-05-22T21:19:14.366194+0000","last_active":"2024-05-22T21:19:15.430565+0000","last_peered":"2024-05-22T21:19:15.430565+0000","last_clean":"2024-05-22T21:19:15.430565+0000","last_became_active":"2024-05-22T21:19:14.365009+0000","last_became_peered":"2024-05-22T21:19:14.365009+0000","last_unstale":"2024-05-22T21:19:15.430565+0000","last_undegraded":"2024-05-22T21:19:15.430565+0000","last_fullsized":"2024-05-22T21:19:15.430565+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:43:11.548823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.780914+0000","last_change":"2024-05-22T21:19:13.974434+0000","last_active":"2024-05-22T21:19:15.780914+0000","last_peered":"2024-05-22T21:19:15.780914+0000","last_clean":"2024-05-22T21:19:15.780914+0000","last_became_active":"2024-05-22T21:19:13.973566+0000","last_became_peered":"2024-05-22T21:19:13.973566+0000","last_unstale":"2024-05-22T21:19:15.780914+0000","last_undegraded":"2024-05-22T21:19:15.780914+0000","last_fullsized":"2024-05-22T21:19:15.780914+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:58:26.400264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.781731+0000","last_change":"2024-05-22T21:19:14.367984+0000","last_active":"2024-05-22T21:19:15.781731+0000","last_peered":"2024-05-22T21:19:15.781731+0000","last_clean":"2024-05-22T21:19:15.781731+0000","last_became_active":"2024-05-22T21:19:14.367393+0000","last_became_peered":"2024-05-22T21:19:14.367393+0000","last_unstale":"2024-05-22T21:19:15.781731+0000","last_undegraded":"2024-05-22T21:19:15.781731+0000","last_fullsized":"2024-05-22T21:19:15.781731+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:29:19.636553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430614+0000","last_change":"2024-05-22T21:19:13.979249+0000","last_active":"2024-05-22T21:19:15.430614+0000","last_peered":"2024-05-22T21:19:15.430614+0000","last_clean":"2024-05-22T21:19:15.430614+0000","last_became_active":"2024-05-22T21:19:13.978885+0000","last_became_peered":"2024-05-22T21:19:13.978885+0000","last_unstale":"2024-05-22T21:19:15.430614+0000","last_undegraded":"2024-05-22T21:19:15.430614+0000","last_fullsized":"2024-05-22T21:19:15.430614+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:24:21.746105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.430492+0000","last_change":"2024-05-22T21:19:14.370527+0000","last_active":"2024-05-22T21:19:15.430492+0000","last_peered":"2024-05-22T21:19:15.430492+0000","last_clean":"2024-05-22T21:19:15.430492+0000","last_became_active":"2024-05-22T21:19:14.370301+0000","last_became_peered":"2024-05-22T21:19:14.370301+0000","last_unstale":"2024-05-22T21:19:15.430492+0000","last_undegraded":"2024-05-22T21:19:15.430492+0000","last_fullsized":"2024-05-22T21:19:15.430492+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:13.957336+0000","last_clean_scrub_stamp":"2024-05-22T21:19:13.957336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:31:00.630803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"124'1","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355757+0000","last_change":"2024-05-22T21:19:13.976882+0000","last_active":"2024-05-22T21:19:15.355757+0000","last_peered":"2024-05-22T21:19:15.355757+0000","last_clean":"2024-05-22T21:19:15.355757+0000","last_became_active":"2024-05-22T21:19:13.976502+0000","last_became_peered":"2024-05-22T21:19:13.976502+0000","last_unstale":"2024-05-22T21:19:15.355757+0000","last_undegraded":"2024-05-22T21:19:15.355757+0000","last_fullsized":"2024-05-22T21:19:15.355757+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:12.949627+0000","last_clean_scrub_stamp":"2024-05-22T21:19:12.949627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:58:26.400264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":271,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-05-22T21:19:15.355849+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:19:15.355849+0000","last_peered":"2024-05-22T21:19:15.355849+0000","last_clean":"2024-05-22T21:19:15.355849+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:19:15.355849+0000","last_undegraded":"2024-05-22T21:19:15.355849+0000","last_fullsized":"2024-05-22T21:19:15.355849+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1250,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194062,"statfs":{"total":200000000,"available":198719875,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70299999999999996,"15min":0.754},"min":{"1min":0.40000000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93700000000000006,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88300000000000001,"5min":1.889,"15min":1.889},"last":0.81799999999999995}]},{"osd":1,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72699999999999998,"15min":0.77000000000000002},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89500000000000002,"5min":3.0920000000000001,"15min":3.0920000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71399999999999997,"15min":0.76000000000000001},"min":{"1min":0.47899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88600000000000001,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.77600000000000002}]},{"osd":2,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69299999999999995,"15min":0.69999999999999996},"min":{"1min":0.503,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81899999999999995,"5min":1.923,"15min":1.923},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.74399999999999999,"15min":0.72699999999999998},"min":{"1min":0.47399999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86099999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.83699999999999997}]},{"osd":3,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.69599999999999995},"min":{"1min":0.53900000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.90600000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.752,"15min":0.79100000000000004},"min":{"1min":0.505,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88600000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.65700000000000003}]},{"osd":4,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.437,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.92000000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.621,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88700000000000001,"5min":1.018,"15min":1.018},"last":0.90900000000000003}]},{"osd":5,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63900000000000001,"15min":0.65500000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84899999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64800000000000002,"15min":0.64200000000000002},"min":{"1min":0.47899999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91000000000000003,"5min":0.996,"15min":0.996},"last":0.70499999999999996}]},{"osd":6,"last update":"Wed May 22 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.71199999999999997},"min":{"1min":0.54700000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85799999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67400000000000004,"15min":0.64400000000000002},"min":{"1min":0.504,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.95299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":674,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194638,"statfs":{"total":200000000,"available":199309703,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.49299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70599999999999996,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.58799999999999997}]},{"osd":1,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95899999999999996,"5min":1.107,"15min":1.107},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.034,"5min":1.081,"15min":1.081},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78300000000000003,"15min":0.75900000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.746,"15min":0.73899999999999999},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.748,"15min":0.74199999999999999},"min":{"1min":0.49399999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0249999999999999,"5min":1.415,"15min":1.415},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.004,"5min":1.788,"15min":1.788},"last":0.60399999999999998}]},{"osd":4,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.41699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.89900000000000002,"5min":1.006,"15min":1.006},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95899999999999996,"5min":2.718,"15min":2.718},"last":0.44400000000000001}]},{"osd":5,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.745,"15min":0.70299999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81799999999999995,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.45500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.42099999999999999}]},{"osd":7,"last update":"Wed May 22 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70999999999999996,"15min":0.73799999999999999},"min":{"1min":0.53600000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0469999999999999,"5min":1.095,"15min":1.095},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72699999999999998,"15min":0.75700000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92500000000000004,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.65100000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":673,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194638,"statfs":{"total":200000000,"available":199310285,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68999999999999995,"15min":0.70399999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.91000000000000003,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69699999999999995,"15min":0.67500000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.96499999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.81899999999999995}]},{"osd":1,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.46500000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.028,"5min":1.216,"15min":1.216},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.44900000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.875,"5min":1.048,"15min":1.048},"last":0.76200000000000001}]},{"osd":2,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.71899999999999997,"15min":0.69899999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91100000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.69799999999999995,"15min":0.67600000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86699999999999999,"5min":1.252,"15min":1.252},"last":0.86099999999999999}]},{"osd":3,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.52800000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91500000000000004,"5min":1.111,"15min":1.111},"last":0.79900000000000004}]},{"osd":4,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.627},"min":{"1min":0.38400000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88800000000000001,"5min":1.272,"15min":1.272},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.629,"15min":0.63200000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.014,"5min":1.038,"15min":1.038},"last":0.57899999999999996}]},{"osd":6,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63500000000000001,"15min":0.63},"min":{"1min":0.41299999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.877,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.442,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.68000000000000005}]},{"osd":7,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93899999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.75900000000000001,"15min":0.79000000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.99399999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.61599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":675,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194637,"statfs":{"total":200000000,"available":199308667,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78200000000000003,"15min":0.748},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.89300000000000002,"5min":6.6539999999999999,"15min":6.6539999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.747,"15min":0.72799999999999998},"min":{"1min":0.49299999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93300000000000005,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":0.84899999999999998}]},{"osd":1,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68200000000000005,"15min":0.71599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88200000000000001,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92200000000000004,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.77300000000000002}]},{"osd":2,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.70199999999999996,"15min":0.68899999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0940000000000001,"5min":1.335,"15min":1.335},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70099999999999996,"15min":0.70399999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95099999999999996,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.54100000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.98999999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76500000000000001,"15min":0.78900000000000003},"min":{"1min":0.53200000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96799999999999997,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.97699999999999998}]},{"osd":5,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.39400000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.93500000000000005,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91900000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.0409999999999999}]},{"osd":6,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65900000000000003,"15min":0.69299999999999995},"min":{"1min":0.46700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91600000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.51800000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.91100000000000003,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.91600000000000004}]},{"osd":7,"last update":"Wed May 22 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.69599999999999995,"15min":0.69999999999999996},"min":{"1min":0.48099999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98199999999999998,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.74099999999999999,"15min":0.78000000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97999999999999998,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.1200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194643,"statfs":{"total":200000000,"available":199315069,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.42999999999999999,"15min":0.40699999999999997},"min":{"1min":0.29699999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.71399999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.442,"15min":0.40799999999999997},"min":{"1min":0.33400000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.49199999999999999}]},{"osd":1,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.46000000000000002,"15min":0.44800000000000001},"min":{"1min":0.28399999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.624,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.625},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71799999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.68600000000000005}]},{"osd":2,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51000000000000001,"15min":0.496},"min":{"1min":0.376,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68999999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51100000000000001,"15min":0.48999999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69899999999999995,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60099999999999998}]},{"osd":4,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.74099999999999999,"15min":0.70599999999999996},"min":{"1min":0.50900000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0209999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.749,"15min":0.71899999999999997},"min":{"1min":0.57599999999999996,"5min":0.435,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1}]},{"osd":5,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.75800000000000001,"15min":0.69899999999999995},"min":{"1min":0.622,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.002,"5min":1.234,"15min":1.234},"last":1.048},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.70299999999999996},"min":{"1min":0.53500000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0449999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.93799999999999994}]},{"osd":6,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.71599999999999997},"min":{"1min":0.48899999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97499999999999998,"5min":1.276,"15min":1.276},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75800000000000001,"15min":0.70799999999999996},"min":{"1min":0.54200000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0760000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.0820000000000001}]},{"osd":7,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.75800000000000001},"min":{"1min":0.52800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.0900000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78500000000000003,"15min":0.74099999999999999},"min":{"1min":0.63300000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0649999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.071}]}]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":673,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194638,"statfs":{"total":200000000,"available":199310321,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.59399999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.504},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.46300000000000002,"15min":0.443},"min":{"1min":0.33200000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.56399999999999995}]},{"osd":1,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.32900000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.624,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54300000000000004,"15min":0.52100000000000002},"min":{"1min":0.442,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.60599999999999998}]},{"osd":3,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.50600000000000001,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83099999999999996,"5min":0.871,"15min":0.871},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.47399999999999998,"15min":0.45900000000000002},"min":{"1min":0.34699999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.68000000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73399999999999999,"15min":0.68999999999999995},"min":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95599999999999996,"5min":1.095,"15min":1.095},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77900000000000003,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.79000000000000004}]},{"osd":5,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.73299999999999998,"15min":0.70699999999999996},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.746},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72599999999999998,"15min":0.70199999999999996},"min":{"1min":0.47899999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92900000000000005,"5min":1.127,"15min":1.127},"last":0.80700000000000005}]},{"osd":6,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73099999999999998,"15min":0.70299999999999996},"min":{"1min":0.52000000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.02,"5min":1.054,"15min":1.054},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69899999999999995,"15min":0.67800000000000005},"min":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.89600000000000002,"5min":1.002,"15min":1.002},"last":0.70699999999999996}]},{"osd":7,"last update":"Wed May 22 21:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.76800000000000002,"15min":0.73199999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.77200000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0249999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.89300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":667,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194645,"statfs":{"total":200000000,"available":199316655,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41599999999999998,"15min":0.42999999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58999999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.46400000000000002,"15min":0.438},"min":{"1min":0.35799999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.67800000000000005,"5min":0.876,"15min":0.876},"last":0.372}]},{"osd":2,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.36099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80500000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.495,"15min":0.53100000000000003},"min":{"1min":0.35199999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64100000000000001,"5min":1.923,"15min":1.923},"last":0.51100000000000001}]},{"osd":3,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53600000000000003,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.497,"15min":0.48899999999999999},"min":{"1min":0.34699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.62,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44400000000000001}]},{"osd":4,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.75900000000000001},"min":{"1min":0.496,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.109,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.72299999999999998,"15min":0.68899999999999995},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0389999999999999,"5min":1.3,"15min":1.3},"last":0.52500000000000002}]},{"osd":5,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.76700000000000002,"15min":0.73599999999999999},"min":{"1min":0.59499999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0169999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.751,"15min":0.753},"min":{"1min":0.621,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97099999999999997,"5min":1.97,"15min":1.97},"last":0.54000000000000004}]},{"osd":6,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.748},"min":{"1min":0.48999999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0009999999999999,"5min":1.272,"15min":1.272},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78600000000000003,"15min":0.75},"min":{"1min":0.57899999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0620000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.61599999999999999}]},{"osd":7,"last update":"Wed May 22 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.76400000000000001,"15min":0.77600000000000002},"min":{"1min":0.63300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98799999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.625},{"interface":"front","average":{"1min":0.754,"5min":0.75600000000000001,"15min":0.77600000000000002},"min":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98899999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.67100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1249,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194062,"statfs":{"total":200000000,"available":198720024,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.42499999999999999,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58099999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.51100000000000001,"15min":0.48499999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69199999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48899999999999999}]},{"osd":2,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.45400000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76200000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.42799999999999999},"min":{"1min":0.29399999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.623,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.442}]},{"osd":3,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45600000000000002,"15min":0.434},"min":{"1min":0.33900000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61699999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53400000000000003,"15min":0.503},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.746,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.54600000000000004}]},{"osd":4,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.71099999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1020000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.57099999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.113,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.86299999999999999}]},{"osd":5,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71899999999999997,"15min":0.70699999999999996},"min":{"1min":0.54200000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.99299999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.76900000000000002,"15min":0.73899999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.143,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.874}]},{"osd":6,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79300000000000004,"15min":0.77300000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1240000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78200000000000003,"15min":0.77200000000000002},"min":{"1min":0.55300000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2090000000000001,"5min":1.5,"15min":1.5},"last":0.84399999999999997}]},{"osd":7,"last update":"Wed May 22 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.75600000000000001,"15min":0.72699999999999998},"min":{"1min":0.57499999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76900000000000002,"15min":0.75900000000000001},"min":{"1min":0.55900000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.0589999999999999,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.72399999999999998}]}]}],"pool_statfs":[]}} 2024-05-22T21:19:18.801 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-22T21:19:18.802 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-22T21:19:18.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.979+0000 7f4c71c00640 1 Processor -- start 2024-05-22T21:19:18.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.979+0000 7f4c71c00640 1 -- start start 2024-05-22T21:19:18.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.979+0000 7f4c71c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4c6c1596b0 0x7f4c6c071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.979+0000 7f4c71c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6c15d250 con 0x7f4c6c06d240 2024-05-22T21:19:18.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.979+0000 7f4c71c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6c159a90 con 0x7f4c6c071db0 2024-05-22T21:19:18.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.979+0000 7f4c71c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6c15d510 con 0x7f4c6c1596b0 2024-05-22T21:19:18.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.979+0000 7f4c6aa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4c6c1596b0 0x7f4c6c071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.979+0000 7f4c6b400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f4c6c06d240 0x7f4c6c0b2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48818/0 (socket says 172.21.15.161:48818) 2024-05-22T21:19:18.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.979+0000 7f4c6b400640 1 -- 172.21.15.161:0/975817121 learned_addr learned my addr 172.21.15.161:0/975817121 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:18.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.979+0000 7f4c6aa00640 1 -- 172.21.15.161:0/975817121 >> v1:172.21.15.179:6789/0 conn(0x7f4c6c071db0 legacy=0x7f4c6c06cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:18.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.979+0000 7f4c6aa00640 1 -- 172.21.15.161:0/975817121 >> v1:172.21.15.161:6789/0 conn(0x7f4c6c06d240 legacy=0x7f4c6c0b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.979+0000 7f4c6aa00640 1 -- 172.21.15.161:0/975817121 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c15d7f0 con 0x7f4c6c1596b0 2024-05-22T21:19:18.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.979+0000 7f4c69600640 1 -- 172.21.15.161:0/975817121 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315285196 0 0) 0x7f4c6c15d250 con 0x7f4c6c06d240 2024-05-22T21:19:18.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.983+0000 7f4c6aa00640 1 --2- 172.21.15.161:0/975817121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c6c1596b0 0x7f4c6c071870 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0097c0 tx=0x7f4c5c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16d8005322d49d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:18.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.983+0000 7f4c69600640 1 -- 172.21.15.161:0/975817121 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c0ab600 con 0x7f4c6c1596b0 2024-05-22T21:19:18.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.983+0000 7f4c69600640 1 -- 172.21.15.161:0/975817121 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5c0a7050 con 0x7f4c6c1596b0 2024-05-22T21:19:18.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.983+0000 7f4c69600640 1 -- 172.21.15.161:0/975817121 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c0ab980 con 0x7f4c6c1596b0 2024-05-22T21:19:18.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.983+0000 7f4c71c00640 1 -- 172.21.15.161:0/975817121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c6c1596b0 msgr2=0x7f4c6c071870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.983+0000 7f4c71c00640 1 --2- 172.21.15.161:0/975817121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c6c1596b0 0x7f4c6c071870 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0097c0 tx=0x7f4c5c01c460 comp rx=0 tx=0).stop 2024-05-22T21:19:18.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.983+0000 7f4c71c00640 1 -- 172.21.15.161:0/975817121 shutdown_connections 2024-05-22T21:19:18.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.983+0000 7f4c71c00640 1 --2- 172.21.15.161:0/975817121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c6c1596b0 0x7f4c6c071870 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.983+0000 7f4c71c00640 1 -- 172.21.15.161:0/975817121 >> 172.21.15.161:0/975817121 conn(0x7f4c6c06a2d0 msgr2=0x7f4c6c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:18.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.983+0000 7f4c71c00640 1 -- 172.21.15.161:0/975817121 shutdown_connections 2024-05-22T21:19:18.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.983+0000 7f4c71c00640 1 -- 172.21.15.161:0/975817121 wait complete. 2024-05-22T21:19:18.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c71c00640 1 Processor -- start 2024-05-22T21:19:18.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c71c00640 1 -- start start 2024-05-22T21:19:18.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c71c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4c6c06d240 0x7f4c6c1456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c71c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c6c071db0 0x7f4c6c15c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c71c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c6c1596b0 0x7f4c6c06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c71c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c15ac30 con 0x7f4c6c1596b0 2024-05-22T21:19:18.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c71c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c15a020 con 0x7f4c6c071db0 2024-05-22T21:19:18.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c71c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c15a300 con 0x7f4c6c06d240 2024-05-22T21:19:18.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c6b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c6c1596b0 0x7f4c6c06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c6b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c6c1596b0 0x7f4c6c06c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44744/0 (socket says 172.21.15.161:44744) 2024-05-22T21:19:18.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c6b400640 1 -- 172.21.15.161:0/3828007491 learned_addr learned my addr 172.21.15.161:0/3828007491 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:18.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c6a000640 1 --2- 172.21.15.161:0/3828007491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c6c071db0 0x7f4c6c15c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c6aa00640 1 --2- 172.21.15.161:0/3828007491 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4c6c06d240 0x7f4c6c1456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c6b400640 1 -- 172.21.15.161:0/3828007491 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4c6c06d240 msgr2=0x7f4c6c1456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c6b400640 1 --2- 172.21.15.161:0/3828007491 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4c6c06d240 0x7f4c6c1456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c6b400640 1 -- 172.21.15.161:0/3828007491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c6c071db0 msgr2=0x7f4c6c15c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:18.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c6b400640 1 --2- 172.21.15.161:0/3828007491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c6c071db0 0x7f4c6c15c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:18.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c6b400640 1 -- 172.21.15.161:0/3828007491 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5c009420 con 0x7f4c6c1596b0 2024-05-22T21:19:18.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c6a000640 1 --2- 172.21.15.161:0/3828007491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c6c071db0 0x7f4c6c15c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:18.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c6b400640 1 --2- 172.21.15.161:0/3828007491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c6c1596b0 0x7f4c6c06c2a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4c64001570 tx=0x7f4c6400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:18.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c62a00640 1 -- 172.21.15.161:0/3828007491 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6400a640 con 0x7f4c6c1596b0 2024-05-22T21:19:18.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c62a00640 1 -- 172.21.15.161:0/3828007491 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6400a7c0 con 0x7f4c6c1596b0 2024-05-22T21:19:18.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c71c00640 1 -- 172.21.15.161:0/3828007491 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6c15a5e0 con 0x7f4c6c1596b0 2024-05-22T21:19:18.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c62a00640 1 -- 172.21.15.161:0/3828007491 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6400ab70 con 0x7f4c6c1596b0 2024-05-22T21:19:18.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c71c00640 1 -- 172.21.15.161:0/3828007491 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6c159cd0 con 0x7f4c6c1596b0 2024-05-22T21:19:18.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.987+0000 7f4c71c00640 1 -- 172.21.15.161:0/3828007491 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6c15ada0 con 0x7f4c6c1596b0 2024-05-22T21:19:18.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.991+0000 7f4c62a00640 1 -- 172.21.15.161:0/3828007491 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4c6400acd0 con 0x7f4c6c1596b0 2024-05-22T21:19:18.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.991+0000 7f4c62a00640 1 --2- 172.21.15.161:0/3828007491 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4c4c066690 0x7f4c4c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:18.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.991+0000 7f4c62a00640 1 -- 172.21.15.161:0/3828007491 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c64083a60 con 0x7f4c6c1596b0 2024-05-22T21:19:18.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.991+0000 7f4c6aa00640 1 --2- 172.21.15.161:0/3828007491 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4c4c066690 0x7f4c4c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:18.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.991+0000 7f4c6aa00640 1 --2- 172.21.15.161:0/3828007491 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4c4c066690 0x7f4c4c068b50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0ad000 tx=0x7f4c5c0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:18.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:18.995+0000 7f4c62a00640 1 -- 172.21.15.161:0/3828007491 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4c64050dc0 con 0x7f4c6c1596b0 2024-05-22T21:19:19.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.199+0000 7f4c71c00640 1 -- 172.21.15.161:0/3828007491 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4c6c067220 con 0x7f4c6c1596b0 2024-05-22T21:19:19.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.203+0000 7f4c62a00640 1 -- 172.21.15.161:0/3828007491 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f4c64018020 con 0x7f4c6c1596b0 2024-05-22T21:19:19.208 INFO:teuthology.orchestra.run.smithi161.stderr:updated caps for client.0 2024-05-22T21:19:19.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.207+0000 7f4c71c00640 1 -- 172.21.15.161:0/3828007491 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4c4c066690 msgr2=0x7f4c4c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:19.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.207+0000 7f4c71c00640 1 --2- 172.21.15.161:0/3828007491 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4c4c066690 0x7f4c4c068b50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0ad000 tx=0x7f4c5c0a6000 comp rx=0 tx=0).stop 2024-05-22T21:19:19.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.207+0000 7f4c71c00640 1 -- 172.21.15.161:0/3828007491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c6c1596b0 msgr2=0x7f4c6c06c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:19.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.207+0000 7f4c71c00640 1 --2- 172.21.15.161:0/3828007491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c6c1596b0 0x7f4c6c06c2a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4c64001570 tx=0x7f4c6400cca0 comp rx=0 tx=0).stop 2024-05-22T21:19:19.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.207+0000 7f4c71c00640 1 -- 172.21.15.161:0/3828007491 shutdown_connections 2024-05-22T21:19:19.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.207+0000 7f4c71c00640 1 --2- 172.21.15.161:0/3828007491 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4c4c066690 0x7f4c4c068b50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:19.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.207+0000 7f4c71c00640 1 --2- 172.21.15.161:0/3828007491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c6c1596b0 0x7f4c6c06c2a0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:19.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.207+0000 7f4c71c00640 1 --2- 172.21.15.161:0/3828007491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c6c071db0 0x7f4c6c15c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:19.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.207+0000 7f4c71c00640 1 --2- 172.21.15.161:0/3828007491 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4c6c06d240 0x7f4c6c1456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:19.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.207+0000 7f4c71c00640 1 -- 172.21.15.161:0/3828007491 >> 172.21.15.161:0/3828007491 conn(0x7f4c6c06a2d0 msgr2=0x7f4c6c0cd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:19.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.207+0000 7f4c71c00640 1 -- 172.21.15.161:0/3828007491 shutdown_connections 2024-05-22T21:19:19.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.207+0000 7f4c71c00640 1 -- 172.21.15.161:0/3828007491 wait complete. 2024-05-22T21:19:19.228 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:19.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.363+0000 7f18cbc00640 1 Processor -- start 2024-05-22T21:19:19.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.363+0000 7f18cbc00640 1 -- start start 2024-05-22T21:19:19.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.363+0000 7f18cbc00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f18c4146c10 0x7f18c40cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:19.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.363+0000 7f18cbc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f18c40717c0 0x7f18c4071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:19.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.363+0000 7f18cbc00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c40b5090 con 0x7f18c415e4b0 2024-05-22T21:19:19.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.363+0000 7f18cbc00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c4072190 con 0x7f18c40717c0 2024-05-22T21:19:19.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.363+0000 7f18cbc00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c4146ff0 con 0x7f18c4146c10 2024-05-22T21:19:19.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.363+0000 7f18c9c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f18c40717c0 0x7f18c4071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:19.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.363+0000 7f18c9c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f18c40717c0 0x7f18c4071c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44758/0 (socket says 172.21.15.161:44758) 2024-05-22T21:19:19.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.363+0000 7f18c9c00640 1 -- 172.21.15.161:0/3489860964 learned_addr learned my addr 172.21.15.161:0/3489860964 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:19.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.363+0000 7f18c9200640 1 --2- 172.21.15.161:0/3489860964 >> v2:172.21.15.179:3300/0 conn(0x7f18c4146c10 0x7f18c40cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:19.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.363+0000 7f18c9c00640 1 -- 172.21.15.161:0/3489860964 >> v2:172.21.15.179:3300/0 conn(0x7f18c4146c10 msgr2=0x7f18c40cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:19.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.363+0000 7f18c9c00640 1 --2- 172.21.15.161:0/3489860964 >> v2:172.21.15.179:3300/0 conn(0x7f18c4146c10 0x7f18c40cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:19.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.363+0000 7f18c9c00640 1 -- 172.21.15.161:0/3489860964 >> v1:172.21.15.179:6789/0 conn(0x7f18c415e4b0 legacy=0x7f18c40b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:19.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.363+0000 7f18c9c00640 1 -- 172.21.15.161:0/3489860964 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c415d9d0 con 0x7f18c40717c0 2024-05-22T21:19:19.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.367+0000 7f18c9c00640 1 --2- 172.21.15.161:0/3489860964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18c40717c0 0x7f18c4071c50 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f18c406ab90 tx=0x7f18bc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43838517d120cb24 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:19.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.371+0000 7f18c3400640 1 -- 172.21.15.161:0/3489860964 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0a56c0 con 0x7f18c40717c0 2024-05-22T21:19:19.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.371+0000 7f18c3400640 1 -- 172.21.15.161:0/3489860964 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18bc0a5820 con 0x7f18c40717c0 2024-05-22T21:19:19.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.371+0000 7f18c3400640 1 -- 172.21.15.161:0/3489860964 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0a5bb0 con 0x7f18c40717c0 2024-05-22T21:19:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.371+0000 7f18cbc00640 1 -- 172.21.15.161:0/3489860964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18c40717c0 msgr2=0x7f18c4071c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.371+0000 7f18cbc00640 1 --2- 172.21.15.161:0/3489860964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18c40717c0 0x7f18c4071c50 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f18c406ab90 tx=0x7f18bc01c9f0 comp rx=0 tx=0).stop 2024-05-22T21:19:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.371+0000 7f18cbc00640 1 -- 172.21.15.161:0/3489860964 shutdown_connections 2024-05-22T21:19:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.371+0000 7f18cbc00640 1 --2- 172.21.15.161:0/3489860964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18c40717c0 0x7f18c4071c50 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.371+0000 7f18cbc00640 1 --2- 172.21.15.161:0/3489860964 >> v2:172.21.15.179:3300/0 conn(0x7f18c4146c10 0x7f18c40cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.371+0000 7f18cbc00640 1 -- 172.21.15.161:0/3489860964 >> 172.21.15.161:0/3489860964 conn(0x7f18c40b1c40 msgr2=0x7f18c40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.371+0000 7f18cbc00640 1 -- 172.21.15.161:0/3489860964 shutdown_connections 2024-05-22T21:19:19.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.371+0000 7f18cbc00640 1 -- 172.21.15.161:0/3489860964 wait complete. 2024-05-22T21:19:19.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18cbc00640 1 Processor -- start 2024-05-22T21:19:19.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18cbc00640 1 -- start start 2024-05-22T21:19:19.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18cbc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18c40717c0 0x7f18c413f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18cbc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18c4146c10 0x7f18c40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18cbc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18c415e4b0 0x7f18c40ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18cbc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c415cd50 con 0x7f18c415e4b0 2024-05-22T21:19:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18cbc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c40ad290 con 0x7f18c4146c10 2024-05-22T21:19:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18cbc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c40addb0 con 0x7f18c40717c0 2024-05-22T21:19:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18c9c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18c415e4b0 0x7f18c40ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18c9c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18c415e4b0 0x7f18c40ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44772/0 (socket says 172.21.15.161:44772) 2024-05-22T21:19:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18c9c00640 1 -- 172.21.15.161:0/3356594326 learned_addr learned my addr 172.21.15.161:0/3356594326 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18c9200640 1 --2- 172.21.15.161:0/3356594326 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18c40717c0 0x7f18c413f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18c3e00640 1 --2- 172.21.15.161:0/3356594326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18c4146c10 0x7f18c40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18c9c00640 1 -- 172.21.15.161:0/3356594326 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18c40717c0 msgr2=0x7f18c413f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18c9c00640 1 --2- 172.21.15.161:0/3356594326 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18c40717c0 0x7f18c413f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:19.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18c9c00640 1 -- 172.21.15.161:0/3356594326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18c4146c10 msgr2=0x7f18c40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:19.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18c9c00640 1 --2- 172.21.15.161:0/3356594326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18c4146c10 0x7f18c40ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:19.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18c9c00640 1 -- 172.21.15.161:0/3356594326 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18bc009b30 con 0x7f18c415e4b0 2024-05-22T21:19:19.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18c9c00640 1 --2- 172.21.15.161:0/3356594326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18c415e4b0 0x7f18c40ae540 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f18bc019600 tx=0x7f18bc019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:19.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18c0c00640 1 -- 172.21.15.161:0/3356594326 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0b8670 con 0x7f18c415e4b0 2024-05-22T21:19:19.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18c0c00640 1 -- 172.21.15.161:0/3356594326 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18bc0b87d0 con 0x7f18c415e4b0 2024-05-22T21:19:19.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18c0c00640 1 -- 172.21.15.161:0/3356594326 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0b8b80 con 0x7f18c415e4b0 2024-05-22T21:19:19.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18cbc00640 1 -- 172.21.15.161:0/3356594326 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c40ad510 con 0x7f18c415e4b0 2024-05-22T21:19:19.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.375+0000 7f18cbc00640 1 -- 172.21.15.161:0/3356594326 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c40adad0 con 0x7f18c415e4b0 2024-05-22T21:19:19.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.379+0000 7f18cbc00640 1 -- 172.21.15.161:0/3356594326 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c406a9a0 con 0x7f18c415e4b0 2024-05-22T21:19:19.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.379+0000 7f18c0c00640 1 -- 172.21.15.161:0/3356594326 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f18bc0b8ce0 con 0x7f18c415e4b0 2024-05-22T21:19:19.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.379+0000 7f18c0c00640 1 --2- 172.21.15.161:0/3356594326 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f18a4066690 0x7f18a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:19.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.379+0000 7f18c0c00640 1 -- 172.21.15.161:0/3356594326 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18bc11f5a0 con 0x7f18c415e4b0 2024-05-22T21:19:19.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.379+0000 7f18c9200640 1 --2- 172.21.15.161:0/3356594326 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f18a4066690 0x7f18a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:19.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.379+0000 7f18c9200640 1 --2- 172.21.15.161:0/3356594326 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f18a4066690 0x7f18a4068b50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f18b800b930 tx=0x7f18b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:19.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.383+0000 7f18c0c00640 1 -- 172.21.15.161:0/3356594326 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f18bc0ecac0 con 0x7f18c415e4b0 2024-05-22T21:19:19.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.571+0000 7f18cbc00640 1 -- 172.21.15.161:0/3356594326 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18c4067220 con 0x7f18c415e4b0 2024-05-22T21:19:19.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.571+0000 7f18c0c00640 1 -- 172.21.15.161:0/3356594326 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f18bc0f00e0 con 0x7f18c415e4b0 2024-05-22T21:19:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:19.574 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3944033857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3944033857},{"type":"v1","addr":"172.21.15.179:6834","nonce":3944033857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6125,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/4216709187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4216709187},{"type":"v1","addr":"172.21.15.161:6836","nonce":4216709187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2364917653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2364917653},{"type":"v1","addr":"172.21.15.179:6835","nonce":2364917653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:14.345936+0000","modified":"2024-05-22T21:19:17.355628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/1424863663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1424863663},{"type":"v1","addr":"172.21.15.161:6837","nonce":1424863663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-22T21:19:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 54 2024-05-22T21:19:19.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.575+0000 7f18cbc00640 1 -- 172.21.15.161:0/3356594326 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f18a4066690 msgr2=0x7f18a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:19.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.575+0000 7f18cbc00640 1 --2- 172.21.15.161:0/3356594326 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f18a4066690 0x7f18a4068b50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f18b800b930 tx=0x7f18b8008040 comp rx=0 tx=0).stop 2024-05-22T21:19:19.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.575+0000 7f18cbc00640 1 -- 172.21.15.161:0/3356594326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18c415e4b0 msgr2=0x7f18c40ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:19.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.575+0000 7f18cbc00640 1 --2- 172.21.15.161:0/3356594326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18c415e4b0 0x7f18c40ae540 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f18bc019600 tx=0x7f18bc019790 comp rx=0 tx=0).stop 2024-05-22T21:19:19.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.575+0000 7f18cbc00640 1 -- 172.21.15.161:0/3356594326 shutdown_connections 2024-05-22T21:19:19.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.575+0000 7f18cbc00640 1 --2- 172.21.15.161:0/3356594326 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f18a4066690 0x7f18a4068b50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:19.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.575+0000 7f18cbc00640 1 --2- 172.21.15.161:0/3356594326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18c415e4b0 0x7f18c40ae540 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:19.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.575+0000 7f18cbc00640 1 --2- 172.21.15.161:0/3356594326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18c4146c10 0x7f18c40ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:19.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.575+0000 7f18cbc00640 1 --2- 172.21.15.161:0/3356594326 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18c40717c0 0x7f18c413f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:19.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.575+0000 7f18cbc00640 1 -- 172.21.15.161:0/3356594326 >> 172.21.15.161:0/3356594326 conn(0x7f18c40b1c40 msgr2=0x7f18c406a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:19.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.575+0000 7f18cbc00640 1 -- 172.21.15.161:0/3356594326 shutdown_connections 2024-05-22T21:19:19.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:19.575+0000 7f18cbc00640 1 -- 172.21.15.161:0/3356594326 wait complete. 2024-05-22T21:19:19.997 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-22T21:19:14.345936+0000', 'modified': '2024-05-22T21:19:17.355628+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7146}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7146': {'gid': 7146, 'name': 'a', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6837/1424863663', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 1424863663}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 1424863663}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-22T21:19:19.997 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-22T21:19:19.998 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-22T21:19:20.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.131+0000 7fe18be00640 1 Processor -- start 2024-05-22T21:19:20.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.131+0000 7fe18be00640 1 -- start start 2024-05-22T21:19:20.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.131+0000 7fe18be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe18c15b050 0x7fe18c0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.131+0000 7fe18be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe18c0b0d80 0x7fe18c0cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.131+0000 7fe18be00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18c0cdd20 con 0x7fe18c0d0ae0 2024-05-22T21:19:20.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.131+0000 7fe18be00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe18c0ce000 con 0x7fe18c0b0d80 2024-05-22T21:19:20.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.131+0000 7fe18be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0ce2e0 con 0x7fe18c15b050 2024-05-22T21:19:20.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.135+0000 7fe18b400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe18c0d0ae0 0x7fe18c0d0f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48824/0 (socket says 172.21.15.161:48824) 2024-05-22T21:19:20.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.135+0000 7fe18b400640 1 -- 172.21.15.161:0/3967949312 learned_addr learned my addr 172.21.15.161:0/3967949312 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:20.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.135+0000 7fe182000640 1 --2- 172.21.15.161:0/3967949312 >> v2:172.21.15.161:3300/0 conn(0x7fe18c0b0d80 0x7fe18c0cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.135+0000 7fe18aa00640 1 --2- 172.21.15.161:0/3967949312 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe18c15b050 0x7fe18c0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.135+0000 7fe182000640 1 -- 172.21.15.161:0/3967949312 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe18c15b050 msgr2=0x7fe18c0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:20.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.135+0000 7fe182000640 1 --2- 172.21.15.161:0/3967949312 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe18c15b050 0x7fe18c0cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.135+0000 7fe182000640 1 -- 172.21.15.161:0/3967949312 >> v1:172.21.15.161:6789/0 conn(0x7fe18c0d0ae0 legacy=0x7fe18c0d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:20.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.135+0000 7fe182000640 1 -- 172.21.15.161:0/3967949312 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c0ce5c0 con 0x7fe18c0b0d80 2024-05-22T21:19:20.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.135+0000 7fe18a000640 1 -- 172.21.15.161:0/3967949312 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1864465291 0 0) 0x7fe18c0cdd20 con 0x7fe18c0d0ae0 2024-05-22T21:19:20.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.135+0000 7fe182000640 1 --2- 172.21.15.161:0/3967949312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe18c0b0d80 0x7fe18c0cf860 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fe1740097c0 tx=0x7fe17401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4980ada6180f75fa server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:20.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.135+0000 7fe18a000640 1 -- 172.21.15.161:0/3967949312 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1740ac600 con 0x7fe18c0b0d80 2024-05-22T21:19:20.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.135+0000 7fe18a000640 1 -- 172.21.15.161:0/3967949312 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1740a8050 con 0x7fe18c0b0d80 2024-05-22T21:19:20.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.135+0000 7fe18a000640 1 -- 172.21.15.161:0/3967949312 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1740ac980 con 0x7fe18c0b0d80 2024-05-22T21:19:20.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18be00640 1 -- 172.21.15.161:0/3967949312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe18c0b0d80 msgr2=0x7fe18c0cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:20.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18be00640 1 --2- 172.21.15.161:0/3967949312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe18c0b0d80 0x7fe18c0cf860 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fe1740097c0 tx=0x7fe17401c380 comp rx=0 tx=0).stop 2024-05-22T21:19:20.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18be00640 1 -- 172.21.15.161:0/3967949312 shutdown_connections 2024-05-22T21:19:20.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18be00640 1 --2- 172.21.15.161:0/3967949312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe18c0b0d80 0x7fe18c0cf860 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18be00640 1 --2- 172.21.15.161:0/3967949312 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe18c15b050 0x7fe18c0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18be00640 1 -- 172.21.15.161:0/3967949312 >> 172.21.15.161:0/3967949312 conn(0x7fe18c099a40 msgr2=0x7fe18c099e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:20.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18be00640 1 -- 172.21.15.161:0/3967949312 shutdown_connections 2024-05-22T21:19:20.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18be00640 1 -- 172.21.15.161:0/3967949312 wait complete. 2024-05-22T21:19:20.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18be00640 1 Processor -- start 2024-05-22T21:19:20.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18be00640 1 -- start start 2024-05-22T21:19:20.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe18c0b0d80 0x7fe18c0c4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe18c0d0ae0 0x7fe18c147170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe18c15b050 0x7fe18c0ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18be00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c157140 con 0x7fe18c0b0d80 2024-05-22T21:19:20.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18be00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c156930 con 0x7fe18c0d0ae0 2024-05-22T21:19:20.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0c6470 con 0x7fe18c15b050 2024-05-22T21:19:20.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe18c0b0d80 0x7fe18c0c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe18c0b0d80 0x7fe18c0c4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44800/0 (socket says 172.21.15.161:44800) 2024-05-22T21:19:20.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18aa00640 1 -- 172.21.15.161:0/1795532084 learned_addr learned my addr 172.21.15.161:0/1795532084 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:20.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18b400640 1 --2- 172.21.15.161:0/1795532084 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe18c15b050 0x7fe18c0ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe182000640 1 --2- 172.21.15.161:0/1795532084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe18c0d0ae0 0x7fe18c147170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18aa00640 1 -- 172.21.15.161:0/1795532084 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe18c15b050 msgr2=0x7fe18c0ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:20.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18aa00640 1 --2- 172.21.15.161:0/1795532084 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe18c15b050 0x7fe18c0ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18aa00640 1 -- 172.21.15.161:0/1795532084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe18c0d0ae0 msgr2=0x7fe18c147170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:20.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18aa00640 1 --2- 172.21.15.161:0/1795532084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe18c0d0ae0 0x7fe18c147170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18aa00640 1 -- 172.21.15.161:0/1795532084 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe174009420 con 0x7fe18c0b0d80 2024-05-22T21:19:20.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18b400640 1 --2- 172.21.15.161:0/1795532084 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe18c15b050 0x7fe18c0ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:20.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe182000640 1 --2- 172.21.15.161:0/1795532084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe18c0d0ae0 0x7fe18c147170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:20.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.139+0000 7fe18aa00640 1 --2- 172.21.15.161:0/1795532084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe18c0b0d80 0x7fe18c0c4980 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe17c007e70 tx=0x7fe17c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:20.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.143+0000 7fe183400640 1 -- 172.21.15.161:0/1795532084 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c014620 con 0x7fe18c0b0d80 2024-05-22T21:19:20.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.143+0000 7fe183400640 1 -- 172.21.15.161:0/1795532084 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17c014780 con 0x7fe18c0b0d80 2024-05-22T21:19:20.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.143+0000 7fe183400640 1 -- 172.21.15.161:0/1795532084 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c014b30 con 0x7fe18c0b0d80 2024-05-22T21:19:20.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.143+0000 7fe18be00640 1 -- 172.21.15.161:0/1795532084 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18c156be0 con 0x7fe18c0b0d80 2024-05-22T21:19:20.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.143+0000 7fe18be00640 1 -- 172.21.15.161:0/1795532084 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c0c69d0 con 0x7fe18c0b0d80 2024-05-22T21:19:20.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.143+0000 7fe183400640 1 -- 172.21.15.161:0/1795532084 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe17c014c90 con 0x7fe18c0b0d80 2024-05-22T21:19:20.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.143+0000 7fe183400640 1 --2- 172.21.15.161:0/1795532084 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe168066690 0x7fe168068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.143+0000 7fe183400640 1 -- 172.21.15.161:0/1795532084 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe17c084b70 con 0x7fe18c0b0d80 2024-05-22T21:19:20.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.143+0000 7fe182000640 1 --2- 172.21.15.161:0/1795532084 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe168066690 0x7fe168068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.143+0000 7fe1949ab000 1 Processor -- start 2024-05-22T21:19:20.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.143+0000 7fe1949ab000 1 -- start start 2024-05-22T21:19:20.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.143+0000 7fe182000640 1 --2- 172.21.15.161:0/1795532084 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe168066690 0x7fe168068b50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe1740ae000 tx=0x7fe174008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:20.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f6956da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f69572e0 0x55b4f690c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f690d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f67d4e80 con 0x55b4f69569c0 2024-05-22T21:19:20.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f6762af0 con 0x55b4f690cec0 2024-05-22T21:19:20.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b4f67600c0 con 0x55b4f69572e0 2024-05-22T21:19:20.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe18aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f6956da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe18aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f6956da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44806/0 (socket says 172.21.15.161:44806) 2024-05-22T21:19:20.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe18aa00640 1 -- 172.21.15.161:0/1438015427 learned_addr learned my addr 172.21.15.161:0/1438015427 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:20.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe18b400640 1 --2- 172.21.15.161:0/1438015427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f690d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe182000640 1 --2- 172.21.15.161:0/1438015427 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f69572e0 0x55b4f690c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe18aa00640 1 -- 172.21.15.161:0/1438015427 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f69572e0 msgr2=0x55b4f690c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:20.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe18aa00640 1 --2- 172.21.15.161:0/1438015427 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f69572e0 0x55b4f690c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe18aa00640 1 -- 172.21.15.161:0/1438015427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 msgr2=0x55b4f690d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:20.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe18aa00640 1 --2- 172.21.15.161:0/1438015427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f690d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe18aa00640 1 -- 172.21.15.161:0/1438015427 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4f6457890 con 0x55b4f69569c0 2024-05-22T21:19:20.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe18aa00640 1 --2- 172.21.15.161:0/1438015427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f6956da0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe17c05d9c0 tx=0x7fe17c055770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b4be7e25bd5583 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:20.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe17b400640 1 -- 172.21.15.161:0/1438015427 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c05def0 con 0x55b4f69569c0 2024-05-22T21:19:20.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe17b400640 1 -- 172.21.15.161:0/1438015427 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17c053ea0 con 0x55b4f69569c0 2024-05-22T21:19:20.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe17b400640 1 -- 172.21.15.161:0/1438015427 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c055d60 con 0x55b4f69569c0 2024-05-22T21:19:20.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 -- 172.21.15.161:0/1438015427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 msgr2=0x55b4f6956da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:20.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 --2- 172.21.15.161:0/1438015427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f6956da0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe17c05d9c0 tx=0x7fe17c055770 comp rx=0 tx=0).stop 2024-05-22T21:19:20.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 -- 172.21.15.161:0/1438015427 shutdown_connections 2024-05-22T21:19:20.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 --2- 172.21.15.161:0/1438015427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f690d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 --2- 172.21.15.161:0/1438015427 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f69572e0 0x55b4f690c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 --2- 172.21.15.161:0/1438015427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f6956da0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 -- 172.21.15.161:0/1438015427 >> 172.21.15.161:0/1438015427 conn(0x55b4f6913440 msgr2=0x55b4f690b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:20.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 -- 172.21.15.161:0/1438015427 shutdown_connections 2024-05-22T21:19:20.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 -- 172.21.15.161:0/1438015427 wait complete. 2024-05-22T21:19:20.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 -- start start 2024-05-22T21:19:20.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.147+0000 7fe1949ab000 1 Processor -- start 2024-05-22T21:19:20.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe1949ab000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f6956200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe1949ab000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f69576c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe1949ab000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f69572e0 0x55b4f6957c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe1949ab000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f675b9b0 con 0x55b4f69569c0 2024-05-22T21:19:20.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe1949ab000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f66fac00 con 0x55b4f690cec0 2024-05-22T21:19:20.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe1949ab000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b4f6719070 con 0x55b4f69572e0 2024-05-22T21:19:20.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe1949ab000 10 client.0 _ll_register_callbacks cb 0x55b4f68f8ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:19:20.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe182000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f69576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe182000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f69576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44808/0 (socket says 172.21.15.161:44808) 2024-05-22T21:19:20.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe182000640 1 -- 172.21.15.161:0/1515443675 learned_addr learned my addr 172.21.15.161:0/1515443675 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:20.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe18b400640 1 --2- 172.21.15.161:0/1515443675 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f69572e0 0x55b4f6957c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe18aa00640 1 --2- 172.21.15.161:0/1515443675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f6956200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe182000640 1 -- 172.21.15.161:0/1515443675 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f69572e0 msgr2=0x55b4f6957c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:20.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe182000640 1 --2- 172.21.15.161:0/1515443675 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f69572e0 0x55b4f6957c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe182000640 1 -- 172.21.15.161:0/1515443675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 msgr2=0x55b4f6956200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:20.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe182000640 1 --2- 172.21.15.161:0/1515443675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f6956200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe182000640 1 -- 172.21.15.161:0/1515443675 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c0c69d0 con 0x55b4f69569c0 2024-05-22T21:19:20.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe182000640 1 --2- 172.21.15.161:0/1515443675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f69576c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe174000c10 tx=0x7fe17401abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f5650c21a7c466 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:20.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe16fe00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:19:20.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe16fe00640 1 -- 172.21.15.161:0/1515443675 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1740b5b10 con 0x55b4f69569c0 2024-05-22T21:19:20.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe16fe00640 1 -- 172.21.15.161:0/1515443675 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1740b5c70 con 0x55b4f69569c0 2024-05-22T21:19:20.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe1949ab000 1 -- 172.21.15.161:0/1515443675 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4f67185f0 con 0x55b4f69569c0 2024-05-22T21:19:20.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe16fe00640 1 -- 172.21.15.161:0/1515443675 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1740b5e30 con 0x55b4f69569c0 2024-05-22T21:19:20.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe1949ab000 10 client.7281 fetch_fsmap learned FSMap version 54 2024-05-22T21:19:20.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe1949ab000 1 -- 172.21.15.161:0/1515443675 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55b4f6720ab0 con 0x55b4f69569c0 2024-05-22T21:19:20.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe16fe00640 1 -- 172.21.15.161:0/1515443675 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b4f67185f0 con 0x55b4f69569c0 2024-05-22T21:19:20.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe16fe00640 1 -- 172.21.15.161:0/1515443675 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 54) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe1740c2440 con 0x55b4f69569c0 2024-05-22T21:19:20.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe1949ab000 10 client.7281 fetch_fsmap finished waiting for FSMap version 54 2024-05-22T21:19:20.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe1949ab000 10 client.7281 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-22T21:19:20.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe1949ab000 20 client.7281 populate_metadata read hostname 'smithi161' 2024-05-22T21:19:20.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe1949ab000 1 --2- 172.21.15.161:0/1515443675 >> [v2:172.21.15.161:6834/1424863663,v1:172.21.15.161:6837/1424863663] conn(0x55b4f6958610 0x55b4f69589f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe1949ab000 4 client.7281 mds_command: new command op to 7146 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T21:19:20.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe1949ab000 1 -- 172.21.15.161:0/1515443675 --> [v2:172.21.15.161:6834/1424863663,v1:172.21.15.161:6837/1424863663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4f6758790 con 0x55b4f6958610 2024-05-22T21:19:20.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe18aa00640 1 --2- 172.21.15.161:0/1515443675 >> [v2:172.21.15.161:6834/1424863663,v1:172.21.15.161:6837/1424863663] conn(0x55b4f6958610 0x55b4f69589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe18aa00640 1 --2- 172.21.15.161:0/1515443675 >> [v2:172.21.15.161:6834/1424863663,v1:172.21.15.161:6837/1424863663] conn(0x55b4f6958610 0x55b4f69589f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95aa5ab46faafabd server_cookie=50719b78e3673517 in_seq=0 out_seq=0 2024-05-22T21:19:20.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe16fe00640 10 client.7281 ms_handle_connect on v2:172.21.15.161:6834/1424863663 2024-05-22T21:19:20.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe16fe00640 1 -- 172.21.15.161:0/1515443675 <== mds.0 v2:172.21.15.161:6834/1424863663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b4f6758790 con 0x55b4f6958610 2024-05-22T21:19:20.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.151+0000 7fe16fe00640 10 client.7281 handle_command_reply: tid=0 2024-05-22T21:19:20.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.155+0000 7fe1949ab000 1 client.7281 shutdown 2024-05-22T21:19:20.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.155+0000 7fe1949ab000 1 -- 172.21.15.161:0/1515443675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 msgr2=0x55b4f69576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:20.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.155+0000 7fe1949ab000 1 --2- 172.21.15.161:0/1515443675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f69576c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe174000c10 tx=0x7fe17401abc0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.155+0000 7fe1949ab000 1 -- 172.21.15.161:0/1515443675 shutdown_connections 2024-05-22T21:19:20.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.155+0000 7fe1949ab000 1 --2- 172.21.15.161:0/1515443675 >> [v2:172.21.15.161:6834/1424863663,v1:172.21.15.161:6837/1424863663] conn(0x55b4f6958610 0x55b4f69589f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.155+0000 7fe1949ab000 1 --2- 172.21.15.161:0/1515443675 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f69572e0 0x55b4f6957c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.155+0000 7fe1949ab000 1 --2- 172.21.15.161:0/1515443675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f69576c0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.155+0000 7fe1949ab000 1 --2- 172.21.15.161:0/1515443675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f6956200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.155+0000 7fe16fe00640 0 client.7281 ms_handle_reset on v2:172.21.15.161:6834/1424863663 2024-05-22T21:19:20.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.155+0000 7fe1949ab000 1 -- 172.21.15.161:0/1515443675 >> 172.21.15.161:0/1515443675 conn(0x55b4f6913440 msgr2=0x55b4f690b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:20.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.155+0000 7fe1949ab000 1 -- 172.21.15.161:0/1515443675 shutdown_connections 2024-05-22T21:19:20.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.155+0000 7fe1949ab000 1 -- 172.21.15.161:0/1515443675 wait complete. 2024-05-22T21:19:20.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.155+0000 7fe1949ab000 20 client.7281 trim_cache size 0 max 16384 2024-05-22T21:19:20.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.167+0000 7fe1949ab000 1 Processor -- start 2024-05-22T21:19:20.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.167+0000 7fe1949ab000 1 -- start start 2024-05-22T21:19:20.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f6913440 0x55b4f69551a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f69556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f695aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f6418940 con 0x55b4f69569c0 2024-05-22T21:19:20.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f66e27c0 con 0x55b4f690cec0 2024-05-22T21:19:20.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b4f69639f0 con 0x55b4f6913440 2024-05-22T21:19:20.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe18b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f695aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe18b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f695aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44816/0 (socket says 172.21.15.161:44816) 2024-05-22T21:19:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe18b400640 1 -- 172.21.15.161:0/3304491171 learned_addr learned my addr 172.21.15.161:0/3304491171 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe18aa00640 1 --2- 172.21.15.161:0/3304491171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f6913440 0x55b4f69551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe182000640 1 --2- 172.21.15.161:0/3304491171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f69556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe18b400640 1 -- 172.21.15.161:0/3304491171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f6913440 msgr2=0x55b4f69551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe18b400640 1 --2- 172.21.15.161:0/3304491171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f6913440 0x55b4f69551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe18b400640 1 -- 172.21.15.161:0/3304491171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 msgr2=0x55b4f69556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe18b400640 1 --2- 172.21.15.161:0/3304491171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f69556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe18b400640 1 -- 172.21.15.161:0/3304491171 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4f6706c00 con 0x55b4f69569c0 2024-05-22T21:19:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe18aa00640 1 --2- 172.21.15.161:0/3304491171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f6913440 0x55b4f69551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe182000640 1 --2- 172.21.15.161:0/3304491171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f69556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe18b400640 1 --2- 172.21.15.161:0/3304491171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f695aa00 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe184007650 tx=0x7fe184004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf4c5576bc03277 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:20.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe17be00640 1 -- 172.21.15.161:0/3304491171 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1840051c0 con 0x55b4f69569c0 2024-05-22T21:19:20.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe17be00640 1 -- 172.21.15.161:0/3304491171 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18400d030 con 0x55b4f69569c0 2024-05-22T21:19:20.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe17be00640 1 -- 172.21.15.161:0/3304491171 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184005520 con 0x55b4f69569c0 2024-05-22T21:19:20.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 -- 172.21.15.161:0/3304491171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 msgr2=0x55b4f695aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:20.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 --2- 172.21.15.161:0/3304491171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f695aa00 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe184007650 tx=0x7fe184004bc0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 -- 172.21.15.161:0/3304491171 shutdown_connections 2024-05-22T21:19:20.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 --2- 172.21.15.161:0/3304491171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f69569c0 0x55b4f695aa00 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 --2- 172.21.15.161:0/3304491171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f69556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 --2- 172.21.15.161:0/3304491171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f6913440 0x55b4f69551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 -- 172.21.15.161:0/3304491171 >> 172.21.15.161:0/3304491171 conn(0x55b4f6957d60 msgr2=0x55b4f6912030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:20.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 -- 172.21.15.161:0/3304491171 shutdown_connections 2024-05-22T21:19:20.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 -- 172.21.15.161:0/3304491171 wait complete. 2024-05-22T21:19:20.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 -- start start 2024-05-22T21:19:20.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.171+0000 7fe1949ab000 1 Processor -- start 2024-05-22T21:19:20.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f695cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f6913440 0x55b4f699cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f69569c0 0x55b4f699d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f67504d0 con 0x55b4f6913440 2024-05-22T21:19:20.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4f6758db0 con 0x55b4f690cec0 2024-05-22T21:19:20.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b4f6722c60 con 0x55b4f69569c0 2024-05-22T21:19:20.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 10 client.0 _ll_register_callbacks cb 0x7fe17c051ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:19:20.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe182000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f6913440 0x55b4f699cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe182000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f6913440 0x55b4f699cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44820/0 (socket says 172.21.15.161:44820) 2024-05-22T21:19:20.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe182000640 1 -- 172.21.15.161:0/4100094035 learned_addr learned my addr 172.21.15.161:0/4100094035 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:20.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe18aa00640 1 --2- 172.21.15.161:0/4100094035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f695cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe18b400640 1 --2- 172.21.15.161:0/4100094035 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f69569c0 0x55b4f699d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe182000640 1 -- 172.21.15.161:0/4100094035 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f69569c0 msgr2=0x55b4f699d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:20.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe182000640 1 --2- 172.21.15.161:0/4100094035 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f69569c0 0x55b4f699d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe182000640 1 -- 172.21.15.161:0/4100094035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 msgr2=0x55b4f695cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:20.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe182000640 1 --2- 172.21.15.161:0/4100094035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f695cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe182000640 1 -- 172.21.15.161:0/4100094035 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe184007300 con 0x55b4f6913440 2024-05-22T21:19:20.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe182000640 1 --2- 172.21.15.161:0/4100094035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f6913440 0x55b4f699cbb0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe174018f90 tx=0x7fe1740a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe03c4d55ad3fce8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:20.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe16fe00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:19:20.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe16fe00640 1 -- 172.21.15.161:0/4100094035 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1740b5b10 con 0x55b4f6913440 2024-05-22T21:19:20.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe16fe00640 1 -- 172.21.15.161:0/4100094035 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1740acc80 con 0x55b4f6913440 2024-05-22T21:19:20.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 -- 172.21.15.161:0/4100094035 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4f672b460 con 0x55b4f6913440 2024-05-22T21:19:20.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe16fe00640 1 -- 172.21.15.161:0/4100094035 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1740c2950 con 0x55b4f6913440 2024-05-22T21:19:20.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 10 client.7287 fetch_fsmap learned FSMap version 54 2024-05-22T21:19:20.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 -- 172.21.15.161:0/4100094035 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55b4f6457cc0 con 0x55b4f6913440 2024-05-22T21:19:20.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe16fe00640 1 -- 172.21.15.161:0/4100094035 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b4f672b460 con 0x55b4f6913440 2024-05-22T21:19:20.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe16fe00640 1 -- 172.21.15.161:0/4100094035 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 54) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe1740b5c70 con 0x55b4f6913440 2024-05-22T21:19:20.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 10 client.7287 fetch_fsmap finished waiting for FSMap version 54 2024-05-22T21:19:20.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 10 client.7287 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-22T21:19:20.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 20 client.7287 populate_metadata read hostname 'smithi161' 2024-05-22T21:19:20.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 --2- 172.21.15.161:0/4100094035 >> [v2:172.21.15.161:6834/1424863663,v1:172.21.15.161:6837/1424863663] conn(0x55b4f69572e0 0x55b4f699d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 4 client.7287 mds_command: new command op to 7146 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-22T21:19:20.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 -- 172.21.15.161:0/4100094035 --> [v2:172.21.15.161:6834/1424863663,v1:172.21.15.161:6837/1424863663] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b4f67249c0 con 0x55b4f69572e0 2024-05-22T21:19:20.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe18aa00640 1 --2- 172.21.15.161:0/4100094035 >> [v2:172.21.15.161:6834/1424863663,v1:172.21.15.161:6837/1424863663] conn(0x55b4f69572e0 0x55b4f699d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe18aa00640 1 --2- 172.21.15.161:0/4100094035 >> [v2:172.21.15.161:6834/1424863663,v1:172.21.15.161:6837/1424863663] conn(0x55b4f69572e0 0x55b4f699d630 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f6cd761c7aed0cf server_cookie=c67912e6d3aec633 in_seq=0 out_seq=0 2024-05-22T21:19:20.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe16fe00640 10 client.7287 ms_handle_connect on v2:172.21.15.161:6834/1424863663 2024-05-22T21:19:20.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe16fe00640 1 -- 172.21.15.161:0/4100094035 <== mds.0 v2:172.21.15.161:6834/1424863663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b4f67249c0 con 0x55b4f69572e0 2024-05-22T21:19:20.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe16fe00640 10 client.7287 handle_command_reply: tid=0 2024-05-22T21:19:20.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 client.7287 shutdown 2024-05-22T21:19:20.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 -- 172.21.15.161:0/4100094035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f6913440 msgr2=0x55b4f699cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:20.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 --2- 172.21.15.161:0/4100094035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f6913440 0x55b4f699cbb0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe174018f90 tx=0x7fe1740a9000 comp rx=0 tx=0).stop 2024-05-22T21:19:20.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 -- 172.21.15.161:0/4100094035 shutdown_connections 2024-05-22T21:19:20.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 --2- 172.21.15.161:0/4100094035 >> [v2:172.21.15.161:6834/1424863663,v1:172.21.15.161:6837/1424863663] conn(0x55b4f69572e0 0x55b4f699d630 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 --2- 172.21.15.161:0/4100094035 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55b4f69569c0 0x55b4f699d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 --2- 172.21.15.161:0/4100094035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4f6913440 0x55b4f699cbb0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 --2- 172.21.15.161:0/4100094035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b4f690cec0 0x55b4f695cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 -- 172.21.15.161:0/4100094035 >> 172.21.15.161:0/4100094035 conn(0x55b4f6957d60 msgr2=0x55b4f6954190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:20.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe16fe00640 0 client.7287 ms_handle_reset on v2:172.21.15.161:6834/1424863663 2024-05-22T21:19:20.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 -- 172.21.15.161:0/4100094035 shutdown_connections 2024-05-22T21:19:20.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.175+0000 7fe1949ab000 1 -- 172.21.15.161:0/4100094035 wait complete. 2024-05-22T21:19:20.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.179+0000 7fe1949ab000 20 client.7287 trim_cache size 0 max 16384 2024-05-22T21:19:20.184 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:19:20.184 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:19:20.185 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 0, 2024-05-22T21:19:20.185 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 7146, 2024-05-22T21:19:20.185 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2024-05-22T21:19:20.185 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2024-05-22T21:19:20.185 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2024-05-22T21:19:20.185 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 5.8321393910000001, 2024-05-22T21:19:20.185 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 54, 2024-05-22T21:19:20.185 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 125, 2024-05-22T21:19:20.185 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 125, 2024-05-22T21:19:20.185 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 10.108471867 2024-05-22T21:19:20.185 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:19:20.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.179+0000 7fe18be00640 1 -- 172.21.15.161:0/1795532084 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe168066690 msgr2=0x7fe168068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:20.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.179+0000 7fe18be00640 1 --2- 172.21.15.161:0/1795532084 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe168066690 0x7fe168068b50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe1740ae000 tx=0x7fe174008040 comp rx=0 tx=0).stop 2024-05-22T21:19:20.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.179+0000 7fe18be00640 1 -- 172.21.15.161:0/1795532084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe18c0b0d80 msgr2=0x7fe18c0c4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:20.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.179+0000 7fe18be00640 1 --2- 172.21.15.161:0/1795532084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe18c0b0d80 0x7fe18c0c4980 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe17c007e70 tx=0x7fe17c00e4b0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.179+0000 7fe18be00640 1 -- 172.21.15.161:0/1795532084 shutdown_connections 2024-05-22T21:19:20.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.179+0000 7fe18be00640 1 --2- 172.21.15.161:0/1795532084 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe168066690 0x7fe168068b50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.179+0000 7fe18be00640 1 --2- 172.21.15.161:0/1795532084 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe18c15b050 0x7fe18c0ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.179+0000 7fe18be00640 1 --2- 172.21.15.161:0/1795532084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe18c0d0ae0 0x7fe18c147170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.179+0000 7fe18be00640 1 --2- 172.21.15.161:0/1795532084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe18c0b0d80 0x7fe18c0c4980 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.179+0000 7fe18be00640 1 -- 172.21.15.161:0/1795532084 >> 172.21.15.161:0/1795532084 conn(0x7fe18c099a40 msgr2=0x7fe18c0ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:20.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.179+0000 7fe18be00640 1 -- 172.21.15.161:0/1795532084 shutdown_connections 2024-05-22T21:19:20.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.179+0000 7fe18be00640 1 -- 172.21.15.161:0/1795532084 wait complete. 2024-05-22T21:19:20.199 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:19:20.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.331+0000 7f7290800640 1 Processor -- start 2024-05-22T21:19:20.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.331+0000 7f7290800640 1 -- start start 2024-05-22T21:19:20.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.331+0000 7f7290800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f728815dd50 0x7f728806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.331+0000 7f7290800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f728815e130 0x7f728815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.331+0000 7f7290800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7288071670 con 0x7f728815d410 2024-05-22T21:19:20.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.331+0000 7f7290800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728815d8a0 con 0x7f728815e130 2024-05-22T21:19:20.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.331+0000 7f7290800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728806de40 con 0x7f728815dd50 2024-05-22T21:19:20.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.331+0000 7f728d400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f728815e130 0x7f728815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.331+0000 7f728e800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f728815d410 0x7f728815a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48836/0 (socket says 172.21.15.161:48836) 2024-05-22T21:19:20.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.331+0000 7f728e800640 1 -- 172.21.15.161:0/2712975169 learned_addr learned my addr 172.21.15.161:0/2712975169 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:20.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.331+0000 7f728de00640 1 --2- 172.21.15.161:0/2712975169 >> v2:172.21.15.179:3300/0 conn(0x7f728815dd50 0x7f728806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.331+0000 7f728d400640 1 -- 172.21.15.161:0/2712975169 >> v2:172.21.15.179:3300/0 conn(0x7f728815dd50 msgr2=0x7f728806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:20.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.331+0000 7f728d400640 1 --2- 172.21.15.161:0/2712975169 >> v2:172.21.15.179:3300/0 conn(0x7f728815dd50 0x7f728806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.331+0000 7f728d400640 1 -- 172.21.15.161:0/2712975169 >> v1:172.21.15.161:6789/0 conn(0x7f728815d410 legacy=0x7f728815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:20.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.331+0000 7f728ca00640 1 -- 172.21.15.161:0/2712975169 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268894953 0 0) 0x7f7288071670 con 0x7f728815d410 2024-05-22T21:19:20.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.331+0000 7f728d400640 1 -- 172.21.15.161:0/2712975169 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7288070a60 con 0x7f728815e130 2024-05-22T21:19:20.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.335+0000 7f728d400640 1 --2- 172.21.15.161:0/2712975169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f728815e130 0x7f728815ced0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f7288072330 tx=0x7f727801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1327f398fb6fcaa server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:20.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.335+0000 7f728ca00640 1 -- 172.21.15.161:0/2712975169 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72780ab650 con 0x7f728815e130 2024-05-22T21:19:20.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.335+0000 7f728ca00640 1 -- 172.21.15.161:0/2712975169 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72780ab7b0 con 0x7f728815e130 2024-05-22T21:19:20.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.335+0000 7f728ca00640 1 -- 172.21.15.161:0/2712975169 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72780abb40 con 0x7f728815e130 2024-05-22T21:19:20.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.335+0000 7f7290800640 1 -- 172.21.15.161:0/2712975169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f728815e130 msgr2=0x7f728815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:20.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.335+0000 7f7290800640 1 --2- 172.21.15.161:0/2712975169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f728815e130 0x7f728815ced0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f7288072330 tx=0x7f727801c450 comp rx=0 tx=0).stop 2024-05-22T21:19:20.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.335+0000 7f7290800640 1 -- 172.21.15.161:0/2712975169 shutdown_connections 2024-05-22T21:19:20.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.335+0000 7f7290800640 1 --2- 172.21.15.161:0/2712975169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f728815e130 0x7f728815ced0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.335+0000 7f7290800640 1 --2- 172.21.15.161:0/2712975169 >> v2:172.21.15.179:3300/0 conn(0x7f728815dd50 0x7f728806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.335+0000 7f7290800640 1 -- 172.21.15.161:0/2712975169 >> 172.21.15.161:0/2712975169 conn(0x7f72880b0bc0 msgr2=0x7f72880b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:20.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.335+0000 7f7290800640 1 -- 172.21.15.161:0/2712975169 shutdown_connections 2024-05-22T21:19:20.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.335+0000 7f7290800640 1 -- 172.21.15.161:0/2712975169 wait complete. 2024-05-22T21:19:20.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f7290800640 1 Processor -- start 2024-05-22T21:19:20.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f7290800640 1 -- start start 2024-05-22T21:19:20.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f7290800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f728815d410 0x7f7288150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f7290800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f728815dd50 0x7f72880b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f7290800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f728815e130 0x7f72881570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f7290800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7288151420 con 0x7f728815e130 2024-05-22T21:19:20.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f7290800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7288157b00 con 0x7f728815dd50 2024-05-22T21:19:20.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f7290800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7288157de0 con 0x7f728815d410 2024-05-22T21:19:20.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f728e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f728815e130 0x7f72881570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f728e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f728815e130 0x7f72881570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44852/0 (socket says 172.21.15.161:44852) 2024-05-22T21:19:20.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f728e800640 1 -- 172.21.15.161:0/3033436098 learned_addr learned my addr 172.21.15.161:0/3033436098 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:20.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f728e800640 1 -- 172.21.15.161:0/3033436098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f728815d410 msgr2=0x7f7288150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:20.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f728e800640 1 --2- 172.21.15.161:0/3033436098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f728815d410 0x7f7288150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f728e800640 1 -- 172.21.15.161:0/3033436098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f728815dd50 msgr2=0x7f72880b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:20.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f728d400640 1 --2- 172.21.15.161:0/3033436098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f728815dd50 0x7f72880b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f728e800640 1 --2- 172.21.15.161:0/3033436098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f728815dd50 0x7f72880b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f728e800640 1 -- 172.21.15.161:0/3033436098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7278009420 con 0x7f728815e130 2024-05-22T21:19:20.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f728d400640 1 --2- 172.21.15.161:0/3033436098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f728815dd50 0x7f72880b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:20.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f728e800640 1 --2- 172.21.15.161:0/3033436098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f728815e130 0x7f72881570b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7284001570 tx=0x7f728400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:20.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f7282000640 1 -- 172.21.15.161:0/3033436098 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728400a640 con 0x7f728815e130 2024-05-22T21:19:20.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f7282000640 1 -- 172.21.15.161:0/3033436098 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728400a7c0 con 0x7f728815e130 2024-05-22T21:19:20.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f7282000640 1 -- 172.21.15.161:0/3033436098 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728400ab70 con 0x7f728815e130 2024-05-22T21:19:20.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f7290800640 1 -- 172.21.15.161:0/3033436098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72881580c0 con 0x7f728815e130 2024-05-22T21:19:20.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.339+0000 7f7290800640 1 -- 172.21.15.161:0/3033436098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7288154e20 con 0x7f728815e130 2024-05-22T21:19:20.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.343+0000 7f7290800640 1 -- 172.21.15.161:0/3033436098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7288155010 con 0x7f728815e130 2024-05-22T21:19:20.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.343+0000 7f7282000640 1 -- 172.21.15.161:0/3033436098 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f728400acd0 con 0x7f728815e130 2024-05-22T21:19:20.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.343+0000 7f7282000640 1 --2- 172.21.15.161:0/3033436098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f726c066690 0x7f726c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:20.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.343+0000 7f7282000640 1 -- 172.21.15.161:0/3033436098 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7284083720 con 0x7f728815e130 2024-05-22T21:19:20.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.343+0000 7f728de00640 1 --2- 172.21.15.161:0/3033436098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f726c066690 0x7f726c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:20.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.343+0000 7f728de00640 1 --2- 172.21.15.161:0/3033436098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f726c066690 0x7f726c068b50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7288159900 tx=0x7f727c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:20.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.347+0000 7f7282000640 1 -- 172.21.15.161:0/3033436098 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7284050a80 con 0x7f728815e130 2024-05-22T21:19:20.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.539+0000 7f7290800640 1 -- 172.21.15.161:0/3033436098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f72880706b0 con 0x7f728815e130 2024-05-22T21:19:20.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.539+0000 7f7282000640 1 -- 172.21.15.161:0/3033436098 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7284018070 con 0x7f728815e130 2024-05-22T21:19:20.541 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:19:20.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.539+0000 7f7290800640 1 -- 172.21.15.161:0/3033436098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f726c066690 msgr2=0x7f726c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:20.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.539+0000 7f7290800640 1 --2- 172.21.15.161:0/3033436098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f726c066690 0x7f726c068b50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7288159900 tx=0x7f727c008040 comp rx=0 tx=0).stop 2024-05-22T21:19:20.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.539+0000 7f7290800640 1 -- 172.21.15.161:0/3033436098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f728815e130 msgr2=0x7f72881570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:20.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.539+0000 7f7290800640 1 --2- 172.21.15.161:0/3033436098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f728815e130 0x7f72881570b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7284001570 tx=0x7f728400cca0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.539+0000 7f7290800640 1 -- 172.21.15.161:0/3033436098 shutdown_connections 2024-05-22T21:19:20.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.539+0000 7f7290800640 1 --2- 172.21.15.161:0/3033436098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f726c066690 0x7f726c068b50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.539+0000 7f7290800640 1 --2- 172.21.15.161:0/3033436098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f728815e130 0x7f72881570b0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.539+0000 7f7290800640 1 --2- 172.21.15.161:0/3033436098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f728815dd50 0x7f72880b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.539+0000 7f7290800640 1 --2- 172.21.15.161:0/3033436098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f728815d410 0x7f7288150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:20.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.539+0000 7f7290800640 1 -- 172.21.15.161:0/3033436098 >> 172.21.15.161:0/3033436098 conn(0x7f72880b0bc0 msgr2=0x7f72880ad500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:20.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.539+0000 7f7290800640 1 -- 172.21.15.161:0/3033436098 shutdown_connections 2024-05-22T21:19:20.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:20.539+0000 7f7290800640 1 -- 172.21.15.161:0/3033436098 wait complete. 2024-05-22T21:19:20.561 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:19:20.561 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:19:20.561 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-05-22T21:19:20.562 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-22T21:19:20.562 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-22T21:19:20.562 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:19:20.562 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:19:20.562 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:19:20.562 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:19:20.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:20.562 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-05-22T21:19:20.571 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:19:20.571 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:19:20.571 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:19:20.572 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:20.572 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-05-22T21:19:20.572 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:20.572 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:19:20.572 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:20.572 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2024-05-22T21:19:20.572 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:19:20.572 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:20.572 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2024-05-22T21:19:20.572 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:19:20.572 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:20.572 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:19:20.573 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:20.573 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:19:20.573 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42132sec preferred_lft 42132sec 2024-05-22T21:19:20.573 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-05-22T21:19:20.573 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:20.573 INFO:teuthology.orchestra.run.smithi161.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:19:20.573 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 02:42:b9:0f:57:df brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:20.573 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:19:20.573 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:20.574 INFO:teuthology.orchestra.run.smithi161.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:19:20.574 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:20.574 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:19:20.574 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:20.574 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:19:20.574 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:20.574 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:19:20.574 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-22T21:19:20.574 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:19:20.574 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:20.575 INFO:teuthology.orchestra.run.smithi161.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:19:20.575 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-22T21:19:20.575 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:19:20.575 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:20.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:20.576 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:19:20.640 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-05-22T21:19:20.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:20.643 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-05-22T21:19:20.695 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-05-22T21:19:20.695 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:19:20.695 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:19:20.695 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:19:20.696 INFO:teuthology.orchestra.run.smithi161.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:19:20.696 INFO:teuthology.orchestra.run.smithi161.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:19:20.696 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:19:20.696 INFO:teuthology.orchestra.run.smithi161.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:19:20.696 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:19:20.696 INFO:teuthology.orchestra.run.smithi161.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:19:20.696 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:19:20.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:20.697 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:19:20.697 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:19:20.697 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:19:20.697 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:19:20.697 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:19:20.697 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:19:20.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:20.787 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-05-22T21:19:20.795 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-22T21:19:20.796 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-22T21:19:20.796 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:19:20.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:20.797 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2024-05-22T21:19:20.858 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:19:20.859 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-22T21:19:20.907 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:19:20.908 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:19:20.908 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:19:20.959 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:19:20.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:20.960 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:19:21.051 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:19:21.051 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:19:21.051 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:19:21.066 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:19:21.066 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:19:21.066 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:19:21.067 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:19:21.067 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:19:21.075 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:19:21.075 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:19:21.075 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:19:21.075 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:19:21.075 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:19:21.075 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:19:21.075 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:19:21.075 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:19:21.079 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:19:21.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:21.079 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:19:21.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:21.099 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:19:21.158 INFO:teuthology.orchestra.run.smithi161.stdout:client7299 2024-05-22T21:19:21.159 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:19:21.159 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:21.211 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:19:21.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.387+0000 7f41e0000640 1 Processor -- start 2024-05-22T21:19:21.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.387+0000 7f41e0000640 1 -- start start 2024-05-22T21:19:21.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.391+0000 7f41e0000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41d815dd50 0x7f41d806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:21.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.391+0000 7f41e0000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f41d815d5e0 0x7f41d815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:21.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.391+0000 7f41e0000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d8071670 con 0x7f41d815e130 2024-05-22T21:19:21.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.391+0000 7f41e0000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d806de40 con 0x7f41d815d5e0 2024-05-22T21:19:21.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.391+0000 7f41e0000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d8070a60 con 0x7f41d815dd50 2024-05-22T21:19:21.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.391+0000 7f41de000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f41d815d5e0 0x7f41d815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:21.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.391+0000 7f41de000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f41d815d5e0 0x7f41d815b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44864/0 (socket says 172.21.15.161:44864) 2024-05-22T21:19:21.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.391+0000 7f41de000640 1 -- 172.21.15.161:0/996615860 learned_addr learned my addr 172.21.15.161:0/996615860 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:21.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.391+0000 7f41dd600640 1 --2- 172.21.15.161:0/996615860 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41d815dd50 0x7f41d806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:21.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.391+0000 7f41de000640 1 -- 172.21.15.161:0/996615860 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41d815dd50 msgr2=0x7f41d806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:21.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.391+0000 7f41de000640 1 --2- 172.21.15.161:0/996615860 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41d815dd50 0x7f41d806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:21.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.391+0000 7f41de000640 1 -- 172.21.15.161:0/996615860 >> v1:172.21.15.179:6789/0 conn(0x7f41d815e130 legacy=0x7f41d815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:21.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.391+0000 7f41de000640 1 -- 172.21.15.161:0/996615860 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d8070d40 con 0x7f41d815d5e0 2024-05-22T21:19:21.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.391+0000 7f41dd600640 1 --2- 172.21.15.161:0/996615860 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41d815dd50 0x7f41d806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:21.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.395+0000 7f41de000640 1 --2- 172.21.15.161:0/996615860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d815d5e0 0x7f41d815b6c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009e50 tx=0x7f41d001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd01ca368d03f88 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:21.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.395+0000 7f41d7e00640 1 -- 172.21.15.161:0/996615860 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d00a5830 con 0x7f41d815d5e0 2024-05-22T21:19:21.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.395+0000 7f41d7e00640 1 -- 172.21.15.161:0/996615860 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d00a5990 con 0x7f41d815d5e0 2024-05-22T21:19:21.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.395+0000 7f41d7e00640 1 -- 172.21.15.161:0/996615860 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d00a5d20 con 0x7f41d815d5e0 2024-05-22T21:19:21.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.395+0000 7f41e0000640 1 -- 172.21.15.161:0/996615860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d815d5e0 msgr2=0x7f41d815b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:21.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.395+0000 7f41e0000640 1 --2- 172.21.15.161:0/996615860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d815d5e0 0x7f41d815b6c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009e50 tx=0x7f41d001c970 comp rx=0 tx=0).stop 2024-05-22T21:19:21.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.395+0000 7f41e0000640 1 -- 172.21.15.161:0/996615860 shutdown_connections 2024-05-22T21:19:21.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.395+0000 7f41e0000640 1 --2- 172.21.15.161:0/996615860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d815d5e0 0x7f41d815b6c0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:21.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.395+0000 7f41e0000640 1 --2- 172.21.15.161:0/996615860 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41d815dd50 0x7f41d806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:21.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.395+0000 7f41e0000640 1 -- 172.21.15.161:0/996615860 >> 172.21.15.161:0/996615860 conn(0x7f41d80b0bc0 msgr2=0x7f41d80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:21.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.395+0000 7f41e0000640 1 -- 172.21.15.161:0/996615860 shutdown_connections 2024-05-22T21:19:21.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.395+0000 7f41e0000640 1 -- 172.21.15.161:0/996615860 wait complete. 2024-05-22T21:19:21.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.395+0000 7f41e0000640 1 Processor -- start 2024-05-22T21:19:21.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.395+0000 7f41e0000640 1 -- start start 2024-05-22T21:19:21.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.395+0000 7f41e0000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d815d5e0 0x7f41d8145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:21.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41e0000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41d815dd50 0x7f41d814b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:21.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41e0000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d815e130 0x7f41d814b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:21.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41e0000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d81464a0 con 0x7f41d815d5e0 2024-05-22T21:19:21.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41e0000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d814bbb0 con 0x7f41d815e130 2024-05-22T21:19:21.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41e0000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d814be90 con 0x7f41d815dd50 2024-05-22T21:19:21.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41dd600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d815d5e0 0x7f41d8145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:21.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41dd600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d815d5e0 0x7f41d8145970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44870/0 (socket says 172.21.15.161:44870) 2024-05-22T21:19:21.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41dd600640 1 -- 172.21.15.161:0/1001475135 learned_addr learned my addr 172.21.15.161:0/1001475135 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:21.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41dd600640 1 -- 172.21.15.161:0/1001475135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41d815dd50 msgr2=0x7f41d814b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:21.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41dd600640 1 --2- 172.21.15.161:0/1001475135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41d815dd50 0x7f41d814b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:21.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41dd600640 1 -- 172.21.15.161:0/1001475135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d815e130 msgr2=0x7f41d814b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:21.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41dd600640 1 --2- 172.21.15.161:0/1001475135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d815e130 0x7f41d814b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:21.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41dd600640 1 -- 172.21.15.161:0/1001475135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d0009b30 con 0x7f41d815d5e0 2024-05-22T21:19:21.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41dd600640 1 --2- 172.21.15.161:0/1001475135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d815d5e0 0x7f41d8145970 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00e810 tx=0x7f41cc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:21.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41d5600640 1 -- 172.21.15.161:0/1001475135 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc00c8d0 con 0x7f41d815d5e0 2024-05-22T21:19:21.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41d5600640 1 -- 172.21.15.161:0/1001475135 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc00ca30 con 0x7f41d815d5e0 2024-05-22T21:19:21.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41d5600640 1 -- 172.21.15.161:0/1001475135 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc00cde0 con 0x7f41d815d5e0 2024-05-22T21:19:21.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41e0000640 1 -- 172.21.15.161:0/1001475135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d814c170 con 0x7f41d815d5e0 2024-05-22T21:19:21.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41e0000640 1 -- 172.21.15.161:0/1001475135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d814c450 con 0x7f41d815d5e0 2024-05-22T21:19:21.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41e0000640 1 -- 172.21.15.161:0/1001475135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d8067220 con 0x7f41d815d5e0 2024-05-22T21:19:21.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41d5600640 1 -- 172.21.15.161:0/1001475135 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f41cc013410 con 0x7f41d815d5e0 2024-05-22T21:19:21.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.399+0000 7f41d5600640 1 --2- 172.21.15.161:0/1001475135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41b8066690 0x7f41b8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:21.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.403+0000 7f41d5600640 1 -- 172.21.15.161:0/1001475135 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41cc083f90 con 0x7f41d815d5e0 2024-05-22T21:19:21.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.403+0000 7f41dcc00640 1 --2- 172.21.15.161:0/1001475135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41b8066690 0x7f41b8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:21.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.403+0000 7f41dcc00640 1 --2- 172.21.15.161:0/1001475135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41b8066690 0x7f41b8068b50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f41c4000f70 tx=0x7f41c40079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:21.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.407+0000 7f41d5600640 1 -- 172.21.15.161:0/1001475135 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f41cc0b5cf0 con 0x7f41d815d5e0 2024-05-22T21:19:21.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.595+0000 7f41e0000640 1 -- 172.21.15.161:0/1001475135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f41d806c7f0 con 0x7f41d815d5e0 2024-05-22T21:19:21.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.595+0000 7f41d5600640 1 -- 172.21.15.161:0/1001475135 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) v1 ==== 53+0+73 (secure 0 0 0) 0x7f41cc0512e0 con 0x7f41d815d5e0 2024-05-22T21:19:21.597 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:19:21.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.595+0000 7f41e0000640 1 -- 172.21.15.161:0/1001475135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41b8066690 msgr2=0x7f41b8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:21.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.595+0000 7f41e0000640 1 --2- 172.21.15.161:0/1001475135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41b8066690 0x7f41b8068b50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f41c4000f70 tx=0x7f41c40079c0 comp rx=0 tx=0).stop 2024-05-22T21:19:21.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.595+0000 7f41e0000640 1 -- 172.21.15.161:0/1001475135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d815d5e0 msgr2=0x7f41d8145970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:21.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.595+0000 7f41e0000640 1 --2- 172.21.15.161:0/1001475135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d815d5e0 0x7f41d8145970 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00e810 tx=0x7f41cc00ecd0 comp rx=0 tx=0).stop 2024-05-22T21:19:21.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.595+0000 7f41e0000640 1 -- 172.21.15.161:0/1001475135 shutdown_connections 2024-05-22T21:19:21.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.595+0000 7f41e0000640 1 --2- 172.21.15.161:0/1001475135 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41b8066690 0x7f41b8068b50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:21.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.595+0000 7f41e0000640 1 --2- 172.21.15.161:0/1001475135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41d815e130 0x7f41d814b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:21.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.595+0000 7f41e0000640 1 --2- 172.21.15.161:0/1001475135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41d815dd50 0x7f41d814b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:21.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.595+0000 7f41e0000640 1 --2- 172.21.15.161:0/1001475135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d815d5e0 0x7f41d8145970 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:21.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.595+0000 7f41e0000640 1 -- 172.21.15.161:0/1001475135 >> 172.21.15.161:0/1001475135 conn(0x7f41d80b0bc0 msgr2=0x7f41d815a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:21.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.595+0000 7f41e0000640 1 -- 172.21.15.161:0/1001475135 shutdown_connections 2024-05-22T21:19:21.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:21.595+0000 7f41e0000640 1 -- 172.21.15.161:0/1001475135 wait complete. 2024-05-22T21:19:21.619 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:19:21.619 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:19:21.619 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-22T21:19:21.619 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-22T21:19:21.619 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-22T21:19:21.619 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:19:21.619 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:19:21.620 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:19:21.620 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:19:21.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:21.620 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-22T21:19:21.631 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:19:21.631 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:19:21.631 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:19:21.631 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:21.631 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-22T21:19:21.631 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:21.632 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:19:21.632 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:21.632 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f0 2024-05-22T21:19:21.632 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:19:21.632 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:21.632 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f1 2024-05-22T21:19:21.632 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:19:21.632 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:21.632 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:19:21.632 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:21.632 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:19:21.633 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42123sec preferred_lft 42123sec 2024-05-22T21:19:21.633 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-22T21:19:21.633 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:21.633 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:19:21.633 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:bd:4e:25:3d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:21.633 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:19:21.633 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:21.633 INFO:teuthology.orchestra.run.smithi179.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:19:21.633 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:21.634 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:19:21.634 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:21.634 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:19:21.634 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:21.634 INFO:teuthology.orchestra.run.smithi179.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:19:21.634 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-22T21:19:21.634 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:19:21.634 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:21.634 INFO:teuthology.orchestra.run.smithi179.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:19:21.634 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-22T21:19:21.634 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:19:21.634 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:21.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:21.635 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:19:21.692 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-22T21:19:21.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:21.695 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-22T21:19:21.753 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-22T21:19:21.753 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:19:21.753 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:19:21.753 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:19:21.754 INFO:teuthology.orchestra.run.smithi179.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:19:21.754 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:19:21.754 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:19:21.754 INFO:teuthology.orchestra.run.smithi179.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:19:21.754 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:19:21.754 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:19:21.754 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:19:21.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:21.754 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:19:21.754 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:19:21.755 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:19:21.755 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:19:21.755 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:19:21.755 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:19:21.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:21.854 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-22T21:19:21.863 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-22T21:19:21.863 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-22T21:19:21.863 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:19:21.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:21.864 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-22T21:19:21.928 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:19:21.928 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-22T21:19:21.981 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:19:21.982 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:19:21.982 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:19:22.031 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:19:22.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:22.032 DEBUG:teuthology.orchestra.run.smithi179:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:19:22.126 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:19:22.126 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:19:22.126 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:19:22.141 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:19:22.141 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:19:22.141 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:19:22.141 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:19:22.141 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:19:22.150 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:19:22.150 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:19:22.151 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:19:22.151 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:19:22.151 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:19:22.151 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:19:22.151 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:19:22.151 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:19:22.155 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:19:22.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:22.155 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:19:22.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:22.174 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:19:22.231 INFO:teuthology.orchestra.run.smithi179.stdout:client7308 2024-05-22T21:19:22.231 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:19:22.231 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:22.280 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:22.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0038800640 1 Processor -- start 2024-05-22T21:19:22.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0038800640 1 -- start start 2024-05-22T21:19:22.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0038800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f003015e130 0x7f003015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:22.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0038800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f003015d410 0x7f003015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:22.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0038800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0030071670 con 0x7f003015dd50 2024-05-22T21:19:22.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0038800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f003015d870 con 0x7f003015d410 2024-05-22T21:19:22.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0038800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0030070a60 con 0x7f003015e130 2024-05-22T21:19:22.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0035e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f003015dd50 0x7f003006d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48838/0 (socket says 172.21.15.161:48838) 2024-05-22T21:19:22.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0035e00640 1 -- 172.21.15.161:0/2278704264 learned_addr learned my addr 172.21.15.161:0/2278704264 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:22.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0036800640 1 --2- 172.21.15.161:0/2278704264 >> v2:172.21.15.161:3300/0 conn(0x7f003015d410 0x7f003015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:22.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0035400640 1 --2- 172.21.15.161:0/2278704264 >> v2:172.21.15.179:3300/0 conn(0x7f003015e130 0x7f003015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:22.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0034a00640 1 -- 172.21.15.161:0/2278704264 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409546291 0 0) 0x7f0030071670 con 0x7f003015dd50 2024-05-22T21:19:22.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0034a00640 1 -- 172.21.15.161:0/2278704264 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0018003530 con 0x7f003015dd50 2024-05-22T21:19:22.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0036800640 1 -- 172.21.15.161:0/2278704264 >> v2:172.21.15.179:3300/0 conn(0x7f003015e130 msgr2=0x7f003015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:22.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0036800640 1 --2- 172.21.15.161:0/2278704264 >> v2:172.21.15.179:3300/0 conn(0x7f003015e130 0x7f003015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:22.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0036800640 1 -- 172.21.15.161:0/2278704264 >> v1:172.21.15.161:6789/0 conn(0x7f003015dd50 legacy=0x7f003006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:22.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.423+0000 7f0036800640 1 -- 172.21.15.161:0/2278704264 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0030070d40 con 0x7f003015d410 2024-05-22T21:19:22.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.427+0000 7f0036800640 1 --2- 172.21.15.161:0/2278704264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003015d410 0x7f003015b6c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f003006c7c0 tx=0x7f002c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3face17bdf84eaf server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:22.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.427+0000 7f0034a00640 1 -- 172.21.15.161:0/2278704264 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c0a59c0 con 0x7f003015d410 2024-05-22T21:19:22.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.427+0000 7f0034a00640 1 -- 172.21.15.161:0/2278704264 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c0a5b20 con 0x7f003015d410 2024-05-22T21:19:22.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.427+0000 7f0034a00640 1 -- 172.21.15.161:0/2278704264 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c0ad440 con 0x7f003015d410 2024-05-22T21:19:22.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.427+0000 7f0038800640 1 -- 172.21.15.161:0/2278704264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003015d410 msgr2=0x7f003015b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:22.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.427+0000 7f0038800640 1 --2- 172.21.15.161:0/2278704264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003015d410 0x7f003015b6c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f003006c7c0 tx=0x7f002c01c900 comp rx=0 tx=0).stop 2024-05-22T21:19:22.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.427+0000 7f0038800640 1 -- 172.21.15.161:0/2278704264 shutdown_connections 2024-05-22T21:19:22.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.427+0000 7f0038800640 1 --2- 172.21.15.161:0/2278704264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003015d410 0x7f003015b6c0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:22.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.427+0000 7f0038800640 1 --2- 172.21.15.161:0/2278704264 >> v2:172.21.15.179:3300/0 conn(0x7f003015e130 0x7f003015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:22.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.427+0000 7f0038800640 1 -- 172.21.15.161:0/2278704264 >> 172.21.15.161:0/2278704264 conn(0x7f00300b0bc0 msgr2=0x7f00300b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:22.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.427+0000 7f0038800640 1 -- 172.21.15.161:0/2278704264 shutdown_connections 2024-05-22T21:19:22.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.427+0000 7f0038800640 1 -- 172.21.15.161:0/2278704264 wait complete. 2024-05-22T21:19:22.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0038800640 1 Processor -- start 2024-05-22T21:19:22.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0038800640 1 -- start start 2024-05-22T21:19:22.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0038800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003015d410 0x7f00301f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:22.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0038800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003015dd50 0x7f00300adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:22.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0038800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003015e130 0x7f0030158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:22.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0038800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00301f6940 con 0x7f003015dd50 2024-05-22T21:19:22.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0038800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003014d630 con 0x7f003015d410 2024-05-22T21:19:22.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0038800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003014d910 con 0x7f003015e130 2024-05-22T21:19:22.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0035400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003015dd50 0x7f00300adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:22.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0035400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003015dd50 0x7f00300adc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44896/0 (socket says 172.21.15.161:44896) 2024-05-22T21:19:22.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0035400640 1 -- 172.21.15.161:0/2738594660 learned_addr learned my addr 172.21.15.161:0/2738594660 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:22.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0035e00640 1 --2- 172.21.15.161:0/2738594660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003015d410 0x7f00301f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:22.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0036800640 1 --2- 172.21.15.161:0/2738594660 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003015e130 0x7f0030158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:22.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0035400640 1 -- 172.21.15.161:0/2738594660 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003015e130 msgr2=0x7f0030158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:22.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0035400640 1 --2- 172.21.15.161:0/2738594660 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003015e130 0x7f0030158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:22.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0035400640 1 -- 172.21.15.161:0/2738594660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003015d410 msgr2=0x7f00301f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:22.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0035400640 1 --2- 172.21.15.161:0/2738594660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003015d410 0x7f00301f6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:22.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0035400640 1 -- 172.21.15.161:0/2738594660 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c009a70 con 0x7f003015dd50 2024-05-22T21:19:22.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0035400640 1 --2- 172.21.15.161:0/2738594660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003015dd50 0x7f00300adc90 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f002000e870 tx=0x7f002000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:22.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f002a000640 1 -- 172.21.15.161:0/2738594660 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020009510 con 0x7f003015dd50 2024-05-22T21:19:22.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f002a000640 1 -- 172.21.15.161:0/2738594660 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002000eec0 con 0x7f003015dd50 2024-05-22T21:19:22.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0038800640 1 -- 172.21.15.161:0/2738594660 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003014dbf0 con 0x7f003015dd50 2024-05-22T21:19:22.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f002a000640 1 -- 172.21.15.161:0/2738594660 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002000cb40 con 0x7f003015dd50 2024-05-22T21:19:22.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0038800640 1 -- 172.21.15.161:0/2738594660 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003014e1b0 con 0x7f003015dd50 2024-05-22T21:19:22.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.431+0000 7f0038800640 1 -- 172.21.15.161:0/2738594660 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0030067220 con 0x7f003015dd50 2024-05-22T21:19:22.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.435+0000 7f002a000640 1 -- 172.21.15.161:0/2738594660 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f002000ccc0 con 0x7f003015dd50 2024-05-22T21:19:22.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.435+0000 7f002a000640 1 --2- 172.21.15.161:0/2738594660 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0014066690 0x7f0014068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:22.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.435+0000 7f002a000640 1 -- 172.21.15.161:0/2738594660 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0020084240 con 0x7f003015dd50 2024-05-22T21:19:22.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.435+0000 7f0035e00640 1 --2- 172.21.15.161:0/2738594660 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0014066690 0x7f0014068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:22.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.435+0000 7f0035e00640 1 --2- 172.21.15.161:0/2738594660 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0014066690 0x7f0014068b50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f00240011f0 tx=0x7f0024007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:22.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.439+0000 7f002a000640 1 -- 172.21.15.161:0/2738594660 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f00200515c0 con 0x7f003015dd50 2024-05-22T21:19:22.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.631+0000 7f0038800640 1 -- 172.21.15.161:0/2738594660 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f003006fef0 con 0x7f003015dd50 2024-05-22T21:19:22.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.631+0000 7f002a000640 1 -- 172.21.15.161:0/2738594660 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f0020054be0 con 0x7f003015dd50 2024-05-22T21:19:22.634 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:22.634 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3944033857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3944033857},{"type":"v1","addr":"172.21.15.179:6834","nonce":3944033857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6125,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/4216709187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4216709187},{"type":"v1","addr":"172.21.15.161:6836","nonce":4216709187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2364917653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2364917653},{"type":"v1","addr":"172.21.15.179:6835","nonce":2364917653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:14.345936+0000","modified":"2024-05-22T21:19:17.355628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/1424863663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1424863663},{"type":"v1","addr":"172.21.15.161:6837","nonce":1424863663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-22T21:19:22.635 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 54 2024-05-22T21:19:22.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.635+0000 7f0038800640 1 -- 172.21.15.161:0/2738594660 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0014066690 msgr2=0x7f0014068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:22.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.635+0000 7f0038800640 1 --2- 172.21.15.161:0/2738594660 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0014066690 0x7f0014068b50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f00240011f0 tx=0x7f0024007d40 comp rx=0 tx=0).stop 2024-05-22T21:19:22.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.635+0000 7f0038800640 1 -- 172.21.15.161:0/2738594660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003015dd50 msgr2=0x7f00300adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:22.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.635+0000 7f0038800640 1 --2- 172.21.15.161:0/2738594660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003015dd50 0x7f00300adc90 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f002000e870 tx=0x7f002000ec70 comp rx=0 tx=0).stop 2024-05-22T21:19:22.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.635+0000 7f0038800640 1 -- 172.21.15.161:0/2738594660 shutdown_connections 2024-05-22T21:19:22.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.635+0000 7f0038800640 1 --2- 172.21.15.161:0/2738594660 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0014066690 0x7f0014068b50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:22.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.635+0000 7f0038800640 1 --2- 172.21.15.161:0/2738594660 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003015e130 0x7f0030158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:22.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.635+0000 7f0038800640 1 --2- 172.21.15.161:0/2738594660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003015dd50 0x7f00300adc90 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:22.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.635+0000 7f0038800640 1 --2- 172.21.15.161:0/2738594660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003015d410 0x7f00301f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:22.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.635+0000 7f0038800640 1 -- 172.21.15.161:0/2738594660 >> 172.21.15.161:0/2738594660 conn(0x7f00300b0bc0 msgr2=0x7f00301594a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:22.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.635+0000 7f0038800640 1 -- 172.21.15.161:0/2738594660 shutdown_connections 2024-05-22T21:19:22.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.635+0000 7f0038800640 1 -- 172.21.15.161:0/2738594660 wait complete. 2024-05-22T21:19:22.656 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-05-22T21:19:22.810 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":7146,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.4484834279999994,"mdsmap_epoch":54,"osdmap_epoch":125,"osdmap_epoch_barrier":125,"uptime":12.724822263} 2024-05-22T21:19:22.811 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 7146, "mdsmap_epoch": 54, "osdmap_epoch": 125, "osdmap_epoch_barrier": 125, "rank_uptime": 8.448483428, "state": "up:active", "uptime": 12.724822263, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:19:22.811 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-05-22T21:19:22.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.947+0000 7fae4cc00640 1 Processor -- start 2024-05-22T21:19:22.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.947+0000 7fae4cc00640 1 -- start start 2024-05-22T21:19:22.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.947+0000 7fae4cc00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fae4813d560 0x7fae480cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:22.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.947+0000 7fae4cc00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae480ce830 con 0x7fae4813cfe0 2024-05-22T21:19:22.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.947+0000 7fae4cc00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae480ceb10 con 0x7fae4815e1d0 2024-05-22T21:19:22.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.947+0000 7fae4cc00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae48140560 con 0x7fae4813d560 2024-05-22T21:19:22.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.947+0000 7fae45e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fae4813cfe0 0x7fae4815dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48840/0 (socket says 172.21.15.161:48840) 2024-05-22T21:19:22.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.947+0000 7fae45e00640 1 -- 172.21.15.161:0/708215267 learned_addr learned my addr 172.21.15.161:0/708215267 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:22.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae3d400640 1 --2- 172.21.15.161:0/708215267 >> v2:172.21.15.179:3300/0 conn(0x7fae4813d560 0x7fae480cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:22.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae45400640 1 -- 172.21.15.161:0/708215267 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400182122 0 0) 0x7fae480ce830 con 0x7fae4813cfe0 2024-05-22T21:19:22.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae45400640 1 -- 172.21.15.161:0/708215267 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae2c003530 con 0x7fae4813cfe0 2024-05-22T21:19:22.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae45400640 1 -- 172.21.15.161:0/708215267 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2279282690 0 0) 0x7fae2c003530 con 0x7fae4813cfe0 2024-05-22T21:19:22.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae45400640 1 -- 172.21.15.161:0/708215267 >> v2:172.21.15.179:3300/0 conn(0x7fae4813d560 msgr2=0x7fae480cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:22.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae45400640 1 --2- 172.21.15.161:0/708215267 >> v2:172.21.15.179:3300/0 conn(0x7fae4813d560 0x7fae480cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:22.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae45400640 1 -- 172.21.15.161:0/708215267 >> v1:172.21.15.179:6789/0 conn(0x7fae4815e1d0 legacy=0x7fae480ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:22.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae45400640 1 -- 172.21.15.161:0/708215267 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4806fd80 con 0x7fae4813cfe0 2024-05-22T21:19:22.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae45400640 1 -- 172.21.15.161:0/708215267 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fae38001680 con 0x7fae4813cfe0 2024-05-22T21:19:22.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae45400640 1 -- 172.21.15.161:0/708215267 >> v1:172.21.15.161:6789/0 conn(0x7fae4813cfe0 legacy=0x7fae4815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:22.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae45400640 1 --2- 172.21.15.161:0/708215267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae2c003980 0x7fae2c003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:22.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae45400640 1 --2- 172.21.15.161:0/708215267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae2c004380 0x7fae2c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:22.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae45400640 1 --2- 172.21.15.161:0/708215267 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae2c0050f0 0x7fae2c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:22.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae45400640 1 -- 172.21.15.161:0/708215267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae48140560 con 0x7fae2c004380 2024-05-22T21:19:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae45400640 1 -- 172.21.15.161:0/708215267 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c00c030 con 0x7fae2c003980 2024-05-22T21:19:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae45400640 1 -- 172.21.15.161:0/708215267 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c00c1f0 con 0x7fae2c0050f0 2024-05-22T21:19:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae3d400640 1 --2- 172.21.15.161:0/708215267 >> v2:172.21.15.179:3300/0 conn(0x7fae4813d560 0x7fae480cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae45e00640 1 --2- 172.21.15.161:0/708215267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae2c003980 0x7fae2c003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae3d400640 1 --2- 172.21.15.161:0/708215267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae2c004380 0x7fae2c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae46800640 1 --2- 172.21.15.161:0/708215267 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae2c0050f0 0x7fae2c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae3d400640 1 -- 172.21.15.161:0/708215267 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae2c0050f0 msgr2=0x7fae2c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae3d400640 1 --2- 172.21.15.161:0/708215267 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae2c0050f0 0x7fae2c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae3d400640 1 -- 172.21.15.161:0/708215267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae2c003980 msgr2=0x7fae2c003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae3d400640 1 --2- 172.21.15.161:0/708215267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae2c003980 0x7fae2c003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae3d400640 1 -- 172.21.15.161:0/708215267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c003530 con 0x7fae2c004380 2024-05-22T21:19:22.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.951+0000 7fae46800640 1 -- 172.21.15.161:0/708215267 reap_dead start 2024-05-22T21:19:22.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae3d400640 1 --2- 172.21.15.161:0/708215267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae2c004380 0x7fae2c004850 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fae30009c60 tx=0x7fae3001cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c861cbeb77a1d29b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:22.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae45400640 1 -- 172.21.15.161:0/708215267 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30004020 con 0x7fae2c004380 2024-05-22T21:19:22.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae45400640 1 -- 172.21.15.161:0/708215267 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae30004180 con 0x7fae2c004380 2024-05-22T21:19:22.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae45400640 1 -- 172.21.15.161:0/708215267 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae300a5760 con 0x7fae2c004380 2024-05-22T21:19:22.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae4cc00640 1 -- 172.21.15.161:0/708215267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae2c004380 msgr2=0x7fae2c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:22.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae4cc00640 1 --2- 172.21.15.161:0/708215267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae2c004380 0x7fae2c004850 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fae30009c60 tx=0x7fae3001cbb0 comp rx=0 tx=0).stop 2024-05-22T21:19:22.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae4cc00640 1 -- 172.21.15.161:0/708215267 shutdown_connections 2024-05-22T21:19:22.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae4cc00640 1 --2- 172.21.15.161:0/708215267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae2c004380 0x7fae2c004850 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:22.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae4cc00640 1 -- 172.21.15.161:0/708215267 >> 172.21.15.161:0/708215267 conn(0x7fae4806a0a0 msgr2=0x7fae4806a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:22.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae4cc00640 1 -- 172.21.15.161:0/708215267 shutdown_connections 2024-05-22T21:19:22.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae4cc00640 1 -- 172.21.15.161:0/708215267 wait complete. 2024-05-22T21:19:22.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae4cc00640 1 Processor -- start 2024-05-22T21:19:22.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae4cc00640 1 -- start start 2024-05-22T21:19:22.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae4cc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae2c004380 0x7fae4815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae4cc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae48159a40 0x7fae48131110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae4cc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae48159ec0 0x7fae48131650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae4cc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae481f3730 con 0x7fae48159ec0 2024-05-22T21:19:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae4cc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae481f2b20 con 0x7fae2c004380 2024-05-22T21:19:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae4cc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae481f2e00 con 0x7fae48159a40 2024-05-22T21:19:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae46800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae48159ec0 0x7fae48131650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae46800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae48159ec0 0x7fae48131650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44916/0 (socket says 172.21.15.161:44916) 2024-05-22T21:19:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.955+0000 7fae46800640 1 -- 172.21.15.161:0/3159688375 learned_addr learned my addr 172.21.15.161:0/3159688375 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae3d400640 1 --2- 172.21.15.161:0/3159688375 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae48159a40 0x7fae48131110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae45e00640 1 --2- 172.21.15.161:0/3159688375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae2c004380 0x7fae4815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae46800640 1 -- 172.21.15.161:0/3159688375 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae48159a40 msgr2=0x7fae48131110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae46800640 1 --2- 172.21.15.161:0/3159688375 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae48159a40 0x7fae48131110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:22.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae46800640 1 -- 172.21.15.161:0/3159688375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae2c004380 msgr2=0x7fae4815c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:22.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae46800640 1 --2- 172.21.15.161:0/3159688375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae2c004380 0x7fae4815c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:22.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae46800640 1 -- 172.21.15.161:0/3159688375 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae481f30e0 con 0x7fae48159ec0 2024-05-22T21:19:22.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae46800640 1 --2- 172.21.15.161:0/3159688375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae48159ec0 0x7fae48131650 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fae4806df00 tx=0x7fae4000f1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:22.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae3ea00640 1 -- 172.21.15.161:0/3159688375 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40006d70 con 0x7fae48159ec0 2024-05-22T21:19:22.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae3ea00640 1 -- 172.21.15.161:0/3159688375 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae40003980 con 0x7fae48159ec0 2024-05-22T21:19:22.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae4cc00640 1 -- 172.21.15.161:0/3159688375 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae481f33c0 con 0x7fae48159ec0 2024-05-22T21:19:22.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae3ea00640 1 -- 172.21.15.161:0/3159688375 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40003d30 con 0x7fae48159ec0 2024-05-22T21:19:22.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae4cc00640 1 -- 172.21.15.161:0/3159688375 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4815cd40 con 0x7fae48159ec0 2024-05-22T21:19:22.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae4cc00640 1 -- 172.21.15.161:0/3159688375 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae48067220 con 0x7fae48159ec0 2024-05-22T21:19:22.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae3ea00640 1 -- 172.21.15.161:0/3159688375 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fae4000f950 con 0x7fae48159ec0 2024-05-22T21:19:22.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae3ea00640 1 --2- 172.21.15.161:0/3159688375 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae240666c0 0x7fae24068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:22.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae3ea00640 1 -- 172.21.15.161:0/3159688375 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae400874f0 con 0x7fae48159ec0 2024-05-22T21:19:22.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.959+0000 7fae45e00640 1 --2- 172.21.15.161:0/3159688375 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae240666c0 0x7fae24068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:22.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.963+0000 7fae45e00640 1 --2- 172.21.15.161:0/3159688375 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae240666c0 0x7fae24068b80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fae48063da0 tx=0x7fae38000fd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:22.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:22.963+0000 7fae3ea00640 1 -- 172.21.15.161:0/3159688375 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fae40054870 con 0x7fae48159ec0 2024-05-22T21:19:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.123+0000 7fae4cc00640 1 -- 172.21.15.161:0/3159688375 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fae481f3550 con 0x7fae48159ec0 2024-05-22T21:19:23.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.387+0000 7fae3ea00640 1 -- 172.21.15.161:0/3159688375 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v126) v1 ==== 181+0+0 (secure 0 0 0) 0x7fae40057e90 con 0x7fae48159ec0 2024-05-22T21:19:23.388 INFO:teuthology.orchestra.run.smithi161.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-05-22T21:19:23.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.387+0000 7fae4cc00640 1 -- 172.21.15.161:0/3159688375 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae240666c0 msgr2=0x7fae24068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:23.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.387+0000 7fae4cc00640 1 --2- 172.21.15.161:0/3159688375 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae240666c0 0x7fae24068b80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fae48063da0 tx=0x7fae38000fd0 comp rx=0 tx=0).stop 2024-05-22T21:19:23.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.387+0000 7fae4cc00640 1 -- 172.21.15.161:0/3159688375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae48159ec0 msgr2=0x7fae48131650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:23.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.387+0000 7fae4cc00640 1 --2- 172.21.15.161:0/3159688375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae48159ec0 0x7fae48131650 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fae4806df00 tx=0x7fae4000f1d0 comp rx=0 tx=0).stop 2024-05-22T21:19:23.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.387+0000 7fae4cc00640 1 -- 172.21.15.161:0/3159688375 shutdown_connections 2024-05-22T21:19:23.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.387+0000 7fae4cc00640 1 --2- 172.21.15.161:0/3159688375 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae240666c0 0x7fae24068b80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:23.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.387+0000 7fae4cc00640 1 --2- 172.21.15.161:0/3159688375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae48159ec0 0x7fae48131650 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:23.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.387+0000 7fae4cc00640 1 --2- 172.21.15.161:0/3159688375 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae48159a40 0x7fae48131110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:23.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.387+0000 7fae4cc00640 1 --2- 172.21.15.161:0/3159688375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae2c004380 0x7fae4815c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:23.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.387+0000 7fae4cc00640 1 -- 172.21.15.161:0/3159688375 >> 172.21.15.161:0/3159688375 conn(0x7fae4806a0a0 msgr2=0x7fae48071300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:23.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.387+0000 7fae4cc00640 1 -- 172.21.15.161:0/3159688375 shutdown_connections 2024-05-22T21:19:23.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.387+0000 7fae4cc00640 1 -- 172.21.15.161:0/3159688375 wait complete. 2024-05-22T21:19:23.410 DEBUG:teuthology.orchestra.run.smithi161:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-22T21:19:23.411 DEBUG:teuthology.orchestra.run.smithi161:> import os 2024-05-22T21:19:23.411 DEBUG:teuthology.orchestra.run.smithi161:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-05-22T21:19:23.411 DEBUG:teuthology.orchestra.run.smithi161:> os.fchmod(fd, 0O666) 2024-05-22T21:19:23.411 DEBUG:teuthology.orchestra.run.smithi161:> os.fsync(fd) 2024-05-22T21:19:23.411 DEBUG:teuthology.orchestra.run.smithi161:> os.close(fd) 2024-05-22T21:19:23.411 DEBUG:teuthology.orchestra.run.smithi161:> ' 2024-05-22T21:19:23.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:23.664 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:19:23.719 INFO:teuthology.orchestra.run.smithi161.stdout:client7299 2024-05-22T21:19:23.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:23.719 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d6fa8528-b1bf-41b6-9f76-c0d715394087.client7299/osdmap') 2024-05-22T21:19:23.778 INFO:teuthology.orchestra.run.smithi161.stdout:epoch 125 barrier 125 flags 0x588000 2024-05-22T21:19:23.778 INFO:teuthology.orchestra.run.smithi161.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:19:23.778 INFO:teuthology.orchestra.run.smithi161.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:19:23.778 INFO:teuthology.orchestra.run.smithi161.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:19:23.779 INFO:teuthology.orchestra.run.smithi161.stdout:osd0 (1)172.21.15.161:6809 100% (exists, up) 100% -1 2024-05-22T21:19:23.779 INFO:teuthology.orchestra.run.smithi161.stdout:osd1 (1)172.21.15.161:6801 100% (exists, up) 100% -1 2024-05-22T21:19:23.779 INFO:teuthology.orchestra.run.smithi161.stdout:osd2 (1)172.21.15.161:6825 100% (exists, up) 100% -1 2024-05-22T21:19:23.779 INFO:teuthology.orchestra.run.smithi161.stdout:osd3 (1)172.21.15.161:6817 100% (exists, up) 100% -1 2024-05-22T21:19:23.779 INFO:teuthology.orchestra.run.smithi161.stdout:osd4 (1)172.21.15.179:6801 100% (exists, up) 100% -1 2024-05-22T21:19:23.779 INFO:teuthology.orchestra.run.smithi161.stdout:osd5 (1)172.21.15.179:6809 100% (exists, up) 100% -1 2024-05-22T21:19:23.779 INFO:teuthology.orchestra.run.smithi161.stdout:osd6 (1)172.21.15.179:6822 100% (exists, up) 100% -1 2024-05-22T21:19:23.779 INFO:teuthology.orchestra.run.smithi161.stdout:osd7 (1)172.21.15.179:6817 100% (exists, up) 100% -1 2024-05-22T21:19:23.780 INFO:teuthology.orchestra.run.smithi161.stderr:1+1 records in 2024-05-22T21:19:23.780 INFO:teuthology.orchestra.run.smithi161.stderr:1+1 records out 2024-05-22T21:19:23.780 INFO:teuthology.orchestra.run.smithi161.stderr:803 bytes copied, 6.2673e-05 s, 12.8 MB/s 2024-05-22T21:19:23.780 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2024-05-22T21:19:23.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.959+0000 7f6437800640 1 Processor -- start 2024-05-22T21:19:23.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6437800640 1 -- start start 2024-05-22T21:19:23.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6437800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f643013cfc0 0x7f64301402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:23.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6437800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f643015af60 0x7f643013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:23.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6437800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f643013fc80 0x7f643015dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:23.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6437800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64301400d0 con 0x7f643013fc80 2024-05-22T21:19:23.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6437800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643013d3a0 con 0x7f643013cfc0 2024-05-22T21:19:23.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6437800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64300cd920 con 0x7f643015af60 2024-05-22T21:19:23.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6435800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f643013fc80 0x7f643015dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:23.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6435800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f643013fc80 0x7f643015dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44920/0 (socket says 172.21.15.161:44920) 2024-05-22T21:19:23.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6435800640 1 -- 172.21.15.161:0/3690071916 learned_addr learned my addr 172.21.15.161:0/3690071916 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:23.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6434e00640 1 --2- 172.21.15.161:0/3690071916 >> v2:172.21.15.179:3300/0 conn(0x7f643013cfc0 0x7f64301402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:23.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6427e00640 1 --2- 172.21.15.161:0/3690071916 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f643015af60 0x7f643013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:23.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6435800640 1 -- 172.21.15.161:0/3690071916 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f643015af60 msgr2=0x7f643013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:23.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6435800640 1 --2- 172.21.15.161:0/3690071916 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f643015af60 0x7f643013f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:23.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6435800640 1 -- 172.21.15.161:0/3690071916 >> v2:172.21.15.179:3300/0 conn(0x7f643013cfc0 msgr2=0x7f64301402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:23.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6435800640 1 --2- 172.21.15.161:0/3690071916 >> v2:172.21.15.179:3300/0 conn(0x7f643013cfc0 0x7f64301402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:23.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6435800640 1 -- 172.21.15.161:0/3690071916 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64300cdc00 con 0x7f643013fc80 2024-05-22T21:19:23.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.963+0000 7f6434e00640 1 --2- 172.21.15.161:0/3690071916 >> v2:172.21.15.179:3300/0 conn(0x7f643013cfc0 0x7f64301402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:23.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.967+0000 7f6435800640 1 --2- 172.21.15.161:0/3690071916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643013fc80 0x7f643015dad0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f643006de90 tx=0x7f642801c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5feacb15dfcc5f58 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:23.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.967+0000 7f642fe00640 1 -- 172.21.15.161:0/3690071916 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280a6500 con 0x7f643013fc80 2024-05-22T21:19:23.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.967+0000 7f642fe00640 1 -- 172.21.15.161:0/3690071916 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64280a6660 con 0x7f643013fc80 2024-05-22T21:19:23.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.967+0000 7f642fe00640 1 -- 172.21.15.161:0/3690071916 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280a69f0 con 0x7f643013fc80 2024-05-22T21:19:23.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.967+0000 7f6437800640 1 -- 172.21.15.161:0/3690071916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643013fc80 msgr2=0x7f643015dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:23.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.967+0000 7f6437800640 1 --2- 172.21.15.161:0/3690071916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643013fc80 0x7f643015dad0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f643006de90 tx=0x7f642801c670 comp rx=0 tx=0).stop 2024-05-22T21:19:23.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.967+0000 7f6437800640 1 -- 172.21.15.161:0/3690071916 shutdown_connections 2024-05-22T21:19:23.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.967+0000 7f6437800640 1 --2- 172.21.15.161:0/3690071916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643013fc80 0x7f643015dad0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:23.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.967+0000 7f6437800640 1 --2- 172.21.15.161:0/3690071916 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f643015af60 0x7f643013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:23.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.967+0000 7f6437800640 1 --2- 172.21.15.161:0/3690071916 >> v2:172.21.15.179:3300/0 conn(0x7f643013cfc0 0x7f64301402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:23.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.967+0000 7f6437800640 1 -- 172.21.15.161:0/3690071916 >> 172.21.15.161:0/3690071916 conn(0x7f643006a2d0 msgr2=0x7f643006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:23.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.967+0000 7f6437800640 1 -- 172.21.15.161:0/3690071916 shutdown_connections 2024-05-22T21:19:23.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.967+0000 7f6437800640 1 -- 172.21.15.161:0/3690071916 wait complete. 2024-05-22T21:19:23.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.967+0000 7f6437800640 1 Processor -- start 2024-05-22T21:19:23.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.967+0000 7f6437800640 1 -- start start 2024-05-22T21:19:23.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6437800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643013cfc0 0x7f64301572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:23.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6437800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f643013fc80 0x7f64300b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:23.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6437800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f643015af60 0x7f643013bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:23.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6437800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6430157810 con 0x7f643013cfc0 2024-05-22T21:19:23.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6437800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64300b3b30 con 0x7f643015af60 2024-05-22T21:19:23.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6437800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64300b3850 con 0x7f643013fc80 2024-05-22T21:19:23.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6434e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643013cfc0 0x7f64301572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:23.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6434e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643013cfc0 0x7f64301572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44926/0 (socket says 172.21.15.161:44926) 2024-05-22T21:19:23.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6434e00640 1 -- 172.21.15.161:0/32801446 learned_addr learned my addr 172.21.15.161:0/32801446 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:23.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6435800640 1 --2- 172.21.15.161:0/32801446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f643015af60 0x7f643013bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:23.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6427e00640 1 --2- 172.21.15.161:0/32801446 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f643013fc80 0x7f64300b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:23.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6434e00640 1 -- 172.21.15.161:0/32801446 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f643013fc80 msgr2=0x7f64300b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:23.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6434e00640 1 --2- 172.21.15.161:0/32801446 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f643013fc80 0x7f64300b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:23.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6434e00640 1 -- 172.21.15.161:0/32801446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f643015af60 msgr2=0x7f643013bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:23.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6434e00640 1 --2- 172.21.15.161:0/32801446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f643015af60 0x7f643013bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:23.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6434e00640 1 -- 172.21.15.161:0/32801446 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6428009b30 con 0x7f643013cfc0 2024-05-22T21:19:23.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6427e00640 1 --2- 172.21.15.161:0/32801446 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f643013fc80 0x7f64300b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:23.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6434e00640 1 --2- 172.21.15.161:0/32801446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643013cfc0 0x7f64301572d0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f642000d800 tx=0x7f642000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:23.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f642d600640 1 -- 172.21.15.161:0/32801446 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642000b6c0 con 0x7f643013cfc0 2024-05-22T21:19:23.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f642d600640 1 -- 172.21.15.161:0/32801446 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642000b820 con 0x7f643013cfc0 2024-05-22T21:19:23.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6437800640 1 -- 172.21.15.161:0/32801446 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6430140410 con 0x7f643013cfc0 2024-05-22T21:19:23.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f642d600640 1 -- 172.21.15.161:0/32801446 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642000bbd0 con 0x7f643013cfc0 2024-05-22T21:19:23.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6437800640 1 -- 172.21.15.161:0/32801446 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64301409d0 con 0x7f643013cfc0 2024-05-22T21:19:23.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f6437800640 1 -- 172.21.15.161:0/32801446 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6430140270 con 0x7f643013cfc0 2024-05-22T21:19:23.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.971+0000 7f642d600640 1 -- 172.21.15.161:0/32801446 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f642000bd30 con 0x7f643013cfc0 2024-05-22T21:19:23.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.975+0000 7f642d600640 1 --2- 172.21.15.161:0/32801446 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f640c066690 0x7f640c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:23.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.975+0000 7f642d600640 1 -- 172.21.15.161:0/32801446 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64200845e0 con 0x7f643013cfc0 2024-05-22T21:19:23.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.975+0000 7f6427e00640 1 --2- 172.21.15.161:0/32801446 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f640c066690 0x7f640c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:23.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.975+0000 7f6427e00640 1 --2- 172.21.15.161:0/32801446 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f640c066690 0x7f640c068b50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f64300cc710 tx=0x7f6418008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:23.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:23.979+0000 7f642d600640 1 -- 172.21.15.161:0/32801446 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6430140270 con 0x7f643013cfc0 2024-05-22T21:19:24.078 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:19:24.082 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:19:24.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.147+0000 7f6437800640 1 -- 172.21.15.161:0/32801446 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7f6430140270 con 0x7f643013cfc0 2024-05-22T21:19:24.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.387+0000 7f642d600640 1 -- 172.21.15.161:0/32801446 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v127) v1 ==== 93+0+0 (secure 0 0 0) 0x7f6430140270 con 0x7f643013cfc0 2024-05-22T21:19:24.390 INFO:teuthology.orchestra.run.smithi161.stderr:pauserd,pausewr is set 2024-05-22T21:19:24.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.391+0000 7f6437800640 1 -- 172.21.15.161:0/32801446 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f640c066690 msgr2=0x7f640c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.391+0000 7f6437800640 1 --2- 172.21.15.161:0/32801446 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f640c066690 0x7f640c068b50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f64300cc710 tx=0x7f6418008040 comp rx=0 tx=0).stop 2024-05-22T21:19:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.391+0000 7f6437800640 1 -- 172.21.15.161:0/32801446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643013cfc0 msgr2=0x7f64301572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.391+0000 7f6437800640 1 --2- 172.21.15.161:0/32801446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643013cfc0 0x7f64301572d0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f642000d800 tx=0x7f642000dcc0 comp rx=0 tx=0).stop 2024-05-22T21:19:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.391+0000 7f6437800640 1 -- 172.21.15.161:0/32801446 shutdown_connections 2024-05-22T21:19:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.391+0000 7f6437800640 1 --2- 172.21.15.161:0/32801446 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f640c066690 0x7f640c068b50 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.391+0000 7f6437800640 1 --2- 172.21.15.161:0/32801446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f643015af60 0x7f643013bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.391+0000 7f6437800640 1 --2- 172.21.15.161:0/32801446 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f643013fc80 0x7f64300b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.391+0000 7f6437800640 1 --2- 172.21.15.161:0/32801446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f643013cfc0 0x7f64301572d0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.391+0000 7f6437800640 1 -- 172.21.15.161:0/32801446 >> 172.21.15.161:0/32801446 conn(0x7f643006a2d0 msgr2=0x7f643006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.391+0000 7f6437800640 1 -- 172.21.15.161:0/32801446 shutdown_connections 2024-05-22T21:19:24.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.391+0000 7f6437800640 1 -- 172.21.15.161:0/32801446 wait complete. 2024-05-22T21:19:24.411 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2024-05-22T21:19:24.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa31aa00640 1 Processor -- start 2024-05-22T21:19:24.549 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa31aa00640 1 -- start start 2024-05-22T21:19:24.549 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa31aa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa3141402e0 0x7fa31413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:24.549 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa31aa00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa31413fc80 0x7fa31406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:24.550 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa31aa00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa314140090 con 0x7fa31413cfc0 2024-05-22T21:19:24.550 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa31aa00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31413d3a0 con 0x7fa3141402e0 2024-05-22T21:19:24.550 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa31aa00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3140cd920 con 0x7fa31413fc80 2024-05-22T21:19:24.550 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa313e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa31413cfc0 0x7fa31415af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48850/0 (socket says 172.21.15.161:48850) 2024-05-22T21:19:24.550 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa313e00640 1 -- 172.21.15.161:0/3342200993 learned_addr learned my addr 172.21.15.161:0/3342200993 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:24.550 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa313400640 1 --2- 172.21.15.161:0/3342200993 >> v2:172.21.15.161:3300/0 conn(0x7fa3141402e0 0x7fa31413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:24.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa318a00640 1 --2- 172.21.15.161:0/3342200993 >> v2:172.21.15.179:3300/0 conn(0x7fa31413fc80 0x7fa31406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:24.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa312a00640 1 -- 172.21.15.161:0/3342200993 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143845745 0 0) 0x7fa314140090 con 0x7fa31413cfc0 2024-05-22T21:19:24.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa312a00640 1 -- 172.21.15.161:0/3342200993 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2f8003530 con 0x7fa31413cfc0 2024-05-22T21:19:24.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa313400640 1 -- 172.21.15.161:0/3342200993 >> v2:172.21.15.179:3300/0 conn(0x7fa31413fc80 msgr2=0x7fa31406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:24.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa313400640 1 --2- 172.21.15.161:0/3342200993 >> v2:172.21.15.179:3300/0 conn(0x7fa31413fc80 0x7fa31406fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:24.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa313400640 1 -- 172.21.15.161:0/3342200993 >> v1:172.21.15.161:6789/0 conn(0x7fa31413cfc0 legacy=0x7fa31415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:24.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa313400640 1 -- 172.21.15.161:0/3342200993 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3140cdc00 con 0x7fa3141402e0 2024-05-22T21:19:24.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.547+0000 7fa318a00640 1 --2- 172.21.15.161:0/3342200993 >> v2:172.21.15.179:3300/0 conn(0x7fa31413fc80 0x7fa31406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:24.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.551+0000 7fa313400640 1 --2- 172.21.15.161:0/3342200993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3141402e0 0x7fa31413f740 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa31406c5e0 tx=0x7fa30001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41de22b935fbfea3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:24.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.551+0000 7fa312a00640 1 -- 172.21.15.161:0/3342200993 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3000ab600 con 0x7fa3141402e0 2024-05-22T21:19:24.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.551+0000 7fa312a00640 1 -- 172.21.15.161:0/3342200993 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3000a7050 con 0x7fa3141402e0 2024-05-22T21:19:24.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.551+0000 7fa312a00640 1 -- 172.21.15.161:0/3342200993 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3000ab980 con 0x7fa3141402e0 2024-05-22T21:19:24.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.551+0000 7fa31aa00640 1 -- 172.21.15.161:0/3342200993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3141402e0 msgr2=0x7fa31413f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:24.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.551+0000 7fa31aa00640 1 --2- 172.21.15.161:0/3342200993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3141402e0 0x7fa31413f740 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa31406c5e0 tx=0x7fa30001c3c0 comp rx=0 tx=0).stop 2024-05-22T21:19:24.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.551+0000 7fa31aa00640 1 -- 172.21.15.161:0/3342200993 shutdown_connections 2024-05-22T21:19:24.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.551+0000 7fa31aa00640 1 --2- 172.21.15.161:0/3342200993 >> v2:172.21.15.179:3300/0 conn(0x7fa31413fc80 0x7fa31406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:24.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.551+0000 7fa31aa00640 1 --2- 172.21.15.161:0/3342200993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3141402e0 0x7fa31413f740 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:24.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.551+0000 7fa31aa00640 1 -- 172.21.15.161:0/3342200993 >> 172.21.15.161:0/3342200993 conn(0x7fa31406a2d0 msgr2=0x7fa31406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:24.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.551+0000 7fa31aa00640 1 -- 172.21.15.161:0/3342200993 shutdown_connections 2024-05-22T21:19:24.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.551+0000 7fa31aa00640 1 -- 172.21.15.161:0/3342200993 wait complete. 2024-05-22T21:19:24.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.551+0000 7fa31aa00640 1 Processor -- start 2024-05-22T21:19:24.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa31aa00640 1 -- start start 2024-05-22T21:19:24.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa31aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa31413cfc0 0x7fa31413ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:24.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa31aa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa31413fc80 0x7fa31415ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:24.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa31aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3141402e0 0x7fa31415c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:24.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa31aa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31415b270 con 0x7fa31413cfc0 2024-05-22T21:19:24.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa31aa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31415b540 con 0x7fa3141402e0 2024-05-22T21:19:24.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa31aa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31415b820 con 0x7fa31413fc80 2024-05-22T21:19:24.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa313e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa31413cfc0 0x7fa31413ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:24.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa313e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa31413cfc0 0x7fa31413ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44946/0 (socket says 172.21.15.161:44946) 2024-05-22T21:19:24.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa313e00640 1 -- 172.21.15.161:0/1765758034 learned_addr learned my addr 172.21.15.161:0/1765758034 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:24.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa313400640 1 --2- 172.21.15.161:0/1765758034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa31413fc80 0x7fa31415ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:24.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa318a00640 1 --2- 172.21.15.161:0/1765758034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3141402e0 0x7fa31415c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:24.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa313e00640 1 -- 172.21.15.161:0/1765758034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa31413fc80 msgr2=0x7fa31415ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:24.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa313e00640 1 --2- 172.21.15.161:0/1765758034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa31413fc80 0x7fa31415ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:24.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa313e00640 1 -- 172.21.15.161:0/1765758034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3141402e0 msgr2=0x7fa31415c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:24.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa313e00640 1 --2- 172.21.15.161:0/1765758034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3141402e0 0x7fa31415c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:24.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa313e00640 1 -- 172.21.15.161:0/1765758034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa300009420 con 0x7fa31413cfc0 2024-05-22T21:19:24.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa313e00640 1 --2- 172.21.15.161:0/1765758034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa31413cfc0 0x7fa31413ef90 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa30400c4d0 tx=0x7fa30400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:24.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa30be00640 1 -- 172.21.15.161:0/1765758034 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa304005cf0 con 0x7fa31413cfc0 2024-05-22T21:19:24.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa30be00640 1 -- 172.21.15.161:0/1765758034 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa304005e50 con 0x7fa31413cfc0 2024-05-22T21:19:24.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa30be00640 1 -- 172.21.15.161:0/1765758034 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa304007270 con 0x7fa31413cfc0 2024-05-22T21:19:24.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa31aa00640 1 -- 172.21.15.161:0/1765758034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31415bb00 con 0x7fa31413cfc0 2024-05-22T21:19:24.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa31aa00640 1 -- 172.21.15.161:0/1765758034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31415c0c0 con 0x7fa31413cfc0 2024-05-22T21:19:24.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.555+0000 7fa31aa00640 1 -- 172.21.15.161:0/1765758034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31415c2b0 con 0x7fa31413cfc0 2024-05-22T21:19:24.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.559+0000 7fa30be00640 1 -- 172.21.15.161:0/1765758034 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3040073d0 con 0x7fa31413cfc0 2024-05-22T21:19:24.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.559+0000 7fa30be00640 1 --2- 172.21.15.161:0/1765758034 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa2f4066690 0x7fa2f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:24.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.559+0000 7fa30be00640 1 -- 172.21.15.161:0/1765758034 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa304083730 con 0x7fa31413cfc0 2024-05-22T21:19:24.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.559+0000 7fa313400640 1 --2- 172.21.15.161:0/1765758034 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa2f4066690 0x7fa2f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:24.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.559+0000 7fa30be00640 1 -- 172.21.15.161:0/1765758034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=128+}) v3 -- 0x7fa2f406e1c0 con 0x7fa31413cfc0 2024-05-22T21:19:24.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.559+0000 7fa313400640 1 --2- 172.21.15.161:0/1765758034 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa2f4066690 0x7fa2f4068b50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa3000ad000 tx=0x7fa300008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:24.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.563+0000 7fa30be00640 1 -- 172.21.15.161:0/1765758034 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa304017060 con 0x7fa31413cfc0 2024-05-22T21:19:24.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:24.727+0000 7fa31aa00640 1 -- 172.21.15.161:0/1765758034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7fa314067220 con 0x7fa31413cfc0 2024-05-22T21:19:25.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.403+0000 7fa30be00640 1 -- 172.21.15.161:0/1765758034 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(128..128 src has 1..128) v4 ==== 288+0+0 (secure 0 0 0) 0x7fa304054770 con 0x7fa31413cfc0 2024-05-22T21:19:25.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.403+0000 7fa30be00640 1 -- 172.21.15.161:0/1765758034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=129}) v3 -- 0x7fa2f406ea70 con 0x7fa31413cfc0 2024-05-22T21:19:25.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.407+0000 7fa30be00640 1 -- 172.21.15.161:0/1765758034 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v128) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa304054540 con 0x7fa31413cfc0 2024-05-22T21:19:25.408 INFO:teuthology.orchestra.run.smithi161.stderr:pauserd,pausewr is unset 2024-05-22T21:19:25.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.407+0000 7fa31aa00640 1 -- 172.21.15.161:0/1765758034 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa2f4066690 msgr2=0x7fa2f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:25.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.407+0000 7fa31aa00640 1 --2- 172.21.15.161:0/1765758034 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa2f4066690 0x7fa2f4068b50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa3000ad000 tx=0x7fa300008040 comp rx=0 tx=0).stop 2024-05-22T21:19:25.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.407+0000 7fa31aa00640 1 -- 172.21.15.161:0/1765758034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa31413cfc0 msgr2=0x7fa31413ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:25.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.407+0000 7fa31aa00640 1 --2- 172.21.15.161:0/1765758034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa31413cfc0 0x7fa31413ef90 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa30400c4d0 tx=0x7fa30400c990 comp rx=0 tx=0).stop 2024-05-22T21:19:25.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.407+0000 7fa31aa00640 1 -- 172.21.15.161:0/1765758034 shutdown_connections 2024-05-22T21:19:25.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.407+0000 7fa31aa00640 1 --2- 172.21.15.161:0/1765758034 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa2f4066690 0x7fa2f4068b50 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:25.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.407+0000 7fa31aa00640 1 --2- 172.21.15.161:0/1765758034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3141402e0 0x7fa31415c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:25.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.407+0000 7fa31aa00640 1 --2- 172.21.15.161:0/1765758034 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa31413fc80 0x7fa31415ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:25.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.407+0000 7fa31aa00640 1 --2- 172.21.15.161:0/1765758034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa31413cfc0 0x7fa31413ef90 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:25.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.407+0000 7fa31aa00640 1 -- 172.21.15.161:0/1765758034 >> 172.21.15.161:0/1765758034 conn(0x7fa31406a2d0 msgr2=0x7fa3140ad020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:25.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.407+0000 7fa31aa00640 1 -- 172.21.15.161:0/1765758034 shutdown_connections 2024-05-22T21:19:25.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.407+0000 7fa31aa00640 1 -- 172.21.15.161:0/1765758034 wait complete. 2024-05-22T21:19:25.433 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:25.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.567+0000 7f565e400640 1 Processor -- start 2024-05-22T21:19:25.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.567+0000 7f565e400640 1 -- start start 2024-05-22T21:19:25.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.567+0000 7f565e400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f565815e130 0x7f565815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:25.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.567+0000 7f565e400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f565815d410 0x7f565815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:25.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.567+0000 7f565e400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5658071670 con 0x7f565815dd50 2024-05-22T21:19:25.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.567+0000 7f565e400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f565815d870 con 0x7f565815e130 2024-05-22T21:19:25.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.567+0000 7f565e400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5658070a60 con 0x7f565815d410 2024-05-22T21:19:25.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.567+0000 7f5656a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f565815e130 0x7f565815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:25.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.567+0000 7f5656a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f565815e130 0x7f565815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44954/0 (socket says 172.21.15.161:44954) 2024-05-22T21:19:25.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.567+0000 7f5656a00640 1 -- 172.21.15.161:0/378043655 learned_addr learned my addr 172.21.15.161:0/378043655 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:25.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.567+0000 7f5657e00640 1 --2- 172.21.15.161:0/378043655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f565815d410 0x7f565815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:25.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.567+0000 7f5656a00640 1 -- 172.21.15.161:0/378043655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f565815d410 msgr2=0x7f565815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:25.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.567+0000 7f5656a00640 1 --2- 172.21.15.161:0/378043655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f565815d410 0x7f565815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:25.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.567+0000 7f5656a00640 1 -- 172.21.15.161:0/378043655 >> v1:172.21.15.179:6789/0 conn(0x7f565815dd50 legacy=0x7f565806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:25.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.567+0000 7f5656a00640 1 -- 172.21.15.161:0/378043655 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5658070d40 con 0x7f565815e130 2024-05-22T21:19:25.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.571+0000 7f5656a00640 1 --2- 172.21.15.161:0/378043655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565815e130 0x7f565815ced0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5644009870 tx=0x7f564401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544a5e5f6bcb832a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:25.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.571+0000 7f5656000640 1 -- 172.21.15.161:0/378043655 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56440aa610 con 0x7f565815e130 2024-05-22T21:19:25.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.571+0000 7f5656000640 1 -- 172.21.15.161:0/378043655 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56440a6040 con 0x7f565815e130 2024-05-22T21:19:25.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.571+0000 7f5656000640 1 -- 172.21.15.161:0/378043655 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56440aa990 con 0x7f565815e130 2024-05-22T21:19:25.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.571+0000 7f565e400640 1 -- 172.21.15.161:0/378043655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565815e130 msgr2=0x7f565815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:25.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.571+0000 7f565e400640 1 --2- 172.21.15.161:0/378043655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565815e130 0x7f565815ced0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5644009870 tx=0x7f564401c4f0 comp rx=0 tx=0).stop 2024-05-22T21:19:25.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.571+0000 7f565e400640 1 -- 172.21.15.161:0/378043655 shutdown_connections 2024-05-22T21:19:25.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.571+0000 7f565e400640 1 --2- 172.21.15.161:0/378043655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f565815d410 0x7f565815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:25.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.571+0000 7f565e400640 1 --2- 172.21.15.161:0/378043655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565815e130 0x7f565815ced0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:25.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.571+0000 7f565e400640 1 -- 172.21.15.161:0/378043655 >> 172.21.15.161:0/378043655 conn(0x7f56580b0bc0 msgr2=0x7f56580b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:25.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.571+0000 7f565e400640 1 -- 172.21.15.161:0/378043655 shutdown_connections 2024-05-22T21:19:25.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.571+0000 7f565e400640 1 -- 172.21.15.161:0/378043655 wait complete. 2024-05-22T21:19:25.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f565e400640 1 Processor -- start 2024-05-22T21:19:25.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f565e400640 1 -- start start 2024-05-22T21:19:25.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f565e400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565815d410 0x7f5658141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:25.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f565e400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f565815dd50 0x7f5658157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:25.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f565e400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f565815e130 0x7f56580cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:25.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f565e400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56580b2ac0 con 0x7f565815d410 2024-05-22T21:19:25.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f565e400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5658158050 con 0x7f565815dd50 2024-05-22T21:19:25.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f565e400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5658158330 con 0x7f565815e130 2024-05-22T21:19:25.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f5657400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565815d410 0x7f5658141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:25.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f5657400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565815d410 0x7f5658141930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44966/0 (socket says 172.21.15.161:44966) 2024-05-22T21:19:25.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f5657400640 1 -- 172.21.15.161:0/395157961 learned_addr learned my addr 172.21.15.161:0/395157961 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:25.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f5656a00640 1 --2- 172.21.15.161:0/395157961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f565815dd50 0x7f5658157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:25.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f5657e00640 1 --2- 172.21.15.161:0/395157961 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f565815e130 0x7f56580cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:25.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f5657400640 1 -- 172.21.15.161:0/395157961 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f565815e130 msgr2=0x7f56580cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:25.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f5657400640 1 --2- 172.21.15.161:0/395157961 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f565815e130 0x7f56580cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:25.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f5657400640 1 -- 172.21.15.161:0/395157961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f565815dd50 msgr2=0x7f5658157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:25.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f5657400640 1 --2- 172.21.15.161:0/395157961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f565815dd50 0x7f5658157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:25.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f5657400640 1 -- 172.21.15.161:0/395157961 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56440094d0 con 0x7f565815d410 2024-05-22T21:19:25.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f5657400640 1 --2- 172.21.15.161:0/395157961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565815d410 0x7f5658141930 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f564800c7c0 tx=0x7f564800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:25.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f564f400640 1 -- 172.21.15.161:0/395157961 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564800aa50 con 0x7f565815d410 2024-05-22T21:19:25.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f564f400640 1 -- 172.21.15.161:0/395157961 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564800abb0 con 0x7f565815d410 2024-05-22T21:19:25.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f564f400640 1 -- 172.21.15.161:0/395157961 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648012420 con 0x7f565815d410 2024-05-22T21:19:25.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f565e400640 1 -- 172.21.15.161:0/395157961 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5658158610 con 0x7f565815d410 2024-05-22T21:19:25.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.575+0000 7f565e400640 1 -- 172.21.15.161:0/395157961 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5658153590 con 0x7f565815d410 2024-05-22T21:19:25.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.579+0000 7f565e400640 1 -- 172.21.15.161:0/395157961 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56580b2890 con 0x7f565815d410 2024-05-22T21:19:25.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.579+0000 7f564f400640 1 -- 172.21.15.161:0/395157961 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5648012580 con 0x7f565815d410 2024-05-22T21:19:25.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.579+0000 7f564f400640 1 --2- 172.21.15.161:0/395157961 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5638066690 0x7f5638068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:25.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.579+0000 7f564f400640 1 -- 172.21.15.161:0/395157961 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5648084430 con 0x7f565815d410 2024-05-22T21:19:25.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.579+0000 7f5656a00640 1 --2- 172.21.15.161:0/395157961 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5638066690 0x7f5638068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:25.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.579+0000 7f5656a00640 1 --2- 172.21.15.161:0/395157961 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5638066690 0x7f5638068b50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f56440ae000 tx=0x7f56440a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:25.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.583+0000 7f564f400640 1 -- 172.21.15.161:0/395157961 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5648051790 con 0x7f565815d410 2024-05-22T21:19:25.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.739+0000 7f565e400640 1 -- 172.21.15.161:0/395157961 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5658067220 con 0x7f565815d410 2024-05-22T21:19:25.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.739+0000 7f564f400640 1 -- 172.21.15.161:0/395157961 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16107 (secure 0 0 0) 0x7f5648054db0 con 0x7f565815d410 2024-05-22T21:19:25.743 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:25.743 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":128,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:19:25.399186+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:12.848545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:13.344629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:25.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.743+0000 7f565e400640 1 -- 172.21.15.161:0/395157961 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5638066690 msgr2=0x7f5638068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:25.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.743+0000 7f565e400640 1 --2- 172.21.15.161:0/395157961 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5638066690 0x7f5638068b50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f56440ae000 tx=0x7f56440a7000 comp rx=0 tx=0).stop 2024-05-22T21:19:25.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.743+0000 7f565e400640 1 -- 172.21.15.161:0/395157961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565815d410 msgr2=0x7f5658141930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:25.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.743+0000 7f565e400640 1 --2- 172.21.15.161:0/395157961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565815d410 0x7f5658141930 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f564800c7c0 tx=0x7f564800cc80 comp rx=0 tx=0).stop 2024-05-22T21:19:25.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.743+0000 7f565e400640 1 -- 172.21.15.161:0/395157961 shutdown_connections 2024-05-22T21:19:25.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.743+0000 7f565e400640 1 --2- 172.21.15.161:0/395157961 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5638066690 0x7f5638068b50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:25.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.743+0000 7f565e400640 1 --2- 172.21.15.161:0/395157961 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f565815e130 0x7f56580cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:25.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.743+0000 7f565e400640 1 --2- 172.21.15.161:0/395157961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f565815dd50 0x7f5658157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:25.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.743+0000 7f565e400640 1 --2- 172.21.15.161:0/395157961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565815d410 0x7f5658141930 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:25.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.743+0000 7f565e400640 1 -- 172.21.15.161:0/395157961 >> 172.21.15.161:0/395157961 conn(0x7f56580b0bc0 msgr2=0x7f56580aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:25.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.743+0000 7f565e400640 1 -- 172.21.15.161:0/395157961 shutdown_connections 2024-05-22T21:19:25.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:25.743+0000 7f565e400640 1 -- 172.21.15.161:0/395157961 wait complete. 2024-05-22T21:19:25.765 DEBUG:teuthology.orchestra.run.smithi161:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-22T21:19:25.765 DEBUG:teuthology.orchestra.run.smithi161:> import os 2024-05-22T21:19:25.765 DEBUG:teuthology.orchestra.run.smithi161:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-05-22T21:19:25.765 DEBUG:teuthology.orchestra.run.smithi161:> os.fchmod(fd, 0O666) 2024-05-22T21:19:25.765 DEBUG:teuthology.orchestra.run.smithi161:> os.fsync(fd) 2024-05-22T21:19:25.765 DEBUG:teuthology.orchestra.run.smithi161:> os.close(fd) 2024-05-22T21:19:25.765 DEBUG:teuthology.orchestra.run.smithi161:> ' 2024-05-22T21:19:26.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:26.017 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:19:26.070 INFO:teuthology.orchestra.run.smithi161.stdout:client7299 2024-05-22T21:19:26.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:26.071 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d6fa8528-b1bf-41b6-9f76-c0d715394087.client7299/osdmap') 2024-05-22T21:19:26.130 INFO:teuthology.orchestra.run.smithi161.stdout:epoch 125 barrier 125 flags 0x588000 2024-05-22T21:19:26.130 INFO:teuthology.orchestra.run.smithi161.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:19:26.130 INFO:teuthology.orchestra.run.smithi161.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:19:26.130 INFO:teuthology.orchestra.run.smithi161.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:19:26.130 INFO:teuthology.orchestra.run.smithi161.stdout:osd0 (1)172.21.15.161:6809 100% (exists, up) 100% -1 2024-05-22T21:19:26.130 INFO:teuthology.orchestra.run.smithi161.stdout:osd1 (1)172.21.15.161:6801 100% (exists, up) 100% -1 2024-05-22T21:19:26.130 INFO:teuthology.orchestra.run.smithi161.stdout:osd2 (1)172.21.15.161:6825 100% (exists, up) 100% -1 2024-05-22T21:19:26.130 INFO:teuthology.orchestra.run.smithi161.stdout:osd3 (1)172.21.15.161:6817 100% (exists, up) 100% -1 2024-05-22T21:19:26.131 INFO:teuthology.orchestra.run.smithi161.stdout:osd4 (1)172.21.15.179:6801 100% (exists, up) 100% -1 2024-05-22T21:19:26.131 INFO:teuthology.orchestra.run.smithi161.stdout:osd5 (1)172.21.15.179:6809 100% (exists, up) 100% -1 2024-05-22T21:19:26.131 INFO:teuthology.orchestra.run.smithi161.stdout:osd6 (1)172.21.15.179:6822 100% (exists, up) 100% -1 2024-05-22T21:19:26.131 INFO:teuthology.orchestra.run.smithi161.stdout:osd7 (1)172.21.15.179:6817 100% (exists, up) 100% -1 2024-05-22T21:19:26.131 INFO:teuthology.orchestra.run.smithi161.stderr:1+1 records in 2024-05-22T21:19:26.131 INFO:teuthology.orchestra.run.smithi161.stderr:1+1 records out 2024-05-22T21:19:26.131 INFO:teuthology.orchestra.run.smithi161.stderr:803 bytes copied, 6.5723e-05 s, 12.2 MB/s 2024-05-22T21:19:26.132 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:26.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa683e00640 1 Processor -- start 2024-05-22T21:19:26.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa683e00640 1 -- start start 2024-05-22T21:19:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa683e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa6840b2590 0x7fa68406d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa683e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa684160cc0 0x7fa684161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa683e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68406bb90 con 0x7fa68415d060 2024-05-22T21:19:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa683e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa684161690 con 0x7fa684160cc0 2024-05-22T21:19:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa683e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6840b3080 con 0x7fa6840b2590 2024-05-22T21:19:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa682a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa68415d060 0x7fa684070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47304/0 (socket says 172.21.15.161:47304) 2024-05-22T21:19:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa682a00640 1 -- 172.21.15.161:0/3969519117 learned_addr learned my addr 172.21.15.161:0/3969519117 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa683400640 1 --2- 172.21.15.161:0/3969519117 >> v2:172.21.15.161:3300/0 conn(0x7fa684160cc0 0x7fa684161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa67be00640 1 --2- 172.21.15.161:0/3969519117 >> v2:172.21.15.179:3300/0 conn(0x7fa6840b2590 0x7fa68406d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa682000640 1 -- 172.21.15.161:0/3969519117 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489112908 0 0) 0x7fa68406bb90 con 0x7fa68415d060 2024-05-22T21:19:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa682000640 1 -- 172.21.15.161:0/3969519117 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa668003530 con 0x7fa68415d060 2024-05-22T21:19:26.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa683400640 1 -- 172.21.15.161:0/3969519117 >> v2:172.21.15.179:3300/0 conn(0x7fa6840b2590 msgr2=0x7fa68406d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:26.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa683400640 1 --2- 172.21.15.161:0/3969519117 >> v2:172.21.15.179:3300/0 conn(0x7fa6840b2590 0x7fa68406d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:26.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa683400640 1 -- 172.21.15.161:0/3969519117 >> v1:172.21.15.161:6789/0 conn(0x7fa68415d060 legacy=0x7fa684070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:26.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.307+0000 7fa683400640 1 -- 172.21.15.161:0/3969519117 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa684161b10 con 0x7fa684160cc0 2024-05-22T21:19:26.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.311+0000 7fa683400640 1 --2- 172.21.15.161:0/3969519117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa684160cc0 0x7fa684161150 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa6840cfc70 tx=0x7fa67c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64dcfdc83dfba05 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:26.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.311+0000 7fa682000640 1 -- 172.21.15.161:0/3969519117 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c0a58e0 con 0x7fa684160cc0 2024-05-22T21:19:26.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.311+0000 7fa682000640 1 -- 172.21.15.161:0/3969519117 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67c0a5a40 con 0x7fa684160cc0 2024-05-22T21:19:26.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.311+0000 7fa682000640 1 -- 172.21.15.161:0/3969519117 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c0a5dd0 con 0x7fa684160cc0 2024-05-22T21:19:26.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.311+0000 7fa683e00640 1 -- 172.21.15.161:0/3969519117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa684160cc0 msgr2=0x7fa684161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:26.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.311+0000 7fa683e00640 1 --2- 172.21.15.161:0/3969519117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa684160cc0 0x7fa684161150 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa6840cfc70 tx=0x7fa67c01c930 comp rx=0 tx=0).stop 2024-05-22T21:19:26.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.311+0000 7fa683e00640 1 -- 172.21.15.161:0/3969519117 shutdown_connections 2024-05-22T21:19:26.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.311+0000 7fa683e00640 1 --2- 172.21.15.161:0/3969519117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa684160cc0 0x7fa684161150 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:26.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.311+0000 7fa683e00640 1 --2- 172.21.15.161:0/3969519117 >> v2:172.21.15.179:3300/0 conn(0x7fa6840b2590 0x7fa68406d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:26.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.311+0000 7fa683e00640 1 -- 172.21.15.161:0/3969519117 >> 172.21.15.161:0/3969519117 conn(0x7fa68406a4d0 msgr2=0x7fa6840b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:26.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.311+0000 7fa683e00640 1 -- 172.21.15.161:0/3969519117 shutdown_connections 2024-05-22T21:19:26.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.311+0000 7fa683e00640 1 -- 172.21.15.161:0/3969519117 wait complete. 2024-05-22T21:19:26.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa683e00640 1 Processor -- start 2024-05-22T21:19:26.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa683e00640 1 -- start start 2024-05-22T21:19:26.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa683e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6840b2590 0x7fa6840cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:26.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa683e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa68415d060 0x7fa684157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:26.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa683e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa684160cc0 0x7fa684151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:26.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa683e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa684151f00 con 0x7fa6840b2590 2024-05-22T21:19:26.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa683e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa684156d40 con 0x7fa68415d060 2024-05-22T21:19:26.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa683e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa684152150 con 0x7fa684160cc0 2024-05-22T21:19:26.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa682a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6840b2590 0x7fa6840cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:26.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa682a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6840b2590 0x7fa6840cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57532/0 (socket says 172.21.15.161:57532) 2024-05-22T21:19:26.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa682a00640 1 -- 172.21.15.161:0/2347871441 learned_addr learned my addr 172.21.15.161:0/2347871441 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:26.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa67be00640 1 --2- 172.21.15.161:0/2347871441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa68415d060 0x7fa684157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:26.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa683400640 1 --2- 172.21.15.161:0/2347871441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa684160cc0 0x7fa684151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:26.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa682a00640 1 -- 172.21.15.161:0/2347871441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa684160cc0 msgr2=0x7fa684151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:26.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa682a00640 1 --2- 172.21.15.161:0/2347871441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa684160cc0 0x7fa684151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:26.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa682a00640 1 -- 172.21.15.161:0/2347871441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa68415d060 msgr2=0x7fa684157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:26.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa682a00640 1 --2- 172.21.15.161:0/2347871441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa68415d060 0x7fa684157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:26.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa682a00640 1 -- 172.21.15.161:0/2347871441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c009a70 con 0x7fa6840b2590 2024-05-22T21:19:26.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa682a00640 1 --2- 172.21.15.161:0/2347871441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6840b2590 0x7fa6840cabf0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa67400c450 tx=0x7fa67400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:26.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa67aa00640 1 -- 172.21.15.161:0/2347871441 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa674005cf0 con 0x7fa6840b2590 2024-05-22T21:19:26.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa67aa00640 1 -- 172.21.15.161:0/2347871441 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa674005e50 con 0x7fa6840b2590 2024-05-22T21:19:26.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa683e00640 1 -- 172.21.15.161:0/2347871441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa684152430 con 0x7fa6840b2590 2024-05-22T21:19:26.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa67aa00640 1 -- 172.21.15.161:0/2347871441 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa674007290 con 0x7fa6840b2590 2024-05-22T21:19:26.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa683e00640 1 -- 172.21.15.161:0/2347871441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa684156960 con 0x7fa6840b2590 2024-05-22T21:19:26.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.315+0000 7fa683e00640 1 -- 172.21.15.161:0/2347871441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa684067070 con 0x7fa6840b2590 2024-05-22T21:19:26.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.319+0000 7fa67aa00640 1 -- 172.21.15.161:0/2347871441 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa6740073f0 con 0x7fa6840b2590 2024-05-22T21:19:26.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.319+0000 7fa67aa00640 1 --2- 172.21.15.161:0/2347871441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6600666c0 0x7fa660068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:26.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.319+0000 7fa67aa00640 1 -- 172.21.15.161:0/2347871441 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa674083660 con 0x7fa6840b2590 2024-05-22T21:19:26.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.319+0000 7fa67be00640 1 --2- 172.21.15.161:0/2347871441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6600666c0 0x7fa660068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:26.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.319+0000 7fa67be00640 1 --2- 172.21.15.161:0/2347871441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6600666c0 0x7fa660068b80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa684167e50 tx=0x7fa66c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:26.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.323+0000 7fa67aa00640 1 -- 172.21.15.161:0/2347871441 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa674050a70 con 0x7fa6840b2590 2024-05-22T21:19:26.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.511+0000 7fa683e00640 1 -- 172.21.15.161:0/2347871441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa684070080 con 0x7fa6840b2590 2024-05-22T21:19:26.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.515+0000 7fa67aa00640 1 -- 172.21.15.161:0/2347871441 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4748 (secure 0 0 0) 0x7fa674017090 con 0x7fa6840b2590 2024-05-22T21:19:26.516 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:26.516 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3944033857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3944033857},{"type":"v1","addr":"172.21.15.179:6834","nonce":3944033857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6125,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/4216709187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4216709187},{"type":"v1","addr":"172.21.15.161:6836","nonce":4216709187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2364917653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2364917653},{"type":"v1","addr":"172.21.15.179:6835","nonce":2364917653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:14.345936+0000","modified":"2024-05-22T21:19:17.355628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/1424863663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1424863663},{"type":"v1","addr":"172.21.15.161:6837","nonce":1424863663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-22T21:19:26.517 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 54 2024-05-22T21:19:26.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.515+0000 7fa683e00640 1 -- 172.21.15.161:0/2347871441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6600666c0 msgr2=0x7fa660068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:26.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.515+0000 7fa683e00640 1 --2- 172.21.15.161:0/2347871441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6600666c0 0x7fa660068b80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa684167e50 tx=0x7fa66c009190 comp rx=0 tx=0).stop 2024-05-22T21:19:26.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.515+0000 7fa683e00640 1 -- 172.21.15.161:0/2347871441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6840b2590 msgr2=0x7fa6840cabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:26.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.515+0000 7fa683e00640 1 --2- 172.21.15.161:0/2347871441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6840b2590 0x7fa6840cabf0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa67400c450 tx=0x7fa67400c910 comp rx=0 tx=0).stop 2024-05-22T21:19:26.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.515+0000 7fa683e00640 1 -- 172.21.15.161:0/2347871441 shutdown_connections 2024-05-22T21:19:26.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.515+0000 7fa683e00640 1 --2- 172.21.15.161:0/2347871441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6600666c0 0x7fa660068b80 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:26.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.515+0000 7fa683e00640 1 --2- 172.21.15.161:0/2347871441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa684160cc0 0x7fa684151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:26.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.515+0000 7fa683e00640 1 --2- 172.21.15.161:0/2347871441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa68415d060 0x7fa684157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:26.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.515+0000 7fa683e00640 1 --2- 172.21.15.161:0/2347871441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6840b2590 0x7fa6840cabf0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:26.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.515+0000 7fa683e00640 1 -- 172.21.15.161:0/2347871441 >> 172.21.15.161:0/2347871441 conn(0x7fa68406a4d0 msgr2=0x7fa6840cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:26.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.515+0000 7fa683e00640 1 -- 172.21.15.161:0/2347871441 shutdown_connections 2024-05-22T21:19:26.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:26.515+0000 7fa683e00640 1 -- 172.21.15.161:0/2347871441 wait complete. 2024-05-22T21:19:26.538 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json osdmap barrier 128 2024-05-22T21:19:26.691 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-05-22T21:19:26.691 DEBUG:teuthology.orchestra.run.smithi161:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-22T21:19:26.691 DEBUG:teuthology.orchestra.run.smithi161:> import os 2024-05-22T21:19:26.691 DEBUG:teuthology.orchestra.run.smithi161:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2024-05-22T21:19:26.691 DEBUG:teuthology.orchestra.run.smithi161:> os.fchmod(fd, 0O666) 2024-05-22T21:19:26.691 DEBUG:teuthology.orchestra.run.smithi161:> os.fsync(fd) 2024-05-22T21:19:26.691 DEBUG:teuthology.orchestra.run.smithi161:> os.close(fd) 2024-05-22T21:19:26.691 DEBUG:teuthology.orchestra.run.smithi161:> ' 2024-05-22T21:19:26.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:26.943 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:19:26.995 INFO:teuthology.orchestra.run.smithi161.stdout:client7299 2024-05-22T21:19:26.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:26.996 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d6fa8528-b1bf-41b6-9f76-c0d715394087.client7299/osdmap') 2024-05-22T21:19:27.062 INFO:teuthology.orchestra.run.smithi161.stdout:epoch 128 barrier 128 flags 0x588000 2024-05-22T21:19:27.062 INFO:teuthology.orchestra.run.smithi161.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:19:27.062 INFO:teuthology.orchestra.run.smithi161.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:19:27.062 INFO:teuthology.orchestra.run.smithi161.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:19:27.062 INFO:teuthology.orchestra.run.smithi161.stdout:osd0 (1)172.21.15.161:6809 100% (exists, up) 100% -1 2024-05-22T21:19:27.062 INFO:teuthology.orchestra.run.smithi161.stdout:osd1 (1)172.21.15.161:6801 100% (exists, up) 100% -1 2024-05-22T21:19:27.063 INFO:teuthology.orchestra.run.smithi161.stdout:osd2 (1)172.21.15.161:6825 100% (exists, up) 100% -1 2024-05-22T21:19:27.063 INFO:teuthology.orchestra.run.smithi161.stdout:osd3 (1)172.21.15.161:6817 100% (exists, up) 100% -1 2024-05-22T21:19:27.063 INFO:teuthology.orchestra.run.smithi161.stdout:osd4 (1)172.21.15.179:6801 100% (exists, up) 100% -1 2024-05-22T21:19:27.063 INFO:teuthology.orchestra.run.smithi161.stdout:osd5 (1)172.21.15.179:6809 100% (exists, up) 100% -1 2024-05-22T21:19:27.063 INFO:teuthology.orchestra.run.smithi161.stdout:osd6 (1)172.21.15.179:6822 100% (exists, up) 100% -1 2024-05-22T21:19:27.063 INFO:teuthology.orchestra.run.smithi161.stdout:osd7 (1)172.21.15.179:6817 100% (exists, up) 100% -1 2024-05-22T21:19:27.063 INFO:teuthology.orchestra.run.smithi161.stderr:1+1 records in 2024-05-22T21:19:27.063 INFO:teuthology.orchestra.run.smithi161.stderr:1+1 records out 2024-05-22T21:19:27.063 INFO:teuthology.orchestra.run.smithi161.stderr:803 bytes copied, 6.3344e-05 s, 12.7 MB/s 2024-05-22T21:19:27.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:27.064 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:19:27.117 INFO:teuthology.orchestra.run.smithi161.stdout:client7299 2024-05-22T21:19:27.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:27.118 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d6fa8528-b1bf-41b6-9f76-c0d715394087.client7299/osdmap') 2024-05-22T21:19:27.181 INFO:teuthology.orchestra.run.smithi161.stdout:epoch 128 barrier 128 flags 0x588000 2024-05-22T21:19:27.181 INFO:teuthology.orchestra.run.smithi161.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:19:27.182 INFO:teuthology.orchestra.run.smithi161.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:19:27.182 INFO:teuthology.orchestra.run.smithi161.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-05-22T21:19:27.182 INFO:teuthology.orchestra.run.smithi161.stdout:osd0 (1)172.21.15.161:6809 100% (exists, up) 100% -1 2024-05-22T21:19:27.182 INFO:teuthology.orchestra.run.smithi161.stdout:osd1 (1)172.21.15.161:6801 100% (exists, up) 100% -1 2024-05-22T21:19:27.182 INFO:teuthology.orchestra.run.smithi161.stdout:osd2 (1)172.21.15.161:6825 100% (exists, up) 100% -1 2024-05-22T21:19:27.182 INFO:teuthology.orchestra.run.smithi161.stdout:osd3 (1)172.21.15.161:6817 100% (exists, up) 100% -1 2024-05-22T21:19:27.182 INFO:teuthology.orchestra.run.smithi161.stdout:osd4 (1)172.21.15.179:6801 100% (exists, up) 100% -1 2024-05-22T21:19:27.182 INFO:teuthology.orchestra.run.smithi161.stdout:osd5 (1)172.21.15.179:6809 100% (exists, up) 100% -1 2024-05-22T21:19:27.182 INFO:teuthology.orchestra.run.smithi161.stdout:osd6 (1)172.21.15.179:6822 100% (exists, up) 100% -1 2024-05-22T21:19:27.182 INFO:teuthology.orchestra.run.smithi161.stdout:osd7 (1)172.21.15.179:6817 100% (exists, up) 100% -1 2024-05-22T21:19:27.182 INFO:teuthology.orchestra.run.smithi161.stderr:1+1 records in 2024-05-22T21:19:27.183 INFO:teuthology.orchestra.run.smithi161.stderr:1+1 records out 2024-05-22T21:19:27.183 INFO:teuthology.orchestra.run.smithi161.stderr:803 bytes copied, 6.247e-05 s, 12.9 MB/s 2024-05-22T21:19:27.183 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-22T21:19:27.183 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:19:27.184 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:19:27.342 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:19:27.443 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:19:27.443 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:19:27.444 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:27.452 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:19:27.452 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:27.503 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:27.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.679+0000 7effddc00640 1 Processor -- start 2024-05-22T21:19:27.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.679+0000 7effddc00640 1 -- start start 2024-05-22T21:19:27.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effddc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effd815dd50 0x7effd806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:27.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effddc00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7effd815e130 0x7effd815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:27.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effddc00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd8071670 con 0x7effd815d410 2024-05-22T21:19:27.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effddc00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd815d8a0 con 0x7effd815e130 2024-05-22T21:19:27.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effddc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd806de40 con 0x7effd815dd50 2024-05-22T21:19:27.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd7400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7effd815d410 0x7effd815a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47320/0 (socket says 172.21.15.161:47320) 2024-05-22T21:19:27.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd7400640 1 -- 172.21.15.161:0/1086505407 learned_addr learned my addr 172.21.15.161:0/1086505407 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:27.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6a00640 1 --2- 172.21.15.161:0/1086505407 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effd815dd50 0x7effd806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:27.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effce000640 1 --2- 172.21.15.161:0/1086505407 >> v2:172.21.15.179:3300/0 conn(0x7effd815e130 0x7effd815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:27.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6000640 1 -- 172.21.15.161:0/1086505407 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271979251 0 0) 0x7effd8071670 con 0x7effd815d410 2024-05-22T21:19:27.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6000640 1 -- 172.21.15.161:0/1086505407 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effbc003530 con 0x7effd815d410 2024-05-22T21:19:27.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6000640 1 -- 172.21.15.161:0/1086505407 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1468726203 0 0) 0x7effbc003530 con 0x7effd815d410 2024-05-22T21:19:27.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6000640 1 -- 172.21.15.161:0/1086505407 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effd815dd50 msgr2=0x7effd806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:27.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6000640 1 --2- 172.21.15.161:0/1086505407 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effd815dd50 0x7effd806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:27.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6000640 1 -- 172.21.15.161:0/1086505407 >> v2:172.21.15.179:3300/0 conn(0x7effd815e130 msgr2=0x7effd815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:27.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6000640 1 --2- 172.21.15.161:0/1086505407 >> v2:172.21.15.179:3300/0 conn(0x7effd815e130 0x7effd815ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:27.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6000640 1 -- 172.21.15.161:0/1086505407 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd8070a60 con 0x7effd815d410 2024-05-22T21:19:27.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6000640 1 -- 172.21.15.161:0/1086505407 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7effd0001a00 con 0x7effd815d410 2024-05-22T21:19:27.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6000640 1 -- 172.21.15.161:0/1086505407 >> v1:172.21.15.161:6789/0 conn(0x7effd815d410 legacy=0x7effd815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:27.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6000640 1 --2- 172.21.15.161:0/1086505407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effbc003980 0x7effbc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:27.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6000640 1 --2- 172.21.15.161:0/1086505407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effbc0047d0 0x7effbc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:27.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6000640 1 --2- 172.21.15.161:0/1086505407 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effd815dd50 0x7effbc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:27.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6000640 1 -- 172.21.15.161:0/1086505407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd815d8a0 con 0x7effbc003980 2024-05-22T21:19:27.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6000640 1 -- 172.21.15.161:0/1086505407 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd806de40 con 0x7effbc0047d0 2024-05-22T21:19:27.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6000640 1 -- 172.21.15.161:0/1086505407 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effbc00dba0 con 0x7effd815dd50 2024-05-22T21:19:27.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6a00640 1 --2- 172.21.15.161:0/1086505407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effbc003980 0x7effbc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:27.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effce000640 1 --2- 172.21.15.161:0/1086505407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effbc0047d0 0x7effbc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:27.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd7400640 1 --2- 172.21.15.161:0/1086505407 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effd815dd50 0x7effbc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:27.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6a00640 1 -- 172.21.15.161:0/1086505407 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effd815dd50 msgr2=0x7effbc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:27.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6a00640 1 --2- 172.21.15.161:0/1086505407 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effd815dd50 0x7effbc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:27.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6a00640 1 -- 172.21.15.161:0/1086505407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effbc0047d0 msgr2=0x7effbc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:27.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6a00640 1 --2- 172.21.15.161:0/1086505407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effbc0047d0 0x7effbc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:27.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.683+0000 7effd6a00640 1 -- 172.21.15.161:0/1086505407 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc003530 con 0x7effbc003980 2024-05-22T21:19:27.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.687+0000 7effd6a00640 1 --2- 172.21.15.161:0/1086505407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effbc003980 0x7effbc003da0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7effc8009b10 tx=0x7effc801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c0aba9e18afbb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:27.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.687+0000 7effd6000640 1 -- 172.21.15.161:0/1086505407 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc80a5530 con 0x7effbc003980 2024-05-22T21:19:27.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.687+0000 7effd6000640 1 -- 172.21.15.161:0/1086505407 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc80a5690 con 0x7effbc003980 2024-05-22T21:19:27.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.687+0000 7effd6000640 1 -- 172.21.15.161:0/1086505407 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc80a5a40 con 0x7effbc003980 2024-05-22T21:19:27.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.687+0000 7effddc00640 1 -- 172.21.15.161:0/1086505407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effbc003980 msgr2=0x7effbc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:27.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.687+0000 7effddc00640 1 --2- 172.21.15.161:0/1086505407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effbc003980 0x7effbc003da0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7effc8009b10 tx=0x7effc801ca40 comp rx=0 tx=0).stop 2024-05-22T21:19:27.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.687+0000 7effd7400640 1 -- 172.21.15.161:0/1086505407 reap_dead start 2024-05-22T21:19:27.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.687+0000 7effddc00640 1 -- 172.21.15.161:0/1086505407 shutdown_connections 2024-05-22T21:19:27.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.687+0000 7effddc00640 1 -- 172.21.15.161:0/1086505407 >> 172.21.15.161:0/1086505407 conn(0x7effd80b0bc0 msgr2=0x7effd80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:27.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.687+0000 7effddc00640 1 -- 172.21.15.161:0/1086505407 shutdown_connections 2024-05-22T21:19:27.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.687+0000 7effddc00640 1 -- 172.21.15.161:0/1086505407 wait complete. 2024-05-22T21:19:27.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effddc00640 1 Processor -- start 2024-05-22T21:19:27.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effddc00640 1 -- start start 2024-05-22T21:19:27.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effddc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effbc003980 0x7effd8157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:27.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effddc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd8156e10 0x7effd8145890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:27.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effddc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effd80ccce0 0x7effd80cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:27.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effddc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd80cd6a0 con 0x7effd8156e10 2024-05-22T21:19:27.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effddc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd81462f0 con 0x7effd80ccce0 2024-05-22T21:19:27.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effddc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd81591d0 con 0x7effbc003980 2024-05-22T21:19:27.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effce000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd8156e10 0x7effd8145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:27.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effce000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd8156e10 0x7effd8145890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57564/0 (socket says 172.21.15.161:57564) 2024-05-22T21:19:27.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effce000640 1 -- 172.21.15.161:0/3412885767 learned_addr learned my addr 172.21.15.161:0/3412885767 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:27.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effd6a00640 1 --2- 172.21.15.161:0/3412885767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effbc003980 0x7effd8157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:27.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effd7400640 1 --2- 172.21.15.161:0/3412885767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effd80ccce0 0x7effd80cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:27.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effce000640 1 -- 172.21.15.161:0/3412885767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effbc003980 msgr2=0x7effd8157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:27.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effce000640 1 --2- 172.21.15.161:0/3412885767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effbc003980 0x7effd8157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:27.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effce000640 1 -- 172.21.15.161:0/3412885767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effd80ccce0 msgr2=0x7effd80cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:27.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effce000640 1 --2- 172.21.15.161:0/3412885767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effd80ccce0 0x7effd80cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:27.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effce000640 1 -- 172.21.15.161:0/3412885767 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd8157cb0 con 0x7effd8156e10 2024-05-22T21:19:27.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effd7400640 1 --2- 172.21.15.161:0/3412885767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effd80ccce0 0x7effd80cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:27.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effce000640 1 --2- 172.21.15.161:0/3412885767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd8156e10 0x7effd8145890 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7effc0011690 tx=0x7effc0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:27.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effcf400640 1 -- 172.21.15.161:0/3412885767 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc000f9c0 con 0x7effd8156e10 2024-05-22T21:19:27.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effcf400640 1 -- 172.21.15.161:0/3412885767 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc000fb20 con 0x7effd8156e10 2024-05-22T21:19:27.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effcf400640 1 -- 172.21.15.161:0/3412885767 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc000fe90 con 0x7effd8156e10 2024-05-22T21:19:27.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effddc00640 1 -- 172.21.15.161:0/3412885767 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd8145dd0 con 0x7effd8156e10 2024-05-22T21:19:27.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effddc00640 1 -- 172.21.15.161:0/3412885767 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd8159400 con 0x7effd8156e10 2024-05-22T21:19:27.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.691+0000 7effddc00640 1 -- 172.21.15.161:0/3412885767 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd8067220 con 0x7effd8156e10 2024-05-22T21:19:27.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.695+0000 7effcf400640 1 -- 172.21.15.161:0/3412885767 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7effc0015460 con 0x7effd8156e10 2024-05-22T21:19:27.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.695+0000 7effcf400640 1 --2- 172.21.15.161:0/3412885767 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7effb4066690 0x7effb4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:27.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.695+0000 7effcf400640 1 -- 172.21.15.161:0/3412885767 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effc00846d0 con 0x7effd8156e10 2024-05-22T21:19:27.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.695+0000 7effd6a00640 1 --2- 172.21.15.161:0/3412885767 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7effb4066690 0x7effb4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:27.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.695+0000 7effd6a00640 1 --2- 172.21.15.161:0/3412885767 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7effb4066690 0x7effb4068b50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7effc80af000 tx=0x7effc80046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:27.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.699+0000 7effcf400640 1 -- 172.21.15.161:0/3412885767 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7effc0051910 con 0x7effd8156e10 2024-05-22T21:19:27.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.851+0000 7effddc00640 1 -- 172.21.15.161:0/3412885767 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7effd806c310 con 0x7effd8156e10 2024-05-22T21:19:27.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.851+0000 7effcf400640 1 -- 172.21.15.161:0/3412885767 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16107 (secure 0 0 0) 0x7effc0054f30 con 0x7effd8156e10 2024-05-22T21:19:27.855 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:27.855 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":128,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:19:25.399186+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:12.848545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:13.344629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:27.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.855+0000 7effddc00640 1 -- 172.21.15.161:0/3412885767 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7effb4066690 msgr2=0x7effb4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:27.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.855+0000 7effddc00640 1 --2- 172.21.15.161:0/3412885767 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7effb4066690 0x7effb4068b50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7effc80af000 tx=0x7effc80046d0 comp rx=0 tx=0).stop 2024-05-22T21:19:27.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.855+0000 7effddc00640 1 -- 172.21.15.161:0/3412885767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd8156e10 msgr2=0x7effd8145890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:27.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.855+0000 7effddc00640 1 --2- 172.21.15.161:0/3412885767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd8156e10 0x7effd8145890 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7effc0011690 tx=0x7effc0011b50 comp rx=0 tx=0).stop 2024-05-22T21:19:27.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.855+0000 7effddc00640 1 -- 172.21.15.161:0/3412885767 shutdown_connections 2024-05-22T21:19:27.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.855+0000 7effddc00640 1 --2- 172.21.15.161:0/3412885767 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7effb4066690 0x7effb4068b50 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:27.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.855+0000 7effddc00640 1 --2- 172.21.15.161:0/3412885767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7effd80ccce0 0x7effd80cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:27.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.855+0000 7effddc00640 1 --2- 172.21.15.161:0/3412885767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd8156e10 0x7effd8145890 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:27.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.855+0000 7effddc00640 1 --2- 172.21.15.161:0/3412885767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7effbc003980 0x7effd8157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:27.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.855+0000 7effddc00640 1 -- 172.21.15.161:0/3412885767 >> 172.21.15.161:0/3412885767 conn(0x7effd80b0bc0 msgr2=0x7effd80aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:27.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.855+0000 7effddc00640 1 -- 172.21.15.161:0/3412885767 shutdown_connections 2024-05-22T21:19:27.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:27.855+0000 7effddc00640 1 -- 172.21.15.161:0/3412885767 wait complete. 2024-05-22T21:19:27.877 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:19:12.848545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:19:13.344629+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:19:27.877 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:19:28.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.011+0000 7f6be9c00640 1 Processor -- start 2024-05-22T21:19:28.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.011+0000 7f6be9c00640 1 -- start start 2024-05-22T21:19:28.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.015+0000 7f6be9c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6be40b1ae0 0x7f6be40cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:28.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.015+0000 7f6be9c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be40b27f0 con 0x7f6be40cdd10 2024-05-22T21:19:28.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.015+0000 7f6be9c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be40b3660 con 0x7f6be406bd60 2024-05-22T21:19:28.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.015+0000 7f6be9c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be40d0010 con 0x7f6be40b1ae0 2024-05-22T21:19:28.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.015+0000 7f6be3e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f6be40cdd10 0x7f6be406b650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47336/0 (socket says 172.21.15.161:47336) 2024-05-22T21:19:28.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.015+0000 7f6be3e00640 1 -- 172.21.15.161:0/3294838006 learned_addr learned my addr 172.21.15.161:0/3294838006 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:28.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.015+0000 7f6bdb400640 1 --2- 172.21.15.161:0/3294838006 >> v2:172.21.15.179:3300/0 conn(0x7f6be40b1ae0 0x7f6be40cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:28.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.015+0000 7f6be3400640 1 -- 172.21.15.161:0/3294838006 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890323501 0 0) 0x7f6be40b27f0 con 0x7f6be40cdd10 2024-05-22T21:19:28.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.015+0000 7f6be3400640 1 -- 172.21.15.161:0/3294838006 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bc8003530 con 0x7f6be40cdd10 2024-05-22T21:19:28.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.015+0000 7f6bdb400640 1 -- 172.21.15.161:0/3294838006 >> v1:172.21.15.179:6789/0 conn(0x7f6be406bd60 legacy=0x7f6be4161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.015+0000 7f6bdb400640 1 -- 172.21.15.161:0/3294838006 >> v1:172.21.15.161:6789/0 conn(0x7f6be40cdd10 legacy=0x7f6be406b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.015+0000 7f6bdb400640 1 -- 172.21.15.161:0/3294838006 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be40b2230 con 0x7f6be40b1ae0 2024-05-22T21:19:28.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6bdb400640 1 --2- 172.21.15.161:0/3294838006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6be40b1ae0 0x7f6be40cfad0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0097c0 tx=0x7f6bcc01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3293bab0377191b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:28.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be3400640 1 -- 172.21.15.161:0/3294838006 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0ab600 con 0x7f6be40b1ae0 2024-05-22T21:19:28.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be3400640 1 -- 172.21.15.161:0/3294838006 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc0a7050 con 0x7f6be40b1ae0 2024-05-22T21:19:28.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be3400640 1 -- 172.21.15.161:0/3294838006 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0ab980 con 0x7f6be40b1ae0 2024-05-22T21:19:28.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be9c00640 1 -- 172.21.15.161:0/3294838006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6be40b1ae0 msgr2=0x7f6be40cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be9c00640 1 --2- 172.21.15.161:0/3294838006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6be40b1ae0 0x7f6be40cfad0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0097c0 tx=0x7f6bcc01c410 comp rx=0 tx=0).stop 2024-05-22T21:19:28.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be9c00640 1 -- 172.21.15.161:0/3294838006 shutdown_connections 2024-05-22T21:19:28.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be9c00640 1 --2- 172.21.15.161:0/3294838006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6be40b1ae0 0x7f6be40cfad0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be9c00640 1 -- 172.21.15.161:0/3294838006 >> 172.21.15.161:0/3294838006 conn(0x7f6be40b29a0 msgr2=0x7f6be40b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:28.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be9c00640 1 -- 172.21.15.161:0/3294838006 shutdown_connections 2024-05-22T21:19:28.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be9c00640 1 -- 172.21.15.161:0/3294838006 wait complete. 2024-05-22T21:19:28.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be9c00640 1 Processor -- start 2024-05-22T21:19:28.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be9c00640 1 -- start start 2024-05-22T21:19:28.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be9c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be406bd60 0x7f6be40c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:28.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be9c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6be40b1ae0 0x7f6be41583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:28.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be9c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6be40cdd10 0x7f6be4158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:28.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be9c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be415db40 con 0x7f6be406bd60 2024-05-22T21:19:28.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be9c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be4158e50 con 0x7f6be40cdd10 2024-05-22T21:19:28.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.019+0000 7f6be9c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be415b790 con 0x7f6be40b1ae0 2024-05-22T21:19:28.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.023+0000 7f6be3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be406bd60 0x7f6be40c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:28.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.023+0000 7f6be3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be406bd60 0x7f6be40c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57576/0 (socket says 172.21.15.161:57576) 2024-05-22T21:19:28.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.023+0000 7f6be3e00640 1 -- 172.21.15.161:0/4112650959 learned_addr learned my addr 172.21.15.161:0/4112650959 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:28.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.023+0000 7f6be3e00640 1 -- 172.21.15.161:0/4112650959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6be40b1ae0 msgr2=0x7f6be41583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.023+0000 7f6be3e00640 1 --2- 172.21.15.161:0/4112650959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6be40b1ae0 0x7f6be41583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.023+0000 7f6be3e00640 1 -- 172.21.15.161:0/4112650959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6be40cdd10 msgr2=0x7f6be4158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.023+0000 7f6be3e00640 1 --2- 172.21.15.161:0/4112650959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6be40cdd10 0x7f6be4158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.023+0000 7f6be3e00640 1 -- 172.21.15.161:0/4112650959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc009420 con 0x7f6be406bd60 2024-05-22T21:19:28.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.023+0000 7f6be3e00640 1 --2- 172.21.15.161:0/4112650959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be406bd60 0x7f6be40c6ca0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400cbc0 tx=0x7f6bd400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:28.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.023+0000 7f6be0c00640 1 -- 172.21.15.161:0/4112650959 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd400ad40 con 0x7f6be406bd60 2024-05-22T21:19:28.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.023+0000 7f6be0c00640 1 -- 172.21.15.161:0/4112650959 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd4012400 con 0x7f6be406bd60 2024-05-22T21:19:28.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.023+0000 7f6be0c00640 1 -- 172.21.15.161:0/4112650959 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd400aea0 con 0x7f6be406bd60 2024-05-22T21:19:28.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.023+0000 7f6be9c00640 1 -- 172.21.15.161:0/4112650959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be40b4a50 con 0x7f6be406bd60 2024-05-22T21:19:28.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.023+0000 7f6be9c00640 1 -- 172.21.15.161:0/4112650959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be40b5010 con 0x7f6be406bd60 2024-05-22T21:19:28.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.023+0000 7f6be9c00640 1 -- 172.21.15.161:0/4112650959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be4067070 con 0x7f6be406bd60 2024-05-22T21:19:28.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.023+0000 7f6be0c00640 1 -- 172.21.15.161:0/4112650959 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6bd4007070 con 0x7f6be406bd60 2024-05-22T21:19:28.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.027+0000 7f6be0c00640 1 --2- 172.21.15.161:0/4112650959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6bc00666c0 0x7f6bc0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:28.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.027+0000 7f6be0c00640 1 -- 172.21.15.161:0/4112650959 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bd4084210 con 0x7f6be406bd60 2024-05-22T21:19:28.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.027+0000 7f6bdb400640 1 --2- 172.21.15.161:0/4112650959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6bc00666c0 0x7f6bc0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:28.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.027+0000 7f6bdb400640 1 --2- 172.21.15.161:0/4112650959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6bc00666c0 0x7f6bc0068b80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0ad000 tx=0x7f6bcc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:28.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.031+0000 7f6be0c00640 1 -- 172.21.15.161:0/4112650959 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6bd40514f0 con 0x7f6be406bd60 2024-05-22T21:19:28.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.183+0000 7f6be9c00640 1 -- 172.21.15.161:0/4112650959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6be40b6dd0 con 0x7f6be406bd60 2024-05-22T21:19:28.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.183+0000 7f6be0c00640 1 -- 172.21.15.161:0/4112650959 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6bd4054b10 con 0x7f6be406bd60 2024-05-22T21:19:28.187 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:19:28.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.187+0000 7f6be9c00640 1 -- 172.21.15.161:0/4112650959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6bc00666c0 msgr2=0x7f6bc0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.187+0000 7f6be9c00640 1 --2- 172.21.15.161:0/4112650959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6bc00666c0 0x7f6bc0068b80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0ad000 tx=0x7f6bcc008040 comp rx=0 tx=0).stop 2024-05-22T21:19:28.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.187+0000 7f6be9c00640 1 -- 172.21.15.161:0/4112650959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be406bd60 msgr2=0x7f6be40c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.187+0000 7f6be9c00640 1 --2- 172.21.15.161:0/4112650959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be406bd60 0x7f6be40c6ca0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400cbc0 tx=0x7f6bd400a430 comp rx=0 tx=0).stop 2024-05-22T21:19:28.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.187+0000 7f6be9c00640 1 -- 172.21.15.161:0/4112650959 shutdown_connections 2024-05-22T21:19:28.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.187+0000 7f6be9c00640 1 --2- 172.21.15.161:0/4112650959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6bc00666c0 0x7f6bc0068b80 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.187+0000 7f6be9c00640 1 --2- 172.21.15.161:0/4112650959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6be40cdd10 0x7f6be4158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.187+0000 7f6be9c00640 1 --2- 172.21.15.161:0/4112650959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6be40b1ae0 0x7f6be41583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.187+0000 7f6be9c00640 1 --2- 172.21.15.161:0/4112650959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be406bd60 0x7f6be40c6ca0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.187+0000 7f6be9c00640 1 -- 172.21.15.161:0/4112650959 >> 172.21.15.161:0/4112650959 conn(0x7f6be40b29a0 msgr2=0x7f6be40b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:28.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.187+0000 7f6be9c00640 1 -- 172.21.15.161:0/4112650959 shutdown_connections 2024-05-22T21:19:28.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.187+0000 7f6be9c00640 1 -- 172.21.15.161:0/4112650959 wait complete. 2024-05-22T21:19:28.208 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:19:28.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.343+0000 7f73d8000640 1 Processor -- start 2024-05-22T21:19:28.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.343+0000 7f73d8000640 1 -- start start 2024-05-22T21:19:28.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.343+0000 7f73d8000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f73d0143c40 0x7f73d015b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:28.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.347+0000 7f73d8000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f73d01444a0 0x7f73d006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:28.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.347+0000 7f73d8000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d015efb0 con 0x7f73d0144900 2024-05-22T21:19:28.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.347+0000 7f73d8000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d015f290 con 0x7f73d01444a0 2024-05-22T21:19:28.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.347+0000 7f73d8000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d015f570 con 0x7f73d0143c40 2024-05-22T21:19:28.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.347+0000 7f73d4c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f73d01444a0 0x7f73d006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:28.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.347+0000 7f73d4c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f73d01444a0 0x7f73d006c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57580/0 (socket says 172.21.15.161:57580) 2024-05-22T21:19:28.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.347+0000 7f73d4c00640 1 -- 172.21.15.161:0/591675557 learned_addr learned my addr 172.21.15.161:0/591675557 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:28.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.347+0000 7f73d5600640 1 --2- 172.21.15.161:0/591675557 >> v2:172.21.15.179:3300/0 conn(0x7f73d0143c40 0x7f73d015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:28.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.347+0000 7f73d4c00640 1 -- 172.21.15.161:0/591675557 >> v2:172.21.15.179:3300/0 conn(0x7f73d0143c40 msgr2=0x7f73d015b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:28.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.347+0000 7f73d4c00640 1 --2- 172.21.15.161:0/591675557 >> v2:172.21.15.179:3300/0 conn(0x7f73d0143c40 0x7f73d015b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.347+0000 7f73d4c00640 1 -- 172.21.15.161:0/591675557 >> v1:172.21.15.179:6789/0 conn(0x7f73d0144900 legacy=0x7f73d00b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.347+0000 7f73d4c00640 1 -- 172.21.15.161:0/591675557 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d015f850 con 0x7f73d01444a0 2024-05-22T21:19:28.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.347+0000 7f73d5600640 1 --2- 172.21.15.161:0/591675557 >> v2:172.21.15.179:3300/0 conn(0x7f73d0143c40 0x7f73d015b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:28.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.347+0000 7f73d4c00640 1 --2- 172.21.15.161:0/591675557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d01444a0 0x7f73d006c2a0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f73d00724b0 tx=0x7f73bc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea495fa1455fec7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:28.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.347+0000 7f73cfe00640 1 -- 172.21.15.161:0/591675557 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc0a7070 con 0x7f73d01444a0 2024-05-22T21:19:28.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73cfe00640 1 -- 172.21.15.161:0/591675557 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73bc0ab430 con 0x7f73d01444a0 2024-05-22T21:19:28.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73cfe00640 1 -- 172.21.15.161:0/591675557 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc0ab7c0 con 0x7f73d01444a0 2024-05-22T21:19:28.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d8000640 1 -- 172.21.15.161:0/591675557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d01444a0 msgr2=0x7f73d006c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d8000640 1 --2- 172.21.15.161:0/591675557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d01444a0 0x7f73d006c2a0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f73d00724b0 tx=0x7f73bc01c520 comp rx=0 tx=0).stop 2024-05-22T21:19:28.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d8000640 1 -- 172.21.15.161:0/591675557 shutdown_connections 2024-05-22T21:19:28.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d8000640 1 --2- 172.21.15.161:0/591675557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d01444a0 0x7f73d006c2a0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d8000640 1 --2- 172.21.15.161:0/591675557 >> v2:172.21.15.179:3300/0 conn(0x7f73d0143c40 0x7f73d015b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d8000640 1 -- 172.21.15.161:0/591675557 >> 172.21.15.161:0/591675557 conn(0x7f73d006a2d0 msgr2=0x7f73d006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:28.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d8000640 1 -- 172.21.15.161:0/591675557 shutdown_connections 2024-05-22T21:19:28.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d8000640 1 -- 172.21.15.161:0/591675557 wait complete. 2024-05-22T21:19:28.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d8000640 1 Processor -- start 2024-05-22T21:19:28.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d8000640 1 -- start start 2024-05-22T21:19:28.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d8000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d0143c40 0x7f73d015aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:28.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d8000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73d01444a0 0x7f73d0153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:28.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d8000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73d0144900 0x7f73d01337c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:28.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d8000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d01540a0 con 0x7f73d0143c40 2024-05-22T21:19:28.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d8000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d0155590 con 0x7f73d01444a0 2024-05-22T21:19:28.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d8000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d01557e0 con 0x7f73d0144900 2024-05-22T21:19:28.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d5600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d0143c40 0x7f73d015aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:28.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d5600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d0143c40 0x7f73d015aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57582/0 (socket says 172.21.15.161:57582) 2024-05-22T21:19:28.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d5600640 1 -- 172.21.15.161:0/2847257366 learned_addr learned my addr 172.21.15.161:0/2847257366 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:28.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d4c00640 1 --2- 172.21.15.161:0/2847257366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73d01444a0 0x7f73d0153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:28.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d6000640 1 --2- 172.21.15.161:0/2847257366 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73d0144900 0x7f73d01337c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:28.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d5600640 1 -- 172.21.15.161:0/2847257366 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73d0144900 msgr2=0x7f73d01337c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d5600640 1 --2- 172.21.15.161:0/2847257366 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73d0144900 0x7f73d01337c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d5600640 1 -- 172.21.15.161:0/2847257366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73d01444a0 msgr2=0x7f73d0153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d5600640 1 --2- 172.21.15.161:0/2847257366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73d01444a0 0x7f73d0153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d5600640 1 -- 172.21.15.161:0/2847257366 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc0094d0 con 0x7f73d0143c40 2024-05-22T21:19:28.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.351+0000 7f73d4c00640 1 --2- 172.21.15.161:0/2847257366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73d01444a0 0x7f73d0153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:28.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.355+0000 7f73d5600640 1 --2- 172.21.15.161:0/2847257366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d0143c40 0x7f73d015aa50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f73c400e810 tx=0x7f73c400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:28.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.355+0000 7f73cd600640 1 -- 172.21.15.161:0/2847257366 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c400c8d0 con 0x7f73d0143c40 2024-05-22T21:19:28.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.355+0000 7f73cd600640 1 -- 172.21.15.161:0/2847257366 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c400ca30 con 0x7f73d0143c40 2024-05-22T21:19:28.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.355+0000 7f73cd600640 1 -- 172.21.15.161:0/2847257366 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c400cdc0 con 0x7f73d0143c40 2024-05-22T21:19:28.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.355+0000 7f73d8000640 1 -- 172.21.15.161:0/2847257366 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d0155ac0 con 0x7f73d0143c40 2024-05-22T21:19:28.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.355+0000 7f73d8000640 1 -- 172.21.15.161:0/2847257366 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d0156080 con 0x7f73d0143c40 2024-05-22T21:19:28.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.355+0000 7f73d8000640 1 -- 172.21.15.161:0/2847257366 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d00cc4a0 con 0x7f73d0143c40 2024-05-22T21:19:28.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.355+0000 7f73cd600640 1 -- 172.21.15.161:0/2847257366 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f73c4013410 con 0x7f73d0143c40 2024-05-22T21:19:28.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.355+0000 7f73cd600640 1 --2- 172.21.15.161:0/2847257366 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73b0066690 0x7f73b0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:28.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.355+0000 7f73cd600640 1 -- 172.21.15.161:0/2847257366 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73c40840a0 con 0x7f73d0143c40 2024-05-22T21:19:28.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.355+0000 7f73d4c00640 1 --2- 172.21.15.161:0/2847257366 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73b0066690 0x7f73b0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:28.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.359+0000 7f73d4c00640 1 --2- 172.21.15.161:0/2847257366 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73b0066690 0x7f73b0068b50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0af000 tx=0x7f73bc0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:28.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.359+0000 7f73cd600640 1 -- 172.21.15.161:0/2847257366 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f73c4051370 con 0x7f73d0143c40 2024-05-22T21:19:28.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.515+0000 7f73d8000640 1 -- 172.21.15.161:0/2847257366 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f73d0067220 con 0x7f73d0143c40 2024-05-22T21:19:28.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.515+0000 7f73cd600640 1 -- 172.21.15.161:0/2847257366 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) v1 ==== 104+0+11 (secure 0 0 0) 0x7f73c4009480 con 0x7f73d0143c40 2024-05-22T21:19:28.518 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:19:28.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.519+0000 7f73d8000640 1 -- 172.21.15.161:0/2847257366 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73b0066690 msgr2=0x7f73b0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.519+0000 7f73d8000640 1 --2- 172.21.15.161:0/2847257366 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73b0066690 0x7f73b0068b50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0af000 tx=0x7f73bc0a7000 comp rx=0 tx=0).stop 2024-05-22T21:19:28.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.519+0000 7f73d8000640 1 -- 172.21.15.161:0/2847257366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d0143c40 msgr2=0x7f73d015aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.519+0000 7f73d8000640 1 --2- 172.21.15.161:0/2847257366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d0143c40 0x7f73d015aa50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f73c400e810 tx=0x7f73c400ecd0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.519+0000 7f73d8000640 1 -- 172.21.15.161:0/2847257366 shutdown_connections 2024-05-22T21:19:28.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.519+0000 7f73d8000640 1 --2- 172.21.15.161:0/2847257366 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f73b0066690 0x7f73b0068b50 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.519+0000 7f73d8000640 1 --2- 172.21.15.161:0/2847257366 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73d0144900 0x7f73d01337c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.519+0000 7f73d8000640 1 --2- 172.21.15.161:0/2847257366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73d01444a0 0x7f73d0153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.519+0000 7f73d8000640 1 --2- 172.21.15.161:0/2847257366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d0143c40 0x7f73d015aa50 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.519+0000 7f73d8000640 1 -- 172.21.15.161:0/2847257366 >> 172.21.15.161:0/2847257366 conn(0x7f73d006a2d0 msgr2=0x7f73d00cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:28.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.519+0000 7f73d8000640 1 -- 172.21.15.161:0/2847257366 shutdown_connections 2024-05-22T21:19:28.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.519+0000 7f73d8000640 1 -- 172.21.15.161:0/2847257366 wait complete. 2024-05-22T21:19:28.540 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:19:28.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.675+0000 7f712ea00640 1 Processor -- start 2024-05-22T21:19:28.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.675+0000 7f712ea00640 1 -- start start 2024-05-22T21:19:28.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.679+0000 7f712ea00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f712813cfc0 0x7f71281402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:28.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.679+0000 7f712ea00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712813d3a0 con 0x7f712813fe50 2024-05-22T21:19:28.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.679+0000 7f712ea00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71280cd920 con 0x7f712815af60 2024-05-22T21:19:28.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.679+0000 7f712ea00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71280cdc00 con 0x7f712813cfc0 2024-05-22T21:19:28.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.679+0000 7f7127e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f712813cfc0 0x7f71281402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:28.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.679+0000 7f712ca00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f712813fe50 0x7f712806fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47352/0 (socket says 172.21.15.161:47352) 2024-05-22T21:19:28.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.679+0000 7f712ca00640 1 -- 172.21.15.161:0/2385678715 learned_addr learned my addr 172.21.15.161:0/2385678715 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:28.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.679+0000 7f7127e00640 1 -- 172.21.15.161:0/2385678715 >> v1:172.21.15.179:6789/0 conn(0x7f712815af60 legacy=0x7f712813f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.679+0000 7f7127e00640 1 -- 172.21.15.161:0/2385678715 >> v1:172.21.15.161:6789/0 conn(0x7f712813fe50 legacy=0x7f712806fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.679+0000 7f7127e00640 1 -- 172.21.15.161:0/2385678715 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71280cdee0 con 0x7f712813cfc0 2024-05-22T21:19:28.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f7127e00640 1 --2- 172.21.15.161:0/2385678715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f712813cfc0 0x7f71281402e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f71180097c0 tx=0x7f711801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a14ad13de9e37c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:28.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f7127400640 1 -- 172.21.15.161:0/2385678715 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71180ac440 con 0x7f712813cfc0 2024-05-22T21:19:28.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f7127400640 1 -- 172.21.15.161:0/2385678715 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71180ac5a0 con 0x7f712813cfc0 2024-05-22T21:19:28.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f7127400640 1 -- 172.21.15.161:0/2385678715 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71180ac930 con 0x7f712813cfc0 2024-05-22T21:19:28.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f712ea00640 1 -- 172.21.15.161:0/2385678715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f712813cfc0 msgr2=0x7f71281402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f712ea00640 1 --2- 172.21.15.161:0/2385678715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f712813cfc0 0x7f71281402e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f71180097c0 tx=0x7f711801c3e0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f712ea00640 1 -- 172.21.15.161:0/2385678715 shutdown_connections 2024-05-22T21:19:28.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f712ea00640 1 --2- 172.21.15.161:0/2385678715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f712813cfc0 0x7f71281402e0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f712ea00640 1 -- 172.21.15.161:0/2385678715 >> 172.21.15.161:0/2385678715 conn(0x7f712806a2d0 msgr2=0x7f712806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:28.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f712ea00640 1 -- 172.21.15.161:0/2385678715 shutdown_connections 2024-05-22T21:19:28.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f712ea00640 1 -- 172.21.15.161:0/2385678715 wait complete. 2024-05-22T21:19:28.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f712ea00640 1 Processor -- start 2024-05-22T21:19:28.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f712ea00640 1 -- start start 2024-05-22T21:19:28.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f712ea00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f712813cfc0 0x7f712813ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:28.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f712ea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f712813fe50 0x7f7128157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:28.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f712ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f712815af60 0x7f71280b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:28.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f712ea00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712813e390 con 0x7f712815af60 2024-05-22T21:19:28.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f712ea00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7128141ac0 con 0x7f712813fe50 2024-05-22T21:19:28.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.683+0000 7f712ea00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7128157dd0 con 0x7f712813cfc0 2024-05-22T21:19:28.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f712ca00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f712815af60 0x7f71280b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:28.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f712ca00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f712815af60 0x7f71280b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57598/0 (socket says 172.21.15.161:57598) 2024-05-22T21:19:28.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f7127e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f712813cfc0 0x7f712813ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:28.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f712ca00640 1 -- 172.21.15.161:0/1695973719 learned_addr learned my addr 172.21.15.161:0/1695973719 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:28.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f711f400640 1 --2- 172.21.15.161:0/1695973719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f712813fe50 0x7f7128157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:28.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f712ca00640 1 -- 172.21.15.161:0/1695973719 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f712813cfc0 msgr2=0x7f712813ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f712ca00640 1 --2- 172.21.15.161:0/1695973719 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f712813cfc0 0x7f712813ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f712ca00640 1 -- 172.21.15.161:0/1695973719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f712813fe50 msgr2=0x7f7128157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f712ca00640 1 --2- 172.21.15.161:0/1695973719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f712813fe50 0x7f7128157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f712ca00640 1 -- 172.21.15.161:0/1695973719 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7118009420 con 0x7f712815af60 2024-05-22T21:19:28.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f712ca00640 1 --2- 172.21.15.161:0/1695973719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f712815af60 0x7f71280b3300 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f712000cbc0 tx=0x7f712000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:28.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f7124c00640 1 -- 172.21.15.161:0/1695973719 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712000ad40 con 0x7f712815af60 2024-05-22T21:19:28.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f7124c00640 1 -- 172.21.15.161:0/1695973719 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7120012400 con 0x7f712815af60 2024-05-22T21:19:28.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f7124c00640 1 -- 172.21.15.161:0/1695973719 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712000aea0 con 0x7f712815af60 2024-05-22T21:19:28.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f712ea00640 1 -- 172.21.15.161:0/1695973719 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7128141220 con 0x7f712815af60 2024-05-22T21:19:28.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f712ea00640 1 -- 172.21.15.161:0/1695973719 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71281417e0 con 0x7f712815af60 2024-05-22T21:19:28.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f712ea00640 1 -- 172.21.15.161:0/1695973719 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712813e500 con 0x7f712815af60 2024-05-22T21:19:28.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f7124c00640 1 -- 172.21.15.161:0/1695973719 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7120007070 con 0x7f712815af60 2024-05-22T21:19:28.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f7124c00640 1 --2- 172.21.15.161:0/1695973719 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7104066690 0x7f7104068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:28.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.687+0000 7f7124c00640 1 -- 172.21.15.161:0/1695973719 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7120083fe0 con 0x7f712815af60 2024-05-22T21:19:28.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.691+0000 7f7127e00640 1 --2- 172.21.15.161:0/1695973719 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7104066690 0x7f7104068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:28.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.691+0000 7f7127e00640 1 --2- 172.21.15.161:0/1695973719 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7104066690 0x7f7104068b50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f71180ae000 tx=0x7f71180a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:28.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.695+0000 7f7124c00640 1 -- 172.21.15.161:0/1695973719 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f71200513c0 con 0x7f712815af60 2024-05-22T21:19:28.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.847+0000 7f712ea00640 1 -- 172.21.15.161:0/1695973719 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7128067220 con 0x7f712815af60 2024-05-22T21:19:28.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.847+0000 7f7124c00640 1 -- 172.21.15.161:0/1695973719 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7120017020 con 0x7f712815af60 2024-05-22T21:19:28.851 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:19:28.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.851+0000 7f712ea00640 1 -- 172.21.15.161:0/1695973719 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7104066690 msgr2=0x7f7104068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.851+0000 7f712ea00640 1 --2- 172.21.15.161:0/1695973719 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7104066690 0x7f7104068b50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f71180ae000 tx=0x7f71180a7000 comp rx=0 tx=0).stop 2024-05-22T21:19:28.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.851+0000 7f712ea00640 1 -- 172.21.15.161:0/1695973719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f712815af60 msgr2=0x7f71280b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:28.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.851+0000 7f712ea00640 1 --2- 172.21.15.161:0/1695973719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f712815af60 0x7f71280b3300 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f712000cbc0 tx=0x7f712000a430 comp rx=0 tx=0).stop 2024-05-22T21:19:28.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.851+0000 7f712ea00640 1 -- 172.21.15.161:0/1695973719 shutdown_connections 2024-05-22T21:19:28.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.851+0000 7f712ea00640 1 --2- 172.21.15.161:0/1695973719 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7104066690 0x7f7104068b50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.851+0000 7f712ea00640 1 --2- 172.21.15.161:0/1695973719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f712815af60 0x7f71280b3300 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.851+0000 7f712ea00640 1 --2- 172.21.15.161:0/1695973719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f712813fe50 0x7f7128157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.851+0000 7f712ea00640 1 --2- 172.21.15.161:0/1695973719 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f712813cfc0 0x7f712813ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:28.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.851+0000 7f712ea00640 1 -- 172.21.15.161:0/1695973719 >> 172.21.15.161:0/1695973719 conn(0x7f712806a2d0 msgr2=0x7f712815da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:28.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.851+0000 7f712ea00640 1 -- 172.21.15.161:0/1695973719 shutdown_connections 2024-05-22T21:19:28.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:28.851+0000 7f712ea00640 1 -- 172.21.15.161:0/1695973719 wait complete. 2024-05-22T21:19:28.872 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:19:29.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.003+0000 7fd1ac800640 1 Processor -- start 2024-05-22T21:19:29.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.007+0000 7fd1ac800640 1 -- start start 2024-05-22T21:19:29.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.007+0000 7fd1ac800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd1a413cfc0 0x7fd1a41402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.007+0000 7fd1ac800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd1a415af60 0x7fd1a413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.007+0000 7fd1ac800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a4140090 con 0x7fd1a413fc80 2024-05-22T21:19:29.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.007+0000 7fd1ac800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a413d3a0 con 0x7fd1a415af60 2024-05-22T21:19:29.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.007+0000 7fd1ac800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a40cd920 con 0x7fd1a413cfc0 2024-05-22T21:19:29.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.007+0000 7fd1aa800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd1a413fc80 0x7fd1a415dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47366/0 (socket says 172.21.15.161:47366) 2024-05-22T21:19:29.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.007+0000 7fd1a9400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd1a415af60 0x7fd1a413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.007+0000 7fd1aa800640 1 -- 172.21.15.161:0/3696277423 learned_addr learned my addr 172.21.15.161:0/3696277423 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:29.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.007+0000 7fd1a9e00640 1 --2- 172.21.15.161:0/3696277423 >> v2:172.21.15.179:3300/0 conn(0x7fd1a413cfc0 0x7fd1a41402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.007+0000 7fd1a9400640 1 -- 172.21.15.161:0/3696277423 >> v2:172.21.15.179:3300/0 conn(0x7fd1a413cfc0 msgr2=0x7fd1a41402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:29.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.007+0000 7fd1a9400640 1 --2- 172.21.15.161:0/3696277423 >> v2:172.21.15.179:3300/0 conn(0x7fd1a413cfc0 0x7fd1a41402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.007+0000 7fd1a9400640 1 -- 172.21.15.161:0/3696277423 >> v1:172.21.15.161:6789/0 conn(0x7fd1a413fc80 legacy=0x7fd1a415dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.007+0000 7fd1a9400640 1 -- 172.21.15.161:0/3696277423 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a40cdc00 con 0x7fd1a415af60 2024-05-22T21:19:29.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.007+0000 7fd1a8a00640 1 -- 172.21.15.161:0/3696277423 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621743210 0 0) 0x7fd1a4140090 con 0x7fd1a413fc80 2024-05-22T21:19:29.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.011+0000 7fd1a9400640 1 --2- 172.21.15.161:0/3696277423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1a415af60 0x7fd1a413f740 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fd1a406de90 tx=0x7fd19401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffb272d23a8bfd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:29.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.011+0000 7fd1a8a00640 1 -- 172.21.15.161:0/3696277423 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1940ab600 con 0x7fd1a415af60 2024-05-22T21:19:29.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.011+0000 7fd1a8a00640 1 -- 172.21.15.161:0/3696277423 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1940a7050 con 0x7fd1a415af60 2024-05-22T21:19:29.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.011+0000 7fd1a8a00640 1 -- 172.21.15.161:0/3696277423 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1940ab980 con 0x7fd1a415af60 2024-05-22T21:19:29.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.011+0000 7fd1ac800640 1 -- 172.21.15.161:0/3696277423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1a415af60 msgr2=0x7fd1a413f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.011+0000 7fd1ac800640 1 --2- 172.21.15.161:0/3696277423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1a415af60 0x7fd1a413f740 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fd1a406de90 tx=0x7fd19401c3f0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.011+0000 7fd1ac800640 1 -- 172.21.15.161:0/3696277423 shutdown_connections 2024-05-22T21:19:29.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.011+0000 7fd1ac800640 1 --2- 172.21.15.161:0/3696277423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1a415af60 0x7fd1a413f740 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.011+0000 7fd1ac800640 1 --2- 172.21.15.161:0/3696277423 >> v2:172.21.15.179:3300/0 conn(0x7fd1a413cfc0 0x7fd1a41402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.011+0000 7fd1ac800640 1 -- 172.21.15.161:0/3696277423 >> 172.21.15.161:0/3696277423 conn(0x7fd1a406a2d0 msgr2=0x7fd1a406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:29.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.011+0000 7fd1ac800640 1 -- 172.21.15.161:0/3696277423 shutdown_connections 2024-05-22T21:19:29.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.011+0000 7fd1ac800640 1 -- 172.21.15.161:0/3696277423 wait complete. 2024-05-22T21:19:29.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.011+0000 7fd1ac800640 1 Processor -- start 2024-05-22T21:19:29.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.011+0000 7fd1ac800640 1 -- start start 2024-05-22T21:19:29.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1ac800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1a413cfc0 0x7fd1a4142320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1ac800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1a413fc80 0x7fd1a40acff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1ac800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1a415af60 0x7fd1a4156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1ac800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a41429b0 con 0x7fd1a413cfc0 2024-05-22T21:19:29.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1ac800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a413d990 con 0x7fd1a415af60 2024-05-22T21:19:29.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1ac800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a413dc30 con 0x7fd1a413fc80 2024-05-22T21:19:29.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1a9e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1a413cfc0 0x7fd1a4142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1a9e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1a413cfc0 0x7fd1a4142320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57620/0 (socket says 172.21.15.161:57620) 2024-05-22T21:19:29.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1a9e00640 1 -- 172.21.15.161:0/82585718 learned_addr learned my addr 172.21.15.161:0/82585718 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:29.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1a9400640 1 --2- 172.21.15.161:0/82585718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1a413fc80 0x7fd1a40acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1a9e00640 1 -- 172.21.15.161:0/82585718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1a413fc80 msgr2=0x7fd1a40acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1aa800640 1 --2- 172.21.15.161:0/82585718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1a415af60 0x7fd1a4156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1a9e00640 1 --2- 172.21.15.161:0/82585718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1a413fc80 0x7fd1a40acff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1a9e00640 1 -- 172.21.15.161:0/82585718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1a415af60 msgr2=0x7fd1a4156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1a9e00640 1 --2- 172.21.15.161:0/82585718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1a415af60 0x7fd1a4156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1a9e00640 1 -- 172.21.15.161:0/82585718 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd194009420 con 0x7fd1a413cfc0 2024-05-22T21:19:29.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1aa800640 1 --2- 172.21.15.161:0/82585718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1a415af60 0x7fd1a4156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:29.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1a9400640 1 --2- 172.21.15.161:0/82585718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1a413fc80 0x7fd1a40acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:29.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1a9e00640 1 --2- 172.21.15.161:0/82585718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1a413cfc0 0x7fd1a4142320 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd19800e870 tx=0x7fd19800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:29.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd19e000640 1 -- 172.21.15.161:0/82585718 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd198009530 con 0x7fd1a413cfc0 2024-05-22T21:19:29.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd19e000640 1 -- 172.21.15.161:0/82585718 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19800c940 con 0x7fd1a413cfc0 2024-05-22T21:19:29.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd19e000640 1 -- 172.21.15.161:0/82585718 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19800ccd0 con 0x7fd1a413cfc0 2024-05-22T21:19:29.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1ac800640 1 -- 172.21.15.161:0/82585718 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a4157a90 con 0x7fd1a413cfc0 2024-05-22T21:19:29.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1ac800640 1 -- 172.21.15.161:0/82585718 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a413e1b0 con 0x7fd1a413cfc0 2024-05-22T21:19:29.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd1ac800640 1 -- 172.21.15.161:0/82585718 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a41430b0 con 0x7fd1a413cfc0 2024-05-22T21:19:29.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.015+0000 7fd19e000640 1 -- 172.21.15.161:0/82585718 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd198013470 con 0x7fd1a413cfc0 2024-05-22T21:19:29.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.019+0000 7fd19e000640 1 --2- 172.21.15.161:0/82585718 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd188066690 0x7fd188068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.019+0000 7fd19e000640 1 -- 172.21.15.161:0/82585718 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd198084350 con 0x7fd1a413cfc0 2024-05-22T21:19:29.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.019+0000 7fd1a9400640 1 --2- 172.21.15.161:0/82585718 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd188066690 0x7fd188068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.019+0000 7fd1a9400640 1 --2- 172.21.15.161:0/82585718 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd188066690 0x7fd188068b50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd1940ad000 tx=0x7fd194008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:29.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.023+0000 7fd19e000640 1 -- 172.21.15.161:0/82585718 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd198051620 con 0x7fd1a413cfc0 2024-05-22T21:19:29.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.187+0000 7fd1ac800640 1 -- 172.21.15.161:0/82585718 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd1a4067220 con 0x7fd1a413cfc0 2024-05-22T21:19:29.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.187+0000 7fd19e000640 1 -- 172.21.15.161:0/82585718 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v128) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd198054c40 con 0x7fd1a413cfc0 2024-05-22T21:19:29.191 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:19:29.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.191+0000 7fd1ac800640 1 -- 172.21.15.161:0/82585718 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd188066690 msgr2=0x7fd188068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.191+0000 7fd1ac800640 1 --2- 172.21.15.161:0/82585718 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd188066690 0x7fd188068b50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd1940ad000 tx=0x7fd194008040 comp rx=0 tx=0).stop 2024-05-22T21:19:29.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.191+0000 7fd1ac800640 1 -- 172.21.15.161:0/82585718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1a413cfc0 msgr2=0x7fd1a4142320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.191+0000 7fd1ac800640 1 --2- 172.21.15.161:0/82585718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1a413cfc0 0x7fd1a4142320 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd19800e870 tx=0x7fd19800ed30 comp rx=0 tx=0).stop 2024-05-22T21:19:29.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.191+0000 7fd1ac800640 1 -- 172.21.15.161:0/82585718 shutdown_connections 2024-05-22T21:19:29.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.191+0000 7fd1ac800640 1 --2- 172.21.15.161:0/82585718 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd188066690 0x7fd188068b50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.191+0000 7fd1ac800640 1 --2- 172.21.15.161:0/82585718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1a415af60 0x7fd1a4156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.191+0000 7fd1ac800640 1 --2- 172.21.15.161:0/82585718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1a413fc80 0x7fd1a40acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.191+0000 7fd1ac800640 1 --2- 172.21.15.161:0/82585718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1a413cfc0 0x7fd1a4142320 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.191+0000 7fd1ac800640 1 -- 172.21.15.161:0/82585718 >> 172.21.15.161:0/82585718 conn(0x7fd1a406a2d0 msgr2=0x7fd1a40b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:29.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.191+0000 7fd1ac800640 1 -- 172.21.15.161:0/82585718 shutdown_connections 2024-05-22T21:19:29.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.191+0000 7fd1ac800640 1 -- 172.21.15.161:0/82585718 wait complete. 2024-05-22T21:19:29.212 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-22T21:19:29.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:29.213 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-22T21:19:29.262 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:19:29.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:29.262 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-22T21:19:29.316 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:19:29.316 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:29.325 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:19:29.325 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:29.380 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:29.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.515+0000 7ff1ce600640 1 Processor -- start 2024-05-22T21:19:29.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.515+0000 7ff1ce600640 1 -- start start 2024-05-22T21:19:29.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.515+0000 7ff1ce600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff1c815e130 0x7ff1c815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.515+0000 7ff1ce600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1c815d410 0x7ff1c815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.515+0000 7ff1ce600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c8071670 con 0x7ff1c815dd50 2024-05-22T21:19:29.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.515+0000 7ff1ce600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c815d870 con 0x7ff1c815e130 2024-05-22T21:19:29.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.515+0000 7ff1ce600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c8070a60 con 0x7ff1c815d410 2024-05-22T21:19:29.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.515+0000 7ff1c6a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff1c815e130 0x7ff1c815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.515+0000 7ff1c6a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff1c815e130 0x7ff1c815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57628/0 (socket says 172.21.15.161:57628) 2024-05-22T21:19:29.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.515+0000 7ff1c6a00640 1 -- 172.21.15.161:0/4098177869 learned_addr learned my addr 172.21.15.161:0/4098177869 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:29.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.515+0000 7ff1c7400640 1 --1- 172.21.15.161:0/4098177869 >> v1:172.21.15.161:6789/0 conn(0x7ff1c815dd50 0x7ff1c806d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47368/0 (socket says 172.21.15.161:47368) 2024-05-22T21:19:29.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.519+0000 7ff1c7e00640 1 --2- 172.21.15.161:0/4098177869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1c815d410 0x7ff1c815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.519+0000 7ff1c6a00640 1 -- 172.21.15.161:0/4098177869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1c815d410 msgr2=0x7ff1c815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:29.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.519+0000 7ff1c6a00640 1 --2- 172.21.15.161:0/4098177869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1c815d410 0x7ff1c815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.519+0000 7ff1c6a00640 1 -- 172.21.15.161:0/4098177869 >> v1:172.21.15.161:6789/0 conn(0x7ff1c815dd50 legacy=0x7ff1c806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.519+0000 7ff1c6a00640 1 -- 172.21.15.161:0/4098177869 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c8070d40 con 0x7ff1c815e130 2024-05-22T21:19:29.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.519+0000 7ff1c6a00640 1 --2- 172.21.15.161:0/4098177869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1c815e130 0x7ff1c815ced0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4009870 tx=0x7ff1b401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e10e257d4b9f6f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:29.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.519+0000 7ff1c6000640 1 -- 172.21.15.161:0/4098177869 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b40a95f0 con 0x7ff1c815e130 2024-05-22T21:19:29.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.519+0000 7ff1c6000640 1 -- 172.21.15.161:0/4098177869 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b40a9750 con 0x7ff1c815e130 2024-05-22T21:19:29.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.519+0000 7ff1c6000640 1 -- 172.21.15.161:0/4098177869 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b40a9ae0 con 0x7ff1c815e130 2024-05-22T21:19:29.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.519+0000 7ff1ce600640 1 -- 172.21.15.161:0/4098177869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1c815e130 msgr2=0x7ff1c815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.519+0000 7ff1ce600640 1 --2- 172.21.15.161:0/4098177869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1c815e130 0x7ff1c815ced0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4009870 tx=0x7ff1b401c530 comp rx=0 tx=0).stop 2024-05-22T21:19:29.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.519+0000 7ff1ce600640 1 -- 172.21.15.161:0/4098177869 shutdown_connections 2024-05-22T21:19:29.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.519+0000 7ff1ce600640 1 --2- 172.21.15.161:0/4098177869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1c815e130 0x7ff1c815ced0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.519+0000 7ff1ce600640 1 --2- 172.21.15.161:0/4098177869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1c815d410 0x7ff1c815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.519+0000 7ff1ce600640 1 -- 172.21.15.161:0/4098177869 >> 172.21.15.161:0/4098177869 conn(0x7ff1c80b0bc0 msgr2=0x7ff1c80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:29.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1ce600640 1 -- 172.21.15.161:0/4098177869 shutdown_connections 2024-05-22T21:19:29.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1ce600640 1 -- 172.21.15.161:0/4098177869 wait complete. 2024-05-22T21:19:29.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1ce600640 1 Processor -- start 2024-05-22T21:19:29.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1ce600640 1 -- start start 2024-05-22T21:19:29.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1ce600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1c815d410 0x7ff1c814d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1ce600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1c815dd50 0x7ff1c814ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1ce600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1c815e130 0x7ff1c8157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1ce600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c81594c0 con 0x7ff1c815dd50 2024-05-22T21:19:29.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1ce600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c80afe50 con 0x7ff1c815d410 2024-05-22T21:19:29.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1ce600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c80b0130 con 0x7ff1c815e130 2024-05-22T21:19:29.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1c6a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1c815dd50 0x7ff1c814ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1c6a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1c815dd50 0x7ff1c814ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57636/0 (socket says 172.21.15.161:57636) 2024-05-22T21:19:29.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1c6a00640 1 -- 172.21.15.161:0/3930713525 learned_addr learned my addr 172.21.15.161:0/3930713525 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:29.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1c7e00640 1 --2- 172.21.15.161:0/3930713525 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1c815e130 0x7ff1c8157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1c7400640 1 --2- 172.21.15.161:0/3930713525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1c815d410 0x7ff1c814d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1c6a00640 1 -- 172.21.15.161:0/3930713525 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1c815e130 msgr2=0x7ff1c8157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1c6a00640 1 --2- 172.21.15.161:0/3930713525 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1c815e130 0x7ff1c8157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1c6a00640 1 -- 172.21.15.161:0/3930713525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1c815d410 msgr2=0x7ff1c814d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1c6a00640 1 --2- 172.21.15.161:0/3930713525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1c815d410 0x7ff1c814d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1c6a00640 1 -- 172.21.15.161:0/3930713525 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b40094d0 con 0x7ff1c815dd50 2024-05-22T21:19:29.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.523+0000 7ff1c6a00640 1 --2- 172.21.15.161:0/3930713525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1c815dd50 0x7ff1c814ca00 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff1b401abe0 tx=0x7ff1b40192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:29.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.527+0000 7ff1bf400640 1 -- 172.21.15.161:0/3930713525 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b40bb640 con 0x7ff1c815dd50 2024-05-22T21:19:29.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.527+0000 7ff1bf400640 1 -- 172.21.15.161:0/3930713525 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b40bb7a0 con 0x7ff1c815dd50 2024-05-22T21:19:29.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.527+0000 7ff1bf400640 1 -- 172.21.15.161:0/3930713525 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b40bbb30 con 0x7ff1c815dd50 2024-05-22T21:19:29.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.527+0000 7ff1ce600640 1 -- 172.21.15.161:0/3930713525 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c8157370 con 0x7ff1c815dd50 2024-05-22T21:19:29.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.527+0000 7ff1ce600640 1 -- 172.21.15.161:0/3930713525 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c8159120 con 0x7ff1c815dd50 2024-05-22T21:19:29.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.527+0000 7ff1ce600640 1 -- 172.21.15.161:0/3930713525 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c8067220 con 0x7ff1c815dd50 2024-05-22T21:19:29.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.527+0000 7ff1bf400640 1 -- 172.21.15.161:0/3930713525 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff1b40b13f0 con 0x7ff1c815dd50 2024-05-22T21:19:29.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.527+0000 7ff1bf400640 1 --2- 172.21.15.161:0/3930713525 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff1a80666c0 0x7ff1a8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.527+0000 7ff1bf400640 1 -- 172.21.15.161:0/3930713525 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1b411e5d0 con 0x7ff1c815dd50 2024-05-22T21:19:29.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.527+0000 7ff1c7400640 1 --2- 172.21.15.161:0/3930713525 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff1a80666c0 0x7ff1a8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.527+0000 7ff1c7400640 1 --2- 172.21.15.161:0/3930713525 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff1a80666c0 0x7ff1a8068b80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80012f0 tx=0x7ff1b8001700 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:29.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.531+0000 7ff1bf400640 1 -- 172.21.15.161:0/3930713525 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff1b40eb9e0 con 0x7ff1c815dd50 2024-05-22T21:19:29.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.687+0000 7ff1ce600640 1 -- 172.21.15.161:0/3930713525 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1c8157820 con 0x7ff1c815dd50 2024-05-22T21:19:29.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.687+0000 7ff1bf400640 1 -- 172.21.15.161:0/3930713525 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16107 (secure 0 0 0) 0x7ff1b40ef000 con 0x7ff1c815dd50 2024-05-22T21:19:29.691 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:29.691 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":128,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:19:25.399186+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:12.848545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:13.344629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:29.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.691+0000 7ff1ce600640 1 -- 172.21.15.161:0/3930713525 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff1a80666c0 msgr2=0x7ff1a8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.691+0000 7ff1ce600640 1 --2- 172.21.15.161:0/3930713525 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff1a80666c0 0x7ff1a8068b80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80012f0 tx=0x7ff1b8001700 comp rx=0 tx=0).stop 2024-05-22T21:19:29.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.691+0000 7ff1ce600640 1 -- 172.21.15.161:0/3930713525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1c815dd50 msgr2=0x7ff1c814ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.691+0000 7ff1ce600640 1 --2- 172.21.15.161:0/3930713525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1c815dd50 0x7ff1c814ca00 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff1b401abe0 tx=0x7ff1b40192d0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.691+0000 7ff1ce600640 1 -- 172.21.15.161:0/3930713525 shutdown_connections 2024-05-22T21:19:29.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.691+0000 7ff1ce600640 1 --2- 172.21.15.161:0/3930713525 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff1a80666c0 0x7ff1a8068b80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.691+0000 7ff1ce600640 1 --2- 172.21.15.161:0/3930713525 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1c815e130 0x7ff1c8157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.691+0000 7ff1ce600640 1 --2- 172.21.15.161:0/3930713525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1c815dd50 0x7ff1c814ca00 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.691+0000 7ff1ce600640 1 --2- 172.21.15.161:0/3930713525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1c815d410 0x7ff1c814d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.691+0000 7ff1ce600640 1 -- 172.21.15.161:0/3930713525 >> 172.21.15.161:0/3930713525 conn(0x7ff1c80b0bc0 msgr2=0x7ff1c80acf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:29.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.691+0000 7ff1ce600640 1 -- 172.21.15.161:0/3930713525 shutdown_connections 2024-05-22T21:19:29.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.691+0000 7ff1ce600640 1 -- 172.21.15.161:0/3930713525 wait complete. 2024-05-22T21:19:29.713 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:19:12.848545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:19:13.344629+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:19:29.713 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:19:29.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.847+0000 7fde46c00640 1 Processor -- start 2024-05-22T21:19:29.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.847+0000 7fde46c00640 1 -- start start 2024-05-22T21:19:29.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde46c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fde4006fa80 0x7fde400cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde46c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde40070b70 con 0x7fde401436d0 2024-05-22T21:19:29.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde46c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde4006ff60 con 0x7fde400b33d0 2024-05-22T21:19:29.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde46c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde400cc720 con 0x7fde4006fa80 2024-05-22T21:19:29.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde45800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fde401436d0 0x7fde400b2cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47382/0 (socket says 172.21.15.161:47382) 2024-05-22T21:19:29.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde45800640 1 -- 172.21.15.161:0/1867519210 learned_addr learned my addr 172.21.15.161:0/1867519210 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:29.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde46200640 1 --2- 172.21.15.161:0/1867519210 >> v2:172.21.15.179:3300/0 conn(0x7fde4006fa80 0x7fde400cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde3fe00640 1 -- 172.21.15.161:0/1867519210 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285321854 0 0) 0x7fde40070b70 con 0x7fde401436d0 2024-05-22T21:19:29.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde3fe00640 1 -- 172.21.15.161:0/1867519210 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde28003530 con 0x7fde401436d0 2024-05-22T21:19:29.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde3fe00640 1 -- 172.21.15.161:0/1867519210 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1975615226 0 0) 0x7fde28003530 con 0x7fde401436d0 2024-05-22T21:19:29.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde3fe00640 1 -- 172.21.15.161:0/1867519210 >> v2:172.21.15.179:3300/0 conn(0x7fde4006fa80 msgr2=0x7fde400cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:29.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde3fe00640 1 --2- 172.21.15.161:0/1867519210 >> v2:172.21.15.179:3300/0 conn(0x7fde4006fa80 0x7fde400cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde3fe00640 1 -- 172.21.15.161:0/1867519210 >> v1:172.21.15.179:6789/0 conn(0x7fde400b33d0 legacy=0x7fde40071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde3fe00640 1 -- 172.21.15.161:0/1867519210 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde40070240 con 0x7fde401436d0 2024-05-22T21:19:29.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde3fe00640 1 -- 172.21.15.161:0/1867519210 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fde34001680 con 0x7fde401436d0 2024-05-22T21:19:29.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde3fe00640 1 -- 172.21.15.161:0/1867519210 >> v1:172.21.15.161:6789/0 conn(0x7fde401436d0 legacy=0x7fde400b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde3fe00640 1 --2- 172.21.15.161:0/1867519210 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde28003980 0x7fde28003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde3fe00640 1 --2- 172.21.15.161:0/1867519210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde280047d0 0x7fde28004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde3fe00640 1 --2- 172.21.15.161:0/1867519210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde28005190 0x7fde2800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde3fe00640 1 -- 172.21.15.161:0/1867519210 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde400cc720 con 0x7fde28005190 2024-05-22T21:19:29.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde3fe00640 1 -- 172.21.15.161:0/1867519210 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2800c130 con 0x7fde280047d0 2024-05-22T21:19:29.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde3fe00640 1 -- 172.21.15.161:0/1867519210 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2800c2f0 con 0x7fde28003980 2024-05-22T21:19:29.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde46200640 1 --2- 172.21.15.161:0/1867519210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde28005190 0x7fde2800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde45800640 1 --2- 172.21.15.161:0/1867519210 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde28003980 0x7fde28003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde44e00640 1 --2- 172.21.15.161:0/1867519210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde280047d0 0x7fde28004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde46200640 1 -- 172.21.15.161:0/1867519210 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde28003980 msgr2=0x7fde28003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:29.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde46200640 1 --2- 172.21.15.161:0/1867519210 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde28003980 0x7fde28003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde46200640 1 -- 172.21.15.161:0/1867519210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde280047d0 msgr2=0x7fde28004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:29.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde46200640 1 --2- 172.21.15.161:0/1867519210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde280047d0 0x7fde28004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.851+0000 7fde46200640 1 -- 172.21.15.161:0/1867519210 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde28003530 con 0x7fde28005190 2024-05-22T21:19:29.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.855+0000 7fde46200640 1 -- 172.21.15.161:0/1867519210 reap_dead start 2024-05-22T21:19:29.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.855+0000 7fde46200640 1 --2- 172.21.15.161:0/1867519210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde28005190 0x7fde2800b990 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fde4006fe90 tx=0x7fde3801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f8d8b77cd5d7dd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:29.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.855+0000 7fde3fe00640 1 -- 172.21.15.161:0/1867519210 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde380a5c10 con 0x7fde28005190 2024-05-22T21:19:29.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.855+0000 7fde3fe00640 1 -- 172.21.15.161:0/1867519210 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3800f040 con 0x7fde28005190 2024-05-22T21:19:29.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.855+0000 7fde3fe00640 1 -- 172.21.15.161:0/1867519210 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde380ad440 con 0x7fde28005190 2024-05-22T21:19:29.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.855+0000 7fde46c00640 1 -- 172.21.15.161:0/1867519210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde28005190 msgr2=0x7fde2800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.855+0000 7fde46c00640 1 --2- 172.21.15.161:0/1867519210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde28005190 0x7fde2800b990 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fde4006fe90 tx=0x7fde3801cc40 comp rx=0 tx=0).stop 2024-05-22T21:19:29.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.855+0000 7fde46c00640 1 -- 172.21.15.161:0/1867519210 shutdown_connections 2024-05-22T21:19:29.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.855+0000 7fde46c00640 1 --2- 172.21.15.161:0/1867519210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde28005190 0x7fde2800b990 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.855+0000 7fde46c00640 1 -- 172.21.15.161:0/1867519210 >> 172.21.15.161:0/1867519210 conn(0x7fde400cec00 msgr2=0x7fde400cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:29.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.855+0000 7fde46c00640 1 -- 172.21.15.161:0/1867519210 shutdown_connections 2024-05-22T21:19:29.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.855+0000 7fde46c00640 1 -- 172.21.15.161:0/1867519210 wait complete. 2024-05-22T21:19:29.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.855+0000 7fde46c00640 1 Processor -- start 2024-05-22T21:19:29.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.855+0000 7fde46c00640 1 -- start start 2024-05-22T21:19:29.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde28005190 0x7fde4013c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde4013cf40 0x7fde4013d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde400ad2c0 0x7fde400ad770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4013e150 con 0x7fde400ad2c0 2024-05-22T21:19:29.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4013d900 con 0x7fde4013cf40 2024-05-22T21:19:29.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde400adcb0 con 0x7fde28005190 2024-05-22T21:19:29.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde400ad2c0 0x7fde400ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde400ad2c0 0x7fde400ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57654/0 (socket says 172.21.15.161:57654) 2024-05-22T21:19:29.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46200640 1 -- 172.21.15.161:0/1763966052 learned_addr learned my addr 172.21.15.161:0/1763966052 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:29.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde44e00640 1 --2- 172.21.15.161:0/1763966052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde4013cf40 0x7fde4013d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde45800640 1 --2- 172.21.15.161:0/1763966052 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde28005190 0x7fde4013c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46200640 1 -- 172.21.15.161:0/1763966052 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde28005190 msgr2=0x7fde4013c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46200640 1 --2- 172.21.15.161:0/1763966052 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde28005190 0x7fde4013c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46200640 1 -- 172.21.15.161:0/1763966052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde4013cf40 msgr2=0x7fde4013d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:29.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46200640 1 --2- 172.21.15.161:0/1763966052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde4013cf40 0x7fde4013d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:29.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46200640 1 -- 172.21.15.161:0/1763966052 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde400adef0 con 0x7fde400ad2c0 2024-05-22T21:19:29.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46200640 1 --2- 172.21.15.161:0/1763966052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde400ad2c0 0x7fde400ad770 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fde3801ca70 tx=0x7fde38009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:29.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde3d600640 1 -- 172.21.15.161:0/1763966052 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde380be070 con 0x7fde400ad2c0 2024-05-22T21:19:29.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde3d600640 1 -- 172.21.15.161:0/1763966052 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde380ad650 con 0x7fde400ad2c0 2024-05-22T21:19:29.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde3d600640 1 -- 172.21.15.161:0/1763966052 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde380ada00 con 0x7fde400ad2c0 2024-05-22T21:19:29.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46c00640 1 -- 172.21.15.161:0/1763966052 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4013fc40 con 0x7fde400ad2c0 2024-05-22T21:19:29.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46c00640 1 -- 172.21.15.161:0/1763966052 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde40140200 con 0x7fde400ad2c0 2024-05-22T21:19:29.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde46c00640 1 -- 172.21.15.161:0/1763966052 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde40140420 con 0x7fde400ad2c0 2024-05-22T21:19:29.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.859+0000 7fde3d600640 1 -- 172.21.15.161:0/1763966052 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fde380b4040 con 0x7fde400ad2c0 2024-05-22T21:19:29.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.863+0000 7fde3d600640 1 --2- 172.21.15.161:0/1763966052 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fde200666c0 0x7fde20068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:29.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.863+0000 7fde3d600640 1 -- 172.21.15.161:0/1763966052 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde3811dd30 con 0x7fde400ad2c0 2024-05-22T21:19:29.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.863+0000 7fde45800640 1 --2- 172.21.15.161:0/1763966052 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fde200666c0 0x7fde20068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:29.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.863+0000 7fde45800640 1 --2- 172.21.15.161:0/1763966052 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fde200666c0 0x7fde20068b80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fde34000c10 tx=0x7fde34001c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:29.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:29.867+0000 7fde3d600640 1 -- 172.21.15.161:0/1763966052 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fde380eb0b0 con 0x7fde400ad2c0 2024-05-22T21:19:30.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.023+0000 7fde46c00640 1 -- 172.21.15.161:0/1763966052 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde40067070 con 0x7fde400ad2c0 2024-05-22T21:19:30.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.023+0000 7fde3d600640 1 -- 172.21.15.161:0/1763966052 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde380ee6d0 con 0x7fde400ad2c0 2024-05-22T21:19:30.026 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:19:30.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.027+0000 7fde46c00640 1 -- 172.21.15.161:0/1763966052 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fde200666c0 msgr2=0x7fde20068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.027+0000 7fde46c00640 1 --2- 172.21.15.161:0/1763966052 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fde200666c0 0x7fde20068b80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fde34000c10 tx=0x7fde34001c10 comp rx=0 tx=0).stop 2024-05-22T21:19:30.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.027+0000 7fde46c00640 1 -- 172.21.15.161:0/1763966052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde400ad2c0 msgr2=0x7fde400ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.027+0000 7fde46c00640 1 --2- 172.21.15.161:0/1763966052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde400ad2c0 0x7fde400ad770 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fde3801ca70 tx=0x7fde38009750 comp rx=0 tx=0).stop 2024-05-22T21:19:30.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.027+0000 7fde46c00640 1 -- 172.21.15.161:0/1763966052 shutdown_connections 2024-05-22T21:19:30.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.027+0000 7fde46c00640 1 --2- 172.21.15.161:0/1763966052 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fde200666c0 0x7fde20068b80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.027+0000 7fde46c00640 1 --2- 172.21.15.161:0/1763966052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde400ad2c0 0x7fde400ad770 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.027+0000 7fde46c00640 1 --2- 172.21.15.161:0/1763966052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde4013cf40 0x7fde4013d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.027+0000 7fde46c00640 1 --2- 172.21.15.161:0/1763966052 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde28005190 0x7fde4013c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.027+0000 7fde46c00640 1 -- 172.21.15.161:0/1763966052 >> 172.21.15.161:0/1763966052 conn(0x7fde400cec00 msgr2=0x7fde400cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:30.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.027+0000 7fde46c00640 1 -- 172.21.15.161:0/1763966052 shutdown_connections 2024-05-22T21:19:30.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.027+0000 7fde46c00640 1 -- 172.21.15.161:0/1763966052 wait complete. 2024-05-22T21:19:30.047 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:19:30.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.183+0000 7f6968200640 1 Processor -- start 2024-05-22T21:19:30.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.183+0000 7f6968200640 1 -- start start 2024-05-22T21:19:30.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.183+0000 7f6968200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f696015b600 0x7f696006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.183+0000 7f6968200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f696006c7e0 0x7f69600b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.183+0000 7f6968200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f696006cc40 con 0x7f6960143c40 2024-05-22T21:19:30.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.183+0000 7f6968200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696015ba60 con 0x7f696006c7e0 2024-05-22T21:19:30.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.183+0000 7f6968200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6960144bb0 con 0x7f696015b600 2024-05-22T21:19:30.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.183+0000 7f6966200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f696006c7e0 0x7f69600b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.183+0000 7f6966200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f696006c7e0 0x7f69600b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57662/0 (socket says 172.21.15.161:57662) 2024-05-22T21:19:30.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.183+0000 7f6966200640 1 -- 172.21.15.161:0/3885212035 learned_addr learned my addr 172.21.15.161:0/3885212035 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:30.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.183+0000 7f6965800640 1 --1- 172.21.15.161:0/3885212035 >> v1:172.21.15.161:6789/0 conn(0x7f6960143c40 0x7f69601444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47386/0 (socket says 172.21.15.161:47386) 2024-05-22T21:19:30.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.183+0000 7f6964e00640 1 --2- 172.21.15.161:0/3885212035 >> v2:172.21.15.179:3300/0 conn(0x7f696015b600 0x7f696006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.183+0000 7f6966200640 1 -- 172.21.15.161:0/3885212035 >> v2:172.21.15.179:3300/0 conn(0x7f696015b600 msgr2=0x7f696006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:30.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.183+0000 7f6966200640 1 --2- 172.21.15.161:0/3885212035 >> v2:172.21.15.179:3300/0 conn(0x7f696015b600 0x7f696006c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.183+0000 7f6966200640 1 -- 172.21.15.161:0/3885212035 >> v1:172.21.15.161:6789/0 conn(0x7f6960143c40 legacy=0x7f69601444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.183+0000 7f6966200640 1 -- 172.21.15.161:0/3885212035 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696015efb0 con 0x7f696006c7e0 2024-05-22T21:19:30.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.187+0000 7f6966200640 1 --2- 172.21.15.161:0/3885212035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f696006c7e0 0x7f69600b2160 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f69600b1c80 tx=0x7f695801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd0f22aaa5378ac server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:30.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.187+0000 7f695fe00640 1 -- 172.21.15.161:0/3885212035 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69580a58b0 con 0x7f696006c7e0 2024-05-22T21:19:30.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.187+0000 7f695fe00640 1 -- 172.21.15.161:0/3885212035 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69580a5a10 con 0x7f696006c7e0 2024-05-22T21:19:30.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.187+0000 7f695fe00640 1 -- 172.21.15.161:0/3885212035 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69580a5da0 con 0x7f696006c7e0 2024-05-22T21:19:30.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.187+0000 7f6968200640 1 -- 172.21.15.161:0/3885212035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f696006c7e0 msgr2=0x7f69600b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.187+0000 7f6968200640 1 --2- 172.21.15.161:0/3885212035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f696006c7e0 0x7f69600b2160 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f69600b1c80 tx=0x7f695801c9f0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.187+0000 7f6968200640 1 -- 172.21.15.161:0/3885212035 shutdown_connections 2024-05-22T21:19:30.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.187+0000 7f6968200640 1 --2- 172.21.15.161:0/3885212035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f696006c7e0 0x7f69600b2160 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.187+0000 7f6968200640 1 --2- 172.21.15.161:0/3885212035 >> v2:172.21.15.179:3300/0 conn(0x7f696015b600 0x7f696006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.187+0000 7f6968200640 1 -- 172.21.15.161:0/3885212035 >> 172.21.15.161:0/3885212035 conn(0x7f696006a2d0 msgr2=0x7f696006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:30.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.187+0000 7f6968200640 1 -- 172.21.15.161:0/3885212035 shutdown_connections 2024-05-22T21:19:30.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.187+0000 7f6968200640 1 -- 172.21.15.161:0/3885212035 wait complete. 2024-05-22T21:19:30.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6968200640 1 Processor -- start 2024-05-22T21:19:30.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6968200640 1 -- start start 2024-05-22T21:19:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6968200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f696006c7e0 0x7f69601397e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6968200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6960143c40 0x7f69600706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6968200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f696015b600 0x7f696013f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6968200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6960070c30 con 0x7f696015b600 2024-05-22T21:19:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6968200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6960070ed0 con 0x7f6960143c40 2024-05-22T21:19:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6968200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6960158ef0 con 0x7f696006c7e0 2024-05-22T21:19:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6966200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f696015b600 0x7f696013f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6966200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f696015b600 0x7f696013f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57672/0 (socket says 172.21.15.161:57672) 2024-05-22T21:19:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6966200640 1 -- 172.21.15.161:0/3081492679 learned_addr learned my addr 172.21.15.161:0/3081492679 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6964e00640 1 --2- 172.21.15.161:0/3081492679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6960143c40 0x7f69600706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6965800640 1 --2- 172.21.15.161:0/3081492679 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f696006c7e0 0x7f69601397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6966200640 1 -- 172.21.15.161:0/3081492679 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f696006c7e0 msgr2=0x7f69601397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6966200640 1 --2- 172.21.15.161:0/3081492679 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f696006c7e0 0x7f69601397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6966200640 1 -- 172.21.15.161:0/3081492679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6960143c40 msgr2=0x7f69600706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6966200640 1 --2- 172.21.15.161:0/3081492679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6960143c40 0x7f69600706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6966200640 1 -- 172.21.15.161:0/3081492679 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6958009b30 con 0x7f696015b600 2024-05-22T21:19:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6964e00640 1 --2- 172.21.15.161:0/3081492679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6960143c40 0x7f69600706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6965800640 1 --2- 172.21.15.161:0/3081492679 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f696006c7e0 0x7f69601397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:30.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6966200640 1 --2- 172.21.15.161:0/3081492679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f696015b600 0x7f696013f530 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f69580af040 tx=0x7f6958019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:30.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f695d600640 1 -- 172.21.15.161:0/3081492679 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69580b8670 con 0x7f696015b600 2024-05-22T21:19:30.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f695d600640 1 -- 172.21.15.161:0/3081492679 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69580b87d0 con 0x7f696015b600 2024-05-22T21:19:30.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6968200640 1 -- 172.21.15.161:0/3081492679 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69601591d0 con 0x7f696015b600 2024-05-22T21:19:30.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f695d600640 1 -- 172.21.15.161:0/3081492679 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69580b8b80 con 0x7f696015b600 2024-05-22T21:19:30.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.191+0000 7f6968200640 1 -- 172.21.15.161:0/3081492679 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6960159790 con 0x7f696015b600 2024-05-22T21:19:30.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.195+0000 7f6968200640 1 -- 172.21.15.161:0/3081492679 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6960159980 con 0x7f696015b600 2024-05-22T21:19:30.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.195+0000 7f695d600640 1 -- 172.21.15.161:0/3081492679 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f69580b8ce0 con 0x7f696015b600 2024-05-22T21:19:30.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.195+0000 7f695d600640 1 --2- 172.21.15.161:0/3081492679 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69400666c0 0x7f6940068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.195+0000 7f695d600640 1 -- 172.21.15.161:0/3081492679 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f695811f6d0 con 0x7f696015b600 2024-05-22T21:19:30.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.195+0000 7f6965800640 1 --2- 172.21.15.161:0/3081492679 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69400666c0 0x7f6940068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.195+0000 7f6965800640 1 --2- 172.21.15.161:0/3081492679 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69400666c0 0x7f6940068b80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6954000f70 tx=0x7f6954007d70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:30.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.199+0000 7f695d600640 1 -- 172.21.15.161:0/3081492679 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f69580ecb10 con 0x7f696015b600 2024-05-22T21:19:30.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.351+0000 7f6968200640 1 -- 172.21.15.161:0/3081492679 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6960067220 con 0x7f696015b600 2024-05-22T21:19:30.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.355+0000 7f695d600640 1 -- 172.21.15.161:0/3081492679 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) v1 ==== 104+0+11 (secure 0 0 0) 0x7f69580f0130 con 0x7f696015b600 2024-05-22T21:19:30.356 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:19:30.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.355+0000 7f6968200640 1 -- 172.21.15.161:0/3081492679 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69400666c0 msgr2=0x7f6940068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.355+0000 7f6968200640 1 --2- 172.21.15.161:0/3081492679 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69400666c0 0x7f6940068b80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6954000f70 tx=0x7f6954007d70 comp rx=0 tx=0).stop 2024-05-22T21:19:30.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.355+0000 7f6968200640 1 -- 172.21.15.161:0/3081492679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f696015b600 msgr2=0x7f696013f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.355+0000 7f6968200640 1 --2- 172.21.15.161:0/3081492679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f696015b600 0x7f696013f530 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f69580af040 tx=0x7f6958019790 comp rx=0 tx=0).stop 2024-05-22T21:19:30.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.355+0000 7f6968200640 1 -- 172.21.15.161:0/3081492679 shutdown_connections 2024-05-22T21:19:30.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.355+0000 7f6968200640 1 --2- 172.21.15.161:0/3081492679 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69400666c0 0x7f6940068b80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.355+0000 7f6968200640 1 --2- 172.21.15.161:0/3081492679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f696015b600 0x7f696013f530 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.355+0000 7f6968200640 1 --2- 172.21.15.161:0/3081492679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6960143c40 0x7f69600706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.355+0000 7f6968200640 1 --2- 172.21.15.161:0/3081492679 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f696006c7e0 0x7f69601397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.355+0000 7f6968200640 1 -- 172.21.15.161:0/3081492679 >> 172.21.15.161:0/3081492679 conn(0x7f696006a2d0 msgr2=0x7f6960139e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:30.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.355+0000 7f6968200640 1 -- 172.21.15.161:0/3081492679 shutdown_connections 2024-05-22T21:19:30.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.355+0000 7f6968200640 1 -- 172.21.15.161:0/3081492679 wait complete. 2024-05-22T21:19:30.379 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:19:30.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.515+0000 7f75fae00640 1 Processor -- start 2024-05-22T21:19:30.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.515+0000 7f75fae00640 1 -- start start 2024-05-22T21:19:30.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.519+0000 7f75fae00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f75f41402e0 0x7f75f413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.519+0000 7f75fae00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f4140090 con 0x7f75f413fc80 2024-05-22T21:19:30.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.519+0000 7f75fae00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f413d3a0 con 0x7f75f413cfc0 2024-05-22T21:19:30.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.519+0000 7f75fae00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f40cd920 con 0x7f75f41402e0 2024-05-22T21:19:30.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.519+0000 7f75eb400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f75f41402e0 0x7f75f413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.519+0000 7f75eb400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f75f41402e0 0x7f75f413f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57684/0 (socket says 172.21.15.161:57684) 2024-05-22T21:19:30.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.519+0000 7f75eb400640 1 -- 172.21.15.161:0/1879475225 learned_addr learned my addr 172.21.15.161:0/1879475225 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:30.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.519+0000 7f75eb400640 1 -- 172.21.15.161:0/1879475225 >> v1:172.21.15.179:6789/0 conn(0x7f75f413cfc0 legacy=0x7f75f415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.519+0000 7f75eb400640 1 -- 172.21.15.161:0/1879475225 >> v1:172.21.15.161:6789/0 conn(0x7f75f413fc80 legacy=0x7f75f406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.519+0000 7f75eb400640 1 -- 172.21.15.161:0/1879475225 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f40cdc00 con 0x7f75f41402e0 2024-05-22T21:19:30.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.523+0000 7f75eb400640 1 --2- 172.21.15.161:0/1879475225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f41402e0 0x7f75f413f740 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009870 tx=0x7f75dc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d0c5098d309024 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:30.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.523+0000 7f75f3400640 1 -- 172.21.15.161:0/1879475225 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0a6040 con 0x7f75f41402e0 2024-05-22T21:19:30.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.523+0000 7f75f3400640 1 -- 172.21.15.161:0/1879475225 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75dc0aa400 con 0x7f75f41402e0 2024-05-22T21:19:30.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.523+0000 7f75f3400640 1 -- 172.21.15.161:0/1879475225 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0aa790 con 0x7f75f41402e0 2024-05-22T21:19:30.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.523+0000 7f75fae00640 1 -- 172.21.15.161:0/1879475225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f41402e0 msgr2=0x7f75f413f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.523+0000 7f75fae00640 1 --2- 172.21.15.161:0/1879475225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f41402e0 0x7f75f413f740 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009870 tx=0x7f75dc01c4a0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.523+0000 7f75fae00640 1 -- 172.21.15.161:0/1879475225 shutdown_connections 2024-05-22T21:19:30.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.523+0000 7f75fae00640 1 --2- 172.21.15.161:0/1879475225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f41402e0 0x7f75f413f740 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.523+0000 7f75fae00640 1 -- 172.21.15.161:0/1879475225 >> 172.21.15.161:0/1879475225 conn(0x7f75f406a2d0 msgr2=0x7f75f406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:30.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.523+0000 7f75fae00640 1 -- 172.21.15.161:0/1879475225 shutdown_connections 2024-05-22T21:19:30.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.523+0000 7f75fae00640 1 -- 172.21.15.161:0/1879475225 wait complete. 2024-05-22T21:19:30.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.523+0000 7f75fae00640 1 Processor -- start 2024-05-22T21:19:30.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.523+0000 7f75fae00640 1 -- start start 2024-05-22T21:19:30.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75fae00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75f413cfc0 0x7f75f413b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75fae00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f413fc80 0x7f75f4135650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75fae00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75f41402e0 0x7f75f4159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75fae00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f4135b90 con 0x7f75f413fc80 2024-05-22T21:19:30.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75fae00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f413a5f0 con 0x7f75f413cfc0 2024-05-22T21:19:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75fae00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f413a8d0 con 0x7f75f41402e0 2024-05-22T21:19:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75eb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f413fc80 0x7f75f4135650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75eb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f413fc80 0x7f75f4135650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57696/0 (socket says 172.21.15.161:57696) 2024-05-22T21:19:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75eb400640 1 -- 172.21.15.161:0/880630727 learned_addr learned my addr 172.21.15.161:0/880630727 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75f3e00640 1 --2- 172.21.15.161:0/880630727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75f413cfc0 0x7f75f413b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75f8e00640 1 --2- 172.21.15.161:0/880630727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75f41402e0 0x7f75f4159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75eb400640 1 -- 172.21.15.161:0/880630727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75f41402e0 msgr2=0x7f75f4159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75eb400640 1 --2- 172.21.15.161:0/880630727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75f41402e0 0x7f75f4159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75eb400640 1 -- 172.21.15.161:0/880630727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75f413cfc0 msgr2=0x7f75f413b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75eb400640 1 --2- 172.21.15.161:0/880630727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75f413cfc0 0x7f75f413b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75eb400640 1 -- 172.21.15.161:0/880630727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc0094d0 con 0x7f75f413fc80 2024-05-22T21:19:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75f3e00640 1 --2- 172.21.15.161:0/880630727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75f413cfc0 0x7f75f413b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:30.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75eb400640 1 --2- 172.21.15.161:0/880630727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f413fc80 0x7f75f4135650 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0097c0 tx=0x7f75dc01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:30.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75f0c00640 1 -- 172.21.15.161:0/880630727 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0b3800 con 0x7f75f413fc80 2024-05-22T21:19:30.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75f0c00640 1 -- 172.21.15.161:0/880630727 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75dc0b39a0 con 0x7f75f413fc80 2024-05-22T21:19:30.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75fae00640 1 -- 172.21.15.161:0/880630727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f413abb0 con 0x7f75f413fc80 2024-05-22T21:19:30.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75f0c00640 1 -- 172.21.15.161:0/880630727 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0b3d50 con 0x7f75f413fc80 2024-05-22T21:19:30.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75fae00640 1 -- 172.21.15.161:0/880630727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f413b170 con 0x7f75f413fc80 2024-05-22T21:19:30.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75fae00640 1 -- 172.21.15.161:0/880630727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f4070f10 con 0x7f75f413fc80 2024-05-22T21:19:30.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.527+0000 7f75f0c00640 1 -- 172.21.15.161:0/880630727 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f75dc0c6430 con 0x7f75f413fc80 2024-05-22T21:19:30.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.531+0000 7f75f0c00640 1 --2- 172.21.15.161:0/880630727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f75d0066690 0x7f75d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.531+0000 7f75f0c00640 1 -- 172.21.15.161:0/880630727 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75dc11ee80 con 0x7f75f413fc80 2024-05-22T21:19:30.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.531+0000 7f75f3e00640 1 --2- 172.21.15.161:0/880630727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f75d0066690 0x7f75d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.531+0000 7f75f3e00640 1 --2- 172.21.15.161:0/880630727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f75d0066690 0x7f75d0068b50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f75f4157f50 tx=0x7f75e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:30.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.535+0000 7f75f0c00640 1 -- 172.21.15.161:0/880630727 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f75dc0ec1e0 con 0x7f75f413fc80 2024-05-22T21:19:30.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.687+0000 7f75fae00640 1 -- 172.21.15.161:0/880630727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f75f4067220 con 0x7f75f413fc80 2024-05-22T21:19:30.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.687+0000 7f75f0c00640 1 -- 172.21.15.161:0/880630727 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) v1 ==== 100+0+11 (secure 0 0 0) 0x7f75dc0ef800 con 0x7f75f413fc80 2024-05-22T21:19:30.690 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:19:30.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.691+0000 7f75fae00640 1 -- 172.21.15.161:0/880630727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f75d0066690 msgr2=0x7f75d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.691+0000 7f75fae00640 1 --2- 172.21.15.161:0/880630727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f75d0066690 0x7f75d0068b50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f75f4157f50 tx=0x7f75e4009040 comp rx=0 tx=0).stop 2024-05-22T21:19:30.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.691+0000 7f75fae00640 1 -- 172.21.15.161:0/880630727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f413fc80 msgr2=0x7f75f4135650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.691+0000 7f75fae00640 1 --2- 172.21.15.161:0/880630727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f413fc80 0x7f75f4135650 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0097c0 tx=0x7f75dc01ace0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.691+0000 7f75fae00640 1 -- 172.21.15.161:0/880630727 shutdown_connections 2024-05-22T21:19:30.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.691+0000 7f75fae00640 1 --2- 172.21.15.161:0/880630727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f75d0066690 0x7f75d0068b50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.691+0000 7f75fae00640 1 --2- 172.21.15.161:0/880630727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75f41402e0 0x7f75f4159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.691+0000 7f75fae00640 1 --2- 172.21.15.161:0/880630727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75f413fc80 0x7f75f4135650 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.691+0000 7f75fae00640 1 --2- 172.21.15.161:0/880630727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75f413cfc0 0x7f75f413b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.691+0000 7f75fae00640 1 -- 172.21.15.161:0/880630727 >> 172.21.15.161:0/880630727 conn(0x7f75f406a2d0 msgr2=0x7f75f40b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:30.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.691+0000 7f75fae00640 1 -- 172.21.15.161:0/880630727 shutdown_connections 2024-05-22T21:19:30.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.691+0000 7f75fae00640 1 -- 172.21.15.161:0/880630727 wait complete. 2024-05-22T21:19:30.711 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:19:30.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.843+0000 7f2521600640 1 Processor -- start 2024-05-22T21:19:30.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.843+0000 7f2521600640 1 -- start start 2024-05-22T21:19:30.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.847+0000 7f2521600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f251c15b600 0x7f251c06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.847+0000 7f2521600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f251c06c7e0 0x7f251c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.847+0000 7f2521600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251c06cc40 con 0x7f251c143c40 2024-05-22T21:19:30.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.847+0000 7f2521600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251c15ba60 con 0x7f251c06c7e0 2024-05-22T21:19:30.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.847+0000 7f2521600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251c144bb0 con 0x7f251c15b600 2024-05-22T21:19:30.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.847+0000 7f251b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f251c06c7e0 0x7f251c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.847+0000 7f251b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f251c06c7e0 0x7f251c0b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57712/0 (socket says 172.21.15.161:57712) 2024-05-22T21:19:30.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.847+0000 7f251b400640 1 -- 172.21.15.161:0/974922967 learned_addr learned my addr 172.21.15.161:0/974922967 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:30.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.847+0000 7f2512000640 1 --2- 172.21.15.161:0/974922967 >> v2:172.21.15.179:3300/0 conn(0x7f251c15b600 0x7f251c06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.847+0000 7f251b400640 1 -- 172.21.15.161:0/974922967 >> v2:172.21.15.179:3300/0 conn(0x7f251c15b600 msgr2=0x7f251c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:30.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.847+0000 7f251b400640 1 --2- 172.21.15.161:0/974922967 >> v2:172.21.15.179:3300/0 conn(0x7f251c15b600 0x7f251c06c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.847+0000 7f251b400640 1 -- 172.21.15.161:0/974922967 >> v1:172.21.15.179:6789/0 conn(0x7f251c143c40 legacy=0x7f251c1444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:30.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.847+0000 7f251b400640 1 -- 172.21.15.161:0/974922967 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c15efb0 con 0x7f251c06c7e0 2024-05-22T21:19:30.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f251b400640 1 --2- 172.21.15.161:0/974922967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c06c7e0 0x7f251c0b2160 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f251c0b1c80 tx=0x7f251401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d176d2615a041c08 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:30.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f251a000640 1 -- 172.21.15.161:0/974922967 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25140aa640 con 0x7f251c06c7e0 2024-05-22T21:19:30.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f251a000640 1 -- 172.21.15.161:0/974922967 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25140aa7a0 con 0x7f251c06c7e0 2024-05-22T21:19:30.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f251a000640 1 -- 172.21.15.161:0/974922967 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25140aab30 con 0x7f251c06c7e0 2024-05-22T21:19:30.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2521600640 1 -- 172.21.15.161:0/974922967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c06c7e0 msgr2=0x7f251c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2521600640 1 --2- 172.21.15.161:0/974922967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c06c7e0 0x7f251c0b2160 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f251c0b1c80 tx=0x7f251401c4e0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2521600640 1 -- 172.21.15.161:0/974922967 shutdown_connections 2024-05-22T21:19:30.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2521600640 1 --2- 172.21.15.161:0/974922967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c06c7e0 0x7f251c0b2160 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2521600640 1 --2- 172.21.15.161:0/974922967 >> v2:172.21.15.179:3300/0 conn(0x7f251c15b600 0x7f251c06c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2521600640 1 -- 172.21.15.161:0/974922967 >> 172.21.15.161:0/974922967 conn(0x7f251c06a2d0 msgr2=0x7f251c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:30.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2521600640 1 -- 172.21.15.161:0/974922967 shutdown_connections 2024-05-22T21:19:30.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2521600640 1 -- 172.21.15.161:0/974922967 wait complete. 2024-05-22T21:19:30.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2521600640 1 Processor -- start 2024-05-22T21:19:30.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2521600640 1 -- start start 2024-05-22T21:19:30.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2521600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f251c06c7e0 0x7f251c129500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2521600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c143c40 0x7f251c153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2521600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f251c15b600 0x7f251c153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2521600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c06fce0 con 0x7f251c143c40 2024-05-22T21:19:30.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2521600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c06ffc0 con 0x7f251c06c7e0 2024-05-22T21:19:30.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2521600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251c0adbf0 con 0x7f251c15b600 2024-05-22T21:19:30.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2512000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c143c40 0x7f251c153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2512000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c143c40 0x7f251c153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57714/0 (socket says 172.21.15.161:57714) 2024-05-22T21:19:30.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f2512000640 1 -- 172.21.15.161:0/609271624 learned_addr learned my addr 172.21.15.161:0/609271624 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:30.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f251aa00640 1 --2- 172.21.15.161:0/609271624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f251c06c7e0 0x7f251c129500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.851+0000 7f251b400640 1 --2- 172.21.15.161:0/609271624 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f251c15b600 0x7f251c153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.855+0000 7f2512000640 1 -- 172.21.15.161:0/609271624 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f251c15b600 msgr2=0x7f251c153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.855+0000 7f2512000640 1 --2- 172.21.15.161:0/609271624 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f251c15b600 0x7f251c153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.855+0000 7f2512000640 1 -- 172.21.15.161:0/609271624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f251c06c7e0 msgr2=0x7f251c129500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:30.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.855+0000 7f2512000640 1 --2- 172.21.15.161:0/609271624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f251c06c7e0 0x7f251c129500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:30.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.855+0000 7f2512000640 1 -- 172.21.15.161:0/609271624 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140094d0 con 0x7f251c143c40 2024-05-22T21:19:30.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.855+0000 7f251b400640 1 --2- 172.21.15.161:0/609271624 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f251c15b600 0x7f251c153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:30.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.855+0000 7f251aa00640 1 --2- 172.21.15.161:0/609271624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f251c06c7e0 0x7f251c129500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:30.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.855+0000 7f2512000640 1 --2- 172.21.15.161:0/609271624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c143c40 0x7f251c153300 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f250400d830 tx=0x7f250400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:30.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.855+0000 7f2513400640 1 -- 172.21.15.161:0/609271624 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250400b6c0 con 0x7f251c143c40 2024-05-22T21:19:30.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.855+0000 7f2513400640 1 -- 172.21.15.161:0/609271624 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250400b820 con 0x7f251c143c40 2024-05-22T21:19:30.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.855+0000 7f2513400640 1 -- 172.21.15.161:0/609271624 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250400bb90 con 0x7f251c143c40 2024-05-22T21:19:30.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.855+0000 7f2521600640 1 -- 172.21.15.161:0/609271624 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251c0aded0 con 0x7f251c143c40 2024-05-22T21:19:30.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.855+0000 7f2521600640 1 -- 172.21.15.161:0/609271624 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251c0ace80 con 0x7f251c143c40 2024-05-22T21:19:30.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.855+0000 7f2521600640 1 -- 172.21.15.161:0/609271624 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251c067220 con 0x7f251c143c40 2024-05-22T21:19:30.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.855+0000 7f2513400640 1 -- 172.21.15.161:0/609271624 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f250401e460 con 0x7f251c143c40 2024-05-22T21:19:30.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.855+0000 7f2513400640 1 --2- 172.21.15.161:0/609271624 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f24f8066690 0x7f24f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:30.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.859+0000 7f2513400640 1 -- 172.21.15.161:0/609271624 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2504084b10 con 0x7f251c143c40 2024-05-22T21:19:30.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.859+0000 7f251aa00640 1 --2- 172.21.15.161:0/609271624 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f24f8066690 0x7f24f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:30.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.859+0000 7f251aa00640 1 --2- 172.21.15.161:0/609271624 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f24f8066690 0x7f24f8068b50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f250c0045c0 tx=0x7f250c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:30.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:30.863+0000 7f2513400640 1 -- 172.21.15.161:0/609271624 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2504051d50 con 0x7f251c143c40 2024-05-22T21:19:31.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.023+0000 7f2521600640 1 -- 172.21.15.161:0/609271624 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f251c1578c0 con 0x7f251c143c40 2024-05-22T21:19:31.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.027+0000 7f2513400640 1 -- 172.21.15.161:0/609271624 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v128) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2504055370 con 0x7f251c143c40 2024-05-22T21:19:31.028 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:19:31.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.027+0000 7f2521600640 1 -- 172.21.15.161:0/609271624 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f24f8066690 msgr2=0x7f24f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:31.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.027+0000 7f2521600640 1 --2- 172.21.15.161:0/609271624 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f24f8066690 0x7f24f8068b50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f250c0045c0 tx=0x7f250c009040 comp rx=0 tx=0).stop 2024-05-22T21:19:31.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.027+0000 7f2521600640 1 -- 172.21.15.161:0/609271624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c143c40 msgr2=0x7f251c153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:31.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.027+0000 7f2521600640 1 --2- 172.21.15.161:0/609271624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c143c40 0x7f251c153300 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f250400d830 tx=0x7f250400dcf0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.027+0000 7f2521600640 1 -- 172.21.15.161:0/609271624 shutdown_connections 2024-05-22T21:19:31.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.027+0000 7f2521600640 1 --2- 172.21.15.161:0/609271624 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f24f8066690 0x7f24f8068b50 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.027+0000 7f2521600640 1 --2- 172.21.15.161:0/609271624 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f251c15b600 0x7f251c153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.027+0000 7f2521600640 1 --2- 172.21.15.161:0/609271624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251c143c40 0x7f251c153300 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.027+0000 7f2521600640 1 --2- 172.21.15.161:0/609271624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f251c06c7e0 0x7f251c129500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.027+0000 7f2521600640 1 -- 172.21.15.161:0/609271624 >> 172.21.15.161:0/609271624 conn(0x7f251c06a2d0 msgr2=0x7f251c154ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.027+0000 7f2521600640 1 -- 172.21.15.161:0/609271624 shutdown_connections 2024-05-22T21:19:31.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.027+0000 7f2521600640 1 -- 172.21.15.161:0/609271624 wait complete. 2024-05-22T21:19:31.049 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-22T21:19:31.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:31.049 DEBUG:teuthology.orchestra.run.smithi179:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-22T21:19:31.078 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:19:31.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:31.078 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-22T21:19:31.129 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:19:31.130 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:31.138 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:19:31.138 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:19:31.138 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:31.181 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:19:31.181 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:31.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.319+0000 7f3425600640 1 Processor -- start 2024-05-22T21:19:31.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.319+0000 7f3425600640 1 -- start start 2024-05-22T21:19:31.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.319+0000 7f3425600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3420143ec0 0x7f342015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:31.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.319+0000 7f3425600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f342015ec90 0x7f342015f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:31.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.319+0000 7f3425600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34201442a0 con 0x7f34200715a0 2024-05-22T21:19:31.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.319+0000 7f3425600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342015f680 con 0x7f3420143ec0 2024-05-22T21:19:31.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.319+0000 7f3425600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3420070500 con 0x7f342015ec90 2024-05-22T21:19:31.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.319+0000 7f341e000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f34200715a0 0x7f342006fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47402/0 (socket says 172.21.15.161:47402) 2024-05-22T21:19:31.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.319+0000 7f341ea00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3420143ec0 0x7f342015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:31.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.319+0000 7f341e000640 1 -- 172.21.15.161:0/4134383748 learned_addr learned my addr 172.21.15.161:0/4134383748 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:31.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.323+0000 7f341f400640 1 --2- 172.21.15.161:0/4134383748 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f342015ec90 0x7f342015f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:31.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.323+0000 7f341ea00640 1 -- 172.21.15.161:0/4134383748 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f342015ec90 msgr2=0x7f342015f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:31.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.323+0000 7f341ea00640 1 --2- 172.21.15.161:0/4134383748 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f342015ec90 0x7f342015f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.323+0000 7f341ea00640 1 -- 172.21.15.161:0/4134383748 >> v1:172.21.15.161:6789/0 conn(0x7f34200715a0 legacy=0x7f342006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:31.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.323+0000 7f341ea00640 1 -- 172.21.15.161:0/4134383748 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34200707e0 con 0x7f3420143ec0 2024-05-22T21:19:31.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.323+0000 7f341d600640 1 -- 172.21.15.161:0/4134383748 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834354566 0 0) 0x7f34201442a0 con 0x7f34200715a0 2024-05-22T21:19:31.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.323+0000 7f341ea00640 1 --2- 172.21.15.161:0/4134383748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3420143ec0 0x7f342015f830 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f3410009d90 tx=0x7f341001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83cd1fa410ba25 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:31.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.323+0000 7f341d600640 1 -- 172.21.15.161:0/4134383748 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34100a5900 con 0x7f3420143ec0 2024-05-22T21:19:31.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.323+0000 7f341d600640 1 -- 172.21.15.161:0/4134383748 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34100a5a60 con 0x7f3420143ec0 2024-05-22T21:19:31.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.323+0000 7f341d600640 1 -- 172.21.15.161:0/4134383748 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34100a5df0 con 0x7f3420143ec0 2024-05-22T21:19:31.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.323+0000 7f3425600640 1 -- 172.21.15.161:0/4134383748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3420143ec0 msgr2=0x7f342015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:31.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.323+0000 7f3425600640 1 --2- 172.21.15.161:0/4134383748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3420143ec0 0x7f342015f830 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f3410009d90 tx=0x7f341001c940 comp rx=0 tx=0).stop 2024-05-22T21:19:31.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f3425600640 1 -- 172.21.15.161:0/4134383748 shutdown_connections 2024-05-22T21:19:31.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f3425600640 1 --2- 172.21.15.161:0/4134383748 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f342015ec90 0x7f342015f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f3425600640 1 --2- 172.21.15.161:0/4134383748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3420143ec0 0x7f342015f830 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f3425600640 1 -- 172.21.15.161:0/4134383748 >> 172.21.15.161:0/4134383748 conn(0x7f34200b0bc0 msgr2=0x7f34200b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:31.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f3425600640 1 -- 172.21.15.161:0/4134383748 shutdown_connections 2024-05-22T21:19:31.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f3425600640 1 -- 172.21.15.161:0/4134383748 wait complete. 2024-05-22T21:19:31.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f3425600640 1 Processor -- start 2024-05-22T21:19:31.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f3425600640 1 -- start start 2024-05-22T21:19:31.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f3425600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3420143ec0 0x7f3420138fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:31.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f3425600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f342015ec90 0x7f342013cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:31.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f3425600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3420154d60 0x7f34201551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:31.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f3425600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3420139980 con 0x7f3420143ec0 2024-05-22T21:19:31.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f3425600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342013d1e0 con 0x7f342015ec90 2024-05-22T21:19:31.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f3425600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34201396a0 con 0x7f3420154d60 2024-05-22T21:19:31.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f341ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3420143ec0 0x7f3420138fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:31.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f341ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3420143ec0 0x7f3420138fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57722/0 (socket says 172.21.15.161:57722) 2024-05-22T21:19:31.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f341ea00640 1 -- 172.21.15.161:0/1787181952 learned_addr learned my addr 172.21.15.161:0/1787181952 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:31.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f341f400640 1 --2- 172.21.15.161:0/1787181952 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3420154d60 0x7f34201551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:31.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f341e000640 1 --2- 172.21.15.161:0/1787181952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f342015ec90 0x7f342013cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:31.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f341ea00640 1 -- 172.21.15.161:0/1787181952 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3420154d60 msgr2=0x7f34201551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:31.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f341ea00640 1 --2- 172.21.15.161:0/1787181952 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3420154d60 0x7f34201551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.332 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f341ea00640 1 -- 172.21.15.161:0/1787181952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f342015ec90 msgr2=0x7f342013cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:31.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f341ea00640 1 --2- 172.21.15.161:0/1787181952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f342015ec90 0x7f342013cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f341ea00640 1 -- 172.21.15.161:0/1787181952 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3410009a70 con 0x7f3420143ec0 2024-05-22T21:19:31.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.327+0000 7f341ea00640 1 --2- 172.21.15.161:0/1787181952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3420143ec0 0x7f3420138fc0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f34100b0040 tx=0x7f34100196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:31.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.331+0000 7f3416a00640 1 -- 172.21.15.161:0/1787181952 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34100b9860 con 0x7f3420143ec0 2024-05-22T21:19:31.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.331+0000 7f3416a00640 1 -- 172.21.15.161:0/1787181952 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34100b99c0 con 0x7f3420143ec0 2024-05-22T21:19:31.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.331+0000 7f3416a00640 1 -- 172.21.15.161:0/1787181952 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34100b9d70 con 0x7f3420143ec0 2024-05-22T21:19:31.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.331+0000 7f3425600640 1 -- 172.21.15.161:0/1787181952 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342013a3c0 con 0x7f3420143ec0 2024-05-22T21:19:31.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.331+0000 7f3425600640 1 -- 172.21.15.161:0/1787181952 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342013a980 con 0x7f3420143ec0 2024-05-22T21:19:31.333 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.331+0000 7f3425600640 1 -- 172.21.15.161:0/1787181952 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342013a220 con 0x7f3420143ec0 2024-05-22T21:19:31.334 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.331+0000 7f3416a00640 1 -- 172.21.15.161:0/1787181952 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f34100c4460 con 0x7f3420143ec0 2024-05-22T21:19:31.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.331+0000 7f3416a00640 1 --2- 172.21.15.161:0/1787181952 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3400066690 0x7f3400068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:31.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.331+0000 7f3416a00640 1 -- 172.21.15.161:0/1787181952 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f341011f7a0 con 0x7f3420143ec0 2024-05-22T21:19:31.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.331+0000 7f341e000640 1 --2- 172.21.15.161:0/1787181952 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3400066690 0x7f3400068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:31.335 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.331+0000 7f341e000640 1 --2- 172.21.15.161:0/1787181952 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3400066690 0x7f3400068b50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3420155e20 tx=0x7f340c007b60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:31.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.335+0000 7f3416a00640 1 -- 172.21.15.161:0/1787181952 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f342013a220 con 0x7f3420143ec0 2024-05-22T21:19:31.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.531+0000 7f3425600640 1 -- 172.21.15.161:0/1787181952 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f342013a220 con 0x7f3420143ec0 2024-05-22T21:19:31.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.531+0000 7f3416a00640 1 -- 172.21.15.161:0/1787181952 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f342013a220 con 0x7f3420143ec0 2024-05-22T21:19:31.532 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:31.533 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3944033857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3944033857},{"type":"v1","addr":"172.21.15.179:6834","nonce":3944033857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6125,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/4216709187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4216709187},{"type":"v1","addr":"172.21.15.161:6836","nonce":4216709187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2364917653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2364917653},{"type":"v1","addr":"172.21.15.179:6835","nonce":2364917653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:14.345936+0000","modified":"2024-05-22T21:19:17.355628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/1424863663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1424863663},{"type":"v1","addr":"172.21.15.161:6837","nonce":1424863663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-22T21:19:31.533 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 54 2024-05-22T21:19:31.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.531+0000 7f3425600640 1 -- 172.21.15.161:0/1787181952 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3400066690 msgr2=0x7f3400068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:31.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.531+0000 7f3425600640 1 --2- 172.21.15.161:0/1787181952 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3400066690 0x7f3400068b50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3420155e20 tx=0x7f340c007b60 comp rx=0 tx=0).stop 2024-05-22T21:19:31.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.531+0000 7f3425600640 1 -- 172.21.15.161:0/1787181952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3420143ec0 msgr2=0x7f3420138fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:31.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.531+0000 7f3425600640 1 --2- 172.21.15.161:0/1787181952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3420143ec0 0x7f3420138fc0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f34100b0040 tx=0x7f34100196e0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.531+0000 7f3425600640 1 -- 172.21.15.161:0/1787181952 shutdown_connections 2024-05-22T21:19:31.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.531+0000 7f3425600640 1 --2- 172.21.15.161:0/1787181952 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3400066690 0x7f3400068b50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.531+0000 7f3425600640 1 --2- 172.21.15.161:0/1787181952 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3420154d60 0x7f34201551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.531+0000 7f3425600640 1 --2- 172.21.15.161:0/1787181952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f342015ec90 0x7f342013cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.531+0000 7f3425600640 1 --2- 172.21.15.161:0/1787181952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3420143ec0 0x7f3420138fc0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.531+0000 7f3425600640 1 -- 172.21.15.161:0/1787181952 >> 172.21.15.161:0/1787181952 conn(0x7f34200b0bc0 msgr2=0x7f3420140de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:31.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.531+0000 7f3425600640 1 -- 172.21.15.161:0/1787181952 shutdown_connections 2024-05-22T21:19:31.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.531+0000 7f3425600640 1 -- 172.21.15.161:0/1787181952 wait complete. 2024-05-22T21:19:31.554 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:31.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.687+0000 7fd61b200640 1 Processor -- start 2024-05-22T21:19:31.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.687+0000 7fd61b200640 1 -- start start 2024-05-22T21:19:31.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.687+0000 7fd61b200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd61415dd50 0x7fd61406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:31.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.687+0000 7fd61b200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd61415d5e0 0x7fd61415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:31.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.687+0000 7fd61b200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd614071670 con 0x7fd61415e130 2024-05-22T21:19:31.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.687+0000 7fd61b200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61406de40 con 0x7fd61415d5e0 2024-05-22T21:19:31.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.687+0000 7fd61b200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd614070a60 con 0x7fd61415dd50 2024-05-22T21:19:31.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.687+0000 7fd60b400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd61415e130 0x7fd61415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47408/0 (socket says 172.21.15.161:47408) 2024-05-22T21:19:31.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.687+0000 7fd60b400640 1 -- 172.21.15.161:0/3223996744 learned_addr learned my addr 172.21.15.161:0/3223996744 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:31.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.687+0000 7fd619200640 1 --2- 172.21.15.161:0/3223996744 >> v2:172.21.15.161:3300/0 conn(0x7fd61415d5e0 0x7fd61415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:31.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.687+0000 7fd613e00640 1 --2- 172.21.15.161:0/3223996744 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd61415dd50 0x7fd61406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:31.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.687+0000 7fd619200640 1 -- 172.21.15.161:0/3223996744 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd61415dd50 msgr2=0x7fd61406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:31.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.687+0000 7fd619200640 1 --2- 172.21.15.161:0/3223996744 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd61415dd50 0x7fd61406d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.687+0000 7fd619200640 1 -- 172.21.15.161:0/3223996744 >> v1:172.21.15.161:6789/0 conn(0x7fd61415e130 legacy=0x7fd61415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:31.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.687+0000 7fd613400640 1 -- 172.21.15.161:0/3223996744 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2281594574 0 0) 0x7fd614071670 con 0x7fd61415e130 2024-05-22T21:19:31.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.687+0000 7fd619200640 1 -- 172.21.15.161:0/3223996744 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd614070d40 con 0x7fd61415d5e0 2024-05-22T21:19:31.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.691+0000 7fd619200640 1 --2- 172.21.15.161:0/3223996744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd61415d5e0 0x7fd61415b6c0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009d90 tx=0x7fd60c01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5a0e9d3b83e759 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:31.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.691+0000 7fd613400640 1 -- 172.21.15.161:0/3223996744 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c0a5970 con 0x7fd61415d5e0 2024-05-22T21:19:31.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.691+0000 7fd613400640 1 -- 172.21.15.161:0/3223996744 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c0a5ad0 con 0x7fd61415d5e0 2024-05-22T21:19:31.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.691+0000 7fd613400640 1 -- 172.21.15.161:0/3223996744 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c0a5e60 con 0x7fd61415d5e0 2024-05-22T21:19:31.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.691+0000 7fd61b200640 1 -- 172.21.15.161:0/3223996744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd61415d5e0 msgr2=0x7fd61415b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:31.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.691+0000 7fd61b200640 1 --2- 172.21.15.161:0/3223996744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd61415d5e0 0x7fd61415b6c0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009d90 tx=0x7fd60c01c8b0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.691+0000 7fd61b200640 1 -- 172.21.15.161:0/3223996744 shutdown_connections 2024-05-22T21:19:31.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.691+0000 7fd61b200640 1 --2- 172.21.15.161:0/3223996744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd61415d5e0 0x7fd61415b6c0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.691+0000 7fd61b200640 1 --2- 172.21.15.161:0/3223996744 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd61415dd50 0x7fd61406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.691+0000 7fd61b200640 1 -- 172.21.15.161:0/3223996744 >> 172.21.15.161:0/3223996744 conn(0x7fd6140b0bc0 msgr2=0x7fd6140b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:31.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.691+0000 7fd61b200640 1 -- 172.21.15.161:0/3223996744 shutdown_connections 2024-05-22T21:19:31.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.691+0000 7fd61b200640 1 -- 172.21.15.161:0/3223996744 wait complete. 2024-05-22T21:19:31.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd61b200640 1 Processor -- start 2024-05-22T21:19:31.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd61b200640 1 -- start start 2024-05-22T21:19:31.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd61b200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd61415d5e0 0x7fd614161350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:31.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd61b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd61415dd50 0x7fd614161890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:31.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd61b200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd61415e130 0x7fd6141f9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:31.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd61b200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6140b3f20 con 0x7fd61415dd50 2024-05-22T21:19:31.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd61b200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61415fe80 con 0x7fd61415d5e0 2024-05-22T21:19:31.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd61b200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61415eb00 con 0x7fd61415e130 2024-05-22T21:19:31.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd60b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd61415dd50 0x7fd614161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:31.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd60b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd61415dd50 0x7fd614161890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57736/0 (socket says 172.21.15.161:57736) 2024-05-22T21:19:31.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd60b400640 1 -- 172.21.15.161:0/2516374577 learned_addr learned my addr 172.21.15.161:0/2516374577 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:31.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd613e00640 1 --2- 172.21.15.161:0/2516374577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd61415d5e0 0x7fd614161350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:31.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd619200640 1 --2- 172.21.15.161:0/2516374577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd61415e130 0x7fd6141f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:31.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd60b400640 1 -- 172.21.15.161:0/2516374577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd61415e130 msgr2=0x7fd6141f9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:31.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd60b400640 1 --2- 172.21.15.161:0/2516374577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd61415e130 0x7fd6141f9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd60b400640 1 -- 172.21.15.161:0/2516374577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd61415d5e0 msgr2=0x7fd614161350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:31.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd60b400640 1 --2- 172.21.15.161:0/2516374577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd61415d5e0 0x7fd614161350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd60b400640 1 -- 172.21.15.161:0/2516374577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60c009a70 con 0x7fd61415dd50 2024-05-22T21:19:31.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd60b400640 1 --2- 172.21.15.161:0/2516374577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd61415dd50 0x7fd614161890 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00c480 tx=0x7fd5fc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:31.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd610c00640 1 -- 172.21.15.161:0/2516374577 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc0074e0 con 0x7fd61415dd50 2024-05-22T21:19:31.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd610c00640 1 -- 172.21.15.161:0/2516374577 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5fc007660 con 0x7fd61415dd50 2024-05-22T21:19:31.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd610c00640 1 -- 172.21.15.161:0/2516374577 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc004a30 con 0x7fd61415dd50 2024-05-22T21:19:31.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd61b200640 1 -- 172.21.15.161:0/2516374577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61415ede0 con 0x7fd61415dd50 2024-05-22T21:19:31.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.695+0000 7fd61b200640 1 -- 172.21.15.161:0/2516374577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61415f3a0 con 0x7fd61415dd50 2024-05-22T21:19:31.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.699+0000 7fd61b200640 1 -- 172.21.15.161:0/2516374577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61406dcb0 con 0x7fd61415dd50 2024-05-22T21:19:31.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.699+0000 7fd610c00640 1 -- 172.21.15.161:0/2516374577 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd5fc004b90 con 0x7fd61415dd50 2024-05-22T21:19:31.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.699+0000 7fd610c00640 1 --2- 172.21.15.161:0/2516374577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd5f0066690 0x7fd5f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:31.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.699+0000 7fd610c00640 1 -- 172.21.15.161:0/2516374577 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5fc0832c0 con 0x7fd61415dd50 2024-05-22T21:19:31.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.699+0000 7fd613e00640 1 --2- 172.21.15.161:0/2516374577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd5f0066690 0x7fd5f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:31.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.699+0000 7fd613e00640 1 --2- 172.21.15.161:0/2516374577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd5f0066690 0x7fd5f0068b50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd604009af0 tx=0x7fd604008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:31.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.703+0000 7fd610c00640 1 -- 172.21.15.161:0/2516374577 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd5fc054150 con 0x7fd61415dd50 2024-05-22T21:19:31.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.859+0000 7fd61b200640 1 -- 172.21.15.161:0/2516374577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd614067220 con 0x7fd61415dd50 2024-05-22T21:19:31.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.859+0000 7fd610c00640 1 -- 172.21.15.161:0/2516374577 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16107 (secure 0 0 0) 0x7fd5fc053ce0 con 0x7fd61415dd50 2024-05-22T21:19:31.861 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:31.861 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":128,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:19:25.399186+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:12.848545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:13.344629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:31.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.859+0000 7fd61b200640 1 -- 172.21.15.161:0/2516374577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd5f0066690 msgr2=0x7fd5f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:31.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.859+0000 7fd61b200640 1 --2- 172.21.15.161:0/2516374577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd5f0066690 0x7fd5f0068b50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd604009af0 tx=0x7fd604008040 comp rx=0 tx=0).stop 2024-05-22T21:19:31.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.859+0000 7fd61b200640 1 -- 172.21.15.161:0/2516374577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd61415dd50 msgr2=0x7fd614161890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:31.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.859+0000 7fd61b200640 1 --2- 172.21.15.161:0/2516374577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd61415dd50 0x7fd614161890 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00c480 tx=0x7fd5fc00c940 comp rx=0 tx=0).stop 2024-05-22T21:19:31.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.859+0000 7fd61b200640 1 -- 172.21.15.161:0/2516374577 shutdown_connections 2024-05-22T21:19:31.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.859+0000 7fd61b200640 1 --2- 172.21.15.161:0/2516374577 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd5f0066690 0x7fd5f0068b50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.859+0000 7fd61b200640 1 --2- 172.21.15.161:0/2516374577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd61415e130 0x7fd6141f9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.859+0000 7fd61b200640 1 --2- 172.21.15.161:0/2516374577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd61415dd50 0x7fd614161890 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.859+0000 7fd61b200640 1 --2- 172.21.15.161:0/2516374577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd61415d5e0 0x7fd614161350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:31.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.859+0000 7fd61b200640 1 -- 172.21.15.161:0/2516374577 >> 172.21.15.161:0/2516374577 conn(0x7fd6140b0bc0 msgr2=0x7fd6140af930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:31.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.859+0000 7fd61b200640 1 -- 172.21.15.161:0/2516374577 shutdown_connections 2024-05-22T21:19:31.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:31.859+0000 7fd61b200640 1 -- 172.21.15.161:0/2516374577 wait complete. 2024-05-22T21:19:31.882 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:19:12.848545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:19:13.344629+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:19:31.883 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:19:32.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.019+0000 7fc3d7600640 1 Processor -- start 2024-05-22T21:19:32.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.019+0000 7fc3d7600640 1 -- start start 2024-05-22T21:19:32.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.019+0000 7fc3d7600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc3d015b600 0x7fc3d006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.019+0000 7fc3d7600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc3d006c7e0 0x7fc3d00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.019+0000 7fc3d7600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d006cc40 con 0x7fc3d0143c40 2024-05-22T21:19:32.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.019+0000 7fc3d7600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d015ba60 con 0x7fc3d006c7e0 2024-05-22T21:19:32.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.019+0000 7fc3d7600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d0144bb0 con 0x7fc3d015b600 2024-05-22T21:19:32.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.019+0000 7fc3d5600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc3d006c7e0 0x7fc3d00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.019+0000 7fc3d5600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc3d006c7e0 0x7fc3d00b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57738/0 (socket says 172.21.15.161:57738) 2024-05-22T21:19:32.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.019+0000 7fc3d5600640 1 -- 172.21.15.161:0/1951973660 learned_addr learned my addr 172.21.15.161:0/1951973660 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:32.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.019+0000 7fc3cfe00640 1 --2- 172.21.15.161:0/1951973660 >> v2:172.21.15.179:3300/0 conn(0x7fc3d015b600 0x7fc3d006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.019+0000 7fc3d5600640 1 -- 172.21.15.161:0/1951973660 >> v2:172.21.15.179:3300/0 conn(0x7fc3d015b600 msgr2=0x7fc3d006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:32.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.019+0000 7fc3d5600640 1 --2- 172.21.15.161:0/1951973660 >> v2:172.21.15.179:3300/0 conn(0x7fc3d015b600 0x7fc3d006c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.019+0000 7fc3d5600640 1 -- 172.21.15.161:0/1951973660 >> v1:172.21.15.179:6789/0 conn(0x7fc3d0143c40 legacy=0x7fc3d01444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:32.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.019+0000 7fc3d5600640 1 -- 172.21.15.161:0/1951973660 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d015efb0 con 0x7fc3d006c7e0 2024-05-22T21:19:32.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.023+0000 7fc3d5600640 1 --2- 172.21.15.161:0/1951973660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3d006c7e0 0x7fc3d00b2160 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc3d00b1c80 tx=0x7fc3c801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23974eb3b7034231 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:32.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.023+0000 7fc3cf400640 1 -- 172.21.15.161:0/1951973660 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80a5870 con 0x7fc3d006c7e0 2024-05-22T21:19:32.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.023+0000 7fc3cf400640 1 -- 172.21.15.161:0/1951973660 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c80a59d0 con 0x7fc3d006c7e0 2024-05-22T21:19:32.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.023+0000 7fc3cf400640 1 -- 172.21.15.161:0/1951973660 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80a5d60 con 0x7fc3d006c7e0 2024-05-22T21:19:32.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.023+0000 7fc3d7600640 1 -- 172.21.15.161:0/1951973660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3d006c7e0 msgr2=0x7fc3d00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.023+0000 7fc3d7600640 1 --2- 172.21.15.161:0/1951973660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3d006c7e0 0x7fc3d00b2160 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc3d00b1c80 tx=0x7fc3c801c9b0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.023+0000 7fc3d7600640 1 -- 172.21.15.161:0/1951973660 shutdown_connections 2024-05-22T21:19:32.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.023+0000 7fc3d7600640 1 --2- 172.21.15.161:0/1951973660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3d006c7e0 0x7fc3d00b2160 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.023+0000 7fc3d7600640 1 --2- 172.21.15.161:0/1951973660 >> v2:172.21.15.179:3300/0 conn(0x7fc3d015b600 0x7fc3d006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.023+0000 7fc3d7600640 1 -- 172.21.15.161:0/1951973660 >> 172.21.15.161:0/1951973660 conn(0x7fc3d006a2d0 msgr2=0x7fc3d006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:32.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.023+0000 7fc3d7600640 1 -- 172.21.15.161:0/1951973660 shutdown_connections 2024-05-22T21:19:32.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.023+0000 7fc3d7600640 1 -- 172.21.15.161:0/1951973660 wait complete. 2024-05-22T21:19:32.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.023+0000 7fc3d7600640 1 Processor -- start 2024-05-22T21:19:32.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.023+0000 7fc3d7600640 1 -- start start 2024-05-22T21:19:32.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3d7600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc3d006c7e0 0x7fc3d0136a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3d7600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3d0143c40 0x7fc3d0154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3d7600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d015b600 0x7fc3d00ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3d7600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d00aec00 con 0x7fc3d0143c40 2024-05-22T21:19:32.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3d7600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d0154ab0 con 0x7fc3d015b600 2024-05-22T21:19:32.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3d7600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d0154d90 con 0x7fc3d006c7e0 2024-05-22T21:19:32.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3cfe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3d0143c40 0x7fc3d0154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3cfe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3d0143c40 0x7fc3d0154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57746/0 (socket says 172.21.15.161:57746) 2024-05-22T21:19:32.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3cfe00640 1 -- 172.21.15.161:0/1603926149 learned_addr learned my addr 172.21.15.161:0/1603926149 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:32.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3d4c00640 1 --2- 172.21.15.161:0/1603926149 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc3d006c7e0 0x7fc3d0136a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3d5600640 1 --2- 172.21.15.161:0/1603926149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d015b600 0x7fc3d00ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3cfe00640 1 -- 172.21.15.161:0/1603926149 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc3d006c7e0 msgr2=0x7fc3d0136a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3cfe00640 1 --2- 172.21.15.161:0/1603926149 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc3d006c7e0 0x7fc3d0136a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3cfe00640 1 -- 172.21.15.161:0/1603926149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d015b600 msgr2=0x7fc3d00ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3cfe00640 1 --2- 172.21.15.161:0/1603926149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d015b600 0x7fc3d00ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3cfe00640 1 -- 172.21.15.161:0/1603926149 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c8009b30 con 0x7fc3d0143c40 2024-05-22T21:19:32.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3cfe00640 1 --2- 172.21.15.161:0/1603926149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3d0143c40 0x7fc3d0154260 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00d5f0 tx=0x7fc3bc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:32.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3ccc00640 1 -- 172.21.15.161:0/1603926149 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0125f0 con 0x7fc3d0143c40 2024-05-22T21:19:32.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3ccc00640 1 -- 172.21.15.161:0/1603926149 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3bc012750 con 0x7fc3d0143c40 2024-05-22T21:19:32.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3ccc00640 1 -- 172.21.15.161:0/1603926149 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc012ae0 con 0x7fc3d0143c40 2024-05-22T21:19:32.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3d7600640 1 -- 172.21.15.161:0/1603926149 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d00ae820 con 0x7fc3d0143c40 2024-05-22T21:19:32.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3d7600640 1 -- 172.21.15.161:0/1603926149 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d01547a0 con 0x7fc3d0143c40 2024-05-22T21:19:32.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3d7600640 1 -- 172.21.15.161:0/1603926149 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d0067220 con 0x7fc3d0143c40 2024-05-22T21:19:32.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.027+0000 7fc3ccc00640 1 -- 172.21.15.161:0/1603926149 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc3bc012c40 con 0x7fc3d0143c40 2024-05-22T21:19:32.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.031+0000 7fc3ccc00640 1 --2- 172.21.15.161:0/1603926149 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc3b0066690 0x7fc3b0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.031+0000 7fc3ccc00640 1 -- 172.21.15.161:0/1603926149 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3bc0844c0 con 0x7fc3d0143c40 2024-05-22T21:19:32.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.031+0000 7fc3d4c00640 1 --2- 172.21.15.161:0/1603926149 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc3b0066690 0x7fc3b0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.031+0000 7fc3d4c00640 1 --2- 172.21.15.161:0/1603926149 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc3b0066690 0x7fc3b0068b50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4007fd0 tx=0x7fc3c4007e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:32.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.035+0000 7fc3ccc00640 1 -- 172.21.15.161:0/1603926149 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc3bc051970 con 0x7fc3d0143c40 2024-05-22T21:19:32.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.187+0000 7fc3d7600640 1 -- 172.21.15.161:0/1603926149 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3d00ceaa0 con 0x7fc3d0143c40 2024-05-22T21:19:32.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.187+0000 7fc3ccc00640 1 -- 172.21.15.161:0/1603926149 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc3bc007380 con 0x7fc3d0143c40 2024-05-22T21:19:32.191 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:19:32.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.191+0000 7fc3d7600640 1 -- 172.21.15.161:0/1603926149 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc3b0066690 msgr2=0x7fc3b0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.191+0000 7fc3d7600640 1 --2- 172.21.15.161:0/1603926149 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc3b0066690 0x7fc3b0068b50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4007fd0 tx=0x7fc3c4007e40 comp rx=0 tx=0).stop 2024-05-22T21:19:32.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.191+0000 7fc3d7600640 1 -- 172.21.15.161:0/1603926149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3d0143c40 msgr2=0x7fc3d0154260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.191+0000 7fc3d7600640 1 --2- 172.21.15.161:0/1603926149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3d0143c40 0x7fc3d0154260 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00d5f0 tx=0x7fc3bc00dab0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.191+0000 7fc3d7600640 1 -- 172.21.15.161:0/1603926149 shutdown_connections 2024-05-22T21:19:32.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.191+0000 7fc3d7600640 1 --2- 172.21.15.161:0/1603926149 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc3b0066690 0x7fc3b0068b50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.191+0000 7fc3d7600640 1 --2- 172.21.15.161:0/1603926149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3d015b600 0x7fc3d00ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.191+0000 7fc3d7600640 1 --2- 172.21.15.161:0/1603926149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3d0143c40 0x7fc3d0154260 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.191+0000 7fc3d7600640 1 --2- 172.21.15.161:0/1603926149 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc3d006c7e0 0x7fc3d0136a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.191+0000 7fc3d7600640 1 -- 172.21.15.161:0/1603926149 >> 172.21.15.161:0/1603926149 conn(0x7fc3d006a2d0 msgr2=0x7fc3d006fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:32.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.191+0000 7fc3d7600640 1 -- 172.21.15.161:0/1603926149 shutdown_connections 2024-05-22T21:19:32.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.191+0000 7fc3d7600640 1 -- 172.21.15.161:0/1603926149 wait complete. 2024-05-22T21:19:32.212 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:19:32.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.347+0000 7eff99c00640 1 Processor -- start 2024-05-22T21:19:32.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.347+0000 7eff99c00640 1 -- start start 2024-05-22T21:19:32.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.347+0000 7eff99c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7eff94143c40 0x7eff9415b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.347+0000 7eff99c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff9415efb0 con 0x7eff9406c9b0 2024-05-22T21:19:32.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.347+0000 7eff99c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff9415f290 con 0x7eff941444a0 2024-05-22T21:19:32.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.347+0000 7eff99c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9415f570 con 0x7eff94143c40 2024-05-22T21:19:32.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.347+0000 7eff93400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7eff9406c9b0 0x7eff940b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47416/0 (socket says 172.21.15.161:47416) 2024-05-22T21:19:32.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.347+0000 7eff92a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7eff94143c40 0x7eff9415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.347+0000 7eff93400640 1 -- 172.21.15.161:0/981513255 learned_addr learned my addr 172.21.15.161:0/981513255 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:32.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.347+0000 7eff91600640 1 -- 172.21.15.161:0/981513255 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3973638911 0 0) 0x7eff9415efb0 con 0x7eff9406c9b0 2024-05-22T21:19:32.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.347+0000 7eff92a00640 1 -- 172.21.15.161:0/981513255 >> v1:172.21.15.179:6789/0 conn(0x7eff941444a0 legacy=0x7eff9406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.347+0000 7eff92a00640 1 -- 172.21.15.161:0/981513255 >> v1:172.21.15.161:6789/0 conn(0x7eff9406c9b0 legacy=0x7eff940b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.347+0000 7eff92a00640 1 -- 172.21.15.161:0/981513255 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9415f850 con 0x7eff94143c40 2024-05-22T21:19:32.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.351+0000 7eff92a00640 1 --2- 172.21.15.161:0/981513255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff94143c40 0x7eff9415b600 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7eff840097c0 tx=0x7eff8401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ae89de8b037c57 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:32.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.351+0000 7eff91600640 1 -- 172.21.15.161:0/981513255 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff840ab600 con 0x7eff94143c40 2024-05-22T21:19:32.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.351+0000 7eff91600640 1 -- 172.21.15.161:0/981513255 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff840ab760 con 0x7eff94143c40 2024-05-22T21:19:32.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.351+0000 7eff91600640 1 -- 172.21.15.161:0/981513255 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff840abaf0 con 0x7eff94143c40 2024-05-22T21:19:32.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.351+0000 7eff99c00640 1 -- 172.21.15.161:0/981513255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff94143c40 msgr2=0x7eff9415b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.351+0000 7eff99c00640 1 --2- 172.21.15.161:0/981513255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff94143c40 0x7eff9415b600 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7eff840097c0 tx=0x7eff8401c4a0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.351+0000 7eff99c00640 1 -- 172.21.15.161:0/981513255 shutdown_connections 2024-05-22T21:19:32.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.351+0000 7eff99c00640 1 --2- 172.21.15.161:0/981513255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff94143c40 0x7eff9415b600 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.351+0000 7eff99c00640 1 -- 172.21.15.161:0/981513255 >> 172.21.15.161:0/981513255 conn(0x7eff9406a2d0 msgr2=0x7eff9406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:32.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.351+0000 7eff99c00640 1 -- 172.21.15.161:0/981513255 shutdown_connections 2024-05-22T21:19:32.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.351+0000 7eff99c00640 1 -- 172.21.15.161:0/981513255 wait complete. 2024-05-22T21:19:32.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff99c00640 1 Processor -- start 2024-05-22T21:19:32.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff99c00640 1 -- start start 2024-05-22T21:19:32.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff99c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff9406c9b0 0x7eff9415e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff99c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7eff94143c40 0x7eff941525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff99c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff941444a0 0x7eff941f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff99c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff94157810 con 0x7eff941444a0 2024-05-22T21:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff99c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff94156b90 con 0x7eff9406c9b0 2024-05-22T21:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff99c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff94156e70 con 0x7eff94143c40 2024-05-22T21:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff93400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff941444a0 0x7eff941f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff93400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff941444a0 0x7eff941f64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57758/0 (socket says 172.21.15.161:57758) 2024-05-22T21:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff93400640 1 -- 172.21.15.161:0/1913482381 learned_addr learned my addr 172.21.15.161:0/1913482381 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff92a00640 1 --2- 172.21.15.161:0/1913482381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff9406c9b0 0x7eff9415e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff92000640 1 --2- 172.21.15.161:0/1913482381 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7eff94143c40 0x7eff941525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff93400640 1 -- 172.21.15.161:0/1913482381 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7eff94143c40 msgr2=0x7eff941525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff93400640 1 --2- 172.21.15.161:0/1913482381 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7eff94143c40 0x7eff941525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff93400640 1 -- 172.21.15.161:0/1913482381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff9406c9b0 msgr2=0x7eff9415e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff93400640 1 --2- 172.21.15.161:0/1913482381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff9406c9b0 0x7eff9415e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff93400640 1 -- 172.21.15.161:0/1913482381 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff84009420 con 0x7eff941444a0 2024-05-22T21:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff93400640 1 --2- 172.21.15.161:0/1913482381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff941444a0 0x7eff941f64c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7eff8c001570 tx=0x7eff8c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff8aa00640 1 -- 172.21.15.161:0/1913482381 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00a640 con 0x7eff941444a0 2024-05-22T21:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff8aa00640 1 -- 172.21.15.161:0/1913482381 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c00a7c0 con 0x7eff941444a0 2024-05-22T21:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff8aa00640 1 -- 172.21.15.161:0/1913482381 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00ab70 con 0x7eff941444a0 2024-05-22T21:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff99c00640 1 -- 172.21.15.161:0/1913482381 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff94157150 con 0x7eff941444a0 2024-05-22T21:19:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff99c00640 1 -- 172.21.15.161:0/1913482381 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff94152af0 con 0x7eff941444a0 2024-05-22T21:19:32.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.355+0000 7eff99c00640 1 -- 172.21.15.161:0/1913482381 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff94157950 con 0x7eff941444a0 2024-05-22T21:19:32.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.359+0000 7eff8aa00640 1 -- 172.21.15.161:0/1913482381 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7eff8c00acd0 con 0x7eff941444a0 2024-05-22T21:19:32.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.359+0000 7eff8aa00640 1 --2- 172.21.15.161:0/1913482381 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7eff74066690 0x7eff74068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.359+0000 7eff8aa00640 1 -- 172.21.15.161:0/1913482381 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff8c083640 con 0x7eff941444a0 2024-05-22T21:19:32.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.359+0000 7eff92a00640 1 --2- 172.21.15.161:0/1913482381 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7eff74066690 0x7eff74068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.359+0000 7eff92a00640 1 --2- 172.21.15.161:0/1913482381 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7eff74066690 0x7eff74068b50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7eff840af000 tx=0x7eff840a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:32.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.363+0000 7eff8aa00640 1 -- 172.21.15.161:0/1913482381 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7eff94157950 con 0x7eff941444a0 2024-05-22T21:19:32.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.519+0000 7eff99c00640 1 -- 172.21.15.161:0/1913482381 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff94157950 con 0x7eff941444a0 2024-05-22T21:19:32.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.519+0000 7eff8aa00640 1 -- 172.21.15.161:0/1913482381 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff94157950 con 0x7eff941444a0 2024-05-22T21:19:32.520 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:19:32.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.519+0000 7eff99c00640 1 -- 172.21.15.161:0/1913482381 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7eff74066690 msgr2=0x7eff74068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.519+0000 7eff99c00640 1 --2- 172.21.15.161:0/1913482381 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7eff74066690 0x7eff74068b50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7eff840af000 tx=0x7eff840a7000 comp rx=0 tx=0).stop 2024-05-22T21:19:32.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.519+0000 7eff99c00640 1 -- 172.21.15.161:0/1913482381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff941444a0 msgr2=0x7eff941f64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.519+0000 7eff99c00640 1 --2- 172.21.15.161:0/1913482381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff941444a0 0x7eff941f64c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7eff8c001570 tx=0x7eff8c00cca0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.519+0000 7eff99c00640 1 -- 172.21.15.161:0/1913482381 shutdown_connections 2024-05-22T21:19:32.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.519+0000 7eff99c00640 1 --2- 172.21.15.161:0/1913482381 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7eff74066690 0x7eff74068b50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.519+0000 7eff99c00640 1 --2- 172.21.15.161:0/1913482381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff941444a0 0x7eff941f64c0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.519+0000 7eff99c00640 1 --2- 172.21.15.161:0/1913482381 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7eff94143c40 0x7eff941525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.519+0000 7eff99c00640 1 --2- 172.21.15.161:0/1913482381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff9406c9b0 0x7eff9415e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.519+0000 7eff99c00640 1 -- 172.21.15.161:0/1913482381 >> 172.21.15.161:0/1913482381 conn(0x7eff9406a2d0 msgr2=0x7eff9406a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:32.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.519+0000 7eff99c00640 1 -- 172.21.15.161:0/1913482381 shutdown_connections 2024-05-22T21:19:32.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.519+0000 7eff99c00640 1 -- 172.21.15.161:0/1913482381 wait complete. 2024-05-22T21:19:32.541 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:19:32.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.675+0000 7f49b7600640 1 Processor -- start 2024-05-22T21:19:32.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.675+0000 7f49b7600640 1 -- start start 2024-05-22T21:19:32.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.679+0000 7f49b7600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49b01402e0 0x7f49b013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.679+0000 7f49b7600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f49b013fc80 0x7f49b006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.679+0000 7f49b7600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b01400d0 con 0x7f49b013cfc0 2024-05-22T21:19:32.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.679+0000 7f49b7600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b013d3a0 con 0x7f49b013fc80 2024-05-22T21:19:32.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.679+0000 7f49b7600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b00cd920 con 0x7f49b01402e0 2024-05-22T21:19:32.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.679+0000 7f49b5600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f49b013fc80 0x7f49b006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.679+0000 7f49b5600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f49b013fc80 0x7f49b006fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57774/0 (socket says 172.21.15.161:57774) 2024-05-22T21:19:32.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.679+0000 7f49b5600640 1 -- 172.21.15.161:0/4254521282 learned_addr learned my addr 172.21.15.161:0/4254521282 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:32.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.679+0000 7f49a7e00640 1 --2- 172.21.15.161:0/4254521282 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49b01402e0 0x7f49b013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.679+0000 7f49b5600640 1 -- 172.21.15.161:0/4254521282 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49b01402e0 msgr2=0x7f49b013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:32.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.679+0000 7f49b5600640 1 --2- 172.21.15.161:0/4254521282 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49b01402e0 0x7f49b013f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.679+0000 7f49b5600640 1 -- 172.21.15.161:0/4254521282 >> v1:172.21.15.179:6789/0 conn(0x7f49b013cfc0 legacy=0x7f49b015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.679+0000 7f49b5600640 1 -- 172.21.15.161:0/4254521282 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b00cdc00 con 0x7f49b013fc80 2024-05-22T21:19:32.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.683+0000 7f49b5600640 1 --2- 172.21.15.161:0/4254521282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49b013fc80 0x7f49b006fce0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009e50 tx=0x7f49a801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f408e8631987880 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:32.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.683+0000 7f49afe00640 1 -- 172.21.15.161:0/4254521282 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a80a9480 con 0x7f49b013fc80 2024-05-22T21:19:32.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.683+0000 7f49afe00640 1 -- 172.21.15.161:0/4254521282 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a80a95e0 con 0x7f49b013fc80 2024-05-22T21:19:32.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.683+0000 7f49afe00640 1 -- 172.21.15.161:0/4254521282 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a80a9970 con 0x7f49b013fc80 2024-05-22T21:19:32.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.683+0000 7f49b7600640 1 -- 172.21.15.161:0/4254521282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49b013fc80 msgr2=0x7f49b006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.683+0000 7f49b7600640 1 --2- 172.21.15.161:0/4254521282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49b013fc80 0x7f49b006fce0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009e50 tx=0x7f49a801c970 comp rx=0 tx=0).stop 2024-05-22T21:19:32.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.683+0000 7f49b7600640 1 -- 172.21.15.161:0/4254521282 shutdown_connections 2024-05-22T21:19:32.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.683+0000 7f49b7600640 1 --2- 172.21.15.161:0/4254521282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49b013fc80 0x7f49b006fce0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.683+0000 7f49b7600640 1 --2- 172.21.15.161:0/4254521282 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49b01402e0 0x7f49b013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.683+0000 7f49b7600640 1 -- 172.21.15.161:0/4254521282 >> 172.21.15.161:0/4254521282 conn(0x7f49b006a2d0 msgr2=0x7f49b006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:32.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.683+0000 7f49b7600640 1 -- 172.21.15.161:0/4254521282 shutdown_connections 2024-05-22T21:19:32.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.683+0000 7f49b7600640 1 -- 172.21.15.161:0/4254521282 wait complete. 2024-05-22T21:19:32.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.683+0000 7f49b7600640 1 Processor -- start 2024-05-22T21:19:32.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.683+0000 7f49b7600640 1 -- start start 2024-05-22T21:19:32.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49b7600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49b013cfc0 0x7f49b01f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49b7600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49b013fc80 0x7f49b013aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49b7600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49b01402e0 0x7f49b0141880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49b7600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b0142460 con 0x7f49b013fc80 2024-05-22T21:19:32.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49b7600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b01f64c0 con 0x7f49b013cfc0 2024-05-22T21:19:32.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49b7600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b0142670 con 0x7f49b01402e0 2024-05-22T21:19:32.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49a7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49b013fc80 0x7f49b013aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49a7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49b013fc80 0x7f49b013aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57784/0 (socket says 172.21.15.161:57784) 2024-05-22T21:19:32.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49a7e00640 1 -- 172.21.15.161:0/1098860018 learned_addr learned my addr 172.21.15.161:0/1098860018 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:32.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49b4c00640 1 --2- 172.21.15.161:0/1098860018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49b013cfc0 0x7f49b01f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49b5600640 1 --2- 172.21.15.161:0/1098860018 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49b01402e0 0x7f49b0141880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49a7e00640 1 -- 172.21.15.161:0/1098860018 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49b01402e0 msgr2=0x7f49b0141880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49a7e00640 1 --2- 172.21.15.161:0/1098860018 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49b01402e0 0x7f49b0141880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49a7e00640 1 -- 172.21.15.161:0/1098860018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49b013cfc0 msgr2=0x7f49b01f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49a7e00640 1 --2- 172.21.15.161:0/1098860018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49b013cfc0 0x7f49b01f5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49a7e00640 1 -- 172.21.15.161:0/1098860018 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a8009b30 con 0x7f49b013fc80 2024-05-22T21:19:32.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49a7e00640 1 --2- 172.21.15.161:0/1098860018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49b013fc80 0x7f49b013aa30 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f499800d830 tx=0x7f499800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:32.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49ad600640 1 -- 172.21.15.161:0/1098860018 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499800b6c0 con 0x7f49b013fc80 2024-05-22T21:19:32.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49ad600640 1 -- 172.21.15.161:0/1098860018 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499800b860 con 0x7f49b013fc80 2024-05-22T21:19:32.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49ad600640 1 -- 172.21.15.161:0/1098860018 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499800bc10 con 0x7f49b013fc80 2024-05-22T21:19:32.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49b7600640 1 -- 172.21.15.161:0/1098860018 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b0142950 con 0x7f49b013fc80 2024-05-22T21:19:32.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49b7600640 1 -- 172.21.15.161:0/1098860018 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b013af70 con 0x7f49b013fc80 2024-05-22T21:19:32.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49b7600640 1 -- 172.21.15.161:0/1098860018 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b0067220 con 0x7f49b013fc80 2024-05-22T21:19:32.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.687+0000 7f49ad600640 1 -- 172.21.15.161:0/1098860018 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f499801e460 con 0x7f49b013fc80 2024-05-22T21:19:32.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.691+0000 7f49ad600640 1 --2- 172.21.15.161:0/1098860018 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f498c066690 0x7f498c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:32.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.691+0000 7f49ad600640 1 -- 172.21.15.161:0/1098860018 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4998084910 con 0x7f49b013fc80 2024-05-22T21:19:32.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.691+0000 7f49b4c00640 1 --2- 172.21.15.161:0/1098860018 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f498c066690 0x7f498c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:32.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.691+0000 7f49b4c00640 1 --2- 172.21.15.161:0/1098860018 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f498c066690 0x7f498c068b50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f49a0007fd0 tx=0x7f49a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:32.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.695+0000 7f49ad600640 1 -- 172.21.15.161:0/1098860018 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4998051c90 con 0x7f49b013fc80 2024-05-22T21:19:32.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.843+0000 7f49b7600640 1 -- 172.21.15.161:0/1098860018 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f49b0071670 con 0x7f49b013fc80 2024-05-22T21:19:32.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.847+0000 7f49ad600640 1 -- 172.21.15.161:0/1098860018 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) v1 ==== 100+0+11 (secure 0 0 0) 0x7f49980552b0 con 0x7f49b013fc80 2024-05-22T21:19:32.848 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:19:32.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.847+0000 7f49b7600640 1 -- 172.21.15.161:0/1098860018 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f498c066690 msgr2=0x7f498c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.847+0000 7f49b7600640 1 --2- 172.21.15.161:0/1098860018 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f498c066690 0x7f498c068b50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f49a0007fd0 tx=0x7f49a0009040 comp rx=0 tx=0).stop 2024-05-22T21:19:32.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.847+0000 7f49b7600640 1 -- 172.21.15.161:0/1098860018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49b013fc80 msgr2=0x7f49b013aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:32.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.847+0000 7f49b7600640 1 --2- 172.21.15.161:0/1098860018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49b013fc80 0x7f49b013aa30 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f499800d830 tx=0x7f499800dcf0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.847+0000 7f49b7600640 1 -- 172.21.15.161:0/1098860018 shutdown_connections 2024-05-22T21:19:32.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.847+0000 7f49b7600640 1 --2- 172.21.15.161:0/1098860018 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f498c066690 0x7f498c068b50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.847+0000 7f49b7600640 1 --2- 172.21.15.161:0/1098860018 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49b01402e0 0x7f49b0141880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.847+0000 7f49b7600640 1 --2- 172.21.15.161:0/1098860018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49b013fc80 0x7f49b013aa30 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.847+0000 7f49b7600640 1 --2- 172.21.15.161:0/1098860018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49b013cfc0 0x7f49b01f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:32.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.847+0000 7f49b7600640 1 -- 172.21.15.161:0/1098860018 >> 172.21.15.161:0/1098860018 conn(0x7f49b006a2d0 msgr2=0x7f49b00b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:32.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.847+0000 7f49b7600640 1 -- 172.21.15.161:0/1098860018 shutdown_connections 2024-05-22T21:19:32.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.847+0000 7f49b7600640 1 -- 172.21.15.161:0/1098860018 wait complete. 2024-05-22T21:19:32.869 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:33.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.999+0000 7f2e7e000640 1 Processor -- start 2024-05-22T21:19:33.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:32.999+0000 7f2e7e000640 1 -- start start 2024-05-22T21:19:33.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.003+0000 7f2e7e000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2e78143ec0 0x7f2e7815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.003+0000 7f2e7e000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e780715a0 0x7f2e7806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.003+0000 7f2e7e000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e781442a0 con 0x7f2e7815ec90 2024-05-22T21:19:33.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.003+0000 7f2e7e000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e78070260 con 0x7f2e78143ec0 2024-05-22T21:19:33.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.003+0000 7f2e7e000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e78070500 con 0x7f2e780715a0 2024-05-22T21:19:33.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.003+0000 7f2e6ea00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e780715a0 0x7f2e7806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.003+0000 7f2e77400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2e78143ec0 0x7f2e7815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.003+0000 7f2e77400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2e78143ec0 0x7f2e7815f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:42852/0 (socket says 172.21.15.161:42852) 2024-05-22T21:19:33.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.003+0000 7f2e6ea00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e780715a0 0x7f2e7806fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:59576/0 (socket says 172.21.15.161:59576) 2024-05-22T21:19:33.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.003+0000 7f2e77400640 1 -- 172.21.15.161:0/2319959167 learned_addr learned my addr 172.21.15.161:0/2319959167 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:33.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.003+0000 7f2e77e00640 1 --1- 172.21.15.161:0/2319959167 >> v1:172.21.15.179:6789/0 conn(0x7f2e7815ec90 0x7f2e7806bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:44762/0 (socket says 172.21.15.161:44762) 2024-05-22T21:19:33.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.003+0000 7f2e77400640 1 -- 172.21.15.161:0/2319959167 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e780715a0 msgr2=0x7f2e7806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:33.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.003+0000 7f2e77400640 1 --2- 172.21.15.161:0/2319959167 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e780715a0 0x7f2e7806fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.003+0000 7f2e77400640 1 -- 172.21.15.161:0/2319959167 >> v1:172.21.15.179:6789/0 conn(0x7f2e7815ec90 legacy=0x7f2e7806bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.003+0000 7f2e77400640 1 -- 172.21.15.161:0/2319959167 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e780707e0 con 0x7f2e78143ec0 2024-05-22T21:19:33.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.003+0000 7f2e76a00640 1 -- 172.21.15.161:0/2319959167 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1744902704 0 0) 0x7f2e781442a0 con 0x7f2e7815ec90 2024-05-22T21:19:33.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.007+0000 7f2e77400640 1 --2- 172.21.15.161:0/2319959167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e78143ec0 0x7f2e7815f830 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2e68009870 tx=0x7f2e6801c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7aed68930557bea server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:33.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.007+0000 7f2e76a00640 1 -- 172.21.15.161:0/2319959167 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e680a95f0 con 0x7f2e78143ec0 2024-05-22T21:19:33.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.007+0000 7f2e76a00640 1 -- 172.21.15.161:0/2319959167 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e680a9750 con 0x7f2e78143ec0 2024-05-22T21:19:33.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.007+0000 7f2e76a00640 1 -- 172.21.15.161:0/2319959167 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e680a9ae0 con 0x7f2e78143ec0 2024-05-22T21:19:33.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.007+0000 7f2e7e000640 1 -- 172.21.15.161:0/2319959167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e78143ec0 msgr2=0x7f2e7815f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.007+0000 7f2e7e000640 1 --2- 172.21.15.161:0/2319959167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e78143ec0 0x7f2e7815f830 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2e68009870 tx=0x7f2e6801c530 comp rx=0 tx=0).stop 2024-05-22T21:19:33.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.007+0000 7f2e7e000640 1 -- 172.21.15.161:0/2319959167 shutdown_connections 2024-05-22T21:19:33.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.007+0000 7f2e7e000640 1 --2- 172.21.15.161:0/2319959167 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e780715a0 0x7f2e7806fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.007+0000 7f2e7e000640 1 --2- 172.21.15.161:0/2319959167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e78143ec0 0x7f2e7815f830 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.007+0000 7f2e7e000640 1 -- 172.21.15.161:0/2319959167 >> 172.21.15.161:0/2319959167 conn(0x7f2e780b0bc0 msgr2=0x7f2e780b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:33.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.007+0000 7f2e7e000640 1 -- 172.21.15.161:0/2319959167 shutdown_connections 2024-05-22T21:19:33.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.007+0000 7f2e7e000640 1 -- 172.21.15.161:0/2319959167 wait complete. 2024-05-22T21:19:33.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.007+0000 7f2e7e000640 1 Processor -- start 2024-05-22T21:19:33.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.007+0000 7f2e7e000640 1 -- start start 2024-05-22T21:19:33.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e7e000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e78143ec0 0x7f2e7815f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e7e000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e7815ec90 0x7f2e78161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e7e000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e78142830 0x7f2e78161550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e7e000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e78160bf0 con 0x7f2e7815ec90 2024-05-22T21:19:33.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e7e000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7815ff70 con 0x7f2e78143ec0 2024-05-22T21:19:33.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e7e000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e78160250 con 0x7f2e78142830 2024-05-22T21:19:33.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e6ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e7815ec90 0x7f2e78161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e6ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e7815ec90 0x7f2e78161010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57796/0 (socket says 172.21.15.161:57796) 2024-05-22T21:19:33.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e77400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e78143ec0 0x7f2e7815f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e6ea00640 1 -- 172.21.15.161:0/749783723 learned_addr learned my addr 172.21.15.161:0/749783723 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:33.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e77e00640 1 --2- 172.21.15.161:0/749783723 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e78142830 0x7f2e78161550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e6ea00640 1 -- 172.21.15.161:0/749783723 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e78142830 msgr2=0x7f2e78161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e6ea00640 1 --2- 172.21.15.161:0/749783723 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e78142830 0x7f2e78161550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e6ea00640 1 -- 172.21.15.161:0/749783723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e78143ec0 msgr2=0x7f2e7815f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e6ea00640 1 --2- 172.21.15.161:0/749783723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e78143ec0 0x7f2e7815f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e6ea00640 1 -- 172.21.15.161:0/749783723 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e680094d0 con 0x7f2e7815ec90 2024-05-22T21:19:33.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e6ea00640 1 --2- 172.21.15.161:0/749783723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e7815ec90 0x7f2e78161010 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000e930 tx=0x7f2e6000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:33.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e6fe00640 1 -- 172.21.15.161:0/749783723 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60009690 con 0x7f2e7815ec90 2024-05-22T21:19:33.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e6fe00640 1 -- 172.21.15.161:0/749783723 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6000ca00 con 0x7f2e7815ec90 2024-05-22T21:19:33.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e7e000640 1 -- 172.21.15.161:0/749783723 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e78160530 con 0x7f2e7815ec90 2024-05-22T21:19:33.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e6fe00640 1 -- 172.21.15.161:0/749783723 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6000cd70 con 0x7f2e7815ec90 2024-05-22T21:19:33.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e7e000640 1 -- 172.21.15.161:0/749783723 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e7813f880 con 0x7f2e7815ec90 2024-05-22T21:19:33.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e7e000640 1 -- 172.21.15.161:0/749783723 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e780afb40 con 0x7f2e7815ec90 2024-05-22T21:19:33.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.011+0000 7f2e6fe00640 1 -- 172.21.15.161:0/749783723 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2e60013470 con 0x7f2e7815ec90 2024-05-22T21:19:33.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.015+0000 7f2e6fe00640 1 --2- 172.21.15.161:0/749783723 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2e540666c0 0x7f2e54068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.015+0000 7f2e6fe00640 1 -- 172.21.15.161:0/749783723 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e600843f0 con 0x7f2e7815ec90 2024-05-22T21:19:33.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.015+0000 7f2e77400640 1 --2- 172.21.15.161:0/749783723 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2e540666c0 0x7f2e54068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.015+0000 7f2e77400640 1 --2- 172.21.15.161:0/749783723 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2e540666c0 0x7f2e54068b80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2e680ae000 tx=0x7f2e680b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:33.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.019+0000 7f2e6fe00640 1 -- 172.21.15.161:0/749783723 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2e60051830 con 0x7f2e7815ec90 2024-05-22T21:19:33.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.207+0000 7f2e7e000640 1 -- 172.21.15.161:0/749783723 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e780cc150 con 0x7f2e7815ec90 2024-05-22T21:19:33.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.207+0000 7f2e6fe00640 1 -- 172.21.15.161:0/749783723 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f2e60054e50 con 0x7f2e7815ec90 2024-05-22T21:19:33.209 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:33.209 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3944033857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3944033857},{"type":"v1","addr":"172.21.15.179:6834","nonce":3944033857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6125,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/4216709187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4216709187},{"type":"v1","addr":"172.21.15.161:6836","nonce":4216709187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2364917653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2364917653},{"type":"v1","addr":"172.21.15.179:6835","nonce":2364917653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:14.345936+0000","modified":"2024-05-22T21:19:17.355628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/1424863663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1424863663},{"type":"v1","addr":"172.21.15.161:6837","nonce":1424863663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-22T21:19:33.209 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 54 2024-05-22T21:19:33.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.207+0000 7f2e7e000640 1 -- 172.21.15.161:0/749783723 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2e540666c0 msgr2=0x7f2e54068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.207+0000 7f2e7e000640 1 --2- 172.21.15.161:0/749783723 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2e540666c0 0x7f2e54068b80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2e680ae000 tx=0x7f2e680b3040 comp rx=0 tx=0).stop 2024-05-22T21:19:33.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.207+0000 7f2e7e000640 1 -- 172.21.15.161:0/749783723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e7815ec90 msgr2=0x7f2e78161010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.207+0000 7f2e7e000640 1 --2- 172.21.15.161:0/749783723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e7815ec90 0x7f2e78161010 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000e930 tx=0x7f2e6000edf0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.207+0000 7f2e7e000640 1 -- 172.21.15.161:0/749783723 shutdown_connections 2024-05-22T21:19:33.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.207+0000 7f2e7e000640 1 --2- 172.21.15.161:0/749783723 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2e540666c0 0x7f2e54068b80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.207+0000 7f2e7e000640 1 --2- 172.21.15.161:0/749783723 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e78142830 0x7f2e78161550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.207+0000 7f2e7e000640 1 --2- 172.21.15.161:0/749783723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e7815ec90 0x7f2e78161010 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.207+0000 7f2e7e000640 1 --2- 172.21.15.161:0/749783723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e78143ec0 0x7f2e7815f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.207+0000 7f2e7e000640 1 -- 172.21.15.161:0/749783723 >> 172.21.15.161:0/749783723 conn(0x7f2e780b0bc0 msgr2=0x7f2e7806bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:33.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.207+0000 7f2e7e000640 1 -- 172.21.15.161:0/749783723 shutdown_connections 2024-05-22T21:19:33.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.207+0000 7f2e7e000640 1 -- 172.21.15.161:0/749783723 wait complete. 2024-05-22T21:19:33.230 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:33.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.363+0000 7ff74bc00640 1 Processor -- start 2024-05-22T21:19:33.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.363+0000 7ff74bc00640 1 -- start start 2024-05-22T21:19:33.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.363+0000 7ff74bc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff74415dd50 0x7ff74406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.363+0000 7ff74bc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff74415d5e0 0x7ff74415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.363+0000 7ff74bc00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff744071670 con 0x7ff74415e130 2024-05-22T21:19:33.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.363+0000 7ff74bc00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff74406de40 con 0x7ff74415d5e0 2024-05-22T21:19:33.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.363+0000 7ff74bc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff744070a60 con 0x7ff74415dd50 2024-05-22T21:19:33.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.363+0000 7ff749c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff74415d5e0 0x7ff74415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.363+0000 7ff743e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff74415e130 0x7ff74415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47420/0 (socket says 172.21.15.161:47420) 2024-05-22T21:19:33.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.363+0000 7ff743e00640 1 -- 172.21.15.161:0/1279414963 learned_addr learned my addr 172.21.15.161:0/1279414963 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:33.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.363+0000 7ff749200640 1 --2- 172.21.15.161:0/1279414963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff74415dd50 0x7ff74406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.363+0000 7ff749c00640 1 -- 172.21.15.161:0/1279414963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff74415dd50 msgr2=0x7ff74406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:33.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.363+0000 7ff749c00640 1 --2- 172.21.15.161:0/1279414963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff74415dd50 0x7ff74406d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.363+0000 7ff749c00640 1 -- 172.21.15.161:0/1279414963 >> v1:172.21.15.161:6789/0 conn(0x7ff74415e130 legacy=0x7ff74415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.363+0000 7ff749c00640 1 -- 172.21.15.161:0/1279414963 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff744070d40 con 0x7ff74415d5e0 2024-05-22T21:19:33.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.363+0000 7ff743400640 1 -- 172.21.15.161:0/1279414963 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165140606 0 0) 0x7ff744071670 con 0x7ff74415e130 2024-05-22T21:19:33.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.367+0000 7ff749c00640 1 --2- 172.21.15.161:0/1279414963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff74415d5e0 0x7ff74415b6c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0097c0 tx=0x7ff73c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f37bd2aebeb0749 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:33.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.367+0000 7ff743400640 1 -- 172.21.15.161:0/1279414963 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0ac640 con 0x7ff74415d5e0 2024-05-22T21:19:33.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.367+0000 7ff743400640 1 -- 172.21.15.161:0/1279414963 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c0ac7a0 con 0x7ff74415d5e0 2024-05-22T21:19:33.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.367+0000 7ff743400640 1 -- 172.21.15.161:0/1279414963 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0acb30 con 0x7ff74415d5e0 2024-05-22T21:19:33.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.367+0000 7ff74bc00640 1 -- 172.21.15.161:0/1279414963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff74415d5e0 msgr2=0x7ff74415b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.367+0000 7ff74bc00640 1 --2- 172.21.15.161:0/1279414963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff74415d5e0 0x7ff74415b6c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0097c0 tx=0x7ff73c01c3e0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.367+0000 7ff74bc00640 1 -- 172.21.15.161:0/1279414963 shutdown_connections 2024-05-22T21:19:33.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.367+0000 7ff74bc00640 1 --2- 172.21.15.161:0/1279414963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff74415d5e0 0x7ff74415b6c0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.367+0000 7ff74bc00640 1 --2- 172.21.15.161:0/1279414963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff74415dd50 0x7ff74406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.367+0000 7ff74bc00640 1 -- 172.21.15.161:0/1279414963 >> 172.21.15.161:0/1279414963 conn(0x7ff7440b0bc0 msgr2=0x7ff7440b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:33.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.367+0000 7ff74bc00640 1 -- 172.21.15.161:0/1279414963 shutdown_connections 2024-05-22T21:19:33.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.367+0000 7ff74bc00640 1 -- 172.21.15.161:0/1279414963 wait complete. 2024-05-22T21:19:33.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff74bc00640 1 Processor -- start 2024-05-22T21:19:33.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff74bc00640 1 -- start start 2024-05-22T21:19:33.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff74bc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff74415d5e0 0x7ff744146ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff74bc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff74415dd50 0x7ff74414c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff74bc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff74415e130 0x7ff7440cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff74bc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74414aea0 con 0x7ff74415d5e0 2024-05-22T21:19:33.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff74bc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74414b180 con 0x7ff74415e130 2024-05-22T21:19:33.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff74bc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74414b460 con 0x7ff74415dd50 2024-05-22T21:19:33.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff749200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff74415d5e0 0x7ff744146ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff749200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff74415d5e0 0x7ff744146ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57818/0 (socket says 172.21.15.161:57818) 2024-05-22T21:19:33.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff749200640 1 -- 172.21.15.161:0/3876486262 learned_addr learned my addr 172.21.15.161:0/3876486262 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:33.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff749c00640 1 --2- 172.21.15.161:0/3876486262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff74415e130 0x7ff7440cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff743e00640 1 --2- 172.21.15.161:0/3876486262 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff74415dd50 0x7ff74414c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff749200640 1 -- 172.21.15.161:0/3876486262 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff74415dd50 msgr2=0x7ff74414c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff749200640 1 --2- 172.21.15.161:0/3876486262 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff74415dd50 0x7ff74414c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff749200640 1 -- 172.21.15.161:0/3876486262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff74415e130 msgr2=0x7ff7440cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff749200640 1 --2- 172.21.15.161:0/3876486262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff74415e130 0x7ff7440cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff749200640 1 -- 172.21.15.161:0/3876486262 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c009420 con 0x7ff74415d5e0 2024-05-22T21:19:33.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff749c00640 1 --2- 172.21.15.161:0/3876486262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff74415e130 0x7ff7440cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:33.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff749200640 1 --2- 172.21.15.161:0/3876486262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff74415d5e0 0x7ff744146ac0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff734007ec0 tx=0x7ff73400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:33.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff740c00640 1 -- 172.21.15.161:0/3876486262 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734014620 con 0x7ff74415d5e0 2024-05-22T21:19:33.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff740c00640 1 -- 172.21.15.161:0/3876486262 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7340147a0 con 0x7ff74415d5e0 2024-05-22T21:19:33.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff740c00640 1 -- 172.21.15.161:0/3876486262 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734014b50 con 0x7ff74415d5e0 2024-05-22T21:19:33.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff74bc00640 1 -- 172.21.15.161:0/3876486262 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74414b740 con 0x7ff74415d5e0 2024-05-22T21:19:33.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.371+0000 7ff74bc00640 1 -- 172.21.15.161:0/3876486262 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74414c9a0 con 0x7ff74415d5e0 2024-05-22T21:19:33.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.375+0000 7ff74bc00640 1 -- 172.21.15.161:0/3876486262 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff744067220 con 0x7ff74415d5e0 2024-05-22T21:19:33.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.375+0000 7ff740c00640 1 -- 172.21.15.161:0/3876486262 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff734014cb0 con 0x7ff74415d5e0 2024-05-22T21:19:33.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.375+0000 7ff740c00640 1 --2- 172.21.15.161:0/3876486262 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff720066690 0x7ff720068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.375+0000 7ff740c00640 1 -- 172.21.15.161:0/3876486262 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff734084bd0 con 0x7ff74415d5e0 2024-05-22T21:19:33.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.375+0000 7ff743e00640 1 --2- 172.21.15.161:0/3876486262 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff720066690 0x7ff720068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.375+0000 7ff743e00640 1 --2- 172.21.15.161:0/3876486262 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff720066690 0x7ff720068b50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff744148490 tx=0x7ff72c007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:33.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.379+0000 7ff740c00640 1 -- 172.21.15.161:0/3876486262 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff734051f40 con 0x7ff74415d5e0 2024-05-22T21:19:33.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.535+0000 7ff74bc00640 1 -- 172.21.15.161:0/3876486262 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff74414bbf0 con 0x7ff74415d5e0 2024-05-22T21:19:33.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.539+0000 7ff740c00640 1 -- 172.21.15.161:0/3876486262 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16107 (secure 0 0 0) 0x7ff734055560 con 0x7ff74415d5e0 2024-05-22T21:19:33.540 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:33.541 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":128,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:19:25.399186+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:12.848545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:13.344629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:33.542 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.539+0000 7ff74bc00640 1 -- 172.21.15.161:0/3876486262 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff720066690 msgr2=0x7ff720068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.542 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.539+0000 7ff74bc00640 1 --2- 172.21.15.161:0/3876486262 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff720066690 0x7ff720068b50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff744148490 tx=0x7ff72c007c10 comp rx=0 tx=0).stop 2024-05-22T21:19:33.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.539+0000 7ff74bc00640 1 -- 172.21.15.161:0/3876486262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff74415d5e0 msgr2=0x7ff744146ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.539+0000 7ff74bc00640 1 --2- 172.21.15.161:0/3876486262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff74415d5e0 0x7ff744146ac0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff734007ec0 tx=0x7ff73400e4b0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.539+0000 7ff74bc00640 1 -- 172.21.15.161:0/3876486262 shutdown_connections 2024-05-22T21:19:33.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.539+0000 7ff74bc00640 1 --2- 172.21.15.161:0/3876486262 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff720066690 0x7ff720068b50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.539+0000 7ff74bc00640 1 --2- 172.21.15.161:0/3876486262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff74415e130 0x7ff7440cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.539+0000 7ff74bc00640 1 --2- 172.21.15.161:0/3876486262 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff74415dd50 0x7ff74414c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.539+0000 7ff74bc00640 1 --2- 172.21.15.161:0/3876486262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff74415d5e0 0x7ff744146ac0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.539+0000 7ff74bc00640 1 -- 172.21.15.161:0/3876486262 >> 172.21.15.161:0/3876486262 conn(0x7ff7440b0bc0 msgr2=0x7ff744158800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:33.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.539+0000 7ff74bc00640 1 -- 172.21.15.161:0/3876486262 shutdown_connections 2024-05-22T21:19:33.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.539+0000 7ff74bc00640 1 -- 172.21.15.161:0/3876486262 wait complete. 2024-05-22T21:19:33.561 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-22T21:19:33.561 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:33.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.695+0000 7fbfc0e00640 1 Processor -- start 2024-05-22T21:19:33.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.695+0000 7fbfc0e00640 1 -- start start 2024-05-22T21:19:33.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.695+0000 7fbfc0e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbfbc070db0 0x7fbfbc0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.695+0000 7fbfc0e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfbc0b2ea0 0x7fbfbc0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.695+0000 7fbfc0e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfbc159bd0 con 0x7fbfbc1597f0 2024-05-22T21:19:33.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.695+0000 7fbfc0e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfbc0cbdd0 con 0x7fbfbc070db0 2024-05-22T21:19:33.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.695+0000 7fbfc0e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc0cc050 con 0x7fbfbc0b2ea0 2024-05-22T21:19:33.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.695+0000 7fbfb9400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbfbc070db0 0x7fbfbc0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.695+0000 7fbfb9400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbfbc070db0 0x7fbfbc0b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57820/0 (socket says 172.21.15.161:57820) 2024-05-22T21:19:33.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.695+0000 7fbfb9400640 1 -- 172.21.15.161:0/1550355160 learned_addr learned my addr 172.21.15.161:0/1550355160 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:33.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.695+0000 7fbfba800640 1 --2- 172.21.15.161:0/1550355160 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfbc0b2ea0 0x7fbfbc0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.695+0000 7fbfb9400640 1 -- 172.21.15.161:0/1550355160 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfbc0b2ea0 msgr2=0x7fbfbc0cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.695+0000 7fbfb9400640 1 --2- 172.21.15.161:0/1550355160 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfbc0b2ea0 0x7fbfbc0cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.695+0000 7fbfb9400640 1 -- 172.21.15.161:0/1550355160 >> v1:172.21.15.179:6789/0 conn(0x7fbfbc1597f0 legacy=0x7fbfbc0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.695+0000 7fbfb9400640 1 -- 172.21.15.161:0/1550355160 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc0b3720 con 0x7fbfbc070db0 2024-05-22T21:19:33.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.699+0000 7fbfb9400640 1 --2- 172.21.15.161:0/1550355160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc070db0 0x7fbfbc0b2960 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009870 tx=0x7fbfa801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6714577134db7e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:33.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.699+0000 7fbfb8a00640 1 -- 172.21.15.161:0/1550355160 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80aa640 con 0x7fbfbc070db0 2024-05-22T21:19:33.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.699+0000 7fbfb8a00640 1 -- 172.21.15.161:0/1550355160 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa80aa7a0 con 0x7fbfbc070db0 2024-05-22T21:19:33.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.699+0000 7fbfb8a00640 1 -- 172.21.15.161:0/1550355160 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80aab30 con 0x7fbfbc070db0 2024-05-22T21:19:33.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.699+0000 7fbfc0e00640 1 -- 172.21.15.161:0/1550355160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc070db0 msgr2=0x7fbfbc0b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.699+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/1550355160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc070db0 0x7fbfbc0b2960 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009870 tx=0x7fbfa801c560 comp rx=0 tx=0).stop 2024-05-22T21:19:33.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.699+0000 7fbfc0e00640 1 -- 172.21.15.161:0/1550355160 shutdown_connections 2024-05-22T21:19:33.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.699+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/1550355160 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfbc0b2ea0 0x7fbfbc0cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.699+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/1550355160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc070db0 0x7fbfbc0b2960 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.699+0000 7fbfc0e00640 1 -- 172.21.15.161:0/1550355160 >> 172.21.15.161:0/1550355160 conn(0x7fbfbc0b0bc0 msgr2=0x7fbfbc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:33.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.699+0000 7fbfc0e00640 1 -- 172.21.15.161:0/1550355160 shutdown_connections 2024-05-22T21:19:33.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.699+0000 7fbfc0e00640 1 -- 172.21.15.161:0/1550355160 wait complete. 2024-05-22T21:19:33.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.699+0000 7fbfc0e00640 1 Processor -- start 2024-05-22T21:19:33.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.699+0000 7fbfc0e00640 1 -- start start 2024-05-22T21:19:33.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfc0e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfbc070db0 0x7fbfbc15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfc0e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfbc0b2ea0 0x7fbfbc146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfc0e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc1597f0 0x7fbfbc14e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfc0e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc15ace0 con 0x7fbfbc1597f0 2024-05-22T21:19:33.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfc0e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc15af80 con 0x7fbfbc0b2ea0 2024-05-22T21:19:33.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfc0e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc15b260 con 0x7fbfbc070db0 2024-05-22T21:19:33.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfba800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc1597f0 0x7fbfbc14e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfba800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc1597f0 0x7fbfbc14e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57830/0 (socket says 172.21.15.161:57830) 2024-05-22T21:19:33.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfba800640 1 -- 172.21.15.161:0/3505722106 learned_addr learned my addr 172.21.15.161:0/3505722106 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:33.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfb9400640 1 --2- 172.21.15.161:0/3505722106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfbc0b2ea0 0x7fbfbc146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfb9e00640 1 --2- 172.21.15.161:0/3505722106 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfbc070db0 0x7fbfbc15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfba800640 1 -- 172.21.15.161:0/3505722106 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfbc070db0 msgr2=0x7fbfbc15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfba800640 1 --2- 172.21.15.161:0/3505722106 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfbc070db0 0x7fbfbc15a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfba800640 1 -- 172.21.15.161:0/3505722106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfbc0b2ea0 msgr2=0x7fbfbc146370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfba800640 1 --2- 172.21.15.161:0/3505722106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfbc0b2ea0 0x7fbfbc146370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfba800640 1 -- 172.21.15.161:0/3505722106 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa80094d0 con 0x7fbfbc1597f0 2024-05-22T21:19:33.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfba800640 1 --2- 172.21.15.161:0/3505722106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc1597f0 0x7fbfbc14e1b0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4007e40 tx=0x7fbfb400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:33.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfb2000640 1 -- 172.21.15.161:0/3505722106 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4014620 con 0x7fbfbc1597f0 2024-05-22T21:19:33.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfb2000640 1 -- 172.21.15.161:0/3505722106 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb400ee90 con 0x7fbfbc1597f0 2024-05-22T21:19:33.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfb2000640 1 -- 172.21.15.161:0/3505722106 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb40149e0 con 0x7fbfbc1597f0 2024-05-22T21:19:33.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfc0e00640 1 -- 172.21.15.161:0/3505722106 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc15b540 con 0x7fbfbc1597f0 2024-05-22T21:19:33.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfc0e00640 1 -- 172.21.15.161:0/3505722106 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc1443a0 con 0x7fbfbc1597f0 2024-05-22T21:19:33.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfc0e00640 1 -- 172.21.15.161:0/3505722106 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfbc067220 con 0x7fbfbc1597f0 2024-05-22T21:19:33.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.703+0000 7fbfb2000640 1 -- 172.21.15.161:0/3505722106 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbfb4014b40 con 0x7fbfbc1597f0 2024-05-22T21:19:33.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.707+0000 7fbfb2000640 1 --2- 172.21.15.161:0/3505722106 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbf9c066690 0x7fbf9c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:33.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.707+0000 7fbfb2000640 1 -- 172.21.15.161:0/3505722106 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfb4084960 con 0x7fbfbc1597f0 2024-05-22T21:19:33.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.707+0000 7fbfb9e00640 1 --2- 172.21.15.161:0/3505722106 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbf9c066690 0x7fbf9c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:33.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.707+0000 7fbfb9e00640 1 --2- 172.21.15.161:0/3505722106 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbf9c066690 0x7fbf9c068b50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc15c1a0 tx=0x7fbfac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:33.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.711+0000 7fbfb2000640 1 -- 172.21.15.161:0/3505722106 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbfb4051e10 con 0x7fbfbc1597f0 2024-05-22T21:19:33.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.895+0000 7fbfc0e00640 1 -- 172.21.15.161:0/3505722106 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfbc1580a0 con 0x7fbfbc1597f0 2024-05-22T21:19:33.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.895+0000 7fbfb2000640 1 -- 172.21.15.161:0/3505722106 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4748 (secure 0 0 0) 0x7fbfb4055430 con 0x7fbfbc1597f0 2024-05-22T21:19:33.899 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:33.899 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3944033857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3944033857},{"type":"v1","addr":"172.21.15.179:6834","nonce":3944033857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6125,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/4216709187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4216709187},{"type":"v1","addr":"172.21.15.161:6836","nonce":4216709187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2364917653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2364917653},{"type":"v1","addr":"172.21.15.179:6835","nonce":2364917653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:14.345936+0000","modified":"2024-05-22T21:19:17.355628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/1424863663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1424863663},{"type":"v1","addr":"172.21.15.161:6837","nonce":1424863663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-22T21:19:33.899 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 54 2024-05-22T21:19:33.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.899+0000 7fbfc0e00640 1 -- 172.21.15.161:0/3505722106 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbf9c066690 msgr2=0x7fbf9c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.899+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/3505722106 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbf9c066690 0x7fbf9c068b50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc15c1a0 tx=0x7fbfac009040 comp rx=0 tx=0).stop 2024-05-22T21:19:33.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.899+0000 7fbfc0e00640 1 -- 172.21.15.161:0/3505722106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc1597f0 msgr2=0x7fbfbc14e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:33.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.899+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/3505722106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc1597f0 0x7fbfbc14e1b0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4007e40 tx=0x7fbfb400e4b0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.899+0000 7fbfc0e00640 1 -- 172.21.15.161:0/3505722106 shutdown_connections 2024-05-22T21:19:33.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.899+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/3505722106 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbf9c066690 0x7fbf9c068b50 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.899+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/3505722106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfbc1597f0 0x7fbfbc14e1b0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.899+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/3505722106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfbc0b2ea0 0x7fbfbc146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.899+0000 7fbfc0e00640 1 --2- 172.21.15.161:0/3505722106 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfbc070db0 0x7fbfbc15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:33.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.899+0000 7fbfc0e00640 1 -- 172.21.15.161:0/3505722106 >> 172.21.15.161:0/3505722106 conn(0x7fbfbc0b0bc0 msgr2=0x7fbfbc1588a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:33.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.899+0000 7fbfc0e00640 1 -- 172.21.15.161:0/3505722106 shutdown_connections 2024-05-22T21:19:33.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:33.899+0000 7fbfc0e00640 1 -- 172.21.15.161:0/3505722106 wait complete. 2024-05-22T21:19:33.919 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:34.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.051+0000 7f0730e00640 1 Processor -- start 2024-05-22T21:19:34.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.051+0000 7f0730e00640 1 -- start start 2024-05-22T21:19:34.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.051+0000 7f0730e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f072c15e130 0x7f072c15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.051+0000 7f0730e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f072c15d410 0x7f072c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.051+0000 7f0730e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072c071670 con 0x7f072c15dd50 2024-05-22T21:19:34.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.051+0000 7f0730e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f072c15d870 con 0x7f072c15d410 2024-05-22T21:19:34.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.051+0000 7f0730e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072c070a60 con 0x7f072c15e130 2024-05-22T21:19:34.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.051+0000 7f072a800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f072c15d410 0x7f072c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.051+0000 7f0729400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f072c15e130 0x7f072c15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.051+0000 7f072a800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f072c15d410 0x7f072c15b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:42888/0 (socket says 172.21.15.161:42888) 2024-05-22T21:19:34.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.051+0000 7f072a800640 1 -- 172.21.15.161:0/2834649925 learned_addr learned my addr 172.21.15.161:0/2834649925 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:34.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.055+0000 7f072a800640 1 -- 172.21.15.161:0/2834649925 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f072c15e130 msgr2=0x7f072c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:34.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.055+0000 7f072a800640 1 --2- 172.21.15.161:0/2834649925 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f072c15e130 0x7f072c15ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.055+0000 7f072a800640 1 -- 172.21.15.161:0/2834649925 >> v1:172.21.15.179:6789/0 conn(0x7f072c15dd50 legacy=0x7f072c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.055+0000 7f072a800640 1 -- 172.21.15.161:0/2834649925 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c070d40 con 0x7f072c15d410 2024-05-22T21:19:34.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.055+0000 7f072a800640 1 --2- 172.21.15.161:0/2834649925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f072c15d410 0x7f072c15b6c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0724009e50 tx=0x7f072401eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46cd4d3785dd07c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:34.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.055+0000 7f0728a00640 1 -- 172.21.15.161:0/2834649925 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07240a7a50 con 0x7f072c15d410 2024-05-22T21:19:34.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.055+0000 7f0728a00640 1 -- 172.21.15.161:0/2834649925 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07240a7bb0 con 0x7f072c15d410 2024-05-22T21:19:34.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.055+0000 7f0728a00640 1 -- 172.21.15.161:0/2834649925 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07240af3f0 con 0x7f072c15d410 2024-05-22T21:19:34.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0730e00640 1 -- 172.21.15.161:0/2834649925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f072c15d410 msgr2=0x7f072c15b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0730e00640 1 --2- 172.21.15.161:0/2834649925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f072c15d410 0x7f072c15b6c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0724009e50 tx=0x7f072401eb80 comp rx=0 tx=0).stop 2024-05-22T21:19:34.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0730e00640 1 -- 172.21.15.161:0/2834649925 shutdown_connections 2024-05-22T21:19:34.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0730e00640 1 --2- 172.21.15.161:0/2834649925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f072c15d410 0x7f072c15b6c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0730e00640 1 --2- 172.21.15.161:0/2834649925 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f072c15e130 0x7f072c15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0730e00640 1 -- 172.21.15.161:0/2834649925 >> 172.21.15.161:0/2834649925 conn(0x7f072c0b0bc0 msgr2=0x7f072c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:34.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0730e00640 1 -- 172.21.15.161:0/2834649925 shutdown_connections 2024-05-22T21:19:34.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0730e00640 1 -- 172.21.15.161:0/2834649925 wait complete. 2024-05-22T21:19:34.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0730e00640 1 Processor -- start 2024-05-22T21:19:34.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0730e00640 1 -- start start 2024-05-22T21:19:34.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0730e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f072c15d410 0x7f072c158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0730e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f072c15dd50 0x7f072c152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0730e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f072c15e130 0x7f072c0b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0730e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c159020 con 0x7f072c15e130 2024-05-22T21:19:34.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0730e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c150ef0 con 0x7f072c15d410 2024-05-22T21:19:34.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0730e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072c14c780 con 0x7f072c15dd50 2024-05-22T21:19:34.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f072a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f072c15e130 0x7f072c0b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f072a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f072c15e130 0x7f072c0b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57838/0 (socket says 172.21.15.161:57838) 2024-05-22T21:19:34.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f072a800640 1 -- 172.21.15.161:0/1337744468 learned_addr learned my addr 172.21.15.161:0/1337744468 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:34.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0729e00640 1 --2- 172.21.15.161:0/1337744468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f072c15d410 0x7f072c158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f072a800640 1 -- 172.21.15.161:0/1337744468 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f072c15dd50 msgr2=0x7f072c152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f072a800640 1 --2- 172.21.15.161:0/1337744468 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f072c15dd50 0x7f072c152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f072a800640 1 -- 172.21.15.161:0/1337744468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f072c15d410 msgr2=0x7f072c158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f072a800640 1 --2- 172.21.15.161:0/1337744468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f072c15d410 0x7f072c158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f072a800640 1 -- 172.21.15.161:0/1337744468 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0724009b30 con 0x7f072c15e130 2024-05-22T21:19:34.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f0729e00640 1 --2- 172.21.15.161:0/1337744468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f072c15d410 0x7f072c158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:34.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.059+0000 7f072a800640 1 --2- 172.21.15.161:0/1337744468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f072c15e130 0x7f072c0b2590 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f072401c7b0 tx=0x7f072401b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:34.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.063+0000 7f0722000640 1 -- 172.21.15.161:0/1337744468 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07240a7a50 con 0x7f072c15e130 2024-05-22T21:19:34.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.063+0000 7f0722000640 1 -- 172.21.15.161:0/1337744468 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0724007dd0 con 0x7f072c15e130 2024-05-22T21:19:34.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.063+0000 7f0722000640 1 -- 172.21.15.161:0/1337744468 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07240ba6b0 con 0x7f072c15e130 2024-05-22T21:19:34.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.063+0000 7f0730e00640 1 -- 172.21.15.161:0/1337744468 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072c150bc0 con 0x7f072c15e130 2024-05-22T21:19:34.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.063+0000 7f0730e00640 1 -- 172.21.15.161:0/1337744468 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072c14cd00 con 0x7f072c15e130 2024-05-22T21:19:34.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.063+0000 7f0730e00640 1 -- 172.21.15.161:0/1337744468 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f072c14d2b0 con 0x7f072c15e130 2024-05-22T21:19:34.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.063+0000 7f0722000640 1 -- 172.21.15.161:0/1337744468 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f07240ba810 con 0x7f072c15e130 2024-05-22T21:19:34.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.063+0000 7f0722000640 1 --2- 172.21.15.161:0/1337744468 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f070c066690 0x7f070c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.063+0000 7f0722000640 1 -- 172.21.15.161:0/1337744468 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0724121bd0 con 0x7f072c15e130 2024-05-22T21:19:34.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.063+0000 7f0729e00640 1 --2- 172.21.15.161:0/1337744468 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f070c066690 0x7f070c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.067+0000 7f0729e00640 1 --2- 172.21.15.161:0/1337744468 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f070c066690 0x7f070c068b50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f071c000f70 tx=0x7f071c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:34.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.067+0000 7f0722000640 1 -- 172.21.15.161:0/1337744468 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f07240ef090 con 0x7f072c15e130 2024-05-22T21:19:34.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.255+0000 7f0730e00640 1 -- 172.21.15.161:0/1337744468 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f072c1510a0 con 0x7f072c15e130 2024-05-22T21:19:34.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.255+0000 7f0722000640 1 -- 172.21.15.161:0/1337744468 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f072c1510a0 con 0x7f072c15e130 2024-05-22T21:19:34.256 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:34.257 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3944033857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3944033857},{"type":"v1","addr":"172.21.15.179:6834","nonce":3944033857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6125,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/4216709187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4216709187},{"type":"v1","addr":"172.21.15.161:6836","nonce":4216709187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2364917653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2364917653},{"type":"v1","addr":"172.21.15.179:6835","nonce":2364917653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:14.345936+0000","modified":"2024-05-22T21:19:17.355628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7146},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7146":{"gid":7146,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/1424863663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1424863663},{"type":"v1","addr":"172.21.15.161:6837","nonce":1424863663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-22T21:19:34.257 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 54 2024-05-22T21:19:34.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.255+0000 7f0730e00640 1 -- 172.21.15.161:0/1337744468 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f070c066690 msgr2=0x7f070c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.255+0000 7f0730e00640 1 --2- 172.21.15.161:0/1337744468 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f070c066690 0x7f070c068b50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f071c000f70 tx=0x7f071c009040 comp rx=0 tx=0).stop 2024-05-22T21:19:34.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.255+0000 7f0730e00640 1 -- 172.21.15.161:0/1337744468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f072c15e130 msgr2=0x7f072c0b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.255+0000 7f0730e00640 1 --2- 172.21.15.161:0/1337744468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f072c15e130 0x7f072c0b2590 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f072401c7b0 tx=0x7f072401b920 comp rx=0 tx=0).stop 2024-05-22T21:19:34.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.255+0000 7f0730e00640 1 -- 172.21.15.161:0/1337744468 shutdown_connections 2024-05-22T21:19:34.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.255+0000 7f0730e00640 1 --2- 172.21.15.161:0/1337744468 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f070c066690 0x7f070c068b50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.255+0000 7f0730e00640 1 --2- 172.21.15.161:0/1337744468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f072c15e130 0x7f072c0b2590 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.255+0000 7f0730e00640 1 --2- 172.21.15.161:0/1337744468 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f072c15dd50 0x7f072c152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.255+0000 7f0730e00640 1 --2- 172.21.15.161:0/1337744468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f072c15d410 0x7f072c158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.255+0000 7f0730e00640 1 -- 172.21.15.161:0/1337744468 >> 172.21.15.161:0/1337744468 conn(0x7f072c0b0bc0 msgr2=0x7f072c0b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:34.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.255+0000 7f0730e00640 1 -- 172.21.15.161:0/1337744468 shutdown_connections 2024-05-22T21:19:34.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.255+0000 7f0730e00640 1 -- 172.21.15.161:0/1337744468 wait complete. 2024-05-22T21:19:34.278 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:34.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c78800640 1 Processor -- start 2024-05-22T21:19:34.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c78800640 1 -- start start 2024-05-22T21:19:34.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c78800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2c7006ef70 0x7f2c70099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c78800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c7006afe0 0x7f2c7006b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c78800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c700cf6e0 con 0x7f2c7006e180 2024-05-22T21:19:34.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c78800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c70160fc0 con 0x7f2c7006ef70 2024-05-22T21:19:34.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c78800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c700cf9c0 con 0x7f2c7006afe0 2024-05-22T21:19:34.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c75400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f2c7006e180 0x7f2c70099f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47432/0 (socket says 172.21.15.161:47432) 2024-05-22T21:19:34.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c75400640 1 -- 172.21.15.161:0/1567119718 learned_addr learned my addr 172.21.15.161:0/1567119718 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:34.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c75e00640 1 --2- 172.21.15.161:0/1567119718 >> v2:172.21.15.179:3300/0 conn(0x7f2c7006ef70 0x7f2c70099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c76800640 1 --2- 172.21.15.161:0/1567119718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c7006afe0 0x7f2c7006b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c74a00640 1 -- 172.21.15.161:0/1567119718 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 400035284 0 0) 0x7f2c700cf6e0 con 0x7f2c7006e180 2024-05-22T21:19:34.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c74a00640 1 -- 172.21.15.161:0/1567119718 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c58003530 con 0x7f2c7006e180 2024-05-22T21:19:34.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c74a00640 1 -- 172.21.15.161:0/1567119718 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1338467768 0 0) 0x7f2c58003530 con 0x7f2c7006e180 2024-05-22T21:19:34.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c74a00640 1 -- 172.21.15.161:0/1567119718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c7006afe0 msgr2=0x7f2c7006b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:34.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c74a00640 1 --2- 172.21.15.161:0/1567119718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c7006afe0 0x7f2c7006b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c74a00640 1 -- 172.21.15.161:0/1567119718 >> v2:172.21.15.179:3300/0 conn(0x7f2c7006ef70 msgr2=0x7f2c70099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:34.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c74a00640 1 --2- 172.21.15.161:0/1567119718 >> v2:172.21.15.179:3300/0 conn(0x7f2c7006ef70 0x7f2c70099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c74a00640 1 -- 172.21.15.161:0/1567119718 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c700cfca0 con 0x7f2c7006e180 2024-05-22T21:19:34.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c74a00640 1 -- 172.21.15.161:0/1567119718 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f2c60001680 con 0x7f2c7006e180 2024-05-22T21:19:34.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c74a00640 1 -- 172.21.15.161:0/1567119718 >> v1:172.21.15.161:6789/0 conn(0x7f2c7006e180 legacy=0x7f2c70099f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c74a00640 1 --2- 172.21.15.161:0/1567119718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c7006afe0 0x7f2c58003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c74a00640 1 --2- 172.21.15.161:0/1567119718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c58003f00 0x7f2c580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c74a00640 1 --2- 172.21.15.161:0/1567119718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c58004d10 0x7f2c58005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c74a00640 1 -- 172.21.15.161:0/1567119718 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c70160fc0 con 0x7f2c58004d10 2024-05-22T21:19:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c74a00640 1 -- 172.21.15.161:0/1567119718 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c700cf9c0 con 0x7f2c58003f00 2024-05-22T21:19:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c74a00640 1 -- 172.21.15.161:0/1567119718 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5800b990 con 0x7f2c7006afe0 2024-05-22T21:19:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.411+0000 7f2c75e00640 1 --2- 172.21.15.161:0/1567119718 >> v2:172.21.15.179:3300/0 conn(0x7f2c7006ef70 0x7f2c70099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.415+0000 7f2c76800640 1 --2- 172.21.15.161:0/1567119718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c58004d10 0x7f2c58005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.415+0000 7f2c75400640 1 --2- 172.21.15.161:0/1567119718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c58003f00 0x7f2c580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.415+0000 7f2c75e00640 1 --2- 172.21.15.161:0/1567119718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c7006afe0 0x7f2c58003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.415+0000 7f2c76800640 1 -- 172.21.15.161:0/1567119718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c7006afe0 msgr2=0x7f2c58003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.415+0000 7f2c76800640 1 --2- 172.21.15.161:0/1567119718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c7006afe0 0x7f2c58003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.415+0000 7f2c76800640 1 -- 172.21.15.161:0/1567119718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c58003f00 msgr2=0x7f2c580047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:34.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.415+0000 7f2c76800640 1 --2- 172.21.15.161:0/1567119718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c58003f00 0x7f2c580047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.415+0000 7f2c76800640 1 -- 172.21.15.161:0/1567119718 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c58003530 con 0x7f2c58004d10 2024-05-22T21:19:34.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.415+0000 7f2c75e00640 1 --2- 172.21.15.161:0/1567119718 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c7006afe0 0x7f2c58003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:34.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.415+0000 7f2c76800640 1 --2- 172.21.15.161:0/1567119718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c58004d10 0x7f2c58005190 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009fc0 tx=0x7f2c6c01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38d3b024c086da7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:34.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.415+0000 7f2c74a00640 1 -- 172.21.15.161:0/1567119718 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c0a5c90 con 0x7f2c58004d10 2024-05-22T21:19:34.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.415+0000 7f2c74a00640 1 -- 172.21.15.161:0/1567119718 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6c00f040 con 0x7f2c58004d10 2024-05-22T21:19:34.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.415+0000 7f2c74a00640 1 -- 172.21.15.161:0/1567119718 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c0ad440 con 0x7f2c58004d10 2024-05-22T21:19:34.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.415+0000 7f2c78800640 1 -- 172.21.15.161:0/1567119718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c58004d10 msgr2=0x7f2c58005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.415+0000 7f2c78800640 1 --2- 172.21.15.161:0/1567119718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c58004d10 0x7f2c58005190 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009fc0 tx=0x7f2c6c01cc20 comp rx=0 tx=0).stop 2024-05-22T21:19:34.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.415+0000 7f2c76800640 1 -- 172.21.15.161:0/1567119718 reap_dead start 2024-05-22T21:19:34.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c78800640 1 -- 172.21.15.161:0/1567119718 shutdown_connections 2024-05-22T21:19:34.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c78800640 1 -- 172.21.15.161:0/1567119718 >> 172.21.15.161:0/1567119718 conn(0x7f2c700b2a40 msgr2=0x7f2c700b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:34.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c78800640 1 -- 172.21.15.161:0/1567119718 shutdown_connections 2024-05-22T21:19:34.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c78800640 1 -- 172.21.15.161:0/1567119718 wait complete. 2024-05-22T21:19:34.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c78800640 1 Processor -- start 2024-05-22T21:19:34.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c78800640 1 -- start start 2024-05-22T21:19:34.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c78800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c58004d10 0x7f2c700b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c78800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c7014cdf0 0x7f2c700b6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c78800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c7014e430 0x7f2c7009aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c78800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7014d3b0 con 0x7f2c7014cdf0 2024-05-22T21:19:34.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c78800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7014d690 con 0x7f2c58004d10 2024-05-22T21:19:34.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c78800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7014d970 con 0x7f2c7014e430 2024-05-22T21:19:34.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c75400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c7014cdf0 0x7f2c700b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c75400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c7014cdf0 0x7f2c700b6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57864/0 (socket says 172.21.15.161:57864) 2024-05-22T21:19:34.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c75400640 1 -- 172.21.15.161:0/3612012293 learned_addr learned my addr 172.21.15.161:0/3612012293 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:34.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c75e00640 1 --2- 172.21.15.161:0/3612012293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c58004d10 0x7f2c700b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c76800640 1 --2- 172.21.15.161:0/3612012293 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c7014e430 0x7f2c7009aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c75400640 1 -- 172.21.15.161:0/3612012293 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c7014e430 msgr2=0x7f2c7009aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c75400640 1 --2- 172.21.15.161:0/3612012293 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c7014e430 0x7f2c7009aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c75400640 1 -- 172.21.15.161:0/3612012293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c58004d10 msgr2=0x7f2c700b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c75400640 1 --2- 172.21.15.161:0/3612012293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c58004d10 0x7f2c700b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c75400640 1 -- 172.21.15.161:0/3612012293 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7014dc50 con 0x7f2c7014cdf0 2024-05-22T21:19:34.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.419+0000 7f2c75400640 1 --2- 172.21.15.161:0/3612012293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c7014cdf0 0x7f2c700b6530 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000e880 tx=0x7f2c6000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:34.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.423+0000 7f2c6a000640 1 -- 172.21.15.161:0/3612012293 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6000c8d0 con 0x7f2c7014cdf0 2024-05-22T21:19:34.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.423+0000 7f2c6a000640 1 -- 172.21.15.161:0/3612012293 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6000ca50 con 0x7f2c7014cdf0 2024-05-22T21:19:34.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.423+0000 7f2c6a000640 1 -- 172.21.15.161:0/3612012293 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6000ce00 con 0x7f2c7014cdf0 2024-05-22T21:19:34.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.423+0000 7f2c78800640 1 -- 172.21.15.161:0/3612012293 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7014df30 con 0x7f2c7014cdf0 2024-05-22T21:19:34.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.423+0000 7f2c78800640 1 -- 172.21.15.161:0/3612012293 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c700b6a70 con 0x7f2c7014cdf0 2024-05-22T21:19:34.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.423+0000 7f2c78800640 1 -- 172.21.15.161:0/3612012293 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c70067220 con 0x7f2c7014cdf0 2024-05-22T21:19:34.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.423+0000 7f2c6a000640 1 -- 172.21.15.161:0/3612012293 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2c6001d460 con 0x7f2c7014cdf0 2024-05-22T21:19:34.426 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.423+0000 7f2c6a000640 1 --2- 172.21.15.161:0/3612012293 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c54066690 0x7f2c54068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.423+0000 7f2c6a000640 1 -- 172.21.15.161:0/3612012293 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c600842b0 con 0x7f2c7014cdf0 2024-05-22T21:19:34.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.423+0000 7f2c75e00640 1 --2- 172.21.15.161:0/3612012293 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c54066690 0x7f2c54068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.423+0000 7f2c75e00640 1 --2- 172.21.15.161:0/3612012293 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c54066690 0x7f2c54068b50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009d80 tx=0x7f2c64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:34.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.427+0000 7f2c6a000640 1 -- 172.21.15.161:0/3612012293 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2c60051630 con 0x7f2c7014cdf0 2024-05-22T21:19:34.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.587+0000 7f2c78800640 1 -- 172.21.15.161:0/3612012293 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c700b6ed0 con 0x7f2c7014cdf0 2024-05-22T21:19:34.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.587+0000 7f2c6a000640 1 -- 172.21.15.161:0/3612012293 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16107 (secure 0 0 0) 0x7f2c60054c50 con 0x7f2c7014cdf0 2024-05-22T21:19:34.590 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:34.590 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":128,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:19:25.399186+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:12.848545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:13.344629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:34.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.591+0000 7f2c78800640 1 -- 172.21.15.161:0/3612012293 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c54066690 msgr2=0x7f2c54068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.591+0000 7f2c78800640 1 --2- 172.21.15.161:0/3612012293 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c54066690 0x7f2c54068b50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009d80 tx=0x7f2c64008040 comp rx=0 tx=0).stop 2024-05-22T21:19:34.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.591+0000 7f2c78800640 1 -- 172.21.15.161:0/3612012293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c7014cdf0 msgr2=0x7f2c700b6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.591+0000 7f2c78800640 1 --2- 172.21.15.161:0/3612012293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c7014cdf0 0x7f2c700b6530 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000e880 tx=0x7f2c6000ed40 comp rx=0 tx=0).stop 2024-05-22T21:19:34.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.591+0000 7f2c78800640 1 -- 172.21.15.161:0/3612012293 shutdown_connections 2024-05-22T21:19:34.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.591+0000 7f2c78800640 1 --2- 172.21.15.161:0/3612012293 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c54066690 0x7f2c54068b50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.591+0000 7f2c78800640 1 --2- 172.21.15.161:0/3612012293 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c7014e430 0x7f2c7009aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.591+0000 7f2c78800640 1 --2- 172.21.15.161:0/3612012293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c7014cdf0 0x7f2c700b6530 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.591+0000 7f2c78800640 1 --2- 172.21.15.161:0/3612012293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c58004d10 0x7f2c700b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.591+0000 7f2c78800640 1 -- 172.21.15.161:0/3612012293 >> 172.21.15.161:0/3612012293 conn(0x7f2c700b2a40 msgr2=0x7f2c700add20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:34.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.591+0000 7f2c78800640 1 -- 172.21.15.161:0/3612012293 shutdown_connections 2024-05-22T21:19:34.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.591+0000 7f2c78800640 1 -- 172.21.15.161:0/3612012293 wait complete. 2024-05-22T21:19:34.612 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-22T21:19:34.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.743+0000 7f159c600640 1 Processor -- start 2024-05-22T21:19:34.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.743+0000 7f159c600640 1 -- start start 2024-05-22T21:19:34.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f159c600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f15941402e0 0x7f159413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f159c600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1594140090 con 0x7f159413fc80 2024-05-22T21:19:34.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f159c600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f159413d3a0 con 0x7f159413cfc0 2024-05-22T21:19:34.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f159c600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15940cd920 con 0x7f15941402e0 2024-05-22T21:19:34.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f159a600640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f159413fc80 0x7f159406fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47440/0 (socket says 172.21.15.161:47440) 2024-05-22T21:19:34.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f159a600640 1 -- 172.21.15.161:0/3733394317 learned_addr learned my addr 172.21.15.161:0/3733394317 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:34.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1599200640 1 --2- 172.21.15.161:0/3733394317 >> v2:172.21.15.179:3300/0 conn(0x7f15941402e0 0x7f159413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 -- 172.21.15.161:0/3733394317 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599546711 0 0) 0x7f1594140090 con 0x7f159413fc80 2024-05-22T21:19:34.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 -- 172.21.15.161:0/3733394317 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f157c003530 con 0x7f159413fc80 2024-05-22T21:19:34.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 -- 172.21.15.161:0/3733394317 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946661629 0 0) 0x7f159413d3a0 con 0x7f159413cfc0 2024-05-22T21:19:34.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 -- 172.21.15.161:0/3733394317 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1594140090 con 0x7f159413cfc0 2024-05-22T21:19:34.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 -- 172.21.15.161:0/3733394317 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4066510873 0 0) 0x7f157c003530 con 0x7f159413fc80 2024-05-22T21:19:34.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 -- 172.21.15.161:0/3733394317 >> v2:172.21.15.179:3300/0 conn(0x7f15941402e0 msgr2=0x7f159413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:34.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 --2- 172.21.15.161:0/3733394317 >> v2:172.21.15.179:3300/0 conn(0x7f15941402e0 0x7f159413f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 -- 172.21.15.161:0/3733394317 >> v1:172.21.15.179:6789/0 conn(0x7f159413cfc0 legacy=0x7f159415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 -- 172.21.15.161:0/3733394317 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15940cdc00 con 0x7f159413fc80 2024-05-22T21:19:34.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 -- 172.21.15.161:0/3733394317 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f158c001a00 con 0x7f159413fc80 2024-05-22T21:19:34.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 -- 172.21.15.161:0/3733394317 >> v1:172.21.15.161:6789/0 conn(0x7f159413fc80 legacy=0x7f159406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 --2- 172.21.15.161:0/3733394317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c003b10 0x7f157c003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 --2- 172.21.15.161:0/3733394317 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f157c0047d0 0x7f157c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 --2- 172.21.15.161:0/3733394317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f157c005190 0x7f157c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 -- 172.21.15.161:0/3733394317 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15940cd920 con 0x7f157c003b10 2024-05-22T21:19:34.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 -- 172.21.15.161:0/3733394317 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c00c0c0 con 0x7f157c005190 2024-05-22T21:19:34.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1593e00640 1 -- 172.21.15.161:0/3733394317 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157c00c280 con 0x7f157c0047d0 2024-05-22T21:19:34.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1599c00640 1 --2- 172.21.15.161:0/3733394317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c003b10 0x7f157c003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f159a600640 1 --2- 172.21.15.161:0/3733394317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f157c005190 0x7f157c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1599200640 1 --2- 172.21.15.161:0/3733394317 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f157c0047d0 0x7f157c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1599c00640 1 -- 172.21.15.161:0/3733394317 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f157c0047d0 msgr2=0x7f157c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:34.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1599c00640 1 --2- 172.21.15.161:0/3733394317 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f157c0047d0 0x7f157c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1599c00640 1 -- 172.21.15.161:0/3733394317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f157c005190 msgr2=0x7f157c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:34.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1599c00640 1 --2- 172.21.15.161:0/3733394317 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f157c005190 0x7f157c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f1599c00640 1 -- 172.21.15.161:0/3733394317 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c003530 con 0x7f157c003b10 2024-05-22T21:19:34.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.747+0000 7f159a600640 1 -- 172.21.15.161:0/3733394317 reap_dead start 2024-05-22T21:19:34.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.751+0000 7f1599c00640 1 --2- 172.21.15.161:0/3733394317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c003b10 0x7f157c003f10 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f15880018d0 tx=0x7f158801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465ff7cef38ddd9f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:34.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.751+0000 7f1593e00640 1 -- 172.21.15.161:0/3733394317 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15880052d0 con 0x7f157c003b10 2024-05-22T21:19:34.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.751+0000 7f1593e00640 1 -- 172.21.15.161:0/3733394317 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1588005450 con 0x7f157c003b10 2024-05-22T21:19:34.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.751+0000 7f1593e00640 1 -- 172.21.15.161:0/3733394317 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588003840 con 0x7f157c003b10 2024-05-22T21:19:34.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.751+0000 7f159c600640 1 -- 172.21.15.161:0/3733394317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c003b10 msgr2=0x7f157c003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.751+0000 7f159c600640 1 --2- 172.21.15.161:0/3733394317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c003b10 0x7f157c003f10 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f15880018d0 tx=0x7f158801c530 comp rx=0 tx=0).stop 2024-05-22T21:19:34.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.751+0000 7f159c600640 1 -- 172.21.15.161:0/3733394317 shutdown_connections 2024-05-22T21:19:34.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.751+0000 7f159c600640 1 --2- 172.21.15.161:0/3733394317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c003b10 0x7f157c003f10 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.751+0000 7f159c600640 1 -- 172.21.15.161:0/3733394317 >> 172.21.15.161:0/3733394317 conn(0x7f159406a2d0 msgr2=0x7f159406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:34.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.751+0000 7f159c600640 1 -- 172.21.15.161:0/3733394317 shutdown_connections 2024-05-22T21:19:34.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.751+0000 7f159c600640 1 -- 172.21.15.161:0/3733394317 wait complete. 2024-05-22T21:19:34.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f159c600640 1 Processor -- start 2024-05-22T21:19:34.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f159c600640 1 -- start start 2024-05-22T21:19:34.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f159c600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c003b10 0x7f1594157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f159c600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f159415be70 0x7f15940b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f159c600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f159415c2d0 0x7f15940b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f159c600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1594157d70 con 0x7f157c003b10 2024-05-22T21:19:34.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f159c600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15941ebd50 con 0x7f159415c2d0 2024-05-22T21:19:34.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f159c600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15941ec030 con 0x7f159415be70 2024-05-22T21:19:34.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f1599c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c003b10 0x7f1594157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f1599c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c003b10 0x7f1594157140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57884/0 (socket says 172.21.15.161:57884) 2024-05-22T21:19:34.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f1599c00640 1 -- 172.21.15.161:0/2720175400 learned_addr learned my addr 172.21.15.161:0/2720175400 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:34.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f159a600640 1 --2- 172.21.15.161:0/2720175400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f159415c2d0 0x7f15940b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f1599200640 1 --2- 172.21.15.161:0/2720175400 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f159415be70 0x7f15940b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f1599c00640 1 -- 172.21.15.161:0/2720175400 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f159415be70 msgr2=0x7f15940b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f1599c00640 1 --2- 172.21.15.161:0/2720175400 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f159415be70 0x7f15940b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f1599c00640 1 -- 172.21.15.161:0/2720175400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f159415c2d0 msgr2=0x7f15940b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f1599c00640 1 --2- 172.21.15.161:0/2720175400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f159415c2d0 0x7f15940b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f1599c00640 1 -- 172.21.15.161:0/2720175400 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159415b4f0 con 0x7f157c003b10 2024-05-22T21:19:34.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f159a600640 1 --2- 172.21.15.161:0/2720175400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f159415c2d0 0x7f15940b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:34.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f1599c00640 1 --2- 172.21.15.161:0/2720175400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c003b10 0x7f1594157140 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1588005040 tx=0x7f157c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:34.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f1591600640 1 -- 172.21.15.161:0/2720175400 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15880050d0 con 0x7f157c003b10 2024-05-22T21:19:34.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f1591600640 1 -- 172.21.15.161:0/2720175400 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1588005230 con 0x7f157c003b10 2024-05-22T21:19:34.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f159c600640 1 -- 172.21.15.161:0/2720175400 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1594140870 con 0x7f157c003b10 2024-05-22T21:19:34.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f1591600640 1 -- 172.21.15.161:0/2720175400 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588003ea0 con 0x7f157c003b10 2024-05-22T21:19:34.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f159c600640 1 -- 172.21.15.161:0/2720175400 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15941eb970 con 0x7f157c003b10 2024-05-22T21:19:34.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.755+0000 7f159c600640 1 -- 172.21.15.161:0/2720175400 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1594067220 con 0x7f157c003b10 2024-05-22T21:19:34.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.759+0000 7f1591600640 1 -- 172.21.15.161:0/2720175400 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f15880067e0 con 0x7f157c003b10 2024-05-22T21:19:34.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.759+0000 7f1591600640 1 --2- 172.21.15.161:0/2720175400 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1574066690 0x7f1574068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:34.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.759+0000 7f1591600640 1 -- 172.21.15.161:0/2720175400 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f158811d880 con 0x7f157c003b10 2024-05-22T21:19:34.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.759+0000 7f1599200640 1 --2- 172.21.15.161:0/2720175400 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1574066690 0x7f1574068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:34.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.759+0000 7f1599200640 1 --2- 172.21.15.161:0/2720175400 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1574066690 0x7f1574068b50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f159415bb10 tx=0x7f1580008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:34.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.763+0000 7f1591600640 1 -- 172.21.15.161:0/2720175400 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f15880eaac0 con 0x7f157c003b10 2024-05-22T21:19:34.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.947+0000 7f159c600640 1 -- 172.21.15.161:0/2720175400 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1594071670 con 0x7f157c003b10 2024-05-22T21:19:34.950 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:19:34.947+0000 7f7d32800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-22T21:19:34.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.963+0000 7f1591600640 1 -- 172.21.15.161:0/2720175400 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7f15880ee0e0 con 0x7f157c003b10 2024-05-22T21:19:34.965 INFO:teuthology.orchestra.run.smithi161.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-22T21:19:34.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.963+0000 7f159c600640 1 -- 172.21.15.161:0/2720175400 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1574066690 msgr2=0x7f1574068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.963+0000 7f159c600640 1 --2- 172.21.15.161:0/2720175400 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1574066690 0x7f1574068b50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f159415bb10 tx=0x7f1580008040 comp rx=0 tx=0).stop 2024-05-22T21:19:34.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.963+0000 7f159c600640 1 -- 172.21.15.161:0/2720175400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c003b10 msgr2=0x7f1594157140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:34.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.963+0000 7f159c600640 1 --2- 172.21.15.161:0/2720175400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c003b10 0x7f1594157140 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1588005040 tx=0x7f157c005800 comp rx=0 tx=0).stop 2024-05-22T21:19:34.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.963+0000 7f159c600640 1 -- 172.21.15.161:0/2720175400 shutdown_connections 2024-05-22T21:19:34.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.963+0000 7f159c600640 1 --2- 172.21.15.161:0/2720175400 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1574066690 0x7f1574068b50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.963+0000 7f159c600640 1 --2- 172.21.15.161:0/2720175400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f159415c2d0 0x7f15940b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.963+0000 7f159c600640 1 --2- 172.21.15.161:0/2720175400 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f159415be70 0x7f15940b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.963+0000 7f159c600640 1 --2- 172.21.15.161:0/2720175400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c003b10 0x7f1594157140 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:34.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.963+0000 7f159c600640 1 -- 172.21.15.161:0/2720175400 >> 172.21.15.161:0/2720175400 conn(0x7f159406a2d0 msgr2=0x7f15940cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:34.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.963+0000 7f159c600640 1 -- 172.21.15.161:0/2720175400 shutdown_connections 2024-05-22T21:19:34.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:34.963+0000 7f159c600640 1 -- 172.21.15.161:0/2720175400 wait complete. 2024-05-22T21:19:34.973 INFO:tasks.ceph.mds.a.smithi161.stderr: -4745> 2024-05-22T21:19:15.951+0000 7f5bd8600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T21:19:34.988 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-22T21:19:35.024 INFO:tasks.ceph.mds.a.smithi161.stdout:starting mds.a at 2024-05-22T21:19:35.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.127+0000 7fbdf9600640 1 Processor -- start 2024-05-22T21:19:35.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.127+0000 7fbdf9600640 1 -- start start 2024-05-22T21:19:35.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf9600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbdf413cfc0 0x7fbdf41402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:35.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf9600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbdf413fe50 0x7fbdf406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:35.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf9600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf413d3a0 con 0x7fbdf415af60 2024-05-22T21:19:35.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf9600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdf40cd920 con 0x7fbdf413fe50 2024-05-22T21:19:35.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf9600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf40cdc00 con 0x7fbdf413cfc0 2024-05-22T21:19:35.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdea000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fbdf415af60 0x7fbdf413f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47464/0 (socket says 172.21.15.161:47464) 2024-05-22T21:19:35.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdea000640 1 -- 172.21.15.161:0/1899008211 learned_addr learned my addr 172.21.15.161:0/1899008211 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:35.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf3400640 1 --2- 172.21.15.161:0/1899008211 >> v2:172.21.15.179:3300/0 conn(0x7fbdf413fe50 0x7fbdf406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:35.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2a00640 1 --2- 172.21.15.161:0/1899008211 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbdf413cfc0 0x7fbdf41402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:35.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2000640 1 -- 172.21.15.161:0/1899008211 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713642792 0 0) 0x7fbdf413d3a0 con 0x7fbdf415af60 2024-05-22T21:19:35.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2000640 1 -- 172.21.15.161:0/1899008211 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdd8005740 con 0x7fbdf415af60 2024-05-22T21:19:35.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2000640 1 -- 172.21.15.161:0/1899008211 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1397948582 0 0) 0x7fbdd8005740 con 0x7fbdf415af60 2024-05-22T21:19:35.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2000640 1 -- 172.21.15.161:0/1899008211 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbdf413cfc0 msgr2=0x7fbdf41402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:35.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2000640 1 --2- 172.21.15.161:0/1899008211 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbdf413cfc0 0x7fbdf41402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:35.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2000640 1 -- 172.21.15.161:0/1899008211 >> v2:172.21.15.179:3300/0 conn(0x7fbdf413fe50 msgr2=0x7fbdf406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:35.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2000640 1 --2- 172.21.15.161:0/1899008211 >> v2:172.21.15.179:3300/0 conn(0x7fbdf413fe50 0x7fbdf406fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:35.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2000640 1 -- 172.21.15.161:0/1899008211 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf40cdee0 con 0x7fbdf415af60 2024-05-22T21:19:35.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2000640 1 -- 172.21.15.161:0/1899008211 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fbddc001680 con 0x7fbdf415af60 2024-05-22T21:19:35.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2a00640 1 --2- 172.21.15.161:0/1899008211 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbdf413cfc0 0x7fbdf41402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:35.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2000640 1 -- 172.21.15.161:0/1899008211 >> v1:172.21.15.161:6789/0 conn(0x7fbdf415af60 legacy=0x7fbdf413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:35.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2000640 1 --2- 172.21.15.161:0/1899008211 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbdf413cfc0 0x7fbdd8005b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:35.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2000640 1 --2- 172.21.15.161:0/1899008211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdd80060d0 0x7fbdd80069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:35.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2000640 1 --2- 172.21.15.161:0/1899008211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd8006f20 0x7fbdd80073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:35.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2000640 1 -- 172.21.15.161:0/1899008211 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf40cd920 con 0x7fbdd80060d0 2024-05-22T21:19:35.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2000640 1 -- 172.21.15.161:0/1899008211 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf40cdc00 con 0x7fbdd8006f20 2024-05-22T21:19:35.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2000640 1 -- 172.21.15.161:0/1899008211 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd800dba0 con 0x7fbdf413cfc0 2024-05-22T21:19:35.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf3400640 1 --2- 172.21.15.161:0/1899008211 >> v2:172.21.15.179:3300/0 conn(0x7fbdf413fe50 0x7fbdf406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:35.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdea000640 1 --2- 172.21.15.161:0/1899008211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdd80060d0 0x7fbdd80069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:35.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf2a00640 1 --2- 172.21.15.161:0/1899008211 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbdf413cfc0 0x7fbdd8005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:35.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf3400640 1 --2- 172.21.15.161:0/1899008211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd8006f20 0x7fbdd80073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:35.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdea000640 1 -- 172.21.15.161:0/1899008211 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbdf413cfc0 msgr2=0x7fbdd8005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:35.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdea000640 1 --2- 172.21.15.161:0/1899008211 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbdf413cfc0 0x7fbdd8005b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:35.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdea000640 1 -- 172.21.15.161:0/1899008211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd8006f20 msgr2=0x7fbdd80073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:35.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdea000640 1 --2- 172.21.15.161:0/1899008211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd8006f20 0x7fbdd80073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:35.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdea000640 1 -- 172.21.15.161:0/1899008211 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde4009420 con 0x7fbdd80060d0 2024-05-22T21:19:35.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.131+0000 7fbdf3400640 1 --2- 172.21.15.161:0/1899008211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd8006f20 0x7fbdd80073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:35.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.135+0000 7fbdea000640 1 --2- 172.21.15.161:0/1899008211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdd80060d0 0x7fbdd80069e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00c240 tx=0x7fbddc01cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b2a44c7db758dd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:35.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.135+0000 7fbdf2000640 1 -- 172.21.15.161:0/1899008211 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc005ae0 con 0x7fbdd80060d0 2024-05-22T21:19:35.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.135+0000 7fbdf2000640 1 -- 172.21.15.161:0/1899008211 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbddc005c40 con 0x7fbdd80060d0 2024-05-22T21:19:35.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.135+0000 7fbdf2000640 1 -- 172.21.15.161:0/1899008211 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc007290 con 0x7fbdd80060d0 2024-05-22T21:19:35.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.135+0000 7fbdf9600640 1 -- 172.21.15.161:0/1899008211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdd80060d0 msgr2=0x7fbdd80069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:35.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.135+0000 7fbdf9600640 1 --2- 172.21.15.161:0/1899008211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdd80060d0 0x7fbdd80069e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00c240 tx=0x7fbddc01cc50 comp rx=0 tx=0).stop 2024-05-22T21:19:35.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.135+0000 7fbdf3400640 1 -- 172.21.15.161:0/1899008211 reap_dead start 2024-05-22T21:19:35.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.135+0000 7fbdf9600640 1 -- 172.21.15.161:0/1899008211 shutdown_connections 2024-05-22T21:19:35.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.135+0000 7fbdf9600640 1 -- 172.21.15.161:0/1899008211 >> 172.21.15.161:0/1899008211 conn(0x7fbdf406a2d0 msgr2=0x7fbdf406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:35.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.135+0000 7fbdf9600640 1 -- 172.21.15.161:0/1899008211 shutdown_connections 2024-05-22T21:19:35.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.135+0000 7fbdf9600640 1 -- 172.21.15.161:0/1899008211 wait complete. 2024-05-22T21:19:35.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.135+0000 7fbdf9600640 1 Processor -- start 2024-05-22T21:19:35.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.135+0000 7fbdf9600640 1 -- start start 2024-05-22T21:19:35.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf9600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbdd80060d0 0x7fbdf4159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:35.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf9600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdf413f3a0 0x7fbdf413f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:35.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf9600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf4157320 0x7fbdf41577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:35.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf9600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf4157ce0 con 0x7fbdf4157320 2024-05-22T21:19:35.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf9600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf413d590 con 0x7fbdf413f3a0 2024-05-22T21:19:35.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf9600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf413d830 con 0x7fbdd80060d0 2024-05-22T21:19:35.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf4157320 0x7fbdf41577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:35.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf2a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbdd80060d0 0x7fbdf4159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:35.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf2a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbdd80060d0 0x7fbdf4159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:59736/0 (socket says 172.21.15.161:59736) 2024-05-22T21:19:35.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf4157320 0x7fbdf41577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57912/0 (socket says 172.21.15.161:57912) 2024-05-22T21:19:35.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf2a00640 1 -- 172.21.15.161:0/2889744259 learned_addr learned my addr 172.21.15.161:0/2889744259 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:35.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdea000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdf413f3a0 0x7fbdf413f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:35.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf3400640 1 -- 172.21.15.161:0/2889744259 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbdd80060d0 msgr2=0x7fbdf4159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:35.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf3400640 1 --2- 172.21.15.161:0/2889744259 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbdd80060d0 0x7fbdf4159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:35.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf3400640 1 -- 172.21.15.161:0/2889744259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdf413f3a0 msgr2=0x7fbdf413f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:35.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf3400640 1 --2- 172.21.15.161:0/2889744259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdf413f3a0 0x7fbdf413f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:35.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf3400640 1 -- 172.21.15.161:0/2889744259 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf413db10 con 0x7fbdf4157320 2024-05-22T21:19:35.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf2a00640 1 --2- 172.21.15.161:0/2889744259 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbdd80060d0 0x7fbdf4159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:35.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf3400640 1 --2- 172.21.15.161:0/2889744259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf4157320 0x7fbdf41577a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fbdf406c630 tx=0x7fbdec0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:35.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdeb400640 1 -- 172.21.15.161:0/2889744259 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec00e070 con 0x7fbdf4157320 2024-05-22T21:19:35.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdeb400640 1 -- 172.21.15.161:0/2889744259 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdec011ed0 con 0x7fbdf4157320 2024-05-22T21:19:35.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf9600640 1 -- 172.21.15.161:0/2889744259 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf413ddf0 con 0x7fbdf4157320 2024-05-22T21:19:35.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdeb400640 1 -- 172.21.15.161:0/2889744259 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec0039a0 con 0x7fbdf4157320 2024-05-22T21:19:35.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf9600640 1 -- 172.21.15.161:0/2889744259 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf413e3b0 con 0x7fbdf4157320 2024-05-22T21:19:35.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.139+0000 7fbdf9600640 1 -- 172.21.15.161:0/2889744259 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf413e5a0 con 0x7fbdf4157320 2024-05-22T21:19:35.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.143+0000 7fbdeb400640 1 -- 172.21.15.161:0/2889744259 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbdec003b00 con 0x7fbdf4157320 2024-05-22T21:19:35.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.143+0000 7fbdeb400640 1 --2- 172.21.15.161:0/2889744259 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbdd00666c0 0x7fbdd0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:35.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.143+0000 7fbdeb400640 1 -- 172.21.15.161:0/2889744259 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbdec084be0 con 0x7fbdf4157320 2024-05-22T21:19:35.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.143+0000 7fbdf2a00640 1 --2- 172.21.15.161:0/2889744259 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbdd00666c0 0x7fbdd0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:35.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.143+0000 7fbdf2a00640 1 --2- 172.21.15.161:0/2889744259 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbdd00666c0 0x7fbdd0068b80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbde40145a0 tx=0x7fbde4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:35.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.147+0000 7fbdeb400640 1 -- 172.21.15.161:0/2889744259 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbdec051e70 con 0x7fbdf4157320 2024-05-22T21:19:35.336 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.335+0000 7fbdf9600640 1 -- 172.21.15.161:0/2889744259 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbdf4067220 con 0x7fbdf4157320 2024-05-22T21:19:35.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.963+0000 7fbdeb400640 1 -- 172.21.15.161:0/2889744259 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbdec055490 con 0x7fbdf4157320 2024-05-22T21:19:35.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.967+0000 7fbdf9600640 1 -- 172.21.15.161:0/2889744259 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbdd00666c0 msgr2=0x7fbdd0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:35.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.967+0000 7fbdf9600640 1 --2- 172.21.15.161:0/2889744259 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbdd00666c0 0x7fbdd0068b80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbde40145a0 tx=0x7fbde4009190 comp rx=0 tx=0).stop 2024-05-22T21:19:35.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.967+0000 7fbdf9600640 1 -- 172.21.15.161:0/2889744259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf4157320 msgr2=0x7fbdf41577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:35.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.967+0000 7fbdf9600640 1 --2- 172.21.15.161:0/2889744259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf4157320 0x7fbdf41577a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fbdf406c630 tx=0x7fbdec0115a0 comp rx=0 tx=0).stop 2024-05-22T21:19:35.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.967+0000 7fbdf9600640 1 -- 172.21.15.161:0/2889744259 shutdown_connections 2024-05-22T21:19:35.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.967+0000 7fbdf9600640 1 --2- 172.21.15.161:0/2889744259 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbdd00666c0 0x7fbdd0068b80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:35.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.967+0000 7fbdf9600640 1 --2- 172.21.15.161:0/2889744259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf4157320 0x7fbdf41577a0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:35.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.967+0000 7fbdf9600640 1 --2- 172.21.15.161:0/2889744259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdf413f3a0 0x7fbdf413f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:35.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.967+0000 7fbdf9600640 1 --2- 172.21.15.161:0/2889744259 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbdd80060d0 0x7fbdf4159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:35.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.967+0000 7fbdf9600640 1 -- 172.21.15.161:0/2889744259 >> 172.21.15.161:0/2889744259 conn(0x7fbdf406a2d0 msgr2=0x7fbdf40ad120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:35.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.967+0000 7fbdf9600640 1 -- 172.21.15.161:0/2889744259 shutdown_connections 2024-05-22T21:19:35.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:35.967+0000 7fbdf9600640 1 -- 172.21.15.161:0/2889744259 wait complete. 2024-05-22T21:19:35.990 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-22T21:19:36.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.167+0000 7f7ef7800640 1 Processor -- start 2024-05-22T21:19:36.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.167+0000 7f7ef7800640 1 -- start start 2024-05-22T21:19:36.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7ef7800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7ef0144980 0x7f7ef015c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:36.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7ef7800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ef006d560 0x7f7ef00cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:36.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7ef7800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ef006d0d0 con 0x7f7ef00cca40 2024-05-22T21:19:36.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7ef7800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef00cced0 con 0x7f7ef0144980 2024-05-22T21:19:36.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7ef7800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef015c680 con 0x7f7ef006d560 2024-05-22T21:19:36.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7ef5800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f7ef00cca40 0x7f7ef015c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43306/0 (socket says 172.21.15.161:43306) 2024-05-22T21:19:36.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7ef5800640 1 -- 172.21.15.161:0/3986567502 learned_addr learned my addr 172.21.15.161:0/3986567502 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:36.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7ef4e00640 1 --2- 172.21.15.161:0/3986567502 >> v2:172.21.15.179:3300/0 conn(0x7f7ef0144980 0x7f7ef015c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:36.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eefe00640 1 --2- 172.21.15.161:0/3986567502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ef006d560 0x7f7ef00cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:36.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eef400640 1 -- 172.21.15.161:0/3986567502 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258832604 0 0) 0x7f7ef006d0d0 con 0x7f7ef00cca40 2024-05-22T21:19:36.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eef400640 1 -- 172.21.15.161:0/3986567502 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ed4003530 con 0x7f7ef00cca40 2024-05-22T21:19:36.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eef400640 1 -- 172.21.15.161:0/3986567502 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3521929315 0 0) 0x7f7ed4003530 con 0x7f7ef00cca40 2024-05-22T21:19:36.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eef400640 1 -- 172.21.15.161:0/3986567502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ef006d560 msgr2=0x7f7ef00cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:36.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eef400640 1 --2- 172.21.15.161:0/3986567502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ef006d560 0x7f7ef00cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:36.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eef400640 1 -- 172.21.15.161:0/3986567502 >> v2:172.21.15.179:3300/0 conn(0x7f7ef0144980 msgr2=0x7f7ef015c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:36.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eef400640 1 --2- 172.21.15.161:0/3986567502 >> v2:172.21.15.179:3300/0 conn(0x7f7ef0144980 0x7f7ef015c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:36.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eef400640 1 -- 172.21.15.161:0/3986567502 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef006c4c0 con 0x7f7ef00cca40 2024-05-22T21:19:36.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eef400640 1 -- 172.21.15.161:0/3986567502 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f7ee8001a00 con 0x7f7ef00cca40 2024-05-22T21:19:36.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eef400640 1 -- 172.21.15.161:0/3986567502 >> v1:172.21.15.161:6789/0 conn(0x7f7ef00cca40 legacy=0x7f7ef015c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:36.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eef400640 1 --2- 172.21.15.161:0/3986567502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ed4003980 0x7f7ed4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:36.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eef400640 1 --2- 172.21.15.161:0/3986567502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ef006d560 0x7f7ed40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:36.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eef400640 1 --2- 172.21.15.161:0/3986567502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ed4004d10 0x7f7ed4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:36.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eef400640 1 -- 172.21.15.161:0/3986567502 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef00cced0 con 0x7f7ed4004d10 2024-05-22T21:19:36.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eef400640 1 -- 172.21.15.161:0/3986567502 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef015c680 con 0x7f7ed4003980 2024-05-22T21:19:36.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eef400640 1 -- 172.21.15.161:0/3986567502 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed400b990 con 0x7f7ef006d560 2024-05-22T21:19:36.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7ef5800640 1 --2- 172.21.15.161:0/3986567502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ed4004d10 0x7f7ed4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:36.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7eefe00640 1 --2- 172.21.15.161:0/3986567502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ef006d560 0x7f7ed40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:36.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7ef4e00640 1 --2- 172.21.15.161:0/3986567502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ed4003980 0x7f7ed4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:36.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7ef5800640 1 -- 172.21.15.161:0/3986567502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ef006d560 msgr2=0x7f7ed40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:36.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7ef5800640 1 --2- 172.21.15.161:0/3986567502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ef006d560 0x7f7ed40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:36.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7ef5800640 1 -- 172.21.15.161:0/3986567502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ed4003980 msgr2=0x7f7ed4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:36.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7ef5800640 1 --2- 172.21.15.161:0/3986567502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ed4003980 0x7f7ed4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:36.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7ef5800640 1 -- 172.21.15.161:0/3986567502 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed4003530 con 0x7f7ed4004d10 2024-05-22T21:19:36.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.171+0000 7f7ef4e00640 1 --2- 172.21.15.161:0/3986567502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ed4003980 0x7f7ed4003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:36.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.175+0000 7f7ef5800640 1 --2- 172.21.15.161:0/3986567502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ed4004d10 0x7f7ed4005190 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800a860 tx=0x7f7ee801d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225cab7f5a8c365 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:36.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.175+0000 7f7eef400640 1 -- 172.21.15.161:0/3986567502 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee80ad630 con 0x7f7ed4004d10 2024-05-22T21:19:36.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.175+0000 7f7eef400640 1 -- 172.21.15.161:0/3986567502 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee80a8070 con 0x7f7ed4004d10 2024-05-22T21:19:36.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.175+0000 7f7eef400640 1 -- 172.21.15.161:0/3986567502 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee80ad9f0 con 0x7f7ed4004d10 2024-05-22T21:19:36.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.175+0000 7f7ef7800640 1 -- 172.21.15.161:0/3986567502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ed4004d10 msgr2=0x7f7ed4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:36.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.175+0000 7f7ef7800640 1 --2- 172.21.15.161:0/3986567502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ed4004d10 0x7f7ed4005190 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800a860 tx=0x7f7ee801d380 comp rx=0 tx=0).stop 2024-05-22T21:19:36.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.175+0000 7f7ef5800640 1 -- 172.21.15.161:0/3986567502 reap_dead start 2024-05-22T21:19:36.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.175+0000 7f7ef7800640 1 -- 172.21.15.161:0/3986567502 shutdown_connections 2024-05-22T21:19:36.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.175+0000 7f7ef7800640 1 -- 172.21.15.161:0/3986567502 >> 172.21.15.161:0/3986567502 conn(0x7f7ef006a0a0 msgr2=0x7f7ef006a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:36.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.175+0000 7f7ef7800640 1 -- 172.21.15.161:0/3986567502 shutdown_connections 2024-05-22T21:19:36.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.175+0000 7f7ef7800640 1 -- 172.21.15.161:0/3986567502 wait complete. 2024-05-22T21:19:36.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.175+0000 7f7ef7800640 1 Processor -- start 2024-05-22T21:19:36.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.175+0000 7f7ef7800640 1 -- start start 2024-05-22T21:19:36.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7ef7800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ed4004d10 0x7f7ef013d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:36.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7ef7800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef0139730 0x7f7ef015a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:36.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7ef7800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ef0139b70 0x7f7ef015a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:36.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7ef7800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef013a690 con 0x7f7ef0139730 2024-05-22T21:19:36.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7ef7800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef0140440 con 0x7f7ef0139b70 2024-05-22T21:19:36.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7ef7800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef0140720 con 0x7f7ed4004d10 2024-05-22T21:19:36.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7eefe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef0139730 0x7f7ef015a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:36.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7eefe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef0139730 0x7f7ef015a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44936/0 (socket says 172.21.15.161:44936) 2024-05-22T21:19:36.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7eefe00640 1 -- 172.21.15.161:0/3198247882 learned_addr learned my addr 172.21.15.161:0/3198247882 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:36.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7ef4e00640 1 --2- 172.21.15.161:0/3198247882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ed4004d10 0x7f7ef013d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:36.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7ef5800640 1 --2- 172.21.15.161:0/3198247882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ef0139b70 0x7f7ef015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:36.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7eefe00640 1 -- 172.21.15.161:0/3198247882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ed4004d10 msgr2=0x7f7ef013d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:36.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7eefe00640 1 --2- 172.21.15.161:0/3198247882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ed4004d10 0x7f7ef013d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:36.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7eefe00640 1 -- 172.21.15.161:0/3198247882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ef0139b70 msgr2=0x7f7ef015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:36.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7eefe00640 1 --2- 172.21.15.161:0/3198247882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ef0139b70 0x7f7ef015a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:36.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7eefe00640 1 -- 172.21.15.161:0/3198247882 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef013a870 con 0x7f7ef0139730 2024-05-22T21:19:36.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7ef5800640 1 --2- 172.21.15.161:0/3198247882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ef0139b70 0x7f7ef015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:36.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7eefe00640 1 --2- 172.21.15.161:0/3198247882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef0139730 0x7f7ef015a330 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0119d0 tx=0x7f7edc011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:36.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7eecc00640 1 -- 172.21.15.161:0/3198247882 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc00fb50 con 0x7f7ef0139730 2024-05-22T21:19:36.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7eecc00640 1 -- 172.21.15.161:0/3198247882 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7edc00fcb0 con 0x7f7ef0139730 2024-05-22T21:19:36.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7ef7800640 1 -- 172.21.15.161:0/3198247882 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef013ab50 con 0x7f7ef0139730 2024-05-22T21:19:36.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7eecc00640 1 -- 172.21.15.161:0/3198247882 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc014440 con 0x7f7ef0139730 2024-05-22T21:19:36.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7ef7800640 1 -- 172.21.15.161:0/3198247882 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef015af80 con 0x7f7ef0139730 2024-05-22T21:19:36.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7ef7800640 1 -- 172.21.15.161:0/3198247882 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef0140940 con 0x7f7ef0139730 2024-05-22T21:19:36.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.179+0000 7f7eecc00640 1 -- 172.21.15.161:0/3198247882 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7edc01d400 con 0x7f7ef0139730 2024-05-22T21:19:36.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.183+0000 7f7eecc00640 1 --2- 172.21.15.161:0/3198247882 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7ed00666c0 0x7f7ed0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:36.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.183+0000 7f7eecc00640 1 -- 172.21.15.161:0/3198247882 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7edc084a50 con 0x7f7ef0139730 2024-05-22T21:19:36.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.183+0000 7f7ef4e00640 1 --2- 172.21.15.161:0/3198247882 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7ed00666c0 0x7f7ed0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:36.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.183+0000 7f7ef4e00640 1 --2- 172.21.15.161:0/3198247882 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7ed00666c0 0x7f7ed0068b80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0006f90 tx=0x7f7ee0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:36.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.187+0000 7f7eecc00640 1 -- 172.21.15.161:0/3198247882 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7edc051d80 con 0x7f7ef0139730 2024-05-22T21:19:36.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.343+0000 7f7ef7800640 1 -- 172.21.15.161:0/3198247882 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7ef0067220 con 0x7f7ef0139730 2024-05-22T21:19:36.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.967+0000 7f7eecc00640 1 -- 172.21.15.161:0/3198247882 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v130) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7edc0553a0 con 0x7f7ef0139730 2024-05-22T21:19:36.970 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' removed 2024-05-22T21:19:36.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.971+0000 7f7ef7800640 1 -- 172.21.15.161:0/3198247882 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7ed00666c0 msgr2=0x7f7ed0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:36.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.971+0000 7f7ef7800640 1 --2- 172.21.15.161:0/3198247882 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7ed00666c0 0x7f7ed0068b80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0006f90 tx=0x7f7ee0008040 comp rx=0 tx=0).stop 2024-05-22T21:19:36.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.971+0000 7f7ef7800640 1 -- 172.21.15.161:0/3198247882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef0139730 msgr2=0x7f7ef015a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:36.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.971+0000 7f7ef7800640 1 --2- 172.21.15.161:0/3198247882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef0139730 0x7f7ef015a330 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0119d0 tx=0x7f7edc011e90 comp rx=0 tx=0).stop 2024-05-22T21:19:36.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.971+0000 7f7ef7800640 1 -- 172.21.15.161:0/3198247882 shutdown_connections 2024-05-22T21:19:36.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.971+0000 7f7ef7800640 1 --2- 172.21.15.161:0/3198247882 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7ed00666c0 0x7f7ed0068b80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:36.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.971+0000 7f7ef7800640 1 --2- 172.21.15.161:0/3198247882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ef0139b70 0x7f7ef015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:36.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.971+0000 7f7ef7800640 1 --2- 172.21.15.161:0/3198247882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef0139730 0x7f7ef015a330 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:36.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.971+0000 7f7ef7800640 1 --2- 172.21.15.161:0/3198247882 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ed4004d10 0x7f7ef013d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:36.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.971+0000 7f7ef7800640 1 -- 172.21.15.161:0/3198247882 >> 172.21.15.161:0/3198247882 conn(0x7f7ef006a0a0 msgr2=0x7f7ef00aeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:36.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.971+0000 7f7ef7800640 1 -- 172.21.15.161:0/3198247882 shutdown_connections 2024-05-22T21:19:36.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:36.971+0000 7f7ef7800640 1 -- 172.21.15.161:0/3198247882 wait complete. 2024-05-22T21:19:36.993 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-22T21:19:37.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.127+0000 7f8c9be00640 1 Processor -- start 2024-05-22T21:19:37.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.131+0000 7f8c9be00640 1 -- start start 2024-05-22T21:19:37.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.131+0000 7f8c9be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c9c15f1e0 0x7f8c9c071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:37.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.131+0000 7f8c9be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c9c06fb50 0x7f8c9c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:37.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.131+0000 7f8c9be00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9c144050 con 0x7f8c9c143c70 2024-05-22T21:19:37.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.131+0000 7f8c9be00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9c0ce490 con 0x7f8c9c06fb50 2024-05-22T21:19:37.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.131+0000 7f8c9be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c071e70 con 0x7f8c9c15f1e0 2024-05-22T21:19:37.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.131+0000 7f8c9b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c9c06fb50 0x7f8c9c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:37.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.131+0000 7f8c9b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c9c06fb50 0x7f8c9c06ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44950/0 (socket says 172.21.15.161:44950) 2024-05-22T21:19:37.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.131+0000 7f8c9b400640 1 -- 172.21.15.161:0/2520266763 learned_addr learned my addr 172.21.15.161:0/2520266763 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:37.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.131+0000 7f8c9a000640 1 --2- 172.21.15.161:0/2520266763 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c9c15f1e0 0x7f8c9c071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:37.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.131+0000 7f8c9b400640 1 -- 172.21.15.161:0/2520266763 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c9c15f1e0 msgr2=0x7f8c9c071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:37.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.131+0000 7f8c9b400640 1 --2- 172.21.15.161:0/2520266763 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c9c15f1e0 0x7f8c9c071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:37.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.131+0000 7f8c9b400640 1 -- 172.21.15.161:0/2520266763 >> v1:172.21.15.179:6789/0 conn(0x7f8c9c143c70 legacy=0x7f8c9c15ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:37.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.131+0000 7f8c9b400640 1 -- 172.21.15.161:0/2520266763 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c071190 con 0x7f8c9c06fb50 2024-05-22T21:19:37.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.135+0000 7f8c9b400640 1 --2- 172.21.15.161:0/2520266763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c9c06fb50 0x7f8c9c06ffe0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f8c94009870 tx=0x7f8c9401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823474a0fada6aa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:37.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.135+0000 7f8c99600640 1 -- 172.21.15.161:0/2520266763 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c940a9620 con 0x7f8c9c06fb50 2024-05-22T21:19:37.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.135+0000 7f8c99600640 1 -- 172.21.15.161:0/2520266763 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c940a9780 con 0x7f8c9c06fb50 2024-05-22T21:19:37.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.135+0000 7f8c99600640 1 -- 172.21.15.161:0/2520266763 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c940a9b10 con 0x7f8c9c06fb50 2024-05-22T21:19:37.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.135+0000 7f8c9be00640 1 -- 172.21.15.161:0/2520266763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c9c06fb50 msgr2=0x7f8c9c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:37.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.135+0000 7f8c9be00640 1 --2- 172.21.15.161:0/2520266763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c9c06fb50 0x7f8c9c06ffe0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f8c94009870 tx=0x7f8c9401c560 comp rx=0 tx=0).stop 2024-05-22T21:19:37.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.135+0000 7f8c9be00640 1 -- 172.21.15.161:0/2520266763 shutdown_connections 2024-05-22T21:19:37.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.135+0000 7f8c9be00640 1 --2- 172.21.15.161:0/2520266763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c9c06fb50 0x7f8c9c06ffe0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:37.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.135+0000 7f8c9be00640 1 --2- 172.21.15.161:0/2520266763 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c9c15f1e0 0x7f8c9c071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:37.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.135+0000 7f8c9be00640 1 -- 172.21.15.161:0/2520266763 >> 172.21.15.161:0/2520266763 conn(0x7f8c9c0cebc0 msgr2=0x7f8c9c0cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:37.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.135+0000 7f8c9be00640 1 -- 172.21.15.161:0/2520266763 shutdown_connections 2024-05-22T21:19:37.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.135+0000 7f8c9be00640 1 -- 172.21.15.161:0/2520266763 wait complete. 2024-05-22T21:19:37.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.135+0000 7f8c9be00640 1 Processor -- start 2024-05-22T21:19:37.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.135+0000 7f8c9be00640 1 -- start start 2024-05-22T21:19:37.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c9c06fb50 0x7f8c9c15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:37.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c9c143c70 0x7f8c9c15db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:37.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c9c15f1e0 0x7f8c9c140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:37.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9be00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c0b3850 con 0x7f8c9c15f1e0 2024-05-22T21:19:37.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9be00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c142880 con 0x7f8c9c143c70 2024-05-22T21:19:37.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c141c70 con 0x7f8c9c06fb50 2024-05-22T21:19:37.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c9c15f1e0 0x7f8c9c140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:37.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c9c15f1e0 0x7f8c9c140d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44952/0 (socket says 172.21.15.161:44952) 2024-05-22T21:19:37.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9b400640 1 -- 172.21.15.161:0/2720686776 learned_addr learned my addr 172.21.15.161:0/2720686776 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:37.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9a000640 1 --2- 172.21.15.161:0/2720686776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c9c143c70 0x7f8c9c15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:37.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9aa00640 1 --2- 172.21.15.161:0/2720686776 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c9c06fb50 0x7f8c9c15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:37.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9b400640 1 -- 172.21.15.161:0/2720686776 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c9c06fb50 msgr2=0x7f8c9c15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:37.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9b400640 1 --2- 172.21.15.161:0/2720686776 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c9c06fb50 0x7f8c9c15a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:37.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9b400640 1 -- 172.21.15.161:0/2720686776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c9c143c70 msgr2=0x7f8c9c15db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:37.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9b400640 1 --2- 172.21.15.161:0/2720686776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c9c143c70 0x7f8c9c15db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:37.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9b400640 1 -- 172.21.15.161:0/2720686776 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c940094d0 con 0x7f8c9c15f1e0 2024-05-22T21:19:37.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9b400640 1 --2- 172.21.15.161:0/2720686776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c9c15f1e0 0x7f8c9c140d60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8c9401ac10 tx=0x7f8c94019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:37.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c92a00640 1 -- 172.21.15.161:0/2720686776 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c940ba680 con 0x7f8c9c15f1e0 2024-05-22T21:19:37.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c92a00640 1 -- 172.21.15.161:0/2720686776 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c940ba800 con 0x7f8c9c15f1e0 2024-05-22T21:19:37.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c92a00640 1 -- 172.21.15.161:0/2720686776 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c940babb0 con 0x7f8c9c15f1e0 2024-05-22T21:19:37.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9be00640 1 -- 172.21.15.161:0/2720686776 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9c141f50 con 0x7f8c9c15f1e0 2024-05-22T21:19:37.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9be00640 1 -- 172.21.15.161:0/2720686776 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9c142510 con 0x7f8c9c15f1e0 2024-05-22T21:19:37.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c9be00640 1 -- 172.21.15.161:0/2720686776 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9c1445a0 con 0x7f8c9c15f1e0 2024-05-22T21:19:37.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.139+0000 7f8c92a00640 1 -- 172.21.15.161:0/2720686776 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8c940bad10 con 0x7f8c9c15f1e0 2024-05-22T21:19:37.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.143+0000 7f8c92a00640 1 --2- 172.21.15.161:0/2720686776 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c7c066690 0x7f8c7c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:37.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.143+0000 7f8c92a00640 1 -- 172.21.15.161:0/2720686776 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f8c9411e530 con 0x7f8c9c15f1e0 2024-05-22T21:19:37.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.143+0000 7f8c9aa00640 1 --2- 172.21.15.161:0/2720686776 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c7c066690 0x7f8c7c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:37.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.143+0000 7f8c9aa00640 1 --2- 172.21.15.161:0/2720686776 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c7c066690 0x7f8c7c068b50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0012f0 tx=0x7f8c8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:37.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.147+0000 7f8c92a00640 1 -- 172.21.15.161:0/2720686776 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8c940eba10 con 0x7f8c9c15f1e0 2024-05-22T21:19:37.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.303+0000 7f8c9be00640 1 -- 172.21.15.161:0/2720686776 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8c9c1429f0 con 0x7f8c9c15f1e0 2024-05-22T21:19:37.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.967+0000 7f8c92a00640 1 -- 172.21.15.161:0/2720686776 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v131) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8c940ef030 con 0x7f8c9c15f1e0 2024-05-22T21:19:37.972 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' removed 2024-05-22T21:19:37.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.971+0000 7f8c9be00640 1 -- 172.21.15.161:0/2720686776 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c7c066690 msgr2=0x7f8c7c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:37.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.971+0000 7f8c9be00640 1 --2- 172.21.15.161:0/2720686776 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c7c066690 0x7f8c7c068b50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0012f0 tx=0x7f8c8c009040 comp rx=0 tx=0).stop 2024-05-22T21:19:37.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.971+0000 7f8c9be00640 1 -- 172.21.15.161:0/2720686776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c9c15f1e0 msgr2=0x7f8c9c140d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:37.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.971+0000 7f8c9be00640 1 --2- 172.21.15.161:0/2720686776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c9c15f1e0 0x7f8c9c140d60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8c9401ac10 tx=0x7f8c94019300 comp rx=0 tx=0).stop 2024-05-22T21:19:37.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.971+0000 7f8c9be00640 1 -- 172.21.15.161:0/2720686776 shutdown_connections 2024-05-22T21:19:37.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.971+0000 7f8c9be00640 1 --2- 172.21.15.161:0/2720686776 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8c7c066690 0x7f8c7c068b50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:37.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.971+0000 7f8c9be00640 1 --2- 172.21.15.161:0/2720686776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c9c15f1e0 0x7f8c9c140d60 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:37.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.971+0000 7f8c9be00640 1 --2- 172.21.15.161:0/2720686776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c9c143c70 0x7f8c9c15db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:37.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.971+0000 7f8c9be00640 1 --2- 172.21.15.161:0/2720686776 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c9c06fb50 0x7f8c9c15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:37.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.971+0000 7f8c9be00640 1 -- 172.21.15.161:0/2720686776 >> 172.21.15.161:0/2720686776 conn(0x7f8c9c0cebc0 msgr2=0x7f8c9c0cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:37.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.971+0000 7f8c9be00640 1 -- 172.21.15.161:0/2720686776 shutdown_connections 2024-05-22T21:19:37.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:37.971+0000 7f8c9be00640 1 -- 172.21.15.161:0/2720686776 wait complete. 2024-05-22T21:19:37.994 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-22T21:19:38.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.127+0000 7faebe200640 1 Processor -- start 2024-05-22T21:19:38.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.127+0000 7faebe200640 1 -- start start 2024-05-22T21:19:38.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.131+0000 7faebe200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7faeb8141730 0x7faeb8141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:38.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.131+0000 7faebe200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7faeb806fcc0 0x7faeb815ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:38.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.131+0000 7faebe200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faeb80cbca0 0x7faeb80cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:38.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.131+0000 7faebe200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb80cc650 con 0x7faeb8141730 2024-05-22T21:19:38.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.131+0000 7faebe200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb80cd150 con 0x7faeb806fcc0 2024-05-22T21:19:38.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.131+0000 7faebe200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb815f550 con 0x7faeb80cbca0 2024-05-22T21:19:38.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.131+0000 7faeb7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7faeb8141730 0x7faeb8141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:38.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.131+0000 7faeb7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7faeb8141730 0x7faeb8141ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44956/0 (socket says 172.21.15.161:44956) 2024-05-22T21:19:38.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.131+0000 7faeb7400640 1 -- 172.21.15.161:0/1823099543 learned_addr learned my addr 172.21.15.161:0/1823099543 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:38.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.131+0000 7faeb6a00640 1 --2- 172.21.15.161:0/1823099543 >> v2:172.21.15.179:3300/0 conn(0x7faeb806fcc0 0x7faeb815ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:38.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.131+0000 7faeb7e00640 1 --2- 172.21.15.161:0/1823099543 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faeb80cbca0 0x7faeb80cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:38.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.131+0000 7faeb7400640 1 -- 172.21.15.161:0/1823099543 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faeb80cbca0 msgr2=0x7faeb80cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:38.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.131+0000 7faeb7400640 1 --2- 172.21.15.161:0/1823099543 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faeb80cbca0 0x7faeb80cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.131+0000 7faeb7400640 1 -- 172.21.15.161:0/1823099543 >> v2:172.21.15.179:3300/0 conn(0x7faeb806fcc0 msgr2=0x7faeb815ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:38.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.131+0000 7faeb7400640 1 --2- 172.21.15.161:0/1823099543 >> v2:172.21.15.179:3300/0 conn(0x7faeb806fcc0 0x7faeb815ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.131+0000 7faeb7400640 1 -- 172.21.15.161:0/1823099543 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb815f830 con 0x7faeb8141730 2024-05-22T21:19:38.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faeb7400640 1 --2- 172.21.15.161:0/1823099543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faeb8141730 0x7faeb8141ed0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7faeb80b1c10 tx=0x7faea801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ec6013737e1c61 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:38.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faeb6000640 1 -- 172.21.15.161:0/1823099543 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea80a5840 con 0x7faeb8141730 2024-05-22T21:19:38.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faeb6000640 1 -- 172.21.15.161:0/1823099543 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea80a59a0 con 0x7faeb8141730 2024-05-22T21:19:38.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faeb6000640 1 -- 172.21.15.161:0/1823099543 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea80a5d30 con 0x7faeb8141730 2024-05-22T21:19:38.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 -- 172.21.15.161:0/1823099543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faeb8141730 msgr2=0x7faeb8141ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:38.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 --2- 172.21.15.161:0/1823099543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faeb8141730 0x7faeb8141ed0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7faeb80b1c10 tx=0x7faea801c9c0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 -- 172.21.15.161:0/1823099543 shutdown_connections 2024-05-22T21:19:38.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 --2- 172.21.15.161:0/1823099543 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faeb80cbca0 0x7faeb80cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 --2- 172.21.15.161:0/1823099543 >> v2:172.21.15.179:3300/0 conn(0x7faeb806fcc0 0x7faeb815ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 --2- 172.21.15.161:0/1823099543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faeb8141730 0x7faeb8141ed0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 -- 172.21.15.161:0/1823099543 >> 172.21.15.161:0/1823099543 conn(0x7faeb806a2d0 msgr2=0x7faeb806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:38.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 -- 172.21.15.161:0/1823099543 shutdown_connections 2024-05-22T21:19:38.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 -- 172.21.15.161:0/1823099543 wait complete. 2024-05-22T21:19:38.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 Processor -- start 2024-05-22T21:19:38.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 -- start start 2024-05-22T21:19:38.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faeb806fcc0 0x7faeb815a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:38.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faeb80cbca0 0x7faeb8139a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:38.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faeb8141730 0x7faeb815d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:38.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb815e0d0 con 0x7faeb806fcc0 2024-05-22T21:19:38.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb815d600 con 0x7faeb8141730 2024-05-22T21:19:38.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.135+0000 7faebe200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb815b030 con 0x7faeb80cbca0 2024-05-22T21:19:38.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faeb7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faeb806fcc0 0x7faeb815a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:38.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faeb7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faeb806fcc0 0x7faeb815a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44960/0 (socket says 172.21.15.161:44960) 2024-05-22T21:19:38.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faeb7400640 1 -- 172.21.15.161:0/3874288617 learned_addr learned my addr 172.21.15.161:0/3874288617 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:38.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faeb7400640 1 -- 172.21.15.161:0/3874288617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faeb80cbca0 msgr2=0x7faeb8139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:38.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faeb7400640 1 --2- 172.21.15.161:0/3874288617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faeb80cbca0 0x7faeb8139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faeb7400640 1 -- 172.21.15.161:0/3874288617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faeb8141730 msgr2=0x7faeb815d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:38.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faeb7400640 1 --2- 172.21.15.161:0/3874288617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faeb8141730 0x7faeb815d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faeb7400640 1 -- 172.21.15.161:0/3874288617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea8009b30 con 0x7faeb806fcc0 2024-05-22T21:19:38.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faeb7400640 1 --2- 172.21.15.161:0/3874288617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faeb806fcc0 0x7faeb815a820 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7faea80af040 tx=0x7faea8019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:38.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faeaf400640 1 -- 172.21.15.161:0/3874288617 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea80b8670 con 0x7faeb806fcc0 2024-05-22T21:19:38.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faeaf400640 1 -- 172.21.15.161:0/3874288617 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea80b87f0 con 0x7faeb806fcc0 2024-05-22T21:19:38.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faeaf400640 1 -- 172.21.15.161:0/3874288617 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea80b8ba0 con 0x7faeb806fcc0 2024-05-22T21:19:38.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faebe200640 1 -- 172.21.15.161:0/3874288617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb815d830 con 0x7faeb806fcc0 2024-05-22T21:19:38.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faebe200640 1 -- 172.21.15.161:0/3874288617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb815ddf0 con 0x7faeb806fcc0 2024-05-22T21:19:38.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faebe200640 1 -- 172.21.15.161:0/3874288617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb8067aa0 con 0x7faeb806fcc0 2024-05-22T21:19:38.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faeaf400640 1 -- 172.21.15.161:0/3874288617 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faea80b8d00 con 0x7faeb806fcc0 2024-05-22T21:19:38.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faeaf400640 1 --2- 172.21.15.161:0/3874288617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae98066690 0x7fae98068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:38.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.139+0000 7faeaf400640 1 -- 172.21.15.161:0/3874288617 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 4943+0+0 (secure 0 0 0) 0x7faea811f6e0 con 0x7faeb806fcc0 2024-05-22T21:19:38.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.143+0000 7faeb6a00640 1 --2- 172.21.15.161:0/3874288617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae98066690 0x7fae98068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:38.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.143+0000 7faeb6a00640 1 --2- 172.21.15.161:0/3874288617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae98066690 0x7fae98068b50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7faeb813bda0 tx=0x7faea4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:38.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.147+0000 7faeaf400640 1 -- 172.21.15.161:0/3874288617 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7faea80ecdd0 con 0x7faeb806fcc0 2024-05-22T21:19:38.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.299+0000 7faebe200640 1 -- 172.21.15.161:0/3874288617 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7faeb8067220 con 0x7faeb806fcc0 2024-05-22T21:19:38.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.303+0000 7faeaf400640 1 -- 172.21.15.161:0/3874288617 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7faea80f03f0 con 0x7faeb806fcc0 2024-05-22T21:19:38.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.307+0000 7faebe200640 1 -- 172.21.15.161:0/3874288617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae98066690 msgr2=0x7fae98068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:38.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.307+0000 7faebe200640 1 --2- 172.21.15.161:0/3874288617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae98066690 0x7fae98068b50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7faeb813bda0 tx=0x7faea4008040 comp rx=0 tx=0).stop 2024-05-22T21:19:38.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.307+0000 7faebe200640 1 -- 172.21.15.161:0/3874288617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faeb806fcc0 msgr2=0x7faeb815a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:38.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.307+0000 7faebe200640 1 --2- 172.21.15.161:0/3874288617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faeb806fcc0 0x7faeb815a820 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7faea80af040 tx=0x7faea8019760 comp rx=0 tx=0).stop 2024-05-22T21:19:38.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.307+0000 7faebe200640 1 -- 172.21.15.161:0/3874288617 shutdown_connections 2024-05-22T21:19:38.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.307+0000 7faebe200640 1 --2- 172.21.15.161:0/3874288617 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fae98066690 0x7fae98068b50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.307+0000 7faebe200640 1 --2- 172.21.15.161:0/3874288617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faeb8141730 0x7faeb815d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.307+0000 7faebe200640 1 --2- 172.21.15.161:0/3874288617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faeb80cbca0 0x7faeb8139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.307+0000 7faebe200640 1 --2- 172.21.15.161:0/3874288617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faeb806fcc0 0x7faeb815a820 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.307+0000 7faebe200640 1 -- 172.21.15.161:0/3874288617 >> 172.21.15.161:0/3874288617 conn(0x7faeb806a2d0 msgr2=0x7faeb815ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:38.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.307+0000 7faebe200640 1 -- 172.21.15.161:0/3874288617 shutdown_connections 2024-05-22T21:19:38.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.307+0000 7faebe200640 1 -- 172.21.15.161:0/3874288617 wait complete. 2024-05-22T21:19:38.329 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2024-05-22T21:19:38.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.463+0000 7f8a38c00640 1 Processor -- start 2024-05-22T21:19:38.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.463+0000 7f8a38c00640 1 -- start start 2024-05-22T21:19:38.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.467+0000 7f8a38c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8a340715a0 0x7f8a3415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:38.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.467+0000 7f8a38c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8a3415ec90 0x7f8a3415f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:38.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.467+0000 7f8a38c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a341442a0 con 0x7f8a34143ec0 2024-05-22T21:19:38.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.467+0000 7f8a38c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a3415f640 con 0x7f8a3415ec90 2024-05-22T21:19:38.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.467+0000 7f8a38c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a34070500 con 0x7f8a340715a0 2024-05-22T21:19:38.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.467+0000 7f8a31e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8a34143ec0 0x7f8a3406fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43316/0 (socket says 172.21.15.161:43316) 2024-05-22T21:19:38.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.467+0000 7f8a31e00640 1 -- 172.21.15.161:0/1906875211 learned_addr learned my addr 172.21.15.161:0/1906875211 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:38.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.467+0000 7f8a32800640 1 --2- 172.21.15.161:0/1906875211 >> v2:172.21.15.161:3300/0 conn(0x7f8a3415ec90 0x7f8a3415f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:38.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.467+0000 7f8a29400640 1 --2- 172.21.15.161:0/1906875211 >> v2:172.21.15.179:3300/0 conn(0x7f8a340715a0 0x7f8a3415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:38.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.467+0000 7f8a31400640 1 -- 172.21.15.161:0/1906875211 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688821711 0 0) 0x7f8a341442a0 con 0x7f8a34143ec0 2024-05-22T21:19:38.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.467+0000 7f8a31400640 1 -- 172.21.15.161:0/1906875211 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a18003530 con 0x7f8a34143ec0 2024-05-22T21:19:38.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.467+0000 7f8a32800640 1 -- 172.21.15.161:0/1906875211 >> v2:172.21.15.179:3300/0 conn(0x7f8a340715a0 msgr2=0x7f8a3415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:38.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.467+0000 7f8a32800640 1 --2- 172.21.15.161:0/1906875211 >> v2:172.21.15.179:3300/0 conn(0x7f8a340715a0 0x7f8a3415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.467+0000 7f8a32800640 1 -- 172.21.15.161:0/1906875211 >> v1:172.21.15.161:6789/0 conn(0x7f8a34143ec0 legacy=0x7f8a3406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:38.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.467+0000 7f8a32800640 1 -- 172.21.15.161:0/1906875211 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a340707e0 con 0x7f8a3415ec90 2024-05-22T21:19:38.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a32800640 1 --2- 172.21.15.161:0/1906875211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3415ec90 0x7f8a3415f100 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f8a34072330 tx=0x7f8a2c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec98b9070ad7d5d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:38.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a31400640 1 -- 172.21.15.161:0/1906875211 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0a58c0 con 0x7f8a3415ec90 2024-05-22T21:19:38.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a31400640 1 -- 172.21.15.161:0/1906875211 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c0a5a20 con 0x7f8a3415ec90 2024-05-22T21:19:38.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a31400640 1 -- 172.21.15.161:0/1906875211 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0a5db0 con 0x7f8a3415ec90 2024-05-22T21:19:38.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a38c00640 1 -- 172.21.15.161:0/1906875211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3415ec90 msgr2=0x7f8a3415f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:38.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a38c00640 1 --2- 172.21.15.161:0/1906875211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3415ec90 0x7f8a3415f100 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f8a34072330 tx=0x7f8a2c01c900 comp rx=0 tx=0).stop 2024-05-22T21:19:38.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a38c00640 1 -- 172.21.15.161:0/1906875211 shutdown_connections 2024-05-22T21:19:38.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a38c00640 1 --2- 172.21.15.161:0/1906875211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3415ec90 0x7f8a3415f100 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a38c00640 1 --2- 172.21.15.161:0/1906875211 >> v2:172.21.15.179:3300/0 conn(0x7f8a340715a0 0x7f8a3415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a38c00640 1 -- 172.21.15.161:0/1906875211 >> 172.21.15.161:0/1906875211 conn(0x7f8a340b0bc0 msgr2=0x7f8a340b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:38.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a38c00640 1 -- 172.21.15.161:0/1906875211 shutdown_connections 2024-05-22T21:19:38.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a38c00640 1 -- 172.21.15.161:0/1906875211 wait complete. 2024-05-22T21:19:38.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a38c00640 1 Processor -- start 2024-05-22T21:19:38.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a38c00640 1 -- start start 2024-05-22T21:19:38.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a38c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a34143ec0 0x7f8a3413aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:38.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a38c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3415ec90 0x7f8a3413bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:38.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a38c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a34159e10 0x7f8a341f8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:38.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a38c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3413b9b0 con 0x7f8a3415ec90 2024-05-22T21:19:38.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a38c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3413b110 con 0x7f8a34143ec0 2024-05-22T21:19:38.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.471+0000 7f8a38c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3413b3f0 con 0x7f8a34159e10 2024-05-22T21:19:38.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a29400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3415ec90 0x7f8a3413bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:38.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a29400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3415ec90 0x7f8a3413bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44974/0 (socket says 172.21.15.161:44974) 2024-05-22T21:19:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a29400640 1 -- 172.21.15.161:0/1989774606 learned_addr learned my addr 172.21.15.161:0/1989774606 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a31e00640 1 --2- 172.21.15.161:0/1989774606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a34143ec0 0x7f8a3413aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a32800640 1 --2- 172.21.15.161:0/1989774606 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a34159e10 0x7f8a341f8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a29400640 1 -- 172.21.15.161:0/1989774606 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a34159e10 msgr2=0x7f8a341f8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a29400640 1 --2- 172.21.15.161:0/1989774606 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a34159e10 0x7f8a341f8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a29400640 1 -- 172.21.15.161:0/1989774606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a34143ec0 msgr2=0x7f8a3413aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a29400640 1 --2- 172.21.15.161:0/1989774606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a34143ec0 0x7f8a3413aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a29400640 1 -- 172.21.15.161:0/1989774606 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2c009a70 con 0x7f8a3415ec90 2024-05-22T21:19:38.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a29400640 1 --2- 172.21.15.161:0/1989774606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3415ec90 0x7f8a3413bea0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00d800 tx=0x7f8a1c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:38.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a2aa00640 1 -- 172.21.15.161:0/1989774606 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c00b6c0 con 0x7f8a3415ec90 2024-05-22T21:19:38.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a2aa00640 1 -- 172.21.15.161:0/1989774606 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1c00b820 con 0x7f8a3415ec90 2024-05-22T21:19:38.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a2aa00640 1 -- 172.21.15.161:0/1989774606 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c00bb90 con 0x7f8a3415ec90 2024-05-22T21:19:38.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a38c00640 1 -- 172.21.15.161:0/1989774606 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a3413b6d0 con 0x7f8a3415ec90 2024-05-22T21:19:38.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a38c00640 1 -- 172.21.15.161:0/1989774606 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a3415a2c0 con 0x7f8a3415ec90 2024-05-22T21:19:38.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a38c00640 1 -- 172.21.15.161:0/1989774606 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a3415a480 con 0x7f8a3415ec90 2024-05-22T21:19:38.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a2aa00640 1 -- 172.21.15.161:0/1989774606 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8a1c00bcf0 con 0x7f8a3415ec90 2024-05-22T21:19:38.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.475+0000 7f8a2aa00640 1 --2- 172.21.15.161:0/1989774606 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a10066690 0x7f8a10068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:38.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.479+0000 7f8a2aa00640 1 -- 172.21.15.161:0/1989774606 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8a1c0842c0 con 0x7f8a3415ec90 2024-05-22T21:19:38.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.479+0000 7f8a31e00640 1 --2- 172.21.15.161:0/1989774606 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a10066690 0x7f8a10068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:38.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.479+0000 7f8a31e00640 1 --2- 172.21.15.161:0/1989774606 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a10066690 0x7f8a10068b50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8a240011f0 tx=0x7f8a24007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:38.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.479+0000 7f8a2aa00640 1 -- 172.21.15.161:0/1989774606 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8a3415a480 con 0x7f8a3415ec90 2024-05-22T21:19:38.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.691+0000 7f8a38c00640 1 -- 172.21.15.161:0/1989774606 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f8a3415a450 con 0x7f8a3415ec90 2024-05-22T21:19:38.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.971+0000 7f8a2aa00640 1 -- 172.21.15.161:0/1989774606 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v333) v1 ==== 128+0+0 (secure 0 0 0) 0x7f8a1c0519d0 con 0x7f8a3415ec90 2024-05-22T21:19:38.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.975+0000 7f8a38c00640 1 -- 172.21.15.161:0/1989774606 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a10066690 msgr2=0x7f8a10068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:38.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.975+0000 7f8a38c00640 1 --2- 172.21.15.161:0/1989774606 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a10066690 0x7f8a10068b50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8a240011f0 tx=0x7f8a24007d40 comp rx=0 tx=0).stop 2024-05-22T21:19:38.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.975+0000 7f8a38c00640 1 -- 172.21.15.161:0/1989774606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3415ec90 msgr2=0x7f8a3413bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:38.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.975+0000 7f8a38c00640 1 --2- 172.21.15.161:0/1989774606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3415ec90 0x7f8a3413bea0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00d800 tx=0x7f8a1c00dcc0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.975+0000 7f8a38c00640 1 -- 172.21.15.161:0/1989774606 shutdown_connections 2024-05-22T21:19:38.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.975+0000 7f8a38c00640 1 --2- 172.21.15.161:0/1989774606 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8a10066690 0x7f8a10068b50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.975+0000 7f8a38c00640 1 --2- 172.21.15.161:0/1989774606 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a34159e10 0x7f8a341f8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.975+0000 7f8a38c00640 1 --2- 172.21.15.161:0/1989774606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a3415ec90 0x7f8a3413bea0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.975+0000 7f8a38c00640 1 --2- 172.21.15.161:0/1989774606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a34143ec0 0x7f8a3413aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:38.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.975+0000 7f8a38c00640 1 -- 172.21.15.161:0/1989774606 >> 172.21.15.161:0/1989774606 conn(0x7f8a340b0bc0 msgr2=0x7f8a3406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:38.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.975+0000 7f8a38c00640 1 -- 172.21.15.161:0/1989774606 shutdown_connections 2024-05-22T21:19:38.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:38.975+0000 7f8a38c00640 1 -- 172.21.15.161:0/1989774606 wait complete. 2024-05-22T21:19:38.997 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2024-05-22T21:19:38.997 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) 2024-05-22T21:19:38.997 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2024-05-22T21:19:39.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.131+0000 7fa8e8e00640 1 Processor -- start 2024-05-22T21:19:39.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.131+0000 7fa8e8e00640 1 -- start start 2024-05-22T21:19:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.131+0000 7fa8e8e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa8e415dd50 0x7fa8e406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.131+0000 7fa8e8e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8e415d5e0 0x7fa8e415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.131+0000 7fa8e8e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e4071670 con 0x7fa8e415e130 2024-05-22T21:19:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.131+0000 7fa8e8e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e406de40 con 0x7fa8e415dd50 2024-05-22T21:19:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.131+0000 7fa8e8e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e4070a60 con 0x7fa8e415d5e0 2024-05-22T21:19:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.131+0000 7fa8e1e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa8e415dd50 0x7fa8e406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.131+0000 7fa8e1e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa8e415dd50 0x7fa8e406d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44990/0 (socket says 172.21.15.161:44990) 2024-05-22T21:19:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.131+0000 7fa8e1e00640 1 -- 172.21.15.161:0/2184669933 learned_addr learned my addr 172.21.15.161:0/2184669933 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.131+0000 7fa8e2800640 1 --2- 172.21.15.161:0/2184669933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8e415d5e0 0x7fa8e415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:39.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.131+0000 7fa8e1e00640 1 -- 172.21.15.161:0/2184669933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8e415d5e0 msgr2=0x7fa8e415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:39.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.131+0000 7fa8e1e00640 1 --2- 172.21.15.161:0/2184669933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8e415d5e0 0x7fa8e415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:39.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.131+0000 7fa8e1e00640 1 -- 172.21.15.161:0/2184669933 >> v1:172.21.15.179:6789/0 conn(0x7fa8e415e130 legacy=0x7fa8e415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:39.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.131+0000 7fa8e1e00640 1 -- 172.21.15.161:0/2184669933 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e4070d40 con 0x7fa8e415dd50 2024-05-22T21:19:39.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.131+0000 7fa8e2800640 1 --2- 172.21.15.161:0/2184669933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8e415d5e0 0x7fa8e415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:39.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.135+0000 7fa8e1e00640 1 --2- 172.21.15.161:0/2184669933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8e415dd50 0x7fa8e406d900 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009870 tx=0x7fa8d401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c8deb456077825 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:39.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.135+0000 7fa8e1400640 1 -- 172.21.15.161:0/2184669933 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40a95f0 con 0x7fa8e415dd50 2024-05-22T21:19:39.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.135+0000 7fa8e1400640 1 -- 172.21.15.161:0/2184669933 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d40a9750 con 0x7fa8e415dd50 2024-05-22T21:19:39.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.135+0000 7fa8e1400640 1 -- 172.21.15.161:0/2184669933 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40a9ae0 con 0x7fa8e415dd50 2024-05-22T21:19:39.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.135+0000 7fa8e8e00640 1 -- 172.21.15.161:0/2184669933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8e415dd50 msgr2=0x7fa8e406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:39.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.135+0000 7fa8e8e00640 1 --2- 172.21.15.161:0/2184669933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8e415dd50 0x7fa8e406d900 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009870 tx=0x7fa8d401c530 comp rx=0 tx=0).stop 2024-05-22T21:19:39.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.135+0000 7fa8e8e00640 1 -- 172.21.15.161:0/2184669933 shutdown_connections 2024-05-22T21:19:39.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.135+0000 7fa8e8e00640 1 --2- 172.21.15.161:0/2184669933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8e415d5e0 0x7fa8e415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:39.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.135+0000 7fa8e8e00640 1 --2- 172.21.15.161:0/2184669933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8e415dd50 0x7fa8e406d900 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:39.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.135+0000 7fa8e8e00640 1 -- 172.21.15.161:0/2184669933 >> 172.21.15.161:0/2184669933 conn(0x7fa8e40b0bc0 msgr2=0x7fa8e40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:39.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.135+0000 7fa8e8e00640 1 -- 172.21.15.161:0/2184669933 shutdown_connections 2024-05-22T21:19:39.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.135+0000 7fa8e8e00640 1 -- 172.21.15.161:0/2184669933 wait complete. 2024-05-22T21:19:39.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e8e00640 1 Processor -- start 2024-05-22T21:19:39.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e8e00640 1 -- start start 2024-05-22T21:19:39.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e8e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8e415d5e0 0x7fa8e4151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e8e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8e415dd50 0x7fa8e414ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e8e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8e415e130 0x7fa8e40aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e8e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e40ccf60 con 0x7fa8e415d5e0 2024-05-22T21:19:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e8e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e40afe60 con 0x7fa8e415dd50 2024-05-22T21:19:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e8e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e40cc630 con 0x7fa8e415e130 2024-05-22T21:19:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8e415d5e0 0x7fa8e4151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8e415d5e0 0x7fa8e4151000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45006/0 (socket says 172.21.15.161:45006) 2024-05-22T21:19:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e1e00640 1 -- 172.21.15.161:0/2797329616 learned_addr learned my addr 172.21.15.161:0/2797329616 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e2800640 1 --2- 172.21.15.161:0/2797329616 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8e415e130 0x7fa8e40aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8dbe00640 1 --2- 172.21.15.161:0/2797329616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8e415dd50 0x7fa8e414ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:39.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e1e00640 1 -- 172.21.15.161:0/2797329616 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8e415e130 msgr2=0x7fa8e40aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:39.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e1e00640 1 --2- 172.21.15.161:0/2797329616 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8e415e130 0x7fa8e40aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:39.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e1e00640 1 -- 172.21.15.161:0/2797329616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8e415dd50 msgr2=0x7fa8e414ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:39.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e1e00640 1 --2- 172.21.15.161:0/2797329616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8e415dd50 0x7fa8e414ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:39.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e1e00640 1 -- 172.21.15.161:0/2797329616 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d40094d0 con 0x7fa8e415d5e0 2024-05-22T21:19:39.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e2800640 1 --2- 172.21.15.161:0/2797329616 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8e415e130 0x7fa8e40aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:39.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e1e00640 1 --2- 172.21.15.161:0/2797329616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8e415d5e0 0x7fa8e4151000 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40ab040 tx=0x7fa8d401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:39.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8da000640 1 -- 172.21.15.161:0/2797329616 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40a95f0 con 0x7fa8e415d5e0 2024-05-22T21:19:39.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8da000640 1 -- 172.21.15.161:0/2797329616 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d40b1820 con 0x7fa8e415d5e0 2024-05-22T21:19:39.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e8e00640 1 -- 172.21.15.161:0/2797329616 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e40cc910 con 0x7fa8e415d5e0 2024-05-22T21:19:39.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8da000640 1 -- 172.21.15.161:0/2797329616 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40b1bd0 con 0x7fa8e415d5e0 2024-05-22T21:19:39.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e8e00640 1 -- 172.21.15.161:0/2797329616 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e40afb10 con 0x7fa8e415d5e0 2024-05-22T21:19:39.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.139+0000 7fa8e8e00640 1 -- 172.21.15.161:0/2797329616 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e4067220 con 0x7fa8e415d5e0 2024-05-22T21:19:39.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.143+0000 7fa8da000640 1 -- 172.21.15.161:0/2797329616 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa8d40b1d30 con 0x7fa8e415d5e0 2024-05-22T21:19:39.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.143+0000 7fa8da000640 1 --2- 172.21.15.161:0/2797329616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8c0066690 0x7fa8c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:39.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.143+0000 7fa8da000640 1 -- 172.21.15.161:0/2797329616 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa8d411eca0 con 0x7fa8e415d5e0 2024-05-22T21:19:39.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.143+0000 7fa8dbe00640 1 --2- 172.21.15.161:0/2797329616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8c0066690 0x7fa8c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:39.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.143+0000 7fa8dbe00640 1 --2- 172.21.15.161:0/2797329616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8c0066690 0x7fa8c0068b50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4067bb0 tx=0x7fa8cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:39.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.147+0000 7fa8da000640 1 -- 172.21.15.161:0/2797329616 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa8d40ec1e0 con 0x7fa8e415d5e0 2024-05-22T21:19:39.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.355+0000 7fa8e8e00640 1 -- 172.21.15.161:0/2797329616 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7fa8e406c310 con 0x7fa8e415d5e0 2024-05-22T21:19:39.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.975+0000 7fa8da000640 1 -- 172.21.15.161:0/2797329616 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v334) v1 ==== 143+0+0 (secure 0 0 0) 0x7fa8d40ef800 con 0x7fa8e415d5e0 2024-05-22T21:19:39.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.979+0000 7fa8e8e00640 1 -- 172.21.15.161:0/2797329616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8c0066690 msgr2=0x7fa8c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:39.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.979+0000 7fa8e8e00640 1 --2- 172.21.15.161:0/2797329616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8c0066690 0x7fa8c0068b50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4067bb0 tx=0x7fa8cc009040 comp rx=0 tx=0).stop 2024-05-22T21:19:39.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.979+0000 7fa8e8e00640 1 -- 172.21.15.161:0/2797329616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8e415d5e0 msgr2=0x7fa8e4151000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:39.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.979+0000 7fa8e8e00640 1 --2- 172.21.15.161:0/2797329616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8e415d5e0 0x7fa8e4151000 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40ab040 tx=0x7fa8d401ad70 comp rx=0 tx=0).stop 2024-05-22T21:19:39.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.979+0000 7fa8e8e00640 1 -- 172.21.15.161:0/2797329616 shutdown_connections 2024-05-22T21:19:39.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.979+0000 7fa8e8e00640 1 --2- 172.21.15.161:0/2797329616 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8c0066690 0x7fa8c0068b50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:39.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.979+0000 7fa8e8e00640 1 --2- 172.21.15.161:0/2797329616 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8e415e130 0x7fa8e40aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:39.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.979+0000 7fa8e8e00640 1 --2- 172.21.15.161:0/2797329616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8e415dd50 0x7fa8e414ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:39.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.979+0000 7fa8e8e00640 1 --2- 172.21.15.161:0/2797329616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8e415d5e0 0x7fa8e4151000 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:39.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.979+0000 7fa8e8e00640 1 -- 172.21.15.161:0/2797329616 >> 172.21.15.161:0/2797329616 conn(0x7fa8e40b0bc0 msgr2=0x7fa8e40b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:39.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.979+0000 7fa8e8e00640 1 -- 172.21.15.161:0/2797329616 shutdown_connections 2024-05-22T21:19:39.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:39.979+0000 7fa8e8e00640 1 -- 172.21.15.161:0/2797329616 wait complete. 2024-05-22T21:19:40.003 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-22T21:19:40.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.139+0000 7f452be00640 1 Processor -- start 2024-05-22T21:19:40.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.139+0000 7f452be00640 1 -- start start 2024-05-22T21:19:40.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.143+0000 7f452be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f45241402e0 0x7f452413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:40.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.143+0000 7f452be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f452413fc80 0x7f452406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:40.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.143+0000 7f452be00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45241400b0 con 0x7f452413cfc0 2024-05-22T21:19:40.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.143+0000 7f452be00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452413d3a0 con 0x7f45241402e0 2024-05-22T21:19:40.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.143+0000 7f452be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45240cd920 con 0x7f452413fc80 2024-05-22T21:19:40.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.143+0000 7f4528a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f45241402e0 0x7f452413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:40.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.143+0000 7f4528a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f45241402e0 0x7f452413f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45008/0 (socket says 172.21.15.161:45008) 2024-05-22T21:19:40.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.143+0000 7f4528a00640 1 -- 172.21.15.161:0/950263420 learned_addr learned my addr 172.21.15.161:0/950263420 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:40.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.143+0000 7f4529e00640 1 --2- 172.21.15.161:0/950263420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f452413fc80 0x7f452406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:40.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.143+0000 7f4528a00640 1 -- 172.21.15.161:0/950263420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f452413fc80 msgr2=0x7f452406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:40.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.143+0000 7f4528a00640 1 --2- 172.21.15.161:0/950263420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f452413fc80 0x7f452406fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.143+0000 7f4528a00640 1 -- 172.21.15.161:0/950263420 >> v1:172.21.15.179:6789/0 conn(0x7f452413cfc0 legacy=0x7f452415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:40.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.143+0000 7f4528a00640 1 -- 172.21.15.161:0/950263420 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45240cdc00 con 0x7f45241402e0 2024-05-22T21:19:40.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.143+0000 7f4523e00640 1 -- 172.21.15.161:0/950263420 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2868715549 0 0) 0x7f45241400b0 con 0x7f452413cfc0 2024-05-22T21:19:40.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.147+0000 7f4528a00640 1 --2- 172.21.15.161:0/950263420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45241402e0 0x7f452413f740 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f4510009870 tx=0x7f451001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f24c0d5af4f5c96 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:40.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.147+0000 7f4523e00640 1 -- 172.21.15.161:0/950263420 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45100ab600 con 0x7f45241402e0 2024-05-22T21:19:40.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.147+0000 7f4523e00640 1 -- 172.21.15.161:0/950263420 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45100a7050 con 0x7f45241402e0 2024-05-22T21:19:40.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.147+0000 7f4523e00640 1 -- 172.21.15.161:0/950263420 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45100ab980 con 0x7f45241402e0 2024-05-22T21:19:40.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.147+0000 7f452be00640 1 -- 172.21.15.161:0/950263420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45241402e0 msgr2=0x7f452413f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:40.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.147+0000 7f452be00640 1 --2- 172.21.15.161:0/950263420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45241402e0 0x7f452413f740 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f4510009870 tx=0x7f451001c4a0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.147+0000 7f452be00640 1 -- 172.21.15.161:0/950263420 shutdown_connections 2024-05-22T21:19:40.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.147+0000 7f452be00640 1 --2- 172.21.15.161:0/950263420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f452413fc80 0x7f452406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.147+0000 7f452be00640 1 --2- 172.21.15.161:0/950263420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45241402e0 0x7f452413f740 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.147+0000 7f452be00640 1 -- 172.21.15.161:0/950263420 >> 172.21.15.161:0/950263420 conn(0x7f452406a2d0 msgr2=0x7f452406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:40.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.147+0000 7f452be00640 1 -- 172.21.15.161:0/950263420 shutdown_connections 2024-05-22T21:19:40.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.147+0000 7f452be00640 1 -- 172.21.15.161:0/950263420 wait complete. 2024-05-22T21:19:40.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.147+0000 7f452be00640 1 Processor -- start 2024-05-22T21:19:40.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.147+0000 7f452be00640 1 -- start start 2024-05-22T21:19:40.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f452be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f452413cfc0 0x7f452413b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:40.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f452be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f452413fc80 0x7f452413e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:40.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f452be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45241402e0 0x7f4524141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:40.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f452be00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45241418d0 con 0x7f452413fc80 2024-05-22T21:19:40.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f452be00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4524141bb0 con 0x7f45241402e0 2024-05-22T21:19:40.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f452be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4524141e90 con 0x7f452413cfc0 2024-05-22T21:19:40.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f4528a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f452413fc80 0x7f452413e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:40.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f4528a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f452413fc80 0x7f452413e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45022/0 (socket says 172.21.15.161:45022) 2024-05-22T21:19:40.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f4528a00640 1 -- 172.21.15.161:0/909252503 learned_addr learned my addr 172.21.15.161:0/909252503 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:40.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f4529400640 1 --2- 172.21.15.161:0/909252503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f452413cfc0 0x7f452413b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:40.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f4529e00640 1 --2- 172.21.15.161:0/909252503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45241402e0 0x7f4524141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:40.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f4528a00640 1 -- 172.21.15.161:0/909252503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f452413cfc0 msgr2=0x7f452413b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:40.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f4528a00640 1 --2- 172.21.15.161:0/909252503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f452413cfc0 0x7f452413b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f4528a00640 1 -- 172.21.15.161:0/909252503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45241402e0 msgr2=0x7f4524141080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:40.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f4528a00640 1 --2- 172.21.15.161:0/909252503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45241402e0 0x7f4524141080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f4528a00640 1 -- 172.21.15.161:0/909252503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45100094d0 con 0x7f452413fc80 2024-05-22T21:19:40.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f4529400640 1 --2- 172.21.15.161:0/909252503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f452413cfc0 0x7f452413b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:40.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f4528a00640 1 --2- 172.21.15.161:0/909252503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f452413fc80 0x7f452413e9e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f45100ae040 tx=0x7f4510019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:40.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f4521600640 1 -- 172.21.15.161:0/909252503 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45100b4420 con 0x7f452413fc80 2024-05-22T21:19:40.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f4521600640 1 -- 172.21.15.161:0/909252503 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45100b4580 con 0x7f452413fc80 2024-05-22T21:19:40.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f452be00640 1 -- 172.21.15.161:0/909252503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4524142170 con 0x7f452413fc80 2024-05-22T21:19:40.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f4521600640 1 -- 172.21.15.161:0/909252503 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45100b4930 con 0x7f452413fc80 2024-05-22T21:19:40.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f452be00640 1 -- 172.21.15.161:0/909252503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45241415c0 con 0x7f452413fc80 2024-05-22T21:19:40.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f452be00640 1 -- 172.21.15.161:0/909252503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4524067220 con 0x7f452413fc80 2024-05-22T21:19:40.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.151+0000 7f4521600640 1 -- 172.21.15.161:0/909252503 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f45100b2070 con 0x7f452413fc80 2024-05-22T21:19:40.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.155+0000 7f4521600640 1 --2- 172.21.15.161:0/909252503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4504066690 0x7f4504068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:40.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.155+0000 7f4521600640 1 -- 172.21.15.161:0/909252503 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f451011d2e0 con 0x7f452413fc80 2024-05-22T21:19:40.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.155+0000 7f4529400640 1 --2- 172.21.15.161:0/909252503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4504066690 0x7f4504068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:40.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.155+0000 7f4529400640 1 --2- 172.21.15.161:0/909252503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4504066690 0x7f4504068b50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4518000f70 tx=0x7f4518009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:40.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.159+0000 7f4521600640 1 -- 172.21.15.161:0/909252503 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f45100ea8b0 con 0x7f452413fc80 2024-05-22T21:19:40.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.311+0000 7f452be00640 1 -- 172.21.15.161:0/909252503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4524142620 con 0x7f452413fc80 2024-05-22T21:19:40.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.315+0000 7f4521600640 1 -- 172.21.15.161:0/909252503 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f45100eded0 con 0x7f452413fc80 2024-05-22T21:19:40.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.319+0000 7f452be00640 1 -- 172.21.15.161:0/909252503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4504066690 msgr2=0x7f4504068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:40.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.319+0000 7f452be00640 1 --2- 172.21.15.161:0/909252503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4504066690 0x7f4504068b50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4518000f70 tx=0x7f4518009040 comp rx=0 tx=0).stop 2024-05-22T21:19:40.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.319+0000 7f452be00640 1 -- 172.21.15.161:0/909252503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f452413fc80 msgr2=0x7f452413e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:40.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.319+0000 7f452be00640 1 --2- 172.21.15.161:0/909252503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f452413fc80 0x7f452413e9e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f45100ae040 tx=0x7f4510019240 comp rx=0 tx=0).stop 2024-05-22T21:19:40.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.319+0000 7f452be00640 1 -- 172.21.15.161:0/909252503 shutdown_connections 2024-05-22T21:19:40.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.319+0000 7f452be00640 1 --2- 172.21.15.161:0/909252503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4504066690 0x7f4504068b50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.319+0000 7f452be00640 1 --2- 172.21.15.161:0/909252503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45241402e0 0x7f4524141080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.319+0000 7f452be00640 1 --2- 172.21.15.161:0/909252503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f452413fc80 0x7f452413e9e0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.319+0000 7f452be00640 1 --2- 172.21.15.161:0/909252503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f452413cfc0 0x7f452413b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.319+0000 7f452be00640 1 -- 172.21.15.161:0/909252503 >> 172.21.15.161:0/909252503 conn(0x7f452406a2d0 msgr2=0x7f452406a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:40.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.319+0000 7f452be00640 1 -- 172.21.15.161:0/909252503 shutdown_connections 2024-05-22T21:19:40.322 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.319+0000 7f452be00640 1 -- 172.21.15.161:0/909252503 wait complete. 2024-05-22T21:19:40.339 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:19:40.340 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:19:40.422 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:19:40.519 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:19:40.519 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:19:40.519 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:40.528 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:19:40.528 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:19:40.528 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:40.538 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:19:40.538 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:19:40.538 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:40.582 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:19:40.583 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:19:40.583 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:40.591 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:19:40.592 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-22T21:19:40.592 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:40.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.755+0000 7f91d1c00640 1 Processor -- start 2024-05-22T21:19:40.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.755+0000 7f91d1c00640 1 -- start start 2024-05-22T21:19:40.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91d1c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f91cc143ec0 0x7f91cc15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:40.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91d1c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91cc1442a0 con 0x7f91cc0715a0 2024-05-22T21:19:40.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91d1c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91cc070500 con 0x7f91cc15ec90 2024-05-22T21:19:40.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91d1c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91cc0707e0 con 0x7f91cc143ec0 2024-05-22T21:19:40.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91c2000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f91cc0715a0 0x7f91cc06fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43320/0 (socket says 172.21.15.161:43320) 2024-05-22T21:19:40.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91c2000640 1 -- 172.21.15.161:0/2922585372 learned_addr learned my addr 172.21.15.161:0/2922585372 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:40.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91caa00640 1 --2- 172.21.15.161:0/2922585372 >> v2:172.21.15.179:3300/0 conn(0x7f91cc143ec0 0x7f91cc15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:40.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91ca000640 1 -- 172.21.15.161:0/2922585372 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127626362 0 0) 0x7f91cc1442a0 con 0x7f91cc0715a0 2024-05-22T21:19:40.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91ca000640 1 -- 172.21.15.161:0/2922585372 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b0003530 con 0x7f91cc0715a0 2024-05-22T21:19:40.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91ca000640 1 -- 172.21.15.161:0/2922585372 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1014491720 0 0) 0x7f91b0003530 con 0x7f91cc0715a0 2024-05-22T21:19:40.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91ca000640 1 -- 172.21.15.161:0/2922585372 >> v2:172.21.15.179:3300/0 conn(0x7f91cc143ec0 msgr2=0x7f91cc15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:40.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91ca000640 1 --2- 172.21.15.161:0/2922585372 >> v2:172.21.15.179:3300/0 conn(0x7f91cc143ec0 0x7f91cc15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91ca000640 1 -- 172.21.15.161:0/2922585372 >> v1:172.21.15.179:6789/0 conn(0x7f91cc15ec90 legacy=0x7f91cc0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:40.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91ca000640 1 -- 172.21.15.161:0/2922585372 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc070ac0 con 0x7f91cc0715a0 2024-05-22T21:19:40.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91ca000640 1 -- 172.21.15.161:0/2922585372 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f91b4001680 con 0x7f91cc0715a0 2024-05-22T21:19:40.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91ca000640 1 -- 172.21.15.161:0/2922585372 >> v1:172.21.15.161:6789/0 conn(0x7f91cc0715a0 legacy=0x7f91cc06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:40.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91ca000640 1 --2- 172.21.15.161:0/2922585372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91b0003980 0x7f91b0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:40.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91ca000640 1 --2- 172.21.15.161:0/2922585372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b00047d0 0x7f91b0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:40.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91ca000640 1 --2- 172.21.15.161:0/2922585372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b0005190 0x7f91b000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:40.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91ca000640 1 -- 172.21.15.161:0/2922585372 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0707e0 con 0x7f91b0005190 2024-05-22T21:19:40.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91ca000640 1 -- 172.21.15.161:0/2922585372 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b000c030 con 0x7f91b00047d0 2024-05-22T21:19:40.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91ca000640 1 -- 172.21.15.161:0/2922585372 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b000c1f0 con 0x7f91b0003980 2024-05-22T21:19:40.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91cb400640 1 --2- 172.21.15.161:0/2922585372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b0005190 0x7f91b000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:40.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91c2000640 1 --2- 172.21.15.161:0/2922585372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b00047d0 0x7f91b0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:40.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91caa00640 1 --2- 172.21.15.161:0/2922585372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91b0003980 0x7f91b0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:40.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91cb400640 1 -- 172.21.15.161:0/2922585372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91b0003980 msgr2=0x7f91b0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:40.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91cb400640 1 --2- 172.21.15.161:0/2922585372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91b0003980 0x7f91b0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91cb400640 1 -- 172.21.15.161:0/2922585372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b00047d0 msgr2=0x7f91b0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:40.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91cb400640 1 --2- 172.21.15.161:0/2922585372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b00047d0 0x7f91b0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.759+0000 7f91cb400640 1 -- 172.21.15.161:0/2922585372 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b0003530 con 0x7f91b0005190 2024-05-22T21:19:40.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.763+0000 7f91cb400640 1 -- 172.21.15.161:0/2922585372 reap_dead start 2024-05-22T21:19:40.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.763+0000 7f91cb400640 1 --2- 172.21.15.161:0/2922585372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b0005190 0x7f91b000b990 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f91c4001e70 tx=0x7f91c401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc55927ebaf160e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:40.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.763+0000 7f91ca000640 1 -- 172.21.15.161:0/2922585372 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4003b80 con 0x7f91b0005190 2024-05-22T21:19:40.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.763+0000 7f91ca000640 1 -- 172.21.15.161:0/2922585372 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c4003d00 con 0x7f91b0005190 2024-05-22T21:19:40.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.763+0000 7f91ca000640 1 -- 172.21.15.161:0/2922585372 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4005260 con 0x7f91b0005190 2024-05-22T21:19:40.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.763+0000 7f91d1c00640 1 -- 172.21.15.161:0/2922585372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b0005190 msgr2=0x7f91b000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:40.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.763+0000 7f91d1c00640 1 --2- 172.21.15.161:0/2922585372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b0005190 0x7f91b000b990 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f91c4001e70 tx=0x7f91c401c780 comp rx=0 tx=0).stop 2024-05-22T21:19:40.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.763+0000 7f91d1c00640 1 -- 172.21.15.161:0/2922585372 shutdown_connections 2024-05-22T21:19:40.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.763+0000 7f91d1c00640 1 --2- 172.21.15.161:0/2922585372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b0005190 0x7f91b000b990 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.763+0000 7f91d1c00640 1 -- 172.21.15.161:0/2922585372 >> 172.21.15.161:0/2922585372 conn(0x7f91cc0b0bc0 msgr2=0x7f91cc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:40.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.763+0000 7f91d1c00640 1 -- 172.21.15.161:0/2922585372 shutdown_connections 2024-05-22T21:19:40.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.763+0000 7f91d1c00640 1 -- 172.21.15.161:0/2922585372 wait complete. 2024-05-22T21:19:40.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.763+0000 7f91d1c00640 1 Processor -- start 2024-05-22T21:19:40.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.763+0000 7f91d1c00640 1 -- start start 2024-05-22T21:19:40.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91d1c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b0005190 0x7f91cc14bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:40.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91d1c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91cc0aa0b0 0x7f91cc0a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:40.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91d1c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc0a89e0 0x7f91cc13c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:40.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91d1c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0a9c20 con 0x7f91b0005190 2024-05-22T21:19:40.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91d1c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0a9010 con 0x7f91cc0aa0b0 2024-05-22T21:19:40.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91d1c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0a92f0 con 0x7f91cc0a89e0 2024-05-22T21:19:40.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91caa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b0005190 0x7f91cc14bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:40.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91caa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b0005190 0x7f91cc14bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45042/0 (socket says 172.21.15.161:45042) 2024-05-22T21:19:40.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91caa00640 1 -- 172.21.15.161:0/2975843426 learned_addr learned my addr 172.21.15.161:0/2975843426 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:40.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91c2000640 1 --2- 172.21.15.161:0/2975843426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91cc0aa0b0 0x7f91cc0a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:40.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91cb400640 1 --2- 172.21.15.161:0/2975843426 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc0a89e0 0x7f91cc13c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:40.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91caa00640 1 -- 172.21.15.161:0/2975843426 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc0a89e0 msgr2=0x7f91cc13c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:40.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91caa00640 1 --2- 172.21.15.161:0/2975843426 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc0a89e0 0x7f91cc13c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91caa00640 1 -- 172.21.15.161:0/2975843426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91cc0aa0b0 msgr2=0x7f91cc0a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:40.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91caa00640 1 --2- 172.21.15.161:0/2975843426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91cc0aa0b0 0x7f91cc0a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91caa00640 1 -- 172.21.15.161:0/2975843426 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc0a95d0 con 0x7f91b0005190 2024-05-22T21:19:40.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91cb400640 1 --2- 172.21.15.161:0/2975843426 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc0a89e0 0x7f91cc13c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:40.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91caa00640 1 --2- 172.21.15.161:0/2975843426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b0005190 0x7f91cc14bc50 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f91bc011960 tx=0x7f91bc011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:40.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91c3400640 1 -- 172.21.15.161:0/2975843426 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc00fa30 con 0x7f91b0005190 2024-05-22T21:19:40.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91c3400640 1 -- 172.21.15.161:0/2975843426 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc00fb90 con 0x7f91b0005190 2024-05-22T21:19:40.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91c3400640 1 -- 172.21.15.161:0/2975843426 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc014440 con 0x7f91b0005190 2024-05-22T21:19:40.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91d1c00640 1 -- 172.21.15.161:0/2975843426 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc0a98b0 con 0x7f91b0005190 2024-05-22T21:19:40.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91d1c00640 1 -- 172.21.15.161:0/2975843426 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91cc14c190 con 0x7f91b0005190 2024-05-22T21:19:40.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91d1c00640 1 -- 172.21.15.161:0/2975843426 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91cc0a8dc0 con 0x7f91b0005190 2024-05-22T21:19:40.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.767+0000 7f91c3400640 1 -- 172.21.15.161:0/2975843426 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f91bc0145a0 con 0x7f91b0005190 2024-05-22T21:19:40.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.771+0000 7f91c3400640 1 --2- 172.21.15.161:0/2975843426 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f91a8066690 0x7f91a8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:40.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.771+0000 7f91c3400640 1 -- 172.21.15.161:0/2975843426 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f91bc084760 con 0x7f91b0005190 2024-05-22T21:19:40.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.771+0000 7f91c2000640 1 --2- 172.21.15.161:0/2975843426 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f91a8066690 0x7f91a8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:40.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.771+0000 7f91c2000640 1 --2- 172.21.15.161:0/2975843426 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f91a8066690 0x7f91a8068b50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0ae730 tx=0x7f91b4001180 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:40.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.775+0000 7f91c3400640 1 -- 172.21.15.161:0/2975843426 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f91bc051ca0 con 0x7f91b0005190 2024-05-22T21:19:40.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.959+0000 7f91d1c00640 1 -- 172.21.15.161:0/2975843426 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91cc067220 con 0x7f91b0005190 2024-05-22T21:19:40.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.959+0000 7f91c3400640 1 -- 172.21.15.161:0/2975843426 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f91bc022020 con 0x7f91b0005190 2024-05-22T21:19:40.963 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:40.963 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/3944033857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3944033857},{"type":"v1","addr":"172.21.15.179:6834","nonce":3944033857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6125,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/4216709187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":4216709187},{"type":"v1","addr":"172.21.15.161:6836","nonce":4216709187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6874,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2364917653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":2364917653},{"type":"v1","addr":"172.21.15.179:6835","nonce":2364917653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7506,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2910979887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2910979887},{"type":"v1","addr":"172.21.15.161:6837","nonce":2910979887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[]} 2024-05-22T21:19:40.963 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 56 2024-05-22T21:19:40.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.963+0000 7f91d1c00640 1 -- 172.21.15.161:0/2975843426 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f91a8066690 msgr2=0x7f91a8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:40.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.963+0000 7f91d1c00640 1 --2- 172.21.15.161:0/2975843426 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f91a8066690 0x7f91a8068b50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0ae730 tx=0x7f91b4001180 comp rx=0 tx=0).stop 2024-05-22T21:19:40.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.963+0000 7f91d1c00640 1 -- 172.21.15.161:0/2975843426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b0005190 msgr2=0x7f91cc14bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:40.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.963+0000 7f91d1c00640 1 --2- 172.21.15.161:0/2975843426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b0005190 0x7f91cc14bc50 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f91bc011960 tx=0x7f91bc011d60 comp rx=0 tx=0).stop 2024-05-22T21:19:40.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.963+0000 7f91d1c00640 1 -- 172.21.15.161:0/2975843426 shutdown_connections 2024-05-22T21:19:40.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.963+0000 7f91d1c00640 1 --2- 172.21.15.161:0/2975843426 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f91a8066690 0x7f91a8068b50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.963+0000 7f91d1c00640 1 --2- 172.21.15.161:0/2975843426 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc0a89e0 0x7f91cc13c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.963+0000 7f91d1c00640 1 --2- 172.21.15.161:0/2975843426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91cc0aa0b0 0x7f91cc0a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.963+0000 7f91d1c00640 1 --2- 172.21.15.161:0/2975843426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91b0005190 0x7f91cc14bc50 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:40.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.963+0000 7f91d1c00640 1 -- 172.21.15.161:0/2975843426 >> 172.21.15.161:0/2975843426 conn(0x7f91cc0b0bc0 msgr2=0x7f91cc06d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:40.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.963+0000 7f91d1c00640 1 -- 172.21.15.161:0/2975843426 shutdown_connections 2024-05-22T21:19:40.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:40.963+0000 7f91d1c00640 1 -- 172.21.15.161:0/2975843426 wait complete. 2024-05-22T21:19:40.985 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-22T21:19:40.985 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-22T21:19:40.985 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-22T21:19:40.985 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:19:40.985 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-22T21:19:40.985 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-22T21:19:40.986 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-22T21:19:40.986 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:19:40.986 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-22T21:19:40.986 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-22T21:19:40.986 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-22T21:19:40.986 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:19:40.986 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-22T21:19:40.986 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-22T21:19:40.987 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-22T21:19:40.987 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:19:40.997 INFO:tasks.ceph.mds.a:Stopped 2024-05-22T21:19:40.997 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-22T21:19:40.999 INFO:tasks.ceph.mds.c:Stopped 2024-05-22T21:19:40.999 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-22T21:19:41.000 INFO:tasks.ceph.mds.b:Stopped 2024-05-22T21:19:41.000 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-22T21:19:41.001 INFO:tasks.ceph.mds.d:Stopped 2024-05-22T21:19:41.001 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-22T21:19:41.045 INFO:tasks.ceph.mds.a:Started 2024-05-22T21:19:41.045 DEBUG:teuthology.parallel:result is None 2024-05-22T21:19:41.047 INFO:tasks.ceph.mds.c:Started 2024-05-22T21:19:41.047 DEBUG:teuthology.parallel:result is None 2024-05-22T21:19:41.050 INFO:tasks.ceph.mds.b:Started 2024-05-22T21:19:41.051 DEBUG:teuthology.parallel:result is None 2024-05-22T21:19:41.052 INFO:tasks.ceph.mds.d:Started 2024-05-22T21:19:41.052 DEBUG:teuthology.parallel:result is None 2024-05-22T21:19:41.053 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-22T21:19:41.132 INFO:tasks.ceph.mds.a.smithi161.stdout:starting mds.a at 2024-05-22T21:19:41.133 INFO:tasks.ceph.mds.c.smithi161.stdout:starting mds.c at 2024-05-22T21:19:41.141 INFO:tasks.ceph.mds.b.smithi179.stdout:starting mds.b at 2024-05-22T21:19:41.143 INFO:tasks.ceph.mds.d.smithi179.stdout:starting mds.d at 2024-05-22T21:19:41.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90df600640 1 Processor -- start 2024-05-22T21:19:41.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90df600640 1 -- start start 2024-05-22T21:19:41.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90df600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90d80cce70 0x7f90d80cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:41.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90df600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f90d80ce010 0x7f90d80d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:41.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90df600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d806e0f0 con 0x7f90d806f190 2024-05-22T21:19:41.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90df600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d80cc580 con 0x7f90d80ce010 2024-05-22T21:19:41.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90df600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d806e3d0 con 0x7f90d80cce70 2024-05-22T21:19:41.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90cfe00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f90d806f190 0x7f90d80cd900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:54056/0 (socket says 172.21.15.161:54056) 2024-05-22T21:19:41.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90cfe00640 1 -- 172.21.15.161:0/3058122470 learned_addr learned my addr 172.21.15.161:0/3058122470 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:41.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90dcc00640 1 --2- 172.21.15.161:0/3058122470 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90d80cce70 0x7f90d80cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:41.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90dd600640 1 --2- 172.21.15.161:0/3058122470 >> v2:172.21.15.179:3300/0 conn(0x7f90d80ce010 0x7f90d80d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:41.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90d7e00640 1 -- 172.21.15.161:0/3058122470 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306596661 0 0) 0x7f90d806e0f0 con 0x7f90d806f190 2024-05-22T21:19:41.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90d7e00640 1 -- 172.21.15.161:0/3058122470 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90bc003530 con 0x7f90d806f190 2024-05-22T21:19:41.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90dcc00640 1 -- 172.21.15.161:0/3058122470 >> v2:172.21.15.179:3300/0 conn(0x7f90d80ce010 msgr2=0x7f90d80d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:41.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90dcc00640 1 --2- 172.21.15.161:0/3058122470 >> v2:172.21.15.179:3300/0 conn(0x7f90d80ce010 0x7f90d80d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:41.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90dcc00640 1 -- 172.21.15.161:0/3058122470 >> v1:172.21.15.179:6789/0 conn(0x7f90d806f190 legacy=0x7f90d80cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:41.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90dcc00640 1 -- 172.21.15.161:0/3058122470 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d806e6b0 con 0x7f90d80cce70 2024-05-22T21:19:41.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.203+0000 7f90dd600640 1 --2- 172.21.15.161:0/3058122470 >> v2:172.21.15.179:3300/0 conn(0x7f90d80ce010 0x7f90d80d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:41.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.207+0000 7f90dcc00640 1 --2- 172.21.15.161:0/3058122470 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90d80cce70 0x7f90d80cf160 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f90c80097c0 tx=0x7f90c801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc991f87ff6917ab server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:41.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.207+0000 7f90d7e00640 1 -- 172.21.15.161:0/3058122470 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c80ab670 con 0x7f90d80cce70 2024-05-22T21:19:41.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.207+0000 7f90d7e00640 1 -- 172.21.15.161:0/3058122470 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c80ab7d0 con 0x7f90d80cce70 2024-05-22T21:19:41.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.207+0000 7f90d7e00640 1 -- 172.21.15.161:0/3058122470 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c80abb60 con 0x7f90d80cce70 2024-05-22T21:19:41.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.207+0000 7f90df600640 1 -- 172.21.15.161:0/3058122470 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90d80cce70 msgr2=0x7f90d80cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:41.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.207+0000 7f90df600640 1 --2- 172.21.15.161:0/3058122470 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90d80cce70 0x7f90d80cf160 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f90c80097c0 tx=0x7f90c801c470 comp rx=0 tx=0).stop 2024-05-22T21:19:41.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.207+0000 7f90df600640 1 -- 172.21.15.161:0/3058122470 shutdown_connections 2024-05-22T21:19:41.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.207+0000 7f90df600640 1 --2- 172.21.15.161:0/3058122470 >> v2:172.21.15.179:3300/0 conn(0x7f90d80ce010 0x7f90d80d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:41.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.207+0000 7f90df600640 1 --2- 172.21.15.161:0/3058122470 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90d80cce70 0x7f90d80cf160 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:41.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.207+0000 7f90df600640 1 -- 172.21.15.161:0/3058122470 >> 172.21.15.161:0/3058122470 conn(0x7f90d80d0f20 msgr2=0x7f90d80d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:41.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.207+0000 7f90df600640 1 -- 172.21.15.161:0/3058122470 shutdown_connections 2024-05-22T21:19:41.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.207+0000 7f90df600640 1 -- 172.21.15.161:0/3058122470 wait complete. 2024-05-22T21:19:41.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90df600640 1 Processor -- start 2024-05-22T21:19:41.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90df600640 1 -- start start 2024-05-22T21:19:41.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90df600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d806f190 0x7f90d80c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:41.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90df600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90d80cce70 0x7f90d80ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:41.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90df600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90d80ce010 0x7f90d806c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:41.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90df600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d814d9f0 con 0x7f90d80ce010 2024-05-22T21:19:41.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90df600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d80b3d70 con 0x7f90d806f190 2024-05-22T21:19:41.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90df600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d814b2f0 con 0x7f90d80cce70 2024-05-22T21:19:41.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90cfe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90d80cce70 0x7f90d80ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:41.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90dcc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d806f190 0x7f90d80c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:41.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90dd600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90d80ce010 0x7f90d806c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:41.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90dd600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90d80ce010 0x7f90d806c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45060/0 (socket says 172.21.15.161:45060) 2024-05-22T21:19:41.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90dcc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d806f190 0x7f90d80c9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:52620/0 (socket says 172.21.15.161:52620) 2024-05-22T21:19:41.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90cfe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90d80cce70 0x7f90d80ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:59518/0 (socket says 172.21.15.161:59518) 2024-05-22T21:19:41.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90dd600640 1 -- 172.21.15.161:0/405782009 learned_addr learned my addr 172.21.15.161:0/405782009 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:41.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90dcc00640 1 -- 172.21.15.161:0/405782009 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90d80cce70 msgr2=0x7f90d80ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:41.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90dcc00640 1 --2- 172.21.15.161:0/405782009 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90d80cce70 0x7f90d80ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:41.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90dcc00640 1 -- 172.21.15.161:0/405782009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90d80ce010 msgr2=0x7f90d806c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:41.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90dcc00640 1 --2- 172.21.15.161:0/405782009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90d80ce010 0x7f90d806c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:41.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90dcc00640 1 -- 172.21.15.161:0/405782009 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d0009a70 con 0x7f90d806f190 2024-05-22T21:19:41.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90dcc00640 1 --2- 172.21.15.161:0/405782009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d806f190 0x7f90d80c9de0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f90d806e510 tx=0x7f90c801acb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:41.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90d5600640 1 -- 172.21.15.161:0/405782009 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c80bf610 con 0x7f90d806f190 2024-05-22T21:19:41.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90d5600640 1 -- 172.21.15.161:0/405782009 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c80bf790 con 0x7f90d806f190 2024-05-22T21:19:41.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90df600640 1 -- 172.21.15.161:0/405782009 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c8009420 con 0x7f90d806f190 2024-05-22T21:19:41.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.211+0000 7f90d5600640 1 -- 172.21.15.161:0/405782009 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c80bfb40 con 0x7f90d806f190 2024-05-22T21:19:41.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.215+0000 7f90df600640 1 -- 172.21.15.161:0/405782009 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d814b8b0 con 0x7f90d806f190 2024-05-22T21:19:41.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.215+0000 7f90df600640 1 -- 172.21.15.161:0/405782009 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d80cff70 con 0x7f90d806f190 2024-05-22T21:19:41.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.215+0000 7f90d5600640 1 -- 172.21.15.161:0/405782009 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f90c80bfca0 con 0x7f90d806f190 2024-05-22T21:19:41.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.215+0000 7f90d5600640 1 --2- 172.21.15.161:0/405782009 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90b4066690 0x7f90b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:41.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.215+0000 7f90d5600640 1 -- 172.21.15.161:0/405782009 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f90c811eec0 con 0x7f90d806f190 2024-05-22T21:19:41.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.215+0000 7f90cfe00640 1 --2- 172.21.15.161:0/405782009 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90b4066690 0x7f90b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:41.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.215+0000 7f90cfe00640 1 --2- 172.21.15.161:0/405782009 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90b4066690 0x7f90b4068b50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f90d80b3b60 tx=0x7f90c0007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:41.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.219+0000 7f90d5600640 1 -- 172.21.15.161:0/405782009 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f90c80ec5c0 con 0x7f90d806f190 2024-05-22T21:19:41.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:41.391+0000 7f90df600640 1 -- 172.21.15.161:0/405782009 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f90d8067220 con 0x7f90d806f190 2024-05-22T21:19:42.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.323+0000 7f90d5600640 1 -- 172.21.15.161:0/405782009 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v136) v1 ==== 97+0+0 (secure 0 0 0) 0x7f90c80efbe0 con 0x7f90d806f190 2024-05-22T21:19:42.326 INFO:teuthology.orchestra.run.smithi161.stderr: removed all blocklist entries 2024-05-22T21:19:42.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.327+0000 7f90df600640 1 -- 172.21.15.161:0/405782009 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90b4066690 msgr2=0x7f90b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:42.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.327+0000 7f90df600640 1 --2- 172.21.15.161:0/405782009 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90b4066690 0x7f90b4068b50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f90d80b3b60 tx=0x7f90c0007d40 comp rx=0 tx=0).stop 2024-05-22T21:19:42.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.327+0000 7f90df600640 1 -- 172.21.15.161:0/405782009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d806f190 msgr2=0x7f90d80c9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:42.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.327+0000 7f90df600640 1 --2- 172.21.15.161:0/405782009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d806f190 0x7f90d80c9de0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f90d806e510 tx=0x7f90c801acb0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.327+0000 7f90df600640 1 -- 172.21.15.161:0/405782009 shutdown_connections 2024-05-22T21:19:42.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.327+0000 7f90df600640 1 --2- 172.21.15.161:0/405782009 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f90b4066690 0x7f90b4068b50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.327+0000 7f90df600640 1 --2- 172.21.15.161:0/405782009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90d80ce010 0x7f90d806c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.327+0000 7f90df600640 1 --2- 172.21.15.161:0/405782009 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f90d80cce70 0x7f90d80ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.327+0000 7f90df600640 1 --2- 172.21.15.161:0/405782009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d806f190 0x7f90d80c9de0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.327+0000 7f90df600640 1 -- 172.21.15.161:0/405782009 >> 172.21.15.161:0/405782009 conn(0x7f90d80d0f20 msgr2=0x7f90d80d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:42.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.327+0000 7f90df600640 1 -- 172.21.15.161:0/405782009 shutdown_connections 2024-05-22T21:19:42.329 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.327+0000 7f90df600640 1 -- 172.21.15.161:0/405782009 wait complete. 2024-05-22T21:19:42.350 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-22T21:19:42.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f4176200640 1 Processor -- start 2024-05-22T21:19:42.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f4176200640 1 -- start start 2024-05-22T21:19:42.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f4176200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f417015b280 0x7f41700ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:42.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f4176200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41700afed0 0x7f41700cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:42.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f4176200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41700cfed0 con 0x7f41700cf650 2024-05-22T21:19:42.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f4176200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41700d1e80 con 0x7f417015b280 2024-05-22T21:19:42.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f4176200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41700cfae0 con 0x7f41700afed0 2024-05-22T21:19:42.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f4166a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41700afed0 0x7f41700cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:42.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f416fe00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f41700cf650 0x7f4170099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:54066/0 (socket says 172.21.15.161:54066) 2024-05-22T21:19:42.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f416f400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f417015b280 0x7f41700ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:42.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f416fe00640 1 -- 172.21.15.161:0/4288362727 learned_addr learned my addr 172.21.15.161:0/4288362727 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:42.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f4166a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41700afed0 0x7f41700cf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:59526/0 (socket says 172.21.15.161:59526) 2024-05-22T21:19:42.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f416ea00640 1 -- 172.21.15.161:0/4288362727 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188092569 0 0) 0x7f41680014a0 con 0x7f41700cf650 2024-05-22T21:19:42.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f416ea00640 1 -- 172.21.15.161:0/4288362727 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4154003530 con 0x7f41700cf650 2024-05-22T21:19:42.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f4166a00640 1 -- 172.21.15.161:0/4288362727 >> v2:172.21.15.179:3300/0 conn(0x7f417015b280 msgr2=0x7f41700ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:42.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f4166a00640 1 --2- 172.21.15.161:0/4288362727 >> v2:172.21.15.179:3300/0 conn(0x7f417015b280 0x7f41700ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f4166a00640 1 -- 172.21.15.161:0/4288362727 >> v1:172.21.15.179:6789/0 conn(0x7f41700cf650 legacy=0x7f4170099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:42.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f4166a00640 1 -- 172.21.15.161:0/4288362727 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41700d1ba0 con 0x7f41700afed0 2024-05-22T21:19:42.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.487+0000 7f416f400640 1 --2- 172.21.15.161:0/4288362727 >> v2:172.21.15.179:3300/0 conn(0x7f417015b280 0x7f41700ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:42.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.491+0000 7f4166a00640 1 --2- 172.21.15.161:0/4288362727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41700afed0 0x7f41700cf110 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f41580097c0 tx=0x7f415801c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fca8fbf581df4b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:42.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.491+0000 7f416ea00640 1 -- 172.21.15.161:0/4288362727 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158004230 con 0x7f41700afed0 2024-05-22T21:19:42.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.491+0000 7f416ea00640 1 -- 172.21.15.161:0/4288362727 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41580a7050 con 0x7f41700afed0 2024-05-22T21:19:42.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.491+0000 7f416ea00640 1 -- 172.21.15.161:0/4288362727 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158003620 con 0x7f41700afed0 2024-05-22T21:19:42.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.491+0000 7f4176200640 1 -- 172.21.15.161:0/4288362727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41700afed0 msgr2=0x7f41700cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:42.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.491+0000 7f4176200640 1 --2- 172.21.15.161:0/4288362727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41700afed0 0x7f41700cf110 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f41580097c0 tx=0x7f415801c350 comp rx=0 tx=0).stop 2024-05-22T21:19:42.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.491+0000 7f4176200640 1 -- 172.21.15.161:0/4288362727 shutdown_connections 2024-05-22T21:19:42.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.491+0000 7f4176200640 1 --2- 172.21.15.161:0/4288362727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41700afed0 0x7f41700cf110 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.491+0000 7f4176200640 1 --2- 172.21.15.161:0/4288362727 >> v2:172.21.15.179:3300/0 conn(0x7f417015b280 0x7f41700ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.491+0000 7f4176200640 1 -- 172.21.15.161:0/4288362727 >> 172.21.15.161:0/4288362727 conn(0x7f41700d0f10 msgr2=0x7f41700d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:42.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.491+0000 7f4176200640 1 -- 172.21.15.161:0/4288362727 shutdown_connections 2024-05-22T21:19:42.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.491+0000 7f4176200640 1 -- 172.21.15.161:0/4288362727 wait complete. 2024-05-22T21:19:42.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f4176200640 1 Processor -- start 2024-05-22T21:19:42.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f4176200640 1 -- start start 2024-05-22T21:19:42.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f4176200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41700afed0 0x7f417015a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:42.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f4176200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41700cf650 0x7f41700b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:42.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f4176200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f417015b280 0x7f417015b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:42.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f4176200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41700c7700 con 0x7f417015b280 2024-05-22T21:19:42.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f4176200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41700c6c10 con 0x7f41700afed0 2024-05-22T21:19:42.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f4176200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41700c4ae0 con 0x7f41700cf650 2024-05-22T21:19:42.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f416fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f417015b280 0x7f417015b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:42.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f416fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f417015b280 0x7f417015b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45072/0 (socket says 172.21.15.161:45072) 2024-05-22T21:19:42.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f416fe00640 1 -- 172.21.15.161:0/4153923064 learned_addr learned my addr 172.21.15.161:0/4153923064 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:42.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f416f400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41700afed0 0x7f417015a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:42.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f4166a00640 1 --2- 172.21.15.161:0/4153923064 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41700cf650 0x7f41700b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:42.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f416fe00640 1 -- 172.21.15.161:0/4153923064 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41700cf650 msgr2=0x7f41700b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:42.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f416fe00640 1 --2- 172.21.15.161:0/4153923064 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41700cf650 0x7f41700b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f416fe00640 1 -- 172.21.15.161:0/4153923064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41700afed0 msgr2=0x7f417015a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:42.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f416fe00640 1 --2- 172.21.15.161:0/4153923064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41700afed0 0x7f417015a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f416fe00640 1 -- 172.21.15.161:0/4153923064 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4160009420 con 0x7f417015b280 2024-05-22T21:19:42.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f416fe00640 1 --2- 172.21.15.161:0/4153923064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f417015b280 0x7f417015b960 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f416800cae0 tx=0x7f416800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:42.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f4167e00640 1 -- 172.21.15.161:0/4153923064 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416800ac40 con 0x7f417015b280 2024-05-22T21:19:42.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f4167e00640 1 -- 172.21.15.161:0/4153923064 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416800ada0 con 0x7f417015b280 2024-05-22T21:19:42.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f4167e00640 1 -- 172.21.15.161:0/4153923064 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168012620 con 0x7f417015b280 2024-05-22T21:19:42.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f4176200640 1 -- 172.21.15.161:0/4153923064 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4158009420 con 0x7f417015b280 2024-05-22T21:19:42.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f4176200640 1 -- 172.21.15.161:0/4153923064 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417015a770 con 0x7f417015b280 2024-05-22T21:19:42.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.495+0000 7f4176200640 1 -- 172.21.15.161:0/4153923064 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41700c52e0 con 0x7f417015b280 2024-05-22T21:19:42.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.499+0000 7f4167e00640 1 -- 172.21.15.161:0/4153923064 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4168007070 con 0x7f417015b280 2024-05-22T21:19:42.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.499+0000 7f4167e00640 1 --2- 172.21.15.161:0/4153923064 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f414c066690 0x7f414c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:42.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.499+0000 7f4167e00640 1 -- 172.21.15.161:0/4153923064 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4168083cb0 con 0x7f417015b280 2024-05-22T21:19:42.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.499+0000 7f416f400640 1 --2- 172.21.15.161:0/4153923064 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f414c066690 0x7f414c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:42.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.499+0000 7f416f400640 1 --2- 172.21.15.161:0/4153923064 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f414c066690 0x7f414c068b50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4160009c50 tx=0x7f4160009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:42.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.503+0000 7f4167e00640 1 -- 172.21.15.161:0/4153923064 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f41700c52e0 con 0x7f417015b280 2024-05-22T21:19:42.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.711+0000 7f4176200640 1 -- 172.21.15.161:0/4153923064 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f41700cfed0 con 0x7f417015b280 2024-05-22T21:19:42.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.711+0000 7f4167e00640 1 -- 172.21.15.161:0/4153923064 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f41700cfed0 con 0x7f417015b280 2024-05-22T21:19:42.715 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:42.715 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:19:42.715 INFO:teuthology.orchestra.run.smithi161.stdout: "auth_dump": [ 2024-05-22T21:19:42.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.716 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.a", 2024-05-22T21:19:42.716 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m00QTFRAAPSRrpBRY75FlcWf3QxOOfg==", 2024-05-22T21:19:42.716 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.716 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:19:42.716 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:19:42.716 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:19:42.716 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:42.716 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.716 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.716 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.b", 2024-05-22T21:19:42.716 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m/ipAIRAA4gj5jGqTpNCr1K2OU/bsAQ==", 2024-05-22T21:19:42.717 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.717 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:19:42.717 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:19:42.717 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:19:42.717 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:42.717 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.717 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.717 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.c", 2024-05-22T21:19:42.717 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mWlM2HBAAEmJJWMDlTG9E7Qzld2gd5w==", 2024-05-22T21:19:42.717 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.717 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:19:42.718 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:19:42.718 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:19:42.718 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:42.718 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.718 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.718 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.d", 2024-05-22T21:19:42.718 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mmsuaKBAAqpCYRZZUGK5ZuxflUBcPgg==", 2024-05-22T21:19:42.718 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.718 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:19:42.718 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:19:42.718 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:19:42.718 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:42.719 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.719 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.0", 2024-05-22T21:19:42.719 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05mJZokLhAAh2vbQnRX25iYUIbvVmURuQ==", 2024-05-22T21:19:42.719 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.719 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:19:42.719 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:19:42.719 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:42.719 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.719 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.719 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.1", 2024-05-22T21:19:42.720 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05m8kWNNhAA6aaVT2xshFkqnk4G4gOttA==", 2024-05-22T21:19:42.720 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.720 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:19:42.720 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:19:42.720 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:42.720 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.720 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.720 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.720 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.2", 2024-05-22T21:19:42.720 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05ml1k9AxAAtL3XaJJI6cilpmvurh18Pw==", 2024-05-22T21:19:42.720 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.720 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:19:42.721 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:19:42.721 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:42.721 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.721 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.721 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.3", 2024-05-22T21:19:42.721 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05mbNBgCxAAas1L+QR4Aez+H1G0EVltfg==", 2024-05-22T21:19:42.721 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.722 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:19:42.722 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:19:42.722 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:42.722 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.722 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.722 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.722 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.4", 2024-05-22T21:19:42.722 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mOPLPABAAZBGIMF/FasNB27n/7di0AA==", 2024-05-22T21:19:42.722 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.722 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:19:42.722 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:19:42.722 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:42.723 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.723 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.5", 2024-05-22T21:19:42.723 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mT1poCRAASXm4tdP01pA1VOviqeoLVw==", 2024-05-22T21:19:42.723 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.723 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:19:42.723 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:19:42.723 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:42.723 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.723 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.723 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.6", 2024-05-22T21:19:42.724 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05m1ZPRERAA2S7WeTwysLCZm1OB/qCrAA==", 2024-05-22T21:19:42.724 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.724 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:19:42.724 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:19:42.724 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:42.724 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.724 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.724 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.7", 2024-05-22T21:19:42.724 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mCrUhGhAAvXoFWV7X3303rfXaxmfWqA==", 2024-05-22T21:19:42.724 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.725 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:19:42.725 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:19:42.725 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:42.725 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.725 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.0", 2024-05-22T21:19:42.725 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mh9B7LBAAevhvj5KpwP9pEIpJcxUvFA==", 2024-05-22T21:19:42.725 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.725 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:19:42.725 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow r", 2024-05-22T21:19:42.725 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-22T21:19:42.725 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.726 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.1", 2024-05-22T21:19:42.726 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mMnHtNRAAg/a/Mb1oRqx73d9efIHCfA==", 2024-05-22T21:19:42.726 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.726 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:19:42.726 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:19:42.726 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:19:42.726 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:19:42.726 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.726 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.726 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.726 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.2", 2024-05-22T21:19:42.727 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mOzdkMhAAbTiNdvyxqIyrluMiZW0cOw==", 2024-05-22T21:19:42.727 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.727 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:19:42.727 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:19:42.727 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:19:42.727 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:19:42.727 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.727 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.727 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.3", 2024-05-22T21:19:42.727 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDYX05mr4oxABAA1+bQQbxhoeU+kb0OCa+Wyw==", 2024-05-22T21:19:42.727 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.727 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:19:42.727 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:19:42.728 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:19:42.728 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:19:42.728 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.728 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.728 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.admin", 2024-05-22T21:19:42.728 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDWX05mHZY0NRAAiBvMDkrCZ+MgG2iwBH+bcg==", 2024-05-22T21:19:42.728 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.728 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:19:42.728 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:19:42.728 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:19:42.728 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:42.728 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.729 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.x", 2024-05-22T21:19:42.729 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mH2kuERAAgigMUWK3M3WaMXlqGUcNrw==", 2024-05-22T21:19:42.729 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.729 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:19:42.729 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:19:42.729 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:42.729 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.729 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:42.729 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:42.729 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.y", 2024-05-22T21:19:42.729 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mZRdiDRAA0vUwD+rPSfyhjgA+9XPp5w==", 2024-05-22T21:19:42.729 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:19:42.730 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:19:42.730 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:19:42.730 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:19:42.730 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.730 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:42.730 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:42.730 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:19:42.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.715+0000 7f4176200640 1 -- 172.21.15.161:0/4153923064 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f414c066690 msgr2=0x7f414c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:42.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.715+0000 7f4176200640 1 --2- 172.21.15.161:0/4153923064 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f414c066690 0x7f414c068b50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4160009c50 tx=0x7f4160009190 comp rx=0 tx=0).stop 2024-05-22T21:19:42.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.715+0000 7f4176200640 1 -- 172.21.15.161:0/4153923064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f417015b280 msgr2=0x7f417015b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:42.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.715+0000 7f4176200640 1 --2- 172.21.15.161:0/4153923064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f417015b280 0x7f417015b960 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f416800cae0 tx=0x7f416800a060 comp rx=0 tx=0).stop 2024-05-22T21:19:42.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.715+0000 7f4176200640 1 -- 172.21.15.161:0/4153923064 shutdown_connections 2024-05-22T21:19:42.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.715+0000 7f4176200640 1 --2- 172.21.15.161:0/4153923064 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f414c066690 0x7f414c068b50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.715+0000 7f4176200640 1 --2- 172.21.15.161:0/4153923064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f417015b280 0x7f417015b960 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.715+0000 7f4176200640 1 --2- 172.21.15.161:0/4153923064 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41700cf650 0x7f41700b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.715+0000 7f4176200640 1 --2- 172.21.15.161:0/4153923064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41700afed0 0x7f417015a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.715+0000 7f4176200640 1 -- 172.21.15.161:0/4153923064 >> 172.21.15.161:0/4153923064 conn(0x7f41700d0f10 msgr2=0x7f4170099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:42.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.715+0000 7f4176200640 1 -- 172.21.15.161:0/4153923064 shutdown_connections 2024-05-22T21:19:42.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.715+0000 7f4176200640 1 -- 172.21.15.161:0/4153923064 wait complete. 2024-05-22T21:19:42.737 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:42.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.871+0000 7f4630200640 1 Processor -- start 2024-05-22T21:19:42.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.871+0000 7f4630200640 1 -- start start 2024-05-22T21:19:42.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.871+0000 7f4630200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f462815dd50 0x7f462806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:42.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.871+0000 7f4630200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f462815d5e0 0x7f462815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:42.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.871+0000 7f4630200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4628071670 con 0x7f462815e130 2024-05-22T21:19:42.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.871+0000 7f4630200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f462806de40 con 0x7f462815d5e0 2024-05-22T21:19:42.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.871+0000 7f4630200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4628070a60 con 0x7f462815dd50 2024-05-22T21:19:42.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.871+0000 7f462ce00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f462815e130 0x7f462815ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43338/0 (socket says 172.21.15.161:43338) 2024-05-22T21:19:42.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.871+0000 7f462e200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f462815d5e0 0x7f462815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:42.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.871+0000 7f462ce00640 1 -- 172.21.15.161:0/2536343143 learned_addr learned my addr 172.21.15.161:0/2536343143 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:42.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.871+0000 7f462d800640 1 --2- 172.21.15.161:0/2536343143 >> v2:172.21.15.179:3300/0 conn(0x7f462815dd50 0x7f462806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:42.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.871+0000 7f462e200640 1 -- 172.21.15.161:0/2536343143 >> v2:172.21.15.179:3300/0 conn(0x7f462815dd50 msgr2=0x7f462806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:42.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.871+0000 7f462e200640 1 --2- 172.21.15.161:0/2536343143 >> v2:172.21.15.179:3300/0 conn(0x7f462815dd50 0x7f462806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.871+0000 7f462e200640 1 -- 172.21.15.161:0/2536343143 >> v1:172.21.15.161:6789/0 conn(0x7f462815e130 legacy=0x7f462815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:42.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.871+0000 7f4627e00640 1 -- 172.21.15.161:0/2536343143 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171481338 0 0) 0x7f4628071670 con 0x7f462815e130 2024-05-22T21:19:42.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.871+0000 7f462e200640 1 -- 172.21.15.161:0/2536343143 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4628070d40 con 0x7f462815d5e0 2024-05-22T21:19:42.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.875+0000 7f462e200640 1 --2- 172.21.15.161:0/2536343143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462815d5e0 0x7f462815b6c0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f4628072330 tx=0x7f462001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7b79e2e11124ee server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:42.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.875+0000 7f4627e00640 1 -- 172.21.15.161:0/2536343143 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46200a59c0 con 0x7f462815d5e0 2024-05-22T21:19:42.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.875+0000 7f4627e00640 1 -- 172.21.15.161:0/2536343143 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46200a5b20 con 0x7f462815d5e0 2024-05-22T21:19:42.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.875+0000 7f4627e00640 1 -- 172.21.15.161:0/2536343143 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46200ad440 con 0x7f462815d5e0 2024-05-22T21:19:42.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.875+0000 7f4630200640 1 -- 172.21.15.161:0/2536343143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462815d5e0 msgr2=0x7f462815b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:42.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.875+0000 7f4630200640 1 --2- 172.21.15.161:0/2536343143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462815d5e0 0x7f462815b6c0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f4628072330 tx=0x7f462001c900 comp rx=0 tx=0).stop 2024-05-22T21:19:42.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.875+0000 7f4630200640 1 -- 172.21.15.161:0/2536343143 shutdown_connections 2024-05-22T21:19:42.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.875+0000 7f4630200640 1 --2- 172.21.15.161:0/2536343143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462815d5e0 0x7f462815b6c0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.875+0000 7f4630200640 1 --2- 172.21.15.161:0/2536343143 >> v2:172.21.15.179:3300/0 conn(0x7f462815dd50 0x7f462806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.875+0000 7f4630200640 1 -- 172.21.15.161:0/2536343143 >> 172.21.15.161:0/2536343143 conn(0x7f46280b0bc0 msgr2=0x7f46280b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:42.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.875+0000 7f4630200640 1 -- 172.21.15.161:0/2536343143 shutdown_connections 2024-05-22T21:19:42.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.875+0000 7f4630200640 1 -- 172.21.15.161:0/2536343143 wait complete. 2024-05-22T21:19:42.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.875+0000 7f4630200640 1 Processor -- start 2024-05-22T21:19:42.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.875+0000 7f4630200640 1 -- start start 2024-05-22T21:19:42.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f4630200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f462815d5e0 0x7f4628155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:42.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f4630200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462815dd50 0x7f462806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:42.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f4630200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f462815e130 0x7f46280ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:42.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f4630200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46280ad5f0 con 0x7f462815dd50 2024-05-22T21:19:42.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f4630200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46280ac9e0 con 0x7f462815e130 2024-05-22T21:19:42.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f4630200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46280accc0 con 0x7f462815d5e0 2024-05-22T21:19:42.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f462ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462815dd50 0x7f462806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:42.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f462ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462815dd50 0x7f462806d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45088/0 (socket says 172.21.15.161:45088) 2024-05-22T21:19:42.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f462ce00640 1 -- 172.21.15.161:0/970704903 learned_addr learned my addr 172.21.15.161:0/970704903 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:42.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f462d800640 1 --2- 172.21.15.161:0/970704903 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f462815d5e0 0x7f4628155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:42.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f462e200640 1 --2- 172.21.15.161:0/970704903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f462815e130 0x7f46280ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:42.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f462ce00640 1 -- 172.21.15.161:0/970704903 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f462815d5e0 msgr2=0x7f4628155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:42.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f462ce00640 1 --2- 172.21.15.161:0/970704903 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f462815d5e0 0x7f4628155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f462ce00640 1 -- 172.21.15.161:0/970704903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f462815e130 msgr2=0x7f46280ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:42.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f462ce00640 1 --2- 172.21.15.161:0/970704903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f462815e130 0x7f46280ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:42.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f462ce00640 1 -- 172.21.15.161:0/970704903 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4620009a70 con 0x7f462815dd50 2024-05-22T21:19:42.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f462ce00640 1 --2- 172.21.15.161:0/970704903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462815dd50 0x7f462806d4b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f461400c480 tx=0x7f461400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:42.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f4625600640 1 -- 172.21.15.161:0/970704903 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46140074e0 con 0x7f462815dd50 2024-05-22T21:19:42.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f4625600640 1 -- 172.21.15.161:0/970704903 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4614007660 con 0x7f462815dd50 2024-05-22T21:19:42.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f4630200640 1 -- 172.21.15.161:0/970704903 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46280acfa0 con 0x7f462815dd50 2024-05-22T21:19:42.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f4625600640 1 -- 172.21.15.161:0/970704903 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614004a30 con 0x7f462815dd50 2024-05-22T21:19:42.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f4630200640 1 -- 172.21.15.161:0/970704903 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462806d9f0 con 0x7f462815dd50 2024-05-22T21:19:42.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.879+0000 7f4630200640 1 -- 172.21.15.161:0/970704903 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46280ad7a0 con 0x7f462815dd50 2024-05-22T21:19:42.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.883+0000 7f4625600640 1 -- 172.21.15.161:0/970704903 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4614004b90 con 0x7f462815dd50 2024-05-22T21:19:42.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.883+0000 7f4625600640 1 --2- 172.21.15.161:0/970704903 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4608066690 0x7f4608068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:42.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.883+0000 7f4625600640 1 -- 172.21.15.161:0/970704903 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4614082f80 con 0x7f462815dd50 2024-05-22T21:19:42.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.883+0000 7f462d800640 1 --2- 172.21.15.161:0/970704903 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4608066690 0x7f4608068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:42.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.883+0000 7f462d800640 1 --2- 172.21.15.161:0/970704903 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4608066690 0x7f4608068b50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f461c007a60 tx=0x7f461c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:42.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:42.887+0000 7f4625600640 1 -- 172.21.15.161:0/970704903 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f46280ad7a0 con 0x7f462815dd50 2024-05-22T21:19:43.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.043+0000 7f4630200640 1 -- 172.21.15.161:0/970704903 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4628067220 con 0x7f462815dd50 2024-05-22T21:19:43.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.043+0000 7f4625600640 1 -- 172.21.15.161:0/970704903 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+12448 (secure 0 0 0) 0x7f4614018020 con 0x7f462815dd50 2024-05-22T21:19:43.044 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:43.045 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":136,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:19:42.319845+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:43.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.043+0000 7f4630200640 1 -- 172.21.15.161:0/970704903 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4608066690 msgr2=0x7f4608068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.043+0000 7f4630200640 1 --2- 172.21.15.161:0/970704903 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4608066690 0x7f4608068b50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f461c007a60 tx=0x7f461c008040 comp rx=0 tx=0).stop 2024-05-22T21:19:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.043+0000 7f4630200640 1 -- 172.21.15.161:0/970704903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462815dd50 msgr2=0x7f462806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.043+0000 7f4630200640 1 --2- 172.21.15.161:0/970704903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462815dd50 0x7f462806d4b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f461400c480 tx=0x7f461400c940 comp rx=0 tx=0).stop 2024-05-22T21:19:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.043+0000 7f4630200640 1 -- 172.21.15.161:0/970704903 shutdown_connections 2024-05-22T21:19:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.043+0000 7f4630200640 1 --2- 172.21.15.161:0/970704903 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4608066690 0x7f4608068b50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.043+0000 7f4630200640 1 --2- 172.21.15.161:0/970704903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f462815e130 0x7f46280ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.043+0000 7f4630200640 1 --2- 172.21.15.161:0/970704903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f462815dd50 0x7f462806d4b0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.043+0000 7f4630200640 1 --2- 172.21.15.161:0/970704903 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f462815d5e0 0x7f4628155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.043+0000 7f4630200640 1 -- 172.21.15.161:0/970704903 >> 172.21.15.161:0/970704903 conn(0x7f46280b0bc0 msgr2=0x7f46280aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.043+0000 7f4630200640 1 -- 172.21.15.161:0/970704903 shutdown_connections 2024-05-22T21:19:43.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.043+0000 7f4630200640 1 -- 172.21.15.161:0/970704903 wait complete. 2024-05-22T21:19:43.065 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:19:43.065 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:19:43.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd3fc00640 1 Processor -- start 2024-05-22T21:19:43.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd3fc00640 1 -- start start 2024-05-22T21:19:43.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd3fc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd38143c40 0x7efd3815b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd3fc00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efd381444a0 0x7efd3806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd3fc00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3815efb0 con 0x7efd38144900 2024-05-22T21:19:43.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd3fc00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3815f290 con 0x7efd381444a0 2024-05-22T21:19:43.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd3fc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3815f570 con 0x7efd38143c40 2024-05-22T21:19:43.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd3dc00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7efd38144900 0x7efd380b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:54082/0 (socket says 172.21.15.161:54082) 2024-05-22T21:19:43.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd3d200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd38143c40 0x7efd3815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd3dc00640 1 -- 172.21.15.161:0/921286288 learned_addr learned my addr 172.21.15.161:0/921286288 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:43.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd37e00640 1 --2- 172.21.15.161:0/921286288 >> v2:172.21.15.179:3300/0 conn(0x7efd381444a0 0x7efd3806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd37400640 1 -- 172.21.15.161:0/921286288 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751354594 0 0) 0x7efd3815efb0 con 0x7efd38144900 2024-05-22T21:19:43.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd37400640 1 -- 172.21.15.161:0/921286288 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd20003530 con 0x7efd38144900 2024-05-22T21:19:43.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd3d200640 1 -- 172.21.15.161:0/921286288 >> v2:172.21.15.179:3300/0 conn(0x7efd381444a0 msgr2=0x7efd3806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:43.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd3d200640 1 --2- 172.21.15.161:0/921286288 >> v2:172.21.15.179:3300/0 conn(0x7efd381444a0 0x7efd3806c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd3d200640 1 -- 172.21.15.161:0/921286288 >> v1:172.21.15.179:6789/0 conn(0x7efd38144900 legacy=0x7efd380b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.203+0000 7efd3d200640 1 -- 172.21.15.161:0/921286288 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3815f850 con 0x7efd38143c40 2024-05-22T21:19:43.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.207+0000 7efd3d200640 1 --2- 172.21.15.161:0/921286288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd38143c40 0x7efd3815b600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efd2c0097c0 tx=0x7efd2c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d0997c4ad71c883 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:43.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.207+0000 7efd37400640 1 -- 172.21.15.161:0/921286288 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c0ac410 con 0x7efd38143c40 2024-05-22T21:19:43.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.207+0000 7efd37400640 1 -- 172.21.15.161:0/921286288 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c0ac570 con 0x7efd38143c40 2024-05-22T21:19:43.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.207+0000 7efd37400640 1 -- 172.21.15.161:0/921286288 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c0ac900 con 0x7efd38143c40 2024-05-22T21:19:43.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.207+0000 7efd3fc00640 1 -- 172.21.15.161:0/921286288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd38143c40 msgr2=0x7efd3815b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.207+0000 7efd3fc00640 1 --2- 172.21.15.161:0/921286288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd38143c40 0x7efd3815b600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efd2c0097c0 tx=0x7efd2c01c4a0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.207+0000 7efd3fc00640 1 -- 172.21.15.161:0/921286288 shutdown_connections 2024-05-22T21:19:43.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.207+0000 7efd3fc00640 1 --2- 172.21.15.161:0/921286288 >> v2:172.21.15.179:3300/0 conn(0x7efd381444a0 0x7efd3806c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.207+0000 7efd3fc00640 1 --2- 172.21.15.161:0/921286288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd38143c40 0x7efd3815b600 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.207+0000 7efd3fc00640 1 -- 172.21.15.161:0/921286288 >> 172.21.15.161:0/921286288 conn(0x7efd3806a2d0 msgr2=0x7efd3806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:43.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.207+0000 7efd3fc00640 1 -- 172.21.15.161:0/921286288 shutdown_connections 2024-05-22T21:19:43.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.207+0000 7efd3fc00640 1 -- 172.21.15.161:0/921286288 wait complete. 2024-05-22T21:19:43.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3fc00640 1 Processor -- start 2024-05-22T21:19:43.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3fc00640 1 -- start start 2024-05-22T21:19:43.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3fc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd38143c40 0x7efd38137f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3fc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd381444a0 0x7efd3815ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3fc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd38144900 0x7efd3815cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3fc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd381559a0 con 0x7efd38143c40 2024-05-22T21:19:43.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3fc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd381564b0 con 0x7efd38144900 2024-05-22T21:19:43.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3fc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd38155b80 con 0x7efd381444a0 2024-05-22T21:19:43.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3d200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd38143c40 0x7efd38137f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3d200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd38143c40 0x7efd38137f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45096/0 (socket says 172.21.15.161:45096) 2024-05-22T21:19:43.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3d200640 1 -- 172.21.15.161:0/1931944296 learned_addr learned my addr 172.21.15.161:0/1931944296 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:43.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd37e00640 1 --2- 172.21.15.161:0/1931944296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd381444a0 0x7efd3815ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3dc00640 1 --2- 172.21.15.161:0/1931944296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd38144900 0x7efd3815cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3d200640 1 -- 172.21.15.161:0/1931944296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd381444a0 msgr2=0x7efd3815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3d200640 1 --2- 172.21.15.161:0/1931944296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd381444a0 0x7efd3815ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3d200640 1 -- 172.21.15.161:0/1931944296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd38144900 msgr2=0x7efd3815cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3d200640 1 --2- 172.21.15.161:0/1931944296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd38144900 0x7efd3815cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3d200640 1 -- 172.21.15.161:0/1931944296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2c009420 con 0x7efd38143c40 2024-05-22T21:19:43.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3dc00640 1 --2- 172.21.15.161:0/1931944296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd38144900 0x7efd3815cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:43.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3d200640 1 --2- 172.21.15.161:0/1931944296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd38143c40 0x7efd38137f70 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7efd2c01a0d0 tx=0x7efd2c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:43.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd34c00640 1 -- 172.21.15.161:0/1931944296 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c006e20 con 0x7efd38143c40 2024-05-22T21:19:43.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd34c00640 1 -- 172.21.15.161:0/1931944296 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c0ace00 con 0x7efd38143c40 2024-05-22T21:19:43.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3fc00640 1 -- 172.21.15.161:0/1931944296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd38155e60 con 0x7efd38143c40 2024-05-22T21:19:43.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd34c00640 1 -- 172.21.15.161:0/1931944296 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c0b5840 con 0x7efd38143c40 2024-05-22T21:19:43.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.211+0000 7efd3fc00640 1 -- 172.21.15.161:0/1931944296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd381384b0 con 0x7efd38143c40 2024-05-22T21:19:43.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.215+0000 7efd3fc00640 1 -- 172.21.15.161:0/1931944296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd38067220 con 0x7efd38143c40 2024-05-22T21:19:43.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.215+0000 7efd34c00640 1 -- 172.21.15.161:0/1931944296 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efd2c0b3040 con 0x7efd38143c40 2024-05-22T21:19:43.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.215+0000 7efd34c00640 1 --2- 172.21.15.161:0/1931944296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd18066690 0x7efd18068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.215+0000 7efd34c00640 1 -- 172.21.15.161:0/1931944296 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efd2c11dbe0 con 0x7efd38143c40 2024-05-22T21:19:43.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.215+0000 7efd37e00640 1 --2- 172.21.15.161:0/1931944296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd18066690 0x7efd18068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.215+0000 7efd37e00640 1 --2- 172.21.15.161:0/1931944296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd18066690 0x7efd18068b50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7efd38165fa0 tx=0x7efd24009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:43.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.219+0000 7efd34c00640 1 -- 172.21.15.161:0/1931944296 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efd2c0eb200 con 0x7efd38143c40 2024-05-22T21:19:43.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.375+0000 7efd3fc00640 1 -- 172.21.15.161:0/1931944296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd380ceaa0 con 0x7efd38143c40 2024-05-22T21:19:43.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.375+0000 7efd34c00640 1 -- 172.21.15.161:0/1931944296 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd2c0b10a0 con 0x7efd38143c40 2024-05-22T21:19:43.377 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:19:43.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.375+0000 7efd3fc00640 1 -- 172.21.15.161:0/1931944296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd18066690 msgr2=0x7efd18068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.375+0000 7efd3fc00640 1 --2- 172.21.15.161:0/1931944296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd18066690 0x7efd18068b50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7efd38165fa0 tx=0x7efd24009190 comp rx=0 tx=0).stop 2024-05-22T21:19:43.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.375+0000 7efd3fc00640 1 -- 172.21.15.161:0/1931944296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd38143c40 msgr2=0x7efd38137f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.375+0000 7efd3fc00640 1 --2- 172.21.15.161:0/1931944296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd38143c40 0x7efd38137f70 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7efd2c01a0d0 tx=0x7efd2c019240 comp rx=0 tx=0).stop 2024-05-22T21:19:43.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.375+0000 7efd3fc00640 1 -- 172.21.15.161:0/1931944296 shutdown_connections 2024-05-22T21:19:43.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.375+0000 7efd3fc00640 1 --2- 172.21.15.161:0/1931944296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efd18066690 0x7efd18068b50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.375+0000 7efd3fc00640 1 --2- 172.21.15.161:0/1931944296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd38144900 0x7efd3815cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.375+0000 7efd3fc00640 1 --2- 172.21.15.161:0/1931944296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd381444a0 0x7efd3815ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.375+0000 7efd3fc00640 1 --2- 172.21.15.161:0/1931944296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd38143c40 0x7efd38137f70 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.375+0000 7efd3fc00640 1 -- 172.21.15.161:0/1931944296 >> 172.21.15.161:0/1931944296 conn(0x7efd3806a2d0 msgr2=0x7efd3815bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:43.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.375+0000 7efd3fc00640 1 -- 172.21.15.161:0/1931944296 shutdown_connections 2024-05-22T21:19:43.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.375+0000 7efd3fc00640 1 -- 172.21.15.161:0/1931944296 wait complete. 2024-05-22T21:19:43.397 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-22T21:19:43.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dcaa00640 1 Processor -- start 2024-05-22T21:19:43.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dcaa00640 1 -- start start 2024-05-22T21:19:43.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dcaa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5dc40715a0 0x7f5dc415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dcaa00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5dc415ec90 0x7f5dc415f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dcaa00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc41442a0 con 0x7f5dc4143ec0 2024-05-22T21:19:43.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dcaa00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc415f640 con 0x7f5dc40715a0 2024-05-22T21:19:43.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dcaa00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc4070500 con 0x7f5dc415ec90 2024-05-22T21:19:43.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dc3e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f5dc4143ec0 0x7f5dc406fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43340/0 (socket says 172.21.15.161:43340) 2024-05-22T21:19:43.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dc3e00640 1 -- 172.21.15.161:0/3759161564 learned_addr learned my addr 172.21.15.161:0/3759161564 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:43.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dbb400640 1 --2- 172.21.15.161:0/3759161564 >> v2:172.21.15.161:3300/0 conn(0x7f5dc40715a0 0x7f5dc415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dc8a00640 1 --2- 172.21.15.161:0/3759161564 >> v2:172.21.15.179:3300/0 conn(0x7f5dc415ec90 0x7f5dc415f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dc3400640 1 -- 172.21.15.161:0/3759161564 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559898310 0 0) 0x7f5dc41442a0 con 0x7f5dc4143ec0 2024-05-22T21:19:43.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dc3400640 1 -- 172.21.15.161:0/3759161564 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5da8003530 con 0x7f5dc4143ec0 2024-05-22T21:19:43.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dbb400640 1 -- 172.21.15.161:0/3759161564 >> v2:172.21.15.179:3300/0 conn(0x7f5dc415ec90 msgr2=0x7f5dc415f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:43.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dbb400640 1 --2- 172.21.15.161:0/3759161564 >> v2:172.21.15.179:3300/0 conn(0x7f5dc415ec90 0x7f5dc415f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dbb400640 1 -- 172.21.15.161:0/3759161564 >> v1:172.21.15.161:6789/0 conn(0x7f5dc4143ec0 legacy=0x7f5dc406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.531+0000 7f5dbb400640 1 -- 172.21.15.161:0/3759161564 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc40707e0 con 0x7f5dc40715a0 2024-05-22T21:19:43.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.535+0000 7f5dbb400640 1 --2- 172.21.15.161:0/3759161564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc40715a0 0x7f5dc415f830 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40cedc0 tx=0x7f5db401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd9ba2f6b3983df server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:43.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.535+0000 7f5dc3400640 1 -- 172.21.15.161:0/3759161564 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4004230 con 0x7f5dc40715a0 2024-05-22T21:19:43.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.535+0000 7f5dc3400640 1 -- 172.21.15.161:0/3759161564 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db40a7050 con 0x7f5dc40715a0 2024-05-22T21:19:43.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.535+0000 7f5dc3400640 1 -- 172.21.15.161:0/3759161564 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4003620 con 0x7f5dc40715a0 2024-05-22T21:19:43.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.535+0000 7f5dcaa00640 1 -- 172.21.15.161:0/3759161564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc40715a0 msgr2=0x7f5dc415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.535+0000 7f5dcaa00640 1 --2- 172.21.15.161:0/3759161564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc40715a0 0x7f5dc415f830 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40cedc0 tx=0x7f5db401c450 comp rx=0 tx=0).stop 2024-05-22T21:19:43.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.535+0000 7f5dcaa00640 1 -- 172.21.15.161:0/3759161564 shutdown_connections 2024-05-22T21:19:43.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.535+0000 7f5dcaa00640 1 --2- 172.21.15.161:0/3759161564 >> v2:172.21.15.179:3300/0 conn(0x7f5dc415ec90 0x7f5dc415f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.535+0000 7f5dcaa00640 1 --2- 172.21.15.161:0/3759161564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc40715a0 0x7f5dc415f830 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.535+0000 7f5dcaa00640 1 -- 172.21.15.161:0/3759161564 >> 172.21.15.161:0/3759161564 conn(0x7f5dc40b0bc0 msgr2=0x7f5dc40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:43.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.535+0000 7f5dcaa00640 1 -- 172.21.15.161:0/3759161564 shutdown_connections 2024-05-22T21:19:43.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.535+0000 7f5dcaa00640 1 -- 172.21.15.161:0/3759161564 wait complete. 2024-05-22T21:19:43.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dcaa00640 1 Processor -- start 2024-05-22T21:19:43.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dcaa00640 1 -- start start 2024-05-22T21:19:43.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dcaa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5dc4143ec0 0x7f5dc415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dcaa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc415ec90 0x7f5dc4138ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dcaa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dc40b39b0 0x7f5dc413a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dcaa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc4139020 con 0x7f5dc415ec90 2024-05-22T21:19:43.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dcaa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc406bff0 con 0x7f5dc40b39b0 2024-05-22T21:19:43.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dcaa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40b2a70 con 0x7f5dc4143ec0 2024-05-22T21:19:43.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dbb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc415ec90 0x7f5dc4138ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dbb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc415ec90 0x7f5dc4138ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45124/0 (socket says 172.21.15.161:45124) 2024-05-22T21:19:43.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dbb400640 1 -- 172.21.15.161:0/3660519980 learned_addr learned my addr 172.21.15.161:0/3660519980 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:43.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dc8a00640 1 --2- 172.21.15.161:0/3660519980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dc40b39b0 0x7f5dc413a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dc3e00640 1 --2- 172.21.15.161:0/3660519980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5dc4143ec0 0x7f5dc415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dbb400640 1 -- 172.21.15.161:0/3660519980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5dc4143ec0 msgr2=0x7f5dc415a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dbb400640 1 --2- 172.21.15.161:0/3660519980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5dc4143ec0 0x7f5dc415a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dbb400640 1 -- 172.21.15.161:0/3660519980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dc40b39b0 msgr2=0x7f5dc413a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dbb400640 1 --2- 172.21.15.161:0/3660519980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dc40b39b0 0x7f5dc413a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dbb400640 1 -- 172.21.15.161:0/3660519980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db4009420 con 0x7f5dc415ec90 2024-05-22T21:19:43.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dc8a00640 1 --2- 172.21.15.161:0/3660519980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dc40b39b0 0x7f5dc413a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:43.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dc3e00640 1 --2- 172.21.15.161:0/3660519980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5dc4143ec0 0x7f5dc415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:43.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dbb400640 1 --2- 172.21.15.161:0/3660519980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc415ec90 0x7f5dc4138ae0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5db401a080 tx=0x7f5db40191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:43.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dc0c00640 1 -- 172.21.15.161:0/3660519980 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4003b70 con 0x7f5dc415ec90 2024-05-22T21:19:43.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dc0c00640 1 -- 172.21.15.161:0/3660519980 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db4003cd0 con 0x7f5dc415ec90 2024-05-22T21:19:43.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dcaa00640 1 -- 172.21.15.161:0/3660519980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc40b2ce0 con 0x7f5dc415ec90 2024-05-22T21:19:43.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dc0c00640 1 -- 172.21.15.161:0/3660519980 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db40ab680 con 0x7f5dc415ec90 2024-05-22T21:19:43.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.539+0000 7f5dcaa00640 1 -- 172.21.15.161:0/3660519980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc415a2c0 con 0x7f5dc415ec90 2024-05-22T21:19:43.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.543+0000 7f5dcaa00640 1 -- 172.21.15.161:0/3660519980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc4067220 con 0x7f5dc415ec90 2024-05-22T21:19:43.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.543+0000 7f5dc0c00640 1 -- 172.21.15.161:0/3660519980 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5db40b2040 con 0x7f5dc415ec90 2024-05-22T21:19:43.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.543+0000 7f5dc0c00640 1 --2- 172.21.15.161:0/3660519980 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5da00666c0 0x7f5da0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.543+0000 7f5dc0c00640 1 -- 172.21.15.161:0/3660519980 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5db411dcd0 con 0x7f5dc415ec90 2024-05-22T21:19:43.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.543+0000 7f5dc3e00640 1 --2- 172.21.15.161:0/3660519980 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5da00666c0 0x7f5da0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.543+0000 7f5dc3e00640 1 --2- 172.21.15.161:0/3660519980 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5da00666c0 0x7f5da0068b80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0011f0 tx=0x7f5dac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:43.549 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.547+0000 7f5dc0c00640 1 -- 172.21.15.161:0/3660519980 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5db40eb2f0 con 0x7f5dc415ec90 2024-05-22T21:19:43.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.699+0000 7f5dcaa00640 1 -- 172.21.15.161:0/3660519980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5dc406c130 con 0x7f5dc415ec90 2024-05-22T21:19:43.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.699+0000 7f5dc0c00640 1 -- 172.21.15.161:0/3660519980 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v136) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5db40b0090 con 0x7f5dc415ec90 2024-05-22T21:19:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:43.703 INFO:teuthology.orchestra.run.smithi161.stdout:[ 2024-05-22T21:19:43.703 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: "poolnum": 1, 2024-05-22T21:19:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: "poolname": ".mgr" 2024-05-22T21:19:43.704 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:43.704 INFO:teuthology.orchestra.run.smithi161.stdout:] 2024-05-22T21:19:43.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.703+0000 7f5dcaa00640 1 -- 172.21.15.161:0/3660519980 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5da00666c0 msgr2=0x7f5da0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.703+0000 7f5dcaa00640 1 --2- 172.21.15.161:0/3660519980 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5da00666c0 0x7f5da0068b80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0011f0 tx=0x7f5dac009040 comp rx=0 tx=0).stop 2024-05-22T21:19:43.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.703+0000 7f5dcaa00640 1 -- 172.21.15.161:0/3660519980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc415ec90 msgr2=0x7f5dc4138ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.703+0000 7f5dcaa00640 1 --2- 172.21.15.161:0/3660519980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc415ec90 0x7f5dc4138ae0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5db401a080 tx=0x7f5db40191f0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.703+0000 7f5dcaa00640 1 -- 172.21.15.161:0/3660519980 shutdown_connections 2024-05-22T21:19:43.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.703+0000 7f5dcaa00640 1 --2- 172.21.15.161:0/3660519980 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5da00666c0 0x7f5da0068b80 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.703+0000 7f5dcaa00640 1 --2- 172.21.15.161:0/3660519980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dc40b39b0 0x7f5dc413a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.703+0000 7f5dcaa00640 1 --2- 172.21.15.161:0/3660519980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dc415ec90 0x7f5dc4138ae0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.703+0000 7f5dcaa00640 1 --2- 172.21.15.161:0/3660519980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5dc4143ec0 0x7f5dc415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.703+0000 7f5dcaa00640 1 -- 172.21.15.161:0/3660519980 >> 172.21.15.161:0/3660519980 conn(0x7f5dc40b0bc0 msgr2=0x7f5dc40af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:43.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.703+0000 7f5dcaa00640 1 -- 172.21.15.161:0/3660519980 shutdown_connections 2024-05-22T21:19:43.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.703+0000 7f5dcaa00640 1 -- 172.21.15.161:0/3660519980 wait complete. 2024-05-22T21:19:43.724 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-22T21:19:43.724 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-22T21:19:43.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.863+0000 7f1e3e400640 1 Processor -- start 2024-05-22T21:19:43.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.863+0000 7f1e3e400640 1 -- start start 2024-05-22T21:19:43.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.863+0000 7f1e3e400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1e3813d560 0x7f1e380cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.863+0000 7f1e3e400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e380ce830 con 0x7f1e3813cfe0 2024-05-22T21:19:43.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.863+0000 7f1e3e400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e380ceb10 con 0x7f1e3815e1d0 2024-05-22T21:19:43.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.863+0000 7f1e3e400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e38140560 con 0x7f1e3813d560 2024-05-22T21:19:43.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.863+0000 7f1e37400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1e3813cfe0 0x7f1e3815dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43344/0 (socket says 172.21.15.161:43344) 2024-05-22T21:19:43.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.863+0000 7f1e37400640 1 -- 172.21.15.161:0/3579323164 learned_addr learned my addr 172.21.15.161:0/3579323164 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:43.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.863+0000 7f1e36a00640 1 --2- 172.21.15.161:0/3579323164 >> v2:172.21.15.179:3300/0 conn(0x7f1e3813d560 0x7f1e380cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.863+0000 7f1e36000640 1 -- 172.21.15.161:0/3579323164 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1624642170 0 0) 0x7f1e380ce830 con 0x7f1e3813cfe0 2024-05-22T21:19:43.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.863+0000 7f1e36000640 1 -- 172.21.15.161:0/3579323164 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e1c003530 con 0x7f1e3813cfe0 2024-05-22T21:19:43.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.863+0000 7f1e36a00640 1 -- 172.21.15.161:0/3579323164 >> v1:172.21.15.179:6789/0 conn(0x7f1e3815e1d0 legacy=0x7f1e380ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.863+0000 7f1e36000640 1 -- 172.21.15.161:0/3579323164 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 951120234 0 0) 0x7f1e1c003530 con 0x7f1e3813cfe0 2024-05-22T21:19:43.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.863+0000 7f1e36a00640 1 -- 172.21.15.161:0/3579323164 >> v1:172.21.15.161:6789/0 conn(0x7f1e3813cfe0 legacy=0x7f1e3815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.863+0000 7f1e36a00640 1 -- 172.21.15.161:0/3579323164 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3806fd80 con 0x7f1e3813d560 2024-05-22T21:19:43.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.867+0000 7f1e36a00640 1 --2- 172.21.15.161:0/3579323164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e3813d560 0x7f1e380cdb00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1e240097c0 tx=0x7f1e2401c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177a66dfec06b67c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:43.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.867+0000 7f1e36000640 1 -- 172.21.15.161:0/3579323164 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28001680 con 0x7f1e3813d560 2024-05-22T21:19:43.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.867+0000 7f1e36000640 1 -- 172.21.15.161:0/3579323164 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e240a7050 con 0x7f1e3813d560 2024-05-22T21:19:43.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.867+0000 7f1e36000640 1 -- 172.21.15.161:0/3579323164 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e240ab620 con 0x7f1e3813d560 2024-05-22T21:19:43.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.867+0000 7f1e3e400640 1 -- 172.21.15.161:0/3579323164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e3813d560 msgr2=0x7f1e380cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.867+0000 7f1e3e400640 1 --2- 172.21.15.161:0/3579323164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e3813d560 0x7f1e380cdb00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1e240097c0 tx=0x7f1e2401c300 comp rx=0 tx=0).stop 2024-05-22T21:19:43.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.867+0000 7f1e3e400640 1 -- 172.21.15.161:0/3579323164 shutdown_connections 2024-05-22T21:19:43.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.867+0000 7f1e3e400640 1 --2- 172.21.15.161:0/3579323164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e3813d560 0x7f1e380cdb00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.867+0000 7f1e3e400640 1 -- 172.21.15.161:0/3579323164 >> 172.21.15.161:0/3579323164 conn(0x7f1e3806a0a0 msgr2=0x7f1e3806a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:43.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.867+0000 7f1e3e400640 1 -- 172.21.15.161:0/3579323164 shutdown_connections 2024-05-22T21:19:43.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.867+0000 7f1e3e400640 1 -- 172.21.15.161:0/3579323164 wait complete. 2024-05-22T21:19:43.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e3e400640 1 Processor -- start 2024-05-22T21:19:43.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e3e400640 1 -- start start 2024-05-22T21:19:43.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e3e400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e3813cfe0 0x7f1e38136a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e3e400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e3813d560 0x7f1e3813c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e3e400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e3815e1d0 0x7f1e38139000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e3e400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3813b440 con 0x7f1e3813cfe0 2024-05-22T21:19:43.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e3e400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e38139540 con 0x7f1e3815e1d0 2024-05-22T21:19:43.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e3e400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3813a830 con 0x7f1e3813d560 2024-05-22T21:19:43.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e37400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e3813cfe0 0x7f1e38136a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e37400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e3813cfe0 0x7f1e38136a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45132/0 (socket says 172.21.15.161:45132) 2024-05-22T21:19:43.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e37400640 1 -- 172.21.15.161:0/1629071990 learned_addr learned my addr 172.21.15.161:0/1629071990 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:43.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e36a00640 1 --2- 172.21.15.161:0/1629071990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e3813d560 0x7f1e3813c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e37e00640 1 --2- 172.21.15.161:0/1629071990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e3815e1d0 0x7f1e38139000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e37400640 1 -- 172.21.15.161:0/1629071990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e3813d560 msgr2=0x7f1e3813c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e37400640 1 --2- 172.21.15.161:0/1629071990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e3813d560 0x7f1e3813c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e37400640 1 -- 172.21.15.161:0/1629071990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e3815e1d0 msgr2=0x7f1e38139000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:43.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e37400640 1 --2- 172.21.15.161:0/1629071990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e3815e1d0 0x7f1e38139000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:43.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e37400640 1 -- 172.21.15.161:0/1629071990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e24009420 con 0x7f1e3813cfe0 2024-05-22T21:19:43.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e37e00640 1 --2- 172.21.15.161:0/1629071990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e3815e1d0 0x7f1e38139000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:43.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e37400640 1 --2- 172.21.15.161:0/1629071990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e3813cfe0 0x7f1e38136a20 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800c780 tx=0x7f1e2800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:43.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e2f400640 1 -- 172.21.15.161:0/1629071990 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28005ae0 con 0x7f1e3813cfe0 2024-05-22T21:19:43.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e2f400640 1 -- 172.21.15.161:0/1629071990 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e28005c40 con 0x7f1e3813cfe0 2024-05-22T21:19:43.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e3e400640 1 -- 172.21.15.161:0/1629071990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3813ab10 con 0x7f1e3813cfe0 2024-05-22T21:19:43.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e2f400640 1 -- 172.21.15.161:0/1629071990 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28007070 con 0x7f1e3813cfe0 2024-05-22T21:19:43.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e3e400640 1 -- 172.21.15.161:0/1629071990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3813b0d0 con 0x7f1e3813cfe0 2024-05-22T21:19:43.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.871+0000 7f1e3e400640 1 -- 172.21.15.161:0/1629071990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e38067220 con 0x7f1e3813cfe0 2024-05-22T21:19:43.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.875+0000 7f1e2f400640 1 -- 172.21.15.161:0/1629071990 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1e280071f0 con 0x7f1e3813cfe0 2024-05-22T21:19:43.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.875+0000 7f1e2f400640 1 --2- 172.21.15.161:0/1629071990 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e18066690 0x7f1e18068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:43.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.875+0000 7f1e2f400640 1 -- 172.21.15.161:0/1629071990 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1e28083bc0 con 0x7f1e3813cfe0 2024-05-22T21:19:43.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.875+0000 7f1e36a00640 1 --2- 172.21.15.161:0/1629071990 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e18066690 0x7f1e18068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:43.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.875+0000 7f1e36a00640 1 --2- 172.21.15.161:0/1629071990 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e18066690 0x7f1e18068b50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1e240ad000 tx=0x7f1e24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:43.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:43.879+0000 7f1e2f400640 1 -- 172.21.15.161:0/1629071990 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1e280511f0 con 0x7f1e3813cfe0 2024-05-22T21:19:44.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:44.031+0000 7f1e3e400640 1 -- 172.21.15.161:0/1629071990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1e38150e90 con 0x7f1e3813cfe0 2024-05-22T21:19:44.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:44.995+0000 7f1e2f400640 1 -- 172.21.15.161:0/1629071990 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v137) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1e28054810 con 0x7f1e3813cfe0 2024-05-22T21:19:44.999 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' created 2024-05-22T21:19:45.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:44.999+0000 7f1e3e400640 1 -- 172.21.15.161:0/1629071990 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e18066690 msgr2=0x7f1e18068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:45.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:44.999+0000 7f1e3e400640 1 --2- 172.21.15.161:0/1629071990 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e18066690 0x7f1e18068b50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1e240ad000 tx=0x7f1e24008040 comp rx=0 tx=0).stop 2024-05-22T21:19:45.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:44.999+0000 7f1e3e400640 1 -- 172.21.15.161:0/1629071990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e3813cfe0 msgr2=0x7f1e38136a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:45.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:44.999+0000 7f1e3e400640 1 --2- 172.21.15.161:0/1629071990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e3813cfe0 0x7f1e38136a20 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800c780 tx=0x7f1e2800cc40 comp rx=0 tx=0).stop 2024-05-22T21:19:45.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:44.999+0000 7f1e3e400640 1 -- 172.21.15.161:0/1629071990 shutdown_connections 2024-05-22T21:19:45.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:44.999+0000 7f1e3e400640 1 --2- 172.21.15.161:0/1629071990 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e18066690 0x7f1e18068b50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:45.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:44.999+0000 7f1e3e400640 1 --2- 172.21.15.161:0/1629071990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e3815e1d0 0x7f1e38139000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:45.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:44.999+0000 7f1e3e400640 1 --2- 172.21.15.161:0/1629071990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e3813d560 0x7f1e3813c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:45.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:44.999+0000 7f1e3e400640 1 --2- 172.21.15.161:0/1629071990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e3813cfe0 0x7f1e38136a20 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:45.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:44.999+0000 7f1e3e400640 1 -- 172.21.15.161:0/1629071990 >> 172.21.15.161:0/1629071990 conn(0x7f1e3806a0a0 msgr2=0x7f1e380b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:45.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:44.999+0000 7f1e3e400640 1 -- 172.21.15.161:0/1629071990 shutdown_connections 2024-05-22T21:19:45.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:44.999+0000 7f1e3e400640 1 -- 172.21.15.161:0/1629071990 wait complete. 2024-05-22T21:19:45.025 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-22T21:19:45.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.167+0000 7fe473600640 1 Processor -- start 2024-05-22T21:19:45.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.167+0000 7fe473600640 1 -- start start 2024-05-22T21:19:45.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe473600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe46c13cfc0 0x7fe46c1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:45.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe473600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46c13d3a0 con 0x7fe46c13fe50 2024-05-22T21:19:45.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe473600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46c0cd920 con 0x7fe46c15af60 2024-05-22T21:19:45.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe473600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c0cdc00 con 0x7fe46c13cfc0 2024-05-22T21:19:45.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe471600640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe46c13fe50 0x7fe46c06fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43358/0 (socket says 172.21.15.161:43358) 2024-05-22T21:19:45.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe471600640 1 -- 172.21.15.161:0/3492261999 learned_addr learned my addr 172.21.15.161:0/3492261999 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:45.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe470c00640 1 --2- 172.21.15.161:0/3492261999 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe46c13cfc0 0x7fe46c1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:45.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe46be00640 1 -- 172.21.15.161:0/3492261999 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377695764 0 0) 0x7fe46c13d3a0 con 0x7fe46c13fe50 2024-05-22T21:19:45.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe46be00640 1 -- 172.21.15.161:0/3492261999 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe450003530 con 0x7fe46c13fe50 2024-05-22T21:19:45.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe46be00640 1 -- 172.21.15.161:0/3492261999 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3543334219 0 0) 0x7fe450003530 con 0x7fe46c13fe50 2024-05-22T21:19:45.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe46be00640 1 -- 172.21.15.161:0/3492261999 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe46c13cfc0 msgr2=0x7fe46c1402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:45.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe46be00640 1 --2- 172.21.15.161:0/3492261999 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe46c13cfc0 0x7fe46c1402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:45.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe46be00640 1 -- 172.21.15.161:0/3492261999 >> v1:172.21.15.179:6789/0 conn(0x7fe46c15af60 legacy=0x7fe46c13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:45.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe46be00640 1 -- 172.21.15.161:0/3492261999 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c0cdee0 con 0x7fe46c13fe50 2024-05-22T21:19:45.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe46be00640 1 -- 172.21.15.161:0/3492261999 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fe464001a00 con 0x7fe46c13fe50 2024-05-22T21:19:45.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe46be00640 1 -- 172.21.15.161:0/3492261999 >> v1:172.21.15.161:6789/0 conn(0x7fe46c13fe50 legacy=0x7fe46c06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:45.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe46be00640 1 --2- 172.21.15.161:0/3492261999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4500039b0 0x7fe450003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:45.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe46be00640 1 --2- 172.21.15.161:0/3492261999 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe46c13cfc0 0x7fe4500047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:45.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe46be00640 1 --2- 172.21.15.161:0/3492261999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe450004d10 0x7fe450005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:45.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe46be00640 1 -- 172.21.15.161:0/3492261999 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c0cdc00 con 0x7fe450004d10 2024-05-22T21:19:45.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe46be00640 1 -- 172.21.15.161:0/3492261999 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45000bac0 con 0x7fe4500039b0 2024-05-22T21:19:45.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe46be00640 1 -- 172.21.15.161:0/3492261999 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45000de90 con 0x7fe46c13cfc0 2024-05-22T21:19:45.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe471600640 1 --2- 172.21.15.161:0/3492261999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe450004d10 0x7fe450005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:45.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe470c00640 1 --2- 172.21.15.161:0/3492261999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4500039b0 0x7fe450003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:45.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe463e00640 1 --2- 172.21.15.161:0/3492261999 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe46c13cfc0 0x7fe4500047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:45.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe471600640 1 -- 172.21.15.161:0/3492261999 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe46c13cfc0 msgr2=0x7fe4500047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:45.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe471600640 1 --2- 172.21.15.161:0/3492261999 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe46c13cfc0 0x7fe4500047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:45.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe471600640 1 -- 172.21.15.161:0/3492261999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4500039b0 msgr2=0x7fe450003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:45.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe471600640 1 --2- 172.21.15.161:0/3492261999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe4500039b0 0x7fe450003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:45.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.171+0000 7fe471600640 1 -- 172.21.15.161:0/3492261999 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe450003530 con 0x7fe450004d10 2024-05-22T21:19:45.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.175+0000 7fe471600640 1 --2- 172.21.15.161:0/3492261999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe450004d10 0x7fe450005190 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe46400a800 tx=0x7fe46401fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0d2cb5a3b6126 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:45.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.175+0000 7fe46be00640 1 -- 172.21.15.161:0/3492261999 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4640aa6b0 con 0x7fe450004d10 2024-05-22T21:19:45.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.175+0000 7fe46be00640 1 -- 172.21.15.161:0/3492261999 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4640aa810 con 0x7fe450004d10 2024-05-22T21:19:45.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.175+0000 7fe46be00640 1 -- 172.21.15.161:0/3492261999 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4640aac00 con 0x7fe450004d10 2024-05-22T21:19:45.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.175+0000 7fe473600640 1 -- 172.21.15.161:0/3492261999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe450004d10 msgr2=0x7fe450005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:45.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.175+0000 7fe473600640 1 --2- 172.21.15.161:0/3492261999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe450004d10 0x7fe450005190 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe46400a800 tx=0x7fe46401fa80 comp rx=0 tx=0).stop 2024-05-22T21:19:45.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.175+0000 7fe471600640 1 -- 172.21.15.161:0/3492261999 reap_dead start 2024-05-22T21:19:45.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.175+0000 7fe473600640 1 -- 172.21.15.161:0/3492261999 shutdown_connections 2024-05-22T21:19:45.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.175+0000 7fe473600640 1 -- 172.21.15.161:0/3492261999 >> 172.21.15.161:0/3492261999 conn(0x7fe46c06a2d0 msgr2=0x7fe46c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:45.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.175+0000 7fe473600640 1 -- 172.21.15.161:0/3492261999 shutdown_connections 2024-05-22T21:19:45.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.175+0000 7fe473600640 1 -- 172.21.15.161:0/3492261999 wait complete. 2024-05-22T21:19:45.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.175+0000 7fe473600640 1 Processor -- start 2024-05-22T21:19:45.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.175+0000 7fe473600640 1 -- start start 2024-05-22T21:19:45.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe473600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe450004d10 0x7fe46c1389a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:45.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe473600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe46c139cd0 0x7fe46c13a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:45.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe473600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe46c0b1f80 0x7fe46c0b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:45.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe473600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c13a670 con 0x7fe46c0b1f80 2024-05-22T21:19:45.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe473600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c138ee0 con 0x7fe46c139cd0 2024-05-22T21:19:45.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe473600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c0ae200 con 0x7fe450004d10 2024-05-22T21:19:45.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe471600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe46c0b1f80 0x7fe46c0b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:45.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe471600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe46c0b1f80 0x7fe46c0b2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45140/0 (socket says 172.21.15.161:45140) 2024-05-22T21:19:45.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe471600640 1 -- 172.21.15.161:0/3505138419 learned_addr learned my addr 172.21.15.161:0/3505138419 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:45.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe470c00640 1 --2- 172.21.15.161:0/3505138419 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe450004d10 0x7fe46c1389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:45.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe463e00640 1 --2- 172.21.15.161:0/3505138419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe46c139cd0 0x7fe46c13a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:45.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe471600640 1 -- 172.21.15.161:0/3505138419 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe450004d10 msgr2=0x7fe46c1389a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:45.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe471600640 1 --2- 172.21.15.161:0/3505138419 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe450004d10 0x7fe46c1389a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:45.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe471600640 1 -- 172.21.15.161:0/3505138419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe46c139cd0 msgr2=0x7fe46c13a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:45.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe471600640 1 --2- 172.21.15.161:0/3505138419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe46c139cd0 0x7fe46c13a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:45.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe471600640 1 -- 172.21.15.161:0/3505138419 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c0ad680 con 0x7fe46c0b1f80 2024-05-22T21:19:45.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe471600640 1 --2- 172.21.15.161:0/3505138419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe46c0b1f80 0x7fe46c0b2430 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe464020320 tx=0x7fe450005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:45.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe469600640 1 -- 172.21.15.161:0/3505138419 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4640aa6b0 con 0x7fe46c0b1f80 2024-05-22T21:19:45.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe469600640 1 -- 172.21.15.161:0/3505138419 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe464001250 con 0x7fe46c0b1f80 2024-05-22T21:19:45.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe469600640 1 -- 172.21.15.161:0/3505138419 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464004db0 con 0x7fe46c0b1f80 2024-05-22T21:19:45.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe473600640 1 -- 172.21.15.161:0/3505138419 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46c0ad960 con 0x7fe46c0b1f80 2024-05-22T21:19:45.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe473600640 1 -- 172.21.15.161:0/3505138419 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46c0adf20 con 0x7fe46c0b1f80 2024-05-22T21:19:45.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.179+0000 7fe473600640 1 -- 172.21.15.161:0/3505138419 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46c067220 con 0x7fe46c0b1f80 2024-05-22T21:19:45.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.183+0000 7fe469600640 1 -- 172.21.15.161:0/3505138419 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe4640017c0 con 0x7fe46c0b1f80 2024-05-22T21:19:45.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.183+0000 7fe469600640 1 --2- 172.21.15.161:0/3505138419 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe448066690 0x7fe448068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:45.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.183+0000 7fe469600640 1 -- 172.21.15.161:0/3505138419 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fe4640b9070 con 0x7fe46c0b1f80 2024-05-22T21:19:45.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.183+0000 7fe470c00640 1 --2- 172.21.15.161:0/3505138419 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe448066690 0x7fe448068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:45.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.183+0000 7fe470c00640 1 --2- 172.21.15.161:0/3505138419 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe448066690 0x7fe448068b50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009be0 tx=0x7fe45c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:45.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.187+0000 7fe469600640 1 -- 172.21.15.161:0/3505138419 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe4640eedd0 con 0x7fe46c0b1f80 2024-05-22T21:19:45.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:45.339+0000 7fe473600640 1 -- 172.21.15.161:0/3505138419 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe46c0b1ce0 con 0x7fe46c0b1f80 2024-05-22T21:19:46.005 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.003+0000 7fe469600640 1 -- 172.21.15.161:0/3505138419 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v138) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe4640f23f0 con 0x7fe46c0b1f80 2024-05-22T21:19:46.005 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' created 2024-05-22T21:19:46.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.007+0000 7fe473600640 1 -- 172.21.15.161:0/3505138419 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe448066690 msgr2=0x7fe448068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.007+0000 7fe473600640 1 --2- 172.21.15.161:0/3505138419 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe448066690 0x7fe448068b50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009be0 tx=0x7fe45c009190 comp rx=0 tx=0).stop 2024-05-22T21:19:46.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.007+0000 7fe473600640 1 -- 172.21.15.161:0/3505138419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe46c0b1f80 msgr2=0x7fe46c0b2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.007+0000 7fe473600640 1 --2- 172.21.15.161:0/3505138419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe46c0b1f80 0x7fe46c0b2430 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe464020320 tx=0x7fe450005800 comp rx=0 tx=0).stop 2024-05-22T21:19:46.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.011+0000 7fe473600640 1 -- 172.21.15.161:0/3505138419 shutdown_connections 2024-05-22T21:19:46.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.011+0000 7fe473600640 1 --2- 172.21.15.161:0/3505138419 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe448066690 0x7fe448068b50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.011+0000 7fe473600640 1 --2- 172.21.15.161:0/3505138419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe46c0b1f80 0x7fe46c0b2430 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.011+0000 7fe473600640 1 --2- 172.21.15.161:0/3505138419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe46c139cd0 0x7fe46c13a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.011+0000 7fe473600640 1 --2- 172.21.15.161:0/3505138419 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe450004d10 0x7fe46c1389a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.011+0000 7fe473600640 1 -- 172.21.15.161:0/3505138419 >> 172.21.15.161:0/3505138419 conn(0x7fe46c06a2d0 msgr2=0x7fe46c06cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:46.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.011+0000 7fe473600640 1 -- 172.21.15.161:0/3505138419 shutdown_connections 2024-05-22T21:19:46.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.011+0000 7fe473600640 1 -- 172.21.15.161:0/3505138419 wait complete. 2024-05-22T21:19:46.039 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-22T21:19:46.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.175+0000 7fe397e00640 1 Processor -- start 2024-05-22T21:19:46.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.175+0000 7fe397e00640 1 -- start start 2024-05-22T21:19:46.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.175+0000 7fe397e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe39806fa80 0x7fe3980cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.175+0000 7fe397e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe398070b70 con 0x7fe3981436d0 2024-05-22T21:19:46.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.175+0000 7fe397e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39806ff60 con 0x7fe3980b33d0 2024-05-22T21:19:46.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.175+0000 7fe397e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3980cc720 con 0x7fe39806fa80 2024-05-22T21:19:46.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.175+0000 7fe396a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe3981436d0 0x7fe3980b2cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48852/0 (socket says 172.21.15.161:48852) 2024-05-22T21:19:46.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.175+0000 7fe396a00640 1 -- 172.21.15.161:0/2292937084 learned_addr learned my addr 172.21.15.161:0/2292937084 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:46.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.175+0000 7fe397400640 1 --2- 172.21.15.161:0/2292937084 >> v2:172.21.15.161:3300/0 conn(0x7fe39806fa80 0x7fe3980cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.175+0000 7fe395600640 1 -- 172.21.15.161:0/2292937084 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295151568 0 0) 0x7fe398070b70 con 0x7fe3981436d0 2024-05-22T21:19:46.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.175+0000 7fe397400640 1 -- 172.21.15.161:0/2292937084 >> v1:172.21.15.179:6789/0 conn(0x7fe3980b33d0 legacy=0x7fe398071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.175+0000 7fe397400640 1 -- 172.21.15.161:0/2292937084 >> v1:172.21.15.161:6789/0 conn(0x7fe3981436d0 legacy=0x7fe3980b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.175+0000 7fe397400640 1 -- 172.21.15.161:0/2292937084 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe398070240 con 0x7fe39806fa80 2024-05-22T21:19:46.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.179+0000 7fe397400640 1 --2- 172.21.15.161:0/2292937084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe39806fa80 0x7fe3980cc1e0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fe390009d90 tx=0x7fe39001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33e367125da54b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:46.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.179+0000 7fe395600640 1 -- 172.21.15.161:0/2292937084 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3900a5970 con 0x7fe39806fa80 2024-05-22T21:19:46.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.179+0000 7fe395600640 1 -- 172.21.15.161:0/2292937084 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3900a5ad0 con 0x7fe39806fa80 2024-05-22T21:19:46.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.179+0000 7fe395600640 1 -- 172.21.15.161:0/2292937084 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3900a5e60 con 0x7fe39806fa80 2024-05-22T21:19:46.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.179+0000 7fe397e00640 1 -- 172.21.15.161:0/2292937084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe39806fa80 msgr2=0x7fe3980cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.179+0000 7fe397e00640 1 --2- 172.21.15.161:0/2292937084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe39806fa80 0x7fe3980cc1e0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fe390009d90 tx=0x7fe39001c8b0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.179+0000 7fe397e00640 1 -- 172.21.15.161:0/2292937084 shutdown_connections 2024-05-22T21:19:46.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.179+0000 7fe397e00640 1 --2- 172.21.15.161:0/2292937084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe39806fa80 0x7fe3980cc1e0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.179+0000 7fe397e00640 1 -- 172.21.15.161:0/2292937084 >> 172.21.15.161:0/2292937084 conn(0x7fe3980cec00 msgr2=0x7fe3980cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:46.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.179+0000 7fe397e00640 1 -- 172.21.15.161:0/2292937084 shutdown_connections 2024-05-22T21:19:46.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.179+0000 7fe397e00640 1 -- 172.21.15.161:0/2292937084 wait complete. 2024-05-22T21:19:46.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe397e00640 1 Processor -- start 2024-05-22T21:19:46.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe397e00640 1 -- start start 2024-05-22T21:19:46.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe397e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe39806fa80 0x7fe398140800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe397e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3980b33d0 0x7fe3980afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe397e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3981436d0 0x7fe398137560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe397e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3980b0410 con 0x7fe39806fa80 2024-05-22T21:19:46.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe397e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39813fd20 con 0x7fe3980b33d0 2024-05-22T21:19:46.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe397e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe398140000 con 0x7fe3981436d0 2024-05-22T21:19:46.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe396a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe39806fa80 0x7fe398140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe396a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe39806fa80 0x7fe398140800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50622/0 (socket says 172.21.15.161:50622) 2024-05-22T21:19:46.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe396a00640 1 -- 172.21.15.161:0/1229878384 learned_addr learned my addr 172.21.15.161:0/1229878384 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:46.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe396000640 1 --2- 172.21.15.161:0/1229878384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3980b33d0 0x7fe3980afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe397400640 1 --2- 172.21.15.161:0/1229878384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3981436d0 0x7fe398137560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe396a00640 1 -- 172.21.15.161:0/1229878384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3981436d0 msgr2=0x7fe398137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe396a00640 1 --2- 172.21.15.161:0/1229878384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3981436d0 0x7fe398137560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe396a00640 1 -- 172.21.15.161:0/1229878384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3980b33d0 msgr2=0x7fe3980afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe396a00640 1 --2- 172.21.15.161:0/1229878384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3980b33d0 0x7fe3980afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe396a00640 1 -- 172.21.15.161:0/1229878384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe390009a70 con 0x7fe39806fa80 2024-05-22T21:19:46.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe397400640 1 --2- 172.21.15.161:0/1229878384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3981436d0 0x7fe398137560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:46.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe396a00640 1 --2- 172.21.15.161:0/1229878384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe39806fa80 0x7fe398140800 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fe38800c410 tx=0x7fe38800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:46.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe38ea00640 1 -- 172.21.15.161:0/1229878384 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3880074e0 con 0x7fe39806fa80 2024-05-22T21:19:46.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe38ea00640 1 -- 172.21.15.161:0/1229878384 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe388007640 con 0x7fe39806fa80 2024-05-22T21:19:46.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe38ea00640 1 -- 172.21.15.161:0/1229878384 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe388004a30 con 0x7fe39806fa80 2024-05-22T21:19:46.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe397e00640 1 -- 172.21.15.161:0/1229878384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3981402e0 con 0x7fe39806fa80 2024-05-22T21:19:46.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.183+0000 7fe397e00640 1 -- 172.21.15.161:0/1229878384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe398137aa0 con 0x7fe39806fa80 2024-05-22T21:19:46.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.187+0000 7fe397e00640 1 -- 172.21.15.161:0/1229878384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe398067070 con 0x7fe39806fa80 2024-05-22T21:19:46.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.187+0000 7fe38ea00640 1 -- 172.21.15.161:0/1229878384 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe388004b90 con 0x7fe39806fa80 2024-05-22T21:19:46.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.187+0000 7fe38ea00640 1 --2- 172.21.15.161:0/1229878384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe3780666c0 0x7fe378068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.187+0000 7fe38ea00640 1 -- 172.21.15.161:0/1229878384 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe388084490 con 0x7fe39806fa80 2024-05-22T21:19:46.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.187+0000 7fe396000640 1 --2- 172.21.15.161:0/1229878384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe3780666c0 0x7fe378068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.187+0000 7fe396000640 1 --2- 172.21.15.161:0/1229878384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe3780666c0 0x7fe378068b80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe3980ccc40 tx=0x7fe384009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:46.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.191+0000 7fe38ea00640 1 -- 172.21.15.161:0/1229878384 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe388051940 con 0x7fe39806fa80 2024-05-22T21:19:46.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.379+0000 7fe397e00640 1 -- 172.21.15.161:0/1229878384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe39806d5c0 con 0x7fe39806fa80 2024-05-22T21:19:46.391 INFO:teuthology.orchestra.run.smithi161.stderr:new fs with metadata pool 16 and data pool 17 2024-05-22T21:19:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.387+0000 7fe38ea00640 1 -- 172.21.15.161:0/1229878384 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v58) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe388054f60 con 0x7fe39806fa80 2024-05-22T21:19:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.391+0000 7fe397e00640 1 -- 172.21.15.161:0/1229878384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe3780666c0 msgr2=0x7fe378068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.391+0000 7fe397e00640 1 --2- 172.21.15.161:0/1229878384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe3780666c0 0x7fe378068b80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe3980ccc40 tx=0x7fe384009040 comp rx=0 tx=0).stop 2024-05-22T21:19:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.391+0000 7fe397e00640 1 -- 172.21.15.161:0/1229878384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe39806fa80 msgr2=0x7fe398140800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.391+0000 7fe397e00640 1 --2- 172.21.15.161:0/1229878384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe39806fa80 0x7fe398140800 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fe38800c410 tx=0x7fe38800c8d0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.391+0000 7fe397e00640 1 -- 172.21.15.161:0/1229878384 shutdown_connections 2024-05-22T21:19:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.391+0000 7fe397e00640 1 --2- 172.21.15.161:0/1229878384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe3780666c0 0x7fe378068b80 secure :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe3980ccc40 tx=0x7fe384009040 comp rx=0 tx=0).stop 2024-05-22T21:19:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.391+0000 7fe397e00640 1 --2- 172.21.15.161:0/1229878384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3981436d0 0x7fe398137560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.391+0000 7fe397e00640 1 --2- 172.21.15.161:0/1229878384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3980b33d0 0x7fe3980afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.391+0000 7fe397e00640 1 --2- 172.21.15.161:0/1229878384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe39806fa80 0x7fe398140800 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.391+0000 7fe397e00640 1 -- 172.21.15.161:0/1229878384 >> 172.21.15.161:0/1229878384 conn(0x7fe3980cec00 msgr2=0x7fe3980b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.391+0000 7fe397e00640 1 -- 172.21.15.161:0/1229878384 shutdown_connections 2024-05-22T21:19:46.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.391+0000 7fe397e00640 1 -- 172.21.15.161:0/1229878384 wait complete. 2024-05-22T21:19:46.425 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:46.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.559+0000 7f14f8400640 1 Processor -- start 2024-05-22T21:19:46.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.559+0000 7f14f8400640 1 -- start start 2024-05-22T21:19:46.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.563+0000 7f14f8400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14f015e130 0x7f14f015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.563+0000 7f14f8400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f14f015d410 0x7f14f015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.563+0000 7f14f8400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f0071670 con 0x7f14f015dd50 2024-05-22T21:19:46.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.563+0000 7f14f8400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f015d870 con 0x7f14f015d410 2024-05-22T21:19:46.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.563+0000 7f14f8400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f0070a60 con 0x7f14f015e130 2024-05-22T21:19:46.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.563+0000 7f14f6400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f14f015d410 0x7f14f015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.563+0000 7f14f5a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f14f015dd50 0x7f14f006d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48854/0 (socket says 172.21.15.161:48854) 2024-05-22T21:19:46.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.563+0000 7f14f5a00640 1 -- 172.21.15.161:0/1318435432 learned_addr learned my addr 172.21.15.161:0/1318435432 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:46.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.563+0000 7f14f5000640 1 --2- 172.21.15.161:0/1318435432 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14f015e130 0x7f14f015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.563+0000 7f14f6400640 1 -- 172.21.15.161:0/1318435432 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14f015e130 msgr2=0x7f14f015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:46.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.563+0000 7f14f6400640 1 --2- 172.21.15.161:0/1318435432 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14f015e130 0x7f14f015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.563+0000 7f14f6400640 1 -- 172.21.15.161:0/1318435432 >> v1:172.21.15.161:6789/0 conn(0x7f14f015dd50 legacy=0x7f14f006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.563+0000 7f14f6400640 1 -- 172.21.15.161:0/1318435432 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f0070d40 con 0x7f14f015d410 2024-05-22T21:19:46.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.563+0000 7f14f5000640 1 --2- 172.21.15.161:0/1318435432 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14f015e130 0x7f14f015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:46.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f6400640 1 --2- 172.21.15.161:0/1318435432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14f015d410 0x7f14f015b6c0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f14e80097c0 tx=0x7f14e801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1404cb163d866f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:46.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14efe00640 1 -- 172.21.15.161:0/1318435432 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e80ac640 con 0x7f14f015d410 2024-05-22T21:19:46.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14efe00640 1 -- 172.21.15.161:0/1318435432 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e80ac7a0 con 0x7f14f015d410 2024-05-22T21:19:46.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14efe00640 1 -- 172.21.15.161:0/1318435432 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e80acb30 con 0x7f14f015d410 2024-05-22T21:19:46.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f8400640 1 -- 172.21.15.161:0/1318435432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14f015d410 msgr2=0x7f14f015b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f8400640 1 --2- 172.21.15.161:0/1318435432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14f015d410 0x7f14f015b6c0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f14e80097c0 tx=0x7f14e801c3e0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f8400640 1 -- 172.21.15.161:0/1318435432 shutdown_connections 2024-05-22T21:19:46.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f8400640 1 --2- 172.21.15.161:0/1318435432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14f015d410 0x7f14f015b6c0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f8400640 1 --2- 172.21.15.161:0/1318435432 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14f015e130 0x7f14f015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f8400640 1 -- 172.21.15.161:0/1318435432 >> 172.21.15.161:0/1318435432 conn(0x7f14f00b0bc0 msgr2=0x7f14f00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:46.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f8400640 1 -- 172.21.15.161:0/1318435432 shutdown_connections 2024-05-22T21:19:46.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f8400640 1 -- 172.21.15.161:0/1318435432 wait complete. 2024-05-22T21:19:46.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f8400640 1 Processor -- start 2024-05-22T21:19:46.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f8400640 1 -- start start 2024-05-22T21:19:46.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f8400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14f015d410 0x7f14f014cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f8400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14f015dd50 0x7f14f014a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f8400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14f015e130 0x7f14f01521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f8400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f014b240 con 0x7f14f015d410 2024-05-22T21:19:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f8400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f014a770 con 0x7f14f015dd50 2024-05-22T21:19:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f8400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f014d9c0 con 0x7f14f015e130 2024-05-22T21:19:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f5a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14f015d410 0x7f14f014cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f5a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14f015d410 0x7f14f014cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50634/0 (socket says 172.21.15.161:50634) 2024-05-22T21:19:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.567+0000 7f14f5a00640 1 -- 172.21.15.161:0/1265426047 learned_addr learned my addr 172.21.15.161:0/1265426047 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14f6400640 1 --2- 172.21.15.161:0/1265426047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14f015e130 0x7f14f01521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14f5000640 1 --2- 172.21.15.161:0/1265426047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14f015dd50 0x7f14f014a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14f5a00640 1 -- 172.21.15.161:0/1265426047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14f015e130 msgr2=0x7f14f01521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14f5a00640 1 --2- 172.21.15.161:0/1265426047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14f015e130 0x7f14f01521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14f5a00640 1 -- 172.21.15.161:0/1265426047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14f015dd50 msgr2=0x7f14f014a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14f5a00640 1 --2- 172.21.15.161:0/1265426047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14f015dd50 0x7f14f014a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14f5a00640 1 -- 172.21.15.161:0/1265426047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e8009420 con 0x7f14f015d410 2024-05-22T21:19:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14f5000640 1 --2- 172.21.15.161:0/1265426047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14f015dd50 0x7f14f014a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14f5a00640 1 --2- 172.21.15.161:0/1265426047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14f015d410 0x7f14f014cfe0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f14e400cba0 tx=0x7f14e400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14ed600640 1 -- 172.21.15.161:0/1265426047 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e400ad40 con 0x7f14f015d410 2024-05-22T21:19:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14ed600640 1 -- 172.21.15.161:0/1265426047 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e4012400 con 0x7f14f015d410 2024-05-22T21:19:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14ed600640 1 -- 172.21.15.161:0/1265426047 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e400aea0 con 0x7f14f015d410 2024-05-22T21:19:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14f8400640 1 -- 172.21.15.161:0/1265426047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f014d6e0 con 0x7f14f015d410 2024-05-22T21:19:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14f8400640 1 -- 172.21.15.161:0/1265426047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f01519d0 con 0x7f14f015d410 2024-05-22T21:19:46.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14f8400640 1 -- 172.21.15.161:0/1265426047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f0067220 con 0x7f14f015d410 2024-05-22T21:19:46.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14ed600640 1 -- 172.21.15.161:0/1265426047 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f14e4007070 con 0x7f14f015d410 2024-05-22T21:19:46.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14ed600640 1 --2- 172.21.15.161:0/1265426047 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f14d0066690 0x7f14d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14ed600640 1 -- 172.21.15.161:0/1265426047 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14e40840d0 con 0x7f14f015d410 2024-05-22T21:19:46.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.571+0000 7f14f5000640 1 --2- 172.21.15.161:0/1265426047 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f14d0066690 0x7f14d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.575+0000 7f14f5000640 1 --2- 172.21.15.161:0/1265426047 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f14d0066690 0x7f14d0068b50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f14f00cb3f0 tx=0x7f14dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:46.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.575+0000 7f14ed600640 1 -- 172.21.15.161:0/1265426047 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f14e4051450 con 0x7f14f015d410 2024-05-22T21:19:46.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.731+0000 7f14f8400640 1 -- 172.21.15.161:0/1265426047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14f0070730 con 0x7f14f015d410 2024-05-22T21:19:46.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.735+0000 7f14ed600640 1 -- 172.21.15.161:0/1265426047 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16121 (secure 0 0 0) 0x7f14e4054a70 con 0x7f14f015d410 2024-05-22T21:19:46.737 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:46.737 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":139,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:19:46.384582+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:44.080742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:45.388618+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:46.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.735+0000 7f14f8400640 1 -- 172.21.15.161:0/1265426047 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f14d0066690 msgr2=0x7f14d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.735+0000 7f14f8400640 1 --2- 172.21.15.161:0/1265426047 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f14d0066690 0x7f14d0068b50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f14f00cb3f0 tx=0x7f14dc008040 comp rx=0 tx=0).stop 2024-05-22T21:19:46.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.735+0000 7f14f8400640 1 -- 172.21.15.161:0/1265426047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14f015d410 msgr2=0x7f14f014cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.735+0000 7f14f8400640 1 --2- 172.21.15.161:0/1265426047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14f015d410 0x7f14f014cfe0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f14e400cba0 tx=0x7f14e400a430 comp rx=0 tx=0).stop 2024-05-22T21:19:46.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.735+0000 7f14f8400640 1 -- 172.21.15.161:0/1265426047 shutdown_connections 2024-05-22T21:19:46.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.735+0000 7f14f8400640 1 --2- 172.21.15.161:0/1265426047 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f14d0066690 0x7f14d0068b50 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.735+0000 7f14f8400640 1 --2- 172.21.15.161:0/1265426047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14f015e130 0x7f14f01521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.735+0000 7f14f8400640 1 --2- 172.21.15.161:0/1265426047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14f015dd50 0x7f14f014a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.735+0000 7f14f8400640 1 --2- 172.21.15.161:0/1265426047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14f015d410 0x7f14f014cfe0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.735+0000 7f14f8400640 1 -- 172.21.15.161:0/1265426047 >> 172.21.15.161:0/1265426047 conn(0x7f14f00b0bc0 msgr2=0x7f14f00aefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:46.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.735+0000 7f14f8400640 1 -- 172.21.15.161:0/1265426047 shutdown_connections 2024-05-22T21:19:46.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.735+0000 7f14f8400640 1 -- 172.21.15.161:0/1265426047 wait complete. 2024-05-22T21:19:46.758 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:46.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.891+0000 7f9bc8c00640 1 Processor -- start 2024-05-22T21:19:46.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.891+0000 7f9bc8c00640 1 -- start start 2024-05-22T21:19:46.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.895+0000 7f9bc8c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9bc4143ec0 0x7f9bc415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.895+0000 7f9bc8c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9bc40715a0 0x7f9bc406fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.895+0000 7f9bc8c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc41442a0 con 0x7f9bc415ec90 2024-05-22T21:19:46.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.895+0000 7f9bc8c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bc4070260 con 0x7f9bc4143ec0 2024-05-22T21:19:46.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.895+0000 7f9bc8c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bc4070500 con 0x7f9bc40715a0 2024-05-22T21:19:46.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.895+0000 7f9bc1e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9bc4143ec0 0x7f9bc415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.895+0000 7f9bc1e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9bc4143ec0 0x7f9bc415f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50642/0 (socket says 172.21.15.161:50642) 2024-05-22T21:19:46.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.895+0000 7f9bc1e00640 1 -- 172.21.15.161:0/169346365 learned_addr learned my addr 172.21.15.161:0/169346365 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:46.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.895+0000 7f9bb9400640 1 --2- 172.21.15.161:0/169346365 >> v2:172.21.15.179:3300/0 conn(0x7f9bc40715a0 0x7f9bc406fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.895+0000 7f9bc1e00640 1 -- 172.21.15.161:0/169346365 >> v2:172.21.15.179:3300/0 conn(0x7f9bc40715a0 msgr2=0x7f9bc406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:46.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.895+0000 7f9bc1e00640 1 --2- 172.21.15.161:0/169346365 >> v2:172.21.15.179:3300/0 conn(0x7f9bc40715a0 0x7f9bc406fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.895+0000 7f9bc1e00640 1 -- 172.21.15.161:0/169346365 >> v1:172.21.15.179:6789/0 conn(0x7f9bc415ec90 legacy=0x7f9bc406bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:46.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.895+0000 7f9bc1e00640 1 -- 172.21.15.161:0/169346365 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc40707e0 con 0x7f9bc4143ec0 2024-05-22T21:19:46.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.895+0000 7f9bb9400640 1 --2- 172.21.15.161:0/169346365 >> v2:172.21.15.179:3300/0 conn(0x7f9bc40715a0 0x7f9bc406fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:46.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.895+0000 7f9bc1e00640 1 --2- 172.21.15.161:0/169346365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bc4143ec0 0x7f9bc415f830 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4074430 tx=0x7f9bb401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2fb7b490ce603 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:46.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.895+0000 7f9bc1400640 1 -- 172.21.15.161:0/169346365 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb40aa610 con 0x7f9bc4143ec0 2024-05-22T21:19:46.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc1400640 1 -- 172.21.15.161:0/169346365 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb40a6040 con 0x7f9bc4143ec0 2024-05-22T21:19:46.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc1400640 1 -- 172.21.15.161:0/169346365 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb40aa990 con 0x7f9bc4143ec0 2024-05-22T21:19:46.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc8c00640 1 -- 172.21.15.161:0/169346365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bc4143ec0 msgr2=0x7f9bc415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc8c00640 1 --2- 172.21.15.161:0/169346365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bc4143ec0 0x7f9bc415f830 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4074430 tx=0x7f9bb401c4b0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc8c00640 1 -- 172.21.15.161:0/169346365 shutdown_connections 2024-05-22T21:19:46.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc8c00640 1 --2- 172.21.15.161:0/169346365 >> v2:172.21.15.179:3300/0 conn(0x7f9bc40715a0 0x7f9bc406fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc8c00640 1 --2- 172.21.15.161:0/169346365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bc4143ec0 0x7f9bc415f830 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc8c00640 1 -- 172.21.15.161:0/169346365 >> 172.21.15.161:0/169346365 conn(0x7f9bc40b0bc0 msgr2=0x7f9bc40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:46.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc8c00640 1 -- 172.21.15.161:0/169346365 shutdown_connections 2024-05-22T21:19:46.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc8c00640 1 -- 172.21.15.161:0/169346365 wait complete. 2024-05-22T21:19:46.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc8c00640 1 Processor -- start 2024-05-22T21:19:46.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc8c00640 1 -- start start 2024-05-22T21:19:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc8c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bc4143ec0 0x7f9bc4163150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc8c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9bc415ec90 0x7f9bc4142c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc8c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bc4161680 0x7f9bc4140e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc8c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc41400f0 con 0x7f9bc4143ec0 2024-05-22T21:19:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc8c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc4161b10 con 0x7f9bc4161680 2024-05-22T21:19:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc8c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc4140390 con 0x7f9bc415ec90 2024-05-22T21:19:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bc4143ec0 0x7f9bc4163150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bc4143ec0 0x7f9bc4163150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50646/0 (socket says 172.21.15.161:50646) 2024-05-22T21:19:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc1e00640 1 -- 172.21.15.161:0/284175396 learned_addr learned my addr 172.21.15.161:0/284175396 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:46.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bb9400640 1 --2- 172.21.15.161:0/284175396 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9bc415ec90 0x7f9bc4142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc2800640 1 --2- 172.21.15.161:0/284175396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bc4161680 0x7f9bc4140e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc1e00640 1 -- 172.21.15.161:0/284175396 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9bc415ec90 msgr2=0x7f9bc4142c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc1e00640 1 --2- 172.21.15.161:0/284175396 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9bc415ec90 0x7f9bc4142c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc1e00640 1 -- 172.21.15.161:0/284175396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bc4161680 msgr2=0x7f9bc4140e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:46.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc1e00640 1 --2- 172.21.15.161:0/284175396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bc4161680 0x7f9bc4140e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:46.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.899+0000 7f9bc1e00640 1 -- 172.21.15.161:0/284175396 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb40094d0 con 0x7f9bc4143ec0 2024-05-22T21:19:46.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.903+0000 7f9bc1e00640 1 --2- 172.21.15.161:0/284175396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bc4143ec0 0x7f9bc4163150 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f9bb401a0e0 tx=0x7f9bb401acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:46.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.903+0000 7f9bbaa00640 1 -- 172.21.15.161:0/284175396 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb40b3600 con 0x7f9bc4143ec0 2024-05-22T21:19:46.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.903+0000 7f9bbaa00640 1 -- 172.21.15.161:0/284175396 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb40b3760 con 0x7f9bc4143ec0 2024-05-22T21:19:46.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.903+0000 7f9bbaa00640 1 -- 172.21.15.161:0/284175396 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb40b3b10 con 0x7f9bc4143ec0 2024-05-22T21:19:46.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.903+0000 7f9bc8c00640 1 -- 172.21.15.161:0/284175396 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc4140670 con 0x7f9bc4143ec0 2024-05-22T21:19:46.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.903+0000 7f9bc8c00640 1 -- 172.21.15.161:0/284175396 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc4163690 con 0x7f9bc4143ec0 2024-05-22T21:19:46.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.903+0000 7f9bc8c00640 1 -- 172.21.15.161:0/284175396 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc40740e0 con 0x7f9bc4143ec0 2024-05-22T21:19:46.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.903+0000 7f9bbaa00640 1 -- 172.21.15.161:0/284175396 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9bb40b3c70 con 0x7f9bc4143ec0 2024-05-22T21:19:46.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.903+0000 7f9bbaa00640 1 --2- 172.21.15.161:0/284175396 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ba0066690 0x7f9ba0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:46.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.903+0000 7f9bbaa00640 1 -- 172.21.15.161:0/284175396 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bc406ce80 con 0x7f9bc4143ec0 2024-05-22T21:19:46.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.903+0000 7f9bb9400640 1 --2- 172.21.15.161:0/284175396 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ba0066690 0x7f9ba0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:46.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.907+0000 7f9bb9400640 1 --2- 172.21.15.161:0/284175396 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ba0066690 0x7f9ba0068b50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4067960 tx=0x7f9bac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:46.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:46.907+0000 7f9bbaa00640 1 -- 172.21.15.161:0/284175396 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9bc40740e0 con 0x7f9bc4143ec0 2024-05-22T21:19:47.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.063+0000 7f9bc8c00640 1 -- 172.21.15.161:0/284175396 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bc40740e0 con 0x7f9bc4143ec0 2024-05-22T21:19:47.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.067+0000 7f9bbaa00640 1 -- 172.21.15.161:0/284175396 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16121 (secure 0 0 0) 0x7f9bc40740e0 con 0x7f9bc4143ec0 2024-05-22T21:19:47.068 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:47.068 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":139,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:19:46.384582+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:44.080742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:45.388618+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:47.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.067+0000 7f9bc8c00640 1 -- 172.21.15.161:0/284175396 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ba0066690 msgr2=0x7f9ba0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:47.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.067+0000 7f9bc8c00640 1 --2- 172.21.15.161:0/284175396 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ba0066690 0x7f9ba0068b50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4067960 tx=0x7f9bac008040 comp rx=0 tx=0).stop 2024-05-22T21:19:47.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.067+0000 7f9bc8c00640 1 -- 172.21.15.161:0/284175396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bc4143ec0 msgr2=0x7f9bc4163150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:47.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.067+0000 7f9bc8c00640 1 --2- 172.21.15.161:0/284175396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bc4143ec0 0x7f9bc4163150 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f9bb401a0e0 tx=0x7f9bb401acf0 comp rx=0 tx=0).stop 2024-05-22T21:19:47.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.067+0000 7f9bc8c00640 1 -- 172.21.15.161:0/284175396 shutdown_connections 2024-05-22T21:19:47.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.067+0000 7f9bc8c00640 1 --2- 172.21.15.161:0/284175396 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9ba0066690 0x7f9ba0068b50 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:47.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.067+0000 7f9bc8c00640 1 --2- 172.21.15.161:0/284175396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bc4161680 0x7f9bc4140e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:47.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.067+0000 7f9bc8c00640 1 --2- 172.21.15.161:0/284175396 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9bc415ec90 0x7f9bc4142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:47.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.067+0000 7f9bc8c00640 1 --2- 172.21.15.161:0/284175396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bc4143ec0 0x7f9bc4163150 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:47.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.067+0000 7f9bc8c00640 1 -- 172.21.15.161:0/284175396 >> 172.21.15.161:0/284175396 conn(0x7f9bc40b0bc0 msgr2=0x7f9bc415d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:47.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.067+0000 7f9bc8c00640 1 -- 172.21.15.161:0/284175396 shutdown_connections 2024-05-22T21:19:47.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.067+0000 7f9bc8c00640 1 -- 172.21.15.161:0/284175396 wait complete. 2024-05-22T21:19:47.089 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-22T21:19:47.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.223+0000 7f8d97200640 1 Processor -- start 2024-05-22T21:19:47.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.223+0000 7f8d97200640 1 -- start start 2024-05-22T21:19:47.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.223+0000 7f8d97200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8d98143ed0 0x7f8d981442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:47.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.227+0000 7f8d97200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8d980ce370 0x7f8d980b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:47.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.227+0000 7f8d97200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d980ce830 con 0x7f8d981448e0 2024-05-22T21:19:47.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.227+0000 7f8d97200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d980b2e60 con 0x7f8d980ce370 2024-05-22T21:19:47.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.227+0000 7f8d97200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d980b3140 con 0x7f8d98143ed0 2024-05-22T21:19:47.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.227+0000 7f8d96800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8d980ce370 0x7f8d980b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:47.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.227+0000 7f8d96800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8d980ce370 0x7f8d980b27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50662/0 (socket says 172.21.15.161:50662) 2024-05-22T21:19:47.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.227+0000 7f8d96800640 1 -- 172.21.15.161:0/2057991148 learned_addr learned my addr 172.21.15.161:0/2057991148 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:47.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.227+0000 7f8d95e00640 1 --2- 172.21.15.161:0/2057991148 >> v2:172.21.15.179:3300/0 conn(0x7f8d98143ed0 0x7f8d981442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:47.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.227+0000 7f8d96800640 1 -- 172.21.15.161:0/2057991148 >> v2:172.21.15.179:3300/0 conn(0x7f8d98143ed0 msgr2=0x7f8d981442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:47.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.227+0000 7f8d96800640 1 --2- 172.21.15.161:0/2057991148 >> v2:172.21.15.179:3300/0 conn(0x7f8d98143ed0 0x7f8d981442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:47.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.227+0000 7f8d96800640 1 -- 172.21.15.161:0/2057991148 >> v1:172.21.15.179:6789/0 conn(0x7f8d981448e0 legacy=0x7f8d980b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:47.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.227+0000 7f8d96800640 1 -- 172.21.15.161:0/2057991148 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d980b3420 con 0x7f8d980ce370 2024-05-22T21:19:47.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.227+0000 7f8d95e00640 1 --2- 172.21.15.161:0/2057991148 >> v2:172.21.15.179:3300/0 conn(0x7f8d98143ed0 0x7f8d981442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:47.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.227+0000 7f8d96800640 1 --2- 172.21.15.161:0/2057991148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d980ce370 0x7f8d980b27f0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f8d98144e50 tx=0x7f8d9001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97f32749598ee3a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:47.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d94a00640 1 -- 172.21.15.161:0/2057991148 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d900ab620 con 0x7f8d980ce370 2024-05-22T21:19:47.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d94a00640 1 -- 172.21.15.161:0/2057991148 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d900ab780 con 0x7f8d980ce370 2024-05-22T21:19:47.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d94a00640 1 -- 172.21.15.161:0/2057991148 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d900abb10 con 0x7f8d980ce370 2024-05-22T21:19:47.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d97200640 1 -- 172.21.15.161:0/2057991148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d980ce370 msgr2=0x7f8d980b27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:47.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d97200640 1 --2- 172.21.15.161:0/2057991148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d980ce370 0x7f8d980b27f0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f8d98144e50 tx=0x7f8d9001c400 comp rx=0 tx=0).stop 2024-05-22T21:19:47.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d97200640 1 -- 172.21.15.161:0/2057991148 shutdown_connections 2024-05-22T21:19:47.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d97200640 1 --2- 172.21.15.161:0/2057991148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d980ce370 0x7f8d980b27f0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:47.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d97200640 1 --2- 172.21.15.161:0/2057991148 >> v2:172.21.15.179:3300/0 conn(0x7f8d98143ed0 0x7f8d981442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:47.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d97200640 1 -- 172.21.15.161:0/2057991148 >> 172.21.15.161:0/2057991148 conn(0x7f8d980cec00 msgr2=0x7f8d980cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:47.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d97200640 1 -- 172.21.15.161:0/2057991148 shutdown_connections 2024-05-22T21:19:47.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d97200640 1 -- 172.21.15.161:0/2057991148 wait complete. 2024-05-22T21:19:47.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d97200640 1 Processor -- start 2024-05-22T21:19:47.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d97200640 1 -- start start 2024-05-22T21:19:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d97200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d980ce370 0x7f8d9813d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d97200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d98143ed0 0x7f8d98070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d97200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d981448e0 0x7f8d98071010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d97200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9813ff00 con 0x7f8d98143ed0 2024-05-22T21:19:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d97200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d98141020 con 0x7f8d980ce370 2024-05-22T21:19:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d97200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d98140260 con 0x7f8d981448e0 2024-05-22T21:19:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d95400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d98143ed0 0x7f8d98070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d95400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d98143ed0 0x7f8d98070ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50672/0 (socket says 172.21.15.161:50672) 2024-05-22T21:19:47.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d95400640 1 -- 172.21.15.161:0/1939205857 learned_addr learned my addr 172.21.15.161:0/1939205857 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:47.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d95e00640 1 --2- 172.21.15.161:0/1939205857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d980ce370 0x7f8d9813d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:47.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.231+0000 7f8d96800640 1 --2- 172.21.15.161:0/1939205857 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d981448e0 0x7f8d98071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:47.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.235+0000 7f8d95400640 1 -- 172.21.15.161:0/1939205857 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d981448e0 msgr2=0x7f8d98071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:47.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.235+0000 7f8d95400640 1 --2- 172.21.15.161:0/1939205857 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d981448e0 0x7f8d98071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:47.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.235+0000 7f8d95400640 1 -- 172.21.15.161:0/1939205857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d980ce370 msgr2=0x7f8d9813d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:47.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.235+0000 7f8d95400640 1 --2- 172.21.15.161:0/1939205857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d980ce370 0x7f8d9813d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:47.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.235+0000 7f8d95400640 1 -- 172.21.15.161:0/1939205857 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d900094d0 con 0x7f8d98143ed0 2024-05-22T21:19:47.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.235+0000 7f8d95400640 1 --2- 172.21.15.161:0/1939205857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d98143ed0 0x7f8d98070ad0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400c3c0 tx=0x7f8d8400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:47.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.235+0000 7f8d8e000640 1 -- 172.21.15.161:0/1939205857 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d840052d0 con 0x7f8d98143ed0 2024-05-22T21:19:47.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.235+0000 7f8d8e000640 1 -- 172.21.15.161:0/1939205857 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d84005430 con 0x7f8d98143ed0 2024-05-22T21:19:47.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.235+0000 7f8d8e000640 1 -- 172.21.15.161:0/1939205857 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84003840 con 0x7f8d98143ed0 2024-05-22T21:19:47.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.235+0000 7f8d97200640 1 -- 172.21.15.161:0/1939205857 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d98140540 con 0x7f8d98143ed0 2024-05-22T21:19:47.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.235+0000 7f8d97200640 1 -- 172.21.15.161:0/1939205857 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d98140b00 con 0x7f8d98143ed0 2024-05-22T21:19:47.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.235+0000 7f8d97200640 1 -- 172.21.15.161:0/1939205857 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d981403a0 con 0x7f8d98143ed0 2024-05-22T21:19:47.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.235+0000 7f8d8e000640 1 -- 172.21.15.161:0/1939205857 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8d840039a0 con 0x7f8d98143ed0 2024-05-22T21:19:47.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.235+0000 7f8d8e000640 1 --2- 172.21.15.161:0/1939205857 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8d78066690 0x7f8d78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:47.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.235+0000 7f8d8e000640 1 -- 172.21.15.161:0/1939205857 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d84083610 con 0x7f8d98143ed0 2024-05-22T21:19:47.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.235+0000 7f8d95e00640 1 --2- 172.21.15.161:0/1939205857 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8d78066690 0x7f8d78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:47.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.239+0000 7f8d95e00640 1 --2- 172.21.15.161:0/1939205857 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8d78066690 0x7f8d78068b50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8d98142500 tx=0x7f8d88009890 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:47.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.239+0000 7f8d8e000640 1 -- 172.21.15.161:0/1939205857 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8d981403a0 con 0x7f8d98143ed0 2024-05-22T21:19:47.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:47.427+0000 7f8d97200640 1 -- 172.21.15.161:0/1939205857 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8d981403a0 con 0x7f8d98143ed0 2024-05-22T21:19:48.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.399+0000 7f8d8e000640 1 -- 172.21.15.161:0/1939205857 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8d981403a0 con 0x7f8d98143ed0 2024-05-22T21:19:48.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.399+0000 7f8d97200640 1 -- 172.21.15.161:0/1939205857 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8d78066690 msgr2=0x7f8d78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:48.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.399+0000 7f8d97200640 1 --2- 172.21.15.161:0/1939205857 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8d78066690 0x7f8d78068b50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8d98142500 tx=0x7f8d88009890 comp rx=0 tx=0).stop 2024-05-22T21:19:48.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.399+0000 7f8d97200640 1 -- 172.21.15.161:0/1939205857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d98143ed0 msgr2=0x7f8d98070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:48.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.399+0000 7f8d97200640 1 --2- 172.21.15.161:0/1939205857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d98143ed0 0x7f8d98070ad0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400c3c0 tx=0x7f8d8400c880 comp rx=0 tx=0).stop 2024-05-22T21:19:48.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.399+0000 7f8d97200640 1 -- 172.21.15.161:0/1939205857 shutdown_connections 2024-05-22T21:19:48.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.399+0000 7f8d97200640 1 --2- 172.21.15.161:0/1939205857 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8d78066690 0x7f8d78068b50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:48.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.399+0000 7f8d97200640 1 --2- 172.21.15.161:0/1939205857 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d981448e0 0x7f8d98071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:48.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.399+0000 7f8d97200640 1 --2- 172.21.15.161:0/1939205857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d98143ed0 0x7f8d98070ad0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:48.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.399+0000 7f8d97200640 1 --2- 172.21.15.161:0/1939205857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d980ce370 0x7f8d9813d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:48.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.399+0000 7f8d97200640 1 -- 172.21.15.161:0/1939205857 >> 172.21.15.161:0/1939205857 conn(0x7f8d980cec00 msgr2=0x7f8d98142880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:48.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.399+0000 7f8d97200640 1 -- 172.21.15.161:0/1939205857 shutdown_connections 2024-05-22T21:19:48.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.399+0000 7f8d97200640 1 -- 172.21.15.161:0/1939205857 wait complete. 2024-05-22T21:19:48.428 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-22T21:19:48.428 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-22T21:19:48.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.567+0000 7fac75e00640 1 Processor -- start 2024-05-22T21:19:48.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.567+0000 7fac75e00640 1 -- start start 2024-05-22T21:19:48.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac75e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fac701436d0 0x7fac700cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:48.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac75e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac7006fa80 0x7fac700b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:48.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac75e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac70070b70 con 0x7fac700b1b10 2024-05-22T21:19:48.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac75e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac700cc720 con 0x7fac701436d0 2024-05-22T21:19:48.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac75e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7006ff60 con 0x7fac7006fa80 2024-05-22T21:19:48.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac75400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fac700b1b10 0x7fac70071000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48870/0 (socket says 172.21.15.161:48870) 2024-05-22T21:19:48.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac75400640 1 -- 172.21.15.161:0/1989011854 learned_addr learned my addr 172.21.15.161:0/1989011854 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:48.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac74a00640 1 --2- 172.21.15.161:0/1989011854 >> v2:172.21.15.179:3300/0 conn(0x7fac701436d0 0x7fac700cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:48.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6fe00640 1 --2- 172.21.15.161:0/1989011854 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac7006fa80 0x7fac700b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:48.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6f400640 1 -- 172.21.15.161:0/1989011854 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988329298 0 0) 0x7fac70070b70 con 0x7fac700b1b10 2024-05-22T21:19:48.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6f400640 1 -- 172.21.15.161:0/1989011854 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac58003530 con 0x7fac700b1b10 2024-05-22T21:19:48.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6f400640 1 -- 172.21.15.161:0/1989011854 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2294546411 0 0) 0x7fac58003530 con 0x7fac700b1b10 2024-05-22T21:19:48.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6f400640 1 -- 172.21.15.161:0/1989011854 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac7006fa80 msgr2=0x7fac700b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:48.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6f400640 1 --2- 172.21.15.161:0/1989011854 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac7006fa80 0x7fac700b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:48.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6f400640 1 -- 172.21.15.161:0/1989011854 >> v2:172.21.15.179:3300/0 conn(0x7fac701436d0 msgr2=0x7fac700cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:48.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6f400640 1 --2- 172.21.15.161:0/1989011854 >> v2:172.21.15.179:3300/0 conn(0x7fac701436d0 0x7fac700cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:48.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6f400640 1 -- 172.21.15.161:0/1989011854 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac70070240 con 0x7fac700b1b10 2024-05-22T21:19:48.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6f400640 1 -- 172.21.15.161:0/1989011854 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fac68001a00 con 0x7fac700b1b10 2024-05-22T21:19:48.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac74a00640 1 --2- 172.21.15.161:0/1989011854 >> v2:172.21.15.179:3300/0 conn(0x7fac701436d0 0x7fac700cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:48.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6f400640 1 -- 172.21.15.161:0/1989011854 >> v1:172.21.15.161:6789/0 conn(0x7fac700b1b10 legacy=0x7fac70071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:48.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6f400640 1 --2- 172.21.15.161:0/1989011854 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac58003980 0x7fac58003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:48.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6f400640 1 --2- 172.21.15.161:0/1989011854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac58004370 0x7fac58004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:48.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6f400640 1 --2- 172.21.15.161:0/1989011854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac58004f60 0x7fac5800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:48.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6f400640 1 -- 172.21.15.161:0/1989011854 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac700cc720 con 0x7fac58004370 2024-05-22T21:19:48.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6f400640 1 -- 172.21.15.161:0/1989011854 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7006ff60 con 0x7fac58004f60 2024-05-22T21:19:48.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6f400640 1 -- 172.21.15.161:0/1989011854 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac5800bf00 con 0x7fac58003980 2024-05-22T21:19:48.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6fe00640 1 --2- 172.21.15.161:0/1989011854 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac7006fa80 0x7fac700b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:48.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6fe00640 1 --2- 172.21.15.161:0/1989011854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac58004370 0x7fac58004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:48.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac75400640 1 --2- 172.21.15.161:0/1989011854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac58004f60 0x7fac5800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:48.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac74a00640 1 --2- 172.21.15.161:0/1989011854 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac58003980 0x7fac58003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:48.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6fe00640 1 -- 172.21.15.161:0/1989011854 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac58003980 msgr2=0x7fac58003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:48.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6fe00640 1 --2- 172.21.15.161:0/1989011854 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac58003980 0x7fac58003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:48.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6fe00640 1 -- 172.21.15.161:0/1989011854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac58004f60 msgr2=0x7fac5800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:48.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6fe00640 1 --2- 172.21.15.161:0/1989011854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac58004f60 0x7fac5800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:48.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.571+0000 7fac6fe00640 1 -- 172.21.15.161:0/1989011854 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac58003530 con 0x7fac58004370 2024-05-22T21:19:48.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.575+0000 7fac6fe00640 1 --2- 172.21.15.161:0/1989011854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac58004370 0x7fac58004850 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fac7006b350 tx=0x7fac5c01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f24589f99dbdfd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:48.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.575+0000 7fac6f400640 1 -- 172.21.15.161:0/1989011854 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c004030 con 0x7fac58004370 2024-05-22T21:19:48.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.575+0000 7fac6f400640 1 -- 172.21.15.161:0/1989011854 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5c004190 con 0x7fac58004370 2024-05-22T21:19:48.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.575+0000 7fac6f400640 1 -- 172.21.15.161:0/1989011854 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c004520 con 0x7fac58004370 2024-05-22T21:19:48.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.575+0000 7fac75e00640 1 -- 172.21.15.161:0/1989011854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac58004370 msgr2=0x7fac58004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:48.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.575+0000 7fac75e00640 1 --2- 172.21.15.161:0/1989011854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac58004370 0x7fac58004850 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fac7006b350 tx=0x7fac5c01ca20 comp rx=0 tx=0).stop 2024-05-22T21:19:48.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.575+0000 7fac75400640 1 -- 172.21.15.161:0/1989011854 reap_dead start 2024-05-22T21:19:48.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.575+0000 7fac75e00640 1 -- 172.21.15.161:0/1989011854 shutdown_connections 2024-05-22T21:19:48.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.575+0000 7fac75e00640 1 -- 172.21.15.161:0/1989011854 >> 172.21.15.161:0/1989011854 conn(0x7fac700cec00 msgr2=0x7fac700cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:48.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.575+0000 7fac75e00640 1 -- 172.21.15.161:0/1989011854 shutdown_connections 2024-05-22T21:19:48.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.575+0000 7fac75e00640 1 -- 172.21.15.161:0/1989011854 wait complete. 2024-05-22T21:19:48.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.575+0000 7fac75e00640 1 Processor -- start 2024-05-22T21:19:48.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75e00640 1 -- start start 2024-05-22T21:19:48.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac58004370 0x7fac701e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:48.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac70154dd0 0x7fac70155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:48.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac70159680 0x7fac70159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:48.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7013e9b0 con 0x7fac70159680 2024-05-22T21:19:48.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7015a040 con 0x7fac70154dd0 2024-05-22T21:19:48.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7013cac0 con 0x7fac58004370 2024-05-22T21:19:48.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac70159680 0x7fac70159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:48.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac70159680 0x7fac70159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50686/0 (socket says 172.21.15.161:50686) 2024-05-22T21:19:48.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75400640 1 -- 172.21.15.161:0/2749208022 learned_addr learned my addr 172.21.15.161:0/2749208022 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:48.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac6fe00640 1 --2- 172.21.15.161:0/2749208022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac70154dd0 0x7fac70155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:48.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac74a00640 1 --2- 172.21.15.161:0/2749208022 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac58004370 0x7fac701e7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:48.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75400640 1 -- 172.21.15.161:0/2749208022 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac58004370 msgr2=0x7fac701e7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:48.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75400640 1 --2- 172.21.15.161:0/2749208022 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac58004370 0x7fac701e7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:48.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75400640 1 -- 172.21.15.161:0/2749208022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac70154dd0 msgr2=0x7fac70155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:48.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75400640 1 --2- 172.21.15.161:0/2749208022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac70154dd0 0x7fac70155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:48.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75400640 1 -- 172.21.15.161:0/2749208022 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7013dd30 con 0x7fac70159680 2024-05-22T21:19:48.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac6fe00640 1 --2- 172.21.15.161:0/2749208022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac70154dd0 0x7fac70155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:48.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75400640 1 --2- 172.21.15.161:0/2749208022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac70159680 0x7fac70159b00 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fac700cbfd0 tx=0x7fac68007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:48.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac6cc00640 1 -- 172.21.15.161:0/2749208022 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6800abc0 con 0x7fac70159680 2024-05-22T21:19:48.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac6cc00640 1 -- 172.21.15.161:0/2749208022 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6800ad60 con 0x7fac70159680 2024-05-22T21:19:48.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75e00640 1 -- 172.21.15.161:0/2749208022 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7013e010 con 0x7fac70159680 2024-05-22T21:19:48.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac6cc00640 1 -- 172.21.15.161:0/2749208022 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac68013630 con 0x7fac70159680 2024-05-22T21:19:48.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75e00640 1 -- 172.21.15.161:0/2749208022 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7013e5d0 con 0x7fac70159680 2024-05-22T21:19:48.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.579+0000 7fac75e00640 1 -- 172.21.15.161:0/2749208022 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac701f4280 con 0x7fac70159680 2024-05-22T21:19:48.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.583+0000 7fac6cc00640 1 -- 172.21.15.161:0/2749208022 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fac68013790 con 0x7fac70159680 2024-05-22T21:19:48.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.583+0000 7fac6cc00640 1 --2- 172.21.15.161:0/2749208022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac500666c0 0x7fac50068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:48.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.583+0000 7fac6cc00640 1 -- 172.21.15.161:0/2749208022 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac68084940 con 0x7fac70159680 2024-05-22T21:19:48.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.583+0000 7fac74a00640 1 --2- 172.21.15.161:0/2749208022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac500666c0 0x7fac50068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:48.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.583+0000 7fac74a00640 1 --2- 172.21.15.161:0/2749208022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac500666c0 0x7fac50068b80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fac64009cb0 tx=0x7fac64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:48.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.587+0000 7fac6cc00640 1 -- 172.21.15.161:0/2749208022 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fac701f4280 con 0x7fac70159680 2024-05-22T21:19:48.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:48.771+0000 7fac75e00640 1 -- 172.21.15.161:0/2749208022 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fac701f4280 con 0x7fac70159680 2024-05-22T21:19:49.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.407+0000 7fac6cc00640 1 -- 172.21.15.161:0/2749208022 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) v1 ==== 122+0+0 (secure 0 0 0) 0x7fac701f4280 con 0x7fac70159680 2024-05-22T21:19:49.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.407+0000 7fac75e00640 1 -- 172.21.15.161:0/2749208022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac500666c0 msgr2=0x7fac50068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:49.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.407+0000 7fac75e00640 1 --2- 172.21.15.161:0/2749208022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac500666c0 0x7fac50068b80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fac64009cb0 tx=0x7fac64008040 comp rx=0 tx=0).stop 2024-05-22T21:19:49.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.407+0000 7fac75e00640 1 -- 172.21.15.161:0/2749208022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac70159680 msgr2=0x7fac70159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:49.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.407+0000 7fac75e00640 1 --2- 172.21.15.161:0/2749208022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac70159680 0x7fac70159b00 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fac700cbfd0 tx=0x7fac68007b00 comp rx=0 tx=0).stop 2024-05-22T21:19:49.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.407+0000 7fac75e00640 1 -- 172.21.15.161:0/2749208022 shutdown_connections 2024-05-22T21:19:49.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.407+0000 7fac75e00640 1 --2- 172.21.15.161:0/2749208022 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac500666c0 0x7fac50068b80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.407+0000 7fac75e00640 1 --2- 172.21.15.161:0/2749208022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac70159680 0x7fac70159b00 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.407+0000 7fac75e00640 1 --2- 172.21.15.161:0/2749208022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac70154dd0 0x7fac70155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.407+0000 7fac75e00640 1 --2- 172.21.15.161:0/2749208022 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac58004370 0x7fac701e7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.407+0000 7fac75e00640 1 -- 172.21.15.161:0/2749208022 >> 172.21.15.161:0/2749208022 conn(0x7fac700cec00 msgr2=0x7fac700af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:49.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.407+0000 7fac75e00640 1 -- 172.21.15.161:0/2749208022 shutdown_connections 2024-05-22T21:19:49.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.407+0000 7fac75e00640 1 -- 172.21.15.161:0/2749208022 wait complete. 2024-05-22T21:19:49.433 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-22T21:19:49.434 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:49.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.567+0000 7f947b400640 1 Processor -- start 2024-05-22T21:19:49.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.567+0000 7f947b400640 1 -- start start 2024-05-22T21:19:49.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f947b400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f947415dd50 0x7f947406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:49.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f947b400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947415e130 0x7f947415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:49.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f947b400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9474071670 con 0x7f947415d410 2024-05-22T21:19:49.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f947b400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f947415d8a0 con 0x7f947415dd50 2024-05-22T21:19:49.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f947b400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947406de40 con 0x7f947415e130 2024-05-22T21:19:49.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f9479400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f947415d410 0x7f947415a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48880/0 (socket says 172.21.15.161:48880) 2024-05-22T21:19:49.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f9479400640 1 -- 172.21.15.161:0/2569152714 learned_addr learned my addr 172.21.15.161:0/2569152714 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:49.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f9473e00640 1 --2- 172.21.15.161:0/2569152714 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947415e130 0x7f947415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:49.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f9478a00640 1 --2- 172.21.15.161:0/2569152714 >> v2:172.21.15.179:3300/0 conn(0x7f947415dd50 0x7f947406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:49.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f9473400640 1 -- 172.21.15.161:0/2569152714 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265447806 0 0) 0x7f9474071670 con 0x7f947415d410 2024-05-22T21:19:49.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f9473400640 1 -- 172.21.15.161:0/2569152714 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9458005740 con 0x7f947415d410 2024-05-22T21:19:49.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f9473e00640 1 -- 172.21.15.161:0/2569152714 >> v2:172.21.15.179:3300/0 conn(0x7f947415dd50 msgr2=0x7f947406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:49.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f9473e00640 1 --2- 172.21.15.161:0/2569152714 >> v2:172.21.15.179:3300/0 conn(0x7f947415dd50 0x7f947406d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f9473400640 1 -- 172.21.15.161:0/2569152714 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4212198033 0 0) 0x7f9458005740 con 0x7f947415d410 2024-05-22T21:19:49.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f9473e00640 1 -- 172.21.15.161:0/2569152714 >> v1:172.21.15.161:6789/0 conn(0x7f947415d410 legacy=0x7f947415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:49.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f9473e00640 1 -- 172.21.15.161:0/2569152714 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9474070a60 con 0x7f947415e130 2024-05-22T21:19:49.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.571+0000 7f9478a00640 1 --2- 172.21.15.161:0/2569152714 >> v2:172.21.15.179:3300/0 conn(0x7f947415dd50 0x7f947406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:49.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f9473e00640 1 --2- 172.21.15.161:0/2569152714 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947415e130 0x7f947415ced0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f94600097c0 tx=0x7f946001c300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2e62159d99924a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:49.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f9473400640 1 -- 172.21.15.161:0/2569152714 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c001680 con 0x7f947415e130 2024-05-22T21:19:49.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f9473400640 1 -- 172.21.15.161:0/2569152714 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94600a7050 con 0x7f947415e130 2024-05-22T21:19:49.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f9473400640 1 -- 172.21.15.161:0/2569152714 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94600ab620 con 0x7f947415e130 2024-05-22T21:19:49.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f947b400640 1 -- 172.21.15.161:0/2569152714 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947415e130 msgr2=0x7f947415ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:49.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f947b400640 1 --2- 172.21.15.161:0/2569152714 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947415e130 0x7f947415ced0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f94600097c0 tx=0x7f946001c300 comp rx=0 tx=0).stop 2024-05-22T21:19:49.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f947b400640 1 -- 172.21.15.161:0/2569152714 shutdown_connections 2024-05-22T21:19:49.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f947b400640 1 --2- 172.21.15.161:0/2569152714 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947415e130 0x7f947415ced0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f947b400640 1 --2- 172.21.15.161:0/2569152714 >> v2:172.21.15.179:3300/0 conn(0x7f947415dd50 0x7f947406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f947b400640 1 -- 172.21.15.161:0/2569152714 >> 172.21.15.161:0/2569152714 conn(0x7f94740b0bc0 msgr2=0x7f94740b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:49.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f947b400640 1 -- 172.21.15.161:0/2569152714 shutdown_connections 2024-05-22T21:19:49.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f947b400640 1 -- 172.21.15.161:0/2569152714 wait complete. 2024-05-22T21:19:49.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f947b400640 1 Processor -- start 2024-05-22T21:19:49.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f947b400640 1 -- start start 2024-05-22T21:19:49.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f947b400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f947415d410 0x7f947414c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:49.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f947b400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947415dd50 0x7f9474150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:49.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f947b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f947415e130 0x7f9474154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:49.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f947b400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94740b2a00 con 0x7f947415e130 2024-05-22T21:19:49.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f947b400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94741514a0 con 0x7f947415d410 2024-05-22T21:19:49.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.575+0000 7f947b400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9474151780 con 0x7f947415dd50 2024-05-22T21:19:49.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f9479400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f947415e130 0x7f9474154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:49.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f9479400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f947415e130 0x7f9474154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50690/0 (socket says 172.21.15.161:50690) 2024-05-22T21:19:49.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f9479400640 1 -- 172.21.15.161:0/1399600012 learned_addr learned my addr 172.21.15.161:0/1399600012 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:49.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f9478a00640 1 --2- 172.21.15.161:0/1399600012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f947415d410 0x7f947414c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:49.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f9473e00640 1 --2- 172.21.15.161:0/1399600012 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947415dd50 0x7f9474150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:49.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f9479400640 1 -- 172.21.15.161:0/1399600012 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947415dd50 msgr2=0x7f9474150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:49.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f9479400640 1 --2- 172.21.15.161:0/1399600012 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947415dd50 0x7f9474150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f9479400640 1 -- 172.21.15.161:0/1399600012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f947415d410 msgr2=0x7f947414c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:49.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f9479400640 1 --2- 172.21.15.161:0/1399600012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f947415d410 0x7f947414c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f9479400640 1 -- 172.21.15.161:0/1399600012 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9464009a70 con 0x7f947415e130 2024-05-22T21:19:49.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f9479400640 1 --2- 172.21.15.161:0/1399600012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f947415e130 0x7f9474154dd0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f946c00c750 tx=0x7f946c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:49.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f9470c00640 1 -- 172.21.15.161:0/1399600012 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c005ae0 con 0x7f947415e130 2024-05-22T21:19:49.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f9470c00640 1 -- 172.21.15.161:0/1399600012 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c005c40 con 0x7f947415e130 2024-05-22T21:19:49.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f947b400640 1 -- 172.21.15.161:0/1399600012 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9460009420 con 0x7f947415e130 2024-05-22T21:19:49.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f9470c00640 1 -- 172.21.15.161:0/1399600012 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c007070 con 0x7f947415e130 2024-05-22T21:19:49.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f947b400640 1 -- 172.21.15.161:0/1399600012 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947415cfd0 con 0x7f947415e130 2024-05-22T21:19:49.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f947b400640 1 -- 172.21.15.161:0/1399600012 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94740b2e50 con 0x7f947415e130 2024-05-22T21:19:49.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.579+0000 7f9470c00640 1 -- 172.21.15.161:0/1399600012 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f946c0071d0 con 0x7f947415e130 2024-05-22T21:19:49.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.583+0000 7f9470c00640 1 --2- 172.21.15.161:0/1399600012 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9454066690 0x7f9454068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:49.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.583+0000 7f9470c00640 1 -- 172.21.15.161:0/1399600012 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f946c083c90 con 0x7f947415e130 2024-05-22T21:19:49.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.583+0000 7f9478a00640 1 --2- 172.21.15.161:0/1399600012 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9454066690 0x7f9454068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:49.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.583+0000 7f9478a00640 1 --2- 172.21.15.161:0/1399600012 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9454066690 0x7f9454068b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f94640114e0 tx=0x7f946400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:49.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.583+0000 7f9470c00640 1 -- 172.21.15.161:0/1399600012 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f94740b2e50 con 0x7f947415e130 2024-05-22T21:19:49.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.771+0000 7f947b400640 1 -- 172.21.15.161:0/1399600012 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94740b2e50 con 0x7f947415e130 2024-05-22T21:19:49.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.771+0000 7f9470c00640 1 -- 172.21.15.161:0/1399600012 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4745 (secure 0 0 0) 0x7f94740b2e50 con 0x7f947415e130 2024-05-22T21:19:49.773 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:49.773 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/524372753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":524372753},{"type":"v1","addr":"172.21.15.161:6836","nonce":524372753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3762119379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3762119379},{"type":"v1","addr":"172.21.15.179:6833","nonce":3762119379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3087241366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3087241366},{"type":"v1","addr":"172.21.15.179:6835","nonce":3087241366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:46.384984+0000","modified":"2024-05-22T21:19:49.400019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7572":{"gid":7572,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/3945715002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3945715002},{"type":"v1","addr":"172.21.15.161:6837","nonce":3945715002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-22T21:19:49.773 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 62 2024-05-22T21:19:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.771+0000 7f947b400640 1 -- 172.21.15.161:0/1399600012 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9454066690 msgr2=0x7f9454068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.771+0000 7f947b400640 1 --2- 172.21.15.161:0/1399600012 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9454066690 0x7f9454068b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f94640114e0 tx=0x7f946400b040 comp rx=0 tx=0).stop 2024-05-22T21:19:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.771+0000 7f947b400640 1 -- 172.21.15.161:0/1399600012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f947415e130 msgr2=0x7f9474154dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.771+0000 7f947b400640 1 --2- 172.21.15.161:0/1399600012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f947415e130 0x7f9474154dd0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f946c00c750 tx=0x7f946c00cc10 comp rx=0 tx=0).stop 2024-05-22T21:19:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.771+0000 7f947b400640 1 -- 172.21.15.161:0/1399600012 shutdown_connections 2024-05-22T21:19:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.771+0000 7f947b400640 1 --2- 172.21.15.161:0/1399600012 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9454066690 0x7f9454068b50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.771+0000 7f947b400640 1 --2- 172.21.15.161:0/1399600012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f947415e130 0x7f9474154dd0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.771+0000 7f947b400640 1 --2- 172.21.15.161:0/1399600012 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947415dd50 0x7f9474150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.771+0000 7f947b400640 1 --2- 172.21.15.161:0/1399600012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f947415d410 0x7f947414c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.771+0000 7f947b400640 1 -- 172.21.15.161:0/1399600012 >> 172.21.15.161:0/1399600012 conn(0x7f94740b0bc0 msgr2=0x7f947414f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:49.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.771+0000 7f947b400640 1 -- 172.21.15.161:0/1399600012 shutdown_connections 2024-05-22T21:19:49.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.771+0000 7f947b400640 1 -- 172.21.15.161:0/1399600012 wait complete. 2024-05-22T21:19:49.795 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:19:49.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.927+0000 7fc8ecc00640 1 Processor -- start 2024-05-22T21:19:49.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.927+0000 7fc8ecc00640 1 -- start start 2024-05-22T21:19:49.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.927+0000 7fc8ecc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8e8143ec0 0x7fc8e815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:49.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.927+0000 7fc8ecc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc8e815ec90 0x7fc8e815f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:49.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.927+0000 7fc8ecc00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e81442a0 con 0x7fc8e80715a0 2024-05-22T21:19:49.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.927+0000 7fc8ecc00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e815f680 con 0x7fc8e815ec90 2024-05-22T21:19:49.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.927+0000 7fc8ecc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e8070500 con 0x7fc8e8143ec0 2024-05-22T21:19:49.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.931+0000 7fc8e6800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc8e815ec90 0x7fc8e815f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:49.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.931+0000 7fc8e6800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc8e815ec90 0x7fc8e815f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50704/0 (socket says 172.21.15.161:50704) 2024-05-22T21:19:49.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.931+0000 7fc8e6800640 1 -- 172.21.15.161:0/3428800899 learned_addr learned my addr 172.21.15.161:0/3428800899 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:49.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.931+0000 7fc8e5e00640 1 --2- 172.21.15.161:0/3428800899 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8e8143ec0 0x7fc8e815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:49.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.931+0000 7fc8e6800640 1 -- 172.21.15.161:0/3428800899 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8e8143ec0 msgr2=0x7fc8e815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:49.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.931+0000 7fc8e6800640 1 --2- 172.21.15.161:0/3428800899 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8e8143ec0 0x7fc8e815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.931+0000 7fc8e6800640 1 -- 172.21.15.161:0/3428800899 >> v1:172.21.15.179:6789/0 conn(0x7fc8e80715a0 legacy=0x7fc8e806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:49.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.931+0000 7fc8e6800640 1 -- 172.21.15.161:0/3428800899 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e80707e0 con 0x7fc8e815ec90 2024-05-22T21:19:49.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.931+0000 7fc8e6800640 1 --2- 172.21.15.161:0/3428800899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e815ec90 0x7fc8e815f140 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009e50 tx=0x7fc8e001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c85e9a0255d209 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:49.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.931+0000 7fc8e5400640 1 -- 172.21.15.161:0/3428800899 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00a57d0 con 0x7fc8e815ec90 2024-05-22T21:19:49.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.931+0000 7fc8e5400640 1 -- 172.21.15.161:0/3428800899 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e00a5930 con 0x7fc8e815ec90 2024-05-22T21:19:49.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.931+0000 7fc8e5400640 1 -- 172.21.15.161:0/3428800899 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00a5cc0 con 0x7fc8e815ec90 2024-05-22T21:19:49.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8ecc00640 1 -- 172.21.15.161:0/3428800899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e815ec90 msgr2=0x7fc8e815f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:49.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8ecc00640 1 --2- 172.21.15.161:0/3428800899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e815ec90 0x7fc8e815f140 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009e50 tx=0x7fc8e001ca00 comp rx=0 tx=0).stop 2024-05-22T21:19:49.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8ecc00640 1 -- 172.21.15.161:0/3428800899 shutdown_connections 2024-05-22T21:19:49.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8ecc00640 1 --2- 172.21.15.161:0/3428800899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e815ec90 0x7fc8e815f140 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8ecc00640 1 --2- 172.21.15.161:0/3428800899 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8e8143ec0 0x7fc8e815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8ecc00640 1 -- 172.21.15.161:0/3428800899 >> 172.21.15.161:0/3428800899 conn(0x7fc8e80b0bc0 msgr2=0x7fc8e80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:49.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8ecc00640 1 -- 172.21.15.161:0/3428800899 shutdown_connections 2024-05-22T21:19:49.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8ecc00640 1 -- 172.21.15.161:0/3428800899 wait complete. 2024-05-22T21:19:49.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8ecc00640 1 Processor -- start 2024-05-22T21:19:49.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8ecc00640 1 -- start start 2024-05-22T21:19:49.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8ecc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8e8143ec0 0x7fc8e8161180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:49.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8ecc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8e815ec90 0x7fc8e8142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:49.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8ecc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e81433e0 0x7fc8e813ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:49.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8ecc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e8160cf0 con 0x7fc8e81433e0 2024-05-22T21:19:49.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8ecc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e813f960 con 0x7fc8e8143ec0 2024-05-22T21:19:49.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8ecc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e81603c0 con 0x7fc8e815ec90 2024-05-22T21:19:49.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8e6800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e81433e0 0x7fc8e813ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:49.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8e6800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e81433e0 0x7fc8e813ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50720/0 (socket says 172.21.15.161:50720) 2024-05-22T21:19:49.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8e6800640 1 -- 172.21.15.161:0/2118861998 learned_addr learned my addr 172.21.15.161:0/2118861998 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:49.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8e5e00640 1 --2- 172.21.15.161:0/2118861998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8e8143ec0 0x7fc8e8161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:49.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8dd400640 1 --2- 172.21.15.161:0/2118861998 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8e815ec90 0x7fc8e8142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:49.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8e6800640 1 -- 172.21.15.161:0/2118861998 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8e815ec90 msgr2=0x7fc8e8142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:49.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8e6800640 1 --2- 172.21.15.161:0/2118861998 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8e815ec90 0x7fc8e8142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8e6800640 1 -- 172.21.15.161:0/2118861998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8e8143ec0 msgr2=0x7fc8e8161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:49.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8e6800640 1 --2- 172.21.15.161:0/2118861998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8e8143ec0 0x7fc8e8161180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:49.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8e6800640 1 -- 172.21.15.161:0/2118861998 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e0009b30 con 0x7fc8e81433e0 2024-05-22T21:19:49.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.935+0000 7fc8e5e00640 1 --2- 172.21.15.161:0/2118861998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8e8143ec0 0x7fc8e8161180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:49.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.939+0000 7fc8e6800640 1 --2- 172.21.15.161:0/2118861998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e81433e0 0x7fc8e813ede0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009e50 tx=0x7fc8e00197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:49.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.939+0000 7fc8dea00640 1 -- 172.21.15.161:0/2118861998 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00b8670 con 0x7fc8e81433e0 2024-05-22T21:19:49.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.939+0000 7fc8dea00640 1 -- 172.21.15.161:0/2118861998 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e00b87d0 con 0x7fc8e81433e0 2024-05-22T21:19:49.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.939+0000 7fc8dea00640 1 -- 172.21.15.161:0/2118861998 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00b8b80 con 0x7fc8e81433e0 2024-05-22T21:19:49.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.939+0000 7fc8ecc00640 1 -- 172.21.15.161:0/2118861998 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e81606a0 con 0x7fc8e81433e0 2024-05-22T21:19:49.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.939+0000 7fc8ecc00640 1 -- 172.21.15.161:0/2118861998 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e813f320 con 0x7fc8e81433e0 2024-05-22T21:19:49.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.939+0000 7fc8ecc00640 1 -- 172.21.15.161:0/2118861998 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e8067220 con 0x7fc8e81433e0 2024-05-22T21:19:49.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.939+0000 7fc8dea00640 1 -- 172.21.15.161:0/2118861998 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc8e00b8ce0 con 0x7fc8e81433e0 2024-05-22T21:19:49.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.939+0000 7fc8dea00640 1 --2- 172.21.15.161:0/2118861998 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc8c4066690 0x7fc8c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:49.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.939+0000 7fc8dea00640 1 -- 172.21.15.161:0/2118861998 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8e011f730 con 0x7fc8e81433e0 2024-05-22T21:19:49.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.939+0000 7fc8e5e00640 1 --2- 172.21.15.161:0/2118861998 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc8c4066690 0x7fc8c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:49.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.943+0000 7fc8e5e00640 1 --2- 172.21.15.161:0/2118861998 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc8c4066690 0x7fc8c4068b50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc8e815ae90 tx=0x7fc8d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:49.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:49.943+0000 7fc8dea00640 1 -- 172.21.15.161:0/2118861998 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc8e00ecbb0 con 0x7fc8e81433e0 2024-05-22T21:19:50.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.099+0000 7fc8ecc00640 1 -- 172.21.15.161:0/2118861998 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8e806dcc0 con 0x7fc8e81433e0 2024-05-22T21:19:50.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.103+0000 7fc8dea00640 1 -- 172.21.15.161:0/2118861998 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16108 (secure 0 0 0) 0x7fc8e00f01d0 con 0x7fc8e81433e0 2024-05-22T21:19:50.104 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:50.104 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":140,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:19:47.387770+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:44.080742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:45.388618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:19:50.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.103+0000 7fc8ecc00640 1 -- 172.21.15.161:0/2118861998 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc8c4066690 msgr2=0x7fc8c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:50.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.103+0000 7fc8ecc00640 1 --2- 172.21.15.161:0/2118861998 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc8c4066690 0x7fc8c4068b50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc8e815ae90 tx=0x7fc8d8008040 comp rx=0 tx=0).stop 2024-05-22T21:19:50.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.103+0000 7fc8ecc00640 1 -- 172.21.15.161:0/2118861998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e81433e0 msgr2=0x7fc8e813ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:50.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.103+0000 7fc8ecc00640 1 --2- 172.21.15.161:0/2118861998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e81433e0 0x7fc8e813ede0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009e50 tx=0x7fc8e00197a0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.103+0000 7fc8ecc00640 1 -- 172.21.15.161:0/2118861998 shutdown_connections 2024-05-22T21:19:50.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.103+0000 7fc8ecc00640 1 --2- 172.21.15.161:0/2118861998 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc8c4066690 0x7fc8c4068b50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.103+0000 7fc8ecc00640 1 --2- 172.21.15.161:0/2118861998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8e81433e0 0x7fc8e813ede0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.103+0000 7fc8ecc00640 1 --2- 172.21.15.161:0/2118861998 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8e815ec90 0x7fc8e8142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.103+0000 7fc8ecc00640 1 --2- 172.21.15.161:0/2118861998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8e8143ec0 0x7fc8e8161180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.103+0000 7fc8ecc00640 1 -- 172.21.15.161:0/2118861998 >> 172.21.15.161:0/2118861998 conn(0x7fc8e80b0bc0 msgr2=0x7fc8e80b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:50.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.103+0000 7fc8ecc00640 1 -- 172.21.15.161:0/2118861998 shutdown_connections 2024-05-22T21:19:50.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.103+0000 7fc8ecc00640 1 -- 172.21.15.161:0/2118861998 wait complete. 2024-05-22T21:19:50.125 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-22T21:19:50.126 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:19:50.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.259+0000 7f124b400640 1 Processor -- start 2024-05-22T21:19:50.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.259+0000 7f124b400640 1 -- start start 2024-05-22T21:19:50.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.259+0000 7f124b400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12440715a0 0x7f124415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:50.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.259+0000 7f124b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f124415ec90 0x7f124415f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:50.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.259+0000 7f124b400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12441442a0 con 0x7f1244143ec0 2024-05-22T21:19:50.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.259+0000 7f124b400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124415f660 con 0x7f124415ec90 2024-05-22T21:19:50.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.259+0000 7f124b400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1244070500 con 0x7f12440715a0 2024-05-22T21:19:50.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.259+0000 7f1249400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f124415ec90 0x7f124415f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:50.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.259+0000 7f1249400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f124415ec90 0x7f124415f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50726/0 (socket says 172.21.15.161:50726) 2024-05-22T21:19:50.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.259+0000 7f1249400640 1 -- 172.21.15.161:0/2028463377 learned_addr learned my addr 172.21.15.161:0/2028463377 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:50.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.263+0000 7f1243e00640 1 --2- 172.21.15.161:0/2028463377 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12440715a0 0x7f124415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:50.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.263+0000 7f1249400640 1 -- 172.21.15.161:0/2028463377 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12440715a0 msgr2=0x7f124415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:50.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.263+0000 7f1249400640 1 --2- 172.21.15.161:0/2028463377 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12440715a0 0x7f124415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.263+0000 7f1249400640 1 -- 172.21.15.161:0/2028463377 >> v1:172.21.15.179:6789/0 conn(0x7f1244143ec0 legacy=0x7f124406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:50.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.263+0000 7f1249400640 1 -- 172.21.15.161:0/2028463377 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12440707e0 con 0x7f124415ec90 2024-05-22T21:19:50.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.263+0000 7f1249400640 1 --2- 172.21.15.161:0/2028463377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f124415ec90 0x7f124415f120 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f123c009e50 tx=0x7f123c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f5a85910298002 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:50.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.263+0000 7f1243400640 1 -- 172.21.15.161:0/2028463377 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c0a57d0 con 0x7f124415ec90 2024-05-22T21:19:50.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.263+0000 7f1243400640 1 -- 172.21.15.161:0/2028463377 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c0a5930 con 0x7f124415ec90 2024-05-22T21:19:50.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.263+0000 7f1243400640 1 -- 172.21.15.161:0/2028463377 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c0a5cc0 con 0x7f124415ec90 2024-05-22T21:19:50.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.263+0000 7f124b400640 1 -- 172.21.15.161:0/2028463377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f124415ec90 msgr2=0x7f124415f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:50.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.263+0000 7f124b400640 1 --2- 172.21.15.161:0/2028463377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f124415ec90 0x7f124415f120 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f123c009e50 tx=0x7f123c01ca00 comp rx=0 tx=0).stop 2024-05-22T21:19:50.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.263+0000 7f124b400640 1 -- 172.21.15.161:0/2028463377 shutdown_connections 2024-05-22T21:19:50.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.263+0000 7f124b400640 1 --2- 172.21.15.161:0/2028463377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f124415ec90 0x7f124415f120 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f124b400640 1 --2- 172.21.15.161:0/2028463377 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12440715a0 0x7f124415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f124b400640 1 -- 172.21.15.161:0/2028463377 >> 172.21.15.161:0/2028463377 conn(0x7f12440b0bc0 msgr2=0x7f12440b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f124b400640 1 -- 172.21.15.161:0/2028463377 shutdown_connections 2024-05-22T21:19:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f124b400640 1 -- 172.21.15.161:0/2028463377 wait complete. 2024-05-22T21:19:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f124b400640 1 Processor -- start 2024-05-22T21:19:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f124b400640 1 -- start start 2024-05-22T21:19:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f124b400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1244143ec0 0x7f12441f9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f124b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f124415ec90 0x7f12441f9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f124b400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1244162240 0x7f12441626f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f124b400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12441602e0 con 0x7f124415ec90 2024-05-22T21:19:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f124b400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124415f9b0 con 0x7f1244143ec0 2024-05-22T21:19:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f124b400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124415fc90 con 0x7f1244162240 2024-05-22T21:19:50.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f1243e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f124415ec90 0x7f12441f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f1243e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f124415ec90 0x7f12441f9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50730/0 (socket says 172.21.15.161:50730) 2024-05-22T21:19:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f1243e00640 1 -- 172.21.15.161:0/2731801468 learned_addr learned my addr 172.21.15.161:0/2731801468 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f1249400640 1 --2- 172.21.15.161:0/2731801468 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1244162240 0x7f12441626f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f1248a00640 1 --2- 172.21.15.161:0/2731801468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1244143ec0 0x7f12441f9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f1243e00640 1 -- 172.21.15.161:0/2731801468 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1244162240 msgr2=0x7f12441626f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f1243e00640 1 --2- 172.21.15.161:0/2731801468 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1244162240 0x7f12441626f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f1243e00640 1 -- 172.21.15.161:0/2731801468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1244143ec0 msgr2=0x7f12441f9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f1243e00640 1 --2- 172.21.15.161:0/2731801468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1244143ec0 0x7f12441f9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f1243e00640 1 -- 172.21.15.161:0/2731801468 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c009b30 con 0x7f124415ec90 2024-05-22T21:19:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f1243e00640 1 --2- 172.21.15.161:0/2731801468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f124415ec90 0x7f12441f9670 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f123000d800 tx=0x7f123000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f1240c00640 1 -- 172.21.15.161:0/2731801468 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123000b6c0 con 0x7f124415ec90 2024-05-22T21:19:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.267+0000 7f1240c00640 1 -- 172.21.15.161:0/2731801468 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123000b860 con 0x7f124415ec90 2024-05-22T21:19:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.271+0000 7f124b400640 1 -- 172.21.15.161:0/2731801468 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124415ff70 con 0x7f124415ec90 2024-05-22T21:19:50.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.271+0000 7f1240c00640 1 -- 172.21.15.161:0/2731801468 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123000bc10 con 0x7f124415ec90 2024-05-22T21:19:50.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.271+0000 7f124b400640 1 -- 172.21.15.161:0/2731801468 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1244160770 con 0x7f124415ec90 2024-05-22T21:19:50.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.271+0000 7f124b400640 1 -- 172.21.15.161:0/2731801468 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1244160490 con 0x7f124415ec90 2024-05-22T21:19:50.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.271+0000 7f1240c00640 1 -- 172.21.15.161:0/2731801468 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f123001e460 con 0x7f124415ec90 2024-05-22T21:19:50.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.271+0000 7f1240c00640 1 --2- 172.21.15.161:0/2731801468 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f12240666c0 0x7f1224068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:50.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.271+0000 7f1240c00640 1 -- 172.21.15.161:0/2731801468 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1230084670 con 0x7f124415ec90 2024-05-22T21:19:50.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.271+0000 7f1248a00640 1 --2- 172.21.15.161:0/2731801468 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f12240666c0 0x7f1224068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:50.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.271+0000 7f1248a00640 1 --2- 172.21.15.161:0/2731801468 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f12240666c0 0x7f1224068b80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1238007fd0 tx=0x7f1238009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:50.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.275+0000 7f1240c00640 1 -- 172.21.15.161:0/2731801468 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1244160490 con 0x7f124415ec90 2024-05-22T21:19:50.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.427+0000 7f124b400640 1 -- 172.21.15.161:0/2731801468 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1244067220 con 0x7f12240666c0 2024-05-22T21:19:50.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.435+0000 7f1240c00640 1 -- 172.21.15.161:0/2731801468 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284698 (secure 0 0 0) 0x7f1244067220 con 0x7f12240666c0 2024-05-22T21:19:50.440 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:50.442 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:19:50.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.439+0000 7f124b400640 1 -- 172.21.15.161:0/2731801468 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f12240666c0 msgr2=0x7f1224068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:50.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.439+0000 7f124b400640 1 --2- 172.21.15.161:0/2731801468 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f12240666c0 0x7f1224068b80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1238007fd0 tx=0x7f1238009040 comp rx=0 tx=0).stop 2024-05-22T21:19:50.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.439+0000 7f124b400640 1 -- 172.21.15.161:0/2731801468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f124415ec90 msgr2=0x7f12441f9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:50.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.439+0000 7f124b400640 1 --2- 172.21.15.161:0/2731801468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f124415ec90 0x7f12441f9670 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f123000d800 tx=0x7f123000dcc0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.439+0000 7f124b400640 1 -- 172.21.15.161:0/2731801468 shutdown_connections 2024-05-22T21:19:50.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.439+0000 7f124b400640 1 --2- 172.21.15.161:0/2731801468 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f12240666c0 0x7f1224068b80 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.439+0000 7f124b400640 1 --2- 172.21.15.161:0/2731801468 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1244162240 0x7f12441626f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.439+0000 7f124b400640 1 --2- 172.21.15.161:0/2731801468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f124415ec90 0x7f12441f9670 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.439+0000 7f124b400640 1 --2- 172.21.15.161:0/2731801468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1244143ec0 0x7f12441f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.439+0000 7f124b400640 1 -- 172.21.15.161:0/2731801468 >> 172.21.15.161:0/2731801468 conn(0x7f12440b0bc0 msgr2=0x7f124406bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:50.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.439+0000 7f124b400640 1 -- 172.21.15.161:0/2731801468 shutdown_connections 2024-05-22T21:19:50.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.439+0000 7f124b400640 1 -- 172.21.15.161:0/2731801468 wait complete. 2024-05-22T21:19:50.459 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-05-22T21:19:48.477723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7097,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555402,"statfs":{"total":1600000000,"available":1592731968,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000845"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.414472+0000","last_change":"2024-05-22T21:19:46.413471+0000","last_active":"2024-05-22T21:19:46.414472+0000","last_peered":"2024-05-22T21:19:46.414472+0000","last_clean":"2024-05-22T21:19:46.414472+0000","last_became_active":"2024-05-22T21:19:46.413210+0000","last_became_peered":"2024-05-22T21:19:46.413210+0000","last_unstale":"2024-05-22T21:19:46.414472+0000","last_undegraded":"2024-05-22T21:19:46.414472+0000","last_fullsized":"2024-05-22T21:19:46.414472+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:26:59.008255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.403761+0000","last_change":"2024-05-22T21:19:46.403546+0000","last_active":"2024-05-22T21:19:46.403761+0000","last_peered":"2024-05-22T21:19:46.403761+0000","last_clean":"2024-05-22T21:19:46.403761+0000","last_became_active":"2024-05-22T21:19:46.402005+0000","last_became_peered":"2024-05-22T21:19:46.402005+0000","last_unstale":"2024-05-22T21:19:46.403761+0000","last_undegraded":"2024-05-22T21:19:46.403761+0000","last_fullsized":"2024-05-22T21:19:46.403761+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:33:07.243615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405477+0000","last_change":"2024-05-22T21:19:46.404603+0000","last_active":"2024-05-22T21:19:46.405477+0000","last_peered":"2024-05-22T21:19:46.405477+0000","last_clean":"2024-05-22T21:19:46.405477+0000","last_became_active":"2024-05-22T21:19:46.403844+0000","last_became_peered":"2024-05-22T21:19:46.403844+0000","last_unstale":"2024-05-22T21:19:46.405477+0000","last_undegraded":"2024-05-22T21:19:46.405477+0000","last_fullsized":"2024-05-22T21:19:46.405477+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:30:01.872224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.411135+0000","last_change":"2024-05-22T21:19:46.411013+0000","last_active":"2024-05-22T21:19:46.411135+0000","last_peered":"2024-05-22T21:19:46.411135+0000","last_clean":"2024-05-22T21:19:46.411135+0000","last_became_active":"2024-05-22T21:19:46.403750+0000","last_became_peered":"2024-05-22T21:19:46.403750+0000","last_unstale":"2024-05-22T21:19:46.411135+0000","last_undegraded":"2024-05-22T21:19:46.411135+0000","last_fullsized":"2024-05-22T21:19:46.411135+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:13:33.884650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.403940+0000","last_change":"2024-05-22T21:19:46.403811+0000","last_active":"2024-05-22T21:19:46.403940+0000","last_peered":"2024-05-22T21:19:46.403940+0000","last_clean":"2024-05-22T21:19:46.403940+0000","last_became_active":"2024-05-22T21:19:46.403566+0000","last_became_peered":"2024-05-22T21:19:46.403566+0000","last_unstale":"2024-05-22T21:19:46.403940+0000","last_undegraded":"2024-05-22T21:19:46.403940+0000","last_fullsized":"2024-05-22T21:19:46.403940+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:39:13.726600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.397048+0000","last_change":"2024-05-22T21:19:46.395576+0000","last_active":"2024-05-22T21:19:46.397048+0000","last_peered":"2024-05-22T21:19:46.397048+0000","last_clean":"2024-05-22T21:19:46.397048+0000","last_became_active":"2024-05-22T21:19:46.395269+0000","last_became_peered":"2024-05-22T21:19:46.395269+0000","last_unstale":"2024-05-22T21:19:46.397048+0000","last_undegraded":"2024-05-22T21:19:46.397048+0000","last_fullsized":"2024-05-22T21:19:46.397048+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:05.182818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.396983+0000","last_change":"2024-05-22T21:19:46.396647+0000","last_active":"2024-05-22T21:19:46.396983+0000","last_peered":"2024-05-22T21:19:46.396983+0000","last_clean":"2024-05-22T21:19:46.396983+0000","last_became_active":"2024-05-22T21:19:46.396394+0000","last_became_peered":"2024-05-22T21:19:46.396394+0000","last_unstale":"2024-05-22T21:19:46.396983+0000","last_undegraded":"2024-05-22T21:19:46.396983+0000","last_fullsized":"2024-05-22T21:19:46.396983+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:42:55.341723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405492+0000","last_change":"2024-05-22T21:19:46.404794+0000","last_active":"2024-05-22T21:19:46.405492+0000","last_peered":"2024-05-22T21:19:46.405492+0000","last_clean":"2024-05-22T21:19:46.405492+0000","last_became_active":"2024-05-22T21:19:46.404191+0000","last_became_peered":"2024-05-22T21:19:46.404191+0000","last_unstale":"2024-05-22T21:19:46.405492+0000","last_undegraded":"2024-05-22T21:19:46.405492+0000","last_fullsized":"2024-05-22T21:19:46.405492+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:21:25.780042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.412104+0000","last_change":"2024-05-22T21:19:46.411730+0000","last_active":"2024-05-22T21:19:46.412104+0000","last_peered":"2024-05-22T21:19:46.412104+0000","last_clean":"2024-05-22T21:19:46.412104+0000","last_became_active":"2024-05-22T21:19:46.411498+0000","last_became_peered":"2024-05-22T21:19:46.411498+0000","last_unstale":"2024-05-22T21:19:46.412104+0000","last_undegraded":"2024-05-22T21:19:46.412104+0000","last_fullsized":"2024-05-22T21:19:46.412104+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:14:36.771922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.408767+0000","last_change":"2024-05-22T21:19:46.405370+0000","last_active":"2024-05-22T21:19:46.408767+0000","last_peered":"2024-05-22T21:19:46.408767+0000","last_clean":"2024-05-22T21:19:46.408767+0000","last_became_active":"2024-05-22T21:19:46.405142+0000","last_became_peered":"2024-05-22T21:19:46.405142+0000","last_unstale":"2024-05-22T21:19:46.408767+0000","last_undegraded":"2024-05-22T21:19:46.408767+0000","last_fullsized":"2024-05-22T21:19:46.408767+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:31:52.992029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405838+0000","last_change":"2024-05-22T21:19:46.405034+0000","last_active":"2024-05-22T21:19:46.405838+0000","last_peered":"2024-05-22T21:19:46.405838+0000","last_clean":"2024-05-22T21:19:46.405838+0000","last_became_active":"2024-05-22T21:19:46.402293+0000","last_became_peered":"2024-05-22T21:19:46.402293+0000","last_unstale":"2024-05-22T21:19:46.405838+0000","last_undegraded":"2024-05-22T21:19:46.405838+0000","last_fullsized":"2024-05-22T21:19:46.405838+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:44:41.425002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405549+0000","last_change":"2024-05-22T21:19:46.400707+0000","last_active":"2024-05-22T21:19:46.405549+0000","last_peered":"2024-05-22T21:19:46.405549+0000","last_clean":"2024-05-22T21:19:46.405549+0000","last_became_active":"2024-05-22T21:19:46.400480+0000","last_became_peered":"2024-05-22T21:19:46.400480+0000","last_unstale":"2024-05-22T21:19:46.405549+0000","last_undegraded":"2024-05-22T21:19:46.405549+0000","last_fullsized":"2024-05-22T21:19:46.405549+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:06:57.565296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.401025+0000","last_change":"2024-05-22T21:19:46.400910+0000","last_active":"2024-05-22T21:19:46.401025+0000","last_peered":"2024-05-22T21:19:46.401025+0000","last_clean":"2024-05-22T21:19:46.401025+0000","last_became_active":"2024-05-22T21:19:46.399810+0000","last_became_peered":"2024-05-22T21:19:46.399810+0000","last_unstale":"2024-05-22T21:19:46.401025+0000","last_undegraded":"2024-05-22T21:19:46.401025+0000","last_fullsized":"2024-05-22T21:19:46.401025+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:01.270410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405428+0000","last_change":"2024-05-22T21:19:46.403555+0000","last_active":"2024-05-22T21:19:46.405428+0000","last_peered":"2024-05-22T21:19:46.405428+0000","last_clean":"2024-05-22T21:19:46.405428+0000","last_became_active":"2024-05-22T21:19:46.402813+0000","last_became_peered":"2024-05-22T21:19:46.402813+0000","last_unstale":"2024-05-22T21:19:46.405428+0000","last_undegraded":"2024-05-22T21:19:46.405428+0000","last_fullsized":"2024-05-22T21:19:46.405428+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:09:13.693246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.401219+0000","last_change":"2024-05-22T21:19:46.401157+0000","last_active":"2024-05-22T21:19:46.401219+0000","last_peered":"2024-05-22T21:19:46.401219+0000","last_clean":"2024-05-22T21:19:46.401219+0000","last_became_active":"2024-05-22T21:19:46.400980+0000","last_became_peered":"2024-05-22T21:19:46.400980+0000","last_unstale":"2024-05-22T21:19:46.401219+0000","last_undegraded":"2024-05-22T21:19:46.401219+0000","last_fullsized":"2024-05-22T21:19:46.401219+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:32:18.996718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.408818+0000","last_change":"2024-05-22T21:19:46.406933+0000","last_active":"2024-05-22T21:19:46.408818+0000","last_peered":"2024-05-22T21:19:46.408818+0000","last_clean":"2024-05-22T21:19:46.408818+0000","last_became_active":"2024-05-22T21:19:46.400710+0000","last_became_peered":"2024-05-22T21:19:46.400710+0000","last_unstale":"2024-05-22T21:19:46.408818+0000","last_undegraded":"2024-05-22T21:19:46.408818+0000","last_fullsized":"2024-05-22T21:19:46.408818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:49:37.475366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.403813+0000","last_change":"2024-05-22T21:19:46.403749+0000","last_active":"2024-05-22T21:19:46.403813+0000","last_peered":"2024-05-22T21:19:46.403813+0000","last_clean":"2024-05-22T21:19:46.403813+0000","last_became_active":"2024-05-22T21:19:46.403297+0000","last_became_peered":"2024-05-22T21:19:46.403297+0000","last_unstale":"2024-05-22T21:19:46.403813+0000","last_undegraded":"2024-05-22T21:19:46.403813+0000","last_fullsized":"2024-05-22T21:19:46.403813+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:54:23.394997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.406003+0000","last_change":"2024-05-22T21:19:46.405501+0000","last_active":"2024-05-22T21:19:46.406003+0000","last_peered":"2024-05-22T21:19:46.406003+0000","last_clean":"2024-05-22T21:19:46.406003+0000","last_became_active":"2024-05-22T21:19:46.405179+0000","last_became_peered":"2024-05-22T21:19:46.405179+0000","last_unstale":"2024-05-22T21:19:46.406003+0000","last_undegraded":"2024-05-22T21:19:46.406003+0000","last_fullsized":"2024-05-22T21:19:46.406003+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:33:07.243615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.399659+0000","last_change":"2024-05-22T21:19:46.399527+0000","last_active":"2024-05-22T21:19:46.399659+0000","last_peered":"2024-05-22T21:19:46.399659+0000","last_clean":"2024-05-22T21:19:46.399659+0000","last_became_active":"2024-05-22T21:19:46.398272+0000","last_became_peered":"2024-05-22T21:19:46.398272+0000","last_unstale":"2024-05-22T21:19:46.399659+0000","last_undegraded":"2024-05-22T21:19:46.399659+0000","last_fullsized":"2024-05-22T21:19:46.399659+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:17.111373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.402954+0000","last_change":"2024-05-22T21:19:46.402668+0000","last_active":"2024-05-22T21:19:46.402954+0000","last_peered":"2024-05-22T21:19:46.402954+0000","last_clean":"2024-05-22T21:19:46.402954+0000","last_became_active":"2024-05-22T21:19:46.402301+0000","last_became_peered":"2024-05-22T21:19:46.402301+0000","last_unstale":"2024-05-22T21:19:46.402954+0000","last_undegraded":"2024-05-22T21:19:46.402954+0000","last_fullsized":"2024-05-22T21:19:46.402954+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:07:33.648876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405929+0000","last_change":"2024-05-22T21:19:46.405740+0000","last_active":"2024-05-22T21:19:46.405929+0000","last_peered":"2024-05-22T21:19:46.405929+0000","last_clean":"2024-05-22T21:19:46.405929+0000","last_became_active":"2024-05-22T21:19:46.405278+0000","last_became_peered":"2024-05-22T21:19:46.405278+0000","last_unstale":"2024-05-22T21:19:46.405929+0000","last_undegraded":"2024-05-22T21:19:46.405929+0000","last_fullsized":"2024-05-22T21:19:46.405929+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:43:42.407532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.397912+0000","last_change":"2024-05-22T21:19:46.397497+0000","last_active":"2024-05-22T21:19:46.397912+0000","last_peered":"2024-05-22T21:19:46.397912+0000","last_clean":"2024-05-22T21:19:46.397912+0000","last_became_active":"2024-05-22T21:19:46.397277+0000","last_became_peered":"2024-05-22T21:19:46.397277+0000","last_unstale":"2024-05-22T21:19:46.397912+0000","last_undegraded":"2024-05-22T21:19:46.397912+0000","last_fullsized":"2024-05-22T21:19:46.397912+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:53:07.064592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405878+0000","last_change":"2024-05-22T21:19:46.405410+0000","last_active":"2024-05-22T21:19:46.405878+0000","last_peered":"2024-05-22T21:19:46.405878+0000","last_clean":"2024-05-22T21:19:46.405878+0000","last_became_active":"2024-05-22T21:19:46.405179+0000","last_became_peered":"2024-05-22T21:19:46.405179+0000","last_unstale":"2024-05-22T21:19:46.405878+0000","last_undegraded":"2024-05-22T21:19:46.405878+0000","last_fullsized":"2024-05-22T21:19:46.405878+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:11:23.114796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.394191+0000","last_change":"2024-05-22T21:19:46.393654+0000","last_active":"2024-05-22T21:19:46.394191+0000","last_peered":"2024-05-22T21:19:46.394191+0000","last_clean":"2024-05-22T21:19:46.394191+0000","last_became_active":"2024-05-22T21:19:46.393402+0000","last_became_peered":"2024-05-22T21:19:46.393402+0000","last_unstale":"2024-05-22T21:19:46.394191+0000","last_undegraded":"2024-05-22T21:19:46.394191+0000","last_fullsized":"2024-05-22T21:19:46.394191+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:01.270410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.399591+0000","last_change":"2024-05-22T21:19:46.398753+0000","last_active":"2024-05-22T21:19:46.399591+0000","last_peered":"2024-05-22T21:19:46.399591+0000","last_clean":"2024-05-22T21:19:46.399591+0000","last_became_active":"2024-05-22T21:19:46.398520+0000","last_became_peered":"2024-05-22T21:19:46.398520+0000","last_unstale":"2024-05-22T21:19:46.399591+0000","last_undegraded":"2024-05-22T21:19:46.399591+0000","last_fullsized":"2024-05-22T21:19:46.399591+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:31:33.029588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.408802+0000","last_change":"2024-05-22T21:19:46.406801+0000","last_active":"2024-05-22T21:19:46.408802+0000","last_peered":"2024-05-22T21:19:46.408802+0000","last_clean":"2024-05-22T21:19:46.408802+0000","last_became_active":"2024-05-22T21:19:46.400207+0000","last_became_peered":"2024-05-22T21:19:46.400207+0000","last_unstale":"2024-05-22T21:19:46.408802+0000","last_undegraded":"2024-05-22T21:19:46.408802+0000","last_fullsized":"2024-05-22T21:19:46.408802+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:19:09.914649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.404510+0000","last_change":"2024-05-22T21:19:46.404098+0000","last_active":"2024-05-22T21:19:46.404510+0000","last_peered":"2024-05-22T21:19:46.404510+0000","last_clean":"2024-05-22T21:19:46.404510+0000","last_became_active":"2024-05-22T21:19:46.403895+0000","last_became_peered":"2024-05-22T21:19:46.403895+0000","last_unstale":"2024-05-22T21:19:46.404510+0000","last_undegraded":"2024-05-22T21:19:46.404510+0000","last_fullsized":"2024-05-22T21:19:46.404510+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:33:03.933467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.409367+0000","last_change":"2024-05-22T21:19:46.407866+0000","last_active":"2024-05-22T21:19:46.409367+0000","last_peered":"2024-05-22T21:19:46.409367+0000","last_clean":"2024-05-22T21:19:46.409367+0000","last_became_active":"2024-05-22T21:19:46.407606+0000","last_became_peered":"2024-05-22T21:19:46.407606+0000","last_unstale":"2024-05-22T21:19:46.409367+0000","last_undegraded":"2024-05-22T21:19:46.409367+0000","last_fullsized":"2024-05-22T21:19:46.409367+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:02:04.279027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.403105+0000","last_change":"2024-05-22T21:19:46.403029+0000","last_active":"2024-05-22T21:19:46.403105+0000","last_peered":"2024-05-22T21:19:46.403105+0000","last_clean":"2024-05-22T21:19:46.403105+0000","last_became_active":"2024-05-22T21:19:46.402789+0000","last_became_peered":"2024-05-22T21:19:46.402789+0000","last_unstale":"2024-05-22T21:19:46.403105+0000","last_undegraded":"2024-05-22T21:19:46.403105+0000","last_fullsized":"2024-05-22T21:19:46.403105+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:01:55.246784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.401966+0000","last_change":"2024-05-22T21:19:46.401860+0000","last_active":"2024-05-22T21:19:46.401966+0000","last_peered":"2024-05-22T21:19:46.401966+0000","last_clean":"2024-05-22T21:19:46.401966+0000","last_became_active":"2024-05-22T21:19:46.401612+0000","last_became_peered":"2024-05-22T21:19:46.401612+0000","last_unstale":"2024-05-22T21:19:46.401966+0000","last_undegraded":"2024-05-22T21:19:46.401966+0000","last_fullsized":"2024-05-22T21:19:46.401966+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:55:58.881775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405527+0000","last_change":"2024-05-22T21:19:46.400487+0000","last_active":"2024-05-22T21:19:46.405527+0000","last_peered":"2024-05-22T21:19:46.405527+0000","last_clean":"2024-05-22T21:19:46.405527+0000","last_became_active":"2024-05-22T21:19:46.399902+0000","last_became_peered":"2024-05-22T21:19:46.399902+0000","last_unstale":"2024-05-22T21:19:46.405527+0000","last_undegraded":"2024-05-22T21:19:46.405527+0000","last_fullsized":"2024-05-22T21:19:46.405527+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:45:25.590631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.399482+0000","last_change":"2024-05-22T21:19:46.399401+0000","last_active":"2024-05-22T21:19:46.399482+0000","last_peered":"2024-05-22T21:19:46.399482+0000","last_clean":"2024-05-22T21:19:46.399482+0000","last_became_active":"2024-05-22T21:19:46.398518+0000","last_became_peered":"2024-05-22T21:19:46.398518+0000","last_unstale":"2024-05-22T21:19:46.399482+0000","last_undegraded":"2024-05-22T21:19:46.399482+0000","last_fullsized":"2024-05-22T21:19:46.399482+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:56:53.774354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.402859+0000","last_change":"2024-05-22T21:19:46.401867+0000","last_active":"2024-05-22T21:19:46.402859+0000","last_peered":"2024-05-22T21:19:46.402859+0000","last_clean":"2024-05-22T21:19:46.402859+0000","last_became_active":"2024-05-22T21:19:46.401655+0000","last_became_peered":"2024-05-22T21:19:46.401655+0000","last_unstale":"2024-05-22T21:19:46.402859+0000","last_undegraded":"2024-05-22T21:19:46.402859+0000","last_fullsized":"2024-05-22T21:19:46.402859+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:18:46.668052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"139'4","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.410934+0000","last_change":"2024-05-22T21:19:46.015891+0000","last_active":"2024-05-22T21:19:46.410934+0000","last_peered":"2024-05-22T21:19:46.410934+0000","last_clean":"2024-05-22T21:19:46.410934+0000","last_became_active":"2024-05-22T21:19:46.013962+0000","last_became_peered":"2024-05-22T21:19:46.013962+0000","last_unstale":"2024-05-22T21:19:46.410934+0000","last_undegraded":"2024-05-22T21:19:46.410934+0000","last_fullsized":"2024-05-22T21:19:46.410934+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:04.172719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.403142+0000","last_change":"2024-05-22T21:19:46.402685+0000","last_active":"2024-05-22T21:19:46.403142+0000","last_peered":"2024-05-22T21:19:46.403142+0000","last_clean":"2024-05-22T21:19:46.403142+0000","last_became_active":"2024-05-22T21:19:46.402436+0000","last_became_peered":"2024-05-22T21:19:46.402436+0000","last_unstale":"2024-05-22T21:19:46.403142+0000","last_undegraded":"2024-05-22T21:19:46.403142+0000","last_fullsized":"2024-05-22T21:19:46.403142+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:44:41.425002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.408116+0000","last_change":"2024-05-22T21:19:46.016005+0000","last_active":"2024-05-22T21:19:46.408116+0000","last_peered":"2024-05-22T21:19:46.408116+0000","last_clean":"2024-05-22T21:19:46.408116+0000","last_became_active":"2024-05-22T21:19:46.015786+0000","last_became_peered":"2024-05-22T21:19:46.015786+0000","last_unstale":"2024-05-22T21:19:46.408116+0000","last_undegraded":"2024-05-22T21:19:46.408116+0000","last_fullsized":"2024-05-22T21:19:46.408116+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:50:21.018391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.401323+0000","last_change":"2024-05-22T21:19:46.401237+0000","last_active":"2024-05-22T21:19:46.401323+0000","last_peered":"2024-05-22T21:19:46.401323+0000","last_clean":"2024-05-22T21:19:46.401323+0000","last_became_active":"2024-05-22T21:19:46.401015+0000","last_became_peered":"2024-05-22T21:19:46.401015+0000","last_unstale":"2024-05-22T21:19:46.401323+0000","last_undegraded":"2024-05-22T21:19:46.401323+0000","last_fullsized":"2024-05-22T21:19:46.401323+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:10:10.164548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.416812+0000","last_change":"2024-05-22T21:19:46.019292+0000","last_active":"2024-05-22T21:19:46.416812+0000","last_peered":"2024-05-22T21:19:46.416812+0000","last_clean":"2024-05-22T21:19:46.416812+0000","last_became_active":"2024-05-22T21:19:46.019044+0000","last_became_peered":"2024-05-22T21:19:46.019044+0000","last_unstale":"2024-05-22T21:19:46.416812+0000","last_undegraded":"2024-05-22T21:19:46.416812+0000","last_fullsized":"2024-05-22T21:19:46.416812+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:05:11.899061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405506+0000","last_change":"2024-05-22T21:19:46.405146+0000","last_active":"2024-05-22T21:19:46.405506+0000","last_peered":"2024-05-22T21:19:46.405506+0000","last_clean":"2024-05-22T21:19:46.405506+0000","last_became_active":"2024-05-22T21:19:46.404788+0000","last_became_peered":"2024-05-22T21:19:46.404788+0000","last_unstale":"2024-05-22T21:19:46.405506+0000","last_undegraded":"2024-05-22T21:19:46.405506+0000","last_fullsized":"2024-05-22T21:19:46.405506+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:03:28.907582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.416577+0000","last_change":"2024-05-22T21:19:46.013623+0000","last_active":"2024-05-22T21:19:46.416577+0000","last_peered":"2024-05-22T21:19:46.416577+0000","last_clean":"2024-05-22T21:19:46.416577+0000","last_became_active":"2024-05-22T21:19:46.013151+0000","last_became_peered":"2024-05-22T21:19:46.013151+0000","last_unstale":"2024-05-22T21:19:46.416577+0000","last_undegraded":"2024-05-22T21:19:46.416577+0000","last_fullsized":"2024-05-22T21:19:46.416577+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:50:09.899680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.408950+0000","last_change":"2024-05-22T21:19:46.408699+0000","last_active":"2024-05-22T21:19:46.408950+0000","last_peered":"2024-05-22T21:19:46.408950+0000","last_clean":"2024-05-22T21:19:46.408950+0000","last_became_active":"2024-05-22T21:19:46.408351+0000","last_became_peered":"2024-05-22T21:19:46.408351+0000","last_unstale":"2024-05-22T21:19:46.408950+0000","last_undegraded":"2024-05-22T21:19:46.408950+0000","last_fullsized":"2024-05-22T21:19:46.408950+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:10:06.354001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.414520+0000","last_change":"2024-05-22T21:19:46.021062+0000","last_active":"2024-05-22T21:19:46.414520+0000","last_peered":"2024-05-22T21:19:46.414520+0000","last_clean":"2024-05-22T21:19:46.414520+0000","last_became_active":"2024-05-22T21:19:46.020815+0000","last_became_peered":"2024-05-22T21:19:46.020815+0000","last_unstale":"2024-05-22T21:19:46.414520+0000","last_undegraded":"2024-05-22T21:19:46.414520+0000","last_fullsized":"2024-05-22T21:19:46.414520+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:56:52.764255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.402560+0000","last_change":"2024-05-22T21:19:46.402418+0000","last_active":"2024-05-22T21:19:46.402560+0000","last_peered":"2024-05-22T21:19:46.402560+0000","last_clean":"2024-05-22T21:19:46.402560+0000","last_became_active":"2024-05-22T21:19:46.401256+0000","last_became_peered":"2024-05-22T21:19:46.401256+0000","last_unstale":"2024-05-22T21:19:46.402560+0000","last_undegraded":"2024-05-22T21:19:46.402560+0000","last_fullsized":"2024-05-22T21:19:46.402560+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:09:42.299345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.412230+0000","last_change":"2024-05-22T21:19:46.018396+0000","last_active":"2024-05-22T21:19:46.412230+0000","last_peered":"2024-05-22T21:19:46.412230+0000","last_clean":"2024-05-22T21:19:46.412230+0000","last_became_active":"2024-05-22T21:19:46.018137+0000","last_became_peered":"2024-05-22T21:19:46.018137+0000","last_unstale":"2024-05-22T21:19:46.412230+0000","last_undegraded":"2024-05-22T21:19:46.412230+0000","last_fullsized":"2024-05-22T21:19:46.412230+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:50:21.018391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.410546+0000","last_change":"2024-05-22T21:19:46.410110+0000","last_active":"2024-05-22T21:19:46.410546+0000","last_peered":"2024-05-22T21:19:46.410546+0000","last_clean":"2024-05-22T21:19:46.410546+0000","last_became_active":"2024-05-22T21:19:46.409710+0000","last_became_peered":"2024-05-22T21:19:46.409710+0000","last_unstale":"2024-05-22T21:19:46.410546+0000","last_undegraded":"2024-05-22T21:19:46.410546+0000","last_fullsized":"2024-05-22T21:19:46.410546+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:29:28.307781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.395978+0000","last_change":"2024-05-22T21:19:46.011421+0000","last_active":"2024-05-22T21:19:46.395978+0000","last_peered":"2024-05-22T21:19:46.395978+0000","last_clean":"2024-05-22T21:19:46.395978+0000","last_became_active":"2024-05-22T21:19:46.011177+0000","last_became_peered":"2024-05-22T21:19:46.011177+0000","last_unstale":"2024-05-22T21:19:46.395978+0000","last_undegraded":"2024-05-22T21:19:46.395978+0000","last_fullsized":"2024-05-22T21:19:46.395978+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:13:36.319481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.400212+0000","last_change":"2024-05-22T21:19:46.400138+0000","last_active":"2024-05-22T21:19:46.400212+0000","last_peered":"2024-05-22T21:19:46.400212+0000","last_clean":"2024-05-22T21:19:46.400212+0000","last_became_active":"2024-05-22T21:19:46.399919+0000","last_became_peered":"2024-05-22T21:19:46.399919+0000","last_unstale":"2024-05-22T21:19:46.400212+0000","last_undegraded":"2024-05-22T21:19:46.400212+0000","last_fullsized":"2024-05-22T21:19:46.400212+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:13:42.580122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.398553+0000","last_change":"2024-05-22T21:19:46.010930+0000","last_active":"2024-05-22T21:19:46.398553+0000","last_peered":"2024-05-22T21:19:46.398553+0000","last_clean":"2024-05-22T21:19:46.398553+0000","last_became_active":"2024-05-22T21:19:46.010703+0000","last_became_peered":"2024-05-22T21:19:46.010703+0000","last_unstale":"2024-05-22T21:19:46.398553+0000","last_undegraded":"2024-05-22T21:19:46.398553+0000","last_fullsized":"2024-05-22T21:19:46.398553+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:57:58.350121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.398967+0000","last_change":"2024-05-22T21:19:46.398748+0000","last_active":"2024-05-22T21:19:46.398967+0000","last_peered":"2024-05-22T21:19:46.398967+0000","last_clean":"2024-05-22T21:19:46.398967+0000","last_became_active":"2024-05-22T21:19:46.398101+0000","last_became_peered":"2024-05-22T21:19:46.398101+0000","last_unstale":"2024-05-22T21:19:46.398967+0000","last_undegraded":"2024-05-22T21:19:46.398967+0000","last_fullsized":"2024-05-22T21:19:46.398967+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:13:37.329580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.393824+0000","last_change":"2024-05-22T21:19:46.014681+0000","last_active":"2024-05-22T21:19:46.393824+0000","last_peered":"2024-05-22T21:19:46.393824+0000","last_clean":"2024-05-22T21:19:46.393824+0000","last_became_active":"2024-05-22T21:19:46.013416+0000","last_became_peered":"2024-05-22T21:19:46.013416+0000","last_unstale":"2024-05-22T21:19:46.393824+0000","last_undegraded":"2024-05-22T21:19:46.393824+0000","last_fullsized":"2024-05-22T21:19:46.393824+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:10:07.010122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.415278+0000","last_change":"2024-05-22T21:19:46.415207+0000","last_active":"2024-05-22T21:19:46.415278+0000","last_peered":"2024-05-22T21:19:46.415278+0000","last_clean":"2024-05-22T21:19:46.415278+0000","last_became_active":"2024-05-22T21:19:46.414932+0000","last_became_peered":"2024-05-22T21:19:46.414932+0000","last_unstale":"2024-05-22T21:19:46.415278+0000","last_undegraded":"2024-05-22T21:19:46.415278+0000","last_fullsized":"2024-05-22T21:19:46.415278+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:09:42.299345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.389585+0000","last_change":"2024-05-22T21:19:46.012649+0000","last_active":"2024-05-22T21:19:46.389585+0000","last_peered":"2024-05-22T21:19:46.389585+0000","last_clean":"2024-05-22T21:19:46.389585+0000","last_became_active":"2024-05-22T21:19:46.012432+0000","last_became_peered":"2024-05-22T21:19:46.012432+0000","last_unstale":"2024-05-22T21:19:46.389585+0000","last_undegraded":"2024-05-22T21:19:46.389585+0000","last_fullsized":"2024-05-22T21:19:46.389585+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:12:01.288530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.401674+0000","last_change":"2024-05-22T21:19:46.400814+0000","last_active":"2024-05-22T21:19:46.401674+0000","last_peered":"2024-05-22T21:19:46.401674+0000","last_clean":"2024-05-22T21:19:46.401674+0000","last_became_active":"2024-05-22T21:19:46.400486+0000","last_became_peered":"2024-05-22T21:19:46.400486+0000","last_unstale":"2024-05-22T21:19:46.401674+0000","last_undegraded":"2024-05-22T21:19:46.401674+0000","last_fullsized":"2024-05-22T21:19:46.401674+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:17.111373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"139'3","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.416244+0000","last_change":"2024-05-22T21:19:46.009675+0000","last_active":"2024-05-22T21:19:46.416244+0000","last_peered":"2024-05-22T21:19:46.416244+0000","last_clean":"2024-05-22T21:19:46.416244+0000","last_became_active":"2024-05-22T21:19:46.009358+0000","last_became_peered":"2024-05-22T21:19:46.009358+0000","last_unstale":"2024-05-22T21:19:46.416244+0000","last_undegraded":"2024-05-22T21:19:46.416244+0000","last_fullsized":"2024-05-22T21:19:46.416244+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:44:40.414903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.397797+0000","last_change":"2024-05-22T21:19:46.397715+0000","last_active":"2024-05-22T21:19:46.397797+0000","last_peered":"2024-05-22T21:19:46.397797+0000","last_clean":"2024-05-22T21:19:46.397797+0000","last_became_active":"2024-05-22T21:19:46.396926+0000","last_became_peered":"2024-05-22T21:19:46.396926+0000","last_unstale":"2024-05-22T21:19:46.397797+0000","last_undegraded":"2024-05-22T21:19:46.397797+0000","last_fullsized":"2024-05-22T21:19:46.397797+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:32:36.123360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"139'6","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.417416+0000","last_change":"2024-05-22T21:19:46.011208+0000","last_active":"2024-05-22T21:19:46.417416+0000","last_peered":"2024-05-22T21:19:46.417416+0000","last_clean":"2024-05-22T21:19:46.417416+0000","last_became_active":"2024-05-22T21:19:46.010687+0000","last_became_peered":"2024-05-22T21:19:46.010687+0000","last_unstale":"2024-05-22T21:19:46.417416+0000","last_undegraded":"2024-05-22T21:19:46.417416+0000","last_fullsized":"2024-05-22T21:19:46.417416+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:53:06.054493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.394950+0000","last_change":"2024-05-22T21:19:46.394831+0000","last_active":"2024-05-22T21:19:46.394950+0000","last_peered":"2024-05-22T21:19:46.394950+0000","last_clean":"2024-05-22T21:19:46.394950+0000","last_became_active":"2024-05-22T21:19:46.393914+0000","last_became_peered":"2024-05-22T21:19:46.393914+0000","last_unstale":"2024-05-22T21:19:46.394950+0000","last_undegraded":"2024-05-22T21:19:46.394950+0000","last_fullsized":"2024-05-22T21:19:46.394950+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:03:28.907582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.410476+0000","last_change":"2024-05-22T21:19:46.005658+0000","last_active":"2024-05-22T21:19:46.410476+0000","last_peered":"2024-05-22T21:19:46.410476+0000","last_clean":"2024-05-22T21:19:46.410476+0000","last_became_active":"2024-05-22T21:19:46.005401+0000","last_became_peered":"2024-05-22T21:19:46.005401+0000","last_unstale":"2024-05-22T21:19:46.410476+0000","last_undegraded":"2024-05-22T21:19:46.410476+0000","last_fullsized":"2024-05-22T21:19:46.410476+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:39:12.716501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.399670+0000","last_change":"2024-05-22T21:19:46.399457+0000","last_active":"2024-05-22T21:19:46.399670+0000","last_peered":"2024-05-22T21:19:46.399670+0000","last_clean":"2024-05-22T21:19:46.399670+0000","last_became_active":"2024-05-22T21:19:46.398507+0000","last_became_peered":"2024-05-22T21:19:46.398507+0000","last_unstale":"2024-05-22T21:19:46.399670+0000","last_undegraded":"2024-05-22T21:19:46.399670+0000","last_fullsized":"2024-05-22T21:19:46.399670+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:21:25.780042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.396253+0000","last_change":"2024-05-22T21:19:46.015382+0000","last_active":"2024-05-22T21:19:46.396253+0000","last_peered":"2024-05-22T21:19:46.396253+0000","last_clean":"2024-05-22T21:19:46.396253+0000","last_became_active":"2024-05-22T21:19:46.014728+0000","last_became_peered":"2024-05-22T21:19:46.014728+0000","last_unstale":"2024-05-22T21:19:46.396253+0000","last_undegraded":"2024-05-22T21:19:46.396253+0000","last_fullsized":"2024-05-22T21:19:46.396253+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:50:21.018391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.403571+0000","last_change":"2024-05-22T21:19:46.403402+0000","last_active":"2024-05-22T21:19:46.403571+0000","last_peered":"2024-05-22T21:19:46.403571+0000","last_clean":"2024-05-22T21:19:46.403571+0000","last_became_active":"2024-05-22T21:19:46.403191+0000","last_became_peered":"2024-05-22T21:19:46.403191+0000","last_unstale":"2024-05-22T21:19:46.403571+0000","last_undegraded":"2024-05-22T21:19:46.403571+0000","last_fullsized":"2024-05-22T21:19:46.403571+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:49:14.153916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"139'8","reported_seq":18,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.417366+0000","last_change":"2024-05-22T21:19:46.011669+0000","last_active":"2024-05-22T21:19:46.417366+0000","last_peered":"2024-05-22T21:19:46.417366+0000","last_clean":"2024-05-22T21:19:46.417366+0000","last_became_active":"2024-05-22T21:19:46.011459+0000","last_became_peered":"2024-05-22T21:19:46.011459+0000","last_unstale":"2024-05-22T21:19:46.417366+0000","last_undegraded":"2024-05-22T21:19:46.417366+0000","last_fullsized":"2024-05-22T21:19:46.417366+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:56:22.229755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.414864+0000","last_change":"2024-05-22T21:19:46.413572+0000","last_active":"2024-05-22T21:19:46.414864+0000","last_peered":"2024-05-22T21:19:46.414864+0000","last_clean":"2024-05-22T21:19:46.414864+0000","last_became_active":"2024-05-22T21:19:46.413349+0000","last_became_peered":"2024-05-22T21:19:46.413349+0000","last_unstale":"2024-05-22T21:19:46.414864+0000","last_undegraded":"2024-05-22T21:19:46.414864+0000","last_fullsized":"2024-05-22T21:19:46.414864+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:28:07.156675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.415086+0000","last_change":"2024-05-22T21:19:46.015585+0000","last_active":"2024-05-22T21:19:46.415086+0000","last_peered":"2024-05-22T21:19:46.415086+0000","last_clean":"2024-05-22T21:19:46.415086+0000","last_became_active":"2024-05-22T21:19:46.015375+0000","last_became_peered":"2024-05-22T21:19:46.015375+0000","last_unstale":"2024-05-22T21:19:46.415086+0000","last_undegraded":"2024-05-22T21:19:46.415086+0000","last_fullsized":"2024-05-22T21:19:46.415086+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:00.260310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405198+0000","last_change":"2024-05-22T21:19:46.404927+0000","last_active":"2024-05-22T21:19:46.405198+0000","last_peered":"2024-05-22T21:19:46.405198+0000","last_clean":"2024-05-22T21:19:46.405198+0000","last_became_active":"2024-05-22T21:19:46.404688+0000","last_became_peered":"2024-05-22T21:19:46.404688+0000","last_unstale":"2024-05-22T21:19:46.405198+0000","last_undegraded":"2024-05-22T21:19:46.405198+0000","last_fullsized":"2024-05-22T21:19:46.405198+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:55:46.617557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.416699+0000","last_change":"2024-05-22T21:19:46.013870+0000","last_active":"2024-05-22T21:19:46.416699+0000","last_peered":"2024-05-22T21:19:46.416699+0000","last_clean":"2024-05-22T21:19:46.416699+0000","last_became_active":"2024-05-22T21:19:46.012435+0000","last_became_peered":"2024-05-22T21:19:46.012435+0000","last_unstale":"2024-05-22T21:19:46.416699+0000","last_undegraded":"2024-05-22T21:19:46.416699+0000","last_fullsized":"2024-05-22T21:19:46.416699+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:19:50.238691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.403359+0000","last_change":"2024-05-22T21:19:46.403283+0000","last_active":"2024-05-22T21:19:46.403359+0000","last_peered":"2024-05-22T21:19:46.403359+0000","last_clean":"2024-05-22T21:19:46.403359+0000","last_became_active":"2024-05-22T21:19:46.403035+0000","last_became_peered":"2024-05-22T21:19:46.403035+0000","last_unstale":"2024-05-22T21:19:46.403359+0000","last_undegraded":"2024-05-22T21:19:46.403359+0000","last_fullsized":"2024-05-22T21:19:46.403359+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:57:59.360221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.410630+0000","last_change":"2024-05-22T21:19:46.016281+0000","last_active":"2024-05-22T21:19:46.410630+0000","last_peered":"2024-05-22T21:19:46.410630+0000","last_clean":"2024-05-22T21:19:46.410630+0000","last_became_active":"2024-05-22T21:19:46.016065+0000","last_became_peered":"2024-05-22T21:19:46.016065+0000","last_unstale":"2024-05-22T21:19:46.410630+0000","last_undegraded":"2024-05-22T21:19:46.410630+0000","last_fullsized":"2024-05-22T21:19:46.410630+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:12:01.288530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.395937+0000","last_change":"2024-05-22T21:19:46.394871+0000","last_active":"2024-05-22T21:19:46.395937+0000","last_peered":"2024-05-22T21:19:46.395937+0000","last_clean":"2024-05-22T21:19:46.395937+0000","last_became_active":"2024-05-22T21:19:46.394643+0000","last_became_peered":"2024-05-22T21:19:46.394643+0000","last_unstale":"2024-05-22T21:19:46.395937+0000","last_undegraded":"2024-05-22T21:19:46.395937+0000","last_fullsized":"2024-05-22T21:19:46.395937+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:06:39.567925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.392872+0000","last_change":"2024-05-22T21:19:46.017189+0000","last_active":"2024-05-22T21:19:46.392872+0000","last_peered":"2024-05-22T21:19:46.392872+0000","last_clean":"2024-05-22T21:19:46.392872+0000","last_became_active":"2024-05-22T21:19:46.016727+0000","last_became_peered":"2024-05-22T21:19:46.016727+0000","last_unstale":"2024-05-22T21:19:46.392872+0000","last_undegraded":"2024-05-22T21:19:46.392872+0000","last_fullsized":"2024-05-22T21:19:46.392872+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:10:09.154449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405447+0000","last_change":"2024-05-22T21:19:46.403704+0000","last_active":"2024-05-22T21:19:46.405447+0000","last_peered":"2024-05-22T21:19:46.405447+0000","last_clean":"2024-05-22T21:19:46.405447+0000","last_became_active":"2024-05-22T21:19:46.403331+0000","last_became_peered":"2024-05-22T21:19:46.403331+0000","last_unstale":"2024-05-22T21:19:46.405447+0000","last_undegraded":"2024-05-22T21:19:46.405447+0000","last_fullsized":"2024-05-22T21:19:46.405447+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:49:14.153916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.415580+0000","last_change":"2024-05-22T21:19:46.013618+0000","last_active":"2024-05-22T21:19:46.415580+0000","last_peered":"2024-05-22T21:19:46.415580+0000","last_clean":"2024-05-22T21:19:46.415580+0000","last_became_active":"2024-05-22T21:19:46.013397+0000","last_became_peered":"2024-05-22T21:19:46.013397+0000","last_unstale":"2024-05-22T21:19:46.415580+0000","last_undegraded":"2024-05-22T21:19:46.415580+0000","last_fullsized":"2024-05-22T21:19:46.415580+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:09:21.911521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.400051+0000","last_change":"2024-05-22T21:19:46.399878+0000","last_active":"2024-05-22T21:19:46.400051+0000","last_peered":"2024-05-22T21:19:46.400051+0000","last_clean":"2024-05-22T21:19:46.400051+0000","last_became_active":"2024-05-22T21:19:46.399633+0000","last_became_peered":"2024-05-22T21:19:46.399633+0000","last_unstale":"2024-05-22T21:19:46.400051+0000","last_undegraded":"2024-05-22T21:19:46.400051+0000","last_fullsized":"2024-05-22T21:19:46.400051+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:00.041735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.392188+0000","last_change":"2024-05-22T21:19:46.017660+0000","last_active":"2024-05-22T21:19:46.392188+0000","last_peered":"2024-05-22T21:19:46.392188+0000","last_clean":"2024-05-22T21:19:46.392188+0000","last_became_active":"2024-05-22T21:19:46.017375+0000","last_became_peered":"2024-05-22T21:19:46.017375+0000","last_unstale":"2024-05-22T21:19:46.392188+0000","last_undegraded":"2024-05-22T21:19:46.392188+0000","last_fullsized":"2024-05-22T21:19:46.392188+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:29:53.472753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405463+0000","last_change":"2024-05-22T21:19:46.404485+0000","last_active":"2024-05-22T21:19:46.405463+0000","last_peered":"2024-05-22T21:19:46.405463+0000","last_clean":"2024-05-22T21:19:46.405463+0000","last_became_active":"2024-05-22T21:19:46.404230+0000","last_became_peered":"2024-05-22T21:19:46.404230+0000","last_unstale":"2024-05-22T21:19:46.405463+0000","last_undegraded":"2024-05-22T21:19:46.405463+0000","last_fullsized":"2024-05-22T21:19:46.405463+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:26:38.251056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.395178+0000","last_change":"2024-05-22T21:19:46.016840+0000","last_active":"2024-05-22T21:19:46.395178+0000","last_peered":"2024-05-22T21:19:46.395178+0000","last_clean":"2024-05-22T21:19:46.395178+0000","last_became_active":"2024-05-22T21:19:46.016441+0000","last_became_peered":"2024-05-22T21:19:46.016441+0000","last_unstale":"2024-05-22T21:19:46.395178+0000","last_undegraded":"2024-05-22T21:19:46.395178+0000","last_fullsized":"2024-05-22T21:19:46.395178+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:28:06.146576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.409191+0000","last_change":"2024-05-22T21:19:46.407277+0000","last_active":"2024-05-22T21:19:46.409191+0000","last_peered":"2024-05-22T21:19:46.409191+0000","last_clean":"2024-05-22T21:19:46.409191+0000","last_became_active":"2024-05-22T21:19:46.407066+0000","last_became_peered":"2024-05-22T21:19:46.407066+0000","last_unstale":"2024-05-22T21:19:46.409191+0000","last_undegraded":"2024-05-22T21:19:46.409191+0000","last_fullsized":"2024-05-22T21:19:46.409191+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:09:13.693246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.395780+0000","last_change":"2024-05-22T21:19:46.014817+0000","last_active":"2024-05-22T21:19:46.395780+0000","last_peered":"2024-05-22T21:19:46.395780+0000","last_clean":"2024-05-22T21:19:46.395780+0000","last_became_active":"2024-05-22T21:19:46.014492+0000","last_became_peered":"2024-05-22T21:19:46.014492+0000","last_unstale":"2024-05-22T21:19:46.395780+0000","last_undegraded":"2024-05-22T21:19:46.395780+0000","last_fullsized":"2024-05-22T21:19:46.395780+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:33:06.233516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.412241+0000","last_change":"2024-05-22T21:19:46.412178+0000","last_active":"2024-05-22T21:19:46.412241+0000","last_peered":"2024-05-22T21:19:46.412241+0000","last_clean":"2024-05-22T21:19:46.412241+0000","last_became_active":"2024-05-22T21:19:46.411925+0000","last_became_peered":"2024-05-22T21:19:46.411925+0000","last_unstale":"2024-05-22T21:19:46.412241+0000","last_undegraded":"2024-05-22T21:19:46.412241+0000","last_fullsized":"2024-05-22T21:19:46.412241+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:22:15.297795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.415261+0000","last_change":"2024-05-22T21:19:46.013294+0000","last_active":"2024-05-22T21:19:46.415261+0000","last_peered":"2024-05-22T21:19:46.415261+0000","last_clean":"2024-05-22T21:19:46.415261+0000","last_became_active":"2024-05-22T21:19:46.013068+0000","last_became_peered":"2024-05-22T21:19:46.013068+0000","last_unstale":"2024-05-22T21:19:46.415261+0000","last_undegraded":"2024-05-22T21:19:46.415261+0000","last_fullsized":"2024-05-22T21:19:46.415261+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:15:09.051559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.408836+0000","last_change":"2024-05-22T21:19:46.406947+0000","last_active":"2024-05-22T21:19:46.408836+0000","last_peered":"2024-05-22T21:19:46.408836+0000","last_clean":"2024-05-22T21:19:46.408836+0000","last_became_active":"2024-05-22T21:19:46.406710+0000","last_became_peered":"2024-05-22T21:19:46.406710+0000","last_unstale":"2024-05-22T21:19:46.408836+0000","last_undegraded":"2024-05-22T21:19:46.408836+0000","last_fullsized":"2024-05-22T21:19:46.408836+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:37:02.707159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.395962+0000","last_change":"2024-05-22T21:19:46.015640+0000","last_active":"2024-05-22T21:19:46.395962+0000","last_peered":"2024-05-22T21:19:46.395962+0000","last_clean":"2024-05-22T21:19:46.395962+0000","last_became_active":"2024-05-22T21:19:46.015404+0000","last_became_peered":"2024-05-22T21:19:46.015404+0000","last_unstale":"2024-05-22T21:19:46.395962+0000","last_undegraded":"2024-05-22T21:19:46.395962+0000","last_fullsized":"2024-05-22T21:19:46.395962+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:03:38.619514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.410529+0000","last_change":"2024-05-22T21:19:46.409946+0000","last_active":"2024-05-22T21:19:46.410529+0000","last_peered":"2024-05-22T21:19:46.410529+0000","last_clean":"2024-05-22T21:19:46.410529+0000","last_became_active":"2024-05-22T21:19:46.409353+0000","last_became_peered":"2024-05-22T21:19:46.409353+0000","last_unstale":"2024-05-22T21:19:46.410529+0000","last_undegraded":"2024-05-22T21:19:46.410529+0000","last_fullsized":"2024-05-22T21:19:46.410529+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:41:15.705579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.393933+0000","last_change":"2024-05-22T21:19:46.015030+0000","last_active":"2024-05-22T21:19:46.393933+0000","last_peered":"2024-05-22T21:19:46.393933+0000","last_clean":"2024-05-22T21:19:46.393933+0000","last_became_active":"2024-05-22T21:19:46.014547+0000","last_became_peered":"2024-05-22T21:19:46.014547+0000","last_unstale":"2024-05-22T21:19:46.393933+0000","last_undegraded":"2024-05-22T21:19:46.393933+0000","last_fullsized":"2024-05-22T21:19:46.393933+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:22:14.287695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.413478+0000","last_change":"2024-05-22T21:19:46.413414+0000","last_active":"2024-05-22T21:19:46.413478+0000","last_peered":"2024-05-22T21:19:46.413478+0000","last_clean":"2024-05-22T21:19:46.413478+0000","last_became_active":"2024-05-22T21:19:46.413182+0000","last_became_peered":"2024-05-22T21:19:46.413182+0000","last_unstale":"2024-05-22T21:19:46.413478+0000","last_undegraded":"2024-05-22T21:19:46.413478+0000","last_fullsized":"2024-05-22T21:19:46.413478+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:34:06.573812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.389287+0000","last_change":"2024-05-22T21:19:46.018873+0000","last_active":"2024-05-22T21:19:46.389287+0000","last_peered":"2024-05-22T21:19:46.389287+0000","last_clean":"2024-05-22T21:19:46.389287+0000","last_became_active":"2024-05-22T21:19:46.018665+0000","last_became_peered":"2024-05-22T21:19:46.018665+0000","last_unstale":"2024-05-22T21:19:46.389287+0000","last_undegraded":"2024-05-22T21:19:46.389287+0000","last_fullsized":"2024-05-22T21:19:46.389287+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:54:22.384897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.401499+0000","last_change":"2024-05-22T21:19:46.400771+0000","last_active":"2024-05-22T21:19:46.401499+0000","last_peered":"2024-05-22T21:19:46.401499+0000","last_clean":"2024-05-22T21:19:46.401499+0000","last_became_active":"2024-05-22T21:19:46.400240+0000","last_became_peered":"2024-05-22T21:19:46.400240+0000","last_unstale":"2024-05-22T21:19:46.401499+0000","last_undegraded":"2024-05-22T21:19:46.401499+0000","last_fullsized":"2024-05-22T21:19:46.401499+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:34:06.573812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.392121+0000","last_change":"2024-05-22T21:19:46.014685+0000","last_active":"2024-05-22T21:19:46.392121+0000","last_peered":"2024-05-22T21:19:46.392121+0000","last_clean":"2024-05-22T21:19:46.392121+0000","last_became_active":"2024-05-22T21:19:46.012483+0000","last_became_peered":"2024-05-22T21:19:46.012483+0000","last_unstale":"2024-05-22T21:19:46.392121+0000","last_undegraded":"2024-05-22T21:19:46.392121+0000","last_fullsized":"2024-05-22T21:19:46.392121+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:29:12.856142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405895+0000","last_change":"2024-05-22T21:19:46.405513+0000","last_active":"2024-05-22T21:19:46.405895+0000","last_peered":"2024-05-22T21:19:46.405895+0000","last_clean":"2024-05-22T21:19:46.405895+0000","last_became_active":"2024-05-22T21:19:46.404797+0000","last_became_peered":"2024-05-22T21:19:46.404797+0000","last_unstale":"2024-05-22T21:19:46.405895+0000","last_undegraded":"2024-05-22T21:19:46.405895+0000","last_fullsized":"2024-05-22T21:19:46.405895+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:50:22.028491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.399818+0000","last_change":"2024-05-22T21:19:46.010542+0000","last_active":"2024-05-22T21:19:46.399818+0000","last_peered":"2024-05-22T21:19:46.399818+0000","last_clean":"2024-05-22T21:19:46.399818+0000","last_became_active":"2024-05-22T21:19:46.009436+0000","last_became_peered":"2024-05-22T21:19:46.009436+0000","last_unstale":"2024-05-22T21:19:46.399818+0000","last_undegraded":"2024-05-22T21:19:46.399818+0000","last_fullsized":"2024-05-22T21:19:46.399818+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:55:57.871676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.403140+0000","last_change":"2024-05-22T21:19:46.403062+0000","last_active":"2024-05-22T21:19:46.403140+0000","last_peered":"2024-05-22T21:19:46.403140+0000","last_clean":"2024-05-22T21:19:46.403140+0000","last_became_active":"2024-05-22T21:19:46.400387+0000","last_became_peered":"2024-05-22T21:19:46.400387+0000","last_unstale":"2024-05-22T21:19:46.403140+0000","last_undegraded":"2024-05-22T21:19:46.403140+0000","last_fullsized":"2024-05-22T21:19:46.403140+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:02:04.279027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.393642+0000","last_change":"2024-05-22T21:19:46.014414+0000","last_active":"2024-05-22T21:19:46.393642+0000","last_peered":"2024-05-22T21:19:46.393642+0000","last_clean":"2024-05-22T21:19:46.393642+0000","last_became_active":"2024-05-22T21:19:46.014225+0000","last_became_peered":"2024-05-22T21:19:46.014225+0000","last_unstale":"2024-05-22T21:19:46.393642+0000","last_undegraded":"2024-05-22T21:19:46.393642+0000","last_fullsized":"2024-05-22T21:19:46.393642+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:32:32.488654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.408786+0000","last_change":"2024-05-22T21:19:46.405524+0000","last_active":"2024-05-22T21:19:46.408786+0000","last_peered":"2024-05-22T21:19:46.408786+0000","last_clean":"2024-05-22T21:19:46.408786+0000","last_became_active":"2024-05-22T21:19:46.405189+0000","last_became_peered":"2024-05-22T21:19:46.405189+0000","last_unstale":"2024-05-22T21:19:46.408786+0000","last_undegraded":"2024-05-22T21:19:46.408786+0000","last_fullsized":"2024-05-22T21:19:46.408786+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:27:54.460213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.394021+0000","last_change":"2024-05-22T21:19:46.015160+0000","last_active":"2024-05-22T21:19:46.394021+0000","last_peered":"2024-05-22T21:19:46.394021+0000","last_clean":"2024-05-22T21:19:46.394021+0000","last_became_active":"2024-05-22T21:19:46.014830+0000","last_became_peered":"2024-05-22T21:19:46.014830+0000","last_unstale":"2024-05-22T21:19:46.394021+0000","last_undegraded":"2024-05-22T21:19:46.394021+0000","last_fullsized":"2024-05-22T21:19:46.394021+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:09:41.289246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.393754+0000","last_change":"2024-05-22T21:19:46.393648+0000","last_active":"2024-05-22T21:19:46.393754+0000","last_peered":"2024-05-22T21:19:46.393754+0000","last_clean":"2024-05-22T21:19:46.393754+0000","last_became_active":"2024-05-22T21:19:46.393424+0000","last_became_peered":"2024-05-22T21:19:46.393424+0000","last_unstale":"2024-05-22T21:19:46.393754+0000","last_undegraded":"2024-05-22T21:19:46.393754+0000","last_fullsized":"2024-05-22T21:19:46.393754+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:26:38.251056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.392791+0000","last_change":"2024-05-22T21:19:46.017323+0000","last_active":"2024-05-22T21:19:46.392791+0000","last_peered":"2024-05-22T21:19:46.392791+0000","last_clean":"2024-05-22T21:19:46.392791+0000","last_became_active":"2024-05-22T21:19:46.017045+0000","last_became_peered":"2024-05-22T21:19:46.017045+0000","last_unstale":"2024-05-22T21:19:46.392791+0000","last_undegraded":"2024-05-22T21:19:46.392791+0000","last_fullsized":"2024-05-22T21:19:46.392791+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:18:45.657953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":285,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.006135+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:19:46.006135+0000","last_peered":"2024-05-22T21:19:46.006135+0000","last_clean":"2024-05-22T21:19:46.006135+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:19:46.006135+0000","last_undegraded":"2024-05-22T21:19:46.006135+0000","last_fullsized":"2024-05-22T21:19:46.006135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1329,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193983,"statfs":{"total":200000000,"available":198639038,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68500000000000005,"15min":0.748},"min":{"1min":0.435,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91100000000000003,"5min":0.998,"15min":2.0369999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91300000000000003,"5min":0.97699999999999998,"15min":1.889},"last":0.78300000000000003}]},{"osd":1,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71199999999999997,"15min":0.76500000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.41699999999999998},"max":{"1min":0.99099999999999999,"5min":1.0840000000000001,"15min":3.0920000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70599999999999996,"15min":0.75700000000000001},"min":{"1min":0.53700000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.877,"5min":0.92700000000000005,"15min":3.1499999999999999},"last":0.53500000000000003}]},{"osd":2,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.38800000000000001},"max":{"1min":0.90000000000000002,"5min":0.94999999999999996,"15min":1.923},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.748,"15min":0.72799999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.71599999999999997}]},{"osd":3,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71899999999999997,"15min":0.69799999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95899999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.74399999999999999,"15min":0.78800000000000003},"min":{"1min":0.47599999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.04,"5min":1.04,"15min":2.1389999999999998},"last":0.80600000000000005}]},{"osd":4,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.38200000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.93100000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90900000000000003,"5min":1.018,"15min":1.018},"last":0.627}]},{"osd":5,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.65100000000000002},"min":{"1min":0.438,"5min":0.371,"15min":0.371},"max":{"1min":0.88500000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.42899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95599999999999996,"5min":0.996,"15min":0.996},"last":0.89500000000000002}]},{"osd":6,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52000000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.68000000000000005,"15min":0.64600000000000002},"min":{"1min":0.42199999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.74199999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":739,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194573,"statfs":{"total":200000000,"available":199243072,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69599999999999995,"15min":0.69299999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0049999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70899999999999996,"15min":0.69099999999999995},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.39500000000000002},"max":{"1min":0.997,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.82699999999999996}]},{"osd":1,"last update":"Wed May 22 21:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70999999999999996,"15min":0.68400000000000005},"min":{"1min":0.55100000000000005,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.91700000000000004,"5min":1.107,"15min":1.107},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.53300000000000003,"5min":0.371,"15min":0.371},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.081},"last":0.746}]},{"osd":2,"last update":"Wed May 22 21:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78600000000000003,"15min":0.76000000000000001},"min":{"1min":0.52000000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.046,"5min":1.087,"15min":1.087},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.745,"15min":0.73899999999999999},"min":{"1min":0.55100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.873}]},{"osd":3,"last update":"Wed May 22 21:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.48199999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.415},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.72199999999999998,"15min":0.72599999999999998},"min":{"1min":0.54500000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91800000000000004,"5min":1.788,"15min":1.788},"last":0.69299999999999995}]},{"osd":4,"last update":"Wed May 22 21:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.45000000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.44400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.0309999999999999,"5min":2.718,"15min":2.718},"last":0.72599999999999998}]},{"osd":5,"last update":"Wed May 22 21:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.73499999999999999,"15min":0.69999999999999996},"min":{"1min":0.45300000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77600000000000002,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70199999999999996,"15min":0.71299999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98499999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.66900000000000004}]},{"osd":7,"last update":"Wed May 22 21:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70599999999999996,"15min":0.73599999999999999},"min":{"1min":0.51600000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.71299999999999997,"15min":0.752},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.98099999999999998,"5min":1.2450000000000001,"15min":1.8009999999999999},"last":0.76100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":745,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194567,"statfs":{"total":200000000,"available":199236797,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.67100000000000004,"15min":0.69799999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.98699999999999999,"15min":1.0409999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.38400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.85899999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.73599999999999999}]},{"osd":1,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69299999999999995,"15min":0.70499999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.68899999999999995},"min":{"1min":0.46400000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.54100000000000004}]},{"osd":2,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73199999999999998,"15min":0.70399999999999996},"min":{"1min":0.53800000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70499999999999996,"15min":0.67900000000000005},"min":{"1min":0.432,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.48799999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.93400000000000005,"5min":0.97199999999999998,"15min":1.1879999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.111},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57299999999999995,"15min":0.621},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77600000000000002,"5min":0.92800000000000005,"15min":1.272},"last":0.433},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.623,"15min":0.63},"min":{"1min":0.433,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88100000000000001,"5min":1.038,"15min":1.038},"last":0.56699999999999995}]},{"osd":6,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63,"15min":0.628},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85699999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.45100000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.80300000000000005,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.58799999999999997}]},{"osd":7,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69599999999999995,"15min":0.71099999999999997},"min":{"1min":0.48999999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96299999999999997,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73499999999999999,"15min":0.78200000000000003},"min":{"1min":0.47399999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91200000000000003,"5min":1.008,"15min":1.2909999999999999},"last":0.70199999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":745,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194567,"statfs":{"total":200000000,"available":199236797,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.78300000000000003,"15min":0.748},"min":{"1min":0.47099999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.94199999999999995,"5min":6.6539999999999999,"15min":6.6539999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.748,"15min":0.72799999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98899999999999999,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":0.61199999999999999}]},{"osd":1,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65800000000000003,"15min":0.70799999999999996},"min":{"1min":0.41999999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86499999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.73099999999999998,"15min":0.70999999999999996},"min":{"1min":0.52300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":0.51100000000000001}]},{"osd":2,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.72999999999999998,"15min":0.69799999999999995},"min":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.4660000000000002,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.71599999999999997,"15min":0.70899999999999996},"min":{"1min":0.53300000000000003,"5min":0.378,"15min":0.378},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.70199999999999996,"15min":0.69499999999999995},"min":{"1min":0.49099999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.77400000000000002,"15min":0.79200000000000004},"min":{"1min":0.629,"5min":0.51500000000000001,"15min":0.47499999999999998},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":0.63400000000000001}]},{"osd":5,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.63900000000000001},"min":{"1min":0.39200000000000002,"5min":0.38,"15min":0.379},"max":{"1min":0.90000000000000002,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.439},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0409999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.52600000000000002}]},{"osd":6,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64800000000000002,"15min":0.68899999999999995},"min":{"1min":0.46600000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.434},"max":{"1min":0.92700000000000005,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.67400000000000004}]},{"osd":7,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.44700000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2150000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72699999999999998,"15min":0.77500000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.4510000000000001},"last":0.873}]}]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":739,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194573,"statfs":{"total":200000000,"available":199243072,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.436,"15min":0.40899999999999997},"min":{"1min":0.25700000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.64700000000000002,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.45100000000000001,"15min":0.41099999999999998},"min":{"1min":0.314,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.58399999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.32800000000000001}]},{"osd":1,"last update":"Wed May 22 21:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.46800000000000003,"15min":0.45000000000000001},"min":{"1min":0.33300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.625,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53600000000000003,"15min":0.51000000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33800000000000002},"max":{"1min":0.70799999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53200000000000003}]},{"osd":2,"last update":"Wed May 22 21:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.50900000000000001,"15min":0.495},"min":{"1min":0.35999999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69099999999999995,"5min":0.70699999999999996,"15min":0.86799999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51200000000000001,"15min":0.48999999999999999},"min":{"1min":0.34399999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.75600000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56899999999999995}]},{"osd":4,"last update":"Wed May 22 21:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.73799999999999999,"15min":0.70499999999999996},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89900000000000002,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.749,"15min":0.71899999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.435},"max":{"1min":1,"5min":1.397,"15min":1.5760000000000001},"last":0.75600000000000001}]},{"osd":5,"last update":"Wed May 22 21:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76400000000000001,"15min":0.70099999999999996},"min":{"1min":0.53700000000000003,"5min":0.48099999999999998,"15min":0.45100000000000001},"max":{"1min":1.048,"5min":1.234,"15min":1.234},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.751,"15min":0.70399999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.93799999999999994,"5min":1.292,"15min":1.3049999999999999},"last":0.80400000000000005}]},{"osd":6,"last update":"Wed May 22 21:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.75,"15min":0.71699999999999997},"min":{"1min":0.54300000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92100000000000004,"5min":1.276,"15min":1.276},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76300000000000001,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0820000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.68600000000000005}]},{"osd":7,"last update":"Wed May 22 21:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.79300000000000004,"15min":0.75700000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.495},"max":{"1min":1.0309999999999999,"5min":1.1000000000000001,"15min":1.2709999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.79000000000000004,"15min":0.74199999999999999},"min":{"1min":0.55600000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.071,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.79700000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":745,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194567,"statfs":{"total":200000000,"available":199236797,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.44400000000000001,"15min":0.40999999999999998},"min":{"1min":0.253,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.48199999999999998,"15min":0.44900000000000001},"min":{"1min":0.26800000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.38300000000000001}]},{"osd":1,"last update":"Wed May 22 21:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.47899999999999998,"15min":0.45400000000000001},"min":{"1min":0.35699999999999998,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.56899999999999995,"15min":0.53000000000000003},"min":{"1min":0.44900000000000001,"5min":0.41199999999999998,"15min":0.39100000000000001},"max":{"1min":2.8159999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.496}]},{"osd":3,"last update":"Wed May 22 21:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53000000000000003,"15min":0.50800000000000001},"min":{"1min":0.39600000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.75,"5min":2.75,"15min":2.75},"last":0.436},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.51500000000000001,"15min":0.47299999999999998},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":3.0619999999999998,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":0.45000000000000001}]},{"osd":4,"last update":"Wed May 22 21:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.77700000000000002,"15min":0.70499999999999996},"min":{"1min":0.54900000000000004,"5min":0.53100000000000003,"15min":0.52700000000000002},"max":{"1min":2.7610000000000001,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.80400000000000005,"15min":0.76000000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.73099999999999998}]},{"osd":5,"last update":"Wed May 22 21:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.76900000000000002,"15min":0.71899999999999997},"min":{"1min":0.498,"5min":0.48099999999999998,"15min":0.47299999999999998},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.75900000000000001,"15min":0.71299999999999997},"min":{"1min":0.53000000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Wed May 22 21:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.76500000000000001,"15min":0.71499999999999997},"min":{"1min":0.56599999999999995,"5min":0.495,"15min":0.495},"max":{"1min":2.8580000000000001,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.73099999999999998,"15min":0.68899999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.6880000000000002,"5min":2.6880000000000002,"15min":2.6880000000000002},"last":0.67700000000000005}]},{"osd":7,"last update":"Wed May 22 21:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.79900000000000004,"15min":0.74299999999999999},"min":{"1min":0.59799999999999998,"5min":0.438,"15min":0.438},"max":{"1min":3.0099999999999998,"5min":3.0099999999999998,"15min":3.0099999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.82299999999999995,"15min":0.78300000000000003},"min":{"1min":0.629,"5min":0.499,"15min":0.499},"max":{"1min":2.827,"5min":2.827,"15min":2.827},"last":0.749}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":739,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194573,"statfs":{"total":200000000,"available":199243072,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.42499999999999999,"15min":0.433},"min":{"1min":0.25800000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46999999999999997,"15min":0.44},"min":{"1min":0.29899999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60199999999999998,"5min":0.84799999999999998,"15min":0.876},"last":0.48399999999999999}]},{"osd":2,"last update":"Wed May 22 21:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.50600000000000001,"15min":0.496},"min":{"1min":0.373,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.52600000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":1.923},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed May 22 21:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52300000000000002,"15min":0.56200000000000006},"min":{"1min":0.34200000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80400000000000005,"5min":0.82799999999999996,"15min":2.1030000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.50700000000000001,"15min":0.49199999999999999},"min":{"1min":0.41299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.73099999999999998},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed May 22 21:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.74099999999999999,"15min":0.76000000000000001},"min":{"1min":0.57599999999999996,"5min":0.46500000000000002,"15min":0.45100000000000001},"max":{"1min":1.0620000000000001,"5min":1.109,"15min":2.1360000000000001},"last":0.623},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.746,"15min":0.69599999999999995},"min":{"1min":0.46500000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Wed May 22 21:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.79100000000000004,"15min":0.74399999999999999},"min":{"1min":0.57699999999999996,"5min":0.53500000000000003,"15min":0.45300000000000001},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.74399999999999999,"15min":0.75},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.40100000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.97},"last":0.66600000000000004}]},{"osd":6,"last update":"Wed May 22 21:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78400000000000003,"15min":0.752},"min":{"1min":0.54200000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.071,"5min":1.085,"15min":1.272},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.81000000000000005,"15min":0.75800000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.53200000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.71799999999999997}]},{"osd":7,"last update":"Wed May 22 21:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.42099999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":2.0310000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.77400000000000002},"min":{"1min":0.56799999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98199999999999998,"5min":1.0449999999999999,"15min":2.0659999999999998},"last":0.70099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1315,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193997,"statfs":{"total":200000000,"available":198653323,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.437,"15min":0.40699999999999997},"min":{"1min":0.245,"5min":0.24299999999999999,"15min":0.22800000000000001},"max":{"1min":0.85899999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51500000000000001,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72899999999999998,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48699999999999999}]},{"osd":2,"last update":"Wed May 22 21:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.495,"15min":0.45700000000000002},"min":{"1min":0.34300000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.88400000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46300000000000002,"15min":0.43099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64500000000000002,"5min":0.76500000000000001,"15min":0.92300000000000004},"last":0.37}]},{"osd":3,"last update":"Wed May 22 21:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46400000000000002,"15min":0.436},"min":{"1min":0.313,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67300000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55200000000000005,"15min":0.50900000000000001},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.46600000000000003}]},{"osd":4,"last update":"Wed May 22 21:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.74099999999999999,"15min":0.71299999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96299999999999997,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.874},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.755,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.49099999999999999},"max":{"1min":1.0429999999999999,"5min":1.113,"15min":1.2709999999999999},"last":0.754}]},{"osd":5,"last update":"Wed May 22 21:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72399999999999998,"15min":0.70899999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93999999999999995,"5min":0.99299999999999999,"15min":1.1339999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.77400000000000002,"15min":0.74099999999999999},"min":{"1min":0.48199999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.111,"5min":1.143,"15min":1.2929999999999999},"last":0.77900000000000003}]},{"osd":6,"last update":"Wed May 22 21:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.80000000000000004,"15min":0.77500000000000002},"min":{"1min":0.60799999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.095,"5min":1.1240000000000001,"15min":1.2090000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78100000000000003,"15min":0.77200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.069,"5min":1.2090000000000001,"15min":1.5},"last":0.69399999999999995}]},{"osd":7,"last update":"Wed May 22 21:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76400000000000001,"15min":0.72999999999999998},"min":{"1min":0.56699999999999995,"5min":0.55000000000000004,"15min":0.503},"max":{"1min":0.93100000000000005,"5min":1.163,"15min":1.163},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.53600000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.01,"5min":1.1990000000000001,"15min":1.3260000000000001},"last":0.73899999999999999}]}]}],"pool_statfs":[]}} 2024-05-22T21:19:50.465 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:19:50.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.639+0000 7f9616400640 1 Processor -- start 2024-05-22T21:19:50.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.639+0000 7f9616400640 1 -- start start 2024-05-22T21:19:50.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.639+0000 7f9616400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f961015dd50 0x7f961006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:50.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.639+0000 7f9616400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f961015e130 0x7f961015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:50.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.639+0000 7f9616400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9610071670 con 0x7f961015d410 2024-05-22T21:19:50.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.639+0000 7f9616400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f961015d8a0 con 0x7f961015dd50 2024-05-22T21:19:50.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.639+0000 7f9616400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961006de40 con 0x7f961015e130 2024-05-22T21:19:50.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.639+0000 7f960fe00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f961015d410 0x7f961015a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48882/0 (socket says 172.21.15.161:48882) 2024-05-22T21:19:50.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.639+0000 7f960fe00640 1 -- 172.21.15.161:0/3844157790 learned_addr learned my addr 172.21.15.161:0/3844157790 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:50.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960f400640 1 --2- 172.21.15.161:0/3844157790 >> v2:172.21.15.179:3300/0 conn(0x7f961015dd50 0x7f961006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:50.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f9606a00640 1 --2- 172.21.15.161:0/3844157790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f961015e130 0x7f961015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:50.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960ea00640 1 -- 172.21.15.161:0/3844157790 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865872513 0 0) 0x7f9610071670 con 0x7f961015d410 2024-05-22T21:19:50.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960ea00640 1 -- 172.21.15.161:0/3844157790 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95f4003530 con 0x7f961015d410 2024-05-22T21:19:50.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960ea00640 1 -- 172.21.15.161:0/3844157790 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2042558481 0 0) 0x7f95f4003530 con 0x7f961015d410 2024-05-22T21:19:50.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960ea00640 1 -- 172.21.15.161:0/3844157790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f961015e130 msgr2=0x7f961015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:50.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960ea00640 1 --2- 172.21.15.161:0/3844157790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f961015e130 0x7f961015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960ea00640 1 -- 172.21.15.161:0/3844157790 >> v2:172.21.15.179:3300/0 conn(0x7f961015dd50 msgr2=0x7f961006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:50.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960ea00640 1 --2- 172.21.15.161:0/3844157790 >> v2:172.21.15.179:3300/0 conn(0x7f961015dd50 0x7f961006d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960ea00640 1 -- 172.21.15.161:0/3844157790 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9610070a60 con 0x7f961015d410 2024-05-22T21:19:50.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960ea00640 1 -- 172.21.15.161:0/3844157790 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f9608001a00 con 0x7f961015d410 2024-05-22T21:19:50.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960ea00640 1 -- 172.21.15.161:0/3844157790 >> v1:172.21.15.161:6789/0 conn(0x7f961015d410 legacy=0x7f961015a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:50.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960ea00640 1 --2- 172.21.15.161:0/3844157790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95f4003980 0x7f95f4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:50.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960ea00640 1 --2- 172.21.15.161:0/3844157790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f40047d0 0x7f95f4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:50.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960ea00640 1 --2- 172.21.15.161:0/3844157790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f961015e130 0x7f95f4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:50.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960ea00640 1 -- 172.21.15.161:0/3844157790 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961015d8a0 con 0x7f95f40047d0 2024-05-22T21:19:50.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960ea00640 1 -- 172.21.15.161:0/3844157790 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961006de40 con 0x7f95f4003980 2024-05-22T21:19:50.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960ea00640 1 -- 172.21.15.161:0/3844157790 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f400dba0 con 0x7f961015e130 2024-05-22T21:19:50.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f9606a00640 1 --2- 172.21.15.161:0/3844157790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f40047d0 0x7f95f4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:50.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960f400640 1 --2- 172.21.15.161:0/3844157790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95f4003980 0x7f95f4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:50.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960fe00640 1 --2- 172.21.15.161:0/3844157790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f961015e130 0x7f95f4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:50.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f9606a00640 1 -- 172.21.15.161:0/3844157790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f961015e130 msgr2=0x7f95f4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:50.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f9606a00640 1 --2- 172.21.15.161:0/3844157790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f961015e130 0x7f95f4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f9606a00640 1 -- 172.21.15.161:0/3844157790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95f4003980 msgr2=0x7f95f4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:50.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f9606a00640 1 --2- 172.21.15.161:0/3844157790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95f4003980 0x7f95f4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f9606a00640 1 -- 172.21.15.161:0/3844157790 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f4003530 con 0x7f95f40047d0 2024-05-22T21:19:50.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.643+0000 7f960fe00640 1 --2- 172.21.15.161:0/3844157790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f961015e130 0x7f95f4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:50.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f9606a00640 1 --2- 172.21.15.161:0/3844157790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f40047d0 0x7f95f4004c50 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009b10 tx=0x7f95f801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fe861d6635ee51 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:50.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f960ea00640 1 -- 172.21.15.161:0/3844157790 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f80a5530 con 0x7f95f40047d0 2024-05-22T21:19:50.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f960ea00640 1 -- 172.21.15.161:0/3844157790 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f80a5690 con 0x7f95f40047d0 2024-05-22T21:19:50.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f960ea00640 1 -- 172.21.15.161:0/3844157790 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f80a5a40 con 0x7f95f40047d0 2024-05-22T21:19:50.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f9616400640 1 -- 172.21.15.161:0/3844157790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f40047d0 msgr2=0x7f95f4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:50.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f9616400640 1 --2- 172.21.15.161:0/3844157790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f40047d0 0x7f95f4004c50 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009b10 tx=0x7f95f801ca40 comp rx=0 tx=0).stop 2024-05-22T21:19:50.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f960fe00640 1 -- 172.21.15.161:0/3844157790 reap_dead start 2024-05-22T21:19:50.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f9616400640 1 -- 172.21.15.161:0/3844157790 shutdown_connections 2024-05-22T21:19:50.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f9616400640 1 -- 172.21.15.161:0/3844157790 >> 172.21.15.161:0/3844157790 conn(0x7f96100b0bc0 msgr2=0x7f96100b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:50.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f9616400640 1 -- 172.21.15.161:0/3844157790 shutdown_connections 2024-05-22T21:19:50.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f9616400640 1 -- 172.21.15.161:0/3844157790 wait complete. 2024-05-22T21:19:50.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f9616400640 1 Processor -- start 2024-05-22T21:19:50.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f9616400640 1 -- start start 2024-05-22T21:19:50.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f9616400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f40047d0 0x7f96100cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:50.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f9616400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9610152ba0 0x7f96101f9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:50.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f9616400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96101f9e40 0x7f9610157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:50.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f9616400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96101f9470 con 0x7f95f40047d0 2024-05-22T21:19:50.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f9616400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9610153060 con 0x7f9610152ba0 2024-05-22T21:19:50.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f9616400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9610153340 con 0x7f96101f9e40 2024-05-22T21:19:50.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f960f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f40047d0 0x7f96100cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:50.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f960f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f40047d0 0x7f96100cdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50746/0 (socket says 172.21.15.161:50746) 2024-05-22T21:19:50.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.647+0000 7f960f400640 1 -- 172.21.15.161:0/1939225784 learned_addr learned my addr 172.21.15.161:0/1939225784 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:50.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f9606a00640 1 --2- 172.21.15.161:0/1939225784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9610152ba0 0x7f96101f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:50.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f960fe00640 1 --2- 172.21.15.161:0/1939225784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96101f9e40 0x7f9610157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:50.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f960f400640 1 -- 172.21.15.161:0/1939225784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96101f9e40 msgr2=0x7f9610157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:50.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f960f400640 1 --2- 172.21.15.161:0/1939225784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96101f9e40 0x7f9610157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f960f400640 1 -- 172.21.15.161:0/1939225784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9610152ba0 msgr2=0x7f96101f9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:50.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f960f400640 1 --2- 172.21.15.161:0/1939225784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9610152ba0 0x7f96101f9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f960f400640 1 -- 172.21.15.161:0/1939225784 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9610153620 con 0x7f95f40047d0 2024-05-22T21:19:50.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f9606a00640 1 --2- 172.21.15.161:0/1939225784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9610152ba0 0x7f96101f9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:50.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f960fe00640 1 --2- 172.21.15.161:0/1939225784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96101f9e40 0x7f9610157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:50.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f960f400640 1 --2- 172.21.15.161:0/1939225784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f40047d0 0x7f96100cdea0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f9600011aa0 tx=0x7f9600011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:50.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f9607e00640 1 -- 172.21.15.161:0/1939225784 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960000fb80 con 0x7f95f40047d0 2024-05-22T21:19:50.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f9607e00640 1 -- 172.21.15.161:0/1939225784 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960000fce0 con 0x7f95f40047d0 2024-05-22T21:19:50.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f9607e00640 1 -- 172.21.15.161:0/1939225784 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600014680 con 0x7f95f40047d0 2024-05-22T21:19:50.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f9616400640 1 -- 172.21.15.161:0/1939225784 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96100ce3e0 con 0x7f95f40047d0 2024-05-22T21:19:50.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f9616400640 1 -- 172.21.15.161:0/1939225784 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9610157730 con 0x7f95f40047d0 2024-05-22T21:19:50.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f9616400640 1 -- 172.21.15.161:0/1939225784 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96101f9240 con 0x7f95f40047d0 2024-05-22T21:19:50.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f9607e00640 1 -- 172.21.15.161:0/1939225784 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f96000147e0 con 0x7f95f40047d0 2024-05-22T21:19:50.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f9607e00640 1 --2- 172.21.15.161:0/1939225784 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f95ec066690 0x7f95ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:50.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f9607e00640 1 -- 172.21.15.161:0/1939225784 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9600084b60 con 0x7f95f40047d0 2024-05-22T21:19:50.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.651+0000 7f9606a00640 1 --2- 172.21.15.161:0/1939225784 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f95ec066690 0x7f95ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:50.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.655+0000 7f9606a00640 1 --2- 172.21.15.161:0/1939225784 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f95ec066690 0x7f95ec068b50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f95f80af000 tx=0x7f95f80046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:50.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.655+0000 7f9607e00640 1 -- 172.21.15.161:0/1939225784 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f96101f9240 con 0x7f95f40047d0 2024-05-22T21:19:50.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.807+0000 7f9616400640 1 -- 172.21.15.161:0/1939225784 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9610070730 con 0x7f95ec066690 2024-05-22T21:19:50.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.815+0000 7f9607e00640 1 -- 172.21.15.161:0/1939225784 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284698 (secure 0 0 0) 0x7f95f80a5800 con 0x7f95ec066690 2024-05-22T21:19:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:50.820 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:19:50.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.819+0000 7f9616400640 1 -- 172.21.15.161:0/1939225784 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f95ec066690 msgr2=0x7f95ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:50.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.819+0000 7f9616400640 1 --2- 172.21.15.161:0/1939225784 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f95ec066690 0x7f95ec068b50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f95f80af000 tx=0x7f95f80046d0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.819+0000 7f9616400640 1 -- 172.21.15.161:0/1939225784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f40047d0 msgr2=0x7f96100cdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:50.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.819+0000 7f9616400640 1 --2- 172.21.15.161:0/1939225784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f40047d0 0x7f96100cdea0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f9600011aa0 tx=0x7f9600011f60 comp rx=0 tx=0).stop 2024-05-22T21:19:50.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.819+0000 7f9616400640 1 -- 172.21.15.161:0/1939225784 shutdown_connections 2024-05-22T21:19:50.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.819+0000 7f9616400640 1 --2- 172.21.15.161:0/1939225784 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f95ec066690 0x7f95ec068b50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.819+0000 7f9616400640 1 --2- 172.21.15.161:0/1939225784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96101f9e40 0x7f9610157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.819+0000 7f9616400640 1 --2- 172.21.15.161:0/1939225784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9610152ba0 0x7f96101f9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.819+0000 7f9616400640 1 --2- 172.21.15.161:0/1939225784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95f40047d0 0x7f96100cdea0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:50.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.819+0000 7f9616400640 1 -- 172.21.15.161:0/1939225784 >> 172.21.15.161:0/1939225784 conn(0x7f96100b0bc0 msgr2=0x7f96100b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:50.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.819+0000 7f9616400640 1 -- 172.21.15.161:0/1939225784 shutdown_connections 2024-05-22T21:19:50.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:50.819+0000 7f9616400640 1 -- 172.21.15.161:0/1939225784 wait complete. 2024-05-22T21:19:50.840 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-05-22T21:19:50.478021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7097,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555402,"statfs":{"total":1600000000,"available":1592731968,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.479295"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.414472+0000","last_change":"2024-05-22T21:19:46.413471+0000","last_active":"2024-05-22T21:19:46.414472+0000","last_peered":"2024-05-22T21:19:46.414472+0000","last_clean":"2024-05-22T21:19:46.414472+0000","last_became_active":"2024-05-22T21:19:46.413210+0000","last_became_peered":"2024-05-22T21:19:46.413210+0000","last_unstale":"2024-05-22T21:19:46.414472+0000","last_undegraded":"2024-05-22T21:19:46.414472+0000","last_fullsized":"2024-05-22T21:19:46.414472+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:26:59.008255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.403761+0000","last_change":"2024-05-22T21:19:46.403546+0000","last_active":"2024-05-22T21:19:46.403761+0000","last_peered":"2024-05-22T21:19:46.403761+0000","last_clean":"2024-05-22T21:19:46.403761+0000","last_became_active":"2024-05-22T21:19:46.402005+0000","last_became_peered":"2024-05-22T21:19:46.402005+0000","last_unstale":"2024-05-22T21:19:46.403761+0000","last_undegraded":"2024-05-22T21:19:46.403761+0000","last_fullsized":"2024-05-22T21:19:46.403761+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:33:07.243615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405477+0000","last_change":"2024-05-22T21:19:46.404603+0000","last_active":"2024-05-22T21:19:46.405477+0000","last_peered":"2024-05-22T21:19:46.405477+0000","last_clean":"2024-05-22T21:19:46.405477+0000","last_became_active":"2024-05-22T21:19:46.403844+0000","last_became_peered":"2024-05-22T21:19:46.403844+0000","last_unstale":"2024-05-22T21:19:46.405477+0000","last_undegraded":"2024-05-22T21:19:46.405477+0000","last_fullsized":"2024-05-22T21:19:46.405477+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:30:01.872224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.411135+0000","last_change":"2024-05-22T21:19:46.411013+0000","last_active":"2024-05-22T21:19:46.411135+0000","last_peered":"2024-05-22T21:19:46.411135+0000","last_clean":"2024-05-22T21:19:46.411135+0000","last_became_active":"2024-05-22T21:19:46.403750+0000","last_became_peered":"2024-05-22T21:19:46.403750+0000","last_unstale":"2024-05-22T21:19:46.411135+0000","last_undegraded":"2024-05-22T21:19:46.411135+0000","last_fullsized":"2024-05-22T21:19:46.411135+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:13:33.884650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.403940+0000","last_change":"2024-05-22T21:19:46.403811+0000","last_active":"2024-05-22T21:19:46.403940+0000","last_peered":"2024-05-22T21:19:46.403940+0000","last_clean":"2024-05-22T21:19:46.403940+0000","last_became_active":"2024-05-22T21:19:46.403566+0000","last_became_peered":"2024-05-22T21:19:46.403566+0000","last_unstale":"2024-05-22T21:19:46.403940+0000","last_undegraded":"2024-05-22T21:19:46.403940+0000","last_fullsized":"2024-05-22T21:19:46.403940+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:39:13.726600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.397048+0000","last_change":"2024-05-22T21:19:46.395576+0000","last_active":"2024-05-22T21:19:46.397048+0000","last_peered":"2024-05-22T21:19:46.397048+0000","last_clean":"2024-05-22T21:19:46.397048+0000","last_became_active":"2024-05-22T21:19:46.395269+0000","last_became_peered":"2024-05-22T21:19:46.395269+0000","last_unstale":"2024-05-22T21:19:46.397048+0000","last_undegraded":"2024-05-22T21:19:46.397048+0000","last_fullsized":"2024-05-22T21:19:46.397048+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:05.182818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.396983+0000","last_change":"2024-05-22T21:19:46.396647+0000","last_active":"2024-05-22T21:19:46.396983+0000","last_peered":"2024-05-22T21:19:46.396983+0000","last_clean":"2024-05-22T21:19:46.396983+0000","last_became_active":"2024-05-22T21:19:46.396394+0000","last_became_peered":"2024-05-22T21:19:46.396394+0000","last_unstale":"2024-05-22T21:19:46.396983+0000","last_undegraded":"2024-05-22T21:19:46.396983+0000","last_fullsized":"2024-05-22T21:19:46.396983+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:42:55.341723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405492+0000","last_change":"2024-05-22T21:19:46.404794+0000","last_active":"2024-05-22T21:19:46.405492+0000","last_peered":"2024-05-22T21:19:46.405492+0000","last_clean":"2024-05-22T21:19:46.405492+0000","last_became_active":"2024-05-22T21:19:46.404191+0000","last_became_peered":"2024-05-22T21:19:46.404191+0000","last_unstale":"2024-05-22T21:19:46.405492+0000","last_undegraded":"2024-05-22T21:19:46.405492+0000","last_fullsized":"2024-05-22T21:19:46.405492+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:21:25.780042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.412104+0000","last_change":"2024-05-22T21:19:46.411730+0000","last_active":"2024-05-22T21:19:46.412104+0000","last_peered":"2024-05-22T21:19:46.412104+0000","last_clean":"2024-05-22T21:19:46.412104+0000","last_became_active":"2024-05-22T21:19:46.411498+0000","last_became_peered":"2024-05-22T21:19:46.411498+0000","last_unstale":"2024-05-22T21:19:46.412104+0000","last_undegraded":"2024-05-22T21:19:46.412104+0000","last_fullsized":"2024-05-22T21:19:46.412104+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:14:36.771922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.408767+0000","last_change":"2024-05-22T21:19:46.405370+0000","last_active":"2024-05-22T21:19:46.408767+0000","last_peered":"2024-05-22T21:19:46.408767+0000","last_clean":"2024-05-22T21:19:46.408767+0000","last_became_active":"2024-05-22T21:19:46.405142+0000","last_became_peered":"2024-05-22T21:19:46.405142+0000","last_unstale":"2024-05-22T21:19:46.408767+0000","last_undegraded":"2024-05-22T21:19:46.408767+0000","last_fullsized":"2024-05-22T21:19:46.408767+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:31:52.992029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405838+0000","last_change":"2024-05-22T21:19:46.405034+0000","last_active":"2024-05-22T21:19:46.405838+0000","last_peered":"2024-05-22T21:19:46.405838+0000","last_clean":"2024-05-22T21:19:46.405838+0000","last_became_active":"2024-05-22T21:19:46.402293+0000","last_became_peered":"2024-05-22T21:19:46.402293+0000","last_unstale":"2024-05-22T21:19:46.405838+0000","last_undegraded":"2024-05-22T21:19:46.405838+0000","last_fullsized":"2024-05-22T21:19:46.405838+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:44:41.425002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405549+0000","last_change":"2024-05-22T21:19:46.400707+0000","last_active":"2024-05-22T21:19:46.405549+0000","last_peered":"2024-05-22T21:19:46.405549+0000","last_clean":"2024-05-22T21:19:46.405549+0000","last_became_active":"2024-05-22T21:19:46.400480+0000","last_became_peered":"2024-05-22T21:19:46.400480+0000","last_unstale":"2024-05-22T21:19:46.405549+0000","last_undegraded":"2024-05-22T21:19:46.405549+0000","last_fullsized":"2024-05-22T21:19:46.405549+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:06:57.565296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.401025+0000","last_change":"2024-05-22T21:19:46.400910+0000","last_active":"2024-05-22T21:19:46.401025+0000","last_peered":"2024-05-22T21:19:46.401025+0000","last_clean":"2024-05-22T21:19:46.401025+0000","last_became_active":"2024-05-22T21:19:46.399810+0000","last_became_peered":"2024-05-22T21:19:46.399810+0000","last_unstale":"2024-05-22T21:19:46.401025+0000","last_undegraded":"2024-05-22T21:19:46.401025+0000","last_fullsized":"2024-05-22T21:19:46.401025+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:01.270410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405428+0000","last_change":"2024-05-22T21:19:46.403555+0000","last_active":"2024-05-22T21:19:46.405428+0000","last_peered":"2024-05-22T21:19:46.405428+0000","last_clean":"2024-05-22T21:19:46.405428+0000","last_became_active":"2024-05-22T21:19:46.402813+0000","last_became_peered":"2024-05-22T21:19:46.402813+0000","last_unstale":"2024-05-22T21:19:46.405428+0000","last_undegraded":"2024-05-22T21:19:46.405428+0000","last_fullsized":"2024-05-22T21:19:46.405428+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:09:13.693246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.401219+0000","last_change":"2024-05-22T21:19:46.401157+0000","last_active":"2024-05-22T21:19:46.401219+0000","last_peered":"2024-05-22T21:19:46.401219+0000","last_clean":"2024-05-22T21:19:46.401219+0000","last_became_active":"2024-05-22T21:19:46.400980+0000","last_became_peered":"2024-05-22T21:19:46.400980+0000","last_unstale":"2024-05-22T21:19:46.401219+0000","last_undegraded":"2024-05-22T21:19:46.401219+0000","last_fullsized":"2024-05-22T21:19:46.401219+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:32:18.996718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.408818+0000","last_change":"2024-05-22T21:19:46.406933+0000","last_active":"2024-05-22T21:19:46.408818+0000","last_peered":"2024-05-22T21:19:46.408818+0000","last_clean":"2024-05-22T21:19:46.408818+0000","last_became_active":"2024-05-22T21:19:46.400710+0000","last_became_peered":"2024-05-22T21:19:46.400710+0000","last_unstale":"2024-05-22T21:19:46.408818+0000","last_undegraded":"2024-05-22T21:19:46.408818+0000","last_fullsized":"2024-05-22T21:19:46.408818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:49:37.475366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.403813+0000","last_change":"2024-05-22T21:19:46.403749+0000","last_active":"2024-05-22T21:19:46.403813+0000","last_peered":"2024-05-22T21:19:46.403813+0000","last_clean":"2024-05-22T21:19:46.403813+0000","last_became_active":"2024-05-22T21:19:46.403297+0000","last_became_peered":"2024-05-22T21:19:46.403297+0000","last_unstale":"2024-05-22T21:19:46.403813+0000","last_undegraded":"2024-05-22T21:19:46.403813+0000","last_fullsized":"2024-05-22T21:19:46.403813+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:54:23.394997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.406003+0000","last_change":"2024-05-22T21:19:46.405501+0000","last_active":"2024-05-22T21:19:46.406003+0000","last_peered":"2024-05-22T21:19:46.406003+0000","last_clean":"2024-05-22T21:19:46.406003+0000","last_became_active":"2024-05-22T21:19:46.405179+0000","last_became_peered":"2024-05-22T21:19:46.405179+0000","last_unstale":"2024-05-22T21:19:46.406003+0000","last_undegraded":"2024-05-22T21:19:46.406003+0000","last_fullsized":"2024-05-22T21:19:46.406003+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:33:07.243615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.399659+0000","last_change":"2024-05-22T21:19:46.399527+0000","last_active":"2024-05-22T21:19:46.399659+0000","last_peered":"2024-05-22T21:19:46.399659+0000","last_clean":"2024-05-22T21:19:46.399659+0000","last_became_active":"2024-05-22T21:19:46.398272+0000","last_became_peered":"2024-05-22T21:19:46.398272+0000","last_unstale":"2024-05-22T21:19:46.399659+0000","last_undegraded":"2024-05-22T21:19:46.399659+0000","last_fullsized":"2024-05-22T21:19:46.399659+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:17.111373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.402954+0000","last_change":"2024-05-22T21:19:46.402668+0000","last_active":"2024-05-22T21:19:46.402954+0000","last_peered":"2024-05-22T21:19:46.402954+0000","last_clean":"2024-05-22T21:19:46.402954+0000","last_became_active":"2024-05-22T21:19:46.402301+0000","last_became_peered":"2024-05-22T21:19:46.402301+0000","last_unstale":"2024-05-22T21:19:46.402954+0000","last_undegraded":"2024-05-22T21:19:46.402954+0000","last_fullsized":"2024-05-22T21:19:46.402954+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:07:33.648876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405929+0000","last_change":"2024-05-22T21:19:46.405740+0000","last_active":"2024-05-22T21:19:46.405929+0000","last_peered":"2024-05-22T21:19:46.405929+0000","last_clean":"2024-05-22T21:19:46.405929+0000","last_became_active":"2024-05-22T21:19:46.405278+0000","last_became_peered":"2024-05-22T21:19:46.405278+0000","last_unstale":"2024-05-22T21:19:46.405929+0000","last_undegraded":"2024-05-22T21:19:46.405929+0000","last_fullsized":"2024-05-22T21:19:46.405929+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:43:42.407532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.397912+0000","last_change":"2024-05-22T21:19:46.397497+0000","last_active":"2024-05-22T21:19:46.397912+0000","last_peered":"2024-05-22T21:19:46.397912+0000","last_clean":"2024-05-22T21:19:46.397912+0000","last_became_active":"2024-05-22T21:19:46.397277+0000","last_became_peered":"2024-05-22T21:19:46.397277+0000","last_unstale":"2024-05-22T21:19:46.397912+0000","last_undegraded":"2024-05-22T21:19:46.397912+0000","last_fullsized":"2024-05-22T21:19:46.397912+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:53:07.064592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405878+0000","last_change":"2024-05-22T21:19:46.405410+0000","last_active":"2024-05-22T21:19:46.405878+0000","last_peered":"2024-05-22T21:19:46.405878+0000","last_clean":"2024-05-22T21:19:46.405878+0000","last_became_active":"2024-05-22T21:19:46.405179+0000","last_became_peered":"2024-05-22T21:19:46.405179+0000","last_unstale":"2024-05-22T21:19:46.405878+0000","last_undegraded":"2024-05-22T21:19:46.405878+0000","last_fullsized":"2024-05-22T21:19:46.405878+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:11:23.114796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.394191+0000","last_change":"2024-05-22T21:19:46.393654+0000","last_active":"2024-05-22T21:19:46.394191+0000","last_peered":"2024-05-22T21:19:46.394191+0000","last_clean":"2024-05-22T21:19:46.394191+0000","last_became_active":"2024-05-22T21:19:46.393402+0000","last_became_peered":"2024-05-22T21:19:46.393402+0000","last_unstale":"2024-05-22T21:19:46.394191+0000","last_undegraded":"2024-05-22T21:19:46.394191+0000","last_fullsized":"2024-05-22T21:19:46.394191+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:01.270410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.399591+0000","last_change":"2024-05-22T21:19:46.398753+0000","last_active":"2024-05-22T21:19:46.399591+0000","last_peered":"2024-05-22T21:19:46.399591+0000","last_clean":"2024-05-22T21:19:46.399591+0000","last_became_active":"2024-05-22T21:19:46.398520+0000","last_became_peered":"2024-05-22T21:19:46.398520+0000","last_unstale":"2024-05-22T21:19:46.399591+0000","last_undegraded":"2024-05-22T21:19:46.399591+0000","last_fullsized":"2024-05-22T21:19:46.399591+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:31:33.029588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.408802+0000","last_change":"2024-05-22T21:19:46.406801+0000","last_active":"2024-05-22T21:19:46.408802+0000","last_peered":"2024-05-22T21:19:46.408802+0000","last_clean":"2024-05-22T21:19:46.408802+0000","last_became_active":"2024-05-22T21:19:46.400207+0000","last_became_peered":"2024-05-22T21:19:46.400207+0000","last_unstale":"2024-05-22T21:19:46.408802+0000","last_undegraded":"2024-05-22T21:19:46.408802+0000","last_fullsized":"2024-05-22T21:19:46.408802+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:19:09.914649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.404510+0000","last_change":"2024-05-22T21:19:46.404098+0000","last_active":"2024-05-22T21:19:46.404510+0000","last_peered":"2024-05-22T21:19:46.404510+0000","last_clean":"2024-05-22T21:19:46.404510+0000","last_became_active":"2024-05-22T21:19:46.403895+0000","last_became_peered":"2024-05-22T21:19:46.403895+0000","last_unstale":"2024-05-22T21:19:46.404510+0000","last_undegraded":"2024-05-22T21:19:46.404510+0000","last_fullsized":"2024-05-22T21:19:46.404510+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:33:03.933467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.409367+0000","last_change":"2024-05-22T21:19:46.407866+0000","last_active":"2024-05-22T21:19:46.409367+0000","last_peered":"2024-05-22T21:19:46.409367+0000","last_clean":"2024-05-22T21:19:46.409367+0000","last_became_active":"2024-05-22T21:19:46.407606+0000","last_became_peered":"2024-05-22T21:19:46.407606+0000","last_unstale":"2024-05-22T21:19:46.409367+0000","last_undegraded":"2024-05-22T21:19:46.409367+0000","last_fullsized":"2024-05-22T21:19:46.409367+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:02:04.279027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.403105+0000","last_change":"2024-05-22T21:19:46.403029+0000","last_active":"2024-05-22T21:19:46.403105+0000","last_peered":"2024-05-22T21:19:46.403105+0000","last_clean":"2024-05-22T21:19:46.403105+0000","last_became_active":"2024-05-22T21:19:46.402789+0000","last_became_peered":"2024-05-22T21:19:46.402789+0000","last_unstale":"2024-05-22T21:19:46.403105+0000","last_undegraded":"2024-05-22T21:19:46.403105+0000","last_fullsized":"2024-05-22T21:19:46.403105+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:01:55.246784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.401966+0000","last_change":"2024-05-22T21:19:46.401860+0000","last_active":"2024-05-22T21:19:46.401966+0000","last_peered":"2024-05-22T21:19:46.401966+0000","last_clean":"2024-05-22T21:19:46.401966+0000","last_became_active":"2024-05-22T21:19:46.401612+0000","last_became_peered":"2024-05-22T21:19:46.401612+0000","last_unstale":"2024-05-22T21:19:46.401966+0000","last_undegraded":"2024-05-22T21:19:46.401966+0000","last_fullsized":"2024-05-22T21:19:46.401966+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:55:58.881775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405527+0000","last_change":"2024-05-22T21:19:46.400487+0000","last_active":"2024-05-22T21:19:46.405527+0000","last_peered":"2024-05-22T21:19:46.405527+0000","last_clean":"2024-05-22T21:19:46.405527+0000","last_became_active":"2024-05-22T21:19:46.399902+0000","last_became_peered":"2024-05-22T21:19:46.399902+0000","last_unstale":"2024-05-22T21:19:46.405527+0000","last_undegraded":"2024-05-22T21:19:46.405527+0000","last_fullsized":"2024-05-22T21:19:46.405527+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:45:25.590631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.399482+0000","last_change":"2024-05-22T21:19:46.399401+0000","last_active":"2024-05-22T21:19:46.399482+0000","last_peered":"2024-05-22T21:19:46.399482+0000","last_clean":"2024-05-22T21:19:46.399482+0000","last_became_active":"2024-05-22T21:19:46.398518+0000","last_became_peered":"2024-05-22T21:19:46.398518+0000","last_unstale":"2024-05-22T21:19:46.399482+0000","last_undegraded":"2024-05-22T21:19:46.399482+0000","last_fullsized":"2024-05-22T21:19:46.399482+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:56:53.774354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.402859+0000","last_change":"2024-05-22T21:19:46.401867+0000","last_active":"2024-05-22T21:19:46.402859+0000","last_peered":"2024-05-22T21:19:46.402859+0000","last_clean":"2024-05-22T21:19:46.402859+0000","last_became_active":"2024-05-22T21:19:46.401655+0000","last_became_peered":"2024-05-22T21:19:46.401655+0000","last_unstale":"2024-05-22T21:19:46.402859+0000","last_undegraded":"2024-05-22T21:19:46.402859+0000","last_fullsized":"2024-05-22T21:19:46.402859+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:18:46.668052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"139'4","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.410934+0000","last_change":"2024-05-22T21:19:46.015891+0000","last_active":"2024-05-22T21:19:46.410934+0000","last_peered":"2024-05-22T21:19:46.410934+0000","last_clean":"2024-05-22T21:19:46.410934+0000","last_became_active":"2024-05-22T21:19:46.013962+0000","last_became_peered":"2024-05-22T21:19:46.013962+0000","last_unstale":"2024-05-22T21:19:46.410934+0000","last_undegraded":"2024-05-22T21:19:46.410934+0000","last_fullsized":"2024-05-22T21:19:46.410934+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:04.172719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.403142+0000","last_change":"2024-05-22T21:19:46.402685+0000","last_active":"2024-05-22T21:19:46.403142+0000","last_peered":"2024-05-22T21:19:46.403142+0000","last_clean":"2024-05-22T21:19:46.403142+0000","last_became_active":"2024-05-22T21:19:46.402436+0000","last_became_peered":"2024-05-22T21:19:46.402436+0000","last_unstale":"2024-05-22T21:19:46.403142+0000","last_undegraded":"2024-05-22T21:19:46.403142+0000","last_fullsized":"2024-05-22T21:19:46.403142+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:44:41.425002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.408116+0000","last_change":"2024-05-22T21:19:46.016005+0000","last_active":"2024-05-22T21:19:46.408116+0000","last_peered":"2024-05-22T21:19:46.408116+0000","last_clean":"2024-05-22T21:19:46.408116+0000","last_became_active":"2024-05-22T21:19:46.015786+0000","last_became_peered":"2024-05-22T21:19:46.015786+0000","last_unstale":"2024-05-22T21:19:46.408116+0000","last_undegraded":"2024-05-22T21:19:46.408116+0000","last_fullsized":"2024-05-22T21:19:46.408116+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:50:21.018391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.401323+0000","last_change":"2024-05-22T21:19:46.401237+0000","last_active":"2024-05-22T21:19:46.401323+0000","last_peered":"2024-05-22T21:19:46.401323+0000","last_clean":"2024-05-22T21:19:46.401323+0000","last_became_active":"2024-05-22T21:19:46.401015+0000","last_became_peered":"2024-05-22T21:19:46.401015+0000","last_unstale":"2024-05-22T21:19:46.401323+0000","last_undegraded":"2024-05-22T21:19:46.401323+0000","last_fullsized":"2024-05-22T21:19:46.401323+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:10:10.164548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.416812+0000","last_change":"2024-05-22T21:19:46.019292+0000","last_active":"2024-05-22T21:19:46.416812+0000","last_peered":"2024-05-22T21:19:46.416812+0000","last_clean":"2024-05-22T21:19:46.416812+0000","last_became_active":"2024-05-22T21:19:46.019044+0000","last_became_peered":"2024-05-22T21:19:46.019044+0000","last_unstale":"2024-05-22T21:19:46.416812+0000","last_undegraded":"2024-05-22T21:19:46.416812+0000","last_fullsized":"2024-05-22T21:19:46.416812+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:05:11.899061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405506+0000","last_change":"2024-05-22T21:19:46.405146+0000","last_active":"2024-05-22T21:19:46.405506+0000","last_peered":"2024-05-22T21:19:46.405506+0000","last_clean":"2024-05-22T21:19:46.405506+0000","last_became_active":"2024-05-22T21:19:46.404788+0000","last_became_peered":"2024-05-22T21:19:46.404788+0000","last_unstale":"2024-05-22T21:19:46.405506+0000","last_undegraded":"2024-05-22T21:19:46.405506+0000","last_fullsized":"2024-05-22T21:19:46.405506+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:03:28.907582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.416577+0000","last_change":"2024-05-22T21:19:46.013623+0000","last_active":"2024-05-22T21:19:46.416577+0000","last_peered":"2024-05-22T21:19:46.416577+0000","last_clean":"2024-05-22T21:19:46.416577+0000","last_became_active":"2024-05-22T21:19:46.013151+0000","last_became_peered":"2024-05-22T21:19:46.013151+0000","last_unstale":"2024-05-22T21:19:46.416577+0000","last_undegraded":"2024-05-22T21:19:46.416577+0000","last_fullsized":"2024-05-22T21:19:46.416577+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:50:09.899680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.408950+0000","last_change":"2024-05-22T21:19:46.408699+0000","last_active":"2024-05-22T21:19:46.408950+0000","last_peered":"2024-05-22T21:19:46.408950+0000","last_clean":"2024-05-22T21:19:46.408950+0000","last_became_active":"2024-05-22T21:19:46.408351+0000","last_became_peered":"2024-05-22T21:19:46.408351+0000","last_unstale":"2024-05-22T21:19:46.408950+0000","last_undegraded":"2024-05-22T21:19:46.408950+0000","last_fullsized":"2024-05-22T21:19:46.408950+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:10:06.354001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.414520+0000","last_change":"2024-05-22T21:19:46.021062+0000","last_active":"2024-05-22T21:19:46.414520+0000","last_peered":"2024-05-22T21:19:46.414520+0000","last_clean":"2024-05-22T21:19:46.414520+0000","last_became_active":"2024-05-22T21:19:46.020815+0000","last_became_peered":"2024-05-22T21:19:46.020815+0000","last_unstale":"2024-05-22T21:19:46.414520+0000","last_undegraded":"2024-05-22T21:19:46.414520+0000","last_fullsized":"2024-05-22T21:19:46.414520+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:56:52.764255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.402560+0000","last_change":"2024-05-22T21:19:46.402418+0000","last_active":"2024-05-22T21:19:46.402560+0000","last_peered":"2024-05-22T21:19:46.402560+0000","last_clean":"2024-05-22T21:19:46.402560+0000","last_became_active":"2024-05-22T21:19:46.401256+0000","last_became_peered":"2024-05-22T21:19:46.401256+0000","last_unstale":"2024-05-22T21:19:46.402560+0000","last_undegraded":"2024-05-22T21:19:46.402560+0000","last_fullsized":"2024-05-22T21:19:46.402560+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:09:42.299345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.412230+0000","last_change":"2024-05-22T21:19:46.018396+0000","last_active":"2024-05-22T21:19:46.412230+0000","last_peered":"2024-05-22T21:19:46.412230+0000","last_clean":"2024-05-22T21:19:46.412230+0000","last_became_active":"2024-05-22T21:19:46.018137+0000","last_became_peered":"2024-05-22T21:19:46.018137+0000","last_unstale":"2024-05-22T21:19:46.412230+0000","last_undegraded":"2024-05-22T21:19:46.412230+0000","last_fullsized":"2024-05-22T21:19:46.412230+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:50:21.018391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.410546+0000","last_change":"2024-05-22T21:19:46.410110+0000","last_active":"2024-05-22T21:19:46.410546+0000","last_peered":"2024-05-22T21:19:46.410546+0000","last_clean":"2024-05-22T21:19:46.410546+0000","last_became_active":"2024-05-22T21:19:46.409710+0000","last_became_peered":"2024-05-22T21:19:46.409710+0000","last_unstale":"2024-05-22T21:19:46.410546+0000","last_undegraded":"2024-05-22T21:19:46.410546+0000","last_fullsized":"2024-05-22T21:19:46.410546+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:29:28.307781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.395978+0000","last_change":"2024-05-22T21:19:46.011421+0000","last_active":"2024-05-22T21:19:46.395978+0000","last_peered":"2024-05-22T21:19:46.395978+0000","last_clean":"2024-05-22T21:19:46.395978+0000","last_became_active":"2024-05-22T21:19:46.011177+0000","last_became_peered":"2024-05-22T21:19:46.011177+0000","last_unstale":"2024-05-22T21:19:46.395978+0000","last_undegraded":"2024-05-22T21:19:46.395978+0000","last_fullsized":"2024-05-22T21:19:46.395978+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:13:36.319481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.400212+0000","last_change":"2024-05-22T21:19:46.400138+0000","last_active":"2024-05-22T21:19:46.400212+0000","last_peered":"2024-05-22T21:19:46.400212+0000","last_clean":"2024-05-22T21:19:46.400212+0000","last_became_active":"2024-05-22T21:19:46.399919+0000","last_became_peered":"2024-05-22T21:19:46.399919+0000","last_unstale":"2024-05-22T21:19:46.400212+0000","last_undegraded":"2024-05-22T21:19:46.400212+0000","last_fullsized":"2024-05-22T21:19:46.400212+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:13:42.580122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.398553+0000","last_change":"2024-05-22T21:19:46.010930+0000","last_active":"2024-05-22T21:19:46.398553+0000","last_peered":"2024-05-22T21:19:46.398553+0000","last_clean":"2024-05-22T21:19:46.398553+0000","last_became_active":"2024-05-22T21:19:46.010703+0000","last_became_peered":"2024-05-22T21:19:46.010703+0000","last_unstale":"2024-05-22T21:19:46.398553+0000","last_undegraded":"2024-05-22T21:19:46.398553+0000","last_fullsized":"2024-05-22T21:19:46.398553+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:57:58.350121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.398967+0000","last_change":"2024-05-22T21:19:46.398748+0000","last_active":"2024-05-22T21:19:46.398967+0000","last_peered":"2024-05-22T21:19:46.398967+0000","last_clean":"2024-05-22T21:19:46.398967+0000","last_became_active":"2024-05-22T21:19:46.398101+0000","last_became_peered":"2024-05-22T21:19:46.398101+0000","last_unstale":"2024-05-22T21:19:46.398967+0000","last_undegraded":"2024-05-22T21:19:46.398967+0000","last_fullsized":"2024-05-22T21:19:46.398967+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:13:37.329580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.393824+0000","last_change":"2024-05-22T21:19:46.014681+0000","last_active":"2024-05-22T21:19:46.393824+0000","last_peered":"2024-05-22T21:19:46.393824+0000","last_clean":"2024-05-22T21:19:46.393824+0000","last_became_active":"2024-05-22T21:19:46.013416+0000","last_became_peered":"2024-05-22T21:19:46.013416+0000","last_unstale":"2024-05-22T21:19:46.393824+0000","last_undegraded":"2024-05-22T21:19:46.393824+0000","last_fullsized":"2024-05-22T21:19:46.393824+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:10:07.010122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.415278+0000","last_change":"2024-05-22T21:19:46.415207+0000","last_active":"2024-05-22T21:19:46.415278+0000","last_peered":"2024-05-22T21:19:46.415278+0000","last_clean":"2024-05-22T21:19:46.415278+0000","last_became_active":"2024-05-22T21:19:46.414932+0000","last_became_peered":"2024-05-22T21:19:46.414932+0000","last_unstale":"2024-05-22T21:19:46.415278+0000","last_undegraded":"2024-05-22T21:19:46.415278+0000","last_fullsized":"2024-05-22T21:19:46.415278+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:09:42.299345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.389585+0000","last_change":"2024-05-22T21:19:46.012649+0000","last_active":"2024-05-22T21:19:46.389585+0000","last_peered":"2024-05-22T21:19:46.389585+0000","last_clean":"2024-05-22T21:19:46.389585+0000","last_became_active":"2024-05-22T21:19:46.012432+0000","last_became_peered":"2024-05-22T21:19:46.012432+0000","last_unstale":"2024-05-22T21:19:46.389585+0000","last_undegraded":"2024-05-22T21:19:46.389585+0000","last_fullsized":"2024-05-22T21:19:46.389585+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:12:01.288530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.401674+0000","last_change":"2024-05-22T21:19:46.400814+0000","last_active":"2024-05-22T21:19:46.401674+0000","last_peered":"2024-05-22T21:19:46.401674+0000","last_clean":"2024-05-22T21:19:46.401674+0000","last_became_active":"2024-05-22T21:19:46.400486+0000","last_became_peered":"2024-05-22T21:19:46.400486+0000","last_unstale":"2024-05-22T21:19:46.401674+0000","last_undegraded":"2024-05-22T21:19:46.401674+0000","last_fullsized":"2024-05-22T21:19:46.401674+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:15:17.111373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"139'3","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.416244+0000","last_change":"2024-05-22T21:19:46.009675+0000","last_active":"2024-05-22T21:19:46.416244+0000","last_peered":"2024-05-22T21:19:46.416244+0000","last_clean":"2024-05-22T21:19:46.416244+0000","last_became_active":"2024-05-22T21:19:46.009358+0000","last_became_peered":"2024-05-22T21:19:46.009358+0000","last_unstale":"2024-05-22T21:19:46.416244+0000","last_undegraded":"2024-05-22T21:19:46.416244+0000","last_fullsized":"2024-05-22T21:19:46.416244+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:44:40.414903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.397797+0000","last_change":"2024-05-22T21:19:46.397715+0000","last_active":"2024-05-22T21:19:46.397797+0000","last_peered":"2024-05-22T21:19:46.397797+0000","last_clean":"2024-05-22T21:19:46.397797+0000","last_became_active":"2024-05-22T21:19:46.396926+0000","last_became_peered":"2024-05-22T21:19:46.396926+0000","last_unstale":"2024-05-22T21:19:46.397797+0000","last_undegraded":"2024-05-22T21:19:46.397797+0000","last_fullsized":"2024-05-22T21:19:46.397797+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:32:36.123360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"139'6","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.417416+0000","last_change":"2024-05-22T21:19:46.011208+0000","last_active":"2024-05-22T21:19:46.417416+0000","last_peered":"2024-05-22T21:19:46.417416+0000","last_clean":"2024-05-22T21:19:46.417416+0000","last_became_active":"2024-05-22T21:19:46.010687+0000","last_became_peered":"2024-05-22T21:19:46.010687+0000","last_unstale":"2024-05-22T21:19:46.417416+0000","last_undegraded":"2024-05-22T21:19:46.417416+0000","last_fullsized":"2024-05-22T21:19:46.417416+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:53:06.054493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.394950+0000","last_change":"2024-05-22T21:19:46.394831+0000","last_active":"2024-05-22T21:19:46.394950+0000","last_peered":"2024-05-22T21:19:46.394950+0000","last_clean":"2024-05-22T21:19:46.394950+0000","last_became_active":"2024-05-22T21:19:46.393914+0000","last_became_peered":"2024-05-22T21:19:46.393914+0000","last_unstale":"2024-05-22T21:19:46.394950+0000","last_undegraded":"2024-05-22T21:19:46.394950+0000","last_fullsized":"2024-05-22T21:19:46.394950+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:03:28.907582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.410476+0000","last_change":"2024-05-22T21:19:46.005658+0000","last_active":"2024-05-22T21:19:46.410476+0000","last_peered":"2024-05-22T21:19:46.410476+0000","last_clean":"2024-05-22T21:19:46.410476+0000","last_became_active":"2024-05-22T21:19:46.005401+0000","last_became_peered":"2024-05-22T21:19:46.005401+0000","last_unstale":"2024-05-22T21:19:46.410476+0000","last_undegraded":"2024-05-22T21:19:46.410476+0000","last_fullsized":"2024-05-22T21:19:46.410476+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:39:12.716501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.399670+0000","last_change":"2024-05-22T21:19:46.399457+0000","last_active":"2024-05-22T21:19:46.399670+0000","last_peered":"2024-05-22T21:19:46.399670+0000","last_clean":"2024-05-22T21:19:46.399670+0000","last_became_active":"2024-05-22T21:19:46.398507+0000","last_became_peered":"2024-05-22T21:19:46.398507+0000","last_unstale":"2024-05-22T21:19:46.399670+0000","last_undegraded":"2024-05-22T21:19:46.399670+0000","last_fullsized":"2024-05-22T21:19:46.399670+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:21:25.780042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.396253+0000","last_change":"2024-05-22T21:19:46.015382+0000","last_active":"2024-05-22T21:19:46.396253+0000","last_peered":"2024-05-22T21:19:46.396253+0000","last_clean":"2024-05-22T21:19:46.396253+0000","last_became_active":"2024-05-22T21:19:46.014728+0000","last_became_peered":"2024-05-22T21:19:46.014728+0000","last_unstale":"2024-05-22T21:19:46.396253+0000","last_undegraded":"2024-05-22T21:19:46.396253+0000","last_fullsized":"2024-05-22T21:19:46.396253+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:50:21.018391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.403571+0000","last_change":"2024-05-22T21:19:46.403402+0000","last_active":"2024-05-22T21:19:46.403571+0000","last_peered":"2024-05-22T21:19:46.403571+0000","last_clean":"2024-05-22T21:19:46.403571+0000","last_became_active":"2024-05-22T21:19:46.403191+0000","last_became_peered":"2024-05-22T21:19:46.403191+0000","last_unstale":"2024-05-22T21:19:46.403571+0000","last_undegraded":"2024-05-22T21:19:46.403571+0000","last_fullsized":"2024-05-22T21:19:46.403571+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:49:14.153916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"139'8","reported_seq":18,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.417366+0000","last_change":"2024-05-22T21:19:46.011669+0000","last_active":"2024-05-22T21:19:46.417366+0000","last_peered":"2024-05-22T21:19:46.417366+0000","last_clean":"2024-05-22T21:19:46.417366+0000","last_became_active":"2024-05-22T21:19:46.011459+0000","last_became_peered":"2024-05-22T21:19:46.011459+0000","last_unstale":"2024-05-22T21:19:46.417366+0000","last_undegraded":"2024-05-22T21:19:46.417366+0000","last_fullsized":"2024-05-22T21:19:46.417366+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:56:22.229755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.414864+0000","last_change":"2024-05-22T21:19:46.413572+0000","last_active":"2024-05-22T21:19:46.414864+0000","last_peered":"2024-05-22T21:19:46.414864+0000","last_clean":"2024-05-22T21:19:46.414864+0000","last_became_active":"2024-05-22T21:19:46.413349+0000","last_became_peered":"2024-05-22T21:19:46.413349+0000","last_unstale":"2024-05-22T21:19:46.414864+0000","last_undegraded":"2024-05-22T21:19:46.414864+0000","last_fullsized":"2024-05-22T21:19:46.414864+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:28:07.156675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.415086+0000","last_change":"2024-05-22T21:19:46.015585+0000","last_active":"2024-05-22T21:19:46.415086+0000","last_peered":"2024-05-22T21:19:46.415086+0000","last_clean":"2024-05-22T21:19:46.415086+0000","last_became_active":"2024-05-22T21:19:46.015375+0000","last_became_peered":"2024-05-22T21:19:46.015375+0000","last_unstale":"2024-05-22T21:19:46.415086+0000","last_undegraded":"2024-05-22T21:19:46.415086+0000","last_fullsized":"2024-05-22T21:19:46.415086+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:13:00.260310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405198+0000","last_change":"2024-05-22T21:19:46.404927+0000","last_active":"2024-05-22T21:19:46.405198+0000","last_peered":"2024-05-22T21:19:46.405198+0000","last_clean":"2024-05-22T21:19:46.405198+0000","last_became_active":"2024-05-22T21:19:46.404688+0000","last_became_peered":"2024-05-22T21:19:46.404688+0000","last_unstale":"2024-05-22T21:19:46.405198+0000","last_undegraded":"2024-05-22T21:19:46.405198+0000","last_fullsized":"2024-05-22T21:19:46.405198+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:55:46.617557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.416699+0000","last_change":"2024-05-22T21:19:46.013870+0000","last_active":"2024-05-22T21:19:46.416699+0000","last_peered":"2024-05-22T21:19:46.416699+0000","last_clean":"2024-05-22T21:19:46.416699+0000","last_became_active":"2024-05-22T21:19:46.012435+0000","last_became_peered":"2024-05-22T21:19:46.012435+0000","last_unstale":"2024-05-22T21:19:46.416699+0000","last_undegraded":"2024-05-22T21:19:46.416699+0000","last_fullsized":"2024-05-22T21:19:46.416699+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:19:50.238691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.403359+0000","last_change":"2024-05-22T21:19:46.403283+0000","last_active":"2024-05-22T21:19:46.403359+0000","last_peered":"2024-05-22T21:19:46.403359+0000","last_clean":"2024-05-22T21:19:46.403359+0000","last_became_active":"2024-05-22T21:19:46.403035+0000","last_became_peered":"2024-05-22T21:19:46.403035+0000","last_unstale":"2024-05-22T21:19:46.403359+0000","last_undegraded":"2024-05-22T21:19:46.403359+0000","last_fullsized":"2024-05-22T21:19:46.403359+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:57:59.360221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.410630+0000","last_change":"2024-05-22T21:19:46.016281+0000","last_active":"2024-05-22T21:19:46.410630+0000","last_peered":"2024-05-22T21:19:46.410630+0000","last_clean":"2024-05-22T21:19:46.410630+0000","last_became_active":"2024-05-22T21:19:46.016065+0000","last_became_peered":"2024-05-22T21:19:46.016065+0000","last_unstale":"2024-05-22T21:19:46.410630+0000","last_undegraded":"2024-05-22T21:19:46.410630+0000","last_fullsized":"2024-05-22T21:19:46.410630+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:12:01.288530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.395937+0000","last_change":"2024-05-22T21:19:46.394871+0000","last_active":"2024-05-22T21:19:46.395937+0000","last_peered":"2024-05-22T21:19:46.395937+0000","last_clean":"2024-05-22T21:19:46.395937+0000","last_became_active":"2024-05-22T21:19:46.394643+0000","last_became_peered":"2024-05-22T21:19:46.394643+0000","last_unstale":"2024-05-22T21:19:46.395937+0000","last_undegraded":"2024-05-22T21:19:46.395937+0000","last_fullsized":"2024-05-22T21:19:46.395937+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:06:39.567925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.392872+0000","last_change":"2024-05-22T21:19:46.017189+0000","last_active":"2024-05-22T21:19:46.392872+0000","last_peered":"2024-05-22T21:19:46.392872+0000","last_clean":"2024-05-22T21:19:46.392872+0000","last_became_active":"2024-05-22T21:19:46.016727+0000","last_became_peered":"2024-05-22T21:19:46.016727+0000","last_unstale":"2024-05-22T21:19:46.392872+0000","last_undegraded":"2024-05-22T21:19:46.392872+0000","last_fullsized":"2024-05-22T21:19:46.392872+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:10:09.154449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405447+0000","last_change":"2024-05-22T21:19:46.403704+0000","last_active":"2024-05-22T21:19:46.405447+0000","last_peered":"2024-05-22T21:19:46.405447+0000","last_clean":"2024-05-22T21:19:46.405447+0000","last_became_active":"2024-05-22T21:19:46.403331+0000","last_became_peered":"2024-05-22T21:19:46.403331+0000","last_unstale":"2024-05-22T21:19:46.405447+0000","last_undegraded":"2024-05-22T21:19:46.405447+0000","last_fullsized":"2024-05-22T21:19:46.405447+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:49:14.153916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.415580+0000","last_change":"2024-05-22T21:19:46.013618+0000","last_active":"2024-05-22T21:19:46.415580+0000","last_peered":"2024-05-22T21:19:46.415580+0000","last_clean":"2024-05-22T21:19:46.415580+0000","last_became_active":"2024-05-22T21:19:46.013397+0000","last_became_peered":"2024-05-22T21:19:46.013397+0000","last_unstale":"2024-05-22T21:19:46.415580+0000","last_undegraded":"2024-05-22T21:19:46.415580+0000","last_fullsized":"2024-05-22T21:19:46.415580+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:09:21.911521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.400051+0000","last_change":"2024-05-22T21:19:46.399878+0000","last_active":"2024-05-22T21:19:46.400051+0000","last_peered":"2024-05-22T21:19:46.400051+0000","last_clean":"2024-05-22T21:19:46.400051+0000","last_became_active":"2024-05-22T21:19:46.399633+0000","last_became_peered":"2024-05-22T21:19:46.399633+0000","last_unstale":"2024-05-22T21:19:46.400051+0000","last_undegraded":"2024-05-22T21:19:46.400051+0000","last_fullsized":"2024-05-22T21:19:46.400051+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:00.041735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.392188+0000","last_change":"2024-05-22T21:19:46.017660+0000","last_active":"2024-05-22T21:19:46.392188+0000","last_peered":"2024-05-22T21:19:46.392188+0000","last_clean":"2024-05-22T21:19:46.392188+0000","last_became_active":"2024-05-22T21:19:46.017375+0000","last_became_peered":"2024-05-22T21:19:46.017375+0000","last_unstale":"2024-05-22T21:19:46.392188+0000","last_undegraded":"2024-05-22T21:19:46.392188+0000","last_fullsized":"2024-05-22T21:19:46.392188+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:29:53.472753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405463+0000","last_change":"2024-05-22T21:19:46.404485+0000","last_active":"2024-05-22T21:19:46.405463+0000","last_peered":"2024-05-22T21:19:46.405463+0000","last_clean":"2024-05-22T21:19:46.405463+0000","last_became_active":"2024-05-22T21:19:46.404230+0000","last_became_peered":"2024-05-22T21:19:46.404230+0000","last_unstale":"2024-05-22T21:19:46.405463+0000","last_undegraded":"2024-05-22T21:19:46.405463+0000","last_fullsized":"2024-05-22T21:19:46.405463+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:26:38.251056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.395178+0000","last_change":"2024-05-22T21:19:46.016840+0000","last_active":"2024-05-22T21:19:46.395178+0000","last_peered":"2024-05-22T21:19:46.395178+0000","last_clean":"2024-05-22T21:19:46.395178+0000","last_became_active":"2024-05-22T21:19:46.016441+0000","last_became_peered":"2024-05-22T21:19:46.016441+0000","last_unstale":"2024-05-22T21:19:46.395178+0000","last_undegraded":"2024-05-22T21:19:46.395178+0000","last_fullsized":"2024-05-22T21:19:46.395178+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:28:06.146576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.409191+0000","last_change":"2024-05-22T21:19:46.407277+0000","last_active":"2024-05-22T21:19:46.409191+0000","last_peered":"2024-05-22T21:19:46.409191+0000","last_clean":"2024-05-22T21:19:46.409191+0000","last_became_active":"2024-05-22T21:19:46.407066+0000","last_became_peered":"2024-05-22T21:19:46.407066+0000","last_unstale":"2024-05-22T21:19:46.409191+0000","last_undegraded":"2024-05-22T21:19:46.409191+0000","last_fullsized":"2024-05-22T21:19:46.409191+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:09:13.693246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.395780+0000","last_change":"2024-05-22T21:19:46.014817+0000","last_active":"2024-05-22T21:19:46.395780+0000","last_peered":"2024-05-22T21:19:46.395780+0000","last_clean":"2024-05-22T21:19:46.395780+0000","last_became_active":"2024-05-22T21:19:46.014492+0000","last_became_peered":"2024-05-22T21:19:46.014492+0000","last_unstale":"2024-05-22T21:19:46.395780+0000","last_undegraded":"2024-05-22T21:19:46.395780+0000","last_fullsized":"2024-05-22T21:19:46.395780+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:33:06.233516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.412241+0000","last_change":"2024-05-22T21:19:46.412178+0000","last_active":"2024-05-22T21:19:46.412241+0000","last_peered":"2024-05-22T21:19:46.412241+0000","last_clean":"2024-05-22T21:19:46.412241+0000","last_became_active":"2024-05-22T21:19:46.411925+0000","last_became_peered":"2024-05-22T21:19:46.411925+0000","last_unstale":"2024-05-22T21:19:46.412241+0000","last_undegraded":"2024-05-22T21:19:46.412241+0000","last_fullsized":"2024-05-22T21:19:46.412241+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:22:15.297795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.415261+0000","last_change":"2024-05-22T21:19:46.013294+0000","last_active":"2024-05-22T21:19:46.415261+0000","last_peered":"2024-05-22T21:19:46.415261+0000","last_clean":"2024-05-22T21:19:46.415261+0000","last_became_active":"2024-05-22T21:19:46.013068+0000","last_became_peered":"2024-05-22T21:19:46.013068+0000","last_unstale":"2024-05-22T21:19:46.415261+0000","last_undegraded":"2024-05-22T21:19:46.415261+0000","last_fullsized":"2024-05-22T21:19:46.415261+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:15:09.051559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.408836+0000","last_change":"2024-05-22T21:19:46.406947+0000","last_active":"2024-05-22T21:19:46.408836+0000","last_peered":"2024-05-22T21:19:46.408836+0000","last_clean":"2024-05-22T21:19:46.408836+0000","last_became_active":"2024-05-22T21:19:46.406710+0000","last_became_peered":"2024-05-22T21:19:46.406710+0000","last_unstale":"2024-05-22T21:19:46.408836+0000","last_undegraded":"2024-05-22T21:19:46.408836+0000","last_fullsized":"2024-05-22T21:19:46.408836+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:37:02.707159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.395962+0000","last_change":"2024-05-22T21:19:46.015640+0000","last_active":"2024-05-22T21:19:46.395962+0000","last_peered":"2024-05-22T21:19:46.395962+0000","last_clean":"2024-05-22T21:19:46.395962+0000","last_became_active":"2024-05-22T21:19:46.015404+0000","last_became_peered":"2024-05-22T21:19:46.015404+0000","last_unstale":"2024-05-22T21:19:46.395962+0000","last_undegraded":"2024-05-22T21:19:46.395962+0000","last_fullsized":"2024-05-22T21:19:46.395962+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:03:38.619514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.410529+0000","last_change":"2024-05-22T21:19:46.409946+0000","last_active":"2024-05-22T21:19:46.410529+0000","last_peered":"2024-05-22T21:19:46.410529+0000","last_clean":"2024-05-22T21:19:46.410529+0000","last_became_active":"2024-05-22T21:19:46.409353+0000","last_became_peered":"2024-05-22T21:19:46.409353+0000","last_unstale":"2024-05-22T21:19:46.410529+0000","last_undegraded":"2024-05-22T21:19:46.410529+0000","last_fullsized":"2024-05-22T21:19:46.410529+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:41:15.705579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.393933+0000","last_change":"2024-05-22T21:19:46.015030+0000","last_active":"2024-05-22T21:19:46.393933+0000","last_peered":"2024-05-22T21:19:46.393933+0000","last_clean":"2024-05-22T21:19:46.393933+0000","last_became_active":"2024-05-22T21:19:46.014547+0000","last_became_peered":"2024-05-22T21:19:46.014547+0000","last_unstale":"2024-05-22T21:19:46.393933+0000","last_undegraded":"2024-05-22T21:19:46.393933+0000","last_fullsized":"2024-05-22T21:19:46.393933+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:22:14.287695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.413478+0000","last_change":"2024-05-22T21:19:46.413414+0000","last_active":"2024-05-22T21:19:46.413478+0000","last_peered":"2024-05-22T21:19:46.413478+0000","last_clean":"2024-05-22T21:19:46.413478+0000","last_became_active":"2024-05-22T21:19:46.413182+0000","last_became_peered":"2024-05-22T21:19:46.413182+0000","last_unstale":"2024-05-22T21:19:46.413478+0000","last_undegraded":"2024-05-22T21:19:46.413478+0000","last_fullsized":"2024-05-22T21:19:46.413478+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:34:06.573812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.389287+0000","last_change":"2024-05-22T21:19:46.018873+0000","last_active":"2024-05-22T21:19:46.389287+0000","last_peered":"2024-05-22T21:19:46.389287+0000","last_clean":"2024-05-22T21:19:46.389287+0000","last_became_active":"2024-05-22T21:19:46.018665+0000","last_became_peered":"2024-05-22T21:19:46.018665+0000","last_unstale":"2024-05-22T21:19:46.389287+0000","last_undegraded":"2024-05-22T21:19:46.389287+0000","last_fullsized":"2024-05-22T21:19:46.389287+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:54:22.384897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.401499+0000","last_change":"2024-05-22T21:19:46.400771+0000","last_active":"2024-05-22T21:19:46.401499+0000","last_peered":"2024-05-22T21:19:46.401499+0000","last_clean":"2024-05-22T21:19:46.401499+0000","last_became_active":"2024-05-22T21:19:46.400240+0000","last_became_peered":"2024-05-22T21:19:46.400240+0000","last_unstale":"2024-05-22T21:19:46.401499+0000","last_undegraded":"2024-05-22T21:19:46.401499+0000","last_fullsized":"2024-05-22T21:19:46.401499+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:34:06.573812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.392121+0000","last_change":"2024-05-22T21:19:46.014685+0000","last_active":"2024-05-22T21:19:46.392121+0000","last_peered":"2024-05-22T21:19:46.392121+0000","last_clean":"2024-05-22T21:19:46.392121+0000","last_became_active":"2024-05-22T21:19:46.012483+0000","last_became_peered":"2024-05-22T21:19:46.012483+0000","last_unstale":"2024-05-22T21:19:46.392121+0000","last_undegraded":"2024-05-22T21:19:46.392121+0000","last_fullsized":"2024-05-22T21:19:46.392121+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:29:12.856142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.405895+0000","last_change":"2024-05-22T21:19:46.405513+0000","last_active":"2024-05-22T21:19:46.405895+0000","last_peered":"2024-05-22T21:19:46.405895+0000","last_clean":"2024-05-22T21:19:46.405895+0000","last_became_active":"2024-05-22T21:19:46.404797+0000","last_became_peered":"2024-05-22T21:19:46.404797+0000","last_unstale":"2024-05-22T21:19:46.405895+0000","last_undegraded":"2024-05-22T21:19:46.405895+0000","last_fullsized":"2024-05-22T21:19:46.405895+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:50:22.028491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.399818+0000","last_change":"2024-05-22T21:19:46.010542+0000","last_active":"2024-05-22T21:19:46.399818+0000","last_peered":"2024-05-22T21:19:46.399818+0000","last_clean":"2024-05-22T21:19:46.399818+0000","last_became_active":"2024-05-22T21:19:46.009436+0000","last_became_peered":"2024-05-22T21:19:46.009436+0000","last_unstale":"2024-05-22T21:19:46.399818+0000","last_undegraded":"2024-05-22T21:19:46.399818+0000","last_fullsized":"2024-05-22T21:19:46.399818+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:55:57.871676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.403140+0000","last_change":"2024-05-22T21:19:46.403062+0000","last_active":"2024-05-22T21:19:46.403140+0000","last_peered":"2024-05-22T21:19:46.403140+0000","last_clean":"2024-05-22T21:19:46.403140+0000","last_became_active":"2024-05-22T21:19:46.400387+0000","last_became_peered":"2024-05-22T21:19:46.400387+0000","last_unstale":"2024-05-22T21:19:46.403140+0000","last_undegraded":"2024-05-22T21:19:46.403140+0000","last_fullsized":"2024-05-22T21:19:46.403140+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:02:04.279027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.393642+0000","last_change":"2024-05-22T21:19:46.014414+0000","last_active":"2024-05-22T21:19:46.393642+0000","last_peered":"2024-05-22T21:19:46.393642+0000","last_clean":"2024-05-22T21:19:46.393642+0000","last_became_active":"2024-05-22T21:19:46.014225+0000","last_became_peered":"2024-05-22T21:19:46.014225+0000","last_unstale":"2024-05-22T21:19:46.393642+0000","last_undegraded":"2024-05-22T21:19:46.393642+0000","last_fullsized":"2024-05-22T21:19:46.393642+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:32:32.488654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.408786+0000","last_change":"2024-05-22T21:19:46.405524+0000","last_active":"2024-05-22T21:19:46.408786+0000","last_peered":"2024-05-22T21:19:46.408786+0000","last_clean":"2024-05-22T21:19:46.408786+0000","last_became_active":"2024-05-22T21:19:46.405189+0000","last_became_peered":"2024-05-22T21:19:46.405189+0000","last_unstale":"2024-05-22T21:19:46.408786+0000","last_undegraded":"2024-05-22T21:19:46.408786+0000","last_fullsized":"2024-05-22T21:19:46.408786+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:27:54.460213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.394021+0000","last_change":"2024-05-22T21:19:46.015160+0000","last_active":"2024-05-22T21:19:46.394021+0000","last_peered":"2024-05-22T21:19:46.394021+0000","last_clean":"2024-05-22T21:19:46.394021+0000","last_became_active":"2024-05-22T21:19:46.014830+0000","last_became_peered":"2024-05-22T21:19:46.014830+0000","last_unstale":"2024-05-22T21:19:46.394021+0000","last_undegraded":"2024-05-22T21:19:46.394021+0000","last_fullsized":"2024-05-22T21:19:46.394021+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:09:41.289246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.393754+0000","last_change":"2024-05-22T21:19:46.393648+0000","last_active":"2024-05-22T21:19:46.393754+0000","last_peered":"2024-05-22T21:19:46.393754+0000","last_clean":"2024-05-22T21:19:46.393754+0000","last_became_active":"2024-05-22T21:19:46.393424+0000","last_became_peered":"2024-05-22T21:19:46.393424+0000","last_unstale":"2024-05-22T21:19:46.393754+0000","last_undegraded":"2024-05-22T21:19:46.393754+0000","last_fullsized":"2024-05-22T21:19:46.393754+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:45.998249+0000","last_clean_scrub_stamp":"2024-05-22T21:19:45.998249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:26:38.251056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.392791+0000","last_change":"2024-05-22T21:19:46.017323+0000","last_active":"2024-05-22T21:19:46.392791+0000","last_peered":"2024-05-22T21:19:46.392791+0000","last_clean":"2024-05-22T21:19:46.392791+0000","last_became_active":"2024-05-22T21:19:46.017045+0000","last_became_peered":"2024-05-22T21:19:46.017045+0000","last_unstale":"2024-05-22T21:19:46.392791+0000","last_undegraded":"2024-05-22T21:19:46.392791+0000","last_fullsized":"2024-05-22T21:19:46.392791+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:19:44.988150+0000","last_clean_scrub_stamp":"2024-05-22T21:19:44.988150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:18:45.657953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":285,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-05-22T21:19:46.006135+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:19:46.006135+0000","last_peered":"2024-05-22T21:19:46.006135+0000","last_clean":"2024-05-22T21:19:46.006135+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:19:46.006135+0000","last_undegraded":"2024-05-22T21:19:46.006135+0000","last_fullsized":"2024-05-22T21:19:46.006135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1329,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193983,"statfs":{"total":200000000,"available":198639038,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68500000000000005,"15min":0.748},"min":{"1min":0.435,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91100000000000003,"5min":0.998,"15min":2.0369999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91300000000000003,"5min":0.97699999999999998,"15min":1.889},"last":0.78300000000000003}]},{"osd":1,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71199999999999997,"15min":0.76500000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.41699999999999998},"max":{"1min":0.99099999999999999,"5min":1.0840000000000001,"15min":3.0920000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70599999999999996,"15min":0.75700000000000001},"min":{"1min":0.53700000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.877,"5min":0.92700000000000005,"15min":3.1499999999999999},"last":0.53500000000000003}]},{"osd":2,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.38800000000000001},"max":{"1min":0.90000000000000002,"5min":0.94999999999999996,"15min":1.923},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.748,"15min":0.72799999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.71599999999999997}]},{"osd":3,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71899999999999997,"15min":0.69799999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95899999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.74399999999999999,"15min":0.78800000000000003},"min":{"1min":0.47599999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.04,"5min":1.04,"15min":2.1389999999999998},"last":0.80600000000000005}]},{"osd":4,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.38200000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.93100000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90900000000000003,"5min":1.018,"15min":1.018},"last":0.627}]},{"osd":5,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.65100000000000002},"min":{"1min":0.438,"5min":0.371,"15min":0.371},"max":{"1min":0.88500000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.42899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95599999999999996,"5min":0.996,"15min":0.996},"last":0.89500000000000002}]},{"osd":6,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52000000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.68000000000000005,"15min":0.64600000000000002},"min":{"1min":0.42199999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.74199999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":739,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194573,"statfs":{"total":200000000,"available":199243072,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69599999999999995,"15min":0.69299999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0049999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70899999999999996,"15min":0.69099999999999995},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.39500000000000002},"max":{"1min":0.997,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.82699999999999996}]},{"osd":1,"last update":"Wed May 22 21:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70999999999999996,"15min":0.68400000000000005},"min":{"1min":0.55100000000000005,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.91700000000000004,"5min":1.107,"15min":1.107},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.53300000000000003,"5min":0.371,"15min":0.371},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.081},"last":0.746}]},{"osd":2,"last update":"Wed May 22 21:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78600000000000003,"15min":0.76000000000000001},"min":{"1min":0.52000000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.046,"5min":1.087,"15min":1.087},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.745,"15min":0.73899999999999999},"min":{"1min":0.55100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.873}]},{"osd":3,"last update":"Wed May 22 21:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.48199999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.415},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.72199999999999998,"15min":0.72599999999999998},"min":{"1min":0.54500000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91800000000000004,"5min":1.788,"15min":1.788},"last":0.69299999999999995}]},{"osd":4,"last update":"Wed May 22 21:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.45000000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.44400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.0309999999999999,"5min":2.718,"15min":2.718},"last":0.72599999999999998}]},{"osd":5,"last update":"Wed May 22 21:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.73499999999999999,"15min":0.69999999999999996},"min":{"1min":0.45300000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77600000000000002,"5min":8.5749999999999993,"15min":8.5749999999999993},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70199999999999996,"15min":0.71299999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98499999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.66900000000000004}]},{"osd":7,"last update":"Wed May 22 21:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.70599999999999996,"15min":0.73599999999999999},"min":{"1min":0.51600000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.71299999999999997,"15min":0.752},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.98099999999999998,"5min":1.2450000000000001,"15min":1.8009999999999999},"last":0.76100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":745,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194567,"statfs":{"total":200000000,"available":199236797,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.67100000000000004,"15min":0.69799999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.98699999999999999,"15min":1.0409999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.38400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.85899999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.73599999999999999}]},{"osd":1,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69299999999999995,"15min":0.70499999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.68899999999999995},"min":{"1min":0.46400000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.54100000000000004}]},{"osd":2,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73199999999999998,"15min":0.70399999999999996},"min":{"1min":0.53800000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70499999999999996,"15min":0.67900000000000005},"min":{"1min":0.432,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.48799999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.93400000000000005,"5min":0.97199999999999998,"15min":1.1879999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.111},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57299999999999995,"15min":0.621},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77600000000000002,"5min":0.92800000000000005,"15min":1.272},"last":0.433},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.623,"15min":0.63},"min":{"1min":0.433,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88100000000000001,"5min":1.038,"15min":1.038},"last":0.56699999999999995}]},{"osd":6,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63,"15min":0.628},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85699999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.45100000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.80300000000000005,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.58799999999999997}]},{"osd":7,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69599999999999995,"15min":0.71099999999999997},"min":{"1min":0.48999999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96299999999999997,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73499999999999999,"15min":0.78200000000000003},"min":{"1min":0.47399999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91200000000000003,"5min":1.008,"15min":1.2909999999999999},"last":0.70199999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":745,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194567,"statfs":{"total":200000000,"available":199236797,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.78300000000000003,"15min":0.748},"min":{"1min":0.47099999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.94199999999999995,"5min":6.6539999999999999,"15min":6.6539999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.748,"15min":0.72799999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98899999999999999,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":0.61199999999999999}]},{"osd":1,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65800000000000003,"15min":0.70799999999999996},"min":{"1min":0.41999999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86499999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.73099999999999998,"15min":0.70999999999999996},"min":{"1min":0.52300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":0.51100000000000001}]},{"osd":2,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.72999999999999998,"15min":0.69799999999999995},"min":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.4660000000000002,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.71599999999999997,"15min":0.70899999999999996},"min":{"1min":0.53300000000000003,"5min":0.378,"15min":0.378},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.70199999999999996,"15min":0.69499999999999995},"min":{"1min":0.49099999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.77400000000000002,"15min":0.79200000000000004},"min":{"1min":0.629,"5min":0.51500000000000001,"15min":0.47499999999999998},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":0.63400000000000001}]},{"osd":5,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.63900000000000001},"min":{"1min":0.39200000000000002,"5min":0.38,"15min":0.379},"max":{"1min":0.90000000000000002,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.439},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0409999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.52600000000000002}]},{"osd":6,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64800000000000002,"15min":0.68899999999999995},"min":{"1min":0.46600000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.434},"max":{"1min":0.92700000000000005,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.67400000000000004}]},{"osd":7,"last update":"Wed May 22 21:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.44700000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2150000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72699999999999998,"15min":0.77500000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.4510000000000001},"last":0.873}]}]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":739,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194573,"statfs":{"total":200000000,"available":199243072,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.436,"15min":0.40899999999999997},"min":{"1min":0.25700000000000001,"5min":0.245,"15min":0.245},"max":{"1min":0.64700000000000002,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.45100000000000001,"15min":0.41099999999999998},"min":{"1min":0.314,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.58399999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.32800000000000001}]},{"osd":1,"last update":"Wed May 22 21:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.46800000000000003,"15min":0.45000000000000001},"min":{"1min":0.33300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.625,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53600000000000003,"15min":0.51000000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33800000000000002},"max":{"1min":0.70799999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53200000000000003}]},{"osd":2,"last update":"Wed May 22 21:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.50900000000000001,"15min":0.495},"min":{"1min":0.35999999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69099999999999995,"5min":0.70699999999999996,"15min":0.86799999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51200000000000001,"15min":0.48999999999999999},"min":{"1min":0.34399999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.75600000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56899999999999995}]},{"osd":4,"last update":"Wed May 22 21:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.73799999999999999,"15min":0.70499999999999996},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89900000000000002,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.749,"15min":0.71899999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.435},"max":{"1min":1,"5min":1.397,"15min":1.5760000000000001},"last":0.75600000000000001}]},{"osd":5,"last update":"Wed May 22 21:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76400000000000001,"15min":0.70099999999999996},"min":{"1min":0.53700000000000003,"5min":0.48099999999999998,"15min":0.45100000000000001},"max":{"1min":1.048,"5min":1.234,"15min":1.234},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.751,"15min":0.70399999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.93799999999999994,"5min":1.292,"15min":1.3049999999999999},"last":0.80400000000000005}]},{"osd":6,"last update":"Wed May 22 21:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.75,"15min":0.71699999999999997},"min":{"1min":0.54300000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92100000000000004,"5min":1.276,"15min":1.276},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76300000000000001,"15min":0.70899999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0820000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.68600000000000005}]},{"osd":7,"last update":"Wed May 22 21:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.79300000000000004,"15min":0.75700000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.495},"max":{"1min":1.0309999999999999,"5min":1.1000000000000001,"15min":1.2709999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.79000000000000004,"15min":0.74199999999999999},"min":{"1min":0.55600000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.071,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.79700000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":745,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194567,"statfs":{"total":200000000,"available":199236797,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.44400000000000001,"15min":0.40999999999999998},"min":{"1min":0.253,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.48199999999999998,"15min":0.44900000000000001},"min":{"1min":0.26800000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.38300000000000001}]},{"osd":1,"last update":"Wed May 22 21:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.47899999999999998,"15min":0.45400000000000001},"min":{"1min":0.35699999999999998,"5min":0.316,"15min":0.29399999999999998},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.56899999999999995,"15min":0.53000000000000003},"min":{"1min":0.44900000000000001,"5min":0.41199999999999998,"15min":0.39100000000000001},"max":{"1min":2.8159999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.496}]},{"osd":3,"last update":"Wed May 22 21:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53000000000000003,"15min":0.50800000000000001},"min":{"1min":0.39600000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.75,"5min":2.75,"15min":2.75},"last":0.436},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.51500000000000001,"15min":0.47299999999999998},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":3.0619999999999998,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":0.45000000000000001}]},{"osd":4,"last update":"Wed May 22 21:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.77700000000000002,"15min":0.70499999999999996},"min":{"1min":0.54900000000000004,"5min":0.53100000000000003,"15min":0.52700000000000002},"max":{"1min":2.7610000000000001,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.80400000000000005,"15min":0.76000000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.73099999999999998}]},{"osd":5,"last update":"Wed May 22 21:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.76900000000000002,"15min":0.71899999999999997},"min":{"1min":0.498,"5min":0.48099999999999998,"15min":0.47299999999999998},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.75900000000000001,"15min":0.71299999999999997},"min":{"1min":0.53000000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Wed May 22 21:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.76500000000000001,"15min":0.71499999999999997},"min":{"1min":0.56599999999999995,"5min":0.495,"15min":0.495},"max":{"1min":2.8580000000000001,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.73099999999999998,"15min":0.68899999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":2.6880000000000002,"5min":2.6880000000000002,"15min":2.6880000000000002},"last":0.67700000000000005}]},{"osd":7,"last update":"Wed May 22 21:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.79900000000000004,"15min":0.74299999999999999},"min":{"1min":0.59799999999999998,"5min":0.438,"15min":0.438},"max":{"1min":3.0099999999999998,"5min":3.0099999999999998,"15min":3.0099999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.82299999999999995,"15min":0.78300000000000003},"min":{"1min":0.629,"5min":0.499,"15min":0.499},"max":{"1min":2.827,"5min":2.827,"15min":2.827},"last":0.749}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":739,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194573,"statfs":{"total":200000000,"available":199243072,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.42499999999999999,"15min":0.433},"min":{"1min":0.25800000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46999999999999997,"15min":0.44},"min":{"1min":0.29899999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60199999999999998,"5min":0.84799999999999998,"15min":0.876},"last":0.48399999999999999}]},{"osd":2,"last update":"Wed May 22 21:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.50600000000000001,"15min":0.496},"min":{"1min":0.373,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.52600000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":1.923},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed May 22 21:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52300000000000002,"15min":0.56200000000000006},"min":{"1min":0.34200000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80400000000000005,"5min":0.82799999999999996,"15min":2.1030000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.50700000000000001,"15min":0.49199999999999999},"min":{"1min":0.41299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.73099999999999998},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed May 22 21:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.74099999999999999,"15min":0.76000000000000001},"min":{"1min":0.57599999999999996,"5min":0.46500000000000002,"15min":0.45100000000000001},"max":{"1min":1.0620000000000001,"5min":1.109,"15min":2.1360000000000001},"last":0.623},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.746,"15min":0.69599999999999995},"min":{"1min":0.46500000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Wed May 22 21:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.79100000000000004,"15min":0.74399999999999999},"min":{"1min":0.57699999999999996,"5min":0.53500000000000003,"15min":0.45300000000000001},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.74399999999999999,"15min":0.75},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.40100000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.97},"last":0.66600000000000004}]},{"osd":6,"last update":"Wed May 22 21:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78400000000000003,"15min":0.752},"min":{"1min":0.54200000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.071,"5min":1.085,"15min":1.272},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.81000000000000005,"15min":0.75800000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.53200000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.71799999999999997}]},{"osd":7,"last update":"Wed May 22 21:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.42099999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":2.0310000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.77400000000000002},"min":{"1min":0.56799999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98199999999999998,"5min":1.0449999999999999,"15min":2.0659999999999998},"last":0.70099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1315,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193997,"statfs":{"total":200000000,"available":198653323,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.437,"15min":0.40699999999999997},"min":{"1min":0.245,"5min":0.24299999999999999,"15min":0.22800000000000001},"max":{"1min":0.85899999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51500000000000001,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72899999999999998,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48699999999999999}]},{"osd":2,"last update":"Wed May 22 21:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.495,"15min":0.45700000000000002},"min":{"1min":0.34300000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.88400000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46300000000000002,"15min":0.43099999999999999},"min":{"1min":0.28399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64500000000000002,"5min":0.76500000000000001,"15min":0.92300000000000004},"last":0.37}]},{"osd":3,"last update":"Wed May 22 21:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46400000000000002,"15min":0.436},"min":{"1min":0.313,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67300000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55200000000000005,"15min":0.50900000000000001},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.46600000000000003}]},{"osd":4,"last update":"Wed May 22 21:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.74099999999999999,"15min":0.71299999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96299999999999997,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.874},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.755,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.49099999999999999},"max":{"1min":1.0429999999999999,"5min":1.113,"15min":1.2709999999999999},"last":0.754}]},{"osd":5,"last update":"Wed May 22 21:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72399999999999998,"15min":0.70899999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93999999999999995,"5min":0.99299999999999999,"15min":1.1339999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.77400000000000002,"15min":0.74099999999999999},"min":{"1min":0.48199999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.111,"5min":1.143,"15min":1.2929999999999999},"last":0.77900000000000003}]},{"osd":6,"last update":"Wed May 22 21:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.80000000000000004,"15min":0.77500000000000002},"min":{"1min":0.60799999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.095,"5min":1.1240000000000001,"15min":1.2090000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78100000000000003,"15min":0.77200000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.069,"5min":1.2090000000000001,"15min":1.5},"last":0.69399999999999995}]},{"osd":7,"last update":"Wed May 22 21:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76400000000000001,"15min":0.72999999999999998},"min":{"1min":0.56699999999999995,"5min":0.55000000000000004,"15min":0.503},"max":{"1min":0.93100000000000005,"5min":1.163,"15min":1.163},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76600000000000001,"15min":0.75800000000000001},"min":{"1min":0.53600000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.01,"5min":1.1990000000000001,"15min":1.3260000000000001},"last":0.73899999999999999}]}]}],"pool_statfs":[]}} 2024-05-22T21:19:50.845 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-22T21:19:50.845 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-22T21:19:51.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.027+0000 7fa76a000640 1 Processor -- start 2024-05-22T21:19:51.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.027+0000 7fa76a000640 1 -- start start 2024-05-22T21:19:51.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.027+0000 7fa76a000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa764143c40 0x7fa76415b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:51.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.027+0000 7fa76a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7641444a0 0x7fa76406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:51.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.027+0000 7fa76a000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa76415efb0 con 0x7fa764144900 2024-05-22T21:19:51.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.027+0000 7fa76a000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76415f290 con 0x7fa7641444a0 2024-05-22T21:19:51.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.027+0000 7fa76a000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76415f570 con 0x7fa764143c40 2024-05-22T21:19:51.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.027+0000 7fa75be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7641444a0 0x7fa76406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:51.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.027+0000 7fa75be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7641444a0 0x7fa76406c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50748/0 (socket says 172.21.15.161:50748) 2024-05-22T21:19:51.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.027+0000 7fa75be00640 1 -- 172.21.15.161:0/1535454614 learned_addr learned my addr 172.21.15.161:0/1535454614 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:51.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.027+0000 7fa763400640 1 --2- 172.21.15.161:0/1535454614 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa764143c40 0x7fa76415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:51.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.027+0000 7fa75be00640 1 -- 172.21.15.161:0/1535454614 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa764143c40 msgr2=0x7fa76415b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:51.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.027+0000 7fa75be00640 1 --2- 172.21.15.161:0/1535454614 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa764143c40 0x7fa76415b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.027+0000 7fa75be00640 1 -- 172.21.15.161:0/1535454614 >> v1:172.21.15.179:6789/0 conn(0x7fa764144900 legacy=0x7fa7640b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:51.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.027+0000 7fa75be00640 1 -- 172.21.15.161:0/1535454614 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76415f850 con 0x7fa7641444a0 2024-05-22T21:19:51.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.027+0000 7fa762a00640 1 -- 172.21.15.161:0/1535454614 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018052567 0 0) 0x7fa76415efb0 con 0x7fa764144900 2024-05-22T21:19:51.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.031+0000 7fa75be00640 1 --2- 172.21.15.161:0/1535454614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7641444a0 0x7fa76406c2a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009870 tx=0x7fa74c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a208e5c76433b29 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:51.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.031+0000 7fa762a00640 1 -- 172.21.15.161:0/1535454614 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0ab670 con 0x7fa7641444a0 2024-05-22T21:19:51.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.031+0000 7fa762a00640 1 -- 172.21.15.161:0/1535454614 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74c0ab7d0 con 0x7fa7641444a0 2024-05-22T21:19:51.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.031+0000 7fa762a00640 1 -- 172.21.15.161:0/1535454614 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0abb60 con 0x7fa7641444a0 2024-05-22T21:19:51.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.031+0000 7fa76a000640 1 -- 172.21.15.161:0/1535454614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7641444a0 msgr2=0x7fa76406c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:51.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.031+0000 7fa76a000640 1 --2- 172.21.15.161:0/1535454614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7641444a0 0x7fa76406c2a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009870 tx=0x7fa74c01c560 comp rx=0 tx=0).stop 2024-05-22T21:19:51.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.031+0000 7fa76a000640 1 -- 172.21.15.161:0/1535454614 shutdown_connections 2024-05-22T21:19:51.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.031+0000 7fa76a000640 1 --2- 172.21.15.161:0/1535454614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7641444a0 0x7fa76406c2a0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.031+0000 7fa76a000640 1 --2- 172.21.15.161:0/1535454614 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa764143c40 0x7fa76415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.031+0000 7fa76a000640 1 -- 172.21.15.161:0/1535454614 >> 172.21.15.161:0/1535454614 conn(0x7fa76406a2d0 msgr2=0x7fa76406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:51.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.031+0000 7fa76a000640 1 -- 172.21.15.161:0/1535454614 shutdown_connections 2024-05-22T21:19:51.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.031+0000 7fa76a000640 1 -- 172.21.15.161:0/1535454614 wait complete. 2024-05-22T21:19:51.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.031+0000 7fa76a000640 1 Processor -- start 2024-05-22T21:19:51.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa76a000640 1 -- start start 2024-05-22T21:19:51.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa76a000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa764143c40 0x7fa7641537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:51.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa76a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7641444a0 0x7fa76415d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:51.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa76a000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa764144900 0x7fa764070d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:51.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa76a000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa764141240 con 0x7fa7641444a0 2024-05-22T21:19:51.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa76a000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76415de80 con 0x7fa764144900 2024-05-22T21:19:51.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa76a000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76415e160 con 0x7fa764143c40 2024-05-22T21:19:51.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa75be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7641444a0 0x7fa76415d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:51.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa75be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7641444a0 0x7fa76415d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50758/0 (socket says 172.21.15.161:50758) 2024-05-22T21:19:51.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa75be00640 1 -- 172.21.15.161:0/3755617900 learned_addr learned my addr 172.21.15.161:0/3755617900 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:51.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa763e00640 1 --2- 172.21.15.161:0/3755617900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa764144900 0x7fa764070d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:51.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa763400640 1 --2- 172.21.15.161:0/3755617900 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa764143c40 0x7fa7641537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:51.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa75be00640 1 -- 172.21.15.161:0/3755617900 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa764143c40 msgr2=0x7fa7641537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:51.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa75be00640 1 --2- 172.21.15.161:0/3755617900 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa764143c40 0x7fa7641537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa75be00640 1 -- 172.21.15.161:0/3755617900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa764144900 msgr2=0x7fa764070d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:51.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa75be00640 1 --2- 172.21.15.161:0/3755617900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa764144900 0x7fa764070d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa75be00640 1 -- 172.21.15.161:0/3755617900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c0094d0 con 0x7fa7641444a0 2024-05-22T21:19:51.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa75be00640 1 --2- 172.21.15.161:0/3755617900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7641444a0 0x7fa76415d510 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fa74c01a190 tx=0x7fa74c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:51.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa75b400640 1 -- 172.21.15.161:0/3755617900 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0ab670 con 0x7fa7641444a0 2024-05-22T21:19:51.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa75b400640 1 -- 172.21.15.161:0/3755617900 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74c0b4800 con 0x7fa7641444a0 2024-05-22T21:19:51.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa76a000640 1 -- 172.21.15.161:0/3755617900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa764141420 con 0x7fa7641444a0 2024-05-22T21:19:51.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa75b400640 1 -- 172.21.15.161:0/3755617900 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0b4bb0 con 0x7fa7641444a0 2024-05-22T21:19:51.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa76a000640 1 -- 172.21.15.161:0/3755617900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa764153ce0 con 0x7fa7641444a0 2024-05-22T21:19:51.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.035+0000 7fa76a000640 1 -- 172.21.15.161:0/3755617900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa764067220 con 0x7fa7641444a0 2024-05-22T21:19:51.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.039+0000 7fa75b400640 1 -- 172.21.15.161:0/3755617900 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa74c0b4d10 con 0x7fa7641444a0 2024-05-22T21:19:51.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.039+0000 7fa75b400640 1 --2- 172.21.15.161:0/3755617900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7400666c0 0x7fa740068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:51.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.039+0000 7fa75b400640 1 -- 172.21.15.161:0/3755617900 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa74c11df20 con 0x7fa7641444a0 2024-05-22T21:19:51.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.039+0000 7fa763400640 1 --2- 172.21.15.161:0/3755617900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7400666c0 0x7fa740068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:51.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.039+0000 7fa763400640 1 --2- 172.21.15.161:0/3755617900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7400666c0 0x7fa740068b80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa754009b60 tx=0x7fa754009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:51.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.043+0000 7fa75b400640 1 -- 172.21.15.161:0/3755617900 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa74c0eb460 con 0x7fa7641444a0 2024-05-22T21:19:51.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.247+0000 7fa76a000640 1 -- 172.21.15.161:0/3755617900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa7640b0910 con 0x7fa7641444a0 2024-05-22T21:19:51.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.255+0000 7fa75b400640 1 -- 172.21.15.161:0/3755617900 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa74c0eea80 con 0x7fa7641444a0 2024-05-22T21:19:51.256 INFO:teuthology.orchestra.run.smithi161.stderr:updated caps for client.0 2024-05-22T21:19:51.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.255+0000 7fa76a000640 1 -- 172.21.15.161:0/3755617900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7400666c0 msgr2=0x7fa740068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:51.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.255+0000 7fa76a000640 1 --2- 172.21.15.161:0/3755617900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7400666c0 0x7fa740068b80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa754009b60 tx=0x7fa754009190 comp rx=0 tx=0).stop 2024-05-22T21:19:51.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.255+0000 7fa76a000640 1 -- 172.21.15.161:0/3755617900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7641444a0 msgr2=0x7fa76415d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:51.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.255+0000 7fa76a000640 1 --2- 172.21.15.161:0/3755617900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7641444a0 0x7fa76415d510 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fa74c01a190 tx=0x7fa74c01ada0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.255+0000 7fa76a000640 1 -- 172.21.15.161:0/3755617900 shutdown_connections 2024-05-22T21:19:51.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.255+0000 7fa76a000640 1 --2- 172.21.15.161:0/3755617900 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7400666c0 0x7fa740068b80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.255+0000 7fa76a000640 1 --2- 172.21.15.161:0/3755617900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa764144900 0x7fa764070d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.255+0000 7fa76a000640 1 --2- 172.21.15.161:0/3755617900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7641444a0 0x7fa76415d510 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.255+0000 7fa76a000640 1 --2- 172.21.15.161:0/3755617900 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa764143c40 0x7fa7641537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.255+0000 7fa76a000640 1 -- 172.21.15.161:0/3755617900 >> 172.21.15.161:0/3755617900 conn(0x7fa76406a2d0 msgr2=0x7fa7640ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:51.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.255+0000 7fa76a000640 1 -- 172.21.15.161:0/3755617900 shutdown_connections 2024-05-22T21:19:51.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.255+0000 7fa76a000640 1 -- 172.21.15.161:0/3755617900 wait complete. 2024-05-22T21:19:51.277 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:51.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.411+0000 7fddf0e00640 1 Processor -- start 2024-05-22T21:19:51.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.411+0000 7fddf0e00640 1 -- start start 2024-05-22T21:19:51.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.411+0000 7fddf0e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fddec0715a0 0x7fddec15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:51.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.411+0000 7fddf0e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddec1442a0 con 0x7fddec143ec0 2024-05-22T21:19:51.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.411+0000 7fddf0e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddec070500 con 0x7fddec15ec90 2024-05-22T21:19:51.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.411+0000 7fddf0e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddec0707e0 con 0x7fddec0715a0 2024-05-22T21:19:51.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.411+0000 7fdde9e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fddec143ec0 0x7fddec06fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48890/0 (socket says 172.21.15.161:48890) 2024-05-22T21:19:51.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.411+0000 7fdde9e00640 1 -- 172.21.15.161:0/2497230897 learned_addr learned my addr 172.21.15.161:0/2497230897 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:51.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.411+0000 7fdde9400640 1 --2- 172.21.15.161:0/2497230897 >> v2:172.21.15.179:3300/0 conn(0x7fddec0715a0 0x7fddec15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:51.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.411+0000 7fdde9400640 1 -- 172.21.15.161:0/2497230897 >> v1:172.21.15.179:6789/0 conn(0x7fddec15ec90 legacy=0x7fddec0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:51.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.411+0000 7fdde9400640 1 -- 172.21.15.161:0/2497230897 >> v1:172.21.15.161:6789/0 conn(0x7fddec143ec0 legacy=0x7fddec06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:51.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.411+0000 7fdde9400640 1 -- 172.21.15.161:0/2497230897 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec070ac0 con 0x7fddec0715a0 2024-05-22T21:19:51.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.415+0000 7fdde9400640 1 --2- 172.21.15.161:0/2497230897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddec0715a0 0x7fddec15f830 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fddd80097c0 tx=0x7fddd801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63a5b17d17b9f397 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:51.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.415+0000 7fdde8a00640 1 -- 172.21.15.161:0/2497230897 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd80ab670 con 0x7fddec0715a0 2024-05-22T21:19:51.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.415+0000 7fdde8a00640 1 -- 172.21.15.161:0/2497230897 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd80ab7d0 con 0x7fddec0715a0 2024-05-22T21:19:51.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.415+0000 7fdde8a00640 1 -- 172.21.15.161:0/2497230897 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd80abb60 con 0x7fddec0715a0 2024-05-22T21:19:51.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.415+0000 7fddf0e00640 1 -- 172.21.15.161:0/2497230897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddec0715a0 msgr2=0x7fddec15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:51.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.415+0000 7fddf0e00640 1 --2- 172.21.15.161:0/2497230897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddec0715a0 0x7fddec15f830 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fddd80097c0 tx=0x7fddd801c470 comp rx=0 tx=0).stop 2024-05-22T21:19:51.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.415+0000 7fddf0e00640 1 -- 172.21.15.161:0/2497230897 shutdown_connections 2024-05-22T21:19:51.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.415+0000 7fddf0e00640 1 --2- 172.21.15.161:0/2497230897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddec0715a0 0x7fddec15f830 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.415+0000 7fddf0e00640 1 -- 172.21.15.161:0/2497230897 >> 172.21.15.161:0/2497230897 conn(0x7fddec0b0bc0 msgr2=0x7fddec0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:51.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.415+0000 7fddf0e00640 1 -- 172.21.15.161:0/2497230897 shutdown_connections 2024-05-22T21:19:51.418 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.415+0000 7fddf0e00640 1 -- 172.21.15.161:0/2497230897 wait complete. 2024-05-22T21:19:51.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fddf0e00640 1 Processor -- start 2024-05-22T21:19:51.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fddf0e00640 1 -- start start 2024-05-22T21:19:51.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fddf0e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddec143ec0 0x7fddec13f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:51.421 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fddf0e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddec15ec90 0x7fddec15b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:51.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fddf0e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddec15ce40 0x7fddec13ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:51.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fddf0e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec15bda0 con 0x7fddec143ec0 2024-05-22T21:19:51.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fddf0e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec15c080 con 0x7fddec15ce40 2024-05-22T21:19:51.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fddf0e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddec15c360 con 0x7fddec15ec90 2024-05-22T21:19:51.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fdde9e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddec143ec0 0x7fddec13f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:51.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fdde9e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddec143ec0 0x7fddec13f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50768/0 (socket says 172.21.15.161:50768) 2024-05-22T21:19:51.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fdde9e00640 1 -- 172.21.15.161:0/1819629312 learned_addr learned my addr 172.21.15.161:0/1819629312 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:51.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fddea800640 1 --2- 172.21.15.161:0/1819629312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddec15ce40 0x7fddec13ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:51.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fdde9400640 1 --2- 172.21.15.161:0/1819629312 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddec15ec90 0x7fddec15b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:51.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fdde9e00640 1 -- 172.21.15.161:0/1819629312 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddec15ec90 msgr2=0x7fddec15b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:51.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fdde9e00640 1 --2- 172.21.15.161:0/1819629312 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddec15ec90 0x7fddec15b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fdde9e00640 1 -- 172.21.15.161:0/1819629312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddec15ce40 msgr2=0x7fddec13ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:51.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fdde9e00640 1 --2- 172.21.15.161:0/1819629312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddec15ce40 0x7fddec13ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fdde9e00640 1 -- 172.21.15.161:0/1819629312 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd8009420 con 0x7fddec143ec0 2024-05-22T21:19:51.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fddea800640 1 --2- 172.21.15.161:0/1819629312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddec15ce40 0x7fddec13ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:51.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fdde9e00640 1 --2- 172.21.15.161:0/1819629312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddec143ec0 0x7fddec13f4a0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00c500 tx=0x7fdddc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:51.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fdde2000640 1 -- 172.21.15.161:0/1819629312 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc005cf0 con 0x7fddec143ec0 2024-05-22T21:19:51.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fdde2000640 1 -- 172.21.15.161:0/1819629312 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdddc005e50 con 0x7fddec143ec0 2024-05-22T21:19:51.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fddf0e00640 1 -- 172.21.15.161:0/1819629312 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddec15c640 con 0x7fddec143ec0 2024-05-22T21:19:51.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fdde2000640 1 -- 172.21.15.161:0/1819629312 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc0072f0 con 0x7fddec143ec0 2024-05-22T21:19:51.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fddf0e00640 1 -- 172.21.15.161:0/1819629312 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddec13f240 con 0x7fddec143ec0 2024-05-22T21:19:51.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.419+0000 7fddf0e00640 1 -- 172.21.15.161:0/1819629312 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddec15cb60 con 0x7fddec143ec0 2024-05-22T21:19:51.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.423+0000 7fdde2000640 1 -- 172.21.15.161:0/1819629312 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdddc007450 con 0x7fddec143ec0 2024-05-22T21:19:51.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.423+0000 7fdde2000640 1 --2- 172.21.15.161:0/1819629312 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fddcc066690 0x7fddcc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:51.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.423+0000 7fdde2000640 1 -- 172.21.15.161:0/1819629312 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdddc0835b0 con 0x7fddec143ec0 2024-05-22T21:19:51.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.423+0000 7fdde9400640 1 --2- 172.21.15.161:0/1819629312 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fddcc066690 0x7fddcc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:51.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.423+0000 7fdde9400640 1 --2- 172.21.15.161:0/1819629312 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fddcc066690 0x7fddcc068b50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fddd80af000 tx=0x7fddd80a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:51.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.427+0000 7fdde2000640 1 -- 172.21.15.161:0/1819629312 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdddc050a60 con 0x7fddec143ec0 2024-05-22T21:19:51.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.611+0000 7fddf0e00640 1 -- 172.21.15.161:0/1819629312 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddec067220 con 0x7fddec143ec0 2024-05-22T21:19:51.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.615+0000 7fdde2000640 1 -- 172.21.15.161:0/1819629312 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4745 (secure 0 0 0) 0x7fdddc054080 con 0x7fddec143ec0 2024-05-22T21:19:51.616 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:51.616 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/524372753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":524372753},{"type":"v1","addr":"172.21.15.161:6836","nonce":524372753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3762119379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3762119379},{"type":"v1","addr":"172.21.15.179:6833","nonce":3762119379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3087241366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3087241366},{"type":"v1","addr":"172.21.15.179:6835","nonce":3087241366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:46.384984+0000","modified":"2024-05-22T21:19:49.400019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7572":{"gid":7572,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/3945715002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3945715002},{"type":"v1","addr":"172.21.15.161:6837","nonce":3945715002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-22T21:19:51.616 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 62 2024-05-22T21:19:51.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.615+0000 7fddf0e00640 1 -- 172.21.15.161:0/1819629312 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fddcc066690 msgr2=0x7fddcc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:51.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.615+0000 7fddf0e00640 1 --2- 172.21.15.161:0/1819629312 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fddcc066690 0x7fddcc068b50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fddd80af000 tx=0x7fddd80a7000 comp rx=0 tx=0).stop 2024-05-22T21:19:51.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.615+0000 7fddf0e00640 1 -- 172.21.15.161:0/1819629312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddec143ec0 msgr2=0x7fddec13f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:51.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.615+0000 7fddf0e00640 1 --2- 172.21.15.161:0/1819629312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddec143ec0 0x7fddec13f4a0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00c500 tx=0x7fdddc00c9c0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.615+0000 7fddf0e00640 1 -- 172.21.15.161:0/1819629312 shutdown_connections 2024-05-22T21:19:51.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.615+0000 7fddf0e00640 1 --2- 172.21.15.161:0/1819629312 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fddcc066690 0x7fddcc068b50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.615+0000 7fddf0e00640 1 --2- 172.21.15.161:0/1819629312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddec15ce40 0x7fddec13ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.615+0000 7fddf0e00640 1 --2- 172.21.15.161:0/1819629312 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddec15ec90 0x7fddec15b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.615+0000 7fddf0e00640 1 --2- 172.21.15.161:0/1819629312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddec143ec0 0x7fddec13f4a0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:51.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.615+0000 7fddf0e00640 1 -- 172.21.15.161:0/1819629312 >> 172.21.15.161:0/1819629312 conn(0x7fddec0b0bc0 msgr2=0x7fddec0af040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:51.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.615+0000 7fddf0e00640 1 -- 172.21.15.161:0/1819629312 shutdown_connections 2024-05-22T21:19:51.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:51.615+0000 7fddf0e00640 1 -- 172.21.15.161:0/1819629312 wait complete. 2024-05-22T21:19:51.990 INFO:tasks.ceph.mds.c.smithi161.stderr:2024-05-22T21:19:51.991+0000 7ff574a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T21:19:52.040 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-22T21:19:46.384984+0000', 'modified': '2024-05-22T21:19:49.400019+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7572}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7572': {'gid': 7572, 'name': 'c', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6837/3945715002', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 3945715002}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 3945715002}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-22T21:19:52.040 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-22T21:19:52.040 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-22T21:19:52.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.175+0000 7f61f3800640 1 Processor -- start 2024-05-22T21:19:52.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.175+0000 7f61f3800640 1 -- start start 2024-05-22T21:19:52.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.175+0000 7f61f3800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61ec143c40 0x7f61ec15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.175+0000 7f61f3800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f61ec1444a0 0x7f61ec06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.175+0000 7f61f3800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ec15efb0 con 0x7f61ec144900 2024-05-22T21:19:52.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.175+0000 7f61f3800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61ec15f290 con 0x7f61ec1444a0 2024-05-22T21:19:52.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.175+0000 7f61f3800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec15f570 con 0x7f61ec143c40 2024-05-22T21:19:52.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.175+0000 7f61e3e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f61ec1444a0 0x7f61ec06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.175+0000 7f61e3e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f61ec1444a0 0x7f61ec06c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50780/0 (socket says 172.21.15.161:50780) 2024-05-22T21:19:52.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.175+0000 7f61e3e00640 1 -- 172.21.15.161:0/22056114 learned_addr learned my addr 172.21.15.161:0/22056114 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:52.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.179+0000 7f61f0e00640 1 --2- 172.21.15.161:0/22056114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61ec143c40 0x7f61ec15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.179+0000 7f61e3e00640 1 -- 172.21.15.161:0/22056114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61ec143c40 msgr2=0x7f61ec15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:52.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.179+0000 7f61e3e00640 1 --2- 172.21.15.161:0/22056114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61ec143c40 0x7f61ec15b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.179+0000 7f61e3e00640 1 -- 172.21.15.161:0/22056114 >> v1:172.21.15.179:6789/0 conn(0x7f61ec144900 legacy=0x7f61ec0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:52.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.179+0000 7f61e3e00640 1 -- 172.21.15.161:0/22056114 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec15f850 con 0x7f61ec1444a0 2024-05-22T21:19:52.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.179+0000 7f61f0e00640 1 --2- 172.21.15.161:0/22056114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61ec143c40 0x7f61ec15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:52.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.179+0000 7f61e3e00640 1 --2- 172.21.15.161:0/22056114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61ec1444a0 0x7f61ec06c2a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009870 tx=0x7f61d401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c56c2fc56feb0a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:52.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.179+0000 7f61ebe00640 1 -- 172.21.15.161:0/22056114 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d40ab670 con 0x7f61ec1444a0 2024-05-22T21:19:52.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.179+0000 7f61ebe00640 1 -- 172.21.15.161:0/22056114 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d40ab7d0 con 0x7f61ec1444a0 2024-05-22T21:19:52.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.179+0000 7f61ebe00640 1 -- 172.21.15.161:0/22056114 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d40abb60 con 0x7f61ec1444a0 2024-05-22T21:19:52.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.179+0000 7f61f3800640 1 -- 172.21.15.161:0/22056114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61ec1444a0 msgr2=0x7f61ec06c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:52.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.179+0000 7f61f3800640 1 --2- 172.21.15.161:0/22056114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61ec1444a0 0x7f61ec06c2a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009870 tx=0x7f61d401c560 comp rx=0 tx=0).stop 2024-05-22T21:19:52.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.179+0000 7f61f3800640 1 -- 172.21.15.161:0/22056114 shutdown_connections 2024-05-22T21:19:52.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.179+0000 7f61f3800640 1 --2- 172.21.15.161:0/22056114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61ec1444a0 0x7f61ec06c2a0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f3800640 1 --2- 172.21.15.161:0/22056114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61ec143c40 0x7f61ec15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f3800640 1 -- 172.21.15.161:0/22056114 >> 172.21.15.161:0/22056114 conn(0x7f61ec06a2d0 msgr2=0x7f61ec06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:52.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f3800640 1 -- 172.21.15.161:0/22056114 shutdown_connections 2024-05-22T21:19:52.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f3800640 1 -- 172.21.15.161:0/22056114 wait complete. 2024-05-22T21:19:52.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f3800640 1 Processor -- start 2024-05-22T21:19:52.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f3800640 1 -- start start 2024-05-22T21:19:52.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f3800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61ec143c40 0x7f61ec136210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f3800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61ec1444a0 0x7f61ec153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f3800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61ec144900 0x7f61ec153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f3800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec136750 con 0x7f61ec143c40 2024-05-22T21:19:52.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f3800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec142ac0 con 0x7f61ec1444a0 2024-05-22T21:19:52.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f3800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec15bc40 con 0x7f61ec144900 2024-05-22T21:19:52.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f0e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61ec143c40 0x7f61ec136210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f0e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61ec143c40 0x7f61ec136210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50794/0 (socket says 172.21.15.161:50794) 2024-05-22T21:19:52.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f0e00640 1 -- 172.21.15.161:0/1318833586 learned_addr learned my addr 172.21.15.161:0/1318833586 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:52.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f1800640 1 --2- 172.21.15.161:0/1318833586 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61ec144900 0x7f61ec153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61e3e00640 1 --2- 172.21.15.161:0/1318833586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61ec1444a0 0x7f61ec153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f0e00640 1 -- 172.21.15.161:0/1318833586 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61ec144900 msgr2=0x7f61ec153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:52.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f0e00640 1 --2- 172.21.15.161:0/1318833586 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61ec144900 0x7f61ec153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f0e00640 1 -- 172.21.15.161:0/1318833586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61ec1444a0 msgr2=0x7f61ec153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:52.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f0e00640 1 --2- 172.21.15.161:0/1318833586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61ec1444a0 0x7f61ec153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f0e00640 1 -- 172.21.15.161:0/1318833586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d40094d0 con 0x7f61ec143c40 2024-05-22T21:19:52.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f0e00640 1 --2- 172.21.15.161:0/1318833586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61ec143c40 0x7f61ec136210 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00e810 tx=0x7f61dc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:52.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61e9600640 1 -- 172.21.15.161:0/1318833586 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc00c8d0 con 0x7f61ec143c40 2024-05-22T21:19:52.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61e9600640 1 -- 172.21.15.161:0/1318833586 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc00ca30 con 0x7f61ec143c40 2024-05-22T21:19:52.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61e9600640 1 -- 172.21.15.161:0/1318833586 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc00cdc0 con 0x7f61ec143c40 2024-05-22T21:19:52.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.183+0000 7f61f3800640 1 -- 172.21.15.161:0/1318833586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ec15bf20 con 0x7f61ec143c40 2024-05-22T21:19:52.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.187+0000 7f61f3800640 1 -- 172.21.15.161:0/1318833586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ec15c4e0 con 0x7f61ec143c40 2024-05-22T21:19:52.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.187+0000 7f61e9600640 1 -- 172.21.15.161:0/1318833586 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f61dc013410 con 0x7f61ec143c40 2024-05-22T21:19:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.187+0000 7f61e9600640 1 --2- 172.21.15.161:0/1318833586 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61c80666c0 0x7f61c8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.187+0000 7f61e9600640 1 -- 172.21.15.161:0/1318833586 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61dc083ef0 con 0x7f61ec143c40 2024-05-22T21:19:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.187+0000 7f61e3e00640 1 --2- 172.21.15.161:0/1318833586 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61c80666c0 0x7f61c8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.187+0000 7f61f6ce0000 1 Processor -- start 2024-05-22T21:19:52.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.187+0000 7f61f6ce0000 1 -- start start 2024-05-22T21:19:52.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.187+0000 7f61e3e00640 1 --2- 172.21.15.161:0/1318833586 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61c80666c0 0x7f61c8068b80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f61d40af000 tx=0x7f61d40b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:52.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d7179c0 0x55e01d717da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7182e0 0x55e01d6cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d6cdec0 0x55e01d6ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01d595e80 con 0x55e01d7179c0 2024-05-22T21:19:52.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01d523af0 con 0x55e01d6cdec0 2024-05-22T21:19:52.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e01d5210c0 con 0x55e01d7182e0 2024-05-22T21:19:52.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f0e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d7179c0 0x55e01d717da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f0e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d7179c0 0x55e01d717da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50808/0 (socket says 172.21.15.161:50808) 2024-05-22T21:19:52.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f0e00640 1 -- 172.21.15.161:0/2530183125 learned_addr learned my addr 172.21.15.161:0/2530183125 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:52.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61e3e00640 1 --2- 172.21.15.161:0/2530183125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7182e0 0x55e01d6cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f1800640 1 --2- 172.21.15.161:0/2530183125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d6cdec0 0x55e01d6ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f0e00640 1 -- 172.21.15.161:0/2530183125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7182e0 msgr2=0x55e01d6cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:52.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f0e00640 1 --2- 172.21.15.161:0/2530183125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7182e0 0x55e01d6cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f0e00640 1 -- 172.21.15.161:0/2530183125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d6cdec0 msgr2=0x55e01d6ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:52.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f0e00640 1 --2- 172.21.15.161:0/2530183125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d6cdec0 0x55e01d6ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f0e00640 1 -- 172.21.15.161:0/2530183125 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e01d218890 con 0x55e01d7179c0 2024-05-22T21:19:52.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f1800640 1 --2- 172.21.15.161:0/2530183125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d6cdec0 0x55e01d6ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:52.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61e3e00640 1 --2- 172.21.15.161:0/2530183125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7182e0 0x55e01d6cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:52.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f0e00640 1 --2- 172.21.15.161:0/2530183125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d7179c0 0x55e01d717da0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f61dc082440 tx=0x7f61dc0549a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc519059ec753a7f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:52.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61e1600640 1 -- 172.21.15.161:0/2530183125 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc055ea0 con 0x55e01d7179c0 2024-05-22T21:19:52.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61e1600640 1 -- 172.21.15.161:0/2530183125 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc05d310 con 0x55e01d7179c0 2024-05-22T21:19:52.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61e1600640 1 -- 172.21.15.161:0/2530183125 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc051650 con 0x55e01d7179c0 2024-05-22T21:19:52.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 -- 172.21.15.161:0/2530183125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d7179c0 msgr2=0x55e01d717da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:52.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/2530183125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d7179c0 0x55e01d717da0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f61dc082440 tx=0x7f61dc0549a0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 -- 172.21.15.161:0/2530183125 shutdown_connections 2024-05-22T21:19:52.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/2530183125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d6cdec0 0x55e01d6ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/2530183125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7182e0 0x55e01d6cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/2530183125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d7179c0 0x55e01d717da0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 -- 172.21.15.161:0/2530183125 >> 172.21.15.161:0/2530183125 conn(0x55e01d6d4440 msgr2=0x55e01d6cc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 -- 172.21.15.161:0/2530183125 shutdown_connections 2024-05-22T21:19:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 -- 172.21.15.161:0/2530183125 wait complete. 2024-05-22T21:19:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 -- start start 2024-05-22T21:19:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 Processor -- start 2024-05-22T21:19:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d6cdec0 0x55e01d6d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7179c0 0x55e01d6d8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d7182e0 0x55e01d6d94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.191+0000 7f61f6ce0000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01d51c9b0 con 0x55e01d6cdec0 2024-05-22T21:19:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01d4bbc00 con 0x55e01d7182e0 2024-05-22T21:19:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e01d4da070 con 0x55e01d7179c0 2024-05-22T21:19:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 10 client.0 _ll_register_callbacks cb 0x55e01d6b9ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:19:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f0e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d6cdec0 0x55e01d6d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f0e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d6cdec0 0x55e01d6d2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50818/0 (socket says 172.21.15.161:50818) 2024-05-22T21:19:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f0e00640 1 -- 172.21.15.161:0/1813397679 learned_addr learned my addr 172.21.15.161:0/1813397679 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61e3e00640 1 --2- 172.21.15.161:0/1813397679 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7179c0 0x55e01d6d8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f1800640 1 --2- 172.21.15.161:0/1813397679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d7182e0 0x55e01d6d94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f0e00640 1 -- 172.21.15.161:0/1813397679 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7179c0 msgr2=0x55e01d6d8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f0e00640 1 --2- 172.21.15.161:0/1813397679 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7179c0 0x55e01d6d8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f0e00640 1 -- 172.21.15.161:0/1813397679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d7182e0 msgr2=0x55e01d6d94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f0e00640 1 --2- 172.21.15.161:0/1813397679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d7182e0 0x55e01d6d94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f0e00640 1 -- 172.21.15.161:0/1813397679 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ec15c4e0 con 0x55e01d6cdec0 2024-05-22T21:19:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f0e00640 1 --2- 172.21.15.161:0/1813397679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d6cdec0 0x55e01d6d2e80 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f61dc006df0 tx=0x7f61dc006d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b215432c5b87b2a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61da000640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:19:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61da000640 1 -- 172.21.15.161:0/1813397679 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc055ea0 con 0x55e01d6cdec0 2024-05-22T21:19:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61da000640 1 -- 172.21.15.161:0/1813397679 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61dc082100 con 0x55e01d6cdec0 2024-05-22T21:19:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 -- 172.21.15.161:0/1813397679 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e01d4d95f0 con 0x55e01d6cdec0 2024-05-22T21:19:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61da000640 1 -- 172.21.15.161:0/1813397679 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61dc05d080 con 0x55e01d6cdec0 2024-05-22T21:19:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 10 client.7704 fetch_fsmap learned FSMap version 62 2024-05-22T21:19:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 -- 172.21.15.161:0/1813397679 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55e01d4e1ab0 con 0x55e01d6cdec0 2024-05-22T21:19:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61da000640 1 -- 172.21.15.161:0/1813397679 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e01d4d95f0 con 0x55e01d6cdec0 2024-05-22T21:19:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61da000640 1 -- 172.21.15.161:0/1813397679 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f61dc06cb30 con 0x55e01d6cdec0 2024-05-22T21:19:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 10 client.7704 fetch_fsmap finished waiting for FSMap version 62 2024-05-22T21:19:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 10 client.7704 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-22T21:19:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 20 client.7704 populate_metadata read hostname 'smithi161' 2024-05-22T21:19:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/1813397679 >> [v2:172.21.15.161:6834/3945715002,v1:172.21.15.161:6837/3945715002] conn(0x55e01d726f20 0x55e01d729310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 4 client.7704 mds_command: new command op to 7572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T21:19:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 -- 172.21.15.161:0/1813397679 --> [v2:172.21.15.161:6834/3945715002,v1:172.21.15.161:6837/3945715002] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e01d519790 con 0x55e01d726f20 2024-05-22T21:19:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61e3e00640 1 --2- 172.21.15.161:0/1813397679 >> [v2:172.21.15.161:6834/3945715002,v1:172.21.15.161:6837/3945715002] conn(0x55e01d726f20 0x55e01d729310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61da000640 10 client.7704 ms_handle_connect on v2:172.21.15.161:6834/3945715002 2024-05-22T21:19:52.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61e3e00640 1 --2- 172.21.15.161:0/1813397679 >> [v2:172.21.15.161:6834/3945715002,v1:172.21.15.161:6837/3945715002] conn(0x55e01d726f20 0x55e01d729310 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae79bc14ba719b95 server_cookie=32a29b4dd87a85c9 in_seq=0 out_seq=0 2024-05-22T21:19:52.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61da000640 1 -- 172.21.15.161:0/1813397679 <== mds.0 v2:172.21.15.161:6834/3945715002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e01d519790 con 0x55e01d726f20 2024-05-22T21:19:52.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61da000640 10 client.7704 handle_command_reply: tid=0 2024-05-22T21:19:52.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 client.7704 shutdown 2024-05-22T21:19:52.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 -- 172.21.15.161:0/1813397679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d6cdec0 msgr2=0x55e01d6d2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:52.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/1813397679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d6cdec0 0x55e01d6d2e80 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f61dc006df0 tx=0x7f61dc006d80 comp rx=0 tx=0).stop 2024-05-22T21:19:52.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 -- 172.21.15.161:0/1813397679 shutdown_connections 2024-05-22T21:19:52.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/1813397679 >> [v2:172.21.15.161:6834/3945715002,v1:172.21.15.161:6837/3945715002] conn(0x55e01d726f20 0x55e01d729310 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/1813397679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d7182e0 0x55e01d6d94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/1813397679 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7179c0 0x55e01d6d8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61da000640 0 client.7704 ms_handle_reset on v2:172.21.15.161:6834/3945715002 2024-05-22T21:19:52.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/1813397679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d6cdec0 0x55e01d6d2e80 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 -- 172.21.15.161:0/1813397679 >> 172.21.15.161:0/1813397679 conn(0x55e01d6d4440 msgr2=0x55e01d6cc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:52.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 -- 172.21.15.161:0/1813397679 shutdown_connections 2024-05-22T21:19:52.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 1 -- 172.21.15.161:0/1813397679 wait complete. 2024-05-22T21:19:52.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.195+0000 7f61f6ce0000 20 client.7704 trim_cache size 0 max 16384 2024-05-22T21:19:52.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.211+0000 7f61f6ce0000 1 Processor -- start 2024-05-22T21:19:52.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.211+0000 7f61f6ce0000 1 -- start start 2024-05-22T21:19:52.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.211+0000 7f61f6ce0000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d6cdec0 0x55e01d728310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.211+0000 7f61f6ce0000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d7179c0 0x55e01d728850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.211+0000 7f61f6ce0000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7182e0 0x55e01d728d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.211+0000 7f61f6ce0000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01d1d9940 con 0x55e01d7179c0 2024-05-22T21:19:52.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.211+0000 7f61f6ce0000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01d4bbc00 con 0x55e01d6cdec0 2024-05-22T21:19:52.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.211+0000 7f61f6ce0000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e01d5114d0 con 0x55e01d7182e0 2024-05-22T21:19:52.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.211+0000 7f61e3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d7179c0 0x55e01d728850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.211+0000 7f61e3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d7179c0 0x55e01d728850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50824/0 (socket says 172.21.15.161:50824) 2024-05-22T21:19:52.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61e3e00640 1 -- 172.21.15.161:0/4289190130 learned_addr learned my addr 172.21.15.161:0/4289190130 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:52.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f1800640 1 --2- 172.21.15.161:0/4289190130 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7182e0 0x55e01d728d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f0e00640 1 --2- 172.21.15.161:0/4289190130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d6cdec0 0x55e01d728310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61e3e00640 1 -- 172.21.15.161:0/4289190130 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7182e0 msgr2=0x55e01d728d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:52.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61e3e00640 1 --2- 172.21.15.161:0/4289190130 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7182e0 0x55e01d728d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61e3e00640 1 -- 172.21.15.161:0/4289190130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d6cdec0 msgr2=0x55e01d728310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:52.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61e3e00640 1 --2- 172.21.15.161:0/4289190130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d6cdec0 0x55e01d728310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61e3e00640 1 -- 172.21.15.161:0/4289190130 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e01d4c7c00 con 0x55e01d7179c0 2024-05-22T21:19:52.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f1800640 1 --2- 172.21.15.161:0/4289190130 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7182e0 0x55e01d728d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:52.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61e3e00640 1 --2- 172.21.15.161:0/4289190130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d7179c0 0x55e01d728850 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f61d401ac10 tx=0x7f61d4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c1fa98a14f1540 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:52.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61e2000640 1 -- 172.21.15.161:0/4289190130 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d40cb480 con 0x55e01d7179c0 2024-05-22T21:19:52.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61e2000640 1 -- 172.21.15.161:0/4289190130 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d40c6040 con 0x55e01d7179c0 2024-05-22T21:19:52.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61e2000640 1 -- 172.21.15.161:0/4289190130 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d40cb7e0 con 0x55e01d7179c0 2024-05-22T21:19:52.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 -- 172.21.15.161:0/4289190130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d7179c0 msgr2=0x55e01d728850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:52.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/4289190130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d7179c0 0x55e01d728850 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f61d401ac10 tx=0x7f61d4004740 comp rx=0 tx=0).stop 2024-05-22T21:19:52.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 -- 172.21.15.161:0/4289190130 shutdown_connections 2024-05-22T21:19:52.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/4289190130 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7182e0 0x55e01d728d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/4289190130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d7179c0 0x55e01d728850 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/4289190130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d6cdec0 0x55e01d728310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 -- 172.21.15.161:0/4289190130 >> 172.21.15.161:0/4289190130 conn(0x55e01d6d4440 msgr2=0x55e01d716800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:52.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 -- 172.21.15.161:0/4289190130 shutdown_connections 2024-05-22T21:19:52.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 -- 172.21.15.161:0/4289190130 wait complete. 2024-05-22T21:19:52.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 -- start start 2024-05-22T21:19:52.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 Processor -- start 2024-05-22T21:19:52.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d6cdec0 0x55e01d75c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7179c0 0x55e01d75cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d7182e0 0x55e01d75d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01d5181a0 con 0x55e01d6cdec0 2024-05-22T21:19:52.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01d520220 con 0x55e01d7182e0 2024-05-22T21:19:52.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e01d51b840 con 0x55e01d7179c0 2024-05-22T21:19:52.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f6ce0000 10 client.0 _ll_register_callbacks cb 0x7f61e40014b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:19:52.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f0e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d6cdec0 0x55e01d75c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f0e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d6cdec0 0x55e01d75c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50830/0 (socket says 172.21.15.161:50830) 2024-05-22T21:19:52.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f0e00640 1 -- 172.21.15.161:0/1216060512 learned_addr learned my addr 172.21.15.161:0/1216060512 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:52.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61f1800640 1 --2- 172.21.15.161:0/1216060512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d7182e0 0x55e01d75d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.215+0000 7f61e3e00640 1 --2- 172.21.15.161:0/1216060512 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7179c0 0x55e01d75cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f0e00640 1 -- 172.21.15.161:0/1216060512 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7179c0 msgr2=0x55e01d75cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:52.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f0e00640 1 --2- 172.21.15.161:0/1216060512 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7179c0 0x55e01d75cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f0e00640 1 -- 172.21.15.161:0/1216060512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d7182e0 msgr2=0x55e01d75d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:52.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f0e00640 1 --2- 172.21.15.161:0/1216060512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d7182e0 0x55e01d75d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f0e00640 1 -- 172.21.15.161:0/1216060512 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ec15f850 con 0x55e01d6cdec0 2024-05-22T21:19:52.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f0e00640 1 --2- 172.21.15.161:0/1216060512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d6cdec0 0x55e01d75c650 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f61dc051440 tx=0x7f61dc051300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de26f0462438fc3e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:52.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61da000640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:19:52.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61da000640 1 -- 172.21.15.161:0/1216060512 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc055ea0 con 0x55e01d6cdec0 2024-05-22T21:19:52.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61da000640 1 -- 172.21.15.161:0/1216060512 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61dc082100 con 0x55e01d6cdec0 2024-05-22T21:19:52.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 1 -- 172.21.15.161:0/1216060512 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e01d4ec460 con 0x55e01d6cdec0 2024-05-22T21:19:52.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61da000640 1 -- 172.21.15.161:0/1216060512 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61dc054d10 con 0x55e01d6cdec0 2024-05-22T21:19:52.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 10 client.7710 fetch_fsmap learned FSMap version 62 2024-05-22T21:19:52.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 1 -- 172.21.15.161:0/1216060512 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55e01d218cc0 con 0x55e01d6cdec0 2024-05-22T21:19:52.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61da000640 1 -- 172.21.15.161:0/1216060512 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e01d4ec460 con 0x55e01d6cdec0 2024-05-22T21:19:52.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61da000640 1 -- 172.21.15.161:0/1216060512 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f61dc053ea0 con 0x55e01d6cdec0 2024-05-22T21:19:52.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 10 client.7710 fetch_fsmap finished waiting for FSMap version 62 2024-05-22T21:19:52.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 10 client.7710 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-22T21:19:52.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 20 client.7710 populate_metadata read hostname 'smithi161' 2024-05-22T21:19:52.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/1216060512 >> [v2:172.21.15.161:6834/3945715002,v1:172.21.15.161:6837/3945715002] conn(0x55e01d75d9f0 0x55e01d75ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 4 client.7710 mds_command: new command op to 7572 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-22T21:19:52.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 1 -- 172.21.15.161:0/1216060512 --> [v2:172.21.15.161:6834/3945715002,v1:172.21.15.161:6837/3945715002] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e01d4e59c0 con 0x55e01d75d9f0 2024-05-22T21:19:52.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61e3e00640 1 --2- 172.21.15.161:0/1216060512 >> [v2:172.21.15.161:6834/3945715002,v1:172.21.15.161:6837/3945715002] conn(0x55e01d75d9f0 0x55e01d75ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61e3e00640 1 --2- 172.21.15.161:0/1216060512 >> [v2:172.21.15.161:6834/3945715002,v1:172.21.15.161:6837/3945715002] conn(0x55e01d75d9f0 0x55e01d75ddd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd7aa172861418bd server_cookie=77ea574994699afe in_seq=0 out_seq=0 2024-05-22T21:19:52.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61da000640 10 client.7710 ms_handle_connect on v2:172.21.15.161:6834/3945715002 2024-05-22T21:19:52.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61da000640 1 -- 172.21.15.161:0/1216060512 <== mds.0 v2:172.21.15.161:6834/3945715002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55e01d4e59c0 con 0x55e01d75d9f0 2024-05-22T21:19:52.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61da000640 10 client.7710 handle_command_reply: tid=0 2024-05-22T21:19:52.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 1 client.7710 shutdown 2024-05-22T21:19:52.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 1 -- 172.21.15.161:0/1216060512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d6cdec0 msgr2=0x55e01d75c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:52.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/1216060512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d6cdec0 0x55e01d75c650 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f61dc051440 tx=0x7f61dc051300 comp rx=0 tx=0).stop 2024-05-22T21:19:52.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 1 -- 172.21.15.161:0/1216060512 shutdown_connections 2024-05-22T21:19:52.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/1216060512 >> [v2:172.21.15.161:6834/3945715002,v1:172.21.15.161:6837/3945715002] conn(0x55e01d75d9f0 0x55e01d75ddd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/1216060512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e01d7182e0 0x55e01d75d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/1216060512 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55e01d7179c0 0x55e01d75cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 1 --2- 172.21.15.161:0/1216060512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e01d6cdec0 0x55e01d75c650 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61da000640 0 client.7710 ms_handle_reset on v2:172.21.15.161:6834/3945715002 2024-05-22T21:19:52.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 1 -- 172.21.15.161:0/1216060512 >> 172.21.15.161:0/1216060512 conn(0x55e01d6d4440 msgr2=0x55e01d727300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:52.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 1 -- 172.21.15.161:0/1216060512 shutdown_connections 2024-05-22T21:19:52.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 1 -- 172.21.15.161:0/1216060512 wait complete. 2024-05-22T21:19:52.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f6ce0000 20 client.7710 trim_cache size 0 max 16384 2024-05-22T21:19:52.227 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:19:52.227 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:19:52.227 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 0, 2024-05-22T21:19:52.227 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 7572, 2024-05-22T21:19:52.227 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2024-05-22T21:19:52.227 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2024-05-22T21:19:52.227 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2024-05-22T21:19:52.227 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 5.8353067750000003, 2024-05-22T21:19:52.227 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 62, 2024-05-22T21:19:52.227 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 140, 2024-05-22T21:19:52.227 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 140, 2024-05-22T21:19:52.228 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 11.085459988 2024-05-22T21:19:52.228 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:19:52.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f3800640 1 -- 172.21.15.161:0/1318833586 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61c80666c0 msgr2=0x7f61c8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:52.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f3800640 1 --2- 172.21.15.161:0/1318833586 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61c80666c0 0x7f61c8068b80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f61d40af000 tx=0x7f61d40b6040 comp rx=0 tx=0).stop 2024-05-22T21:19:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f3800640 1 -- 172.21.15.161:0/1318833586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61ec143c40 msgr2=0x7f61ec136210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f3800640 1 --2- 172.21.15.161:0/1318833586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61ec143c40 0x7f61ec136210 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00e810 tx=0x7f61dc00ecd0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f3800640 1 -- 172.21.15.161:0/1318833586 shutdown_connections 2024-05-22T21:19:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f3800640 1 --2- 172.21.15.161:0/1318833586 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f61c80666c0 0x7f61c8068b80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f3800640 1 --2- 172.21.15.161:0/1318833586 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61ec144900 0x7f61ec153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f3800640 1 --2- 172.21.15.161:0/1318833586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61ec1444a0 0x7f61ec153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f3800640 1 --2- 172.21.15.161:0/1318833586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61ec143c40 0x7f61ec136210 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f3800640 1 -- 172.21.15.161:0/1318833586 >> 172.21.15.161:0/1318833586 conn(0x7f61ec06a2d0 msgr2=0x7f61ec0cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f3800640 1 -- 172.21.15.161:0/1318833586 shutdown_connections 2024-05-22T21:19:52.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.219+0000 7f61f3800640 1 -- 172.21.15.161:0/1318833586 wait complete. 2024-05-22T21:19:52.241 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:19:52.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f5fa00640 1 Processor -- start 2024-05-22T21:19:52.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f5fa00640 1 -- start start 2024-05-22T21:19:52.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f5fa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f580b4420 0x7f0f5806aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f5fa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0f5806b420 0x7f0f5806c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f5fa00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0f5806c8f0 0x7f0f58160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f5fa00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5815db70 con 0x7f0f5806b420 2024-05-22T21:19:52.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f5fa00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5806b880 con 0x7f0f5806c8f0 2024-05-22T21:19:52.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f5fa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f580ad610 con 0x7f0f580b4420 2024-05-22T21:19:52.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f57e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0f5806b420 0x7f0f5806c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f57e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0f5806b420 0x7f0f5806c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50838/0 (socket says 172.21.15.161:50838) 2024-05-22T21:19:52.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f57e00640 1 -- 172.21.15.161:0/167128982 learned_addr learned my addr 172.21.15.161:0/167128982 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:52.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f5da00640 1 --2- 172.21.15.161:0/167128982 >> v2:172.21.15.179:3300/0 conn(0x7f0f5806c8f0 0x7f0f58160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f5d000640 1 --2- 172.21.15.161:0/167128982 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f580b4420 0x7f0f5806aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f57e00640 1 -- 172.21.15.161:0/167128982 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f580b4420 msgr2=0x7f0f5806aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:52.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f57e00640 1 --2- 172.21.15.161:0/167128982 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f580b4420 0x7f0f5806aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f57e00640 1 -- 172.21.15.161:0/167128982 >> v2:172.21.15.179:3300/0 conn(0x7f0f5806c8f0 msgr2=0x7f0f58160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:52.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f57e00640 1 --2- 172.21.15.161:0/167128982 >> v2:172.21.15.179:3300/0 conn(0x7f0f5806c8f0 0x7f0f58160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f57e00640 1 -- 172.21.15.161:0/167128982 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5806cd20 con 0x7f0f5806b420 2024-05-22T21:19:52.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.375+0000 7f0f5d000640 1 --2- 172.21.15.161:0/167128982 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f580b4420 0x7f0f5806aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:52.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.379+0000 7f0f57e00640 1 --2- 172.21.15.161:0/167128982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5806b420 0x7f0f5806c3b0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f0f5806db40 tx=0x7f0f4401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bbccbec21205e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:52.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.379+0000 7f0f57400640 1 -- 172.21.15.161:0/167128982 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f440ab600 con 0x7f0f5806b420 2024-05-22T21:19:52.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.379+0000 7f0f57400640 1 -- 172.21.15.161:0/167128982 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f440a7050 con 0x7f0f5806b420 2024-05-22T21:19:52.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.379+0000 7f0f57400640 1 -- 172.21.15.161:0/167128982 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f440ab980 con 0x7f0f5806b420 2024-05-22T21:19:52.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.379+0000 7f0f5fa00640 1 -- 172.21.15.161:0/167128982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5806b420 msgr2=0x7f0f5806c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:52.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.379+0000 7f0f5fa00640 1 --2- 172.21.15.161:0/167128982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5806b420 0x7f0f5806c3b0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f0f5806db40 tx=0x7f0f4401c260 comp rx=0 tx=0).stop 2024-05-22T21:19:52.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.379+0000 7f0f5fa00640 1 -- 172.21.15.161:0/167128982 shutdown_connections 2024-05-22T21:19:52.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.379+0000 7f0f5fa00640 1 --2- 172.21.15.161:0/167128982 >> v2:172.21.15.179:3300/0 conn(0x7f0f5806c8f0 0x7f0f58160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.379+0000 7f0f5fa00640 1 --2- 172.21.15.161:0/167128982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5806b420 0x7f0f5806c3b0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.379+0000 7f0f5fa00640 1 --2- 172.21.15.161:0/167128982 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f580b4420 0x7f0f5806aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.379+0000 7f0f5fa00640 1 -- 172.21.15.161:0/167128982 >> 172.21.15.161:0/167128982 conn(0x7f0f58070630 msgr2=0x7f0f58070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:52.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.379+0000 7f0f5fa00640 1 -- 172.21.15.161:0/167128982 shutdown_connections 2024-05-22T21:19:52.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.379+0000 7f0f5fa00640 1 -- 172.21.15.161:0/167128982 wait complete. 2024-05-22T21:19:52.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5fa00640 1 Processor -- start 2024-05-22T21:19:52.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5fa00640 1 -- start start 2024-05-22T21:19:52.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5fa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5806b420 0x7f0f58156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5fa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f5806c8f0 0x7f0f5815fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5fa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f580b4420 0x7f0f58149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5fa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58157110 con 0x7f0f5806b420 2024-05-22T21:19:52.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5fa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58156110 con 0x7f0f580b4420 2024-05-22T21:19:52.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5fa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58152e10 con 0x7f0f5806c8f0 2024-05-22T21:19:52.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5806b420 0x7f0f58156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5806b420 0x7f0f58156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50852/0 (socket says 172.21.15.161:50852) 2024-05-22T21:19:52.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5d000640 1 -- 172.21.15.161:0/3940728560 learned_addr learned my addr 172.21.15.161:0/3940728560 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:52.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f57e00640 1 --2- 172.21.15.161:0/3940728560 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f5806c8f0 0x7f0f5815fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5d000640 1 -- 172.21.15.161:0/3940728560 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f5806c8f0 msgr2=0x7f0f5815fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:52.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5d000640 1 --2- 172.21.15.161:0/3940728560 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f5806c8f0 0x7f0f5815fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5da00640 1 --2- 172.21.15.161:0/3940728560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f580b4420 0x7f0f58149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5d000640 1 -- 172.21.15.161:0/3940728560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f580b4420 msgr2=0x7f0f58149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:52.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5d000640 1 --2- 172.21.15.161:0/3940728560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f580b4420 0x7f0f58149ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5d000640 1 -- 172.21.15.161:0/3940728560 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f440094d0 con 0x7f0f5806b420 2024-05-22T21:19:52.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5da00640 1 --2- 172.21.15.161:0/3940728560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f580b4420 0x7f0f58149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:19:52.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5d000640 1 --2- 172.21.15.161:0/3940728560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5806b420 0x7f0f58156bd0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800d800 tx=0x7f0f4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:52.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f54c00640 1 -- 172.21.15.161:0/3940728560 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800b6c0 con 0x7f0f5806b420 2024-05-22T21:19:52.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f54c00640 1 -- 172.21.15.161:0/3940728560 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4800b820 con 0x7f0f5806b420 2024-05-22T21:19:52.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5fa00640 1 -- 172.21.15.161:0/3940728560 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f581563d0 con 0x7f0f5806b420 2024-05-22T21:19:52.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f54c00640 1 -- 172.21.15.161:0/3940728560 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800bbd0 con 0x7f0f5806b420 2024-05-22T21:19:52.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.383+0000 7f0f5fa00640 1 -- 172.21.15.161:0/3940728560 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f581600b0 con 0x7f0f5806b420 2024-05-22T21:19:52.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.387+0000 7f0f5fa00640 1 -- 172.21.15.161:0/3940728560 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f581602a0 con 0x7f0f5806b420 2024-05-22T21:19:52.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.387+0000 7f0f54c00640 1 -- 172.21.15.161:0/3940728560 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0f4800bd30 con 0x7f0f5806b420 2024-05-22T21:19:52.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.387+0000 7f0f54c00640 1 --2- 172.21.15.161:0/3940728560 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f38066690 0x7f0f38068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:52.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.387+0000 7f0f54c00640 1 -- 172.21.15.161:0/3940728560 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f480845f0 con 0x7f0f5806b420 2024-05-22T21:19:52.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.387+0000 7f0f57e00640 1 --2- 172.21.15.161:0/3940728560 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f38066690 0x7f0f38068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:52.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.387+0000 7f0f57e00640 1 --2- 172.21.15.161:0/3940728560 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f38066690 0x7f0f38068b50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0f440ad000 tx=0x7f0f44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:52.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.391+0000 7f0f54c00640 1 -- 172.21.15.161:0/3940728560 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0f48051960 con 0x7f0f5806b420 2024-05-22T21:19:52.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.583+0000 7f0f5fa00640 1 -- 172.21.15.161:0/3940728560 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0f58067220 con 0x7f0f5806b420 2024-05-22T21:19:52.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.583+0000 7f0f54c00640 1 -- 172.21.15.161:0/3940728560 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0f48054f80 con 0x7f0f5806b420 2024-05-22T21:19:52.585 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:19:52.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.583+0000 7f0f5fa00640 1 -- 172.21.15.161:0/3940728560 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f38066690 msgr2=0x7f0f38068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:52.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.583+0000 7f0f5fa00640 1 --2- 172.21.15.161:0/3940728560 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f38066690 0x7f0f38068b50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0f440ad000 tx=0x7f0f44008040 comp rx=0 tx=0).stop 2024-05-22T21:19:52.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.583+0000 7f0f5fa00640 1 -- 172.21.15.161:0/3940728560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5806b420 msgr2=0x7f0f58156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:52.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.583+0000 7f0f5fa00640 1 --2- 172.21.15.161:0/3940728560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5806b420 0x7f0f58156bd0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800d800 tx=0x7f0f4800dcc0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.583+0000 7f0f5fa00640 1 -- 172.21.15.161:0/3940728560 shutdown_connections 2024-05-22T21:19:52.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.583+0000 7f0f5fa00640 1 --2- 172.21.15.161:0/3940728560 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0f38066690 0x7f0f38068b50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.583+0000 7f0f5fa00640 1 --2- 172.21.15.161:0/3940728560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f580b4420 0x7f0f58149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.583+0000 7f0f5fa00640 1 --2- 172.21.15.161:0/3940728560 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0f5806c8f0 0x7f0f5815fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.583+0000 7f0f5fa00640 1 --2- 172.21.15.161:0/3940728560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f5806b420 0x7f0f58156bd0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:52.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.583+0000 7f0f5fa00640 1 -- 172.21.15.161:0/3940728560 >> 172.21.15.161:0/3940728560 conn(0x7f0f58070630 msgr2=0x7f0f5815b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:52.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.583+0000 7f0f5fa00640 1 -- 172.21.15.161:0/3940728560 shutdown_connections 2024-05-22T21:19:52.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:52.583+0000 7f0f5fa00640 1 -- 172.21.15.161:0/3940728560 wait complete. 2024-05-22T21:19:52.606 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:19:52.606 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:19:52.606 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-05-22T21:19:52.607 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-22T21:19:52.607 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-22T21:19:52.607 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:19:52.607 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:19:52.607 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:19:52.607 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:19:52.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:52.607 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-05-22T21:19:52.617 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:19:52.618 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:19:52.618 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:19:52.618 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:52.618 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-05-22T21:19:52.618 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:52.618 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:19:52.619 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:52.619 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2024-05-22T21:19:52.619 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:19:52.619 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:52.619 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2024-05-22T21:19:52.619 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:19:52.620 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:52.620 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:19:52.620 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:52.620 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:19:52.620 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42100sec preferred_lft 42100sec 2024-05-22T21:19:52.620 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-05-22T21:19:52.620 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:52.621 INFO:teuthology.orchestra.run.smithi161.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:19:52.621 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 02:42:b9:0f:57:df brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:52.621 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:19:52.621 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:52.621 INFO:teuthology.orchestra.run.smithi161.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:19:52.621 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:52.621 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:19:52.621 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:52.621 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:19:52.622 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:52.622 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:19:52.622 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-22T21:19:52.622 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:19:52.622 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:52.622 INFO:teuthology.orchestra.run.smithi161.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:19:52.622 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-22T21:19:52.622 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:19:52.622 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:52.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:52.623 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:19:52.676 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-05-22T21:19:52.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:52.679 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-05-22T21:19:52.730 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-05-22T21:19:52.730 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:19:52.731 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:19:52.731 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:19:52.731 INFO:teuthology.orchestra.run.smithi161.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:19:52.731 INFO:teuthology.orchestra.run.smithi161.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:19:52.731 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:19:52.731 INFO:teuthology.orchestra.run.smithi161.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:19:52.731 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:19:52.731 INFO:teuthology.orchestra.run.smithi161.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:19:52.731 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:19:52.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:52.732 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:19:52.732 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:19:52.732 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:19:52.732 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:19:52.732 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:19:52.732 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:19:52.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:52.819 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-05-22T21:19:52.871 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-22T21:19:52.871 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-22T21:19:52.872 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:19:52.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:52.872 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2024-05-22T21:19:52.934 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:19:52.934 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-22T21:19:52.983 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:19:52.983 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:19:52.983 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:19:53.030 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:19:53.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:53.031 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:19:53.118 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:19:53.118 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:19:53.118 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:19:53.133 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:19:53.133 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:19:53.133 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:19:53.133 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:19:53.133 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:19:53.141 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:19:53.141 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:19:53.142 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:19:53.142 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:19:53.142 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:19:53.142 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:19:53.142 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:19:53.142 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:19:53.145 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:19:53.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:53.145 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:19:53.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:53.207 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:19:53.264 INFO:teuthology.orchestra.run.smithi161.stdout:client6497 2024-05-22T21:19:53.264 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:19:53.265 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:53.315 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:19:53.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.490+0000 7fe8c3000640 1 Processor -- start 2024-05-22T21:19:53.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.490+0000 7fe8c3000640 1 -- start start 2024-05-22T21:19:53.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.490+0000 7fe8c3000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe8bc0715a0 0x7fe8bc15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:53.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.490+0000 7fe8c3000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8bc1442a0 con 0x7fe8bc143ec0 2024-05-22T21:19:53.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.490+0000 7fe8c3000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8bc070500 con 0x7fe8bc15ec90 2024-05-22T21:19:53.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.490+0000 7fe8c3000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8bc0707e0 con 0x7fe8bc0715a0 2024-05-22T21:19:53.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.490+0000 7fe8bbe00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe8bc143ec0 0x7fe8bc06fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48906/0 (socket says 172.21.15.161:48906) 2024-05-22T21:19:53.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.490+0000 7fe8bbe00640 1 -- 172.21.15.161:0/1245571319 learned_addr learned my addr 172.21.15.161:0/1245571319 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:53.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8b3400640 1 --2- 172.21.15.161:0/1245571319 >> v2:172.21.15.179:3300/0 conn(0x7fe8bc0715a0 0x7fe8bc15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:53.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 -- 172.21.15.161:0/1245571319 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654628848 0 0) 0x7fe8bc1442a0 con 0x7fe8bc143ec0 2024-05-22T21:19:53.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 -- 172.21.15.161:0/1245571319 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8a0003530 con 0x7fe8bc143ec0 2024-05-22T21:19:53.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 -- 172.21.15.161:0/1245571319 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189567130 0 0) 0x7fe8bc070500 con 0x7fe8bc15ec90 2024-05-22T21:19:53.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 -- 172.21.15.161:0/1245571319 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8bc1442a0 con 0x7fe8bc15ec90 2024-05-22T21:19:53.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 -- 172.21.15.161:0/1245571319 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 737857664 0 0) 0x7fe8a0003530 con 0x7fe8bc143ec0 2024-05-22T21:19:53.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 -- 172.21.15.161:0/1245571319 >> v2:172.21.15.179:3300/0 conn(0x7fe8bc0715a0 msgr2=0x7fe8bc15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:53.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 --2- 172.21.15.161:0/1245571319 >> v2:172.21.15.179:3300/0 conn(0x7fe8bc0715a0 0x7fe8bc15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:53.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 -- 172.21.15.161:0/1245571319 >> v1:172.21.15.179:6789/0 conn(0x7fe8bc15ec90 legacy=0x7fe8bc0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:53.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 -- 172.21.15.161:0/1245571319 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc070ac0 con 0x7fe8bc143ec0 2024-05-22T21:19:53.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 -- 172.21.15.161:0/1245571319 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fe8a4001680 con 0x7fe8bc143ec0 2024-05-22T21:19:53.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8b3400640 1 --2- 172.21.15.161:0/1245571319 >> v2:172.21.15.179:3300/0 conn(0x7fe8bc0715a0 0x7fe8bc15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:53.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 -- 172.21.15.161:0/1245571319 >> v1:172.21.15.161:6789/0 conn(0x7fe8bc143ec0 legacy=0x7fe8bc06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:53.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 --2- 172.21.15.161:0/1245571319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a0003b10 0x7fe8a0003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:53.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 --2- 172.21.15.161:0/1245571319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8a00047d0 0x7fe8a0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:53.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 --2- 172.21.15.161:0/1245571319 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8a0005190 0x7fe8a000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:53.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 -- 172.21.15.161:0/1245571319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0707e0 con 0x7fe8a0003b10 2024-05-22T21:19:53.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 -- 172.21.15.161:0/1245571319 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a000c010 con 0x7fe8a00047d0 2024-05-22T21:19:53.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bb400640 1 -- 172.21.15.161:0/1245571319 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a000c1d0 con 0x7fe8a0005190 2024-05-22T21:19:53.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bbe00640 1 --2- 172.21.15.161:0/1245571319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a0003b10 0x7fe8a0003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:53.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8b3400640 1 --2- 172.21.15.161:0/1245571319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8a00047d0 0x7fe8a0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:53.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8c1000640 1 --2- 172.21.15.161:0/1245571319 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8a0005190 0x7fe8a000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:53.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bbe00640 1 -- 172.21.15.161:0/1245571319 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8a0005190 msgr2=0x7fe8a000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:53.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bbe00640 1 --2- 172.21.15.161:0/1245571319 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8a0005190 0x7fe8a000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:53.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bbe00640 1 -- 172.21.15.161:0/1245571319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8a00047d0 msgr2=0x7fe8a0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:53.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bbe00640 1 --2- 172.21.15.161:0/1245571319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8a00047d0 0x7fe8a0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:53.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8bbe00640 1 -- 172.21.15.161:0/1245571319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a0003530 con 0x7fe8a0003b10 2024-05-22T21:19:53.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.494+0000 7fe8c1000640 1 -- 172.21.15.161:0/1245571319 reap_dead start 2024-05-22T21:19:53.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8bbe00640 1 --2- 172.21.15.161:0/1245571319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a0003b10 0x7fe8a0003f10 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400c160 tx=0x7fe8a401cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07afe08738e8a2c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:53.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8bb400640 1 -- 172.21.15.161:0/1245571319 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40a5940 con 0x7fe8a0003b10 2024-05-22T21:19:53.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8bb400640 1 -- 172.21.15.161:0/1245571319 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a40a5aa0 con 0x7fe8a0003b10 2024-05-22T21:19:53.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8bb400640 1 -- 172.21.15.161:0/1245571319 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40058e0 con 0x7fe8a0003b10 2024-05-22T21:19:53.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8c3000640 1 -- 172.21.15.161:0/1245571319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a0003b10 msgr2=0x7fe8a0003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:53.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8c3000640 1 --2- 172.21.15.161:0/1245571319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a0003b10 0x7fe8a0003f10 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400c160 tx=0x7fe8a401cbb0 comp rx=0 tx=0).stop 2024-05-22T21:19:53.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8c3000640 1 -- 172.21.15.161:0/1245571319 shutdown_connections 2024-05-22T21:19:53.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8c3000640 1 --2- 172.21.15.161:0/1245571319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a0003b10 0x7fe8a0003f10 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:53.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8c3000640 1 -- 172.21.15.161:0/1245571319 >> 172.21.15.161:0/1245571319 conn(0x7fe8bc0b0bc0 msgr2=0x7fe8bc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:53.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8c3000640 1 -- 172.21.15.161:0/1245571319 shutdown_connections 2024-05-22T21:19:53.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8c3000640 1 -- 172.21.15.161:0/1245571319 wait complete. 2024-05-22T21:19:53.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8c3000640 1 Processor -- start 2024-05-22T21:19:53.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8c3000640 1 -- start start 2024-05-22T21:19:53.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8c3000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8a0003b10 0x7fe8bc13e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:53.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8c3000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8bc0c3e30 0x7fe8bc0c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:53.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8c3000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8bc0c42b0 0x7fe8bc0c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:53.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.498+0000 7fe8c3000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc1f74b0 con 0x7fe8bc0c3e30 2024-05-22T21:19:53.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8c3000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc1f6830 con 0x7fe8bc0c42b0 2024-05-22T21:19:53.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8c3000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc1f6b10 con 0x7fe8a0003b10 2024-05-22T21:19:53.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8b3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8bc0c3e30 0x7fe8bc0c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:53.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8b3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8bc0c3e30 0x7fe8bc0c47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50868/0 (socket says 172.21.15.161:50868) 2024-05-22T21:19:53.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8b3400640 1 -- 172.21.15.161:0/3265571565 learned_addr learned my addr 172.21.15.161:0/3265571565 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:53.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8bbe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8a0003b10 0x7fe8bc13e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:53.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8c1000640 1 --2- 172.21.15.161:0/3265571565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8bc0c42b0 0x7fe8bc0c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:53.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8bbe00640 1 -- 172.21.15.161:0/3265571565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8bc0c42b0 msgr2=0x7fe8bc0c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:53.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8bbe00640 1 --2- 172.21.15.161:0/3265571565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8bc0c42b0 0x7fe8bc0c4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:53.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8bbe00640 1 -- 172.21.15.161:0/3265571565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8bc0c3e30 msgr2=0x7fe8bc0c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:53.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8bbe00640 1 --2- 172.21.15.161:0/3265571565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8bc0c3e30 0x7fe8bc0c47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:53.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8bbe00640 1 -- 172.21.15.161:0/3265571565 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc1f6df0 con 0x7fe8a0003b10 2024-05-22T21:19:53.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8b3400640 1 --2- 172.21.15.161:0/3265571565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8bc0c3e30 0x7fe8bc0c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:19:53.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8bbe00640 1 --2- 172.21.15.161:0/3265571565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8a0003b10 0x7fe8bc13e460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009000 tx=0x7fe8a0005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:53.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8b8c00640 1 -- 172.21.15.161:0/3265571565 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40a5940 con 0x7fe8a0003b10 2024-05-22T21:19:53.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8b8c00640 1 -- 172.21.15.161:0/3265571565 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a40b2070 con 0x7fe8a0003b10 2024-05-22T21:19:53.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8b8c00640 1 -- 172.21.15.161:0/3265571565 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40074f0 con 0x7fe8a0003b10 2024-05-22T21:19:53.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8c3000640 1 -- 172.21.15.161:0/3265571565 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8bc1f70d0 con 0x7fe8a0003b10 2024-05-22T21:19:53.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8c3000640 1 -- 172.21.15.161:0/3265571565 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8bc0adfe0 con 0x7fe8a0003b10 2024-05-22T21:19:53.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8c3000640 1 -- 172.21.15.161:0/3265571565 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8bc1f7720 con 0x7fe8a0003b10 2024-05-22T21:19:53.507 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.502+0000 7fe8b8c00640 1 -- 172.21.15.161:0/3265571565 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe8a4007650 con 0x7fe8a0003b10 2024-05-22T21:19:53.507 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.506+0000 7fe8b8c00640 1 --2- 172.21.15.161:0/3265571565 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe898066690 0x7fe898068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:53.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.506+0000 7fe8b8c00640 1 -- 172.21.15.161:0/3265571565 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8a411dcc0 con 0x7fe8a0003b10 2024-05-22T21:19:53.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.506+0000 7fe8b3400640 1 --2- 172.21.15.161:0/3265571565 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe898066690 0x7fe898068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:53.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.506+0000 7fe8b3400640 1 --2- 172.21.15.161:0/3265571565 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe898066690 0x7fe898068b50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc142a20 tx=0x7fe8ac009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:53.512 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.510+0000 7fe8b8c00640 1 -- 172.21.15.161:0/3265571565 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe8a4124080 con 0x7fe8a0003b10 2024-05-22T21:19:53.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.694+0000 7fe8c3000640 1 -- 172.21.15.161:0/3265571565 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe8bc1f6c50 con 0x7fe8a0003b10 2024-05-22T21:19:53.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.698+0000 7fe8b8c00640 1 -- 172.21.15.161:0/3265571565 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe8bc1f6c50 con 0x7fe8a0003b10 2024-05-22T21:19:53.700 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:19:53.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.698+0000 7fe8c3000640 1 -- 172.21.15.161:0/3265571565 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe898066690 msgr2=0x7fe898068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:53.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.698+0000 7fe8c3000640 1 --2- 172.21.15.161:0/3265571565 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe898066690 0x7fe898068b50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc142a20 tx=0x7fe8ac009190 comp rx=0 tx=0).stop 2024-05-22T21:19:53.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.698+0000 7fe8c3000640 1 -- 172.21.15.161:0/3265571565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8a0003b10 msgr2=0x7fe8bc13e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:53.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.698+0000 7fe8c3000640 1 --2- 172.21.15.161:0/3265571565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8a0003b10 0x7fe8bc13e460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009000 tx=0x7fe8a0005800 comp rx=0 tx=0).stop 2024-05-22T21:19:53.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.698+0000 7fe8c3000640 1 -- 172.21.15.161:0/3265571565 shutdown_connections 2024-05-22T21:19:53.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.698+0000 7fe8c3000640 1 --2- 172.21.15.161:0/3265571565 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe898066690 0x7fe898068b50 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:53.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.698+0000 7fe8c3000640 1 --2- 172.21.15.161:0/3265571565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8bc0c42b0 0x7fe8bc0c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:53.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.698+0000 7fe8c3000640 1 --2- 172.21.15.161:0/3265571565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8bc0c3e30 0x7fe8bc0c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:53.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.698+0000 7fe8c3000640 1 --2- 172.21.15.161:0/3265571565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe8a0003b10 0x7fe8bc13e460 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:53.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.698+0000 7fe8c3000640 1 -- 172.21.15.161:0/3265571565 >> 172.21.15.161:0/3265571565 conn(0x7fe8bc0b0bc0 msgr2=0x7fe8bc0cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:53.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.698+0000 7fe8c3000640 1 -- 172.21.15.161:0/3265571565 shutdown_connections 2024-05-22T21:19:53.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:53.698+0000 7fe8c3000640 1 -- 172.21.15.161:0/3265571565 wait complete. 2024-05-22T21:19:53.722 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:19:53.722 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:19:53.722 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-22T21:19:53.722 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-22T21:19:53.722 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-22T21:19:53.723 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:19:53.723 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:19:53.723 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:19:53.723 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:19:53.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:53.723 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-22T21:19:53.734 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:19:53.734 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:19:53.734 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:19:53.734 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:53.734 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-22T21:19:53.735 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:53.735 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:19:53.735 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:53.735 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f0 2024-05-22T21:19:53.735 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:19:53.735 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:53.735 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f1 2024-05-22T21:19:53.735 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:19:53.735 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:53.736 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:19:53.736 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:53.736 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:19:53.736 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42091sec preferred_lft 42091sec 2024-05-22T21:19:53.736 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-22T21:19:53.736 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:53.736 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:19:53.736 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:bd:4e:25:3d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:53.736 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:19:53.737 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:53.737 INFO:teuthology.orchestra.run.smithi179.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:19:53.737 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:19:53.737 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:19:53.737 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:53.737 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:19:53.737 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:53.737 INFO:teuthology.orchestra.run.smithi179.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:19:53.737 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-22T21:19:53.737 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:19:53.738 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:53.738 INFO:teuthology.orchestra.run.smithi179.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:19:53.738 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-22T21:19:53.738 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:19:53.738 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:19:53.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:53.739 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:19:53.804 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-22T21:19:53.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:53.806 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-22T21:19:53.859 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-22T21:19:53.859 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:19:53.860 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:19:53.860 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:19:53.860 INFO:teuthology.orchestra.run.smithi179.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:19:53.860 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:19:53.860 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:19:53.860 INFO:teuthology.orchestra.run.smithi179.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:19:53.860 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:19:53.860 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:19:53.860 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:19:53.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:53.860 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:19:53.861 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:19:53.861 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:19:53.861 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:19:53.861 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:19:53.861 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:19:53.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:53.960 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-22T21:19:53.969 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-22T21:19:53.969 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-22T21:19:53.970 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:19:53.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:53.970 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-22T21:19:54.035 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:19:54.036 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-22T21:19:54.090 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:19:54.090 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:19:54.090 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:19:54.139 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:19:54.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:54.140 DEBUG:teuthology.orchestra.run.smithi179:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:19:54.234 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:19:54.234 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:19:54.234 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:19:54.249 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:19:54.250 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:19:54.250 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:19:54.250 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:19:54.250 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:19:54.257 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:19:54.258 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:19:54.258 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:19:54.258 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:19:54.258 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:19:54.258 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:19:54.258 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:19:54.258 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:19:54.262 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:19:54.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:54.262 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:19:54.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:54.282 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:19:54.297 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:19:54.301 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:19:54.339 INFO:teuthology.orchestra.run.smithi179.stdout:client6503 2024-05-22T21:19:54.339 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:19:54.339 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:54.423 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:19:54.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.558+0000 7f75edc00640 1 Processor -- start 2024-05-22T21:19:54.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.558+0000 7f75edc00640 1 -- start start 2024-05-22T21:19:54.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.558+0000 7f75edc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75e80715a0 0x7f75e815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:54.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.558+0000 7f75edc00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e81442a0 con 0x7f75e8143ec0 2024-05-22T21:19:54.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.558+0000 7f75edc00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e8070500 con 0x7f75e815ec90 2024-05-22T21:19:54.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.558+0000 7f75edc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e80707e0 con 0x7f75e80715a0 2024-05-22T21:19:54.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.558+0000 7f75e6a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f75e8143ec0 0x7f75e806fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48916/0 (socket says 172.21.15.161:48916) 2024-05-22T21:19:54.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.558+0000 7f75e6a00640 1 -- 172.21.15.161:0/337809664 learned_addr learned my addr 172.21.15.161:0/337809664 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:54.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.558+0000 7f75e6000640 1 --2- 172.21.15.161:0/337809664 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75e80715a0 0x7f75e815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:54.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.562+0000 7f75e5600640 1 -- 172.21.15.161:0/337809664 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701172946 0 0) 0x7f75e81442a0 con 0x7f75e8143ec0 2024-05-22T21:19:54.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.562+0000 7f75e5600640 1 -- 172.21.15.161:0/337809664 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75cc003530 con 0x7f75e8143ec0 2024-05-22T21:19:54.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.562+0000 7f75e5600640 1 -- 172.21.15.161:0/337809664 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558292725 0 0) 0x7f75e8070500 con 0x7f75e815ec90 2024-05-22T21:19:54.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.562+0000 7f75e5600640 1 -- 172.21.15.161:0/337809664 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e81442a0 con 0x7f75e815ec90 2024-05-22T21:19:54.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.562+0000 7f75e6000640 1 -- 172.21.15.161:0/337809664 >> v1:172.21.15.179:6789/0 conn(0x7f75e815ec90 legacy=0x7f75e80cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:54.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.562+0000 7f75e6000640 1 -- 172.21.15.161:0/337809664 >> v1:172.21.15.161:6789/0 conn(0x7f75e8143ec0 legacy=0x7f75e806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:54.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.562+0000 7f75e5600640 1 -- 172.21.15.161:0/337809664 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 737836873 0 0) 0x7f75cc003530 con 0x7f75e8143ec0 2024-05-22T21:19:54.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.562+0000 7f75e6000640 1 -- 172.21.15.161:0/337809664 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e8070ac0 con 0x7f75e80715a0 2024-05-22T21:19:54.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.562+0000 7f75e6000640 1 --2- 172.21.15.161:0/337809664 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75e80715a0 0x7f75e815f830 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f75d40097f0 tx=0x7f75d401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d78620f96fa8a22f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:54.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.562+0000 7f75e5600640 1 -- 172.21.15.161:0/337809664 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d40aa610 con 0x7f75e80715a0 2024-05-22T21:19:54.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.562+0000 7f75e5600640 1 -- 172.21.15.161:0/337809664 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d40a6040 con 0x7f75e80715a0 2024-05-22T21:19:54.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.562+0000 7f75e5600640 1 -- 172.21.15.161:0/337809664 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d40aa990 con 0x7f75e80715a0 2024-05-22T21:19:54.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.562+0000 7f75edc00640 1 -- 172.21.15.161:0/337809664 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75e80715a0 msgr2=0x7f75e815f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:54.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.562+0000 7f75edc00640 1 --2- 172.21.15.161:0/337809664 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75e80715a0 0x7f75e815f830 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f75d40097f0 tx=0x7f75d401c4b0 comp rx=0 tx=0).stop 2024-05-22T21:19:54.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75edc00640 1 -- 172.21.15.161:0/337809664 shutdown_connections 2024-05-22T21:19:54.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75edc00640 1 --2- 172.21.15.161:0/337809664 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75e80715a0 0x7f75e815f830 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:54.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75edc00640 1 -- 172.21.15.161:0/337809664 >> 172.21.15.161:0/337809664 conn(0x7f75e80b0bc0 msgr2=0x7f75e80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:54.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75edc00640 1 -- 172.21.15.161:0/337809664 shutdown_connections 2024-05-22T21:19:54.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75edc00640 1 -- 172.21.15.161:0/337809664 wait complete. 2024-05-22T21:19:54.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75edc00640 1 Processor -- start 2024-05-22T21:19:54.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75edc00640 1 -- start start 2024-05-22T21:19:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75edc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e8143ec0 0x7f75e815d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75edc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75e815ec90 0x7f75e8139650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75edc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75e815b6e0 0x7f75e81598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75edc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e815c0a0 con 0x7f75e815b6e0 2024-05-22T21:19:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75edc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e815c380 con 0x7f75e8143ec0 2024-05-22T21:19:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75edc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e815c660 con 0x7f75e815ec90 2024-05-22T21:19:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75e7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75e815b6e0 0x7f75e81598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75e7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75e815b6e0 0x7f75e81598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50874/0 (socket says 172.21.15.161:50874) 2024-05-22T21:19:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75e7400640 1 -- 172.21.15.161:0/2688028047 learned_addr learned my addr 172.21.15.161:0/2688028047 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75e6a00640 1 --2- 172.21.15.161:0/2688028047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e8143ec0 0x7f75e815d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:54.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75e6000640 1 --2- 172.21.15.161:0/2688028047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75e815ec90 0x7f75e8139650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:54.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75e7400640 1 -- 172.21.15.161:0/2688028047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75e815ec90 msgr2=0x7f75e8139650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:54.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75e7400640 1 --2- 172.21.15.161:0/2688028047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75e815ec90 0x7f75e8139650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:54.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75e7400640 1 -- 172.21.15.161:0/2688028047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e8143ec0 msgr2=0x7f75e815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:54.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75e7400640 1 --2- 172.21.15.161:0/2688028047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e8143ec0 0x7f75e815d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:54.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75e7400640 1 -- 172.21.15.161:0/2688028047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d4009420 con 0x7f75e815b6e0 2024-05-22T21:19:54.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75e6000640 1 --2- 172.21.15.161:0/2688028047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75e815ec90 0x7f75e8139650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:54.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75e6a00640 1 --2- 172.21.15.161:0/2688028047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e8143ec0 0x7f75e815d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:19:54.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75e7400640 1 --2- 172.21.15.161:0/2688028047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75e815b6e0 0x7f75e81598f0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f75e000c3c0 tx=0x7f75e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:54.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75dea00640 1 -- 172.21.15.161:0/2688028047 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e00052d0 con 0x7f75e815b6e0 2024-05-22T21:19:54.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75dea00640 1 -- 172.21.15.161:0/2688028047 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e0005430 con 0x7f75e815b6e0 2024-05-22T21:19:54.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.566+0000 7f75dea00640 1 -- 172.21.15.161:0/2688028047 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0003820 con 0x7f75e815b6e0 2024-05-22T21:19:54.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.570+0000 7f75edc00640 1 -- 172.21.15.161:0/2688028047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e815c940 con 0x7f75e815b6e0 2024-05-22T21:19:54.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.570+0000 7f75edc00640 1 -- 172.21.15.161:0/2688028047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e815d680 con 0x7f75e815b6e0 2024-05-22T21:19:54.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.570+0000 7f75edc00640 1 -- 172.21.15.161:0/2688028047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e8073af0 con 0x7f75e815b6e0 2024-05-22T21:19:54.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.570+0000 7f75dea00640 1 -- 172.21.15.161:0/2688028047 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f75e0003980 con 0x7f75e815b6e0 2024-05-22T21:19:54.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.570+0000 7f75dea00640 1 --2- 172.21.15.161:0/2688028047 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f75c80666c0 0x7f75c8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:54.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.570+0000 7f75dea00640 1 -- 172.21.15.161:0/2688028047 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75e0083800 con 0x7f75e815b6e0 2024-05-22T21:19:54.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.570+0000 7f75e6a00640 1 --2- 172.21.15.161:0/2688028047 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f75c80666c0 0x7f75c8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:54.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.570+0000 7f75e6a00640 1 --2- 172.21.15.161:0/2688028047 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f75c80666c0 0x7f75c8068b80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f75d80012f0 tx=0x7f75d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:54.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.574+0000 7f75dea00640 1 -- 172.21.15.161:0/2688028047 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f75e0050a40 con 0x7f75e815b6e0 2024-05-22T21:19:54.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.762+0000 7f75edc00640 1 -- 172.21.15.161:0/2688028047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75e8067220 con 0x7f75e815b6e0 2024-05-22T21:19:54.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.762+0000 7f75dea00640 1 -- 172.21.15.161:0/2688028047 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4745 (secure 0 0 0) 0x7f75e0017090 con 0x7f75e815b6e0 2024-05-22T21:19:54.766 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:54.766 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/524372753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":524372753},{"type":"v1","addr":"172.21.15.161:6836","nonce":524372753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3762119379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3762119379},{"type":"v1","addr":"172.21.15.179:6833","nonce":3762119379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3087241366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3087241366},{"type":"v1","addr":"172.21.15.179:6835","nonce":3087241366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:46.384984+0000","modified":"2024-05-22T21:19:49.400019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7572":{"gid":7572,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/3945715002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3945715002},{"type":"v1","addr":"172.21.15.161:6837","nonce":3945715002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-22T21:19:54.766 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 62 2024-05-22T21:19:54.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.766+0000 7f75edc00640 1 -- 172.21.15.161:0/2688028047 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f75c80666c0 msgr2=0x7f75c8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:54.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.766+0000 7f75edc00640 1 --2- 172.21.15.161:0/2688028047 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f75c80666c0 0x7f75c8068b80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f75d80012f0 tx=0x7f75d8009040 comp rx=0 tx=0).stop 2024-05-22T21:19:54.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.766+0000 7f75edc00640 1 -- 172.21.15.161:0/2688028047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75e815b6e0 msgr2=0x7f75e81598f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:54.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.766+0000 7f75edc00640 1 --2- 172.21.15.161:0/2688028047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75e815b6e0 0x7f75e81598f0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f75e000c3c0 tx=0x7f75e000c880 comp rx=0 tx=0).stop 2024-05-22T21:19:54.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.766+0000 7f75edc00640 1 -- 172.21.15.161:0/2688028047 shutdown_connections 2024-05-22T21:19:54.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.766+0000 7f75edc00640 1 --2- 172.21.15.161:0/2688028047 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f75c80666c0 0x7f75c8068b80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:54.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.766+0000 7f75edc00640 1 --2- 172.21.15.161:0/2688028047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75e815b6e0 0x7f75e81598f0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:54.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.766+0000 7f75edc00640 1 --2- 172.21.15.161:0/2688028047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f75e815ec90 0x7f75e8139650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:54.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.766+0000 7f75edc00640 1 --2- 172.21.15.161:0/2688028047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e8143ec0 0x7f75e815d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:54.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.766+0000 7f75edc00640 1 -- 172.21.15.161:0/2688028047 >> 172.21.15.161:0/2688028047 conn(0x7f75e80b0bc0 msgr2=0x7f75e8142f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:54.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.766+0000 7f75edc00640 1 -- 172.21.15.161:0/2688028047 shutdown_connections 2024-05-22T21:19:54.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:54.766+0000 7f75edc00640 1 -- 172.21.15.161:0/2688028047 wait complete. 2024-05-22T21:19:54.787 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-05-22T21:19:54.941 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":7572,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.539351796,"mdsmap_epoch":62,"osdmap_epoch":140,"osdmap_epoch_barrier":140,"uptime":13.789493273} 2024-05-22T21:19:54.941 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 7572, "mdsmap_epoch": 62, "osdmap_epoch": 140, "osdmap_epoch_barrier": 140, "rank_uptime": 8.539351796, "state": "up:active", "uptime": 13.789493273, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:19:54.941 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-05-22T21:19:55.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cbee00640 1 Processor -- start 2024-05-22T21:19:55.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cbee00640 1 -- start start 2024-05-22T21:19:55.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cbee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0cb8143a10 0x7f0cb806d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:55.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cbee00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb806d290 con 0x7f0cb8161510 2024-05-22T21:19:55.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cbee00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb80adf60 con 0x7f0cb8143630 2024-05-22T21:19:55.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cbee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb806cc40 con 0x7f0cb8143a10 2024-05-22T21:19:55.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cbce00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0cb8161510 0x7f0cb806fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48926/0 (socket says 172.21.15.161:48926) 2024-05-22T21:19:55.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cbce00640 1 -- 172.21.15.161:0/1229423196 learned_addr learned my addr 172.21.15.161:0/1229423196 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:55.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cb7400640 1 --2- 172.21.15.161:0/1229423196 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0cb8143a10 0x7f0cb806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:55.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cb6a00640 1 -- 172.21.15.161:0/1229423196 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190829875 0 0) 0x7f0cb806d290 con 0x7f0cb8161510 2024-05-22T21:19:55.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cb6a00640 1 -- 172.21.15.161:0/1229423196 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c9c003530 con 0x7f0cb8161510 2024-05-22T21:19:55.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cb6a00640 1 -- 172.21.15.161:0/1229423196 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652717020 0 0) 0x7f0cb80adf60 con 0x7f0cb8143630 2024-05-22T21:19:55.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cb6a00640 1 -- 172.21.15.161:0/1229423196 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cb806d290 con 0x7f0cb8143630 2024-05-22T21:19:55.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cb7400640 1 -- 172.21.15.161:0/1229423196 >> v1:172.21.15.179:6789/0 conn(0x7f0cb8143630 legacy=0x7f0cb8160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:55.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cb7400640 1 -- 172.21.15.161:0/1229423196 >> v1:172.21.15.161:6789/0 conn(0x7f0cb8161510 legacy=0x7f0cb806fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:55.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cb6a00640 1 -- 172.21.15.161:0/1229423196 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1723115790 0 0) 0x7f0c9c003530 con 0x7f0cb8161510 2024-05-22T21:19:55.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.078+0000 7f0cb7400640 1 -- 172.21.15.161:0/1229423196 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb806cf20 con 0x7f0cb8143a10 2024-05-22T21:19:55.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.082+0000 7f0cb7400640 1 --2- 172.21.15.161:0/1229423196 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0cb8143a10 0x7f0cb806d720 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40097f0 tx=0x7f0ca401c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eb01152f7c4e94f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:55.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.082+0000 7f0cb6a00640 1 -- 172.21.15.161:0/1229423196 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0001680 con 0x7f0cb8143a10 2024-05-22T21:19:55.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.082+0000 7f0cb6a00640 1 -- 172.21.15.161:0/1229423196 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca40a7050 con 0x7f0cb8143a10 2024-05-22T21:19:55.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.082+0000 7f0cb6a00640 1 -- 172.21.15.161:0/1229423196 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca40ab620 con 0x7f0cb8143a10 2024-05-22T21:19:55.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.082+0000 7f0cbee00640 1 -- 172.21.15.161:0/1229423196 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0cb8143a10 msgr2=0x7f0cb806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:55.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.082+0000 7f0cbee00640 1 --2- 172.21.15.161:0/1229423196 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0cb8143a10 0x7f0cb806d720 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40097f0 tx=0x7f0ca401c340 comp rx=0 tx=0).stop 2024-05-22T21:19:55.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.082+0000 7f0cbee00640 1 -- 172.21.15.161:0/1229423196 shutdown_connections 2024-05-22T21:19:55.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.082+0000 7f0cbee00640 1 --2- 172.21.15.161:0/1229423196 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0cb8143a10 0x7f0cb806d720 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:55.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.082+0000 7f0cbee00640 1 -- 172.21.15.161:0/1229423196 >> 172.21.15.161:0/1229423196 conn(0x7f0cb806a0a0 msgr2=0x7f0cb806a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:55.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.082+0000 7f0cbee00640 1 -- 172.21.15.161:0/1229423196 shutdown_connections 2024-05-22T21:19:55.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.082+0000 7f0cbee00640 1 -- 172.21.15.161:0/1229423196 wait complete. 2024-05-22T21:19:55.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbee00640 1 Processor -- start 2024-05-22T21:19:55.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbee00640 1 -- start start 2024-05-22T21:19:55.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0cb8143630 0x7f0cb8144970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:55.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cb8143a10 0x7f0cb8144eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:55.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbee00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cb8161510 0x7f0cb81fa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:55.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbee00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb8145c20 con 0x7f0cb8161510 2024-05-22T21:19:55.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb815af40 con 0x7f0cb8143a10 2024-05-22T21:19:55.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb81453f0 con 0x7f0cb8143630 2024-05-22T21:19:55.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cb8161510 0x7f0cb81fa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:55.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cb8161510 0x7f0cb81fa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50888/0 (socket says 172.21.15.161:50888) 2024-05-22T21:19:55.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbce00640 1 -- 172.21.15.161:0/1704130618 learned_addr learned my addr 172.21.15.161:0/1704130618 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:55.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cb7e00640 1 --2- 172.21.15.161:0/1704130618 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0cb8143630 0x7f0cb8144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:55.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cb7400640 1 --2- 172.21.15.161:0/1704130618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cb8143a10 0x7f0cb8144eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:55.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbce00640 1 -- 172.21.15.161:0/1704130618 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0cb8143630 msgr2=0x7f0cb8144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:55.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbce00640 1 --2- 172.21.15.161:0/1704130618 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0cb8143630 0x7f0cb8144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:55.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbce00640 1 -- 172.21.15.161:0/1704130618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cb8143a10 msgr2=0x7f0cb8144eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:55.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbce00640 1 --2- 172.21.15.161:0/1704130618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cb8143a10 0x7f0cb8144eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:55.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbce00640 1 -- 172.21.15.161:0/1704130618 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca4009420 con 0x7f0cb8161510 2024-05-22T21:19:55.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbce00640 1 --2- 172.21.15.161:0/1704130618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cb8161510 0x7f0cb81fa360 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0001ec0 tx=0x7f0cb000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:55.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cafe00640 1 -- 172.21.15.161:0/1704130618 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0005ae0 con 0x7f0cb8161510 2024-05-22T21:19:55.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cafe00640 1 -- 172.21.15.161:0/1704130618 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb0005c40 con 0x7f0cb8161510 2024-05-22T21:19:55.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cafe00640 1 -- 172.21.15.161:0/1704130618 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0007070 con 0x7f0cb8161510 2024-05-22T21:19:55.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbee00640 1 -- 172.21.15.161:0/1704130618 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb8145660 con 0x7f0cb8161510 2024-05-22T21:19:55.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.086+0000 7f0cbee00640 1 -- 172.21.15.161:0/1704130618 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb815abd0 con 0x7f0cb8161510 2024-05-22T21:19:55.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.090+0000 7f0cbee00640 1 -- 172.21.15.161:0/1704130618 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb815aa30 con 0x7f0cb8161510 2024-05-22T21:19:55.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.090+0000 7f0cafe00640 1 -- 172.21.15.161:0/1704130618 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0cb00071d0 con 0x7f0cb8161510 2024-05-22T21:19:55.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.090+0000 7f0cafe00640 1 --2- 172.21.15.161:0/1704130618 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0c98066690 0x7f0c98068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:55.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.090+0000 7f0cafe00640 1 -- 172.21.15.161:0/1704130618 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cb0083ca0 con 0x7f0cb8161510 2024-05-22T21:19:55.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.090+0000 7f0cb7e00640 1 --2- 172.21.15.161:0/1704130618 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0c98066690 0x7f0c98068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:55.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.090+0000 7f0cb7e00640 1 --2- 172.21.15.161:0/1704130618 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0c98066690 0x7f0c98068b50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8001570 tx=0x7f0ca8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:55.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.094+0000 7f0cafe00640 1 -- 172.21.15.161:0/1704130618 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0cb815aa30 con 0x7f0cb8161510 2024-05-22T21:19:55.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.250+0000 7f0cbee00640 1 -- 172.21.15.161:0/1704130618 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f0cb815b0f0 con 0x7f0cb8161510 2024-05-22T21:19:55.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.426+0000 7f0cafe00640 1 -- 172.21.15.161:0/1704130618 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v141) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0cb815b0f0 con 0x7f0cb8161510 2024-05-22T21:19:55.429 INFO:teuthology.orchestra.run.smithi161.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-05-22T21:19:55.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.426+0000 7f0cbee00640 1 -- 172.21.15.161:0/1704130618 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0c98066690 msgr2=0x7f0c98068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:55.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.426+0000 7f0cbee00640 1 --2- 172.21.15.161:0/1704130618 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0c98066690 0x7f0c98068b50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8001570 tx=0x7f0ca8009040 comp rx=0 tx=0).stop 2024-05-22T21:19:55.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.426+0000 7f0cbee00640 1 -- 172.21.15.161:0/1704130618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cb8161510 msgr2=0x7f0cb81fa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:55.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.426+0000 7f0cbee00640 1 --2- 172.21.15.161:0/1704130618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cb8161510 0x7f0cb81fa360 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0001ec0 tx=0x7f0cb000cbe0 comp rx=0 tx=0).stop 2024-05-22T21:19:55.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.426+0000 7f0cbee00640 1 -- 172.21.15.161:0/1704130618 shutdown_connections 2024-05-22T21:19:55.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.426+0000 7f0cbee00640 1 --2- 172.21.15.161:0/1704130618 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0c98066690 0x7f0c98068b50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:55.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.426+0000 7f0cbee00640 1 --2- 172.21.15.161:0/1704130618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cb8161510 0x7f0cb81fa360 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:55.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.426+0000 7f0cbee00640 1 --2- 172.21.15.161:0/1704130618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cb8143a10 0x7f0cb8144eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:55.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.426+0000 7f0cbee00640 1 --2- 172.21.15.161:0/1704130618 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0cb8143630 0x7f0cb8144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:55.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.426+0000 7f0cbee00640 1 -- 172.21.15.161:0/1704130618 >> 172.21.15.161:0/1704130618 conn(0x7f0cb806a0a0 msgr2=0x7f0cb80aeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:55.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.426+0000 7f0cbee00640 1 -- 172.21.15.161:0/1704130618 shutdown_connections 2024-05-22T21:19:55.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:55.430+0000 7f0cbee00640 1 -- 172.21.15.161:0/1704130618 wait complete. 2024-05-22T21:19:55.451 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-05-22T21:19:55.603 INFO:teuthology.orchestra.run.smithi161.stdout:{"osd_mon_report_interval":"5"} 2024-05-22T21:19:55.603 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-05-22T21:19:55.603 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2024-05-22T21:19:55.604 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:19:55.604 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:55.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:55.613 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2024-05-22T21:19:55.820 INFO:teuthology.orchestra.run.smithi161.stderr:16+0 records in 2024-05-22T21:19:55.820 INFO:teuthology.orchestra.run.smithi161.stderr:16+0 records out 2024-05-22T21:19:55.820 INFO:teuthology.orchestra.run.smithi161.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.149277 s, 112 MB/s 2024-05-22T21:19:55.821 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:19:55.822 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:19:55.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:19:55.830 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2024-05-22T21:19:56.059 INFO:teuthology.orchestra.run.smithi161.stderr:17+0 records in 2024-05-22T21:19:56.059 INFO:teuthology.orchestra.run.smithi161.stderr:17+0 records out 2024-05-22T21:19:56.059 INFO:teuthology.orchestra.run.smithi161.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.17479 s, 102 MB/s 2024-05-22T21:19:56.060 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-05-22T21:19:56.060 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:19:56.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.190+0000 7fcb83800640 1 Processor -- start 2024-05-22T21:19:56.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.190+0000 7fcb83800640 1 -- start start 2024-05-22T21:19:56.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.194+0000 7fcb83800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcb7c15dd50 0x7fcb7c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:56.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.194+0000 7fcb83800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb7c15e130 0x7fcb7c15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:56.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.194+0000 7fcb83800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb7c071670 con 0x7fcb7c15d410 2024-05-22T21:19:56.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.194+0000 7fcb83800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb7c15d8a0 con 0x7fcb7c15dd50 2024-05-22T21:19:56.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.194+0000 7fcb83800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c06de40 con 0x7fcb7c15e130 2024-05-22T21:19:56.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.194+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb7c15e130 0x7fcb7c15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:56.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.194+0000 7fcb81800640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fcb7c15d410 0x7fcb7c15a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:44268/0 (socket says 172.21.15.161:44268) 2024-05-22T21:19:56.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.194+0000 7fcb81800640 1 -- 172.21.15.161:0/3625928183 learned_addr learned my addr 172.21.15.161:0/3625928183 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:19:56.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.194+0000 7fcb80e00640 1 --2- 172.21.15.161:0/3625928183 >> v2:172.21.15.179:3300/0 conn(0x7fcb7c15dd50 0x7fcb7c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:56.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.194+0000 7fcb7be00640 1 -- 172.21.15.161:0/3625928183 >> v2:172.21.15.179:3300/0 conn(0x7fcb7c15dd50 msgr2=0x7fcb7c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:19:56.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.194+0000 7fcb7be00640 1 --2- 172.21.15.161:0/3625928183 >> v2:172.21.15.179:3300/0 conn(0x7fcb7c15dd50 0x7fcb7c06d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:56.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.194+0000 7fcb7be00640 1 -- 172.21.15.161:0/3625928183 >> v1:172.21.15.179:6789/0 conn(0x7fcb7c15d410 legacy=0x7fcb7c15a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:56.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.194+0000 7fcb7be00640 1 -- 172.21.15.161:0/3625928183 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c070a60 con 0x7fcb7c15e130 2024-05-22T21:19:56.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.194+0000 7fcb7b400640 1 -- 172.21.15.161:0/3625928183 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435448458 0 0) 0x7fcb7c071670 con 0x7fcb7c15d410 2024-05-22T21:19:56.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.198+0000 7fcb7be00640 1 --2- 172.21.15.161:0/3625928183 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb7c15e130 0x7fcb7c15ced0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcb680097c0 tx=0x7fcb6801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fa09f6b36032e34 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:56.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.198+0000 7fcb7b400640 1 -- 172.21.15.161:0/3625928183 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb680ac640 con 0x7fcb7c15e130 2024-05-22T21:19:56.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.198+0000 7fcb7b400640 1 -- 172.21.15.161:0/3625928183 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb680ac7a0 con 0x7fcb7c15e130 2024-05-22T21:19:56.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.198+0000 7fcb7b400640 1 -- 172.21.15.161:0/3625928183 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb680acb30 con 0x7fcb7c15e130 2024-05-22T21:19:56.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.198+0000 7fcb83800640 1 -- 172.21.15.161:0/3625928183 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb7c15e130 msgr2=0x7fcb7c15ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:56.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.198+0000 7fcb83800640 1 --2- 172.21.15.161:0/3625928183 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb7c15e130 0x7fcb7c15ced0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcb680097c0 tx=0x7fcb6801c3e0 comp rx=0 tx=0).stop 2024-05-22T21:19:56.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.198+0000 7fcb83800640 1 -- 172.21.15.161:0/3625928183 shutdown_connections 2024-05-22T21:19:56.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.198+0000 7fcb83800640 1 --2- 172.21.15.161:0/3625928183 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb7c15e130 0x7fcb7c15ced0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:56.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.198+0000 7fcb83800640 1 --2- 172.21.15.161:0/3625928183 >> v2:172.21.15.179:3300/0 conn(0x7fcb7c15dd50 0x7fcb7c06d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:56.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.198+0000 7fcb83800640 1 -- 172.21.15.161:0/3625928183 >> 172.21.15.161:0/3625928183 conn(0x7fcb7c0b0bc0 msgr2=0x7fcb7c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:56.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.198+0000 7fcb83800640 1 -- 172.21.15.161:0/3625928183 shutdown_connections 2024-05-22T21:19:56.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.198+0000 7fcb83800640 1 -- 172.21.15.161:0/3625928183 wait complete. 2024-05-22T21:19:56.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.198+0000 7fcb83800640 1 Processor -- start 2024-05-22T21:19:56.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.198+0000 7fcb83800640 1 -- start start 2024-05-22T21:19:56.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb83800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb7c15d410 0x7fcb7c1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:56.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb83800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb7c15dd50 0x7fcb7c1fa090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:56.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb83800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb7c15e130 0x7fcb7c141850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:56.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb83800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c1fa5d0 con 0x7fcb7c15d410 2024-05-22T21:19:56.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb83800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c1556f0 con 0x7fcb7c15e130 2024-05-22T21:19:56.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb83800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c141d90 con 0x7fcb7c15dd50 2024-05-22T21:19:56.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb80e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb7c15d410 0x7fcb7c1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:56.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb80e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb7c15d410 0x7fcb7c1567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43508/0 (socket says 172.21.15.161:43508) 2024-05-22T21:19:56.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb80e00640 1 -- 172.21.15.161:0/3661353184 learned_addr learned my addr 172.21.15.161:0/3661353184 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:19:56.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb7be00640 1 --2- 172.21.15.161:0/3661353184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb7c15dd50 0x7fcb7c1fa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:56.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb81800640 1 --2- 172.21.15.161:0/3661353184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb7c15e130 0x7fcb7c141850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:56.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb80e00640 1 -- 172.21.15.161:0/3661353184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb7c15dd50 msgr2=0x7fcb7c1fa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:56.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb80e00640 1 --2- 172.21.15.161:0/3661353184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb7c15dd50 0x7fcb7c1fa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:56.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb80e00640 1 -- 172.21.15.161:0/3661353184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb7c15e130 msgr2=0x7fcb7c141850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:56.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb80e00640 1 --2- 172.21.15.161:0/3661353184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb7c15e130 0x7fcb7c141850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:56.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb80e00640 1 -- 172.21.15.161:0/3661353184 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb68009420 con 0x7fcb7c15d410 2024-05-22T21:19:56.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb80e00640 1 --2- 172.21.15.161:0/3661353184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb7c15d410 0x7fcb7c1567e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c00e870 tx=0x7fcb6c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:56.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb78c00640 1 -- 172.21.15.161:0/3661353184 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c009510 con 0x7fcb7c15d410 2024-05-22T21:19:56.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb78c00640 1 -- 172.21.15.161:0/3661353184 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c00eec0 con 0x7fcb7c15d410 2024-05-22T21:19:56.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb78c00640 1 -- 172.21.15.161:0/3661353184 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c00cb40 con 0x7fcb7c15d410 2024-05-22T21:19:56.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb83800640 1 -- 172.21.15.161:0/3661353184 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb7c156ee0 con 0x7fcb7c15d410 2024-05-22T21:19:56.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb83800640 1 -- 172.21.15.161:0/3661353184 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7c1550a0 con 0x7fcb7c15d410 2024-05-22T21:19:56.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb83800640 1 -- 172.21.15.161:0/3661353184 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb7c154a00 con 0x7fcb7c15d410 2024-05-22T21:19:56.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.202+0000 7fcb78c00640 1 -- 172.21.15.161:0/3661353184 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcb6c00cca0 con 0x7fcb7c15d410 2024-05-22T21:19:56.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.206+0000 7fcb78c00640 1 --2- 172.21.15.161:0/3661353184 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcb5c066690 0x7fcb5c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:19:56.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.206+0000 7fcb78c00640 1 -- 172.21.15.161:0/3661353184 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb6c084200 con 0x7fcb7c15d410 2024-05-22T21:19:56.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.206+0000 7fcb7be00640 1 --2- 172.21.15.161:0/3661353184 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcb5c066690 0x7fcb5c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:19:56.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.206+0000 7fcb7be00640 1 --2- 172.21.15.161:0/3661353184 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcb5c066690 0x7fcb5c068b50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcb680ae000 tx=0x7fcb68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:19:56.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.210+0000 7fcb78c00640 1 -- 172.21.15.161:0/3661353184 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcb6c051600 con 0x7fcb7c15d410 2024-05-22T21:19:56.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.362+0000 7fcb83800640 1 -- 172.21.15.161:0/3661353184 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fcb7c067220 con 0x7fcb7c15d410 2024-05-22T21:19:56.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.366+0000 7fcb78c00640 1 -- 172.21.15.161:0/3661353184 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v141) v1 ==== 81+0+33550 (secure 0 0 0) 0x7fcb6c054c20 con 0x7fcb7c15d410 2024-05-22T21:19:56.368 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:19:56.368 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:19:56.368 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 141, 2024-05-22T21:19:56.368 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:19:56.368 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:19:56.369 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:19:55.423659+0000", 2024-05-22T21:19:56.369 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:19:56.369 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:19:56.369 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:19:56.369 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:19:56.369 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:19:56.369 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:19:56.369 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:19:56.369 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:19:56.369 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:19:56.369 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:19:56.369 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:19:56.369 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:19:56.370 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:19:56.370 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:19:56.370 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:19:56.370 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 17, 2024-05-22T21:19:56.370 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:19:56.370 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:19:56.371 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:19:56.371 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:19:56.371 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:19:56.371 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:19:56.371 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.371 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:19:56.371 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:19:56.371 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:19:56.371 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:19:56.372 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:19:56.372 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:19:56.372 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:19:56.372 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:19:56.372 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:19:56.372 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:19:56.372 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:19:56.372 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:19:56.372 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:19:56.372 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:19:56.372 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:19:56.372 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:19:56.372 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:19:56.373 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:19:56.373 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:19:56.373 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:19:56.373 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:19:56.373 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:19:56.373 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:19:56.373 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:19:56.373 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:19:56.373 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:19:56.373 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:19:56.373 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.373 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:19:56.373 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:19:56.374 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:19:56.374 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:19:56.374 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:19:56.374 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:19:56.374 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:19:56.374 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:19:56.374 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:19:56.374 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:19:56.374 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:19:56.374 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:19:56.374 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:19:56.374 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:19:56.375 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:19:56.375 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:19:56.375 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:19:56.375 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:19:56.375 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:19:56.375 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:19:56.375 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:19:56.375 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:19:56.375 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:19:56.375 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:19:56.375 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:19:56.375 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:19:56.375 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:19:56.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.376 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:19:56.376 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:19:56.376 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:19:56.376 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:19:56.376 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:19:56.376 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:19:56.376 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:19:56.376 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:19:56.376 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:19:56.376 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:19:56.376 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:19:56.377 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:19:56.377 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:19:56.377 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:19:56.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.377 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:19:56.377 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:19:56.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.377 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:19:56.377 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:19:56.377 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:19:56.377 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:19:56.377 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:19:56.377 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:19:56.378 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:19:56.378 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:19:56.378 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:19:56.378 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.378 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.378 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 16, 2024-05-22T21:19:56.378 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:19:56.378 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:19:44.080742+0000", 2024-05-22T21:19:56.378 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:19:56.378 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:19:56.378 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:19:56.378 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:19:56.379 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:19:56.379 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:19:56.379 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:19:56.379 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:19:56.379 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:19:56.379 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:19:56.379 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:19:56.379 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:19:56.379 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:19:56.379 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:19:56.379 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:19:56.379 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:19:56.379 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:19:56.380 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:19:56.380 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:19:56.380 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:19:56.380 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:19:56.380 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:19:56.380 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:19:56.380 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:19:56.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.380 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "139", 2024-05-22T21:19:56.380 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:19:56.380 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:19:56.380 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:19:56.380 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:19:56.381 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:19:56.381 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:19:56.381 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:19:56.381 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:19:56.381 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:19:56.381 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:19:56.381 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:19:56.381 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:19:56.381 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:19:56.381 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:19:56.381 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:19:56.381 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:19:56.381 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:19:56.381 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:19:56.382 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:19:56.382 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:19:56.382 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:19:56.382 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:19:56.382 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:19:56.382 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:19:56.382 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:19:56.382 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:19:56.382 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.382 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:19:56.382 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:19:56.382 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:19:56.383 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:19:56.383 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:19:56.383 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:19:56.383 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:19:56.383 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:19:56.383 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:19:56.383 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:19:56.383 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:19:56.383 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:19:56.383 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:19:56.383 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:19:56.383 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:19:56.383 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.384 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:19:56.384 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:19:56.384 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:19:56.384 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.384 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:19:56.384 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:19:56.384 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:19:56.384 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:19:56.384 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:19:56.384 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:19:56.384 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:19:56.384 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:19:56.385 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:19:56.385 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.385 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.385 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 17, 2024-05-22T21:19:56.385 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:19:56.385 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:19:45.388618+0000", 2024-05-22T21:19:56.385 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:19:56.385 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:19:56.385 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:19:56.385 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:19:56.385 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:19:56.385 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:19:56.386 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:19:56.386 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:19:56.386 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:19:56.386 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:19:56.386 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:19:56.386 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:19:56.386 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:19:56.386 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:19:56.386 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:19:56.386 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:19:56.386 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:19:56.386 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:19:56.386 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:19:56.387 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:19:56.387 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:19:56.387 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:19:56.387 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:19:56.387 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:19:56.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.387 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "141", 2024-05-22T21:19:56.387 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:19:56.387 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:19:56.387 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:19:56.387 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:19:56.387 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:19:56.387 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:19:56.387 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:19:56.388 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:19:56.388 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:19:56.388 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 33554432, 2024-05-22T21:19:56.388 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:19:56.388 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:19:56.388 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:19:56.388 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:19:56.388 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:19:56.388 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:19:56.388 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:19:56.388 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:19:56.388 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:19:56.388 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:19:56.388 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:19:56.388 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:19:56.389 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:19:56.389 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:19:56.389 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:19:56.389 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:19:56.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.389 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:19:56.389 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:19:56.389 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:19:56.389 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:19:56.389 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:19:56.389 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:19:56.389 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:19:56.389 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:19:56.389 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:19:56.390 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:19:56.390 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:19:56.390 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:19:56.390 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:19:56.390 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:19:56.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.390 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:19:56.390 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:19:56.390 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:19:56.390 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.390 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:19:56.390 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:19:56.391 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:19:56.391 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:19:56.391 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:19:56.391 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:19:56.391 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:19:56.391 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:19:56.391 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:19:56.391 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.391 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.391 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:19:56.391 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:19:56.391 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.391 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:19:56.391 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:19:56.392 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:19:56.392 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:19:56.392 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:19:56.392 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:19:56.392 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:19:56.392 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:19:56.392 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:19:56.392 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:19:56.392 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:19:56.392 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:19:56.392 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:19:56.392 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.392 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.392 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.393 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:19:56.393 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:19:56.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.393 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.393 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.393 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:19:56.393 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:19:56.393 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.393 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.393 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:19:56.393 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.394 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.394 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.394 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:19:56.394 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:19:56.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.394 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.394 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.394 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:19:56.394 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:19:56.395 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.395 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.395 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:19:56.395 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.395 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.395 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.395 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:19:56.395 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:19:56.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.395 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.395 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.395 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:19:56.396 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:19:56.396 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.396 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.396 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:19:56.396 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.396 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.396 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.396 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:19:56.396 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:19:56.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.396 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.396 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.396 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:19:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:19:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:19:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:19:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:19:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:19:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:19:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:19:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:19:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.397 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:19:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:19:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:19:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:19:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:19:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:19:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:19:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:19:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:19:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:19:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:19:56.398 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:19:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:19:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:19:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:19:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:19:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:19:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.399 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:19:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:19:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:19:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:19:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:19:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.400 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:19:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:19:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:19:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:19:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:19:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.401 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:19:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:19:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:19:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:19:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:19:56.402 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:19:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:19:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:19:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:19:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:19:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:19:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:19:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.403 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:19:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:19:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:19:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:19:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:19:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:19:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:19:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:19:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:19:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:19:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:19:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:19:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:19:56.404 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:19:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:19:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:19:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:19:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:19:56.405 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.406 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.406 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.406 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:19:56.406 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:19:56.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.406 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.406 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.406 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:19:56.406 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:19:56.406 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.406 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:19:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:19:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:19:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:19:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:19:56.407 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:19:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:19:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:19:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:19:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:19:56.408 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:19:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:19:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:19:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:19:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:19:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:19:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:19:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.409 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:19:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:19:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:19:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:19:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:19:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:19:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:19:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:19:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:19:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:19:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:19:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:19:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:19:56.410 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:19:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:19:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:19:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:19:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.411 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:19:56.413 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.413 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:19:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:19:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:19:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:19:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:19:56.414 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:19:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:19:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:19:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:19:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:19:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:19:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:19:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:19:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:19:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.416 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:19:56.417 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:19:56.417 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:19:56.417 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:19:56.417 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:19:56.417 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:19:56.417 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:19:56.417 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:19:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:19:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:19:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:19:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:19:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:19:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:19:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:19:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:19:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:19:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:19:56.418 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:19:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:19:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:19:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:19:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:19:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:19:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:19:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:19:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:19:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:19:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:19:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.420 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:19:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:19:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:19:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:19:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:19:56.421 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:19:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:19:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:19:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:19:56.422 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:19:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:19:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:19:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:19:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:19:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:19:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:19:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:19:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:19:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:19:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:19:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:19:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:19:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:19:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:19:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:19:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:19:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:19:56.424 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:19:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:19:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:19:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:19:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:19:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:19:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:19:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.425 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.426 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:19:56.426 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.426 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.426 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:19:56.426 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:19:56.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.426 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.426 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:19:56.426 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:19:56.427 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.427 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.427 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:19:56.427 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.427 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.427 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:19:56.427 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:19:56.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.427 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.427 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:19:56.427 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:19:56.428 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.428 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.428 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:19:56.428 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.428 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.428 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:19:56.428 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:19:56.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.428 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.428 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:19:56.429 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:19:56.429 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.429 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.429 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:19:56.429 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:19:56.429 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:19:56.429 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:19:56.429 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:19:56.429 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:19:56.429 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:19:56.429 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.430 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:19:56.430 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:19:56.430 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:19:56.430 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:19:56.430 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:19:56.430 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:19:56.430 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:19:56.430 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:19:56.430 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:19:56.430 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:19:56.430 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:19:56.430 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:19:56.431 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:19:56.431 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.431 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.431 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:19:56.431 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:19:56.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.431 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.431 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:19:56.431 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:19:56.431 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.431 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.432 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:19:56.432 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.432 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.432 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:19:56.432 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:19:56.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.432 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.432 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:19:56.432 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:19:56.432 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.432 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.433 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:19:56.433 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.433 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.433 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:19:56.433 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:19:56.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.433 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.433 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:19:56.433 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:19:56.433 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.433 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.434 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:19:56.434 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.434 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.434 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.434 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:19:56.434 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:19:56.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.434 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.434 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.434 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:19:56.434 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:19:56.434 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.435 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.435 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:19:56.435 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:19:56.435 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:19:56.435 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:19:56.435 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:19:56.435 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:19:56.435 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:19:56.435 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.435 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.435 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:19:56.435 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:19:56.436 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:19:56.436 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:19:56.436 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:19:56.436 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:19:56.436 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:19:56.436 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:19:56.436 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:19:56.436 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:19:56.436 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:19:56.436 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:19:56.436 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:19:56.436 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.436 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.437 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:19:56.437 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:19:56.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.437 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.437 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:19:56.437 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:19:56.437 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.437 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.437 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:19:56.437 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.437 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.438 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:19:56.438 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:19:56.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.438 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.438 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:19:56.438 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:19:56.438 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.438 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.438 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:19:56.438 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.439 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.439 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:19:56.439 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:19:56.439 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.439 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.439 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.439 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:19:56.439 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:19:56.439 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.439 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.439 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.439 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:19:56.439 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:19:56.439 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.440 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:19:56.440 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:19:56.440 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:19:56.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.440 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.440 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:19:56.440 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:19:56.440 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:19:56.440 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.440 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.440 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.440 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:19:56.440 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:19:56.441 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:19:56.441 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:19:56.441 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:19:56.441 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:19:56.441 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:19:56.441 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:19:56.441 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.441 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:19:56.441 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:19:56.441 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.441 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:19:56.441 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:19:56.441 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:19:56.441 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:19:56.442 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:19:56.442 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:19:56.442 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:19:56.442 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:19:56.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.442 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.442 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:19:56.442 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:19:56.442 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:19:56.442 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:19:56.442 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:19:56.442 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:19:56.442 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:19:56.443 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:19:56.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.443 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.443 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:19:56.443 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:19:56.443 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:19:56.443 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:19:56.443 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:19:56.443 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:19:56.443 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:19:56.443 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:19:56.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.443 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.443 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:19:56.443 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:19:56.444 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:19:56.444 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:19:56.444 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:19:56.444 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:19:56.444 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:19:56.444 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:19:56.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.444 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.444 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:19:56.444 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:19:56.444 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:19:56.444 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:19:56.444 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:19:56.445 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:19:56.445 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:19:56.445 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:19:56.445 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.445 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.445 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:19:56.445 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:19:56.445 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:19:56.445 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:19:56.445 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:19:56.445 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:19:56.445 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:19:56.445 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:19:56.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.446 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:19:56.446 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:19:56.446 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:19:56.446 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:19:56.446 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:19:56.446 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:19:56.446 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:19:56.446 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:19:56.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.446 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:19:56.446 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:19:56.447 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:19:56.447 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:19:56.447 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:19:56.447 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:19:56.447 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:19:56.447 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:19:56.447 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:19:56.447 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.447 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:19:56.447 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:19:56.447 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:19:56.447 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:19:56.447 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:19:56.448 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:19:56.448 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:19:56.448 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:19:56.448 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:19:56.448 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:19:56.448 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:19:56.448 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:19:56.448 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:19:56.448 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:19:56.448 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:19:56.448 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:19:56.448 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:19:56.448 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:19:56.449 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:19:56.449 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:19:56.449 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:19:56.449 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:19:56.449 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:19:56.449 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:19:56.449 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:19:56.449 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:19:56.449 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:19:56.449 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:19:56.449 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:19:56.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.366+0000 7fcb83800640 1 -- 172.21.15.161:0/3661353184 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcb5c066690 msgr2=0x7fcb5c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:56.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.366+0000 7fcb83800640 1 --2- 172.21.15.161:0/3661353184 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcb5c066690 0x7fcb5c068b50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcb680ae000 tx=0x7fcb68008040 comp rx=0 tx=0).stop 2024-05-22T21:19:56.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.366+0000 7fcb83800640 1 -- 172.21.15.161:0/3661353184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb7c15d410 msgr2=0x7fcb7c1567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:19:56.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.366+0000 7fcb83800640 1 --2- 172.21.15.161:0/3661353184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb7c15d410 0x7fcb7c1567e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c00e870 tx=0x7fcb6c00ec70 comp rx=0 tx=0).stop 2024-05-22T21:19:56.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.366+0000 7fcb83800640 1 -- 172.21.15.161:0/3661353184 shutdown_connections 2024-05-22T21:19:56.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.366+0000 7fcb83800640 1 --2- 172.21.15.161:0/3661353184 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcb5c066690 0x7fcb5c068b50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:56.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.366+0000 7fcb83800640 1 --2- 172.21.15.161:0/3661353184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb7c15e130 0x7fcb7c141850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:56.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.366+0000 7fcb83800640 1 --2- 172.21.15.161:0/3661353184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb7c15dd50 0x7fcb7c1fa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:56.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.366+0000 7fcb83800640 1 --2- 172.21.15.161:0/3661353184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb7c15d410 0x7fcb7c1567e0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:19:56.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.366+0000 7fcb83800640 1 -- 172.21.15.161:0/3661353184 >> 172.21.15.161:0/3661353184 conn(0x7fcb7c0b0bc0 msgr2=0x7fcb7c1542f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:19:56.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.366+0000 7fcb83800640 1 -- 172.21.15.161:0/3661353184 shutdown_connections 2024-05-22T21:19:56.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:19:56.366+0000 7fcb83800640 1 -- 172.21.15.161:0/3661353184 wait complete. 2024-05-22T21:19:56.452 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-05-22T21:20:01.453 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:20:01.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.598+0000 7f564ae00640 1 Processor -- start 2024-05-22T21:20:01.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.598+0000 7f564ae00640 1 -- start start 2024-05-22T21:20:01.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.598+0000 7f564ae00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f564415e130 0x7f564415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:01.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.598+0000 7f564ae00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f564415d410 0x7f564415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:01.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.598+0000 7f564ae00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5644071670 con 0x7f564415dd50 2024-05-22T21:20:01.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.598+0000 7f564ae00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f564415d870 con 0x7f564415d410 2024-05-22T21:20:01.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.598+0000 7f564ae00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5644070a60 con 0x7f564415e130 2024-05-22T21:20:01.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.598+0000 7f5648e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f564415d410 0x7f564415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:01.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.598+0000 7f5648e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f564415d410 0x7f564415b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43512/0 (socket says 172.21.15.161:43512) 2024-05-22T21:20:01.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.598+0000 7f5648e00640 1 -- 172.21.15.161:0/2522682163 learned_addr learned my addr 172.21.15.161:0/2522682163 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:01.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.598+0000 7f563b400640 1 --2- 172.21.15.161:0/2522682163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f564415e130 0x7f564415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:01.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.598+0000 7f5648e00640 1 -- 172.21.15.161:0/2522682163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f564415e130 msgr2=0x7f564415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:01.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.598+0000 7f5648e00640 1 --2- 172.21.15.161:0/2522682163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f564415e130 0x7f564415ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:01.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.598+0000 7f5648e00640 1 -- 172.21.15.161:0/2522682163 >> v1:172.21.15.179:6789/0 conn(0x7f564415dd50 legacy=0x7f564406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:01.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.598+0000 7f5648e00640 1 -- 172.21.15.161:0/2522682163 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5644070d40 con 0x7f564415d410 2024-05-22T21:20:01.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.598+0000 7f563b400640 1 --2- 172.21.15.161:0/2522682163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f564415e130 0x7f564415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:01.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.602+0000 7f5648e00640 1 --2- 172.21.15.161:0/2522682163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f564415d410 0x7f564415b6c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f563c009870 tx=0x7f563c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbe38f97c24d4ca server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:01.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.602+0000 7f5643400640 1 -- 172.21.15.161:0/2522682163 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c0ab620 con 0x7f564415d410 2024-05-22T21:20:01.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.602+0000 7f5643400640 1 -- 172.21.15.161:0/2522682163 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563c0ab780 con 0x7f564415d410 2024-05-22T21:20:01.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.602+0000 7f5643400640 1 -- 172.21.15.161:0/2522682163 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c0abb10 con 0x7f564415d410 2024-05-22T21:20:01.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.602+0000 7f564ae00640 1 -- 172.21.15.161:0/2522682163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f564415d410 msgr2=0x7f564415b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:01.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.602+0000 7f564ae00640 1 --2- 172.21.15.161:0/2522682163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f564415d410 0x7f564415b6c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f563c009870 tx=0x7f563c01c460 comp rx=0 tx=0).stop 2024-05-22T21:20:01.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.602+0000 7f564ae00640 1 -- 172.21.15.161:0/2522682163 shutdown_connections 2024-05-22T21:20:01.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.602+0000 7f564ae00640 1 --2- 172.21.15.161:0/2522682163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f564415d410 0x7f564415b6c0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:01.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.602+0000 7f564ae00640 1 --2- 172.21.15.161:0/2522682163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f564415e130 0x7f564415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:01.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.602+0000 7f564ae00640 1 -- 172.21.15.161:0/2522682163 >> 172.21.15.161:0/2522682163 conn(0x7f56440b0bc0 msgr2=0x7f56440b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:01.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.602+0000 7f564ae00640 1 -- 172.21.15.161:0/2522682163 shutdown_connections 2024-05-22T21:20:01.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.602+0000 7f564ae00640 1 -- 172.21.15.161:0/2522682163 wait complete. 2024-05-22T21:20:01.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f564ae00640 1 Processor -- start 2024-05-22T21:20:01.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f564ae00640 1 -- start start 2024-05-22T21:20:01.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f564ae00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f564415d410 0x7f564413de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:01.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f564ae00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f564415dd50 0x7f56440b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:01.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f564ae00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f564415e130 0x7f56441f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:01.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f564ae00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564413d9c0 con 0x7f564415dd50 2024-05-22T21:20:01.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f564ae00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56441f9e00 con 0x7f564415e130 2024-05-22T21:20:01.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f564ae00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56441f9b20 con 0x7f564415d410 2024-05-22T21:20:01.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f563b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f564415dd50 0x7f56440b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:01.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f5643e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f564415d410 0x7f564413de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:01.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f563b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f564415dd50 0x7f56440b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43516/0 (socket says 172.21.15.161:43516) 2024-05-22T21:20:01.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f563b400640 1 -- 172.21.15.161:0/1122689858 learned_addr learned my addr 172.21.15.161:0/1122689858 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:01.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f5648e00640 1 --2- 172.21.15.161:0/1122689858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f564415e130 0x7f56441f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:01.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f563b400640 1 -- 172.21.15.161:0/1122689858 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f564415d410 msgr2=0x7f564413de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:01.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f563b400640 1 --2- 172.21.15.161:0/1122689858 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f564415d410 0x7f564413de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:01.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f563b400640 1 -- 172.21.15.161:0/1122689858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f564415e130 msgr2=0x7f56441f8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:01.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f563b400640 1 --2- 172.21.15.161:0/1122689858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f564415e130 0x7f56441f8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:01.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f563b400640 1 -- 172.21.15.161:0/1122689858 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c0094d0 con 0x7f564415dd50 2024-05-22T21:20:01.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f563b400640 1 --2- 172.21.15.161:0/1122689858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f564415dd50 0x7f56440b1430 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f562c00c7f0 tx=0x7f562c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:01.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f5640c00640 1 -- 172.21.15.161:0/1122689858 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c007820 con 0x7f564415dd50 2024-05-22T21:20:01.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f5640c00640 1 -- 172.21.15.161:0/1122689858 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c007980 con 0x7f564415dd50 2024-05-22T21:20:01.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f5640c00640 1 -- 172.21.15.161:0/1122689858 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c007d30 con 0x7f564415dd50 2024-05-22T21:20:01.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f564ae00640 1 -- 172.21.15.161:0/1122689858 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564413d370 con 0x7f564415dd50 2024-05-22T21:20:01.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f564ae00640 1 -- 172.21.15.161:0/1122689858 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56441415b0 con 0x7f564415dd50 2024-05-22T21:20:01.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.606+0000 7f564ae00640 1 -- 172.21.15.161:0/1122689858 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5644067220 con 0x7f564415dd50 2024-05-22T21:20:01.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.610+0000 7f5640c00640 1 -- 172.21.15.161:0/1122689858 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f562c0113f0 con 0x7f564415dd50 2024-05-22T21:20:01.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.610+0000 7f5640c00640 1 --2- 172.21.15.161:0/1122689858 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5620066690 0x7f5620068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:01.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.610+0000 7f5640c00640 1 -- 172.21.15.161:0/1122689858 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f562c084440 con 0x7f564415dd50 2024-05-22T21:20:01.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.610+0000 7f5643e00640 1 --2- 172.21.15.161:0/1122689858 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5620066690 0x7f5620068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:01.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.610+0000 7f5640c00640 1 -- 172.21.15.161:0/1122689858 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=143}) v3 -- 0x7f562006e1c0 con 0x7f564415dd50 2024-05-22T21:20:01.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.610+0000 7f5643e00640 1 --2- 172.21.15.161:0/1122689858 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5620066690 0x7f5620068b50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f564413b110 tx=0x7f5634003dd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:01.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.614+0000 7f5640c00640 1 -- 172.21.15.161:0/1122689858 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f562c01f060 con 0x7f564415dd50 2024-05-22T21:20:01.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.766+0000 7f564ae00640 1 -- 172.21.15.161:0/1122689858 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5644159a60 con 0x7f564415dd50 2024-05-22T21:20:01.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.770+0000 7f5640c00640 1 -- 172.21.15.161:0/1122689858 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v142) v1 ==== 81+0+33569 (secure 0 0 0) 0x7f562c051850 con 0x7f564415dd50 2024-05-22T21:20:01.773 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:01.773 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:20:01.773 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 142, 2024-05-22T21:20:01.773 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:20:01.773 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:20:01.773 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:20:00.256219+0000", 2024-05-22T21:20:01.773 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:20:01.773 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:20:01.773 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:20:01.773 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:20:01.774 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:20:01.774 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:20:01.774 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:20:01.774 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:20:01.774 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:20:01.774 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:01.774 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:20:01.774 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:20:01.774 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:20:01.774 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:20:01.774 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:20:01.774 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 17, 2024-05-22T21:20:01.775 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:20:01.775 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:20:01.775 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:20:01.775 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:20:01.775 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:20:01.775 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:20:01.775 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.775 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:20:01.775 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:20:01.775 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:20:01.775 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:20:01.775 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:20:01.775 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:01.775 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:01.776 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:01.776 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:01.776 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:01.776 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:01.776 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:01.776 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:01.776 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:01.776 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:01.776 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:20:01.776 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:20:01.776 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:20:01.776 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:20:01.776 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:20:01.777 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:01.777 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:01.777 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:01.777 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:01.777 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:01.777 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:01.777 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:01.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.777 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:20:01.777 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:01.777 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:01.777 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:01.777 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:01.777 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:01.778 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:01.778 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:01.778 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:01.778 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:01.778 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:20:01.778 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:01.778 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:01.778 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:01.778 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:01.778 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:01.778 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:01.778 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:01.778 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:01.778 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:01.778 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:01.779 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:01.779 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:01.779 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:01.779 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:01.779 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:01.779 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:01.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.779 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:01.779 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:01.779 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:01.779 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:01.779 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:01.779 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:01.780 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:01.780 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:01.780 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:01.780 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:01.780 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:01.780 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:01.780 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:20:01.780 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:20:01.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.780 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:01.780 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:20:01.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.780 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:01.780 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:20:01.780 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:20:01.781 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:20:01.781 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:20:01.781 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:20:01.781 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:01.781 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:01.781 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:01.781 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.781 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.781 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 16, 2024-05-22T21:20:01.781 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:20:01.781 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:19:44.080742+0000", 2024-05-22T21:20:01.781 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:20:01.781 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:20:01.782 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:01.782 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:01.782 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:01.782 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:01.782 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:01.782 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:01.782 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:01.782 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:01.782 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:01.782 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:01.782 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:20:01.782 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:20:01.783 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:20:01.783 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:20:01.783 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:20:01.783 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:01.783 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:01.783 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:01.783 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:01.783 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:01.783 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:01.783 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:01.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.784 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "139", 2024-05-22T21:20:01.784 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:01.784 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:01.784 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:01.784 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:01.784 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:01.784 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:01.784 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:01.784 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:01.784 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:01.784 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:20:01.784 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:01.785 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:01.785 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:01.785 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:01.785 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:01.785 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:01.785 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:01.785 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:01.785 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:01.785 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:01.785 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:01.785 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:01.785 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:01.785 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:01.785 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:01.786 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:01.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.786 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:01.786 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:01.786 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:01.786 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:01.786 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:01.786 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:01.786 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:01.786 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:01.786 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:01.786 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:01.786 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:01.787 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:01.787 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:20:01.787 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:20:01.787 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:20:01.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.787 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:01.787 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:20:01.787 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:20:01.787 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.787 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:01.787 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:20:01.787 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:20:01.788 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:20:01.788 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:20:01.788 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:20:01.788 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:01.788 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:01.788 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:01.788 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.788 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.788 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 17, 2024-05-22T21:20:01.788 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:20:01.788 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:19:45.388618+0000", 2024-05-22T21:20:01.788 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1027, 2024-05-22T21:20:01.789 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full,full_quota", 2024-05-22T21:20:01.789 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:01.789 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:01.789 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:01.789 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:01.789 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:01.789 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:01.789 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:01.789 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:01.789 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:01.789 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:01.789 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:20:01.789 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:20:01.790 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:20:01.790 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:20:01.790 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:20:01.790 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:01.790 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:01.790 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:01.790 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:01.790 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:01.790 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:01.790 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:01.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.791 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "142", 2024-05-22T21:20:01.791 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:01.791 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:01.791 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:01.791 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:01.791 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:01.791 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:01.791 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:01.791 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:01.792 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:01.792 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 33554432, 2024-05-22T21:20:01.792 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:01.792 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:01.792 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:01.792 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:01.792 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:01.792 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:01.792 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:01.792 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:01.792 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:01.793 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:01.793 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:01.793 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:01.793 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:01.793 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:01.793 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:01.793 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:01.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.793 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:01.793 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:01.794 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:01.794 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:01.794 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:01.794 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:01.794 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:01.794 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:01.794 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:01.794 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:01.794 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:01.794 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:01.794 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:20:01.794 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:20:01.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.795 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:01.795 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:20:01.795 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:20:01.795 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.795 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:01.795 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:20:01.795 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:20:01.795 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:20:01.795 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:20:01.795 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:20:01.796 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:01.796 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:01.796 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:01.796 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.796 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.796 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:01.796 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:20:01.796 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.796 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:20:01.797 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:01.797 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:01.797 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:01.797 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:01.797 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:01.797 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:01.797 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:01.797 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:01.797 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:01.797 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:01.797 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:01.797 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:01.797 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.798 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.798 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.798 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:20:01.798 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:01.798 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.798 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.798 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.798 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:20:01.798 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:01.798 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.798 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.798 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.799 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:01.799 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.799 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.799 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.799 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:20:01.799 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:01.799 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.799 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.799 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.799 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:20:01.799 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:01.799 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.799 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.800 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:01.800 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.800 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.800 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.800 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:20:01.800 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:01.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.800 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.800 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.800 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:20:01.800 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:01.800 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.801 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.801 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.801 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:01.801 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.801 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.801 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.801 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:20:01.801 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:01.801 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.801 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.801 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.801 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:20:01.801 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:01.802 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.802 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.802 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.802 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:20:01.802 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:20:01.802 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:20:01.802 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:20:01.802 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:01.802 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:01.802 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:01.802 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.802 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.802 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.803 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:20:01.803 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:01.803 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:01.803 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:01.803 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:01.803 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:01.803 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:01.803 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:01.803 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:01.803 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:01.804 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:01.804 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:01.804 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:01.804 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.804 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.804 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.804 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:20:01.804 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:01.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.804 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.804 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.804 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:20:01.804 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:01.805 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.805 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.805 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:01.805 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.805 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.805 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.805 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:20:01.805 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:01.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.805 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.805 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.805 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:20:01.806 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:01.806 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.806 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.806 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.806 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:01.806 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.806 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.806 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.806 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:20:01.806 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:01.806 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.806 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.807 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.807 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:20:01.807 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:01.807 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.807 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.807 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.807 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:01.807 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.807 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.807 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.807 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:20:01.807 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:20:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:20:01.808 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:20:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:20:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:20:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:20:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:20:01.810 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:20:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.811 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:20:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:20:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:01.812 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:20:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:20:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.813 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:20:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:20:01.814 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:20:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:20:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:20:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:20:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.815 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:20:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:01.816 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:20:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:20:01.817 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:20:01.818 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:01.819 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.819 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.819 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.819 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:20:01.819 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:01.819 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.819 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.819 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.819 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:01.819 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.819 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:20:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:20:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.820 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:20:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:20:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.821 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:20:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:20:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:20:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:20:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.822 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:20:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:01.823 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:20:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:20:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.824 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:20:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:20:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.825 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:20:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:20:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:01.826 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:20:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:20:01.827 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:20:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:20:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:20:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:20:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.828 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:20:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:01.829 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:20:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:20:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.830 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:20:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:20:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.831 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:20:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:20:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.832 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:20:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:20:01.833 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:20:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:20:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:20:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:20:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.834 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.835 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:20:01.835 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:01.835 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:01.835 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:01.835 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:01.835 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:01.835 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:01.835 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:01.835 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:01.835 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:01.835 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:01.835 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:01.835 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:20:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:20:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:01.836 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:20:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.837 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:20:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:20:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.838 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:20:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:20:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:01.839 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:20:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:20:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:20:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:20:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:20:01.840 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:20:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:01.841 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:20:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.842 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:20:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:20:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.843 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:20:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:20:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:01.844 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:20:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:20:01.845 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:20:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:20:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:20:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:20:01.846 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:20:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:20:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:20:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:01.847 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:20:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:20:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:20:01.848 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:20:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:20:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:20:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:20:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:20:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:01.850 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:20:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:20:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:01.851 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:20:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:20:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:20:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:01.852 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:20:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:20:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:20:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:20:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:20:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:20:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:20:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:20:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:20:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:20:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:20:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:20:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:20:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:20:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:20:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:20:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:20:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:20:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:20:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:20:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:20:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:20:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:20:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:20:01.855 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:20:01.856 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:20:01.856 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:20:01.856 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:01.856 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:20:01.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.770+0000 7f564ae00640 1 -- 172.21.15.161:0/1122689858 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5620066690 msgr2=0x7f5620068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:01.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.770+0000 7f564ae00640 1 --2- 172.21.15.161:0/1122689858 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5620066690 0x7f5620068b50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f564413b110 tx=0x7f5634003dd0 comp rx=0 tx=0).stop 2024-05-22T21:20:01.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.770+0000 7f564ae00640 1 -- 172.21.15.161:0/1122689858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f564415dd50 msgr2=0x7f56440b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:01.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.770+0000 7f564ae00640 1 --2- 172.21.15.161:0/1122689858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f564415dd50 0x7f56440b1430 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f562c00c7f0 tx=0x7f562c00ccb0 comp rx=0 tx=0).stop 2024-05-22T21:20:01.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.770+0000 7f564ae00640 1 -- 172.21.15.161:0/1122689858 shutdown_connections 2024-05-22T21:20:01.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.770+0000 7f564ae00640 1 --2- 172.21.15.161:0/1122689858 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5620066690 0x7f5620068b50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:01.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.770+0000 7f564ae00640 1 --2- 172.21.15.161:0/1122689858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f564415e130 0x7f56441f8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:01.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.770+0000 7f564ae00640 1 --2- 172.21.15.161:0/1122689858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f564415dd50 0x7f56440b1430 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:01.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.770+0000 7f564ae00640 1 --2- 172.21.15.161:0/1122689858 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f564415d410 0x7f564413de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:01.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.770+0000 7f564ae00640 1 -- 172.21.15.161:0/1122689858 >> 172.21.15.161:0/1122689858 conn(0x7f56440b0bc0 msgr2=0x7f56440aec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:01.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.770+0000 7f564ae00640 1 -- 172.21.15.161:0/1122689858 shutdown_connections 2024-05-22T21:20:01.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:01.770+0000 7f564ae00640 1 -- 172.21.15.161:0/1122689858 wait complete. 2024-05-22T21:20:01.859 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-05-22T21:20:01.859 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:20:01.860 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:20:01.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:01.868 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2024-05-22T21:20:02.201 INFO:teuthology.orchestra.run.smithi161.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-05-22T21:20:02.202 INFO:teuthology.orchestra.run.smithi161.stderr:50+0 records in 2024-05-22T21:20:02.202 INFO:teuthology.orchestra.run.smithi161.stderr:50+0 records out 2024-05-22T21:20:02.202 INFO:teuthology.orchestra.run.smithi161.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.279288 s, 188 MB/s 2024-05-22T21:20:02.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:20:02.203 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:20:02.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.334+0000 7f6fe6800640 1 Processor -- start 2024-05-22T21:20:02.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.334+0000 7f6fe6800640 1 -- start start 2024-05-22T21:20:02.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.338+0000 7f6fe6800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fe0143ec0 0x7f6fe015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:02.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.338+0000 7f6fe6800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6fe00715a0 0x7f6fe006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:02.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.338+0000 7f6fe6800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6fe015ec90 0x7f6fe015f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:02.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.338+0000 7f6fe6800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe015f660 con 0x7f6fe015ec90 2024-05-22T21:20:02.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.338+0000 7f6fe6800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe01442a0 con 0x7f6fe00715a0 2024-05-22T21:20:02.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.338+0000 7f6fe6800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe0070260 con 0x7f6fe0143ec0 2024-05-22T21:20:02.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.338+0000 7f6fdfe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6fe015ec90 0x7f6fe015f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:02.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.338+0000 7f6fdfe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6fe015ec90 0x7f6fe015f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43526/0 (socket says 172.21.15.161:43526) 2024-05-22T21:20:02.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.338+0000 7f6fdfe00640 1 -- 172.21.15.161:0/3484035227 learned_addr learned my addr 172.21.15.161:0/3484035227 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:02.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.338+0000 7f6fdf400640 1 --2- 172.21.15.161:0/3484035227 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fe0143ec0 0x7f6fe015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:02.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.338+0000 7f6fdea00640 1 --2- 172.21.15.161:0/3484035227 >> v2:172.21.15.179:3300/0 conn(0x7f6fe00715a0 0x7f6fe006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:02.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.338+0000 7f6fdfe00640 1 -- 172.21.15.161:0/3484035227 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fe0143ec0 msgr2=0x7f6fe015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:02.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.338+0000 7f6fdfe00640 1 --2- 172.21.15.161:0/3484035227 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fe0143ec0 0x7f6fe015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.338+0000 7f6fdfe00640 1 -- 172.21.15.161:0/3484035227 >> v2:172.21.15.179:3300/0 conn(0x7f6fe00715a0 msgr2=0x7f6fe006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:02.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.338+0000 7f6fdfe00640 1 --2- 172.21.15.161:0/3484035227 >> v2:172.21.15.179:3300/0 conn(0x7f6fe00715a0 0x7f6fe006fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.338+0000 7f6fdfe00640 1 -- 172.21.15.161:0/3484035227 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe0070500 con 0x7f6fe015ec90 2024-05-22T21:20:02.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fdfe00640 1 --2- 172.21.15.161:0/3484035227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe015ec90 0x7f6fe015f120 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0072510 tx=0x7f6fd801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91fb97d3f16dc091 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:02.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fde000640 1 -- 172.21.15.161:0/3484035227 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd80a5840 con 0x7f6fe015ec90 2024-05-22T21:20:02.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fde000640 1 -- 172.21.15.161:0/3484035227 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd80a59a0 con 0x7f6fe015ec90 2024-05-22T21:20:02.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fde000640 1 -- 172.21.15.161:0/3484035227 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd80a5d30 con 0x7f6fe015ec90 2024-05-22T21:20:02.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 -- 172.21.15.161:0/3484035227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe015ec90 msgr2=0x7f6fe015f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:02.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 --2- 172.21.15.161:0/3484035227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe015ec90 0x7f6fe015f120 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0072510 tx=0x7f6fd801c9c0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 -- 172.21.15.161:0/3484035227 shutdown_connections 2024-05-22T21:20:02.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 --2- 172.21.15.161:0/3484035227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe015ec90 0x7f6fe015f120 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 --2- 172.21.15.161:0/3484035227 >> v2:172.21.15.179:3300/0 conn(0x7f6fe00715a0 0x7f6fe006fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 --2- 172.21.15.161:0/3484035227 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fe0143ec0 0x7f6fe015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 -- 172.21.15.161:0/3484035227 >> 172.21.15.161:0/3484035227 conn(0x7f6fe00b0bc0 msgr2=0x7f6fe00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:02.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 -- 172.21.15.161:0/3484035227 shutdown_connections 2024-05-22T21:20:02.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 -- 172.21.15.161:0/3484035227 wait complete. 2024-05-22T21:20:02.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 Processor -- start 2024-05-22T21:20:02.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 -- start start 2024-05-22T21:20:02.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fe0143ec0 0x7f6fe00a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:02.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe015ec90 0x7f6fe0152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:02.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fe0138690 0x7f6fe013b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:02.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe0152ec0 con 0x7f6fe015ec90 2024-05-22T21:20:02.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe01531a0 con 0x7f6fe0143ec0 2024-05-22T21:20:02.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fe6800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe0153480 con 0x7f6fe0138690 2024-05-22T21:20:02.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.342+0000 7f6fdea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe015ec90 0x7f6fe0152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:02.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fdea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe015ec90 0x7f6fe0152670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43542/0 (socket says 172.21.15.161:43542) 2024-05-22T21:20:02.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fdea00640 1 -- 172.21.15.161:0/2081694983 learned_addr learned my addr 172.21.15.161:0/2081694983 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:02.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fdf400640 1 --2- 172.21.15.161:0/2081694983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fe0143ec0 0x7f6fe00a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:02.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fdfe00640 1 --2- 172.21.15.161:0/2081694983 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fe0138690 0x7f6fe013b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:02.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fdea00640 1 -- 172.21.15.161:0/2081694983 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fe0138690 msgr2=0x7f6fe013b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:02.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fdea00640 1 --2- 172.21.15.161:0/2081694983 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fe0138690 0x7f6fe013b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fdea00640 1 -- 172.21.15.161:0/2081694983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fe0143ec0 msgr2=0x7f6fe00a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:02.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fdea00640 1 --2- 172.21.15.161:0/2081694983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fe0143ec0 0x7f6fe00a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fdea00640 1 -- 172.21.15.161:0/2081694983 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd8009b30 con 0x7f6fe015ec90 2024-05-22T21:20:02.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fdea00640 1 --2- 172.21.15.161:0/2081694983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe015ec90 0x7f6fe0152670 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000e930 tx=0x7f6fd000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:02.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fd7400640 1 -- 172.21.15.161:0/2081694983 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0009690 con 0x7f6fe015ec90 2024-05-22T21:20:02.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fd7400640 1 -- 172.21.15.161:0/2081694983 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd000ca00 con 0x7f6fe015ec90 2024-05-22T21:20:02.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fe6800640 1 -- 172.21.15.161:0/2081694983 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe013ab60 con 0x7f6fe015ec90 2024-05-22T21:20:02.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fd7400640 1 -- 172.21.15.161:0/2081694983 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd000cd70 con 0x7f6fe015ec90 2024-05-22T21:20:02.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fe6800640 1 -- 172.21.15.161:0/2081694983 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe0152bb0 con 0x7f6fe015ec90 2024-05-22T21:20:02.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fe6800640 1 -- 172.21.15.161:0/2081694983 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe0067220 con 0x7f6fe015ec90 2024-05-22T21:20:02.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fd7400640 1 -- 172.21.15.161:0/2081694983 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6fd0013470 con 0x7f6fe015ec90 2024-05-22T21:20:02.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.346+0000 7f6fd7400640 1 --2- 172.21.15.161:0/2081694983 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6fbc0666c0 0x7f6fbc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:02.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.350+0000 7f6fd7400640 1 -- 172.21.15.161:0/2081694983 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fd00843c0 con 0x7f6fe015ec90 2024-05-22T21:20:02.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.350+0000 7f6fdf400640 1 --2- 172.21.15.161:0/2081694983 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6fbc0666c0 0x7f6fbc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:02.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.350+0000 7f6fd7400640 1 -- 172.21.15.161:0/2081694983 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=143}) v3 -- 0x7f6fbc06e1f0 con 0x7f6fe015ec90 2024-05-22T21:20:02.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.350+0000 7f6fdf400640 1 --2- 172.21.15.161:0/2081694983 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6fbc0666c0 0x7f6fbc068b80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009ae0 tx=0x7f6fc8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:02.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.354+0000 7f6fd7400640 1 -- 172.21.15.161:0/2081694983 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6fd0051800 con 0x7f6fe015ec90 2024-05-22T21:20:02.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.514+0000 7f6fe6800640 1 -- 172.21.15.161:0/2081694983 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6fe00ccd30 con 0x7f6fe015ec90 2024-05-22T21:20:02.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.514+0000 7f6fd7400640 1 -- 172.21.15.161:0/2081694983 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v142) v1 ==== 81+0+33569 (secure 0 0 0) 0x7f6fd0021020 con 0x7f6fe015ec90 2024-05-22T21:20:02.518 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:02.518 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:20:02.518 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 142, 2024-05-22T21:20:02.518 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:20:02.518 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:20:02.518 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:20:00.256219+0000", 2024-05-22T21:20:02.519 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:20:02.519 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:20:02.519 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:20:02.519 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:20:02.519 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:20:02.519 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:20:02.519 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:20:02.519 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:20:02.519 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:20:02.519 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:02.519 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:20:02.519 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:20:02.519 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:20:02.520 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:20:02.520 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:20:02.520 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 17, 2024-05-22T21:20:02.520 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:20:02.520 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:20:02.520 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:20:02.520 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:20:02.520 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:20:02.520 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:20:02.520 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.520 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:20:02.520 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:20:02.521 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:20:02.521 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:20:02.521 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:20:02.521 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:02.521 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:02.521 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:02.521 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:02.521 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:02.521 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:02.521 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:02.521 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:02.521 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:02.522 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:02.522 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:20:02.522 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:20:02.522 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:20:02.522 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:20:02.522 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:20:02.522 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:02.522 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:02.522 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:02.522 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:02.522 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:02.522 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:02.523 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:02.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.523 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:20:02.523 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:02.523 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:02.523 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:02.523 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:02.523 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:02.523 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:02.523 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:02.523 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:02.523 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:02.523 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:20:02.524 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:02.524 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:02.524 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:02.524 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:02.524 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:02.524 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:02.524 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:02.524 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:02.524 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:02.524 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:02.524 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:02.524 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:02.524 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:02.525 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:02.525 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:02.525 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:02.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.525 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:02.525 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:02.525 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:02.525 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:02.525 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:02.525 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:02.525 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:02.525 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:02.525 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:02.525 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:02.526 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:02.526 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:02.526 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:20:02.526 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:20:02.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.526 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:02.526 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:20:02.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.526 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:02.526 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:20:02.526 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:20:02.526 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:20:02.527 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:20:02.527 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:20:02.527 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:02.527 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:02.527 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:02.527 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.527 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 16, 2024-05-22T21:20:02.527 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:20:02.527 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:19:44.080742+0000", 2024-05-22T21:20:02.527 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:20:02.527 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:20:02.528 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:02.528 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:02.528 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:02.528 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:02.528 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:02.528 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:02.528 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:02.528 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:02.528 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:02.528 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:02.528 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:20:02.528 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:20:02.528 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:20:02.529 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:20:02.529 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:20:02.529 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:02.529 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:02.529 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:02.529 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:02.529 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:02.529 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:02.529 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:02.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.529 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "139", 2024-05-22T21:20:02.529 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:02.529 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:02.530 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:02.530 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:02.530 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:02.530 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:02.530 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:02.530 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:02.530 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:02.530 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:20:02.530 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:02.530 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:02.530 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:02.530 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:02.531 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:02.531 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:02.531 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:02.531 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:02.531 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:02.531 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:02.531 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:02.531 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:02.531 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:02.531 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:02.531 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:02.531 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:02.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.532 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:02.532 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:02.532 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:02.532 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:02.532 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:02.532 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:02.532 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:02.532 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:02.532 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:02.532 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:02.532 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:02.532 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:02.532 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:20:02.532 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:20:02.533 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:20:02.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.533 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:02.533 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:20:02.533 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:20:02.533 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.533 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:02.533 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:20:02.533 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:20:02.533 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:20:02.533 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:20:02.533 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:20:02.534 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:02.534 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:02.534 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:02.534 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.534 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 17, 2024-05-22T21:20:02.534 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:20:02.534 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:19:45.388618+0000", 2024-05-22T21:20:02.534 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1027, 2024-05-22T21:20:02.534 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full,full_quota", 2024-05-22T21:20:02.534 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:02.534 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:20:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:20:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:20:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:20:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:20:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "142", 2024-05-22T21:20:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 33554432, 2024-05-22T21:20:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:20:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:20:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:20:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:20:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:20:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:20:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:20:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:20:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:20:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:20:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:20:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:20:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:20:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:20:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:20:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:20:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:20:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:20:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:02.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.546 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.546 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:20:02.546 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:02.546 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.546 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.546 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:20:02.546 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:20:02.546 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:20:02.546 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:20:02.547 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:02.547 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:02.547 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:02.547 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.547 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.547 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:20:02.547 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:02.547 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:02.547 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:02.547 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:02.547 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:02.547 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:02.548 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:02.548 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:02.548 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:02.548 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:02.548 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:02.548 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:02.548 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.548 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.548 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:20:02.548 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:02.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.549 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.549 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:20:02.549 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:02.549 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.549 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.549 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:02.549 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.549 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.549 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:20:02.549 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:02.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.550 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.550 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:20:02.550 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:02.550 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.550 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.550 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:02.550 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.550 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.550 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:20:02.550 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:02.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.551 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.551 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:20:02.551 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:02.551 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.551 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.551 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:02.551 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.551 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.551 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:20:02.552 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:02.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.552 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.552 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.552 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:20:02.552 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:02.552 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.552 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.552 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:20:02.552 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:20:02.552 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:20:02.552 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:20:02.553 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:02.553 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:02.553 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:02.553 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.553 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:20:02.553 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:02.553 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:02.553 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:02.553 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:02.553 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:02.554 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:02.554 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:02.554 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:02.554 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:02.554 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:02.554 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:02.554 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:02.554 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.554 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.554 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.554 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:20:02.554 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:02.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.555 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.555 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:20:02.555 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:02.555 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.555 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.557 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:02.557 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.557 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.557 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:20:02.557 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:02.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.558 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.558 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:20:02.558 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:02.558 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.558 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.558 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:02.558 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.558 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.558 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:20:02.559 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:02.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.559 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.559 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:20:02.559 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:02.559 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.559 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.559 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:02.559 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.559 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.559 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.560 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:20:02.560 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:02.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.560 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.560 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:20:02.560 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:02.560 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.560 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.560 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:20:02.560 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:20:02.560 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:20:02.561 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:20:02.561 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:02.561 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:02.561 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:02.561 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.561 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:20:02.561 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:02.561 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:02.561 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:02.561 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:02.561 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:02.562 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:02.562 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:02.562 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:02.562 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:02.562 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:02.562 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:02.562 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:02.562 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.562 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.562 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:20:02.562 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:02.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.563 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.563 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:20:02.563 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:02.563 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.563 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.563 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:02.563 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.563 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.563 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:20:02.563 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:02.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.564 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.564 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:20:02.564 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:02.564 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.564 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.564 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:02.564 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.564 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.564 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:20:02.564 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:02.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.565 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.565 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:20:02.565 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:02.565 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.565 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.565 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:02.565 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.565 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.565 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:20:02.566 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:02.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.566 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.566 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:20:02.566 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:02.566 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.566 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.566 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:20:02.566 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:20:02.566 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:20:02.566 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:20:02.567 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:02.567 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:02.567 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:02.567 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.567 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.567 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:20:02.567 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:02.567 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:02.567 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:02.567 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:02.567 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:02.567 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:02.568 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:02.568 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:02.568 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:02.568 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:02.568 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:02.568 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:02.568 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.568 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.568 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:20:02.568 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:02.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.568 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.569 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:20:02.569 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:02.569 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.569 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.569 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:02.569 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.569 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.569 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:20:02.569 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:02.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.569 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.570 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:20:02.570 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:02.570 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.570 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.570 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:02.570 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.570 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.570 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:20:02.570 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:02.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.571 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.571 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:20:02.571 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:02.571 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.571 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.571 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:02.571 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.571 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.571 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:20:02.571 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:02.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.572 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.572 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:20:02.572 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:02.572 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.572 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.572 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:20:02.572 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:20:02.572 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:20:02.572 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:20:02.572 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:02.572 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:02.573 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:02.573 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.573 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.573 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:20:02.573 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:02.573 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:02.573 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:02.573 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:02.573 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:02.573 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:02.573 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:02.573 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:02.573 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:20:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.574 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:20:02.575 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:02.575 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.575 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.575 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:02.575 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.575 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.575 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:20:02.575 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:02.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.575 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.575 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:20:02.576 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:02.576 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.576 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.576 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:02.576 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.576 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.576 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:20:02.576 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:02.576 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.576 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.576 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.577 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:20:02.577 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:02.577 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.577 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.577 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:02.577 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.577 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.577 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:20:02.577 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:02.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.577 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.578 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:20:02.578 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:02.578 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.578 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.578 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:20:02.578 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:20:02.578 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:20:02.578 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:20:02.578 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:02.578 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:02.578 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:02.579 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.579 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:20:02.579 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:02.579 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:02.579 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:02.579 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:02.579 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:02.579 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:02.579 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:02.579 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:02.579 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:02.579 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:02.580 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:02.580 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:02.580 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.580 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.580 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:20:02.580 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:02.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.580 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.580 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:20:02.580 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:02.580 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.581 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.581 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:02.581 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.581 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.581 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:20:02.581 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:02.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.581 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.581 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:20:02.581 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:02.582 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.582 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.582 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:02.582 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.582 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.582 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:20:02.582 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:02.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.582 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.582 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:20:02.583 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:02.583 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.583 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.583 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:02.583 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.583 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.583 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:20:02.583 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:02.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.584 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.584 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:20:02.584 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:02.584 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.584 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.584 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:20:02.584 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:20:02.584 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:20:02.584 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:20:02.584 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:02.584 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:02.584 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:02.585 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.585 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:20:02.585 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:02.585 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:02.585 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:02.585 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:02.585 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:02.585 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:02.585 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:02.585 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:02.585 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:02.586 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:02.586 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:02.586 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:02.586 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.586 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.586 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:20:02.586 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:02.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.586 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.586 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:20:02.586 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:02.587 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.587 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.587 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:02.587 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.587 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.587 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:20:02.587 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:02.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.588 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.588 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:20:02.588 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:02.588 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.588 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.588 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:02.588 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.588 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.588 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.588 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:20:02.588 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:02.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.589 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.589 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:20:02.589 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:02.589 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.589 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.589 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:02.589 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:02.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.589 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:02.589 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:20:02.589 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:02.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.590 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:02.590 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:20:02.590 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:02.590 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.590 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.590 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:20:02.590 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:20:02.590 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:20:02.590 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:20:02.591 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:02.591 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:02.591 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:02.591 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:02.591 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.591 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:02.591 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:20:02.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.591 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:20:02.591 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:02.591 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:02.591 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:02.591 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:02.592 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:02.592 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:20:02.592 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:02.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.592 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:20:02.592 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:02.592 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:02.592 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:02.592 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:02.592 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:02.592 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:20:02.593 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:02.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.593 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:20:02.593 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:02.593 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:02.593 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:02.593 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:02.593 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:02.593 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:20:02.593 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:02.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.593 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.594 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:20:02.594 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:02.594 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:02.594 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:02.594 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:02.594 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:02.594 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:20:02.594 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:02.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.594 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:20:02.594 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:02.595 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:02.595 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:02.595 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:02.595 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:02.595 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:20:02.595 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:02.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.595 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:20:02.595 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:02.595 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:02.595 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:02.596 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:02.596 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:02.596 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:20:02.596 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:02.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.596 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:20:02.596 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:02.596 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:02.596 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:02.596 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:02.596 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:02.596 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:20:02.597 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:02.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:02.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:20:02.597 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:02.597 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:02.597 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:02.597 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:02.597 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:02.597 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:20:02.597 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:02.597 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.597 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:02.598 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:20:02.598 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:20:02.598 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:20:02.598 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:20:02.598 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:20:02.598 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:20:02.598 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:20:02.598 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:20:02.598 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:20:02.598 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:20:02.598 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:20:02.598 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:20:02.599 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:20:02.599 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:20:02.599 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:02.599 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:20:02.599 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:20:02.599 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:20:02.599 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:20:02.599 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:20:02.599 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:20:02.599 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:20:02.599 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:20:02.599 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:20:02.599 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:20:02.600 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:20:02.600 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:02.600 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:20:02.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.514+0000 7f6fe6800640 1 -- 172.21.15.161:0/2081694983 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6fbc0666c0 msgr2=0x7f6fbc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:02.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.514+0000 7f6fe6800640 1 --2- 172.21.15.161:0/2081694983 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6fbc0666c0 0x7f6fbc068b80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009ae0 tx=0x7f6fc8009190 comp rx=0 tx=0).stop 2024-05-22T21:20:02.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.514+0000 7f6fe6800640 1 -- 172.21.15.161:0/2081694983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe015ec90 msgr2=0x7f6fe0152670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:02.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.514+0000 7f6fe6800640 1 --2- 172.21.15.161:0/2081694983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe015ec90 0x7f6fe0152670 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000e930 tx=0x7f6fd000edf0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.518+0000 7f6fe6800640 1 -- 172.21.15.161:0/2081694983 shutdown_connections 2024-05-22T21:20:02.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.518+0000 7f6fe6800640 1 --2- 172.21.15.161:0/2081694983 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6fbc0666c0 0x7f6fbc068b80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.518+0000 7f6fe6800640 1 --2- 172.21.15.161:0/2081694983 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fe0138690 0x7f6fe013b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.518+0000 7f6fe6800640 1 --2- 172.21.15.161:0/2081694983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe015ec90 0x7f6fe0152670 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.518+0000 7f6fe6800640 1 --2- 172.21.15.161:0/2081694983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fe0143ec0 0x7f6fe00a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.518+0000 7f6fe6800640 1 -- 172.21.15.161:0/2081694983 >> 172.21.15.161:0/2081694983 conn(0x7f6fe00b0bc0 msgr2=0x7f6fe015e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:02.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.518+0000 7f6fe6800640 1 -- 172.21.15.161:0/2081694983 shutdown_connections 2024-05-22T21:20:02.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.518+0000 7f6fe6800640 1 -- 172.21.15.161:0/2081694983 wait complete. 2024-05-22T21:20:02.603 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:20:02.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.734+0000 7f11dae00640 1 Processor -- start 2024-05-22T21:20:02.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.734+0000 7f11dae00640 1 -- start start 2024-05-22T21:20:02.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.738+0000 7f11dae00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f11d41597f0 0x7f11d40706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:02.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.738+0000 7f11dae00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f11d4070be0 0x7f11d40b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:02.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.738+0000 7f11dae00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d4071040 con 0x7f11d40b2ea0 2024-05-22T21:20:02.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.738+0000 7f11dae00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d4159bd0 con 0x7f11d4070be0 2024-05-22T21:20:02.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.738+0000 7f11dae00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d40cbdd0 con 0x7f11d41597f0 2024-05-22T21:20:02.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.738+0000 7f11d8e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f11d40b2ea0 0x7f11d406c7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56210/0 (socket says 172.21.15.161:56210) 2024-05-22T21:20:02.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.738+0000 7f11d8e00640 1 -- 172.21.15.161:0/329188124 learned_addr learned my addr 172.21.15.161:0/329188124 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:02.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.738+0000 7f11d3400640 1 --2- 172.21.15.161:0/329188124 >> v2:172.21.15.161:3300/0 conn(0x7f11d4070be0 0x7f11d40b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:02.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.738+0000 7f11d3e00640 1 --2- 172.21.15.161:0/329188124 >> v2:172.21.15.179:3300/0 conn(0x7f11d41597f0 0x7f11d40706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:02.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.738+0000 7f11d2a00640 1 -- 172.21.15.161:0/329188124 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556099897 0 0) 0x7f11d4071040 con 0x7f11d40b2ea0 2024-05-22T21:20:02.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.738+0000 7f11d2a00640 1 -- 172.21.15.161:0/329188124 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11b8003530 con 0x7f11d40b2ea0 2024-05-22T21:20:02.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.738+0000 7f11d3400640 1 -- 172.21.15.161:0/329188124 >> v2:172.21.15.179:3300/0 conn(0x7f11d41597f0 msgr2=0x7f11d40706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:02.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.738+0000 7f11d3400640 1 --2- 172.21.15.161:0/329188124 >> v2:172.21.15.179:3300/0 conn(0x7f11d41597f0 0x7f11d40706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.738+0000 7f11d3400640 1 -- 172.21.15.161:0/329188124 >> v1:172.21.15.161:6789/0 conn(0x7f11d40b2ea0 legacy=0x7f11d406c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:02.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.738+0000 7f11d3400640 1 -- 172.21.15.161:0/329188124 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d40cc050 con 0x7f11d4070be0 2024-05-22T21:20:02.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11d3400640 1 --2- 172.21.15.161:0/329188124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d4070be0 0x7f11d40b2960 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f11d4072330 tx=0x7f11c001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2370adbae7a39f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:02.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11d2a00640 1 -- 172.21.15.161:0/329188124 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c00ac640 con 0x7f11d4070be0 2024-05-22T21:20:02.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11d2a00640 1 -- 172.21.15.161:0/329188124 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c00ac7a0 con 0x7f11d4070be0 2024-05-22T21:20:02.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11d2a00640 1 -- 172.21.15.161:0/329188124 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c00acb30 con 0x7f11d4070be0 2024-05-22T21:20:02.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11dae00640 1 -- 172.21.15.161:0/329188124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d4070be0 msgr2=0x7f11d40b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:02.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11dae00640 1 --2- 172.21.15.161:0/329188124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d4070be0 0x7f11d40b2960 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f11d4072330 tx=0x7f11c001c460 comp rx=0 tx=0).stop 2024-05-22T21:20:02.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11dae00640 1 -- 172.21.15.161:0/329188124 shutdown_connections 2024-05-22T21:20:02.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11dae00640 1 --2- 172.21.15.161:0/329188124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d4070be0 0x7f11d40b2960 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11dae00640 1 --2- 172.21.15.161:0/329188124 >> v2:172.21.15.179:3300/0 conn(0x7f11d41597f0 0x7f11d40706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11dae00640 1 -- 172.21.15.161:0/329188124 >> 172.21.15.161:0/329188124 conn(0x7f11d40b0bc0 msgr2=0x7f11d40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:02.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11dae00640 1 -- 172.21.15.161:0/329188124 shutdown_connections 2024-05-22T21:20:02.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11dae00640 1 -- 172.21.15.161:0/329188124 wait complete. 2024-05-22T21:20:02.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11dae00640 1 Processor -- start 2024-05-22T21:20:02.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11dae00640 1 -- start start 2024-05-22T21:20:02.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11dae00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11d4070be0 0x7f11d4142730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:02.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11dae00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11d40b2ea0 0x7f11d414e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:02.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11dae00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d41597f0 0x7f11d414e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:02.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11dae00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d414b4d0 con 0x7f11d41597f0 2024-05-22T21:20:02.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11dae00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d4142c70 con 0x7f11d40b2ea0 2024-05-22T21:20:02.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.742+0000 7f11dae00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d41fa1f0 con 0x7f11d4070be0 2024-05-22T21:20:02.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11d8e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d41597f0 0x7f11d414e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:02.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11d8e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d41597f0 0x7f11d414e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43552/0 (socket says 172.21.15.161:43552) 2024-05-22T21:20:02.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11d8e00640 1 -- 172.21.15.161:0/351583279 learned_addr learned my addr 172.21.15.161:0/351583279 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:02.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11d3400640 1 --2- 172.21.15.161:0/351583279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11d40b2ea0 0x7f11d414e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:02.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11d3e00640 1 --2- 172.21.15.161:0/351583279 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11d4070be0 0x7f11d4142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:02.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11d8e00640 1 -- 172.21.15.161:0/351583279 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11d4070be0 msgr2=0x7f11d4142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:02.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11d8e00640 1 --2- 172.21.15.161:0/351583279 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11d4070be0 0x7f11d4142730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11d8e00640 1 -- 172.21.15.161:0/351583279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11d40b2ea0 msgr2=0x7f11d414e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:02.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11d8e00640 1 --2- 172.21.15.161:0/351583279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11d40b2ea0 0x7f11d414e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11d8e00640 1 -- 172.21.15.161:0/351583279 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c0009420 con 0x7f11d41597f0 2024-05-22T21:20:02.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11d3400640 1 --2- 172.21.15.161:0/351583279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11d40b2ea0 0x7f11d414e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:02.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11d8e00640 1 --2- 172.21.15.161:0/351583279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d41597f0 0x7f11d414e990 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00c450 tx=0x7f11cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:02.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11cbe00640 1 -- 172.21.15.161:0/351583279 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc005cf0 con 0x7f11d41597f0 2024-05-22T21:20:02.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11cbe00640 1 -- 172.21.15.161:0/351583279 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11cc005e50 con 0x7f11d41597f0 2024-05-22T21:20:02.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11dae00640 1 -- 172.21.15.161:0/351583279 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d414efc0 con 0x7f11d41597f0 2024-05-22T21:20:02.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11cbe00640 1 -- 172.21.15.161:0/351583279 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc0072b0 con 0x7f11d41597f0 2024-05-22T21:20:02.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11dae00640 1 -- 172.21.15.161:0/351583279 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d41f9710 con 0x7f11d41597f0 2024-05-22T21:20:02.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11dae00640 1 -- 172.21.15.161:0/351583279 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d4067220 con 0x7f11d41597f0 2024-05-22T21:20:02.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.746+0000 7f11cbe00640 1 -- 172.21.15.161:0/351583279 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f11cc007410 con 0x7f11d41597f0 2024-05-22T21:20:02.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.750+0000 7f11cbe00640 1 --2- 172.21.15.161:0/351583279 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f11b4066690 0x7f11b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:02.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.750+0000 7f11cbe00640 1 -- 172.21.15.161:0/351583279 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11cc083620 con 0x7f11d41597f0 2024-05-22T21:20:02.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.750+0000 7f11d3e00640 1 --2- 172.21.15.161:0/351583279 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f11b4066690 0x7f11b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:02.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.750+0000 7f11cbe00640 1 -- 172.21.15.161:0/351583279 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=143}) v3 -- 0x7f11b406e1c0 con 0x7f11d41597f0 2024-05-22T21:20:02.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.750+0000 7f11d3e00640 1 --2- 172.21.15.161:0/351583279 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f11b4066690 0x7f11b4068b50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f11c4011340 tx=0x7f11c400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:02.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.754+0000 7f11cbe00640 1 -- 172.21.15.161:0/351583279 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f11cc0509a0 con 0x7f11d41597f0 2024-05-22T21:20:02.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.938+0000 7f11dae00640 1 -- 172.21.15.161:0/351583279 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11d414f760 con 0x7f11d41597f0 2024-05-22T21:20:02.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.938+0000 7f11cbe00640 1 -- 172.21.15.161:0/351583279 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4745 (secure 0 0 0) 0x7f11cc017070 con 0x7f11d41597f0 2024-05-22T21:20:02.940 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:02.940 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/524372753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":524372753},{"type":"v1","addr":"172.21.15.161:6836","nonce":524372753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3762119379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3762119379},{"type":"v1","addr":"172.21.15.179:6833","nonce":3762119379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3087241366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3087241366},{"type":"v1","addr":"172.21.15.179:6835","nonce":3087241366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:46.384984+0000","modified":"2024-05-22T21:19:49.400019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7572":{"gid":7572,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/3945715002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3945715002},{"type":"v1","addr":"172.21.15.161:6837","nonce":3945715002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-22T21:20:02.941 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 62 2024-05-22T21:20:02.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.938+0000 7f11dae00640 1 -- 172.21.15.161:0/351583279 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f11b4066690 msgr2=0x7f11b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:02.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.938+0000 7f11dae00640 1 --2- 172.21.15.161:0/351583279 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f11b4066690 0x7f11b4068b50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f11c4011340 tx=0x7f11c400b040 comp rx=0 tx=0).stop 2024-05-22T21:20:02.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.938+0000 7f11dae00640 1 -- 172.21.15.161:0/351583279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d41597f0 msgr2=0x7f11d414e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:02.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.938+0000 7f11dae00640 1 --2- 172.21.15.161:0/351583279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d41597f0 0x7f11d414e990 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00c450 tx=0x7f11cc00c910 comp rx=0 tx=0).stop 2024-05-22T21:20:02.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.938+0000 7f11dae00640 1 -- 172.21.15.161:0/351583279 shutdown_connections 2024-05-22T21:20:02.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.938+0000 7f11dae00640 1 --2- 172.21.15.161:0/351583279 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f11b4066690 0x7f11b4068b50 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.938+0000 7f11dae00640 1 --2- 172.21.15.161:0/351583279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11d41597f0 0x7f11d414e990 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.938+0000 7f11dae00640 1 --2- 172.21.15.161:0/351583279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11d40b2ea0 0x7f11d414e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.938+0000 7f11dae00640 1 --2- 172.21.15.161:0/351583279 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11d4070be0 0x7f11d4142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:02.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.938+0000 7f11dae00640 1 -- 172.21.15.161:0/351583279 >> 172.21.15.161:0/351583279 conn(0x7f11d40b0bc0 msgr2=0x7f11d406c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:02.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.938+0000 7f11dae00640 1 -- 172.21.15.161:0/351583279 shutdown_connections 2024-05-22T21:20:02.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:02.938+0000 7f11dae00640 1 -- 172.21.15.161:0/351583279 wait complete. 2024-05-22T21:20:02.961 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-05-22T21:20:03.117 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":7572,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":16.714131814000002,"mdsmap_epoch":62,"osdmap_epoch":142,"osdmap_epoch_barrier":142,"uptime":21.964274708000001} 2024-05-22T21:20:03.117 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 7572, "mdsmap_epoch": 62, "osdmap_epoch": 142, "osdmap_epoch_barrier": 142, "rank_uptime": 16.714131814, "state": "up:active", "uptime": 21.964274708, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:20:03.117 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-22T21:20:03.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:03.117 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-05-22T21:20:03.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:03.128 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-05-22T21:20:03.184 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:20:03.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.358+0000 7fb12ee00640 1 Processor -- start 2024-05-22T21:20:03.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.358+0000 7fb12ee00640 1 -- start start 2024-05-22T21:20:03.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.358+0000 7fb12ee00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb128143ec0 0x7fb12815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:03.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.358+0000 7fb12ee00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb1280715a0 0x7fb12806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:03.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.358+0000 7fb12ee00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1281442a0 con 0x7fb12815ec90 2024-05-22T21:20:03.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.358+0000 7fb12ee00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb128070260 con 0x7fb1280715a0 2024-05-22T21:20:03.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.358+0000 7fb12ee00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb128070500 con 0x7fb128143ec0 2024-05-22T21:20:03.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.362+0000 7fb11f400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb1280715a0 0x7fb12806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:03.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.362+0000 7fb11f400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb1280715a0 0x7fb12806fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43556/0 (socket says 172.21.15.161:43556) 2024-05-22T21:20:03.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.362+0000 7fb11f400640 1 -- 172.21.15.161:0/1641933620 learned_addr learned my addr 172.21.15.161:0/1641933620 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:03.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.362+0000 7fb127e00640 1 --2- 172.21.15.161:0/1641933620 >> v2:172.21.15.179:3300/0 conn(0x7fb128143ec0 0x7fb12815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:03.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.362+0000 7fb11f400640 1 -- 172.21.15.161:0/1641933620 >> v2:172.21.15.179:3300/0 conn(0x7fb128143ec0 msgr2=0x7fb12815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:03.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.362+0000 7fb11f400640 1 --2- 172.21.15.161:0/1641933620 >> v2:172.21.15.179:3300/0 conn(0x7fb128143ec0 0x7fb12815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:03.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.362+0000 7fb11f400640 1 -- 172.21.15.161:0/1641933620 >> v1:172.21.15.179:6789/0 conn(0x7fb12815ec90 legacy=0x7fb12806bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:03.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.362+0000 7fb11f400640 1 -- 172.21.15.161:0/1641933620 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1280707e0 con 0x7fb1280715a0 2024-05-22T21:20:03.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.362+0000 7fb11f400640 1 --2- 172.21.15.161:0/1641933620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1280715a0 0x7fb12806fd20 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb1280724e0 tx=0x7fb11001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7df3e72637f6e65 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.362+0000 7fb127400640 1 -- 172.21.15.161:0/1641933620 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1100aa650 con 0x7fb1280715a0 2024-05-22T21:20:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.362+0000 7fb127400640 1 -- 172.21.15.161:0/1641933620 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1100aa7b0 con 0x7fb1280715a0 2024-05-22T21:20:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.362+0000 7fb127400640 1 -- 172.21.15.161:0/1641933620 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1100aab40 con 0x7fb1280715a0 2024-05-22T21:20:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ee00640 1 -- 172.21.15.161:0/1641933620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1280715a0 msgr2=0x7fb12806fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ee00640 1 --2- 172.21.15.161:0/1641933620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1280715a0 0x7fb12806fd20 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb1280724e0 tx=0x7fb11001c4f0 comp rx=0 tx=0).stop 2024-05-22T21:20:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ee00640 1 -- 172.21.15.161:0/1641933620 shutdown_connections 2024-05-22T21:20:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ee00640 1 --2- 172.21.15.161:0/1641933620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1280715a0 0x7fb12806fd20 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ee00640 1 --2- 172.21.15.161:0/1641933620 >> v2:172.21.15.179:3300/0 conn(0x7fb128143ec0 0x7fb12815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ee00640 1 -- 172.21.15.161:0/1641933620 >> 172.21.15.161:0/1641933620 conn(0x7fb1280b0bc0 msgr2=0x7fb1280b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ee00640 1 -- 172.21.15.161:0/1641933620 shutdown_connections 2024-05-22T21:20:03.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ee00640 1 -- 172.21.15.161:0/1641933620 wait complete. 2024-05-22T21:20:03.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ee00640 1 Processor -- start 2024-05-22T21:20:03.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ee00640 1 -- start start 2024-05-22T21:20:03.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ee00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb128143ec0 0x7fb12815a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:03.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb12815ec90 0x7fb12815cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:03.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb12815d240 0x7fb1281625d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:03.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ee00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1281603b0 con 0x7fb128143ec0 2024-05-22T21:20:03.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12815d6f0 con 0x7fb12815ec90 2024-05-22T21:20:03.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12815f7c0 con 0x7fb12815d240 2024-05-22T21:20:03.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb127e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb128143ec0 0x7fb12815a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:03.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb127e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb128143ec0 0x7fb12815a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43568/0 (socket says 172.21.15.161:43568) 2024-05-22T21:20:03.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb127e00640 1 -- 172.21.15.161:0/3117602112 learned_addr learned my addr 172.21.15.161:0/3117602112 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:03.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb11f400640 1 --2- 172.21.15.161:0/3117602112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb12815ec90 0x7fb12815cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:03.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb12ce00640 1 --2- 172.21.15.161:0/3117602112 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb12815d240 0x7fb1281625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:03.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb127e00640 1 -- 172.21.15.161:0/3117602112 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb12815d240 msgr2=0x7fb1281625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:03.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb127e00640 1 --2- 172.21.15.161:0/3117602112 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb12815d240 0x7fb1281625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:03.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb127e00640 1 -- 172.21.15.161:0/3117602112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb12815ec90 msgr2=0x7fb12815cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:03.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb127e00640 1 --2- 172.21.15.161:0/3117602112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb12815ec90 0x7fb12815cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:03.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.366+0000 7fb127e00640 1 -- 172.21.15.161:0/3117602112 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1100094d0 con 0x7fb128143ec0 2024-05-22T21:20:03.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.370+0000 7fb127e00640 1 --2- 172.21.15.161:0/3117602112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb128143ec0 0x7fb12815a5b0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb11800d830 tx=0x7fb11800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:03.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.370+0000 7fb124c00640 1 -- 172.21.15.161:0/3117602112 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11800b6c0 con 0x7fb128143ec0 2024-05-22T21:20:03.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.370+0000 7fb124c00640 1 -- 172.21.15.161:0/3117602112 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11800b820 con 0x7fb128143ec0 2024-05-22T21:20:03.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.370+0000 7fb124c00640 1 -- 172.21.15.161:0/3117602112 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11800bb90 con 0x7fb128143ec0 2024-05-22T21:20:03.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.370+0000 7fb12ee00640 1 -- 172.21.15.161:0/3117602112 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb12815f630 con 0x7fb128143ec0 2024-05-22T21:20:03.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.370+0000 7fb12ee00640 1 -- 172.21.15.161:0/3117602112 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12815fcf0 con 0x7fb128143ec0 2024-05-22T21:20:03.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.370+0000 7fb12ee00640 1 -- 172.21.15.161:0/3117602112 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb128067220 con 0x7fb128143ec0 2024-05-22T21:20:03.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.370+0000 7fb124c00640 1 -- 172.21.15.161:0/3117602112 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb11801e460 con 0x7fb128143ec0 2024-05-22T21:20:03.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.370+0000 7fb124c00640 1 --2- 172.21.15.161:0/3117602112 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb1040666c0 0x7fb104068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:03.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.370+0000 7fb124c00640 1 -- 172.21.15.161:0/3117602112 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1180847c0 con 0x7fb128143ec0 2024-05-22T21:20:03.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.370+0000 7fb11f400640 1 --2- 172.21.15.161:0/3117602112 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb1040666c0 0x7fb104068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:03.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.370+0000 7fb124c00640 1 -- 172.21.15.161:0/3117602112 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=143}) v3 -- 0x7fb10406e1f0 con 0x7fb128143ec0 2024-05-22T21:20:03.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.370+0000 7fb11f400640 1 --2- 172.21.15.161:0/3117602112 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb1040666c0 0x7fb104068b80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb1100ae000 tx=0x7fb1100b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:03.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.374+0000 7fb124c00640 1 -- 172.21.15.161:0/3117602112 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb118051bd0 con 0x7fb128143ec0 2024-05-22T21:20:03.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.530+0000 7fb12ee00640 1 -- 172.21.15.161:0/3117602112 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb12806dcc0 con 0x7fb128143ec0 2024-05-22T21:20:03.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.534+0000 7fb124c00640 1 -- 172.21.15.161:0/3117602112 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v142) v1 ==== 81+0+33569 (secure 0 0 0) 0x7fb1180551f0 con 0x7fb128143ec0 2024-05-22T21:20:03.536 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:03.536 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:20:03.536 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 142, 2024-05-22T21:20:03.536 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:20:03.536 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:20:03.537 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:20:00.256219+0000", 2024-05-22T21:20:03.537 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:20:03.537 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:20:03.537 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:20:03.537 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:20:03.537 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:20:03.537 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:20:03.537 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:20:03.537 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:20:03.537 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:20:03.537 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:03.537 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:20:03.537 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:20:03.538 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:20:03.538 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:20:03.538 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:20:03.538 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 17, 2024-05-22T21:20:03.538 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:20:03.538 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:20:03.538 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:20:03.538 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:20:03.538 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:20:03.538 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:20:03.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.538 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:20:03.538 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:20:03.539 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:20:03.539 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:20:03.539 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:20:03.539 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:03.539 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:03.539 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:03.539 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:03.539 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:03.539 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:03.539 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:03.539 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:03.539 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:03.539 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:03.540 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:20:03.540 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:20:03.540 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:20:03.540 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:20:03.540 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:20:03.540 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:03.540 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:03.541 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:03.541 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:03.541 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:03.541 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:03.541 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:03.541 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.541 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:20:03.541 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:03.541 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:03.541 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:03.541 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:03.542 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:03.542 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:03.542 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:03.542 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:03.542 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:03.542 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:20:03.542 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:03.542 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:03.542 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:03.542 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:03.542 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:03.542 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:03.542 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:03.543 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:03.543 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:03.543 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:03.543 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:03.543 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:03.543 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:03.543 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:03.543 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:03.543 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:03.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.543 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:03.543 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:03.543 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:03.544 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:03.544 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:03.544 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:03.544 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:03.544 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:03.544 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:03.544 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:03.544 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:03.544 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:03.544 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:20:03.544 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:20:03.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.544 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:03.545 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:20:03.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.545 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:03.545 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:20:03.545 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:20:03.545 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:20:03.545 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:20:03.545 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:20:03.545 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:03.545 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:03.546 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:03.546 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.546 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 16, 2024-05-22T21:20:03.546 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:20:03.546 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:19:44.080742+0000", 2024-05-22T21:20:03.546 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:20:03.546 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:20:03.546 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:03.546 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:03.546 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:03.547 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:03.547 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:03.547 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:03.547 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:03.547 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:03.547 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:03.547 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:03.547 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:20:03.547 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:20:03.547 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:20:03.547 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:20:03.547 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:20:03.547 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:03.547 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:03.548 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:03.548 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:03.548 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:03.548 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:03.548 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:03.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.548 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "139", 2024-05-22T21:20:03.548 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:03.548 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:03.548 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:03.548 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:03.548 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:03.548 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:03.549 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:03.549 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:03.549 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:03.549 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:20:03.549 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:03.549 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:03.549 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:03.549 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:03.549 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:03.549 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:03.549 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:03.549 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:03.549 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:03.550 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:03.550 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:03.550 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:03.550 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:03.550 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:03.550 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:03.550 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:03.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.550 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:03.550 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:03.550 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:03.550 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:03.551 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:03.551 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:03.551 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:03.551 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:03.551 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:03.551 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:03.551 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:03.551 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:03.551 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:20:03.551 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:20:03.551 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:20:03.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.552 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:03.552 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:20:03.552 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:20:03.552 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.552 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:03.552 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:20:03.552 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:20:03.552 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:20:03.552 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:20:03.552 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:20:03.552 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:03.552 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:03.553 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:03.553 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.553 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 17, 2024-05-22T21:20:03.553 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:20:03.553 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:19:45.388618+0000", 2024-05-22T21:20:03.553 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1027, 2024-05-22T21:20:03.553 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full,full_quota", 2024-05-22T21:20:03.553 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:03.553 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:03.553 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:03.553 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:03.554 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:03.554 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:03.554 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:03.554 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:03.554 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:03.554 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:03.554 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:20:03.554 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:20:03.554 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:20:03.554 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:20:03.554 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:20:03.554 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:03.555 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:03.555 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:03.555 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:03.555 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:03.555 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:03.555 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:03.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.555 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "142", 2024-05-22T21:20:03.555 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:03.555 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:03.555 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:03.555 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:03.555 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:03.556 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:03.556 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:03.556 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:03.556 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:03.556 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 33554432, 2024-05-22T21:20:03.556 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:03.556 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:03.556 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:03.556 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:03.556 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:03.556 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:03.556 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:03.557 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:03.557 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:03.557 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:03.557 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:03.557 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:03.557 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:03.557 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:03.557 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:03.557 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:03.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.557 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:03.557 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:03.557 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:03.558 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:03.558 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:03.558 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:03.558 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:03.558 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:03.558 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:03.558 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:03.558 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:03.558 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:03.558 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:20:03.558 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:20:03.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.558 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:20:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:20:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:20:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:20:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:20:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:20:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:20:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:20:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:20:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:03.561 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:03.561 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:03.561 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:03.561 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:03.561 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:03.561 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:03.561 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:03.561 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:03.561 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.561 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.561 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.561 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:20:03.562 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:03.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.562 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.562 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:20:03.562 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:03.562 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.562 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.562 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:03.562 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.562 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.563 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.563 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:20:03.563 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:03.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.563 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.563 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.563 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:20:03.563 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:03.563 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.563 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.563 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:03.563 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.564 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.564 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:20:03.564 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:03.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.564 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.564 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.564 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:20:03.564 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:03.564 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.564 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.564 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:03.565 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.565 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.565 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:20:03.565 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:03.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.565 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.565 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.565 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:20:03.565 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:03.565 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.565 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.566 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:20:03.566 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:20:03.566 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:20:03.566 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:20:03.566 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:03.566 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:03.566 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:03.566 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.566 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.566 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:20:03.567 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:03.567 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:03.567 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:03.567 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:03.567 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:03.567 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:03.567 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:03.567 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:03.567 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:03.567 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:03.567 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:03.567 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:03.567 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.568 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.568 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:20:03.568 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:03.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.568 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.568 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.568 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:20:03.568 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:03.568 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.568 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.568 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:03.569 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.569 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.569 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:20:03.569 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:03.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.569 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.569 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.569 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:20:03.569 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:03.569 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.569 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.570 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:03.570 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.570 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.570 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:20:03.570 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:03.570 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.570 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.570 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.570 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:20:03.570 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:03.570 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.570 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.571 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:03.571 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.571 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.571 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:20:03.571 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:03.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.571 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.571 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.571 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:20:03.571 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:03.572 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.572 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.572 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:20:03.572 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:20:03.572 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:20:03.572 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:20:03.572 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:03.572 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:03.572 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:03.572 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.572 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.572 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.573 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:20:03.573 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:03.573 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:03.573 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:03.573 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:03.573 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:03.573 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:03.573 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:03.573 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:03.573 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:03.573 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:03.573 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:03.574 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:03.574 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.574 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.574 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:20:03.574 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:03.574 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.574 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.574 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.574 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:20:03.574 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:03.574 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.574 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.575 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:03.575 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.575 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.577 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.577 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:20:03.577 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:03.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.577 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.578 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:20:03.578 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:03.578 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.578 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.578 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:03.578 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.578 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.578 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.578 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:20:03.578 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:03.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.579 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.579 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:20:03.579 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:03.579 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.579 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.579 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:03.579 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.579 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.579 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.580 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:20:03.580 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:03.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.580 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.580 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.580 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:20:03.580 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:03.580 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.580 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.580 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:20:03.580 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:20:03.581 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:20:03.581 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:20:03.581 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:03.581 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:03.581 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:03.581 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.581 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.581 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:20:03.581 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:03.581 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:03.581 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:03.582 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:03.582 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:03.582 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:03.582 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:03.582 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:03.582 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:03.582 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:03.582 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:03.582 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:03.582 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.582 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.582 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.583 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:20:03.583 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:03.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.583 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.583 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:20:03.583 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:03.583 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.583 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.583 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:03.583 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.583 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.584 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.584 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:20:03.584 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:03.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.584 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.584 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:20:03.584 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:03.584 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.584 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.584 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:03.585 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.585 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.585 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:20:03.585 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:03.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.585 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.585 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.585 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:20:03.585 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:03.585 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.585 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.586 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:03.586 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.586 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.586 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:20:03.586 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:03.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.586 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.586 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.586 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:20:03.586 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:03.587 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.587 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.587 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:20:03.587 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:20:03.587 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:20:03.587 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:20:03.587 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:03.587 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:03.587 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:03.587 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.587 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.588 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:20:03.588 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:03.588 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:03.588 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:03.588 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:03.588 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:03.588 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:03.588 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:03.588 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:03.588 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:03.588 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:03.588 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:03.588 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:03.589 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.589 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.589 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:20:03.589 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:03.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.589 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.589 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.589 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:20:03.589 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:03.589 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.589 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.590 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:03.590 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.590 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.590 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:20:03.590 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:03.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.590 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.590 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.590 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:20:03.590 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:03.591 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.591 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.591 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:03.591 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.591 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.591 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:20:03.591 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:03.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.591 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.591 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:20:03.592 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:03.592 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.592 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.592 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:03.592 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.592 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.592 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:20:03.592 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:03.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.592 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.592 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.593 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:20:03.593 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:03.593 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.593 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.593 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:20:03.593 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:20:03.593 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:20:03.593 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:20:03.593 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:03.593 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:03.593 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:03.593 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.594 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.594 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:20:03.594 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:03.594 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:03.594 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:03.594 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:03.594 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:03.594 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:03.594 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:03.594 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:03.594 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:03.595 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:03.595 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:03.595 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:03.595 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.595 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.595 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:20:03.595 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:03.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.595 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.595 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.595 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:20:03.596 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:03.596 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.596 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.596 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:03.596 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.596 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.596 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:20:03.596 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:03.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.596 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.596 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.597 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:20:03.597 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:03.597 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.597 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.597 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:03.597 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.597 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.597 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:20:03.597 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:03.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.597 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.598 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.598 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:20:03.598 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:03.598 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.598 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.598 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:03.598 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.598 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.598 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:20:03.599 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:03.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.599 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.599 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:20:03.599 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:03.599 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.599 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.599 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:20:03.599 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:20:03.599 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:20:03.599 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:20:03.600 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:03.600 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:03.600 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:03.600 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.600 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:20:03.600 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:03.600 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:03.600 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:03.600 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:03.600 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:03.600 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:03.601 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:03.601 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:03.601 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:03.601 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:03.601 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:03.601 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:03.601 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.601 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.601 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:20:03.601 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:03.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.602 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.602 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:20:03.602 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:03.602 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.602 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.602 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:03.602 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.602 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.602 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:20:03.602 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:03.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.603 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.603 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:20:03.603 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:03.603 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.603 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.603 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:03.603 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.603 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.603 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:20:03.604 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:03.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.604 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.604 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:20:03.604 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:03.604 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.604 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.604 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:03.604 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.604 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.605 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:20:03.605 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:03.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.605 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.605 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:20:03.605 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:03.605 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.605 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.605 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:20:03.605 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:20:03.605 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:20:03.606 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:20:03.606 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:03.606 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:03.606 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:03.606 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.606 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:20:03.606 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:03.606 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:03.606 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:03.606 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:03.607 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:03.607 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:03.607 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:03.607 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:03.607 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:03.607 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:03.607 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:03.607 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:03.607 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.607 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.607 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:20:03.607 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:03.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.608 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.608 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:20:03.608 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:03.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.608 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.608 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:03.608 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.608 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.608 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:20:03.609 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:03.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.609 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.609 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:20:03.609 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:03.609 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.609 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.609 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:03.609 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.609 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.609 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.609 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:20:03.610 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:03.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.610 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.610 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:20:03.610 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:03.610 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.610 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.610 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:03.610 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:03.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.610 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:03.611 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:20:03.611 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:03.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.611 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:03.611 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:20:03.611 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:03.611 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.611 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.611 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:20:03.611 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:20:03.611 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:20:03.612 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:20:03.612 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:03.612 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:03.612 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:03.612 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:03.612 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.612 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:03.612 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:20:03.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.612 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:20:03.612 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:03.612 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:03.612 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:03.613 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:03.613 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:03.613 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:20:03.613 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:03.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.613 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:20:03.613 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:03.613 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:03.613 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:03.613 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:03.613 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:03.613 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:20:03.613 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:03.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.614 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:20:03.614 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:03.614 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:03.614 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:03.614 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:03.614 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:03.614 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:20:03.614 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:03.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.614 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:20:03.615 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:03.615 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:03.615 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:03.615 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:03.615 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:03.615 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:20:03.615 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:03.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.615 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:20:03.615 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:03.615 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:03.615 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:03.616 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:03.616 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:03.616 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:20:03.616 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:03.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.616 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:20:03.616 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:03.616 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:03.616 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:03.616 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:03.616 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:03.617 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:20:03.617 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:03.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.617 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:20:03.617 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:03.617 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:03.617 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:03.617 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:03.617 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:03.617 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:20:03.617 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:03.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:03.618 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:20:03.618 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:03.618 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:03.618 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:03.618 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:03.618 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:03.618 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:20:03.618 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:03.618 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.618 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:03.618 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:20:03.619 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:20:03.619 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:20:03.619 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:20:03.619 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:20:03.619 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:20:03.619 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:20:03.619 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:20:03.619 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:20:03.619 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:20:03.619 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:20:03.619 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:20:03.619 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:20:03.619 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:20:03.619 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:03.620 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:20:03.620 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:20:03.620 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:20:03.620 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:20:03.620 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:20:03.620 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:20:03.620 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:20:03.620 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:20:03.620 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:20:03.620 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:20:03.620 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:20:03.620 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:03.621 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:20:03.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.534+0000 7fb12ee00640 1 -- 172.21.15.161:0/3117602112 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb1040666c0 msgr2=0x7fb104068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:03.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.534+0000 7fb12ee00640 1 --2- 172.21.15.161:0/3117602112 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb1040666c0 0x7fb104068b80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb1100ae000 tx=0x7fb1100b5040 comp rx=0 tx=0).stop 2024-05-22T21:20:03.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.534+0000 7fb12ee00640 1 -- 172.21.15.161:0/3117602112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb128143ec0 msgr2=0x7fb12815a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:03.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.534+0000 7fb12ee00640 1 --2- 172.21.15.161:0/3117602112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb128143ec0 0x7fb12815a5b0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb11800d830 tx=0x7fb11800dcf0 comp rx=0 tx=0).stop 2024-05-22T21:20:03.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.534+0000 7fb12ee00640 1 -- 172.21.15.161:0/3117602112 shutdown_connections 2024-05-22T21:20:03.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.534+0000 7fb12ee00640 1 --2- 172.21.15.161:0/3117602112 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb1040666c0 0x7fb104068b80 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:03.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.534+0000 7fb12ee00640 1 --2- 172.21.15.161:0/3117602112 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb12815d240 0x7fb1281625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:03.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.534+0000 7fb12ee00640 1 --2- 172.21.15.161:0/3117602112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb12815ec90 0x7fb12815cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:03.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.534+0000 7fb12ee00640 1 --2- 172.21.15.161:0/3117602112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb128143ec0 0x7fb12815a5b0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:03.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.534+0000 7fb12ee00640 1 -- 172.21.15.161:0/3117602112 >> 172.21.15.161:0/3117602112 conn(0x7fb1280b0bc0 msgr2=0x7fb1280b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:03.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.534+0000 7fb12ee00640 1 -- 172.21.15.161:0/3117602112 shutdown_connections 2024-05-22T21:20:03.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:03.534+0000 7fb12ee00640 1 -- 172.21.15.161:0/3117602112 wait complete. 2024-05-22T21:20:03.624 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-05-22T21:20:08.625 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:20:08.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.770+0000 7fe566c00640 1 Processor -- start 2024-05-22T21:20:08.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.770+0000 7fe566c00640 1 -- start start 2024-05-22T21:20:08.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.770+0000 7fe566c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe56015d040 0x7fe5600b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:08.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.770+0000 7fe566c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe5600d0cf0 0x7fe5600d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:08.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.770+0000 7fe566c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5600d16c0 con 0x7fe5600cd840 2024-05-22T21:20:08.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.770+0000 7fe566c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5600ce390 con 0x7fe5600d0cf0 2024-05-22T21:20:08.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.770+0000 7fe566c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5600b13d0 con 0x7fe56015d040 2024-05-22T21:20:08.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.770+0000 7fe564e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe5600d0cf0 0x7fe5600d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:08.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.770+0000 7fe566200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe5600cd840 0x7fe5600cdc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44064/0 (socket says 172.21.15.161:44064) 2024-05-22T21:20:08.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.770+0000 7fe566200640 1 -- 172.21.15.161:0/1811741986 learned_addr learned my addr 172.21.15.161:0/1811741986 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:08.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.770+0000 7fe565800640 1 --2- 172.21.15.161:0/1811741986 >> v2:172.21.15.179:3300/0 conn(0x7fe56015d040 0x7fe5600b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:08.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.770+0000 7fe564e00640 1 -- 172.21.15.161:0/1811741986 >> v2:172.21.15.179:3300/0 conn(0x7fe56015d040 msgr2=0x7fe5600b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:08.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.770+0000 7fe564e00640 1 --2- 172.21.15.161:0/1811741986 >> v2:172.21.15.179:3300/0 conn(0x7fe56015d040 0x7fe5600b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:08.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.770+0000 7fe564e00640 1 -- 172.21.15.161:0/1811741986 >> v1:172.21.15.161:6789/0 conn(0x7fe5600cd840 legacy=0x7fe5600cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:08.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.770+0000 7fe564e00640 1 -- 172.21.15.161:0/1811741986 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5600b0aa0 con 0x7fe5600d0cf0 2024-05-22T21:20:08.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.770+0000 7fe55fe00640 1 -- 172.21.15.161:0/1811741986 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471516203 0 0) 0x7fe5600d16c0 con 0x7fe5600cd840 2024-05-22T21:20:08.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.774+0000 7fe564e00640 1 --2- 172.21.15.161:0/1811741986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5600d0cf0 0x7fe5600d1180 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fe56009a880 tx=0x7fe54c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20936dbedbfd787 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:08.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.774+0000 7fe55fe00640 1 -- 172.21.15.161:0/1811741986 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c0ab650 con 0x7fe5600d0cf0 2024-05-22T21:20:08.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.774+0000 7fe55fe00640 1 -- 172.21.15.161:0/1811741986 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c0ab7b0 con 0x7fe5600d0cf0 2024-05-22T21:20:08.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.774+0000 7fe55fe00640 1 -- 172.21.15.161:0/1811741986 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c0abb40 con 0x7fe5600d0cf0 2024-05-22T21:20:08.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.774+0000 7fe566c00640 1 -- 172.21.15.161:0/1811741986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5600d0cf0 msgr2=0x7fe5600d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:08.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.774+0000 7fe566c00640 1 --2- 172.21.15.161:0/1811741986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5600d0cf0 0x7fe5600d1180 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fe56009a880 tx=0x7fe54c01c450 comp rx=0 tx=0).stop 2024-05-22T21:20:08.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.774+0000 7fe566c00640 1 -- 172.21.15.161:0/1811741986 shutdown_connections 2024-05-22T21:20:08.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.774+0000 7fe566c00640 1 --2- 172.21.15.161:0/1811741986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5600d0cf0 0x7fe5600d1180 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:08.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.774+0000 7fe566c00640 1 --2- 172.21.15.161:0/1811741986 >> v2:172.21.15.179:3300/0 conn(0x7fe56015d040 0x7fe5600b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:08.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.774+0000 7fe566c00640 1 -- 172.21.15.161:0/1811741986 >> 172.21.15.161:0/1811741986 conn(0x7fe560099a40 msgr2=0x7fe560099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:08.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.774+0000 7fe566c00640 1 -- 172.21.15.161:0/1811741986 shutdown_connections 2024-05-22T21:20:08.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.774+0000 7fe566c00640 1 -- 172.21.15.161:0/1811741986 wait complete. 2024-05-22T21:20:08.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.774+0000 7fe566c00640 1 Processor -- start 2024-05-22T21:20:08.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.774+0000 7fe566c00640 1 -- start start 2024-05-22T21:20:08.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5600cd840 0x7fe56015c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:08.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe5600d0cf0 0x7fe5600c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:08.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe56015d040 0x7fe5600aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:08.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5601582e0 con 0x7fe56015d040 2024-05-22T21:20:08.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5601576d0 con 0x7fe5600cd840 2024-05-22T21:20:08.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5601579b0 con 0x7fe5600d0cf0 2024-05-22T21:20:08.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe56015d040 0x7fe5600aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:08.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe565800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5600cd840 0x7fe56015c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:08.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe56015d040 0x7fe5600aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53006/0 (socket says 172.21.15.161:53006) 2024-05-22T21:20:08.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566200640 1 -- 172.21.15.161:0/3863287470 learned_addr learned my addr 172.21.15.161:0/3863287470 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:08.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe565800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5600cd840 0x7fe56015c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:59442/0 (socket says 172.21.15.161:59442) 2024-05-22T21:20:08.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe564e00640 1 --2- 172.21.15.161:0/3863287470 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe5600d0cf0 0x7fe5600c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:08.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566200640 1 -- 172.21.15.161:0/3863287470 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe5600d0cf0 msgr2=0x7fe5600c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:08.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566200640 1 --2- 172.21.15.161:0/3863287470 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe5600d0cf0 0x7fe5600c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:08.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566200640 1 -- 172.21.15.161:0/3863287470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5600cd840 msgr2=0x7fe56015c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:08.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566200640 1 --2- 172.21.15.161:0/3863287470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5600cd840 0x7fe56015c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:08.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566200640 1 -- 172.21.15.161:0/3863287470 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c009420 con 0x7fe56015d040 2024-05-22T21:20:08.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566200640 1 --2- 172.21.15.161:0/3863287470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe56015d040 0x7fe5600aed60 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fe558001570 tx=0x7fe55800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:08.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe55d600640 1 -- 172.21.15.161:0/3863287470 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800a640 con 0x7fe56015d040 2024-05-22T21:20:08.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe55d600640 1 -- 172.21.15.161:0/3863287470 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55800a7c0 con 0x7fe56015d040 2024-05-22T21:20:08.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe55d600640 1 -- 172.21.15.161:0/3863287470 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800ab70 con 0x7fe56015d040 2024-05-22T21:20:08.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566c00640 1 -- 172.21.15.161:0/3863287470 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe560157c90 con 0x7fe56015d040 2024-05-22T21:20:08.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566c00640 1 -- 172.21.15.161:0/3863287470 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe560159f20 con 0x7fe56015d040 2024-05-22T21:20:08.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe566c00640 1 -- 172.21.15.161:0/3863287470 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe560158450 con 0x7fe56015d040 2024-05-22T21:20:08.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.778+0000 7fe55d600640 1 -- 172.21.15.161:0/3863287470 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe55800acd0 con 0x7fe56015d040 2024-05-22T21:20:08.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.782+0000 7fe55d600640 1 --2- 172.21.15.161:0/3863287470 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe5400666c0 0x7fe540068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:08.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.782+0000 7fe55d600640 1 -- 172.21.15.161:0/3863287470 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe558083610 con 0x7fe56015d040 2024-05-22T21:20:08.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.782+0000 7fe565800640 1 --2- 172.21.15.161:0/3863287470 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe5400666c0 0x7fe540068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:08.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.782+0000 7fe55d600640 1 -- 172.21.15.161:0/3863287470 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=143}) v3 -- 0x7fe54006e1f0 con 0x7fe56015d040 2024-05-22T21:20:08.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.782+0000 7fe565800640 1 --2- 172.21.15.161:0/3863287470 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe5400666c0 0x7fe540068b80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe554009ac0 tx=0x7fe554009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:08.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.786+0000 7fe55d600640 1 -- 172.21.15.161:0/3863287470 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe558018060 con 0x7fe56015d040 2024-05-22T21:20:08.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.942+0000 7fe566c00640 1 -- 172.21.15.161:0/3863287470 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe560067070 con 0x7fe56015d040 2024-05-22T21:20:08.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.942+0000 7fe55d600640 1 -- 172.21.15.161:0/3863287470 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v142) v1 ==== 81+0+33569 (secure 0 0 0) 0x7fe558050990 con 0x7fe56015d040 2024-05-22T21:20:08.947 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:08.947 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:20:08.947 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 142, 2024-05-22T21:20:08.947 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:20:08.947 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:20:08.947 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:20:00.256219+0000", 2024-05-22T21:20:08.947 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:20:08.948 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:20:08.948 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:20:08.948 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:20:08.948 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:20:08.948 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:20:08.948 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:20:08.948 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:20:08.948 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:20:08.948 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:08.948 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:20:08.948 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:20:08.948 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:20:08.948 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:20:08.949 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:20:08.949 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 17, 2024-05-22T21:20:08.949 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:20:08.949 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:20:08.949 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:20:08.949 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:20:08.949 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:20:08.949 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:20:08.949 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.949 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:20:08.949 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:20:08.949 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:20:08.949 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:20:08.949 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:20:08.950 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:08.950 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:08.950 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:08.950 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:08.950 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:08.950 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:08.950 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:08.950 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:08.950 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:08.950 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:08.950 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:20:08.950 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:20:08.951 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:20:08.951 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:20:08.951 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:20:08.951 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:08.951 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:08.951 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:08.951 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:08.951 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:08.951 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:08.951 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:08.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.951 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:20:08.951 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:08.951 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:08.952 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:08.952 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:08.952 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:08.952 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:08.952 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:08.952 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:08.952 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:08.952 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:20:08.952 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:08.952 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:08.952 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:08.952 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:08.952 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:08.952 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:08.953 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:08.953 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:08.953 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:08.953 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:08.953 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:08.953 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:08.953 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:08.953 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:08.953 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:08.953 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:08.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.953 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:08.953 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:08.954 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:08.954 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:08.954 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:08.954 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:08.954 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:08.954 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:08.954 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:08.954 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:08.954 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:08.954 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:08.954 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:20:08.954 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:20:08.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.955 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:08.955 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:20:08.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.955 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:08.955 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:20:08.955 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:20:08.955 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:20:08.955 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:20:08.955 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:20:08.955 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:08.955 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:08.955 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:08.955 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.956 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.956 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 16, 2024-05-22T21:20:08.956 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:20:08.956 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:19:44.080742+0000", 2024-05-22T21:20:08.956 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:20:08.956 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:20:08.956 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:08.956 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:08.956 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:08.956 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:08.956 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:08.956 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:08.956 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:08.957 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:08.957 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:08.957 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:08.957 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:20:08.957 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:20:08.957 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:20:08.957 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:20:08.957 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:20:08.957 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:08.957 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:08.957 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:08.957 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:08.957 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:08.957 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:08.958 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:08.958 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.958 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "139", 2024-05-22T21:20:08.958 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:08.958 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:08.958 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:08.958 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:08.958 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:08.958 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:08.958 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:08.958 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:08.958 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:08.958 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:20:08.958 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:08.959 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:08.959 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:08.959 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:08.959 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:08.959 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:08.959 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:08.959 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:08.959 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:08.959 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:08.959 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:08.959 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:08.959 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:08.959 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:08.960 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:08.960 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:08.960 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.960 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:08.960 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:08.960 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:08.960 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:08.960 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:08.960 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:08.960 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:08.960 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:08.960 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:08.960 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:08.960 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:08.961 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:08.961 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:20:08.961 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:20:08.961 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:20:08.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.961 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:08.961 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:20:08.961 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:20:08.961 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.961 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:08.961 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:20:08.961 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:20:08.961 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:20:08.961 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:20:08.962 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:20:08.962 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:08.962 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:08.962 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:08.962 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.962 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 17, 2024-05-22T21:20:08.962 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:20:08.962 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:19:45.388618+0000", 2024-05-22T21:20:08.962 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1027, 2024-05-22T21:20:08.962 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full,full_quota", 2024-05-22T21:20:08.962 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:08.963 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:08.963 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:08.963 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:08.963 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:08.963 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:08.963 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:08.963 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:08.963 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:08.963 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:08.963 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:20:08.963 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:20:08.963 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:20:08.963 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:20:08.963 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:20:08.964 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:08.964 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:08.964 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:08.964 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:08.964 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:08.964 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:08.964 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:08.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.964 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "142", 2024-05-22T21:20:08.964 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:08.964 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:08.964 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:08.964 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:08.964 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:08.965 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:08.965 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:08.965 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:08.965 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:08.965 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 33554432, 2024-05-22T21:20:08.965 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:08.965 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:08.965 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:08.965 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:08.965 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:08.965 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:08.965 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:08.965 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:08.965 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:08.966 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:08.966 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:08.966 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:08.966 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:08.966 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:08.966 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:08.966 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:08.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.966 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:08.966 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:08.966 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:08.966 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:08.966 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:08.966 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:08.967 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:08.967 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:08.967 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:08.967 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:08.967 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:08.967 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:08.967 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:20:08.967 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:20:08.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.967 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:08.967 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:20:08.967 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:20:08.967 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.968 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:08.968 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:20:08.968 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:20:08.968 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:20:08.968 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:20:08.968 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:20:08.968 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:08.968 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:08.968 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:08.968 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.968 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.968 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:08.968 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:20:08.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.969 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:20:08.969 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:08.969 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:08.969 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:08.969 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:08.969 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:08.969 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:08.969 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:08.969 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:08.969 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:08.969 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:08.969 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:08.969 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:08.969 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.969 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.970 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.970 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:20:08.970 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:08.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.970 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.970 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:20:08.970 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:08.970 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.970 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.970 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:08.970 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.970 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.971 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.971 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:20:08.971 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:08.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.971 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.971 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.971 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:20:08.971 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:08.971 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.971 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.971 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:08.971 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.971 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.972 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.972 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:20:08.972 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:08.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.972 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.972 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:20:08.972 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:08.972 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.972 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.972 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:08.972 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.972 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.973 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.973 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:20:08.973 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:08.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.973 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.973 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.973 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:20:08.973 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:08.973 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.973 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.973 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:20:08.973 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:20:08.973 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:20:08.973 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:20:08.974 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:08.974 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:08.974 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:08.974 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.974 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.974 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:20:08.974 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:08.974 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:08.974 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:08.974 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:08.974 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:08.974 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:08.975 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:08.975 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:08.975 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:08.975 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:08.975 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:08.975 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:08.975 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.975 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.975 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:20:08.975 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:08.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.975 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.975 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:20:08.976 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:08.976 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.976 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.976 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:08.976 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.976 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.976 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:20:08.976 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:08.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.976 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.976 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.976 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:20:08.977 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:08.977 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.977 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.977 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:08.977 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.977 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.977 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:20:08.977 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:08.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.977 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.977 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.977 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:20:08.978 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:08.978 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.978 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.978 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:08.978 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.978 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.978 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:20:08.978 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:08.978 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.978 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.978 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.978 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:20:08.979 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:08.979 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.979 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.979 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:20:08.979 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:20:08.979 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:20:08.979 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:20:08.979 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:08.979 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:08.979 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:08.979 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.979 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:20:08.980 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:08.980 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:08.980 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:08.980 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:08.980 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:08.980 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:08.980 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:08.980 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:08.980 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:08.980 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:08.980 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:08.980 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:08.980 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.980 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.981 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.981 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:20:08.981 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:08.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.981 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.981 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:20:08.981 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:08.981 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.981 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.981 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:08.981 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.981 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.982 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.982 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:20:08.982 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:08.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.982 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.982 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:20:08.982 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:08.982 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.982 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.982 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:08.983 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.983 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.983 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:20:08.983 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:08.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.983 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.983 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.983 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:20:08.983 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:08.983 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.983 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.983 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.983 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:08.984 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.984 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.984 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:20:08.984 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:08.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.984 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.984 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.984 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:20:08.984 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:08.984 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.984 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.984 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:20:08.985 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:20:08.985 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:20:08.985 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:20:08.985 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:08.985 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:08.985 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:08.985 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.985 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.985 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.985 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:20:08.985 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:08.985 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:08.985 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:08.986 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:08.986 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:08.986 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:08.986 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:08.986 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:08.986 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:08.986 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:08.986 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:08.986 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:08.986 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.986 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.986 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.986 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:20:08.987 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:08.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.987 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.987 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:20:08.987 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:08.987 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.987 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.987 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:08.987 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.987 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.987 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.988 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:20:08.988 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:08.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.988 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.988 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:20:08.988 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:08.988 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.988 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.988 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:08.988 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.988 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.988 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.988 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:20:08.989 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:08.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.989 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.989 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:20:08.989 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:08.989 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.989 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.989 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:08.989 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.989 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.989 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:20:08.990 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:08.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.990 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.990 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.990 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:20:08.990 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:08.990 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.990 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.990 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:20:08.990 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:20:08.990 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:20:08.991 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:20:08.991 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:08.991 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:08.991 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:08.991 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.991 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:20:08.991 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:08.991 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:08.991 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:08.991 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:08.991 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:08.991 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:08.992 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:08.992 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:08.992 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:08.992 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:08.992 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:08.992 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:08.992 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.992 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.992 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:20:08.992 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:08.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.993 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.993 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:20:08.993 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:08.993 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.993 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.993 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:08.993 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.993 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.993 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.993 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:20:08.993 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:08.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.994 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.994 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:20:08.994 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:08.994 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.994 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.994 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:08.994 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.994 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.994 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:20:08.995 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:08.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.995 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.995 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:20:08.995 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:08.995 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.995 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.995 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:08.995 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.995 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.995 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.996 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:20:08.996 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:08.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.996 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.996 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:20:08.996 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:08.996 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.996 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.996 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:20:08.996 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:20:08.996 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:20:08.996 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:20:08.997 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:08.997 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:08.997 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:08.997 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.997 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.997 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:20:08.997 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:08.997 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:08.997 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:08.997 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:08.997 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:08.997 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:08.998 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:08.998 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:08.998 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:08.998 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:08.998 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:08.998 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:08.998 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.998 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.998 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:20:08.998 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:08.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.999 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:08.999 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:20:08.999 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:08.999 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:08.999 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:08.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:08.999 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:08.999 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:08.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:08.999 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:08.999 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:20:08.999 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:08.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.000 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:09.000 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:20:09.000 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:09.000 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:09.000 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:09.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.000 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:09.000 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:09.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.000 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:09.000 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:20:09.000 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:09.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.001 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.001 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:09.001 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:20:09.001 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:09.001 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:09.001 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:09.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.001 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:09.001 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:09.001 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.001 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:09.001 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:20:09.002 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:09.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.002 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:09.002 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:20:09.002 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:09.002 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:09.002 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:09.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.002 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:20:09.002 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:20:09.002 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:20:09.003 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:20:09.003 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:09.003 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:09.003 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:09.003 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:09.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.003 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.003 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:20:09.003 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:09.003 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:09.003 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:09.004 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:09.004 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:09.004 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:09.004 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:09.004 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:09.004 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:09.004 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:09.004 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:09.004 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:09.004 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:09.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.004 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:09.004 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:20:09.005 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:09.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.005 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:09.005 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:20:09.005 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:09.005 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:09.005 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:09.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.005 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:09.005 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:09.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.005 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:09.006 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:20:09.006 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:09.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.006 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:09.006 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:20:09.006 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:09.006 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:09.006 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:09.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.006 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:09.006 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:09.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.007 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:09.007 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:20:09.007 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:09.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.007 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:09.007 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:20:09.007 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:09.007 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:09.007 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:09.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.008 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:09.008 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:09.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.008 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:09.008 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:20:09.008 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:09.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.008 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:09.008 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:20:09.008 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:09.008 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:09.008 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:09.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.009 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:20:09.009 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:20:09.009 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:20:09.009 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:20:09.009 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:09.009 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:09.009 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:09.009 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:09.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.009 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:20:09.009 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:09.009 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:09.009 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:09.010 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:09.010 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:09.010 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:09.010 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:09.010 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:09.010 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:09.010 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:09.010 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:09.010 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:09.010 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:09.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.010 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:09.010 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:20:09.011 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:09.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.011 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:09.011 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:20:09.011 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:09.011 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:09.011 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:09.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.011 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:09.011 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:09.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.011 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:09.012 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:20:09.012 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:09.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.012 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:09.012 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:20:09.012 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:09.012 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:09.012 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:09.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.012 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:09.012 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:09.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.012 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:09.013 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:20:09.013 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:09.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.013 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:09.013 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:20:09.013 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:09.013 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:09.013 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:09.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.013 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:09.013 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:09.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.014 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:09.014 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:20:09.014 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:09.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.014 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:09.014 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:20:09.014 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:09.014 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:09.014 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:09.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.014 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:20:09.015 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:20:09.015 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:20:09.015 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:20:09.015 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:09.015 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:09.015 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:09.015 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:09.015 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:09.015 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:09.015 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:20:09.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.015 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:20:09.016 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:09.016 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:09.016 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:09.016 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:09.016 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:09.016 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:20:09.016 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:09.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.016 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:20:09.016 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:09.016 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:09.016 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:09.016 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:20:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:20:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:20:09.017 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:20:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:20:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.018 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:20:09.019 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:09.019 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:09.019 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:09.019 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:09.019 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:09.019 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:20:09.019 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:09.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.019 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:20:09.019 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:09.019 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:09.020 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:09.020 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:09.020 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:09.020 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:20:09.020 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:09.020 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.020 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.020 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:20:09.020 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:09.020 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:09.020 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:09.020 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:09.020 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:09.020 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:20:09.021 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:09.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:09.021 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:20:09.021 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:09.021 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:09.021 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:09.021 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:09.021 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:09.021 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:20:09.021 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:09.021 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:09.021 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:09.022 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:20:09.022 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:20:09.022 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:20:09.022 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:20:09.022 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:20:09.022 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:20:09.022 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:20:09.022 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:20:09.022 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:20:09.022 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:20:09.022 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:20:09.022 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:20:09.023 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:20:09.023 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:20:09.023 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:09.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:09.023 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:20:09.023 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:20:09.023 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:20:09.023 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:20:09.023 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:20:09.023 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:20:09.023 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:20:09.023 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:20:09.023 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:20:09.023 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:20:09.024 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:20:09.024 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:09.024 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:20:09.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.946+0000 7fe566c00640 1 -- 172.21.15.161:0/3863287470 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe5400666c0 msgr2=0x7fe540068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:09.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.946+0000 7fe566c00640 1 --2- 172.21.15.161:0/3863287470 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe5400666c0 0x7fe540068b80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe554009ac0 tx=0x7fe554009190 comp rx=0 tx=0).stop 2024-05-22T21:20:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.946+0000 7fe566c00640 1 -- 172.21.15.161:0/3863287470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe56015d040 msgr2=0x7fe5600aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.946+0000 7fe566c00640 1 --2- 172.21.15.161:0/3863287470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe56015d040 0x7fe5600aed60 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fe558001570 tx=0x7fe55800cca0 comp rx=0 tx=0).stop 2024-05-22T21:20:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.946+0000 7fe566c00640 1 -- 172.21.15.161:0/3863287470 shutdown_connections 2024-05-22T21:20:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.946+0000 7fe566c00640 1 --2- 172.21.15.161:0/3863287470 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe5400666c0 0x7fe540068b80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.946+0000 7fe566c00640 1 --2- 172.21.15.161:0/3863287470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe56015d040 0x7fe5600aed60 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.946+0000 7fe566c00640 1 --2- 172.21.15.161:0/3863287470 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe5600d0cf0 0x7fe5600c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.946+0000 7fe566c00640 1 --2- 172.21.15.161:0/3863287470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5600cd840 0x7fe56015c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.946+0000 7fe566c00640 1 -- 172.21.15.161:0/3863287470 >> 172.21.15.161:0/3863287470 conn(0x7fe560099a40 msgr2=0x7fe5600d1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.946+0000 7fe566c00640 1 -- 172.21.15.161:0/3863287470 shutdown_connections 2024-05-22T21:20:09.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:08.946+0000 7fe566c00640 1 -- 172.21.15.161:0/3863287470 wait complete. 2024-05-22T21:20:09.027 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-05-22T21:20:14.028 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:20:14.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.174+0000 7f5e6e600640 1 Processor -- start 2024-05-22T21:20:14.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.174+0000 7f5e6e600640 1 -- start start 2024-05-22T21:20:14.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.174+0000 7f5e6e600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e680cbfa0 0x7f5e6815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.174+0000 7f5e6e600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e6815c650 con 0x7f5e6806cff0 2024-05-22T21:20:14.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.174+0000 7f5e6e600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e680711f0 con 0x7f5e680d0ba0 2024-05-22T21:20:14.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.174+0000 7f5e6e600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e681590f0 con 0x7f5e680cbfa0 2024-05-22T21:20:14.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.174+0000 7f5e67e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f5e6806cff0 0x7f5e68071680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44076/0 (socket says 172.21.15.161:44076) 2024-05-22T21:20:14.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.174+0000 7f5e67e00640 1 -- 172.21.15.161:0/4142603797 learned_addr learned my addr 172.21.15.161:0/4142603797 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:14.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.174+0000 7f5e67400640 1 --2- 172.21.15.161:0/4142603797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e680cbfa0 0x7f5e6815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.174+0000 7f5e66a00640 1 -- 172.21.15.161:0/4142603797 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698319306 0 0) 0x7f5e6815c650 con 0x7f5e6806cff0 2024-05-22T21:20:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.174+0000 7f5e66a00640 1 -- 172.21.15.161:0/4142603797 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e4c003530 con 0x7f5e6806cff0 2024-05-22T21:20:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.174+0000 7f5e66a00640 1 -- 172.21.15.161:0/4142603797 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994691341 0 0) 0x7f5e680711f0 con 0x7f5e680d0ba0 2024-05-22T21:20:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.174+0000 7f5e66a00640 1 -- 172.21.15.161:0/4142603797 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e6815c650 con 0x7f5e680d0ba0 2024-05-22T21:20:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.174+0000 7f5e67400640 1 -- 172.21.15.161:0/4142603797 >> v1:172.21.15.179:6789/0 conn(0x7f5e680d0ba0 legacy=0x7f5e680cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.174+0000 7f5e67400640 1 -- 172.21.15.161:0/4142603797 >> v1:172.21.15.161:6789/0 conn(0x7f5e6806cff0 legacy=0x7f5e68071680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.174+0000 7f5e67400640 1 -- 172.21.15.161:0/4142603797 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e680ca900 con 0x7f5e680cbfa0 2024-05-22T21:20:14.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.178+0000 7f5e67400640 1 --2- 172.21.15.161:0/4142603797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e680cbfa0 0x7f5e6815b6b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5e580097f0 tx=0x7f5e5801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca3e5f5be0d70489 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:14.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.178+0000 7f5e66a00640 1 -- 172.21.15.161:0/4142603797 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e580ab600 con 0x7f5e680cbfa0 2024-05-22T21:20:14.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.178+0000 7f5e66a00640 1 -- 172.21.15.161:0/4142603797 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e580a7050 con 0x7f5e680cbfa0 2024-05-22T21:20:14.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.178+0000 7f5e66a00640 1 -- 172.21.15.161:0/4142603797 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e580ab980 con 0x7f5e680cbfa0 2024-05-22T21:20:14.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.178+0000 7f5e6e600640 1 -- 172.21.15.161:0/4142603797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e680cbfa0 msgr2=0x7f5e6815b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.178+0000 7f5e6e600640 1 --2- 172.21.15.161:0/4142603797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e680cbfa0 0x7f5e6815b6b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5e580097f0 tx=0x7f5e5801c3c0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.178+0000 7f5e6e600640 1 -- 172.21.15.161:0/4142603797 shutdown_connections 2024-05-22T21:20:14.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.178+0000 7f5e6e600640 1 --2- 172.21.15.161:0/4142603797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e680cbfa0 0x7f5e6815b6b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.178+0000 7f5e6e600640 1 -- 172.21.15.161:0/4142603797 >> 172.21.15.161:0/4142603797 conn(0x7f5e680d00b0 msgr2=0x7f5e680d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:14.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.178+0000 7f5e6e600640 1 -- 172.21.15.161:0/4142603797 shutdown_connections 2024-05-22T21:20:14.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.178+0000 7f5e6e600640 1 -- 172.21.15.161:0/4142603797 wait complete. 2024-05-22T21:20:14.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e6e600640 1 Processor -- start 2024-05-22T21:20:14.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e6e600640 1 -- start start 2024-05-22T21:20:14.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e6e600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e6806cff0 0x7f5e680b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e6e600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e680cbfa0 0x7f5e680b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e6e600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e680d0ba0 0x7f5e680adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e6e600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e680b36b0 con 0x7f5e680d0ba0 2024-05-22T21:20:14.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e6e600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e681518b0 con 0x7f5e6806cff0 2024-05-22T21:20:14.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e6e600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e680cf600 con 0x7f5e680cbfa0 2024-05-22T21:20:14.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e67e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e680d0ba0 0x7f5e680adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e67e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e680d0ba0 0x7f5e680adf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53016/0 (socket says 172.21.15.161:53016) 2024-05-22T21:20:14.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e67e00640 1 -- 172.21.15.161:0/1532681512 learned_addr learned my addr 172.21.15.161:0/1532681512 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:14.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e67400640 1 --2- 172.21.15.161:0/1532681512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e6806cff0 0x7f5e680b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e5ea00640 1 --2- 172.21.15.161:0/1532681512 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e680cbfa0 0x7f5e680b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e67e00640 1 -- 172.21.15.161:0/1532681512 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e680cbfa0 msgr2=0x7f5e680b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e67e00640 1 --2- 172.21.15.161:0/1532681512 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e680cbfa0 0x7f5e680b3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e67e00640 1 -- 172.21.15.161:0/1532681512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e6806cff0 msgr2=0x7f5e680b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e67e00640 1 --2- 172.21.15.161:0/1532681512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e6806cff0 0x7f5e680b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e67e00640 1 -- 172.21.15.161:0/1532681512 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e58009420 con 0x7f5e680d0ba0 2024-05-22T21:20:14.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e67e00640 1 --2- 172.21.15.161:0/1532681512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e680d0ba0 0x7f5e680adf90 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000caa0 tx=0x7f5e6000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:14.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e5fe00640 1 -- 172.21.15.161:0/1532681512 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6000ac00 con 0x7f5e680d0ba0 2024-05-22T21:20:14.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e5fe00640 1 -- 172.21.15.161:0/1532681512 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6000ad60 con 0x7f5e680d0ba0 2024-05-22T21:20:14.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e5fe00640 1 -- 172.21.15.161:0/1532681512 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e60012620 con 0x7f5e680d0ba0 2024-05-22T21:20:14.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e6e600640 1 -- 172.21.15.161:0/1532681512 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e680cee10 con 0x7f5e680d0ba0 2024-05-22T21:20:14.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e6e600640 1 -- 172.21.15.161:0/1532681512 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e680cfb80 con 0x7f5e680d0ba0 2024-05-22T21:20:14.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.182+0000 7f5e6e600640 1 -- 172.21.15.161:0/1532681512 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e68067220 con 0x7f5e680d0ba0 2024-05-22T21:20:14.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.186+0000 7f5e5fe00640 1 -- 172.21.15.161:0/1532681512 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5e60007070 con 0x7f5e680d0ba0 2024-05-22T21:20:14.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.186+0000 7f5e5fe00640 1 --2- 172.21.15.161:0/1532681512 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5e44066690 0x7f5e44068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.186+0000 7f5e5fe00640 1 -- 172.21.15.161:0/1532681512 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5e600840e0 con 0x7f5e680d0ba0 2024-05-22T21:20:14.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.186+0000 7f5e67400640 1 --2- 172.21.15.161:0/1532681512 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5e44066690 0x7f5e44068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.186+0000 7f5e67400640 1 --2- 172.21.15.161:0/1532681512 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5e44066690 0x7f5e44068b50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5e580ad000 tx=0x7f5e580a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:14.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.190+0000 7f5e5fe00640 1 -- 172.21.15.161:0/1532681512 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5e600514c0 con 0x7f5e680d0ba0 2024-05-22T21:20:14.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.346+0000 7f5e6e600640 1 -- 172.21.15.161:0/1532681512 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5e68072180 con 0x7f5e680d0ba0 2024-05-22T21:20:14.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.350+0000 7f5e5fe00640 1 -- 172.21.15.161:0/1532681512 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v145) v1 ==== 81+0+33982 (secure 0 0 0) 0x7f5e60054ae0 con 0x7f5e680d0ba0 2024-05-22T21:20:14.353 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:14.353 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:20:14.353 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 145, 2024-05-22T21:20:14.353 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:20:14.353 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:20:14.353 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:20:13.272722+0000", 2024-05-22T21:20:14.354 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:20:14.354 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:20:14.354 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:20:14.354 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:20:14.354 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:20:14.354 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:20:14.354 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:20:14.354 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:20:14.354 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:20:14.354 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:14.354 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:20:14.354 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:20:14.354 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:20:14.355 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:20:14.355 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:20:14.355 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 17, 2024-05-22T21:20:14.355 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:20:14.355 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:20:14.355 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "luminous", 2024-05-22T21:20:14.355 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:20:14.355 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:20:14.355 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:20:14.355 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.355 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:20:14.355 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:20:14.355 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:20:14.356 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:20:14.356 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:20:14.356 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:14.356 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:14.356 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:14.356 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:14.356 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:14.356 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:14.356 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:14.356 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:14.356 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:14.356 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:14.356 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:20:14.356 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:20:14.357 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:20:14.357 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:20:14.357 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:20:14.357 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:14.357 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:14.357 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:14.357 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:14.357 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:14.357 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:14.357 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:14.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.357 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:20:14.357 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:14.358 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:14.358 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:14.358 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:14.358 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:14.358 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:14.358 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:14.358 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:14.358 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:14.358 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:20:14.358 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:14.358 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:14.358 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:14.358 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:14.359 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:14.359 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:14.359 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:14.359 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:14.359 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:14.359 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:14.359 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:14.359 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:14.359 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:14.359 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:14.359 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:14.359 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:14.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.360 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:14.360 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:14.360 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:14.360 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:14.360 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:14.360 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:14.360 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:14.360 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:14.360 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:14.360 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:14.360 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:14.360 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:14.360 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:20:14.361 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:20:14.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.361 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:14.361 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:20:14.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.361 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:14.361 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:20:14.361 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:20:14.361 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:20:14.361 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:20:14.361 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:20:14.361 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:14.361 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:14.362 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:14.362 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.362 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.362 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.362 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 16, 2024-05-22T21:20:14.362 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:20:14.362 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:19:44.080742+0000", 2024-05-22T21:20:14.362 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:20:14.362 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:20:14.362 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:14.362 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:14.362 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:14.362 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:14.363 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:14.363 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:14.363 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:14.363 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:14.363 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:14.363 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:14.363 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:20:14.363 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:20:14.363 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:20:14.363 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:20:14.363 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:20:14.363 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:14.363 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:14.364 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:14.364 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:14.364 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:14.364 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:14.364 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:14.364 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.364 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "139", 2024-05-22T21:20:14.364 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:14.364 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:14.364 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:14.364 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:14.364 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:14.364 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:14.364 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:14.365 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:14.365 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:14.365 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:20:14.365 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:14.365 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:14.365 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:14.365 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:14.365 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:14.365 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:14.365 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:14.365 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:14.365 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:14.365 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:14.366 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:14.366 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:14.366 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:14.366 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:14.366 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:14.366 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:14.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.366 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:14.366 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:14.366 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:14.366 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:14.366 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:14.366 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:14.367 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:14.367 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:14.367 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:14.367 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:14.367 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:14.367 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:14.367 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:20:14.367 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:20:14.367 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:20:14.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.367 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:14.367 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:20:14.367 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:20:14.368 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.368 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.368 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:14.368 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:20:14.368 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:20:14.368 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:20:14.368 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:20:14.368 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:20:14.368 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:14.368 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:14.368 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:14.368 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.368 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.368 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.369 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 17, 2024-05-22T21:20:14.369 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:20:14.369 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:19:45.388618+0000", 2024-05-22T21:20:14.369 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:20:14.369 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:20:14.369 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:14.369 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:14.369 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:14.369 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:14.369 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:14.370 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:14.370 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:14.370 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:14.370 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:14.370 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:14.370 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:20:14.370 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:20:14.370 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:20:14.370 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:20:14.370 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:20:14.370 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:14.370 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:14.370 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:14.371 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:14.371 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:14.371 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:14.371 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:14.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.371 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "143", 2024-05-22T21:20:14.371 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:14.371 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:14.371 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:14.371 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:14.371 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:14.371 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:14.372 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:14.372 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:14.372 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:14.372 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 33554432, 2024-05-22T21:20:14.372 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:14.372 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:14.372 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:14.372 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:14.372 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:14.372 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:14.372 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:14.372 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:14.372 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:14.373 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:14.373 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:14.373 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:14.373 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:14.373 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:14.373 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:14.373 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:14.373 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.373 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:14.373 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:14.373 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:14.373 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:14.373 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:14.373 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:14.374 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:14.374 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:14.374 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:14.374 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:14.374 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:14.374 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:14.374 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:20:14.374 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:20:14.374 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.374 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:14.374 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:20:14.375 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:20:14.375 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.375 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:14.375 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.6299999952316284, 2024-05-22T21:20:14.375 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.6299999952316284, 2024-05-22T21:20:14.375 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:20:14.375 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.6299999952316284, 2024-05-22T21:20:14.375 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.6299999952316284, 2024-05-22T21:20:14.375 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:14.375 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:14.376 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:14.376 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.376 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.376 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:14.376 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:20:14.376 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.376 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:20:14.376 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:14.376 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:14.376 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:14.376 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:14.376 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:14.376 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:14.376 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:14.377 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:14.377 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:14.377 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:14.377 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:14.377 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:14.377 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.377 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.377 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.377 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:20:14.377 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:14.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.377 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.377 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.378 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:20:14.378 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:14.378 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.378 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.378 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:14.378 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.378 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.378 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.378 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:20:14.378 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:14.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.378 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.379 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.379 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:20:14.379 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:14.379 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.379 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.379 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:14.379 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.379 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.379 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.379 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:20:14.379 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:14.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.380 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.380 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.380 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:20:14.380 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:14.380 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.380 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.380 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:14.380 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.380 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.380 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.380 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:20:14.380 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:14.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.381 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.381 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.381 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:20:14.381 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:14.381 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.381 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.381 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.381 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:20:14.381 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:20:14.381 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:20:14.381 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:20:14.381 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:14.381 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:14.382 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:14.382 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.382 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.382 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.382 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:20:14.382 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:14.382 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:14.382 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:14.382 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:14.382 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:14.382 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:14.382 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:14.382 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:14.383 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:14.383 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:14.383 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:14.383 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:14.383 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.383 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.383 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.383 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:20:14.383 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:14.383 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.383 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.383 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.383 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:20:14.384 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:14.384 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.384 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.384 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:14.384 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.384 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.384 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.384 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:20:14.384 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:14.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.384 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.384 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.384 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:20:14.385 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:14.385 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.385 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.385 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:14.385 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.385 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.385 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.385 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:20:14.385 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:14.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.385 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.385 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.386 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:20:14.386 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:14.386 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.386 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.386 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.386 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:14.386 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.386 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.386 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.386 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:20:14.386 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:14.386 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.386 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.387 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.387 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:20:14.387 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:14.387 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.387 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.387 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:20:14.387 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:20:14.387 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:20:14.387 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:20:14.387 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:14.387 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:14.387 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:14.387 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.388 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.388 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:20:14.388 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:14.388 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:14.388 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:14.388 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:14.388 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:14.388 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:14.388 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:14.388 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:14.388 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 144, 2024-05-22T21:20:14.388 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:14.389 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:14.389 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:14.389 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.389 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.389 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.389 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:20:14.389 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:14.389 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.389 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.389 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.389 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:20:14.389 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:14.389 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.389 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.390 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:14.390 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.390 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.390 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.390 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:20:14.390 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:14.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.390 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.390 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.390 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:20:14.390 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:14.391 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.391 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.391 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:14.391 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.391 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.391 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.391 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:20:14.391 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:14.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.391 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.391 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.391 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:20:14.392 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:14.392 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.392 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.392 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:14.392 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.392 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.392 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.392 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:20:14.392 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:14.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.392 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.392 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.393 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:20:14.393 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:14.393 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.393 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.393 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:20:14.393 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:20:14.393 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:20:14.393 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:20:14.393 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:14.393 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:14.393 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:14.393 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.394 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.394 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:20:14.394 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:14.394 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:14.394 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:14.394 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:14.394 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:14.394 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:14.394 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:14.394 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:14.394 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:14.395 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:14.395 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:14.395 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:14.395 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.395 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.395 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.395 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:20:14.395 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:14.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.395 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.395 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.395 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:20:14.395 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:14.396 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.396 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.396 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:14.396 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.396 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.396 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.396 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:20:14.396 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:14.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.396 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.396 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.396 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:20:14.397 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:14.397 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.397 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.397 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.397 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:14.397 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.397 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.397 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.397 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:20:14.397 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:14.397 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.397 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.397 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.398 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:20:14.398 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:14.398 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.398 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.398 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.398 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:14.398 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.398 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.398 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.398 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:20:14.398 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:14.398 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.399 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.399 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.399 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:20:14.399 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:14.399 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.399 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.399 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.399 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:20:14.399 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:20:14.399 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:20:14.399 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:20:14.399 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:14.399 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:14.400 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:14.400 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.400 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.400 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:20:14.400 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:14.400 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:14.400 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:14.400 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:14.400 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:14.400 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:14.400 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:14.400 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:14.401 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:14.401 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:14.401 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:14.401 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:14.401 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.401 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.401 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.401 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:20:14.401 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:14.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.401 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.401 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.401 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:20:14.402 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:14.402 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.402 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.402 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:14.402 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.402 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.402 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.402 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:20:14.402 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:14.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.402 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.403 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.403 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:20:14.403 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:14.403 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.403 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.403 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:14.403 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.403 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.403 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.403 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:20:14.403 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:14.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.404 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.404 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.404 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:20:14.404 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:14.404 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.404 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.404 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:14.404 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.404 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.404 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.404 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:20:14.404 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:14.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.405 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.405 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.405 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:20:14.405 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:14.405 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.405 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.405 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:20:14.405 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:20:14.405 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:20:14.405 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:20:14.405 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:14.405 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:14.405 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:14.406 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.406 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.406 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:20:14.406 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:14.406 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:14.406 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:14.406 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:14.406 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:14.406 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:14.406 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:14.406 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:14.406 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 144, 2024-05-22T21:20:14.407 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:14.407 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:14.407 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:14.407 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.407 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.407 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:20:14.407 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:14.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.407 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.407 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.407 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:20:14.407 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:14.407 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.408 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.408 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:14.408 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.408 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.408 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:20:14.408 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:14.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.408 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.408 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.408 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:20:14.408 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:14.409 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.409 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.409 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:14.409 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.409 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.409 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.409 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:20:14.409 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:14.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.409 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.409 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.409 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:20:14.409 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:14.410 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.410 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.410 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:14.410 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.410 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.410 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.410 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:20:14.410 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:14.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.410 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.410 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.410 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:20:14.411 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:14.411 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.411 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.411 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:20:14.411 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:20:14.411 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:20:14.411 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:20:14.411 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:14.411 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:14.411 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:14.411 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.412 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.412 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:20:14.412 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:14.412 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:14.412 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:14.412 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:14.412 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:14.412 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:14.412 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:14.412 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:14.412 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:14.412 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:14.412 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:14.413 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:14.413 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.413 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.413 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:20:14.413 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:14.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.413 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.413 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.413 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:20:14.413 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:14.413 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.413 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.414 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:14.414 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.414 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.414 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.414 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:20:14.414 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:14.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.414 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.414 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.414 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:20:14.414 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:14.414 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.414 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.415 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:14.415 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.415 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.415 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.415 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:20:14.415 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:14.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.415 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.415 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.415 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:20:14.415 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:14.415 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.415 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.416 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:14.416 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.416 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.416 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:20:14.416 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:14.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.416 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.416 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.416 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:20:14.416 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:14.416 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.417 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.417 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:20:14.417 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:20:14.417 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:20:14.417 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:20:14.417 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:14.417 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:14.417 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:14.417 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.417 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:20:14.417 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:14.418 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:14.418 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:14.418 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:14.418 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:14.418 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:14.418 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:14.418 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:14.418 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:14.418 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:14.418 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:14.418 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:14.418 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.418 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.419 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.419 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:20:14.419 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:14.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.419 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.419 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.419 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:20:14.419 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:14.419 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.419 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.420 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:14.420 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.420 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.420 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:20:14.420 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:14.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.420 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.420 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.420 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:20:14.420 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:14.420 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.420 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.421 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:14.421 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.421 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.421 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:20:14.421 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:14.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.421 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.421 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.421 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:20:14.421 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:14.421 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.422 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.422 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:14.422 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.422 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.422 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.422 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:20:14.422 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:14.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.422 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.422 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.422 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:20:14.422 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:14.423 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.423 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.423 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:20:14.423 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:20:14.423 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:20:14.423 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:20:14.423 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:14.423 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:14.423 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:14.423 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.423 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.423 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:14.424 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:20:14.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.424 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:20:14.424 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:14.424 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:14.424 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:14.424 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:14.424 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:14.424 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:20:14.424 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:14.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.424 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:20:14.425 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:14.425 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:14.425 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:14.425 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:14.425 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:14.425 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:20:14.425 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:14.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.425 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:20:14.425 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:14.425 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:14.425 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:14.426 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:14.426 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:14.426 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:20:14.426 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:14.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.426 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:20:14.426 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:14.426 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:14.426 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:14.426 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:14.426 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:14.426 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:20:14.427 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:14.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.427 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:20:14.427 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:14.427 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:14.427 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:14.427 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:14.427 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:14.427 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:20:14.427 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:14.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.427 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:20:14.428 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:14.429 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:14.429 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:14.429 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:14.429 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:14.429 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:20:14.429 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:14.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.430 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.430 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:20:14.430 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:14.430 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:14.430 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:14.430 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:14.430 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:14.430 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:20:14.430 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:14.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.430 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.430 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:20:14.431 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:14.431 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:14.431 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:14.431 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:14.431 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:14.431 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:20:14.431 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:14.431 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.431 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:14.432 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:20:14.432 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [ 2024-05-22T21:20:14.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.432 INFO:teuthology.orchestra.run.smithi161.stdout: "pgid": "17.e", 2024-05-22T21:20:14.432 INFO:teuthology.orchestra.run.smithi161.stdout: "mappings": [ 2024-05-22T21:20:14.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.432 INFO:teuthology.orchestra.run.smithi161.stdout: "from": 0, 2024-05-22T21:20:14.432 INFO:teuthology.orchestra.run.smithi161.stdout: "to": 5 2024-05-22T21:20:14.432 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.432 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.432 INFO:teuthology.orchestra.run.smithi161.stdout: "pgid": "17.30", 2024-05-22T21:20:14.433 INFO:teuthology.orchestra.run.smithi161.stdout: "mappings": [ 2024-05-22T21:20:14.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.433 INFO:teuthology.orchestra.run.smithi161.stdout: "from": 0, 2024-05-22T21:20:14.433 INFO:teuthology.orchestra.run.smithi161.stdout: "to": 5 2024-05-22T21:20:14.433 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.433 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.433 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.433 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:14.433 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:20:14.433 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:20:14.433 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:20:14.433 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:20:14.433 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:20:14.433 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:20:14.434 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:20:14.434 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:20:14.434 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:20:14.434 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:20:14.434 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:20:14.434 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:20:14.434 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.434 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:20:14.434 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:20:14.434 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:20:14.434 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:20:14.434 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:20:14.435 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:20:14.435 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:20:14.435 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:20:14.435 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:20:14.435 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:20:14.435 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:20:14.435 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.435 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:20:14.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.350+0000 7f5e6e600640 1 -- 172.21.15.161:0/1532681512 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5e44066690 msgr2=0x7f5e44068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.350+0000 7f5e6e600640 1 --2- 172.21.15.161:0/1532681512 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5e44066690 0x7f5e44068b50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5e580ad000 tx=0x7f5e580a6000 comp rx=0 tx=0).stop 2024-05-22T21:20:14.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.350+0000 7f5e6e600640 1 -- 172.21.15.161:0/1532681512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e680d0ba0 msgr2=0x7f5e680adf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.350+0000 7f5e6e600640 1 --2- 172.21.15.161:0/1532681512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e680d0ba0 0x7f5e680adf90 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000caa0 tx=0x7f5e6000cf60 comp rx=0 tx=0).stop 2024-05-22T21:20:14.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.350+0000 7f5e6e600640 1 -- 172.21.15.161:0/1532681512 shutdown_connections 2024-05-22T21:20:14.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.350+0000 7f5e6e600640 1 --2- 172.21.15.161:0/1532681512 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5e44066690 0x7f5e44068b50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.350+0000 7f5e6e600640 1 --2- 172.21.15.161:0/1532681512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e680d0ba0 0x7f5e680adf90 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.350+0000 7f5e6e600640 1 --2- 172.21.15.161:0/1532681512 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e680cbfa0 0x7f5e680b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.350+0000 7f5e6e600640 1 --2- 172.21.15.161:0/1532681512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e6806cff0 0x7f5e680b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.350+0000 7f5e6e600640 1 -- 172.21.15.161:0/1532681512 >> 172.21.15.161:0/1532681512 conn(0x7f5e680d00b0 msgr2=0x7f5e680cd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:14.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.350+0000 7f5e6e600640 1 -- 172.21.15.161:0/1532681512 shutdown_connections 2024-05-22T21:20:14.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.350+0000 7f5e6e600640 1 -- 172.21.15.161:0/1532681512 wait complete. 2024-05-22T21:20:14.439 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-05-22T21:20:14.439 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:20:14.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.570+0000 7f97dfe00640 1 Processor -- start 2024-05-22T21:20:14.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.570+0000 7f97dfe00640 1 -- start start 2024-05-22T21:20:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.574+0000 7f97dfe00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f97d815dd50 0x7f97d806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.574+0000 7f97dfe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f97d815e130 0x7f97d815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.574+0000 7f97dfe00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d8071670 con 0x7f97d815d410 2024-05-22T21:20:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.574+0000 7f97dfe00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d815d8a0 con 0x7f97d815e130 2024-05-22T21:20:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.574+0000 7f97dfe00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d806de40 con 0x7f97d815dd50 2024-05-22T21:20:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.574+0000 7f97dca00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f97d815e130 0x7f97d815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.574+0000 7f97dca00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f97d815e130 0x7f97d815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53032/0 (socket says 172.21.15.161:53032) 2024-05-22T21:20:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.574+0000 7f97dca00640 1 -- 172.21.15.161:0/3487700700 learned_addr learned my addr 172.21.15.161:0/3487700700 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.574+0000 7f97dd400640 1 --2- 172.21.15.161:0/3487700700 >> v2:172.21.15.179:3300/0 conn(0x7f97d815dd50 0x7f97d806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.574+0000 7f97dca00640 1 -- 172.21.15.161:0/3487700700 >> v2:172.21.15.179:3300/0 conn(0x7f97d815dd50 msgr2=0x7f97d806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.574+0000 7f97dca00640 1 --2- 172.21.15.161:0/3487700700 >> v2:172.21.15.179:3300/0 conn(0x7f97d815dd50 0x7f97d806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.574+0000 7f97dca00640 1 -- 172.21.15.161:0/3487700700 >> v1:172.21.15.179:6789/0 conn(0x7f97d815d410 legacy=0x7f97d815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.574+0000 7f97dca00640 1 -- 172.21.15.161:0/3487700700 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d8070a60 con 0x7f97d815e130 2024-05-22T21:20:14.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.578+0000 7f97dca00640 1 --2- 172.21.15.161:0/3487700700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d815e130 0x7f97d815ced0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f97d8072330 tx=0x7f97c401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef12f9ca51529b67 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:14.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.578+0000 7f97d7e00640 1 -- 172.21.15.161:0/3487700700 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c40aa650 con 0x7f97d815e130 2024-05-22T21:20:14.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.578+0000 7f97d7e00640 1 -- 172.21.15.161:0/3487700700 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c40aa7b0 con 0x7f97d815e130 2024-05-22T21:20:14.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.578+0000 7f97d7e00640 1 -- 172.21.15.161:0/3487700700 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c40aab40 con 0x7f97d815e130 2024-05-22T21:20:14.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.578+0000 7f97dfe00640 1 -- 172.21.15.161:0/3487700700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d815e130 msgr2=0x7f97d815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.578+0000 7f97dfe00640 1 --2- 172.21.15.161:0/3487700700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d815e130 0x7f97d815ced0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f97d8072330 tx=0x7f97c401c4f0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.578+0000 7f97dfe00640 1 -- 172.21.15.161:0/3487700700 shutdown_connections 2024-05-22T21:20:14.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.578+0000 7f97dfe00640 1 --2- 172.21.15.161:0/3487700700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d815e130 0x7f97d815ced0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.578+0000 7f97dfe00640 1 --2- 172.21.15.161:0/3487700700 >> v2:172.21.15.179:3300/0 conn(0x7f97d815dd50 0x7f97d806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.578+0000 7f97dfe00640 1 -- 172.21.15.161:0/3487700700 >> 172.21.15.161:0/3487700700 conn(0x7f97d80b0bc0 msgr2=0x7f97d80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:14.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.578+0000 7f97dfe00640 1 -- 172.21.15.161:0/3487700700 shutdown_connections 2024-05-22T21:20:14.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.578+0000 7f97dfe00640 1 -- 172.21.15.161:0/3487700700 wait complete. 2024-05-22T21:20:14.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.578+0000 7f97dfe00640 1 Processor -- start 2024-05-22T21:20:14.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.578+0000 7f97dfe00640 1 -- start start 2024-05-22T21:20:14.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.578+0000 7f97dfe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97d815d410 0x7f97d814d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.578+0000 7f97dfe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97d815dd50 0x7f97d80cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dfe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d815e130 0x7f97d8153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dfe00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d80b3f40 con 0x7f97d815e130 2024-05-22T21:20:14.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dfe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d80b36a0 con 0x7f97d815d410 2024-05-22T21:20:14.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dfe00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d80b3980 con 0x7f97d815dd50 2024-05-22T21:20:14.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dde00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d815e130 0x7f97d8153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dde00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d815e130 0x7f97d8153c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53040/0 (socket says 172.21.15.161:53040) 2024-05-22T21:20:14.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dde00640 1 -- 172.21.15.161:0/560894957 learned_addr learned my addr 172.21.15.161:0/560894957 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:14.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dd400640 1 --2- 172.21.15.161:0/560894957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97d815d410 0x7f97d814d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dca00640 1 --2- 172.21.15.161:0/560894957 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97d815dd50 0x7f97d80cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dde00640 1 -- 172.21.15.161:0/560894957 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97d815dd50 msgr2=0x7f97d80cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dde00640 1 --2- 172.21.15.161:0/560894957 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97d815dd50 0x7f97d80cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dde00640 1 -- 172.21.15.161:0/560894957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97d815d410 msgr2=0x7f97d814d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dde00640 1 --2- 172.21.15.161:0/560894957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97d815d410 0x7f97d814d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dde00640 1 -- 172.21.15.161:0/560894957 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c40094d0 con 0x7f97d815e130 2024-05-22T21:20:14.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dde00640 1 --2- 172.21.15.161:0/560894957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d815e130 0x7f97d8153c30 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f97d000c770 tx=0x7f97d000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:14.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97d5600640 1 -- 172.21.15.161:0/560894957 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0003b80 con 0x7f97d815e130 2024-05-22T21:20:14.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97d5600640 1 -- 172.21.15.161:0/560894957 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d0003ce0 con 0x7f97d815e130 2024-05-22T21:20:14.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97d5600640 1 -- 172.21.15.161:0/560894957 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d000aa90 con 0x7f97d815e130 2024-05-22T21:20:14.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dfe00640 1 -- 172.21.15.161:0/560894957 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d80b3c60 con 0x7f97d815e130 2024-05-22T21:20:14.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dfe00640 1 -- 172.21.15.161:0/560894957 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d80b3300 con 0x7f97d815e130 2024-05-22T21:20:14.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97dfe00640 1 -- 172.21.15.161:0/560894957 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d8067220 con 0x7f97d815e130 2024-05-22T21:20:14.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.582+0000 7f97d5600640 1 -- 172.21.15.161:0/560894957 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f97d000abf0 con 0x7f97d815e130 2024-05-22T21:20:14.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.586+0000 7f97d5600640 1 --2- 172.21.15.161:0/560894957 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97b8066690 0x7f97b8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.586+0000 7f97d5600640 1 -- 172.21.15.161:0/560894957 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f97d0083ee0 con 0x7f97d815e130 2024-05-22T21:20:14.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.586+0000 7f97dd400640 1 --2- 172.21.15.161:0/560894957 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97b8066690 0x7f97b8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.586+0000 7f97dd400640 1 --2- 172.21.15.161:0/560894957 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97b8066690 0x7f97b8068b50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f97d80b4110 tx=0x7f97cc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:14.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.590+0000 7f97d5600640 1 -- 172.21.15.161:0/560894957 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f97d00512b0 con 0x7f97d815e130 2024-05-22T21:20:14.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.742+0000 7f97dfe00640 1 -- 172.21.15.161:0/560894957 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f97d8151ae0 con 0x7f97d815e130 2024-05-22T21:20:14.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.746+0000 7f97d5600640 1 -- 172.21.15.161:0/560894957 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v145) v1 ==== 81+0+33982 (secure 0 0 0) 0x7f97d00548d0 con 0x7f97d815e130 2024-05-22T21:20:14.748 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:14.748 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:20:14.748 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 145, 2024-05-22T21:20:14.748 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:20:14.748 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:20:14.748 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:20:13.272722+0000", 2024-05-22T21:20:14.749 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:20:14.749 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:20:14.749 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:20:14.749 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:20:14.749 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:20:14.749 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:20:14.749 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:20:14.749 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:20:14.749 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:20:14.749 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:14.749 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:20:14.749 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:20:14.750 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:20:14.750 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:20:14.750 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:20:14.750 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 17, 2024-05-22T21:20:14.750 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:20:14.750 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:20:14.750 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "luminous", 2024-05-22T21:20:14.750 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:20:14.750 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:20:14.750 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:20:14.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.750 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:20:14.751 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:20:14.751 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:20:14.751 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:20:14.751 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:20:14.751 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:14.751 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:14.751 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:14.751 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:14.751 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:14.751 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:14.751 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:14.751 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:14.752 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:14.752 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:14.752 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:20:14.752 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:20:14.752 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:20:14.752 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:20:14.752 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:20:14.752 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:14.752 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:14.752 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:14.752 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:14.752 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:14.752 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:14.753 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:14.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.753 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:20:14.753 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:14.753 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:14.753 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:14.753 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:14.753 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:14.753 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:14.753 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:14.753 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:14.753 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:14.754 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:20:14.754 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:14.754 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:14.754 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:14.754 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:14.754 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:14.754 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:14.754 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:14.754 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:14.754 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:14.754 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:14.754 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:14.755 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:14.755 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:14.755 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:14.755 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:14.755 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:14.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.755 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:14.755 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:14.755 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:14.755 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:14.755 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:14.755 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:14.755 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:14.755 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:14.756 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:14.756 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:14.756 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:14.756 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:14.756 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:20:14.756 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:20:14.756 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.756 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:14.756 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:20:14.756 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.756 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:14.756 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:20:14.756 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:20:14.756 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:20:14.757 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:20:14.757 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:20:14.757 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:14.757 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:14.757 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:14.757 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.757 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.757 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 16, 2024-05-22T21:20:14.757 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:20:14.757 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:19:44.080742+0000", 2024-05-22T21:20:14.757 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:20:14.757 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:20:14.757 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:14.758 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:14.758 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:14.758 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:14.758 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:14.758 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:14.758 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:14.758 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:14.758 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:14.758 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:14.758 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:20:14.758 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:20:14.758 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:20:14.758 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:20:14.759 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:20:14.759 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:14.759 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:14.759 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:14.759 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:14.759 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:14.759 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:14.759 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:14.759 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.759 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "139", 2024-05-22T21:20:14.759 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:14.759 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:14.759 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:14.760 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:14.760 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:14.760 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:14.760 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:14.760 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:14.760 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:14.760 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:20:14.760 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:14.760 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:14.760 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:14.760 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:14.760 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:14.760 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:14.760 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:14.761 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:14.761 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:14.761 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:14.761 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:14.761 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:14.761 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:14.761 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:14.761 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:14.761 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:14.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.761 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:14.761 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:14.761 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:14.761 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:14.761 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:14.762 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:14.762 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:14.762 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:14.762 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:14.762 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:14.762 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:14.762 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:14.762 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:20:14.762 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:20:14.762 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:20:14.762 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.762 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:14.762 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:20:14.763 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:20:14.763 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.763 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.763 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:14.763 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.75, 2024-05-22T21:20:14.763 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.75, 2024-05-22T21:20:14.763 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:20:14.763 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.75, 2024-05-22T21:20:14.763 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.75, 2024-05-22T21:20:14.763 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:14.763 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:14.763 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:14.763 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.763 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.763 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.764 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 17, 2024-05-22T21:20:14.764 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:20:14.764 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:19:45.388618+0000", 2024-05-22T21:20:14.764 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:20:14.764 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:20:14.764 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:20:14.764 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:20:14.764 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:20:14.764 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:20:14.764 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:20:14.764 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:20:14.764 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:20:14.764 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:20:14.765 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:20:14.765 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:20:14.765 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:20:14.765 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:20:14.765 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:20:14.765 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:20:14.765 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:20:14.765 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:20:14.765 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:20:14.765 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:20:14.765 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:20:14.765 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:20:14.765 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:20:14.765 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:20:14.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.766 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "143", 2024-05-22T21:20:14.766 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:20:14.766 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:20:14.766 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:20:14.766 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:20:14.766 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:20:14.766 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:20:14.766 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:20:14.766 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:20:14.766 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:20:14.767 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 33554432, 2024-05-22T21:20:14.767 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:20:14.767 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:20:14.767 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:20:14.767 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:20:14.767 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:20:14.767 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:20:14.767 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:20:14.767 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:20:14.767 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:20:14.767 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:20:14.767 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:20:14.767 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:20:14.768 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:20:14.768 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:20:14.768 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:20:14.768 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:20:14.768 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.768 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:20:14.768 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:20:14.768 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:20:14.768 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:20:14.768 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:20:14.768 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:20:14.768 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:20:14.768 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:20:14.769 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:20:14.769 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:20:14.769 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:20:14.769 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:20:14.769 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:20:14.769 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:20:14.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.769 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:20:14.769 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:20:14.769 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:20:14.769 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.769 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:20:14.770 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.6299999952316284, 2024-05-22T21:20:14.770 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.6299999952316284, 2024-05-22T21:20:14.770 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:20:14.770 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.6299999952316284, 2024-05-22T21:20:14.770 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.6299999952316284, 2024-05-22T21:20:14.770 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:20:14.770 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:20:14.770 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:20:14.770 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.770 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.770 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:14.770 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:20:14.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.771 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:20:14.771 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:14.771 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:14.771 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:14.771 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:14.771 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:14.771 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:14.771 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:14.771 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:14.771 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:14.771 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:14.771 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:14.772 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:14.772 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.772 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.772 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.772 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:20:14.772 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:14.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.772 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.772 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.772 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:20:14.772 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:14.772 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.772 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.773 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:14.773 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.773 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.773 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.773 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:20:14.773 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:14.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.773 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.773 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.773 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:20:14.773 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:14.773 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.773 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.774 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:14.774 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.774 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.774 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:20:14.774 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:14.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.774 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.774 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:20:14.774 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:14.774 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.774 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.775 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:14.775 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.775 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.775 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.775 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:20:14.775 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:14.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.775 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.775 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.775 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:20:14.775 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:20:14.775 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.775 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.775 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:20:14.776 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:20:14.776 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:20:14.776 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:20:14.776 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:14.776 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:14.776 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:14.776 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.776 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.776 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:20:14.776 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:14.776 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:14.776 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:14.776 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:14.777 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:14.777 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:14.777 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:14.777 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:14.777 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:14.777 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:14.777 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:14.777 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:14.777 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.777 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.777 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:20:14.777 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:14.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.778 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.778 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:20:14.778 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:14.778 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.778 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.778 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:14.778 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.778 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.778 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.778 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:20:14.778 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:14.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.779 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.779 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.779 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:20:14.779 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:14.779 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.779 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.779 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:14.779 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.779 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.779 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.779 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:20:14.779 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:14.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.780 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.780 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.780 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:20:14.780 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:14.780 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.780 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.780 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:14.780 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.780 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.780 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.780 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:20:14.781 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:14.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.781 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.781 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.781 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:20:14.781 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:20:14.781 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.781 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.781 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:20:14.781 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:20:14.781 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:20:14.781 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:20:14.781 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:14.781 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:14.782 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:14.782 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.782 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.782 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:20:14.782 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:14.782 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:14.782 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:14.782 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:14.782 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:14.782 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:14.782 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:14.782 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:14.783 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 144, 2024-05-22T21:20:14.783 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:14.783 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:14.783 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:14.783 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.783 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.783 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:20:14.783 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:14.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.783 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.783 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:20:14.783 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:14.783 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.784 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.784 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:14.784 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.784 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.784 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:20:14.784 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:14.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.784 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.784 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:20:14.784 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:14.784 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.784 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.785 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:14.785 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.785 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.785 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:20:14.785 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:14.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.785 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.785 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:20:14.785 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:14.785 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.785 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.786 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:14.786 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.786 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.786 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:20:14.786 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:14.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.786 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.786 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:20:14.786 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:20:14.786 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.786 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.787 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:20:14.787 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:20:14.787 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:20:14.787 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:20:14.787 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:14.787 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:14.787 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:14.787 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.787 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.787 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:20:14.787 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:14.787 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:14.788 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:14.788 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:14.788 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:14.788 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:14.788 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:14.788 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:14.788 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:14.788 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:14.788 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:14.788 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:14.788 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.788 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.788 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:20:14.789 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:14.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.789 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.789 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.789 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:20:14.789 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:14.789 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.789 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.789 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:14.789 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.789 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.789 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.789 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:20:14.790 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:14.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.790 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.790 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.790 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:20:14.790 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:14.790 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.790 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.790 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:14.790 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.790 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.790 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.791 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:20:14.791 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:14.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.791 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.791 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:20:14.791 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:14.791 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.791 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.791 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:14.791 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.791 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.792 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:20:14.792 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:14.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.792 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.792 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.792 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:20:14.792 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:20:14.792 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.792 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.792 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:20:14.792 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:20:14.792 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:20:14.792 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:20:14.793 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:14.793 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:14.793 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:14.793 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.793 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.793 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:20:14.793 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:14.793 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:14.793 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:14.793 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:14.793 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:14.793 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:14.793 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:14.794 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:14.794 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:14.794 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:14.794 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:14.794 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:14.794 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.794 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.794 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.794 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:20:14.794 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:14.794 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.794 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.795 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.795 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:20:14.795 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:14.795 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.795 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.795 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:14.795 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.795 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.795 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.795 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:20:14.795 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:14.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.796 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.796 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.796 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:20:14.796 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:14.796 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.796 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.796 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:14.796 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.796 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.796 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.796 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:20:14.796 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:14.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.797 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.797 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.797 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:20:14.797 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:14.797 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.797 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.797 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.797 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:14.797 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.797 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.797 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.797 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:20:14.797 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:14.797 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.797 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.798 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.798 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:20:14.798 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:20:14.798 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.798 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.798 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.798 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:20:14.798 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:20:14.798 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:20:14.798 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:20:14.798 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:14.798 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:14.799 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:14.799 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.799 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.799 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.799 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:20:14.799 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:14.799 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:14.799 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:14.799 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:14.799 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:14.799 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:14.799 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:14.799 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:14.799 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 144, 2024-05-22T21:20:14.800 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:14.800 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:14.800 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:14.800 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.800 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.800 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.800 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:20:14.800 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:14.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.800 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.800 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.800 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:20:14.800 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:14.800 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.800 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.801 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.801 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:14.801 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.801 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.801 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.801 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:20:14.801 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:14.801 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.801 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.801 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.801 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:20:14.801 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:14.801 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.801 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.802 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.802 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:14.802 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.802 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.802 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.802 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:20:14.802 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:14.802 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.802 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.802 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.802 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:20:14.802 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:14.802 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.802 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:20:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:20:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:20:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.803 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.804 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:20:14.804 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:20:14.804 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:20:14.804 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:20:14.804 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:14.804 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:14.804 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:14.804 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.804 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.804 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:20:14.804 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:14.804 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:14.804 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:14.804 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:14.805 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:14.805 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:14.805 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:14.805 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:14.805 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:14.805 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:14.805 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:14.805 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:14.805 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.805 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.805 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.805 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:20:14.805 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:14.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.805 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.806 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.806 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:20:14.806 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:14.806 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.806 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.806 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.806 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:14.806 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.806 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.806 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.806 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:20:14.806 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:14.806 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.807 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.807 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.807 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:20:14.807 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:14.807 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.807 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.807 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.807 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:14.807 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.807 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.807 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.807 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:20:14.807 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:14.807 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.808 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.808 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.808 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:20:14.808 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:14.808 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.808 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.808 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:14.808 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.808 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.808 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.808 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:20:14.808 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:14.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.809 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.809 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.809 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:20:14.809 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:20:14.809 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.809 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.809 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:20:14.809 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:20:14.809 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:20:14.809 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:20:14.809 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:14.809 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:14.809 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:14.809 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.810 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.810 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.810 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:20:14.810 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:20:14.810 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:20:14.810 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:20:14.810 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:20:14.810 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:20:14.810 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:20:14.810 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:20:14.810 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:20:14.810 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 138, 2024-05-22T21:20:14.811 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:20:14.811 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:20:14.811 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:20:14.811 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.811 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.811 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.811 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:20:14.811 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:14.811 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.811 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.811 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.811 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:20:14.811 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:14.811 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.811 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.812 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.812 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:20:14.812 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.812 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.812 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.812 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:20:14.812 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:14.812 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.812 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.812 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.812 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:20:14.812 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:14.812 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.812 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.812 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.813 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:20:14.813 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.813 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.813 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.813 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:20:14.813 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:14.813 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.813 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.813 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.813 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:20:14.813 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:14.813 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.813 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.813 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.813 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:20:14.814 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:20:14.814 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.814 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:20:14.814 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:20:14.814 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:14.814 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.814 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.814 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:20:14.814 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:20:14.814 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:20:14.814 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.814 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.814 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.814 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:20:14.815 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:20:14.815 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:20:14.815 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:20:14.815 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:20:14.815 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:20:14.815 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:20:14.815 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.815 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.815 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:14.815 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:20:14.815 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.815 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:20:14.815 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:14.815 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:14.815 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:14.816 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:14.816 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:14.816 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:20:14.816 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:14.816 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.816 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.816 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:20:14.816 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:14.816 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:14.816 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:14.816 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:14.816 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:14.816 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:20:14.816 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:14.816 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.817 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.817 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:20:14.817 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:14.817 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:14.817 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:14.817 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:14.817 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:14.817 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:20:14.817 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:14.817 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.817 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.817 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:20:14.817 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:14.817 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:14.817 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:14.818 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:14.818 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:14.818 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:20:14.818 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:14.818 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.818 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.818 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:20:14.818 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:14.818 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:14.818 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:14.818 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:14.818 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:14.818 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:20:14.819 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:14.819 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.819 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.819 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:20:14.819 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:14.819 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:14.819 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:14.819 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:14.819 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:14.819 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:20:14.819 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:14.819 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.819 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.819 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:20:14.820 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:14.820 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:14.820 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:14.820 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:14.820 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:14.820 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:20:14.820 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:14.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.820 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.820 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:20:14.820 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:20:14.820 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:20:14.820 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:20:14.821 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:20:14.821 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:20:14.821 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:20:14.821 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:20:14.821 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.821 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:14.821 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:20:14.821 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [ 2024-05-22T21:20:14.821 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.821 INFO:teuthology.orchestra.run.smithi161.stdout: "pgid": "17.e", 2024-05-22T21:20:14.821 INFO:teuthology.orchestra.run.smithi161.stdout: "mappings": [ 2024-05-22T21:20:14.821 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.821 INFO:teuthology.orchestra.run.smithi161.stdout: "from": 0, 2024-05-22T21:20:14.822 INFO:teuthology.orchestra.run.smithi161.stdout: "to": 5 2024-05-22T21:20:14.822 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.822 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.822 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.822 INFO:teuthology.orchestra.run.smithi161.stdout: "pgid": "17.30", 2024-05-22T21:20:14.822 INFO:teuthology.orchestra.run.smithi161.stdout: "mappings": [ 2024-05-22T21:20:14.822 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:14.822 INFO:teuthology.orchestra.run.smithi161.stdout: "from": 0, 2024-05-22T21:20:14.822 INFO:teuthology.orchestra.run.smithi161.stdout: "to": 5 2024-05-22T21:20:14.822 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.822 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:14.823 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.823 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:20:14.823 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:20:14.823 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:20:14.823 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:20:14.823 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:20:14.823 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:20:14.823 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:20:14.823 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:20:14.823 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:20:14.823 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:20:14.823 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:20:14.823 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:20:14.824 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:20:14.824 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:14.824 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:20:14.824 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:20:14.824 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:20:14.824 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:20:14.824 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:20:14.824 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:20:14.824 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:20:14.824 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:20:14.824 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:20:14.824 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:20:14.825 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:20:14.825 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:14.825 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:20:14.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.746+0000 7f97dfe00640 1 -- 172.21.15.161:0/560894957 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97b8066690 msgr2=0x7f97b8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.746+0000 7f97dfe00640 1 --2- 172.21.15.161:0/560894957 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97b8066690 0x7f97b8068b50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f97d80b4110 tx=0x7f97cc009190 comp rx=0 tx=0).stop 2024-05-22T21:20:14.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.746+0000 7f97dfe00640 1 -- 172.21.15.161:0/560894957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d815e130 msgr2=0x7f97d8153c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.746+0000 7f97dfe00640 1 --2- 172.21.15.161:0/560894957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d815e130 0x7f97d8153c30 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f97d000c770 tx=0x7f97d000cc30 comp rx=0 tx=0).stop 2024-05-22T21:20:14.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.746+0000 7f97dfe00640 1 -- 172.21.15.161:0/560894957 shutdown_connections 2024-05-22T21:20:14.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.746+0000 7f97dfe00640 1 --2- 172.21.15.161:0/560894957 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f97b8066690 0x7f97b8068b50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.746+0000 7f97dfe00640 1 --2- 172.21.15.161:0/560894957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97d815e130 0x7f97d8153c30 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.746+0000 7f97dfe00640 1 --2- 172.21.15.161:0/560894957 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f97d815dd50 0x7f97d80cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.746+0000 7f97dfe00640 1 --2- 172.21.15.161:0/560894957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97d815d410 0x7f97d814d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.746+0000 7f97dfe00640 1 -- 172.21.15.161:0/560894957 >> 172.21.15.161:0/560894957 conn(0x7f97d80b0bc0 msgr2=0x7f97d81598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:14.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.746+0000 7f97dfe00640 1 -- 172.21.15.161:0/560894957 shutdown_connections 2024-05-22T21:20:14.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.746+0000 7f97dfe00640 1 -- 172.21.15.161:0/560894957 wait complete. 2024-05-22T21:20:14.828 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:20:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.958+0000 7fbb54600640 1 Processor -- start 2024-05-22T21:20:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.958+0000 7fbb54600640 1 -- start start 2024-05-22T21:20:14.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.958+0000 7fbb54600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbb4c15c6d0 0x7fbb4c164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.958+0000 7fbb54600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4c071670 con 0x7fbb4c15e130 2024-05-22T21:20:14.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.958+0000 7fbb54600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4c070a60 con 0x7fbb4c15dd50 2024-05-22T21:20:14.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.958+0000 7fbb54600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb4c070d40 con 0x7fbb4c15c6d0 2024-05-22T21:20:14.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb51200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fbb4c15e130 0x7fbb4c15ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44092/0 (socket says 172.21.15.161:44092) 2024-05-22T21:20:14.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb51200640 1 -- 172.21.15.161:0/933233649 learned_addr learned my addr 172.21.15.161:0/933233649 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:14.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb52600640 1 --2- 172.21.15.161:0/933233649 >> v2:172.21.15.179:3300/0 conn(0x7fbb4c15c6d0 0x7fbb4c164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb4be00640 1 -- 172.21.15.161:0/933233649 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004899042 0 0) 0x7fbb4c071670 con 0x7fbb4c15e130 2024-05-22T21:20:14.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb4be00640 1 -- 172.21.15.161:0/933233649 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb34003530 con 0x7fbb4c15e130 2024-05-22T21:20:14.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb4be00640 1 -- 172.21.15.161:0/933233649 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1032395992 0 0) 0x7fbb34003530 con 0x7fbb4c15e130 2024-05-22T21:20:14.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb4be00640 1 -- 172.21.15.161:0/933233649 >> v2:172.21.15.179:3300/0 conn(0x7fbb4c15c6d0 msgr2=0x7fbb4c164e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:14.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb4be00640 1 --2- 172.21.15.161:0/933233649 >> v2:172.21.15.179:3300/0 conn(0x7fbb4c15c6d0 0x7fbb4c164e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb4be00640 1 -- 172.21.15.161:0/933233649 >> v1:172.21.15.179:6789/0 conn(0x7fbb4c15dd50 legacy=0x7fbb4c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb4be00640 1 -- 172.21.15.161:0/933233649 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4c071020 con 0x7fbb4c15e130 2024-05-22T21:20:14.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb4be00640 1 -- 172.21.15.161:0/933233649 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fbb38001680 con 0x7fbb4c15e130 2024-05-22T21:20:14.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb4be00640 1 -- 172.21.15.161:0/933233649 >> v1:172.21.15.161:6789/0 conn(0x7fbb4c15e130 legacy=0x7fbb4c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb4be00640 1 --2- 172.21.15.161:0/933233649 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb34003980 0x7fbb34003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb4be00640 1 --2- 172.21.15.161:0/933233649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb340047d0 0x7fbb34004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb4be00640 1 --2- 172.21.15.161:0/933233649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb34005190 0x7fbb3400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb4be00640 1 -- 172.21.15.161:0/933233649 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c070d40 con 0x7fbb34005190 2024-05-22T21:20:14.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb4be00640 1 -- 172.21.15.161:0/933233649 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3400c030 con 0x7fbb340047d0 2024-05-22T21:20:14.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb4be00640 1 -- 172.21.15.161:0/933233649 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3400c1f0 con 0x7fbb34003980 2024-05-22T21:20:14.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb52600640 1 --2- 172.21.15.161:0/933233649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb34005190 0x7fbb3400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb51c00640 1 --2- 172.21.15.161:0/933233649 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb34003980 0x7fbb34003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb51200640 1 --2- 172.21.15.161:0/933233649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb340047d0 0x7fbb34004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb52600640 1 -- 172.21.15.161:0/933233649 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb34003980 msgr2=0x7fbb34003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:14.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb52600640 1 --2- 172.21.15.161:0/933233649 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb34003980 0x7fbb34003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb52600640 1 -- 172.21.15.161:0/933233649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb340047d0 msgr2=0x7fbb34004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:14.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb52600640 1 --2- 172.21.15.161:0/933233649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb340047d0 0x7fbb34004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.962+0000 7fbb52600640 1 -- 172.21.15.161:0/933233649 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb34003530 con 0x7fbb34005190 2024-05-22T21:20:14.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.966+0000 7fbb52600640 1 -- 172.21.15.161:0/933233649 reap_dead start 2024-05-22T21:20:14.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.966+0000 7fbb52600640 1 --2- 172.21.15.161:0/933233649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb34005190 0x7fbb3400b990 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009ab0 tx=0x7fbb4401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c717530b7c404088 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:14.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.966+0000 7fbb4be00640 1 -- 172.21.15.161:0/933233649 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb440a5680 con 0x7fbb34005190 2024-05-22T21:20:14.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.966+0000 7fbb4be00640 1 -- 172.21.15.161:0/933233649 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb440a57e0 con 0x7fbb34005190 2024-05-22T21:20:14.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.966+0000 7fbb4be00640 1 -- 172.21.15.161:0/933233649 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb440a5b70 con 0x7fbb34005190 2024-05-22T21:20:14.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.966+0000 7fbb54600640 1 -- 172.21.15.161:0/933233649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb34005190 msgr2=0x7fbb3400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.966+0000 7fbb54600640 1 --2- 172.21.15.161:0/933233649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb34005190 0x7fbb3400b990 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009ab0 tx=0x7fbb4401ca90 comp rx=0 tx=0).stop 2024-05-22T21:20:14.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.966+0000 7fbb54600640 1 -- 172.21.15.161:0/933233649 shutdown_connections 2024-05-22T21:20:14.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.966+0000 7fbb54600640 1 --2- 172.21.15.161:0/933233649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb34005190 0x7fbb3400b990 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.966+0000 7fbb54600640 1 -- 172.21.15.161:0/933233649 >> 172.21.15.161:0/933233649 conn(0x7fbb4c0b0bc0 msgr2=0x7fbb4c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:14.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.966+0000 7fbb54600640 1 -- 172.21.15.161:0/933233649 shutdown_connections 2024-05-22T21:20:14.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.966+0000 7fbb54600640 1 -- 172.21.15.161:0/933233649 wait complete. 2024-05-22T21:20:14.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.966+0000 7fbb54600640 1 Processor -- start 2024-05-22T21:20:14.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.966+0000 7fbb54600640 1 -- start start 2024-05-22T21:20:14.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb54600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb34005190 0x7fbb4c14a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb54600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb4c14ae80 0x7fbb4c1ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb54600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c1eff00 0x7fbb4c157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb54600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c14a770 con 0x7fbb4c1eff00 2024-05-22T21:20:14.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb54600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c154a40 con 0x7fbb4c14ae80 2024-05-22T21:20:14.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb54600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c154480 con 0x7fbb34005190 2024-05-22T21:20:14.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb52600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c1eff00 0x7fbb4c157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb52600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c1eff00 0x7fbb4c157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:53050/0 (socket says 172.21.15.161:53050) 2024-05-22T21:20:14.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb52600640 1 -- 172.21.15.161:0/3274227319 learned_addr learned my addr 172.21.15.161:0/3274227319 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:14.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb51200640 1 --2- 172.21.15.161:0/3274227319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb4c14ae80 0x7fbb4c1ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb51c00640 1 --2- 172.21.15.161:0/3274227319 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb34005190 0x7fbb4c14a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb52600640 1 -- 172.21.15.161:0/3274227319 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb34005190 msgr2=0x7fbb4c14a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb52600640 1 --2- 172.21.15.161:0/3274227319 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb34005190 0x7fbb4c14a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb52600640 1 -- 172.21.15.161:0/3274227319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb4c14ae80 msgr2=0x7fbb4c1ef9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:14.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb52600640 1 --2- 172.21.15.161:0/3274227319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb4c14ae80 0x7fbb4c1ef9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:14.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb52600640 1 -- 172.21.15.161:0/3274227319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4c154760 con 0x7fbb4c1eff00 2024-05-22T21:20:14.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb52600640 1 --2- 172.21.15.161:0/3274227319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c1eff00 0x7fbb4c157d60 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fbb4401d330 tx=0x7fbb34005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:14.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb49600640 1 -- 172.21.15.161:0/3274227319 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb440a5680 con 0x7fbb4c1eff00 2024-05-22T21:20:14.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb49600640 1 -- 172.21.15.161:0/3274227319 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb440ad850 con 0x7fbb4c1eff00 2024-05-22T21:20:14.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb49600640 1 -- 172.21.15.161:0/3274227319 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb440adbe0 con 0x7fbb4c1eff00 2024-05-22T21:20:14.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb54600640 1 -- 172.21.15.161:0/3274227319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4c1ef1c0 con 0x7fbb4c1eff00 2024-05-22T21:20:14.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb54600640 1 -- 172.21.15.161:0/3274227319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4c152a20 con 0x7fbb4c1eff00 2024-05-22T21:20:14.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb54600640 1 -- 172.21.15.161:0/3274227319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4c155e40 con 0x7fbb4c1eff00 2024-05-22T21:20:14.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.970+0000 7fbb49600640 1 -- 172.21.15.161:0/3274227319 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbb440add40 con 0x7fbb4c1eff00 2024-05-22T21:20:14.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.974+0000 7fbb49600640 1 --2- 172.21.15.161:0/3274227319 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb2c066690 0x7fbb2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:14.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.974+0000 7fbb49600640 1 -- 172.21.15.161:0/3274227319 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbb4411d720 con 0x7fbb4c1eff00 2024-05-22T21:20:14.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.974+0000 7fbb51c00640 1 --2- 172.21.15.161:0/3274227319 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb2c066690 0x7fbb2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:14.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.974+0000 7fbb51c00640 1 --2- 172.21.15.161:0/3274227319 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb2c066690 0x7fbb2c068b50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbb40001480 tx=0x7fbb40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:14.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:14.978+0000 7fbb49600640 1 -- 172.21.15.161:0/3274227319 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbb440eabc0 con 0x7fbb4c1eff00 2024-05-22T21:20:15.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:15.162+0000 7fbb54600640 1 -- 172.21.15.161:0/3274227319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb4c157070 con 0x7fbb4c1eff00 2024-05-22T21:20:15.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:15.162+0000 7fbb49600640 1 -- 172.21.15.161:0/3274227319 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4745 (secure 0 0 0) 0x7fbb440bd020 con 0x7fbb4c1eff00 2024-05-22T21:20:15.166 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:15.166 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/524372753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":524372753},{"type":"v1","addr":"172.21.15.161:6836","nonce":524372753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3762119379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3762119379},{"type":"v1","addr":"172.21.15.179:6833","nonce":3762119379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3087241366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3087241366},{"type":"v1","addr":"172.21.15.179:6835","nonce":3087241366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:46.384984+0000","modified":"2024-05-22T21:19:49.400019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7572":{"gid":7572,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/3945715002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3945715002},{"type":"v1","addr":"172.21.15.161:6837","nonce":3945715002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-22T21:20:15.166 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 62 2024-05-22T21:20:15.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:15.162+0000 7fbb54600640 1 -- 172.21.15.161:0/3274227319 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb2c066690 msgr2=0x7fbb2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:15.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:15.162+0000 7fbb54600640 1 --2- 172.21.15.161:0/3274227319 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb2c066690 0x7fbb2c068b50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbb40001480 tx=0x7fbb40009040 comp rx=0 tx=0).stop 2024-05-22T21:20:15.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:15.162+0000 7fbb54600640 1 -- 172.21.15.161:0/3274227319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c1eff00 msgr2=0x7fbb4c157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:15.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:15.162+0000 7fbb54600640 1 --2- 172.21.15.161:0/3274227319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c1eff00 0x7fbb4c157d60 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fbb4401d330 tx=0x7fbb34005800 comp rx=0 tx=0).stop 2024-05-22T21:20:15.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:15.166+0000 7fbb54600640 1 -- 172.21.15.161:0/3274227319 shutdown_connections 2024-05-22T21:20:15.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:15.166+0000 7fbb54600640 1 --2- 172.21.15.161:0/3274227319 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbb2c066690 0x7fbb2c068b50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:15.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:15.166+0000 7fbb54600640 1 --2- 172.21.15.161:0/3274227319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb4c1eff00 0x7fbb4c157d60 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:15.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:15.166+0000 7fbb54600640 1 --2- 172.21.15.161:0/3274227319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb4c14ae80 0x7fbb4c1ef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:15.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:15.166+0000 7fbb54600640 1 --2- 172.21.15.161:0/3274227319 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbb34005190 0x7fbb4c14a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:15.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:15.166+0000 7fbb54600640 1 -- 172.21.15.161:0/3274227319 >> 172.21.15.161:0/3274227319 conn(0x7fbb4c0b0bc0 msgr2=0x7fbb4c0aed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:15.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:15.166+0000 7fbb54600640 1 -- 172.21.15.161:0/3274227319 shutdown_connections 2024-05-22T21:20:15.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:15.166+0000 7fbb54600640 1 -- 172.21.15.161:0/3274227319 wait complete. 2024-05-22T21:20:15.187 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-05-22T21:20:15.342 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":7572,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":28.941183432999999,"mdsmap_epoch":62,"osdmap_epoch":145,"osdmap_epoch_barrier":142,"uptime":34.191324010999999} 2024-05-22T21:20:15.342 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 7572, "mdsmap_epoch": 62, "osdmap_epoch": 145, "osdmap_epoch_barrier": 142, "rank_uptime": 28.941183433, "state": "up:active", "uptime": 34.191324011, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:20:15.343 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-22T21:20:15.343 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:20:15.343 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:20:15.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:15.352 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-05-22T21:20:15.841 INFO:teuthology.orchestra.run.smithi161.stderr:50+0 records in 2024-05-22T21:20:15.842 INFO:teuthology.orchestra.run.smithi161.stderr:50+0 records out 2024-05-22T21:20:15.842 INFO:teuthology.orchestra.run.smithi161.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.437335 s, 120 MB/s 2024-05-22T21:20:15.843 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:20:15.843 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:20:15.942 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:20:16.027 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:20:16.027 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:20:16.027 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:20:16.035 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:20:16.035 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:20:16.087 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:20:16.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.262+0000 7f8148800640 1 Processor -- start 2024-05-22T21:20:16.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.262+0000 7f8148800640 1 -- start start 2024-05-22T21:20:16.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.262+0000 7f8148800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f814015c6d0 0x7f8140164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.262+0000 7f8148800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8140071670 con 0x7f814015dd50 2024-05-22T21:20:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.262+0000 7f8148800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8140070a60 con 0x7f814015e130 2024-05-22T21:20:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.262+0000 7f8148800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8140070d40 con 0x7f814015c6d0 2024-05-22T21:20:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.262+0000 7f8145e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f814015dd50 0x7f814006d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51192/0 (socket says 172.21.15.161:51192) 2024-05-22T21:20:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.262+0000 7f8145e00640 1 -- 172.21.15.161:0/1754779801 learned_addr learned my addr 172.21.15.161:0/1754779801 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.262+0000 7f8146800640 1 --2- 172.21.15.161:0/1754779801 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f814015c6d0 0x7f8140164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.262+0000 7f8144a00640 1 -- 172.21.15.161:0/1754779801 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883460480 0 0) 0x7f8140071670 con 0x7f814015dd50 2024-05-22T21:20:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.266+0000 7f8144a00640 1 -- 172.21.15.161:0/1754779801 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8128003530 con 0x7f814015dd50 2024-05-22T21:20:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.266+0000 7f8144a00640 1 -- 172.21.15.161:0/1754779801 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097382308 0 0) 0x7f8140070a60 con 0x7f814015e130 2024-05-22T21:20:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.266+0000 7f8144a00640 1 -- 172.21.15.161:0/1754779801 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8140071670 con 0x7f814015e130 2024-05-22T21:20:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.266+0000 7f8146800640 1 -- 172.21.15.161:0/1754779801 >> v1:172.21.15.179:6789/0 conn(0x7f814015e130 legacy=0x7f814015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.266+0000 7f8146800640 1 -- 172.21.15.161:0/1754779801 >> v1:172.21.15.161:6789/0 conn(0x7f814015dd50 legacy=0x7f814006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.266+0000 7f8146800640 1 -- 172.21.15.161:0/1754779801 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8140071020 con 0x7f814015c6d0 2024-05-22T21:20:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.266+0000 7f8146800640 1 --2- 172.21.15.161:0/1754779801 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f814015c6d0 0x7f8140164e80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f813c009dc0 tx=0x7f813c01c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5e38d15e4fc86a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.266+0000 7f8144a00640 1 -- 172.21.15.161:0/1754779801 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c0a59e0 con 0x7f814015c6d0 2024-05-22T21:20:16.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.266+0000 7f8144a00640 1 -- 172.21.15.161:0/1754779801 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c0a5b40 con 0x7f814015c6d0 2024-05-22T21:20:16.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.266+0000 7f8144a00640 1 -- 172.21.15.161:0/1754779801 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c0ad440 con 0x7f814015c6d0 2024-05-22T21:20:16.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.266+0000 7f8148800640 1 -- 172.21.15.161:0/1754779801 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f814015c6d0 msgr2=0x7f8140164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8148800640 1 --2- 172.21.15.161:0/1754779801 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f814015c6d0 0x7f8140164e80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f813c009dc0 tx=0x7f813c01c960 comp rx=0 tx=0).stop 2024-05-22T21:20:16.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8148800640 1 -- 172.21.15.161:0/1754779801 shutdown_connections 2024-05-22T21:20:16.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8148800640 1 --2- 172.21.15.161:0/1754779801 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f814015c6d0 0x7f8140164e80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8148800640 1 -- 172.21.15.161:0/1754779801 >> 172.21.15.161:0/1754779801 conn(0x7f81400b0bc0 msgr2=0x7f81400b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:16.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8148800640 1 -- 172.21.15.161:0/1754779801 shutdown_connections 2024-05-22T21:20:16.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8148800640 1 -- 172.21.15.161:0/1754779801 wait complete. 2024-05-22T21:20:16.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8148800640 1 Processor -- start 2024-05-22T21:20:16.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8148800640 1 -- start start 2024-05-22T21:20:16.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8148800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f814015c6d0 0x7f81400b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8148800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f814015dd50 0x7f81400b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8148800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f814015e130 0x7f8140151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8148800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81400b32b0 con 0x7f814015c6d0 2024-05-22T21:20:16.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8148800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81400b4160 con 0x7f814015e130 2024-05-22T21:20:16.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8148800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81400b26a0 con 0x7f814015dd50 2024-05-22T21:20:16.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8145e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f814015c6d0 0x7f81400b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8145e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f814015c6d0 0x7f81400b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45068/0 (socket says 172.21.15.161:45068) 2024-05-22T21:20:16.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8145e00640 1 -- 172.21.15.161:0/2872188046 learned_addr learned my addr 172.21.15.161:0/2872188046 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:16.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8145400640 1 --2- 172.21.15.161:0/2872188046 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f814015dd50 0x7f81400b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8146800640 1 --2- 172.21.15.161:0/2872188046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f814015e130 0x7f8140151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8145e00640 1 -- 172.21.15.161:0/2872188046 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f814015dd50 msgr2=0x7f81400b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8145e00640 1 --2- 172.21.15.161:0/2872188046 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f814015dd50 0x7f81400b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8145e00640 1 -- 172.21.15.161:0/2872188046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f814015e130 msgr2=0x7f8140151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8145e00640 1 --2- 172.21.15.161:0/2872188046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f814015e130 0x7f8140151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8145e00640 1 -- 172.21.15.161:0/2872188046 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c009a70 con 0x7f814015c6d0 2024-05-22T21:20:16.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8146800640 1 --2- 172.21.15.161:0/2872188046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f814015e130 0x7f8140151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:16.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.270+0000 7f8145e00640 1 --2- 172.21.15.161:0/2872188046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f814015c6d0 0x7f81400b3c20 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f813400c420 tx=0x7f813400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:16.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.274+0000 7f813a000640 1 -- 172.21.15.161:0/2872188046 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8134005cf0 con 0x7f814015c6d0 2024-05-22T21:20:16.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.274+0000 7f813a000640 1 -- 172.21.15.161:0/2872188046 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8134005e50 con 0x7f814015c6d0 2024-05-22T21:20:16.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.274+0000 7f813a000640 1 -- 172.21.15.161:0/2872188046 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8134007310 con 0x7f814015c6d0 2024-05-22T21:20:16.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.274+0000 7f8148800640 1 -- 172.21.15.161:0/2872188046 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81400b2980 con 0x7f814015c6d0 2024-05-22T21:20:16.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.274+0000 7f8148800640 1 -- 172.21.15.161:0/2872188046 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81400b2f40 con 0x7f814015c6d0 2024-05-22T21:20:16.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.274+0000 7f8148800640 1 -- 172.21.15.161:0/2872188046 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8140067220 con 0x7f814015c6d0 2024-05-22T21:20:16.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.274+0000 7f813a000640 1 -- 172.21.15.161:0/2872188046 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8134007470 con 0x7f814015c6d0 2024-05-22T21:20:16.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.274+0000 7f813a000640 1 --2- 172.21.15.161:0/2872188046 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8124066690 0x7f8124068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.274+0000 7f813a000640 1 -- 172.21.15.161:0/2872188046 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f81340836a0 con 0x7f814015c6d0 2024-05-22T21:20:16.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.274+0000 7f8145400640 1 --2- 172.21.15.161:0/2872188046 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8124066690 0x7f8124068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.274+0000 7f8145400640 1 --2- 172.21.15.161:0/2872188046 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8124066690 0x7f8124068b50 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f81401f99b0 tx=0x7f8130009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:16.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.278+0000 7f813a000640 1 -- 172.21.15.161:0/2872188046 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8134050b10 con 0x7f814015c6d0 2024-05-22T21:20:16.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.434+0000 7f8148800640 1 -- 172.21.15.161:0/2872188046 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f814015b020 con 0x7f814015c6d0 2024-05-22T21:20:16.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.434+0000 7f813a000640 1 -- 172.21.15.161:0/2872188046 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+16268 (secure 0 0 0) 0x7f8134054130 con 0x7f814015c6d0 2024-05-22T21:20:16.438 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:16.438 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":145,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:20:13.272722+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:44.080742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:45.388618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.30","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:20:16.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.438+0000 7f8148800640 1 -- 172.21.15.161:0/2872188046 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8124066690 msgr2=0x7f8124068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.438+0000 7f8148800640 1 --2- 172.21.15.161:0/2872188046 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8124066690 0x7f8124068b50 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f81401f99b0 tx=0x7f8130009040 comp rx=0 tx=0).stop 2024-05-22T21:20:16.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.438+0000 7f8148800640 1 -- 172.21.15.161:0/2872188046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f814015c6d0 msgr2=0x7f81400b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.438+0000 7f8148800640 1 --2- 172.21.15.161:0/2872188046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f814015c6d0 0x7f81400b3c20 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f813400c420 tx=0x7f813400c8e0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.438+0000 7f8148800640 1 -- 172.21.15.161:0/2872188046 shutdown_connections 2024-05-22T21:20:16.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.438+0000 7f8148800640 1 --2- 172.21.15.161:0/2872188046 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8124066690 0x7f8124068b50 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.438+0000 7f8148800640 1 --2- 172.21.15.161:0/2872188046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f814015e130 0x7f8140151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.438+0000 7f8148800640 1 --2- 172.21.15.161:0/2872188046 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f814015dd50 0x7f81400b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.438+0000 7f8148800640 1 --2- 172.21.15.161:0/2872188046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f814015c6d0 0x7f81400b3c20 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.438+0000 7f8148800640 1 -- 172.21.15.161:0/2872188046 >> 172.21.15.161:0/2872188046 conn(0x7f81400b0bc0 msgr2=0x7f81400af320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:16.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.438+0000 7f8148800640 1 -- 172.21.15.161:0/2872188046 shutdown_connections 2024-05-22T21:20:16.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.438+0000 7f8148800640 1 -- 172.21.15.161:0/2872188046 wait complete. 2024-05-22T21:20:16.460 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:19:44.080742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:19:45.388618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:20:16.460 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:20:16.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.594+0000 7f43dac00640 1 Processor -- start 2024-05-22T21:20:16.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.594+0000 7f43dac00640 1 -- start start 2024-05-22T21:20:16.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.598+0000 7f43dac00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f43d413cfc0 0x7f43d41402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.598+0000 7f43dac00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f43d413fe50 0x7f43d406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.598+0000 7f43dac00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d413d3a0 con 0x7f43d415af60 2024-05-22T21:20:16.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.598+0000 7f43dac00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d40cd920 con 0x7f43d413cfc0 2024-05-22T21:20:16.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.598+0000 7f43dac00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d40cdc00 con 0x7f43d413fe50 2024-05-22T21:20:16.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.598+0000 7f43d3e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f43d413cfc0 0x7f43d41402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.598+0000 7f43d3e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f43d413cfc0 0x7f43d41402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45070/0 (socket says 172.21.15.161:45070) 2024-05-22T21:20:16.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.598+0000 7f43d3e00640 1 -- 172.21.15.161:0/2125967049 learned_addr learned my addr 172.21.15.161:0/2125967049 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:16.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.598+0000 7f43d3e00640 1 -- 172.21.15.161:0/2125967049 >> v2:172.21.15.179:3300/0 conn(0x7f43d413fe50 msgr2=0x7f43d406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:16.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.598+0000 7f43d8c00640 1 --2- 172.21.15.161:0/2125967049 >> v2:172.21.15.179:3300/0 conn(0x7f43d413fe50 0x7f43d406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.598+0000 7f43d3e00640 1 --2- 172.21.15.161:0/2125967049 >> v2:172.21.15.179:3300/0 conn(0x7f43d413fe50 0x7f43d406fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.598+0000 7f43d3e00640 1 -- 172.21.15.161:0/2125967049 >> v1:172.21.15.179:6789/0 conn(0x7f43d415af60 legacy=0x7f43d413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:16.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.598+0000 7f43d3e00640 1 -- 172.21.15.161:0/2125967049 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d40cdee0 con 0x7f43d413cfc0 2024-05-22T21:20:16.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.598+0000 7f43d8c00640 1 --2- 172.21.15.161:0/2125967049 >> v2:172.21.15.179:3300/0 conn(0x7f43d413fe50 0x7f43d406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:16.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.602+0000 7f43d3e00640 1 --2- 172.21.15.161:0/2125967049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43d413cfc0 0x7f43d41402e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f43d406c5e0 tx=0x7f43c401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a4e5ae593306c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:16.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.602+0000 7f43d2a00640 1 -- 172.21.15.161:0/2125967049 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c40a8450 con 0x7f43d413cfc0 2024-05-22T21:20:16.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.602+0000 7f43d2a00640 1 -- 172.21.15.161:0/2125967049 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c40a85b0 con 0x7f43d413cfc0 2024-05-22T21:20:16.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.602+0000 7f43d2a00640 1 -- 172.21.15.161:0/2125967049 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c40a8940 con 0x7f43d413cfc0 2024-05-22T21:20:16.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.602+0000 7f43dac00640 1 -- 172.21.15.161:0/2125967049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43d413cfc0 msgr2=0x7f43d41402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.602+0000 7f43dac00640 1 --2- 172.21.15.161:0/2125967049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43d413cfc0 0x7f43d41402e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f43d406c5e0 tx=0x7f43c401c8f0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.602+0000 7f43dac00640 1 -- 172.21.15.161:0/2125967049 shutdown_connections 2024-05-22T21:20:16.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.602+0000 7f43dac00640 1 --2- 172.21.15.161:0/2125967049 >> v2:172.21.15.179:3300/0 conn(0x7f43d413fe50 0x7f43d406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.602+0000 7f43dac00640 1 --2- 172.21.15.161:0/2125967049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43d413cfc0 0x7f43d41402e0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.602+0000 7f43dac00640 1 -- 172.21.15.161:0/2125967049 >> 172.21.15.161:0/2125967049 conn(0x7f43d406a2d0 msgr2=0x7f43d406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:16.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.602+0000 7f43dac00640 1 -- 172.21.15.161:0/2125967049 shutdown_connections 2024-05-22T21:20:16.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.602+0000 7f43dac00640 1 -- 172.21.15.161:0/2125967049 wait complete. 2024-05-22T21:20:16.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.602+0000 7f43dac00640 1 Processor -- start 2024-05-22T21:20:16.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.602+0000 7f43dac00640 1 -- start start 2024-05-22T21:20:16.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43dac00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43d413cfc0 0x7f43d4141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43dac00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43d413fe50 0x7f43d4157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43dac00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43d415af60 0x7f43d413f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43dac00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d413fb40 con 0x7f43d413cfc0 2024-05-22T21:20:16.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43dac00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d415b630 con 0x7f43d413fe50 2024-05-22T21:20:16.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43dac00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d415b910 con 0x7f43d415af60 2024-05-22T21:20:16.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43d3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43d413cfc0 0x7f43d4141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43d3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43d413cfc0 0x7f43d4141bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45074/0 (socket says 172.21.15.161:45074) 2024-05-22T21:20:16.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43d3e00640 1 -- 172.21.15.161:0/3273749388 learned_addr learned my addr 172.21.15.161:0/3273749388 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:16.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43d3400640 1 --2- 172.21.15.161:0/3273749388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43d413fe50 0x7f43d4157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43d8c00640 1 --2- 172.21.15.161:0/3273749388 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43d415af60 0x7f43d413f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43d3e00640 1 -- 172.21.15.161:0/3273749388 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43d415af60 msgr2=0x7f43d413f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43d3e00640 1 --2- 172.21.15.161:0/3273749388 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43d415af60 0x7f43d413f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43d3e00640 1 -- 172.21.15.161:0/3273749388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43d413fe50 msgr2=0x7f43d4157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43d3e00640 1 --2- 172.21.15.161:0/3273749388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43d413fe50 0x7f43d4157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43d3e00640 1 -- 172.21.15.161:0/3273749388 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c4009b30 con 0x7f43d413cfc0 2024-05-22T21:20:16.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43d3400640 1 --2- 172.21.15.161:0/3273749388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43d413fe50 0x7f43d4157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:16.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43d3e00640 1 --2- 172.21.15.161:0/3273749388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43d413cfc0 0x7f43d4141bd0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f43c401afa0 tx=0x7f43c4019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:16.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43cbe00640 1 -- 172.21.15.161:0/3273749388 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c40b6070 con 0x7f43d413cfc0 2024-05-22T21:20:16.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43cbe00640 1 -- 172.21.15.161:0/3273749388 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c4009510 con 0x7f43d413cfc0 2024-05-22T21:20:16.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43dac00640 1 -- 172.21.15.161:0/3273749388 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d415bbf0 con 0x7f43d413cfc0 2024-05-22T21:20:16.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43cbe00640 1 -- 172.21.15.161:0/3273749388 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c40b1450 con 0x7f43d413cfc0 2024-05-22T21:20:16.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43dac00640 1 -- 172.21.15.161:0/3273749388 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d413b810 con 0x7f43d413cfc0 2024-05-22T21:20:16.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43dac00640 1 -- 172.21.15.161:0/3273749388 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d4067220 con 0x7f43d413cfc0 2024-05-22T21:20:16.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.606+0000 7f43cbe00640 1 -- 172.21.15.161:0/3273749388 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f43c40ba070 con 0x7f43d413cfc0 2024-05-22T21:20:16.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.610+0000 7f43cbe00640 1 --2- 172.21.15.161:0/3273749388 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f43b40666c0 0x7f43b4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.610+0000 7f43cbe00640 1 -- 172.21.15.161:0/3273749388 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f43c411e870 con 0x7f43d413cfc0 2024-05-22T21:20:16.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.610+0000 7f43d3400640 1 --2- 172.21.15.161:0/3273749388 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f43b40666c0 0x7f43b4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.610+0000 7f43d3400640 1 --2- 172.21.15.161:0/3273749388 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f43b40666c0 0x7f43b4068b80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f43d4141ae0 tx=0x7f43c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:16.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.614+0000 7f43cbe00640 1 -- 172.21.15.161:0/3273749388 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f43c40ebcb0 con 0x7f43d413cfc0 2024-05-22T21:20:16.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.766+0000 7f43dac00640 1 -- 172.21.15.161:0/3273749388 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43d415c0a0 con 0x7f43d413cfc0 2024-05-22T21:20:16.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.766+0000 7f43cbe00640 1 -- 172.21.15.161:0/3273749388 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43c40ef2d0 con 0x7f43d413cfc0 2024-05-22T21:20:16.769 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:20:16.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.766+0000 7f43dac00640 1 -- 172.21.15.161:0/3273749388 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f43b40666c0 msgr2=0x7f43b4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.766+0000 7f43dac00640 1 --2- 172.21.15.161:0/3273749388 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f43b40666c0 0x7f43b4068b80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f43d4141ae0 tx=0x7f43c0008040 comp rx=0 tx=0).stop 2024-05-22T21:20:16.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.766+0000 7f43dac00640 1 -- 172.21.15.161:0/3273749388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43d413cfc0 msgr2=0x7f43d4141bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.766+0000 7f43dac00640 1 --2- 172.21.15.161:0/3273749388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43d413cfc0 0x7f43d4141bd0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f43c401afa0 tx=0x7f43c4019690 comp rx=0 tx=0).stop 2024-05-22T21:20:16.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.766+0000 7f43dac00640 1 -- 172.21.15.161:0/3273749388 shutdown_connections 2024-05-22T21:20:16.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.766+0000 7f43dac00640 1 --2- 172.21.15.161:0/3273749388 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f43b40666c0 0x7f43b4068b80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.766+0000 7f43dac00640 1 --2- 172.21.15.161:0/3273749388 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43d415af60 0x7f43d413f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.766+0000 7f43dac00640 1 --2- 172.21.15.161:0/3273749388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43d413fe50 0x7f43d4157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.766+0000 7f43dac00640 1 --2- 172.21.15.161:0/3273749388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43d413cfc0 0x7f43d4141bd0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.766+0000 7f43dac00640 1 -- 172.21.15.161:0/3273749388 >> 172.21.15.161:0/3273749388 conn(0x7f43d406a2d0 msgr2=0x7f43d406a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:16.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.766+0000 7f43dac00640 1 -- 172.21.15.161:0/3273749388 shutdown_connections 2024-05-22T21:20:16.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.766+0000 7f43dac00640 1 -- 172.21.15.161:0/3273749388 wait complete. 2024-05-22T21:20:16.790 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:20:16.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e59200640 1 Processor -- start 2024-05-22T21:20:16.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e59200640 1 -- start start 2024-05-22T21:20:16.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e59200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3e5415b600 0x7f3e5406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e59200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5406cc40 con 0x7f3e54143c40 2024-05-22T21:20:16.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e59200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e54144bb0 con 0x7f3e5406c7e0 2024-05-22T21:20:16.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e59200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e5415ba60 con 0x7f3e5415b600 2024-05-22T21:20:16.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e51e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3e54143c40 0x7f3e541444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51196/0 (socket says 172.21.15.161:51196) 2024-05-22T21:20:16.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e51e00640 1 -- 172.21.15.161:0/707629760 learned_addr learned my addr 172.21.15.161:0/707629760 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:16.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e50a00640 1 -- 172.21.15.161:0/707629760 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336458637 0 0) 0x7f3e5406cc40 con 0x7f3e54143c40 2024-05-22T21:20:16.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e50a00640 1 -- 172.21.15.161:0/707629760 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e38003530 con 0x7f3e54143c40 2024-05-22T21:20:16.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e51400640 1 --2- 172.21.15.161:0/707629760 >> v2:172.21.15.179:3300/0 conn(0x7f3e5415b600 0x7f3e5406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e50a00640 1 -- 172.21.15.161:0/707629760 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3259272851 0 0) 0x7f3e38003530 con 0x7f3e54143c40 2024-05-22T21:20:16.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e50a00640 1 -- 172.21.15.161:0/707629760 >> v2:172.21.15.179:3300/0 conn(0x7f3e5415b600 msgr2=0x7f3e5406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:16.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e50a00640 1 --2- 172.21.15.161:0/707629760 >> v2:172.21.15.179:3300/0 conn(0x7f3e5415b600 0x7f3e5406c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e50a00640 1 -- 172.21.15.161:0/707629760 >> v1:172.21.15.179:6789/0 conn(0x7f3e5406c7e0 legacy=0x7f3e540b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e50a00640 1 -- 172.21.15.161:0/707629760 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5415efb0 con 0x7f3e54143c40 2024-05-22T21:20:16.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e50a00640 1 -- 172.21.15.161:0/707629760 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f3e44001680 con 0x7f3e54143c40 2024-05-22T21:20:16.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e50a00640 1 -- 172.21.15.161:0/707629760 >> v1:172.21.15.161:6789/0 conn(0x7f3e54143c40 legacy=0x7f3e541444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e50a00640 1 --2- 172.21.15.161:0/707629760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e38003980 0x7f3e38003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e50a00640 1 --2- 172.21.15.161:0/707629760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e380047d0 0x7f3e38004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e50a00640 1 --2- 172.21.15.161:0/707629760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e38005190 0x7f3e3800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e50a00640 1 -- 172.21.15.161:0/707629760 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5415ba60 con 0x7f3e380047d0 2024-05-22T21:20:16.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e50a00640 1 -- 172.21.15.161:0/707629760 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3800c0e0 con 0x7f3e38005190 2024-05-22T21:20:16.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e50a00640 1 -- 172.21.15.161:0/707629760 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3800c2a0 con 0x7f3e38003980 2024-05-22T21:20:16.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e51400640 1 --2- 172.21.15.161:0/707629760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e380047d0 0x7f3e38004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e51e00640 1 --2- 172.21.15.161:0/707629760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e38003980 0x7f3e38003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e52800640 1 --2- 172.21.15.161:0/707629760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e38005190 0x7f3e3800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e51400640 1 -- 172.21.15.161:0/707629760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e38003980 msgr2=0x7f3e38003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:16.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e51400640 1 --2- 172.21.15.161:0/707629760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e38003980 0x7f3e38003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e51400640 1 -- 172.21.15.161:0/707629760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e38005190 msgr2=0x7f3e3800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:16.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e51400640 1 --2- 172.21.15.161:0/707629760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e38005190 0x7f3e3800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e51400640 1 -- 172.21.15.161:0/707629760 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e38003530 con 0x7f3e380047d0 2024-05-22T21:20:16.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e52800640 1 -- 172.21.15.161:0/707629760 reap_dead start 2024-05-22T21:20:16.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.926+0000 7f3e51e00640 1 --2- 172.21.15.161:0/707629760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e38003980 0x7f3e38003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:16.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.930+0000 7f3e51400640 1 --2- 172.21.15.161:0/707629760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e380047d0 0x7f3e38004c50 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000ba30 tx=0x7f3e4001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18efcdc9ebd4445 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:16.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.930+0000 7f3e50a00640 1 -- 172.21.15.161:0/707629760 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e400a9680 con 0x7f3e380047d0 2024-05-22T21:20:16.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.930+0000 7f3e50a00640 1 -- 172.21.15.161:0/707629760 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e400a97e0 con 0x7f3e380047d0 2024-05-22T21:20:16.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.930+0000 7f3e50a00640 1 -- 172.21.15.161:0/707629760 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e400a9b90 con 0x7f3e380047d0 2024-05-22T21:20:16.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.930+0000 7f3e59200640 1 -- 172.21.15.161:0/707629760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e380047d0 msgr2=0x7f3e38004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.930+0000 7f3e59200640 1 --2- 172.21.15.161:0/707629760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e380047d0 0x7f3e38004c50 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000ba30 tx=0x7f3e4001c6a0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.930+0000 7f3e59200640 1 -- 172.21.15.161:0/707629760 shutdown_connections 2024-05-22T21:20:16.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.930+0000 7f3e59200640 1 --2- 172.21.15.161:0/707629760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e380047d0 0x7f3e38004c50 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.930+0000 7f3e59200640 1 -- 172.21.15.161:0/707629760 >> 172.21.15.161:0/707629760 conn(0x7f3e5406a2d0 msgr2=0x7f3e5406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:16.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.930+0000 7f3e59200640 1 -- 172.21.15.161:0/707629760 shutdown_connections 2024-05-22T21:20:16.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.930+0000 7f3e59200640 1 -- 172.21.15.161:0/707629760 wait complete. 2024-05-22T21:20:16.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e59200640 1 Processor -- start 2024-05-22T21:20:16.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e59200640 1 -- start start 2024-05-22T21:20:16.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e59200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e380047d0 0x7f3e541536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e59200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5413eb50 0x7f3e5413efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e59200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e5415cba0 0x7f3e5415d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e59200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5413f510 con 0x7f3e5413eb50 2024-05-22T21:20:16.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e59200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5415d560 con 0x7f3e380047d0 2024-05-22T21:20:16.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e59200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e541f6990 con 0x7f3e5415cba0 2024-05-22T21:20:16.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e51400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5413eb50 0x7f3e5413efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e51400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5413eb50 0x7f3e5413efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45094/0 (socket says 172.21.15.161:45094) 2024-05-22T21:20:16.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e51400640 1 -- 172.21.15.161:0/3893536231 learned_addr learned my addr 172.21.15.161:0/3893536231 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:16.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e51e00640 1 --2- 172.21.15.161:0/3893536231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e380047d0 0x7f3e541536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e52800640 1 --2- 172.21.15.161:0/3893536231 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e5415cba0 0x7f3e5415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e51400640 1 -- 172.21.15.161:0/3893536231 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e5415cba0 msgr2=0x7f3e5415d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e51400640 1 --2- 172.21.15.161:0/3893536231 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e5415cba0 0x7f3e5415d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e51400640 1 -- 172.21.15.161:0/3893536231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e380047d0 msgr2=0x7f3e541536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:16.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e51400640 1 --2- 172.21.15.161:0/3893536231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e380047d0 0x7f3e541536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:16.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e51400640 1 -- 172.21.15.161:0/3893536231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e541f6c70 con 0x7f3e5413eb50 2024-05-22T21:20:16.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e51400640 1 --2- 172.21.15.161:0/3893536231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5413eb50 0x7f3e5413efd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3e4001cc10 tx=0x7f3e38005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:16.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e4a000640 1 -- 172.21.15.161:0/3893536231 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40004020 con 0x7f3e5413eb50 2024-05-22T21:20:16.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e4a000640 1 -- 172.21.15.161:0/3893536231 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e40004180 con 0x7f3e5413eb50 2024-05-22T21:20:16.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e4a000640 1 -- 172.21.15.161:0/3893536231 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e400044f0 con 0x7f3e5413eb50 2024-05-22T21:20:16.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e59200640 1 -- 172.21.15.161:0/3893536231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e541f6f50 con 0x7f3e5413eb50 2024-05-22T21:20:16.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e59200640 1 -- 172.21.15.161:0/3893536231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e541f7510 con 0x7f3e5413eb50 2024-05-22T21:20:16.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.934+0000 7f3e59200640 1 -- 172.21.15.161:0/3893536231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e540b1ec0 con 0x7f3e5413eb50 2024-05-22T21:20:16.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.938+0000 7f3e4a000640 1 -- 172.21.15.161:0/3893536231 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3e40003620 con 0x7f3e5413eb50 2024-05-22T21:20:16.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.938+0000 7f3e4a000640 1 --2- 172.21.15.161:0/3893536231 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e340666c0 0x7f3e34068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:16.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.938+0000 7f3e4a000640 1 -- 172.21.15.161:0/3893536231 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3e4011eca0 con 0x7f3e5413eb50 2024-05-22T21:20:16.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.938+0000 7f3e51e00640 1 --2- 172.21.15.161:0/3893536231 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e340666c0 0x7f3e34068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:16.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.938+0000 7f3e51e00640 1 --2- 172.21.15.161:0/3893536231 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e340666c0 0x7f3e34068b80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3e54063da0 tx=0x7f3e44009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:16.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:16.942+0000 7f3e4a000640 1 -- 172.21.15.161:0/3893536231 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3e400ebfb0 con 0x7f3e5413eb50 2024-05-22T21:20:17.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.094+0000 7f3e59200640 1 -- 172.21.15.161:0/3893536231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3e54067220 con 0x7f3e5413eb50 2024-05-22T21:20:17.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.094+0000 7f3e4a000640 1 -- 172.21.15.161:0/3893536231 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3e400ef5d0 con 0x7f3e5413eb50 2024-05-22T21:20:17.101 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:20:17.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.098+0000 7f3e59200640 1 -- 172.21.15.161:0/3893536231 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e340666c0 msgr2=0x7f3e34068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:17.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.098+0000 7f3e59200640 1 --2- 172.21.15.161:0/3893536231 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e340666c0 0x7f3e34068b80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3e54063da0 tx=0x7f3e44009040 comp rx=0 tx=0).stop 2024-05-22T21:20:17.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.098+0000 7f3e59200640 1 -- 172.21.15.161:0/3893536231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5413eb50 msgr2=0x7f3e5413efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:17.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.098+0000 7f3e59200640 1 --2- 172.21.15.161:0/3893536231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5413eb50 0x7f3e5413efd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3e4001cc10 tx=0x7f3e38005800 comp rx=0 tx=0).stop 2024-05-22T21:20:17.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.098+0000 7f3e59200640 1 -- 172.21.15.161:0/3893536231 shutdown_connections 2024-05-22T21:20:17.102 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.098+0000 7f3e59200640 1 --2- 172.21.15.161:0/3893536231 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e340666c0 0x7f3e34068b80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.098+0000 7f3e59200640 1 --2- 172.21.15.161:0/3893536231 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e5415cba0 0x7f3e5415d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.098+0000 7f3e59200640 1 --2- 172.21.15.161:0/3893536231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5413eb50 0x7f3e5413efd0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.098+0000 7f3e59200640 1 --2- 172.21.15.161:0/3893536231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e380047d0 0x7f3e541536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.098+0000 7f3e59200640 1 -- 172.21.15.161:0/3893536231 >> 172.21.15.161:0/3893536231 conn(0x7f3e5406a2d0 msgr2=0x7f3e54142670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:17.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.098+0000 7f3e59200640 1 -- 172.21.15.161:0/3893536231 shutdown_connections 2024-05-22T21:20:17.103 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.098+0000 7f3e59200640 1 -- 172.21.15.161:0/3893536231 wait complete. 2024-05-22T21:20:17.119 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:20:17.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa49f000640 1 Processor -- start 2024-05-22T21:20:17.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa49f000640 1 -- start start 2024-05-22T21:20:17.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa49f000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa498099a40 0x7fa4980cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:17.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa49f000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa49806f0a0 0x7fa49806f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:17.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa49f000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4980cff60 con 0x7fa4980ceb40 2024-05-22T21:20:17.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa49f000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4980cd050 con 0x7fa49806f0a0 2024-05-22T21:20:17.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa49f000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49815d8e0 con 0x7fa498099a40 2024-05-22T21:20:17.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa48f400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa4980ceb40 0x7fa498099e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51206/0 (socket says 172.21.15.161:51206) 2024-05-22T21:20:17.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa48f400640 1 -- 172.21.15.161:0/27340115 learned_addr learned my addr 172.21.15.161:0/27340115 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:17.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa497e00640 1 --2- 172.21.15.161:0/27340115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa498099a40 0x7fa4980cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:17.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa49d000640 1 --2- 172.21.15.161:0/27340115 >> v2:172.21.15.179:3300/0 conn(0x7fa49806f0a0 0x7fa49806f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:17.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa497400640 1 -- 172.21.15.161:0/27340115 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239925085 0 0) 0x7fa4980cff60 con 0x7fa4980ceb40 2024-05-22T21:20:17.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa497400640 1 -- 172.21.15.161:0/27340115 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa47c003530 con 0x7fa4980ceb40 2024-05-22T21:20:17.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa497e00640 1 -- 172.21.15.161:0/27340115 >> v2:172.21.15.179:3300/0 conn(0x7fa49806f0a0 msgr2=0x7fa49806f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:17.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa497e00640 1 --2- 172.21.15.161:0/27340115 >> v2:172.21.15.179:3300/0 conn(0x7fa49806f0a0 0x7fa49806f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa497e00640 1 -- 172.21.15.161:0/27340115 >> v1:172.21.15.161:6789/0 conn(0x7fa4980ceb40 legacy=0x7fa498099e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:17.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa497400640 1 -- 172.21.15.161:0/27340115 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 465696512 0 0) 0x7fa47c003530 con 0x7fa4980ceb40 2024-05-22T21:20:17.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa497e00640 1 -- 172.21.15.161:0/27340115 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49806e000 con 0x7fa498099a40 2024-05-22T21:20:17.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.254+0000 7fa49d000640 1 --2- 172.21.15.161:0/27340115 >> v2:172.21.15.179:3300/0 conn(0x7fa49806f0a0 0x7fa49806f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:20:17.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.258+0000 7fa497e00640 1 --2- 172.21.15.161:0/27340115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa498099a40 0x7fa4980cfa20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa488009d90 tx=0x7fa48801c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=362c5453d7f81726 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:17.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.258+0000 7fa497400640 1 -- 172.21.15.161:0/27340115 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480001680 con 0x7fa498099a40 2024-05-22T21:20:17.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.258+0000 7fa497400640 1 -- 172.21.15.161:0/27340115 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4880a5660 con 0x7fa498099a40 2024-05-22T21:20:17.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.258+0000 7fa497400640 1 -- 172.21.15.161:0/27340115 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4880a59f0 con 0x7fa498099a40 2024-05-22T21:20:17.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.258+0000 7fa49f000640 1 -- 172.21.15.161:0/27340115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa498099a40 msgr2=0x7fa4980cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:17.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.258+0000 7fa49f000640 1 --2- 172.21.15.161:0/27340115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa498099a40 0x7fa4980cfa20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa488009d90 tx=0x7fa48801c860 comp rx=0 tx=0).stop 2024-05-22T21:20:17.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.258+0000 7fa49f000640 1 -- 172.21.15.161:0/27340115 shutdown_connections 2024-05-22T21:20:17.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.258+0000 7fa49f000640 1 --2- 172.21.15.161:0/27340115 >> v2:172.21.15.179:3300/0 conn(0x7fa49806f0a0 0x7fa49806f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.258+0000 7fa49f000640 1 --2- 172.21.15.161:0/27340115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa498099a40 0x7fa4980cfa20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.258+0000 7fa49f000640 1 -- 172.21.15.161:0/27340115 >> 172.21.15.161:0/27340115 conn(0x7fa4980b2a60 msgr2=0x7fa4980b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:17.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.258+0000 7fa49f000640 1 -- 172.21.15.161:0/27340115 shutdown_connections 2024-05-22T21:20:17.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49f000640 1 -- 172.21.15.161:0/27340115 wait complete. 2024-05-22T21:20:17.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49f000640 1 Processor -- start 2024-05-22T21:20:17.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49f000640 1 -- start start 2024-05-22T21:20:17.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49f000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa49806f0a0 0x7fa49814a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:17.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49f000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa498099a40 0x7fa49815da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:17.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49f000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4980ceb40 0x7fa4980c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:17.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49f000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4980c7020 con 0x7fa4980ceb40 2024-05-22T21:20:17.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49f000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4980c8410 con 0x7fa498099a40 2024-05-22T21:20:17.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49f000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4980c72b0 con 0x7fa49806f0a0 2024-05-22T21:20:17.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4980ceb40 0x7fa4980c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:17.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4980ceb40 0x7fa4980c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45104/0 (socket says 172.21.15.161:45104) 2024-05-22T21:20:17.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49d000640 1 -- 172.21.15.161:0/1343031239 learned_addr learned my addr 172.21.15.161:0/1343031239 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:17.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa48f400640 1 --2- 172.21.15.161:0/1343031239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa498099a40 0x7fa49815da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:17.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa497e00640 1 --2- 172.21.15.161:0/1343031239 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa49806f0a0 0x7fa49814a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:17.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49d000640 1 -- 172.21.15.161:0/1343031239 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa49806f0a0 msgr2=0x7fa49814a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:17.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49d000640 1 --2- 172.21.15.161:0/1343031239 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa49806f0a0 0x7fa49814a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49d000640 1 -- 172.21.15.161:0/1343031239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa498099a40 msgr2=0x7fa49815da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:17.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49d000640 1 --2- 172.21.15.161:0/1343031239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa498099a40 0x7fa49815da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49d000640 1 -- 172.21.15.161:0/1343031239 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa490009420 con 0x7fa4980ceb40 2024-05-22T21:20:17.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49d000640 1 --2- 172.21.15.161:0/1343031239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4980ceb40 0x7fa4980c5cc0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fa49000e950 tx=0x7fa49000ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:17.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa494c00640 1 -- 172.21.15.161:0/1343031239 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490009720 con 0x7fa4980ceb40 2024-05-22T21:20:17.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa494c00640 1 -- 172.21.15.161:0/1343031239 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49000ca00 con 0x7fa4980ceb40 2024-05-22T21:20:17.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49f000640 1 -- 172.21.15.161:0/1343031239 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa488009a70 con 0x7fa4980ceb40 2024-05-22T21:20:17.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa494c00640 1 -- 172.21.15.161:0/1343031239 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49000cd90 con 0x7fa4980ceb40 2024-05-22T21:20:17.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.262+0000 7fa49f000640 1 -- 172.21.15.161:0/1343031239 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4980c86b0 con 0x7fa4980ceb40 2024-05-22T21:20:17.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.266+0000 7fa49f000640 1 -- 172.21.15.161:0/1343031239 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa498073ce0 con 0x7fa4980ceb40 2024-05-22T21:20:17.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.266+0000 7fa494c00640 1 -- 172.21.15.161:0/1343031239 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa490013470 con 0x7fa4980ceb40 2024-05-22T21:20:17.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.266+0000 7fa494c00640 1 --2- 172.21.15.161:0/1343031239 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa474066690 0x7fa474068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:17.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.266+0000 7fa494c00640 1 -- 172.21.15.161:0/1343031239 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa490083e60 con 0x7fa4980ceb40 2024-05-22T21:20:17.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.266+0000 7fa497e00640 1 --2- 172.21.15.161:0/1343031239 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa474066690 0x7fa474068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:17.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.266+0000 7fa497e00640 1 --2- 172.21.15.161:0/1343031239 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa474066690 0x7fa474068b50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa4880af000 tx=0x7fa4880a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:17.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.270+0000 7fa494c00640 1 -- 172.21.15.161:0/1343031239 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa490051180 con 0x7fa4980ceb40 2024-05-22T21:20:17.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.426+0000 7fa49f000640 1 -- 172.21.15.161:0/1343031239 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa498067220 con 0x7fa4980ceb40 2024-05-22T21:20:17.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.426+0000 7fa494c00640 1 -- 172.21.15.161:0/1343031239 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4900547a0 con 0x7fa4980ceb40 2024-05-22T21:20:17.428 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:20:17.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.426+0000 7fa49f000640 1 -- 172.21.15.161:0/1343031239 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa474066690 msgr2=0x7fa474068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:17.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.426+0000 7fa49f000640 1 --2- 172.21.15.161:0/1343031239 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa474066690 0x7fa474068b50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa4880af000 tx=0x7fa4880a7070 comp rx=0 tx=0).stop 2024-05-22T21:20:17.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.426+0000 7fa49f000640 1 -- 172.21.15.161:0/1343031239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4980ceb40 msgr2=0x7fa4980c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:17.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.426+0000 7fa49f000640 1 --2- 172.21.15.161:0/1343031239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4980ceb40 0x7fa4980c5cc0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fa49000e950 tx=0x7fa49000ee10 comp rx=0 tx=0).stop 2024-05-22T21:20:17.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.426+0000 7fa49f000640 1 -- 172.21.15.161:0/1343031239 shutdown_connections 2024-05-22T21:20:17.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.426+0000 7fa49f000640 1 --2- 172.21.15.161:0/1343031239 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa474066690 0x7fa474068b50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.426+0000 7fa49f000640 1 --2- 172.21.15.161:0/1343031239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4980ceb40 0x7fa4980c5cc0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.426+0000 7fa49f000640 1 --2- 172.21.15.161:0/1343031239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa498099a40 0x7fa49815da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.426+0000 7fa49f000640 1 --2- 172.21.15.161:0/1343031239 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa49806f0a0 0x7fa49814a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.426+0000 7fa49f000640 1 -- 172.21.15.161:0/1343031239 >> 172.21.15.161:0/1343031239 conn(0x7fa4980b2a60 msgr2=0x7fa4980b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:17.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.426+0000 7fa49f000640 1 -- 172.21.15.161:0/1343031239 shutdown_connections 2024-05-22T21:20:17.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.426+0000 7fa49f000640 1 -- 172.21.15.161:0/1343031239 wait complete. 2024-05-22T21:20:17.450 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:20:17.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f531b200640 1 Processor -- start 2024-05-22T21:20:17.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f531b200640 1 -- start start 2024-05-22T21:20:17.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f531b200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f531413cfc0 0x7f53141402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:17.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f531b200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f531415af60 0x7f531413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:17.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f531b200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f531413fc80 0x7f531415dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:17.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f531b200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53141400d0 con 0x7f531415af60 2024-05-22T21:20:17.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f531b200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531413d3a0 con 0x7f531413fc80 2024-05-22T21:20:17.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f531b200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53140cd920 con 0x7f531413cfc0 2024-05-22T21:20:17.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f530b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f531415af60 0x7f531413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:17.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f530b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f531415af60 0x7f531413f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45116/0 (socket says 172.21.15.161:45116) 2024-05-22T21:20:17.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f530b400640 1 -- 172.21.15.161:0/3383687451 learned_addr learned my addr 172.21.15.161:0/3383687451 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:17.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f5319200640 1 --2- 172.21.15.161:0/3383687451 >> v2:172.21.15.179:3300/0 conn(0x7f531413fc80 0x7f531415dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:17.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f5313e00640 1 --2- 172.21.15.161:0/3383687451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f531413cfc0 0x7f53141402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:17.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f530b400640 1 -- 172.21.15.161:0/3383687451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f531413cfc0 msgr2=0x7f53141402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:17.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f530b400640 1 --2- 172.21.15.161:0/3383687451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f531413cfc0 0x7f53141402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f530b400640 1 -- 172.21.15.161:0/3383687451 >> v2:172.21.15.179:3300/0 conn(0x7f531413fc80 msgr2=0x7f531415dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:17.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f530b400640 1 --2- 172.21.15.161:0/3383687451 >> v2:172.21.15.179:3300/0 conn(0x7f531413fc80 0x7f531415dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f530b400640 1 -- 172.21.15.161:0/3383687451 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53140cdc00 con 0x7f531415af60 2024-05-22T21:20:17.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.586+0000 7f5313e00640 1 --2- 172.21.15.161:0/3383687451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f531413cfc0 0x7f53141402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:17.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.590+0000 7f530b400640 1 --2- 172.21.15.161:0/3383687451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531415af60 0x7f531413f740 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f531406de40 tx=0x7f52fc01c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596adbec076eb889 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:17.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.590+0000 7f5313400640 1 -- 172.21.15.161:0/3383687451 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc0ab600 con 0x7f531415af60 2024-05-22T21:20:17.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.590+0000 7f5313400640 1 -- 172.21.15.161:0/3383687451 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc0a7050 con 0x7f531415af60 2024-05-22T21:20:17.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.590+0000 7f5313400640 1 -- 172.21.15.161:0/3383687451 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc0ab980 con 0x7f531415af60 2024-05-22T21:20:17.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.590+0000 7f531b200640 1 -- 172.21.15.161:0/3383687451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531415af60 msgr2=0x7f531413f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:17.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.590+0000 7f531b200640 1 --2- 172.21.15.161:0/3383687451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531415af60 0x7f531413f740 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f531406de40 tx=0x7f52fc01c1a0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.590+0000 7f531b200640 1 -- 172.21.15.161:0/3383687451 shutdown_connections 2024-05-22T21:20:17.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.590+0000 7f531b200640 1 --2- 172.21.15.161:0/3383687451 >> v2:172.21.15.179:3300/0 conn(0x7f531413fc80 0x7f531415dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.590+0000 7f531b200640 1 --2- 172.21.15.161:0/3383687451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531415af60 0x7f531413f740 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.590+0000 7f531b200640 1 --2- 172.21.15.161:0/3383687451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f531413cfc0 0x7f53141402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.590+0000 7f531b200640 1 -- 172.21.15.161:0/3383687451 >> 172.21.15.161:0/3383687451 conn(0x7f531406a2d0 msgr2=0x7f531406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:17.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.590+0000 7f531b200640 1 -- 172.21.15.161:0/3383687451 shutdown_connections 2024-05-22T21:20:17.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.590+0000 7f531b200640 1 -- 172.21.15.161:0/3383687451 wait complete. 2024-05-22T21:20:17.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f531b200640 1 Processor -- start 2024-05-22T21:20:17.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f531b200640 1 -- start start 2024-05-22T21:20:17.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f531b200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f531413cfc0 0x7f53140ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:17.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f531b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531413fc80 0x7f53140ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:17.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f531b200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f531415af60 0x7f5314158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:17.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f531b200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53140b3100 con 0x7f531413fc80 2024-05-22T21:20:17.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f531b200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53140b33e0 con 0x7f531415af60 2024-05-22T21:20:17.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f531b200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53140b36c0 con 0x7f531413cfc0 2024-05-22T21:20:17.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f530b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531413fc80 0x7f53140ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:17.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f530b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531413fc80 0x7f53140ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45134/0 (socket says 172.21.15.161:45134) 2024-05-22T21:20:17.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f530b400640 1 -- 172.21.15.161:0/2381419921 learned_addr learned my addr 172.21.15.161:0/2381419921 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:17.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f5313e00640 1 --2- 172.21.15.161:0/2381419921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f531413cfc0 0x7f53140ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:17.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f5319200640 1 --2- 172.21.15.161:0/2381419921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f531415af60 0x7f5314158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:17.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f530b400640 1 -- 172.21.15.161:0/2381419921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f531413cfc0 msgr2=0x7f53140ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:17.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f530b400640 1 --2- 172.21.15.161:0/2381419921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f531413cfc0 0x7f53140ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f530b400640 1 -- 172.21.15.161:0/2381419921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f531415af60 msgr2=0x7f5314158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:17.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f530b400640 1 --2- 172.21.15.161:0/2381419921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f531415af60 0x7f5314158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f530b400640 1 -- 172.21.15.161:0/2381419921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc0094d0 con 0x7f531413fc80 2024-05-22T21:20:17.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f530b400640 1 --2- 172.21.15.161:0/2381419921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531413fc80 0x7f53140ade40 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0ae040 tx=0x7f52fc018f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:17.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f5310c00640 1 -- 172.21.15.161:0/2381419921 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc0acdc0 con 0x7f531413fc80 2024-05-22T21:20:17.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f5310c00640 1 -- 172.21.15.161:0/2381419921 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc0b6420 con 0x7f531413fc80 2024-05-22T21:20:17.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f5310c00640 1 -- 172.21.15.161:0/2381419921 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc0b67d0 con 0x7f531413fc80 2024-05-22T21:20:17.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f531b200640 1 -- 172.21.15.161:0/2381419921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53140b39a0 con 0x7f531413fc80 2024-05-22T21:20:17.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f531b200640 1 -- 172.21.15.161:0/2381419921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531413e540 con 0x7f531413fc80 2024-05-22T21:20:17.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.594+0000 7f531b200640 1 -- 172.21.15.161:0/2381419921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53140b3b90 con 0x7f531413fc80 2024-05-22T21:20:17.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.598+0000 7f5310c00640 1 -- 172.21.15.161:0/2381419921 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f52fc0b2040 con 0x7f531413fc80 2024-05-22T21:20:17.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.598+0000 7f5310c00640 1 --2- 172.21.15.161:0/2381419921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52f00666c0 0x7f52f0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:17.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.598+0000 7f5310c00640 1 -- 172.21.15.161:0/2381419921 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f52fc11e420 con 0x7f531413fc80 2024-05-22T21:20:17.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.598+0000 7f5313e00640 1 --2- 172.21.15.161:0/2381419921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52f00666c0 0x7f52f0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:17.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.598+0000 7f5313e00640 1 --2- 172.21.15.161:0/2381419921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52f00666c0 0x7f52f0068b80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f530400b930 tx=0x7f5304008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:17.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.602+0000 7f5310c00640 1 -- 172.21.15.161:0/2381419921 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f52fc0eb6b0 con 0x7f531413fc80 2024-05-22T21:20:17.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.766+0000 7f531b200640 1 -- 172.21.15.161:0/2381419921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f53140b3240 con 0x7f531413fc80 2024-05-22T21:20:17.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.766+0000 7f5310c00640 1 -- 172.21.15.161:0/2381419921 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) v1 ==== 80+0+0 (secure 0 0 0) 0x7f53140b3240 con 0x7f531413fc80 2024-05-22T21:20:17.771 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:20:17.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.770+0000 7f531b200640 1 -- 172.21.15.161:0/2381419921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52f00666c0 msgr2=0x7f52f0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:17.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.770+0000 7f531b200640 1 --2- 172.21.15.161:0/2381419921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52f00666c0 0x7f52f0068b80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f530400b930 tx=0x7f5304008040 comp rx=0 tx=0).stop 2024-05-22T21:20:17.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.770+0000 7f531b200640 1 -- 172.21.15.161:0/2381419921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531413fc80 msgr2=0x7f53140ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:17.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.770+0000 7f531b200640 1 --2- 172.21.15.161:0/2381419921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531413fc80 0x7f53140ade40 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0ae040 tx=0x7f52fc018f40 comp rx=0 tx=0).stop 2024-05-22T21:20:17.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.770+0000 7f531b200640 1 -- 172.21.15.161:0/2381419921 shutdown_connections 2024-05-22T21:20:17.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.770+0000 7f531b200640 1 --2- 172.21.15.161:0/2381419921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f52f00666c0 0x7f52f0068b80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.770+0000 7f531b200640 1 --2- 172.21.15.161:0/2381419921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f531415af60 0x7f5314158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.770+0000 7f531b200640 1 --2- 172.21.15.161:0/2381419921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531413fc80 0x7f53140ade40 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.770+0000 7f531b200640 1 --2- 172.21.15.161:0/2381419921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f531413cfc0 0x7f53140ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:17.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.770+0000 7f531b200640 1 -- 172.21.15.161:0/2381419921 >> 172.21.15.161:0/2381419921 conn(0x7f531406a2d0 msgr2=0x7f531406cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:17.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.770+0000 7f531b200640 1 -- 172.21.15.161:0/2381419921 shutdown_connections 2024-05-22T21:20:17.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:17.770+0000 7f531b200640 1 -- 172.21.15.161:0/2381419921 wait complete. 2024-05-22T21:20:17.791 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-22T21:20:17.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:17.792 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-22T21:20:17.844 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:20:17.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:17.844 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-22T21:20:17.861 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:20:17.861 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:20:17.870 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:20:17.870 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:20:17.922 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:20:18.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.054+0000 7ff01fa00640 1 Processor -- start 2024-05-22T21:20:18.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.054+0000 7ff01fa00640 1 -- start start 2024-05-22T21:20:18.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff01fa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff01815dd50 0x7ff01806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff01fa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff01815e130 0x7ff01815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff01fa00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff01815d410 0x7ff01815a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff01fa00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff018071670 con 0x7ff01815dd50 2024-05-22T21:20:18.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff01fa00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff01815d8a0 con 0x7ff01815d410 2024-05-22T21:20:18.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff01fa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01806de40 con 0x7ff01815e130 2024-05-22T21:20:18.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff01d000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff01815dd50 0x7ff01806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff01d000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff01815dd50 0x7ff01806d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45148/0 (socket says 172.21.15.161:45148) 2024-05-22T21:20:18.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff01d000640 1 -- 172.21.15.161:0/3466340549 learned_addr learned my addr 172.21.15.161:0/3466340549 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:18.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff01da00640 1 --2- 172.21.15.161:0/3466340549 >> v2:172.21.15.179:3300/0 conn(0x7ff01815d410 0x7ff01815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff00fe00640 1 --2- 172.21.15.161:0/3466340549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff01815e130 0x7ff01815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff01d000640 1 -- 172.21.15.161:0/3466340549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff01815e130 msgr2=0x7ff01815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:18.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff01d000640 1 --2- 172.21.15.161:0/3466340549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff01815e130 0x7ff01815ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff01d000640 1 -- 172.21.15.161:0/3466340549 >> v2:172.21.15.179:3300/0 conn(0x7ff01815d410 msgr2=0x7ff01815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:18.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff01d000640 1 --2- 172.21.15.161:0/3466340549 >> v2:172.21.15.179:3300/0 conn(0x7ff01815d410 0x7ff01815a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff01d000640 1 -- 172.21.15.161:0/3466340549 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff018070a60 con 0x7ff01815dd50 2024-05-22T21:20:18.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff00fe00640 1 --2- 172.21.15.161:0/3466340549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff01815e130 0x7ff01815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:18.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.058+0000 7ff01da00640 1 --2- 172.21.15.161:0/3466340549 >> v2:172.21.15.179:3300/0 conn(0x7ff01815d410 0x7ff01815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:18.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.062+0000 7ff01d000640 1 --2- 172.21.15.161:0/3466340549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff01815dd50 0x7ff01806d900 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff0180721d0 tx=0x7ff00801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c041f91c558eabca server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:18.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.062+0000 7ff017e00640 1 -- 172.21.15.161:0/3466340549 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0080ab620 con 0x7ff01815dd50 2024-05-22T21:20:18.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.062+0000 7ff017e00640 1 -- 172.21.15.161:0/3466340549 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0080ab780 con 0x7ff01815dd50 2024-05-22T21:20:18.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.062+0000 7ff017e00640 1 -- 172.21.15.161:0/3466340549 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0080abb10 con 0x7ff01815dd50 2024-05-22T21:20:18.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.062+0000 7ff01fa00640 1 -- 172.21.15.161:0/3466340549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff01815dd50 msgr2=0x7ff01806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.062+0000 7ff01fa00640 1 --2- 172.21.15.161:0/3466340549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff01815dd50 0x7ff01806d900 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff0180721d0 tx=0x7ff00801c460 comp rx=0 tx=0).stop 2024-05-22T21:20:18.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.062+0000 7ff01fa00640 1 -- 172.21.15.161:0/3466340549 shutdown_connections 2024-05-22T21:20:18.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.062+0000 7ff01fa00640 1 --2- 172.21.15.161:0/3466340549 >> v2:172.21.15.179:3300/0 conn(0x7ff01815d410 0x7ff01815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.062+0000 7ff01fa00640 1 --2- 172.21.15.161:0/3466340549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff01815e130 0x7ff01815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.062+0000 7ff01fa00640 1 --2- 172.21.15.161:0/3466340549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff01815dd50 0x7ff01806d900 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.062+0000 7ff01fa00640 1 -- 172.21.15.161:0/3466340549 >> 172.21.15.161:0/3466340549 conn(0x7ff0180b0bc0 msgr2=0x7ff0180b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:18.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.062+0000 7ff01fa00640 1 -- 172.21.15.161:0/3466340549 shutdown_connections 2024-05-22T21:20:18.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.062+0000 7ff01fa00640 1 -- 172.21.15.161:0/3466340549 wait complete. 2024-05-22T21:20:18.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.062+0000 7ff01fa00640 1 Processor -- start 2024-05-22T21:20:18.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.062+0000 7ff01fa00640 1 -- start start 2024-05-22T21:20:18.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff01fa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff01815d410 0x7ff0181531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff01fa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff01815dd50 0x7ff01814ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff01fa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff01815e130 0x7ff0180b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff01fa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01814d790 con 0x7ff01815dd50 2024-05-22T21:20:18.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff01fa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01814d9f0 con 0x7ff01815e130 2024-05-22T21:20:18.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff01fa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01814dcd0 con 0x7ff01815d410 2024-05-22T21:20:18.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff00fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff01815dd50 0x7ff01814ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff00fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff01815dd50 0x7ff01814ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45156/0 (socket says 172.21.15.161:45156) 2024-05-22T21:20:18.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff00fe00640 1 -- 172.21.15.161:0/162384365 learned_addr learned my addr 172.21.15.161:0/162384365 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:18.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff01d000640 1 --2- 172.21.15.161:0/162384365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff01815d410 0x7ff0181531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff01da00640 1 --2- 172.21.15.161:0/162384365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff01815e130 0x7ff0180b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff00fe00640 1 -- 172.21.15.161:0/162384365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff01815d410 msgr2=0x7ff0181531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff00fe00640 1 --2- 172.21.15.161:0/162384365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff01815d410 0x7ff0181531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff00fe00640 1 -- 172.21.15.161:0/162384365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff01815e130 msgr2=0x7ff0180b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff00fe00640 1 --2- 172.21.15.161:0/162384365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff01815e130 0x7ff0180b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff00fe00640 1 -- 172.21.15.161:0/162384365 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0080094d0 con 0x7ff01815dd50 2024-05-22T21:20:18.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff01da00640 1 --2- 172.21.15.161:0/162384365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff01815e130 0x7ff0180b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:18.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff00fe00640 1 --2- 172.21.15.161:0/162384365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff01815dd50 0x7ff01814ea90 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff00000c7f0 tx=0x7ff00000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:18.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff015600640 1 -- 172.21.15.161:0/162384365 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000007820 con 0x7ff01815dd50 2024-05-22T21:20:18.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff015600640 1 -- 172.21.15.161:0/162384365 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff000007980 con 0x7ff01815dd50 2024-05-22T21:20:18.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff015600640 1 -- 172.21.15.161:0/162384365 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000007d30 con 0x7ff01815dd50 2024-05-22T21:20:18.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff01fa00640 1 -- 172.21.15.161:0/162384365 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff01814dfb0 con 0x7ff01815dd50 2024-05-22T21:20:18.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff01fa00640 1 -- 172.21.15.161:0/162384365 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01814e570 con 0x7ff01815dd50 2024-05-22T21:20:18.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff01fa00640 1 -- 172.21.15.161:0/162384365 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff01814e790 con 0x7ff01815dd50 2024-05-22T21:20:18.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.066+0000 7ff015600640 1 -- 172.21.15.161:0/162384365 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff0000113f0 con 0x7ff01815dd50 2024-05-22T21:20:18.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.070+0000 7ff015600640 1 --2- 172.21.15.161:0/162384365 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feff4066690 0x7feff4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.070+0000 7ff015600640 1 -- 172.21.15.161:0/162384365 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff000084430 con 0x7ff01815dd50 2024-05-22T21:20:18.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.070+0000 7ff01d000640 1 --2- 172.21.15.161:0/162384365 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feff4066690 0x7feff4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.070+0000 7ff01d000640 1 --2- 172.21.15.161:0/162384365 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feff4066690 0x7feff4068b50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff0080af000 tx=0x7ff0080b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:18.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.074+0000 7ff015600640 1 -- 172.21.15.161:0/162384365 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff000051800 con 0x7ff01815dd50 2024-05-22T21:20:18.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.226+0000 7ff01fa00640 1 -- 172.21.15.161:0/162384365 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff018067220 con 0x7ff01815dd50 2024-05-22T21:20:18.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.226+0000 7ff015600640 1 -- 172.21.15.161:0/162384365 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+16268 (secure 0 0 0) 0x7ff000054e20 con 0x7ff01815dd50 2024-05-22T21:20:18.231 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:18.231 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":145,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:20:13.272722+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:44.080742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:45.388618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.30","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:20:18.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.230+0000 7ff01fa00640 1 -- 172.21.15.161:0/162384365 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feff4066690 msgr2=0x7feff4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.230+0000 7ff01fa00640 1 --2- 172.21.15.161:0/162384365 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feff4066690 0x7feff4068b50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff0080af000 tx=0x7ff0080b6040 comp rx=0 tx=0).stop 2024-05-22T21:20:18.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.230+0000 7ff01fa00640 1 -- 172.21.15.161:0/162384365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff01815dd50 msgr2=0x7ff01814ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.230+0000 7ff01fa00640 1 --2- 172.21.15.161:0/162384365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff01815dd50 0x7ff01814ea90 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff00000c7f0 tx=0x7ff00000ccb0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.230+0000 7ff01fa00640 1 -- 172.21.15.161:0/162384365 shutdown_connections 2024-05-22T21:20:18.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.230+0000 7ff01fa00640 1 --2- 172.21.15.161:0/162384365 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feff4066690 0x7feff4068b50 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.230+0000 7ff01fa00640 1 --2- 172.21.15.161:0/162384365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff01815e130 0x7ff0180b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.230+0000 7ff01fa00640 1 --2- 172.21.15.161:0/162384365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff01815dd50 0x7ff01814ea90 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.230+0000 7ff01fa00640 1 --2- 172.21.15.161:0/162384365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff01815d410 0x7ff0181531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.230+0000 7ff01fa00640 1 -- 172.21.15.161:0/162384365 >> 172.21.15.161:0/162384365 conn(0x7ff0180b0bc0 msgr2=0x7ff01815b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:18.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.230+0000 7ff01fa00640 1 -- 172.21.15.161:0/162384365 shutdown_connections 2024-05-22T21:20:18.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.230+0000 7ff01fa00640 1 -- 172.21.15.161:0/162384365 wait complete. 2024-05-22T21:20:18.251 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:19:44.080742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:19:45.388618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:20:18.252 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:20:18.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.386+0000 7f8e73000640 1 Processor -- start 2024-05-22T21:20:18.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.386+0000 7f8e73000640 1 -- start start 2024-05-22T21:20:18.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.386+0000 7f8e73000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8e6c13cfc0 0x7f8e6c1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.386+0000 7f8e73000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e6c13fe50 0x7f8e6c06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.386+0000 7f8e73000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e6c13d3a0 con 0x7f8e6c15af60 2024-05-22T21:20:18.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.386+0000 7f8e73000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e6c0cd920 con 0x7f8e6c13cfc0 2024-05-22T21:20:18.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.386+0000 7f8e73000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c0cdc00 con 0x7f8e6c13fe50 2024-05-22T21:20:18.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.390+0000 7f8e6be00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8e6c13cfc0 0x7f8e6c1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.390+0000 7f8e71000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e6c13fe50 0x7f8e6c06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.390+0000 7f8e63400640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8e6c15af60 0x7f8e6c13f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:52672/0 (socket says 172.21.15.161:52672) 2024-05-22T21:20:18.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.390+0000 7f8e63400640 1 -- 172.21.15.161:0/3031927986 learned_addr learned my addr 172.21.15.161:0/3031927986 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:18.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.390+0000 7f8e6be00640 1 -- 172.21.15.161:0/3031927986 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e6c13fe50 msgr2=0x7f8e6c06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:18.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.390+0000 7f8e6be00640 1 --2- 172.21.15.161:0/3031927986 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e6c13fe50 0x7f8e6c06fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.390+0000 7f8e6be00640 1 -- 172.21.15.161:0/3031927986 >> v1:172.21.15.179:6789/0 conn(0x7f8e6c15af60 legacy=0x7f8e6c13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.390+0000 7f8e6be00640 1 -- 172.21.15.161:0/3031927986 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6c0cdee0 con 0x7f8e6c13cfc0 2024-05-22T21:20:18.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.390+0000 7f8e6be00640 1 --2- 172.21.15.161:0/3031927986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e6c13cfc0 0x7f8e6c1402e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0097c0 tx=0x7f8e5c01e480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b15799e91720338 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:18.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.394+0000 7f8e6b400640 1 -- 172.21.15.161:0/3031927986 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c0ad600 con 0x7f8e6c13cfc0 2024-05-22T21:20:18.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.394+0000 7f8e6b400640 1 -- 172.21.15.161:0/3031927986 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5c0a9050 con 0x7f8e6c13cfc0 2024-05-22T21:20:18.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.394+0000 7f8e6b400640 1 -- 172.21.15.161:0/3031927986 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c0ad980 con 0x7f8e6c13cfc0 2024-05-22T21:20:18.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.394+0000 7f8e73000640 1 -- 172.21.15.161:0/3031927986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e6c13cfc0 msgr2=0x7f8e6c1402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.394+0000 7f8e73000640 1 --2- 172.21.15.161:0/3031927986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e6c13cfc0 0x7f8e6c1402e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0097c0 tx=0x7f8e5c01e480 comp rx=0 tx=0).stop 2024-05-22T21:20:18.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.394+0000 7f8e73000640 1 -- 172.21.15.161:0/3031927986 shutdown_connections 2024-05-22T21:20:18.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.394+0000 7f8e73000640 1 --2- 172.21.15.161:0/3031927986 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e6c13fe50 0x7f8e6c06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.394+0000 7f8e73000640 1 --2- 172.21.15.161:0/3031927986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e6c13cfc0 0x7f8e6c1402e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.394+0000 7f8e73000640 1 -- 172.21.15.161:0/3031927986 >> 172.21.15.161:0/3031927986 conn(0x7f8e6c06a2d0 msgr2=0x7f8e6c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:18.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.394+0000 7f8e73000640 1 -- 172.21.15.161:0/3031927986 shutdown_connections 2024-05-22T21:20:18.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.394+0000 7f8e73000640 1 -- 172.21.15.161:0/3031927986 wait complete. 2024-05-22T21:20:18.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.394+0000 7f8e73000640 1 Processor -- start 2024-05-22T21:20:18.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.394+0000 7f8e73000640 1 -- start start 2024-05-22T21:20:18.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e73000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e6c13cfc0 0x7f8e6c12c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e73000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e6c13fe50 0x7f8e6c0a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e73000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e6c15af60 0x7f8e6c13e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e73000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c13f900 con 0x7f8e6c13fe50 2024-05-22T21:20:18.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e73000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c157ed0 con 0x7f8e6c15af60 2024-05-22T21:20:18.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e73000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c12c9e0 con 0x7f8e6c13cfc0 2024-05-22T21:20:18.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e63400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e6c13fe50 0x7f8e6c0a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e63400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e6c13fe50 0x7f8e6c0a8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45170/0 (socket says 172.21.15.161:45170) 2024-05-22T21:20:18.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e63400640 1 -- 172.21.15.161:0/3150041182 learned_addr learned my addr 172.21.15.161:0/3150041182 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:18.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e6be00640 1 --2- 172.21.15.161:0/3150041182 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e6c13cfc0 0x7f8e6c12c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e71000640 1 --2- 172.21.15.161:0/3150041182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e6c15af60 0x7f8e6c13e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e63400640 1 -- 172.21.15.161:0/3150041182 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e6c13cfc0 msgr2=0x7f8e6c12c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e63400640 1 --2- 172.21.15.161:0/3150041182 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e6c13cfc0 0x7f8e6c12c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e63400640 1 -- 172.21.15.161:0/3150041182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e6c15af60 msgr2=0x7f8e6c13e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e63400640 1 --2- 172.21.15.161:0/3150041182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e6c15af60 0x7f8e6c13e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e63400640 1 -- 172.21.15.161:0/3150041182 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c009420 con 0x7f8e6c13fe50 2024-05-22T21:20:18.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e71000640 1 --2- 172.21.15.161:0/3150041182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e6c15af60 0x7f8e6c13e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:18.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e63400640 1 --2- 172.21.15.161:0/3150041182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e6c13fe50 0x7f8e6c0a8eb0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400c7b0 tx=0x7f8e5400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:18.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e68c00640 1 -- 172.21.15.161:0/3150041182 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e540072d0 con 0x7f8e6c13fe50 2024-05-22T21:20:18.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e68c00640 1 -- 172.21.15.161:0/3150041182 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e54007430 con 0x7f8e6c13fe50 2024-05-22T21:20:18.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e73000640 1 -- 172.21.15.161:0/3150041182 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6c141e10 con 0x7f8e6c13fe50 2024-05-22T21:20:18.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e68c00640 1 -- 172.21.15.161:0/3150041182 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e540047f0 con 0x7f8e6c13fe50 2024-05-22T21:20:18.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e73000640 1 -- 172.21.15.161:0/3150041182 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6c1423d0 con 0x7f8e6c13fe50 2024-05-22T21:20:18.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.398+0000 7f8e73000640 1 -- 172.21.15.161:0/3150041182 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e6c067220 con 0x7f8e6c13fe50 2024-05-22T21:20:18.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.402+0000 7f8e68c00640 1 -- 172.21.15.161:0/3150041182 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8e54004950 con 0x7f8e6c13fe50 2024-05-22T21:20:18.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.402+0000 7f8e68c00640 1 --2- 172.21.15.161:0/3150041182 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e48066690 0x7f8e48068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.402+0000 7f8e68c00640 1 -- 172.21.15.161:0/3150041182 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8e540837c0 con 0x7f8e6c13fe50 2024-05-22T21:20:18.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.402+0000 7f8e6be00640 1 --2- 172.21.15.161:0/3150041182 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e48066690 0x7f8e48068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.402+0000 7f8e6be00640 1 --2- 172.21.15.161:0/3150041182 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e48066690 0x7f8e48068b50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0af000 tx=0x7f8e5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:18.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.406+0000 7f8e68c00640 1 -- 172.21.15.161:0/3150041182 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8e54050b00 con 0x7f8e6c13fe50 2024-05-22T21:20:18.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.562+0000 7f8e73000640 1 -- 172.21.15.161:0/3150041182 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e6c070e00 con 0x7f8e6c13fe50 2024-05-22T21:20:18.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.562+0000 7f8e68c00640 1 -- 172.21.15.161:0/3150041182 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e540180a0 con 0x7f8e6c13fe50 2024-05-22T21:20:18.564 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:20:18.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.562+0000 7f8e73000640 1 -- 172.21.15.161:0/3150041182 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e48066690 msgr2=0x7f8e48068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.562+0000 7f8e73000640 1 --2- 172.21.15.161:0/3150041182 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e48066690 0x7f8e48068b50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0af000 tx=0x7f8e5c008040 comp rx=0 tx=0).stop 2024-05-22T21:20:18.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.562+0000 7f8e73000640 1 -- 172.21.15.161:0/3150041182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e6c13fe50 msgr2=0x7f8e6c0a8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.562+0000 7f8e73000640 1 --2- 172.21.15.161:0/3150041182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e6c13fe50 0x7f8e6c0a8eb0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400c7b0 tx=0x7f8e5400cc70 comp rx=0 tx=0).stop 2024-05-22T21:20:18.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.562+0000 7f8e73000640 1 -- 172.21.15.161:0/3150041182 shutdown_connections 2024-05-22T21:20:18.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.562+0000 7f8e73000640 1 --2- 172.21.15.161:0/3150041182 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e48066690 0x7f8e48068b50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.562+0000 7f8e73000640 1 --2- 172.21.15.161:0/3150041182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e6c15af60 0x7f8e6c13e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.562+0000 7f8e73000640 1 --2- 172.21.15.161:0/3150041182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e6c13fe50 0x7f8e6c0a8eb0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.562+0000 7f8e73000640 1 --2- 172.21.15.161:0/3150041182 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e6c13cfc0 0x7f8e6c12c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.562+0000 7f8e73000640 1 -- 172.21.15.161:0/3150041182 >> 172.21.15.161:0/3150041182 conn(0x7f8e6c06a2d0 msgr2=0x7f8e6c0af560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:18.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.562+0000 7f8e73000640 1 -- 172.21.15.161:0/3150041182 shutdown_connections 2024-05-22T21:20:18.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.562+0000 7f8e73000640 1 -- 172.21.15.161:0/3150041182 wait complete. 2024-05-22T21:20:18.585 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:20:18.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f3914200640 1 Processor -- start 2024-05-22T21:20:18.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f3914200640 1 -- start start 2024-05-22T21:20:18.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f3914200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f390c15cfa0 0x7f390c15e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f3914200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f390c0aefc0 0x7f390c0b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f3914200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390c15d8b0 con 0x7f390c15d380 2024-05-22T21:20:18.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f3914200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390c0b4a00 con 0x7f390c0aefc0 2024-05-22T21:20:18.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f3914200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390c15db50 con 0x7f390c15cfa0 2024-05-22T21:20:18.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f3912200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f390c15d380 0x7f390c069ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51218/0 (socket says 172.21.15.161:51218) 2024-05-22T21:20:18.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f3912200640 1 -- 172.21.15.161:0/313213708 learned_addr learned my addr 172.21.15.161:0/313213708 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:18.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f3911800640 1 --2- 172.21.15.161:0/313213708 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f390c15cfa0 0x7f390c15e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f3910e00640 1 --2- 172.21.15.161:0/313213708 >> v2:172.21.15.179:3300/0 conn(0x7f390c0aefc0 0x7f390c0b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f390be00640 1 -- 172.21.15.161:0/313213708 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553573302 0 0) 0x7f390c15d8b0 con 0x7f390c15d380 2024-05-22T21:20:18.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f390be00640 1 -- 172.21.15.161:0/313213708 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38f4003530 con 0x7f390c15d380 2024-05-22T21:20:18.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f390be00640 1 -- 172.21.15.161:0/313213708 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 826615467 0 0) 0x7f38f4003530 con 0x7f390c15d380 2024-05-22T21:20:18.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f390be00640 1 -- 172.21.15.161:0/313213708 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f390c15cfa0 msgr2=0x7f390c15e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:18.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f390be00640 1 --2- 172.21.15.161:0/313213708 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f390c15cfa0 0x7f390c15e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f390be00640 1 -- 172.21.15.161:0/313213708 >> v2:172.21.15.179:3300/0 conn(0x7f390c0aefc0 msgr2=0x7f390c0b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:18.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f390be00640 1 --2- 172.21.15.161:0/313213708 >> v2:172.21.15.179:3300/0 conn(0x7f390c0aefc0 0x7f390c0b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f390be00640 1 -- 172.21.15.161:0/313213708 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c15de30 con 0x7f390c15d380 2024-05-22T21:20:18.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f390be00640 1 -- 172.21.15.161:0/313213708 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f3904001a00 con 0x7f390c15d380 2024-05-22T21:20:18.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f390be00640 1 -- 172.21.15.161:0/313213708 >> v1:172.21.15.161:6789/0 conn(0x7f390c15d380 legacy=0x7f390c069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f390be00640 1 --2- 172.21.15.161:0/313213708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38f4003980 0x7f38f4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f390be00640 1 --2- 172.21.15.161:0/313213708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38f40047d0 0x7f38f4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f390be00640 1 --2- 172.21.15.161:0/313213708 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f390c15cfa0 0x7f38f4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f390be00640 1 -- 172.21.15.161:0/313213708 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c0b4a00 con 0x7f38f40047d0 2024-05-22T21:20:18.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f390be00640 1 -- 172.21.15.161:0/313213708 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c15db50 con 0x7f38f4003980 2024-05-22T21:20:18.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f390be00640 1 -- 172.21.15.161:0/313213708 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f400b990 con 0x7f390c15cfa0 2024-05-22T21:20:18.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.722+0000 7f3910e00640 1 --2- 172.21.15.161:0/313213708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38f40047d0 0x7f38f4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f3912200640 1 --2- 172.21.15.161:0/313213708 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f390c15cfa0 0x7f38f4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f3911800640 1 --2- 172.21.15.161:0/313213708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38f4003980 0x7f38f4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f3910e00640 1 -- 172.21.15.161:0/313213708 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f390c15cfa0 msgr2=0x7f38f4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:18.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f3910e00640 1 --2- 172.21.15.161:0/313213708 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f390c15cfa0 0x7f38f4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f3910e00640 1 -- 172.21.15.161:0/313213708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38f4003980 msgr2=0x7f38f4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:18.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f3910e00640 1 --2- 172.21.15.161:0/313213708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38f4003980 0x7f38f4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f3910e00640 1 -- 172.21.15.161:0/313213708 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f4003530 con 0x7f38f40047d0 2024-05-22T21:20:18.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f3911800640 1 --2- 172.21.15.161:0/313213708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38f4003980 0x7f38f4003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:18.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f3910e00640 1 --2- 172.21.15.161:0/313213708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38f40047d0 0x7f38f4004c50 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009ab0 tx=0x7f38f801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae8f0e1fa9aab2d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:18.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f390be00640 1 -- 172.21.15.161:0/313213708 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f80a64a0 con 0x7f38f40047d0 2024-05-22T21:20:18.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f390be00640 1 -- 172.21.15.161:0/313213708 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f80a6600 con 0x7f38f40047d0 2024-05-22T21:20:18.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f390be00640 1 -- 172.21.15.161:0/313213708 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f80a69b0 con 0x7f38f40047d0 2024-05-22T21:20:18.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f3914200640 1 -- 172.21.15.161:0/313213708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38f40047d0 msgr2=0x7f38f4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f3914200640 1 --2- 172.21.15.161:0/313213708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38f40047d0 0x7f38f4004c50 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009ab0 tx=0x7f38f801ca90 comp rx=0 tx=0).stop 2024-05-22T21:20:18.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f3912200640 1 -- 172.21.15.161:0/313213708 reap_dead start 2024-05-22T21:20:18.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f3914200640 1 -- 172.21.15.161:0/313213708 shutdown_connections 2024-05-22T21:20:18.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.726+0000 7f3914200640 1 -- 172.21.15.161:0/313213708 >> 172.21.15.161:0/313213708 conn(0x7f390c0b1bd0 msgr2=0x7f390c0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:18.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3914200640 1 -- 172.21.15.161:0/313213708 shutdown_connections 2024-05-22T21:20:18.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3914200640 1 -- 172.21.15.161:0/313213708 wait complete. 2024-05-22T21:20:18.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3914200640 1 Processor -- start 2024-05-22T21:20:18.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3914200640 1 -- start start 2024-05-22T21:20:18.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3914200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38f40047d0 0x7f390c1505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3914200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f390c15ed70 0x7f390c1519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3914200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f390c150d70 0x7f390c151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3914200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c15fe30 con 0x7f390c15ed70 2024-05-22T21:20:18.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3914200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c15f2b0 con 0x7f390c150d70 2024-05-22T21:20:18.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3914200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390c15f590 con 0x7f38f40047d0 2024-05-22T21:20:18.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3910e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f390c15ed70 0x7f390c1519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3910e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f390c15ed70 0x7f390c1519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45178/0 (socket says 172.21.15.161:45178) 2024-05-22T21:20:18.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3910e00640 1 -- 172.21.15.161:0/463017080 learned_addr learned my addr 172.21.15.161:0/463017080 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:18.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3911800640 1 --2- 172.21.15.161:0/463017080 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38f40047d0 0x7f390c1505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3912200640 1 --2- 172.21.15.161:0/463017080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f390c150d70 0x7f390c151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3910e00640 1 -- 172.21.15.161:0/463017080 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38f40047d0 msgr2=0x7f390c1505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3910e00640 1 --2- 172.21.15.161:0/463017080 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38f40047d0 0x7f390c1505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3910e00640 1 -- 172.21.15.161:0/463017080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f390c150d70 msgr2=0x7f390c151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3910e00640 1 --2- 172.21.15.161:0/463017080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f390c150d70 0x7f390c151f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3910e00640 1 -- 172.21.15.161:0/463017080 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c15f870 con 0x7f390c15ed70 2024-05-22T21:20:18.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3912200640 1 --2- 172.21.15.161:0/463017080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f390c150d70 0x7f390c151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:18.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3910e00640 1 --2- 172.21.15.161:0/463017080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f390c15ed70 0x7f390c1519d0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009ab0 tx=0x7f38f4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:18.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3909600640 1 -- 172.21.15.161:0/463017080 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f80a7680 con 0x7f390c15ed70 2024-05-22T21:20:18.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3909600640 1 -- 172.21.15.161:0/463017080 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f80a77e0 con 0x7f390c15ed70 2024-05-22T21:20:18.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3909600640 1 -- 172.21.15.161:0/463017080 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f80a7b70 con 0x7f390c15ed70 2024-05-22T21:20:18.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.730+0000 7f3914200640 1 -- 172.21.15.161:0/463017080 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390c15fb50 con 0x7f390c15ed70 2024-05-22T21:20:18.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.734+0000 7f3914200640 1 -- 172.21.15.161:0/463017080 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390c151410 con 0x7f390c15ed70 2024-05-22T21:20:18.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.734+0000 7f3914200640 1 -- 172.21.15.161:0/463017080 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390c151600 con 0x7f390c15ed70 2024-05-22T21:20:18.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.734+0000 7f3909600640 1 -- 172.21.15.161:0/463017080 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f38f80a7cd0 con 0x7f390c15ed70 2024-05-22T21:20:18.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.734+0000 7f3909600640 1 --2- 172.21.15.161:0/463017080 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f38ec0666c0 0x7f38ec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:18.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.734+0000 7f3909600640 1 -- 172.21.15.161:0/463017080 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f38f811eaf0 con 0x7f390c15ed70 2024-05-22T21:20:18.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.734+0000 7f3911800640 1 --2- 172.21.15.161:0/463017080 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f38ec0666c0 0x7f38ec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:18.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.734+0000 7f3911800640 1 --2- 172.21.15.161:0/463017080 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f38ec0666c0 0x7f38ec068b80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3900009a80 tx=0x7f3900008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:18.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.738+0000 7f3909600640 1 -- 172.21.15.161:0/463017080 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f390c151600 con 0x7f390c15ed70 2024-05-22T21:20:18.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.890+0000 7f3914200640 1 -- 172.21.15.161:0/463017080 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f390c151600 con 0x7f390c15ed70 2024-05-22T21:20:18.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.890+0000 7f3909600640 1 -- 172.21.15.161:0/463017080 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) v1 ==== 104+0+11 (secure 0 0 0) 0x7f390c151600 con 0x7f390c15ed70 2024-05-22T21:20:18.895 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:20:18.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.894+0000 7f3914200640 1 -- 172.21.15.161:0/463017080 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f38ec0666c0 msgr2=0x7f38ec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.894+0000 7f3914200640 1 --2- 172.21.15.161:0/463017080 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f38ec0666c0 0x7f38ec068b80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3900009a80 tx=0x7f3900008040 comp rx=0 tx=0).stop 2024-05-22T21:20:18.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.894+0000 7f3914200640 1 -- 172.21.15.161:0/463017080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f390c15ed70 msgr2=0x7f390c1519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:18.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.894+0000 7f3914200640 1 --2- 172.21.15.161:0/463017080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f390c15ed70 0x7f390c1519d0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009ab0 tx=0x7f38f4005800 comp rx=0 tx=0).stop 2024-05-22T21:20:18.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.894+0000 7f3914200640 1 -- 172.21.15.161:0/463017080 shutdown_connections 2024-05-22T21:20:18.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.894+0000 7f3914200640 1 --2- 172.21.15.161:0/463017080 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f38ec0666c0 0x7f38ec068b80 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.894+0000 7f3914200640 1 --2- 172.21.15.161:0/463017080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f390c150d70 0x7f390c151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.894+0000 7f3914200640 1 --2- 172.21.15.161:0/463017080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f390c15ed70 0x7f390c1519d0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.894+0000 7f3914200640 1 --2- 172.21.15.161:0/463017080 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38f40047d0 0x7f390c1505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:18.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.894+0000 7f3914200640 1 -- 172.21.15.161:0/463017080 >> 172.21.15.161:0/463017080 conn(0x7f390c0b1bd0 msgr2=0x7f390c160580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:18.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.894+0000 7f3914200640 1 -- 172.21.15.161:0/463017080 shutdown_connections 2024-05-22T21:20:18.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:18.894+0000 7f3914200640 1 -- 172.21.15.161:0/463017080 wait complete. 2024-05-22T21:20:18.916 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:20:19.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.050+0000 7f0bc4e00640 1 Processor -- start 2024-05-22T21:20:19.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.050+0000 7f0bc4e00640 1 -- start start 2024-05-22T21:20:19.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.054+0000 7f0bc4e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0bc013cfc0 0x7f0bc01402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.054+0000 7f0bc4e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0bc013fe50 0x7f0bc006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.054+0000 7f0bc4e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc013d3a0 con 0x7f0bc015af60 2024-05-22T21:20:19.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.054+0000 7f0bc4e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc00cd920 con 0x7f0bc013cfc0 2024-05-22T21:20:19.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.054+0000 7f0bc4e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc00cdc00 con 0x7f0bc013fe50 2024-05-22T21:20:19.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.054+0000 7f0bbde00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0bc013cfc0 0x7f0bc01402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.054+0000 7f0bbde00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0bc013cfc0 0x7f0bc01402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45186/0 (socket says 172.21.15.161:45186) 2024-05-22T21:20:19.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.054+0000 7f0bbde00640 1 -- 172.21.15.161:0/1714190625 learned_addr learned my addr 172.21.15.161:0/1714190625 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:19.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.054+0000 7f0bbe800640 1 --2- 172.21.15.161:0/1714190625 >> v2:172.21.15.179:3300/0 conn(0x7f0bc013fe50 0x7f0bc006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.054+0000 7f0bbde00640 1 -- 172.21.15.161:0/1714190625 >> v2:172.21.15.179:3300/0 conn(0x7f0bc013fe50 msgr2=0x7f0bc006fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:19.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.054+0000 7f0bbde00640 1 --2- 172.21.15.161:0/1714190625 >> v2:172.21.15.179:3300/0 conn(0x7f0bc013fe50 0x7f0bc006fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.054+0000 7f0bbde00640 1 -- 172.21.15.161:0/1714190625 >> v1:172.21.15.179:6789/0 conn(0x7f0bc015af60 legacy=0x7f0bc013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.054+0000 7f0bbde00640 1 -- 172.21.15.161:0/1714190625 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc00cdee0 con 0x7f0bc013cfc0 2024-05-22T21:20:19.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bbde00640 1 --2- 172.21.15.161:0/1714190625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bc013cfc0 0x7f0bc01402e0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f0bc006c5e0 tx=0x7f0bb001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26309985e89c9d35 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:19.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bbd400640 1 -- 172.21.15.161:0/1714190625 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00ab600 con 0x7f0bc013cfc0 2024-05-22T21:20:19.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bbd400640 1 -- 172.21.15.161:0/1714190625 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb00a7050 con 0x7f0bc013cfc0 2024-05-22T21:20:19.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bbd400640 1 -- 172.21.15.161:0/1714190625 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00ab980 con 0x7f0bc013cfc0 2024-05-22T21:20:19.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bc4e00640 1 -- 172.21.15.161:0/1714190625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bc013cfc0 msgr2=0x7f0bc01402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bc4e00640 1 --2- 172.21.15.161:0/1714190625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bc013cfc0 0x7f0bc01402e0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f0bc006c5e0 tx=0x7f0bb001c460 comp rx=0 tx=0).stop 2024-05-22T21:20:19.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bc4e00640 1 -- 172.21.15.161:0/1714190625 shutdown_connections 2024-05-22T21:20:19.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bc4e00640 1 --2- 172.21.15.161:0/1714190625 >> v2:172.21.15.179:3300/0 conn(0x7f0bc013fe50 0x7f0bc006fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bc4e00640 1 --2- 172.21.15.161:0/1714190625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bc013cfc0 0x7f0bc01402e0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bc4e00640 1 -- 172.21.15.161:0/1714190625 >> 172.21.15.161:0/1714190625 conn(0x7f0bc006a2d0 msgr2=0x7f0bc006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:19.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bc4e00640 1 -- 172.21.15.161:0/1714190625 shutdown_connections 2024-05-22T21:20:19.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bc4e00640 1 -- 172.21.15.161:0/1714190625 wait complete. 2024-05-22T21:20:19.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bc4e00640 1 Processor -- start 2024-05-22T21:20:19.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bc4e00640 1 -- start start 2024-05-22T21:20:19.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bc4e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bc013cfc0 0x7f0bc0141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bc4e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bc013fe50 0x7f0bc0156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bc4e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bc015af60 0x7f0bc00b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bc4e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc0139c70 con 0x7f0bc013cfc0 2024-05-22T21:20:19.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bc4e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc0138ff0 con 0x7f0bc015af60 2024-05-22T21:20:19.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bc4e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc01392d0 con 0x7f0bc013fe50 2024-05-22T21:20:19.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bbde00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bc013cfc0 0x7f0bc0141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bbde00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bc013cfc0 0x7f0bc0141fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45202/0 (socket says 172.21.15.161:45202) 2024-05-22T21:20:19.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.058+0000 7f0bbde00640 1 -- 172.21.15.161:0/1317599906 learned_addr learned my addr 172.21.15.161:0/1317599906 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:19.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bbde00640 1 -- 172.21.15.161:0/1317599906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bc013fe50 msgr2=0x7f0bc0156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bbe800640 1 --2- 172.21.15.161:0/1317599906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bc015af60 0x7f0bc00b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bbde00640 1 --2- 172.21.15.161:0/1317599906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bc013fe50 0x7f0bc0156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bbde00640 1 -- 172.21.15.161:0/1317599906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bc015af60 msgr2=0x7f0bc00b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bbde00640 1 --2- 172.21.15.161:0/1317599906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bc015af60 0x7f0bc00b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bbde00640 1 -- 172.21.15.161:0/1317599906 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb00094d0 con 0x7f0bc013cfc0 2024-05-22T21:20:19.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bbe800640 1 --2- 172.21.15.161:0/1317599906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bc015af60 0x7f0bc00b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:19.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bbde00640 1 --2- 172.21.15.161:0/1317599906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bc013cfc0 0x7f0bc0141fd0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0019070 tx=0x7f0bb0019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:19.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bb6a00640 1 -- 172.21.15.161:0/1317599906 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00b37d0 con 0x7f0bc013cfc0 2024-05-22T21:20:19.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bb6a00640 1 -- 172.21.15.161:0/1317599906 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb00b3930 con 0x7f0bc013cfc0 2024-05-22T21:20:19.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bb6a00640 1 -- 172.21.15.161:0/1317599906 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00b3cc0 con 0x7f0bc013cfc0 2024-05-22T21:20:19.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bc4e00640 1 -- 172.21.15.161:0/1317599906 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc01395b0 con 0x7f0bc013cfc0 2024-05-22T21:20:19.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bc4e00640 1 -- 172.21.15.161:0/1317599906 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc0138d00 con 0x7f0bc013cfc0 2024-05-22T21:20:19.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bc4e00640 1 -- 172.21.15.161:0/1317599906 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc0139f90 con 0x7f0bc013cfc0 2024-05-22T21:20:19.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bb6a00640 1 -- 172.21.15.161:0/1317599906 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0bb00b2070 con 0x7f0bc013cfc0 2024-05-22T21:20:19.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bb6a00640 1 --2- 172.21.15.161:0/1317599906 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b9c066690 0x7f0b9c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bb6a00640 1 -- 172.21.15.161:0/1317599906 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0bb011e620 con 0x7f0bc013cfc0 2024-05-22T21:20:19.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.062+0000 7f0bb5400640 1 --2- 172.21.15.161:0/1317599906 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b9c066690 0x7f0b9c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.066+0000 7f0bb5400640 1 --2- 172.21.15.161:0/1317599906 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b9c066690 0x7f0b9c068b50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8000f70 tx=0x7f0ba80079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:19.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.066+0000 7f0bb6a00640 1 -- 172.21.15.161:0/1317599906 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0bb00eb8b0 con 0x7f0bc013cfc0 2024-05-22T21:20:19.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.222+0000 7f0bc4e00640 1 -- 172.21.15.161:0/1317599906 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0bc0067220 con 0x7f0bc013cfc0 2024-05-22T21:20:19.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.222+0000 7f0bb6a00640 1 -- 172.21.15.161:0/1317599906 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0bb00eeed0 con 0x7f0bc013cfc0 2024-05-22T21:20:19.227 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:20:19.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.226+0000 7f0bc4e00640 1 -- 172.21.15.161:0/1317599906 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b9c066690 msgr2=0x7f0b9c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.226+0000 7f0bc4e00640 1 --2- 172.21.15.161:0/1317599906 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b9c066690 0x7f0b9c068b50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8000f70 tx=0x7f0ba80079c0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.226+0000 7f0bc4e00640 1 -- 172.21.15.161:0/1317599906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bc013cfc0 msgr2=0x7f0bc0141fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.226+0000 7f0bc4e00640 1 --2- 172.21.15.161:0/1317599906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bc013cfc0 0x7f0bc0141fd0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0019070 tx=0x7f0bb0019200 comp rx=0 tx=0).stop 2024-05-22T21:20:19.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.226+0000 7f0bc4e00640 1 -- 172.21.15.161:0/1317599906 shutdown_connections 2024-05-22T21:20:19.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.226+0000 7f0bc4e00640 1 --2- 172.21.15.161:0/1317599906 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0b9c066690 0x7f0b9c068b50 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.226+0000 7f0bc4e00640 1 --2- 172.21.15.161:0/1317599906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bc015af60 0x7f0bc00b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.226+0000 7f0bc4e00640 1 --2- 172.21.15.161:0/1317599906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bc013fe50 0x7f0bc0156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.226+0000 7f0bc4e00640 1 --2- 172.21.15.161:0/1317599906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bc013cfc0 0x7f0bc0141fd0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.226+0000 7f0bc4e00640 1 -- 172.21.15.161:0/1317599906 >> 172.21.15.161:0/1317599906 conn(0x7f0bc006a2d0 msgr2=0x7f0bc00b0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:19.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.226+0000 7f0bc4e00640 1 -- 172.21.15.161:0/1317599906 shutdown_connections 2024-05-22T21:20:19.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.226+0000 7f0bc4e00640 1 -- 172.21.15.161:0/1317599906 wait complete. 2024-05-22T21:20:19.246 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:20:19.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.378+0000 7f93c0800640 1 Processor -- start 2024-05-22T21:20:19.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.378+0000 7f93c0800640 1 -- start start 2024-05-22T21:20:19.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93c0800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f93b815b600 0x7f93b806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93c0800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b806cc40 con 0x7f93b8143c40 2024-05-22T21:20:19.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93c0800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b8144bb0 con 0x7f93b806c7e0 2024-05-22T21:20:19.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93c0800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b815ba60 con 0x7f93b815b600 2024-05-22T21:20:19.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bde00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f93b8143c40 0x7f93b81444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51222/0 (socket says 172.21.15.161:51222) 2024-05-22T21:20:19.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bde00640 1 -- 172.21.15.161:0/21601897 learned_addr learned my addr 172.21.15.161:0/21601897 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:19.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bd400640 1 --2- 172.21.15.161:0/21601897 >> v2:172.21.15.179:3300/0 conn(0x7f93b815b600 0x7f93b806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bca00640 1 -- 172.21.15.161:0/21601897 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292948241 0 0) 0x7f93b806cc40 con 0x7f93b8143c40 2024-05-22T21:20:19.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bca00640 1 -- 172.21.15.161:0/21601897 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93a0003530 con 0x7f93b8143c40 2024-05-22T21:20:19.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bca00640 1 -- 172.21.15.161:0/21601897 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1712455315 0 0) 0x7f93a0003530 con 0x7f93b8143c40 2024-05-22T21:20:19.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bca00640 1 -- 172.21.15.161:0/21601897 >> v2:172.21.15.179:3300/0 conn(0x7f93b815b600 msgr2=0x7f93b806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:19.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bca00640 1 --2- 172.21.15.161:0/21601897 >> v2:172.21.15.179:3300/0 conn(0x7f93b815b600 0x7f93b806c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bca00640 1 -- 172.21.15.161:0/21601897 >> v1:172.21.15.179:6789/0 conn(0x7f93b806c7e0 legacy=0x7f93b80b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bca00640 1 -- 172.21.15.161:0/21601897 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b815efb0 con 0x7f93b8143c40 2024-05-22T21:20:19.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bca00640 1 -- 172.21.15.161:0/21601897 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f93ac001680 con 0x7f93b8143c40 2024-05-22T21:20:19.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bca00640 1 -- 172.21.15.161:0/21601897 >> v1:172.21.15.161:6789/0 conn(0x7f93b8143c40 legacy=0x7f93b81444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bca00640 1 --2- 172.21.15.161:0/21601897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a0003980 0x7f93a0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bca00640 1 --2- 172.21.15.161:0/21601897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93a00047d0 0x7f93a0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bca00640 1 --2- 172.21.15.161:0/21601897 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93a0005190 0x7f93a000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bca00640 1 -- 172.21.15.161:0/21601897 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b815ba60 con 0x7f93a0003980 2024-05-22T21:20:19.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bca00640 1 -- 172.21.15.161:0/21601897 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a000c030 con 0x7f93a00047d0 2024-05-22T21:20:19.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bca00640 1 -- 172.21.15.161:0/21601897 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a000c1f0 con 0x7f93a0005190 2024-05-22T21:20:19.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bde00640 1 --2- 172.21.15.161:0/21601897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a0003980 0x7f93a0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93be800640 1 --2- 172.21.15.161:0/21601897 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93a0005190 0x7f93a000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bd400640 1 --2- 172.21.15.161:0/21601897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93a00047d0 0x7f93a0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bde00640 1 -- 172.21.15.161:0/21601897 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93a0005190 msgr2=0x7f93a000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:19.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bde00640 1 --2- 172.21.15.161:0/21601897 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93a0005190 0x7f93a000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bde00640 1 -- 172.21.15.161:0/21601897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93a00047d0 msgr2=0x7f93a0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:19.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bde00640 1 --2- 172.21.15.161:0/21601897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93a00047d0 0x7f93a0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93bde00640 1 -- 172.21.15.161:0/21601897 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a0003530 con 0x7f93a0003980 2024-05-22T21:20:19.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93be800640 1 --2- 172.21.15.161:0/21601897 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93a0005190 0x7f93a000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:19.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.382+0000 7f93be800640 1 -- 172.21.15.161:0/21601897 reap_dead start 2024-05-22T21:20:19.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.386+0000 7f93bde00640 1 --2- 172.21.15.161:0/21601897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a0003980 0x7f93a0003da0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00c240 tx=0x7f93ac01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db2872290c3703d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:19.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.386+0000 7f93bca00640 1 -- 172.21.15.161:0/21601897 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac005ae0 con 0x7f93a0003980 2024-05-22T21:20:19.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.386+0000 7f93bca00640 1 -- 172.21.15.161:0/21601897 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ac005c40 con 0x7f93a0003980 2024-05-22T21:20:19.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.386+0000 7f93bca00640 1 -- 172.21.15.161:0/21601897 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac0a5a60 con 0x7f93a0003980 2024-05-22T21:20:19.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.386+0000 7f93c0800640 1 -- 172.21.15.161:0/21601897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a0003980 msgr2=0x7f93a0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.386+0000 7f93c0800640 1 --2- 172.21.15.161:0/21601897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a0003980 0x7f93a0003da0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00c240 tx=0x7f93ac01cc60 comp rx=0 tx=0).stop 2024-05-22T21:20:19.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.386+0000 7f93c0800640 1 -- 172.21.15.161:0/21601897 shutdown_connections 2024-05-22T21:20:19.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.386+0000 7f93c0800640 1 --2- 172.21.15.161:0/21601897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a0003980 0x7f93a0003da0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.386+0000 7f93c0800640 1 -- 172.21.15.161:0/21601897 >> 172.21.15.161:0/21601897 conn(0x7f93b806a2d0 msgr2=0x7f93b806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:19.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.386+0000 7f93c0800640 1 -- 172.21.15.161:0/21601897 shutdown_connections 2024-05-22T21:20:19.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.386+0000 7f93c0800640 1 -- 172.21.15.161:0/21601897 wait complete. 2024-05-22T21:20:19.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.386+0000 7f93c0800640 1 Processor -- start 2024-05-22T21:20:19.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.386+0000 7f93c0800640 1 -- start start 2024-05-22T21:20:19.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93c0800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93a0003980 0x7f93b81fa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93c0800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93b81fa950 0x7f93b81e8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93c0800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b81fadd0 0x7f93b81e93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93c0800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b81e7e80 con 0x7f93b81fadd0 2024-05-22T21:20:19.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93c0800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b81e80f0 con 0x7f93a0003980 2024-05-22T21:20:19.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93c0800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b81e83d0 con 0x7f93b81fa950 2024-05-22T21:20:19.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93be800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b81fadd0 0x7f93b81e93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93be800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b81fadd0 0x7f93b81e93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45234/0 (socket says 172.21.15.161:45234) 2024-05-22T21:20:19.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93be800640 1 -- 172.21.15.161:0/413706489 learned_addr learned my addr 172.21.15.161:0/413706489 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:19.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93bde00640 1 --2- 172.21.15.161:0/413706489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93a0003980 0x7f93b81fa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93bd400640 1 --2- 172.21.15.161:0/413706489 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93b81fa950 0x7f93b81e8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93be800640 1 -- 172.21.15.161:0/413706489 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93b81fa950 msgr2=0x7f93b81e8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93be800640 1 --2- 172.21.15.161:0/413706489 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93b81fa950 0x7f93b81e8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93be800640 1 -- 172.21.15.161:0/413706489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93a0003980 msgr2=0x7f93b81fa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93be800640 1 --2- 172.21.15.161:0/413706489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93a0003980 0x7f93b81fa410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93be800640 1 -- 172.21.15.161:0/413706489 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b81e86b0 con 0x7f93b81fadd0 2024-05-22T21:20:19.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93be800640 1 --2- 172.21.15.161:0/413706489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b81fadd0 0x7f93b81e93d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f93b80b4560 tx=0x7f93b400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:19.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93b2000640 1 -- 172.21.15.161:0/413706489 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b400eed0 con 0x7f93b81fadd0 2024-05-22T21:20:19.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93b2000640 1 -- 172.21.15.161:0/413706489 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b4003980 con 0x7f93b81fadd0 2024-05-22T21:20:19.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93c0800640 1 -- 172.21.15.161:0/413706489 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b81e8990 con 0x7f93b81fadd0 2024-05-22T21:20:19.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93b2000640 1 -- 172.21.15.161:0/413706489 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b4003d30 con 0x7f93b81fadd0 2024-05-22T21:20:19.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93c0800640 1 -- 172.21.15.161:0/413706489 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b81e6f00 con 0x7f93b81fadd0 2024-05-22T21:20:19.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93c0800640 1 -- 172.21.15.161:0/413706489 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b8067220 con 0x7f93b81fadd0 2024-05-22T21:20:19.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.390+0000 7f93b2000640 1 -- 172.21.15.161:0/413706489 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f93b40050d0 con 0x7f93b81fadd0 2024-05-22T21:20:19.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.394+0000 7f93b2000640 1 --2- 172.21.15.161:0/413706489 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f939c066690 0x7f939c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.394+0000 7f93b2000640 1 -- 172.21.15.161:0/413706489 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f93b4083c00 con 0x7f93b81fadd0 2024-05-22T21:20:19.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.394+0000 7f93bde00640 1 --2- 172.21.15.161:0/413706489 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f939c066690 0x7f939c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.394+0000 7f93bde00640 1 --2- 172.21.15.161:0/413706489 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f939c066690 0x7f939c068b50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0ab000 tx=0x7f93ac0b1040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:19.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.398+0000 7f93b2000640 1 -- 172.21.15.161:0/413706489 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f93b4050f40 con 0x7f93b81fadd0 2024-05-22T21:20:19.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.570+0000 7f93c0800640 1 -- 172.21.15.161:0/413706489 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f93b815b3b0 con 0x7f93b81fadd0 2024-05-22T21:20:19.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.570+0000 7f93b2000640 1 -- 172.21.15.161:0/413706489 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) v1 ==== 80+0+0 (secure 0 0 0) 0x7f93b4054560 con 0x7f93b81fadd0 2024-05-22T21:20:19.573 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:20:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.570+0000 7f93c0800640 1 -- 172.21.15.161:0/413706489 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f939c066690 msgr2=0x7f939c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.570+0000 7f93c0800640 1 --2- 172.21.15.161:0/413706489 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f939c066690 0x7f939c068b50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0ab000 tx=0x7f93ac0b1040 comp rx=0 tx=0).stop 2024-05-22T21:20:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.570+0000 7f93c0800640 1 -- 172.21.15.161:0/413706489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b81fadd0 msgr2=0x7f93b81e93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.570+0000 7f93c0800640 1 --2- 172.21.15.161:0/413706489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b81fadd0 0x7f93b81e93d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f93b80b4560 tx=0x7f93b400eb90 comp rx=0 tx=0).stop 2024-05-22T21:20:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.570+0000 7f93c0800640 1 -- 172.21.15.161:0/413706489 shutdown_connections 2024-05-22T21:20:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.570+0000 7f93c0800640 1 --2- 172.21.15.161:0/413706489 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f939c066690 0x7f939c068b50 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.570+0000 7f93c0800640 1 --2- 172.21.15.161:0/413706489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b81fadd0 0x7f93b81e93d0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.570+0000 7f93c0800640 1 --2- 172.21.15.161:0/413706489 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93b81fa950 0x7f93b81e8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.570+0000 7f93c0800640 1 --2- 172.21.15.161:0/413706489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93a0003980 0x7f93b81fa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.570+0000 7f93c0800640 1 -- 172.21.15.161:0/413706489 >> 172.21.15.161:0/413706489 conn(0x7f93b806a2d0 msgr2=0x7f93b80cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:19.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.570+0000 7f93c0800640 1 -- 172.21.15.161:0/413706489 shutdown_connections 2024-05-22T21:20:19.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.570+0000 7f93c0800640 1 -- 172.21.15.161:0/413706489 wait complete. 2024-05-22T21:20:19.594 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-22T21:20:19.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:19.594 DEBUG:teuthology.orchestra.run.smithi179:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-22T21:20:19.622 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:20:19.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:19.623 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-22T21:20:19.673 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:20:19.673 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:20:19.682 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:20:19.682 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:20:19.682 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:20:19.724 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:20:19.724 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:20:19.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.854+0000 7f1f74e00640 1 Processor -- start 2024-05-22T21:20:19.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.854+0000 7f1f74e00640 1 -- start start 2024-05-22T21:20:19.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.858+0000 7f1f74e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f70143ec0 0x7f1f7015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.858+0000 7f1f74e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1f700715a0 0x7f1f7006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.858+0000 7f1f74e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f701442a0 con 0x7f1f7015ec90 2024-05-22T21:20:19.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.858+0000 7f1f74e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f70070260 con 0x7f1f700715a0 2024-05-22T21:20:19.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.858+0000 7f1f74e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f70070500 con 0x7f1f70143ec0 2024-05-22T21:20:19.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.858+0000 7f1f6de00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f70143ec0 0x7f1f7015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.858+0000 7f1f6d400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1f700715a0 0x7f1f7006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.858+0000 7f1f6de00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f70143ec0 0x7f1f7015f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:59060/0 (socket says 172.21.15.161:59060) 2024-05-22T21:20:19.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.858+0000 7f1f6de00640 1 -- 172.21.15.161:0/3469156619 learned_addr learned my addr 172.21.15.161:0/3469156619 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:19.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.858+0000 7f1f6e800640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f1f7015ec90 0x7f1f7006bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:52710/0 (socket says 172.21.15.161:52710) 2024-05-22T21:20:19.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.858+0000 7f1f6d400640 1 -- 172.21.15.161:0/3469156619 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f70143ec0 msgr2=0x7f1f7015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:19.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.858+0000 7f1f6d400640 1 --2- 172.21.15.161:0/3469156619 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f70143ec0 0x7f1f7015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.858+0000 7f1f6ca00640 1 -- 172.21.15.161:0/3469156619 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3762228014 0 0) 0x7f1f701442a0 con 0x7f1f7015ec90 2024-05-22T21:20:19.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.858+0000 7f1f6d400640 1 -- 172.21.15.161:0/3469156619 >> v1:172.21.15.179:6789/0 conn(0x7f1f7015ec90 legacy=0x7f1f7006bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.858+0000 7f1f6d400640 1 -- 172.21.15.161:0/3469156619 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f700707e0 con 0x7f1f700715a0 2024-05-22T21:20:19.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.858+0000 7f1f6de00640 1 --2- 172.21.15.161:0/3469156619 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f70143ec0 0x7f1f7015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:20:19.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.862+0000 7f1f6d400640 1 --2- 172.21.15.161:0/3469156619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f700715a0 0x7f1f7006fd20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0097c0 tx=0x7f1f5c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f8d3d88c5523ccd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:19.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.862+0000 7f1f6ca00640 1 -- 172.21.15.161:0/3469156619 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0ab670 con 0x7f1f700715a0 2024-05-22T21:20:19.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.862+0000 7f1f6ca00640 1 -- 172.21.15.161:0/3469156619 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c0ab7d0 con 0x7f1f700715a0 2024-05-22T21:20:19.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.862+0000 7f1f6ca00640 1 -- 172.21.15.161:0/3469156619 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0abb60 con 0x7f1f700715a0 2024-05-22T21:20:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.862+0000 7f1f74e00640 1 -- 172.21.15.161:0/3469156619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f700715a0 msgr2=0x7f1f7006fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.862+0000 7f1f74e00640 1 --2- 172.21.15.161:0/3469156619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f700715a0 0x7f1f7006fd20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0097c0 tx=0x7f1f5c01c470 comp rx=0 tx=0).stop 2024-05-22T21:20:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.862+0000 7f1f74e00640 1 -- 172.21.15.161:0/3469156619 shutdown_connections 2024-05-22T21:20:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.862+0000 7f1f74e00640 1 --2- 172.21.15.161:0/3469156619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f700715a0 0x7f1f7006fd20 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.862+0000 7f1f74e00640 1 --2- 172.21.15.161:0/3469156619 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f70143ec0 0x7f1f7015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.862+0000 7f1f74e00640 1 -- 172.21.15.161:0/3469156619 >> 172.21.15.161:0/3469156619 conn(0x7f1f700b0bc0 msgr2=0x7f1f700b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.862+0000 7f1f74e00640 1 -- 172.21.15.161:0/3469156619 shutdown_connections 2024-05-22T21:20:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.862+0000 7f1f74e00640 1 -- 172.21.15.161:0/3469156619 wait complete. 2024-05-22T21:20:19.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.862+0000 7f1f74e00640 1 Processor -- start 2024-05-22T21:20:19.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f74e00640 1 -- start start 2024-05-22T21:20:19.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f74e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f70143ec0 0x7f1f700b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f74e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f7015ec90 0x7f1f700adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f74e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f70137750 0x7f1f70137c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f74e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f701524f0 con 0x7f1f70143ec0 2024-05-22T21:20:19.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f74e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f70138140 con 0x7f1f70137750 2024-05-22T21:20:19.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f74e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f700aea90 con 0x7f1f7015ec90 2024-05-22T21:20:19.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f6de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f70143ec0 0x7f1f700b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f6de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f70143ec0 0x7f1f700b0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45236/0 (socket says 172.21.15.161:45236) 2024-05-22T21:20:19.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f6de00640 1 -- 172.21.15.161:0/1144326583 learned_addr learned my addr 172.21.15.161:0/1144326583 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:19.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f6d400640 1 --2- 172.21.15.161:0/1144326583 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f7015ec90 0x7f1f700adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f6e800640 1 --2- 172.21.15.161:0/1144326583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f70137750 0x7f1f70137c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f6de00640 1 -- 172.21.15.161:0/1144326583 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f7015ec90 msgr2=0x7f1f700adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f6de00640 1 --2- 172.21.15.161:0/1144326583 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f7015ec90 0x7f1f700adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f6de00640 1 -- 172.21.15.161:0/1144326583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f70137750 msgr2=0x7f1f70137c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:19.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f6de00640 1 --2- 172.21.15.161:0/1144326583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f70137750 0x7f1f70137c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f6de00640 1 -- 172.21.15.161:0/1144326583 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f60009b30 con 0x7f1f70143ec0 2024-05-22T21:20:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f6e800640 1 --2- 172.21.15.161:0/1144326583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f70137750 0x7f1f70137c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f6d400640 1 --2- 172.21.15.161:0/1144326583 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f7015ec90 0x7f1f700adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f6de00640 1 --2- 172.21.15.161:0/1144326583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f70143ec0 0x7f1f700b0380 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000efc0 tx=0x7f1f6000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f66000640 1 -- 172.21.15.161:0/1144326583 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60014480 con 0x7f1f70143ec0 2024-05-22T21:20:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f66000640 1 -- 172.21.15.161:0/1144326583 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6000f040 con 0x7f1f70143ec0 2024-05-22T21:20:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f74e00640 1 -- 172.21.15.161:0/1144326583 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c009420 con 0x7f1f70143ec0 2024-05-22T21:20:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f66000640 1 -- 172.21.15.161:0/1144326583 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f600147e0 con 0x7f1f70143ec0 2024-05-22T21:20:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f74e00640 1 -- 172.21.15.161:0/1144326583 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f70151e30 con 0x7f1f70143ec0 2024-05-22T21:20:19.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.866+0000 7f1f74e00640 1 -- 172.21.15.161:0/1144326583 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f70067220 con 0x7f1f70143ec0 2024-05-22T21:20:19.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.870+0000 7f1f66000640 1 -- 172.21.15.161:0/1144326583 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1f60014940 con 0x7f1f70143ec0 2024-05-22T21:20:19.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.870+0000 7f1f66000640 1 --2- 172.21.15.161:0/1144326583 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f500666c0 0x7f1f50068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:19.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.870+0000 7f1f66000640 1 -- 172.21.15.161:0/1144326583 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1f60084aa0 con 0x7f1f70143ec0 2024-05-22T21:20:19.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.870+0000 7f1f6d400640 1 --2- 172.21.15.161:0/1144326583 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f500666c0 0x7f1f50068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:19.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.870+0000 7f1f6d400640 1 --2- 172.21.15.161:0/1144326583 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f500666c0 0x7f1f50068b80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0af000 tx=0x7f1f5c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:19.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:19.874+0000 7f1f66000640 1 -- 172.21.15.161:0/1144326583 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1f60051dd0 con 0x7f1f70143ec0 2024-05-22T21:20:20.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.058+0000 7f1f74e00640 1 -- 172.21.15.161:0/1144326583 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f7006dcc0 con 0x7f1f70143ec0 2024-05-22T21:20:20.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.058+0000 7f1f66000640 1 -- 172.21.15.161:0/1144326583 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4745 (secure 0 0 0) 0x7f1f600553f0 con 0x7f1f70143ec0 2024-05-22T21:20:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:20.062 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/524372753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":524372753},{"type":"v1","addr":"172.21.15.161:6836","nonce":524372753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3762119379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3762119379},{"type":"v1","addr":"172.21.15.179:6833","nonce":3762119379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3087241366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3087241366},{"type":"v1","addr":"172.21.15.179:6835","nonce":3087241366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:46.384984+0000","modified":"2024-05-22T21:19:49.400019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7572":{"gid":7572,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/3945715002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3945715002},{"type":"v1","addr":"172.21.15.161:6837","nonce":3945715002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-22T21:20:20.062 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 62 2024-05-22T21:20:20.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.058+0000 7f1f74e00640 1 -- 172.21.15.161:0/1144326583 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f500666c0 msgr2=0x7f1f50068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.058+0000 7f1f74e00640 1 --2- 172.21.15.161:0/1144326583 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f500666c0 0x7f1f50068b80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0af000 tx=0x7f1f5c0a7000 comp rx=0 tx=0).stop 2024-05-22T21:20:20.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.058+0000 7f1f74e00640 1 -- 172.21.15.161:0/1144326583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f70143ec0 msgr2=0x7f1f700b0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.058+0000 7f1f74e00640 1 --2- 172.21.15.161:0/1144326583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f70143ec0 0x7f1f700b0380 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000efc0 tx=0x7f1f6000c490 comp rx=0 tx=0).stop 2024-05-22T21:20:20.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.058+0000 7f1f74e00640 1 -- 172.21.15.161:0/1144326583 shutdown_connections 2024-05-22T21:20:20.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.058+0000 7f1f74e00640 1 --2- 172.21.15.161:0/1144326583 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1f500666c0 0x7f1f50068b80 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.058+0000 7f1f74e00640 1 --2- 172.21.15.161:0/1144326583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f70137750 0x7f1f70137c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.058+0000 7f1f74e00640 1 --2- 172.21.15.161:0/1144326583 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f7015ec90 0x7f1f700adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.058+0000 7f1f74e00640 1 --2- 172.21.15.161:0/1144326583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f70143ec0 0x7f1f700b0380 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.058+0000 7f1f74e00640 1 -- 172.21.15.161:0/1144326583 >> 172.21.15.161:0/1144326583 conn(0x7f1f700b0bc0 msgr2=0x7f1f7015bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:20.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.058+0000 7f1f74e00640 1 -- 172.21.15.161:0/1144326583 shutdown_connections 2024-05-22T21:20:20.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.062+0000 7f1f74e00640 1 -- 172.21.15.161:0/1144326583 wait complete. 2024-05-22T21:20:20.082 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:20:20.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.214+0000 7f60b3000640 1 Processor -- start 2024-05-22T21:20:20.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.214+0000 7f60b3000640 1 -- start start 2024-05-22T21:20:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.218+0000 7f60b3000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60ac143ec0 0x7f60ac15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.218+0000 7f60b3000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f60ac15ec90 0x7f60ac15f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.218+0000 7f60b3000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ac1442a0 con 0x7f60ac0715a0 2024-05-22T21:20:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.218+0000 7f60b3000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ac15f680 con 0x7f60ac15ec90 2024-05-22T21:20:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.218+0000 7f60b3000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac070500 con 0x7f60ac143ec0 2024-05-22T21:20:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.218+0000 7f60b1000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f60ac15ec90 0x7f60ac15f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.218+0000 7f60b1000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f60ac15ec90 0x7f60ac15f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45246/0 (socket says 172.21.15.161:45246) 2024-05-22T21:20:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.218+0000 7f60b1000640 1 -- 172.21.15.161:0/4184822859 learned_addr learned my addr 172.21.15.161:0/4184822859 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:20.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.218+0000 7f60abe00640 1 --2- 172.21.15.161:0/4184822859 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60ac143ec0 0x7f60ac15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.218+0000 7f60b1000640 1 -- 172.21.15.161:0/4184822859 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60ac143ec0 msgr2=0x7f60ac15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:20.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.218+0000 7f60b1000640 1 --2- 172.21.15.161:0/4184822859 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60ac143ec0 0x7f60ac15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.218+0000 7f60b1000640 1 -- 172.21.15.161:0/4184822859 >> v1:172.21.15.179:6789/0 conn(0x7f60ac0715a0 legacy=0x7f60ac06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:20.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.218+0000 7f60b1000640 1 -- 172.21.15.161:0/4184822859 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac0707e0 con 0x7f60ac15ec90 2024-05-22T21:20:20.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.218+0000 7f60abe00640 1 --2- 172.21.15.161:0/4184822859 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60ac143ec0 0x7f60ac15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:20.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.218+0000 7f60b1000640 1 --2- 172.21.15.161:0/4184822859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ac15ec90 0x7f60ac15f140 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009870 tx=0x7f60a401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3abb65cadf7c7e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:20.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60aaa00640 1 -- 172.21.15.161:0/4184822859 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a40a7070 con 0x7f60ac15ec90 2024-05-22T21:20:20.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60aaa00640 1 -- 172.21.15.161:0/4184822859 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a40ab430 con 0x7f60ac15ec90 2024-05-22T21:20:20.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60aaa00640 1 -- 172.21.15.161:0/4184822859 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a40ab7c0 con 0x7f60ac15ec90 2024-05-22T21:20:20.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60b3000640 1 -- 172.21.15.161:0/4184822859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ac15ec90 msgr2=0x7f60ac15f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60b3000640 1 --2- 172.21.15.161:0/4184822859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ac15ec90 0x7f60ac15f140 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009870 tx=0x7f60a401c530 comp rx=0 tx=0).stop 2024-05-22T21:20:20.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60b3000640 1 -- 172.21.15.161:0/4184822859 shutdown_connections 2024-05-22T21:20:20.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60b3000640 1 --2- 172.21.15.161:0/4184822859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ac15ec90 0x7f60ac15f140 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60b3000640 1 --2- 172.21.15.161:0/4184822859 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60ac143ec0 0x7f60ac15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60b3000640 1 -- 172.21.15.161:0/4184822859 >> 172.21.15.161:0/4184822859 conn(0x7f60ac0b0bc0 msgr2=0x7f60ac0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:20.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60b3000640 1 -- 172.21.15.161:0/4184822859 shutdown_connections 2024-05-22T21:20:20.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60b3000640 1 -- 172.21.15.161:0/4184822859 wait complete. 2024-05-22T21:20:20.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60b3000640 1 Processor -- start 2024-05-22T21:20:20.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60b3000640 1 -- start start 2024-05-22T21:20:20.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60b3000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ac143ec0 0x7f60ac159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60b3000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60ac15ec90 0x7f60ac06c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60b3000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60ac15a2b0 0x7f60ac06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60b3000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac159b90 con 0x7f60ac143ec0 2024-05-22T21:20:20.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60b3000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac0aaf80 con 0x7f60ac15ec90 2024-05-22T21:20:20.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60b3000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac0b2060 con 0x7f60ac15a2b0 2024-05-22T21:20:20.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60abe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ac143ec0 0x7f60ac159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60abe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ac143ec0 0x7f60ac159650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45254/0 (socket says 172.21.15.161:45254) 2024-05-22T21:20:20.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.222+0000 7f60abe00640 1 -- 172.21.15.161:0/842192759 learned_addr learned my addr 172.21.15.161:0/842192759 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:20.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60ab400640 1 --2- 172.21.15.161:0/842192759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60ac15ec90 0x7f60ac06c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60b1000640 1 --2- 172.21.15.161:0/842192759 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60ac15a2b0 0x7f60ac06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60abe00640 1 -- 172.21.15.161:0/842192759 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60ac15a2b0 msgr2=0x7f60ac06ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60abe00640 1 --2- 172.21.15.161:0/842192759 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60ac15a2b0 0x7f60ac06ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60abe00640 1 -- 172.21.15.161:0/842192759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60ac15ec90 msgr2=0x7f60ac06c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60abe00640 1 --2- 172.21.15.161:0/842192759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60ac15ec90 0x7f60ac06c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60abe00640 1 -- 172.21.15.161:0/842192759 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a40094d0 con 0x7f60ac143ec0 2024-05-22T21:20:20.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60ab400640 1 --2- 172.21.15.161:0/842192759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60ac15ec90 0x7f60ac06c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:20.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60abe00640 1 --2- 172.21.15.161:0/842192759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ac143ec0 0x7f60ac159650 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f609c00cf60 tx=0x7f609c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:20.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60a3e00640 1 -- 172.21.15.161:0/842192759 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c00d840 con 0x7f60ac143ec0 2024-05-22T21:20:20.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60a3e00640 1 -- 172.21.15.161:0/842192759 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609c00d9a0 con 0x7f60ac143ec0 2024-05-22T21:20:20.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60a3e00640 1 -- 172.21.15.161:0/842192759 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c00dd30 con 0x7f60ac143ec0 2024-05-22T21:20:20.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60b3000640 1 -- 172.21.15.161:0/842192759 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ac142200 con 0x7f60ac143ec0 2024-05-22T21:20:20.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60b3000640 1 -- 172.21.15.161:0/842192759 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac0b25e0 con 0x7f60ac143ec0 2024-05-22T21:20:20.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60b3000640 1 -- 172.21.15.161:0/842192759 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ac067220 con 0x7f60ac143ec0 2024-05-22T21:20:20.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60a3e00640 1 -- 172.21.15.161:0/842192759 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f609c015440 con 0x7f60ac143ec0 2024-05-22T21:20:20.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.226+0000 7f60a3e00640 1 --2- 172.21.15.161:0/842192759 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f608c066690 0x7f608c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.230+0000 7f60a3e00640 1 -- 172.21.15.161:0/842192759 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f609c084c80 con 0x7f60ac143ec0 2024-05-22T21:20:20.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.230+0000 7f60ab400640 1 --2- 172.21.15.161:0/842192759 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f608c066690 0x7f608c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.230+0000 7f60ab400640 1 --2- 172.21.15.161:0/842192759 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f608c066690 0x7f608c068b50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f60ac067c00 tx=0x7f6098009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:20.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.230+0000 7f60a3e00640 1 -- 172.21.15.161:0/842192759 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f609c051fc0 con 0x7f60ac143ec0 2024-05-22T21:20:20.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.390+0000 7f60b3000640 1 -- 172.21.15.161:0/842192759 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60ac06dcc0 con 0x7f60ac143ec0 2024-05-22T21:20:20.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.390+0000 7f60a3e00640 1 -- 172.21.15.161:0/842192759 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f609c01a020 con 0x7f60ac143ec0 2024-05-22T21:20:20.395 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:20.395 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":146,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:20:20.260480+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:44.080742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:45.388618+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.30","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:20:20.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.394+0000 7f60b3000640 1 -- 172.21.15.161:0/842192759 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f608c066690 msgr2=0x7f608c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.394+0000 7f60b3000640 1 --2- 172.21.15.161:0/842192759 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f608c066690 0x7f608c068b50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f60ac067c00 tx=0x7f6098009040 comp rx=0 tx=0).stop 2024-05-22T21:20:20.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.394+0000 7f60b3000640 1 -- 172.21.15.161:0/842192759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ac143ec0 msgr2=0x7f60ac159650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.394+0000 7f60b3000640 1 --2- 172.21.15.161:0/842192759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ac143ec0 0x7f60ac159650 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f609c00cf60 tx=0x7f609c00cf90 comp rx=0 tx=0).stop 2024-05-22T21:20:20.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.394+0000 7f60b3000640 1 -- 172.21.15.161:0/842192759 shutdown_connections 2024-05-22T21:20:20.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.394+0000 7f60b3000640 1 --2- 172.21.15.161:0/842192759 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f608c066690 0x7f608c068b50 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.394+0000 7f60b3000640 1 --2- 172.21.15.161:0/842192759 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60ac15a2b0 0x7f60ac06ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.394+0000 7f60b3000640 1 --2- 172.21.15.161:0/842192759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60ac15ec90 0x7f60ac06c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.394+0000 7f60b3000640 1 --2- 172.21.15.161:0/842192759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60ac143ec0 0x7f60ac159650 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.394+0000 7f60b3000640 1 -- 172.21.15.161:0/842192759 >> 172.21.15.161:0/842192759 conn(0x7f60ac0b0bc0 msgr2=0x7f60ac15cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:20.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.394+0000 7f60b3000640 1 -- 172.21.15.161:0/842192759 shutdown_connections 2024-05-22T21:20:20.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.394+0000 7f60b3000640 1 -- 172.21.15.161:0/842192759 wait complete. 2024-05-22T21:20:20.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:19:44.080742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:19:45.388618+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '146', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:20:20.417 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:20:20.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa90dc00640 1 Processor -- start 2024-05-22T21:20:20.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa90dc00640 1 -- start start 2024-05-22T21:20:20.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa90dc00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa90813cfc0 0x7fa9081402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa90dc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa90813fe50 0x7fa90806fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa90dc00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90813d3a0 con 0x7fa90815af60 2024-05-22T21:20:20.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa90dc00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9080cd920 con 0x7fa90813cfc0 2024-05-22T21:20:20.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa90dc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080cdc00 con 0x7fa90813fe50 2024-05-22T21:20:20.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa8fe000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa90815af60 0x7fa90813f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51236/0 (socket says 172.21.15.161:51236) 2024-05-22T21:20:20.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa8fe000640 1 -- 172.21.15.161:0/2495197551 learned_addr learned my addr 172.21.15.161:0/2495197551 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:20.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa906a00640 1 --2- 172.21.15.161:0/2495197551 >> v2:172.21.15.179:3300/0 conn(0x7fa90813cfc0 0x7fa9081402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa907400640 1 --2- 172.21.15.161:0/2495197551 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa90813fe50 0x7fa90806fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa906000640 1 -- 172.21.15.161:0/2495197551 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393452310 0 0) 0x7fa90813d3a0 con 0x7fa90815af60 2024-05-22T21:20:20.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa906000640 1 -- 172.21.15.161:0/2495197551 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8ec003530 con 0x7fa90815af60 2024-05-22T21:20:20.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa906000640 1 -- 172.21.15.161:0/2495197551 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2825839062 0 0) 0x7fa8ec003530 con 0x7fa90815af60 2024-05-22T21:20:20.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa906000640 1 -- 172.21.15.161:0/2495197551 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa90813fe50 msgr2=0x7fa90806fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:20.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa906000640 1 --2- 172.21.15.161:0/2495197551 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa90813fe50 0x7fa90806fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa906000640 1 -- 172.21.15.161:0/2495197551 >> v2:172.21.15.179:3300/0 conn(0x7fa90813cfc0 msgr2=0x7fa9081402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:20.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa906000640 1 --2- 172.21.15.161:0/2495197551 >> v2:172.21.15.179:3300/0 conn(0x7fa90813cfc0 0x7fa9081402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa906000640 1 -- 172.21.15.161:0/2495197551 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9080cdee0 con 0x7fa90815af60 2024-05-22T21:20:20.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa906a00640 1 --2- 172.21.15.161:0/2495197551 >> v2:172.21.15.179:3300/0 conn(0x7fa90813cfc0 0x7fa9081402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:20:20.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.554+0000 7fa906000640 1 -- 172.21.15.161:0/2495197551 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fa8f8001680 con 0x7fa90815af60 2024-05-22T21:20:20.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa906000640 1 -- 172.21.15.161:0/2495197551 >> v1:172.21.15.161:6789/0 conn(0x7fa90815af60 legacy=0x7fa90813f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa906000640 1 --2- 172.21.15.161:0/2495197551 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa90813fe50 0x7fa8ec003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa906000640 1 --2- 172.21.15.161:0/2495197551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8ec003f00 0x7fa8ec0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa906000640 1 --2- 172.21.15.161:0/2495197551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8ec004d10 0x7fa8ec005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa906000640 1 -- 172.21.15.161:0/2495197551 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080cd920 con 0x7fa8ec004d10 2024-05-22T21:20:20.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa906000640 1 -- 172.21.15.161:0/2495197551 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080cdc00 con 0x7fa8ec003f00 2024-05-22T21:20:20.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa906000640 1 -- 172.21.15.161:0/2495197551 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec00b9e0 con 0x7fa90813fe50 2024-05-22T21:20:20.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa907400640 1 --2- 172.21.15.161:0/2495197551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8ec004d10 0x7fa8ec005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa906a00640 1 --2- 172.21.15.161:0/2495197551 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa90813fe50 0x7fa8ec003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa8fe000640 1 --2- 172.21.15.161:0/2495197551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8ec003f00 0x7fa8ec0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa907400640 1 -- 172.21.15.161:0/2495197551 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa90813fe50 msgr2=0x7fa8ec003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:20.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa907400640 1 --2- 172.21.15.161:0/2495197551 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa90813fe50 0x7fa8ec003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa907400640 1 -- 172.21.15.161:0/2495197551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8ec003f00 msgr2=0x7fa8ec0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:20.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa907400640 1 --2- 172.21.15.161:0/2495197551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8ec003f00 0x7fa8ec0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa907400640 1 -- 172.21.15.161:0/2495197551 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ec003530 con 0x7fa8ec004d10 2024-05-22T21:20:20.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa8fe000640 1 --2- 172.21.15.161:0/2495197551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8ec003f00 0x7fa8ec0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:20.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa907400640 1 --2- 172.21.15.161:0/2495197551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8ec004d10 0x7fa8ec005190 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fa900009fa0 tx=0x7fa90001cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2190428d85c24b21 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:20.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa906000640 1 -- 172.21.15.161:0/2495197551 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9000a5cd0 con 0x7fa8ec004d10 2024-05-22T21:20:20.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa906000640 1 -- 172.21.15.161:0/2495197551 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90000f040 con 0x7fa8ec004d10 2024-05-22T21:20:20.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.558+0000 7fa906000640 1 -- 172.21.15.161:0/2495197551 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9000ad660 con 0x7fa8ec004d10 2024-05-22T21:20:20.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa90dc00640 1 -- 172.21.15.161:0/2495197551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8ec004d10 msgr2=0x7fa8ec005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa90dc00640 1 --2- 172.21.15.161:0/2495197551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8ec004d10 0x7fa8ec005190 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fa900009fa0 tx=0x7fa90001cc00 comp rx=0 tx=0).stop 2024-05-22T21:20:20.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa907400640 1 -- 172.21.15.161:0/2495197551 reap_dead start 2024-05-22T21:20:20.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa90dc00640 1 -- 172.21.15.161:0/2495197551 shutdown_connections 2024-05-22T21:20:20.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa90dc00640 1 -- 172.21.15.161:0/2495197551 >> 172.21.15.161:0/2495197551 conn(0x7fa90806a2d0 msgr2=0x7fa90806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:20.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa90dc00640 1 -- 172.21.15.161:0/2495197551 shutdown_connections 2024-05-22T21:20:20.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa90dc00640 1 -- 172.21.15.161:0/2495197551 wait complete. 2024-05-22T21:20:20.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa90dc00640 1 Processor -- start 2024-05-22T21:20:20.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa90dc00640 1 -- start start 2024-05-22T21:20:20.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa90dc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8ec004d10 0x7fa90813ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa90dc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9081f3e40 0x7fa9080ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa90dc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9080a9a20 0x7fa90813c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa90dc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9081f43e0 con 0x7fa8ec004d10 2024-05-22T21:20:20.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa90dc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9081f46c0 con 0x7fa9081f3e40 2024-05-22T21:20:20.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa90dc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9081f49a0 con 0x7fa9080a9a20 2024-05-22T21:20:20.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa906a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8ec004d10 0x7fa90813ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa906a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8ec004d10 0x7fa90813ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45266/0 (socket says 172.21.15.161:45266) 2024-05-22T21:20:20.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa906a00640 1 -- 172.21.15.161:0/4048303770 learned_addr learned my addr 172.21.15.161:0/4048303770 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:20.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa8fe000640 1 --2- 172.21.15.161:0/4048303770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9081f3e40 0x7fa9080ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa907400640 1 --2- 172.21.15.161:0/4048303770 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9080a9a20 0x7fa90813c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa906a00640 1 -- 172.21.15.161:0/4048303770 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9080a9a20 msgr2=0x7fa90813c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa906a00640 1 --2- 172.21.15.161:0/4048303770 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9080a9a20 0x7fa90813c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.562+0000 7fa906a00640 1 -- 172.21.15.161:0/4048303770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9081f3e40 msgr2=0x7fa9080ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.566+0000 7fa906a00640 1 --2- 172.21.15.161:0/4048303770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9081f3e40 0x7fa9080ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.566+0000 7fa906a00640 1 -- 172.21.15.161:0/4048303770 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9081f4c80 con 0x7fa8ec004d10 2024-05-22T21:20:20.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.566+0000 7fa907400640 1 --2- 172.21.15.161:0/4048303770 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9080a9a20 0x7fa90813c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:20.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.566+0000 7fa906a00640 1 --2- 172.21.15.161:0/4048303770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8ec004d10 0x7fa90813ea40 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0011cd0 tx=0x7fa8f000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:20.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.566+0000 7fa8ff400640 1 -- 172.21.15.161:0/4048303770 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0011e80 con 0x7fa8ec004d10 2024-05-22T21:20:20.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.566+0000 7fa8ff400640 1 -- 172.21.15.161:0/4048303770 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f0004020 con 0x7fa8ec004d10 2024-05-22T21:20:20.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.566+0000 7fa8ff400640 1 -- 172.21.15.161:0/4048303770 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f00043b0 con 0x7fa8ec004d10 2024-05-22T21:20:20.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.566+0000 7fa90dc00640 1 -- 172.21.15.161:0/4048303770 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9080a84a0 con 0x7fa8ec004d10 2024-05-22T21:20:20.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.566+0000 7fa90dc00640 1 -- 172.21.15.161:0/4048303770 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9080abcf0 con 0x7fa8ec004d10 2024-05-22T21:20:20.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.566+0000 7fa90dc00640 1 -- 172.21.15.161:0/4048303770 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa908067220 con 0x7fa8ec004d10 2024-05-22T21:20:20.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.566+0000 7fa8ff400640 1 -- 172.21.15.161:0/4048303770 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa8f0003620 con 0x7fa8ec004d10 2024-05-22T21:20:20.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.566+0000 7fa8ff400640 1 --2- 172.21.15.161:0/4048303770 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8e4066690 0x7fa8e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.566+0000 7fa8ff400640 1 -- 172.21.15.161:0/4048303770 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa8f0084970 con 0x7fa8ec004d10 2024-05-22T21:20:20.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.566+0000 7fa8fe000640 1 --2- 172.21.15.161:0/4048303770 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8e4066690 0x7fa8e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.566+0000 7fa8ff400640 1 -- 172.21.15.161:0/4048303770 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=147}) v3 -- 0x7fa8e406e280 con 0x7fa8ec004d10 2024-05-22T21:20:20.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.570+0000 7fa8fe000640 1 --2- 172.21.15.161:0/4048303770 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8e4066690 0x7fa8e4068b50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa90806ff70 tx=0x7fa8f8001e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:20.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.570+0000 7fa8ff400640 1 -- 172.21.15.161:0/4048303770 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa8f0022070 con 0x7fa8ec004d10 2024-05-22T21:20:20.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.734+0000 7fa90dc00640 1 -- 172.21.15.161:0/4048303770 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9081f4520 con 0x7fa8ec004d10 2024-05-22T21:20:20.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.734+0000 7fa8ff400640 1 -- 172.21.15.161:0/4048303770 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v146) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9081f4520 con 0x7fa8ec004d10 2024-05-22T21:20:20.736 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:20:20.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.734+0000 7fa90dc00640 1 -- 172.21.15.161:0/4048303770 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8e4066690 msgr2=0x7fa8e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.734+0000 7fa90dc00640 1 --2- 172.21.15.161:0/4048303770 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8e4066690 0x7fa8e4068b50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa90806ff70 tx=0x7fa8f8001e50 comp rx=0 tx=0).stop 2024-05-22T21:20:20.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.734+0000 7fa90dc00640 1 -- 172.21.15.161:0/4048303770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8ec004d10 msgr2=0x7fa90813ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.734+0000 7fa90dc00640 1 --2- 172.21.15.161:0/4048303770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8ec004d10 0x7fa90813ea40 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0011cd0 tx=0x7fa8f000f610 comp rx=0 tx=0).stop 2024-05-22T21:20:20.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.734+0000 7fa90dc00640 1 -- 172.21.15.161:0/4048303770 shutdown_connections 2024-05-22T21:20:20.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.734+0000 7fa90dc00640 1 --2- 172.21.15.161:0/4048303770 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa8e4066690 0x7fa8e4068b50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.734+0000 7fa90dc00640 1 --2- 172.21.15.161:0/4048303770 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9080a9a20 0x7fa90813c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.734+0000 7fa90dc00640 1 --2- 172.21.15.161:0/4048303770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9081f3e40 0x7fa9080ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.734+0000 7fa90dc00640 1 --2- 172.21.15.161:0/4048303770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8ec004d10 0x7fa90813ea40 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.734+0000 7fa90dc00640 1 -- 172.21.15.161:0/4048303770 >> 172.21.15.161:0/4048303770 conn(0x7fa90806a2d0 msgr2=0x7fa908158370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:20.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.734+0000 7fa90dc00640 1 -- 172.21.15.161:0/4048303770 shutdown_connections 2024-05-22T21:20:20.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.734+0000 7fa90dc00640 1 -- 172.21.15.161:0/4048303770 wait complete. 2024-05-22T21:20:20.758 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:20:20.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.890+0000 7fb2d0200640 1 Processor -- start 2024-05-22T21:20:20.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.890+0000 7fb2d0200640 1 -- start start 2024-05-22T21:20:20.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.894+0000 7fb2d0200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb2c8143c40 0x7fb2c815b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.894+0000 7fb2d0200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb2c81444a0 0x7fb2c806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.894+0000 7fb2d0200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c815efb0 con 0x7fb2c8144900 2024-05-22T21:20:20.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.894+0000 7fb2d0200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c815f290 con 0x7fb2c8143c40 2024-05-22T21:20:20.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.894+0000 7fb2d0200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c815f570 con 0x7fb2c81444a0 2024-05-22T21:20:20.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.894+0000 7fb2cd800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb2c8143c40 0x7fb2c815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.894+0000 7fb2cd800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb2c8143c40 0x7fb2c815b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45268/0 (socket says 172.21.15.161:45268) 2024-05-22T21:20:20.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.894+0000 7fb2cd800640 1 -- 172.21.15.161:0/3580670199 learned_addr learned my addr 172.21.15.161:0/3580670199 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:20.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.894+0000 7fb2cce00640 1 --2- 172.21.15.161:0/3580670199 >> v2:172.21.15.179:3300/0 conn(0x7fb2c81444a0 0x7fb2c806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.894+0000 7fb2cd800640 1 -- 172.21.15.161:0/3580670199 >> v2:172.21.15.179:3300/0 conn(0x7fb2c81444a0 msgr2=0x7fb2c806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:20.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.894+0000 7fb2cd800640 1 --2- 172.21.15.161:0/3580670199 >> v2:172.21.15.179:3300/0 conn(0x7fb2c81444a0 0x7fb2c806c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.894+0000 7fb2cd800640 1 -- 172.21.15.161:0/3580670199 >> v1:172.21.15.179:6789/0 conn(0x7fb2c8144900 legacy=0x7fb2c80b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.894+0000 7fb2cd800640 1 -- 172.21.15.161:0/3580670199 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c815f850 con 0x7fb2c8143c40 2024-05-22T21:20:20.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.898+0000 7fb2cd800640 1 --2- 172.21.15.161:0/3580670199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c8143c40 0x7fb2c815b600 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80b1c80 tx=0x7fb2bc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2f3084d77a0c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:20.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.898+0000 7fb2c7e00640 1 -- 172.21.15.161:0/3580670199 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0a7070 con 0x7fb2c8143c40 2024-05-22T21:20:20.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.898+0000 7fb2c7e00640 1 -- 172.21.15.161:0/3580670199 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc0ab430 con 0x7fb2c8143c40 2024-05-22T21:20:20.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.898+0000 7fb2c7e00640 1 -- 172.21.15.161:0/3580670199 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0ab7c0 con 0x7fb2c8143c40 2024-05-22T21:20:20.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.898+0000 7fb2d0200640 1 -- 172.21.15.161:0/3580670199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c8143c40 msgr2=0x7fb2c815b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.898+0000 7fb2d0200640 1 --2- 172.21.15.161:0/3580670199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c8143c40 0x7fb2c815b600 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80b1c80 tx=0x7fb2bc01c520 comp rx=0 tx=0).stop 2024-05-22T21:20:20.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.898+0000 7fb2d0200640 1 -- 172.21.15.161:0/3580670199 shutdown_connections 2024-05-22T21:20:20.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.898+0000 7fb2d0200640 1 --2- 172.21.15.161:0/3580670199 >> v2:172.21.15.179:3300/0 conn(0x7fb2c81444a0 0x7fb2c806c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.898+0000 7fb2d0200640 1 --2- 172.21.15.161:0/3580670199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c8143c40 0x7fb2c815b600 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.898+0000 7fb2d0200640 1 -- 172.21.15.161:0/3580670199 >> 172.21.15.161:0/3580670199 conn(0x7fb2c806a2d0 msgr2=0x7fb2c806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:20.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.898+0000 7fb2d0200640 1 -- 172.21.15.161:0/3580670199 shutdown_connections 2024-05-22T21:20:20.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.898+0000 7fb2d0200640 1 -- 172.21.15.161:0/3580670199 wait complete. 2024-05-22T21:20:20.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.898+0000 7fb2d0200640 1 Processor -- start 2024-05-22T21:20:20.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.898+0000 7fb2d0200640 1 -- start start 2024-05-22T21:20:20.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2d0200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb2c8143c40 0x7fb2c8157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2d0200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c81444a0 0x7fb2c80ad350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2d0200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2c8144900 0x7fb2c815de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2d0200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81582b0 con 0x7fb2c81444a0 2024-05-22T21:20:20.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2d0200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81585b0 con 0x7fb2c8144900 2024-05-22T21:20:20.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2d0200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c8158890 con 0x7fb2c8143c40 2024-05-22T21:20:20.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2cce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c81444a0 0x7fb2c80ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2cce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c81444a0 0x7fb2c80ad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45278/0 (socket says 172.21.15.161:45278) 2024-05-22T21:20:20.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2cce00640 1 -- 172.21.15.161:0/2411050282 learned_addr learned my addr 172.21.15.161:0/2411050282 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:20.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2cd800640 1 --2- 172.21.15.161:0/2411050282 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb2c8143c40 0x7fb2c8157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2ce200640 1 --2- 172.21.15.161:0/2411050282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2c8144900 0x7fb2c815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2cce00640 1 -- 172.21.15.161:0/2411050282 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb2c8143c40 msgr2=0x7fb2c8157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2cce00640 1 --2- 172.21.15.161:0/2411050282 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb2c8143c40 0x7fb2c8157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2cce00640 1 -- 172.21.15.161:0/2411050282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2c8144900 msgr2=0x7fb2c815de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:20.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2cce00640 1 --2- 172.21.15.161:0/2411050282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2c8144900 0x7fb2c815de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:20.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2cce00640 1 -- 172.21.15.161:0/2411050282 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc0094d0 con 0x7fb2c81444a0 2024-05-22T21:20:20.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2ce200640 1 --2- 172.21.15.161:0/2411050282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2c8144900 0x7fb2c815de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:20.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2cce00640 1 --2- 172.21.15.161:0/2411050282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c81444a0 0x7fb2c80ad350 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400d800 tx=0x7fb2b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:20.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2c5600640 1 -- 172.21.15.161:0/2411050282 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400b6c0 con 0x7fb2c81444a0 2024-05-22T21:20:20.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2c5600640 1 -- 172.21.15.161:0/2411050282 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b400b820 con 0x7fb2c81444a0 2024-05-22T21:20:20.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2c5600640 1 -- 172.21.15.161:0/2411050282 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400bb90 con 0x7fb2c81444a0 2024-05-22T21:20:20.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2d0200640 1 -- 172.21.15.161:0/2411050282 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c8158b70 con 0x7fb2c81444a0 2024-05-22T21:20:20.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2d0200640 1 -- 172.21.15.161:0/2411050282 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c8159130 con 0x7fb2c81444a0 2024-05-22T21:20:20.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2d0200640 1 -- 172.21.15.161:0/2411050282 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c8067220 con 0x7fb2c81444a0 2024-05-22T21:20:20.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.902+0000 7fb2c5600640 1 -- 172.21.15.161:0/2411050282 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb2b400bcf0 con 0x7fb2c81444a0 2024-05-22T21:20:20.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.906+0000 7fb2c5600640 1 --2- 172.21.15.161:0/2411050282 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb2a8066690 0x7fb2a8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:20.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.906+0000 7fb2c5600640 1 -- 172.21.15.161:0/2411050282 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb2b4084770 con 0x7fb2c81444a0 2024-05-22T21:20:20.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.906+0000 7fb2c5600640 1 -- 172.21.15.161:0/2411050282 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=147}) v3 -- 0x7fb2a806e280 con 0x7fb2c81444a0 2024-05-22T21:20:20.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.906+0000 7fb2cd800640 1 --2- 172.21.15.161:0/2411050282 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb2a8066690 0x7fb2a8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:20.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.906+0000 7fb2cd800640 1 --2- 172.21.15.161:0/2411050282 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb2a8066690 0x7fb2a8068b50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0af000 tx=0x7fb2bc0b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:20.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:20.910+0000 7fb2c5600640 1 -- 172.21.15.161:0/2411050282 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb2b4051a00 con 0x7fb2c81444a0 2024-05-22T21:20:21.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.058+0000 7fb2d0200640 1 -- 172.21.15.161:0/2411050282 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb2c80ceaa0 con 0x7fb2c81444a0 2024-05-22T21:20:21.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.062+0000 7fb2c5600640 1 -- 172.21.15.161:0/2411050282 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v146) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb2b4055020 con 0x7fb2c81444a0 2024-05-22T21:20:21.064 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:20:21.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.062+0000 7fb2d0200640 1 -- 172.21.15.161:0/2411050282 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb2a8066690 msgr2=0x7fb2a8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.062+0000 7fb2d0200640 1 --2- 172.21.15.161:0/2411050282 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb2a8066690 0x7fb2a8068b50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0af000 tx=0x7fb2bc0b6040 comp rx=0 tx=0).stop 2024-05-22T21:20:21.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.062+0000 7fb2d0200640 1 -- 172.21.15.161:0/2411050282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c81444a0 msgr2=0x7fb2c80ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.062+0000 7fb2d0200640 1 --2- 172.21.15.161:0/2411050282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c81444a0 0x7fb2c80ad350 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400d800 tx=0x7fb2b400dcc0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.062+0000 7fb2d0200640 1 -- 172.21.15.161:0/2411050282 shutdown_connections 2024-05-22T21:20:21.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.062+0000 7fb2d0200640 1 --2- 172.21.15.161:0/2411050282 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb2a8066690 0x7fb2a8068b50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.062+0000 7fb2d0200640 1 --2- 172.21.15.161:0/2411050282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2c8144900 0x7fb2c815de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.062+0000 7fb2d0200640 1 --2- 172.21.15.161:0/2411050282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2c81444a0 0x7fb2c80ad350 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.062+0000 7fb2d0200640 1 --2- 172.21.15.161:0/2411050282 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb2c8143c40 0x7fb2c8157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.062+0000 7fb2d0200640 1 -- 172.21.15.161:0/2411050282 >> 172.21.15.161:0/2411050282 conn(0x7fb2c806a2d0 msgr2=0x7fb2c815b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:21.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.062+0000 7fb2d0200640 1 -- 172.21.15.161:0/2411050282 shutdown_connections 2024-05-22T21:20:21.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.062+0000 7fb2d0200640 1 -- 172.21.15.161:0/2411050282 wait complete. 2024-05-22T21:20:21.085 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:20:21.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f0038200640 1 Processor -- start 2024-05-22T21:20:21.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f0038200640 1 -- start start 2024-05-22T21:20:21.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f0038200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f00301402e0 0x7f003013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f0038200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00301400d0 con 0x7f003013cfc0 2024-05-22T21:20:21.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f0038200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003013d3a0 con 0x7f003013fc80 2024-05-22T21:20:21.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f0038200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00300cd920 con 0x7f00301402e0 2024-05-22T21:20:21.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f0035800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f003013cfc0 0x7f003015af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51242/0 (socket says 172.21.15.161:51242) 2024-05-22T21:20:21.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f0035800640 1 -- 172.21.15.161:0/1164875788 learned_addr learned my addr 172.21.15.161:0/1164875788 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:21.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f0034e00640 1 --2- 172.21.15.161:0/1164875788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f00301402e0 0x7f003013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f002fe00640 1 -- 172.21.15.161:0/1164875788 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246206427 0 0) 0x7f00240014a0 con 0x7f003013cfc0 2024-05-22T21:20:21.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f002fe00640 1 -- 172.21.15.161:0/1164875788 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0018003530 con 0x7f003013cfc0 2024-05-22T21:20:21.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f002fe00640 1 -- 172.21.15.161:0/1164875788 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653567846 0 0) 0x7f003013d3a0 con 0x7f003013fc80 2024-05-22T21:20:21.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f002fe00640 1 -- 172.21.15.161:0/1164875788 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00240014a0 con 0x7f003013fc80 2024-05-22T21:20:21.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f002fe00640 1 -- 172.21.15.161:0/1164875788 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3795698073 0 0) 0x7f0018003530 con 0x7f003013cfc0 2024-05-22T21:20:21.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f0034e00640 1 -- 172.21.15.161:0/1164875788 >> v1:172.21.15.179:6789/0 conn(0x7f003013fc80 legacy=0x7f003006fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f0034e00640 1 -- 172.21.15.161:0/1164875788 >> v1:172.21.15.161:6789/0 conn(0x7f003013cfc0 legacy=0x7f003015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.222+0000 7f0034e00640 1 -- 172.21.15.161:0/1164875788 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00300cdc00 con 0x7f00301402e0 2024-05-22T21:20:21.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.226+0000 7f0034e00640 1 --2- 172.21.15.161:0/1164875788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f00301402e0 0x7f003013f740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f001c0097f0 tx=0x7f001c01c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb1ea4fb254be4dd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:21.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.226+0000 7f002fe00640 1 -- 172.21.15.161:0/1164875788 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024001b90 con 0x7f00301402e0 2024-05-22T21:20:21.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.226+0000 7f002fe00640 1 -- 172.21.15.161:0/1164875788 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001c0a7050 con 0x7f00301402e0 2024-05-22T21:20:21.229 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.226+0000 7f002fe00640 1 -- 172.21.15.161:0/1164875788 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c0ab400 con 0x7f00301402e0 2024-05-22T21:20:21.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.226+0000 7f0038200640 1 -- 172.21.15.161:0/1164875788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f00301402e0 msgr2=0x7f003013f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.226+0000 7f0038200640 1 --2- 172.21.15.161:0/1164875788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f00301402e0 0x7f003013f740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f001c0097f0 tx=0x7f001c01c340 comp rx=0 tx=0).stop 2024-05-22T21:20:21.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.226+0000 7f0038200640 1 -- 172.21.15.161:0/1164875788 shutdown_connections 2024-05-22T21:20:21.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.226+0000 7f0038200640 1 --2- 172.21.15.161:0/1164875788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f00301402e0 0x7f003013f740 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.226+0000 7f0038200640 1 -- 172.21.15.161:0/1164875788 >> 172.21.15.161:0/1164875788 conn(0x7f003006a2d0 msgr2=0x7f003006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:21.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.226+0000 7f0038200640 1 -- 172.21.15.161:0/1164875788 shutdown_connections 2024-05-22T21:20:21.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.226+0000 7f0038200640 1 -- 172.21.15.161:0/1164875788 wait complete. 2024-05-22T21:20:21.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0038200640 1 Processor -- start 2024-05-22T21:20:21.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0038200640 1 -- start start 2024-05-22T21:20:21.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0038200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003013cfc0 0x7f003015ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0038200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003013fc80 0x7f0030156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0038200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00301402e0 0x7f00301569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0038200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00300b2b90 con 0x7f003013fc80 2024-05-22T21:20:21.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0038200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00300b3710 con 0x7f00301402e0 2024-05-22T21:20:21.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0038200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00300b2de0 con 0x7f003013cfc0 2024-05-22T21:20:21.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0034e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003013fc80 0x7f0030156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0034e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003013fc80 0x7f0030156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45286/0 (socket says 172.21.15.161:45286) 2024-05-22T21:20:21.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0034e00640 1 -- 172.21.15.161:0/3286635055 learned_addr learned my addr 172.21.15.161:0/3286635055 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:21.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0036200640 1 --2- 172.21.15.161:0/3286635055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00301402e0 0x7f00301569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0035800640 1 --2- 172.21.15.161:0/3286635055 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003013cfc0 0x7f003015ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0034e00640 1 -- 172.21.15.161:0/3286635055 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003013cfc0 msgr2=0x7f003015ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0034e00640 1 --2- 172.21.15.161:0/3286635055 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003013cfc0 0x7f003015ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0034e00640 1 -- 172.21.15.161:0/3286635055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00301402e0 msgr2=0x7f00301569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0034e00640 1 --2- 172.21.15.161:0/3286635055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00301402e0 0x7f00301569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0034e00640 1 -- 172.21.15.161:0/3286635055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c009420 con 0x7f003013fc80 2024-05-22T21:20:21.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0035800640 1 --2- 172.21.15.161:0/3286635055 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003013cfc0 0x7f003015ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:21.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0034e00640 1 --2- 172.21.15.161:0/3286635055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003013fc80 0x7f0030156470 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f001c009740 tx=0x7f001c0190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:21.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f002d600640 1 -- 172.21.15.161:0/3286635055 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c0b5470 con 0x7f003013fc80 2024-05-22T21:20:21.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f002d600640 1 -- 172.21.15.161:0/3286635055 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001c0b55d0 con 0x7f003013fc80 2024-05-22T21:20:21.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0038200640 1 -- 172.21.15.161:0/3286635055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00300b30c0 con 0x7f003013fc80 2024-05-22T21:20:21.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f002d600640 1 -- 172.21.15.161:0/3286635055 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c0b5980 con 0x7f003013fc80 2024-05-22T21:20:21.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0038200640 1 -- 172.21.15.161:0/3286635055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00300b3ba0 con 0x7f003013fc80 2024-05-22T21:20:21.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.230+0000 7f0038200640 1 -- 172.21.15.161:0/3286635055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00300cd040 con 0x7f003013fc80 2024-05-22T21:20:21.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.234+0000 7f002d600640 1 -- 172.21.15.161:0/3286635055 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f001c0b2040 con 0x7f003013fc80 2024-05-22T21:20:21.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.234+0000 7f002d600640 1 --2- 172.21.15.161:0/3286635055 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0010066690 0x7f0010068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.234+0000 7f002d600640 1 -- 172.21.15.161:0/3286635055 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f001c11e380 con 0x7f003013fc80 2024-05-22T21:20:21.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.234+0000 7f0035800640 1 --2- 172.21.15.161:0/3286635055 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0010066690 0x7f0010068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.234+0000 7f002d600640 1 -- 172.21.15.161:0/3286635055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=147}) v3 -- 0x7f001006e280 con 0x7f003013fc80 2024-05-22T21:20:21.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.234+0000 7f0035800640 1 --2- 172.21.15.161:0/3286635055 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0010066690 0x7f0010068b50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0024002240 tx=0x7f0024009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:21.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.238+0000 7f002d600640 1 -- 172.21.15.161:0/3286635055 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f001c0b9060 con 0x7f003013fc80 2024-05-22T21:20:21.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.390+0000 7f0038200640 1 -- 172.21.15.161:0/3286635055 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0030067220 con 0x7f003013fc80 2024-05-22T21:20:21.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.394+0000 7f002d600640 1 -- 172.21.15.161:0/3286635055 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v146) v1 ==== 100+0+11 (secure 0 0 0) 0x7f001c0b0020 con 0x7f003013fc80 2024-05-22T21:20:21.396 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:20:21.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.394+0000 7f0038200640 1 -- 172.21.15.161:0/3286635055 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0010066690 msgr2=0x7f0010068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.394+0000 7f0038200640 1 --2- 172.21.15.161:0/3286635055 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0010066690 0x7f0010068b50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0024002240 tx=0x7f0024009040 comp rx=0 tx=0).stop 2024-05-22T21:20:21.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.394+0000 7f0038200640 1 -- 172.21.15.161:0/3286635055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003013fc80 msgr2=0x7f0030156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.394+0000 7f0038200640 1 --2- 172.21.15.161:0/3286635055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003013fc80 0x7f0030156470 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f001c009740 tx=0x7f001c0190e0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.394+0000 7f0038200640 1 -- 172.21.15.161:0/3286635055 shutdown_connections 2024-05-22T21:20:21.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.394+0000 7f0038200640 1 --2- 172.21.15.161:0/3286635055 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0010066690 0x7f0010068b50 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.394+0000 7f0038200640 1 --2- 172.21.15.161:0/3286635055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00301402e0 0x7f00301569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.394+0000 7f0038200640 1 --2- 172.21.15.161:0/3286635055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f003013fc80 0x7f0030156470 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.394+0000 7f0038200640 1 --2- 172.21.15.161:0/3286635055 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003013cfc0 0x7f003015ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.394+0000 7f0038200640 1 -- 172.21.15.161:0/3286635055 >> 172.21.15.161:0/3286635055 conn(0x7f003006a2d0 msgr2=0x7f003006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:21.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.394+0000 7f0038200640 1 -- 172.21.15.161:0/3286635055 shutdown_connections 2024-05-22T21:20:21.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.394+0000 7f0038200640 1 -- 172.21.15.161:0/3286635055 wait complete. 2024-05-22T21:20:21.417 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:20:21.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.550+0000 7f4a1e400640 1 Processor -- start 2024-05-22T21:20:21.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.550+0000 7f4a1e400640 1 -- start start 2024-05-22T21:20:21.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.554+0000 7f4a1e400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4a18143ec0 0x7f4a1815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.554+0000 7f4a1e400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4a180715a0 0x7f4a1806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.554+0000 7f4a1e400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a181442a0 con 0x7f4a1815ec90 2024-05-22T21:20:21.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.554+0000 7f4a1e400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a18070260 con 0x7f4a180715a0 2024-05-22T21:20:21.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.554+0000 7f4a1e400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a18070500 con 0x7f4a18143ec0 2024-05-22T21:20:21.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.554+0000 7f4a0ea00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4a180715a0 0x7f4a1806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.554+0000 7f4a17e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f4a1815ec90 0x7f4a1806bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51244/0 (socket says 172.21.15.161:51244) 2024-05-22T21:20:21.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.554+0000 7f4a17e00640 1 -- 172.21.15.161:0/2304098067 learned_addr learned my addr 172.21.15.161:0/2304098067 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:21.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.554+0000 7f4a17400640 1 --2- 172.21.15.161:0/2304098067 >> v2:172.21.15.179:3300/0 conn(0x7f4a18143ec0 0x7f4a1815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.554+0000 7f4a0ea00640 1 -- 172.21.15.161:0/2304098067 >> v2:172.21.15.179:3300/0 conn(0x7f4a18143ec0 msgr2=0x7f4a1815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:21.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.554+0000 7f4a0ea00640 1 --2- 172.21.15.161:0/2304098067 >> v2:172.21.15.179:3300/0 conn(0x7f4a18143ec0 0x7f4a1815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.554+0000 7f4a0ea00640 1 -- 172.21.15.161:0/2304098067 >> v1:172.21.15.161:6789/0 conn(0x7f4a1815ec90 legacy=0x7f4a1806bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.554+0000 7f4a16a00640 1 -- 172.21.15.161:0/2304098067 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4134691542 0 0) 0x7f4a181442a0 con 0x7f4a1815ec90 2024-05-22T21:20:21.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.554+0000 7f4a0ea00640 1 -- 172.21.15.161:0/2304098067 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a180707e0 con 0x7f4a180715a0 2024-05-22T21:20:21.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a0ea00640 1 --2- 172.21.15.161:0/2304098067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a180715a0 0x7f4a1806fd20 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f4a180724e0 tx=0x7f4a0001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15a76a20edefa88 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:21.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a16a00640 1 -- 172.21.15.161:0/2304098067 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000ab650 con 0x7f4a180715a0 2024-05-22T21:20:21.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a16a00640 1 -- 172.21.15.161:0/2304098067 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a000ab7b0 con 0x7f4a180715a0 2024-05-22T21:20:21.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a16a00640 1 -- 172.21.15.161:0/2304098067 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000abb40 con 0x7f4a180715a0 2024-05-22T21:20:21.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a1e400640 1 -- 172.21.15.161:0/2304098067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a180715a0 msgr2=0x7f4a1806fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a1e400640 1 --2- 172.21.15.161:0/2304098067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a180715a0 0x7f4a1806fd20 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f4a180724e0 tx=0x7f4a0001c450 comp rx=0 tx=0).stop 2024-05-22T21:20:21.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a1e400640 1 -- 172.21.15.161:0/2304098067 shutdown_connections 2024-05-22T21:20:21.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a1e400640 1 --2- 172.21.15.161:0/2304098067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a180715a0 0x7f4a1806fd20 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a1e400640 1 --2- 172.21.15.161:0/2304098067 >> v2:172.21.15.179:3300/0 conn(0x7f4a18143ec0 0x7f4a1815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a1e400640 1 -- 172.21.15.161:0/2304098067 >> 172.21.15.161:0/2304098067 conn(0x7f4a180b0bc0 msgr2=0x7f4a180b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:21.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a1e400640 1 -- 172.21.15.161:0/2304098067 shutdown_connections 2024-05-22T21:20:21.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a1e400640 1 -- 172.21.15.161:0/2304098067 wait complete. 2024-05-22T21:20:21.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a1e400640 1 Processor -- start 2024-05-22T21:20:21.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a1e400640 1 -- start start 2024-05-22T21:20:21.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a1e400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a18143ec0 0x7f4a180add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a1e400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a1815ec90 0x7f4a180b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.558+0000 7f4a1e400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4a180b3960 0x7f4a181f9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a1e400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180b3de0 con 0x7f4a1815ec90 2024-05-22T21:20:21.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a1e400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180ad270 con 0x7f4a18143ec0 2024-05-22T21:20:21.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a1e400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180ad550 con 0x7f4a180b3960 2024-05-22T21:20:21.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a0ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a1815ec90 0x7f4a180b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a0ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a1815ec90 0x7f4a180b3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45304/0 (socket says 172.21.15.161:45304) 2024-05-22T21:20:21.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a0ea00640 1 -- 172.21.15.161:0/3808084687 learned_addr learned my addr 172.21.15.161:0/3808084687 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:21.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a17e00640 1 --2- 172.21.15.161:0/3808084687 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4a180b3960 0x7f4a181f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a17400640 1 --2- 172.21.15.161:0/3808084687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a18143ec0 0x7f4a180add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a0ea00640 1 -- 172.21.15.161:0/3808084687 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4a180b3960 msgr2=0x7f4a181f9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a0ea00640 1 --2- 172.21.15.161:0/3808084687 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4a180b3960 0x7f4a181f9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a0ea00640 1 -- 172.21.15.161:0/3808084687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a18143ec0 msgr2=0x7f4a180add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a0ea00640 1 --2- 172.21.15.161:0/3808084687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a18143ec0 0x7f4a180add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a0ea00640 1 -- 172.21.15.161:0/3808084687 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a00009420 con 0x7f4a1815ec90 2024-05-22T21:20:21.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a0ea00640 1 --2- 172.21.15.161:0/3808084687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a1815ec90 0x7f4a180b3420 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f4a00019060 tx=0x7f4a0001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:21.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a0fe00640 1 -- 172.21.15.161:0/3808084687 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00004220 con 0x7f4a1815ec90 2024-05-22T21:20:21.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a0fe00640 1 -- 172.21.15.161:0/3808084687 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a000b9070 con 0x7f4a1815ec90 2024-05-22T21:20:21.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a0fe00640 1 -- 172.21.15.161:0/3808084687 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00003620 con 0x7f4a1815ec90 2024-05-22T21:20:21.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a1e400640 1 -- 172.21.15.161:0/3808084687 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1815f630 con 0x7f4a1815ec90 2024-05-22T21:20:21.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a1e400640 1 -- 172.21.15.161:0/3808084687 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a180ae290 con 0x7f4a1815ec90 2024-05-22T21:20:21.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a1e400640 1 -- 172.21.15.161:0/3808084687 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a18067220 con 0x7f4a1815ec90 2024-05-22T21:20:21.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.562+0000 7f4a0fe00640 1 -- 172.21.15.161:0/3808084687 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4a00003780 con 0x7f4a1815ec90 2024-05-22T21:20:21.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.566+0000 7f4a0fe00640 1 --2- 172.21.15.161:0/3808084687 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f49f4066690 0x7f49f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.566+0000 7f4a0fe00640 1 -- 172.21.15.161:0/3808084687 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4a000bb070 con 0x7f4a1815ec90 2024-05-22T21:20:21.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.566+0000 7f4a0fe00640 1 -- 172.21.15.161:0/3808084687 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=147}) v3 -- 0x7f49f406e280 con 0x7f4a1815ec90 2024-05-22T21:20:21.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.566+0000 7f4a17400640 1 --2- 172.21.15.161:0/3808084687 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f49f4066690 0x7f49f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.566+0000 7f4a17400640 1 --2- 172.21.15.161:0/3808084687 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f49f4066690 0x7f49f4068b50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4a08007330 tx=0x7f4a0800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:21.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.570+0000 7f4a0fe00640 1 -- 172.21.15.161:0/3808084687 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4a000c4070 con 0x7f4a1815ec90 2024-05-22T21:20:21.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.754+0000 7f4a1e400640 1 -- 172.21.15.161:0/3808084687 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a1806dcc0 con 0x7f4a1815ec90 2024-05-22T21:20:21.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.754+0000 7f4a0fe00640 1 -- 172.21.15.161:0/3808084687 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4745 (secure 0 0 0) 0x7f4a000ec0c0 con 0x7f4a1815ec90 2024-05-22T21:20:21.756 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:21.756 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/524372753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":524372753},{"type":"v1","addr":"172.21.15.161:6836","nonce":524372753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3762119379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3762119379},{"type":"v1","addr":"172.21.15.179:6833","nonce":3762119379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3087241366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3087241366},{"type":"v1","addr":"172.21.15.179:6835","nonce":3087241366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:46.384984+0000","modified":"2024-05-22T21:19:49.400019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7572":{"gid":7572,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/3945715002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3945715002},{"type":"v1","addr":"172.21.15.161:6837","nonce":3945715002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-22T21:20:21.757 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 62 2024-05-22T21:20:21.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.754+0000 7f4a1e400640 1 -- 172.21.15.161:0/3808084687 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f49f4066690 msgr2=0x7f49f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.754+0000 7f4a1e400640 1 --2- 172.21.15.161:0/3808084687 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f49f4066690 0x7f49f4068b50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4a08007330 tx=0x7f4a0800b040 comp rx=0 tx=0).stop 2024-05-22T21:20:21.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.754+0000 7f4a1e400640 1 -- 172.21.15.161:0/3808084687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a1815ec90 msgr2=0x7f4a180b3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.754+0000 7f4a1e400640 1 --2- 172.21.15.161:0/3808084687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a1815ec90 0x7f4a180b3420 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f4a00019060 tx=0x7f4a0001ac90 comp rx=0 tx=0).stop 2024-05-22T21:20:21.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.754+0000 7f4a1e400640 1 -- 172.21.15.161:0/3808084687 shutdown_connections 2024-05-22T21:20:21.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.754+0000 7f4a1e400640 1 --2- 172.21.15.161:0/3808084687 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f49f4066690 0x7f49f4068b50 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.754+0000 7f4a1e400640 1 --2- 172.21.15.161:0/3808084687 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4a180b3960 0x7f4a181f9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.754+0000 7f4a1e400640 1 --2- 172.21.15.161:0/3808084687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a1815ec90 0x7f4a180b3420 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.754+0000 7f4a1e400640 1 --2- 172.21.15.161:0/3808084687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a18143ec0 0x7f4a180add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.754+0000 7f4a1e400640 1 -- 172.21.15.161:0/3808084687 >> 172.21.15.161:0/3808084687 conn(0x7f4a180b0bc0 msgr2=0x7f4a180cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:21.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.754+0000 7f4a1e400640 1 -- 172.21.15.161:0/3808084687 shutdown_connections 2024-05-22T21:20:21.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.754+0000 7f4a1e400640 1 -- 172.21.15.161:0/3808084687 wait complete. 2024-05-22T21:20:21.777 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:20:21.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd18c600640 1 Processor -- start 2024-05-22T21:20:21.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd18c600640 1 -- start start 2024-05-22T21:20:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd18c600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd18406bf50 0x7fd1840cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd18c600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd18406b1f0 0x7fd1840cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd18c600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1840d1d50 con 0x7fd18415d560 2024-05-22T21:20:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd18c600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18406b680 con 0x7fd18406b1f0 2024-05-22T21:20:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd18c600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1840cbce0 con 0x7fd18406bf50 2024-05-22T21:20:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd189c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd18415d560 0x7fd18406b840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51254/0 (socket says 172.21.15.161:51254) 2024-05-22T21:20:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd189c00640 1 -- 172.21.15.161:0/1566973662 learned_addr learned my addr 172.21.15.161:0/1566973662 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd18a600640 1 --2- 172.21.15.161:0/1566973662 >> v2:172.21.15.161:3300/0 conn(0x7fd18406b1f0 0x7fd1840cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd189200640 1 --2- 172.21.15.161:0/1566973662 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd18406bf50 0x7fd1840cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd183e00640 1 -- 172.21.15.161:0/1566973662 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4082840867 0 0) 0x7fd1840d1d50 con 0x7fd18415d560 2024-05-22T21:20:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd183e00640 1 -- 172.21.15.161:0/1566973662 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd16c003530 con 0x7fd18415d560 2024-05-22T21:20:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd18a600640 1 -- 172.21.15.161:0/1566973662 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd18406bf50 msgr2=0x7fd1840cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:21.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd18a600640 1 --2- 172.21.15.161:0/1566973662 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd18406bf50 0x7fd1840cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd18a600640 1 -- 172.21.15.161:0/1566973662 >> v1:172.21.15.161:6789/0 conn(0x7fd18415d560 legacy=0x7fd18406b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.910+0000 7fd18a600640 1 -- 172.21.15.161:0/1566973662 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1840d0470 con 0x7fd18406b1f0 2024-05-22T21:20:21.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.914+0000 7fd18a600640 1 --2- 172.21.15.161:0/1566973662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd18406b1f0 0x7fd1840cc170 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009d90 tx=0x7fd17c01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538d7d320df822c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:21.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.914+0000 7fd183e00640 1 -- 172.21.15.161:0/1566973662 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0a5970 con 0x7fd18406b1f0 2024-05-22T21:20:21.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.914+0000 7fd183e00640 1 -- 172.21.15.161:0/1566973662 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c0a5ad0 con 0x7fd18406b1f0 2024-05-22T21:20:21.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.914+0000 7fd183e00640 1 -- 172.21.15.161:0/1566973662 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0a5e60 con 0x7fd18406b1f0 2024-05-22T21:20:21.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.914+0000 7fd18c600640 1 -- 172.21.15.161:0/1566973662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd18406b1f0 msgr2=0x7fd1840cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.914+0000 7fd18c600640 1 --2- 172.21.15.161:0/1566973662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd18406b1f0 0x7fd1840cc170 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009d90 tx=0x7fd17c01c8b0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.914+0000 7fd18c600640 1 -- 172.21.15.161:0/1566973662 shutdown_connections 2024-05-22T21:20:21.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.914+0000 7fd18c600640 1 --2- 172.21.15.161:0/1566973662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd18406b1f0 0x7fd1840cc170 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.914+0000 7fd18c600640 1 --2- 172.21.15.161:0/1566973662 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd18406bf50 0x7fd1840cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.914+0000 7fd18c600640 1 -- 172.21.15.161:0/1566973662 >> 172.21.15.161:0/1566973662 conn(0x7fd1840d0f10 msgr2=0x7fd1840d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:21.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.914+0000 7fd18c600640 1 -- 172.21.15.161:0/1566973662 shutdown_connections 2024-05-22T21:20:21.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.914+0000 7fd18c600640 1 -- 172.21.15.161:0/1566973662 wait complete. 2024-05-22T21:20:21.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18c600640 1 Processor -- start 2024-05-22T21:20:21.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18c600640 1 -- start start 2024-05-22T21:20:21.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18c600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd18406b1f0 0x7fd1841588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18c600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd18406bf50 0x7fd1840c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18c600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd18415d560 0x7fd18409c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18c600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1840c7900 con 0x7fd18415d560 2024-05-22T21:20:21.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18c600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1840c8cd0 con 0x7fd18406bf50 2024-05-22T21:20:21.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18c600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1840c8fb0 con 0x7fd18406b1f0 2024-05-22T21:20:21.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18a600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd18415d560 0x7fd18409c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18a600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd18415d560 0x7fd18409c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45326/0 (socket says 172.21.15.161:45326) 2024-05-22T21:20:21.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18a600640 1 -- 172.21.15.161:0/505583051 learned_addr learned my addr 172.21.15.161:0/505583051 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:21.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd189c00640 1 --2- 172.21.15.161:0/505583051 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd18406b1f0 0x7fd1841588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd189200640 1 --2- 172.21.15.161:0/505583051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd18406bf50 0x7fd1840c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18a600640 1 -- 172.21.15.161:0/505583051 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd18406b1f0 msgr2=0x7fd1841588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18a600640 1 --2- 172.21.15.161:0/505583051 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd18406b1f0 0x7fd1841588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18a600640 1 -- 172.21.15.161:0/505583051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd18406bf50 msgr2=0x7fd1840c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:21.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18a600640 1 --2- 172.21.15.161:0/505583051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd18406bf50 0x7fd1840c73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:21.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18a600640 1 -- 172.21.15.161:0/505583051 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c009a70 con 0x7fd18415d560 2024-05-22T21:20:21.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18a600640 1 --2- 172.21.15.161:0/505583051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd18415d560 0x7fd18409c000 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0b0040 tx=0x7fd17c019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:21.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd181600640 1 -- 172.21.15.161:0/505583051 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0adce0 con 0x7fd18415d560 2024-05-22T21:20:21.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd181600640 1 -- 172.21.15.161:0/505583051 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c0ade40 con 0x7fd18415d560 2024-05-22T21:20:21.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18c600640 1 -- 172.21.15.161:0/505583051 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1840c9b90 con 0x7fd18415d560 2024-05-22T21:20:21.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd181600640 1 -- 172.21.15.161:0/505583051 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0b7650 con 0x7fd18415d560 2024-05-22T21:20:21.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.918+0000 7fd18c600640 1 -- 172.21.15.161:0/505583051 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1840c89c0 con 0x7fd18415d560 2024-05-22T21:20:21.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.922+0000 7fd18c600640 1 -- 172.21.15.161:0/505583051 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd184067220 con 0x7fd18415d560 2024-05-22T21:20:21.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.922+0000 7fd181600640 1 -- 172.21.15.161:0/505583051 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd17c0b4050 con 0x7fd18415d560 2024-05-22T21:20:21.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.922+0000 7fd181600640 1 --2- 172.21.15.161:0/505583051 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd164066690 0x7fd164068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:21.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.922+0000 7fd181600640 1 -- 172.21.15.161:0/505583051 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd17c11f950 con 0x7fd18415d560 2024-05-22T21:20:21.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.922+0000 7fd189c00640 1 --2- 172.21.15.161:0/505583051 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd164066690 0x7fd164068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:21.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.922+0000 7fd181600640 1 -- 172.21.15.161:0/505583051 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=147}) v3 -- 0x7fd16406e280 con 0x7fd18415d560 2024-05-22T21:20:21.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.922+0000 7fd189c00640 1 --2- 172.21.15.161:0/505583051 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd164066690 0x7fd164068b50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd1780011f0 tx=0x7fd178007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:21.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:21.926+0000 7fd181600640 1 -- 172.21.15.161:0/505583051 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd17c0c7060 con 0x7fd18415d560 2024-05-22T21:20:22.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.086+0000 7fd18c600640 1 -- 172.21.15.161:0/505583051 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1840ca040 con 0x7fd18415d560 2024-05-22T21:20:22.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.086+0000 7fd181600640 1 -- 172.21.15.161:0/505583051 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fd17c0b2020 con 0x7fd18415d560 2024-05-22T21:20:22.089 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:22.089 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":146,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:20:20.260480+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:44.080742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:45.388618+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.30","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:20:22.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.086+0000 7fd18c600640 1 -- 172.21.15.161:0/505583051 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd164066690 msgr2=0x7fd164068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.086+0000 7fd18c600640 1 --2- 172.21.15.161:0/505583051 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd164066690 0x7fd164068b50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd1780011f0 tx=0x7fd178007d40 comp rx=0 tx=0).stop 2024-05-22T21:20:22.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.086+0000 7fd18c600640 1 -- 172.21.15.161:0/505583051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd18415d560 msgr2=0x7fd18409c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.086+0000 7fd18c600640 1 --2- 172.21.15.161:0/505583051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd18415d560 0x7fd18409c000 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0b0040 tx=0x7fd17c019650 comp rx=0 tx=0).stop 2024-05-22T21:20:22.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.086+0000 7fd18c600640 1 -- 172.21.15.161:0/505583051 shutdown_connections 2024-05-22T21:20:22.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.086+0000 7fd18c600640 1 --2- 172.21.15.161:0/505583051 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd164066690 0x7fd164068b50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.086+0000 7fd18c600640 1 --2- 172.21.15.161:0/505583051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd18415d560 0x7fd18409c000 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.086+0000 7fd18c600640 1 --2- 172.21.15.161:0/505583051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd18406bf50 0x7fd1840c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.086+0000 7fd18c600640 1 --2- 172.21.15.161:0/505583051 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd18406b1f0 0x7fd1841588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.086+0000 7fd18c600640 1 -- 172.21.15.161:0/505583051 >> 172.21.15.161:0/505583051 conn(0x7fd1840d0f10 msgr2=0x7fd18406e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:22.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.086+0000 7fd18c600640 1 -- 172.21.15.161:0/505583051 shutdown_connections 2024-05-22T21:20:22.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.086+0000 7fd18c600640 1 -- 172.21.15.161:0/505583051 wait complete. 2024-05-22T21:20:22.112 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-22T21:20:22.112 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:20:22.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a8ee00640 1 Processor -- start 2024-05-22T21:20:22.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a8ee00640 1 -- start start 2024-05-22T21:20:22.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a8ee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a88143ec0 0x7f3a8815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a8ee00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3a880715a0 0x7f3a8806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a8ee00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a881442a0 con 0x7f3a8815ec90 2024-05-22T21:20:22.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a8ee00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a88070260 con 0x7f3a880715a0 2024-05-22T21:20:22.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a8ee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a88070500 con 0x7f3a88143ec0 2024-05-22T21:20:22.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a8ce00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3a8815ec90 0x7f3a8806bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51262/0 (socket says 172.21.15.161:51262) 2024-05-22T21:20:22.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a8ce00640 1 -- 172.21.15.161:0/516195906 learned_addr learned my addr 172.21.15.161:0/516195906 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:22.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a87400640 1 --2- 172.21.15.161:0/516195906 >> v2:172.21.15.179:3300/0 conn(0x7f3a880715a0 0x7f3a8806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a87e00640 1 --2- 172.21.15.161:0/516195906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a88143ec0 0x7f3a8815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a86a00640 1 -- 172.21.15.161:0/516195906 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1062676897 0 0) 0x7f3a881442a0 con 0x7f3a8815ec90 2024-05-22T21:20:22.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a86a00640 1 -- 172.21.15.161:0/516195906 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a6c003530 con 0x7f3a8815ec90 2024-05-22T21:20:22.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a86a00640 1 -- 172.21.15.161:0/516195906 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2493398218 0 0) 0x7f3a6c003530 con 0x7f3a8815ec90 2024-05-22T21:20:22.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a86a00640 1 -- 172.21.15.161:0/516195906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a88143ec0 msgr2=0x7f3a8815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:22.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a86a00640 1 --2- 172.21.15.161:0/516195906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a88143ec0 0x7f3a8815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a86a00640 1 -- 172.21.15.161:0/516195906 >> v2:172.21.15.179:3300/0 conn(0x7f3a880715a0 msgr2=0x7f3a8806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a86a00640 1 --2- 172.21.15.161:0/516195906 >> v2:172.21.15.179:3300/0 conn(0x7f3a880715a0 0x7f3a8806fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a86a00640 1 -- 172.21.15.161:0/516195906 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a880707e0 con 0x7f3a8815ec90 2024-05-22T21:20:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a86a00640 1 -- 172.21.15.161:0/516195906 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f3a80001a00 con 0x7f3a8815ec90 2024-05-22T21:20:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a86a00640 1 -- 172.21.15.161:0/516195906 >> v1:172.21.15.161:6789/0 conn(0x7f3a8815ec90 legacy=0x7f3a8806bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a86a00640 1 --2- 172.21.15.161:0/516195906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a88143ec0 0x7f3a6c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a86a00640 1 --2- 172.21.15.161:0/516195906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a6c003f00 0x7f3a6c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a86a00640 1 --2- 172.21.15.161:0/516195906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a6c004d10 0x7f3a6c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a86a00640 1 -- 172.21.15.161:0/516195906 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a88070260 con 0x7f3a6c003f00 2024-05-22T21:20:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a86a00640 1 -- 172.21.15.161:0/516195906 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a88070500 con 0x7f3a6c004d10 2024-05-22T21:20:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a86a00640 1 -- 172.21.15.161:0/516195906 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c00b990 con 0x7f3a88143ec0 2024-05-22T21:20:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a87400640 1 --2- 172.21.15.161:0/516195906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a6c003f00 0x7f3a6c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a87e00640 1 --2- 172.21.15.161:0/516195906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a88143ec0 0x7f3a6c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a8ce00640 1 --2- 172.21.15.161:0/516195906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a6c004d10 0x7f3a6c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a87400640 1 -- 172.21.15.161:0/516195906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a88143ec0 msgr2=0x7f3a6c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:22.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a87400640 1 --2- 172.21.15.161:0/516195906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a88143ec0 0x7f3a6c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a87400640 1 -- 172.21.15.161:0/516195906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a6c004d10 msgr2=0x7f3a6c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:22.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a87400640 1 --2- 172.21.15.161:0/516195906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a6c004d10 0x7f3a6c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.246+0000 7f3a87400640 1 -- 172.21.15.161:0/516195906 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c003530 con 0x7f3a6c003f00 2024-05-22T21:20:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.250+0000 7f3a87400640 1 --2- 172.21.15.161:0/516195906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a6c003f00 0x7f3a6c0047d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009ae0 tx=0x7f3a7401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea71889dddbe1a4a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.250+0000 7f3a86a00640 1 -- 172.21.15.161:0/516195906 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a740a5590 con 0x7f3a6c003f00 2024-05-22T21:20:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.250+0000 7f3a86a00640 1 -- 172.21.15.161:0/516195906 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a740a56f0 con 0x7f3a6c003f00 2024-05-22T21:20:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.250+0000 7f3a86a00640 1 -- 172.21.15.161:0/516195906 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a740a5aa0 con 0x7f3a6c003f00 2024-05-22T21:20:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.250+0000 7f3a8ee00640 1 -- 172.21.15.161:0/516195906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a6c003f00 msgr2=0x7f3a6c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.250+0000 7f3a8ee00640 1 --2- 172.21.15.161:0/516195906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a6c003f00 0x7f3a6c0047d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009ae0 tx=0x7f3a7401ca50 comp rx=0 tx=0).stop 2024-05-22T21:20:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.250+0000 7f3a8ce00640 1 -- 172.21.15.161:0/516195906 reap_dead start 2024-05-22T21:20:22.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.250+0000 7f3a8ee00640 1 -- 172.21.15.161:0/516195906 shutdown_connections 2024-05-22T21:20:22.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.250+0000 7f3a8ee00640 1 -- 172.21.15.161:0/516195906 >> 172.21.15.161:0/516195906 conn(0x7f3a880b0bc0 msgr2=0x7f3a880b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:22.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.250+0000 7f3a8ee00640 1 -- 172.21.15.161:0/516195906 shutdown_connections 2024-05-22T21:20:22.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.250+0000 7f3a8ee00640 1 -- 172.21.15.161:0/516195906 wait complete. 2024-05-22T21:20:22.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ee00640 1 Processor -- start 2024-05-22T21:20:22.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ee00640 1 -- start start 2024-05-22T21:20:22.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a6c003f00 0x7f3a881e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a881e76f0 0x7f3a8815a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ee00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a8815d8e0 0x7f3a8815dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ee00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a881e8be0 con 0x7f3a8815d8e0 2024-05-22T21:20:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a881e6310 con 0x7f3a6c003f00 2024-05-22T21:20:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8815e2b0 con 0x7f3a881e76f0 2024-05-22T21:20:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a8815d8e0 0x7f3a8815dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a8815d8e0 0x7f3a8815dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45342/0 (socket says 172.21.15.161:45342) 2024-05-22T21:20:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ce00640 1 -- 172.21.15.161:0/2915587961 learned_addr learned my addr 172.21.15.161:0/2915587961 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ce00640 1 -- 172.21.15.161:0/2915587961 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a881e76f0 msgr2=0x7f3a8815a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ce00640 1 --2- 172.21.15.161:0/2915587961 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a881e76f0 0x7f3a8815a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ce00640 1 -- 172.21.15.161:0/2915587961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a6c003f00 msgr2=0x7f3a881e5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ce00640 1 --2- 172.21.15.161:0/2915587961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a6c003f00 0x7f3a881e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ce00640 1 -- 172.21.15.161:0/2915587961 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a881e7f60 con 0x7f3a8815d8e0 2024-05-22T21:20:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ce00640 1 --2- 172.21.15.161:0/2915587961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a8815d8e0 0x7f3a8815dd70 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3a800012f0 tx=0x7f3a80011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a7fe00640 1 -- 172.21.15.161:0/2915587961 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000f920 con 0x7f3a8815d8e0 2024-05-22T21:20:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a7fe00640 1 -- 172.21.15.161:0/2915587961 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8000faa0 con 0x7f3a8815d8e0 2024-05-22T21:20:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ee00640 1 -- 172.21.15.161:0/2915587961 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a881e8240 con 0x7f3a8815d8e0 2024-05-22T21:20:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a7fe00640 1 -- 172.21.15.161:0/2915587961 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000fe50 con 0x7f3a8815d8e0 2024-05-22T21:20:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.254+0000 7f3a8ee00640 1 -- 172.21.15.161:0/2915587961 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a881e8800 con 0x7f3a8815d8e0 2024-05-22T21:20:22.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.258+0000 7f3a8ee00640 1 -- 172.21.15.161:0/2915587961 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a881e8d20 con 0x7f3a8815d8e0 2024-05-22T21:20:22.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.258+0000 7f3a7fe00640 1 -- 172.21.15.161:0/2915587961 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3a8001f460 con 0x7f3a8815d8e0 2024-05-22T21:20:22.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.258+0000 7f3a7fe00640 1 --2- 172.21.15.161:0/2915587961 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a68066690 0x7f3a68068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.258+0000 7f3a7fe00640 1 -- 172.21.15.161:0/2915587961 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3a800871d0 con 0x7f3a8815d8e0 2024-05-22T21:20:22.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.258+0000 7f3a87e00640 1 --2- 172.21.15.161:0/2915587961 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a68066690 0x7f3a68068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.258+0000 7f3a7fe00640 1 -- 172.21.15.161:0/2915587961 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=147}) v3 -- 0x7f3a6806e280 con 0x7f3a8815d8e0 2024-05-22T21:20:22.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.258+0000 7f3a87e00640 1 --2- 172.21.15.161:0/2915587961 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a68066690 0x7f3a68068b50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3a78006d50 tx=0x7f3a78006c40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:22.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.262+0000 7f3a7fe00640 1 -- 172.21.15.161:0/2915587961 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3a881e8d20 con 0x7f3a8815d8e0 2024-05-22T21:20:22.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.446+0000 7f3a8ee00640 1 -- 172.21.15.161:0/2915587961 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a881e8d20 con 0x7f3a8815d8e0 2024-05-22T21:20:22.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.446+0000 7f3a7fe00640 1 -- 172.21.15.161:0/2915587961 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4745 (secure 0 0 0) 0x7f3a881e8d20 con 0x7f3a8815d8e0 2024-05-22T21:20:22.450 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:22.450 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/524372753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":524372753},{"type":"v1","addr":"172.21.15.161:6836","nonce":524372753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3762119379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3762119379},{"type":"v1","addr":"172.21.15.179:6833","nonce":3762119379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3087241366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3087241366},{"type":"v1","addr":"172.21.15.179:6835","nonce":3087241366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:46.384984+0000","modified":"2024-05-22T21:19:49.400019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7572":{"gid":7572,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/3945715002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3945715002},{"type":"v1","addr":"172.21.15.161:6837","nonce":3945715002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-22T21:20:22.451 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 62 2024-05-22T21:20:22.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.450+0000 7f3a8ee00640 1 -- 172.21.15.161:0/2915587961 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a68066690 msgr2=0x7f3a68068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.450+0000 7f3a8ee00640 1 --2- 172.21.15.161:0/2915587961 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a68066690 0x7f3a68068b50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3a78006d50 tx=0x7f3a78006c40 comp rx=0 tx=0).stop 2024-05-22T21:20:22.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.450+0000 7f3a8ee00640 1 -- 172.21.15.161:0/2915587961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a8815d8e0 msgr2=0x7f3a8815dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.450+0000 7f3a8ee00640 1 --2- 172.21.15.161:0/2915587961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a8815d8e0 0x7f3a8815dd70 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3a800012f0 tx=0x7f3a80011c20 comp rx=0 tx=0).stop 2024-05-22T21:20:22.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.450+0000 7f3a8ee00640 1 -- 172.21.15.161:0/2915587961 shutdown_connections 2024-05-22T21:20:22.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.450+0000 7f3a8ee00640 1 --2- 172.21.15.161:0/2915587961 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a68066690 0x7f3a68068b50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.450+0000 7f3a8ee00640 1 --2- 172.21.15.161:0/2915587961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a8815d8e0 0x7f3a8815dd70 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.450+0000 7f3a8ee00640 1 --2- 172.21.15.161:0/2915587961 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a881e76f0 0x7f3a8815a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.450+0000 7f3a8ee00640 1 --2- 172.21.15.161:0/2915587961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a6c003f00 0x7f3a881e5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.450+0000 7f3a8ee00640 1 -- 172.21.15.161:0/2915587961 >> 172.21.15.161:0/2915587961 conn(0x7f3a880b0bc0 msgr2=0x7f3a880cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:22.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.450+0000 7f3a8ee00640 1 -- 172.21.15.161:0/2915587961 shutdown_connections 2024-05-22T21:20:22.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.450+0000 7f3a8ee00640 1 -- 172.21.15.161:0/2915587961 wait complete. 2024-05-22T21:20:22.471 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:20:22.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad8d000640 1 Processor -- start 2024-05-22T21:20:22.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad8d000640 1 -- start start 2024-05-22T21:20:22.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad8d000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fad8815d430 0x7fad880707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad8d000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad88071fd0 con 0x7fad880accf0 2024-05-22T21:20:22.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad8d000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad88070e80 con 0x7fad880ac8b0 2024-05-22T21:20:22.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad8d000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8806c510 con 0x7fad8815d430 2024-05-22T21:20:22.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad86800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fad880accf0 0x7fad88160e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51274/0 (socket says 172.21.15.161:51274) 2024-05-22T21:20:22.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad86800640 1 -- 172.21.15.161:0/3224058349 learned_addr learned my addr 172.21.15.161:0/3224058349 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad85e00640 1 --2- 172.21.15.161:0/3224058349 >> v2:172.21.15.179:3300/0 conn(0x7fad8815d430 0x7fad880707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad85400640 1 -- 172.21.15.161:0/3224058349 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592647551 0 0) 0x7fad88071fd0 con 0x7fad880accf0 2024-05-22T21:20:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad85400640 1 -- 172.21.15.161:0/3224058349 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad6c003530 con 0x7fad880accf0 2024-05-22T21:20:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad85400640 1 -- 172.21.15.161:0/3224058349 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2691737533 0 0) 0x7fad6c003530 con 0x7fad880accf0 2024-05-22T21:20:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad85400640 1 -- 172.21.15.161:0/3224058349 >> v2:172.21.15.179:3300/0 conn(0x7fad8815d430 msgr2=0x7fad880707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad85400640 1 --2- 172.21.15.161:0/3224058349 >> v2:172.21.15.179:3300/0 conn(0x7fad8815d430 0x7fad880707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad85400640 1 -- 172.21.15.161:0/3224058349 >> v1:172.21.15.179:6789/0 conn(0x7fad880ac8b0 legacy=0x7fad8806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad85400640 1 -- 172.21.15.161:0/3224058349 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8806c7f0 con 0x7fad880accf0 2024-05-22T21:20:22.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad85400640 1 -- 172.21.15.161:0/3224058349 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fad80001a00 con 0x7fad880accf0 2024-05-22T21:20:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad85400640 1 -- 172.21.15.161:0/3224058349 >> v1:172.21.15.161:6789/0 conn(0x7fad880accf0 legacy=0x7fad88160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad85400640 1 --2- 172.21.15.161:0/3224058349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad6c003980 0x7fad6c003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad85400640 1 --2- 172.21.15.161:0/3224058349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad6c004380 0x7fad6c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad85400640 1 --2- 172.21.15.161:0/3224058349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad6c0050f0 0x7fad6c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad85400640 1 -- 172.21.15.161:0/3224058349 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8806c510 con 0x7fad6c0050f0 2024-05-22T21:20:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad85400640 1 -- 172.21.15.161:0/3224058349 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c00c030 con 0x7fad6c004380 2024-05-22T21:20:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad85400640 1 -- 172.21.15.161:0/3224058349 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c00c1f0 con 0x7fad6c003980 2024-05-22T21:20:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.606+0000 7fad86800640 1 --2- 172.21.15.161:0/3224058349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad6c0050f0 0x7fad6c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.610+0000 7fad7d400640 1 --2- 172.21.15.161:0/3224058349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad6c004380 0x7fad6c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.610+0000 7fad85e00640 1 --2- 172.21.15.161:0/3224058349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad6c003980 0x7fad6c003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.610+0000 7fad86800640 1 -- 172.21.15.161:0/3224058349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad6c003980 msgr2=0x7fad6c003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.610+0000 7fad86800640 1 --2- 172.21.15.161:0/3224058349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad6c003980 0x7fad6c003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.610+0000 7fad86800640 1 -- 172.21.15.161:0/3224058349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad6c004380 msgr2=0x7fad6c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:22.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.610+0000 7fad86800640 1 --2- 172.21.15.161:0/3224058349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad6c004380 0x7fad6c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.610+0000 7fad86800640 1 -- 172.21.15.161:0/3224058349 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c003530 con 0x7fad6c0050f0 2024-05-22T21:20:22.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.610+0000 7fad86800640 1 -- 172.21.15.161:0/3224058349 reap_dead start 2024-05-22T21:20:22.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.610+0000 7fad86800640 1 --2- 172.21.15.161:0/3224058349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad6c0050f0 0x7fad6c00b990 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fad88071cd0 tx=0x7fad8001d2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3334f7d0e85bcbb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:22.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.610+0000 7fad85400640 1 -- 172.21.15.161:0/3224058349 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad800ad630 con 0x7fad6c0050f0 2024-05-22T21:20:22.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.610+0000 7fad85400640 1 -- 172.21.15.161:0/3224058349 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad800a8070 con 0x7fad6c0050f0 2024-05-22T21:20:22.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.610+0000 7fad85400640 1 -- 172.21.15.161:0/3224058349 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad800ad9f0 con 0x7fad6c0050f0 2024-05-22T21:20:22.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad8d000640 1 -- 172.21.15.161:0/3224058349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad6c0050f0 msgr2=0x7fad6c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad8d000640 1 --2- 172.21.15.161:0/3224058349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad6c0050f0 0x7fad6c00b990 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fad88071cd0 tx=0x7fad8001d2f0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad8d000640 1 -- 172.21.15.161:0/3224058349 shutdown_connections 2024-05-22T21:20:22.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad8d000640 1 --2- 172.21.15.161:0/3224058349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad6c0050f0 0x7fad6c00b990 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad8d000640 1 -- 172.21.15.161:0/3224058349 >> 172.21.15.161:0/3224058349 conn(0x7fad880b0bc0 msgr2=0x7fad880b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:22.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad8d000640 1 -- 172.21.15.161:0/3224058349 shutdown_connections 2024-05-22T21:20:22.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad8d000640 1 -- 172.21.15.161:0/3224058349 wait complete. 2024-05-22T21:20:22.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad8d000640 1 Processor -- start 2024-05-22T21:20:22.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad8d000640 1 -- start start 2024-05-22T21:20:22.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad8d000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad6c0050f0 0x7fad88160400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad8d000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad880b1200 0x7fad881565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad8d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad880b1680 0x7fad88159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad8d000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad88156160 con 0x7fad880b1680 2024-05-22T21:20:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad8d000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad88155b80 con 0x7fad880b1200 2024-05-22T21:20:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad8d000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad881585c0 con 0x7fad6c0050f0 2024-05-22T21:20:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad86800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad880b1680 0x7fad88159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad86800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad880b1680 0x7fad88159e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45366/0 (socket says 172.21.15.161:45366) 2024-05-22T21:20:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad86800640 1 -- 172.21.15.161:0/3182671114 learned_addr learned my addr 172.21.15.161:0/3182671114 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad85e00640 1 --2- 172.21.15.161:0/3182671114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad6c0050f0 0x7fad88160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad7d400640 1 --2- 172.21.15.161:0/3182671114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad880b1200 0x7fad881565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad86800640 1 -- 172.21.15.161:0/3182671114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad6c0050f0 msgr2=0x7fad88160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad86800640 1 --2- 172.21.15.161:0/3182671114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad6c0050f0 0x7fad88160400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad86800640 1 -- 172.21.15.161:0/3182671114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad880b1200 msgr2=0x7fad881565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad86800640 1 --2- 172.21.15.161:0/3182671114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad880b1200 0x7fad881565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad86800640 1 -- 172.21.15.161:0/3182671114 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad88155df0 con 0x7fad880b1680 2024-05-22T21:20:22.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad85e00640 1 --2- 172.21.15.161:0/3182671114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad6c0050f0 0x7fad88160400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:22.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.614+0000 7fad7d400640 1 --2- 172.21.15.161:0/3182671114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad880b1200 0x7fad881565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:22.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.618+0000 7fad86800640 1 --2- 172.21.15.161:0/3182671114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad880b1680 0x7fad88159e70 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fad8001db90 tx=0x7fad6c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:22.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.618+0000 7fad7ea00640 1 -- 172.21.15.161:0/3182671114 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80001a00 con 0x7fad880b1680 2024-05-22T21:20:22.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.618+0000 7fad7ea00640 1 -- 172.21.15.161:0/3182671114 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad80001b60 con 0x7fad880b1680 2024-05-22T21:20:22.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.618+0000 7fad7ea00640 1 -- 172.21.15.161:0/3182671114 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad800ad630 con 0x7fad880b1680 2024-05-22T21:20:22.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.618+0000 7fad8d000640 1 -- 172.21.15.161:0/3182671114 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad88150760 con 0x7fad880b1680 2024-05-22T21:20:22.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.618+0000 7fad8d000640 1 -- 172.21.15.161:0/3182671114 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad88158170 con 0x7fad880b1680 2024-05-22T21:20:22.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.618+0000 7fad8d000640 1 -- 172.21.15.161:0/3182671114 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad881562d0 con 0x7fad880b1680 2024-05-22T21:20:22.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.618+0000 7fad7ea00640 1 -- 172.21.15.161:0/3182671114 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fad800b6070 con 0x7fad880b1680 2024-05-22T21:20:22.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.618+0000 7fad7ea00640 1 --2- 172.21.15.161:0/3182671114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fad640666c0 0x7fad64068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.618+0000 7fad7ea00640 1 -- 172.21.15.161:0/3182671114 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fad8011d890 con 0x7fad880b1680 2024-05-22T21:20:22.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.618+0000 7fad85e00640 1 --2- 172.21.15.161:0/3182671114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fad640666c0 0x7fad64068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.622+0000 7fad7ea00640 1 -- 172.21.15.161:0/3182671114 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=147}) v3 -- 0x7fad6406e2b0 con 0x7fad880b1680 2024-05-22T21:20:22.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.622+0000 7fad85e00640 1 --2- 172.21.15.161:0/3182671114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fad640666c0 0x7fad64068b80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fad70009b60 tx=0x7fad70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:22.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.622+0000 7fad7ea00640 1 -- 172.21.15.161:0/3182671114 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fad800eabe0 con 0x7fad880b1680 2024-05-22T21:20:22.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.810+0000 7fad8d000640 1 -- 172.21.15.161:0/3182671114 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad88067220 con 0x7fad880b1680 2024-05-22T21:20:22.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.810+0000 7fad7ea00640 1 -- 172.21.15.161:0/3182671114 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4745 (secure 0 0 0) 0x7fad800ee200 con 0x7fad880b1680 2024-05-22T21:20:22.813 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:22.813 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/524372753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":524372753},{"type":"v1","addr":"172.21.15.161:6836","nonce":524372753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3762119379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3762119379},{"type":"v1","addr":"172.21.15.179:6833","nonce":3762119379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3087241366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3087241366},{"type":"v1","addr":"172.21.15.179:6835","nonce":3087241366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:19:46.384984+0000","modified":"2024-05-22T21:19:49.400019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7572":{"gid":7572,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/3945715002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3945715002},{"type":"v1","addr":"172.21.15.161:6837","nonce":3945715002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-05-22T21:20:22.814 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 62 2024-05-22T21:20:22.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.810+0000 7fad8d000640 1 -- 172.21.15.161:0/3182671114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fad640666c0 msgr2=0x7fad64068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.810+0000 7fad8d000640 1 --2- 172.21.15.161:0/3182671114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fad640666c0 0x7fad64068b80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fad70009b60 tx=0x7fad70008040 comp rx=0 tx=0).stop 2024-05-22T21:20:22.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.810+0000 7fad8d000640 1 -- 172.21.15.161:0/3182671114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad880b1680 msgr2=0x7fad88159e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.810+0000 7fad8d000640 1 --2- 172.21.15.161:0/3182671114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad880b1680 0x7fad88159e70 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fad8001db90 tx=0x7fad6c005800 comp rx=0 tx=0).stop 2024-05-22T21:20:22.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.810+0000 7fad8d000640 1 -- 172.21.15.161:0/3182671114 shutdown_connections 2024-05-22T21:20:22.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.810+0000 7fad8d000640 1 --2- 172.21.15.161:0/3182671114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fad640666c0 0x7fad64068b80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.810+0000 7fad8d000640 1 --2- 172.21.15.161:0/3182671114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad880b1680 0x7fad88159e70 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.810+0000 7fad8d000640 1 --2- 172.21.15.161:0/3182671114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad880b1200 0x7fad881565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.810+0000 7fad8d000640 1 --2- 172.21.15.161:0/3182671114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fad6c0050f0 0x7fad88160400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.810+0000 7fad8d000640 1 -- 172.21.15.161:0/3182671114 >> 172.21.15.161:0/3182671114 conn(0x7fad880b0bc0 msgr2=0x7fad880aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:22.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.810+0000 7fad8d000640 1 -- 172.21.15.161:0/3182671114 shutdown_connections 2024-05-22T21:20:22.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.810+0000 7fad8d000640 1 -- 172.21.15.161:0/3182671114 wait complete. 2024-05-22T21:20:22.834 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:20:22.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.966+0000 7f06a1a00640 1 Processor -- start 2024-05-22T21:20:22.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.966+0000 7f06a1a00640 1 -- start start 2024-05-22T21:20:22.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.966+0000 7f06a1a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f069c070db0 0x7f069c06cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.966+0000 7f06a1a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f069c0b2960 0x7f069c0b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.966+0000 7f06a1a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069c159bd0 con 0x7f069c1597f0 2024-05-22T21:20:22.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.966+0000 7f06a1a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069c0cbdd0 con 0x7f069c0b2960 2024-05-22T21:20:22.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.966+0000 7f06a1a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0cc050 con 0x7f069c070db0 2024-05-22T21:20:22.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.966+0000 7f069b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f069c0b2960 0x7f069c0b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.966+0000 7f069b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f069c0b2960 0x7f069c0b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45372/0 (socket says 172.21.15.161:45372) 2024-05-22T21:20:22.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.966+0000 7f069b400640 1 -- 172.21.15.161:0/2972418114 learned_addr learned my addr 172.21.15.161:0/2972418114 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:22.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.966+0000 7f069a000640 1 --2- 172.21.15.161:0/2972418114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f069c070db0 0x7f069c06cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.966+0000 7f069b400640 1 -- 172.21.15.161:0/2972418114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f069c070db0 msgr2=0x7f069c06cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:22.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.970+0000 7f069b400640 1 --2- 172.21.15.161:0/2972418114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f069c070db0 0x7f069c06cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.970+0000 7f069b400640 1 -- 172.21.15.161:0/2972418114 >> v1:172.21.15.161:6789/0 conn(0x7f069c1597f0 legacy=0x7f069c0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.970+0000 7f069b400640 1 -- 172.21.15.161:0/2972418114 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c0b3720 con 0x7f069c0b2960 2024-05-22T21:20:22.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.970+0000 7f069b400640 1 --2- 172.21.15.161:0/2972418114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f069c0b2960 0x7f069c0b2df0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f0694009870 tx=0x7f069401e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2606f876919c1f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:22.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.970+0000 7f0699600640 1 -- 172.21.15.161:0/2972418114 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06940a7730 con 0x7f069c0b2960 2024-05-22T21:20:22.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.970+0000 7f0699600640 1 -- 172.21.15.161:0/2972418114 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06940a7890 con 0x7f069c0b2960 2024-05-22T21:20:22.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.970+0000 7f0699600640 1 -- 172.21.15.161:0/2972418114 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06940a7c20 con 0x7f069c0b2960 2024-05-22T21:20:22.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.970+0000 7f06a1a00640 1 -- 172.21.15.161:0/2972418114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f069c0b2960 msgr2=0x7f069c0b2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.970+0000 7f06a1a00640 1 --2- 172.21.15.161:0/2972418114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f069c0b2960 0x7f069c0b2df0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f0694009870 tx=0x7f069401e770 comp rx=0 tx=0).stop 2024-05-22T21:20:22.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.970+0000 7f06a1a00640 1 -- 172.21.15.161:0/2972418114 shutdown_connections 2024-05-22T21:20:22.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.970+0000 7f06a1a00640 1 --2- 172.21.15.161:0/2972418114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f069c0b2960 0x7f069c0b2df0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.970+0000 7f06a1a00640 1 --2- 172.21.15.161:0/2972418114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f069c070db0 0x7f069c06cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.970+0000 7f06a1a00640 1 -- 172.21.15.161:0/2972418114 >> 172.21.15.161:0/2972418114 conn(0x7f069c0b0bc0 msgr2=0x7f069c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:22.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.970+0000 7f06a1a00640 1 -- 172.21.15.161:0/2972418114 shutdown_connections 2024-05-22T21:20:22.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.970+0000 7f06a1a00640 1 -- 172.21.15.161:0/2972418114 wait complete. 2024-05-22T21:20:22.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f06a1a00640 1 Processor -- start 2024-05-22T21:20:22.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f06a1a00640 1 -- start start 2024-05-22T21:20:22.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f06a1a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f069c070db0 0x7f069c13e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f06a1a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f069c0b2960 0x7f069c148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f06a1a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f069c1597f0 0x7f069c06d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f06a1a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0ac8b0 con 0x7f069c070db0 2024-05-22T21:20:22.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f06a1a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c159d60 con 0x7f069c1597f0 2024-05-22T21:20:22.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f06a1a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c15a040 con 0x7f069c0b2960 2024-05-22T21:20:22.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f069aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f069c070db0 0x7f069c13e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f069aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f069c070db0 0x7f069c13e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45382/0 (socket says 172.21.15.161:45382) 2024-05-22T21:20:22.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f069aa00640 1 -- 172.21.15.161:0/3088265646 learned_addr learned my addr 172.21.15.161:0/3088265646 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:22.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f069a000640 1 --2- 172.21.15.161:0/3088265646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f069c0b2960 0x7f069c148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f069b400640 1 --2- 172.21.15.161:0/3088265646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f069c1597f0 0x7f069c06d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f069aa00640 1 -- 172.21.15.161:0/3088265646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f069c0b2960 msgr2=0x7f069c148c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f069aa00640 1 --2- 172.21.15.161:0/3088265646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f069c0b2960 0x7f069c148c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f069aa00640 1 -- 172.21.15.161:0/3088265646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f069c1597f0 msgr2=0x7f069c06d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:22.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f069aa00640 1 --2- 172.21.15.161:0/3088265646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f069c1597f0 0x7f069c06d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:22.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f069aa00640 1 -- 172.21.15.161:0/3088265646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06940094d0 con 0x7f069c070db0 2024-05-22T21:20:22.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f069aa00640 1 --2- 172.21.15.161:0/3088265646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f069c070db0 0x7f069c13e170 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f068c00c7a0 tx=0x7f068c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:22.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f0692a00640 1 -- 172.21.15.161:0/3088265646 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c003b80 con 0x7f069c070db0 2024-05-22T21:20:22.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f0692a00640 1 -- 172.21.15.161:0/3088265646 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068c003ce0 con 0x7f069c070db0 2024-05-22T21:20:22.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f0692a00640 1 -- 172.21.15.161:0/3088265646 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c00aa90 con 0x7f069c070db0 2024-05-22T21:20:22.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f06a1a00640 1 -- 172.21.15.161:0/3088265646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c15a320 con 0x7f069c070db0 2024-05-22T21:20:22.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.974+0000 7f06a1a00640 1 -- 172.21.15.161:0/3088265646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c15a8e0 con 0x7f069c070db0 2024-05-22T21:20:22.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.978+0000 7f06a1a00640 1 -- 172.21.15.161:0/3088265646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c13f3c0 con 0x7f069c070db0 2024-05-22T21:20:22.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.978+0000 7f0692a00640 1 -- 172.21.15.161:0/3088265646 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f068c00abf0 con 0x7f069c070db0 2024-05-22T21:20:22.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.978+0000 7f0692a00640 1 --2- 172.21.15.161:0/3088265646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f067c066690 0x7f067c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:22.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.978+0000 7f0692a00640 1 -- 172.21.15.161:0/3088265646 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f068c0840c0 con 0x7f069c070db0 2024-05-22T21:20:22.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.978+0000 7f069a000640 1 --2- 172.21.15.161:0/3088265646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f067c066690 0x7f067c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:22.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.978+0000 7f0692a00640 1 -- 172.21.15.161:0/3088265646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=147}) v3 -- 0x7f067c06e280 con 0x7f069c070db0 2024-05-22T21:20:22.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.978+0000 7f069a000640 1 --2- 172.21.15.161:0/3088265646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f067c066690 0x7f067c068b50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f069c149770 tx=0x7f0688008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:22.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:22.982+0000 7f0692a00640 1 -- 172.21.15.161:0/3088265646 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f068c051350 con 0x7f069c070db0 2024-05-22T21:20:23.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.138+0000 7f06a1a00640 1 -- 172.21.15.161:0/3088265646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f069c067220 con 0x7f069c070db0 2024-05-22T21:20:23.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.142+0000 7f0692a00640 1 -- 172.21.15.161:0/3088265646 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f068c054970 con 0x7f069c070db0 2024-05-22T21:20:23.144 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:23.144 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":146,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:20:20.260480+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:19:44.080742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-22T21:19:45.388618+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.30","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:20:23.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.142+0000 7f06a1a00640 1 -- 172.21.15.161:0/3088265646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f067c066690 msgr2=0x7f067c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:23.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.142+0000 7f06a1a00640 1 --2- 172.21.15.161:0/3088265646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f067c066690 0x7f067c068b50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f069c149770 tx=0x7f0688008040 comp rx=0 tx=0).stop 2024-05-22T21:20:23.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.142+0000 7f06a1a00640 1 -- 172.21.15.161:0/3088265646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f069c070db0 msgr2=0x7f069c13e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:23.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.142+0000 7f06a1a00640 1 --2- 172.21.15.161:0/3088265646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f069c070db0 0x7f069c13e170 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f068c00c7a0 tx=0x7f068c00cc60 comp rx=0 tx=0).stop 2024-05-22T21:20:23.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.142+0000 7f06a1a00640 1 -- 172.21.15.161:0/3088265646 shutdown_connections 2024-05-22T21:20:23.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.142+0000 7f06a1a00640 1 --2- 172.21.15.161:0/3088265646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f067c066690 0x7f067c068b50 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.142+0000 7f06a1a00640 1 --2- 172.21.15.161:0/3088265646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f069c1597f0 0x7f069c06d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.142+0000 7f06a1a00640 1 --2- 172.21.15.161:0/3088265646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f069c0b2960 0x7f069c148c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.142+0000 7f06a1a00640 1 --2- 172.21.15.161:0/3088265646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f069c070db0 0x7f069c13e170 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.142+0000 7f06a1a00640 1 -- 172.21.15.161:0/3088265646 >> 172.21.15.161:0/3088265646 conn(0x7f069c0b0bc0 msgr2=0x7f069c14a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:23.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.142+0000 7f06a1a00640 1 -- 172.21.15.161:0/3088265646 shutdown_connections 2024-05-22T21:20:23.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.142+0000 7f06a1a00640 1 -- 172.21.15.161:0/3088265646 wait complete. 2024-05-22T21:20:23.165 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-22T21:20:23.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.298+0000 7fbc36200640 1 Processor -- start 2024-05-22T21:20:23.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.298+0000 7fbc36200640 1 -- start start 2024-05-22T21:20:23.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.298+0000 7fbc36200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbc3013cfc0 0x7fbc301402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:23.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.298+0000 7fbc36200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbc3013fe50 0x7fbc3006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:23.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.298+0000 7fbc36200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3013d3a0 con 0x7fbc3015af60 2024-05-22T21:20:23.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.298+0000 7fbc36200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc300cd920 con 0x7fbc3013fe50 2024-05-22T21:20:23.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.298+0000 7fbc36200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc300cdc00 con 0x7fbc3013cfc0 2024-05-22T21:20:23.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.302+0000 7fbc2fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbc3013fe50 0x7fbc3006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:23.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.302+0000 7fbc2fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbc3013fe50 0x7fbc3006fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45392/0 (socket says 172.21.15.161:45392) 2024-05-22T21:20:23.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.302+0000 7fbc2fe00640 1 -- 172.21.15.161:0/928199522 learned_addr learned my addr 172.21.15.161:0/928199522 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:23.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.302+0000 7fbc2f400640 1 --2- 172.21.15.161:0/928199522 >> v2:172.21.15.179:3300/0 conn(0x7fbc3013cfc0 0x7fbc301402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:23.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.302+0000 7fbc2fe00640 1 -- 172.21.15.161:0/928199522 >> v2:172.21.15.179:3300/0 conn(0x7fbc3013cfc0 msgr2=0x7fbc301402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:23.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.302+0000 7fbc2fe00640 1 --2- 172.21.15.161:0/928199522 >> v2:172.21.15.179:3300/0 conn(0x7fbc3013cfc0 0x7fbc301402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.302+0000 7fbc2fe00640 1 -- 172.21.15.161:0/928199522 >> v1:172.21.15.179:6789/0 conn(0x7fbc3015af60 legacy=0x7fbc3013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:23.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.302+0000 7fbc2fe00640 1 -- 172.21.15.161:0/928199522 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc300cdee0 con 0x7fbc3013fe50 2024-05-22T21:20:23.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.302+0000 7fbc2f400640 1 --2- 172.21.15.161:0/928199522 >> v2:172.21.15.179:3300/0 conn(0x7fbc3013cfc0 0x7fbc301402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:23.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.302+0000 7fbc2fe00640 1 --2- 172.21.15.161:0/928199522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3013fe50 0x7fbc3006fce0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fbc3006de90 tx=0x7fbc2801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf9e64da118970d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:23.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.302+0000 7fbc2ea00640 1 -- 172.21.15.161:0/928199522 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc280a6430 con 0x7fbc3013fe50 2024-05-22T21:20:23.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.302+0000 7fbc2ea00640 1 -- 172.21.15.161:0/928199522 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc280a6590 con 0x7fbc3013fe50 2024-05-22T21:20:23.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.302+0000 7fbc2ea00640 1 -- 172.21.15.161:0/928199522 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc280a6920 con 0x7fbc3013fe50 2024-05-22T21:20:23.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc36200640 1 -- 172.21.15.161:0/928199522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3013fe50 msgr2=0x7fbc3006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:23.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc36200640 1 --2- 172.21.15.161:0/928199522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3013fe50 0x7fbc3006fce0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fbc3006de90 tx=0x7fbc2801c930 comp rx=0 tx=0).stop 2024-05-22T21:20:23.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc36200640 1 -- 172.21.15.161:0/928199522 shutdown_connections 2024-05-22T21:20:23.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc36200640 1 --2- 172.21.15.161:0/928199522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3013fe50 0x7fbc3006fce0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc36200640 1 --2- 172.21.15.161:0/928199522 >> v2:172.21.15.179:3300/0 conn(0x7fbc3013cfc0 0x7fbc301402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc36200640 1 -- 172.21.15.161:0/928199522 >> 172.21.15.161:0/928199522 conn(0x7fbc3006a2d0 msgr2=0x7fbc3006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:23.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc36200640 1 -- 172.21.15.161:0/928199522 shutdown_connections 2024-05-22T21:20:23.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc36200640 1 -- 172.21.15.161:0/928199522 wait complete. 2024-05-22T21:20:23.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc36200640 1 Processor -- start 2024-05-22T21:20:23.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc36200640 1 -- start start 2024-05-22T21:20:23.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc36200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc3013cfc0 0x7fbc300b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:23.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc36200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3013fe50 0x7fbc301593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:23.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc36200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc3015af60 0x7fbc3013a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:23.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc36200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc30157e00 con 0x7fbc3013fe50 2024-05-22T21:20:23.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc36200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3015b9d0 con 0x7fbc3013cfc0 2024-05-22T21:20:23.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc36200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3015bcb0 con 0x7fbc3015af60 2024-05-22T21:20:23.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc26a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3013fe50 0x7fbc301593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:23.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc26a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3013fe50 0x7fbc301593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45398/0 (socket says 172.21.15.161:45398) 2024-05-22T21:20:23.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc26a00640 1 -- 172.21.15.161:0/1321281989 learned_addr learned my addr 172.21.15.161:0/1321281989 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:23.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc2f400640 1 --2- 172.21.15.161:0/1321281989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc3013cfc0 0x7fbc300b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:23.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc2fe00640 1 --2- 172.21.15.161:0/1321281989 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc3015af60 0x7fbc3013a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:23.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc26a00640 1 -- 172.21.15.161:0/1321281989 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc3015af60 msgr2=0x7fbc3013a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:23.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc26a00640 1 --2- 172.21.15.161:0/1321281989 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc3015af60 0x7fbc3013a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc26a00640 1 -- 172.21.15.161:0/1321281989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc3013cfc0 msgr2=0x7fbc300b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:23.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc26a00640 1 --2- 172.21.15.161:0/1321281989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc3013cfc0 0x7fbc300b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc26a00640 1 -- 172.21.15.161:0/1321281989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc28009b30 con 0x7fbc3013fe50 2024-05-22T21:20:23.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.306+0000 7fbc2fe00640 1 --2- 172.21.15.161:0/1321281989 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc3015af60 0x7fbc3013a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:23.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.310+0000 7fbc26a00640 1 --2- 172.21.15.161:0/1321281989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3013fe50 0x7fbc301593b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800c3c0 tx=0x7fbc1800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:23.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.310+0000 7fbc27e00640 1 -- 172.21.15.161:0/1321281989 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc180052d0 con 0x7fbc3013fe50 2024-05-22T21:20:23.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.310+0000 7fbc27e00640 1 -- 172.21.15.161:0/1321281989 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc18005430 con 0x7fbc3013fe50 2024-05-22T21:20:23.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.310+0000 7fbc36200640 1 -- 172.21.15.161:0/1321281989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3015bf90 con 0x7fbc3013fe50 2024-05-22T21:20:23.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.310+0000 7fbc27e00640 1 -- 172.21.15.161:0/1321281989 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc18003840 con 0x7fbc3013fe50 2024-05-22T21:20:23.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.310+0000 7fbc36200640 1 -- 172.21.15.161:0/1321281989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3015c550 con 0x7fbc3013fe50 2024-05-22T21:20:23.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.310+0000 7fbc36200640 1 -- 172.21.15.161:0/1321281989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3015c770 con 0x7fbc3013fe50 2024-05-22T21:20:23.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.310+0000 7fbc27e00640 1 -- 172.21.15.161:0/1321281989 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbc180039a0 con 0x7fbc3013fe50 2024-05-22T21:20:23.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.310+0000 7fbc27e00640 1 --2- 172.21.15.161:0/1321281989 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc0c066690 0x7fbc0c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:23.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.310+0000 7fbc27e00640 1 -- 172.21.15.161:0/1321281989 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbc180836d0 con 0x7fbc3013fe50 2024-05-22T21:20:23.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.310+0000 7fbc2f400640 1 --2- 172.21.15.161:0/1321281989 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc0c066690 0x7fbc0c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:23.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.310+0000 7fbc27e00640 1 -- 172.21.15.161:0/1321281989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=147}) v3 -- 0x7fbc0c06e280 con 0x7fbc3013fe50 2024-05-22T21:20:23.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.310+0000 7fbc2f400640 1 --2- 172.21.15.161:0/1321281989 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc0c066690 0x7fbc0c068b50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbc301f9580 tx=0x7fbc20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:23.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.314+0000 7fbc27e00640 1 -- 172.21.15.161:0/1321281989 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fbc18050a10 con 0x7fbc3013fe50 2024-05-22T21:20:23.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.502+0000 7fbc36200640 1 -- 172.21.15.161:0/1321281989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbc30067220 con 0x7fbc3013fe50 2024-05-22T21:20:23.505 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:20:23.502+0000 7f7d32800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-22T21:20:23.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.506+0000 7fbc27e00640 1 -- 172.21.15.161:0/1321281989 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(147..147 src has 1..147) v4 ==== 374+0+0 (secure 0 0 0) 0x7fbc18058450 con 0x7fbc3013fe50 2024-05-22T21:20:23.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.506+0000 7fbc27e00640 1 -- 172.21.15.161:0/1321281989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=148}) v3 -- 0x7fbc0c06eb30 con 0x7fbc3013fe50 2024-05-22T21:20:23.516 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.514+0000 7fbc27e00640 1 -- 172.21.15.161:0/1321281989 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbc18017030 con 0x7fbc3013fe50 2024-05-22T21:20:23.516 INFO:teuthology.orchestra.run.smithi161.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-22T21:20:23.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.514+0000 7fbc36200640 1 -- 172.21.15.161:0/1321281989 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc0c066690 msgr2=0x7fbc0c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:23.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.514+0000 7fbc36200640 1 --2- 172.21.15.161:0/1321281989 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc0c066690 0x7fbc0c068b50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbc301f9580 tx=0x7fbc20008040 comp rx=0 tx=0).stop 2024-05-22T21:20:23.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.514+0000 7fbc36200640 1 -- 172.21.15.161:0/1321281989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3013fe50 msgr2=0x7fbc301593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:23.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.514+0000 7fbc36200640 1 --2- 172.21.15.161:0/1321281989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3013fe50 0x7fbc301593b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800c3c0 tx=0x7fbc1800c880 comp rx=0 tx=0).stop 2024-05-22T21:20:23.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.514+0000 7fbc36200640 1 -- 172.21.15.161:0/1321281989 shutdown_connections 2024-05-22T21:20:23.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.514+0000 7fbc36200640 1 --2- 172.21.15.161:0/1321281989 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fbc0c066690 0x7fbc0c068b50 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.514+0000 7fbc36200640 1 --2- 172.21.15.161:0/1321281989 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc3015af60 0x7fbc3013a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.514+0000 7fbc36200640 1 --2- 172.21.15.161:0/1321281989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3013fe50 0x7fbc301593b0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.514+0000 7fbc36200640 1 --2- 172.21.15.161:0/1321281989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc3013cfc0 0x7fbc300b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.514+0000 7fbc36200640 1 -- 172.21.15.161:0/1321281989 >> 172.21.15.161:0/1321281989 conn(0x7fbc3006a2d0 msgr2=0x7fbc3015da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:23.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.514+0000 7fbc36200640 1 -- 172.21.15.161:0/1321281989 shutdown_connections 2024-05-22T21:20:23.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.514+0000 7fbc36200640 1 -- 172.21.15.161:0/1321281989 wait complete. 2024-05-22T21:20:23.528 INFO:tasks.ceph.mds.c.smithi161.stderr: -8829> 2024-05-22T21:19:51.991+0000 7ff574a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T21:20:23.541 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-22T21:20:23.583 INFO:tasks.ceph.mds.c.smithi161.stdout:starting mds.c at 2024-05-22T21:20:23.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f0697c00640 1 Processor -- start 2024-05-22T21:20:23.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f0697c00640 1 -- start start 2024-05-22T21:20:23.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f0697c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0690143c40 0x7f069015b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:23.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f0697c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06901444a0 0x7f069006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:23.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f0697c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069015efb0 con 0x7f0690144900 2024-05-22T21:20:23.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f0697c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069015f290 con 0x7f0690143c40 2024-05-22T21:20:23.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f0697c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069015f570 con 0x7f06901444a0 2024-05-22T21:20:23.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f0695c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0690144900 0x7f06900b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51288/0 (socket says 172.21.15.161:51288) 2024-05-22T21:20:23.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f0695c00640 1 -- 172.21.15.161:0/325817683 learned_addr learned my addr 172.21.15.161:0/325817683 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:23.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f068fe00640 1 --2- 172.21.15.161:0/325817683 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06901444a0 0x7f069006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:23.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f0695200640 1 --2- 172.21.15.161:0/325817683 >> v2:172.21.15.179:3300/0 conn(0x7f0690143c40 0x7f069015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:23.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f068fe00640 1 -- 172.21.15.161:0/325817683 >> v2:172.21.15.179:3300/0 conn(0x7f0690143c40 msgr2=0x7f069015b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:23.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f068fe00640 1 --2- 172.21.15.161:0/325817683 >> v2:172.21.15.179:3300/0 conn(0x7f0690143c40 0x7f069015b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f068fe00640 1 -- 172.21.15.161:0/325817683 >> v1:172.21.15.161:6789/0 conn(0x7f0690144900 legacy=0x7f06900b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:23.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f068f400640 1 -- 172.21.15.161:0/325817683 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816294408 0 0) 0x7f069015efb0 con 0x7f0690144900 2024-05-22T21:20:23.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f068fe00640 1 -- 172.21.15.161:0/325817683 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069015f850 con 0x7f06901444a0 2024-05-22T21:20:23.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.682+0000 7f0695200640 1 --2- 172.21.15.161:0/325817683 >> v2:172.21.15.179:3300/0 conn(0x7f0690143c40 0x7f069015b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:20:23.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.686+0000 7f068fe00640 1 --2- 172.21.15.161:0/325817683 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06901444a0 0x7f069006c2a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f067c0097c0 tx=0x7f067c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36289094e9b6c4ec server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:23.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.686+0000 7f068f400640 1 -- 172.21.15.161:0/325817683 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c0ac410 con 0x7f06901444a0 2024-05-22T21:20:23.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.686+0000 7f068f400640 1 -- 172.21.15.161:0/325817683 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c0ac570 con 0x7f06901444a0 2024-05-22T21:20:23.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.686+0000 7f068f400640 1 -- 172.21.15.161:0/325817683 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c0ac900 con 0x7f06901444a0 2024-05-22T21:20:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.686+0000 7f0697c00640 1 -- 172.21.15.161:0/325817683 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06901444a0 msgr2=0x7f069006c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.686+0000 7f0697c00640 1 --2- 172.21.15.161:0/325817683 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06901444a0 0x7f069006c2a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f067c0097c0 tx=0x7f067c01c4a0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.686+0000 7f0697c00640 1 -- 172.21.15.161:0/325817683 shutdown_connections 2024-05-22T21:20:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.686+0000 7f0697c00640 1 --2- 172.21.15.161:0/325817683 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f06901444a0 0x7f069006c2a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.686+0000 7f0697c00640 1 --2- 172.21.15.161:0/325817683 >> v2:172.21.15.179:3300/0 conn(0x7f0690143c40 0x7f069015b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.686+0000 7f0697c00640 1 -- 172.21.15.161:0/325817683 >> 172.21.15.161:0/325817683 conn(0x7f069006a2d0 msgr2=0x7f069006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.686+0000 7f0697c00640 1 -- 172.21.15.161:0/325817683 shutdown_connections 2024-05-22T21:20:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.686+0000 7f0697c00640 1 -- 172.21.15.161:0/325817683 wait complete. 2024-05-22T21:20:23.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f0697c00640 1 Processor -- start 2024-05-22T21:20:23.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f0697c00640 1 -- start start 2024-05-22T21:20:23.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f0697c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0690143c40 0x7f06901427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:23.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f0697c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06901444a0 0x7f0690135ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:23.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f0697c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0690144900 0x7f0690157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:23.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f0697c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069015d240 con 0x7f06901444a0 2024-05-22T21:20:23.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f0697c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069015cc70 con 0x7f0690144900 2024-05-22T21:20:23.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f0697c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0690142d10 con 0x7f0690143c40 2024-05-22T21:20:23.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f068fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06901444a0 0x7f0690135ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:23.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f068fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06901444a0 0x7f0690135ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45430/0 (socket says 172.21.15.161:45430) 2024-05-22T21:20:23.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f068fe00640 1 -- 172.21.15.161:0/1158795715 learned_addr learned my addr 172.21.15.161:0/1158795715 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:23.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f068fe00640 1 -- 172.21.15.161:0/1158795715 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0690143c40 msgr2=0x7f06901427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:23.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f068fe00640 1 --2- 172.21.15.161:0/1158795715 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0690143c40 0x7f06901427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f068fe00640 1 -- 172.21.15.161:0/1158795715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0690144900 msgr2=0x7f0690157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:23.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f068fe00640 1 --2- 172.21.15.161:0/1158795715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0690144900 0x7f0690157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:23.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f068fe00640 1 -- 172.21.15.161:0/1158795715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0680009a70 con 0x7f06901444a0 2024-05-22T21:20:23.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f068fe00640 1 --2- 172.21.15.161:0/1158795715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06901444a0 0x7f0690135ca0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f067c01a0d0 tx=0x7f067c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:23.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f068cc00640 1 -- 172.21.15.161:0/1158795715 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c006e20 con 0x7f06901444a0 2024-05-22T21:20:23.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f068cc00640 1 -- 172.21.15.161:0/1158795715 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c0b5640 con 0x7f06901444a0 2024-05-22T21:20:23.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f068cc00640 1 -- 172.21.15.161:0/1158795715 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c0b59d0 con 0x7f06901444a0 2024-05-22T21:20:23.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f0697c00640 1 -- 172.21.15.161:0/1158795715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067c009420 con 0x7f06901444a0 2024-05-22T21:20:23.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.690+0000 7f0697c00640 1 -- 172.21.15.161:0/1158795715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0690142fb0 con 0x7f06901444a0 2024-05-22T21:20:23.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.694+0000 7f0697c00640 1 -- 172.21.15.161:0/1158795715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0690067220 con 0x7f06901444a0 2024-05-22T21:20:23.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.694+0000 7f068cc00640 1 -- 172.21.15.161:0/1158795715 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f067c0b3040 con 0x7f06901444a0 2024-05-22T21:20:23.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.694+0000 7f068cc00640 1 --2- 172.21.15.161:0/1158795715 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f06700666c0 0x7f0670068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:23.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.694+0000 7f068cc00640 1 -- 172.21.15.161:0/1158795715 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f067c11e1e0 con 0x7f06901444a0 2024-05-22T21:20:23.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.694+0000 7f0695200640 1 --2- 172.21.15.161:0/1158795715 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f06700666c0 0x7f0670068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:23.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.694+0000 7f068cc00640 1 -- 172.21.15.161:0/1158795715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=148}) v3 -- 0x7f067006e3c0 con 0x7f06901444a0 2024-05-22T21:20:23.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.694+0000 7f0695200640 1 --2- 172.21.15.161:0/1158795715 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f06700666c0 0x7f0670068b80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f068000e330 tx=0x7f068000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:23.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.698+0000 7f068cc00640 1 -- 172.21.15.161:0/1158795715 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f067c0eb420 con 0x7f06901444a0 2024-05-22T21:20:23.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:23.886+0000 7f0697c00640 1 -- 172.21.15.161:0/1158795715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f069015d3f0 con 0x7f06901444a0 2024-05-22T21:20:24.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.526+0000 7f068cc00640 1 -- 172.21.15.161:0/1158795715 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7f069015d3f0 con 0x7f06901444a0 2024-05-22T21:20:24.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.530+0000 7f0697c00640 1 -- 172.21.15.161:0/1158795715 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f06700666c0 msgr2=0x7f0670068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:24.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.530+0000 7f0697c00640 1 --2- 172.21.15.161:0/1158795715 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f06700666c0 0x7f0670068b80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f068000e330 tx=0x7f068000b040 comp rx=0 tx=0).stop 2024-05-22T21:20:24.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.530+0000 7f0697c00640 1 -- 172.21.15.161:0/1158795715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06901444a0 msgr2=0x7f0690135ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:24.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.530+0000 7f0697c00640 1 --2- 172.21.15.161:0/1158795715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06901444a0 0x7f0690135ca0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f067c01a0d0 tx=0x7f067c019240 comp rx=0 tx=0).stop 2024-05-22T21:20:24.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.530+0000 7f0697c00640 1 -- 172.21.15.161:0/1158795715 shutdown_connections 2024-05-22T21:20:24.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.530+0000 7f0697c00640 1 --2- 172.21.15.161:0/1158795715 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f06700666c0 0x7f0670068b80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:24.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.530+0000 7f0697c00640 1 --2- 172.21.15.161:0/1158795715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0690144900 0x7f0690157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:24.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.530+0000 7f0697c00640 1 --2- 172.21.15.161:0/1158795715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06901444a0 0x7f0690135ca0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:24.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.530+0000 7f0697c00640 1 --2- 172.21.15.161:0/1158795715 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0690143c40 0x7f06901427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:24.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.530+0000 7f0697c00640 1 -- 172.21.15.161:0/1158795715 >> 172.21.15.161:0/1158795715 conn(0x7f069006a2d0 msgr2=0x7f0690156c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:24.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.530+0000 7f0697c00640 1 -- 172.21.15.161:0/1158795715 shutdown_connections 2024-05-22T21:20:24.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.530+0000 7f0697c00640 1 -- 172.21.15.161:0/1158795715 wait complete. 2024-05-22T21:20:24.555 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-22T21:20:24.606 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:20:24.650 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:20:24.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.738+0000 7f2475800640 1 Processor -- start 2024-05-22T21:20:24.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.738+0000 7f2475800640 1 -- start start 2024-05-22T21:20:24.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.738+0000 7f2475800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f247006e270 0x7f247015f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:24.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.738+0000 7f2475800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f247015b6f0 0x7f247015bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:24.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.738+0000 7f2475800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24700cb9b0 con 0x7f24700d0a30 2024-05-22T21:20:24.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.738+0000 7f2475800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24700cbe40 con 0x7f247006e270 2024-05-22T21:20:24.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.738+0000 7f2475800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24700b6c50 con 0x7f247015b6f0 2024-05-22T21:20:24.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.738+0000 7f246f400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f247015b6f0 0x7f247015bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:24.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.738+0000 7f246f400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f247015b6f0 0x7f247015bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:59264/0 (socket says 172.21.15.161:59264) 2024-05-22T21:20:24.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.738+0000 7f246f400640 1 -- 172.21.15.161:0/3197523383 learned_addr learned my addr 172.21.15.161:0/3197523383 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:24.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.738+0000 7f246ea00640 1 --2- 172.21.15.161:0/3197523383 >> v2:172.21.15.179:3300/0 conn(0x7f247006e270 0x7f247015f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:24.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.738+0000 7f246f400640 1 -- 172.21.15.161:0/3197523383 >> v2:172.21.15.179:3300/0 conn(0x7f247006e270 msgr2=0x7f247015f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:24.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.738+0000 7f246f400640 1 --2- 172.21.15.161:0/3197523383 >> v2:172.21.15.179:3300/0 conn(0x7f247006e270 0x7f247015f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:24.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.738+0000 7f246f400640 1 -- 172.21.15.161:0/3197523383 >> v1:172.21.15.179:6789/0 conn(0x7f24700d0a30 legacy=0x7f247006e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:24.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.738+0000 7f246f400640 1 -- 172.21.15.161:0/3197523383 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247006f0a0 con 0x7f247015b6f0 2024-05-22T21:20:24.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.742+0000 7f246f400640 1 --2- 172.21.15.161:0/3197523383 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f247015b6f0 0x7f247015bb80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2468009870 tx=0x7f246801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d243ef730b2e7869 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:24.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.742+0000 7f246d600640 1 -- 172.21.15.161:0/3197523383 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24680a9620 con 0x7f247015b6f0 2024-05-22T21:20:24.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.742+0000 7f246d600640 1 -- 172.21.15.161:0/3197523383 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24680a9780 con 0x7f247015b6f0 2024-05-22T21:20:24.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.742+0000 7f246d600640 1 -- 172.21.15.161:0/3197523383 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24680a9b10 con 0x7f247015b6f0 2024-05-22T21:20:24.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.742+0000 7f2475800640 1 -- 172.21.15.161:0/3197523383 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f247015b6f0 msgr2=0x7f247015bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:24.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.742+0000 7f2475800640 1 --2- 172.21.15.161:0/3197523383 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f247015b6f0 0x7f247015bb80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2468009870 tx=0x7f246801c560 comp rx=0 tx=0).stop 2024-05-22T21:20:24.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.742+0000 7f2475800640 1 -- 172.21.15.161:0/3197523383 shutdown_connections 2024-05-22T21:20:24.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.742+0000 7f2475800640 1 --2- 172.21.15.161:0/3197523383 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f247015b6f0 0x7f247015bb80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:24.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.742+0000 7f2475800640 1 --2- 172.21.15.161:0/3197523383 >> v2:172.21.15.179:3300/0 conn(0x7f247006e270 0x7f247015f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:24.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.742+0000 7f2475800640 1 -- 172.21.15.161:0/3197523383 >> 172.21.15.161:0/3197523383 conn(0x7f24700d0ff0 msgr2=0x7f24700d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:24.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.742+0000 7f2475800640 1 -- 172.21.15.161:0/3197523383 shutdown_connections 2024-05-22T21:20:24.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.742+0000 7f2475800640 1 -- 172.21.15.161:0/3197523383 wait complete. 2024-05-22T21:20:24.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f2475800640 1 Processor -- start 2024-05-22T21:20:24.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f2475800640 1 -- start start 2024-05-22T21:20:24.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f2475800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f247006e270 0x7f247006afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:24.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f2475800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f24700d0a30 0x7f2470151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:24.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f2475800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f247015b6f0 0x7f24700c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:24.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f2475800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24700b2160 con 0x7f247015b6f0 2024-05-22T21:20:24.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f2475800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247006b520 con 0x7f247006e270 2024-05-22T21:20:24.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f2475800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24700c6980 con 0x7f24700d0a30 2024-05-22T21:20:24.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f246f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f247015b6f0 0x7f24700c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:24.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f246f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f247015b6f0 0x7f24700c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45444/0 (socket says 172.21.15.161:45444) 2024-05-22T21:20:24.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f246f400640 1 -- 172.21.15.161:0/3755155139 learned_addr learned my addr 172.21.15.161:0/3755155139 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:24.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f246ea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f247006e270 0x7f247006afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:24.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f246e000640 1 --2- 172.21.15.161:0/3755155139 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f24700d0a30 0x7f2470151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:24.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f246f400640 1 -- 172.21.15.161:0/3755155139 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f24700d0a30 msgr2=0x7f2470151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:24.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f246f400640 1 --2- 172.21.15.161:0/3755155139 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f24700d0a30 0x7f2470151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:24.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f246f400640 1 -- 172.21.15.161:0/3755155139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f247006e270 msgr2=0x7f247006afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:24.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f246f400640 1 --2- 172.21.15.161:0/3755155139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f247006e270 0x7f247006afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:24.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f246f400640 1 -- 172.21.15.161:0/3755155139 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24680094d0 con 0x7f247015b6f0 2024-05-22T21:20:24.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f246f400640 1 --2- 172.21.15.161:0/3755155139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f247015b6f0 0x7f24700c6d80 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f246801ac10 tx=0x7f2468019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:24.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f2466a00640 1 -- 172.21.15.161:0/3755155139 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24680bb640 con 0x7f247015b6f0 2024-05-22T21:20:24.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f2466a00640 1 -- 172.21.15.161:0/3755155139 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24680bb7a0 con 0x7f247015b6f0 2024-05-22T21:20:24.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f2475800640 1 -- 172.21.15.161:0/3755155139 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24700b1dc0 con 0x7f247015b6f0 2024-05-22T21:20:24.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f2466a00640 1 -- 172.21.15.161:0/3755155139 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24680bbb50 con 0x7f247015b6f0 2024-05-22T21:20:24.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.746+0000 7f2475800640 1 -- 172.21.15.161:0/3755155139 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24700c66a0 con 0x7f247015b6f0 2024-05-22T21:20:24.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.750+0000 7f2475800640 1 -- 172.21.15.161:0/3755155139 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2470073ce0 con 0x7f247015b6f0 2024-05-22T21:20:24.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.750+0000 7f2466a00640 1 -- 172.21.15.161:0/3755155139 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f24680bbcb0 con 0x7f247015b6f0 2024-05-22T21:20:24.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.750+0000 7f2466a00640 1 --2- 172.21.15.161:0/3755155139 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2450066690 0x7f2450068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:24.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.750+0000 7f2466a00640 1 -- 172.21.15.161:0/3755155139 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f246811e6e0 con 0x7f247015b6f0 2024-05-22T21:20:24.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.750+0000 7f246ea00640 1 --2- 172.21.15.161:0/3755155139 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2450066690 0x7f2450068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:24.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.750+0000 7f2466a00640 1 -- 172.21.15.161:0/3755155139 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=148}) v3 -- 0x7f245006e390 con 0x7f247015b6f0 2024-05-22T21:20:24.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.750+0000 7f246ea00640 1 --2- 172.21.15.161:0/3755155139 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2450066690 0x7f2450068b50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f24600113e0 tx=0x7f246000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:24.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.754+0000 7f2466a00640 1 -- 172.21.15.161:0/3755155139 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f24680eba60 con 0x7f247015b6f0 2024-05-22T21:20:24.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:24.906+0000 7f2475800640 1 -- 172.21.15.161:0/3755155139 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f24700b22a0 con 0x7f247015b6f0 2024-05-22T21:20:25.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.530+0000 7f2466a00640 1 -- 172.21.15.161:0/3755155139 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(148..148 src has 1..148) v4 ==== 296+0+0 (secure 0 0 0) 0x7f24680ef720 con 0x7f247015b6f0 2024-05-22T21:20:25.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.530+0000 7f2466a00640 1 -- 172.21.15.161:0/3755155139 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=149}) v3 -- 0x7f245006c7c0 con 0x7f247015b6f0 2024-05-22T21:20:25.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.534+0000 7f2466a00640 1 -- 172.21.15.161:0/3755155139 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v148) v1 ==== 181+0+0 (secure 0 0 0) 0x7f24680c0070 con 0x7f247015b6f0 2024-05-22T21:20:25.541 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' removed 2024-05-22T21:20:25.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.542+0000 7f2475800640 1 -- 172.21.15.161:0/3755155139 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2450066690 msgr2=0x7f2450068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:25.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.542+0000 7f2475800640 1 --2- 172.21.15.161:0/3755155139 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2450066690 0x7f2450068b50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f24600113e0 tx=0x7f246000b040 comp rx=0 tx=0).stop 2024-05-22T21:20:25.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.542+0000 7f2475800640 1 -- 172.21.15.161:0/3755155139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f247015b6f0 msgr2=0x7f24700c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:25.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.542+0000 7f2475800640 1 --2- 172.21.15.161:0/3755155139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f247015b6f0 0x7f24700c6d80 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f246801ac10 tx=0x7f2468019300 comp rx=0 tx=0).stop 2024-05-22T21:20:25.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.542+0000 7f2475800640 1 -- 172.21.15.161:0/3755155139 shutdown_connections 2024-05-22T21:20:25.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.542+0000 7f2475800640 1 --2- 172.21.15.161:0/3755155139 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2450066690 0x7f2450068b50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:25.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.542+0000 7f2475800640 1 --2- 172.21.15.161:0/3755155139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f247015b6f0 0x7f24700c6d80 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:25.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.542+0000 7f2475800640 1 --2- 172.21.15.161:0/3755155139 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f24700d0a30 0x7f2470151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:25.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.542+0000 7f2475800640 1 --2- 172.21.15.161:0/3755155139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f247006e270 0x7f247006afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:25.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.542+0000 7f2475800640 1 -- 172.21.15.161:0/3755155139 >> 172.21.15.161:0/3755155139 conn(0x7f24700d0ff0 msgr2=0x7f247015abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:25.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.542+0000 7f2475800640 1 -- 172.21.15.161:0/3755155139 shutdown_connections 2024-05-22T21:20:25.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.542+0000 7f2475800640 1 -- 172.21.15.161:0/3755155139 wait complete. 2024-05-22T21:20:25.565 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-22T21:20:25.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.702+0000 7f6dc4c00640 1 Processor -- start 2024-05-22T21:20:25.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.702+0000 7f6dc4c00640 1 -- start start 2024-05-22T21:20:25.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.702+0000 7f6dc4c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6dc013d560 0x7f6dc00cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:25.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.702+0000 7f6dc4c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6dc015e1d0 0x7f6dc00ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:25.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.702+0000 7f6dc4c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc00ce580 con 0x7f6dc013cfe0 2024-05-22T21:20:25.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.702+0000 7f6dc4c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc0140560 con 0x7f6dc013d560 2024-05-22T21:20:25.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.702+0000 7f6dc4c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc00ce830 con 0x7f6dc015e1d0 2024-05-22T21:20:25.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.702+0000 7f6db5400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6dc013d560 0x7f6dc00cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:25.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.702+0000 7f6dbde00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f6dc013cfe0 0x7f6dc015dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:51294/0 (socket says 172.21.15.161:51294) 2024-05-22T21:20:25.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.702+0000 7f6dbde00640 1 -- 172.21.15.161:0/522098991 learned_addr learned my addr 172.21.15.161:0/522098991 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:25.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.702+0000 7f6dbe800640 1 --2- 172.21.15.161:0/522098991 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6dc015e1d0 0x7f6dc00ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:25.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.702+0000 7f6db5400640 1 -- 172.21.15.161:0/522098991 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6dc015e1d0 msgr2=0x7f6dc00ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:25.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.702+0000 7f6db5400640 1 --2- 172.21.15.161:0/522098991 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6dc015e1d0 0x7f6dc00ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:25.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.702+0000 7f6db5400640 1 -- 172.21.15.161:0/522098991 >> v1:172.21.15.161:6789/0 conn(0x7f6dc013cfe0 legacy=0x7f6dc015dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:25.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.702+0000 7f6db5400640 1 -- 172.21.15.161:0/522098991 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc00ceb10 con 0x7f6dc013d560 2024-05-22T21:20:25.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.702+0000 7f6dbd400640 1 -- 172.21.15.161:0/522098991 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771191745 0 0) 0x7f6dc00ce580 con 0x7f6dc013cfe0 2024-05-22T21:20:25.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.706+0000 7f6db5400640 1 --2- 172.21.15.161:0/522098991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc013d560 0x7f6dc00cdb00 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009d90 tx=0x7f6da801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a790d6878da812bd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:25.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.706+0000 7f6dbd400640 1 -- 172.21.15.161:0/522098991 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da80a5970 con 0x7f6dc013d560 2024-05-22T21:20:25.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.706+0000 7f6dbd400640 1 -- 172.21.15.161:0/522098991 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da80a5ad0 con 0x7f6dc013d560 2024-05-22T21:20:25.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.706+0000 7f6dbd400640 1 -- 172.21.15.161:0/522098991 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da80a5e60 con 0x7f6dc013d560 2024-05-22T21:20:25.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.706+0000 7f6dc4c00640 1 -- 172.21.15.161:0/522098991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc013d560 msgr2=0x7f6dc00cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:25.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.706+0000 7f6dc4c00640 1 --2- 172.21.15.161:0/522098991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc013d560 0x7f6dc00cdb00 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009d90 tx=0x7f6da801c8b0 comp rx=0 tx=0).stop 2024-05-22T21:20:25.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.706+0000 7f6dc4c00640 1 -- 172.21.15.161:0/522098991 shutdown_connections 2024-05-22T21:20:25.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.706+0000 7f6dc4c00640 1 --2- 172.21.15.161:0/522098991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc013d560 0x7f6dc00cdb00 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:25.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.706+0000 7f6dc4c00640 1 --2- 172.21.15.161:0/522098991 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6dc015e1d0 0x7f6dc00ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:25.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.706+0000 7f6dc4c00640 1 -- 172.21.15.161:0/522098991 >> 172.21.15.161:0/522098991 conn(0x7f6dc006a0a0 msgr2=0x7f6dc006a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:25.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.706+0000 7f6dc4c00640 1 -- 172.21.15.161:0/522098991 shutdown_connections 2024-05-22T21:20:25.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.706+0000 7f6dc4c00640 1 -- 172.21.15.161:0/522098991 wait complete. 2024-05-22T21:20:25.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6dc4c00640 1 Processor -- start 2024-05-22T21:20:25.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6dc4c00640 1 -- start start 2024-05-22T21:20:25.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6dc4c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6dc013cfe0 0x7f6dc015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:25.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6dc4c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc013d560 0x7f6dc0143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:25.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6dc4c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dc015e1d0 0x7f6dc0135650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:25.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6dc4c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc01380b0 con 0x7f6dc013d560 2024-05-22T21:20:25.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6dc4c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc0142d60 con 0x7f6dc015e1d0 2024-05-22T21:20:25.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6dc4c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc0135b90 con 0x7f6dc013cfe0 2024-05-22T21:20:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6db5400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc013d560 0x7f6dc0143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6db5400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc013d560 0x7f6dc0143120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:45470/0 (socket says 172.21.15.161:45470) 2024-05-22T21:20:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6db5400640 1 -- 172.21.15.161:0/3342883503 learned_addr learned my addr 172.21.15.161:0/3342883503 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6dbe800640 1 --2- 172.21.15.161:0/3342883503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dc015e1d0 0x7f6dc0135650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6dbde00640 1 --2- 172.21.15.161:0/3342883503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6dc013cfe0 0x7f6dc015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6db5400640 1 -- 172.21.15.161:0/3342883503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6dc013cfe0 msgr2=0x7f6dc015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6db5400640 1 --2- 172.21.15.161:0/3342883503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6dc013cfe0 0x7f6dc015cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6db5400640 1 -- 172.21.15.161:0/3342883503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dc015e1d0 msgr2=0x7f6dc0135650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6db5400640 1 --2- 172.21.15.161:0/3342883503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dc015e1d0 0x7f6dc0135650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6db5400640 1 -- 172.21.15.161:0/3342883503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da8009a70 con 0x7f6dc013d560 2024-05-22T21:20:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6db5400640 1 --2- 172.21.15.161:0/3342883503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc013d560 0x7f6dc0143120 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009d90 tx=0x7f6da8019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6db6a00640 1 -- 172.21.15.161:0/3342883503 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da80a5970 con 0x7f6dc013d560 2024-05-22T21:20:25.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6db6a00640 1 -- 172.21.15.161:0/3342883503 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da8009450 con 0x7f6dc013d560 2024-05-22T21:20:25.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6db6a00640 1 -- 172.21.15.161:0/3342883503 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da80b7650 con 0x7f6dc013d560 2024-05-22T21:20:25.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6dc4c00640 1 -- 172.21.15.161:0/3342883503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc015c8d0 con 0x7f6dc013d560 2024-05-22T21:20:25.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6dc4c00640 1 -- 172.21.15.161:0/3342883503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc0137dd0 con 0x7f6dc013d560 2024-05-22T21:20:25.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.710+0000 7f6dc4c00640 1 -- 172.21.15.161:0/3342883503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc0067220 con 0x7f6dc013d560 2024-05-22T21:20:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.714+0000 7f6db6a00640 1 -- 172.21.15.161:0/3342883503 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6da80b4050 con 0x7f6dc013d560 2024-05-22T21:20:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.714+0000 7f6db6a00640 1 --2- 172.21.15.161:0/3342883503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6d9c066690 0x7f6d9c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.714+0000 7f6db6a00640 1 -- 172.21.15.161:0/3342883503 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f6da811f430 con 0x7f6dc013d560 2024-05-22T21:20:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.714+0000 7f6dbde00640 1 --2- 172.21.15.161:0/3342883503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6d9c066690 0x7f6d9c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.714+0000 7f6db6a00640 1 -- 172.21.15.161:0/3342883503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=149}) v3 -- 0x7f6d9c06d520 con 0x7f6dc013d560 2024-05-22T21:20:25.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.714+0000 7f6dbde00640 1 --2- 172.21.15.161:0/3342883503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6d9c066690 0x7f6d9c068b50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6db0000f70 tx=0x7f6db0007b60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:25.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.718+0000 7f6db6a00640 1 -- 172.21.15.161:0/3342883503 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6da80b2020 con 0x7f6dc013d560 2024-05-22T21:20:25.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:25.874+0000 7f6dc4c00640 1 -- 172.21.15.161:0/3342883503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6dc00ccc60 con 0x7f6dc013d560 2024-05-22T21:20:26.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.538+0000 7f6db6a00640 1 -- 172.21.15.161:0/3342883503 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(149..149 src has 1..149) v4 ==== 330+0+0 (secure 0 0 0) 0x7f6da80f0b50 con 0x7f6dc013d560 2024-05-22T21:20:26.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.538+0000 7f6db6a00640 1 -- 172.21.15.161:0/3342883503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=150}) v3 -- 0x7f6d9c06c2b0 con 0x7f6dc013d560 2024-05-22T21:20:26.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.542+0000 7f6db6a00640 1 -- 172.21.15.161:0/3342883503 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v149) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6da80ec790 con 0x7f6dc013d560 2024-05-22T21:20:26.548 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' removed 2024-05-22T21:20:26.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.550+0000 7f6dc4c00640 1 -- 172.21.15.161:0/3342883503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6d9c066690 msgr2=0x7f6d9c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:26.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.550+0000 7f6dc4c00640 1 --2- 172.21.15.161:0/3342883503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6d9c066690 0x7f6d9c068b50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6db0000f70 tx=0x7f6db0007b60 comp rx=0 tx=0).stop 2024-05-22T21:20:26.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.550+0000 7f6dc4c00640 1 -- 172.21.15.161:0/3342883503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc013d560 msgr2=0x7f6dc0143120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:26.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.550+0000 7f6dc4c00640 1 --2- 172.21.15.161:0/3342883503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc013d560 0x7f6dc0143120 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009d90 tx=0x7f6da8019650 comp rx=0 tx=0).stop 2024-05-22T21:20:26.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.550+0000 7f6dc4c00640 1 -- 172.21.15.161:0/3342883503 shutdown_connections 2024-05-22T21:20:26.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.550+0000 7f6dc4c00640 1 --2- 172.21.15.161:0/3342883503 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6d9c066690 0x7f6d9c068b50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:26.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.550+0000 7f6dc4c00640 1 --2- 172.21.15.161:0/3342883503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dc015e1d0 0x7f6dc0135650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:26.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.550+0000 7f6dc4c00640 1 --2- 172.21.15.161:0/3342883503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc013d560 0x7f6dc0143120 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:26.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.550+0000 7f6dc4c00640 1 --2- 172.21.15.161:0/3342883503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6dc013cfe0 0x7f6dc015cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:26.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.550+0000 7f6dc4c00640 1 -- 172.21.15.161:0/3342883503 >> 172.21.15.161:0/3342883503 conn(0x7f6dc006a0a0 msgr2=0x7f6dc013f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:26.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.550+0000 7f6dc4c00640 1 -- 172.21.15.161:0/3342883503 shutdown_connections 2024-05-22T21:20:26.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.550+0000 7f6dc4c00640 1 -- 172.21.15.161:0/3342883503 wait complete. 2024-05-22T21:20:26.574 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-22T21:20:26.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.706+0000 7fb511a00640 1 Processor -- start 2024-05-22T21:20:26.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.706+0000 7fb511a00640 1 -- start start 2024-05-22T21:20:26.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.710+0000 7fb511a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb50c13cfc0 0x7fb50c1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:26.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.710+0000 7fb511a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50c13d3a0 con 0x7fb50c15af60 2024-05-22T21:20:26.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.710+0000 7fb511a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50c0cd920 con 0x7fb50c13fe50 2024-05-22T21:20:26.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.710+0000 7fb511a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c0cdc00 con 0x7fb50c13cfc0 2024-05-22T21:20:26.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.710+0000 7fb50a000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb50c15af60 0x7fb50c13f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54174/0 (socket says 172.21.15.161:54174) 2024-05-22T21:20:26.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.710+0000 7fb50a000640 1 -- 172.21.15.161:0/1064405605 learned_addr learned my addr 172.21.15.161:0/1064405605 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:26.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.710+0000 7fb50aa00640 1 --2- 172.21.15.161:0/1064405605 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb50c13cfc0 0x7fb50c1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:26.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.710+0000 7fb509600640 1 -- 172.21.15.161:0/1064405605 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062786328 0 0) 0x7fb50c13d3a0 con 0x7fb50c15af60 2024-05-22T21:20:26.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.710+0000 7fb509600640 1 -- 172.21.15.161:0/1064405605 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4f0003530 con 0x7fb50c15af60 2024-05-22T21:20:26.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.710+0000 7fb509600640 1 -- 172.21.15.161:0/1064405605 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128798376 0 0) 0x7fb50c0cd920 con 0x7fb50c13fe50 2024-05-22T21:20:26.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.710+0000 7fb509600640 1 -- 172.21.15.161:0/1064405605 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb50c13d3a0 con 0x7fb50c13fe50 2024-05-22T21:20:26.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.710+0000 7fb50aa00640 1 -- 172.21.15.161:0/1064405605 >> v1:172.21.15.179:6789/0 conn(0x7fb50c13fe50 legacy=0x7fb50c06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:26.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.710+0000 7fb50aa00640 1 -- 172.21.15.161:0/1064405605 >> v1:172.21.15.161:6789/0 conn(0x7fb50c15af60 legacy=0x7fb50c13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:26.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.710+0000 7fb50aa00640 1 -- 172.21.15.161:0/1064405605 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c0cdee0 con 0x7fb50c13cfc0 2024-05-22T21:20:26.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.714+0000 7fb50aa00640 1 --2- 172.21.15.161:0/1064405605 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb50c13cfc0 0x7fb50c1402e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009dc0 tx=0x7fb4fc01c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c87e324f883c97a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:26.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.714+0000 7fb509600640 1 -- 172.21.15.161:0/1064405605 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc0a5790 con 0x7fb50c13cfc0 2024-05-22T21:20:26.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.714+0000 7fb509600640 1 -- 172.21.15.161:0/1064405605 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4fc0a58f0 con 0x7fb50c13cfc0 2024-05-22T21:20:26.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.714+0000 7fb509600640 1 -- 172.21.15.161:0/1064405605 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc0a5c80 con 0x7fb50c13cfc0 2024-05-22T21:20:26.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.714+0000 7fb511a00640 1 -- 172.21.15.161:0/1064405605 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb50c13cfc0 msgr2=0x7fb50c1402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:26.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.714+0000 7fb511a00640 1 --2- 172.21.15.161:0/1064405605 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb50c13cfc0 0x7fb50c1402e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009dc0 tx=0x7fb4fc01c8d0 comp rx=0 tx=0).stop 2024-05-22T21:20:26.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.714+0000 7fb511a00640 1 -- 172.21.15.161:0/1064405605 shutdown_connections 2024-05-22T21:20:26.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.714+0000 7fb511a00640 1 --2- 172.21.15.161:0/1064405605 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb50c13cfc0 0x7fb50c1402e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:26.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.714+0000 7fb511a00640 1 -- 172.21.15.161:0/1064405605 >> 172.21.15.161:0/1064405605 conn(0x7fb50c06a2d0 msgr2=0x7fb50c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:26.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.714+0000 7fb511a00640 1 -- 172.21.15.161:0/1064405605 shutdown_connections 2024-05-22T21:20:26.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.714+0000 7fb511a00640 1 -- 172.21.15.161:0/1064405605 wait complete. 2024-05-22T21:20:26.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.714+0000 7fb511a00640 1 Processor -- start 2024-05-22T21:20:26.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.714+0000 7fb511a00640 1 -- start start 2024-05-22T21:20:26.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb511a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb50c13cfc0 0x7fb50c1571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:26.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb511a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb50c13fe50 0x7fb50c0a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:26.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb511a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb50c15af60 0x7fb50c156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:26.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb511a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c157af0 con 0x7fb50c15af60 2024-05-22T21:20:26.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb511a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c15cba0 con 0x7fb50c13cfc0 2024-05-22T21:20:26.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb511a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c15c000 con 0x7fb50c13fe50 2024-05-22T21:20:26.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb50b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb50c15af60 0x7fb50c156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:26.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb50b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb50c15af60 0x7fb50c156a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54926/0 (socket says 172.21.15.161:54926) 2024-05-22T21:20:26.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb50b400640 1 -- 172.21.15.161:0/2768868959 learned_addr learned my addr 172.21.15.161:0/2768868959 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:26.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb50a000640 1 --2- 172.21.15.161:0/2768868959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb50c13fe50 0x7fb50c0a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:26.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb50aa00640 1 --2- 172.21.15.161:0/2768868959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb50c13cfc0 0x7fb50c1571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:26.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb50b400640 1 -- 172.21.15.161:0/2768868959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb50c13fe50 msgr2=0x7fb50c0a8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:26.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb50b400640 1 --2- 172.21.15.161:0/2768868959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb50c13fe50 0x7fb50c0a8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:26.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb50b400640 1 -- 172.21.15.161:0/2768868959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb50c13cfc0 msgr2=0x7fb50c1571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:26.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb50b400640 1 --2- 172.21.15.161:0/2768868959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb50c13cfc0 0x7fb50c1571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:26.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb50b400640 1 -- 172.21.15.161:0/2768868959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc009a70 con 0x7fb50c15af60 2024-05-22T21:20:26.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb50aa00640 1 --2- 172.21.15.161:0/2768868959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb50c13cfc0 0x7fb50c1571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:26.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb50b400640 1 --2- 172.21.15.161:0/2768868959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb50c15af60 0x7fb50c156a50 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb50400c440 tx=0x7fb50400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:26.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb502a00640 1 -- 172.21.15.161:0/2768868959 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5040052d0 con 0x7fb50c15af60 2024-05-22T21:20:26.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb502a00640 1 -- 172.21.15.161:0/2768868959 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb504005430 con 0x7fb50c15af60 2024-05-22T21:20:26.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb502a00640 1 -- 172.21.15.161:0/2768868959 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504012830 con 0x7fb50c15af60 2024-05-22T21:20:26.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb511a00640 1 -- 172.21.15.161:0/2768868959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50c0a93f0 con 0x7fb50c15af60 2024-05-22T21:20:26.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb511a00640 1 -- 172.21.15.161:0/2768868959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50c15c550 con 0x7fb50c15af60 2024-05-22T21:20:26.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb511a00640 1 -- 172.21.15.161:0/2768868959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50c15cd30 con 0x7fb50c15af60 2024-05-22T21:20:26.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.718+0000 7fb502a00640 1 -- 172.21.15.161:0/2768868959 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb504012990 con 0x7fb50c15af60 2024-05-22T21:20:26.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.722+0000 7fb502a00640 1 --2- 172.21.15.161:0/2768868959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb4ec066690 0x7fb4ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:26.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.722+0000 7fb502a00640 1 -- 172.21.15.161:0/2768868959 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb504083990 con 0x7fb50c15af60 2024-05-22T21:20:26.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.722+0000 7fb50aa00640 1 --2- 172.21.15.161:0/2768868959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb4ec066690 0x7fb4ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:26.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.722+0000 7fb50aa00640 1 --2- 172.21.15.161:0/2768868959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb4ec066690 0x7fb4ec068b50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0af000 tx=0x7fb4fc0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:26.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.726+0000 7fb502a00640 1 -- 172.21.15.161:0/2768868959 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb504050f60 con 0x7fb50c15af60 2024-05-22T21:20:26.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.878+0000 7fb511a00640 1 -- 172.21.15.161:0/2768868959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb50c067220 con 0x7fb50c15af60 2024-05-22T21:20:26.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.886+0000 7fb502a00640 1 -- 172.21.15.161:0/2768868959 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb504054580 con 0x7fb50c15af60 2024-05-22T21:20:26.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.886+0000 7fb511a00640 1 -- 172.21.15.161:0/2768868959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb4ec066690 msgr2=0x7fb4ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:26.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.886+0000 7fb511a00640 1 --2- 172.21.15.161:0/2768868959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb4ec066690 0x7fb4ec068b50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0af000 tx=0x7fb4fc0a7070 comp rx=0 tx=0).stop 2024-05-22T21:20:26.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.886+0000 7fb511a00640 1 -- 172.21.15.161:0/2768868959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb50c15af60 msgr2=0x7fb50c156a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:26.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.886+0000 7fb511a00640 1 --2- 172.21.15.161:0/2768868959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb50c15af60 0x7fb50c156a50 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb50400c440 tx=0x7fb50400c900 comp rx=0 tx=0).stop 2024-05-22T21:20:26.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.886+0000 7fb511a00640 1 -- 172.21.15.161:0/2768868959 shutdown_connections 2024-05-22T21:20:26.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.886+0000 7fb511a00640 1 --2- 172.21.15.161:0/2768868959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb4ec066690 0x7fb4ec068b50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:26.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.886+0000 7fb511a00640 1 --2- 172.21.15.161:0/2768868959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb50c15af60 0x7fb50c156a50 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:26.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.886+0000 7fb511a00640 1 --2- 172.21.15.161:0/2768868959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb50c13fe50 0x7fb50c0a8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:26.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.886+0000 7fb511a00640 1 --2- 172.21.15.161:0/2768868959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb50c13cfc0 0x7fb50c1571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:26.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.886+0000 7fb511a00640 1 -- 172.21.15.161:0/2768868959 >> 172.21.15.161:0/2768868959 conn(0x7fb50c06a2d0 msgr2=0x7fb50c13c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:26.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.886+0000 7fb511a00640 1 -- 172.21.15.161:0/2768868959 shutdown_connections 2024-05-22T21:20:26.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:26.886+0000 7fb511a00640 1 -- 172.21.15.161:0/2768868959 wait complete. 2024-05-22T21:20:26.909 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2024-05-22T21:20:27.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.042+0000 7fd548400640 1 Processor -- start 2024-05-22T21:20:27.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.042+0000 7fd548400640 1 -- start start 2024-05-22T21:20:27.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.042+0000 7fd548400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd54015dd50 0x7fd54006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:27.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.042+0000 7fd548400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd54015d5e0 0x7fd54015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:27.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.042+0000 7fd548400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd540071670 con 0x7fd54015e130 2024-05-22T21:20:27.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.042+0000 7fd548400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54006de40 con 0x7fd54015d5e0 2024-05-22T21:20:27.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.042+0000 7fd548400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd540070a60 con 0x7fd54015dd50 2024-05-22T21:20:27.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.042+0000 7fd546400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd54015d5e0 0x7fd54015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:27.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.042+0000 7fd546400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd54015d5e0 0x7fd54015b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54938/0 (socket says 172.21.15.161:54938) 2024-05-22T21:20:27.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.042+0000 7fd546400640 1 -- 172.21.15.161:0/1517109165 learned_addr learned my addr 172.21.15.161:0/1517109165 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:27.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.046+0000 7fd545a00640 1 --2- 172.21.15.161:0/1517109165 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd54015dd50 0x7fd54006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:27.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.046+0000 7fd546400640 1 -- 172.21.15.161:0/1517109165 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd54015dd50 msgr2=0x7fd54006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:27.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.046+0000 7fd546400640 1 --2- 172.21.15.161:0/1517109165 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd54015dd50 0x7fd54006d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:27.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.046+0000 7fd546400640 1 -- 172.21.15.161:0/1517109165 >> v1:172.21.15.179:6789/0 conn(0x7fd54015e130 legacy=0x7fd54015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:27.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.046+0000 7fd546400640 1 -- 172.21.15.161:0/1517109165 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd540070d40 con 0x7fd54015d5e0 2024-05-22T21:20:27.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.046+0000 7fd546400640 1 --2- 172.21.15.161:0/1517109165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd54015d5e0 0x7fd54015b6c0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fd53800bb10 tx=0x7fd53801eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533b159f4b263a94 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:27.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.046+0000 7fd53fe00640 1 -- 172.21.15.161:0/1517109165 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5380a7c40 con 0x7fd54015d5e0 2024-05-22T21:20:27.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.046+0000 7fd53fe00640 1 -- 172.21.15.161:0/1517109165 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5380a7da0 con 0x7fd54015d5e0 2024-05-22T21:20:27.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.046+0000 7fd53fe00640 1 -- 172.21.15.161:0/1517109165 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5380af660 con 0x7fd54015d5e0 2024-05-22T21:20:27.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd548400640 1 -- 172.21.15.161:0/1517109165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd54015d5e0 msgr2=0x7fd54015b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:27.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd548400640 1 --2- 172.21.15.161:0/1517109165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd54015d5e0 0x7fd54015b6c0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fd53800bb10 tx=0x7fd53801eb70 comp rx=0 tx=0).stop 2024-05-22T21:20:27.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd548400640 1 -- 172.21.15.161:0/1517109165 shutdown_connections 2024-05-22T21:20:27.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd548400640 1 --2- 172.21.15.161:0/1517109165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd54015d5e0 0x7fd54015b6c0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:27.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd548400640 1 --2- 172.21.15.161:0/1517109165 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd54015dd50 0x7fd54006d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:27.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd548400640 1 -- 172.21.15.161:0/1517109165 >> 172.21.15.161:0/1517109165 conn(0x7fd5400b0bc0 msgr2=0x7fd5400b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:27.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd548400640 1 -- 172.21.15.161:0/1517109165 shutdown_connections 2024-05-22T21:20:27.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd548400640 1 -- 172.21.15.161:0/1517109165 wait complete. 2024-05-22T21:20:27.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd548400640 1 Processor -- start 2024-05-22T21:20:27.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd548400640 1 -- start start 2024-05-22T21:20:27.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd548400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd54015d5e0 0x7fd54014b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:27.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd548400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd54015dd50 0x7fd540152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:27.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd548400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd54015e130 0x7fd5400af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:27.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd548400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54014f7d0 con 0x7fd54015e130 2024-05-22T21:20:27.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd548400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54014fab0 con 0x7fd54015d5e0 2024-05-22T21:20:27.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd548400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54014fd90 con 0x7fd54015dd50 2024-05-22T21:20:27.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd546400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd54015e130 0x7fd5400af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:27.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd546400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd54015e130 0x7fd5400af0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54948/0 (socket says 172.21.15.161:54948) 2024-05-22T21:20:27.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd546400640 1 -- 172.21.15.161:0/2390588003 learned_addr learned my addr 172.21.15.161:0/2390588003 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:27.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd545a00640 1 --2- 172.21.15.161:0/2390588003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd54015d5e0 0x7fd54014b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:27.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd545000640 1 --2- 172.21.15.161:0/2390588003 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd54015dd50 0x7fd540152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:27.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd546400640 1 -- 172.21.15.161:0/2390588003 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd54015dd50 msgr2=0x7fd540152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:27.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd546400640 1 --2- 172.21.15.161:0/2390588003 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd54015dd50 0x7fd540152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:27.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd546400640 1 -- 172.21.15.161:0/2390588003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd54015d5e0 msgr2=0x7fd54014b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:27.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd546400640 1 --2- 172.21.15.161:0/2390588003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd54015d5e0 0x7fd54014b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:27.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd546400640 1 -- 172.21.15.161:0/2390588003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53800b770 con 0x7fd54015e130 2024-05-22T21:20:27.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.050+0000 7fd546400640 1 --2- 172.21.15.161:0/2390588003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd54015e130 0x7fd5400af0d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fd53800b2c0 tx=0x7fd53801b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:27.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.054+0000 7fd53d600640 1 -- 172.21.15.161:0/2390588003 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5380afd90 con 0x7fd54015e130 2024-05-22T21:20:27.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.054+0000 7fd53d600640 1 -- 172.21.15.161:0/2390588003 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5380ba410 con 0x7fd54015e130 2024-05-22T21:20:27.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.054+0000 7fd53d600640 1 -- 172.21.15.161:0/2390588003 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5380ba7c0 con 0x7fd54015e130 2024-05-22T21:20:27.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.054+0000 7fd548400640 1 -- 172.21.15.161:0/2390588003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5400b2090 con 0x7fd54015e130 2024-05-22T21:20:27.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.054+0000 7fd548400640 1 -- 172.21.15.161:0/2390588003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54014bb20 con 0x7fd54015e130 2024-05-22T21:20:27.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.054+0000 7fd548400640 1 -- 172.21.15.161:0/2390588003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd540067220 con 0x7fd54015e130 2024-05-22T21:20:27.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.054+0000 7fd53d600640 1 -- 172.21.15.161:0/2390588003 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd5380c3450 con 0x7fd54015e130 2024-05-22T21:20:27.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.054+0000 7fd53d600640 1 --2- 172.21.15.161:0/2390588003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd520066690 0x7fd520068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:27.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.054+0000 7fd53d600640 1 -- 172.21.15.161:0/2390588003 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd53811e750 con 0x7fd54015e130 2024-05-22T21:20:27.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.054+0000 7fd545a00640 1 --2- 172.21.15.161:0/2390588003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd520066690 0x7fd520068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:27.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.054+0000 7fd545a00640 1 --2- 172.21.15.161:0/2390588003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd520066690 0x7fd520068b50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd54006dd40 tx=0x7fd53400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:27.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.058+0000 7fd53d600640 1 -- 172.21.15.161:0/2390588003 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd5380ebdd0 con 0x7fd54015e130 2024-05-22T21:20:27.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.266+0000 7fd548400640 1 -- 172.21.15.161:0/2390588003 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7fd54014f910 con 0x7fd54015e130 2024-05-22T21:20:27.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.542+0000 7fd53d600640 1 -- 172.21.15.161:0/2390588003 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v378) v1 ==== 140+0+0 (secure 0 0 0) 0x7fd54014f910 con 0x7fd54015e130 2024-05-22T21:20:27.547 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.542+0000 7fd548400640 1 -- 172.21.15.161:0/2390588003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd520066690 msgr2=0x7fd520068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:27.547 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.542+0000 7fd548400640 1 --2- 172.21.15.161:0/2390588003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd520066690 0x7fd520068b50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd54006dd40 tx=0x7fd53400b040 comp rx=0 tx=0).stop 2024-05-22T21:20:27.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.542+0000 7fd548400640 1 -- 172.21.15.161:0/2390588003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd54015e130 msgr2=0x7fd5400af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:27.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.542+0000 7fd548400640 1 --2- 172.21.15.161:0/2390588003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd54015e130 0x7fd5400af0d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fd53800b2c0 tx=0x7fd53801b910 comp rx=0 tx=0).stop 2024-05-22T21:20:27.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.542+0000 7fd548400640 1 -- 172.21.15.161:0/2390588003 shutdown_connections 2024-05-22T21:20:27.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.542+0000 7fd548400640 1 --2- 172.21.15.161:0/2390588003 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd520066690 0x7fd520068b50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:27.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.542+0000 7fd548400640 1 --2- 172.21.15.161:0/2390588003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd54015e130 0x7fd5400af0d0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:27.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.542+0000 7fd548400640 1 --2- 172.21.15.161:0/2390588003 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd54015dd50 0x7fd540152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:27.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.542+0000 7fd548400640 1 --2- 172.21.15.161:0/2390588003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd54015d5e0 0x7fd54014b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:27.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.542+0000 7fd548400640 1 -- 172.21.15.161:0/2390588003 >> 172.21.15.161:0/2390588003 conn(0x7fd5400b0bc0 msgr2=0x7fd54015a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:27.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.546+0000 7fd548400640 1 -- 172.21.15.161:0/2390588003 shutdown_connections 2024-05-22T21:20:27.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.546+0000 7fd548400640 1 -- 172.21.15.161:0/2390588003 wait complete. 2024-05-22T21:20:27.566 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-05-22T21:20:27.567 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) 2024-05-22T21:20:27.567 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2024-05-22T21:20:27.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.698+0000 7fac5e400640 1 Processor -- start 2024-05-22T21:20:27.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.698+0000 7fac5e400640 1 -- start start 2024-05-22T21:20:27.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.702+0000 7fac5e400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac5815dd50 0x7fac5806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:27.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.702+0000 7fac5e400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac58071670 con 0x7fac5815d5e0 2024-05-22T21:20:27.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.702+0000 7fac5e400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5806de40 con 0x7fac5815e130 2024-05-22T21:20:27.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.702+0000 7fac5e400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac58070a60 con 0x7fac5815dd50 2024-05-22T21:20:27.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.702+0000 7fac57e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fac5815d5e0 0x7fac5815b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54176/0 (socket says 172.21.15.161:54176) 2024-05-22T21:20:27.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.702+0000 7fac57e00640 1 -- 172.21.15.161:0/1800366273 learned_addr learned my addr 172.21.15.161:0/1800366273 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:27.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.702+0000 7fac57400640 1 --2- 172.21.15.161:0/1800366273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac5815dd50 0x7fac5806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:27.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.702+0000 7fac56000640 1 -- 172.21.15.161:0/1800366273 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357073930 0 0) 0x7fac58071670 con 0x7fac5815d5e0 2024-05-22T21:20:27.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.702+0000 7fac56000640 1 -- 172.21.15.161:0/1800366273 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac3c003530 con 0x7fac5815d5e0 2024-05-22T21:20:27.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.702+0000 7fac56000640 1 -- 172.21.15.161:0/1800366273 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3448731219 0 0) 0x7fac5806de40 con 0x7fac5815e130 2024-05-22T21:20:27.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.702+0000 7fac56000640 1 -- 172.21.15.161:0/1800366273 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac58071670 con 0x7fac5815e130 2024-05-22T21:20:27.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.702+0000 7fac56000640 1 -- 172.21.15.161:0/1800366273 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3747532403 0 0) 0x7fac3c003530 con 0x7fac5815d5e0 2024-05-22T21:20:27.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.702+0000 7fac57400640 1 -- 172.21.15.161:0/1800366273 >> v1:172.21.15.179:6789/0 conn(0x7fac5815e130 legacy=0x7fac5815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:27.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.702+0000 7fac57400640 1 -- 172.21.15.161:0/1800366273 >> v1:172.21.15.161:6789/0 conn(0x7fac5815d5e0 legacy=0x7fac5815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:27.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.702+0000 7fac57400640 1 -- 172.21.15.161:0/1800366273 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac58070d40 con 0x7fac5815dd50 2024-05-22T21:20:27.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.706+0000 7fac57400640 1 --2- 172.21.15.161:0/1800366273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac5815dd50 0x7fac5806d900 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fac48009dc0 tx=0x7fac4801c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f96e73c89f312518 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:27.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.706+0000 7fac56000640 1 -- 172.21.15.161:0/1800366273 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50001680 con 0x7fac5815dd50 2024-05-22T21:20:27.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.706+0000 7fac56000640 1 -- 172.21.15.161:0/1800366273 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac480a5630 con 0x7fac5815dd50 2024-05-22T21:20:27.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.706+0000 7fac56000640 1 -- 172.21.15.161:0/1800366273 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac480a59c0 con 0x7fac5815dd50 2024-05-22T21:20:27.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.706+0000 7fac5e400640 1 -- 172.21.15.161:0/1800366273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac5815dd50 msgr2=0x7fac5806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:27.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.706+0000 7fac5e400640 1 --2- 172.21.15.161:0/1800366273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac5815dd50 0x7fac5806d900 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fac48009dc0 tx=0x7fac4801c7f0 comp rx=0 tx=0).stop 2024-05-22T21:20:27.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.706+0000 7fac5e400640 1 -- 172.21.15.161:0/1800366273 shutdown_connections 2024-05-22T21:20:27.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.706+0000 7fac5e400640 1 --2- 172.21.15.161:0/1800366273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac5815dd50 0x7fac5806d900 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:27.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.706+0000 7fac5e400640 1 -- 172.21.15.161:0/1800366273 >> 172.21.15.161:0/1800366273 conn(0x7fac580b0bc0 msgr2=0x7fac580b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:27.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.706+0000 7fac5e400640 1 -- 172.21.15.161:0/1800366273 shutdown_connections 2024-05-22T21:20:27.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.706+0000 7fac5e400640 1 -- 172.21.15.161:0/1800366273 wait complete. 2024-05-22T21:20:27.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.706+0000 7fac5e400640 1 Processor -- start 2024-05-22T21:20:27.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.706+0000 7fac5e400640 1 -- start start 2024-05-22T21:20:27.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac5e400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac5815d5e0 0x7fac58141300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:27.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac5e400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac5815dd50 0x7fac58140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:27.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac5e400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac5815e130 0x7fac581f6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:27.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac5e400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac580ccfd0 con 0x7fac5815d5e0 2024-05-22T21:20:27.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac5e400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac580cca10 con 0x7fac5815e130 2024-05-22T21:20:27.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac5e400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac580cccf0 con 0x7fac5815dd50 2024-05-22T21:20:27.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac57400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac5815d5e0 0x7fac58141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:27.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac57400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac5815d5e0 0x7fac58141300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54962/0 (socket says 172.21.15.161:54962) 2024-05-22T21:20:27.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac57400640 1 -- 172.21.15.161:0/3199748272 learned_addr learned my addr 172.21.15.161:0/3199748272 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:27.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac57e00640 1 --2- 172.21.15.161:0/3199748272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac5815e130 0x7fac581f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:27.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac56a00640 1 --2- 172.21.15.161:0/3199748272 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac5815dd50 0x7fac58140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:27.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac57400640 1 -- 172.21.15.161:0/3199748272 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac5815dd50 msgr2=0x7fac58140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:27.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac57400640 1 --2- 172.21.15.161:0/3199748272 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac5815dd50 0x7fac58140d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:27.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac57400640 1 -- 172.21.15.161:0/3199748272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac5815e130 msgr2=0x7fac581f6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:27.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac57400640 1 --2- 172.21.15.161:0/3199748272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac5815e130 0x7fac581f6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:27.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac57400640 1 -- 172.21.15.161:0/3199748272 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac48009a70 con 0x7fac5815d5e0 2024-05-22T21:20:27.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac57400640 1 --2- 172.21.15.161:0/3199748272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac5815d5e0 0x7fac58141300 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fac4801aea0 tx=0x7fac48019590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:27.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac4f400640 1 -- 172.21.15.161:0/3199748272 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac480a5ea0 con 0x7fac5815d5e0 2024-05-22T21:20:27.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac4f400640 1 -- 172.21.15.161:0/3199748272 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac48003bc0 con 0x7fac5815d5e0 2024-05-22T21:20:27.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac4f400640 1 -- 172.21.15.161:0/3199748272 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48009450 con 0x7fac5815d5e0 2024-05-22T21:20:27.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac5e400640 1 -- 172.21.15.161:0/3199748272 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac581f5e50 con 0x7fac5815d5e0 2024-05-22T21:20:27.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac5e400640 1 -- 172.21.15.161:0/3199748272 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac581f6b90 con 0x7fac5815d5e0 2024-05-22T21:20:27.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac5e400640 1 -- 172.21.15.161:0/3199748272 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac58067220 con 0x7fac5815d5e0 2024-05-22T21:20:27.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.710+0000 7fac4f400640 1 -- 172.21.15.161:0/3199748272 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fac480b4050 con 0x7fac5815d5e0 2024-05-22T21:20:27.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.714+0000 7fac4f400640 1 --2- 172.21.15.161:0/3199748272 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac38066690 0x7fac38068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:27.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.714+0000 7fac4f400640 1 -- 172.21.15.161:0/3199748272 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fac4811eb60 con 0x7fac5815d5e0 2024-05-22T21:20:27.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.714+0000 7fac56a00640 1 --2- 172.21.15.161:0/3199748272 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac38066690 0x7fac38068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:27.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.714+0000 7fac56a00640 1 --2- 172.21.15.161:0/3199748272 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac38066690 0x7fac38068b50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fac5814ed80 tx=0x7fac44009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:27.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.718+0000 7fac4f400640 1 -- 172.21.15.161:0/3199748272 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fac480ec130 con 0x7fac5815d5e0 2024-05-22T21:20:27.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:27.922+0000 7fac5e400640 1 -- 172.21.15.161:0/3199748272 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7fac581f6300 con 0x7fac5815d5e0 2024-05-22T21:20:28.550 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.546+0000 7fac4f400640 1 -- 172.21.15.161:0/3199748272 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v379) v1 ==== 135+0+0 (secure 0 0 0) 0x7fac480b2020 con 0x7fac5815d5e0 2024-05-22T21:20:28.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.550+0000 7fac5e400640 1 -- 172.21.15.161:0/3199748272 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac38066690 msgr2=0x7fac38068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:28.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.550+0000 7fac5e400640 1 --2- 172.21.15.161:0/3199748272 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac38066690 0x7fac38068b50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fac5814ed80 tx=0x7fac44009040 comp rx=0 tx=0).stop 2024-05-22T21:20:28.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.550+0000 7fac5e400640 1 -- 172.21.15.161:0/3199748272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac5815d5e0 msgr2=0x7fac58141300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:28.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.550+0000 7fac5e400640 1 --2- 172.21.15.161:0/3199748272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac5815d5e0 0x7fac58141300 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fac4801aea0 tx=0x7fac48019590 comp rx=0 tx=0).stop 2024-05-22T21:20:28.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.550+0000 7fac5e400640 1 -- 172.21.15.161:0/3199748272 shutdown_connections 2024-05-22T21:20:28.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.550+0000 7fac5e400640 1 --2- 172.21.15.161:0/3199748272 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac38066690 0x7fac38068b50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:28.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.550+0000 7fac5e400640 1 --2- 172.21.15.161:0/3199748272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac5815e130 0x7fac581f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:28.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.550+0000 7fac5e400640 1 --2- 172.21.15.161:0/3199748272 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac5815dd50 0x7fac58140d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:28.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.550+0000 7fac5e400640 1 --2- 172.21.15.161:0/3199748272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac5815d5e0 0x7fac58141300 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:28.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.550+0000 7fac5e400640 1 -- 172.21.15.161:0/3199748272 >> 172.21.15.161:0/3199748272 conn(0x7fac580b0bc0 msgr2=0x7fac5814eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:28.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.550+0000 7fac5e400640 1 -- 172.21.15.161:0/3199748272 shutdown_connections 2024-05-22T21:20:28.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.550+0000 7fac5e400640 1 -- 172.21.15.161:0/3199748272 wait complete. 2024-05-22T21:20:28.575 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-22T21:20:28.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.714+0000 7f93bbc00640 1 Processor -- start 2024-05-22T21:20:28.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.714+0000 7f93bbc00640 1 -- start start 2024-05-22T21:20:28.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.714+0000 7f93bbc00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f93b41402b0 0x7f93b4070590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:28.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.714+0000 7f93bbc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f93b406c2a0 0x7f93b406c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:28.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.714+0000 7f93bbc00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b415d9d0 con 0x7f93b4070f90 2024-05-22T21:20:28.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.714+0000 7f93bbc00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b406cc90 con 0x7f93b406c2a0 2024-05-22T21:20:28.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.714+0000 7f93bbc00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b415d7c0 con 0x7f93b41402b0 2024-05-22T21:20:28.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.714+0000 7f93b3e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f93b406c2a0 0x7f93b406c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:28.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.714+0000 7f93b9c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f93b4070f90 0x7f93b4071400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54188/0 (socket says 172.21.15.161:54188) 2024-05-22T21:20:28.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.714+0000 7f93b9c00640 1 -- 172.21.15.161:0/3265062538 learned_addr learned my addr 172.21.15.161:0/3265062538 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:28.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.714+0000 7f93b9200640 1 --2- 172.21.15.161:0/3265062538 >> v2:172.21.15.179:3300/0 conn(0x7f93b41402b0 0x7f93b4070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:28.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.714+0000 7f93b3e00640 1 -- 172.21.15.161:0/3265062538 >> v2:172.21.15.179:3300/0 conn(0x7f93b41402b0 msgr2=0x7f93b4070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:28.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.714+0000 7f93b3e00640 1 --2- 172.21.15.161:0/3265062538 >> v2:172.21.15.179:3300/0 conn(0x7f93b41402b0 0x7f93b4070590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:28.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.714+0000 7f93b3e00640 1 -- 172.21.15.161:0/3265062538 >> v1:172.21.15.161:6789/0 conn(0x7f93b4070f90 legacy=0x7f93b4071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:28.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.714+0000 7f93b3400640 1 -- 172.21.15.161:0/3265062538 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449037042 0 0) 0x7f93b415d9d0 con 0x7f93b4070f90 2024-05-22T21:20:28.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.714+0000 7f93b3e00640 1 -- 172.21.15.161:0/3265062538 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b415dcb0 con 0x7f93b406c2a0 2024-05-22T21:20:28.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.718+0000 7f93b3e00640 1 --2- 172.21.15.161:0/3265062538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b406c2a0 0x7f93b406c750 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f93b406dec0 tx=0x7f93a001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3110449901ced4da server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:28.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.718+0000 7f93b3400640 1 -- 172.21.15.161:0/3265062538 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a00a6040 con 0x7f93b406c2a0 2024-05-22T21:20:28.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.718+0000 7f93b3400640 1 -- 172.21.15.161:0/3265062538 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a00aa600 con 0x7f93b406c2a0 2024-05-22T21:20:28.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.718+0000 7f93b3400640 1 -- 172.21.15.161:0/3265062538 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a00aa990 con 0x7f93b406c2a0 2024-05-22T21:20:28.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.718+0000 7f93bbc00640 1 -- 172.21.15.161:0/3265062538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b406c2a0 msgr2=0x7f93b406c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:28.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.718+0000 7f93bbc00640 1 --2- 172.21.15.161:0/3265062538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b406c2a0 0x7f93b406c750 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f93b406dec0 tx=0x7f93a001c480 comp rx=0 tx=0).stop 2024-05-22T21:20:28.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.718+0000 7f93bbc00640 1 -- 172.21.15.161:0/3265062538 shutdown_connections 2024-05-22T21:20:28.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.718+0000 7f93bbc00640 1 --2- 172.21.15.161:0/3265062538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b406c2a0 0x7f93b406c750 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:28.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.718+0000 7f93bbc00640 1 --2- 172.21.15.161:0/3265062538 >> v2:172.21.15.179:3300/0 conn(0x7f93b41402b0 0x7f93b4070590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:28.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.718+0000 7f93bbc00640 1 -- 172.21.15.161:0/3265062538 >> 172.21.15.161:0/3265062538 conn(0x7f93b406a2d0 msgr2=0x7f93b406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:28.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.718+0000 7f93bbc00640 1 -- 172.21.15.161:0/3265062538 shutdown_connections 2024-05-22T21:20:28.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.718+0000 7f93bbc00640 1 -- 172.21.15.161:0/3265062538 wait complete. 2024-05-22T21:20:28.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.718+0000 7f93bbc00640 1 Processor -- start 2024-05-22T21:20:28.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93bbc00640 1 -- start start 2024-05-22T21:20:28.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93bbc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b406c2a0 0x7f93b413c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:28.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93bbc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b4070f90 0x7f93b4141390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:28.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93bbc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93b41402b0 0x7f93b413e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:28.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93bbc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b4157e60 con 0x7f93b406c2a0 2024-05-22T21:20:28.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93bbc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b4141a50 con 0x7f93b4070f90 2024-05-22T21:20:28.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93bbc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b4141d30 con 0x7f93b41402b0 2024-05-22T21:20:28.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93b9200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b406c2a0 0x7f93b413c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:28.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93b9200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b406c2a0 0x7f93b413c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54978/0 (socket says 172.21.15.161:54978) 2024-05-22T21:20:28.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93b9200640 1 -- 172.21.15.161:0/3652363699 learned_addr learned my addr 172.21.15.161:0/3652363699 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:28.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93b3e00640 1 --2- 172.21.15.161:0/3652363699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b4070f90 0x7f93b4141390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:28.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93b9c00640 1 --2- 172.21.15.161:0/3652363699 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93b41402b0 0x7f93b413e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:28.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93b9200640 1 -- 172.21.15.161:0/3652363699 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93b41402b0 msgr2=0x7f93b413e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:28.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93b9200640 1 --2- 172.21.15.161:0/3652363699 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93b41402b0 0x7f93b413e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:28.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93b9200640 1 -- 172.21.15.161:0/3652363699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b4070f90 msgr2=0x7f93b4141390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:28.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93b9200640 1 --2- 172.21.15.161:0/3652363699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b4070f90 0x7f93b4141390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:28.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93b9200640 1 -- 172.21.15.161:0/3652363699 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a0009420 con 0x7f93b406c2a0 2024-05-22T21:20:28.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93b9c00640 1 --2- 172.21.15.161:0/3652363699 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93b41402b0 0x7f93b413e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:28.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93b9200640 1 --2- 172.21.15.161:0/3652363699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b406c2a0 0x7f93b413c7a0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f93a800d5f0 tx=0x7f93a800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:28.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93b0c00640 1 -- 172.21.15.161:0/3652363699 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a80125f0 con 0x7f93b406c2a0 2024-05-22T21:20:28.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93b0c00640 1 -- 172.21.15.161:0/3652363699 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a8012750 con 0x7f93b406c2a0 2024-05-22T21:20:28.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93b0c00640 1 -- 172.21.15.161:0/3652363699 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8012ae0 con 0x7f93b406c2a0 2024-05-22T21:20:28.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93bbc00640 1 -- 172.21.15.161:0/3652363699 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b4142010 con 0x7f93b406c2a0 2024-05-22T21:20:28.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93bbc00640 1 -- 172.21.15.161:0/3652363699 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b413ebd0 con 0x7f93b406c2a0 2024-05-22T21:20:28.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93bbc00640 1 -- 172.21.15.161:0/3652363699 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b413edc0 con 0x7f93b406c2a0 2024-05-22T21:20:28.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.722+0000 7f93b0c00640 1 -- 172.21.15.161:0/3652363699 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f93a8012c40 con 0x7f93b406c2a0 2024-05-22T21:20:28.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.726+0000 7f93b0c00640 1 --2- 172.21.15.161:0/3652363699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9394066690 0x7f9394068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:28.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.726+0000 7f93b0c00640 1 -- 172.21.15.161:0/3652363699 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f93a80844f0 con 0x7f93b406c2a0 2024-05-22T21:20:28.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.726+0000 7f93b3e00640 1 --2- 172.21.15.161:0/3652363699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9394066690 0x7f9394068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:28.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.726+0000 7f93b3e00640 1 --2- 172.21.15.161:0/3652363699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9394066690 0x7f9394068b50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f93a00ae000 tx=0x7f93a00a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:28.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.730+0000 7f93b0c00640 1 -- 172.21.15.161:0/3652363699 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f93b413edc0 con 0x7f93b406c2a0 2024-05-22T21:20:28.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.886+0000 7f93bbc00640 1 -- 172.21.15.161:0/3652363699 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f93b4157bc0 con 0x7f93b406c2a0 2024-05-22T21:20:28.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.890+0000 7f93b0c00640 1 -- 172.21.15.161:0/3652363699 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f93b4157bc0 con 0x7f93b406c2a0 2024-05-22T21:20:28.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.894+0000 7f93bbc00640 1 -- 172.21.15.161:0/3652363699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9394066690 msgr2=0x7f9394068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:28.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.894+0000 7f93bbc00640 1 --2- 172.21.15.161:0/3652363699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9394066690 0x7f9394068b50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f93a00ae000 tx=0x7f93a00a7000 comp rx=0 tx=0).stop 2024-05-22T21:20:28.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.894+0000 7f93bbc00640 1 -- 172.21.15.161:0/3652363699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b406c2a0 msgr2=0x7f93b413c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:28.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.894+0000 7f93bbc00640 1 --2- 172.21.15.161:0/3652363699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b406c2a0 0x7f93b413c7a0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f93a800d5f0 tx=0x7f93a800dab0 comp rx=0 tx=0).stop 2024-05-22T21:20:28.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.894+0000 7f93bbc00640 1 -- 172.21.15.161:0/3652363699 shutdown_connections 2024-05-22T21:20:28.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.894+0000 7f93bbc00640 1 --2- 172.21.15.161:0/3652363699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9394066690 0x7f9394068b50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:28.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.894+0000 7f93bbc00640 1 --2- 172.21.15.161:0/3652363699 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f93b41402b0 0x7f93b413e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:28.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.894+0000 7f93bbc00640 1 --2- 172.21.15.161:0/3652363699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b4070f90 0x7f93b4141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:28.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.894+0000 7f93bbc00640 1 --2- 172.21.15.161:0/3652363699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93b406c2a0 0x7f93b413c7a0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:28.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.894+0000 7f93bbc00640 1 -- 172.21.15.161:0/3652363699 >> 172.21.15.161:0/3652363699 conn(0x7f93b406a2d0 msgr2=0x7f93b415b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:28.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.894+0000 7f93bbc00640 1 -- 172.21.15.161:0/3652363699 shutdown_connections 2024-05-22T21:20:28.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:28.894+0000 7f93bbc00640 1 -- 172.21.15.161:0/3652363699 wait complete. 2024-05-22T21:20:28.916 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:20:28.916 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:20:28.994 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:20:29.055 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:20:29.055 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:20:29.055 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:20:29.064 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:20:29.064 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:20:29.064 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:20:29.073 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:20:29.074 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:20:29.074 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:20:29.115 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:20:29.115 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:20:29.115 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:20:29.128 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:20:29.128 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-22T21:20:29.128 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:20:29.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.290+0000 7fc798200640 1 Processor -- start 2024-05-22T21:20:29.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.290+0000 7fc798200640 1 -- start start 2024-05-22T21:20:29.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.290+0000 7fc798200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc79015dd50 0x7fc79006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:29.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.290+0000 7fc798200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc790071670 con 0x7fc79015e130 2024-05-22T21:20:29.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.290+0000 7fc798200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79006de40 con 0x7fc79015d5e0 2024-05-22T21:20:29.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.290+0000 7fc798200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc790070a60 con 0x7fc79015dd50 2024-05-22T21:20:29.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.290+0000 7fc795800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc79015dd50 0x7fc79006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:29.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.290+0000 7fc795800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc79015dd50 0x7fc79006d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54994/0 (socket says 172.21.15.161:54994) 2024-05-22T21:20:29.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.290+0000 7fc794e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc79015e130 0x7fc79015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54202/0 (socket says 172.21.15.161:54202) 2024-05-22T21:20:29.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.290+0000 7fc795800640 1 -- 172.21.15.161:0/3170235626 learned_addr learned my addr 172.21.15.161:0/3170235626 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:29.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.290+0000 7fc795800640 1 -- 172.21.15.161:0/3170235626 >> v1:172.21.15.179:6789/0 conn(0x7fc79015d5e0 legacy=0x7fc79015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:29.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.290+0000 7fc795800640 1 -- 172.21.15.161:0/3170235626 >> v1:172.21.15.161:6789/0 conn(0x7fc79015e130 legacy=0x7fc79015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:29.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.290+0000 7fc795800640 1 -- 172.21.15.161:0/3170235626 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc790070d40 con 0x7fc79015dd50 2024-05-22T21:20:29.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.290+0000 7fc78fe00640 1 -- 172.21.15.161:0/3170235626 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746474303 0 0) 0x7fc790071670 con 0x7fc79015e130 2024-05-22T21:20:29.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.294+0000 7fc795800640 1 --2- 172.21.15.161:0/3170235626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc79015dd50 0x7fc79006d900 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fc78400bb10 tx=0x7fc78401ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3185b6b73d276da4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:29.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.294+0000 7fc78fe00640 1 -- 172.21.15.161:0/3170235626 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7840a7cd0 con 0x7fc79015dd50 2024-05-22T21:20:29.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.294+0000 7fc78fe00640 1 -- 172.21.15.161:0/3170235626 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7840a7e30 con 0x7fc79015dd50 2024-05-22T21:20:29.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.294+0000 7fc78fe00640 1 -- 172.21.15.161:0/3170235626 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7840af660 con 0x7fc79015dd50 2024-05-22T21:20:29.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.294+0000 7fc798200640 1 -- 172.21.15.161:0/3170235626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc79015dd50 msgr2=0x7fc79006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:29.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.294+0000 7fc798200640 1 --2- 172.21.15.161:0/3170235626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc79015dd50 0x7fc79006d900 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fc78400bb10 tx=0x7fc78401ec00 comp rx=0 tx=0).stop 2024-05-22T21:20:29.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.294+0000 7fc798200640 1 -- 172.21.15.161:0/3170235626 shutdown_connections 2024-05-22T21:20:29.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.294+0000 7fc798200640 1 --2- 172.21.15.161:0/3170235626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc79015dd50 0x7fc79006d900 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:29.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.294+0000 7fc798200640 1 -- 172.21.15.161:0/3170235626 >> 172.21.15.161:0/3170235626 conn(0x7fc7900b0bc0 msgr2=0x7fc7900b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:29.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.294+0000 7fc798200640 1 -- 172.21.15.161:0/3170235626 shutdown_connections 2024-05-22T21:20:29.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.294+0000 7fc798200640 1 -- 172.21.15.161:0/3170235626 wait complete. 2024-05-22T21:20:29.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc798200640 1 Processor -- start 2024-05-22T21:20:29.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc798200640 1 -- start start 2024-05-22T21:20:29.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc798200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc79015d5e0 0x7fc79014ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:29.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc798200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc79015dd50 0x7fc7900b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:29.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc798200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc79015e130 0x7fc7900b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:29.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc798200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79014cf40 con 0x7fc79015dd50 2024-05-22T21:20:29.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc798200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900b3120 con 0x7fc79015e130 2024-05-22T21:20:29.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc798200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900b2b60 con 0x7fc79015d5e0 2024-05-22T21:20:29.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc794e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc79015dd50 0x7fc7900b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:29.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc794e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc79015dd50 0x7fc7900b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55008/0 (socket says 172.21.15.161:55008) 2024-05-22T21:20:29.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc794e00640 1 -- 172.21.15.161:0/2756988960 learned_addr learned my addr 172.21.15.161:0/2756988960 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:29.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc794e00640 1 -- 172.21.15.161:0/2756988960 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc79015d5e0 msgr2=0x7fc79014ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:29.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc794e00640 1 --2- 172.21.15.161:0/2756988960 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc79015d5e0 0x7fc79014ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:29.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc794e00640 1 -- 172.21.15.161:0/2756988960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc79015e130 msgr2=0x7fc7900b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:29.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc796200640 1 --2- 172.21.15.161:0/2756988960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc79015e130 0x7fc7900b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:29.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc794e00640 1 --2- 172.21.15.161:0/2756988960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc79015e130 0x7fc7900b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:29.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc794e00640 1 -- 172.21.15.161:0/2756988960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78400b770 con 0x7fc79015dd50 2024-05-22T21:20:29.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc796200640 1 --2- 172.21.15.161:0/2756988960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc79015e130 0x7fc7900b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:29.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc794e00640 1 --2- 172.21.15.161:0/2756988960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc79015dd50 0x7fc7900b1ff0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00c4a0 tx=0x7fc77c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:29.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc78d600640 1 -- 172.21.15.161:0/2756988960 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c0074e0 con 0x7fc79015dd50 2024-05-22T21:20:29.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc78d600640 1 -- 172.21.15.161:0/2756988960 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c007640 con 0x7fc79015dd50 2024-05-22T21:20:29.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc78d600640 1 -- 172.21.15.161:0/2756988960 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c004a30 con 0x7fc79015dd50 2024-05-22T21:20:29.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc798200640 1 -- 172.21.15.161:0/2756988960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7900b2e40 con 0x7fc79015dd50 2024-05-22T21:20:29.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.298+0000 7fc798200640 1 -- 172.21.15.161:0/2756988960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc790154cc0 con 0x7fc79015dd50 2024-05-22T21:20:29.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.302+0000 7fc798200640 1 -- 172.21.15.161:0/2756988960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc790154840 con 0x7fc79015dd50 2024-05-22T21:20:29.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.302+0000 7fc78d600640 1 -- 172.21.15.161:0/2756988960 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc77c004b90 con 0x7fc79015dd50 2024-05-22T21:20:29.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.302+0000 7fc78d600640 1 --2- 172.21.15.161:0/2756988960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc770066690 0x7fc770068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:29.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.302+0000 7fc78d600640 1 -- 172.21.15.161:0/2756988960 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc77c083180 con 0x7fc79015dd50 2024-05-22T21:20:29.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.302+0000 7fc795800640 1 --2- 172.21.15.161:0/2756988960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc770066690 0x7fc770068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:29.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.302+0000 7fc795800640 1 --2- 172.21.15.161:0/2756988960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc770066690 0x7fc770068b50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc7840b1000 tx=0x7fc784008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:29.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.306+0000 7fc78d600640 1 -- 172.21.15.161:0/2756988960 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc790154840 con 0x7fc79015dd50 2024-05-22T21:20:29.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.494+0000 7fc798200640 1 -- 172.21.15.161:0/2756988960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc790154840 con 0x7fc79015dd50 2024-05-22T21:20:29.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.494+0000 7fc78d600640 1 -- 172.21.15.161:0/2756988960 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+3538 (secure 0 0 0) 0x7fc790154840 con 0x7fc79015dd50 2024-05-22T21:20:29.498 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:29.498 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":64,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/524372753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":524372753},{"type":"v1","addr":"172.21.15.161:6836","nonce":524372753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3762119379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3762119379},{"type":"v1","addr":"172.21.15.179:6833","nonce":3762119379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3087241366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3087241366},{"type":"v1","addr":"172.21.15.179:6835","nonce":3087241366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3905842546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3905842546},{"type":"v1","addr":"172.21.15.161:6837","nonce":3905842546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[]} 2024-05-22T21:20:29.498 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 64 2024-05-22T21:20:29.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.494+0000 7fc798200640 1 -- 172.21.15.161:0/2756988960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc770066690 msgr2=0x7fc770068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:29.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.494+0000 7fc798200640 1 --2- 172.21.15.161:0/2756988960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc770066690 0x7fc770068b50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc7840b1000 tx=0x7fc784008040 comp rx=0 tx=0).stop 2024-05-22T21:20:29.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.494+0000 7fc798200640 1 -- 172.21.15.161:0/2756988960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc79015dd50 msgr2=0x7fc7900b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:29.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.494+0000 7fc798200640 1 --2- 172.21.15.161:0/2756988960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc79015dd50 0x7fc7900b1ff0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00c4a0 tx=0x7fc77c00c960 comp rx=0 tx=0).stop 2024-05-22T21:20:29.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.498+0000 7fc798200640 1 -- 172.21.15.161:0/2756988960 shutdown_connections 2024-05-22T21:20:29.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.498+0000 7fc798200640 1 --2- 172.21.15.161:0/2756988960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc770066690 0x7fc770068b50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:29.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.498+0000 7fc798200640 1 --2- 172.21.15.161:0/2756988960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc79015e130 0x7fc7900b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:29.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.498+0000 7fc798200640 1 --2- 172.21.15.161:0/2756988960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc79015dd50 0x7fc7900b1ff0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:29.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.498+0000 7fc798200640 1 --2- 172.21.15.161:0/2756988960 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc79015d5e0 0x7fc79014ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:29.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.498+0000 7fc798200640 1 -- 172.21.15.161:0/2756988960 >> 172.21.15.161:0/2756988960 conn(0x7fc7900b0bc0 msgr2=0x7fc7900aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:29.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.498+0000 7fc798200640 1 -- 172.21.15.161:0/2756988960 shutdown_connections 2024-05-22T21:20:29.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.498+0000 7fc798200640 1 -- 172.21.15.161:0/2756988960 wait complete. 2024-05-22T21:20:29.518 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-22T21:20:29.519 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-22T21:20:29.519 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-22T21:20:29.519 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:20:29.519 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-22T21:20:29.519 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-22T21:20:29.519 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-22T21:20:29.520 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:20:29.520 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-22T21:20:29.520 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-22T21:20:29.520 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-22T21:20:29.520 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:20:29.520 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-22T21:20:29.520 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-22T21:20:29.521 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-22T21:20:29.521 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:20:29.532 INFO:tasks.ceph.mds.a:Stopped 2024-05-22T21:20:29.532 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-22T21:20:29.533 INFO:tasks.ceph.mds.c:Stopped 2024-05-22T21:20:29.533 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-22T21:20:29.535 INFO:tasks.ceph.mds.b:Stopped 2024-05-22T21:20:29.535 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-22T21:20:29.536 INFO:tasks.ceph.mds.d:Stopped 2024-05-22T21:20:29.536 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-22T21:20:29.577 INFO:tasks.ceph.mds.a:Started 2024-05-22T21:20:29.578 DEBUG:teuthology.parallel:result is None 2024-05-22T21:20:29.579 INFO:tasks.ceph.mds.c:Started 2024-05-22T21:20:29.579 DEBUG:teuthology.parallel:result is None 2024-05-22T21:20:29.586 INFO:tasks.ceph.mds.b:Started 2024-05-22T21:20:29.587 DEBUG:teuthology.parallel:result is None 2024-05-22T21:20:29.588 INFO:tasks.ceph.mds.d:Started 2024-05-22T21:20:29.588 DEBUG:teuthology.parallel:result is None 2024-05-22T21:20:29.588 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-22T21:20:29.663 INFO:tasks.ceph.mds.c.smithi161.stdout:starting mds.c at 2024-05-22T21:20:29.664 INFO:tasks.ceph.mds.a.smithi161.stdout:starting mds.a at 2024-05-22T21:20:29.677 INFO:tasks.ceph.mds.d.smithi179.stdout:starting mds.d at 2024-05-22T21:20:29.677 INFO:tasks.ceph.mds.b.smithi179.stdout:starting mds.b at 2024-05-22T21:20:29.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.734+0000 7f3f1f200640 1 Processor -- start 2024-05-22T21:20:29.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.734+0000 7f3f1f200640 1 -- start start 2024-05-22T21:20:29.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.734+0000 7f3f1f200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3f1813cfc0 0x7f3f181402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:29.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.734+0000 7f3f1f200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1813d3a0 con 0x7f3f1813fe50 2024-05-22T21:20:29.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.734+0000 7f3f1f200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f180cd920 con 0x7f3f1815af60 2024-05-22T21:20:29.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.734+0000 7f3f1f200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f180cdc00 con 0x7f3f1813cfc0 2024-05-22T21:20:29.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.734+0000 7f3f1d200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3f1813fe50 0x7f3f1806fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54226/0 (socket says 172.21.15.161:54226) 2024-05-22T21:20:29.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.734+0000 7f3f1d200640 1 -- 172.21.15.161:0/706260429 learned_addr learned my addr 172.21.15.161:0/706260429 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:29.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.734+0000 7f3f17e00640 1 --2- 172.21.15.161:0/706260429 >> v2:172.21.15.179:3300/0 conn(0x7f3f1813cfc0 0x7f3f181402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:29.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.734+0000 7f3f17400640 1 -- 172.21.15.161:0/706260429 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1873570587 0 0) 0x7f3f1813d3a0 con 0x7f3f1813fe50 2024-05-22T21:20:29.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.734+0000 7f3f17400640 1 -- 172.21.15.161:0/706260429 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3efc003530 con 0x7f3f1813fe50 2024-05-22T21:20:29.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.734+0000 7f3f17e00640 1 -- 172.21.15.161:0/706260429 >> v1:172.21.15.179:6789/0 conn(0x7f3f1815af60 legacy=0x7f3f1813f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:29.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.734+0000 7f3f17e00640 1 -- 172.21.15.161:0/706260429 >> v1:172.21.15.161:6789/0 conn(0x7f3f1813fe50 legacy=0x7f3f1806fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:29.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.734+0000 7f3f17e00640 1 -- 172.21.15.161:0/706260429 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f180cdee0 con 0x7f3f1813cfc0 2024-05-22T21:20:29.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.738+0000 7f3f17e00640 1 --2- 172.21.15.161:0/706260429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f1813cfc0 0x7f3f181402e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3f08009d90 tx=0x7f3f0801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81127936435ffb8c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:29.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.738+0000 7f3f17400640 1 -- 172.21.15.161:0/706260429 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f080a5770 con 0x7f3f1813cfc0 2024-05-22T21:20:29.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.738+0000 7f3f17400640 1 -- 172.21.15.161:0/706260429 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f080a58d0 con 0x7f3f1813cfc0 2024-05-22T21:20:29.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.738+0000 7f3f17400640 1 -- 172.21.15.161:0/706260429 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f080a5c60 con 0x7f3f1813cfc0 2024-05-22T21:20:29.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.738+0000 7f3f1f200640 1 -- 172.21.15.161:0/706260429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f1813cfc0 msgr2=0x7f3f181402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:29.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.738+0000 7f3f1f200640 1 --2- 172.21.15.161:0/706260429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f1813cfc0 0x7f3f181402e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3f08009d90 tx=0x7f3f0801c8b0 comp rx=0 tx=0).stop 2024-05-22T21:20:29.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.738+0000 7f3f1f200640 1 -- 172.21.15.161:0/706260429 shutdown_connections 2024-05-22T21:20:29.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.738+0000 7f3f1f200640 1 --2- 172.21.15.161:0/706260429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f1813cfc0 0x7f3f181402e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:29.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.738+0000 7f3f1f200640 1 -- 172.21.15.161:0/706260429 >> 172.21.15.161:0/706260429 conn(0x7f3f1806a2d0 msgr2=0x7f3f1806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:29.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.738+0000 7f3f1f200640 1 -- 172.21.15.161:0/706260429 shutdown_connections 2024-05-22T21:20:29.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.738+0000 7f3f1f200640 1 -- 172.21.15.161:0/706260429 wait complete. 2024-05-22T21:20:29.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.738+0000 7f3f1f200640 1 Processor -- start 2024-05-22T21:20:29.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f1f200640 1 -- start start 2024-05-22T21:20:29.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f1f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f1813cfc0 0x7f3f1815c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:29.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f1f200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f1813fe50 0x7f3f18157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:29.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f1f200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f1815af60 0x7f3f181559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:29.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f1f200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f18157970 con 0x7f3f1813cfc0 2024-05-22T21:20:29.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f1f200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f18155ef0 con 0x7f3f1813fe50 2024-05-22T21:20:29.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f1f200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f18156b90 con 0x7f3f1815af60 2024-05-22T21:20:29.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f17e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f1813cfc0 0x7f3f1815c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:29.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f17e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f1813cfc0 0x7f3f1815c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55044/0 (socket says 172.21.15.161:55044) 2024-05-22T21:20:29.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f17e00640 1 -- 172.21.15.161:0/3070199138 learned_addr learned my addr 172.21.15.161:0/3070199138 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:29.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f0f400640 1 --2- 172.21.15.161:0/3070199138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f1813fe50 0x7f3f18157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:29.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f1d200640 1 --2- 172.21.15.161:0/3070199138 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f1815af60 0x7f3f181559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:29.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f17e00640 1 -- 172.21.15.161:0/3070199138 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f1815af60 msgr2=0x7f3f181559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:29.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f17e00640 1 --2- 172.21.15.161:0/3070199138 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f1815af60 0x7f3f181559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:29.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f17e00640 1 -- 172.21.15.161:0/3070199138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f1813fe50 msgr2=0x7f3f18157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:29.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f17e00640 1 --2- 172.21.15.161:0/3070199138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f1813fe50 0x7f3f18157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:29.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f17e00640 1 -- 172.21.15.161:0/3070199138 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f08009a70 con 0x7f3f1813cfc0 2024-05-22T21:20:29.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f0f400640 1 --2- 172.21.15.161:0/3070199138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f1813fe50 0x7f3f18157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:29.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f17e00640 1 --2- 172.21.15.161:0/3070199138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f1813cfc0 0x7f3f1815c010 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f3f08009d90 tx=0x7f3f08019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:29.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f14c00640 1 -- 172.21.15.161:0/3070199138 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f080b9860 con 0x7f3f1813cfc0 2024-05-22T21:20:29.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f14c00640 1 -- 172.21.15.161:0/3070199138 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f080b99c0 con 0x7f3f1813cfc0 2024-05-22T21:20:29.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f1f200640 1 -- 172.21.15.161:0/3070199138 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f18156e70 con 0x7f3f1813cfc0 2024-05-22T21:20:29.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f14c00640 1 -- 172.21.15.161:0/3070199138 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f080b9d30 con 0x7f3f1813cfc0 2024-05-22T21:20:29.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f1f200640 1 -- 172.21.15.161:0/3070199138 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1815c5f0 con 0x7f3f1813cfc0 2024-05-22T21:20:29.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.742+0000 7f3f1f200640 1 -- 172.21.15.161:0/3070199138 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f18067220 con 0x7f3f1813cfc0 2024-05-22T21:20:29.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.746+0000 7f3f14c00640 1 -- 172.21.15.161:0/3070199138 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3f080ad440 con 0x7f3f1813cfc0 2024-05-22T21:20:29.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.746+0000 7f3f14c00640 1 --2- 172.21.15.161:0/3070199138 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ef4066690 0x7f3ef4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:29.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.746+0000 7f3f14c00640 1 -- 172.21.15.161:0/3070199138 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3f0811f180 con 0x7f3f1813cfc0 2024-05-22T21:20:29.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.746+0000 7f3f0f400640 1 --2- 172.21.15.161:0/3070199138 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ef4066690 0x7f3ef4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:29.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.746+0000 7f3f0f400640 1 --2- 172.21.15.161:0/3070199138 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ef4066690 0x7f3ef4068b50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3f18138a90 tx=0x7f3f00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:29.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.750+0000 7f3f14c00640 1 -- 172.21.15.161:0/3070199138 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3f080ec900 con 0x7f3f1813cfc0 2024-05-22T21:20:29.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:29.910+0000 7f3f1f200640 1 -- 172.21.15.161:0/3070199138 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3f18136f40 con 0x7f3f1813cfc0 2024-05-22T21:20:30.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.702+0000 7f3f14c00640 1 -- 172.21.15.161:0/3070199138 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v154) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3f080eff20 con 0x7f3f1813cfc0 2024-05-22T21:20:30.706 INFO:teuthology.orchestra.run.smithi161.stderr: removed all blocklist entries 2024-05-22T21:20:30.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.706+0000 7f3f1f200640 1 -- 172.21.15.161:0/3070199138 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ef4066690 msgr2=0x7f3ef4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:30.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.706+0000 7f3f1f200640 1 --2- 172.21.15.161:0/3070199138 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ef4066690 0x7f3ef4068b50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3f18138a90 tx=0x7f3f00009040 comp rx=0 tx=0).stop 2024-05-22T21:20:30.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.706+0000 7f3f1f200640 1 -- 172.21.15.161:0/3070199138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f1813cfc0 msgr2=0x7f3f1815c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:30.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.706+0000 7f3f1f200640 1 --2- 172.21.15.161:0/3070199138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f1813cfc0 0x7f3f1815c010 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f3f08009d90 tx=0x7f3f08019650 comp rx=0 tx=0).stop 2024-05-22T21:20:30.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.706+0000 7f3f1f200640 1 -- 172.21.15.161:0/3070199138 shutdown_connections 2024-05-22T21:20:30.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.706+0000 7f3f1f200640 1 --2- 172.21.15.161:0/3070199138 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ef4066690 0x7f3ef4068b50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:30.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.706+0000 7f3f1f200640 1 --2- 172.21.15.161:0/3070199138 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f1815af60 0x7f3f181559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:30.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.706+0000 7f3f1f200640 1 --2- 172.21.15.161:0/3070199138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f1813fe50 0x7f3f18157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:30.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.706+0000 7f3f1f200640 1 --2- 172.21.15.161:0/3070199138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3f1813cfc0 0x7f3f1815c010 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:30.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.706+0000 7f3f1f200640 1 -- 172.21.15.161:0/3070199138 >> 172.21.15.161:0/3070199138 conn(0x7f3f1806a2d0 msgr2=0x7f3f18142700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:30.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.706+0000 7f3f1f200640 1 -- 172.21.15.161:0/3070199138 shutdown_connections 2024-05-22T21:20:30.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.706+0000 7f3f1f200640 1 -- 172.21.15.161:0/3070199138 wait complete. 2024-05-22T21:20:30.727 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-22T21:20:30.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.858+0000 7f81ac800640 1 Processor -- start 2024-05-22T21:20:30.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.858+0000 7f81ac800640 1 -- start start 2024-05-22T21:20:30.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.858+0000 7f81ac800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f81a415dd50 0x7f81a406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:30.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.858+0000 7f81ac800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f81a415e130 0x7f81a415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:30.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.858+0000 7f81ac800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a4071670 con 0x7f81a415d410 2024-05-22T21:20:30.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.858+0000 7f81ac800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a415d8a0 con 0x7f81a415e130 2024-05-22T21:20:30.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.858+0000 7f81ac800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a406de40 con 0x7f81a415dd50 2024-05-22T21:20:30.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.858+0000 7f81aa800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f81a415d410 0x7f81a415a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54238/0 (socket says 172.21.15.161:54238) 2024-05-22T21:20:30.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.858+0000 7f81aa800640 1 -- 172.21.15.161:0/187306841 learned_addr learned my addr 172.21.15.161:0/187306841 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:30.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.862+0000 7f81a9e00640 1 --2- 172.21.15.161:0/187306841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f81a415dd50 0x7f81a406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:30.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.862+0000 7f81a9400640 1 --2- 172.21.15.161:0/187306841 >> v2:172.21.15.179:3300/0 conn(0x7f81a415e130 0x7f81a415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:30.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.862+0000 7f81a8a00640 1 -- 172.21.15.161:0/187306841 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078003548 0 0) 0x7f81a4071670 con 0x7f81a415d410 2024-05-22T21:20:30.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.862+0000 7f81a8a00640 1 -- 172.21.15.161:0/187306841 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f818c003530 con 0x7f81a415d410 2024-05-22T21:20:30.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.862+0000 7f81a9400640 1 -- 172.21.15.161:0/187306841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f81a415dd50 msgr2=0x7f81a406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:30.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.862+0000 7f81a9400640 1 --2- 172.21.15.161:0/187306841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f81a415dd50 0x7f81a406d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:30.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.862+0000 7f81a9400640 1 -- 172.21.15.161:0/187306841 >> v1:172.21.15.161:6789/0 conn(0x7f81a415d410 legacy=0x7f81a415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:30.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.862+0000 7f81a9400640 1 -- 172.21.15.161:0/187306841 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a4070a60 con 0x7f81a415e130 2024-05-22T21:20:30.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.862+0000 7f81a9e00640 1 --2- 172.21.15.161:0/187306841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f81a415dd50 0x7f81a406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:20:30.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.862+0000 7f81a9400640 1 --2- 172.21.15.161:0/187306841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81a415e130 0x7f81a415ced0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f81940097c0 tx=0x7f819401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13c633a5a3b8f446 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:30.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.862+0000 7f81a8a00640 1 -- 172.21.15.161:0/187306841 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940ac640 con 0x7f81a415e130 2024-05-22T21:20:30.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.862+0000 7f81a8a00640 1 -- 172.21.15.161:0/187306841 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81940ac7a0 con 0x7f81a415e130 2024-05-22T21:20:30.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.862+0000 7f81a8a00640 1 -- 172.21.15.161:0/187306841 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940acb30 con 0x7f81a415e130 2024-05-22T21:20:30.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81ac800640 1 -- 172.21.15.161:0/187306841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81a415e130 msgr2=0x7f81a415ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:30.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81ac800640 1 --2- 172.21.15.161:0/187306841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81a415e130 0x7f81a415ced0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f81940097c0 tx=0x7f819401c3e0 comp rx=0 tx=0).stop 2024-05-22T21:20:30.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81ac800640 1 -- 172.21.15.161:0/187306841 shutdown_connections 2024-05-22T21:20:30.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81ac800640 1 --2- 172.21.15.161:0/187306841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81a415e130 0x7f81a415ced0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:30.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81ac800640 1 --2- 172.21.15.161:0/187306841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f81a415dd50 0x7f81a406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:30.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81ac800640 1 -- 172.21.15.161:0/187306841 >> 172.21.15.161:0/187306841 conn(0x7f81a40b0bc0 msgr2=0x7f81a40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:30.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81ac800640 1 -- 172.21.15.161:0/187306841 shutdown_connections 2024-05-22T21:20:30.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81ac800640 1 -- 172.21.15.161:0/187306841 wait complete. 2024-05-22T21:20:30.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81ac800640 1 Processor -- start 2024-05-22T21:20:30.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81ac800640 1 -- start start 2024-05-22T21:20:30.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81ac800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81a415d410 0x7f81a4157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:30.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81ac800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81a415dd50 0x7f81a4152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:30.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81ac800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f81a415e130 0x7f81a41567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:30.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81ac800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a4149220 con 0x7f81a415dd50 2024-05-22T21:20:30.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81ac800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a4154c70 con 0x7f81a415d410 2024-05-22T21:20:30.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81ac800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a4154f50 con 0x7f81a415e130 2024-05-22T21:20:30.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81a9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81a415dd50 0x7f81a4152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:30.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81a9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81a415dd50 0x7f81a4152a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55052/0 (socket says 172.21.15.161:55052) 2024-05-22T21:20:30.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81a9e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81a415d410 0x7f81a4157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:30.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81a9400640 1 -- 172.21.15.161:0/258277155 learned_addr learned my addr 172.21.15.161:0/258277155 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:30.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.866+0000 7f81aa800640 1 --2- 172.21.15.161:0/258277155 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f81a415e130 0x7f81a41567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f81a9400640 1 -- 172.21.15.161:0/258277155 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f81a415e130 msgr2=0x7f81a41567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f81a9400640 1 --2- 172.21.15.161:0/258277155 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f81a415e130 0x7f81a41567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f81a9400640 1 -- 172.21.15.161:0/258277155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81a415d410 msgr2=0x7f81a4157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f81a9400640 1 --2- 172.21.15.161:0/258277155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81a415d410 0x7f81a4157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f81a9400640 1 -- 172.21.15.161:0/258277155 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8198009420 con 0x7f81a415dd50 2024-05-22T21:20:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f81a9e00640 1 --2- 172.21.15.161:0/258277155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81a415d410 0x7f81a4157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:20:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f81a9400640 1 --2- 172.21.15.161:0/258277155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81a415dd50 0x7f81a4152a30 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f81940af040 tx=0x7f8194019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f819e000640 1 -- 172.21.15.161:0/258277155 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940b5610 con 0x7f81a415dd50 2024-05-22T21:20:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f819e000640 1 -- 172.21.15.161:0/258277155 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81940b5770 con 0x7f81a415dd50 2024-05-22T21:20:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f819e000640 1 -- 172.21.15.161:0/258277155 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940b5b20 con 0x7f81a415dd50 2024-05-22T21:20:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f81ac800640 1 -- 172.21.15.161:0/258277155 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8194009420 con 0x7f81a415dd50 2024-05-22T21:20:30.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f81ac800640 1 -- 172.21.15.161:0/258277155 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a4157680 con 0x7f81a415dd50 2024-05-22T21:20:30.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f81ac800640 1 -- 172.21.15.161:0/258277155 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a4067220 con 0x7f81a415dd50 2024-05-22T21:20:30.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f819e000640 1 -- 172.21.15.161:0/258277155 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f81940b3040 con 0x7f81a415dd50 2024-05-22T21:20:30.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f819e000640 1 --2- 172.21.15.161:0/258277155 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8188066690 0x7f8188068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:30.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f819e000640 1 -- 172.21.15.161:0/258277155 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f819411e1d0 con 0x7f81a415dd50 2024-05-22T21:20:30.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.870+0000 7f81a9e00640 1 --2- 172.21.15.161:0/258277155 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8188066690 0x7f8188068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:30.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.874+0000 7f81a9e00640 1 --2- 172.21.15.161:0/258277155 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8188066690 0x7f8188068b50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8198011310 tx=0x7f8198009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:30.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:30.874+0000 7f819e000640 1 -- 172.21.15.161:0/258277155 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f81940eb930 con 0x7f81a415dd50 2024-05-22T21:20:31.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.086+0000 7f81ac800640 1 -- 172.21.15.161:0/258277155 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f81a406c310 con 0x7f81a415dd50 2024-05-22T21:20:31.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.086+0000 7f819e000640 1 -- 172.21.15.161:0/258277155 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f81940b1090 con 0x7f81a415dd50 2024-05-22T21:20:31.089 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:31.089 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:20:31.089 INFO:teuthology.orchestra.run.smithi161.stdout: "auth_dump": [ 2024-05-22T21:20:31.089 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.089 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.a", 2024-05-22T21:20:31.089 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m00QTFRAAPSRrpBRY75FlcWf3QxOOfg==", 2024-05-22T21:20:31.089 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.089 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:20:31.089 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:20:31.089 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:20:31.090 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:20:31.090 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.090 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.090 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.b", 2024-05-22T21:20:31.090 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m/ipAIRAA4gj5jGqTpNCr1K2OU/bsAQ==", 2024-05-22T21:20:31.090 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.090 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:20:31.090 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:20:31.090 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:20:31.090 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:20:31.091 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.091 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.c", 2024-05-22T21:20:31.091 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mWlM2HBAAEmJJWMDlTG9E7Qzld2gd5w==", 2024-05-22T21:20:31.091 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.091 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:20:31.091 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:20:31.091 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:20:31.091 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:20:31.091 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.091 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.092 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.d", 2024-05-22T21:20:31.092 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mmsuaKBAAqpCYRZZUGK5ZuxflUBcPgg==", 2024-05-22T21:20:31.092 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.092 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:20:31.092 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:20:31.092 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:20:31.092 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:20:31.092 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.092 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.0", 2024-05-22T21:20:31.092 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05mJZokLhAAh2vbQnRX25iYUIbvVmURuQ==", 2024-05-22T21:20:31.092 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.093 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:20:31.093 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:20:31.093 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:20:31.093 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.093 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.093 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.1", 2024-05-22T21:20:31.093 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05m8kWNNhAA6aaVT2xshFkqnk4G4gOttA==", 2024-05-22T21:20:31.093 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.093 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:20:31.093 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:20:31.093 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:20:31.093 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.094 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.2", 2024-05-22T21:20:31.094 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05ml1k9AxAAtL3XaJJI6cilpmvurh18Pw==", 2024-05-22T21:20:31.094 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.094 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:20:31.094 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:20:31.094 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:20:31.094 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.094 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.094 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.3", 2024-05-22T21:20:31.094 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05mbNBgCxAAas1L+QR4Aez+H1G0EVltfg==", 2024-05-22T21:20:31.095 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.095 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:20:31.095 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:20:31.095 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:20:31.095 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.095 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.095 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.4", 2024-05-22T21:20:31.095 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mOPLPABAAZBGIMF/FasNB27n/7di0AA==", 2024-05-22T21:20:31.095 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.095 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:20:31.095 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:20:31.095 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:20:31.096 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.096 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.5", 2024-05-22T21:20:31.096 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mT1poCRAASXm4tdP01pA1VOviqeoLVw==", 2024-05-22T21:20:31.096 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.096 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:20:31.096 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:20:31.096 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:20:31.096 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.096 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.096 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.6", 2024-05-22T21:20:31.097 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05m1ZPRERAA2S7WeTwysLCZm1OB/qCrAA==", 2024-05-22T21:20:31.097 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.097 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:20:31.097 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:20:31.097 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:20:31.097 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.097 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.097 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.7", 2024-05-22T21:20:31.097 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mCrUhGhAAvXoFWV7X3303rfXaxmfWqA==", 2024-05-22T21:20:31.097 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.097 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:20:31.097 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:20:31.098 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:20:31.098 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.098 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.098 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.0", 2024-05-22T21:20:31.098 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mh9B7LBAAevhvj5KpwP9pEIpJcxUvFA==", 2024-05-22T21:20:31.098 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.098 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:20:31.098 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow r", 2024-05-22T21:20:31.098 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-22T21:20:31.098 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.099 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.1", 2024-05-22T21:20:31.099 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mMnHtNRAAg/a/Mb1oRqx73d9efIHCfA==", 2024-05-22T21:20:31.099 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.099 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:20:31.099 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:20:31.099 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:20:31.099 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:20:31.099 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.099 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.099 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.2", 2024-05-22T21:20:31.100 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mOzdkMhAAbTiNdvyxqIyrluMiZW0cOw==", 2024-05-22T21:20:31.100 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.100 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:20:31.100 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:20:31.100 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:20:31.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:20:31.100 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.100 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.3", 2024-05-22T21:20:31.100 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDYX05mr4oxABAA1+bQQbxhoeU+kb0OCa+Wyw==", 2024-05-22T21:20:31.100 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.100 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:20:31.100 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:20:31.101 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:20:31.101 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:20:31.101 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.101 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.admin", 2024-05-22T21:20:31.101 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDWX05mHZY0NRAAiBvMDkrCZ+MgG2iwBH+bcg==", 2024-05-22T21:20:31.101 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.101 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:20:31.101 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:20:31.101 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:20:31.101 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:20:31.101 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.102 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.x", 2024-05-22T21:20:31.102 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mH2kuERAAgigMUWK3M3WaMXlqGUcNrw==", 2024-05-22T21:20:31.102 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.102 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:20:31.102 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:20:31.102 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:20:31.102 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:20:31.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:31.103 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.y", 2024-05-22T21:20:31.103 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mZRdiDRAA0vUwD+rPSfyhjgA+9XPp5w==", 2024-05-22T21:20:31.103 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:20:31.103 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:20:31.103 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:20:31.103 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:20:31.103 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.103 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:31.103 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:20:31.103 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:20:31.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.086+0000 7f81ac800640 1 -- 172.21.15.161:0/258277155 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8188066690 msgr2=0x7f8188068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.086+0000 7f81ac800640 1 --2- 172.21.15.161:0/258277155 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8188066690 0x7f8188068b50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8198011310 tx=0x7f8198009190 comp rx=0 tx=0).stop 2024-05-22T21:20:31.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.086+0000 7f81ac800640 1 -- 172.21.15.161:0/258277155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81a415dd50 msgr2=0x7f81a4152a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.086+0000 7f81ac800640 1 --2- 172.21.15.161:0/258277155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81a415dd50 0x7f81a4152a30 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f81940af040 tx=0x7f8194019180 comp rx=0 tx=0).stop 2024-05-22T21:20:31.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.086+0000 7f81ac800640 1 -- 172.21.15.161:0/258277155 shutdown_connections 2024-05-22T21:20:31.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.086+0000 7f81ac800640 1 --2- 172.21.15.161:0/258277155 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8188066690 0x7f8188068b50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.086+0000 7f81ac800640 1 --2- 172.21.15.161:0/258277155 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f81a415e130 0x7f81a41567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.086+0000 7f81ac800640 1 --2- 172.21.15.161:0/258277155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81a415dd50 0x7f81a4152a30 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.086+0000 7f81ac800640 1 --2- 172.21.15.161:0/258277155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81a415d410 0x7f81a4157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.086+0000 7f81ac800640 1 -- 172.21.15.161:0/258277155 >> 172.21.15.161:0/258277155 conn(0x7f81a40b0bc0 msgr2=0x7f81a40adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:31.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.086+0000 7f81ac800640 1 -- 172.21.15.161:0/258277155 shutdown_connections 2024-05-22T21:20:31.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.086+0000 7f81ac800640 1 -- 172.21.15.161:0/258277155 wait complete. 2024-05-22T21:20:31.111 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:20:31.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2b4c00640 1 Processor -- start 2024-05-22T21:20:31.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2b4c00640 1 -- start start 2024-05-22T21:20:31.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2b4c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff2b015dd50 0x7ff2b006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2b4c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff2b015d5e0 0x7ff2b015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2b4c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b0071670 con 0x7ff2b015e130 2024-05-22T21:20:31.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2b4c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b006de40 con 0x7ff2b015dd50 2024-05-22T21:20:31.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2b4c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b0070a60 con 0x7ff2b015d5e0 2024-05-22T21:20:31.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2ade00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff2b015dd50 0x7ff2b006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2ad400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff2b015e130 0x7ff2b015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54244/0 (socket says 172.21.15.161:54244) 2024-05-22T21:20:31.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2ade00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff2b015dd50 0x7ff2b006d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55060/0 (socket says 172.21.15.161:55060) 2024-05-22T21:20:31.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2ad400640 1 -- 172.21.15.161:0/264658991 learned_addr learned my addr 172.21.15.161:0/264658991 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:31.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2ae800640 1 --2- 172.21.15.161:0/264658991 >> v2:172.21.15.179:3300/0 conn(0x7ff2b015d5e0 0x7ff2b015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2ade00640 1 -- 172.21.15.161:0/264658991 >> v2:172.21.15.179:3300/0 conn(0x7ff2b015d5e0 msgr2=0x7ff2b015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:31.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2ade00640 1 --2- 172.21.15.161:0/264658991 >> v2:172.21.15.179:3300/0 conn(0x7ff2b015d5e0 0x7ff2b015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2ade00640 1 -- 172.21.15.161:0/264658991 >> v1:172.21.15.161:6789/0 conn(0x7ff2b015e130 legacy=0x7ff2b015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2ade00640 1 -- 172.21.15.161:0/264658991 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b0070d40 con 0x7ff2b015dd50 2024-05-22T21:20:31.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.246+0000 7ff2aca00640 1 -- 172.21.15.161:0/264658991 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228885692 0 0) 0x7ff2b0071670 con 0x7ff2b015e130 2024-05-22T21:20:31.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.250+0000 7ff2ade00640 1 --2- 172.21.15.161:0/264658991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2b015dd50 0x7ff2b006d900 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7ff2b015e5f0 tx=0x7ff2a001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2367db9a619ac3c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:31.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.250+0000 7ff2aca00640 1 -- 172.21.15.161:0/264658991 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00ab650 con 0x7ff2b015dd50 2024-05-22T21:20:31.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.250+0000 7ff2aca00640 1 -- 172.21.15.161:0/264658991 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a00ab7b0 con 0x7ff2b015dd50 2024-05-22T21:20:31.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.250+0000 7ff2aca00640 1 -- 172.21.15.161:0/264658991 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00abb40 con 0x7ff2b015dd50 2024-05-22T21:20:31.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.250+0000 7ff2b4c00640 1 -- 172.21.15.161:0/264658991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2b015dd50 msgr2=0x7ff2b006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.250+0000 7ff2b4c00640 1 --2- 172.21.15.161:0/264658991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2b015dd50 0x7ff2b006d900 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7ff2b015e5f0 tx=0x7ff2a001c450 comp rx=0 tx=0).stop 2024-05-22T21:20:31.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.250+0000 7ff2b4c00640 1 -- 172.21.15.161:0/264658991 shutdown_connections 2024-05-22T21:20:31.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.250+0000 7ff2b4c00640 1 --2- 172.21.15.161:0/264658991 >> v2:172.21.15.179:3300/0 conn(0x7ff2b015d5e0 0x7ff2b015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.250+0000 7ff2b4c00640 1 --2- 172.21.15.161:0/264658991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2b015dd50 0x7ff2b006d900 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.250+0000 7ff2b4c00640 1 -- 172.21.15.161:0/264658991 >> 172.21.15.161:0/264658991 conn(0x7ff2b00b0bc0 msgr2=0x7ff2b00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:31.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.250+0000 7ff2b4c00640 1 -- 172.21.15.161:0/264658991 shutdown_connections 2024-05-22T21:20:31.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.250+0000 7ff2b4c00640 1 -- 172.21.15.161:0/264658991 wait complete. 2024-05-22T21:20:31.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2b4c00640 1 Processor -- start 2024-05-22T21:20:31.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2b4c00640 1 -- start start 2024-05-22T21:20:31.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2b4c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2b015d5e0 0x7ff2b00ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2b4c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2b015dd50 0x7ff2b01f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2b4c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2b015e130 0x7ff2b01f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2b4c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b0152da0 con 0x7ff2b015d5e0 2024-05-22T21:20:31.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2b4c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b0151fe0 con 0x7ff2b015dd50 2024-05-22T21:20:31.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2b4c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b01522c0 con 0x7ff2b015e130 2024-05-22T21:20:31.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2ade00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2b015d5e0 0x7ff2b00ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2ade00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2b015d5e0 0x7ff2b00ccf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55066/0 (socket says 172.21.15.161:55066) 2024-05-22T21:20:31.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2ade00640 1 -- 172.21.15.161:0/3506272727 learned_addr learned my addr 172.21.15.161:0/3506272727 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:31.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2ae800640 1 --2- 172.21.15.161:0/3506272727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2b015e130 0x7ff2b01f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2ad400640 1 --2- 172.21.15.161:0/3506272727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2b015dd50 0x7ff2b01f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2ade00640 1 -- 172.21.15.161:0/3506272727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2b015e130 msgr2=0x7ff2b01f5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2ade00640 1 --2- 172.21.15.161:0/3506272727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2b015e130 0x7ff2b01f5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2ade00640 1 -- 172.21.15.161:0/3506272727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2b015dd50 msgr2=0x7ff2b01f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2ade00640 1 --2- 172.21.15.161:0/3506272727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2b015dd50 0x7ff2b01f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2ade00640 1 -- 172.21.15.161:0/3506272727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a0009420 con 0x7ff2b015d5e0 2024-05-22T21:20:31.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2ad400640 1 --2- 172.21.15.161:0/3506272727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2b015dd50 0x7ff2b01f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:31.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2ade00640 1 --2- 172.21.15.161:0/3506272727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2b015d5e0 0x7ff2b00ccf20 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7ff2a001ab00 tx=0x7ff2a001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:31.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2a6000640 1 -- 172.21.15.161:0/3506272727 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0004220 con 0x7ff2b015d5e0 2024-05-22T21:20:31.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2a6000640 1 -- 172.21.15.161:0/3506272727 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a00b9070 con 0x7ff2b015d5e0 2024-05-22T21:20:31.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2b4c00640 1 -- 172.21.15.161:0/3506272727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b01525a0 con 0x7ff2b015d5e0 2024-05-22T21:20:31.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2a6000640 1 -- 172.21.15.161:0/3506272727 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0003620 con 0x7ff2b015d5e0 2024-05-22T21:20:31.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2b4c00640 1 -- 172.21.15.161:0/3506272727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b00cd460 con 0x7ff2b015d5e0 2024-05-22T21:20:31.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.254+0000 7ff2b4c00640 1 -- 172.21.15.161:0/3506272727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b00cb9e0 con 0x7ff2b015d5e0 2024-05-22T21:20:31.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.258+0000 7ff2a6000640 1 -- 172.21.15.161:0/3506272727 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff2a0003780 con 0x7ff2b015d5e0 2024-05-22T21:20:31.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.258+0000 7ff2a6000640 1 --2- 172.21.15.161:0/3506272727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff28c066690 0x7ff28c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.258+0000 7ff2a6000640 1 -- 172.21.15.161:0/3506272727 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff2a00bb070 con 0x7ff2b015d5e0 2024-05-22T21:20:31.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.258+0000 7ff2ad400640 1 --2- 172.21.15.161:0/3506272727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff28c066690 0x7ff28c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.258+0000 7ff2ad400640 1 --2- 172.21.15.161:0/3506272727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff28c066690 0x7ff28c068b50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff2b015b990 tx=0x7ff298007cb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:31.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.262+0000 7ff2a6000640 1 -- 172.21.15.161:0/3506272727 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff2a00ec0e0 con 0x7ff2b015d5e0 2024-05-22T21:20:31.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.418+0000 7ff2b4c00640 1 -- 172.21.15.161:0/3506272727 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2b0070710 con 0x7ff2b015d5e0 2024-05-22T21:20:31.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.418+0000 7ff2a6000640 1 -- 172.21.15.161:0/3506272727 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+12448 (secure 0 0 0) 0x7ff2b0070710 con 0x7ff2b015d5e0 2024-05-22T21:20:31.421 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:31.421 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":154,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:20:30.698799+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:20:31.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.418+0000 7ff2b4c00640 1 -- 172.21.15.161:0/3506272727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff28c066690 msgr2=0x7ff28c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.418+0000 7ff2b4c00640 1 --2- 172.21.15.161:0/3506272727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff28c066690 0x7ff28c068b50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff2b015b990 tx=0x7ff298007cb0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.418+0000 7ff2b4c00640 1 -- 172.21.15.161:0/3506272727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2b015d5e0 msgr2=0x7ff2b00ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.418+0000 7ff2b4c00640 1 --2- 172.21.15.161:0/3506272727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2b015d5e0 0x7ff2b00ccf20 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7ff2a001ab00 tx=0x7ff2a001ac90 comp rx=0 tx=0).stop 2024-05-22T21:20:31.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.418+0000 7ff2b4c00640 1 -- 172.21.15.161:0/3506272727 shutdown_connections 2024-05-22T21:20:31.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.418+0000 7ff2b4c00640 1 --2- 172.21.15.161:0/3506272727 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff28c066690 0x7ff28c068b50 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.418+0000 7ff2b4c00640 1 --2- 172.21.15.161:0/3506272727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2b015e130 0x7ff2b01f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.418+0000 7ff2b4c00640 1 --2- 172.21.15.161:0/3506272727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2b015dd50 0x7ff2b01f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.418+0000 7ff2b4c00640 1 --2- 172.21.15.161:0/3506272727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2b015d5e0 0x7ff2b00ccf20 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.418+0000 7ff2b4c00640 1 -- 172.21.15.161:0/3506272727 >> 172.21.15.161:0/3506272727 conn(0x7ff2b00b0bc0 msgr2=0x7ff2b006d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:31.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.418+0000 7ff2b4c00640 1 -- 172.21.15.161:0/3506272727 shutdown_connections 2024-05-22T21:20:31.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.418+0000 7ff2b4c00640 1 -- 172.21.15.161:0/3506272727 wait complete. 2024-05-22T21:20:31.442 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:20:31.442 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:20:31.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.578+0000 7f64c3e00640 1 Processor -- start 2024-05-22T21:20:31.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.578+0000 7f64c3e00640 1 -- start start 2024-05-22T21:20:31.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.578+0000 7f64c3e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f64bc13cfc0 0x7f64bc1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.578+0000 7f64c3e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64bc15af60 0x7f64bc13f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.578+0000 7f64c3e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64bc1400d0 con 0x7f64bc13fc80 2024-05-22T21:20:31.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.578+0000 7f64c3e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64bc13d3a0 con 0x7f64bc13cfc0 2024-05-22T21:20:31.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.578+0000 7f64c3e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc0cd920 con 0x7f64bc15af60 2024-05-22T21:20:31.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.582+0000 7f64c0a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64bc15af60 0x7f64bc13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.582+0000 7f64c1e00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f64bc13fc80 0x7f64bc15dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:60766/0 (socket says 172.21.15.161:60766) 2024-05-22T21:20:31.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.582+0000 7f64c1400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f64bc13cfc0 0x7f64bc1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.582+0000 7f64c1e00640 1 -- 172.21.15.161:0/2456013103 learned_addr learned my addr 172.21.15.161:0/2456013103 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:31.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.582+0000 7f64bbe00640 1 -- 172.21.15.161:0/2456013103 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611280924 0 0) 0x7f64b40014a0 con 0x7f64bc13fc80 2024-05-22T21:20:31.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.582+0000 7f64bbe00640 1 -- 172.21.15.161:0/2456013103 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64a4005740 con 0x7f64bc13fc80 2024-05-22T21:20:31.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.582+0000 7f64c0a00640 1 -- 172.21.15.161:0/2456013103 >> v2:172.21.15.179:3300/0 conn(0x7f64bc13cfc0 msgr2=0x7f64bc1402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:31.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.582+0000 7f64c0a00640 1 --2- 172.21.15.161:0/2456013103 >> v2:172.21.15.179:3300/0 conn(0x7f64bc13cfc0 0x7f64bc1402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.582+0000 7f64c0a00640 1 -- 172.21.15.161:0/2456013103 >> v1:172.21.15.179:6789/0 conn(0x7f64bc13fc80 legacy=0x7f64bc15dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.582+0000 7f64c0a00640 1 -- 172.21.15.161:0/2456013103 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64bc0cdc00 con 0x7f64bc15af60 2024-05-22T21:20:31.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.582+0000 7f64c1400640 1 --2- 172.21.15.161:0/2456013103 >> v2:172.21.15.179:3300/0 conn(0x7f64bc13cfc0 0x7f64bc1402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:20:31.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.582+0000 7f64c0a00640 1 --2- 172.21.15.161:0/2456013103 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64bc15af60 0x7f64bc13f740 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f64a80097c0 tx=0x7f64a801c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57e84e24c924e677 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:31.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64bbe00640 1 -- 172.21.15.161:0/2456013103 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a80ab600 con 0x7f64bc15af60 2024-05-22T21:20:31.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64bbe00640 1 -- 172.21.15.161:0/2456013103 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a80a7050 con 0x7f64bc15af60 2024-05-22T21:20:31.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64bbe00640 1 -- 172.21.15.161:0/2456013103 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a80ab980 con 0x7f64bc15af60 2024-05-22T21:20:31.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c3e00640 1 -- 172.21.15.161:0/2456013103 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64bc15af60 msgr2=0x7f64bc13f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c3e00640 1 --2- 172.21.15.161:0/2456013103 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64bc15af60 0x7f64bc13f740 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f64a80097c0 tx=0x7f64a801c3d0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c3e00640 1 -- 172.21.15.161:0/2456013103 shutdown_connections 2024-05-22T21:20:31.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c3e00640 1 --2- 172.21.15.161:0/2456013103 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64bc15af60 0x7f64bc13f740 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c3e00640 1 --2- 172.21.15.161:0/2456013103 >> v2:172.21.15.179:3300/0 conn(0x7f64bc13cfc0 0x7f64bc1402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c3e00640 1 -- 172.21.15.161:0/2456013103 >> 172.21.15.161:0/2456013103 conn(0x7f64bc06a2d0 msgr2=0x7f64bc06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:31.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c3e00640 1 -- 172.21.15.161:0/2456013103 shutdown_connections 2024-05-22T21:20:31.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c3e00640 1 -- 172.21.15.161:0/2456013103 wait complete. 2024-05-22T21:20:31.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c3e00640 1 Processor -- start 2024-05-22T21:20:31.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c3e00640 1 -- start start 2024-05-22T21:20:31.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c3e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64bc13cfc0 0x7f64bc134980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64bc13fc80 0x7f64bc1386f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c3e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64bc15af60 0x7f64bc1542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c3e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc132070 con 0x7f64bc13fc80 2024-05-22T21:20:31.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c3e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc0b3bc0 con 0x7f64bc13cfc0 2024-05-22T21:20:31.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c3e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc0b3ea0 con 0x7f64bc15af60 2024-05-22T21:20:31.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c0a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64bc13fc80 0x7f64bc1386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c0a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64bc13fc80 0x7f64bc1386f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55072/0 (socket says 172.21.15.161:55072) 2024-05-22T21:20:31.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c0a00640 1 -- 172.21.15.161:0/1590689171 learned_addr learned my addr 172.21.15.161:0/1590689171 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:31.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c1400640 1 --2- 172.21.15.161:0/1590689171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64bc13cfc0 0x7f64bc134980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.586+0000 7f64c1e00640 1 --2- 172.21.15.161:0/1590689171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64bc15af60 0x7f64bc1542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64c0a00640 1 -- 172.21.15.161:0/1590689171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64bc15af60 msgr2=0x7f64bc1542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64c0a00640 1 --2- 172.21.15.161:0/1590689171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64bc15af60 0x7f64bc1542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64c0a00640 1 -- 172.21.15.161:0/1590689171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64bc13cfc0 msgr2=0x7f64bc134980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64c0a00640 1 --2- 172.21.15.161:0/1590689171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64bc13cfc0 0x7f64bc134980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64c0a00640 1 -- 172.21.15.161:0/1590689171 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b0009420 con 0x7f64bc13fc80 2024-05-22T21:20:31.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64c1400640 1 --2- 172.21.15.161:0/1590689171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64bc13cfc0 0x7f64bc134980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:20:31.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64c0a00640 1 --2- 172.21.15.161:0/1590689171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64bc13fc80 0x7f64bc1386f0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f64a801aa80 tx=0x7f64a8019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:31.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64b9600640 1 -- 172.21.15.161:0/1590689171 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a80b4400 con 0x7f64bc13fc80 2024-05-22T21:20:31.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64b9600640 1 -- 172.21.15.161:0/1590689171 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a80b4580 con 0x7f64bc13fc80 2024-05-22T21:20:31.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64c3e00640 1 -- 172.21.15.161:0/1590689171 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a8009420 con 0x7f64bc13fc80 2024-05-22T21:20:31.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64b9600640 1 -- 172.21.15.161:0/1590689171 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a80b4930 con 0x7f64bc13fc80 2024-05-22T21:20:31.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64c3e00640 1 -- 172.21.15.161:0/1590689171 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64bc132560 con 0x7f64bc13fc80 2024-05-22T21:20:31.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64c3e00640 1 -- 172.21.15.161:0/1590689171 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64bc0b4060 con 0x7f64bc13fc80 2024-05-22T21:20:31.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64b9600640 1 -- 172.21.15.161:0/1590689171 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f64a80b2040 con 0x7f64bc13fc80 2024-05-22T21:20:31.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64b9600640 1 --2- 172.21.15.161:0/1590689171 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f649c066690 0x7f649c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64b9600640 1 -- 172.21.15.161:0/1590689171 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f64a811dd50 con 0x7f64bc13fc80 2024-05-22T21:20:31.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.590+0000 7f64c1400640 1 --2- 172.21.15.161:0/1590689171 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f649c066690 0x7f649c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.594+0000 7f64c1400640 1 --2- 172.21.15.161:0/1590689171 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f649c066690 0x7f649c068b50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f64b0011310 tx=0x7f64b0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:31.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.594+0000 7f64b9600640 1 -- 172.21.15.161:0/1590689171 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f64a80eb4b0 con 0x7f64bc13fc80 2024-05-22T21:20:31.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.750+0000 7f64c3e00640 1 -- 172.21.15.161:0/1590689171 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f64bc067220 con 0x7f64bc13fc80 2024-05-22T21:20:31.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.750+0000 7f64b9600640 1 -- 172.21.15.161:0/1590689171 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v154) v1 ==== 93+0+10 (secure 0 0 0) 0x7f64a80eead0 con 0x7f64bc13fc80 2024-05-22T21:20:31.753 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:20:31.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.750+0000 7f64c3e00640 1 -- 172.21.15.161:0/1590689171 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f649c066690 msgr2=0x7f649c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.750+0000 7f64c3e00640 1 --2- 172.21.15.161:0/1590689171 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f649c066690 0x7f649c068b50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f64b0011310 tx=0x7f64b0009190 comp rx=0 tx=0).stop 2024-05-22T21:20:31.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.750+0000 7f64c3e00640 1 -- 172.21.15.161:0/1590689171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64bc13fc80 msgr2=0x7f64bc1386f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.750+0000 7f64c3e00640 1 --2- 172.21.15.161:0/1590689171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64bc13fc80 0x7f64bc1386f0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f64a801aa80 tx=0x7f64a8019170 comp rx=0 tx=0).stop 2024-05-22T21:20:31.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.750+0000 7f64c3e00640 1 -- 172.21.15.161:0/1590689171 shutdown_connections 2024-05-22T21:20:31.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.750+0000 7f64c3e00640 1 --2- 172.21.15.161:0/1590689171 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f649c066690 0x7f649c068b50 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.750+0000 7f64c3e00640 1 --2- 172.21.15.161:0/1590689171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64bc15af60 0x7f64bc1542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.750+0000 7f64c3e00640 1 --2- 172.21.15.161:0/1590689171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64bc13fc80 0x7f64bc1386f0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.750+0000 7f64c3e00640 1 --2- 172.21.15.161:0/1590689171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64bc13cfc0 0x7f64bc134980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.750+0000 7f64c3e00640 1 -- 172.21.15.161:0/1590689171 >> 172.21.15.161:0/1590689171 conn(0x7f64bc06a2d0 msgr2=0x7f64bc159230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:31.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.750+0000 7f64c3e00640 1 -- 172.21.15.161:0/1590689171 shutdown_connections 2024-05-22T21:20:31.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.754+0000 7f64c3e00640 1 -- 172.21.15.161:0/1590689171 wait complete. 2024-05-22T21:20:31.775 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-22T21:20:31.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.906+0000 7f2c10200640 1 Processor -- start 2024-05-22T21:20:31.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.906+0000 7f2c10200640 1 -- start start 2024-05-22T21:20:31.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.906+0000 7f2c10200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2c08143ec0 0x7f2c0815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.906+0000 7f2c10200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2c0815ec90 0x7f2c0815f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.906+0000 7f2c10200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c081442a0 con 0x7f2c080715a0 2024-05-22T21:20:31.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.906+0000 7f2c10200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c0815f660 con 0x7f2c08143ec0 2024-05-22T21:20:31.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.906+0000 7f2c10200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c08070500 con 0x7f2c0815ec90 2024-05-22T21:20:31.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.906+0000 7f2c0d800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2c08143ec0 0x7f2c0815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.906+0000 7f2c0d800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2c08143ec0 0x7f2c0815f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55082/0 (socket says 172.21.15.161:55082) 2024-05-22T21:20:31.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.906+0000 7f2c0d800640 1 -- 172.21.15.161:0/2260012915 learned_addr learned my addr 172.21.15.161:0/2260012915 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:31.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.906+0000 7f2c0e200640 1 --2- 172.21.15.161:0/2260012915 >> v2:172.21.15.179:3300/0 conn(0x7f2c0815ec90 0x7f2c0815f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.910+0000 7f2c0d800640 1 -- 172.21.15.161:0/2260012915 >> v2:172.21.15.179:3300/0 conn(0x7f2c0815ec90 msgr2=0x7f2c0815f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:31.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.910+0000 7f2c0d800640 1 --2- 172.21.15.161:0/2260012915 >> v2:172.21.15.179:3300/0 conn(0x7f2c0815ec90 0x7f2c0815f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.910+0000 7f2c0d800640 1 -- 172.21.15.161:0/2260012915 >> v1:172.21.15.179:6789/0 conn(0x7f2c080715a0 legacy=0x7f2c0806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:31.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.910+0000 7f2c0d800640 1 -- 172.21.15.161:0/2260012915 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c080707e0 con 0x7f2c08143ec0 2024-05-22T21:20:31.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.910+0000 7f2c0d800640 1 --2- 172.21.15.161:0/2260012915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c08143ec0 0x7f2c0815f830 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f2c080cedc0 tx=0x7f2bfc01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88978659858efa06 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:31.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.910+0000 7f2c07e00640 1 -- 172.21.15.161:0/2260012915 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0ac410 con 0x7f2c08143ec0 2024-05-22T21:20:31.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.910+0000 7f2c07e00640 1 -- 172.21.15.161:0/2260012915 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bfc0ac570 con 0x7f2c08143ec0 2024-05-22T21:20:31.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.910+0000 7f2c07e00640 1 -- 172.21.15.161:0/2260012915 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0ac900 con 0x7f2c08143ec0 2024-05-22T21:20:31.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c10200640 1 -- 172.21.15.161:0/2260012915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c08143ec0 msgr2=0x7f2c0815f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c10200640 1 --2- 172.21.15.161:0/2260012915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c08143ec0 0x7f2c0815f830 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f2c080cedc0 tx=0x7f2bfc01c4b0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c10200640 1 -- 172.21.15.161:0/2260012915 shutdown_connections 2024-05-22T21:20:31.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c10200640 1 --2- 172.21.15.161:0/2260012915 >> v2:172.21.15.179:3300/0 conn(0x7f2c0815ec90 0x7f2c0815f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c10200640 1 --2- 172.21.15.161:0/2260012915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c08143ec0 0x7f2c0815f830 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c10200640 1 -- 172.21.15.161:0/2260012915 >> 172.21.15.161:0/2260012915 conn(0x7f2c080b0bc0 msgr2=0x7f2c080b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:31.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c10200640 1 -- 172.21.15.161:0/2260012915 shutdown_connections 2024-05-22T21:20:31.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c10200640 1 -- 172.21.15.161:0/2260012915 wait complete. 2024-05-22T21:20:31.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c10200640 1 Processor -- start 2024-05-22T21:20:31.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c10200640 1 -- start start 2024-05-22T21:20:31.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c10200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c08143ec0 0x7f2c0813df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c10200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c0815ec90 0x7f2c080accb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c10200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c080ad1f0 0x7f2c080abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c10200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c080a9500 con 0x7f2c0815ec90 2024-05-22T21:20:31.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c10200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c080a8880 con 0x7f2c080ad1f0 2024-05-22T21:20:31.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c10200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c080a8b60 con 0x7f2c08143ec0 2024-05-22T21:20:31.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c0ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c0815ec90 0x7f2c080accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c0ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c0815ec90 0x7f2c080accb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55084/0 (socket says 172.21.15.161:55084) 2024-05-22T21:20:31.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c0ce00640 1 -- 172.21.15.161:0/483262702 learned_addr learned my addr 172.21.15.161:0/483262702 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:31.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c0d800640 1 --2- 172.21.15.161:0/483262702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c08143ec0 0x7f2c0813df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c0e200640 1 --2- 172.21.15.161:0/483262702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c080ad1f0 0x7f2c080abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c0ce00640 1 -- 172.21.15.161:0/483262702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c08143ec0 msgr2=0x7f2c0813df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c0ce00640 1 --2- 172.21.15.161:0/483262702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c08143ec0 0x7f2c0813df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c0ce00640 1 -- 172.21.15.161:0/483262702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c080ad1f0 msgr2=0x7f2c080abbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:31.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c0ce00640 1 --2- 172.21.15.161:0/483262702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c080ad1f0 0x7f2c080abbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:31.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.914+0000 7f2c0ce00640 1 -- 172.21.15.161:0/483262702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc0094d0 con 0x7f2c0815ec90 2024-05-22T21:20:31.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.918+0000 7f2c0ce00640 1 --2- 172.21.15.161:0/483262702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c0815ec90 0x7f2c080accb0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400c410 tx=0x7f2bf400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:31.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.918+0000 7f2c05600640 1 -- 172.21.15.161:0/483262702 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4011790 con 0x7f2c0815ec90 2024-05-22T21:20:31.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.918+0000 7f2c05600640 1 -- 172.21.15.161:0/483262702 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf40118f0 con 0x7f2c0815ec90 2024-05-22T21:20:31.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.918+0000 7f2c05600640 1 -- 172.21.15.161:0/483262702 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4011ce0 con 0x7f2c0815ec90 2024-05-22T21:20:31.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.918+0000 7f2c10200640 1 -- 172.21.15.161:0/483262702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c080a8e40 con 0x7f2c0815ec90 2024-05-22T21:20:31.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.918+0000 7f2c10200640 1 -- 172.21.15.161:0/483262702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c080a84a0 con 0x7f2c0815ec90 2024-05-22T21:20:31.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.918+0000 7f2c10200640 1 -- 172.21.15.161:0/483262702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c080aa5b0 con 0x7f2c0815ec90 2024-05-22T21:20:31.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.918+0000 7f2c05600640 1 -- 172.21.15.161:0/483262702 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2bf401c460 con 0x7f2c0815ec90 2024-05-22T21:20:31.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.918+0000 7f2c05600640 1 --2- 172.21.15.161:0/483262702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2be80666c0 0x7f2be8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:31.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.918+0000 7f2c05600640 1 -- 172.21.15.161:0/483262702 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2bf4083a00 con 0x7f2c0815ec90 2024-05-22T21:20:31.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.918+0000 7f2c0d800640 1 --2- 172.21.15.161:0/483262702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2be80666c0 0x7f2be8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:31.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.922+0000 7f2c0d800640 1 --2- 172.21.15.161:0/483262702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2be80666c0 0x7f2be8068b80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0ae000 tx=0x7f2bfc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:31.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:31.922+0000 7f2c05600640 1 -- 172.21.15.161:0/483262702 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2c080aa5b0 con 0x7f2c0815ec90 2024-05-22T21:20:32.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.078+0000 7f2c10200640 1 -- 172.21.15.161:0/483262702 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2c080aa5b0 con 0x7f2c0815ec90 2024-05-22T21:20:32.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.078+0000 7f2c05600640 1 -- 172.21.15.161:0/483262702 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v154) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2c080aa5b0 con 0x7f2c0815ec90 2024-05-22T21:20:32.082 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:32.082 INFO:teuthology.orchestra.run.smithi161.stdout:[ 2024-05-22T21:20:32.082 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:20:32.082 INFO:teuthology.orchestra.run.smithi161.stdout: "poolnum": 1, 2024-05-22T21:20:32.082 INFO:teuthology.orchestra.run.smithi161.stdout: "poolname": ".mgr" 2024-05-22T21:20:32.082 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:20:32.082 INFO:teuthology.orchestra.run.smithi161.stdout:] 2024-05-22T21:20:32.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.082+0000 7f2c10200640 1 -- 172.21.15.161:0/483262702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2be80666c0 msgr2=0x7f2be8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:32.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.082+0000 7f2c10200640 1 --2- 172.21.15.161:0/483262702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2be80666c0 0x7f2be8068b80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0ae000 tx=0x7f2bfc009040 comp rx=0 tx=0).stop 2024-05-22T21:20:32.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.082+0000 7f2c10200640 1 -- 172.21.15.161:0/483262702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c0815ec90 msgr2=0x7f2c080accb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:32.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.082+0000 7f2c10200640 1 --2- 172.21.15.161:0/483262702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c0815ec90 0x7f2c080accb0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400c410 tx=0x7f2bf400c8d0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.082+0000 7f2c10200640 1 -- 172.21.15.161:0/483262702 shutdown_connections 2024-05-22T21:20:32.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.082+0000 7f2c10200640 1 --2- 172.21.15.161:0/483262702 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2be80666c0 0x7f2be8068b80 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.082+0000 7f2c10200640 1 --2- 172.21.15.161:0/483262702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c080ad1f0 0x7f2c080abbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.082+0000 7f2c10200640 1 --2- 172.21.15.161:0/483262702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c0815ec90 0x7f2c080accb0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.082+0000 7f2c10200640 1 --2- 172.21.15.161:0/483262702 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c08143ec0 0x7f2c0813df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.082+0000 7f2c10200640 1 -- 172.21.15.161:0/483262702 >> 172.21.15.161:0/483262702 conn(0x7f2c080b0bc0 msgr2=0x7f2c0815d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:32.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.082+0000 7f2c10200640 1 -- 172.21.15.161:0/483262702 shutdown_connections 2024-05-22T21:20:32.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.082+0000 7f2c10200640 1 -- 172.21.15.161:0/483262702 wait complete. 2024-05-22T21:20:32.103 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-22T21:20:32.104 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-22T21:20:32.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.238+0000 7f72e3600640 1 Processor -- start 2024-05-22T21:20:32.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.238+0000 7f72e3600640 1 -- start start 2024-05-22T21:20:32.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.242+0000 7f72e3600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f72dc13cfe0 0x7f72dc15dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:32.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.242+0000 7f72e3600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f72dc13d560 0x7f72dc15e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:32.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.242+0000 7f72e3600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72dc0ce830 con 0x7f72dc0cdb00 2024-05-22T21:20:32.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.242+0000 7f72e3600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72dc140560 con 0x7f72dc13cfe0 2024-05-22T21:20:32.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.242+0000 7f72e3600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72dc0ceb10 con 0x7f72dc13d560 2024-05-22T21:20:32.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.242+0000 7f72e0c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f72dc13cfe0 0x7f72dc15dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:32.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.242+0000 7f72e0c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f72dc13cfe0 0x7f72dc15dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55100/0 (socket says 172.21.15.161:55100) 2024-05-22T21:20:32.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.242+0000 7f72e0c00640 1 -- 172.21.15.161:0/47200637 learned_addr learned my addr 172.21.15.161:0/47200637 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:32.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.242+0000 7f72dbe00640 1 --2- 172.21.15.161:0/47200637 >> v2:172.21.15.179:3300/0 conn(0x7f72dc13d560 0x7f72dc15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:32.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.242+0000 7f72e0c00640 1 -- 172.21.15.161:0/47200637 >> v2:172.21.15.179:3300/0 conn(0x7f72dc13d560 msgr2=0x7f72dc15e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:32.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.242+0000 7f72e0c00640 1 --2- 172.21.15.161:0/47200637 >> v2:172.21.15.179:3300/0 conn(0x7f72dc13d560 0x7f72dc15e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.242+0000 7f72e0c00640 1 -- 172.21.15.161:0/47200637 >> v1:172.21.15.179:6789/0 conn(0x7f72dc0cdb00 legacy=0x7f72dc0cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:32.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.242+0000 7f72e0c00640 1 -- 172.21.15.161:0/47200637 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc06fd80 con 0x7f72dc13cfe0 2024-05-22T21:20:32.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.246+0000 7f72e0c00640 1 --2- 172.21.15.161:0/47200637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72dc13cfe0 0x7f72dc15dac0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f72dc06de40 tx=0x7f72cc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72688992109f37cd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:32.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.246+0000 7f72db400640 1 -- 172.21.15.161:0/47200637 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0ab650 con 0x7f72dc13cfe0 2024-05-22T21:20:32.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.246+0000 7f72db400640 1 -- 172.21.15.161:0/47200637 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc0ab7b0 con 0x7f72dc13cfe0 2024-05-22T21:20:32.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.246+0000 7f72db400640 1 -- 172.21.15.161:0/47200637 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0abb40 con 0x7f72dc13cfe0 2024-05-22T21:20:32.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.246+0000 7f72e3600640 1 -- 172.21.15.161:0/47200637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72dc13cfe0 msgr2=0x7f72dc15dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:32.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.246+0000 7f72e3600640 1 --2- 172.21.15.161:0/47200637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72dc13cfe0 0x7f72dc15dac0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f72dc06de40 tx=0x7f72cc01c450 comp rx=0 tx=0).stop 2024-05-22T21:20:32.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.246+0000 7f72e3600640 1 -- 172.21.15.161:0/47200637 shutdown_connections 2024-05-22T21:20:32.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.246+0000 7f72e3600640 1 --2- 172.21.15.161:0/47200637 >> v2:172.21.15.179:3300/0 conn(0x7f72dc13d560 0x7f72dc15e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.246+0000 7f72e3600640 1 --2- 172.21.15.161:0/47200637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72dc13cfe0 0x7f72dc15dac0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.246+0000 7f72e3600640 1 -- 172.21.15.161:0/47200637 >> 172.21.15.161:0/47200637 conn(0x7f72dc06a0a0 msgr2=0x7f72dc06a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:32.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.246+0000 7f72e3600640 1 -- 172.21.15.161:0/47200637 shutdown_connections 2024-05-22T21:20:32.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.246+0000 7f72e3600640 1 -- 172.21.15.161:0/47200637 wait complete. 2024-05-22T21:20:32.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.246+0000 7f72e3600640 1 Processor -- start 2024-05-22T21:20:32.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.246+0000 7f72e3600640 1 -- start start 2024-05-22T21:20:32.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72e3600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72dc0cdb00 0x7f72dc141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:32.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72e3600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72dc13cfe0 0x7f72dc143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:32.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72e3600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72dc13d560 0x7f72dc1410f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:32.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72e3600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc13e930 con 0x7f72dc13cfe0 2024-05-22T21:20:32.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72e3600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc13dd40 con 0x7f72dc13d560 2024-05-22T21:20:32.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72e3600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc13df90 con 0x7f72dc0cdb00 2024-05-22T21:20:32.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72dbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72dc13cfe0 0x7f72dc143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:32.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72dbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72dc13cfe0 0x7f72dc143230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55106/0 (socket says 172.21.15.161:55106) 2024-05-22T21:20:32.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72dbe00640 1 -- 172.21.15.161:0/1610976091 learned_addr learned my addr 172.21.15.161:0/1610976091 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:32.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72e0c00640 1 --2- 172.21.15.161:0/1610976091 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72dc0cdb00 0x7f72dc141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:32.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72e1600640 1 --2- 172.21.15.161:0/1610976091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72dc13d560 0x7f72dc1410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:32.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72dbe00640 1 -- 172.21.15.161:0/1610976091 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72dc0cdb00 msgr2=0x7f72dc141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:32.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72dbe00640 1 --2- 172.21.15.161:0/1610976091 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72dc0cdb00 0x7f72dc141a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72dbe00640 1 -- 172.21.15.161:0/1610976091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72dc13d560 msgr2=0x7f72dc1410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:32.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72dbe00640 1 --2- 172.21.15.161:0/1610976091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72dc13d560 0x7f72dc1410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72dbe00640 1 -- 172.21.15.161:0/1610976091 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc0094d0 con 0x7f72dc13cfe0 2024-05-22T21:20:32.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72e1600640 1 --2- 172.21.15.161:0/1610976091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72dc13d560 0x7f72dc1410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:32.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72dbe00640 1 --2- 172.21.15.161:0/1610976091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72dc13cfe0 0x7f72dc143230 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f72c800d7d0 tx=0x7f72c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:32.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72d8c00640 1 -- 172.21.15.161:0/1610976091 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c800b6c0 con 0x7f72dc13cfe0 2024-05-22T21:20:32.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72d8c00640 1 -- 172.21.15.161:0/1610976091 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c800b820 con 0x7f72dc13cfe0 2024-05-22T21:20:32.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72e3600640 1 -- 172.21.15.161:0/1610976091 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc13e270 con 0x7f72dc13cfe0 2024-05-22T21:20:32.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72d8c00640 1 -- 172.21.15.161:0/1610976091 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c800bbb0 con 0x7f72dc13cfe0 2024-05-22T21:20:32.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72e3600640 1 -- 172.21.15.161:0/1610976091 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc143770 con 0x7f72dc13cfe0 2024-05-22T21:20:32.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72e3600640 1 -- 172.21.15.161:0/1610976091 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc067220 con 0x7f72dc13cfe0 2024-05-22T21:20:32.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.250+0000 7f72d8c00640 1 -- 172.21.15.161:0/1610976091 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f72c800bd10 con 0x7f72dc13cfe0 2024-05-22T21:20:32.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.254+0000 7f72d8c00640 1 --2- 172.21.15.161:0/1610976091 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72bc066690 0x7f72bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:32.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.254+0000 7f72d8c00640 1 -- 172.21.15.161:0/1610976091 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f72c80841f0 con 0x7f72dc13cfe0 2024-05-22T21:20:32.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.254+0000 7f72e0c00640 1 --2- 172.21.15.161:0/1610976091 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72bc066690 0x7f72bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:32.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.254+0000 7f72e0c00640 1 --2- 172.21.15.161:0/1610976091 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72bc066690 0x7f72bc068b50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0af000 tx=0x7f72cc0b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:32.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.258+0000 7f72d8c00640 1 -- 172.21.15.161:0/1610976091 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f72c8051940 con 0x7f72dc13cfe0 2024-05-22T21:20:32.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.410+0000 7f72e3600640 1 -- 172.21.15.161:0/1610976091 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f72dc071d50 con 0x7f72dc13cfe0 2024-05-22T21:20:32.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.562+0000 7f72d8c00640 1 -- 172.21.15.161:0/1610976091 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v155) v1 ==== 138+0+0 (secure 0 0 0) 0x7f72c8054f60 con 0x7f72dc13cfe0 2024-05-22T21:20:32.566 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' created 2024-05-22T21:20:32.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.566+0000 7f72e3600640 1 -- 172.21.15.161:0/1610976091 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72bc066690 msgr2=0x7f72bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:32.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.566+0000 7f72e3600640 1 --2- 172.21.15.161:0/1610976091 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72bc066690 0x7f72bc068b50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0af000 tx=0x7f72cc0b6040 comp rx=0 tx=0).stop 2024-05-22T21:20:32.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.566+0000 7f72e3600640 1 -- 172.21.15.161:0/1610976091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72dc13cfe0 msgr2=0x7f72dc143230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:32.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.566+0000 7f72e3600640 1 --2- 172.21.15.161:0/1610976091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72dc13cfe0 0x7f72dc143230 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f72c800d7d0 tx=0x7f72c800dc90 comp rx=0 tx=0).stop 2024-05-22T21:20:32.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.566+0000 7f72e3600640 1 -- 172.21.15.161:0/1610976091 shutdown_connections 2024-05-22T21:20:32.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.566+0000 7f72e3600640 1 --2- 172.21.15.161:0/1610976091 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72bc066690 0x7f72bc068b50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.566+0000 7f72e3600640 1 --2- 172.21.15.161:0/1610976091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72dc13d560 0x7f72dc1410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.566+0000 7f72e3600640 1 --2- 172.21.15.161:0/1610976091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72dc13cfe0 0x7f72dc143230 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.566+0000 7f72e3600640 1 --2- 172.21.15.161:0/1610976091 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72dc0cdb00 0x7f72dc141a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.566+0000 7f72e3600640 1 -- 172.21.15.161:0/1610976091 >> 172.21.15.161:0/1610976091 conn(0x7f72dc06a0a0 msgr2=0x7f72dc06d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:32.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.566+0000 7f72e3600640 1 -- 172.21.15.161:0/1610976091 shutdown_connections 2024-05-22T21:20:32.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.566+0000 7f72e3600640 1 -- 172.21.15.161:0/1610976091 wait complete. 2024-05-22T21:20:32.590 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-22T21:20:32.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.730+0000 7f5ff4a00640 1 Processor -- start 2024-05-22T21:20:32.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.730+0000 7f5ff4a00640 1 -- start start 2024-05-22T21:20:32.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.734+0000 7f5ff4a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5ff015b600 0x7f5ff006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:32.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.734+0000 7f5ff4a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff0144bb0 con 0x7f5ff0143c40 2024-05-22T21:20:32.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.734+0000 7f5ff4a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff015efb0 con 0x7f5ff006c7e0 2024-05-22T21:20:32.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.734+0000 7f5ff4a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff015ba60 con 0x7f5ff015b600 2024-05-22T21:20:32.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.734+0000 7f5fede00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f5ff0143c40 0x7f5ff01444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54256/0 (socket says 172.21.15.161:54256) 2024-05-22T21:20:32.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.734+0000 7f5fede00640 1 -- 172.21.15.161:0/1812464094 learned_addr learned my addr 172.21.15.161:0/1812464094 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:32.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.734+0000 7f5fed400640 1 --2- 172.21.15.161:0/1812464094 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5ff015b600 0x7f5ff006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:32.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.734+0000 7f5feca00640 1 -- 172.21.15.161:0/1812464094 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150714198 0 0) 0x7f5ff0144bb0 con 0x7f5ff0143c40 2024-05-22T21:20:32.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.734+0000 7f5feca00640 1 -- 172.21.15.161:0/1812464094 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fd4003530 con 0x7f5ff0143c40 2024-05-22T21:20:32.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.734+0000 7f5feca00640 1 -- 172.21.15.161:0/1812464094 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600238247 0 0) 0x7f5ff015efb0 con 0x7f5ff006c7e0 2024-05-22T21:20:32.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.734+0000 7f5feca00640 1 -- 172.21.15.161:0/1812464094 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ff0144bb0 con 0x7f5ff006c7e0 2024-05-22T21:20:32.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.734+0000 7f5feca00640 1 -- 172.21.15.161:0/1812464094 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1907978099 0 0) 0x7f5fd4003530 con 0x7f5ff0143c40 2024-05-22T21:20:32.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.734+0000 7f5fed400640 1 -- 172.21.15.161:0/1812464094 >> v1:172.21.15.179:6789/0 conn(0x7f5ff006c7e0 legacy=0x7f5ff00b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:32.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.734+0000 7f5fed400640 1 -- 172.21.15.161:0/1812464094 >> v1:172.21.15.161:6789/0 conn(0x7f5ff0143c40 legacy=0x7f5ff01444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:32.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.734+0000 7f5fed400640 1 -- 172.21.15.161:0/1812464094 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff015f290 con 0x7f5ff015b600 2024-05-22T21:20:32.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5fed400640 1 --2- 172.21.15.161:0/1812464094 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5ff015b600 0x7f5ff006c2a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0097f0 tx=0x7f5fdc01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f50d133986a1c89e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:32.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5feca00640 1 -- 172.21.15.161:0/1812464094 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0001a00 con 0x7f5ff015b600 2024-05-22T21:20:32.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5feca00640 1 -- 172.21.15.161:0/1812464094 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc0a7050 con 0x7f5ff015b600 2024-05-22T21:20:32.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5feca00640 1 -- 172.21.15.161:0/1812464094 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc0ab620 con 0x7f5ff015b600 2024-05-22T21:20:32.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5ff4a00640 1 -- 172.21.15.161:0/1812464094 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5ff015b600 msgr2=0x7f5ff006c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:32.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5ff4a00640 1 --2- 172.21.15.161:0/1812464094 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5ff015b600 0x7f5ff006c2a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0097f0 tx=0x7f5fdc01c400 comp rx=0 tx=0).stop 2024-05-22T21:20:32.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5ff4a00640 1 -- 172.21.15.161:0/1812464094 shutdown_connections 2024-05-22T21:20:32.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5ff4a00640 1 --2- 172.21.15.161:0/1812464094 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5ff015b600 0x7f5ff006c2a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5ff4a00640 1 -- 172.21.15.161:0/1812464094 >> 172.21.15.161:0/1812464094 conn(0x7f5ff006a2d0 msgr2=0x7f5ff006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:32.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5ff4a00640 1 -- 172.21.15.161:0/1812464094 shutdown_connections 2024-05-22T21:20:32.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5ff4a00640 1 -- 172.21.15.161:0/1812464094 wait complete. 2024-05-22T21:20:32.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5ff4a00640 1 Processor -- start 2024-05-22T21:20:32.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5ff4a00640 1 -- start start 2024-05-22T21:20:32.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5ff4a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ff006c7e0 0x7f5ff013dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:32.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5ff4a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ff0143c40 0x7f5ff0140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:32.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5ff4a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5ff015b600 0x7f5ff015d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:32.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5ff4a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff013e200 con 0x7f5ff0143c40 2024-05-22T21:20:32.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5ff4a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff0140c60 con 0x7f5ff006c7e0 2024-05-22T21:20:32.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5ff4a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff015de50 con 0x7f5ff015b600 2024-05-22T21:20:32.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5fed400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ff0143c40 0x7f5ff0140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:32.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5fed400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ff0143c40 0x7f5ff0140720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55118/0 (socket says 172.21.15.161:55118) 2024-05-22T21:20:32.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.738+0000 7f5fed400640 1 -- 172.21.15.161:0/386905821 learned_addr learned my addr 172.21.15.161:0/386905821 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:32.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.742+0000 7f5fede00640 1 --2- 172.21.15.161:0/386905821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ff006c7e0 0x7f5ff013dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:32.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.742+0000 7f5fee800640 1 --2- 172.21.15.161:0/386905821 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5ff015b600 0x7f5ff015d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:32.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.742+0000 7f5fed400640 1 -- 172.21.15.161:0/386905821 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5ff015b600 msgr2=0x7f5ff015d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:32.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.742+0000 7f5fed400640 1 --2- 172.21.15.161:0/386905821 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5ff015b600 0x7f5ff015d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.742+0000 7f5fed400640 1 -- 172.21.15.161:0/386905821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ff006c7e0 msgr2=0x7f5ff013dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:32.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.742+0000 7f5fed400640 1 --2- 172.21.15.161:0/386905821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ff006c7e0 0x7f5ff013dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:32.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.742+0000 7f5fed400640 1 -- 172.21.15.161:0/386905821 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc009420 con 0x7f5ff0143c40 2024-05-22T21:20:32.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.742+0000 7f5fede00640 1 --2- 172.21.15.161:0/386905821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ff006c7e0 0x7f5ff013dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:32.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.742+0000 7f5fed400640 1 --2- 172.21.15.161:0/386905821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ff0143c40 0x7f5ff0140720 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0ae040 tx=0x7f5fdc01ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:32.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.742+0000 7f5fe6000640 1 -- 172.21.15.161:0/386905821 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc0abcd0 con 0x7f5ff0143c40 2024-05-22T21:20:32.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.742+0000 7f5fe6000640 1 -- 172.21.15.161:0/386905821 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc0abe50 con 0x7f5ff0143c40 2024-05-22T21:20:32.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.742+0000 7f5ff4a00640 1 -- 172.21.15.161:0/386905821 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff015cae0 con 0x7f5ff0143c40 2024-05-22T21:20:32.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.742+0000 7f5fe6000640 1 -- 172.21.15.161:0/386905821 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc004260 con 0x7f5ff0143c40 2024-05-22T21:20:32.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.742+0000 7f5ff4a00640 1 -- 172.21.15.161:0/386905821 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff0154720 con 0x7f5ff0143c40 2024-05-22T21:20:32.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.742+0000 7f5ff4a00640 1 -- 172.21.15.161:0/386905821 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff01449e0 con 0x7f5ff0143c40 2024-05-22T21:20:32.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.742+0000 7f5fe6000640 1 -- 172.21.15.161:0/386905821 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5fdc0043c0 con 0x7f5ff0143c40 2024-05-22T21:20:32.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.746+0000 7f5fe6000640 1 --2- 172.21.15.161:0/386905821 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5fd0066690 0x7f5fd0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:32.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.746+0000 7f5fe6000640 1 -- 172.21.15.161:0/386905821 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f5fdc11dfe0 con 0x7f5ff0143c40 2024-05-22T21:20:32.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.746+0000 7f5fede00640 1 --2- 172.21.15.161:0/386905821 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5fd0066690 0x7f5fd0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:32.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.746+0000 7f5fede00640 1 --2- 172.21.15.161:0/386905821 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5fd0066690 0x7f5fd0068b50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00012f0 tx=0x7f5fe0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:32.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.750+0000 7f5fe6000640 1 -- 172.21.15.161:0/386905821 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5fdc0eb540 con 0x7f5ff0143c40 2024-05-22T21:20:32.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:32.902+0000 7f5ff4a00640 1 -- 172.21.15.161:0/386905821 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5ff0067220 con 0x7f5ff0143c40 2024-05-22T21:20:33.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.574+0000 7f5fe6000640 1 -- 172.21.15.161:0/386905821 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v156) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5fdc0eeb60 con 0x7f5ff0143c40 2024-05-22T21:20:33.581 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' created 2024-05-22T21:20:33.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.582+0000 7f5ff4a00640 1 -- 172.21.15.161:0/386905821 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5fd0066690 msgr2=0x7f5fd0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:33.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.582+0000 7f5ff4a00640 1 --2- 172.21.15.161:0/386905821 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5fd0066690 0x7f5fd0068b50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00012f0 tx=0x7f5fe0009040 comp rx=0 tx=0).stop 2024-05-22T21:20:33.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.582+0000 7f5ff4a00640 1 -- 172.21.15.161:0/386905821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ff0143c40 msgr2=0x7f5ff0140720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:33.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.582+0000 7f5ff4a00640 1 --2- 172.21.15.161:0/386905821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ff0143c40 0x7f5ff0140720 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0ae040 tx=0x7f5fdc01ac40 comp rx=0 tx=0).stop 2024-05-22T21:20:33.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.582+0000 7f5ff4a00640 1 -- 172.21.15.161:0/386905821 shutdown_connections 2024-05-22T21:20:33.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.582+0000 7f5ff4a00640 1 --2- 172.21.15.161:0/386905821 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5fd0066690 0x7f5fd0068b50 secure :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00012f0 tx=0x7f5fe0009040 comp rx=0 tx=0).stop 2024-05-22T21:20:33.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.582+0000 7f5ff4a00640 1 --2- 172.21.15.161:0/386905821 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5ff015b600 0x7f5ff015d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:33.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.582+0000 7f5ff4a00640 1 --2- 172.21.15.161:0/386905821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ff0143c40 0x7f5ff0140720 secure :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0ae040 tx=0x7f5fdc01ac40 comp rx=0 tx=0).stop 2024-05-22T21:20:33.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.582+0000 7f5ff4a00640 1 --2- 172.21.15.161:0/386905821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ff006c7e0 0x7f5ff013dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:33.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.582+0000 7f5ff4a00640 1 -- 172.21.15.161:0/386905821 >> 172.21.15.161:0/386905821 conn(0x7f5ff006a2d0 msgr2=0x7f5ff0153310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:33.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.582+0000 7f5ff4a00640 1 -- 172.21.15.161:0/386905821 shutdown_connections 2024-05-22T21:20:33.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.582+0000 7f5ff4a00640 1 -- 172.21.15.161:0/386905821 wait complete. 2024-05-22T21:20:33.612 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-22T21:20:33.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf19800640 1 Processor -- start 2024-05-22T21:20:33.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf19800640 1 -- start start 2024-05-22T21:20:33.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf19800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf1415b600 0x7fcf1406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:33.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf19800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcf1406c7e0 0x7fcf140b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:33.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf19800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf14144bb0 con 0x7fcf14143c40 2024-05-22T21:20:33.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf19800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf1415ba60 con 0x7fcf1406c7e0 2024-05-22T21:20:33.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf19800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1406cc70 con 0x7fcf1415b600 2024-05-22T21:20:33.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf12a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fcf14143c40 0x7fcf141444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54266/0 (socket says 172.21.15.161:54266) 2024-05-22T21:20:33.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf12a00640 1 -- 172.21.15.161:0/2521078152 learned_addr learned my addr 172.21.15.161:0/2521078152 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:33.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf11600640 1 -- 172.21.15.161:0/2521078152 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311562792 0 0) 0x7fcf14144bb0 con 0x7fcf14143c40 2024-05-22T21:20:33.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf11600640 1 -- 172.21.15.161:0/2521078152 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcef8003530 con 0x7fcf14143c40 2024-05-22T21:20:33.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf12000640 1 --2- 172.21.15.161:0/2521078152 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf1415b600 0x7fcf1406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:33.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf13400640 1 --2- 172.21.15.161:0/2521078152 >> v2:172.21.15.179:3300/0 conn(0x7fcf1406c7e0 0x7fcf140b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:33.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf11600640 1 -- 172.21.15.161:0/2521078152 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2007127173 0 0) 0x7fcef8003530 con 0x7fcf14143c40 2024-05-22T21:20:33.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf11600640 1 -- 172.21.15.161:0/2521078152 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf1415b600 msgr2=0x7fcf1406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:33.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf11600640 1 --2- 172.21.15.161:0/2521078152 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf1415b600 0x7fcf1406c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:33.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf11600640 1 -- 172.21.15.161:0/2521078152 >> v2:172.21.15.179:3300/0 conn(0x7fcf1406c7e0 msgr2=0x7fcf140b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:33.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf11600640 1 --2- 172.21.15.161:0/2521078152 >> v2:172.21.15.179:3300/0 conn(0x7fcf1406c7e0 0x7fcf140b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:33.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf11600640 1 -- 172.21.15.161:0/2521078152 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1415efb0 con 0x7fcf14143c40 2024-05-22T21:20:33.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf11600640 1 -- 172.21.15.161:0/2521078152 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fcf04001a00 con 0x7fcf14143c40 2024-05-22T21:20:33.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf11600640 1 -- 172.21.15.161:0/2521078152 >> v1:172.21.15.161:6789/0 conn(0x7fcf14143c40 legacy=0x7fcf141444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:33.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.750+0000 7fcf11600640 1 --2- 172.21.15.161:0/2521078152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef8003980 0x7fcef8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:33.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf11600640 1 --2- 172.21.15.161:0/2521078152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcef80047d0 0x7fcef8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:33.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf11600640 1 --2- 172.21.15.161:0/2521078152 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf1415b600 0x7fcef8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:33.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf11600640 1 -- 172.21.15.161:0/2521078152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1415ba60 con 0x7fcef8003980 2024-05-22T21:20:33.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf11600640 1 -- 172.21.15.161:0/2521078152 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1406cc70 con 0x7fcef80047d0 2024-05-22T21:20:33.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf11600640 1 -- 172.21.15.161:0/2521078152 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef800ba40 con 0x7fcf1415b600 2024-05-22T21:20:33.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf12a00640 1 --2- 172.21.15.161:0/2521078152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef8003980 0x7fcef8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:33.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf12000640 1 --2- 172.21.15.161:0/2521078152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcef80047d0 0x7fcef8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:33.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf13400640 1 --2- 172.21.15.161:0/2521078152 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf1415b600 0x7fcef8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:33.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf12a00640 1 -- 172.21.15.161:0/2521078152 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf1415b600 msgr2=0x7fcef8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:33.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf12a00640 1 --2- 172.21.15.161:0/2521078152 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf1415b600 0x7fcef8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:33.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf12a00640 1 -- 172.21.15.161:0/2521078152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcef80047d0 msgr2=0x7fcef8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:33.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf12a00640 1 --2- 172.21.15.161:0/2521078152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcef80047d0 0x7fcef8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:33.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf12a00640 1 -- 172.21.15.161:0/2521078152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef8003530 con 0x7fcef8003980 2024-05-22T21:20:33.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf12a00640 1 --2- 172.21.15.161:0/2521078152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef8003980 0x7fcef8003da0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400a830 tx=0x7fcf0401d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1a87efb13d9a88 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:33.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf11600640 1 -- 172.21.15.161:0/2521078152 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf040ad630 con 0x7fcef8003980 2024-05-22T21:20:33.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf11600640 1 -- 172.21.15.161:0/2521078152 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf040a8070 con 0x7fcef8003980 2024-05-22T21:20:33.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.754+0000 7fcf11600640 1 -- 172.21.15.161:0/2521078152 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf040ad9f0 con 0x7fcef8003980 2024-05-22T21:20:33.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf19800640 1 -- 172.21.15.161:0/2521078152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef8003980 msgr2=0x7fcef8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:33.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf19800640 1 --2- 172.21.15.161:0/2521078152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcef8003980 0x7fcef8003da0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400a830 tx=0x7fcf0401d340 comp rx=0 tx=0).stop 2024-05-22T21:20:33.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf13400640 1 -- 172.21.15.161:0/2521078152 reap_dead start 2024-05-22T21:20:33.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf19800640 1 -- 172.21.15.161:0/2521078152 shutdown_connections 2024-05-22T21:20:33.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf19800640 1 -- 172.21.15.161:0/2521078152 >> 172.21.15.161:0/2521078152 conn(0x7fcf1406a2d0 msgr2=0x7fcf1406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:33.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf19800640 1 -- 172.21.15.161:0/2521078152 shutdown_connections 2024-05-22T21:20:33.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf19800640 1 -- 172.21.15.161:0/2521078152 wait complete. 2024-05-22T21:20:33.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf19800640 1 Processor -- start 2024-05-22T21:20:33.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf19800640 1 -- start start 2024-05-22T21:20:33.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf19800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf141546c0 0x7fcf14142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:33.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf19800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1413f0f0 0x7fcf1413f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:33.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf19800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf141f9c60 0x7fcf141fa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:33.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf19800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1413fab0 con 0x7fcf1413f0f0 2024-05-22T21:20:33.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf19800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf141fa620 con 0x7fcf141f9c60 2024-05-22T21:20:33.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf19800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf14138bf0 con 0x7fcf141546c0 2024-05-22T21:20:33.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf12000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1413f0f0 0x7fcf1413f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:33.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf12000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1413f0f0 0x7fcf1413f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55142/0 (socket says 172.21.15.161:55142) 2024-05-22T21:20:33.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf12000640 1 -- 172.21.15.161:0/115970533 learned_addr learned my addr 172.21.15.161:0/115970533 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:33.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf12a00640 1 --2- 172.21.15.161:0/115970533 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf141546c0 0x7fcf14142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:33.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf13400640 1 --2- 172.21.15.161:0/115970533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf141f9c60 0x7fcf141fa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:33.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf12000640 1 -- 172.21.15.161:0/115970533 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf141546c0 msgr2=0x7fcf14142fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:33.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf12000640 1 --2- 172.21.15.161:0/115970533 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf141546c0 0x7fcf14142fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:33.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf12000640 1 -- 172.21.15.161:0/115970533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf141f9c60 msgr2=0x7fcf141fa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:33.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf12000640 1 --2- 172.21.15.161:0/115970533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf141f9c60 0x7fcf141fa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:33.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.758+0000 7fcf12000640 1 -- 172.21.15.161:0/115970533 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf14142b20 con 0x7fcf1413f0f0 2024-05-22T21:20:33.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.762+0000 7fcf12000640 1 --2- 172.21.15.161:0/115970533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1413f0f0 0x7fcf1413f570 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000b850 tx=0x7fcf0000bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:33.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.762+0000 7fcf0aa00640 1 -- 172.21.15.161:0/115970533 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf000040f0 con 0x7fcf1413f0f0 2024-05-22T21:20:33.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.762+0000 7fcf0aa00640 1 -- 172.21.15.161:0/115970533 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf000107d0 con 0x7fcf1413f0f0 2024-05-22T21:20:33.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.762+0000 7fcf0aa00640 1 -- 172.21.15.161:0/115970533 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00010b60 con 0x7fcf1413f0f0 2024-05-22T21:20:33.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.762+0000 7fcf19800640 1 -- 172.21.15.161:0/115970533 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf141fab20 con 0x7fcf1413f0f0 2024-05-22T21:20:33.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.762+0000 7fcf19800640 1 -- 172.21.15.161:0/115970533 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf141424d0 con 0x7fcf1413f0f0 2024-05-22T21:20:33.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.762+0000 7fcf19800640 1 -- 172.21.15.161:0/115970533 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf14142330 con 0x7fcf1413f0f0 2024-05-22T21:20:33.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.762+0000 7fcf0aa00640 1 -- 172.21.15.161:0/115970533 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcf00027480 con 0x7fcf1413f0f0 2024-05-22T21:20:33.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.762+0000 7fcf0aa00640 1 --2- 172.21.15.161:0/115970533 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcef4066690 0x7fcef4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:33.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.766+0000 7fcf0aa00640 1 -- 172.21.15.161:0/115970533 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fcf00084b10 con 0x7fcf1413f0f0 2024-05-22T21:20:33.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.766+0000 7fcf12a00640 1 --2- 172.21.15.161:0/115970533 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcef4066690 0x7fcef4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:33.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.766+0000 7fcf12a00640 1 --2- 172.21.15.161:0/115970533 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcef4066690 0x7fcef4068b50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcf040ad000 tx=0x7fcf04007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:33.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.766+0000 7fcf0aa00640 1 -- 172.21.15.161:0/115970533 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcf00051f70 con 0x7fcf1413f0f0 2024-05-22T21:20:33.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.958+0000 7fcf19800640 1 -- 172.21.15.161:0/115970533 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcf14142330 con 0x7fcf1413f0f0 2024-05-22T21:20:33.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.966+0000 7fcf0aa00640 1 -- 172.21.15.161:0/115970533 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v66) v1 ==== 174+0+0 (secure 0 0 0) 0x7fcf14142330 con 0x7fcf1413f0f0 2024-05-22T21:20:33.974 INFO:teuthology.orchestra.run.smithi161.stderr:new fs with metadata pool 18 and data pool 19 2024-05-22T21:20:33.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.974+0000 7fcf19800640 1 -- 172.21.15.161:0/115970533 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcef4066690 msgr2=0x7fcef4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:33.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.974+0000 7fcf19800640 1 --2- 172.21.15.161:0/115970533 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcef4066690 0x7fcef4068b50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcf040ad000 tx=0x7fcf04007040 comp rx=0 tx=0).stop 2024-05-22T21:20:33.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.974+0000 7fcf19800640 1 -- 172.21.15.161:0/115970533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1413f0f0 msgr2=0x7fcf1413f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:33.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.974+0000 7fcf19800640 1 --2- 172.21.15.161:0/115970533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1413f0f0 0x7fcf1413f570 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000b850 tx=0x7fcf0000bd10 comp rx=0 tx=0).stop 2024-05-22T21:20:33.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.974+0000 7fcf19800640 1 -- 172.21.15.161:0/115970533 shutdown_connections 2024-05-22T21:20:33.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.974+0000 7fcf19800640 1 --2- 172.21.15.161:0/115970533 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcef4066690 0x7fcef4068b50 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:33.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.974+0000 7fcf19800640 1 --2- 172.21.15.161:0/115970533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf141f9c60 0x7fcf141fa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:33.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.974+0000 7fcf19800640 1 --2- 172.21.15.161:0/115970533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1413f0f0 0x7fcf1413f570 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:33.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.974+0000 7fcf19800640 1 --2- 172.21.15.161:0/115970533 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcf141546c0 0x7fcf14142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:33.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.974+0000 7fcf19800640 1 -- 172.21.15.161:0/115970533 >> 172.21.15.161:0/115970533 conn(0x7fcf1406a2d0 msgr2=0x7fcf140aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:33.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.978+0000 7fcf19800640 1 -- 172.21.15.161:0/115970533 shutdown_connections 2024-05-22T21:20:33.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:33.978+0000 7fcf19800640 1 -- 172.21.15.161:0/115970533 wait complete. 2024-05-22T21:20:34.007 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:20:34.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.138+0000 7f2a52e00640 1 Processor -- start 2024-05-22T21:20:34.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.142+0000 7f2a52e00640 1 -- start start 2024-05-22T21:20:34.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.142+0000 7f2a52e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a4c0d0a40 0x7f2a4c0d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.142+0000 7f2a52e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2a4c0cc320 0x7f2a4c0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.142+0000 7f2a52e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4c0cb8a0 con 0x7f2a4c15af00 2024-05-22T21:20:34.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.142+0000 7f2a52e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a4c0d13f0 con 0x7f2a4c0cc320 2024-05-22T21:20:34.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.142+0000 7f2a52e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c15f530 con 0x7f2a4c0d0a40 2024-05-22T21:20:34.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.142+0000 7f2a52400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2a4c0cc320 0x7f2a4c0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.142+0000 7f2a52400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2a4c0cc320 0x7f2a4c0cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55148/0 (socket says 172.21.15.161:55148) 2024-05-22T21:20:34.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.142+0000 7f2a52400640 1 -- 172.21.15.161:0/54873047 learned_addr learned my addr 172.21.15.161:0/54873047 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:34.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.142+0000 7f2a51000640 1 --2- 172.21.15.161:0/54873047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a4c0d0a40 0x7f2a4c0d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.142+0000 7f2a52400640 1 -- 172.21.15.161:0/54873047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a4c0d0a40 msgr2=0x7f2a4c0d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:34.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.142+0000 7f2a52400640 1 --2- 172.21.15.161:0/54873047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a4c0d0a40 0x7f2a4c0d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.142+0000 7f2a52400640 1 -- 172.21.15.161:0/54873047 >> v1:172.21.15.179:6789/0 conn(0x7f2a4c15af00 legacy=0x7f2a4c15eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:34.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.142+0000 7f2a52400640 1 -- 172.21.15.161:0/54873047 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c15f810 con 0x7f2a4c0cc320 2024-05-22T21:20:34.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.142+0000 7f2a51000640 1 --2- 172.21.15.161:0/54873047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a4c0d0a40 0x7f2a4c0d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:34.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.146+0000 7f2a52400640 1 --2- 172.21.15.161:0/54873047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c0cc320 0x7f2a4c0cc770 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009e50 tx=0x7f2a4401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187df8170a5dd66c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:34.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.146+0000 7f2a4be00640 1 -- 172.21.15.161:0/54873047 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a440a5870 con 0x7f2a4c0cc320 2024-05-22T21:20:34.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.146+0000 7f2a4be00640 1 -- 172.21.15.161:0/54873047 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a440a59d0 con 0x7f2a4c0cc320 2024-05-22T21:20:34.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.146+0000 7f2a4be00640 1 -- 172.21.15.161:0/54873047 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a440a5d60 con 0x7f2a4c0cc320 2024-05-22T21:20:34.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.146+0000 7f2a52e00640 1 -- 172.21.15.161:0/54873047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c0cc320 msgr2=0x7f2a4c0cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:34.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.146+0000 7f2a52e00640 1 --2- 172.21.15.161:0/54873047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c0cc320 0x7f2a4c0cc770 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009e50 tx=0x7f2a4401c9f0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.146+0000 7f2a52e00640 1 -- 172.21.15.161:0/54873047 shutdown_connections 2024-05-22T21:20:34.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.146+0000 7f2a52e00640 1 --2- 172.21.15.161:0/54873047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c0cc320 0x7f2a4c0cc770 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.146+0000 7f2a52e00640 1 --2- 172.21.15.161:0/54873047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a4c0d0a40 0x7f2a4c0d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.146+0000 7f2a52e00640 1 -- 172.21.15.161:0/54873047 >> 172.21.15.161:0/54873047 conn(0x7f2a4c099a40 msgr2=0x7f2a4c099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:34.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.146+0000 7f2a52e00640 1 -- 172.21.15.161:0/54873047 shutdown_connections 2024-05-22T21:20:34.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.146+0000 7f2a52e00640 1 -- 172.21.15.161:0/54873047 wait complete. 2024-05-22T21:20:34.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.146+0000 7f2a52e00640 1 Processor -- start 2024-05-22T21:20:34.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.146+0000 7f2a52e00640 1 -- start start 2024-05-22T21:20:34.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a52e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c0cc320 0x7f2a4c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a52e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a4c0d0a40 0x7f2a4c159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a52e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a4c15af00 0x7f2a4c0c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a52e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c159fa0 con 0x7f2a4c0cc320 2024-05-22T21:20:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a52e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c15d8f0 con 0x7f2a4c15af00 2024-05-22T21:20:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a52e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c15db90 con 0x7f2a4c0d0a40 2024-05-22T21:20:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a51a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c0cc320 0x7f2a4c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a51a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c0cc320 0x7f2a4c15b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55154/0 (socket says 172.21.15.161:55154) 2024-05-22T21:20:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a51a00640 1 -- 172.21.15.161:0/3966504436 learned_addr learned my addr 172.21.15.161:0/3966504436 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a51000640 1 --2- 172.21.15.161:0/3966504436 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a4c0d0a40 0x7f2a4c159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a52400640 1 --2- 172.21.15.161:0/3966504436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a4c15af00 0x7f2a4c0c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a51a00640 1 -- 172.21.15.161:0/3966504436 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a4c0d0a40 msgr2=0x7f2a4c159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a51a00640 1 --2- 172.21.15.161:0/3966504436 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a4c0d0a40 0x7f2a4c159380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a51a00640 1 -- 172.21.15.161:0/3966504436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a4c15af00 msgr2=0x7f2a4c0c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a51a00640 1 --2- 172.21.15.161:0/3966504436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a4c15af00 0x7f2a4c0c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a51a00640 1 -- 172.21.15.161:0/3966504436 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a44009b30 con 0x7f2a4c0cc320 2024-05-22T21:20:34.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a51a00640 1 --2- 172.21.15.161:0/3966504436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c0cc320 0x7f2a4c15b600 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000b410 tx=0x7f2a4000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:34.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a49600640 1 -- 172.21.15.161:0/3966504436 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a40004020 con 0x7f2a4c0cc320 2024-05-22T21:20:34.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a49600640 1 -- 172.21.15.161:0/3966504436 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a40004180 con 0x7f2a4c0cc320 2024-05-22T21:20:34.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a49600640 1 -- 172.21.15.161:0/3966504436 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a40011630 con 0x7f2a4c0cc320 2024-05-22T21:20:34.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a52e00640 1 -- 172.21.15.161:0/3966504436 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4c15a1a0 con 0x7f2a4c0cc320 2024-05-22T21:20:34.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a52e00640 1 -- 172.21.15.161:0/3966504436 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4c1598c0 con 0x7f2a4c0cc320 2024-05-22T21:20:34.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a52e00640 1 -- 172.21.15.161:0/3966504436 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a4c067070 con 0x7f2a4c0cc320 2024-05-22T21:20:34.155 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.150+0000 7f2a49600640 1 -- 172.21.15.161:0/3966504436 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2a40011790 con 0x7f2a4c0cc320 2024-05-22T21:20:34.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.154+0000 7f2a49600640 1 --2- 172.21.15.161:0/3966504436 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a2c066690 0x7f2a2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.154+0000 7f2a49600640 1 -- 172.21.15.161:0/3966504436 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a40083e60 con 0x7f2a4c0cc320 2024-05-22T21:20:34.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.154+0000 7f2a51000640 1 --2- 172.21.15.161:0/3966504436 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a2c066690 0x7f2a2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.154+0000 7f2a51000640 1 --2- 172.21.15.161:0/3966504436 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a2c066690 0x7f2a2c068b50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c06c940 tx=0x7f2a38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:34.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.158+0000 7f2a49600640 1 -- 172.21.15.161:0/3966504436 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2a400511e0 con 0x7f2a4c0cc320 2024-05-22T21:20:34.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.310+0000 7f2a52e00640 1 -- 172.21.15.161:0/3966504436 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a4c0cb1c0 con 0x7f2a4c0cc320 2024-05-22T21:20:34.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.310+0000 7f2a49600640 1 -- 172.21.15.161:0/3966504436 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16173 (secure 0 0 0) 0x7f2a40054800 con 0x7f2a4c0cc320 2024-05-22T21:20:34.315 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:34.315 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":157,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:20:33.964105+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:20:32.457234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-22T21:20:32.951241+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:20:34.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.314+0000 7f2a52e00640 1 -- 172.21.15.161:0/3966504436 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a2c066690 msgr2=0x7f2a2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:34.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.314+0000 7f2a52e00640 1 --2- 172.21.15.161:0/3966504436 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a2c066690 0x7f2a2c068b50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c06c940 tx=0x7f2a38008040 comp rx=0 tx=0).stop 2024-05-22T21:20:34.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.314+0000 7f2a52e00640 1 -- 172.21.15.161:0/3966504436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c0cc320 msgr2=0x7f2a4c15b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:34.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.314+0000 7f2a52e00640 1 --2- 172.21.15.161:0/3966504436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c0cc320 0x7f2a4c15b600 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000b410 tx=0x7f2a4000b8d0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.314+0000 7f2a52e00640 1 -- 172.21.15.161:0/3966504436 shutdown_connections 2024-05-22T21:20:34.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.314+0000 7f2a52e00640 1 --2- 172.21.15.161:0/3966504436 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2a2c066690 0x7f2a2c068b50 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.314+0000 7f2a52e00640 1 --2- 172.21.15.161:0/3966504436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a4c15af00 0x7f2a4c0c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.314+0000 7f2a52e00640 1 --2- 172.21.15.161:0/3966504436 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2a4c0d0a40 0x7f2a4c159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.314+0000 7f2a52e00640 1 --2- 172.21.15.161:0/3966504436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a4c0cc320 0x7f2a4c15b600 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.314+0000 7f2a52e00640 1 -- 172.21.15.161:0/3966504436 >> 172.21.15.161:0/3966504436 conn(0x7f2a4c099a40 msgr2=0x7f2a4c06dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:34.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.314+0000 7f2a52e00640 1 -- 172.21.15.161:0/3966504436 shutdown_connections 2024-05-22T21:20:34.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.314+0000 7f2a52e00640 1 -- 172.21.15.161:0/3966504436 wait complete. 2024-05-22T21:20:34.336 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:20:34.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.466+0000 7f7813a00640 1 Processor -- start 2024-05-22T21:20:34.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.466+0000 7f7813a00640 1 -- start start 2024-05-22T21:20:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.470+0000 7f7813a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f780c15d430 0x7f780c0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.470+0000 7f7813a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f780c0accf0 0x7f780c160e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.470+0000 7f7813a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f780c071fd0 con 0x7f780c0ac8b0 2024-05-22T21:20:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.470+0000 7f7813a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f780c070e80 con 0x7f780c15d430 2024-05-22T21:20:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.470+0000 7f7813a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f780c06c510 con 0x7f780c0accf0 2024-05-22T21:20:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.470+0000 7f7811000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f780c15d430 0x7f780c0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.470+0000 7f7803e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f780c0ac8b0 0x7f780c06cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54274/0 (socket says 172.21.15.161:54274) 2024-05-22T21:20:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.470+0000 7f7803e00640 1 -- 172.21.15.161:0/2698347343 learned_addr learned my addr 172.21.15.161:0/2698347343 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.470+0000 7f7811a00640 1 --2- 172.21.15.161:0/2698347343 >> v2:172.21.15.179:3300/0 conn(0x7f780c0accf0 0x7f780c160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.470+0000 7f7811000640 1 -- 172.21.15.161:0/2698347343 >> v2:172.21.15.179:3300/0 conn(0x7f780c0accf0 msgr2=0x7f780c160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.470+0000 7f7811000640 1 --2- 172.21.15.161:0/2698347343 >> v2:172.21.15.179:3300/0 conn(0x7f780c0accf0 0x7f780c160e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.470+0000 7f7811000640 1 -- 172.21.15.161:0/2698347343 >> v1:172.21.15.161:6789/0 conn(0x7f780c0ac8b0 legacy=0x7f780c06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.470+0000 7f7811000640 1 -- 172.21.15.161:0/2698347343 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c06c7f0 con 0x7f780c15d430 2024-05-22T21:20:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.470+0000 7f7811a00640 1 --2- 172.21.15.161:0/2698347343 >> v2:172.21.15.179:3300/0 conn(0x7f780c0accf0 0x7f780c160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:34.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.470+0000 7f780be00640 1 -- 172.21.15.161:0/2698347343 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037605428 0 0) 0x7f780c071fd0 con 0x7f780c0ac8b0 2024-05-22T21:20:34.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f7811000640 1 --2- 172.21.15.161:0/2698347343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f780c15d430 0x7f780c0707c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f780c0aeac0 tx=0x7f77fc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3b155dbb03b52b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:34.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f780be00640 1 -- 172.21.15.161:0/2698347343 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0ab600 con 0x7f780c15d430 2024-05-22T21:20:34.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f780be00640 1 -- 172.21.15.161:0/2698347343 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc0a7050 con 0x7f780c15d430 2024-05-22T21:20:34.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f780be00640 1 -- 172.21.15.161:0/2698347343 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0ab980 con 0x7f780c15d430 2024-05-22T21:20:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f7813a00640 1 -- 172.21.15.161:0/2698347343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f780c15d430 msgr2=0x7f780c0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f7813a00640 1 --2- 172.21.15.161:0/2698347343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f780c15d430 0x7f780c0707c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f780c0aeac0 tx=0x7f77fc01c3c0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f7813a00640 1 -- 172.21.15.161:0/2698347343 shutdown_connections 2024-05-22T21:20:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f7813a00640 1 --2- 172.21.15.161:0/2698347343 >> v2:172.21.15.179:3300/0 conn(0x7f780c0accf0 0x7f780c160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f7813a00640 1 --2- 172.21.15.161:0/2698347343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f780c15d430 0x7f780c0707c0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f7813a00640 1 -- 172.21.15.161:0/2698347343 >> 172.21.15.161:0/2698347343 conn(0x7f780c0b0bc0 msgr2=0x7f780c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f7813a00640 1 -- 172.21.15.161:0/2698347343 shutdown_connections 2024-05-22T21:20:34.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f7813a00640 1 -- 172.21.15.161:0/2698347343 wait complete. 2024-05-22T21:20:34.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f7813a00640 1 Processor -- start 2024-05-22T21:20:34.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f7813a00640 1 -- start start 2024-05-22T21:20:34.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f7813a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f780c0ac8b0 0x7f780c14c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f7813a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f780c0accf0 0x7f780c15ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f7813a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f780c15d430 0x7f780c149130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.474+0000 7f7813a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c15f280 con 0x7f780c15d430 2024-05-22T21:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7813a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c149670 con 0x7f780c0accf0 2024-05-22T21:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7813a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780c14af60 con 0x7f780c0ac8b0 2024-05-22T21:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7811a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f780c15d430 0x7f780c149130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7811a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f780c15d430 0x7f780c149130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55170/0 (socket says 172.21.15.161:55170) 2024-05-22T21:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7811a00640 1 -- 172.21.15.161:0/1391864475 learned_addr learned my addr 172.21.15.161:0/1391864475 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7803e00640 1 --2- 172.21.15.161:0/1391864475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f780c0accf0 0x7f780c15ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7811000640 1 --2- 172.21.15.161:0/1391864475 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f780c0ac8b0 0x7f780c14c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7811a00640 1 -- 172.21.15.161:0/1391864475 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f780c0ac8b0 msgr2=0x7f780c14c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7811a00640 1 --2- 172.21.15.161:0/1391864475 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f780c0ac8b0 0x7f780c14c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7811a00640 1 -- 172.21.15.161:0/1391864475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f780c0accf0 msgr2=0x7f780c15ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:34.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7811a00640 1 --2- 172.21.15.161:0/1391864475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f780c0accf0 0x7f780c15ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7811a00640 1 -- 172.21.15.161:0/1391864475 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc009420 con 0x7f780c15d430 2024-05-22T21:20:34.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7811a00640 1 --2- 172.21.15.161:0/1391864475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f780c15d430 0x7f780c149130 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f780400cdb0 tx=0x7f780400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:34.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7809600640 1 -- 172.21.15.161:0/1391864475 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804003ba0 con 0x7f780c15d430 2024-05-22T21:20:34.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7809600640 1 -- 172.21.15.161:0/1391864475 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7804003d20 con 0x7f780c15d430 2024-05-22T21:20:34.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7809600640 1 -- 172.21.15.161:0/1391864475 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804004260 con 0x7f780c15d430 2024-05-22T21:20:34.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7813a00640 1 -- 172.21.15.161:0/1391864475 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780c14b240 con 0x7f780c15d430 2024-05-22T21:20:34.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7813a00640 1 -- 172.21.15.161:0/1391864475 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780c14b800 con 0x7f780c15d430 2024-05-22T21:20:34.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7813a00640 1 -- 172.21.15.161:0/1391864475 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780c067220 con 0x7f780c15d430 2024-05-22T21:20:34.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7809600640 1 -- 172.21.15.161:0/1391864475 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f780400b040 con 0x7f780c15d430 2024-05-22T21:20:34.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7809600640 1 --2- 172.21.15.161:0/1391864475 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f77e8066690 0x7f77e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.478+0000 7f7809600640 1 -- 172.21.15.161:0/1391864475 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78040848d0 con 0x7f780c15d430 2024-05-22T21:20:34.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.482+0000 7f7811000640 1 --2- 172.21.15.161:0/1391864475 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f77e8066690 0x7f77e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.482+0000 7f7811000640 1 --2- 172.21.15.161:0/1391864475 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f77e8066690 0x7f77e8068b50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0ad000 tx=0x7f77fc0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:34.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.486+0000 7f7809600640 1 -- 172.21.15.161:0/1391864475 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7804051ba0 con 0x7f780c15d430 2024-05-22T21:20:34.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.642+0000 7f7813a00640 1 -- 172.21.15.161:0/1391864475 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f780c0ae7a0 con 0x7f780c15d430 2024-05-22T21:20:34.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.642+0000 7f7809600640 1 -- 172.21.15.161:0/1391864475 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16173 (secure 0 0 0) 0x7f780401d020 con 0x7f780c15d430 2024-05-22T21:20:34.645 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:34.645 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":157,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:20:33.964105+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:20:32.457234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-22T21:20:32.951241+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:20:34.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.642+0000 7f7813a00640 1 -- 172.21.15.161:0/1391864475 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f77e8066690 msgr2=0x7f77e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:34.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.642+0000 7f7813a00640 1 --2- 172.21.15.161:0/1391864475 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f77e8066690 0x7f77e8068b50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0ad000 tx=0x7f77fc0a6000 comp rx=0 tx=0).stop 2024-05-22T21:20:34.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.642+0000 7f7813a00640 1 -- 172.21.15.161:0/1391864475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f780c15d430 msgr2=0x7f780c149130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:34.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.642+0000 7f7813a00640 1 --2- 172.21.15.161:0/1391864475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f780c15d430 0x7f780c149130 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f780400cdb0 tx=0x7f780400a650 comp rx=0 tx=0).stop 2024-05-22T21:20:34.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.646+0000 7f7813a00640 1 -- 172.21.15.161:0/1391864475 shutdown_connections 2024-05-22T21:20:34.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.646+0000 7f7813a00640 1 --2- 172.21.15.161:0/1391864475 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f77e8066690 0x7f77e8068b50 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.646+0000 7f7813a00640 1 --2- 172.21.15.161:0/1391864475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f780c15d430 0x7f780c149130 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.646+0000 7f7813a00640 1 --2- 172.21.15.161:0/1391864475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f780c0accf0 0x7f780c15ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.646+0000 7f7813a00640 1 --2- 172.21.15.161:0/1391864475 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f780c0ac8b0 0x7f780c14c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.646+0000 7f7813a00640 1 -- 172.21.15.161:0/1391864475 >> 172.21.15.161:0/1391864475 conn(0x7f780c0b0bc0 msgr2=0x7f780c155b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:34.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.646+0000 7f7813a00640 1 -- 172.21.15.161:0/1391864475 shutdown_connections 2024-05-22T21:20:34.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.646+0000 7f7813a00640 1 -- 172.21.15.161:0/1391864475 wait complete. 2024-05-22T21:20:34.667 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-22T21:20:34.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.802+0000 7fec41400640 1 Processor -- start 2024-05-22T21:20:34.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.802+0000 7fec41400640 1 -- start start 2024-05-22T21:20:34.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.806+0000 7fec41400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fec3c1402e0 0x7fec3c13f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.806+0000 7fec41400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3c13fc80 0x7fec3c06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.806+0000 7fec41400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec3c1400b0 con 0x7fec3c13cfc0 2024-05-22T21:20:34.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.806+0000 7fec41400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec3c13d3a0 con 0x7fec3c1402e0 2024-05-22T21:20:34.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.806+0000 7fec41400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0cd920 con 0x7fec3c13fc80 2024-05-22T21:20:34.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.806+0000 7fec3a800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fec3c13cfc0 0x7fec3c15af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54284/0 (socket says 172.21.15.161:54284) 2024-05-22T21:20:34.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.806+0000 7fec3a800640 1 -- 172.21.15.161:0/790923202 learned_addr learned my addr 172.21.15.161:0/790923202 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:34.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.806+0000 7fec31e00640 1 --2- 172.21.15.161:0/790923202 >> v2:172.21.15.161:3300/0 conn(0x7fec3c1402e0 0x7fec3c13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.806+0000 7fec39e00640 1 -- 172.21.15.161:0/790923202 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524753166 0 0) 0x7fec3c1400b0 con 0x7fec3c13cfc0 2024-05-22T21:20:34.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.806+0000 7fec39e00640 1 -- 172.21.15.161:0/790923202 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec20003530 con 0x7fec3c13cfc0 2024-05-22T21:20:34.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.806+0000 7fec3b200640 1 --2- 172.21.15.161:0/790923202 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3c13fc80 0x7fec3c06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.806+0000 7fec31e00640 1 -- 172.21.15.161:0/790923202 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3c13fc80 msgr2=0x7fec3c06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:34.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.806+0000 7fec31e00640 1 --2- 172.21.15.161:0/790923202 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3c13fc80 0x7fec3c06fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.806+0000 7fec31e00640 1 -- 172.21.15.161:0/790923202 >> v1:172.21.15.161:6789/0 conn(0x7fec3c13cfc0 legacy=0x7fec3c15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:34.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.806+0000 7fec31e00640 1 -- 172.21.15.161:0/790923202 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c0cdc00 con 0x7fec3c1402e0 2024-05-22T21:20:34.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.806+0000 7fec3b200640 1 --2- 172.21.15.161:0/790923202 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3c13fc80 0x7fec3c06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:34.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.810+0000 7fec31e00640 1 --2- 172.21.15.161:0/790923202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec3c1402e0 0x7fec3c13f740 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fec240097c0 tx=0x7fec2401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e8ae9a689d9856 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:34.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.810+0000 7fec39e00640 1 -- 172.21.15.161:0/790923202 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec240ab600 con 0x7fec3c1402e0 2024-05-22T21:20:34.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.810+0000 7fec39e00640 1 -- 172.21.15.161:0/790923202 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec240a7050 con 0x7fec3c1402e0 2024-05-22T21:20:34.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.810+0000 7fec39e00640 1 -- 172.21.15.161:0/790923202 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec240ab980 con 0x7fec3c1402e0 2024-05-22T21:20:34.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.810+0000 7fec41400640 1 -- 172.21.15.161:0/790923202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec3c1402e0 msgr2=0x7fec3c13f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:34.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.810+0000 7fec41400640 1 --2- 172.21.15.161:0/790923202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec3c1402e0 0x7fec3c13f740 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fec240097c0 tx=0x7fec2401c3e0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.810+0000 7fec41400640 1 -- 172.21.15.161:0/790923202 shutdown_connections 2024-05-22T21:20:34.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.810+0000 7fec41400640 1 --2- 172.21.15.161:0/790923202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec3c1402e0 0x7fec3c13f740 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.810+0000 7fec41400640 1 --2- 172.21.15.161:0/790923202 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3c13fc80 0x7fec3c06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.810+0000 7fec41400640 1 -- 172.21.15.161:0/790923202 >> 172.21.15.161:0/790923202 conn(0x7fec3c06a2d0 msgr2=0x7fec3c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:34.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.810+0000 7fec41400640 1 -- 172.21.15.161:0/790923202 shutdown_connections 2024-05-22T21:20:34.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.810+0000 7fec41400640 1 -- 172.21.15.161:0/790923202 wait complete. 2024-05-22T21:20:34.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.810+0000 7fec41400640 1 Processor -- start 2024-05-22T21:20:34.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.810+0000 7fec41400640 1 -- start start 2024-05-22T21:20:34.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.810+0000 7fec41400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3c13cfc0 0x7fec3c13bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.810+0000 7fec41400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec3c13fc80 0x7fec3c142ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec41400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3c1402e0 0x7fec3c13b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec41400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c13f8f0 con 0x7fec3c13fc80 2024-05-22T21:20:34.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec41400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c13bdf0 con 0x7fec3c13cfc0 2024-05-22T21:20:34.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec41400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c13f370 con 0x7fec3c1402e0 2024-05-22T21:20:34.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec31e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec3c13fc80 0x7fec3c142ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec31e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec3c13fc80 0x7fec3c142ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55176/0 (socket says 172.21.15.161:55176) 2024-05-22T21:20:34.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec31e00640 1 -- 172.21.15.161:0/3365276121 learned_addr learned my addr 172.21.15.161:0/3365276121 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:34.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec3a800640 1 --2- 172.21.15.161:0/3365276121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3c13cfc0 0x7fec3c13bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec3b200640 1 --2- 172.21.15.161:0/3365276121 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3c1402e0 0x7fec3c13b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec31e00640 1 -- 172.21.15.161:0/3365276121 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3c1402e0 msgr2=0x7fec3c13b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:34.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec31e00640 1 --2- 172.21.15.161:0/3365276121 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3c1402e0 0x7fec3c13b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec31e00640 1 -- 172.21.15.161:0/3365276121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3c13cfc0 msgr2=0x7fec3c13bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:34.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec31e00640 1 --2- 172.21.15.161:0/3365276121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3c13cfc0 0x7fec3c13bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:34.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec31e00640 1 -- 172.21.15.161:0/3365276121 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec24009420 con 0x7fec3c13fc80 2024-05-22T21:20:34.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec3a800640 1 --2- 172.21.15.161:0/3365276121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3c13cfc0 0x7fec3c13bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:34.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec31e00640 1 --2- 172.21.15.161:0/3365276121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec3c13fc80 0x7fec3c142ca0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fec240ae040 tx=0x7fec24019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:34.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec33400640 1 -- 172.21.15.161:0/3365276121 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec240b4430 con 0x7fec3c13fc80 2024-05-22T21:20:34.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec33400640 1 -- 172.21.15.161:0/3365276121 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec240b4590 con 0x7fec3c13fc80 2024-05-22T21:20:34.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec33400640 1 -- 172.21.15.161:0/3365276121 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec240b4940 con 0x7fec3c13fc80 2024-05-22T21:20:34.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec41400640 1 -- 172.21.15.161:0/3365276121 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3c142200 con 0x7fec3c13fc80 2024-05-22T21:20:34.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec41400640 1 -- 172.21.15.161:0/3365276121 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3c1424a0 con 0x7fec3c13fc80 2024-05-22T21:20:34.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec41400640 1 -- 172.21.15.161:0/3365276121 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3c067220 con 0x7fec3c13fc80 2024-05-22T21:20:34.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec33400640 1 -- 172.21.15.161:0/3365276121 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fec240b2040 con 0x7fec3c13fc80 2024-05-22T21:20:34.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.814+0000 7fec33400640 1 --2- 172.21.15.161:0/3365276121 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fec18066690 0x7fec18068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:34.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.818+0000 7fec33400640 1 -- 172.21.15.161:0/3365276121 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec2411e0b0 con 0x7fec3c13fc80 2024-05-22T21:20:34.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.818+0000 7fec3a800640 1 --2- 172.21.15.161:0/3365276121 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fec18066690 0x7fec18068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:34.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.818+0000 7fec3a800640 1 --2- 172.21.15.161:0/3365276121 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fec18066690 0x7fec18068b50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0011f0 tx=0x7fec2c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:34.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:34.818+0000 7fec33400640 1 -- 172.21.15.161:0/3365276121 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fec240eb490 con 0x7fec3c13fc80 2024-05-22T21:20:35.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:35.010+0000 7fec41400640 1 -- 172.21.15.161:0/3365276121 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fec3c071d00 con 0x7fec3c13fc80 2024-05-22T21:20:35.563 INFO:tasks.ceph.mds.b.smithi179.stderr:2024-05-22T21:20:35.559+0000 7f0e71400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T21:20:35.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:35.986+0000 7fec33400640 1 -- 172.21.15.161:0/3365276121 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v69) v1 ==== 118+0+0 (secure 0 0 0) 0x7fec240eeab0 con 0x7fec3c13fc80 2024-05-22T21:20:35.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:35.986+0000 7fec41400640 1 -- 172.21.15.161:0/3365276121 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fec18066690 msgr2=0x7fec18068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:35.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:35.986+0000 7fec41400640 1 --2- 172.21.15.161:0/3365276121 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fec18066690 0x7fec18068b50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0011f0 tx=0x7fec2c009040 comp rx=0 tx=0).stop 2024-05-22T21:20:35.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:35.986+0000 7fec41400640 1 -- 172.21.15.161:0/3365276121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec3c13fc80 msgr2=0x7fec3c142ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:35.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:35.986+0000 7fec41400640 1 --2- 172.21.15.161:0/3365276121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec3c13fc80 0x7fec3c142ca0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fec240ae040 tx=0x7fec24019180 comp rx=0 tx=0).stop 2024-05-22T21:20:35.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:35.986+0000 7fec41400640 1 -- 172.21.15.161:0/3365276121 shutdown_connections 2024-05-22T21:20:35.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:35.986+0000 7fec41400640 1 --2- 172.21.15.161:0/3365276121 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fec18066690 0x7fec18068b50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:35.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:35.986+0000 7fec41400640 1 --2- 172.21.15.161:0/3365276121 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec3c1402e0 0x7fec3c13b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:35.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:35.986+0000 7fec41400640 1 --2- 172.21.15.161:0/3365276121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec3c13fc80 0x7fec3c142ca0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:35.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:35.986+0000 7fec41400640 1 --2- 172.21.15.161:0/3365276121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec3c13cfc0 0x7fec3c13bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:35.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:35.986+0000 7fec41400640 1 -- 172.21.15.161:0/3365276121 >> 172.21.15.161:0/3365276121 conn(0x7fec3c06a2d0 msgr2=0x7fec3c0b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:35.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:35.986+0000 7fec41400640 1 -- 172.21.15.161:0/3365276121 shutdown_connections 2024-05-22T21:20:35.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:35.986+0000 7fec41400640 1 -- 172.21.15.161:0/3365276121 wait complete. 2024-05-22T21:20:36.013 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-22T21:20:36.014 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-22T21:20:36.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.154+0000 7f9637800640 1 Processor -- start 2024-05-22T21:20:36.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.154+0000 7f9637800640 1 -- start start 2024-05-22T21:20:36.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.154+0000 7f9637800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96301402e0 0x7f963013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:36.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.154+0000 7f9637800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f963013fc80 0x7f963006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:36.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.154+0000 7f9637800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96301400d0 con 0x7f963013cfc0 2024-05-22T21:20:36.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.154+0000 7f9637800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963013d3a0 con 0x7f963013fc80 2024-05-22T21:20:36.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.154+0000 7f9637800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96300cd920 con 0x7f96301402e0 2024-05-22T21:20:36.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.154+0000 7f9635800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f963013fc80 0x7f963006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:36.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.154+0000 7f9635800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f963013fc80 0x7f963006fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32800/0 (socket says 172.21.15.161:32800) 2024-05-22T21:20:36.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.154+0000 7f9635800640 1 -- 172.21.15.161:0/4183918955 learned_addr learned my addr 172.21.15.161:0/4183918955 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:36.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.154+0000 7f962fe00640 1 --2- 172.21.15.161:0/4183918955 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96301402e0 0x7f963013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:36.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.154+0000 7f9635800640 1 -- 172.21.15.161:0/4183918955 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96301402e0 msgr2=0x7f963013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:36.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.154+0000 7f9635800640 1 --2- 172.21.15.161:0/4183918955 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96301402e0 0x7f963013f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:36.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.154+0000 7f9635800640 1 -- 172.21.15.161:0/4183918955 >> v1:172.21.15.179:6789/0 conn(0x7f963013cfc0 legacy=0x7f963015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:36.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.154+0000 7f9635800640 1 -- 172.21.15.161:0/4183918955 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96300cdc00 con 0x7f963013fc80 2024-05-22T21:20:36.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.158+0000 7f9635800640 1 --2- 172.21.15.161:0/4183918955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963013fc80 0x7f963006fce0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f962800d150 tx=0x7f962801eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29014692017bc990 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:36.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.158+0000 7f962f400640 1 -- 172.21.15.161:0/4183918955 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962800b2d0 con 0x7f963013fc80 2024-05-22T21:20:36.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.158+0000 7f962f400640 1 -- 172.21.15.161:0/4183918955 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962800b430 con 0x7f963013fc80 2024-05-22T21:20:36.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.158+0000 7f962f400640 1 -- 172.21.15.161:0/4183918955 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9628009820 con 0x7f963013fc80 2024-05-22T21:20:36.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.158+0000 7f9637800640 1 -- 172.21.15.161:0/4183918955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963013fc80 msgr2=0x7f963006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:36.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.158+0000 7f9637800640 1 --2- 172.21.15.161:0/4183918955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963013fc80 0x7f963006fce0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f962800d150 tx=0x7f962801eb30 comp rx=0 tx=0).stop 2024-05-22T21:20:36.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.158+0000 7f9637800640 1 -- 172.21.15.161:0/4183918955 shutdown_connections 2024-05-22T21:20:36.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.158+0000 7f9637800640 1 --2- 172.21.15.161:0/4183918955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963013fc80 0x7f963006fce0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:36.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.158+0000 7f9637800640 1 --2- 172.21.15.161:0/4183918955 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f96301402e0 0x7f963013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:36.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.158+0000 7f9637800640 1 -- 172.21.15.161:0/4183918955 >> 172.21.15.161:0/4183918955 conn(0x7f963006a2d0 msgr2=0x7f963006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:36.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.158+0000 7f9637800640 1 -- 172.21.15.161:0/4183918955 shutdown_connections 2024-05-22T21:20:36.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.158+0000 7f9637800640 1 -- 172.21.15.161:0/4183918955 wait complete. 2024-05-22T21:20:36.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9637800640 1 Processor -- start 2024-05-22T21:20:36.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9637800640 1 -- start start 2024-05-22T21:20:36.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9637800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963013cfc0 0x7f963013c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:36.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9637800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f963013fc80 0x7f96301f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:36.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9637800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96301402e0 0x7f96300b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:36.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9637800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963013c560 con 0x7f963013cfc0 2024-05-22T21:20:36.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9637800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963013c9f0 con 0x7f96301402e0 2024-05-22T21:20:36.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9637800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963013f610 con 0x7f963013fc80 2024-05-22T21:20:36.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9634e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963013cfc0 0x7f963013c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:36.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9634e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963013cfc0 0x7f963013c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32814/0 (socket says 172.21.15.161:32814) 2024-05-22T21:20:36.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9634e00640 1 -- 172.21.15.161:0/2068924114 learned_addr learned my addr 172.21.15.161:0/2068924114 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:36.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9635800640 1 --2- 172.21.15.161:0/2068924114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96301402e0 0x7f96300b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:36.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f962fe00640 1 --2- 172.21.15.161:0/2068924114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f963013fc80 0x7f96301f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:36.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9634e00640 1 -- 172.21.15.161:0/2068924114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f963013fc80 msgr2=0x7f96301f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:36.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9634e00640 1 --2- 172.21.15.161:0/2068924114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f963013fc80 0x7f96301f5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:36.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9634e00640 1 -- 172.21.15.161:0/2068924114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96301402e0 msgr2=0x7f96300b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:36.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9634e00640 1 --2- 172.21.15.161:0/2068924114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96301402e0 0x7f96300b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:36.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9634e00640 1 -- 172.21.15.161:0/2068924114 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962800bd40 con 0x7f963013cfc0 2024-05-22T21:20:36.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9634e00640 1 --2- 172.21.15.161:0/2068924114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963013cfc0 0x7f963013c020 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f962000c450 tx=0x7f962000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:36.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f962cc00640 1 -- 172.21.15.161:0/2068924114 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620012830 con 0x7f963013cfc0 2024-05-22T21:20:36.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f962cc00640 1 -- 172.21.15.161:0/2068924114 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96200129b0 con 0x7f963013cfc0 2024-05-22T21:20:36.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f962cc00640 1 -- 172.21.15.161:0/2068924114 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620012d60 con 0x7f963013cfc0 2024-05-22T21:20:36.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9637800640 1 -- 172.21.15.161:0/2068924114 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f963013f8f0 con 0x7f963013cfc0 2024-05-22T21:20:36.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9637800640 1 -- 172.21.15.161:0/2068924114 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963015b010 con 0x7f963013cfc0 2024-05-22T21:20:36.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.162+0000 7f9637800640 1 -- 172.21.15.161:0/2068924114 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9630067220 con 0x7f963013cfc0 2024-05-22T21:20:36.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.166+0000 7f962cc00640 1 -- 172.21.15.161:0/2068924114 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f962001c460 con 0x7f963013cfc0 2024-05-22T21:20:36.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.166+0000 7f962cc00640 1 --2- 172.21.15.161:0/2068924114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9610066690 0x7f9610068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:36.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.166+0000 7f962cc00640 1 -- 172.21.15.161:0/2068924114 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9620083ec0 con 0x7f963013cfc0 2024-05-22T21:20:36.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.166+0000 7f962fe00640 1 --2- 172.21.15.161:0/2068924114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9610066690 0x7f9610068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:36.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.166+0000 7f962fe00640 1 --2- 172.21.15.161:0/2068924114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9610066690 0x7f9610068b50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f96300b4200 tx=0x7f961c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:36.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.170+0000 7f962cc00640 1 -- 172.21.15.161:0/2068924114 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9620051240 con 0x7f963013cfc0 2024-05-22T21:20:36.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.358+0000 7f9637800640 1 -- 172.21.15.161:0/2068924114 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9630071670 con 0x7f963013cfc0 2024-05-22T21:20:36.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.990+0000 7f962cc00640 1 -- 172.21.15.161:0/2068924114 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9620054860 con 0x7f963013cfc0 2024-05-22T21:20:36.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.994+0000 7f9637800640 1 -- 172.21.15.161:0/2068924114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9610066690 msgr2=0x7f9610068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:36.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.994+0000 7f9637800640 1 --2- 172.21.15.161:0/2068924114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9610066690 0x7f9610068b50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f96300b4200 tx=0x7f961c008040 comp rx=0 tx=0).stop 2024-05-22T21:20:36.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.994+0000 7f9637800640 1 -- 172.21.15.161:0/2068924114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963013cfc0 msgr2=0x7f963013c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:36.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.994+0000 7f9637800640 1 --2- 172.21.15.161:0/2068924114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963013cfc0 0x7f963013c020 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f962000c450 tx=0x7f962000c910 comp rx=0 tx=0).stop 2024-05-22T21:20:36.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.994+0000 7f9637800640 1 -- 172.21.15.161:0/2068924114 shutdown_connections 2024-05-22T21:20:36.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.994+0000 7f9637800640 1 --2- 172.21.15.161:0/2068924114 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9610066690 0x7f9610068b50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:36.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.994+0000 7f9637800640 1 --2- 172.21.15.161:0/2068924114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96301402e0 0x7f96300b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:36.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.994+0000 7f9637800640 1 --2- 172.21.15.161:0/2068924114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f963013fc80 0x7f96301f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:36.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.994+0000 7f9637800640 1 --2- 172.21.15.161:0/2068924114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963013cfc0 0x7f963013c020 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:36.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.994+0000 7f9637800640 1 -- 172.21.15.161:0/2068924114 >> 172.21.15.161:0/2068924114 conn(0x7f963006a2d0 msgr2=0x7f96300af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:36.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.994+0000 7f9637800640 1 -- 172.21.15.161:0/2068924114 shutdown_connections 2024-05-22T21:20:36.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:36.994+0000 7f9637800640 1 -- 172.21.15.161:0/2068924114 wait complete. 2024-05-22T21:20:37.018 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-22T21:20:37.018 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:20:37.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.154+0000 7f4b55000640 1 Processor -- start 2024-05-22T21:20:37.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.154+0000 7f4b55000640 1 -- start start 2024-05-22T21:20:37.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.154+0000 7f4b55000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4b5015e130 0x7f4b5015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.154+0000 7f4b55000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b5015d410 0x7f4b5015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.154+0000 7f4b55000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b50071670 con 0x7f4b5015dd50 2024-05-22T21:20:37.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.154+0000 7f4b55000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5015d870 con 0x7f4b5015e130 2024-05-22T21:20:37.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.154+0000 7f4b55000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50070a60 con 0x7f4b5015d410 2024-05-22T21:20:37.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.154+0000 7f4b4de00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f4b5015dd50 0x7f4b5006d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60532/0 (socket says 172.21.15.161:60532) 2024-05-22T21:20:37.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.154+0000 7f4b4de00640 1 -- 172.21.15.161:0/2978601192 learned_addr learned my addr 172.21.15.161:0/2978601192 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:37.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.154+0000 7f4b4d400640 1 --2- 172.21.15.161:0/2978601192 >> v2:172.21.15.161:3300/0 conn(0x7f4b5015e130 0x7f4b5015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.154+0000 7f4b4e800640 1 --2- 172.21.15.161:0/2978601192 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b5015d410 0x7f4b5015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.154+0000 7f4b4d400640 1 -- 172.21.15.161:0/2978601192 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b5015d410 msgr2=0x7f4b5015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:37.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.154+0000 7f4b4d400640 1 --2- 172.21.15.161:0/2978601192 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b5015d410 0x7f4b5015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.154+0000 7f4b4d400640 1 -- 172.21.15.161:0/2978601192 >> v1:172.21.15.161:6789/0 conn(0x7f4b5015dd50 legacy=0x7f4b5006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:37.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.154+0000 7f4b4d400640 1 -- 172.21.15.161:0/2978601192 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b50070d40 con 0x7f4b5015e130 2024-05-22T21:20:37.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.154+0000 7f4b4ca00640 1 -- 172.21.15.161:0/2978601192 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766141123 0 0) 0x7f4b50071670 con 0x7f4b5015dd50 2024-05-22T21:20:37.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.158+0000 7f4b4d400640 1 --2- 172.21.15.161:0/2978601192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b5015e130 0x7f4b5015ced0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0097c0 tx=0x7f4b3c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a831d0a41c427fa server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:37.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.158+0000 7f4b4ca00640 1 -- 172.21.15.161:0/2978601192 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0ab670 con 0x7f4b5015e130 2024-05-22T21:20:37.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.158+0000 7f4b4ca00640 1 -- 172.21.15.161:0/2978601192 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c0ab7d0 con 0x7f4b5015e130 2024-05-22T21:20:37.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.158+0000 7f4b4ca00640 1 -- 172.21.15.161:0/2978601192 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0abb60 con 0x7f4b5015e130 2024-05-22T21:20:37.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.158+0000 7f4b55000640 1 -- 172.21.15.161:0/2978601192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b5015e130 msgr2=0x7f4b5015ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:37.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.158+0000 7f4b55000640 1 --2- 172.21.15.161:0/2978601192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b5015e130 0x7f4b5015ced0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0097c0 tx=0x7f4b3c01c470 comp rx=0 tx=0).stop 2024-05-22T21:20:37.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.158+0000 7f4b55000640 1 -- 172.21.15.161:0/2978601192 shutdown_connections 2024-05-22T21:20:37.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.158+0000 7f4b55000640 1 --2- 172.21.15.161:0/2978601192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b5015e130 0x7f4b5015ced0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.158+0000 7f4b55000640 1 --2- 172.21.15.161:0/2978601192 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b5015d410 0x7f4b5015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.158+0000 7f4b55000640 1 -- 172.21.15.161:0/2978601192 >> 172.21.15.161:0/2978601192 conn(0x7f4b500b0bc0 msgr2=0x7f4b500b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:37.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.158+0000 7f4b55000640 1 -- 172.21.15.161:0/2978601192 shutdown_connections 2024-05-22T21:20:37.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.158+0000 7f4b55000640 1 -- 172.21.15.161:0/2978601192 wait complete. 2024-05-22T21:20:37.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.158+0000 7f4b55000640 1 Processor -- start 2024-05-22T21:20:37.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.158+0000 7f4b55000640 1 -- start start 2024-05-22T21:20:37.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b55000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b5015d410 0x7f4b5014b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b55000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b5015dd50 0x7f4b50156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b55000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b5015e130 0x7f4b50148700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b55000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50148e30 con 0x7f4b5015dd50 2024-05-22T21:20:37.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b55000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50158720 con 0x7f4b5015d410 2024-05-22T21:20:37.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b55000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50148c40 con 0x7f4b5015e130 2024-05-22T21:20:37.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b4d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b5015dd50 0x7f4b50156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b4d400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b5015dd50 0x7f4b50156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32822/0 (socket says 172.21.15.161:32822) 2024-05-22T21:20:37.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b4d400640 1 -- 172.21.15.161:0/2324506690 learned_addr learned my addr 172.21.15.161:0/2324506690 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:37.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b4de00640 1 --2- 172.21.15.161:0/2324506690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b5015d410 0x7f4b5014b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b4e800640 1 --2- 172.21.15.161:0/2324506690 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b5015e130 0x7f4b50148700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b4d400640 1 -- 172.21.15.161:0/2324506690 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b5015e130 msgr2=0x7f4b50148700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:37.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b4d400640 1 --2- 172.21.15.161:0/2324506690 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b5015e130 0x7f4b50148700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b4d400640 1 -- 172.21.15.161:0/2324506690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b5015d410 msgr2=0x7f4b5014b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:37.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b4d400640 1 --2- 172.21.15.161:0/2324506690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b5015d410 0x7f4b5014b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b4d400640 1 -- 172.21.15.161:0/2324506690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c009420 con 0x7f4b5015dd50 2024-05-22T21:20:37.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b4d400640 1 --2- 172.21.15.161:0/2324506690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b5015dd50 0x7f4b50156720 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c01ab20 tx=0x7f4b3c019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:37.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b46000640 1 -- 172.21.15.161:0/2324506690 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0abe90 con 0x7f4b5015dd50 2024-05-22T21:20:37.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b46000640 1 -- 172.21.15.161:0/2324506690 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c0bf450 con 0x7f4b5015dd50 2024-05-22T21:20:37.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b46000640 1 -- 172.21.15.161:0/2324506690 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0bf7e0 con 0x7f4b5015dd50 2024-05-22T21:20:37.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b55000640 1 -- 172.21.15.161:0/2324506690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5014c2d0 con 0x7f4b5015dd50 2024-05-22T21:20:37.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b55000640 1 -- 172.21.15.161:0/2324506690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5014bd10 con 0x7f4b5015dd50 2024-05-22T21:20:37.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.162+0000 7f4b55000640 1 -- 172.21.15.161:0/2324506690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b500706b0 con 0x7f4b5015dd50 2024-05-22T21:20:37.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.166+0000 7f4b46000640 1 -- 172.21.15.161:0/2324506690 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4b3c0bf940 con 0x7f4b5015dd50 2024-05-22T21:20:37.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.166+0000 7f4b46000640 1 --2- 172.21.15.161:0/2324506690 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4b30066690 0x7f4b30068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.166+0000 7f4b46000640 1 -- 172.21.15.161:0/2324506690 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b3c11eb80 con 0x7f4b5015dd50 2024-05-22T21:20:37.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.166+0000 7f4b4de00640 1 --2- 172.21.15.161:0/2324506690 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4b30066690 0x7f4b30068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.166+0000 7f4b4de00640 1 --2- 172.21.15.161:0/2324506690 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4b30066690 0x7f4b30068b50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4b40007fd0 tx=0x7f4b40007d60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:37.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.170+0000 7f4b46000640 1 -- 172.21.15.161:0/2324506690 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4b3c0ebf90 con 0x7f4b5015dd50 2024-05-22T21:20:37.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.358+0000 7f4b55000640 1 -- 172.21.15.161:0/2324506690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b50067220 con 0x7f4b5015dd50 2024-05-22T21:20:37.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.358+0000 7f4b46000640 1 -- 172.21.15.161:0/2324506690 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f4b3c11ee60 con 0x7f4b5015dd50 2024-05-22T21:20:37.362 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:37.362 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":70,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7992,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/207877183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":207877183},{"type":"v1","addr":"172.21.15.161:6836","nonce":207877183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2152702106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2152702106},{"type":"v1","addr":"172.21.15.161:6837","nonce":2152702106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/749927648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":749927648},{"type":"v1","addr":"172.21.15.179:6834","nonce":749927648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:20:33.964441+0000","modified":"2024-05-22T21:20:36.988238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8004},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8004":{"gid":8004,"name":"b","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/93167194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":93167194},{"type":"v1","addr":"172.21.15.179:6835","nonce":93167194}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-05-22T21:20:37.362 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 70 2024-05-22T21:20:37.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.358+0000 7f4b55000640 1 -- 172.21.15.161:0/2324506690 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4b30066690 msgr2=0x7f4b30068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:37.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.358+0000 7f4b55000640 1 --2- 172.21.15.161:0/2324506690 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4b30066690 0x7f4b30068b50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4b40007fd0 tx=0x7f4b40007d60 comp rx=0 tx=0).stop 2024-05-22T21:20:37.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.358+0000 7f4b55000640 1 -- 172.21.15.161:0/2324506690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b5015dd50 msgr2=0x7f4b50156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:37.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.358+0000 7f4b55000640 1 --2- 172.21.15.161:0/2324506690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b5015dd50 0x7f4b50156720 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c01ab20 tx=0x7f4b3c019210 comp rx=0 tx=0).stop 2024-05-22T21:20:37.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.362+0000 7f4b55000640 1 -- 172.21.15.161:0/2324506690 shutdown_connections 2024-05-22T21:20:37.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.362+0000 7f4b55000640 1 --2- 172.21.15.161:0/2324506690 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4b30066690 0x7f4b30068b50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.362+0000 7f4b55000640 1 --2- 172.21.15.161:0/2324506690 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b5015e130 0x7f4b50148700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.362+0000 7f4b55000640 1 --2- 172.21.15.161:0/2324506690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b5015dd50 0x7f4b50156720 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.362+0000 7f4b55000640 1 --2- 172.21.15.161:0/2324506690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b5015d410 0x7f4b5014b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.362+0000 7f4b55000640 1 -- 172.21.15.161:0/2324506690 >> 172.21.15.161:0/2324506690 conn(0x7f4b500b0bc0 msgr2=0x7f4b5006c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:37.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.362+0000 7f4b55000640 1 -- 172.21.15.161:0/2324506690 shutdown_connections 2024-05-22T21:20:37.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.362+0000 7f4b55000640 1 -- 172.21.15.161:0/2324506690 wait complete. 2024-05-22T21:20:37.382 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:20:37.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.518+0000 7ff466e00640 1 Processor -- start 2024-05-22T21:20:37.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.518+0000 7ff466e00640 1 -- start start 2024-05-22T21:20:37.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.518+0000 7ff466e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff460070db0 0x7ff4600b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.518+0000 7ff466e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4600b2ea0 0x7ff4600cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.518+0000 7ff466e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff460159bd0 con 0x7ff4601597f0 2024-05-22T21:20:37.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.518+0000 7ff466e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4600cbdd0 con 0x7ff460070db0 2024-05-22T21:20:37.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.518+0000 7ff466e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600cc050 con 0x7ff4600b2ea0 2024-05-22T21:20:37.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.518+0000 7ff457400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff460070db0 0x7ff4600b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.518+0000 7ff457400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff460070db0 0x7ff4600b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32824/0 (socket says 172.21.15.161:32824) 2024-05-22T21:20:37.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.518+0000 7ff457400640 1 -- 172.21.15.161:0/4046556386 learned_addr learned my addr 172.21.15.161:0/4046556386 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:37.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.518+0000 7ff464e00640 1 --2- 172.21.15.161:0/4046556386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4600b2ea0 0x7ff4600cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.518+0000 7ff457400640 1 -- 172.21.15.161:0/4046556386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4600b2ea0 msgr2=0x7ff4600cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:37.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.518+0000 7ff457400640 1 --2- 172.21.15.161:0/4046556386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4600b2ea0 0x7ff4600cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.518+0000 7ff457400640 1 -- 172.21.15.161:0/4046556386 >> v1:172.21.15.179:6789/0 conn(0x7ff4601597f0 legacy=0x7ff4600706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:37.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.522+0000 7ff457400640 1 -- 172.21.15.161:0/4046556386 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4600b3720 con 0x7ff460070db0 2024-05-22T21:20:37.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.522+0000 7ff464e00640 1 --2- 172.21.15.161:0/4046556386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4600b2ea0 0x7ff4600cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:20:37.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.522+0000 7ff457400640 1 --2- 172.21.15.161:0/4046556386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff460070db0 0x7ff4600b2960 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7ff44800bb10 tx=0x7ff44801ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b2acc2d4102ee0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:37.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.522+0000 7ff45f400640 1 -- 172.21.15.161:0/4046556386 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4480a7c80 con 0x7ff460070db0 2024-05-22T21:20:37.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.522+0000 7ff45f400640 1 -- 172.21.15.161:0/4046556386 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4480a7de0 con 0x7ff460070db0 2024-05-22T21:20:37.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.522+0000 7ff45f400640 1 -- 172.21.15.161:0/4046556386 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4480af660 con 0x7ff460070db0 2024-05-22T21:20:37.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.522+0000 7ff466e00640 1 -- 172.21.15.161:0/4046556386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff460070db0 msgr2=0x7ff4600b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:37.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.522+0000 7ff466e00640 1 --2- 172.21.15.161:0/4046556386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff460070db0 0x7ff4600b2960 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7ff44800bb10 tx=0x7ff44801ec30 comp rx=0 tx=0).stop 2024-05-22T21:20:37.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.522+0000 7ff466e00640 1 -- 172.21.15.161:0/4046556386 shutdown_connections 2024-05-22T21:20:37.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.522+0000 7ff466e00640 1 --2- 172.21.15.161:0/4046556386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4600b2ea0 0x7ff4600cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.522+0000 7ff466e00640 1 --2- 172.21.15.161:0/4046556386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff460070db0 0x7ff4600b2960 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.522+0000 7ff466e00640 1 -- 172.21.15.161:0/4046556386 >> 172.21.15.161:0/4046556386 conn(0x7ff4600b0bc0 msgr2=0x7ff4600b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:37.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.522+0000 7ff466e00640 1 -- 172.21.15.161:0/4046556386 shutdown_connections 2024-05-22T21:20:37.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.522+0000 7ff466e00640 1 -- 172.21.15.161:0/4046556386 wait complete. 2024-05-22T21:20:37.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff466e00640 1 Processor -- start 2024-05-22T21:20:37.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff466e00640 1 -- start start 2024-05-22T21:20:37.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff466e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff460070db0 0x7ff4601403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff466e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4600b2ea0 0x7ff4600ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff466e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4601597f0 0x7ff46014b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff466e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46014cde0 con 0x7ff4601597f0 2024-05-22T21:20:37.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff466e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46015c6f0 con 0x7ff460070db0 2024-05-22T21:20:37.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff466e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46015c130 con 0x7ff4600b2ea0 2024-05-22T21:20:37.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff464e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4601597f0 0x7ff46014b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff464e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4601597f0 0x7ff46014b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32834/0 (socket says 172.21.15.161:32834) 2024-05-22T21:20:37.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff464e00640 1 -- 172.21.15.161:0/1051879186 learned_addr learned my addr 172.21.15.161:0/1051879186 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:37.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff457400640 1 --2- 172.21.15.161:0/1051879186 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4600b2ea0 0x7ff4600ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff45fe00640 1 --2- 172.21.15.161:0/1051879186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff460070db0 0x7ff4601403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff464e00640 1 -- 172.21.15.161:0/1051879186 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4600b2ea0 msgr2=0x7ff4600ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:37.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff464e00640 1 --2- 172.21.15.161:0/1051879186 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4600b2ea0 0x7ff4600ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff464e00640 1 -- 172.21.15.161:0/1051879186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff460070db0 msgr2=0x7ff4601403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:37.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff464e00640 1 --2- 172.21.15.161:0/1051879186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff460070db0 0x7ff4601403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff464e00640 1 -- 172.21.15.161:0/1051879186 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff458009a70 con 0x7ff4601597f0 2024-05-22T21:20:37.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff45fe00640 1 --2- 172.21.15.161:0/1051879186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff460070db0 0x7ff4601403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:37.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff464e00640 1 --2- 172.21.15.161:0/1051879186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4601597f0 0x7ff46014b3c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7ff458009f00 tx=0x7ff45800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:37.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff45cc00640 1 -- 172.21.15.161:0/1051879186 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458009d70 con 0x7ff4601597f0 2024-05-22T21:20:37.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff45cc00640 1 -- 172.21.15.161:0/1051879186 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45800f040 con 0x7ff4601597f0 2024-05-22T21:20:37.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff45cc00640 1 -- 172.21.15.161:0/1051879186 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458003bc0 con 0x7ff4601597f0 2024-05-22T21:20:37.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.526+0000 7ff466e00640 1 -- 172.21.15.161:0/1051879186 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44800b770 con 0x7ff4601597f0 2024-05-22T21:20:37.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.530+0000 7ff466e00640 1 -- 172.21.15.161:0/1051879186 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4601549b0 con 0x7ff4601597f0 2024-05-22T21:20:37.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.530+0000 7ff466e00640 1 -- 172.21.15.161:0/1051879186 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff460067220 con 0x7ff4601597f0 2024-05-22T21:20:37.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.530+0000 7ff45cc00640 1 -- 172.21.15.161:0/1051879186 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff458014480 con 0x7ff4601597f0 2024-05-22T21:20:37.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.530+0000 7ff45cc00640 1 --2- 172.21.15.161:0/1051879186 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff43c0666c0 0x7ff43c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.530+0000 7ff45cc00640 1 -- 172.21.15.161:0/1051879186 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff458084d20 con 0x7ff4601597f0 2024-05-22T21:20:37.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.530+0000 7ff45fe00640 1 --2- 172.21.15.161:0/1051879186 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff43c0666c0 0x7ff43c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.530+0000 7ff45fe00640 1 --2- 172.21.15.161:0/1051879186 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff43c0666c0 0x7ff43c068b80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff450000f70 tx=0x7ff450009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:37.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.534+0000 7ff45cc00640 1 -- 172.21.15.161:0/1051879186 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff458052080 con 0x7ff4601597f0 2024-05-22T21:20:37.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.698+0000 7ff466e00640 1 -- 172.21.15.161:0/1051879186 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff460158dd0 con 0x7ff4601597f0 2024-05-22T21:20:37.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.698+0000 7ff45cc00640 1 -- 172.21.15.161:0/1051879186 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16160 (secure 0 0 0) 0x7ff458018090 con 0x7ff4601597f0 2024-05-22T21:20:37.701 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:37.701 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":158,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:20:34.969416+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:20:32.457234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-22T21:20:32.951241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:20:37.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.698+0000 7ff466e00640 1 -- 172.21.15.161:0/1051879186 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff43c0666c0 msgr2=0x7ff43c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:37.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.698+0000 7ff466e00640 1 --2- 172.21.15.161:0/1051879186 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff43c0666c0 0x7ff43c068b80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff450000f70 tx=0x7ff450009040 comp rx=0 tx=0).stop 2024-05-22T21:20:37.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.698+0000 7ff466e00640 1 -- 172.21.15.161:0/1051879186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4601597f0 msgr2=0x7ff46014b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:37.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.698+0000 7ff466e00640 1 --2- 172.21.15.161:0/1051879186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4601597f0 0x7ff46014b3c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7ff458009f00 tx=0x7ff45800c490 comp rx=0 tx=0).stop 2024-05-22T21:20:37.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.698+0000 7ff466e00640 1 -- 172.21.15.161:0/1051879186 shutdown_connections 2024-05-22T21:20:37.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.698+0000 7ff466e00640 1 --2- 172.21.15.161:0/1051879186 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff43c0666c0 0x7ff43c068b80 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.698+0000 7ff466e00640 1 --2- 172.21.15.161:0/1051879186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4601597f0 0x7ff46014b3c0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.698+0000 7ff466e00640 1 --2- 172.21.15.161:0/1051879186 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4600b2ea0 0x7ff4600ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.698+0000 7ff466e00640 1 --2- 172.21.15.161:0/1051879186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff460070db0 0x7ff4601403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.698+0000 7ff466e00640 1 -- 172.21.15.161:0/1051879186 >> 172.21.15.161:0/1051879186 conn(0x7ff4600b0bc0 msgr2=0x7ff46006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:37.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.698+0000 7ff466e00640 1 -- 172.21.15.161:0/1051879186 shutdown_connections 2024-05-22T21:20:37.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.698+0000 7ff466e00640 1 -- 172.21.15.161:0/1051879186 wait complete. 2024-05-22T21:20:37.723 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-22T21:20:37.723 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:20:37.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.854+0000 7fa481600640 1 Processor -- start 2024-05-22T21:20:37.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.854+0000 7fa481600640 1 -- start start 2024-05-22T21:20:37.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.858+0000 7fa481600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa47c1597f0 0x7fa47c0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.858+0000 7fa481600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa47c0b3070 0x7fa47c0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.858+0000 7fa481600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c071040 con 0x7fa47c070be0 2024-05-22T21:20:37.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.858+0000 7fa481600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47c0b3500 con 0x7fa47c0b3070 2024-05-22T21:20:37.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.858+0000 7fa481600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c159bd0 con 0x7fa47c1597f0 2024-05-22T21:20:37.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.858+0000 7fa47b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa47c0b3070 0x7fa47c0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.858+0000 7fa47b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa47c0b3070 0x7fa47c0cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32842/0 (socket says 172.21.15.161:32842) 2024-05-22T21:20:37.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.858+0000 7fa47b400640 1 -- 172.21.15.161:0/3516892521 learned_addr learned my addr 172.21.15.161:0/3516892521 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:37.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.858+0000 7fa47aa00640 1 --2- 172.21.15.161:0/3516892521 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa47c1597f0 0x7fa47c0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.858+0000 7fa47b400640 1 -- 172.21.15.161:0/3516892521 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa47c1597f0 msgr2=0x7fa47c0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:37.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.858+0000 7fa47b400640 1 --2- 172.21.15.161:0/3516892521 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa47c1597f0 0x7fa47c0706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.858+0000 7fa47b400640 1 -- 172.21.15.161:0/3516892521 >> v1:172.21.15.179:6789/0 conn(0x7fa47c070be0 legacy=0x7fa47c0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:37.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.858+0000 7fa47b400640 1 -- 172.21.15.161:0/3516892521 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c0cbdd0 con 0x7fa47c0b3070 2024-05-22T21:20:37.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.858+0000 7fa47b400640 1 --2- 172.21.15.161:0/3516892521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa47c0b3070 0x7fa47c0cc550 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fa47400af60 tx=0x7fa47401db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628aff4f9cd528ce server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:37.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa47a000640 1 -- 172.21.15.161:0/3516892521 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47400a2d0 con 0x7fa47c0b3070 2024-05-22T21:20:37.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa47a000640 1 -- 172.21.15.161:0/3516892521 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4740a6970 con 0x7fa47c0b3070 2024-05-22T21:20:37.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa47a000640 1 -- 172.21.15.161:0/3516892521 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4740a6d00 con 0x7fa47c0b3070 2024-05-22T21:20:37.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa481600640 1 -- 172.21.15.161:0/3516892521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa47c0b3070 msgr2=0x7fa47c0cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:37.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa481600640 1 --2- 172.21.15.161:0/3516892521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa47c0b3070 0x7fa47c0cc550 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fa47400af60 tx=0x7fa47401db30 comp rx=0 tx=0).stop 2024-05-22T21:20:37.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa481600640 1 -- 172.21.15.161:0/3516892521 shutdown_connections 2024-05-22T21:20:37.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa481600640 1 --2- 172.21.15.161:0/3516892521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa47c0b3070 0x7fa47c0cc550 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa481600640 1 --2- 172.21.15.161:0/3516892521 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa47c1597f0 0x7fa47c0706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa481600640 1 -- 172.21.15.161:0/3516892521 >> 172.21.15.161:0/3516892521 conn(0x7fa47c0b0bc0 msgr2=0x7fa47c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:37.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa481600640 1 -- 172.21.15.161:0/3516892521 shutdown_connections 2024-05-22T21:20:37.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa481600640 1 -- 172.21.15.161:0/3516892521 wait complete. 2024-05-22T21:20:37.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa481600640 1 Processor -- start 2024-05-22T21:20:37.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa481600640 1 -- start start 2024-05-22T21:20:37.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa481600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa47c070be0 0x7fa47c0ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa481600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa47c0b3070 0x7fa47c1396a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa481600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa47c1597f0 0x7fa47c145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa481600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c149660 con 0x7fa47c070be0 2024-05-22T21:20:37.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa481600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c14b550 con 0x7fa47c1597f0 2024-05-22T21:20:37.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa481600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0ac950 con 0x7fa47c0b3070 2024-05-22T21:20:37.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa47aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa47c070be0 0x7fa47c0ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa47aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa47c070be0 0x7fa47c0ad3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32856/0 (socket says 172.21.15.161:32856) 2024-05-22T21:20:37.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa47aa00640 1 -- 172.21.15.161:0/2933081509 learned_addr learned my addr 172.21.15.161:0/2933081509 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:37.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa472000640 1 --2- 172.21.15.161:0/2933081509 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa47c0b3070 0x7fa47c1396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.862+0000 7fa47b400640 1 --2- 172.21.15.161:0/2933081509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa47c1597f0 0x7fa47c145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.866+0000 7fa47aa00640 1 -- 172.21.15.161:0/2933081509 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa47c0b3070 msgr2=0x7fa47c1396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:37.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.866+0000 7fa47aa00640 1 --2- 172.21.15.161:0/2933081509 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa47c0b3070 0x7fa47c1396a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.866+0000 7fa47aa00640 1 -- 172.21.15.161:0/2933081509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa47c1597f0 msgr2=0x7fa47c145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:37.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.866+0000 7fa47aa00640 1 --2- 172.21.15.161:0/2933081509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa47c1597f0 0x7fa47c145470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:37.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.866+0000 7fa47aa00640 1 -- 172.21.15.161:0/2933081509 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47400ac40 con 0x7fa47c070be0 2024-05-22T21:20:37.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.866+0000 7fa47aa00640 1 --2- 172.21.15.161:0/2933081509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa47c070be0 0x7fa47c0ad3f0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00e870 tx=0x7fa46c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:37.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.866+0000 7fa473400640 1 -- 172.21.15.161:0/2933081509 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c009510 con 0x7fa47c070be0 2024-05-22T21:20:37.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.866+0000 7fa473400640 1 -- 172.21.15.161:0/2933081509 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c00eec0 con 0x7fa47c070be0 2024-05-22T21:20:37.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.866+0000 7fa473400640 1 -- 172.21.15.161:0/2933081509 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c00cb40 con 0x7fa47c070be0 2024-05-22T21:20:37.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.866+0000 7fa481600640 1 -- 172.21.15.161:0/2933081509 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47c1498b0 con 0x7fa47c070be0 2024-05-22T21:20:37.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.866+0000 7fa481600640 1 -- 172.21.15.161:0/2933081509 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47c0aced0 con 0x7fa47c070be0 2024-05-22T21:20:37.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.866+0000 7fa481600640 1 -- 172.21.15.161:0/2933081509 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa47c0ad060 con 0x7fa47c070be0 2024-05-22T21:20:37.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.866+0000 7fa473400640 1 -- 172.21.15.161:0/2933081509 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa46c00cca0 con 0x7fa47c070be0 2024-05-22T21:20:37.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.866+0000 7fa473400640 1 --2- 172.21.15.161:0/2933081509 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa458066690 0x7fa458068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:37.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.866+0000 7fa473400640 1 -- 172.21.15.161:0/2933081509 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa46c084280 con 0x7fa47c070be0 2024-05-22T21:20:37.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.866+0000 7fa472000640 1 --2- 172.21.15.161:0/2933081509 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa458066690 0x7fa458068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:37.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.870+0000 7fa472000640 1 --2- 172.21.15.161:0/2933081509 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa458066690 0x7fa458068b50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa47c1f9870 tx=0x7fa464009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:37.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:37.870+0000 7fa473400640 1 -- 172.21.15.161:0/2933081509 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa46c051600 con 0x7fa47c070be0 2024-05-22T21:20:38.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.022+0000 7fa481600640 1 -- 172.21.15.161:0/2933081509 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa47c0cfaa0 con 0x7fa458066690 2024-05-22T21:20:38.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.030+0000 7fa473400640 1 -- 172.21.15.161:0/2933081509 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285205 (secure 0 0 0) 0x7fa47c0cfaa0 con 0x7fa458066690 2024-05-22T21:20:38.036 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:38.038 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:20:38.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.034+0000 7fa481600640 1 -- 172.21.15.161:0/2933081509 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa458066690 msgr2=0x7fa458068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:38.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.034+0000 7fa481600640 1 --2- 172.21.15.161:0/2933081509 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa458066690 0x7fa458068b50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa47c1f9870 tx=0x7fa464009040 comp rx=0 tx=0).stop 2024-05-22T21:20:38.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.034+0000 7fa481600640 1 -- 172.21.15.161:0/2933081509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa47c070be0 msgr2=0x7fa47c0ad3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:38.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.034+0000 7fa481600640 1 --2- 172.21.15.161:0/2933081509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa47c070be0 0x7fa47c0ad3f0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00e870 tx=0x7fa46c00ec70 comp rx=0 tx=0).stop 2024-05-22T21:20:38.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.034+0000 7fa481600640 1 -- 172.21.15.161:0/2933081509 shutdown_connections 2024-05-22T21:20:38.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.034+0000 7fa481600640 1 --2- 172.21.15.161:0/2933081509 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa458066690 0x7fa458068b50 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.034+0000 7fa481600640 1 --2- 172.21.15.161:0/2933081509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa47c1597f0 0x7fa47c145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.034+0000 7fa481600640 1 --2- 172.21.15.161:0/2933081509 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa47c0b3070 0x7fa47c1396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.034+0000 7fa481600640 1 --2- 172.21.15.161:0/2933081509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa47c070be0 0x7fa47c0ad3f0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.034+0000 7fa481600640 1 -- 172.21.15.161:0/2933081509 >> 172.21.15.161:0/2933081509 conn(0x7fa47c0b0bc0 msgr2=0x7fa47c06c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:38.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.034+0000 7fa481600640 1 -- 172.21.15.161:0/2933081509 shutdown_connections 2024-05-22T21:20:38.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.034+0000 7fa481600640 1 -- 172.21.15.161:0/2933081509 wait complete. 2024-05-22T21:20:38.057 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-05-22T21:20:36.488688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8049,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554450,"statfs":{"total":1600000000,"available":1591757487,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000633"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.981119+0000","last_change":"2024-05-22T21:20:33.987530+0000","last_active":"2024-05-22T21:20:34.981119+0000","last_peered":"2024-05-22T21:20:34.981119+0000","last_clean":"2024-05-22T21:20:34.981119+0000","last_became_active":"2024-05-22T21:20:33.987187+0000","last_became_peered":"2024-05-22T21:20:33.987187+0000","last_unstale":"2024-05-22T21:20:34.981119+0000","last_undegraded":"2024-05-22T21:20:34.981119+0000","last_fullsized":"2024-05-22T21:20:34.981119+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:06:23.454383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084595+0000","last_change":"2024-05-22T21:20:33.989948+0000","last_active":"2024-05-22T21:20:35.084595+0000","last_peered":"2024-05-22T21:20:35.084595+0000","last_clean":"2024-05-22T21:20:35.084595+0000","last_became_active":"2024-05-22T21:20:33.989761+0000","last_became_peered":"2024-05-22T21:20:33.989761+0000","last_unstale":"2024-05-22T21:20:35.084595+0000","last_undegraded":"2024-05-22T21:20:35.084595+0000","last_fullsized":"2024-05-22T21:20:35.084595+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:40:45.608132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010058+0000","last_change":"2024-05-22T21:20:33.992444+0000","last_active":"2024-05-22T21:20:35.010058+0000","last_peered":"2024-05-22T21:20:35.010058+0000","last_clean":"2024-05-22T21:20:35.010058+0000","last_became_active":"2024-05-22T21:20:33.988764+0000","last_became_peered":"2024-05-22T21:20:33.988764+0000","last_unstale":"2024-05-22T21:20:35.010058+0000","last_undegraded":"2024-05-22T21:20:35.010058+0000","last_fullsized":"2024-05-22T21:20:35.010058+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:55:50.256135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.980760+0000","last_change":"2024-05-22T21:20:33.986053+0000","last_active":"2024-05-22T21:20:34.980760+0000","last_peered":"2024-05-22T21:20:34.980760+0000","last_clean":"2024-05-22T21:20:34.980760+0000","last_became_active":"2024-05-22T21:20:33.984019+0000","last_became_peered":"2024-05-22T21:20:33.984019+0000","last_unstale":"2024-05-22T21:20:34.980760+0000","last_undegraded":"2024-05-22T21:20:34.980760+0000","last_fullsized":"2024-05-22T21:20:34.980760+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:14:18.962972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984949+0000","last_change":"2024-05-22T21:20:33.984277+0000","last_active":"2024-05-22T21:20:34.984949+0000","last_peered":"2024-05-22T21:20:34.984949+0000","last_clean":"2024-05-22T21:20:34.984949+0000","last_became_active":"2024-05-22T21:20:33.983977+0000","last_became_peered":"2024-05-22T21:20:33.983977+0000","last_unstale":"2024-05-22T21:20:34.984949+0000","last_undegraded":"2024-05-22T21:20:34.984949+0000","last_fullsized":"2024-05-22T21:20:34.984949+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:53:21.407922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984088+0000","last_change":"2024-05-22T21:20:33.986489+0000","last_active":"2024-05-22T21:20:34.984088+0000","last_peered":"2024-05-22T21:20:34.984088+0000","last_clean":"2024-05-22T21:20:34.984088+0000","last_became_active":"2024-05-22T21:20:33.986268+0000","last_became_peered":"2024-05-22T21:20:33.986268+0000","last_unstale":"2024-05-22T21:20:34.984088+0000","last_undegraded":"2024-05-22T21:20:34.984088+0000","last_fullsized":"2024-05-22T21:20:34.984088+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:13:10.680899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.983660+0000","last_change":"2024-05-22T21:20:33.990749+0000","last_active":"2024-05-22T21:20:34.983660+0000","last_peered":"2024-05-22T21:20:34.983660+0000","last_clean":"2024-05-22T21:20:34.983660+0000","last_became_active":"2024-05-22T21:20:33.989220+0000","last_became_peered":"2024-05-22T21:20:33.989220+0000","last_unstale":"2024-05-22T21:20:34.983660+0000","last_undegraded":"2024-05-22T21:20:34.983660+0000","last_fullsized":"2024-05-22T21:20:34.983660+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:46:01.370439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010202+0000","last_change":"2024-05-22T21:20:33.992912+0000","last_active":"2024-05-22T21:20:35.010202+0000","last_peered":"2024-05-22T21:20:35.010202+0000","last_clean":"2024-05-22T21:20:35.010202+0000","last_became_active":"2024-05-22T21:20:33.989515+0000","last_became_peered":"2024-05-22T21:20:33.989515+0000","last_unstale":"2024-05-22T21:20:35.010202+0000","last_undegraded":"2024-05-22T21:20:35.010202+0000","last_fullsized":"2024-05-22T21:20:35.010202+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:34:29.353240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010069+0000","last_change":"2024-05-22T21:20:33.988082+0000","last_active":"2024-05-22T21:20:35.010069+0000","last_peered":"2024-05-22T21:20:35.010069+0000","last_clean":"2024-05-22T21:20:35.010069+0000","last_became_active":"2024-05-22T21:20:33.987851+0000","last_became_peered":"2024-05-22T21:20:33.987851+0000","last_unstale":"2024-05-22T21:20:35.010069+0000","last_undegraded":"2024-05-22T21:20:35.010069+0000","last_fullsized":"2024-05-22T21:20:35.010069+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:33:05.996672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084822+0000","last_change":"2024-05-22T21:20:33.987659+0000","last_active":"2024-05-22T21:20:35.084822+0000","last_peered":"2024-05-22T21:20:35.084822+0000","last_clean":"2024-05-22T21:20:35.084822+0000","last_became_active":"2024-05-22T21:20:33.987129+0000","last_became_peered":"2024-05-22T21:20:33.987129+0000","last_unstale":"2024-05-22T21:20:35.084822+0000","last_undegraded":"2024-05-22T21:20:35.084822+0000","last_fullsized":"2024-05-22T21:20:35.084822+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:53:24.255863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181683+0000","last_change":"2024-05-22T21:20:33.977208+0000","last_active":"2024-05-22T21:20:35.181683+0000","last_peered":"2024-05-22T21:20:35.181683+0000","last_clean":"2024-05-22T21:20:35.181683+0000","last_became_active":"2024-05-22T21:20:33.976995+0000","last_became_peered":"2024-05-22T21:20:33.976995+0000","last_unstale":"2024-05-22T21:20:35.181683+0000","last_undegraded":"2024-05-22T21:20:35.181683+0000","last_fullsized":"2024-05-22T21:20:35.181683+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:25:33.574679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010222+0000","last_change":"2024-05-22T21:20:33.992317+0000","last_active":"2024-05-22T21:20:35.010222+0000","last_peered":"2024-05-22T21:20:35.010222+0000","last_clean":"2024-05-22T21:20:35.010222+0000","last_became_active":"2024-05-22T21:20:33.991918+0000","last_became_peered":"2024-05-22T21:20:33.991918+0000","last_unstale":"2024-05-22T21:20:35.010222+0000","last_undegraded":"2024-05-22T21:20:35.010222+0000","last_fullsized":"2024-05-22T21:20:35.010222+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:14:08.304638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.980880+0000","last_change":"2024-05-22T21:20:33.986816+0000","last_active":"2024-05-22T21:20:34.980880+0000","last_peered":"2024-05-22T21:20:34.980880+0000","last_clean":"2024-05-22T21:20:34.980880+0000","last_became_active":"2024-05-22T21:20:33.986618+0000","last_became_peered":"2024-05-22T21:20:33.986618+0000","last_unstale":"2024-05-22T21:20:34.980880+0000","last_undegraded":"2024-05-22T21:20:34.980880+0000","last_fullsized":"2024-05-22T21:20:34.980880+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:15.376070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084507+0000","last_change":"2024-05-22T21:20:33.979997+0000","last_active":"2024-05-22T21:20:35.084507+0000","last_peered":"2024-05-22T21:20:35.084507+0000","last_clean":"2024-05-22T21:20:35.084507+0000","last_became_active":"2024-05-22T21:20:33.979792+0000","last_became_peered":"2024-05-22T21:20:33.979792+0000","last_unstale":"2024-05-22T21:20:35.084507+0000","last_undegraded":"2024-05-22T21:20:35.084507+0000","last_fullsized":"2024-05-22T21:20:35.084507+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:57:40.927650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.981082+0000","last_change":"2024-05-22T21:20:33.985176+0000","last_active":"2024-05-22T21:20:34.981082+0000","last_peered":"2024-05-22T21:20:34.981082+0000","last_clean":"2024-05-22T21:20:34.981082+0000","last_became_active":"2024-05-22T21:20:33.984964+0000","last_became_peered":"2024-05-22T21:20:33.984964+0000","last_unstale":"2024-05-22T21:20:34.981082+0000","last_undegraded":"2024-05-22T21:20:34.981082+0000","last_fullsized":"2024-05-22T21:20:34.981082+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:57:49.306839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181112+0000","last_change":"2024-05-22T21:20:33.981222+0000","last_active":"2024-05-22T21:20:35.181112+0000","last_peered":"2024-05-22T21:20:35.181112+0000","last_clean":"2024-05-22T21:20:35.181112+0000","last_became_active":"2024-05-22T21:20:33.980857+0000","last_became_peered":"2024-05-22T21:20:33.980857+0000","last_unstale":"2024-05-22T21:20:35.181112+0000","last_undegraded":"2024-05-22T21:20:35.181112+0000","last_fullsized":"2024-05-22T21:20:35.181112+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:01:11.747849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.983862+0000","last_change":"2024-05-22T21:20:33.991411+0000","last_active":"2024-05-22T21:20:34.983862+0000","last_peered":"2024-05-22T21:20:34.983862+0000","last_clean":"2024-05-22T21:20:34.983862+0000","last_became_active":"2024-05-22T21:20:33.991158+0000","last_became_peered":"2024-05-22T21:20:33.991158+0000","last_unstale":"2024-05-22T21:20:34.983862+0000","last_undegraded":"2024-05-22T21:20:34.983862+0000","last_fullsized":"2024-05-22T21:20:34.983862+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:47:35.142416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181175+0000","last_change":"2024-05-22T21:20:33.987504+0000","last_active":"2024-05-22T21:20:35.181175+0000","last_peered":"2024-05-22T21:20:35.181175+0000","last_clean":"2024-05-22T21:20:35.181175+0000","last_became_active":"2024-05-22T21:20:33.987169+0000","last_became_peered":"2024-05-22T21:20:33.987169+0000","last_unstale":"2024-05-22T21:20:35.181175+0000","last_undegraded":"2024-05-22T21:20:35.181175+0000","last_fullsized":"2024-05-22T21:20:35.181175+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:10:01.358237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181018+0000","last_change":"2024-05-22T21:20:33.986480+0000","last_active":"2024-05-22T21:20:35.181018+0000","last_peered":"2024-05-22T21:20:35.181018+0000","last_clean":"2024-05-22T21:20:35.181018+0000","last_became_active":"2024-05-22T21:20:33.986231+0000","last_became_peered":"2024-05-22T21:20:33.986231+0000","last_unstale":"2024-05-22T21:20:35.181018+0000","last_undegraded":"2024-05-22T21:20:35.181018+0000","last_fullsized":"2024-05-22T21:20:35.181018+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:46:49.859734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181433+0000","last_change":"2024-05-22T21:20:33.979538+0000","last_active":"2024-05-22T21:20:35.181433+0000","last_peered":"2024-05-22T21:20:35.181433+0000","last_clean":"2024-05-22T21:20:35.181433+0000","last_became_active":"2024-05-22T21:20:33.979318+0000","last_became_peered":"2024-05-22T21:20:33.979318+0000","last_unstale":"2024-05-22T21:20:35.181433+0000","last_undegraded":"2024-05-22T21:20:35.181433+0000","last_fullsized":"2024-05-22T21:20:35.181433+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:00:42.483473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984570+0000","last_change":"2024-05-22T21:20:33.985123+0000","last_active":"2024-05-22T21:20:34.984570+0000","last_peered":"2024-05-22T21:20:34.984570+0000","last_clean":"2024-05-22T21:20:34.984570+0000","last_became_active":"2024-05-22T21:20:33.984161+0000","last_became_peered":"2024-05-22T21:20:33.984161+0000","last_unstale":"2024-05-22T21:20:34.984570+0000","last_undegraded":"2024-05-22T21:20:34.984570+0000","last_fullsized":"2024-05-22T21:20:34.984570+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:49:28.392673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181323+0000","last_change":"2024-05-22T21:20:33.980318+0000","last_active":"2024-05-22T21:20:35.181323+0000","last_peered":"2024-05-22T21:20:35.181323+0000","last_clean":"2024-05-22T21:20:35.181323+0000","last_became_active":"2024-05-22T21:20:33.979682+0000","last_became_peered":"2024-05-22T21:20:33.979682+0000","last_unstale":"2024-05-22T21:20:35.181323+0000","last_undegraded":"2024-05-22T21:20:35.181323+0000","last_fullsized":"2024-05-22T21:20:35.181323+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:07:03.573164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010534+0000","last_change":"2024-05-22T21:20:33.989882+0000","last_active":"2024-05-22T21:20:35.010534+0000","last_peered":"2024-05-22T21:20:35.010534+0000","last_clean":"2024-05-22T21:20:35.010534+0000","last_became_active":"2024-05-22T21:20:33.988460+0000","last_became_peered":"2024-05-22T21:20:33.988460+0000","last_unstale":"2024-05-22T21:20:35.010534+0000","last_undegraded":"2024-05-22T21:20:35.010534+0000","last_fullsized":"2024-05-22T21:20:35.010534+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:13:10.680899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181106+0000","last_change":"2024-05-22T21:20:33.983548+0000","last_active":"2024-05-22T21:20:35.181106+0000","last_peered":"2024-05-22T21:20:35.181106+0000","last_clean":"2024-05-22T21:20:35.181106+0000","last_became_active":"2024-05-22T21:20:33.983299+0000","last_became_peered":"2024-05-22T21:20:33.983299+0000","last_unstale":"2024-05-22T21:20:35.181106+0000","last_undegraded":"2024-05-22T21:20:35.181106+0000","last_fullsized":"2024-05-22T21:20:35.181106+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:13:01.144330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084864+0000","last_change":"2024-05-22T21:20:33.985027+0000","last_active":"2024-05-22T21:20:35.084864+0000","last_peered":"2024-05-22T21:20:35.084864+0000","last_clean":"2024-05-22T21:20:35.084864+0000","last_became_active":"2024-05-22T21:20:33.984340+0000","last_became_peered":"2024-05-22T21:20:33.984340+0000","last_unstale":"2024-05-22T21:20:35.084864+0000","last_undegraded":"2024-05-22T21:20:35.084864+0000","last_fullsized":"2024-05-22T21:20:35.084864+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:00:38.509323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181163+0000","last_change":"2024-05-22T21:20:33.981091+0000","last_active":"2024-05-22T21:20:35.181163+0000","last_peered":"2024-05-22T21:20:35.181163+0000","last_clean":"2024-05-22T21:20:35.181163+0000","last_became_active":"2024-05-22T21:20:33.979598+0000","last_became_peered":"2024-05-22T21:20:33.979598+0000","last_unstale":"2024-05-22T21:20:35.181163+0000","last_undegraded":"2024-05-22T21:20:35.181163+0000","last_fullsized":"2024-05-22T21:20:35.181163+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:15:12.227118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.980762+0000","last_change":"2024-05-22T21:20:33.991797+0000","last_active":"2024-05-22T21:20:34.980762+0000","last_peered":"2024-05-22T21:20:34.980762+0000","last_clean":"2024-05-22T21:20:34.980762+0000","last_became_active":"2024-05-22T21:20:33.991580+0000","last_became_peered":"2024-05-22T21:20:33.991580+0000","last_unstale":"2024-05-22T21:20:34.980762+0000","last_undegraded":"2024-05-22T21:20:34.980762+0000","last_fullsized":"2024-05-22T21:20:34.980762+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:23:04.919505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.182107+0000","last_change":"2024-05-22T21:20:33.985691+0000","last_active":"2024-05-22T21:20:35.182107+0000","last_peered":"2024-05-22T21:20:35.182107+0000","last_clean":"2024-05-22T21:20:35.182107+0000","last_became_active":"2024-05-22T21:20:33.985094+0000","last_became_peered":"2024-05-22T21:20:33.985094+0000","last_unstale":"2024-05-22T21:20:35.182107+0000","last_undegraded":"2024-05-22T21:20:35.182107+0000","last_fullsized":"2024-05-22T21:20:35.182107+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:52:42.816671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.980938+0000","last_change":"2024-05-22T21:20:33.982490+0000","last_active":"2024-05-22T21:20:34.980938+0000","last_peered":"2024-05-22T21:20:34.980938+0000","last_clean":"2024-05-22T21:20:34.980938+0000","last_became_active":"2024-05-22T21:20:33.982208+0000","last_became_peered":"2024-05-22T21:20:33.982208+0000","last_unstale":"2024-05-22T21:20:34.980938+0000","last_undegraded":"2024-05-22T21:20:34.980938+0000","last_fullsized":"2024-05-22T21:20:34.980938+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:49:22.599564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.981178+0000","last_change":"2024-05-22T21:20:33.989217+0000","last_active":"2024-05-22T21:20:34.981178+0000","last_peered":"2024-05-22T21:20:34.981178+0000","last_clean":"2024-05-22T21:20:34.981178+0000","last_became_active":"2024-05-22T21:20:33.988809+0000","last_became_peered":"2024-05-22T21:20:33.988809+0000","last_unstale":"2024-05-22T21:20:34.981178+0000","last_undegraded":"2024-05-22T21:20:34.981178+0000","last_fullsized":"2024-05-22T21:20:34.981178+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:17:15.251407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984677+0000","last_change":"2024-05-22T21:20:33.985274+0000","last_active":"2024-05-22T21:20:34.984677+0000","last_peered":"2024-05-22T21:20:34.984677+0000","last_clean":"2024-05-22T21:20:34.984677+0000","last_became_active":"2024-05-22T21:20:33.984511+0000","last_became_peered":"2024-05-22T21:20:33.984511+0000","last_unstale":"2024-05-22T21:20:34.984677+0000","last_undegraded":"2024-05-22T21:20:34.984677+0000","last_fullsized":"2024-05-22T21:20:34.984677+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:56:13.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.983767+0000","last_change":"2024-05-22T21:20:33.989756+0000","last_active":"2024-05-22T21:20:34.983767+0000","last_peered":"2024-05-22T21:20:34.983767+0000","last_clean":"2024-05-22T21:20:34.983767+0000","last_became_active":"2024-05-22T21:20:33.988822+0000","last_became_peered":"2024-05-22T21:20:33.988822+0000","last_unstale":"2024-05-22T21:20:34.983767+0000","last_undegraded":"2024-05-22T21:20:34.983767+0000","last_fullsized":"2024-05-22T21:20:34.983767+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:39:43.907759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181055+0000","last_change":"2024-05-22T21:20:33.984066+0000","last_active":"2024-05-22T21:20:35.181055+0000","last_peered":"2024-05-22T21:20:35.181055+0000","last_clean":"2024-05-22T21:20:35.181055+0000","last_became_active":"2024-05-22T21:20:33.983845+0000","last_became_peered":"2024-05-22T21:20:33.983845+0000","last_unstale":"2024-05-22T21:20:35.181055+0000","last_undegraded":"2024-05-22T21:20:35.181055+0000","last_fullsized":"2024-05-22T21:20:35.181055+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:37:48.073294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"157'4","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984283+0000","last_change":"2024-05-22T21:20:33.592688+0000","last_active":"2024-05-22T21:20:34.984283+0000","last_peered":"2024-05-22T21:20:34.984283+0000","last_clean":"2024-05-22T21:20:34.984283+0000","last_became_active":"2024-05-22T21:20:33.592449+0000","last_became_peered":"2024-05-22T21:20:33.592449+0000","last_unstale":"2024-05-22T21:20:34.984283+0000","last_undegraded":"2024-05-22T21:20:34.984283+0000","last_fullsized":"2024-05-22T21:20:34.984283+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:33:04.990291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010313+0000","last_change":"2024-05-22T21:20:33.992717+0000","last_active":"2024-05-22T21:20:35.010313+0000","last_peered":"2024-05-22T21:20:35.010313+0000","last_clean":"2024-05-22T21:20:35.010313+0000","last_became_active":"2024-05-22T21:20:33.989166+0000","last_became_peered":"2024-05-22T21:20:33.989166+0000","last_unstale":"2024-05-22T21:20:35.010313+0000","last_undegraded":"2024-05-22T21:20:35.010313+0000","last_fullsized":"2024-05-22T21:20:35.010313+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:36:01.329591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"157'1","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.182148+0000","last_change":"2024-05-22T21:20:33.592953+0000","last_active":"2024-05-22T21:20:35.182148+0000","last_peered":"2024-05-22T21:20:35.182148+0000","last_clean":"2024-05-22T21:20:35.182148+0000","last_became_active":"2024-05-22T21:20:33.590822+0000","last_became_peered":"2024-05-22T21:20:33.590822+0000","last_unstale":"2024-05-22T21:20:35.182148+0000","last_undegraded":"2024-05-22T21:20:35.182148+0000","last_fullsized":"2024-05-22T21:20:35.182148+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:53:59.333062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984778+0000","last_change":"2024-05-22T21:20:33.990586+0000","last_active":"2024-05-22T21:20:34.984778+0000","last_peered":"2024-05-22T21:20:34.984778+0000","last_clean":"2024-05-22T21:20:34.984778+0000","last_became_active":"2024-05-22T21:20:33.990357+0000","last_became_peered":"2024-05-22T21:20:33.990357+0000","last_unstale":"2024-05-22T21:20:34.984778+0000","last_undegraded":"2024-05-22T21:20:34.984778+0000","last_fullsized":"2024-05-22T21:20:34.984778+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:01:11.747849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"157'2","reported_seq":13,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181392+0000","last_change":"2024-05-22T21:20:33.584011+0000","last_active":"2024-05-22T21:20:35.181392+0000","last_peered":"2024-05-22T21:20:35.181392+0000","last_clean":"2024-05-22T21:20:35.181392+0000","last_became_active":"2024-05-22T21:20:33.583750+0000","last_became_peered":"2024-05-22T21:20:33.583750+0000","last_unstale":"2024-05-22T21:20:35.181392+0000","last_undegraded":"2024-05-22T21:20:35.181392+0000","last_fullsized":"2024-05-22T21:20:35.181392+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:17:12.215032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.985033+0000","last_change":"2024-05-22T21:20:33.986461+0000","last_active":"2024-05-22T21:20:34.985033+0000","last_peered":"2024-05-22T21:20:34.985033+0000","last_clean":"2024-05-22T21:20:34.985033+0000","last_became_active":"2024-05-22T21:20:33.986202+0000","last_became_peered":"2024-05-22T21:20:33.986202+0000","last_unstale":"2024-05-22T21:20:34.985033+0000","last_undegraded":"2024-05-22T21:20:34.985033+0000","last_fullsized":"2024-05-22T21:20:34.985033+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:10:01.358237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"157'2","reported_seq":13,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084665+0000","last_change":"2024-05-22T21:20:33.589018+0000","last_active":"2024-05-22T21:20:35.084665+0000","last_peered":"2024-05-22T21:20:35.084665+0000","last_clean":"2024-05-22T21:20:35.084665+0000","last_became_active":"2024-05-22T21:20:33.588811+0000","last_became_peered":"2024-05-22T21:20:33.588811+0000","last_unstale":"2024-05-22T21:20:35.084665+0000","last_undegraded":"2024-05-22T21:20:35.084665+0000","last_fullsized":"2024-05-22T21:20:35.084665+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:22:10.147717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181261+0000","last_change":"2024-05-22T21:20:33.987370+0000","last_active":"2024-05-22T21:20:35.181261+0000","last_peered":"2024-05-22T21:20:35.181261+0000","last_clean":"2024-05-22T21:20:35.181261+0000","last_became_active":"2024-05-22T21:20:33.986840+0000","last_became_peered":"2024-05-22T21:20:33.986840+0000","last_unstale":"2024-05-22T21:20:35.181261+0000","last_undegraded":"2024-05-22T21:20:35.181261+0000","last_fullsized":"2024-05-22T21:20:35.181261+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:53:21.407922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"157'1","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084689+0000","last_change":"2024-05-22T21:20:33.589728+0000","last_active":"2024-05-22T21:20:35.084689+0000","last_peered":"2024-05-22T21:20:35.084689+0000","last_clean":"2024-05-22T21:20:35.084689+0000","last_became_active":"2024-05-22T21:20:33.589488+0000","last_became_peered":"2024-05-22T21:20:33.589488+0000","last_unstale":"2024-05-22T21:20:35.084689+0000","last_undegraded":"2024-05-22T21:20:35.084689+0000","last_fullsized":"2024-05-22T21:20:35.084689+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:14.369689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.983966+0000","last_change":"2024-05-22T21:20:33.990896+0000","last_active":"2024-05-22T21:20:34.983966+0000","last_peered":"2024-05-22T21:20:34.983966+0000","last_clean":"2024-05-22T21:20:34.983966+0000","last_became_active":"2024-05-22T21:20:33.989550+0000","last_became_peered":"2024-05-22T21:20:33.989550+0000","last_unstale":"2024-05-22T21:20:34.983966+0000","last_undegraded":"2024-05-22T21:20:34.983966+0000","last_fullsized":"2024-05-22T21:20:34.983966+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:55:50.256135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"157'1","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181339+0000","last_change":"2024-05-22T21:20:33.586334+0000","last_active":"2024-05-22T21:20:35.181339+0000","last_peered":"2024-05-22T21:20:35.181339+0000","last_clean":"2024-05-22T21:20:35.181339+0000","last_became_active":"2024-05-22T21:20:33.585375+0000","last_became_peered":"2024-05-22T21:20:33.585375+0000","last_unstale":"2024-05-22T21:20:35.181339+0000","last_undegraded":"2024-05-22T21:20:35.181339+0000","last_fullsized":"2024-05-22T21:20:35.181339+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:13:09.674518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010241+0000","last_change":"2024-05-22T21:20:33.988721+0000","last_active":"2024-05-22T21:20:35.010241+0000","last_peered":"2024-05-22T21:20:35.010241+0000","last_clean":"2024-05-22T21:20:35.010241+0000","last_became_active":"2024-05-22T21:20:33.987610+0000","last_became_peered":"2024-05-22T21:20:33.987610+0000","last_unstale":"2024-05-22T21:20:35.010241+0000","last_undegraded":"2024-05-22T21:20:35.010241+0000","last_fullsized":"2024-05-22T21:20:35.010241+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:46:01.370439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.983757+0000","last_change":"2024-05-22T21:20:33.591934+0000","last_active":"2024-05-22T21:20:34.983757+0000","last_peered":"2024-05-22T21:20:34.983757+0000","last_clean":"2024-05-22T21:20:34.983757+0000","last_became_active":"2024-05-22T21:20:33.591250+0000","last_became_peered":"2024-05-22T21:20:33.591250+0000","last_unstale":"2024-05-22T21:20:34.983757+0000","last_undegraded":"2024-05-22T21:20:34.983757+0000","last_fullsized":"2024-05-22T21:20:34.983757+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:43:56.234498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084616+0000","last_change":"2024-05-22T21:20:33.977727+0000","last_active":"2024-05-22T21:20:35.084616+0000","last_peered":"2024-05-22T21:20:35.084616+0000","last_clean":"2024-05-22T21:20:35.084616+0000","last_became_active":"2024-05-22T21:20:33.977496+0000","last_became_peered":"2024-05-22T21:20:33.977496+0000","last_unstale":"2024-05-22T21:20:35.084616+0000","last_undegraded":"2024-05-22T21:20:35.084616+0000","last_fullsized":"2024-05-22T21:20:35.084616+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:10:32.784200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084941+0000","last_change":"2024-05-22T21:20:33.591209+0000","last_active":"2024-05-22T21:20:35.084941+0000","last_peered":"2024-05-22T21:20:35.084941+0000","last_clean":"2024-05-22T21:20:35.084941+0000","last_became_active":"2024-05-22T21:20:33.590997+0000","last_became_peered":"2024-05-22T21:20:33.590997+0000","last_unstale":"2024-05-22T21:20:35.084941+0000","last_undegraded":"2024-05-22T21:20:35.084941+0000","last_fullsized":"2024-05-22T21:20:35.084941+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:05:58.495735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.980943+0000","last_change":"2024-05-22T21:20:33.981952+0000","last_active":"2024-05-22T21:20:34.980943+0000","last_peered":"2024-05-22T21:20:34.980943+0000","last_clean":"2024-05-22T21:20:34.980943+0000","last_became_active":"2024-05-22T21:20:33.981749+0000","last_became_peered":"2024-05-22T21:20:33.981749+0000","last_unstale":"2024-05-22T21:20:34.980943+0000","last_undegraded":"2024-05-22T21:20:34.980943+0000","last_fullsized":"2024-05-22T21:20:34.980943+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:32.511970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.182142+0000","last_change":"2024-05-22T21:20:33.593354+0000","last_active":"2024-05-22T21:20:35.182142+0000","last_peered":"2024-05-22T21:20:35.182142+0000","last_clean":"2024-05-22T21:20:35.182142+0000","last_became_active":"2024-05-22T21:20:33.592907+0000","last_became_peered":"2024-05-22T21:20:33.592907+0000","last_unstale":"2024-05-22T21:20:35.182142+0000","last_undegraded":"2024-05-22T21:20:35.182142+0000","last_fullsized":"2024-05-22T21:20:35.182142+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:47:05.722164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181988+0000","last_change":"2024-05-22T21:20:33.985266+0000","last_active":"2024-05-22T21:20:35.181988+0000","last_peered":"2024-05-22T21:20:35.181988+0000","last_clean":"2024-05-22T21:20:35.181988+0000","last_became_active":"2024-05-22T21:20:33.985005+0000","last_became_peered":"2024-05-22T21:20:33.985005+0000","last_unstale":"2024-05-22T21:20:35.181988+0000","last_undegraded":"2024-05-22T21:20:35.181988+0000","last_fullsized":"2024-05-22T21:20:35.181988+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:57:40.927650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181652+0000","last_change":"2024-05-22T21:20:33.595367+0000","last_active":"2024-05-22T21:20:35.181652+0000","last_peered":"2024-05-22T21:20:35.181652+0000","last_clean":"2024-05-22T21:20:35.181652+0000","last_became_active":"2024-05-22T21:20:33.595132+0000","last_became_peered":"2024-05-22T21:20:33.595132+0000","last_unstale":"2024-05-22T21:20:35.181652+0000","last_undegraded":"2024-05-22T21:20:35.181652+0000","last_fullsized":"2024-05-22T21:20:35.181652+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:14:07.298257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984223+0000","last_change":"2024-05-22T21:20:33.978008+0000","last_active":"2024-05-22T21:20:34.984223+0000","last_peered":"2024-05-22T21:20:34.984223+0000","last_clean":"2024-05-22T21:20:34.984223+0000","last_became_active":"2024-05-22T21:20:33.977772+0000","last_became_peered":"2024-05-22T21:20:33.977772+0000","last_unstale":"2024-05-22T21:20:34.984223+0000","last_undegraded":"2024-05-22T21:20:34.984223+0000","last_fullsized":"2024-05-22T21:20:34.984223+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:15:12.227118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"157'3","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.182214+0000","last_change":"2024-05-22T21:20:33.589624+0000","last_active":"2024-05-22T21:20:35.182214+0000","last_peered":"2024-05-22T21:20:35.182214+0000","last_clean":"2024-05-22T21:20:35.182214+0000","last_became_active":"2024-05-22T21:20:33.589049+0000","last_became_peered":"2024-05-22T21:20:33.589049+0000","last_unstale":"2024-05-22T21:20:35.182214+0000","last_undegraded":"2024-05-22T21:20:35.182214+0000","last_fullsized":"2024-05-22T21:20:35.182214+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:57:48.300459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984190+0000","last_change":"2024-05-22T21:20:33.989493+0000","last_active":"2024-05-22T21:20:34.984190+0000","last_peered":"2024-05-22T21:20:34.984190+0000","last_clean":"2024-05-22T21:20:34.984190+0000","last_became_active":"2024-05-22T21:20:33.989267+0000","last_became_peered":"2024-05-22T21:20:33.989267+0000","last_unstale":"2024-05-22T21:20:34.984190+0000","last_undegraded":"2024-05-22T21:20:34.984190+0000","last_fullsized":"2024-05-22T21:20:34.984190+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:14:32.778352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"157'6","reported_seq":17,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010411+0000","last_change":"2024-05-22T21:20:33.597863+0000","last_active":"2024-05-22T21:20:35.010411+0000","last_peered":"2024-05-22T21:20:35.010411+0000","last_clean":"2024-05-22T21:20:35.010411+0000","last_became_active":"2024-05-22T21:20:33.597573+0000","last_became_peered":"2024-05-22T21:20:33.597573+0000","last_unstale":"2024-05-22T21:20:35.010411+0000","last_undegraded":"2024-05-22T21:20:35.010411+0000","last_fullsized":"2024-05-22T21:20:35.010411+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:33:03.240282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.981320+0000","last_change":"2024-05-22T21:20:33.986706+0000","last_active":"2024-05-22T21:20:34.981320+0000","last_peered":"2024-05-22T21:20:34.981320+0000","last_clean":"2024-05-22T21:20:34.981320+0000","last_became_active":"2024-05-22T21:20:33.986498+0000","last_became_peered":"2024-05-22T21:20:33.986498+0000","last_unstale":"2024-05-22T21:20:34.981320+0000","last_undegraded":"2024-05-22T21:20:34.981320+0000","last_fullsized":"2024-05-22T21:20:34.981320+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:35:17.584421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.981166+0000","last_change":"2024-05-22T21:20:33.592145+0000","last_active":"2024-05-22T21:20:34.981166+0000","last_peered":"2024-05-22T21:20:34.981166+0000","last_clean":"2024-05-22T21:20:34.981166+0000","last_became_active":"2024-05-22T21:20:33.591759+0000","last_became_peered":"2024-05-22T21:20:33.591759+0000","last_unstale":"2024-05-22T21:20:34.981166+0000","last_undegraded":"2024-05-22T21:20:34.981166+0000","last_fullsized":"2024-05-22T21:20:34.981166+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:37:49.268832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084507+0000","last_change":"2024-05-22T21:20:33.985312+0000","last_active":"2024-05-22T21:20:35.084507+0000","last_peered":"2024-05-22T21:20:35.084507+0000","last_clean":"2024-05-22T21:20:35.084507+0000","last_became_active":"2024-05-22T21:20:33.985063+0000","last_became_peered":"2024-05-22T21:20:33.985063+0000","last_unstale":"2024-05-22T21:20:35.084507+0000","last_undegraded":"2024-05-22T21:20:35.084507+0000","last_fullsized":"2024-05-22T21:20:35.084507+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:52:42.816671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084774+0000","last_change":"2024-05-22T21:20:33.592526+0000","last_active":"2024-05-22T21:20:35.084774+0000","last_peered":"2024-05-22T21:20:35.084774+0000","last_clean":"2024-05-22T21:20:35.084774+0000","last_became_active":"2024-05-22T21:20:33.592295+0000","last_became_peered":"2024-05-22T21:20:33.592295+0000","last_unstale":"2024-05-22T21:20:35.084774+0000","last_undegraded":"2024-05-22T21:20:35.084774+0000","last_fullsized":"2024-05-22T21:20:35.084774+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:35:16.578041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010304+0000","last_change":"2024-05-22T21:20:33.989472+0000","last_active":"2024-05-22T21:20:35.010304+0000","last_peered":"2024-05-22T21:20:35.010304+0000","last_clean":"2024-05-22T21:20:35.010304+0000","last_became_active":"2024-05-22T21:20:33.989263+0000","last_became_peered":"2024-05-22T21:20:33.989263+0000","last_unstale":"2024-05-22T21:20:35.010304+0000","last_undegraded":"2024-05-22T21:20:35.010304+0000","last_fullsized":"2024-05-22T21:20:35.010304+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:47:35.142416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"157'8","reported_seq":19,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010427+0000","last_change":"2024-05-22T21:20:33.596302+0000","last_active":"2024-05-22T21:20:35.010427+0000","last_peered":"2024-05-22T21:20:35.010427+0000","last_clean":"2024-05-22T21:20:35.010427+0000","last_became_active":"2024-05-22T21:20:33.596064+0000","last_became_peered":"2024-05-22T21:20:33.596064+0000","last_unstale":"2024-05-22T21:20:35.010427+0000","last_undegraded":"2024-05-22T21:20:35.010427+0000","last_fullsized":"2024-05-22T21:20:35.010427+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:43:56.234498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010340+0000","last_change":"2024-05-22T21:20:33.988060+0000","last_active":"2024-05-22T21:20:35.010340+0000","last_peered":"2024-05-22T21:20:35.010340+0000","last_clean":"2024-05-22T21:20:35.010340+0000","last_became_active":"2024-05-22T21:20:33.987266+0000","last_became_peered":"2024-05-22T21:20:33.987266+0000","last_unstale":"2024-05-22T21:20:35.010340+0000","last_undegraded":"2024-05-22T21:20:35.010340+0000","last_fullsized":"2024-05-22T21:20:35.010340+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:56:13.371399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"157'1","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.182211+0000","last_change":"2024-05-22T21:20:33.593172+0000","last_active":"2024-05-22T21:20:35.182211+0000","last_peered":"2024-05-22T21:20:35.182211+0000","last_clean":"2024-05-22T21:20:35.182211+0000","last_became_active":"2024-05-22T21:20:33.592638+0000","last_became_peered":"2024-05-22T21:20:33.592638+0000","last_unstale":"2024-05-22T21:20:35.182211+0000","last_undegraded":"2024-05-22T21:20:35.182211+0000","last_fullsized":"2024-05-22T21:20:35.182211+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:23:27.892841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181959+0000","last_change":"2024-05-22T21:20:33.991500+0000","last_active":"2024-05-22T21:20:35.181959+0000","last_peered":"2024-05-22T21:20:35.181959+0000","last_clean":"2024-05-22T21:20:35.181959+0000","last_became_active":"2024-05-22T21:20:33.988446+0000","last_became_peered":"2024-05-22T21:20:33.988446+0000","last_unstale":"2024-05-22T21:20:35.181959+0000","last_undegraded":"2024-05-22T21:20:35.181959+0000","last_fullsized":"2024-05-22T21:20:35.181959+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:47:58.618118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"157'2","reported_seq":13,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181477+0000","last_change":"2024-05-22T21:20:33.595497+0000","last_active":"2024-05-22T21:20:35.181477+0000","last_peered":"2024-05-22T21:20:35.181477+0000","last_clean":"2024-05-22T21:20:35.181477+0000","last_became_active":"2024-05-22T21:20:33.595246+0000","last_became_peered":"2024-05-22T21:20:33.595246+0000","last_unstale":"2024-05-22T21:20:35.181477+0000","last_undegraded":"2024-05-22T21:20:35.181477+0000","last_fullsized":"2024-05-22T21:20:35.181477+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:20:35.100019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181605+0000","last_change":"2024-05-22T21:20:33.980190+0000","last_active":"2024-05-22T21:20:35.181605+0000","last_peered":"2024-05-22T21:20:35.181605+0000","last_clean":"2024-05-22T21:20:35.181605+0000","last_became_active":"2024-05-22T21:20:33.979921+0000","last_became_peered":"2024-05-22T21:20:33.979921+0000","last_unstale":"2024-05-22T21:20:35.181605+0000","last_undegraded":"2024-05-22T21:20:35.181605+0000","last_fullsized":"2024-05-22T21:20:35.181605+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:17:13.221413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"157'1","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984350+0000","last_change":"2024-05-22T21:20:33.592852+0000","last_active":"2024-05-22T21:20:34.984350+0000","last_peered":"2024-05-22T21:20:34.984350+0000","last_clean":"2024-05-22T21:20:34.984350+0000","last_became_active":"2024-05-22T21:20:33.592586+0000","last_became_peered":"2024-05-22T21:20:33.592586+0000","last_unstale":"2024-05-22T21:20:34.984350+0000","last_undegraded":"2024-05-22T21:20:34.984350+0000","last_fullsized":"2024-05-22T21:20:34.984350+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:33:03.240282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084734+0000","last_change":"2024-05-22T21:20:33.977590+0000","last_active":"2024-05-22T21:20:35.084734+0000","last_peered":"2024-05-22T21:20:35.084734+0000","last_clean":"2024-05-22T21:20:35.084734+0000","last_became_active":"2024-05-22T21:20:33.977363+0000","last_became_peered":"2024-05-22T21:20:33.977363+0000","last_unstale":"2024-05-22T21:20:35.084734+0000","last_undegraded":"2024-05-22T21:20:35.084734+0000","last_fullsized":"2024-05-22T21:20:35.084734+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:49:03.076890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.182083+0000","last_change":"2024-05-22T21:20:33.590581+0000","last_active":"2024-05-22T21:20:35.182083+0000","last_peered":"2024-05-22T21:20:35.182083+0000","last_clean":"2024-05-22T21:20:35.182083+0000","last_became_active":"2024-05-22T21:20:33.590354+0000","last_became_peered":"2024-05-22T21:20:33.590354+0000","last_unstale":"2024-05-22T21:20:35.182083+0000","last_undegraded":"2024-05-22T21:20:35.182083+0000","last_fullsized":"2024-05-22T21:20:35.182083+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:06:22.448002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.981029+0000","last_change":"2024-05-22T21:20:33.986252+0000","last_active":"2024-05-22T21:20:34.981029+0000","last_peered":"2024-05-22T21:20:34.981029+0000","last_clean":"2024-05-22T21:20:34.981029+0000","last_became_active":"2024-05-22T21:20:33.985637+0000","last_became_peered":"2024-05-22T21:20:33.985637+0000","last_unstale":"2024-05-22T21:20:34.981029+0000","last_undegraded":"2024-05-22T21:20:34.981029+0000","last_fullsized":"2024-05-22T21:20:34.981029+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:22:11.154098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"157'2","reported_seq":13,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.182306+0000","last_change":"2024-05-22T21:20:33.591607+0000","last_active":"2024-05-22T21:20:35.182306+0000","last_peered":"2024-05-22T21:20:35.182306+0000","last_clean":"2024-05-22T21:20:35.182306+0000","last_became_active":"2024-05-22T21:20:33.589408+0000","last_became_peered":"2024-05-22T21:20:33.589408+0000","last_unstale":"2024-05-22T21:20:35.182306+0000","last_undegraded":"2024-05-22T21:20:35.182306+0000","last_fullsized":"2024-05-22T21:20:35.182306+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:17:14.245027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.983574+0000","last_change":"2024-05-22T21:20:33.994103+0000","last_active":"2024-05-22T21:20:34.983574+0000","last_peered":"2024-05-22T21:20:34.983574+0000","last_clean":"2024-05-22T21:20:34.983574+0000","last_became_active":"2024-05-22T21:20:33.993145+0000","last_became_peered":"2024-05-22T21:20:33.993145+0000","last_unstale":"2024-05-22T21:20:34.983574+0000","last_undegraded":"2024-05-22T21:20:34.983574+0000","last_fullsized":"2024-05-22T21:20:34.983574+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:14:08.304638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181232+0000","last_change":"2024-05-22T21:20:33.589913+0000","last_active":"2024-05-22T21:20:35.181232+0000","last_peered":"2024-05-22T21:20:35.181232+0000","last_clean":"2024-05-22T21:20:35.181232+0000","last_became_active":"2024-05-22T21:20:33.589674+0000","last_became_peered":"2024-05-22T21:20:33.589674+0000","last_unstale":"2024-05-22T21:20:35.181232+0000","last_undegraded":"2024-05-22T21:20:35.181232+0000","last_fullsized":"2024-05-22T21:20:35.181232+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:47:34.136035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084572+0000","last_change":"2024-05-22T21:20:33.978990+0000","last_active":"2024-05-22T21:20:35.084572+0000","last_peered":"2024-05-22T21:20:35.084572+0000","last_clean":"2024-05-22T21:20:35.084572+0000","last_became_active":"2024-05-22T21:20:33.978613+0000","last_became_peered":"2024-05-22T21:20:33.978613+0000","last_unstale":"2024-05-22T21:20:35.084572+0000","last_undegraded":"2024-05-22T21:20:35.084572+0000","last_fullsized":"2024-05-22T21:20:35.084572+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:47:58.618118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181600+0000","last_change":"2024-05-22T21:20:33.586119+0000","last_active":"2024-05-22T21:20:35.181600+0000","last_peered":"2024-05-22T21:20:35.181600+0000","last_clean":"2024-05-22T21:20:35.181600+0000","last_became_active":"2024-05-22T21:20:33.585883+0000","last_became_peered":"2024-05-22T21:20:33.585883+0000","last_unstale":"2024-05-22T21:20:35.181600+0000","last_undegraded":"2024-05-22T21:20:35.181600+0000","last_fullsized":"2024-05-22T21:20:35.181600+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:33:04.990291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984862+0000","last_change":"2024-05-22T21:20:33.982986+0000","last_active":"2024-05-22T21:20:34.984862+0000","last_peered":"2024-05-22T21:20:34.984862+0000","last_clean":"2024-05-22T21:20:34.984862+0000","last_became_active":"2024-05-22T21:20:33.982716+0000","last_became_peered":"2024-05-22T21:20:33.982716+0000","last_unstale":"2024-05-22T21:20:34.984862+0000","last_undegraded":"2024-05-22T21:20:34.984862+0000","last_fullsized":"2024-05-22T21:20:34.984862+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:20:36.106400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181765+0000","last_change":"2024-05-22T21:20:33.595338+0000","last_active":"2024-05-22T21:20:35.181765+0000","last_peered":"2024-05-22T21:20:35.181765+0000","last_clean":"2024-05-22T21:20:35.181765+0000","last_became_active":"2024-05-22T21:20:33.595081+0000","last_became_peered":"2024-05-22T21:20:33.595081+0000","last_unstale":"2024-05-22T21:20:35.181765+0000","last_undegraded":"2024-05-22T21:20:35.181765+0000","last_fullsized":"2024-05-22T21:20:35.181765+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:56:12.154973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.983683+0000","last_change":"2024-05-22T21:20:33.996909+0000","last_active":"2024-05-22T21:20:34.983683+0000","last_peered":"2024-05-22T21:20:34.983683+0000","last_clean":"2024-05-22T21:20:34.983683+0000","last_became_active":"2024-05-22T21:20:33.993773+0000","last_became_peered":"2024-05-22T21:20:33.993773+0000","last_unstale":"2024-05-22T21:20:34.983683+0000","last_undegraded":"2024-05-22T21:20:34.983683+0000","last_fullsized":"2024-05-22T21:20:34.983683+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:36:01.329591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"157'2","reported_seq":13,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.981132+0000","last_change":"2024-05-22T21:20:33.585549+0000","last_active":"2024-05-22T21:20:34.981132+0000","last_peered":"2024-05-22T21:20:34.981132+0000","last_clean":"2024-05-22T21:20:34.981132+0000","last_became_active":"2024-05-22T21:20:33.585324+0000","last_became_peered":"2024-05-22T21:20:33.585324+0000","last_unstale":"2024-05-22T21:20:34.981132+0000","last_undegraded":"2024-05-22T21:20:34.981132+0000","last_fullsized":"2024-05-22T21:20:34.981132+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:07:44.126970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.981028+0000","last_change":"2024-05-22T21:20:33.984622+0000","last_active":"2024-05-22T21:20:34.981028+0000","last_peered":"2024-05-22T21:20:34.981028+0000","last_clean":"2024-05-22T21:20:34.981028+0000","last_became_active":"2024-05-22T21:20:33.984396+0000","last_became_peered":"2024-05-22T21:20:33.984396+0000","last_unstale":"2024-05-22T21:20:34.981028+0000","last_undegraded":"2024-05-22T21:20:34.981028+0000","last_fullsized":"2024-05-22T21:20:34.981028+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:22:01.171038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984374+0000","last_change":"2024-05-22T21:20:33.589346+0000","last_active":"2024-05-22T21:20:34.984374+0000","last_peered":"2024-05-22T21:20:34.984374+0000","last_clean":"2024-05-22T21:20:34.984374+0000","last_became_active":"2024-05-22T21:20:33.589020+0000","last_became_peered":"2024-05-22T21:20:33.589020+0000","last_unstale":"2024-05-22T21:20:34.984374+0000","last_undegraded":"2024-05-22T21:20:34.984374+0000","last_fullsized":"2024-05-22T21:20:34.984374+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:14:07.298257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084905+0000","last_change":"2024-05-22T21:20:33.985160+0000","last_active":"2024-05-22T21:20:35.084905+0000","last_peered":"2024-05-22T21:20:35.084905+0000","last_clean":"2024-05-22T21:20:35.084905+0000","last_became_active":"2024-05-22T21:20:33.984696+0000","last_became_peered":"2024-05-22T21:20:33.984696+0000","last_unstale":"2024-05-22T21:20:35.084905+0000","last_undegraded":"2024-05-22T21:20:35.084905+0000","last_fullsized":"2024-05-22T21:20:35.084905+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:43:03.172403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084714+0000","last_change":"2024-05-22T21:20:33.593120+0000","last_active":"2024-05-22T21:20:35.084714+0000","last_peered":"2024-05-22T21:20:35.084714+0000","last_clean":"2024-05-22T21:20:35.084714+0000","last_became_active":"2024-05-22T21:20:33.592899+0000","last_became_peered":"2024-05-22T21:20:33.592899+0000","last_unstale":"2024-05-22T21:20:35.084714+0000","last_undegraded":"2024-05-22T21:20:35.084714+0000","last_fullsized":"2024-05-22T21:20:35.084714+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:08:03.810592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.983657+0000","last_change":"2024-05-22T21:20:33.984699+0000","last_active":"2024-05-22T21:20:34.983657+0000","last_peered":"2024-05-22T21:20:34.983657+0000","last_clean":"2024-05-22T21:20:34.983657+0000","last_became_active":"2024-05-22T21:20:33.984347+0000","last_became_peered":"2024-05-22T21:20:33.984347+0000","last_unstale":"2024-05-22T21:20:34.983657+0000","last_undegraded":"2024-05-22T21:20:34.983657+0000","last_fullsized":"2024-05-22T21:20:34.983657+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:33:05.996672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181575+0000","last_change":"2024-05-22T21:20:33.586878+0000","last_active":"2024-05-22T21:20:35.181575+0000","last_peered":"2024-05-22T21:20:35.181575+0000","last_clean":"2024-05-22T21:20:35.181575+0000","last_became_active":"2024-05-22T21:20:33.586618+0000","last_became_peered":"2024-05-22T21:20:33.586618+0000","last_unstale":"2024-05-22T21:20:35.181575+0000","last_undegraded":"2024-05-22T21:20:35.181575+0000","last_fullsized":"2024-05-22T21:20:35.181575+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:53:20.401541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984442+0000","last_change":"2024-05-22T21:20:33.991695+0000","last_active":"2024-05-22T21:20:34.984442+0000","last_peered":"2024-05-22T21:20:34.984442+0000","last_clean":"2024-05-22T21:20:34.984442+0000","last_became_active":"2024-05-22T21:20:33.991406+0000","last_became_peered":"2024-05-22T21:20:33.991406+0000","last_unstale":"2024-05-22T21:20:34.984442+0000","last_undegraded":"2024-05-22T21:20:34.984442+0000","last_fullsized":"2024-05-22T21:20:34.984442+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:40:50.727562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084792+0000","last_change":"2024-05-22T21:20:33.594743+0000","last_active":"2024-05-22T21:20:35.084792+0000","last_peered":"2024-05-22T21:20:35.084792+0000","last_clean":"2024-05-22T21:20:35.084792+0000","last_became_active":"2024-05-22T21:20:33.594509+0000","last_became_peered":"2024-05-22T21:20:33.594509+0000","last_unstale":"2024-05-22T21:20:35.084792+0000","last_undegraded":"2024-05-22T21:20:35.084792+0000","last_fullsized":"2024-05-22T21:20:35.084792+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:31.505589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.182084+0000","last_change":"2024-05-22T21:20:33.988470+0000","last_active":"2024-05-22T21:20:35.182084+0000","last_peered":"2024-05-22T21:20:35.182084+0000","last_clean":"2024-05-22T21:20:35.182084+0000","last_became_active":"2024-05-22T21:20:33.988144+0000","last_became_peered":"2024-05-22T21:20:33.988144+0000","last_unstale":"2024-05-22T21:20:35.182084+0000","last_undegraded":"2024-05-22T21:20:35.182084+0000","last_fullsized":"2024-05-22T21:20:35.182084+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:10:32.784200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984598+0000","last_change":"2024-05-22T21:20:33.591896+0000","last_active":"2024-05-22T21:20:34.984598+0000","last_peered":"2024-05-22T21:20:34.984598+0000","last_clean":"2024-05-22T21:20:34.984598+0000","last_became_active":"2024-05-22T21:20:33.589730+0000","last_became_peered":"2024-05-22T21:20:33.589730+0000","last_unstale":"2024-05-22T21:20:34.984598+0000","last_undegraded":"2024-05-22T21:20:34.984598+0000","last_fullsized":"2024-05-22T21:20:34.984598+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:36:00.323210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181149+0000","last_change":"2024-05-22T21:20:33.981881+0000","last_active":"2024-05-22T21:20:35.181149+0000","last_peered":"2024-05-22T21:20:35.181149+0000","last_clean":"2024-05-22T21:20:35.181149+0000","last_became_active":"2024-05-22T21:20:33.981670+0000","last_became_peered":"2024-05-22T21:20:33.981670+0000","last_unstale":"2024-05-22T21:20:35.181149+0000","last_undegraded":"2024-05-22T21:20:35.181149+0000","last_fullsized":"2024-05-22T21:20:35.181149+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:40:50.727562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181412+0000","last_change":"2024-05-22T21:20:33.595495+0000","last_active":"2024-05-22T21:20:35.181412+0000","last_peered":"2024-05-22T21:20:35.181412+0000","last_clean":"2024-05-22T21:20:35.181412+0000","last_became_active":"2024-05-22T21:20:33.595246+0000","last_became_peered":"2024-05-22T21:20:33.595246+0000","last_unstale":"2024-05-22T21:20:35.181412+0000","last_undegraded":"2024-05-22T21:20:35.181412+0000","last_fullsized":"2024-05-22T21:20:35.181412+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:36:00.323210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084619+0000","last_change":"2024-05-22T21:20:33.978228+0000","last_active":"2024-05-22T21:20:35.084619+0000","last_peered":"2024-05-22T21:20:35.084619+0000","last_clean":"2024-05-22T21:20:35.084619+0000","last_became_active":"2024-05-22T21:20:33.977454+0000","last_became_peered":"2024-05-22T21:20:33.977454+0000","last_unstale":"2024-05-22T21:20:35.084619+0000","last_undegraded":"2024-05-22T21:20:35.084619+0000","last_fullsized":"2024-05-22T21:20:35.084619+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:05:23.557893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181180+0000","last_change":"2024-05-22T21:20:33.590422+0000","last_active":"2024-05-22T21:20:35.181180+0000","last_peered":"2024-05-22T21:20:35.181180+0000","last_clean":"2024-05-22T21:20:35.181180+0000","last_became_active":"2024-05-22T21:20:33.590181+0000","last_became_peered":"2024-05-22T21:20:33.590181+0000","last_unstale":"2024-05-22T21:20:35.181180+0000","last_undegraded":"2024-05-22T21:20:35.181180+0000","last_fullsized":"2024-05-22T21:20:35.181180+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:10:00.351856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984884+0000","last_change":"2024-05-22T21:20:33.985435+0000","last_active":"2024-05-22T21:20:34.984884+0000","last_peered":"2024-05-22T21:20:34.984884+0000","last_clean":"2024-05-22T21:20:34.984884+0000","last_became_active":"2024-05-22T21:20:33.984822+0000","last_became_peered":"2024-05-22T21:20:33.984822+0000","last_unstale":"2024-05-22T21:20:34.984884+0000","last_undegraded":"2024-05-22T21:20:34.984884+0000","last_fullsized":"2024-05-22T21:20:34.984884+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:17:13.221413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"157'2","reported_seq":13,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.985121+0000","last_change":"2024-05-22T21:20:33.594652+0000","last_active":"2024-05-22T21:20:34.985121+0000","last_peered":"2024-05-22T21:20:34.985121+0000","last_clean":"2024-05-22T21:20:34.985121+0000","last_became_active":"2024-05-22T21:20:33.594282+0000","last_became_peered":"2024-05-22T21:20:33.594282+0000","last_unstale":"2024-05-22T21:20:34.985121+0000","last_undegraded":"2024-05-22T21:20:34.985121+0000","last_fullsized":"2024-05-22T21:20:34.985121+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:13:09.674518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":304,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-22T21:20:33.974973+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:20:33.974973+0000","last_peered":"2024-05-22T21:20:33.974973+0000","last_clean":"2024-05-22T21:20:33.974973+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:20:33.974973+0000","last_undegraded":"2024-05-22T21:20:33.974973+0000","last_fullsized":"2024-05-22T21:20:33.974973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1440,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193872,"statfs":{"total":200000000,"available":198525221,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69199999999999995,"15min":0.74099999999999999},"min":{"1min":0.45300000000000001,"5min":0.40000000000000002,"15min":0.36699999999999999},"max":{"1min":0.92700000000000005,"5min":0.98899999999999999,"15min":2.0369999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.889},"last":0.80000000000000004}]},{"osd":1,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72099999999999997,"15min":0.76100000000000001},"min":{"1min":0.53100000000000003,"5min":0.42799999999999999,"15min":0.41699999999999998},"max":{"1min":1.0329999999999999,"5min":1.0840000000000001,"15min":3.0920000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71699999999999997,"15min":0.751},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.39800000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":3.1499999999999999},"last":0.72399999999999998}]},{"osd":2,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70099999999999996,"15min":0.69899999999999995},"min":{"1min":0.48299999999999998,"5min":0.44400000000000001,"15min":0.38800000000000001},"max":{"1min":0.84499999999999997,"5min":0.94999999999999996,"15min":1.923},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75800000000000001,"15min":0.72999999999999998},"min":{"1min":0.61599999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0720000000000001,"5min":1.099,"15min":1.099},"last":0.64900000000000002}]},{"osd":3,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72799999999999998,"15min":0.69999999999999996},"min":{"1min":0.52200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90800000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.753,"15min":0.78300000000000003},"min":{"1min":0.501,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98899999999999999,"5min":1.04,"15min":2.1389999999999998},"last":0.77600000000000002}]},{"osd":4,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.85699999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.624,"15min":0.61499999999999999},"min":{"1min":0.40500000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84099999999999997,"5min":0.90900000000000003,"15min":1.018},"last":0.81200000000000006}]},{"osd":5,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.628,"15min":0.65000000000000002},"min":{"1min":0.41099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.0289999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65700000000000003,"15min":0.64600000000000002},"min":{"1min":0.437,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97899999999999998,"5min":0.996,"15min":0.996},"last":0.89800000000000002}]},{"osd":6,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.71599999999999997,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.872},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68000000000000005,"15min":0.65000000000000002},"min":{"1min":0.497,"5min":0.42199999999999999,"15min":0.41199999999999998},"max":{"1min":0.89700000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.91100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":863,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194449,"statfs":{"total":200000000,"available":199116227,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71099999999999997,"15min":0.69499999999999995},"min":{"1min":0.52000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.91500000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71299999999999997,"15min":0.69099999999999995},"min":{"1min":0.46300000000000002,"5min":0.41599999999999998,"15min":0.39500000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.82299999999999995}]},{"osd":1,"last update":"Wed May 22 21:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69999999999999996,"15min":0.68500000000000005},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.877,"5min":0.95899999999999996,"15min":1.107},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.70999999999999996},"min":{"1min":0.51100000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.92100000000000004,"5min":1.0449999999999999,"15min":1.081},"last":0.75}]},{"osd":2,"last update":"Wed May 22 21:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78100000000000003,"15min":0.76200000000000001},"min":{"1min":0.56100000000000005,"5min":0.442,"15min":0.442},"max":{"1min":0.94599999999999995,"5min":1.087,"15min":1.087},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.64100000000000001}]},{"osd":3,"last update":"Wed May 22 21:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.745},"min":{"1min":0.53000000000000003,"5min":0.49399999999999999,"15min":0.48199999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.415},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70899999999999996,"15min":0.72399999999999998},"min":{"1min":0.48999999999999999,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":0.93300000000000005,"5min":1.087,"15min":1.788},"last":0.627}]},{"osd":4,"last update":"Wed May 22 21:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.435,"5min":0.41699999999999998,"15min":0.374},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65000000000000002,"15min":0.68200000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":2.718},"last":0.86399999999999999}]},{"osd":5,"last update":"Wed May 22 21:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61499999999999999,"15min":0.69599999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":8.5749999999999993},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.71199999999999997},"min":{"1min":0.44800000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90700000000000003,"5min":0.98499999999999999,"15min":1.1140000000000001},"last":0.79700000000000004}]},{"osd":7,"last update":"Wed May 22 21:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71099999999999997,"15min":0.73599999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42399999999999999},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68899999999999995,"15min":0.746},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88400000000000001,"5min":0.98099999999999998,"15min":1.8009999999999999},"last":0.72999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":861,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194450,"statfs":{"total":200000000,"available":199117679,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.66200000000000003,"15min":0.69099999999999995},"min":{"1min":0.38600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93600000000000005,"5min":0.98699999999999999,"15min":1.0409999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.48699999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.81699999999999995,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.55900000000000005}]},{"osd":1,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.45300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.93000000000000005,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.69699999999999995,"15min":0.68600000000000005},"min":{"1min":0.45500000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.85499999999999998,"5min":1.321,"15min":1.321},"last":0.57799999999999996}]},{"osd":2,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.71399999999999997,"15min":0.69999999999999996},"min":{"1min":0.502,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.03,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.71299999999999997,"15min":0.68100000000000005},"min":{"1min":0.50700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84599999999999997,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.66000000000000003}]},{"osd":3,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.57199999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91400000000000003,"5min":0.97199999999999998,"15min":1.1879999999999999},"last":0.504},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.83699999999999997,"5min":1.0469999999999999,"15min":1.111},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58699999999999997,"15min":0.61799999999999999},"min":{"1min":0.40500000000000003,"5min":0.38400000000000001,"15min":0.35099999999999998},"max":{"1min":0.92400000000000004,"5min":0.92800000000000005,"15min":1.272},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95499999999999996,"5min":1.038,"15min":1.038},"last":0.74199999999999999}]},{"osd":6,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64600000000000002,"15min":0.63200000000000001},"min":{"1min":0.46600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.91300000000000003,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.503,"5min":0.41499999999999998,"15min":0.41099999999999998},"max":{"1min":1.0069999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.71399999999999997,"15min":0.71499999999999997},"min":{"1min":0.63200000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.751,"15min":0.78200000000000003},"min":{"1min":0.67700000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98499999999999999,"5min":1.008,"15min":1.2909999999999999},"last":0.76000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":861,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194450,"statfs":{"total":200000000,"available":199117721,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.79200000000000004,"15min":0.747},"min":{"1min":0.41299999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.98899999999999999,"5min":6.6539999999999999,"15min":6.6539999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.72699999999999998},"min":{"1min":0.53900000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.92500000000000004,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":0.66200000000000003}]},{"osd":1,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67000000000000004,"15min":0.70499999999999996},"min":{"1min":0.45100000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95799999999999996,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.73999999999999999,"15min":0.70999999999999996},"min":{"1min":0.48899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90900000000000003,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":0.872}]},{"osd":2,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.73999999999999999,"15min":0.69999999999999996},"min":{"1min":0.51500000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.89000000000000001,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71599999999999997,"15min":0.70899999999999996},"min":{"1min":0.48599999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.95399999999999996,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.63500000000000001}]},{"osd":3,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71499999999999997,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.042,"5min":1.857,"15min":1.857},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.78200000000000003,"15min":0.79000000000000004},"min":{"1min":0.56799999999999995,"5min":0.51500000000000001,"15min":0.47499999999999998},"max":{"1min":1.012,"5min":1.897,"15min":1.897},"last":0.76800000000000002}]},{"osd":5,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.627,"15min":0.64000000000000001},"min":{"1min":0.438,"5min":0.38,"15min":0.379},"max":{"1min":1.0169999999999999,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.41599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.90600000000000003}]},{"osd":6,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65800000000000003,"15min":0.68899999999999995},"min":{"1min":0.44800000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.044,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70299999999999996,"15min":0.69599999999999995},"min":{"1min":0.47399999999999998,"5min":0.44700000000000001,"15min":0.434},"max":{"1min":1.095,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.94799999999999995}]},{"osd":7,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.69699999999999995},"min":{"1min":0.495,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1579999999999999,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.02},{"interface":"front","average":{"1min":0.753,"5min":0.73099999999999998,"15min":0.77200000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.069,"5min":1.1200000000000001,"15min":1.4510000000000001},"last":0.97399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":861,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194450,"statfs":{"total":200000000,"available":199117631,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.443,"15min":0.40899999999999997},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53500000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.44800000000000001,"15min":0.41299999999999998},"min":{"1min":0.30399999999999999,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.57699999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.44}]},{"osd":1,"last update":"Wed May 22 21:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46999999999999997,"15min":0.45100000000000001},"min":{"1min":0.33300000000000002,"5min":0.28399999999999997,"15min":0.26500000000000001},"max":{"1min":0.63800000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54400000000000004,"15min":0.51000000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71599999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.58299999999999996}]},{"osd":2,"last update":"Wed May 22 21:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51400000000000001,"15min":0.498},"min":{"1min":0.40400000000000003,"5min":0.35999999999999999,"15min":0.30599999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.37,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68700000000000006,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed May 22 21:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73699999999999999,"15min":0.70699999999999996},"min":{"1min":0.497,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.96699999999999997,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.753,"15min":0.72299999999999998},"min":{"1min":0.52000000000000002,"5min":0.49199999999999999,"15min":0.435},"max":{"1min":1.0489999999999999,"5min":1.397,"15min":1.5760000000000001},"last":0.49399999999999999}]},{"osd":5,"last update":"Wed May 22 21:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.76100000000000001,"15min":0.70499999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.45100000000000001},"max":{"1min":0.996,"5min":1.234,"15min":1.234},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.755,"15min":0.70799999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95499999999999996,"5min":1.292,"15min":1.3049999999999999},"last":0.622}]},{"osd":6,"last update":"Wed May 22 21:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.74299999999999999,"15min":0.71699999999999997},"min":{"1min":0.51900000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98899999999999999,"5min":1.276,"15min":1.276},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.76400000000000001,"15min":0.71399999999999997},"min":{"1min":0.59199999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94099999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.53400000000000003}]},{"osd":7,"last update":"Wed May 22 21:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78900000000000003,"15min":0.76000000000000001},"min":{"1min":0.56000000000000005,"5min":0.501,"15min":0.495},"max":{"1min":0.97199999999999998,"5min":1.1000000000000001,"15min":1.2709999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78700000000000003,"15min":0.745},"min":{"1min":0.56100000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.92500000000000004,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.52500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":862,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194450,"statfs":{"total":200000000,"available":199117175,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.44900000000000001,"15min":0.41299999999999998},"min":{"1min":0.29699999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.66500000000000004,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.49199999999999999,"15min":0.45000000000000001},"min":{"1min":0.26900000000000002,"5min":0.26800000000000002,"15min":0.23400000000000001},"max":{"1min":0.77400000000000002,"5min":2.532,"15min":2.532},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed May 22 21:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47899999999999998,"15min":0.45500000000000002},"min":{"1min":0.34899999999999998,"5min":0.32900000000000001,"15min":0.29399999999999998},"max":{"1min":0.63600000000000001,"5min":2.948,"15min":2.948},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.57099999999999995,"15min":0.53100000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.39100000000000001},"max":{"1min":0.71799999999999997,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.46200000000000002}]},{"osd":3,"last update":"Wed May 22 21:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53800000000000003,"15min":0.50900000000000001},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.32700000000000001},"max":{"1min":0.69999999999999996,"5min":2.75,"15min":2.75},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.52200000000000002,"15min":0.47399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":0.43099999999999999}]},{"osd":4,"last update":"Wed May 22 21:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.77900000000000003,"15min":0.70799999999999996},"min":{"1min":0.56299999999999994,"5min":0.53100000000000003,"15min":0.52700000000000002},"max":{"1min":0.94799999999999995,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.76200000000000001},"min":{"1min":0.57899999999999996,"5min":0.375,"15min":0.375},"max":{"1min":0.96999999999999997,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Wed May 22 21:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.78200000000000003,"15min":0.72099999999999997},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.47299999999999998},"max":{"1min":0.98199999999999998,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.749,"15min":0.71199999999999997},"min":{"1min":0.52200000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.84799999999999998,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.55900000000000005}]},{"osd":6,"last update":"Wed May 22 21:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.76900000000000002,"15min":0.71799999999999997},"min":{"1min":0.60799999999999998,"5min":0.495,"15min":0.495},"max":{"1min":0.91400000000000003,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73599999999999999,"15min":0.69199999999999995},"min":{"1min":0.496,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.89500000000000002,"5min":2.6880000000000002,"15min":2.6880000000000002},"last":0.66800000000000004}]},{"osd":7,"last update":"Wed May 22 21:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.80000000000000004,"15min":0.745},"min":{"1min":0.50700000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.91200000000000003,"5min":3.0099999999999998,"15min":3.0099999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.82399999999999995,"15min":0.78400000000000003},"min":{"1min":0.58799999999999997,"5min":0.499,"15min":0.499},"max":{"1min":0.97099999999999997,"5min":2.827,"15min":2.827},"last":0.58999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":861,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194450,"statfs":{"total":200000000,"available":199117791,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42599999999999999,"15min":0.432},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.72999999999999998,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.46600000000000003,"15min":0.44},"min":{"1min":0.29999999999999999,"5min":0.26900000000000002,"15min":0.26000000000000001},"max":{"1min":0.65400000000000003,"5min":0.84799999999999998,"15min":0.876},"last":0.36099999999999999}]},{"osd":2,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.505,"15min":0.49399999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.63700000000000001,"5min":1.76,"15min":1.76},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.47599999999999998,"15min":0.51800000000000002},"min":{"1min":0.29799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.54700000000000004,"5min":0.73899999999999999,"15min":1.923},"last":0.42599999999999999}]},{"osd":3,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52400000000000002,"15min":0.55700000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.61299999999999999,"5min":0.82799999999999996,"15min":2.1030000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.505,"15min":0.49099999999999999},"min":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.60999999999999999,"5min":0.72099999999999997,"15min":0.73099999999999998},"last":0.44800000000000001}]},{"osd":4,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.73599999999999999,"15min":0.753},"min":{"1min":0.52300000000000002,"5min":0.46500000000000002,"15min":0.45100000000000001},"max":{"1min":0.89000000000000001,"5min":1.109,"15min":2.1360000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.747,"15min":0.69899999999999995},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94599999999999995,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.74399999999999999},"min":{"1min":0.57799999999999996,"5min":0.53500000000000003,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":1.819,"15min":1.819},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.48799999999999999,"5min":0.46300000000000002,"15min":0.40100000000000002},"max":{"1min":0.94699999999999995,"5min":1.1060000000000001,"15min":1.97},"last":0.63400000000000001}]},{"osd":6,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.77400000000000002,"15min":0.749},"min":{"1min":0.52100000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88700000000000001,"5min":1.085,"15min":1.272},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.80800000000000005,"15min":0.76000000000000001},"min":{"1min":0.60599999999999998,"5min":0.55000000000000004,"15min":0.53200000000000003},"max":{"1min":0.93100000000000005,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.83599999999999997}]},{"osd":7,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76200000000000001,"15min":0.77100000000000002},"min":{"1min":0.50900000000000001,"5min":0.48599999999999999,"15min":0.42099999999999999},"max":{"1min":0.90400000000000003,"5min":1.1379999999999999,"15min":2.0310000000000001},"last":0.754},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.74199999999999999,"15min":0.76800000000000002},"min":{"1min":0.54900000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.80600000000000005,"5min":1.008,"15min":2.0659999999999998},"last":0.64400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574939,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1437,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193875,"statfs":{"total":200000000,"available":198528042,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.43099999999999999,"15min":0.40899999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.22800000000000001},"max":{"1min":0.63900000000000001,"5min":0.85899999999999999,"15min":0.88300000000000001},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.505,"15min":0.48699999999999999},"min":{"1min":0.35999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.61299999999999999,"5min":0.80600000000000005,"15min":0.82999999999999996},"last":0.37}]},{"osd":2,"last update":"Wed May 22 21:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48299999999999998,"15min":0.45800000000000002},"min":{"1min":0.34899999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69399999999999995,"5min":0.88400000000000001,"15min":0.93799999999999994},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.46300000000000002,"15min":0.434},"min":{"1min":0.33800000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70799999999999996,"5min":0.76500000000000001,"15min":0.92300000000000004},"last":0.47999999999999998}]},{"osd":3,"last update":"Wed May 22 21:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45900000000000002,"15min":0.438},"min":{"1min":0.318,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":1.0760000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54700000000000004,"15min":0.51000000000000001},"min":{"1min":0.38900000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66100000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.39600000000000002}]},{"osd":4,"last update":"Wed May 22 21:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73999999999999999,"15min":0.71399999999999997},"min":{"1min":0.47399999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97699999999999998,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.755,"15min":0.73299999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1,"5min":1.113,"15min":1.2709999999999999},"last":0.58699999999999997}]},{"osd":5,"last update":"Wed May 22 21:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.1339999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76600000000000001,"15min":0.74099999999999999},"min":{"1min":0.51500000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0149999999999999,"5min":1.143,"15min":1.2929999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Wed May 22 21:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.80000000000000004,"15min":0.77500000000000002},"min":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.99299999999999999,"5min":1.1240000000000001,"15min":1.2090000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77800000000000002,"15min":0.77000000000000002},"min":{"1min":0.50800000000000001,"5min":0.502,"15min":0.502},"max":{"1min":0.98599999999999999,"5min":1.2090000000000001,"15min":1.5},"last":0.745}]},{"osd":7,"last update":"Wed May 22 21:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.76900000000000002,"15min":0.73199999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0349999999999999,"5min":1.163,"15min":1.163},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.55100000000000005,"5min":0.497,"15min":0.497},"max":{"1min":0.90700000000000003,"5min":1.1990000000000001,"15min":1.3260000000000001},"last":0.61499999999999999}]}]}],"pool_statfs":[]}} 2024-05-22T21:20:38.063 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:20:38.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.230+0000 7fdab5000640 1 Processor -- start 2024-05-22T21:20:38.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.230+0000 7fdab5000640 1 -- start start 2024-05-22T21:20:38.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.230+0000 7fdab5000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdab015dd50 0x7fdab006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:38.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.230+0000 7fdab5000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdab015e130 0x7fdab015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:38.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.230+0000 7fdab5000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdab015d410 0x7fdab015a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:38.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.230+0000 7fdab5000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab0071670 con 0x7fdab015d410 2024-05-22T21:20:38.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.230+0000 7fdab5000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab015d8a0 con 0x7fdab015e130 2024-05-22T21:20:38.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.230+0000 7fdab5000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab006de40 con 0x7fdab015dd50 2024-05-22T21:20:38.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.230+0000 7fdaae800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdab015d410 0x7fdab015a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:38.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.234+0000 7fdaae800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdab015d410 0x7fdab015a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32870/0 (socket says 172.21.15.161:32870) 2024-05-22T21:20:38.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.234+0000 7fdaae800640 1 -- 172.21.15.161:0/1132193430 learned_addr learned my addr 172.21.15.161:0/1132193430 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:38.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.234+0000 7fdaade00640 1 --2- 172.21.15.161:0/1132193430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdab015dd50 0x7fdab006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:38.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.234+0000 7fdaa5400640 1 --2- 172.21.15.161:0/1132193430 >> v2:172.21.15.179:3300/0 conn(0x7fdab015e130 0x7fdab015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:38.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.234+0000 7fdaae800640 1 -- 172.21.15.161:0/1132193430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdab015dd50 msgr2=0x7fdab006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:38.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.234+0000 7fdaae800640 1 --2- 172.21.15.161:0/1132193430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdab015dd50 0x7fdab006d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.234+0000 7fdaae800640 1 -- 172.21.15.161:0/1132193430 >> v2:172.21.15.179:3300/0 conn(0x7fdab015e130 msgr2=0x7fdab015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:38.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.234+0000 7fdaae800640 1 --2- 172.21.15.161:0/1132193430 >> v2:172.21.15.179:3300/0 conn(0x7fdab015e130 0x7fdab015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.234+0000 7fdaae800640 1 -- 172.21.15.161:0/1132193430 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab0070a60 con 0x7fdab015d410 2024-05-22T21:20:38.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.234+0000 7fdaae800640 1 --2- 172.21.15.161:0/1132193430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab015d410 0x7fdab015a500 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fdab0072510 tx=0x7fdaa801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a4e6cd8fc842e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:38.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.234+0000 7fdaad400640 1 -- 172.21.15.161:0/1132193430 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80a57b0 con 0x7fdab015d410 2024-05-22T21:20:38.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.234+0000 7fdaad400640 1 -- 172.21.15.161:0/1132193430 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa80a5910 con 0x7fdab015d410 2024-05-22T21:20:38.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.234+0000 7fdaad400640 1 -- 172.21.15.161:0/1132193430 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80a5ca0 con 0x7fdab015d410 2024-05-22T21:20:38.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 -- 172.21.15.161:0/1132193430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab015d410 msgr2=0x7fdab015a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:38.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 --2- 172.21.15.161:0/1132193430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab015d410 0x7fdab015a500 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fdab0072510 tx=0x7fdaa801c930 comp rx=0 tx=0).stop 2024-05-22T21:20:38.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 -- 172.21.15.161:0/1132193430 shutdown_connections 2024-05-22T21:20:38.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 --2- 172.21.15.161:0/1132193430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab015d410 0x7fdab015a500 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 --2- 172.21.15.161:0/1132193430 >> v2:172.21.15.179:3300/0 conn(0x7fdab015e130 0x7fdab015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 --2- 172.21.15.161:0/1132193430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdab015dd50 0x7fdab006d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 -- 172.21.15.161:0/1132193430 >> 172.21.15.161:0/1132193430 conn(0x7fdab00b0bc0 msgr2=0x7fdab00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:38.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 -- 172.21.15.161:0/1132193430 shutdown_connections 2024-05-22T21:20:38.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 -- 172.21.15.161:0/1132193430 wait complete. 2024-05-22T21:20:38.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 Processor -- start 2024-05-22T21:20:38.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 -- start start 2024-05-22T21:20:38.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab015d410 0x7fdab00b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:38.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdab015dd50 0x7fdab00c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:38.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab015e130 0x7fdab00b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:38.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab0158be0 con 0x7fdab015e130 2024-05-22T21:20:38.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab00b1120 con 0x7fdab015d410 2024-05-22T21:20:38.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdab5000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab00b1400 con 0x7fdab015dd50 2024-05-22T21:20:38.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdaae800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab015e130 0x7fdab00b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:38.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdaae800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab015e130 0x7fdab00b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32876/0 (socket says 172.21.15.161:32876) 2024-05-22T21:20:38.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdaae800640 1 -- 172.21.15.161:0/323392257 learned_addr learned my addr 172.21.15.161:0/323392257 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:38.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdaade00640 1 --2- 172.21.15.161:0/323392257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab015d410 0x7fdab00b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:38.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdaa5400640 1 --2- 172.21.15.161:0/323392257 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdab015dd50 0x7fdab00c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:38.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdaae800640 1 -- 172.21.15.161:0/323392257 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdab015dd50 msgr2=0x7fdab00c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:38.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdaae800640 1 --2- 172.21.15.161:0/323392257 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdab015dd50 0x7fdab00c3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdaae800640 1 -- 172.21.15.161:0/323392257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab015d410 msgr2=0x7fdab00b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:38.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdaae800640 1 --2- 172.21.15.161:0/323392257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab015d410 0x7fdab00b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.238+0000 7fdaae800640 1 -- 172.21.15.161:0/323392257 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa8009b30 con 0x7fdab015e130 2024-05-22T21:20:38.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.242+0000 7fdaae800640 1 --2- 172.21.15.161:0/323392257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab015e130 0x7fdab00b1ee0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80af040 tx=0x7fdaa80196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:38.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.242+0000 7fdaa6a00640 1 -- 172.21.15.161:0/323392257 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80b8670 con 0x7fdab015e130 2024-05-22T21:20:38.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.242+0000 7fdaa6a00640 1 -- 172.21.15.161:0/323392257 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa80b87d0 con 0x7fdab015e130 2024-05-22T21:20:38.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.242+0000 7fdab5000640 1 -- 172.21.15.161:0/323392257 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab00b16e0 con 0x7fdab015e130 2024-05-22T21:20:38.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.242+0000 7fdaa6a00640 1 -- 172.21.15.161:0/323392257 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80b8b80 con 0x7fdab015e130 2024-05-22T21:20:38.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.242+0000 7fdab5000640 1 -- 172.21.15.161:0/323392257 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab00b3a70 con 0x7fdab015e130 2024-05-22T21:20:38.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.242+0000 7fdab5000640 1 -- 172.21.15.161:0/323392257 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab0067220 con 0x7fdab015e130 2024-05-22T21:20:38.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.242+0000 7fdaa6a00640 1 -- 172.21.15.161:0/323392257 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdaa80b8ce0 con 0x7fdab015e130 2024-05-22T21:20:38.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.242+0000 7fdaa6a00640 1 --2- 172.21.15.161:0/323392257 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda8c066690 0x7fda8c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:38.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.242+0000 7fdaa6a00640 1 -- 172.21.15.161:0/323392257 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdaa811f7b0 con 0x7fdab015e130 2024-05-22T21:20:38.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.242+0000 7fdaade00640 1 --2- 172.21.15.161:0/323392257 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda8c066690 0x7fda8c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:38.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.242+0000 7fdaade00640 1 --2- 172.21.15.161:0/323392257 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda8c066690 0x7fda8c068b50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0007a60 tx=0x7fdaa0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:38.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.246+0000 7fdaa6a00640 1 -- 172.21.15.161:0/323392257 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdaa80ecc70 con 0x7fdab015e130 2024-05-22T21:20:38.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.398+0000 7fdab5000640 1 -- 172.21.15.161:0/323392257 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdab00b1290 con 0x7fda8c066690 2024-05-22T21:20:38.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.406+0000 7fdaa6a00640 1 -- 172.21.15.161:0/323392257 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285205 (secure 0 0 0) 0x7fdab00b1290 con 0x7fda8c066690 2024-05-22T21:20:38.413 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:38.414 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:20:38.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.410+0000 7fdab5000640 1 -- 172.21.15.161:0/323392257 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda8c066690 msgr2=0x7fda8c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:38.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.410+0000 7fdab5000640 1 --2- 172.21.15.161:0/323392257 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda8c066690 0x7fda8c068b50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0007a60 tx=0x7fdaa0008040 comp rx=0 tx=0).stop 2024-05-22T21:20:38.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.410+0000 7fdab5000640 1 -- 172.21.15.161:0/323392257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab015e130 msgr2=0x7fdab00b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:38.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.410+0000 7fdab5000640 1 --2- 172.21.15.161:0/323392257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab015e130 0x7fdab00b1ee0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80af040 tx=0x7fdaa80196d0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.410+0000 7fdab5000640 1 -- 172.21.15.161:0/323392257 shutdown_connections 2024-05-22T21:20:38.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.410+0000 7fdab5000640 1 --2- 172.21.15.161:0/323392257 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fda8c066690 0x7fda8c068b50 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.410+0000 7fdab5000640 1 --2- 172.21.15.161:0/323392257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab015e130 0x7fdab00b1ee0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.410+0000 7fdab5000640 1 --2- 172.21.15.161:0/323392257 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdab015dd50 0x7fdab00c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.410+0000 7fdab5000640 1 --2- 172.21.15.161:0/323392257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab015d410 0x7fdab00b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.410+0000 7fdab5000640 1 -- 172.21.15.161:0/323392257 >> 172.21.15.161:0/323392257 conn(0x7fdab00b0bc0 msgr2=0x7fdab00af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:38.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.410+0000 7fdab5000640 1 -- 172.21.15.161:0/323392257 shutdown_connections 2024-05-22T21:20:38.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.410+0000 7fdab5000640 1 -- 172.21.15.161:0/323392257 wait complete. 2024-05-22T21:20:38.433 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-05-22T21:20:36.488688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8049,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554450,"statfs":{"total":1600000000,"available":1591757487,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000633"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.981119+0000","last_change":"2024-05-22T21:20:33.987530+0000","last_active":"2024-05-22T21:20:34.981119+0000","last_peered":"2024-05-22T21:20:34.981119+0000","last_clean":"2024-05-22T21:20:34.981119+0000","last_became_active":"2024-05-22T21:20:33.987187+0000","last_became_peered":"2024-05-22T21:20:33.987187+0000","last_unstale":"2024-05-22T21:20:34.981119+0000","last_undegraded":"2024-05-22T21:20:34.981119+0000","last_fullsized":"2024-05-22T21:20:34.981119+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:06:23.454383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084595+0000","last_change":"2024-05-22T21:20:33.989948+0000","last_active":"2024-05-22T21:20:35.084595+0000","last_peered":"2024-05-22T21:20:35.084595+0000","last_clean":"2024-05-22T21:20:35.084595+0000","last_became_active":"2024-05-22T21:20:33.989761+0000","last_became_peered":"2024-05-22T21:20:33.989761+0000","last_unstale":"2024-05-22T21:20:35.084595+0000","last_undegraded":"2024-05-22T21:20:35.084595+0000","last_fullsized":"2024-05-22T21:20:35.084595+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:40:45.608132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010058+0000","last_change":"2024-05-22T21:20:33.992444+0000","last_active":"2024-05-22T21:20:35.010058+0000","last_peered":"2024-05-22T21:20:35.010058+0000","last_clean":"2024-05-22T21:20:35.010058+0000","last_became_active":"2024-05-22T21:20:33.988764+0000","last_became_peered":"2024-05-22T21:20:33.988764+0000","last_unstale":"2024-05-22T21:20:35.010058+0000","last_undegraded":"2024-05-22T21:20:35.010058+0000","last_fullsized":"2024-05-22T21:20:35.010058+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:55:50.256135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.980760+0000","last_change":"2024-05-22T21:20:33.986053+0000","last_active":"2024-05-22T21:20:34.980760+0000","last_peered":"2024-05-22T21:20:34.980760+0000","last_clean":"2024-05-22T21:20:34.980760+0000","last_became_active":"2024-05-22T21:20:33.984019+0000","last_became_peered":"2024-05-22T21:20:33.984019+0000","last_unstale":"2024-05-22T21:20:34.980760+0000","last_undegraded":"2024-05-22T21:20:34.980760+0000","last_fullsized":"2024-05-22T21:20:34.980760+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:14:18.962972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984949+0000","last_change":"2024-05-22T21:20:33.984277+0000","last_active":"2024-05-22T21:20:34.984949+0000","last_peered":"2024-05-22T21:20:34.984949+0000","last_clean":"2024-05-22T21:20:34.984949+0000","last_became_active":"2024-05-22T21:20:33.983977+0000","last_became_peered":"2024-05-22T21:20:33.983977+0000","last_unstale":"2024-05-22T21:20:34.984949+0000","last_undegraded":"2024-05-22T21:20:34.984949+0000","last_fullsized":"2024-05-22T21:20:34.984949+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:53:21.407922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984088+0000","last_change":"2024-05-22T21:20:33.986489+0000","last_active":"2024-05-22T21:20:34.984088+0000","last_peered":"2024-05-22T21:20:34.984088+0000","last_clean":"2024-05-22T21:20:34.984088+0000","last_became_active":"2024-05-22T21:20:33.986268+0000","last_became_peered":"2024-05-22T21:20:33.986268+0000","last_unstale":"2024-05-22T21:20:34.984088+0000","last_undegraded":"2024-05-22T21:20:34.984088+0000","last_fullsized":"2024-05-22T21:20:34.984088+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:13:10.680899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.983660+0000","last_change":"2024-05-22T21:20:33.990749+0000","last_active":"2024-05-22T21:20:34.983660+0000","last_peered":"2024-05-22T21:20:34.983660+0000","last_clean":"2024-05-22T21:20:34.983660+0000","last_became_active":"2024-05-22T21:20:33.989220+0000","last_became_peered":"2024-05-22T21:20:33.989220+0000","last_unstale":"2024-05-22T21:20:34.983660+0000","last_undegraded":"2024-05-22T21:20:34.983660+0000","last_fullsized":"2024-05-22T21:20:34.983660+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:46:01.370439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010202+0000","last_change":"2024-05-22T21:20:33.992912+0000","last_active":"2024-05-22T21:20:35.010202+0000","last_peered":"2024-05-22T21:20:35.010202+0000","last_clean":"2024-05-22T21:20:35.010202+0000","last_became_active":"2024-05-22T21:20:33.989515+0000","last_became_peered":"2024-05-22T21:20:33.989515+0000","last_unstale":"2024-05-22T21:20:35.010202+0000","last_undegraded":"2024-05-22T21:20:35.010202+0000","last_fullsized":"2024-05-22T21:20:35.010202+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:34:29.353240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010069+0000","last_change":"2024-05-22T21:20:33.988082+0000","last_active":"2024-05-22T21:20:35.010069+0000","last_peered":"2024-05-22T21:20:35.010069+0000","last_clean":"2024-05-22T21:20:35.010069+0000","last_became_active":"2024-05-22T21:20:33.987851+0000","last_became_peered":"2024-05-22T21:20:33.987851+0000","last_unstale":"2024-05-22T21:20:35.010069+0000","last_undegraded":"2024-05-22T21:20:35.010069+0000","last_fullsized":"2024-05-22T21:20:35.010069+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:33:05.996672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084822+0000","last_change":"2024-05-22T21:20:33.987659+0000","last_active":"2024-05-22T21:20:35.084822+0000","last_peered":"2024-05-22T21:20:35.084822+0000","last_clean":"2024-05-22T21:20:35.084822+0000","last_became_active":"2024-05-22T21:20:33.987129+0000","last_became_peered":"2024-05-22T21:20:33.987129+0000","last_unstale":"2024-05-22T21:20:35.084822+0000","last_undegraded":"2024-05-22T21:20:35.084822+0000","last_fullsized":"2024-05-22T21:20:35.084822+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:53:24.255863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181683+0000","last_change":"2024-05-22T21:20:33.977208+0000","last_active":"2024-05-22T21:20:35.181683+0000","last_peered":"2024-05-22T21:20:35.181683+0000","last_clean":"2024-05-22T21:20:35.181683+0000","last_became_active":"2024-05-22T21:20:33.976995+0000","last_became_peered":"2024-05-22T21:20:33.976995+0000","last_unstale":"2024-05-22T21:20:35.181683+0000","last_undegraded":"2024-05-22T21:20:35.181683+0000","last_fullsized":"2024-05-22T21:20:35.181683+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:25:33.574679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010222+0000","last_change":"2024-05-22T21:20:33.992317+0000","last_active":"2024-05-22T21:20:35.010222+0000","last_peered":"2024-05-22T21:20:35.010222+0000","last_clean":"2024-05-22T21:20:35.010222+0000","last_became_active":"2024-05-22T21:20:33.991918+0000","last_became_peered":"2024-05-22T21:20:33.991918+0000","last_unstale":"2024-05-22T21:20:35.010222+0000","last_undegraded":"2024-05-22T21:20:35.010222+0000","last_fullsized":"2024-05-22T21:20:35.010222+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:14:08.304638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.980880+0000","last_change":"2024-05-22T21:20:33.986816+0000","last_active":"2024-05-22T21:20:34.980880+0000","last_peered":"2024-05-22T21:20:34.980880+0000","last_clean":"2024-05-22T21:20:34.980880+0000","last_became_active":"2024-05-22T21:20:33.986618+0000","last_became_peered":"2024-05-22T21:20:33.986618+0000","last_unstale":"2024-05-22T21:20:34.980880+0000","last_undegraded":"2024-05-22T21:20:34.980880+0000","last_fullsized":"2024-05-22T21:20:34.980880+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:15.376070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084507+0000","last_change":"2024-05-22T21:20:33.979997+0000","last_active":"2024-05-22T21:20:35.084507+0000","last_peered":"2024-05-22T21:20:35.084507+0000","last_clean":"2024-05-22T21:20:35.084507+0000","last_became_active":"2024-05-22T21:20:33.979792+0000","last_became_peered":"2024-05-22T21:20:33.979792+0000","last_unstale":"2024-05-22T21:20:35.084507+0000","last_undegraded":"2024-05-22T21:20:35.084507+0000","last_fullsized":"2024-05-22T21:20:35.084507+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:57:40.927650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.981082+0000","last_change":"2024-05-22T21:20:33.985176+0000","last_active":"2024-05-22T21:20:34.981082+0000","last_peered":"2024-05-22T21:20:34.981082+0000","last_clean":"2024-05-22T21:20:34.981082+0000","last_became_active":"2024-05-22T21:20:33.984964+0000","last_became_peered":"2024-05-22T21:20:33.984964+0000","last_unstale":"2024-05-22T21:20:34.981082+0000","last_undegraded":"2024-05-22T21:20:34.981082+0000","last_fullsized":"2024-05-22T21:20:34.981082+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:57:49.306839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181112+0000","last_change":"2024-05-22T21:20:33.981222+0000","last_active":"2024-05-22T21:20:35.181112+0000","last_peered":"2024-05-22T21:20:35.181112+0000","last_clean":"2024-05-22T21:20:35.181112+0000","last_became_active":"2024-05-22T21:20:33.980857+0000","last_became_peered":"2024-05-22T21:20:33.980857+0000","last_unstale":"2024-05-22T21:20:35.181112+0000","last_undegraded":"2024-05-22T21:20:35.181112+0000","last_fullsized":"2024-05-22T21:20:35.181112+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:01:11.747849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.983862+0000","last_change":"2024-05-22T21:20:33.991411+0000","last_active":"2024-05-22T21:20:34.983862+0000","last_peered":"2024-05-22T21:20:34.983862+0000","last_clean":"2024-05-22T21:20:34.983862+0000","last_became_active":"2024-05-22T21:20:33.991158+0000","last_became_peered":"2024-05-22T21:20:33.991158+0000","last_unstale":"2024-05-22T21:20:34.983862+0000","last_undegraded":"2024-05-22T21:20:34.983862+0000","last_fullsized":"2024-05-22T21:20:34.983862+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:47:35.142416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181175+0000","last_change":"2024-05-22T21:20:33.987504+0000","last_active":"2024-05-22T21:20:35.181175+0000","last_peered":"2024-05-22T21:20:35.181175+0000","last_clean":"2024-05-22T21:20:35.181175+0000","last_became_active":"2024-05-22T21:20:33.987169+0000","last_became_peered":"2024-05-22T21:20:33.987169+0000","last_unstale":"2024-05-22T21:20:35.181175+0000","last_undegraded":"2024-05-22T21:20:35.181175+0000","last_fullsized":"2024-05-22T21:20:35.181175+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:10:01.358237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181018+0000","last_change":"2024-05-22T21:20:33.986480+0000","last_active":"2024-05-22T21:20:35.181018+0000","last_peered":"2024-05-22T21:20:35.181018+0000","last_clean":"2024-05-22T21:20:35.181018+0000","last_became_active":"2024-05-22T21:20:33.986231+0000","last_became_peered":"2024-05-22T21:20:33.986231+0000","last_unstale":"2024-05-22T21:20:35.181018+0000","last_undegraded":"2024-05-22T21:20:35.181018+0000","last_fullsized":"2024-05-22T21:20:35.181018+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:46:49.859734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181433+0000","last_change":"2024-05-22T21:20:33.979538+0000","last_active":"2024-05-22T21:20:35.181433+0000","last_peered":"2024-05-22T21:20:35.181433+0000","last_clean":"2024-05-22T21:20:35.181433+0000","last_became_active":"2024-05-22T21:20:33.979318+0000","last_became_peered":"2024-05-22T21:20:33.979318+0000","last_unstale":"2024-05-22T21:20:35.181433+0000","last_undegraded":"2024-05-22T21:20:35.181433+0000","last_fullsized":"2024-05-22T21:20:35.181433+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:00:42.483473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984570+0000","last_change":"2024-05-22T21:20:33.985123+0000","last_active":"2024-05-22T21:20:34.984570+0000","last_peered":"2024-05-22T21:20:34.984570+0000","last_clean":"2024-05-22T21:20:34.984570+0000","last_became_active":"2024-05-22T21:20:33.984161+0000","last_became_peered":"2024-05-22T21:20:33.984161+0000","last_unstale":"2024-05-22T21:20:34.984570+0000","last_undegraded":"2024-05-22T21:20:34.984570+0000","last_fullsized":"2024-05-22T21:20:34.984570+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:49:28.392673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181323+0000","last_change":"2024-05-22T21:20:33.980318+0000","last_active":"2024-05-22T21:20:35.181323+0000","last_peered":"2024-05-22T21:20:35.181323+0000","last_clean":"2024-05-22T21:20:35.181323+0000","last_became_active":"2024-05-22T21:20:33.979682+0000","last_became_peered":"2024-05-22T21:20:33.979682+0000","last_unstale":"2024-05-22T21:20:35.181323+0000","last_undegraded":"2024-05-22T21:20:35.181323+0000","last_fullsized":"2024-05-22T21:20:35.181323+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:07:03.573164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010534+0000","last_change":"2024-05-22T21:20:33.989882+0000","last_active":"2024-05-22T21:20:35.010534+0000","last_peered":"2024-05-22T21:20:35.010534+0000","last_clean":"2024-05-22T21:20:35.010534+0000","last_became_active":"2024-05-22T21:20:33.988460+0000","last_became_peered":"2024-05-22T21:20:33.988460+0000","last_unstale":"2024-05-22T21:20:35.010534+0000","last_undegraded":"2024-05-22T21:20:35.010534+0000","last_fullsized":"2024-05-22T21:20:35.010534+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:13:10.680899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181106+0000","last_change":"2024-05-22T21:20:33.983548+0000","last_active":"2024-05-22T21:20:35.181106+0000","last_peered":"2024-05-22T21:20:35.181106+0000","last_clean":"2024-05-22T21:20:35.181106+0000","last_became_active":"2024-05-22T21:20:33.983299+0000","last_became_peered":"2024-05-22T21:20:33.983299+0000","last_unstale":"2024-05-22T21:20:35.181106+0000","last_undegraded":"2024-05-22T21:20:35.181106+0000","last_fullsized":"2024-05-22T21:20:35.181106+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:13:01.144330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084864+0000","last_change":"2024-05-22T21:20:33.985027+0000","last_active":"2024-05-22T21:20:35.084864+0000","last_peered":"2024-05-22T21:20:35.084864+0000","last_clean":"2024-05-22T21:20:35.084864+0000","last_became_active":"2024-05-22T21:20:33.984340+0000","last_became_peered":"2024-05-22T21:20:33.984340+0000","last_unstale":"2024-05-22T21:20:35.084864+0000","last_undegraded":"2024-05-22T21:20:35.084864+0000","last_fullsized":"2024-05-22T21:20:35.084864+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:00:38.509323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181163+0000","last_change":"2024-05-22T21:20:33.981091+0000","last_active":"2024-05-22T21:20:35.181163+0000","last_peered":"2024-05-22T21:20:35.181163+0000","last_clean":"2024-05-22T21:20:35.181163+0000","last_became_active":"2024-05-22T21:20:33.979598+0000","last_became_peered":"2024-05-22T21:20:33.979598+0000","last_unstale":"2024-05-22T21:20:35.181163+0000","last_undegraded":"2024-05-22T21:20:35.181163+0000","last_fullsized":"2024-05-22T21:20:35.181163+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:15:12.227118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.980762+0000","last_change":"2024-05-22T21:20:33.991797+0000","last_active":"2024-05-22T21:20:34.980762+0000","last_peered":"2024-05-22T21:20:34.980762+0000","last_clean":"2024-05-22T21:20:34.980762+0000","last_became_active":"2024-05-22T21:20:33.991580+0000","last_became_peered":"2024-05-22T21:20:33.991580+0000","last_unstale":"2024-05-22T21:20:34.980762+0000","last_undegraded":"2024-05-22T21:20:34.980762+0000","last_fullsized":"2024-05-22T21:20:34.980762+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:23:04.919505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.182107+0000","last_change":"2024-05-22T21:20:33.985691+0000","last_active":"2024-05-22T21:20:35.182107+0000","last_peered":"2024-05-22T21:20:35.182107+0000","last_clean":"2024-05-22T21:20:35.182107+0000","last_became_active":"2024-05-22T21:20:33.985094+0000","last_became_peered":"2024-05-22T21:20:33.985094+0000","last_unstale":"2024-05-22T21:20:35.182107+0000","last_undegraded":"2024-05-22T21:20:35.182107+0000","last_fullsized":"2024-05-22T21:20:35.182107+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:52:42.816671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.980938+0000","last_change":"2024-05-22T21:20:33.982490+0000","last_active":"2024-05-22T21:20:34.980938+0000","last_peered":"2024-05-22T21:20:34.980938+0000","last_clean":"2024-05-22T21:20:34.980938+0000","last_became_active":"2024-05-22T21:20:33.982208+0000","last_became_peered":"2024-05-22T21:20:33.982208+0000","last_unstale":"2024-05-22T21:20:34.980938+0000","last_undegraded":"2024-05-22T21:20:34.980938+0000","last_fullsized":"2024-05-22T21:20:34.980938+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:49:22.599564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.981178+0000","last_change":"2024-05-22T21:20:33.989217+0000","last_active":"2024-05-22T21:20:34.981178+0000","last_peered":"2024-05-22T21:20:34.981178+0000","last_clean":"2024-05-22T21:20:34.981178+0000","last_became_active":"2024-05-22T21:20:33.988809+0000","last_became_peered":"2024-05-22T21:20:33.988809+0000","last_unstale":"2024-05-22T21:20:34.981178+0000","last_undegraded":"2024-05-22T21:20:34.981178+0000","last_fullsized":"2024-05-22T21:20:34.981178+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:17:15.251407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984677+0000","last_change":"2024-05-22T21:20:33.985274+0000","last_active":"2024-05-22T21:20:34.984677+0000","last_peered":"2024-05-22T21:20:34.984677+0000","last_clean":"2024-05-22T21:20:34.984677+0000","last_became_active":"2024-05-22T21:20:33.984511+0000","last_became_peered":"2024-05-22T21:20:33.984511+0000","last_unstale":"2024-05-22T21:20:34.984677+0000","last_undegraded":"2024-05-22T21:20:34.984677+0000","last_fullsized":"2024-05-22T21:20:34.984677+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:56:13.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.983767+0000","last_change":"2024-05-22T21:20:33.989756+0000","last_active":"2024-05-22T21:20:34.983767+0000","last_peered":"2024-05-22T21:20:34.983767+0000","last_clean":"2024-05-22T21:20:34.983767+0000","last_became_active":"2024-05-22T21:20:33.988822+0000","last_became_peered":"2024-05-22T21:20:33.988822+0000","last_unstale":"2024-05-22T21:20:34.983767+0000","last_undegraded":"2024-05-22T21:20:34.983767+0000","last_fullsized":"2024-05-22T21:20:34.983767+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:39:43.907759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181055+0000","last_change":"2024-05-22T21:20:33.984066+0000","last_active":"2024-05-22T21:20:35.181055+0000","last_peered":"2024-05-22T21:20:35.181055+0000","last_clean":"2024-05-22T21:20:35.181055+0000","last_became_active":"2024-05-22T21:20:33.983845+0000","last_became_peered":"2024-05-22T21:20:33.983845+0000","last_unstale":"2024-05-22T21:20:35.181055+0000","last_undegraded":"2024-05-22T21:20:35.181055+0000","last_fullsized":"2024-05-22T21:20:35.181055+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:37:48.073294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"157'4","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984283+0000","last_change":"2024-05-22T21:20:33.592688+0000","last_active":"2024-05-22T21:20:34.984283+0000","last_peered":"2024-05-22T21:20:34.984283+0000","last_clean":"2024-05-22T21:20:34.984283+0000","last_became_active":"2024-05-22T21:20:33.592449+0000","last_became_peered":"2024-05-22T21:20:33.592449+0000","last_unstale":"2024-05-22T21:20:34.984283+0000","last_undegraded":"2024-05-22T21:20:34.984283+0000","last_fullsized":"2024-05-22T21:20:34.984283+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:33:04.990291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010313+0000","last_change":"2024-05-22T21:20:33.992717+0000","last_active":"2024-05-22T21:20:35.010313+0000","last_peered":"2024-05-22T21:20:35.010313+0000","last_clean":"2024-05-22T21:20:35.010313+0000","last_became_active":"2024-05-22T21:20:33.989166+0000","last_became_peered":"2024-05-22T21:20:33.989166+0000","last_unstale":"2024-05-22T21:20:35.010313+0000","last_undegraded":"2024-05-22T21:20:35.010313+0000","last_fullsized":"2024-05-22T21:20:35.010313+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:36:01.329591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"157'1","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.182148+0000","last_change":"2024-05-22T21:20:33.592953+0000","last_active":"2024-05-22T21:20:35.182148+0000","last_peered":"2024-05-22T21:20:35.182148+0000","last_clean":"2024-05-22T21:20:35.182148+0000","last_became_active":"2024-05-22T21:20:33.590822+0000","last_became_peered":"2024-05-22T21:20:33.590822+0000","last_unstale":"2024-05-22T21:20:35.182148+0000","last_undegraded":"2024-05-22T21:20:35.182148+0000","last_fullsized":"2024-05-22T21:20:35.182148+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:53:59.333062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984778+0000","last_change":"2024-05-22T21:20:33.990586+0000","last_active":"2024-05-22T21:20:34.984778+0000","last_peered":"2024-05-22T21:20:34.984778+0000","last_clean":"2024-05-22T21:20:34.984778+0000","last_became_active":"2024-05-22T21:20:33.990357+0000","last_became_peered":"2024-05-22T21:20:33.990357+0000","last_unstale":"2024-05-22T21:20:34.984778+0000","last_undegraded":"2024-05-22T21:20:34.984778+0000","last_fullsized":"2024-05-22T21:20:34.984778+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:01:11.747849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"157'2","reported_seq":13,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181392+0000","last_change":"2024-05-22T21:20:33.584011+0000","last_active":"2024-05-22T21:20:35.181392+0000","last_peered":"2024-05-22T21:20:35.181392+0000","last_clean":"2024-05-22T21:20:35.181392+0000","last_became_active":"2024-05-22T21:20:33.583750+0000","last_became_peered":"2024-05-22T21:20:33.583750+0000","last_unstale":"2024-05-22T21:20:35.181392+0000","last_undegraded":"2024-05-22T21:20:35.181392+0000","last_fullsized":"2024-05-22T21:20:35.181392+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:17:12.215032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.985033+0000","last_change":"2024-05-22T21:20:33.986461+0000","last_active":"2024-05-22T21:20:34.985033+0000","last_peered":"2024-05-22T21:20:34.985033+0000","last_clean":"2024-05-22T21:20:34.985033+0000","last_became_active":"2024-05-22T21:20:33.986202+0000","last_became_peered":"2024-05-22T21:20:33.986202+0000","last_unstale":"2024-05-22T21:20:34.985033+0000","last_undegraded":"2024-05-22T21:20:34.985033+0000","last_fullsized":"2024-05-22T21:20:34.985033+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:10:01.358237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"157'2","reported_seq":13,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084665+0000","last_change":"2024-05-22T21:20:33.589018+0000","last_active":"2024-05-22T21:20:35.084665+0000","last_peered":"2024-05-22T21:20:35.084665+0000","last_clean":"2024-05-22T21:20:35.084665+0000","last_became_active":"2024-05-22T21:20:33.588811+0000","last_became_peered":"2024-05-22T21:20:33.588811+0000","last_unstale":"2024-05-22T21:20:35.084665+0000","last_undegraded":"2024-05-22T21:20:35.084665+0000","last_fullsized":"2024-05-22T21:20:35.084665+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:22:10.147717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181261+0000","last_change":"2024-05-22T21:20:33.987370+0000","last_active":"2024-05-22T21:20:35.181261+0000","last_peered":"2024-05-22T21:20:35.181261+0000","last_clean":"2024-05-22T21:20:35.181261+0000","last_became_active":"2024-05-22T21:20:33.986840+0000","last_became_peered":"2024-05-22T21:20:33.986840+0000","last_unstale":"2024-05-22T21:20:35.181261+0000","last_undegraded":"2024-05-22T21:20:35.181261+0000","last_fullsized":"2024-05-22T21:20:35.181261+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:53:21.407922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"157'1","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084689+0000","last_change":"2024-05-22T21:20:33.589728+0000","last_active":"2024-05-22T21:20:35.084689+0000","last_peered":"2024-05-22T21:20:35.084689+0000","last_clean":"2024-05-22T21:20:35.084689+0000","last_became_active":"2024-05-22T21:20:33.589488+0000","last_became_peered":"2024-05-22T21:20:33.589488+0000","last_unstale":"2024-05-22T21:20:35.084689+0000","last_undegraded":"2024-05-22T21:20:35.084689+0000","last_fullsized":"2024-05-22T21:20:35.084689+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:16:14.369689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.983966+0000","last_change":"2024-05-22T21:20:33.990896+0000","last_active":"2024-05-22T21:20:34.983966+0000","last_peered":"2024-05-22T21:20:34.983966+0000","last_clean":"2024-05-22T21:20:34.983966+0000","last_became_active":"2024-05-22T21:20:33.989550+0000","last_became_peered":"2024-05-22T21:20:33.989550+0000","last_unstale":"2024-05-22T21:20:34.983966+0000","last_undegraded":"2024-05-22T21:20:34.983966+0000","last_fullsized":"2024-05-22T21:20:34.983966+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:55:50.256135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"157'1","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181339+0000","last_change":"2024-05-22T21:20:33.586334+0000","last_active":"2024-05-22T21:20:35.181339+0000","last_peered":"2024-05-22T21:20:35.181339+0000","last_clean":"2024-05-22T21:20:35.181339+0000","last_became_active":"2024-05-22T21:20:33.585375+0000","last_became_peered":"2024-05-22T21:20:33.585375+0000","last_unstale":"2024-05-22T21:20:35.181339+0000","last_undegraded":"2024-05-22T21:20:35.181339+0000","last_fullsized":"2024-05-22T21:20:35.181339+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:13:09.674518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010241+0000","last_change":"2024-05-22T21:20:33.988721+0000","last_active":"2024-05-22T21:20:35.010241+0000","last_peered":"2024-05-22T21:20:35.010241+0000","last_clean":"2024-05-22T21:20:35.010241+0000","last_became_active":"2024-05-22T21:20:33.987610+0000","last_became_peered":"2024-05-22T21:20:33.987610+0000","last_unstale":"2024-05-22T21:20:35.010241+0000","last_undegraded":"2024-05-22T21:20:35.010241+0000","last_fullsized":"2024-05-22T21:20:35.010241+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:46:01.370439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.983757+0000","last_change":"2024-05-22T21:20:33.591934+0000","last_active":"2024-05-22T21:20:34.983757+0000","last_peered":"2024-05-22T21:20:34.983757+0000","last_clean":"2024-05-22T21:20:34.983757+0000","last_became_active":"2024-05-22T21:20:33.591250+0000","last_became_peered":"2024-05-22T21:20:33.591250+0000","last_unstale":"2024-05-22T21:20:34.983757+0000","last_undegraded":"2024-05-22T21:20:34.983757+0000","last_fullsized":"2024-05-22T21:20:34.983757+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:43:56.234498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084616+0000","last_change":"2024-05-22T21:20:33.977727+0000","last_active":"2024-05-22T21:20:35.084616+0000","last_peered":"2024-05-22T21:20:35.084616+0000","last_clean":"2024-05-22T21:20:35.084616+0000","last_became_active":"2024-05-22T21:20:33.977496+0000","last_became_peered":"2024-05-22T21:20:33.977496+0000","last_unstale":"2024-05-22T21:20:35.084616+0000","last_undegraded":"2024-05-22T21:20:35.084616+0000","last_fullsized":"2024-05-22T21:20:35.084616+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:10:32.784200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084941+0000","last_change":"2024-05-22T21:20:33.591209+0000","last_active":"2024-05-22T21:20:35.084941+0000","last_peered":"2024-05-22T21:20:35.084941+0000","last_clean":"2024-05-22T21:20:35.084941+0000","last_became_active":"2024-05-22T21:20:33.590997+0000","last_became_peered":"2024-05-22T21:20:33.590997+0000","last_unstale":"2024-05-22T21:20:35.084941+0000","last_undegraded":"2024-05-22T21:20:35.084941+0000","last_fullsized":"2024-05-22T21:20:35.084941+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:05:58.495735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.980943+0000","last_change":"2024-05-22T21:20:33.981952+0000","last_active":"2024-05-22T21:20:34.980943+0000","last_peered":"2024-05-22T21:20:34.980943+0000","last_clean":"2024-05-22T21:20:34.980943+0000","last_became_active":"2024-05-22T21:20:33.981749+0000","last_became_peered":"2024-05-22T21:20:33.981749+0000","last_unstale":"2024-05-22T21:20:34.980943+0000","last_undegraded":"2024-05-22T21:20:34.980943+0000","last_fullsized":"2024-05-22T21:20:34.980943+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:32.511970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.182142+0000","last_change":"2024-05-22T21:20:33.593354+0000","last_active":"2024-05-22T21:20:35.182142+0000","last_peered":"2024-05-22T21:20:35.182142+0000","last_clean":"2024-05-22T21:20:35.182142+0000","last_became_active":"2024-05-22T21:20:33.592907+0000","last_became_peered":"2024-05-22T21:20:33.592907+0000","last_unstale":"2024-05-22T21:20:35.182142+0000","last_undegraded":"2024-05-22T21:20:35.182142+0000","last_fullsized":"2024-05-22T21:20:35.182142+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:47:05.722164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181988+0000","last_change":"2024-05-22T21:20:33.985266+0000","last_active":"2024-05-22T21:20:35.181988+0000","last_peered":"2024-05-22T21:20:35.181988+0000","last_clean":"2024-05-22T21:20:35.181988+0000","last_became_active":"2024-05-22T21:20:33.985005+0000","last_became_peered":"2024-05-22T21:20:33.985005+0000","last_unstale":"2024-05-22T21:20:35.181988+0000","last_undegraded":"2024-05-22T21:20:35.181988+0000","last_fullsized":"2024-05-22T21:20:35.181988+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:57:40.927650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181652+0000","last_change":"2024-05-22T21:20:33.595367+0000","last_active":"2024-05-22T21:20:35.181652+0000","last_peered":"2024-05-22T21:20:35.181652+0000","last_clean":"2024-05-22T21:20:35.181652+0000","last_became_active":"2024-05-22T21:20:33.595132+0000","last_became_peered":"2024-05-22T21:20:33.595132+0000","last_unstale":"2024-05-22T21:20:35.181652+0000","last_undegraded":"2024-05-22T21:20:35.181652+0000","last_fullsized":"2024-05-22T21:20:35.181652+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:14:07.298257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984223+0000","last_change":"2024-05-22T21:20:33.978008+0000","last_active":"2024-05-22T21:20:34.984223+0000","last_peered":"2024-05-22T21:20:34.984223+0000","last_clean":"2024-05-22T21:20:34.984223+0000","last_became_active":"2024-05-22T21:20:33.977772+0000","last_became_peered":"2024-05-22T21:20:33.977772+0000","last_unstale":"2024-05-22T21:20:34.984223+0000","last_undegraded":"2024-05-22T21:20:34.984223+0000","last_fullsized":"2024-05-22T21:20:34.984223+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:15:12.227118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"157'3","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.182214+0000","last_change":"2024-05-22T21:20:33.589624+0000","last_active":"2024-05-22T21:20:35.182214+0000","last_peered":"2024-05-22T21:20:35.182214+0000","last_clean":"2024-05-22T21:20:35.182214+0000","last_became_active":"2024-05-22T21:20:33.589049+0000","last_became_peered":"2024-05-22T21:20:33.589049+0000","last_unstale":"2024-05-22T21:20:35.182214+0000","last_undegraded":"2024-05-22T21:20:35.182214+0000","last_fullsized":"2024-05-22T21:20:35.182214+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:57:48.300459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984190+0000","last_change":"2024-05-22T21:20:33.989493+0000","last_active":"2024-05-22T21:20:34.984190+0000","last_peered":"2024-05-22T21:20:34.984190+0000","last_clean":"2024-05-22T21:20:34.984190+0000","last_became_active":"2024-05-22T21:20:33.989267+0000","last_became_peered":"2024-05-22T21:20:33.989267+0000","last_unstale":"2024-05-22T21:20:34.984190+0000","last_undegraded":"2024-05-22T21:20:34.984190+0000","last_fullsized":"2024-05-22T21:20:34.984190+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:14:32.778352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"157'6","reported_seq":17,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010411+0000","last_change":"2024-05-22T21:20:33.597863+0000","last_active":"2024-05-22T21:20:35.010411+0000","last_peered":"2024-05-22T21:20:35.010411+0000","last_clean":"2024-05-22T21:20:35.010411+0000","last_became_active":"2024-05-22T21:20:33.597573+0000","last_became_peered":"2024-05-22T21:20:33.597573+0000","last_unstale":"2024-05-22T21:20:35.010411+0000","last_undegraded":"2024-05-22T21:20:35.010411+0000","last_fullsized":"2024-05-22T21:20:35.010411+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:33:03.240282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.981320+0000","last_change":"2024-05-22T21:20:33.986706+0000","last_active":"2024-05-22T21:20:34.981320+0000","last_peered":"2024-05-22T21:20:34.981320+0000","last_clean":"2024-05-22T21:20:34.981320+0000","last_became_active":"2024-05-22T21:20:33.986498+0000","last_became_peered":"2024-05-22T21:20:33.986498+0000","last_unstale":"2024-05-22T21:20:34.981320+0000","last_undegraded":"2024-05-22T21:20:34.981320+0000","last_fullsized":"2024-05-22T21:20:34.981320+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:35:17.584421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.981166+0000","last_change":"2024-05-22T21:20:33.592145+0000","last_active":"2024-05-22T21:20:34.981166+0000","last_peered":"2024-05-22T21:20:34.981166+0000","last_clean":"2024-05-22T21:20:34.981166+0000","last_became_active":"2024-05-22T21:20:33.591759+0000","last_became_peered":"2024-05-22T21:20:33.591759+0000","last_unstale":"2024-05-22T21:20:34.981166+0000","last_undegraded":"2024-05-22T21:20:34.981166+0000","last_fullsized":"2024-05-22T21:20:34.981166+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:37:49.268832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084507+0000","last_change":"2024-05-22T21:20:33.985312+0000","last_active":"2024-05-22T21:20:35.084507+0000","last_peered":"2024-05-22T21:20:35.084507+0000","last_clean":"2024-05-22T21:20:35.084507+0000","last_became_active":"2024-05-22T21:20:33.985063+0000","last_became_peered":"2024-05-22T21:20:33.985063+0000","last_unstale":"2024-05-22T21:20:35.084507+0000","last_undegraded":"2024-05-22T21:20:35.084507+0000","last_fullsized":"2024-05-22T21:20:35.084507+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:52:42.816671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084774+0000","last_change":"2024-05-22T21:20:33.592526+0000","last_active":"2024-05-22T21:20:35.084774+0000","last_peered":"2024-05-22T21:20:35.084774+0000","last_clean":"2024-05-22T21:20:35.084774+0000","last_became_active":"2024-05-22T21:20:33.592295+0000","last_became_peered":"2024-05-22T21:20:33.592295+0000","last_unstale":"2024-05-22T21:20:35.084774+0000","last_undegraded":"2024-05-22T21:20:35.084774+0000","last_fullsized":"2024-05-22T21:20:35.084774+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:35:16.578041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010304+0000","last_change":"2024-05-22T21:20:33.989472+0000","last_active":"2024-05-22T21:20:35.010304+0000","last_peered":"2024-05-22T21:20:35.010304+0000","last_clean":"2024-05-22T21:20:35.010304+0000","last_became_active":"2024-05-22T21:20:33.989263+0000","last_became_peered":"2024-05-22T21:20:33.989263+0000","last_unstale":"2024-05-22T21:20:35.010304+0000","last_undegraded":"2024-05-22T21:20:35.010304+0000","last_fullsized":"2024-05-22T21:20:35.010304+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:47:35.142416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"157'8","reported_seq":19,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010427+0000","last_change":"2024-05-22T21:20:33.596302+0000","last_active":"2024-05-22T21:20:35.010427+0000","last_peered":"2024-05-22T21:20:35.010427+0000","last_clean":"2024-05-22T21:20:35.010427+0000","last_became_active":"2024-05-22T21:20:33.596064+0000","last_became_peered":"2024-05-22T21:20:33.596064+0000","last_unstale":"2024-05-22T21:20:35.010427+0000","last_undegraded":"2024-05-22T21:20:35.010427+0000","last_fullsized":"2024-05-22T21:20:35.010427+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:43:56.234498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.010340+0000","last_change":"2024-05-22T21:20:33.988060+0000","last_active":"2024-05-22T21:20:35.010340+0000","last_peered":"2024-05-22T21:20:35.010340+0000","last_clean":"2024-05-22T21:20:35.010340+0000","last_became_active":"2024-05-22T21:20:33.987266+0000","last_became_peered":"2024-05-22T21:20:33.987266+0000","last_unstale":"2024-05-22T21:20:35.010340+0000","last_undegraded":"2024-05-22T21:20:35.010340+0000","last_fullsized":"2024-05-22T21:20:35.010340+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:56:13.371399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"157'1","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.182211+0000","last_change":"2024-05-22T21:20:33.593172+0000","last_active":"2024-05-22T21:20:35.182211+0000","last_peered":"2024-05-22T21:20:35.182211+0000","last_clean":"2024-05-22T21:20:35.182211+0000","last_became_active":"2024-05-22T21:20:33.592638+0000","last_became_peered":"2024-05-22T21:20:33.592638+0000","last_unstale":"2024-05-22T21:20:35.182211+0000","last_undegraded":"2024-05-22T21:20:35.182211+0000","last_fullsized":"2024-05-22T21:20:35.182211+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:23:27.892841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181959+0000","last_change":"2024-05-22T21:20:33.991500+0000","last_active":"2024-05-22T21:20:35.181959+0000","last_peered":"2024-05-22T21:20:35.181959+0000","last_clean":"2024-05-22T21:20:35.181959+0000","last_became_active":"2024-05-22T21:20:33.988446+0000","last_became_peered":"2024-05-22T21:20:33.988446+0000","last_unstale":"2024-05-22T21:20:35.181959+0000","last_undegraded":"2024-05-22T21:20:35.181959+0000","last_fullsized":"2024-05-22T21:20:35.181959+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:47:58.618118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"157'2","reported_seq":13,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181477+0000","last_change":"2024-05-22T21:20:33.595497+0000","last_active":"2024-05-22T21:20:35.181477+0000","last_peered":"2024-05-22T21:20:35.181477+0000","last_clean":"2024-05-22T21:20:35.181477+0000","last_became_active":"2024-05-22T21:20:33.595246+0000","last_became_peered":"2024-05-22T21:20:33.595246+0000","last_unstale":"2024-05-22T21:20:35.181477+0000","last_undegraded":"2024-05-22T21:20:35.181477+0000","last_fullsized":"2024-05-22T21:20:35.181477+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:20:35.100019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181605+0000","last_change":"2024-05-22T21:20:33.980190+0000","last_active":"2024-05-22T21:20:35.181605+0000","last_peered":"2024-05-22T21:20:35.181605+0000","last_clean":"2024-05-22T21:20:35.181605+0000","last_became_active":"2024-05-22T21:20:33.979921+0000","last_became_peered":"2024-05-22T21:20:33.979921+0000","last_unstale":"2024-05-22T21:20:35.181605+0000","last_undegraded":"2024-05-22T21:20:35.181605+0000","last_fullsized":"2024-05-22T21:20:35.181605+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:17:13.221413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"157'1","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984350+0000","last_change":"2024-05-22T21:20:33.592852+0000","last_active":"2024-05-22T21:20:34.984350+0000","last_peered":"2024-05-22T21:20:34.984350+0000","last_clean":"2024-05-22T21:20:34.984350+0000","last_became_active":"2024-05-22T21:20:33.592586+0000","last_became_peered":"2024-05-22T21:20:33.592586+0000","last_unstale":"2024-05-22T21:20:34.984350+0000","last_undegraded":"2024-05-22T21:20:34.984350+0000","last_fullsized":"2024-05-22T21:20:34.984350+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:33:03.240282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084734+0000","last_change":"2024-05-22T21:20:33.977590+0000","last_active":"2024-05-22T21:20:35.084734+0000","last_peered":"2024-05-22T21:20:35.084734+0000","last_clean":"2024-05-22T21:20:35.084734+0000","last_became_active":"2024-05-22T21:20:33.977363+0000","last_became_peered":"2024-05-22T21:20:33.977363+0000","last_unstale":"2024-05-22T21:20:35.084734+0000","last_undegraded":"2024-05-22T21:20:35.084734+0000","last_fullsized":"2024-05-22T21:20:35.084734+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:49:03.076890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.182083+0000","last_change":"2024-05-22T21:20:33.590581+0000","last_active":"2024-05-22T21:20:35.182083+0000","last_peered":"2024-05-22T21:20:35.182083+0000","last_clean":"2024-05-22T21:20:35.182083+0000","last_became_active":"2024-05-22T21:20:33.590354+0000","last_became_peered":"2024-05-22T21:20:33.590354+0000","last_unstale":"2024-05-22T21:20:35.182083+0000","last_undegraded":"2024-05-22T21:20:35.182083+0000","last_fullsized":"2024-05-22T21:20:35.182083+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:06:22.448002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.981029+0000","last_change":"2024-05-22T21:20:33.986252+0000","last_active":"2024-05-22T21:20:34.981029+0000","last_peered":"2024-05-22T21:20:34.981029+0000","last_clean":"2024-05-22T21:20:34.981029+0000","last_became_active":"2024-05-22T21:20:33.985637+0000","last_became_peered":"2024-05-22T21:20:33.985637+0000","last_unstale":"2024-05-22T21:20:34.981029+0000","last_undegraded":"2024-05-22T21:20:34.981029+0000","last_fullsized":"2024-05-22T21:20:34.981029+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:22:11.154098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"157'2","reported_seq":13,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.182306+0000","last_change":"2024-05-22T21:20:33.591607+0000","last_active":"2024-05-22T21:20:35.182306+0000","last_peered":"2024-05-22T21:20:35.182306+0000","last_clean":"2024-05-22T21:20:35.182306+0000","last_became_active":"2024-05-22T21:20:33.589408+0000","last_became_peered":"2024-05-22T21:20:33.589408+0000","last_unstale":"2024-05-22T21:20:35.182306+0000","last_undegraded":"2024-05-22T21:20:35.182306+0000","last_fullsized":"2024-05-22T21:20:35.182306+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:17:14.245027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.983574+0000","last_change":"2024-05-22T21:20:33.994103+0000","last_active":"2024-05-22T21:20:34.983574+0000","last_peered":"2024-05-22T21:20:34.983574+0000","last_clean":"2024-05-22T21:20:34.983574+0000","last_became_active":"2024-05-22T21:20:33.993145+0000","last_became_peered":"2024-05-22T21:20:33.993145+0000","last_unstale":"2024-05-22T21:20:34.983574+0000","last_undegraded":"2024-05-22T21:20:34.983574+0000","last_fullsized":"2024-05-22T21:20:34.983574+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:14:08.304638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181232+0000","last_change":"2024-05-22T21:20:33.589913+0000","last_active":"2024-05-22T21:20:35.181232+0000","last_peered":"2024-05-22T21:20:35.181232+0000","last_clean":"2024-05-22T21:20:35.181232+0000","last_became_active":"2024-05-22T21:20:33.589674+0000","last_became_peered":"2024-05-22T21:20:33.589674+0000","last_unstale":"2024-05-22T21:20:35.181232+0000","last_undegraded":"2024-05-22T21:20:35.181232+0000","last_fullsized":"2024-05-22T21:20:35.181232+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:47:34.136035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084572+0000","last_change":"2024-05-22T21:20:33.978990+0000","last_active":"2024-05-22T21:20:35.084572+0000","last_peered":"2024-05-22T21:20:35.084572+0000","last_clean":"2024-05-22T21:20:35.084572+0000","last_became_active":"2024-05-22T21:20:33.978613+0000","last_became_peered":"2024-05-22T21:20:33.978613+0000","last_unstale":"2024-05-22T21:20:35.084572+0000","last_undegraded":"2024-05-22T21:20:35.084572+0000","last_fullsized":"2024-05-22T21:20:35.084572+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:47:58.618118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181600+0000","last_change":"2024-05-22T21:20:33.586119+0000","last_active":"2024-05-22T21:20:35.181600+0000","last_peered":"2024-05-22T21:20:35.181600+0000","last_clean":"2024-05-22T21:20:35.181600+0000","last_became_active":"2024-05-22T21:20:33.585883+0000","last_became_peered":"2024-05-22T21:20:33.585883+0000","last_unstale":"2024-05-22T21:20:35.181600+0000","last_undegraded":"2024-05-22T21:20:35.181600+0000","last_fullsized":"2024-05-22T21:20:35.181600+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:33:04.990291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984862+0000","last_change":"2024-05-22T21:20:33.982986+0000","last_active":"2024-05-22T21:20:34.984862+0000","last_peered":"2024-05-22T21:20:34.984862+0000","last_clean":"2024-05-22T21:20:34.984862+0000","last_became_active":"2024-05-22T21:20:33.982716+0000","last_became_peered":"2024-05-22T21:20:33.982716+0000","last_unstale":"2024-05-22T21:20:34.984862+0000","last_undegraded":"2024-05-22T21:20:34.984862+0000","last_fullsized":"2024-05-22T21:20:34.984862+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:20:36.106400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181765+0000","last_change":"2024-05-22T21:20:33.595338+0000","last_active":"2024-05-22T21:20:35.181765+0000","last_peered":"2024-05-22T21:20:35.181765+0000","last_clean":"2024-05-22T21:20:35.181765+0000","last_became_active":"2024-05-22T21:20:33.595081+0000","last_became_peered":"2024-05-22T21:20:33.595081+0000","last_unstale":"2024-05-22T21:20:35.181765+0000","last_undegraded":"2024-05-22T21:20:35.181765+0000","last_fullsized":"2024-05-22T21:20:35.181765+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:56:12.154973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.983683+0000","last_change":"2024-05-22T21:20:33.996909+0000","last_active":"2024-05-22T21:20:34.983683+0000","last_peered":"2024-05-22T21:20:34.983683+0000","last_clean":"2024-05-22T21:20:34.983683+0000","last_became_active":"2024-05-22T21:20:33.993773+0000","last_became_peered":"2024-05-22T21:20:33.993773+0000","last_unstale":"2024-05-22T21:20:34.983683+0000","last_undegraded":"2024-05-22T21:20:34.983683+0000","last_fullsized":"2024-05-22T21:20:34.983683+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:36:01.329591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"157'2","reported_seq":13,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.981132+0000","last_change":"2024-05-22T21:20:33.585549+0000","last_active":"2024-05-22T21:20:34.981132+0000","last_peered":"2024-05-22T21:20:34.981132+0000","last_clean":"2024-05-22T21:20:34.981132+0000","last_became_active":"2024-05-22T21:20:33.585324+0000","last_became_peered":"2024-05-22T21:20:33.585324+0000","last_unstale":"2024-05-22T21:20:34.981132+0000","last_undegraded":"2024-05-22T21:20:34.981132+0000","last_fullsized":"2024-05-22T21:20:34.981132+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:07:44.126970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.981028+0000","last_change":"2024-05-22T21:20:33.984622+0000","last_active":"2024-05-22T21:20:34.981028+0000","last_peered":"2024-05-22T21:20:34.981028+0000","last_clean":"2024-05-22T21:20:34.981028+0000","last_became_active":"2024-05-22T21:20:33.984396+0000","last_became_peered":"2024-05-22T21:20:33.984396+0000","last_unstale":"2024-05-22T21:20:34.981028+0000","last_undegraded":"2024-05-22T21:20:34.981028+0000","last_fullsized":"2024-05-22T21:20:34.981028+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:22:01.171038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984374+0000","last_change":"2024-05-22T21:20:33.589346+0000","last_active":"2024-05-22T21:20:34.984374+0000","last_peered":"2024-05-22T21:20:34.984374+0000","last_clean":"2024-05-22T21:20:34.984374+0000","last_became_active":"2024-05-22T21:20:33.589020+0000","last_became_peered":"2024-05-22T21:20:33.589020+0000","last_unstale":"2024-05-22T21:20:34.984374+0000","last_undegraded":"2024-05-22T21:20:34.984374+0000","last_fullsized":"2024-05-22T21:20:34.984374+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:14:07.298257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084905+0000","last_change":"2024-05-22T21:20:33.985160+0000","last_active":"2024-05-22T21:20:35.084905+0000","last_peered":"2024-05-22T21:20:35.084905+0000","last_clean":"2024-05-22T21:20:35.084905+0000","last_became_active":"2024-05-22T21:20:33.984696+0000","last_became_peered":"2024-05-22T21:20:33.984696+0000","last_unstale":"2024-05-22T21:20:35.084905+0000","last_undegraded":"2024-05-22T21:20:35.084905+0000","last_fullsized":"2024-05-22T21:20:35.084905+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:43:03.172403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084714+0000","last_change":"2024-05-22T21:20:33.593120+0000","last_active":"2024-05-22T21:20:35.084714+0000","last_peered":"2024-05-22T21:20:35.084714+0000","last_clean":"2024-05-22T21:20:35.084714+0000","last_became_active":"2024-05-22T21:20:33.592899+0000","last_became_peered":"2024-05-22T21:20:33.592899+0000","last_unstale":"2024-05-22T21:20:35.084714+0000","last_undegraded":"2024-05-22T21:20:35.084714+0000","last_fullsized":"2024-05-22T21:20:35.084714+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:08:03.810592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.983657+0000","last_change":"2024-05-22T21:20:33.984699+0000","last_active":"2024-05-22T21:20:34.983657+0000","last_peered":"2024-05-22T21:20:34.983657+0000","last_clean":"2024-05-22T21:20:34.983657+0000","last_became_active":"2024-05-22T21:20:33.984347+0000","last_became_peered":"2024-05-22T21:20:33.984347+0000","last_unstale":"2024-05-22T21:20:34.983657+0000","last_undegraded":"2024-05-22T21:20:34.983657+0000","last_fullsized":"2024-05-22T21:20:34.983657+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:33:05.996672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181575+0000","last_change":"2024-05-22T21:20:33.586878+0000","last_active":"2024-05-22T21:20:35.181575+0000","last_peered":"2024-05-22T21:20:35.181575+0000","last_clean":"2024-05-22T21:20:35.181575+0000","last_became_active":"2024-05-22T21:20:33.586618+0000","last_became_peered":"2024-05-22T21:20:33.586618+0000","last_unstale":"2024-05-22T21:20:35.181575+0000","last_undegraded":"2024-05-22T21:20:35.181575+0000","last_fullsized":"2024-05-22T21:20:35.181575+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:53:20.401541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984442+0000","last_change":"2024-05-22T21:20:33.991695+0000","last_active":"2024-05-22T21:20:34.984442+0000","last_peered":"2024-05-22T21:20:34.984442+0000","last_clean":"2024-05-22T21:20:34.984442+0000","last_became_active":"2024-05-22T21:20:33.991406+0000","last_became_peered":"2024-05-22T21:20:33.991406+0000","last_unstale":"2024-05-22T21:20:34.984442+0000","last_undegraded":"2024-05-22T21:20:34.984442+0000","last_fullsized":"2024-05-22T21:20:34.984442+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:40:50.727562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084792+0000","last_change":"2024-05-22T21:20:33.594743+0000","last_active":"2024-05-22T21:20:35.084792+0000","last_peered":"2024-05-22T21:20:35.084792+0000","last_clean":"2024-05-22T21:20:35.084792+0000","last_became_active":"2024-05-22T21:20:33.594509+0000","last_became_peered":"2024-05-22T21:20:33.594509+0000","last_unstale":"2024-05-22T21:20:35.084792+0000","last_undegraded":"2024-05-22T21:20:35.084792+0000","last_fullsized":"2024-05-22T21:20:35.084792+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:57:31.505589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.182084+0000","last_change":"2024-05-22T21:20:33.988470+0000","last_active":"2024-05-22T21:20:35.182084+0000","last_peered":"2024-05-22T21:20:35.182084+0000","last_clean":"2024-05-22T21:20:35.182084+0000","last_became_active":"2024-05-22T21:20:33.988144+0000","last_became_peered":"2024-05-22T21:20:33.988144+0000","last_unstale":"2024-05-22T21:20:35.182084+0000","last_undegraded":"2024-05-22T21:20:35.182084+0000","last_fullsized":"2024-05-22T21:20:35.182084+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:10:32.784200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984598+0000","last_change":"2024-05-22T21:20:33.591896+0000","last_active":"2024-05-22T21:20:34.984598+0000","last_peered":"2024-05-22T21:20:34.984598+0000","last_clean":"2024-05-22T21:20:34.984598+0000","last_became_active":"2024-05-22T21:20:33.589730+0000","last_became_peered":"2024-05-22T21:20:33.589730+0000","last_unstale":"2024-05-22T21:20:34.984598+0000","last_undegraded":"2024-05-22T21:20:34.984598+0000","last_fullsized":"2024-05-22T21:20:34.984598+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:36:00.323210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181149+0000","last_change":"2024-05-22T21:20:33.981881+0000","last_active":"2024-05-22T21:20:35.181149+0000","last_peered":"2024-05-22T21:20:35.181149+0000","last_clean":"2024-05-22T21:20:35.181149+0000","last_became_active":"2024-05-22T21:20:33.981670+0000","last_became_peered":"2024-05-22T21:20:33.981670+0000","last_unstale":"2024-05-22T21:20:35.181149+0000","last_undegraded":"2024-05-22T21:20:35.181149+0000","last_fullsized":"2024-05-22T21:20:35.181149+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:40:50.727562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181412+0000","last_change":"2024-05-22T21:20:33.595495+0000","last_active":"2024-05-22T21:20:35.181412+0000","last_peered":"2024-05-22T21:20:35.181412+0000","last_clean":"2024-05-22T21:20:35.181412+0000","last_became_active":"2024-05-22T21:20:33.595246+0000","last_became_peered":"2024-05-22T21:20:33.595246+0000","last_unstale":"2024-05-22T21:20:35.181412+0000","last_undegraded":"2024-05-22T21:20:35.181412+0000","last_fullsized":"2024-05-22T21:20:35.181412+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:36:00.323210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.084619+0000","last_change":"2024-05-22T21:20:33.978228+0000","last_active":"2024-05-22T21:20:35.084619+0000","last_peered":"2024-05-22T21:20:35.084619+0000","last_clean":"2024-05-22T21:20:35.084619+0000","last_became_active":"2024-05-22T21:20:33.977454+0000","last_became_peered":"2024-05-22T21:20:33.977454+0000","last_unstale":"2024-05-22T21:20:35.084619+0000","last_undegraded":"2024-05-22T21:20:35.084619+0000","last_fullsized":"2024-05-22T21:20:35.084619+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:05:23.557893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:35.181180+0000","last_change":"2024-05-22T21:20:33.590422+0000","last_active":"2024-05-22T21:20:35.181180+0000","last_peered":"2024-05-22T21:20:35.181180+0000","last_clean":"2024-05-22T21:20:35.181180+0000","last_became_active":"2024-05-22T21:20:33.590181+0000","last_became_peered":"2024-05-22T21:20:33.590181+0000","last_unstale":"2024-05-22T21:20:35.181180+0000","last_undegraded":"2024-05-22T21:20:35.181180+0000","last_fullsized":"2024-05-22T21:20:35.181180+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:10:00.351856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.984884+0000","last_change":"2024-05-22T21:20:33.985435+0000","last_active":"2024-05-22T21:20:34.984884+0000","last_peered":"2024-05-22T21:20:34.984884+0000","last_clean":"2024-05-22T21:20:34.984884+0000","last_became_active":"2024-05-22T21:20:33.984822+0000","last_became_peered":"2024-05-22T21:20:33.984822+0000","last_unstale":"2024-05-22T21:20:34.984884+0000","last_undegraded":"2024-05-22T21:20:34.984884+0000","last_fullsized":"2024-05-22T21:20:34.984884+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:33.566303+0000","last_clean_scrub_stamp":"2024-05-22T21:20:33.566303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:17:13.221413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"157'2","reported_seq":13,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-22T21:20:34.985121+0000","last_change":"2024-05-22T21:20:33.594652+0000","last_active":"2024-05-22T21:20:34.985121+0000","last_peered":"2024-05-22T21:20:34.985121+0000","last_clean":"2024-05-22T21:20:34.985121+0000","last_became_active":"2024-05-22T21:20:33.594282+0000","last_became_peered":"2024-05-22T21:20:33.594282+0000","last_unstale":"2024-05-22T21:20:34.985121+0000","last_undegraded":"2024-05-22T21:20:34.985121+0000","last_fullsized":"2024-05-22T21:20:34.985121+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:20:32.559922+0000","last_clean_scrub_stamp":"2024-05-22T21:20:32.559922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:13:09.674518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":304,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-22T21:20:33.974973+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:20:33.974973+0000","last_peered":"2024-05-22T21:20:33.974973+0000","last_clean":"2024-05-22T21:20:33.974973+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:20:33.974973+0000","last_undegraded":"2024-05-22T21:20:33.974973+0000","last_fullsized":"2024-05-22T21:20:33.974973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1440,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193872,"statfs":{"total":200000000,"available":198525221,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69199999999999995,"15min":0.74099999999999999},"min":{"1min":0.45300000000000001,"5min":0.40000000000000002,"15min":0.36699999999999999},"max":{"1min":0.92700000000000005,"5min":0.98899999999999999,"15min":2.0369999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.889},"last":0.80000000000000004}]},{"osd":1,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72099999999999997,"15min":0.76100000000000001},"min":{"1min":0.53100000000000003,"5min":0.42799999999999999,"15min":0.41699999999999998},"max":{"1min":1.0329999999999999,"5min":1.0840000000000001,"15min":3.0920000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71699999999999997,"15min":0.751},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.39800000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":3.1499999999999999},"last":0.72399999999999998}]},{"osd":2,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70099999999999996,"15min":0.69899999999999995},"min":{"1min":0.48299999999999998,"5min":0.44400000000000001,"15min":0.38800000000000001},"max":{"1min":0.84499999999999997,"5min":0.94999999999999996,"15min":1.923},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75800000000000001,"15min":0.72999999999999998},"min":{"1min":0.61599999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0720000000000001,"5min":1.099,"15min":1.099},"last":0.64900000000000002}]},{"osd":3,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72799999999999998,"15min":0.69999999999999996},"min":{"1min":0.52200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90800000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.753,"15min":0.78300000000000003},"min":{"1min":0.501,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98899999999999999,"5min":1.04,"15min":2.1389999999999998},"last":0.77600000000000002}]},{"osd":4,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.85699999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.624,"15min":0.61499999999999999},"min":{"1min":0.40500000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84099999999999997,"5min":0.90900000000000003,"15min":1.018},"last":0.81200000000000006}]},{"osd":5,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.628,"15min":0.65000000000000002},"min":{"1min":0.41099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.0289999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65700000000000003,"15min":0.64600000000000002},"min":{"1min":0.437,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97899999999999998,"5min":0.996,"15min":0.996},"last":0.89800000000000002}]},{"osd":6,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.71599999999999997,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.872},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68000000000000005,"15min":0.65000000000000002},"min":{"1min":0.497,"5min":0.42199999999999999,"15min":0.41199999999999998},"max":{"1min":0.89700000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.91100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":863,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194449,"statfs":{"total":200000000,"available":199116227,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71099999999999997,"15min":0.69499999999999995},"min":{"1min":0.52000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.91500000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71299999999999997,"15min":0.69099999999999995},"min":{"1min":0.46300000000000002,"5min":0.41599999999999998,"15min":0.39500000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.82299999999999995}]},{"osd":1,"last update":"Wed May 22 21:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69999999999999996,"15min":0.68500000000000005},"min":{"1min":0.42699999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.877,"5min":0.95899999999999996,"15min":1.107},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.70999999999999996},"min":{"1min":0.51100000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.92100000000000004,"5min":1.0449999999999999,"15min":1.081},"last":0.75}]},{"osd":2,"last update":"Wed May 22 21:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78100000000000003,"15min":0.76200000000000001},"min":{"1min":0.56100000000000005,"5min":0.442,"15min":0.442},"max":{"1min":0.94599999999999995,"5min":1.087,"15min":1.087},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.64100000000000001}]},{"osd":3,"last update":"Wed May 22 21:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.745},"min":{"1min":0.53000000000000003,"5min":0.49399999999999999,"15min":0.48199999999999998},"max":{"1min":1.28,"5min":1.28,"15min":1.415},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70899999999999996,"15min":0.72399999999999998},"min":{"1min":0.48999999999999999,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":0.93300000000000005,"5min":1.087,"15min":1.788},"last":0.627}]},{"osd":4,"last update":"Wed May 22 21:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.435,"5min":0.41699999999999998,"15min":0.374},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65000000000000002,"15min":0.68200000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":2.718},"last":0.86399999999999999}]},{"osd":5,"last update":"Wed May 22 21:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61499999999999999,"15min":0.69599999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":8.5749999999999993},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.71199999999999997},"min":{"1min":0.44800000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90700000000000003,"5min":0.98499999999999999,"15min":1.1140000000000001},"last":0.79700000000000004}]},{"osd":7,"last update":"Wed May 22 21:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71099999999999997,"15min":0.73599999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42399999999999999},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68899999999999995,"15min":0.746},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88400000000000001,"5min":0.98099999999999998,"15min":1.8009999999999999},"last":0.72999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":861,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194450,"statfs":{"total":200000000,"available":199117679,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.66200000000000003,"15min":0.69099999999999995},"min":{"1min":0.38600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93600000000000005,"5min":0.98699999999999999,"15min":1.0409999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.48699999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.81699999999999995,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.55900000000000005}]},{"osd":1,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.45300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.93000000000000005,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.69699999999999995,"15min":0.68600000000000005},"min":{"1min":0.45500000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.85499999999999998,"5min":1.321,"15min":1.321},"last":0.57799999999999996}]},{"osd":2,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.71399999999999997,"15min":0.69999999999999996},"min":{"1min":0.502,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.03,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.71299999999999997,"15min":0.68100000000000005},"min":{"1min":0.50700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84599999999999997,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.66000000000000003}]},{"osd":3,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.57199999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.91400000000000003,"5min":0.97199999999999998,"15min":1.1879999999999999},"last":0.504},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.83699999999999997,"5min":1.0469999999999999,"15min":1.111},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58699999999999997,"15min":0.61799999999999999},"min":{"1min":0.40500000000000003,"5min":0.38400000000000001,"15min":0.35099999999999998},"max":{"1min":0.92400000000000004,"5min":0.92800000000000005,"15min":1.272},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95499999999999996,"5min":1.038,"15min":1.038},"last":0.74199999999999999}]},{"osd":6,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64600000000000002,"15min":0.63200000000000001},"min":{"1min":0.46600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.91300000000000003,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.503,"5min":0.41499999999999998,"15min":0.41099999999999998},"max":{"1min":1.0069999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.71399999999999997,"15min":0.71499999999999997},"min":{"1min":0.63200000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.751,"15min":0.78200000000000003},"min":{"1min":0.67700000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98499999999999999,"5min":1.008,"15min":1.2909999999999999},"last":0.76000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":861,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194450,"statfs":{"total":200000000,"available":199117721,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.79200000000000004,"15min":0.747},"min":{"1min":0.41299999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.98899999999999999,"5min":6.6539999999999999,"15min":6.6539999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.72699999999999998},"min":{"1min":0.53900000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.92500000000000004,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":0.66200000000000003}]},{"osd":1,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67000000000000004,"15min":0.70499999999999996},"min":{"1min":0.45100000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95799999999999996,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.73999999999999999,"15min":0.70999999999999996},"min":{"1min":0.48899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90900000000000003,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":0.872}]},{"osd":2,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.73999999999999999,"15min":0.69999999999999996},"min":{"1min":0.51500000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.89000000000000001,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71599999999999997,"15min":0.70899999999999996},"min":{"1min":0.48599999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.95399999999999996,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.63500000000000001}]},{"osd":3,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71499999999999997,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.042,"5min":1.857,"15min":1.857},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.78200000000000003,"15min":0.79000000000000004},"min":{"1min":0.56799999999999995,"5min":0.51500000000000001,"15min":0.47499999999999998},"max":{"1min":1.012,"5min":1.897,"15min":1.897},"last":0.76800000000000002}]},{"osd":5,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.627,"15min":0.64000000000000001},"min":{"1min":0.438,"5min":0.38,"15min":0.379},"max":{"1min":1.0169999999999999,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.41599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.90600000000000003}]},{"osd":6,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65800000000000003,"15min":0.68899999999999995},"min":{"1min":0.44800000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.044,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.70299999999999996,"15min":0.69599999999999995},"min":{"1min":0.47399999999999998,"5min":0.44700000000000001,"15min":0.434},"max":{"1min":1.095,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.94799999999999995}]},{"osd":7,"last update":"Wed May 22 21:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.69699999999999995},"min":{"1min":0.495,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1579999999999999,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.02},{"interface":"front","average":{"1min":0.753,"5min":0.73099999999999998,"15min":0.77200000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.069,"5min":1.1200000000000001,"15min":1.4510000000000001},"last":0.97399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":861,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194450,"statfs":{"total":200000000,"available":199117631,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.443,"15min":0.40899999999999997},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53500000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.44800000000000001,"15min":0.41299999999999998},"min":{"1min":0.30399999999999999,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.57699999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.44}]},{"osd":1,"last update":"Wed May 22 21:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46999999999999997,"15min":0.45100000000000001},"min":{"1min":0.33300000000000002,"5min":0.28399999999999997,"15min":0.26500000000000001},"max":{"1min":0.63800000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54400000000000004,"15min":0.51000000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71599999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.58299999999999996}]},{"osd":2,"last update":"Wed May 22 21:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51400000000000001,"15min":0.498},"min":{"1min":0.40400000000000003,"5min":0.35999999999999999,"15min":0.30599999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.37,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68700000000000006,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed May 22 21:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73699999999999999,"15min":0.70699999999999996},"min":{"1min":0.497,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.96699999999999997,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.753,"15min":0.72299999999999998},"min":{"1min":0.52000000000000002,"5min":0.49199999999999999,"15min":0.435},"max":{"1min":1.0489999999999999,"5min":1.397,"15min":1.5760000000000001},"last":0.49399999999999999}]},{"osd":5,"last update":"Wed May 22 21:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.76100000000000001,"15min":0.70499999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.45100000000000001},"max":{"1min":0.996,"5min":1.234,"15min":1.234},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.755,"15min":0.70799999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95499999999999996,"5min":1.292,"15min":1.3049999999999999},"last":0.622}]},{"osd":6,"last update":"Wed May 22 21:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.74299999999999999,"15min":0.71699999999999997},"min":{"1min":0.51900000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98899999999999999,"5min":1.276,"15min":1.276},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.76400000000000001,"15min":0.71399999999999997},"min":{"1min":0.59199999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94099999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.53400000000000003}]},{"osd":7,"last update":"Wed May 22 21:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78900000000000003,"15min":0.76000000000000001},"min":{"1min":0.56000000000000005,"5min":0.501,"15min":0.495},"max":{"1min":0.97199999999999998,"5min":1.1000000000000001,"15min":1.2709999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78700000000000003,"15min":0.745},"min":{"1min":0.56100000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.92500000000000004,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.52500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":862,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194450,"statfs":{"total":200000000,"available":199117175,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.44900000000000001,"15min":0.41299999999999998},"min":{"1min":0.29699999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.66500000000000004,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.49199999999999999,"15min":0.45000000000000001},"min":{"1min":0.26900000000000002,"5min":0.26800000000000002,"15min":0.23400000000000001},"max":{"1min":0.77400000000000002,"5min":2.532,"15min":2.532},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed May 22 21:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47899999999999998,"15min":0.45500000000000002},"min":{"1min":0.34899999999999998,"5min":0.32900000000000001,"15min":0.29399999999999998},"max":{"1min":0.63600000000000001,"5min":2.948,"15min":2.948},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.57099999999999995,"15min":0.53100000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.39100000000000001},"max":{"1min":0.71799999999999997,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.46200000000000002}]},{"osd":3,"last update":"Wed May 22 21:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53800000000000003,"15min":0.50900000000000001},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.32700000000000001},"max":{"1min":0.69999999999999996,"5min":2.75,"15min":2.75},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.52200000000000002,"15min":0.47399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":0.63700000000000001,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":0.43099999999999999}]},{"osd":4,"last update":"Wed May 22 21:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.77900000000000003,"15min":0.70799999999999996},"min":{"1min":0.56299999999999994,"5min":0.53100000000000003,"15min":0.52700000000000002},"max":{"1min":0.94799999999999995,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.76200000000000001},"min":{"1min":0.57899999999999996,"5min":0.375,"15min":0.375},"max":{"1min":0.96999999999999997,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Wed May 22 21:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.78200000000000003,"15min":0.72099999999999997},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.47299999999999998},"max":{"1min":0.98199999999999998,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.749,"15min":0.71199999999999997},"min":{"1min":0.52200000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.84799999999999998,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.55900000000000005}]},{"osd":6,"last update":"Wed May 22 21:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.76900000000000002,"15min":0.71799999999999997},"min":{"1min":0.60799999999999998,"5min":0.495,"15min":0.495},"max":{"1min":0.91400000000000003,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73599999999999999,"15min":0.69199999999999995},"min":{"1min":0.496,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.89500000000000002,"5min":2.6880000000000002,"15min":2.6880000000000002},"last":0.66800000000000004}]},{"osd":7,"last update":"Wed May 22 21:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.80000000000000004,"15min":0.745},"min":{"1min":0.50700000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.91200000000000003,"5min":3.0099999999999998,"15min":3.0099999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.82399999999999995,"15min":0.78400000000000003},"min":{"1min":0.58799999999999997,"5min":0.499,"15min":0.499},"max":{"1min":0.97099999999999997,"5min":2.827,"15min":2.827},"last":0.58999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":861,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194450,"statfs":{"total":200000000,"available":199117791,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42599999999999999,"15min":0.432},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.72999999999999998,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.46600000000000003,"15min":0.44},"min":{"1min":0.29999999999999999,"5min":0.26900000000000002,"15min":0.26000000000000001},"max":{"1min":0.65400000000000003,"5min":0.84799999999999998,"15min":0.876},"last":0.36099999999999999}]},{"osd":2,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.505,"15min":0.49399999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.63700000000000001,"5min":1.76,"15min":1.76},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.47599999999999998,"15min":0.51800000000000002},"min":{"1min":0.29799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.54700000000000004,"5min":0.73899999999999999,"15min":1.923},"last":0.42599999999999999}]},{"osd":3,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52400000000000002,"15min":0.55700000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.61299999999999999,"5min":0.82799999999999996,"15min":2.1030000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.505,"15min":0.49099999999999999},"min":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.60999999999999999,"5min":0.72099999999999997,"15min":0.73099999999999998},"last":0.44800000000000001}]},{"osd":4,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.73599999999999999,"15min":0.753},"min":{"1min":0.52300000000000002,"5min":0.46500000000000002,"15min":0.45100000000000001},"max":{"1min":0.89000000000000001,"5min":1.109,"15min":2.1360000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.747,"15min":0.69899999999999995},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94599999999999995,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.74399999999999999},"min":{"1min":0.57799999999999996,"5min":0.53500000000000003,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":1.819,"15min":1.819},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.48799999999999999,"5min":0.46300000000000002,"15min":0.40100000000000002},"max":{"1min":0.94699999999999995,"5min":1.1060000000000001,"15min":1.97},"last":0.63400000000000001}]},{"osd":6,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.77400000000000002,"15min":0.749},"min":{"1min":0.52100000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88700000000000001,"5min":1.085,"15min":1.272},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.80800000000000005,"15min":0.76000000000000001},"min":{"1min":0.60599999999999998,"5min":0.55000000000000004,"15min":0.53200000000000003},"max":{"1min":0.93100000000000005,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.83599999999999997}]},{"osd":7,"last update":"Wed May 22 21:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76200000000000001,"15min":0.77100000000000002},"min":{"1min":0.50900000000000001,"5min":0.48599999999999999,"15min":0.42099999999999999},"max":{"1min":0.90400000000000003,"5min":1.1379999999999999,"15min":2.0310000000000001},"last":0.754},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.74199999999999999,"15min":0.76800000000000002},"min":{"1min":0.54900000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.80600000000000005,"5min":1.008,"15min":2.0659999999999998},"last":0.64400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574939,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1437,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193875,"statfs":{"total":200000000,"available":198528042,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.43099999999999999,"15min":0.40899999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.22800000000000001},"max":{"1min":0.63900000000000001,"5min":0.85899999999999999,"15min":0.88300000000000001},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.505,"15min":0.48699999999999999},"min":{"1min":0.35999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.61299999999999999,"5min":0.80600000000000005,"15min":0.82999999999999996},"last":0.37}]},{"osd":2,"last update":"Wed May 22 21:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48299999999999998,"15min":0.45800000000000002},"min":{"1min":0.34899999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69399999999999995,"5min":0.88400000000000001,"15min":0.93799999999999994},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.46300000000000002,"15min":0.434},"min":{"1min":0.33800000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70799999999999996,"5min":0.76500000000000001,"15min":0.92300000000000004},"last":0.47999999999999998}]},{"osd":3,"last update":"Wed May 22 21:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45900000000000002,"15min":0.438},"min":{"1min":0.318,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":1.0760000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54700000000000004,"15min":0.51000000000000001},"min":{"1min":0.38900000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66100000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.39600000000000002}]},{"osd":4,"last update":"Wed May 22 21:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73999999999999999,"15min":0.71399999999999997},"min":{"1min":0.47399999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97699999999999998,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.755,"15min":0.73299999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1,"5min":1.113,"15min":1.2709999999999999},"last":0.58699999999999997}]},{"osd":5,"last update":"Wed May 22 21:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.1339999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76600000000000001,"15min":0.74099999999999999},"min":{"1min":0.51500000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0149999999999999,"5min":1.143,"15min":1.2929999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Wed May 22 21:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.80000000000000004,"15min":0.77500000000000002},"min":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.99299999999999999,"5min":1.1240000000000001,"15min":1.2090000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77800000000000002,"15min":0.77000000000000002},"min":{"1min":0.50800000000000001,"5min":0.502,"15min":0.502},"max":{"1min":0.98599999999999999,"5min":1.2090000000000001,"15min":1.5},"last":0.745}]},{"osd":7,"last update":"Wed May 22 21:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.76900000000000002,"15min":0.73199999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0349999999999999,"5min":1.163,"15min":1.163},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.55100000000000005,"5min":0.497,"15min":0.497},"max":{"1min":0.90700000000000003,"5min":1.1990000000000001,"15min":1.3260000000000001},"last":0.61499999999999999}]}]}],"pool_statfs":[]}} 2024-05-22T21:20:38.437 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-22T21:20:38.438 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-22T21:20:38.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.618+0000 7f6a15600640 1 Processor -- start 2024-05-22T21:20:38.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.618+0000 7f6a15600640 1 -- start start 2024-05-22T21:20:38.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a15600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6a101436d0 0x7f6a100cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:38.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a15600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a1006fa80 0x7f6a100b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:38.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a15600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6a100b1b10 0x7f6a100b3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:38.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a15600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a100cc720 con 0x7f6a101436d0 2024-05-22T21:20:38.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a15600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a10070b70 con 0x7f6a100b1b10 2024-05-22T21:20:38.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a15600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1006ff60 con 0x7f6a1006fa80 2024-05-22T21:20:38.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a0fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6a101436d0 0x7f6a100cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:38.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a0fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6a101436d0 0x7f6a100cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32880/0 (socket says 172.21.15.161:32880) 2024-05-22T21:20:38.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a0fe00640 1 -- 172.21.15.161:0/2804123294 learned_addr learned my addr 172.21.15.161:0/2804123294 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:38.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a14c00640 1 --2- 172.21.15.161:0/2804123294 >> v2:172.21.15.179:3300/0 conn(0x7f6a100b1b10 0x7f6a100b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:38.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a07400640 1 --2- 172.21.15.161:0/2804123294 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a1006fa80 0x7f6a100b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:38.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a0fe00640 1 -- 172.21.15.161:0/2804123294 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a1006fa80 msgr2=0x7f6a100b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:38.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a0fe00640 1 --2- 172.21.15.161:0/2804123294 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a1006fa80 0x7f6a100b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a0fe00640 1 -- 172.21.15.161:0/2804123294 >> v2:172.21.15.179:3300/0 conn(0x7f6a100b1b10 msgr2=0x7f6a100b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:38.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a0fe00640 1 --2- 172.21.15.161:0/2804123294 >> v2:172.21.15.179:3300/0 conn(0x7f6a100b1b10 0x7f6a100b3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a0fe00640 1 -- 172.21.15.161:0/2804123294 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a10070240 con 0x7f6a101436d0 2024-05-22T21:20:38.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a14c00640 1 --2- 172.21.15.161:0/2804123294 >> v2:172.21.15.179:3300/0 conn(0x7f6a100b1b10 0x7f6a100b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:38.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a07400640 1 --2- 172.21.15.161:0/2804123294 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a1006fa80 0x7f6a100b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:38.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a0fe00640 1 --2- 172.21.15.161:0/2804123294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a101436d0 0x7f6a100cc1e0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f6a1006f3a0 tx=0x7f69f801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbb6804aa6a74cc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:38.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a0f400640 1 -- 172.21.15.161:0/2804123294 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f80ab620 con 0x7f6a101436d0 2024-05-22T21:20:38.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a0f400640 1 -- 172.21.15.161:0/2804123294 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f80ab780 con 0x7f6a101436d0 2024-05-22T21:20:38.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.622+0000 7f6a0f400640 1 -- 172.21.15.161:0/2804123294 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f80abb10 con 0x7f6a101436d0 2024-05-22T21:20:38.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 -- 172.21.15.161:0/2804123294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a101436d0 msgr2=0x7f6a100cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:38.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 --2- 172.21.15.161:0/2804123294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a101436d0 0x7f6a100cc1e0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f6a1006f3a0 tx=0x7f69f801c460 comp rx=0 tx=0).stop 2024-05-22T21:20:38.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 -- 172.21.15.161:0/2804123294 shutdown_connections 2024-05-22T21:20:38.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 --2- 172.21.15.161:0/2804123294 >> v2:172.21.15.179:3300/0 conn(0x7f6a100b1b10 0x7f6a100b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 --2- 172.21.15.161:0/2804123294 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a1006fa80 0x7f6a100b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 --2- 172.21.15.161:0/2804123294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a101436d0 0x7f6a100cc1e0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 -- 172.21.15.161:0/2804123294 >> 172.21.15.161:0/2804123294 conn(0x7f6a100cec00 msgr2=0x7f6a100cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:38.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 -- 172.21.15.161:0/2804123294 shutdown_connections 2024-05-22T21:20:38.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 -- 172.21.15.161:0/2804123294 wait complete. 2024-05-22T21:20:38.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 Processor -- start 2024-05-22T21:20:38.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 -- start start 2024-05-22T21:20:38.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a1006fa80 0x7f6a10152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:38.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a100b1b10 0x7f6a10157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:38.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a101436d0 0x7f6a10158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:38.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a101598e0 con 0x7f6a100b1b10 2024-05-22T21:20:38.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a10158840 con 0x7f6a1006fa80 2024-05-22T21:20:38.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a15600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a10158b20 con 0x7f6a101436d0 2024-05-22T21:20:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a07400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a100b1b10 0x7f6a10157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a07400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a100b1b10 0x7f6a10157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32892/0 (socket says 172.21.15.161:32892) 2024-05-22T21:20:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a07400640 1 -- 172.21.15.161:0/534187991 learned_addr learned my addr 172.21.15.161:0/534187991 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a0fe00640 1 --2- 172.21.15.161:0/534187991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a1006fa80 0x7f6a10152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.626+0000 7f6a14c00640 1 --2- 172.21.15.161:0/534187991 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a101436d0 0x7f6a10158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.630+0000 7f6a07400640 1 -- 172.21.15.161:0/534187991 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a101436d0 msgr2=0x7f6a10158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.630+0000 7f6a07400640 1 --2- 172.21.15.161:0/534187991 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a101436d0 0x7f6a10158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.630+0000 7f6a07400640 1 -- 172.21.15.161:0/534187991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a1006fa80 msgr2=0x7f6a10152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.630+0000 7f6a07400640 1 --2- 172.21.15.161:0/534187991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a1006fa80 0x7f6a10152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.630+0000 7f6a07400640 1 -- 172.21.15.161:0/534187991 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f80094d0 con 0x7f6a100b1b10 2024-05-22T21:20:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.630+0000 7f6a07400640 1 --2- 172.21.15.161:0/534187991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a100b1b10 0x7f6a10157ac0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000c7c0 tx=0x7f6a0000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:38.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.630+0000 7f6a0cc00640 1 -- 172.21.15.161:0/534187991 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00007a30 con 0x7f6a100b1b10 2024-05-22T21:20:38.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.630+0000 7f6a0cc00640 1 -- 172.21.15.161:0/534187991 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a00007b90 con 0x7f6a100b1b10 2024-05-22T21:20:38.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.630+0000 7f6a0cc00640 1 -- 172.21.15.161:0/534187991 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0000ce70 con 0x7f6a100b1b10 2024-05-22T21:20:38.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.630+0000 7f6a15600640 1 -- 172.21.15.161:0/534187991 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a10158e00 con 0x7f6a100b1b10 2024-05-22T21:20:38.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.630+0000 7f6a15600640 1 -- 172.21.15.161:0/534187991 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a101593c0 con 0x7f6a100b1b10 2024-05-22T21:20:38.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.630+0000 7f6a15600640 1 -- 172.21.15.161:0/534187991 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a10067070 con 0x7f6a100b1b10 2024-05-22T21:20:38.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.630+0000 7f6a0cc00640 1 -- 172.21.15.161:0/534187991 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6a000113f0 con 0x7f6a100b1b10 2024-05-22T21:20:38.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.630+0000 7f6a0cc00640 1 --2- 172.21.15.161:0/534187991 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69ec0666c0 0x7f69ec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:38.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.630+0000 7f6a0cc00640 1 -- 172.21.15.161:0/534187991 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a000841f0 con 0x7f6a100b1b10 2024-05-22T21:20:38.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.630+0000 7f6a0fe00640 1 --2- 172.21.15.161:0/534187991 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69ec0666c0 0x7f69ec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:38.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.634+0000 7f6a0fe00640 1 --2- 172.21.15.161:0/534187991 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69ec0666c0 0x7f69ec068b80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f69f80af000 tx=0x7f69f80b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:38.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.634+0000 7f6a0cc00640 1 -- 172.21.15.161:0/534187991 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6a00051600 con 0x7f6a100b1b10 2024-05-22T21:20:38.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.842+0000 7f6a15600640 1 -- 172.21.15.161:0/534187991 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6a100ce880 con 0x7f6a100b1b10 2024-05-22T21:20:38.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.846+0000 7f6a0cc00640 1 -- 172.21.15.161:0/534187991 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6a00054c20 con 0x7f6a100b1b10 2024-05-22T21:20:38.850 INFO:teuthology.orchestra.run.smithi161.stderr:updated caps for client.0 2024-05-22T21:20:38.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.850+0000 7f6a15600640 1 -- 172.21.15.161:0/534187991 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69ec0666c0 msgr2=0x7f69ec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:38.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.850+0000 7f6a15600640 1 --2- 172.21.15.161:0/534187991 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69ec0666c0 0x7f69ec068b80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f69f80af000 tx=0x7f69f80b6040 comp rx=0 tx=0).stop 2024-05-22T21:20:38.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.850+0000 7f6a15600640 1 -- 172.21.15.161:0/534187991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a100b1b10 msgr2=0x7f6a10157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:38.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.850+0000 7f6a15600640 1 --2- 172.21.15.161:0/534187991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a100b1b10 0x7f6a10157ac0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000c7c0 tx=0x7f6a0000cc80 comp rx=0 tx=0).stop 2024-05-22T21:20:38.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.850+0000 7f6a15600640 1 -- 172.21.15.161:0/534187991 shutdown_connections 2024-05-22T21:20:38.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.850+0000 7f6a15600640 1 --2- 172.21.15.161:0/534187991 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69ec0666c0 0x7f69ec068b80 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.850+0000 7f6a15600640 1 --2- 172.21.15.161:0/534187991 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a101436d0 0x7f6a10158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.850+0000 7f6a15600640 1 --2- 172.21.15.161:0/534187991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a100b1b10 0x7f6a10157ac0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.850+0000 7f6a15600640 1 --2- 172.21.15.161:0/534187991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a1006fa80 0x7f6a10152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:38.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.850+0000 7f6a15600640 1 -- 172.21.15.161:0/534187991 >> 172.21.15.161:0/534187991 conn(0x7f6a100cec00 msgr2=0x7f6a1013d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:38.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.850+0000 7f6a15600640 1 -- 172.21.15.161:0/534187991 shutdown_connections 2024-05-22T21:20:38.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:38.850+0000 7f6a15600640 1 -- 172.21.15.161:0/534187991 wait complete. 2024-05-22T21:20:38.872 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:20:39.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.006+0000 7fea7f800640 1 Processor -- start 2024-05-22T21:20:39.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.006+0000 7fea7f800640 1 -- start start 2024-05-22T21:20:39.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.006+0000 7fea7f800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fea7806bfb0 0x7fea780cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.006+0000 7fea7f800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea7806c440 con 0x7fea7815d4a0 2024-05-22T21:20:39.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.006+0000 7fea7f800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea780cf190 con 0x7fea780b0490 2024-05-22T21:20:39.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.006+0000 7fea7f800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea7815d880 con 0x7fea7806bfb0 2024-05-22T21:20:39.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.006+0000 7fea7ce00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fea7815d4a0 0x7fea78099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60536/0 (socket says 172.21.15.161:60536) 2024-05-22T21:20:39.010 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.006+0000 7fea7ce00640 1 -- 172.21.15.161:0/2493411810 learned_addr learned my addr 172.21.15.161:0/2493411810 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:39.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.006+0000 7fea7d800640 1 --2- 172.21.15.161:0/2493411810 >> v2:172.21.15.161:3300/0 conn(0x7fea7806bfb0 0x7fea780cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.006+0000 7fea77400640 1 -- 172.21.15.161:0/2493411810 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240051647 0 0) 0x7fea6c001120 con 0x7fea7815d4a0 2024-05-22T21:20:39.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.006+0000 7fea7d800640 1 -- 172.21.15.161:0/2493411810 >> v1:172.21.15.179:6789/0 conn(0x7fea780b0490 legacy=0x7fea780cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.006+0000 7fea7d800640 1 -- 172.21.15.161:0/2493411810 >> v1:172.21.15.161:6789/0 conn(0x7fea7815d4a0 legacy=0x7fea78099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.006+0000 7fea7d800640 1 -- 172.21.15.161:0/2493411810 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7806b160 con 0x7fea7806bfb0 2024-05-22T21:20:39.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.010+0000 7fea7d800640 1 --2- 172.21.15.161:0/2493411810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea7806bfb0 0x7fea780cfd80 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fea70009d90 tx=0x7fea7001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fc4b62ed81d821 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:39.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.010+0000 7fea77400640 1 -- 172.21.15.161:0/2493411810 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea700a5940 con 0x7fea7806bfb0 2024-05-22T21:20:39.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.010+0000 7fea77400640 1 -- 172.21.15.161:0/2493411810 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea700a5aa0 con 0x7fea7806bfb0 2024-05-22T21:20:39.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.010+0000 7fea77400640 1 -- 172.21.15.161:0/2493411810 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea700a5e30 con 0x7fea7806bfb0 2024-05-22T21:20:39.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.010+0000 7fea7f800640 1 -- 172.21.15.161:0/2493411810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea7806bfb0 msgr2=0x7fea780cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.010+0000 7fea7f800640 1 --2- 172.21.15.161:0/2493411810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea7806bfb0 0x7fea780cfd80 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fea70009d90 tx=0x7fea7001c970 comp rx=0 tx=0).stop 2024-05-22T21:20:39.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.010+0000 7fea7f800640 1 -- 172.21.15.161:0/2493411810 shutdown_connections 2024-05-22T21:20:39.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.010+0000 7fea7f800640 1 --2- 172.21.15.161:0/2493411810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea7806bfb0 0x7fea780cfd80 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.010+0000 7fea7f800640 1 -- 172.21.15.161:0/2493411810 >> 172.21.15.161:0/2493411810 conn(0x7fea780b2a40 msgr2=0x7fea780b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:39.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.010+0000 7fea7f800640 1 -- 172.21.15.161:0/2493411810 shutdown_connections 2024-05-22T21:20:39.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.010+0000 7fea7f800640 1 -- 172.21.15.161:0/2493411810 wait complete. 2024-05-22T21:20:39.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7f800640 1 Processor -- start 2024-05-22T21:20:39.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7f800640 1 -- start start 2024-05-22T21:20:39.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7f800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea780b0490 0x7fea780b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7f800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea7815d4a0 0x7fea780b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7f800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea780b3fb0 0x7fea7814d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7f800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea780c9390 con 0x7fea780b0490 2024-05-22T21:20:39.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7f800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea780c8d10 con 0x7fea7815d4a0 2024-05-22T21:20:39.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7f800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea780b4510 con 0x7fea780b3fb0 2024-05-22T21:20:39.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea780b0490 0x7fea780b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea780b0490 0x7fea780b60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32904/0 (socket says 172.21.15.161:32904) 2024-05-22T21:20:39.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7ce00640 1 -- 172.21.15.161:0/3754429411 learned_addr learned my addr 172.21.15.161:0/3754429411 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:39.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea77e00640 1 --2- 172.21.15.161:0/3754429411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea7815d4a0 0x7fea780b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7d800640 1 --2- 172.21.15.161:0/3754429411 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea780b3fb0 0x7fea7814d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7ce00640 1 -- 172.21.15.161:0/3754429411 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea780b3fb0 msgr2=0x7fea7814d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7ce00640 1 --2- 172.21.15.161:0/3754429411 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea780b3fb0 0x7fea7814d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7ce00640 1 -- 172.21.15.161:0/3754429411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea7815d4a0 msgr2=0x7fea780b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7ce00640 1 --2- 172.21.15.161:0/3754429411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea7815d4a0 0x7fea780b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7ce00640 1 -- 172.21.15.161:0/3754429411 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea70009a70 con 0x7fea780b0490 2024-05-22T21:20:39.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7ce00640 1 --2- 172.21.15.161:0/3754429411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea780b0490 0x7fea780b60d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00c780 tx=0x7fea6c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:39.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea74c00640 1 -- 172.21.15.161:0/3754429411 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0072d0 con 0x7fea780b0490 2024-05-22T21:20:39.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea74c00640 1 -- 172.21.15.161:0/3754429411 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6c007430 con 0x7fea780b0490 2024-05-22T21:20:39.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea74c00640 1 -- 172.21.15.161:0/3754429411 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c001d20 con 0x7fea780b0490 2024-05-22T21:20:39.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7f800640 1 -- 172.21.15.161:0/3754429411 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea780c8fb0 con 0x7fea780b0490 2024-05-22T21:20:39.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7f800640 1 -- 172.21.15.161:0/3754429411 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea780b47f0 con 0x7fea780b0490 2024-05-22T21:20:39.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.014+0000 7fea7f800640 1 -- 172.21.15.161:0/3754429411 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea780c94d0 con 0x7fea780b0490 2024-05-22T21:20:39.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.018+0000 7fea74c00640 1 -- 172.21.15.161:0/3754429411 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fea6c0047f0 con 0x7fea780b0490 2024-05-22T21:20:39.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.018+0000 7fea74c00640 1 --2- 172.21.15.161:0/3754429411 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea58066690 0x7fea58068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.018+0000 7fea74c00640 1 -- 172.21.15.161:0/3754429411 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea6c0833d0 con 0x7fea780b0490 2024-05-22T21:20:39.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.018+0000 7fea77e00640 1 --2- 172.21.15.161:0/3754429411 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea58066690 0x7fea58068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.018+0000 7fea77e00640 1 --2- 172.21.15.161:0/3754429411 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea58066690 0x7fea58068b50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fea78154410 tx=0x7fea64009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:39.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.022+0000 7fea74c00640 1 -- 172.21.15.161:0/3754429411 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fea780c94d0 con 0x7fea780b0490 2024-05-22T21:20:39.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.206+0000 7fea7f800640 1 -- 172.21.15.161:0/3754429411 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea7806c440 con 0x7fea780b0490 2024-05-22T21:20:39.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.206+0000 7fea74c00640 1 -- 172.21.15.161:0/3754429411 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4736 (secure 0 0 0) 0x7fea7806c440 con 0x7fea780b0490 2024-05-22T21:20:39.210 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:39.210 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":70,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7992,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/207877183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":207877183},{"type":"v1","addr":"172.21.15.161:6836","nonce":207877183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2152702106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2152702106},{"type":"v1","addr":"172.21.15.161:6837","nonce":2152702106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/749927648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":749927648},{"type":"v1","addr":"172.21.15.179:6834","nonce":749927648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:20:33.964441+0000","modified":"2024-05-22T21:20:36.988238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8004},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8004":{"gid":8004,"name":"b","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/93167194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":93167194},{"type":"v1","addr":"172.21.15.179:6835","nonce":93167194}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-05-22T21:20:39.211 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 70 2024-05-22T21:20:39.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.210+0000 7fea7f800640 1 -- 172.21.15.161:0/3754429411 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea58066690 msgr2=0x7fea58068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.210+0000 7fea7f800640 1 --2- 172.21.15.161:0/3754429411 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea58066690 0x7fea58068b50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fea78154410 tx=0x7fea64009040 comp rx=0 tx=0).stop 2024-05-22T21:20:39.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.210+0000 7fea7f800640 1 -- 172.21.15.161:0/3754429411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea780b0490 msgr2=0x7fea780b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.210+0000 7fea7f800640 1 --2- 172.21.15.161:0/3754429411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea780b0490 0x7fea780b60d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00c780 tx=0x7fea6c00cc40 comp rx=0 tx=0).stop 2024-05-22T21:20:39.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.210+0000 7fea7f800640 1 -- 172.21.15.161:0/3754429411 shutdown_connections 2024-05-22T21:20:39.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.210+0000 7fea7f800640 1 --2- 172.21.15.161:0/3754429411 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea58066690 0x7fea58068b50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.210+0000 7fea7f800640 1 --2- 172.21.15.161:0/3754429411 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea780b3fb0 0x7fea7814d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.210+0000 7fea7f800640 1 --2- 172.21.15.161:0/3754429411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea7815d4a0 0x7fea780b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.210+0000 7fea7f800640 1 --2- 172.21.15.161:0/3754429411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea780b0490 0x7fea780b60d0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.210+0000 7fea7f800640 1 -- 172.21.15.161:0/3754429411 >> 172.21.15.161:0/3754429411 conn(0x7fea780b2a40 msgr2=0x7fea780cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:39.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.210+0000 7fea7f800640 1 -- 172.21.15.161:0/3754429411 shutdown_connections 2024-05-22T21:20:39.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.210+0000 7fea7f800640 1 -- 172.21.15.161:0/3754429411 wait complete. 2024-05-22T21:20:39.635 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-22T21:20:33.964441+0000', 'modified': '2024-05-22T21:20:36.988238+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8004}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8004': {'gid': 8004, 'name': 'b', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.179:6835/93167194', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6833', 'nonce': 93167194}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 93167194}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-22T21:20:39.635 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-22T21:20:39.636 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-22T21:20:39.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.770+0000 7f2b7f000640 1 Processor -- start 2024-05-22T21:20:39.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.770+0000 7f2b7f000640 1 -- start start 2024-05-22T21:20:39.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.770+0000 7f2b7f000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b7815d400 0x7f2b780cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.770+0000 7f2b7f000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2b7806edb0 0x7f2b780cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.770+0000 7f2b7f000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b780d1c00 con 0x7f2b780cdce0 2024-05-22T21:20:39.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.770+0000 7f2b7f000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b780ce170 con 0x7f2b7806edb0 2024-05-22T21:20:39.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.770+0000 7f2b7f000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7806e920 con 0x7f2b7815d400 2024-05-22T21:20:39.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.770+0000 7f2b77400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2b7806edb0 0x7f2b780cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.770+0000 7f2b77400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2b7806edb0 0x7f2b780cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32914/0 (socket says 172.21.15.161:32914) 2024-05-22T21:20:39.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.770+0000 7f2b77400640 1 -- 172.21.15.161:0/1646348307 learned_addr learned my addr 172.21.15.161:0/1646348307 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:39.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.770+0000 7f2b77e00640 1 --2- 172.21.15.161:0/1646348307 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b7815d400 0x7f2b780cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.770+0000 7f2b77400640 1 -- 172.21.15.161:0/1646348307 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b7815d400 msgr2=0x7f2b780cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:39.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.770+0000 7f2b77400640 1 --2- 172.21.15.161:0/1646348307 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b7815d400 0x7f2b780cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.770+0000 7f2b77400640 1 -- 172.21.15.161:0/1646348307 >> v1:172.21.15.161:6789/0 conn(0x7f2b780cdce0 legacy=0x7f2b780cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.770+0000 7f2b77400640 1 -- 172.21.15.161:0/1646348307 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b780d0180 con 0x7f2b7806edb0 2024-05-22T21:20:39.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.770+0000 7f2b76a00640 1 -- 172.21.15.161:0/1646348307 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204485658 0 0) 0x7f2b780d1c00 con 0x7f2b780cdce0 2024-05-22T21:20:39.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.774+0000 7f2b77400640 1 --2- 172.21.15.161:0/1646348307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b7806edb0 0x7f2b780cd7a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009870 tx=0x7f2b6401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb14e68e461ce1b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:39.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.774+0000 7f2b76a00640 1 -- 172.21.15.161:0/1646348307 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b640ac410 con 0x7f2b7806edb0 2024-05-22T21:20:39.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.774+0000 7f2b76a00640 1 -- 172.21.15.161:0/1646348307 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b640ac570 con 0x7f2b7806edb0 2024-05-22T21:20:39.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.774+0000 7f2b76a00640 1 -- 172.21.15.161:0/1646348307 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b640ac900 con 0x7f2b7806edb0 2024-05-22T21:20:39.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.774+0000 7f2b7f000640 1 -- 172.21.15.161:0/1646348307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b7806edb0 msgr2=0x7f2b780cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.774+0000 7f2b7f000640 1 --2- 172.21.15.161:0/1646348307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b7806edb0 0x7f2b780cd7a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009870 tx=0x7f2b6401c4a0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.774+0000 7f2b7f000640 1 -- 172.21.15.161:0/1646348307 shutdown_connections 2024-05-22T21:20:39.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.774+0000 7f2b7f000640 1 --2- 172.21.15.161:0/1646348307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b7806edb0 0x7f2b780cd7a0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.774+0000 7f2b7f000640 1 --2- 172.21.15.161:0/1646348307 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b7815d400 0x7f2b780cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.774+0000 7f2b7f000640 1 -- 172.21.15.161:0/1646348307 >> 172.21.15.161:0/1646348307 conn(0x7f2b780d0f20 msgr2=0x7f2b780d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:39.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.774+0000 7f2b7f000640 1 -- 172.21.15.161:0/1646348307 shutdown_connections 2024-05-22T21:20:39.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.774+0000 7f2b7f000640 1 -- 172.21.15.161:0/1646348307 wait complete. 2024-05-22T21:20:39.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b7f000640 1 Processor -- start 2024-05-22T21:20:39.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b7f000640 1 -- start start 2024-05-22T21:20:39.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b7f000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b7806edb0 0x7f2b7814f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b7f000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b780cdce0 0x7f2b780b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b7f000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b7815d400 0x7f2b780b6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b7f000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b780b6a80 con 0x7f2b7806edb0 2024-05-22T21:20:39.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b7f000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b780b4bc0 con 0x7f2b780cdce0 2024-05-22T21:20:39.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b7f000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b780b4920 con 0x7f2b7815d400 2024-05-22T21:20:39.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b77e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b7806edb0 0x7f2b7814f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b77e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b7806edb0 0x7f2b7814f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32920/0 (socket says 172.21.15.161:32920) 2024-05-22T21:20:39.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b77e00640 1 -- 172.21.15.161:0/2733869914 learned_addr learned my addr 172.21.15.161:0/2733869914 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:39.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b77e00640 1 -- 172.21.15.161:0/2733869914 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b7815d400 msgr2=0x7f2b780b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b77e00640 1 --2- 172.21.15.161:0/2733869914 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b7815d400 0x7f2b780b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b77e00640 1 -- 172.21.15.161:0/2733869914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b780cdce0 msgr2=0x7f2b780b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b77400640 1 --2- 172.21.15.161:0/2733869914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b780cdce0 0x7f2b780b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b77e00640 1 --2- 172.21.15.161:0/2733869914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b780cdce0 0x7f2b780b6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b77e00640 1 -- 172.21.15.161:0/2733869914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b640094d0 con 0x7f2b7806edb0 2024-05-22T21:20:39.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b77e00640 1 --2- 172.21.15.161:0/2733869914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b7806edb0 0x7f2b7814f430 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f2b68007ef0 tx=0x7f2b6800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:39.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b6fe00640 1 -- 172.21.15.161:0/2733869914 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b68014620 con 0x7f2b7806edb0 2024-05-22T21:20:39.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b6fe00640 1 -- 172.21.15.161:0/2733869914 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b68014780 con 0x7f2b7806edb0 2024-05-22T21:20:39.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b6fe00640 1 -- 172.21.15.161:0/2733869914 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b68014b30 con 0x7f2b7806edb0 2024-05-22T21:20:39.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b7f000640 1 -- 172.21.15.161:0/2733869914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b780c8af0 con 0x7f2b7806edb0 2024-05-22T21:20:39.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.778+0000 7f2b7f000640 1 -- 172.21.15.161:0/2733869914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b781607d0 con 0x7f2b7806edb0 2024-05-22T21:20:39.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.782+0000 7f2b6fe00640 1 -- 172.21.15.161:0/2733869914 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2b68014c90 con 0x7f2b7806edb0 2024-05-22T21:20:39.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.782+0000 7f2b6fe00640 1 --2- 172.21.15.161:0/2733869914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2b58066690 0x7f2b58068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.782+0000 7f2b6fe00640 1 -- 172.21.15.161:0/2733869914 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b68084b60 con 0x7f2b7806edb0 2024-05-22T21:20:39.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.782+0000 7f2b77400640 1 --2- 172.21.15.161:0/2733869914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2b58066690 0x7f2b58068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.782+0000 7f2b82495000 1 Processor -- start 2024-05-22T21:20:39.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.782+0000 7f2b82495000 1 -- start start 2024-05-22T21:20:39.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.782+0000 7f2b77400640 1 --2- 172.21.15.161:0/2733869914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2b58066690 0x7f2b58068b50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2b640ae000 tx=0x7f2b640a6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:39.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.782+0000 7f2b82495000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4f39c0 0x55cd9c4f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.782+0000 7f2b82495000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f42e0 0x55cd9c4a9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.782+0000 7f2b82495000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4a9ec0 0x55cd9c4aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.782+0000 7f2b82495000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9c371e80 con 0x55cd9c4f42e0 2024-05-22T21:20:39.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.782+0000 7f2b82495000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9c2ffaf0 con 0x55cd9c4a9ec0 2024-05-22T21:20:39.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.782+0000 7f2b82495000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd9c2fd0c0 con 0x55cd9c4f39c0 2024-05-22T21:20:39.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.782+0000 7f2b77400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f42e0 0x55cd9c4a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.782+0000 7f2b77400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f42e0 0x55cd9c4a9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32924/0 (socket says 172.21.15.161:32924) 2024-05-22T21:20:39.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.782+0000 7f2b77400640 1 -- 172.21.15.161:0/466412016 learned_addr learned my addr 172.21.15.161:0/466412016 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:39.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b77e00640 1 --2- 172.21.15.161:0/466412016 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4f39c0 0x55cd9c4f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b77400640 1 -- 172.21.15.161:0/466412016 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4f39c0 msgr2=0x55cd9c4f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:39.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b7d000640 1 --2- 172.21.15.161:0/466412016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4a9ec0 0x55cd9c4aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b77400640 1 --2- 172.21.15.161:0/466412016 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4f39c0 0x55cd9c4f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b77400640 1 -- 172.21.15.161:0/466412016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4a9ec0 msgr2=0x55cd9c4aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:39.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b77400640 1 --2- 172.21.15.161:0/466412016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4a9ec0 0x55cd9c4aa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b77400640 1 -- 172.21.15.161:0/466412016 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd9bff4890 con 0x55cd9c4f42e0 2024-05-22T21:20:39.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b77e00640 1 --2- 172.21.15.161:0/466412016 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4f39c0 0x55cd9c4f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:39.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b7d000640 1 --2- 172.21.15.161:0/466412016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4a9ec0 0x55cd9c4aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:39.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b77400640 1 --2- 172.21.15.161:0/466412016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f42e0 0x55cd9c4a9980 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f2b640af040 tx=0x7f2b64004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865aec84dd528128 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:39.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b6cc00640 1 -- 172.21.15.161:0/466412016 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b640bd070 con 0x55cd9c4f42e0 2024-05-22T21:20:39.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b6cc00640 1 -- 172.21.15.161:0/466412016 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b640b8620 con 0x55cd9c4f42e0 2024-05-22T21:20:39.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b6cc00640 1 -- 172.21.15.161:0/466412016 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b640b87b0 con 0x55cd9c4f42e0 2024-05-22T21:20:39.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 -- 172.21.15.161:0/466412016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f42e0 msgr2=0x55cd9c4a9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 --2- 172.21.15.161:0/466412016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f42e0 0x55cd9c4a9980 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f2b640af040 tx=0x7f2b64004740 comp rx=0 tx=0).stop 2024-05-22T21:20:39.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 -- 172.21.15.161:0/466412016 shutdown_connections 2024-05-22T21:20:39.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 --2- 172.21.15.161:0/466412016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4a9ec0 0x55cd9c4aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 --2- 172.21.15.161:0/466412016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f42e0 0x55cd9c4a9980 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 --2- 172.21.15.161:0/466412016 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4f39c0 0x55cd9c4f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 -- 172.21.15.161:0/466412016 >> 172.21.15.161:0/466412016 conn(0x55cd9c4b0440 msgr2=0x55cd9c4a8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:39.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 -- 172.21.15.161:0/466412016 shutdown_connections 2024-05-22T21:20:39.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 -- 172.21.15.161:0/466412016 wait complete. 2024-05-22T21:20:39.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 -- start start 2024-05-22T21:20:39.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 Processor -- start 2024-05-22T21:20:39.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4a9ec0 0x55cd9c4fab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4f39c0 0x55cd9c4fb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4f42e0 0x55cd9c4b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9c2f89b0 con 0x55cd9c4a9ec0 2024-05-22T21:20:39.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9c297c00 con 0x55cd9c4f42e0 2024-05-22T21:20:39.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd9c2b6070 con 0x55cd9c4f39c0 2024-05-22T21:20:39.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b82495000 10 client.0 _ll_register_callbacks cb 0x55cd9c495ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:20:39.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b77e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4a9ec0 0x55cd9c4fab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b77e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4a9ec0 0x55cd9c4fab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32938/0 (socket says 172.21.15.161:32938) 2024-05-22T21:20:39.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b77e00640 1 -- 172.21.15.161:0/2917354449 learned_addr learned my addr 172.21.15.161:0/2917354449 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:39.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b77e00640 1 -- 172.21.15.161:0/2917354449 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4f39c0 msgr2=0x55cd9c4fb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:39.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b77e00640 1 --2- 172.21.15.161:0/2917354449 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4f39c0 0x55cd9c4fb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b77e00640 1 -- 172.21.15.161:0/2917354449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4f42e0 msgr2=0x55cd9c4b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:39.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b77e00640 1 --2- 172.21.15.161:0/2917354449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4f42e0 0x55cd9c4b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.786+0000 7f2b77e00640 1 -- 172.21.15.161:0/2917354449 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b780d0180 con 0x55cd9c4a9ec0 2024-05-22T21:20:39.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b77e00640 1 --2- 172.21.15.161:0/2917354449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4a9ec0 0x55cd9c4fab10 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f2b68083140 tx=0x7f2b68055c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4456f60fc10e5bad server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:39.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b61600640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:20:39.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b61600640 1 -- 172.21.15.161:0/2917354449 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6806d070 con 0x55cd9c4a9ec0 2024-05-22T21:20:39.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b61600640 1 -- 172.21.15.161:0/2917354449 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(158..158 src has 1..158) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b68055e80 con 0x55cd9c4a9ec0 2024-05-22T21:20:39.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 1 -- 172.21.15.161:0/2917354449 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd9c2b55f0 con 0x55cd9c4a9ec0 2024-05-22T21:20:39.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b61600640 1 -- 172.21.15.161:0/2917354449 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b68056040 con 0x55cd9c4a9ec0 2024-05-22T21:20:39.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 10 client.8139 fetch_fsmap learned FSMap version 70 2024-05-22T21:20:39.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 1 -- 172.21.15.161:0/2917354449 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x55cd9c2bdab0 con 0x55cd9c4a9ec0 2024-05-22T21:20:39.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b61600640 1 -- 172.21.15.161:0/2917354449 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd9c2b55f0 con 0x55cd9c4a9ec0 2024-05-22T21:20:39.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b61600640 1 -- 172.21.15.161:0/2917354449 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 70) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2b68005d20 con 0x55cd9c4a9ec0 2024-05-22T21:20:39.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 10 client.8139 fetch_fsmap finished waiting for FSMap version 70 2024-05-22T21:20:39.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 10 client.8139 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-22T21:20:39.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 20 client.8139 populate_metadata read hostname 'smithi161' 2024-05-22T21:20:39.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 1 --2- 172.21.15.161:0/2917354449 >> [v2:172.21.15.179:6833/93167194,v1:172.21.15.179:6835/93167194] conn(0x55cd9c4b5d20 0x55cd9c4b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 4 client.8139 mds_command: new command op to 8004 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T21:20:39.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 1 -- 172.21.15.161:0/2917354449 --> [v2:172.21.15.179:6833/93167194,v1:172.21.15.179:6835/93167194] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd9c2f5790 con 0x55cd9c4b5d20 2024-05-22T21:20:39.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b77400640 1 --2- 172.21.15.161:0/2917354449 >> [v2:172.21.15.179:6833/93167194,v1:172.21.15.179:6835/93167194] conn(0x55cd9c4b5d20 0x55cd9c4b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b77400640 1 --2- 172.21.15.161:0/2917354449 >> [v2:172.21.15.179:6833/93167194,v1:172.21.15.179:6835/93167194] conn(0x55cd9c4b5d20 0x55cd9c4b6100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4badc155ca5654a server_cookie=9cbddb6c762c5f2f in_seq=0 out_seq=0 2024-05-22T21:20:39.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b61600640 10 client.8139 ms_handle_connect on v2:172.21.15.179:6833/93167194 2024-05-22T21:20:39.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b61600640 1 -- 172.21.15.161:0/2917354449 <== mds.0 v2:172.21.15.179:6833/93167194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cd9c2f5790 con 0x55cd9c4b5d20 2024-05-22T21:20:39.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b61600640 10 client.8139 handle_command_reply: tid=0 2024-05-22T21:20:39.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 1 client.8139 shutdown 2024-05-22T21:20:39.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 1 -- 172.21.15.161:0/2917354449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4a9ec0 msgr2=0x55cd9c4fab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 1 --2- 172.21.15.161:0/2917354449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4a9ec0 0x55cd9c4fab10 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f2b68083140 tx=0x7f2b68055c20 comp rx=0 tx=0).stop 2024-05-22T21:20:39.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 1 -- 172.21.15.161:0/2917354449 shutdown_connections 2024-05-22T21:20:39.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 1 --2- 172.21.15.161:0/2917354449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4f42e0 0x55cd9c4b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 1 --2- 172.21.15.161:0/2917354449 >> [v2:172.21.15.179:6833/93167194,v1:172.21.15.179:6835/93167194] conn(0x55cd9c4b5d20 0x55cd9c4b6100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 1 --2- 172.21.15.161:0/2917354449 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4f39c0 0x55cd9c4fb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 1 --2- 172.21.15.161:0/2917354449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4a9ec0 0x55cd9c4fab10 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b61600640 0 client.8139 ms_handle_reset on v2:172.21.15.179:6833/93167194 2024-05-22T21:20:39.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 1 -- 172.21.15.161:0/2917354449 >> 172.21.15.161:0/2917354449 conn(0x55cd9c4b0440 msgr2=0x55cd9c4a8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:39.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 1 -- 172.21.15.161:0/2917354449 shutdown_connections 2024-05-22T21:20:39.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 1 -- 172.21.15.161:0/2917354449 wait complete. 2024-05-22T21:20:39.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.790+0000 7f2b82495000 20 client.8139 trim_cache size 0 max 16384 2024-05-22T21:20:39.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.806+0000 7f2b82495000 1 Processor -- start 2024-05-22T21:20:39.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.806+0000 7f2b82495000 1 -- start start 2024-05-22T21:20:39.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.806+0000 7f2b82495000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4a9ec0 0x55cd9c4af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.806+0000 7f2b82495000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f39c0 0x55cd9c4afa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.806+0000 7f2b82495000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4f42e0 0x55cd9c4f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.806+0000 7f2b82495000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9bfb5940 con 0x55cd9c4f39c0 2024-05-22T21:20:39.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.806+0000 7f2b82495000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9c2fc9b0 con 0x55cd9c4f42e0 2024-05-22T21:20:39.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.806+0000 7f2b82495000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd9c2f41a0 con 0x55cd9c4a9ec0 2024-05-22T21:20:39.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f39c0 0x55cd9c4afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f39c0 0x55cd9c4afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32944/0 (socket says 172.21.15.161:32944) 2024-05-22T21:20:39.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 -- 172.21.15.161:0/3994591667 learned_addr learned my addr 172.21.15.161:0/3994591667 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:39.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77e00640 1 --2- 172.21.15.161:0/3994591667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4a9ec0 0x55cd9c4af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b7d000640 1 --2- 172.21.15.161:0/3994591667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4f42e0 0x55cd9c4f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 -- 172.21.15.161:0/3994591667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4a9ec0 msgr2=0x55cd9c4af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:39.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 --2- 172.21.15.161:0/3994591667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4a9ec0 0x55cd9c4af4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 -- 172.21.15.161:0/3994591667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4f42e0 msgr2=0x55cd9c4f29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:39.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 --2- 172.21.15.161:0/3994591667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4f42e0 0x55cd9c4f29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 -- 172.21.15.161:0/3994591667 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd9c2a3c00 con 0x55cd9c4f39c0 2024-05-22T21:20:39.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b7d000640 1 --2- 172.21.15.161:0/3994591667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4f42e0 0x55cd9c4f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:39.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 --2- 172.21.15.161:0/3994591667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f39c0 0x55cd9c4afa00 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f2b640af040 tx=0x7f2b640046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9538b7752b08bc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:39.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b6d600640 1 -- 172.21.15.161:0/3994591667 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b640bd040 con 0x55cd9c4f39c0 2024-05-22T21:20:39.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b6d600640 1 -- 172.21.15.161:0/3994591667 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b64004200 con 0x55cd9c4f39c0 2024-05-22T21:20:39.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b6d600640 1 -- 172.21.15.161:0/3994591667 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b640b8620 con 0x55cd9c4f39c0 2024-05-22T21:20:39.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 -- 172.21.15.161:0/3994591667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f39c0 msgr2=0x55cd9c4afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 --2- 172.21.15.161:0/3994591667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f39c0 0x55cd9c4afa00 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f2b640af040 tx=0x7f2b640046d0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 -- 172.21.15.161:0/3994591667 shutdown_connections 2024-05-22T21:20:39.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 --2- 172.21.15.161:0/3994591667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4f42e0 0x55cd9c4f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 --2- 172.21.15.161:0/3994591667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f39c0 0x55cd9c4afa00 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 --2- 172.21.15.161:0/3994591667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4a9ec0 0x55cd9c4af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 -- 172.21.15.161:0/3994591667 >> 172.21.15.161:0/3994591667 conn(0x55cd9c4b0440 msgr2=0x55cd9c4f22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:39.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 -- 172.21.15.161:0/3994591667 shutdown_connections 2024-05-22T21:20:39.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 -- 172.21.15.161:0/3994591667 wait complete. 2024-05-22T21:20:39.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 -- start start 2024-05-22T21:20:39.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 Processor -- start 2024-05-22T21:20:39.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4a9ec0 0x55cd9c551990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f39c0 0x55cd9c551ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4f42e0 0x55cd9c552410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9c29c860 con 0x55cd9c4f39c0 2024-05-22T21:20:39.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9c4a16d0 con 0x55cd9c4f42e0 2024-05-22T21:20:39.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd9c2fc290 con 0x55cd9c4a9ec0 2024-05-22T21:20:39.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b82495000 10 client.0 _ll_register_callbacks cb 0x7f2b640af070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:20:39.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f39c0 0x55cd9c551ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f39c0 0x55cd9c551ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32960/0 (socket says 172.21.15.161:32960) 2024-05-22T21:20:39.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 -- 172.21.15.161:0/2043818897 learned_addr learned my addr 172.21.15.161:0/2043818897 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:39.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77e00640 1 --2- 172.21.15.161:0/2043818897 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4a9ec0 0x55cd9c551990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b7d000640 1 --2- 172.21.15.161:0/2043818897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4f42e0 0x55cd9c552410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 -- 172.21.15.161:0/2043818897 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4a9ec0 msgr2=0x55cd9c551990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:39.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 --2- 172.21.15.161:0/2043818897 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4a9ec0 0x55cd9c551990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 -- 172.21.15.161:0/2043818897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4f42e0 msgr2=0x55cd9c552410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:39.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 --2- 172.21.15.161:0/2043818897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4f42e0 0x55cd9c552410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.810+0000 7f2b77400640 1 -- 172.21.15.161:0/2043818897 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd9bff4890 con 0x55cd9c4f39c0 2024-05-22T21:20:39.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b77400640 1 --2- 172.21.15.161:0/2043818897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f39c0 0x55cd9c551ed0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f2b6401c980 tx=0x7f2b6401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5adad2273ccc59b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:39.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b61600640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:20:39.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b61600640 1 -- 172.21.15.161:0/2043818897 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b640bd070 con 0x55cd9c4f39c0 2024-05-22T21:20:39.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b61600640 1 -- 172.21.15.161:0/2043818897 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(158..158 src has 1..158) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b64004460 con 0x55cd9c4f39c0 2024-05-22T21:20:39.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 1 -- 172.21.15.161:0/2043818897 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd9c2c8460 con 0x55cd9c4f39c0 2024-05-22T21:20:39.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b61600640 1 -- 172.21.15.161:0/2043818897 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b640b8620 con 0x55cd9c4f39c0 2024-05-22T21:20:39.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 10 client.8145 fetch_fsmap learned FSMap version 70 2024-05-22T21:20:39.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 1 -- 172.21.15.161:0/2043818897 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x55cd9bff4cc0 con 0x55cd9c4f39c0 2024-05-22T21:20:39.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b61600640 1 -- 172.21.15.161:0/2043818897 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd9c2c8460 con 0x55cd9c4f39c0 2024-05-22T21:20:39.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b61600640 1 -- 172.21.15.161:0/2043818897 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 70) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2b640b8780 con 0x55cd9c4f39c0 2024-05-22T21:20:39.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 10 client.8145 fetch_fsmap finished waiting for FSMap version 70 2024-05-22T21:20:39.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 10 client.8145 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-22T21:20:39.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 20 client.8145 populate_metadata read hostname 'smithi161' 2024-05-22T21:20:39.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 1 --2- 172.21.15.161:0/2043818897 >> [v2:172.21.15.179:6833/93167194,v1:172.21.15.179:6835/93167194] conn(0x55cd9c552950 0x55cd9c55d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 4 client.8145 mds_command: new command op to 8004 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-22T21:20:39.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 1 -- 172.21.15.161:0/2043818897 --> [v2:172.21.15.179:6833/93167194,v1:172.21.15.179:6835/93167194] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cd9c2c19c0 con 0x55cd9c552950 2024-05-22T21:20:39.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b77e00640 1 --2- 172.21.15.161:0/2043818897 >> [v2:172.21.15.179:6833/93167194,v1:172.21.15.179:6835/93167194] conn(0x55cd9c552950 0x55cd9c55d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b61600640 10 client.8145 ms_handle_connect on v2:172.21.15.179:6833/93167194 2024-05-22T21:20:39.822 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b77e00640 1 --2- 172.21.15.161:0/2043818897 >> [v2:172.21.15.179:6833/93167194,v1:172.21.15.179:6835/93167194] conn(0x55cd9c552950 0x55cd9c55d140 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2edad361056c7066 server_cookie=7fdd9199d78fafeb in_seq=0 out_seq=0 2024-05-22T21:20:39.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b61600640 1 -- 172.21.15.161:0/2043818897 <== mds.0 v2:172.21.15.179:6833/93167194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55cd9c2c19c0 con 0x55cd9c552950 2024-05-22T21:20:39.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b61600640 10 client.8145 handle_command_reply: tid=0 2024-05-22T21:20:39.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 1 client.8145 shutdown 2024-05-22T21:20:39.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 1 -- 172.21.15.161:0/2043818897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f39c0 msgr2=0x55cd9c551ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 1 --2- 172.21.15.161:0/2043818897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f39c0 0x55cd9c551ed0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f2b6401c980 tx=0x7f2b6401ace0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 1 -- 172.21.15.161:0/2043818897 shutdown_connections 2024-05-22T21:20:39.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 1 --2- 172.21.15.161:0/2043818897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cd9c4f42e0 0x55cd9c552410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 1 --2- 172.21.15.161:0/2043818897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd9c4f39c0 0x55cd9c551ed0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 1 --2- 172.21.15.161:0/2043818897 >> [v2:172.21.15.179:6833/93167194,v1:172.21.15.179:6835/93167194] conn(0x55cd9c552950 0x55cd9c55d140 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 1 --2- 172.21.15.161:0/2043818897 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55cd9c4a9ec0 0x55cd9c551990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b61600640 0 client.8145 ms_handle_reset on v2:172.21.15.179:6833/93167194 2024-05-22T21:20:39.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 1 -- 172.21.15.161:0/2043818897 >> 172.21.15.161:0/2043818897 conn(0x55cd9c4b0440 msgr2=0x55cd9c4f2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:39.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 1 -- 172.21.15.161:0/2043818897 shutdown_connections 2024-05-22T21:20:39.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 1 -- 172.21.15.161:0/2043818897 wait complete. 2024-05-22T21:20:39.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b82495000 20 client.8145 trim_cache size 0 max 16384 2024-05-22T21:20:39.824 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:20:39.824 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:20:39.824 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 0, 2024-05-22T21:20:39.824 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 8004, 2024-05-22T21:20:39.825 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2024-05-22T21:20:39.825 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2024-05-22T21:20:39.825 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2024-05-22T21:20:39.825 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 5.8499313300000004, 2024-05-22T21:20:39.825 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 70, 2024-05-22T21:20:39.825 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 158, 2024-05-22T21:20:39.825 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 158, 2024-05-22T21:20:39.825 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 10.136357135000001 2024-05-22T21:20:39.825 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:20:39.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b7f000640 1 -- 172.21.15.161:0/2733869914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2b58066690 msgr2=0x7f2b58068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b7f000640 1 --2- 172.21.15.161:0/2733869914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2b58066690 0x7f2b58068b50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2b640ae000 tx=0x7f2b640a6040 comp rx=0 tx=0).stop 2024-05-22T21:20:39.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b7f000640 1 -- 172.21.15.161:0/2733869914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b7806edb0 msgr2=0x7f2b7814f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.826 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b7f000640 1 --2- 172.21.15.161:0/2733869914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b7806edb0 0x7f2b7814f430 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f2b68007ef0 tx=0x7f2b6800e4b0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b7f000640 1 -- 172.21.15.161:0/2733869914 shutdown_connections 2024-05-22T21:20:39.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b7f000640 1 --2- 172.21.15.161:0/2733869914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2b58066690 0x7f2b58068b50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b7f000640 1 --2- 172.21.15.161:0/2733869914 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2b7815d400 0x7f2b780b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b7f000640 1 --2- 172.21.15.161:0/2733869914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b780cdce0 0x7f2b780b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b7f000640 1 --2- 172.21.15.161:0/2733869914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b7806edb0 0x7f2b7814f430 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b7f000640 1 -- 172.21.15.161:0/2733869914 >> 172.21.15.161:0/2733869914 conn(0x7f2b780d0f20 msgr2=0x7f2b780af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:39.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b7f000640 1 -- 172.21.15.161:0/2733869914 shutdown_connections 2024-05-22T21:20:39.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.814+0000 7f2b7f000640 1 -- 172.21.15.161:0/2733869914 wait complete. 2024-05-22T21:20:39.838 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:20:39.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.970+0000 7f5a0b800640 1 Processor -- start 2024-05-22T21:20:39.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.970+0000 7f5a0b800640 1 -- start start 2024-05-22T21:20:39.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.970+0000 7f5a0b800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a04143ec0 0x7f5a0415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.970+0000 7f5a0b800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5a040715a0 0x7f5a0406fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.970+0000 7f5a0b800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a041442a0 con 0x7f5a0415ec90 2024-05-22T21:20:39.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.970+0000 7f5a0b800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a04070260 con 0x7f5a040715a0 2024-05-22T21:20:39.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.970+0000 7f5a0b800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a04070500 con 0x7f5a04143ec0 2024-05-22T21:20:39.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.970+0000 7f5a03e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5a040715a0 0x7f5a0406fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.970+0000 7f5a03e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5a040715a0 0x7f5a0406fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32962/0 (socket says 172.21.15.161:32962) 2024-05-22T21:20:39.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.970+0000 7f5a09800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f5a0415ec90 0x7f5a0406bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60550/0 (socket says 172.21.15.161:60550) 2024-05-22T21:20:39.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.970+0000 7f5a03e00640 1 -- 172.21.15.161:0/1384526963 learned_addr learned my addr 172.21.15.161:0/1384526963 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:39.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.970+0000 7f5a08e00640 1 --2- 172.21.15.161:0/1384526963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a04143ec0 0x7f5a0415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.970+0000 7f5a03e00640 1 -- 172.21.15.161:0/1384526963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a04143ec0 msgr2=0x7f5a0415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:39.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.974+0000 7f5a03e00640 1 --2- 172.21.15.161:0/1384526963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a04143ec0 0x7f5a0415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.974+0000 7f5a03e00640 1 -- 172.21.15.161:0/1384526963 >> v1:172.21.15.161:6789/0 conn(0x7f5a0415ec90 legacy=0x7f5a0406bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.974+0000 7f5a03400640 1 -- 172.21.15.161:0/1384526963 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1322780604 0 0) 0x7f5a041442a0 con 0x7f5a0415ec90 2024-05-22T21:20:39.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.974+0000 7f5a03e00640 1 -- 172.21.15.161:0/1384526963 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a040707e0 con 0x7f5a040715a0 2024-05-22T21:20:39.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.974+0000 7f5a03e00640 1 --2- 172.21.15.161:0/1384526963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a040715a0 0x7f5a0406fd20 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009870 tx=0x7f59f001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada7bd7b087575df server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:39.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.974+0000 7f5a03400640 1 -- 172.21.15.161:0/1384526963 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f00a95f0 con 0x7f5a040715a0 2024-05-22T21:20:39.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.974+0000 7f5a03400640 1 -- 172.21.15.161:0/1384526963 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f00a9750 con 0x7f5a040715a0 2024-05-22T21:20:39.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.974+0000 7f5a03400640 1 -- 172.21.15.161:0/1384526963 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f00a9ae0 con 0x7f5a040715a0 2024-05-22T21:20:39.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.974+0000 7f5a0b800640 1 -- 172.21.15.161:0/1384526963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a040715a0 msgr2=0x7f5a0406fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.974+0000 7f5a0b800640 1 --2- 172.21.15.161:0/1384526963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a040715a0 0x7f5a0406fd20 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009870 tx=0x7f59f001c530 comp rx=0 tx=0).stop 2024-05-22T21:20:39.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.974+0000 7f5a0b800640 1 -- 172.21.15.161:0/1384526963 shutdown_connections 2024-05-22T21:20:39.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.974+0000 7f5a0b800640 1 --2- 172.21.15.161:0/1384526963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a040715a0 0x7f5a0406fd20 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.974+0000 7f5a0b800640 1 --2- 172.21.15.161:0/1384526963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a04143ec0 0x7f5a0415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.974+0000 7f5a0b800640 1 -- 172.21.15.161:0/1384526963 >> 172.21.15.161:0/1384526963 conn(0x7f5a040b0bc0 msgr2=0x7f5a040b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:39.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.974+0000 7f5a0b800640 1 -- 172.21.15.161:0/1384526963 shutdown_connections 2024-05-22T21:20:39.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.974+0000 7f5a0b800640 1 -- 172.21.15.161:0/1384526963 wait complete. 2024-05-22T21:20:39.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a0b800640 1 Processor -- start 2024-05-22T21:20:39.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a0b800640 1 -- start start 2024-05-22T21:20:39.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a0b800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a04143ec0 0x7f5a041576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a0b800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0415ec90 0x7f5a04155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a0b800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a041559e0 0x7f5a04158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a0b800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a040af630 con 0x7f5a04143ec0 2024-05-22T21:20:39.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a0b800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a04142ab0 con 0x7f5a041559e0 2024-05-22T21:20:39.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a0b800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a04142530 con 0x7f5a0415ec90 2024-05-22T21:20:39.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a08e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a04143ec0 0x7f5a041576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a08e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a04143ec0 0x7f5a041576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32968/0 (socket says 172.21.15.161:32968) 2024-05-22T21:20:39.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a08e00640 1 -- 172.21.15.161:0/3074539273 learned_addr learned my addr 172.21.15.161:0/3074539273 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:39.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a03e00640 1 --2- 172.21.15.161:0/3074539273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0415ec90 0x7f5a04155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a09800640 1 --2- 172.21.15.161:0/3074539273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a041559e0 0x7f5a04158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a08e00640 1 -- 172.21.15.161:0/3074539273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0415ec90 msgr2=0x7f5a04155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a08e00640 1 --2- 172.21.15.161:0/3074539273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0415ec90 0x7f5a04155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a08e00640 1 -- 172.21.15.161:0/3074539273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a041559e0 msgr2=0x7f5a04158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:39.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a08e00640 1 --2- 172.21.15.161:0/3074539273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a041559e0 0x7f5a04158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:39.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a08e00640 1 -- 172.21.15.161:0/3074539273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f00094d0 con 0x7f5a04143ec0 2024-05-22T21:20:39.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a09800640 1 --2- 172.21.15.161:0/3074539273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a041559e0 0x7f5a04158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:20:39.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a08e00640 1 --2- 172.21.15.161:0/3074539273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a04143ec0 0x7f5a041576c0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f59f400d800 tx=0x7f59f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:39.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a00c00640 1 -- 172.21.15.161:0/3074539273 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f400b6c0 con 0x7f5a04143ec0 2024-05-22T21:20:39.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a00c00640 1 -- 172.21.15.161:0/3074539273 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f400b820 con 0x7f5a04143ec0 2024-05-22T21:20:39.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a00c00640 1 -- 172.21.15.161:0/3074539273 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f400bb90 con 0x7f5a04143ec0 2024-05-22T21:20:39.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a0b800640 1 -- 172.21.15.161:0/3074539273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a041427d0 con 0x7f5a04143ec0 2024-05-22T21:20:39.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.978+0000 7f5a0b800640 1 -- 172.21.15.161:0/3074539273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a04157c00 con 0x7f5a04143ec0 2024-05-22T21:20:39.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.982+0000 7f5a0b800640 1 -- 172.21.15.161:0/3074539273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a040af8a0 con 0x7f5a04143ec0 2024-05-22T21:20:39.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.982+0000 7f5a00c00640 1 -- 172.21.15.161:0/3074539273 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59f400bcf0 con 0x7f5a04143ec0 2024-05-22T21:20:39.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.982+0000 7f5a00c00640 1 --2- 172.21.15.161:0/3074539273 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59e4066690 0x7f59e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:39.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.982+0000 7f5a00c00640 1 -- 172.21.15.161:0/3074539273 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59f4084690 con 0x7f5a04143ec0 2024-05-22T21:20:39.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.982+0000 7f5a03e00640 1 --2- 172.21.15.161:0/3074539273 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59e4066690 0x7f59e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:39.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.982+0000 7f5a03e00640 1 --2- 172.21.15.161:0/3074539273 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59e4066690 0x7f59e4068b50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f59f00ae000 tx=0x7f59f00b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:39.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:39.986+0000 7f5a00c00640 1 -- 172.21.15.161:0/3074539273 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f59f4051a90 con 0x7f5a04143ec0 2024-05-22T21:20:40.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:40.174+0000 7f5a0b800640 1 -- 172.21.15.161:0/3074539273 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5a04157e50 con 0x7f5a04143ec0 2024-05-22T21:20:40.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:40.174+0000 7f5a00c00640 1 -- 172.21.15.161:0/3074539273 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) v1 ==== 53+0+73 (secure 0 0 0) 0x7f59f4011340 con 0x7f5a04143ec0 2024-05-22T21:20:40.179 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:20:40.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:40.178+0000 7f5a0b800640 1 -- 172.21.15.161:0/3074539273 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59e4066690 msgr2=0x7f59e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:40.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:40.178+0000 7f5a0b800640 1 --2- 172.21.15.161:0/3074539273 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59e4066690 0x7f59e4068b50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f59f00ae000 tx=0x7f59f00b3040 comp rx=0 tx=0).stop 2024-05-22T21:20:40.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:40.178+0000 7f5a0b800640 1 -- 172.21.15.161:0/3074539273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a04143ec0 msgr2=0x7f5a041576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:40.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:40.178+0000 7f5a0b800640 1 --2- 172.21.15.161:0/3074539273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a04143ec0 0x7f5a041576c0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f59f400d800 tx=0x7f59f400dcc0 comp rx=0 tx=0).stop 2024-05-22T21:20:40.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:40.178+0000 7f5a0b800640 1 -- 172.21.15.161:0/3074539273 shutdown_connections 2024-05-22T21:20:40.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:40.178+0000 7f5a0b800640 1 --2- 172.21.15.161:0/3074539273 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59e4066690 0x7f59e4068b50 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:40.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:40.178+0000 7f5a0b800640 1 --2- 172.21.15.161:0/3074539273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a041559e0 0x7f5a04158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:40.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:40.178+0000 7f5a0b800640 1 --2- 172.21.15.161:0/3074539273 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0415ec90 0x7f5a04155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:40.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:40.178+0000 7f5a0b800640 1 --2- 172.21.15.161:0/3074539273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a04143ec0 0x7f5a041576c0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:40.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:40.178+0000 7f5a0b800640 1 -- 172.21.15.161:0/3074539273 >> 172.21.15.161:0/3074539273 conn(0x7f5a040b0bc0 msgr2=0x7f5a04139060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:40.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:40.178+0000 7f5a0b800640 1 -- 172.21.15.161:0/3074539273 shutdown_connections 2024-05-22T21:20:40.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:40.178+0000 7f5a0b800640 1 -- 172.21.15.161:0/3074539273 wait complete. 2024-05-22T21:20:40.200 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:20:40.200 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:20:40.201 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-05-22T21:20:40.201 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-22T21:20:40.201 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-22T21:20:40.201 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:20:40.201 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:20:40.201 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:20:40.201 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:20:40.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:40.202 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-05-22T21:20:40.211 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:20:40.211 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:20:40.211 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:20:40.211 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:20:40.211 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-05-22T21:20:40.211 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:20:40.211 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:20:40.211 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-05-22T21:20:40.212 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2024-05-22T21:20:40.212 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:20:40.212 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:20:40.212 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2024-05-22T21:20:40.212 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:20:40.212 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:20:40.212 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:20:40.212 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:20:40.212 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:20:40.212 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42052sec preferred_lft 42052sec 2024-05-22T21:20:40.212 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-05-22T21:20:40.212 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:20:40.212 INFO:teuthology.orchestra.run.smithi161.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:20:40.213 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 02:42:b9:0f:57:df brd ff:ff:ff:ff:ff:ff 2024-05-22T21:20:40.213 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:20:40.213 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:20:40.213 INFO:teuthology.orchestra.run.smithi161.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:20:40.213 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:20:40.213 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:20:40.213 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:20:40.213 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:20:40.213 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:20:40.213 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:20:40.213 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-22T21:20:40.213 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:20:40.213 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:20:40.214 INFO:teuthology.orchestra.run.smithi161.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:20:40.214 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-22T21:20:40.214 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:20:40.214 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:20:40.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:40.215 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:20:40.276 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-05-22T21:20:40.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:40.278 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-05-22T21:20:40.334 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-05-22T21:20:40.334 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:20:40.334 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:20:40.334 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:20:40.334 INFO:teuthology.orchestra.run.smithi161.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:20:40.334 INFO:teuthology.orchestra.run.smithi161.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:20:40.334 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:20:40.335 INFO:teuthology.orchestra.run.smithi161.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:20:40.335 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:20:40.335 INFO:teuthology.orchestra.run.smithi161.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:20:40.335 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:20:40.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:40.335 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:20:40.335 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:20:40.335 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:20:40.335 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:20:40.335 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:20:40.336 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:20:40.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:40.427 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-05-22T21:20:40.435 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-22T21:20:40.436 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-22T21:20:40.436 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:20:40.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:40.437 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2024-05-22T21:20:40.499 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:20:40.499 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-22T21:20:40.546 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:20:40.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:20:40.547 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:20:40.596 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:20:40.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:40.596 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:20:40.687 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:20:40.687 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:20:40.687 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:20:40.702 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:20:40.702 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:20:40.702 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:20:40.702 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:20:40.703 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:20:40.711 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:20:40.711 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:20:40.711 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:20:40.712 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:20:40.712 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:20:40.712 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:20:40.712 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:20:40.712 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:20:40.716 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:20:40.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:40.716 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:20:40.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:40.783 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:20:40.843 INFO:teuthology.orchestra.run.smithi161.stdout:client7723 2024-05-22T21:20:40.844 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:20:40.844 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:20:40.895 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:20:41.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.074+0000 7f1714800640 1 Processor -- start 2024-05-22T21:20:41.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.074+0000 7f1714800640 1 -- start start 2024-05-22T21:20:41.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.074+0000 7f1714800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f170c15d4a0 0x7f170c099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:41.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.074+0000 7f1714800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c0b0490 0x7f170c099f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:41.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.074+0000 7f1714800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170c06c440 con 0x7f170c06bfb0 2024-05-22T21:20:41.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.074+0000 7f1714800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170c15d880 con 0x7f170c15d4a0 2024-05-22T21:20:41.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.074+0000 7f1714800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c06b440 con 0x7f170c0b0490 2024-05-22T21:20:41.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.074+0000 7f1711e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f170c15d4a0 0x7f170c099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:41.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.074+0000 7f1711e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f170c15d4a0 0x7f170c099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32982/0 (socket says 172.21.15.161:32982) 2024-05-22T21:20:41.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.074+0000 7f1711e00640 1 -- 172.21.15.161:0/4213462554 learned_addr learned my addr 172.21.15.161:0/4213462554 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:41.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.074+0000 7f1711e00640 1 -- 172.21.15.161:0/4213462554 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c0b0490 msgr2=0x7f170c099f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:41.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.074+0000 7f1711e00640 1 --2- 172.21.15.161:0/4213462554 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c0b0490 0x7f170c099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:41.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.074+0000 7f1711e00640 1 -- 172.21.15.161:0/4213462554 >> v1:172.21.15.179:6789/0 conn(0x7f170c06bfb0 legacy=0x7f170c160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:41.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.074+0000 7f1711e00640 1 -- 172.21.15.161:0/4213462554 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c0cf190 con 0x7f170c15d4a0 2024-05-22T21:20:41.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.078+0000 7f1711e00640 1 --2- 172.21.15.161:0/4213462554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c15d4a0 0x7f170c099a40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f1700009870 tx=0x7f170001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40a13b153009dbd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:41.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.078+0000 7f1710a00640 1 -- 172.21.15.161:0/4213462554 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17000aa610 con 0x7f170c15d4a0 2024-05-22T21:20:41.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.078+0000 7f1710a00640 1 -- 172.21.15.161:0/4213462554 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17000a6040 con 0x7f170c15d4a0 2024-05-22T21:20:41.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.078+0000 7f1710a00640 1 -- 172.21.15.161:0/4213462554 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17000aa990 con 0x7f170c15d4a0 2024-05-22T21:20:41.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.078+0000 7f1714800640 1 -- 172.21.15.161:0/4213462554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c15d4a0 msgr2=0x7f170c099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:41.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.078+0000 7f1714800640 1 --2- 172.21.15.161:0/4213462554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c15d4a0 0x7f170c099a40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f1700009870 tx=0x7f170001c4f0 comp rx=0 tx=0).stop 2024-05-22T21:20:41.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.078+0000 7f1714800640 1 -- 172.21.15.161:0/4213462554 shutdown_connections 2024-05-22T21:20:41.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.078+0000 7f1714800640 1 --2- 172.21.15.161:0/4213462554 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c0b0490 0x7f170c099f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:41.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.078+0000 7f1714800640 1 --2- 172.21.15.161:0/4213462554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c15d4a0 0x7f170c099a40 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:41.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.078+0000 7f1714800640 1 -- 172.21.15.161:0/4213462554 >> 172.21.15.161:0/4213462554 conn(0x7f170c0b2a40 msgr2=0x7f170c0b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:41.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.078+0000 7f1714800640 1 -- 172.21.15.161:0/4213462554 shutdown_connections 2024-05-22T21:20:41.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.078+0000 7f1714800640 1 -- 172.21.15.161:0/4213462554 wait complete. 2024-05-22T21:20:41.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1714800640 1 Processor -- start 2024-05-22T21:20:41.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1714800640 1 -- start start 2024-05-22T21:20:41.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1714800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c0b0490 0x7f170c151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:41.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1714800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f170c15d4a0 0x7f170c1581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:41.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1714800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c159650 0x7f170c15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:41.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1714800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c15aa10 con 0x7f170c0b0490 2024-05-22T21:20:41.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1714800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c159e00 con 0x7f170c15d4a0 2024-05-22T21:20:41.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1714800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c15a0e0 con 0x7f170c159650 2024-05-22T21:20:41.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1711e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c0b0490 0x7f170c151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:41.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1711e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c0b0490 0x7f170c151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:32986/0 (socket says 172.21.15.161:32986) 2024-05-22T21:20:41.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1711e00640 1 -- 172.21.15.161:0/2654077547 learned_addr learned my addr 172.21.15.161:0/2654077547 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:41.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1711400640 1 --2- 172.21.15.161:0/2654077547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f170c15d4a0 0x7f170c1581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:41.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1712800640 1 --2- 172.21.15.161:0/2654077547 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c159650 0x7f170c15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:41.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1711e00640 1 -- 172.21.15.161:0/2654077547 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c159650 msgr2=0x7f170c15beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:41.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1711e00640 1 --2- 172.21.15.161:0/2654077547 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c159650 0x7f170c15beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:41.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1711e00640 1 -- 172.21.15.161:0/2654077547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f170c15d4a0 msgr2=0x7f170c1581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:41.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1711e00640 1 --2- 172.21.15.161:0/2654077547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f170c15d4a0 0x7f170c1581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:41.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1711e00640 1 -- 172.21.15.161:0/2654077547 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17000094d0 con 0x7f170c0b0490 2024-05-22T21:20:41.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1711400640 1 --2- 172.21.15.161:0/2654077547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f170c15d4a0 0x7f170c1581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:20:41.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1711e00640 1 --2- 172.21.15.161:0/2654077547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c0b0490 0x7f170c151140 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f17000ac040 tx=0x7f170001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:41.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1706000640 1 -- 172.21.15.161:0/2654077547 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17000bd5f0 con 0x7f170c0b0490 2024-05-22T21:20:41.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1706000640 1 -- 172.21.15.161:0/2654077547 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17000bd770 con 0x7f170c0b0490 2024-05-22T21:20:41.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1706000640 1 -- 172.21.15.161:0/2654077547 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17000bdb20 con 0x7f170c0b0490 2024-05-22T21:20:41.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1714800640 1 -- 172.21.15.161:0/2654077547 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170c0cb7b0 con 0x7f170c0b0490 2024-05-22T21:20:41.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.082+0000 7f1714800640 1 -- 172.21.15.161:0/2654077547 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c15a6a0 con 0x7f170c0b0490 2024-05-22T21:20:41.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.086+0000 7f1714800640 1 -- 172.21.15.161:0/2654077547 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170c1586e0 con 0x7f170c0b0490 2024-05-22T21:20:41.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.086+0000 7f1706000640 1 -- 172.21.15.161:0/2654077547 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f17000b33f0 con 0x7f170c0b0490 2024-05-22T21:20:41.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.086+0000 7f1706000640 1 --2- 172.21.15.161:0/2654077547 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16f0066690 0x7f16f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:41.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.086+0000 7f1706000640 1 -- 172.21.15.161:0/2654077547 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f170011eb80 con 0x7f170c0b0490 2024-05-22T21:20:41.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.086+0000 7f1711400640 1 --2- 172.21.15.161:0/2654077547 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16f0066690 0x7f16f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:41.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.086+0000 7f1711400640 1 --2- 172.21.15.161:0/2654077547 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16f0066690 0x7f16f0068b50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f170c09aa10 tx=0x7f16fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:41.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.090+0000 7f1706000640 1 -- 172.21.15.161:0/2654077547 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f17000ebfa0 con 0x7f170c0b0490 2024-05-22T21:20:41.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.274+0000 7f1714800640 1 -- 172.21.15.161:0/2654077547 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f170c06c440 con 0x7f170c0b0490 2024-05-22T21:20:41.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.278+0000 7f1706000640 1 -- 172.21.15.161:0/2654077547 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) v1 ==== 53+0+73 (secure 0 0 0) 0x7f17000c3060 con 0x7f170c0b0490 2024-05-22T21:20:41.279 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:20:41.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.278+0000 7f1714800640 1 -- 172.21.15.161:0/2654077547 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16f0066690 msgr2=0x7f16f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:41.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.278+0000 7f1714800640 1 --2- 172.21.15.161:0/2654077547 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16f0066690 0x7f16f0068b50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f170c09aa10 tx=0x7f16fc008040 comp rx=0 tx=0).stop 2024-05-22T21:20:41.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.278+0000 7f1714800640 1 -- 172.21.15.161:0/2654077547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c0b0490 msgr2=0x7f170c151140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:41.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.278+0000 7f1714800640 1 --2- 172.21.15.161:0/2654077547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c0b0490 0x7f170c151140 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f17000ac040 tx=0x7f170001ad30 comp rx=0 tx=0).stop 2024-05-22T21:20:41.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.278+0000 7f1714800640 1 -- 172.21.15.161:0/2654077547 shutdown_connections 2024-05-22T21:20:41.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.278+0000 7f1714800640 1 --2- 172.21.15.161:0/2654077547 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16f0066690 0x7f16f0068b50 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:41.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.278+0000 7f1714800640 1 --2- 172.21.15.161:0/2654077547 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c159650 0x7f170c15beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:41.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.278+0000 7f1714800640 1 --2- 172.21.15.161:0/2654077547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f170c15d4a0 0x7f170c1581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:41.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.278+0000 7f1714800640 1 --2- 172.21.15.161:0/2654077547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c0b0490 0x7f170c151140 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:41.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.278+0000 7f1714800640 1 -- 172.21.15.161:0/2654077547 >> 172.21.15.161:0/2654077547 conn(0x7f170c0b2a40 msgr2=0x7f170c0cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:41.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.278+0000 7f1714800640 1 -- 172.21.15.161:0/2654077547 shutdown_connections 2024-05-22T21:20:41.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:41.278+0000 7f1714800640 1 -- 172.21.15.161:0/2654077547 wait complete. 2024-05-22T21:20:41.300 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:20:41.301 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:20:41.301 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-22T21:20:41.301 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-22T21:20:41.301 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-22T21:20:41.301 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:20:41.301 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:20:41.301 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:20:41.301 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:20:41.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:41.301 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-22T21:20:41.311 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:20:41.312 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:20:41.312 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:20:41.312 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:20:41.312 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-22T21:20:41.312 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:20:41.312 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:20:41.312 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:20:41.312 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f0 2024-05-22T21:20:41.312 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:20:41.312 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:20:41.312 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f1 2024-05-22T21:20:41.312 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:20:41.312 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-22T21:20:41.313 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:20:41.313 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-22T21:20:41.313 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:20:41.313 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 42043sec preferred_lft 42043sec 2024-05-22T21:20:41.313 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-22T21:20:41.313 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:20:41.313 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:20:41.313 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:bd:4e:25:3d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:20:41.313 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:20:41.313 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:20:41.313 INFO:teuthology.orchestra.run.smithi179.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:20:41.314 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:20:41.314 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:20:41.314 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:20:41.314 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:20:41.314 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:20:41.314 INFO:teuthology.orchestra.run.smithi179.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:20:41.314 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-22T21:20:41.314 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:20:41.314 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:20:41.314 INFO:teuthology.orchestra.run.smithi179.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:20:41.314 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-22T21:20:41.314 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:20:41.315 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:20:41.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:41.315 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:20:41.384 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-22T21:20:41.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:41.386 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-22T21:20:41.440 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-22T21:20:41.440 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:20:41.440 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:20:41.440 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:20:41.441 INFO:teuthology.orchestra.run.smithi179.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:20:41.441 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:20:41.441 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:20:41.441 INFO:teuthology.orchestra.run.smithi179.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:20:41.441 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:20:41.441 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:20:41.441 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:20:41.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:41.441 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:20:41.441 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:20:41.442 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:20:41.442 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:20:41.442 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:20:41.442 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:20:41.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:41.537 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-22T21:20:41.546 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-22T21:20:41.546 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-22T21:20:41.547 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:20:41.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:41.547 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-22T21:20:41.611 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:20:41.612 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-22T21:20:41.663 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:20:41.664 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:20:41.664 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:20:41.718 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:20:41.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:41.719 DEBUG:teuthology.orchestra.run.smithi179:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:20:41.807 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:20:41.808 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:20:41.808 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:20:41.823 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:20:41.823 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:20:41.823 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:20:41.823 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:20:41.823 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:20:41.831 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:20:41.831 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:20:41.831 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:20:41.831 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:20:41.831 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:20:41.831 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:20:41.831 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:20:41.831 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:20:41.835 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:20:41.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:41.836 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:20:41.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:20:41.855 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:20:41.909 INFO:teuthology.orchestra.run.smithi179.stdout:client6797 2024-05-22T21:20:41.910 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:20:41.910 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:20:41.961 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:20:42.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.102+0000 7fb531000640 1 Processor -- start 2024-05-22T21:20:42.105 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.102+0000 7fb531000640 1 -- start start 2024-05-22T21:20:42.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.102+0000 7fb531000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb52c15e130 0x7fb52c15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:42.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.102+0000 7fb531000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52c071670 con 0x7fb52c15d410 2024-05-22T21:20:42.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.102+0000 7fb531000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52c15d870 con 0x7fb52c15dd50 2024-05-22T21:20:42.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.102+0000 7fb531000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52c070a60 con 0x7fb52c15e130 2024-05-22T21:20:42.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.102+0000 7fb529400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb52c15e130 0x7fb52c15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:42.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.102+0000 7fb52a800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb52c15d410 0x7fb52c15b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60552/0 (socket says 172.21.15.161:60552) 2024-05-22T21:20:42.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.102+0000 7fb52a800640 1 -- 172.21.15.161:0/614102407 learned_addr learned my addr 172.21.15.161:0/614102407 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:20:42.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.102+0000 7fb529400640 1 -- 172.21.15.161:0/614102407 >> v1:172.21.15.179:6789/0 conn(0x7fb52c15dd50 legacy=0x7fb52c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:42.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.102+0000 7fb529400640 1 -- 172.21.15.161:0/614102407 >> v1:172.21.15.161:6789/0 conn(0x7fb52c15d410 legacy=0x7fb52c15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:42.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.102+0000 7fb529400640 1 -- 172.21.15.161:0/614102407 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c070d40 con 0x7fb52c15e130 2024-05-22T21:20:42.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.102+0000 7fb528a00640 1 -- 172.21.15.161:0/614102407 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164661936 0 0) 0x7fb52c071670 con 0x7fb52c15d410 2024-05-22T21:20:42.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.106+0000 7fb529400640 1 --2- 172.21.15.161:0/614102407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb52c15e130 0x7fb52c15ced0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fb5180097c0 tx=0x7fb51801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3deb78ef653089f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:42.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.106+0000 7fb528a00640 1 -- 172.21.15.161:0/614102407 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5180ab670 con 0x7fb52c15e130 2024-05-22T21:20:42.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.106+0000 7fb528a00640 1 -- 172.21.15.161:0/614102407 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5180ab7d0 con 0x7fb52c15e130 2024-05-22T21:20:42.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.106+0000 7fb528a00640 1 -- 172.21.15.161:0/614102407 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5180abb60 con 0x7fb52c15e130 2024-05-22T21:20:42.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.106+0000 7fb531000640 1 -- 172.21.15.161:0/614102407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb52c15e130 msgr2=0x7fb52c15ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:42.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.106+0000 7fb531000640 1 --2- 172.21.15.161:0/614102407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb52c15e130 0x7fb52c15ced0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fb5180097c0 tx=0x7fb51801c470 comp rx=0 tx=0).stop 2024-05-22T21:20:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.106+0000 7fb531000640 1 -- 172.21.15.161:0/614102407 shutdown_connections 2024-05-22T21:20:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.106+0000 7fb531000640 1 --2- 172.21.15.161:0/614102407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb52c15e130 0x7fb52c15ced0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.106+0000 7fb531000640 1 -- 172.21.15.161:0/614102407 >> 172.21.15.161:0/614102407 conn(0x7fb52c0b0bc0 msgr2=0x7fb52c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.106+0000 7fb531000640 1 -- 172.21.15.161:0/614102407 shutdown_connections 2024-05-22T21:20:42.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.106+0000 7fb531000640 1 -- 172.21.15.161:0/614102407 wait complete. 2024-05-22T21:20:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb531000640 1 Processor -- start 2024-05-22T21:20:42.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb531000640 1 -- start start 2024-05-22T21:20:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb531000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb52c15d410 0x7fb52c155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb531000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c15dd50 0x7fb52c151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb531000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb52c15e130 0x7fb52c0b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb531000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c1560d0 con 0x7fb52c15d410 2024-05-22T21:20:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb531000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c155df0 con 0x7fb52c15dd50 2024-05-22T21:20:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb531000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c157ef0 con 0x7fb52c15e130 2024-05-22T21:20:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb529e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb52c15d410 0x7fb52c155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb529e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb52c15d410 0x7fb52c155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33010/0 (socket says 172.21.15.161:33010) 2024-05-22T21:20:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb529e00640 1 -- 172.21.15.161:0/1718027930 learned_addr learned my addr 172.21.15.161:0/1718027930 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:42.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb529400640 1 --2- 172.21.15.161:0/1718027930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c15dd50 0x7fb52c151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:42.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb52a800640 1 --2- 172.21.15.161:0/1718027930 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb52c15e130 0x7fb52c0b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:42.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb529e00640 1 -- 172.21.15.161:0/1718027930 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb52c15e130 msgr2=0x7fb52c0b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:42.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb529e00640 1 --2- 172.21.15.161:0/1718027930 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb52c15e130 0x7fb52c0b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:42.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb529e00640 1 -- 172.21.15.161:0/1718027930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c15dd50 msgr2=0x7fb52c151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:42.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb529e00640 1 --2- 172.21.15.161:0/1718027930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c15dd50 0x7fb52c151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:42.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb529e00640 1 -- 172.21.15.161:0/1718027930 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb518009420 con 0x7fb52c15d410 2024-05-22T21:20:42.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb529e00640 1 --2- 172.21.15.161:0/1718027930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb52c15d410 0x7fb52c155810 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00c3c0 tx=0x7fb51c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:42.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb522000640 1 -- 172.21.15.161:0/1718027930 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0052d0 con 0x7fb52c15d410 2024-05-22T21:20:42.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb522000640 1 -- 172.21.15.161:0/1718027930 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c005430 con 0x7fb52c15d410 2024-05-22T21:20:42.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb531000640 1 -- 172.21.15.161:0/1718027930 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c1581d0 con 0x7fb52c15d410 2024-05-22T21:20:42.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb522000640 1 -- 172.21.15.161:0/1718027930 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c003840 con 0x7fb52c15d410 2024-05-22T21:20:42.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.110+0000 7fb531000640 1 -- 172.21.15.161:0/1718027930 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c14cc60 con 0x7fb52c15d410 2024-05-22T21:20:42.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.114+0000 7fb531000640 1 -- 172.21.15.161:0/1718027930 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c157d50 con 0x7fb52c15d410 2024-05-22T21:20:42.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.114+0000 7fb522000640 1 -- 172.21.15.161:0/1718027930 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb51c0039a0 con 0x7fb52c15d410 2024-05-22T21:20:42.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.114+0000 7fb522000640 1 --2- 172.21.15.161:0/1718027930 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb50c066690 0x7fb50c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:42.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.114+0000 7fb522000640 1 -- 172.21.15.161:0/1718027930 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb51c083600 con 0x7fb52c15d410 2024-05-22T21:20:42.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.114+0000 7fb529400640 1 --2- 172.21.15.161:0/1718027930 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb50c066690 0x7fb50c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:42.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.114+0000 7fb529400640 1 --2- 172.21.15.161:0/1718027930 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb50c066690 0x7fb50c068b50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb5180af000 tx=0x7fb5180b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:42.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.118+0000 7fb522000640 1 -- 172.21.15.161:0/1718027930 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb52c157d50 con 0x7fb52c15d410 2024-05-22T21:20:42.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.306+0000 7fb531000640 1 -- 172.21.15.161:0/1718027930 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb52c157d50 con 0x7fb52c15d410 2024-05-22T21:20:42.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.306+0000 7fb522000640 1 -- 172.21.15.161:0/1718027930 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4736 (secure 0 0 0) 0x7fb52c157d50 con 0x7fb52c15d410 2024-05-22T21:20:42.309 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:20:42.309 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":70,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7992,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/207877183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":207877183},{"type":"v1","addr":"172.21.15.161:6836","nonce":207877183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2152702106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2152702106},{"type":"v1","addr":"172.21.15.161:6837","nonce":2152702106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/749927648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":749927648},{"type":"v1","addr":"172.21.15.179:6834","nonce":749927648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:20:33.964441+0000","modified":"2024-05-22T21:20:36.988238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8004},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8004":{"gid":8004,"name":"b","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/93167194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":93167194},{"type":"v1","addr":"172.21.15.179:6835","nonce":93167194}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-05-22T21:20:42.309 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 70 2024-05-22T21:20:42.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.306+0000 7fb531000640 1 -- 172.21.15.161:0/1718027930 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb50c066690 msgr2=0x7fb50c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:42.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.306+0000 7fb531000640 1 --2- 172.21.15.161:0/1718027930 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb50c066690 0x7fb50c068b50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb5180af000 tx=0x7fb5180b6040 comp rx=0 tx=0).stop 2024-05-22T21:20:42.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.306+0000 7fb531000640 1 -- 172.21.15.161:0/1718027930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb52c15d410 msgr2=0x7fb52c155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:42.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.306+0000 7fb531000640 1 --2- 172.21.15.161:0/1718027930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb52c15d410 0x7fb52c155810 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00c3c0 tx=0x7fb51c00c880 comp rx=0 tx=0).stop 2024-05-22T21:20:42.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.306+0000 7fb531000640 1 -- 172.21.15.161:0/1718027930 shutdown_connections 2024-05-22T21:20:42.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.306+0000 7fb531000640 1 --2- 172.21.15.161:0/1718027930 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb50c066690 0x7fb50c068b50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:42.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.306+0000 7fb531000640 1 --2- 172.21.15.161:0/1718027930 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb52c15e130 0x7fb52c0b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:42.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.306+0000 7fb531000640 1 --2- 172.21.15.161:0/1718027930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c15dd50 0x7fb52c151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:42.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.306+0000 7fb531000640 1 --2- 172.21.15.161:0/1718027930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb52c15d410 0x7fb52c155810 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:42.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.306+0000 7fb531000640 1 -- 172.21.15.161:0/1718027930 >> 172.21.15.161:0/1718027930 conn(0x7fb52c0b0bc0 msgr2=0x7fb52c156ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:42.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.306+0000 7fb531000640 1 -- 172.21.15.161:0/1718027930 shutdown_connections 2024-05-22T21:20:42.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.306+0000 7fb531000640 1 -- 172.21.15.161:0/1718027930 wait complete. 2024-05-22T21:20:42.330 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-05-22T21:20:42.487 INFO:teuthology.orchestra.run.smithi179.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":8004,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.5048682479999993,"mdsmap_epoch":70,"osdmap_epoch":158,"osdmap_epoch_barrier":158,"uptime":12.791295649} 2024-05-22T21:20:42.488 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 8004, "mdsmap_epoch": 70, "osdmap_epoch": 158, "osdmap_epoch_barrier": 158, "rank_uptime": 8.504868248, "state": "up:active", "uptime": 12.791295649, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:20:42.488 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-05-22T21:20:42.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.626+0000 7f883fc00640 1 Processor -- start 2024-05-22T21:20:42.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.626+0000 7f883fc00640 1 -- start start 2024-05-22T21:20:42.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.626+0000 7f883fc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f883813cfe0 0x7f883815dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:42.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.626+0000 7f883fc00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f883813d560 0x7f883815e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:42.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.626+0000 7f883fc00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88380ce830 con 0x7f88380cdb00 2024-05-22T21:20:42.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.626+0000 7f883fc00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88380ce650 con 0x7f883813d560 2024-05-22T21:20:42.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.626+0000 7f883fc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8838140560 con 0x7f883813cfe0 2024-05-22T21:20:42.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.630+0000 7f882fe00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f883813d560 0x7f883815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:42.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.630+0000 7f882fe00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f883813d560 0x7f883815e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:54164/0 (socket says 172.21.15.161:54164) 2024-05-22T21:20:42.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.630+0000 7f882fe00640 1 -- 172.21.15.161:0/2346186087 learned_addr learned my addr 172.21.15.161:0/2346186087 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:42.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.630+0000 7f883dc00640 1 --1- 172.21.15.161:0/2346186087 >> v1:172.21.15.179:6789/0 conn(0x7f88380cdb00 0x7f88380cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:34084/0 (socket says 172.21.15.161:34084) 2024-05-22T21:20:42.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.630+0000 7f883d200640 1 --2- 172.21.15.161:0/2346186087 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f883813cfe0 0x7f883815dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:42.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.630+0000 7f882fe00640 1 -- 172.21.15.161:0/2346186087 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f883813cfe0 msgr2=0x7f883815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:20:42.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.630+0000 7f882fe00640 1 --2- 172.21.15.161:0/2346186087 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f883813cfe0 0x7f883815dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:42.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.630+0000 7f882fe00640 1 -- 172.21.15.161:0/2346186087 >> v1:172.21.15.179:6789/0 conn(0x7f88380cdb00 legacy=0x7f88380cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:42.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.630+0000 7f882fe00640 1 -- 172.21.15.161:0/2346186087 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88380ceb10 con 0x7f883813d560 2024-05-22T21:20:42.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.630+0000 7f8837e00640 1 -- 172.21.15.161:0/2346186087 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663855121 0 0) 0x7f88380ce830 con 0x7f88380cdb00 2024-05-22T21:20:42.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.630+0000 7f882fe00640 1 --2- 172.21.15.161:0/2346186087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f883813d560 0x7f883815e000 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8820009870 tx=0x7f882001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cddfb963fe8d2692 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:42.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.630+0000 7f8837e00640 1 -- 172.21.15.161:0/2346186087 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88200a6040 con 0x7f883813d560 2024-05-22T21:20:42.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.630+0000 7f8837e00640 1 -- 172.21.15.161:0/2346186087 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88200aa600 con 0x7f883813d560 2024-05-22T21:20:42.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.630+0000 7f8837e00640 1 -- 172.21.15.161:0/2346186087 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88200aa990 con 0x7f883813d560 2024-05-22T21:20:42.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883fc00640 1 -- 172.21.15.161:0/2346186087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f883813d560 msgr2=0x7f883815e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:42.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883fc00640 1 --2- 172.21.15.161:0/2346186087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f883813d560 0x7f883815e000 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8820009870 tx=0x7f882001c4a0 comp rx=0 tx=0).stop 2024-05-22T21:20:42.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883fc00640 1 -- 172.21.15.161:0/2346186087 shutdown_connections 2024-05-22T21:20:42.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883fc00640 1 --2- 172.21.15.161:0/2346186087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f883813d560 0x7f883815e000 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:42.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883fc00640 1 --2- 172.21.15.161:0/2346186087 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f883813cfe0 0x7f883815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:42.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883fc00640 1 -- 172.21.15.161:0/2346186087 >> 172.21.15.161:0/2346186087 conn(0x7f883806a0a0 msgr2=0x7f883806a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:42.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883fc00640 1 -- 172.21.15.161:0/2346186087 shutdown_connections 2024-05-22T21:20:42.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883fc00640 1 -- 172.21.15.161:0/2346186087 wait complete. 2024-05-22T21:20:42.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883fc00640 1 Processor -- start 2024-05-22T21:20:42.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883fc00640 1 -- start start 2024-05-22T21:20:42.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883fc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f88380cdb00 0x7f8838139e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:42.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883fc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f883813cfe0 0x7f883813f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:42.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883fc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883813d560 0x7f883813f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:42.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883fc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8838156eb0 con 0x7f883813d560 2024-05-22T21:20:42.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883fc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8838156a20 con 0x7f883813cfe0 2024-05-22T21:20:42.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883fc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8838155e10 con 0x7f88380cdb00 2024-05-22T21:20:42.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883dc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883813d560 0x7f883813f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:42.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883dc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883813d560 0x7f883813f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33024/0 (socket says 172.21.15.161:33024) 2024-05-22T21:20:42.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883dc00640 1 -- 172.21.15.161:0/3065652308 learned_addr learned my addr 172.21.15.161:0/3065652308 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:20:42.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883d200640 1 --2- 172.21.15.161:0/3065652308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f88380cdb00 0x7f8838139e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:42.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f882fe00640 1 --2- 172.21.15.161:0/3065652308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f883813cfe0 0x7f883813f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:42.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883dc00640 1 -- 172.21.15.161:0/3065652308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f88380cdb00 msgr2=0x7f8838139e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:42.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883dc00640 1 --2- 172.21.15.161:0/3065652308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f88380cdb00 0x7f8838139e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:42.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883dc00640 1 -- 172.21.15.161:0/3065652308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f883813cfe0 msgr2=0x7f883813f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:42.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883dc00640 1 --2- 172.21.15.161:0/3065652308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f883813cfe0 0x7f883813f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:42.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883dc00640 1 -- 172.21.15.161:0/3065652308 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88200094d0 con 0x7f883813d560 2024-05-22T21:20:42.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.634+0000 7f883d200640 1 --2- 172.21.15.161:0/3065652308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f88380cdb00 0x7f8838139e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:20:42.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.638+0000 7f883dc00640 1 --2- 172.21.15.161:0/3065652308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883813d560 0x7f883813f610 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f8830001570 tx=0x7f883000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:42.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.638+0000 7f8835600640 1 -- 172.21.15.161:0/3065652308 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883000a640 con 0x7f883813d560 2024-05-22T21:20:42.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.638+0000 7f8835600640 1 -- 172.21.15.161:0/3065652308 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883000a7c0 con 0x7f883813d560 2024-05-22T21:20:42.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.638+0000 7f883fc00640 1 -- 172.21.15.161:0/3065652308 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88381560f0 con 0x7f883813d560 2024-05-22T21:20:42.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.638+0000 7f8835600640 1 -- 172.21.15.161:0/3065652308 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883000ab70 con 0x7f883813d560 2024-05-22T21:20:42.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.638+0000 7f883fc00640 1 -- 172.21.15.161:0/3065652308 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88381566b0 con 0x7f883813d560 2024-05-22T21:20:42.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.638+0000 7f883fc00640 1 -- 172.21.15.161:0/3065652308 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8838156b90 con 0x7f883813d560 2024-05-22T21:20:42.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.638+0000 7f8835600640 1 -- 172.21.15.161:0/3065652308 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f883000acd0 con 0x7f883813d560 2024-05-22T21:20:42.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.638+0000 7f8835600640 1 --2- 172.21.15.161:0/3065652308 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8814066690 0x7f8814068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:20:42.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.638+0000 7f8835600640 1 -- 172.21.15.161:0/3065652308 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8830083940 con 0x7f883813d560 2024-05-22T21:20:42.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.638+0000 7f883d200640 1 --2- 172.21.15.161:0/3065652308 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8814066690 0x7f8814068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:20:42.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.642+0000 7f883d200640 1 --2- 172.21.15.161:0/3065652308 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8814066690 0x7f8814068b50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8828009f50 tx=0x7f8828009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:20:42.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.642+0000 7f8835600640 1 -- 172.21.15.161:0/3065652308 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8830050d60 con 0x7f883813d560 2024-05-22T21:20:42.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:42.802+0000 7f883fc00640 1 -- 172.21.15.161:0/3065652308 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f8838067220 con 0x7f883813d560 2024-05-22T21:20:43.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:43.014+0000 7f8835600640 1 -- 172.21.15.161:0/3065652308 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v159) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8830018020 con 0x7f883813d560 2024-05-22T21:20:43.016 INFO:teuthology.orchestra.run.smithi161.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-05-22T21:20:43.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:43.014+0000 7f883fc00640 1 -- 172.21.15.161:0/3065652308 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8814066690 msgr2=0x7f8814068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:43.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:43.014+0000 7f883fc00640 1 --2- 172.21.15.161:0/3065652308 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8814066690 0x7f8814068b50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8828009f50 tx=0x7f8828009190 comp rx=0 tx=0).stop 2024-05-22T21:20:43.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:43.014+0000 7f883fc00640 1 -- 172.21.15.161:0/3065652308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883813d560 msgr2=0x7f883813f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:20:43.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:43.014+0000 7f883fc00640 1 --2- 172.21.15.161:0/3065652308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883813d560 0x7f883813f610 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f8830001570 tx=0x7f883000cca0 comp rx=0 tx=0).stop 2024-05-22T21:20:43.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:43.014+0000 7f883fc00640 1 -- 172.21.15.161:0/3065652308 shutdown_connections 2024-05-22T21:20:43.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:43.014+0000 7f883fc00640 1 --2- 172.21.15.161:0/3065652308 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8814066690 0x7f8814068b50 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:43.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:43.014+0000 7f883fc00640 1 --2- 172.21.15.161:0/3065652308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883813d560 0x7f883813f610 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:43.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:43.014+0000 7f883fc00640 1 --2- 172.21.15.161:0/3065652308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f883813cfe0 0x7f883813f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:43.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:43.014+0000 7f883fc00640 1 --2- 172.21.15.161:0/3065652308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f88380cdb00 0x7f8838139e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:20:43.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:43.014+0000 7f883fc00640 1 -- 172.21.15.161:0/3065652308 >> 172.21.15.161:0/3065652308 conn(0x7f883806a0a0 msgr2=0x7f883806a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:20:43.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:43.014+0000 7f883fc00640 1 -- 172.21.15.161:0/3065652308 shutdown_connections 2024-05-22T21:20:43.020 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:20:43.014+0000 7f883fc00640 1 -- 172.21.15.161:0/3065652308 wait complete. 2024-05-22T21:20:43.040 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-05-22T21:20:43.191 INFO:teuthology.orchestra.run.smithi161.stdout:{"osd_mon_report_interval":"5"} 2024-05-22T21:20:43.192 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-05-22T21:20:43.192 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-05-22T21:20:43.338 INFO:teuthology.orchestra.run.smithi161.stdout:{"mon_tick_interval":"5"} 2024-05-22T21:20:43.338 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-05-22T21:20:43.338 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2024-05-22T21:20:43.339 DEBUG:teuthology.orchestra.run.smithi161:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-22T21:20:43.339 DEBUG:teuthology.orchestra.run.smithi161:> import time 2024-05-22T21:20:43.339 DEBUG:teuthology.orchestra.run.smithi161:> import datetime 2024-05-22T21:20:43.339 DEBUG:teuthology.orchestra.run.smithi161:> import subprocess 2024-05-22T21:20:43.339 DEBUG:teuthology.orchestra.run.smithi161:> import os 2024-05-22T21:20:43.339 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:20:43.339 DEBUG:teuthology.orchestra.run.smithi161:> # Write some buffered data through before going full, all should be well 2024-05-22T21:20:43.339 DEBUG:teuthology.orchestra.run.smithi161:> print("writing some data through which we expect to succeed") 2024-05-22T21:20:43.339 DEBUG:teuthology.orchestra.run.smithi161:> bytes = 0 2024-05-22T21:20:43.339 DEBUG:teuthology.orchestra.run.smithi161:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-05-22T21:20:43.339 DEBUG:teuthology.orchestra.run.smithi161:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-05-22T21:20:43.339 DEBUG:teuthology.orchestra.run.smithi161:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2024-05-22T21:20:43.339 DEBUG:teuthology.orchestra.run.smithi161:> os.fsync(f) 2024-05-22T21:20:43.339 DEBUG:teuthology.orchestra.run.smithi161:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-05-22T21:20:43.340 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:20:43.340 DEBUG:teuthology.orchestra.run.smithi161:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-05-22T21:20:43.340 DEBUG:teuthology.orchestra.run.smithi161:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-05-22T21:20:43.340 DEBUG:teuthology.orchestra.run.smithi161:> # buffered IO, the error should always only appear from fsync and not 2024-05-22T21:20:43.340 DEBUG:teuthology.orchestra.run.smithi161:> # from write 2024-05-22T21:20:43.340 DEBUG:teuthology.orchestra.run.smithi161:> full = False 2024-05-22T21:20:43.340 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:20:43.340 DEBUG:teuthology.orchestra.run.smithi161:> for n in range(0, int(32 * 0.9)): 2024-05-22T21:20:43.340 DEBUG:teuthology.orchestra.run.smithi161:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-05-22T21:20:43.340 DEBUG:teuthology.orchestra.run.smithi161:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2024-05-22T21:20:43.340 DEBUG:teuthology.orchestra.run.smithi161:> print("done writing {0} bytes".format(bytes)) 2024-05-22T21:20:43.340 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:20:43.340 DEBUG:teuthology.orchestra.run.smithi161:> # OK, now we should sneak in under the full condition 2024-05-22T21:20:43.340 DEBUG:teuthology.orchestra.run.smithi161:> # due to the time it takes the OSDs to report to the 2024-05-22T21:20:43.340 DEBUG:teuthology.orchestra.run.smithi161:> # mons, and get a successful fsync on our full-making data 2024-05-22T21:20:43.341 DEBUG:teuthology.orchestra.run.smithi161:> os.fsync(f) 2024-05-22T21:20:43.341 DEBUG:teuthology.orchestra.run.smithi161:> print("successfully fsync'"'"'ed prior to getting full state reported") 2024-05-22T21:20:43.341 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:20:43.341 DEBUG:teuthology.orchestra.run.smithi161:> # buffered write, add more dirty data to the buffer 2024-05-22T21:20:43.341 DEBUG:teuthology.orchestra.run.smithi161:> print("starting buffered write") 2024-05-22T21:20:43.341 DEBUG:teuthology.orchestra.run.smithi161:> try: 2024-05-22T21:20:43.341 DEBUG:teuthology.orchestra.run.smithi161:> for n in range(0, int(32 * 0.2)): 2024-05-22T21:20:43.341 DEBUG:teuthology.orchestra.run.smithi161:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-05-22T21:20:43.341 DEBUG:teuthology.orchestra.run.smithi161:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-05-22T21:20:43.341 DEBUG:teuthology.orchestra.run.smithi161:> time.sleep(15.0) 2024-05-22T21:20:43.341 DEBUG:teuthology.orchestra.run.smithi161:> except OSError: 2024-05-22T21:20:43.341 DEBUG:teuthology.orchestra.run.smithi161:> pass; 2024-05-22T21:20:43.341 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:20:43.341 DEBUG:teuthology.orchestra.run.smithi161:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2024-05-22T21:20:43.341 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:20:43.342 DEBUG:teuthology.orchestra.run.smithi161:> # Wait long enough for a background flush that should fail 2024-05-22T21:20:43.342 DEBUG:teuthology.orchestra.run.smithi161:> time.sleep(30) 2024-05-22T21:20:43.342 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:20:43.342 DEBUG:teuthology.orchestra.run.smithi161:> if False: 2024-05-22T21:20:43.342 DEBUG:teuthology.orchestra.run.smithi161:> # ...and check that the failed background flush is reflected in fclose 2024-05-22T21:20:43.342 DEBUG:teuthology.orchestra.run.smithi161:> try: 2024-05-22T21:20:43.342 DEBUG:teuthology.orchestra.run.smithi161:> os.close(f) 2024-05-22T21:20:43.342 DEBUG:teuthology.orchestra.run.smithi161:> except OSError: 2024-05-22T21:20:43.342 DEBUG:teuthology.orchestra.run.smithi161:> print("close() returned an error as expected") 2024-05-22T21:20:43.342 DEBUG:teuthology.orchestra.run.smithi161:> else: 2024-05-22T21:20:43.342 DEBUG:teuthology.orchestra.run.smithi161:> raise RuntimeError("close() failed to raise error") 2024-05-22T21:20:43.342 DEBUG:teuthology.orchestra.run.smithi161:> else: 2024-05-22T21:20:43.342 DEBUG:teuthology.orchestra.run.smithi161:> # The kernel cephfs client does not raise errors on fclose 2024-05-22T21:20:43.342 DEBUG:teuthology.orchestra.run.smithi161:> os.close(f) 2024-05-22T21:20:43.343 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:20:43.343 DEBUG:teuthology.orchestra.run.smithi161:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-05-22T21:20:43.343 DEBUG:teuthology.orchestra.run.smithi161:> ' 2024-05-22T21:20:54.697 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:20:54.701 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:21:24.753 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:21:24.758 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:21:54.818 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:21:54.823 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:22:24.883 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:22:24.887 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:22:44.976 INFO:teuthology.orchestra.run.smithi161.stdout:writing some data through which we expect to succeed 2024-05-22T21:22:44.977 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-05-22T21:22:44.977 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 1572864 bytes via buffered write, may repeat 2024-05-22T21:22:44.977 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 2621440 bytes via buffered write, may repeat 2024-05-22T21:22:44.977 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 3670016 bytes via buffered write, may repeat 2024-05-22T21:22:44.977 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 4718592 bytes via buffered write, may repeat 2024-05-22T21:22:44.977 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 5767168 bytes via buffered write, may repeat 2024-05-22T21:22:44.977 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 6815744 bytes via buffered write, may repeat 2024-05-22T21:22:44.977 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 7864320 bytes via buffered write, may repeat 2024-05-22T21:22:44.977 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 8912896 bytes via buffered write, may repeat 2024-05-22T21:22:44.977 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 9961472 bytes via buffered write, may repeat 2024-05-22T21:22:44.977 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 11010048 bytes via buffered write, may repeat 2024-05-22T21:22:44.978 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 12058624 bytes via buffered write, may repeat 2024-05-22T21:22:44.978 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 13107200 bytes via buffered write, may repeat 2024-05-22T21:22:44.978 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 14155776 bytes via buffered write, may repeat 2024-05-22T21:22:44.978 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 15204352 bytes via buffered write, may repeat 2024-05-22T21:22:44.978 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 16252928 bytes via buffered write, may repeat 2024-05-22T21:22:44.978 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 17301504 bytes via buffered write, may repeat 2024-05-22T21:22:44.978 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 18350080 bytes via buffered write, may repeat 2024-05-22T21:22:44.978 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 19398656 bytes via buffered write, may repeat 2024-05-22T21:22:44.978 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 20447232 bytes via buffered write, may repeat 2024-05-22T21:22:44.978 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 21495808 bytes via buffered write, may repeat 2024-05-22T21:22:44.978 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 22544384 bytes via buffered write, may repeat 2024-05-22T21:22:44.979 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 23592960 bytes via buffered write, may repeat 2024-05-22T21:22:44.979 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 24641536 bytes via buffered write, may repeat 2024-05-22T21:22:44.979 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 25690112 bytes via buffered write, may repeat 2024-05-22T21:22:44.979 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 26738688 bytes via buffered write, may repeat 2024-05-22T21:22:44.979 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 27787264 bytes via buffered write, may repeat 2024-05-22T21:22:44.979 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 28835840 bytes via buffered write, may repeat 2024-05-22T21:22:44.979 INFO:teuthology.orchestra.run.smithi161.stdout:wrote 29884416 bytes via buffered write, may repeat 2024-05-22T21:22:44.979 INFO:teuthology.orchestra.run.smithi161.stdout:done writing 29884416 bytes 2024-05-22T21:22:44.979 INFO:teuthology.orchestra.run.smithi161.stdout:successfully fsync'ed prior to getting full state reported 2024-05-22T21:22:44.979 INFO:teuthology.orchestra.run.smithi161.stdout:starting buffered write 2024-05-22T21:22:44.979 INFO:teuthology.orchestra.run.smithi161.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-05-22T21:22:44.979 INFO:teuthology.orchestra.run.smithi161.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-05-22T21:22:44.979 INFO:teuthology.orchestra.run.smithi161.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-05-22T21:22:44.980 INFO:teuthology.orchestra.run.smithi161.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-05-22T21:22:44.980 INFO:teuthology.orchestra.run.smithi161.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-05-22T21:22:44.980 INFO:teuthology.orchestra.run.smithi161.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-05-22T21:22:44.980 INFO:teuthology.orchestra.run.smithi161.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2024-05-22T21:22:45.180 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:22:45.180 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:22:45.229 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:22:45.309 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:22:45.309 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:22:45.309 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:22:45.319 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:22:45.319 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:22:45.375 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:22:45.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.564+0000 7f36a1000640 1 Processor -- start 2024-05-22T21:22:45.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.564+0000 7f36a1000640 1 -- start start 2024-05-22T21:22:45.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.564+0000 7f36a1000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f369c15e130 0x7f369c15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:45.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.564+0000 7f36a1000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f369c15d410 0x7f369c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:45.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.564+0000 7f36a1000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f369c071670 con 0x7f369c15dd50 2024-05-22T21:22:45.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.564+0000 7f36a1000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f369c15d870 con 0x7f369c15e130 2024-05-22T21:22:45.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.564+0000 7f36a1000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369c070a60 con 0x7f369c15d410 2024-05-22T21:22:45.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.564+0000 7f3699400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f369c15e130 0x7f369c15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:45.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.564+0000 7f3699e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f369c15dd50 0x7f369c06d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39920/0 (socket says 172.21.15.161:39920) 2024-05-22T21:22:45.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.564+0000 7f3699e00640 1 -- 172.21.15.161:0/2348856455 learned_addr learned my addr 172.21.15.161:0/2348856455 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:45.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.564+0000 7f3699400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f369c15e130 0x7f369c15ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60786/0 (socket says 172.21.15.161:60786) 2024-05-22T21:22:45.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.568+0000 7f369a800640 1 --2- 172.21.15.161:0/2348856455 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f369c15d410 0x7f369c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:45.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.568+0000 7f3699400640 1 -- 172.21.15.161:0/2348856455 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f369c15d410 msgr2=0x7f369c15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:45.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.568+0000 7f3699400640 1 --2- 172.21.15.161:0/2348856455 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f369c15d410 0x7f369c15b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:45.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.568+0000 7f3699400640 1 -- 172.21.15.161:0/2348856455 >> v1:172.21.15.161:6789/0 conn(0x7f369c15dd50 legacy=0x7f369c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:45.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.568+0000 7f3699400640 1 -- 172.21.15.161:0/2348856455 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c070d40 con 0x7f369c15e130 2024-05-22T21:22:45.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.568+0000 7f3698a00640 1 -- 172.21.15.161:0/2348856455 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261648065 0 0) 0x7f369c071670 con 0x7f369c15dd50 2024-05-22T21:22:45.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.568+0000 7f3699400640 1 --2- 172.21.15.161:0/2348856455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f369c15e130 0x7f369c15ced0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f36880097c0 tx=0x7f368801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e787fbe386b0cc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:45.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.568+0000 7f3698a00640 1 -- 172.21.15.161:0/2348856455 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36880ab670 con 0x7f369c15e130 2024-05-22T21:22:45.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.568+0000 7f3698a00640 1 -- 172.21.15.161:0/2348856455 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36880ab7d0 con 0x7f369c15e130 2024-05-22T21:22:45.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.568+0000 7f3698a00640 1 -- 172.21.15.161:0/2348856455 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36880abb60 con 0x7f369c15e130 2024-05-22T21:22:45.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f36a1000640 1 -- 172.21.15.161:0/2348856455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f369c15e130 msgr2=0x7f369c15ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:45.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f36a1000640 1 --2- 172.21.15.161:0/2348856455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f369c15e130 0x7f369c15ced0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f36880097c0 tx=0x7f368801c470 comp rx=0 tx=0).stop 2024-05-22T21:22:45.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f36a1000640 1 -- 172.21.15.161:0/2348856455 shutdown_connections 2024-05-22T21:22:45.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f36a1000640 1 --2- 172.21.15.161:0/2348856455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f369c15e130 0x7f369c15ced0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:45.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f36a1000640 1 --2- 172.21.15.161:0/2348856455 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f369c15d410 0x7f369c15b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:45.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f36a1000640 1 -- 172.21.15.161:0/2348856455 >> 172.21.15.161:0/2348856455 conn(0x7f369c0b0bc0 msgr2=0x7f369c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:45.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f36a1000640 1 -- 172.21.15.161:0/2348856455 shutdown_connections 2024-05-22T21:22:45.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f36a1000640 1 -- 172.21.15.161:0/2348856455 wait complete. 2024-05-22T21:22:45.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f36a1000640 1 Processor -- start 2024-05-22T21:22:45.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f36a1000640 1 -- start start 2024-05-22T21:22:45.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f36a1000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f369c15d410 0x7f369c154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:45.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f36a1000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f369c15dd50 0x7f369c157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:45.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f36a1000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f369c15e130 0x7f369c1407d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:45.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f36a1000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0b2d60 con 0x7f369c15dd50 2024-05-22T21:22:45.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f36a1000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0b2150 con 0x7f369c15d410 2024-05-22T21:22:45.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f36a1000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0b2430 con 0x7f369c15e130 2024-05-22T21:22:45.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f3699400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f369c15dd50 0x7f369c157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:45.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f3699400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f369c15dd50 0x7f369c157240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60798/0 (socket says 172.21.15.161:60798) 2024-05-22T21:22:45.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f3699400640 1 -- 172.21.15.161:0/4158659280 learned_addr learned my addr 172.21.15.161:0/4158659280 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:45.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f3699e00640 1 --2- 172.21.15.161:0/4158659280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f369c15d410 0x7f369c154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:45.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f369a800640 1 --2- 172.21.15.161:0/4158659280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f369c15e130 0x7f369c1407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:45.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f3699400640 1 -- 172.21.15.161:0/4158659280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f369c15e130 msgr2=0x7f369c1407d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:45.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f3699400640 1 --2- 172.21.15.161:0/4158659280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f369c15e130 0x7f369c1407d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:45.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f3699400640 1 -- 172.21.15.161:0/4158659280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f369c15d410 msgr2=0x7f369c154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:45.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f3699400640 1 --2- 172.21.15.161:0/4158659280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f369c15d410 0x7f369c154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:45.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.572+0000 7f3699400640 1 -- 172.21.15.161:0/4158659280 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3688009420 con 0x7f369c15dd50 2024-05-22T21:22:45.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.576+0000 7f3699400640 1 --2- 172.21.15.161:0/4158659280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f369c15dd50 0x7f369c157240 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f368801a0a0 tx=0x7f3688019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:45.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.576+0000 7f3692000640 1 -- 172.21.15.161:0/4158659280 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36880abe90 con 0x7f369c15dd50 2024-05-22T21:22:45.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.576+0000 7f3692000640 1 -- 172.21.15.161:0/4158659280 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36880bf450 con 0x7f369c15dd50 2024-05-22T21:22:45.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.576+0000 7f3692000640 1 -- 172.21.15.161:0/4158659280 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36880bf7e0 con 0x7f369c15dd50 2024-05-22T21:22:45.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.576+0000 7f36a1000640 1 -- 172.21.15.161:0/4158659280 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f369c0b2710 con 0x7f369c15dd50 2024-05-22T21:22:45.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.576+0000 7f36a1000640 1 -- 172.21.15.161:0/4158659280 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369c140d10 con 0x7f369c15dd50 2024-05-22T21:22:45.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.576+0000 7f36a1000640 1 -- 172.21.15.161:0/4158659280 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f369c067220 con 0x7f369c15dd50 2024-05-22T21:22:45.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.576+0000 7f3692000640 1 -- 172.21.15.161:0/4158659280 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f36880b43f0 con 0x7f369c15dd50 2024-05-22T21:22:45.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.576+0000 7f3692000640 1 --2- 172.21.15.161:0/4158659280 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f367c0666c0 0x7f367c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:45.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.576+0000 7f3692000640 1 -- 172.21.15.161:0/4158659280 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f368811ecc0 con 0x7f369c15dd50 2024-05-22T21:22:45.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.576+0000 7f3692000640 1 -- 172.21.15.161:0/4158659280 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7f367c06e2b0 con 0x7f369c15dd50 2024-05-22T21:22:45.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.576+0000 7f3699e00640 1 --2- 172.21.15.161:0/4158659280 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f367c0666c0 0x7f367c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:45.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.580+0000 7f3699e00640 1 --2- 172.21.15.161:0/4158659280 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f367c0666c0 0x7f367c068b80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f368c007fd0 tx=0x7f368c007d60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:45.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.580+0000 7f3692000640 1 -- 172.21.15.161:0/4158659280 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f36880c4020 con 0x7f369c15dd50 2024-05-22T21:22:45.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.736+0000 7f36a1000640 1 -- 172.21.15.161:0/4158659280 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f369c06c310 con 0x7f369c15dd50 2024-05-22T21:22:45.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.736+0000 7f3692000640 1 -- 172.21.15.161:0/4158659280 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16228 (secure 0 0 0) 0x7f3688125080 con 0x7f369c15dd50 2024-05-22T21:22:45.738 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:22:45.738 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":162,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:22:05.280635+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:20:32.457234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-22T21:20:32.951241+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.16","mappings":[{"from":2,"to":5}]},{"pgid":"19.25","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:22:45.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.740+0000 7f36a1000640 1 -- 172.21.15.161:0/4158659280 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f367c0666c0 msgr2=0x7f367c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:45.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.740+0000 7f36a1000640 1 --2- 172.21.15.161:0/4158659280 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f367c0666c0 0x7f367c068b80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f368c007fd0 tx=0x7f368c007d60 comp rx=0 tx=0).stop 2024-05-22T21:22:45.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.740+0000 7f36a1000640 1 -- 172.21.15.161:0/4158659280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f369c15dd50 msgr2=0x7f369c157240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:45.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.740+0000 7f36a1000640 1 --2- 172.21.15.161:0/4158659280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f369c15dd50 0x7f369c157240 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f368801a0a0 tx=0x7f3688019210 comp rx=0 tx=0).stop 2024-05-22T21:22:45.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.740+0000 7f36a1000640 1 -- 172.21.15.161:0/4158659280 shutdown_connections 2024-05-22T21:22:45.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.740+0000 7f36a1000640 1 --2- 172.21.15.161:0/4158659280 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f367c0666c0 0x7f367c068b80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:45.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.740+0000 7f36a1000640 1 --2- 172.21.15.161:0/4158659280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f369c15e130 0x7f369c1407d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:45.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.740+0000 7f36a1000640 1 --2- 172.21.15.161:0/4158659280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f369c15dd50 0x7f369c157240 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:45.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.740+0000 7f36a1000640 1 --2- 172.21.15.161:0/4158659280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f369c15d410 0x7f369c154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:45.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.740+0000 7f36a1000640 1 -- 172.21.15.161:0/4158659280 >> 172.21.15.161:0/4158659280 conn(0x7f369c0b0bc0 msgr2=0x7f369c06c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:45.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.740+0000 7f36a1000640 1 -- 172.21.15.161:0/4158659280 shutdown_connections 2024-05-22T21:22:45.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.740+0000 7f36a1000640 1 -- 172.21.15.161:0/4158659280 wait complete. 2024-05-22T21:22:45.759 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:20:32.457234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '157', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:20:32.951241+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:22:45.760 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:22:45.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.900+0000 7f42bdc00640 1 Processor -- start 2024-05-22T21:22:45.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.900+0000 7f42bdc00640 1 -- start start 2024-05-22T21:22:45.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.900+0000 7f42bdc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f42b813cfc0 0x7f42b81402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:45.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.900+0000 7f42bdc00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b813d3a0 con 0x7f42b815af60 2024-05-22T21:22:45.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.900+0000 7f42bdc00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b80cd920 con 0x7f42b813fe50 2024-05-22T21:22:45.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.900+0000 7f42bdc00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b80cdc00 con 0x7f42b813cfc0 2024-05-22T21:22:45.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.900+0000 7f42b6a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f42b813cfc0 0x7f42b81402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:45.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.900+0000 7f42b6000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f42b815af60 0x7f42b813f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:43996/0 (socket says 172.21.15.161:43996) 2024-05-22T21:22:45.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.900+0000 7f42b6000640 1 -- 172.21.15.161:0/2834922664 learned_addr learned my addr 172.21.15.161:0/2834922664 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:45.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.900+0000 7f42b6a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f42b813cfc0 0x7f42b81402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49562/0 (socket says 172.21.15.161:49562) 2024-05-22T21:22:45.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.900+0000 7f42b6a00640 1 -- 172.21.15.161:0/2834922664 >> v1:172.21.15.179:6789/0 conn(0x7f42b813fe50 legacy=0x7f42b806fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:45.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.900+0000 7f42b6a00640 1 -- 172.21.15.161:0/2834922664 >> v1:172.21.15.161:6789/0 conn(0x7f42b815af60 legacy=0x7f42b813f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:45.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.900+0000 7f42b6a00640 1 -- 172.21.15.161:0/2834922664 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b80cdee0 con 0x7f42b813cfc0 2024-05-22T21:22:45.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.904+0000 7f42b6a00640 1 --2- 172.21.15.161:0/2834922664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42b813cfc0 0x7f42b81402e0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f42a800ba60 tx=0x7f42a801ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36b3977db6a1bfb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:45.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.904+0000 7f42b5600640 1 -- 172.21.15.161:0/2834922664 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a80a7940 con 0x7f42b813cfc0 2024-05-22T21:22:45.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.904+0000 7f42b5600640 1 -- 172.21.15.161:0/2834922664 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a80a7aa0 con 0x7f42b813cfc0 2024-05-22T21:22:45.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.904+0000 7f42b5600640 1 -- 172.21.15.161:0/2834922664 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a80a7e30 con 0x7f42b813cfc0 2024-05-22T21:22:45.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.904+0000 7f42bdc00640 1 -- 172.21.15.161:0/2834922664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42b813cfc0 msgr2=0x7f42b81402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:45.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.904+0000 7f42bdc00640 1 --2- 172.21.15.161:0/2834922664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42b813cfc0 0x7f42b81402e0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f42a800ba60 tx=0x7f42a801ea70 comp rx=0 tx=0).stop 2024-05-22T21:22:45.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.904+0000 7f42bdc00640 1 -- 172.21.15.161:0/2834922664 shutdown_connections 2024-05-22T21:22:45.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.904+0000 7f42bdc00640 1 --2- 172.21.15.161:0/2834922664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42b813cfc0 0x7f42b81402e0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:45.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.904+0000 7f42bdc00640 1 -- 172.21.15.161:0/2834922664 >> 172.21.15.161:0/2834922664 conn(0x7f42b806a2d0 msgr2=0x7f42b806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:45.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.904+0000 7f42bdc00640 1 -- 172.21.15.161:0/2834922664 shutdown_connections 2024-05-22T21:22:45.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.904+0000 7f42bdc00640 1 -- 172.21.15.161:0/2834922664 wait complete. 2024-05-22T21:22:45.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42bdc00640 1 Processor -- start 2024-05-22T21:22:45.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42bdc00640 1 -- start start 2024-05-22T21:22:45.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42bdc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42b813cfc0 0x7f42b81375c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:45.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42bdc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42b813fe50 0x7f42b8154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:45.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42bdc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42b815af60 0x7f42b815c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:45.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42bdc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b81561e0 con 0x7f42b813fe50 2024-05-22T21:22:45.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42bdc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b8155860 con 0x7f42b813cfc0 2024-05-22T21:22:45.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42bdc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b8155490 con 0x7f42b815af60 2024-05-22T21:22:45.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42b6000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42b813fe50 0x7f42b8154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:45.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42b6000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42b813fe50 0x7f42b8154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49576/0 (socket says 172.21.15.161:49576) 2024-05-22T21:22:45.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42b6000640 1 -- 172.21.15.161:0/2949931077 learned_addr learned my addr 172.21.15.161:0/2949931077 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:45.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42b6a00640 1 --2- 172.21.15.161:0/2949931077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42b813cfc0 0x7f42b81375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:45.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42b7400640 1 --2- 172.21.15.161:0/2949931077 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42b815af60 0x7f42b815c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:45.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42b6000640 1 -- 172.21.15.161:0/2949931077 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42b815af60 msgr2=0x7f42b815c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:45.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42b6000640 1 --2- 172.21.15.161:0/2949931077 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42b815af60 0x7f42b815c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:45.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42b6000640 1 -- 172.21.15.161:0/2949931077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42b813cfc0 msgr2=0x7f42b81375c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:45.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42b6000640 1 --2- 172.21.15.161:0/2949931077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42b813cfc0 0x7f42b81375c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:45.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42b6000640 1 -- 172.21.15.161:0/2949931077 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a800b6c0 con 0x7f42b813fe50 2024-05-22T21:22:45.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42b6000640 1 --2- 172.21.15.161:0/2949931077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42b813fe50 0x7f42b8154ad0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f42a400c400 tx=0x7f42a400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:45.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42aea00640 1 -- 172.21.15.161:0/2949931077 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a4005cf0 con 0x7f42b813fe50 2024-05-22T21:22:45.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42aea00640 1 -- 172.21.15.161:0/2949931077 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a4005e50 con 0x7f42b813fe50 2024-05-22T21:22:45.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42aea00640 1 -- 172.21.15.161:0/2949931077 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a40072d0 con 0x7f42b813fe50 2024-05-22T21:22:45.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42bdc00640 1 -- 172.21.15.161:0/2949931077 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b8155b20 con 0x7f42b813fe50 2024-05-22T21:22:45.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.908+0000 7f42bdc00640 1 -- 172.21.15.161:0/2949931077 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b8137b00 con 0x7f42b813fe50 2024-05-22T21:22:45.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.912+0000 7f42bdc00640 1 -- 172.21.15.161:0/2949931077 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b8156320 con 0x7f42b813fe50 2024-05-22T21:22:45.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.912+0000 7f42aea00640 1 -- 172.21.15.161:0/2949931077 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f42a4007430 con 0x7f42b813fe50 2024-05-22T21:22:45.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.912+0000 7f42aea00640 1 --2- 172.21.15.161:0/2949931077 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4298066690 0x7f4298068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:45.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.912+0000 7f42aea00640 1 -- 172.21.15.161:0/2949931077 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f42a40836c0 con 0x7f42b813fe50 2024-05-22T21:22:45.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.912+0000 7f42b6a00640 1 --2- 172.21.15.161:0/2949931077 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4298066690 0x7f4298068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:45.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.912+0000 7f42aea00640 1 -- 172.21.15.161:0/2949931077 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7f429806e280 con 0x7f42b813fe50 2024-05-22T21:22:45.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.912+0000 7f42b6a00640 1 --2- 172.21.15.161:0/2949931077 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4298066690 0x7f4298068b50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f42a80b1000 tx=0x7f42a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:45.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:45.916+0000 7f42aea00640 1 -- 172.21.15.161:0/2949931077 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f42b8156320 con 0x7f42b813fe50 2024-05-22T21:22:46.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.068+0000 7f42bdc00640 1 -- 172.21.15.161:0/2949931077 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f42b8156320 con 0x7f42b813fe50 2024-05-22T21:22:46.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.072+0000 7f42aea00640 1 -- 172.21.15.161:0/2949931077 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v162) v1 ==== 93+0+10 (secure 0 0 0) 0x7f42b8156320 con 0x7f42b813fe50 2024-05-22T21:22:46.071 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:22:46.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.072+0000 7f42bdc00640 1 -- 172.21.15.161:0/2949931077 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4298066690 msgr2=0x7f4298068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.072+0000 7f42bdc00640 1 --2- 172.21.15.161:0/2949931077 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4298066690 0x7f4298068b50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f42a80b1000 tx=0x7f42a8008040 comp rx=0 tx=0).stop 2024-05-22T21:22:46.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.072+0000 7f42bdc00640 1 -- 172.21.15.161:0/2949931077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42b813fe50 msgr2=0x7f42b8154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.072+0000 7f42bdc00640 1 --2- 172.21.15.161:0/2949931077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42b813fe50 0x7f42b8154ad0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f42a400c400 tx=0x7f42a400c8c0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.072+0000 7f42bdc00640 1 -- 172.21.15.161:0/2949931077 shutdown_connections 2024-05-22T21:22:46.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.072+0000 7f42bdc00640 1 --2- 172.21.15.161:0/2949931077 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4298066690 0x7f4298068b50 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.072+0000 7f42bdc00640 1 --2- 172.21.15.161:0/2949931077 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f42b815af60 0x7f42b815c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.072+0000 7f42bdc00640 1 --2- 172.21.15.161:0/2949931077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42b813fe50 0x7f42b8154ad0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.072+0000 7f42bdc00640 1 --2- 172.21.15.161:0/2949931077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42b813cfc0 0x7f42b81375c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.072+0000 7f42bdc00640 1 -- 172.21.15.161:0/2949931077 >> 172.21.15.161:0/2949931077 conn(0x7f42b806a2d0 msgr2=0x7f42b806a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:46.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.072+0000 7f42bdc00640 1 -- 172.21.15.161:0/2949931077 shutdown_connections 2024-05-22T21:22:46.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.072+0000 7f42bdc00640 1 -- 172.21.15.161:0/2949931077 wait complete. 2024-05-22T21:22:46.092 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:22:46.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.228+0000 7fcddb600640 1 Processor -- start 2024-05-22T21:22:46.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.228+0000 7fcddb600640 1 -- start start 2024-05-22T21:22:46.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.232+0000 7fcddb600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcdd41402e0 0x7fcdd413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:46.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.232+0000 7fcddb600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcdd413fc80 0x7fcdd406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:46.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.232+0000 7fcddb600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd41400d0 con 0x7fcdd413cfc0 2024-05-22T21:22:46.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.232+0000 7fcddb600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd413d3a0 con 0x7fcdd413fc80 2024-05-22T21:22:46.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.232+0000 7fcddb600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd40cd920 con 0x7fcdd41402e0 2024-05-22T21:22:46.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.232+0000 7fcdd9600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcdd413fc80 0x7fcdd406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:46.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.232+0000 7fcdd9600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcdd413fc80 0x7fcdd406fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:58186/0 (socket says 172.21.15.161:58186) 2024-05-22T21:22:46.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.232+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcdd41402e0 0x7fcdd413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:46.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.232+0000 7fcdd8c00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fcdd413cfc0 0x7fcdd415af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:38780/0 (socket says 172.21.15.161:38780) 2024-05-22T21:22:46.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.232+0000 7fcdd9600640 1 -- 172.21.15.161:0/2761582816 learned_addr learned my addr 172.21.15.161:0/2761582816 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:46.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.232+0000 7fcdd9600640 1 -- 172.21.15.161:0/2761582816 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcdd41402e0 msgr2=0x7fcdd413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:46.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.232+0000 7fcdd9600640 1 --2- 172.21.15.161:0/2761582816 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcdd41402e0 0x7fcdd413f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.232+0000 7fcdd9600640 1 -- 172.21.15.161:0/2761582816 >> v1:172.21.15.179:6789/0 conn(0x7fcdd413cfc0 legacy=0x7fcdd415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.232+0000 7fcdd9600640 1 -- 172.21.15.161:0/2761582816 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd40cdc00 con 0x7fcdd413fc80 2024-05-22T21:22:46.233 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.232+0000 7fcdd3400640 1 -- 172.21.15.161:0/2761582816 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1135788601 0 0) 0x7fcdc4001120 con 0x7fcdd413cfc0 2024-05-22T21:22:46.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.236+0000 7fcdd9600640 1 --2- 172.21.15.161:0/2761582816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdd413fc80 0x7fcdd406fce0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009e50 tx=0x7fcdcc01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a03cbc0765cc37b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:46.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.236+0000 7fcdd3400640 1 -- 172.21.15.161:0/2761582816 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0a9480 con 0x7fcdd413fc80 2024-05-22T21:22:46.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.236+0000 7fcdd3400640 1 -- 172.21.15.161:0/2761582816 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc0a95e0 con 0x7fcdd413fc80 2024-05-22T21:22:46.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.236+0000 7fcdd3400640 1 -- 172.21.15.161:0/2761582816 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0a9970 con 0x7fcdd413fc80 2024-05-22T21:22:46.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.236+0000 7fcddb600640 1 -- 172.21.15.161:0/2761582816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdd413fc80 msgr2=0x7fcdd406fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.236+0000 7fcddb600640 1 --2- 172.21.15.161:0/2761582816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdd413fc80 0x7fcdd406fce0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009e50 tx=0x7fcdcc01c970 comp rx=0 tx=0).stop 2024-05-22T21:22:46.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.236+0000 7fcddb600640 1 -- 172.21.15.161:0/2761582816 shutdown_connections 2024-05-22T21:22:46.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.236+0000 7fcddb600640 1 --2- 172.21.15.161:0/2761582816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdd413fc80 0x7fcdd406fce0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.236+0000 7fcddb600640 1 --2- 172.21.15.161:0/2761582816 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcdd41402e0 0x7fcdd413f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.236+0000 7fcddb600640 1 -- 172.21.15.161:0/2761582816 >> 172.21.15.161:0/2761582816 conn(0x7fcdd406a2d0 msgr2=0x7fcdd406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:46.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.236+0000 7fcddb600640 1 -- 172.21.15.161:0/2761582816 shutdown_connections 2024-05-22T21:22:46.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.236+0000 7fcddb600640 1 -- 172.21.15.161:0/2761582816 wait complete. 2024-05-22T21:22:46.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.236+0000 7fcddb600640 1 Processor -- start 2024-05-22T21:22:46.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcddb600640 1 -- start start 2024-05-22T21:22:46.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcddb600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdd413cfc0 0x7fcdd4154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:46.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcddb600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcdd413fc80 0x7fcdd413ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:46.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcddb600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdd41402e0 0x7fcdd4155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:46.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcddb600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd4155210 con 0x7fcdd413cfc0 2024-05-22T21:22:46.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcddb600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd41423f0 con 0x7fcdd41402e0 2024-05-22T21:22:46.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcddb600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd4141870 con 0x7fcdd413fc80 2024-05-22T21:22:46.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcdd8c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdd413cfc0 0x7fcdd4154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:46.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcdd8c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdd413cfc0 0x7fcdd4154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49588/0 (socket says 172.21.15.161:49588) 2024-05-22T21:22:46.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcdd8c00640 1 -- 172.21.15.161:0/4119550588 learned_addr learned my addr 172.21.15.161:0/4119550588 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:46.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcdd3e00640 1 --2- 172.21.15.161:0/4119550588 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcdd413fc80 0x7fcdd413ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:46.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcdd9600640 1 --2- 172.21.15.161:0/4119550588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdd41402e0 0x7fcdd4155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:46.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcdd8c00640 1 -- 172.21.15.161:0/4119550588 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcdd413fc80 msgr2=0x7fcdd413ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcdd8c00640 1 --2- 172.21.15.161:0/4119550588 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcdd413fc80 0x7fcdd413ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcdd8c00640 1 -- 172.21.15.161:0/4119550588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdd41402e0 msgr2=0x7fcdd4155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcdd8c00640 1 --2- 172.21.15.161:0/4119550588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdd41402e0 0x7fcdd4155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcdd8c00640 1 -- 172.21.15.161:0/4119550588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdcc009b30 con 0x7fcdd413cfc0 2024-05-22T21:22:46.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcdd8c00640 1 --2- 172.21.15.161:0/4119550588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdd413cfc0 0x7fcdd4154cd0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400c480 tx=0x7fcdc400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:46.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcdd0c00640 1 -- 172.21.15.161:0/4119550588 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc40074e0 con 0x7fcdd413cfc0 2024-05-22T21:22:46.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcdd0c00640 1 -- 172.21.15.161:0/4119550588 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc4007660 con 0x7fcdd413cfc0 2024-05-22T21:22:46.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcdd0c00640 1 -- 172.21.15.161:0/4119550588 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4004a30 con 0x7fcdd413cfc0 2024-05-22T21:22:46.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcddb600640 1 -- 172.21.15.161:0/4119550588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd4141b50 con 0x7fcdd413cfc0 2024-05-22T21:22:46.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcddb600640 1 -- 172.21.15.161:0/4119550588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd4142110 con 0x7fcdd413cfc0 2024-05-22T21:22:46.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.240+0000 7fcddb600640 1 -- 172.21.15.161:0/4119550588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd4067220 con 0x7fcdd413cfc0 2024-05-22T21:22:46.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.244+0000 7fcdd0c00640 1 -- 172.21.15.161:0/4119550588 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcdc4004b90 con 0x7fcdd413cfc0 2024-05-22T21:22:46.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.244+0000 7fcdd0c00640 1 --2- 172.21.15.161:0/4119550588 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcdb4066690 0x7fcdb4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:46.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.244+0000 7fcdd0c00640 1 -- 172.21.15.161:0/4119550588 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcdc4083170 con 0x7fcdd413cfc0 2024-05-22T21:22:46.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.244+0000 7fcdd3e00640 1 --2- 172.21.15.161:0/4119550588 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcdb4066690 0x7fcdb4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:46.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.244+0000 7fcdd0c00640 1 -- 172.21.15.161:0/4119550588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7fcdb406e280 con 0x7fcdd413cfc0 2024-05-22T21:22:46.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.244+0000 7fcdd3e00640 1 --2- 172.21.15.161:0/4119550588 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcdb4066690 0x7fcdb4068b50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40b2120 tx=0x7fcdc0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:46.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.248+0000 7fcdd0c00640 1 -- 172.21.15.161:0/4119550588 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fcdc4050540 con 0x7fcdd413cfc0 2024-05-22T21:22:46.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.400+0000 7fcddb600640 1 -- 172.21.15.161:0/4119550588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcdd41400d0 con 0x7fcdd413cfc0 2024-05-22T21:22:46.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.400+0000 7fcdd0c00640 1 -- 172.21.15.161:0/4119550588 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v162) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcdd41400d0 con 0x7fcdd413cfc0 2024-05-22T21:22:46.401 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:22:46.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.404+0000 7fcddb600640 1 -- 172.21.15.161:0/4119550588 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcdb4066690 msgr2=0x7fcdb4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.404+0000 7fcddb600640 1 --2- 172.21.15.161:0/4119550588 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcdb4066690 0x7fcdb4068b50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40b2120 tx=0x7fcdc0009190 comp rx=0 tx=0).stop 2024-05-22T21:22:46.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.404+0000 7fcddb600640 1 -- 172.21.15.161:0/4119550588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdd413cfc0 msgr2=0x7fcdd4154cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.404+0000 7fcddb600640 1 --2- 172.21.15.161:0/4119550588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdd413cfc0 0x7fcdd4154cd0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400c480 tx=0x7fcdc400c940 comp rx=0 tx=0).stop 2024-05-22T21:22:46.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.404+0000 7fcddb600640 1 -- 172.21.15.161:0/4119550588 shutdown_connections 2024-05-22T21:22:46.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.404+0000 7fcddb600640 1 --2- 172.21.15.161:0/4119550588 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fcdb4066690 0x7fcdb4068b50 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.404+0000 7fcddb600640 1 --2- 172.21.15.161:0/4119550588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdd41402e0 0x7fcdd4155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.404+0000 7fcddb600640 1 --2- 172.21.15.161:0/4119550588 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcdd413fc80 0x7fcdd413ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.404+0000 7fcddb600640 1 --2- 172.21.15.161:0/4119550588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdd413cfc0 0x7fcdd4154cd0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.404+0000 7fcddb600640 1 -- 172.21.15.161:0/4119550588 >> 172.21.15.161:0/4119550588 conn(0x7fcdd406a2d0 msgr2=0x7fcdd40b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:46.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.404+0000 7fcddb600640 1 -- 172.21.15.161:0/4119550588 shutdown_connections 2024-05-22T21:22:46.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.404+0000 7fcddb600640 1 -- 172.21.15.161:0/4119550588 wait complete. 2024-05-22T21:22:46.422 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:22:46.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.560+0000 7fb5d2c00640 1 Processor -- start 2024-05-22T21:22:46.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.560+0000 7fb5d2c00640 1 -- start start 2024-05-22T21:22:46.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.564+0000 7fb5d2c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb5cc13cfc0 0x7fb5cc1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:46.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.564+0000 7fb5d2c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5cc13d3a0 con 0x7fb5cc13fe50 2024-05-22T21:22:46.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.564+0000 7fb5d2c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5cc0cd920 con 0x7fb5cc15af60 2024-05-22T21:22:46.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.564+0000 7fb5d2c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5cc0cdc00 con 0x7fb5cc13cfc0 2024-05-22T21:22:46.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.564+0000 7fb5cbe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb5cc13cfc0 0x7fb5cc1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:46.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.564+0000 7fb5cbe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb5cc13cfc0 0x7fb5cc1402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49592/0 (socket says 172.21.15.161:49592) 2024-05-22T21:22:46.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.564+0000 7fb5d0c00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb5cc13fe50 0x7fb5cc06fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44008/0 (socket says 172.21.15.161:44008) 2024-05-22T21:22:46.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.564+0000 7fb5cbe00640 1 -- 172.21.15.161:0/1012288969 learned_addr learned my addr 172.21.15.161:0/1012288969 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:46.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.564+0000 7fb5cbe00640 1 -- 172.21.15.161:0/1012288969 >> v1:172.21.15.179:6789/0 conn(0x7fb5cc15af60 legacy=0x7fb5cc13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.564+0000 7fb5cbe00640 1 -- 172.21.15.161:0/1012288969 >> v1:172.21.15.161:6789/0 conn(0x7fb5cc13fe50 legacy=0x7fb5cc06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.564+0000 7fb5cbe00640 1 -- 172.21.15.161:0/1012288969 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc0cdee0 con 0x7fb5cc13cfc0 2024-05-22T21:22:46.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.568+0000 7fb5cbe00640 1 --2- 172.21.15.161:0/1012288969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc13cfc0 0x7fb5cc1402e0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009870 tx=0x7fb5bc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ab71a95ebbb119 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:46.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.568+0000 7fb5caa00640 1 -- 172.21.15.161:0/1012288969 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc0a6040 con 0x7fb5cc13cfc0 2024-05-22T21:22:46.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.568+0000 7fb5caa00640 1 -- 172.21.15.161:0/1012288969 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc0aa400 con 0x7fb5cc13cfc0 2024-05-22T21:22:46.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.568+0000 7fb5caa00640 1 -- 172.21.15.161:0/1012288969 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc0aa790 con 0x7fb5cc13cfc0 2024-05-22T21:22:46.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.568+0000 7fb5d2c00640 1 -- 172.21.15.161:0/1012288969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc13cfc0 msgr2=0x7fb5cc1402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.568+0000 7fb5d2c00640 1 --2- 172.21.15.161:0/1012288969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc13cfc0 0x7fb5cc1402e0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009870 tx=0x7fb5bc01c4a0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.568+0000 7fb5d2c00640 1 -- 172.21.15.161:0/1012288969 shutdown_connections 2024-05-22T21:22:46.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.568+0000 7fb5d2c00640 1 --2- 172.21.15.161:0/1012288969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc13cfc0 0x7fb5cc1402e0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.568+0000 7fb5d2c00640 1 -- 172.21.15.161:0/1012288969 >> 172.21.15.161:0/1012288969 conn(0x7fb5cc06a2d0 msgr2=0x7fb5cc06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:46.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.568+0000 7fb5d2c00640 1 -- 172.21.15.161:0/1012288969 shutdown_connections 2024-05-22T21:22:46.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.568+0000 7fb5d2c00640 1 -- 172.21.15.161:0/1012288969 wait complete. 2024-05-22T21:22:46.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.568+0000 7fb5d2c00640 1 Processor -- start 2024-05-22T21:22:46.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.568+0000 7fb5d2c00640 1 -- start start 2024-05-22T21:22:46.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5d2c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5cc13cfc0 0x7fb5cc15c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:46.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5d2c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc13fe50 0x7fb5cc1595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:46.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5d2c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5cc15af60 0x7fb5cc156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:46.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5d2c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc1425a0 con 0x7fb5cc13fe50 2024-05-22T21:22:46.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5d2c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc1fa350 con 0x7fb5cc13cfc0 2024-05-22T21:22:46.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5d2c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc140700 con 0x7fb5cc15af60 2024-05-22T21:22:46.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5cb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc13fe50 0x7fb5cc1595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:46.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5cb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc13fe50 0x7fb5cc1595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49602/0 (socket says 172.21.15.161:49602) 2024-05-22T21:22:46.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5cb400640 1 -- 172.21.15.161:0/2237873532 learned_addr learned my addr 172.21.15.161:0/2237873532 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5cbe00640 1 --2- 172.21.15.161:0/2237873532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5cc13cfc0 0x7fb5cc15c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5d0c00640 1 --2- 172.21.15.161:0/2237873532 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5cc15af60 0x7fb5cc156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5cb400640 1 -- 172.21.15.161:0/2237873532 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5cc15af60 msgr2=0x7fb5cc156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5cb400640 1 --2- 172.21.15.161:0/2237873532 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5cc15af60 0x7fb5cc156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5cb400640 1 -- 172.21.15.161:0/2237873532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5cc13cfc0 msgr2=0x7fb5cc15c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5cb400640 1 --2- 172.21.15.161:0/2237873532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5cc13cfc0 0x7fb5cc15c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5cb400640 1 -- 172.21.15.161:0/2237873532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc0094d0 con 0x7fb5cc13fe50 2024-05-22T21:22:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5cb400640 1 --2- 172.21.15.161:0/2237873532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc13fe50 0x7fb5cc1595e0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800c530 tx=0x7fb5b800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5c3e00640 1 -- 172.21.15.161:0/2237873532 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8005220 con 0x7fb5cc13fe50 2024-05-22T21:22:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5c3e00640 1 -- 172.21.15.161:0/2237873532 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b8005380 con 0x7fb5cc13fe50 2024-05-22T21:22:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5d2c00640 1 -- 172.21.15.161:0/2237873532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5cc1f9590 con 0x7fb5cc13fe50 2024-05-22T21:22:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5c3e00640 1 -- 172.21.15.161:0/2237873532 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8007270 con 0x7fb5cc13fe50 2024-05-22T21:22:46.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5d2c00640 1 -- 172.21.15.161:0/2237873532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5cc1f9b50 con 0x7fb5cc13fe50 2024-05-22T21:22:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5d2c00640 1 -- 172.21.15.161:0/2237873532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5cc067220 con 0x7fb5cc13fe50 2024-05-22T21:22:46.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.572+0000 7fb5c3e00640 1 -- 172.21.15.161:0/2237873532 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb5b80073d0 con 0x7fb5cc13fe50 2024-05-22T21:22:46.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.576+0000 7fb5c3e00640 1 --2- 172.21.15.161:0/2237873532 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb5ac0666c0 0x7fb5ac068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:46.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.576+0000 7fb5c3e00640 1 -- 172.21.15.161:0/2237873532 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb5b8083a70 con 0x7fb5cc13fe50 2024-05-22T21:22:46.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.576+0000 7fb5cbe00640 1 --2- 172.21.15.161:0/2237873532 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb5ac0666c0 0x7fb5ac068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:46.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.576+0000 7fb5c3e00640 1 -- 172.21.15.161:0/2237873532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7fb5ac06e2b0 con 0x7fb5cc13fe50 2024-05-22T21:22:46.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.576+0000 7fb5cbe00640 1 --2- 172.21.15.161:0/2237873532 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb5ac0666c0 0x7fb5ac068b80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0ae000 tx=0x7fb5bc0b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:46.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.580+0000 7fb5c3e00640 1 -- 172.21.15.161:0/2237873532 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb5b8050db0 con 0x7fb5cc13fe50 2024-05-22T21:22:46.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.732+0000 7fb5d2c00640 1 -- 172.21.15.161:0/2237873532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb5cc13cbd0 con 0x7fb5cc13fe50 2024-05-22T21:22:46.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.732+0000 7fb5c3e00640 1 -- 172.21.15.161:0/2237873532 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v162) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb5b80543d0 con 0x7fb5cc13fe50 2024-05-22T21:22:46.731 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:22:46.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.732+0000 7fb5d2c00640 1 -- 172.21.15.161:0/2237873532 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb5ac0666c0 msgr2=0x7fb5ac068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.732+0000 7fb5d2c00640 1 --2- 172.21.15.161:0/2237873532 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb5ac0666c0 0x7fb5ac068b80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0ae000 tx=0x7fb5bc0b5040 comp rx=0 tx=0).stop 2024-05-22T21:22:46.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.732+0000 7fb5d2c00640 1 -- 172.21.15.161:0/2237873532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc13fe50 msgr2=0x7fb5cc1595e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.732+0000 7fb5d2c00640 1 --2- 172.21.15.161:0/2237873532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc13fe50 0x7fb5cc1595e0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800c530 tx=0x7fb5b800c9f0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.732+0000 7fb5d2c00640 1 -- 172.21.15.161:0/2237873532 shutdown_connections 2024-05-22T21:22:46.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.732+0000 7fb5d2c00640 1 --2- 172.21.15.161:0/2237873532 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb5ac0666c0 0x7fb5ac068b80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.732+0000 7fb5d2c00640 1 --2- 172.21.15.161:0/2237873532 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5cc15af60 0x7fb5cc156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.732+0000 7fb5d2c00640 1 --2- 172.21.15.161:0/2237873532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc13fe50 0x7fb5cc1595e0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.732+0000 7fb5d2c00640 1 --2- 172.21.15.161:0/2237873532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5cc13cfc0 0x7fb5cc15c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.732+0000 7fb5d2c00640 1 -- 172.21.15.161:0/2237873532 >> 172.21.15.161:0/2237873532 conn(0x7fb5cc06a2d0 msgr2=0x7fb5cc139bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:46.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.732+0000 7fb5d2c00640 1 -- 172.21.15.161:0/2237873532 shutdown_connections 2024-05-22T21:22:46.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.732+0000 7fb5d2c00640 1 -- 172.21.15.161:0/2237873532 wait complete. 2024-05-22T21:22:46.753 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:22:46.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.892+0000 7f860da00640 1 Processor -- start 2024-05-22T21:22:46.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.892+0000 7f860da00640 1 -- start start 2024-05-22T21:22:46.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.896+0000 7f860da00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f860815b600 0x7f860806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:46.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.896+0000 7f860da00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f860806cc40 con 0x7f860806c7e0 2024-05-22T21:22:46.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.896+0000 7f860da00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8608144bb0 con 0x7f8608143c40 2024-05-22T21:22:46.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.896+0000 7f860da00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f860815ba60 con 0x7f860815b600 2024-05-22T21:22:46.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.896+0000 7f85fe000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f860815b600 0x7f860806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:46.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.896+0000 7f85fe000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f860815b600 0x7f860806c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49606/0 (socket says 172.21.15.161:49606) 2024-05-22T21:22:46.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.896+0000 7f85fe000640 1 -- 172.21.15.161:0/1796594173 learned_addr learned my addr 172.21.15.161:0/1796594173 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:46.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.896+0000 7f8607400640 1 --1- 172.21.15.161:0/1796594173 >> v1:172.21.15.161:6789/0 conn(0x7f860806c7e0 0x7f86080b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44018/0 (socket says 172.21.15.161:44018) 2024-05-22T21:22:46.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.896+0000 7f85fe000640 1 -- 172.21.15.161:0/1796594173 >> v1:172.21.15.179:6789/0 conn(0x7f8608143c40 legacy=0x7f86081444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:46.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.896+0000 7f85fe000640 1 -- 172.21.15.161:0/1796594173 >> v1:172.21.15.161:6789/0 conn(0x7f860806c7e0 legacy=0x7f86080b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.896+0000 7f85fe000640 1 -- 172.21.15.161:0/1796594173 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860815efb0 con 0x7f860815b600 2024-05-22T21:22:46.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.896+0000 7f8606000640 1 -- 172.21.15.161:0/1796594173 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313534395 0 0) 0x7f860806cc40 con 0x7f860806c7e0 2024-05-22T21:22:46.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.896+0000 7f85fe000640 1 --2- 172.21.15.161:0/1796594173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f860815b600 0x7f860806c2a0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f85f0009870 tx=0x7f85f001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5143a7289c196529 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:46.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.896+0000 7f8606000640 1 -- 172.21.15.161:0/1796594173 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f00aa680 con 0x7f860815b600 2024-05-22T21:22:46.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.896+0000 7f8606000640 1 -- 172.21.15.161:0/1796594173 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f00aa7e0 con 0x7f860815b600 2024-05-22T21:22:46.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.896+0000 7f8606000640 1 -- 172.21.15.161:0/1796594173 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f00aab70 con 0x7f860815b600 2024-05-22T21:22:46.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f860da00640 1 -- 172.21.15.161:0/1796594173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f860815b600 msgr2=0x7f860806c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f860da00640 1 --2- 172.21.15.161:0/1796594173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f860815b600 0x7f860806c2a0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f85f0009870 tx=0x7f85f001c520 comp rx=0 tx=0).stop 2024-05-22T21:22:46.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f860da00640 1 -- 172.21.15.161:0/1796594173 shutdown_connections 2024-05-22T21:22:46.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f860da00640 1 --2- 172.21.15.161:0/1796594173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f860815b600 0x7f860806c2a0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f860da00640 1 -- 172.21.15.161:0/1796594173 >> 172.21.15.161:0/1796594173 conn(0x7f860806a2d0 msgr2=0x7f860806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:46.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f860da00640 1 -- 172.21.15.161:0/1796594173 shutdown_connections 2024-05-22T21:22:46.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f860da00640 1 -- 172.21.15.161:0/1796594173 wait complete. 2024-05-22T21:22:46.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f860da00640 1 Processor -- start 2024-05-22T21:22:46.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f860da00640 1 -- start start 2024-05-22T21:22:46.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f860da00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f860806c7e0 0x7f8608142f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:46.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f860da00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8608143c40 0x7f860815d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:46.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f860da00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f860815b600 0x7f860813cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:46.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f860da00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860815ea90 con 0x7f860806c7e0 2024-05-22T21:22:46.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f860da00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860815dcb0 con 0x7f860815b600 2024-05-22T21:22:46.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f860da00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f860815df10 con 0x7f8608143c40 2024-05-22T21:22:46.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f8606a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f860806c7e0 0x7f8608142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:46.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f8606a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f860806c7e0 0x7f8608142f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49614/0 (socket says 172.21.15.161:49614) 2024-05-22T21:22:46.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f8606a00640 1 -- 172.21.15.161:0/928085786 learned_addr learned my addr 172.21.15.161:0/928085786 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:46.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f85fe000640 1 --2- 172.21.15.161:0/928085786 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8608143c40 0x7f860815d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:46.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f8607400640 1 --2- 172.21.15.161:0/928085786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f860815b600 0x7f860813cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:46.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f8606a00640 1 -- 172.21.15.161:0/928085786 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8608143c40 msgr2=0x7f860815d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f8606a00640 1 --2- 172.21.15.161:0/928085786 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8608143c40 0x7f860815d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f8606a00640 1 -- 172.21.15.161:0/928085786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f860815b600 msgr2=0x7f860813cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f8606a00640 1 --2- 172.21.15.161:0/928085786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f860815b600 0x7f860813cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.900+0000 7f8606a00640 1 -- 172.21.15.161:0/928085786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f00094d0 con 0x7f860806c7e0 2024-05-22T21:22:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.904+0000 7f8606a00640 1 --2- 172.21.15.161:0/928085786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f860806c7e0 0x7f8608142f40 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c450 tx=0x7f85f800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.904+0000 7f85ff400640 1 -- 172.21.15.161:0/928085786 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8012830 con 0x7f860806c7e0 2024-05-22T21:22:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.904+0000 7f85ff400640 1 -- 172.21.15.161:0/928085786 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f80129b0 con 0x7f860806c7e0 2024-05-22T21:22:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.904+0000 7f85ff400640 1 -- 172.21.15.161:0/928085786 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8012d60 con 0x7f860806c7e0 2024-05-22T21:22:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.904+0000 7f860da00640 1 -- 172.21.15.161:0/928085786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f860815e1f0 con 0x7f860806c7e0 2024-05-22T21:22:46.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.904+0000 7f860da00640 1 -- 172.21.15.161:0/928085786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860815e7b0 con 0x7f860806c7e0 2024-05-22T21:22:46.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.904+0000 7f860da00640 1 -- 172.21.15.161:0/928085786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f860814a370 con 0x7f860806c7e0 2024-05-22T21:22:46.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.904+0000 7f85ff400640 1 -- 172.21.15.161:0/928085786 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f85f801c460 con 0x7f860806c7e0 2024-05-22T21:22:46.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.904+0000 7f85ff400640 1 --2- 172.21.15.161:0/928085786 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85e4066690 0x7f85e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:46.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.904+0000 7f85ff400640 1 -- 172.21.15.161:0/928085786 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f85f8084070 con 0x7f860806c7e0 2024-05-22T21:22:46.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.904+0000 7f85fe000640 1 --2- 172.21.15.161:0/928085786 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85e4066690 0x7f85e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:46.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.908+0000 7f85ff400640 1 -- 172.21.15.161:0/928085786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7f85e406e280 con 0x7f860806c7e0 2024-05-22T21:22:46.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.908+0000 7f85fe000640 1 --2- 172.21.15.161:0/928085786 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85e4066690 0x7f85e4068b50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f85f00ae000 tx=0x7f85f00a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:46.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:46.908+0000 7f85ff400640 1 -- 172.21.15.161:0/928085786 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f85f80b5cf0 con 0x7f860806c7e0 2024-05-22T21:22:47.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.076+0000 7f860da00640 1 -- 172.21.15.161:0/928085786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f860815ebd0 con 0x7f860806c7e0 2024-05-22T21:22:47.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.076+0000 7f85ff400640 1 -- 172.21.15.161:0/928085786 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v162) v1 ==== 80+0+0 (secure 0 0 0) 0x7f860814a370 con 0x7f860806c7e0 2024-05-22T21:22:47.076 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:22:47.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.076+0000 7f860da00640 1 -- 172.21.15.161:0/928085786 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85e4066690 msgr2=0x7f85e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:47.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.076+0000 7f860da00640 1 --2- 172.21.15.161:0/928085786 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85e4066690 0x7f85e4068b50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f85f00ae000 tx=0x7f85f00a7000 comp rx=0 tx=0).stop 2024-05-22T21:22:47.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.076+0000 7f860da00640 1 -- 172.21.15.161:0/928085786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f860806c7e0 msgr2=0x7f8608142f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:47.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.076+0000 7f860da00640 1 --2- 172.21.15.161:0/928085786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f860806c7e0 0x7f8608142f40 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c450 tx=0x7f85f800c910 comp rx=0 tx=0).stop 2024-05-22T21:22:47.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.076+0000 7f860da00640 1 -- 172.21.15.161:0/928085786 shutdown_connections 2024-05-22T21:22:47.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.076+0000 7f860da00640 1 --2- 172.21.15.161:0/928085786 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f85e4066690 0x7f85e4068b50 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.076+0000 7f860da00640 1 --2- 172.21.15.161:0/928085786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f860815b600 0x7f860813cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.076+0000 7f860da00640 1 --2- 172.21.15.161:0/928085786 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8608143c40 0x7f860815d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.076+0000 7f860da00640 1 --2- 172.21.15.161:0/928085786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f860806c7e0 0x7f8608142f40 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.076+0000 7f860da00640 1 -- 172.21.15.161:0/928085786 >> 172.21.15.161:0/928085786 conn(0x7f860806a2d0 msgr2=0x7f86080aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:47.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.076+0000 7f860da00640 1 -- 172.21.15.161:0/928085786 shutdown_connections 2024-05-22T21:22:47.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.080+0000 7f860da00640 1 -- 172.21.15.161:0/928085786 wait complete. 2024-05-22T21:22:47.097 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-22T21:22:47.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:22:47.097 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-22T21:22:47.126 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:22:47.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:22:47.126 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-22T21:22:47.183 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:22:47.183 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:22:47.193 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:22:47.193 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:22:47.243 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:22:47.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.380+0000 7f9e4be00640 1 Processor -- start 2024-05-22T21:22:47.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.380+0000 7f9e4be00640 1 -- start start 2024-05-22T21:22:47.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.380+0000 7f9e4be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e4c15af00 0x7f9e4c15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:47.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.380+0000 7f9e4be00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9e4c0d1150 0x7f9e4c0cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:47.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.380+0000 7f9e4be00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4c0cb8a0 con 0x7f9e4c15f040 2024-05-22T21:22:47.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.380+0000 7f9e4be00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4c15f530 con 0x7f9e4c0d1150 2024-05-22T21:22:47.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.380+0000 7f9e4be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c15f810 con 0x7f9e4c15af00 2024-05-22T21:22:47.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.380+0000 7f9e4b400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9e4c0d1150 0x7f9e4c0cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:47.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.380+0000 7f9e43e00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f9e4c15f040 0x7f9e4c0d0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:38802/0 (socket says 172.21.15.161:38802) 2024-05-22T21:22:47.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.380+0000 7f9e43e00640 1 -- 172.21.15.161:0/1806287869 learned_addr learned my addr 172.21.15.161:0/1806287869 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:47.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.380+0000 7f9e4aa00640 1 --2- 172.21.15.161:0/1806287869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e4c15af00 0x7f9e4c15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:47.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.380+0000 7f9e4b400640 1 -- 172.21.15.161:0/1806287869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e4c15af00 msgr2=0x7f9e4c15eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:47.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.380+0000 7f9e4b400640 1 --2- 172.21.15.161:0/1806287869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e4c15af00 0x7f9e4c15eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.380+0000 7f9e4b400640 1 -- 172.21.15.161:0/1806287869 >> v1:172.21.15.179:6789/0 conn(0x7f9e4c15f040 legacy=0x7f9e4c0d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:47.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.380+0000 7f9e4b400640 1 -- 172.21.15.161:0/1806287869 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c0cbb20 con 0x7f9e4c0d1150 2024-05-22T21:22:47.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.384+0000 7f9e4b400640 1 --2- 172.21.15.161:0/1806287869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e4c0d1150 0x7f9e4c0cc320 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009d90 tx=0x7f9e4401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb51c8901f490aab server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:47.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.384+0000 7f9e4a000640 1 -- 172.21.15.161:0/1806287869 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e440a58b0 con 0x7f9e4c0d1150 2024-05-22T21:22:47.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.384+0000 7f9e4a000640 1 -- 172.21.15.161:0/1806287869 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e440a5a10 con 0x7f9e4c0d1150 2024-05-22T21:22:47.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.384+0000 7f9e4a000640 1 -- 172.21.15.161:0/1806287869 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e440a5da0 con 0x7f9e4c0d1150 2024-05-22T21:22:47.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.384+0000 7f9e4be00640 1 -- 172.21.15.161:0/1806287869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e4c0d1150 msgr2=0x7f9e4c0cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:47.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.384+0000 7f9e4be00640 1 --2- 172.21.15.161:0/1806287869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e4c0d1150 0x7f9e4c0cc320 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009d90 tx=0x7f9e4401c8e0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.384+0000 7f9e4be00640 1 -- 172.21.15.161:0/1806287869 shutdown_connections 2024-05-22T21:22:47.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.384+0000 7f9e4be00640 1 --2- 172.21.15.161:0/1806287869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e4c0d1150 0x7f9e4c0cc320 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.384+0000 7f9e4be00640 1 --2- 172.21.15.161:0/1806287869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e4c15af00 0x7f9e4c15eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.384+0000 7f9e4be00640 1 -- 172.21.15.161:0/1806287869 >> 172.21.15.161:0/1806287869 conn(0x7f9e4c099a40 msgr2=0x7f9e4c099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:47.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.384+0000 7f9e4be00640 1 -- 172.21.15.161:0/1806287869 shutdown_connections 2024-05-22T21:22:47.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.384+0000 7f9e4be00640 1 -- 172.21.15.161:0/1806287869 wait complete. 2024-05-22T21:22:47.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e4be00640 1 Processor -- start 2024-05-22T21:22:47.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e4be00640 1 -- start start 2024-05-22T21:22:47.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e4be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e4c0d1150 0x7f9e4c14bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:47.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e4be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c15af00 0x7f9e4c153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e4be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e4c15f040 0x7f9e4c1548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e4be00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c0c8d10 con 0x7f9e4c15af00 2024-05-22T21:22:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e4be00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c1524a0 con 0x7f9e4c0d1150 2024-05-22T21:22:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e4be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c1539f0 con 0x7f9e4c15f040 2024-05-22T21:22:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e43e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c15af00 0x7f9e4c153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e4aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e4c0d1150 0x7f9e4c14bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e43e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c15af00 0x7f9e4c153e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49630/0 (socket says 172.21.15.161:49630) 2024-05-22T21:22:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e4aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e4c0d1150 0x7f9e4c14bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:58242/0 (socket says 172.21.15.161:58242) 2024-05-22T21:22:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e43e00640 1 -- 172.21.15.161:0/4125458751 learned_addr learned my addr 172.21.15.161:0/4125458751 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e4b400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e4c15f040 0x7f9e4c1548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e43e00640 1 -- 172.21.15.161:0/4125458751 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e4c15f040 msgr2=0x7f9e4c1548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e43e00640 1 --2- 172.21.15.161:0/4125458751 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e4c15f040 0x7f9e4c1548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e43e00640 1 -- 172.21.15.161:0/4125458751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e4c0d1150 msgr2=0x7f9e4c14bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:47.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e43e00640 1 --2- 172.21.15.161:0/4125458751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e4c0d1150 0x7f9e4c14bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e43e00640 1 -- 172.21.15.161:0/4125458751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e44009a70 con 0x7f9e4c15af00 2024-05-22T21:22:47.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e43e00640 1 --2- 172.21.15.161:0/4125458751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c15af00 0x7f9e4c153e80 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400c4d0 tx=0x7f9e3400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:47.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e42a00640 1 -- 172.21.15.161:0/4125458751 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34005cf0 con 0x7f9e4c15af00 2024-05-22T21:22:47.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e42a00640 1 -- 172.21.15.161:0/4125458751 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e34005e50 con 0x7f9e4c15af00 2024-05-22T21:22:47.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e42a00640 1 -- 172.21.15.161:0/4125458751 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34007270 con 0x7f9e4c15af00 2024-05-22T21:22:47.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e4be00640 1 -- 172.21.15.161:0/4125458751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4c1552d0 con 0x7f9e4c15af00 2024-05-22T21:22:47.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.388+0000 7f9e4be00640 1 -- 172.21.15.161:0/4125458751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4c155850 con 0x7f9e4c15af00 2024-05-22T21:22:47.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.392+0000 7f9e4be00640 1 -- 172.21.15.161:0/4125458751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4c14c0f0 con 0x7f9e4c15af00 2024-05-22T21:22:47.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.392+0000 7f9e42a00640 1 -- 172.21.15.161:0/4125458751 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9e340073d0 con 0x7f9e4c15af00 2024-05-22T21:22:47.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.392+0000 7f9e42a00640 1 --2- 172.21.15.161:0/4125458751 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e28066690 0x7f9e28068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:47.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.392+0000 7f9e42a00640 1 -- 172.21.15.161:0/4125458751 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9e34083690 con 0x7f9e4c15af00 2024-05-22T21:22:47.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.392+0000 7f9e4aa00640 1 --2- 172.21.15.161:0/4125458751 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e28066690 0x7f9e28068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:47.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.392+0000 7f9e42a00640 1 -- 172.21.15.161:0/4125458751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7f9e2806e280 con 0x7f9e4c15af00 2024-05-22T21:22:47.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.392+0000 7f9e4aa00640 1 --2- 172.21.15.161:0/4125458751 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e28066690 0x7f9e28068b50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c165790 tx=0x7f9e3c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:47.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.396+0000 7f9e42a00640 1 -- 172.21.15.161:0/4125458751 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9e34017060 con 0x7f9e4c15af00 2024-05-22T21:22:47.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.552+0000 7f9e4be00640 1 -- 172.21.15.161:0/4125458751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e4c067070 con 0x7f9e4c15af00 2024-05-22T21:22:47.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.552+0000 7f9e42a00640 1 -- 172.21.15.161:0/4125458751 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16228 (secure 0 0 0) 0x7f9e340509d0 con 0x7f9e4c15af00 2024-05-22T21:22:47.552 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:22:47.552 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":162,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:22:05.280635+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:20:32.457234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-22T21:20:32.951241+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.16","mappings":[{"from":2,"to":5}]},{"pgid":"19.25","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:22:47.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.552+0000 7f9e4be00640 1 -- 172.21.15.161:0/4125458751 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e28066690 msgr2=0x7f9e28068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:47.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.552+0000 7f9e4be00640 1 --2- 172.21.15.161:0/4125458751 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e28066690 0x7f9e28068b50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c165790 tx=0x7f9e3c009190 comp rx=0 tx=0).stop 2024-05-22T21:22:47.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.552+0000 7f9e4be00640 1 -- 172.21.15.161:0/4125458751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c15af00 msgr2=0x7f9e4c153e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:47.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.552+0000 7f9e4be00640 1 --2- 172.21.15.161:0/4125458751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c15af00 0x7f9e4c153e80 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400c4d0 tx=0x7f9e3400c990 comp rx=0 tx=0).stop 2024-05-22T21:22:47.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.552+0000 7f9e4be00640 1 -- 172.21.15.161:0/4125458751 shutdown_connections 2024-05-22T21:22:47.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.552+0000 7f9e4be00640 1 --2- 172.21.15.161:0/4125458751 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9e28066690 0x7f9e28068b50 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.552+0000 7f9e4be00640 1 --2- 172.21.15.161:0/4125458751 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9e4c15f040 0x7f9e4c1548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.552+0000 7f9e4be00640 1 --2- 172.21.15.161:0/4125458751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e4c15af00 0x7f9e4c153e80 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.552+0000 7f9e4be00640 1 --2- 172.21.15.161:0/4125458751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e4c0d1150 0x7f9e4c14bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.552+0000 7f9e4be00640 1 -- 172.21.15.161:0/4125458751 >> 172.21.15.161:0/4125458751 conn(0x7f9e4c099a40 msgr2=0x7f9e4c06ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:47.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.552+0000 7f9e4be00640 1 -- 172.21.15.161:0/4125458751 shutdown_connections 2024-05-22T21:22:47.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.552+0000 7f9e4be00640 1 -- 172.21.15.161:0/4125458751 wait complete. 2024-05-22T21:22:47.575 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:20:32.457234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '157', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:20:32.951241+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:22:47.575 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:22:47.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f887a400640 1 Processor -- start 2024-05-22T21:22:47.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f887a400640 1 -- start start 2024-05-22T21:22:47.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f887a400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f887413cfc0 0x7f88741402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:47.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f887a400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f887415af60 0x7f887413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:47.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f887a400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88741400d0 con 0x7f887413fc80 2024-05-22T21:22:47.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f887a400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887413d3a0 con 0x7f887413cfc0 2024-05-22T21:22:47.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f887a400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88740cd920 con 0x7f887415af60 2024-05-22T21:22:47.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8873e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f887413fc80 0x7f887415dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44028/0 (socket says 172.21.15.161:44028) 2024-05-22T21:22:47.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8873e00640 1 -- 172.21.15.161:0/622251393 learned_addr learned my addr 172.21.15.161:0/622251393 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:47.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8873400640 1 --2- 172.21.15.161:0/622251393 >> v2:172.21.15.179:3300/0 conn(0x7f887413cfc0 0x7f88741402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:47.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872a00640 1 --2- 172.21.15.161:0/622251393 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f887415af60 0x7f887413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:47.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872000640 1 -- 172.21.15.161:0/622251393 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466165207 0 0) 0x7f886c0014a0 con 0x7f887413fc80 2024-05-22T21:22:47.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872000640 1 -- 172.21.15.161:0/622251393 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8858003530 con 0x7f887413fc80 2024-05-22T21:22:47.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872000640 1 -- 172.21.15.161:0/622251393 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1380786009 0 0) 0x7f8858003530 con 0x7f887413fc80 2024-05-22T21:22:47.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872000640 1 -- 172.21.15.161:0/622251393 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f887415af60 msgr2=0x7f887413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:47.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872000640 1 --2- 172.21.15.161:0/622251393 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f887415af60 0x7f887413f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872000640 1 -- 172.21.15.161:0/622251393 >> v2:172.21.15.179:3300/0 conn(0x7f887413cfc0 msgr2=0x7f88741402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:47.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872000640 1 --2- 172.21.15.161:0/622251393 >> v2:172.21.15.179:3300/0 conn(0x7f887413cfc0 0x7f88741402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872000640 1 -- 172.21.15.161:0/622251393 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88740cdc00 con 0x7f887413fc80 2024-05-22T21:22:47.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872000640 1 -- 172.21.15.161:0/622251393 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f886c001b90 con 0x7f887413fc80 2024-05-22T21:22:47.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872000640 1 -- 172.21.15.161:0/622251393 >> v1:172.21.15.161:6789/0 conn(0x7f887413fc80 legacy=0x7f887415dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:47.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872000640 1 --2- 172.21.15.161:0/622251393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8858003980 0x7f8858003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:47.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872000640 1 --2- 172.21.15.161:0/622251393 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f887415af60 0x7f88580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:47.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872000640 1 --2- 172.21.15.161:0/622251393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8858004d10 0x7f8858005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:47.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872000640 1 -- 172.21.15.161:0/622251393 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887413d3a0 con 0x7f8858004d10 2024-05-22T21:22:47.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872000640 1 -- 172.21.15.161:0/622251393 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88740cd920 con 0x7f8858003980 2024-05-22T21:22:47.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872000640 1 -- 172.21.15.161:0/622251393 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885800b9c0 con 0x7f887415af60 2024-05-22T21:22:47.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8873e00640 1 --2- 172.21.15.161:0/622251393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8858004d10 0x7f8858005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:47.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8873400640 1 --2- 172.21.15.161:0/622251393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8858003980 0x7f8858003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:47.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8872a00640 1 --2- 172.21.15.161:0/622251393 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f887415af60 0x7f88580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:47.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8873e00640 1 -- 172.21.15.161:0/622251393 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f887415af60 msgr2=0x7f88580047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:47.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8873e00640 1 --2- 172.21.15.161:0/622251393 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f887415af60 0x7f88580047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8873e00640 1 -- 172.21.15.161:0/622251393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8858003980 msgr2=0x7f8858003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:47.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8873e00640 1 --2- 172.21.15.161:0/622251393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8858003980 0x7f8858003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.716+0000 7f8873e00640 1 -- 172.21.15.161:0/622251393 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8858003530 con 0x7f8858004d10 2024-05-22T21:22:47.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.720+0000 7f8873e00640 1 --2- 172.21.15.161:0/622251393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8858004d10 0x7f8858005190 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f886c00a830 tx=0x7f886c01fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9c562ba349cecc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:47.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.720+0000 7f8872000640 1 -- 172.21.15.161:0/622251393 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c0aa710 con 0x7f8858004d10 2024-05-22T21:22:47.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.720+0000 7f8872000640 1 -- 172.21.15.161:0/622251393 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c0aa890 con 0x7f8858004d10 2024-05-22T21:22:47.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.720+0000 7f8872000640 1 -- 172.21.15.161:0/622251393 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c0aac40 con 0x7f8858004d10 2024-05-22T21:22:47.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.720+0000 7f887a400640 1 -- 172.21.15.161:0/622251393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8858004d10 msgr2=0x7f8858005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:47.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.720+0000 7f887a400640 1 --2- 172.21.15.161:0/622251393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8858004d10 0x7f8858005190 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f886c00a830 tx=0x7f886c01fae0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.720+0000 7f8873e00640 1 -- 172.21.15.161:0/622251393 reap_dead start 2024-05-22T21:22:47.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.720+0000 7f887a400640 1 -- 172.21.15.161:0/622251393 shutdown_connections 2024-05-22T21:22:47.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.720+0000 7f887a400640 1 -- 172.21.15.161:0/622251393 >> 172.21.15.161:0/622251393 conn(0x7f887406a2d0 msgr2=0x7f887406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:47.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.720+0000 7f887a400640 1 -- 172.21.15.161:0/622251393 shutdown_connections 2024-05-22T21:22:47.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.720+0000 7f887a400640 1 -- 172.21.15.161:0/622251393 wait complete. 2024-05-22T21:22:47.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f887a400640 1 Processor -- start 2024-05-22T21:22:47.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f887a400640 1 -- start start 2024-05-22T21:22:47.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f887a400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8858004d10 0x7f887413c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:47.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f887a400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88740b3930 0x7f88741412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:47.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f887a400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88740b3d90 0x7f8874141800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:47.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f887a400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88740af750 con 0x7f88740b3d90 2024-05-22T21:22:47.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f887a400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88740b23d0 con 0x7f88740b3930 2024-05-22T21:22:47.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f887a400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88740af470 con 0x7f8858004d10 2024-05-22T21:22:47.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f8873e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88740b3d90 0x7f8874141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:47.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f8873e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88740b3d90 0x7f8874141800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49636/0 (socket says 172.21.15.161:49636) 2024-05-22T21:22:47.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f8873e00640 1 -- 172.21.15.161:0/58534566 learned_addr learned my addr 172.21.15.161:0/58534566 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:47.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f8873400640 1 --2- 172.21.15.161:0/58534566 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8858004d10 0x7f887413c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:47.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f8872a00640 1 --2- 172.21.15.161:0/58534566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88740b3930 0x7f88741412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:47.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f8873e00640 1 -- 172.21.15.161:0/58534566 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8858004d10 msgr2=0x7f887413c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:47.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f8873e00640 1 --2- 172.21.15.161:0/58534566 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8858004d10 0x7f887413c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f8873e00640 1 -- 172.21.15.161:0/58534566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88740b3930 msgr2=0x7f88741412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:47.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f8873e00640 1 --2- 172.21.15.161:0/58534566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88740b3930 0x7f88741412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f8873e00640 1 -- 172.21.15.161:0/58534566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88740b1d80 con 0x7f88740b3d90 2024-05-22T21:22:47.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f8873e00640 1 --2- 172.21.15.161:0/58534566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88740b3d90 0x7f8874141800 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f886c00a830 tx=0x7f8858005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:47.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f886b400640 1 -- 172.21.15.161:0/58534566 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c00a070 con 0x7f88740b3d90 2024-05-22T21:22:47.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f886b400640 1 -- 172.21.15.161:0/58534566 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c0aa710 con 0x7f88740b3d90 2024-05-22T21:22:47.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f886b400640 1 -- 172.21.15.161:0/58534566 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c001b70 con 0x7f88740b3d90 2024-05-22T21:22:47.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f887a400640 1 -- 172.21.15.161:0/58534566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88740b2060 con 0x7f88740b3d90 2024-05-22T21:22:47.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.724+0000 7f887a400640 1 -- 172.21.15.161:0/58534566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887413ca40 con 0x7f88740b3d90 2024-05-22T21:22:47.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.728+0000 7f887a400640 1 -- 172.21.15.161:0/58534566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88741291b0 con 0x7f88740b3d90 2024-05-22T21:22:47.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.728+0000 7f886b400640 1 -- 172.21.15.161:0/58534566 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f886c004b70 con 0x7f88740b3d90 2024-05-22T21:22:47.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.728+0000 7f886b400640 1 --2- 172.21.15.161:0/58534566 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f88540666c0 0x7f8854068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:47.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.728+0000 7f886b400640 1 -- 172.21.15.161:0/58534566 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f886c0b9070 con 0x7f88740b3d90 2024-05-22T21:22:47.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.728+0000 7f8873400640 1 --2- 172.21.15.161:0/58534566 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f88540666c0 0x7f8854068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:47.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.728+0000 7f886b400640 1 -- 172.21.15.161:0/58534566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7f885406e2b0 con 0x7f88740b3d90 2024-05-22T21:22:47.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.728+0000 7f8873400640 1 --2- 172.21.15.161:0/58534566 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f88540666c0 0x7f8854068b80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8864009b40 tx=0x7f8864009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:47.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.732+0000 7f886b400640 1 -- 172.21.15.161:0/58534566 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f88741291b0 con 0x7f88740b3d90 2024-05-22T21:22:47.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.888+0000 7f887a400640 1 -- 172.21.15.161:0/58534566 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88741400d0 con 0x7f88740b3d90 2024-05-22T21:22:47.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.888+0000 7f886b400640 1 -- 172.21.15.161:0/58534566 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v162) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88741400d0 con 0x7f88740b3d90 2024-05-22T21:22:47.888 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:22:47.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.888+0000 7f887a400640 1 -- 172.21.15.161:0/58534566 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f88540666c0 msgr2=0x7f8854068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:47.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.888+0000 7f887a400640 1 --2- 172.21.15.161:0/58534566 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f88540666c0 0x7f8854068b80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8864009b40 tx=0x7f8864009190 comp rx=0 tx=0).stop 2024-05-22T21:22:47.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.888+0000 7f887a400640 1 -- 172.21.15.161:0/58534566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88740b3d90 msgr2=0x7f8874141800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:47.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.888+0000 7f887a400640 1 --2- 172.21.15.161:0/58534566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88740b3d90 0x7f8874141800 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f886c00a830 tx=0x7f8858005800 comp rx=0 tx=0).stop 2024-05-22T21:22:47.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.888+0000 7f887a400640 1 -- 172.21.15.161:0/58534566 shutdown_connections 2024-05-22T21:22:47.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.888+0000 7f887a400640 1 --2- 172.21.15.161:0/58534566 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f88540666c0 0x7f8854068b80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.888+0000 7f887a400640 1 --2- 172.21.15.161:0/58534566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88740b3d90 0x7f8874141800 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.888+0000 7f887a400640 1 --2- 172.21.15.161:0/58534566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88740b3930 0x7f88741412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.888+0000 7f887a400640 1 --2- 172.21.15.161:0/58534566 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8858004d10 0x7f887413c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:47.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.888+0000 7f887a400640 1 -- 172.21.15.161:0/58534566 >> 172.21.15.161:0/58534566 conn(0x7f887406a2d0 msgr2=0x7f887415da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:47.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.888+0000 7f887a400640 1 -- 172.21.15.161:0/58534566 shutdown_connections 2024-05-22T21:22:47.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:47.888+0000 7f887a400640 1 -- 172.21.15.161:0/58534566 wait complete. 2024-05-22T21:22:47.910 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:22:48.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.048+0000 7fa804400640 1 Processor -- start 2024-05-22T21:22:48.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.048+0000 7fa804400640 1 -- start start 2024-05-22T21:22:48.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.048+0000 7fa804400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa7fc13cfc0 0x7fa7fc1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.048+0000 7fa804400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7fc15af60 0x7fa7fc13f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.048+0000 7fa804400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7fc1400d0 con 0x7fa7fc13fc80 2024-05-22T21:22:48.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.048+0000 7fa804400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7fc13d3a0 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.048+0000 7fa804400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc0cd920 con 0x7fa7fc15af60 2024-05-22T21:22:48.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.048+0000 7fa801a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa7fc13cfc0 0x7fa7fc1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.048+0000 7fa802400640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa7fc13fc80 0x7fa7fc15dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:38808/0 (socket says 172.21.15.161:38808) 2024-05-22T21:22:48.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.048+0000 7fa802400640 1 -- 172.21.15.161:0/1510893547 learned_addr learned my addr 172.21.15.161:0/1510893547 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:48.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.048+0000 7fa801000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7fc15af60 0x7fa7fc13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.048+0000 7fa801a00640 1 -- 172.21.15.161:0/1510893547 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7fc15af60 msgr2=0x7fa7fc13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:48.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.048+0000 7fa801a00640 1 --2- 172.21.15.161:0/1510893547 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7fc15af60 0x7fa7fc13f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.048+0000 7fa801a00640 1 -- 172.21.15.161:0/1510893547 >> v1:172.21.15.179:6789/0 conn(0x7fa7fc13fc80 legacy=0x7fa7fc15dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.048+0000 7fa801a00640 1 -- 172.21.15.161:0/1510893547 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc0cdc00 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.048+0000 7fa7fbe00640 1 -- 172.21.15.161:0/1510893547 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910821459 0 0) 0x7fa7f40014a0 con 0x7fa7fc13fc80 2024-05-22T21:22:48.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.052+0000 7fa801a00640 1 --2- 172.21.15.161:0/1510893547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7fc13cfc0 0x7fa7fc1402e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00097c0 tx=0x7fa7f001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b787251eb5126168 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:48.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.052+0000 7fa7fbe00640 1 -- 172.21.15.161:0/1510893547 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00ab600 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.052+0000 7fa7fbe00640 1 -- 172.21.15.161:0/1510893547 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f00a7050 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.052+0000 7fa7fbe00640 1 -- 172.21.15.161:0/1510893547 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00ab980 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.052+0000 7fa804400640 1 -- 172.21.15.161:0/1510893547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7fc13cfc0 msgr2=0x7fa7fc1402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.052+0000 7fa804400640 1 --2- 172.21.15.161:0/1510893547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7fc13cfc0 0x7fa7fc1402e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00097c0 tx=0x7fa7f001c410 comp rx=0 tx=0).stop 2024-05-22T21:22:48.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.052+0000 7fa804400640 1 -- 172.21.15.161:0/1510893547 shutdown_connections 2024-05-22T21:22:48.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.052+0000 7fa804400640 1 --2- 172.21.15.161:0/1510893547 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7fc15af60 0x7fa7fc13f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.052+0000 7fa804400640 1 --2- 172.21.15.161:0/1510893547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7fc13cfc0 0x7fa7fc1402e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.052+0000 7fa804400640 1 -- 172.21.15.161:0/1510893547 >> 172.21.15.161:0/1510893547 conn(0x7fa7fc06a2d0 msgr2=0x7fa7fc06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:48.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.052+0000 7fa804400640 1 -- 172.21.15.161:0/1510893547 shutdown_connections 2024-05-22T21:22:48.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.052+0000 7fa804400640 1 -- 172.21.15.161:0/1510893547 wait complete. 2024-05-22T21:22:48.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa804400640 1 Processor -- start 2024-05-22T21:22:48.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa804400640 1 -- start start 2024-05-22T21:22:48.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa804400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7fc13cfc0 0x7fa7fc13c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa804400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7fc13fc80 0x7fa7fc15c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa804400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7fc15af60 0x7fa7fc13ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa804400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc13da70 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa804400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc13dd50 con 0x7fa7fc15af60 2024-05-22T21:22:48.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa804400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc13e030 con 0x7fa7fc13fc80 2024-05-22T21:22:48.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa801a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7fc13cfc0 0x7fa7fc13c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa801a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7fc13cfc0 0x7fa7fc13c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49648/0 (socket says 172.21.15.161:49648) 2024-05-22T21:22:48.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa801a00640 1 -- 172.21.15.161:0/1299882534 learned_addr learned my addr 172.21.15.161:0/1299882534 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:48.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa802400640 1 --2- 172.21.15.161:0/1299882534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7fc15af60 0x7fa7fc13ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa801000640 1 --2- 172.21.15.161:0/1299882534 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7fc13fc80 0x7fa7fc15c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa801a00640 1 -- 172.21.15.161:0/1299882534 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7fc13fc80 msgr2=0x7fa7fc15c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa801a00640 1 --2- 172.21.15.161:0/1299882534 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7fc13fc80 0x7fa7fc15c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa801a00640 1 -- 172.21.15.161:0/1299882534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7fc15af60 msgr2=0x7fa7fc13ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa801a00640 1 --2- 172.21.15.161:0/1299882534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7fc15af60 0x7fa7fc13ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa801a00640 1 -- 172.21.15.161:0/1299882534 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f0009420 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa801a00640 1 --2- 172.21.15.161:0/1299882534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7fc13cfc0 0x7fa7fc13c590 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00ae040 tx=0x7fa7f001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:48.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa7f9600640 1 -- 172.21.15.161:0/1299882534 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00b7450 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa7f9600640 1 -- 172.21.15.161:0/1299882534 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f00b75b0 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa804400640 1 -- 172.21.15.161:0/1299882534 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7fc13e310 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa7f9600640 1 -- 172.21.15.161:0/1299882534 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00b7960 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa804400640 1 -- 172.21.15.161:0/1299882534 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7fc13cad0 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.056+0000 7fa804400640 1 -- 172.21.15.161:0/1299882534 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7fc067220 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.060+0000 7fa7f9600640 1 -- 172.21.15.161:0/1299882534 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa7f00b7ac0 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.060+0000 7fa7f9600640 1 --2- 172.21.15.161:0/1299882534 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7dc066690 0x7fa7dc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.060+0000 7fa7f9600640 1 -- 172.21.15.161:0/1299882534 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa7f011ea40 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.060+0000 7fa801000640 1 --2- 172.21.15.161:0/1299882534 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7dc066690 0x7fa7dc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.060+0000 7fa7f9600640 1 -- 172.21.15.161:0/1299882534 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7fa7dc06e280 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.060+0000 7fa801000640 1 --2- 172.21.15.161:0/1299882534 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7dc066690 0x7fa7dc068b50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc159f30 tx=0x7fa7e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:48.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.064+0000 7fa7f9600640 1 -- 172.21.15.161:0/1299882534 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa7f00b0060 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.224+0000 7fa804400640 1 -- 172.21.15.161:0/1299882534 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa7fc0cc5c0 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.224+0000 7fa7f9600640 1 -- 172.21.15.161:0/1299882534 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v162) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa7f00bb020 con 0x7fa7fc13cfc0 2024-05-22T21:22:48.225 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:22:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.228+0000 7fa804400640 1 -- 172.21.15.161:0/1299882534 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7dc066690 msgr2=0x7fa7dc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.228+0000 7fa804400640 1 --2- 172.21.15.161:0/1299882534 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7dc066690 0x7fa7dc068b50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc159f30 tx=0x7fa7e8008040 comp rx=0 tx=0).stop 2024-05-22T21:22:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.228+0000 7fa804400640 1 -- 172.21.15.161:0/1299882534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7fc13cfc0 msgr2=0x7fa7fc13c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.228+0000 7fa804400640 1 --2- 172.21.15.161:0/1299882534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7fc13cfc0 0x7fa7fc13c590 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00ae040 tx=0x7fa7f001ac50 comp rx=0 tx=0).stop 2024-05-22T21:22:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.228+0000 7fa804400640 1 -- 172.21.15.161:0/1299882534 shutdown_connections 2024-05-22T21:22:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.228+0000 7fa804400640 1 --2- 172.21.15.161:0/1299882534 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7dc066690 0x7fa7dc068b50 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.228+0000 7fa804400640 1 --2- 172.21.15.161:0/1299882534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7fc15af60 0x7fa7fc13ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.228+0000 7fa804400640 1 --2- 172.21.15.161:0/1299882534 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7fc13fc80 0x7fa7fc15c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.228+0000 7fa804400640 1 --2- 172.21.15.161:0/1299882534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7fc13cfc0 0x7fa7fc13c590 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.228+0000 7fa804400640 1 -- 172.21.15.161:0/1299882534 >> 172.21.15.161:0/1299882534 conn(0x7fa7fc06a2d0 msgr2=0x7fa7fc0aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:48.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.228+0000 7fa804400640 1 -- 172.21.15.161:0/1299882534 shutdown_connections 2024-05-22T21:22:48.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.228+0000 7fa804400640 1 -- 172.21.15.161:0/1299882534 wait complete. 2024-05-22T21:22:48.245 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:22:48.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.384+0000 7f2caee00640 1 Processor -- start 2024-05-22T21:22:48.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.384+0000 7f2caee00640 1 -- start start 2024-05-22T21:22:48.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.384+0000 7f2caee00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2ca81402e0 0x7f2ca813f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.384+0000 7f2caee00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca8140090 con 0x7f2ca813fc80 2024-05-22T21:22:48.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.384+0000 7f2caee00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca813d3a0 con 0x7f2ca813cfc0 2024-05-22T21:22:48.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.384+0000 7f2caee00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca80cd920 con 0x7f2ca81402e0 2024-05-22T21:22:48.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.384+0000 7f2ca7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2ca81402e0 0x7f2ca813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.384+0000 7f2ca7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2ca81402e0 0x7f2ca813f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49662/0 (socket says 172.21.15.161:49662) 2024-05-22T21:22:48.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.384+0000 7f2ca7400640 1 -- 172.21.15.161:0/715657 learned_addr learned my addr 172.21.15.161:0/715657 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:48.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.384+0000 7f2cace00640 1 --1- 172.21.15.161:0/715657 >> v1:172.21.15.161:6789/0 conn(0x7f2ca813fc80 0x7f2ca806fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44044/0 (socket says 172.21.15.161:44044) 2024-05-22T21:22:48.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.384+0000 7f2ca7400640 1 -- 172.21.15.161:0/715657 >> v1:172.21.15.179:6789/0 conn(0x7f2ca813cfc0 legacy=0x7f2ca815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:48.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.384+0000 7f2ca6a00640 1 -- 172.21.15.161:0/715657 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938274504 0 0) 0x7f2ca8140090 con 0x7f2ca813fc80 2024-05-22T21:22:48.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.384+0000 7f2ca7400640 1 -- 172.21.15.161:0/715657 >> v1:172.21.15.161:6789/0 conn(0x7f2ca813fc80 legacy=0x7f2ca806fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.384+0000 7f2ca7400640 1 -- 172.21.15.161:0/715657 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80cdc00 con 0x7f2ca81402e0 2024-05-22T21:22:48.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.388+0000 7f2ca7400640 1 --2- 172.21.15.161:0/715657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ca81402e0 0x7f2ca813f740 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009870 tx=0x7f2c9401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be918e6091f19bb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:48.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.388+0000 7f2ca6a00640 1 -- 172.21.15.161:0/715657 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c940a6040 con 0x7f2ca81402e0 2024-05-22T21:22:48.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.388+0000 7f2ca6a00640 1 -- 172.21.15.161:0/715657 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c940aa400 con 0x7f2ca81402e0 2024-05-22T21:22:48.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.388+0000 7f2ca6a00640 1 -- 172.21.15.161:0/715657 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c940aa790 con 0x7f2ca81402e0 2024-05-22T21:22:48.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.388+0000 7f2caee00640 1 -- 172.21.15.161:0/715657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ca81402e0 msgr2=0x7f2ca813f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.388+0000 7f2caee00640 1 --2- 172.21.15.161:0/715657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ca81402e0 0x7f2ca813f740 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009870 tx=0x7f2c9401c4a0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.388+0000 7f2caee00640 1 -- 172.21.15.161:0/715657 shutdown_connections 2024-05-22T21:22:48.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.388+0000 7f2caee00640 1 --2- 172.21.15.161:0/715657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ca81402e0 0x7f2ca813f740 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.388+0000 7f2caee00640 1 -- 172.21.15.161:0/715657 >> 172.21.15.161:0/715657 conn(0x7f2ca806a2d0 msgr2=0x7f2ca806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:48.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.388+0000 7f2caee00640 1 -- 172.21.15.161:0/715657 shutdown_connections 2024-05-22T21:22:48.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.388+0000 7f2caee00640 1 -- 172.21.15.161:0/715657 wait complete. 2024-05-22T21:22:48.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2caee00640 1 Processor -- start 2024-05-22T21:22:48.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2caee00640 1 -- start start 2024-05-22T21:22:48.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2caee00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ca813cfc0 0x7f2ca815b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2caee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ca813fc80 0x7f2ca813a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2caee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ca81402e0 0x7f2ca8156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2caee00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca813f8a0 con 0x7f2ca813cfc0 2024-05-22T21:22:48.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2caee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca813f5c0 con 0x7f2ca81402e0 2024-05-22T21:22:48.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2caee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca813d6c0 con 0x7f2ca813fc80 2024-05-22T21:22:48.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2ca7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ca813cfc0 0x7f2ca815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2ca7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ca813cfc0 0x7f2ca815b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49666/0 (socket says 172.21.15.161:49666) 2024-05-22T21:22:48.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2ca7e00640 1 -- 172.21.15.161:0/1036076710 learned_addr learned my addr 172.21.15.161:0/1036076710 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:48.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2cace00640 1 --2- 172.21.15.161:0/1036076710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ca81402e0 0x7f2ca8156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2ca7400640 1 --2- 172.21.15.161:0/1036076710 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ca813fc80 0x7f2ca813a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2ca7e00640 1 -- 172.21.15.161:0/1036076710 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ca813fc80 msgr2=0x7f2ca813a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2ca7e00640 1 --2- 172.21.15.161:0/1036076710 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ca813fc80 0x7f2ca813a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2ca7e00640 1 -- 172.21.15.161:0/1036076710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ca81402e0 msgr2=0x7f2ca8156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2ca7e00640 1 --2- 172.21.15.161:0/1036076710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ca81402e0 0x7f2ca8156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2ca7e00640 1 -- 172.21.15.161:0/1036076710 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c940094d0 con 0x7f2ca813cfc0 2024-05-22T21:22:48.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2ca7400640 1 --2- 172.21.15.161:0/1036076710 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ca813fc80 0x7f2ca813a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:22:48.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2ca7e00640 1 --2- 172.21.15.161:0/1036076710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ca813cfc0 0x7f2ca815b740 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800c3c0 tx=0x7f2c9800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:48.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2c9fe00640 1 -- 172.21.15.161:0/1036076710 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980052d0 con 0x7f2ca813cfc0 2024-05-22T21:22:48.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2c9fe00640 1 -- 172.21.15.161:0/1036076710 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c98005430 con 0x7f2ca813cfc0 2024-05-22T21:22:48.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2c9fe00640 1 -- 172.21.15.161:0/1036076710 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98003840 con 0x7f2ca813cfc0 2024-05-22T21:22:48.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2caee00640 1 -- 172.21.15.161:0/1036076710 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca813d9a0 con 0x7f2ca813cfc0 2024-05-22T21:22:48.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.392+0000 7f2caee00640 1 -- 172.21.15.161:0/1036076710 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca8159ca0 con 0x7f2ca813cfc0 2024-05-22T21:22:48.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.396+0000 7f2caee00640 1 -- 172.21.15.161:0/1036076710 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca8067220 con 0x7f2ca813cfc0 2024-05-22T21:22:48.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.396+0000 7f2c9fe00640 1 -- 172.21.15.161:0/1036076710 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2c980039a0 con 0x7f2ca813cfc0 2024-05-22T21:22:48.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.396+0000 7f2c9fe00640 1 --2- 172.21.15.161:0/1036076710 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c88066690 0x7f2c88068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.396+0000 7f2c9fe00640 1 -- 172.21.15.161:0/1036076710 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2c98083660 con 0x7f2ca813cfc0 2024-05-22T21:22:48.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.396+0000 7f2ca7400640 1 --2- 172.21.15.161:0/1036076710 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c88066690 0x7f2c88068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.396+0000 7f2c9fe00640 1 -- 172.21.15.161:0/1036076710 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7f2c8806e280 con 0x7f2ca813cfc0 2024-05-22T21:22:48.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.396+0000 7f2ca7400640 1 --2- 172.21.15.161:0/1036076710 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c88066690 0x7f2c88068b50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2c940ae000 tx=0x7f2c940a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:48.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.400+0000 7f2c9fe00640 1 -- 172.21.15.161:0/1036076710 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2c98017020 con 0x7f2ca813cfc0 2024-05-22T21:22:48.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.556+0000 7f2caee00640 1 -- 172.21.15.161:0/1036076710 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2ca8071d00 con 0x7f2ca813cfc0 2024-05-22T21:22:48.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.556+0000 7f2c9fe00640 1 -- 172.21.15.161:0/1036076710 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v162) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2c98050a30 con 0x7f2ca813cfc0 2024-05-22T21:22:48.557 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:22:48.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.560+0000 7f2caee00640 1 -- 172.21.15.161:0/1036076710 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c88066690 msgr2=0x7f2c88068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.560+0000 7f2caee00640 1 --2- 172.21.15.161:0/1036076710 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c88066690 0x7f2c88068b50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2c940ae000 tx=0x7f2c940a7000 comp rx=0 tx=0).stop 2024-05-22T21:22:48.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.560+0000 7f2caee00640 1 -- 172.21.15.161:0/1036076710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ca813cfc0 msgr2=0x7f2ca815b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.560+0000 7f2caee00640 1 --2- 172.21.15.161:0/1036076710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ca813cfc0 0x7f2ca815b740 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800c3c0 tx=0x7f2c9800c880 comp rx=0 tx=0).stop 2024-05-22T21:22:48.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.560+0000 7f2caee00640 1 -- 172.21.15.161:0/1036076710 shutdown_connections 2024-05-22T21:22:48.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.560+0000 7f2caee00640 1 --2- 172.21.15.161:0/1036076710 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2c88066690 0x7f2c88068b50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.560+0000 7f2caee00640 1 --2- 172.21.15.161:0/1036076710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ca81402e0 0x7f2ca8156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.560+0000 7f2caee00640 1 --2- 172.21.15.161:0/1036076710 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ca813fc80 0x7f2ca813a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.560+0000 7f2caee00640 1 --2- 172.21.15.161:0/1036076710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ca813cfc0 0x7f2ca815b740 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.560+0000 7f2caee00640 1 -- 172.21.15.161:0/1036076710 >> 172.21.15.161:0/1036076710 conn(0x7f2ca806a2d0 msgr2=0x7f2ca806fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:48.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.560+0000 7f2caee00640 1 -- 172.21.15.161:0/1036076710 shutdown_connections 2024-05-22T21:22:48.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.560+0000 7f2caee00640 1 -- 172.21.15.161:0/1036076710 wait complete. 2024-05-22T21:22:48.579 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:22:48.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.712+0000 7fc455200640 1 Processor -- start 2024-05-22T21:22:48.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc455200640 1 -- start start 2024-05-22T21:22:48.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc455200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc450143c40 0x7fc45015b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc455200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc4501444a0 0x7fc45006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc455200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc450144900 0x7fc45006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc455200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45015efb0 con 0x7fc450144900 2024-05-22T21:22:48.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc455200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45015f290 con 0x7fc4501444a0 2024-05-22T21:22:48.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc455200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45015f570 con 0x7fc450143c40 2024-05-22T21:22:48.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc44e800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc450144900 0x7fc45006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc44e800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc450144900 0x7fc45006c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49676/0 (socket says 172.21.15.161:49676) 2024-05-22T21:22:48.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc44e800640 1 -- 172.21.15.161:0/3550828109 learned_addr learned my addr 172.21.15.161:0/3550828109 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:48.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc445400640 1 --2- 172.21.15.161:0/3550828109 >> v2:172.21.15.179:3300/0 conn(0x7fc4501444a0 0x7fc45006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc44de00640 1 --2- 172.21.15.161:0/3550828109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc450143c40 0x7fc45015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc44e800640 1 -- 172.21.15.161:0/3550828109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc450143c40 msgr2=0x7fc45015b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:48.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc44e800640 1 --2- 172.21.15.161:0/3550828109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc450143c40 0x7fc45015b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc44e800640 1 -- 172.21.15.161:0/3550828109 >> v2:172.21.15.179:3300/0 conn(0x7fc4501444a0 msgr2=0x7fc45006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:48.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc44e800640 1 --2- 172.21.15.161:0/3550828109 >> v2:172.21.15.179:3300/0 conn(0x7fc4501444a0 0x7fc45006c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc44e800640 1 -- 172.21.15.161:0/3550828109 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45015f850 con 0x7fc450144900 2024-05-22T21:22:48.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.716+0000 7fc44de00640 1 --2- 172.21.15.161:0/3550828109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc450143c40 0x7fc45015b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:22:48.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.720+0000 7fc44e800640 1 --2- 172.21.15.161:0/3550828109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc450144900 0x7fc45006c7e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fc450071fe0 tx=0x7fc44801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c8840f298a8050 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:48.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.720+0000 7fc44d400640 1 -- 172.21.15.161:0/3550828109 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4480ab640 con 0x7fc450144900 2024-05-22T21:22:48.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.720+0000 7fc44d400640 1 -- 172.21.15.161:0/3550828109 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4480a7070 con 0x7fc450144900 2024-05-22T21:22:48.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.720+0000 7fc44d400640 1 -- 172.21.15.161:0/3550828109 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4480ab9c0 con 0x7fc450144900 2024-05-22T21:22:48.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.720+0000 7fc455200640 1 -- 172.21.15.161:0/3550828109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc450144900 msgr2=0x7fc45006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.720+0000 7fc455200640 1 --2- 172.21.15.161:0/3550828109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc450144900 0x7fc45006c7e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fc450071fe0 tx=0x7fc44801c520 comp rx=0 tx=0).stop 2024-05-22T21:22:48.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.720+0000 7fc455200640 1 -- 172.21.15.161:0/3550828109 shutdown_connections 2024-05-22T21:22:48.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.720+0000 7fc455200640 1 --2- 172.21.15.161:0/3550828109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc450144900 0x7fc45006c7e0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.720+0000 7fc455200640 1 --2- 172.21.15.161:0/3550828109 >> v2:172.21.15.179:3300/0 conn(0x7fc4501444a0 0x7fc45006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.720+0000 7fc455200640 1 --2- 172.21.15.161:0/3550828109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc450143c40 0x7fc45015b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.720+0000 7fc455200640 1 -- 172.21.15.161:0/3550828109 >> 172.21.15.161:0/3550828109 conn(0x7fc45006a2d0 msgr2=0x7fc45006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:48.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.720+0000 7fc455200640 1 -- 172.21.15.161:0/3550828109 shutdown_connections 2024-05-22T21:22:48.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.720+0000 7fc455200640 1 -- 172.21.15.161:0/3550828109 wait complete. 2024-05-22T21:22:48.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.720+0000 7fc455200640 1 Processor -- start 2024-05-22T21:22:48.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.720+0000 7fc455200640 1 -- start start 2024-05-22T21:22:48.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc455200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc450143c40 0x7fc45015a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc455200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4501444a0 0x7fc450136330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc455200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc450144900 0x7fc45006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc455200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4500713c0 con 0x7fc450143c40 2024-05-22T21:22:48.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc455200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc450136870 con 0x7fc4501444a0 2024-05-22T21:22:48.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc455200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4500717e0 con 0x7fc450144900 2024-05-22T21:22:48.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc44de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc450143c40 0x7fc45015a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc44de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc450143c40 0x7fc45015a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49690/0 (socket says 172.21.15.161:49690) 2024-05-22T21:22:48.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc44de00640 1 -- 172.21.15.161:0/152231484 learned_addr learned my addr 172.21.15.161:0/152231484 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:48.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc44e800640 1 --2- 172.21.15.161:0/152231484 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc450144900 0x7fc45006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc445400640 1 --2- 172.21.15.161:0/152231484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4501444a0 0x7fc450136330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc44de00640 1 -- 172.21.15.161:0/152231484 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc450144900 msgr2=0x7fc45006fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc44de00640 1 --2- 172.21.15.161:0/152231484 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc450144900 0x7fc45006fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc44de00640 1 -- 172.21.15.161:0/152231484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4501444a0 msgr2=0x7fc450136330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc44de00640 1 --2- 172.21.15.161:0/152231484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4501444a0 0x7fc450136330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc44de00640 1 -- 172.21.15.161:0/152231484 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4480094d0 con 0x7fc450143c40 2024-05-22T21:22:48.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc44de00640 1 --2- 172.21.15.161:0/152231484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc450143c40 0x7fc45015a860 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc44000e810 tx=0x7fc44000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:48.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc446a00640 1 -- 172.21.15.161:0/152231484 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440004030 con 0x7fc450143c40 2024-05-22T21:22:48.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc446a00640 1 -- 172.21.15.161:0/152231484 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc440004190 con 0x7fc450143c40 2024-05-22T21:22:48.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc446a00640 1 -- 172.21.15.161:0/152231484 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440004520 con 0x7fc450143c40 2024-05-22T21:22:48.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc455200640 1 -- 172.21.15.161:0/152231484 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc450070740 con 0x7fc450143c40 2024-05-22T21:22:48.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc455200640 1 -- 172.21.15.161:0/152231484 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc450070d00 con 0x7fc450143c40 2024-05-22T21:22:48.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.724+0000 7fc455200640 1 -- 172.21.15.161:0/152231484 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc450067220 con 0x7fc450143c40 2024-05-22T21:22:48.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.728+0000 7fc446a00640 1 -- 172.21.15.161:0/152231484 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc44000c6d0 con 0x7fc450143c40 2024-05-22T21:22:48.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.728+0000 7fc446a00640 1 --2- 172.21.15.161:0/152231484 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc42c066690 0x7fc42c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:48.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.728+0000 7fc446a00640 1 -- 172.21.15.161:0/152231484 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc440083e10 con 0x7fc450143c40 2024-05-22T21:22:48.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.728+0000 7fc445400640 1 --2- 172.21.15.161:0/152231484 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc42c066690 0x7fc42c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:48.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.728+0000 7fc446a00640 1 -- 172.21.15.161:0/152231484 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7fc42c06e280 con 0x7fc450143c40 2024-05-22T21:22:48.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.728+0000 7fc445400640 1 --2- 172.21.15.161:0/152231484 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc42c066690 0x7fc42c068b50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc4500cd990 tx=0x7fc43800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:48.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.732+0000 7fc446a00640 1 -- 172.21.15.161:0/152231484 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc440020060 con 0x7fc450143c40 2024-05-22T21:22:48.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.896+0000 7fc455200640 1 -- 172.21.15.161:0/152231484 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc4500ceaa0 con 0x7fc450143c40 2024-05-22T21:22:48.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.896+0000 7fc446a00640 1 -- 172.21.15.161:0/152231484 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v162) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc440020070 con 0x7fc450143c40 2024-05-22T21:22:48.898 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:22:48.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.900+0000 7fc455200640 1 -- 172.21.15.161:0/152231484 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc42c066690 msgr2=0x7fc42c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.900+0000 7fc455200640 1 --2- 172.21.15.161:0/152231484 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc42c066690 0x7fc42c068b50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc4500cd990 tx=0x7fc43800b040 comp rx=0 tx=0).stop 2024-05-22T21:22:48.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.900+0000 7fc455200640 1 -- 172.21.15.161:0/152231484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc450143c40 msgr2=0x7fc45015a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:48.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.900+0000 7fc455200640 1 --2- 172.21.15.161:0/152231484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc450143c40 0x7fc45015a860 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc44000e810 tx=0x7fc44000ecd0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.900+0000 7fc455200640 1 -- 172.21.15.161:0/152231484 shutdown_connections 2024-05-22T21:22:48.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.900+0000 7fc455200640 1 --2- 172.21.15.161:0/152231484 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc42c066690 0x7fc42c068b50 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.900+0000 7fc455200640 1 --2- 172.21.15.161:0/152231484 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc450144900 0x7fc45006fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.900+0000 7fc455200640 1 --2- 172.21.15.161:0/152231484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4501444a0 0x7fc450136330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.900+0000 7fc455200640 1 --2- 172.21.15.161:0/152231484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc450143c40 0x7fc45015a860 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:48.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.900+0000 7fc455200640 1 -- 172.21.15.161:0/152231484 >> 172.21.15.161:0/152231484 conn(0x7fc45006a2d0 msgr2=0x7fc4501548d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:48.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.900+0000 7fc455200640 1 -- 172.21.15.161:0/152231484 shutdown_connections 2024-05-22T21:22:48.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:48.900+0000 7fc455200640 1 -- 172.21.15.161:0/152231484 wait complete. 2024-05-22T21:22:48.919 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-22T21:22:48.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:22:48.919 DEBUG:teuthology.orchestra.run.smithi179:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-22T21:22:48.949 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:22:48.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:22:48.949 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-22T21:22:48.999 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:22:48.999 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:22:49.008 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:22:49.009 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:22:49.009 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:22:49.051 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:22:49.051 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:22:49.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.184+0000 7f59d4800640 1 Processor -- start 2024-05-22T21:22:49.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.184+0000 7f59d4800640 1 -- start start 2024-05-22T21:22:49.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.184+0000 7f59d4800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59cc15e130 0x7f59cc15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.184+0000 7f59d4800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f59cc15d410 0x7f59cc15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.184+0000 7f59d4800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59cc071670 con 0x7f59cc15dd50 2024-05-22T21:22:49.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.184+0000 7f59d4800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59cc15d870 con 0x7f59cc15d410 2024-05-22T21:22:49.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.184+0000 7f59d4800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc070a60 con 0x7f59cc15e130 2024-05-22T21:22:49.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.188+0000 7f59d2800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f59cc15d410 0x7f59cc15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.188+0000 7f59d2800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f59cc15d410 0x7f59cc15b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49700/0 (socket says 172.21.15.161:49700) 2024-05-22T21:22:49.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.188+0000 7f59d2800640 1 -- 172.21.15.161:0/3818843534 learned_addr learned my addr 172.21.15.161:0/3818843534 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:49.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.188+0000 7f59d1400640 1 --2- 172.21.15.161:0/3818843534 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59cc15e130 0x7f59cc15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.188+0000 7f59d2800640 1 -- 172.21.15.161:0/3818843534 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59cc15e130 msgr2=0x7f59cc15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:49.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.188+0000 7f59d2800640 1 --2- 172.21.15.161:0/3818843534 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59cc15e130 0x7f59cc15ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.188+0000 7f59d2800640 1 -- 172.21.15.161:0/3818843534 >> v1:172.21.15.179:6789/0 conn(0x7f59cc15dd50 legacy=0x7f59cc06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:49.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.188+0000 7f59d2800640 1 -- 172.21.15.161:0/3818843534 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc070d40 con 0x7f59cc15d410 2024-05-22T21:22:49.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.188+0000 7f59d2800640 1 --2- 172.21.15.161:0/3818843534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15d410 0x7f59cc15b6c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009e50 tx=0x7f59c801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85291439833b9941 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:49.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.188+0000 7f59d0a00640 1 -- 172.21.15.161:0/3818843534 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c80a5830 con 0x7f59cc15d410 2024-05-22T21:22:49.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.188+0000 7f59d0a00640 1 -- 172.21.15.161:0/3818843534 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c80a5990 con 0x7f59cc15d410 2024-05-22T21:22:49.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.188+0000 7f59d0a00640 1 -- 172.21.15.161:0/3818843534 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c80a5d20 con 0x7f59cc15d410 2024-05-22T21:22:49.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d4800640 1 -- 172.21.15.161:0/3818843534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15d410 msgr2=0x7f59cc15b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:49.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d4800640 1 --2- 172.21.15.161:0/3818843534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15d410 0x7f59cc15b6c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009e50 tx=0x7f59c801c970 comp rx=0 tx=0).stop 2024-05-22T21:22:49.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d4800640 1 -- 172.21.15.161:0/3818843534 shutdown_connections 2024-05-22T21:22:49.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d4800640 1 --2- 172.21.15.161:0/3818843534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15d410 0x7f59cc15b6c0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d4800640 1 --2- 172.21.15.161:0/3818843534 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59cc15e130 0x7f59cc15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d4800640 1 -- 172.21.15.161:0/3818843534 >> 172.21.15.161:0/3818843534 conn(0x7f59cc0b0bc0 msgr2=0x7f59cc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:49.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d4800640 1 -- 172.21.15.161:0/3818843534 shutdown_connections 2024-05-22T21:22:49.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d4800640 1 -- 172.21.15.161:0/3818843534 wait complete. 2024-05-22T21:22:49.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d4800640 1 Processor -- start 2024-05-22T21:22:49.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d4800640 1 -- start start 2024-05-22T21:22:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d4800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15d410 0x7f59cc0adef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d4800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc15dd50 0x7f59cc0b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d4800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59cc15e130 0x7f59cc06c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d4800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc0b2390 con 0x7f59cc15d410 2024-05-22T21:22:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d4800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc0b2670 con 0x7f59cc15dd50 2024-05-22T21:22:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d4800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc0b2950 con 0x7f59cc15e130 2024-05-22T21:22:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15d410 0x7f59cc0adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15d410 0x7f59cc0adef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49704/0 (socket says 172.21.15.161:49704) 2024-05-22T21:22:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d1e00640 1 -- 172.21.15.161:0/1259304803 learned_addr learned my addr 172.21.15.161:0/1259304803 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:49.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d1400640 1 --2- 172.21.15.161:0/1259304803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc15dd50 0x7f59cc0b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d2800640 1 --2- 172.21.15.161:0/1259304803 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59cc15e130 0x7f59cc06c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d1e00640 1 -- 172.21.15.161:0/1259304803 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59cc15e130 msgr2=0x7f59cc06c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d1e00640 1 --2- 172.21.15.161:0/1259304803 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59cc15e130 0x7f59cc06c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d1e00640 1 -- 172.21.15.161:0/1259304803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc15dd50 msgr2=0x7f59cc0b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d1e00640 1 --2- 172.21.15.161:0/1259304803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc15dd50 0x7f59cc0b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.192+0000 7f59d1e00640 1 -- 172.21.15.161:0/1259304803 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c8009b30 con 0x7f59cc15d410 2024-05-22T21:22:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.196+0000 7f59d1e00640 1 --2- 172.21.15.161:0/1259304803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15d410 0x7f59cc0adef0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f59c000c500 tx=0x7f59c000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.196+0000 7f59c6000640 1 -- 172.21.15.161:0/1259304803 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c0005220 con 0x7f59cc15d410 2024-05-22T21:22:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.196+0000 7f59c6000640 1 -- 172.21.15.161:0/1259304803 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c0005380 con 0x7f59cc15d410 2024-05-22T21:22:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.196+0000 7f59d4800640 1 -- 172.21.15.161:0/1259304803 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59cc13a360 con 0x7f59cc15d410 2024-05-22T21:22:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.196+0000 7f59c6000640 1 -- 172.21.15.161:0/1259304803 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c0007270 con 0x7f59cc15d410 2024-05-22T21:22:49.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.196+0000 7f59d4800640 1 -- 172.21.15.161:0/1259304803 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc14ee70 con 0x7f59cc15d410 2024-05-22T21:22:49.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.196+0000 7f59d4800640 1 -- 172.21.15.161:0/1259304803 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59cc0706b0 con 0x7f59cc15d410 2024-05-22T21:22:49.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.196+0000 7f59c6000640 1 -- 172.21.15.161:0/1259304803 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59c00073d0 con 0x7f59cc15d410 2024-05-22T21:22:49.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.196+0000 7f59c6000640 1 --2- 172.21.15.161:0/1259304803 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59b00666c0 0x7f59b0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.196+0000 7f59c6000640 1 -- 172.21.15.161:0/1259304803 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f59c0083920 con 0x7f59cc15d410 2024-05-22T21:22:49.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.196+0000 7f59c6000640 1 -- 172.21.15.161:0/1259304803 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7f59b006e2b0 con 0x7f59cc15d410 2024-05-22T21:22:49.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.196+0000 7f59d1400640 1 --2- 172.21.15.161:0/1259304803 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59b00666c0 0x7f59b0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.196+0000 7f59d1400640 1 --2- 172.21.15.161:0/1259304803 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59b00666c0 0x7f59b0068b80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f59cc14f550 tx=0x7f59bc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:49.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.200+0000 7f59c6000640 1 -- 172.21.15.161:0/1259304803 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f59c0050d10 con 0x7f59cc15d410 2024-05-22T21:22:49.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.388+0000 7f59d4800640 1 -- 172.21.15.161:0/1259304803 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59cc067220 con 0x7f59cc15d410 2024-05-22T21:22:49.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.388+0000 7f59c6000640 1 -- 172.21.15.161:0/1259304803 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f59c0054330 con 0x7f59cc15d410 2024-05-22T21:22:49.389 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:22:49.389 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":70,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7992,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/207877183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":207877183},{"type":"v1","addr":"172.21.15.161:6836","nonce":207877183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2152702106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2152702106},{"type":"v1","addr":"172.21.15.161:6837","nonce":2152702106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/749927648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":749927648},{"type":"v1","addr":"172.21.15.179:6834","nonce":749927648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:20:33.964441+0000","modified":"2024-05-22T21:20:36.988238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8004},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8004":{"gid":8004,"name":"b","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/93167194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":93167194},{"type":"v1","addr":"172.21.15.179:6835","nonce":93167194}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-05-22T21:22:49.389 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 70 2024-05-22T21:22:49.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.392+0000 7f59d4800640 1 -- 172.21.15.161:0/1259304803 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59b00666c0 msgr2=0x7f59b0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:49.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.392+0000 7f59d4800640 1 --2- 172.21.15.161:0/1259304803 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59b00666c0 0x7f59b0068b80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f59cc14f550 tx=0x7f59bc009190 comp rx=0 tx=0).stop 2024-05-22T21:22:49.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.392+0000 7f59d4800640 1 -- 172.21.15.161:0/1259304803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15d410 msgr2=0x7f59cc0adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:49.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.392+0000 7f59d4800640 1 --2- 172.21.15.161:0/1259304803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15d410 0x7f59cc0adef0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f59c000c500 tx=0x7f59c000c9c0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.392+0000 7f59d4800640 1 -- 172.21.15.161:0/1259304803 shutdown_connections 2024-05-22T21:22:49.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.392+0000 7f59d4800640 1 --2- 172.21.15.161:0/1259304803 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59b00666c0 0x7f59b0068b80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.392+0000 7f59d4800640 1 --2- 172.21.15.161:0/1259304803 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59cc15e130 0x7f59cc06c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.392+0000 7f59d4800640 1 --2- 172.21.15.161:0/1259304803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc15dd50 0x7f59cc0b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.392+0000 7f59d4800640 1 --2- 172.21.15.161:0/1259304803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59cc15d410 0x7f59cc0adef0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.392+0000 7f59d4800640 1 -- 172.21.15.161:0/1259304803 >> 172.21.15.161:0/1259304803 conn(0x7f59cc0b0bc0 msgr2=0x7f59cc0aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:49.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.392+0000 7f59d4800640 1 -- 172.21.15.161:0/1259304803 shutdown_connections 2024-05-22T21:22:49.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.392+0000 7f59d4800640 1 -- 172.21.15.161:0/1259304803 wait complete. 2024-05-22T21:22:49.409 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:22:49.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.544+0000 7f420be00640 1 Processor -- start 2024-05-22T21:22:49.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.544+0000 7f420be00640 1 -- start start 2024-05-22T21:22:49.547 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f420be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f420415dd50 0x7f420406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.547 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f420be00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f420415e130 0x7f420415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.547 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f420be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f420415d410 0x7f420415a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.547 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f420be00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4204071670 con 0x7f420415dd50 2024-05-22T21:22:49.547 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f420be00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f420415d8a0 con 0x7f420415e130 2024-05-22T21:22:49.547 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f420be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420406de40 con 0x7f420415d410 2024-05-22T21:22:49.547 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f4209400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f420415dd50 0x7f420406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.547 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f4209400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f420415dd50 0x7f420406d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49714/0 (socket says 172.21.15.161:49714) 2024-05-22T21:22:49.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f4209400640 1 -- 172.21.15.161:0/2487280793 learned_addr learned my addr 172.21.15.161:0/2487280793 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:49.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f4208a00640 1 --2- 172.21.15.161:0/2487280793 >> v2:172.21.15.179:3300/0 conn(0x7f420415e130 0x7f420415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f4209e00640 1 --2- 172.21.15.161:0/2487280793 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f420415d410 0x7f420415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f4209400640 1 -- 172.21.15.161:0/2487280793 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f420415d410 msgr2=0x7f420415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:49.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f4209400640 1 --2- 172.21.15.161:0/2487280793 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f420415d410 0x7f420415a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f4209400640 1 -- 172.21.15.161:0/2487280793 >> v2:172.21.15.179:3300/0 conn(0x7f420415e130 msgr2=0x7f420415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:49.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f4209400640 1 --2- 172.21.15.161:0/2487280793 >> v2:172.21.15.179:3300/0 conn(0x7f420415e130 0x7f420415ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.549 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f4209400640 1 -- 172.21.15.161:0/2487280793 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4204070a60 con 0x7f420415dd50 2024-05-22T21:22:49.550 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.548+0000 7f4209400640 1 --2- 172.21.15.161:0/2487280793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420415dd50 0x7f420406d900 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f420406c7c0 tx=0x7f41f801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b97a6349421559 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:49.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f4203e00640 1 -- 172.21.15.161:0/2487280793 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f80aa610 con 0x7f420415dd50 2024-05-22T21:22:49.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f4203e00640 1 -- 172.21.15.161:0/2487280793 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f80a6040 con 0x7f420415dd50 2024-05-22T21:22:49.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f4203e00640 1 -- 172.21.15.161:0/2487280793 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f80aa990 con 0x7f420415dd50 2024-05-22T21:22:49.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 -- 172.21.15.161:0/2487280793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420415dd50 msgr2=0x7f420406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:49.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 --2- 172.21.15.161:0/2487280793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420415dd50 0x7f420406d900 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f420406c7c0 tx=0x7f41f801c4f0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 -- 172.21.15.161:0/2487280793 shutdown_connections 2024-05-22T21:22:49.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 --2- 172.21.15.161:0/2487280793 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f420415d410 0x7f420415a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 --2- 172.21.15.161:0/2487280793 >> v2:172.21.15.179:3300/0 conn(0x7f420415e130 0x7f420415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 --2- 172.21.15.161:0/2487280793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420415dd50 0x7f420406d900 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 -- 172.21.15.161:0/2487280793 >> 172.21.15.161:0/2487280793 conn(0x7f42040b0bc0 msgr2=0x7f42040b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:49.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 -- 172.21.15.161:0/2487280793 shutdown_connections 2024-05-22T21:22:49.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 -- 172.21.15.161:0/2487280793 wait complete. 2024-05-22T21:22:49.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 Processor -- start 2024-05-22T21:22:49.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 -- start start 2024-05-22T21:22:49.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420415d410 0x7f4204152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f420415dd50 0x7f4204154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f420415e130 0x7f4204154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4204153390 con 0x7f420415d410 2024-05-22T21:22:49.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4204153670 con 0x7f420415e130 2024-05-22T21:22:49.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f420be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4204153950 con 0x7f420415dd50 2024-05-22T21:22:49.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f4209400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420415d410 0x7f4204152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f4209400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420415d410 0x7f4204152db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49722/0 (socket says 172.21.15.161:49722) 2024-05-22T21:22:49.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f4209400640 1 -- 172.21.15.161:0/919615910 learned_addr learned my addr 172.21.15.161:0/919615910 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:49.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f4208a00640 1 --2- 172.21.15.161:0/919615910 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f420415dd50 0x7f4204154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.552+0000 7f4209e00640 1 --2- 172.21.15.161:0/919615910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f420415e130 0x7f4204154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.556+0000 7f4209400640 1 -- 172.21.15.161:0/919615910 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f420415dd50 msgr2=0x7f4204154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:49.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.556+0000 7f4209400640 1 --2- 172.21.15.161:0/919615910 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f420415dd50 0x7f4204154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.556+0000 7f4209400640 1 -- 172.21.15.161:0/919615910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f420415e130 msgr2=0x7f4204154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:49.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.556+0000 7f4209400640 1 --2- 172.21.15.161:0/919615910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f420415e130 0x7f4204154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.556+0000 7f4209400640 1 -- 172.21.15.161:0/919615910 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f80094d0 con 0x7f420415d410 2024-05-22T21:22:49.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.556+0000 7f4209400640 1 --2- 172.21.15.161:0/919615910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420415d410 0x7f4204152db0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f41f801aba0 tx=0x7f41f801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:49.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.556+0000 7f4201600640 1 -- 172.21.15.161:0/919615910 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f80bd3f0 con 0x7f420415d410 2024-05-22T21:22:49.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.556+0000 7f4201600640 1 -- 172.21.15.161:0/919615910 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f80bd550 con 0x7f420415d410 2024-05-22T21:22:49.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.556+0000 7f4201600640 1 -- 172.21.15.161:0/919615910 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f80bd8e0 con 0x7f420415d410 2024-05-22T21:22:49.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.556+0000 7f420be00640 1 -- 172.21.15.161:0/919615910 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4204153c30 con 0x7f420415d410 2024-05-22T21:22:49.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.556+0000 7f420be00640 1 -- 172.21.15.161:0/919615910 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4204155090 con 0x7f420415d410 2024-05-22T21:22:49.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.556+0000 7f420be00640 1 -- 172.21.15.161:0/919615910 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4204067220 con 0x7f420415d410 2024-05-22T21:22:49.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.556+0000 7f4201600640 1 -- 172.21.15.161:0/919615910 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f41f80b33f0 con 0x7f420415d410 2024-05-22T21:22:49.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.560+0000 7f4201600640 1 --2- 172.21.15.161:0/919615910 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41e4066690 0x7f41e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.560+0000 7f4201600640 1 -- 172.21.15.161:0/919615910 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f41f811e9d0 con 0x7f420415d410 2024-05-22T21:22:49.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.560+0000 7f4208a00640 1 --2- 172.21.15.161:0/919615910 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41e4066690 0x7f41e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.560+0000 7f4201600640 1 -- 172.21.15.161:0/919615910 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7f41e406e280 con 0x7f420415d410 2024-05-22T21:22:49.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.560+0000 7f4208a00640 1 --2- 172.21.15.161:0/919615910 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41e4066690 0x7f41e4068b50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f42041521a0 tx=0x7f41f0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:49.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.564+0000 7f4201600640 1 -- 172.21.15.161:0/919615910 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f41f80c3060 con 0x7f420415d410 2024-05-22T21:22:49.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.724+0000 7f420be00640 1 -- 172.21.15.161:0/919615910 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42041540e0 con 0x7f420415d410 2024-05-22T21:22:49.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.728+0000 7f4201600640 1 -- 172.21.15.161:0/919615910 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16228 (secure 0 0 0) 0x7f41f80ebe20 con 0x7f420415d410 2024-05-22T21:22:49.727 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:22:49.727 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":162,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:22:05.280635+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:20:32.457234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-22T21:20:32.951241+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.16","mappings":[{"from":2,"to":5}]},{"pgid":"19.25","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:22:49.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.728+0000 7f420be00640 1 -- 172.21.15.161:0/919615910 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41e4066690 msgr2=0x7f41e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:49.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.728+0000 7f420be00640 1 --2- 172.21.15.161:0/919615910 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41e4066690 0x7f41e4068b50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f42041521a0 tx=0x7f41f0009190 comp rx=0 tx=0).stop 2024-05-22T21:22:49.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.728+0000 7f420be00640 1 -- 172.21.15.161:0/919615910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420415d410 msgr2=0x7f4204152db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:49.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.728+0000 7f420be00640 1 --2- 172.21.15.161:0/919615910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420415d410 0x7f4204152db0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f41f801aba0 tx=0x7f41f801ad30 comp rx=0 tx=0).stop 2024-05-22T21:22:49.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.728+0000 7f420be00640 1 -- 172.21.15.161:0/919615910 shutdown_connections 2024-05-22T21:22:49.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.728+0000 7f420be00640 1 --2- 172.21.15.161:0/919615910 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f41e4066690 0x7f41e4068b50 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.728+0000 7f420be00640 1 --2- 172.21.15.161:0/919615910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f420415e130 0x7f4204154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.728+0000 7f420be00640 1 --2- 172.21.15.161:0/919615910 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f420415dd50 0x7f4204154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.728+0000 7f420be00640 1 --2- 172.21.15.161:0/919615910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f420415d410 0x7f4204152db0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.728+0000 7f420be00640 1 -- 172.21.15.161:0/919615910 >> 172.21.15.161:0/919615910 conn(0x7f42040b0bc0 msgr2=0x7f42040b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:49.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.728+0000 7f420be00640 1 -- 172.21.15.161:0/919615910 shutdown_connections 2024-05-22T21:22:49.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.728+0000 7f420be00640 1 -- 172.21.15.161:0/919615910 wait complete. 2024-05-22T21:22:49.747 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:20:32.457234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '157', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:20:32.951241+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:22:49.747 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:22:49.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.884+0000 7f41b2a00640 1 Processor -- start 2024-05-22T21:22:49.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.884+0000 7f41b2a00640 1 -- start start 2024-05-22T21:22:49.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.884+0000 7f41b2a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41ac13cfc0 0x7f41ac1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.884+0000 7f41b2a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f41ac15af60 0x7f41ac13f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.884+0000 7f41b2a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41ac1400d0 con 0x7f41ac13fc80 2024-05-22T21:22:49.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.884+0000 7f41b2a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41ac13d3a0 con 0x7f41ac15af60 2024-05-22T21:22:49.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.884+0000 7f41b2a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac0cd920 con 0x7f41ac13cfc0 2024-05-22T21:22:49.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.888+0000 7f41b0a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f41ac13fc80 0x7f41ac15dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44050/0 (socket says 172.21.15.161:44050) 2024-05-22T21:22:49.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.888+0000 7f41b0a00640 1 -- 172.21.15.161:0/2333336597 learned_addr learned my addr 172.21.15.161:0/2333336597 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:49.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.888+0000 7f41abe00640 1 --2- 172.21.15.161:0/2333336597 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41ac13cfc0 0x7f41ac1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.888+0000 7f41ab400640 1 --2- 172.21.15.161:0/2333336597 >> v2:172.21.15.179:3300/0 conn(0x7f41ac15af60 0x7f41ac13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.888+0000 7f41aaa00640 1 -- 172.21.15.161:0/2333336597 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 617435099 0 0) 0x7f41a40014a0 con 0x7f41ac13fc80 2024-05-22T21:22:49.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.888+0000 7f41aaa00640 1 -- 172.21.15.161:0/2333336597 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4190003530 con 0x7f41ac13fc80 2024-05-22T21:22:49.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.888+0000 7f41abe00640 1 -- 172.21.15.161:0/2333336597 >> v2:172.21.15.179:3300/0 conn(0x7f41ac15af60 msgr2=0x7f41ac13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:49.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.888+0000 7f41abe00640 1 --2- 172.21.15.161:0/2333336597 >> v2:172.21.15.179:3300/0 conn(0x7f41ac15af60 0x7f41ac13f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.888+0000 7f41abe00640 1 -- 172.21.15.161:0/2333336597 >> v1:172.21.15.161:6789/0 conn(0x7f41ac13fc80 legacy=0x7f41ac15dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:49.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.888+0000 7f41abe00640 1 -- 172.21.15.161:0/2333336597 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac0cdc00 con 0x7f41ac13cfc0 2024-05-22T21:22:49.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.888+0000 7f41ab400640 1 --2- 172.21.15.161:0/2333336597 >> v2:172.21.15.179:3300/0 conn(0x7f41ac15af60 0x7f41ac13f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:22:49.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.888+0000 7f41abe00640 1 --2- 172.21.15.161:0/2333336597 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41ac13cfc0 0x7f41ac1402e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f419c0097c0 tx=0x7f419c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3d87f7d85d57c91 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:49.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41aaa00640 1 -- 172.21.15.161:0/2333336597 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c0ab600 con 0x7f41ac13cfc0 2024-05-22T21:22:49.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41aaa00640 1 -- 172.21.15.161:0/2333336597 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419c0a7050 con 0x7f41ac13cfc0 2024-05-22T21:22:49.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41aaa00640 1 -- 172.21.15.161:0/2333336597 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c0ab980 con 0x7f41ac13cfc0 2024-05-22T21:22:49.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b2a00640 1 -- 172.21.15.161:0/2333336597 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41ac13cfc0 msgr2=0x7f41ac1402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:49.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b2a00640 1 --2- 172.21.15.161:0/2333336597 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41ac13cfc0 0x7f41ac1402e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f419c0097c0 tx=0x7f419c01c410 comp rx=0 tx=0).stop 2024-05-22T21:22:49.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b2a00640 1 -- 172.21.15.161:0/2333336597 shutdown_connections 2024-05-22T21:22:49.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b2a00640 1 --2- 172.21.15.161:0/2333336597 >> v2:172.21.15.179:3300/0 conn(0x7f41ac15af60 0x7f41ac13f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b2a00640 1 --2- 172.21.15.161:0/2333336597 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41ac13cfc0 0x7f41ac1402e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b2a00640 1 -- 172.21.15.161:0/2333336597 >> 172.21.15.161:0/2333336597 conn(0x7f41ac06a2d0 msgr2=0x7f41ac06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:49.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b2a00640 1 -- 172.21.15.161:0/2333336597 shutdown_connections 2024-05-22T21:22:49.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b2a00640 1 -- 172.21.15.161:0/2333336597 wait complete. 2024-05-22T21:22:49.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b2a00640 1 Processor -- start 2024-05-22T21:22:49.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b2a00640 1 -- start start 2024-05-22T21:22:49.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ac13cfc0 0x7f41ac13a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b2a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41ac13fc80 0x7f41ac0b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b2a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41ac15af60 0x7f41ac0b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b2a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac139490 con 0x7f41ac13cfc0 2024-05-22T21:22:49.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b2a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac139770 con 0x7f41ac15af60 2024-05-22T21:22:49.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b2a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac139a50 con 0x7f41ac13fc80 2024-05-22T21:22:49.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41abe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ac13cfc0 0x7f41ac13a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41abe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ac13cfc0 0x7f41ac13a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49738/0 (socket says 172.21.15.161:49738) 2024-05-22T21:22:49.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41abe00640 1 -- 172.21.15.161:0/1833927107 learned_addr learned my addr 172.21.15.161:0/1833927107 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:49.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41b0a00640 1 --2- 172.21.15.161:0/1833927107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41ac15af60 0x7f41ac0b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41ab400640 1 --2- 172.21.15.161:0/1833927107 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41ac13fc80 0x7f41ac0b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41abe00640 1 -- 172.21.15.161:0/1833927107 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41ac13fc80 msgr2=0x7f41ac0b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:49.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41abe00640 1 --2- 172.21.15.161:0/1833927107 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41ac13fc80 0x7f41ac0b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41abe00640 1 -- 172.21.15.161:0/1833927107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41ac15af60 msgr2=0x7f41ac0b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:49.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41abe00640 1 --2- 172.21.15.161:0/1833927107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41ac15af60 0x7f41ac0b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:49.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.892+0000 7f41abe00640 1 -- 172.21.15.161:0/1833927107 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4198009420 con 0x7f41ac13cfc0 2024-05-22T21:22:49.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.896+0000 7f41abe00640 1 --2- 172.21.15.161:0/1833927107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ac13cfc0 0x7f41ac13a530 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f419c0ae040 tx=0x7f419c01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:49.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.896+0000 7f41a3e00640 1 -- 172.21.15.161:0/1833927107 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c0b7450 con 0x7f41ac13cfc0 2024-05-22T21:22:49.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.896+0000 7f41a3e00640 1 -- 172.21.15.161:0/1833927107 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419c0b75d0 con 0x7f41ac13cfc0 2024-05-22T21:22:49.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.896+0000 7f41b2a00640 1 -- 172.21.15.161:0/1833927107 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419c009420 con 0x7f41ac13cfc0 2024-05-22T21:22:49.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.896+0000 7f41a3e00640 1 -- 172.21.15.161:0/1833927107 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c0b7980 con 0x7f41ac13cfc0 2024-05-22T21:22:49.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.896+0000 7f41b2a00640 1 -- 172.21.15.161:0/1833927107 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ac13a010 con 0x7f41ac13cfc0 2024-05-22T21:22:49.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.896+0000 7f41b2a00640 1 -- 172.21.15.161:0/1833927107 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41ac067220 con 0x7f41ac13cfc0 2024-05-22T21:22:49.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.896+0000 7f41a3e00640 1 -- 172.21.15.161:0/1833927107 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f419c0b7ae0 con 0x7f41ac13cfc0 2024-05-22T21:22:49.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.896+0000 7f41a3e00640 1 --2- 172.21.15.161:0/1833927107 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f418c066690 0x7f418c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:49.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.896+0000 7f41a3e00640 1 -- 172.21.15.161:0/1833927107 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f419c11e9a0 con 0x7f41ac13cfc0 2024-05-22T21:22:49.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.896+0000 7f41ab400640 1 --2- 172.21.15.161:0/1833927107 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f418c066690 0x7f418c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:49.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.896+0000 7f41a3e00640 1 -- 172.21.15.161:0/1833927107 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7f418c06e280 con 0x7f41ac13cfc0 2024-05-22T21:22:49.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.900+0000 7f41ab400640 1 --2- 172.21.15.161:0/1833927107 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f418c066690 0x7f418c068b50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f41ac1f71d0 tx=0x7f4198008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:49.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:49.900+0000 7f41a3e00640 1 -- 172.21.15.161:0/1833927107 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f419c0ebc30 con 0x7f41ac13cfc0 2024-05-22T21:22:50.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.112+0000 7f41b2a00640 1 -- 172.21.15.161:0/1833927107 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41ac071670 con 0x7f41ac13cfc0 2024-05-22T21:22:50.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.112+0000 7f41a3e00640 1 -- 172.21.15.161:0/1833927107 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v162) v1 ==== 93+0+10 (secure 0 0 0) 0x7f419c0b0020 con 0x7f41ac13cfc0 2024-05-22T21:22:50.114 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:22:50.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.116+0000 7f41b2a00640 1 -- 172.21.15.161:0/1833927107 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f418c066690 msgr2=0x7f418c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.116+0000 7f41b2a00640 1 --2- 172.21.15.161:0/1833927107 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f418c066690 0x7f418c068b50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f41ac1f71d0 tx=0x7f4198008040 comp rx=0 tx=0).stop 2024-05-22T21:22:50.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.116+0000 7f41b2a00640 1 -- 172.21.15.161:0/1833927107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ac13cfc0 msgr2=0x7f41ac13a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.116+0000 7f41b2a00640 1 --2- 172.21.15.161:0/1833927107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ac13cfc0 0x7f41ac13a530 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f419c0ae040 tx=0x7f419c01ac50 comp rx=0 tx=0).stop 2024-05-22T21:22:50.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.116+0000 7f41b2a00640 1 -- 172.21.15.161:0/1833927107 shutdown_connections 2024-05-22T21:22:50.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.116+0000 7f41b2a00640 1 --2- 172.21.15.161:0/1833927107 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f418c066690 0x7f418c068b50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.116+0000 7f41b2a00640 1 --2- 172.21.15.161:0/1833927107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41ac15af60 0x7f41ac0b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.116+0000 7f41b2a00640 1 --2- 172.21.15.161:0/1833927107 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41ac13fc80 0x7f41ac0b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.116+0000 7f41b2a00640 1 --2- 172.21.15.161:0/1833927107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ac13cfc0 0x7f41ac13a530 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.116+0000 7f41b2a00640 1 -- 172.21.15.161:0/1833927107 >> 172.21.15.161:0/1833927107 conn(0x7f41ac06a2d0 msgr2=0x7f41ac06a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:50.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.116+0000 7f41b2a00640 1 -- 172.21.15.161:0/1833927107 shutdown_connections 2024-05-22T21:22:50.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.116+0000 7f41b2a00640 1 -- 172.21.15.161:0/1833927107 wait complete. 2024-05-22T21:22:50.135 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:22:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.272+0000 7f5a12800640 1 Processor -- start 2024-05-22T21:22:50.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.272+0000 7f5a12800640 1 -- start start 2024-05-22T21:22:50.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a12800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c13cfc0 0x7f5a0c1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a12800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5a0c13fe50 0x7f5a0c06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a12800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0c13d3a0 con 0x7f5a0c15af60 2024-05-22T21:22:50.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a12800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0c0cd920 con 0x7f5a0c13fe50 2024-05-22T21:22:50.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a12800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c0cdc00 con 0x7f5a0c13cfc0 2024-05-22T21:22:50.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a02a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f5a0c15af60 0x7f5a0c13f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44056/0 (socket says 172.21.15.161:44056) 2024-05-22T21:22:50.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a02a00640 1 -- 172.21.15.161:0/2774641472 learned_addr learned my addr 172.21.15.161:0/2774641472 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:50.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0aa00640 1 -- 172.21.15.161:0/2774641472 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412616452 0 0) 0x7f5a0c13d3a0 con 0x7f5a0c15af60 2024-05-22T21:22:50.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0be00640 1 --2- 172.21.15.161:0/2774641472 >> v2:172.21.15.179:3300/0 conn(0x7f5a0c13fe50 0x7f5a0c06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0b400640 1 --2- 172.21.15.161:0/2774641472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c13cfc0 0x7f5a0c1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0aa00640 1 -- 172.21.15.161:0/2774641472 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59f0003530 con 0x7f5a0c15af60 2024-05-22T21:22:50.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0aa00640 1 -- 172.21.15.161:0/2774641472 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 809744496 0 0) 0x7f59f0003530 con 0x7f5a0c15af60 2024-05-22T21:22:50.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0aa00640 1 -- 172.21.15.161:0/2774641472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c13cfc0 msgr2=0x7f5a0c1402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:50.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0aa00640 1 --2- 172.21.15.161:0/2774641472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c13cfc0 0x7f5a0c1402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0aa00640 1 -- 172.21.15.161:0/2774641472 >> v2:172.21.15.179:3300/0 conn(0x7f5a0c13fe50 msgr2=0x7f5a0c06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:50.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0aa00640 1 --2- 172.21.15.161:0/2774641472 >> v2:172.21.15.179:3300/0 conn(0x7f5a0c13fe50 0x7f5a0c06fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0aa00640 1 -- 172.21.15.161:0/2774641472 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c0cdee0 con 0x7f5a0c15af60 2024-05-22T21:22:50.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0aa00640 1 -- 172.21.15.161:0/2774641472 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f59f4001680 con 0x7f5a0c15af60 2024-05-22T21:22:50.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0aa00640 1 -- 172.21.15.161:0/2774641472 >> v1:172.21.15.161:6789/0 conn(0x7f5a0c15af60 legacy=0x7f5a0c13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0aa00640 1 --2- 172.21.15.161:0/2774641472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59f0003980 0x7f59f0003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0aa00640 1 --2- 172.21.15.161:0/2774641472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59f0004330 0x7f59f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0aa00640 1 --2- 172.21.15.161:0/2774641472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c13cfc0 0x7f59f0004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0aa00640 1 -- 172.21.15.161:0/2774641472 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c0cd920 con 0x7f59f0004330 2024-05-22T21:22:50.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0aa00640 1 -- 172.21.15.161:0/2774641472 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c0cdc00 con 0x7f59f0003980 2024-05-22T21:22:50.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0aa00640 1 -- 172.21.15.161:0/2774641472 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f000b990 con 0x7f5a0c13cfc0 2024-05-22T21:22:50.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a02a00640 1 --2- 172.21.15.161:0/2774641472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59f0004330 0x7f59f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0b400640 1 --2- 172.21.15.161:0/2774641472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59f0003980 0x7f59f0003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a0be00640 1 --2- 172.21.15.161:0/2774641472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c13cfc0 0x7f59f0004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a02a00640 1 -- 172.21.15.161:0/2774641472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c13cfc0 msgr2=0x7f59f0004f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:50.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a02a00640 1 --2- 172.21.15.161:0/2774641472 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c13cfc0 0x7f59f0004f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a02a00640 1 -- 172.21.15.161:0/2774641472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59f0003980 msgr2=0x7f59f0003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:50.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a02a00640 1 --2- 172.21.15.161:0/2774641472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59f0003980 0x7f59f0003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.276+0000 7f5a02a00640 1 -- 172.21.15.161:0/2774641472 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f0003530 con 0x7f59f0004330 2024-05-22T21:22:50.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.280+0000 7f5a02a00640 1 --2- 172.21.15.161:0/2774641472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59f0004330 0x7f59f0004810 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f59f4005060 tx=0x7f59f401d270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bdb54a749e8c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:50.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.280+0000 7f5a0aa00640 1 -- 172.21.15.161:0/2774641472 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f40a5a90 con 0x7f59f0004330 2024-05-22T21:22:50.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.280+0000 7f5a0aa00640 1 -- 172.21.15.161:0/2774641472 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f40a5c10 con 0x7f59f0004330 2024-05-22T21:22:50.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.280+0000 7f5a0aa00640 1 -- 172.21.15.161:0/2774641472 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f40058e0 con 0x7f59f0004330 2024-05-22T21:22:50.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.280+0000 7f5a12800640 1 -- 172.21.15.161:0/2774641472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59f0004330 msgr2=0x7f59f0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.280+0000 7f5a12800640 1 --2- 172.21.15.161:0/2774641472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59f0004330 0x7f59f0004810 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f59f4005060 tx=0x7f59f401d270 comp rx=0 tx=0).stop 2024-05-22T21:22:50.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.280+0000 7f5a0be00640 1 -- 172.21.15.161:0/2774641472 reap_dead start 2024-05-22T21:22:50.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.280+0000 7f5a12800640 1 -- 172.21.15.161:0/2774641472 shutdown_connections 2024-05-22T21:22:50.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.280+0000 7f5a12800640 1 -- 172.21.15.161:0/2774641472 >> 172.21.15.161:0/2774641472 conn(0x7f5a0c06a2d0 msgr2=0x7f5a0c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:50.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.280+0000 7f5a12800640 1 -- 172.21.15.161:0/2774641472 shutdown_connections 2024-05-22T21:22:50.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.280+0000 7f5a12800640 1 -- 172.21.15.161:0/2774641472 wait complete. 2024-05-22T21:22:50.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a12800640 1 Processor -- start 2024-05-22T21:22:50.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a12800640 1 -- start start 2024-05-22T21:22:50.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a12800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59f0004330 0x7f5a0c1f9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a12800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c142280 0x7f5a0c1633f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a12800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a0c0b2180 0x7f5a0c0b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a12800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c1f6140 con 0x7f59f0004330 2024-05-22T21:22:50.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a12800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c1fa410 con 0x7f5a0c0b2180 2024-05-22T21:22:50.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a12800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c1f9b70 con 0x7f5a0c142280 2024-05-22T21:22:50.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a0b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59f0004330 0x7f5a0c1f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a0b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59f0004330 0x7f5a0c1f9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49748/0 (socket says 172.21.15.161:49748) 2024-05-22T21:22:50.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a0b400640 1 -- 172.21.15.161:0/1833359496 learned_addr learned my addr 172.21.15.161:0/1833359496 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:50.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a0be00640 1 --2- 172.21.15.161:0/1833359496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a0c0b2180 0x7f5a0c0b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a02a00640 1 --2- 172.21.15.161:0/1833359496 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c142280 0x7f5a0c1633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a0b400640 1 -- 172.21.15.161:0/1833359496 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c142280 msgr2=0x7f5a0c1633f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a0b400640 1 --2- 172.21.15.161:0/1833359496 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c142280 0x7f5a0c1633f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a0b400640 1 -- 172.21.15.161:0/1833359496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a0c0b2180 msgr2=0x7f5a0c0b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a0b400640 1 --2- 172.21.15.161:0/1833359496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a0c0b2180 0x7f5a0c0b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a0b400640 1 -- 172.21.15.161:0/1833359496 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c1f9e50 con 0x7f59f0004330 2024-05-22T21:22:50.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a0b400640 1 --2- 172.21.15.161:0/1833359496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59f0004330 0x7f5a0c1f9590 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f59fc011960 tx=0x7f59fc011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:50.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a03e00640 1 -- 172.21.15.161:0/1833359496 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc00faf0 con 0x7f59f0004330 2024-05-22T21:22:50.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a03e00640 1 -- 172.21.15.161:0/1833359496 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc00fc50 con 0x7f59f0004330 2024-05-22T21:22:50.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a12800640 1 -- 172.21.15.161:0/1833359496 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0c1fa130 con 0x7f59f0004330 2024-05-22T21:22:50.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a03e00640 1 -- 172.21.15.161:0/1833359496 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc014440 con 0x7f59f0004330 2024-05-22T21:22:50.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a12800640 1 -- 172.21.15.161:0/1833359496 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0c1426e0 con 0x7f59f0004330 2024-05-22T21:22:50.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.284+0000 7f5a12800640 1 -- 172.21.15.161:0/1833359496 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a0c067220 con 0x7f59f0004330 2024-05-22T21:22:50.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.288+0000 7f5a03e00640 1 -- 172.21.15.161:0/1833359496 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59fc0145a0 con 0x7f59f0004330 2024-05-22T21:22:50.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.288+0000 7f5a03e00640 1 --2- 172.21.15.161:0/1833359496 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59e8066690 0x7f59e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.288+0000 7f5a03e00640 1 -- 172.21.15.161:0/1833359496 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f59fc084a60 con 0x7f59f0004330 2024-05-22T21:22:50.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.288+0000 7f5a02a00640 1 --2- 172.21.15.161:0/1833359496 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59e8066690 0x7f59e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.288+0000 7f5a03e00640 1 -- 172.21.15.161:0/1833359496 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7f59e806e280 con 0x7f59f0004330 2024-05-22T21:22:50.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.288+0000 7f5a02a00640 1 --2- 172.21.15.161:0/1833359496 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59e8066690 0x7f59e8068b50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f59f40aa000 tx=0x7f59f40a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:50.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.292+0000 7f5a03e00640 1 -- 172.21.15.161:0/1833359496 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f59fc051cf0 con 0x7f59f0004330 2024-05-22T21:22:50.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.444+0000 7f5a12800640 1 -- 172.21.15.161:0/1833359496 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a0c071670 con 0x7f59f0004330 2024-05-22T21:22:50.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.448+0000 7f5a03e00640 1 -- 172.21.15.161:0/1833359496 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v162) v1 ==== 104+0+11 (secure 0 0 0) 0x7f59fc055310 con 0x7f59f0004330 2024-05-22T21:22:50.447 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:22:50.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.448+0000 7f5a12800640 1 -- 172.21.15.161:0/1833359496 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59e8066690 msgr2=0x7f59e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.448+0000 7f5a12800640 1 --2- 172.21.15.161:0/1833359496 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59e8066690 0x7f59e8068b50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f59f40aa000 tx=0x7f59f40a7070 comp rx=0 tx=0).stop 2024-05-22T21:22:50.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.448+0000 7f5a12800640 1 -- 172.21.15.161:0/1833359496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59f0004330 msgr2=0x7f5a0c1f9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.448+0000 7f5a12800640 1 --2- 172.21.15.161:0/1833359496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59f0004330 0x7f5a0c1f9590 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f59fc011960 tx=0x7f59fc011e20 comp rx=0 tx=0).stop 2024-05-22T21:22:50.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.448+0000 7f5a12800640 1 -- 172.21.15.161:0/1833359496 shutdown_connections 2024-05-22T21:22:50.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.448+0000 7f5a12800640 1 --2- 172.21.15.161:0/1833359496 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f59e8066690 0x7f59e8068b50 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.448+0000 7f5a12800640 1 --2- 172.21.15.161:0/1833359496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a0c0b2180 0x7f5a0c0b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.448+0000 7f5a12800640 1 --2- 172.21.15.161:0/1833359496 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c142280 0x7f5a0c1633f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.448+0000 7f5a12800640 1 --2- 172.21.15.161:0/1833359496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59f0004330 0x7f5a0c1f9590 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.448+0000 7f5a12800640 1 -- 172.21.15.161:0/1833359496 >> 172.21.15.161:0/1833359496 conn(0x7f5a0c06a2d0 msgr2=0x7f5a0c142b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:50.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.448+0000 7f5a12800640 1 -- 172.21.15.161:0/1833359496 shutdown_connections 2024-05-22T21:22:50.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.448+0000 7f5a12800640 1 -- 172.21.15.161:0/1833359496 wait complete. 2024-05-22T21:22:50.467 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:22:50.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.604+0000 7ff3a0c00640 1 Processor -- start 2024-05-22T21:22:50.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.604+0000 7ff3a0c00640 1 -- start start 2024-05-22T21:22:50.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff3a0c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff39c15b050 0x7ff39c0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff3a0c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff39c0b0d80 0x7ff39c0cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff3a0c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff39c0d0ae0 0x7ff39c0d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff3a0c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff39c0cdd20 con 0x7ff39c0d0ae0 2024-05-22T21:22:50.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff3a0c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff39c0ce000 con 0x7ff39c15b050 2024-05-22T21:22:50.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff3a0c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c0ce2e0 con 0x7ff39c0b0d80 2024-05-22T21:22:50.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff39be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff39c0d0ae0 0x7ff39c0d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff39be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff39c0d0ae0 0x7ff39c0d0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49764/0 (socket says 172.21.15.161:49764) 2024-05-22T21:22:50.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff39be00640 1 -- 172.21.15.161:0/2443686944 learned_addr learned my addr 172.21.15.161:0/2443686944 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:50.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff39b400640 1 --2- 172.21.15.161:0/2443686944 >> v2:172.21.15.179:3300/0 conn(0x7ff39c15b050 0x7ff39c0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff392a00640 1 --2- 172.21.15.161:0/2443686944 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff39c0b0d80 0x7ff39c0cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff39be00640 1 -- 172.21.15.161:0/2443686944 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff39c0b0d80 msgr2=0x7ff39c0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:50.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff39be00640 1 --2- 172.21.15.161:0/2443686944 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff39c0b0d80 0x7ff39c0cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff39be00640 1 -- 172.21.15.161:0/2443686944 >> v2:172.21.15.179:3300/0 conn(0x7ff39c15b050 msgr2=0x7ff39c0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:50.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff39be00640 1 --2- 172.21.15.161:0/2443686944 >> v2:172.21.15.179:3300/0 conn(0x7ff39c15b050 0x7ff39c0cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff39be00640 1 -- 172.21.15.161:0/2443686944 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39c0ce5c0 con 0x7ff39c0d0ae0 2024-05-22T21:22:50.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.608+0000 7ff39be00640 1 --2- 172.21.15.161:0/2443686944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39c0d0ae0 0x7ff39c0d0f70 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0b2910 tx=0x7ff39401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2726f2d63f865f51 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:50.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff39aa00640 1 -- 172.21.15.161:0/2443686944 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3940a5740 con 0x7ff39c0d0ae0 2024-05-22T21:22:50.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff39aa00640 1 -- 172.21.15.161:0/2443686944 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3940a58a0 con 0x7ff39c0d0ae0 2024-05-22T21:22:50.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff39aa00640 1 -- 172.21.15.161:0/2443686944 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3940a5c30 con 0x7ff39c0d0ae0 2024-05-22T21:22:50.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 -- 172.21.15.161:0/2443686944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39c0d0ae0 msgr2=0x7ff39c0d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 --2- 172.21.15.161:0/2443686944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39c0d0ae0 0x7ff39c0d0f70 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0b2910 tx=0x7ff39401c930 comp rx=0 tx=0).stop 2024-05-22T21:22:50.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 -- 172.21.15.161:0/2443686944 shutdown_connections 2024-05-22T21:22:50.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 --2- 172.21.15.161:0/2443686944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39c0d0ae0 0x7ff39c0d0f70 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 --2- 172.21.15.161:0/2443686944 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff39c0b0d80 0x7ff39c0cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 --2- 172.21.15.161:0/2443686944 >> v2:172.21.15.179:3300/0 conn(0x7ff39c15b050 0x7ff39c0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 -- 172.21.15.161:0/2443686944 >> 172.21.15.161:0/2443686944 conn(0x7ff39c099a40 msgr2=0x7ff39c099e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:50.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 -- 172.21.15.161:0/2443686944 shutdown_connections 2024-05-22T21:22:50.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 -- 172.21.15.161:0/2443686944 wait complete. 2024-05-22T21:22:50.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 Processor -- start 2024-05-22T21:22:50.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 -- start start 2024-05-22T21:22:50.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39c0b0d80 0x7ff39c1515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff39c0d0ae0 0x7ff39c1573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff39c15b050 0x7ff39c155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c150350 con 0x7ff39c0b0d80 2024-05-22T21:22:50.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c153920 con 0x7ff39c15b050 2024-05-22T21:22:50.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.612+0000 7ff3a0c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c153c00 con 0x7ff39c0d0ae0 2024-05-22T21:22:50.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff39b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39c0b0d80 0x7ff39c1515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff39b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39c0b0d80 0x7ff39c1515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49766/0 (socket says 172.21.15.161:49766) 2024-05-22T21:22:50.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff39b400640 1 -- 172.21.15.161:0/3801488071 learned_addr learned my addr 172.21.15.161:0/3801488071 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:50.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff39be00640 1 --2- 172.21.15.161:0/3801488071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff39c15b050 0x7ff39c155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff392a00640 1 --2- 172.21.15.161:0/3801488071 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff39c0d0ae0 0x7ff39c1573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff39b400640 1 -- 172.21.15.161:0/3801488071 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff39c0d0ae0 msgr2=0x7ff39c1573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff39b400640 1 --2- 172.21.15.161:0/3801488071 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff39c0d0ae0 0x7ff39c1573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff39b400640 1 -- 172.21.15.161:0/3801488071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff39c15b050 msgr2=0x7ff39c155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff39b400640 1 --2- 172.21.15.161:0/3801488071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff39c15b050 0x7ff39c155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff39b400640 1 -- 172.21.15.161:0/3801488071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff394009b30 con 0x7ff39c0b0d80 2024-05-22T21:22:50.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff39b400640 1 --2- 172.21.15.161:0/3801488071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39c0b0d80 0x7ff39c1515d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00e870 tx=0x7ff38c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:50.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff393e00640 1 -- 172.21.15.161:0/3801488071 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c009510 con 0x7ff39c0b0d80 2024-05-22T21:22:50.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff393e00640 1 -- 172.21.15.161:0/3801488071 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38c00eec0 con 0x7ff39c0b0d80 2024-05-22T21:22:50.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff393e00640 1 -- 172.21.15.161:0/3801488071 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00cb40 con 0x7ff39c0b0d80 2024-05-22T21:22:50.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff3a0c00640 1 -- 172.21.15.161:0/3801488071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39c150af0 con 0x7ff39c0b0d80 2024-05-22T21:22:50.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff3a0c00640 1 -- 172.21.15.161:0/3801488071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39c1510b0 con 0x7ff39c0b0d80 2024-05-22T21:22:50.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff3a0c00640 1 -- 172.21.15.161:0/3801488071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff39c067070 con 0x7ff39c0b0d80 2024-05-22T21:22:50.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff393e00640 1 -- 172.21.15.161:0/3801488071 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff38c00cca0 con 0x7ff39c0b0d80 2024-05-22T21:22:50.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff393e00640 1 --2- 172.21.15.161:0/3801488071 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff378066690 0x7ff378068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.616+0000 7ff393e00640 1 -- 172.21.15.161:0/3801488071 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff38c0842c0 con 0x7ff39c0b0d80 2024-05-22T21:22:50.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.620+0000 7ff392a00640 1 --2- 172.21.15.161:0/3801488071 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff378066690 0x7ff378068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.620+0000 7ff393e00640 1 -- 172.21.15.161:0/3801488071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7ff37806e280 con 0x7ff39c0b0d80 2024-05-22T21:22:50.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.620+0000 7ff392a00640 1 --2- 172.21.15.161:0/3801488071 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff378066690 0x7ff378068b50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff39c1563c0 tx=0x7ff384009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:50.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.620+0000 7ff393e00640 1 -- 172.21.15.161:0/3801488071 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff38c051600 con 0x7ff39c0b0d80 2024-05-22T21:22:50.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.776+0000 7ff3a0c00640 1 -- 172.21.15.161:0/3801488071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff39c0b06d0 con 0x7ff39c0b0d80 2024-05-22T21:22:50.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.776+0000 7ff393e00640 1 -- 172.21.15.161:0/3801488071 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v162) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff38c0210a0 con 0x7ff39c0b0d80 2024-05-22T21:22:50.777 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:22:50.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.780+0000 7ff3a0c00640 1 -- 172.21.15.161:0/3801488071 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff378066690 msgr2=0x7ff378068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.780+0000 7ff3a0c00640 1 --2- 172.21.15.161:0/3801488071 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff378066690 0x7ff378068b50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff39c1563c0 tx=0x7ff384009190 comp rx=0 tx=0).stop 2024-05-22T21:22:50.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.780+0000 7ff3a0c00640 1 -- 172.21.15.161:0/3801488071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39c0b0d80 msgr2=0x7ff39c1515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.780+0000 7ff3a0c00640 1 --2- 172.21.15.161:0/3801488071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39c0b0d80 0x7ff39c1515d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00e870 tx=0x7ff38c00ec70 comp rx=0 tx=0).stop 2024-05-22T21:22:50.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.780+0000 7ff3a0c00640 1 -- 172.21.15.161:0/3801488071 shutdown_connections 2024-05-22T21:22:50.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.780+0000 7ff3a0c00640 1 --2- 172.21.15.161:0/3801488071 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff378066690 0x7ff378068b50 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.780+0000 7ff3a0c00640 1 --2- 172.21.15.161:0/3801488071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff39c15b050 0x7ff39c155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.780+0000 7ff3a0c00640 1 --2- 172.21.15.161:0/3801488071 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff39c0d0ae0 0x7ff39c1573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.780+0000 7ff3a0c00640 1 --2- 172.21.15.161:0/3801488071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39c0b0d80 0x7ff39c1515d0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.780+0000 7ff3a0c00640 1 -- 172.21.15.161:0/3801488071 >> 172.21.15.161:0/3801488071 conn(0x7ff39c099a40 msgr2=0x7ff39c0adca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:50.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.780+0000 7ff3a0c00640 1 -- 172.21.15.161:0/3801488071 shutdown_connections 2024-05-22T21:22:50.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.780+0000 7ff3a0c00640 1 -- 172.21.15.161:0/3801488071 wait complete. 2024-05-22T21:22:50.798 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:22:50.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.932+0000 7f3bb7a00640 1 Processor -- start 2024-05-22T21:22:50.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.932+0000 7f3bb7a00640 1 -- start start 2024-05-22T21:22:50.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bb7a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3bb01437e0 0x7f3bb0143bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bb7a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3bb00ce5a0 0x7f3bb00b3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bb7a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb0070db0 con 0x7f3bb006d470 2024-05-22T21:22:50.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bb7a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb0071000 con 0x7f3bb01437e0 2024-05-22T21:22:50.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bb7a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb00712e0 con 0x7f3bb00ce5a0 2024-05-22T21:22:50.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3ba7e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3bb006d470 0x7f3bb00cde90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44060/0 (socket says 172.21.15.161:44060) 2024-05-22T21:22:50.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3ba7e00640 1 -- 172.21.15.161:0/3967945655 learned_addr learned my addr 172.21.15.161:0/3967945655 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:50.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bb5000640 1 --2- 172.21.15.161:0/3967945655 >> v2:172.21.15.179:3300/0 conn(0x7f3bb01437e0 0x7f3bb0143bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bb5a00640 1 --2- 172.21.15.161:0/3967945655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3bb00ce5a0 0x7f3bb00b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bafe00640 1 -- 172.21.15.161:0/3967945655 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475392414 0 0) 0x7f3bb0070db0 con 0x7f3bb006d470 2024-05-22T21:22:50.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bafe00640 1 -- 172.21.15.161:0/3967945655 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b94003530 con 0x7f3bb006d470 2024-05-22T21:22:50.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bafe00640 1 -- 172.21.15.161:0/3967945655 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 446281079 0 0) 0x7f3b94003530 con 0x7f3bb006d470 2024-05-22T21:22:50.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bafe00640 1 -- 172.21.15.161:0/3967945655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3bb00ce5a0 msgr2=0x7f3bb00b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:50.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bafe00640 1 --2- 172.21.15.161:0/3967945655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3bb00ce5a0 0x7f3bb00b3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bafe00640 1 -- 172.21.15.161:0/3967945655 >> v2:172.21.15.179:3300/0 conn(0x7f3bb01437e0 msgr2=0x7f3bb0143bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:50.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bafe00640 1 --2- 172.21.15.161:0/3967945655 >> v2:172.21.15.179:3300/0 conn(0x7f3bb01437e0 0x7f3bb0143bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bafe00640 1 -- 172.21.15.161:0/3967945655 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb00715c0 con 0x7f3bb006d470 2024-05-22T21:22:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bafe00640 1 -- 172.21.15.161:0/3967945655 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f3b98001680 con 0x7f3bb006d470 2024-05-22T21:22:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bafe00640 1 -- 172.21.15.161:0/3967945655 >> v1:172.21.15.161:6789/0 conn(0x7f3bb006d470 legacy=0x7f3bb00cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bafe00640 1 --2- 172.21.15.161:0/3967945655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b94003980 0x7f3b94003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bafe00640 1 --2- 172.21.15.161:0/3967945655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b940047d0 0x7f3b94004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bafe00640 1 --2- 172.21.15.161:0/3967945655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3bb00ce5a0 0x7f3b94005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bafe00640 1 -- 172.21.15.161:0/3967945655 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb0071000 con 0x7f3b94003980 2024-05-22T21:22:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bafe00640 1 -- 172.21.15.161:0/3967945655 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb00712e0 con 0x7f3b940047d0 2024-05-22T21:22:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bafe00640 1 -- 172.21.15.161:0/3967945655 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9400b990 con 0x7f3bb00ce5a0 2024-05-22T21:22:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bb5000640 1 --2- 172.21.15.161:0/3967945655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b94003980 0x7f3b94003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3ba7e00640 1 --2- 172.21.15.161:0/3967945655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b940047d0 0x7f3b94004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bb5a00640 1 --2- 172.21.15.161:0/3967945655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3bb00ce5a0 0x7f3b94005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bb5000640 1 -- 172.21.15.161:0/3967945655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3bb00ce5a0 msgr2=0x7f3b94005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:50.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bb5000640 1 --2- 172.21.15.161:0/3967945655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3bb00ce5a0 0x7f3b94005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bb5000640 1 -- 172.21.15.161:0/3967945655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b940047d0 msgr2=0x7f3b94004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:50.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bb5000640 1 --2- 172.21.15.161:0/3967945655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b940047d0 0x7f3b94004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.936+0000 7f3bb5000640 1 -- 172.21.15.161:0/3967945655 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b94003530 con 0x7f3b94003980 2024-05-22T21:22:50.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.940+0000 7f3bb5000640 1 --2- 172.21.15.161:0/3967945655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b94003980 0x7f3b94003da0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009ae0 tx=0x7f3ba001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0607593b825fb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:50.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.940+0000 7f3bafe00640 1 -- 172.21.15.161:0/3967945655 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00b0400 con 0x7f3b94003980 2024-05-22T21:22:50.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.940+0000 7f3bafe00640 1 -- 172.21.15.161:0/3967945655 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba00b0560 con 0x7f3b94003980 2024-05-22T21:22:50.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.940+0000 7f3bafe00640 1 -- 172.21.15.161:0/3967945655 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00b0910 con 0x7f3b94003980 2024-05-22T21:22:50.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.940+0000 7f3bb7a00640 1 -- 172.21.15.161:0/3967945655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b94003980 msgr2=0x7f3b94003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.940+0000 7f3bb7a00640 1 --2- 172.21.15.161:0/3967945655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b94003980 0x7f3b94003da0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009ae0 tx=0x7f3ba001cab0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.940+0000 7f3bb5a00640 1 -- 172.21.15.161:0/3967945655 reap_dead start 2024-05-22T21:22:50.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.940+0000 7f3bb7a00640 1 -- 172.21.15.161:0/3967945655 shutdown_connections 2024-05-22T21:22:50.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.940+0000 7f3bb7a00640 1 -- 172.21.15.161:0/3967945655 >> 172.21.15.161:0/3967945655 conn(0x7f3bb00b5540 msgr2=0x7f3bb00b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:50.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.940+0000 7f3bb7a00640 1 -- 172.21.15.161:0/3967945655 shutdown_connections 2024-05-22T21:22:50.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.940+0000 7f3bb7a00640 1 -- 172.21.15.161:0/3967945655 wait complete. 2024-05-22T21:22:50.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb7a00640 1 Processor -- start 2024-05-22T21:22:50.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb7a00640 1 -- start start 2024-05-22T21:22:50.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb7a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3b94003980 0x7f3bb013d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb7a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bb01f6630 0x7f3bb01f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb7a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bb00a8970 0x7f3bb00a8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb7a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb01f6bf0 con 0x7f3bb00a8970 2024-05-22T21:22:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb7a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb00a9330 con 0x7f3bb01f6630 2024-05-22T21:22:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb7a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb00a9780 con 0x7f3b94003980 2024-05-22T21:22:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb5a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bb00a8970 0x7f3bb00a8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb5a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bb00a8970 0x7f3bb00a8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49788/0 (socket says 172.21.15.161:49788) 2024-05-22T21:22:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb5a00640 1 -- 172.21.15.161:0/2166750875 learned_addr learned my addr 172.21.15.161:0/2166750875 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb5a00640 1 -- 172.21.15.161:0/2166750875 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3b94003980 msgr2=0x7f3bb013d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb5a00640 1 --2- 172.21.15.161:0/2166750875 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3b94003980 0x7f3bb013d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb5a00640 1 -- 172.21.15.161:0/2166750875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bb01f6630 msgr2=0x7f3bb01f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:50.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb5a00640 1 --2- 172.21.15.161:0/2166750875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bb01f6630 0x7f3bb01f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:50.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb5a00640 1 -- 172.21.15.161:0/2166750875 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb00aa430 con 0x7f3bb00a8970 2024-05-22T21:22:50.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb5a00640 1 --2- 172.21.15.161:0/2166750875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bb00a8970 0x7f3bb00a8df0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00cd4e0 tx=0x7f3ba800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:50.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bad600640 1 -- 172.21.15.161:0/2166750875 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800e040 con 0x7f3bb00a8970 2024-05-22T21:22:50.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bad600640 1 -- 172.21.15.161:0/2166750875 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba800f040 con 0x7f3bb00a8970 2024-05-22T21:22:50.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bad600640 1 -- 172.21.15.161:0/2166750875 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8003c00 con 0x7f3bb00a8970 2024-05-22T21:22:50.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb7a00640 1 -- 172.21.15.161:0/2166750875 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb00aa710 con 0x7f3bb00a8970 2024-05-22T21:22:50.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb7a00640 1 -- 172.21.15.161:0/2166750875 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb00aacd0 con 0x7f3bb00a8970 2024-05-22T21:22:50.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.944+0000 7f3bb7a00640 1 -- 172.21.15.161:0/2166750875 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb00aaef0 con 0x7f3bb00a8970 2024-05-22T21:22:50.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.948+0000 7f3bad600640 1 -- 172.21.15.161:0/2166750875 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3ba8004030 con 0x7f3bb00a8970 2024-05-22T21:22:50.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.948+0000 7f3bad600640 1 --2- 172.21.15.161:0/2166750875 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3b8c0666c0 0x7f3b8c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:50.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.948+0000 7f3bad600640 1 -- 172.21.15.161:0/2166750875 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3ba8084a20 con 0x7f3bb00a8970 2024-05-22T21:22:50.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.948+0000 7f3bb5000640 1 --2- 172.21.15.161:0/2166750875 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3b8c0666c0 0x7f3b8c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:50.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.948+0000 7f3bad600640 1 -- 172.21.15.161:0/2166750875 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7f3b8c06e2b0 con 0x7f3bb00a8970 2024-05-22T21:22:50.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.948+0000 7f3bb5000640 1 --2- 172.21.15.161:0/2166750875 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3b8c0666c0 0x7f3b8c068b80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00af000 tx=0x7f3ba00aa040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:50.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:50.952+0000 7f3bad600640 1 -- 172.21.15.161:0/2166750875 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3ba8055870 con 0x7f3bb00a8970 2024-05-22T21:22:51.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.136+0000 7f3bb7a00640 1 -- 172.21.15.161:0/2166750875 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bb0067220 con 0x7f3bb00a8970 2024-05-22T21:22:51.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.136+0000 7f3bad600640 1 -- 172.21.15.161:0/2166750875 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f3ba8023030 con 0x7f3bb00a8970 2024-05-22T21:22:51.137 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:22:51.137 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":70,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7992,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/207877183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":207877183},{"type":"v1","addr":"172.21.15.161:6836","nonce":207877183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2152702106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2152702106},{"type":"v1","addr":"172.21.15.161:6837","nonce":2152702106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/749927648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":749927648},{"type":"v1","addr":"172.21.15.179:6834","nonce":749927648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:20:33.964441+0000","modified":"2024-05-22T21:20:36.988238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8004},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8004":{"gid":8004,"name":"b","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/93167194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":93167194},{"type":"v1","addr":"172.21.15.179:6835","nonce":93167194}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-05-22T21:22:51.138 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 70 2024-05-22T21:22:51.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.140+0000 7f3bb7a00640 1 -- 172.21.15.161:0/2166750875 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3b8c0666c0 msgr2=0x7f3b8c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.140+0000 7f3bb7a00640 1 --2- 172.21.15.161:0/2166750875 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3b8c0666c0 0x7f3b8c068b80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00af000 tx=0x7f3ba00aa040 comp rx=0 tx=0).stop 2024-05-22T21:22:51.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.140+0000 7f3bb7a00640 1 -- 172.21.15.161:0/2166750875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bb00a8970 msgr2=0x7f3bb00a8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.140+0000 7f3bb7a00640 1 --2- 172.21.15.161:0/2166750875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bb00a8970 0x7f3bb00a8df0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00cd4e0 tx=0x7f3ba800f580 comp rx=0 tx=0).stop 2024-05-22T21:22:51.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.140+0000 7f3bb7a00640 1 -- 172.21.15.161:0/2166750875 shutdown_connections 2024-05-22T21:22:51.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.140+0000 7f3bb7a00640 1 --2- 172.21.15.161:0/2166750875 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3b8c0666c0 0x7f3b8c068b80 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.140+0000 7f3bb7a00640 1 --2- 172.21.15.161:0/2166750875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bb00a8970 0x7f3bb00a8df0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.140+0000 7f3bb7a00640 1 --2- 172.21.15.161:0/2166750875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bb01f6630 0x7f3bb01f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.140+0000 7f3bb7a00640 1 --2- 172.21.15.161:0/2166750875 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3b94003980 0x7f3bb013d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.140+0000 7f3bb7a00640 1 -- 172.21.15.161:0/2166750875 >> 172.21.15.161:0/2166750875 conn(0x7f3bb00b5540 msgr2=0x7f3bb0141ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:51.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.140+0000 7f3bb7a00640 1 -- 172.21.15.161:0/2166750875 shutdown_connections 2024-05-22T21:22:51.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.140+0000 7f3bb7a00640 1 -- 172.21.15.161:0/2166750875 wait complete. 2024-05-22T21:22:51.159 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:22:51.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.292+0000 7f0e4e400640 1 Processor -- start 2024-05-22T21:22:51.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.292+0000 7f0e4e400640 1 -- start start 2024-05-22T21:22:51.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.296+0000 7f0e4e400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4815e130 0x7f0e4815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.296+0000 7f0e4e400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0e4815d410 0x7f0e4815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.296+0000 7f0e4e400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e48071670 con 0x7f0e4815dd50 2024-05-22T21:22:51.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.296+0000 7f0e4e400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4815d870 con 0x7f0e4815d410 2024-05-22T21:22:51.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.296+0000 7f0e4e400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e48070a60 con 0x7f0e4815e130 2024-05-22T21:22:51.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.296+0000 7f0e47400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0e4815dd50 0x7f0e4806d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44062/0 (socket says 172.21.15.161:44062) 2024-05-22T21:22:51.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.296+0000 7f0e47400640 1 -- 172.21.15.161:0/1313659151 learned_addr learned my addr 172.21.15.161:0/1313659151 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:51.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.296+0000 7f0e47e00640 1 --2- 172.21.15.161:0/1313659151 >> v2:172.21.15.161:3300/0 conn(0x7f0e4815d410 0x7f0e4815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.296+0000 7f0e46a00640 1 --2- 172.21.15.161:0/1313659151 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4815e130 0x7f0e4815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.296+0000 7f0e46000640 1 -- 172.21.15.161:0/1313659151 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563080762 0 0) 0x7f0e48071670 con 0x7f0e4815dd50 2024-05-22T21:22:51.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.296+0000 7f0e46000640 1 -- 172.21.15.161:0/1313659151 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e2c005740 con 0x7f0e4815dd50 2024-05-22T21:22:51.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.296+0000 7f0e47e00640 1 -- 172.21.15.161:0/1313659151 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4815e130 msgr2=0x7f0e4815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:51.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.296+0000 7f0e47e00640 1 --2- 172.21.15.161:0/1313659151 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4815e130 0x7f0e4815ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.296+0000 7f0e47e00640 1 -- 172.21.15.161:0/1313659151 >> v1:172.21.15.161:6789/0 conn(0x7f0e4815dd50 legacy=0x7f0e4806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.296+0000 7f0e47e00640 1 -- 172.21.15.161:0/1313659151 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e48070d40 con 0x7f0e4815d410 2024-05-22T21:22:51.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.296+0000 7f0e46a00640 1 --2- 172.21.15.161:0/1313659151 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4815e130 0x7f0e4815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:22:51.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e47e00640 1 --2- 172.21.15.161:0/1313659151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4815d410 0x7f0e4815b6c0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009d90 tx=0x7f0e4001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d714171d73de5e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:51.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e46000640 1 -- 172.21.15.161:0/1313659151 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e400a5970 con 0x7f0e4815d410 2024-05-22T21:22:51.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e46000640 1 -- 172.21.15.161:0/1313659151 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e400a5ad0 con 0x7f0e4815d410 2024-05-22T21:22:51.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e46000640 1 -- 172.21.15.161:0/1313659151 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e400a5e60 con 0x7f0e4815d410 2024-05-22T21:22:51.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e4e400640 1 -- 172.21.15.161:0/1313659151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4815d410 msgr2=0x7f0e4815b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e4e400640 1 --2- 172.21.15.161:0/1313659151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4815d410 0x7f0e4815b6c0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009d90 tx=0x7f0e4001c8b0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e4e400640 1 -- 172.21.15.161:0/1313659151 shutdown_connections 2024-05-22T21:22:51.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e4e400640 1 --2- 172.21.15.161:0/1313659151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4815d410 0x7f0e4815b6c0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e4e400640 1 --2- 172.21.15.161:0/1313659151 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4815e130 0x7f0e4815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e4e400640 1 -- 172.21.15.161:0/1313659151 >> 172.21.15.161:0/1313659151 conn(0x7f0e480b0bc0 msgr2=0x7f0e480b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:51.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e4e400640 1 -- 172.21.15.161:0/1313659151 shutdown_connections 2024-05-22T21:22:51.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e4e400640 1 -- 172.21.15.161:0/1313659151 wait complete. 2024-05-22T21:22:51.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e4e400640 1 Processor -- start 2024-05-22T21:22:51.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e4e400640 1 -- start start 2024-05-22T21:22:51.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e4e400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4815d410 0x7f0e48152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e4e400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e4815dd50 0x7f0e4814bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e4e400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4815e130 0x7f0e4814c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e4e400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e48158e30 con 0x7f0e4815d410 2024-05-22T21:22:51.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e4e400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4814cd30 con 0x7f0e4815dd50 2024-05-22T21:22:51.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e4e400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e48153440 con 0x7f0e4815e130 2024-05-22T21:22:51.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e47400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4815d410 0x7f0e48152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e47400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4815d410 0x7f0e48152f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49818/0 (socket says 172.21.15.161:49818) 2024-05-22T21:22:51.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.300+0000 7f0e47400640 1 -- 172.21.15.161:0/1832902151 learned_addr learned my addr 172.21.15.161:0/1832902151 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:51.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e46a00640 1 --2- 172.21.15.161:0/1832902151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e4815dd50 0x7f0e4814bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e47e00640 1 --2- 172.21.15.161:0/1832902151 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4815e130 0x7f0e4814c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e47400640 1 -- 172.21.15.161:0/1832902151 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4815e130 msgr2=0x7f0e4814c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e47400640 1 --2- 172.21.15.161:0/1832902151 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4815e130 0x7f0e4814c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e47400640 1 -- 172.21.15.161:0/1832902151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e4815dd50 msgr2=0x7f0e4814bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e47400640 1 --2- 172.21.15.161:0/1832902151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e4815dd50 0x7f0e4814bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e47400640 1 -- 172.21.15.161:0/1832902151 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e40009a70 con 0x7f0e4815d410 2024-05-22T21:22:51.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e46a00640 1 --2- 172.21.15.161:0/1832902151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e4815dd50 0x7f0e4814bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:22:51.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e47e00640 1 --2- 172.21.15.161:0/1832902151 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4815e130 0x7f0e4814c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:22:51.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e47400640 1 --2- 172.21.15.161:0/1832902151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4815d410 0x7f0e48152f00 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800c500 tx=0x7f0e3800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:51.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e3f400640 1 -- 172.21.15.161:0/1832902151 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38005cf0 con 0x7f0e4815d410 2024-05-22T21:22:51.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e3f400640 1 -- 172.21.15.161:0/1832902151 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e38005e50 con 0x7f0e4815d410 2024-05-22T21:22:51.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e4e400640 1 -- 172.21.15.161:0/1832902151 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e48158870 con 0x7f0e4815d410 2024-05-22T21:22:51.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e3f400640 1 -- 172.21.15.161:0/1832902151 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380072f0 con 0x7f0e4815d410 2024-05-22T21:22:51.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e4e400640 1 -- 172.21.15.161:0/1832902151 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e480b1140 con 0x7f0e4815d410 2024-05-22T21:22:51.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e4e400640 1 -- 172.21.15.161:0/1832902151 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e480b0fa0 con 0x7f0e4815d410 2024-05-22T21:22:51.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e3f400640 1 -- 172.21.15.161:0/1832902151 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0e38007450 con 0x7f0e4815d410 2024-05-22T21:22:51.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e3f400640 1 --2- 172.21.15.161:0/1832902151 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e28066690 0x7f0e28068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e3f400640 1 -- 172.21.15.161:0/1832902151 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0e380836d0 con 0x7f0e4815d410 2024-05-22T21:22:51.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e46a00640 1 --2- 172.21.15.161:0/1832902151 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e28066690 0x7f0e28068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.304+0000 7f0e3f400640 1 -- 172.21.15.161:0/1832902151 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7f0e2806e280 con 0x7f0e4815d410 2024-05-22T21:22:51.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.308+0000 7f0e46a00640 1 --2- 172.21.15.161:0/1832902151 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e28066690 0x7f0e28068b50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0e48154870 tx=0x7f0e34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:51.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.308+0000 7f0e3f400640 1 -- 172.21.15.161:0/1832902151 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0e38017060 con 0x7f0e4815d410 2024-05-22T21:22:51.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.468+0000 7f0e4e400640 1 -- 172.21.15.161:0/1832902151 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e48070730 con 0x7f0e4815d410 2024-05-22T21:22:51.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.468+0000 7f0e3f400640 1 -- 172.21.15.161:0/1832902151 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16228 (secure 0 0 0) 0x7f0e480b0fa0 con 0x7f0e4815d410 2024-05-22T21:22:51.468 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:22:51.468 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":162,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:22:05.280635+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:20:32.457234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-22T21:20:32.951241+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.16","mappings":[{"from":2,"to":5}]},{"pgid":"19.25","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:22:51.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.468+0000 7f0e4e400640 1 -- 172.21.15.161:0/1832902151 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e28066690 msgr2=0x7f0e28068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.468+0000 7f0e4e400640 1 --2- 172.21.15.161:0/1832902151 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e28066690 0x7f0e28068b50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0e48154870 tx=0x7f0e34008040 comp rx=0 tx=0).stop 2024-05-22T21:22:51.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.468+0000 7f0e4e400640 1 -- 172.21.15.161:0/1832902151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4815d410 msgr2=0x7f0e48152f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.468+0000 7f0e4e400640 1 --2- 172.21.15.161:0/1832902151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4815d410 0x7f0e48152f00 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800c500 tx=0x7f0e3800c9c0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.468+0000 7f0e4e400640 1 -- 172.21.15.161:0/1832902151 shutdown_connections 2024-05-22T21:22:51.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.468+0000 7f0e4e400640 1 --2- 172.21.15.161:0/1832902151 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e28066690 0x7f0e28068b50 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.468+0000 7f0e4e400640 1 --2- 172.21.15.161:0/1832902151 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4815e130 0x7f0e4814c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.468+0000 7f0e4e400640 1 --2- 172.21.15.161:0/1832902151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e4815dd50 0x7f0e4814bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.468+0000 7f0e4e400640 1 --2- 172.21.15.161:0/1832902151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4815d410 0x7f0e48152f00 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.468+0000 7f0e4e400640 1 -- 172.21.15.161:0/1832902151 >> 172.21.15.161:0/1832902151 conn(0x7f0e480b0bc0 msgr2=0x7f0e480b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:51.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.468+0000 7f0e4e400640 1 -- 172.21.15.161:0/1832902151 shutdown_connections 2024-05-22T21:22:51.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.468+0000 7f0e4e400640 1 -- 172.21.15.161:0/1832902151 wait complete. 2024-05-22T21:22:51.489 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-22T21:22:51.489 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:22:51.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.624+0000 7fc0b0200640 1 Processor -- start 2024-05-22T21:22:51.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.624+0000 7fc0b0200640 1 -- start start 2024-05-22T21:22:51.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.624+0000 7fc0b0200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0a815dd50 0x7fc0a806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.624+0000 7fc0b0200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc0a815e130 0x7fc0a815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.624+0000 7fc0b0200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a8071670 con 0x7fc0a815d410 2024-05-22T21:22:51.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.624+0000 7fc0b0200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a815d8a0 con 0x7fc0a815e130 2024-05-22T21:22:51.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.624+0000 7fc0b0200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a806de40 con 0x7fc0a815dd50 2024-05-22T21:22:51.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.624+0000 7fc0ae200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc0a815d410 0x7fc0a815a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44072/0 (socket says 172.21.15.161:44072) 2024-05-22T21:22:51.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.624+0000 7fc0ae200640 1 -- 172.21.15.161:0/3384082660 learned_addr learned my addr 172.21.15.161:0/3384082660 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:51.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.624+0000 7fc0ad800640 1 --2- 172.21.15.161:0/3384082660 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0a815dd50 0x7fc0a806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.628+0000 7fc0ace00640 1 --2- 172.21.15.161:0/3384082660 >> v2:172.21.15.179:3300/0 conn(0x7fc0a815e130 0x7fc0a815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.628+0000 7fc0a7e00640 1 -- 172.21.15.161:0/3384082660 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710102311 0 0) 0x7fc0a8071670 con 0x7fc0a815d410 2024-05-22T21:22:51.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.628+0000 7fc0a7e00640 1 -- 172.21.15.161:0/3384082660 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc090003530 con 0x7fc0a815d410 2024-05-22T21:22:51.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.628+0000 7fc0ad800640 1 -- 172.21.15.161:0/3384082660 >> v2:172.21.15.179:3300/0 conn(0x7fc0a815e130 msgr2=0x7fc0a815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:51.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.628+0000 7fc0ad800640 1 --2- 172.21.15.161:0/3384082660 >> v2:172.21.15.179:3300/0 conn(0x7fc0a815e130 0x7fc0a815ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.628+0000 7fc0ad800640 1 -- 172.21.15.161:0/3384082660 >> v1:172.21.15.161:6789/0 conn(0x7fc0a815d410 legacy=0x7fc0a815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.628+0000 7fc0ad800640 1 -- 172.21.15.161:0/3384082660 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a8070a60 con 0x7fc0a815dd50 2024-05-22T21:22:51.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.628+0000 7fc0ad800640 1 --2- 172.21.15.161:0/3384082660 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0a815dd50 0x7fc0a806d900 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0097c0 tx=0x7fc09c01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9fe7f285ea63d5d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:51.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.628+0000 7fc0a7e00640 1 -- 172.21.15.161:0/3384082660 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c0ac640 con 0x7fc0a815dd50 2024-05-22T21:22:51.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.628+0000 7fc0a7e00640 1 -- 172.21.15.161:0/3384082660 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09c0ac7a0 con 0x7fc0a815dd50 2024-05-22T21:22:51.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.628+0000 7fc0a7e00640 1 -- 172.21.15.161:0/3384082660 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c0acb10 con 0x7fc0a815dd50 2024-05-22T21:22:51.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0b0200640 1 -- 172.21.15.161:0/3384082660 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0a815dd50 msgr2=0x7fc0a806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0b0200640 1 --2- 172.21.15.161:0/3384082660 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0a815dd50 0x7fc0a806d900 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0097c0 tx=0x7fc09c01c3e0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0b0200640 1 -- 172.21.15.161:0/3384082660 shutdown_connections 2024-05-22T21:22:51.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0b0200640 1 --2- 172.21.15.161:0/3384082660 >> v2:172.21.15.179:3300/0 conn(0x7fc0a815e130 0x7fc0a815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0b0200640 1 --2- 172.21.15.161:0/3384082660 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0a815dd50 0x7fc0a806d900 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0b0200640 1 -- 172.21.15.161:0/3384082660 >> 172.21.15.161:0/3384082660 conn(0x7fc0a80b0bc0 msgr2=0x7fc0a80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:51.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0b0200640 1 -- 172.21.15.161:0/3384082660 shutdown_connections 2024-05-22T21:22:51.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0b0200640 1 -- 172.21.15.161:0/3384082660 wait complete. 2024-05-22T21:22:51.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0b0200640 1 Processor -- start 2024-05-22T21:22:51.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0b0200640 1 -- start start 2024-05-22T21:22:51.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0b0200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0a815d410 0x7fc0a81f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0b0200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0a815dd50 0x7fc0a8150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0b0200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc0a815e130 0x7fc0a8150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0b0200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a80cdda0 con 0x7fc0a815dd50 2024-05-22T21:22:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0b0200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a8147320 con 0x7fc0a815e130 2024-05-22T21:22:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0b0200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a80cdac0 con 0x7fc0a815d410 2024-05-22T21:22:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0ace00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0a815dd50 0x7fc0a8150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0ace00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0a815dd50 0x7fc0a8150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49834/0 (socket says 172.21.15.161:49834) 2024-05-22T21:22:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0ace00640 1 -- 172.21.15.161:0/285002098 learned_addr learned my addr 172.21.15.161:0/285002098 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0ad800640 1 --2- 172.21.15.161:0/285002098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0a815d410 0x7fc0a81f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0ae200640 1 --2- 172.21.15.161:0/285002098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc0a815e130 0x7fc0a8150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0ace00640 1 -- 172.21.15.161:0/285002098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0a815d410 msgr2=0x7fc0a81f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0ace00640 1 --2- 172.21.15.161:0/285002098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0a815d410 0x7fc0a81f6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0ace00640 1 -- 172.21.15.161:0/285002098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc0a815e130 msgr2=0x7fc0a8150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0ace00640 1 --2- 172.21.15.161:0/285002098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc0a815e130 0x7fc0a8150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0ace00640 1 -- 172.21.15.161:0/285002098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c009420 con 0x7fc0a815dd50 2024-05-22T21:22:51.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0ae200640 1 --2- 172.21.15.161:0/285002098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc0a815e130 0x7fc0a8150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:22:51.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0ad800640 1 --2- 172.21.15.161:0/285002098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0a815d410 0x7fc0a81f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:22:51.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.632+0000 7fc0ace00640 1 --2- 172.21.15.161:0/285002098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0a815dd50 0x7fc0a8150740 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fc094008620 tx=0x7fc094008ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:51.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.636+0000 7fc0a5600640 1 -- 172.21.15.161:0/285002098 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094010040 con 0x7fc0a815dd50 2024-05-22T21:22:51.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.636+0000 7fc0a5600640 1 -- 172.21.15.161:0/285002098 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0940163f0 con 0x7fc0a815dd50 2024-05-22T21:22:51.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.636+0000 7fc0b0200640 1 -- 172.21.15.161:0/285002098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a81475c0 con 0x7fc0a815dd50 2024-05-22T21:22:51.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.636+0000 7fc0a5600640 1 -- 172.21.15.161:0/285002098 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094009ea0 con 0x7fc0a815dd50 2024-05-22T21:22:51.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.636+0000 7fc0b0200640 1 -- 172.21.15.161:0/285002098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a81493f0 con 0x7fc0a815dd50 2024-05-22T21:22:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.636+0000 7fc0b0200640 1 -- 172.21.15.161:0/285002098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a8147ae0 con 0x7fc0a815dd50 2024-05-22T21:22:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.636+0000 7fc0a5600640 1 -- 172.21.15.161:0/285002098 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc094016780 con 0x7fc0a815dd50 2024-05-22T21:22:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.636+0000 7fc0a5600640 1 --2- 172.21.15.161:0/285002098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc088066690 0x7fc088068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.636+0000 7fc0a5600640 1 -- 172.21.15.161:0/285002098 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc094084280 con 0x7fc0a815dd50 2024-05-22T21:22:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.636+0000 7fc0ad800640 1 --2- 172.21.15.161:0/285002098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc088066690 0x7fc088068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.636+0000 7fc0a5600640 1 -- 172.21.15.161:0/285002098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7fc08806e280 con 0x7fc0a815dd50 2024-05-22T21:22:51.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.636+0000 7fc0ad800640 1 --2- 172.21.15.161:0/285002098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc088066690 0x7fc088068b50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0ae000 tx=0x7fc09c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:51.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.640+0000 7fc0a5600640 1 -- 172.21.15.161:0/285002098 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc0a8147ae0 con 0x7fc0a815dd50 2024-05-22T21:22:51.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.828+0000 7fc0b0200640 1 -- 172.21.15.161:0/285002098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0a8147ae0 con 0x7fc0a815dd50 2024-05-22T21:22:51.829 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.828+0000 7fc0a5600640 1 -- 172.21.15.161:0/285002098 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4736 (secure 0 0 0) 0x7fc0a8147ae0 con 0x7fc0a815dd50 2024-05-22T21:22:51.829 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:22:51.829 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":70,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7992,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/207877183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":207877183},{"type":"v1","addr":"172.21.15.161:6836","nonce":207877183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2152702106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2152702106},{"type":"v1","addr":"172.21.15.161:6837","nonce":2152702106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/749927648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":749927648},{"type":"v1","addr":"172.21.15.179:6834","nonce":749927648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:20:33.964441+0000","modified":"2024-05-22T21:20:36.988238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8004},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8004":{"gid":8004,"name":"b","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/93167194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":93167194},{"type":"v1","addr":"172.21.15.179:6835","nonce":93167194}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-05-22T21:22:51.829 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 70 2024-05-22T21:22:51.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.832+0000 7fc0b0200640 1 -- 172.21.15.161:0/285002098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc088066690 msgr2=0x7fc088068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.832+0000 7fc0b0200640 1 --2- 172.21.15.161:0/285002098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc088066690 0x7fc088068b50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0ae000 tx=0x7fc09c0a7000 comp rx=0 tx=0).stop 2024-05-22T21:22:51.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.832+0000 7fc0b0200640 1 -- 172.21.15.161:0/285002098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0a815dd50 msgr2=0x7fc0a8150740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.832+0000 7fc0b0200640 1 --2- 172.21.15.161:0/285002098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0a815dd50 0x7fc0a8150740 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fc094008620 tx=0x7fc094008ae0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.832+0000 7fc0b0200640 1 -- 172.21.15.161:0/285002098 shutdown_connections 2024-05-22T21:22:51.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.832+0000 7fc0b0200640 1 --2- 172.21.15.161:0/285002098 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc088066690 0x7fc088068b50 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.832+0000 7fc0b0200640 1 --2- 172.21.15.161:0/285002098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc0a815e130 0x7fc0a8150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.832+0000 7fc0b0200640 1 --2- 172.21.15.161:0/285002098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0a815dd50 0x7fc0a8150740 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.832+0000 7fc0b0200640 1 --2- 172.21.15.161:0/285002098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc0a815d410 0x7fc0a81f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.832+0000 7fc0b0200640 1 -- 172.21.15.161:0/285002098 >> 172.21.15.161:0/285002098 conn(0x7fc0a80b0bc0 msgr2=0x7fc0a80cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:51.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.832+0000 7fc0b0200640 1 -- 172.21.15.161:0/285002098 shutdown_connections 2024-05-22T21:22:51.832 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.832+0000 7fc0b0200640 1 -- 172.21.15.161:0/285002098 wait complete. 2024-05-22T21:22:51.851 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:22:51.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.984+0000 7f8ef9000640 1 Processor -- start 2024-05-22T21:22:51.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.984+0000 7f8ef9000640 1 -- start start 2024-05-22T21:22:51.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.988+0000 7f8ef9000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8ef406d700 0x7f8ef4168210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.988+0000 7f8ef9000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef4071fd0 con 0x7f8ef40707c0 2024-05-22T21:22:51.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.988+0000 7f8ef9000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef4070c00 con 0x7f8ef415d430 2024-05-22T21:22:51.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.988+0000 7f8ef9000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef40acfc0 con 0x7f8ef406d700 2024-05-22T21:22:51.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.988+0000 7f8ef1400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8ef40707c0 0x7f8ef406cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44074/0 (socket says 172.21.15.161:44074) 2024-05-22T21:22:51.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.988+0000 7f8ef1400640 1 -- 172.21.15.161:0/1239818568 learned_addr learned my addr 172.21.15.161:0/1239818568 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:51.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.988+0000 7f8ef2800640 1 --2- 172.21.15.161:0/1239818568 >> v2:172.21.15.179:3300/0 conn(0x7f8ef406d700 0x7f8ef4168210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.988+0000 7f8ef0a00640 1 -- 172.21.15.161:0/1239818568 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1902585419 0 0) 0x7f8ef4071fd0 con 0x7f8ef40707c0 2024-05-22T21:22:51.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.988+0000 7f8ef0a00640 1 -- 172.21.15.161:0/1239818568 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ed8003530 con 0x7f8ef40707c0 2024-05-22T21:22:51.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.988+0000 7f8ef2800640 1 -- 172.21.15.161:0/1239818568 >> v1:172.21.15.179:6789/0 conn(0x7f8ef415d430 legacy=0x7f8ef40ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.988+0000 7f8ef2800640 1 -- 172.21.15.161:0/1239818568 >> v1:172.21.15.161:6789/0 conn(0x7f8ef40707c0 legacy=0x7f8ef406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.988+0000 7f8ef2800640 1 -- 172.21.15.161:0/1239818568 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef4070e80 con 0x7f8ef406d700 2024-05-22T21:22:51.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.988+0000 7f8ef2800640 1 --2- 172.21.15.161:0/1239818568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ef406d700 0x7f8ef4168210 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0097c0 tx=0x7f8eec01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=748976cf878a4643 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:51.990 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef0a00640 1 -- 172.21.15.161:0/1239818568 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0ac640 con 0x7f8ef406d700 2024-05-22T21:22:51.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef0a00640 1 -- 172.21.15.161:0/1239818568 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eec0ac7a0 con 0x7f8ef406d700 2024-05-22T21:22:51.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef0a00640 1 -- 172.21.15.161:0/1239818568 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0acb30 con 0x7f8ef406d700 2024-05-22T21:22:51.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef9000640 1 -- 172.21.15.161:0/1239818568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ef406d700 msgr2=0x7f8ef4168210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef9000640 1 --2- 172.21.15.161:0/1239818568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ef406d700 0x7f8ef4168210 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0097c0 tx=0x7f8eec01c3e0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef9000640 1 -- 172.21.15.161:0/1239818568 shutdown_connections 2024-05-22T21:22:51.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef9000640 1 --2- 172.21.15.161:0/1239818568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ef406d700 0x7f8ef4168210 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef9000640 1 -- 172.21.15.161:0/1239818568 >> 172.21.15.161:0/1239818568 conn(0x7f8ef40b0bc0 msgr2=0x7f8ef40b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:51.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef9000640 1 -- 172.21.15.161:0/1239818568 shutdown_connections 2024-05-22T21:22:51.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef9000640 1 -- 172.21.15.161:0/1239818568 wait complete. 2024-05-22T21:22:51.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef9000640 1 Processor -- start 2024-05-22T21:22:51.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef9000640 1 -- start start 2024-05-22T21:22:51.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef9000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ef406d700 0x7f8ef4152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef9000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ef40707c0 0x7f8ef41f9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef9000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ef415d430 0x7f8ef41f9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef9000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef414dc40 con 0x7f8ef406d700 2024-05-22T21:22:51.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef9000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef414b600 con 0x7f8ef40707c0 2024-05-22T21:22:51.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef9000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef4152ea0 con 0x7f8ef415d430 2024-05-22T21:22:51.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ef406d700 0x7f8ef4152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ef406d700 0x7f8ef4152960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49844/0 (socket says 172.21.15.161:49844) 2024-05-22T21:22:51.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef1e00640 1 -- 172.21.15.161:0/532119521 learned_addr learned my addr 172.21.15.161:0/532119521 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:51.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef1400640 1 --2- 172.21.15.161:0/532119521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ef40707c0 0x7f8ef41f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.992+0000 7f8ef2800640 1 --2- 172.21.15.161:0/532119521 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ef415d430 0x7f8ef41f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.996+0000 7f8ef1e00640 1 -- 172.21.15.161:0/532119521 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ef415d430 msgr2=0x7f8ef41f9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.996+0000 7f8ef1e00640 1 --2- 172.21.15.161:0/532119521 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ef415d430 0x7f8ef41f9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.996+0000 7f8ef1e00640 1 -- 172.21.15.161:0/532119521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ef40707c0 msgr2=0x7f8ef41f9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:51.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.996+0000 7f8ef1e00640 1 --2- 172.21.15.161:0/532119521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ef40707c0 0x7f8ef41f9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:51.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.996+0000 7f8ef1e00640 1 -- 172.21.15.161:0/532119521 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eec009420 con 0x7f8ef406d700 2024-05-22T21:22:51.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.996+0000 7f8ef2800640 1 --2- 172.21.15.161:0/532119521 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ef415d430 0x7f8ef41f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:22:51.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.996+0000 7f8ef1e00640 1 --2- 172.21.15.161:0/532119521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ef406d700 0x7f8ef4152960 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400c760 tx=0x7f8ee400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:51.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.996+0000 7f8eea000640 1 -- 172.21.15.161:0/532119521 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4003ba0 con 0x7f8ef406d700 2024-05-22T21:22:51.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.996+0000 7f8eea000640 1 -- 172.21.15.161:0/532119521 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee4003d00 con 0x7f8ef406d700 2024-05-22T21:22:51.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.996+0000 7f8eea000640 1 -- 172.21.15.161:0/532119521 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee400aa90 con 0x7f8ef406d700 2024-05-22T21:22:51.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.996+0000 7f8ef9000640 1 -- 172.21.15.161:0/532119521 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef4153110 con 0x7f8ef406d700 2024-05-22T21:22:51.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.996+0000 7f8ef9000640 1 -- 172.21.15.161:0/532119521 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef414d270 con 0x7f8ef406d700 2024-05-22T21:22:51.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.996+0000 7f8ef9000640 1 -- 172.21.15.161:0/532119521 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef4067220 con 0x7f8ef406d700 2024-05-22T21:22:51.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.996+0000 7f8eea000640 1 -- 172.21.15.161:0/532119521 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8ee400abf0 con 0x7f8ef406d700 2024-05-22T21:22:51.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.996+0000 7f8eea000640 1 --2- 172.21.15.161:0/532119521 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ed4066690 0x7f8ed4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:51.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:51.996+0000 7f8eea000640 1 -- 172.21.15.161:0/532119521 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8ee4083f00 con 0x7f8ef406d700 2024-05-22T21:22:51.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.000+0000 7f8eea000640 1 -- 172.21.15.161:0/532119521 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7f8ed406e280 con 0x7f8ef406d700 2024-05-22T21:22:51.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.000+0000 7f8ef1400640 1 --2- 172.21.15.161:0/532119521 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ed4066690 0x7f8ed4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:51.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.000+0000 7f8ef1400640 1 --2- 172.21.15.161:0/532119521 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ed4066690 0x7f8ed4068b50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8ef414c1a0 tx=0x7f8ee0007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:52.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.000+0000 7f8eea000640 1 -- 172.21.15.161:0/532119521 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8ee4051230 con 0x7f8ef406d700 2024-05-22T21:22:52.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.184+0000 7f8ef9000640 1 -- 172.21.15.161:0/532119521 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ef40ae7a0 con 0x7f8ef406d700 2024-05-22T21:22:52.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.188+0000 7f8eea000640 1 -- 172.21.15.161:0/532119521 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f8ee4054850 con 0x7f8ef406d700 2024-05-22T21:22:52.187 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:22:52.187 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":70,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7992,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/207877183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":207877183},{"type":"v1","addr":"172.21.15.161:6836","nonce":207877183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2152702106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2152702106},{"type":"v1","addr":"172.21.15.161:6837","nonce":2152702106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/749927648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":749927648},{"type":"v1","addr":"172.21.15.179:6834","nonce":749927648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:20:33.964441+0000","modified":"2024-05-22T21:20:36.988238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8004},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8004":{"gid":8004,"name":"b","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.179:6835/93167194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":93167194},{"type":"v1","addr":"172.21.15.179:6835","nonce":93167194}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-05-22T21:22:52.187 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 70 2024-05-22T21:22:52.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.188+0000 7f8ef9000640 1 -- 172.21.15.161:0/532119521 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ed4066690 msgr2=0x7f8ed4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:52.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.188+0000 7f8ef9000640 1 --2- 172.21.15.161:0/532119521 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ed4066690 0x7f8ed4068b50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8ef414c1a0 tx=0x7f8ee0007d40 comp rx=0 tx=0).stop 2024-05-22T21:22:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.188+0000 7f8ef9000640 1 -- 172.21.15.161:0/532119521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ef406d700 msgr2=0x7f8ef4152960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.188+0000 7f8ef9000640 1 --2- 172.21.15.161:0/532119521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ef406d700 0x7f8ef4152960 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400c760 tx=0x7f8ee400cc20 comp rx=0 tx=0).stop 2024-05-22T21:22:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.188+0000 7f8ef9000640 1 -- 172.21.15.161:0/532119521 shutdown_connections 2024-05-22T21:22:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.188+0000 7f8ef9000640 1 --2- 172.21.15.161:0/532119521 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8ed4066690 0x7f8ed4068b50 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.188+0000 7f8ef9000640 1 --2- 172.21.15.161:0/532119521 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ef415d430 0x7f8ef41f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.188+0000 7f8ef9000640 1 --2- 172.21.15.161:0/532119521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ef40707c0 0x7f8ef41f9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.188+0000 7f8ef9000640 1 --2- 172.21.15.161:0/532119521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ef406d700 0x7f8ef4152960 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.188+0000 7f8ef9000640 1 -- 172.21.15.161:0/532119521 >> 172.21.15.161:0/532119521 conn(0x7f8ef40b0bc0 msgr2=0x7f8ef4161560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.188+0000 7f8ef9000640 1 -- 172.21.15.161:0/532119521 shutdown_connections 2024-05-22T21:22:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.188+0000 7f8ef9000640 1 -- 172.21.15.161:0/532119521 wait complete. 2024-05-22T21:22:52.209 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:22:52.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.344+0000 7f4fbd000640 1 Processor -- start 2024-05-22T21:22:52.346 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.344+0000 7f4fbd000640 1 -- start start 2024-05-22T21:22:52.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.348+0000 7f4fbd000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4fb80707c0 0x7f4fb806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:52.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.348+0000 7f4fbd000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4fb806d530 0x7f4fb8160e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:52.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.348+0000 7f4fbd000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb8071fd0 con 0x7f4fb815d430 2024-05-22T21:22:52.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.348+0000 7f4fbd000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb80acfc0 con 0x7f4fb80707c0 2024-05-22T21:22:52.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.348+0000 7f4fbd000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb8070c00 con 0x7f4fb806d530 2024-05-22T21:22:52.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.348+0000 7f4fad400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4fb80707c0 0x7f4fb806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:52.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.348+0000 7f4fb5e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f4fb815d430 0x7f4fb80ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44086/0 (socket says 172.21.15.161:44086) 2024-05-22T21:22:52.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.348+0000 7f4fb5e00640 1 -- 172.21.15.161:0/582008979 learned_addr learned my addr 172.21.15.161:0/582008979 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:52.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.348+0000 7f4fb6800640 1 --2- 172.21.15.161:0/582008979 >> v2:172.21.15.179:3300/0 conn(0x7f4fb806d530 0x7f4fb8160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:52.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.348+0000 7f4fad400640 1 -- 172.21.15.161:0/582008979 >> v2:172.21.15.179:3300/0 conn(0x7f4fb806d530 msgr2=0x7f4fb8160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:52.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.348+0000 7f4fad400640 1 --2- 172.21.15.161:0/582008979 >> v2:172.21.15.179:3300/0 conn(0x7f4fb806d530 0x7f4fb8160e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.348+0000 7f4fad400640 1 -- 172.21.15.161:0/582008979 >> v1:172.21.15.161:6789/0 conn(0x7f4fb815d430 legacy=0x7f4fb80ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:52.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.348+0000 7f4fad400640 1 -- 172.21.15.161:0/582008979 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb8070e80 con 0x7f4fb80707c0 2024-05-22T21:22:52.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.352+0000 7f4fad400640 1 --2- 172.21.15.161:0/582008979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb80707c0 0x7f4fb806cff0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80aeac0 tx=0x7f4fa001bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b622b9626aefabf server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:52.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.352+0000 7f4fb5400640 1 -- 172.21.15.161:0/582008979 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa00ac440 con 0x7f4fb80707c0 2024-05-22T21:22:52.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.352+0000 7f4fb5400640 1 -- 172.21.15.161:0/582008979 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa00ac5a0 con 0x7f4fb80707c0 2024-05-22T21:22:52.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.352+0000 7f4fb5400640 1 -- 172.21.15.161:0/582008979 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa00ac930 con 0x7f4fb80707c0 2024-05-22T21:22:52.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.352+0000 7f4fbd000640 1 -- 172.21.15.161:0/582008979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb80707c0 msgr2=0x7f4fb806cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:52.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.352+0000 7f4fbd000640 1 --2- 172.21.15.161:0/582008979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb80707c0 0x7f4fb806cff0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80aeac0 tx=0x7f4fa001bf90 comp rx=0 tx=0).stop 2024-05-22T21:22:52.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.352+0000 7f4fbd000640 1 -- 172.21.15.161:0/582008979 shutdown_connections 2024-05-22T21:22:52.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.352+0000 7f4fbd000640 1 --2- 172.21.15.161:0/582008979 >> v2:172.21.15.179:3300/0 conn(0x7f4fb806d530 0x7f4fb8160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.352+0000 7f4fbd000640 1 --2- 172.21.15.161:0/582008979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb80707c0 0x7f4fb806cff0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.352+0000 7f4fbd000640 1 -- 172.21.15.161:0/582008979 >> 172.21.15.161:0/582008979 conn(0x7f4fb80b0bc0 msgr2=0x7f4fb80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:52.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.352+0000 7f4fbd000640 1 -- 172.21.15.161:0/582008979 shutdown_connections 2024-05-22T21:22:52.352 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.352+0000 7f4fbd000640 1 -- 172.21.15.161:0/582008979 wait complete. 2024-05-22T21:22:52.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.352+0000 7f4fbd000640 1 Processor -- start 2024-05-22T21:22:52.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.352+0000 7f4fbd000640 1 -- start start 2024-05-22T21:22:52.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fbd000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4fb806d530 0x7f4fb81529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:52.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fbd000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb80707c0 0x7f4fb8158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:52.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fbd000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4fb815d430 0x7f4fb81588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:52.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fbd000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8159500 con 0x7f4fb80707c0 2024-05-22T21:22:52.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fbd000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8157750 con 0x7f4fb806d530 2024-05-22T21:22:52.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fbd000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8158f40 con 0x7f4fb815d430 2024-05-22T21:22:52.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fad400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb80707c0 0x7f4fb8158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:52.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fad400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb80707c0 0x7f4fb8158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49858/0 (socket says 172.21.15.161:49858) 2024-05-22T21:22:52.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fad400640 1 -- 172.21.15.161:0/3357021251 learned_addr learned my addr 172.21.15.161:0/3357021251 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:52.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fb5e00640 1 --2- 172.21.15.161:0/3357021251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4fb806d530 0x7f4fb81529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:52.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fb6800640 1 --2- 172.21.15.161:0/3357021251 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4fb815d430 0x7f4fb81588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:52.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fad400640 1 -- 172.21.15.161:0/3357021251 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4fb815d430 msgr2=0x7f4fb81588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:52.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fad400640 1 --2- 172.21.15.161:0/3357021251 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4fb815d430 0x7f4fb81588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fad400640 1 -- 172.21.15.161:0/3357021251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4fb806d530 msgr2=0x7f4fb81529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:52.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fad400640 1 --2- 172.21.15.161:0/3357021251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4fb806d530 0x7f4fb81529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fad400640 1 -- 172.21.15.161:0/3357021251 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa0009420 con 0x7f4fb80707c0 2024-05-22T21:22:52.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fb6800640 1 --2- 172.21.15.161:0/3357021251 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4fb815d430 0x7f4fb81588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:22:52.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fb5e00640 1 --2- 172.21.15.161:0/3357021251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4fb806d530 0x7f4fb81529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:22:52.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fad400640 1 --2- 172.21.15.161:0/3357021251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb80707c0 0x7f4fb8158390 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00af040 tx=0x7f4fa0018d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:52.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4faea00640 1 -- 172.21.15.161:0/3357021251 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa00b5820 con 0x7f4fb80707c0 2024-05-22T21:22:52.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4faea00640 1 -- 172.21.15.161:0/3357021251 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa00b5980 con 0x7f4fb80707c0 2024-05-22T21:22:52.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fbd000640 1 -- 172.21.15.161:0/3357021251 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb8159220 con 0x7f4fb80707c0 2024-05-22T21:22:52.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4faea00640 1 -- 172.21.15.161:0/3357021251 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa00b5d30 con 0x7f4fb80707c0 2024-05-22T21:22:52.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fbd000640 1 -- 172.21.15.161:0/3357021251 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb814e5c0 con 0x7f4fb80707c0 2024-05-22T21:22:52.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4fbd000640 1 -- 172.21.15.161:0/3357021251 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb814e7b0 con 0x7f4fb80707c0 2024-05-22T21:22:52.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.356+0000 7f4faea00640 1 -- 172.21.15.161:0/3357021251 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4fa00b3040 con 0x7f4fb80707c0 2024-05-22T21:22:52.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.360+0000 7f4faea00640 1 --2- 172.21.15.161:0/3357021251 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4f94066690 0x7f4f94068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:52.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.360+0000 7f4faea00640 1 -- 172.21.15.161:0/3357021251 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4fa011e480 con 0x7f4fb80707c0 2024-05-22T21:22:52.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.360+0000 7f4faea00640 1 -- 172.21.15.161:0/3357021251 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7f4f9406e280 con 0x7f4fb80707c0 2024-05-22T21:22:52.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.360+0000 7f4fb5e00640 1 --2- 172.21.15.161:0/3357021251 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4f94066690 0x7f4f94068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:52.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.360+0000 7f4fb5e00640 1 --2- 172.21.15.161:0/3357021251 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4f94066690 0x7f4f94068b50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8005fd0 tx=0x7f4fa8005b80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:52.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.364+0000 7f4faea00640 1 -- 172.21.15.161:0/3357021251 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4fa00eb7a0 con 0x7f4fb80707c0 2024-05-22T21:22:52.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.516+0000 7f4fbd000640 1 -- 172.21.15.161:0/3357021251 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fb8067220 con 0x7f4fb80707c0 2024-05-22T21:22:52.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.520+0000 7f4faea00640 1 -- 172.21.15.161:0/3357021251 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16228 (secure 0 0 0) 0x7f4fa00b1070 con 0x7f4fb80707c0 2024-05-22T21:22:52.519 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:22:52.519 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":162,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:22:05.280635+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:20:32.457234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-22T21:20:32.951241+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.16","mappings":[{"from":2,"to":5}]},{"pgid":"19.25","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:22:52.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.520+0000 7f4fbd000640 1 -- 172.21.15.161:0/3357021251 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4f94066690 msgr2=0x7f4f94068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:52.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.520+0000 7f4fbd000640 1 --2- 172.21.15.161:0/3357021251 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4f94066690 0x7f4f94068b50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8005fd0 tx=0x7f4fa8005b80 comp rx=0 tx=0).stop 2024-05-22T21:22:52.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.520+0000 7f4fbd000640 1 -- 172.21.15.161:0/3357021251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb80707c0 msgr2=0x7f4fb8158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:52.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.520+0000 7f4fbd000640 1 --2- 172.21.15.161:0/3357021251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb80707c0 0x7f4fb8158390 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00af040 tx=0x7f4fa0018d30 comp rx=0 tx=0).stop 2024-05-22T21:22:52.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.520+0000 7f4fbd000640 1 -- 172.21.15.161:0/3357021251 shutdown_connections 2024-05-22T21:22:52.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.520+0000 7f4fbd000640 1 --2- 172.21.15.161:0/3357021251 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4f94066690 0x7f4f94068b50 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.520+0000 7f4fbd000640 1 --2- 172.21.15.161:0/3357021251 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4fb815d430 0x7f4fb81588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.520+0000 7f4fbd000640 1 --2- 172.21.15.161:0/3357021251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fb80707c0 0x7f4fb8158390 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.520+0000 7f4fbd000640 1 --2- 172.21.15.161:0/3357021251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4fb806d530 0x7f4fb81529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.520+0000 7f4fbd000640 1 -- 172.21.15.161:0/3357021251 >> 172.21.15.161:0/3357021251 conn(0x7f4fb80b0bc0 msgr2=0x7f4fb80ad120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:52.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.520+0000 7f4fbd000640 1 -- 172.21.15.161:0/3357021251 shutdown_connections 2024-05-22T21:22:52.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.520+0000 7f4fbd000640 1 -- 172.21.15.161:0/3357021251 wait complete. 2024-05-22T21:22:52.541 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-22T21:22:52.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.676+0000 7f03e8c00640 1 Processor -- start 2024-05-22T21:22:52.678 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.676+0000 7f03e8c00640 1 -- start start 2024-05-22T21:22:52.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.680+0000 7f03e8c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03e413cfc0 0x7f03e41402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:52.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.680+0000 7f03e8c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f03e413fe50 0x7f03e406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:52.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.680+0000 7f03e8c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e413d3a0 con 0x7f03e415af60 2024-05-22T21:22:52.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.680+0000 7f03e8c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e40cd920 con 0x7f03e413fe50 2024-05-22T21:22:52.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.680+0000 7f03e8c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40cdc00 con 0x7f03e413cfc0 2024-05-22T21:22:52.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.680+0000 7f03d9400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f03e415af60 0x7f03e413f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44090/0 (socket says 172.21.15.161:44090) 2024-05-22T21:22:52.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.680+0000 7f03d9400640 1 -- 172.21.15.161:0/2395483647 learned_addr learned my addr 172.21.15.161:0/2395483647 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:52.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.680+0000 7f03e1e00640 1 --2- 172.21.15.161:0/2395483647 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03e413cfc0 0x7f03e41402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:52.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.680+0000 7f03e2800640 1 --2- 172.21.15.161:0/2395483647 >> v2:172.21.15.179:3300/0 conn(0x7f03e413fe50 0x7f03e406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:52.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.680+0000 7f03e1400640 1 -- 172.21.15.161:0/2395483647 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457791199 0 0) 0x7f03e413d3a0 con 0x7f03e415af60 2024-05-22T21:22:52.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.680+0000 7f03e1400640 1 -- 172.21.15.161:0/2395483647 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03c8005740 con 0x7f03e415af60 2024-05-22T21:22:52.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.680+0000 7f03e2800640 1 -- 172.21.15.161:0/2395483647 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03e413cfc0 msgr2=0x7f03e41402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:52.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.680+0000 7f03e2800640 1 --2- 172.21.15.161:0/2395483647 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03e413cfc0 0x7f03e41402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.680+0000 7f03e2800640 1 -- 172.21.15.161:0/2395483647 >> v1:172.21.15.161:6789/0 conn(0x7f03e415af60 legacy=0x7f03e413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:52.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.680+0000 7f03e2800640 1 -- 172.21.15.161:0/2395483647 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e40cdee0 con 0x7f03e413fe50 2024-05-22T21:22:52.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.680+0000 7f03e1e00640 1 --2- 172.21.15.161:0/2395483647 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03e413cfc0 0x7f03e41402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:22:52.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.684+0000 7f03e2800640 1 --2- 172.21.15.161:0/2395483647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03e413fe50 0x7f03e406fce0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009d90 tx=0x7f03dc01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad08802d5e0f1455 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:52.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.684+0000 7f03e1400640 1 -- 172.21.15.161:0/2395483647 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0a8610 con 0x7f03e413fe50 2024-05-22T21:22:52.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.684+0000 7f03e1400640 1 -- 172.21.15.161:0/2395483647 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03dc0a8770 con 0x7f03e413fe50 2024-05-22T21:22:52.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.684+0000 7f03e1400640 1 -- 172.21.15.161:0/2395483647 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0a8b00 con 0x7f03e413fe50 2024-05-22T21:22:52.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.684+0000 7f03e8c00640 1 -- 172.21.15.161:0/2395483647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03e413fe50 msgr2=0x7f03e406fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:52.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.684+0000 7f03e8c00640 1 --2- 172.21.15.161:0/2395483647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03e413fe50 0x7f03e406fce0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009d90 tx=0x7f03dc01c8b0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.684+0000 7f03e8c00640 1 -- 172.21.15.161:0/2395483647 shutdown_connections 2024-05-22T21:22:52.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.684+0000 7f03e8c00640 1 --2- 172.21.15.161:0/2395483647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03e413fe50 0x7f03e406fce0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.684+0000 7f03e8c00640 1 --2- 172.21.15.161:0/2395483647 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03e413cfc0 0x7f03e41402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.684+0000 7f03e8c00640 1 -- 172.21.15.161:0/2395483647 >> 172.21.15.161:0/2395483647 conn(0x7f03e406a2d0 msgr2=0x7f03e406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:52.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.684+0000 7f03e8c00640 1 -- 172.21.15.161:0/2395483647 shutdown_connections 2024-05-22T21:22:52.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.684+0000 7f03e8c00640 1 -- 172.21.15.161:0/2395483647 wait complete. 2024-05-22T21:22:52.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.684+0000 7f03e8c00640 1 Processor -- start 2024-05-22T21:22:52.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.684+0000 7f03e8c00640 1 -- start start 2024-05-22T21:22:52.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e8c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03e413cfc0 0x7f03e41579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:52.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e8c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03e413fe50 0x7f03e4140c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:52.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e8c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e415af60 0x7f03e41364b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:52.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e8c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e4157f30 con 0x7f03e415af60 2024-05-22T21:22:52.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e8c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e41415b0 con 0x7f03e413fe50 2024-05-22T21:22:52.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e8c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e4141890 con 0x7f03e413cfc0 2024-05-22T21:22:52.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e2800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e415af60 0x7f03e41364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:52.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e2800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e415af60 0x7f03e41364b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49870/0 (socket says 172.21.15.161:49870) 2024-05-22T21:22:52.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e2800640 1 -- 172.21.15.161:0/3468467384 learned_addr learned my addr 172.21.15.161:0/3468467384 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:52.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03d9400640 1 --2- 172.21.15.161:0/3468467384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03e413fe50 0x7f03e4140c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:52.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e1e00640 1 --2- 172.21.15.161:0/3468467384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03e413cfc0 0x7f03e41579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:52.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e2800640 1 -- 172.21.15.161:0/3468467384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03e413cfc0 msgr2=0x7f03e41579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:52.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e2800640 1 --2- 172.21.15.161:0/3468467384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03e413cfc0 0x7f03e41579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e2800640 1 -- 172.21.15.161:0/3468467384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03e413fe50 msgr2=0x7f03e4140c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e2800640 1 --2- 172.21.15.161:0/3468467384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03e413fe50 0x7f03e4140c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e2800640 1 -- 172.21.15.161:0/3468467384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d4009420 con 0x7f03e415af60 2024-05-22T21:22:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e2800640 1 --2- 172.21.15.161:0/3468467384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e415af60 0x7f03e41364b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009d90 tx=0x7f03dc019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03daa00640 1 -- 172.21.15.161:0/3468467384 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0b0600 con 0x7f03e415af60 2024-05-22T21:22:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03daa00640 1 -- 172.21.15.161:0/3468467384 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03dc0b6040 con 0x7f03e415af60 2024-05-22T21:22:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e8c00640 1 -- 172.21.15.161:0/3468467384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03dc009a70 con 0x7f03e415af60 2024-05-22T21:22:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03daa00640 1 -- 172.21.15.161:0/3468467384 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0b09a0 con 0x7f03e415af60 2024-05-22T21:22:52.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e8c00640 1 -- 172.21.15.161:0/3468467384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e4141e50 con 0x7f03e415af60 2024-05-22T21:22:52.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03e8c00640 1 -- 172.21.15.161:0/3468467384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e4067220 con 0x7f03e415af60 2024-05-22T21:22:52.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.688+0000 7f03daa00640 1 -- 172.21.15.161:0/3468467384 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f03dc0a6040 con 0x7f03e415af60 2024-05-22T21:22:52.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.692+0000 7f03daa00640 1 --2- 172.21.15.161:0/3468467384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f03c0066690 0x7f03c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:52.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.692+0000 7f03daa00640 1 -- 172.21.15.161:0/3468467384 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f03dc11f140 con 0x7f03e415af60 2024-05-22T21:22:52.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.692+0000 7f03e1e00640 1 --2- 172.21.15.161:0/3468467384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f03c0066690 0x7f03c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:52.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.692+0000 7f03daa00640 1 -- 172.21.15.161:0/3468467384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=163}) v3 -- 0x7f03c006e280 con 0x7f03e415af60 2024-05-22T21:22:52.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.692+0000 7f03e1e00640 1 --2- 172.21.15.161:0/3468467384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f03c0066690 0x7f03c0068b50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009cf0 tx=0x7f03d4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:52.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.696+0000 7f03daa00640 1 -- 172.21.15.161:0/3468467384 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f03dc0ec5a0 con 0x7f03e415af60 2024-05-22T21:22:52.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.884+0000 7f03e8c00640 1 -- 172.21.15.161:0/3468467384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f03e40679d0 con 0x7f03e415af60 2024-05-22T21:22:52.885 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:22:52.884+0000 7f7d32800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-22T21:22:52.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.892+0000 7f03daa00640 1 -- 172.21.15.161:0/3468467384 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(163..163 src has 1..163) v4 ==== 374+0+0 (secure 0 0 0) 0x7f03dc0f0260 con 0x7f03e415af60 2024-05-22T21:22:52.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.892+0000 7f03daa00640 1 -- 172.21.15.161:0/3468467384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=164}) v3 -- 0x7f03c006eac0 con 0x7f03e415af60 2024-05-22T21:22:52.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.900+0000 7f03daa00640 1 -- 172.21.15.161:0/3468467384 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v71) v1 ==== 161+0+0 (secure 0 0 0) 0x7f03dc009450 con 0x7f03e415af60 2024-05-22T21:22:52.900 INFO:teuthology.orchestra.run.smithi161.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-22T21:22:52.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.904+0000 7f03e8c00640 1 -- 172.21.15.161:0/3468467384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f03c0066690 msgr2=0x7f03c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:52.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.904+0000 7f03e8c00640 1 --2- 172.21.15.161:0/3468467384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f03c0066690 0x7f03c0068b50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009cf0 tx=0x7f03d4009190 comp rx=0 tx=0).stop 2024-05-22T21:22:52.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.904+0000 7f03e8c00640 1 -- 172.21.15.161:0/3468467384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e415af60 msgr2=0x7f03e41364b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:52.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.904+0000 7f03e8c00640 1 --2- 172.21.15.161:0/3468467384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e415af60 0x7f03e41364b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009d90 tx=0x7f03dc019650 comp rx=0 tx=0).stop 2024-05-22T21:22:52.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.904+0000 7f03e8c00640 1 -- 172.21.15.161:0/3468467384 shutdown_connections 2024-05-22T21:22:52.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.904+0000 7f03e8c00640 1 --2- 172.21.15.161:0/3468467384 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f03c0066690 0x7f03c0068b50 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.904+0000 7f03e8c00640 1 --2- 172.21.15.161:0/3468467384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e415af60 0x7f03e41364b0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.904+0000 7f03e8c00640 1 --2- 172.21.15.161:0/3468467384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03e413fe50 0x7f03e4140c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.904+0000 7f03e8c00640 1 --2- 172.21.15.161:0/3468467384 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03e413cfc0 0x7f03e41579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:52.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.904+0000 7f03e8c00640 1 -- 172.21.15.161:0/3468467384 >> 172.21.15.161:0/3468467384 conn(0x7f03e406a2d0 msgr2=0x7f03e40cc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:52.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.904+0000 7f03e8c00640 1 -- 172.21.15.161:0/3468467384 shutdown_connections 2024-05-22T21:22:52.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:52.904+0000 7f03e8c00640 1 -- 172.21.15.161:0/3468467384 wait complete. 2024-05-22T21:22:52.922 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-22T21:22:52.967 INFO:tasks.ceph.mds.b.smithi179.stdout:starting mds.b at 2024-05-22T21:22:53.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.060+0000 7f2895a00640 1 Processor -- start 2024-05-22T21:22:53.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.060+0000 7f2895a00640 1 -- start start 2024-05-22T21:22:53.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.060+0000 7f2895a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2890143c40 0x7f289015b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:53.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.060+0000 7f2895a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f289006c9b0 0x7f28900b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:53.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.060+0000 7f2895a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f289015efb0 con 0x7f28901444a0 2024-05-22T21:22:53.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.060+0000 7f2895a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f289015f290 con 0x7f289006c9b0 2024-05-22T21:22:53.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.060+0000 7f2895a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289015f570 con 0x7f2890143c40 2024-05-22T21:22:53.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.060+0000 7f2886000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f28901444a0 0x7f289006c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:44098/0 (socket says 172.21.15.161:44098) 2024-05-22T21:22:53.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.060+0000 7f2886000640 1 -- 172.21.15.161:0/2170657148 learned_addr learned my addr 172.21.15.161:0/2170657148 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:53.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.060+0000 7f288f400640 1 --2- 172.21.15.161:0/2170657148 >> v2:172.21.15.179:3300/0 conn(0x7f289006c9b0 0x7f28900b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:53.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.060+0000 7f288ea00640 1 --2- 172.21.15.161:0/2170657148 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2890143c40 0x7f289015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:53.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.060+0000 7f288e000640 1 -- 172.21.15.161:0/2170657148 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3185013867 0 0) 0x7f289015efb0 con 0x7f28901444a0 2024-05-22T21:22:53.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.060+0000 7f288e000640 1 -- 172.21.15.161:0/2170657148 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2874003530 con 0x7f28901444a0 2024-05-22T21:22:53.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.064+0000 7f288e000640 1 -- 172.21.15.161:0/2170657148 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 192498121 0 0) 0x7f2874003530 con 0x7f28901444a0 2024-05-22T21:22:53.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.064+0000 7f288ea00640 1 -- 172.21.15.161:0/2170657148 >> v2:172.21.15.179:3300/0 conn(0x7f289006c9b0 msgr2=0x7f28900b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:53.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.064+0000 7f288ea00640 1 --2- 172.21.15.161:0/2170657148 >> v2:172.21.15.179:3300/0 conn(0x7f289006c9b0 0x7f28900b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:53.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.064+0000 7f288ea00640 1 -- 172.21.15.161:0/2170657148 >> v1:172.21.15.161:6789/0 conn(0x7f28901444a0 legacy=0x7f289006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:53.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.064+0000 7f288ea00640 1 -- 172.21.15.161:0/2170657148 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289015f850 con 0x7f2890143c40 2024-05-22T21:22:53.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.064+0000 7f288ea00640 1 --2- 172.21.15.161:0/2170657148 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2890143c40 0x7f289015b600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2880009d90 tx=0x7f288001c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78dcc527e8b639c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:53.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.064+0000 7f288e000640 1 -- 172.21.15.161:0/2170657148 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2878001680 con 0x7f2890143c40 2024-05-22T21:22:53.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.064+0000 7f288e000640 1 -- 172.21.15.161:0/2170657148 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28800a55f0 con 0x7f2890143c40 2024-05-22T21:22:53.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.064+0000 7f288e000640 1 -- 172.21.15.161:0/2170657148 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28800a5980 con 0x7f2890143c40 2024-05-22T21:22:53.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.064+0000 7f2895a00640 1 -- 172.21.15.161:0/2170657148 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2890143c40 msgr2=0x7f289015b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:53.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.064+0000 7f2895a00640 1 --2- 172.21.15.161:0/2170657148 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2890143c40 0x7f289015b600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2880009d90 tx=0x7f288001c890 comp rx=0 tx=0).stop 2024-05-22T21:22:53.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2895a00640 1 -- 172.21.15.161:0/2170657148 shutdown_connections 2024-05-22T21:22:53.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2895a00640 1 --2- 172.21.15.161:0/2170657148 >> v2:172.21.15.179:3300/0 conn(0x7f289006c9b0 0x7f28900b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:53.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2895a00640 1 --2- 172.21.15.161:0/2170657148 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2890143c40 0x7f289015b600 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:53.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2895a00640 1 -- 172.21.15.161:0/2170657148 >> 172.21.15.161:0/2170657148 conn(0x7f289006a2d0 msgr2=0x7f289006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:53.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2895a00640 1 -- 172.21.15.161:0/2170657148 shutdown_connections 2024-05-22T21:22:53.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2895a00640 1 -- 172.21.15.161:0/2170657148 wait complete. 2024-05-22T21:22:53.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2895a00640 1 Processor -- start 2024-05-22T21:22:53.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2895a00640 1 -- start start 2024-05-22T21:22:53.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2895a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f289006c9b0 0x7f28901397a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:53.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2895a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2890143c40 0x7f2890137c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:53.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2895a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f28901444a0 0x7f28901419f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:53.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2895a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2890158970 con 0x7f289006c9b0 2024-05-22T21:22:53.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2895a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2890157df0 con 0x7f2890143c40 2024-05-22T21:22:53.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2895a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28901580d0 con 0x7f28901444a0 2024-05-22T21:22:53.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f288ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f289006c9b0 0x7f28901397a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:53.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f288ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f289006c9b0 0x7f28901397a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49882/0 (socket says 172.21.15.161:49882) 2024-05-22T21:22:53.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f288ea00640 1 -- 172.21.15.161:0/2962598603 learned_addr learned my addr 172.21.15.161:0/2962598603 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:53.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f288ea00640 1 -- 172.21.15.161:0/2962598603 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f28901444a0 msgr2=0x7f28901419f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:53.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f288ea00640 1 --2- 172.21.15.161:0/2962598603 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f28901444a0 0x7f28901419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:53.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f288ea00640 1 -- 172.21.15.161:0/2962598603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2890143c40 msgr2=0x7f2890137c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:53.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f288ea00640 1 --2- 172.21.15.161:0/2962598603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2890143c40 0x7f2890137c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:53.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f288ea00640 1 -- 172.21.15.161:0/2962598603 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2880009a70 con 0x7f289006c9b0 2024-05-22T21:22:53.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f288ea00640 1 --2- 172.21.15.161:0/2962598603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f289006c9b0 0x7f28901397a0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f28800b0040 tx=0x7f2880019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:53.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2887400640 1 -- 172.21.15.161:0/2962598603 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28800ad850 con 0x7f289006c9b0 2024-05-22T21:22:53.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2887400640 1 -- 172.21.15.161:0/2962598603 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28800ad9b0 con 0x7f289006c9b0 2024-05-22T21:22:53.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2887400640 1 -- 172.21.15.161:0/2962598603 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28800add60 con 0x7f289006c9b0 2024-05-22T21:22:53.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.068+0000 7f2895a00640 1 -- 172.21.15.161:0/2962598603 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28901583b0 con 0x7f289006c9b0 2024-05-22T21:22:53.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.072+0000 7f2895a00640 1 -- 172.21.15.161:0/2962598603 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2890139440 con 0x7f289006c9b0 2024-05-22T21:22:53.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.072+0000 7f2895a00640 1 -- 172.21.15.161:0/2962598603 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2890067220 con 0x7f289006c9b0 2024-05-22T21:22:53.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.072+0000 7f2887400640 1 -- 172.21.15.161:0/2962598603 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f28800b4050 con 0x7f289006c9b0 2024-05-22T21:22:53.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.072+0000 7f2887400640 1 --2- 172.21.15.161:0/2962598603 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f286c066690 0x7f286c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:53.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.072+0000 7f2887400640 1 -- 172.21.15.161:0/2962598603 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f288011eea0 con 0x7f289006c9b0 2024-05-22T21:22:53.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.072+0000 7f2886000640 1 --2- 172.21.15.161:0/2962598603 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f286c066690 0x7f286c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:53.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.072+0000 7f2887400640 1 -- 172.21.15.161:0/2962598603 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=164}) v3 -- 0x7f286c06e390 con 0x7f289006c9b0 2024-05-22T21:22:53.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.072+0000 7f2886000640 1 --2- 172.21.15.161:0/2962598603 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f286c066690 0x7f286c068b50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2878001ec0 tx=0x7f2878001990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:53.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.076+0000 7f2887400640 1 -- 172.21.15.161:0/2962598603 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f28800b2070 con 0x7f289006c9b0 2024-05-22T21:22:53.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.268+0000 7f2895a00640 1 -- 172.21.15.161:0/2962598603 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f28900ceaa0 con 0x7f289006c9b0 2024-05-22T21:22:53.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.904+0000 7f2887400640 1 -- 172.21.15.161:0/2962598603 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v72) v1 ==== 104+0+0 (secure 0 0 0) 0x7f28800ec050 con 0x7f289006c9b0 2024-05-22T21:22:53.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.908+0000 7f2895a00640 1 -- 172.21.15.161:0/2962598603 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f286c066690 msgr2=0x7f286c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:53.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.908+0000 7f2895a00640 1 --2- 172.21.15.161:0/2962598603 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f286c066690 0x7f286c068b50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2878001ec0 tx=0x7f2878001990 comp rx=0 tx=0).stop 2024-05-22T21:22:53.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.908+0000 7f2895a00640 1 -- 172.21.15.161:0/2962598603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f289006c9b0 msgr2=0x7f28901397a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:53.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.908+0000 7f2895a00640 1 --2- 172.21.15.161:0/2962598603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f289006c9b0 0x7f28901397a0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f28800b0040 tx=0x7f2880019630 comp rx=0 tx=0).stop 2024-05-22T21:22:53.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.908+0000 7f2895a00640 1 -- 172.21.15.161:0/2962598603 shutdown_connections 2024-05-22T21:22:53.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.908+0000 7f2895a00640 1 --2- 172.21.15.161:0/2962598603 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f286c066690 0x7f286c068b50 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:53.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.908+0000 7f2895a00640 1 --2- 172.21.15.161:0/2962598603 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f28901444a0 0x7f28901419f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:53.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.908+0000 7f2895a00640 1 --2- 172.21.15.161:0/2962598603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2890143c40 0x7f2890137c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:53.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.908+0000 7f2895a00640 1 --2- 172.21.15.161:0/2962598603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f289006c9b0 0x7f28901397a0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:53.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.908+0000 7f2895a00640 1 -- 172.21.15.161:0/2962598603 >> 172.21.15.161:0/2962598603 conn(0x7f289006a2d0 msgr2=0x7f2890140790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:53.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.908+0000 7f2895a00640 1 -- 172.21.15.161:0/2962598603 shutdown_connections 2024-05-22T21:22:53.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:53.908+0000 7f2895a00640 1 -- 172.21.15.161:0/2962598603 wait complete. 2024-05-22T21:22:53.930 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-22T21:22:54.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.072+0000 7f6b15000640 1 Processor -- start 2024-05-22T21:22:54.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.072+0000 7f6b15000640 1 -- start start 2024-05-22T21:22:54.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.072+0000 7f6b15000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6b1013cfe0 0x7f6b1015dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:54.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.072+0000 7f6b15000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b1013d560 0x7f6b1015e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:54.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.072+0000 7f6b15000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b100ce830 con 0x7f6b100cdb00 2024-05-22T21:22:54.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.072+0000 7f6b15000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b100ce650 con 0x7f6b1013cfe0 2024-05-22T21:22:54.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.072+0000 7f6b15000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b10140560 con 0x7f6b1013d560 2024-05-22T21:22:54.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.072+0000 7f6b0de00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6b1013cfe0 0x7f6b1015dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:54.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.072+0000 7f6b0de00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6b1013cfe0 0x7f6b1015dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49888/0 (socket says 172.21.15.161:49888) 2024-05-22T21:22:54.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.072+0000 7f6b0de00640 1 -- 172.21.15.161:0/675812607 learned_addr learned my addr 172.21.15.161:0/675812607 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:54.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.076+0000 7f6b0d400640 1 --2- 172.21.15.161:0/675812607 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b1013d560 0x7f6b1015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:54.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.076+0000 7f6b0de00640 1 -- 172.21.15.161:0/675812607 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b1013d560 msgr2=0x7f6b1015e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:54.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.076+0000 7f6b0de00640 1 --2- 172.21.15.161:0/675812607 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b1013d560 0x7f6b1015e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:54.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.076+0000 7f6b0de00640 1 -- 172.21.15.161:0/675812607 >> v1:172.21.15.179:6789/0 conn(0x7f6b100cdb00 legacy=0x7f6b100cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:54.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.076+0000 7f6b0de00640 1 -- 172.21.15.161:0/675812607 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b100ceb10 con 0x7f6b1013cfe0 2024-05-22T21:22:54.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.076+0000 7f6b0de00640 1 --2- 172.21.15.161:0/675812607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b1013cfe0 0x7f6b1015dac0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009870 tx=0x7f6b0001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558f12f6f2113ec2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:54.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.076+0000 7f6b0ca00640 1 -- 172.21.15.161:0/675812607 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b000a6040 con 0x7f6b1013cfe0 2024-05-22T21:22:54.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.076+0000 7f6b0ca00640 1 -- 172.21.15.161:0/675812607 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b000aa600 con 0x7f6b1013cfe0 2024-05-22T21:22:54.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.076+0000 7f6b0ca00640 1 -- 172.21.15.161:0/675812607 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b000aa990 con 0x7f6b1013cfe0 2024-05-22T21:22:54.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b15000640 1 -- 172.21.15.161:0/675812607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b1013cfe0 msgr2=0x7f6b1015dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:54.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b15000640 1 --2- 172.21.15.161:0/675812607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b1013cfe0 0x7f6b1015dac0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009870 tx=0x7f6b0001c4a0 comp rx=0 tx=0).stop 2024-05-22T21:22:54.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b15000640 1 -- 172.21.15.161:0/675812607 shutdown_connections 2024-05-22T21:22:54.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b15000640 1 --2- 172.21.15.161:0/675812607 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b1013d560 0x7f6b1015e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:54.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b15000640 1 --2- 172.21.15.161:0/675812607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b1013cfe0 0x7f6b1015dac0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:54.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b15000640 1 -- 172.21.15.161:0/675812607 >> 172.21.15.161:0/675812607 conn(0x7f6b1006a0a0 msgr2=0x7f6b1006a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:54.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b15000640 1 -- 172.21.15.161:0/675812607 shutdown_connections 2024-05-22T21:22:54.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b15000640 1 -- 172.21.15.161:0/675812607 wait complete. 2024-05-22T21:22:54.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b15000640 1 Processor -- start 2024-05-22T21:22:54.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b15000640 1 -- start start 2024-05-22T21:22:54.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b15000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b100cdb00 0x7f6b101537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:54.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b15000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b1013cfe0 0x7f6b101f9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:54.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b15000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b1013d560 0x7f6b101f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:54.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b15000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b10154010 con 0x7f6b1013d560 2024-05-22T21:22:54.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b15000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b101542f0 con 0x7f6b1013cfe0 2024-05-22T21:22:54.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b15000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b10141790 con 0x7f6b100cdb00 2024-05-22T21:22:54.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b0e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b1013d560 0x7f6b101f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:54.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b0e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b1013d560 0x7f6b101f4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49890/0 (socket says 172.21.15.161:49890) 2024-05-22T21:22:54.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b0e800640 1 -- 172.21.15.161:0/165841270 learned_addr learned my addr 172.21.15.161:0/165841270 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:54.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b0de00640 1 --2- 172.21.15.161:0/165841270 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b100cdb00 0x7f6b101537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:54.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b0d400640 1 --2- 172.21.15.161:0/165841270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b1013cfe0 0x7f6b101f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:54.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b0e800640 1 -- 172.21.15.161:0/165841270 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b100cdb00 msgr2=0x7f6b101537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:54.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b0e800640 1 --2- 172.21.15.161:0/165841270 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b100cdb00 0x7f6b101537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:54.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b0e800640 1 -- 172.21.15.161:0/165841270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b1013cfe0 msgr2=0x7f6b101f9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:54.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b0e800640 1 --2- 172.21.15.161:0/165841270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b1013cfe0 0x7f6b101f9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:54.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b0e800640 1 -- 172.21.15.161:0/165841270 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b000094d0 con 0x7f6b1013d560 2024-05-22T21:22:54.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.080+0000 7f6b0e800640 1 --2- 172.21.15.161:0/165841270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b1013d560 0x7f6b101f4c00 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800c770 tx=0x7f6b0800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:54.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.084+0000 7f6b06000640 1 -- 172.21.15.161:0/165841270 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08003b80 con 0x7f6b1013d560 2024-05-22T21:22:54.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.084+0000 7f6b06000640 1 -- 172.21.15.161:0/165841270 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b08003ce0 con 0x7f6b1013d560 2024-05-22T21:22:54.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.084+0000 7f6b06000640 1 -- 172.21.15.161:0/165841270 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800aa90 con 0x7f6b1013d560 2024-05-22T21:22:54.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.084+0000 7f6b15000640 1 -- 172.21.15.161:0/165841270 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b10141a70 con 0x7f6b1013d560 2024-05-22T21:22:54.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.084+0000 7f6b15000640 1 -- 172.21.15.161:0/165841270 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b10153d00 con 0x7f6b1013d560 2024-05-22T21:22:54.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.084+0000 7f6b15000640 1 -- 172.21.15.161:0/165841270 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b10141c90 con 0x7f6b1013d560 2024-05-22T21:22:54.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.084+0000 7f6b06000640 1 -- 172.21.15.161:0/165841270 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6b0800abf0 con 0x7f6b1013d560 2024-05-22T21:22:54.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.084+0000 7f6b06000640 1 --2- 172.21.15.161:0/165841270 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6af0066690 0x7f6af0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:54.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.084+0000 7f6b06000640 1 -- 172.21.15.161:0/165841270 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6b08083fc0 con 0x7f6b1013d560 2024-05-22T21:22:54.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.084+0000 7f6b06000640 1 -- 172.21.15.161:0/165841270 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=164}) v3 -- 0x7f6af006e390 con 0x7f6b1013d560 2024-05-22T21:22:54.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.084+0000 7f6b0de00640 1 --2- 172.21.15.161:0/165841270 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6af0066690 0x7f6af0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:54.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.084+0000 7f6b0de00640 1 --2- 172.21.15.161:0/165841270 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6af0066690 0x7f6af0068b50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6b000ae000 tx=0x7f6b000b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:54.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.088+0000 7f6b06000640 1 -- 172.21.15.161:0/165841270 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6b080512b0 con 0x7f6b1013d560 2024-05-22T21:22:54.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.244+0000 7f6b15000640 1 -- 172.21.15.161:0/165841270 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6b10067220 con 0x7f6b1013d560 2024-05-22T21:22:54.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.916+0000 7f6b06000640 1 -- 172.21.15.161:0/165841270 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(164..164 src has 1..164) v4 ==== 296+0+0 (secure 0 0 0) 0x7f6b08054f70 con 0x7f6b1013d560 2024-05-22T21:22:54.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.916+0000 7f6b06000640 1 -- 172.21.15.161:0/165841270 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=165}) v3 -- 0x7f6af006c7c0 con 0x7f6b1013d560 2024-05-22T21:22:54.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.916+0000 7f6b06000640 1 -- 172.21.15.161:0/165841270 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v164) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6b080548d0 con 0x7f6b1013d560 2024-05-22T21:22:54.919 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' removed 2024-05-22T21:22:54.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.924+0000 7f6b15000640 1 -- 172.21.15.161:0/165841270 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6af0066690 msgr2=0x7f6af0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:54.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.924+0000 7f6b15000640 1 --2- 172.21.15.161:0/165841270 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6af0066690 0x7f6af0068b50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6b000ae000 tx=0x7f6b000b5040 comp rx=0 tx=0).stop 2024-05-22T21:22:54.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.924+0000 7f6b15000640 1 -- 172.21.15.161:0/165841270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b1013d560 msgr2=0x7f6b101f4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:54.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.924+0000 7f6b15000640 1 --2- 172.21.15.161:0/165841270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b1013d560 0x7f6b101f4c00 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800c770 tx=0x7f6b0800cc30 comp rx=0 tx=0).stop 2024-05-22T21:22:54.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.924+0000 7f6b15000640 1 -- 172.21.15.161:0/165841270 shutdown_connections 2024-05-22T21:22:54.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.924+0000 7f6b15000640 1 --2- 172.21.15.161:0/165841270 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6af0066690 0x7f6af0068b50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:54.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.924+0000 7f6b15000640 1 --2- 172.21.15.161:0/165841270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b1013d560 0x7f6b101f4c00 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:54.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.924+0000 7f6b15000640 1 --2- 172.21.15.161:0/165841270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b1013cfe0 0x7f6b101f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:54.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.924+0000 7f6b15000640 1 --2- 172.21.15.161:0/165841270 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b100cdb00 0x7f6b101537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:54.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.924+0000 7f6b15000640 1 -- 172.21.15.161:0/165841270 >> 172.21.15.161:0/165841270 conn(0x7f6b1006a0a0 msgr2=0x7f6b101642f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:54.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.924+0000 7f6b15000640 1 -- 172.21.15.161:0/165841270 shutdown_connections 2024-05-22T21:22:54.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:54.924+0000 7f6b15000640 1 -- 172.21.15.161:0/165841270 wait complete. 2024-05-22T21:22:54.938 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:22:54.942 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:22:54.947 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-22T21:22:55.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.164+0000 7f5c7be00640 1 Processor -- start 2024-05-22T21:22:55.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.164+0000 7f5c7be00640 1 -- start start 2024-05-22T21:22:55.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.168+0000 7f5c7be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5c74143630 0x7f5c7406d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:55.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.168+0000 7f5c7be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c74143a10 0x7f5c741619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:55.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.168+0000 7f5c7be00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c740ad9f0 con 0x7f5c740ad560 2024-05-22T21:22:55.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.168+0000 7f5c7be00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7406d290 con 0x7f5c74143630 2024-05-22T21:22:55.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.168+0000 7f5c7be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740adf60 con 0x7f5c74143a10 2024-05-22T21:22:55.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.168+0000 7f5c79400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5c74143630 0x7f5c7406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:55.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.168+0000 7f5c79400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5c74143630 0x7f5c7406d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49904/0 (socket says 172.21.15.161:49904) 2024-05-22T21:22:55.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.168+0000 7f5c79400640 1 -- 172.21.15.161:0/3418273304 learned_addr learned my addr 172.21.15.161:0/3418273304 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:55.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.168+0000 7f5c78a00640 1 --2- 172.21.15.161:0/3418273304 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c74143a10 0x7f5c741619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:55.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.168+0000 7f5c79400640 1 -- 172.21.15.161:0/3418273304 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c74143a10 msgr2=0x7f5c741619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:55.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.168+0000 7f5c79400640 1 --2- 172.21.15.161:0/3418273304 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c74143a10 0x7f5c741619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:55.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.168+0000 7f5c79400640 1 -- 172.21.15.161:0/3418273304 >> v1:172.21.15.179:6789/0 conn(0x7f5c740ad560 legacy=0x7f5c74160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:55.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.168+0000 7f5c79400640 1 -- 172.21.15.161:0/3418273304 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7406cc40 con 0x7f5c74143630 2024-05-22T21:22:55.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.172+0000 7f5c79400640 1 --2- 172.21.15.161:0/3418273304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c74143630 0x7f5c7406d720 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009e50 tx=0x7f5c6801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf59a3d6cee6a69 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:55.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.172+0000 7f5c73e00640 1 -- 172.21.15.161:0/3418273304 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c680a57a0 con 0x7f5c74143630 2024-05-22T21:22:55.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.172+0000 7f5c73e00640 1 -- 172.21.15.161:0/3418273304 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c680a5900 con 0x7f5c74143630 2024-05-22T21:22:55.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.172+0000 7f5c73e00640 1 -- 172.21.15.161:0/3418273304 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c680a5c90 con 0x7f5c74143630 2024-05-22T21:22:55.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.172+0000 7f5c7be00640 1 -- 172.21.15.161:0/3418273304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c74143630 msgr2=0x7f5c7406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:55.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.172+0000 7f5c7be00640 1 --2- 172.21.15.161:0/3418273304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c74143630 0x7f5c7406d720 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009e50 tx=0x7f5c6801c9c0 comp rx=0 tx=0).stop 2024-05-22T21:22:55.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.172+0000 7f5c7be00640 1 -- 172.21.15.161:0/3418273304 shutdown_connections 2024-05-22T21:22:55.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.172+0000 7f5c7be00640 1 --2- 172.21.15.161:0/3418273304 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c74143a10 0x7f5c741619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:55.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.172+0000 7f5c7be00640 1 --2- 172.21.15.161:0/3418273304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c74143630 0x7f5c7406d720 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:55.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.172+0000 7f5c7be00640 1 -- 172.21.15.161:0/3418273304 >> 172.21.15.161:0/3418273304 conn(0x7f5c7406a0a0 msgr2=0x7f5c7406a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:55.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.172+0000 7f5c7be00640 1 -- 172.21.15.161:0/3418273304 shutdown_connections 2024-05-22T21:22:55.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.172+0000 7f5c7be00640 1 -- 172.21.15.161:0/3418273304 wait complete. 2024-05-22T21:22:55.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.172+0000 7f5c7be00640 1 Processor -- start 2024-05-22T21:22:55.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.172+0000 7f5c7be00640 1 -- start start 2024-05-22T21:22:55.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c7be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c740ad560 0x7f5c741fa130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:55.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c7be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c74143630 0x7f5c7415c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:55.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c7be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c74143a10 0x7f5c7415e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:55.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c7be00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c741fa670 con 0x7f5c740ad560 2024-05-22T21:22:55.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c7be00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7415cb50 con 0x7f5c74143630 2024-05-22T21:22:55.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c7be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7415d050 con 0x7f5c74143a10 2024-05-22T21:22:55.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c79400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c740ad560 0x7f5c741fa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:55.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c79400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c740ad560 0x7f5c741fa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49910/0 (socket says 172.21.15.161:49910) 2024-05-22T21:22:55.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c79400640 1 -- 172.21.15.161:0/327141001 learned_addr learned my addr 172.21.15.161:0/327141001 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:55.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c79e00640 1 --2- 172.21.15.161:0/327141001 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c74143a10 0x7f5c7415e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:55.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c78a00640 1 --2- 172.21.15.161:0/327141001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c74143630 0x7f5c7415c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:55.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c79400640 1 -- 172.21.15.161:0/327141001 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c74143a10 msgr2=0x7f5c7415e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:55.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c79400640 1 --2- 172.21.15.161:0/327141001 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c74143a10 0x7f5c7415e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:55.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c79400640 1 -- 172.21.15.161:0/327141001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c74143630 msgr2=0x7f5c7415c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:55.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c79400640 1 --2- 172.21.15.161:0/327141001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c74143630 0x7f5c7415c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:55.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c79400640 1 -- 172.21.15.161:0/327141001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68009b30 con 0x7f5c740ad560 2024-05-22T21:22:55.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c79e00640 1 --2- 172.21.15.161:0/327141001 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c74143a10 0x7f5c7415e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:22:55.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c78a00640 1 --2- 172.21.15.161:0/327141001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c74143630 0x7f5c7415c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:22:55.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c79400640 1 --2- 172.21.15.161:0/327141001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c740ad560 0x7f5c741fa130 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f5c6801b070 tx=0x7f5c68019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:55.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c71600640 1 -- 172.21.15.161:0/327141001 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c680bd070 con 0x7f5c740ad560 2024-05-22T21:22:55.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c71600640 1 -- 172.21.15.161:0/327141001 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c680b8470 con 0x7f5c740ad560 2024-05-22T21:22:55.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c7be00640 1 -- 172.21.15.161:0/327141001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7413ef90 con 0x7f5c740ad560 2024-05-22T21:22:55.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c71600640 1 -- 172.21.15.161:0/327141001 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c680b8820 con 0x7f5c740ad560 2024-05-22T21:22:55.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c7be00640 1 -- 172.21.15.161:0/327141001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7415d940 con 0x7f5c740ad560 2024-05-22T21:22:55.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c7be00640 1 -- 172.21.15.161:0/327141001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c74067220 con 0x7f5c740ad560 2024-05-22T21:22:55.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c71600640 1 -- 172.21.15.161:0/327141001 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5c680b8980 con 0x7f5c740ad560 2024-05-22T21:22:55.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c71600640 1 --2- 172.21.15.161:0/327141001 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5c54066690 0x7f5c54068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:55.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.176+0000 7f5c71600640 1 -- 172.21.15.161:0/327141001 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f5c6811eeb0 con 0x7f5c740ad560 2024-05-22T21:22:55.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.180+0000 7f5c78a00640 1 --2- 172.21.15.161:0/327141001 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5c54066690 0x7f5c54068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:55.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.180+0000 7f5c71600640 1 -- 172.21.15.161:0/327141001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=165}) v3 -- 0x7f5c5406d520 con 0x7f5c740ad560 2024-05-22T21:22:55.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.180+0000 7f5c78a00640 1 --2- 172.21.15.161:0/327141001 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5c54066690 0x7f5c54068b50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5c7413dcb0 tx=0x7f5c60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:55.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.180+0000 7f5c71600640 1 -- 172.21.15.161:0/327141001 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f5c680ec2a0 con 0x7f5c740ad560 2024-05-22T21:22:55.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.288+0000 7f5c71600640 1 -- 172.21.15.161:0/327141001 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(165..165 src has 1..165) v4 ==== 679+0+0 (secure 0 0 0) 0x7f5c680f0660 con 0x7f5c740ad560 2024-05-22T21:22:55.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.288+0000 7f5c71600640 1 -- 172.21.15.161:0/327141001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=166}) v3 -- 0x7f5c5406df40 con 0x7f5c740ad560 2024-05-22T21:22:55.338 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:55.340+0000 7f5c7be00640 1 -- 172.21.15.161:0/327141001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5c74071b10 con 0x7f5c740ad560 2024-05-22T21:22:56.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.292+0000 7f5c71600640 1 -- 172.21.15.161:0/327141001 <== mon.0 v2:172.21.15.161:3300/0 8 ==== osd_map(166..166 src has 1..166) v4 ==== 330+0+0 (secure 0 0 0) 0x7f5c68120070 con 0x7f5c740ad560 2024-05-22T21:22:56.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.296+0000 7f5c71600640 1 -- 172.21.15.161:0/327141001 <== mon.0 v2:172.21.15.161:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v166) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5c680ef8c0 con 0x7f5c740ad560 2024-05-22T21:22:56.296 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' removed 2024-05-22T21:22:56.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.300+0000 7f5c7be00640 1 -- 172.21.15.161:0/327141001 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5c54066690 msgr2=0x7f5c54068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:56.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.300+0000 7f5c7be00640 1 --2- 172.21.15.161:0/327141001 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5c54066690 0x7f5c54068b50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5c7413dcb0 tx=0x7f5c60008040 comp rx=0 tx=0).stop 2024-05-22T21:22:56.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.300+0000 7f5c7be00640 1 -- 172.21.15.161:0/327141001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c740ad560 msgr2=0x7f5c741fa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:56.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.300+0000 7f5c7be00640 1 --2- 172.21.15.161:0/327141001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c740ad560 0x7f5c741fa130 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f5c6801b070 tx=0x7f5c68019760 comp rx=0 tx=0).stop 2024-05-22T21:22:56.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.300+0000 7f5c7be00640 1 -- 172.21.15.161:0/327141001 shutdown_connections 2024-05-22T21:22:56.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.300+0000 7f5c7be00640 1 --2- 172.21.15.161:0/327141001 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5c54066690 0x7f5c54068b50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.300+0000 7f5c7be00640 1 --2- 172.21.15.161:0/327141001 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c74143a10 0x7f5c7415e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.300+0000 7f5c7be00640 1 --2- 172.21.15.161:0/327141001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c74143630 0x7f5c7415c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.300+0000 7f5c7be00640 1 --2- 172.21.15.161:0/327141001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c740ad560 0x7f5c741fa130 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.300+0000 7f5c7be00640 1 -- 172.21.15.161:0/327141001 >> 172.21.15.161:0/327141001 conn(0x7f5c7406a0a0 msgr2=0x7f5c7406c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:56.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.300+0000 7f5c7be00640 1 -- 172.21.15.161:0/327141001 shutdown_connections 2024-05-22T21:22:56.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.300+0000 7f5c7be00640 1 -- 172.21.15.161:0/327141001 wait complete. 2024-05-22T21:22:56.325 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-22T21:22:56.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.464+0000 7faa8e000640 1 Processor -- start 2024-05-22T21:22:56.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.464+0000 7faa8e000640 1 -- start start 2024-05-22T21:22:56.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.468+0000 7faa8e000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7faa8813cfc0 0x7faa881402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:56.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.468+0000 7faa8e000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7faa8815af60 0x7faa8813f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:56.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.468+0000 7faa8e000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa8813fc80 0x7faa8815dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:56.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.468+0000 7faa8e000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa881400b0 con 0x7faa8813cfc0 2024-05-22T21:22:56.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.468+0000 7faa8e000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa8813d3a0 con 0x7faa8815af60 2024-05-22T21:22:56.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.468+0000 7faa8e000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa880cd920 con 0x7faa8813fc80 2024-05-22T21:22:56.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.468+0000 7faa87400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7faa8813cfc0 0x7faa881402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:56.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.468+0000 7faa87400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7faa8813cfc0 0x7faa881402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:40950/0 (socket says 172.21.15.161:40950) 2024-05-22T21:22:56.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.468+0000 7faa87400640 1 -- 172.21.15.161:0/3840673348 learned_addr learned my addr 172.21.15.161:0/3840673348 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:56.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.468+0000 7faa7ea00640 1 --2- 172.21.15.161:0/3840673348 >> v2:172.21.15.179:3300/0 conn(0x7faa8815af60 0x7faa8813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:56.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.468+0000 7faa87e00640 1 --2- 172.21.15.161:0/3840673348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa8813fc80 0x7faa8815dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:56.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.468+0000 7faa87400640 1 -- 172.21.15.161:0/3840673348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa8813fc80 msgr2=0x7faa8815dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:56.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.468+0000 7faa87400640 1 --2- 172.21.15.161:0/3840673348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa8813fc80 0x7faa8815dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.468+0000 7faa87400640 1 -- 172.21.15.161:0/3840673348 >> v2:172.21.15.179:3300/0 conn(0x7faa8815af60 msgr2=0x7faa8813f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:56.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.468+0000 7faa87400640 1 --2- 172.21.15.161:0/3840673348 >> v2:172.21.15.179:3300/0 conn(0x7faa8815af60 0x7faa8813f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.468+0000 7faa87400640 1 -- 172.21.15.161:0/3840673348 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa880cdc00 con 0x7faa8813cfc0 2024-05-22T21:22:56.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa87400640 1 --2- 172.21.15.161:0/3840673348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa8813cfc0 0x7faa881402e0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7faa8806e1d0 tx=0x7faa7801c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95dd86ad2df61bec server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:56.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa86a00640 1 -- 172.21.15.161:0/3840673348 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa780ab600 con 0x7faa8813cfc0 2024-05-22T21:22:56.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa86a00640 1 -- 172.21.15.161:0/3840673348 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa780a7050 con 0x7faa8813cfc0 2024-05-22T21:22:56.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa86a00640 1 -- 172.21.15.161:0/3840673348 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa780ab980 con 0x7faa8813cfc0 2024-05-22T21:22:56.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 -- 172.21.15.161:0/3840673348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa8813cfc0 msgr2=0x7faa881402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:56.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 --2- 172.21.15.161:0/3840673348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa8813cfc0 0x7faa881402e0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7faa8806e1d0 tx=0x7faa7801c1a0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 -- 172.21.15.161:0/3840673348 shutdown_connections 2024-05-22T21:22:56.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 --2- 172.21.15.161:0/3840673348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa8813fc80 0x7faa8815dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 --2- 172.21.15.161:0/3840673348 >> v2:172.21.15.179:3300/0 conn(0x7faa8815af60 0x7faa8813f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 --2- 172.21.15.161:0/3840673348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa8813cfc0 0x7faa881402e0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 -- 172.21.15.161:0/3840673348 >> 172.21.15.161:0/3840673348 conn(0x7faa8806a2d0 msgr2=0x7faa8806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:56.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 -- 172.21.15.161:0/3840673348 shutdown_connections 2024-05-22T21:22:56.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 -- 172.21.15.161:0/3840673348 wait complete. 2024-05-22T21:22:56.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 Processor -- start 2024-05-22T21:22:56.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 -- start start 2024-05-22T21:22:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa8813cfc0 0x7faa881578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa8813fc80 0x7faa88140250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa8815af60 0x7faa88139680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa881418b0 con 0x7faa8815af60 2024-05-22T21:22:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa88157e00 con 0x7faa8813cfc0 2024-05-22T21:22:56.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa8e000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa88140c30 con 0x7faa8813fc80 2024-05-22T21:22:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.472+0000 7faa87e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa8815af60 0x7faa88139680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa87e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa8815af60 0x7faa88139680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:40952/0 (socket says 172.21.15.161:40952) 2024-05-22T21:22:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa87e00640 1 -- 172.21.15.161:0/2683835794 learned_addr learned my addr 172.21.15.161:0/2683835794 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa7ea00640 1 --2- 172.21.15.161:0/2683835794 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa8813fc80 0x7faa88140250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa87400640 1 --2- 172.21.15.161:0/2683835794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa8813cfc0 0x7faa881578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa87e00640 1 -- 172.21.15.161:0/2683835794 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa8813fc80 msgr2=0x7faa88140250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa87e00640 1 --2- 172.21.15.161:0/2683835794 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa8813fc80 0x7faa88140250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa87e00640 1 -- 172.21.15.161:0/2683835794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa8813cfc0 msgr2=0x7faa881578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa87e00640 1 --2- 172.21.15.161:0/2683835794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa8813cfc0 0x7faa881578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa87e00640 1 -- 172.21.15.161:0/2683835794 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa780094d0 con 0x7faa8815af60 2024-05-22T21:22:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa87e00640 1 --2- 172.21.15.161:0/2683835794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa8815af60 0x7faa88139680 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7faa80007e60 tx=0x7faa8000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa7fe00640 1 -- 172.21.15.161:0/2683835794 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80014420 con 0x7faa8815af60 2024-05-22T21:22:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa7fe00640 1 -- 172.21.15.161:0/2683835794 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa800145a0 con 0x7faa8815af60 2024-05-22T21:22:56.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa7fe00640 1 -- 172.21.15.161:0/2683835794 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80014950 con 0x7faa8815af60 2024-05-22T21:22:56.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa8e000640 1 -- 172.21.15.161:0/2683835794 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8815b4f0 con 0x7faa8815af60 2024-05-22T21:22:56.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa8e000640 1 -- 172.21.15.161:0/2683835794 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa881411f0 con 0x7faa8815af60 2024-05-22T21:22:56.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa8e000640 1 -- 172.21.15.161:0/2683835794 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa881419f0 con 0x7faa8815af60 2024-05-22T21:22:56.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa7fe00640 1 -- 172.21.15.161:0/2683835794 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faa80014ab0 con 0x7faa8815af60 2024-05-22T21:22:56.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.476+0000 7faa7fe00640 1 --2- 172.21.15.161:0/2683835794 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faa64066690 0x7faa64068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:56.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.480+0000 7faa7fe00640 1 -- 172.21.15.161:0/2683835794 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 4943+0+0 (secure 0 0 0) 0x7faa80085360 con 0x7faa8815af60 2024-05-22T21:22:56.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.480+0000 7faa87400640 1 --2- 172.21.15.161:0/2683835794 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faa64066690 0x7faa64068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:56.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.480+0000 7faa87400640 1 --2- 172.21.15.161:0/2683835794 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faa64066690 0x7faa64068b50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7faa780ad000 tx=0x7faa78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:56.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.480+0000 7faa7fe00640 1 -- 172.21.15.161:0/2683835794 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7faa80052af0 con 0x7faa8815af60 2024-05-22T21:22:56.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.636+0000 7faa8e000640 1 -- 172.21.15.161:0/2683835794 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7faa881419f0 con 0x7faa8815af60 2024-05-22T21:22:56.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.640+0000 7faa7fe00640 1 -- 172.21.15.161:0/2683835794 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7faa881419f0 con 0x7faa8815af60 2024-05-22T21:22:56.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.644+0000 7faa8e000640 1 -- 172.21.15.161:0/2683835794 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faa64066690 msgr2=0x7faa64068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:56.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.644+0000 7faa8e000640 1 --2- 172.21.15.161:0/2683835794 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faa64066690 0x7faa64068b50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7faa780ad000 tx=0x7faa78008040 comp rx=0 tx=0).stop 2024-05-22T21:22:56.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.644+0000 7faa8e000640 1 -- 172.21.15.161:0/2683835794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa8815af60 msgr2=0x7faa88139680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:56.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.644+0000 7faa8e000640 1 --2- 172.21.15.161:0/2683835794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa8815af60 0x7faa88139680 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7faa80007e60 tx=0x7faa8000e4b0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.644+0000 7faa8e000640 1 -- 172.21.15.161:0/2683835794 shutdown_connections 2024-05-22T21:22:56.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.644+0000 7faa8e000640 1 --2- 172.21.15.161:0/2683835794 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faa64066690 0x7faa64068b50 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.644+0000 7faa8e000640 1 --2- 172.21.15.161:0/2683835794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa8815af60 0x7faa88139680 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.644+0000 7faa8e000640 1 --2- 172.21.15.161:0/2683835794 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa8813fc80 0x7faa88140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.644+0000 7faa8e000640 1 --2- 172.21.15.161:0/2683835794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa8813cfc0 0x7faa881578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.644+0000 7faa8e000640 1 -- 172.21.15.161:0/2683835794 >> 172.21.15.161:0/2683835794 conn(0x7faa8806a2d0 msgr2=0x7faa8806a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:56.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.644+0000 7faa8e000640 1 -- 172.21.15.161:0/2683835794 shutdown_connections 2024-05-22T21:22:56.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.644+0000 7faa8e000640 1 -- 172.21.15.161:0/2683835794 wait complete. 2024-05-22T21:22:56.664 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2024-05-22T21:22:56.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe42ba00640 1 Processor -- start 2024-05-22T21:22:56.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe42ba00640 1 -- start start 2024-05-22T21:22:56.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe42ba00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe42415dd50 0x7fe42406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:56.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe42ba00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe42415e130 0x7fe42415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:56.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe42ba00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe424071670 con 0x7fe42415d410 2024-05-22T21:22:56.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe42ba00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42415d8a0 con 0x7fe42415e130 2024-05-22T21:22:56.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe42ba00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42406de40 con 0x7fe42415dd50 2024-05-22T21:22:56.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe423e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe42415e130 0x7fe42415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:56.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe429a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe42415d410 0x7fe42415a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:45104/0 (socket says 172.21.15.161:45104) 2024-05-22T21:22:56.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe429a00640 1 -- 172.21.15.161:0/2946898242 learned_addr learned my addr 172.21.15.161:0/2946898242 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:56.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe429000640 1 --2- 172.21.15.161:0/2946898242 >> v2:172.21.15.179:3300/0 conn(0x7fe42415dd50 0x7fe42406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:56.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe423400640 1 -- 172.21.15.161:0/2946898242 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942159915 0 0) 0x7fe424071670 con 0x7fe42415d410 2024-05-22T21:22:56.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe423400640 1 -- 172.21.15.161:0/2946898242 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe40c005740 con 0x7fe42415d410 2024-05-22T21:22:56.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe423e00640 1 -- 172.21.15.161:0/2946898242 >> v2:172.21.15.179:3300/0 conn(0x7fe42415dd50 msgr2=0x7fe42406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:56.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe423e00640 1 --2- 172.21.15.161:0/2946898242 >> v2:172.21.15.179:3300/0 conn(0x7fe42415dd50 0x7fe42406d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe423e00640 1 -- 172.21.15.161:0/2946898242 >> v1:172.21.15.161:6789/0 conn(0x7fe42415d410 legacy=0x7fe42415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:56.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe423e00640 1 -- 172.21.15.161:0/2946898242 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe424070a60 con 0x7fe42415e130 2024-05-22T21:22:56.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.800+0000 7fe429000640 1 --2- 172.21.15.161:0/2946898242 >> v2:172.21.15.179:3300/0 conn(0x7fe42415dd50 0x7fe42406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:22:56.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.804+0000 7fe423e00640 1 --2- 172.21.15.161:0/2946898242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe42415e130 0x7fe42415ced0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fe424072330 tx=0x7fe41001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d00a30995c79c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:56.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.804+0000 7fe423400640 1 -- 172.21.15.161:0/2946898242 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4100ab650 con 0x7fe42415e130 2024-05-22T21:22:56.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.804+0000 7fe423400640 1 -- 172.21.15.161:0/2946898242 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4100ab7b0 con 0x7fe42415e130 2024-05-22T21:22:56.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.804+0000 7fe423400640 1 -- 172.21.15.161:0/2946898242 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4100abb40 con 0x7fe42415e130 2024-05-22T21:22:56.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.804+0000 7fe42ba00640 1 -- 172.21.15.161:0/2946898242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe42415e130 msgr2=0x7fe42415ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:56.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.804+0000 7fe42ba00640 1 --2- 172.21.15.161:0/2946898242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe42415e130 0x7fe42415ced0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fe424072330 tx=0x7fe41001c450 comp rx=0 tx=0).stop 2024-05-22T21:22:56.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.804+0000 7fe42ba00640 1 -- 172.21.15.161:0/2946898242 shutdown_connections 2024-05-22T21:22:56.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.804+0000 7fe42ba00640 1 --2- 172.21.15.161:0/2946898242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe42415e130 0x7fe42415ced0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.804+0000 7fe42ba00640 1 --2- 172.21.15.161:0/2946898242 >> v2:172.21.15.179:3300/0 conn(0x7fe42415dd50 0x7fe42406d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.804+0000 7fe42ba00640 1 -- 172.21.15.161:0/2946898242 >> 172.21.15.161:0/2946898242 conn(0x7fe4240b0bc0 msgr2=0x7fe4240b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:56.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.804+0000 7fe42ba00640 1 -- 172.21.15.161:0/2946898242 shutdown_connections 2024-05-22T21:22:56.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.804+0000 7fe42ba00640 1 -- 172.21.15.161:0/2946898242 wait complete. 2024-05-22T21:22:56.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe42ba00640 1 Processor -- start 2024-05-22T21:22:56.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe42ba00640 1 -- start start 2024-05-22T21:22:56.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe42ba00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe42415d410 0x7fe42414ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe42ba00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe42415dd50 0x7fe424152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe42ba00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe42415e130 0x7fe4240b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe42ba00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4240b3e70 con 0x7fe42415dd50 2024-05-22T21:22:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe42ba00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe424153a80 con 0x7fe42415e130 2024-05-22T21:22:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe42ba00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4241531e0 con 0x7fe42415d410 2024-05-22T21:22:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe423e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe42415dd50 0x7fe424152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe423e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe42415dd50 0x7fe424152ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:40980/0 (socket says 172.21.15.161:40980) 2024-05-22T21:22:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe423e00640 1 -- 172.21.15.161:0/667284280 learned_addr learned my addr 172.21.15.161:0/667284280 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe429a00640 1 --2- 172.21.15.161:0/667284280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe42415e130 0x7fe4240b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe429000640 1 --2- 172.21.15.161:0/667284280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe42415d410 0x7fe42414ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe423e00640 1 -- 172.21.15.161:0/667284280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe42415d410 msgr2=0x7fe42414ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:56.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe423e00640 1 --2- 172.21.15.161:0/667284280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe42415d410 0x7fe42414ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe423e00640 1 -- 172.21.15.161:0/667284280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe42415e130 msgr2=0x7fe4240b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:56.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe423e00640 1 --2- 172.21.15.161:0/667284280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe42415e130 0x7fe4240b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:56.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe423e00640 1 -- 172.21.15.161:0/667284280 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe410009420 con 0x7fe42415dd50 2024-05-22T21:22:56.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe429000640 1 --2- 172.21.15.161:0/667284280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe42415d410 0x7fe42414ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:22:56.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe423e00640 1 --2- 172.21.15.161:0/667284280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe42415dd50 0x7fe424152ae0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fe410019060 tx=0x7fe41001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:56.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe420c00640 1 -- 172.21.15.161:0/667284280 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4100bf610 con 0x7fe42415dd50 2024-05-22T21:22:56.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe420c00640 1 -- 172.21.15.161:0/667284280 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4100bf770 con 0x7fe42415dd50 2024-05-22T21:22:56.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe420c00640 1 -- 172.21.15.161:0/667284280 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4100bfb20 con 0x7fe42415dd50 2024-05-22T21:22:56.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe42ba00640 1 -- 172.21.15.161:0/667284280 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4241534c0 con 0x7fe42415dd50 2024-05-22T21:22:56.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe42ba00640 1 -- 172.21.15.161:0/667284280 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4240cc6d0 con 0x7fe42415dd50 2024-05-22T21:22:56.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.808+0000 7fe42ba00640 1 -- 172.21.15.161:0/667284280 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4240cc8c0 con 0x7fe42415dd50 2024-05-22T21:22:56.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.812+0000 7fe420c00640 1 -- 172.21.15.161:0/667284280 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe4100bfc80 con 0x7fe42415dd50 2024-05-22T21:22:56.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.812+0000 7fe420c00640 1 --2- 172.21.15.161:0/667284280 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe404066690 0x7fe404068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:56.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.812+0000 7fe420c00640 1 -- 172.21.15.161:0/667284280 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe41011e920 con 0x7fe42415dd50 2024-05-22T21:22:56.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.812+0000 7fe429000640 1 --2- 172.21.15.161:0/667284280 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe404066690 0x7fe404068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:56.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.812+0000 7fe429000640 1 --2- 172.21.15.161:0/667284280 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe404066690 0x7fe404068b50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe418009980 tx=0x7fe418008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:56.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:56.816+0000 7fe420c00640 1 -- 172.21.15.161:0/667284280 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe4100ebef0 con 0x7fe42415dd50 2024-05-22T21:22:57.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.024+0000 7fe42ba00640 1 -- 172.21.15.161:0/667284280 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7fe424067220 con 0x7fe42415dd50 2024-05-22T21:22:57.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.300+0000 7fe420c00640 1 -- 172.21.15.161:0/667284280 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v469) v1 ==== 132+0+0 (secure 0 0 0) 0x7fe4100ef510 con 0x7fe42415dd50 2024-05-22T21:22:57.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.300+0000 7fe42ba00640 1 -- 172.21.15.161:0/667284280 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe404066690 msgr2=0x7fe404068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:57.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.300+0000 7fe42ba00640 1 --2- 172.21.15.161:0/667284280 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe404066690 0x7fe404068b50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe418009980 tx=0x7fe418008040 comp rx=0 tx=0).stop 2024-05-22T21:22:57.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.300+0000 7fe42ba00640 1 -- 172.21.15.161:0/667284280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe42415dd50 msgr2=0x7fe424152ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:57.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.300+0000 7fe42ba00640 1 --2- 172.21.15.161:0/667284280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe42415dd50 0x7fe424152ae0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fe410019060 tx=0x7fe41001ac90 comp rx=0 tx=0).stop 2024-05-22T21:22:57.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.300+0000 7fe42ba00640 1 -- 172.21.15.161:0/667284280 shutdown_connections 2024-05-22T21:22:57.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.300+0000 7fe42ba00640 1 --2- 172.21.15.161:0/667284280 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe404066690 0x7fe404068b50 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:57.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.300+0000 7fe42ba00640 1 --2- 172.21.15.161:0/667284280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe42415e130 0x7fe4240b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:57.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.300+0000 7fe42ba00640 1 --2- 172.21.15.161:0/667284280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe42415dd50 0x7fe424152ae0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:57.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.300+0000 7fe42ba00640 1 --2- 172.21.15.161:0/667284280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe42415d410 0x7fe42414ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:57.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.300+0000 7fe42ba00640 1 -- 172.21.15.161:0/667284280 >> 172.21.15.161:0/667284280 conn(0x7fe4240b0bc0 msgr2=0x7fe4240adbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:57.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.300+0000 7fe42ba00640 1 -- 172.21.15.161:0/667284280 shutdown_connections 2024-05-22T21:22:57.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.300+0000 7fe42ba00640 1 -- 172.21.15.161:0/667284280 wait complete. 2024-05-22T21:22:57.321 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-05-22T21:22:57.322 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) That when the full flag is encountered during asynchronous 2024-05-22T21:22:57.322 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) 2024-05-22T21:22:57.322 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2024-05-22T21:22:57.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.456+0000 7f2f27e00640 1 Processor -- start 2024-05-22T21:22:57.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.456+0000 7f2f27e00640 1 -- start start 2024-05-22T21:22:57.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.456+0000 7f2f27e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2f2015dd50 0x7f2f2006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:57.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.456+0000 7f2f27e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2f2015e130 0x7f2f2015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:57.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.456+0000 7f2f27e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f20071670 con 0x7f2f2015d410 2024-05-22T21:22:57.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.456+0000 7f2f27e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2015d8a0 con 0x7f2f2015dd50 2024-05-22T21:22:57.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.456+0000 7f2f27e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2006de40 con 0x7f2f2015e130 2024-05-22T21:22:57.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.456+0000 7f2f25400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2f2015dd50 0x7f2f2006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:57.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.456+0000 7f2f25400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2f2015dd50 0x7f2f2006d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:40994/0 (socket says 172.21.15.161:40994) 2024-05-22T21:22:57.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.456+0000 7f2f25400640 1 -- 172.21.15.161:0/1623054175 learned_addr learned my addr 172.21.15.161:0/1623054175 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:57.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.456+0000 7f2f24a00640 1 --2- 172.21.15.161:0/1623054175 >> v2:172.21.15.179:3300/0 conn(0x7f2f2015e130 0x7f2f2015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:57.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.456+0000 7f2f25400640 1 -- 172.21.15.161:0/1623054175 >> v2:172.21.15.179:3300/0 conn(0x7f2f2015e130 msgr2=0x7f2f2015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:57.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.456+0000 7f2f25400640 1 --2- 172.21.15.161:0/1623054175 >> v2:172.21.15.179:3300/0 conn(0x7f2f2015e130 0x7f2f2015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:57.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.456+0000 7f2f25400640 1 -- 172.21.15.161:0/1623054175 >> v1:172.21.15.179:6789/0 conn(0x7f2f2015d410 legacy=0x7f2f2015a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:57.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.456+0000 7f2f25400640 1 -- 172.21.15.161:0/1623054175 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f20070a60 con 0x7f2f2015dd50 2024-05-22T21:22:57.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.460+0000 7f2f24a00640 1 --2- 172.21.15.161:0/1623054175 >> v2:172.21.15.179:3300/0 conn(0x7f2f2015e130 0x7f2f2015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:22:57.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.460+0000 7f2f25400640 1 --2- 172.21.15.161:0/1623054175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f2015dd50 0x7f2f2006d900 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f2f2006c7c0 tx=0x7f2f1401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784dddfa9f1b5f34 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:57.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.460+0000 7f2f1fe00640 1 -- 172.21.15.161:0/1623054175 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f140a5880 con 0x7f2f2015dd50 2024-05-22T21:22:57.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.460+0000 7f2f1fe00640 1 -- 172.21.15.161:0/1623054175 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f140a59e0 con 0x7f2f2015dd50 2024-05-22T21:22:57.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.460+0000 7f2f1fe00640 1 -- 172.21.15.161:0/1623054175 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f140a5d70 con 0x7f2f2015dd50 2024-05-22T21:22:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.460+0000 7f2f27e00640 1 -- 172.21.15.161:0/1623054175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f2015dd50 msgr2=0x7f2f2006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.460+0000 7f2f27e00640 1 --2- 172.21.15.161:0/1623054175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f2015dd50 0x7f2f2006d900 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f2f2006c7c0 tx=0x7f2f1401c9c0 comp rx=0 tx=0).stop 2024-05-22T21:22:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.460+0000 7f2f27e00640 1 -- 172.21.15.161:0/1623054175 shutdown_connections 2024-05-22T21:22:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.460+0000 7f2f27e00640 1 --2- 172.21.15.161:0/1623054175 >> v2:172.21.15.179:3300/0 conn(0x7f2f2015e130 0x7f2f2015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.460+0000 7f2f27e00640 1 --2- 172.21.15.161:0/1623054175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f2015dd50 0x7f2f2006d900 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.460+0000 7f2f27e00640 1 -- 172.21.15.161:0/1623054175 >> 172.21.15.161:0/1623054175 conn(0x7f2f200b0bc0 msgr2=0x7f2f200b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:57.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.460+0000 7f2f27e00640 1 -- 172.21.15.161:0/1623054175 shutdown_connections 2024-05-22T21:22:57.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.460+0000 7f2f27e00640 1 -- 172.21.15.161:0/1623054175 wait complete. 2024-05-22T21:22:57.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f27e00640 1 Processor -- start 2024-05-22T21:22:57.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f27e00640 1 -- start start 2024-05-22T21:22:57.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f27e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f2015d410 0x7f2f2014ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:57.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f27e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2f2015dd50 0x7f2f201511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:57.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f27e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f2015e130 0x7f2f2014c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:57.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f27e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f200cd530 con 0x7f2f2015d410 2024-05-22T21:22:57.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f27e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2014b190 con 0x7f2f2015e130 2024-05-22T21:22:57.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f27e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2014b530 con 0x7f2f2015dd50 2024-05-22T21:22:57.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f25400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f2015d410 0x7f2f2014ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f25400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f2015d410 0x7f2f2014ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41008/0 (socket says 172.21.15.161:41008) 2024-05-22T21:22:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f25400640 1 -- 172.21.15.161:0/38653967 learned_addr learned my addr 172.21.15.161:0/38653967 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f24a00640 1 --2- 172.21.15.161:0/38653967 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2f2015dd50 0x7f2f201511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f25e00640 1 --2- 172.21.15.161:0/38653967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f2015e130 0x7f2f2014c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f25400640 1 -- 172.21.15.161:0/38653967 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2f2015dd50 msgr2=0x7f2f201511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f25400640 1 --2- 172.21.15.161:0/38653967 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2f2015dd50 0x7f2f201511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f25400640 1 -- 172.21.15.161:0/38653967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f2015e130 msgr2=0x7f2f2014c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f25400640 1 --2- 172.21.15.161:0/38653967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f2015e130 0x7f2f2014c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f25400640 1 -- 172.21.15.161:0/38653967 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f14009b30 con 0x7f2f2015d410 2024-05-22T21:22:57.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f24a00640 1 --2- 172.21.15.161:0/38653967 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2f2015dd50 0x7f2f201511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:22:57.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f25400640 1 --2- 172.21.15.161:0/38653967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f2015d410 0x7f2f2014ac50 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f2f140af040 tx=0x7f2f14019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:57.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f1d600640 1 -- 172.21.15.161:0/38653967 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f140b8880 con 0x7f2f2015d410 2024-05-22T21:22:57.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f1d600640 1 -- 172.21.15.161:0/38653967 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f140b8a20 con 0x7f2f2015d410 2024-05-22T21:22:57.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f27e00640 1 -- 172.21.15.161:0/38653967 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2014b810 con 0x7f2f2015d410 2024-05-22T21:22:57.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f1d600640 1 -- 172.21.15.161:0/38653967 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f140b8dd0 con 0x7f2f2015d410 2024-05-22T21:22:57.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.464+0000 7f2f27e00640 1 -- 172.21.15.161:0/38653967 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f200cd150 con 0x7f2f2015d410 2024-05-22T21:22:57.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.468+0000 7f2f27e00640 1 -- 172.21.15.161:0/38653967 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f200cd670 con 0x7f2f2015d410 2024-05-22T21:22:57.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.468+0000 7f2f1d600640 1 -- 172.21.15.161:0/38653967 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2f140c3460 con 0x7f2f2015d410 2024-05-22T21:22:57.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.468+0000 7f2f1d600640 1 --2- 172.21.15.161:0/38653967 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f00066690 0x7f2f00068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:57.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.468+0000 7f2f1d600640 1 -- 172.21.15.161:0/38653967 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2f1411fe80 con 0x7f2f2015d410 2024-05-22T21:22:57.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.468+0000 7f2f24a00640 1 --2- 172.21.15.161:0/38653967 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f00066690 0x7f2f00068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:57.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.468+0000 7f2f24a00640 1 --2- 172.21.15.161:0/38653967 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f00066690 0x7f2f00068b50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2f200b1df0 tx=0x7f2f0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:57.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.472+0000 7f2f1d600640 1 -- 172.21.15.161:0/38653967 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2f140ed4d0 con 0x7f2f2015d410 2024-05-22T21:22:57.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:57.684+0000 7f2f27e00640 1 -- 172.21.15.161:0/38653967 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7f2f200cd670 con 0x7f2f2015d410 2024-05-22T21:22:58.303 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.304+0000 7f2f1d600640 1 -- 172.21.15.161:0/38653967 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v470) v1 ==== 134+0+0 (secure 0 0 0) 0x7f2f200cd670 con 0x7f2f2015d410 2024-05-22T21:22:58.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.304+0000 7f2f27e00640 1 -- 172.21.15.161:0/38653967 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f00066690 msgr2=0x7f2f00068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:58.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.304+0000 7f2f27e00640 1 --2- 172.21.15.161:0/38653967 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f00066690 0x7f2f00068b50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2f200b1df0 tx=0x7f2f0c008040 comp rx=0 tx=0).stop 2024-05-22T21:22:58.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.304+0000 7f2f27e00640 1 -- 172.21.15.161:0/38653967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f2015d410 msgr2=0x7f2f2014ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:58.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.304+0000 7f2f27e00640 1 --2- 172.21.15.161:0/38653967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f2015d410 0x7f2f2014ac50 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f2f140af040 tx=0x7f2f14019760 comp rx=0 tx=0).stop 2024-05-22T21:22:58.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.304+0000 7f2f27e00640 1 -- 172.21.15.161:0/38653967 shutdown_connections 2024-05-22T21:22:58.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.304+0000 7f2f27e00640 1 --2- 172.21.15.161:0/38653967 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2f00066690 0x7f2f00068b50 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:58.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.304+0000 7f2f27e00640 1 --2- 172.21.15.161:0/38653967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f2015e130 0x7f2f2014c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:58.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.304+0000 7f2f27e00640 1 --2- 172.21.15.161:0/38653967 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2f2015dd50 0x7f2f201511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:58.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.304+0000 7f2f27e00640 1 --2- 172.21.15.161:0/38653967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f2015d410 0x7f2f2014ac50 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:58.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.304+0000 7f2f27e00640 1 -- 172.21.15.161:0/38653967 >> 172.21.15.161:0/38653967 conn(0x7f2f200b0bc0 msgr2=0x7f2f2006c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:58.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.304+0000 7f2f27e00640 1 -- 172.21.15.161:0/38653967 shutdown_connections 2024-05-22T21:22:58.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.304+0000 7f2f27e00640 1 -- 172.21.15.161:0/38653967 wait complete. 2024-05-22T21:22:58.328 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-22T21:22:58.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.468+0000 7f0135a00640 1 Processor -- start 2024-05-22T21:22:58.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.468+0000 7f0135a00640 1 -- start start 2024-05-22T21:22:58.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.472+0000 7f0135a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f013013fe50 0x7f0130164e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:58.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.472+0000 7f0135a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f013013d3a0 con 0x7f013013cfc0 2024-05-22T21:22:58.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.472+0000 7f0135a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01300cd920 con 0x7f01301402e0 2024-05-22T21:22:58.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.472+0000 7f0135a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01300cdc00 con 0x7f013013fe50 2024-05-22T21:22:58.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.472+0000 7f012ea00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f013013cfc0 0x7f013015af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:45110/0 (socket says 172.21.15.161:45110) 2024-05-22T21:22:58.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.472+0000 7f012ea00640 1 -- 172.21.15.161:0/3522783104 learned_addr learned my addr 172.21.15.161:0/3522783104 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:58.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.472+0000 7f012f400640 1 --2- 172.21.15.161:0/3522783104 >> v2:172.21.15.161:3300/0 conn(0x7f013013fe50 0x7f0130164e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:58.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.472+0000 7f012e000640 1 -- 172.21.15.161:0/3522783104 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069582584 0 0) 0x7f013013d3a0 con 0x7f013013cfc0 2024-05-22T21:22:58.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.472+0000 7f012f400640 1 -- 172.21.15.161:0/3522783104 >> v1:172.21.15.179:6789/0 conn(0x7f01301402e0 legacy=0x7f013013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:58.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.472+0000 7f012f400640 1 -- 172.21.15.161:0/3522783104 >> v1:172.21.15.161:6789/0 conn(0x7f013013cfc0 legacy=0x7f013015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:58.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.472+0000 7f012f400640 1 -- 172.21.15.161:0/3522783104 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01300cdee0 con 0x7f013013fe50 2024-05-22T21:22:58.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.476+0000 7f012f400640 1 --2- 172.21.15.161:0/3522783104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f013013fe50 0x7f0130164e60 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f0128009d90 tx=0x7f012801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db659fbc6c211386 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:58.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.476+0000 7f012e000640 1 -- 172.21.15.161:0/3522783104 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01280a5850 con 0x7f013013fe50 2024-05-22T21:22:58.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.476+0000 7f012e000640 1 -- 172.21.15.161:0/3522783104 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01280a59b0 con 0x7f013013fe50 2024-05-22T21:22:58.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.476+0000 7f012e000640 1 -- 172.21.15.161:0/3522783104 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01280a5d40 con 0x7f013013fe50 2024-05-22T21:22:58.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.476+0000 7f0135a00640 1 -- 172.21.15.161:0/3522783104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f013013fe50 msgr2=0x7f0130164e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:58.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.476+0000 7f0135a00640 1 --2- 172.21.15.161:0/3522783104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f013013fe50 0x7f0130164e60 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f0128009d90 tx=0x7f012801c8a0 comp rx=0 tx=0).stop 2024-05-22T21:22:58.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.476+0000 7f0135a00640 1 -- 172.21.15.161:0/3522783104 shutdown_connections 2024-05-22T21:22:58.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.476+0000 7f0135a00640 1 --2- 172.21.15.161:0/3522783104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f013013fe50 0x7f0130164e60 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:58.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.476+0000 7f0135a00640 1 -- 172.21.15.161:0/3522783104 >> 172.21.15.161:0/3522783104 conn(0x7f013006a2d0 msgr2=0x7f013006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:58.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.476+0000 7f0135a00640 1 -- 172.21.15.161:0/3522783104 shutdown_connections 2024-05-22T21:22:58.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.476+0000 7f0135a00640 1 -- 172.21.15.161:0/3522783104 wait complete. 2024-05-22T21:22:58.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.476+0000 7f0135a00640 1 Processor -- start 2024-05-22T21:22:58.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.476+0000 7f0135a00640 1 -- start start 2024-05-22T21:22:58.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f0135a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f013013cfc0 0x7f01300ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:58.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f0135a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f013013fe50 0x7f0130141b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:58.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f0135a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01301402e0 0x7f013013bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:58.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f0135a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01300adb50 con 0x7f013013cfc0 2024-05-22T21:22:58.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f0135a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01301420d0 con 0x7f01301402e0 2024-05-22T21:22:58.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f0135a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01300acf40 con 0x7f013013fe50 2024-05-22T21:22:58.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f012ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f013013cfc0 0x7f01300ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:58.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f012ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f013013cfc0 0x7f01300ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41040/0 (socket says 172.21.15.161:41040) 2024-05-22T21:22:58.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f012ea00640 1 -- 172.21.15.161:0/641491941 learned_addr learned my addr 172.21.15.161:0/641491941 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:58.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f0126000640 1 --2- 172.21.15.161:0/641491941 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f013013fe50 0x7f0130141b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:58.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f012f400640 1 --2- 172.21.15.161:0/641491941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01301402e0 0x7f013013bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:58.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f012ea00640 1 -- 172.21.15.161:0/641491941 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f013013fe50 msgr2=0x7f0130141b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:58.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f012ea00640 1 --2- 172.21.15.161:0/641491941 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f013013fe50 0x7f0130141b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:58.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f012ea00640 1 -- 172.21.15.161:0/641491941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01301402e0 msgr2=0x7f013013bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:58.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f012ea00640 1 --2- 172.21.15.161:0/641491941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01301402e0 0x7f013013bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:58.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f012ea00640 1 -- 172.21.15.161:0/641491941 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0128009a70 con 0x7f013013cfc0 2024-05-22T21:22:58.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f012f400640 1 --2- 172.21.15.161:0/641491941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01301402e0 0x7f013013bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:22:58.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f012ea00640 1 --2- 172.21.15.161:0/641491941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f013013cfc0 0x7f01300ac8b0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f012000c480 tx=0x7f012000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:58.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f0127400640 1 -- 172.21.15.161:0/641491941 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01200074e0 con 0x7f013013cfc0 2024-05-22T21:22:58.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f0127400640 1 -- 172.21.15.161:0/641491941 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0120007660 con 0x7f013013cfc0 2024-05-22T21:22:58.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f0127400640 1 -- 172.21.15.161:0/641491941 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120004a30 con 0x7f013013cfc0 2024-05-22T21:22:58.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f0135a00640 1 -- 172.21.15.161:0/641491941 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01300ad220 con 0x7f013013cfc0 2024-05-22T21:22:58.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f0135a00640 1 -- 172.21.15.161:0/641491941 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01300ad7e0 con 0x7f013013cfc0 2024-05-22T21:22:58.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f0135a00640 1 -- 172.21.15.161:0/641491941 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0130067220 con 0x7f013013cfc0 2024-05-22T21:22:58.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.480+0000 7f0127400640 1 -- 172.21.15.161:0/641491941 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f012001b360 con 0x7f013013cfc0 2024-05-22T21:22:58.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.484+0000 7f0127400640 1 --2- 172.21.15.161:0/641491941 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f010c066690 0x7f010c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:58.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.484+0000 7f0127400640 1 -- 172.21.15.161:0/641491941 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f01200832d0 con 0x7f013013cfc0 2024-05-22T21:22:58.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.484+0000 7f0126000640 1 --2- 172.21.15.161:0/641491941 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f010c066690 0x7f010c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:58.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.484+0000 7f0126000640 1 --2- 172.21.15.161:0/641491941 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f010c066690 0x7f010c068b50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f01301f92b0 tx=0x7f0118007e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:58.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.484+0000 7f0127400640 1 -- 172.21.15.161:0/641491941 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0120050a60 con 0x7f013013cfc0 2024-05-22T21:22:58.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.640+0000 7f0135a00640 1 -- 172.21.15.161:0/641491941 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0130071d00 con 0x7f013013cfc0 2024-05-22T21:22:58.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.644+0000 7f0127400640 1 -- 172.21.15.161:0/641491941 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f01200180c0 con 0x7f013013cfc0 2024-05-22T21:22:58.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.644+0000 7f0135a00640 1 -- 172.21.15.161:0/641491941 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f010c066690 msgr2=0x7f010c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:58.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.644+0000 7f0135a00640 1 --2- 172.21.15.161:0/641491941 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f010c066690 0x7f010c068b50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f01301f92b0 tx=0x7f0118007e40 comp rx=0 tx=0).stop 2024-05-22T21:22:58.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.644+0000 7f0135a00640 1 -- 172.21.15.161:0/641491941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f013013cfc0 msgr2=0x7f01300ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:58.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.644+0000 7f0135a00640 1 --2- 172.21.15.161:0/641491941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f013013cfc0 0x7f01300ac8b0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f012000c480 tx=0x7f012000c940 comp rx=0 tx=0).stop 2024-05-22T21:22:58.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.644+0000 7f0135a00640 1 -- 172.21.15.161:0/641491941 shutdown_connections 2024-05-22T21:22:58.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.644+0000 7f0135a00640 1 --2- 172.21.15.161:0/641491941 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f010c066690 0x7f010c068b50 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:58.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.644+0000 7f0135a00640 1 --2- 172.21.15.161:0/641491941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01301402e0 0x7f013013bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:58.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.644+0000 7f0135a00640 1 --2- 172.21.15.161:0/641491941 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f013013fe50 0x7f0130141b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:58.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.644+0000 7f0135a00640 1 --2- 172.21.15.161:0/641491941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f013013cfc0 0x7f01300ac8b0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:58.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.644+0000 7f0135a00640 1 -- 172.21.15.161:0/641491941 >> 172.21.15.161:0/641491941 conn(0x7f013006a2d0 msgr2=0x7f01300af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:58.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.644+0000 7f0135a00640 1 -- 172.21.15.161:0/641491941 shutdown_connections 2024-05-22T21:22:58.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:58.644+0000 7f0135a00640 1 -- 172.21.15.161:0/641491941 wait complete. 2024-05-22T21:22:58.665 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:22:58.665 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:22:58.753 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:22:58.814 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:22:58.815 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:22:58.815 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:22:58.823 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:22:58.824 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:22:58.824 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:22:58.833 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:22:58.833 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:22:58.833 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:22:58.874 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:22:58.874 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:22:58.874 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:22:58.886 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:22:58.886 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-22T21:22:58.886 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:22:59.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc49a800640 1 Processor -- start 2024-05-22T21:22:59.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc49a800640 1 -- start start 2024-05-22T21:22:59.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc49a800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc49406e980 0x7fc4940cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:59.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc49a800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc49406ed60 0x7fc49406e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:59.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc49a800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4940cf8c0 con 0x7fc494160f30 2024-05-22T21:22:59.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc49a800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4940cfba0 con 0x7fc49406e980 2024-05-22T21:22:59.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc49a800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4940cfe80 con 0x7fc49406ed60 2024-05-22T21:22:59.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc493400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc49406e980 0x7fc4940cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:59.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc493e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc494160f30 0x7fc494161380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:45124/0 (socket says 172.21.15.161:45124) 2024-05-22T21:22:59.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc493e00640 1 -- 172.21.15.161:0/3910556609 learned_addr learned my addr 172.21.15.161:0/3910556609 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:59.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc492a00640 1 --2- 172.21.15.161:0/3910556609 >> v2:172.21.15.179:3300/0 conn(0x7fc49406ed60 0x7fc49406e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:59.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc492000640 1 -- 172.21.15.161:0/3910556609 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239641352 0 0) 0x7fc4940cf8c0 con 0x7fc494160f30 2024-05-22T21:22:59.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc492000640 1 -- 172.21.15.161:0/3910556609 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc478003530 con 0x7fc494160f30 2024-05-22T21:22:59.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc493400640 1 -- 172.21.15.161:0/3910556609 >> v2:172.21.15.179:3300/0 conn(0x7fc49406ed60 msgr2=0x7fc49406e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:22:59.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc493400640 1 --2- 172.21.15.161:0/3910556609 >> v2:172.21.15.179:3300/0 conn(0x7fc49406ed60 0x7fc49406e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:59.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc493400640 1 -- 172.21.15.161:0/3910556609 >> v1:172.21.15.161:6789/0 conn(0x7fc494160f30 legacy=0x7fc494161380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:59.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc493400640 1 -- 172.21.15.161:0/3910556609 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4940d0160 con 0x7fc49406e980 2024-05-22T21:22:59.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.052+0000 7fc492a00640 1 --2- 172.21.15.161:0/3910556609 >> v2:172.21.15.179:3300/0 conn(0x7fc49406ed60 0x7fc49406e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:22:59.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.056+0000 7fc493400640 1 --2- 172.21.15.161:0/3910556609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc49406e980 0x7fc4940cf290 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fc494071000 tx=0x7fc48401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6159cf1998f195f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:59.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.056+0000 7fc492000640 1 -- 172.21.15.161:0/3910556609 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4840a5880 con 0x7fc49406e980 2024-05-22T21:22:59.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.056+0000 7fc492000640 1 -- 172.21.15.161:0/3910556609 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4840a59e0 con 0x7fc49406e980 2024-05-22T21:22:59.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.056+0000 7fc492000640 1 -- 172.21.15.161:0/3910556609 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4840a5d50 con 0x7fc49406e980 2024-05-22T21:22:59.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.056+0000 7fc49a800640 1 -- 172.21.15.161:0/3910556609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc49406e980 msgr2=0x7fc4940cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:59.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.056+0000 7fc49a800640 1 --2- 172.21.15.161:0/3910556609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc49406e980 0x7fc4940cf290 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fc494071000 tx=0x7fc48401c930 comp rx=0 tx=0).stop 2024-05-22T21:22:59.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.056+0000 7fc49a800640 1 -- 172.21.15.161:0/3910556609 shutdown_connections 2024-05-22T21:22:59.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.056+0000 7fc49a800640 1 --2- 172.21.15.161:0/3910556609 >> v2:172.21.15.179:3300/0 conn(0x7fc49406ed60 0x7fc49406e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:59.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.056+0000 7fc49a800640 1 --2- 172.21.15.161:0/3910556609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc49406e980 0x7fc4940cf290 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:59.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.056+0000 7fc49a800640 1 -- 172.21.15.161:0/3910556609 >> 172.21.15.161:0/3910556609 conn(0x7fc4940b2a40 msgr2=0x7fc4940b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:59.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.056+0000 7fc49a800640 1 -- 172.21.15.161:0/3910556609 shutdown_connections 2024-05-22T21:22:59.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.056+0000 7fc49a800640 1 -- 172.21.15.161:0/3910556609 wait complete. 2024-05-22T21:22:59.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc49a800640 1 Processor -- start 2024-05-22T21:22:59.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc49a800640 1 -- start start 2024-05-22T21:22:59.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc49a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc49406e980 0x7fc494157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:59.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc49a800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc49406ed60 0x7fc4940b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:59.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc49a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc494160f30 0x7fc4941590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:59.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc49a800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940b4510 con 0x7fc494160f30 2024-05-22T21:22:59.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc49a800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4941580d0 con 0x7fc49406e980 2024-05-22T21:22:59.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc49a800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4941545b0 con 0x7fc49406ed60 2024-05-22T21:22:59.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc493e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc494160f30 0x7fc4941590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:59.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc493e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc494160f30 0x7fc4941590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41068/0 (socket says 172.21.15.161:41068) 2024-05-22T21:22:59.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc493e00640 1 -- 172.21.15.161:0/512272792 learned_addr learned my addr 172.21.15.161:0/512272792 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:59.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc493400640 1 --2- 172.21.15.161:0/512272792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc49406e980 0x7fc494157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:59.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc492a00640 1 --2- 172.21.15.161:0/512272792 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc49406ed60 0x7fc4940b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:59.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc493e00640 1 -- 172.21.15.161:0/512272792 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc49406ed60 msgr2=0x7fc4940b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:59.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc493e00640 1 --2- 172.21.15.161:0/512272792 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc49406ed60 0x7fc4940b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:59.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc493e00640 1 -- 172.21.15.161:0/512272792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc49406e980 msgr2=0x7fc494157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:59.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc493e00640 1 --2- 172.21.15.161:0/512272792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc49406e980 0x7fc494157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:59.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc493e00640 1 -- 172.21.15.161:0/512272792 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc484009a70 con 0x7fc494160f30 2024-05-22T21:22:59.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc493400640 1 --2- 172.21.15.161:0/512272792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc49406e980 0x7fc494157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:22:59.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc492a00640 1 --2- 172.21.15.161:0/512272792 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc49406ed60 0x7fc4940b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:22:59.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc493e00640 1 --2- 172.21.15.161:0/512272792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc494160f30 0x7fc4941590b0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00c3c0 tx=0x7fc48c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:59.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc48b400640 1 -- 172.21.15.161:0/512272792 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c005cf0 con 0x7fc494160f30 2024-05-22T21:22:59.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc48b400640 1 -- 172.21.15.161:0/512272792 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c005e50 con 0x7fc494160f30 2024-05-22T21:22:59.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc48b400640 1 -- 172.21.15.161:0/512272792 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c007310 con 0x7fc494160f30 2024-05-22T21:22:59.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc49a800640 1 -- 172.21.15.161:0/512272792 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4940b4760 con 0x7fc494160f30 2024-05-22T21:22:59.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.060+0000 7fc49a800640 1 -- 172.21.15.161:0/512272792 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49415b8f0 con 0x7fc494160f30 2024-05-22T21:22:59.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.064+0000 7fc49a800640 1 -- 172.21.15.161:0/512272792 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc494067220 con 0x7fc494160f30 2024-05-22T21:22:59.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.064+0000 7fc48b400640 1 -- 172.21.15.161:0/512272792 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc48c007470 con 0x7fc494160f30 2024-05-22T21:22:59.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.064+0000 7fc48b400640 1 --2- 172.21.15.161:0/512272792 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc474066690 0x7fc474068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:59.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.064+0000 7fc48b400640 1 -- 172.21.15.161:0/512272792 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc48c0830a0 con 0x7fc494160f30 2024-05-22T21:22:59.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.064+0000 7fc493400640 1 --2- 172.21.15.161:0/512272792 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc474066690 0x7fc474068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:59.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.064+0000 7fc493400640 1 --2- 172.21.15.161:0/512272792 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc474066690 0x7fc474068b50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc4840af000 tx=0x7fc4840a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:59.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.068+0000 7fc48b400640 1 -- 172.21.15.161:0/512272792 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc48c050700 con 0x7fc494160f30 2024-05-22T21:22:59.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.252+0000 7fc49a800640 1 -- 172.21.15.161:0/512272792 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4940b6ed0 con 0x7fc494160f30 2024-05-22T21:22:59.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.252+0000 7fc48b400640 1 -- 172.21.15.161:0/512272792 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fc48c017090 con 0x7fc494160f30 2024-05-22T21:22:59.254 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:22:59.254 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":72,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/4226627816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":4226627816},{"type":"v1","addr":"172.21.15.179:6835","nonce":4226627816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7992,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/207877183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":207877183},{"type":"v1","addr":"172.21.15.161:6836","nonce":207877183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2152702106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2152702106},{"type":"v1","addr":"172.21.15.161:6837","nonce":2152702106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/749927648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":749927648},{"type":"v1","addr":"172.21.15.179:6834","nonce":749927648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[]} 2024-05-22T21:22:59.254 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 72 2024-05-22T21:22:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.256+0000 7fc49a800640 1 -- 172.21.15.161:0/512272792 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc474066690 msgr2=0x7fc474068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.256+0000 7fc49a800640 1 --2- 172.21.15.161:0/512272792 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc474066690 0x7fc474068b50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc4840af000 tx=0x7fc4840a7070 comp rx=0 tx=0).stop 2024-05-22T21:22:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.256+0000 7fc49a800640 1 -- 172.21.15.161:0/512272792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc494160f30 msgr2=0x7fc4941590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.256+0000 7fc49a800640 1 --2- 172.21.15.161:0/512272792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc494160f30 0x7fc4941590b0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00c3c0 tx=0x7fc48c00c880 comp rx=0 tx=0).stop 2024-05-22T21:22:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.256+0000 7fc49a800640 1 -- 172.21.15.161:0/512272792 shutdown_connections 2024-05-22T21:22:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.256+0000 7fc49a800640 1 --2- 172.21.15.161:0/512272792 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc474066690 0x7fc474068b50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.256+0000 7fc49a800640 1 --2- 172.21.15.161:0/512272792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc494160f30 0x7fc4941590b0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:59.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.256+0000 7fc49a800640 1 --2- 172.21.15.161:0/512272792 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc49406ed60 0x7fc4940b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:59.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.256+0000 7fc49a800640 1 --2- 172.21.15.161:0/512272792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc49406e980 0x7fc494157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:59.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.256+0000 7fc49a800640 1 -- 172.21.15.161:0/512272792 >> 172.21.15.161:0/512272792 conn(0x7fc4940b2a40 msgr2=0x7fc4940cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:59.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.256+0000 7fc49a800640 1 -- 172.21.15.161:0/512272792 shutdown_connections 2024-05-22T21:22:59.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.256+0000 7fc49a800640 1 -- 172.21.15.161:0/512272792 wait complete. 2024-05-22T21:22:59.278 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-22T21:22:59.278 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-22T21:22:59.278 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-22T21:22:59.278 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:22:59.278 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-22T21:22:59.279 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-22T21:22:59.279 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-22T21:22:59.279 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:22:59.279 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-22T21:22:59.279 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-22T21:22:59.280 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-22T21:22:59.280 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:22:59.280 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-22T21:22:59.280 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-22T21:22:59.280 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-22T21:22:59.280 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:22:59.290 INFO:tasks.ceph.mds.c:Stopped 2024-05-22T21:22:59.291 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-22T21:22:59.291 INFO:tasks.ceph.mds.a:Stopped 2024-05-22T21:22:59.292 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-22T21:22:59.294 INFO:tasks.ceph.mds.b:Stopped 2024-05-22T21:22:59.294 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-22T21:22:59.295 INFO:tasks.ceph.mds.d:Stopped 2024-05-22T21:22:59.296 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-22T21:22:59.341 INFO:tasks.ceph.mds.c:Started 2024-05-22T21:22:59.341 DEBUG:teuthology.parallel:result is None 2024-05-22T21:22:59.342 INFO:tasks.ceph.mds.a:Started 2024-05-22T21:22:59.343 DEBUG:teuthology.parallel:result is None 2024-05-22T21:22:59.344 INFO:tasks.ceph.mds.b:Started 2024-05-22T21:22:59.345 DEBUG:teuthology.parallel:result is None 2024-05-22T21:22:59.347 INFO:tasks.ceph.mds.d:Started 2024-05-22T21:22:59.347 DEBUG:teuthology.parallel:result is None 2024-05-22T21:22:59.347 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-22T21:22:59.426 INFO:tasks.ceph.mds.a.smithi161.stdout:starting mds.a at 2024-05-22T21:22:59.427 INFO:tasks.ceph.mds.c.smithi161.stdout:starting mds.c at 2024-05-22T21:22:59.431 INFO:tasks.ceph.mds.d.smithi179.stdout:starting mds.d at 2024-05-22T21:22:59.431 INFO:tasks.ceph.mds.b.smithi179.stdout:starting mds.b at 2024-05-22T21:22:59.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.492+0000 7fe849a00640 1 Processor -- start 2024-05-22T21:22:59.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.492+0000 7fe849a00640 1 -- start start 2024-05-22T21:22:59.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.496+0000 7fe849a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe84406c9b0 0x7fe8440b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:59.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.496+0000 7fe849a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe844144bb0 con 0x7fe844143c40 2024-05-22T21:22:59.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.496+0000 7fe849a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe84415efb0 con 0x7fe84415b600 2024-05-22T21:22:59.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.496+0000 7fe849a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe84415ba60 con 0x7fe84406c9b0 2024-05-22T21:22:59.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.496+0000 7fe843400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe84406c9b0 0x7fe8440b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:59.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.496+0000 7fe842a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe844143c40 0x7fe8441444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:45144/0 (socket says 172.21.15.161:45144) 2024-05-22T21:22:59.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.496+0000 7fe842a00640 1 -- 172.21.15.161:0/506831095 learned_addr learned my addr 172.21.15.161:0/506831095 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:22:59.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.496+0000 7fe843400640 1 -- 172.21.15.161:0/506831095 >> v1:172.21.15.179:6789/0 conn(0x7fe84415b600 legacy=0x7fe84406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:59.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.496+0000 7fe843400640 1 -- 172.21.15.161:0/506831095 >> v1:172.21.15.161:6789/0 conn(0x7fe844143c40 legacy=0x7fe8441444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:59.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.496+0000 7fe841600640 1 -- 172.21.15.161:0/506831095 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266453402 0 0) 0x7fe84415efb0 con 0x7fe84415b600 2024-05-22T21:22:59.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.496+0000 7fe843400640 1 -- 172.21.15.161:0/506831095 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84415f290 con 0x7fe84406c9b0 2024-05-22T21:22:59.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe843400640 1 --2- 172.21.15.161:0/506831095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe84406c9b0 0x7fe8440b2160 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0097c0 tx=0x7fe83c01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45a7925ae289fa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:59.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe841600640 1 -- 172.21.15.161:0/506831095 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c0ab600 con 0x7fe84406c9b0 2024-05-22T21:22:59.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe841600640 1 -- 172.21.15.161:0/506831095 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c0a7050 con 0x7fe84406c9b0 2024-05-22T21:22:59.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe841600640 1 -- 172.21.15.161:0/506831095 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c0ab980 con 0x7fe84406c9b0 2024-05-22T21:22:59.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe849a00640 1 -- 172.21.15.161:0/506831095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe84406c9b0 msgr2=0x7fe8440b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:59.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe849a00640 1 --2- 172.21.15.161:0/506831095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe84406c9b0 0x7fe8440b2160 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0097c0 tx=0x7fe83c01c330 comp rx=0 tx=0).stop 2024-05-22T21:22:59.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe849a00640 1 -- 172.21.15.161:0/506831095 shutdown_connections 2024-05-22T21:22:59.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe849a00640 1 --2- 172.21.15.161:0/506831095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe84406c9b0 0x7fe8440b2160 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:59.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe849a00640 1 -- 172.21.15.161:0/506831095 >> 172.21.15.161:0/506831095 conn(0x7fe84406a2d0 msgr2=0x7fe84406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:22:59.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe849a00640 1 -- 172.21.15.161:0/506831095 shutdown_connections 2024-05-22T21:22:59.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe849a00640 1 -- 172.21.15.161:0/506831095 wait complete. 2024-05-22T21:22:59.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe849a00640 1 Processor -- start 2024-05-22T21:22:59.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe849a00640 1 -- start start 2024-05-22T21:22:59.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe849a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84406c9b0 0x7fe8441361f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:59.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe849a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe844143c40 0x7fe84413c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:59.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe849a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe84415b600 0x7fe84413a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:59.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe849a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8441543b0 con 0x7fe844143c40 2024-05-22T21:22:59.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe849a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84413d0f0 con 0x7fe84406c9b0 2024-05-22T21:22:59.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe849a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84413d3d0 con 0x7fe84415b600 2024-05-22T21:22:59.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.500+0000 7fe842000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe844143c40 0x7fe84413c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:59.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe842000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe844143c40 0x7fe84413c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41116/0 (socket says 172.21.15.161:41116) 2024-05-22T21:22:59.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe842000640 1 -- 172.21.15.161:0/2796677764 learned_addr learned my addr 172.21.15.161:0/2796677764 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:22:59.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe842a00640 1 --2- 172.21.15.161:0/2796677764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84406c9b0 0x7fe8441361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:59.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe842000640 1 -- 172.21.15.161:0/2796677764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe84415b600 msgr2=0x7fe84413a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:59.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe842000640 1 --2- 172.21.15.161:0/2796677764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe84415b600 0x7fe84413a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:59.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe842000640 1 -- 172.21.15.161:0/2796677764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84406c9b0 msgr2=0x7fe8441361f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:22:59.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe842000640 1 --2- 172.21.15.161:0/2796677764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84406c9b0 0x7fe8441361f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:22:59.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe842000640 1 -- 172.21.15.161:0/2796677764 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c009420 con 0x7fe844143c40 2024-05-22T21:22:59.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe842a00640 1 --2- 172.21.15.161:0/2796677764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84406c9b0 0x7fe8441361f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:22:59.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe842000640 1 --2- 172.21.15.161:0/2796677764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe844143c40 0x7fe84413c610 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fe82c001ec0 tx=0x7fe82c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:59.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe83aa00640 1 -- 172.21.15.161:0/2796677764 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c0054e0 con 0x7fe844143c40 2024-05-22T21:22:59.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe83aa00640 1 -- 172.21.15.161:0/2796677764 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82c003600 con 0x7fe844143c40 2024-05-22T21:22:59.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe83aa00640 1 -- 172.21.15.161:0/2796677764 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c0039b0 con 0x7fe844143c40 2024-05-22T21:22:59.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe849a00640 1 -- 172.21.15.161:0/2796677764 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe844154600 con 0x7fe844143c40 2024-05-22T21:22:59.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe849a00640 1 -- 172.21.15.161:0/2796677764 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe844136730 con 0x7fe844143c40 2024-05-22T21:22:59.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe849a00640 1 -- 172.21.15.161:0/2796677764 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe844067220 con 0x7fe844143c40 2024-05-22T21:22:59.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe83aa00640 1 -- 172.21.15.161:0/2796677764 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe82c003b10 con 0x7fe844143c40 2024-05-22T21:22:59.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe83aa00640 1 --2- 172.21.15.161:0/2796677764 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe820066690 0x7fe820068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:22:59.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe83aa00640 1 -- 172.21.15.161:0/2796677764 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe82c0834b0 con 0x7fe844143c40 2024-05-22T21:22:59.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.504+0000 7fe842a00640 1 --2- 172.21.15.161:0/2796677764 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe820066690 0x7fe820068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:22:59.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.508+0000 7fe842a00640 1 --2- 172.21.15.161:0/2796677764 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe820066690 0x7fe820068b50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe8340012f0 tx=0x7fe834009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:22:59.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.508+0000 7fe83aa00640 1 -- 172.21.15.161:0/2796677764 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe82c050b10 con 0x7fe844143c40 2024-05-22T21:22:59.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:22:59.676+0000 7fe849a00640 1 -- 172.21.15.161:0/2796677764 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe844154ab0 con 0x7fe844143c40 2024-05-22T21:23:00.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.468+0000 7fe83aa00640 1 -- 172.21.15.161:0/2796677764 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v171) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe82c017020 con 0x7fe844143c40 2024-05-22T21:23:00.469 INFO:teuthology.orchestra.run.smithi161.stderr: removed all blocklist entries 2024-05-22T21:23:00.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.472+0000 7fe849a00640 1 -- 172.21.15.161:0/2796677764 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe820066690 msgr2=0x7fe820068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:00.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.472+0000 7fe849a00640 1 --2- 172.21.15.161:0/2796677764 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe820066690 0x7fe820068b50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe8340012f0 tx=0x7fe834009040 comp rx=0 tx=0).stop 2024-05-22T21:23:00.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.472+0000 7fe849a00640 1 -- 172.21.15.161:0/2796677764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe844143c40 msgr2=0x7fe84413c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:00.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.472+0000 7fe849a00640 1 --2- 172.21.15.161:0/2796677764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe844143c40 0x7fe84413c610 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fe82c001ec0 tx=0x7fe82c00c8e0 comp rx=0 tx=0).stop 2024-05-22T21:23:00.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.472+0000 7fe849a00640 1 -- 172.21.15.161:0/2796677764 shutdown_connections 2024-05-22T21:23:00.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.472+0000 7fe849a00640 1 --2- 172.21.15.161:0/2796677764 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe820066690 0x7fe820068b50 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:00.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.472+0000 7fe849a00640 1 --2- 172.21.15.161:0/2796677764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe84415b600 0x7fe84413a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:00.472 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.472+0000 7fe849a00640 1 --2- 172.21.15.161:0/2796677764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe844143c40 0x7fe84413c610 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:00.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.472+0000 7fe849a00640 1 --2- 172.21.15.161:0/2796677764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe84406c9b0 0x7fe8441361f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:00.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.472+0000 7fe849a00640 1 -- 172.21.15.161:0/2796677764 >> 172.21.15.161:0/2796677764 conn(0x7fe84406a2d0 msgr2=0x7fe844156b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:00.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.472+0000 7fe849a00640 1 -- 172.21.15.161:0/2796677764 shutdown_connections 2024-05-22T21:23:00.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.472+0000 7fe849a00640 1 -- 172.21.15.161:0/2796677764 wait complete. 2024-05-22T21:23:00.493 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-22T21:23:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3aece00640 1 Processor -- start 2024-05-22T21:23:00.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3aece00640 1 -- start start 2024-05-22T21:23:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3aece00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ae815dd50 0x7f3ae806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3aece00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae8071670 con 0x7f3ae815d5e0 2024-05-22T21:23:00.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3aece00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae806de40 con 0x7f3ae815e130 2024-05-22T21:23:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3aece00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae8070a60 con 0x7f3ae815dd50 2024-05-22T21:23:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae6800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3ae815d5e0 0x7f3ae815b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:45150/0 (socket says 172.21.15.161:45150) 2024-05-22T21:23:00.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae6800640 1 -- 172.21.15.161:0/2612900601 learned_addr learned my addr 172.21.15.161:0/2612900601 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:23:00.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae5e00640 1 --2- 172.21.15.161:0/2612900601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ae815dd50 0x7f3ae806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:00.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 -- 172.21.15.161:0/2612900601 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2518424757 0 0) 0x7f3ae8071670 con 0x7f3ae815d5e0 2024-05-22T21:23:00.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 -- 172.21.15.161:0/2612900601 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3acc003530 con 0x7f3ae815d5e0 2024-05-22T21:23:00.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 -- 172.21.15.161:0/2612900601 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057350451 0 0) 0x7f3ae806de40 con 0x7f3ae815e130 2024-05-22T21:23:00.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 -- 172.21.15.161:0/2612900601 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ae8071670 con 0x7f3ae815e130 2024-05-22T21:23:00.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 -- 172.21.15.161:0/2612900601 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3508380790 0 0) 0x7f3acc003530 con 0x7f3ae815d5e0 2024-05-22T21:23:00.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 -- 172.21.15.161:0/2612900601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ae815dd50 msgr2=0x7f3ae806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:00.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 --2- 172.21.15.161:0/2612900601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ae815dd50 0x7f3ae806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:00.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 -- 172.21.15.161:0/2612900601 >> v1:172.21.15.179:6789/0 conn(0x7f3ae815e130 legacy=0x7f3ae815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:00.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 -- 172.21.15.161:0/2612900601 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae8070d40 con 0x7f3ae815d5e0 2024-05-22T21:23:00.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 -- 172.21.15.161:0/2612900601 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f3ae0001a00 con 0x7f3ae815d5e0 2024-05-22T21:23:00.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 -- 172.21.15.161:0/2612900601 >> v1:172.21.15.161:6789/0 conn(0x7f3ae815d5e0 legacy=0x7f3ae815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:00.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 --2- 172.21.15.161:0/2612900601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ae815dd50 0x7f3acc003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:00.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 --2- 172.21.15.161:0/2612900601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3acc0040a0 0x7f3acc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:00.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 --2- 172.21.15.161:0/2612900601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3acc004d10 0x7f3acc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:00.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 -- 172.21.15.161:0/2612900601 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae8070a60 con 0x7f3acc0040a0 2024-05-22T21:23:00.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 -- 172.21.15.161:0/2612900601 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc00dd30 con 0x7f3acc004d10 2024-05-22T21:23:00.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae4a00640 1 -- 172.21.15.161:0/2612900601 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc00dec0 con 0x7f3ae815dd50 2024-05-22T21:23:00.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae5400640 1 --2- 172.21.15.161:0/2612900601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3acc0040a0 0x7f3acc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:00.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae6800640 1 --2- 172.21.15.161:0/2612900601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3acc004d10 0x7f3acc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:00.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae5e00640 1 --2- 172.21.15.161:0/2612900601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ae815dd50 0x7f3acc003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:00.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae5400640 1 -- 172.21.15.161:0/2612900601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ae815dd50 msgr2=0x7f3acc003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:00.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae5400640 1 --2- 172.21.15.161:0/2612900601 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ae815dd50 0x7f3acc003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:00.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae5400640 1 -- 172.21.15.161:0/2612900601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3acc004d10 msgr2=0x7f3acc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:00.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae5400640 1 --2- 172.21.15.161:0/2612900601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3acc004d10 0x7f3acc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:00.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.632+0000 7f3ae5400640 1 -- 172.21.15.161:0/2612900601 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc003530 con 0x7f3acc0040a0 2024-05-22T21:23:00.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.636+0000 7f3ae5400640 1 --2- 172.21.15.161:0/2612900601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3acc0040a0 0x7f3acc0047d0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40018d0 tx=0x7f3ad401c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b52bc879c8fbb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:00.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.636+0000 7f3ae4a00640 1 -- 172.21.15.161:0/2612900601 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad40052d0 con 0x7f3acc0040a0 2024-05-22T21:23:00.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.636+0000 7f3ae4a00640 1 -- 172.21.15.161:0/2612900601 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad4005450 con 0x7f3acc0040a0 2024-05-22T21:23:00.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.636+0000 7f3ae4a00640 1 -- 172.21.15.161:0/2612900601 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4003840 con 0x7f3acc0040a0 2024-05-22T21:23:00.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.636+0000 7f3aece00640 1 -- 172.21.15.161:0/2612900601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3acc0040a0 msgr2=0x7f3acc0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:00.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.636+0000 7f3aece00640 1 --2- 172.21.15.161:0/2612900601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3acc0040a0 0x7f3acc0047d0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40018d0 tx=0x7f3ad401c550 comp rx=0 tx=0).stop 2024-05-22T21:23:00.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.636+0000 7f3ae6800640 1 -- 172.21.15.161:0/2612900601 reap_dead start 2024-05-22T21:23:00.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.636+0000 7f3aece00640 1 -- 172.21.15.161:0/2612900601 shutdown_connections 2024-05-22T21:23:00.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.636+0000 7f3aece00640 1 -- 172.21.15.161:0/2612900601 >> 172.21.15.161:0/2612900601 conn(0x7f3ae80b0bc0 msgr2=0x7f3ae80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:00.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.636+0000 7f3aece00640 1 -- 172.21.15.161:0/2612900601 shutdown_connections 2024-05-22T21:23:00.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.636+0000 7f3aece00640 1 -- 172.21.15.161:0/2612900601 wait complete. 2024-05-22T21:23:00.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3aece00640 1 Processor -- start 2024-05-22T21:23:00.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3aece00640 1 -- start start 2024-05-22T21:23:00.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3aece00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc0040a0 0x7f3ae81e5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:00.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3aece00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ae80b3000 0x7f3ae8152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:00.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3aece00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ae81f9270 0x7f3ae81f96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:00.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3aece00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae81f9c30 con 0x7f3ae80b3000 2024-05-22T21:23:00.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3aece00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae81531a0 con 0x7f3ae81f9270 2024-05-22T21:23:00.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3aece00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae81545a0 con 0x7f3acc0040a0 2024-05-22T21:23:00.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3ae5400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ae80b3000 0x7f3ae8152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:00.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3ae5e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc0040a0 0x7f3ae81e5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:00.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3ae5400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ae80b3000 0x7f3ae8152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41126/0 (socket says 172.21.15.161:41126) 2024-05-22T21:23:00.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3ae5400640 1 -- 172.21.15.161:0/2561870058 learned_addr learned my addr 172.21.15.161:0/2561870058 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:00.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3ae6800640 1 --2- 172.21.15.161:0/2561870058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ae81f9270 0x7f3ae81f96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:00.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3ae5e00640 1 -- 172.21.15.161:0/2561870058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ae81f9270 msgr2=0x7f3ae81f96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:00.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3ae5e00640 1 --2- 172.21.15.161:0/2561870058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ae81f9270 0x7f3ae81f96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:00.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3ae5e00640 1 -- 172.21.15.161:0/2561870058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ae80b3000 msgr2=0x7f3ae8152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:00.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3ae5e00640 1 --2- 172.21.15.161:0/2561870058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ae80b3000 0x7f3ae8152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:00.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3ae5e00640 1 -- 172.21.15.161:0/2561870058 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae8154880 con 0x7f3acc0040a0 2024-05-22T21:23:00.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3ae5e00640 1 --2- 172.21.15.161:0/2561870058 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc0040a0 0x7f3ae81e5e30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8011960 tx=0x7f3ad8011e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:00.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3ade000640 1 -- 172.21.15.161:0/2561870058 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8004300 con 0x7f3acc0040a0 2024-05-22T21:23:00.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3ade000640 1 -- 172.21.15.161:0/2561870058 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad8004460 con 0x7f3acc0040a0 2024-05-22T21:23:00.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3ade000640 1 -- 172.21.15.161:0/2561870058 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad800fa20 con 0x7f3acc0040a0 2024-05-22T21:23:00.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3aece00640 1 -- 172.21.15.161:0/2561870058 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae8154b60 con 0x7f3acc0040a0 2024-05-22T21:23:00.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.640+0000 7f3aece00640 1 -- 172.21.15.161:0/2561870058 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae8155120 con 0x7f3acc0040a0 2024-05-22T21:23:00.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.644+0000 7f3aece00640 1 -- 172.21.15.161:0/2561870058 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae81552e0 con 0x7f3acc0040a0 2024-05-22T21:23:00.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.644+0000 7f3ade000640 1 -- 172.21.15.161:0/2561870058 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3ad800fb80 con 0x7f3acc0040a0 2024-05-22T21:23:00.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.644+0000 7f3ade000640 1 --2- 172.21.15.161:0/2561870058 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ac8066690 0x7f3ac8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:00.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.644+0000 7f3ade000640 1 -- 172.21.15.161:0/2561870058 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3ad8084350 con 0x7f3acc0040a0 2024-05-22T21:23:00.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.644+0000 7f3ae5400640 1 --2- 172.21.15.161:0/2561870058 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ac8066690 0x7f3ac8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:00.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.648+0000 7f3ae5400640 1 --2- 172.21.15.161:0/2561870058 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ac8066690 0x7f3ac8068b50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3ad401c380 tx=0x7f3ad40a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:00.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.648+0000 7f3ade000640 1 -- 172.21.15.161:0/2561870058 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3ad8051ab0 con 0x7f3acc0040a0 2024-05-22T21:23:00.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.868+0000 7f3aece00640 1 -- 172.21.15.161:0/2561870058 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3ae8067220 con 0x7f3acc0040a0 2024-05-22T21:23:00.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.868+0000 7f3ade000640 1 -- 172.21.15.161:0/2561870058 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3ad80550d0 con 0x7f3acc0040a0 2024-05-22T21:23:00.869 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:23:00.869 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:23:00.870 INFO:teuthology.orchestra.run.smithi161.stdout: "auth_dump": [ 2024-05-22T21:23:00.870 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.870 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.a", 2024-05-22T21:23:00.870 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m00QTFRAAPSRrpBRY75FlcWf3QxOOfg==", 2024-05-22T21:23:00.870 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.870 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:23:00.870 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:23:00.870 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:23:00.870 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:23:00.870 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.870 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.870 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.871 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.b", 2024-05-22T21:23:00.871 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m/ipAIRAA4gj5jGqTpNCr1K2OU/bsAQ==", 2024-05-22T21:23:00.871 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.871 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:23:00.871 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:23:00.871 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:23:00.871 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:23:00.871 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.871 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.871 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.871 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.c", 2024-05-22T21:23:00.872 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mWlM2HBAAEmJJWMDlTG9E7Qzld2gd5w==", 2024-05-22T21:23:00.872 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.872 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:23:00.872 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:23:00.872 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:23:00.872 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:23:00.872 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.872 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.872 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.872 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.d", 2024-05-22T21:23:00.872 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mmsuaKBAAqpCYRZZUGK5ZuxflUBcPgg==", 2024-05-22T21:23:00.872 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.872 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:23:00.873 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:23:00.873 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:23:00.873 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:23:00.873 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.873 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.873 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.873 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.0", 2024-05-22T21:23:00.873 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05mJZokLhAAh2vbQnRX25iYUIbvVmURuQ==", 2024-05-22T21:23:00.873 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.873 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:23:00.873 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:23:00.873 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:23:00.873 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.873 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.874 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.874 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.1", 2024-05-22T21:23:00.874 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05m8kWNNhAA6aaVT2xshFkqnk4G4gOttA==", 2024-05-22T21:23:00.874 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.874 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:23:00.874 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:23:00.874 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:23:00.874 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.874 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.874 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.874 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.2", 2024-05-22T21:23:00.874 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05ml1k9AxAAtL3XaJJI6cilpmvurh18Pw==", 2024-05-22T21:23:00.874 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.875 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:23:00.875 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:23:00.875 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:23:00.875 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.875 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.875 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.875 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.3", 2024-05-22T21:23:00.875 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05mbNBgCxAAas1L+QR4Aez+H1G0EVltfg==", 2024-05-22T21:23:00.875 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.876 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:23:00.876 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:23:00.876 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:23:00.876 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.876 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.876 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.876 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.4", 2024-05-22T21:23:00.876 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mOPLPABAAZBGIMF/FasNB27n/7di0AA==", 2024-05-22T21:23:00.876 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.877 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:23:00.877 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:23:00.877 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:23:00.877 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.877 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.877 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.877 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.5", 2024-05-22T21:23:00.877 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mT1poCRAASXm4tdP01pA1VOviqeoLVw==", 2024-05-22T21:23:00.877 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.877 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:23:00.877 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:23:00.877 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:23:00.877 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.877 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.878 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.878 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.6", 2024-05-22T21:23:00.878 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05m1ZPRERAA2S7WeTwysLCZm1OB/qCrAA==", 2024-05-22T21:23:00.878 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.878 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:23:00.878 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:23:00.878 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:23:00.878 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.878 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.878 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.878 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.7", 2024-05-22T21:23:00.878 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mCrUhGhAAvXoFWV7X3303rfXaxmfWqA==", 2024-05-22T21:23:00.879 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.879 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:23:00.879 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:23:00.879 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:23:00.879 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.879 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.879 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.879 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.0", 2024-05-22T21:23:00.879 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mh9B7LBAAevhvj5KpwP9pEIpJcxUvFA==", 2024-05-22T21:23:00.879 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.879 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:23:00.879 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow r", 2024-05-22T21:23:00.879 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-22T21:23:00.879 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.880 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.880 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.880 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.1", 2024-05-22T21:23:00.880 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mMnHtNRAAg/a/Mb1oRqx73d9efIHCfA==", 2024-05-22T21:23:00.880 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.880 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:23:00.880 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:23:00.880 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:23:00.880 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:23:00.880 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.880 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.880 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.880 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.2", 2024-05-22T21:23:00.880 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mOzdkMhAAbTiNdvyxqIyrluMiZW0cOw==", 2024-05-22T21:23:00.881 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.881 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:23:00.881 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:23:00.881 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:23:00.881 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:23:00.881 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.881 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.881 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.881 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.3", 2024-05-22T21:23:00.881 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDYX05mr4oxABAA1+bQQbxhoeU+kb0OCa+Wyw==", 2024-05-22T21:23:00.881 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.881 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:23:00.881 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:23:00.881 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:23:00.882 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:23:00.882 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.882 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.882 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.882 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.admin", 2024-05-22T21:23:00.882 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDWX05mHZY0NRAAiBvMDkrCZ+MgG2iwBH+bcg==", 2024-05-22T21:23:00.882 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.882 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:23:00.882 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:23:00.882 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:23:00.882 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:23:00.882 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.883 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.883 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.883 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.x", 2024-05-22T21:23:00.883 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mH2kuERAAgigMUWK3M3WaMXlqGUcNrw==", 2024-05-22T21:23:00.883 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.883 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:23:00.883 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:23:00.883 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:23:00.883 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.883 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:23:00.883 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:00.883 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.y", 2024-05-22T21:23:00.883 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mZRdiDRAA0vUwD+rPSfyhjgA+9XPp5w==", 2024-05-22T21:23:00.884 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:23:00.884 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:23:00.884 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:23:00.884 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:23:00.884 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.884 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:00.884 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:23:00.884 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:23:00.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.872+0000 7f3aece00640 1 -- 172.21.15.161:0/2561870058 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ac8066690 msgr2=0x7f3ac8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:00.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.872+0000 7f3aece00640 1 --2- 172.21.15.161:0/2561870058 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ac8066690 0x7f3ac8068b50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3ad401c380 tx=0x7f3ad40a7000 comp rx=0 tx=0).stop 2024-05-22T21:23:00.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.872+0000 7f3aece00640 1 -- 172.21.15.161:0/2561870058 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc0040a0 msgr2=0x7f3ae81e5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:00.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.872+0000 7f3aece00640 1 --2- 172.21.15.161:0/2561870058 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc0040a0 0x7f3ae81e5e30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8011960 tx=0x7f3ad8011e20 comp rx=0 tx=0).stop 2024-05-22T21:23:00.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.872+0000 7f3aece00640 1 -- 172.21.15.161:0/2561870058 shutdown_connections 2024-05-22T21:23:00.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.872+0000 7f3aece00640 1 --2- 172.21.15.161:0/2561870058 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ac8066690 0x7f3ac8068b50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:00.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.872+0000 7f3aece00640 1 --2- 172.21.15.161:0/2561870058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ae81f9270 0x7f3ae81f96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:00.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.872+0000 7f3aece00640 1 --2- 172.21.15.161:0/2561870058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ae80b3000 0x7f3ae8152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:00.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.872+0000 7f3aece00640 1 --2- 172.21.15.161:0/2561870058 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3acc0040a0 0x7f3ae81e5e30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:00.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.872+0000 7f3aece00640 1 -- 172.21.15.161:0/2561870058 >> 172.21.15.161:0/2561870058 conn(0x7f3ae80b0bc0 msgr2=0x7f3ae80adde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:00.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.872+0000 7f3aece00640 1 -- 172.21.15.161:0/2561870058 shutdown_connections 2024-05-22T21:23:00.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:00.872+0000 7f3aece00640 1 -- 172.21.15.161:0/2561870058 wait complete. 2024-05-22T21:23:00.891 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:23:01.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.028+0000 7f6a72c00640 1 Processor -- start 2024-05-22T21:23:01.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.028+0000 7f6a72c00640 1 -- start start 2024-05-22T21:23:01.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.028+0000 7f6a72c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6a6c0d0a40 0x7f6a6c0d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.028+0000 7f6a72c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c0cb8a0 con 0x7f6a6c15af00 2024-05-22T21:23:01.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.028+0000 7f6a72c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c0d13f0 con 0x7f6a6c0cc320 2024-05-22T21:23:01.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.028+0000 7f6a72c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6c15f530 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.028+0000 7f6a71800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f6a6c15af00 0x7f6a6c15eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:45156/0 (socket says 172.21.15.161:45156) 2024-05-22T21:23:01.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.028+0000 7f6a71800640 1 -- 172.21.15.161:0/1463090687 learned_addr learned my addr 172.21.15.161:0/1463090687 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:23:01.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.028+0000 7f6a70e00640 1 --2- 172.21.15.161:0/1463090687 >> v2:172.21.15.179:3300/0 conn(0x7f6a6c0d0a40 0x7f6a6c0d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:01.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.028+0000 7f6a6be00640 1 -- 172.21.15.161:0/1463090687 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210158279 0 0) 0x7f6a6c0cb8a0 con 0x7f6a6c15af00 2024-05-22T21:23:01.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.028+0000 7f6a6be00640 1 -- 172.21.15.161:0/1463090687 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a54003530 con 0x7f6a6c15af00 2024-05-22T21:23:01.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.028+0000 7f6a6be00640 1 -- 172.21.15.161:0/1463090687 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4178869090 0 0) 0x7f6a54003530 con 0x7f6a6c15af00 2024-05-22T21:23:01.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.028+0000 7f6a70e00640 1 -- 172.21.15.161:0/1463090687 >> v1:172.21.15.179:6789/0 conn(0x7f6a6c0cc320 legacy=0x7f6a6c0cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.028+0000 7f6a70e00640 1 -- 172.21.15.161:0/1463090687 >> v1:172.21.15.161:6789/0 conn(0x7f6a6c15af00 legacy=0x7f6a6c15eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.028+0000 7f6a70e00640 1 -- 172.21.15.161:0/1463090687 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c15f810 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.032+0000 7f6a70e00640 1 --2- 172.21.15.161:0/1463090687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a6c0d0a40 0x7f6a6c0d0eb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6a580097c0 tx=0x7f6a5801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=766f3972a68e35f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:01.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.032+0000 7f6a6be00640 1 -- 172.21.15.161:0/1463090687 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60001680 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.032+0000 7f6a6be00640 1 -- 172.21.15.161:0/1463090687 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a580a7050 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.032+0000 7f6a6be00640 1 -- 172.21.15.161:0/1463090687 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a580ab620 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.032+0000 7f6a72c00640 1 -- 172.21.15.161:0/1463090687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a6c0d0a40 msgr2=0x7f6a6c0d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.032+0000 7f6a72c00640 1 --2- 172.21.15.161:0/1463090687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a6c0d0a40 0x7f6a6c0d0eb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6a580097c0 tx=0x7f6a5801c3c0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.032+0000 7f6a72c00640 1 -- 172.21.15.161:0/1463090687 shutdown_connections 2024-05-22T21:23:01.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.032+0000 7f6a72c00640 1 --2- 172.21.15.161:0/1463090687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a6c0d0a40 0x7f6a6c0d0eb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.032+0000 7f6a72c00640 1 -- 172.21.15.161:0/1463090687 >> 172.21.15.161:0/1463090687 conn(0x7f6a6c099a40 msgr2=0x7f6a6c099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:01.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.032+0000 7f6a72c00640 1 -- 172.21.15.161:0/1463090687 shutdown_connections 2024-05-22T21:23:01.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.032+0000 7f6a72c00640 1 -- 172.21.15.161:0/1463090687 wait complete. 2024-05-22T21:23:01.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a72c00640 1 Processor -- start 2024-05-22T21:23:01.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a72c00640 1 -- start start 2024-05-22T21:23:01.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a72c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a6c0cc320 0x7f6a6c1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a72c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a6c0d0a40 0x7f6a6c0c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a72c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a6c15af00 0x7f6a6c15b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a72c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0c5240 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a72c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0c4980 con 0x7f6a6c15af00 2024-05-22T21:23:01.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a72c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0c4bf0 con 0x7f6a6c0cc320 2024-05-22T21:23:01.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a70e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a6c0d0a40 0x7f6a6c0c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:01.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a70e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a6c0d0a40 0x7f6a6c0c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41140/0 (socket says 172.21.15.161:41140) 2024-05-22T21:23:01.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a70e00640 1 -- 172.21.15.161:0/800395082 learned_addr learned my addr 172.21.15.161:0/800395082 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:01.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a70e00640 1 -- 172.21.15.161:0/800395082 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a6c0cc320 msgr2=0x7f6a6c1561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a70e00640 1 --2- 172.21.15.161:0/800395082 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a6c0cc320 0x7f6a6c1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a70e00640 1 -- 172.21.15.161:0/800395082 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a6c15af00 msgr2=0x7f6a6c15b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a70e00640 1 --2- 172.21.15.161:0/800395082 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a6c15af00 0x7f6a6c15b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a70e00640 1 -- 172.21.15.161:0/800395082 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a58009420 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a70e00640 1 --2- 172.21.15.161:0/800395082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a6c0d0a40 0x7f6a6c0c9b30 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f6a58018fd0 tx=0x7f6a58019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:01.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a69600640 1 -- 172.21.15.161:0/800395082 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a580ba070 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a69600640 1 -- 172.21.15.161:0/800395082 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a580abd40 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a69600640 1 -- 172.21.15.161:0/800395082 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a580b4600 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a72c00640 1 -- 172.21.15.161:0/800395082 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6c0c4ed0 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a72c00640 1 -- 172.21.15.161:0/800395082 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c165560 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.036+0000 7f6a72c00640 1 -- 172.21.15.161:0/800395082 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6c073630 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.040+0000 7f6a69600640 1 -- 172.21.15.161:0/800395082 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6a580b2040 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.040+0000 7f6a69600640 1 --2- 172.21.15.161:0/800395082 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a4c066690 0x7f6a4c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.040+0000 7f6a69600640 1 -- 172.21.15.161:0/800395082 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6a5811d590 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.040+0000 7f6a71800640 1 --2- 172.21.15.161:0/800395082 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a4c066690 0x7f6a4c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:01.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.040+0000 7f6a71800640 1 --2- 172.21.15.161:0/800395082 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a4c066690 0x7f6a4c068b50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6a60001ec0 tx=0x7f6a60001a10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:01.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.044+0000 7f6a69600640 1 -- 172.21.15.161:0/800395082 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6a580eac40 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.200+0000 7f6a72c00640 1 -- 172.21.15.161:0/800395082 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a6c0c5060 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.200+0000 7f6a69600640 1 -- 172.21.15.161:0/800395082 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+12448 (secure 0 0 0) 0x7f6a580ee260 con 0x7f6a6c0d0a40 2024-05-22T21:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:23:01.201 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":171,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:23:00.468134+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:23:01.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.204+0000 7f6a72c00640 1 -- 172.21.15.161:0/800395082 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a4c066690 msgr2=0x7f6a4c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.204+0000 7f6a72c00640 1 --2- 172.21.15.161:0/800395082 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a4c066690 0x7f6a4c068b50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6a60001ec0 tx=0x7f6a60001a10 comp rx=0 tx=0).stop 2024-05-22T21:23:01.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.204+0000 7f6a72c00640 1 -- 172.21.15.161:0/800395082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a6c0d0a40 msgr2=0x7f6a6c0c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.204+0000 7f6a72c00640 1 --2- 172.21.15.161:0/800395082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a6c0d0a40 0x7f6a6c0c9b30 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f6a58018fd0 tx=0x7f6a58019160 comp rx=0 tx=0).stop 2024-05-22T21:23:01.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.204+0000 7f6a72c00640 1 -- 172.21.15.161:0/800395082 shutdown_connections 2024-05-22T21:23:01.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.204+0000 7f6a72c00640 1 --2- 172.21.15.161:0/800395082 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6a4c066690 0x7f6a4c068b50 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.204+0000 7f6a72c00640 1 --2- 172.21.15.161:0/800395082 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a6c15af00 0x7f6a6c15b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.204+0000 7f6a72c00640 1 --2- 172.21.15.161:0/800395082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a6c0d0a40 0x7f6a6c0c9b30 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.204+0000 7f6a72c00640 1 --2- 172.21.15.161:0/800395082 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6a6c0cc320 0x7f6a6c1561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.204+0000 7f6a72c00640 1 -- 172.21.15.161:0/800395082 >> 172.21.15.161:0/800395082 conn(0x7f6a6c099a40 msgr2=0x7f6a6c06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:01.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.204+0000 7f6a72c00640 1 -- 172.21.15.161:0/800395082 shutdown_connections 2024-05-22T21:23:01.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.204+0000 7f6a72c00640 1 -- 172.21.15.161:0/800395082 wait complete. 2024-05-22T21:23:01.222 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:23:01.223 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:23:01.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.360+0000 7f73aee00640 1 Processor -- start 2024-05-22T21:23:01.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.360+0000 7f73aee00640 1 -- start start 2024-05-22T21:23:01.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.364+0000 7f73aee00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f73a813cfc0 0x7f73a81402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.364+0000 7f73aee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73a815af60 0x7f73a813f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.364+0000 7f73aee00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a81400d0 con 0x7f73a813fc80 2024-05-22T21:23:01.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.364+0000 7f73aee00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a813d3a0 con 0x7f73a813cfc0 2024-05-22T21:23:01.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.364+0000 7f73aee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80cd920 con 0x7f73a815af60 2024-05-22T21:23:01.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.364+0000 7f73a7e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f73a813cfc0 0x7f73a81402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:01.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.364+0000 7f73a7e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f73a813cfc0 0x7f73a81402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41142/0 (socket says 172.21.15.161:41142) 2024-05-22T21:23:01.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.364+0000 7f73a7e00640 1 -- 172.21.15.161:0/2008932502 learned_addr learned my addr 172.21.15.161:0/2008932502 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:01.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.364+0000 7f739f400640 1 --2- 172.21.15.161:0/2008932502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73a815af60 0x7f73a813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:01.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.364+0000 7f73a7e00640 1 -- 172.21.15.161:0/2008932502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73a815af60 msgr2=0x7f73a813f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:01.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.364+0000 7f73a7e00640 1 --2- 172.21.15.161:0/2008932502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73a815af60 0x7f73a813f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.364+0000 7f73a7e00640 1 -- 172.21.15.161:0/2008932502 >> v1:172.21.15.179:6789/0 conn(0x7f73a813fc80 legacy=0x7f73a815dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.364+0000 7f73a7e00640 1 -- 172.21.15.161:0/2008932502 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a80cdc00 con 0x7f73a813cfc0 2024-05-22T21:23:01.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.368+0000 7f73a7e00640 1 --2- 172.21.15.161:0/2008932502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a813cfc0 0x7f73a81402e0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f7398009870 tx=0x7f739801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a90aca39668b22 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:01.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.368+0000 7f73a7400640 1 -- 172.21.15.161:0/2008932502 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73980ab600 con 0x7f73a813cfc0 2024-05-22T21:23:01.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.368+0000 7f73a7400640 1 -- 172.21.15.161:0/2008932502 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73980a7050 con 0x7f73a813cfc0 2024-05-22T21:23:01.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.368+0000 7f73a7400640 1 -- 172.21.15.161:0/2008932502 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73980ab980 con 0x7f73a813cfc0 2024-05-22T21:23:01.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.368+0000 7f73aee00640 1 -- 172.21.15.161:0/2008932502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a813cfc0 msgr2=0x7f73a81402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.368+0000 7f73aee00640 1 --2- 172.21.15.161:0/2008932502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a813cfc0 0x7f73a81402e0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f7398009870 tx=0x7f739801c4d0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.368+0000 7f73aee00640 1 -- 172.21.15.161:0/2008932502 shutdown_connections 2024-05-22T21:23:01.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.368+0000 7f73aee00640 1 --2- 172.21.15.161:0/2008932502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73a815af60 0x7f73a813f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.368+0000 7f73aee00640 1 --2- 172.21.15.161:0/2008932502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a813cfc0 0x7f73a81402e0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.368+0000 7f73aee00640 1 -- 172.21.15.161:0/2008932502 >> 172.21.15.161:0/2008932502 conn(0x7f73a806a2d0 msgr2=0x7f73a806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:01.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.368+0000 7f73aee00640 1 -- 172.21.15.161:0/2008932502 shutdown_connections 2024-05-22T21:23:01.368 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.368+0000 7f73aee00640 1 -- 172.21.15.161:0/2008932502 wait complete. 2024-05-22T21:23:01.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.368+0000 7f73aee00640 1 Processor -- start 2024-05-22T21:23:01.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.368+0000 7f73aee00640 1 -- start start 2024-05-22T21:23:01.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f73aee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73a813cfc0 0x7f73a8157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f73aee00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a813fc80 0x7f73a8159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f73aee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73a815af60 0x7f73a813ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f73aee00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80b2220 con 0x7f73a813fc80 2024-05-22T21:23:01.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f73aee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80b24c0 con 0x7f73a813cfc0 2024-05-22T21:23:01.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f73aee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80b27a0 con 0x7f73a815af60 2024-05-22T21:23:01.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f739f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a813fc80 0x7f73a8159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:01.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f739f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a813fc80 0x7f73a8159590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41148/0 (socket says 172.21.15.161:41148) 2024-05-22T21:23:01.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f739f400640 1 -- 172.21.15.161:0/1950242227 learned_addr learned my addr 172.21.15.161:0/1950242227 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:01.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f73a7e00640 1 --2- 172.21.15.161:0/1950242227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73a813cfc0 0x7f73a8157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:01.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f73ace00640 1 --2- 172.21.15.161:0/1950242227 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73a815af60 0x7f73a813ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:01.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f739f400640 1 -- 172.21.15.161:0/1950242227 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73a815af60 msgr2=0x7f73a813ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f739f400640 1 --2- 172.21.15.161:0/1950242227 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73a815af60 0x7f73a813ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f739f400640 1 -- 172.21.15.161:0/1950242227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73a813cfc0 msgr2=0x7f73a8157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f739f400640 1 --2- 172.21.15.161:0/1950242227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73a813cfc0 0x7f73a8157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f739f400640 1 -- 172.21.15.161:0/1950242227 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73980094d0 con 0x7f73a813fc80 2024-05-22T21:23:01.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f73ace00640 1 --2- 172.21.15.161:0/1950242227 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73a815af60 0x7f73a813ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:23:01.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f739f400640 1 --2- 172.21.15.161:0/1950242227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a813fc80 0x7f73a8159590 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f739000e930 tx=0x7f739000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:01.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f73a4c00640 1 -- 172.21.15.161:0/1950242227 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7390009690 con 0x7f73a813fc80 2024-05-22T21:23:01.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f73a4c00640 1 -- 172.21.15.161:0/1950242227 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739000ca00 con 0x7f73a813fc80 2024-05-22T21:23:01.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f73aee00640 1 -- 172.21.15.161:0/1950242227 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a80b2a80 con 0x7f73a813fc80 2024-05-22T21:23:01.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f73a4c00640 1 -- 172.21.15.161:0/1950242227 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739000cd70 con 0x7f73a813fc80 2024-05-22T21:23:01.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f73aee00640 1 -- 172.21.15.161:0/1950242227 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a8140270 con 0x7f73a813fc80 2024-05-22T21:23:01.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f73aee00640 1 -- 172.21.15.161:0/1950242227 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a80b2c10 con 0x7f73a813fc80 2024-05-22T21:23:01.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.372+0000 7f73a4c00640 1 -- 172.21.15.161:0/1950242227 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7390013470 con 0x7f73a813fc80 2024-05-22T21:23:01.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.376+0000 7f73a4c00640 1 --2- 172.21.15.161:0/1950242227 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7384066690 0x7f7384068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.376+0000 7f73a4c00640 1 -- 172.21.15.161:0/1950242227 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7390084040 con 0x7f73a813fc80 2024-05-22T21:23:01.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.376+0000 7f73a7e00640 1 --2- 172.21.15.161:0/1950242227 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7384066690 0x7f7384068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:01.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.376+0000 7f73a7e00640 1 --2- 172.21.15.161:0/1950242227 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7384066690 0x7f7384068b50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f73980ad000 tx=0x7f73980a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:01.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.380+0000 7f73a4c00640 1 -- 172.21.15.161:0/1950242227 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7390051870 con 0x7f73a813fc80 2024-05-22T21:23:01.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.536+0000 7f73aee00640 1 -- 172.21.15.161:0/1950242227 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73a8067220 con 0x7f73a813fc80 2024-05-22T21:23:01.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.536+0000 7f73a4c00640 1 -- 172.21.15.161:0/1950242227 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v171) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7390017020 con 0x7f73a813fc80 2024-05-22T21:23:01.537 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:23:01.538 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.540+0000 7f73aee00640 1 -- 172.21.15.161:0/1950242227 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7384066690 msgr2=0x7f7384068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.540+0000 7f73aee00640 1 --2- 172.21.15.161:0/1950242227 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7384066690 0x7f7384068b50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f73980ad000 tx=0x7f73980a6000 comp rx=0 tx=0).stop 2024-05-22T21:23:01.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.540+0000 7f73aee00640 1 -- 172.21.15.161:0/1950242227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a813fc80 msgr2=0x7f73a8159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.540+0000 7f73aee00640 1 --2- 172.21.15.161:0/1950242227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a813fc80 0x7f73a8159590 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f739000e930 tx=0x7f739000edf0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.540+0000 7f73aee00640 1 -- 172.21.15.161:0/1950242227 shutdown_connections 2024-05-22T21:23:01.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.540+0000 7f73aee00640 1 --2- 172.21.15.161:0/1950242227 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7384066690 0x7f7384068b50 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.540+0000 7f73aee00640 1 --2- 172.21.15.161:0/1950242227 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f73a815af60 0x7f73a813ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.540+0000 7f73aee00640 1 --2- 172.21.15.161:0/1950242227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a813fc80 0x7f73a8159590 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.540+0000 7f73aee00640 1 --2- 172.21.15.161:0/1950242227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73a813cfc0 0x7f73a8157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.540+0000 7f73aee00640 1 -- 172.21.15.161:0/1950242227 >> 172.21.15.161:0/1950242227 conn(0x7f73a806a2d0 msgr2=0x7f73a806d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:01.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.540+0000 7f73aee00640 1 -- 172.21.15.161:0/1950242227 shutdown_connections 2024-05-22T21:23:01.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.540+0000 7f73aee00640 1 -- 172.21.15.161:0/1950242227 wait complete. 2024-05-22T21:23:01.559 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-22T21:23:01.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.696+0000 7f8f6f000640 1 Processor -- start 2024-05-22T21:23:01.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.696+0000 7f8f6f000640 1 -- start start 2024-05-22T21:23:01.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.696+0000 7f8f6f000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f680b4fa0 0x7f8f6815e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.696+0000 7f8f6f000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8f68160f70 0x7f8f681613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.696+0000 7f8f6f000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6815e000 con 0x7f8f68069ed0 2024-05-22T21:23:01.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.696+0000 7f8f6f000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6815e9d0 con 0x7f8f68160f70 2024-05-22T21:23:01.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.696+0000 7f8f6f000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f68161a30 con 0x7f8f680b4fa0 2024-05-22T21:23:01.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.696+0000 7f8f67400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8f68160f70 0x7f8f681613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:01.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.696+0000 7f8f67e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f680b4fa0 0x7f8f6815e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:01.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.696+0000 7f8f67e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f680b4fa0 0x7f8f6815e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:43112/0 (socket says 172.21.15.161:43112) 2024-05-22T21:23:01.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.696+0000 7f8f67e00640 1 -- 172.21.15.161:0/1091043203 learned_addr learned my addr 172.21.15.161:0/1091043203 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:01.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.696+0000 7f8f67400640 1 -- 172.21.15.161:0/1091043203 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f680b4fa0 msgr2=0x7f8f6815e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:01.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.696+0000 7f8f67400640 1 --2- 172.21.15.161:0/1091043203 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f680b4fa0 0x7f8f6815e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.696+0000 7f8f67400640 1 -- 172.21.15.161:0/1091043203 >> v1:172.21.15.179:6789/0 conn(0x7f8f68069ed0 legacy=0x7f8f680cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.696+0000 7f8f67400640 1 -- 172.21.15.161:0/1091043203 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f68161d10 con 0x7f8f68160f70 2024-05-22T21:23:01.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.700+0000 7f8f67400640 1 --2- 172.21.15.161:0/1091043203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f68160f70 0x7f8f681613f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8f540097c0 tx=0x7f8f5401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35af2c4d284eef9f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:01.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.700+0000 7f8f66a00640 1 -- 172.21.15.161:0/1091043203 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f540ab610 con 0x7f8f68160f70 2024-05-22T21:23:01.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.700+0000 7f8f66a00640 1 -- 172.21.15.161:0/1091043203 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f540ab770 con 0x7f8f68160f70 2024-05-22T21:23:01.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.700+0000 7f8f66a00640 1 -- 172.21.15.161:0/1091043203 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f540abb00 con 0x7f8f68160f70 2024-05-22T21:23:01.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.700+0000 7f8f6f000640 1 -- 172.21.15.161:0/1091043203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f68160f70 msgr2=0x7f8f681613f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.700+0000 7f8f6f000640 1 --2- 172.21.15.161:0/1091043203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f68160f70 0x7f8f681613f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8f540097c0 tx=0x7f8f5401c470 comp rx=0 tx=0).stop 2024-05-22T21:23:01.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.700+0000 7f8f6f000640 1 -- 172.21.15.161:0/1091043203 shutdown_connections 2024-05-22T21:23:01.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.700+0000 7f8f6f000640 1 --2- 172.21.15.161:0/1091043203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f68160f70 0x7f8f681613f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.700+0000 7f8f6f000640 1 --2- 172.21.15.161:0/1091043203 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f680b4fa0 0x7f8f6815e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.700+0000 7f8f6f000640 1 -- 172.21.15.161:0/1091043203 >> 172.21.15.161:0/1091043203 conn(0x7f8f680b1c40 msgr2=0x7f8f680b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:01.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.700+0000 7f8f6f000640 1 -- 172.21.15.161:0/1091043203 shutdown_connections 2024-05-22T21:23:01.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.700+0000 7f8f6f000640 1 -- 172.21.15.161:0/1091043203 wait complete. 2024-05-22T21:23:01.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f6f000640 1 Processor -- start 2024-05-22T21:23:01.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f6f000640 1 -- start start 2024-05-22T21:23:01.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f6f000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f68069ed0 0x7f8f68140d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f6f000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f680b4fa0 0x7f8f680ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f6f000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f68160f70 0x7f8f681415c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f6f000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f680aed00 con 0x7f8f680b4fa0 2024-05-22T21:23:01.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f6f000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6806d090 con 0x7f8f68160f70 2024-05-22T21:23:01.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f6f000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f680aef30 con 0x7f8f68069ed0 2024-05-22T21:23:01.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f67400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f680b4fa0 0x7f8f680ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:01.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f67400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f680b4fa0 0x7f8f680ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41154/0 (socket says 172.21.15.161:41154) 2024-05-22T21:23:01.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f67400640 1 -- 172.21.15.161:0/3212202508 learned_addr learned my addr 172.21.15.161:0/3212202508 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:01.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f67e00640 1 --2- 172.21.15.161:0/3212202508 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f68069ed0 0x7f8f68140d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:01.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f6d000640 1 --2- 172.21.15.161:0/3212202508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f68160f70 0x7f8f681415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:01.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f67400640 1 -- 172.21.15.161:0/3212202508 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f68069ed0 msgr2=0x7f8f68140d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f67400640 1 --2- 172.21.15.161:0/3212202508 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f68069ed0 0x7f8f68140d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f67400640 1 -- 172.21.15.161:0/3212202508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f68160f70 msgr2=0x7f8f681415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f67400640 1 --2- 172.21.15.161:0/3212202508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f68160f70 0x7f8f681415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f67400640 1 -- 172.21.15.161:0/3212202508 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f54009420 con 0x7f8f680b4fa0 2024-05-22T21:23:01.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f67400640 1 --2- 172.21.15.161:0/3212202508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f680b4fa0 0x7f8f680ae7c0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f8f540ad040 tx=0x7f8f5401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:01.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f5fe00640 1 -- 172.21.15.161:0/3212202508 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f540bf610 con 0x7f8f680b4fa0 2024-05-22T21:23:01.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f5fe00640 1 -- 172.21.15.161:0/3212202508 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f540bf770 con 0x7f8f680b4fa0 2024-05-22T21:23:01.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f5fe00640 1 -- 172.21.15.161:0/3212202508 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f540bfb20 con 0x7f8f680b4fa0 2024-05-22T21:23:01.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f6f000640 1 -- 172.21.15.161:0/3212202508 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6806a870 con 0x7f8f680b4fa0 2024-05-22T21:23:01.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f6f000640 1 -- 172.21.15.161:0/3212202508 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6806d5c0 con 0x7f8f680b4fa0 2024-05-22T21:23:01.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.704+0000 7f8f6f000640 1 -- 172.21.15.161:0/3212202508 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f68067220 con 0x7f8f680b4fa0 2024-05-22T21:23:01.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.708+0000 7f8f5fe00640 1 -- 172.21.15.161:0/3212202508 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8f540bfc80 con 0x7f8f680b4fa0 2024-05-22T21:23:01.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.708+0000 7f8f5fe00640 1 --2- 172.21.15.161:0/3212202508 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8f48066690 0x7f8f48068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:01.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.708+0000 7f8f5fe00640 1 -- 172.21.15.161:0/3212202508 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8f5411e8c0 con 0x7f8f680b4fa0 2024-05-22T21:23:01.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.708+0000 7f8f67e00640 1 --2- 172.21.15.161:0/3212202508 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8f48066690 0x7f8f48068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:01.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.708+0000 7f8f67e00640 1 --2- 172.21.15.161:0/3212202508 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8f48066690 0x7f8f48068b50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009b50 tx=0x7f8f58009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.712+0000 7f8f5fe00640 1 -- 172.21.15.161:0/3212202508 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8f540ec020 con 0x7f8f680b4fa0 2024-05-22T21:23:01.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.868+0000 7f8f6f000640 1 -- 172.21.15.161:0/3212202508 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8f68071290 con 0x7f8f680b4fa0 2024-05-22T21:23:01.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.868+0000 7f8f5fe00640 1 -- 172.21.15.161:0/3212202508 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v171) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8f540ef640 con 0x7f8f680b4fa0 2024-05-22T21:23:01.868 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:23:01.868 INFO:teuthology.orchestra.run.smithi161.stdout:[ 2024-05-22T21:23:01.868 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:23:01.868 INFO:teuthology.orchestra.run.smithi161.stdout: "poolnum": 1, 2024-05-22T21:23:01.868 INFO:teuthology.orchestra.run.smithi161.stdout: "poolname": ".mgr" 2024-05-22T21:23:01.868 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:23:01.868 INFO:teuthology.orchestra.run.smithi161.stdout:] 2024-05-22T21:23:01.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.868+0000 7f8f6f000640 1 -- 172.21.15.161:0/3212202508 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8f48066690 msgr2=0x7f8f48068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.868+0000 7f8f6f000640 1 --2- 172.21.15.161:0/3212202508 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8f48066690 0x7f8f48068b50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009b50 tx=0x7f8f58009240 comp rx=0 tx=0).stop 2024-05-22T21:23:01.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.868+0000 7f8f6f000640 1 -- 172.21.15.161:0/3212202508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f680b4fa0 msgr2=0x7f8f680ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:01.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.868+0000 7f8f6f000640 1 --2- 172.21.15.161:0/3212202508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f680b4fa0 0x7f8f680ae7c0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f8f540ad040 tx=0x7f8f5401acb0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.868+0000 7f8f6f000640 1 -- 172.21.15.161:0/3212202508 shutdown_connections 2024-05-22T21:23:01.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.868+0000 7f8f6f000640 1 --2- 172.21.15.161:0/3212202508 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8f48066690 0x7f8f48068b50 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.868+0000 7f8f6f000640 1 --2- 172.21.15.161:0/3212202508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f68160f70 0x7f8f681415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.868+0000 7f8f6f000640 1 --2- 172.21.15.161:0/3212202508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f680b4fa0 0x7f8f680ae7c0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.868+0000 7f8f6f000640 1 --2- 172.21.15.161:0/3212202508 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f68069ed0 0x7f8f68140d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:01.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.868+0000 7f8f6f000640 1 -- 172.21.15.161:0/3212202508 >> 172.21.15.161:0/3212202508 conn(0x7f8f680b1c40 msgr2=0x7f8f680cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:01.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.868+0000 7f8f6f000640 1 -- 172.21.15.161:0/3212202508 shutdown_connections 2024-05-22T21:23:01.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:01.868+0000 7f8f6f000640 1 -- 172.21.15.161:0/3212202508 wait complete. 2024-05-22T21:23:01.887 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-22T21:23:01.888 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-22T21:23:02.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.024+0000 7fa6e5800640 1 Processor -- start 2024-05-22T21:23:02.026 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.024+0000 7fa6e5800640 1 -- start start 2024-05-22T21:23:02.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.028+0000 7fa6e5800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6e013cfe0 0x7fa6e015dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:02.027 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.028+0000 7fa6e5800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa6e00ce210 0x7fa6e015e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:02.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.028+0000 7fa6e5800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e00ce830 con 0x7fa6e013d560 2024-05-22T21:23:02.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.028+0000 7fa6e5800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e00ce650 con 0x7fa6e00ce210 2024-05-22T21:23:02.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.028+0000 7fa6e5800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e0140560 con 0x7fa6e013cfe0 2024-05-22T21:23:02.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.028+0000 7fa6df400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa6e00ce210 0x7fa6e015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:02.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.028+0000 7fa6d6000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa6e013d560 0x7fa6e00cdb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:45166/0 (socket says 172.21.15.161:45166) 2024-05-22T21:23:02.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.028+0000 7fa6d6000640 1 -- 172.21.15.161:0/3301682225 learned_addr learned my addr 172.21.15.161:0/3301682225 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:23:02.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.028+0000 7fa6dea00640 1 --2- 172.21.15.161:0/3301682225 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6e013cfe0 0x7fa6e015dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:02.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.028+0000 7fa6df400640 1 -- 172.21.15.161:0/3301682225 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6e013cfe0 msgr2=0x7fa6e015dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:02.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.028+0000 7fa6df400640 1 --2- 172.21.15.161:0/3301682225 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6e013cfe0 0x7fa6e015dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:02.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.028+0000 7fa6df400640 1 -- 172.21.15.161:0/3301682225 >> v1:172.21.15.161:6789/0 conn(0x7fa6e013d560 legacy=0x7fa6e00cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:02.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.028+0000 7fa6df400640 1 -- 172.21.15.161:0/3301682225 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e00ceb10 con 0x7fa6e00ce210 2024-05-22T21:23:02.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.028+0000 7fa6dea00640 1 --2- 172.21.15.161:0/3301682225 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6e013cfe0 0x7fa6e015dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:23:02.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6df400640 1 --2- 172.21.15.161:0/3301682225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6e00ce210 0x7fa6e015e000 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80097c0 tx=0x7fa6d801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4ee1604c25dfb server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:02.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6de000640 1 -- 172.21.15.161:0/3301682225 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d80ac640 con 0x7fa6e00ce210 2024-05-22T21:23:02.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6de000640 1 -- 172.21.15.161:0/3301682225 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d80ac7a0 con 0x7fa6e00ce210 2024-05-22T21:23:02.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6de000640 1 -- 172.21.15.161:0/3301682225 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d80acb30 con 0x7fa6e00ce210 2024-05-22T21:23:02.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6e5800640 1 -- 172.21.15.161:0/3301682225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6e00ce210 msgr2=0x7fa6e015e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:02.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6e5800640 1 --2- 172.21.15.161:0/3301682225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6e00ce210 0x7fa6e015e000 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80097c0 tx=0x7fa6d801c3e0 comp rx=0 tx=0).stop 2024-05-22T21:23:02.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6e5800640 1 -- 172.21.15.161:0/3301682225 shutdown_connections 2024-05-22T21:23:02.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6e5800640 1 --2- 172.21.15.161:0/3301682225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6e00ce210 0x7fa6e015e000 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:02.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6e5800640 1 --2- 172.21.15.161:0/3301682225 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6e013cfe0 0x7fa6e015dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:02.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6e5800640 1 -- 172.21.15.161:0/3301682225 >> 172.21.15.161:0/3301682225 conn(0x7fa6e006a0a0 msgr2=0x7fa6e006a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:02.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6e5800640 1 -- 172.21.15.161:0/3301682225 shutdown_connections 2024-05-22T21:23:02.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6e5800640 1 -- 172.21.15.161:0/3301682225 wait complete. 2024-05-22T21:23:02.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6e5800640 1 Processor -- start 2024-05-22T21:23:02.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6e5800640 1 -- start start 2024-05-22T21:23:02.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6e5800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6e00ce210 0x7fa6e0137b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:02.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6e5800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6e013cfe0 0x7fa6e0139e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:02.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.032+0000 7fa6e5800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6e013d560 0x7fa6e015c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:02.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6e5800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e0139130 con 0x7fa6e013cfe0 2024-05-22T21:23:02.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6e5800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e0138090 con 0x7fa6e00ce210 2024-05-22T21:23:02.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6e5800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e0138210 con 0x7fa6e013d560 2024-05-22T21:23:02.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6d6000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6e013cfe0 0x7fa6e0139e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:02.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6d6000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6e013cfe0 0x7fa6e0139e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41176/0 (socket says 172.21.15.161:41176) 2024-05-22T21:23:02.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6d6000640 1 -- 172.21.15.161:0/2303518437 learned_addr learned my addr 172.21.15.161:0/2303518437 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:02.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6dea00640 1 --2- 172.21.15.161:0/2303518437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6e00ce210 0x7fa6e0137b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:02.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6df400640 1 --2- 172.21.15.161:0/2303518437 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6e013d560 0x7fa6e015c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:02.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6d6000640 1 -- 172.21.15.161:0/2303518437 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6e013d560 msgr2=0x7fa6e015c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:02.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6d6000640 1 --2- 172.21.15.161:0/2303518437 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6e013d560 0x7fa6e015c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:02.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6d6000640 1 -- 172.21.15.161:0/2303518437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6e00ce210 msgr2=0x7fa6e0137b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:02.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6d6000640 1 --2- 172.21.15.161:0/2303518437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6e00ce210 0x7fa6e0137b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:02.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6d6000640 1 -- 172.21.15.161:0/2303518437 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d8009420 con 0x7fa6e013cfe0 2024-05-22T21:23:02.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6d6000640 1 --2- 172.21.15.161:0/2303518437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6e013cfe0 0x7fa6e0139e00 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800c3a0 tx=0x7fa6c800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:02.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6d7400640 1 -- 172.21.15.161:0/2303518437 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8005cf0 con 0x7fa6e013cfe0 2024-05-22T21:23:02.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6d7400640 1 -- 172.21.15.161:0/2303518437 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c8005e50 con 0x7fa6e013cfe0 2024-05-22T21:23:02.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6d7400640 1 -- 172.21.15.161:0/2303518437 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c80072d0 con 0x7fa6e013cfe0 2024-05-22T21:23:02.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6e5800640 1 -- 172.21.15.161:0/2303518437 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e01384b0 con 0x7fa6e013cfe0 2024-05-22T21:23:02.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6e5800640 1 -- 172.21.15.161:0/2303518437 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e0138a70 con 0x7fa6e013cfe0 2024-05-22T21:23:02.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6e5800640 1 -- 172.21.15.161:0/2303518437 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e015b2a0 con 0x7fa6e013cfe0 2024-05-22T21:23:02.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6d7400640 1 -- 172.21.15.161:0/2303518437 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa6c8007430 con 0x7fa6e013cfe0 2024-05-22T21:23:02.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.036+0000 7fa6d7400640 1 --2- 172.21.15.161:0/2303518437 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6bc066690 0x7fa6bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:02.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.040+0000 7fa6d7400640 1 -- 172.21.15.161:0/2303518437 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa6c8083350 con 0x7fa6e013cfe0 2024-05-22T21:23:02.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.040+0000 7fa6dea00640 1 --2- 172.21.15.161:0/2303518437 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6bc066690 0x7fa6bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:02.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.040+0000 7fa6dea00640 1 --2- 172.21.15.161:0/2303518437 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6bc066690 0x7fa6bc068b50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0003b90 tx=0x7fa6d0009890 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:02.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.044+0000 7fa6d7400640 1 -- 172.21.15.161:0/2303518437 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa6c8050990 con 0x7fa6e013cfe0 2024-05-22T21:23:02.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.196+0000 7fa6e5800640 1 -- 172.21.15.161:0/2303518437 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa6e0067220 con 0x7fa6e013cfe0 2024-05-22T21:23:02.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.480+0000 7fa6d7400640 1 -- 172.21.15.161:0/2303518437 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v172) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa6c8053fb0 con 0x7fa6e013cfe0 2024-05-22T21:23:02.480 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' created 2024-05-22T21:23:02.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.484+0000 7fa6e5800640 1 -- 172.21.15.161:0/2303518437 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6bc066690 msgr2=0x7fa6bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:02.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.484+0000 7fa6e5800640 1 --2- 172.21.15.161:0/2303518437 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6bc066690 0x7fa6bc068b50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0003b90 tx=0x7fa6d0009890 comp rx=0 tx=0).stop 2024-05-22T21:23:02.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.484+0000 7fa6e5800640 1 -- 172.21.15.161:0/2303518437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6e013cfe0 msgr2=0x7fa6e0139e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:02.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.484+0000 7fa6e5800640 1 --2- 172.21.15.161:0/2303518437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6e013cfe0 0x7fa6e0139e00 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800c3a0 tx=0x7fa6c800c860 comp rx=0 tx=0).stop 2024-05-22T21:23:02.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.484+0000 7fa6e5800640 1 -- 172.21.15.161:0/2303518437 shutdown_connections 2024-05-22T21:23:02.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.484+0000 7fa6e5800640 1 --2- 172.21.15.161:0/2303518437 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6bc066690 0x7fa6bc068b50 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:02.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.484+0000 7fa6e5800640 1 --2- 172.21.15.161:0/2303518437 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa6e013d560 0x7fa6e015c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:02.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.484+0000 7fa6e5800640 1 --2- 172.21.15.161:0/2303518437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6e013cfe0 0x7fa6e0139e00 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:02.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.484+0000 7fa6e5800640 1 --2- 172.21.15.161:0/2303518437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6e00ce210 0x7fa6e0137b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:02.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.484+0000 7fa6e5800640 1 -- 172.21.15.161:0/2303518437 >> 172.21.15.161:0/2303518437 conn(0x7fa6e006a0a0 msgr2=0x7fa6e006ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:02.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.484+0000 7fa6e5800640 1 -- 172.21.15.161:0/2303518437 shutdown_connections 2024-05-22T21:23:02.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.484+0000 7fa6e5800640 1 -- 172.21.15.161:0/2303518437 wait complete. 2024-05-22T21:23:02.504 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-22T21:23:02.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.648+0000 7f80aa400640 1 Processor -- start 2024-05-22T21:23:02.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.648+0000 7f80aa400640 1 -- start start 2024-05-22T21:23:02.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.648+0000 7f80aa400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f80a41402e0 0x7f80a413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:02.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.648+0000 7f80aa400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a4140090 con 0x7f80a413cfc0 2024-05-22T21:23:02.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.648+0000 7f80aa400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a413d3a0 con 0x7f80a413fc80 2024-05-22T21:23:02.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.648+0000 7f80aa400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a40cd920 con 0x7f80a41402e0 2024-05-22T21:23:02.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.648+0000 7f809be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f80a41402e0 0x7f80a413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:02.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.648+0000 7f809be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f80a41402e0 0x7f80a413f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41184/0 (socket says 172.21.15.161:41184) 2024-05-22T21:23:02.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.648+0000 7f809be00640 1 -- 172.21.15.161:0/642204684 learned_addr learned my addr 172.21.15.161:0/642204684 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:02.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.648+0000 7f809be00640 1 -- 172.21.15.161:0/642204684 >> v1:172.21.15.179:6789/0 conn(0x7f80a413fc80 legacy=0x7f80a406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:02.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.648+0000 7f809be00640 1 -- 172.21.15.161:0/642204684 >> v1:172.21.15.161:6789/0 conn(0x7f80a413cfc0 legacy=0x7f80a415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:02.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.648+0000 7f809be00640 1 -- 172.21.15.161:0/642204684 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a40cdc00 con 0x7f80a41402e0 2024-05-22T21:23:02.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.652+0000 7f809be00640 1 --2- 172.21.15.161:0/642204684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a41402e0 0x7f80a413f740 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f808c009870 tx=0x7f808c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4574cacd578ad419 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:02.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.652+0000 7f80a2a00640 1 -- 172.21.15.161:0/642204684 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c0a6040 con 0x7f80a41402e0 2024-05-22T21:23:02.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.652+0000 7f80a2a00640 1 -- 172.21.15.161:0/642204684 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808c0aa400 con 0x7f80a41402e0 2024-05-22T21:23:02.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.652+0000 7f80a2a00640 1 -- 172.21.15.161:0/642204684 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c0aa790 con 0x7f80a41402e0 2024-05-22T21:23:02.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.652+0000 7f80aa400640 1 -- 172.21.15.161:0/642204684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a41402e0 msgr2=0x7f80a413f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:02.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.652+0000 7f80aa400640 1 --2- 172.21.15.161:0/642204684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a41402e0 0x7f80a413f740 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f808c009870 tx=0x7f808c01c4a0 comp rx=0 tx=0).stop 2024-05-22T21:23:02.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.652+0000 7f80aa400640 1 -- 172.21.15.161:0/642204684 shutdown_connections 2024-05-22T21:23:02.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.652+0000 7f80aa400640 1 --2- 172.21.15.161:0/642204684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a41402e0 0x7f80a413f740 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:02.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.652+0000 7f80aa400640 1 -- 172.21.15.161:0/642204684 >> 172.21.15.161:0/642204684 conn(0x7f80a406a2d0 msgr2=0x7f80a406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:02.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.652+0000 7f80aa400640 1 -- 172.21.15.161:0/642204684 shutdown_connections 2024-05-22T21:23:02.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.652+0000 7f80aa400640 1 -- 172.21.15.161:0/642204684 wait complete. 2024-05-22T21:23:02.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.652+0000 7f80aa400640 1 Processor -- start 2024-05-22T21:23:02.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.652+0000 7f80aa400640 1 -- start start 2024-05-22T21:23:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80aa400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a413cfc0 0x7f80a4155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80aa400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80a413fc80 0x7f80a413e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80aa400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80a41402e0 0x7f80a415da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80aa400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a4155ec0 con 0x7f80a413cfc0 2024-05-22T21:23:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80aa400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a413b820 con 0x7f80a41402e0 2024-05-22T21:23:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80aa400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a413bff0 con 0x7f80a413fc80 2024-05-22T21:23:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80a3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a413cfc0 0x7f80a4155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80a3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a413cfc0 0x7f80a4155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41200/0 (socket says 172.21.15.161:41200) 2024-05-22T21:23:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80a3400640 1 -- 172.21.15.161:0/4084865090 learned_addr learned my addr 172.21.15.161:0/4084865090 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f809be00640 1 --2- 172.21.15.161:0/4084865090 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80a413fc80 0x7f80a413e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80a3e00640 1 --2- 172.21.15.161:0/4084865090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80a41402e0 0x7f80a415da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80a3400640 1 -- 172.21.15.161:0/4084865090 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80a413fc80 msgr2=0x7f80a413e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80a3400640 1 --2- 172.21.15.161:0/4084865090 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80a413fc80 0x7f80a413e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80a3400640 1 -- 172.21.15.161:0/4084865090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80a41402e0 msgr2=0x7f80a415da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80a3400640 1 --2- 172.21.15.161:0/4084865090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80a41402e0 0x7f80a415da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80a3400640 1 -- 172.21.15.161:0/4084865090 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c0094d0 con 0x7f80a413cfc0 2024-05-22T21:23:02.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80a3400640 1 --2- 172.21.15.161:0/4084865090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a413cfc0 0x7f80a4155980 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f809400c3f0 tx=0x7f809400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:02.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f809b400640 1 -- 172.21.15.161:0/4084865090 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80940052d0 con 0x7f80a413cfc0 2024-05-22T21:23:02.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f809b400640 1 -- 172.21.15.161:0/4084865090 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8094005430 con 0x7f80a413cfc0 2024-05-22T21:23:02.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f809b400640 1 -- 172.21.15.161:0/4084865090 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094003840 con 0x7f80a413cfc0 2024-05-22T21:23:02.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80aa400640 1 -- 172.21.15.161:0/4084865090 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a413ba70 con 0x7f80a413cfc0 2024-05-22T21:23:02.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80aa400640 1 -- 172.21.15.161:0/4084865090 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a413e620 con 0x7f80a413cfc0 2024-05-22T21:23:02.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.656+0000 7f80aa400640 1 -- 172.21.15.161:0/4084865090 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a4067220 con 0x7f80a413cfc0 2024-05-22T21:23:02.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.660+0000 7f809b400640 1 -- 172.21.15.161:0/4084865090 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f80940039a0 con 0x7f80a413cfc0 2024-05-22T21:23:02.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.660+0000 7f809b400640 1 --2- 172.21.15.161:0/4084865090 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8080066690 0x7f8080068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:02.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.660+0000 7f809b400640 1 -- 172.21.15.161:0/4084865090 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f80940838b0 con 0x7f80a413cfc0 2024-05-22T21:23:02.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.660+0000 7f809be00640 1 --2- 172.21.15.161:0/4084865090 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8080066690 0x7f8080068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:02.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.660+0000 7f809be00640 1 --2- 172.21.15.161:0/4084865090 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8080066690 0x7f8080068b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f808c0ae000 tx=0x7f808c0b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:02.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.664+0000 7f809b400640 1 -- 172.21.15.161:0/4084865090 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8094050f20 con 0x7f80a413cfc0 2024-05-22T21:23:02.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:02.816+0000 7f80aa400640 1 -- 172.21.15.161:0/4084865090 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f80a4071670 con 0x7f80a413cfc0 2024-05-22T21:23:03.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.488+0000 7f809b400640 1 -- 172.21.15.161:0/4084865090 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v173) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8094054540 con 0x7f80a413cfc0 2024-05-22T21:23:03.489 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' created 2024-05-22T21:23:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.492+0000 7f80aa400640 1 -- 172.21.15.161:0/4084865090 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8080066690 msgr2=0x7f8080068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.492+0000 7f80aa400640 1 --2- 172.21.15.161:0/4084865090 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8080066690 0x7f8080068b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f808c0ae000 tx=0x7f808c0b5040 comp rx=0 tx=0).stop 2024-05-22T21:23:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.492+0000 7f80aa400640 1 -- 172.21.15.161:0/4084865090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a413cfc0 msgr2=0x7f80a4155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.492+0000 7f80aa400640 1 --2- 172.21.15.161:0/4084865090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a413cfc0 0x7f80a4155980 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f809400c3f0 tx=0x7f809400c8b0 comp rx=0 tx=0).stop 2024-05-22T21:23:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.492+0000 7f80aa400640 1 -- 172.21.15.161:0/4084865090 shutdown_connections 2024-05-22T21:23:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.492+0000 7f80aa400640 1 --2- 172.21.15.161:0/4084865090 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8080066690 0x7f8080068b50 secure :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f808c0ae000 tx=0x7f808c0b5040 comp rx=0 tx=0).stop 2024-05-22T21:23:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.492+0000 7f80aa400640 1 --2- 172.21.15.161:0/4084865090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80a41402e0 0x7f80a415da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.492+0000 7f80aa400640 1 --2- 172.21.15.161:0/4084865090 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80a413fc80 0x7f80a413e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.492+0000 7f80aa400640 1 --2- 172.21.15.161:0/4084865090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80a413cfc0 0x7f80a4155980 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:03.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.492+0000 7f80aa400640 1 -- 172.21.15.161:0/4084865090 >> 172.21.15.161:0/4084865090 conn(0x7f80a406a2d0 msgr2=0x7f80a40af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:03.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.496+0000 7f80aa400640 1 -- 172.21.15.161:0/4084865090 shutdown_connections 2024-05-22T21:23:03.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.496+0000 7f80aa400640 1 -- 172.21.15.161:0/4084865090 wait complete. 2024-05-22T21:23:03.520 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-22T21:23:03.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.660+0000 7fca1b200640 1 Processor -- start 2024-05-22T21:23:03.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.660+0000 7fca1b200640 1 -- start start 2024-05-22T21:23:03.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.660+0000 7fca1b200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fca1c0ce370 0x7fca1c1442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:03.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.660+0000 7fca1b200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fca1c1448e0 0x7fca1c0b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:03.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.660+0000 7fca1b200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca1c0ce830 con 0x7fca1c143ed0 2024-05-22T21:23:03.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.660+0000 7fca1b200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca1c0b2e60 con 0x7fca1c0ce370 2024-05-22T21:23:03.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.660+0000 7fca1b200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c0b3140 con 0x7fca1c1448e0 2024-05-22T21:23:03.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.660+0000 7fca11400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fca1c0ce370 0x7fca1c1442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:03.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.660+0000 7fca11400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fca1c0ce370 0x7fca1c1442b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41216/0 (socket says 172.21.15.161:41216) 2024-05-22T21:23:03.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.660+0000 7fca11400640 1 -- 172.21.15.161:0/1100350022 learned_addr learned my addr 172.21.15.161:0/1100350022 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:03.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.660+0000 7fca1a800640 1 --2- 172.21.15.161:0/1100350022 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fca1c1448e0 0x7fca1c0b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:03.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.660+0000 7fca11400640 1 -- 172.21.15.161:0/1100350022 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fca1c1448e0 msgr2=0x7fca1c0b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:03.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.660+0000 7fca11400640 1 --2- 172.21.15.161:0/1100350022 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fca1c1448e0 0x7fca1c0b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:03.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.660+0000 7fca11400640 1 -- 172.21.15.161:0/1100350022 >> v1:172.21.15.179:6789/0 conn(0x7fca1c143ed0 legacy=0x7fca1c0b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:03.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.660+0000 7fca11400640 1 -- 172.21.15.161:0/1100350022 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c0b3420 con 0x7fca1c0ce370 2024-05-22T21:23:03.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.664+0000 7fca11400640 1 --2- 172.21.15.161:0/1100350022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca1c0ce370 0x7fca1c1442b0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fca04009870 tx=0x7fca0401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123cab14d115876e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:03.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.664+0000 7fca19400640 1 -- 172.21.15.161:0/1100350022 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca040ab660 con 0x7fca1c0ce370 2024-05-22T21:23:03.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.664+0000 7fca19400640 1 -- 172.21.15.161:0/1100350022 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca040ab7c0 con 0x7fca1c0ce370 2024-05-22T21:23:03.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.664+0000 7fca19400640 1 -- 172.21.15.161:0/1100350022 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca040abb50 con 0x7fca1c0ce370 2024-05-22T21:23:03.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.664+0000 7fca1b200640 1 -- 172.21.15.161:0/1100350022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca1c0ce370 msgr2=0x7fca1c1442b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:03.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.664+0000 7fca1b200640 1 --2- 172.21.15.161:0/1100350022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca1c0ce370 0x7fca1c1442b0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fca04009870 tx=0x7fca0401c440 comp rx=0 tx=0).stop 2024-05-22T21:23:03.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.664+0000 7fca1b200640 1 -- 172.21.15.161:0/1100350022 shutdown_connections 2024-05-22T21:23:03.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.664+0000 7fca1b200640 1 --2- 172.21.15.161:0/1100350022 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fca1c1448e0 0x7fca1c0b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:03.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.664+0000 7fca1b200640 1 --2- 172.21.15.161:0/1100350022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca1c0ce370 0x7fca1c1442b0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:03.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.664+0000 7fca1b200640 1 -- 172.21.15.161:0/1100350022 >> 172.21.15.161:0/1100350022 conn(0x7fca1c0cec00 msgr2=0x7fca1c0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:03.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.664+0000 7fca1b200640 1 -- 172.21.15.161:0/1100350022 shutdown_connections 2024-05-22T21:23:03.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.664+0000 7fca1b200640 1 -- 172.21.15.161:0/1100350022 wait complete. 2024-05-22T21:23:03.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca1b200640 1 Processor -- start 2024-05-22T21:23:03.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca1b200640 1 -- start start 2024-05-22T21:23:03.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca1b200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fca1c0ce370 0x7fca1c0c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:03.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca1b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca1c143ed0 0x7fca1c159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:03.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca1b200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca1c1448e0 0x7fca1c159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:03.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca1b200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c139610 con 0x7fca1c143ed0 2024-05-22T21:23:03.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca1b200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c139860 con 0x7fca1c1448e0 2024-05-22T21:23:03.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca1b200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c139b40 con 0x7fca1c0ce370 2024-05-22T21:23:03.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca11400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca1c143ed0 0x7fca1c159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:03.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca11400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca1c143ed0 0x7fca1c159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41224/0 (socket says 172.21.15.161:41224) 2024-05-22T21:23:03.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca11400640 1 -- 172.21.15.161:0/2314456329 learned_addr learned my addr 172.21.15.161:0/2314456329 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:03.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca19e00640 1 --2- 172.21.15.161:0/2314456329 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fca1c0ce370 0x7fca1c0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:03.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca1a800640 1 --2- 172.21.15.161:0/2314456329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca1c1448e0 0x7fca1c159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:03.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca11400640 1 -- 172.21.15.161:0/2314456329 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fca1c0ce370 msgr2=0x7fca1c0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:03.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca11400640 1 --2- 172.21.15.161:0/2314456329 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fca1c0ce370 0x7fca1c0c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:03.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca11400640 1 -- 172.21.15.161:0/2314456329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca1c1448e0 msgr2=0x7fca1c159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:03.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca11400640 1 --2- 172.21.15.161:0/2314456329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca1c1448e0 0x7fca1c159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:03.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca11400640 1 -- 172.21.15.161:0/2314456329 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040094d0 con 0x7fca1c143ed0 2024-05-22T21:23:03.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca1a800640 1 --2- 172.21.15.161:0/2314456329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca1c1448e0 0x7fca1c159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:23:03.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca11400640 1 --2- 172.21.15.161:0/2314456329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca1c143ed0 0x7fca1c159a60 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fca040ad040 tx=0x7fca0401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:03.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca12a00640 1 -- 172.21.15.161:0/2314456329 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca040aac00 con 0x7fca1c143ed0 2024-05-22T21:23:03.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca12a00640 1 -- 172.21.15.161:0/2314456329 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca040aad60 con 0x7fca1c143ed0 2024-05-22T21:23:03.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca12a00640 1 -- 172.21.15.161:0/2314456329 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca040be630 con 0x7fca1c143ed0 2024-05-22T21:23:03.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca1b200640 1 -- 172.21.15.161:0/2314456329 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca1c139e20 con 0x7fca1c143ed0 2024-05-22T21:23:03.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca1b200640 1 -- 172.21.15.161:0/2314456329 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1c13a3e0 con 0x7fca1c143ed0 2024-05-22T21:23:03.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.668+0000 7fca1b200640 1 -- 172.21.15.161:0/2314456329 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca1c067070 con 0x7fca1c143ed0 2024-05-22T21:23:03.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.672+0000 7fca12a00640 1 -- 172.21.15.161:0/2314456329 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fca040a9070 con 0x7fca1c143ed0 2024-05-22T21:23:03.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.672+0000 7fca12a00640 1 --2- 172.21.15.161:0/2314456329 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc9f8066690 0x7fc9f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:03.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.672+0000 7fca12a00640 1 -- 172.21.15.161:0/2314456329 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fca0411f280 con 0x7fca1c143ed0 2024-05-22T21:23:03.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.672+0000 7fca19e00640 1 --2- 172.21.15.161:0/2314456329 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc9f8066690 0x7fc9f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:03.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.672+0000 7fca19e00640 1 --2- 172.21.15.161:0/2314456329 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc9f8066690 0x7fc9f8068b50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fca0c000f70 tx=0x7fca0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:03.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.676+0000 7fca12a00640 1 -- 172.21.15.161:0/2314456329 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fca040ec750 con 0x7fca1c143ed0 2024-05-22T21:23:03.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.860+0000 7fca1b200640 1 -- 172.21.15.161:0/2314456329 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fca1c139c80 con 0x7fca1c143ed0 2024-05-22T21:23:03.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.868+0000 7fca12a00640 1 -- 172.21.15.161:0/2314456329 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v74) v1 ==== 174+0+0 (secure 0 0 0) 0x7fca1c139c80 con 0x7fca1c143ed0 2024-05-22T21:23:03.869 INFO:teuthology.orchestra.run.smithi161.stderr:new fs with metadata pool 20 and data pool 21 2024-05-22T21:23:03.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.872+0000 7fca1b200640 1 -- 172.21.15.161:0/2314456329 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc9f8066690 msgr2=0x7fc9f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:03.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.872+0000 7fca1b200640 1 --2- 172.21.15.161:0/2314456329 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc9f8066690 0x7fc9f8068b50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fca0c000f70 tx=0x7fca0c009040 comp rx=0 tx=0).stop 2024-05-22T21:23:03.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.872+0000 7fca1b200640 1 -- 172.21.15.161:0/2314456329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca1c143ed0 msgr2=0x7fca1c159a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:03.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.872+0000 7fca1b200640 1 --2- 172.21.15.161:0/2314456329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca1c143ed0 0x7fca1c159a60 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fca040ad040 tx=0x7fca0401ac80 comp rx=0 tx=0).stop 2024-05-22T21:23:03.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.876+0000 7fca1b200640 1 -- 172.21.15.161:0/2314456329 shutdown_connections 2024-05-22T21:23:03.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.876+0000 7fca1b200640 1 --2- 172.21.15.161:0/2314456329 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc9f8066690 0x7fc9f8068b50 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:03.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.876+0000 7fca1b200640 1 --2- 172.21.15.161:0/2314456329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca1c1448e0 0x7fca1c159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:03.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.876+0000 7fca1b200640 1 --2- 172.21.15.161:0/2314456329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca1c143ed0 0x7fca1c159a60 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:03.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.876+0000 7fca1b200640 1 --2- 172.21.15.161:0/2314456329 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fca1c0ce370 0x7fca1c0c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:03.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.876+0000 7fca1b200640 1 -- 172.21.15.161:0/2314456329 >> 172.21.15.161:0/2314456329 conn(0x7fca1c0cec00 msgr2=0x7fca1c0b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:03.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.880+0000 7fca1b200640 1 -- 172.21.15.161:0/2314456329 shutdown_connections 2024-05-22T21:23:03.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:03.880+0000 7fca1b200640 1 -- 172.21.15.161:0/2314456329 wait complete. 2024-05-22T21:23:03.904 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:23:04.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.040+0000 7f9df4200640 1 Processor -- start 2024-05-22T21:23:04.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.040+0000 7f9df4200640 1 -- start start 2024-05-22T21:23:04.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.040+0000 7f9df4200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9dec15dd50 0x7f9dec06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.040+0000 7f9df4200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9dec15e130 0x7f9dec15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.040+0000 7f9df4200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dec071670 con 0x7f9dec15d410 2024-05-22T21:23:04.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.040+0000 7f9df4200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dec15d8a0 con 0x7f9dec15dd50 2024-05-22T21:23:04.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.040+0000 7f9df4200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec06de40 con 0x7f9dec15e130 2024-05-22T21:23:04.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.040+0000 7f9df1800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9dec15dd50 0x7f9dec06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:04.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.040+0000 7f9df1800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9dec15dd50 0x7f9dec06d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41232/0 (socket says 172.21.15.161:41232) 2024-05-22T21:23:04.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.040+0000 7f9df1800640 1 -- 172.21.15.161:0/267446166 learned_addr learned my addr 172.21.15.161:0/267446166 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:04.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.040+0000 7f9df1800640 1 -- 172.21.15.161:0/267446166 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9dec15e130 msgr2=0x7f9dec15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:04.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.040+0000 7f9df0e00640 1 --2- 172.21.15.161:0/267446166 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9dec15e130 0x7f9dec15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:04.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.040+0000 7f9df1800640 1 --2- 172.21.15.161:0/267446166 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9dec15e130 0x7f9dec15ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.040+0000 7f9df1800640 1 -- 172.21.15.161:0/267446166 >> v1:172.21.15.179:6789/0 conn(0x7f9dec15d410 legacy=0x7f9dec15a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:04.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.040+0000 7f9df1800640 1 -- 172.21.15.161:0/267446166 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dec070a60 con 0x7f9dec15dd50 2024-05-22T21:23:04.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.040+0000 7f9df0e00640 1 --2- 172.21.15.161:0/267446166 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9dec15e130 0x7f9dec15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:23:04.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.044+0000 7f9df1800640 1 --2- 172.21.15.161:0/267446166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dec15dd50 0x7f9dec06d900 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009870 tx=0x7f9de001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389c8ef97a6bbdfd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:04.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.044+0000 7f9debe00640 1 -- 172.21.15.161:0/267446166 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de00ab620 con 0x7f9dec15dd50 2024-05-22T21:23:04.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.044+0000 7f9debe00640 1 -- 172.21.15.161:0/267446166 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de00ab780 con 0x7f9dec15dd50 2024-05-22T21:23:04.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.044+0000 7f9debe00640 1 -- 172.21.15.161:0/267446166 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de00abb10 con 0x7f9dec15dd50 2024-05-22T21:23:04.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.044+0000 7f9df4200640 1 -- 172.21.15.161:0/267446166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dec15dd50 msgr2=0x7f9dec06d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:04.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.044+0000 7f9df4200640 1 --2- 172.21.15.161:0/267446166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dec15dd50 0x7f9dec06d900 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009870 tx=0x7f9de001c460 comp rx=0 tx=0).stop 2024-05-22T21:23:04.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.044+0000 7f9df4200640 1 -- 172.21.15.161:0/267446166 shutdown_connections 2024-05-22T21:23:04.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.044+0000 7f9df4200640 1 --2- 172.21.15.161:0/267446166 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9dec15e130 0x7f9dec15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.044+0000 7f9df4200640 1 --2- 172.21.15.161:0/267446166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dec15dd50 0x7f9dec06d900 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.044+0000 7f9df4200640 1 -- 172.21.15.161:0/267446166 >> 172.21.15.161:0/267446166 conn(0x7f9dec0b0bc0 msgr2=0x7f9dec0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:04.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.044+0000 7f9df4200640 1 -- 172.21.15.161:0/267446166 shutdown_connections 2024-05-22T21:23:04.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.044+0000 7f9df4200640 1 -- 172.21.15.161:0/267446166 wait complete. 2024-05-22T21:23:04.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df4200640 1 Processor -- start 2024-05-22T21:23:04.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df4200640 1 -- start start 2024-05-22T21:23:04.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df4200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9dec15d410 0x7f9dec14de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df4200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9dec15dd50 0x7f9dec0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df4200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dec15e130 0x7f9dec158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df4200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec0ccce0 con 0x7f9dec15e130 2024-05-22T21:23:04.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df4200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec0cc6d0 con 0x7f9dec15d410 2024-05-22T21:23:04.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df4200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec14efb0 con 0x7f9dec15dd50 2024-05-22T21:23:04.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df2200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dec15e130 0x7f9dec158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:04.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df2200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dec15e130 0x7f9dec158b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41242/0 (socket says 172.21.15.161:41242) 2024-05-22T21:23:04.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df2200640 1 -- 172.21.15.161:0/3344824105 learned_addr learned my addr 172.21.15.161:0/3344824105 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:04.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df0e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9dec15dd50 0x7f9dec0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:04.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df1800640 1 --2- 172.21.15.161:0/3344824105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9dec15d410 0x7f9dec14de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:04.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df2200640 1 -- 172.21.15.161:0/3344824105 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9dec15dd50 msgr2=0x7f9dec0b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:04.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df2200640 1 --2- 172.21.15.161:0/3344824105 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9dec15dd50 0x7f9dec0b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df2200640 1 -- 172.21.15.161:0/3344824105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9dec15d410 msgr2=0x7f9dec14de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:04.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df2200640 1 --2- 172.21.15.161:0/3344824105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9dec15d410 0x7f9dec14de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df2200640 1 -- 172.21.15.161:0/3344824105 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de00094d0 con 0x7f9dec15e130 2024-05-22T21:23:04.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df2200640 1 --2- 172.21.15.161:0/3344824105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dec15e130 0x7f9dec158b20 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f9de400c780 tx=0x7f9de400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:04.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9de9600640 1 -- 172.21.15.161:0/3344824105 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de4004b70 con 0x7f9dec15e130 2024-05-22T21:23:04.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9de9600640 1 -- 172.21.15.161:0/3344824105 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de400ce90 con 0x7f9dec15e130 2024-05-22T21:23:04.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9de9600640 1 -- 172.21.15.161:0/3344824105 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de400a850 con 0x7f9dec15e130 2024-05-22T21:23:04.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df4200640 1 -- 172.21.15.161:0/3344824105 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dec0cc970 con 0x7f9dec15e130 2024-05-22T21:23:04.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df4200640 1 -- 172.21.15.161:0/3344824105 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dec14e340 con 0x7f9dec15e130 2024-05-22T21:23:04.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.048+0000 7f9df4200640 1 -- 172.21.15.161:0/3344824105 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dec0cce50 con 0x7f9dec15e130 2024-05-22T21:23:04.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.052+0000 7f9de9600640 1 -- 172.21.15.161:0/3344824105 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9de400a9b0 con 0x7f9dec15e130 2024-05-22T21:23:04.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.052+0000 7f9de9600640 1 --2- 172.21.15.161:0/3344824105 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9dcc066690 0x7f9dcc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.052+0000 7f9de9600640 1 -- 172.21.15.161:0/3344824105 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9de40841d0 con 0x7f9dec15e130 2024-05-22T21:23:04.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.052+0000 7f9df1800640 1 --2- 172.21.15.161:0/3344824105 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9dcc066690 0x7f9dcc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:04.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.052+0000 7f9df1800640 1 --2- 172.21.15.161:0/3344824105 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9dcc066690 0x7f9dcc068b50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9de00af000 tx=0x7f9de00b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:04.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.056+0000 7f9de9600640 1 -- 172.21.15.161:0/3344824105 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9de40515e0 con 0x7f9dec15e130 2024-05-22T21:23:04.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.208+0000 7f9df4200640 1 -- 172.21.15.161:0/3344824105 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9dec067220 con 0x7f9dec15e130 2024-05-22T21:23:04.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.212+0000 7f9de9600640 1 -- 172.21.15.161:0/3344824105 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+16173 (secure 0 0 0) 0x7f9de4054c00 con 0x7f9dec15e130 2024-05-22T21:23:04.211 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:23:04.212 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":174,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:23:03.863215+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:23:02.242525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-05-22T21:23:02.864916+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:23:04.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.212+0000 7f9df4200640 1 -- 172.21.15.161:0/3344824105 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9dcc066690 msgr2=0x7f9dcc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:04.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.212+0000 7f9df4200640 1 --2- 172.21.15.161:0/3344824105 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9dcc066690 0x7f9dcc068b50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9de00af000 tx=0x7f9de00b6040 comp rx=0 tx=0).stop 2024-05-22T21:23:04.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.212+0000 7f9df4200640 1 -- 172.21.15.161:0/3344824105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dec15e130 msgr2=0x7f9dec158b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:04.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.212+0000 7f9df4200640 1 --2- 172.21.15.161:0/3344824105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dec15e130 0x7f9dec158b20 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f9de400c780 tx=0x7f9de400cc40 comp rx=0 tx=0).stop 2024-05-22T21:23:04.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.212+0000 7f9df4200640 1 -- 172.21.15.161:0/3344824105 shutdown_connections 2024-05-22T21:23:04.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.212+0000 7f9df4200640 1 --2- 172.21.15.161:0/3344824105 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9dcc066690 0x7f9dcc068b50 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.212+0000 7f9df4200640 1 --2- 172.21.15.161:0/3344824105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9dec15e130 0x7f9dec158b20 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.212+0000 7f9df4200640 1 --2- 172.21.15.161:0/3344824105 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9dec15dd50 0x7f9dec0b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.212+0000 7f9df4200640 1 --2- 172.21.15.161:0/3344824105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9dec15d410 0x7f9dec14de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.212+0000 7f9df4200640 1 -- 172.21.15.161:0/3344824105 >> 172.21.15.161:0/3344824105 conn(0x7f9dec0b0bc0 msgr2=0x7f9dec0cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:04.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.212+0000 7f9df4200640 1 -- 172.21.15.161:0/3344824105 shutdown_connections 2024-05-22T21:23:04.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.212+0000 7f9df4200640 1 -- 172.21.15.161:0/3344824105 wait complete. 2024-05-22T21:23:04.233 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:23:04.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f392da00640 1 Processor -- start 2024-05-22T21:23:04.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f392da00640 1 -- start start 2024-05-22T21:23:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f392da00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f392815d430 0x7f39280707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f392da00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f39280ac8b0 0x7f392806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f392da00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3928071fd0 con 0x7f39280accf0 2024-05-22T21:23:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f392da00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3928070e80 con 0x7f392815d430 2024-05-22T21:23:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f392da00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392806c510 con 0x7f39280ac8b0 2024-05-22T21:23:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f3926a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f392815d430 0x7f39280707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f3927400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f39280accf0 0x7f39280b1ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:45190/0 (socket says 172.21.15.161:45190) 2024-05-22T21:23:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f3927400640 1 -- 172.21.15.161:0/4043922380 learned_addr learned my addr 172.21.15.161:0/4043922380 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:23:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f391e000640 1 --2- 172.21.15.161:0/4043922380 >> v2:172.21.15.179:3300/0 conn(0x7f39280ac8b0 0x7f392806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:04.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f3926a00640 1 -- 172.21.15.161:0/4043922380 >> v2:172.21.15.179:3300/0 conn(0x7f39280ac8b0 msgr2=0x7f392806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:04.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f3926a00640 1 --2- 172.21.15.161:0/4043922380 >> v2:172.21.15.179:3300/0 conn(0x7f39280ac8b0 0x7f392806cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f3926a00640 1 -- 172.21.15.161:0/4043922380 >> v1:172.21.15.161:6789/0 conn(0x7f39280accf0 legacy=0x7f39280b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:04.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f3926a00640 1 -- 172.21.15.161:0/4043922380 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392806c7f0 con 0x7f392815d430 2024-05-22T21:23:04.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f391e000640 1 --2- 172.21.15.161:0/4043922380 >> v2:172.21.15.179:3300/0 conn(0x7f39280ac8b0 0x7f392806cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:23:04.370 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.368+0000 7f3926000640 1 -- 172.21.15.161:0/4043922380 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989969178 0 0) 0x7f3928071fd0 con 0x7f39280accf0 2024-05-22T21:23:04.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.372+0000 7f3926a00640 1 --2- 172.21.15.161:0/4043922380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f392815d430 0x7f39280707c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f3928072330 tx=0x7f391801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9438d1ac6bba44f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:04.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.372+0000 7f3926000640 1 -- 172.21.15.161:0/4043922380 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39180ab600 con 0x7f392815d430 2024-05-22T21:23:04.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.372+0000 7f3926000640 1 -- 172.21.15.161:0/4043922380 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39180a7050 con 0x7f392815d430 2024-05-22T21:23:04.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.372+0000 7f3926000640 1 -- 172.21.15.161:0/4043922380 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39180ab980 con 0x7f392815d430 2024-05-22T21:23:04.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.372+0000 7f392da00640 1 -- 172.21.15.161:0/4043922380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f392815d430 msgr2=0x7f39280707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:04.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.372+0000 7f392da00640 1 --2- 172.21.15.161:0/4043922380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f392815d430 0x7f39280707c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f3928072330 tx=0x7f391801c3c0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.372+0000 7f392da00640 1 -- 172.21.15.161:0/4043922380 shutdown_connections 2024-05-22T21:23:04.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.372+0000 7f392da00640 1 --2- 172.21.15.161:0/4043922380 >> v2:172.21.15.179:3300/0 conn(0x7f39280ac8b0 0x7f392806cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.372+0000 7f392da00640 1 --2- 172.21.15.161:0/4043922380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f392815d430 0x7f39280707c0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.372+0000 7f392da00640 1 -- 172.21.15.161:0/4043922380 >> 172.21.15.161:0/4043922380 conn(0x7f39280b0bc0 msgr2=0x7f39280b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:04.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.372+0000 7f392da00640 1 -- 172.21.15.161:0/4043922380 shutdown_connections 2024-05-22T21:23:04.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.372+0000 7f392da00640 1 -- 172.21.15.161:0/4043922380 wait complete. 2024-05-22T21:23:04.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f392da00640 1 Processor -- start 2024-05-22T21:23:04.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f392da00640 1 -- start start 2024-05-22T21:23:04.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f392da00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39280ac8b0 0x7f3928148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f392da00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39280accf0 0x7f3928139740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f392da00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f392815d430 0x7f392815f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f392da00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39281489e0 con 0x7f39280ac8b0 2024-05-22T21:23:04.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f392da00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3928147dd0 con 0x7f39280accf0 2024-05-22T21:23:04.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f392da00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39281480b0 con 0x7f392815d430 2024-05-22T21:23:04.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f3926a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39280ac8b0 0x7f3928148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:04.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f3926a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39280ac8b0 0x7f3928148da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41264/0 (socket says 172.21.15.161:41264) 2024-05-22T21:23:04.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f3926a00640 1 -- 172.21.15.161:0/2088371443 learned_addr learned my addr 172.21.15.161:0/2088371443 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:04.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f3926a00640 1 -- 172.21.15.161:0/2088371443 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f392815d430 msgr2=0x7f392815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:04.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f3926a00640 1 --2- 172.21.15.161:0/2088371443 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f392815d430 0x7f392815f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f3926a00640 1 -- 172.21.15.161:0/2088371443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39280accf0 msgr2=0x7f3928139740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:04.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f3926a00640 1 --2- 172.21.15.161:0/2088371443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39280accf0 0x7f3928139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f3926a00640 1 -- 172.21.15.161:0/2088371443 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3918009420 con 0x7f39280ac8b0 2024-05-22T21:23:04.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f3926a00640 1 --2- 172.21.15.161:0/2088371443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39280ac8b0 0x7f3928148da0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f391801aa70 tx=0x7f391801ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:04.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f391f400640 1 -- 172.21.15.161:0/2088371443 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39180b7650 con 0x7f39280ac8b0 2024-05-22T21:23:04.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f391f400640 1 -- 172.21.15.161:0/2088371443 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39180b77b0 con 0x7f39280ac8b0 2024-05-22T21:23:04.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f392da00640 1 -- 172.21.15.161:0/2088371443 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3928148390 con 0x7f39280ac8b0 2024-05-22T21:23:04.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f391f400640 1 -- 172.21.15.161:0/2088371443 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39180b7b60 con 0x7f39280ac8b0 2024-05-22T21:23:04.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f392da00640 1 -- 172.21.15.161:0/2088371443 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3928139c80 con 0x7f39280ac8b0 2024-05-22T21:23:04.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.376+0000 7f392da00640 1 -- 172.21.15.161:0/2088371443 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3928067220 con 0x7f39280ac8b0 2024-05-22T21:23:04.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.380+0000 7f391f400640 1 -- 172.21.15.161:0/2088371443 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f39180b7cc0 con 0x7f39280ac8b0 2024-05-22T21:23:04.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.380+0000 7f391f400640 1 --2- 172.21.15.161:0/2088371443 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3904066690 0x7f3904068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.380+0000 7f391f400640 1 -- 172.21.15.161:0/2088371443 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f391811ea50 con 0x7f39280ac8b0 2024-05-22T21:23:04.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.380+0000 7f391e000640 1 --2- 172.21.15.161:0/2088371443 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3904066690 0x7f3904068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:04.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.380+0000 7f391e000640 1 --2- 172.21.15.161:0/2088371443 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3904066690 0x7f3904068b50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3910006f10 tx=0x7f3910008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:04.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.384+0000 7f391f400640 1 -- 172.21.15.161:0/2088371443 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f39180ebd20 con 0x7f39280ac8b0 2024-05-22T21:23:04.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.540+0000 7f392da00640 1 -- 172.21.15.161:0/2088371443 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39280ae7a0 con 0x7f39280ac8b0 2024-05-22T21:23:04.541 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.540+0000 7f391f400640 1 -- 172.21.15.161:0/2088371443 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+16173 (secure 0 0 0) 0x7f39180bb090 con 0x7f39280ac8b0 2024-05-22T21:23:04.542 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:23:04.542 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":174,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:23:03.863215+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:23:02.242525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-05-22T21:23:02.864916+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:23:04.544 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.544+0000 7f392da00640 1 -- 172.21.15.161:0/2088371443 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3904066690 msgr2=0x7f3904068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:04.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.544+0000 7f392da00640 1 --2- 172.21.15.161:0/2088371443 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3904066690 0x7f3904068b50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3910006f10 tx=0x7f3910008040 comp rx=0 tx=0).stop 2024-05-22T21:23:04.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.544+0000 7f392da00640 1 -- 172.21.15.161:0/2088371443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39280ac8b0 msgr2=0x7f3928148da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:04.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.544+0000 7f392da00640 1 --2- 172.21.15.161:0/2088371443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39280ac8b0 0x7f3928148da0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f391801aa70 tx=0x7f391801ac00 comp rx=0 tx=0).stop 2024-05-22T21:23:04.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.544+0000 7f392da00640 1 -- 172.21.15.161:0/2088371443 shutdown_connections 2024-05-22T21:23:04.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.544+0000 7f392da00640 1 --2- 172.21.15.161:0/2088371443 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3904066690 0x7f3904068b50 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.544+0000 7f392da00640 1 --2- 172.21.15.161:0/2088371443 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f392815d430 0x7f392815f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.544+0000 7f392da00640 1 --2- 172.21.15.161:0/2088371443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39280accf0 0x7f3928139740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.544+0000 7f392da00640 1 --2- 172.21.15.161:0/2088371443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39280ac8b0 0x7f3928148da0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.544+0000 7f392da00640 1 -- 172.21.15.161:0/2088371443 >> 172.21.15.161:0/2088371443 conn(0x7f39280b0bc0 msgr2=0x7f39280aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:04.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.544+0000 7f392da00640 1 -- 172.21.15.161:0/2088371443 shutdown_connections 2024-05-22T21:23:04.547 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.544+0000 7f392da00640 1 -- 172.21.15.161:0/2088371443 wait complete. 2024-05-22T21:23:04.563 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-22T21:23:04.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.700+0000 7f8226600640 1 Processor -- start 2024-05-22T21:23:04.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.700+0000 7f8226600640 1 -- start start 2024-05-22T21:23:04.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.704+0000 7f8226600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f82201596b0 0x7f8220071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.703 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.704+0000 7f8226600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8220071db0 0x7f822006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.704+0000 7f8226600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f822006d070 0x7f822006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.704+0000 7f8226600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822006d4e0 con 0x7f822006d070 2024-05-22T21:23:04.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.704+0000 7f8226600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822015d250 con 0x7f82201596b0 2024-05-22T21:23:04.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.704+0000 7f8226600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8220159a90 con 0x7f8220071db0 2024-05-22T21:23:04.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.704+0000 7f821fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f822006d070 0x7f822006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:04.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.704+0000 7f821fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f822006d070 0x7f822006fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41274/0 (socket says 172.21.15.161:41274) 2024-05-22T21:23:04.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.704+0000 7f821fe00640 1 -- 172.21.15.161:0/1228899676 learned_addr learned my addr 172.21.15.161:0/1228899676 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:04.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.704+0000 7f821fe00640 1 -- 172.21.15.161:0/1228899676 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8220071db0 msgr2=0x7f822006cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:04.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.704+0000 7f821fe00640 1 --2- 172.21.15.161:0/1228899676 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8220071db0 0x7f822006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.704+0000 7f821fe00640 1 -- 172.21.15.161:0/1228899676 >> v2:172.21.15.179:3300/0 conn(0x7f82201596b0 msgr2=0x7f8220071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:04.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.704+0000 7f821fe00640 1 --2- 172.21.15.161:0/1228899676 >> v2:172.21.15.179:3300/0 conn(0x7f82201596b0 0x7f8220071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.704+0000 7f821fe00640 1 -- 172.21.15.161:0/1228899676 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822015d510 con 0x7f822006d070 2024-05-22T21:23:04.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.708+0000 7f821fe00640 1 --2- 172.21.15.161:0/1228899676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f822006d070 0x7f822006fce0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f82200724b0 tx=0x7f821801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18be1046e945648e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:04.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.708+0000 7f821e000640 1 -- 172.21.15.161:0/1228899676 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180ab600 con 0x7f822006d070 2024-05-22T21:23:04.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.708+0000 7f821e000640 1 -- 172.21.15.161:0/1228899676 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82180a7050 con 0x7f822006d070 2024-05-22T21:23:04.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.708+0000 7f821e000640 1 -- 172.21.15.161:0/1228899676 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180ab980 con 0x7f822006d070 2024-05-22T21:23:04.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.708+0000 7f8226600640 1 -- 172.21.15.161:0/1228899676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f822006d070 msgr2=0x7f822006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:04.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.708+0000 7f8226600640 1 --2- 172.21.15.161:0/1228899676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f822006d070 0x7f822006fce0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f82200724b0 tx=0x7f821801c520 comp rx=0 tx=0).stop 2024-05-22T21:23:04.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.708+0000 7f8226600640 1 -- 172.21.15.161:0/1228899676 shutdown_connections 2024-05-22T21:23:04.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.708+0000 7f8226600640 1 --2- 172.21.15.161:0/1228899676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f822006d070 0x7f822006fce0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.708+0000 7f8226600640 1 --2- 172.21.15.161:0/1228899676 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8220071db0 0x7f822006cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.708+0000 7f8226600640 1 --2- 172.21.15.161:0/1228899676 >> v2:172.21.15.179:3300/0 conn(0x7f82201596b0 0x7f8220071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.708+0000 7f8226600640 1 -- 172.21.15.161:0/1228899676 >> 172.21.15.161:0/1228899676 conn(0x7f822006a2d0 msgr2=0x7f822006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:04.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.708+0000 7f8226600640 1 -- 172.21.15.161:0/1228899676 shutdown_connections 2024-05-22T21:23:04.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.708+0000 7f8226600640 1 -- 172.21.15.161:0/1228899676 wait complete. 2024-05-22T21:23:04.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.708+0000 7f8226600640 1 Processor -- start 2024-05-22T21:23:04.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.708+0000 7f8226600640 1 -- start start 2024-05-22T21:23:04.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f8226600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f822006d070 0x7f8220153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f8226600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8220071db0 0x7f8220155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f8226600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82201596b0 0x7f822014a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f8226600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82201557a0 con 0x7f82201596b0 2024-05-22T21:23:04.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f8226600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822014bd30 con 0x7f8220071db0 2024-05-22T21:23:04.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f8226600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822014a9b0 con 0x7f822006d070 2024-05-22T21:23:04.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f821fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82201596b0 0x7f822014a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:04.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f821fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82201596b0 0x7f822014a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41282/0 (socket says 172.21.15.161:41282) 2024-05-22T21:23:04.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f821fe00640 1 -- 172.21.15.161:0/3108957764 learned_addr learned my addr 172.21.15.161:0/3108957764 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:04.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f821f400640 1 --2- 172.21.15.161:0/3108957764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f822006d070 0x7f8220153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:04.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f821ea00640 1 --2- 172.21.15.161:0/3108957764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8220071db0 0x7f8220155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:04.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f821fe00640 1 -- 172.21.15.161:0/3108957764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f822006d070 msgr2=0x7f8220153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:04.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f821fe00640 1 --2- 172.21.15.161:0/3108957764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f822006d070 0x7f8220153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f821fe00640 1 -- 172.21.15.161:0/3108957764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8220071db0 msgr2=0x7f8220155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:04.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f821fe00640 1 --2- 172.21.15.161:0/3108957764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8220071db0 0x7f8220155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:04.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f821fe00640 1 -- 172.21.15.161:0/3108957764 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82180094d0 con 0x7f82201596b0 2024-05-22T21:23:04.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f821fe00640 1 --2- 172.21.15.161:0/3108957764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82201596b0 0x7f822014a470 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f821801abd0 tx=0x7f821801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:04.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f8217400640 1 -- 172.21.15.161:0/3108957764 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180b6620 con 0x7f82201596b0 2024-05-22T21:23:04.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f8217400640 1 -- 172.21.15.161:0/3108957764 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82180b6780 con 0x7f82201596b0 2024-05-22T21:23:04.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f8226600640 1 -- 172.21.15.161:0/3108957764 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8220153d10 con 0x7f82201596b0 2024-05-22T21:23:04.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f8217400640 1 -- 172.21.15.161:0/3108957764 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180b6b30 con 0x7f82201596b0 2024-05-22T21:23:04.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f8226600640 1 -- 172.21.15.161:0/3108957764 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82201542d0 con 0x7f82201596b0 2024-05-22T21:23:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f8226600640 1 -- 172.21.15.161:0/3108957764 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8220067220 con 0x7f82201596b0 2024-05-22T21:23:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.712+0000 7f8217400640 1 -- 172.21.15.161:0/3108957764 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f82180b2040 con 0x7f82201596b0 2024-05-22T21:23:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.716+0000 7f8217400640 1 --2- 172.21.15.161:0/3108957764 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8200066690 0x7f8200068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.716+0000 7f8217400640 1 -- 172.21.15.161:0/3108957764 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f821811e350 con 0x7f82201596b0 2024-05-22T21:23:04.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.716+0000 7f821f400640 1 --2- 172.21.15.161:0/3108957764 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8200066690 0x7f8200068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:04.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.716+0000 7f821f400640 1 --2- 172.21.15.161:0/3108957764 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8200066690 0x7f8200068b50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f821000a400 tx=0x7f821000d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:04.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.720+0000 7f8217400640 1 -- 172.21.15.161:0/3108957764 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f82180eb870 con 0x7f82201596b0 2024-05-22T21:23:04.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:04.908+0000 7f8226600640 1 -- 172.21.15.161:0/3108957764 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f822014be70 con 0x7f82201596b0 2024-05-22T21:23:05.319 INFO:tasks.ceph.mds.a.smithi161.stderr:2024-05-22T21:23:05.320+0000 7f9cefe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T21:23:05.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:05.876+0000 7f8217400640 1 -- 172.21.15.161:0/3108957764 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v77) v1 ==== 118+0+0 (secure 0 0 0) 0x7f82180eee90 con 0x7f82201596b0 2024-05-22T21:23:05.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:05.880+0000 7f8226600640 1 -- 172.21.15.161:0/3108957764 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8200066690 msgr2=0x7f8200068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:05.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:05.880+0000 7f8226600640 1 --2- 172.21.15.161:0/3108957764 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8200066690 0x7f8200068b50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f821000a400 tx=0x7f821000d040 comp rx=0 tx=0).stop 2024-05-22T21:23:05.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:05.880+0000 7f8226600640 1 -- 172.21.15.161:0/3108957764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82201596b0 msgr2=0x7f822014a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:05.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:05.880+0000 7f8226600640 1 --2- 172.21.15.161:0/3108957764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82201596b0 0x7f822014a470 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f821801abd0 tx=0x7f821801ad60 comp rx=0 tx=0).stop 2024-05-22T21:23:05.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:05.880+0000 7f8226600640 1 -- 172.21.15.161:0/3108957764 shutdown_connections 2024-05-22T21:23:05.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:05.880+0000 7f8226600640 1 --2- 172.21.15.161:0/3108957764 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8200066690 0x7f8200068b50 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:05.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:05.880+0000 7f8226600640 1 --2- 172.21.15.161:0/3108957764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82201596b0 0x7f822014a470 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:05.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:05.880+0000 7f8226600640 1 --2- 172.21.15.161:0/3108957764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8220071db0 0x7f8220155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:05.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:05.880+0000 7f8226600640 1 --2- 172.21.15.161:0/3108957764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f822006d070 0x7f8220153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:05.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:05.880+0000 7f8226600640 1 -- 172.21.15.161:0/3108957764 >> 172.21.15.161:0/3108957764 conn(0x7f822006a2d0 msgr2=0x7f82200ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:05.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:05.880+0000 7f8226600640 1 -- 172.21.15.161:0/3108957764 shutdown_connections 2024-05-22T21:23:05.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:05.880+0000 7f8226600640 1 -- 172.21.15.161:0/3108957764 wait complete. 2024-05-22T21:23:05.900 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-22T21:23:05.900 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-22T21:23:06.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.040+0000 7f3273a00640 1 Processor -- start 2024-05-22T21:23:06.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.040+0000 7f3273a00640 1 -- start start 2024-05-22T21:23:06.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.040+0000 7f3273a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f326c1596b0 0x7f326c071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:06.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.040+0000 7f3273a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f326c06d240 0x7f326c0b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:06.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.040+0000 7f3273a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326c15d250 con 0x7f326c071db0 2024-05-22T21:23:06.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.040+0000 7f3273a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326c159a90 con 0x7f326c1596b0 2024-05-22T21:23:06.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.040+0000 7f3273a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c15d510 con 0x7f326c06d240 2024-05-22T21:23:06.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.040+0000 7f3271000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f326c1596b0 0x7f326c071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:06.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.040+0000 7f3271a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f326c06d240 0x7f326c0b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:06.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.040+0000 7f3271000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f326c1596b0 0x7f326c071870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:56284/0 (socket says 172.21.15.161:56284) 2024-05-22T21:23:06.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.040+0000 7f3271000640 1 -- 172.21.15.161:0/418731877 learned_addr learned my addr 172.21.15.161:0/418731877 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:06.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.044+0000 7f3271000640 1 -- 172.21.15.161:0/418731877 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f326c06d240 msgr2=0x7f326c0b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:06.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.044+0000 7f3271000640 1 --2- 172.21.15.161:0/418731877 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f326c06d240 0x7f326c0b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:06.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.044+0000 7f3271000640 1 -- 172.21.15.161:0/418731877 >> v1:172.21.15.179:6789/0 conn(0x7f326c071db0 legacy=0x7f326c06cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:06.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.044+0000 7f3271000640 1 -- 172.21.15.161:0/418731877 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c15d7f0 con 0x7f326c1596b0 2024-05-22T21:23:06.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.044+0000 7f3271a00640 1 --2- 172.21.15.161:0/418731877 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f326c06d240 0x7f326c0b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:23:06.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.044+0000 7f3271000640 1 --2- 172.21.15.161:0/418731877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f326c1596b0 0x7f326c071870 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f325c006fd0 tx=0x7f325c01e9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d86cf1704f2082e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:06.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.044+0000 7f326b400640 1 -- 172.21.15.161:0/418731877 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c004230 con 0x7f326c1596b0 2024-05-22T21:23:06.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.044+0000 7f326b400640 1 -- 172.21.15.161:0/418731877 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325c0ab070 con 0x7f326c1596b0 2024-05-22T21:23:06.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.044+0000 7f326b400640 1 -- 172.21.15.161:0/418731877 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c0a7750 con 0x7f326c1596b0 2024-05-22T21:23:06.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3273a00640 1 -- 172.21.15.161:0/418731877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f326c1596b0 msgr2=0x7f326c071870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:06.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3273a00640 1 --2- 172.21.15.161:0/418731877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f326c1596b0 0x7f326c071870 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f325c006fd0 tx=0x7f325c01e9f0 comp rx=0 tx=0).stop 2024-05-22T21:23:06.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3273a00640 1 -- 172.21.15.161:0/418731877 shutdown_connections 2024-05-22T21:23:06.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3273a00640 1 --2- 172.21.15.161:0/418731877 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f326c06d240 0x7f326c0b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:06.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3273a00640 1 --2- 172.21.15.161:0/418731877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f326c1596b0 0x7f326c071870 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:06.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3273a00640 1 -- 172.21.15.161:0/418731877 >> 172.21.15.161:0/418731877 conn(0x7f326c06a2d0 msgr2=0x7f326c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:06.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3273a00640 1 -- 172.21.15.161:0/418731877 shutdown_connections 2024-05-22T21:23:06.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3273a00640 1 -- 172.21.15.161:0/418731877 wait complete. 2024-05-22T21:23:06.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3273a00640 1 Processor -- start 2024-05-22T21:23:06.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3273a00640 1 -- start start 2024-05-22T21:23:06.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3273a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f326c06d240 0x7f326c14a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:06.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3273a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f326c071db0 0x7f326c15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:06.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3273a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f326c1596b0 0x7f326c0afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:06.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3273a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c149850 con 0x7f326c06d240 2024-05-22T21:23:06.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3273a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c149b30 con 0x7f326c1596b0 2024-05-22T21:23:06.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3273a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c149e10 con 0x7f326c071db0 2024-05-22T21:23:06.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3271000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f326c06d240 0x7f326c14a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:06.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3271000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f326c06d240 0x7f326c14a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42298/0 (socket says 172.21.15.161:42298) 2024-05-22T21:23:06.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3271000640 1 -- 172.21.15.161:0/1139878105 learned_addr learned my addr 172.21.15.161:0/1139878105 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:06.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3271a00640 1 --2- 172.21.15.161:0/1139878105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f326c1596b0 0x7f326c0afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:06.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f326be00640 1 --2- 172.21.15.161:0/1139878105 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f326c071db0 0x7f326c15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:06.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3271000640 1 -- 172.21.15.161:0/1139878105 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f326c071db0 msgr2=0x7f326c15b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:06.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3271000640 1 --2- 172.21.15.161:0/1139878105 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f326c071db0 0x7f326c15b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:06.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3271000640 1 -- 172.21.15.161:0/1139878105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f326c1596b0 msgr2=0x7f326c0afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:06.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3271000640 1 --2- 172.21.15.161:0/1139878105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f326c1596b0 0x7f326c0afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:06.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3271000640 1 -- 172.21.15.161:0/1139878105 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3264009a70 con 0x7f326c06d240 2024-05-22T21:23:06.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3271a00640 1 --2- 172.21.15.161:0/1139878105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f326c1596b0 0x7f326c0afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:23:06.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f326be00640 1 --2- 172.21.15.161:0/1139878105 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f326c071db0 0x7f326c15b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:23:06.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.048+0000 7f3271000640 1 --2- 172.21.15.161:0/1139878105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f326c06d240 0x7f326c14a8f0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f325c000c10 tx=0x7f325c01b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:06.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.052+0000 7f3268c00640 1 -- 172.21.15.161:0/1139878105 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c0bf070 con 0x7f326c06d240 2024-05-22T21:23:06.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.052+0000 7f3268c00640 1 -- 172.21.15.161:0/1139878105 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325c0a79e0 con 0x7f326c06d240 2024-05-22T21:23:06.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.052+0000 7f3268c00640 1 -- 172.21.15.161:0/1139878105 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c0a7bb0 con 0x7f326c06d240 2024-05-22T21:23:06.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.052+0000 7f3273a00640 1 -- 172.21.15.161:0/1139878105 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325c00b780 con 0x7f326c06d240 2024-05-22T21:23:06.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.052+0000 7f3273a00640 1 -- 172.21.15.161:0/1139878105 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c14a3d0 con 0x7f326c06d240 2024-05-22T21:23:06.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.052+0000 7f3273a00640 1 -- 172.21.15.161:0/1139878105 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326c067220 con 0x7f326c06d240 2024-05-22T21:23:06.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.052+0000 7f3268c00640 1 -- 172.21.15.161:0/1139878105 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f325c0b9050 con 0x7f326c06d240 2024-05-22T21:23:06.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.052+0000 7f3268c00640 1 --2- 172.21.15.161:0/1139878105 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f324c066690 0x7f324c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:06.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.052+0000 7f3268c00640 1 -- 172.21.15.161:0/1139878105 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f325c120350 con 0x7f326c06d240 2024-05-22T21:23:06.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.052+0000 7f326be00640 1 --2- 172.21.15.161:0/1139878105 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f324c066690 0x7f324c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:06.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.056+0000 7f326be00640 1 --2- 172.21.15.161:0/1139878105 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f324c066690 0x7f324c068b50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f326c0b3670 tx=0x7f3258009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:06.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.056+0000 7f3268c00640 1 -- 172.21.15.161:0/1139878105 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f325c0ed6c0 con 0x7f326c06d240 2024-05-22T21:23:06.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.244+0000 7f3273a00640 1 -- 172.21.15.161:0/1139878105 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f326c071420 con 0x7f326c06d240 2024-05-22T21:23:06.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.880+0000 7f3268c00640 1 -- 172.21.15.161:0/1139878105 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v78) v1 ==== 122+0+0 (secure 0 0 0) 0x7f325c0f0ce0 con 0x7f326c06d240 2024-05-22T21:23:06.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.880+0000 7f3273a00640 1 -- 172.21.15.161:0/1139878105 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f324c066690 msgr2=0x7f324c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:06.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.880+0000 7f3273a00640 1 --2- 172.21.15.161:0/1139878105 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f324c066690 0x7f324c068b50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f326c0b3670 tx=0x7f3258009040 comp rx=0 tx=0).stop 2024-05-22T21:23:06.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.880+0000 7f3273a00640 1 -- 172.21.15.161:0/1139878105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f326c06d240 msgr2=0x7f326c14a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:06.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.880+0000 7f3273a00640 1 --2- 172.21.15.161:0/1139878105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f326c06d240 0x7f326c14a8f0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f325c000c10 tx=0x7f325c01b790 comp rx=0 tx=0).stop 2024-05-22T21:23:06.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.880+0000 7f3273a00640 1 -- 172.21.15.161:0/1139878105 shutdown_connections 2024-05-22T21:23:06.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.880+0000 7f3273a00640 1 --2- 172.21.15.161:0/1139878105 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f324c066690 0x7f324c068b50 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:06.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.880+0000 7f3273a00640 1 --2- 172.21.15.161:0/1139878105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f326c1596b0 0x7f326c0afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:06.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.880+0000 7f3273a00640 1 --2- 172.21.15.161:0/1139878105 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f326c071db0 0x7f326c15b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:06.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.880+0000 7f3273a00640 1 --2- 172.21.15.161:0/1139878105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f326c06d240 0x7f326c14a8f0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:06.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.880+0000 7f3273a00640 1 -- 172.21.15.161:0/1139878105 >> 172.21.15.161:0/1139878105 conn(0x7f326c06a2d0 msgr2=0x7f326c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:06.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.880+0000 7f3273a00640 1 -- 172.21.15.161:0/1139878105 shutdown_connections 2024-05-22T21:23:06.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:06.880+0000 7f3273a00640 1 -- 172.21.15.161:0/1139878105 wait complete. 2024-05-22T21:23:06.901 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-22T21:23:06.901 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:23:07.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.040+0000 7fd3f8000640 1 Processor -- start 2024-05-22T21:23:07.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.040+0000 7fd3f8000640 1 -- start start 2024-05-22T21:23:07.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.040+0000 7fd3f8000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd3f015d270 0x7fd3f00ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.040+0000 7fd3f8000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd3f006e140 0x7fd3f006ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.040+0000 7fd3f8000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f006a9a0 con 0x7fd3f006b370 2024-05-22T21:23:07.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.040+0000 7fd3f8000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f006cc40 con 0x7fd3f006e140 2024-05-22T21:23:07.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.040+0000 7fd3f8000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f006a070 con 0x7fd3f015d270 2024-05-22T21:23:07.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.040+0000 7fd3f4c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd3f006e140 0x7fd3f006ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.040+0000 7fd3f4c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd3f006e140 0x7fd3f006ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42314/0 (socket says 172.21.15.161:42314) 2024-05-22T21:23:07.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.040+0000 7fd3f4c00640 1 -- 172.21.15.161:0/1056197872 learned_addr learned my addr 172.21.15.161:0/1056197872 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:07.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.040+0000 7fd3f5600640 1 --2- 172.21.15.161:0/1056197872 >> v2:172.21.15.179:3300/0 conn(0x7fd3f015d270 0x7fd3f00ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.040+0000 7fd3f4c00640 1 -- 172.21.15.161:0/1056197872 >> v2:172.21.15.179:3300/0 conn(0x7fd3f015d270 msgr2=0x7fd3f00ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:07.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.040+0000 7fd3f4c00640 1 --2- 172.21.15.161:0/1056197872 >> v2:172.21.15.179:3300/0 conn(0x7fd3f015d270 0x7fd3f00ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.040+0000 7fd3f4c00640 1 -- 172.21.15.161:0/1056197872 >> v1:172.21.15.179:6789/0 conn(0x7fd3f006b370 legacy=0x7fd3f00b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.040+0000 7fd3f4c00640 1 -- 172.21.15.161:0/1056197872 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f006a350 con 0x7fd3f006e140 2024-05-22T21:23:07.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.044+0000 7fd3f4c00640 1 --2- 172.21.15.161:0/1056197872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3f006e140 0x7fd3f006ae30 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd3f006baa0 tx=0x7fd3dc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac11565c1c36c848 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:07.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.044+0000 7fd3efe00640 1 -- 172.21.15.161:0/1056197872 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0ab600 con 0x7fd3f006e140 2024-05-22T21:23:07.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.044+0000 7fd3efe00640 1 -- 172.21.15.161:0/1056197872 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc0a7050 con 0x7fd3f006e140 2024-05-22T21:23:07.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.044+0000 7fd3efe00640 1 -- 172.21.15.161:0/1056197872 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0ab980 con 0x7fd3f006e140 2024-05-22T21:23:07.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.044+0000 7fd3f8000640 1 -- 172.21.15.161:0/1056197872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3f006e140 msgr2=0x7fd3f006ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.044+0000 7fd3f8000640 1 --2- 172.21.15.161:0/1056197872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3f006e140 0x7fd3f006ae30 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd3f006baa0 tx=0x7fd3dc01c520 comp rx=0 tx=0).stop 2024-05-22T21:23:07.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.044+0000 7fd3f8000640 1 -- 172.21.15.161:0/1056197872 shutdown_connections 2024-05-22T21:23:07.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.044+0000 7fd3f8000640 1 --2- 172.21.15.161:0/1056197872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3f006e140 0x7fd3f006ae30 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.044+0000 7fd3f8000640 1 --2- 172.21.15.161:0/1056197872 >> v2:172.21.15.179:3300/0 conn(0x7fd3f015d270 0x7fd3f00ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.044+0000 7fd3f8000640 1 -- 172.21.15.161:0/1056197872 >> 172.21.15.161:0/1056197872 conn(0x7fd3f00cf610 msgr2=0x7fd3f00cfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:07.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.044+0000 7fd3f8000640 1 -- 172.21.15.161:0/1056197872 shutdown_connections 2024-05-22T21:23:07.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.044+0000 7fd3f8000640 1 -- 172.21.15.161:0/1056197872 wait complete. 2024-05-22T21:23:07.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f8000640 1 Processor -- start 2024-05-22T21:23:07.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f8000640 1 -- start start 2024-05-22T21:23:07.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f8000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3f006b370 0x7fd3f0152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f8000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3f006e140 0x7fd3f013cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f8000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3f015d270 0x7fd3f015b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f8000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f0152930 con 0x7fd3f006b370 2024-05-22T21:23:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f8000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f013d5b0 con 0x7fd3f006e140 2024-05-22T21:23:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f8000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f013d890 con 0x7fd3f015d270 2024-05-22T21:23:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f5600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3f006b370 0x7fd3f0152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f5600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3f006b370 0x7fd3f0152030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42326/0 (socket says 172.21.15.161:42326) 2024-05-22T21:23:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f5600640 1 -- 172.21.15.161:0/1280419634 learned_addr learned my addr 172.21.15.161:0/1280419634 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f6000640 1 --2- 172.21.15.161:0/1280419634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3f015d270 0x7fd3f015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f4c00640 1 --2- 172.21.15.161:0/1280419634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3f006e140 0x7fd3f013cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f5600640 1 -- 172.21.15.161:0/1280419634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3f015d270 msgr2=0x7fd3f015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f5600640 1 --2- 172.21.15.161:0/1280419634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3f015d270 0x7fd3f015b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f5600640 1 -- 172.21.15.161:0/1280419634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3f006e140 msgr2=0x7fd3f013cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f5600640 1 --2- 172.21.15.161:0/1280419634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3f006e140 0x7fd3f013cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f5600640 1 -- 172.21.15.161:0/1280419634 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3dc0094d0 con 0x7fd3f006b370 2024-05-22T21:23:07.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f5600640 1 --2- 172.21.15.161:0/1280419634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3f006b370 0x7fd3f0152030 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400d800 tx=0x7fd3e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:07.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3ed600640 1 -- 172.21.15.161:0/1280419634 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e400b6c0 con 0x7fd3f006b370 2024-05-22T21:23:07.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3ed600640 1 -- 172.21.15.161:0/1280419634 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e400b820 con 0x7fd3f006b370 2024-05-22T21:23:07.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3ed600640 1 -- 172.21.15.161:0/1280419634 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e400bb90 con 0x7fd3f006b370 2024-05-22T21:23:07.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f8000640 1 -- 172.21.15.161:0/1280419634 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f013db70 con 0x7fd3f006b370 2024-05-22T21:23:07.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f8000640 1 -- 172.21.15.161:0/1280419634 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f0159250 con 0x7fd3f006b370 2024-05-22T21:23:07.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.048+0000 7fd3f8000640 1 -- 172.21.15.161:0/1280419634 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f00acdd0 con 0x7fd3f006b370 2024-05-22T21:23:07.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.052+0000 7fd3ed600640 1 -- 172.21.15.161:0/1280419634 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd3e400bcf0 con 0x7fd3f006b370 2024-05-22T21:23:07.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.052+0000 7fd3ed600640 1 --2- 172.21.15.161:0/1280419634 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd3d0066690 0x7fd3d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.052+0000 7fd3ed600640 1 -- 172.21.15.161:0/1280419634 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3e4084670 con 0x7fd3f006b370 2024-05-22T21:23:07.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.052+0000 7fd3f4c00640 1 --2- 172.21.15.161:0/1280419634 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd3d0066690 0x7fd3d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.052+0000 7fd3f4c00640 1 --2- 172.21.15.161:0/1280419634 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd3d0066690 0x7fd3d0068b50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0ad000 tx=0x7fd3dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:07.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.056+0000 7fd3ed600640 1 -- 172.21.15.161:0/1280419634 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd3f00acdd0 con 0x7fd3f006b370 2024-05-22T21:23:07.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.244+0000 7fd3f8000640 1 -- 172.21.15.161:0/1280419634 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3f00acdd0 con 0x7fd3f006b370 2024-05-22T21:23:07.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.248+0000 7fd3ed600640 1 -- 172.21.15.161:0/1280419634 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4747 (secure 0 0 0) 0x7fd3f00acdd0 con 0x7fd3f006b370 2024-05-22T21:23:07.247 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:23:07.247 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2405126804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2405126804},{"type":"v1","addr":"172.21.15.179:6834","nonce":2405126804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":6962,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3984924056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3984924056},{"type":"v1","addr":"172.21.15.179:6835","nonce":3984924056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8391,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/368424851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":368424851},{"type":"v1","addr":"172.21.15.161:6836","nonce":368424851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:23:03.863540+0000","modified":"2024-05-22T21:23:06.877759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2279022192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2279022192},{"type":"v1","addr":"172.21.15.161:6837","nonce":2279022192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-05-22T21:23:07.247 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 78 2024-05-22T21:23:07.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.248+0000 7fd3f8000640 1 -- 172.21.15.161:0/1280419634 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd3d0066690 msgr2=0x7fd3d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.248+0000 7fd3f8000640 1 --2- 172.21.15.161:0/1280419634 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd3d0066690 0x7fd3d0068b50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0ad000 tx=0x7fd3dc008040 comp rx=0 tx=0).stop 2024-05-22T21:23:07.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.248+0000 7fd3f8000640 1 -- 172.21.15.161:0/1280419634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3f006b370 msgr2=0x7fd3f0152030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.248+0000 7fd3f8000640 1 --2- 172.21.15.161:0/1280419634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3f006b370 0x7fd3f0152030 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400d800 tx=0x7fd3e400dcc0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.248+0000 7fd3f8000640 1 -- 172.21.15.161:0/1280419634 shutdown_connections 2024-05-22T21:23:07.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.248+0000 7fd3f8000640 1 --2- 172.21.15.161:0/1280419634 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd3d0066690 0x7fd3d0068b50 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.248+0000 7fd3f8000640 1 --2- 172.21.15.161:0/1280419634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3f015d270 0x7fd3f015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.248+0000 7fd3f8000640 1 --2- 172.21.15.161:0/1280419634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3f006e140 0x7fd3f013cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.248+0000 7fd3f8000640 1 --2- 172.21.15.161:0/1280419634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3f006b370 0x7fd3f0152030 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.248+0000 7fd3f8000640 1 -- 172.21.15.161:0/1280419634 >> 172.21.15.161:0/1280419634 conn(0x7fd3f00cf610 msgr2=0x7fd3f015fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:07.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.248+0000 7fd3f8000640 1 -- 172.21.15.161:0/1280419634 shutdown_connections 2024-05-22T21:23:07.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.248+0000 7fd3f8000640 1 -- 172.21.15.161:0/1280419634 wait complete. 2024-05-22T21:23:07.267 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:23:07.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4b4c00640 1 Processor -- start 2024-05-22T21:23:07.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4b4c00640 1 -- start start 2024-05-22T21:23:07.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4b4c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb4b0143ec0 0x7fb4b015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4b4c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb4b00715a0 0x7fb4b006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4b4c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb4b015ec90 0x7fb4b015f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4b4c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b015f660 con 0x7fb4b00715a0 2024-05-22T21:23:07.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4b4c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b01442a0 con 0x7fb4b015ec90 2024-05-22T21:23:07.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4b4c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b0070260 con 0x7fb4b0143ec0 2024-05-22T21:23:07.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4ad400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb4b00715a0 0x7fb4b006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4ad400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb4b00715a0 0x7fb4b006fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42340/0 (socket says 172.21.15.161:42340) 2024-05-22T21:23:07.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4ad400640 1 -- 172.21.15.161:0/2477134534 learned_addr learned my addr 172.21.15.161:0/2477134534 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:07.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4ade00640 1 --2- 172.21.15.161:0/2477134534 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb4b0143ec0 0x7fb4b015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4ae800640 1 --2- 172.21.15.161:0/2477134534 >> v2:172.21.15.179:3300/0 conn(0x7fb4b015ec90 0x7fb4b015f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4ad400640 1 -- 172.21.15.161:0/2477134534 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb4b0143ec0 msgr2=0x7fb4b015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:07.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4ad400640 1 --2- 172.21.15.161:0/2477134534 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb4b0143ec0 0x7fb4b015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4ad400640 1 -- 172.21.15.161:0/2477134534 >> v2:172.21.15.179:3300/0 conn(0x7fb4b015ec90 msgr2=0x7fb4b015f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:07.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4ad400640 1 --2- 172.21.15.161:0/2477134534 >> v2:172.21.15.179:3300/0 conn(0x7fb4b015ec90 0x7fb4b015f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.404+0000 7fb4ad400640 1 -- 172.21.15.161:0/2477134534 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b0070500 con 0x7fb4b00715a0 2024-05-22T21:23:07.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.408+0000 7fb4ad400640 1 --2- 172.21.15.161:0/2477134534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4b00715a0 0x7fb4b006fd20 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fb4b00b1be0 tx=0x7fb49c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a757a71839e3892 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:07.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.408+0000 7fb4aca00640 1 -- 172.21.15.161:0/2477134534 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c0aa610 con 0x7fb4b00715a0 2024-05-22T21:23:07.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.408+0000 7fb4aca00640 1 -- 172.21.15.161:0/2477134534 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49c0a6040 con 0x7fb4b00715a0 2024-05-22T21:23:07.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.408+0000 7fb4aca00640 1 -- 172.21.15.161:0/2477134534 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c0aa990 con 0x7fb4b00715a0 2024-05-22T21:23:07.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.408+0000 7fb4b4c00640 1 -- 172.21.15.161:0/2477134534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4b00715a0 msgr2=0x7fb4b006fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.408+0000 7fb4b4c00640 1 --2- 172.21.15.161:0/2477134534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4b00715a0 0x7fb4b006fd20 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fb4b00b1be0 tx=0x7fb49c01c4f0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.408+0000 7fb4b4c00640 1 -- 172.21.15.161:0/2477134534 shutdown_connections 2024-05-22T21:23:07.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.408+0000 7fb4b4c00640 1 --2- 172.21.15.161:0/2477134534 >> v2:172.21.15.179:3300/0 conn(0x7fb4b015ec90 0x7fb4b015f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.408+0000 7fb4b4c00640 1 --2- 172.21.15.161:0/2477134534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4b00715a0 0x7fb4b006fd20 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.408+0000 7fb4b4c00640 1 --2- 172.21.15.161:0/2477134534 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb4b0143ec0 0x7fb4b015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.408+0000 7fb4b4c00640 1 -- 172.21.15.161:0/2477134534 >> 172.21.15.161:0/2477134534 conn(0x7fb4b00b0bc0 msgr2=0x7fb4b00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:07.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.408+0000 7fb4b4c00640 1 -- 172.21.15.161:0/2477134534 shutdown_connections 2024-05-22T21:23:07.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.408+0000 7fb4b4c00640 1 -- 172.21.15.161:0/2477134534 wait complete. 2024-05-22T21:23:07.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.408+0000 7fb4b4c00640 1 Processor -- start 2024-05-22T21:23:07.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.408+0000 7fb4b4c00640 1 -- start start 2024-05-22T21:23:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4b4c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb4b0143ec0 0x7fb4b015a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4b4c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b015ec90 0x7fb4b0141450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4b4c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4b015d6f0 0x7fb4b01f7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4b4c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b013ef50 con 0x7fb4b015d6f0 2024-05-22T21:23:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4b4c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b0141ca0 con 0x7fb4b015ec90 2024-05-22T21:23:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4b4c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b013e900 con 0x7fb4b0143ec0 2024-05-22T21:23:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4ae800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4b015d6f0 0x7fb4b01f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4ae800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4b015d6f0 0x7fb4b01f7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42350/0 (socket says 172.21.15.161:42350) 2024-05-22T21:23:07.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4ae800640 1 -- 172.21.15.161:0/798552430 learned_addr learned my addr 172.21.15.161:0/798552430 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:07.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4ad400640 1 --2- 172.21.15.161:0/798552430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b015ec90 0x7fb4b0141450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4ae800640 1 -- 172.21.15.161:0/798552430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb4b0143ec0 msgr2=0x7fb4b015a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4ade00640 1 --2- 172.21.15.161:0/798552430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb4b0143ec0 0x7fb4b015a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4ae800640 1 --2- 172.21.15.161:0/798552430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb4b0143ec0 0x7fb4b015a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4ae800640 1 -- 172.21.15.161:0/798552430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b015ec90 msgr2=0x7fb4b0141450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4ae800640 1 --2- 172.21.15.161:0/798552430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b015ec90 0x7fb4b0141450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4ae800640 1 -- 172.21.15.161:0/798552430 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c0094d0 con 0x7fb4b015d6f0 2024-05-22T21:23:07.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4ad400640 1 --2- 172.21.15.161:0/798552430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b015ec90 0x7fb4b0141450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:23:07.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4ade00640 1 --2- 172.21.15.161:0/798552430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb4b0143ec0 0x7fb4b015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:23:07.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4ae800640 1 --2- 172.21.15.161:0/798552430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4b015d6f0 0x7fb4b01f7390 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8007e60 tx=0x7fb4a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:07.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4a6000640 1 -- 172.21.15.161:0/798552430 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8014420 con 0x7fb4b015d6f0 2024-05-22T21:23:07.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4a6000640 1 -- 172.21.15.161:0/798552430 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a80145a0 con 0x7fb4b015d6f0 2024-05-22T21:23:07.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4b4c00640 1 -- 172.21.15.161:0/798552430 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b013ebe0 con 0x7fb4b015d6f0 2024-05-22T21:23:07.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4a6000640 1 -- 172.21.15.161:0/798552430 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8014950 con 0x7fb4b015d6f0 2024-05-22T21:23:07.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4b4c00640 1 -- 172.21.15.161:0/798552430 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b0141990 con 0x7fb4b015d6f0 2024-05-22T21:23:07.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4b4c00640 1 -- 172.21.15.161:0/798552430 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b0067220 con 0x7fb4b015d6f0 2024-05-22T21:23:07.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.412+0000 7fb4a6000640 1 -- 172.21.15.161:0/798552430 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb4a8014ab0 con 0x7fb4b015d6f0 2024-05-22T21:23:07.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.416+0000 7fb4a6000640 1 --2- 172.21.15.161:0/798552430 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb490066690 0x7fb490068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.416+0000 7fb4a6000640 1 -- 172.21.15.161:0/798552430 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4a8084a60 con 0x7fb4b015d6f0 2024-05-22T21:23:07.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.416+0000 7fb4ade00640 1 --2- 172.21.15.161:0/798552430 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb490066690 0x7fb490068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.416+0000 7fb4ade00640 1 --2- 172.21.15.161:0/798552430 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb490066690 0x7fb490068b50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb4b006d590 tx=0x7fb4a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:07.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.420+0000 7fb4a6000640 1 -- 172.21.15.161:0/798552430 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb4a8051dc0 con 0x7fb4b015d6f0 2024-05-22T21:23:07.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.576+0000 7fb4b4c00640 1 -- 172.21.15.161:0/798552430 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4b0165b50 con 0x7fb4b015d6f0 2024-05-22T21:23:07.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.576+0000 7fb4a6000640 1 -- 172.21.15.161:0/798552430 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+16160 (secure 0 0 0) 0x7fb4a80553e0 con 0x7fb4b015d6f0 2024-05-22T21:23:07.577 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:23:07.577 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":175,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:23:04.866317+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:23:02.242525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-05-22T21:23:02.864916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:23:07.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.580+0000 7fb4b4c00640 1 -- 172.21.15.161:0/798552430 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb490066690 msgr2=0x7fb490068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.580+0000 7fb4b4c00640 1 --2- 172.21.15.161:0/798552430 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb490066690 0x7fb490068b50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb4b006d590 tx=0x7fb4a0008040 comp rx=0 tx=0).stop 2024-05-22T21:23:07.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.580+0000 7fb4b4c00640 1 -- 172.21.15.161:0/798552430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4b015d6f0 msgr2=0x7fb4b01f7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.580+0000 7fb4b4c00640 1 --2- 172.21.15.161:0/798552430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4b015d6f0 0x7fb4b01f7390 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8007e60 tx=0x7fb4a800e4b0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.580+0000 7fb4b4c00640 1 -- 172.21.15.161:0/798552430 shutdown_connections 2024-05-22T21:23:07.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.580+0000 7fb4b4c00640 1 --2- 172.21.15.161:0/798552430 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb490066690 0x7fb490068b50 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.580+0000 7fb4b4c00640 1 --2- 172.21.15.161:0/798552430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4b015d6f0 0x7fb4b01f7390 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.580+0000 7fb4b4c00640 1 --2- 172.21.15.161:0/798552430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4b015ec90 0x7fb4b0141450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.580+0000 7fb4b4c00640 1 --2- 172.21.15.161:0/798552430 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb4b0143ec0 0x7fb4b015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.580+0000 7fb4b4c00640 1 -- 172.21.15.161:0/798552430 >> 172.21.15.161:0/798552430 conn(0x7fb4b00b0bc0 msgr2=0x7fb4b015c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:07.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.580+0000 7fb4b4c00640 1 -- 172.21.15.161:0/798552430 shutdown_connections 2024-05-22T21:23:07.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.580+0000 7fb4b4c00640 1 -- 172.21.15.161:0/798552430 wait complete. 2024-05-22T21:23:07.598 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-22T21:23:07.598 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:23:07.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.732+0000 7f580e200640 1 Processor -- start 2024-05-22T21:23:07.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.732+0000 7f580e200640 1 -- start start 2024-05-22T21:23:07.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f580e200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f580815dd50 0x7f580806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f580e200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f580815e130 0x7f580815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f580e200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5808071670 con 0x7f580815d410 2024-05-22T21:23:07.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f580e200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580815d8a0 con 0x7f580815e130 2024-05-22T21:23:07.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f580e200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580806de40 con 0x7f580815dd50 2024-05-22T21:23:07.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f5806a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f580815e130 0x7f580815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f5806a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f580815e130 0x7f580815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42354/0 (socket says 172.21.15.161:42354) 2024-05-22T21:23:07.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f5806a00640 1 -- 172.21.15.161:0/626139572 learned_addr learned my addr 172.21.15.161:0/626139572 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:07.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f5807400640 1 --2- 172.21.15.161:0/626139572 >> v2:172.21.15.179:3300/0 conn(0x7f580815dd50 0x7f580806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f5806a00640 1 -- 172.21.15.161:0/626139572 >> v2:172.21.15.179:3300/0 conn(0x7f580815dd50 msgr2=0x7f580806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:07.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f5806a00640 1 --2- 172.21.15.161:0/626139572 >> v2:172.21.15.179:3300/0 conn(0x7f580815dd50 0x7f580806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f5806a00640 1 -- 172.21.15.161:0/626139572 >> v1:172.21.15.161:6789/0 conn(0x7f580815d410 legacy=0x7f580815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f5806a00640 1 -- 172.21.15.161:0/626139572 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5808070a60 con 0x7f580815e130 2024-05-22T21:23:07.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f5806000640 1 -- 172.21.15.161:0/626139572 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507872345 0 0) 0x7f5808071670 con 0x7f580815d410 2024-05-22T21:23:07.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f5806a00640 1 --2- 172.21.15.161:0/626139572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f580815e130 0x7f580815ced0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f5808072330 tx=0x7f57f401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f980ab34f6e4aa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:07.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f5806000640 1 -- 172.21.15.161:0/626139572 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f40aa650 con 0x7f580815e130 2024-05-22T21:23:07.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f5806000640 1 -- 172.21.15.161:0/626139572 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f40aa7b0 con 0x7f580815e130 2024-05-22T21:23:07.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.736+0000 7f5806000640 1 -- 172.21.15.161:0/626139572 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f40aab40 con 0x7f580815e130 2024-05-22T21:23:07.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f580e200640 1 -- 172.21.15.161:0/626139572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f580815e130 msgr2=0x7f580815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f580e200640 1 --2- 172.21.15.161:0/626139572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f580815e130 0x7f580815ced0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f5808072330 tx=0x7f57f401c4f0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f580e200640 1 -- 172.21.15.161:0/626139572 shutdown_connections 2024-05-22T21:23:07.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f580e200640 1 --2- 172.21.15.161:0/626139572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f580815e130 0x7f580815ced0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f580e200640 1 --2- 172.21.15.161:0/626139572 >> v2:172.21.15.179:3300/0 conn(0x7f580815dd50 0x7f580806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f580e200640 1 -- 172.21.15.161:0/626139572 >> 172.21.15.161:0/626139572 conn(0x7f58080b0bc0 msgr2=0x7f58080b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:07.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f580e200640 1 -- 172.21.15.161:0/626139572 shutdown_connections 2024-05-22T21:23:07.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f580e200640 1 -- 172.21.15.161:0/626139572 wait complete. 2024-05-22T21:23:07.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f580e200640 1 Processor -- start 2024-05-22T21:23:07.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f580e200640 1 -- start start 2024-05-22T21:23:07.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f580e200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f580815d410 0x7f5808150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f580e200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f580815dd50 0x7f58080b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f580e200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f580815e130 0x7f58081570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f580e200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5808151420 con 0x7f580815d410 2024-05-22T21:23:07.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f580e200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5808157b00 con 0x7f580815e130 2024-05-22T21:23:07.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f580e200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5808157de0 con 0x7f580815dd50 2024-05-22T21:23:07.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f5807400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f580815d410 0x7f5808150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f5807400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f580815d410 0x7f5808150ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42364/0 (socket says 172.21.15.161:42364) 2024-05-22T21:23:07.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f5807400640 1 -- 172.21.15.161:0/514692363 learned_addr learned my addr 172.21.15.161:0/514692363 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:07.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f5807400640 1 -- 172.21.15.161:0/514692363 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f580815dd50 msgr2=0x7f58080b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f5807400640 1 --2- 172.21.15.161:0/514692363 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f580815dd50 0x7f58080b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f5807e00640 1 --2- 172.21.15.161:0/514692363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f580815e130 0x7f58081570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f5807400640 1 -- 172.21.15.161:0/514692363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f580815e130 msgr2=0x7f58081570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f5807400640 1 --2- 172.21.15.161:0/514692363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f580815e130 0x7f58081570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f5807400640 1 -- 172.21.15.161:0/514692363 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f40094d0 con 0x7f580815d410 2024-05-22T21:23:07.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.740+0000 7f5807e00640 1 --2- 172.21.15.161:0/514692363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f580815e130 0x7f58081570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:23:07.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.744+0000 7f5807400640 1 --2- 172.21.15.161:0/514692363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f580815d410 0x7f5808150ee0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f57f8007ef0 tx=0x7f57f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:07.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.744+0000 7f57ff400640 1 -- 172.21.15.161:0/514692363 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8014620 con 0x7f580815d410 2024-05-22T21:23:07.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.744+0000 7f57ff400640 1 -- 172.21.15.161:0/514692363 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f8014780 con 0x7f580815d410 2024-05-22T21:23:07.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.744+0000 7f580e200640 1 -- 172.21.15.161:0/514692363 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58081580c0 con 0x7f580815d410 2024-05-22T21:23:07.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.744+0000 7f57ff400640 1 -- 172.21.15.161:0/514692363 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8014b30 con 0x7f580815d410 2024-05-22T21:23:07.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.744+0000 7f580e200640 1 -- 172.21.15.161:0/514692363 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5808154e20 con 0x7f580815d410 2024-05-22T21:23:07.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.744+0000 7f580e200640 1 -- 172.21.15.161:0/514692363 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58080706b0 con 0x7f580815d410 2024-05-22T21:23:07.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.744+0000 7f57ff400640 1 -- 172.21.15.161:0/514692363 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f57f8014c90 con 0x7f580815d410 2024-05-22T21:23:07.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.744+0000 7f57ff400640 1 --2- 172.21.15.161:0/514692363 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f57e8066690 0x7f57e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:07.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.744+0000 7f57ff400640 1 -- 172.21.15.161:0/514692363 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57f8084bb0 con 0x7f580815d410 2024-05-22T21:23:07.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.744+0000 7f5806a00640 1 --2- 172.21.15.161:0/514692363 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f57e8066690 0x7f57e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:07.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.748+0000 7f5806a00640 1 --2- 172.21.15.161:0/514692363 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f57e8066690 0x7f57e8068b50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f57f40ae000 tx=0x7f57f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:07.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.748+0000 7f57ff400640 1 -- 172.21.15.161:0/514692363 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f57f8051f10 con 0x7f580815d410 2024-05-22T21:23:07.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.900+0000 7f580e200640 1 -- 172.21.15.161:0/514692363 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f580815e890 con 0x7f57e8066690 2024-05-22T21:23:07.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.908+0000 7f57ff400640 1 -- 172.21.15.161:0/514692363 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284948 (secure 0 0 0) 0x7f57f40aa570 con 0x7f57e8066690 2024-05-22T21:23:07.911 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:23:07.912 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:23:07.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.912+0000 7f580e200640 1 -- 172.21.15.161:0/514692363 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f57e8066690 msgr2=0x7f57e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.912+0000 7f580e200640 1 --2- 172.21.15.161:0/514692363 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f57e8066690 0x7f57e8068b50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f57f40ae000 tx=0x7f57f4008040 comp rx=0 tx=0).stop 2024-05-22T21:23:07.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.912+0000 7f580e200640 1 -- 172.21.15.161:0/514692363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f580815d410 msgr2=0x7f5808150ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:07.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.912+0000 7f580e200640 1 --2- 172.21.15.161:0/514692363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f580815d410 0x7f5808150ee0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f57f8007ef0 tx=0x7f57f800e4b0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.912+0000 7f580e200640 1 -- 172.21.15.161:0/514692363 shutdown_connections 2024-05-22T21:23:07.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.912+0000 7f580e200640 1 --2- 172.21.15.161:0/514692363 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f57e8066690 0x7f57e8068b50 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.912+0000 7f580e200640 1 --2- 172.21.15.161:0/514692363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f580815e130 0x7f58081570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.912+0000 7f580e200640 1 --2- 172.21.15.161:0/514692363 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f580815dd50 0x7f58080b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.912+0000 7f580e200640 1 --2- 172.21.15.161:0/514692363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f580815d410 0x7f5808150ee0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:07.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.912+0000 7f580e200640 1 -- 172.21.15.161:0/514692363 >> 172.21.15.161:0/514692363 conn(0x7f58080b0bc0 msgr2=0x7f58080ad500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:07.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.912+0000 7f580e200640 1 -- 172.21.15.161:0/514692363 shutdown_connections 2024-05-22T21:23:07.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:07.912+0000 7f580e200640 1 -- 172.21.15.161:0/514692363 wait complete. 2024-05-22T21:23:07.930 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-05-22T21:23:06.528109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8825,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553674,"statfs":{"total":1600000000,"available":1590962791,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.046253"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918607+0000","last_change":"2024-05-22T21:23:03.879894+0000","last_active":"2024-05-22T21:23:04.918607+0000","last_peered":"2024-05-22T21:23:04.918607+0000","last_clean":"2024-05-22T21:23:04.918607+0000","last_became_active":"2024-05-22T21:23:03.879301+0000","last_became_peered":"2024-05-22T21:23:03.879301+0000","last_unstale":"2024-05-22T21:23:04.918607+0000","last_undegraded":"2024-05-22T21:23:04.918607+0000","last_fullsized":"2024-05-22T21:23:04.918607+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:09:55.160766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918799+0000","last_change":"2024-05-22T21:23:03.877341+0000","last_active":"2024-05-22T21:23:04.918799+0000","last_peered":"2024-05-22T21:23:04.918799+0000","last_clean":"2024-05-22T21:23:04.918799+0000","last_became_active":"2024-05-22T21:23:03.877124+0000","last_became_peered":"2024-05-22T21:23:03.877124+0000","last_unstale":"2024-05-22T21:23:04.918799+0000","last_undegraded":"2024-05-22T21:23:04.918799+0000","last_fullsized":"2024-05-22T21:23:04.918799+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:40:40.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920611+0000","last_change":"2024-05-22T21:23:03.884129+0000","last_active":"2024-05-22T21:23:04.920611+0000","last_peered":"2024-05-22T21:23:04.920611+0000","last_clean":"2024-05-22T21:23:04.920611+0000","last_became_active":"2024-05-22T21:23:03.883911+0000","last_became_peered":"2024-05-22T21:23:03.883911+0000","last_unstale":"2024-05-22T21:23:04.920611+0000","last_undegraded":"2024-05-22T21:23:04.920611+0000","last_fullsized":"2024-05-22T21:23:04.920611+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:24:12.146293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883537+0000","last_change":"2024-05-22T21:23:03.878683+0000","last_active":"2024-05-22T21:23:04.883537+0000","last_peered":"2024-05-22T21:23:04.883537+0000","last_clean":"2024-05-22T21:23:04.883537+0000","last_became_active":"2024-05-22T21:23:03.878406+0000","last_became_peered":"2024-05-22T21:23:03.878406+0000","last_unstale":"2024-05-22T21:23:04.883537+0000","last_undegraded":"2024-05-22T21:23:04.883537+0000","last_fullsized":"2024-05-22T21:23:04.883537+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:20.274667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920360+0000","last_change":"2024-05-22T21:23:03.886007+0000","last_active":"2024-05-22T21:23:04.920360+0000","last_peered":"2024-05-22T21:23:04.920360+0000","last_clean":"2024-05-22T21:23:04.920360+0000","last_became_active":"2024-05-22T21:23:03.884536+0000","last_became_peered":"2024-05-22T21:23:03.884536+0000","last_unstale":"2024-05-22T21:23:04.920360+0000","last_undegraded":"2024-05-22T21:23:04.920360+0000","last_fullsized":"2024-05-22T21:23:04.920360+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:34:14.648009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918760+0000","last_change":"2024-05-22T21:23:03.884920+0000","last_active":"2024-05-22T21:23:04.918760+0000","last_peered":"2024-05-22T21:23:04.918760+0000","last_clean":"2024-05-22T21:23:04.918760+0000","last_became_active":"2024-05-22T21:23:03.884527+0000","last_became_peered":"2024-05-22T21:23:03.884527+0000","last_unstale":"2024-05-22T21:23:04.918760+0000","last_undegraded":"2024-05-22T21:23:04.918760+0000","last_fullsized":"2024-05-22T21:23:04.918760+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:04:35.477076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.877090+0000","last_change":"2024-05-22T21:23:03.877885+0000","last_active":"2024-05-22T21:23:04.877090+0000","last_peered":"2024-05-22T21:23:04.877090+0000","last_clean":"2024-05-22T21:23:04.877090+0000","last_became_active":"2024-05-22T21:23:03.877499+0000","last_became_peered":"2024-05-22T21:23:03.877499+0000","last_unstale":"2024-05-22T21:23:04.877090+0000","last_undegraded":"2024-05-22T21:23:04.877090+0000","last_fullsized":"2024-05-22T21:23:04.877090+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:15:26.229883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918587+0000","last_change":"2024-05-22T21:23:03.883719+0000","last_active":"2024-05-22T21:23:04.918587+0000","last_peered":"2024-05-22T21:23:04.918587+0000","last_clean":"2024-05-22T21:23:04.918587+0000","last_became_active":"2024-05-22T21:23:03.883511+0000","last_became_peered":"2024-05-22T21:23:03.883511+0000","last_unstale":"2024-05-22T21:23:04.918587+0000","last_undegraded":"2024-05-22T21:23:04.918587+0000","last_fullsized":"2024-05-22T21:23:04.918587+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:09:45.135262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919513+0000","last_change":"2024-05-22T21:23:03.872664+0000","last_active":"2024-05-22T21:23:04.919513+0000","last_peered":"2024-05-22T21:23:04.919513+0000","last_clean":"2024-05-22T21:23:04.919513+0000","last_became_active":"2024-05-22T21:23:03.872441+0000","last_became_peered":"2024-05-22T21:23:03.872441+0000","last_unstale":"2024-05-22T21:23:04.919513+0000","last_undegraded":"2024-05-22T21:23:04.919513+0000","last_fullsized":"2024-05-22T21:23:04.919513+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:31:11.627686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884380+0000","last_change":"2024-05-22T21:23:03.887317+0000","last_active":"2024-05-22T21:23:04.884380+0000","last_peered":"2024-05-22T21:23:04.884380+0000","last_clean":"2024-05-22T21:23:04.884380+0000","last_became_active":"2024-05-22T21:23:03.886830+0000","last_became_peered":"2024-05-22T21:23:03.886830+0000","last_unstale":"2024-05-22T21:23:04.884380+0000","last_undegraded":"2024-05-22T21:23:04.884380+0000","last_fullsized":"2024-05-22T21:23:04.884380+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:52:34.048705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918842+0000","last_change":"2024-05-22T21:23:03.883070+0000","last_active":"2024-05-22T21:23:04.918842+0000","last_peered":"2024-05-22T21:23:04.918842+0000","last_clean":"2024-05-22T21:23:04.918842+0000","last_became_active":"2024-05-22T21:23:03.882874+0000","last_became_peered":"2024-05-22T21:23:03.882874+0000","last_unstale":"2024-05-22T21:23:04.918842+0000","last_undegraded":"2024-05-22T21:23:04.918842+0000","last_fullsized":"2024-05-22T21:23:04.918842+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:30:35.247289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883204+0000","last_change":"2024-05-22T21:23:03.883008+0000","last_active":"2024-05-22T21:23:04.883204+0000","last_peered":"2024-05-22T21:23:04.883204+0000","last_clean":"2024-05-22T21:23:04.883204+0000","last_became_active":"2024-05-22T21:23:03.882782+0000","last_became_peered":"2024-05-22T21:23:03.882782+0000","last_unstale":"2024-05-22T21:23:04.883204+0000","last_undegraded":"2024-05-22T21:23:04.883204+0000","last_fullsized":"2024-05-22T21:23:04.883204+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:22:02.297408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920058+0000","last_change":"2024-05-22T21:23:03.887705+0000","last_active":"2024-05-22T21:23:04.920058+0000","last_peered":"2024-05-22T21:23:04.920058+0000","last_clean":"2024-05-22T21:23:04.920058+0000","last_became_active":"2024-05-22T21:23:03.887204+0000","last_became_peered":"2024-05-22T21:23:03.887204+0000","last_unstale":"2024-05-22T21:23:04.920058+0000","last_undegraded":"2024-05-22T21:23:04.920058+0000","last_fullsized":"2024-05-22T21:23:04.920058+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:06:28.393977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883634+0000","last_change":"2024-05-22T21:23:03.884347+0000","last_active":"2024-05-22T21:23:04.883634+0000","last_peered":"2024-05-22T21:23:04.883634+0000","last_clean":"2024-05-22T21:23:04.883634+0000","last_became_active":"2024-05-22T21:23:03.883895+0000","last_became_peered":"2024-05-22T21:23:03.883895+0000","last_unstale":"2024-05-22T21:23:04.883634+0000","last_undegraded":"2024-05-22T21:23:04.883634+0000","last_fullsized":"2024-05-22T21:23:04.883634+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:26:35.850390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884426+0000","last_change":"2024-05-22T21:23:03.883626+0000","last_active":"2024-05-22T21:23:04.884426+0000","last_peered":"2024-05-22T21:23:04.884426+0000","last_clean":"2024-05-22T21:23:04.884426+0000","last_became_active":"2024-05-22T21:23:03.883104+0000","last_became_peered":"2024-05-22T21:23:03.883104+0000","last_unstale":"2024-05-22T21:23:04.884426+0000","last_undegraded":"2024-05-22T21:23:04.884426+0000","last_fullsized":"2024-05-22T21:23:04.884426+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:05:23.728167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919421+0000","last_change":"2024-05-22T21:23:03.876394+0000","last_active":"2024-05-22T21:23:04.919421+0000","last_peered":"2024-05-22T21:23:04.919421+0000","last_clean":"2024-05-22T21:23:04.919421+0000","last_became_active":"2024-05-22T21:23:03.876120+0000","last_became_peered":"2024-05-22T21:23:03.876120+0000","last_unstale":"2024-05-22T21:23:04.919421+0000","last_undegraded":"2024-05-22T21:23:04.919421+0000","last_fullsized":"2024-05-22T21:23:04.919421+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:35:16.124266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883484+0000","last_change":"2024-05-22T21:23:03.876919+0000","last_active":"2024-05-22T21:23:04.883484+0000","last_peered":"2024-05-22T21:23:04.883484+0000","last_clean":"2024-05-22T21:23:04.883484+0000","last_became_active":"2024-05-22T21:23:03.876692+0000","last_became_peered":"2024-05-22T21:23:03.876692+0000","last_unstale":"2024-05-22T21:23:04.883484+0000","last_undegraded":"2024-05-22T21:23:04.883484+0000","last_fullsized":"2024-05-22T21:23:04.883484+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:26:00.113338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920337+0000","last_change":"2024-05-22T21:23:03.890379+0000","last_active":"2024-05-22T21:23:04.920337+0000","last_peered":"2024-05-22T21:23:04.920337+0000","last_clean":"2024-05-22T21:23:04.920337+0000","last_became_active":"2024-05-22T21:23:03.888170+0000","last_became_peered":"2024-05-22T21:23:03.888170+0000","last_unstale":"2024-05-22T21:23:04.920337+0000","last_undegraded":"2024-05-22T21:23:04.920337+0000","last_fullsized":"2024-05-22T21:23:04.920337+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:27:30.545155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918960+0000","last_change":"2024-05-22T21:23:03.878118+0000","last_active":"2024-05-22T21:23:04.918960+0000","last_peered":"2024-05-22T21:23:04.918960+0000","last_clean":"2024-05-22T21:23:04.918960+0000","last_became_active":"2024-05-22T21:23:03.877648+0000","last_became_peered":"2024-05-22T21:23:03.877648+0000","last_unstale":"2024-05-22T21:23:04.918960+0000","last_undegraded":"2024-05-22T21:23:04.918960+0000","last_fullsized":"2024-05-22T21:23:04.918960+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:06:28.393977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919488+0000","last_change":"2024-05-22T21:23:03.877612+0000","last_active":"2024-05-22T21:23:04.919488+0000","last_peered":"2024-05-22T21:23:04.919488+0000","last_clean":"2024-05-22T21:23:04.919488+0000","last_became_active":"2024-05-22T21:23:03.877360+0000","last_became_peered":"2024-05-22T21:23:03.877360+0000","last_unstale":"2024-05-22T21:23:04.919488+0000","last_undegraded":"2024-05-22T21:23:04.919488+0000","last_fullsized":"2024-05-22T21:23:04.919488+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:34:14.648009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.882971+0000","last_change":"2024-05-22T21:23:03.881364+0000","last_active":"2024-05-22T21:23:04.882971+0000","last_peered":"2024-05-22T21:23:04.882971+0000","last_clean":"2024-05-22T21:23:04.882971+0000","last_became_active":"2024-05-22T21:23:03.880568+0000","last_became_peered":"2024-05-22T21:23:03.880568+0000","last_unstale":"2024-05-22T21:23:04.882971+0000","last_undegraded":"2024-05-22T21:23:04.882971+0000","last_fullsized":"2024-05-22T21:23:04.882971+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:15:48.218923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883727+0000","last_change":"2024-05-22T21:23:03.879480+0000","last_active":"2024-05-22T21:23:04.883727+0000","last_peered":"2024-05-22T21:23:04.883727+0000","last_clean":"2024-05-22T21:23:04.883727+0000","last_became_active":"2024-05-22T21:23:03.878956+0000","last_became_peered":"2024-05-22T21:23:03.878956+0000","last_unstale":"2024-05-22T21:23:04.883727+0000","last_undegraded":"2024-05-22T21:23:04.883727+0000","last_fullsized":"2024-05-22T21:23:04.883727+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:24:12.146293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883371+0000","last_change":"2024-05-22T21:23:03.883548+0000","last_active":"2024-05-22T21:23:04.883371+0000","last_peered":"2024-05-22T21:23:04.883371+0000","last_clean":"2024-05-22T21:23:04.883371+0000","last_became_active":"2024-05-22T21:23:03.883193+0000","last_became_peered":"2024-05-22T21:23:03.883193+0000","last_unstale":"2024-05-22T21:23:04.883371+0000","last_undegraded":"2024-05-22T21:23:04.883371+0000","last_fullsized":"2024-05-22T21:23:04.883371+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:13:00.978006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883046+0000","last_change":"2024-05-22T21:23:03.881375+0000","last_active":"2024-05-22T21:23:04.883046+0000","last_peered":"2024-05-22T21:23:04.883046+0000","last_clean":"2024-05-22T21:23:04.883046+0000","last_became_active":"2024-05-22T21:23:03.880587+0000","last_became_peered":"2024-05-22T21:23:03.880587+0000","last_unstale":"2024-05-22T21:23:04.883046+0000","last_undegraded":"2024-05-22T21:23:04.883046+0000","last_fullsized":"2024-05-22T21:23:04.883046+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:10:59.460151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919535+0000","last_change":"2024-05-22T21:23:03.875646+0000","last_active":"2024-05-22T21:23:04.919535+0000","last_peered":"2024-05-22T21:23:04.919535+0000","last_clean":"2024-05-22T21:23:04.919535+0000","last_became_active":"2024-05-22T21:23:03.875441+0000","last_became_peered":"2024-05-22T21:23:03.875441+0000","last_unstale":"2024-05-22T21:23:04.919535+0000","last_undegraded":"2024-05-22T21:23:04.919535+0000","last_fullsized":"2024-05-22T21:23:04.919535+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:51:34.278141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918927+0000","last_change":"2024-05-22T21:23:03.879494+0000","last_active":"2024-05-22T21:23:04.918927+0000","last_peered":"2024-05-22T21:23:04.918927+0000","last_clean":"2024-05-22T21:23:04.918927+0000","last_became_active":"2024-05-22T21:23:03.878931+0000","last_became_peered":"2024-05-22T21:23:03.878931+0000","last_unstale":"2024-05-22T21:23:04.918927+0000","last_undegraded":"2024-05-22T21:23:04.918927+0000","last_fullsized":"2024-05-22T21:23:04.918927+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:06:25.045909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919110+0000","last_change":"2024-05-22T21:23:03.882340+0000","last_active":"2024-05-22T21:23:04.919110+0000","last_peered":"2024-05-22T21:23:04.919110+0000","last_clean":"2024-05-22T21:23:04.919110+0000","last_became_active":"2024-05-22T21:23:03.882014+0000","last_became_peered":"2024-05-22T21:23:03.882014+0000","last_unstale":"2024-05-22T21:23:04.919110+0000","last_undegraded":"2024-05-22T21:23:04.919110+0000","last_fullsized":"2024-05-22T21:23:04.919110+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:26:21.878604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.877107+0000","last_change":"2024-05-22T21:23:03.885708+0000","last_active":"2024-05-22T21:23:04.877107+0000","last_peered":"2024-05-22T21:23:04.877107+0000","last_clean":"2024-05-22T21:23:04.877107+0000","last_became_active":"2024-05-22T21:23:03.885470+0000","last_became_peered":"2024-05-22T21:23:03.885470+0000","last_unstale":"2024-05-22T21:23:04.877107+0000","last_undegraded":"2024-05-22T21:23:04.877107+0000","last_fullsized":"2024-05-22T21:23:04.877107+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:30:27.417750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918726+0000","last_change":"2024-05-22T21:23:03.877852+0000","last_active":"2024-05-22T21:23:04.918726+0000","last_peered":"2024-05-22T21:23:04.918726+0000","last_clean":"2024-05-22T21:23:04.918726+0000","last_became_active":"2024-05-22T21:23:03.877330+0000","last_became_peered":"2024-05-22T21:23:03.877330+0000","last_unstale":"2024-05-22T21:23:04.918726+0000","last_undegraded":"2024-05-22T21:23:04.918726+0000","last_fullsized":"2024-05-22T21:23:04.918726+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:09:36.781044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884538+0000","last_change":"2024-05-22T21:23:03.882795+0000","last_active":"2024-05-22T21:23:04.884538+0000","last_peered":"2024-05-22T21:23:04.884538+0000","last_clean":"2024-05-22T21:23:04.884538+0000","last_became_active":"2024-05-22T21:23:03.882516+0000","last_became_peered":"2024-05-22T21:23:03.882516+0000","last_unstale":"2024-05-22T21:23:04.884538+0000","last_undegraded":"2024-05-22T21:23:04.884538+0000","last_fullsized":"2024-05-22T21:23:04.884538+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:04:00.574818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919334+0000","last_change":"2024-05-22T21:23:03.884061+0000","last_active":"2024-05-22T21:23:04.919334+0000","last_peered":"2024-05-22T21:23:04.919334+0000","last_clean":"2024-05-22T21:23:04.919334+0000","last_became_active":"2024-05-22T21:23:03.883845+0000","last_became_peered":"2024-05-22T21:23:03.883845+0000","last_unstale":"2024-05-22T21:23:04.919334+0000","last_undegraded":"2024-05-22T21:23:04.919334+0000","last_fullsized":"2024-05-22T21:23:04.919334+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:02:58.646363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919206+0000","last_change":"2024-05-22T21:23:03.882214+0000","last_active":"2024-05-22T21:23:04.919206+0000","last_peered":"2024-05-22T21:23:04.919206+0000","last_clean":"2024-05-22T21:23:04.919206+0000","last_became_active":"2024-05-22T21:23:03.881589+0000","last_became_peered":"2024-05-22T21:23:03.881589+0000","last_unstale":"2024-05-22T21:23:04.919206+0000","last_undegraded":"2024-05-22T21:23:04.919206+0000","last_fullsized":"2024-05-22T21:23:04.919206+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:26:27.533773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883147+0000","last_change":"2024-05-22T21:23:03.877625+0000","last_active":"2024-05-22T21:23:04.883147+0000","last_peered":"2024-05-22T21:23:04.883147+0000","last_clean":"2024-05-22T21:23:04.883147+0000","last_became_active":"2024-05-22T21:23:03.877338+0000","last_became_peered":"2024-05-22T21:23:03.877338+0000","last_unstale":"2024-05-22T21:23:04.883147+0000","last_undegraded":"2024-05-22T21:23:04.883147+0000","last_fullsized":"2024-05-22T21:23:04.883147+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:55:03.432809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"174'4","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883246+0000","last_change":"2024-05-22T21:23:03.502676+0000","last_active":"2024-05-22T21:23:04.883246+0000","last_peered":"2024-05-22T21:23:04.883246+0000","last_clean":"2024-05-22T21:23:04.883246+0000","last_became_active":"2024-05-22T21:23:03.502407+0000","last_became_peered":"2024-05-22T21:23:03.502407+0000","last_unstale":"2024-05-22T21:23:04.883246+0000","last_undegraded":"2024-05-22T21:23:04.883246+0000","last_fullsized":"2024-05-22T21:23:04.883246+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:12:38.116914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918987+0000","last_change":"2024-05-22T21:23:03.876393+0000","last_active":"2024-05-22T21:23:04.918987+0000","last_peered":"2024-05-22T21:23:04.918987+0000","last_clean":"2024-05-22T21:23:04.918987+0000","last_became_active":"2024-05-22T21:23:03.876168+0000","last_became_peered":"2024-05-22T21:23:03.876168+0000","last_unstale":"2024-05-22T21:23:04.918987+0000","last_undegraded":"2024-05-22T21:23:04.918987+0000","last_fullsized":"2024-05-22T21:23:04.918987+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:12:39.124394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919602+0000","last_change":"2024-05-22T21:23:03.499748+0000","last_active":"2024-05-22T21:23:04.919602+0000","last_peered":"2024-05-22T21:23:04.919602+0000","last_clean":"2024-05-22T21:23:04.919602+0000","last_became_active":"2024-05-22T21:23:03.499260+0000","last_became_peered":"2024-05-22T21:23:03.499260+0000","last_unstale":"2024-05-22T21:23:04.919602+0000","last_undegraded":"2024-05-22T21:23:04.919602+0000","last_fullsized":"2024-05-22T21:23:04.919602+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:26:34.842909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919015+0000","last_change":"2024-05-22T21:23:03.880013+0000","last_active":"2024-05-22T21:23:04.919015+0000","last_peered":"2024-05-22T21:23:04.919015+0000","last_clean":"2024-05-22T21:23:04.919015+0000","last_became_active":"2024-05-22T21:23:03.879692+0000","last_became_peered":"2024-05-22T21:23:03.879692+0000","last_unstale":"2024-05-22T21:23:04.919015+0000","last_undegraded":"2024-05-22T21:23:04.919015+0000","last_fullsized":"2024-05-22T21:23:04.919015+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:44:02.437988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884678+0000","last_change":"2024-05-22T21:23:03.504699+0000","last_active":"2024-05-22T21:23:04.884678+0000","last_peered":"2024-05-22T21:23:04.884678+0000","last_clean":"2024-05-22T21:23:04.884678+0000","last_became_active":"2024-05-22T21:23:03.504457+0000","last_became_peered":"2024-05-22T21:23:03.504457+0000","last_unstale":"2024-05-22T21:23:04.884678+0000","last_undegraded":"2024-05-22T21:23:04.884678+0000","last_fullsized":"2024-05-22T21:23:04.884678+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:44:01.430507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884472+0000","last_change":"2024-05-22T21:23:03.883811+0000","last_active":"2024-05-22T21:23:04.884472+0000","last_peered":"2024-05-22T21:23:04.884472+0000","last_clean":"2024-05-22T21:23:04.884472+0000","last_became_active":"2024-05-22T21:23:03.883412+0000","last_became_peered":"2024-05-22T21:23:03.883412+0000","last_unstale":"2024-05-22T21:23:04.884472+0000","last_undegraded":"2024-05-22T21:23:04.884472+0000","last_fullsized":"2024-05-22T21:23:04.884472+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:55:23.600882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884013+0000","last_change":"2024-05-22T21:23:03.504387+0000","last_active":"2024-05-22T21:23:04.884013+0000","last_peered":"2024-05-22T21:23:04.884013+0000","last_clean":"2024-05-22T21:23:04.884013+0000","last_became_active":"2024-05-22T21:23:03.503974+0000","last_became_peered":"2024-05-22T21:23:03.503974+0000","last_unstale":"2024-05-22T21:23:04.884013+0000","last_undegraded":"2024-05-22T21:23:04.884013+0000","last_fullsized":"2024-05-22T21:23:04.884013+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:09:35.773564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919063+0000","last_change":"2024-05-22T21:23:03.884097+0000","last_active":"2024-05-22T21:23:04.919063+0000","last_peered":"2024-05-22T21:23:04.919063+0000","last_clean":"2024-05-22T21:23:04.919063+0000","last_became_active":"2024-05-22T21:23:03.883741+0000","last_became_peered":"2024-05-22T21:23:03.883741+0000","last_unstale":"2024-05-22T21:23:04.919063+0000","last_undegraded":"2024-05-22T21:23:04.919063+0000","last_fullsized":"2024-05-22T21:23:04.919063+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:23:57.604650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919383+0000","last_change":"2024-05-22T21:23:03.500065+0000","last_active":"2024-05-22T21:23:04.919383+0000","last_peered":"2024-05-22T21:23:04.919383+0000","last_clean":"2024-05-22T21:23:04.919383+0000","last_became_active":"2024-05-22T21:23:03.498678+0000","last_became_peered":"2024-05-22T21:23:03.498678+0000","last_unstale":"2024-05-22T21:23:04.919383+0000","last_undegraded":"2024-05-22T21:23:04.919383+0000","last_fullsized":"2024-05-22T21:23:04.919383+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:26:51.655443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883817+0000","last_change":"2024-05-22T21:23:03.878032+0000","last_active":"2024-05-22T21:23:04.883817+0000","last_peered":"2024-05-22T21:23:04.883817+0000","last_clean":"2024-05-22T21:23:04.883817+0000","last_became_active":"2024-05-22T21:23:03.877762+0000","last_became_peered":"2024-05-22T21:23:03.877762+0000","last_unstale":"2024-05-22T21:23:04.883817+0000","last_undegraded":"2024-05-22T21:23:04.883817+0000","last_fullsized":"2024-05-22T21:23:04.883817+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:50:02.421057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920533+0000","last_change":"2024-05-22T21:23:03.505146+0000","last_active":"2024-05-22T21:23:04.920533+0000","last_peered":"2024-05-22T21:23:04.920533+0000","last_clean":"2024-05-22T21:23:04.920533+0000","last_became_active":"2024-05-22T21:23:03.504683+0000","last_became_peered":"2024-05-22T21:23:03.504683+0000","last_unstale":"2024-05-22T21:23:04.920533+0000","last_undegraded":"2024-05-22T21:23:04.920533+0000","last_fullsized":"2024-05-22T21:23:04.920533+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:00:40.034709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919490+0000","last_change":"2024-05-22T21:23:03.881027+0000","last_active":"2024-05-22T21:23:04.919490+0000","last_peered":"2024-05-22T21:23:04.919490+0000","last_clean":"2024-05-22T21:23:04.919490+0000","last_became_active":"2024-05-22T21:23:03.880824+0000","last_became_peered":"2024-05-22T21:23:03.880824+0000","last_unstale":"2024-05-22T21:23:04.919490+0000","last_undegraded":"2024-05-22T21:23:04.919490+0000","last_fullsized":"2024-05-22T21:23:04.919490+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:06:44.193264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919323+0000","last_change":"2024-05-22T21:23:03.501791+0000","last_active":"2024-05-22T21:23:04.919323+0000","last_peered":"2024-05-22T21:23:04.919323+0000","last_clean":"2024-05-22T21:23:04.919323+0000","last_became_active":"2024-05-22T21:23:03.501582+0000","last_became_peered":"2024-05-22T21:23:03.501582+0000","last_unstale":"2024-05-22T21:23:04.919323+0000","last_undegraded":"2024-05-22T21:23:04.919323+0000","last_fullsized":"2024-05-22T21:23:04.919323+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:19:21.792180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920096+0000","last_change":"2024-05-22T21:23:03.881307+0000","last_active":"2024-05-22T21:23:04.920096+0000","last_peered":"2024-05-22T21:23:04.920096+0000","last_clean":"2024-05-22T21:23:04.920096+0000","last_became_active":"2024-05-22T21:23:03.881047+0000","last_became_peered":"2024-05-22T21:23:03.881047+0000","last_unstale":"2024-05-22T21:23:04.920096+0000","last_undegraded":"2024-05-22T21:23:04.920096+0000","last_fullsized":"2024-05-22T21:23:04.920096+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:26:27.533773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884759+0000","last_change":"2024-05-22T21:23:03.503260+0000","last_active":"2024-05-22T21:23:04.884759+0000","last_peered":"2024-05-22T21:23:04.884759+0000","last_clean":"2024-05-22T21:23:04.884759+0000","last_became_active":"2024-05-22T21:23:03.502992+0000","last_became_peered":"2024-05-22T21:23:03.502992+0000","last_unstale":"2024-05-22T21:23:04.884759+0000","last_undegraded":"2024-05-22T21:23:04.884759+0000","last_fullsized":"2024-05-22T21:23:04.884759+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:13:33.310983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920314+0000","last_change":"2024-05-22T21:23:03.885333+0000","last_active":"2024-05-22T21:23:04.920314+0000","last_peered":"2024-05-22T21:23:04.920314+0000","last_clean":"2024-05-22T21:23:04.920314+0000","last_became_active":"2024-05-22T21:23:03.884648+0000","last_became_peered":"2024-05-22T21:23:03.884648+0000","last_unstale":"2024-05-22T21:23:04.920314+0000","last_undegraded":"2024-05-22T21:23:04.920314+0000","last_fullsized":"2024-05-22T21:23:04.920314+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:26:21.878604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919414+0000","last_change":"2024-05-22T21:23:03.500679+0000","last_active":"2024-05-22T21:23:04.919414+0000","last_peered":"2024-05-22T21:23:04.919414+0000","last_clean":"2024-05-22T21:23:04.919414+0000","last_became_active":"2024-05-22T21:23:03.500429+0000","last_became_peered":"2024-05-22T21:23:03.500429+0000","last_unstale":"2024-05-22T21:23:04.919414+0000","last_undegraded":"2024-05-22T21:23:04.919414+0000","last_fullsized":"2024-05-22T21:23:04.919414+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:55.213155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883095+0000","last_change":"2024-05-22T21:23:03.876887+0000","last_active":"2024-05-22T21:23:04.883095+0000","last_peered":"2024-05-22T21:23:04.883095+0000","last_clean":"2024-05-22T21:23:04.883095+0000","last_became_active":"2024-05-22T21:23:03.876626+0000","last_became_peered":"2024-05-22T21:23:03.876626+0000","last_unstale":"2024-05-22T21:23:04.883095+0000","last_undegraded":"2024-05-22T21:23:04.883095+0000","last_fullsized":"2024-05-22T21:23:04.883095+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:52:59.054693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883391+0000","last_change":"2024-05-22T21:23:03.507647+0000","last_active":"2024-05-22T21:23:04.883391+0000","last_peered":"2024-05-22T21:23:04.883391+0000","last_clean":"2024-05-22T21:23:04.883391+0000","last_became_active":"2024-05-22T21:23:03.506896+0000","last_became_peered":"2024-05-22T21:23:03.506896+0000","last_unstale":"2024-05-22T21:23:04.883391+0000","last_undegraded":"2024-05-22T21:23:04.883391+0000","last_fullsized":"2024-05-22T21:23:04.883391+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:23:56.597169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919229+0000","last_change":"2024-05-22T21:23:03.876890+0000","last_active":"2024-05-22T21:23:04.919229+0000","last_peered":"2024-05-22T21:23:04.919229+0000","last_clean":"2024-05-22T21:23:04.919229+0000","last_became_active":"2024-05-22T21:23:03.876650+0000","last_became_peered":"2024-05-22T21:23:03.876650+0000","last_unstale":"2024-05-22T21:23:04.919229+0000","last_undegraded":"2024-05-22T21:23:04.919229+0000","last_fullsized":"2024-05-22T21:23:04.919229+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:20:48.302649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"174'6","reported_seq":17,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.876850+0000","last_change":"2024-05-22T21:23:03.495709+0000","last_active":"2024-05-22T21:23:04.876850+0000","last_peered":"2024-05-22T21:23:04.876850+0000","last_clean":"2024-05-22T21:23:04.876850+0000","last_became_active":"2024-05-22T21:23:03.495136+0000","last_became_peered":"2024-05-22T21:23:03.495136+0000","last_unstale":"2024-05-22T21:23:04.876850+0000","last_undegraded":"2024-05-22T21:23:04.876850+0000","last_fullsized":"2024-05-22T21:23:04.876850+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:31:10.620206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920788+0000","last_change":"2024-05-22T21:23:03.887898+0000","last_active":"2024-05-22T21:23:04.920788+0000","last_peered":"2024-05-22T21:23:04.920788+0000","last_clean":"2024-05-22T21:23:04.920788+0000","last_became_active":"2024-05-22T21:23:03.887243+0000","last_became_peered":"2024-05-22T21:23:03.887243+0000","last_unstale":"2024-05-22T21:23:04.920788+0000","last_undegraded":"2024-05-22T21:23:04.920788+0000","last_fullsized":"2024-05-22T21:23:04.920788+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:56.220635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920722+0000","last_change":"2024-05-22T21:23:03.505356+0000","last_active":"2024-05-22T21:23:04.920722+0000","last_peered":"2024-05-22T21:23:04.920722+0000","last_clean":"2024-05-22T21:23:04.920722+0000","last_became_active":"2024-05-22T21:23:03.505102+0000","last_became_peered":"2024-05-22T21:23:03.505102+0000","last_unstale":"2024-05-22T21:23:04.920722+0000","last_undegraded":"2024-05-22T21:23:04.920722+0000","last_fullsized":"2024-05-22T21:23:04.920722+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:32.797433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919274+0000","last_change":"2024-05-22T21:23:03.877777+0000","last_active":"2024-05-22T21:23:04.919274+0000","last_peered":"2024-05-22T21:23:04.919274+0000","last_clean":"2024-05-22T21:23:04.919274+0000","last_became_active":"2024-05-22T21:23:03.877570+0000","last_became_peered":"2024-05-22T21:23:03.877570+0000","last_unstale":"2024-05-22T21:23:04.919274+0000","last_undegraded":"2024-05-22T21:23:04.919274+0000","last_fullsized":"2024-05-22T21:23:04.919274+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:59:12.425717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920854+0000","last_change":"2024-05-22T21:23:03.503186+0000","last_active":"2024-05-22T21:23:04.920854+0000","last_peered":"2024-05-22T21:23:04.920854+0000","last_clean":"2024-05-22T21:23:04.920854+0000","last_became_active":"2024-05-22T21:23:03.502941+0000","last_became_peered":"2024-05-22T21:23:03.502941+0000","last_unstale":"2024-05-22T21:23:04.920854+0000","last_undegraded":"2024-05-22T21:23:04.920854+0000","last_fullsized":"2024-05-22T21:23:04.920854+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:40:20.750446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920169+0000","last_change":"2024-05-22T21:23:03.887561+0000","last_active":"2024-05-22T21:23:04.920169+0000","last_peered":"2024-05-22T21:23:04.920169+0000","last_clean":"2024-05-22T21:23:04.920169+0000","last_became_active":"2024-05-22T21:23:03.886909+0000","last_became_peered":"2024-05-22T21:23:03.886909+0000","last_unstale":"2024-05-22T21:23:04.920169+0000","last_undegraded":"2024-05-22T21:23:04.920169+0000","last_fullsized":"2024-05-22T21:23:04.920169+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:02:40.386927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918515+0000","last_change":"2024-05-22T21:23:03.496061+0000","last_active":"2024-05-22T21:23:04.918515+0000","last_peered":"2024-05-22T21:23:04.918515+0000","last_clean":"2024-05-22T21:23:04.918515+0000","last_became_active":"2024-05-22T21:23:03.495850+0000","last_became_peered":"2024-05-22T21:23:03.495850+0000","last_unstale":"2024-05-22T21:23:04.918515+0000","last_undegraded":"2024-05-22T21:23:04.918515+0000","last_fullsized":"2024-05-22T21:23:04.918515+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:32:32.161836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920716+0000","last_change":"2024-05-22T21:23:03.888426+0000","last_active":"2024-05-22T21:23:04.920716+0000","last_peered":"2024-05-22T21:23:04.920716+0000","last_clean":"2024-05-22T21:23:04.920716+0000","last_became_active":"2024-05-22T21:23:03.883306+0000","last_became_peered":"2024-05-22T21:23:03.883306+0000","last_unstale":"2024-05-22T21:23:04.920716+0000","last_undegraded":"2024-05-22T21:23:04.920716+0000","last_fullsized":"2024-05-22T21:23:04.920716+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:20.274667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"174'8","reported_seq":19,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919313+0000","last_change":"2024-05-22T21:23:03.505247+0000","last_active":"2024-05-22T21:23:04.919313+0000","last_peered":"2024-05-22T21:23:04.919313+0000","last_clean":"2024-05-22T21:23:04.919313+0000","last_became_active":"2024-05-22T21:23:03.505022+0000","last_became_peered":"2024-05-22T21:23:03.505022+0000","last_unstale":"2024-05-22T21:23:04.919313+0000","last_undegraded":"2024-05-22T21:23:04.919313+0000","last_fullsized":"2024-05-22T21:23:04.919313+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:44:00.938486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884519+0000","last_change":"2024-05-22T21:23:03.884246+0000","last_active":"2024-05-22T21:23:04.884519+0000","last_peered":"2024-05-22T21:23:04.884519+0000","last_clean":"2024-05-22T21:23:04.884519+0000","last_became_active":"2024-05-22T21:23:03.884036+0000","last_became_peered":"2024-05-22T21:23:03.884036+0000","last_unstale":"2024-05-22T21:23:04.884519+0000","last_undegraded":"2024-05-22T21:23:04.884519+0000","last_fullsized":"2024-05-22T21:23:04.884519+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:00:17.253740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920272+0000","last_change":"2024-05-22T21:23:03.505737+0000","last_active":"2024-05-22T21:23:04.920272+0000","last_peered":"2024-05-22T21:23:04.920272+0000","last_clean":"2024-05-22T21:23:04.920272+0000","last_became_active":"2024-05-22T21:23:03.505355+0000","last_became_peered":"2024-05-22T21:23:03.505355+0000","last_unstale":"2024-05-22T21:23:04.920272+0000","last_undegraded":"2024-05-22T21:23:04.920272+0000","last_fullsized":"2024-05-22T21:23:04.920272+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:31:12.385932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919165+0000","last_change":"2024-05-22T21:23:03.881207+0000","last_active":"2024-05-22T21:23:04.919165+0000","last_peered":"2024-05-22T21:23:04.919165+0000","last_clean":"2024-05-22T21:23:04.919165+0000","last_became_active":"2024-05-22T21:23:03.880905+0000","last_became_peered":"2024-05-22T21:23:03.880905+0000","last_unstale":"2024-05-22T21:23:04.919165+0000","last_undegraded":"2024-05-22T21:23:04.919165+0000","last_fullsized":"2024-05-22T21:23:04.919165+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:59.125519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919573+0000","last_change":"2024-05-22T21:23:03.503749+0000","last_active":"2024-05-22T21:23:04.919573+0000","last_peered":"2024-05-22T21:23:04.919573+0000","last_clean":"2024-05-22T21:23:04.919573+0000","last_became_active":"2024-05-22T21:23:03.503530+0000","last_became_peered":"2024-05-22T21:23:03.503530+0000","last_unstale":"2024-05-22T21:23:04.919573+0000","last_undegraded":"2024-05-22T21:23:04.919573+0000","last_fullsized":"2024-05-22T21:23:04.919573+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:12:59.970525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920790+0000","last_change":"2024-05-22T21:23:03.886291+0000","last_active":"2024-05-22T21:23:04.920790+0000","last_peered":"2024-05-22T21:23:04.920790+0000","last_clean":"2024-05-22T21:23:04.920790+0000","last_became_active":"2024-05-22T21:23:03.882943+0000","last_became_peered":"2024-05-22T21:23:03.882943+0000","last_unstale":"2024-05-22T21:23:04.920790+0000","last_undegraded":"2024-05-22T21:23:04.920790+0000","last_fullsized":"2024-05-22T21:23:04.920790+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:50:02.421057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919574+0000","last_change":"2024-05-22T21:23:03.497137+0000","last_active":"2024-05-22T21:23:04.919574+0000","last_peered":"2024-05-22T21:23:04.919574+0000","last_clean":"2024-05-22T21:23:04.919574+0000","last_became_active":"2024-05-22T21:23:03.496631+0000","last_became_peered":"2024-05-22T21:23:03.496631+0000","last_unstale":"2024-05-22T21:23:04.919574+0000","last_undegraded":"2024-05-22T21:23:04.919574+0000","last_fullsized":"2024-05-22T21:23:04.919574+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:22:52.446757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919156+0000","last_change":"2024-05-22T21:23:03.884773+0000","last_active":"2024-05-22T21:23:04.919156+0000","last_peered":"2024-05-22T21:23:04.919156+0000","last_clean":"2024-05-22T21:23:04.919156+0000","last_became_active":"2024-05-22T21:23:03.884239+0000","last_became_peered":"2024-05-22T21:23:03.884239+0000","last_unstale":"2024-05-22T21:23:04.919156+0000","last_undegraded":"2024-05-22T21:23:04.919156+0000","last_fullsized":"2024-05-22T21:23:04.919156+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:37.096853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918950+0000","last_change":"2024-05-22T21:23:03.501930+0000","last_active":"2024-05-22T21:23:04.918950+0000","last_peered":"2024-05-22T21:23:04.918950+0000","last_clean":"2024-05-22T21:23:04.918950+0000","last_became_active":"2024-05-22T21:23:03.501581+0000","last_became_peered":"2024-05-22T21:23:03.501581+0000","last_unstale":"2024-05-22T21:23:04.918950+0000","last_undegraded":"2024-05-22T21:23:04.918950+0000","last_fullsized":"2024-05-22T21:23:04.918950+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:24:11.138812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883734+0000","last_change":"2024-05-22T21:23:03.881211+0000","last_active":"2024-05-22T21:23:04.883734+0000","last_peered":"2024-05-22T21:23:04.883734+0000","last_clean":"2024-05-22T21:23:04.883734+0000","last_became_active":"2024-05-22T21:23:03.880995+0000","last_became_peered":"2024-05-22T21:23:03.880995+0000","last_unstale":"2024-05-22T21:23:04.883734+0000","last_undegraded":"2024-05-22T21:23:04.883734+0000","last_fullsized":"2024-05-22T21:23:04.883734+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:56.220635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.877199+0000","last_change":"2024-05-22T21:23:03.505393+0000","last_active":"2024-05-22T21:23:04.877199+0000","last_peered":"2024-05-22T21:23:04.877199+0000","last_clean":"2024-05-22T21:23:04.877199+0000","last_became_active":"2024-05-22T21:23:03.505116+0000","last_became_peered":"2024-05-22T21:23:03.505116+0000","last_unstale":"2024-05-22T21:23:04.877199+0000","last_undegraded":"2024-05-22T21:23:04.877199+0000","last_fullsized":"2024-05-22T21:23:04.877199+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:59:11.418236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.877156+0000","last_change":"2024-05-22T21:23:03.879909+0000","last_active":"2024-05-22T21:23:04.877156+0000","last_peered":"2024-05-22T21:23:04.877156+0000","last_clean":"2024-05-22T21:23:04.877156+0000","last_became_active":"2024-05-22T21:23:03.879636+0000","last_became_peered":"2024-05-22T21:23:03.879636+0000","last_unstale":"2024-05-22T21:23:04.877156+0000","last_undegraded":"2024-05-22T21:23:04.877156+0000","last_fullsized":"2024-05-22T21:23:04.877156+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:37:41.376144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883350+0000","last_change":"2024-05-22T21:23:03.499340+0000","last_active":"2024-05-22T21:23:04.883350+0000","last_peered":"2024-05-22T21:23:04.883350+0000","last_clean":"2024-05-22T21:23:04.883350+0000","last_became_active":"2024-05-22T21:23:03.498048+0000","last_became_peered":"2024-05-22T21:23:03.498048+0000","last_unstale":"2024-05-22T21:23:04.883350+0000","last_undegraded":"2024-05-22T21:23:04.883350+0000","last_fullsized":"2024-05-22T21:23:04.883350+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:21:25.641549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883902+0000","last_change":"2024-05-22T21:23:03.880182+0000","last_active":"2024-05-22T21:23:04.883902+0000","last_peered":"2024-05-22T21:23:04.883902+0000","last_clean":"2024-05-22T21:23:04.883902+0000","last_became_active":"2024-05-22T21:23:03.879938+0000","last_became_peered":"2024-05-22T21:23:03.879938+0000","last_unstale":"2024-05-22T21:23:04.883902+0000","last_undegraded":"2024-05-22T21:23:04.883902+0000","last_fullsized":"2024-05-22T21:23:04.883902+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:59:47.725165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884562+0000","last_change":"2024-05-22T21:23:03.498487+0000","last_active":"2024-05-22T21:23:04.884562+0000","last_peered":"2024-05-22T21:23:04.884562+0000","last_clean":"2024-05-22T21:23:04.884562+0000","last_became_active":"2024-05-22T21:23:03.498233+0000","last_became_peered":"2024-05-22T21:23:03.498233+0000","last_unstale":"2024-05-22T21:23:04.884562+0000","last_undegraded":"2024-05-22T21:23:04.884562+0000","last_fullsized":"2024-05-22T21:23:04.884562+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:02:57.638883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.877246+0000","last_change":"2024-05-22T21:23:03.878599+0000","last_active":"2024-05-22T21:23:04.877246+0000","last_peered":"2024-05-22T21:23:04.877246+0000","last_clean":"2024-05-22T21:23:04.877246+0000","last_became_active":"2024-05-22T21:23:03.875474+0000","last_became_peered":"2024-05-22T21:23:03.875474+0000","last_unstale":"2024-05-22T21:23:04.877246+0000","last_undegraded":"2024-05-22T21:23:04.877246+0000","last_fullsized":"2024-05-22T21:23:04.877246+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:45:43.497247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920904+0000","last_change":"2024-05-22T21:23:03.502189+0000","last_active":"2024-05-22T21:23:04.920904+0000","last_peered":"2024-05-22T21:23:04.920904+0000","last_clean":"2024-05-22T21:23:04.920904+0000","last_became_active":"2024-05-22T21:23:03.501893+0000","last_became_peered":"2024-05-22T21:23:03.501893+0000","last_unstale":"2024-05-22T21:23:04.920904+0000","last_undegraded":"2024-05-22T21:23:04.920904+0000","last_fullsized":"2024-05-22T21:23:04.920904+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:32:10.359880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920358+0000","last_change":"2024-05-22T21:23:03.887421+0000","last_active":"2024-05-22T21:23:04.920358+0000","last_peered":"2024-05-22T21:23:04.920358+0000","last_clean":"2024-05-22T21:23:04.920358+0000","last_became_active":"2024-05-22T21:23:03.886618+0000","last_became_peered":"2024-05-22T21:23:03.886618+0000","last_unstale":"2024-05-22T21:23:04.920358+0000","last_undegraded":"2024-05-22T21:23:04.920358+0000","last_fullsized":"2024-05-22T21:23:04.920358+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:53.702637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883567+0000","last_change":"2024-05-22T21:23:03.507143+0000","last_active":"2024-05-22T21:23:04.883567+0000","last_peered":"2024-05-22T21:23:04.883567+0000","last_clean":"2024-05-22T21:23:04.883567+0000","last_became_active":"2024-05-22T21:23:03.504672+0000","last_became_peered":"2024-05-22T21:23:03.504672+0000","last_unstale":"2024-05-22T21:23:04.883567+0000","last_undegraded":"2024-05-22T21:23:04.883567+0000","last_fullsized":"2024-05-22T21:23:04.883567+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:59:11.418236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920850+0000","last_change":"2024-05-22T21:23:03.883520+0000","last_active":"2024-05-22T21:23:04.920850+0000","last_peered":"2024-05-22T21:23:04.920850+0000","last_clean":"2024-05-22T21:23:04.920850+0000","last_became_active":"2024-05-22T21:23:03.882636+0000","last_became_peered":"2024-05-22T21:23:03.882636+0000","last_unstale":"2024-05-22T21:23:04.920850+0000","last_undegraded":"2024-05-22T21:23:04.920850+0000","last_fullsized":"2024-05-22T21:23:04.920850+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:26:35.850390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918671+0000","last_change":"2024-05-22T21:23:03.499566+0000","last_active":"2024-05-22T21:23:04.918671+0000","last_peered":"2024-05-22T21:23:04.918671+0000","last_clean":"2024-05-22T21:23:04.918671+0000","last_became_active":"2024-05-22T21:23:03.499164+0000","last_became_peered":"2024-05-22T21:23:03.499164+0000","last_unstale":"2024-05-22T21:23:04.918671+0000","last_undegraded":"2024-05-22T21:23:04.918671+0000","last_fullsized":"2024-05-22T21:23:04.918671+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:36:33.702861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919357+0000","last_change":"2024-05-22T21:23:03.885996+0000","last_active":"2024-05-22T21:23:04.919357+0000","last_peered":"2024-05-22T21:23:04.919357+0000","last_clean":"2024-05-22T21:23:04.919357+0000","last_became_active":"2024-05-22T21:23:03.885790+0000","last_became_peered":"2024-05-22T21:23:03.885790+0000","last_unstale":"2024-05-22T21:23:04.919357+0000","last_undegraded":"2024-05-22T21:23:04.919357+0000","last_fullsized":"2024-05-22T21:23:04.919357+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:41:15.882835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919041+0000","last_change":"2024-05-22T21:23:03.499933+0000","last_active":"2024-05-22T21:23:04.919041+0000","last_peered":"2024-05-22T21:23:04.919041+0000","last_clean":"2024-05-22T21:23:04.919041+0000","last_became_active":"2024-05-22T21:23:03.499274+0000","last_became_peered":"2024-05-22T21:23:03.499274+0000","last_unstale":"2024-05-22T21:23:04.919041+0000","last_undegraded":"2024-05-22T21:23:04.919041+0000","last_fullsized":"2024-05-22T21:23:04.919041+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:20:47.295168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918815+0000","last_change":"2024-05-22T21:23:03.879664+0000","last_active":"2024-05-22T21:23:04.918815+0000","last_peered":"2024-05-22T21:23:04.918815+0000","last_clean":"2024-05-22T21:23:04.918815+0000","last_became_active":"2024-05-22T21:23:03.879445+0000","last_became_peered":"2024-05-22T21:23:03.879445+0000","last_unstale":"2024-05-22T21:23:04.918815+0000","last_undegraded":"2024-05-22T21:23:04.918815+0000","last_fullsized":"2024-05-22T21:23:04.918815+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:13:34.318464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918798+0000","last_change":"2024-05-22T21:23:03.498403+0000","last_active":"2024-05-22T21:23:04.918798+0000","last_peered":"2024-05-22T21:23:04.918798+0000","last_clean":"2024-05-22T21:23:04.918798+0000","last_became_active":"2024-05-22T21:23:03.497966+0000","last_became_peered":"2024-05-22T21:23:03.497966+0000","last_unstale":"2024-05-22T21:23:04.918798+0000","last_undegraded":"2024-05-22T21:23:04.918798+0000","last_fullsized":"2024-05-22T21:23:04.918798+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:51:33.270661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919296+0000","last_change":"2024-05-22T21:23:03.879434+0000","last_active":"2024-05-22T21:23:04.919296+0000","last_peered":"2024-05-22T21:23:04.919296+0000","last_clean":"2024-05-22T21:23:04.919296+0000","last_became_active":"2024-05-22T21:23:03.879230+0000","last_became_peered":"2024-05-22T21:23:03.879230+0000","last_unstale":"2024-05-22T21:23:04.919296+0000","last_undegraded":"2024-05-22T21:23:04.919296+0000","last_fullsized":"2024-05-22T21:23:04.919296+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:46:30.050409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.877107+0000","last_change":"2024-05-22T21:23:03.500735+0000","last_active":"2024-05-22T21:23:04.877107+0000","last_peered":"2024-05-22T21:23:04.877107+0000","last_clean":"2024-05-22T21:23:04.877107+0000","last_became_active":"2024-05-22T21:23:03.500489+0000","last_became_peered":"2024-05-22T21:23:03.500489+0000","last_unstale":"2024-05-22T21:23:04.877107+0000","last_undegraded":"2024-05-22T21:23:04.877107+0000","last_fullsized":"2024-05-22T21:23:04.877107+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:23:56.597169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920370+0000","last_change":"2024-05-22T21:23:03.891467+0000","last_active":"2024-05-22T21:23:04.920370+0000","last_peered":"2024-05-22T21:23:04.920370+0000","last_clean":"2024-05-22T21:23:04.920370+0000","last_became_active":"2024-05-22T21:23:03.891234+0000","last_became_peered":"2024-05-22T21:23:03.891234+0000","last_unstale":"2024-05-22T21:23:04.920370+0000","last_undegraded":"2024-05-22T21:23:04.920370+0000","last_fullsized":"2024-05-22T21:23:04.920370+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:33:36.236974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883645+0000","last_change":"2024-05-22T21:23:03.507443+0000","last_active":"2024-05-22T21:23:04.883645+0000","last_peered":"2024-05-22T21:23:04.883645+0000","last_clean":"2024-05-22T21:23:04.883645+0000","last_became_active":"2024-05-22T21:23:03.505194+0000","last_became_peered":"2024-05-22T21:23:03.505194+0000","last_unstale":"2024-05-22T21:23:04.883645+0000","last_undegraded":"2024-05-22T21:23:04.883645+0000","last_fullsized":"2024-05-22T21:23:04.883645+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:40:39.781464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883252+0000","last_change":"2024-05-22T21:23:03.885111+0000","last_active":"2024-05-22T21:23:04.883252+0000","last_peered":"2024-05-22T21:23:04.883252+0000","last_clean":"2024-05-22T21:23:04.883252+0000","last_became_active":"2024-05-22T21:23:03.881157+0000","last_became_peered":"2024-05-22T21:23:03.881157+0000","last_unstale":"2024-05-22T21:23:04.883252+0000","last_undegraded":"2024-05-22T21:23:04.883252+0000","last_fullsized":"2024-05-22T21:23:04.883252+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:06:32.394718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920818+0000","last_change":"2024-05-22T21:23:03.507764+0000","last_active":"2024-05-22T21:23:04.920818+0000","last_peered":"2024-05-22T21:23:04.920818+0000","last_clean":"2024-05-22T21:23:04.920818+0000","last_became_active":"2024-05-22T21:23:03.506771+0000","last_became_peered":"2024-05-22T21:23:03.506771+0000","last_unstale":"2024-05-22T21:23:04.920818+0000","last_undegraded":"2024-05-22T21:23:04.920818+0000","last_fullsized":"2024-05-22T21:23:04.920818+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:44:45.599821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920438+0000","last_change":"2024-05-22T21:23:03.881367+0000","last_active":"2024-05-22T21:23:04.920438+0000","last_peered":"2024-05-22T21:23:04.920438+0000","last_clean":"2024-05-22T21:23:04.920438+0000","last_became_active":"2024-05-22T21:23:03.881079+0000","last_became_peered":"2024-05-22T21:23:03.881079+0000","last_unstale":"2024-05-22T21:23:04.920438+0000","last_undegraded":"2024-05-22T21:23:04.920438+0000","last_fullsized":"2024-05-22T21:23:04.920438+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:37.096853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919678+0000","last_change":"2024-05-22T21:23:03.499886+0000","last_active":"2024-05-22T21:23:04.919678+0000","last_peered":"2024-05-22T21:23:04.919678+0000","last_clean":"2024-05-22T21:23:04.919678+0000","last_became_active":"2024-05-22T21:23:03.499545+0000","last_became_peered":"2024-05-22T21:23:03.499545+0000","last_unstale":"2024-05-22T21:23:04.919678+0000","last_undegraded":"2024-05-22T21:23:04.919678+0000","last_fullsized":"2024-05-22T21:23:04.919678+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:03:59.567337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.876945+0000","last_change":"2024-05-22T21:23:03.879540+0000","last_active":"2024-05-22T21:23:04.876945+0000","last_peered":"2024-05-22T21:23:04.876945+0000","last_clean":"2024-05-22T21:23:04.876945+0000","last_became_active":"2024-05-22T21:23:03.879329+0000","last_became_peered":"2024-05-22T21:23:03.879329+0000","last_unstale":"2024-05-22T21:23:04.876945+0000","last_undegraded":"2024-05-22T21:23:04.876945+0000","last_fullsized":"2024-05-22T21:23:04.876945+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:22:27.742690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"174'3","reported_seq":14,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920725+0000","last_change":"2024-05-22T21:23:03.507316+0000","last_active":"2024-05-22T21:23:04.920725+0000","last_peered":"2024-05-22T21:23:04.920725+0000","last_clean":"2024-05-22T21:23:04.920725+0000","last_became_active":"2024-05-22T21:23:03.506228+0000","last_became_peered":"2024-05-22T21:23:03.506228+0000","last_unstale":"2024-05-22T21:23:04.920725+0000","last_undegraded":"2024-05-22T21:23:04.920725+0000","last_fullsized":"2024-05-22T21:23:04.920725+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:48:30.664137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":322,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-05-22T21:23:03.872023+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:23:03.872023+0000","last_peered":"2024-05-22T21:23:03.872023+0000","last_clean":"2024-05-22T21:23:03.872023+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:23:03.872023+0000","last_undegraded":"2024-05-22T21:23:03.872023+0000","last_fullsized":"2024-05-22T21:23:03.872023+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574969,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1535,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193776,"statfs":{"total":200000000,"available":198427434,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.69399999999999995,"15min":0.73299999999999998},"min":{"1min":0.48599999999999999,"5min":0.40000000000000002,"15min":0.36699999999999999},"max":{"1min":1.022,"5min":1.022,"15min":2.0369999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.54700000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.889},"last":0.754}]},{"osd":1,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71499999999999997,"15min":0.752},"min":{"1min":0.53200000000000003,"5min":0.42799999999999999,"15min":0.41699999999999998},"max":{"1min":1.008,"5min":1.0329999999999999,"15min":3.0920000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.71999999999999997,"15min":0.74399999999999999},"min":{"1min":0.50600000000000001,"5min":0.45200000000000001,"15min":0.39800000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":3.1499999999999999},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.70299999999999996,"15min":0.70199999999999996},"min":{"1min":0.60899999999999999,"5min":0.44400000000000001,"15min":0.38800000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.923},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.56200000000000006,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.878}]},{"osd":3,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72799999999999998,"15min":0.70799999999999996},"min":{"1min":0.57299999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.752,"15min":0.77900000000000003},"min":{"1min":0.64100000000000001,"5min":0.47599999999999998,"15min":0.44500000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":2.1389999999999998},"last":0.97699999999999998}]},{"osd":4,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57399999999999995,"15min":0.58099999999999996},"min":{"1min":0.41099999999999998,"5min":0.36199999999999999,"15min":0.29399999999999998},"max":{"1min":0.80200000000000005,"5min":0.93100000000000005,"15min":0.96599999999999997},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.433,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77700000000000002,"5min":0.90900000000000003,"15min":1.018},"last":0.95399999999999996}]},{"osd":5,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.621,"15min":0.64100000000000001},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.92100000000000004,"5min":0.94499999999999995,"15min":1.0289999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.33400000000000002},"max":{"1min":0.86499999999999999,"5min":0.97899999999999998,"15min":0.996},"last":0.99299999999999999}]},{"osd":6,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.70599999999999996,"15min":0.70999999999999996},"min":{"1min":0.504,"5min":0.5,"15min":0.36799999999999999},"max":{"1min":0.97499999999999998,"5min":1.052,"15min":1.052},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67800000000000005,"15min":0.65600000000000003},"min":{"1min":0.48699999999999999,"5min":0.42199999999999999,"15min":0.41199999999999998},"max":{"1min":0.93500000000000005,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.83799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":958,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194353,"statfs":{"total":200000000,"available":199018308,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.70499999999999996,"15min":0.69599999999999995},"min":{"1min":0.46999999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.872,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.623},{"interface":"front","average":{"1min":0.752,"5min":0.71499999999999997,"15min":0.69799999999999995},"min":{"1min":0.64600000000000002,"5min":0.443,"15min":0.39500000000000002},"max":{"1min":0.88700000000000001,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.64400000000000002}]},{"osd":1,"last update":"Wed May 22 21:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.47799999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94299999999999995,"5min":0.95899999999999996,"15min":1.107},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70299999999999996,"15min":0.70799999999999996},"min":{"1min":0.51700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.84699999999999998,"5min":1.0449999999999999,"15min":1.081},"last":0.70299999999999996}]},{"osd":2,"last update":"Wed May 22 21:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77400000000000002,"15min":0.76700000000000002},"min":{"1min":0.61099999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.012,"5min":1.087,"15min":1.087},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.74299999999999999},"min":{"1min":0.54100000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.99099999999999999,"5min":1.419,"15min":1.419},"last":0.68700000000000006}]},{"osd":3,"last update":"Wed May 22 21:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.75900000000000001,"15min":0.749},"min":{"1min":0.57499999999999996,"5min":0.49399999999999999,"15min":0.48199999999999998},"max":{"1min":0.92500000000000004,"5min":1.28,"15min":1.415},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.53400000000000003,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":0.90900000000000003,"5min":1.004,"15min":1.788},"last":0.66600000000000004}]},{"osd":4,"last update":"Wed May 22 21:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41899999999999998,"5min":0.41699999999999998,"15min":0.374},"max":{"1min":0.878,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66600000000000004,"15min":0.67800000000000005},"min":{"1min":0.439,"5min":0.378,"15min":0.378},"max":{"1min":0.93000000000000005,"5min":1.1850000000000001,"15min":2.718},"last":0.48799999999999999}]},{"osd":5,"last update":"Wed May 22 21:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.622,"15min":0.68700000000000006},"min":{"1min":0.41399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82799999999999996,"5min":0.96999999999999997,"15min":8.5749999999999993},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68300000000000005,"15min":0.70599999999999996},"min":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":0.98499999999999999,"15min":1.1140000000000001},"last":0.76600000000000001}]},{"osd":7,"last update":"Wed May 22 21:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72299999999999998,"15min":0.73099999999999998},"min":{"1min":0.51300000000000001,"5min":0.42899999999999999,"15min":0.42399999999999999},"max":{"1min":0.95599999999999996,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69599999999999995,"15min":0.73699999999999999},"min":{"1min":0.53300000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89700000000000002,"5min":0.98099999999999998,"15min":1.8009999999999999},"last":0.71799999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":958,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194353,"statfs":{"total":200000000,"available":199018432,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.67200000000000004,"15min":0.69399999999999995},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0409999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.48699999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.93300000000000005,"5min":0.96499999999999997,"15min":1.0089999999999999},"last":0.75600000000000001}]},{"osd":1,"last update":"Wed May 22 21:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.51100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.0489999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.70299999999999996,"15min":0.69299999999999995},"min":{"1min":0.56000000000000005,"5min":0.44900000000000001,"15min":0.44},"max":{"1min":0.92600000000000005,"5min":1.321,"15min":1.321},"last":0.77000000000000002}]},{"osd":2,"last update":"Wed May 22 21:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70599999999999996,"15min":0.70699999999999996},"min":{"1min":0.48999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.98699999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72099999999999997,"15min":0.69499999999999995},"min":{"1min":0.63800000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92500000000000004,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.745}]},{"osd":3,"last update":"Wed May 22 21:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.59199999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90300000000000002,"5min":1.004,"15min":1.1879999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73099999999999998,"15min":0.73499999999999999},"min":{"1min":0.61599999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.96699999999999997,"5min":1.0469999999999999,"15min":1.111},"last":0.80100000000000005}]},{"osd":4,"last update":"Wed May 22 21:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58399999999999996,"15min":0.61099999999999999},"min":{"1min":0.439,"5min":0.38400000000000001,"15min":0.35099999999999998},"max":{"1min":0.86599999999999999,"5min":0.92400000000000004,"15min":1.272},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.625,"15min":0.63100000000000001},"min":{"1min":0.46400000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85099999999999998,"5min":1.014,"15min":1.038},"last":0.58599999999999997}]},{"osd":6,"last update":"Wed May 22 21:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.626,"15min":0.63100000000000001},"min":{"1min":0.442,"5min":0.371,"15min":0.371},"max":{"1min":0.82799999999999996,"5min":0.95199999999999996,"15min":0.98899999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.442,"15min":0.41099999999999998},"max":{"1min":0.94899999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.72899999999999998}]},{"osd":7,"last update":"Wed May 22 21:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70799999999999996,"15min":0.71399999999999997},"min":{"1min":0.56699999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96599999999999997,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.74299999999999999,"15min":0.77600000000000002},"min":{"1min":0.53000000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.2909999999999999},"last":0.78100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":958,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194353,"statfs":{"total":200000000,"available":199018454,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73499999999999999,"15min":0.749},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.30399999999999999},"max":{"1min":0.94899999999999995,"5min":0.98899999999999999,"15min":6.6539999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.56100000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.96299999999999997,"5min":0.98899999999999999,"15min":5.4370000000000003},"last":0.63100000000000001}]},{"osd":1,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.67600000000000005,"15min":0.70099999999999996},"min":{"1min":0.52500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88500000000000001,"5min":0.95799999999999996,"15min":1.7749999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.53600000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.878,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.73899999999999999,"15min":0.70499999999999996},"min":{"1min":0.45600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91700000000000004,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72799999999999998,"15min":0.71199999999999997},"min":{"1min":0.625,"5min":0.378,"15min":0.378},"max":{"1min":0.90200000000000002,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.621}]},{"osd":3,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72499999999999998,"15min":0.69899999999999995},"min":{"1min":0.51000000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89500000000000002,"5min":1.857,"15min":1.857},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78100000000000003,"15min":0.78500000000000003},"min":{"1min":0.58799999999999997,"5min":0.53200000000000003,"15min":0.47499999999999998},"max":{"1min":0.95599999999999996,"5min":1.897,"15min":1.897},"last":0.81599999999999995}]},{"osd":5,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.39200000000000002,"15min":0.379},"max":{"1min":0.80900000000000005,"5min":1.0169999999999999,"15min":1.1850000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.35599999999999998},"max":{"1min":0.83499999999999996,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.79700000000000004}]},{"osd":6,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64700000000000002,"15min":0.67700000000000005},"min":{"1min":0.45300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78400000000000003,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.68400000000000005,"15min":0.68999999999999995},"min":{"1min":0.502,"5min":0.44700000000000001,"15min":0.434},"max":{"1min":0.76000000000000001,"5min":1.095,"15min":1.1419999999999999},"last":0.70699999999999996}]},{"osd":7,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67000000000000004,"15min":0.68899999999999995},"min":{"1min":0.46400000000000002,"5min":0.42699999999999999,"15min":0.40799999999999997},"max":{"1min":0.80900000000000005,"5min":1.2150000000000001,"15min":1.7390000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71899999999999997,"15min":0.75700000000000001},"min":{"1min":0.48599999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.80400000000000005,"5min":1.1200000000000001,"15min":1.4510000000000001},"last":0.90400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574969,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":958,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194353,"statfs":{"total":200000000,"available":199018364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.436,"15min":0.41799999999999998},"min":{"1min":0.30299999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.68500000000000005,"5min":0.73099999999999998,"15min":0.86799999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.45800000000000002,"15min":0.42699999999999999},"min":{"1min":0.33000000000000002,"5min":0.30399999999999999,"15min":0.28399999999999997},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.34499999999999997}]},{"osd":1,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47199999999999998,"15min":0.45700000000000002},"min":{"1min":0.33600000000000002,"5min":0.28399999999999997,"15min":0.26500000000000001},"max":{"1min":0.60099999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55000000000000004,"15min":0.52300000000000002},"min":{"1min":0.40999999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.90000000000000002},"last":0.48799999999999999}]},{"osd":2,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.41099999999999998,"5min":0.35999999999999999,"15min":0.30599999999999999},"max":{"1min":0.69799999999999995,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.52400000000000002,"15min":0.503},"min":{"1min":0.38,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.93400000000000005},"last":0.44900000000000001}]},{"osd":4,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72499999999999998,"15min":0.71399999999999997},"min":{"1min":0.56599999999999995,"5min":0.47699999999999998,"15min":0.39400000000000002},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.2050000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.57799999999999996,"5min":0.49199999999999999,"15min":0.435},"max":{"1min":1.093,"5min":1.093,"15min":1.5760000000000001},"last":0.83799999999999997}]},{"osd":5,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.754,"15min":0.71999999999999997},"min":{"1min":0.55000000000000004,"5min":0.46800000000000003,"15min":0.45100000000000001},"max":{"1min":1,"5min":1.048,"15min":1.234},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73699999999999999,"15min":0.71799999999999997},"min":{"1min":0.60899999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.024,"5min":1.0449999999999999,"15min":1.3049999999999999},"last":0.80000000000000004}]},{"osd":6,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.73799999999999999,"15min":0.72499999999999998},"min":{"1min":0.60199999999999998,"5min":0.48499999999999999,"15min":0.44900000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.276},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.75900000000000001,"15min":0.72699999999999998},"min":{"1min":0.59399999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.05,"5min":1.0820000000000001,"15min":1.2869999999999999},"last":0.97299999999999998}]},{"osd":7,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.78200000000000003,"15min":0.77000000000000002},"min":{"1min":0.59399999999999997,"5min":0.501,"15min":0.495},"max":{"1min":1.0880000000000001,"5min":1.0900000000000001,"15min":1.2709999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.78200000000000003,"15min":0.75600000000000001},"min":{"1min":0.623,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.2969999999999999},"last":0.72899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574969,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":960,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194352,"statfs":{"total":200000000,"available":199016820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.46000000000000002,"15min":0.41999999999999998},"min":{"1min":0.32900000000000001,"5min":0.253,"15min":0.23599999999999999},"max":{"1min":0.64800000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.49099999999999999,"15min":0.45600000000000002},"min":{"1min":0.35999999999999999,"5min":0.26800000000000002,"15min":0.23400000000000001},"max":{"1min":0.61099999999999999,"5min":2.532,"15min":2.532},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48399999999999999,"15min":0.45800000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.29399999999999998},"max":{"1min":0.68899999999999995,"5min":2.948,"15min":2.948},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58299999999999996,"15min":0.54000000000000004},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.39100000000000001},"max":{"1min":0.71399999999999997,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.42899999999999999}]},{"osd":3,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55000000000000004,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.36899999999999999,"15min":0.32700000000000001},"max":{"1min":0.83499999999999996,"5min":2.75,"15min":2.75},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.48399999999999999},"min":{"1min":0.41999999999999998,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":0.66600000000000004,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.79900000000000004,"15min":0.72799999999999998},"min":{"1min":0.64400000000000002,"5min":0.53100000000000003,"15min":0.52700000000000002},"max":{"1min":1.103,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.83699999999999997,"15min":0.77700000000000002},"min":{"1min":0.71299999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.208,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.81499999999999995}]},{"osd":5,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.79600000000000004,"15min":0.73399999999999999},"min":{"1min":0.628,"5min":0.498,"15min":0.47299999999999998},"max":{"1min":1.133,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.72299999999999998},"min":{"1min":0.58099999999999996,"5min":0.47899999999999998,"15min":0.45900000000000002},"max":{"1min":1.0549999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.78400000000000003,"15min":0.73199999999999998},"min":{"1min":0.56999999999999995,"5min":0.51300000000000001,"15min":0.495},"max":{"1min":1.048,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.76100000000000001,"15min":0.70699999999999996},"min":{"1min":0.59899999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.147,"5min":2.6880000000000002,"15min":2.6880000000000002},"last":0.68100000000000005}]},{"osd":7,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.82399999999999995,"15min":0.76100000000000001},"min":{"1min":0.629,"5min":0.438,"15min":0.438},"max":{"1min":1.2210000000000001,"5min":3.0099999999999998,"15min":3.0099999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.84599999999999997,"15min":0.79700000000000004},"min":{"1min":0.63600000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.1339999999999999,"5min":2.827,"15min":2.827},"last":0.77500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":959,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194353,"statfs":{"total":200000000,"available":199017872,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.439,"15min":0.433},"min":{"1min":0.28599999999999998,"5min":0.216,"15min":0.216},"max":{"1min":0.73499999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.254},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.47399999999999998,"15min":0.44900000000000001},"min":{"1min":0.35199999999999998,"5min":0.29299999999999998,"15min":0.26000000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed May 22 21:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.50900000000000001,"15min":0.496},"min":{"1min":0.36199999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.73099999999999998,"5min":1.76,"15min":1.76},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.48399999999999999,"15min":0.51400000000000001},"min":{"1min":0.32200000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":1.923},"last":0.44600000000000001}]},{"osd":3,"last update":"Wed May 22 21:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54200000000000004,"15min":0.55700000000000005},"min":{"1min":0.46000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":2.1030000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.52500000000000002,"15min":0.5},"min":{"1min":0.433,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73299999999999998,"5min":0.745,"15min":0.745},"last":0.379}]},{"osd":4,"last update":"Wed May 22 21:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.45100000000000001},"max":{"1min":0.95899999999999996,"5min":1.109,"15min":2.1360000000000001},"last":0.628},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73999999999999999,"15min":0.70599999999999996},"min":{"1min":0.53600000000000003,"5min":0.46500000000000002,"15min":0.44900000000000001},"max":{"1min":0.91800000000000004,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.64000000000000001}]},{"osd":5,"last update":"Wed May 22 21:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.77200000000000002,"15min":0.749},"min":{"1min":0.58599999999999997,"5min":0.56000000000000005,"15min":0.45300000000000001},"max":{"1min":0.92000000000000004,"5min":1.819,"15min":1.819},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.745,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.46300000000000002,"15min":0.40100000000000002},"max":{"1min":1.034,"5min":1.1060000000000001,"15min":1.97},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed May 22 21:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.77100000000000002,"15min":0.75600000000000001},"min":{"1min":0.56100000000000005,"5min":0.48999999999999999,"15min":0.47299999999999998},"max":{"1min":0.94499999999999995,"5min":1.1359999999999999,"15min":1.272},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80900000000000005,"15min":0.77000000000000002},"min":{"1min":0.63900000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98999999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.68100000000000005}]},{"osd":7,"last update":"Wed May 22 21:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77200000000000002,"15min":0.77300000000000002},"min":{"1min":0.65400000000000003,"5min":0.48599999999999999,"15min":0.42099999999999999},"max":{"1min":0.98799999999999999,"5min":1.1379999999999999,"15min":2.0310000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.76200000000000001,"15min":0.77000000000000002},"min":{"1min":0.626,"5min":0.54900000000000004,"15min":0.52000000000000002},"max":{"1min":0.96099999999999997,"5min":1.0660000000000001,"15min":2.0659999999999998},"last":0.66300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574969,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1536,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193776,"statfs":{"total":200000000,"available":198427107,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.441,"15min":0.41999999999999998},"min":{"1min":0.32900000000000001,"5min":0.24199999999999999,"15min":0.22800000000000001},"max":{"1min":0.67600000000000005,"5min":0.85899999999999999,"15min":0.88300000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.52300000000000002,"15min":0.5},"min":{"1min":0.46100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed May 22 21:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.5,"15min":0.47199999999999998},"min":{"1min":0.40699999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.85899999999999999,"5min":0.88400000000000001,"15min":0.93799999999999994},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.46899999999999997,"15min":0.441},"min":{"1min":0.33700000000000002,"5min":0.28399999999999997,"15min":0.27900000000000003},"max":{"1min":0.64600000000000002,"5min":0.70799999999999996,"15min":0.92300000000000004},"last":0.47999999999999998}]},{"osd":3,"last update":"Wed May 22 21:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46700000000000003,"15min":0.44600000000000001},"min":{"1min":0.38,"5min":0.313,"15min":0.28499999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":1.0760000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56000000000000005,"15min":0.52200000000000002},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89700000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed May 22 21:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.73799999999999999,"15min":0.72099999999999997},"min":{"1min":0.61799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.76900000000000002,"15min":0.74299999999999999},"min":{"1min":0.65300000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2709999999999999},"last":0.871}]},{"osd":5,"last update":"Wed May 22 21:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76000000000000001,"15min":0.72299999999999998},"min":{"1min":0.56899999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.76100000000000001,"15min":0.747},"min":{"1min":0.58399999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.84799999999999998}]},{"osd":6,"last update":"Wed May 22 21:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.80500000000000005,"15min":0.78500000000000003},"min":{"1min":0.68200000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.2090000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.79000000000000004,"15min":0.77700000000000002},"min":{"1min":0.66600000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.5},"last":0.88600000000000001}]},{"osd":7,"last update":"Wed May 22 21:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.77100000000000002,"15min":0.74199999999999999},"min":{"1min":0.60499999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.76600000000000001,"15min":0.76200000000000001},"min":{"1min":0.64500000000000002,"5min":0.52800000000000002,"15min":0.497},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.89600000000000002}]}]}],"pool_statfs":[]}} 2024-05-22T21:23:07.936 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:23:08.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87bee00640 1 Processor -- start 2024-05-22T21:23:08.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87bee00640 1 -- start start 2024-05-22T21:23:08.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87bee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87b815e130 0x7f87b815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87bee00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f87b815d410 0x7f87b815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87bee00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b8071670 con 0x7f87b815dd50 2024-05-22T21:23:08.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87bee00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b815d870 con 0x7f87b815d410 2024-05-22T21:23:08.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87bee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b8070a60 con 0x7f87b815e130 2024-05-22T21:23:08.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87b7e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f87b815dd50 0x7f87b806d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:40722/0 (socket says 172.21.15.161:40722) 2024-05-22T21:23:08.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87b7e00640 1 -- 172.21.15.161:0/793373424 learned_addr learned my addr 172.21.15.161:0/793373424 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:23:08.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87bce00640 1 --2- 172.21.15.161:0/793373424 >> v2:172.21.15.179:3300/0 conn(0x7f87b815d410 0x7f87b815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87b7400640 1 --2- 172.21.15.161:0/793373424 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87b815e130 0x7f87b815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87b6a00640 1 -- 172.21.15.161:0/793373424 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133649939 0 0) 0x7f87b8071670 con 0x7f87b815dd50 2024-05-22T21:23:08.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87b6a00640 1 -- 172.21.15.161:0/793373424 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f879c003530 con 0x7f87b815dd50 2024-05-22T21:23:08.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87bce00640 1 -- 172.21.15.161:0/793373424 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87b815e130 msgr2=0x7f87b815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:08.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87bce00640 1 --2- 172.21.15.161:0/793373424 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87b815e130 0x7f87b815ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87bce00640 1 -- 172.21.15.161:0/793373424 >> v1:172.21.15.161:6789/0 conn(0x7f87b815dd50 legacy=0x7f87b806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:08.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87b6a00640 1 -- 172.21.15.161:0/793373424 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1487843708 0 0) 0x7f879c003530 con 0x7f87b815dd50 2024-05-22T21:23:08.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.112+0000 7f87bce00640 1 -- 172.21.15.161:0/793373424 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b8070d40 con 0x7f87b815d410 2024-05-22T21:23:08.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.116+0000 7f87bce00640 1 --2- 172.21.15.161:0/793373424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b815d410 0x7f87b815b6c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009d90 tx=0x7f87b001c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=534722ecf970bb87 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:08.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.116+0000 7f87b6a00640 1 -- 172.21.15.161:0/793373424 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0001680 con 0x7f87b815d410 2024-05-22T21:23:08.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.116+0000 7f87b6a00640 1 -- 172.21.15.161:0/793373424 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b00a55d0 con 0x7f87b815d410 2024-05-22T21:23:08.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.116+0000 7f87b6a00640 1 -- 172.21.15.161:0/793373424 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b00a5940 con 0x7f87b815d410 2024-05-22T21:23:08.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.116+0000 7f87bee00640 1 -- 172.21.15.161:0/793373424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b815d410 msgr2=0x7f87b815b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:08.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.116+0000 7f87bee00640 1 --2- 172.21.15.161:0/793373424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b815d410 0x7f87b815b6c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009d90 tx=0x7f87b001c7d0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.116+0000 7f87bee00640 1 -- 172.21.15.161:0/793373424 shutdown_connections 2024-05-22T21:23:08.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.116+0000 7f87bee00640 1 --2- 172.21.15.161:0/793373424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b815d410 0x7f87b815b6c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.116+0000 7f87bee00640 1 --2- 172.21.15.161:0/793373424 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87b815e130 0x7f87b815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.116+0000 7f87bee00640 1 -- 172.21.15.161:0/793373424 >> 172.21.15.161:0/793373424 conn(0x7f87b80b0bc0 msgr2=0x7f87b80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:08.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.116+0000 7f87bee00640 1 -- 172.21.15.161:0/793373424 shutdown_connections 2024-05-22T21:23:08.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87bee00640 1 -- 172.21.15.161:0/793373424 wait complete. 2024-05-22T21:23:08.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87bee00640 1 Processor -- start 2024-05-22T21:23:08.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87bee00640 1 -- start start 2024-05-22T21:23:08.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87bee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87b815d410 0x7f87b814e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87bee00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b815dd50 0x7f87b814f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87bee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b815e130 0x7f87b80cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87bee00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b814e8d0 con 0x7f87b815dd50 2024-05-22T21:23:08.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87bee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80cd3b0 con 0x7f87b815e130 2024-05-22T21:23:08.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87bee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b8168ab0 con 0x7f87b815d410 2024-05-22T21:23:08.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87b7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b815dd50 0x7f87b814f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87b7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b815dd50 0x7f87b814f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42366/0 (socket says 172.21.15.161:42366) 2024-05-22T21:23:08.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87b7400640 1 -- 172.21.15.161:0/2327919950 learned_addr learned my addr 172.21.15.161:0/2327919950 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:08.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87b7e00640 1 --2- 172.21.15.161:0/2327919950 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87b815d410 0x7f87b814e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.122 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87bce00640 1 --2- 172.21.15.161:0/2327919950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b815e130 0x7f87b80cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87b7400640 1 -- 172.21.15.161:0/2327919950 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87b815d410 msgr2=0x7f87b814e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:08.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87b7400640 1 --2- 172.21.15.161:0/2327919950 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87b815d410 0x7f87b814e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87b7400640 1 -- 172.21.15.161:0/2327919950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b815e130 msgr2=0x7f87b80cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:08.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87b7400640 1 --2- 172.21.15.161:0/2327919950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b815e130 0x7f87b80cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87b7400640 1 -- 172.21.15.161:0/2327919950 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b0009a70 con 0x7f87b815dd50 2024-05-22T21:23:08.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87bce00640 1 --2- 172.21.15.161:0/2327919950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b815e130 0x7f87b80cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:23:08.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87b7400640 1 --2- 172.21.15.161:0/2327919950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b815dd50 0x7f87b814f720 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f87a800e870 tx=0x7f87a800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:08.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.120+0000 7f87afe00640 1 -- 172.21.15.161:0/2327919950 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8009510 con 0x7f87b815dd50 2024-05-22T21:23:08.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.124+0000 7f87afe00640 1 -- 172.21.15.161:0/2327919950 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a800eec0 con 0x7f87b815dd50 2024-05-22T21:23:08.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.124+0000 7f87bee00640 1 -- 172.21.15.161:0/2327919950 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b8168d90 con 0x7f87b815dd50 2024-05-22T21:23:08.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.124+0000 7f87afe00640 1 -- 172.21.15.161:0/2327919950 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a800cb40 con 0x7f87b815dd50 2024-05-22T21:23:08.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.124+0000 7f87bee00640 1 -- 172.21.15.161:0/2327919950 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b814e020 con 0x7f87b815dd50 2024-05-22T21:23:08.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.124+0000 7f87bee00640 1 -- 172.21.15.161:0/2327919950 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b8070710 con 0x7f87b815dd50 2024-05-22T21:23:08.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.124+0000 7f87afe00640 1 -- 172.21.15.161:0/2327919950 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f87a800cca0 con 0x7f87b815dd50 2024-05-22T21:23:08.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.124+0000 7f87afe00640 1 --2- 172.21.15.161:0/2327919950 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8794066690 0x7f8794068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.124+0000 7f87afe00640 1 -- 172.21.15.161:0/2327919950 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87a80842a0 con 0x7f87b815dd50 2024-05-22T21:23:08.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.124+0000 7f87b7e00640 1 --2- 172.21.15.161:0/2327919950 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8794066690 0x7f8794068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.124+0000 7f87b7e00640 1 --2- 172.21.15.161:0/2327919950 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8794066690 0x7f8794068b50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f87a0001190 tx=0x7f87a0001120 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:08.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.128+0000 7f87afe00640 1 -- 172.21.15.161:0/2327919950 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f87b8070710 con 0x7f87b815dd50 2024-05-22T21:23:08.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.276+0000 7f87bee00640 1 -- 172.21.15.161:0/2327919950 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87b8168c20 con 0x7f8794066690 2024-05-22T21:23:08.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.284+0000 7f87afe00640 1 -- 172.21.15.161:0/2327919950 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284948 (secure 0 0 0) 0x7f87b8168c20 con 0x7f8794066690 2024-05-22T21:23:08.289 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:23:08.290 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:23:08.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.288+0000 7f87bee00640 1 -- 172.21.15.161:0/2327919950 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8794066690 msgr2=0x7f8794068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:08.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.288+0000 7f87bee00640 1 --2- 172.21.15.161:0/2327919950 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8794066690 0x7f8794068b50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f87a0001190 tx=0x7f87a0001120 comp rx=0 tx=0).stop 2024-05-22T21:23:08.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.288+0000 7f87bee00640 1 -- 172.21.15.161:0/2327919950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b815dd50 msgr2=0x7f87b814f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:08.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.288+0000 7f87bee00640 1 --2- 172.21.15.161:0/2327919950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b815dd50 0x7f87b814f720 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f87a800e870 tx=0x7f87a800ec70 comp rx=0 tx=0).stop 2024-05-22T21:23:08.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.288+0000 7f87bee00640 1 -- 172.21.15.161:0/2327919950 shutdown_connections 2024-05-22T21:23:08.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.288+0000 7f87bee00640 1 --2- 172.21.15.161:0/2327919950 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8794066690 0x7f8794068b50 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.288+0000 7f87bee00640 1 --2- 172.21.15.161:0/2327919950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87b815e130 0x7f87b80cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.288+0000 7f87bee00640 1 --2- 172.21.15.161:0/2327919950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b815dd50 0x7f87b814f720 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.288+0000 7f87bee00640 1 --2- 172.21.15.161:0/2327919950 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87b815d410 0x7f87b814e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.288+0000 7f87bee00640 1 -- 172.21.15.161:0/2327919950 >> 172.21.15.161:0/2327919950 conn(0x7f87b80b0bc0 msgr2=0x7f87b815a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:08.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.288+0000 7f87bee00640 1 -- 172.21.15.161:0/2327919950 shutdown_connections 2024-05-22T21:23:08.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.288+0000 7f87bee00640 1 -- 172.21.15.161:0/2327919950 wait complete. 2024-05-22T21:23:08.309 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-05-22T21:23:06.528109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8825,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553674,"statfs":{"total":1600000000,"available":1590962791,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.046253"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918607+0000","last_change":"2024-05-22T21:23:03.879894+0000","last_active":"2024-05-22T21:23:04.918607+0000","last_peered":"2024-05-22T21:23:04.918607+0000","last_clean":"2024-05-22T21:23:04.918607+0000","last_became_active":"2024-05-22T21:23:03.879301+0000","last_became_peered":"2024-05-22T21:23:03.879301+0000","last_unstale":"2024-05-22T21:23:04.918607+0000","last_undegraded":"2024-05-22T21:23:04.918607+0000","last_fullsized":"2024-05-22T21:23:04.918607+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:09:55.160766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918799+0000","last_change":"2024-05-22T21:23:03.877341+0000","last_active":"2024-05-22T21:23:04.918799+0000","last_peered":"2024-05-22T21:23:04.918799+0000","last_clean":"2024-05-22T21:23:04.918799+0000","last_became_active":"2024-05-22T21:23:03.877124+0000","last_became_peered":"2024-05-22T21:23:03.877124+0000","last_unstale":"2024-05-22T21:23:04.918799+0000","last_undegraded":"2024-05-22T21:23:04.918799+0000","last_fullsized":"2024-05-22T21:23:04.918799+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:40:40.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920611+0000","last_change":"2024-05-22T21:23:03.884129+0000","last_active":"2024-05-22T21:23:04.920611+0000","last_peered":"2024-05-22T21:23:04.920611+0000","last_clean":"2024-05-22T21:23:04.920611+0000","last_became_active":"2024-05-22T21:23:03.883911+0000","last_became_peered":"2024-05-22T21:23:03.883911+0000","last_unstale":"2024-05-22T21:23:04.920611+0000","last_undegraded":"2024-05-22T21:23:04.920611+0000","last_fullsized":"2024-05-22T21:23:04.920611+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:24:12.146293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883537+0000","last_change":"2024-05-22T21:23:03.878683+0000","last_active":"2024-05-22T21:23:04.883537+0000","last_peered":"2024-05-22T21:23:04.883537+0000","last_clean":"2024-05-22T21:23:04.883537+0000","last_became_active":"2024-05-22T21:23:03.878406+0000","last_became_peered":"2024-05-22T21:23:03.878406+0000","last_unstale":"2024-05-22T21:23:04.883537+0000","last_undegraded":"2024-05-22T21:23:04.883537+0000","last_fullsized":"2024-05-22T21:23:04.883537+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:20.274667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920360+0000","last_change":"2024-05-22T21:23:03.886007+0000","last_active":"2024-05-22T21:23:04.920360+0000","last_peered":"2024-05-22T21:23:04.920360+0000","last_clean":"2024-05-22T21:23:04.920360+0000","last_became_active":"2024-05-22T21:23:03.884536+0000","last_became_peered":"2024-05-22T21:23:03.884536+0000","last_unstale":"2024-05-22T21:23:04.920360+0000","last_undegraded":"2024-05-22T21:23:04.920360+0000","last_fullsized":"2024-05-22T21:23:04.920360+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:34:14.648009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918760+0000","last_change":"2024-05-22T21:23:03.884920+0000","last_active":"2024-05-22T21:23:04.918760+0000","last_peered":"2024-05-22T21:23:04.918760+0000","last_clean":"2024-05-22T21:23:04.918760+0000","last_became_active":"2024-05-22T21:23:03.884527+0000","last_became_peered":"2024-05-22T21:23:03.884527+0000","last_unstale":"2024-05-22T21:23:04.918760+0000","last_undegraded":"2024-05-22T21:23:04.918760+0000","last_fullsized":"2024-05-22T21:23:04.918760+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:04:35.477076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.877090+0000","last_change":"2024-05-22T21:23:03.877885+0000","last_active":"2024-05-22T21:23:04.877090+0000","last_peered":"2024-05-22T21:23:04.877090+0000","last_clean":"2024-05-22T21:23:04.877090+0000","last_became_active":"2024-05-22T21:23:03.877499+0000","last_became_peered":"2024-05-22T21:23:03.877499+0000","last_unstale":"2024-05-22T21:23:04.877090+0000","last_undegraded":"2024-05-22T21:23:04.877090+0000","last_fullsized":"2024-05-22T21:23:04.877090+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:15:26.229883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918587+0000","last_change":"2024-05-22T21:23:03.883719+0000","last_active":"2024-05-22T21:23:04.918587+0000","last_peered":"2024-05-22T21:23:04.918587+0000","last_clean":"2024-05-22T21:23:04.918587+0000","last_became_active":"2024-05-22T21:23:03.883511+0000","last_became_peered":"2024-05-22T21:23:03.883511+0000","last_unstale":"2024-05-22T21:23:04.918587+0000","last_undegraded":"2024-05-22T21:23:04.918587+0000","last_fullsized":"2024-05-22T21:23:04.918587+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:09:45.135262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919513+0000","last_change":"2024-05-22T21:23:03.872664+0000","last_active":"2024-05-22T21:23:04.919513+0000","last_peered":"2024-05-22T21:23:04.919513+0000","last_clean":"2024-05-22T21:23:04.919513+0000","last_became_active":"2024-05-22T21:23:03.872441+0000","last_became_peered":"2024-05-22T21:23:03.872441+0000","last_unstale":"2024-05-22T21:23:04.919513+0000","last_undegraded":"2024-05-22T21:23:04.919513+0000","last_fullsized":"2024-05-22T21:23:04.919513+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:31:11.627686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884380+0000","last_change":"2024-05-22T21:23:03.887317+0000","last_active":"2024-05-22T21:23:04.884380+0000","last_peered":"2024-05-22T21:23:04.884380+0000","last_clean":"2024-05-22T21:23:04.884380+0000","last_became_active":"2024-05-22T21:23:03.886830+0000","last_became_peered":"2024-05-22T21:23:03.886830+0000","last_unstale":"2024-05-22T21:23:04.884380+0000","last_undegraded":"2024-05-22T21:23:04.884380+0000","last_fullsized":"2024-05-22T21:23:04.884380+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:52:34.048705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918842+0000","last_change":"2024-05-22T21:23:03.883070+0000","last_active":"2024-05-22T21:23:04.918842+0000","last_peered":"2024-05-22T21:23:04.918842+0000","last_clean":"2024-05-22T21:23:04.918842+0000","last_became_active":"2024-05-22T21:23:03.882874+0000","last_became_peered":"2024-05-22T21:23:03.882874+0000","last_unstale":"2024-05-22T21:23:04.918842+0000","last_undegraded":"2024-05-22T21:23:04.918842+0000","last_fullsized":"2024-05-22T21:23:04.918842+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:30:35.247289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883204+0000","last_change":"2024-05-22T21:23:03.883008+0000","last_active":"2024-05-22T21:23:04.883204+0000","last_peered":"2024-05-22T21:23:04.883204+0000","last_clean":"2024-05-22T21:23:04.883204+0000","last_became_active":"2024-05-22T21:23:03.882782+0000","last_became_peered":"2024-05-22T21:23:03.882782+0000","last_unstale":"2024-05-22T21:23:04.883204+0000","last_undegraded":"2024-05-22T21:23:04.883204+0000","last_fullsized":"2024-05-22T21:23:04.883204+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:22:02.297408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920058+0000","last_change":"2024-05-22T21:23:03.887705+0000","last_active":"2024-05-22T21:23:04.920058+0000","last_peered":"2024-05-22T21:23:04.920058+0000","last_clean":"2024-05-22T21:23:04.920058+0000","last_became_active":"2024-05-22T21:23:03.887204+0000","last_became_peered":"2024-05-22T21:23:03.887204+0000","last_unstale":"2024-05-22T21:23:04.920058+0000","last_undegraded":"2024-05-22T21:23:04.920058+0000","last_fullsized":"2024-05-22T21:23:04.920058+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:06:28.393977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883634+0000","last_change":"2024-05-22T21:23:03.884347+0000","last_active":"2024-05-22T21:23:04.883634+0000","last_peered":"2024-05-22T21:23:04.883634+0000","last_clean":"2024-05-22T21:23:04.883634+0000","last_became_active":"2024-05-22T21:23:03.883895+0000","last_became_peered":"2024-05-22T21:23:03.883895+0000","last_unstale":"2024-05-22T21:23:04.883634+0000","last_undegraded":"2024-05-22T21:23:04.883634+0000","last_fullsized":"2024-05-22T21:23:04.883634+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:26:35.850390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884426+0000","last_change":"2024-05-22T21:23:03.883626+0000","last_active":"2024-05-22T21:23:04.884426+0000","last_peered":"2024-05-22T21:23:04.884426+0000","last_clean":"2024-05-22T21:23:04.884426+0000","last_became_active":"2024-05-22T21:23:03.883104+0000","last_became_peered":"2024-05-22T21:23:03.883104+0000","last_unstale":"2024-05-22T21:23:04.884426+0000","last_undegraded":"2024-05-22T21:23:04.884426+0000","last_fullsized":"2024-05-22T21:23:04.884426+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:05:23.728167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919421+0000","last_change":"2024-05-22T21:23:03.876394+0000","last_active":"2024-05-22T21:23:04.919421+0000","last_peered":"2024-05-22T21:23:04.919421+0000","last_clean":"2024-05-22T21:23:04.919421+0000","last_became_active":"2024-05-22T21:23:03.876120+0000","last_became_peered":"2024-05-22T21:23:03.876120+0000","last_unstale":"2024-05-22T21:23:04.919421+0000","last_undegraded":"2024-05-22T21:23:04.919421+0000","last_fullsized":"2024-05-22T21:23:04.919421+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:35:16.124266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883484+0000","last_change":"2024-05-22T21:23:03.876919+0000","last_active":"2024-05-22T21:23:04.883484+0000","last_peered":"2024-05-22T21:23:04.883484+0000","last_clean":"2024-05-22T21:23:04.883484+0000","last_became_active":"2024-05-22T21:23:03.876692+0000","last_became_peered":"2024-05-22T21:23:03.876692+0000","last_unstale":"2024-05-22T21:23:04.883484+0000","last_undegraded":"2024-05-22T21:23:04.883484+0000","last_fullsized":"2024-05-22T21:23:04.883484+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:26:00.113338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920337+0000","last_change":"2024-05-22T21:23:03.890379+0000","last_active":"2024-05-22T21:23:04.920337+0000","last_peered":"2024-05-22T21:23:04.920337+0000","last_clean":"2024-05-22T21:23:04.920337+0000","last_became_active":"2024-05-22T21:23:03.888170+0000","last_became_peered":"2024-05-22T21:23:03.888170+0000","last_unstale":"2024-05-22T21:23:04.920337+0000","last_undegraded":"2024-05-22T21:23:04.920337+0000","last_fullsized":"2024-05-22T21:23:04.920337+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:27:30.545155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918960+0000","last_change":"2024-05-22T21:23:03.878118+0000","last_active":"2024-05-22T21:23:04.918960+0000","last_peered":"2024-05-22T21:23:04.918960+0000","last_clean":"2024-05-22T21:23:04.918960+0000","last_became_active":"2024-05-22T21:23:03.877648+0000","last_became_peered":"2024-05-22T21:23:03.877648+0000","last_unstale":"2024-05-22T21:23:04.918960+0000","last_undegraded":"2024-05-22T21:23:04.918960+0000","last_fullsized":"2024-05-22T21:23:04.918960+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:06:28.393977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919488+0000","last_change":"2024-05-22T21:23:03.877612+0000","last_active":"2024-05-22T21:23:04.919488+0000","last_peered":"2024-05-22T21:23:04.919488+0000","last_clean":"2024-05-22T21:23:04.919488+0000","last_became_active":"2024-05-22T21:23:03.877360+0000","last_became_peered":"2024-05-22T21:23:03.877360+0000","last_unstale":"2024-05-22T21:23:04.919488+0000","last_undegraded":"2024-05-22T21:23:04.919488+0000","last_fullsized":"2024-05-22T21:23:04.919488+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:34:14.648009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.882971+0000","last_change":"2024-05-22T21:23:03.881364+0000","last_active":"2024-05-22T21:23:04.882971+0000","last_peered":"2024-05-22T21:23:04.882971+0000","last_clean":"2024-05-22T21:23:04.882971+0000","last_became_active":"2024-05-22T21:23:03.880568+0000","last_became_peered":"2024-05-22T21:23:03.880568+0000","last_unstale":"2024-05-22T21:23:04.882971+0000","last_undegraded":"2024-05-22T21:23:04.882971+0000","last_fullsized":"2024-05-22T21:23:04.882971+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:15:48.218923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883727+0000","last_change":"2024-05-22T21:23:03.879480+0000","last_active":"2024-05-22T21:23:04.883727+0000","last_peered":"2024-05-22T21:23:04.883727+0000","last_clean":"2024-05-22T21:23:04.883727+0000","last_became_active":"2024-05-22T21:23:03.878956+0000","last_became_peered":"2024-05-22T21:23:03.878956+0000","last_unstale":"2024-05-22T21:23:04.883727+0000","last_undegraded":"2024-05-22T21:23:04.883727+0000","last_fullsized":"2024-05-22T21:23:04.883727+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:24:12.146293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883371+0000","last_change":"2024-05-22T21:23:03.883548+0000","last_active":"2024-05-22T21:23:04.883371+0000","last_peered":"2024-05-22T21:23:04.883371+0000","last_clean":"2024-05-22T21:23:04.883371+0000","last_became_active":"2024-05-22T21:23:03.883193+0000","last_became_peered":"2024-05-22T21:23:03.883193+0000","last_unstale":"2024-05-22T21:23:04.883371+0000","last_undegraded":"2024-05-22T21:23:04.883371+0000","last_fullsized":"2024-05-22T21:23:04.883371+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:13:00.978006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883046+0000","last_change":"2024-05-22T21:23:03.881375+0000","last_active":"2024-05-22T21:23:04.883046+0000","last_peered":"2024-05-22T21:23:04.883046+0000","last_clean":"2024-05-22T21:23:04.883046+0000","last_became_active":"2024-05-22T21:23:03.880587+0000","last_became_peered":"2024-05-22T21:23:03.880587+0000","last_unstale":"2024-05-22T21:23:04.883046+0000","last_undegraded":"2024-05-22T21:23:04.883046+0000","last_fullsized":"2024-05-22T21:23:04.883046+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:10:59.460151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919535+0000","last_change":"2024-05-22T21:23:03.875646+0000","last_active":"2024-05-22T21:23:04.919535+0000","last_peered":"2024-05-22T21:23:04.919535+0000","last_clean":"2024-05-22T21:23:04.919535+0000","last_became_active":"2024-05-22T21:23:03.875441+0000","last_became_peered":"2024-05-22T21:23:03.875441+0000","last_unstale":"2024-05-22T21:23:04.919535+0000","last_undegraded":"2024-05-22T21:23:04.919535+0000","last_fullsized":"2024-05-22T21:23:04.919535+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:51:34.278141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918927+0000","last_change":"2024-05-22T21:23:03.879494+0000","last_active":"2024-05-22T21:23:04.918927+0000","last_peered":"2024-05-22T21:23:04.918927+0000","last_clean":"2024-05-22T21:23:04.918927+0000","last_became_active":"2024-05-22T21:23:03.878931+0000","last_became_peered":"2024-05-22T21:23:03.878931+0000","last_unstale":"2024-05-22T21:23:04.918927+0000","last_undegraded":"2024-05-22T21:23:04.918927+0000","last_fullsized":"2024-05-22T21:23:04.918927+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:06:25.045909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919110+0000","last_change":"2024-05-22T21:23:03.882340+0000","last_active":"2024-05-22T21:23:04.919110+0000","last_peered":"2024-05-22T21:23:04.919110+0000","last_clean":"2024-05-22T21:23:04.919110+0000","last_became_active":"2024-05-22T21:23:03.882014+0000","last_became_peered":"2024-05-22T21:23:03.882014+0000","last_unstale":"2024-05-22T21:23:04.919110+0000","last_undegraded":"2024-05-22T21:23:04.919110+0000","last_fullsized":"2024-05-22T21:23:04.919110+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:26:21.878604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.877107+0000","last_change":"2024-05-22T21:23:03.885708+0000","last_active":"2024-05-22T21:23:04.877107+0000","last_peered":"2024-05-22T21:23:04.877107+0000","last_clean":"2024-05-22T21:23:04.877107+0000","last_became_active":"2024-05-22T21:23:03.885470+0000","last_became_peered":"2024-05-22T21:23:03.885470+0000","last_unstale":"2024-05-22T21:23:04.877107+0000","last_undegraded":"2024-05-22T21:23:04.877107+0000","last_fullsized":"2024-05-22T21:23:04.877107+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:30:27.417750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918726+0000","last_change":"2024-05-22T21:23:03.877852+0000","last_active":"2024-05-22T21:23:04.918726+0000","last_peered":"2024-05-22T21:23:04.918726+0000","last_clean":"2024-05-22T21:23:04.918726+0000","last_became_active":"2024-05-22T21:23:03.877330+0000","last_became_peered":"2024-05-22T21:23:03.877330+0000","last_unstale":"2024-05-22T21:23:04.918726+0000","last_undegraded":"2024-05-22T21:23:04.918726+0000","last_fullsized":"2024-05-22T21:23:04.918726+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:09:36.781044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884538+0000","last_change":"2024-05-22T21:23:03.882795+0000","last_active":"2024-05-22T21:23:04.884538+0000","last_peered":"2024-05-22T21:23:04.884538+0000","last_clean":"2024-05-22T21:23:04.884538+0000","last_became_active":"2024-05-22T21:23:03.882516+0000","last_became_peered":"2024-05-22T21:23:03.882516+0000","last_unstale":"2024-05-22T21:23:04.884538+0000","last_undegraded":"2024-05-22T21:23:04.884538+0000","last_fullsized":"2024-05-22T21:23:04.884538+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:04:00.574818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919334+0000","last_change":"2024-05-22T21:23:03.884061+0000","last_active":"2024-05-22T21:23:04.919334+0000","last_peered":"2024-05-22T21:23:04.919334+0000","last_clean":"2024-05-22T21:23:04.919334+0000","last_became_active":"2024-05-22T21:23:03.883845+0000","last_became_peered":"2024-05-22T21:23:03.883845+0000","last_unstale":"2024-05-22T21:23:04.919334+0000","last_undegraded":"2024-05-22T21:23:04.919334+0000","last_fullsized":"2024-05-22T21:23:04.919334+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:02:58.646363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919206+0000","last_change":"2024-05-22T21:23:03.882214+0000","last_active":"2024-05-22T21:23:04.919206+0000","last_peered":"2024-05-22T21:23:04.919206+0000","last_clean":"2024-05-22T21:23:04.919206+0000","last_became_active":"2024-05-22T21:23:03.881589+0000","last_became_peered":"2024-05-22T21:23:03.881589+0000","last_unstale":"2024-05-22T21:23:04.919206+0000","last_undegraded":"2024-05-22T21:23:04.919206+0000","last_fullsized":"2024-05-22T21:23:04.919206+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:26:27.533773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883147+0000","last_change":"2024-05-22T21:23:03.877625+0000","last_active":"2024-05-22T21:23:04.883147+0000","last_peered":"2024-05-22T21:23:04.883147+0000","last_clean":"2024-05-22T21:23:04.883147+0000","last_became_active":"2024-05-22T21:23:03.877338+0000","last_became_peered":"2024-05-22T21:23:03.877338+0000","last_unstale":"2024-05-22T21:23:04.883147+0000","last_undegraded":"2024-05-22T21:23:04.883147+0000","last_fullsized":"2024-05-22T21:23:04.883147+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:55:03.432809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"174'4","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883246+0000","last_change":"2024-05-22T21:23:03.502676+0000","last_active":"2024-05-22T21:23:04.883246+0000","last_peered":"2024-05-22T21:23:04.883246+0000","last_clean":"2024-05-22T21:23:04.883246+0000","last_became_active":"2024-05-22T21:23:03.502407+0000","last_became_peered":"2024-05-22T21:23:03.502407+0000","last_unstale":"2024-05-22T21:23:04.883246+0000","last_undegraded":"2024-05-22T21:23:04.883246+0000","last_fullsized":"2024-05-22T21:23:04.883246+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:12:38.116914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918987+0000","last_change":"2024-05-22T21:23:03.876393+0000","last_active":"2024-05-22T21:23:04.918987+0000","last_peered":"2024-05-22T21:23:04.918987+0000","last_clean":"2024-05-22T21:23:04.918987+0000","last_became_active":"2024-05-22T21:23:03.876168+0000","last_became_peered":"2024-05-22T21:23:03.876168+0000","last_unstale":"2024-05-22T21:23:04.918987+0000","last_undegraded":"2024-05-22T21:23:04.918987+0000","last_fullsized":"2024-05-22T21:23:04.918987+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:12:39.124394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919602+0000","last_change":"2024-05-22T21:23:03.499748+0000","last_active":"2024-05-22T21:23:04.919602+0000","last_peered":"2024-05-22T21:23:04.919602+0000","last_clean":"2024-05-22T21:23:04.919602+0000","last_became_active":"2024-05-22T21:23:03.499260+0000","last_became_peered":"2024-05-22T21:23:03.499260+0000","last_unstale":"2024-05-22T21:23:04.919602+0000","last_undegraded":"2024-05-22T21:23:04.919602+0000","last_fullsized":"2024-05-22T21:23:04.919602+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:26:34.842909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919015+0000","last_change":"2024-05-22T21:23:03.880013+0000","last_active":"2024-05-22T21:23:04.919015+0000","last_peered":"2024-05-22T21:23:04.919015+0000","last_clean":"2024-05-22T21:23:04.919015+0000","last_became_active":"2024-05-22T21:23:03.879692+0000","last_became_peered":"2024-05-22T21:23:03.879692+0000","last_unstale":"2024-05-22T21:23:04.919015+0000","last_undegraded":"2024-05-22T21:23:04.919015+0000","last_fullsized":"2024-05-22T21:23:04.919015+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:44:02.437988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884678+0000","last_change":"2024-05-22T21:23:03.504699+0000","last_active":"2024-05-22T21:23:04.884678+0000","last_peered":"2024-05-22T21:23:04.884678+0000","last_clean":"2024-05-22T21:23:04.884678+0000","last_became_active":"2024-05-22T21:23:03.504457+0000","last_became_peered":"2024-05-22T21:23:03.504457+0000","last_unstale":"2024-05-22T21:23:04.884678+0000","last_undegraded":"2024-05-22T21:23:04.884678+0000","last_fullsized":"2024-05-22T21:23:04.884678+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:44:01.430507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884472+0000","last_change":"2024-05-22T21:23:03.883811+0000","last_active":"2024-05-22T21:23:04.884472+0000","last_peered":"2024-05-22T21:23:04.884472+0000","last_clean":"2024-05-22T21:23:04.884472+0000","last_became_active":"2024-05-22T21:23:03.883412+0000","last_became_peered":"2024-05-22T21:23:03.883412+0000","last_unstale":"2024-05-22T21:23:04.884472+0000","last_undegraded":"2024-05-22T21:23:04.884472+0000","last_fullsized":"2024-05-22T21:23:04.884472+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:55:23.600882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884013+0000","last_change":"2024-05-22T21:23:03.504387+0000","last_active":"2024-05-22T21:23:04.884013+0000","last_peered":"2024-05-22T21:23:04.884013+0000","last_clean":"2024-05-22T21:23:04.884013+0000","last_became_active":"2024-05-22T21:23:03.503974+0000","last_became_peered":"2024-05-22T21:23:03.503974+0000","last_unstale":"2024-05-22T21:23:04.884013+0000","last_undegraded":"2024-05-22T21:23:04.884013+0000","last_fullsized":"2024-05-22T21:23:04.884013+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:09:35.773564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919063+0000","last_change":"2024-05-22T21:23:03.884097+0000","last_active":"2024-05-22T21:23:04.919063+0000","last_peered":"2024-05-22T21:23:04.919063+0000","last_clean":"2024-05-22T21:23:04.919063+0000","last_became_active":"2024-05-22T21:23:03.883741+0000","last_became_peered":"2024-05-22T21:23:03.883741+0000","last_unstale":"2024-05-22T21:23:04.919063+0000","last_undegraded":"2024-05-22T21:23:04.919063+0000","last_fullsized":"2024-05-22T21:23:04.919063+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:23:57.604650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919383+0000","last_change":"2024-05-22T21:23:03.500065+0000","last_active":"2024-05-22T21:23:04.919383+0000","last_peered":"2024-05-22T21:23:04.919383+0000","last_clean":"2024-05-22T21:23:04.919383+0000","last_became_active":"2024-05-22T21:23:03.498678+0000","last_became_peered":"2024-05-22T21:23:03.498678+0000","last_unstale":"2024-05-22T21:23:04.919383+0000","last_undegraded":"2024-05-22T21:23:04.919383+0000","last_fullsized":"2024-05-22T21:23:04.919383+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:26:51.655443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883817+0000","last_change":"2024-05-22T21:23:03.878032+0000","last_active":"2024-05-22T21:23:04.883817+0000","last_peered":"2024-05-22T21:23:04.883817+0000","last_clean":"2024-05-22T21:23:04.883817+0000","last_became_active":"2024-05-22T21:23:03.877762+0000","last_became_peered":"2024-05-22T21:23:03.877762+0000","last_unstale":"2024-05-22T21:23:04.883817+0000","last_undegraded":"2024-05-22T21:23:04.883817+0000","last_fullsized":"2024-05-22T21:23:04.883817+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:50:02.421057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920533+0000","last_change":"2024-05-22T21:23:03.505146+0000","last_active":"2024-05-22T21:23:04.920533+0000","last_peered":"2024-05-22T21:23:04.920533+0000","last_clean":"2024-05-22T21:23:04.920533+0000","last_became_active":"2024-05-22T21:23:03.504683+0000","last_became_peered":"2024-05-22T21:23:03.504683+0000","last_unstale":"2024-05-22T21:23:04.920533+0000","last_undegraded":"2024-05-22T21:23:04.920533+0000","last_fullsized":"2024-05-22T21:23:04.920533+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:00:40.034709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919490+0000","last_change":"2024-05-22T21:23:03.881027+0000","last_active":"2024-05-22T21:23:04.919490+0000","last_peered":"2024-05-22T21:23:04.919490+0000","last_clean":"2024-05-22T21:23:04.919490+0000","last_became_active":"2024-05-22T21:23:03.880824+0000","last_became_peered":"2024-05-22T21:23:03.880824+0000","last_unstale":"2024-05-22T21:23:04.919490+0000","last_undegraded":"2024-05-22T21:23:04.919490+0000","last_fullsized":"2024-05-22T21:23:04.919490+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:06:44.193264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919323+0000","last_change":"2024-05-22T21:23:03.501791+0000","last_active":"2024-05-22T21:23:04.919323+0000","last_peered":"2024-05-22T21:23:04.919323+0000","last_clean":"2024-05-22T21:23:04.919323+0000","last_became_active":"2024-05-22T21:23:03.501582+0000","last_became_peered":"2024-05-22T21:23:03.501582+0000","last_unstale":"2024-05-22T21:23:04.919323+0000","last_undegraded":"2024-05-22T21:23:04.919323+0000","last_fullsized":"2024-05-22T21:23:04.919323+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:19:21.792180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920096+0000","last_change":"2024-05-22T21:23:03.881307+0000","last_active":"2024-05-22T21:23:04.920096+0000","last_peered":"2024-05-22T21:23:04.920096+0000","last_clean":"2024-05-22T21:23:04.920096+0000","last_became_active":"2024-05-22T21:23:03.881047+0000","last_became_peered":"2024-05-22T21:23:03.881047+0000","last_unstale":"2024-05-22T21:23:04.920096+0000","last_undegraded":"2024-05-22T21:23:04.920096+0000","last_fullsized":"2024-05-22T21:23:04.920096+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:26:27.533773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884759+0000","last_change":"2024-05-22T21:23:03.503260+0000","last_active":"2024-05-22T21:23:04.884759+0000","last_peered":"2024-05-22T21:23:04.884759+0000","last_clean":"2024-05-22T21:23:04.884759+0000","last_became_active":"2024-05-22T21:23:03.502992+0000","last_became_peered":"2024-05-22T21:23:03.502992+0000","last_unstale":"2024-05-22T21:23:04.884759+0000","last_undegraded":"2024-05-22T21:23:04.884759+0000","last_fullsized":"2024-05-22T21:23:04.884759+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:13:33.310983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920314+0000","last_change":"2024-05-22T21:23:03.885333+0000","last_active":"2024-05-22T21:23:04.920314+0000","last_peered":"2024-05-22T21:23:04.920314+0000","last_clean":"2024-05-22T21:23:04.920314+0000","last_became_active":"2024-05-22T21:23:03.884648+0000","last_became_peered":"2024-05-22T21:23:03.884648+0000","last_unstale":"2024-05-22T21:23:04.920314+0000","last_undegraded":"2024-05-22T21:23:04.920314+0000","last_fullsized":"2024-05-22T21:23:04.920314+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:26:21.878604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919414+0000","last_change":"2024-05-22T21:23:03.500679+0000","last_active":"2024-05-22T21:23:04.919414+0000","last_peered":"2024-05-22T21:23:04.919414+0000","last_clean":"2024-05-22T21:23:04.919414+0000","last_became_active":"2024-05-22T21:23:03.500429+0000","last_became_peered":"2024-05-22T21:23:03.500429+0000","last_unstale":"2024-05-22T21:23:04.919414+0000","last_undegraded":"2024-05-22T21:23:04.919414+0000","last_fullsized":"2024-05-22T21:23:04.919414+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:55.213155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883095+0000","last_change":"2024-05-22T21:23:03.876887+0000","last_active":"2024-05-22T21:23:04.883095+0000","last_peered":"2024-05-22T21:23:04.883095+0000","last_clean":"2024-05-22T21:23:04.883095+0000","last_became_active":"2024-05-22T21:23:03.876626+0000","last_became_peered":"2024-05-22T21:23:03.876626+0000","last_unstale":"2024-05-22T21:23:04.883095+0000","last_undegraded":"2024-05-22T21:23:04.883095+0000","last_fullsized":"2024-05-22T21:23:04.883095+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:52:59.054693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883391+0000","last_change":"2024-05-22T21:23:03.507647+0000","last_active":"2024-05-22T21:23:04.883391+0000","last_peered":"2024-05-22T21:23:04.883391+0000","last_clean":"2024-05-22T21:23:04.883391+0000","last_became_active":"2024-05-22T21:23:03.506896+0000","last_became_peered":"2024-05-22T21:23:03.506896+0000","last_unstale":"2024-05-22T21:23:04.883391+0000","last_undegraded":"2024-05-22T21:23:04.883391+0000","last_fullsized":"2024-05-22T21:23:04.883391+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:23:56.597169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919229+0000","last_change":"2024-05-22T21:23:03.876890+0000","last_active":"2024-05-22T21:23:04.919229+0000","last_peered":"2024-05-22T21:23:04.919229+0000","last_clean":"2024-05-22T21:23:04.919229+0000","last_became_active":"2024-05-22T21:23:03.876650+0000","last_became_peered":"2024-05-22T21:23:03.876650+0000","last_unstale":"2024-05-22T21:23:04.919229+0000","last_undegraded":"2024-05-22T21:23:04.919229+0000","last_fullsized":"2024-05-22T21:23:04.919229+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:20:48.302649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"174'6","reported_seq":17,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.876850+0000","last_change":"2024-05-22T21:23:03.495709+0000","last_active":"2024-05-22T21:23:04.876850+0000","last_peered":"2024-05-22T21:23:04.876850+0000","last_clean":"2024-05-22T21:23:04.876850+0000","last_became_active":"2024-05-22T21:23:03.495136+0000","last_became_peered":"2024-05-22T21:23:03.495136+0000","last_unstale":"2024-05-22T21:23:04.876850+0000","last_undegraded":"2024-05-22T21:23:04.876850+0000","last_fullsized":"2024-05-22T21:23:04.876850+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:31:10.620206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920788+0000","last_change":"2024-05-22T21:23:03.887898+0000","last_active":"2024-05-22T21:23:04.920788+0000","last_peered":"2024-05-22T21:23:04.920788+0000","last_clean":"2024-05-22T21:23:04.920788+0000","last_became_active":"2024-05-22T21:23:03.887243+0000","last_became_peered":"2024-05-22T21:23:03.887243+0000","last_unstale":"2024-05-22T21:23:04.920788+0000","last_undegraded":"2024-05-22T21:23:04.920788+0000","last_fullsized":"2024-05-22T21:23:04.920788+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:56.220635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920722+0000","last_change":"2024-05-22T21:23:03.505356+0000","last_active":"2024-05-22T21:23:04.920722+0000","last_peered":"2024-05-22T21:23:04.920722+0000","last_clean":"2024-05-22T21:23:04.920722+0000","last_became_active":"2024-05-22T21:23:03.505102+0000","last_became_peered":"2024-05-22T21:23:03.505102+0000","last_unstale":"2024-05-22T21:23:04.920722+0000","last_undegraded":"2024-05-22T21:23:04.920722+0000","last_fullsized":"2024-05-22T21:23:04.920722+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:54:32.797433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919274+0000","last_change":"2024-05-22T21:23:03.877777+0000","last_active":"2024-05-22T21:23:04.919274+0000","last_peered":"2024-05-22T21:23:04.919274+0000","last_clean":"2024-05-22T21:23:04.919274+0000","last_became_active":"2024-05-22T21:23:03.877570+0000","last_became_peered":"2024-05-22T21:23:03.877570+0000","last_unstale":"2024-05-22T21:23:04.919274+0000","last_undegraded":"2024-05-22T21:23:04.919274+0000","last_fullsized":"2024-05-22T21:23:04.919274+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:59:12.425717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920854+0000","last_change":"2024-05-22T21:23:03.503186+0000","last_active":"2024-05-22T21:23:04.920854+0000","last_peered":"2024-05-22T21:23:04.920854+0000","last_clean":"2024-05-22T21:23:04.920854+0000","last_became_active":"2024-05-22T21:23:03.502941+0000","last_became_peered":"2024-05-22T21:23:03.502941+0000","last_unstale":"2024-05-22T21:23:04.920854+0000","last_undegraded":"2024-05-22T21:23:04.920854+0000","last_fullsized":"2024-05-22T21:23:04.920854+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:40:20.750446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920169+0000","last_change":"2024-05-22T21:23:03.887561+0000","last_active":"2024-05-22T21:23:04.920169+0000","last_peered":"2024-05-22T21:23:04.920169+0000","last_clean":"2024-05-22T21:23:04.920169+0000","last_became_active":"2024-05-22T21:23:03.886909+0000","last_became_peered":"2024-05-22T21:23:03.886909+0000","last_unstale":"2024-05-22T21:23:04.920169+0000","last_undegraded":"2024-05-22T21:23:04.920169+0000","last_fullsized":"2024-05-22T21:23:04.920169+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:02:40.386927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918515+0000","last_change":"2024-05-22T21:23:03.496061+0000","last_active":"2024-05-22T21:23:04.918515+0000","last_peered":"2024-05-22T21:23:04.918515+0000","last_clean":"2024-05-22T21:23:04.918515+0000","last_became_active":"2024-05-22T21:23:03.495850+0000","last_became_peered":"2024-05-22T21:23:03.495850+0000","last_unstale":"2024-05-22T21:23:04.918515+0000","last_undegraded":"2024-05-22T21:23:04.918515+0000","last_fullsized":"2024-05-22T21:23:04.918515+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:32:32.161836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920716+0000","last_change":"2024-05-22T21:23:03.888426+0000","last_active":"2024-05-22T21:23:04.920716+0000","last_peered":"2024-05-22T21:23:04.920716+0000","last_clean":"2024-05-22T21:23:04.920716+0000","last_became_active":"2024-05-22T21:23:03.883306+0000","last_became_peered":"2024-05-22T21:23:03.883306+0000","last_unstale":"2024-05-22T21:23:04.920716+0000","last_undegraded":"2024-05-22T21:23:04.920716+0000","last_fullsized":"2024-05-22T21:23:04.920716+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:15:20.274667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"174'8","reported_seq":19,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919313+0000","last_change":"2024-05-22T21:23:03.505247+0000","last_active":"2024-05-22T21:23:04.919313+0000","last_peered":"2024-05-22T21:23:04.919313+0000","last_clean":"2024-05-22T21:23:04.919313+0000","last_became_active":"2024-05-22T21:23:03.505022+0000","last_became_peered":"2024-05-22T21:23:03.505022+0000","last_unstale":"2024-05-22T21:23:04.919313+0000","last_undegraded":"2024-05-22T21:23:04.919313+0000","last_fullsized":"2024-05-22T21:23:04.919313+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:44:00.938486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884519+0000","last_change":"2024-05-22T21:23:03.884246+0000","last_active":"2024-05-22T21:23:04.884519+0000","last_peered":"2024-05-22T21:23:04.884519+0000","last_clean":"2024-05-22T21:23:04.884519+0000","last_became_active":"2024-05-22T21:23:03.884036+0000","last_became_peered":"2024-05-22T21:23:03.884036+0000","last_unstale":"2024-05-22T21:23:04.884519+0000","last_undegraded":"2024-05-22T21:23:04.884519+0000","last_fullsized":"2024-05-22T21:23:04.884519+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:00:17.253740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920272+0000","last_change":"2024-05-22T21:23:03.505737+0000","last_active":"2024-05-22T21:23:04.920272+0000","last_peered":"2024-05-22T21:23:04.920272+0000","last_clean":"2024-05-22T21:23:04.920272+0000","last_became_active":"2024-05-22T21:23:03.505355+0000","last_became_peered":"2024-05-22T21:23:03.505355+0000","last_unstale":"2024-05-22T21:23:04.920272+0000","last_undegraded":"2024-05-22T21:23:04.920272+0000","last_fullsized":"2024-05-22T21:23:04.920272+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:31:12.385932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919165+0000","last_change":"2024-05-22T21:23:03.881207+0000","last_active":"2024-05-22T21:23:04.919165+0000","last_peered":"2024-05-22T21:23:04.919165+0000","last_clean":"2024-05-22T21:23:04.919165+0000","last_became_active":"2024-05-22T21:23:03.880905+0000","last_became_peered":"2024-05-22T21:23:03.880905+0000","last_unstale":"2024-05-22T21:23:04.919165+0000","last_undegraded":"2024-05-22T21:23:04.919165+0000","last_fullsized":"2024-05-22T21:23:04.919165+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:30:59.125519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919573+0000","last_change":"2024-05-22T21:23:03.503749+0000","last_active":"2024-05-22T21:23:04.919573+0000","last_peered":"2024-05-22T21:23:04.919573+0000","last_clean":"2024-05-22T21:23:04.919573+0000","last_became_active":"2024-05-22T21:23:03.503530+0000","last_became_peered":"2024-05-22T21:23:03.503530+0000","last_unstale":"2024-05-22T21:23:04.919573+0000","last_undegraded":"2024-05-22T21:23:04.919573+0000","last_fullsized":"2024-05-22T21:23:04.919573+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:12:59.970525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920790+0000","last_change":"2024-05-22T21:23:03.886291+0000","last_active":"2024-05-22T21:23:04.920790+0000","last_peered":"2024-05-22T21:23:04.920790+0000","last_clean":"2024-05-22T21:23:04.920790+0000","last_became_active":"2024-05-22T21:23:03.882943+0000","last_became_peered":"2024-05-22T21:23:03.882943+0000","last_unstale":"2024-05-22T21:23:04.920790+0000","last_undegraded":"2024-05-22T21:23:04.920790+0000","last_fullsized":"2024-05-22T21:23:04.920790+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:50:02.421057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919574+0000","last_change":"2024-05-22T21:23:03.497137+0000","last_active":"2024-05-22T21:23:04.919574+0000","last_peered":"2024-05-22T21:23:04.919574+0000","last_clean":"2024-05-22T21:23:04.919574+0000","last_became_active":"2024-05-22T21:23:03.496631+0000","last_became_peered":"2024-05-22T21:23:03.496631+0000","last_unstale":"2024-05-22T21:23:04.919574+0000","last_undegraded":"2024-05-22T21:23:04.919574+0000","last_fullsized":"2024-05-22T21:23:04.919574+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:22:52.446757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919156+0000","last_change":"2024-05-22T21:23:03.884773+0000","last_active":"2024-05-22T21:23:04.919156+0000","last_peered":"2024-05-22T21:23:04.919156+0000","last_clean":"2024-05-22T21:23:04.919156+0000","last_became_active":"2024-05-22T21:23:03.884239+0000","last_became_peered":"2024-05-22T21:23:03.884239+0000","last_unstale":"2024-05-22T21:23:04.919156+0000","last_undegraded":"2024-05-22T21:23:04.919156+0000","last_fullsized":"2024-05-22T21:23:04.919156+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:37.096853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918950+0000","last_change":"2024-05-22T21:23:03.501930+0000","last_active":"2024-05-22T21:23:04.918950+0000","last_peered":"2024-05-22T21:23:04.918950+0000","last_clean":"2024-05-22T21:23:04.918950+0000","last_became_active":"2024-05-22T21:23:03.501581+0000","last_became_peered":"2024-05-22T21:23:03.501581+0000","last_unstale":"2024-05-22T21:23:04.918950+0000","last_undegraded":"2024-05-22T21:23:04.918950+0000","last_fullsized":"2024-05-22T21:23:04.918950+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:24:11.138812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883734+0000","last_change":"2024-05-22T21:23:03.881211+0000","last_active":"2024-05-22T21:23:04.883734+0000","last_peered":"2024-05-22T21:23:04.883734+0000","last_clean":"2024-05-22T21:23:04.883734+0000","last_became_active":"2024-05-22T21:23:03.880995+0000","last_became_peered":"2024-05-22T21:23:03.880995+0000","last_unstale":"2024-05-22T21:23:04.883734+0000","last_undegraded":"2024-05-22T21:23:04.883734+0000","last_fullsized":"2024-05-22T21:23:04.883734+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:11:56.220635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.877199+0000","last_change":"2024-05-22T21:23:03.505393+0000","last_active":"2024-05-22T21:23:04.877199+0000","last_peered":"2024-05-22T21:23:04.877199+0000","last_clean":"2024-05-22T21:23:04.877199+0000","last_became_active":"2024-05-22T21:23:03.505116+0000","last_became_peered":"2024-05-22T21:23:03.505116+0000","last_unstale":"2024-05-22T21:23:04.877199+0000","last_undegraded":"2024-05-22T21:23:04.877199+0000","last_fullsized":"2024-05-22T21:23:04.877199+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:59:11.418236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.877156+0000","last_change":"2024-05-22T21:23:03.879909+0000","last_active":"2024-05-22T21:23:04.877156+0000","last_peered":"2024-05-22T21:23:04.877156+0000","last_clean":"2024-05-22T21:23:04.877156+0000","last_became_active":"2024-05-22T21:23:03.879636+0000","last_became_peered":"2024-05-22T21:23:03.879636+0000","last_unstale":"2024-05-22T21:23:04.877156+0000","last_undegraded":"2024-05-22T21:23:04.877156+0000","last_fullsized":"2024-05-22T21:23:04.877156+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:37:41.376144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883350+0000","last_change":"2024-05-22T21:23:03.499340+0000","last_active":"2024-05-22T21:23:04.883350+0000","last_peered":"2024-05-22T21:23:04.883350+0000","last_clean":"2024-05-22T21:23:04.883350+0000","last_became_active":"2024-05-22T21:23:03.498048+0000","last_became_peered":"2024-05-22T21:23:03.498048+0000","last_unstale":"2024-05-22T21:23:04.883350+0000","last_undegraded":"2024-05-22T21:23:04.883350+0000","last_fullsized":"2024-05-22T21:23:04.883350+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:21:25.641549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883902+0000","last_change":"2024-05-22T21:23:03.880182+0000","last_active":"2024-05-22T21:23:04.883902+0000","last_peered":"2024-05-22T21:23:04.883902+0000","last_clean":"2024-05-22T21:23:04.883902+0000","last_became_active":"2024-05-22T21:23:03.879938+0000","last_became_peered":"2024-05-22T21:23:03.879938+0000","last_unstale":"2024-05-22T21:23:04.883902+0000","last_undegraded":"2024-05-22T21:23:04.883902+0000","last_fullsized":"2024-05-22T21:23:04.883902+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:59:47.725165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.884562+0000","last_change":"2024-05-22T21:23:03.498487+0000","last_active":"2024-05-22T21:23:04.884562+0000","last_peered":"2024-05-22T21:23:04.884562+0000","last_clean":"2024-05-22T21:23:04.884562+0000","last_became_active":"2024-05-22T21:23:03.498233+0000","last_became_peered":"2024-05-22T21:23:03.498233+0000","last_unstale":"2024-05-22T21:23:04.884562+0000","last_undegraded":"2024-05-22T21:23:04.884562+0000","last_fullsized":"2024-05-22T21:23:04.884562+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:02:57.638883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.877246+0000","last_change":"2024-05-22T21:23:03.878599+0000","last_active":"2024-05-22T21:23:04.877246+0000","last_peered":"2024-05-22T21:23:04.877246+0000","last_clean":"2024-05-22T21:23:04.877246+0000","last_became_active":"2024-05-22T21:23:03.875474+0000","last_became_peered":"2024-05-22T21:23:03.875474+0000","last_unstale":"2024-05-22T21:23:04.877246+0000","last_undegraded":"2024-05-22T21:23:04.877246+0000","last_fullsized":"2024-05-22T21:23:04.877246+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:45:43.497247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920904+0000","last_change":"2024-05-22T21:23:03.502189+0000","last_active":"2024-05-22T21:23:04.920904+0000","last_peered":"2024-05-22T21:23:04.920904+0000","last_clean":"2024-05-22T21:23:04.920904+0000","last_became_active":"2024-05-22T21:23:03.501893+0000","last_became_peered":"2024-05-22T21:23:03.501893+0000","last_unstale":"2024-05-22T21:23:04.920904+0000","last_undegraded":"2024-05-22T21:23:04.920904+0000","last_fullsized":"2024-05-22T21:23:04.920904+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:32:10.359880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920358+0000","last_change":"2024-05-22T21:23:03.887421+0000","last_active":"2024-05-22T21:23:04.920358+0000","last_peered":"2024-05-22T21:23:04.920358+0000","last_clean":"2024-05-22T21:23:04.920358+0000","last_became_active":"2024-05-22T21:23:03.886618+0000","last_became_peered":"2024-05-22T21:23:03.886618+0000","last_unstale":"2024-05-22T21:23:04.920358+0000","last_undegraded":"2024-05-22T21:23:04.920358+0000","last_fullsized":"2024-05-22T21:23:04.920358+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:53.702637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883567+0000","last_change":"2024-05-22T21:23:03.507143+0000","last_active":"2024-05-22T21:23:04.883567+0000","last_peered":"2024-05-22T21:23:04.883567+0000","last_clean":"2024-05-22T21:23:04.883567+0000","last_became_active":"2024-05-22T21:23:03.504672+0000","last_became_peered":"2024-05-22T21:23:03.504672+0000","last_unstale":"2024-05-22T21:23:04.883567+0000","last_undegraded":"2024-05-22T21:23:04.883567+0000","last_fullsized":"2024-05-22T21:23:04.883567+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:59:11.418236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920850+0000","last_change":"2024-05-22T21:23:03.883520+0000","last_active":"2024-05-22T21:23:04.920850+0000","last_peered":"2024-05-22T21:23:04.920850+0000","last_clean":"2024-05-22T21:23:04.920850+0000","last_became_active":"2024-05-22T21:23:03.882636+0000","last_became_peered":"2024-05-22T21:23:03.882636+0000","last_unstale":"2024-05-22T21:23:04.920850+0000","last_undegraded":"2024-05-22T21:23:04.920850+0000","last_fullsized":"2024-05-22T21:23:04.920850+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:26:35.850390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918671+0000","last_change":"2024-05-22T21:23:03.499566+0000","last_active":"2024-05-22T21:23:04.918671+0000","last_peered":"2024-05-22T21:23:04.918671+0000","last_clean":"2024-05-22T21:23:04.918671+0000","last_became_active":"2024-05-22T21:23:03.499164+0000","last_became_peered":"2024-05-22T21:23:03.499164+0000","last_unstale":"2024-05-22T21:23:04.918671+0000","last_undegraded":"2024-05-22T21:23:04.918671+0000","last_fullsized":"2024-05-22T21:23:04.918671+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:36:33.702861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919357+0000","last_change":"2024-05-22T21:23:03.885996+0000","last_active":"2024-05-22T21:23:04.919357+0000","last_peered":"2024-05-22T21:23:04.919357+0000","last_clean":"2024-05-22T21:23:04.919357+0000","last_became_active":"2024-05-22T21:23:03.885790+0000","last_became_peered":"2024-05-22T21:23:03.885790+0000","last_unstale":"2024-05-22T21:23:04.919357+0000","last_undegraded":"2024-05-22T21:23:04.919357+0000","last_fullsized":"2024-05-22T21:23:04.919357+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:41:15.882835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919041+0000","last_change":"2024-05-22T21:23:03.499933+0000","last_active":"2024-05-22T21:23:04.919041+0000","last_peered":"2024-05-22T21:23:04.919041+0000","last_clean":"2024-05-22T21:23:04.919041+0000","last_became_active":"2024-05-22T21:23:03.499274+0000","last_became_peered":"2024-05-22T21:23:03.499274+0000","last_unstale":"2024-05-22T21:23:04.919041+0000","last_undegraded":"2024-05-22T21:23:04.919041+0000","last_fullsized":"2024-05-22T21:23:04.919041+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:20:47.295168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918815+0000","last_change":"2024-05-22T21:23:03.879664+0000","last_active":"2024-05-22T21:23:04.918815+0000","last_peered":"2024-05-22T21:23:04.918815+0000","last_clean":"2024-05-22T21:23:04.918815+0000","last_became_active":"2024-05-22T21:23:03.879445+0000","last_became_peered":"2024-05-22T21:23:03.879445+0000","last_unstale":"2024-05-22T21:23:04.918815+0000","last_undegraded":"2024-05-22T21:23:04.918815+0000","last_fullsized":"2024-05-22T21:23:04.918815+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:13:34.318464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.918798+0000","last_change":"2024-05-22T21:23:03.498403+0000","last_active":"2024-05-22T21:23:04.918798+0000","last_peered":"2024-05-22T21:23:04.918798+0000","last_clean":"2024-05-22T21:23:04.918798+0000","last_became_active":"2024-05-22T21:23:03.497966+0000","last_became_peered":"2024-05-22T21:23:03.497966+0000","last_unstale":"2024-05-22T21:23:04.918798+0000","last_undegraded":"2024-05-22T21:23:04.918798+0000","last_fullsized":"2024-05-22T21:23:04.918798+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:51:33.270661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919296+0000","last_change":"2024-05-22T21:23:03.879434+0000","last_active":"2024-05-22T21:23:04.919296+0000","last_peered":"2024-05-22T21:23:04.919296+0000","last_clean":"2024-05-22T21:23:04.919296+0000","last_became_active":"2024-05-22T21:23:03.879230+0000","last_became_peered":"2024-05-22T21:23:03.879230+0000","last_unstale":"2024-05-22T21:23:04.919296+0000","last_undegraded":"2024-05-22T21:23:04.919296+0000","last_fullsized":"2024-05-22T21:23:04.919296+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:46:30.050409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.877107+0000","last_change":"2024-05-22T21:23:03.500735+0000","last_active":"2024-05-22T21:23:04.877107+0000","last_peered":"2024-05-22T21:23:04.877107+0000","last_clean":"2024-05-22T21:23:04.877107+0000","last_became_active":"2024-05-22T21:23:03.500489+0000","last_became_peered":"2024-05-22T21:23:03.500489+0000","last_unstale":"2024-05-22T21:23:04.877107+0000","last_undegraded":"2024-05-22T21:23:04.877107+0000","last_fullsized":"2024-05-22T21:23:04.877107+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:23:56.597169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920370+0000","last_change":"2024-05-22T21:23:03.891467+0000","last_active":"2024-05-22T21:23:04.920370+0000","last_peered":"2024-05-22T21:23:04.920370+0000","last_clean":"2024-05-22T21:23:04.920370+0000","last_became_active":"2024-05-22T21:23:03.891234+0000","last_became_peered":"2024-05-22T21:23:03.891234+0000","last_unstale":"2024-05-22T21:23:04.920370+0000","last_undegraded":"2024-05-22T21:23:04.920370+0000","last_fullsized":"2024-05-22T21:23:04.920370+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:33:36.236974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883645+0000","last_change":"2024-05-22T21:23:03.507443+0000","last_active":"2024-05-22T21:23:04.883645+0000","last_peered":"2024-05-22T21:23:04.883645+0000","last_clean":"2024-05-22T21:23:04.883645+0000","last_became_active":"2024-05-22T21:23:03.505194+0000","last_became_peered":"2024-05-22T21:23:03.505194+0000","last_unstale":"2024-05-22T21:23:04.883645+0000","last_undegraded":"2024-05-22T21:23:04.883645+0000","last_fullsized":"2024-05-22T21:23:04.883645+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:40:39.781464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.883252+0000","last_change":"2024-05-22T21:23:03.885111+0000","last_active":"2024-05-22T21:23:04.883252+0000","last_peered":"2024-05-22T21:23:04.883252+0000","last_clean":"2024-05-22T21:23:04.883252+0000","last_became_active":"2024-05-22T21:23:03.881157+0000","last_became_peered":"2024-05-22T21:23:03.881157+0000","last_unstale":"2024-05-22T21:23:04.883252+0000","last_undegraded":"2024-05-22T21:23:04.883252+0000","last_fullsized":"2024-05-22T21:23:04.883252+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:06:32.394718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920818+0000","last_change":"2024-05-22T21:23:03.507764+0000","last_active":"2024-05-22T21:23:04.920818+0000","last_peered":"2024-05-22T21:23:04.920818+0000","last_clean":"2024-05-22T21:23:04.920818+0000","last_became_active":"2024-05-22T21:23:03.506771+0000","last_became_peered":"2024-05-22T21:23:03.506771+0000","last_unstale":"2024-05-22T21:23:04.920818+0000","last_undegraded":"2024-05-22T21:23:04.920818+0000","last_fullsized":"2024-05-22T21:23:04.920818+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:44:45.599821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920438+0000","last_change":"2024-05-22T21:23:03.881367+0000","last_active":"2024-05-22T21:23:04.920438+0000","last_peered":"2024-05-22T21:23:04.920438+0000","last_clean":"2024-05-22T21:23:04.920438+0000","last_became_active":"2024-05-22T21:23:03.881079+0000","last_became_peered":"2024-05-22T21:23:03.881079+0000","last_unstale":"2024-05-22T21:23:04.920438+0000","last_undegraded":"2024-05-22T21:23:04.920438+0000","last_fullsized":"2024-05-22T21:23:04.920438+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:18:37.096853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.919678+0000","last_change":"2024-05-22T21:23:03.499886+0000","last_active":"2024-05-22T21:23:04.919678+0000","last_peered":"2024-05-22T21:23:04.919678+0000","last_clean":"2024-05-22T21:23:04.919678+0000","last_became_active":"2024-05-22T21:23:03.499545+0000","last_became_peered":"2024-05-22T21:23:03.499545+0000","last_unstale":"2024-05-22T21:23:04.919678+0000","last_undegraded":"2024-05-22T21:23:04.919678+0000","last_fullsized":"2024-05-22T21:23:04.919678+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:03:59.567337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.876945+0000","last_change":"2024-05-22T21:23:03.879540+0000","last_active":"2024-05-22T21:23:04.876945+0000","last_peered":"2024-05-22T21:23:04.876945+0000","last_clean":"2024-05-22T21:23:04.876945+0000","last_became_active":"2024-05-22T21:23:03.879329+0000","last_became_peered":"2024-05-22T21:23:03.879329+0000","last_unstale":"2024-05-22T21:23:04.876945+0000","last_undegraded":"2024-05-22T21:23:04.876945+0000","last_fullsized":"2024-05-22T21:23:04.876945+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:03.479722+0000","last_clean_scrub_stamp":"2024-05-22T21:23:03.479722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:22:27.742690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"174'3","reported_seq":14,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-05-22T21:23:04.920725+0000","last_change":"2024-05-22T21:23:03.507316+0000","last_active":"2024-05-22T21:23:04.920725+0000","last_peered":"2024-05-22T21:23:04.920725+0000","last_clean":"2024-05-22T21:23:04.920725+0000","last_became_active":"2024-05-22T21:23:03.506228+0000","last_became_peered":"2024-05-22T21:23:03.506228+0000","last_unstale":"2024-05-22T21:23:04.920725+0000","last_undegraded":"2024-05-22T21:23:04.920725+0000","last_fullsized":"2024-05-22T21:23:04.920725+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:23:02.472241+0000","last_clean_scrub_stamp":"2024-05-22T21:23:02.472241+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:48:30.664137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":322,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-05-22T21:23:03.872023+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:23:03.872023+0000","last_peered":"2024-05-22T21:23:03.872023+0000","last_clean":"2024-05-22T21:23:03.872023+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:23:03.872023+0000","last_undegraded":"2024-05-22T21:23:03.872023+0000","last_fullsized":"2024-05-22T21:23:03.872023+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574969,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1535,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193776,"statfs":{"total":200000000,"available":198427434,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.69399999999999995,"15min":0.73299999999999998},"min":{"1min":0.48599999999999999,"5min":0.40000000000000002,"15min":0.36699999999999999},"max":{"1min":1.022,"5min":1.022,"15min":2.0369999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.54700000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.889},"last":0.754}]},{"osd":1,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71499999999999997,"15min":0.752},"min":{"1min":0.53200000000000003,"5min":0.42799999999999999,"15min":0.41699999999999998},"max":{"1min":1.008,"5min":1.0329999999999999,"15min":3.0920000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.71999999999999997,"15min":0.74399999999999999},"min":{"1min":0.50600000000000001,"5min":0.45200000000000001,"15min":0.39800000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":3.1499999999999999},"last":0.57299999999999995}]},{"osd":2,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.70299999999999996,"15min":0.70199999999999996},"min":{"1min":0.60899999999999999,"5min":0.44400000000000001,"15min":0.38800000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.923},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.56200000000000006,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.878}]},{"osd":3,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72799999999999998,"15min":0.70799999999999996},"min":{"1min":0.57299999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.752,"15min":0.77900000000000003},"min":{"1min":0.64100000000000001,"5min":0.47599999999999998,"15min":0.44500000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":2.1389999999999998},"last":0.97699999999999998}]},{"osd":4,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57399999999999995,"15min":0.58099999999999996},"min":{"1min":0.41099999999999998,"5min":0.36199999999999999,"15min":0.29399999999999998},"max":{"1min":0.80200000000000005,"5min":0.93100000000000005,"15min":0.96599999999999997},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.433,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77700000000000002,"5min":0.90900000000000003,"15min":1.018},"last":0.95399999999999996}]},{"osd":5,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.621,"15min":0.64100000000000001},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.92100000000000004,"5min":0.94499999999999995,"15min":1.0289999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.33400000000000002},"max":{"1min":0.86499999999999999,"5min":0.97899999999999998,"15min":0.996},"last":0.99299999999999999}]},{"osd":6,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.70599999999999996,"15min":0.70999999999999996},"min":{"1min":0.504,"5min":0.5,"15min":0.36799999999999999},"max":{"1min":0.97499999999999998,"5min":1.052,"15min":1.052},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67800000000000005,"15min":0.65600000000000003},"min":{"1min":0.48699999999999999,"5min":0.42199999999999999,"15min":0.41199999999999998},"max":{"1min":0.93500000000000005,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.83799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":958,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194353,"statfs":{"total":200000000,"available":199018308,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.70499999999999996,"15min":0.69599999999999995},"min":{"1min":0.46999999999999997,"5min":0.379,"15min":0.379},"max":{"1min":0.872,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.623},{"interface":"front","average":{"1min":0.752,"5min":0.71499999999999997,"15min":0.69799999999999995},"min":{"1min":0.64600000000000002,"5min":0.443,"15min":0.39500000000000002},"max":{"1min":0.88700000000000001,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.64400000000000002}]},{"osd":1,"last update":"Wed May 22 21:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.47799999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94299999999999995,"5min":0.95899999999999996,"15min":1.107},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70299999999999996,"15min":0.70799999999999996},"min":{"1min":0.51700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.84699999999999998,"5min":1.0449999999999999,"15min":1.081},"last":0.70299999999999996}]},{"osd":2,"last update":"Wed May 22 21:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77400000000000002,"15min":0.76700000000000002},"min":{"1min":0.61099999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.012,"5min":1.087,"15min":1.087},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.74299999999999999},"min":{"1min":0.54100000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.99099999999999999,"5min":1.419,"15min":1.419},"last":0.68700000000000006}]},{"osd":3,"last update":"Wed May 22 21:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.75900000000000001,"15min":0.749},"min":{"1min":0.57499999999999996,"5min":0.49399999999999999,"15min":0.48199999999999998},"max":{"1min":0.92500000000000004,"5min":1.28,"15min":1.415},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.53400000000000003,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":0.90900000000000003,"5min":1.004,"15min":1.788},"last":0.66600000000000004}]},{"osd":4,"last update":"Wed May 22 21:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41899999999999998,"5min":0.41699999999999998,"15min":0.374},"max":{"1min":0.878,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66600000000000004,"15min":0.67800000000000005},"min":{"1min":0.439,"5min":0.378,"15min":0.378},"max":{"1min":0.93000000000000005,"5min":1.1850000000000001,"15min":2.718},"last":0.48799999999999999}]},{"osd":5,"last update":"Wed May 22 21:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.622,"15min":0.68700000000000006},"min":{"1min":0.41399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82799999999999996,"5min":0.96999999999999997,"15min":8.5749999999999993},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68300000000000005,"15min":0.70599999999999996},"min":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":0.98499999999999999,"15min":1.1140000000000001},"last":0.76600000000000001}]},{"osd":7,"last update":"Wed May 22 21:22:29 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72299999999999998,"15min":0.73099999999999998},"min":{"1min":0.51300000000000001,"5min":0.42899999999999999,"15min":0.42399999999999999},"max":{"1min":0.95599999999999996,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69599999999999995,"15min":0.73699999999999999},"min":{"1min":0.53300000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89700000000000002,"5min":0.98099999999999998,"15min":1.8009999999999999},"last":0.71799999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":958,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194353,"statfs":{"total":200000000,"available":199018432,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.67200000000000004,"15min":0.69399999999999995},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0409999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.48699999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.93300000000000005,"5min":0.96499999999999997,"15min":1.0089999999999999},"last":0.75600000000000001}]},{"osd":1,"last update":"Wed May 22 21:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.51100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.0489999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.70299999999999996,"15min":0.69299999999999995},"min":{"1min":0.56000000000000005,"5min":0.44900000000000001,"15min":0.44},"max":{"1min":0.92600000000000005,"5min":1.321,"15min":1.321},"last":0.77000000000000002}]},{"osd":2,"last update":"Wed May 22 21:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70599999999999996,"15min":0.70699999999999996},"min":{"1min":0.48999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.98699999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72099999999999997,"15min":0.69499999999999995},"min":{"1min":0.63800000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92500000000000004,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.745}]},{"osd":3,"last update":"Wed May 22 21:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.59199999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90300000000000002,"5min":1.004,"15min":1.1879999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73099999999999998,"15min":0.73499999999999999},"min":{"1min":0.61599999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.96699999999999997,"5min":1.0469999999999999,"15min":1.111},"last":0.80100000000000005}]},{"osd":4,"last update":"Wed May 22 21:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58399999999999996,"15min":0.61099999999999999},"min":{"1min":0.439,"5min":0.38400000000000001,"15min":0.35099999999999998},"max":{"1min":0.86599999999999999,"5min":0.92400000000000004,"15min":1.272},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.625,"15min":0.63100000000000001},"min":{"1min":0.46400000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85099999999999998,"5min":1.014,"15min":1.038},"last":0.58599999999999997}]},{"osd":6,"last update":"Wed May 22 21:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.626,"15min":0.63100000000000001},"min":{"1min":0.442,"5min":0.371,"15min":0.371},"max":{"1min":0.82799999999999996,"5min":0.95199999999999996,"15min":0.98899999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.442,"15min":0.41099999999999998},"max":{"1min":0.94899999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.72899999999999998}]},{"osd":7,"last update":"Wed May 22 21:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70799999999999996,"15min":0.71399999999999997},"min":{"1min":0.56699999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96599999999999997,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.74299999999999999,"15min":0.77600000000000002},"min":{"1min":0.53000000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.2909999999999999},"last":0.78100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":958,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194353,"statfs":{"total":200000000,"available":199018454,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73499999999999999,"15min":0.749},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.30399999999999999},"max":{"1min":0.94899999999999995,"5min":0.98899999999999999,"15min":6.6539999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.56100000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.96299999999999997,"5min":0.98899999999999999,"15min":5.4370000000000003},"last":0.63100000000000001}]},{"osd":1,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.67600000000000005,"15min":0.70099999999999996},"min":{"1min":0.52500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88500000000000001,"5min":0.95799999999999996,"15min":1.7749999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.53600000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.878,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":0.55800000000000005}]},{"osd":2,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.73899999999999999,"15min":0.70499999999999996},"min":{"1min":0.45600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91700000000000004,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72799999999999998,"15min":0.71199999999999997},"min":{"1min":0.625,"5min":0.378,"15min":0.378},"max":{"1min":0.90200000000000002,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.621}]},{"osd":3,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72499999999999998,"15min":0.69899999999999995},"min":{"1min":0.51000000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89500000000000002,"5min":1.857,"15min":1.857},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78100000000000003,"15min":0.78500000000000003},"min":{"1min":0.58799999999999997,"5min":0.53200000000000003,"15min":0.47499999999999998},"max":{"1min":0.95599999999999996,"5min":1.897,"15min":1.897},"last":0.81599999999999995}]},{"osd":5,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.39200000000000002,"15min":0.379},"max":{"1min":0.80900000000000005,"5min":1.0169999999999999,"15min":1.1850000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.35599999999999998},"max":{"1min":0.83499999999999996,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.79700000000000004}]},{"osd":6,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64700000000000002,"15min":0.67700000000000005},"min":{"1min":0.45300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78400000000000003,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.68400000000000005,"15min":0.68999999999999995},"min":{"1min":0.502,"5min":0.44700000000000001,"15min":0.434},"max":{"1min":0.76000000000000001,"5min":1.095,"15min":1.1419999999999999},"last":0.70699999999999996}]},{"osd":7,"last update":"Wed May 22 21:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67000000000000004,"15min":0.68899999999999995},"min":{"1min":0.46400000000000002,"5min":0.42699999999999999,"15min":0.40799999999999997},"max":{"1min":0.80900000000000005,"5min":1.2150000000000001,"15min":1.7390000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71899999999999997,"15min":0.75700000000000001},"min":{"1min":0.48599999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.80400000000000005,"5min":1.1200000000000001,"15min":1.4510000000000001},"last":0.90400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574969,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":958,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194353,"statfs":{"total":200000000,"available":199018364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.436,"15min":0.41799999999999998},"min":{"1min":0.30299999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.68500000000000005,"5min":0.73099999999999998,"15min":0.86799999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.45800000000000002,"15min":0.42699999999999999},"min":{"1min":0.33000000000000002,"5min":0.30399999999999999,"15min":0.28399999999999997},"max":{"1min":0.73599999999999999,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.34499999999999997}]},{"osd":1,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47199999999999998,"15min":0.45700000000000002},"min":{"1min":0.33600000000000002,"5min":0.28399999999999997,"15min":0.26500000000000001},"max":{"1min":0.60099999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55000000000000004,"15min":0.52300000000000002},"min":{"1min":0.40999999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.90000000000000002},"last":0.48799999999999999}]},{"osd":2,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.41099999999999998,"5min":0.35999999999999999,"15min":0.30599999999999999},"max":{"1min":0.69799999999999995,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.52400000000000002,"15min":0.503},"min":{"1min":0.38,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.93400000000000005},"last":0.44900000000000001}]},{"osd":4,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72499999999999998,"15min":0.71399999999999997},"min":{"1min":0.56599999999999995,"5min":0.47699999999999998,"15min":0.39400000000000002},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.2050000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.57799999999999996,"5min":0.49199999999999999,"15min":0.435},"max":{"1min":1.093,"5min":1.093,"15min":1.5760000000000001},"last":0.83799999999999997}]},{"osd":5,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.754,"15min":0.71999999999999997},"min":{"1min":0.55000000000000004,"5min":0.46800000000000003,"15min":0.45100000000000001},"max":{"1min":1,"5min":1.048,"15min":1.234},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73699999999999999,"15min":0.71799999999999997},"min":{"1min":0.60899999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.024,"5min":1.0449999999999999,"15min":1.3049999999999999},"last":0.80000000000000004}]},{"osd":6,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.73799999999999999,"15min":0.72499999999999998},"min":{"1min":0.60199999999999998,"5min":0.48499999999999999,"15min":0.44900000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.276},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.75900000000000001,"15min":0.72699999999999998},"min":{"1min":0.59399999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.05,"5min":1.0820000000000001,"15min":1.2869999999999999},"last":0.97299999999999998}]},{"osd":7,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.78200000000000003,"15min":0.77000000000000002},"min":{"1min":0.59399999999999997,"5min":0.501,"15min":0.495},"max":{"1min":1.0880000000000001,"5min":1.0900000000000001,"15min":1.2709999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.78200000000000003,"15min":0.75600000000000001},"min":{"1min":0.623,"5min":0.52500000000000002,"15min":0.51100000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.2969999999999999},"last":0.72899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574969,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":960,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194352,"statfs":{"total":200000000,"available":199016820,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.46000000000000002,"15min":0.41999999999999998},"min":{"1min":0.32900000000000001,"5min":0.253,"15min":0.23599999999999999},"max":{"1min":0.64800000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.49099999999999999,"15min":0.45600000000000002},"min":{"1min":0.35999999999999999,"5min":0.26800000000000002,"15min":0.23400000000000001},"max":{"1min":0.61099999999999999,"5min":2.532,"15min":2.532},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48399999999999999,"15min":0.45800000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.29399999999999998},"max":{"1min":0.68899999999999995,"5min":2.948,"15min":2.948},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58299999999999996,"15min":0.54000000000000004},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.39100000000000001},"max":{"1min":0.71399999999999997,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.42899999999999999}]},{"osd":3,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55000000000000004,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.36899999999999999,"15min":0.32700000000000001},"max":{"1min":0.83499999999999996,"5min":2.75,"15min":2.75},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.48399999999999999},"min":{"1min":0.41999999999999998,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":0.66600000000000004,"5min":3.0619999999999998,"15min":3.0619999999999998},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.79900000000000004,"15min":0.72799999999999998},"min":{"1min":0.64400000000000002,"5min":0.53100000000000003,"15min":0.52700000000000002},"max":{"1min":1.103,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.83699999999999997,"15min":0.77700000000000002},"min":{"1min":0.71299999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.208,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.81499999999999995}]},{"osd":5,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.79600000000000004,"15min":0.73399999999999999},"min":{"1min":0.628,"5min":0.498,"15min":0.47299999999999998},"max":{"1min":1.133,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.75700000000000001,"15min":0.72299999999999998},"min":{"1min":0.58099999999999996,"5min":0.47899999999999998,"15min":0.45900000000000002},"max":{"1min":1.0549999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.78400000000000003,"15min":0.73199999999999998},"min":{"1min":0.56999999999999995,"5min":0.51300000000000001,"15min":0.495},"max":{"1min":1.048,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.76100000000000001,"15min":0.70699999999999996},"min":{"1min":0.59899999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.147,"5min":2.6880000000000002,"15min":2.6880000000000002},"last":0.68100000000000005}]},{"osd":7,"last update":"Wed May 22 21:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.82399999999999995,"15min":0.76100000000000001},"min":{"1min":0.629,"5min":0.438,"15min":0.438},"max":{"1min":1.2210000000000001,"5min":3.0099999999999998,"15min":3.0099999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.84599999999999997,"15min":0.79700000000000004},"min":{"1min":0.63600000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.1339999999999999,"5min":2.827,"15min":2.827},"last":0.77500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":959,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194353,"statfs":{"total":200000000,"available":199017872,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.439,"15min":0.433},"min":{"1min":0.28599999999999998,"5min":0.216,"15min":0.216},"max":{"1min":0.73499999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.254},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.47399999999999998,"15min":0.44900000000000001},"min":{"1min":0.35199999999999998,"5min":0.29299999999999998,"15min":0.26000000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed May 22 21:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.50900000000000001,"15min":0.496},"min":{"1min":0.36199999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.73099999999999998,"5min":1.76,"15min":1.76},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.48399999999999999,"15min":0.51400000000000001},"min":{"1min":0.32200000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":1.923},"last":0.44600000000000001}]},{"osd":3,"last update":"Wed May 22 21:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54200000000000004,"15min":0.55700000000000005},"min":{"1min":0.46000000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":2.1030000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.52500000000000002,"15min":0.5},"min":{"1min":0.433,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73299999999999998,"5min":0.745,"15min":0.745},"last":0.379}]},{"osd":4,"last update":"Wed May 22 21:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.45100000000000001},"max":{"1min":0.95899999999999996,"5min":1.109,"15min":2.1360000000000001},"last":0.628},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73999999999999999,"15min":0.70599999999999996},"min":{"1min":0.53600000000000003,"5min":0.46500000000000002,"15min":0.44900000000000001},"max":{"1min":0.91800000000000004,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.64000000000000001}]},{"osd":5,"last update":"Wed May 22 21:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.77200000000000002,"15min":0.749},"min":{"1min":0.58599999999999997,"5min":0.56000000000000005,"15min":0.45300000000000001},"max":{"1min":0.92000000000000004,"5min":1.819,"15min":1.819},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.745,"15min":0.75},"min":{"1min":0.54300000000000004,"5min":0.46300000000000002,"15min":0.40100000000000002},"max":{"1min":1.034,"5min":1.1060000000000001,"15min":1.97},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed May 22 21:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.77100000000000002,"15min":0.75600000000000001},"min":{"1min":0.56100000000000005,"5min":0.48999999999999999,"15min":0.47299999999999998},"max":{"1min":0.94499999999999995,"5min":1.1359999999999999,"15min":1.272},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80900000000000005,"15min":0.77000000000000002},"min":{"1min":0.63900000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98999999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.68100000000000005}]},{"osd":7,"last update":"Wed May 22 21:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77200000000000002,"15min":0.77300000000000002},"min":{"1min":0.65400000000000003,"5min":0.48599999999999999,"15min":0.42099999999999999},"max":{"1min":0.98799999999999999,"5min":1.1379999999999999,"15min":2.0310000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.76200000000000001,"15min":0.77000000000000002},"min":{"1min":0.626,"5min":0.54900000000000004,"15min":0.52000000000000002},"max":{"1min":0.96099999999999997,"5min":1.0660000000000001,"15min":2.0659999999999998},"last":0.66300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574969,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1536,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193776,"statfs":{"total":200000000,"available":198427107,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.441,"15min":0.41999999999999998},"min":{"1min":0.32900000000000001,"5min":0.24199999999999999,"15min":0.22800000000000001},"max":{"1min":0.67600000000000005,"5min":0.85899999999999999,"15min":0.88300000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.52300000000000002,"15min":0.5},"min":{"1min":0.46100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed May 22 21:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.5,"15min":0.47199999999999998},"min":{"1min":0.40699999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.85899999999999999,"5min":0.88400000000000001,"15min":0.93799999999999994},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.46899999999999997,"15min":0.441},"min":{"1min":0.33700000000000002,"5min":0.28399999999999997,"15min":0.27900000000000003},"max":{"1min":0.64600000000000002,"5min":0.70799999999999996,"15min":0.92300000000000004},"last":0.47999999999999998}]},{"osd":3,"last update":"Wed May 22 21:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46700000000000003,"15min":0.44600000000000001},"min":{"1min":0.38,"5min":0.313,"15min":0.28499999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":1.0760000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56000000000000005,"15min":0.52200000000000002},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89700000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed May 22 21:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.73799999999999999,"15min":0.72099999999999997},"min":{"1min":0.61799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.76900000000000002,"15min":0.74299999999999999},"min":{"1min":0.65300000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2709999999999999},"last":0.871}]},{"osd":5,"last update":"Wed May 22 21:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76000000000000001,"15min":0.72299999999999998},"min":{"1min":0.56899999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.76100000000000001,"15min":0.747},"min":{"1min":0.58399999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.84799999999999998}]},{"osd":6,"last update":"Wed May 22 21:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.80500000000000005,"15min":0.78500000000000003},"min":{"1min":0.68200000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.2090000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.79000000000000004,"15min":0.77700000000000002},"min":{"1min":0.66600000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.5},"last":0.88600000000000001}]},{"osd":7,"last update":"Wed May 22 21:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.77100000000000002,"15min":0.74199999999999999},"min":{"1min":0.60499999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.76600000000000001,"15min":0.76200000000000001},"min":{"1min":0.64500000000000002,"5min":0.52800000000000002,"15min":0.497},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.89600000000000002}]}]}],"pool_statfs":[]}} 2024-05-22T21:23:08.314 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-22T21:23:08.314 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-22T21:23:08.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b2a400640 1 Processor -- start 2024-05-22T21:23:08.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b2a400640 1 -- start start 2024-05-22T21:23:08.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b2a400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b24143ed0 0x7f4b241442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b2a400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4b240ce370 0x7f4b240b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b2a400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b240ce830 con 0x7f4b241448e0 2024-05-22T21:23:08.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b2a400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b240b2e60 con 0x7f4b240ce370 2024-05-22T21:23:08.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b2a400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b240b3140 con 0x7f4b24143ed0 2024-05-22T21:23:08.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b29a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4b240ce370 0x7f4b240b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b29000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b24143ed0 0x7f4b241442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b29a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4b240ce370 0x7f4b240b27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:56360/0 (socket says 172.21.15.161:56360) 2024-05-22T21:23:08.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b29a00640 1 -- 172.21.15.161:0/3602586197 learned_addr learned my addr 172.21.15.161:0/3602586197 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:08.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b23e00640 1 --1- 172.21.15.161:0/3602586197 >> v1:172.21.15.179:6789/0 conn(0x7f4b241448e0 0x7f4b240b20e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:57948/0 (socket says 172.21.15.161:57948) 2024-05-22T21:23:08.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b23400640 1 -- 172.21.15.161:0/3602586197 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630033282 0 0) 0x7f4b240ce830 con 0x7f4b241448e0 2024-05-22T21:23:08.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b29000640 1 -- 172.21.15.161:0/3602586197 >> v2:172.21.15.179:3300/0 conn(0x7f4b240ce370 msgr2=0x7f4b240b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:08.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b29000640 1 --2- 172.21.15.161:0/3602586197 >> v2:172.21.15.179:3300/0 conn(0x7f4b240ce370 0x7f4b240b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b29000640 1 -- 172.21.15.161:0/3602586197 >> v1:172.21.15.179:6789/0 conn(0x7f4b241448e0 legacy=0x7f4b240b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:08.494 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.492+0000 7f4b29000640 1 -- 172.21.15.161:0/3602586197 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b240b3420 con 0x7f4b24143ed0 2024-05-22T21:23:08.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.496+0000 7f4b29000640 1 --2- 172.21.15.161:0/3602586197 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b24143ed0 0x7f4b241442b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4b180097c0 tx=0x7f4b1801c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbb033958b57cb23 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:08.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.496+0000 7f4b23400640 1 -- 172.21.15.161:0/3602586197 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b180ac600 con 0x7f4b24143ed0 2024-05-22T21:23:08.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.496+0000 7f4b23400640 1 -- 172.21.15.161:0/3602586197 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b180a8050 con 0x7f4b24143ed0 2024-05-22T21:23:08.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.496+0000 7f4b23400640 1 -- 172.21.15.161:0/3602586197 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b180ac980 con 0x7f4b24143ed0 2024-05-22T21:23:08.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.496+0000 7f4b2a400640 1 -- 172.21.15.161:0/3602586197 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b24143ed0 msgr2=0x7f4b241442b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:08.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.496+0000 7f4b2a400640 1 --2- 172.21.15.161:0/3602586197 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b24143ed0 0x7f4b241442b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4b180097c0 tx=0x7f4b1801c380 comp rx=0 tx=0).stop 2024-05-22T21:23:08.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.496+0000 7f4b2a400640 1 -- 172.21.15.161:0/3602586197 shutdown_connections 2024-05-22T21:23:08.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.496+0000 7f4b2a400640 1 --2- 172.21.15.161:0/3602586197 >> v2:172.21.15.179:3300/0 conn(0x7f4b240ce370 0x7f4b240b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.496+0000 7f4b2a400640 1 --2- 172.21.15.161:0/3602586197 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b24143ed0 0x7f4b241442b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.496+0000 7f4b2a400640 1 -- 172.21.15.161:0/3602586197 >> 172.21.15.161:0/3602586197 conn(0x7f4b240cec00 msgr2=0x7f4b240cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:08.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.496+0000 7f4b2a400640 1 -- 172.21.15.161:0/3602586197 shutdown_connections 2024-05-22T21:23:08.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.496+0000 7f4b2a400640 1 -- 172.21.15.161:0/3602586197 wait complete. 2024-05-22T21:23:08.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b2a400640 1 Processor -- start 2024-05-22T21:23:08.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b2a400640 1 -- start start 2024-05-22T21:23:08.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b2a400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b240ce370 0x7f4b24071060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b2a400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b24143ed0 0x7f4b24140470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b2a400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b241448e0 0x7f4b240accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b2a400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b241409b0 con 0x7f4b240ce370 2024-05-22T21:23:08.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b2a400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b240ad230 con 0x7f4b241448e0 2024-05-22T21:23:08.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b2a400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b240ad500 con 0x7f4b24143ed0 2024-05-22T21:23:08.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b29000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b240ce370 0x7f4b24071060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b29000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b240ce370 0x7f4b24071060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42370/0 (socket says 172.21.15.161:42370) 2024-05-22T21:23:08.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b29000640 1 -- 172.21.15.161:0/208613344 learned_addr learned my addr 172.21.15.161:0/208613344 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:08.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b23e00640 1 --2- 172.21.15.161:0/208613344 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b24143ed0 0x7f4b24140470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b29a00640 1 --2- 172.21.15.161:0/208613344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b241448e0 0x7f4b240accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b29000640 1 -- 172.21.15.161:0/208613344 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b24143ed0 msgr2=0x7f4b24140470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:08.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b29000640 1 --2- 172.21.15.161:0/208613344 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b24143ed0 0x7f4b24140470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b29000640 1 -- 172.21.15.161:0/208613344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b241448e0 msgr2=0x7f4b240accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:08.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b29000640 1 --2- 172.21.15.161:0/208613344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b241448e0 0x7f4b240accf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b29000640 1 -- 172.21.15.161:0/208613344 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b18009420 con 0x7f4b240ce370 2024-05-22T21:23:08.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.500+0000 7f4b29000640 1 --2- 172.21.15.161:0/208613344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b240ce370 0x7f4b24071060 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f4b1801aa30 tx=0x7f4b1801abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:08.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.504+0000 7f4b20c00640 1 -- 172.21.15.161:0/208613344 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b180b7830 con 0x7f4b240ce370 2024-05-22T21:23:08.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.504+0000 7f4b20c00640 1 -- 172.21.15.161:0/208613344 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b180b7990 con 0x7f4b240ce370 2024-05-22T21:23:08.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.504+0000 7f4b20c00640 1 -- 172.21.15.161:0/208613344 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b180b7d40 con 0x7f4b240ce370 2024-05-22T21:23:08.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.504+0000 7f4b2a400640 1 -- 172.21.15.161:0/208613344 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b240ad7e0 con 0x7f4b240ce370 2024-05-22T21:23:08.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.504+0000 7f4b2a400640 1 -- 172.21.15.161:0/208613344 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b240adda0 con 0x7f4b240ce370 2024-05-22T21:23:08.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.504+0000 7f4b2a400640 1 -- 172.21.15.161:0/208613344 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b24067070 con 0x7f4b240ce370 2024-05-22T21:23:08.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.504+0000 7f4b20c00640 1 -- 172.21.15.161:0/208613344 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4b180b3040 con 0x7f4b240ce370 2024-05-22T21:23:08.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.504+0000 7f4b20c00640 1 --2- 172.21.15.161:0/208613344 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4b04066690 0x7f4b04068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.504+0000 7f4b20c00640 1 -- 172.21.15.161:0/208613344 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b1811eb30 con 0x7f4b240ce370 2024-05-22T21:23:08.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.504+0000 7f4b23e00640 1 --2- 172.21.15.161:0/208613344 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4b04066690 0x7f4b04068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.508+0000 7f4b23e00640 1 --2- 172.21.15.161:0/208613344 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4b04066690 0x7f4b04068b50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4b2406d510 tx=0x7f4b10007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:08.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.508+0000 7f4b20c00640 1 -- 172.21.15.161:0/208613344 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4b180ebe90 con 0x7f4b240ce370 2024-05-22T21:23:08.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.716+0000 7f4b2a400640 1 -- 172.21.15.161:0/208613344 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4b24141f30 con 0x7f4b240ce370 2024-05-22T21:23:08.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.720+0000 7f4b20c00640 1 -- 172.21.15.161:0/208613344 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7f4b180ef4b0 con 0x7f4b240ce370 2024-05-22T21:23:08.722 INFO:teuthology.orchestra.run.smithi161.stderr:updated caps for client.0 2024-05-22T21:23:08.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.724+0000 7f4b2a400640 1 -- 172.21.15.161:0/208613344 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4b04066690 msgr2=0x7f4b04068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:08.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.724+0000 7f4b2a400640 1 --2- 172.21.15.161:0/208613344 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4b04066690 0x7f4b04068b50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4b2406d510 tx=0x7f4b10007c10 comp rx=0 tx=0).stop 2024-05-22T21:23:08.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.724+0000 7f4b2a400640 1 -- 172.21.15.161:0/208613344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b240ce370 msgr2=0x7f4b24071060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:08.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.724+0000 7f4b2a400640 1 --2- 172.21.15.161:0/208613344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b240ce370 0x7f4b24071060 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f4b1801aa30 tx=0x7f4b1801abc0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.724+0000 7f4b2a400640 1 -- 172.21.15.161:0/208613344 shutdown_connections 2024-05-22T21:23:08.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.724+0000 7f4b2a400640 1 --2- 172.21.15.161:0/208613344 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4b04066690 0x7f4b04068b50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.724+0000 7f4b2a400640 1 --2- 172.21.15.161:0/208613344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b241448e0 0x7f4b240accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.724+0000 7f4b2a400640 1 --2- 172.21.15.161:0/208613344 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4b24143ed0 0x7f4b24140470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.724+0000 7f4b2a400640 1 --2- 172.21.15.161:0/208613344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b240ce370 0x7f4b24071060 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.724+0000 7f4b2a400640 1 -- 172.21.15.161:0/208613344 >> 172.21.15.161:0/208613344 conn(0x7f4b240cec00 msgr2=0x7f4b24141820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:08.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.724+0000 7f4b2a400640 1 -- 172.21.15.161:0/208613344 shutdown_connections 2024-05-22T21:23:08.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.724+0000 7f4b2a400640 1 -- 172.21.15.161:0/208613344 wait complete. 2024-05-22T21:23:08.744 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:23:08.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc808600640 1 Processor -- start 2024-05-22T21:23:08.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc808600640 1 -- start start 2024-05-22T21:23:08.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc808600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc800143ec0 0x7fc80015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc808600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8000715a0 0x7fc80006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc808600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8001442a0 con 0x7fc80015ec90 2024-05-22T21:23:08.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc808600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc800070260 con 0x7fc800143ec0 2024-05-22T21:23:08.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc808600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc800070500 con 0x7fc8000715a0 2024-05-22T21:23:08.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc806600640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc80015ec90 0x7fc80006bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:40738/0 (socket says 172.21.15.161:40738) 2024-05-22T21:23:08.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc806600640 1 -- 172.21.15.161:0/3769230799 learned_addr learned my addr 172.21.15.161:0/3769230799 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:23:08.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc805200640 1 --2- 172.21.15.161:0/3769230799 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8000715a0 0x7fc80006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc805c00640 1 --2- 172.21.15.161:0/3769230799 >> v2:172.21.15.179:3300/0 conn(0x7fc800143ec0 0x7fc80015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc7ffe00640 1 -- 172.21.15.161:0/3769230799 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711537781 0 0) 0x7fc8001442a0 con 0x7fc80015ec90 2024-05-22T21:23:08.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc7ffe00640 1 -- 172.21.15.161:0/3769230799 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7e8003530 con 0x7fc80015ec90 2024-05-22T21:23:08.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc805200640 1 -- 172.21.15.161:0/3769230799 >> v2:172.21.15.179:3300/0 conn(0x7fc800143ec0 msgr2=0x7fc80015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:08.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc805200640 1 --2- 172.21.15.161:0/3769230799 >> v2:172.21.15.179:3300/0 conn(0x7fc800143ec0 0x7fc80015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc805200640 1 -- 172.21.15.161:0/3769230799 >> v1:172.21.15.161:6789/0 conn(0x7fc80015ec90 legacy=0x7fc80006bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:08.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.880+0000 7fc805200640 1 -- 172.21.15.161:0/3769230799 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8000707e0 con 0x7fc8000715a0 2024-05-22T21:23:08.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.884+0000 7fc805200640 1 --2- 172.21.15.161:0/3769230799 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8000715a0 0x7fc80006fd20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0097c0 tx=0x7fc7ec01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5491933a706cdbc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:08.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.884+0000 7fc7ffe00640 1 -- 172.21.15.161:0/3769230799 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec0ab670 con 0x7fc8000715a0 2024-05-22T21:23:08.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.884+0000 7fc7ffe00640 1 -- 172.21.15.161:0/3769230799 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ec0ab7d0 con 0x7fc8000715a0 2024-05-22T21:23:08.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.884+0000 7fc7ffe00640 1 -- 172.21.15.161:0/3769230799 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec0abb40 con 0x7fc8000715a0 2024-05-22T21:23:08.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.884+0000 7fc808600640 1 -- 172.21.15.161:0/3769230799 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8000715a0 msgr2=0x7fc80006fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:08.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.884+0000 7fc808600640 1 --2- 172.21.15.161:0/3769230799 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8000715a0 0x7fc80006fd20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0097c0 tx=0x7fc7ec01c470 comp rx=0 tx=0).stop 2024-05-22T21:23:08.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.884+0000 7fc808600640 1 -- 172.21.15.161:0/3769230799 shutdown_connections 2024-05-22T21:23:08.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.884+0000 7fc808600640 1 --2- 172.21.15.161:0/3769230799 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8000715a0 0x7fc80006fd20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.884+0000 7fc808600640 1 --2- 172.21.15.161:0/3769230799 >> v2:172.21.15.179:3300/0 conn(0x7fc800143ec0 0x7fc80015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.884+0000 7fc808600640 1 -- 172.21.15.161:0/3769230799 >> 172.21.15.161:0/3769230799 conn(0x7fc8000b0bc0 msgr2=0x7fc8000b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:08.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.884+0000 7fc808600640 1 -- 172.21.15.161:0/3769230799 shutdown_connections 2024-05-22T21:23:08.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.884+0000 7fc808600640 1 -- 172.21.15.161:0/3769230799 wait complete. 2024-05-22T21:23:08.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc808600640 1 Processor -- start 2024-05-22T21:23:08.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc808600640 1 -- start start 2024-05-22T21:23:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc808600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc800143ec0 0x7fc800157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc808600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc80015ec90 0x7fc80013a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc808600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc80013ad20 0x7fc8000cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc808600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc800140690 con 0x7fc80013ad20 2024-05-22T21:23:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc808600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc800159ad0 con 0x7fc80015ec90 2024-05-22T21:23:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc808600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc800159db0 con 0x7fc800143ec0 2024-05-22T21:23:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc806600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc80013ad20 0x7fc8000cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc806600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc80013ad20 0x7fc8000cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42376/0 (socket says 172.21.15.161:42376) 2024-05-22T21:23:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc806600640 1 -- 172.21.15.161:0/2707966043 learned_addr learned my addr 172.21.15.161:0/2707966043 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc805c00640 1 --2- 172.21.15.161:0/2707966043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc800143ec0 0x7fc800157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc805200640 1 --2- 172.21.15.161:0/2707966043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc80015ec90 0x7fc80013a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc806600640 1 -- 172.21.15.161:0/2707966043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc800143ec0 msgr2=0x7fc800157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc806600640 1 --2- 172.21.15.161:0/2707966043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc800143ec0 0x7fc800157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc806600640 1 -- 172.21.15.161:0/2707966043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc80015ec90 msgr2=0x7fc80013a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc806600640 1 --2- 172.21.15.161:0/2707966043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc80015ec90 0x7fc80013a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc806600640 1 -- 172.21.15.161:0/2707966043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec009420 con 0x7fc80013ad20 2024-05-22T21:23:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc805200640 1 --2- 172.21.15.161:0/2707966043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc80015ec90 0x7fc80013a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:23:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc806600640 1 --2- 172.21.15.161:0/2707966043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc80013ad20 0x7fc8000cdd90 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800caa0 tx=0x7fc7f800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc7fd600640 1 -- 172.21.15.161:0/2707966043 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f800ac00 con 0x7fc80013ad20 2024-05-22T21:23:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc7fd600640 1 -- 172.21.15.161:0/2707966043 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f800ad60 con 0x7fc80013ad20 2024-05-22T21:23:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc7fd600640 1 -- 172.21.15.161:0/2707966043 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8012620 con 0x7fc80013ad20 2024-05-22T21:23:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc808600640 1 -- 172.21.15.161:0/2707966043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80015f630 con 0x7fc80013ad20 2024-05-22T21:23:08.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.888+0000 7fc808600640 1 -- 172.21.15.161:0/2707966043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80015a370 con 0x7fc80013ad20 2024-05-22T21:23:08.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.892+0000 7fc808600640 1 -- 172.21.15.161:0/2707966043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc800067220 con 0x7fc80013ad20 2024-05-22T21:23:08.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.892+0000 7fc7fd600640 1 -- 172.21.15.161:0/2707966043 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc7f8007070 con 0x7fc80013ad20 2024-05-22T21:23:08.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.892+0000 7fc7fd600640 1 --2- 172.21.15.161:0/2707966043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7e00666c0 0x7fc7e0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:08.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.892+0000 7fc7fd600640 1 -- 172.21.15.161:0/2707966043 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7f80840f0 con 0x7fc80013ad20 2024-05-22T21:23:08.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.892+0000 7fc805c00640 1 --2- 172.21.15.161:0/2707966043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7e00666c0 0x7fc7e0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:08.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.892+0000 7fc805c00640 1 --2- 172.21.15.161:0/2707966043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7e00666c0 0x7fc7e0068b80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009b60 tx=0x7fc7f4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:08.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:08.896+0000 7fc7fd600640 1 -- 172.21.15.161:0/2707966043 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc7f8051470 con 0x7fc80013ad20 2024-05-22T21:23:09.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.080+0000 7fc808600640 1 -- 172.21.15.161:0/2707966043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc80006dcc0 con 0x7fc80013ad20 2024-05-22T21:23:09.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.080+0000 7fc7fd600640 1 -- 172.21.15.161:0/2707966043 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4747 (secure 0 0 0) 0x7fc7f8054a90 con 0x7fc80013ad20 2024-05-22T21:23:09.081 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:23:09.081 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2405126804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2405126804},{"type":"v1","addr":"172.21.15.179:6834","nonce":2405126804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":6962,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3984924056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3984924056},{"type":"v1","addr":"172.21.15.179:6835","nonce":3984924056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8391,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/368424851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":368424851},{"type":"v1","addr":"172.21.15.161:6836","nonce":368424851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:23:03.863540+0000","modified":"2024-05-22T21:23:06.877759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2279022192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2279022192},{"type":"v1","addr":"172.21.15.161:6837","nonce":2279022192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-05-22T21:23:09.081 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 78 2024-05-22T21:23:09.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.084+0000 7fc808600640 1 -- 172.21.15.161:0/2707966043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7e00666c0 msgr2=0x7fc7e0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:09.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.084+0000 7fc808600640 1 --2- 172.21.15.161:0/2707966043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7e00666c0 0x7fc7e0068b80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009b60 tx=0x7fc7f4009190 comp rx=0 tx=0).stop 2024-05-22T21:23:09.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.084+0000 7fc808600640 1 -- 172.21.15.161:0/2707966043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc80013ad20 msgr2=0x7fc8000cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:09.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.084+0000 7fc808600640 1 --2- 172.21.15.161:0/2707966043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc80013ad20 0x7fc8000cdd90 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800caa0 tx=0x7fc7f800cf60 comp rx=0 tx=0).stop 2024-05-22T21:23:09.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.084+0000 7fc808600640 1 -- 172.21.15.161:0/2707966043 shutdown_connections 2024-05-22T21:23:09.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.084+0000 7fc808600640 1 --2- 172.21.15.161:0/2707966043 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc7e00666c0 0x7fc7e0068b80 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.084+0000 7fc808600640 1 --2- 172.21.15.161:0/2707966043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc80013ad20 0x7fc8000cdd90 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.084+0000 7fc808600640 1 --2- 172.21.15.161:0/2707966043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc80015ec90 0x7fc80013a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.084+0000 7fc808600640 1 --2- 172.21.15.161:0/2707966043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc800143ec0 0x7fc800157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.084+0000 7fc808600640 1 -- 172.21.15.161:0/2707966043 >> 172.21.15.161:0/2707966043 conn(0x7fc8000b0bc0 msgr2=0x7fc80006d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:09.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.084+0000 7fc808600640 1 -- 172.21.15.161:0/2707966043 shutdown_connections 2024-05-22T21:23:09.084 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.084+0000 7fc808600640 1 -- 172.21.15.161:0/2707966043 wait complete. 2024-05-22T21:23:09.505 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-22T21:23:03.863540+0000', 'modified': '2024-05-22T21:23:06.877759+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8394}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8394': {'gid': 8394, 'name': 'a', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6837/2279022192', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6835', 'nonce': 2279022192}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 2279022192}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-22T21:23:09.505 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-22T21:23:09.505 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-22T21:23:09.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e51400640 1 Processor -- start 2024-05-22T21:23:09.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e51400640 1 -- start start 2024-05-22T21:23:09.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e51400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0e4c143c40 0x7f0e4c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e51400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0e4c06c9b0 0x7f0e4c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e51400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4c15efb0 con 0x7f0e4c1444a0 2024-05-22T21:23:09.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e51400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4c15f290 con 0x7f0e4c06c9b0 2024-05-22T21:23:09.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e51400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4c15f570 con 0x7f0e4c143c40 2024-05-22T21:23:09.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e4b200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0e4c06c9b0 0x7f0e4c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e4b200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0e4c06c9b0 0x7f0e4c0b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42392/0 (socket says 172.21.15.161:42392) 2024-05-22T21:23:09.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e4b200640 1 -- 172.21.15.161:0/732000511 learned_addr learned my addr 172.21.15.161:0/732000511 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:09.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e43e00640 1 --1- 172.21.15.161:0/732000511 >> v1:172.21.15.161:6789/0 conn(0x7f0e4c1444a0 0x7f0e4c06c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:40750/0 (socket says 172.21.15.161:40750) 2024-05-22T21:23:09.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e4a800640 1 --2- 172.21.15.161:0/732000511 >> v2:172.21.15.179:3300/0 conn(0x7f0e4c143c40 0x7f0e4c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e4b200640 1 -- 172.21.15.161:0/732000511 >> v2:172.21.15.179:3300/0 conn(0x7f0e4c143c40 msgr2=0x7f0e4c15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:09.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e4b200640 1 --2- 172.21.15.161:0/732000511 >> v2:172.21.15.179:3300/0 conn(0x7f0e4c143c40 0x7f0e4c15b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e4b200640 1 -- 172.21.15.161:0/732000511 >> v1:172.21.15.161:6789/0 conn(0x7f0e4c1444a0 legacy=0x7f0e4c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:09.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e4b200640 1 -- 172.21.15.161:0/732000511 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4c15f850 con 0x7f0e4c06c9b0 2024-05-22T21:23:09.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e49e00640 1 -- 172.21.15.161:0/732000511 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408243982 0 0) 0x7f0e4c15efb0 con 0x7f0e4c1444a0 2024-05-22T21:23:09.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.644+0000 7f0e4a800640 1 --2- 172.21.15.161:0/732000511 >> v2:172.21.15.179:3300/0 conn(0x7f0e4c143c40 0x7f0e4c15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:23:09.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.648+0000 7f0e4b200640 1 --2- 172.21.15.161:0/732000511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4c06c9b0 0x7f0e4c0b2160 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c0724b0 tx=0x7f0e4401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cf96963611b35 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:09.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.648+0000 7f0e49e00640 1 -- 172.21.15.161:0/732000511 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e440a5820 con 0x7f0e4c06c9b0 2024-05-22T21:23:09.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.648+0000 7f0e49e00640 1 -- 172.21.15.161:0/732000511 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e440a5980 con 0x7f0e4c06c9b0 2024-05-22T21:23:09.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.648+0000 7f0e49e00640 1 -- 172.21.15.161:0/732000511 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e440a5d10 con 0x7f0e4c06c9b0 2024-05-22T21:23:09.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.648+0000 7f0e51400640 1 -- 172.21.15.161:0/732000511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4c06c9b0 msgr2=0x7f0e4c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:09.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.648+0000 7f0e51400640 1 --2- 172.21.15.161:0/732000511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4c06c9b0 0x7f0e4c0b2160 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c0724b0 tx=0x7f0e4401c9f0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.648+0000 7f0e51400640 1 -- 172.21.15.161:0/732000511 shutdown_connections 2024-05-22T21:23:09.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.648+0000 7f0e51400640 1 --2- 172.21.15.161:0/732000511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4c06c9b0 0x7f0e4c0b2160 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.648+0000 7f0e51400640 1 --2- 172.21.15.161:0/732000511 >> v2:172.21.15.179:3300/0 conn(0x7f0e4c143c40 0x7f0e4c15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.648+0000 7f0e51400640 1 -- 172.21.15.161:0/732000511 >> 172.21.15.161:0/732000511 conn(0x7f0e4c06a2d0 msgr2=0x7f0e4c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:09.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.648+0000 7f0e51400640 1 -- 172.21.15.161:0/732000511 shutdown_connections 2024-05-22T21:23:09.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.648+0000 7f0e51400640 1 -- 172.21.15.161:0/732000511 wait complete. 2024-05-22T21:23:09.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.648+0000 7f0e51400640 1 Processor -- start 2024-05-22T21:23:09.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.648+0000 7f0e51400640 1 -- start start 2024-05-22T21:23:09.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e51400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4c06c9b0 0x7f0e4c0ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e51400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4c143c40 0x7f0e4c154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e51400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e4c1444a0 0x7f0e4c06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e51400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c0ad3d0 con 0x7f0e4c143c40 2024-05-22T21:23:09.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e51400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c070930 con 0x7f0e4c1444a0 2024-05-22T21:23:09.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e51400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c0aca30 con 0x7f0e4c06c9b0 2024-05-22T21:23:09.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4c143c40 0x7f0e4c154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4c143c40 0x7f0e4c154810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42406/0 (socket says 172.21.15.161:42406) 2024-05-22T21:23:09.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e43e00640 1 -- 172.21.15.161:0/2588745682 learned_addr learned my addr 172.21.15.161:0/2588745682 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:09.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e4b200640 1 --2- 172.21.15.161:0/2588745682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e4c1444a0 0x7f0e4c06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e43e00640 1 -- 172.21.15.161:0/2588745682 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4c06c9b0 msgr2=0x7f0e4c0ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:09.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e4a800640 1 --2- 172.21.15.161:0/2588745682 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4c06c9b0 0x7f0e4c0ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e43e00640 1 --2- 172.21.15.161:0/2588745682 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4c06c9b0 0x7f0e4c0ad7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e43e00640 1 -- 172.21.15.161:0/2588745682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e4c1444a0 msgr2=0x7f0e4c06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:09.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e43e00640 1 --2- 172.21.15.161:0/2588745682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e4c1444a0 0x7f0e4c06fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.653 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e43e00640 1 -- 172.21.15.161:0/2588745682 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e44009b30 con 0x7f0e4c143c40 2024-05-22T21:23:09.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e4b200640 1 --2- 172.21.15.161:0/2588745682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e4c1444a0 0x7f0e4c06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:23:09.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e4a800640 1 --2- 172.21.15.161:0/2588745682 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4c06c9b0 0x7f0e4c0ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:23:09.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e43e00640 1 --2- 172.21.15.161:0/2588745682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4c143c40 0x7f0e4c154810 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400c480 tx=0x7f0e3400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:09.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e42a00640 1 -- 172.21.15.161:0/2588745682 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e340074e0 con 0x7f0e4c143c40 2024-05-22T21:23:09.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e42a00640 1 -- 172.21.15.161:0/2588745682 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e34007660 con 0x7f0e4c143c40 2024-05-22T21:23:09.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e42a00640 1 -- 172.21.15.161:0/2588745682 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34004a30 con 0x7f0e4c143c40 2024-05-22T21:23:09.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e51400640 1 -- 172.21.15.161:0/2588745682 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4c0acd10 con 0x7f0e4c143c40 2024-05-22T21:23:09.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.652+0000 7f0e51400640 1 -- 172.21.15.161:0/2588745682 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4c154d50 con 0x7f0e4c143c40 2024-05-22T21:23:09.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e42a00640 1 -- 172.21.15.161:0/2588745682 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0e34004b90 con 0x7f0e4c143c40 2024-05-22T21:23:09.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e42a00640 1 --2- 172.21.15.161:0/2588745682 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e28066690 0x7f0e28068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e42a00640 1 -- 172.21.15.161:0/2588745682 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e34083360 con 0x7f0e4c143c40 2024-05-22T21:23:09.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e4a800640 1 --2- 172.21.15.161:0/2588745682 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e28066690 0x7f0e28068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e548ca000 1 Processor -- start 2024-05-22T21:23:09.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e548ca000 1 -- start start 2024-05-22T21:23:09.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e4a800640 1 --2- 172.21.15.161:0/2588745682 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e28066690 0x7f0e28068b50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c008930 tx=0x7f0e3c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:09.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e548ca000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6629c0 0x55884a662da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e548ca000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a6632e0 0x55884a618980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e548ca000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a618ec0 0x55884a6192a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e548ca000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884a4e0e80 con 0x55884a618ec0 2024-05-22T21:23:09.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e548ca000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884a46eaf0 con 0x55884a6629c0 2024-05-22T21:23:09.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e548ca000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55884a46c0c0 con 0x55884a6632e0 2024-05-22T21:23:09.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e4b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a618ec0 0x55884a6192a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e4b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a618ec0 0x55884a6192a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42420/0 (socket says 172.21.15.161:42420) 2024-05-22T21:23:09.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e4b200640 1 -- 172.21.15.161:0/1186918663 learned_addr learned my addr 172.21.15.161:0/1186918663 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:09.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e43e00640 1 --2- 172.21.15.161:0/1186918663 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a6632e0 0x55884a618980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e4a800640 1 --2- 172.21.15.161:0/1186918663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6629c0 0x55884a662da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e4b200640 1 -- 172.21.15.161:0/1186918663 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a6632e0 msgr2=0x55884a618980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:09.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e4b200640 1 --2- 172.21.15.161:0/1186918663 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a6632e0 0x55884a618980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e4b200640 1 -- 172.21.15.161:0/1186918663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6629c0 msgr2=0x55884a662da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:09.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e4b200640 1 --2- 172.21.15.161:0/1186918663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6629c0 0x55884a662da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e4b200640 1 -- 172.21.15.161:0/1186918663 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55884a163890 con 0x55884a618ec0 2024-05-22T21:23:09.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e43e00640 1 --2- 172.21.15.161:0/1186918663 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a6632e0 0x55884a618980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:23:09.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e4b200640 1 --2- 172.21.15.161:0/1186918663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a618ec0 0x55884a6192a0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f0e440af040 tx=0x7f0e44004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861f376abe63d967 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:09.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e3b400640 1 -- 172.21.15.161:0/1186918663 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e440a5cf0 con 0x55884a618ec0 2024-05-22T21:23:09.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e3b400640 1 -- 172.21.15.161:0/1186918663 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e440a5e50 con 0x55884a618ec0 2024-05-22T21:23:09.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.656+0000 7f0e3b400640 1 -- 172.21.15.161:0/1186918663 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e440b96b0 con 0x55884a618ec0 2024-05-22T21:23:09.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 -- 172.21.15.161:0/1186918663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a618ec0 msgr2=0x55884a6192a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:09.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 --2- 172.21.15.161:0/1186918663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a618ec0 0x55884a6192a0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f0e440af040 tx=0x7f0e44004ac0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 -- 172.21.15.161:0/1186918663 shutdown_connections 2024-05-22T21:23:09.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 --2- 172.21.15.161:0/1186918663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a618ec0 0x55884a6192a0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 --2- 172.21.15.161:0/1186918663 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a6632e0 0x55884a618980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 --2- 172.21.15.161:0/1186918663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6629c0 0x55884a662da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 -- 172.21.15.161:0/1186918663 >> 172.21.15.161:0/1186918663 conn(0x55884a61f440 msgr2=0x55884a617260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:09.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 -- 172.21.15.161:0/1186918663 shutdown_connections 2024-05-22T21:23:09.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 -- 172.21.15.161:0/1186918663 wait complete. 2024-05-22T21:23:09.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 -- start start 2024-05-22T21:23:09.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 Processor -- start 2024-05-22T21:23:09.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a618ec0 0x55884a669b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6629c0 0x55884a66a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6632e0 0x55884a623fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884a4679b0 con 0x55884a6629c0 2024-05-22T21:23:09.664 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884a406c00 con 0x55884a6632e0 2024-05-22T21:23:09.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55884a425070 con 0x55884a618ec0 2024-05-22T21:23:09.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 10 client.0 _ll_register_callbacks cb 0x55884a604ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:23:09.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6629c0 0x55884a66a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6629c0 0x55884a66a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42428/0 (socket says 172.21.15.161:42428) 2024-05-22T21:23:09.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e43e00640 1 -- 172.21.15.161:0/3239687605 learned_addr learned my addr 172.21.15.161:0/3239687605 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:09.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e4a800640 1 --2- 172.21.15.161:0/3239687605 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a618ec0 0x55884a669b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e4b200640 1 --2- 172.21.15.161:0/3239687605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6632e0 0x55884a623fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e43e00640 1 -- 172.21.15.161:0/3239687605 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a618ec0 msgr2=0x55884a669b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:09.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e43e00640 1 --2- 172.21.15.161:0/3239687605 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a618ec0 0x55884a669b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e43e00640 1 -- 172.21.15.161:0/3239687605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6632e0 msgr2=0x55884a623fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:09.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e43e00640 1 --2- 172.21.15.161:0/3239687605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6632e0 0x55884a623fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e43e00640 1 -- 172.21.15.161:0/3239687605 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4c15f850 con 0x55884a6629c0 2024-05-22T21:23:09.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e43e00640 1 --2- 172.21.15.161:0/3239687605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6629c0 0x55884a66a050 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400c450 tx=0x7f0e340582f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4926a3636ac876e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:09.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:23:09.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e2fe00640 1 -- 172.21.15.161:0/3239687605 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34053870 con 0x55884a6629c0 2024-05-22T21:23:09.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e2fe00640 1 -- 172.21.15.161:0/3239687605 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e3407ce30 con 0x55884a6629c0 2024-05-22T21:23:09.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 -- 172.21.15.161:0/3239687605 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55884a4245f0 con 0x55884a6629c0 2024-05-22T21:23:09.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e2fe00640 1 -- 172.21.15.161:0/3239687605 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e340539d0 con 0x55884a6629c0 2024-05-22T21:23:09.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 10 client.8529 fetch_fsmap learned FSMap version 78 2024-05-22T21:23:09.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.660+0000 7f0e548ca000 1 -- 172.21.15.161:0/3239687605 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55884a42cab0 con 0x55884a6629c0 2024-05-22T21:23:09.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e2fe00640 1 -- 172.21.15.161:0/3239687605 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3406f020 con 0x55884a6629c0 2024-05-22T21:23:09.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e2fe00640 1 -- 172.21.15.161:0/3239687605 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 78) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0e3406b030 con 0x55884a6629c0 2024-05-22T21:23:09.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 10 client.8529 fetch_fsmap finished waiting for FSMap version 78 2024-05-22T21:23:09.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 10 client.8529 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-22T21:23:09.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 20 client.8529 populate_metadata read hostname 'smithi161' 2024-05-22T21:23:09.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 1 --2- 172.21.15.161:0/3239687605 >> [v2:172.21.15.161:6835/2279022192,v1:172.21.15.161:6837/2279022192] conn(0x55884a624d50 0x55884a625130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 4 client.8529 mds_command: new command op to 8394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T21:23:09.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 1 -- 172.21.15.161:0/3239687605 --> [v2:172.21.15.161:6835/2279022192,v1:172.21.15.161:6837/2279022192] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55884a464790 con 0x55884a624d50 2024-05-22T21:23:09.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e4a800640 1 --2- 172.21.15.161:0/3239687605 >> [v2:172.21.15.161:6835/2279022192,v1:172.21.15.161:6837/2279022192] conn(0x55884a624d50 0x55884a625130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e4a800640 1 --2- 172.21.15.161:0/3239687605 >> [v2:172.21.15.161:6835/2279022192,v1:172.21.15.161:6837/2279022192] conn(0x55884a624d50 0x55884a625130 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61af7cd509d466c server_cookie=fbcfdc3f27cdfd4a in_seq=0 out_seq=0 2024-05-22T21:23:09.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e2fe00640 10 client.8529 ms_handle_connect on v2:172.21.15.161:6835/2279022192 2024-05-22T21:23:09.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e2fe00640 1 -- 172.21.15.161:0/3239687605 <== mds.0 v2:172.21.15.161:6835/2279022192 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55884a464790 con 0x55884a624d50 2024-05-22T21:23:09.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e2fe00640 10 client.8529 handle_command_reply: tid=0 2024-05-22T21:23:09.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 1 client.8529 shutdown 2024-05-22T21:23:09.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 1 -- 172.21.15.161:0/3239687605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6629c0 msgr2=0x55884a66a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:09.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 1 --2- 172.21.15.161:0/3239687605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6629c0 0x55884a66a050 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400c450 tx=0x7f0e340582f0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 1 -- 172.21.15.161:0/3239687605 shutdown_connections 2024-05-22T21:23:09.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 1 --2- 172.21.15.161:0/3239687605 >> [v2:172.21.15.161:6835/2279022192,v1:172.21.15.161:6837/2279022192] conn(0x55884a624d50 0x55884a625130 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 1 --2- 172.21.15.161:0/3239687605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6632e0 0x55884a623fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 1 --2- 172.21.15.161:0/3239687605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6629c0 0x55884a66a050 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 1 --2- 172.21.15.161:0/3239687605 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a618ec0 0x55884a669b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 1 -- 172.21.15.161:0/3239687605 >> 172.21.15.161:0/3239687605 conn(0x55884a61f440 msgr2=0x55884a617960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:09.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e2fe00640 0 client.8529 ms_handle_reset on v2:172.21.15.161:6835/2279022192 2024-05-22T21:23:09.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 1 -- 172.21.15.161:0/3239687605 shutdown_connections 2024-05-22T21:23:09.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 1 -- 172.21.15.161:0/3239687605 wait complete. 2024-05-22T21:23:09.668 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.664+0000 7f0e548ca000 20 client.8529 trim_cache size 0 max 16384 2024-05-22T21:23:09.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e548ca000 1 Processor -- start 2024-05-22T21:23:09.680 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e548ca000 1 -- start start 2024-05-22T21:23:09.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e548ca000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a618ec0 0x55884a6c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e548ca000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6632e0 0x55884a6c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e548ca000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6c7850 0x55884a61d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e548ca000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884a124940 con 0x55884a6632e0 2024-05-22T21:23:09.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e548ca000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884a464db0 con 0x55884a6c7850 2024-05-22T21:23:09.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e548ca000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55884a46a510 con 0x55884a618ec0 2024-05-22T21:23:09.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6632e0 0x55884a6c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6632e0 0x55884a6c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42438/0 (socket says 172.21.15.161:42438) 2024-05-22T21:23:09.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e43e00640 1 -- 172.21.15.161:0/67060165 learned_addr learned my addr 172.21.15.161:0/67060165 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:09.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e4b200640 1 --2- 172.21.15.161:0/67060165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6c7850 0x55884a61d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e4a800640 1 --2- 172.21.15.161:0/67060165 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a618ec0 0x55884a6c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e43e00640 1 -- 172.21.15.161:0/67060165 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a618ec0 msgr2=0x55884a6c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:09.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e43e00640 1 --2- 172.21.15.161:0/67060165 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a618ec0 0x55884a6c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e43e00640 1 -- 172.21.15.161:0/67060165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6c7850 msgr2=0x55884a61d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:09.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e43e00640 1 --2- 172.21.15.161:0/67060165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6c7850 0x55884a61d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e43e00640 1 -- 172.21.15.161:0/67060165 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55884a412c00 con 0x55884a6632e0 2024-05-22T21:23:09.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e43e00640 1 --2- 172.21.15.161:0/67060165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6632e0 0x55884a6c7310 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f0e34052970 tx=0x7f0e34083050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb974dbca6b7b68 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:09.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e3be00640 1 -- 172.21.15.161:0/67060165 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3400e040 con 0x55884a6632e0 2024-05-22T21:23:09.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e3be00640 1 -- 172.21.15.161:0/67060165 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3406f040 con 0x55884a6632e0 2024-05-22T21:23:09.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.680+0000 7f0e3be00640 1 -- 172.21.15.161:0/67060165 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34053a70 con 0x55884a6632e0 2024-05-22T21:23:09.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 -- 172.21.15.161:0/67060165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6632e0 msgr2=0x55884a6c7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:09.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 --2- 172.21.15.161:0/67060165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6632e0 0x55884a6c7310 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f0e34052970 tx=0x7f0e34083050 comp rx=0 tx=0).stop 2024-05-22T21:23:09.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 -- 172.21.15.161:0/67060165 shutdown_connections 2024-05-22T21:23:09.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 --2- 172.21.15.161:0/67060165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6c7850 0x55884a61d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 --2- 172.21.15.161:0/67060165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6632e0 0x55884a6c7310 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 --2- 172.21.15.161:0/67060165 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a618ec0 0x55884a6c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 -- 172.21.15.161:0/67060165 >> 172.21.15.161:0/67060165 conn(0x55884a61f440 msgr2=0x55884a660f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:09.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 -- 172.21.15.161:0/67060165 shutdown_connections 2024-05-22T21:23:09.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 -- 172.21.15.161:0/67060165 wait complete. 2024-05-22T21:23:09.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 -- start start 2024-05-22T21:23:09.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 Processor -- start 2024-05-22T21:23:09.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a618ec0 0x55884a6b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6c7850 0x55884a6b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6b4840 0x55884a6b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884a4631a0 con 0x55884a6b4840 2024-05-22T21:23:09.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55884a6106d0 con 0x55884a6c7850 2024-05-22T21:23:09.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55884a46b9b0 con 0x55884a618ec0 2024-05-22T21:23:09.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 10 client.0 _ll_register_callbacks cb 0x7f0e44009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:23:09.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e4b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6b4840 0x55884a6b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e4b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6b4840 0x55884a6b4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42454/0 (socket says 172.21.15.161:42454) 2024-05-22T21:23:09.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e4b200640 1 -- 172.21.15.161:0/3776152216 learned_addr learned my addr 172.21.15.161:0/3776152216 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:09.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e43e00640 1 --2- 172.21.15.161:0/3776152216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6c7850 0x55884a6b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e4a800640 1 --2- 172.21.15.161:0/3776152216 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a618ec0 0x55884a6b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e4b200640 1 -- 172.21.15.161:0/3776152216 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a618ec0 msgr2=0x55884a6b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:09.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e4b200640 1 --2- 172.21.15.161:0/3776152216 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a618ec0 0x55884a6b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.686 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e4b200640 1 -- 172.21.15.161:0/3776152216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6c7850 msgr2=0x55884a6b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:09.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e4b200640 1 --2- 172.21.15.161:0/3776152216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6c7850 0x55884a6b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e4b200640 1 -- 172.21.15.161:0/3776152216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55884a42cab0 con 0x55884a6b4840 2024-05-22T21:23:09.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e4b200640 1 --2- 172.21.15.161:0/3776152216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6b4840 0x55884a6b4c20 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f0e4401ca30 tx=0x7f0e4401b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41de7dcf64b294ff server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:09.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:23:09.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e2fe00640 1 -- 172.21.15.161:0/3776152216 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e440cb070 con 0x55884a6b4840 2024-05-22T21:23:09.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e2fe00640 1 -- 172.21.15.161:0/3776152216 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e440039a0 con 0x55884a6b4840 2024-05-22T21:23:09.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 -- 172.21.15.161:0/3776152216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55884a437460 con 0x55884a6b4840 2024-05-22T21:23:09.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e2fe00640 1 -- 172.21.15.161:0/3776152216 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e44003ba0 con 0x55884a6b4840 2024-05-22T21:23:09.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 10 client.8535 fetch_fsmap learned FSMap version 78 2024-05-22T21:23:09.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 -- 172.21.15.161:0/3776152216 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55884a163cc0 con 0x55884a6b4840 2024-05-22T21:23:09.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e2fe00640 1 -- 172.21.15.161:0/3776152216 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55884a437460 con 0x55884a6b4840 2024-05-22T21:23:09.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e2fe00640 1 -- 172.21.15.161:0/3776152216 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 78) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0e440ce020 con 0x55884a6b4840 2024-05-22T21:23:09.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 10 client.8535 fetch_fsmap finished waiting for FSMap version 78 2024-05-22T21:23:09.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 10 client.8535 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-22T21:23:09.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 20 client.8535 populate_metadata read hostname 'smithi161' 2024-05-22T21:23:09.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 --2- 172.21.15.161:0/3776152216 >> [v2:172.21.15.161:6835/2279022192,v1:172.21.15.161:6837/2279022192] conn(0x55884a6d2cb0 0x55884a6d50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 4 client.8535 mds_command: new command op to 8394 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-22T21:23:09.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.684+0000 7f0e548ca000 1 -- 172.21.15.161:0/3776152216 --> [v2:172.21.15.161:6835/2279022192,v1:172.21.15.161:6837/2279022192] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55884a4309c0 con 0x55884a6d2cb0 2024-05-22T21:23:09.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e4a800640 1 --2- 172.21.15.161:0/3776152216 >> [v2:172.21.15.161:6835/2279022192,v1:172.21.15.161:6837/2279022192] conn(0x55884a6d2cb0 0x55884a6d50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e4a800640 1 --2- 172.21.15.161:0/3776152216 >> [v2:172.21.15.161:6835/2279022192,v1:172.21.15.161:6837/2279022192] conn(0x55884a6d2cb0 0x55884a6d50a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3faa60d69700045 server_cookie=b960699f6bedf1f2 in_seq=0 out_seq=0 2024-05-22T21:23:09.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e2fe00640 10 client.8535 ms_handle_connect on v2:172.21.15.161:6835/2279022192 2024-05-22T21:23:09.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e2fe00640 1 -- 172.21.15.161:0/3776152216 <== mds.0 v2:172.21.15.161:6835/2279022192 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55884a4309c0 con 0x55884a6d2cb0 2024-05-22T21:23:09.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e2fe00640 10 client.8535 handle_command_reply: tid=0 2024-05-22T21:23:09.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e548ca000 1 client.8535 shutdown 2024-05-22T21:23:09.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e548ca000 1 -- 172.21.15.161:0/3776152216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6b4840 msgr2=0x55884a6b4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:09.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e548ca000 1 --2- 172.21.15.161:0/3776152216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6b4840 0x55884a6b4c20 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f0e4401ca30 tx=0x7f0e4401b230 comp rx=0 tx=0).stop 2024-05-22T21:23:09.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e548ca000 1 -- 172.21.15.161:0/3776152216 shutdown_connections 2024-05-22T21:23:09.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e548ca000 1 --2- 172.21.15.161:0/3776152216 >> [v2:172.21.15.161:6835/2279022192,v1:172.21.15.161:6837/2279022192] conn(0x55884a6d2cb0 0x55884a6d50a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e548ca000 1 --2- 172.21.15.161:0/3776152216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55884a6b4840 0x55884a6b4c20 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e548ca000 1 --2- 172.21.15.161:0/3776152216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55884a6c7850 0x55884a6b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e2fe00640 0 client.8535 ms_handle_reset on v2:172.21.15.161:6835/2279022192 2024-05-22T21:23:09.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e548ca000 1 --2- 172.21.15.161:0/3776152216 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55884a618ec0 0x55884a6b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e548ca000 1 -- 172.21.15.161:0/3776152216 >> 172.21.15.161:0/3776152216 conn(0x55884a61f440 msgr2=0x55884a6617c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:09.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e548ca000 1 -- 172.21.15.161:0/3776152216 shutdown_connections 2024-05-22T21:23:09.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e548ca000 1 -- 172.21.15.161:0/3776152216 wait complete. 2024-05-22T21:23:09.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e548ca000 20 client.8535 trim_cache size 0 max 16384 2024-05-22T21:23:09.691 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:23:09.691 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:23:09.691 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 0, 2024-05-22T21:23:09.691 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 8394, 2024-05-22T21:23:09.691 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2024-05-22T21:23:09.691 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2024-05-22T21:23:09.692 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2024-05-22T21:23:09.692 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 5.8237585620000001, 2024-05-22T21:23:09.692 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 78, 2024-05-22T21:23:09.692 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 175, 2024-05-22T21:23:09.692 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 175, 2024-05-22T21:23:09.692 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 10.257072245 2024-05-22T21:23:09.692 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:23:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e51400640 1 -- 172.21.15.161:0/2588745682 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e28066690 msgr2=0x7f0e28068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e51400640 1 --2- 172.21.15.161:0/2588745682 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e28066690 0x7f0e28068b50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c008930 tx=0x7f0e3c008040 comp rx=0 tx=0).stop 2024-05-22T21:23:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e51400640 1 -- 172.21.15.161:0/2588745682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4c143c40 msgr2=0x7f0e4c154810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e51400640 1 --2- 172.21.15.161:0/2588745682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4c143c40 0x7f0e4c154810 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400c480 tx=0x7f0e3400c940 comp rx=0 tx=0).stop 2024-05-22T21:23:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e51400640 1 -- 172.21.15.161:0/2588745682 shutdown_connections 2024-05-22T21:23:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e51400640 1 --2- 172.21.15.161:0/2588745682 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e28066690 0x7f0e28068b50 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e51400640 1 --2- 172.21.15.161:0/2588745682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e4c1444a0 0x7f0e4c06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e51400640 1 --2- 172.21.15.161:0/2588745682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e4c143c40 0x7f0e4c154810 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e51400640 1 --2- 172.21.15.161:0/2588745682 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e4c06c9b0 0x7f0e4c0ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e51400640 1 -- 172.21.15.161:0/2588745682 >> 172.21.15.161:0/2588745682 conn(0x7f0e4c06a2d0 msgr2=0x7f0e4c157670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e51400640 1 -- 172.21.15.161:0/2588745682 shutdown_connections 2024-05-22T21:23:09.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.688+0000 7f0e51400640 1 -- 172.21.15.161:0/2588745682 wait complete. 2024-05-22T21:23:09.707 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:23:09.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.840+0000 7fac6ec00640 1 Processor -- start 2024-05-22T21:23:09.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.840+0000 7fac6ec00640 1 -- start start 2024-05-22T21:23:09.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.844+0000 7fac6ec00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fac6815e130 0x7fac6815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.844+0000 7fac6ec00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac6815d410 0x7fac6815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.844+0000 7fac6ec00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac68071670 con 0x7fac6815dd50 2024-05-22T21:23:09.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.844+0000 7fac6ec00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6815d870 con 0x7fac6815e130 2024-05-22T21:23:09.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.844+0000 7fac6ec00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac68070a60 con 0x7fac6815d410 2024-05-22T21:23:09.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.844+0000 7fac67400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fac6815e130 0x7fac6815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.844+0000 7fac67400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fac6815e130 0x7fac6815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42456/0 (socket says 172.21.15.161:42456) 2024-05-22T21:23:09.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.844+0000 7fac67400640 1 -- 172.21.15.161:0/1794215985 learned_addr learned my addr 172.21.15.161:0/1794215985 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:09.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.844+0000 7fac6cc00640 1 --2- 172.21.15.161:0/1794215985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac6815d410 0x7fac6815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.844+0000 7fac67400640 1 -- 172.21.15.161:0/1794215985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac6815d410 msgr2=0x7fac6815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:09.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.844+0000 7fac67400640 1 --2- 172.21.15.161:0/1794215985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac6815d410 0x7fac6815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.844+0000 7fac67400640 1 -- 172.21.15.161:0/1794215985 >> v1:172.21.15.179:6789/0 conn(0x7fac6815dd50 legacy=0x7fac6806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:09.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.844+0000 7fac67400640 1 -- 172.21.15.161:0/1794215985 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac68070d40 con 0x7fac6815e130 2024-05-22T21:23:09.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.848+0000 7fac67400640 1 --2- 172.21.15.161:0/1794215985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6815e130 0x7fac6815ced0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fac54009870 tx=0x7fac5401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f5e6710602d0ed server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:09.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.848+0000 7fac66a00640 1 -- 172.21.15.161:0/1794215985 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac540a95f0 con 0x7fac6815e130 2024-05-22T21:23:09.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.848+0000 7fac66a00640 1 -- 172.21.15.161:0/1794215985 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac540a9750 con 0x7fac6815e130 2024-05-22T21:23:09.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.848+0000 7fac66a00640 1 -- 172.21.15.161:0/1794215985 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac540a9ae0 con 0x7fac6815e130 2024-05-22T21:23:09.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.848+0000 7fac6ec00640 1 -- 172.21.15.161:0/1794215985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6815e130 msgr2=0x7fac6815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:09.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.848+0000 7fac6ec00640 1 --2- 172.21.15.161:0/1794215985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6815e130 0x7fac6815ced0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fac54009870 tx=0x7fac5401c530 comp rx=0 tx=0).stop 2024-05-22T21:23:09.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.848+0000 7fac6ec00640 1 -- 172.21.15.161:0/1794215985 shutdown_connections 2024-05-22T21:23:09.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.848+0000 7fac6ec00640 1 --2- 172.21.15.161:0/1794215985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac6815d410 0x7fac6815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.848+0000 7fac6ec00640 1 --2- 172.21.15.161:0/1794215985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6815e130 0x7fac6815ced0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.848+0000 7fac6ec00640 1 -- 172.21.15.161:0/1794215985 >> 172.21.15.161:0/1794215985 conn(0x7fac680b0bc0 msgr2=0x7fac680b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:09.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.848+0000 7fac6ec00640 1 -- 172.21.15.161:0/1794215985 shutdown_connections 2024-05-22T21:23:09.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.848+0000 7fac6ec00640 1 -- 172.21.15.161:0/1794215985 wait complete. 2024-05-22T21:23:09.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.848+0000 7fac6ec00640 1 Processor -- start 2024-05-22T21:23:09.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.848+0000 7fac6ec00640 1 -- start start 2024-05-22T21:23:09.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.848+0000 7fac6ec00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac6815d410 0x7fac6814cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6ec00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac6815dd50 0x7fac680afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6ec00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6815e130 0x7fac680b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6ec00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac680b3d30 con 0x7fac6815e130 2024-05-22T21:23:09.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6ec00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac680b31b0 con 0x7fac6815dd50 2024-05-22T21:23:09.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6ec00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac680b3490 con 0x7fac6815d410 2024-05-22T21:23:09.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6cc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6815e130 0x7fac680b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6cc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6815e130 0x7fac680b0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42468/0 (socket says 172.21.15.161:42468) 2024-05-22T21:23:09.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6cc00640 1 -- 172.21.15.161:0/322169852 learned_addr learned my addr 172.21.15.161:0/322169852 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:09.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac67e00640 1 --2- 172.21.15.161:0/322169852 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac6815d410 0x7fac6814cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac67400640 1 --2- 172.21.15.161:0/322169852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac6815dd50 0x7fac680afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6cc00640 1 -- 172.21.15.161:0/322169852 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac6815d410 msgr2=0x7fac6814cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:09.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6cc00640 1 --2- 172.21.15.161:0/322169852 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac6815d410 0x7fac6814cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6cc00640 1 -- 172.21.15.161:0/322169852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac6815dd50 msgr2=0x7fac680afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:09.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6cc00640 1 --2- 172.21.15.161:0/322169852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac6815dd50 0x7fac680afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:09.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6cc00640 1 -- 172.21.15.161:0/322169852 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac540094d0 con 0x7fac6815e130 2024-05-22T21:23:09.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6cc00640 1 --2- 172.21.15.161:0/322169852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6815e130 0x7fac680b0260 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fac6000d800 tx=0x7fac6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:09.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac5fe00640 1 -- 172.21.15.161:0/322169852 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6000b6c0 con 0x7fac6815e130 2024-05-22T21:23:09.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac5fe00640 1 -- 172.21.15.161:0/322169852 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6000b820 con 0x7fac6815e130 2024-05-22T21:23:09.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac5fe00640 1 -- 172.21.15.161:0/322169852 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6000bbd0 con 0x7fac6815e130 2024-05-22T21:23:09.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6ec00640 1 -- 172.21.15.161:0/322169852 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac680b3770 con 0x7fac6815e130 2024-05-22T21:23:09.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6ec00640 1 -- 172.21.15.161:0/322169852 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac680b2dd0 con 0x7fac6815e130 2024-05-22T21:23:09.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac6ec00640 1 -- 172.21.15.161:0/322169852 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac68067220 con 0x7fac6815e130 2024-05-22T21:23:09.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.852+0000 7fac5fe00640 1 -- 172.21.15.161:0/322169852 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fac6000bd30 con 0x7fac6815e130 2024-05-22T21:23:09.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.856+0000 7fac5fe00640 1 --2- 172.21.15.161:0/322169852 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac48066690 0x7fac48068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:09.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.856+0000 7fac5fe00640 1 -- 172.21.15.161:0/322169852 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac60084640 con 0x7fac6815e130 2024-05-22T21:23:09.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.856+0000 7fac67e00640 1 --2- 172.21.15.161:0/322169852 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac48066690 0x7fac48068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:09.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.856+0000 7fac67e00640 1 --2- 172.21.15.161:0/322169852 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac48066690 0x7fac48068b50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fac580012f0 tx=0x7fac58009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:09.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:09.856+0000 7fac5fe00640 1 -- 172.21.15.161:0/322169852 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fac600519c0 con 0x7fac6815e130 2024-05-22T21:23:10.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.048+0000 7fac6ec00640 1 -- 172.21.15.161:0/322169852 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fac68154720 con 0x7fac6815e130 2024-05-22T21:23:10.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.048+0000 7fac5fe00640 1 -- 172.21.15.161:0/322169852 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) v1 ==== 53+0+73 (secure 0 0 0) 0x7fac60054fe0 con 0x7fac6815e130 2024-05-22T21:23:10.050 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:23:10.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.052+0000 7fac6ec00640 1 -- 172.21.15.161:0/322169852 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac48066690 msgr2=0x7fac48068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:10.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.052+0000 7fac6ec00640 1 --2- 172.21.15.161:0/322169852 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac48066690 0x7fac48068b50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fac580012f0 tx=0x7fac58009040 comp rx=0 tx=0).stop 2024-05-22T21:23:10.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.052+0000 7fac6ec00640 1 -- 172.21.15.161:0/322169852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6815e130 msgr2=0x7fac680b0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:10.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.052+0000 7fac6ec00640 1 --2- 172.21.15.161:0/322169852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6815e130 0x7fac680b0260 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fac6000d800 tx=0x7fac6000dcc0 comp rx=0 tx=0).stop 2024-05-22T21:23:10.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.052+0000 7fac6ec00640 1 -- 172.21.15.161:0/322169852 shutdown_connections 2024-05-22T21:23:10.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.052+0000 7fac6ec00640 1 --2- 172.21.15.161:0/322169852 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fac48066690 0x7fac48068b50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:10.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.052+0000 7fac6ec00640 1 --2- 172.21.15.161:0/322169852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac6815e130 0x7fac680b0260 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:10.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.052+0000 7fac6ec00640 1 --2- 172.21.15.161:0/322169852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac6815dd50 0x7fac680afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:10.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.052+0000 7fac6ec00640 1 --2- 172.21.15.161:0/322169852 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fac6815d410 0x7fac6814cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:10.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.052+0000 7fac6ec00640 1 -- 172.21.15.161:0/322169852 >> 172.21.15.161:0/322169852 conn(0x7fac680b0bc0 msgr2=0x7fac680acf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:10.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.052+0000 7fac6ec00640 1 -- 172.21.15.161:0/322169852 shutdown_connections 2024-05-22T21:23:10.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.052+0000 7fac6ec00640 1 -- 172.21.15.161:0/322169852 wait complete. 2024-05-22T21:23:10.070 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:23:10.071 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:23:10.071 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-05-22T21:23:10.071 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-22T21:23:10.071 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-22T21:23:10.071 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:23:10.071 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:23:10.071 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:23:10.071 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:23:10.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:10.071 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-05-22T21:23:10.080 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:23:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:23:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:23:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:23:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-05-22T21:23:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:23:10.080 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:23:10.080 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-05-22T21:23:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2024-05-22T21:23:10.081 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:23:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:23:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2024-05-22T21:23:10.081 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:23:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:23:10.081 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:23:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:23:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:23:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 41902sec preferred_lft 41902sec 2024-05-22T21:23:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-05-22T21:23:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:23:10.081 INFO:teuthology.orchestra.run.smithi161.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:23:10.081 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 02:42:b9:0f:57:df brd ff:ff:ff:ff:ff:ff 2024-05-22T21:23:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:23:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:23:10.082 INFO:teuthology.orchestra.run.smithi161.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:23:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:23:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:23:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:23:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:23:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:23:10.082 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:23:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-22T21:23:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:23:10.082 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:23:10.082 INFO:teuthology.orchestra.run.smithi161.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:23:10.083 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-22T21:23:10.083 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:23:10.083 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:23:10.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:10.084 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:23:10.143 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-05-22T21:23:10.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:10.145 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-05-22T21:23:10.196 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-05-22T21:23:10.196 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:23:10.197 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:23:10.197 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:23:10.197 INFO:teuthology.orchestra.run.smithi161.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:23:10.197 INFO:teuthology.orchestra.run.smithi161.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:23:10.197 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:23:10.197 INFO:teuthology.orchestra.run.smithi161.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:23:10.197 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:23:10.197 INFO:teuthology.orchestra.run.smithi161.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:23:10.197 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:23:10.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:10.198 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:23:10.198 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:23:10.198 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:23:10.198 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:23:10.198 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:23:10.198 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:23:10.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:10.286 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-05-22T21:23:10.342 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-22T21:23:10.342 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-22T21:23:10.342 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:23:10.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:10.343 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2024-05-22T21:23:10.405 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:23:10.406 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-22T21:23:10.458 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:23:10.458 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:23:10.458 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:23:10.509 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:23:10.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:10.509 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:23:10.602 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:23:10.602 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:23:10.602 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:23:10.617 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:23:10.617 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:23:10.617 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:23:10.617 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:23:10.617 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:23:10.625 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:23:10.625 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:23:10.625 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:23:10.625 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:23:10.625 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:23:10.625 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:23:10.626 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:23:10.626 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:23:10.629 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:23:10.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:10.629 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:23:10.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:10.649 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:23:10.709 INFO:teuthology.orchestra.run.smithi161.stdout:client8044 2024-05-22T21:23:10.709 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:23:10.709 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:23:10.758 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:23:10.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.936+0000 7f1088000640 1 Processor -- start 2024-05-22T21:23:10.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.936+0000 7f1088000640 1 -- start start 2024-05-22T21:23:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.940+0000 7f1088000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f108015e130 0x7f108015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:10.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.940+0000 7f1088000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f108015d410 0x7f108015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:10.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.940+0000 7f1088000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1080071670 con 0x7f108015dd50 2024-05-22T21:23:10.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.940+0000 7f1088000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108015d870 con 0x7f108015e130 2024-05-22T21:23:10.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.940+0000 7f1088000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1080070a60 con 0x7f108015d410 2024-05-22T21:23:10.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.940+0000 7f1084c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f108015e130 0x7f108015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:10.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.940+0000 7f1084c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f108015e130 0x7f108015ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42482/0 (socket says 172.21.15.161:42482) 2024-05-22T21:23:10.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.940+0000 7f1084c00640 1 -- 172.21.15.161:0/2825246146 learned_addr learned my addr 172.21.15.161:0/2825246146 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:10.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.940+0000 7f1085600640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f108015dd50 0x7f108006d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:40766/0 (socket says 172.21.15.161:40766) 2024-05-22T21:23:10.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.940+0000 7f1086000640 1 --2- 172.21.15.161:0/2825246146 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f108015d410 0x7f108015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:10.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.940+0000 7f1084c00640 1 -- 172.21.15.161:0/2825246146 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f108015d410 msgr2=0x7f108015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:10.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.940+0000 7f1084c00640 1 --2- 172.21.15.161:0/2825246146 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f108015d410 0x7f108015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:10.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.940+0000 7f1084c00640 1 -- 172.21.15.161:0/2825246146 >> v1:172.21.15.161:6789/0 conn(0x7f108015dd50 legacy=0x7f108006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:10.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.940+0000 7f1084c00640 1 -- 172.21.15.161:0/2825246146 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1080070d40 con 0x7f108015e130 2024-05-22T21:23:10.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.940+0000 7f1084c00640 1 --2- 172.21.15.161:0/2825246146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108015e130 0x7f108015ced0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f106c009870 tx=0x7f106c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32247beb2a1ed17 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:10.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f107fe00640 1 -- 172.21.15.161:0/2825246146 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c0a95f0 con 0x7f108015e130 2024-05-22T21:23:10.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f107fe00640 1 -- 172.21.15.161:0/2825246146 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c0a9750 con 0x7f108015e130 2024-05-22T21:23:10.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f107fe00640 1 -- 172.21.15.161:0/2825246146 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c0a9ae0 con 0x7f108015e130 2024-05-22T21:23:10.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1088000640 1 -- 172.21.15.161:0/2825246146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108015e130 msgr2=0x7f108015ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:10.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1088000640 1 --2- 172.21.15.161:0/2825246146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108015e130 0x7f108015ced0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f106c009870 tx=0x7f106c01c530 comp rx=0 tx=0).stop 2024-05-22T21:23:10.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1088000640 1 -- 172.21.15.161:0/2825246146 shutdown_connections 2024-05-22T21:23:10.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1088000640 1 --2- 172.21.15.161:0/2825246146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108015e130 0x7f108015ced0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:10.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1088000640 1 --2- 172.21.15.161:0/2825246146 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f108015d410 0x7f108015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:10.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1088000640 1 -- 172.21.15.161:0/2825246146 >> 172.21.15.161:0/2825246146 conn(0x7f10800b0bc0 msgr2=0x7f10800b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:10.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1088000640 1 -- 172.21.15.161:0/2825246146 shutdown_connections 2024-05-22T21:23:10.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1088000640 1 -- 172.21.15.161:0/2825246146 wait complete. 2024-05-22T21:23:10.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1088000640 1 Processor -- start 2024-05-22T21:23:10.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1088000640 1 -- start start 2024-05-22T21:23:10.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1088000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f108015d410 0x7f10800b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:10.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1088000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108015dd50 0x7f108014d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:10.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1088000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f108015e130 0x7f108013e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:10.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1088000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108014dde0 con 0x7f108015dd50 2024-05-22T21:23:10.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1088000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108013edd0 con 0x7f108015e130 2024-05-22T21:23:10.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1088000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108014d4a0 con 0x7f108015d410 2024-05-22T21:23:10.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1084c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108015dd50 0x7f108014d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:10.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1084c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108015dd50 0x7f108014d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42494/0 (socket says 172.21.15.161:42494) 2024-05-22T21:23:10.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1084c00640 1 -- 172.21.15.161:0/3909689963 learned_addr learned my addr 172.21.15.161:0/3909689963 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:10.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1085600640 1 --2- 172.21.15.161:0/3909689963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f108015d410 0x7f10800b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:10.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.944+0000 7f1086000640 1 --2- 172.21.15.161:0/3909689963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f108015e130 0x7f108013e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:10.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.948+0000 7f1084c00640 1 -- 172.21.15.161:0/3909689963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f108015d410 msgr2=0x7f10800b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:10.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.948+0000 7f1084c00640 1 --2- 172.21.15.161:0/3909689963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f108015d410 0x7f10800b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:10.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.948+0000 7f1084c00640 1 -- 172.21.15.161:0/3909689963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f108015e130 msgr2=0x7f108013e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:10.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.948+0000 7f1084c00640 1 --2- 172.21.15.161:0/3909689963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f108015e130 0x7f108013e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:10.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.948+0000 7f1084c00640 1 -- 172.21.15.161:0/3909689963 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c0094d0 con 0x7f108015dd50 2024-05-22T21:23:10.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.948+0000 7f1084c00640 1 --2- 172.21.15.161:0/3909689963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108015dd50 0x7f108014d8a0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f106c01abe0 tx=0x7f106c0192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:10.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.948+0000 7f107d600640 1 -- 172.21.15.161:0/3909689963 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c0bb640 con 0x7f108015dd50 2024-05-22T21:23:10.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.948+0000 7f107d600640 1 -- 172.21.15.161:0/3909689963 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c0bb7a0 con 0x7f108015dd50 2024-05-22T21:23:10.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.948+0000 7f107d600640 1 -- 172.21.15.161:0/3909689963 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c0bbb30 con 0x7f108015dd50 2024-05-22T21:23:10.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.948+0000 7f1088000640 1 -- 172.21.15.161:0/3909689963 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10801f5e20 con 0x7f108015dd50 2024-05-22T21:23:10.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.948+0000 7f1088000640 1 -- 172.21.15.161:0/3909689963 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10801f63e0 con 0x7f108015dd50 2024-05-22T21:23:10.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.948+0000 7f1088000640 1 -- 172.21.15.161:0/3909689963 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1080067220 con 0x7f108015dd50 2024-05-22T21:23:10.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.948+0000 7f107d600640 1 -- 172.21.15.161:0/3909689963 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f106c0b13f0 con 0x7f108015dd50 2024-05-22T21:23:10.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.948+0000 7f107d600640 1 --2- 172.21.15.161:0/3909689963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1060066690 0x7f1060068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:10.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.948+0000 7f107d600640 1 -- 172.21.15.161:0/3909689963 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f106c11e630 con 0x7f108015dd50 2024-05-22T21:23:10.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.948+0000 7f1085600640 1 --2- 172.21.15.161:0/3909689963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1060066690 0x7f1060068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:10.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.952+0000 7f1085600640 1 --2- 172.21.15.161:0/3909689963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1060066690 0x7f1060068b50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1074000f70 tx=0x7f1074007e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:10.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:10.952+0000 7f107d600640 1 -- 172.21.15.161:0/3909689963 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f106c0eba40 con 0x7f108015dd50 2024-05-22T21:23:11.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.144+0000 7f1088000640 1 -- 172.21.15.161:0/3909689963 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1080154af0 con 0x7f108015dd50 2024-05-22T21:23:11.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.144+0000 7f107d600640 1 -- 172.21.15.161:0/3909689963 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) v1 ==== 53+0+73 (secure 0 0 0) 0x7f106c0ef060 con 0x7f108015dd50 2024-05-22T21:23:11.144 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:23:11.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.144+0000 7f1088000640 1 -- 172.21.15.161:0/3909689963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1060066690 msgr2=0x7f1060068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:11.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.144+0000 7f1088000640 1 --2- 172.21.15.161:0/3909689963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1060066690 0x7f1060068b50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1074000f70 tx=0x7f1074007e50 comp rx=0 tx=0).stop 2024-05-22T21:23:11.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.144+0000 7f1088000640 1 -- 172.21.15.161:0/3909689963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108015dd50 msgr2=0x7f108014d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:11.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.144+0000 7f1088000640 1 --2- 172.21.15.161:0/3909689963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108015dd50 0x7f108014d8a0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f106c01abe0 tx=0x7f106c0192d0 comp rx=0 tx=0).stop 2024-05-22T21:23:11.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.144+0000 7f1088000640 1 -- 172.21.15.161:0/3909689963 shutdown_connections 2024-05-22T21:23:11.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.144+0000 7f1088000640 1 --2- 172.21.15.161:0/3909689963 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1060066690 0x7f1060068b50 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:11.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.144+0000 7f1088000640 1 --2- 172.21.15.161:0/3909689963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f108015e130 0x7f108013e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:11.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.144+0000 7f1088000640 1 --2- 172.21.15.161:0/3909689963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f108015dd50 0x7f108014d8a0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:11.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.144+0000 7f1088000640 1 --2- 172.21.15.161:0/3909689963 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f108015d410 0x7f10800b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:11.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.144+0000 7f1088000640 1 -- 172.21.15.161:0/3909689963 >> 172.21.15.161:0/3909689963 conn(0x7f10800b0bc0 msgr2=0x7f1080159410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:11.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.144+0000 7f1088000640 1 -- 172.21.15.161:0/3909689963 shutdown_connections 2024-05-22T21:23:11.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.144+0000 7f1088000640 1 -- 172.21.15.161:0/3909689963 wait complete. 2024-05-22T21:23:11.164 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:23:11.164 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:23:11.164 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-22T21:23:11.164 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-22T21:23:11.164 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-22T21:23:11.164 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:23:11.164 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:23:11.164 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:23:11.164 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:23:11.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:11.165 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-22T21:23:11.175 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:23:11.175 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:23:11.175 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:23:11.175 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:23:11.175 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-22T21:23:11.175 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:23:11.175 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:23:11.175 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:23:11.175 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f0 2024-05-22T21:23:11.175 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:23:11.175 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:23:11.175 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f1 2024-05-22T21:23:11.175 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:23:11.176 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-22T21:23:11.176 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:23:11.176 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-22T21:23:11.176 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:23:11.176 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 41893sec preferred_lft 41893sec 2024-05-22T21:23:11.176 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-22T21:23:11.176 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:23:11.176 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:23:11.176 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:bd:4e:25:3d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:23:11.176 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:23:11.176 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:23:11.176 INFO:teuthology.orchestra.run.smithi179.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:23:11.176 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:23:11.176 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:23:11.176 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:23:11.177 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:23:11.177 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:23:11.177 INFO:teuthology.orchestra.run.smithi179.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:23:11.177 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-22T21:23:11.177 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:23:11.177 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:23:11.177 INFO:teuthology.orchestra.run.smithi179.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:23:11.177 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-22T21:23:11.177 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:23:11.177 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:23:11.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:11.178 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:23:11.242 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-22T21:23:11.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:11.244 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-22T21:23:11.298 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-22T21:23:11.299 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:23:11.299 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:23:11.299 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:23:11.299 INFO:teuthology.orchestra.run.smithi179.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:23:11.299 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:23:11.299 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:23:11.299 INFO:teuthology.orchestra.run.smithi179.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:23:11.299 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:23:11.299 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:23:11.299 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:23:11.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:11.300 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:23:11.300 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:23:11.300 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:23:11.300 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:23:11.300 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:23:11.300 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:23:11.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:11.398 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-22T21:23:11.407 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-22T21:23:11.407 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-22T21:23:11.407 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:23:11.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:11.408 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-22T21:23:11.474 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:23:11.474 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-22T21:23:11.530 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:23:11.530 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:23:11.530 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:23:11.581 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:23:11.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:11.582 DEBUG:teuthology.orchestra.run.smithi179:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:23:11.670 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:23:11.670 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:23:11.670 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:23:11.685 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:23:11.685 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:23:11.685 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:23:11.686 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:23:11.686 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:23:11.693 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:23:11.693 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:23:11.693 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:23:11.693 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:23:11.693 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:23:11.694 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:23:11.694 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:23:11.694 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:23:11.697 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:23:11.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:11.697 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:23:11.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:23:11.716 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:23:11.768 INFO:teuthology.orchestra.run.smithi179.stdout:client7055 2024-05-22T21:23:11.768 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:23:11.768 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:23:11.818 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:23:11.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.956+0000 7f4913200640 1 Processor -- start 2024-05-22T21:23:11.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.956+0000 7f4913200640 1 -- start start 2024-05-22T21:23:11.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.960+0000 7f4913200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f490c143ec0 0x7f490c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:11.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.960+0000 7f4913200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f490c0715a0 0x7f490c06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:11.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.960+0000 7f4913200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c1442a0 con 0x7f490c15ec90 2024-05-22T21:23:11.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.960+0000 7f4913200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490c070260 con 0x7f490c0715a0 2024-05-22T21:23:11.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.960+0000 7f4913200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490c070500 con 0x7f490c143ec0 2024-05-22T21:23:11.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.960+0000 7f4903400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f490c0715a0 0x7f490c06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:11.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.960+0000 7f4911200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f490c15ec90 0x7f490c06bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:40774/0 (socket says 172.21.15.161:40774) 2024-05-22T21:23:11.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.960+0000 7f4911200640 1 -- 172.21.15.161:0/2066175085 learned_addr learned my addr 172.21.15.161:0/2066175085 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:23:11.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.960+0000 7f490be00640 1 --2- 172.21.15.161:0/2066175085 >> v2:172.21.15.179:3300/0 conn(0x7f490c143ec0 0x7f490c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:11.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.960+0000 7f4903400640 1 -- 172.21.15.161:0/2066175085 >> v2:172.21.15.179:3300/0 conn(0x7f490c143ec0 msgr2=0x7f490c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:11.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.960+0000 7f4903400640 1 --2- 172.21.15.161:0/2066175085 >> v2:172.21.15.179:3300/0 conn(0x7f490c143ec0 0x7f490c15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:11.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.960+0000 7f4903400640 1 -- 172.21.15.161:0/2066175085 >> v1:172.21.15.161:6789/0 conn(0x7f490c15ec90 legacy=0x7f490c06bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:11.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.960+0000 7f4903400640 1 -- 172.21.15.161:0/2066175085 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c0707e0 con 0x7f490c0715a0 2024-05-22T21:23:11.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.960+0000 7f490b400640 1 -- 172.21.15.161:0/2066175085 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908691531 0 0) 0x7f490c1442a0 con 0x7f490c15ec90 2024-05-22T21:23:11.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4903400640 1 --2- 172.21.15.161:0/2066175085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f490c0715a0 0x7f490c06fd20 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f490c0724e0 tx=0x7f48f401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4b05dbbbb1606c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:11.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f490b400640 1 -- 172.21.15.161:0/2066175085 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f40ab650 con 0x7f490c0715a0 2024-05-22T21:23:11.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f490b400640 1 -- 172.21.15.161:0/2066175085 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f40ab7b0 con 0x7f490c0715a0 2024-05-22T21:23:11.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f490b400640 1 -- 172.21.15.161:0/2066175085 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f40abb40 con 0x7f490c0715a0 2024-05-22T21:23:11.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4913200640 1 -- 172.21.15.161:0/2066175085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f490c0715a0 msgr2=0x7f490c06fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:11.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4913200640 1 --2- 172.21.15.161:0/2066175085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f490c0715a0 0x7f490c06fd20 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f490c0724e0 tx=0x7f48f401c450 comp rx=0 tx=0).stop 2024-05-22T21:23:11.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4913200640 1 -- 172.21.15.161:0/2066175085 shutdown_connections 2024-05-22T21:23:11.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4913200640 1 --2- 172.21.15.161:0/2066175085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f490c0715a0 0x7f490c06fd20 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:11.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4913200640 1 --2- 172.21.15.161:0/2066175085 >> v2:172.21.15.179:3300/0 conn(0x7f490c143ec0 0x7f490c15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:11.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4913200640 1 -- 172.21.15.161:0/2066175085 >> 172.21.15.161:0/2066175085 conn(0x7f490c0b0bc0 msgr2=0x7f490c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:11.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4913200640 1 -- 172.21.15.161:0/2066175085 shutdown_connections 2024-05-22T21:23:11.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4913200640 1 -- 172.21.15.161:0/2066175085 wait complete. 2024-05-22T21:23:11.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4913200640 1 Processor -- start 2024-05-22T21:23:11.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4913200640 1 -- start start 2024-05-22T21:23:11.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4913200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f490c143ec0 0x7f490c163060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:11.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4913200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f490c15ec90 0x7f490c13ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:11.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4913200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f490c13f700 0x7f490c0ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:11.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4913200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c15c760 con 0x7f490c13f700 2024-05-22T21:23:11.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4913200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c15c9b0 con 0x7f490c15ec90 2024-05-22T21:23:11.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4913200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c15cc90 con 0x7f490c143ec0 2024-05-22T21:23:11.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4911200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f490c13f700 0x7f490c0ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:11.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4911200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f490c13f700 0x7f490c0ad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42506/0 (socket says 172.21.15.161:42506) 2024-05-22T21:23:11.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4911200640 1 -- 172.21.15.161:0/1259633583 learned_addr learned my addr 172.21.15.161:0/1259633583 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:11.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.964+0000 7f4903400640 1 --2- 172.21.15.161:0/1259633583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f490c15ec90 0x7f490c13ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:11.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f490be00640 1 --2- 172.21.15.161:0/1259633583 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f490c143ec0 0x7f490c163060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:11.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f4911200640 1 -- 172.21.15.161:0/1259633583 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f490c143ec0 msgr2=0x7f490c163060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:11.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f4911200640 1 --2- 172.21.15.161:0/1259633583 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f490c143ec0 0x7f490c163060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:11.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f4911200640 1 -- 172.21.15.161:0/1259633583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f490c15ec90 msgr2=0x7f490c13ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:11.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f4911200640 1 --2- 172.21.15.161:0/1259633583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f490c15ec90 0x7f490c13ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:11.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f4911200640 1 -- 172.21.15.161:0/1259633583 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f4009420 con 0x7f490c13f700 2024-05-22T21:23:11.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f4911200640 1 --2- 172.21.15.161:0/1259633583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f490c13f700 0x7f490c0ad020 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f490400c780 tx=0x7f490400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:11.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f4908c00640 1 -- 172.21.15.161:0/1259633583 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490400ce90 con 0x7f490c13f700 2024-05-22T21:23:11.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f4908c00640 1 -- 172.21.15.161:0/1259633583 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490400a640 con 0x7f490c13f700 2024-05-22T21:23:11.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f4908c00640 1 -- 172.21.15.161:0/1259633583 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490400a9d0 con 0x7f490c13f700 2024-05-22T21:23:11.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f4913200640 1 -- 172.21.15.161:0/1259633583 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c15f630 con 0x7f490c13f700 2024-05-22T21:23:11.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f4913200640 1 -- 172.21.15.161:0/1259633583 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c15d250 con 0x7f490c13f700 2024-05-22T21:23:11.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f4913200640 1 -- 172.21.15.161:0/1259633583 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490c067220 con 0x7f490c13f700 2024-05-22T21:23:11.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f4908c00640 1 -- 172.21.15.161:0/1259633583 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f490400ab30 con 0x7f490c13f700 2024-05-22T21:23:11.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f4908c00640 1 --2- 172.21.15.161:0/1259633583 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f48e8066690 0x7f48e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:11.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f4908c00640 1 -- 172.21.15.161:0/1259633583 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4904083690 con 0x7f490c13f700 2024-05-22T21:23:11.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.968+0000 7f490be00640 1 --2- 172.21.15.161:0/1259633583 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f48e8066690 0x7f48e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:11.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.972+0000 7f490be00640 1 --2- 172.21.15.161:0/1259633583 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f48e8066690 0x7f48e8068b50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f48fc007a60 tx=0x7f48fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:11.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:11.972+0000 7f4908c00640 1 -- 172.21.15.161:0/1259633583 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4904050a10 con 0x7f490c13f700 2024-05-22T21:23:12.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.160+0000 7f4913200640 1 -- 172.21.15.161:0/1259633583 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f490c06dcc0 con 0x7f490c13f700 2024-05-22T21:23:12.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.160+0000 7f4908c00640 1 -- 172.21.15.161:0/1259633583 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4747 (secure 0 0 0) 0x7f4904018090 con 0x7f490c13f700 2024-05-22T21:23:12.159 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:23:12.159 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2405126804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2405126804},{"type":"v1","addr":"172.21.15.179:6834","nonce":2405126804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":6962,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3984924056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3984924056},{"type":"v1","addr":"172.21.15.179:6835","nonce":3984924056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8391,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/368424851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":368424851},{"type":"v1","addr":"172.21.15.161:6836","nonce":368424851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:23:03.863540+0000","modified":"2024-05-22T21:23:06.877759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2279022192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2279022192},{"type":"v1","addr":"172.21.15.161:6837","nonce":2279022192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-05-22T21:23:12.160 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 78 2024-05-22T21:23:12.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.160+0000 7f4913200640 1 -- 172.21.15.161:0/1259633583 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f48e8066690 msgr2=0x7f48e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:12.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.160+0000 7f4913200640 1 --2- 172.21.15.161:0/1259633583 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f48e8066690 0x7f48e8068b50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f48fc007a60 tx=0x7f48fc008040 comp rx=0 tx=0).stop 2024-05-22T21:23:12.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.160+0000 7f4913200640 1 -- 172.21.15.161:0/1259633583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f490c13f700 msgr2=0x7f490c0ad020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:12.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.160+0000 7f4913200640 1 --2- 172.21.15.161:0/1259633583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f490c13f700 0x7f490c0ad020 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f490400c780 tx=0x7f490400cc40 comp rx=0 tx=0).stop 2024-05-22T21:23:12.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.160+0000 7f4913200640 1 -- 172.21.15.161:0/1259633583 shutdown_connections 2024-05-22T21:23:12.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.160+0000 7f4913200640 1 --2- 172.21.15.161:0/1259633583 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f48e8066690 0x7f48e8068b50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:12.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.160+0000 7f4913200640 1 --2- 172.21.15.161:0/1259633583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f490c13f700 0x7f490c0ad020 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:12.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.160+0000 7f4913200640 1 --2- 172.21.15.161:0/1259633583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f490c15ec90 0x7f490c13ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:12.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.160+0000 7f4913200640 1 --2- 172.21.15.161:0/1259633583 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f490c143ec0 0x7f490c163060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:12.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.160+0000 7f4913200640 1 -- 172.21.15.161:0/1259633583 >> 172.21.15.161:0/1259633583 conn(0x7f490c0b0bc0 msgr2=0x7f490c0ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:12.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.160+0000 7f4913200640 1 -- 172.21.15.161:0/1259633583 shutdown_connections 2024-05-22T21:23:12.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.160+0000 7f4913200640 1 -- 172.21.15.161:0/1259633583 wait complete. 2024-05-22T21:23:12.180 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-05-22T21:23:12.334 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":8394,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.4559627469999992,"mdsmap_epoch":78,"osdmap_epoch":175,"osdmap_epoch_barrier":175,"uptime":12.889275946} 2024-05-22T21:23:12.334 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 8394, "mdsmap_epoch": 78, "osdmap_epoch": 175, "osdmap_epoch_barrier": 175, "rank_uptime": 8.455962747, "state": "up:active", "uptime": 12.889275946, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:23:12.335 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-05-22T21:23:12.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.472+0000 7f7cbb400640 1 Processor -- start 2024-05-22T21:23:12.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.472+0000 7f7cbb400640 1 -- start start 2024-05-22T21:23:12.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cbb400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cb40ccc10 0x7f7cb415c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:12.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cbb400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb406d0d0 con 0x7f7cb4144980 2024-05-22T21:23:12.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cbb400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb406c4c0 con 0x7f7cb406d560 2024-05-22T21:23:12.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cbb400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb406c7a0 con 0x7f7cb40ccc10 2024-05-22T21:23:12.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb8a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f7cb4144980 0x7f7cb415c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:40786/0 (socket says 172.21.15.161:40786) 2024-05-22T21:23:12.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb8a00640 1 -- 172.21.15.161:0/1952273737 learned_addr learned my addr 172.21.15.161:0/1952273737 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:23:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb3400640 1 -- 172.21.15.161:0/1952273737 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180236595 0 0) 0x7f7cb406d0d0 con 0x7f7cb4144980 2024-05-22T21:23:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb3400640 1 -- 172.21.15.161:0/1952273737 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c98003530 con 0x7f7cb4144980 2024-05-22T21:23:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb9400640 1 --2- 172.21.15.161:0/1952273737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cb40ccc10 0x7f7cb415c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb3400640 1 -- 172.21.15.161:0/1952273737 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2669488707 0 0) 0x7f7c98003530 con 0x7f7cb4144980 2024-05-22T21:23:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb3400640 1 -- 172.21.15.161:0/1952273737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cb40ccc10 msgr2=0x7f7cb415c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb3400640 1 --2- 172.21.15.161:0/1952273737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cb40ccc10 0x7f7cb415c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb3400640 1 -- 172.21.15.161:0/1952273737 >> v1:172.21.15.179:6789/0 conn(0x7f7cb406d560 legacy=0x7f7cb40cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb3400640 1 -- 172.21.15.161:0/1952273737 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb406ca80 con 0x7f7cb4144980 2024-05-22T21:23:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb3400640 1 -- 172.21.15.161:0/1952273737 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f7ca4001680 con 0x7f7cb4144980 2024-05-22T21:23:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb3400640 1 -- 172.21.15.161:0/1952273737 >> v1:172.21.15.161:6789/0 conn(0x7f7cb4144980 legacy=0x7f7cb415c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:12.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb3400640 1 --2- 172.21.15.161:0/1952273737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c980039b0 0x7f7c98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:12.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb3400640 1 --2- 172.21.15.161:0/1952273737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cb40ccc10 0x7f7c980047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:12.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb3400640 1 --2- 172.21.15.161:0/1952273737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c98004d10 0x7f7c98005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:12.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb3400640 1 -- 172.21.15.161:0/1952273737 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb406c7a0 con 0x7f7c980039b0 2024-05-22T21:23:12.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb3400640 1 -- 172.21.15.161:0/1952273737 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9800ba70 con 0x7f7c98004d10 2024-05-22T21:23:12.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb3400640 1 -- 172.21.15.161:0/1952273737 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9800bc30 con 0x7f7cb40ccc10 2024-05-22T21:23:12.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb8a00640 1 --2- 172.21.15.161:0/1952273737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c980039b0 0x7f7c98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:12.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb3e00640 1 --2- 172.21.15.161:0/1952273737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cb40ccc10 0x7f7c980047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:12.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb9400640 1 --2- 172.21.15.161:0/1952273737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c98004d10 0x7f7c98005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:12.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb8a00640 1 -- 172.21.15.161:0/1952273737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cb40ccc10 msgr2=0x7f7c980047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:12.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb8a00640 1 --2- 172.21.15.161:0/1952273737 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cb40ccc10 0x7f7c980047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:12.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb8a00640 1 -- 172.21.15.161:0/1952273737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c98004d10 msgr2=0x7f7c98005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:23:12.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb8a00640 1 --2- 172.21.15.161:0/1952273737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c98004d10 0x7f7c98005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:12.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.476+0000 7f7cb8a00640 1 -- 172.21.15.161:0/1952273737 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c98003530 con 0x7f7c980039b0 2024-05-22T21:23:12.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.480+0000 7f7cb8a00640 1 --2- 172.21.15.161:0/1952273737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c980039b0 0x7f7c98003dd0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400c210 tx=0x7f7ca401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5141bb1822c31c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:12.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.480+0000 7f7cb3400640 1 -- 172.21.15.161:0/1952273737 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4005ae0 con 0x7f7c980039b0 2024-05-22T21:23:12.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.480+0000 7f7cb3400640 1 -- 172.21.15.161:0/1952273737 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca4005c60 con 0x7f7c980039b0 2024-05-22T21:23:12.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.480+0000 7f7cb3400640 1 -- 172.21.15.161:0/1952273737 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4007070 con 0x7f7c980039b0 2024-05-22T21:23:12.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.480+0000 7f7cbb400640 1 -- 172.21.15.161:0/1952273737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c980039b0 msgr2=0x7f7c98003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:12.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.480+0000 7f7cbb400640 1 --2- 172.21.15.161:0/1952273737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c980039b0 0x7f7c98003dd0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400c210 tx=0x7f7ca401cc40 comp rx=0 tx=0).stop 2024-05-22T21:23:12.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.480+0000 7f7cb9400640 1 -- 172.21.15.161:0/1952273737 reap_dead start 2024-05-22T21:23:12.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.480+0000 7f7cbb400640 1 -- 172.21.15.161:0/1952273737 shutdown_connections 2024-05-22T21:23:12.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.480+0000 7f7cbb400640 1 -- 172.21.15.161:0/1952273737 >> 172.21.15.161:0/1952273737 conn(0x7f7cb406a0a0 msgr2=0x7f7cb406a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:12.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.480+0000 7f7cbb400640 1 -- 172.21.15.161:0/1952273737 shutdown_connections 2024-05-22T21:23:12.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.480+0000 7f7cbb400640 1 -- 172.21.15.161:0/1952273737 wait complete. 2024-05-22T21:23:12.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.480+0000 7f7cbb400640 1 Processor -- start 2024-05-22T21:23:12.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.480+0000 7f7cbb400640 1 -- start start 2024-05-22T21:23:12.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cbb400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c980039b0 0x7f7cb413b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:12.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cbb400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cb406fce0 0x7f7cb40adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:12.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cbb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cb4141500 0x7f7cb4141990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:12.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cbb400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4070df0 con 0x7f7cb4141500 2024-05-22T21:23:12.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cbb400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb413bb50 con 0x7f7c980039b0 2024-05-22T21:23:12.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cbb400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4141ed0 con 0x7f7cb406fce0 2024-05-22T21:23:12.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cb9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cb4141500 0x7f7cb4141990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:12.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cb9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cb4141500 0x7f7cb4141990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42532/0 (socket says 172.21.15.161:42532) 2024-05-22T21:23:12.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cb9400640 1 -- 172.21.15.161:0/1556295858 learned_addr learned my addr 172.21.15.161:0/1556295858 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:23:12.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cb3e00640 1 --2- 172.21.15.161:0/1556295858 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cb406fce0 0x7f7cb40adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:12.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cb8a00640 1 --2- 172.21.15.161:0/1556295858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c980039b0 0x7f7cb413b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:12.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cb9400640 1 -- 172.21.15.161:0/1556295858 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cb406fce0 msgr2=0x7f7cb40adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:12.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cb9400640 1 --2- 172.21.15.161:0/1556295858 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cb406fce0 0x7f7cb40adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:12.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cb9400640 1 -- 172.21.15.161:0/1556295858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c980039b0 msgr2=0x7f7cb413b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:12.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cb9400640 1 --2- 172.21.15.161:0/1556295858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c980039b0 0x7f7cb413b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:12.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cb9400640 1 -- 172.21.15.161:0/1556295858 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb413b5a0 con 0x7f7cb4141500 2024-05-22T21:23:12.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cb9400640 1 --2- 172.21.15.161:0/1556295858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cb4141500 0x7f7cb4141990 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40aea10 tx=0x7f7cac00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:12.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cb0c00640 1 -- 172.21.15.161:0/1556295858 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac00e040 con 0x7f7cb4141500 2024-05-22T21:23:12.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cb0c00640 1 -- 172.21.15.161:0/1556295858 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac00f040 con 0x7f7cb4141500 2024-05-22T21:23:12.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cbb400640 1 -- 172.21.15.161:0/1556295858 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb4070170 con 0x7f7cb4141500 2024-05-22T21:23:12.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cb0c00640 1 -- 172.21.15.161:0/1556295858 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac015430 con 0x7f7cb4141500 2024-05-22T21:23:12.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cbb400640 1 -- 172.21.15.161:0/1556295858 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb4070730 con 0x7f7cb4141500 2024-05-22T21:23:12.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.484+0000 7f7cbb400640 1 -- 172.21.15.161:0/1556295858 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb4070f30 con 0x7f7cb4141500 2024-05-22T21:23:12.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.488+0000 7f7cb0c00640 1 -- 172.21.15.161:0/1556295858 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7cac015590 con 0x7f7cb4141500 2024-05-22T21:23:12.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.488+0000 7f7cb0c00640 1 --2- 172.21.15.161:0/1556295858 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7c94066690 0x7f7c94068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:23:12.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.488+0000 7f7cb0c00640 1 -- 172.21.15.161:0/1556295858 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cac0849f0 con 0x7f7cb4141500 2024-05-22T21:23:12.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.488+0000 7f7cb8a00640 1 --2- 172.21.15.161:0/1556295858 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7c94066690 0x7f7c94068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:23:12.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.488+0000 7f7cb8a00640 1 --2- 172.21.15.161:0/1556295858 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7c94066690 0x7f7c94068b50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40ab000 tx=0x7f7ca40a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:23:12.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.492+0000 7f7cb0c00640 1 -- 172.21.15.161:0/1556295858 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7cb4070f30 con 0x7f7cb4141500 2024-05-22T21:23:12.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:12.652+0000 7f7cbb400640 1 -- 172.21.15.161:0/1556295858 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f7cb4070f30 con 0x7f7cb4141500 2024-05-22T21:23:13.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:13.524+0000 7f7cb0c00640 1 -- 172.21.15.161:0/1556295858 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v177) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7cb4070f30 con 0x7f7cb4141500 2024-05-22T21:23:13.523 INFO:teuthology.orchestra.run.smithi161.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-05-22T21:23:13.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:13.524+0000 7f7cbb400640 1 -- 172.21.15.161:0/1556295858 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7c94066690 msgr2=0x7f7c94068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:13.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:13.524+0000 7f7cbb400640 1 --2- 172.21.15.161:0/1556295858 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7c94066690 0x7f7c94068b50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40ab000 tx=0x7f7ca40a7070 comp rx=0 tx=0).stop 2024-05-22T21:23:13.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:13.524+0000 7f7cbb400640 1 -- 172.21.15.161:0/1556295858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cb4141500 msgr2=0x7f7cb4141990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:23:13.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:13.524+0000 7f7cbb400640 1 --2- 172.21.15.161:0/1556295858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cb4141500 0x7f7cb4141990 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40aea10 tx=0x7f7cac00f580 comp rx=0 tx=0).stop 2024-05-22T21:23:13.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:13.524+0000 7f7cbb400640 1 -- 172.21.15.161:0/1556295858 shutdown_connections 2024-05-22T21:23:13.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:13.524+0000 7f7cbb400640 1 --2- 172.21.15.161:0/1556295858 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7c94066690 0x7f7c94068b50 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:13.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:13.524+0000 7f7cbb400640 1 --2- 172.21.15.161:0/1556295858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cb4141500 0x7f7cb4141990 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:13.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:13.524+0000 7f7cbb400640 1 --2- 172.21.15.161:0/1556295858 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7cb406fce0 0x7f7cb40adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:13.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:13.524+0000 7f7cbb400640 1 --2- 172.21.15.161:0/1556295858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c980039b0 0x7f7cb413b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:23:13.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:13.524+0000 7f7cbb400640 1 -- 172.21.15.161:0/1556295858 >> 172.21.15.161:0/1556295858 conn(0x7f7cb406a0a0 msgr2=0x7f7cb4143930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:23:13.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:13.524+0000 7f7cbb400640 1 -- 172.21.15.161:0/1556295858 shutdown_connections 2024-05-22T21:23:13.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:23:13.524+0000 7f7cbb400640 1 -- 172.21.15.161:0/1556295858 wait complete. 2024-05-22T21:23:13.551 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-05-22T21:23:13.705 INFO:teuthology.orchestra.run.smithi161.stdout:{"osd_mon_report_interval":"5"} 2024-05-22T21:23:13.706 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-05-22T21:23:13.706 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-05-22T21:23:13.852 INFO:teuthology.orchestra.run.smithi161.stdout:{"mon_tick_interval":"5"} 2024-05-22T21:23:13.852 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-05-22T21:23:13.853 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2024-05-22T21:23:13.853 DEBUG:teuthology.orchestra.run.smithi161:> adjust-ulimits daemon-helper kill python3 -c ' 2024-05-22T21:23:13.853 DEBUG:teuthology.orchestra.run.smithi161:> import time 2024-05-22T21:23:13.853 DEBUG:teuthology.orchestra.run.smithi161:> import datetime 2024-05-22T21:23:13.853 DEBUG:teuthology.orchestra.run.smithi161:> import subprocess 2024-05-22T21:23:13.853 DEBUG:teuthology.orchestra.run.smithi161:> import os 2024-05-22T21:23:13.853 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:23:13.853 DEBUG:teuthology.orchestra.run.smithi161:> # Write some buffered data through before going full, all should be well 2024-05-22T21:23:13.853 DEBUG:teuthology.orchestra.run.smithi161:> print("writing some data through which we expect to succeed") 2024-05-22T21:23:13.853 DEBUG:teuthology.orchestra.run.smithi161:> bytes = 0 2024-05-22T21:23:13.854 DEBUG:teuthology.orchestra.run.smithi161:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-05-22T21:23:13.854 DEBUG:teuthology.orchestra.run.smithi161:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-05-22T21:23:13.854 DEBUG:teuthology.orchestra.run.smithi161:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2024-05-22T21:23:13.854 DEBUG:teuthology.orchestra.run.smithi161:> os.fsync(f) 2024-05-22T21:23:13.854 DEBUG:teuthology.orchestra.run.smithi161:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-05-22T21:23:13.854 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:23:13.854 DEBUG:teuthology.orchestra.run.smithi161:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-05-22T21:23:13.854 DEBUG:teuthology.orchestra.run.smithi161:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-05-22T21:23:13.854 DEBUG:teuthology.orchestra.run.smithi161:> # buffered IO, the error should always only appear from fsync and not 2024-05-22T21:23:13.854 DEBUG:teuthology.orchestra.run.smithi161:> # from write 2024-05-22T21:23:13.854 DEBUG:teuthology.orchestra.run.smithi161:> full = False 2024-05-22T21:23:13.854 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:23:13.854 DEBUG:teuthology.orchestra.run.smithi161:> for n in range(0, int(32 * 1.1)): 2024-05-22T21:23:13.854 DEBUG:teuthology.orchestra.run.smithi161:> try: 2024-05-22T21:23:13.855 DEBUG:teuthology.orchestra.run.smithi161:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-05-22T21:23:13.855 DEBUG:teuthology.orchestra.run.smithi161:> print("wrote bytes via buffered write, moving on to fsync") 2024-05-22T21:23:13.855 DEBUG:teuthology.orchestra.run.smithi161:> except OSError as e: 2024-05-22T21:23:13.855 DEBUG:teuthology.orchestra.run.smithi161:> if False: 2024-05-22T21:23:13.855 DEBUG:teuthology.orchestra.run.smithi161:> print("Unexpected error %s from write() instead of fsync()" % e) 2024-05-22T21:23:13.855 DEBUG:teuthology.orchestra.run.smithi161:> raise 2024-05-22T21:23:13.855 DEBUG:teuthology.orchestra.run.smithi161:> else: 2024-05-22T21:23:13.855 DEBUG:teuthology.orchestra.run.smithi161:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-05-22T21:23:13.855 DEBUG:teuthology.orchestra.run.smithi161:> full = True 2024-05-22T21:23:13.855 DEBUG:teuthology.orchestra.run.smithi161:> break 2024-05-22T21:23:13.855 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:23:13.855 DEBUG:teuthology.orchestra.run.smithi161:> try: 2024-05-22T21:23:13.855 DEBUG:teuthology.orchestra.run.smithi161:> os.fsync(f) 2024-05-22T21:23:13.855 DEBUG:teuthology.orchestra.run.smithi161:> print("fsync'"'"'ed successfully") 2024-05-22T21:23:13.855 DEBUG:teuthology.orchestra.run.smithi161:> except OSError as e: 2024-05-22T21:23:13.856 DEBUG:teuthology.orchestra.run.smithi161:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-05-22T21:23:13.856 DEBUG:teuthology.orchestra.run.smithi161:> full = True 2024-05-22T21:23:13.856 DEBUG:teuthology.orchestra.run.smithi161:> break 2024-05-22T21:23:13.856 DEBUG:teuthology.orchestra.run.smithi161:> else: 2024-05-22T21:23:13.856 DEBUG:teuthology.orchestra.run.smithi161:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-05-22T21:23:13.856 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:23:13.856 DEBUG:teuthology.orchestra.run.smithi161:> if n > 32 * 0.9: 2024-05-22T21:23:13.856 DEBUG:teuthology.orchestra.run.smithi161:> # Be cautious in the last region where we expect to hit 2024-05-22T21:23:13.856 DEBUG:teuthology.orchestra.run.smithi161:> # the full condition, so that we don'"'"'t overshoot too dramatically 2024-05-22T21:23:13.856 DEBUG:teuthology.orchestra.run.smithi161:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-05-22T21:23:13.856 DEBUG:teuthology.orchestra.run.smithi161:> time.sleep(15.0) 2024-05-22T21:23:13.856 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:23:13.856 DEBUG:teuthology.orchestra.run.smithi161:> if not full: 2024-05-22T21:23:13.856 DEBUG:teuthology.orchestra.run.smithi161:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2024-05-22T21:23:13.856 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:23:13.857 DEBUG:teuthology.orchestra.run.smithi161:> # close() should not raise an error because we already caught it in 2024-05-22T21:23:13.857 DEBUG:teuthology.orchestra.run.smithi161:> # fsync. There shouldn'"'"'t have been any more writeback errors 2024-05-22T21:23:13.857 DEBUG:teuthology.orchestra.run.smithi161:> # since then because all IOs got cancelled on the full flag. 2024-05-22T21:23:13.857 DEBUG:teuthology.orchestra.run.smithi161:> print("calling close") 2024-05-22T21:23:13.857 DEBUG:teuthology.orchestra.run.smithi161:> os.close(f) 2024-05-22T21:23:13.857 DEBUG:teuthology.orchestra.run.smithi161:> print("close() did not raise error") 2024-05-22T21:23:13.857 DEBUG:teuthology.orchestra.run.smithi161:> 2024-05-22T21:23:13.857 DEBUG:teuthology.orchestra.run.smithi161:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-05-22T21:23:13.857 DEBUG:teuthology.orchestra.run.smithi161:> ' 2024-05-22T21:23:24.995 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:23:24.999 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:23:55.060 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:23:55.065 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:24:14.900 INFO:teuthology.orchestra.run.smithi161.stdout:writing some data through which we expect to succeed 2024-05-22T21:24:14.900 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-05-22T21:24:14.900 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.900 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.900 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 1.00 MB 2024-05-22T21:24:14.901 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.901 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.901 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 2.00 MB 2024-05-22T21:24:14.901 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.901 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.901 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 3.00 MB 2024-05-22T21:24:14.901 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.901 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.901 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 4.00 MB 2024-05-22T21:24:14.901 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.901 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.901 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 5.00 MB 2024-05-22T21:24:14.902 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.902 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.902 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 6.00 MB 2024-05-22T21:24:14.902 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.902 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.902 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 7.00 MB 2024-05-22T21:24:14.902 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.902 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.902 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 8.00 MB 2024-05-22T21:24:14.902 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.903 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.903 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 9.00 MB 2024-05-22T21:24:14.903 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.903 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.903 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 10.00 MB 2024-05-22T21:24:14.903 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.903 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.903 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 11.00 MB 2024-05-22T21:24:14.903 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.903 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.903 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 12.00 MB 2024-05-22T21:24:14.903 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.904 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.904 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 13.00 MB 2024-05-22T21:24:14.904 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.904 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.904 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 14.00 MB 2024-05-22T21:24:14.904 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.904 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.904 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 15.00 MB 2024-05-22T21:24:14.904 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.904 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.904 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 16.00 MB 2024-05-22T21:24:14.905 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.905 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.905 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 17.00 MB 2024-05-22T21:24:14.905 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.905 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.905 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 18.00 MB 2024-05-22T21:24:14.905 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.905 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.905 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 19.00 MB 2024-05-22T21:24:14.905 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.906 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.906 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 20.00 MB 2024-05-22T21:24:14.906 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.906 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.906 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 21.00 MB 2024-05-22T21:24:14.906 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.906 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.906 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 22.00 MB 2024-05-22T21:24:14.906 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.906 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.906 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 23.00 MB 2024-05-22T21:24:14.907 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.907 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.907 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 24.00 MB 2024-05-22T21:24:14.907 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.907 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.907 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 25.00 MB 2024-05-22T21:24:14.907 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.907 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.907 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 26.00 MB 2024-05-22T21:24:14.907 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.907 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.907 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 27.00 MB 2024-05-22T21:24:14.907 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.908 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.908 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 28.00 MB 2024-05-22T21:24:14.908 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.908 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.908 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 29.00 MB 2024-05-22T21:24:14.908 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.908 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.908 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 30.00 MB 2024-05-22T21:24:14.908 INFO:teuthology.orchestra.run.smithi161.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-05-22T21:24:14.908 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.908 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.908 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 31.00 MB 2024-05-22T21:24:14.908 INFO:teuthology.orchestra.run.smithi161.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-05-22T21:24:14.909 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.909 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.909 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 32.00 MB 2024-05-22T21:24:14.909 INFO:teuthology.orchestra.run.smithi161.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-05-22T21:24:14.909 INFO:teuthology.orchestra.run.smithi161.stdout:wrote bytes via buffered write, moving on to fsync 2024-05-22T21:24:14.909 INFO:teuthology.orchestra.run.smithi161.stdout:fsync'ed successfully 2024-05-22T21:24:14.909 INFO:teuthology.orchestra.run.smithi161.stdout:Not full yet after 33.00 MB 2024-05-22T21:24:14.909 INFO:teuthology.orchestra.run.smithi161.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-05-22T21:24:14.909 INFO:teuthology.orchestra.run.smithi161.stdout:Reached fullness after 33.00 MB 2024-05-22T21:24:14.909 INFO:teuthology.orchestra.run.smithi161.stdout:calling close 2024-05-22T21:24:14.909 INFO:teuthology.orchestra.run.smithi161.stdout:close() did not raise error 2024-05-22T21:24:15.002 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:24:15.002 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:24:15.065 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:24:15.129 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:24:15.129 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:24:15.129 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:24:15.139 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:24:15.139 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:24:15.190 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:24:15.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe21e400640 1 Processor -- start 2024-05-22T21:24:15.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe21e400640 1 -- start start 2024-05-22T21:24:15.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe21e400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe21806d2d0 0x7fe218158990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:15.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe21e400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe21806d6b0 0x7fe21815c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:15.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe21e400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2180cd390 con 0x7fe2180cd820 2024-05-22T21:24:15.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe21e400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2180cca60 con 0x7fe21806d6b0 2024-05-22T21:24:15.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe21e400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2180ccd40 con 0x7fe21806d2d0 2024-05-22T21:24:15.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe217e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe21806d6b0 0x7fe21815c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:15.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe216a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe2180cd820 0x7fe21815ba50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59716/0 (socket says 172.21.15.161:59716) 2024-05-22T21:24:15.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe216a00640 1 -- 172.21.15.161:0/102911710 learned_addr learned my addr 172.21.15.161:0/102911710 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:15.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe217e00640 1 -- 172.21.15.161:0/102911710 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe21806d2d0 msgr2=0x7fe218158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:15.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe217400640 1 --2- 172.21.15.161:0/102911710 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe21806d2d0 0x7fe218158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:15.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe217e00640 1 --2- 172.21.15.161:0/102911710 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe21806d2d0 0x7fe218158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe216000640 1 -- 172.21.15.161:0/102911710 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075130530 0 0) 0x7fe2180cd390 con 0x7fe2180cd820 2024-05-22T21:24:15.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe217e00640 1 -- 172.21.15.161:0/102911710 >> v1:172.21.15.161:6789/0 conn(0x7fe2180cd820 legacy=0x7fe21815ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:15.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe217e00640 1 -- 172.21.15.161:0/102911710 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2180cd020 con 0x7fe21806d6b0 2024-05-22T21:24:15.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.379+0000 7fe217400640 1 --2- 172.21.15.161:0/102911710 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe21806d2d0 0x7fe218158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:15.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.383+0000 7fe217e00640 1 --2- 172.21.15.161:0/102911710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe21806d6b0 0x7fe21815c160 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fe210009d90 tx=0x7fe21001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b9fd7242779d11 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:15.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.383+0000 7fe216000640 1 -- 172.21.15.161:0/102911710 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2100a59c0 con 0x7fe21806d6b0 2024-05-22T21:24:15.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.383+0000 7fe216000640 1 -- 172.21.15.161:0/102911710 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2100a5b20 con 0x7fe21806d6b0 2024-05-22T21:24:15.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.383+0000 7fe216000640 1 -- 172.21.15.161:0/102911710 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2100ad440 con 0x7fe21806d6b0 2024-05-22T21:24:15.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.383+0000 7fe21e400640 1 -- 172.21.15.161:0/102911710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe21806d6b0 msgr2=0x7fe21815c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:15.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.383+0000 7fe21e400640 1 --2- 172.21.15.161:0/102911710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe21806d6b0 0x7fe21815c160 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fe210009d90 tx=0x7fe21001c8e0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.383+0000 7fe21e400640 1 -- 172.21.15.161:0/102911710 shutdown_connections 2024-05-22T21:24:15.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.383+0000 7fe21e400640 1 --2- 172.21.15.161:0/102911710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe21806d6b0 0x7fe21815c160 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.383+0000 7fe21e400640 1 --2- 172.21.15.161:0/102911710 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe21806d2d0 0x7fe218158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.383+0000 7fe21e400640 1 -- 172.21.15.161:0/102911710 >> 172.21.15.161:0/102911710 conn(0x7fe2180b5540 msgr2=0x7fe2180b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:15.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.383+0000 7fe21e400640 1 -- 172.21.15.161:0/102911710 shutdown_connections 2024-05-22T21:24:15.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.383+0000 7fe21e400640 1 -- 172.21.15.161:0/102911710 wait complete. 2024-05-22T21:24:15.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe21e400640 1 Processor -- start 2024-05-22T21:24:15.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe21e400640 1 -- start start 2024-05-22T21:24:15.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe21e400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe21806d2d0 0x7fe2181573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:15.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe21e400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe21806d6b0 0x7fe218143690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe21e400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2180cd820 0x7fe21815da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe21e400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2181440e0 con 0x7fe2180cd820 2024-05-22T21:24:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe21e400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21813dbe0 con 0x7fe21806d6b0 2024-05-22T21:24:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe21e400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21813dec0 con 0x7fe21806d2d0 2024-05-22T21:24:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe217e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2180cd820 0x7fe21815da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe217e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2180cd820 0x7fe21815da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35534/0 (socket says 172.21.15.161:35534) 2024-05-22T21:24:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe217e00640 1 -- 172.21.15.161:0/1091111157 learned_addr learned my addr 172.21.15.161:0/1091111157 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe217400640 1 --2- 172.21.15.161:0/1091111157 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe21806d2d0 0x7fe2181573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe216a00640 1 --2- 172.21.15.161:0/1091111157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe21806d6b0 0x7fe218143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe217e00640 1 -- 172.21.15.161:0/1091111157 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe21806d2d0 msgr2=0x7fe2181573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe217e00640 1 --2- 172.21.15.161:0/1091111157 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe21806d2d0 0x7fe2181573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe217e00640 1 -- 172.21.15.161:0/1091111157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe21806d6b0 msgr2=0x7fe218143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:15.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe217e00640 1 --2- 172.21.15.161:0/1091111157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe21806d6b0 0x7fe218143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe217e00640 1 -- 172.21.15.161:0/1091111157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe210009a70 con 0x7fe2180cd820 2024-05-22T21:24:15.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe216a00640 1 --2- 172.21.15.161:0/1091111157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe21806d6b0 0x7fe218143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:15.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe217e00640 1 --2- 172.21.15.161:0/1091111157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2180cd820 0x7fe21815da30 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fe2100b0040 tx=0x7fe210019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:15.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe20f400640 1 -- 172.21.15.161:0/1091111157 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2100b9650 con 0x7fe2180cd820 2024-05-22T21:24:15.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe20f400640 1 -- 172.21.15.161:0/1091111157 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2100b97b0 con 0x7fe2180cd820 2024-05-22T21:24:15.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe21e400640 1 -- 172.21.15.161:0/1091111157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21813e1a0 con 0x7fe2180cd820 2024-05-22T21:24:15.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe20f400640 1 -- 172.21.15.161:0/1091111157 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2100b9b40 con 0x7fe2180cd820 2024-05-22T21:24:15.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe21e400640 1 -- 172.21.15.161:0/1091111157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe218157910 con 0x7fe2180cd820 2024-05-22T21:24:15.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.387+0000 7fe21e400640 1 -- 172.21.15.161:0/1091111157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21813e6c0 con 0x7fe2180cd820 2024-05-22T21:24:15.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.391+0000 7fe20f400640 1 -- 172.21.15.161:0/1091111157 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe2100b9ca0 con 0x7fe2180cd820 2024-05-22T21:24:15.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.391+0000 7fe20f400640 1 --2- 172.21.15.161:0/1091111157 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe1f4066690 0x7fe1f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:15.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.391+0000 7fe20f400640 1 -- 172.21.15.161:0/1091111157 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe21011ff90 con 0x7fe2180cd820 2024-05-22T21:24:15.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.391+0000 7fe217400640 1 --2- 172.21.15.161:0/1091111157 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe1f4066690 0x7fe1f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:15.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.391+0000 7fe20f400640 1 -- 172.21.15.161:0/1091111157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7fe1f406e220 con 0x7fe2180cd820 2024-05-22T21:24:15.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.391+0000 7fe217400640 1 --2- 172.21.15.161:0/1091111157 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe1f4066690 0x7fe1f4068b50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe21815c670 tx=0x7fe200008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:15.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.395+0000 7fe20f400640 1 -- 172.21.15.161:0/1091111157 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe21813e6c0 con 0x7fe2180cd820 2024-05-22T21:24:15.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.551+0000 7fe21e400640 1 -- 172.21.15.161:0/1091111157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe21813e6c0 con 0x7fe2180cd820 2024-05-22T21:24:15.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.555+0000 7fe20f400640 1 -- 172.21.15.161:0/1091111157 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+16235 (secure 0 0 0) 0x7fe21813e6c0 con 0x7fe2180cd820 2024-05-22T21:24:15.554 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:15.554 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":178,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:23:50.299404+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:23:02.242525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-05-22T21:23:02.864916+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:24:15.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.555+0000 7fe21e400640 1 -- 172.21.15.161:0/1091111157 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe1f4066690 msgr2=0x7fe1f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:15.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.555+0000 7fe21e400640 1 --2- 172.21.15.161:0/1091111157 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe1f4066690 0x7fe1f4068b50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe21815c670 tx=0x7fe200008040 comp rx=0 tx=0).stop 2024-05-22T21:24:15.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.555+0000 7fe21e400640 1 -- 172.21.15.161:0/1091111157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2180cd820 msgr2=0x7fe21815da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:15.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.555+0000 7fe21e400640 1 --2- 172.21.15.161:0/1091111157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2180cd820 0x7fe21815da30 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fe2100b0040 tx=0x7fe210019680 comp rx=0 tx=0).stop 2024-05-22T21:24:15.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.555+0000 7fe21e400640 1 -- 172.21.15.161:0/1091111157 shutdown_connections 2024-05-22T21:24:15.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.555+0000 7fe21e400640 1 --2- 172.21.15.161:0/1091111157 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe1f4066690 0x7fe1f4068b50 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.555+0000 7fe21e400640 1 --2- 172.21.15.161:0/1091111157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2180cd820 0x7fe21815da30 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.555+0000 7fe21e400640 1 --2- 172.21.15.161:0/1091111157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe21806d6b0 0x7fe218143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.555+0000 7fe21e400640 1 --2- 172.21.15.161:0/1091111157 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe21806d2d0 0x7fe2181573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.555+0000 7fe21e400640 1 -- 172.21.15.161:0/1091111157 >> 172.21.15.161:0/1091111157 conn(0x7fe2180b5540 msgr2=0x7fe2180b5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:15.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.555+0000 7fe21e400640 1 -- 172.21.15.161:0/1091111157 shutdown_connections 2024-05-22T21:24:15.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.555+0000 7fe21e400640 1 -- 172.21.15.161:0/1091111157 wait complete. 2024-05-22T21:24:15.575 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:23:02.242525+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '174', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:23:02.864916+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '178', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:24:15.576 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:24:15.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e63200640 1 Processor -- start 2024-05-22T21:24:15.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e63200640 1 -- start start 2024-05-22T21:24:15.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e63200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3e5c0cf160 0x7f3e5c0cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:15.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e63200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e5c0cdce0 0x7f3e5c0cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:15.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e63200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5c0d1c00 con 0x7f3e5c15d400 2024-05-22T21:24:15.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e63200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e5c0ce170 con 0x7f3e5c0cf160 2024-05-22T21:24:15.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e63200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c06e920 con 0x7f3e5c0cdce0 2024-05-22T21:24:15.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e5be00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3e5c15d400 0x7f3e5c06edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59728/0 (socket says 172.21.15.161:59728) 2024-05-22T21:24:15.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e5be00640 1 -- 172.21.15.161:0/3647011091 learned_addr learned my addr 172.21.15.161:0/3647011091 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:15.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e5b400640 1 --2- 172.21.15.161:0/3647011091 >> v2:172.21.15.179:3300/0 conn(0x7f3e5c0cf160 0x7f3e5c0cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:15.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e61200640 1 --2- 172.21.15.161:0/3647011091 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e5c0cdce0 0x7f3e5c0cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:15.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e5aa00640 1 -- 172.21.15.161:0/3647011091 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74571026 0 0) 0x7f3e5c0d1c00 con 0x7f3e5c15d400 2024-05-22T21:24:15.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e5aa00640 1 -- 172.21.15.161:0/3647011091 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e40003530 con 0x7f3e5c15d400 2024-05-22T21:24:15.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e61200640 1 -- 172.21.15.161:0/3647011091 >> v2:172.21.15.179:3300/0 conn(0x7f3e5c0cf160 msgr2=0x7f3e5c0cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:15.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e61200640 1 --2- 172.21.15.161:0/3647011091 >> v2:172.21.15.179:3300/0 conn(0x7f3e5c0cf160 0x7f3e5c0cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e61200640 1 -- 172.21.15.161:0/3647011091 >> v1:172.21.15.161:6789/0 conn(0x7f3e5c15d400 legacy=0x7f3e5c06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:15.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e61200640 1 -- 172.21.15.161:0/3647011091 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c0cf5a0 con 0x7f3e5c0cdce0 2024-05-22T21:24:15.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.715+0000 7f3e5b400640 1 --2- 172.21.15.161:0/3647011091 >> v2:172.21.15.179:3300/0 conn(0x7f3e5c0cf160 0x7f3e5c0cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:24:15.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.719+0000 7f3e61200640 1 --2- 172.21.15.161:0/3647011091 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e5c0cdce0 0x7f3e5c0cb570 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e540097c0 tx=0x7f3e5401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3adb1b43a24a6d54 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:15.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.719+0000 7f3e5aa00640 1 -- 172.21.15.161:0/3647011091 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e540ac640 con 0x7f3e5c0cdce0 2024-05-22T21:24:15.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.719+0000 7f3e5aa00640 1 -- 172.21.15.161:0/3647011091 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e540ac7a0 con 0x7f3e5c0cdce0 2024-05-22T21:24:15.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.719+0000 7f3e5aa00640 1 -- 172.21.15.161:0/3647011091 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e540acb10 con 0x7f3e5c0cdce0 2024-05-22T21:24:15.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.719+0000 7f3e63200640 1 -- 172.21.15.161:0/3647011091 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e5c0cdce0 msgr2=0x7f3e5c0cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:15.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.719+0000 7f3e63200640 1 --2- 172.21.15.161:0/3647011091 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e5c0cdce0 0x7f3e5c0cb570 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e540097c0 tx=0x7f3e5401c3e0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.719+0000 7f3e63200640 1 -- 172.21.15.161:0/3647011091 shutdown_connections 2024-05-22T21:24:15.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.719+0000 7f3e63200640 1 --2- 172.21.15.161:0/3647011091 >> v2:172.21.15.179:3300/0 conn(0x7f3e5c0cf160 0x7f3e5c0cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.719+0000 7f3e63200640 1 --2- 172.21.15.161:0/3647011091 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e5c0cdce0 0x7f3e5c0cb570 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.719+0000 7f3e63200640 1 -- 172.21.15.161:0/3647011091 >> 172.21.15.161:0/3647011091 conn(0x7f3e5c0d0f20 msgr2=0x7f3e5c0d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:15.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.719+0000 7f3e63200640 1 -- 172.21.15.161:0/3647011091 shutdown_connections 2024-05-22T21:24:15.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.719+0000 7f3e63200640 1 -- 172.21.15.161:0/3647011091 wait complete. 2024-05-22T21:24:15.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e63200640 1 Processor -- start 2024-05-22T21:24:15.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e63200640 1 -- start start 2024-05-22T21:24:15.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e63200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e5c0cdce0 0x7f3e5c0b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:15.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e63200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5c0cf160 0x7f3e5c0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:15.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e63200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e5c15d400 0x7f3e5c15e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:15.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e63200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0b2630 con 0x7f3e5c0cf160 2024-05-22T21:24:15.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e63200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0b6b80 con 0x7f3e5c0cdce0 2024-05-22T21:24:15.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e63200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0b28d0 con 0x7f3e5c15d400 2024-05-22T21:24:15.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e5b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5c0cf160 0x7f3e5c0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:15.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e5b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5c0cf160 0x7f3e5c0b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35542/0 (socket says 172.21.15.161:35542) 2024-05-22T21:24:15.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e5b400640 1 -- 172.21.15.161:0/899950420 learned_addr learned my addr 172.21.15.161:0/899950420 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:15.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e5be00640 1 --2- 172.21.15.161:0/899950420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e5c0cdce0 0x7f3e5c0b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e61200640 1 --2- 172.21.15.161:0/899950420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e5c15d400 0x7f3e5c15e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e5b400640 1 -- 172.21.15.161:0/899950420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e5c15d400 msgr2=0x7f3e5c15e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e5b400640 1 --2- 172.21.15.161:0/899950420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e5c15d400 0x7f3e5c15e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e5b400640 1 -- 172.21.15.161:0/899950420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e5c0cdce0 msgr2=0x7f3e5c0b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e5b400640 1 --2- 172.21.15.161:0/899950420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e5c0cdce0 0x7f3e5c0b64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e5b400640 1 -- 172.21.15.161:0/899950420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e48009420 con 0x7f3e5c0cf160 2024-05-22T21:24:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e5b400640 1 --2- 172.21.15.161:0/899950420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5c0cf160 0x7f3e5c0b4630 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800ebe0 tx=0x7f3e4800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e53e00640 1 -- 172.21.15.161:0/899950420 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4800eee0 con 0x7f3e5c0cf160 2024-05-22T21:24:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e53e00640 1 -- 172.21.15.161:0/899950420 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4800cc30 con 0x7f3e5c0cf160 2024-05-22T21:24:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e63200640 1 -- 172.21.15.161:0/899950420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e54009420 con 0x7f3e5c0cf160 2024-05-22T21:24:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e53e00640 1 -- 172.21.15.161:0/899950420 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48013470 con 0x7f3e5c0cf160 2024-05-22T21:24:15.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.723+0000 7f3e63200640 1 -- 172.21.15.161:0/899950420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5c0b2e90 con 0x7f3e5c0cf160 2024-05-22T21:24:15.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.727+0000 7f3e63200640 1 -- 172.21.15.161:0/899950420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5c067220 con 0x7f3e5c0cf160 2024-05-22T21:24:15.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.727+0000 7f3e53e00640 1 -- 172.21.15.161:0/899950420 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3e480135d0 con 0x7f3e5c0cf160 2024-05-22T21:24:15.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.727+0000 7f3e53e00640 1 --2- 172.21.15.161:0/899950420 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e3c066690 0x7f3e3c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:15.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.727+0000 7f3e53e00640 1 -- 172.21.15.161:0/899950420 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f3e48084480 con 0x7f3e5c0cf160 2024-05-22T21:24:15.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.727+0000 7f3e53e00640 1 -- 172.21.15.161:0/899950420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7f3e3c06e220 con 0x7f3e5c0cf160 2024-05-22T21:24:15.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.727+0000 7f3e5be00640 1 --2- 172.21.15.161:0/899950420 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e3c066690 0x7f3e3c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:15.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.727+0000 7f3e5be00640 1 --2- 172.21.15.161:0/899950420 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e3c066690 0x7f3e3c068b50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c005c10 tx=0x7f3e4c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:15.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.731+0000 7f3e53e00640 1 -- 172.21.15.161:0/899950420 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3e48051870 con 0x7f3e5c0cf160 2024-05-22T21:24:15.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.887+0000 7f3e63200640 1 -- 172.21.15.161:0/899950420 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e5c0b6ea0 con 0x7f3e5c0cf160 2024-05-22T21:24:15.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.887+0000 7f3e53e00640 1 -- 172.21.15.161:0/899950420 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v178) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e480170a0 con 0x7f3e5c0cf160 2024-05-22T21:24:15.888 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:24:15.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.887+0000 7f3e63200640 1 -- 172.21.15.161:0/899950420 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e3c066690 msgr2=0x7f3e3c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:15.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.887+0000 7f3e63200640 1 --2- 172.21.15.161:0/899950420 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e3c066690 0x7f3e3c068b50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c005c10 tx=0x7f3e4c009040 comp rx=0 tx=0).stop 2024-05-22T21:24:15.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.887+0000 7f3e63200640 1 -- 172.21.15.161:0/899950420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5c0cf160 msgr2=0x7f3e5c0b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:15.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.887+0000 7f3e63200640 1 --2- 172.21.15.161:0/899950420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5c0cf160 0x7f3e5c0b4630 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800ebe0 tx=0x7f3e4800c520 comp rx=0 tx=0).stop 2024-05-22T21:24:15.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.891+0000 7f3e63200640 1 -- 172.21.15.161:0/899950420 shutdown_connections 2024-05-22T21:24:15.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.891+0000 7f3e63200640 1 --2- 172.21.15.161:0/899950420 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e3c066690 0x7f3e3c068b50 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.891+0000 7f3e63200640 1 --2- 172.21.15.161:0/899950420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e5c15d400 0x7f3e5c15e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.891+0000 7f3e63200640 1 --2- 172.21.15.161:0/899950420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5c0cf160 0x7f3e5c0b4630 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.891+0000 7f3e63200640 1 --2- 172.21.15.161:0/899950420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e5c0cdce0 0x7f3e5c0b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:15.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.891+0000 7f3e63200640 1 -- 172.21.15.161:0/899950420 >> 172.21.15.161:0/899950420 conn(0x7f3e5c0d0f20 msgr2=0x7f3e5c0add40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:15.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.891+0000 7f3e63200640 1 -- 172.21.15.161:0/899950420 shutdown_connections 2024-05-22T21:24:15.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:15.891+0000 7f3e63200640 1 -- 172.21.15.161:0/899950420 wait complete. 2024-05-22T21:24:15.909 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:24:16.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.047+0000 7f6502a00640 1 Processor -- start 2024-05-22T21:24:16.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.047+0000 7f6502a00640 1 -- start start 2024-05-22T21:24:16.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.047+0000 7f6502a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64fc13cfc0 0x7f64fc1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.047+0000 7f6502a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc13d3a0 con 0x7f64fc15af60 2024-05-22T21:24:16.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.047+0000 7f6502a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc0cd920 con 0x7f64fc13fe50 2024-05-22T21:24:16.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.047+0000 7f6502a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc0cdc00 con 0x7f64fc13cfc0 2024-05-22T21:24:16.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.047+0000 7f64f3400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f64fc15af60 0x7f64fc13f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56842/0 (socket says 172.21.15.161:56842) 2024-05-22T21:24:16.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.047+0000 7f64f3400640 1 -- 172.21.15.161:0/2670437777 learned_addr learned my addr 172.21.15.161:0/2670437777 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:16.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.047+0000 7f64fbe00640 1 --2- 172.21.15.161:0/2670437777 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64fc13cfc0 0x7f64fc1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.047+0000 7f64fb400640 1 -- 172.21.15.161:0/2670437777 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495601752 0 0) 0x7f64fc13d3a0 con 0x7f64fc15af60 2024-05-22T21:24:16.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.047+0000 7f64fb400640 1 -- 172.21.15.161:0/2670437777 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e0003530 con 0x7f64fc15af60 2024-05-22T21:24:16.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.047+0000 7f64fbe00640 1 -- 172.21.15.161:0/2670437777 >> v1:172.21.15.179:6789/0 conn(0x7f64fc13fe50 legacy=0x7f64fc06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.047+0000 7f64fb400640 1 -- 172.21.15.161:0/2670437777 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641265996 0 0) 0x7f64fc0cd920 con 0x7f64fc13fe50 2024-05-22T21:24:16.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.047+0000 7f64fbe00640 1 -- 172.21.15.161:0/2670437777 >> v1:172.21.15.161:6789/0 conn(0x7f64fc15af60 legacy=0x7f64fc13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.047+0000 7f64fbe00640 1 -- 172.21.15.161:0/2670437777 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc0cdee0 con 0x7f64fc13cfc0 2024-05-22T21:24:16.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.051+0000 7f64fbe00640 1 --2- 172.21.15.161:0/2670437777 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64fc13cfc0 0x7f64fc1402e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0097f0 tx=0x7f64ec01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b0633567c113ce server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:16.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.051+0000 7f64fb400640 1 -- 172.21.15.161:0/2670437777 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0ab420 con 0x7f64fc13cfc0 2024-05-22T21:24:16.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.051+0000 7f64fb400640 1 -- 172.21.15.161:0/2670437777 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ec0ab580 con 0x7f64fc13cfc0 2024-05-22T21:24:16.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.051+0000 7f64fb400640 1 -- 172.21.15.161:0/2670437777 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0ab910 con 0x7f64fc13cfc0 2024-05-22T21:24:16.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.051+0000 7f6502a00640 1 -- 172.21.15.161:0/2670437777 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64fc13cfc0 msgr2=0x7f64fc1402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.051+0000 7f6502a00640 1 --2- 172.21.15.161:0/2670437777 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64fc13cfc0 0x7f64fc1402e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0097f0 tx=0x7f64ec01c420 comp rx=0 tx=0).stop 2024-05-22T21:24:16.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.051+0000 7f6502a00640 1 -- 172.21.15.161:0/2670437777 shutdown_connections 2024-05-22T21:24:16.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.051+0000 7f6502a00640 1 --2- 172.21.15.161:0/2670437777 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64fc13cfc0 0x7f64fc1402e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.051+0000 7f6502a00640 1 -- 172.21.15.161:0/2670437777 >> 172.21.15.161:0/2670437777 conn(0x7f64fc06a2d0 msgr2=0x7f64fc06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:16.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.051+0000 7f6502a00640 1 -- 172.21.15.161:0/2670437777 shutdown_connections 2024-05-22T21:24:16.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.051+0000 7f6502a00640 1 -- 172.21.15.161:0/2670437777 wait complete. 2024-05-22T21:24:16.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f6502a00640 1 Processor -- start 2024-05-22T21:24:16.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f6502a00640 1 -- start start 2024-05-22T21:24:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f6502a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64fc13cfc0 0x7f64fc156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f6502a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64fc13fe50 0x7f64fc13a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f6502a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64fc15af60 0x7f64fc1395c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f6502a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc0b2180 con 0x7f64fc13cfc0 2024-05-22T21:24:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f6502a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc0ad7c0 con 0x7f64fc15af60 2024-05-22T21:24:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f6502a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc0adaa0 con 0x7f64fc13fe50 2024-05-22T21:24:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f64fbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64fc13cfc0 0x7f64fc156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f64fbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64fc13cfc0 0x7f64fc156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57002/0 (socket says 172.21.15.161:57002) 2024-05-22T21:24:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f64fbe00640 1 -- 172.21.15.161:0/3345335989 learned_addr learned my addr 172.21.15.161:0/3345335989 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:16.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f64f3400640 1 --2- 172.21.15.161:0/3345335989 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64fc13fe50 0x7f64fc13a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f6500a00640 1 --2- 172.21.15.161:0/3345335989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64fc15af60 0x7f64fc1395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f64fbe00640 1 -- 172.21.15.161:0/3345335989 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64fc13fe50 msgr2=0x7f64fc13a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f64fbe00640 1 --2- 172.21.15.161:0/3345335989 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64fc13fe50 0x7f64fc13a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f64fbe00640 1 -- 172.21.15.161:0/3345335989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64fc15af60 msgr2=0x7f64fc1395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f64fbe00640 1 --2- 172.21.15.161:0/3345335989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64fc15af60 0x7f64fc1395c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f64fbe00640 1 -- 172.21.15.161:0/3345335989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec009420 con 0x7f64fc13cfc0 2024-05-22T21:24:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f6500a00640 1 --2- 172.21.15.161:0/3345335989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64fc15af60 0x7f64fc1395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f64fbe00640 1 --2- 172.21.15.161:0/3345335989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64fc13cfc0 0x7f64fc156300 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0ad040 tx=0x7f64ec01ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f64f8c00640 1 -- 172.21.15.161:0/3345335989 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0abd40 con 0x7f64fc13cfc0 2024-05-22T21:24:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f64f8c00640 1 -- 172.21.15.161:0/3345335989 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ec0bf410 con 0x7f64fc13cfc0 2024-05-22T21:24:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f64f8c00640 1 -- 172.21.15.161:0/3345335989 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0abea0 con 0x7f64fc13cfc0 2024-05-22T21:24:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.055+0000 7f6502a00640 1 -- 172.21.15.161:0/3345335989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc0add80 con 0x7f64fc13cfc0 2024-05-22T21:24:16.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.059+0000 7f6502a00640 1 -- 172.21.15.161:0/3345335989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc15cbe0 con 0x7f64fc13cfc0 2024-05-22T21:24:16.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.059+0000 7f6502a00640 1 -- 172.21.15.161:0/3345335989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64fc06db30 con 0x7f64fc13cfc0 2024-05-22T21:24:16.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.059+0000 7f64f8c00640 1 -- 172.21.15.161:0/3345335989 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f64ec0bf7c0 con 0x7f64fc13cfc0 2024-05-22T21:24:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.059+0000 7f64f8c00640 1 --2- 172.21.15.161:0/3345335989 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f64d80666c0 0x7f64d8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.059+0000 7f64f8c00640 1 -- 172.21.15.161:0/3345335989 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f64ec11eb20 con 0x7f64fc13cfc0 2024-05-22T21:24:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.059+0000 7f64f8c00640 1 -- 172.21.15.161:0/3345335989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7f64d806e250 con 0x7f64fc13cfc0 2024-05-22T21:24:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.059+0000 7f64f3400640 1 --2- 172.21.15.161:0/3345335989 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f64d80666c0 0x7f64d8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.059+0000 7f64f3400640 1 --2- 172.21.15.161:0/3345335989 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f64d80666c0 0x7f64d8068b80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f64fc157fd0 tx=0x7f64e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:16.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.063+0000 7f64f8c00640 1 -- 172.21.15.161:0/3345335989 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f64ec0c4020 con 0x7f64fc13cfc0 2024-05-22T21:24:16.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.215+0000 7f6502a00640 1 -- 172.21.15.161:0/3345335989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64fc0adbe0 con 0x7f64fc13cfc0 2024-05-22T21:24:16.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.215+0000 7f64f8c00640 1 -- 172.21.15.161:0/3345335989 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v178) v1 ==== 104+0+11 (secure 0 0 0) 0x7f64fc0adbe0 con 0x7f64fc13cfc0 2024-05-22T21:24:16.219 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:24:16.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.219+0000 7f6502a00640 1 -- 172.21.15.161:0/3345335989 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f64d80666c0 msgr2=0x7f64d8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.219+0000 7f6502a00640 1 --2- 172.21.15.161:0/3345335989 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f64d80666c0 0x7f64d8068b80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f64fc157fd0 tx=0x7f64e4009040 comp rx=0 tx=0).stop 2024-05-22T21:24:16.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.219+0000 7f6502a00640 1 -- 172.21.15.161:0/3345335989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64fc13cfc0 msgr2=0x7f64fc156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.219+0000 7f6502a00640 1 --2- 172.21.15.161:0/3345335989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64fc13cfc0 0x7f64fc156300 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0ad040 tx=0x7f64ec01ac60 comp rx=0 tx=0).stop 2024-05-22T21:24:16.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.219+0000 7f6502a00640 1 -- 172.21.15.161:0/3345335989 shutdown_connections 2024-05-22T21:24:16.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.219+0000 7f6502a00640 1 --2- 172.21.15.161:0/3345335989 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f64d80666c0 0x7f64d8068b80 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.219+0000 7f6502a00640 1 --2- 172.21.15.161:0/3345335989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64fc15af60 0x7f64fc1395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.219+0000 7f6502a00640 1 --2- 172.21.15.161:0/3345335989 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64fc13fe50 0x7f64fc13a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.219+0000 7f6502a00640 1 --2- 172.21.15.161:0/3345335989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64fc13cfc0 0x7f64fc156300 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.219+0000 7f6502a00640 1 -- 172.21.15.161:0/3345335989 >> 172.21.15.161:0/3345335989 conn(0x7f64fc06a2d0 msgr2=0x7f64fc06cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:16.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.219+0000 7f6502a00640 1 -- 172.21.15.161:0/3345335989 shutdown_connections 2024-05-22T21:24:16.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.219+0000 7f6502a00640 1 -- 172.21.15.161:0/3345335989 wait complete. 2024-05-22T21:24:16.238 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:24:16.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.375+0000 7fd8d5a00640 1 Processor -- start 2024-05-22T21:24:16.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.375+0000 7fd8d5a00640 1 -- start start 2024-05-22T21:24:16.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8d5a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd8d00ce450 0x7fd8d006c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8d5a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d015d690 con 0x7fd8d00711d0 2024-05-22T21:24:16.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8d5a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d00cd830 con 0x7fd8d00cd330 2024-05-22T21:24:16.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8d5a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d00cdb10 con 0x7fd8d00ce450 2024-05-22T21:24:16.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8d5000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd8d00711d0 0x7fd8d00cb020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56854/0 (socket says 172.21.15.161:56854) 2024-05-22T21:24:16.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8d5000640 1 -- 172.21.15.161:0/1610399109 learned_addr learned my addr 172.21.15.161:0/1610399109 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:16.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cfe00640 1 --2- 172.21.15.161:0/1610399109 >> v2:172.21.15.179:3300/0 conn(0x7fd8d00ce450 0x7fd8d006c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cea00640 1 -- 172.21.15.161:0/1610399109 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405481318 0 0) 0x7fd8d015d690 con 0x7fd8d00711d0 2024-05-22T21:24:16.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cea00640 1 -- 172.21.15.161:0/1610399109 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8b4003530 con 0x7fd8d00711d0 2024-05-22T21:24:16.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cea00640 1 -- 172.21.15.161:0/1610399109 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3947534666 0 0) 0x7fd8b4003530 con 0x7fd8d00711d0 2024-05-22T21:24:16.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cea00640 1 -- 172.21.15.161:0/1610399109 >> v2:172.21.15.179:3300/0 conn(0x7fd8d00ce450 msgr2=0x7fd8d006c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:16.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cea00640 1 --2- 172.21.15.161:0/1610399109 >> v2:172.21.15.179:3300/0 conn(0x7fd8d00ce450 0x7fd8d006c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cea00640 1 -- 172.21.15.161:0/1610399109 >> v1:172.21.15.179:6789/0 conn(0x7fd8d00cd330 legacy=0x7fd8d00ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cea00640 1 -- 172.21.15.161:0/1610399109 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d00cddf0 con 0x7fd8d00711d0 2024-05-22T21:24:16.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cea00640 1 -- 172.21.15.161:0/1610399109 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fd8c8001a00 con 0x7fd8d00711d0 2024-05-22T21:24:16.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cea00640 1 -- 172.21.15.161:0/1610399109 >> v1:172.21.15.161:6789/0 conn(0x7fd8d00711d0 legacy=0x7fd8d00cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cea00640 1 --2- 172.21.15.161:0/1610399109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8b4003980 0x7fd8b4003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cea00640 1 --2- 172.21.15.161:0/1610399109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4004380 0x7fd8b4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cea00640 1 --2- 172.21.15.161:0/1610399109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8b4005080 0x7fd8b400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cea00640 1 -- 172.21.15.161:0/1610399109 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00cdb10 con 0x7fd8b4004380 2024-05-22T21:24:16.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cea00640 1 -- 172.21.15.161:0/1610399109 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b400bf30 con 0x7fd8b4005080 2024-05-22T21:24:16.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cea00640 1 -- 172.21.15.161:0/1610399109 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b400c0f0 con 0x7fd8b4003980 2024-05-22T21:24:16.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cfe00640 1 --2- 172.21.15.161:0/1610399109 >> v2:172.21.15.179:3300/0 conn(0x7fd8d00ce450 0x7fd8d006c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:24:16.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cf400640 1 --2- 172.21.15.161:0/1610399109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4004380 0x7fd8b4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8d5000640 1 --2- 172.21.15.161:0/1610399109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8b4005080 0x7fd8b400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cfe00640 1 --2- 172.21.15.161:0/1610399109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8b4003980 0x7fd8b4003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cf400640 1 -- 172.21.15.161:0/1610399109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8b4003980 msgr2=0x7fd8b4003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:16.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cf400640 1 --2- 172.21.15.161:0/1610399109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8b4003980 0x7fd8b4003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cf400640 1 -- 172.21.15.161:0/1610399109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8b4005080 msgr2=0x7fd8b400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:16.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cf400640 1 --2- 172.21.15.161:0/1610399109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8b4005080 0x7fd8b400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8cf400640 1 -- 172.21.15.161:0/1610399109 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b4003530 con 0x7fd8b4004380 2024-05-22T21:24:16.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.379+0000 7fd8d5000640 1 -- 172.21.15.161:0/1610399109 reap_dead start 2024-05-22T21:24:16.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.383+0000 7fd8cf400640 1 --2- 172.21.15.161:0/1610399109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4004380 0x7fd8b4004850 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc007fd0 tx=0x7fd8bc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ea166dcb289221 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:16.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.383+0000 7fd8cea00640 1 -- 172.21.15.161:0/1610399109 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc0ac620 con 0x7fd8b4004380 2024-05-22T21:24:16.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.383+0000 7fd8cea00640 1 -- 172.21.15.161:0/1610399109 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc0ac780 con 0x7fd8b4004380 2024-05-22T21:24:16.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.383+0000 7fd8cea00640 1 -- 172.21.15.161:0/1610399109 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc0acb70 con 0x7fd8b4004380 2024-05-22T21:24:16.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.383+0000 7fd8d5a00640 1 -- 172.21.15.161:0/1610399109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4004380 msgr2=0x7fd8b4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.383+0000 7fd8d5a00640 1 --2- 172.21.15.161:0/1610399109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4004380 0x7fd8b4004850 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc007fd0 tx=0x7fd8bc01c480 comp rx=0 tx=0).stop 2024-05-22T21:24:16.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.383+0000 7fd8d5a00640 1 -- 172.21.15.161:0/1610399109 shutdown_connections 2024-05-22T21:24:16.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.383+0000 7fd8d5a00640 1 --2- 172.21.15.161:0/1610399109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4004380 0x7fd8b4004850 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.383+0000 7fd8d5a00640 1 -- 172.21.15.161:0/1610399109 >> 172.21.15.161:0/1610399109 conn(0x7fd8d00703f0 msgr2=0x7fd8d0070800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:16.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.383+0000 7fd8d5a00640 1 -- 172.21.15.161:0/1610399109 shutdown_connections 2024-05-22T21:24:16.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.383+0000 7fd8d5a00640 1 -- 172.21.15.161:0/1610399109 wait complete. 2024-05-22T21:24:16.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.383+0000 7fd8d5a00640 1 Processor -- start 2024-05-22T21:24:16.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.383+0000 7fd8d5a00640 1 -- start start 2024-05-22T21:24:16.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8d5a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4004380 0x7fd8d01e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8d5a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8d0158550 0x7fd8d01589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8d5a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d0158f10 0x7fd8d0143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8d5a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d0159380 con 0x7fd8b4004380 2024-05-22T21:24:16.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8d5a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d0145bd0 con 0x7fd8d0158f10 2024-05-22T21:24:16.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8d5a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d0142d20 con 0x7fd8d0158550 2024-05-22T21:24:16.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8cfe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4004380 0x7fd8d01e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8cfe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4004380 0x7fd8d01e71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57014/0 (socket says 172.21.15.161:57014) 2024-05-22T21:24:16.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8cfe00640 1 -- 172.21.15.161:0/3376937732 learned_addr learned my addr 172.21.15.161:0/3376937732 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:16.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8cfe00640 1 -- 172.21.15.161:0/3376937732 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8d0158550 msgr2=0x7fd8d01589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8cfe00640 1 --2- 172.21.15.161:0/3376937732 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8d0158550 0x7fd8d01589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8cfe00640 1 -- 172.21.15.161:0/3376937732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d0158f10 msgr2=0x7fd8d0143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8d5000640 1 --2- 172.21.15.161:0/3376937732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d0158f10 0x7fd8d0143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8cfe00640 1 --2- 172.21.15.161:0/3376937732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d0158f10 0x7fd8d0143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8cfe00640 1 -- 172.21.15.161:0/3376937732 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d0143000 con 0x7fd8b4004380 2024-05-22T21:24:16.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8d5000640 1 --2- 172.21.15.161:0/3376937732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d0158f10 0x7fd8d0143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:16.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8cfe00640 1 --2- 172.21.15.161:0/3376937732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4004380 0x7fd8d01e71f0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0011800 tx=0x7fd8c0011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:16.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8c7e00640 1 -- 172.21.15.161:0/3376937732 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c000f9c0 con 0x7fd8b4004380 2024-05-22T21:24:16.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8c7e00640 1 -- 172.21.15.161:0/3376937732 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c000fb20 con 0x7fd8b4004380 2024-05-22T21:24:16.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8d5a00640 1 -- 172.21.15.161:0/3376937732 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d01432e0 con 0x7fd8b4004380 2024-05-22T21:24:16.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8c7e00640 1 -- 172.21.15.161:0/3376937732 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c000fe90 con 0x7fd8b4004380 2024-05-22T21:24:16.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8d5a00640 1 -- 172.21.15.161:0/3376937732 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d01438a0 con 0x7fd8b4004380 2024-05-22T21:24:16.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8d5a00640 1 -- 172.21.15.161:0/3376937732 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d0067070 con 0x7fd8b4004380 2024-05-22T21:24:16.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.387+0000 7fd8c7e00640 1 -- 172.21.15.161:0/3376937732 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd8c0015460 con 0x7fd8b4004380 2024-05-22T21:24:16.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.391+0000 7fd8c7e00640 1 --2- 172.21.15.161:0/3376937732 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd8b0066690 0x7fd8b0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.391+0000 7fd8c7e00640 1 -- 172.21.15.161:0/3376937732 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd8c0084c40 con 0x7fd8b4004380 2024-05-22T21:24:16.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.391+0000 7fd8cf400640 1 --2- 172.21.15.161:0/3376937732 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd8b0066690 0x7fd8b0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.391+0000 7fd8c7e00640 1 -- 172.21.15.161:0/3376937732 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7fd8b006e220 con 0x7fd8b4004380 2024-05-22T21:24:16.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.391+0000 7fd8cf400640 1 --2- 172.21.15.161:0/3376937732 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd8b0066690 0x7fd8b0068b50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0ae000 tx=0x7fd8bc001240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:16.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.395+0000 7fd8c7e00640 1 -- 172.21.15.161:0/3376937732 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd8c0051fa0 con 0x7fd8b4004380 2024-05-22T21:24:16.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.547+0000 7fd8d5a00640 1 -- 172.21.15.161:0/3376937732 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8d00af430 con 0x7fd8b4004380 2024-05-22T21:24:16.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.547+0000 7fd8c7e00640 1 -- 172.21.15.161:0/3376937732 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v178) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd8c00555c0 con 0x7fd8b4004380 2024-05-22T21:24:16.549 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:24:16.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.551+0000 7fd8d5a00640 1 -- 172.21.15.161:0/3376937732 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd8b0066690 msgr2=0x7fd8b0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.551+0000 7fd8d5a00640 1 --2- 172.21.15.161:0/3376937732 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd8b0066690 0x7fd8b0068b50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0ae000 tx=0x7fd8bc001240 comp rx=0 tx=0).stop 2024-05-22T21:24:16.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.551+0000 7fd8d5a00640 1 -- 172.21.15.161:0/3376937732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4004380 msgr2=0x7fd8d01e71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.551+0000 7fd8d5a00640 1 --2- 172.21.15.161:0/3376937732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4004380 0x7fd8d01e71f0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0011800 tx=0x7fd8c0011cc0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.551+0000 7fd8d5a00640 1 -- 172.21.15.161:0/3376937732 shutdown_connections 2024-05-22T21:24:16.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.551+0000 7fd8d5a00640 1 --2- 172.21.15.161:0/3376937732 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd8b0066690 0x7fd8b0068b50 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.551+0000 7fd8d5a00640 1 --2- 172.21.15.161:0/3376937732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8d0158f10 0x7fd8d0143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.551+0000 7fd8d5a00640 1 --2- 172.21.15.161:0/3376937732 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8d0158550 0x7fd8d01589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.551+0000 7fd8d5a00640 1 --2- 172.21.15.161:0/3376937732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4004380 0x7fd8d01e71f0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.551+0000 7fd8d5a00640 1 -- 172.21.15.161:0/3376937732 >> 172.21.15.161:0/3376937732 conn(0x7fd8d00703f0 msgr2=0x7fd8d006c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:16.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.551+0000 7fd8d5a00640 1 -- 172.21.15.161:0/3376937732 shutdown_connections 2024-05-22T21:24:16.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.551+0000 7fd8d5a00640 1 -- 172.21.15.161:0/3376937732 wait complete. 2024-05-22T21:24:16.570 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:24:16.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.707+0000 7f0d15800640 1 Processor -- start 2024-05-22T21:24:16.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.707+0000 7f0d15800640 1 -- start start 2024-05-22T21:24:16.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.707+0000 7f0d15800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d1015b600 0x7f0d1006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d15800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d10144bb0 con 0x7f0d10143c40 2024-05-22T21:24:16.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d15800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1015efb0 con 0x7f0d1006c7e0 2024-05-22T21:24:16.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d15800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1015ba60 con 0x7f0d1015b600 2024-05-22T21:24:16.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0ea00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0d10143c40 0x7f0d101444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56868/0 (socket says 172.21.15.161:56868) 2024-05-22T21:24:16.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0ea00640 1 -- 172.21.15.161:0/521140506 learned_addr learned my addr 172.21.15.161:0/521140506 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:16.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0e000640 1 -- 172.21.15.161:0/521140506 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050029764 0 0) 0x7f0d10144bb0 con 0x7f0d10143c40 2024-05-22T21:24:16.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0e000640 1 -- 172.21.15.161:0/521140506 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cf4003530 con 0x7f0d10143c40 2024-05-22T21:24:16.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d06000640 1 --2- 172.21.15.161:0/521140506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d1015b600 0x7f0d1006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0e000640 1 -- 172.21.15.161:0/521140506 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2347195563 0 0) 0x7f0cf4003530 con 0x7f0d10143c40 2024-05-22T21:24:16.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0e000640 1 -- 172.21.15.161:0/521140506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d1015b600 msgr2=0x7f0d1006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:16.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0e000640 1 --2- 172.21.15.161:0/521140506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d1015b600 0x7f0d1006c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0e000640 1 -- 172.21.15.161:0/521140506 >> v1:172.21.15.179:6789/0 conn(0x7f0d1006c7e0 legacy=0x7f0d100b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0e000640 1 -- 172.21.15.161:0/521140506 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1015f290 con 0x7f0d10143c40 2024-05-22T21:24:16.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0e000640 1 -- 172.21.15.161:0/521140506 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f0d00001a00 con 0x7f0d10143c40 2024-05-22T21:24:16.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0e000640 1 -- 172.21.15.161:0/521140506 >> v1:172.21.15.161:6789/0 conn(0x7f0d10143c40 legacy=0x7f0d101444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0e000640 1 --2- 172.21.15.161:0/521140506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf40039b0 0x7f0cf4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0e000640 1 --2- 172.21.15.161:0/521140506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d1015b600 0x7f0cf40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0e000640 1 --2- 172.21.15.161:0/521140506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf4004d10 0x7f0cf4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0e000640 1 -- 172.21.15.161:0/521140506 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1015ba60 con 0x7f0cf40039b0 2024-05-22T21:24:16.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0e000640 1 -- 172.21.15.161:0/521140506 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf400bb20 con 0x7f0cf4004d10 2024-05-22T21:24:16.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0e000640 1 -- 172.21.15.161:0/521140506 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf400bcb0 con 0x7f0d1015b600 2024-05-22T21:24:16.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0ea00640 1 --2- 172.21.15.161:0/521140506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf40039b0 0x7f0cf4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0f400640 1 --2- 172.21.15.161:0/521140506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf4004d10 0x7f0cf4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d06000640 1 --2- 172.21.15.161:0/521140506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d1015b600 0x7f0cf40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0ea00640 1 -- 172.21.15.161:0/521140506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d1015b600 msgr2=0x7f0cf40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:16.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0ea00640 1 --2- 172.21.15.161:0/521140506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d1015b600 0x7f0cf40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0ea00640 1 -- 172.21.15.161:0/521140506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf4004d10 msgr2=0x7f0cf4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:16.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0ea00640 1 --2- 172.21.15.161:0/521140506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf4004d10 0x7f0cf4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0ea00640 1 -- 172.21.15.161:0/521140506 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf4003530 con 0x7f0cf40039b0 2024-05-22T21:24:16.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d06000640 1 --2- 172.21.15.161:0/521140506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d1015b600 0x7f0cf40047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:16.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.711+0000 7f0d0f400640 1 --2- 172.21.15.161:0/521140506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf4004d10 0x7f0cf4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:16.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.715+0000 7f0d0ea00640 1 --2- 172.21.15.161:0/521140506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf40039b0 0x7f0cf4003dd0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000a860 tx=0x7f0d0001d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5786d44b4b9d60b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:16.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.715+0000 7f0d0e000640 1 -- 172.21.15.161:0/521140506 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d000072d0 con 0x7f0cf40039b0 2024-05-22T21:24:16.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.715+0000 7f0d0e000640 1 -- 172.21.15.161:0/521140506 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d000a8070 con 0x7f0cf40039b0 2024-05-22T21:24:16.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.715+0000 7f0d0e000640 1 -- 172.21.15.161:0/521140506 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d000076b0 con 0x7f0cf40039b0 2024-05-22T21:24:16.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.715+0000 7f0d15800640 1 -- 172.21.15.161:0/521140506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf40039b0 msgr2=0x7f0cf4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.715+0000 7f0d15800640 1 --2- 172.21.15.161:0/521140506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf40039b0 0x7f0cf4003dd0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000a860 tx=0x7f0d0001d340 comp rx=0 tx=0).stop 2024-05-22T21:24:16.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.715+0000 7f0d0f400640 1 -- 172.21.15.161:0/521140506 reap_dead start 2024-05-22T21:24:16.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.715+0000 7f0d15800640 1 -- 172.21.15.161:0/521140506 shutdown_connections 2024-05-22T21:24:16.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.715+0000 7f0d15800640 1 -- 172.21.15.161:0/521140506 >> 172.21.15.161:0/521140506 conn(0x7f0d1006a2d0 msgr2=0x7f0d1006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:16.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.715+0000 7f0d15800640 1 -- 172.21.15.161:0/521140506 shutdown_connections 2024-05-22T21:24:16.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.715+0000 7f0d15800640 1 -- 172.21.15.161:0/521140506 wait complete. 2024-05-22T21:24:16.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.715+0000 7f0d15800640 1 Processor -- start 2024-05-22T21:24:16.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.715+0000 7f0d15800640 1 -- start start 2024-05-22T21:24:16.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d15800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf40039b0 0x7f0d1015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d15800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d100add60 0x7f0d101f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d15800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d101f7b60 0x7f0d101f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d15800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100ae1e0 con 0x7f0d100add60 2024-05-22T21:24:16.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d15800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100ad410 con 0x7f0cf40039b0 2024-05-22T21:24:16.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d15800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100ac650 con 0x7f0d101f7b60 2024-05-22T21:24:16.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d06000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d100add60 0x7f0d101f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d06000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d100add60 0x7f0d101f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57032/0 (socket says 172.21.15.161:57032) 2024-05-22T21:24:16.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d06000640 1 -- 172.21.15.161:0/3030917013 learned_addr learned my addr 172.21.15.161:0/3030917013 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:16.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d0ea00640 1 --2- 172.21.15.161:0/3030917013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf40039b0 0x7f0d1015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d0f400640 1 --2- 172.21.15.161:0/3030917013 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d101f7b60 0x7f0d101f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d06000640 1 -- 172.21.15.161:0/3030917013 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d101f7b60 msgr2=0x7f0d101f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d06000640 1 --2- 172.21.15.161:0/3030917013 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d101f7b60 0x7f0d101f8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d06000640 1 -- 172.21.15.161:0/3030917013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf40039b0 msgr2=0x7f0d1015a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d06000640 1 --2- 172.21.15.161:0/3030917013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf40039b0 0x7f0d1015a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d06000640 1 -- 172.21.15.161:0/3030917013 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d100ac930 con 0x7f0d100add60 2024-05-22T21:24:16.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d06000640 1 --2- 172.21.15.161:0/3030917013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d100add60 0x7f0d101f7620 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800f510 tx=0x7f0cf800f910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:16.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d07400640 1 -- 172.21.15.161:0/3030917013 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8014870 con 0x7f0d100add60 2024-05-22T21:24:16.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d07400640 1 -- 172.21.15.161:0/3030917013 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf80149d0 con 0x7f0d100add60 2024-05-22T21:24:16.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d07400640 1 -- 172.21.15.161:0/3030917013 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8014d60 con 0x7f0d100add60 2024-05-22T21:24:16.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d15800640 1 -- 172.21.15.161:0/3030917013 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d100acc10 con 0x7f0d100add60 2024-05-22T21:24:16.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d15800640 1 -- 172.21.15.161:0/3030917013 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1015b150 con 0x7f0d100add60 2024-05-22T21:24:16.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d15800640 1 -- 172.21.15.161:0/3030917013 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d10067220 con 0x7f0d100add60 2024-05-22T21:24:16.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.719+0000 7f0d07400640 1 -- 172.21.15.161:0/3030917013 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0cf801d400 con 0x7f0d100add60 2024-05-22T21:24:16.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.723+0000 7f0d07400640 1 --2- 172.21.15.161:0/3030917013 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0cec0666c0 0x7f0cec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:16.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.723+0000 7f0d07400640 1 -- 172.21.15.161:0/3030917013 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f0cf80846c0 con 0x7f0d100add60 2024-05-22T21:24:16.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.723+0000 7f0d0ea00640 1 --2- 172.21.15.161:0/3030917013 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0cec0666c0 0x7f0cec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:16.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.723+0000 7f0d07400640 1 -- 172.21.15.161:0/3030917013 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7f0cec06e250 con 0x7f0d100add60 2024-05-22T21:24:16.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.723+0000 7f0d0ea00640 1 --2- 172.21.15.161:0/3030917013 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0cec0666c0 0x7f0cec068b80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0d000ad000 tx=0x7f0d00009050 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:16.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.727+0000 7f0d07400640 1 -- 172.21.15.161:0/3030917013 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0cf8051a00 con 0x7f0d100add60 2024-05-22T21:24:16.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.891+0000 7f0d15800640 1 -- 172.21.15.161:0/3030917013 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0d100ceaa0 con 0x7f0d100add60 2024-05-22T21:24:16.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.891+0000 7f0d07400640 1 -- 172.21.15.161:0/3030917013 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v178) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0cf8055020 con 0x7f0d100add60 2024-05-22T21:24:16.890 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:24:16.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.891+0000 7f0d15800640 1 -- 172.21.15.161:0/3030917013 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0cec0666c0 msgr2=0x7f0cec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.891+0000 7f0d15800640 1 --2- 172.21.15.161:0/3030917013 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0cec0666c0 0x7f0cec068b80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0d000ad000 tx=0x7f0d00009050 comp rx=0 tx=0).stop 2024-05-22T21:24:16.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.891+0000 7f0d15800640 1 -- 172.21.15.161:0/3030917013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d100add60 msgr2=0x7f0d101f7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:16.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.891+0000 7f0d15800640 1 --2- 172.21.15.161:0/3030917013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d100add60 0x7f0d101f7620 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800f510 tx=0x7f0cf800f910 comp rx=0 tx=0).stop 2024-05-22T21:24:16.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.891+0000 7f0d15800640 1 -- 172.21.15.161:0/3030917013 shutdown_connections 2024-05-22T21:24:16.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.891+0000 7f0d15800640 1 --2- 172.21.15.161:0/3030917013 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0cec0666c0 0x7f0cec068b80 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.891+0000 7f0d15800640 1 --2- 172.21.15.161:0/3030917013 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d101f7b60 0x7f0d101f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.891+0000 7f0d15800640 1 --2- 172.21.15.161:0/3030917013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d100add60 0x7f0d101f7620 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.891+0000 7f0d15800640 1 --2- 172.21.15.161:0/3030917013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0cf40039b0 0x7f0d1015a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:16.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.891+0000 7f0d15800640 1 -- 172.21.15.161:0/3030917013 >> 172.21.15.161:0/3030917013 conn(0x7f0d1006a2d0 msgr2=0x7f0d100b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:16.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.891+0000 7f0d15800640 1 -- 172.21.15.161:0/3030917013 shutdown_connections 2024-05-22T21:24:16.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:16.891+0000 7f0d15800640 1 -- 172.21.15.161:0/3030917013 wait complete. 2024-05-22T21:24:16.911 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-22T21:24:16.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:16.912 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-22T21:24:16.938 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:24:16.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:16.939 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-22T21:24:16.990 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:24:16.990 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:24:17.000 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:24:17.000 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:24:17.050 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:24:17.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.183+0000 7f57fb000640 1 Processor -- start 2024-05-22T21:24:17.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.187+0000 7f57fb000640 1 -- start start 2024-05-22T21:24:17.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.187+0000 7f57fb000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f57f4143760 0x7f57f4070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.187+0000 7f57fb000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f57f406bfc0 0x7f57f4071180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.187+0000 7f57fb000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f4143b40 con 0x7f57f415e640 2024-05-22T21:24:17.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.187+0000 7f57fb000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f406c7c0 con 0x7f57f406bfc0 2024-05-22T21:24:17.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.187+0000 7f57fb000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40b3380 con 0x7f57f4143760 2024-05-22T21:24:17.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.187+0000 7f57fa600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f57f406bfc0 0x7f57f4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:17.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.187+0000 7f57f9200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f57f415e640 0x7f57f406b8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56874/0 (socket says 172.21.15.161:56874) 2024-05-22T21:24:17.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.187+0000 7f57fa600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f57f406bfc0 0x7f57f4071180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57044/0 (socket says 172.21.15.161:57044) 2024-05-22T21:24:17.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.187+0000 7f57f9200640 1 -- 172.21.15.161:0/209614246 learned_addr learned my addr 172.21.15.161:0/209614246 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:17.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.187+0000 7f57f9c00640 1 --2- 172.21.15.161:0/209614246 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f57f4143760 0x7f57f4070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:17.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.187+0000 7f57fa600640 1 -- 172.21.15.161:0/209614246 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f57f4143760 msgr2=0x7f57f4070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:17.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.187+0000 7f57fa600640 1 --2- 172.21.15.161:0/209614246 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f57f4143760 0x7f57f4070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.187+0000 7f57fa600640 1 -- 172.21.15.161:0/209614246 >> v1:172.21.15.161:6789/0 conn(0x7f57f415e640 legacy=0x7f57f406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:17.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.187+0000 7f57f3e00640 1 -- 172.21.15.161:0/209614246 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918540868 0 0) 0x7f57f4143b40 con 0x7f57f415e640 2024-05-22T21:24:17.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.187+0000 7f57fa600640 1 -- 172.21.15.161:0/209614246 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f406c4e0 con 0x7f57f406bfc0 2024-05-22T21:24:17.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.191+0000 7f57fa600640 1 --2- 172.21.15.161:0/209614246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57f406bfc0 0x7f57f4071180 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009dc0 tx=0x7f57ec01c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c46dbeb6f07806b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:17.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.191+0000 7f57f3e00640 1 -- 172.21.15.161:0/209614246 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0a5920 con 0x7f57f406bfc0 2024-05-22T21:24:17.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.191+0000 7f57f3e00640 1 -- 172.21.15.161:0/209614246 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec0a5a80 con 0x7f57f406bfc0 2024-05-22T21:24:17.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.191+0000 7f57f3e00640 1 -- 172.21.15.161:0/209614246 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0a5e10 con 0x7f57f406bfc0 2024-05-22T21:24:17.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.191+0000 7f57fb000640 1 -- 172.21.15.161:0/209614246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57f406bfc0 msgr2=0x7f57f4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:17.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.191+0000 7f57fb000640 1 --2- 172.21.15.161:0/209614246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57f406bfc0 0x7f57f4071180 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009dc0 tx=0x7f57ec01c860 comp rx=0 tx=0).stop 2024-05-22T21:24:17.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.191+0000 7f57fb000640 1 -- 172.21.15.161:0/209614246 shutdown_connections 2024-05-22T21:24:17.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.191+0000 7f57fb000640 1 --2- 172.21.15.161:0/209614246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57f406bfc0 0x7f57f4071180 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.191+0000 7f57fb000640 1 --2- 172.21.15.161:0/209614246 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f57f4143760 0x7f57f4070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.191+0000 7f57fb000640 1 -- 172.21.15.161:0/209614246 >> 172.21.15.161:0/209614246 conn(0x7f57f406a4d0 msgr2=0x7f57f40b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:17.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.191+0000 7f57fb000640 1 -- 172.21.15.161:0/209614246 shutdown_connections 2024-05-22T21:24:17.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.191+0000 7f57fb000640 1 -- 172.21.15.161:0/209614246 wait complete. 2024-05-22T21:24:17.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.191+0000 7f57fb000640 1 Processor -- start 2024-05-22T21:24:17.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.191+0000 7f57fb000640 1 -- start start 2024-05-22T21:24:17.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fb000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f57f406bfc0 0x7f57f40ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fb000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57f4143760 0x7f57f413db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fb000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57f415e640 0x7f57f4158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fb000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40c3fc0 con 0x7f57f415e640 2024-05-22T21:24:17.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fb000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f4159cc0 con 0x7f57f4143760 2024-05-22T21:24:17.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fb000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f4159240 con 0x7f57f406bfc0 2024-05-22T21:24:17.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fa600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57f415e640 0x7f57f4158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:17.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fa600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57f415e640 0x7f57f4158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57048/0 (socket says 172.21.15.161:57048) 2024-05-22T21:24:17.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fa600640 1 -- 172.21.15.161:0/278944453 learned_addr learned my addr 172.21.15.161:0/278944453 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:17.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fa600640 1 -- 172.21.15.161:0/278944453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f57f406bfc0 msgr2=0x7f57f40ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:17.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fa600640 1 --2- 172.21.15.161:0/278944453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f57f406bfc0 0x7f57f40ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fa600640 1 -- 172.21.15.161:0/278944453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57f4143760 msgr2=0x7f57f413db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:17.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fa600640 1 --2- 172.21.15.161:0/278944453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57f4143760 0x7f57f413db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fa600640 1 -- 172.21.15.161:0/278944453 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec009a70 con 0x7f57f415e640 2024-05-22T21:24:17.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fa600640 1 --2- 172.21.15.161:0/278944453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57f415e640 0x7f57f4158d00 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0b0040 tx=0x7f57ec019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:17.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57f1600640 1 -- 172.21.15.161:0/278944453 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0ad850 con 0x7f57f415e640 2024-05-22T21:24:17.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57f1600640 1 -- 172.21.15.161:0/278944453 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec0ad9d0 con 0x7f57f415e640 2024-05-22T21:24:17.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57f1600640 1 -- 172.21.15.161:0/278944453 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0add80 con 0x7f57f415e640 2024-05-22T21:24:17.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fb000640 1 -- 172.21.15.161:0/278944453 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f40c3c70 con 0x7f57f415e640 2024-05-22T21:24:17.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fb000640 1 -- 172.21.15.161:0/278944453 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f41f6850 con 0x7f57f415e640 2024-05-22T21:24:17.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57fb000640 1 -- 172.21.15.161:0/278944453 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f40c4170 con 0x7f57f415e640 2024-05-22T21:24:17.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.195+0000 7f57f1600640 1 -- 172.21.15.161:0/278944453 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f57ec0b4050 con 0x7f57f415e640 2024-05-22T21:24:17.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.199+0000 7f57f1600640 1 --2- 172.21.15.161:0/278944453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f57d4066690 0x7f57d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.199+0000 7f57f1600640 1 -- 172.21.15.161:0/278944453 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f57ec11f190 con 0x7f57f415e640 2024-05-22T21:24:17.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.199+0000 7f57f9c00640 1 --2- 172.21.15.161:0/278944453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f57d4066690 0x7f57d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:17.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.199+0000 7f57f1600640 1 -- 172.21.15.161:0/278944453 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7f57d406e220 con 0x7f57f415e640 2024-05-22T21:24:17.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.199+0000 7f57f9c00640 1 --2- 172.21.15.161:0/278944453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f57d4066690 0x7f57d4068b50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f57f4073370 tx=0x7f57e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:17.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.203+0000 7f57f1600640 1 -- 172.21.15.161:0/278944453 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f57ec0ec6b0 con 0x7f57f415e640 2024-05-22T21:24:17.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.359+0000 7f57fb000640 1 -- 172.21.15.161:0/278944453 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57f41f6a40 con 0x7f57f415e640 2024-05-22T21:24:17.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.359+0000 7f57f1600640 1 -- 172.21.15.161:0/278944453 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+16235 (secure 0 0 0) 0x7f57ec0c7020 con 0x7f57f415e640 2024-05-22T21:24:17.361 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:17.361 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":178,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:23:50.299404+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:23:02.242525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-05-22T21:23:02.864916+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:24:17.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.363+0000 7f57fb000640 1 -- 172.21.15.161:0/278944453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f57d4066690 msgr2=0x7f57d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:17.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.363+0000 7f57fb000640 1 --2- 172.21.15.161:0/278944453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f57d4066690 0x7f57d4068b50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f57f4073370 tx=0x7f57e8008040 comp rx=0 tx=0).stop 2024-05-22T21:24:17.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.363+0000 7f57fb000640 1 -- 172.21.15.161:0/278944453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57f415e640 msgr2=0x7f57f4158d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:17.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.363+0000 7f57fb000640 1 --2- 172.21.15.161:0/278944453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57f415e640 0x7f57f4158d00 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0b0040 tx=0x7f57ec019600 comp rx=0 tx=0).stop 2024-05-22T21:24:17.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.363+0000 7f57fb000640 1 -- 172.21.15.161:0/278944453 shutdown_connections 2024-05-22T21:24:17.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.363+0000 7f57fb000640 1 --2- 172.21.15.161:0/278944453 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f57d4066690 0x7f57d4068b50 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.363+0000 7f57fb000640 1 --2- 172.21.15.161:0/278944453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57f415e640 0x7f57f4158d00 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.363+0000 7f57fb000640 1 --2- 172.21.15.161:0/278944453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57f4143760 0x7f57f413db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.363+0000 7f57fb000640 1 --2- 172.21.15.161:0/278944453 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f57f406bfc0 0x7f57f40ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.363+0000 7f57fb000640 1 -- 172.21.15.161:0/278944453 >> 172.21.15.161:0/278944453 conn(0x7f57f406a4d0 msgr2=0x7f57f40b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:17.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.363+0000 7f57fb000640 1 -- 172.21.15.161:0/278944453 shutdown_connections 2024-05-22T21:24:17.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.363+0000 7f57fb000640 1 -- 172.21.15.161:0/278944453 wait complete. 2024-05-22T21:24:17.383 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:23:02.242525+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '174', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:23:02.864916+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '178', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:24:17.383 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:24:17.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.519+0000 7fa9e6400640 1 Processor -- start 2024-05-22T21:24:17.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.519+0000 7fa9e6400640 1 -- start start 2024-05-22T21:24:17.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.523+0000 7fa9e6400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa9e0143c40 0x7fa9e015b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.523+0000 7fa9e6400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa9e006c9b0 0x7fa9e00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.523+0000 7fa9e6400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e015efb0 con 0x7fa9e01444a0 2024-05-22T21:24:17.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.523+0000 7fa9e6400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e015f290 con 0x7fa9e0143c40 2024-05-22T21:24:17.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.523+0000 7fa9e6400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e015f570 con 0x7fa9e006c9b0 2024-05-22T21:24:17.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.523+0000 7fa9df400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa9e0143c40 0x7fa9e015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:17.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.523+0000 7fa9df400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa9e0143c40 0x7fa9e015b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57050/0 (socket says 172.21.15.161:57050) 2024-05-22T21:24:17.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.523+0000 7fa9df400640 1 -- 172.21.15.161:0/2135445740 learned_addr learned my addr 172.21.15.161:0/2135445740 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:17.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.523+0000 7fa9dfe00640 1 --2- 172.21.15.161:0/2135445740 >> v2:172.21.15.179:3300/0 conn(0x7fa9e006c9b0 0x7fa9e00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:17.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.523+0000 7fa9df400640 1 -- 172.21.15.161:0/2135445740 >> v2:172.21.15.179:3300/0 conn(0x7fa9e006c9b0 msgr2=0x7fa9e00b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:17.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.523+0000 7fa9df400640 1 --2- 172.21.15.161:0/2135445740 >> v2:172.21.15.179:3300/0 conn(0x7fa9e006c9b0 0x7fa9e00b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.523+0000 7fa9df400640 1 -- 172.21.15.161:0/2135445740 >> v1:172.21.15.179:6789/0 conn(0x7fa9e01444a0 legacy=0x7fa9e006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:17.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.523+0000 7fa9df400640 1 -- 172.21.15.161:0/2135445740 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e015f850 con 0x7fa9e0143c40 2024-05-22T21:24:17.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.527+0000 7fa9df400640 1 --2- 172.21.15.161:0/2135445740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9e0143c40 0x7fa9e015b600 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00aead0 tx=0x7fa9d001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92477182d7bf3a75 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.527+0000 7fa9de000640 1 -- 172.21.15.161:0/2135445740 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00a5820 con 0x7fa9e0143c40 2024-05-22T21:24:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.527+0000 7fa9de000640 1 -- 172.21.15.161:0/2135445740 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d00a5980 con 0x7fa9e0143c40 2024-05-22T21:24:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.527+0000 7fa9de000640 1 -- 172.21.15.161:0/2135445740 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00a5d10 con 0x7fa9e0143c40 2024-05-22T21:24:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.527+0000 7fa9e6400640 1 -- 172.21.15.161:0/2135445740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9e0143c40 msgr2=0x7fa9e015b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.527+0000 7fa9e6400640 1 --2- 172.21.15.161:0/2135445740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9e0143c40 0x7fa9e015b600 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00aead0 tx=0x7fa9d001c9f0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.527+0000 7fa9e6400640 1 -- 172.21.15.161:0/2135445740 shutdown_connections 2024-05-22T21:24:17.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.527+0000 7fa9e6400640 1 --2- 172.21.15.161:0/2135445740 >> v2:172.21.15.179:3300/0 conn(0x7fa9e006c9b0 0x7fa9e00b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.527+0000 7fa9e6400640 1 --2- 172.21.15.161:0/2135445740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9e0143c40 0x7fa9e015b600 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.527+0000 7fa9e6400640 1 -- 172.21.15.161:0/2135445740 >> 172.21.15.161:0/2135445740 conn(0x7fa9e006a2d0 msgr2=0x7fa9e006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:17.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.527+0000 7fa9e6400640 1 -- 172.21.15.161:0/2135445740 shutdown_connections 2024-05-22T21:24:17.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.527+0000 7fa9e6400640 1 -- 172.21.15.161:0/2135445740 wait complete. 2024-05-22T21:24:17.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.527+0000 7fa9e6400640 1 Processor -- start 2024-05-22T21:24:17.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.527+0000 7fa9e6400640 1 -- start start 2024-05-22T21:24:17.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9e6400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9e006c9b0 0x7fa9e01430d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9e6400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9e0143c40 0x7fa9e01fa110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9e6400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9e01444a0 0x7fa9e0153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9e6400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e01536d0 con 0x7fa9e0143c40 2024-05-22T21:24:17.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9e6400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e015aff0 con 0x7fa9e006c9b0 2024-05-22T21:24:17.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9e6400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e01fa960 con 0x7fa9e01444a0 2024-05-22T21:24:17.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9dea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9e0143c40 0x7fa9e01fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:17.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9dea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9e0143c40 0x7fa9e01fa110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57056/0 (socket says 172.21.15.161:57056) 2024-05-22T21:24:17.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9df400640 1 --2- 172.21.15.161:0/291832699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9e006c9b0 0x7fa9e01430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:17.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9dea00640 1 -- 172.21.15.161:0/291832699 learned_addr learned my addr 172.21.15.161:0/291832699 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:17.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9dfe00640 1 --2- 172.21.15.161:0/291832699 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9e01444a0 0x7fa9e0153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:17.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9dea00640 1 -- 172.21.15.161:0/291832699 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9e01444a0 msgr2=0x7fa9e0153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:17.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9dea00640 1 --2- 172.21.15.161:0/291832699 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9e01444a0 0x7fa9e0153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9dea00640 1 -- 172.21.15.161:0/291832699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9e006c9b0 msgr2=0x7fa9e01430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:17.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9dea00640 1 --2- 172.21.15.161:0/291832699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9e006c9b0 0x7fa9e01430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9dea00640 1 -- 172.21.15.161:0/291832699 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d0009b30 con 0x7fa9e0143c40 2024-05-22T21:24:17.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9dea00640 1 --2- 172.21.15.161:0/291832699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9e0143c40 0x7fa9e01fa110 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c3f0 tx=0x7fa9cc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:17.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9d7400640 1 -- 172.21.15.161:0/291832699 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0052d0 con 0x7fa9e0143c40 2024-05-22T21:24:17.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9d7400640 1 -- 172.21.15.161:0/291832699 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc005430 con 0x7fa9e0143c40 2024-05-22T21:24:17.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9d7400640 1 -- 172.21.15.161:0/291832699 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc003840 con 0x7fa9e0143c40 2024-05-22T21:24:17.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9e6400640 1 -- 172.21.15.161:0/291832699 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e01fac40 con 0x7fa9e0143c40 2024-05-22T21:24:17.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9e6400640 1 -- 172.21.15.161:0/291832699 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e01532f0 con 0x7fa9e0143c40 2024-05-22T21:24:17.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9e6400640 1 -- 172.21.15.161:0/291832699 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e0153810 con 0x7fa9e0143c40 2024-05-22T21:24:17.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.531+0000 7fa9d7400640 1 -- 172.21.15.161:0/291832699 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa9cc0039a0 con 0x7fa9e0143c40 2024-05-22T21:24:17.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.535+0000 7fa9d7400640 1 --2- 172.21.15.161:0/291832699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa9c0066690 0x7fa9c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.535+0000 7fa9d7400640 1 -- 172.21.15.161:0/291832699 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fa9cc083690 con 0x7fa9e0143c40 2024-05-22T21:24:17.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.535+0000 7fa9d7400640 1 -- 172.21.15.161:0/291832699 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7fa9c006e220 con 0x7fa9e0143c40 2024-05-22T21:24:17.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.535+0000 7fa9df400640 1 --2- 172.21.15.161:0/291832699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa9c0066690 0x7fa9c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:17.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.535+0000 7fa9df400640 1 --2- 172.21.15.161:0/291832699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa9c0066690 0x7fa9c0068b50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00ae000 tx=0x7fa9d00a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:17.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.539+0000 7fa9d7400640 1 -- 172.21.15.161:0/291832699 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa9e0153810 con 0x7fa9e0143c40 2024-05-22T21:24:17.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.695+0000 7fa9e6400640 1 -- 172.21.15.161:0/291832699 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9e0153810 con 0x7fa9e0143c40 2024-05-22T21:24:17.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.695+0000 7fa9d7400640 1 -- 172.21.15.161:0/291832699 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v178) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9e0153810 con 0x7fa9e0143c40 2024-05-22T21:24:17.695 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:24:17.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.695+0000 7fa9e6400640 1 -- 172.21.15.161:0/291832699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa9c0066690 msgr2=0x7fa9c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.695+0000 7fa9e6400640 1 --2- 172.21.15.161:0/291832699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa9c0066690 0x7fa9c0068b50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00ae000 tx=0x7fa9d00a7070 comp rx=0 tx=0).stop 2024-05-22T21:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.695+0000 7fa9e6400640 1 -- 172.21.15.161:0/291832699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9e0143c40 msgr2=0x7fa9e01fa110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.695+0000 7fa9e6400640 1 --2- 172.21.15.161:0/291832699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9e0143c40 0x7fa9e01fa110 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c3f0 tx=0x7fa9cc00c8b0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.695+0000 7fa9e6400640 1 -- 172.21.15.161:0/291832699 shutdown_connections 2024-05-22T21:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.695+0000 7fa9e6400640 1 --2- 172.21.15.161:0/291832699 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa9c0066690 0x7fa9c0068b50 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.695+0000 7fa9e6400640 1 --2- 172.21.15.161:0/291832699 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa9e01444a0 0x7fa9e0153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.695+0000 7fa9e6400640 1 --2- 172.21.15.161:0/291832699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9e0143c40 0x7fa9e01fa110 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.695+0000 7fa9e6400640 1 --2- 172.21.15.161:0/291832699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9e006c9b0 0x7fa9e01430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.695+0000 7fa9e6400640 1 -- 172.21.15.161:0/291832699 >> 172.21.15.161:0/291832699 conn(0x7fa9e006a2d0 msgr2=0x7fa9e0158030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.695+0000 7fa9e6400640 1 -- 172.21.15.161:0/291832699 shutdown_connections 2024-05-22T21:24:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.699+0000 7fa9e6400640 1 -- 172.21.15.161:0/291832699 wait complete. 2024-05-22T21:24:17.717 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:24:17.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba3aa00640 1 Processor -- start 2024-05-22T21:24:17.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba3aa00640 1 -- start start 2024-05-22T21:24:17.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba3aa00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fba3406df50 0x7fba340cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba3aa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fba3406e3e0 0x7fba340cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba3aa00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba340cb9f0 con 0x7fba3415b290 2024-05-22T21:24:17.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba3aa00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba340cf300 con 0x7fba3406df50 2024-05-22T21:24:17.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba3aa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba340ceaf0 con 0x7fba3406e3e0 2024-05-22T21:24:17.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba33400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fba3406df50 0x7fba340cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:17.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba38a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fba3406e3e0 0x7fba340cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:17.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba33e00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fba3415b290 0x7fba340ce130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:43014/0 (socket says 172.21.15.161:43014) 2024-05-22T21:24:17.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba33e00640 1 -- 172.21.15.161:0/1168020977 learned_addr learned my addr 172.21.15.161:0/1168020977 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:17.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba33400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fba3406df50 0x7fba340cf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:58866/0 (socket says 172.21.15.161:58866) 2024-05-22T21:24:17.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba38a00640 1 -- 172.21.15.161:0/1168020977 >> v2:172.21.15.179:3300/0 conn(0x7fba3406df50 msgr2=0x7fba340cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:17.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba38a00640 1 --2- 172.21.15.161:0/1168020977 >> v2:172.21.15.179:3300/0 conn(0x7fba3406df50 0x7fba340cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba32a00640 1 -- 172.21.15.161:0/1168020977 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068652624 0 0) 0x7fba340cb9f0 con 0x7fba3415b290 2024-05-22T21:24:17.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba38a00640 1 -- 172.21.15.161:0/1168020977 >> v1:172.21.15.179:6789/0 conn(0x7fba3415b290 legacy=0x7fba340ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:17.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.855+0000 7fba38a00640 1 -- 172.21.15.161:0/1168020977 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba340ce840 con 0x7fba3406e3e0 2024-05-22T21:24:17.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.859+0000 7fba38a00640 1 --2- 172.21.15.161:0/1168020977 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fba3406e3e0 0x7fba340cfa20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0097f0 tx=0x7fba2c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f1481cc7a5b9955 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:17.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.859+0000 7fba32a00640 1 -- 172.21.15.161:0/1168020977 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0a6040 con 0x7fba3406e3e0 2024-05-22T21:24:17.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.859+0000 7fba32a00640 1 -- 172.21.15.161:0/1168020977 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2c0aa600 con 0x7fba3406e3e0 2024-05-22T21:24:17.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.859+0000 7fba32a00640 1 -- 172.21.15.161:0/1168020977 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0aa990 con 0x7fba3406e3e0 2024-05-22T21:24:17.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.859+0000 7fba3aa00640 1 -- 172.21.15.161:0/1168020977 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fba3406e3e0 msgr2=0x7fba340cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:17.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.859+0000 7fba3aa00640 1 --2- 172.21.15.161:0/1168020977 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fba3406e3e0 0x7fba340cfa20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0097f0 tx=0x7fba2c01c4a0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.859+0000 7fba3aa00640 1 -- 172.21.15.161:0/1168020977 shutdown_connections 2024-05-22T21:24:17.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.859+0000 7fba3aa00640 1 --2- 172.21.15.161:0/1168020977 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fba3406e3e0 0x7fba340cfa20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.859+0000 7fba3aa00640 1 --2- 172.21.15.161:0/1168020977 >> v2:172.21.15.179:3300/0 conn(0x7fba3406df50 0x7fba340cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.859+0000 7fba3aa00640 1 -- 172.21.15.161:0/1168020977 >> 172.21.15.161:0/1168020977 conn(0x7fba340d0f20 msgr2=0x7fba340d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:17.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.859+0000 7fba3aa00640 1 -- 172.21.15.161:0/1168020977 shutdown_connections 2024-05-22T21:24:17.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.859+0000 7fba3aa00640 1 -- 172.21.15.161:0/1168020977 wait complete. 2024-05-22T21:24:17.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba3aa00640 1 Processor -- start 2024-05-22T21:24:17.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba3aa00640 1 -- start start 2024-05-22T21:24:17.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba3aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba3406df50 0x7fba340c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba3aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba3406e3e0 0x7fba34156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba3aa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fba3415b290 0x7fba340aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba3aa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba340ca480 con 0x7fba3406df50 2024-05-22T21:24:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba3aa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba341575f0 con 0x7fba3406e3e0 2024-05-22T21:24:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba3aa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba34156b60 con 0x7fba3415b290 2024-05-22T21:24:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba33e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba3406df50 0x7fba340c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba33e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba3406df50 0x7fba340c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57066/0 (socket says 172.21.15.161:57066) 2024-05-22T21:24:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba33e00640 1 -- 172.21.15.161:0/4034296892 learned_addr learned my addr 172.21.15.161:0/4034296892 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba33400640 1 --2- 172.21.15.161:0/4034296892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba3406e3e0 0x7fba34156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba38a00640 1 --2- 172.21.15.161:0/4034296892 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fba3415b290 0x7fba340aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba33e00640 1 -- 172.21.15.161:0/4034296892 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fba3415b290 msgr2=0x7fba340aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba33e00640 1 --2- 172.21.15.161:0/4034296892 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fba3415b290 0x7fba340aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba33e00640 1 -- 172.21.15.161:0/4034296892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba3406e3e0 msgr2=0x7fba34156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:17.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba33e00640 1 --2- 172.21.15.161:0/4034296892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba3406e3e0 0x7fba34156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:17.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba33e00640 1 -- 172.21.15.161:0/4034296892 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c009420 con 0x7fba3406df50 2024-05-22T21:24:17.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba33e00640 1 --2- 172.21.15.161:0/4034296892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba3406df50 0x7fba340c9f40 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fba2400c780 tx=0x7fba2400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:17.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba2be00640 1 -- 172.21.15.161:0/4034296892 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2400ce90 con 0x7fba3406df50 2024-05-22T21:24:17.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba2be00640 1 -- 172.21.15.161:0/4034296892 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2400a640 con 0x7fba3406df50 2024-05-22T21:24:17.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba2be00640 1 -- 172.21.15.161:0/4034296892 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2400aa70 con 0x7fba3406df50 2024-05-22T21:24:17.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba3aa00640 1 -- 172.21.15.161:0/4034296892 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba34159270 con 0x7fba3406df50 2024-05-22T21:24:17.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.863+0000 7fba3aa00640 1 -- 172.21.15.161:0/4034296892 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba34157cc0 con 0x7fba3406df50 2024-05-22T21:24:17.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.867+0000 7fba3aa00640 1 -- 172.21.15.161:0/4034296892 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba34157840 con 0x7fba3406df50 2024-05-22T21:24:17.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.867+0000 7fba2be00640 1 -- 172.21.15.161:0/4034296892 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fba2400abd0 con 0x7fba3406df50 2024-05-22T21:24:17.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.867+0000 7fba2be00640 1 --2- 172.21.15.161:0/4034296892 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fba14066690 0x7fba14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:17.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.867+0000 7fba2be00640 1 -- 172.21.15.161:0/4034296892 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fba24083730 con 0x7fba3406df50 2024-05-22T21:24:17.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.867+0000 7fba33400640 1 --2- 172.21.15.161:0/4034296892 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fba14066690 0x7fba14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:17.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.867+0000 7fba2be00640 1 -- 172.21.15.161:0/4034296892 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7fba1406e220 con 0x7fba3406df50 2024-05-22T21:24:17.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.867+0000 7fba33400640 1 --2- 172.21.15.161:0/4034296892 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fba14066690 0x7fba14068b50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fba340b2280 tx=0x7fba20009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:17.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:17.871+0000 7fba2be00640 1 -- 172.21.15.161:0/4034296892 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fba24018020 con 0x7fba3406df50 2024-05-22T21:24:18.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.027+0000 7fba3aa00640 1 -- 172.21.15.161:0/4034296892 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba34067220 con 0x7fba3406df50 2024-05-22T21:24:18.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.031+0000 7fba2be00640 1 -- 172.21.15.161:0/4034296892 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v178) v1 ==== 104+0+11 (secure 0 0 0) 0x7fba34157840 con 0x7fba3406df50 2024-05-22T21:24:18.030 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:24:18.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.031+0000 7fba3aa00640 1 -- 172.21.15.161:0/4034296892 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fba14066690 msgr2=0x7fba14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:18.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.031+0000 7fba3aa00640 1 --2- 172.21.15.161:0/4034296892 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fba14066690 0x7fba14068b50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fba340b2280 tx=0x7fba20009190 comp rx=0 tx=0).stop 2024-05-22T21:24:18.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.031+0000 7fba3aa00640 1 -- 172.21.15.161:0/4034296892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba3406df50 msgr2=0x7fba340c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:18.032 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.031+0000 7fba3aa00640 1 --2- 172.21.15.161:0/4034296892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba3406df50 0x7fba340c9f40 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fba2400c780 tx=0x7fba2400cc40 comp rx=0 tx=0).stop 2024-05-22T21:24:18.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.031+0000 7fba3aa00640 1 -- 172.21.15.161:0/4034296892 shutdown_connections 2024-05-22T21:24:18.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.031+0000 7fba3aa00640 1 --2- 172.21.15.161:0/4034296892 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fba14066690 0x7fba14068b50 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.031+0000 7fba3aa00640 1 --2- 172.21.15.161:0/4034296892 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fba3415b290 0x7fba340aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.031+0000 7fba3aa00640 1 --2- 172.21.15.161:0/4034296892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba3406e3e0 0x7fba34156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.031+0000 7fba3aa00640 1 --2- 172.21.15.161:0/4034296892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba3406df50 0x7fba340c9f40 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.031+0000 7fba3aa00640 1 -- 172.21.15.161:0/4034296892 >> 172.21.15.161:0/4034296892 conn(0x7fba340d0f20 msgr2=0x7fba340b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:18.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.031+0000 7fba3aa00640 1 -- 172.21.15.161:0/4034296892 shutdown_connections 2024-05-22T21:24:18.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.031+0000 7fba3aa00640 1 -- 172.21.15.161:0/4034296892 wait complete. 2024-05-22T21:24:18.054 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:24:18.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.191+0000 7f7aeae00640 1 Processor -- start 2024-05-22T21:24:18.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.191+0000 7f7aeae00640 1 -- start start 2024-05-22T21:24:18.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7aeae00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7ae40b33d0 0x7f7ae40cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:18.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7aeae00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ae406fa80 0x7f7ae4071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:18.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7aeae00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae4070b70 con 0x7f7ae41436d0 2024-05-22T21:24:18.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7aeae00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae40cc720 con 0x7f7ae40b33d0 2024-05-22T21:24:18.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7aeae00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae406ff60 con 0x7f7ae406fa80 2024-05-22T21:24:18.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae9a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f7ae41436d0 0x7f7ae40b2cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56886/0 (socket says 172.21.15.161:56886) 2024-05-22T21:24:18.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae9a00640 1 -- 172.21.15.161:0/2750551288 learned_addr learned my addr 172.21.15.161:0/2750551288 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:18.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae3e00640 1 -- 172.21.15.161:0/2750551288 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799707446 0 0) 0x7f7ae4070b70 con 0x7f7ae41436d0 2024-05-22T21:24:18.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae3e00640 1 -- 172.21.15.161:0/2750551288 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7acc003530 con 0x7f7ae41436d0 2024-05-22T21:24:18.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7aea400640 1 --2- 172.21.15.161:0/2750551288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ae406fa80 0x7f7ae4071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:18.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae9000640 1 --2- 172.21.15.161:0/2750551288 >> v2:172.21.15.179:3300/0 conn(0x7f7ae40b33d0 0x7f7ae40cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:18.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae3e00640 1 -- 172.21.15.161:0/2750551288 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3993851482 0 0) 0x7f7acc003530 con 0x7f7ae41436d0 2024-05-22T21:24:18.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae3e00640 1 -- 172.21.15.161:0/2750551288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ae406fa80 msgr2=0x7f7ae4071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:18.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae3e00640 1 --2- 172.21.15.161:0/2750551288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ae406fa80 0x7f7ae4071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae3e00640 1 -- 172.21.15.161:0/2750551288 >> v2:172.21.15.179:3300/0 conn(0x7f7ae40b33d0 msgr2=0x7f7ae40cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:18.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae3e00640 1 --2- 172.21.15.161:0/2750551288 >> v2:172.21.15.179:3300/0 conn(0x7f7ae40b33d0 0x7f7ae40cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae3e00640 1 -- 172.21.15.161:0/2750551288 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae4070240 con 0x7f7ae41436d0 2024-05-22T21:24:18.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae3e00640 1 -- 172.21.15.161:0/2750551288 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f7ad8001680 con 0x7f7ae41436d0 2024-05-22T21:24:18.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae3e00640 1 -- 172.21.15.161:0/2750551288 >> v1:172.21.15.161:6789/0 conn(0x7f7ae41436d0 legacy=0x7f7ae40b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:18.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae3e00640 1 --2- 172.21.15.161:0/2750551288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7acc003980 0x7f7acc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:18.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae3e00640 1 --2- 172.21.15.161:0/2750551288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ae406fa80 0x7f7acc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:18.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae3e00640 1 --2- 172.21.15.161:0/2750551288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7acc004d10 0x7f7acc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:18.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae3e00640 1 -- 172.21.15.161:0/2750551288 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40cc720 con 0x7f7acc003980 2024-05-22T21:24:18.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae3e00640 1 -- 172.21.15.161:0/2750551288 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae406ff60 con 0x7f7acc004d10 2024-05-22T21:24:18.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae3e00640 1 -- 172.21.15.161:0/2750551288 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc00baf0 con 0x7f7ae406fa80 2024-05-22T21:24:18.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae9a00640 1 --2- 172.21.15.161:0/2750551288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7acc003980 0x7f7acc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:18.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7aea400640 1 --2- 172.21.15.161:0/2750551288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7acc004d10 0x7f7acc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:18.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae9000640 1 --2- 172.21.15.161:0/2750551288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ae406fa80 0x7f7acc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:18.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae9a00640 1 -- 172.21.15.161:0/2750551288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ae406fa80 msgr2=0x7f7acc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:18.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae9a00640 1 --2- 172.21.15.161:0/2750551288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ae406fa80 0x7f7acc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae9a00640 1 -- 172.21.15.161:0/2750551288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7acc004d10 msgr2=0x7f7acc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:18.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae9a00640 1 --2- 172.21.15.161:0/2750551288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7acc004d10 0x7f7acc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae9a00640 1 -- 172.21.15.161:0/2750551288 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc003530 con 0x7f7acc003980 2024-05-22T21:24:18.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.195+0000 7f7ae9000640 1 --2- 172.21.15.161:0/2750551288 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ae406fa80 0x7f7acc0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:18.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.199+0000 7f7ae9a00640 1 --2- 172.21.15.161:0/2750551288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7acc003980 0x7f7acc003da0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800c240 tx=0x7f7ad801cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd55e175ad5f6be server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:18.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.199+0000 7f7ae3e00640 1 -- 172.21.15.161:0/2750551288 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80a5930 con 0x7f7acc003980 2024-05-22T21:24:18.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.199+0000 7f7ae3e00640 1 -- 172.21.15.161:0/2750551288 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad80a5a90 con 0x7f7acc003980 2024-05-22T21:24:18.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.199+0000 7f7ae3e00640 1 -- 172.21.15.161:0/2750551288 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80a5e60 con 0x7f7acc003980 2024-05-22T21:24:18.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.199+0000 7f7aeae00640 1 -- 172.21.15.161:0/2750551288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7acc003980 msgr2=0x7f7acc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:18.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.199+0000 7f7aeae00640 1 --2- 172.21.15.161:0/2750551288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7acc003980 0x7f7acc003da0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800c240 tx=0x7f7ad801cc50 comp rx=0 tx=0).stop 2024-05-22T21:24:18.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.199+0000 7f7aea400640 1 -- 172.21.15.161:0/2750551288 reap_dead start 2024-05-22T21:24:18.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.199+0000 7f7aeae00640 1 -- 172.21.15.161:0/2750551288 shutdown_connections 2024-05-22T21:24:18.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.199+0000 7f7aeae00640 1 -- 172.21.15.161:0/2750551288 >> 172.21.15.161:0/2750551288 conn(0x7f7ae40cec00 msgr2=0x7f7ae40cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:18.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.199+0000 7f7aeae00640 1 -- 172.21.15.161:0/2750551288 shutdown_connections 2024-05-22T21:24:18.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.199+0000 7f7aeae00640 1 -- 172.21.15.161:0/2750551288 wait complete. 2024-05-22T21:24:18.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.199+0000 7f7aeae00640 1 Processor -- start 2024-05-22T21:24:18.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.199+0000 7f7aeae00640 1 -- start start 2024-05-22T21:24:18.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7aeae00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7acc003980 0x7f7ae413af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:18.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7aeae00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ae41f9f00 0x7f7ae41f92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:18.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7aeae00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae41f9830 0x7f7ae413f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:18.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7aeae00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae4156e50 con 0x7f7acc003980 2024-05-22T21:24:18.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7aeae00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae41f9cb0 con 0x7f7ae41f9830 2024-05-22T21:24:18.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7aeae00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40ae220 con 0x7f7ae41f9f00 2024-05-22T21:24:18.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7ae9a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7acc003980 0x7f7ae413af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:18.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7ae9a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7acc003980 0x7f7ae413af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57098/0 (socket says 172.21.15.161:57098) 2024-05-22T21:24:18.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7ae9a00640 1 -- 172.21.15.161:0/3542428259 learned_addr learned my addr 172.21.15.161:0/3542428259 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:18.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7aea400640 1 --2- 172.21.15.161:0/3542428259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae41f9830 0x7f7ae413f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:18.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7ae9000640 1 --2- 172.21.15.161:0/3542428259 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ae41f9f00 0x7f7ae41f92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:18.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7ae9a00640 1 -- 172.21.15.161:0/3542428259 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ae41f9f00 msgr2=0x7f7ae41f92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:18.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7ae9a00640 1 --2- 172.21.15.161:0/3542428259 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ae41f9f00 0x7f7ae41f92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7ae9a00640 1 -- 172.21.15.161:0/3542428259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae41f9830 msgr2=0x7f7ae413f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:18.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7ae9a00640 1 --2- 172.21.15.161:0/3542428259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae41f9830 0x7f7ae413f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7ae9a00640 1 -- 172.21.15.161:0/3542428259 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae40add90 con 0x7f7acc003980 2024-05-22T21:24:18.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7aea400640 1 --2- 172.21.15.161:0/3542428259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae41f9830 0x7f7ae413f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:18.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7ae9a00640 1 --2- 172.21.15.161:0/3542428259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7acc003980 0x7f7ae413af60 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800c240 tx=0x7f7acc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:18.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7ae1600640 1 -- 172.21.15.161:0/3542428259 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8004ab0 con 0x7f7acc003980 2024-05-22T21:24:18.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7ae1600640 1 -- 172.21.15.161:0/3542428259 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad80b3070 con 0x7f7acc003980 2024-05-22T21:24:18.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7ae1600640 1 -- 172.21.15.161:0/3542428259 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8004e50 con 0x7f7acc003980 2024-05-22T21:24:18.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7aeae00640 1 -- 172.21.15.161:0/3542428259 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae40ad460 con 0x7f7acc003980 2024-05-22T21:24:18.204 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7aeae00640 1 -- 172.21.15.161:0/3542428259 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae40ada20 con 0x7f7acc003980 2024-05-22T21:24:18.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7aeae00640 1 -- 172.21.15.161:0/3542428259 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae4067070 con 0x7f7acc003980 2024-05-22T21:24:18.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.203+0000 7f7ae1600640 1 -- 172.21.15.161:0/3542428259 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7ad8007070 con 0x7f7acc003980 2024-05-22T21:24:18.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.207+0000 7f7ae1600640 1 --2- 172.21.15.161:0/3542428259 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7ac4066690 0x7f7ac4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:18.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.207+0000 7f7ae1600640 1 -- 172.21.15.161:0/3542428259 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7ad80b5070 con 0x7f7acc003980 2024-05-22T21:24:18.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.207+0000 7f7ae1600640 1 -- 172.21.15.161:0/3542428259 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7f7ac406e220 con 0x7f7acc003980 2024-05-22T21:24:18.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.207+0000 7f7ae9000640 1 --2- 172.21.15.161:0/3542428259 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7ac4066690 0x7f7ac4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:18.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.207+0000 7f7ae9000640 1 --2- 172.21.15.161:0/3542428259 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7ac4066690 0x7f7ac4068b50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40b2510 tx=0x7f7ad0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:18.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.211+0000 7f7ae1600640 1 -- 172.21.15.161:0/3542428259 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7ad80bf020 con 0x7f7acc003980 2024-05-22T21:24:18.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.363+0000 7f7aeae00640 1 -- 172.21.15.161:0/3542428259 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7ae406d5c0 con 0x7f7acc003980 2024-05-22T21:24:18.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.363+0000 7f7ae1600640 1 -- 172.21.15.161:0/3542428259 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v178) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7ad80ef2a0 con 0x7f7acc003980 2024-05-22T21:24:18.364 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:24:18.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.367+0000 7f7aeae00640 1 -- 172.21.15.161:0/3542428259 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7ac4066690 msgr2=0x7f7ac4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:18.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.367+0000 7f7aeae00640 1 --2- 172.21.15.161:0/3542428259 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7ac4066690 0x7f7ac4068b50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40b2510 tx=0x7f7ad0008040 comp rx=0 tx=0).stop 2024-05-22T21:24:18.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.367+0000 7f7aeae00640 1 -- 172.21.15.161:0/3542428259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7acc003980 msgr2=0x7f7ae413af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:18.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.367+0000 7f7aeae00640 1 --2- 172.21.15.161:0/3542428259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7acc003980 0x7f7ae413af60 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800c240 tx=0x7f7acc005800 comp rx=0 tx=0).stop 2024-05-22T21:24:18.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.367+0000 7f7aeae00640 1 -- 172.21.15.161:0/3542428259 shutdown_connections 2024-05-22T21:24:18.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.367+0000 7f7aeae00640 1 --2- 172.21.15.161:0/3542428259 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7ac4066690 0x7f7ac4068b50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.367+0000 7f7aeae00640 1 --2- 172.21.15.161:0/3542428259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae41f9830 0x7f7ae413f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.367+0000 7f7aeae00640 1 --2- 172.21.15.161:0/3542428259 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ae41f9f00 0x7f7ae41f92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.367+0000 7f7aeae00640 1 --2- 172.21.15.161:0/3542428259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7acc003980 0x7f7ae413af60 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.367+0000 7f7aeae00640 1 -- 172.21.15.161:0/3542428259 >> 172.21.15.161:0/3542428259 conn(0x7f7ae40cec00 msgr2=0x7f7ae40ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:18.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.367+0000 7f7aeae00640 1 -- 172.21.15.161:0/3542428259 shutdown_connections 2024-05-22T21:24:18.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.367+0000 7f7aeae00640 1 -- 172.21.15.161:0/3542428259 wait complete. 2024-05-22T21:24:18.386 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:24:18.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.523+0000 7fedf9600640 1 Processor -- start 2024-05-22T21:24:18.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.523+0000 7fedf9600640 1 -- start start 2024-05-22T21:24:18.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.523+0000 7fedf9600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fedf406fa80 0x7fedf40cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:18.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.523+0000 7fedf9600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf4070b70 con 0x7fedf41436d0 2024-05-22T21:24:18.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.523+0000 7fedf9600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf406ff60 con 0x7fedf40b33d0 2024-05-22T21:24:18.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.523+0000 7fedf9600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf40cc720 con 0x7fedf406fa80 2024-05-22T21:24:18.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.523+0000 7fedf8c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fedf406fa80 0x7fedf40cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:18.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.523+0000 7fedf3e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fedf41436d0 0x7fedf40b2cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56902/0 (socket says 172.21.15.161:56902) 2024-05-22T21:24:18.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.523+0000 7fedf3e00640 1 -- 172.21.15.161:0/2754701109 learned_addr learned my addr 172.21.15.161:0/2754701109 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:18.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.523+0000 7fedf8c00640 1 -- 172.21.15.161:0/2754701109 >> v1:172.21.15.179:6789/0 conn(0x7fedf40b33d0 legacy=0x7fedf4071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:18.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.523+0000 7fedf3400640 1 -- 172.21.15.161:0/2754701109 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1467503644 0 0) 0x7fedf4070b70 con 0x7fedf41436d0 2024-05-22T21:24:18.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.523+0000 7fedf8c00640 1 -- 172.21.15.161:0/2754701109 >> v1:172.21.15.161:6789/0 conn(0x7fedf41436d0 legacy=0x7fedf40b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:18.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.523+0000 7fedf8c00640 1 -- 172.21.15.161:0/2754701109 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf4070240 con 0x7fedf406fa80 2024-05-22T21:24:18.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.527+0000 7fedf8c00640 1 --2- 172.21.15.161:0/2754701109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf406fa80 0x7fedf40cc1e0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fedec009d90 tx=0x7fedec01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3905b6e3e83e47b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:18.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.527+0000 7fedf3400640 1 -- 172.21.15.161:0/2754701109 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec0a5930 con 0x7fedf406fa80 2024-05-22T21:24:18.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.527+0000 7fedf3400640 1 -- 172.21.15.161:0/2754701109 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec0a5a90 con 0x7fedf406fa80 2024-05-22T21:24:18.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.527+0000 7fedf3400640 1 -- 172.21.15.161:0/2754701109 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec0a5e20 con 0x7fedf406fa80 2024-05-22T21:24:18.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.527+0000 7fedf9600640 1 -- 172.21.15.161:0/2754701109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf406fa80 msgr2=0x7fedf40cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:18.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.527+0000 7fedf9600640 1 --2- 172.21.15.161:0/2754701109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf406fa80 0x7fedf40cc1e0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fedec009d90 tx=0x7fedec01c870 comp rx=0 tx=0).stop 2024-05-22T21:24:18.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.527+0000 7fedf9600640 1 -- 172.21.15.161:0/2754701109 shutdown_connections 2024-05-22T21:24:18.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.527+0000 7fedf9600640 1 --2- 172.21.15.161:0/2754701109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf406fa80 0x7fedf40cc1e0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.527+0000 7fedf9600640 1 -- 172.21.15.161:0/2754701109 >> 172.21.15.161:0/2754701109 conn(0x7fedf40cec00 msgr2=0x7fedf40cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:18.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.527+0000 7fedf9600640 1 -- 172.21.15.161:0/2754701109 shutdown_connections 2024-05-22T21:24:18.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.527+0000 7fedf9600640 1 -- 172.21.15.161:0/2754701109 wait complete. 2024-05-22T21:24:18.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf9600640 1 Processor -- start 2024-05-22T21:24:18.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf9600640 1 -- start start 2024-05-22T21:24:18.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf9600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedf406fa80 0x7fedf41f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:18.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf9600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fedf40b33d0 0x7fedf4136f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:18.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf9600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf41436d0 0x7fedf413b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:18.531 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf9600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf41f6a20 con 0x7fedf41436d0 2024-05-22T21:24:18.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf9600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf4137460 con 0x7fedf406fa80 2024-05-22T21:24:18.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf9600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf413a620 con 0x7fedf40b33d0 2024-05-22T21:24:18.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf8c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf41436d0 0x7fedf413b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:18.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf8c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf41436d0 0x7fedf413b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57108/0 (socket says 172.21.15.161:57108) 2024-05-22T21:24:18.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf8c00640 1 -- 172.21.15.161:0/3710257097 learned_addr learned my addr 172.21.15.161:0/3710257097 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:18.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf3e00640 1 --2- 172.21.15.161:0/3710257097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedf406fa80 0x7fedf41f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:18.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedeb400640 1 --2- 172.21.15.161:0/3710257097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fedf40b33d0 0x7fedf4136f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:18.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf8c00640 1 -- 172.21.15.161:0/3710257097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fedf40b33d0 msgr2=0x7fedf4136f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:18.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf8c00640 1 --2- 172.21.15.161:0/3710257097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fedf40b33d0 0x7fedf4136f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf8c00640 1 -- 172.21.15.161:0/3710257097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedf406fa80 msgr2=0x7fedf41f64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:18.533 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf8c00640 1 --2- 172.21.15.161:0/3710257097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedf406fa80 0x7fedf41f64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf8c00640 1 -- 172.21.15.161:0/3710257097 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedec009a70 con 0x7fedf41436d0 2024-05-22T21:24:18.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf3e00640 1 --2- 172.21.15.161:0/3710257097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedf406fa80 0x7fedf41f64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:18.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedeb400640 1 --2- 172.21.15.161:0/3710257097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fedf40b33d0 0x7fedf4136f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:18.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf8c00640 1 --2- 172.21.15.161:0/3710257097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf41436d0 0x7fedf413b6c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fedec01a4a0 tx=0x7fedec019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:18.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf0c00640 1 -- 172.21.15.161:0/3710257097 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec0ade40 con 0x7fedf41436d0 2024-05-22T21:24:18.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf0c00640 1 -- 172.21.15.161:0/3710257097 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec0b7410 con 0x7fedf41436d0 2024-05-22T21:24:18.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf0c00640 1 -- 172.21.15.161:0/3710257097 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec0b77a0 con 0x7fedf41436d0 2024-05-22T21:24:18.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf9600640 1 -- 172.21.15.161:0/3710257097 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf413a900 con 0x7fedf41436d0 2024-05-22T21:24:18.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.531+0000 7fedf9600640 1 -- 172.21.15.161:0/3710257097 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf413aec0 con 0x7fedf41436d0 2024-05-22T21:24:18.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.535+0000 7fedf9600640 1 -- 172.21.15.161:0/3710257097 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf4067070 con 0x7fedf41436d0 2024-05-22T21:24:18.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.535+0000 7fedf0c00640 1 -- 172.21.15.161:0/3710257097 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fedec0b4050 con 0x7fedf41436d0 2024-05-22T21:24:18.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.535+0000 7fedf0c00640 1 --2- 172.21.15.161:0/3710257097 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fedd0066690 0x7fedd0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:18.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.535+0000 7fedf0c00640 1 -- 172.21.15.161:0/3710257097 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fedec11f6e0 con 0x7fedf41436d0 2024-05-22T21:24:18.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.535+0000 7fedf3e00640 1 --2- 172.21.15.161:0/3710257097 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fedd0066690 0x7fedd0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:18.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.535+0000 7fedf0c00640 1 -- 172.21.15.161:0/3710257097 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7fedd006e220 con 0x7fedf41436d0 2024-05-22T21:24:18.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.535+0000 7fedf3e00640 1 --2- 172.21.15.161:0/3710257097 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fedd0066690 0x7fedd0068b50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fede4007fd0 tx=0x7fede4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:18.539 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.539+0000 7fedf0c00640 1 -- 172.21.15.161:0/3710257097 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fedec0b2060 con 0x7fedf41436d0 2024-05-22T21:24:18.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.715+0000 7fedf9600640 1 -- 172.21.15.161:0/3710257097 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fedf406d5c0 con 0x7fedf41436d0 2024-05-22T21:24:18.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.715+0000 7fedf0c00640 1 -- 172.21.15.161:0/3710257097 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v178) v1 ==== 80+0+0 (secure 0 0 0) 0x7fedec0ecc00 con 0x7fedf41436d0 2024-05-22T21:24:18.714 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:24:18.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.715+0000 7fedf9600640 1 -- 172.21.15.161:0/3710257097 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fedd0066690 msgr2=0x7fedd0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:18.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.715+0000 7fedf9600640 1 --2- 172.21.15.161:0/3710257097 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fedd0066690 0x7fedd0068b50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fede4007fd0 tx=0x7fede4009040 comp rx=0 tx=0).stop 2024-05-22T21:24:18.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.715+0000 7fedf9600640 1 -- 172.21.15.161:0/3710257097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf41436d0 msgr2=0x7fedf413b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:18.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.715+0000 7fedf9600640 1 --2- 172.21.15.161:0/3710257097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf41436d0 0x7fedf413b6c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fedec01a4a0 tx=0x7fedec019610 comp rx=0 tx=0).stop 2024-05-22T21:24:18.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.715+0000 7fedf9600640 1 -- 172.21.15.161:0/3710257097 shutdown_connections 2024-05-22T21:24:18.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.715+0000 7fedf9600640 1 --2- 172.21.15.161:0/3710257097 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fedd0066690 0x7fedd0068b50 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.715+0000 7fedf9600640 1 --2- 172.21.15.161:0/3710257097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedf41436d0 0x7fedf413b6c0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.715+0000 7fedf9600640 1 --2- 172.21.15.161:0/3710257097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fedf40b33d0 0x7fedf4136f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.715+0000 7fedf9600640 1 --2- 172.21.15.161:0/3710257097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedf406fa80 0x7fedf41f64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:18.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.715+0000 7fedf9600640 1 -- 172.21.15.161:0/3710257097 >> 172.21.15.161:0/3710257097 conn(0x7fedf40cec00 msgr2=0x7fedf40b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:18.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.715+0000 7fedf9600640 1 -- 172.21.15.161:0/3710257097 shutdown_connections 2024-05-22T21:24:18.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:18.715+0000 7fedf9600640 1 -- 172.21.15.161:0/3710257097 wait complete. 2024-05-22T21:24:18.735 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-22T21:24:18.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:18.735 DEBUG:teuthology.orchestra.run.smithi179:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-22T21:24:18.764 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:24:18.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:18.764 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-22T21:24:18.815 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:24:18.815 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:24:18.824 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:24:18.825 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:24:18.825 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:24:18.870 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:24:18.870 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:24:19.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.007+0000 7f72e7400640 1 Processor -- start 2024-05-22T21:24:19.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.007+0000 7f72e7400640 1 -- start start 2024-05-22T21:24:19.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.007+0000 7f72e7400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72e01641f0 0x7f72e01665e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.007+0000 7f72e7400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e00b3560 con 0x7f72e015e7d0 2024-05-22T21:24:19.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.007+0000 7f72e7400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e00cf530 con 0x7f72e00b1df0 2024-05-22T21:24:19.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.007+0000 7f72e7400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00ccdb0 con 0x7f72e01641f0 2024-05-22T21:24:19.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.007+0000 7f72e4a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f72e015e7d0 0x7f72e015eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56914/0 (socket says 172.21.15.161:56914) 2024-05-22T21:24:19.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.007+0000 7f72e4a00640 1 -- 172.21.15.161:0/4173522971 learned_addr learned my addr 172.21.15.161:0/4173522971 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:19.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.007+0000 7f72e5400640 1 --2- 172.21.15.161:0/4173522971 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72e01641f0 0x7f72e01665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:19.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.007+0000 7f72df400640 1 -- 172.21.15.161:0/4173522971 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766280991 0 0) 0x7f72e00b3560 con 0x7f72e015e7d0 2024-05-22T21:24:19.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.007+0000 7f72df400640 1 -- 172.21.15.161:0/4173522971 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72c8003530 con 0x7f72e015e7d0 2024-05-22T21:24:19.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.007+0000 7f72df400640 1 -- 172.21.15.161:0/4173522971 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3733693290 0 0) 0x7f72e00cf530 con 0x7f72e00b1df0 2024-05-22T21:24:19.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.007+0000 7f72df400640 1 -- 172.21.15.161:0/4173522971 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72e00b3560 con 0x7f72e00b1df0 2024-05-22T21:24:19.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.007+0000 7f72e5400640 1 -- 172.21.15.161:0/4173522971 >> v1:172.21.15.179:6789/0 conn(0x7f72e00b1df0 legacy=0x7f72e015f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.007+0000 7f72e5400640 1 -- 172.21.15.161:0/4173522971 >> v1:172.21.15.161:6789/0 conn(0x7f72e015e7d0 legacy=0x7f72e015eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.009 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.007+0000 7f72e5400640 1 -- 172.21.15.161:0/4173522971 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e015fcf0 con 0x7f72e01641f0 2024-05-22T21:24:19.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.011+0000 7f72e5400640 1 --2- 172.21.15.161:0/4173522971 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72e01641f0 0x7f72e01665e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009dc0 tx=0x7f72d801c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e729ce52b6ad9a99 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:19.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.011+0000 7f72df400640 1 -- 172.21.15.161:0/4173522971 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d80a5930 con 0x7f72e01641f0 2024-05-22T21:24:19.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.011+0000 7f72df400640 1 -- 172.21.15.161:0/4173522971 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d80a5a90 con 0x7f72e01641f0 2024-05-22T21:24:19.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.011+0000 7f72df400640 1 -- 172.21.15.161:0/4173522971 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d80a5e20 con 0x7f72e01641f0 2024-05-22T21:24:19.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.011+0000 7f72e7400640 1 -- 172.21.15.161:0/4173522971 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72e01641f0 msgr2=0x7f72e01665e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.011+0000 7f72e7400640 1 --2- 172.21.15.161:0/4173522971 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72e01641f0 0x7f72e01665e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009dc0 tx=0x7f72d801c960 comp rx=0 tx=0).stop 2024-05-22T21:24:19.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.011+0000 7f72e7400640 1 -- 172.21.15.161:0/4173522971 shutdown_connections 2024-05-22T21:24:19.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.011+0000 7f72e7400640 1 --2- 172.21.15.161:0/4173522971 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72e01641f0 0x7f72e01665e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.011+0000 7f72e7400640 1 -- 172.21.15.161:0/4173522971 >> 172.21.15.161:0/4173522971 conn(0x7f72e00b2a40 msgr2=0x7f72e00b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:19.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.011+0000 7f72e7400640 1 -- 172.21.15.161:0/4173522971 shutdown_connections 2024-05-22T21:24:19.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.011+0000 7f72e7400640 1 -- 172.21.15.161:0/4173522971 wait complete. 2024-05-22T21:24:19.013 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72e7400640 1 Processor -- start 2024-05-22T21:24:19.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72e7400640 1 -- start start 2024-05-22T21:24:19.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72e7400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72e00b1df0 0x7f72e00cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72e7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72e015e7d0 0x7f72e00c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72e7400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72e01641f0 0x7f72e00af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72e7400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e0153f10 con 0x7f72e015e7d0 2024-05-22T21:24:19.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72e7400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00b07a0 con 0x7f72e01641f0 2024-05-22T21:24:19.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72e7400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00ca830 con 0x7f72e00b1df0 2024-05-22T21:24:19.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72dfe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72e015e7d0 0x7f72e00c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:19.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72dfe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72e015e7d0 0x7f72e00c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57120/0 (socket says 172.21.15.161:57120) 2024-05-22T21:24:19.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72dfe00640 1 -- 172.21.15.161:0/4131782921 learned_addr learned my addr 172.21.15.161:0/4131782921 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:19.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72e5400640 1 --2- 172.21.15.161:0/4131782921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72e01641f0 0x7f72e00af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:19.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72e4a00640 1 --2- 172.21.15.161:0/4131782921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72e00b1df0 0x7f72e00cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:19.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72dfe00640 1 -- 172.21.15.161:0/4131782921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72e00b1df0 msgr2=0x7f72e00cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72dfe00640 1 --2- 172.21.15.161:0/4131782921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72e00b1df0 0x7f72e00cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72dfe00640 1 -- 172.21.15.161:0/4131782921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72e01641f0 msgr2=0x7f72e00af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72dfe00640 1 --2- 172.21.15.161:0/4131782921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72e01641f0 0x7f72e00af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.016 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72dfe00640 1 -- 172.21.15.161:0/4131782921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d8009a70 con 0x7f72e015e7d0 2024-05-22T21:24:19.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72dfe00640 1 --2- 172.21.15.161:0/4131782921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72e015e7d0 0x7f72e00c8a30 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00c490 tx=0x7f72cc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:19.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72dcc00640 1 -- 172.21.15.161:0/4131782921 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0052d0 con 0x7f72e015e7d0 2024-05-22T21:24:19.017 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72e7400640 1 -- 172.21.15.161:0/4131782921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e00c9c20 con 0x7f72e015e7d0 2024-05-22T21:24:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72dcc00640 1 -- 172.21.15.161:0/4131782921 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc005430 con 0x7f72e015e7d0 2024-05-22T21:24:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72dcc00640 1 -- 172.21.15.161:0/4131782921 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc012870 con 0x7f72e015e7d0 2024-05-22T21:24:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72e7400640 1 -- 172.21.15.161:0/4131782921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e00ca1e0 con 0x7f72e015e7d0 2024-05-22T21:24:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.015+0000 7f72e7400640 1 -- 172.21.15.161:0/4131782921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e0067220 con 0x7f72e015e7d0 2024-05-22T21:24:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.019+0000 7f72dcc00640 1 -- 172.21.15.161:0/4131782921 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f72cc0129d0 con 0x7f72e015e7d0 2024-05-22T21:24:19.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.019+0000 7f72dcc00640 1 --2- 172.21.15.161:0/4131782921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72c0066690 0x7f72c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.019+0000 7f72dcc00640 1 -- 172.21.15.161:0/4131782921 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f72cc083e10 con 0x7f72e015e7d0 2024-05-22T21:24:19.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.019+0000 7f72e4a00640 1 --2- 172.21.15.161:0/4131782921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72c0066690 0x7f72c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:19.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.019+0000 7f72dcc00640 1 -- 172.21.15.161:0/4131782921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7f72c006e220 con 0x7f72e015e7d0 2024-05-22T21:24:19.019 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.019+0000 7f72e4a00640 1 --2- 172.21.15.161:0/4131782921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72c0066690 0x7f72c0068b50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f72d40011f0 tx=0x7f72d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:19.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.023+0000 7f72dcc00640 1 -- 172.21.15.161:0/4131782921 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f72cc087030 con 0x7f72e015e7d0 2024-05-22T21:24:19.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.215+0000 7f72e7400640 1 -- 172.21.15.161:0/4131782921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72e015e000 con 0x7f72e015e7d0 2024-05-22T21:24:19.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.215+0000 7f72dcc00640 1 -- 172.21.15.161:0/4131782921 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4747 (secure 0 0 0) 0x7f72cc0510d0 con 0x7f72e015e7d0 2024-05-22T21:24:19.216 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:19.216 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2405126804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2405126804},{"type":"v1","addr":"172.21.15.179:6834","nonce":2405126804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":6962,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3984924056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3984924056},{"type":"v1","addr":"172.21.15.179:6835","nonce":3984924056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8391,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/368424851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":368424851},{"type":"v1","addr":"172.21.15.161:6836","nonce":368424851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:23:03.863540+0000","modified":"2024-05-22T21:23:06.877759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2279022192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2279022192},{"type":"v1","addr":"172.21.15.161:6837","nonce":2279022192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-05-22T21:24:19.216 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 78 2024-05-22T21:24:19.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.219+0000 7f72e7400640 1 -- 172.21.15.161:0/4131782921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72c0066690 msgr2=0x7f72c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.219+0000 7f72e7400640 1 --2- 172.21.15.161:0/4131782921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72c0066690 0x7f72c0068b50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f72d40011f0 tx=0x7f72d4009040 comp rx=0 tx=0).stop 2024-05-22T21:24:19.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.219+0000 7f72e7400640 1 -- 172.21.15.161:0/4131782921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72e015e7d0 msgr2=0x7f72e00c8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.219+0000 7f72e7400640 1 --2- 172.21.15.161:0/4131782921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72e015e7d0 0x7f72e00c8a30 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00c490 tx=0x7f72cc00c950 comp rx=0 tx=0).stop 2024-05-22T21:24:19.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.219+0000 7f72e7400640 1 -- 172.21.15.161:0/4131782921 shutdown_connections 2024-05-22T21:24:19.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.219+0000 7f72e7400640 1 --2- 172.21.15.161:0/4131782921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72c0066690 0x7f72c0068b50 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.219+0000 7f72e7400640 1 --2- 172.21.15.161:0/4131782921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72e01641f0 0x7f72e00af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.219+0000 7f72e7400640 1 --2- 172.21.15.161:0/4131782921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72e015e7d0 0x7f72e00c8a30 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.219+0000 7f72e7400640 1 --2- 172.21.15.161:0/4131782921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72e00b1df0 0x7f72e00cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.219+0000 7f72e7400640 1 -- 172.21.15.161:0/4131782921 >> 172.21.15.161:0/4131782921 conn(0x7f72e00b2a40 msgr2=0x7f72e00b5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:19.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.219+0000 7f72e7400640 1 -- 172.21.15.161:0/4131782921 shutdown_connections 2024-05-22T21:24:19.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.219+0000 7f72e7400640 1 -- 172.21.15.161:0/4131782921 wait complete. 2024-05-22T21:24:19.237 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:24:19.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.371+0000 7f8ea3c00640 1 Processor -- start 2024-05-22T21:24:19.373 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.371+0000 7f8ea3c00640 1 -- start start 2024-05-22T21:24:19.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.375+0000 7f8ea3c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e9c15dd50 0x7f8e9c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.375+0000 7f8ea3c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8e9c15d5e0 0x7f8e9c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.375+0000 7f8ea3c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e9c071670 con 0x7f8e9c15e130 2024-05-22T21:24:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.375+0000 7f8ea3c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e9c06de40 con 0x7f8e9c15d5e0 2024-05-22T21:24:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.375+0000 7f8ea3c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c070a60 con 0x7f8e9c15dd50 2024-05-22T21:24:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.375+0000 7f8e93e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8e9c15e130 0x7f8e9c15ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56916/0 (socket says 172.21.15.161:56916) 2024-05-22T21:24:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.375+0000 7f8e93e00640 1 -- 172.21.15.161:0/3470303665 learned_addr learned my addr 172.21.15.161:0/3470303665 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.375+0000 7f8ea1c00640 1 --2- 172.21.15.161:0/3470303665 >> v2:172.21.15.161:3300/0 conn(0x7f8e9c15d5e0 0x7f8e9c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.375+0000 7f8ea1200640 1 --2- 172.21.15.161:0/3470303665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e9c15dd50 0x7f8e9c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.375+0000 7f8e9be00640 1 -- 172.21.15.161:0/3470303665 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431595440 0 0) 0x7f8e9c071670 con 0x7f8e9c15e130 2024-05-22T21:24:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.375+0000 7f8e9be00640 1 -- 172.21.15.161:0/3470303665 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e80005740 con 0x7f8e9c15e130 2024-05-22T21:24:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.375+0000 7f8ea1c00640 1 -- 172.21.15.161:0/3470303665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e9c15dd50 msgr2=0x7f8e9c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:19.375 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.375+0000 7f8ea1c00640 1 --2- 172.21.15.161:0/3470303665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e9c15dd50 0x7f8e9c06d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.375+0000 7f8ea1c00640 1 -- 172.21.15.161:0/3470303665 >> v1:172.21.15.161:6789/0 conn(0x7f8e9c15e130 legacy=0x7f8e9c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.375+0000 7f8ea1c00640 1 -- 172.21.15.161:0/3470303665 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c070d40 con 0x7f8e9c15d5e0 2024-05-22T21:24:19.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea1c00640 1 --2- 172.21.15.161:0/3470303665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9c15d5e0 0x7f8e9c15b6c0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009d90 tx=0x7f8e9401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49181350f2d959fd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:19.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8e9be00640 1 -- 172.21.15.161:0/3470303665 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e940a5970 con 0x7f8e9c15d5e0 2024-05-22T21:24:19.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8e9be00640 1 -- 172.21.15.161:0/3470303665 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e940a5ad0 con 0x7f8e9c15d5e0 2024-05-22T21:24:19.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8e9be00640 1 -- 172.21.15.161:0/3470303665 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e940a5e60 con 0x7f8e9c15d5e0 2024-05-22T21:24:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea3c00640 1 -- 172.21.15.161:0/3470303665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9c15d5e0 msgr2=0x7f8e9c15b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea3c00640 1 --2- 172.21.15.161:0/3470303665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9c15d5e0 0x7f8e9c15b6c0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009d90 tx=0x7f8e9401c8b0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea3c00640 1 -- 172.21.15.161:0/3470303665 shutdown_connections 2024-05-22T21:24:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea3c00640 1 --2- 172.21.15.161:0/3470303665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9c15d5e0 0x7f8e9c15b6c0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea3c00640 1 --2- 172.21.15.161:0/3470303665 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e9c15dd50 0x7f8e9c06d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea3c00640 1 -- 172.21.15.161:0/3470303665 >> 172.21.15.161:0/3470303665 conn(0x7f8e9c0b0bc0 msgr2=0x7f8e9c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea3c00640 1 -- 172.21.15.161:0/3470303665 shutdown_connections 2024-05-22T21:24:19.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea3c00640 1 -- 172.21.15.161:0/3470303665 wait complete. 2024-05-22T21:24:19.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea3c00640 1 Processor -- start 2024-05-22T21:24:19.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea3c00640 1 -- start start 2024-05-22T21:24:19.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea3c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e9c15d5e0 0x7f8e9c155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea3c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9c15dd50 0x7f8e9c1407b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea3c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e9c15e130 0x7f8e9c0b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea3c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c144200 con 0x7f8e9c15dd50 2024-05-22T21:24:19.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea3c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c154cc0 con 0x7f8e9c15d5e0 2024-05-22T21:24:19.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8ea3c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c156b90 con 0x7f8e9c15e130 2024-05-22T21:24:19.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8e93e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9c15dd50 0x7f8e9c1407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:19.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8e93e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9c15dd50 0x7f8e9c1407b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57126/0 (socket says 172.21.15.161:57126) 2024-05-22T21:24:19.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.379+0000 7f8e93e00640 1 -- 172.21.15.161:0/1374633680 learned_addr learned my addr 172.21.15.161:0/1374633680 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:19.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8ea1c00640 1 --2- 172.21.15.161:0/1374633680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e9c15e130 0x7f8e9c0b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:19.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8ea1200640 1 --2- 172.21.15.161:0/1374633680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e9c15d5e0 0x7f8e9c155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:19.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8e93e00640 1 -- 172.21.15.161:0/1374633680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e9c15e130 msgr2=0x7f8e9c0b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8e93e00640 1 --2- 172.21.15.161:0/1374633680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e9c15e130 0x7f8e9c0b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8e93e00640 1 -- 172.21.15.161:0/1374633680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e9c15d5e0 msgr2=0x7f8e9c155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8e93e00640 1 --2- 172.21.15.161:0/1374633680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e9c15d5e0 0x7f8e9c155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8e93e00640 1 -- 172.21.15.161:0/1374633680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e94009a70 con 0x7f8e9c15dd50 2024-05-22T21:24:19.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8ea1c00640 1 --2- 172.21.15.161:0/1374633680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e9c15e130 0x7f8e9c0b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:19.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8ea1200640 1 --2- 172.21.15.161:0/1374633680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e9c15d5e0 0x7f8e9c155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:19.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8e93e00640 1 --2- 172.21.15.161:0/1374633680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9c15dd50 0x7f8e9c1407b0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400c420 tx=0x7f8e8400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:19.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8e99600640 1 -- 172.21.15.161:0/1374633680 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84005cf0 con 0x7f8e9c15dd50 2024-05-22T21:24:19.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8e99600640 1 -- 172.21.15.161:0/1374633680 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e84005e50 con 0x7f8e9c15dd50 2024-05-22T21:24:19.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8ea3c00640 1 -- 172.21.15.161:0/1374633680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9c154f80 con 0x7f8e9c15dd50 2024-05-22T21:24:19.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8e99600640 1 -- 172.21.15.161:0/1374633680 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e840072d0 con 0x7f8e9c15dd50 2024-05-22T21:24:19.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8ea3c00640 1 -- 172.21.15.161:0/1374633680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e9c140cf0 con 0x7f8e9c15dd50 2024-05-22T21:24:19.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8ea3c00640 1 -- 172.21.15.161:0/1374633680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e9c067220 con 0x7f8e9c15dd50 2024-05-22T21:24:19.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8e99600640 1 -- 172.21.15.161:0/1374633680 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8e84007430 con 0x7f8e9c15dd50 2024-05-22T21:24:19.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8e99600640 1 --2- 172.21.15.161:0/1374633680 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e78066690 0x7f8e78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8e99600640 1 -- 172.21.15.161:0/1374633680 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8e84083730 con 0x7f8e9c15dd50 2024-05-22T21:24:19.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8ea1200640 1 --2- 172.21.15.161:0/1374633680 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e78066690 0x7f8e78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:19.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.383+0000 7f8e99600640 1 -- 172.21.15.161:0/1374633680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7f8e7806e220 con 0x7f8e9c15dd50 2024-05-22T21:24:19.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.387+0000 7f8ea1200640 1 --2- 172.21.15.161:0/1374633680 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e78066690 0x7f8e78068b50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c007a70 tx=0x7f8e8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:19.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.387+0000 7f8e99600640 1 -- 172.21.15.161:0/1374633680 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8e84017060 con 0x7f8e9c15dd50 2024-05-22T21:24:19.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.547+0000 7f8ea3c00640 1 -- 172.21.15.161:0/1374633680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e9c06c310 con 0x7f8e9c15dd50 2024-05-22T21:24:19.549 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.547+0000 7f8e99600640 1 -- 172.21.15.161:0/1374633680 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+16235 (secure 0 0 0) 0x7f8e84050a90 con 0x7f8e9c15dd50 2024-05-22T21:24:19.550 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:19.550 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":178,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:23:50.299404+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:23:02.242525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-05-22T21:23:02.864916+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:24:19.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.551+0000 7f8ea3c00640 1 -- 172.21.15.161:0/1374633680 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e78066690 msgr2=0x7f8e78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.551+0000 7f8ea3c00640 1 --2- 172.21.15.161:0/1374633680 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e78066690 0x7f8e78068b50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c007a70 tx=0x7f8e8c008040 comp rx=0 tx=0).stop 2024-05-22T21:24:19.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.551+0000 7f8ea3c00640 1 -- 172.21.15.161:0/1374633680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9c15dd50 msgr2=0x7f8e9c1407b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.551+0000 7f8ea3c00640 1 --2- 172.21.15.161:0/1374633680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9c15dd50 0x7f8e9c1407b0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400c420 tx=0x7f8e8400c8e0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.551+0000 7f8ea3c00640 1 -- 172.21.15.161:0/1374633680 shutdown_connections 2024-05-22T21:24:19.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.551+0000 7f8ea3c00640 1 --2- 172.21.15.161:0/1374633680 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8e78066690 0x7f8e78068b50 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.551+0000 7f8ea3c00640 1 --2- 172.21.15.161:0/1374633680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8e9c15e130 0x7f8e9c0b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.551+0000 7f8ea3c00640 1 --2- 172.21.15.161:0/1374633680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9c15dd50 0x7f8e9c1407b0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.551+0000 7f8ea3c00640 1 --2- 172.21.15.161:0/1374633680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e9c15d5e0 0x7f8e9c155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.551+0000 7f8ea3c00640 1 -- 172.21.15.161:0/1374633680 >> 172.21.15.161:0/1374633680 conn(0x7f8e9c0b0bc0 msgr2=0x7f8e9c0adc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:19.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.551+0000 7f8ea3c00640 1 -- 172.21.15.161:0/1374633680 shutdown_connections 2024-05-22T21:24:19.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.551+0000 7f8ea3c00640 1 -- 172.21.15.161:0/1374633680 wait complete. 2024-05-22T21:24:19.570 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:23:02.242525+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '174', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:23:02.864916+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '178', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:24:19.571 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:24:19.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.707+0000 7f6af4600640 1 Processor -- start 2024-05-22T21:24:19.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.711+0000 7f6af4600640 1 -- start start 2024-05-22T21:24:19.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.711+0000 7f6af4600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6aec143c40 0x7f6aec15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.711+0000 7f6af4600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6aec06c9b0 0x7f6aec0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.711+0000 7f6af4600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec15efb0 con 0x7f6aec1444a0 2024-05-22T21:24:19.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.711+0000 7f6af4600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aec15f290 con 0x7f6aec143c40 2024-05-22T21:24:19.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.711+0000 7f6af4600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec15f570 con 0x7f6aec06c9b0 2024-05-22T21:24:19.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.711+0000 7f6af1c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6aec143c40 0x7f6aec15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:19.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.711+0000 7f6af1200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f6aec1444a0 0x7f6aec06c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56924/0 (socket says 172.21.15.161:56924) 2024-05-22T21:24:19.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.711+0000 7f6af1200640 1 -- 172.21.15.161:0/2198969820 learned_addr learned my addr 172.21.15.161:0/2198969820 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:19.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.711+0000 7f6af1c00640 1 -- 172.21.15.161:0/2198969820 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6aec06c9b0 msgr2=0x7f6aec0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:19.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.711+0000 7f6af1c00640 1 --2- 172.21.15.161:0/2198969820 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6aec06c9b0 0x7f6aec0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.711+0000 7f6af1c00640 1 -- 172.21.15.161:0/2198969820 >> v1:172.21.15.161:6789/0 conn(0x7f6aec1444a0 legacy=0x7f6aec06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.711+0000 7f6af1c00640 1 -- 172.21.15.161:0/2198969820 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec15f850 con 0x7f6aec143c40 2024-05-22T21:24:19.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.711+0000 7f6aebe00640 1 -- 172.21.15.161:0/2198969820 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539826113 0 0) 0x7f6aec15efb0 con 0x7f6aec1444a0 2024-05-22T21:24:19.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.715+0000 7f6af1c00640 1 --2- 172.21.15.161:0/2198969820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec143c40 0x7f6aec15b600 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00097c0 tx=0x7f6ae001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1947e8511aaaf5ff server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:19.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.715+0000 7f6aebe00640 1 -- 172.21.15.161:0/2198969820 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00ab600 con 0x7f6aec143c40 2024-05-22T21:24:19.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.715+0000 7f6aebe00640 1 -- 172.21.15.161:0/2198969820 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae00ab760 con 0x7f6aec143c40 2024-05-22T21:24:19.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.715+0000 7f6aebe00640 1 -- 172.21.15.161:0/2198969820 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00abaf0 con 0x7f6aec143c40 2024-05-22T21:24:19.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.715+0000 7f6af4600640 1 -- 172.21.15.161:0/2198969820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec143c40 msgr2=0x7f6aec15b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.715+0000 7f6af4600640 1 --2- 172.21.15.161:0/2198969820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec143c40 0x7f6aec15b600 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00097c0 tx=0x7f6ae001c4a0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.715+0000 7f6af4600640 1 -- 172.21.15.161:0/2198969820 shutdown_connections 2024-05-22T21:24:19.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.715+0000 7f6af4600640 1 --2- 172.21.15.161:0/2198969820 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6aec06c9b0 0x7f6aec0b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.715+0000 7f6af4600640 1 --2- 172.21.15.161:0/2198969820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec143c40 0x7f6aec15b600 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.715+0000 7f6af4600640 1 -- 172.21.15.161:0/2198969820 >> 172.21.15.161:0/2198969820 conn(0x7f6aec06a2d0 msgr2=0x7f6aec06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:19.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.715+0000 7f6af4600640 1 -- 172.21.15.161:0/2198969820 shutdown_connections 2024-05-22T21:24:19.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.715+0000 7f6af4600640 1 -- 172.21.15.161:0/2198969820 wait complete. 2024-05-22T21:24:19.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.715+0000 7f6af4600640 1 Processor -- start 2024-05-22T21:24:19.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.715+0000 7f6af4600640 1 -- start start 2024-05-22T21:24:19.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af4600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6aec06c9b0 0x7f6aec159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af4600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec143c40 0x7f6aec159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af4600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6aec1444a0 0x7f6aec139a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af4600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec158410 con 0x7f6aec143c40 2024-05-22T21:24:19.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af4600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec159210 con 0x7f6aec06c9b0 2024-05-22T21:24:19.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af4600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec158600 con 0x7f6aec1444a0 2024-05-22T21:24:19.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af1200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec143c40 0x7f6aec159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:19.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af1200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec143c40 0x7f6aec159eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57154/0 (socket says 172.21.15.161:57154) 2024-05-22T21:24:19.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af1200640 1 -- 172.21.15.161:0/153412487 learned_addr learned my addr 172.21.15.161:0/153412487 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:19.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af1c00640 1 --2- 172.21.15.161:0/153412487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6aec06c9b0 0x7f6aec159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:19.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af1200640 1 -- 172.21.15.161:0/153412487 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6aec1444a0 msgr2=0x7f6aec139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af2600640 1 --2- 172.21.15.161:0/153412487 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6aec1444a0 0x7f6aec139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:19.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af1200640 1 --2- 172.21.15.161:0/153412487 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6aec1444a0 0x7f6aec139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af1200640 1 -- 172.21.15.161:0/153412487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6aec06c9b0 msgr2=0x7f6aec159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af1200640 1 --2- 172.21.15.161:0/153412487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6aec06c9b0 0x7f6aec159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af1200640 1 -- 172.21.15.161:0/153412487 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae0009420 con 0x7f6aec143c40 2024-05-22T21:24:19.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af1c00640 1 --2- 172.21.15.161:0/153412487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6aec06c9b0 0x7f6aec159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:19.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af2600640 1 --2- 172.21.15.161:0/153412487 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6aec1444a0 0x7f6aec139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:19.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af1200640 1 --2- 172.21.15.161:0/153412487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec143c40 0x7f6aec159eb0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800c480 tx=0x7f6ad800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:19.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6ae9600640 1 -- 172.21.15.161:0/153412487 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad80074e0 con 0x7f6aec143c40 2024-05-22T21:24:19.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6ae9600640 1 -- 172.21.15.161:0/153412487 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad8007660 con 0x7f6aec143c40 2024-05-22T21:24:19.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6ae9600640 1 -- 172.21.15.161:0/153412487 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8004a30 con 0x7f6aec143c40 2024-05-22T21:24:19.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af4600640 1 -- 172.21.15.161:0/153412487 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec1588e0 con 0x7f6aec143c40 2024-05-22T21:24:19.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af4600640 1 -- 172.21.15.161:0/153412487 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec158ea0 con 0x7f6aec143c40 2024-05-22T21:24:19.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.719+0000 7f6af4600640 1 -- 172.21.15.161:0/153412487 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aec1593c0 con 0x7f6aec143c40 2024-05-22T21:24:19.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.723+0000 7f6ae9600640 1 -- 172.21.15.161:0/153412487 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6ad8004b90 con 0x7f6aec143c40 2024-05-22T21:24:19.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.723+0000 7f6ae9600640 1 --2- 172.21.15.161:0/153412487 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6acc066690 0x7f6acc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:19.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.723+0000 7f6ae9600640 1 -- 172.21.15.161:0/153412487 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f6ad8083320 con 0x7f6aec143c40 2024-05-22T21:24:19.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.723+0000 7f6af1c00640 1 --2- 172.21.15.161:0/153412487 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6acc066690 0x7f6acc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:19.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.723+0000 7f6ae9600640 1 -- 172.21.15.161:0/153412487 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7f6acc06e220 con 0x7f6aec143c40 2024-05-22T21:24:19.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.723+0000 7f6af1c00640 1 --2- 172.21.15.161:0/153412487 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6acc066690 0x7f6acc068b50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00af000 tx=0x7f6ae0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:19.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.727+0000 7f6ae9600640 1 -- 172.21.15.161:0/153412487 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6ad8018060 con 0x7f6aec143c40 2024-05-22T21:24:19.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.879+0000 7f6af4600640 1 -- 172.21.15.161:0/153412487 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6aec159030 con 0x7f6aec143c40 2024-05-22T21:24:19.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.883+0000 7f6ae9600640 1 -- 172.21.15.161:0/153412487 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v178) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ad8054100 con 0x7f6aec143c40 2024-05-22T21:24:19.882 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:24:19.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.883+0000 7f6af4600640 1 -- 172.21.15.161:0/153412487 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6acc066690 msgr2=0x7f6acc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.883+0000 7f6af4600640 1 --2- 172.21.15.161:0/153412487 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6acc066690 0x7f6acc068b50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00af000 tx=0x7f6ae0008040 comp rx=0 tx=0).stop 2024-05-22T21:24:19.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.883+0000 7f6af4600640 1 -- 172.21.15.161:0/153412487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec143c40 msgr2=0x7f6aec159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:19.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.883+0000 7f6af4600640 1 --2- 172.21.15.161:0/153412487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec143c40 0x7f6aec159eb0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800c480 tx=0x7f6ad800c940 comp rx=0 tx=0).stop 2024-05-22T21:24:19.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.883+0000 7f6af4600640 1 -- 172.21.15.161:0/153412487 shutdown_connections 2024-05-22T21:24:19.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.883+0000 7f6af4600640 1 --2- 172.21.15.161:0/153412487 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6acc066690 0x7f6acc068b50 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.883+0000 7f6af4600640 1 --2- 172.21.15.161:0/153412487 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6aec1444a0 0x7f6aec139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.883+0000 7f6af4600640 1 --2- 172.21.15.161:0/153412487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec143c40 0x7f6aec159eb0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.883+0000 7f6af4600640 1 --2- 172.21.15.161:0/153412487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6aec06c9b0 0x7f6aec159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:19.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.883+0000 7f6af4600640 1 -- 172.21.15.161:0/153412487 >> 172.21.15.161:0/153412487 conn(0x7f6aec06a2d0 msgr2=0x7f6aec13f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:19.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.883+0000 7f6af4600640 1 -- 172.21.15.161:0/153412487 shutdown_connections 2024-05-22T21:24:19.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:19.883+0000 7f6af4600640 1 -- 172.21.15.161:0/153412487 wait complete. 2024-05-22T21:24:19.903 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:24:20.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.043+0000 7fd12c400640 1 Processor -- start 2024-05-22T21:24:20.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.043+0000 7fd12c400640 1 -- start start 2024-05-22T21:24:20.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.047+0000 7fd12c400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd1241596b0 0x7fd124071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.047+0000 7fd12c400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd124071db0 0x7fd12406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.047+0000 7fd12c400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1240b3110 con 0x7fd12406d070 2024-05-22T21:24:20.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.047+0000 7fd12c400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12406d4e0 con 0x7fd1241596b0 2024-05-22T21:24:20.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.047+0000 7fd12c400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12415d250 con 0x7fd124071db0 2024-05-22T21:24:20.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.047+0000 7fd129a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd1241596b0 0x7fd124071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.047+0000 7fd129a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd1241596b0 0x7fd124071870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57164/0 (socket says 172.21.15.161:57164) 2024-05-22T21:24:20.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.047+0000 7fd129a00640 1 -- 172.21.15.161:0/54991921 learned_addr learned my addr 172.21.15.161:0/54991921 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:20.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.047+0000 7fd12a400640 1 --1- 172.21.15.161:0/54991921 >> v1:172.21.15.161:6789/0 conn(0x7fd12406d070 0x7fd12406fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56934/0 (socket says 172.21.15.161:56934) 2024-05-22T21:24:20.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.047+0000 7fd129000640 1 --2- 172.21.15.161:0/54991921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd124071db0 0x7fd12406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.047+0000 7fd129a00640 1 -- 172.21.15.161:0/54991921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd124071db0 msgr2=0x7fd12406cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:20.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.047+0000 7fd129a00640 1 --2- 172.21.15.161:0/54991921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd124071db0 0x7fd12406cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.047+0000 7fd129a00640 1 -- 172.21.15.161:0/54991921 >> v1:172.21.15.161:6789/0 conn(0x7fd12406d070 legacy=0x7fd12406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:20.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.047+0000 7fd129a00640 1 -- 172.21.15.161:0/54991921 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd124159a90 con 0x7fd1241596b0 2024-05-22T21:24:20.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.047+0000 7fd123e00640 1 -- 172.21.15.161:0/54991921 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392344705 0 0) 0x7fd1240b3110 con 0x7fd12406d070 2024-05-22T21:24:20.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd129a00640 1 --2- 172.21.15.161:0/54991921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1241596b0 0x7fd124071870 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fd118006fd0 tx=0x7fd11801e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e91b7fea74876c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:20.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd123e00640 1 -- 172.21.15.161:0/54991921 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118004230 con 0x7fd1241596b0 2024-05-22T21:24:20.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd123e00640 1 -- 172.21.15.161:0/54991921 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1180ab070 con 0x7fd1241596b0 2024-05-22T21:24:20.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd123e00640 1 -- 172.21.15.161:0/54991921 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1180a7750 con 0x7fd1241596b0 2024-05-22T21:24:20.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd12c400640 1 -- 172.21.15.161:0/54991921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1241596b0 msgr2=0x7fd124071870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:20.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd12c400640 1 --2- 172.21.15.161:0/54991921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1241596b0 0x7fd124071870 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fd118006fd0 tx=0x7fd11801e9f0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd12c400640 1 -- 172.21.15.161:0/54991921 shutdown_connections 2024-05-22T21:24:20.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd12c400640 1 --2- 172.21.15.161:0/54991921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd124071db0 0x7fd12406cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd12c400640 1 --2- 172.21.15.161:0/54991921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1241596b0 0x7fd124071870 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd12c400640 1 -- 172.21.15.161:0/54991921 >> 172.21.15.161:0/54991921 conn(0x7fd12406a2d0 msgr2=0x7fd12406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:20.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd12c400640 1 -- 172.21.15.161:0/54991921 shutdown_connections 2024-05-22T21:24:20.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd12c400640 1 -- 172.21.15.161:0/54991921 wait complete. 2024-05-22T21:24:20.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd12c400640 1 Processor -- start 2024-05-22T21:24:20.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd12c400640 1 -- start start 2024-05-22T21:24:20.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd12c400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd12406d070 0x7fd12415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd12c400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd124071db0 0x7fd12414da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd12c400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1241596b0 0x7fd124146fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd12c400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1241408f0 con 0x7fd124071db0 2024-05-22T21:24:20.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd12c400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12414ea50 con 0x7fd1241596b0 2024-05-22T21:24:20.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd12c400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12414ed30 con 0x7fd12406d070 2024-05-22T21:24:20.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd129000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd124071db0 0x7fd12414da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.051+0000 7fd129000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd124071db0 0x7fd12414da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57176/0 (socket says 172.21.15.161:57176) 2024-05-22T21:24:20.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.055+0000 7fd129000640 1 -- 172.21.15.161:0/359321416 learned_addr learned my addr 172.21.15.161:0/359321416 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:20.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.055+0000 7fd129a00640 1 --2- 172.21.15.161:0/359321416 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd12406d070 0x7fd12415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.055+0000 7fd12a400640 1 --2- 172.21.15.161:0/359321416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1241596b0 0x7fd124146fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.055+0000 7fd129000640 1 -- 172.21.15.161:0/359321416 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd12406d070 msgr2=0x7fd12415b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:20.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.055+0000 7fd129000640 1 --2- 172.21.15.161:0/359321416 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd12406d070 0x7fd12415b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.055+0000 7fd129000640 1 -- 172.21.15.161:0/359321416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1241596b0 msgr2=0x7fd124146fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:20.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.055+0000 7fd129000640 1 --2- 172.21.15.161:0/359321416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1241596b0 0x7fd124146fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.055+0000 7fd129000640 1 -- 172.21.15.161:0/359321416 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11800b780 con 0x7fd124071db0 2024-05-22T21:24:20.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.055+0000 7fd129000640 1 --2- 172.21.15.161:0/359321416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd124071db0 0x7fd12414da80 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fd11000d7d0 tx=0x7fd11000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:20.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.055+0000 7fd121600640 1 -- 172.21.15.161:0/359321416 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11000b6c0 con 0x7fd124071db0 2024-05-22T21:24:20.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.055+0000 7fd121600640 1 -- 172.21.15.161:0/359321416 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11000b820 con 0x7fd124071db0 2024-05-22T21:24:20.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.055+0000 7fd121600640 1 -- 172.21.15.161:0/359321416 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11000bbd0 con 0x7fd124071db0 2024-05-22T21:24:20.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.055+0000 7fd12c400640 1 -- 172.21.15.161:0/359321416 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12414f010 con 0x7fd124071db0 2024-05-22T21:24:20.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.055+0000 7fd12c400640 1 -- 172.21.15.161:0/359321416 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12414f5d0 con 0x7fd124071db0 2024-05-22T21:24:20.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.055+0000 7fd12c400640 1 -- 172.21.15.161:0/359321416 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12414f790 con 0x7fd124071db0 2024-05-22T21:24:20.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.055+0000 7fd121600640 1 -- 172.21.15.161:0/359321416 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd11000bd30 con 0x7fd124071db0 2024-05-22T21:24:20.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.059+0000 7fd121600640 1 --2- 172.21.15.161:0/359321416 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd104066690 0x7fd104068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.059+0000 7fd121600640 1 -- 172.21.15.161:0/359321416 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd1100847c0 con 0x7fd124071db0 2024-05-22T21:24:20.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.059+0000 7fd129a00640 1 --2- 172.21.15.161:0/359321416 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd104066690 0x7fd104068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.059+0000 7fd121600640 1 -- 172.21.15.161:0/359321416 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7fd10406e220 con 0x7fd124071db0 2024-05-22T21:24:20.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.059+0000 7fd129a00640 1 --2- 172.21.15.161:0/359321416 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd104066690 0x7fd104068b50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd1180b1000 tx=0x7fd1180b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:20.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.063+0000 7fd121600640 1 -- 172.21.15.161:0/359321416 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd1100519e0 con 0x7fd124071db0 2024-05-22T21:24:20.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.215+0000 7fd12c400640 1 -- 172.21.15.161:0/359321416 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd124067220 con 0x7fd124071db0 2024-05-22T21:24:20.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.215+0000 7fd121600640 1 -- 172.21.15.161:0/359321416 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v178) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd110055000 con 0x7fd124071db0 2024-05-22T21:24:20.217 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:24:20.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.219+0000 7fd12c400640 1 -- 172.21.15.161:0/359321416 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd104066690 msgr2=0x7fd104068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:20.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.219+0000 7fd12c400640 1 --2- 172.21.15.161:0/359321416 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd104066690 0x7fd104068b50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd1180b1000 tx=0x7fd1180b3040 comp rx=0 tx=0).stop 2024-05-22T21:24:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.219+0000 7fd12c400640 1 -- 172.21.15.161:0/359321416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd124071db0 msgr2=0x7fd12414da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.219+0000 7fd12c400640 1 --2- 172.21.15.161:0/359321416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd124071db0 0x7fd12414da80 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fd11000d7d0 tx=0x7fd11000dc90 comp rx=0 tx=0).stop 2024-05-22T21:24:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.219+0000 7fd12c400640 1 -- 172.21.15.161:0/359321416 shutdown_connections 2024-05-22T21:24:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.219+0000 7fd12c400640 1 --2- 172.21.15.161:0/359321416 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd104066690 0x7fd104068b50 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.219+0000 7fd12c400640 1 --2- 172.21.15.161:0/359321416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1241596b0 0x7fd124146fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.219+0000 7fd12c400640 1 --2- 172.21.15.161:0/359321416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd124071db0 0x7fd12414da80 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.219+0000 7fd12c400640 1 --2- 172.21.15.161:0/359321416 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd12406d070 0x7fd12415b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.219+0000 7fd12c400640 1 -- 172.21.15.161:0/359321416 >> 172.21.15.161:0/359321416 conn(0x7fd12406a2d0 msgr2=0x7fd1240aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.219+0000 7fd12c400640 1 -- 172.21.15.161:0/359321416 shutdown_connections 2024-05-22T21:24:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.219+0000 7fd12c400640 1 -- 172.21.15.161:0/359321416 wait complete. 2024-05-22T21:24:20.238 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:24:20.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.375+0000 7f20e8c00640 1 Processor -- start 2024-05-22T21:24:20.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.375+0000 7f20e8c00640 1 -- start start 2024-05-22T21:24:20.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.375+0000 7f20e8c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f20e413cfc0 0x7f20e41402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.375+0000 7f20e8c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20e413fe50 0x7f20e406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.375+0000 7f20e8c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e413d3a0 con 0x7f20e415af60 2024-05-22T21:24:20.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.375+0000 7f20e8c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e40cd920 con 0x7f20e413cfc0 2024-05-22T21:24:20.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.375+0000 7f20e8c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40cdc00 con 0x7f20e413fe50 2024-05-22T21:24:20.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.375+0000 7f20e1e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f20e413cfc0 0x7f20e41402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.379+0000 7f20e1e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f20e413cfc0 0x7f20e41402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57184/0 (socket says 172.21.15.161:57184) 2024-05-22T21:24:20.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.379+0000 7f20e1e00640 1 -- 172.21.15.161:0/269381220 learned_addr learned my addr 172.21.15.161:0/269381220 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:20.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.379+0000 7f20e2800640 1 --2- 172.21.15.161:0/269381220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20e413fe50 0x7f20e406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.379+0000 7f20e1e00640 1 -- 172.21.15.161:0/269381220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20e413fe50 msgr2=0x7f20e406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:20.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.379+0000 7f20e1e00640 1 --2- 172.21.15.161:0/269381220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20e413fe50 0x7f20e406fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.379+0000 7f20e1e00640 1 -- 172.21.15.161:0/269381220 >> v1:172.21.15.179:6789/0 conn(0x7f20e415af60 legacy=0x7f20e413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:20.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.379+0000 7f20e1e00640 1 -- 172.21.15.161:0/269381220 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e40cdee0 con 0x7f20e413cfc0 2024-05-22T21:24:20.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.379+0000 7f20e2800640 1 --2- 172.21.15.161:0/269381220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20e413fe50 0x7f20e406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:20.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.379+0000 7f20e1e00640 1 --2- 172.21.15.161:0/269381220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20e413cfc0 0x7f20e41402e0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009870 tx=0x7f20cc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ca1ef922496c63 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:20.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.379+0000 7f20e1400640 1 -- 172.21.15.161:0/269381220 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc0ab600 con 0x7f20e413cfc0 2024-05-22T21:24:20.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.379+0000 7f20e1400640 1 -- 172.21.15.161:0/269381220 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc0a7050 con 0x7f20e413cfc0 2024-05-22T21:24:20.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.379+0000 7f20e1400640 1 -- 172.21.15.161:0/269381220 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc0ab980 con 0x7f20e413cfc0 2024-05-22T21:24:20.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e8c00640 1 -- 172.21.15.161:0/269381220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20e413cfc0 msgr2=0x7f20e41402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:20.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e8c00640 1 --2- 172.21.15.161:0/269381220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20e413cfc0 0x7f20e41402e0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009870 tx=0x7f20cc01c460 comp rx=0 tx=0).stop 2024-05-22T21:24:20.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e8c00640 1 -- 172.21.15.161:0/269381220 shutdown_connections 2024-05-22T21:24:20.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e8c00640 1 --2- 172.21.15.161:0/269381220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20e413fe50 0x7f20e406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e8c00640 1 --2- 172.21.15.161:0/269381220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20e413cfc0 0x7f20e41402e0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e8c00640 1 -- 172.21.15.161:0/269381220 >> 172.21.15.161:0/269381220 conn(0x7f20e406a2d0 msgr2=0x7f20e406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:20.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e8c00640 1 -- 172.21.15.161:0/269381220 shutdown_connections 2024-05-22T21:24:20.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e8c00640 1 -- 172.21.15.161:0/269381220 wait complete. 2024-05-22T21:24:20.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e8c00640 1 Processor -- start 2024-05-22T21:24:20.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e8c00640 1 -- start start 2024-05-22T21:24:20.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e8c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20e413cfc0 0x7f20e413a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e8c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20e413fe50 0x7f20e4158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e8c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20e415af60 0x7f20e4158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e8c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e413f6e0 con 0x7f20e415af60 2024-05-22T21:24:20.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e8c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40b2d20 con 0x7f20e413cfc0 2024-05-22T21:24:20.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e8c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40b3000 con 0x7f20e413fe50 2024-05-22T21:24:20.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e2800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20e415af60 0x7f20e4158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e2800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20e415af60 0x7f20e4158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57200/0 (socket says 172.21.15.161:57200) 2024-05-22T21:24:20.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e2800640 1 -- 172.21.15.161:0/3144228962 learned_addr learned my addr 172.21.15.161:0/3144228962 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:20.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e1e00640 1 --2- 172.21.15.161:0/3144228962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20e413cfc0 0x7f20e413a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20d9400640 1 --2- 172.21.15.161:0/3144228962 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20e413fe50 0x7f20e4158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e2800640 1 -- 172.21.15.161:0/3144228962 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20e413fe50 msgr2=0x7f20e4158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:20.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e2800640 1 --2- 172.21.15.161:0/3144228962 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20e413fe50 0x7f20e4158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e2800640 1 -- 172.21.15.161:0/3144228962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20e413cfc0 msgr2=0x7f20e413a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:20.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e2800640 1 --2- 172.21.15.161:0/3144228962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20e413cfc0 0x7f20e413a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e2800640 1 -- 172.21.15.161:0/3144228962 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc0094d0 con 0x7f20e415af60 2024-05-22T21:24:20.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.383+0000 7f20e2800640 1 --2- 172.21.15.161:0/3144228962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20e415af60 0x7f20e4158640 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00cea0 tx=0x7f20dc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:20.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.387+0000 7f20daa00640 1 -- 172.21.15.161:0/3144228962 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc003ba0 con 0x7f20e415af60 2024-05-22T21:24:20.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.387+0000 7f20daa00640 1 -- 172.21.15.161:0/3144228962 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc003d20 con 0x7f20e415af60 2024-05-22T21:24:20.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.387+0000 7f20daa00640 1 -- 172.21.15.161:0/3144228962 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc00d880 con 0x7f20e415af60 2024-05-22T21:24:20.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.387+0000 7f20e8c00640 1 -- 172.21.15.161:0/3144228962 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e40b32e0 con 0x7f20e415af60 2024-05-22T21:24:20.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.387+0000 7f20e8c00640 1 -- 172.21.15.161:0/3144228962 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e413a700 con 0x7f20e415af60 2024-05-22T21:24:20.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.387+0000 7f20e8c00640 1 -- 172.21.15.161:0/3144228962 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e4067220 con 0x7f20e415af60 2024-05-22T21:24:20.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.387+0000 7f20daa00640 1 -- 172.21.15.161:0/3144228962 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f20dc00d9e0 con 0x7f20e415af60 2024-05-22T21:24:20.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.387+0000 7f20daa00640 1 --2- 172.21.15.161:0/3144228962 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f20c0066690 0x7f20c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.387+0000 7f20daa00640 1 -- 172.21.15.161:0/3144228962 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f20dc084ab0 con 0x7f20e415af60 2024-05-22T21:24:20.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.387+0000 7f20daa00640 1 -- 172.21.15.161:0/3144228962 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7f20c006e220 con 0x7f20e415af60 2024-05-22T21:24:20.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.387+0000 7f20e1e00640 1 --2- 172.21.15.161:0/3144228962 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f20c0066690 0x7f20c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.387+0000 7f20e1e00640 1 --2- 172.21.15.161:0/3144228962 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f20c0066690 0x7f20c0068b50 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f20cc0ad000 tx=0x7f20cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:20.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.391+0000 7f20daa00640 1 -- 172.21.15.161:0/3144228962 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f20dc051e10 con 0x7f20e415af60 2024-05-22T21:24:20.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.547+0000 7f20e8c00640 1 -- 172.21.15.161:0/3144228962 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20e4071d00 con 0x7f20e415af60 2024-05-22T21:24:20.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.547+0000 7f20daa00640 1 -- 172.21.15.161:0/3144228962 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v178) v1 ==== 100+0+11 (secure 0 0 0) 0x7f20dc055430 con 0x7f20e415af60 2024-05-22T21:24:20.548 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:24:20.550 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.551+0000 7f20e8c00640 1 -- 172.21.15.161:0/3144228962 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f20c0066690 msgr2=0x7f20c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:20.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.551+0000 7f20e8c00640 1 --2- 172.21.15.161:0/3144228962 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f20c0066690 0x7f20c0068b50 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f20cc0ad000 tx=0x7f20cc008040 comp rx=0 tx=0).stop 2024-05-22T21:24:20.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.551+0000 7f20e8c00640 1 -- 172.21.15.161:0/3144228962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20e415af60 msgr2=0x7f20e4158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:20.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.551+0000 7f20e8c00640 1 --2- 172.21.15.161:0/3144228962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20e415af60 0x7f20e4158640 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00cea0 tx=0x7f20dc00cf90 comp rx=0 tx=0).stop 2024-05-22T21:24:20.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.551+0000 7f20e8c00640 1 -- 172.21.15.161:0/3144228962 shutdown_connections 2024-05-22T21:24:20.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.551+0000 7f20e8c00640 1 --2- 172.21.15.161:0/3144228962 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f20c0066690 0x7f20c0068b50 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.551+0000 7f20e8c00640 1 --2- 172.21.15.161:0/3144228962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20e415af60 0x7f20e4158640 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.551+0000 7f20e8c00640 1 --2- 172.21.15.161:0/3144228962 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20e413fe50 0x7f20e4158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.551+0000 7f20e8c00640 1 --2- 172.21.15.161:0/3144228962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20e413cfc0 0x7f20e413a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.551+0000 7f20e8c00640 1 -- 172.21.15.161:0/3144228962 >> 172.21.15.161:0/3144228962 conn(0x7f20e406a2d0 msgr2=0x7f20e40ad7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:20.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.551+0000 7f20e8c00640 1 -- 172.21.15.161:0/3144228962 shutdown_connections 2024-05-22T21:24:20.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.551+0000 7f20e8c00640 1 -- 172.21.15.161:0/3144228962 wait complete. 2024-05-22T21:24:20.570 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:24:20.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d2b600640 1 Processor -- start 2024-05-22T21:24:20.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d2b600640 1 -- start start 2024-05-22T21:24:20.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d2b600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9d2415d4b0 0x7f9d240d38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d2b600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9d240d3e00 0x7f9d240cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d2b600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d240d0100 0x7f9d240d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d2b600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d240cf0d0 con 0x7f9d240d3e00 2024-05-22T21:24:20.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d2b600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d240d0590 con 0x7f9d2415d4b0 2024-05-22T21:24:20.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d2b600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d240c6ec0 con 0x7f9d240d0100 2024-05-22T21:24:20.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d1be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9d240d3e00 0x7f9d240cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d1be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9d240d3e00 0x7f9d240cfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57202/0 (socket says 172.21.15.161:57202) 2024-05-22T21:24:20.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d1be00640 1 -- 172.21.15.161:0/413071621 learned_addr learned my addr 172.21.15.161:0/413071621 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:20.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d28c00640 1 --2- 172.21.15.161:0/413071621 >> v2:172.21.15.179:3300/0 conn(0x7f9d2415d4b0 0x7f9d240d38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d29600640 1 --2- 172.21.15.161:0/413071621 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d240d0100 0x7f9d240d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d1be00640 1 -- 172.21.15.161:0/413071621 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d240d0100 msgr2=0x7f9d240d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:20.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d1be00640 1 --2- 172.21.15.161:0/413071621 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d240d0100 0x7f9d240d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d1be00640 1 -- 172.21.15.161:0/413071621 >> v2:172.21.15.179:3300/0 conn(0x7f9d2415d4b0 msgr2=0x7f9d240d38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:20.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d1be00640 1 --2- 172.21.15.161:0/413071621 >> v2:172.21.15.179:3300/0 conn(0x7f9d2415d4b0 0x7f9d240d38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d1be00640 1 -- 172.21.15.161:0/413071621 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d240ce870 con 0x7f9d240d3e00 2024-05-22T21:24:20.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d29600640 1 --2- 172.21.15.161:0/413071621 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d240d0100 0x7f9d240d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:20.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.707+0000 7f9d28c00640 1 --2- 172.21.15.161:0/413071621 >> v2:172.21.15.179:3300/0 conn(0x7f9d2415d4b0 0x7f9d240d38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:20.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.711+0000 7f9d1be00640 1 --2- 172.21.15.161:0/413071621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d240d3e00 0x7f9d240cfbc0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f9d2408cb10 tx=0x7f9d0c01c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22bdeaa73a260bc server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:20.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.711+0000 7f9d23e00640 1 -- 172.21.15.161:0/413071621 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0ab600 con 0x7f9d240d3e00 2024-05-22T21:24:20.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.711+0000 7f9d23e00640 1 -- 172.21.15.161:0/413071621 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c0a7050 con 0x7f9d240d3e00 2024-05-22T21:24:20.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.711+0000 7f9d23e00640 1 -- 172.21.15.161:0/413071621 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0ab980 con 0x7f9d240d3e00 2024-05-22T21:24:20.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.711+0000 7f9d2b600640 1 -- 172.21.15.161:0/413071621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d240d3e00 msgr2=0x7f9d240cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:20.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.711+0000 7f9d2b600640 1 --2- 172.21.15.161:0/413071621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d240d3e00 0x7f9d240cfbc0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f9d2408cb10 tx=0x7f9d0c01c140 comp rx=0 tx=0).stop 2024-05-22T21:24:20.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.711+0000 7f9d2b600640 1 -- 172.21.15.161:0/413071621 shutdown_connections 2024-05-22T21:24:20.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.711+0000 7f9d2b600640 1 --2- 172.21.15.161:0/413071621 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d240d0100 0x7f9d240d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.711+0000 7f9d2b600640 1 --2- 172.21.15.161:0/413071621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d240d3e00 0x7f9d240cfbc0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.711+0000 7f9d2b600640 1 --2- 172.21.15.161:0/413071621 >> v2:172.21.15.179:3300/0 conn(0x7f9d2415d4b0 0x7f9d240d38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.711+0000 7f9d2b600640 1 -- 172.21.15.161:0/413071621 >> 172.21.15.161:0/413071621 conn(0x7f9d2409c4a0 msgr2=0x7f9d240d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:20.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.711+0000 7f9d2b600640 1 -- 172.21.15.161:0/413071621 shutdown_connections 2024-05-22T21:24:20.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.711+0000 7f9d2b600640 1 -- 172.21.15.161:0/413071621 wait complete. 2024-05-22T21:24:20.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d2b600640 1 Processor -- start 2024-05-22T21:24:20.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d2b600640 1 -- start start 2024-05-22T21:24:20.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d2b600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d240d0100 0x7f9d240ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d2b600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d240d3e00 0x7f9d2415ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d2b600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d2415d4b0 0x7f9d240cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d2b600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2415b590 con 0x7f9d240d3e00 2024-05-22T21:24:20.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d2b600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2415b870 con 0x7f9d2415d4b0 2024-05-22T21:24:20.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d2b600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2415bb50 con 0x7f9d240d0100 2024-05-22T21:24:20.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d1be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d240d3e00 0x7f9d2415ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d1be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d240d3e00 0x7f9d2415ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57206/0 (socket says 172.21.15.161:57206) 2024-05-22T21:24:20.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d1be00640 1 -- 172.21.15.161:0/3643968674 learned_addr learned my addr 172.21.15.161:0/3643968674 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:20.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d29600640 1 --2- 172.21.15.161:0/3643968674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d2415d4b0 0x7f9d240cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d28c00640 1 --2- 172.21.15.161:0/3643968674 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d240d0100 0x7f9d240ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d1be00640 1 -- 172.21.15.161:0/3643968674 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d240d0100 msgr2=0x7f9d240ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:20.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d1be00640 1 --2- 172.21.15.161:0/3643968674 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d240d0100 0x7f9d240ccf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d1be00640 1 -- 172.21.15.161:0/3643968674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d2415d4b0 msgr2=0x7f9d240cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:20.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d1be00640 1 --2- 172.21.15.161:0/3643968674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d2415d4b0 0x7f9d240cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d1be00640 1 -- 172.21.15.161:0/3643968674 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0094d0 con 0x7f9d240d3e00 2024-05-22T21:24:20.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d1be00640 1 --2- 172.21.15.161:0/3643968674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d240d3e00 0x7f9d2415ad60 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0ae040 tx=0x7f9d0c01a980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:20.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d21600640 1 -- 172.21.15.161:0/3643968674 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0b6620 con 0x7f9d240d3e00 2024-05-22T21:24:20.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d21600640 1 -- 172.21.15.161:0/3643968674 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c0b67a0 con 0x7f9d240d3e00 2024-05-22T21:24:20.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d2b600640 1 -- 172.21.15.161:0/3643968674 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2415be30 con 0x7f9d240d3e00 2024-05-22T21:24:20.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d21600640 1 -- 172.21.15.161:0/3643968674 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0b6b50 con 0x7f9d240d3e00 2024-05-22T21:24:20.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d2b600640 1 -- 172.21.15.161:0/3643968674 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2415b2a0 con 0x7f9d240d3e00 2024-05-22T21:24:20.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.715+0000 7f9d2b600640 1 -- 172.21.15.161:0/3643968674 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d240b90a0 con 0x7f9d240d3e00 2024-05-22T21:24:20.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.719+0000 7f9d21600640 1 -- 172.21.15.161:0/3643968674 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9d0c0b2040 con 0x7f9d240d3e00 2024-05-22T21:24:20.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.719+0000 7f9d21600640 1 --2- 172.21.15.161:0/3643968674 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d00066690 0x7f9d00068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:20.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.719+0000 7f9d21600640 1 -- 172.21.15.161:0/3643968674 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9d0c11e760 con 0x7f9d240d3e00 2024-05-22T21:24:20.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.719+0000 7f9d28c00640 1 --2- 172.21.15.161:0/3643968674 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d00066690 0x7f9d00068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:20.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.719+0000 7f9d21600640 1 -- 172.21.15.161:0/3643968674 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7f9d0006e220 con 0x7f9d240d3e00 2024-05-22T21:24:20.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.719+0000 7f9d28c00640 1 --2- 172.21.15.161:0/3643968674 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d00066690 0x7f9d00068b50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9d14006fd0 tx=0x7f9d14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:20.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.723+0000 7f9d21600640 1 -- 172.21.15.161:0/3643968674 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f9d0c0ebac0 con 0x7f9d240d3e00 2024-05-22T21:24:20.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.907+0000 7f9d2b600640 1 -- 172.21.15.161:0/3643968674 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d24067220 con 0x7f9d240d3e00 2024-05-22T21:24:20.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.907+0000 7f9d21600640 1 -- 172.21.15.161:0/3643968674 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4747 (secure 0 0 0) 0x7f9d0c0ba090 con 0x7f9d240d3e00 2024-05-22T21:24:20.908 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:20.908 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2405126804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2405126804},{"type":"v1","addr":"172.21.15.179:6834","nonce":2405126804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":6962,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3984924056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3984924056},{"type":"v1","addr":"172.21.15.179:6835","nonce":3984924056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8391,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/368424851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":368424851},{"type":"v1","addr":"172.21.15.161:6836","nonce":368424851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:23:03.863540+0000","modified":"2024-05-22T21:23:06.877759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2279022192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2279022192},{"type":"v1","addr":"172.21.15.161:6837","nonce":2279022192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-05-22T21:24:20.908 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 78 2024-05-22T21:24:20.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.907+0000 7f9d2b600640 1 -- 172.21.15.161:0/3643968674 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d00066690 msgr2=0x7f9d00068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:20.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.907+0000 7f9d2b600640 1 --2- 172.21.15.161:0/3643968674 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d00066690 0x7f9d00068b50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9d14006fd0 tx=0x7f9d14008040 comp rx=0 tx=0).stop 2024-05-22T21:24:20.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.907+0000 7f9d2b600640 1 -- 172.21.15.161:0/3643968674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d240d3e00 msgr2=0x7f9d2415ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:20.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.907+0000 7f9d2b600640 1 --2- 172.21.15.161:0/3643968674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d240d3e00 0x7f9d2415ad60 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0ae040 tx=0x7f9d0c01a980 comp rx=0 tx=0).stop 2024-05-22T21:24:20.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.907+0000 7f9d2b600640 1 -- 172.21.15.161:0/3643968674 shutdown_connections 2024-05-22T21:24:20.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.907+0000 7f9d2b600640 1 --2- 172.21.15.161:0/3643968674 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9d00066690 0x7f9d00068b50 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.907+0000 7f9d2b600640 1 --2- 172.21.15.161:0/3643968674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d2415d4b0 0x7f9d240cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.907+0000 7f9d2b600640 1 --2- 172.21.15.161:0/3643968674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d240d3e00 0x7f9d2415ad60 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.907+0000 7f9d2b600640 1 --2- 172.21.15.161:0/3643968674 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d240d0100 0x7f9d240ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:20.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.907+0000 7f9d2b600640 1 -- 172.21.15.161:0/3643968674 >> 172.21.15.161:0/3643968674 conn(0x7f9d2409c4a0 msgr2=0x7f9d240b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:20.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.911+0000 7f9d2b600640 1 -- 172.21.15.161:0/3643968674 shutdown_connections 2024-05-22T21:24:20.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:20.911+0000 7f9d2b600640 1 -- 172.21.15.161:0/3643968674 wait complete. 2024-05-22T21:24:20.931 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:24:21.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.067+0000 7f4ffd200640 1 Processor -- start 2024-05-22T21:24:21.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.067+0000 7f4ffd200640 1 -- start start 2024-05-22T21:24:21.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.067+0000 7f4ffd200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4ff815e130 0x7f4ff815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.067+0000 7f4ffd200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4ff815d410 0x7f4ff815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.067+0000 7f4ffd200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff8071670 con 0x7f4ff815dd50 2024-05-22T21:24:21.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.067+0000 7f4ffd200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff815d870 con 0x7f4ff815e130 2024-05-22T21:24:21.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.067+0000 7f4ffd200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff8070a60 con 0x7f4ff815d410 2024-05-22T21:24:21.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.067+0000 7f4fed400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4ff815e130 0x7f4ff815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.067+0000 7f4fed400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4ff815e130 0x7f4ff815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57214/0 (socket says 172.21.15.161:57214) 2024-05-22T21:24:21.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.067+0000 7f4fed400640 1 -- 172.21.15.161:0/187049566 learned_addr learned my addr 172.21.15.161:0/187049566 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:21.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.067+0000 7f4fed400640 1 -- 172.21.15.161:0/187049566 >> v2:172.21.15.179:3300/0 conn(0x7f4ff815d410 msgr2=0x7f4ff815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:21.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.067+0000 7f4ff6800640 1 --2- 172.21.15.161:0/187049566 >> v2:172.21.15.179:3300/0 conn(0x7f4ff815d410 0x7f4ff815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.067+0000 7f4fed400640 1 --2- 172.21.15.161:0/187049566 >> v2:172.21.15.179:3300/0 conn(0x7f4ff815d410 0x7f4ff815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.067+0000 7f4fed400640 1 -- 172.21.15.161:0/187049566 >> v1:172.21.15.179:6789/0 conn(0x7f4ff815dd50 legacy=0x7f4ff806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:21.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.067+0000 7f4fed400640 1 -- 172.21.15.161:0/187049566 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff8070d40 con 0x7f4ff815e130 2024-05-22T21:24:21.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.067+0000 7f4ff6800640 1 --2- 172.21.15.161:0/187049566 >> v2:172.21.15.179:3300/0 conn(0x7f4ff815d410 0x7f4ff815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:21.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.071+0000 7f4fed400640 1 --2- 172.21.15.161:0/187049566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ff815e130 0x7f4ff815ced0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f4ff815e5c0 tx=0x7f4fe001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f639f19cd38166e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:21.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.071+0000 7f4ff5400640 1 -- 172.21.15.161:0/187049566 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00aa610 con 0x7f4ff815e130 2024-05-22T21:24:21.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.071+0000 7f4ff5400640 1 -- 172.21.15.161:0/187049566 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe00a6040 con 0x7f4ff815e130 2024-05-22T21:24:21.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.071+0000 7f4ff5400640 1 -- 172.21.15.161:0/187049566 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00aa990 con 0x7f4ff815e130 2024-05-22T21:24:21.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.071+0000 7f4ffd200640 1 -- 172.21.15.161:0/187049566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ff815e130 msgr2=0x7f4ff815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:21.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.071+0000 7f4ffd200640 1 --2- 172.21.15.161:0/187049566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ff815e130 0x7f4ff815ced0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f4ff815e5c0 tx=0x7f4fe001c4b0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.071+0000 7f4ffd200640 1 -- 172.21.15.161:0/187049566 shutdown_connections 2024-05-22T21:24:21.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.071+0000 7f4ffd200640 1 --2- 172.21.15.161:0/187049566 >> v2:172.21.15.179:3300/0 conn(0x7f4ff815d410 0x7f4ff815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.071+0000 7f4ffd200640 1 --2- 172.21.15.161:0/187049566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ff815e130 0x7f4ff815ced0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.071+0000 7f4ffd200640 1 -- 172.21.15.161:0/187049566 >> 172.21.15.161:0/187049566 conn(0x7f4ff80b0bc0 msgr2=0x7f4ff80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:21.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.071+0000 7f4ffd200640 1 -- 172.21.15.161:0/187049566 shutdown_connections 2024-05-22T21:24:21.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.071+0000 7f4ffd200640 1 -- 172.21.15.161:0/187049566 wait complete. 2024-05-22T21:24:21.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ffd200640 1 Processor -- start 2024-05-22T21:24:21.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ffd200640 1 -- start start 2024-05-22T21:24:21.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ffd200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ff815d410 0x7f4ff81561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ffd200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4ff815dd50 0x7f4ff8152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ffd200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ff815e130 0x7f4ff814c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ffd200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff80b3b50 con 0x7f4ff815d410 2024-05-22T21:24:21.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ffd200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff80b2f40 con 0x7f4ff815e130 2024-05-22T21:24:21.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ffd200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff80b3220 con 0x7f4ff815dd50 2024-05-22T21:24:21.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ff5e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ff815d410 0x7f4ff81561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ff5e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ff815d410 0x7f4ff81561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57218/0 (socket says 172.21.15.161:57218) 2024-05-22T21:24:21.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ff5e00640 1 -- 172.21.15.161:0/2159072819 learned_addr learned my addr 172.21.15.161:0/2159072819 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:21.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ff6800640 1 --2- 172.21.15.161:0/2159072819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ff815e130 0x7f4ff814c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4fed400640 1 --2- 172.21.15.161:0/2159072819 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4ff815dd50 0x7f4ff8152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ff5e00640 1 -- 172.21.15.161:0/2159072819 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4ff815dd50 msgr2=0x7f4ff8152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:21.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ff5e00640 1 --2- 172.21.15.161:0/2159072819 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4ff815dd50 0x7f4ff8152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ff5e00640 1 -- 172.21.15.161:0/2159072819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ff815e130 msgr2=0x7f4ff814c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:21.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ff5e00640 1 --2- 172.21.15.161:0/2159072819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ff815e130 0x7f4ff814c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ff5e00640 1 -- 172.21.15.161:0/2159072819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe00094d0 con 0x7f4ff815d410 2024-05-22T21:24:21.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ff6800640 1 --2- 172.21.15.161:0/2159072819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ff815e130 0x7f4ff814c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:21.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ff5e00640 1 --2- 172.21.15.161:0/2159072819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ff815d410 0x7f4ff81561d0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800c7c0 tx=0x7f4fe800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:21.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4feea00640 1 -- 172.21.15.161:0/2159072819 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe800aa50 con 0x7f4ff815d410 2024-05-22T21:24:21.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4feea00640 1 -- 172.21.15.161:0/2159072819 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe800abb0 con 0x7f4ff815d410 2024-05-22T21:24:21.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ffd200640 1 -- 172.21.15.161:0/2159072819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff80b3500 con 0x7f4ff815d410 2024-05-22T21:24:21.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4feea00640 1 -- 172.21.15.161:0/2159072819 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8012420 con 0x7f4ff815d410 2024-05-22T21:24:21.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ffd200640 1 -- 172.21.15.161:0/2159072819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff80b2bd0 con 0x7f4ff815d410 2024-05-22T21:24:21.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.075+0000 7f4ffd200640 1 -- 172.21.15.161:0/2159072819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff8067220 con 0x7f4ff815d410 2024-05-22T21:24:21.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.079+0000 7f4feea00640 1 -- 172.21.15.161:0/2159072819 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4fe8012580 con 0x7f4ff815d410 2024-05-22T21:24:21.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.079+0000 7f4feea00640 1 --2- 172.21.15.161:0/2159072819 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4fd4066690 0x7f4fd4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.079+0000 7f4feea00640 1 -- 172.21.15.161:0/2159072819 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f4fe8084450 con 0x7f4ff815d410 2024-05-22T21:24:21.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.079+0000 7f4fed400640 1 --2- 172.21.15.161:0/2159072819 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4fd4066690 0x7f4fd4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.079+0000 7f4feea00640 1 -- 172.21.15.161:0/2159072819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7f4fd406e220 con 0x7f4ff815d410 2024-05-22T21:24:21.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.079+0000 7f4fed400640 1 --2- 172.21.15.161:0/2159072819 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4fd4066690 0x7f4fd4068b50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00ae000 tx=0x7f4fe00a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:21.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.083+0000 7f4feea00640 1 -- 172.21.15.161:0/2159072819 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f4fe8051790 con 0x7f4ff815d410 2024-05-22T21:24:21.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.239+0000 7f4ffd200640 1 -- 172.21.15.161:0/2159072819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ff806c310 con 0x7f4ff815d410 2024-05-22T21:24:21.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.239+0000 7f4feea00640 1 -- 172.21.15.161:0/2159072819 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+16235 (secure 0 0 0) 0x7f4fe8016020 con 0x7f4ff815d410 2024-05-22T21:24:21.240 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:21.240 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":178,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:23:50.299404+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:23:02.242525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-05-22T21:23:02.864916+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:24:21.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.243+0000 7f4ffd200640 1 -- 172.21.15.161:0/2159072819 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4fd4066690 msgr2=0x7f4fd4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:21.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.243+0000 7f4ffd200640 1 --2- 172.21.15.161:0/2159072819 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4fd4066690 0x7f4fd4068b50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00ae000 tx=0x7f4fe00a7000 comp rx=0 tx=0).stop 2024-05-22T21:24:21.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.243+0000 7f4ffd200640 1 -- 172.21.15.161:0/2159072819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ff815d410 msgr2=0x7f4ff81561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:21.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.243+0000 7f4ffd200640 1 --2- 172.21.15.161:0/2159072819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ff815d410 0x7f4ff81561d0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800c7c0 tx=0x7f4fe800cc80 comp rx=0 tx=0).stop 2024-05-22T21:24:21.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.243+0000 7f4ffd200640 1 -- 172.21.15.161:0/2159072819 shutdown_connections 2024-05-22T21:24:21.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.243+0000 7f4ffd200640 1 --2- 172.21.15.161:0/2159072819 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4fd4066690 0x7f4fd4068b50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.243+0000 7f4ffd200640 1 --2- 172.21.15.161:0/2159072819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ff815e130 0x7f4ff814c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.243+0000 7f4ffd200640 1 --2- 172.21.15.161:0/2159072819 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4ff815dd50 0x7f4ff8152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.243+0000 7f4ffd200640 1 --2- 172.21.15.161:0/2159072819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ff815d410 0x7f4ff81561d0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.243+0000 7f4ffd200640 1 -- 172.21.15.161:0/2159072819 >> 172.21.15.161:0/2159072819 conn(0x7f4ff80b0bc0 msgr2=0x7f4ff80af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:21.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.243+0000 7f4ffd200640 1 -- 172.21.15.161:0/2159072819 shutdown_connections 2024-05-22T21:24:21.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.243+0000 7f4ffd200640 1 -- 172.21.15.161:0/2159072819 wait complete. 2024-05-22T21:24:21.262 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-22T21:24:21.262 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:24:21.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.395+0000 7f323e600640 1 Processor -- start 2024-05-22T21:24:21.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.395+0000 7f323e600640 1 -- start start 2024-05-22T21:24:21.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.399+0000 7f323e600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f323815dd50 0x7f323806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.399+0000 7f323e600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f323815e130 0x7f323815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.399+0000 7f323e600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3238071670 con 0x7f323815d410 2024-05-22T21:24:21.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.399+0000 7f323e600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323815d8a0 con 0x7f323815e130 2024-05-22T21:24:21.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.399+0000 7f323e600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323806de40 con 0x7f323815dd50 2024-05-22T21:24:21.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.399+0000 7f3237e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f323815d410 0x7f323815a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56942/0 (socket says 172.21.15.161:56942) 2024-05-22T21:24:21.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.399+0000 7f3237e00640 1 -- 172.21.15.161:0/2679850209 learned_addr learned my addr 172.21.15.161:0/2679850209 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:21.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.399+0000 7f322ea00640 1 --2- 172.21.15.161:0/2679850209 >> v2:172.21.15.179:3300/0 conn(0x7f323815e130 0x7f323815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.399+0000 7f3237400640 1 --2- 172.21.15.161:0/2679850209 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f323815dd50 0x7f323806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.399+0000 7f3236a00640 1 -- 172.21.15.161:0/2679850209 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50806593 0 0) 0x7f3238071670 con 0x7f323815d410 2024-05-22T21:24:21.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.399+0000 7f3236a00640 1 -- 172.21.15.161:0/2679850209 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f321c003530 con 0x7f323815d410 2024-05-22T21:24:21.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.399+0000 7f322ea00640 1 -- 172.21.15.161:0/2679850209 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f323815dd50 msgr2=0x7f323806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:21.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.399+0000 7f322ea00640 1 --2- 172.21.15.161:0/2679850209 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f323815dd50 0x7f323806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.399+0000 7f322ea00640 1 -- 172.21.15.161:0/2679850209 >> v1:172.21.15.161:6789/0 conn(0x7f323815d410 legacy=0x7f323815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:21.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.399+0000 7f322ea00640 1 -- 172.21.15.161:0/2679850209 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3238070a60 con 0x7f323815e130 2024-05-22T21:24:21.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.399+0000 7f3237400640 1 --2- 172.21.15.161:0/2679850209 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f323815dd50 0x7f323806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:24:21.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.403+0000 7f322ea00640 1 --2- 172.21.15.161:0/2679850209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f323815e130 0x7f323815ced0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f32200097c0 tx=0x7f322001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc26ba2e547afed7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:21.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.403+0000 7f3236a00640 1 -- 172.21.15.161:0/2679850209 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32200ac640 con 0x7f323815e130 2024-05-22T21:24:21.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.403+0000 7f3236a00640 1 -- 172.21.15.161:0/2679850209 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32200ac7a0 con 0x7f323815e130 2024-05-22T21:24:21.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.403+0000 7f3236a00640 1 -- 172.21.15.161:0/2679850209 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32200acb30 con 0x7f323815e130 2024-05-22T21:24:21.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.403+0000 7f323e600640 1 -- 172.21.15.161:0/2679850209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f323815e130 msgr2=0x7f323815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:21.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.403+0000 7f323e600640 1 --2- 172.21.15.161:0/2679850209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f323815e130 0x7f323815ced0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f32200097c0 tx=0x7f322001c3e0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.403+0000 7f323e600640 1 -- 172.21.15.161:0/2679850209 shutdown_connections 2024-05-22T21:24:21.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.403+0000 7f323e600640 1 --2- 172.21.15.161:0/2679850209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f323815e130 0x7f323815ced0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.403+0000 7f323e600640 1 --2- 172.21.15.161:0/2679850209 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f323815dd50 0x7f323806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.403+0000 7f323e600640 1 -- 172.21.15.161:0/2679850209 >> 172.21.15.161:0/2679850209 conn(0x7f32380b0bc0 msgr2=0x7f32380b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:21.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.403+0000 7f323e600640 1 -- 172.21.15.161:0/2679850209 shutdown_connections 2024-05-22T21:24:21.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.403+0000 7f323e600640 1 -- 172.21.15.161:0/2679850209 wait complete. 2024-05-22T21:24:21.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.403+0000 7f323e600640 1 Processor -- start 2024-05-22T21:24:21.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f323e600640 1 -- start start 2024-05-22T21:24:21.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f323e600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f323815d410 0x7f32380b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f323e600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f323815dd50 0x7f32381425c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f323e600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f323815e130 0x7f32380ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f323e600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32380b2660 con 0x7f323815dd50 2024-05-22T21:24:21.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f323e600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32380b1a50 con 0x7f323815d410 2024-05-22T21:24:21.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f323e600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32380b1d30 con 0x7f323815e130 2024-05-22T21:24:21.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f322ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f323815dd50 0x7f32381425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f322ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f323815dd50 0x7f32381425c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57228/0 (socket says 172.21.15.161:57228) 2024-05-22T21:24:21.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f322ea00640 1 -- 172.21.15.161:0/2827451870 learned_addr learned my addr 172.21.15.161:0/2827451870 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:21.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f3237400640 1 --2- 172.21.15.161:0/2827451870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f323815d410 0x7f32380b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f3237e00640 1 --2- 172.21.15.161:0/2827451870 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f323815e130 0x7f32380ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f322ea00640 1 -- 172.21.15.161:0/2827451870 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f323815e130 msgr2=0x7f32380ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:21.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f322ea00640 1 --2- 172.21.15.161:0/2827451870 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f323815e130 0x7f32380ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f322ea00640 1 -- 172.21.15.161:0/2827451870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f323815d410 msgr2=0x7f32380b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:21.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f322ea00640 1 --2- 172.21.15.161:0/2827451870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f323815d410 0x7f32380b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f322ea00640 1 -- 172.21.15.161:0/2827451870 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3228009420 con 0x7f323815dd50 2024-05-22T21:24:21.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f322ea00640 1 --2- 172.21.15.161:0/2827451870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f323815dd50 0x7f32381425c0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f32200af040 tx=0x7f3220019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:21.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f322fe00640 1 -- 172.21.15.161:0/2827451870 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32200b5610 con 0x7f323815dd50 2024-05-22T21:24:21.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f322fe00640 1 -- 172.21.15.161:0/2827451870 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32200b5770 con 0x7f323815dd50 2024-05-22T21:24:21.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f323e600640 1 -- 172.21.15.161:0/2827451870 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3220009420 con 0x7f323815dd50 2024-05-22T21:24:21.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f322fe00640 1 -- 172.21.15.161:0/2827451870 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32200b5b20 con 0x7f323815dd50 2024-05-22T21:24:21.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f323e600640 1 -- 172.21.15.161:0/2827451870 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32380b22f0 con 0x7f323815dd50 2024-05-22T21:24:21.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.407+0000 7f323e600640 1 -- 172.21.15.161:0/2827451870 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32380b27d0 con 0x7f323815dd50 2024-05-22T21:24:21.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.411+0000 7f322fe00640 1 -- 172.21.15.161:0/2827451870 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f32200b3040 con 0x7f323815dd50 2024-05-22T21:24:21.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.411+0000 7f322fe00640 1 --2- 172.21.15.161:0/2827451870 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3214066690 0x7f3214068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.411+0000 7f322fe00640 1 -- 172.21.15.161:0/2827451870 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f322011e500 con 0x7f323815dd50 2024-05-22T21:24:21.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.411+0000 7f3237400640 1 --2- 172.21.15.161:0/2827451870 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3214066690 0x7f3214068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.411+0000 7f322fe00640 1 -- 172.21.15.161:0/2827451870 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7f321406e220 con 0x7f323815dd50 2024-05-22T21:24:21.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.411+0000 7f3237400640 1 --2- 172.21.15.161:0/2827451870 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3214066690 0x7f3214068b50 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3238152780 tx=0x7f3228009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:21.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.415+0000 7f322fe00640 1 -- 172.21.15.161:0/2827451870 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f32200eb8f0 con 0x7f323815dd50 2024-05-22T21:24:21.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.607+0000 7f323e600640 1 -- 172.21.15.161:0/2827451870 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3238067220 con 0x7f323815dd50 2024-05-22T21:24:21.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.607+0000 7f322fe00640 1 -- 172.21.15.161:0/2827451870 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4747 (secure 0 0 0) 0x7f32200b1070 con 0x7f323815dd50 2024-05-22T21:24:21.608 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:21.608 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2405126804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2405126804},{"type":"v1","addr":"172.21.15.179:6834","nonce":2405126804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":6962,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3984924056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3984924056},{"type":"v1","addr":"172.21.15.179:6835","nonce":3984924056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8391,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/368424851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":368424851},{"type":"v1","addr":"172.21.15.161:6836","nonce":368424851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:23:03.863540+0000","modified":"2024-05-22T21:23:06.877759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2279022192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2279022192},{"type":"v1","addr":"172.21.15.161:6837","nonce":2279022192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-05-22T21:24:21.608 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 78 2024-05-22T21:24:21.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.611+0000 7f323e600640 1 -- 172.21.15.161:0/2827451870 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3214066690 msgr2=0x7f3214068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:21.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.611+0000 7f323e600640 1 --2- 172.21.15.161:0/2827451870 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3214066690 0x7f3214068b50 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3238152780 tx=0x7f3228009190 comp rx=0 tx=0).stop 2024-05-22T21:24:21.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.611+0000 7f323e600640 1 -- 172.21.15.161:0/2827451870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f323815dd50 msgr2=0x7f32381425c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:21.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.611+0000 7f323e600640 1 --2- 172.21.15.161:0/2827451870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f323815dd50 0x7f32381425c0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f32200af040 tx=0x7f3220019180 comp rx=0 tx=0).stop 2024-05-22T21:24:21.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.611+0000 7f323e600640 1 -- 172.21.15.161:0/2827451870 shutdown_connections 2024-05-22T21:24:21.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.611+0000 7f323e600640 1 --2- 172.21.15.161:0/2827451870 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3214066690 0x7f3214068b50 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.611+0000 7f323e600640 1 --2- 172.21.15.161:0/2827451870 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f323815e130 0x7f32380ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.611+0000 7f323e600640 1 --2- 172.21.15.161:0/2827451870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f323815dd50 0x7f32381425c0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.611+0000 7f323e600640 1 --2- 172.21.15.161:0/2827451870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f323815d410 0x7f32380b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.611+0000 7f323e600640 1 -- 172.21.15.161:0/2827451870 >> 172.21.15.161:0/2827451870 conn(0x7f32380b0bc0 msgr2=0x7f32380b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:21.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.611+0000 7f323e600640 1 -- 172.21.15.161:0/2827451870 shutdown_connections 2024-05-22T21:24:21.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.611+0000 7f323e600640 1 -- 172.21.15.161:0/2827451870 wait complete. 2024-05-22T21:24:21.629 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:24:21.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb2ce00640 1 Processor -- start 2024-05-22T21:24:21.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb2ce00640 1 -- start start 2024-05-22T21:24:21.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb2ce00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb28143ec0 0x7feb2815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb2ce00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feb280715a0 0x7feb2806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb2ce00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7feb2815ec90 0x7feb2815f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb2ce00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2815f660 con 0x7feb280715a0 2024-05-22T21:24:21.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb2ce00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb281442a0 con 0x7feb2815ec90 2024-05-22T21:24:21.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb2ce00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb28070260 con 0x7feb28143ec0 2024-05-22T21:24:21.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb25400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feb280715a0 0x7feb2806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb25400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feb280715a0 0x7feb2806fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57230/0 (socket says 172.21.15.161:57230) 2024-05-22T21:24:21.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb25400640 1 -- 172.21.15.161:0/3877283751 learned_addr learned my addr 172.21.15.161:0/3877283751 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:21.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb25e00640 1 --2- 172.21.15.161:0/3877283751 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb28143ec0 0x7feb2815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb26800640 1 --2- 172.21.15.161:0/3877283751 >> v2:172.21.15.179:3300/0 conn(0x7feb2815ec90 0x7feb2815f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb25400640 1 -- 172.21.15.161:0/3877283751 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb28143ec0 msgr2=0x7feb2815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:21.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb25400640 1 --2- 172.21.15.161:0/3877283751 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb28143ec0 0x7feb2815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb25400640 1 -- 172.21.15.161:0/3877283751 >> v2:172.21.15.179:3300/0 conn(0x7feb2815ec90 msgr2=0x7feb2815f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:21.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb25400640 1 --2- 172.21.15.161:0/3877283751 >> v2:172.21.15.179:3300/0 conn(0x7feb2815ec90 0x7feb2815f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.767+0000 7feb25400640 1 -- 172.21.15.161:0/3877283751 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb28070500 con 0x7feb280715a0 2024-05-22T21:24:21.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.771+0000 7feb25400640 1 --2- 172.21.15.161:0/3877283751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb280715a0 0x7feb2806fd20 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7feb280b1be0 tx=0x7feb1401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ac1d822addddd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:21.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.771+0000 7feb24a00640 1 -- 172.21.15.161:0/3877283751 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb140aa610 con 0x7feb280715a0 2024-05-22T21:24:21.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.771+0000 7feb24a00640 1 -- 172.21.15.161:0/3877283751 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb140a6040 con 0x7feb280715a0 2024-05-22T21:24:21.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.771+0000 7feb24a00640 1 -- 172.21.15.161:0/3877283751 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb140aa990 con 0x7feb280715a0 2024-05-22T21:24:21.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.771+0000 7feb2ce00640 1 -- 172.21.15.161:0/3877283751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb280715a0 msgr2=0x7feb2806fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:21.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.771+0000 7feb2ce00640 1 --2- 172.21.15.161:0/3877283751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb280715a0 0x7feb2806fd20 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7feb280b1be0 tx=0x7feb1401c4f0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.771+0000 7feb2ce00640 1 -- 172.21.15.161:0/3877283751 shutdown_connections 2024-05-22T21:24:21.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.771+0000 7feb2ce00640 1 --2- 172.21.15.161:0/3877283751 >> v2:172.21.15.179:3300/0 conn(0x7feb2815ec90 0x7feb2815f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.771+0000 7feb2ce00640 1 --2- 172.21.15.161:0/3877283751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb280715a0 0x7feb2806fd20 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.771+0000 7feb2ce00640 1 --2- 172.21.15.161:0/3877283751 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb28143ec0 0x7feb2815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.771+0000 7feb2ce00640 1 -- 172.21.15.161:0/3877283751 >> 172.21.15.161:0/3877283751 conn(0x7feb280b0bc0 msgr2=0x7feb280b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:21.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.771+0000 7feb2ce00640 1 -- 172.21.15.161:0/3877283751 shutdown_connections 2024-05-22T21:24:21.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.771+0000 7feb2ce00640 1 -- 172.21.15.161:0/3877283751 wait complete. 2024-05-22T21:24:21.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb2ce00640 1 Processor -- start 2024-05-22T21:24:21.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb2ce00640 1 -- start start 2024-05-22T21:24:21.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb2ce00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb28143ec0 0x7feb2815a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb2ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb2815ec90 0x7feb28141450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb2ce00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb2815d6f0 0x7feb281f7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb2ce00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2813ef50 con 0x7feb2815ec90 2024-05-22T21:24:21.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb2ce00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb28141ca0 con 0x7feb2815d6f0 2024-05-22T21:24:21.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb2ce00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2813e900 con 0x7feb28143ec0 2024-05-22T21:24:21.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb25400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb2815ec90 0x7feb28141450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb25400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb2815ec90 0x7feb28141450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57244/0 (socket says 172.21.15.161:57244) 2024-05-22T21:24:21.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb25400640 1 -- 172.21.15.161:0/1870628797 learned_addr learned my addr 172.21.15.161:0/1870628797 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:21.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb25e00640 1 --2- 172.21.15.161:0/1870628797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb28143ec0 0x7feb2815a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb26800640 1 --2- 172.21.15.161:0/1870628797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb2815d6f0 0x7feb281f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb25400640 1 -- 172.21.15.161:0/1870628797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb28143ec0 msgr2=0x7feb2815a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:21.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb25400640 1 --2- 172.21.15.161:0/1870628797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb28143ec0 0x7feb2815a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb25400640 1 -- 172.21.15.161:0/1870628797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb2815d6f0 msgr2=0x7feb281f7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:21.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb25400640 1 --2- 172.21.15.161:0/1870628797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb2815d6f0 0x7feb281f7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb25400640 1 -- 172.21.15.161:0/1870628797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb140094d0 con 0x7feb2815ec90 2024-05-22T21:24:21.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb26800640 1 --2- 172.21.15.161:0/1870628797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb2815d6f0 0x7feb281f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:21.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb25400640 1 --2- 172.21.15.161:0/1870628797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb2815ec90 0x7feb28141450 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7feb1401aba0 tx=0x7feb1401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:21.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb1e000640 1 -- 172.21.15.161:0/1870628797 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb140bd5f0 con 0x7feb2815ec90 2024-05-22T21:24:21.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb1e000640 1 -- 172.21.15.161:0/1870628797 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb140bd750 con 0x7feb2815ec90 2024-05-22T21:24:21.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb2ce00640 1 -- 172.21.15.161:0/1870628797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2813ebe0 con 0x7feb2815ec90 2024-05-22T21:24:21.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb1e000640 1 -- 172.21.15.161:0/1870628797 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb140bdae0 con 0x7feb2815ec90 2024-05-22T21:24:21.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb2ce00640 1 -- 172.21.15.161:0/1870628797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb28141990 con 0x7feb2815ec90 2024-05-22T21:24:21.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.775+0000 7feb2ce00640 1 -- 172.21.15.161:0/1870628797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2813f0c0 con 0x7feb2815ec90 2024-05-22T21:24:21.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.779+0000 7feb1e000640 1 -- 172.21.15.161:0/1870628797 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feb140b33f0 con 0x7feb2815ec90 2024-05-22T21:24:21.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.779+0000 7feb1e000640 1 --2- 172.21.15.161:0/1870628797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feb08066690 0x7feb08068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:21.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.779+0000 7feb1e000640 1 -- 172.21.15.161:0/1870628797 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7feb1411e6e0 con 0x7feb2815ec90 2024-05-22T21:24:21.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.779+0000 7feb25e00640 1 --2- 172.21.15.161:0/1870628797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feb08066690 0x7feb08068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:21.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.779+0000 7feb1e000640 1 -- 172.21.15.161:0/1870628797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7feb0806e220 con 0x7feb2815ec90 2024-05-22T21:24:21.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.779+0000 7feb25e00640 1 --2- 172.21.15.161:0/1870628797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feb08066690 0x7feb08068b50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7feb2806d590 tx=0x7feb180098d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:21.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.783+0000 7feb1e000640 1 -- 172.21.15.161:0/1870628797 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7feb140ebad0 con 0x7feb2815ec90 2024-05-22T21:24:21.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.971+0000 7feb2ce00640 1 -- 172.21.15.161:0/1870628797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb28067220 con 0x7feb2815ec90 2024-05-22T21:24:21.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.971+0000 7feb1e000640 1 -- 172.21.15.161:0/1870628797 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4747 (secure 0 0 0) 0x7feb140ef0f0 con 0x7feb2815ec90 2024-05-22T21:24:21.970 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:21.971 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":78,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2405126804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2405126804},{"type":"v1","addr":"172.21.15.179:6834","nonce":2405126804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":6962,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3984924056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3984924056},{"type":"v1","addr":"172.21.15.179:6835","nonce":3984924056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8391,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/368424851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":368424851},{"type":"v1","addr":"172.21.15.161:6836","nonce":368424851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:23:03.863540+0000","modified":"2024-05-22T21:23:06.877759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2279022192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2279022192},{"type":"v1","addr":"172.21.15.161:6837","nonce":2279022192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-05-22T21:24:21.971 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 78 2024-05-22T21:24:21.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.971+0000 7feb2ce00640 1 -- 172.21.15.161:0/1870628797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feb08066690 msgr2=0x7feb08068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:21.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.971+0000 7feb2ce00640 1 --2- 172.21.15.161:0/1870628797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feb08066690 0x7feb08068b50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7feb2806d590 tx=0x7feb180098d0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.971+0000 7feb2ce00640 1 -- 172.21.15.161:0/1870628797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb2815ec90 msgr2=0x7feb28141450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:21.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.971+0000 7feb2ce00640 1 --2- 172.21.15.161:0/1870628797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb2815ec90 0x7feb28141450 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7feb1401aba0 tx=0x7feb1401ad30 comp rx=0 tx=0).stop 2024-05-22T21:24:21.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.971+0000 7feb2ce00640 1 -- 172.21.15.161:0/1870628797 shutdown_connections 2024-05-22T21:24:21.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.971+0000 7feb2ce00640 1 --2- 172.21.15.161:0/1870628797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feb08066690 0x7feb08068b50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.971+0000 7feb2ce00640 1 --2- 172.21.15.161:0/1870628797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb2815d6f0 0x7feb281f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.971+0000 7feb2ce00640 1 --2- 172.21.15.161:0/1870628797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb2815ec90 0x7feb28141450 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.971+0000 7feb2ce00640 1 --2- 172.21.15.161:0/1870628797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb28143ec0 0x7feb2815a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:21.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.971+0000 7feb2ce00640 1 -- 172.21.15.161:0/1870628797 >> 172.21.15.161:0/1870628797 conn(0x7feb280b0bc0 msgr2=0x7feb2815c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:21.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.971+0000 7feb2ce00640 1 -- 172.21.15.161:0/1870628797 shutdown_connections 2024-05-22T21:24:21.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:21.971+0000 7feb2ce00640 1 -- 172.21.15.161:0/1870628797 wait complete. 2024-05-22T21:24:21.991 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:24:22.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.127+0000 7fe11b400640 1 Processor -- start 2024-05-22T21:24:22.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.127+0000 7fe11b400640 1 -- start start 2024-05-22T21:24:22.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.127+0000 7fe11b400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe114070db0 0x7fe1140b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.127+0000 7fe11b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe1140b2ea0 0x7fe1140cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.127+0000 7fe11b400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe114159bd0 con 0x7fe1141597f0 2024-05-22T21:24:22.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.127+0000 7fe11b400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1140cbdd0 con 0x7fe1140b2ea0 2024-05-22T21:24:22.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.127+0000 7fe11b400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1140cc050 con 0x7fe114070db0 2024-05-22T21:24:22.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.127+0000 7fe119400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe1140b2ea0 0x7fe1140cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.127+0000 7fe119400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe1140b2ea0 0x7fe1140cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57254/0 (socket says 172.21.15.161:57254) 2024-05-22T21:24:22.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.127+0000 7fe119400640 1 -- 172.21.15.161:0/1099232531 learned_addr learned my addr 172.21.15.161:0/1099232531 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:22.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.127+0000 7fe113e00640 1 --2- 172.21.15.161:0/1099232531 >> v2:172.21.15.179:3300/0 conn(0x7fe114070db0 0x7fe1140b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.127+0000 7fe119400640 1 -- 172.21.15.161:0/1099232531 >> v2:172.21.15.179:3300/0 conn(0x7fe114070db0 msgr2=0x7fe1140b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:22.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.127+0000 7fe119400640 1 --2- 172.21.15.161:0/1099232531 >> v2:172.21.15.179:3300/0 conn(0x7fe114070db0 0x7fe1140b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.127+0000 7fe119400640 1 -- 172.21.15.161:0/1099232531 >> v1:172.21.15.179:6789/0 conn(0x7fe1141597f0 legacy=0x7fe1140706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:22.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.127+0000 7fe119400640 1 -- 172.21.15.161:0/1099232531 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1140b3720 con 0x7fe1140b2ea0 2024-05-22T21:24:22.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.131+0000 7fe119400640 1 --2- 172.21.15.161:0/1099232531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1140b2ea0 0x7fe1140cc550 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fe114073ce0 tx=0x7fe10c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6a2a53be59a98b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:22.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.131+0000 7fe113400640 1 -- 172.21.15.161:0/1099232531 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c0a5830 con 0x7fe1140b2ea0 2024-05-22T21:24:22.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.131+0000 7fe113400640 1 -- 172.21.15.161:0/1099232531 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c0a5990 con 0x7fe1140b2ea0 2024-05-22T21:24:22.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.131+0000 7fe113400640 1 -- 172.21.15.161:0/1099232531 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c0a5d20 con 0x7fe1140b2ea0 2024-05-22T21:24:22.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.131+0000 7fe11b400640 1 -- 172.21.15.161:0/1099232531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1140b2ea0 msgr2=0x7fe1140cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:22.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.131+0000 7fe11b400640 1 --2- 172.21.15.161:0/1099232531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1140b2ea0 0x7fe1140cc550 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fe114073ce0 tx=0x7fe10c01c9f0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.131+0000 7fe11b400640 1 -- 172.21.15.161:0/1099232531 shutdown_connections 2024-05-22T21:24:22.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.131+0000 7fe11b400640 1 --2- 172.21.15.161:0/1099232531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1140b2ea0 0x7fe1140cc550 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.131+0000 7fe11b400640 1 --2- 172.21.15.161:0/1099232531 >> v2:172.21.15.179:3300/0 conn(0x7fe114070db0 0x7fe1140b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.131+0000 7fe11b400640 1 -- 172.21.15.161:0/1099232531 >> 172.21.15.161:0/1099232531 conn(0x7fe1140b0bc0 msgr2=0x7fe1140b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:22.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.131+0000 7fe11b400640 1 -- 172.21.15.161:0/1099232531 shutdown_connections 2024-05-22T21:24:22.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.131+0000 7fe11b400640 1 -- 172.21.15.161:0/1099232531 wait complete. 2024-05-22T21:24:22.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.131+0000 7fe11b400640 1 Processor -- start 2024-05-22T21:24:22.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.131+0000 7fe11b400640 1 -- start start 2024-05-22T21:24:22.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe11b400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe114070db0 0x7fe11415bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe11b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1140b2ea0 0x7fe114156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe11b400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe1141597f0 0x7fe1141f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe11b400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11415c000 con 0x7fe1140b2ea0 2024-05-22T21:24:22.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe11b400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1141566a0 con 0x7fe114070db0 2024-05-22T21:24:22.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe11b400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1141f6040 con 0x7fe1141597f0 2024-05-22T21:24:22.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe113e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1140b2ea0 0x7fe114156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe113e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1140b2ea0 0x7fe114156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57270/0 (socket says 172.21.15.161:57270) 2024-05-22T21:24:22.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe113e00640 1 -- 172.21.15.161:0/3048217788 learned_addr learned my addr 172.21.15.161:0/3048217788 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:22.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe118a00640 1 --2- 172.21.15.161:0/3048217788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe114070db0 0x7fe11415bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe119400640 1 --2- 172.21.15.161:0/3048217788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe1141597f0 0x7fe1141f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe113e00640 1 -- 172.21.15.161:0/3048217788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe1141597f0 msgr2=0x7fe1141f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:22.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe113e00640 1 --2- 172.21.15.161:0/3048217788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe1141597f0 0x7fe1141f5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe113e00640 1 -- 172.21.15.161:0/3048217788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe114070db0 msgr2=0x7fe11415bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:22.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe113e00640 1 --2- 172.21.15.161:0/3048217788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe114070db0 0x7fe11415bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe113e00640 1 -- 172.21.15.161:0/3048217788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c009b30 con 0x7fe1140b2ea0 2024-05-22T21:24:22.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe119400640 1 --2- 172.21.15.161:0/3048217788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe1141597f0 0x7fe1141f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:22.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe118a00640 1 --2- 172.21.15.161:0/3048217788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe114070db0 0x7fe11415bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:22.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe113e00640 1 --2- 172.21.15.161:0/3048217788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1140b2ea0 0x7fe114156160 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fe10000e930 tx=0x7fe10000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:22.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe110c00640 1 -- 172.21.15.161:0/3048217788 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100009690 con 0x7fe1140b2ea0 2024-05-22T21:24:22.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe110c00640 1 -- 172.21.15.161:0/3048217788 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10000ca00 con 0x7fe1140b2ea0 2024-05-22T21:24:22.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe11b400640 1 -- 172.21.15.161:0/3048217788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1141f62b0 con 0x7fe1140b2ea0 2024-05-22T21:24:22.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe110c00640 1 -- 172.21.15.161:0/3048217788 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10000cd70 con 0x7fe1140b2ea0 2024-05-22T21:24:22.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe11b400640 1 -- 172.21.15.161:0/3048217788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1141f6870 con 0x7fe1140b2ea0 2024-05-22T21:24:22.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.135+0000 7fe11b400640 1 -- 172.21.15.161:0/3048217788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1141f6a90 con 0x7fe1140b2ea0 2024-05-22T21:24:22.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.139+0000 7fe110c00640 1 -- 172.21.15.161:0/3048217788 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe100013470 con 0x7fe1140b2ea0 2024-05-22T21:24:22.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.139+0000 7fe110c00640 1 --2- 172.21.15.161:0/3048217788 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe0f4066690 0x7fe0f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.139+0000 7fe110c00640 1 -- 172.21.15.161:0/3048217788 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe1000844a0 con 0x7fe1140b2ea0 2024-05-22T21:24:22.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.139+0000 7fe110c00640 1 -- 172.21.15.161:0/3048217788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7fe0f406e220 con 0x7fe1140b2ea0 2024-05-22T21:24:22.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.139+0000 7fe118a00640 1 --2- 172.21.15.161:0/3048217788 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe0f4066690 0x7fe0f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.139 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.139+0000 7fe118a00640 1 --2- 172.21.15.161:0/3048217788 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe0f4066690 0x7fe0f4068b50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe108000f70 tx=0x7fe108009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:22.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.143+0000 7fe110c00640 1 -- 172.21.15.161:0/3048217788 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe100051810 con 0x7fe1140b2ea0 2024-05-22T21:24:22.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.295+0000 7fe11b400640 1 -- 172.21.15.161:0/3048217788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe114067220 con 0x7fe1140b2ea0 2024-05-22T21:24:22.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.299+0000 7fe110c00640 1 -- 172.21.15.161:0/3048217788 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+16235 (secure 0 0 0) 0x7fe100021020 con 0x7fe1140b2ea0 2024-05-22T21:24:22.298 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:22.298 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":178,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:23:50.299404+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:23:02.242525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-05-22T21:23:02.864916+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:24:22.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.299+0000 7fe11b400640 1 -- 172.21.15.161:0/3048217788 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe0f4066690 msgr2=0x7fe0f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:22.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.299+0000 7fe11b400640 1 --2- 172.21.15.161:0/3048217788 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe0f4066690 0x7fe0f4068b50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe108000f70 tx=0x7fe108009040 comp rx=0 tx=0).stop 2024-05-22T21:24:22.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.299+0000 7fe11b400640 1 -- 172.21.15.161:0/3048217788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1140b2ea0 msgr2=0x7fe114156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:22.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.299+0000 7fe11b400640 1 --2- 172.21.15.161:0/3048217788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1140b2ea0 0x7fe114156160 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fe10000e930 tx=0x7fe10000edf0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.299+0000 7fe11b400640 1 -- 172.21.15.161:0/3048217788 shutdown_connections 2024-05-22T21:24:22.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.299+0000 7fe11b400640 1 --2- 172.21.15.161:0/3048217788 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe0f4066690 0x7fe0f4068b50 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.299+0000 7fe11b400640 1 --2- 172.21.15.161:0/3048217788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe1141597f0 0x7fe1141f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.299+0000 7fe11b400640 1 --2- 172.21.15.161:0/3048217788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1140b2ea0 0x7fe114156160 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.299+0000 7fe11b400640 1 --2- 172.21.15.161:0/3048217788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe114070db0 0x7fe11415bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.299+0000 7fe11b400640 1 -- 172.21.15.161:0/3048217788 >> 172.21.15.161:0/3048217788 conn(0x7fe1140b0bc0 msgr2=0x7fe1141502f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:22.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.299+0000 7fe11b400640 1 -- 172.21.15.161:0/3048217788 shutdown_connections 2024-05-22T21:24:22.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.299+0000 7fe11b400640 1 -- 172.21.15.161:0/3048217788 wait complete. 2024-05-22T21:24:22.319 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-22T21:24:22.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.455+0000 7f777be00640 1 Processor -- start 2024-05-22T21:24:22.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.455+0000 7f777be00640 1 -- start start 2024-05-22T21:24:22.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.455+0000 7f777be00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f777c15d180 0x7f777c06f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.455+0000 7f777be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f777c0af070 0x7f777c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.455+0000 7f777be00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777c06a820 con 0x7f777c06a410 2024-05-22T21:24:22.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.455+0000 7f777be00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777c15d560 con 0x7f777c15d180 2024-05-22T21:24:22.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.455+0000 7f777be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777c0adb00 con 0x7f777c0af070 2024-05-22T21:24:22.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.455+0000 7f777aa00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f777c15d180 0x7f777c06f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.455+0000 7f777b400640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f777c06a410 0x7f777c070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:43058/0 (socket says 172.21.15.161:43058) 2024-05-22T21:24:22.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.455+0000 7f777b400640 1 -- 172.21.15.161:0/3985560107 learned_addr learned my addr 172.21.15.161:0/3985560107 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:22.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.455+0000 7f7772000640 1 --2- 172.21.15.161:0/3985560107 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f777c0af070 0x7f777c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.455+0000 7f777aa00640 1 -- 172.21.15.161:0/3985560107 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f777c0af070 msgr2=0x7f777c069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:22.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.455+0000 7f777aa00640 1 --2- 172.21.15.161:0/3985560107 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f777c0af070 0x7f777c069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.455+0000 7f777aa00640 1 -- 172.21.15.161:0/3985560107 >> v1:172.21.15.179:6789/0 conn(0x7f777c06a410 legacy=0x7f777c070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:22.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.455+0000 7f777aa00640 1 -- 172.21.15.161:0/3985560107 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c15d800 con 0x7f777c15d180 2024-05-22T21:24:22.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.455+0000 7f777a000640 1 -- 172.21.15.161:0/3985560107 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 437145225 0 0) 0x7f777c06a820 con 0x7f777c06a410 2024-05-22T21:24:22.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.459+0000 7f777aa00640 1 --2- 172.21.15.161:0/3985560107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f777c15d180 0x7f777c06f2b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f776c0097c0 tx=0x7f776c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6fdcd03e549abc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:22.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.459+0000 7f777a000640 1 -- 172.21.15.161:0/3985560107 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c0ab600 con 0x7f777c15d180 2024-05-22T21:24:22.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.459+0000 7f777a000640 1 -- 172.21.15.161:0/3985560107 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776c0a7050 con 0x7f777c15d180 2024-05-22T21:24:22.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.459+0000 7f777a000640 1 -- 172.21.15.161:0/3985560107 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c0ab980 con 0x7f777c15d180 2024-05-22T21:24:22.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.459+0000 7f777be00640 1 -- 172.21.15.161:0/3985560107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f777c15d180 msgr2=0x7f777c06f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:22.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.459+0000 7f777be00640 1 --2- 172.21.15.161:0/3985560107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f777c15d180 0x7f777c06f2b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f776c0097c0 tx=0x7f776c01c4a0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.459+0000 7f777be00640 1 -- 172.21.15.161:0/3985560107 shutdown_connections 2024-05-22T21:24:22.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.459+0000 7f777be00640 1 --2- 172.21.15.161:0/3985560107 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f777c0af070 0x7f777c069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.459+0000 7f777be00640 1 --2- 172.21.15.161:0/3985560107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f777c15d180 0x7f777c06f2b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.459+0000 7f777be00640 1 -- 172.21.15.161:0/3985560107 >> 172.21.15.161:0/3985560107 conn(0x7f777c06b9d0 msgr2=0x7f777c0b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:22.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.459+0000 7f777be00640 1 -- 172.21.15.161:0/3985560107 shutdown_connections 2024-05-22T21:24:22.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.459+0000 7f777be00640 1 -- 172.21.15.161:0/3985560107 wait complete. 2024-05-22T21:24:22.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f777be00640 1 Processor -- start 2024-05-22T21:24:22.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f777be00640 1 -- start start 2024-05-22T21:24:22.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f777be00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f777c06a410 0x7f777c156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f777be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777c0af070 0x7f777c157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f777be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f777c15d180 0x7f777c1578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f777be00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c151750 con 0x7f777c0af070 2024-05-22T21:24:22.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f777be00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c150ad0 con 0x7f777c15d180 2024-05-22T21:24:22.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f777be00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777c150db0 con 0x7f777c06a410 2024-05-22T21:24:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f7772000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777c0af070 0x7f777c157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f7772000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777c0af070 0x7f777c157390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57278/0 (socket says 172.21.15.161:57278) 2024-05-22T21:24:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f7772000640 1 -- 172.21.15.161:0/2465822011 learned_addr learned my addr 172.21.15.161:0/2465822011 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f777aa00640 1 --2- 172.21.15.161:0/2465822011 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f777c06a410 0x7f777c156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f777b400640 1 --2- 172.21.15.161:0/2465822011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f777c15d180 0x7f777c1578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f7772000640 1 -- 172.21.15.161:0/2465822011 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f777c06a410 msgr2=0x7f777c156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f7772000640 1 --2- 172.21.15.161:0/2465822011 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f777c06a410 0x7f777c156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f7772000640 1 -- 172.21.15.161:0/2465822011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f777c15d180 msgr2=0x7f777c1578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f7772000640 1 --2- 172.21.15.161:0/2465822011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f777c15d180 0x7f777c1578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f7772000640 1 -- 172.21.15.161:0/2465822011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c009420 con 0x7f777c0af070 2024-05-22T21:24:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f7772000640 1 --2- 172.21.15.161:0/2465822011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777c0af070 0x7f777c157390 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f776400e930 tx=0x7f776400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:22.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f7773400640 1 -- 172.21.15.161:0/2465822011 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764009690 con 0x7f777c0af070 2024-05-22T21:24:22.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f7773400640 1 -- 172.21.15.161:0/2465822011 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776400ca00 con 0x7f777c0af070 2024-05-22T21:24:22.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f7773400640 1 -- 172.21.15.161:0/2465822011 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776400cd70 con 0x7f777c0af070 2024-05-22T21:24:22.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f777be00640 1 -- 172.21.15.161:0/2465822011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777c151090 con 0x7f777c0af070 2024-05-22T21:24:22.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.463+0000 7f777be00640 1 -- 172.21.15.161:0/2465822011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777c158020 con 0x7f777c0af070 2024-05-22T21:24:22.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.467+0000 7f777be00640 1 -- 172.21.15.161:0/2465822011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777c067070 con 0x7f777c0af070 2024-05-22T21:24:22.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.467+0000 7f7773400640 1 -- 172.21.15.161:0/2465822011 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7764013470 con 0x7f777c0af070 2024-05-22T21:24:22.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.467+0000 7f7773400640 1 --2- 172.21.15.161:0/2465822011 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7758066690 0x7f7758068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.467+0000 7f7773400640 1 -- 172.21.15.161:0/2465822011 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f77640843e0 con 0x7f777c0af070 2024-05-22T21:24:22.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.467+0000 7f7773400640 1 -- 172.21.15.161:0/2465822011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=179}) v3 -- 0x7f775806e220 con 0x7f777c0af070 2024-05-22T21:24:22.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.467+0000 7f777aa00640 1 --2- 172.21.15.161:0/2465822011 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7758066690 0x7f7758068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.467+0000 7f777aa00640 1 --2- 172.21.15.161:0/2465822011 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7758066690 0x7f7758068b50 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f776c0ad000 tx=0x7f776c0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:22.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.471+0000 7f7773400640 1 -- 172.21.15.161:0/2465822011 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7764051890 con 0x7f777c0af070 2024-05-22T21:24:22.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.659+0000 7f777be00640 1 -- 172.21.15.161:0/2465822011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f777c070e00 con 0x7f777c0af070 2024-05-22T21:24:22.661 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:24:22.659+0000 7f7d32800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-22T21:24:22.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.663+0000 7f7773400640 1 -- 172.21.15.161:0/2465822011 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(179..179 src has 1..179) v4 ==== 374+0+0 (secure 0 0 0) 0x7f77640592d0 con 0x7f777c0af070 2024-05-22T21:24:22.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.663+0000 7f7773400640 1 -- 172.21.15.161:0/2465822011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=180}) v3 -- 0x7f775806ead0 con 0x7f777c0af070 2024-05-22T21:24:22.670 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.671+0000 7f7773400640 1 -- 172.21.15.161:0/2465822011 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v79) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7764021020 con 0x7f777c0af070 2024-05-22T21:24:22.670 INFO:teuthology.orchestra.run.smithi161.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-22T21:24:22.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.671+0000 7f777be00640 1 -- 172.21.15.161:0/2465822011 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7758066690 msgr2=0x7f7758068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:22.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.671+0000 7f777be00640 1 --2- 172.21.15.161:0/2465822011 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7758066690 0x7f7758068b50 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f776c0ad000 tx=0x7f776c0a6000 comp rx=0 tx=0).stop 2024-05-22T21:24:22.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.671+0000 7f777be00640 1 -- 172.21.15.161:0/2465822011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777c0af070 msgr2=0x7f777c157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:22.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.671+0000 7f777be00640 1 --2- 172.21.15.161:0/2465822011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777c0af070 0x7f777c157390 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f776400e930 tx=0x7f776400edf0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.671+0000 7f777be00640 1 -- 172.21.15.161:0/2465822011 shutdown_connections 2024-05-22T21:24:22.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.671+0000 7f777be00640 1 --2- 172.21.15.161:0/2465822011 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7758066690 0x7f7758068b50 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.671+0000 7f777be00640 1 --2- 172.21.15.161:0/2465822011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f777c15d180 0x7f777c1578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.671+0000 7f777be00640 1 --2- 172.21.15.161:0/2465822011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777c0af070 0x7f777c157390 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.671+0000 7f777be00640 1 --2- 172.21.15.161:0/2465822011 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f777c06a410 0x7f777c156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.671+0000 7f777be00640 1 -- 172.21.15.161:0/2465822011 >> 172.21.15.161:0/2465822011 conn(0x7f777c06b9d0 msgr2=0x7f777c152470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:22.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.671+0000 7f777be00640 1 -- 172.21.15.161:0/2465822011 shutdown_connections 2024-05-22T21:24:22.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.671+0000 7f777be00640 1 -- 172.21.15.161:0/2465822011 wait complete. 2024-05-22T21:24:22.692 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-22T21:24:22.733 INFO:tasks.ceph.mds.a.smithi161.stdout:starting mds.a at 2024-05-22T21:24:22.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.831+0000 7fe43e200640 1 Processor -- start 2024-05-22T21:24:22.833 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.831+0000 7fe43e200640 1 -- start start 2024-05-22T21:24:22.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.835+0000 7fe43e200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe43813cfc0 0x7fe4381402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.835+0000 7fe43e200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe43815af60 0x7fe43813f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.835+0000 7fe43e200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4381400d0 con 0x7fe43813fc80 2024-05-22T21:24:22.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.835+0000 7fe43e200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe43813d3a0 con 0x7fe43813cfc0 2024-05-22T21:24:22.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.835+0000 7fe43e200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4380cd920 con 0x7fe43815af60 2024-05-22T21:24:22.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.835+0000 7fe437400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe43813cfc0 0x7fe4381402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.835+0000 7fe437e00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe43813fc80 0x7fe43815dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:43068/0 (socket says 172.21.15.161:43068) 2024-05-22T21:24:22.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.835+0000 7fe437e00640 1 -- 172.21.15.161:0/3131403211 learned_addr learned my addr 172.21.15.161:0/3131403211 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:22.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.835+0000 7fe42fe00640 1 --2- 172.21.15.161:0/3131403211 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe43815af60 0x7fe43813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.835+0000 7fe437400640 1 -- 172.21.15.161:0/3131403211 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe43815af60 msgr2=0x7fe43813f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:22.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.835+0000 7fe437400640 1 --2- 172.21.15.161:0/3131403211 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe43815af60 0x7fe43813f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.835+0000 7fe437400640 1 -- 172.21.15.161:0/3131403211 >> v1:172.21.15.179:6789/0 conn(0x7fe43813fc80 legacy=0x7fe43815dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:22.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.835+0000 7fe437400640 1 -- 172.21.15.161:0/3131403211 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4380cdc00 con 0x7fe43813cfc0 2024-05-22T21:24:22.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.835+0000 7fe436a00640 1 -- 172.21.15.161:0/3131403211 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419545273 0 0) 0x7fe4300014a0 con 0x7fe43813fc80 2024-05-22T21:24:22.838 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.839+0000 7fe437400640 1 --2- 172.21.15.161:0/3131403211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe43813cfc0 0x7fe4381402e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe4280097c0 tx=0x7fe42801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d49e44fc7335c02 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:22.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.839+0000 7fe436a00640 1 -- 172.21.15.161:0/3131403211 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4280ab600 con 0x7fe43813cfc0 2024-05-22T21:24:22.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.839+0000 7fe436a00640 1 -- 172.21.15.161:0/3131403211 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4280a7050 con 0x7fe43813cfc0 2024-05-22T21:24:22.839 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.839+0000 7fe436a00640 1 -- 172.21.15.161:0/3131403211 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4280ab980 con 0x7fe43813cfc0 2024-05-22T21:24:22.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.839+0000 7fe43e200640 1 -- 172.21.15.161:0/3131403211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe43813cfc0 msgr2=0x7fe4381402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:22.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.839+0000 7fe43e200640 1 --2- 172.21.15.161:0/3131403211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe43813cfc0 0x7fe4381402e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe4280097c0 tx=0x7fe42801c410 comp rx=0 tx=0).stop 2024-05-22T21:24:22.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.839+0000 7fe43e200640 1 -- 172.21.15.161:0/3131403211 shutdown_connections 2024-05-22T21:24:22.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.839+0000 7fe43e200640 1 --2- 172.21.15.161:0/3131403211 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe43815af60 0x7fe43813f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.839+0000 7fe43e200640 1 --2- 172.21.15.161:0/3131403211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe43813cfc0 0x7fe4381402e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.839+0000 7fe43e200640 1 -- 172.21.15.161:0/3131403211 >> 172.21.15.161:0/3131403211 conn(0x7fe43806a2d0 msgr2=0x7fe43806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:22.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.839+0000 7fe43e200640 1 -- 172.21.15.161:0/3131403211 shutdown_connections 2024-05-22T21:24:22.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.839+0000 7fe43e200640 1 -- 172.21.15.161:0/3131403211 wait complete. 2024-05-22T21:24:22.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.839+0000 7fe43e200640 1 Processor -- start 2024-05-22T21:24:22.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.839+0000 7fe43e200640 1 -- start start 2024-05-22T21:24:22.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe43e200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe43813cfc0 0x7fe4380b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe43e200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe43813fc80 0x7fe4381594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe43e200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe43815af60 0x7fe438156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe43e200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4380b3d60 con 0x7fe43815af60 2024-05-22T21:24:22.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe43e200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4380b0930 con 0x7fe43813cfc0 2024-05-22T21:24:22.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe43e200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4380b3150 con 0x7fe43813fc80 2024-05-22T21:24:22.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe437e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe43815af60 0x7fe438156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe437e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe43815af60 0x7fe438156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57318/0 (socket says 172.21.15.161:57318) 2024-05-22T21:24:22.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe437e00640 1 -- 172.21.15.161:0/3901971520 learned_addr learned my addr 172.21.15.161:0/3901971520 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:22.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe437400640 1 --2- 172.21.15.161:0/3901971520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe43813cfc0 0x7fe4380b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe42fe00640 1 --2- 172.21.15.161:0/3901971520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe43813fc80 0x7fe4381594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe437e00640 1 -- 172.21.15.161:0/3901971520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe43813fc80 msgr2=0x7fe4381594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:22.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe437e00640 1 --2- 172.21.15.161:0/3901971520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe43813fc80 0x7fe4381594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe437e00640 1 -- 172.21.15.161:0/3901971520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe43813cfc0 msgr2=0x7fe4380b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:22.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe437e00640 1 --2- 172.21.15.161:0/3901971520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe43813cfc0 0x7fe4380b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:22.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe437e00640 1 -- 172.21.15.161:0/3901971520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe428009420 con 0x7fe43815af60 2024-05-22T21:24:22.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe437e00640 1 --2- 172.21.15.161:0/3901971520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe43815af60 0x7fe438156380 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fe43000c7e0 tx=0x7fe43000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:22.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe42f400640 1 -- 172.21.15.161:0/3901971520 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43000a640 con 0x7fe43815af60 2024-05-22T21:24:22.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe42f400640 1 -- 172.21.15.161:0/3901971520 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43000a7c0 con 0x7fe43815af60 2024-05-22T21:24:22.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe42f400640 1 -- 172.21.15.161:0/3901971520 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43000abb0 con 0x7fe43815af60 2024-05-22T21:24:22.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe43e200640 1 -- 172.21.15.161:0/3901971520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4380b3430 con 0x7fe43815af60 2024-05-22T21:24:22.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe43e200640 1 -- 172.21.15.161:0/3901971520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4380b39f0 con 0x7fe43815af60 2024-05-22T21:24:22.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe43e200640 1 -- 172.21.15.161:0/3901971520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4380b3f10 con 0x7fe43815af60 2024-05-22T21:24:22.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.843+0000 7fe42f400640 1 -- 172.21.15.161:0/3901971520 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe43000ad10 con 0x7fe43815af60 2024-05-22T21:24:22.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.847+0000 7fe42f400640 1 --2- 172.21.15.161:0/3901971520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe4140666c0 0x7fe414068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:22.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.847+0000 7fe42f400640 1 -- 172.21.15.161:0/3901971520 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe430083490 con 0x7fe43815af60 2024-05-22T21:24:22.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.847+0000 7fe437400640 1 --2- 172.21.15.161:0/3901971520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe4140666c0 0x7fe414068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:22.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.847+0000 7fe42f400640 1 -- 172.21.15.161:0/3901971520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=180}) v3 -- 0x7fe41406e360 con 0x7fe43815af60 2024-05-22T21:24:22.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.847+0000 7fe437400640 1 --2- 172.21.15.161:0/3901971520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe4140666c0 0x7fe414068b80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe4280ad000 tx=0x7fe4280a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:22.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:22.851+0000 7fe42f400640 1 -- 172.21.15.161:0/3901971520 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe430018060 con 0x7fe43815af60 2024-05-22T21:24:23.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.031+0000 7fe43e200640 1 -- 172.21.15.161:0/3901971520 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe4381400d0 con 0x7fe43815af60 2024-05-22T21:24:23.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.671+0000 7fe42f400640 1 -- 172.21.15.161:0/3901971520 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v80) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe4381400d0 con 0x7fe43815af60 2024-05-22T21:24:23.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.675+0000 7fe43e200640 1 -- 172.21.15.161:0/3901971520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe4140666c0 msgr2=0x7fe414068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:23.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.675+0000 7fe43e200640 1 --2- 172.21.15.161:0/3901971520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe4140666c0 0x7fe414068b80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe4280ad000 tx=0x7fe4280a6000 comp rx=0 tx=0).stop 2024-05-22T21:24:23.674 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.675+0000 7fe43e200640 1 -- 172.21.15.161:0/3901971520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe43815af60 msgr2=0x7fe438156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:23.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.675+0000 7fe43e200640 1 --2- 172.21.15.161:0/3901971520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe43815af60 0x7fe438156380 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fe43000c7e0 tx=0x7fe43000cca0 comp rx=0 tx=0).stop 2024-05-22T21:24:23.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.675+0000 7fe43e200640 1 -- 172.21.15.161:0/3901971520 shutdown_connections 2024-05-22T21:24:23.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.675+0000 7fe43e200640 1 --2- 172.21.15.161:0/3901971520 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe4140666c0 0x7fe414068b80 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:23.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.675+0000 7fe43e200640 1 --2- 172.21.15.161:0/3901971520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe43815af60 0x7fe438156380 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:23.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.675+0000 7fe43e200640 1 --2- 172.21.15.161:0/3901971520 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe43813fc80 0x7fe4381594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:23.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.675+0000 7fe43e200640 1 --2- 172.21.15.161:0/3901971520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe43813cfc0 0x7fe4380b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:23.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.675+0000 7fe43e200640 1 -- 172.21.15.161:0/3901971520 >> 172.21.15.161:0/3901971520 conn(0x7fe43806a2d0 msgr2=0x7fe438142480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:23.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.675+0000 7fe43e200640 1 -- 172.21.15.161:0/3901971520 shutdown_connections 2024-05-22T21:24:23.675 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.675+0000 7fe43e200640 1 -- 172.21.15.161:0/3901971520 wait complete. 2024-05-22T21:24:23.695 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-22T21:24:23.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.883+0000 7f0bf3800640 1 Processor -- start 2024-05-22T21:24:23.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.883+0000 7f0bf3800640 1 -- start start 2024-05-22T21:24:23.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.883+0000 7f0bf3800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bec160c90 0x7f0bec0afef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:23.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.883+0000 7f0bf3800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bec06b9f0 con 0x7f0bec09a560 2024-05-22T21:24:23.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.883+0000 7f0bf3800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bec06bcd0 con 0x7f0bec0cd820 2024-05-22T21:24:23.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.883+0000 7f0bf3800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec06bfb0 con 0x7f0bec160c90 2024-05-22T21:24:23.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.883+0000 7f0bebe00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0bec09a560 0x7f0bec0cb5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56948/0 (socket says 172.21.15.161:56948) 2024-05-22T21:24:23.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.883+0000 7f0bebe00640 1 -- 172.21.15.161:0/226186244 learned_addr learned my addr 172.21.15.161:0/226186244 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:23.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.883+0000 7f0bf0e00640 1 --2- 172.21.15.161:0/226186244 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bec160c90 0x7f0bec0afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:23.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.883+0000 7f0beb400640 1 -- 172.21.15.161:0/226186244 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 691341115 0 0) 0x7f0bec06b9f0 con 0x7f0bec09a560 2024-05-22T21:24:23.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.883+0000 7f0beb400640 1 -- 172.21.15.161:0/226186244 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bd4003530 con 0x7f0bec09a560 2024-05-22T21:24:23.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.883+0000 7f0beb400640 1 -- 172.21.15.161:0/226186244 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4225747996 0 0) 0x7f0bec06bcd0 con 0x7f0bec0cd820 2024-05-22T21:24:23.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.883+0000 7f0bf0e00640 1 -- 172.21.15.161:0/226186244 >> v1:172.21.15.179:6789/0 conn(0x7f0bec0cd820 legacy=0x7f0bec0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:23.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.883+0000 7f0bf0e00640 1 -- 172.21.15.161:0/226186244 >> v1:172.21.15.161:6789/0 conn(0x7f0bec09a560 legacy=0x7f0bec0cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:23.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.883+0000 7f0bf0e00640 1 -- 172.21.15.161:0/226186244 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec0cf340 con 0x7f0bec160c90 2024-05-22T21:24:23.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.887+0000 7f0bf0e00640 1 --2- 172.21.15.161:0/226186244 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bec160c90 0x7f0bec0afef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0be00097f0 tx=0x7f0be001c0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4521ff616183c28c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:23.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.887+0000 7f0beb400640 1 -- 172.21.15.161:0/226186244 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be00a47d0 con 0x7f0bec160c90 2024-05-22T21:24:23.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.887+0000 7f0beb400640 1 -- 172.21.15.161:0/226186244 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be00a4930 con 0x7f0bec160c90 2024-05-22T21:24:23.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.887+0000 7f0beb400640 1 -- 172.21.15.161:0/226186244 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be00a4cc0 con 0x7f0bec160c90 2024-05-22T21:24:23.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.887+0000 7f0bf3800640 1 -- 172.21.15.161:0/226186244 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bec160c90 msgr2=0x7f0bec0afef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:23.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.887+0000 7f0bf3800640 1 --2- 172.21.15.161:0/226186244 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bec160c90 0x7f0bec0afef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0be00097f0 tx=0x7f0be001c0a0 comp rx=0 tx=0).stop 2024-05-22T21:24:23.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.887+0000 7f0bf3800640 1 -- 172.21.15.161:0/226186244 shutdown_connections 2024-05-22T21:24:23.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.887+0000 7f0bf3800640 1 --2- 172.21.15.161:0/226186244 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bec160c90 0x7f0bec0afef0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:23.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.887+0000 7f0bf3800640 1 -- 172.21.15.161:0/226186244 >> 172.21.15.161:0/226186244 conn(0x7f0bec099a40 msgr2=0x7f0bec099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:23.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.887+0000 7f0bf3800640 1 -- 172.21.15.161:0/226186244 shutdown_connections 2024-05-22T21:24:23.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.887+0000 7f0bf3800640 1 -- 172.21.15.161:0/226186244 wait complete. 2024-05-22T21:24:23.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf3800640 1 Processor -- start 2024-05-22T21:24:23.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf3800640 1 -- start start 2024-05-22T21:24:23.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf3800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bec09a560 0x7f0bec14cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:23.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf3800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bec0cd820 0x7f0bec0c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:23.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf3800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bec160c90 0x7f0bec0aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:23.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf3800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0b6ec0 con 0x7f0bec160c90 2024-05-22T21:24:23.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf3800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0b4690 con 0x7f0bec0cd820 2024-05-22T21:24:23.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf3800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0c9900 con 0x7f0bec09a560 2024-05-22T21:24:23.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf1800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bec160c90 0x7f0bec0aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:23.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf1800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bec160c90 0x7f0bec0aed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57326/0 (socket says 172.21.15.161:57326) 2024-05-22T21:24:23.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf1800640 1 -- 172.21.15.161:0/563701059 learned_addr learned my addr 172.21.15.161:0/563701059 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:23.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf0e00640 1 --2- 172.21.15.161:0/563701059 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bec09a560 0x7f0bec14cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:23.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bebe00640 1 --2- 172.21.15.161:0/563701059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bec0cd820 0x7f0bec0c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:23.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf1800640 1 -- 172.21.15.161:0/563701059 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bec09a560 msgr2=0x7f0bec14cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:23.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf1800640 1 --2- 172.21.15.161:0/563701059 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bec09a560 0x7f0bec14cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:23.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf1800640 1 -- 172.21.15.161:0/563701059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bec0cd820 msgr2=0x7f0bec0c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:23.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf1800640 1 --2- 172.21.15.161:0/563701059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bec0cd820 0x7f0bec0c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:23.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf1800640 1 -- 172.21.15.161:0/563701059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be0009420 con 0x7f0bec160c90 2024-05-22T21:24:23.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf1800640 1 --2- 172.21.15.161:0/563701059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bec160c90 0x7f0bec0aed00 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f0be4002240 tx=0x7f0be400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:23.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0be8c00640 1 -- 172.21.15.161:0/563701059 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be4003db0 con 0x7f0bec160c90 2024-05-22T21:24:23.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0be8c00640 1 -- 172.21.15.161:0/563701059 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be400ce30 con 0x7f0bec160c90 2024-05-22T21:24:23.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0be8c00640 1 -- 172.21.15.161:0/563701059 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be4005270 con 0x7f0bec160c90 2024-05-22T21:24:23.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf3800640 1 -- 172.21.15.161:0/563701059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bec0c9be0 con 0x7f0bec160c90 2024-05-22T21:24:23.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.891+0000 7f0bf3800640 1 -- 172.21.15.161:0/563701059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bec0ca1a0 con 0x7f0bec160c90 2024-05-22T21:24:23.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.895+0000 7f0bf3800640 1 -- 172.21.15.161:0/563701059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bec0b4850 con 0x7f0bec160c90 2024-05-22T21:24:23.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.895+0000 7f0be8c00640 1 -- 172.21.15.161:0/563701059 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0be40053d0 con 0x7f0bec160c90 2024-05-22T21:24:23.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.895+0000 7f0be8c00640 1 --2- 172.21.15.161:0/563701059 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0bcc066690 0x7f0bcc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:23.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.895+0000 7f0be8c00640 1 -- 172.21.15.161:0/563701059 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f0be4083900 con 0x7f0bec160c90 2024-05-22T21:24:23.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.895+0000 7f0bf0e00640 1 --2- 172.21.15.161:0/563701059 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0bcc066690 0x7f0bcc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:23.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.895+0000 7f0be8c00640 1 -- 172.21.15.161:0/563701059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=180}) v3 -- 0x7f0bcc06e330 con 0x7f0bec160c90 2024-05-22T21:24:23.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.895+0000 7f0bf0e00640 1 --2- 172.21.15.161:0/563701059 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0bcc066690 0x7f0bcc068b50 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0be00ae000 tx=0x7f0be00a6070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:23.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:23.899+0000 7f0be8c00640 1 -- 172.21.15.161:0/563701059 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0be4017070 con 0x7f0bec160c90 2024-05-22T21:24:24.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.055+0000 7f0bf3800640 1 -- 172.21.15.161:0/563701059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0bec067220 con 0x7f0bec160c90 2024-05-22T21:24:24.676 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.675+0000 7f0be8c00640 1 -- 172.21.15.161:0/563701059 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(180..180 src has 1..180) v4 ==== 296+0+0 (secure 0 0 0) 0x7f0be4054980 con 0x7f0bec160c90 2024-05-22T21:24:24.677 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.675+0000 7f0be8c00640 1 -- 172.21.15.161:0/563701059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=181}) v3 -- 0x7f0bcc06c760 con 0x7f0bec160c90 2024-05-22T21:24:24.679 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.679+0000 7f0be8c00640 1 -- 172.21.15.161:0/563701059 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v180) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0be4050cc0 con 0x7f0bec160c90 2024-05-22T21:24:24.679 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' removed 2024-05-22T21:24:24.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.683+0000 7f0bf3800640 1 -- 172.21.15.161:0/563701059 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0bcc066690 msgr2=0x7f0bcc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:24.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.683+0000 7f0bf3800640 1 --2- 172.21.15.161:0/563701059 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0bcc066690 0x7f0bcc068b50 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0be00ae000 tx=0x7f0be00a6070 comp rx=0 tx=0).stop 2024-05-22T21:24:24.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.683+0000 7f0bf3800640 1 -- 172.21.15.161:0/563701059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bec160c90 msgr2=0x7f0bec0aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:24.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.683+0000 7f0bf3800640 1 --2- 172.21.15.161:0/563701059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bec160c90 0x7f0bec0aed00 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f0be4002240 tx=0x7f0be400cbe0 comp rx=0 tx=0).stop 2024-05-22T21:24:24.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.683+0000 7f0bf3800640 1 -- 172.21.15.161:0/563701059 shutdown_connections 2024-05-22T21:24:24.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.683+0000 7f0bf3800640 1 --2- 172.21.15.161:0/563701059 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0bcc066690 0x7f0bcc068b50 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:24.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.683+0000 7f0bf3800640 1 --2- 172.21.15.161:0/563701059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bec160c90 0x7f0bec0aed00 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:24.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.683+0000 7f0bf3800640 1 --2- 172.21.15.161:0/563701059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bec0cd820 0x7f0bec0c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:24.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.683+0000 7f0bf3800640 1 --2- 172.21.15.161:0/563701059 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bec09a560 0x7f0bec14cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:24.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.683+0000 7f0bf3800640 1 -- 172.21.15.161:0/563701059 >> 172.21.15.161:0/563701059 conn(0x7f0bec099a40 msgr2=0x7f0bec06ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:24.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.683+0000 7f0bf3800640 1 -- 172.21.15.161:0/563701059 shutdown_connections 2024-05-22T21:24:24.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.683+0000 7f0bf3800640 1 -- 172.21.15.161:0/563701059 wait complete. 2024-05-22T21:24:24.704 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-22T21:24:24.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.843+0000 7fdfa6000640 1 Processor -- start 2024-05-22T21:24:24.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.843+0000 7fdfa6000640 1 -- start start 2024-05-22T21:24:24.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.843+0000 7fdfa6000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfa0071b40 0x7fdfa006ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:24.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.843+0000 7fdfa6000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa00cd770 con 0x7fdfa00ce660 2024-05-22T21:24:24.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.843+0000 7fdfa6000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa00cce80 con 0x7fdfa0159640 2024-05-22T21:24:24.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.843+0000 7fdfa6000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00ce310 con 0x7fdfa0071b40 2024-05-22T21:24:24.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9fe00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fdfa00ce660 0x7fdfa00b1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56958/0 (socket says 172.21.15.161:56958) 2024-05-22T21:24:24.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9fe00640 1 -- 172.21.15.161:0/4202358188 learned_addr learned my addr 172.21.15.161:0/4202358188 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9ea00640 1 --2- 172.21.15.161:0/4202358188 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfa0071b40 0x7fdfa006ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 -- 172.21.15.161:0/4202358188 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401825354 0 0) 0x7fdfa00cd770 con 0x7fdfa00ce660 2024-05-22T21:24:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 -- 172.21.15.161:0/4202358188 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf84003450 con 0x7fdfa00ce660 2024-05-22T21:24:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 -- 172.21.15.161:0/4202358188 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677099218 0 0) 0x7fdfa00cce80 con 0x7fdfa0159640 2024-05-22T21:24:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 -- 172.21.15.161:0/4202358188 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfa00cd770 con 0x7fdfa0159640 2024-05-22T21:24:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 -- 172.21.15.161:0/4202358188 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1263188518 0 0) 0x7fdf84003450 con 0x7fdfa00ce660 2024-05-22T21:24:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 -- 172.21.15.161:0/4202358188 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfa0071b40 msgr2=0x7fdfa006ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 --2- 172.21.15.161:0/4202358188 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfa0071b40 0x7fdfa006ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 -- 172.21.15.161:0/4202358188 >> v1:172.21.15.179:6789/0 conn(0x7fdfa0159640 legacy=0x7fdfa00cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 -- 172.21.15.161:0/4202358188 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa015d390 con 0x7fdfa00ce660 2024-05-22T21:24:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 -- 172.21.15.161:0/4202358188 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fdf98001680 con 0x7fdfa00ce660 2024-05-22T21:24:24.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 -- 172.21.15.161:0/4202358188 >> v1:172.21.15.161:6789/0 conn(0x7fdfa00ce660 legacy=0x7fdfa00b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 --2- 172.21.15.161:0/4202358188 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfa0071b40 0x7fdf84003c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 --2- 172.21.15.161:0/4202358188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf84008860 0x7fdf8400ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 --2- 172.21.15.161:0/4202358188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf8400b190 0x7fdf8400d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 -- 172.21.15.161:0/4202358188 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00ce310 con 0x7fdf84008860 2024-05-22T21:24:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 -- 172.21.15.161:0/4202358188 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8400dd00 con 0x7fdf8400b190 2024-05-22T21:24:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9e000640 1 -- 172.21.15.161:0/4202358188 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8400dec0 con 0x7fdfa0071b40 2024-05-22T21:24:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9ea00640 1 --2- 172.21.15.161:0/4202358188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf84008860 0x7fdf8400ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9fe00640 1 --2- 172.21.15.161:0/4202358188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf8400b190 0x7fdf8400d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9f400640 1 --2- 172.21.15.161:0/4202358188 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfa0071b40 0x7fdf84003c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9ea00640 1 -- 172.21.15.161:0/4202358188 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfa0071b40 msgr2=0x7fdf84003c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9ea00640 1 --2- 172.21.15.161:0/4202358188 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfa0071b40 0x7fdf84003c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9ea00640 1 -- 172.21.15.161:0/4202358188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf8400b190 msgr2=0x7fdf8400d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:24.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9ea00640 1 --2- 172.21.15.161:0/4202358188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf8400b190 0x7fdf8400d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:24.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.847+0000 7fdf9ea00640 1 -- 172.21.15.161:0/4202358188 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf84003450 con 0x7fdf84008860 2024-05-22T21:24:24.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.851+0000 7fdf9ea00640 1 --2- 172.21.15.161:0/4202358188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf84008860 0x7fdf8400ac50 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009ae0 tx=0x7fdf8c01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e13947096caa40 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:24.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.851+0000 7fdf9e000640 1 -- 172.21.15.161:0/4202358188 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0a5520 con 0x7fdf84008860 2024-05-22T21:24:24.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.851+0000 7fdf9e000640 1 -- 172.21.15.161:0/4202358188 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8c0a5680 con 0x7fdf84008860 2024-05-22T21:24:24.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.851+0000 7fdf9e000640 1 -- 172.21.15.161:0/4202358188 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0a5a30 con 0x7fdf84008860 2024-05-22T21:24:24.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.851+0000 7fdfa6000640 1 -- 172.21.15.161:0/4202358188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf84008860 msgr2=0x7fdf8400ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:24.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.851+0000 7fdfa6000640 1 --2- 172.21.15.161:0/4202358188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf84008860 0x7fdf8400ac50 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009ae0 tx=0x7fdf8c01ca40 comp rx=0 tx=0).stop 2024-05-22T21:24:24.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.851+0000 7fdf9fe00640 1 -- 172.21.15.161:0/4202358188 reap_dead start 2024-05-22T21:24:24.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.851+0000 7fdfa6000640 1 -- 172.21.15.161:0/4202358188 shutdown_connections 2024-05-22T21:24:24.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.851+0000 7fdfa6000640 1 -- 172.21.15.161:0/4202358188 >> 172.21.15.161:0/4202358188 conn(0x7fdfa006a0a0 msgr2=0x7fdfa006a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:24.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.851+0000 7fdfa6000640 1 -- 172.21.15.161:0/4202358188 shutdown_connections 2024-05-22T21:24:24.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.851+0000 7fdfa6000640 1 -- 172.21.15.161:0/4202358188 wait complete. 2024-05-22T21:24:24.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.851+0000 7fdfa6000640 1 Processor -- start 2024-05-22T21:24:24.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.851+0000 7fdfa6000640 1 -- start start 2024-05-22T21:24:24.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.851+0000 7fdfa6000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf84008860 0x7fdfa0143570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:24.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdfa6000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfa00aeac0 0x7fdfa015a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:24.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdfa6000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfa015bda0 0x7fdfa015d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:24.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdfa6000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa015ad00 con 0x7fdfa00aeac0 2024-05-22T21:24:24.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdfa6000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa015afe0 con 0x7fdf84008860 2024-05-22T21:24:24.854 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdfa6000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa015b2c0 con 0x7fdfa015bda0 2024-05-22T21:24:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdf9ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfa00aeac0 0x7fdfa015a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdf9ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfa00aeac0 0x7fdfa015a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57346/0 (socket says 172.21.15.161:57346) 2024-05-22T21:24:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdf9ea00640 1 -- 172.21.15.161:0/1192738517 learned_addr learned my addr 172.21.15.161:0/1192738517 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdf9fe00640 1 --2- 172.21.15.161:0/1192738517 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfa015bda0 0x7fdfa015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdf9f400640 1 --2- 172.21.15.161:0/1192738517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf84008860 0x7fdfa0143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdf9ea00640 1 -- 172.21.15.161:0/1192738517 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfa015bda0 msgr2=0x7fdfa015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdf9ea00640 1 --2- 172.21.15.161:0/1192738517 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfa015bda0 0x7fdfa015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdf9ea00640 1 -- 172.21.15.161:0/1192738517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf84008860 msgr2=0x7fdfa0143570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdf9ea00640 1 --2- 172.21.15.161:0/1192738517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf84008860 0x7fdfa0143570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdf9ea00640 1 -- 172.21.15.161:0/1192738517 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa015b5a0 con 0x7fdfa00aeac0 2024-05-22T21:24:24.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdf9f400640 1 --2- 172.21.15.161:0/1192738517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf84008860 0x7fdfa0143570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:24.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdf9ea00640 1 --2- 172.21.15.161:0/1192738517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfa00aeac0 0x7fdfa015a3d0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c01c870 tx=0x7fdf8c0197e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:24.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdf97400640 1 -- 172.21.15.161:0/1192738517 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0bd070 con 0x7fdfa00aeac0 2024-05-22T21:24:24.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdf97400640 1 -- 172.21.15.161:0/1192738517 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8c004020 con 0x7fdfa00aeac0 2024-05-22T21:24:24.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdfa6000640 1 -- 172.21.15.161:0/1192738517 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa015b880 con 0x7fdfa00aeac0 2024-05-22T21:24:24.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdf97400640 1 -- 172.21.15.161:0/1192738517 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0043d0 con 0x7fdfa00aeac0 2024-05-22T21:24:24.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdfa6000640 1 -- 172.21.15.161:0/1192738517 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa015a910 con 0x7fdfa00aeac0 2024-05-22T21:24:24.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdfa6000640 1 -- 172.21.15.161:0/1192738517 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa015ba10 con 0x7fdfa00aeac0 2024-05-22T21:24:24.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.855+0000 7fdf97400640 1 -- 172.21.15.161:0/1192738517 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf8c0b4070 con 0x7fdfa00aeac0 2024-05-22T21:24:24.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.859+0000 7fdf97400640 1 --2- 172.21.15.161:0/1192738517 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdf80066690 0x7fdf80068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:24.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.859+0000 7fdf97400640 1 -- 172.21.15.161:0/1192738517 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5386+0+0 (secure 0 0 0) 0x7fdf8c11d570 con 0x7fdfa00aeac0 2024-05-22T21:24:24.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.859+0000 7fdf9f400640 1 --2- 172.21.15.161:0/1192738517 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdf80066690 0x7fdf80068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:24.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.859+0000 7fdf97400640 1 -- 172.21.15.161:0/1192738517 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=181}) v3 -- 0x7fdf8006d4c0 con 0x7fdfa00aeac0 2024-05-22T21:24:24.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.859+0000 7fdf9f400640 1 --2- 172.21.15.161:0/1192738517 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdf80066690 0x7fdf80068b50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fdf90000f90 tx=0x7fdf90009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:24.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:24.863+0000 7fdf97400640 1 -- 172.21.15.161:0/1192738517 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdf8c0eaa30 con 0x7fdfa00aeac0 2024-05-22T21:24:25.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.015+0000 7fdfa6000640 1 -- 172.21.15.161:0/1192738517 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdfa0067220 con 0x7fdfa00aeac0 2024-05-22T21:24:25.124 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:24:25.128 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:24:25.304 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.303+0000 7fdf97400640 1 -- 172.21.15.161:0/1192738517 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(181..181 src has 1..181) v4 ==== 704+0+0 (secure 0 0 0) 0x7fdf8c0ee6f0 con 0x7fdfa00aeac0 2024-05-22T21:24:25.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.303+0000 7fdf97400640 1 -- 172.21.15.161:0/1192738517 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=182}) v3 -- 0x7fdf8006c250 con 0x7fdfa00aeac0 2024-05-22T21:24:25.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.307+0000 7fdf97400640 1 -- 172.21.15.161:0/1192738517 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v181) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdf8c0b2070 con 0x7fdfa00aeac0 2024-05-22T21:24:25.311 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' removed 2024-05-22T21:24:25.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.315+0000 7fdfa6000640 1 -- 172.21.15.161:0/1192738517 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdf80066690 msgr2=0x7fdf80068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:25.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.315+0000 7fdfa6000640 1 --2- 172.21.15.161:0/1192738517 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdf80066690 0x7fdf80068b50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fdf90000f90 tx=0x7fdf90009040 comp rx=0 tx=0).stop 2024-05-22T21:24:25.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.315+0000 7fdfa6000640 1 -- 172.21.15.161:0/1192738517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfa00aeac0 msgr2=0x7fdfa015a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:25.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.315+0000 7fdfa6000640 1 --2- 172.21.15.161:0/1192738517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfa00aeac0 0x7fdfa015a3d0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c01c870 tx=0x7fdf8c0197e0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.315+0000 7fdfa6000640 1 -- 172.21.15.161:0/1192738517 shutdown_connections 2024-05-22T21:24:25.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.315+0000 7fdfa6000640 1 --2- 172.21.15.161:0/1192738517 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fdf80066690 0x7fdf80068b50 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.315+0000 7fdfa6000640 1 --2- 172.21.15.161:0/1192738517 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfa015bda0 0x7fdfa015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.315+0000 7fdfa6000640 1 --2- 172.21.15.161:0/1192738517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfa00aeac0 0x7fdfa015a3d0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.315+0000 7fdfa6000640 1 --2- 172.21.15.161:0/1192738517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf84008860 0x7fdfa0143570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.315+0000 7fdfa6000640 1 -- 172.21.15.161:0/1192738517 >> 172.21.15.161:0/1192738517 conn(0x7fdfa006a0a0 msgr2=0x7fdfa015f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.315+0000 7fdfa6000640 1 -- 172.21.15.161:0/1192738517 shutdown_connections 2024-05-22T21:24:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.315+0000 7fdfa6000640 1 -- 172.21.15.161:0/1192738517 wait complete. 2024-05-22T21:24:25.336 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-22T21:24:25.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.475+0000 7f6305600640 1 Processor -- start 2024-05-22T21:24:25.475 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.475+0000 7f6305600640 1 -- start start 2024-05-22T21:24:25.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.475+0000 7f6305600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f63000ceb40 0x7f63000cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:25.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.475+0000 7f6305600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63000cff60 con 0x7f6300099a40 2024-05-22T21:24:25.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.475+0000 7f6305600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630015d8e0 con 0x7f630006f0a0 2024-05-22T21:24:25.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.475+0000 7f6305600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63000cd050 con 0x7f63000ceb40 2024-05-22T21:24:25.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.475+0000 7f62fea00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f6300099a40 0x7f6300099e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56972/0 (socket says 172.21.15.161:56972) 2024-05-22T21:24:25.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.475+0000 7f62f6000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f63000ceb40 0x7f63000cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:25.476 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.475+0000 7f62fea00640 1 -- 172.21.15.161:0/306903335 learned_addr learned my addr 172.21.15.161:0/306903335 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:25.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.475+0000 7f62f6000640 1 -- 172.21.15.161:0/306903335 >> v1:172.21.15.179:6789/0 conn(0x7f630006f0a0 legacy=0x7f630006f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:25.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.475+0000 7f62f6000640 1 -- 172.21.15.161:0/306903335 >> v1:172.21.15.161:6789/0 conn(0x7f6300099a40 legacy=0x7f6300099e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:25.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.475+0000 7f62f6000640 1 -- 172.21.15.161:0/306903335 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630006e000 con 0x7f63000ceb40 2024-05-22T21:24:25.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.479+0000 7f62f6000640 1 --2- 172.21.15.161:0/306903335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63000ceb40 0x7f63000cfa20 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f62e80097c0 tx=0x7f62e801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b324003f376c16 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:25.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.479+0000 7f62fe000640 1 -- 172.21.15.161:0/306903335 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e80aa5f0 con 0x7f63000ceb40 2024-05-22T21:24:25.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.479+0000 7f62fe000640 1 -- 172.21.15.161:0/306903335 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e80aa750 con 0x7f63000ceb40 2024-05-22T21:24:25.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.479+0000 7f62fe000640 1 -- 172.21.15.161:0/306903335 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e80aaae0 con 0x7f63000ceb40 2024-05-22T21:24:25.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.479+0000 7f6305600640 1 -- 172.21.15.161:0/306903335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63000ceb40 msgr2=0x7f63000cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:25.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.479+0000 7f6305600640 1 --2- 172.21.15.161:0/306903335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63000ceb40 0x7f63000cfa20 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f62e80097c0 tx=0x7f62e801c470 comp rx=0 tx=0).stop 2024-05-22T21:24:25.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.479+0000 7f6305600640 1 -- 172.21.15.161:0/306903335 shutdown_connections 2024-05-22T21:24:25.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.479+0000 7f6305600640 1 --2- 172.21.15.161:0/306903335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63000ceb40 0x7f63000cfa20 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.479+0000 7f6305600640 1 -- 172.21.15.161:0/306903335 >> 172.21.15.161:0/306903335 conn(0x7f63000b2a60 msgr2=0x7f63000b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:25.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.479+0000 7f6305600640 1 -- 172.21.15.161:0/306903335 shutdown_connections 2024-05-22T21:24:25.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.479+0000 7f6305600640 1 -- 172.21.15.161:0/306903335 wait complete. 2024-05-22T21:24:25.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f6305600640 1 Processor -- start 2024-05-22T21:24:25.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f6305600640 1 -- start start 2024-05-22T21:24:25.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f6305600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f630006f0a0 0x7f630014f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:25.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f6305600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6300099a40 0x7f63000ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:25.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f6305600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63000ceb40 0x7f630014fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:25.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f6305600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630014fbb0 con 0x7f6300099a40 2024-05-22T21:24:25.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f6305600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6300154e20 con 0x7f63000ceb40 2024-05-22T21:24:25.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f6305600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6300155100 con 0x7f630006f0a0 2024-05-22T21:24:25.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f62f6000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6300099a40 0x7f63000ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:25.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f62f6000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6300099a40 0x7f63000ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57364/0 (socket says 172.21.15.161:57364) 2024-05-22T21:24:25.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f62f6000640 1 -- 172.21.15.161:0/3322970261 learned_addr learned my addr 172.21.15.161:0/3322970261 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:25.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f62fea00640 1 --2- 172.21.15.161:0/3322970261 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f630006f0a0 0x7f630014f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:25.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f62ff400640 1 --2- 172.21.15.161:0/3322970261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63000ceb40 0x7f630014fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:25.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f62f6000640 1 -- 172.21.15.161:0/3322970261 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f630006f0a0 msgr2=0x7f630014f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:25.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f62f6000640 1 --2- 172.21.15.161:0/3322970261 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f630006f0a0 0x7f630014f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f62f6000640 1 -- 172.21.15.161:0/3322970261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63000ceb40 msgr2=0x7f630014fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:25.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f62f6000640 1 --2- 172.21.15.161:0/3322970261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63000ceb40 0x7f630014fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f62f6000640 1 -- 172.21.15.161:0/3322970261 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e8009420 con 0x7f6300099a40 2024-05-22T21:24:25.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f62ff400640 1 --2- 172.21.15.161:0/3322970261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63000ceb40 0x7f630014fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:25.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f62fea00640 1 --2- 172.21.15.161:0/3322970261 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f630006f0a0 0x7f630014f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:25.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f62f6000640 1 --2- 172.21.15.161:0/3322970261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6300099a40 0x7f63000ca7e0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f62e80ac040 tx=0x7f62e801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:25.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f62f7400640 1 -- 172.21.15.161:0/3322970261 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e80b4810 con 0x7f6300099a40 2024-05-22T21:24:25.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f62f7400640 1 -- 172.21.15.161:0/3322970261 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e80b49b0 con 0x7f6300099a40 2024-05-22T21:24:25.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f62f7400640 1 -- 172.21.15.161:0/3322970261 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e80b4d60 con 0x7f6300099a40 2024-05-22T21:24:25.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f6305600640 1 -- 172.21.15.161:0/3322970261 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63001553e0 con 0x7f6300099a40 2024-05-22T21:24:25.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f6305600640 1 -- 172.21.15.161:0/3322970261 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63001559a0 con 0x7f6300099a40 2024-05-22T21:24:25.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.483+0000 7f6305600640 1 -- 172.21.15.161:0/3322970261 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6300073ce0 con 0x7f6300099a40 2024-05-22T21:24:25.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.487+0000 7f62f7400640 1 -- 172.21.15.161:0/3322970261 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f62e80b3390 con 0x7f6300099a40 2024-05-22T21:24:25.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.487+0000 7f62f7400640 1 --2- 172.21.15.161:0/3322970261 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f62dc066690 0x7f62dc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:25.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.487+0000 7f62f7400640 1 -- 172.21.15.161:0/3322970261 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f62e80a7070 con 0x7f6300099a40 2024-05-22T21:24:25.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.487+0000 7f62fea00640 1 --2- 172.21.15.161:0/3322970261 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f62dc066690 0x7f62dc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:25.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.487+0000 7f62fea00640 1 --2- 172.21.15.161:0/3322970261 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f62dc066690 0x7f62dc068b50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f62f0005890 tx=0x7f62f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:25.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.491+0000 7f62f7400640 1 -- 172.21.15.161:0/3322970261 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f62e80ebf50 con 0x7f6300099a40 2024-05-22T21:24:25.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.643+0000 7f6305600640 1 -- 172.21.15.161:0/3322970261 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6300155b90 con 0x7f6300099a40 2024-05-22T21:24:25.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.647+0000 7f62f7400640 1 -- 172.21.15.161:0/3322970261 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f62e80ef570 con 0x7f6300099a40 2024-05-22T21:24:25.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.651+0000 7f6305600640 1 -- 172.21.15.161:0/3322970261 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f62dc066690 msgr2=0x7f62dc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:25.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.651+0000 7f6305600640 1 --2- 172.21.15.161:0/3322970261 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f62dc066690 0x7f62dc068b50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f62f0005890 tx=0x7f62f0009040 comp rx=0 tx=0).stop 2024-05-22T21:24:25.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.651+0000 7f6305600640 1 -- 172.21.15.161:0/3322970261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6300099a40 msgr2=0x7f63000ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:25.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.651+0000 7f6305600640 1 --2- 172.21.15.161:0/3322970261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6300099a40 0x7f63000ca7e0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f62e80ac040 tx=0x7f62e801acb0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.651+0000 7f6305600640 1 -- 172.21.15.161:0/3322970261 shutdown_connections 2024-05-22T21:24:25.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.651+0000 7f6305600640 1 --2- 172.21.15.161:0/3322970261 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f62dc066690 0x7f62dc068b50 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.651+0000 7f6305600640 1 --2- 172.21.15.161:0/3322970261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63000ceb40 0x7f630014fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.651+0000 7f6305600640 1 --2- 172.21.15.161:0/3322970261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6300099a40 0x7f63000ca7e0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.651+0000 7f6305600640 1 --2- 172.21.15.161:0/3322970261 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f630006f0a0 0x7f630014f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.651+0000 7f6305600640 1 -- 172.21.15.161:0/3322970261 >> 172.21.15.161:0/3322970261 conn(0x7f63000b2a60 msgr2=0x7f63000d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:25.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.651+0000 7f6305600640 1 -- 172.21.15.161:0/3322970261 shutdown_connections 2024-05-22T21:24:25.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.651+0000 7f6305600640 1 -- 172.21.15.161:0/3322970261 wait complete. 2024-05-22T21:24:25.670 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2024-05-22T21:24:25.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.807+0000 7f2d97200640 1 Processor -- start 2024-05-22T21:24:25.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.807+0000 7f2d97200640 1 -- start start 2024-05-22T21:24:25.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.807+0000 7f2d97200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2d98144550 0x7f2d980cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:25.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.807+0000 7f2d97200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2d9815b420 0x7f2d9815b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:25.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.807+0000 7f2d97200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d9806de20 con 0x7f2d9815ea20 2024-05-22T21:24:25.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.807+0000 7f2d97200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9806ba50 con 0x7f2d9815b420 2024-05-22T21:24:25.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.807+0000 7f2d97200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9815be10 con 0x7f2d98144550 2024-05-22T21:24:25.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.807+0000 7f2d95400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2d9815b420 0x7f2d9815b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:25.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.807+0000 7f2d95400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2d9815b420 0x7f2d9815b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57380/0 (socket says 172.21.15.161:57380) 2024-05-22T21:24:25.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.807+0000 7f2d95400640 1 -- 172.21.15.161:0/3783416083 learned_addr learned my addr 172.21.15.161:0/3783416083 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:25.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.807+0000 7f2d95e00640 1 --2- 172.21.15.161:0/3783416083 >> v2:172.21.15.179:3300/0 conn(0x7f2d98144550 0x7f2d980cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:25.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.807+0000 7f2d95400640 1 -- 172.21.15.161:0/3783416083 >> v2:172.21.15.179:3300/0 conn(0x7f2d98144550 msgr2=0x7f2d980cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:25.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.807+0000 7f2d95400640 1 --2- 172.21.15.161:0/3783416083 >> v2:172.21.15.179:3300/0 conn(0x7f2d98144550 0x7f2d980cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.807+0000 7f2d95400640 1 -- 172.21.15.161:0/3783416083 >> v1:172.21.15.179:6789/0 conn(0x7f2d9815ea20 legacy=0x7f2d9815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:25.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.807+0000 7f2d95400640 1 -- 172.21.15.161:0/3783416083 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d980cdd20 con 0x7f2d9815b420 2024-05-22T21:24:25.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.811+0000 7f2d95400640 1 --2- 172.21.15.161:0/3783416083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d9815b420 0x7f2d9815b8d0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f2d9806e040 tx=0x7f2d8401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a9681c153fc77b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:25.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.811+0000 7f2d94a00640 1 -- 172.21.15.161:0/3783416083 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d840ab660 con 0x7f2d9815b420 2024-05-22T21:24:25.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.811+0000 7f2d94a00640 1 -- 172.21.15.161:0/3783416083 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d840ab7c0 con 0x7f2d9815b420 2024-05-22T21:24:25.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.811+0000 7f2d94a00640 1 -- 172.21.15.161:0/3783416083 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d840abb50 con 0x7f2d9815b420 2024-05-22T21:24:25.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.811+0000 7f2d97200640 1 -- 172.21.15.161:0/3783416083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d9815b420 msgr2=0x7f2d9815b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:25.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.811+0000 7f2d97200640 1 --2- 172.21.15.161:0/3783416083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d9815b420 0x7f2d9815b8d0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f2d9806e040 tx=0x7f2d8401c460 comp rx=0 tx=0).stop 2024-05-22T21:24:25.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.811+0000 7f2d97200640 1 -- 172.21.15.161:0/3783416083 shutdown_connections 2024-05-22T21:24:25.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.811+0000 7f2d97200640 1 --2- 172.21.15.161:0/3783416083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d9815b420 0x7f2d9815b8d0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.811+0000 7f2d97200640 1 --2- 172.21.15.161:0/3783416083 >> v2:172.21.15.179:3300/0 conn(0x7f2d98144550 0x7f2d980cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.811+0000 7f2d97200640 1 -- 172.21.15.161:0/3783416083 >> 172.21.15.161:0/3783416083 conn(0x7f2d9806a4d0 msgr2=0x7f2d980b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:25.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.811+0000 7f2d97200640 1 -- 172.21.15.161:0/3783416083 shutdown_connections 2024-05-22T21:24:25.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.811+0000 7f2d97200640 1 -- 172.21.15.161:0/3783416083 wait complete. 2024-05-22T21:24:25.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d97200640 1 Processor -- start 2024-05-22T21:24:25.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d97200640 1 -- start start 2024-05-22T21:24:25.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d97200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d98144550 0x7f2d98142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:25.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d97200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d9815b420 0x7f2d9813ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:25.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d97200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d9815ea20 0x7f2d980afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:25.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d97200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9813de60 con 0x7f2d9815ea20 2024-05-22T21:24:25.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d97200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9813e0d0 con 0x7f2d9815b420 2024-05-22T21:24:25.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d97200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9813e3b0 con 0x7f2d98144550 2024-05-22T21:24:25.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d96800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d9815ea20 0x7f2d980afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:25.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d95400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d9815b420 0x7f2d9813ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:25.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d95e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d98144550 0x7f2d98142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:25.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d96800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d9815ea20 0x7f2d980afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57394/0 (socket says 172.21.15.161:57394) 2024-05-22T21:24:25.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d95e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d98144550 0x7f2d98142f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:48128/0 (socket says 172.21.15.161:48128) 2024-05-22T21:24:25.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d96800640 1 -- 172.21.15.161:0/223012576 learned_addr learned my addr 172.21.15.161:0/223012576 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:25.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d95400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d9815b420 0x7f2d9813ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:59138/0 (socket says 172.21.15.161:59138) 2024-05-22T21:24:25.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d96800640 1 -- 172.21.15.161:0/223012576 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d98144550 msgr2=0x7f2d98142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:25.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d96800640 1 --2- 172.21.15.161:0/223012576 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d98144550 0x7f2d98142f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d96800640 1 -- 172.21.15.161:0/223012576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d9815b420 msgr2=0x7f2d9813ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:25.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d96800640 1 --2- 172.21.15.161:0/223012576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d9815b420 0x7f2d9813ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:25.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d96800640 1 -- 172.21.15.161:0/223012576 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d840094d0 con 0x7f2d9815ea20 2024-05-22T21:24:25.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d96800640 1 --2- 172.21.15.161:0/223012576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d9815ea20 0x7f2d980afbb0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000c340 tx=0x7f2d9000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:25.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d8e000640 1 -- 172.21.15.161:0/223012576 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90005220 con 0x7f2d9815ea20 2024-05-22T21:24:25.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d8e000640 1 -- 172.21.15.161:0/223012576 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d90005380 con 0x7f2d9815ea20 2024-05-22T21:24:25.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d8e000640 1 -- 172.21.15.161:0/223012576 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90007270 con 0x7f2d9815ea20 2024-05-22T21:24:25.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d97200640 1 -- 172.21.15.161:0/223012576 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9813e690 con 0x7f2d9815ea20 2024-05-22T21:24:25.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.815+0000 7f2d97200640 1 -- 172.21.15.161:0/223012576 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9813f0f0 con 0x7f2d9815ea20 2024-05-22T21:24:25.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.819+0000 7f2d97200640 1 -- 172.21.15.161:0/223012576 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d9813e880 con 0x7f2d9815ea20 2024-05-22T21:24:25.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.819+0000 7f2d8e000640 1 -- 172.21.15.161:0/223012576 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2d900073d0 con 0x7f2d9815ea20 2024-05-22T21:24:25.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.819+0000 7f2d8e000640 1 --2- 172.21.15.161:0/223012576 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d78066690 0x7f2d78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:25.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.819+0000 7f2d8e000640 1 -- 172.21.15.161:0/223012576 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2d900831d0 con 0x7f2d9815ea20 2024-05-22T21:24:25.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.819+0000 7f2d95e00640 1 --2- 172.21.15.161:0/223012576 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d78066690 0x7f2d78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:25.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.819+0000 7f2d95e00640 1 --2- 172.21.15.161:0/223012576 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d78066690 0x7f2d78068b50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2d88003b90 tx=0x7f2d880098d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:25.824 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:25.823+0000 7f2d8e000640 1 -- 172.21.15.161:0/223012576 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2d90050980 con 0x7f2d9815ea20 2024-05-22T21:24:26.034 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.035+0000 7f2d97200640 1 -- 172.21.15.161:0/223012576 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7f2d98067070 con 0x7f2d9815ea20 2024-05-22T21:24:26.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.683+0000 7f2d8e000640 1 -- 172.21.15.161:0/223012576 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v530) v1 ==== 131+0+0 (secure 0 0 0) 0x7f2d90053fa0 con 0x7f2d9815ea20 2024-05-22T21:24:26.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.687+0000 7f2d97200640 1 -- 172.21.15.161:0/223012576 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d78066690 msgr2=0x7f2d78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:26.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.687+0000 7f2d97200640 1 --2- 172.21.15.161:0/223012576 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d78066690 0x7f2d78068b50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2d88003b90 tx=0x7f2d880098d0 comp rx=0 tx=0).stop 2024-05-22T21:24:26.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.687+0000 7f2d97200640 1 -- 172.21.15.161:0/223012576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d9815ea20 msgr2=0x7f2d980afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:26.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.687+0000 7f2d97200640 1 --2- 172.21.15.161:0/223012576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d9815ea20 0x7f2d980afbb0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000c340 tx=0x7f2d9000c800 comp rx=0 tx=0).stop 2024-05-22T21:24:26.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.687+0000 7f2d97200640 1 -- 172.21.15.161:0/223012576 shutdown_connections 2024-05-22T21:24:26.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.687+0000 7f2d97200640 1 --2- 172.21.15.161:0/223012576 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2d78066690 0x7f2d78068b50 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:26.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.687+0000 7f2d97200640 1 --2- 172.21.15.161:0/223012576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d9815ea20 0x7f2d980afbb0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:26.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.687+0000 7f2d97200640 1 --2- 172.21.15.161:0/223012576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d9815b420 0x7f2d9813ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:26.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.687+0000 7f2d97200640 1 --2- 172.21.15.161:0/223012576 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2d98144550 0x7f2d98142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:26.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.687+0000 7f2d97200640 1 -- 172.21.15.161:0/223012576 >> 172.21.15.161:0/223012576 conn(0x7f2d9806a4d0 msgr2=0x7f2d980ae7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:26.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.687+0000 7f2d97200640 1 -- 172.21.15.161:0/223012576 shutdown_connections 2024-05-22T21:24:26.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.687+0000 7f2d97200640 1 -- 172.21.15.161:0/223012576 wait complete. 2024-05-22T21:24:26.708 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2024-05-22T21:24:26.708 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) 2024-05-22T21:24:26.708 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2024-05-22T21:24:26.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.843+0000 7fead1600640 1 Processor -- start 2024-05-22T21:24:26.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.843+0000 7fead1600640 1 -- start start 2024-05-22T21:24:26.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7fead1600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7feacc06ce80 0x7feacc165db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:26.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7fead1600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feacc06de20 con 0x7feacc15b420 2024-05-22T21:24:26.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7fead1600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feacc0cbcb0 con 0x7feacc144550 2024-05-22T21:24:26.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7fead1600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feacc06ba50 con 0x7feacc06ce80 2024-05-22T21:24:26.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feac3e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7feacc15b420 0x7feacc15b8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41838/0 (socket says 172.21.15.161:41838) 2024-05-22T21:24:26.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feac3e00640 1 -- 172.21.15.161:0/759267345 learned_addr learned my addr 172.21.15.161:0/759267345 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:26.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7fead0c00640 1 --2- 172.21.15.161:0/759267345 >> v2:172.21.15.179:3300/0 conn(0x7feacc06ce80 0x7feacc165db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:26.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feacb400640 1 -- 172.21.15.161:0/759267345 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926268830 0 0) 0x7feacc06de20 con 0x7feacc15b420 2024-05-22T21:24:26.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feacb400640 1 -- 172.21.15.161:0/759267345 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feab0003530 con 0x7feacc15b420 2024-05-22T21:24:26.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feacb400640 1 -- 172.21.15.161:0/759267345 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2348163713 0 0) 0x7feab0003530 con 0x7feacc15b420 2024-05-22T21:24:26.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feacb400640 1 -- 172.21.15.161:0/759267345 >> v2:172.21.15.179:3300/0 conn(0x7feacc06ce80 msgr2=0x7feacc165db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:26.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feacb400640 1 --2- 172.21.15.161:0/759267345 >> v2:172.21.15.179:3300/0 conn(0x7feacc06ce80 0x7feacc165db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:26.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feacb400640 1 -- 172.21.15.161:0/759267345 >> v1:172.21.15.179:6789/0 conn(0x7feacc144550 legacy=0x7feacc0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:26.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feacb400640 1 -- 172.21.15.161:0/759267345 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc0cdd20 con 0x7feacc15b420 2024-05-22T21:24:26.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feacb400640 1 -- 172.21.15.161:0/759267345 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7feab4001680 con 0x7feacc15b420 2024-05-22T21:24:26.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feacb400640 1 -- 172.21.15.161:0/759267345 >> v1:172.21.15.161:6789/0 conn(0x7feacc15b420 legacy=0x7feacc15b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:26.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feacb400640 1 --2- 172.21.15.161:0/759267345 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feab0003980 0x7feab0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:26.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feacb400640 1 --2- 172.21.15.161:0/759267345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab00047d0 0x7feab0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:26.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feacb400640 1 --2- 172.21.15.161:0/759267345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feab0005190 0x7feab000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:26.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feacb400640 1 -- 172.21.15.161:0/759267345 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc06ba50 con 0x7feab00047d0 2024-05-22T21:24:26.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feacb400640 1 -- 172.21.15.161:0/759267345 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab000c130 con 0x7feab0005190 2024-05-22T21:24:26.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feacb400640 1 -- 172.21.15.161:0/759267345 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab000c2f0 con 0x7feab0003980 2024-05-22T21:24:26.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feac3e00640 1 --2- 172.21.15.161:0/759267345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab00047d0 0x7feab0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:26.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7fead0c00640 1 --2- 172.21.15.161:0/759267345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feab0005190 0x7feab000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:26.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feacbe00640 1 --2- 172.21.15.161:0/759267345 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feab0003980 0x7feab0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:26.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feac3e00640 1 -- 172.21.15.161:0/759267345 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feab0003980 msgr2=0x7feab0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:26.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feac3e00640 1 --2- 172.21.15.161:0/759267345 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feab0003980 0x7feab0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:26.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feac3e00640 1 -- 172.21.15.161:0/759267345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feab0005190 msgr2=0x7feab000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:26.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feac3e00640 1 --2- 172.21.15.161:0/759267345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feab0005190 0x7feab000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:26.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7feac3e00640 1 -- 172.21.15.161:0/759267345 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab0003530 con 0x7feab00047d0 2024-05-22T21:24:26.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7fead0c00640 1 --2- 172.21.15.161:0/759267345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feab0005190 0x7feab000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:24:26.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.847+0000 7fead0c00640 1 -- 172.21.15.161:0/759267345 reap_dead start 2024-05-22T21:24:26.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.851+0000 7feac3e00640 1 --2- 172.21.15.161:0/759267345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab00047d0 0x7feab0004c50 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7feab400c210 tx=0x7feab401cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fb7fdb0b22c0d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:26.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.851+0000 7feacb400640 1 -- 172.21.15.161:0/759267345 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4005ae0 con 0x7feab00047d0 2024-05-22T21:24:26.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.851+0000 7feacb400640 1 -- 172.21.15.161:0/759267345 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab4005c40 con 0x7feab00047d0 2024-05-22T21:24:26.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.851+0000 7feacb400640 1 -- 172.21.15.161:0/759267345 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40072b0 con 0x7feab00047d0 2024-05-22T21:24:26.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.851+0000 7fead1600640 1 -- 172.21.15.161:0/759267345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab00047d0 msgr2=0x7feab0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:26.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.851+0000 7fead1600640 1 --2- 172.21.15.161:0/759267345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab00047d0 0x7feab0004c50 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7feab400c210 tx=0x7feab401cb90 comp rx=0 tx=0).stop 2024-05-22T21:24:26.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.851+0000 7fead1600640 1 -- 172.21.15.161:0/759267345 shutdown_connections 2024-05-22T21:24:26.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.851+0000 7fead1600640 1 --2- 172.21.15.161:0/759267345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab00047d0 0x7feab0004c50 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:26.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.851+0000 7fead1600640 1 -- 172.21.15.161:0/759267345 >> 172.21.15.161:0/759267345 conn(0x7feacc06a4d0 msgr2=0x7feacc0b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:26.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.851+0000 7fead1600640 1 -- 172.21.15.161:0/759267345 shutdown_connections 2024-05-22T21:24:26.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.851+0000 7fead1600640 1 -- 172.21.15.161:0/759267345 wait complete. 2024-05-22T21:24:26.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.851+0000 7fead1600640 1 Processor -- start 2024-05-22T21:24:26.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.851+0000 7fead1600640 1 -- start start 2024-05-22T21:24:26.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7fead1600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab00047d0 0x7feacc0b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:26.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7fead1600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feacc15a980 0x7feacc14d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:26.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7fead1600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feacc0ae7f0 0x7feacc13f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:26.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7fead1600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc0af0f0 con 0x7feab00047d0 2024-05-22T21:24:26.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7fead1600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc0af3d0 con 0x7feacc15a980 2024-05-22T21:24:26.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7fead1600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feacc0af6b0 con 0x7feacc0ae7f0 2024-05-22T21:24:26.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7feacbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab00047d0 0x7feacc0b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:26.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7feacbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab00047d0 0x7feacc0b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60498/0 (socket says 172.21.15.161:60498) 2024-05-22T21:24:26.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7feacbe00640 1 -- 172.21.15.161:0/1157996002 learned_addr learned my addr 172.21.15.161:0/1157996002 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:26.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7feac3e00640 1 --2- 172.21.15.161:0/1157996002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feacc15a980 0x7feacc14d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:26.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7fead0c00640 1 --2- 172.21.15.161:0/1157996002 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feacc0ae7f0 0x7feacc13f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:26.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7feacbe00640 1 -- 172.21.15.161:0/1157996002 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feacc0ae7f0 msgr2=0x7feacc13f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:26.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7feacbe00640 1 --2- 172.21.15.161:0/1157996002 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feacc0ae7f0 0x7feacc13f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:26.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7feacbe00640 1 -- 172.21.15.161:0/1157996002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feacc15a980 msgr2=0x7feacc14d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:26.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7feacbe00640 1 --2- 172.21.15.161:0/1157996002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feacc15a980 0x7feacc14d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:26.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7feacbe00640 1 -- 172.21.15.161:0/1157996002 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc0af990 con 0x7feab00047d0 2024-05-22T21:24:26.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7fead0c00640 1 --2- 172.21.15.161:0/1157996002 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feacc0ae7f0 0x7feacc13f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:26.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7feacbe00640 1 --2- 172.21.15.161:0/1157996002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab00047d0 0x7feacc0b0190 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7feabc00f5f0 tx=0x7feabc00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:26.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7feac8c00640 1 -- 172.21.15.161:0/1157996002 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc0052e0 con 0x7feab00047d0 2024-05-22T21:24:26.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7feac8c00640 1 -- 172.21.15.161:0/1157996002 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc005440 con 0x7feab00047d0 2024-05-22T21:24:26.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7fead1600640 1 -- 172.21.15.161:0/1157996002 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feacc0afc70 con 0x7feab00047d0 2024-05-22T21:24:26.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7feac8c00640 1 -- 172.21.15.161:0/1157996002 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc003820 con 0x7feab00047d0 2024-05-22T21:24:26.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7fead1600640 1 -- 172.21.15.161:0/1157996002 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feacc0aec80 con 0x7feab00047d0 2024-05-22T21:24:26.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7fead1600640 1 -- 172.21.15.161:0/1157996002 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feacc0aee70 con 0x7feab00047d0 2024-05-22T21:24:26.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.855+0000 7feac8c00640 1 -- 172.21.15.161:0/1157996002 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feabc003980 con 0x7feab00047d0 2024-05-22T21:24:26.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.859+0000 7feac8c00640 1 --2- 172.21.15.161:0/1157996002 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feaa80666c0 0x7feaa8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:26.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.859+0000 7feac8c00640 1 -- 172.21.15.161:0/1157996002 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 4943+0+0 (secure 0 0 0) 0x7feabc083e80 con 0x7feab00047d0 2024-05-22T21:24:26.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.859+0000 7feac3e00640 1 --2- 172.21.15.161:0/1157996002 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feaa80666c0 0x7feaa8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:26.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.859+0000 7feac3e00640 1 --2- 172.21.15.161:0/1157996002 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feaa80666c0 0x7feaa8068b80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7feab40ab000 tx=0x7feab40a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:26.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:26.863+0000 7feac8c00640 1 -- 172.21.15.161:0/1157996002 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7feabc051450 con 0x7feab00047d0 2024-05-22T21:24:27.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.067+0000 7fead1600640 1 -- 172.21.15.161:0/1157996002 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7feacc067070 con 0x7feab00047d0 2024-05-22T21:24:27.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.687+0000 7feac8c00640 1 -- 172.21.15.161:0/1157996002 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v531) v1 ==== 138+0+0 (secure 0 0 0) 0x7feabc059e60 con 0x7feab00047d0 2024-05-22T21:24:27.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.691+0000 7fead1600640 1 -- 172.21.15.161:0/1157996002 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feaa80666c0 msgr2=0x7feaa8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:27.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.691+0000 7fead1600640 1 --2- 172.21.15.161:0/1157996002 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feaa80666c0 0x7feaa8068b80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7feab40ab000 tx=0x7feab40a7070 comp rx=0 tx=0).stop 2024-05-22T21:24:27.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.691+0000 7fead1600640 1 -- 172.21.15.161:0/1157996002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab00047d0 msgr2=0x7feacc0b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:27.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.691+0000 7fead1600640 1 --2- 172.21.15.161:0/1157996002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab00047d0 0x7feacc0b0190 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7feabc00f5f0 tx=0x7feabc00fab0 comp rx=0 tx=0).stop 2024-05-22T21:24:27.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.691+0000 7fead1600640 1 -- 172.21.15.161:0/1157996002 shutdown_connections 2024-05-22T21:24:27.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.691+0000 7fead1600640 1 --2- 172.21.15.161:0/1157996002 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7feaa80666c0 0x7feaa8068b80 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:27.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.691+0000 7fead1600640 1 --2- 172.21.15.161:0/1157996002 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feacc0ae7f0 0x7feacc13f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:27.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.691+0000 7fead1600640 1 --2- 172.21.15.161:0/1157996002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feacc15a980 0x7feacc14d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:27.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.691+0000 7fead1600640 1 --2- 172.21.15.161:0/1157996002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab00047d0 0x7feacc0b0190 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:27.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.691+0000 7fead1600640 1 -- 172.21.15.161:0/1157996002 >> 172.21.15.161:0/1157996002 conn(0x7feacc06a4d0 msgr2=0x7feacc070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:27.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.691+0000 7fead1600640 1 -- 172.21.15.161:0/1157996002 shutdown_connections 2024-05-22T21:24:27.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.691+0000 7fead1600640 1 -- 172.21.15.161:0/1157996002 wait complete. 2024-05-22T21:24:27.712 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-22T21:24:27.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.851+0000 7fa879800640 1 Processor -- start 2024-05-22T21:24:27.853 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.851+0000 7fa879800640 1 -- start start 2024-05-22T21:24:27.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.855+0000 7fa879800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8741402e0 0x7fa87413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:27.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.855+0000 7fa879800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8741400d0 con 0x7fa87413cfc0 2024-05-22T21:24:27.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.855+0000 7fa879800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87413d3a0 con 0x7fa87413fc80 2024-05-22T21:24:27.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.855+0000 7fa879800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8740cd920 con 0x7fa8741402e0 2024-05-22T21:24:27.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.855+0000 7fa872a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa87413cfc0 0x7fa87415af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41840/0 (socket says 172.21.15.161:41840) 2024-05-22T21:24:27.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.855+0000 7fa872a00640 1 -- 172.21.15.161:0/2708680836 learned_addr learned my addr 172.21.15.161:0/2708680836 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:27.855 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.855+0000 7fa86a000640 1 --2- 172.21.15.161:0/2708680836 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8741402e0 0x7fa87413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:27.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.855+0000 7fa872000640 1 -- 172.21.15.161:0/2708680836 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828603155 0 0) 0x7fa8640014a0 con 0x7fa87413cfc0 2024-05-22T21:24:27.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.855+0000 7fa872000640 1 -- 172.21.15.161:0/2708680836 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa858005740 con 0x7fa87413cfc0 2024-05-22T21:24:27.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.855+0000 7fa872000640 1 -- 172.21.15.161:0/2708680836 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585419282 0 0) 0x7fa87413d3a0 con 0x7fa87413fc80 2024-05-22T21:24:27.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.855+0000 7fa872000640 1 -- 172.21.15.161:0/2708680836 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8640014a0 con 0x7fa87413fc80 2024-05-22T21:24:27.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.855+0000 7fa872000640 1 -- 172.21.15.161:0/2708680836 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2850805522 0 0) 0x7fa858005740 con 0x7fa87413cfc0 2024-05-22T21:24:27.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.855+0000 7fa86a000640 1 -- 172.21.15.161:0/2708680836 >> v1:172.21.15.179:6789/0 conn(0x7fa87413fc80 legacy=0x7fa87406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:27.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.855+0000 7fa86a000640 1 -- 172.21.15.161:0/2708680836 >> v1:172.21.15.161:6789/0 conn(0x7fa87413cfc0 legacy=0x7fa87415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:27.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.855+0000 7fa86a000640 1 -- 172.21.15.161:0/2708680836 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8740cdc00 con 0x7fa8741402e0 2024-05-22T21:24:27.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.859+0000 7fa86a000640 1 --2- 172.21.15.161:0/2708680836 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8741402e0 0x7fa87413f740 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0097f0 tx=0x7fa85c01c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a6c65765b42bd93 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:27.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.859+0000 7fa872000640 1 -- 172.21.15.161:0/2708680836 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864001b90 con 0x7fa8741402e0 2024-05-22T21:24:27.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.859+0000 7fa872000640 1 -- 172.21.15.161:0/2708680836 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c0a7050 con 0x7fa8741402e0 2024-05-22T21:24:27.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.859+0000 7fa872000640 1 -- 172.21.15.161:0/2708680836 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c0ab400 con 0x7fa8741402e0 2024-05-22T21:24:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.859+0000 7fa879800640 1 -- 172.21.15.161:0/2708680836 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8741402e0 msgr2=0x7fa87413f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.859+0000 7fa879800640 1 --2- 172.21.15.161:0/2708680836 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8741402e0 0x7fa87413f740 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0097f0 tx=0x7fa85c01c340 comp rx=0 tx=0).stop 2024-05-22T21:24:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.859+0000 7fa879800640 1 -- 172.21.15.161:0/2708680836 shutdown_connections 2024-05-22T21:24:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.859+0000 7fa879800640 1 --2- 172.21.15.161:0/2708680836 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8741402e0 0x7fa87413f740 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.859+0000 7fa879800640 1 -- 172.21.15.161:0/2708680836 >> 172.21.15.161:0/2708680836 conn(0x7fa87406a2d0 msgr2=0x7fa87406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.859+0000 7fa879800640 1 -- 172.21.15.161:0/2708680836 shutdown_connections 2024-05-22T21:24:27.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.859+0000 7fa879800640 1 -- 172.21.15.161:0/2708680836 wait complete. 2024-05-22T21:24:27.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.859+0000 7fa879800640 1 Processor -- start 2024-05-22T21:24:27.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.859+0000 7fa879800640 1 -- start start 2024-05-22T21:24:27.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa879800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa87413cfc0 0x7fa8741578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:27.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa879800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa87413fc80 0x7fa87415b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:27.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa879800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8741402e0 0x7fa87415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:27.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa879800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87413d9c0 con 0x7fa8741402e0 2024-05-22T21:24:27.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa879800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa874157e10 con 0x7fa87413fc80 2024-05-22T21:24:27.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa879800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87415c520 con 0x7fa87413cfc0 2024-05-22T21:24:27.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa873400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8741402e0 0x7fa87415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:27.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa873400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8741402e0 0x7fa87415bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60500/0 (socket says 172.21.15.161:60500) 2024-05-22T21:24:27.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa873400640 1 -- 172.21.15.161:0/920633312 learned_addr learned my addr 172.21.15.161:0/920633312 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:27.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa872a00640 1 --2- 172.21.15.161:0/920633312 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa87413cfc0 0x7fa8741578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:27.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa86a000640 1 --2- 172.21.15.161:0/920633312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa87413fc80 0x7fa87415b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:27.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa873400640 1 -- 172.21.15.161:0/920633312 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa87413cfc0 msgr2=0x7fa8741578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:27.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa873400640 1 --2- 172.21.15.161:0/920633312 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa87413cfc0 0x7fa8741578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:27.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa873400640 1 -- 172.21.15.161:0/920633312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa87413fc80 msgr2=0x7fa87415b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:27.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa873400640 1 --2- 172.21.15.161:0/920633312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa87413fc80 0x7fa87415b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:27.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa873400640 1 -- 172.21.15.161:0/920633312 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c009420 con 0x7fa8741402e0 2024-05-22T21:24:27.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa873400640 1 --2- 172.21.15.161:0/920633312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8741402e0 0x7fa87415bcf0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00c410 tx=0x7fa86c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:27.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa86b400640 1 -- 172.21.15.161:0/920633312 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0052d0 con 0x7fa8741402e0 2024-05-22T21:24:27.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa86b400640 1 -- 172.21.15.161:0/920633312 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c005430 con 0x7fa8741402e0 2024-05-22T21:24:27.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa879800640 1 -- 172.21.15.161:0/920633312 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87415c800 con 0x7fa8741402e0 2024-05-22T21:24:27.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa86b400640 1 -- 172.21.15.161:0/920633312 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c003800 con 0x7fa8741402e0 2024-05-22T21:24:27.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa879800640 1 -- 172.21.15.161:0/920633312 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87415cdc0 con 0x7fa8741402e0 2024-05-22T21:24:27.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.863+0000 7fa879800640 1 -- 172.21.15.161:0/920633312 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87413d790 con 0x7fa8741402e0 2024-05-22T21:24:27.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.867+0000 7fa86b400640 1 -- 172.21.15.161:0/920633312 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa86c003960 con 0x7fa8741402e0 2024-05-22T21:24:27.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.867+0000 7fa86b400640 1 --2- 172.21.15.161:0/920633312 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa850066690 0x7fa850068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:27.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.867+0000 7fa86b400640 1 -- 172.21.15.161:0/920633312 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa86c083320 con 0x7fa8741402e0 2024-05-22T21:24:27.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.867+0000 7fa872a00640 1 --2- 172.21.15.161:0/920633312 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa850066690 0x7fa850068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:27.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.867+0000 7fa872a00640 1 --2- 172.21.15.161:0/920633312 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa850066690 0x7fa850068b50 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa8640012f0 tx=0x7fa864009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:27.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:27.871+0000 7fa86b400640 1 -- 172.21.15.161:0/920633312 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa87413d790 con 0x7fa8741402e0 2024-05-22T21:24:28.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.023+0000 7fa879800640 1 -- 172.21.15.161:0/920633312 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa87413d790 con 0x7fa8741402e0 2024-05-22T21:24:28.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.027+0000 7fa86b400640 1 -- 172.21.15.161:0/920633312 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa87413d790 con 0x7fa8741402e0 2024-05-22T21:24:28.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.031+0000 7fa879800640 1 -- 172.21.15.161:0/920633312 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa850066690 msgr2=0x7fa850068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:28.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.031+0000 7fa879800640 1 --2- 172.21.15.161:0/920633312 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa850066690 0x7fa850068b50 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa8640012f0 tx=0x7fa864009040 comp rx=0 tx=0).stop 2024-05-22T21:24:28.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.031+0000 7fa879800640 1 -- 172.21.15.161:0/920633312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8741402e0 msgr2=0x7fa87415bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:28.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.031+0000 7fa879800640 1 --2- 172.21.15.161:0/920633312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8741402e0 0x7fa87415bcf0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00c410 tx=0x7fa86c00c8d0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.031+0000 7fa879800640 1 -- 172.21.15.161:0/920633312 shutdown_connections 2024-05-22T21:24:28.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.031+0000 7fa879800640 1 --2- 172.21.15.161:0/920633312 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa850066690 0x7fa850068b50 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.031+0000 7fa879800640 1 --2- 172.21.15.161:0/920633312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8741402e0 0x7fa87415bcf0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.031+0000 7fa879800640 1 --2- 172.21.15.161:0/920633312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa87413fc80 0x7fa87415b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.031+0000 7fa879800640 1 --2- 172.21.15.161:0/920633312 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa87413cfc0 0x7fa8741578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.031+0000 7fa879800640 1 -- 172.21.15.161:0/920633312 >> 172.21.15.161:0/920633312 conn(0x7fa87406a2d0 msgr2=0x7fa87406a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:28.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.031+0000 7fa879800640 1 -- 172.21.15.161:0/920633312 shutdown_connections 2024-05-22T21:24:28.031 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.031+0000 7fa879800640 1 -- 172.21.15.161:0/920633312 wait complete. 2024-05-22T21:24:28.049 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:24:28.049 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:24:28.132 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:24:28.220 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:24:28.220 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:24:28.221 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:24:28.229 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:24:28.229 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:24:28.229 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:24:28.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:24:28.239 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:24:28.239 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:24:28.281 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:24:28.281 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:24:28.281 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:24:28.290 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:24:28.290 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-22T21:24:28.290 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:24:28.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.455+0000 7fc453e00640 1 Processor -- start 2024-05-22T21:24:28.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.455+0000 7fc453e00640 1 -- start start 2024-05-22T21:24:28.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.455+0000 7fc453e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc44c143ec0 0x7fc44c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:28.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.455+0000 7fc453e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc44c0715a0 0x7fc44c06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:28.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.455+0000 7fc453e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c1442a0 con 0x7fc44c15ec90 2024-05-22T21:24:28.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.455+0000 7fc453e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44c070260 con 0x7fc44c143ec0 2024-05-22T21:24:28.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.455+0000 7fc453e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c070500 con 0x7fc44c0715a0 2024-05-22T21:24:28.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.459+0000 7fc451400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc44c143ec0 0x7fc44c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:28.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.459+0000 7fc450a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc44c0715a0 0x7fc44c06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:28.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.459+0000 7fc451e00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fc44c15ec90 0x7fc44c06bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:46284/0 (socket says 172.21.15.161:46284) 2024-05-22T21:24:28.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.459+0000 7fc451e00640 1 -- 172.21.15.161:0/4084227751 learned_addr learned my addr 172.21.15.161:0/4084227751 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:28.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.459+0000 7fc451400640 1 -- 172.21.15.161:0/4084227751 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc44c0715a0 msgr2=0x7fc44c06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:28.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.459+0000 7fc451400640 1 --2- 172.21.15.161:0/4084227751 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc44c0715a0 0x7fc44c06fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.459+0000 7fc451400640 1 -- 172.21.15.161:0/4084227751 >> v1:172.21.15.179:6789/0 conn(0x7fc44c15ec90 legacy=0x7fc44c06bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:28.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.459+0000 7fc451400640 1 -- 172.21.15.161:0/4084227751 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c0707e0 con 0x7fc44c143ec0 2024-05-22T21:24:28.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.459+0000 7fc450a00640 1 --2- 172.21.15.161:0/4084227751 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc44c0715a0 0x7fc44c06fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:24:28.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.459+0000 7fc451400640 1 --2- 172.21.15.161:0/4084227751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc44c143ec0 0x7fc44c15f830 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc4400097c0 tx=0x7fc44001c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e262de0e08e5aa server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:28.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc44be00640 1 -- 172.21.15.161:0/4084227751 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4400ab600 con 0x7fc44c143ec0 2024-05-22T21:24:28.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc44be00640 1 -- 172.21.15.161:0/4084227751 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4400a7050 con 0x7fc44c143ec0 2024-05-22T21:24:28.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc44be00640 1 -- 172.21.15.161:0/4084227751 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4400ab980 con 0x7fc44c143ec0 2024-05-22T21:24:28.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc453e00640 1 -- 172.21.15.161:0/4084227751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc44c143ec0 msgr2=0x7fc44c15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:28.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc453e00640 1 --2- 172.21.15.161:0/4084227751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc44c143ec0 0x7fc44c15f830 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc4400097c0 tx=0x7fc44001c330 comp rx=0 tx=0).stop 2024-05-22T21:24:28.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc453e00640 1 -- 172.21.15.161:0/4084227751 shutdown_connections 2024-05-22T21:24:28.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc453e00640 1 --2- 172.21.15.161:0/4084227751 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc44c0715a0 0x7fc44c06fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc453e00640 1 --2- 172.21.15.161:0/4084227751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc44c143ec0 0x7fc44c15f830 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc453e00640 1 -- 172.21.15.161:0/4084227751 >> 172.21.15.161:0/4084227751 conn(0x7fc44c0b0bc0 msgr2=0x7fc44c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:28.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc453e00640 1 -- 172.21.15.161:0/4084227751 shutdown_connections 2024-05-22T21:24:28.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc453e00640 1 -- 172.21.15.161:0/4084227751 wait complete. 2024-05-22T21:24:28.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc453e00640 1 Processor -- start 2024-05-22T21:24:28.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc453e00640 1 -- start start 2024-05-22T21:24:28.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc453e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc44c143ec0 0x7fc44c131680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:28.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc453e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc44c15ec90 0x7fc44c0c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:28.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc453e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc44c1302f0 0x7fc44c0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:28.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc453e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c137270 con 0x7fc44c143ec0 2024-05-22T21:24:28.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc453e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c1329a0 con 0x7fc44c1302f0 2024-05-22T21:24:28.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc453e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0c46c0 con 0x7fc44c15ec90 2024-05-22T21:24:28.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc451400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc44c143ec0 0x7fc44c131680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:28.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc451400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc44c143ec0 0x7fc44c131680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60516/0 (socket says 172.21.15.161:60516) 2024-05-22T21:24:28.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.463+0000 7fc451400640 1 -- 172.21.15.161:0/908414985 learned_addr learned my addr 172.21.15.161:0/908414985 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:28.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.467+0000 7fc450a00640 1 --2- 172.21.15.161:0/908414985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc44c15ec90 0x7fc44c0c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:28.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.467+0000 7fc451e00640 1 --2- 172.21.15.161:0/908414985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc44c1302f0 0x7fc44c0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:28.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.467+0000 7fc451400640 1 -- 172.21.15.161:0/908414985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc44c15ec90 msgr2=0x7fc44c0c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:28.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.467+0000 7fc451400640 1 --2- 172.21.15.161:0/908414985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc44c15ec90 0x7fc44c0c4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.467+0000 7fc451400640 1 -- 172.21.15.161:0/908414985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc44c1302f0 msgr2=0x7fc44c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:28.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.467+0000 7fc451400640 1 --2- 172.21.15.161:0/908414985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc44c1302f0 0x7fc44c0ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.467+0000 7fc451400640 1 -- 172.21.15.161:0/908414985 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc438009420 con 0x7fc44c143ec0 2024-05-22T21:24:28.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.467+0000 7fc451400640 1 --2- 172.21.15.161:0/908414985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc44c143ec0 0x7fc44c131680 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fc44001a9e0 tx=0x7fc44001ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:28.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.467+0000 7fc449600640 1 -- 172.21.15.161:0/908414985 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4400b7860 con 0x7fc44c143ec0 2024-05-22T21:24:28.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.467+0000 7fc449600640 1 -- 172.21.15.161:0/908414985 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4400b79e0 con 0x7fc44c143ec0 2024-05-22T21:24:28.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.467+0000 7fc449600640 1 -- 172.21.15.161:0/908414985 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4400b7d90 con 0x7fc44c143ec0 2024-05-22T21:24:28.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.467+0000 7fc453e00640 1 -- 172.21.15.161:0/908414985 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc440009420 con 0x7fc44c143ec0 2024-05-22T21:24:28.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.467+0000 7fc453e00640 1 -- 172.21.15.161:0/908414985 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c132000 con 0x7fc44c143ec0 2024-05-22T21:24:28.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.467+0000 7fc453e00640 1 -- 172.21.15.161:0/908414985 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44c067220 con 0x7fc44c143ec0 2024-05-22T21:24:28.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.467+0000 7fc449600640 1 -- 172.21.15.161:0/908414985 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc4400b5400 con 0x7fc44c143ec0 2024-05-22T21:24:28.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.467+0000 7fc449600640 1 --2- 172.21.15.161:0/908414985 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc42c066690 0x7fc42c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:28.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.471+0000 7fc449600640 1 -- 172.21.15.161:0/908414985 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc44011eac0 con 0x7fc44c143ec0 2024-05-22T21:24:28.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.471+0000 7fc450a00640 1 --2- 172.21.15.161:0/908414985 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc42c066690 0x7fc42c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:28.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.471+0000 7fc450a00640 1 --2- 172.21.15.161:0/908414985 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc42c066690 0x7fc42c068b50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc44c133dc0 tx=0x7fc438009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:28.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.471+0000 7fc449600640 1 -- 172.21.15.161:0/908414985 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc4400ec2f0 con 0x7fc44c143ec0 2024-05-22T21:24:28.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.659+0000 7fc453e00640 1 -- 172.21.15.161:0/908414985 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc44c06dcc0 con 0x7fc44c143ec0 2024-05-22T21:24:28.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.659+0000 7fc449600640 1 -- 172.21.15.161:0/908414985 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+3538 (secure 0 0 0) 0x7fc4400b0df0 con 0x7fc44c143ec0 2024-05-22T21:24:28.660 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:28.660 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":80,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/2405126804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2405126804},{"type":"v1","addr":"172.21.15.179:6834","nonce":2405126804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":6962,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3984924056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3984924056},{"type":"v1","addr":"172.21.15.179:6835","nonce":3984924056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8391,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/368424851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":368424851},{"type":"v1","addr":"172.21.15.161:6836","nonce":368424851}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2127416220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2127416220},{"type":"v1","addr":"172.21.15.161:6837","nonce":2127416220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[]} 2024-05-22T21:24:28.661 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 80 2024-05-22T21:24:28.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.663+0000 7fc453e00640 1 -- 172.21.15.161:0/908414985 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc42c066690 msgr2=0x7fc42c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:28.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.663+0000 7fc453e00640 1 --2- 172.21.15.161:0/908414985 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc42c066690 0x7fc42c068b50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc44c133dc0 tx=0x7fc438009190 comp rx=0 tx=0).stop 2024-05-22T21:24:28.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.663+0000 7fc453e00640 1 -- 172.21.15.161:0/908414985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc44c143ec0 msgr2=0x7fc44c131680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:28.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.663+0000 7fc453e00640 1 --2- 172.21.15.161:0/908414985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc44c143ec0 0x7fc44c131680 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fc44001a9e0 tx=0x7fc44001ab70 comp rx=0 tx=0).stop 2024-05-22T21:24:28.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.663+0000 7fc453e00640 1 -- 172.21.15.161:0/908414985 shutdown_connections 2024-05-22T21:24:28.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.663+0000 7fc453e00640 1 --2- 172.21.15.161:0/908414985 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc42c066690 0x7fc42c068b50 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.663+0000 7fc453e00640 1 --2- 172.21.15.161:0/908414985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc44c1302f0 0x7fc44c0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.663+0000 7fc453e00640 1 --2- 172.21.15.161:0/908414985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc44c15ec90 0x7fc44c0c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.663+0000 7fc453e00640 1 --2- 172.21.15.161:0/908414985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc44c143ec0 0x7fc44c131680 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.663+0000 7fc453e00640 1 -- 172.21.15.161:0/908414985 >> 172.21.15.161:0/908414985 conn(0x7fc44c0b0bc0 msgr2=0x7fc44c0ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:28.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.663+0000 7fc453e00640 1 -- 172.21.15.161:0/908414985 shutdown_connections 2024-05-22T21:24:28.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.663+0000 7fc453e00640 1 -- 172.21.15.161:0/908414985 wait complete. 2024-05-22T21:24:28.682 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-22T21:24:28.682 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-22T21:24:28.683 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-22T21:24:28.683 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:24:28.683 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-22T21:24:28.683 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-22T21:24:28.683 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-22T21:24:28.683 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:24:28.683 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-22T21:24:28.683 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-22T21:24:28.684 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-22T21:24:28.684 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:24:28.684 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-22T21:24:28.684 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-22T21:24:28.684 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-22T21:24:28.684 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:24:28.695 INFO:tasks.ceph.mds.a:Stopped 2024-05-22T21:24:28.695 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-22T21:24:28.696 INFO:tasks.ceph.mds.c:Stopped 2024-05-22T21:24:28.696 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-22T21:24:28.699 INFO:tasks.ceph.mds.d:Stopped 2024-05-22T21:24:28.699 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-22T21:24:28.700 INFO:tasks.ceph.mds.b:Stopped 2024-05-22T21:24:28.700 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-22T21:24:28.740 INFO:tasks.ceph.mds.a:Started 2024-05-22T21:24:28.740 DEBUG:teuthology.parallel:result is None 2024-05-22T21:24:28.741 INFO:tasks.ceph.mds.c:Started 2024-05-22T21:24:28.742 DEBUG:teuthology.parallel:result is None 2024-05-22T21:24:28.745 INFO:tasks.ceph.mds.d:Started 2024-05-22T21:24:28.745 DEBUG:teuthology.parallel:result is None 2024-05-22T21:24:28.746 INFO:tasks.ceph.mds.b:Started 2024-05-22T21:24:28.746 DEBUG:teuthology.parallel:result is None 2024-05-22T21:24:28.747 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-22T21:24:28.836 INFO:tasks.ceph.mds.d.smithi179.stdout:starting mds.d at 2024-05-22T21:24:28.837 INFO:tasks.ceph.mds.b.smithi179.stdout:starting mds.b at 2024-05-22T21:24:28.840 INFO:tasks.ceph.mds.a.smithi161.stdout:starting mds.a at 2024-05-22T21:24:28.840 INFO:tasks.ceph.mds.c.smithi161.stdout:starting mds.c at 2024-05-22T21:24:28.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7af800640 1 Processor -- start 2024-05-22T21:24:28.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7af800640 1 -- start start 2024-05-22T21:24:28.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7af800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7a815b300 0x7ff7a80b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:28.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7af800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff7a815ece0 0x7ff7a815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:28.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7af800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a80ccd80 con 0x7ff7a80cc920 2024-05-22T21:24:28.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7af800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a815f6d0 con 0x7ff7a815ece0 2024-05-22T21:24:28.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7af800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a80cfd30 con 0x7ff7a815b300 2024-05-22T21:24:28.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7ace00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7a815b300 0x7ff7a80b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:28.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7ad800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff7a815ece0 0x7ff7a815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:28.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7ace00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7a815b300 0x7ff7a80b35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:52704/0 (socket says 172.21.15.161:52704) 2024-05-22T21:24:28.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7ace00640 1 -- 172.21.15.161:0/264850348 learned_addr learned my addr 172.21.15.161:0/264850348 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:28.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7ad800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff7a815ece0 0x7ff7a815f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:51142/0 (socket says 172.21.15.161:51142) 2024-05-22T21:24:28.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7ad800640 1 -- 172.21.15.161:0/264850348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7a815b300 msgr2=0x7ff7a80b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:28.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7ad800640 1 --2- 172.21.15.161:0/264850348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7a815b300 0x7ff7a80b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7ad800640 1 -- 172.21.15.161:0/264850348 >> v1:172.21.15.179:6789/0 conn(0x7ff7a80cc920 legacy=0x7ff7a80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:28.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7ad800640 1 -- 172.21.15.161:0/264850348 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a80cbb60 con 0x7ff7a815ece0 2024-05-22T21:24:28.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7a7e00640 1 -- 172.21.15.161:0/264850348 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599369391 0 0) 0x7ff7a80ccd80 con 0x7ff7a80cc920 2024-05-22T21:24:28.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.895+0000 7ff7ace00640 1 --2- 172.21.15.161:0/264850348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7a815b300 0x7ff7a80b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:24:28.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.899+0000 7ff7ad800640 1 --2- 172.21.15.161:0/264850348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7a815ece0 0x7ff7a815f190 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009d90 tx=0x7ff7a001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=699e3586fd21429b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:28.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.899+0000 7ff7a7e00640 1 -- 172.21.15.161:0/264850348 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00a5a00 con 0x7ff7a815ece0 2024-05-22T21:24:28.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.899+0000 7ff7a7e00640 1 -- 172.21.15.161:0/264850348 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a00a5b60 con 0x7ff7a815ece0 2024-05-22T21:24:28.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.899+0000 7ff7a7e00640 1 -- 172.21.15.161:0/264850348 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00ad440 con 0x7ff7a815ece0 2024-05-22T21:24:28.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.899+0000 7ff7af800640 1 -- 172.21.15.161:0/264850348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7a815ece0 msgr2=0x7ff7a815f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:28.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.899+0000 7ff7af800640 1 --2- 172.21.15.161:0/264850348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7a815ece0 0x7ff7a815f190 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009d90 tx=0x7ff7a001c940 comp rx=0 tx=0).stop 2024-05-22T21:24:28.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.899+0000 7ff7af800640 1 -- 172.21.15.161:0/264850348 shutdown_connections 2024-05-22T21:24:28.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.899+0000 7ff7af800640 1 --2- 172.21.15.161:0/264850348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7a815ece0 0x7ff7a815f190 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.899+0000 7ff7af800640 1 --2- 172.21.15.161:0/264850348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7a815b300 0x7ff7a80b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.899+0000 7ff7af800640 1 -- 172.21.15.161:0/264850348 >> 172.21.15.161:0/264850348 conn(0x7ff7a80b2a60 msgr2=0x7ff7a80b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:28.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.899+0000 7ff7af800640 1 -- 172.21.15.161:0/264850348 shutdown_connections 2024-05-22T21:24:28.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.899+0000 7ff7af800640 1 -- 172.21.15.161:0/264850348 wait complete. 2024-05-22T21:24:28.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7af800640 1 Processor -- start 2024-05-22T21:24:28.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7af800640 1 -- start start 2024-05-22T21:24:28.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7af800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7a80cc920 0x7ff7a80c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:28.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7af800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7a815b300 0x7ff7a815de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:28.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7af800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7a815ece0 0x7ff7a8154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:28.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7af800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a80c62e0 con 0x7ff7a80cc920 2024-05-22T21:24:28.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7af800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a80c65c0 con 0x7ff7a815b300 2024-05-22T21:24:28.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7af800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a80c68a0 con 0x7ff7a815ece0 2024-05-22T21:24:28.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7ace00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7a80cc920 0x7ff7a80c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:28.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7ace00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7a80cc920 0x7ff7a80c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60558/0 (socket says 172.21.15.161:60558) 2024-05-22T21:24:28.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7ace00640 1 -- 172.21.15.161:0/1941194797 learned_addr learned my addr 172.21.15.161:0/1941194797 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:28.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff79fe00640 1 --2- 172.21.15.161:0/1941194797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7a815b300 0x7ff7a815de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:28.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7ad800640 1 --2- 172.21.15.161:0/1941194797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7a815ece0 0x7ff7a8154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:28.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7ace00640 1 -- 172.21.15.161:0/1941194797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7a815ece0 msgr2=0x7ff7a8154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:28.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7ace00640 1 --2- 172.21.15.161:0/1941194797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7a815ece0 0x7ff7a8154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7ace00640 1 -- 172.21.15.161:0/1941194797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7a815b300 msgr2=0x7ff7a815de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:28.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7ace00640 1 --2- 172.21.15.161:0/1941194797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7a815b300 0x7ff7a815de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:28.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7ace00640 1 -- 172.21.15.161:0/1941194797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7980094d0 con 0x7ff7a80cc920 2024-05-22T21:24:28.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff79fe00640 1 --2- 172.21.15.161:0/1941194797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7a815b300 0x7ff7a815de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:24:28.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7ace00640 1 --2- 172.21.15.161:0/1941194797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7a80cc920 0x7ff7a80c7380 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ff79800e930 tx=0x7ff79800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:28.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7a5600640 1 -- 172.21.15.161:0/1941194797 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79800ca00 con 0x7ff7a80cc920 2024-05-22T21:24:28.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7a5600640 1 -- 172.21.15.161:0/1941194797 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79800cb60 con 0x7ff7a80cc920 2024-05-22T21:24:28.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7a5600640 1 -- 172.21.15.161:0/1941194797 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798013470 con 0x7ff7a80cc920 2024-05-22T21:24:28.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7af800640 1 -- 172.21.15.161:0/1941194797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a0009a70 con 0x7ff7a80cc920 2024-05-22T21:24:28.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7af800640 1 -- 172.21.15.161:0/1941194797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a80c6e60 con 0x7ff7a80cc920 2024-05-22T21:24:28.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.903+0000 7ff7af800640 1 -- 172.21.15.161:0/1941194797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a8073ce0 con 0x7ff7a80cc920 2024-05-22T21:24:28.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.907+0000 7ff7a5600640 1 -- 172.21.15.161:0/1941194797 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff7980135d0 con 0x7ff7a80cc920 2024-05-22T21:24:28.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.907+0000 7ff7a5600640 1 --2- 172.21.15.161:0/1941194797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff784066690 0x7ff784068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:28.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.907+0000 7ff7a5600640 1 -- 172.21.15.161:0/1941194797 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff798084120 con 0x7ff7a80cc920 2024-05-22T21:24:28.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.907+0000 7ff79fe00640 1 --2- 172.21.15.161:0/1941194797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff784066690 0x7ff784068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:28.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.907+0000 7ff79fe00640 1 --2- 172.21.15.161:0/1941194797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff784066690 0x7ff784068b50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80cb750 tx=0x7ff790009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:28.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:28.911+0000 7ff7a5600640 1 -- 172.21.15.161:0/1941194797 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff7980517a0 con 0x7ff7a80cc920 2024-05-22T21:24:29.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:29.075+0000 7ff7af800640 1 -- 172.21.15.161:0/1941194797 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff7a8067220 con 0x7ff7a80cc920 2024-05-22T21:24:30.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.035+0000 7ff7a5600640 1 -- 172.21.15.161:0/1941194797 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v186) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff798054dc0 con 0x7ff7a80cc920 2024-05-22T21:24:30.035 INFO:teuthology.orchestra.run.smithi161.stderr: removed all blocklist entries 2024-05-22T21:24:30.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.035+0000 7ff7af800640 1 -- 172.21.15.161:0/1941194797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff784066690 msgr2=0x7ff784068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.035+0000 7ff7af800640 1 --2- 172.21.15.161:0/1941194797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff784066690 0x7ff784068b50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80cb750 tx=0x7ff790009040 comp rx=0 tx=0).stop 2024-05-22T21:24:30.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.035+0000 7ff7af800640 1 -- 172.21.15.161:0/1941194797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7a80cc920 msgr2=0x7ff7a80c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.035+0000 7ff7af800640 1 --2- 172.21.15.161:0/1941194797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7a80cc920 0x7ff7a80c7380 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ff79800e930 tx=0x7ff79800edf0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.035+0000 7ff7af800640 1 -- 172.21.15.161:0/1941194797 shutdown_connections 2024-05-22T21:24:30.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.035+0000 7ff7af800640 1 --2- 172.21.15.161:0/1941194797 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff784066690 0x7ff784068b50 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.035+0000 7ff7af800640 1 --2- 172.21.15.161:0/1941194797 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7a815ece0 0x7ff7a8154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.035+0000 7ff7af800640 1 --2- 172.21.15.161:0/1941194797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7a815b300 0x7ff7a815de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.035+0000 7ff7af800640 1 --2- 172.21.15.161:0/1941194797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7a80cc920 0x7ff7a80c7380 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.035+0000 7ff7af800640 1 -- 172.21.15.161:0/1941194797 >> 172.21.15.161:0/1941194797 conn(0x7ff7a80b2a60 msgr2=0x7ff7a809a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:30.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.035+0000 7ff7af800640 1 -- 172.21.15.161:0/1941194797 shutdown_connections 2024-05-22T21:24:30.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.035+0000 7ff7af800640 1 -- 172.21.15.161:0/1941194797 wait complete. 2024-05-22T21:24:30.061 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-22T21:24:30.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.199+0000 7f14ae200640 1 Processor -- start 2024-05-22T21:24:30.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.199+0000 7f14ae200640 1 -- start start 2024-05-22T21:24:30.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.199+0000 7f14ae200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f14a80715a0 0x7f14a815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.199+0000 7f14ae200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a81442a0 con 0x7f14a815ec90 2024-05-22T21:24:30.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.199+0000 7f14ae200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a8070500 con 0x7f14a8143ec0 2024-05-22T21:24:30.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.199+0000 7f14ae200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a80707e0 con 0x7f14a80715a0 2024-05-22T21:24:30.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.199+0000 7f149fe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f14a80715a0 0x7f14a815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.199+0000 7f14a7e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f14a815ec90 0x7f14a80cd710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41872/0 (socket says 172.21.15.161:41872) 2024-05-22T21:24:30.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.203+0000 7f14a7e00640 1 -- 172.21.15.161:0/1332549660 learned_addr learned my addr 172.21.15.161:0/1332549660 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:30.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.203+0000 7f149fe00640 1 -- 172.21.15.161:0/1332549660 >> v1:172.21.15.179:6789/0 conn(0x7f14a8143ec0 legacy=0x7f14a806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.203+0000 7f149fe00640 1 -- 172.21.15.161:0/1332549660 >> v1:172.21.15.161:6789/0 conn(0x7f14a815ec90 legacy=0x7f14a80cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.203+0000 7f149fe00640 1 -- 172.21.15.161:0/1332549660 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a8070ac0 con 0x7f14a80715a0 2024-05-22T21:24:30.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.203+0000 7f14a6a00640 1 -- 172.21.15.161:0/1332549660 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942630694 0 0) 0x7f14a81442a0 con 0x7f14a815ec90 2024-05-22T21:24:30.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.203+0000 7f149fe00640 1 --2- 172.21.15.161:0/1332549660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a80715a0 0x7f14a815f830 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f14900097c0 tx=0x7f149001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66192702623cfd8c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:30.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.203+0000 7f14a6a00640 1 -- 172.21.15.161:0/1332549660 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14900ab670 con 0x7f14a80715a0 2024-05-22T21:24:30.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.203+0000 7f14a6a00640 1 -- 172.21.15.161:0/1332549660 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14900ab7d0 con 0x7f14a80715a0 2024-05-22T21:24:30.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.203+0000 7f14a6a00640 1 -- 172.21.15.161:0/1332549660 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14900abb60 con 0x7f14a80715a0 2024-05-22T21:24:30.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14ae200640 1 -- 172.21.15.161:0/1332549660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a80715a0 msgr2=0x7f14a815f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14ae200640 1 --2- 172.21.15.161:0/1332549660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a80715a0 0x7f14a815f830 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f14900097c0 tx=0x7f149001c470 comp rx=0 tx=0).stop 2024-05-22T21:24:30.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14ae200640 1 -- 172.21.15.161:0/1332549660 shutdown_connections 2024-05-22T21:24:30.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14ae200640 1 --2- 172.21.15.161:0/1332549660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a80715a0 0x7f14a815f830 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14ae200640 1 -- 172.21.15.161:0/1332549660 >> 172.21.15.161:0/1332549660 conn(0x7f14a80b0bc0 msgr2=0x7f14a80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:30.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14ae200640 1 -- 172.21.15.161:0/1332549660 shutdown_connections 2024-05-22T21:24:30.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14ae200640 1 -- 172.21.15.161:0/1332549660 wait complete. 2024-05-22T21:24:30.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14ae200640 1 Processor -- start 2024-05-22T21:24:30.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14ae200640 1 -- start start 2024-05-22T21:24:30.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14ae200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a8143ec0 0x7f14a8159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14ae200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14a815ec90 0x7f14a815b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14ae200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14a815a100 0x7f14a80cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14ae200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a813f760 con 0x7f14a8143ec0 2024-05-22T21:24:30.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14ae200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a8159e50 con 0x7f14a815ec90 2024-05-22T21:24:30.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14ae200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a813ee30 con 0x7f14a815a100 2024-05-22T21:24:30.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14a7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a8143ec0 0x7f14a8159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14a7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a8143ec0 0x7f14a8159910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60576/0 (socket says 172.21.15.161:60576) 2024-05-22T21:24:30.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14a7400640 1 -- 172.21.15.161:0/4071510296 learned_addr learned my addr 172.21.15.161:0/4071510296 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:30.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f149fe00640 1 --2- 172.21.15.161:0/4071510296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14a815ec90 0x7f14a815b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14a7e00640 1 --2- 172.21.15.161:0/4071510296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14a815a100 0x7f14a80cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14a7400640 1 -- 172.21.15.161:0/4071510296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14a815a100 msgr2=0x7f14a80cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14a7400640 1 --2- 172.21.15.161:0/4071510296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14a815a100 0x7f14a80cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14a7400640 1 -- 172.21.15.161:0/4071510296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14a815ec90 msgr2=0x7f14a815b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14a7400640 1 --2- 172.21.15.161:0/4071510296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14a815ec90 0x7f14a815b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14a7400640 1 -- 172.21.15.161:0/4071510296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1490009420 con 0x7f14a8143ec0 2024-05-22T21:24:30.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14a7e00640 1 --2- 172.21.15.161:0/4071510296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14a815a100 0x7f14a80cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:30.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f14a7400640 1 --2- 172.21.15.161:0/4071510296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a8143ec0 0x7f14a8159910 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f149800c530 tx=0x7f149800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:30.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.207+0000 7f149f400640 1 -- 172.21.15.161:0/4071510296 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498005220 con 0x7f14a8143ec0 2024-05-22T21:24:30.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.211+0000 7f149f400640 1 -- 172.21.15.161:0/4071510296 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1498005380 con 0x7f14a8143ec0 2024-05-22T21:24:30.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.211+0000 7f149f400640 1 -- 172.21.15.161:0/4071510296 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498007270 con 0x7f14a8143ec0 2024-05-22T21:24:30.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.211+0000 7f14ae200640 1 -- 172.21.15.161:0/4071510296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a813f110 con 0x7f14a8143ec0 2024-05-22T21:24:30.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.211+0000 7f14ae200640 1 -- 172.21.15.161:0/4071510296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a815c0f0 con 0x7f14a8143ec0 2024-05-22T21:24:30.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.211+0000 7f14ae200640 1 -- 172.21.15.161:0/4071510296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a8067220 con 0x7f14a8143ec0 2024-05-22T21:24:30.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.211+0000 7f149f400640 1 -- 172.21.15.161:0/4071510296 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f14980073d0 con 0x7f14a8143ec0 2024-05-22T21:24:30.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.211+0000 7f149f400640 1 --2- 172.21.15.161:0/4071510296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1484066690 0x7f1484068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.211+0000 7f149f400640 1 -- 172.21.15.161:0/4071510296 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1498083700 con 0x7f14a8143ec0 2024-05-22T21:24:30.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.211+0000 7f149fe00640 1 --2- 172.21.15.161:0/4071510296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1484066690 0x7f1484068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.211+0000 7f149fe00640 1 --2- 172.21.15.161:0/4071510296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1484066690 0x7f1484068b50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f14900af000 tx=0x7f14900a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:30.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.215+0000 7f149f400640 1 -- 172.21.15.161:0/4071510296 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1498050db0 con 0x7f14a8143ec0 2024-05-22T21:24:30.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.423+0000 7f14ae200640 1 -- 172.21.15.161:0/4071510296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f14a806dcc0 con 0x7f14a8143ec0 2024-05-22T21:24:30.423 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.423+0000 7f149f400640 1 -- 172.21.15.161:0/4071510296 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f14980543d0 con 0x7f14a8143ec0 2024-05-22T21:24:30.424 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:30.424 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:24:30.424 INFO:teuthology.orchestra.run.smithi161.stdout: "auth_dump": [ 2024-05-22T21:24:30.424 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.424 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.a", 2024-05-22T21:24:30.424 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m00QTFRAAPSRrpBRY75FlcWf3QxOOfg==", 2024-05-22T21:24:30.424 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.424 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:24:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:24:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:24:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:24:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.b", 2024-05-22T21:24:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05m/ipAIRAA4gj5jGqTpNCr1K2OU/bsAQ==", 2024-05-22T21:24:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:24:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:24:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:24:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:24:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.c", 2024-05-22T21:24:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mWlM2HBAAEmJJWMDlTG9E7Qzld2gd5w==", 2024-05-22T21:24:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:24:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:24:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:24:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:24:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mds.d", 2024-05-22T21:24:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mmsuaKBAAqpCYRZZUGK5ZuxflUBcPgg==", 2024-05-22T21:24:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:24:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:24:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:24:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:24:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.0", 2024-05-22T21:24:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05mJZokLhAAh2vbQnRX25iYUIbvVmURuQ==", 2024-05-22T21:24:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:24:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:24:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:24:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.1", 2024-05-22T21:24:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDaX05m8kWNNhAA6aaVT2xshFkqnk4G4gOttA==", 2024-05-22T21:24:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:24:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:24:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:24:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.2", 2024-05-22T21:24:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05ml1k9AxAAtL3XaJJI6cilpmvurh18Pw==", 2024-05-22T21:24:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:24:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:24:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:24:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.3", 2024-05-22T21:24:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDbX05mbNBgCxAAas1L+QR4Aez+H1G0EVltfg==", 2024-05-22T21:24:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:24:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:24:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:24:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.4", 2024-05-22T21:24:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mOPLPABAAZBGIMF/FasNB27n/7di0AA==", 2024-05-22T21:24:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:24:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:24:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:24:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.5", 2024-05-22T21:24:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mT1poCRAASXm4tdP01pA1VOviqeoLVw==", 2024-05-22T21:24:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:24:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:24:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:24:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.6", 2024-05-22T21:24:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05m1ZPRERAA2S7WeTwysLCZm1OB/qCrAA==", 2024-05-22T21:24:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:24:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:24:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:24:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "osd.7", 2024-05-22T21:24:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDeX05mCrUhGhAAvXoFWV7X3303rfXaxmfWqA==", 2024-05-22T21:24:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow profile osd", 2024-05-22T21:24:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile osd", 2024-05-22T21:24:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:24:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.0", 2024-05-22T21:24:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mh9B7LBAAevhvj5KpwP9pEIpJcxUvFA==", 2024-05-22T21:24:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:24:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow r", 2024-05-22T21:24:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-22T21:24:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.1", 2024-05-22T21:24:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mMnHtNRAAg/a/Mb1oRqx73d9efIHCfA==", 2024-05-22T21:24:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:24:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:24:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:24:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:24:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.435 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.435 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.2", 2024-05-22T21:24:30.435 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mOzdkMhAAbTiNdvyxqIyrluMiZW0cOw==", 2024-05-22T21:24:30.435 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.435 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:24:30.435 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:24:30.435 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:24:30.435 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:24:30.435 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.3", 2024-05-22T21:24:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDYX05mr4oxABAA1+bQQbxhoeU+kb0OCa+Wyw==", 2024-05-22T21:24:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow", 2024-05-22T21:24:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow r", 2024-05-22T21:24:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow rw", 2024-05-22T21:24:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow rwx" 2024-05-22T21:24:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "client.admin", 2024-05-22T21:24:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDWX05mHZY0NRAAiBvMDkrCZ+MgG2iwBH+bcg==", 2024-05-22T21:24:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:24:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": "allow *", 2024-05-22T21:24:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow *", 2024-05-22T21:24:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:24:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.x", 2024-05-22T21:24:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mH2kuERAAgigMUWK3M3WaMXlqGUcNrw==", 2024-05-22T21:24:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:24:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:24:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:24:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: "entity": "mgr.y", 2024-05-22T21:24:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: "key": "AQDXX05mZRdiDRAA0vUwD+rPSfyhjgA+9XPp5w==", 2024-05-22T21:24:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": { 2024-05-22T21:24:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": "allow *", 2024-05-22T21:24:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: "mon": "allow profile mgr", 2024-05-22T21:24:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": "allow *" 2024-05-22T21:24:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.439 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:30.439 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:30.439 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:24:30.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.423+0000 7f14ae200640 1 -- 172.21.15.161:0/4071510296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1484066690 msgr2=0x7f1484068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.423+0000 7f14ae200640 1 --2- 172.21.15.161:0/4071510296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1484066690 0x7f1484068b50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f14900af000 tx=0x7f14900a7000 comp rx=0 tx=0).stop 2024-05-22T21:24:30.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.423+0000 7f14ae200640 1 -- 172.21.15.161:0/4071510296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a8143ec0 msgr2=0x7f14a8159910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.423+0000 7f14ae200640 1 --2- 172.21.15.161:0/4071510296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a8143ec0 0x7f14a8159910 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f149800c530 tx=0x7f149800c9f0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.423+0000 7f14ae200640 1 -- 172.21.15.161:0/4071510296 shutdown_connections 2024-05-22T21:24:30.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.423+0000 7f14ae200640 1 --2- 172.21.15.161:0/4071510296 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1484066690 0x7f1484068b50 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.423+0000 7f14ae200640 1 --2- 172.21.15.161:0/4071510296 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f14a815a100 0x7f14a80cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.423+0000 7f14ae200640 1 --2- 172.21.15.161:0/4071510296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14a815ec90 0x7f14a815b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.423+0000 7f14ae200640 1 --2- 172.21.15.161:0/4071510296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a8143ec0 0x7f14a8159910 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.427+0000 7f14ae200640 1 -- 172.21.15.161:0/4071510296 >> 172.21.15.161:0/4071510296 conn(0x7f14a80b0bc0 msgr2=0x7f14a815e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:30.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.427+0000 7f14ae200640 1 -- 172.21.15.161:0/4071510296 shutdown_connections 2024-05-22T21:24:30.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.427+0000 7f14ae200640 1 -- 172.21.15.161:0/4071510296 wait complete. 2024-05-22T21:24:30.445 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:24:30.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.579+0000 7fa4a2200640 1 Processor -- start 2024-05-22T21:24:30.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.579+0000 7fa4a2200640 1 -- start start 2024-05-22T21:24:30.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa4a2200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa49c143ec0 0x7fa49c15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa4a2200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa49c0715a0 0x7fa49c06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa4a2200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa49c15ec90 0x7fa49c15f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa4a2200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49c15f660 con 0x7fa49c15ec90 2024-05-22T21:24:30.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa4a2200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49c1442a0 con 0x7fa49c0715a0 2024-05-22T21:24:30.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa4a2200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c070260 con 0x7fa49c143ec0 2024-05-22T21:24:30.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa49be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa49c15ec90 0x7fa49c15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa49be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa49c15ec90 0x7fa49c15f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60578/0 (socket says 172.21.15.161:60578) 2024-05-22T21:24:30.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa49be00640 1 -- 172.21.15.161:0/3911289555 learned_addr learned my addr 172.21.15.161:0/3911289555 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:30.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa493e00640 1 --2- 172.21.15.161:0/3911289555 >> v2:172.21.15.179:3300/0 conn(0x7fa49c0715a0 0x7fa49c06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa49b400640 1 --2- 172.21.15.161:0/3911289555 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa49c143ec0 0x7fa49c15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa49be00640 1 -- 172.21.15.161:0/3911289555 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa49c143ec0 msgr2=0x7fa49c15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:30.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa49be00640 1 --2- 172.21.15.161:0/3911289555 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa49c143ec0 0x7fa49c15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa49be00640 1 -- 172.21.15.161:0/3911289555 >> v2:172.21.15.179:3300/0 conn(0x7fa49c0715a0 msgr2=0x7fa49c06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:30.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa49be00640 1 --2- 172.21.15.161:0/3911289555 >> v2:172.21.15.179:3300/0 conn(0x7fa49c0715a0 0x7fa49c06fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa49be00640 1 -- 172.21.15.161:0/3911289555 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c070500 con 0x7fa49c15ec90 2024-05-22T21:24:30.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.583+0000 7fa493e00640 1 --2- 172.21.15.161:0/3911289555 >> v2:172.21.15.179:3300/0 conn(0x7fa49c0715a0 0x7fa49c06fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:30.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.587+0000 7fa49be00640 1 --2- 172.21.15.161:0/3911289555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c15ec90 0x7fa49c15f120 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fa49c072510 tx=0x7fa49401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74f275dde08f255 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:30.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.587+0000 7fa49aa00640 1 -- 172.21.15.161:0/3911289555 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4940a5840 con 0x7fa49c15ec90 2024-05-22T21:24:30.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.587+0000 7fa49aa00640 1 -- 172.21.15.161:0/3911289555 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4940a59a0 con 0x7fa49c15ec90 2024-05-22T21:24:30.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.587+0000 7fa49aa00640 1 -- 172.21.15.161:0/3911289555 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4940a5d30 con 0x7fa49c15ec90 2024-05-22T21:24:30.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.587+0000 7fa4a2200640 1 -- 172.21.15.161:0/3911289555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c15ec90 msgr2=0x7fa49c15f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.587+0000 7fa4a2200640 1 --2- 172.21.15.161:0/3911289555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c15ec90 0x7fa49c15f120 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fa49c072510 tx=0x7fa49401c9c0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.587+0000 7fa4a2200640 1 -- 172.21.15.161:0/3911289555 shutdown_connections 2024-05-22T21:24:30.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.587+0000 7fa4a2200640 1 --2- 172.21.15.161:0/3911289555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c15ec90 0x7fa49c15f120 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.587+0000 7fa4a2200640 1 --2- 172.21.15.161:0/3911289555 >> v2:172.21.15.179:3300/0 conn(0x7fa49c0715a0 0x7fa49c06fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.587+0000 7fa4a2200640 1 --2- 172.21.15.161:0/3911289555 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa49c143ec0 0x7fa49c15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.587+0000 7fa4a2200640 1 -- 172.21.15.161:0/3911289555 >> 172.21.15.161:0/3911289555 conn(0x7fa49c0b0bc0 msgr2=0x7fa49c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:30.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.587+0000 7fa4a2200640 1 -- 172.21.15.161:0/3911289555 shutdown_connections 2024-05-22T21:24:30.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.587+0000 7fa4a2200640 1 -- 172.21.15.161:0/3911289555 wait complete. 2024-05-22T21:24:30.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.587+0000 7fa4a2200640 1 Processor -- start 2024-05-22T21:24:30.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.587+0000 7fa4a2200640 1 -- start start 2024-05-22T21:24:30.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa4a2200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa49c143ec0 0x7fa49c1550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa4a2200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c15ec90 0x7fa49c139740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa4a2200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa49c15c160 0x7fa49c15c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa4a2200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c15cb50 con 0x7fa49c15ec90 2024-05-22T21:24:30.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa4a2200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c156950 con 0x7fa49c143ec0 2024-05-22T21:24:30.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa4a2200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c155620 con 0x7fa49c15c160 2024-05-22T21:24:30.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa493e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c15ec90 0x7fa49c139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa493e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c15ec90 0x7fa49c139740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60584/0 (socket says 172.21.15.161:60584) 2024-05-22T21:24:30.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa493e00640 1 -- 172.21.15.161:0/3142651182 learned_addr learned my addr 172.21.15.161:0/3142651182 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:30.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa49b400640 1 --2- 172.21.15.161:0/3142651182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa49c143ec0 0x7fa49c1550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa49be00640 1 --2- 172.21.15.161:0/3142651182 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa49c15c160 0x7fa49c15c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa493e00640 1 -- 172.21.15.161:0/3142651182 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa49c15c160 msgr2=0x7fa49c15c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa493e00640 1 --2- 172.21.15.161:0/3142651182 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa49c15c160 0x7fa49c15c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa493e00640 1 -- 172.21.15.161:0/3142651182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa49c143ec0 msgr2=0x7fa49c1550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa493e00640 1 --2- 172.21.15.161:0/3142651182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa49c143ec0 0x7fa49c1550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa493e00640 1 -- 172.21.15.161:0/3142651182 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa494009b30 con 0x7fa49c15ec90 2024-05-22T21:24:30.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa493e00640 1 --2- 172.21.15.161:0/3142651182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c15ec90 0x7fa49c139740 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fa48400e810 tx=0x7fa48400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:30.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa493400640 1 -- 172.21.15.161:0/3142651182 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48400c870 con 0x7fa49c15ec90 2024-05-22T21:24:30.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa493400640 1 -- 172.21.15.161:0/3142651182 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48400c9d0 con 0x7fa49c15ec90 2024-05-22T21:24:30.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa493400640 1 -- 172.21.15.161:0/3142651182 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48400cd80 con 0x7fa49c15ec90 2024-05-22T21:24:30.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa4a2200640 1 -- 172.21.15.161:0/3142651182 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c135a00 con 0x7fa49c15ec90 2024-05-22T21:24:30.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa4a2200640 1 -- 172.21.15.161:0/3142651182 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c135ca0 con 0x7fa49c15ec90 2024-05-22T21:24:30.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa4a2200640 1 -- 172.21.15.161:0/3142651182 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49c067220 con 0x7fa49c15ec90 2024-05-22T21:24:30.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa493400640 1 -- 172.21.15.161:0/3142651182 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa484013410 con 0x7fa49c15ec90 2024-05-22T21:24:30.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.591+0000 7fa493400640 1 --2- 172.21.15.161:0/3142651182 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa4780666c0 0x7fa478068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.595+0000 7fa493400640 1 -- 172.21.15.161:0/3142651182 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa484083bc0 con 0x7fa49c15ec90 2024-05-22T21:24:30.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.595+0000 7fa49b400640 1 --2- 172.21.15.161:0/3142651182 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa4780666c0 0x7fa478068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.595+0000 7fa49b400640 1 --2- 172.21.15.161:0/3142651182 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa4780666c0 0x7fa478068b80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0079e0 tx=0x7fa48c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:30.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.599+0000 7fa493400640 1 -- 172.21.15.161:0/3142651182 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa484051290 con 0x7fa49c15ec90 2024-05-22T21:24:30.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.755+0000 7fa4a2200640 1 -- 172.21.15.161:0/3142651182 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa49c156a90 con 0x7fa49c15ec90 2024-05-22T21:24:30.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.755+0000 7fa493400640 1 -- 172.21.15.161:0/3142651182 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+12448 (secure 0 0 0) 0x7fa4840548b0 con 0x7fa49c15ec90 2024-05-22T21:24:30.755 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:30.755 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":186,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:24:30.033209+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:24:30.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.755+0000 7fa4a2200640 1 -- 172.21.15.161:0/3142651182 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa4780666c0 msgr2=0x7fa478068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.755+0000 7fa4a2200640 1 --2- 172.21.15.161:0/3142651182 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa4780666c0 0x7fa478068b80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0079e0 tx=0x7fa48c008040 comp rx=0 tx=0).stop 2024-05-22T21:24:30.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.755+0000 7fa4a2200640 1 -- 172.21.15.161:0/3142651182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c15ec90 msgr2=0x7fa49c139740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.755+0000 7fa4a2200640 1 --2- 172.21.15.161:0/3142651182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c15ec90 0x7fa49c139740 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fa48400e810 tx=0x7fa48400ecd0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.755+0000 7fa4a2200640 1 -- 172.21.15.161:0/3142651182 shutdown_connections 2024-05-22T21:24:30.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.755+0000 7fa4a2200640 1 --2- 172.21.15.161:0/3142651182 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa4780666c0 0x7fa478068b80 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.755+0000 7fa4a2200640 1 --2- 172.21.15.161:0/3142651182 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa49c15c160 0x7fa49c15c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.755+0000 7fa4a2200640 1 --2- 172.21.15.161:0/3142651182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa49c15ec90 0x7fa49c139740 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.755+0000 7fa4a2200640 1 --2- 172.21.15.161:0/3142651182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa49c143ec0 0x7fa49c1550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.755+0000 7fa4a2200640 1 -- 172.21.15.161:0/3142651182 >> 172.21.15.161:0/3142651182 conn(0x7fa49c0b0bc0 msgr2=0x7fa49c15e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:30.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.755+0000 7fa4a2200640 1 -- 172.21.15.161:0/3142651182 shutdown_connections 2024-05-22T21:24:30.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.759+0000 7fa4a2200640 1 -- 172.21.15.161:0/3142651182 wait complete. 2024-05-22T21:24:30.777 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:24:30.778 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:24:30.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.915+0000 7ff0da600640 1 Processor -- start 2024-05-22T21:24:30.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.915+0000 7ff0da600640 1 -- start start 2024-05-22T21:24:30.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.915+0000 7ff0da600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff0d413cfc0 0x7ff0d41402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.915+0000 7ff0da600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d413d3a0 con 0x7ff0d413fe50 2024-05-22T21:24:30.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.915+0000 7ff0da600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d40cd920 con 0x7ff0d415af60 2024-05-22T21:24:30.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.915+0000 7ff0da600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d40cdc00 con 0x7ff0d413cfc0 2024-05-22T21:24:30.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.915+0000 7ff0d3e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff0d413fe50 0x7ff0d406fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41884/0 (socket says 172.21.15.161:41884) 2024-05-22T21:24:30.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.915+0000 7ff0d3e00640 1 -- 172.21.15.161:0/167396052 learned_addr learned my addr 172.21.15.161:0/167396052 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:30.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.915+0000 7ff0d3400640 1 --2- 172.21.15.161:0/167396052 >> v2:172.21.15.179:3300/0 conn(0x7ff0d413cfc0 0x7ff0d41402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.919+0000 7ff0d2000640 1 -- 172.21.15.161:0/167396052 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2207667095 0 0) 0x7ff0d413d3a0 con 0x7ff0d413fe50 2024-05-22T21:24:30.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.919+0000 7ff0d2000640 1 -- 172.21.15.161:0/167396052 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0b8003530 con 0x7ff0d413fe50 2024-05-22T21:24:30.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.919+0000 7ff0d2000640 1 -- 172.21.15.161:0/167396052 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960350957 0 0) 0x7ff0d40cd920 con 0x7ff0d415af60 2024-05-22T21:24:30.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.919+0000 7ff0d3400640 1 -- 172.21.15.161:0/167396052 >> v1:172.21.15.179:6789/0 conn(0x7ff0d415af60 legacy=0x7ff0d413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.919+0000 7ff0d3400640 1 -- 172.21.15.161:0/167396052 >> v1:172.21.15.161:6789/0 conn(0x7ff0d413fe50 legacy=0x7ff0d406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.919+0000 7ff0d3400640 1 -- 172.21.15.161:0/167396052 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d40cdee0 con 0x7ff0d413cfc0 2024-05-22T21:24:30.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.919+0000 7ff0d3400640 1 --2- 172.21.15.161:0/167396052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d413cfc0 0x7ff0d41402e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40097c0 tx=0x7ff0c401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63f5a34a979201d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:30.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.919+0000 7ff0d2000640 1 -- 172.21.15.161:0/167396052 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40ac440 con 0x7ff0d413cfc0 2024-05-22T21:24:30.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.919+0000 7ff0d2000640 1 -- 172.21.15.161:0/167396052 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c40ac5a0 con 0x7ff0d413cfc0 2024-05-22T21:24:30.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.919+0000 7ff0d2000640 1 -- 172.21.15.161:0/167396052 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40ac930 con 0x7ff0d413cfc0 2024-05-22T21:24:30.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0da600640 1 -- 172.21.15.161:0/167396052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d413cfc0 msgr2=0x7ff0d41402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0da600640 1 --2- 172.21.15.161:0/167396052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d413cfc0 0x7ff0d41402e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40097c0 tx=0x7ff0c401c3e0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0da600640 1 -- 172.21.15.161:0/167396052 shutdown_connections 2024-05-22T21:24:30.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0da600640 1 --2- 172.21.15.161:0/167396052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d413cfc0 0x7ff0d41402e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0da600640 1 -- 172.21.15.161:0/167396052 >> 172.21.15.161:0/167396052 conn(0x7ff0d406a2d0 msgr2=0x7ff0d406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:30.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0da600640 1 -- 172.21.15.161:0/167396052 shutdown_connections 2024-05-22T21:24:30.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0da600640 1 -- 172.21.15.161:0/167396052 wait complete. 2024-05-22T21:24:30.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0da600640 1 Processor -- start 2024-05-22T21:24:30.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0da600640 1 -- start start 2024-05-22T21:24:30.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0da600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff0d413cfc0 0x7ff0d4157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0da600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0d413fe50 0x7ff0d41404f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0da600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d415af60 0x7ff0d4159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0da600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d413d5b0 con 0x7ff0d413fe50 2024-05-22T21:24:30.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0da600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d4142210 con 0x7ff0d415af60 2024-05-22T21:24:30.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0da600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d40cc0c0 con 0x7ff0d413cfc0 2024-05-22T21:24:30.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0d2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0d413fe50 0x7ff0d41404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0d2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0d413fe50 0x7ff0d41404f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60600/0 (socket says 172.21.15.161:60600) 2024-05-22T21:24:30.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0d2a00640 1 -- 172.21.15.161:0/487826750 learned_addr learned my addr 172.21.15.161:0/487826750 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:30.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0d3e00640 1 --2- 172.21.15.161:0/487826750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d415af60 0x7ff0d4159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0d3400640 1 --2- 172.21.15.161:0/487826750 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff0d413cfc0 0x7ff0d4157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0d2a00640 1 -- 172.21.15.161:0/487826750 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff0d413cfc0 msgr2=0x7ff0d4157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0d2a00640 1 --2- 172.21.15.161:0/487826750 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff0d413cfc0 0x7ff0d4157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0d2a00640 1 -- 172.21.15.161:0/487826750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d415af60 msgr2=0x7ff0d4159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:30.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0d2a00640 1 --2- 172.21.15.161:0/487826750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d415af60 0x7ff0d4159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:30.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.923+0000 7ff0d2a00640 1 -- 172.21.15.161:0/487826750 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c4009420 con 0x7ff0d413fe50 2024-05-22T21:24:30.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.927+0000 7ff0d2a00640 1 --2- 172.21.15.161:0/487826750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0d413fe50 0x7ff0d41404f0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000c500 tx=0x7ff0c000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:30.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.927+0000 7ff0cb400640 1 -- 172.21.15.161:0/487826750 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0003600 con 0x7ff0d413fe50 2024-05-22T21:24:30.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.927+0000 7ff0cb400640 1 -- 172.21.15.161:0/487826750 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c0003760 con 0x7ff0d413fe50 2024-05-22T21:24:30.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.927+0000 7ff0da600640 1 -- 172.21.15.161:0/487826750 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d415c3c0 con 0x7ff0d413fe50 2024-05-22T21:24:30.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.927+0000 7ff0cb400640 1 -- 172.21.15.161:0/487826750 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0003b10 con 0x7ff0d413fe50 2024-05-22T21:24:30.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.927+0000 7ff0da600640 1 -- 172.21.15.161:0/487826750 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d40cc3a0 con 0x7ff0d413fe50 2024-05-22T21:24:30.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.927+0000 7ff0da600640 1 -- 172.21.15.161:0/487826750 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d4141fe0 con 0x7ff0d413fe50 2024-05-22T21:24:30.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.927+0000 7ff0cb400640 1 -- 172.21.15.161:0/487826750 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff0c0003c70 con 0x7ff0d413fe50 2024-05-22T21:24:30.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.927+0000 7ff0cb400640 1 --2- 172.21.15.161:0/487826750 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff0b4066690 0x7ff0b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:30.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.927+0000 7ff0cb400640 1 -- 172.21.15.161:0/487826750 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff0c0083420 con 0x7ff0d413fe50 2024-05-22T21:24:30.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.927+0000 7ff0d3400640 1 --2- 172.21.15.161:0/487826750 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff0b4066690 0x7ff0b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:30.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.931+0000 7ff0d3400640 1 --2- 172.21.15.161:0/487826750 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff0b4066690 0x7ff0b4068b50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40ae000 tx=0x7ff0c40a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:30.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:30.931+0000 7ff0cb400640 1 -- 172.21.15.161:0/487826750 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff0d4141fe0 con 0x7ff0d413fe50 2024-05-22T21:24:31.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.087+0000 7ff0da600640 1 -- 172.21.15.161:0/487826750 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0d4141fe0 con 0x7ff0d413fe50 2024-05-22T21:24:31.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.087+0000 7ff0cb400640 1 -- 172.21.15.161:0/487826750 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v186) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff0c0050c40 con 0x7ff0d413fe50 2024-05-22T21:24:31.088 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:24:31.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.091+0000 7ff0da600640 1 -- 172.21.15.161:0/487826750 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff0b4066690 msgr2=0x7ff0b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:31.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.091+0000 7ff0da600640 1 --2- 172.21.15.161:0/487826750 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff0b4066690 0x7ff0b4068b50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40ae000 tx=0x7ff0c40a7000 comp rx=0 tx=0).stop 2024-05-22T21:24:31.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.091+0000 7ff0da600640 1 -- 172.21.15.161:0/487826750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0d413fe50 msgr2=0x7ff0d41404f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:31.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.091+0000 7ff0da600640 1 --2- 172.21.15.161:0/487826750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0d413fe50 0x7ff0d41404f0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000c500 tx=0x7ff0c000c9c0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.091+0000 7ff0da600640 1 -- 172.21.15.161:0/487826750 shutdown_connections 2024-05-22T21:24:31.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.091+0000 7ff0da600640 1 --2- 172.21.15.161:0/487826750 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff0b4066690 0x7ff0b4068b50 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.091+0000 7ff0da600640 1 --2- 172.21.15.161:0/487826750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d415af60 0x7ff0d4159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.091+0000 7ff0da600640 1 --2- 172.21.15.161:0/487826750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0d413fe50 0x7ff0d41404f0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.091+0000 7ff0da600640 1 --2- 172.21.15.161:0/487826750 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff0d413cfc0 0x7ff0d4157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.091+0000 7ff0da600640 1 -- 172.21.15.161:0/487826750 >> 172.21.15.161:0/487826750 conn(0x7ff0d406a2d0 msgr2=0x7ff0d406cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:31.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.091+0000 7ff0da600640 1 -- 172.21.15.161:0/487826750 shutdown_connections 2024-05-22T21:24:31.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.091+0000 7ff0da600640 1 -- 172.21.15.161:0/487826750 wait complete. 2024-05-22T21:24:31.109 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-22T21:24:31.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.243+0000 7ff68e200640 1 Processor -- start 2024-05-22T21:24:31.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.243+0000 7ff68e200640 1 -- start start 2024-05-22T21:24:31.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.247+0000 7ff68e200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6881597f0 0x7ff6880706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:31.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.247+0000 7ff68e200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff6880b3070 0x7ff6880cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:31.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.247+0000 7ff68e200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff688071040 con 0x7ff688070be0 2024-05-22T21:24:31.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.247+0000 7ff68e200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6880b3500 con 0x7ff6880b3070 2024-05-22T21:24:31.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.247+0000 7ff68e200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff688159bd0 con 0x7ff6881597f0 2024-05-22T21:24:31.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.247+0000 7ff686a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff688070be0 0x7ff6880b2960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41892/0 (socket says 172.21.15.161:41892) 2024-05-22T21:24:31.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.247+0000 7ff686a00640 1 -- 172.21.15.161:0/1312537110 learned_addr learned my addr 172.21.15.161:0/1312537110 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:31.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.247+0000 7ff687e00640 1 --2- 172.21.15.161:0/1312537110 >> v2:172.21.15.179:3300/0 conn(0x7ff6880b3070 0x7ff6880cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:31.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.247+0000 7ff687400640 1 --2- 172.21.15.161:0/1312537110 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6881597f0 0x7ff6880706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:31.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.247+0000 7ff686000640 1 -- 172.21.15.161:0/1312537110 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441698071 0 0) 0x7ff688071040 con 0x7ff688070be0 2024-05-22T21:24:31.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.247+0000 7ff686000640 1 -- 172.21.15.161:0/1312537110 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff66c003530 con 0x7ff688070be0 2024-05-22T21:24:31.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.247+0000 7ff687e00640 1 -- 172.21.15.161:0/1312537110 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6881597f0 msgr2=0x7ff6880706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:31.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.247+0000 7ff687e00640 1 --2- 172.21.15.161:0/1312537110 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6881597f0 0x7ff6880706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.247+0000 7ff687e00640 1 -- 172.21.15.161:0/1312537110 >> v1:172.21.15.161:6789/0 conn(0x7ff688070be0 legacy=0x7ff6880b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:31.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.247+0000 7ff687e00640 1 -- 172.21.15.161:0/1312537110 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6880cbdd0 con 0x7ff6880b3070 2024-05-22T21:24:31.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.251+0000 7ff687e00640 1 --2- 172.21.15.161:0/1312537110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6880b3070 0x7ff6880cc550 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff6800097c0 tx=0x7ff68001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bba0caee84558217 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:31.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.251+0000 7ff686000640 1 -- 172.21.15.161:0/1312537110 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6800ab620 con 0x7ff6880b3070 2024-05-22T21:24:31.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.251+0000 7ff686000640 1 -- 172.21.15.161:0/1312537110 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6800ab780 con 0x7ff6880b3070 2024-05-22T21:24:31.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.251+0000 7ff686000640 1 -- 172.21.15.161:0/1312537110 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6800abb10 con 0x7ff6880b3070 2024-05-22T21:24:31.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.251+0000 7ff68e200640 1 -- 172.21.15.161:0/1312537110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6880b3070 msgr2=0x7ff6880cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:31.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.251+0000 7ff68e200640 1 --2- 172.21.15.161:0/1312537110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6880b3070 0x7ff6880cc550 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff6800097c0 tx=0x7ff68001c4a0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.251+0000 7ff68e200640 1 -- 172.21.15.161:0/1312537110 shutdown_connections 2024-05-22T21:24:31.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.251+0000 7ff68e200640 1 --2- 172.21.15.161:0/1312537110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6880b3070 0x7ff6880cc550 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.251+0000 7ff68e200640 1 --2- 172.21.15.161:0/1312537110 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6881597f0 0x7ff6880706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.251+0000 7ff68e200640 1 -- 172.21.15.161:0/1312537110 >> 172.21.15.161:0/1312537110 conn(0x7ff6880b0bc0 msgr2=0x7ff6880b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:31.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.251+0000 7ff68e200640 1 -- 172.21.15.161:0/1312537110 shutdown_connections 2024-05-22T21:24:31.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.251+0000 7ff68e200640 1 -- 172.21.15.161:0/1312537110 wait complete. 2024-05-22T21:24:31.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.251+0000 7ff68e200640 1 Processor -- start 2024-05-22T21:24:31.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.251+0000 7ff68e200640 1 -- start start 2024-05-22T21:24:31.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff68e200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff688070be0 0x7ff688145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:31.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff68e200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6880b3070 0x7ff6881552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:31.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff68e200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6881597f0 0x7ff6881443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:31.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff68e200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68815b8e0 con 0x7ff6880b3070 2024-05-22T21:24:31.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff68e200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68815bb80 con 0x7ff6881597f0 2024-05-22T21:24:31.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff68e200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68815be60 con 0x7ff688070be0 2024-05-22T21:24:31.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff686a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6880b3070 0x7ff6881552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:31.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff686a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6880b3070 0x7ff6881552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60610/0 (socket says 172.21.15.161:60610) 2024-05-22T21:24:31.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff686a00640 1 -- 172.21.15.161:0/3447249230 learned_addr learned my addr 172.21.15.161:0/3447249230 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:31.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff687e00640 1 --2- 172.21.15.161:0/3447249230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6881597f0 0x7ff6881443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:31.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff686a00640 1 -- 172.21.15.161:0/3447249230 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff688070be0 msgr2=0x7ff688145c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:31.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff687400640 1 --2- 172.21.15.161:0/3447249230 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff688070be0 0x7ff688145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:31.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff686a00640 1 --2- 172.21.15.161:0/3447249230 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff688070be0 0x7ff688145c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff686a00640 1 -- 172.21.15.161:0/3447249230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6881597f0 msgr2=0x7ff6881443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:31.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff686a00640 1 --2- 172.21.15.161:0/3447249230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6881597f0 0x7ff6881443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff686a00640 1 -- 172.21.15.161:0/3447249230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff680009420 con 0x7ff6880b3070 2024-05-22T21:24:31.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff687e00640 1 --2- 172.21.15.161:0/3447249230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6881597f0 0x7ff6881443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:31.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff686a00640 1 --2- 172.21.15.161:0/3447249230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6880b3070 0x7ff6881552c0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7ff67400c4d0 tx=0x7ff67400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:31.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff67f400640 1 -- 172.21.15.161:0/3447249230 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674005cf0 con 0x7ff6880b3070 2024-05-22T21:24:31.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff67f400640 1 -- 172.21.15.161:0/3447249230 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff674005e50 con 0x7ff6880b3070 2024-05-22T21:24:31.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff67f400640 1 -- 172.21.15.161:0/3447249230 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674007270 con 0x7ff6880b3070 2024-05-22T21:24:31.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff68e200640 1 -- 172.21.15.161:0/3447249230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68815c140 con 0x7ff6880b3070 2024-05-22T21:24:31.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff68e200640 1 -- 172.21.15.161:0/3447249230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff688146160 con 0x7ff6880b3070 2024-05-22T21:24:31.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff68e200640 1 -- 172.21.15.161:0/3447249230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68815c660 con 0x7ff6880b3070 2024-05-22T21:24:31.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.255+0000 7ff67f400640 1 -- 172.21.15.161:0/3447249230 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff6740073d0 con 0x7ff6880b3070 2024-05-22T21:24:31.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.259+0000 7ff67f400640 1 --2- 172.21.15.161:0/3447249230 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff668066690 0x7ff668068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:31.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.259+0000 7ff67f400640 1 -- 172.21.15.161:0/3447249230 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff6740833c0 con 0x7ff6880b3070 2024-05-22T21:24:31.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.259+0000 7ff687400640 1 --2- 172.21.15.161:0/3447249230 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff668066690 0x7ff668068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:31.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.259+0000 7ff687400640 1 --2- 172.21.15.161:0/3447249230 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff668066690 0x7ff668068b50 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff678007380 tx=0x7ff67800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:31.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.263+0000 7ff67f400640 1 -- 172.21.15.161:0/3447249230 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff68815c660 con 0x7ff6880b3070 2024-05-22T21:24:31.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.415+0000 7ff68e200640 1 -- 172.21.15.161:0/3447249230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff68815c660 con 0x7ff6880b3070 2024-05-22T21:24:31.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.415+0000 7ff67f400640 1 -- 172.21.15.161:0/3447249230 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v186) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff674017060 con 0x7ff6880b3070 2024-05-22T21:24:31.417 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:31.417 INFO:teuthology.orchestra.run.smithi161.stdout:[ 2024-05-22T21:24:31.417 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:31.417 INFO:teuthology.orchestra.run.smithi161.stdout: "poolnum": 1, 2024-05-22T21:24:31.417 INFO:teuthology.orchestra.run.smithi161.stdout: "poolname": ".mgr" 2024-05-22T21:24:31.417 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:31.417 INFO:teuthology.orchestra.run.smithi161.stdout:] 2024-05-22T21:24:31.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.419+0000 7ff68e200640 1 -- 172.21.15.161:0/3447249230 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff668066690 msgr2=0x7ff668068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:31.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.419+0000 7ff68e200640 1 --2- 172.21.15.161:0/3447249230 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff668066690 0x7ff668068b50 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff678007380 tx=0x7ff67800b040 comp rx=0 tx=0).stop 2024-05-22T21:24:31.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.419+0000 7ff68e200640 1 -- 172.21.15.161:0/3447249230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6880b3070 msgr2=0x7ff6881552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:31.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.419+0000 7ff68e200640 1 --2- 172.21.15.161:0/3447249230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6880b3070 0x7ff6881552c0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7ff67400c4d0 tx=0x7ff67400c990 comp rx=0 tx=0).stop 2024-05-22T21:24:31.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.419+0000 7ff68e200640 1 -- 172.21.15.161:0/3447249230 shutdown_connections 2024-05-22T21:24:31.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.419+0000 7ff68e200640 1 --2- 172.21.15.161:0/3447249230 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff668066690 0x7ff668068b50 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.419+0000 7ff68e200640 1 --2- 172.21.15.161:0/3447249230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6881597f0 0x7ff6881443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.419+0000 7ff68e200640 1 --2- 172.21.15.161:0/3447249230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6880b3070 0x7ff6881552c0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.419+0000 7ff68e200640 1 --2- 172.21.15.161:0/3447249230 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff688070be0 0x7ff688145c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.419+0000 7ff68e200640 1 -- 172.21.15.161:0/3447249230 >> 172.21.15.161:0/3447249230 conn(0x7ff6880b0bc0 msgr2=0x7ff6881574b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:31.419 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.419+0000 7ff68e200640 1 -- 172.21.15.161:0/3447249230 shutdown_connections 2024-05-22T21:24:31.420 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.419+0000 7ff68e200640 1 -- 172.21.15.161:0/3447249230 wait complete. 2024-05-22T21:24:31.438 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-22T21:24:31.438 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-22T21:24:31.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.579+0000 7f6eab600640 1 Processor -- start 2024-05-22T21:24:31.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.579+0000 7f6eab600640 1 -- start start 2024-05-22T21:24:31.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.579+0000 7f6eab600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ea413cfe0 0x7f6ea415dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:31.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.579+0000 7f6eab600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6ea40ce210 0x7f6ea415e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:31.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.579+0000 7f6eab600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea40ce830 con 0x7f6ea413d560 2024-05-22T21:24:31.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.579+0000 7f6eab600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea40ce650 con 0x7f6ea40ce210 2024-05-22T21:24:31.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.579+0000 7f6eab600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea4140560 con 0x7f6ea413cfe0 2024-05-22T21:24:31.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.579+0000 7f6ea9600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6ea40ce210 0x7f6ea415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:31.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.579+0000 7f6ea9600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6ea40ce210 0x7f6ea415e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60618/0 (socket says 172.21.15.161:60618) 2024-05-22T21:24:31.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.579+0000 7f6ea9600640 1 -- 172.21.15.161:0/1603254201 learned_addr learned my addr 172.21.15.161:0/1603254201 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:31.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.579+0000 7f6ea8c00640 1 --2- 172.21.15.161:0/1603254201 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ea413cfe0 0x7f6ea415dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:31.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.579+0000 7f6ea9600640 1 -- 172.21.15.161:0/1603254201 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ea413cfe0 msgr2=0x7f6ea415dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:31.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.579+0000 7f6ea9600640 1 --2- 172.21.15.161:0/1603254201 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ea413cfe0 0x7f6ea415dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.579+0000 7f6ea9600640 1 -- 172.21.15.161:0/1603254201 >> v1:172.21.15.179:6789/0 conn(0x7f6ea413d560 legacy=0x7f6ea40cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:31.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.579+0000 7f6ea9600640 1 -- 172.21.15.161:0/1603254201 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea40ceb10 con 0x7f6ea40ce210 2024-05-22T21:24:31.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.583+0000 7f6ea9600640 1 --2- 172.21.15.161:0/1603254201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea40ce210 0x7f6ea415e000 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009e50 tx=0x7f6e9c01eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19da25621a645f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:31.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.583+0000 7f6ea3e00640 1 -- 172.21.15.161:0/1603254201 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c0a7a50 con 0x7f6ea40ce210 2024-05-22T21:24:31.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.583+0000 7f6ea3e00640 1 -- 172.21.15.161:0/1603254201 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9c0a7bb0 con 0x7f6ea40ce210 2024-05-22T21:24:31.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.583+0000 7f6ea3e00640 1 -- 172.21.15.161:0/1603254201 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c0af3f0 con 0x7f6ea40ce210 2024-05-22T21:24:31.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.583+0000 7f6eab600640 1 -- 172.21.15.161:0/1603254201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea40ce210 msgr2=0x7f6ea415e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:31.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.583+0000 7f6eab600640 1 --2- 172.21.15.161:0/1603254201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea40ce210 0x7f6ea415e000 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009e50 tx=0x7f6e9c01eb80 comp rx=0 tx=0).stop 2024-05-22T21:24:31.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.583+0000 7f6eab600640 1 -- 172.21.15.161:0/1603254201 shutdown_connections 2024-05-22T21:24:31.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.583+0000 7f6eab600640 1 --2- 172.21.15.161:0/1603254201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea40ce210 0x7f6ea415e000 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.583+0000 7f6eab600640 1 --2- 172.21.15.161:0/1603254201 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ea413cfe0 0x7f6ea415dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.583+0000 7f6eab600640 1 -- 172.21.15.161:0/1603254201 >> 172.21.15.161:0/1603254201 conn(0x7f6ea406a0a0 msgr2=0x7f6ea406a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:31.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.583+0000 7f6eab600640 1 -- 172.21.15.161:0/1603254201 shutdown_connections 2024-05-22T21:24:31.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.583+0000 7f6eab600640 1 -- 172.21.15.161:0/1603254201 wait complete. 2024-05-22T21:24:31.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6eab600640 1 Processor -- start 2024-05-22T21:24:31.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6eab600640 1 -- start start 2024-05-22T21:24:31.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6eab600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ea40ce210 0x7f6ea4153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:31.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6eab600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ea413cfe0 0x7f6ea415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:31.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6eab600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea413d560 0x7f6ea415ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:31.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6eab600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea41536c0 con 0x7f6ea413d560 2024-05-22T21:24:31.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6eab600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea4154470 con 0x7f6ea40ce210 2024-05-22T21:24:31.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6eab600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea41538f0 con 0x7f6ea413cfe0 2024-05-22T21:24:31.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6ea9600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea413d560 0x7f6ea415ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:31.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6ea9600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea413d560 0x7f6ea415ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60624/0 (socket says 172.21.15.161:60624) 2024-05-22T21:24:31.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6ea9600640 1 -- 172.21.15.161:0/1279760501 learned_addr learned my addr 172.21.15.161:0/1279760501 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:31.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6ea8c00640 1 --2- 172.21.15.161:0/1279760501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ea40ce210 0x7f6ea4153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:31.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6e9be00640 1 --2- 172.21.15.161:0/1279760501 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ea413cfe0 0x7f6ea415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:31.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6ea9600640 1 -- 172.21.15.161:0/1279760501 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ea413cfe0 msgr2=0x7f6ea415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:31.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6ea9600640 1 --2- 172.21.15.161:0/1279760501 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ea413cfe0 0x7f6ea415a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6ea9600640 1 -- 172.21.15.161:0/1279760501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ea40ce210 msgr2=0x7f6ea4153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:31.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6ea9600640 1 --2- 172.21.15.161:0/1279760501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ea40ce210 0x7f6ea4153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:31.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6ea9600640 1 -- 172.21.15.161:0/1279760501 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c009b30 con 0x7f6ea413d560 2024-05-22T21:24:31.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6ea9600640 1 --2- 172.21.15.161:0/1279760501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea413d560 0x7f6ea415ad00 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c01c7b0 tx=0x7f6e9c01b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:31.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6ea1600640 1 -- 172.21.15.161:0/1279760501 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c0a7a50 con 0x7f6ea413d560 2024-05-22T21:24:31.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6ea1600640 1 -- 172.21.15.161:0/1279760501 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9c00e150 con 0x7f6ea413d560 2024-05-22T21:24:31.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6ea1600640 1 -- 172.21.15.161:0/1279760501 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c007ea0 con 0x7f6ea413d560 2024-05-22T21:24:31.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6eab600640 1 -- 172.21.15.161:0/1279760501 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea4153bd0 con 0x7f6ea413d560 2024-05-22T21:24:31.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6eab600640 1 -- 172.21.15.161:0/1279760501 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea4154190 con 0x7f6ea413d560 2024-05-22T21:24:31.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.587+0000 7f6eab600640 1 -- 172.21.15.161:0/1279760501 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea412a0a0 con 0x7f6ea413d560 2024-05-22T21:24:31.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.591+0000 7f6ea1600640 1 -- 172.21.15.161:0/1279760501 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6e9c0af3f0 con 0x7f6ea413d560 2024-05-22T21:24:31.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.591+0000 7f6ea1600640 1 --2- 172.21.15.161:0/1279760501 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e80066690 0x7f6e80068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:31.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.591+0000 7f6ea1600640 1 -- 172.21.15.161:0/1279760501 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6e9c11e260 con 0x7f6ea413d560 2024-05-22T21:24:31.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.591+0000 7f6ea8c00640 1 --2- 172.21.15.161:0/1279760501 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e80066690 0x7f6e80068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:31.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.591+0000 7f6ea8c00640 1 --2- 172.21.15.161:0/1279760501 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e80066690 0x7f6e80068b50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6e94007a60 tx=0x7f6e94008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:31.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.595+0000 7f6ea1600640 1 -- 172.21.15.161:0/1279760501 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6ea412a0a0 con 0x7f6ea413d560 2024-05-22T21:24:31.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:31.751+0000 7f6eab600640 1 -- 172.21.15.161:0/1279760501 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6ea412a0a0 con 0x7f6ea413d560 2024-05-22T21:24:32.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.707+0000 7f6ea1600640 1 -- 172.21.15.161:0/1279760501 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v187) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6ea412a0a0 con 0x7f6ea413d560 2024-05-22T21:24:32.709 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' created 2024-05-22T21:24:32.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.711+0000 7f6eab600640 1 -- 172.21.15.161:0/1279760501 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e80066690 msgr2=0x7f6e80068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:32.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.711+0000 7f6eab600640 1 --2- 172.21.15.161:0/1279760501 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e80066690 0x7f6e80068b50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6e94007a60 tx=0x7f6e94008040 comp rx=0 tx=0).stop 2024-05-22T21:24:32.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.711+0000 7f6eab600640 1 -- 172.21.15.161:0/1279760501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea413d560 msgr2=0x7f6ea415ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:32.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.711+0000 7f6eab600640 1 --2- 172.21.15.161:0/1279760501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea413d560 0x7f6ea415ad00 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c01c7b0 tx=0x7f6e9c01b920 comp rx=0 tx=0).stop 2024-05-22T21:24:32.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.711+0000 7f6eab600640 1 -- 172.21.15.161:0/1279760501 shutdown_connections 2024-05-22T21:24:32.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.711+0000 7f6eab600640 1 --2- 172.21.15.161:0/1279760501 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6e80066690 0x7f6e80068b50 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:32.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.711+0000 7f6eab600640 1 --2- 172.21.15.161:0/1279760501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ea413d560 0x7f6ea415ad00 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:32.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.711+0000 7f6eab600640 1 --2- 172.21.15.161:0/1279760501 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ea413cfe0 0x7f6ea415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:32.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.711+0000 7f6eab600640 1 --2- 172.21.15.161:0/1279760501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ea40ce210 0x7f6ea4153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:32.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.711+0000 7f6eab600640 1 -- 172.21.15.161:0/1279760501 >> 172.21.15.161:0/1279760501 conn(0x7f6ea406a0a0 msgr2=0x7f6ea406c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:32.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.711+0000 7f6eab600640 1 -- 172.21.15.161:0/1279760501 shutdown_connections 2024-05-22T21:24:32.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.715+0000 7f6eab600640 1 -- 172.21.15.161:0/1279760501 wait complete. 2024-05-22T21:24:32.735 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-22T21:24:32.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.879+0000 7f431a600640 1 Processor -- start 2024-05-22T21:24:32.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.879+0000 7f431a600640 1 -- start start 2024-05-22T21:24:32.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.879+0000 7f431a600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f431415b600 0x7f431406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:32.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.879+0000 7f431a600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f431406c7e0 0x7f43140b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:32.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.879+0000 7f431a600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4314144bb0 con 0x7f4314143c40 2024-05-22T21:24:32.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.879+0000 7f431a600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f431415ba60 con 0x7f431406c7e0 2024-05-22T21:24:32.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.879+0000 7f431a600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431406cc70 con 0x7f431415b600 2024-05-22T21:24:32.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.879+0000 7f4312a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f431415b600 0x7f431406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:32.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.879+0000 7f4313e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f431406c7e0 0x7f43140b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:32.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.879+0000 7f4312a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f431415b600 0x7f431406c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:52782/0 (socket says 172.21.15.161:52782) 2024-05-22T21:24:32.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.879+0000 7f4312a00640 1 -- 172.21.15.161:0/1193155667 learned_addr learned my addr 172.21.15.161:0/1193155667 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:32.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.879+0000 7f4313400640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f4314143c40 0x7f43141444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:46358/0 (socket says 172.21.15.161:46358) 2024-05-22T21:24:32.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.879+0000 7f4312a00640 1 -- 172.21.15.161:0/1193155667 >> v2:172.21.15.179:3300/0 conn(0x7f431406c7e0 msgr2=0x7f43140b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:32.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.879+0000 7f4312a00640 1 --2- 172.21.15.161:0/1193155667 >> v2:172.21.15.179:3300/0 conn(0x7f431406c7e0 0x7f43140b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:32.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.879+0000 7f4312a00640 1 -- 172.21.15.161:0/1193155667 >> v1:172.21.15.179:6789/0 conn(0x7f4314143c40 legacy=0x7f43141444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:32.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.879+0000 7f4312a00640 1 -- 172.21.15.161:0/1193155667 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431415efb0 con 0x7f431415b600 2024-05-22T21:24:32.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.883+0000 7f4312a00640 1 --2- 172.21.15.161:0/1193155667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f431415b600 0x7f431406c2a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4304009870 tx=0x7f430401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b044aea19939e12e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:32.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.883+0000 7f4312000640 1 -- 172.21.15.161:0/1193155667 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43040a9620 con 0x7f431415b600 2024-05-22T21:24:32.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.883+0000 7f4312000640 1 -- 172.21.15.161:0/1193155667 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43040a9780 con 0x7f431415b600 2024-05-22T21:24:32.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.883+0000 7f4312000640 1 -- 172.21.15.161:0/1193155667 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43040a9b10 con 0x7f431415b600 2024-05-22T21:24:32.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.883+0000 7f431a600640 1 -- 172.21.15.161:0/1193155667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f431415b600 msgr2=0x7f431406c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:32.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.883+0000 7f431a600640 1 --2- 172.21.15.161:0/1193155667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f431415b600 0x7f431406c2a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4304009870 tx=0x7f430401c560 comp rx=0 tx=0).stop 2024-05-22T21:24:32.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.883+0000 7f431a600640 1 -- 172.21.15.161:0/1193155667 shutdown_connections 2024-05-22T21:24:32.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.883+0000 7f431a600640 1 --2- 172.21.15.161:0/1193155667 >> v2:172.21.15.179:3300/0 conn(0x7f431406c7e0 0x7f43140b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:32.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.883+0000 7f431a600640 1 --2- 172.21.15.161:0/1193155667 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f431415b600 0x7f431406c2a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:32.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.883+0000 7f431a600640 1 -- 172.21.15.161:0/1193155667 >> 172.21.15.161:0/1193155667 conn(0x7f431406a2d0 msgr2=0x7f431406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:32.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.883+0000 7f431a600640 1 -- 172.21.15.161:0/1193155667 shutdown_connections 2024-05-22T21:24:32.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.883+0000 7f431a600640 1 -- 172.21.15.161:0/1193155667 wait complete. 2024-05-22T21:24:32.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f431a600640 1 Processor -- start 2024-05-22T21:24:32.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f431a600640 1 -- start start 2024-05-22T21:24:32.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f431a600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431406c7e0 0x7f431413b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:32.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f431a600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4314143c40 0x7f43141364c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:32.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f431a600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f431415b600 0x7f431415e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:32.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f431a600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4314158ea0 con 0x7f431406c7e0 2024-05-22T21:24:32.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f431a600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431413ba10 con 0x7f4314143c40 2024-05-22T21:24:32.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f431a600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431413bcf0 con 0x7f431415b600 2024-05-22T21:24:32.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f4313400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431406c7e0 0x7f431413b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:32.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f4313400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431406c7e0 0x7f431413b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60634/0 (socket says 172.21.15.161:60634) 2024-05-22T21:24:32.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f4313400640 1 -- 172.21.15.161:0/3465802960 learned_addr learned my addr 172.21.15.161:0/3465802960 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:32.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f4312a00640 1 --2- 172.21.15.161:0/3465802960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4314143c40 0x7f43141364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:32.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f4313e00640 1 --2- 172.21.15.161:0/3465802960 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f431415b600 0x7f431415e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:32.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f4313400640 1 -- 172.21.15.161:0/3465802960 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f431415b600 msgr2=0x7f431415e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:32.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f4313400640 1 --2- 172.21.15.161:0/3465802960 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f431415b600 0x7f431415e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:32.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f4313400640 1 -- 172.21.15.161:0/3465802960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4314143c40 msgr2=0x7f43141364c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:32.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f4313400640 1 --2- 172.21.15.161:0/3465802960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4314143c40 0x7f43141364c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:32.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f4313400640 1 -- 172.21.15.161:0/3465802960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43040094d0 con 0x7f431406c7e0 2024-05-22T21:24:32.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f4313400640 1 --2- 172.21.15.161:0/3465802960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431406c7e0 0x7f431413b150 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00c490 tx=0x7f42fc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:32.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f430b400640 1 -- 172.21.15.161:0/3465802960 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc007480 con 0x7f431406c7e0 2024-05-22T21:24:32.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f430b400640 1 -- 172.21.15.161:0/3465802960 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42fc0075e0 con 0x7f431406c7e0 2024-05-22T21:24:32.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f430b400640 1 -- 172.21.15.161:0/3465802960 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc004a10 con 0x7f431406c7e0 2024-05-22T21:24:32.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f431a600640 1 -- 172.21.15.161:0/3465802960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431413bfd0 con 0x7f431406c7e0 2024-05-22T21:24:32.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.887+0000 7f431a600640 1 -- 172.21.15.161:0/3465802960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431413b690 con 0x7f431406c7e0 2024-05-22T21:24:32.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.891+0000 7f431a600640 1 -- 172.21.15.161:0/3465802960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4314159010 con 0x7f431406c7e0 2024-05-22T21:24:32.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.891+0000 7f430b400640 1 -- 172.21.15.161:0/3465802960 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f42fc004b70 con 0x7f431406c7e0 2024-05-22T21:24:32.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.891+0000 7f430b400640 1 --2- 172.21.15.161:0/3465802960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f42f0066690 0x7f42f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:32.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.891+0000 7f430b400640 1 -- 172.21.15.161:0/3465802960 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f42fc083270 con 0x7f431406c7e0 2024-05-22T21:24:32.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.891+0000 7f4312a00640 1 --2- 172.21.15.161:0/3465802960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f42f0066690 0x7f42f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:32.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.891+0000 7f4312a00640 1 --2- 172.21.15.161:0/3465802960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f42f0066690 0x7f42f0068b50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f43040ae000 tx=0x7f4304009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:32.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:32.895+0000 7f430b400640 1 -- 172.21.15.161:0/3465802960 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f42fc0506c0 con 0x7f431406c7e0 2024-05-22T21:24:33.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.051+0000 7f431a600640 1 -- 172.21.15.161:0/3465802960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4314067220 con 0x7f431406c7e0 2024-05-22T21:24:33.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.719+0000 7f430b400640 1 -- 172.21.15.161:0/3465802960 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v188) v1 ==== 170+0+0 (secure 0 0 0) 0x7f42fc053ce0 con 0x7f431406c7e0 2024-05-22T21:24:33.721 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' created 2024-05-22T21:24:33.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.727+0000 7f431a600640 1 -- 172.21.15.161:0/3465802960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f42f0066690 msgr2=0x7f42f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:33.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.727+0000 7f431a600640 1 --2- 172.21.15.161:0/3465802960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f42f0066690 0x7f42f0068b50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f43040ae000 tx=0x7f4304009040 comp rx=0 tx=0).stop 2024-05-22T21:24:33.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.727+0000 7f431a600640 1 -- 172.21.15.161:0/3465802960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431406c7e0 msgr2=0x7f431413b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:33.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.727+0000 7f431a600640 1 --2- 172.21.15.161:0/3465802960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431406c7e0 0x7f431413b150 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00c490 tx=0x7f42fc00c950 comp rx=0 tx=0).stop 2024-05-22T21:24:33.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.731+0000 7f431a600640 1 -- 172.21.15.161:0/3465802960 shutdown_connections 2024-05-22T21:24:33.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.731+0000 7f431a600640 1 --2- 172.21.15.161:0/3465802960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f42f0066690 0x7f42f0068b50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:33.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.731+0000 7f431a600640 1 --2- 172.21.15.161:0/3465802960 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f431415b600 0x7f431415e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:33.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.731+0000 7f431a600640 1 --2- 172.21.15.161:0/3465802960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4314143c40 0x7f43141364c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:33.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.731+0000 7f431a600640 1 --2- 172.21.15.161:0/3465802960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431406c7e0 0x7f431413b150 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:33.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.731+0000 7f431a600640 1 -- 172.21.15.161:0/3465802960 >> 172.21.15.161:0/3465802960 conn(0x7f431406a2d0 msgr2=0x7f43140cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:33.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.731+0000 7f431a600640 1 -- 172.21.15.161:0/3465802960 shutdown_connections 2024-05-22T21:24:33.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.731+0000 7f431a600640 1 -- 172.21.15.161:0/3465802960 wait complete. 2024-05-22T21:24:33.754 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-22T21:24:33.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.891+0000 7f3a55a00640 1 Processor -- start 2024-05-22T21:24:33.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.891+0000 7f3a55a00640 1 -- start start 2024-05-22T21:24:33.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.891+0000 7f3a55a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a5013fe50 0x7f3a50164e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:33.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.891+0000 7f3a55a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a5013d3a0 con 0x7f3a501402e0 2024-05-22T21:24:33.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.891+0000 7f3a55a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a500cd920 con 0x7f3a5013cfc0 2024-05-22T21:24:33.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.891+0000 7f3a55a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500cdc00 con 0x7f3a5013fe50 2024-05-22T21:24:33.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.891+0000 7f3a4e000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3a501402e0 0x7f3a5013f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41902/0 (socket says 172.21.15.161:41902) 2024-05-22T21:24:33.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.891+0000 7f3a4e000640 1 -- 172.21.15.161:0/2348343135 learned_addr learned my addr 172.21.15.161:0/2348343135 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:33.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.891+0000 7f3a4f400640 1 --2- 172.21.15.161:0/2348343135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a5013fe50 0x7f3a50164e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:33.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 -- 172.21.15.161:0/2348343135 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471975110 0 0) 0x7f3a5013d3a0 con 0x7f3a501402e0 2024-05-22T21:24:33.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 -- 172.21.15.161:0/2348343135 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a34005740 con 0x7f3a501402e0 2024-05-22T21:24:33.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 -- 172.21.15.161:0/2348343135 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780633221 0 0) 0x7f3a500cd920 con 0x7f3a5013cfc0 2024-05-22T21:24:33.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 -- 172.21.15.161:0/2348343135 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a5013d3a0 con 0x7f3a5013cfc0 2024-05-22T21:24:33.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 -- 172.21.15.161:0/2348343135 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1863455689 0 0) 0x7f3a34005740 con 0x7f3a501402e0 2024-05-22T21:24:33.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 -- 172.21.15.161:0/2348343135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a5013fe50 msgr2=0x7f3a50164e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:33.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 --2- 172.21.15.161:0/2348343135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a5013fe50 0x7f3a50164e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:33.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 -- 172.21.15.161:0/2348343135 >> v1:172.21.15.179:6789/0 conn(0x7f3a5013cfc0 legacy=0x7f3a5015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:33.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 -- 172.21.15.161:0/2348343135 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a500cdee0 con 0x7f3a501402e0 2024-05-22T21:24:33.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 -- 172.21.15.161:0/2348343135 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f3a3c001680 con 0x7f3a501402e0 2024-05-22T21:24:33.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 -- 172.21.15.161:0/2348343135 >> v1:172.21.15.161:6789/0 conn(0x7f3a501402e0 legacy=0x7f3a5013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:33.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 --2- 172.21.15.161:0/2348343135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a34005d50 0x7f3a34006150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:33.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 --2- 172.21.15.161:0/2348343135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a5013fe50 0x7f3a340069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:33.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 --2- 172.21.15.161:0/2348343135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a34006f20 0x7f3a340073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:33.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 -- 172.21.15.161:0/2348343135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500cdc00 con 0x7f3a34005d50 2024-05-22T21:24:33.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 -- 172.21.15.161:0/2348343135 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3400dcd0 con 0x7f3a34006f20 2024-05-22T21:24:33.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4d600640 1 -- 172.21.15.161:0/2348343135 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3400de90 con 0x7f3a5013fe50 2024-05-22T21:24:33.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4ea00640 1 --2- 172.21.15.161:0/2348343135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a34005d50 0x7f3a34006150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:33.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4f400640 1 --2- 172.21.15.161:0/2348343135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a34006f20 0x7f3a340073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:33.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4e000640 1 --2- 172.21.15.161:0/2348343135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a5013fe50 0x7f3a340069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:33.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4ea00640 1 -- 172.21.15.161:0/2348343135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a5013fe50 msgr2=0x7f3a340069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:33.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4ea00640 1 --2- 172.21.15.161:0/2348343135 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a5013fe50 0x7f3a340069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:33.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4ea00640 1 -- 172.21.15.161:0/2348343135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a34006f20 msgr2=0x7f3a340073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:33.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4ea00640 1 --2- 172.21.15.161:0/2348343135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a34006f20 0x7f3a340073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:33.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.895+0000 7f3a4ea00640 1 -- 172.21.15.161:0/2348343135 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a34005740 con 0x7f3a34005d50 2024-05-22T21:24:33.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a4ea00640 1 --2- 172.21.15.161:0/2348343135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a34005d50 0x7f3a34006150 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f3a40001fd0 tx=0x7f3a4001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef9e72735871de1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:33.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a4d600640 1 -- 172.21.15.161:0/2348343135 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40003b80 con 0x7f3a34005d50 2024-05-22T21:24:33.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a4d600640 1 -- 172.21.15.161:0/2348343135 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a40003d00 con 0x7f3a34005d50 2024-05-22T21:24:33.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a4d600640 1 -- 172.21.15.161:0/2348343135 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40005310 con 0x7f3a34005d50 2024-05-22T21:24:33.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a55a00640 1 -- 172.21.15.161:0/2348343135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a34005d50 msgr2=0x7f3a34006150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:33.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a55a00640 1 --2- 172.21.15.161:0/2348343135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a34005d50 0x7f3a34006150 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f3a40001fd0 tx=0x7f3a4001c900 comp rx=0 tx=0).stop 2024-05-22T21:24:33.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a4f400640 1 -- 172.21.15.161:0/2348343135 reap_dead start 2024-05-22T21:24:33.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a55a00640 1 -- 172.21.15.161:0/2348343135 shutdown_connections 2024-05-22T21:24:33.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a55a00640 1 -- 172.21.15.161:0/2348343135 >> 172.21.15.161:0/2348343135 conn(0x7f3a5006a2d0 msgr2=0x7f3a5006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:33.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a55a00640 1 -- 172.21.15.161:0/2348343135 shutdown_connections 2024-05-22T21:24:33.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a55a00640 1 -- 172.21.15.161:0/2348343135 wait complete. 2024-05-22T21:24:33.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a55a00640 1 Processor -- start 2024-05-22T21:24:33.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a55a00640 1 -- start start 2024-05-22T21:24:33.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a55a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a501576d0 0x7f3a500afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:33.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a55a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a500b0120 0x7f3a500ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:33.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a55a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a500adae0 0x7f3a5013e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:33.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a55a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500b05a0 con 0x7f3a500b0120 2024-05-22T21:24:33.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a55a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500adf60 con 0x7f3a500adae0 2024-05-22T21:24:33.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a55a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500b13b0 con 0x7f3a501576d0 2024-05-22T21:24:33.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a4e000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a500b0120 0x7f3a500ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:33.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a4e000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a500b0120 0x7f3a500ad5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60648/0 (socket says 172.21.15.161:60648) 2024-05-22T21:24:33.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.899+0000 7f3a4e000640 1 -- 172.21.15.161:0/422379690 learned_addr learned my addr 172.21.15.161:0/422379690 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:33.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.903+0000 7f3a4ea00640 1 --2- 172.21.15.161:0/422379690 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a501576d0 0x7f3a500afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:33.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.903+0000 7f3a4f400640 1 --2- 172.21.15.161:0/422379690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a500adae0 0x7f3a5013e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:33.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.903+0000 7f3a4e000640 1 -- 172.21.15.161:0/422379690 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a501576d0 msgr2=0x7f3a500afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:33.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.903+0000 7f3a4e000640 1 --2- 172.21.15.161:0/422379690 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a501576d0 0x7f3a500afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:33.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.903+0000 7f3a4e000640 1 -- 172.21.15.161:0/422379690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a500adae0 msgr2=0x7f3a5013e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:33.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.903+0000 7f3a4e000640 1 --2- 172.21.15.161:0/422379690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a500adae0 0x7f3a5013e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:33.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.903+0000 7f3a4e000640 1 -- 172.21.15.161:0/422379690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a500b07a0 con 0x7f3a500b0120 2024-05-22T21:24:33.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.903+0000 7f3a4e000640 1 --2- 172.21.15.161:0/422379690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a500b0120 0x7f3a500ad5a0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c007340 tx=0x7f3a3c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:33.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.903+0000 7f3a46a00640 1 -- 172.21.15.161:0/422379690 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c00f9c0 con 0x7f3a500b0120 2024-05-22T21:24:33.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.903+0000 7f3a46a00640 1 -- 172.21.15.161:0/422379690 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c00fb40 con 0x7f3a500b0120 2024-05-22T21:24:33.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.903+0000 7f3a55a00640 1 -- 172.21.15.161:0/422379690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a500b0a80 con 0x7f3a500b0120 2024-05-22T21:24:33.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.903+0000 7f3a46a00640 1 -- 172.21.15.161:0/422379690 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c0133f0 con 0x7f3a500b0120 2024-05-22T21:24:33.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.903+0000 7f3a55a00640 1 -- 172.21.15.161:0/422379690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a500b1040 con 0x7f3a500b0120 2024-05-22T21:24:33.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.903+0000 7f3a55a00640 1 -- 172.21.15.161:0/422379690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a500b1200 con 0x7f3a500b0120 2024-05-22T21:24:33.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.903+0000 7f3a46a00640 1 -- 172.21.15.161:0/422379690 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3a3c013550 con 0x7f3a500b0120 2024-05-22T21:24:33.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.907+0000 7f3a46a00640 1 --2- 172.21.15.161:0/422379690 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a300666c0 0x7f3a30068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:33.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.907+0000 7f3a46a00640 1 -- 172.21.15.161:0/422379690 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3a3c084900 con 0x7f3a500b0120 2024-05-22T21:24:33.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.907+0000 7f3a4ea00640 1 --2- 172.21.15.161:0/422379690 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a300666c0 0x7f3a30068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:33.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.907+0000 7f3a4ea00640 1 --2- 172.21.15.161:0/422379690 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a300666c0 0x7f3a30068b80 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3a400ad000 tx=0x7f3a400a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:33.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:33.907+0000 7f3a46a00640 1 -- 172.21.15.161:0/422379690 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3a3c051cb0 con 0x7f3a500b0120 2024-05-22T21:24:34.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.091+0000 7f3a55a00640 1 -- 172.21.15.161:0/422379690 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3a5006ac20 con 0x7f3a500b0120 2024-05-22T21:24:34.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.099+0000 7f3a46a00640 1 -- 172.21.15.161:0/422379690 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v82) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3a3c0552d0 con 0x7f3a500b0120 2024-05-22T21:24:34.104 INFO:teuthology.orchestra.run.smithi161.stderr:new fs with metadata pool 22 and data pool 23 2024-05-22T21:24:34.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.111+0000 7f3a55a00640 1 -- 172.21.15.161:0/422379690 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a300666c0 msgr2=0x7f3a30068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.111+0000 7f3a55a00640 1 --2- 172.21.15.161:0/422379690 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a300666c0 0x7f3a30068b80 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3a400ad000 tx=0x7f3a400a7000 comp rx=0 tx=0).stop 2024-05-22T21:24:34.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.111+0000 7f3a55a00640 1 -- 172.21.15.161:0/422379690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a500b0120 msgr2=0x7f3a500ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.111+0000 7f3a55a00640 1 --2- 172.21.15.161:0/422379690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a500b0120 0x7f3a500ad5a0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c007340 tx=0x7f3a3c007800 comp rx=0 tx=0).stop 2024-05-22T21:24:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.115+0000 7f3a55a00640 1 -- 172.21.15.161:0/422379690 shutdown_connections 2024-05-22T21:24:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.115+0000 7f3a55a00640 1 --2- 172.21.15.161:0/422379690 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3a300666c0 0x7f3a30068b80 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.115+0000 7f3a55a00640 1 --2- 172.21.15.161:0/422379690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a500adae0 0x7f3a5013e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.115+0000 7f3a55a00640 1 --2- 172.21.15.161:0/422379690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a500b0120 0x7f3a500ad5a0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.115+0000 7f3a55a00640 1 --2- 172.21.15.161:0/422379690 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a501576d0 0x7f3a500afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.115+0000 7f3a55a00640 1 -- 172.21.15.161:0/422379690 >> 172.21.15.161:0/422379690 conn(0x7f3a5006a2d0 msgr2=0x7f3a501592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:34.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.115+0000 7f3a55a00640 1 -- 172.21.15.161:0/422379690 shutdown_connections 2024-05-22T21:24:34.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.115+0000 7f3a55a00640 1 -- 172.21.15.161:0/422379690 wait complete. 2024-05-22T21:24:34.137 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:24:34.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.271+0000 7f0e6da00640 1 Processor -- start 2024-05-22T21:24:34.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.271+0000 7f0e6da00640 1 -- start start 2024-05-22T21:24:34.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.275+0000 7f0e6da00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0e680cb880 0x7f0e680b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.275+0000 7f0e6da00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0e680b2b80 0x7f0e6806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.275+0000 7f0e6da00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6806fb30 con 0x7f0e680b2ff0 2024-05-22T21:24:34.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.275+0000 7f0e6da00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e680b3500 con 0x7f0e680cb880 2024-05-22T21:24:34.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.275+0000 7f0e6da00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e680b37e0 con 0x7f0e680b2b80 2024-05-22T21:24:34.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.275+0000 7f0e67e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0e680cb880 0x7f0e680b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:34.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.275+0000 7f0e67e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0e680cb880 0x7f0e680b0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60650/0 (socket says 172.21.15.161:60650) 2024-05-22T21:24:34.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.275+0000 7f0e67e00640 1 -- 172.21.15.161:0/1977602149 learned_addr learned my addr 172.21.15.161:0/1977602149 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:34.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.275+0000 7f0e6d000640 1 --1- 172.21.15.161:0/1977602149 >> v1:172.21.15.161:6789/0 conn(0x7f0e680b2ff0 0x7f0e6806bdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41904/0 (socket says 172.21.15.161:41904) 2024-05-22T21:24:34.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.275+0000 7f0e5f400640 1 --2- 172.21.15.161:0/1977602149 >> v2:172.21.15.179:3300/0 conn(0x7f0e680b2b80 0x7f0e6806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:34.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.275+0000 7f0e67e00640 1 -- 172.21.15.161:0/1977602149 >> v2:172.21.15.179:3300/0 conn(0x7f0e680b2b80 msgr2=0x7f0e6806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:34.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.275+0000 7f0e67e00640 1 --2- 172.21.15.161:0/1977602149 >> v2:172.21.15.179:3300/0 conn(0x7f0e680b2b80 0x7f0e6806b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.275+0000 7f0e67e00640 1 -- 172.21.15.161:0/1977602149 >> v1:172.21.15.161:6789/0 conn(0x7f0e680b2ff0 legacy=0x7f0e6806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.275+0000 7f0e67e00640 1 -- 172.21.15.161:0/1977602149 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6806fd80 con 0x7f0e680cb880 2024-05-22T21:24:34.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.275+0000 7f0e67400640 1 -- 172.21.15.161:0/1977602149 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333248755 0 0) 0x7f0e6806fb30 con 0x7f0e680b2ff0 2024-05-22T21:24:34.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e67e00640 1 --2- 172.21.15.161:0/1977602149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e680cb880 0x7f0e680b0f90 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f0e6806cfa0 tx=0x7f0e5801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff78402e61c217f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:34.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e67400640 1 -- 172.21.15.161:0/1977602149 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e580ad680 con 0x7f0e680cb880 2024-05-22T21:24:34.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e67400640 1 -- 172.21.15.161:0/1977602149 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e580ad7e0 con 0x7f0e680cb880 2024-05-22T21:24:34.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e67400640 1 -- 172.21.15.161:0/1977602149 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e580adb70 con 0x7f0e680cb880 2024-05-22T21:24:34.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6da00640 1 -- 172.21.15.161:0/1977602149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e680cb880 msgr2=0x7f0e680b0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6da00640 1 --2- 172.21.15.161:0/1977602149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e680cb880 0x7f0e680b0f90 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f0e6806cfa0 tx=0x7f0e5801c460 comp rx=0 tx=0).stop 2024-05-22T21:24:34.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6da00640 1 -- 172.21.15.161:0/1977602149 shutdown_connections 2024-05-22T21:24:34.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6da00640 1 --2- 172.21.15.161:0/1977602149 >> v2:172.21.15.179:3300/0 conn(0x7f0e680b2b80 0x7f0e6806b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6da00640 1 --2- 172.21.15.161:0/1977602149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e680cb880 0x7f0e680b0f90 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6da00640 1 -- 172.21.15.161:0/1977602149 >> 172.21.15.161:0/1977602149 conn(0x7f0e6806a4d0 msgr2=0x7f0e680b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:34.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6da00640 1 -- 172.21.15.161:0/1977602149 shutdown_connections 2024-05-22T21:24:34.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6da00640 1 -- 172.21.15.161:0/1977602149 wait complete. 2024-05-22T21:24:34.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6da00640 1 Processor -- start 2024-05-22T21:24:34.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6da00640 1 -- start start 2024-05-22T21:24:34.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6da00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e680b2b80 0x7f0e68143900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6da00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e680b2ff0 0x7f0e6815b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6da00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e680cb880 0x7f0e6814b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6da00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e68142780 con 0x7f0e680cb880 2024-05-22T21:24:34.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6da00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e680cc1d0 con 0x7f0e680b2b80 2024-05-22T21:24:34.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6da00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e680cc4b0 con 0x7f0e680b2ff0 2024-05-22T21:24:34.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.279+0000 7f0e6d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e680cb880 0x7f0e6814b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:34.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e6d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e680cb880 0x7f0e6814b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60658/0 (socket says 172.21.15.161:60658) 2024-05-22T21:24:34.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e6d000640 1 -- 172.21.15.161:0/1507359891 learned_addr learned my addr 172.21.15.161:0/1507359891 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:34.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e67e00640 1 --2- 172.21.15.161:0/1507359891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e680b2b80 0x7f0e68143900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e6d000640 1 -- 172.21.15.161:0/1507359891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e680b2ff0 msgr2=0x7f0e6815b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e6d000640 1 --2- 172.21.15.161:0/1507359891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e680b2ff0 0x7f0e6815b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e6d000640 1 -- 172.21.15.161:0/1507359891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e680b2b80 msgr2=0x7f0e68143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e6d000640 1 --2- 172.21.15.161:0/1507359891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e680b2b80 0x7f0e68143900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e6d000640 1 -- 172.21.15.161:0/1507359891 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e580094d0 con 0x7f0e680cb880 2024-05-22T21:24:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e67e00640 1 --2- 172.21.15.161:0/1507359891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e680b2b80 0x7f0e68143900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:34.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e6d000640 1 --2- 172.21.15.161:0/1507359891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e680cb880 0x7f0e6814b470 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f0e60001570 tx=0x7f0e6000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:34.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e64c00640 1 -- 172.21.15.161:0/1507359891 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6000a640 con 0x7f0e680cb880 2024-05-22T21:24:34.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e64c00640 1 -- 172.21.15.161:0/1507359891 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6000a7c0 con 0x7f0e680cb880 2024-05-22T21:24:34.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e6da00640 1 -- 172.21.15.161:0/1507359891 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e680cc790 con 0x7f0e680cb880 2024-05-22T21:24:34.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e64c00640 1 -- 172.21.15.161:0/1507359891 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6000ab70 con 0x7f0e680cb880 2024-05-22T21:24:34.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e6da00640 1 -- 172.21.15.161:0/1507359891 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e680ccd50 con 0x7f0e680cb880 2024-05-22T21:24:34.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e6da00640 1 -- 172.21.15.161:0/1507359891 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e680ccf70 con 0x7f0e680cb880 2024-05-22T21:24:34.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e64c00640 1 -- 172.21.15.161:0/1507359891 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0e6000acd0 con 0x7f0e680cb880 2024-05-22T21:24:34.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e64c00640 1 --2- 172.21.15.161:0/1507359891 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e44066690 0x7f0e44068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e64c00640 1 -- 172.21.15.161:0/1507359891 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e60083830 con 0x7f0e680cb880 2024-05-22T21:24:34.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.283+0000 7f0e67e00640 1 --2- 172.21.15.161:0/1507359891 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e44066690 0x7f0e44068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:34.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.287+0000 7f0e67e00640 1 --2- 172.21.15.161:0/1507359891 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e44066690 0x7f0e44068b50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0e580af000 tx=0x7f0e58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:34.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.287+0000 7f0e64c00640 1 -- 172.21.15.161:0/1507359891 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0e60050b00 con 0x7f0e680cb880 2024-05-22T21:24:34.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.443+0000 7f0e6da00640 1 -- 172.21.15.161:0/1507359891 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e68067070 con 0x7f0e680cb880 2024-05-22T21:24:34.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.443+0000 7f0e64c00640 1 -- 172.21.15.161:0/1507359891 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+16165 (secure 0 0 0) 0x7f0e60018020 con 0x7f0e680cb880 2024-05-22T21:24:34.445 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:34.445 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":189,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:24:34.096309+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:24:31.797440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-05-22T21:24:33.098240+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:24:34.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.447+0000 7f0e6da00640 1 -- 172.21.15.161:0/1507359891 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e44066690 msgr2=0x7f0e44068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.447+0000 7f0e6da00640 1 --2- 172.21.15.161:0/1507359891 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e44066690 0x7f0e44068b50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0e580af000 tx=0x7f0e58008040 comp rx=0 tx=0).stop 2024-05-22T21:24:34.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.447+0000 7f0e6da00640 1 -- 172.21.15.161:0/1507359891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e680cb880 msgr2=0x7f0e6814b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.447+0000 7f0e6da00640 1 --2- 172.21.15.161:0/1507359891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e680cb880 0x7f0e6814b470 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f0e60001570 tx=0x7f0e6000cca0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.447+0000 7f0e6da00640 1 -- 172.21.15.161:0/1507359891 shutdown_connections 2024-05-22T21:24:34.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.447+0000 7f0e6da00640 1 --2- 172.21.15.161:0/1507359891 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0e44066690 0x7f0e44068b50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.447+0000 7f0e6da00640 1 --2- 172.21.15.161:0/1507359891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e680cb880 0x7f0e6814b470 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.447+0000 7f0e6da00640 1 --2- 172.21.15.161:0/1507359891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0e680b2ff0 0x7f0e6815b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.447+0000 7f0e6da00640 1 --2- 172.21.15.161:0/1507359891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e680b2b80 0x7f0e68143900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.447+0000 7f0e6da00640 1 -- 172.21.15.161:0/1507359891 >> 172.21.15.161:0/1507359891 conn(0x7f0e6806a4d0 msgr2=0x7f0e68156df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:34.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.447+0000 7f0e6da00640 1 -- 172.21.15.161:0/1507359891 shutdown_connections 2024-05-22T21:24:34.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.447+0000 7f0e6da00640 1 -- 172.21.15.161:0/1507359891 wait complete. 2024-05-22T21:24:34.467 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:24:34.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.603+0000 7f7d3d000640 1 Processor -- start 2024-05-22T21:24:34.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.603+0000 7f7d3d000640 1 -- start start 2024-05-22T21:24:34.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.603+0000 7f7d3d000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7d3815dd50 0x7f7d3806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.603+0000 7f7d3d000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7d3815e130 0x7f7d3815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.603+0000 7f7d3d000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d38071670 con 0x7f7d3815d410 2024-05-22T21:24:34.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.603+0000 7f7d3d000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3815d8a0 con 0x7f7d3815e130 2024-05-22T21:24:34.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.603+0000 7f7d3d000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3806de40 con 0x7f7d3815dd50 2024-05-22T21:24:34.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.603+0000 7f7d36800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f7d3815d410 0x7f7d3815a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:41918/0 (socket says 172.21.15.161:41918) 2024-05-22T21:24:34.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.603+0000 7f7d36800640 1 -- 172.21.15.161:0/2174038441 learned_addr learned my addr 172.21.15.161:0/2174038441 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:34.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.603+0000 7f7d2fe00640 1 --2- 172.21.15.161:0/2174038441 >> v2:172.21.15.179:3300/0 conn(0x7f7d3815e130 0x7f7d3815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:34.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.603+0000 7f7d35e00640 1 --2- 172.21.15.161:0/2174038441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7d3815dd50 0x7f7d3806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:34.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.603+0000 7f7d35400640 1 -- 172.21.15.161:0/2174038441 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3916999393 0 0) 0x7f7d38071670 con 0x7f7d3815d410 2024-05-22T21:24:34.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.603+0000 7f7d35400640 1 -- 172.21.15.161:0/2174038441 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d1c005740 con 0x7f7d3815d410 2024-05-22T21:24:34.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.603+0000 7f7d35400640 1 -- 172.21.15.161:0/2174038441 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3692000608 0 0) 0x7f7d1c005740 con 0x7f7d3815d410 2024-05-22T21:24:34.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.603+0000 7f7d2fe00640 1 -- 172.21.15.161:0/2174038441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7d3815dd50 msgr2=0x7f7d3806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:34.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.607+0000 7f7d2fe00640 1 --2- 172.21.15.161:0/2174038441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7d3815dd50 0x7f7d3806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.607+0000 7f7d2fe00640 1 -- 172.21.15.161:0/2174038441 >> v1:172.21.15.161:6789/0 conn(0x7f7d3815d410 legacy=0x7f7d3815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.607+0000 7f7d2fe00640 1 -- 172.21.15.161:0/2174038441 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d38070a60 con 0x7f7d3815e130 2024-05-22T21:24:34.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.607+0000 7f7d2fe00640 1 --2- 172.21.15.161:0/2174038441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d3815e130 0x7f7d3815ced0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7d200097c0 tx=0x7f7d2001c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16f1d7506316c7f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:34.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.607+0000 7f7d35400640 1 -- 172.21.15.161:0/2174038441 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30001a00 con 0x7f7d3815e130 2024-05-22T21:24:34.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.607+0000 7f7d35400640 1 -- 172.21.15.161:0/2174038441 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d200a7050 con 0x7f7d3815e130 2024-05-22T21:24:34.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.607+0000 7f7d35400640 1 -- 172.21.15.161:0/2174038441 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d200ab620 con 0x7f7d3815e130 2024-05-22T21:24:34.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.607+0000 7f7d3d000640 1 -- 172.21.15.161:0/2174038441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d3815e130 msgr2=0x7f7d3815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.607+0000 7f7d3d000640 1 --2- 172.21.15.161:0/2174038441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d3815e130 0x7f7d3815ced0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7d200097c0 tx=0x7f7d2001c300 comp rx=0 tx=0).stop 2024-05-22T21:24:34.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.607+0000 7f7d3d000640 1 -- 172.21.15.161:0/2174038441 shutdown_connections 2024-05-22T21:24:34.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.607+0000 7f7d3d000640 1 --2- 172.21.15.161:0/2174038441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d3815e130 0x7f7d3815ced0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.607+0000 7f7d3d000640 1 --2- 172.21.15.161:0/2174038441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7d3815dd50 0x7f7d3806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.607+0000 7f7d3d000640 1 -- 172.21.15.161:0/2174038441 >> 172.21.15.161:0/2174038441 conn(0x7f7d380b0bc0 msgr2=0x7f7d380b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:34.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.607+0000 7f7d3d000640 1 -- 172.21.15.161:0/2174038441 shutdown_connections 2024-05-22T21:24:34.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d3d000640 1 -- 172.21.15.161:0/2174038441 wait complete. 2024-05-22T21:24:34.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d3d000640 1 Processor -- start 2024-05-22T21:24:34.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d3d000640 1 -- start start 2024-05-22T21:24:34.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d3d000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7d3815d410 0x7f7d380b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d3d000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d3815dd50 0x7f7d38156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d3d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d3815e130 0x7f7d381573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d3d000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d38150f70 con 0x7f7d3815e130 2024-05-22T21:24:34.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d3d000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d38154b50 con 0x7f7d3815dd50 2024-05-22T21:24:34.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d3d000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d381555f0 con 0x7f7d3815d410 2024-05-22T21:24:34.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d36800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d3815e130 0x7f7d381573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:34.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d36800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d3815e130 0x7f7d381573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60662/0 (socket says 172.21.15.161:60662) 2024-05-22T21:24:34.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d36800640 1 -- 172.21.15.161:0/3242826131 learned_addr learned my addr 172.21.15.161:0/3242826131 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:34.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d35e00640 1 --2- 172.21.15.161:0/3242826131 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7d3815d410 0x7f7d380b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:34.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d2fe00640 1 --2- 172.21.15.161:0/3242826131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d3815dd50 0x7f7d38156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:34.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d36800640 1 -- 172.21.15.161:0/3242826131 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7d3815d410 msgr2=0x7f7d380b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d36800640 1 --2- 172.21.15.161:0/3242826131 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7d3815d410 0x7f7d380b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d36800640 1 -- 172.21.15.161:0/3242826131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d3815dd50 msgr2=0x7f7d38156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d36800640 1 --2- 172.21.15.161:0/3242826131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d3815dd50 0x7f7d38156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d36800640 1 -- 172.21.15.161:0/3242826131 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20009420 con 0x7f7d3815e130 2024-05-22T21:24:34.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d36800640 1 --2- 172.21.15.161:0/3242826131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d3815e130 0x7f7d381573d0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000cca0 tx=0x7f7d3000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:34.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d2e000640 1 -- 172.21.15.161:0/3242826131 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30012400 con 0x7f7d3815e130 2024-05-22T21:24:34.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d2e000640 1 -- 172.21.15.161:0/3242826131 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d30012560 con 0x7f7d3815e130 2024-05-22T21:24:34.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d3d000640 1 -- 172.21.15.161:0/3242826131 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d381558d0 con 0x7f7d3815e130 2024-05-22T21:24:34.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d2e000640 1 -- 172.21.15.161:0/3242826131 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30012910 con 0x7f7d3815e130 2024-05-22T21:24:34.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.611+0000 7f7d3d000640 1 -- 172.21.15.161:0/3242826131 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d38155e90 con 0x7f7d3815e130 2024-05-22T21:24:34.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.615+0000 7f7d3d000640 1 -- 172.21.15.161:0/3242826131 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d380706b0 con 0x7f7d3815e130 2024-05-22T21:24:34.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.615+0000 7f7d2e000640 1 -- 172.21.15.161:0/3242826131 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7d30007070 con 0x7f7d3815e130 2024-05-22T21:24:34.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.615+0000 7f7d2e000640 1 --2- 172.21.15.161:0/3242826131 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7d14066690 0x7f7d14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.615+0000 7f7d2e000640 1 -- 172.21.15.161:0/3242826131 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d300843c0 con 0x7f7d3815e130 2024-05-22T21:24:34.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.615+0000 7f7d35e00640 1 --2- 172.21.15.161:0/3242826131 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7d14066690 0x7f7d14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:34.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.615+0000 7f7d35e00640 1 --2- 172.21.15.161:0/3242826131 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7d14066690 0x7f7d14068b50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7d38159c50 tx=0x7f7d28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:34.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.619+0000 7f7d2e000640 1 -- 172.21.15.161:0/3242826131 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7d30051690 con 0x7f7d3815e130 2024-05-22T21:24:34.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.775+0000 7f7d3d000640 1 -- 172.21.15.161:0/3242826131 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d38067220 con 0x7f7d3815e130 2024-05-22T21:24:34.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.775+0000 7f7d2e000640 1 -- 172.21.15.161:0/3242826131 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+16165 (secure 0 0 0) 0x7f7d30054cb0 con 0x7f7d3815e130 2024-05-22T21:24:34.777 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:34.777 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":189,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:24:34.096309+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:24:31.797440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-05-22T21:24:33.098240+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:24:34.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.779+0000 7f7d3d000640 1 -- 172.21.15.161:0/3242826131 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7d14066690 msgr2=0x7f7d14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.779+0000 7f7d3d000640 1 --2- 172.21.15.161:0/3242826131 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7d14066690 0x7f7d14068b50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7d38159c50 tx=0x7f7d28008040 comp rx=0 tx=0).stop 2024-05-22T21:24:34.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.779+0000 7f7d3d000640 1 -- 172.21.15.161:0/3242826131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d3815e130 msgr2=0x7f7d381573d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.779+0000 7f7d3d000640 1 --2- 172.21.15.161:0/3242826131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d3815e130 0x7f7d381573d0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000cca0 tx=0x7f7d3000a430 comp rx=0 tx=0).stop 2024-05-22T21:24:34.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.779+0000 7f7d3d000640 1 -- 172.21.15.161:0/3242826131 shutdown_connections 2024-05-22T21:24:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.779+0000 7f7d3d000640 1 --2- 172.21.15.161:0/3242826131 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7d14066690 0x7f7d14068b50 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.779+0000 7f7d3d000640 1 --2- 172.21.15.161:0/3242826131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d3815e130 0x7f7d381573d0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.779+0000 7f7d3d000640 1 --2- 172.21.15.161:0/3242826131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d3815dd50 0x7f7d38156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.779+0000 7f7d3d000640 1 --2- 172.21.15.161:0/3242826131 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7d3815d410 0x7f7d380b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.779+0000 7f7d3d000640 1 -- 172.21.15.161:0/3242826131 >> 172.21.15.161:0/3242826131 conn(0x7f7d380b0bc0 msgr2=0x7f7d380ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.779+0000 7f7d3d000640 1 -- 172.21.15.161:0/3242826131 shutdown_connections 2024-05-22T21:24:34.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.779+0000 7f7d3d000640 1 -- 172.21.15.161:0/3242826131 wait complete. 2024-05-22T21:24:34.799 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-22T21:24:34.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.935+0000 7fc255a00640 1 Processor -- start 2024-05-22T21:24:34.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.935+0000 7fc255a00640 1 -- start start 2024-05-22T21:24:34.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.939+0000 7fc255a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc25013cfc0 0x7fc2501402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.939+0000 7fc255a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc25013d3a0 con 0x7fc25013fe50 2024-05-22T21:24:34.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.939+0000 7fc255a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2500cd920 con 0x7fc25015af60 2024-05-22T21:24:34.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.939+0000 7fc255a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2500cdc00 con 0x7fc25013cfc0 2024-05-22T21:24:34.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.939+0000 7fc24ea00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc25013cfc0 0x7fc2501402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:34.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.939+0000 7fc24ea00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc25013cfc0 0x7fc2501402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60676/0 (socket says 172.21.15.161:60676) 2024-05-22T21:24:34.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.939+0000 7fc24ea00640 1 -- 172.21.15.161:0/89272475 learned_addr learned my addr 172.21.15.161:0/89272475 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:34.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.939+0000 7fc24ea00640 1 -- 172.21.15.161:0/89272475 >> v1:172.21.15.179:6789/0 conn(0x7fc25015af60 legacy=0x7fc25013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.939+0000 7fc24ea00640 1 -- 172.21.15.161:0/89272475 >> v1:172.21.15.161:6789/0 conn(0x7fc25013fe50 legacy=0x7fc25006fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.939+0000 7fc24ea00640 1 -- 172.21.15.161:0/89272475 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2500cdee0 con 0x7fc25013cfc0 2024-05-22T21:24:34.940 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.939+0000 7fc24e000640 1 -- 172.21.15.161:0/89272475 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338663504 0 0) 0x7fc25013d3a0 con 0x7fc25013fe50 2024-05-22T21:24:34.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc24ea00640 1 --2- 172.21.15.161:0/89272475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25013cfc0 0x7fc2501402e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fc240009870 tx=0x7fc24001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98554293a9f1085 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:34.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc24e000640 1 -- 172.21.15.161:0/89272475 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2400a6040 con 0x7fc25013cfc0 2024-05-22T21:24:34.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc24e000640 1 -- 172.21.15.161:0/89272475 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2400aa400 con 0x7fc25013cfc0 2024-05-22T21:24:34.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc24e000640 1 -- 172.21.15.161:0/89272475 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2400aa790 con 0x7fc25013cfc0 2024-05-22T21:24:34.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc255a00640 1 -- 172.21.15.161:0/89272475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25013cfc0 msgr2=0x7fc2501402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc255a00640 1 --2- 172.21.15.161:0/89272475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25013cfc0 0x7fc2501402e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fc240009870 tx=0x7fc24001c4a0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc255a00640 1 -- 172.21.15.161:0/89272475 shutdown_connections 2024-05-22T21:24:34.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc255a00640 1 --2- 172.21.15.161:0/89272475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25013cfc0 0x7fc2501402e0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc255a00640 1 -- 172.21.15.161:0/89272475 >> 172.21.15.161:0/89272475 conn(0x7fc25006a2d0 msgr2=0x7fc25006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:34.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc255a00640 1 -- 172.21.15.161:0/89272475 shutdown_connections 2024-05-22T21:24:34.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc255a00640 1 -- 172.21.15.161:0/89272475 wait complete. 2024-05-22T21:24:34.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc255a00640 1 Processor -- start 2024-05-22T21:24:34.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc255a00640 1 -- start start 2024-05-22T21:24:34.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc255a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc25013cfc0 0x7fc25015c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc255a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25013fe50 0x7fc250154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc255a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc25015af60 0x7fc2501551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc255a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc250155730 con 0x7fc25013fe50 2024-05-22T21:24:34.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc255a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25013db60 con 0x7fc25013cfc0 2024-05-22T21:24:34.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc255a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25015c8c0 con 0x7fc25015af60 2024-05-22T21:24:34.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc246000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25013fe50 0x7fc250154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:34.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc246000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25013fe50 0x7fc250154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60686/0 (socket says 172.21.15.161:60686) 2024-05-22T21:24:34.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc246000640 1 -- 172.21.15.161:0/1411001001 learned_addr learned my addr 172.21.15.161:0/1411001001 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:34.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.943+0000 7fc24ea00640 1 --2- 172.21.15.161:0/1411001001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc25013cfc0 0x7fc25015c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:34.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc24f400640 1 --2- 172.21.15.161:0/1411001001 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc25015af60 0x7fc2501551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:34.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc246000640 1 -- 172.21.15.161:0/1411001001 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc25015af60 msgr2=0x7fc2501551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc246000640 1 --2- 172.21.15.161:0/1411001001 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc25015af60 0x7fc2501551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc246000640 1 -- 172.21.15.161:0/1411001001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc25013cfc0 msgr2=0x7fc25015c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:34.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc246000640 1 --2- 172.21.15.161:0/1411001001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc25013cfc0 0x7fc25015c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:34.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc246000640 1 -- 172.21.15.161:0/1411001001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2400094d0 con 0x7fc25013fe50 2024-05-22T21:24:34.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc246000640 1 --2- 172.21.15.161:0/1411001001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25013fe50 0x7fc250154cb0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fc23800c500 tx=0x7fc23800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:34.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc247400640 1 -- 172.21.15.161:0/1411001001 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238005220 con 0x7fc25013fe50 2024-05-22T21:24:34.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc247400640 1 -- 172.21.15.161:0/1411001001 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc238005380 con 0x7fc25013fe50 2024-05-22T21:24:34.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc247400640 1 -- 172.21.15.161:0/1411001001 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238007270 con 0x7fc25013fe50 2024-05-22T21:24:34.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc255a00640 1 -- 172.21.15.161:0/1411001001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25015cba0 con 0x7fc25013fe50 2024-05-22T21:24:34.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc255a00640 1 -- 172.21.15.161:0/1411001001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25013de00 con 0x7fc25013fe50 2024-05-22T21:24:34.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc255a00640 1 -- 172.21.15.161:0/1411001001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25013dff0 con 0x7fc25013fe50 2024-05-22T21:24:34.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc247400640 1 -- 172.21.15.161:0/1411001001 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc2380073d0 con 0x7fc25013fe50 2024-05-22T21:24:34.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc247400640 1 --2- 172.21.15.161:0/1411001001 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc22c066690 0x7fc22c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:34.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc247400640 1 -- 172.21.15.161:0/1411001001 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc238083b20 con 0x7fc25013fe50 2024-05-22T21:24:34.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.947+0000 7fc24ea00640 1 --2- 172.21.15.161:0/1411001001 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc22c066690 0x7fc22c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:34.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.951+0000 7fc24ea00640 1 --2- 172.21.15.161:0/1411001001 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc22c066690 0x7fc22c068b50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc2400ae000 tx=0x7fc2400a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:34.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:34.951+0000 7fc247400640 1 -- 172.21.15.161:0/1411001001 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc238050e00 con 0x7fc25013fe50 2024-05-22T21:24:35.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:35.139+0000 7fc255a00640 1 -- 172.21.15.161:0/1411001001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc250067220 con 0x7fc25013fe50 2024-05-22T21:24:36.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.111+0000 7fc247400640 1 -- 172.21.15.161:0/1411001001 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v85) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc238054420 con 0x7fc25013fe50 2024-05-22T21:24:36.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.111+0000 7fc255a00640 1 -- 172.21.15.161:0/1411001001 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc22c066690 msgr2=0x7fc22c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:36.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.111+0000 7fc255a00640 1 --2- 172.21.15.161:0/1411001001 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc22c066690 0x7fc22c068b50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc2400ae000 tx=0x7fc2400a7000 comp rx=0 tx=0).stop 2024-05-22T21:24:36.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.111+0000 7fc255a00640 1 -- 172.21.15.161:0/1411001001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25013fe50 msgr2=0x7fc250154cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:36.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.111+0000 7fc255a00640 1 --2- 172.21.15.161:0/1411001001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25013fe50 0x7fc250154cb0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fc23800c500 tx=0x7fc23800c9c0 comp rx=0 tx=0).stop 2024-05-22T21:24:36.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.111+0000 7fc255a00640 1 -- 172.21.15.161:0/1411001001 shutdown_connections 2024-05-22T21:24:36.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.111+0000 7fc255a00640 1 --2- 172.21.15.161:0/1411001001 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc22c066690 0x7fc22c068b50 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:36.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.111+0000 7fc255a00640 1 --2- 172.21.15.161:0/1411001001 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc25015af60 0x7fc2501551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:36.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.111+0000 7fc255a00640 1 --2- 172.21.15.161:0/1411001001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc25013fe50 0x7fc250154cb0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:36.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.111+0000 7fc255a00640 1 --2- 172.21.15.161:0/1411001001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc25013cfc0 0x7fc25015c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:36.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.111+0000 7fc255a00640 1 -- 172.21.15.161:0/1411001001 >> 172.21.15.161:0/1411001001 conn(0x7fc25006a2d0 msgr2=0x7fc25006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:36.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.111+0000 7fc255a00640 1 -- 172.21.15.161:0/1411001001 shutdown_connections 2024-05-22T21:24:36.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.111+0000 7fc255a00640 1 -- 172.21.15.161:0/1411001001 wait complete. 2024-05-22T21:24:36.134 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-22T21:24:36.134 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-22T21:24:36.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.275+0000 7f6b3a000640 1 Processor -- start 2024-05-22T21:24:36.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.275+0000 7f6b3a000640 1 -- start start 2024-05-22T21:24:36.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.275+0000 7f6b3a000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b3413cfc0 0x7f6b341402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:36.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.275+0000 7f6b3a000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6b3413fe50 0x7f6b3406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:36.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.275+0000 7f6b3a000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b3413d3a0 con 0x7f6b3415af60 2024-05-22T21:24:36.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.275+0000 7f6b3a000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b340cd920 con 0x7f6b3413fe50 2024-05-22T21:24:36.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.275+0000 7f6b3a000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b340cdc00 con 0x7f6b3413cfc0 2024-05-22T21:24:36.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.275+0000 7f6b33400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b3413cfc0 0x7f6b341402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:36.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.275+0000 7f6b33e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6b3413fe50 0x7f6b3406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:36.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.275+0000 7f6b33400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b3413cfc0 0x7f6b341402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:57924/0 (socket says 172.21.15.161:57924) 2024-05-22T21:24:36.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.275+0000 7f6b33400640 1 -- 172.21.15.161:0/4151172536 learned_addr learned my addr 172.21.15.161:0/4151172536 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:36.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.275+0000 7f6b33e00640 1 -- 172.21.15.161:0/4151172536 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b3413cfc0 msgr2=0x7f6b341402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:36.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.275+0000 7f6b33e00640 1 --2- 172.21.15.161:0/4151172536 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b3413cfc0 0x7f6b341402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:36.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.275+0000 7f6b33e00640 1 -- 172.21.15.161:0/4151172536 >> v1:172.21.15.179:6789/0 conn(0x7f6b3415af60 legacy=0x7f6b3413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:36.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.275+0000 7f6b33e00640 1 -- 172.21.15.161:0/4151172536 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b340cdee0 con 0x7f6b3413fe50 2024-05-22T21:24:36.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.279+0000 7f6b33e00640 1 --2- 172.21.15.161:0/4151172536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b3413fe50 0x7f6b3406fce0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c009d90 tx=0x7f6b2c01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cea56489de451259 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:36.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.279+0000 7f6b32000640 1 -- 172.21.15.161:0/4151172536 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c0a8610 con 0x7f6b3413fe50 2024-05-22T21:24:36.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.279+0000 7f6b32000640 1 -- 172.21.15.161:0/4151172536 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2c0a8770 con 0x7f6b3413fe50 2024-05-22T21:24:36.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.279+0000 7f6b32000640 1 -- 172.21.15.161:0/4151172536 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c0a8b00 con 0x7f6b3413fe50 2024-05-22T21:24:36.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.279+0000 7f6b3a000640 1 -- 172.21.15.161:0/4151172536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b3413fe50 msgr2=0x7f6b3406fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:36.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.279+0000 7f6b3a000640 1 --2- 172.21.15.161:0/4151172536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b3413fe50 0x7f6b3406fce0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c009d90 tx=0x7f6b2c01c8b0 comp rx=0 tx=0).stop 2024-05-22T21:24:36.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.279+0000 7f6b3a000640 1 -- 172.21.15.161:0/4151172536 shutdown_connections 2024-05-22T21:24:36.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.279+0000 7f6b3a000640 1 --2- 172.21.15.161:0/4151172536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b3413fe50 0x7f6b3406fce0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:36.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.279+0000 7f6b3a000640 1 --2- 172.21.15.161:0/4151172536 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b3413cfc0 0x7f6b341402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:36.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.279+0000 7f6b3a000640 1 -- 172.21.15.161:0/4151172536 >> 172.21.15.161:0/4151172536 conn(0x7f6b3406a2d0 msgr2=0x7f6b3406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:36.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.279+0000 7f6b3a000640 1 -- 172.21.15.161:0/4151172536 shutdown_connections 2024-05-22T21:24:36.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.279+0000 7f6b3a000640 1 -- 172.21.15.161:0/4151172536 wait complete. 2024-05-22T21:24:36.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b3a000640 1 Processor -- start 2024-05-22T21:24:36.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b3a000640 1 -- start start 2024-05-22T21:24:36.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b3a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b3413cfc0 0x7f6b34154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:36.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b3a000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b3413fe50 0x7f6b34141a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:36.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b3a000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b3415af60 0x7f6b3413ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:36.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b3a000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b341417c0 con 0x7f6b3413cfc0 2024-05-22T21:24:36.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b3a000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b34140f20 con 0x7f6b3413fe50 2024-05-22T21:24:36.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b3a000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b34141200 con 0x7f6b3415af60 2024-05-22T21:24:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b33400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b3413cfc0 0x7f6b34154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b33400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b3413cfc0 0x7f6b34154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59644/0 (socket says 172.21.15.161:59644) 2024-05-22T21:24:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b33400640 1 -- 172.21.15.161:0/74045287 learned_addr learned my addr 172.21.15.161:0/74045287 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b33e00640 1 --2- 172.21.15.161:0/74045287 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b3415af60 0x7f6b3413ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b32a00640 1 --2- 172.21.15.161:0/74045287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b3413fe50 0x7f6b34141a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b33400640 1 -- 172.21.15.161:0/74045287 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b3415af60 msgr2=0x7f6b3413ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b33400640 1 --2- 172.21.15.161:0/74045287 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b3415af60 0x7f6b3413ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b33400640 1 -- 172.21.15.161:0/74045287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b3413fe50 msgr2=0x7f6b34141a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b33400640 1 --2- 172.21.15.161:0/74045287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b3413fe50 0x7f6b34141a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b33400640 1 -- 172.21.15.161:0/74045287 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c009a70 con 0x7f6b3413cfc0 2024-05-22T21:24:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b33e00640 1 --2- 172.21.15.161:0/74045287 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b3415af60 0x7f6b3413ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:36.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b33400640 1 --2- 172.21.15.161:0/74045287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b3413cfc0 0x7f6b34154bf0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f6b24010d40 tx=0x7f6b2400e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:36.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b2b400640 1 -- 172.21.15.161:0/74045287 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400b8c0 con 0x7f6b3413cfc0 2024-05-22T21:24:36.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b2b400640 1 -- 172.21.15.161:0/74045287 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b24011650 con 0x7f6b3413cfc0 2024-05-22T21:24:36.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b2b400640 1 -- 172.21.15.161:0/74045287 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24011a00 con 0x7f6b3413cfc0 2024-05-22T21:24:36.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b3a000640 1 -- 172.21.15.161:0/74045287 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b341414e0 con 0x7f6b3413cfc0 2024-05-22T21:24:36.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.283+0000 7f6b3a000640 1 -- 172.21.15.161:0/74045287 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b34155130 con 0x7f6b3413cfc0 2024-05-22T21:24:36.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.287+0000 7f6b3a000640 1 -- 172.21.15.161:0/74045287 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b34067220 con 0x7f6b3413cfc0 2024-05-22T21:24:36.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.287+0000 7f6b2b400640 1 -- 172.21.15.161:0/74045287 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6b24011b60 con 0x7f6b3413cfc0 2024-05-22T21:24:36.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.287+0000 7f6b2b400640 1 --2- 172.21.15.161:0/74045287 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6b14066690 0x7f6b14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:36.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.287+0000 7f6b2b400640 1 -- 172.21.15.161:0/74045287 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b24086f50 con 0x7f6b3413cfc0 2024-05-22T21:24:36.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.287+0000 7f6b32a00640 1 --2- 172.21.15.161:0/74045287 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6b14066690 0x7f6b14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:36.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.287+0000 7f6b32a00640 1 --2- 172.21.15.161:0/74045287 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6b14066690 0x7f6b14068b50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6b340b0ac0 tx=0x7f6b20009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:36.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.291+0000 7f6b2b400640 1 -- 172.21.15.161:0/74045287 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6b240542d0 con 0x7f6b3413cfc0 2024-05-22T21:24:36.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:36.479+0000 7f6b3a000640 1 -- 172.21.15.161:0/74045287 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6b3413b160 con 0x7f6b3413cfc0 2024-05-22T21:24:37.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.115+0000 7f6b2b400640 1 -- 172.21.15.161:0/74045287 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v86) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6b2401e0c0 con 0x7f6b3413cfc0 2024-05-22T21:24:37.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.119+0000 7f6b3a000640 1 -- 172.21.15.161:0/74045287 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6b14066690 msgr2=0x7f6b14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.119+0000 7f6b3a000640 1 --2- 172.21.15.161:0/74045287 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6b14066690 0x7f6b14068b50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6b340b0ac0 tx=0x7f6b20009040 comp rx=0 tx=0).stop 2024-05-22T21:24:37.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.119+0000 7f6b3a000640 1 -- 172.21.15.161:0/74045287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b3413cfc0 msgr2=0x7f6b34154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.119+0000 7f6b3a000640 1 --2- 172.21.15.161:0/74045287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b3413cfc0 0x7f6b34154bf0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f6b24010d40 tx=0x7f6b2400e730 comp rx=0 tx=0).stop 2024-05-22T21:24:37.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.119+0000 7f6b3a000640 1 -- 172.21.15.161:0/74045287 shutdown_connections 2024-05-22T21:24:37.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.119+0000 7f6b3a000640 1 --2- 172.21.15.161:0/74045287 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6b14066690 0x7f6b14068b50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.119+0000 7f6b3a000640 1 --2- 172.21.15.161:0/74045287 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6b3415af60 0x7f6b3413ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.119+0000 7f6b3a000640 1 --2- 172.21.15.161:0/74045287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b3413fe50 0x7f6b34141a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.119+0000 7f6b3a000640 1 --2- 172.21.15.161:0/74045287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b3413cfc0 0x7f6b34154bf0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.119+0000 7f6b3a000640 1 -- 172.21.15.161:0/74045287 >> 172.21.15.161:0/74045287 conn(0x7f6b3406a2d0 msgr2=0x7f6b340cc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:37.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.119+0000 7f6b3a000640 1 -- 172.21.15.161:0/74045287 shutdown_connections 2024-05-22T21:24:37.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.119+0000 7f6b3a000640 1 -- 172.21.15.161:0/74045287 wait complete. 2024-05-22T21:24:37.141 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-22T21:24:37.141 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:24:37.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31f8400640 1 Processor -- start 2024-05-22T21:24:37.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31f8400640 1 -- start start 2024-05-22T21:24:37.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31f8400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f31f015e130 0x7f31f015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31f8400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f0071670 con 0x7f31f015dd50 2024-05-22T21:24:37.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31f8400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f015d870 con 0x7f31f015d410 2024-05-22T21:24:37.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31f8400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f0070a60 con 0x7f31f015e130 2024-05-22T21:24:37.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31f5a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f31f015dd50 0x7f31f006d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59110/0 (socket says 172.21.15.161:59110) 2024-05-22T21:24:37.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31f5a00640 1 -- 172.21.15.161:0/1776567048 learned_addr learned my addr 172.21.15.161:0/1776567048 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:37.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31f5000640 1 --2- 172.21.15.161:0/1776567048 >> v2:172.21.15.179:3300/0 conn(0x7f31f015e130 0x7f31f015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31efe00640 1 -- 172.21.15.161:0/1776567048 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966677702 0 0) 0x7f31f0071670 con 0x7f31f015dd50 2024-05-22T21:24:37.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31efe00640 1 -- 172.21.15.161:0/1776567048 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d8003530 con 0x7f31f015dd50 2024-05-22T21:24:37.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31efe00640 1 -- 172.21.15.161:0/1776567048 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2544259831 0 0) 0x7f31d8003530 con 0x7f31f015dd50 2024-05-22T21:24:37.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31efe00640 1 -- 172.21.15.161:0/1776567048 >> v2:172.21.15.179:3300/0 conn(0x7f31f015e130 msgr2=0x7f31f015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:37.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31efe00640 1 --2- 172.21.15.161:0/1776567048 >> v2:172.21.15.179:3300/0 conn(0x7f31f015e130 0x7f31f015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31efe00640 1 -- 172.21.15.161:0/1776567048 >> v1:172.21.15.179:6789/0 conn(0x7f31f015d410 legacy=0x7f31f015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.282 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31efe00640 1 -- 172.21.15.161:0/1776567048 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f0070d40 con 0x7f31f015dd50 2024-05-22T21:24:37.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31efe00640 1 -- 172.21.15.161:0/1776567048 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f31e4001680 con 0x7f31f015dd50 2024-05-22T21:24:37.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31efe00640 1 -- 172.21.15.161:0/1776567048 >> v1:172.21.15.161:6789/0 conn(0x7f31f015dd50 legacy=0x7f31f006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31efe00640 1 --2- 172.21.15.161:0/1776567048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d8003980 0x7f31d8003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31efe00640 1 --2- 172.21.15.161:0/1776567048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31d8004380 0x7f31d8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31efe00640 1 --2- 172.21.15.161:0/1776567048 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f31d8005050 0x7f31d800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31efe00640 1 -- 172.21.15.161:0/1776567048 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f0070a60 con 0x7f31d8003980 2024-05-22T21:24:37.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31efe00640 1 -- 172.21.15.161:0/1776567048 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d800bfb0 con 0x7f31d8004380 2024-05-22T21:24:37.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31efe00640 1 -- 172.21.15.161:0/1776567048 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d800e380 con 0x7f31d8005050 2024-05-22T21:24:37.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31f5000640 1 --2- 172.21.15.161:0/1776567048 >> v2:172.21.15.179:3300/0 conn(0x7f31f015e130 0x7f31f015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:24:37.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.279+0000 7f31f5a00640 1 --2- 172.21.15.161:0/1776567048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d8003980 0x7f31d8003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31f6400640 1 --2- 172.21.15.161:0/1776567048 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f31d8005050 0x7f31d800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31f5000640 1 --2- 172.21.15.161:0/1776567048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31d8004380 0x7f31d8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31f5a00640 1 -- 172.21.15.161:0/1776567048 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f31d8005050 msgr2=0x7f31d800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:37.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31f5a00640 1 --2- 172.21.15.161:0/1776567048 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f31d8005050 0x7f31d800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31f5a00640 1 -- 172.21.15.161:0/1776567048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31d8004380 msgr2=0x7f31d8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:37.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31f5a00640 1 --2- 172.21.15.161:0/1776567048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31d8004380 0x7f31d8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31f5a00640 1 -- 172.21.15.161:0/1776567048 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d8003530 con 0x7f31d8003980 2024-05-22T21:24:37.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31f6400640 1 -- 172.21.15.161:0/1776567048 reap_dead start 2024-05-22T21:24:37.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31f5a00640 1 --2- 172.21.15.161:0/1776567048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d8003980 0x7f31d8003d60 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f31e400c210 tx=0x7f31e401cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3382c6eccfd0e3c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:37.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31efe00640 1 -- 172.21.15.161:0/1776567048 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e40058e0 con 0x7f31d8003980 2024-05-22T21:24:37.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31efe00640 1 -- 172.21.15.161:0/1776567048 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e4005a60 con 0x7f31d8003980 2024-05-22T21:24:37.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31efe00640 1 -- 172.21.15.161:0/1776567048 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e4005e50 con 0x7f31d8003980 2024-05-22T21:24:37.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31f8400640 1 -- 172.21.15.161:0/1776567048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d8003980 msgr2=0x7f31d8003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31f8400640 1 --2- 172.21.15.161:0/1776567048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d8003980 0x7f31d8003d60 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f31e400c210 tx=0x7f31e401cc50 comp rx=0 tx=0).stop 2024-05-22T21:24:37.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31f8400640 1 -- 172.21.15.161:0/1776567048 shutdown_connections 2024-05-22T21:24:37.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31f8400640 1 --2- 172.21.15.161:0/1776567048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d8003980 0x7f31d8003d60 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31f8400640 1 -- 172.21.15.161:0/1776567048 >> 172.21.15.161:0/1776567048 conn(0x7f31f00b0bc0 msgr2=0x7f31f00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:37.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31f8400640 1 -- 172.21.15.161:0/1776567048 shutdown_connections 2024-05-22T21:24:37.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.283+0000 7f31f8400640 1 -- 172.21.15.161:0/1776567048 wait complete. 2024-05-22T21:24:37.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f8400640 1 Processor -- start 2024-05-22T21:24:37.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f8400640 1 -- start start 2024-05-22T21:24:37.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f8400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31d8003980 0x7f31f0144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f8400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f31f00cdae0 0x7f31f01f1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f8400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31f00cdf60 0x7f31f0151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f8400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00b3fc0 con 0x7f31f00cdf60 2024-05-22T21:24:37.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f8400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00cd320 con 0x7f31d8003980 2024-05-22T21:24:37.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f8400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00b3730 con 0x7f31f00cdae0 2024-05-22T21:24:37.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f6400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31f00cdf60 0x7f31f0151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f6400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31f00cdf60 0x7f31f0151a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59668/0 (socket says 172.21.15.161:59668) 2024-05-22T21:24:37.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f6400640 1 -- 172.21.15.161:0/3990826316 learned_addr learned my addr 172.21.15.161:0/3990826316 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f5a00640 1 --2- 172.21.15.161:0/3990826316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31d8003980 0x7f31f0144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f5000640 1 --2- 172.21.15.161:0/3990826316 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f31f00cdae0 0x7f31f01f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f6400640 1 -- 172.21.15.161:0/3990826316 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f31f00cdae0 msgr2=0x7f31f01f1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f6400640 1 --2- 172.21.15.161:0/3990826316 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f31f00cdae0 0x7f31f01f1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f6400640 1 -- 172.21.15.161:0/3990826316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31d8003980 msgr2=0x7f31f0144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f6400640 1 --2- 172.21.15.161:0/3990826316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31d8003980 0x7f31f0144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f6400640 1 -- 172.21.15.161:0/3990826316 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f00b3a00 con 0x7f31f00cdf60 2024-05-22T21:24:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f5000640 1 --2- 172.21.15.161:0/3990826316 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f31f00cdae0 0x7f31f01f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f6400640 1 --2- 172.21.15.161:0/3990826316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31f00cdf60 0x7f31f0151a60 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f31f00721a0 tx=0x7f31e800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31ed600640 1 -- 172.21.15.161:0/3990826316 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8003980 con 0x7f31f00cdf60 2024-05-22T21:24:37.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31ed600640 1 -- 172.21.15.161:0/3990826316 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e8003ae0 con 0x7f31f00cdf60 2024-05-22T21:24:37.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31ed600640 1 -- 172.21.15.161:0/3990826316 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8003e90 con 0x7f31f00cdf60 2024-05-22T21:24:37.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f8400640 1 -- 172.21.15.161:0/3990826316 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f00b3ce0 con 0x7f31f00cdf60 2024-05-22T21:24:37.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.287+0000 7f31f8400640 1 -- 172.21.15.161:0/3990826316 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f01f15b0 con 0x7f31f00cdf60 2024-05-22T21:24:37.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.291+0000 7f31f8400640 1 -- 172.21.15.161:0/3990826316 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f00cd180 con 0x7f31f00cdf60 2024-05-22T21:24:37.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.291+0000 7f31ed600640 1 -- 172.21.15.161:0/3990826316 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f31e80050d0 con 0x7f31f00cdf60 2024-05-22T21:24:37.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.291+0000 7f31ed600640 1 --2- 172.21.15.161:0/3990826316 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f31d0066690 0x7f31d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.291+0000 7f31ed600640 1 -- 172.21.15.161:0/3990826316 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31e80840e0 con 0x7f31f00cdf60 2024-05-22T21:24:37.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.291+0000 7f31f5a00640 1 --2- 172.21.15.161:0/3990826316 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f31d0066690 0x7f31d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.291+0000 7f31f5a00640 1 --2- 172.21.15.161:0/3990826316 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f31d0066690 0x7f31d0068b50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f31e40aa000 tx=0x7f31e4001a10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:37.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.295+0000 7f31ed600640 1 -- 172.21.15.161:0/3990826316 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f31f00cd180 con 0x7f31f00cdf60 2024-05-22T21:24:37.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.479+0000 7f31f8400640 1 -- 172.21.15.161:0/3990826316 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31f00cd180 con 0x7f31f00cdf60 2024-05-22T21:24:37.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.479+0000 7f31ed600640 1 -- 172.21.15.161:0/3990826316 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4744 (secure 0 0 0) 0x7f31f00cd180 con 0x7f31f00cdf60 2024-05-22T21:24:37.480 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:37.481 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":86,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3304182060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3304182060},{"type":"v1","addr":"172.21.15.179:6835","nonce":3304182060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196309272},{"type":"v1","addr":"172.21.15.161:6836","nonce":196309272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/396408969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":396408969},{"type":"v1","addr":"172.21.15.179:6834","nonce":396408969}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:24:34.096673+0000","modified":"2024-05-22T21:24:37.112845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8784},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8784":{"gid":8784,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2172511349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2172511349},{"type":"v1","addr":"172.21.15.161:6837","nonce":2172511349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-05-22T21:24:37.481 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 86 2024-05-22T21:24:37.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.483+0000 7f31f8400640 1 -- 172.21.15.161:0/3990826316 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f31d0066690 msgr2=0x7f31d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.483+0000 7f31f8400640 1 --2- 172.21.15.161:0/3990826316 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f31d0066690 0x7f31d0068b50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f31e40aa000 tx=0x7f31e4001a10 comp rx=0 tx=0).stop 2024-05-22T21:24:37.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.483+0000 7f31f8400640 1 -- 172.21.15.161:0/3990826316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31f00cdf60 msgr2=0x7f31f0151a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.483+0000 7f31f8400640 1 --2- 172.21.15.161:0/3990826316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31f00cdf60 0x7f31f0151a60 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f31f00721a0 tx=0x7f31e800eb90 comp rx=0 tx=0).stop 2024-05-22T21:24:37.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.483+0000 7f31f8400640 1 -- 172.21.15.161:0/3990826316 shutdown_connections 2024-05-22T21:24:37.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.483+0000 7f31f8400640 1 --2- 172.21.15.161:0/3990826316 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f31d0066690 0x7f31d0068b50 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.483+0000 7f31f8400640 1 --2- 172.21.15.161:0/3990826316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31f00cdf60 0x7f31f0151a60 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.483+0000 7f31f8400640 1 --2- 172.21.15.161:0/3990826316 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f31f00cdae0 0x7f31f01f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.483+0000 7f31f8400640 1 --2- 172.21.15.161:0/3990826316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31d8003980 0x7f31f0144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.483+0000 7f31f8400640 1 -- 172.21.15.161:0/3990826316 >> 172.21.15.161:0/3990826316 conn(0x7f31f00b0bc0 msgr2=0x7f31f00b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:37.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.483+0000 7f31f8400640 1 -- 172.21.15.161:0/3990826316 shutdown_connections 2024-05-22T21:24:37.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.483+0000 7f31f8400640 1 -- 172.21.15.161:0/3990826316 wait complete. 2024-05-22T21:24:37.503 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:24:37.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f636f800640 1 Processor -- start 2024-05-22T21:24:37.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f636f800640 1 -- start start 2024-05-22T21:24:37.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f636f800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6368143ec0 0x7f636815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f636f800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f63680715a0 0x7f636806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f636f800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63681442a0 con 0x7f636815ec90 2024-05-22T21:24:37.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f636f800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6368070260 con 0x7f63680715a0 2024-05-22T21:24:37.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f636f800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6368070500 con 0x7f6368143ec0 2024-05-22T21:24:37.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f6367e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f63680715a0 0x7f636806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f6367e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f63680715a0 0x7f636806fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59674/0 (socket says 172.21.15.161:59674) 2024-05-22T21:24:37.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f6367e00640 1 -- 172.21.15.161:0/810977234 learned_addr learned my addr 172.21.15.161:0/810977234 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:37.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f636d800640 1 --1- 172.21.15.161:0/810977234 >> v1:172.21.15.161:6789/0 conn(0x7f636815ec90 0x7f636806bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59116/0 (socket says 172.21.15.161:59116) 2024-05-22T21:24:37.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f636ce00640 1 --2- 172.21.15.161:0/810977234 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6368143ec0 0x7f636815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f6367e00640 1 -- 172.21.15.161:0/810977234 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6368143ec0 msgr2=0x7f636815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:37.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f6367e00640 1 --2- 172.21.15.161:0/810977234 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6368143ec0 0x7f636815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f6367e00640 1 -- 172.21.15.161:0/810977234 >> v1:172.21.15.161:6789/0 conn(0x7f636815ec90 legacy=0x7f636806bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f6367e00640 1 -- 172.21.15.161:0/810977234 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63680707e0 con 0x7f63680715a0 2024-05-22T21:24:37.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.639+0000 7f6367400640 1 -- 172.21.15.161:0/810977234 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930784066 0 0) 0x7f63681442a0 con 0x7f636815ec90 2024-05-22T21:24:37.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.643+0000 7f6367e00640 1 --2- 172.21.15.161:0/810977234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63680715a0 0x7f636806fd20 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f6354009870 tx=0x7f635401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26315fd1ce4e9188 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:37.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.643+0000 7f6367400640 1 -- 172.21.15.161:0/810977234 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63540a95f0 con 0x7f63680715a0 2024-05-22T21:24:37.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.643+0000 7f6367400640 1 -- 172.21.15.161:0/810977234 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63540a9750 con 0x7f63680715a0 2024-05-22T21:24:37.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.643+0000 7f6367400640 1 -- 172.21.15.161:0/810977234 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63540a9ae0 con 0x7f63680715a0 2024-05-22T21:24:37.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.643+0000 7f636f800640 1 -- 172.21.15.161:0/810977234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63680715a0 msgr2=0x7f636806fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.643+0000 7f636f800640 1 --2- 172.21.15.161:0/810977234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63680715a0 0x7f636806fd20 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f6354009870 tx=0x7f635401c530 comp rx=0 tx=0).stop 2024-05-22T21:24:37.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.643+0000 7f636f800640 1 -- 172.21.15.161:0/810977234 shutdown_connections 2024-05-22T21:24:37.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.643+0000 7f636f800640 1 --2- 172.21.15.161:0/810977234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63680715a0 0x7f636806fd20 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.643+0000 7f636f800640 1 --2- 172.21.15.161:0/810977234 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6368143ec0 0x7f636815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.643+0000 7f636f800640 1 -- 172.21.15.161:0/810977234 >> 172.21.15.161:0/810977234 conn(0x7f63680b0bc0 msgr2=0x7f63680b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:37.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.643+0000 7f636f800640 1 -- 172.21.15.161:0/810977234 shutdown_connections 2024-05-22T21:24:37.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.643+0000 7f636f800640 1 -- 172.21.15.161:0/810977234 wait complete. 2024-05-22T21:24:37.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636f800640 1 Processor -- start 2024-05-22T21:24:37.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636f800640 1 -- start start 2024-05-22T21:24:37.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636f800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6368143ec0 0x7f636813fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636f800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f636815ec90 0x7f6368156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636f800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6368142220 0x7f63681426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636f800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6368157420 con 0x7f6368142220 2024-05-22T21:24:37.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636f800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63680b0230 con 0x7f636815ec90 2024-05-22T21:24:37.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636f800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63680b0510 con 0x7f6368143ec0 2024-05-22T21:24:37.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636d800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6368142220 0x7f63681426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636d800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6368142220 0x7f63681426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59680/0 (socket says 172.21.15.161:59680) 2024-05-22T21:24:37.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636d800640 1 -- 172.21.15.161:0/2982766646 learned_addr learned my addr 172.21.15.161:0/2982766646 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:37.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636ce00640 1 --2- 172.21.15.161:0/2982766646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6368143ec0 0x7f636813fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f6367e00640 1 --2- 172.21.15.161:0/2982766646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f636815ec90 0x7f6368156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.649 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636d800640 1 -- 172.21.15.161:0/2982766646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6368143ec0 msgr2=0x7f636813fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636d800640 1 --2- 172.21.15.161:0/2982766646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6368143ec0 0x7f636813fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636d800640 1 -- 172.21.15.161:0/2982766646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f636815ec90 msgr2=0x7f6368156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636d800640 1 --2- 172.21.15.161:0/2982766646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f636815ec90 0x7f6368156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636d800640 1 -- 172.21.15.161:0/2982766646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63540094d0 con 0x7f6368142220 2024-05-22T21:24:37.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636ce00640 1 --2- 172.21.15.161:0/2982766646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6368143ec0 0x7f636813fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:37.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636d800640 1 --2- 172.21.15.161:0/2982766646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6368142220 0x7f63681426a0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f636000c780 tx=0x7f636000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:37.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f6364c00640 1 -- 172.21.15.161:0/2982766646 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636000ce90 con 0x7f6368142220 2024-05-22T21:24:37.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f6364c00640 1 -- 172.21.15.161:0/2982766646 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636000a640 con 0x7f6368142220 2024-05-22T21:24:37.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636f800640 1 -- 172.21.15.161:0/2982766646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636815c510 con 0x7f6368142220 2024-05-22T21:24:37.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f6364c00640 1 -- 172.21.15.161:0/2982766646 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636000a9d0 con 0x7f6368142220 2024-05-22T21:24:37.650 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.647+0000 7f636f800640 1 -- 172.21.15.161:0/2982766646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636815cad0 con 0x7f6368142220 2024-05-22T21:24:37.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.651+0000 7f636f800640 1 -- 172.21.15.161:0/2982766646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6368067220 con 0x7f6368142220 2024-05-22T21:24:37.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.651+0000 7f6364c00640 1 -- 172.21.15.161:0/2982766646 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f636000ab30 con 0x7f6368142220 2024-05-22T21:24:37.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.651+0000 7f6364c00640 1 --2- 172.21.15.161:0/2982766646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f63480666c0 0x7f6348068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.651+0000 7f6364c00640 1 -- 172.21.15.161:0/2982766646 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6360083830 con 0x7f6368142220 2024-05-22T21:24:37.651 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.651+0000 7f636ce00640 1 --2- 172.21.15.161:0/2982766646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f63480666c0 0x7f6348068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.651+0000 7f636ce00640 1 --2- 172.21.15.161:0/2982766646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f63480666c0 0x7f6348068b80 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f635c009bb0 tx=0x7f635c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:37.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.655+0000 7f6364c00640 1 -- 172.21.15.161:0/2982766646 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6360050c40 con 0x7f6368142220 2024-05-22T21:24:37.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.811+0000 7f636f800640 1 -- 172.21.15.161:0/2982766646 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63680cca70 con 0x7f6368142220 2024-05-22T21:24:37.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.811+0000 7f6364c00640 1 -- 172.21.15.161:0/2982766646 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+16152 (secure 0 0 0) 0x7f6360018090 con 0x7f6368142220 2024-05-22T21:24:37.812 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:37.812 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":190,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:24:35.099389+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:24:31.797440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-05-22T21:24:33.098240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:24:37.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.815+0000 7f636f800640 1 -- 172.21.15.161:0/2982766646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f63480666c0 msgr2=0x7f6348068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.815+0000 7f636f800640 1 --2- 172.21.15.161:0/2982766646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f63480666c0 0x7f6348068b80 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f635c009bb0 tx=0x7f635c008040 comp rx=0 tx=0).stop 2024-05-22T21:24:37.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.815+0000 7f636f800640 1 -- 172.21.15.161:0/2982766646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6368142220 msgr2=0x7f63681426a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.815+0000 7f636f800640 1 --2- 172.21.15.161:0/2982766646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6368142220 0x7f63681426a0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f636000c780 tx=0x7f636000cc40 comp rx=0 tx=0).stop 2024-05-22T21:24:37.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.815+0000 7f636f800640 1 -- 172.21.15.161:0/2982766646 shutdown_connections 2024-05-22T21:24:37.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.815+0000 7f636f800640 1 --2- 172.21.15.161:0/2982766646 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f63480666c0 0x7f6348068b80 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.815+0000 7f636f800640 1 --2- 172.21.15.161:0/2982766646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6368142220 0x7f63681426a0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.815+0000 7f636f800640 1 --2- 172.21.15.161:0/2982766646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f636815ec90 0x7f6368156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.815+0000 7f636f800640 1 --2- 172.21.15.161:0/2982766646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6368143ec0 0x7f636813fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.815+0000 7f636f800640 1 -- 172.21.15.161:0/2982766646 >> 172.21.15.161:0/2982766646 conn(0x7f63680b0bc0 msgr2=0x7f636813f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:37.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.815+0000 7f636f800640 1 -- 172.21.15.161:0/2982766646 shutdown_connections 2024-05-22T21:24:37.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.815+0000 7f636f800640 1 -- 172.21.15.161:0/2982766646 wait complete. 2024-05-22T21:24:37.833 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-22T21:24:37.834 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:24:37.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.967+0000 7fa55da00640 1 Processor -- start 2024-05-22T21:24:37.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.967+0000 7fa55da00640 1 -- start start 2024-05-22T21:24:37.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.971+0000 7fa55da00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa55815e130 0x7fa55815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.971+0000 7fa55da00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa558071670 con 0x7fa55815dd50 2024-05-22T21:24:37.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.971+0000 7fa55da00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55815d870 con 0x7fa55815d410 2024-05-22T21:24:37.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.971+0000 7fa55da00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa558070a60 con 0x7fa55815e130 2024-05-22T21:24:37.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.971+0000 7fa556a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa55815dd50 0x7fa55806d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59126/0 (socket says 172.21.15.161:59126) 2024-05-22T21:24:37.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.971+0000 7fa556a00640 1 -- 172.21.15.161:0/3182824644 learned_addr learned my addr 172.21.15.161:0/3182824644 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:37.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.971+0000 7fa556000640 1 --2- 172.21.15.161:0/3182824644 >> v2:172.21.15.161:3300/0 conn(0x7fa55815e130 0x7fa55815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.971+0000 7fa556000640 1 -- 172.21.15.161:0/3182824644 >> v1:172.21.15.179:6789/0 conn(0x7fa55815d410 legacy=0x7fa55815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:37.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.971+0000 7fa556000640 1 -- 172.21.15.161:0/3182824644 >> v1:172.21.15.161:6789/0 conn(0x7fa55815dd50 legacy=0x7fa55806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.971+0000 7fa556000640 1 -- 172.21.15.161:0/3182824644 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa558070d40 con 0x7fa55815e130 2024-05-22T21:24:37.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.971+0000 7fa555600640 1 -- 172.21.15.161:0/3182824644 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111742120 0 0) 0x7fa558071670 con 0x7fa55815dd50 2024-05-22T21:24:37.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa556000640 1 --2- 172.21.15.161:0/3182824644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55815e130 0x7fa55815ced0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fa5400097c0 tx=0x7fa54001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78b907cd949f0f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:37.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa555600640 1 -- 172.21.15.161:0/3182824644 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5400a7070 con 0x7fa55815e130 2024-05-22T21:24:37.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa555600640 1 -- 172.21.15.161:0/3182824644 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5400ab430 con 0x7fa55815e130 2024-05-22T21:24:37.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa555600640 1 -- 172.21.15.161:0/3182824644 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5400ab7c0 con 0x7fa55815e130 2024-05-22T21:24:37.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa55da00640 1 -- 172.21.15.161:0/3182824644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55815e130 msgr2=0x7fa55815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa55da00640 1 --2- 172.21.15.161:0/3182824644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55815e130 0x7fa55815ced0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fa5400097c0 tx=0x7fa54001c430 comp rx=0 tx=0).stop 2024-05-22T21:24:37.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa55da00640 1 -- 172.21.15.161:0/3182824644 shutdown_connections 2024-05-22T21:24:37.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa55da00640 1 --2- 172.21.15.161:0/3182824644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55815e130 0x7fa55815ced0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa55da00640 1 -- 172.21.15.161:0/3182824644 >> 172.21.15.161:0/3182824644 conn(0x7fa5580b0bc0 msgr2=0x7fa5580b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:37.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa55da00640 1 -- 172.21.15.161:0/3182824644 shutdown_connections 2024-05-22T21:24:37.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa55da00640 1 -- 172.21.15.161:0/3182824644 wait complete. 2024-05-22T21:24:37.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa55da00640 1 Processor -- start 2024-05-22T21:24:37.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa55da00640 1 -- start start 2024-05-22T21:24:37.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa55da00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa55815d410 0x7fa5580b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa55da00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa55815dd50 0x7fa5581f9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa55da00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55815e130 0x7fa55814b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa55da00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580ce600 con 0x7fa55815e130 2024-05-22T21:24:37.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa55da00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5581f9d60 con 0x7fa55815dd50 2024-05-22T21:24:37.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.975+0000 7fa55da00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580ce0d0 con 0x7fa55815d410 2024-05-22T21:24:37.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa557400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55815e130 0x7fa55814b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa557400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55815e130 0x7fa55814b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59702/0 (socket says 172.21.15.161:59702) 2024-05-22T21:24:37.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa557400640 1 -- 172.21.15.161:0/2809452762 learned_addr learned my addr 172.21.15.161:0/2809452762 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:37.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa556a00640 1 --2- 172.21.15.161:0/2809452762 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa55815d410 0x7fa5580b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa556000640 1 --2- 172.21.15.161:0/2809452762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa55815dd50 0x7fa5581f9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa557400640 1 -- 172.21.15.161:0/2809452762 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa55815d410 msgr2=0x7fa5580b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa557400640 1 --2- 172.21.15.161:0/2809452762 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa55815d410 0x7fa5580b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa557400640 1 -- 172.21.15.161:0/2809452762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa55815dd50 msgr2=0x7fa5581f9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:37.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa557400640 1 --2- 172.21.15.161:0/2809452762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa55815dd50 0x7fa5581f9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:37.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa557400640 1 -- 172.21.15.161:0/2809452762 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa540009420 con 0x7fa55815e130 2024-05-22T21:24:37.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa556000640 1 --2- 172.21.15.161:0/2809452762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa55815dd50 0x7fa5581f9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:37.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa557400640 1 --2- 172.21.15.161:0/2809452762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55815e130 0x7fa55814b860 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fa55000b970 tx=0x7fa55000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:37.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa54ea00640 1 -- 172.21.15.161:0/2809452762 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55000cb40 con 0x7fa55815e130 2024-05-22T21:24:37.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa54ea00640 1 -- 172.21.15.161:0/2809452762 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55000cca0 con 0x7fa55815e130 2024-05-22T21:24:37.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa54ea00640 1 -- 172.21.15.161:0/2809452762 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550014680 con 0x7fa55815e130 2024-05-22T21:24:37.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa55da00640 1 -- 172.21.15.161:0/2809452762 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5580ce320 con 0x7fa55815e130 2024-05-22T21:24:37.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa55da00640 1 -- 172.21.15.161:0/2809452762 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55814fd50 con 0x7fa55815e130 2024-05-22T21:24:37.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa55da00640 1 -- 172.21.15.161:0/2809452762 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa558151650 con 0x7fa55815e130 2024-05-22T21:24:37.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa54ea00640 1 -- 172.21.15.161:0/2809452762 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa55001d3b0 con 0x7fa55815e130 2024-05-22T21:24:37.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.979+0000 7fa54ea00640 1 --2- 172.21.15.161:0/2809452762 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa5340666c0 0x7fa534068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:37.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.983+0000 7fa54ea00640 1 -- 172.21.15.161:0/2809452762 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa550083f90 con 0x7fa55815e130 2024-05-22T21:24:37.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.983+0000 7fa556a00640 1 --2- 172.21.15.161:0/2809452762 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa5340666c0 0x7fa534068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:37.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.983+0000 7fa556a00640 1 --2- 172.21.15.161:0/2809452762 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa5340666c0 0x7fa534068b80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa548000f70 tx=0x7fa548007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:37.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:37.983+0000 7fa54ea00640 1 -- 172.21.15.161:0/2809452762 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa558151650 con 0x7fa55815e130 2024-05-22T21:24:38.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.135+0000 7fa55da00640 1 -- 172.21.15.161:0/2809452762 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa558070730 con 0x7fa5340666c0 2024-05-22T21:24:38.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.143+0000 7fa54ea00640 1 -- 172.21.15.161:0/2809452762 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285127 (secure 0 0 0) 0x7fa5480072a0 con 0x7fa5340666c0 2024-05-22T21:24:38.148 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:38.150 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:24:38.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.147+0000 7fa55da00640 1 -- 172.21.15.161:0/2809452762 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa5340666c0 msgr2=0x7fa534068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:38.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.147+0000 7fa55da00640 1 --2- 172.21.15.161:0/2809452762 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa5340666c0 0x7fa534068b80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa548000f70 tx=0x7fa548007df0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.147+0000 7fa55da00640 1 -- 172.21.15.161:0/2809452762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55815e130 msgr2=0x7fa55814b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:38.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.147+0000 7fa55da00640 1 --2- 172.21.15.161:0/2809452762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55815e130 0x7fa55814b860 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fa55000b970 tx=0x7fa55000be30 comp rx=0 tx=0).stop 2024-05-22T21:24:38.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.151+0000 7fa55da00640 1 -- 172.21.15.161:0/2809452762 shutdown_connections 2024-05-22T21:24:38.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.151+0000 7fa55da00640 1 --2- 172.21.15.161:0/2809452762 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa5340666c0 0x7fa534068b80 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.151+0000 7fa55da00640 1 --2- 172.21.15.161:0/2809452762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa55815e130 0x7fa55814b860 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.151+0000 7fa55da00640 1 --2- 172.21.15.161:0/2809452762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa55815dd50 0x7fa5581f9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.151+0000 7fa55da00640 1 --2- 172.21.15.161:0/2809452762 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa55815d410 0x7fa5580b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.151+0000 7fa55da00640 1 -- 172.21.15.161:0/2809452762 >> 172.21.15.161:0/2809452762 conn(0x7fa5580b0bc0 msgr2=0x7fa5580ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:38.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.151+0000 7fa55da00640 1 -- 172.21.15.161:0/2809452762 shutdown_connections 2024-05-22T21:24:38.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.151+0000 7fa55da00640 1 -- 172.21.15.161:0/2809452762 wait complete. 2024-05-22T21:24:38.170 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":524,"stamp":"2024-05-22T21:24:36.550572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9479,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553020,"statfs":{"total":1600000000,"available":1590293379,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000743"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.108999+0000","last_change":"2024-05-22T21:24:34.113887+0000","last_active":"2024-05-22T21:24:35.108999+0000","last_peered":"2024-05-22T21:24:35.108999+0000","last_clean":"2024-05-22T21:24:35.108999+0000","last_became_active":"2024-05-22T21:24:34.113298+0000","last_became_peered":"2024-05-22T21:24:34.113298+0000","last_unstale":"2024-05-22T21:24:35.108999+0000","last_undegraded":"2024-05-22T21:24:35.108999+0000","last_fullsized":"2024-05-22T21:24:35.108999+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:10:34.453663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452422+0000","last_change":"2024-05-22T21:24:34.116680+0000","last_active":"2024-05-22T21:24:36.452422+0000","last_peered":"2024-05-22T21:24:36.452422+0000","last_clean":"2024-05-22T21:24:36.452422+0000","last_became_active":"2024-05-22T21:24:34.115180+0000","last_became_peered":"2024-05-22T21:24:34.115180+0000","last_unstale":"2024-05-22T21:24:36.452422+0000","last_undegraded":"2024-05-22T21:24:36.452422+0000","last_fullsized":"2024-05-22T21:24:36.452422+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:22:41.425404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.453002+0000","last_change":"2024-05-22T21:24:34.113906+0000","last_active":"2024-05-22T21:24:36.453002+0000","last_peered":"2024-05-22T21:24:36.453002+0000","last_clean":"2024-05-22T21:24:36.453002+0000","last_became_active":"2024-05-22T21:24:34.113191+0000","last_became_peered":"2024-05-22T21:24:34.113191+0000","last_unstale":"2024-05-22T21:24:36.453002+0000","last_undegraded":"2024-05-22T21:24:36.453002+0000","last_fullsized":"2024-05-22T21:24:36.453002+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:00:28.671610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451007+0000","last_change":"2024-05-22T21:24:34.112336+0000","last_active":"2024-05-22T21:24:36.451007+0000","last_peered":"2024-05-22T21:24:36.451007+0000","last_clean":"2024-05-22T21:24:36.451007+0000","last_became_active":"2024-05-22T21:24:34.112003+0000","last_became_peered":"2024-05-22T21:24:34.112003+0000","last_unstale":"2024-05-22T21:24:36.451007+0000","last_undegraded":"2024-05-22T21:24:36.451007+0000","last_fullsized":"2024-05-22T21:24:36.451007+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:30:31.031688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452953+0000","last_change":"2024-05-22T21:24:34.114924+0000","last_active":"2024-05-22T21:24:36.452953+0000","last_peered":"2024-05-22T21:24:36.452953+0000","last_clean":"2024-05-22T21:24:36.452953+0000","last_became_active":"2024-05-22T21:24:34.114674+0000","last_became_peered":"2024-05-22T21:24:34.114674+0000","last_unstale":"2024-05-22T21:24:36.452953+0000","last_undegraded":"2024-05-22T21:24:36.452953+0000","last_fullsized":"2024-05-22T21:24:36.452953+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:36.398308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450677+0000","last_change":"2024-05-22T21:24:34.110047+0000","last_active":"2024-05-22T21:24:36.450677+0000","last_peered":"2024-05-22T21:24:36.450677+0000","last_clean":"2024-05-22T21:24:36.450677+0000","last_became_active":"2024-05-22T21:24:34.109839+0000","last_became_peered":"2024-05-22T21:24:34.109839+0000","last_unstale":"2024-05-22T21:24:36.450677+0000","last_undegraded":"2024-05-22T21:24:36.450677+0000","last_fullsized":"2024-05-22T21:24:36.450677+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:09:44.245622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451104+0000","last_change":"2024-05-22T21:24:34.110619+0000","last_active":"2024-05-22T21:24:36.451104+0000","last_peered":"2024-05-22T21:24:36.451104+0000","last_clean":"2024-05-22T21:24:36.451104+0000","last_became_active":"2024-05-22T21:24:34.110319+0000","last_became_peered":"2024-05-22T21:24:34.110319+0000","last_unstale":"2024-05-22T21:24:36.451104+0000","last_undegraded":"2024-05-22T21:24:36.451104+0000","last_fullsized":"2024-05-22T21:24:36.451104+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:56:27.788790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452313+0000","last_change":"2024-05-22T21:24:34.114810+0000","last_active":"2024-05-22T21:24:36.452313+0000","last_peered":"2024-05-22T21:24:36.452313+0000","last_clean":"2024-05-22T21:24:36.452313+0000","last_became_active":"2024-05-22T21:24:34.114554+0000","last_became_peered":"2024-05-22T21:24:34.114554+0000","last_unstale":"2024-05-22T21:24:36.452313+0000","last_undegraded":"2024-05-22T21:24:36.452313+0000","last_fullsized":"2024-05-22T21:24:36.452313+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:24:39.530150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452597+0000","last_change":"2024-05-22T21:24:34.111703+0000","last_active":"2024-05-22T21:24:36.452597+0000","last_peered":"2024-05-22T21:24:36.452597+0000","last_clean":"2024-05-22T21:24:36.452597+0000","last_became_active":"2024-05-22T21:24:34.111192+0000","last_became_peered":"2024-05-22T21:24:34.111192+0000","last_unstale":"2024-05-22T21:24:36.452597+0000","last_undegraded":"2024-05-22T21:24:36.452597+0000","last_fullsized":"2024-05-22T21:24:36.452597+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:19:40.350127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.453038+0000","last_change":"2024-05-22T21:24:34.117375+0000","last_active":"2024-05-22T21:24:36.453038+0000","last_peered":"2024-05-22T21:24:36.453038+0000","last_clean":"2024-05-22T21:24:36.453038+0000","last_became_active":"2024-05-22T21:24:34.117119+0000","last_became_peered":"2024-05-22T21:24:34.117119+0000","last_unstale":"2024-05-22T21:24:36.453038+0000","last_undegraded":"2024-05-22T21:24:36.453038+0000","last_fullsized":"2024-05-22T21:24:36.453038+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:28:06.948625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.453357+0000","last_change":"2024-05-22T21:24:34.116325+0000","last_active":"2024-05-22T21:24:36.453357+0000","last_peered":"2024-05-22T21:24:36.453357+0000","last_clean":"2024-05-22T21:24:36.453357+0000","last_became_active":"2024-05-22T21:24:34.116107+0000","last_became_peered":"2024-05-22T21:24:34.116107+0000","last_unstale":"2024-05-22T21:24:36.453357+0000","last_undegraded":"2024-05-22T21:24:36.453357+0000","last_fullsized":"2024-05-22T21:24:36.453357+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:07:48.906348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452534+0000","last_change":"2024-05-22T21:24:34.115392+0000","last_active":"2024-05-22T21:24:36.452534+0000","last_peered":"2024-05-22T21:24:36.452534+0000","last_clean":"2024-05-22T21:24:36.452534+0000","last_became_active":"2024-05-22T21:24:34.114854+0000","last_became_peered":"2024-05-22T21:24:34.114854+0000","last_unstale":"2024-05-22T21:24:36.452534+0000","last_undegraded":"2024-05-22T21:24:36.452534+0000","last_fullsized":"2024-05-22T21:24:36.452534+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:02:21.712263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452687+0000","last_change":"2024-05-22T21:24:34.114415+0000","last_active":"2024-05-22T21:24:36.452687+0000","last_peered":"2024-05-22T21:24:36.452687+0000","last_clean":"2024-05-22T21:24:36.452687+0000","last_became_active":"2024-05-22T21:24:34.114183+0000","last_became_peered":"2024-05-22T21:24:34.114183+0000","last_unstale":"2024-05-22T21:24:36.452687+0000","last_undegraded":"2024-05-22T21:24:36.452687+0000","last_fullsized":"2024-05-22T21:24:36.452687+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:35:38.621202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452392+0000","last_change":"2024-05-22T21:24:34.117506+0000","last_active":"2024-05-22T21:24:36.452392+0000","last_peered":"2024-05-22T21:24:36.452392+0000","last_clean":"2024-05-22T21:24:36.452392+0000","last_became_active":"2024-05-22T21:24:34.117053+0000","last_became_peered":"2024-05-22T21:24:34.117053+0000","last_unstale":"2024-05-22T21:24:36.452392+0000","last_undegraded":"2024-05-22T21:24:36.452392+0000","last_fullsized":"2024-05-22T21:24:36.452392+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:55:04.400396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.108700+0000","last_change":"2024-05-22T21:24:34.111338+0000","last_active":"2024-05-22T21:24:35.108700+0000","last_peered":"2024-05-22T21:24:35.108700+0000","last_clean":"2024-05-22T21:24:35.108700+0000","last_became_active":"2024-05-22T21:24:34.111094+0000","last_became_peered":"2024-05-22T21:24:34.111094+0000","last_unstale":"2024-05-22T21:24:35.108700+0000","last_undegraded":"2024-05-22T21:24:35.108700+0000","last_fullsized":"2024-05-22T21:24:35.108700+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:33:42.410676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451196+0000","last_change":"2024-05-22T21:24:34.119754+0000","last_active":"2024-05-22T21:24:36.451196+0000","last_peered":"2024-05-22T21:24:36.451196+0000","last_clean":"2024-05-22T21:24:36.451196+0000","last_became_active":"2024-05-22T21:24:34.116424+0000","last_became_peered":"2024-05-22T21:24:34.116424+0000","last_unstale":"2024-05-22T21:24:36.451196+0000","last_undegraded":"2024-05-22T21:24:36.451196+0000","last_fullsized":"2024-05-22T21:24:36.451196+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:51.640158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451090+0000","last_change":"2024-05-22T21:24:34.118252+0000","last_active":"2024-05-22T21:24:36.451090+0000","last_peered":"2024-05-22T21:24:36.451090+0000","last_clean":"2024-05-22T21:24:36.451090+0000","last_became_active":"2024-05-22T21:24:34.117847+0000","last_became_peered":"2024-05-22T21:24:34.117847+0000","last_unstale":"2024-05-22T21:24:36.451090+0000","last_undegraded":"2024-05-22T21:24:36.451090+0000","last_fullsized":"2024-05-22T21:24:36.451090+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:05:35.517631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106191+0000","last_change":"2024-05-22T21:24:34.116233+0000","last_active":"2024-05-22T21:24:35.106191+0000","last_peered":"2024-05-22T21:24:35.106191+0000","last_clean":"2024-05-22T21:24:35.106191+0000","last_became_active":"2024-05-22T21:24:34.115667+0000","last_became_peered":"2024-05-22T21:24:34.115667+0000","last_unstale":"2024-05-22T21:24:35.106191+0000","last_undegraded":"2024-05-22T21:24:35.106191+0000","last_fullsized":"2024-05-22T21:24:35.106191+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:39:13.059738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451583+0000","last_change":"2024-05-22T21:24:34.115165+0000","last_active":"2024-05-22T21:24:36.451583+0000","last_peered":"2024-05-22T21:24:36.451583+0000","last_clean":"2024-05-22T21:24:36.451583+0000","last_became_active":"2024-05-22T21:24:34.114479+0000","last_became_peered":"2024-05-22T21:24:34.114479+0000","last_unstale":"2024-05-22T21:24:36.451583+0000","last_undegraded":"2024-05-22T21:24:36.451583+0000","last_fullsized":"2024-05-22T21:24:36.451583+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:59:49.394372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452601+0000","last_change":"2024-05-22T21:24:34.116803+0000","last_active":"2024-05-22T21:24:36.452601+0000","last_peered":"2024-05-22T21:24:36.452601+0000","last_clean":"2024-05-22T21:24:36.452601+0000","last_became_active":"2024-05-22T21:24:34.116219+0000","last_became_peered":"2024-05-22T21:24:34.116219+0000","last_unstale":"2024-05-22T21:24:36.452601+0000","last_undegraded":"2024-05-22T21:24:36.452601+0000","last_fullsized":"2024-05-22T21:24:36.452601+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:09:44.245622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450760+0000","last_change":"2024-05-22T21:24:34.119155+0000","last_active":"2024-05-22T21:24:36.450760+0000","last_peered":"2024-05-22T21:24:36.450760+0000","last_clean":"2024-05-22T21:24:36.450760+0000","last_became_active":"2024-05-22T21:24:34.117404+0000","last_became_peered":"2024-05-22T21:24:34.117404+0000","last_unstale":"2024-05-22T21:24:36.450760+0000","last_undegraded":"2024-05-22T21:24:36.450760+0000","last_fullsized":"2024-05-22T21:24:36.450760+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:10:58.574775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452674+0000","last_change":"2024-05-22T21:24:34.111859+0000","last_active":"2024-05-22T21:24:36.452674+0000","last_peered":"2024-05-22T21:24:36.452674+0000","last_clean":"2024-05-22T21:24:36.452674+0000","last_became_active":"2024-05-22T21:24:34.111567+0000","last_became_peered":"2024-05-22T21:24:34.111567+0000","last_unstale":"2024-05-22T21:24:36.452674+0000","last_undegraded":"2024-05-22T21:24:36.452674+0000","last_fullsized":"2024-05-22T21:24:36.452674+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:35:47.933629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450782+0000","last_change":"2024-05-22T21:24:34.114137+0000","last_active":"2024-05-22T21:24:36.450782+0000","last_peered":"2024-05-22T21:24:36.450782+0000","last_clean":"2024-05-22T21:24:36.450782+0000","last_became_active":"2024-05-22T21:24:34.113933+0000","last_became_peered":"2024-05-22T21:24:34.113933+0000","last_unstale":"2024-05-22T21:24:36.450782+0000","last_undegraded":"2024-05-22T21:24:36.450782+0000","last_fullsized":"2024-05-22T21:24:36.450782+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:58:32.687466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452315+0000","last_change":"2024-05-22T21:24:34.112238+0000","last_active":"2024-05-22T21:24:36.452315+0000","last_peered":"2024-05-22T21:24:36.452315+0000","last_clean":"2024-05-22T21:24:36.452315+0000","last_became_active":"2024-05-22T21:24:34.111480+0000","last_became_peered":"2024-05-22T21:24:34.111480+0000","last_unstale":"2024-05-22T21:24:36.452315+0000","last_undegraded":"2024-05-22T21:24:36.452315+0000","last_fullsized":"2024-05-22T21:24:36.452315+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:04:25.576271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452723+0000","last_change":"2024-05-22T21:24:34.114269+0000","last_active":"2024-05-22T21:24:36.452723+0000","last_peered":"2024-05-22T21:24:36.452723+0000","last_clean":"2024-05-22T21:24:36.452723+0000","last_became_active":"2024-05-22T21:24:34.114027+0000","last_became_peered":"2024-05-22T21:24:34.114027+0000","last_unstale":"2024-05-22T21:24:36.452723+0000","last_undegraded":"2024-05-22T21:24:36.452723+0000","last_fullsized":"2024-05-22T21:24:36.452723+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:19:52.242393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451106+0000","last_change":"2024-05-22T21:24:34.109702+0000","last_active":"2024-05-22T21:24:36.451106+0000","last_peered":"2024-05-22T21:24:36.451106+0000","last_clean":"2024-05-22T21:24:36.451106+0000","last_became_active":"2024-05-22T21:24:34.109443+0000","last_became_peered":"2024-05-22T21:24:34.109443+0000","last_unstale":"2024-05-22T21:24:36.451106+0000","last_undegraded":"2024-05-22T21:24:36.451106+0000","last_fullsized":"2024-05-22T21:24:36.451106+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:14:06.124437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452838+0000","last_change":"2024-05-22T21:24:34.111487+0000","last_active":"2024-05-22T21:24:36.452838+0000","last_peered":"2024-05-22T21:24:36.452838+0000","last_clean":"2024-05-22T21:24:36.452838+0000","last_became_active":"2024-05-22T21:24:34.111244+0000","last_became_peered":"2024-05-22T21:24:34.111244+0000","last_unstale":"2024-05-22T21:24:36.452838+0000","last_undegraded":"2024-05-22T21:24:36.452838+0000","last_fullsized":"2024-05-22T21:24:36.452838+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:14:40.086666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451142+0000","last_change":"2024-05-22T21:24:34.105705+0000","last_active":"2024-05-22T21:24:36.451142+0000","last_peered":"2024-05-22T21:24:36.451142+0000","last_clean":"2024-05-22T21:24:36.451142+0000","last_became_active":"2024-05-22T21:24:34.105479+0000","last_became_peered":"2024-05-22T21:24:34.105479+0000","last_unstale":"2024-05-22T21:24:36.451142+0000","last_undegraded":"2024-05-22T21:24:36.451142+0000","last_fullsized":"2024-05-22T21:24:36.451142+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:25:40.227199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452500+0000","last_change":"2024-05-22T21:24:34.117364+0000","last_active":"2024-05-22T21:24:36.452500+0000","last_peered":"2024-05-22T21:24:36.452500+0000","last_clean":"2024-05-22T21:24:36.452500+0000","last_became_active":"2024-05-22T21:24:34.116751+0000","last_became_peered":"2024-05-22T21:24:34.116751+0000","last_unstale":"2024-05-22T21:24:36.452500+0000","last_undegraded":"2024-05-22T21:24:36.452500+0000","last_fullsized":"2024-05-22T21:24:36.452500+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:12:42.267144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109616+0000","last_change":"2024-05-22T21:24:34.106292+0000","last_active":"2024-05-22T21:24:35.109616+0000","last_peered":"2024-05-22T21:24:35.109616+0000","last_clean":"2024-05-22T21:24:35.109616+0000","last_became_active":"2024-05-22T21:24:34.106043+0000","last_became_peered":"2024-05-22T21:24:34.106043+0000","last_unstale":"2024-05-22T21:24:35.109616+0000","last_undegraded":"2024-05-22T21:24:35.109616+0000","last_fullsized":"2024-05-22T21:24:35.109616+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:11:39.188888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109668+0000","last_change":"2024-05-22T21:24:34.111431+0000","last_active":"2024-05-22T21:24:35.109668+0000","last_peered":"2024-05-22T21:24:35.109668+0000","last_clean":"2024-05-22T21:24:35.109668+0000","last_became_active":"2024-05-22T21:24:34.111089+0000","last_became_peered":"2024-05-22T21:24:34.111089+0000","last_unstale":"2024-05-22T21:24:35.109668+0000","last_undegraded":"2024-05-22T21:24:35.109668+0000","last_fullsized":"2024-05-22T21:24:35.109668+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:23:28.758003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106081+0000","last_change":"2024-05-22T21:24:34.115265+0000","last_active":"2024-05-22T21:24:35.106081+0000","last_peered":"2024-05-22T21:24:35.106081+0000","last_clean":"2024-05-22T21:24:35.106081+0000","last_became_active":"2024-05-22T21:24:34.114756+0000","last_became_peered":"2024-05-22T21:24:34.114756+0000","last_unstale":"2024-05-22T21:24:35.106081+0000","last_undegraded":"2024-05-22T21:24:35.106081+0000","last_fullsized":"2024-05-22T21:24:35.106081+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:23:47.941680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450561+0000","last_change":"2024-05-22T21:24:34.108101+0000","last_active":"2024-05-22T21:24:36.450561+0000","last_peered":"2024-05-22T21:24:36.450561+0000","last_clean":"2024-05-22T21:24:36.450561+0000","last_became_active":"2024-05-22T21:24:34.107898+0000","last_became_peered":"2024-05-22T21:24:34.107898+0000","last_unstale":"2024-05-22T21:24:36.450561+0000","last_undegraded":"2024-05-22T21:24:36.450561+0000","last_fullsized":"2024-05-22T21:24:36.450561+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:31:45.904007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"189'4","reported_seq":15,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.108595+0000","last_change":"2024-05-22T21:24:33.737621+0000","last_active":"2024-05-22T21:24:35.108595+0000","last_peered":"2024-05-22T21:24:35.108595+0000","last_clean":"2024-05-22T21:24:35.108595+0000","last_became_active":"2024-05-22T21:24:33.737371+0000","last_became_peered":"2024-05-22T21:24:33.737371+0000","last_unstale":"2024-05-22T21:24:35.108595+0000","last_undegraded":"2024-05-22T21:24:35.108595+0000","last_fullsized":"2024-05-22T21:24:35.108595+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:54:11.566309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109561+0000","last_change":"2024-05-22T21:24:34.115450+0000","last_active":"2024-05-22T21:24:35.109561+0000","last_peered":"2024-05-22T21:24:35.109561+0000","last_clean":"2024-05-22T21:24:35.109561+0000","last_became_active":"2024-05-22T21:24:34.115228+0000","last_became_peered":"2024-05-22T21:24:34.115228+0000","last_unstale":"2024-05-22T21:24:35.109561+0000","last_undegraded":"2024-05-22T21:24:35.109561+0000","last_fullsized":"2024-05-22T21:24:35.109561+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:34:13.785204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"189'1","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109306+0000","last_change":"2024-05-22T21:24:33.733654+0000","last_active":"2024-05-22T21:24:35.109306+0000","last_peered":"2024-05-22T21:24:35.109306+0000","last_clean":"2024-05-22T21:24:35.109306+0000","last_became_active":"2024-05-22T21:24:33.730353+0000","last_became_peered":"2024-05-22T21:24:33.730353+0000","last_unstale":"2024-05-22T21:24:35.109306+0000","last_undegraded":"2024-05-22T21:24:35.109306+0000","last_fullsized":"2024-05-22T21:24:35.109306+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:27:57.830133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450840+0000","last_change":"2024-05-22T21:24:34.114629+0000","last_active":"2024-05-22T21:24:36.450840+0000","last_peered":"2024-05-22T21:24:36.450840+0000","last_clean":"2024-05-22T21:24:36.450840+0000","last_became_active":"2024-05-22T21:24:34.114368+0000","last_became_peered":"2024-05-22T21:24:34.114368+0000","last_unstale":"2024-05-22T21:24:36.450840+0000","last_undegraded":"2024-05-22T21:24:36.450840+0000","last_fullsized":"2024-05-22T21:24:36.450840+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:35:38.621202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"189'2","reported_seq":13,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451640+0000","last_change":"2024-05-22T21:24:33.731105+0000","last_active":"2024-05-22T21:24:36.451640+0000","last_peered":"2024-05-22T21:24:36.451640+0000","last_clean":"2024-05-22T21:24:36.451640+0000","last_became_active":"2024-05-22T21:24:33.730882+0000","last_became_peered":"2024-05-22T21:24:33.730882+0000","last_unstale":"2024-05-22T21:24:36.451640+0000","last_undegraded":"2024-05-22T21:24:36.451640+0000","last_fullsized":"2024-05-22T21:24:36.451640+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:00:27.664691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452828+0000","last_change":"2024-05-22T21:24:34.116987+0000","last_active":"2024-05-22T21:24:36.452828+0000","last_peered":"2024-05-22T21:24:36.452828+0000","last_clean":"2024-05-22T21:24:36.452828+0000","last_became_active":"2024-05-22T21:24:34.116508+0000","last_became_peered":"2024-05-22T21:24:34.116508+0000","last_unstale":"2024-05-22T21:24:36.452828+0000","last_undegraded":"2024-05-22T21:24:36.452828+0000","last_fullsized":"2024-05-22T21:24:36.452828+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:28:27.579964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"189'2","reported_seq":13,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451486+0000","last_change":"2024-05-22T21:24:33.732542+0000","last_active":"2024-05-22T21:24:36.451486+0000","last_peered":"2024-05-22T21:24:36.451486+0000","last_clean":"2024-05-22T21:24:36.451486+0000","last_became_active":"2024-05-22T21:24:33.732322+0000","last_became_peered":"2024-05-22T21:24:33.732322+0000","last_unstale":"2024-05-22T21:24:36.451486+0000","last_undegraded":"2024-05-22T21:24:36.451486+0000","last_fullsized":"2024-05-22T21:24:36.451486+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:18:21.802589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450704+0000","last_change":"2024-05-22T21:24:34.113280+0000","last_active":"2024-05-22T21:24:36.450704+0000","last_peered":"2024-05-22T21:24:36.450704+0000","last_clean":"2024-05-22T21:24:36.450704+0000","last_became_active":"2024-05-22T21:24:34.113007+0000","last_became_peered":"2024-05-22T21:24:34.113007+0000","last_unstale":"2024-05-22T21:24:36.450704+0000","last_undegraded":"2024-05-22T21:24:36.450704+0000","last_fullsized":"2024-05-22T21:24:36.450704+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:04:25.576271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"189'1","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452999+0000","last_change":"2024-05-22T21:24:33.737047+0000","last_active":"2024-05-22T21:24:36.452999+0000","last_peered":"2024-05-22T21:24:36.452999+0000","last_clean":"2024-05-22T21:24:36.452999+0000","last_became_active":"2024-05-22T21:24:33.736636+0000","last_became_peered":"2024-05-22T21:24:33.736636+0000","last_unstale":"2024-05-22T21:24:36.452999+0000","last_undegraded":"2024-05-22T21:24:36.452999+0000","last_fullsized":"2024-05-22T21:24:36.452999+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:20:16.845129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451980+0000","last_change":"2024-05-22T21:24:34.108250+0000","last_active":"2024-05-22T21:24:36.451980+0000","last_peered":"2024-05-22T21:24:36.451980+0000","last_clean":"2024-05-22T21:24:36.451980+0000","last_became_active":"2024-05-22T21:24:34.107431+0000","last_became_peered":"2024-05-22T21:24:34.107431+0000","last_unstale":"2024-05-22T21:24:36.451980+0000","last_undegraded":"2024-05-22T21:24:36.451980+0000","last_fullsized":"2024-05-22T21:24:36.451980+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:27:58.837052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"189'1","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451227+0000","last_change":"2024-05-22T21:24:33.732662+0000","last_active":"2024-05-22T21:24:36.451227+0000","last_peered":"2024-05-22T21:24:36.451227+0000","last_clean":"2024-05-22T21:24:36.451227+0000","last_became_active":"2024-05-22T21:24:33.732302+0000","last_became_peered":"2024-05-22T21:24:33.732302+0000","last_unstale":"2024-05-22T21:24:36.451227+0000","last_undegraded":"2024-05-22T21:24:36.451227+0000","last_fullsized":"2024-05-22T21:24:36.451227+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:05:34.510712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451412+0000","last_change":"2024-05-22T21:24:34.105228+0000","last_active":"2024-05-22T21:24:36.451412+0000","last_peered":"2024-05-22T21:24:36.451412+0000","last_clean":"2024-05-22T21:24:36.451412+0000","last_became_active":"2024-05-22T21:24:34.105011+0000","last_became_peered":"2024-05-22T21:24:34.105011+0000","last_unstale":"2024-05-22T21:24:36.451412+0000","last_undegraded":"2024-05-22T21:24:36.451412+0000","last_fullsized":"2024-05-22T21:24:36.451412+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:34:14.259089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452490+0000","last_change":"2024-05-22T21:24:33.739724+0000","last_active":"2024-05-22T21:24:36.452490+0000","last_peered":"2024-05-22T21:24:36.452490+0000","last_clean":"2024-05-22T21:24:36.452490+0000","last_became_active":"2024-05-22T21:24:33.739165+0000","last_became_peered":"2024-05-22T21:24:33.739165+0000","last_unstale":"2024-05-22T21:24:36.452490+0000","last_undegraded":"2024-05-22T21:24:36.452490+0000","last_fullsized":"2024-05-22T21:24:36.452490+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:11:38.181969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451657+0000","last_change":"2024-05-22T21:24:34.118026+0000","last_active":"2024-05-22T21:24:36.451657+0000","last_peered":"2024-05-22T21:24:36.451657+0000","last_clean":"2024-05-22T21:24:36.451657+0000","last_became_active":"2024-05-22T21:24:34.117822+0000","last_became_peered":"2024-05-22T21:24:34.117822+0000","last_unstale":"2024-05-22T21:24:36.451657+0000","last_undegraded":"2024-05-22T21:24:36.451657+0000","last_fullsized":"2024-05-22T21:24:36.451657+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:31:37.180335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451473+0000","last_change":"2024-05-22T21:24:33.728849+0000","last_active":"2024-05-22T21:24:36.451473+0000","last_peered":"2024-05-22T21:24:36.451473+0000","last_clean":"2024-05-22T21:24:36.451473+0000","last_became_active":"2024-05-22T21:24:33.728637+0000","last_became_peered":"2024-05-22T21:24:33.728637+0000","last_unstale":"2024-05-22T21:24:36.451473+0000","last_undegraded":"2024-05-22T21:24:36.451473+0000","last_fullsized":"2024-05-22T21:24:36.451473+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:29:46.561661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106052+0000","last_change":"2024-05-22T21:24:34.116873+0000","last_active":"2024-05-22T21:24:35.106052+0000","last_peered":"2024-05-22T21:24:35.106052+0000","last_clean":"2024-05-22T21:24:35.106052+0000","last_became_active":"2024-05-22T21:24:34.116343+0000","last_became_peered":"2024-05-22T21:24:34.116343+0000","last_unstale":"2024-05-22T21:24:35.106052+0000","last_undegraded":"2024-05-22T21:24:35.106052+0000","last_fullsized":"2024-05-22T21:24:35.106052+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:28:06.948625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451161+0000","last_change":"2024-05-22T21:24:33.736492+0000","last_active":"2024-05-22T21:24:36.451161+0000","last_peered":"2024-05-22T21:24:36.451161+0000","last_clean":"2024-05-22T21:24:36.451161+0000","last_became_active":"2024-05-22T21:24:33.736255+0000","last_became_peered":"2024-05-22T21:24:33.736255+0000","last_unstale":"2024-05-22T21:24:36.451161+0000","last_undegraded":"2024-05-22T21:24:36.451161+0000","last_fullsized":"2024-05-22T21:24:36.451161+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:55:41.991906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106095+0000","last_change":"2024-05-22T21:24:34.111367+0000","last_active":"2024-05-22T21:24:35.106095+0000","last_peered":"2024-05-22T21:24:35.106095+0000","last_clean":"2024-05-22T21:24:35.106095+0000","last_became_active":"2024-05-22T21:24:34.111161+0000","last_became_peered":"2024-05-22T21:24:34.111161+0000","last_unstale":"2024-05-22T21:24:35.106095+0000","last_undegraded":"2024-05-22T21:24:35.106095+0000","last_fullsized":"2024-05-22T21:24:35.106095+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:14:06.124437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"189'3","reported_seq":14,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450870+0000","last_change":"2024-05-22T21:24:33.734810+0000","last_active":"2024-05-22T21:24:36.450870+0000","last_peered":"2024-05-22T21:24:36.450870+0000","last_clean":"2024-05-22T21:24:36.450870+0000","last_became_active":"2024-05-22T21:24:33.734579+0000","last_became_peered":"2024-05-22T21:24:33.734579+0000","last_unstale":"2024-05-22T21:24:36.450870+0000","last_undegraded":"2024-05-22T21:24:36.450870+0000","last_fullsized":"2024-05-22T21:24:36.450870+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:13:43.138691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109675+0000","last_change":"2024-05-22T21:24:34.109809+0000","last_active":"2024-05-22T21:24:35.109675+0000","last_peered":"2024-05-22T21:24:35.109675+0000","last_clean":"2024-05-22T21:24:35.109675+0000","last_became_active":"2024-05-22T21:24:34.109538+0000","last_became_peered":"2024-05-22T21:24:34.109538+0000","last_unstale":"2024-05-22T21:24:35.109675+0000","last_undegraded":"2024-05-22T21:24:35.109675+0000","last_fullsized":"2024-05-22T21:24:35.109675+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:18:42.658301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"189'6","reported_seq":17,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109541+0000","last_change":"2024-05-22T21:24:33.730577+0000","last_active":"2024-05-22T21:24:35.109541+0000","last_peered":"2024-05-22T21:24:35.109541+0000","last_clean":"2024-05-22T21:24:35.109541+0000","last_became_active":"2024-05-22T21:24:33.730000+0000","last_became_peered":"2024-05-22T21:24:33.730000+0000","last_unstale":"2024-05-22T21:24:35.109541+0000","last_undegraded":"2024-05-22T21:24:35.109541+0000","last_fullsized":"2024-05-22T21:24:35.109541+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:26:36.568013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106147+0000","last_change":"2024-05-22T21:24:34.118827+0000","last_active":"2024-05-22T21:24:35.106147+0000","last_peered":"2024-05-22T21:24:35.106147+0000","last_clean":"2024-05-22T21:24:35.106147+0000","last_became_active":"2024-05-22T21:24:34.116654+0000","last_became_peered":"2024-05-22T21:24:34.116654+0000","last_unstale":"2024-05-22T21:24:35.106147+0000","last_undegraded":"2024-05-22T21:24:35.106147+0000","last_fullsized":"2024-05-22T21:24:35.106147+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:30:37.702584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"189'2","reported_seq":13,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451723+0000","last_change":"2024-05-22T21:24:33.727854+0000","last_active":"2024-05-22T21:24:36.451723+0000","last_peered":"2024-05-22T21:24:36.451723+0000","last_clean":"2024-05-22T21:24:36.451723+0000","last_became_active":"2024-05-22T21:24:33.727595+0000","last_became_peered":"2024-05-22T21:24:33.727595+0000","last_unstale":"2024-05-22T21:24:36.451723+0000","last_undegraded":"2024-05-22T21:24:36.451723+0000","last_fullsized":"2024-05-22T21:24:36.451723+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:09:43.238703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451396+0000","last_change":"2024-05-22T21:24:34.110202+0000","last_active":"2024-05-22T21:24:36.451396+0000","last_peered":"2024-05-22T21:24:36.451396+0000","last_clean":"2024-05-22T21:24:36.451396+0000","last_became_active":"2024-05-22T21:24:34.109974+0000","last_became_peered":"2024-05-22T21:24:34.109974+0000","last_unstale":"2024-05-22T21:24:36.451396+0000","last_undegraded":"2024-05-22T21:24:36.451396+0000","last_fullsized":"2024-05-22T21:24:36.451396+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:20:03.445202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106349+0000","last_change":"2024-05-22T21:24:33.731871+0000","last_active":"2024-05-22T21:24:35.106349+0000","last_peered":"2024-05-22T21:24:35.106349+0000","last_clean":"2024-05-22T21:24:35.106349+0000","last_became_active":"2024-05-22T21:24:33.731479+0000","last_became_peered":"2024-05-22T21:24:33.731479+0000","last_unstale":"2024-05-22T21:24:35.106349+0000","last_undegraded":"2024-05-22T21:24:35.106349+0000","last_fullsized":"2024-05-22T21:24:35.106349+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:42:25.944396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451058+0000","last_change":"2024-05-22T21:24:34.112801+0000","last_active":"2024-05-22T21:24:36.451058+0000","last_peered":"2024-05-22T21:24:36.451058+0000","last_clean":"2024-05-22T21:24:36.451058+0000","last_became_active":"2024-05-22T21:24:34.112525+0000","last_became_peered":"2024-05-22T21:24:34.112525+0000","last_unstale":"2024-05-22T21:24:36.451058+0000","last_undegraded":"2024-05-22T21:24:36.451058+0000","last_fullsized":"2024-05-22T21:24:36.451058+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:31:23.471827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452207+0000","last_change":"2024-05-22T21:24:33.730489+0000","last_active":"2024-05-22T21:24:36.452207+0000","last_peered":"2024-05-22T21:24:36.452207+0000","last_clean":"2024-05-22T21:24:36.452207+0000","last_became_active":"2024-05-22T21:24:33.730272+0000","last_became_peered":"2024-05-22T21:24:33.730272+0000","last_unstale":"2024-05-22T21:24:36.452207+0000","last_undegraded":"2024-05-22T21:24:36.452207+0000","last_fullsized":"2024-05-22T21:24:36.452207+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:31:36.173417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452905+0000","last_change":"2024-05-22T21:24:34.115931+0000","last_active":"2024-05-22T21:24:36.452905+0000","last_peered":"2024-05-22T21:24:36.452905+0000","last_clean":"2024-05-22T21:24:36.452905+0000","last_became_active":"2024-05-22T21:24:34.115647+0000","last_became_peered":"2024-05-22T21:24:34.115647+0000","last_unstale":"2024-05-22T21:24:36.452905+0000","last_undegraded":"2024-05-22T21:24:36.452905+0000","last_fullsized":"2024-05-22T21:24:36.452905+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:18:22.809508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"189'8","reported_seq":19,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109758+0000","last_change":"2024-05-22T21:24:33.728520+0000","last_active":"2024-05-22T21:24:35.109758+0000","last_peered":"2024-05-22T21:24:35.109758+0000","last_clean":"2024-05-22T21:24:35.109758+0000","last_became_active":"2024-05-22T21:24:33.728263+0000","last_became_peered":"2024-05-22T21:24:33.728263+0000","last_unstale":"2024-05-22T21:24:35.109758+0000","last_undegraded":"2024-05-22T21:24:35.109758+0000","last_fullsized":"2024-05-22T21:24:35.109758+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:32:13.689008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451558+0000","last_change":"2024-05-22T21:24:34.114448+0000","last_active":"2024-05-22T21:24:36.451558+0000","last_peered":"2024-05-22T21:24:36.451558+0000","last_clean":"2024-05-22T21:24:36.451558+0000","last_became_active":"2024-05-22T21:24:34.114182+0000","last_became_peered":"2024-05-22T21:24:34.114182+0000","last_unstale":"2024-05-22T21:24:36.451558+0000","last_undegraded":"2024-05-22T21:24:36.451558+0000","last_fullsized":"2024-05-22T21:24:36.451558+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:33:12.291343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"189'1","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452721+0000","last_change":"2024-05-22T21:24:33.736025+0000","last_active":"2024-05-22T21:24:36.452721+0000","last_peered":"2024-05-22T21:24:36.452721+0000","last_clean":"2024-05-22T21:24:36.452721+0000","last_became_active":"2024-05-22T21:24:33.735545+0000","last_became_peered":"2024-05-22T21:24:33.735545+0000","last_unstale":"2024-05-22T21:24:36.452721+0000","last_undegraded":"2024-05-22T21:24:36.452721+0000","last_fullsized":"2024-05-22T21:24:36.452721+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:32:09.388182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106200+0000","last_change":"2024-05-22T21:24:34.118506+0000","last_active":"2024-05-22T21:24:35.106200+0000","last_peered":"2024-05-22T21:24:35.106200+0000","last_clean":"2024-05-22T21:24:35.106200+0000","last_became_active":"2024-05-22T21:24:34.113673+0000","last_became_peered":"2024-05-22T21:24:34.113673+0000","last_unstale":"2024-05-22T21:24:35.106200+0000","last_undegraded":"2024-05-22T21:24:35.106200+0000","last_fullsized":"2024-05-22T21:24:35.106200+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:04:25.576271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"189'2","reported_seq":13,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.453199+0000","last_change":"2024-05-22T21:24:33.739366+0000","last_active":"2024-05-22T21:24:36.453199+0000","last_peered":"2024-05-22T21:24:36.453199+0000","last_clean":"2024-05-22T21:24:36.453199+0000","last_became_active":"2024-05-22T21:24:33.738926+0000","last_became_peered":"2024-05-22T21:24:33.738926+0000","last_unstale":"2024-05-22T21:24:36.453199+0000","last_undegraded":"2024-05-22T21:24:36.453199+0000","last_fullsized":"2024-05-22T21:24:36.453199+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:27:37.325046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452125+0000","last_change":"2024-05-22T21:24:34.107683+0000","last_active":"2024-05-22T21:24:36.452125+0000","last_peered":"2024-05-22T21:24:36.452125+0000","last_clean":"2024-05-22T21:24:36.452125+0000","last_became_active":"2024-05-22T21:24:34.107145+0000","last_became_peered":"2024-05-22T21:24:34.107145+0000","last_unstale":"2024-05-22T21:24:36.452125+0000","last_undegraded":"2024-05-22T21:24:36.452125+0000","last_fullsized":"2024-05-22T21:24:36.452125+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:26:37.574931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"189'1","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109089+0000","last_change":"2024-05-22T21:24:33.732972+0000","last_active":"2024-05-22T21:24:35.109089+0000","last_peered":"2024-05-22T21:24:35.109089+0000","last_clean":"2024-05-22T21:24:35.109089+0000","last_became_active":"2024-05-22T21:24:33.732310+0000","last_became_peered":"2024-05-22T21:24:33.732310+0000","last_unstale":"2024-05-22T21:24:35.109089+0000","last_undegraded":"2024-05-22T21:24:35.109089+0000","last_fullsized":"2024-05-22T21:24:35.109089+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:06:06.856153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451280+0000","last_change":"2024-05-22T21:24:34.116623+0000","last_active":"2024-05-22T21:24:36.451280+0000","last_peered":"2024-05-22T21:24:36.451280+0000","last_clean":"2024-05-22T21:24:36.451280+0000","last_became_active":"2024-05-22T21:24:34.116129+0000","last_became_peered":"2024-05-22T21:24:34.116129+0000","last_unstale":"2024-05-22T21:24:36.451280+0000","last_undegraded":"2024-05-22T21:24:36.451280+0000","last_fullsized":"2024-05-22T21:24:36.451280+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:39:13.059738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.108808+0000","last_change":"2024-05-22T21:24:33.737995+0000","last_active":"2024-05-22T21:24:35.108808+0000","last_peered":"2024-05-22T21:24:35.108808+0000","last_clean":"2024-05-22T21:24:35.108808+0000","last_became_active":"2024-05-22T21:24:33.737732+0000","last_became_peered":"2024-05-22T21:24:33.737732+0000","last_unstale":"2024-05-22T21:24:35.108808+0000","last_undegraded":"2024-05-22T21:24:35.108808+0000","last_fullsized":"2024-05-22T21:24:35.108808+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:49:38.137481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452612+0000","last_change":"2024-05-22T21:24:34.115193+0000","last_active":"2024-05-22T21:24:36.452612+0000","last_peered":"2024-05-22T21:24:36.452612+0000","last_clean":"2024-05-22T21:24:36.452612+0000","last_became_active":"2024-05-22T21:24:34.113165+0000","last_became_peered":"2024-05-22T21:24:34.113165+0000","last_unstale":"2024-05-22T21:24:36.452612+0000","last_undegraded":"2024-05-22T21:24:36.452612+0000","last_fullsized":"2024-05-22T21:24:36.452612+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:51:26.318295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"189'2","reported_seq":13,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451317+0000","last_change":"2024-05-22T21:24:33.732530+0000","last_active":"2024-05-22T21:24:36.451317+0000","last_peered":"2024-05-22T21:24:36.451317+0000","last_clean":"2024-05-22T21:24:36.451317+0000","last_became_active":"2024-05-22T21:24:33.731856+0000","last_became_peered":"2024-05-22T21:24:33.731856+0000","last_unstale":"2024-05-22T21:24:36.451317+0000","last_undegraded":"2024-05-22T21:24:36.451317+0000","last_fullsized":"2024-05-22T21:24:36.451317+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:10:57.567857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450839+0000","last_change":"2024-05-22T21:24:34.116014+0000","last_active":"2024-05-22T21:24:36.450839+0000","last_peered":"2024-05-22T21:24:36.450839+0000","last_clean":"2024-05-22T21:24:36.450839+0000","last_became_active":"2024-05-22T21:24:34.115805+0000","last_became_peered":"2024-05-22T21:24:34.115805+0000","last_unstale":"2024-05-22T21:24:36.450839+0000","last_undegraded":"2024-05-22T21:24:36.450839+0000","last_fullsized":"2024-05-22T21:24:36.450839+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:28:06.948625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.108791+0000","last_change":"2024-05-22T21:24:33.733103+0000","last_active":"2024-05-22T21:24:35.108791+0000","last_peered":"2024-05-22T21:24:35.108791+0000","last_clean":"2024-05-22T21:24:35.108791+0000","last_became_active":"2024-05-22T21:24:33.732341+0000","last_became_peered":"2024-05-22T21:24:33.732341+0000","last_unstale":"2024-05-22T21:24:35.108791+0000","last_undegraded":"2024-05-22T21:24:35.108791+0000","last_fullsized":"2024-05-22T21:24:35.108791+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:22:45.506309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.108902+0000","last_change":"2024-05-22T21:24:34.114041+0000","last_active":"2024-05-22T21:24:35.108902+0000","last_peered":"2024-05-22T21:24:35.108902+0000","last_clean":"2024-05-22T21:24:35.108902+0000","last_became_active":"2024-05-22T21:24:34.113589+0000","last_became_peered":"2024-05-22T21:24:34.113589+0000","last_unstale":"2024-05-22T21:24:35.108902+0000","last_undegraded":"2024-05-22T21:24:35.108902+0000","last_fullsized":"2024-05-22T21:24:35.108902+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:22:09.318242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451797+0000","last_change":"2024-05-22T21:24:33.734864+0000","last_active":"2024-05-22T21:24:36.451797+0000","last_peered":"2024-05-22T21:24:36.451797+0000","last_clean":"2024-05-22T21:24:36.451797+0000","last_became_active":"2024-05-22T21:24:33.734649+0000","last_became_peered":"2024-05-22T21:24:33.734649+0000","last_unstale":"2024-05-22T21:24:36.451797+0000","last_undegraded":"2024-05-22T21:24:36.451797+0000","last_fullsized":"2024-05-22T21:24:36.451797+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:35:37.614284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451641+0000","last_change":"2024-05-22T21:24:34.115089+0000","last_active":"2024-05-22T21:24:36.451641+0000","last_peered":"2024-05-22T21:24:36.451641+0000","last_clean":"2024-05-22T21:24:36.451641+0000","last_became_active":"2024-05-22T21:24:34.114873+0000","last_became_peered":"2024-05-22T21:24:34.114873+0000","last_unstale":"2024-05-22T21:24:36.451641+0000","last_undegraded":"2024-05-22T21:24:36.451641+0000","last_fullsized":"2024-05-22T21:24:36.451641+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:02:58.346797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109178+0000","last_change":"2024-05-22T21:24:33.733126+0000","last_active":"2024-05-22T21:24:35.109178+0000","last_peered":"2024-05-22T21:24:35.109178+0000","last_clean":"2024-05-22T21:24:35.109178+0000","last_became_active":"2024-05-22T21:24:33.732702+0000","last_became_peered":"2024-05-22T21:24:33.732702+0000","last_unstale":"2024-05-22T21:24:35.109178+0000","last_undegraded":"2024-05-22T21:24:35.109178+0000","last_fullsized":"2024-05-22T21:24:35.109178+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:35:40.028344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106282+0000","last_change":"2024-05-22T21:24:34.114522+0000","last_active":"2024-05-22T21:24:35.106282+0000","last_peered":"2024-05-22T21:24:35.106282+0000","last_clean":"2024-05-22T21:24:35.106282+0000","last_became_active":"2024-05-22T21:24:34.114297+0000","last_became_peered":"2024-05-22T21:24:34.114297+0000","last_unstale":"2024-05-22T21:24:35.106282+0000","last_undegraded":"2024-05-22T21:24:35.106282+0000","last_fullsized":"2024-05-22T21:24:35.106282+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:24:39.530150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"189'2","reported_seq":13,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106381+0000","last_change":"2024-05-22T21:24:33.733577+0000","last_active":"2024-05-22T21:24:35.106381+0000","last_peered":"2024-05-22T21:24:35.106381+0000","last_clean":"2024-05-22T21:24:35.106381+0000","last_became_active":"2024-05-22T21:24:33.732656+0000","last_became_peered":"2024-05-22T21:24:33.732656+0000","last_unstale":"2024-05-22T21:24:35.106381+0000","last_undegraded":"2024-05-22T21:24:35.106381+0000","last_fullsized":"2024-05-22T21:24:35.106381+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:12:37.414160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451191+0000","last_change":"2024-05-22T21:24:34.113899+0000","last_active":"2024-05-22T21:24:36.451191+0000","last_peered":"2024-05-22T21:24:36.451191+0000","last_clean":"2024-05-22T21:24:36.451191+0000","last_became_active":"2024-05-22T21:24:34.113610+0000","last_became_peered":"2024-05-22T21:24:34.113610+0000","last_unstale":"2024-05-22T21:24:36.451191+0000","last_undegraded":"2024-05-22T21:24:36.451191+0000","last_fullsized":"2024-05-22T21:24:36.451191+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:30:37.702584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.453088+0000","last_change":"2024-05-22T21:24:33.737243+0000","last_active":"2024-05-22T21:24:36.453088+0000","last_peered":"2024-05-22T21:24:36.453088+0000","last_clean":"2024-05-22T21:24:36.453088+0000","last_became_active":"2024-05-22T21:24:33.736981+0000","last_became_peered":"2024-05-22T21:24:33.736981+0000","last_unstale":"2024-05-22T21:24:36.453088+0000","last_undegraded":"2024-05-22T21:24:36.453088+0000","last_fullsized":"2024-05-22T21:24:36.453088+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:32:03.762097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451725+0000","last_change":"2024-05-22T21:24:34.115655+0000","last_active":"2024-05-22T21:24:36.451725+0000","last_peered":"2024-05-22T21:24:36.451725+0000","last_clean":"2024-05-22T21:24:36.451725+0000","last_became_active":"2024-05-22T21:24:34.115406+0000","last_became_peered":"2024-05-22T21:24:34.115406+0000","last_unstale":"2024-05-22T21:24:36.451725+0000","last_undegraded":"2024-05-22T21:24:36.451725+0000","last_fullsized":"2024-05-22T21:24:36.451725+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:56:01.598828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452702+0000","last_change":"2024-05-22T21:24:33.739499+0000","last_active":"2024-05-22T21:24:36.452702+0000","last_peered":"2024-05-22T21:24:36.452702+0000","last_clean":"2024-05-22T21:24:36.452702+0000","last_became_active":"2024-05-22T21:24:33.738730+0000","last_became_peered":"2024-05-22T21:24:33.738730+0000","last_unstale":"2024-05-22T21:24:36.452702+0000","last_undegraded":"2024-05-22T21:24:36.452702+0000","last_fullsized":"2024-05-22T21:24:36.452702+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:42:25.944396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106289+0000","last_change":"2024-05-22T21:24:34.108832+0000","last_active":"2024-05-22T21:24:35.106289+0000","last_peered":"2024-05-22T21:24:35.106289+0000","last_clean":"2024-05-22T21:24:35.106289+0000","last_became_active":"2024-05-22T21:24:34.108629+0000","last_became_peered":"2024-05-22T21:24:34.108629+0000","last_unstale":"2024-05-22T21:24:35.106289+0000","last_undegraded":"2024-05-22T21:24:35.106289+0000","last_fullsized":"2024-05-22T21:24:35.106289+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:22:41.425404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106544+0000","last_change":"2024-05-22T21:24:33.735095+0000","last_active":"2024-05-22T21:24:35.106544+0000","last_peered":"2024-05-22T21:24:35.106544+0000","last_clean":"2024-05-22T21:24:35.106544+0000","last_became_active":"2024-05-22T21:24:33.734799+0000","last_became_peered":"2024-05-22T21:24:33.734799+0000","last_unstale":"2024-05-22T21:24:35.106544+0000","last_undegraded":"2024-05-22T21:24:35.106544+0000","last_fullsized":"2024-05-22T21:24:35.106544+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:11:38.181969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106288+0000","last_change":"2024-05-22T21:24:34.115785+0000","last_active":"2024-05-22T21:24:35.106288+0000","last_peered":"2024-05-22T21:24:35.106288+0000","last_clean":"2024-05-22T21:24:35.106288+0000","last_became_active":"2024-05-22T21:24:34.115049+0000","last_became_peered":"2024-05-22T21:24:34.115049+0000","last_unstale":"2024-05-22T21:24:35.106288+0000","last_undegraded":"2024-05-22T21:24:35.106288+0000","last_fullsized":"2024-05-22T21:24:35.106288+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:35:47.933629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109971+0000","last_change":"2024-05-22T21:24:33.729321+0000","last_active":"2024-05-22T21:24:35.109971+0000","last_peered":"2024-05-22T21:24:35.109971+0000","last_clean":"2024-05-22T21:24:35.109971+0000","last_became_active":"2024-05-22T21:24:33.727866+0000","last_became_peered":"2024-05-22T21:24:33.727866+0000","last_unstale":"2024-05-22T21:24:35.109971+0000","last_undegraded":"2024-05-22T21:24:35.109971+0000","last_fullsized":"2024-05-22T21:24:35.109971+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:22:34.599259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109757+0000","last_change":"2024-05-22T21:24:34.111301+0000","last_active":"2024-05-22T21:24:35.109757+0000","last_peered":"2024-05-22T21:24:35.109757+0000","last_clean":"2024-05-22T21:24:35.109757+0000","last_became_active":"2024-05-22T21:24:34.110763+0000","last_became_peered":"2024-05-22T21:24:34.110763+0000","last_unstale":"2024-05-22T21:24:35.109757+0000","last_undegraded":"2024-05-22T21:24:35.109757+0000","last_fullsized":"2024-05-22T21:24:35.109757+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:12:38.421079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452284+0000","last_change":"2024-05-22T21:24:33.728901+0000","last_active":"2024-05-22T21:24:36.452284+0000","last_peered":"2024-05-22T21:24:36.452284+0000","last_clean":"2024-05-22T21:24:36.452284+0000","last_became_active":"2024-05-22T21:24:33.728050+0000","last_became_peered":"2024-05-22T21:24:33.728050+0000","last_unstale":"2024-05-22T21:24:36.452284+0000","last_undegraded":"2024-05-22T21:24:36.452284+0000","last_fullsized":"2024-05-22T21:24:36.452284+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:55:03.393477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450467+0000","last_change":"2024-05-22T21:24:34.109817+0000","last_active":"2024-05-22T21:24:36.450467+0000","last_peered":"2024-05-22T21:24:36.450467+0000","last_clean":"2024-05-22T21:24:36.450467+0000","last_became_active":"2024-05-22T21:24:34.109611+0000","last_became_peered":"2024-05-22T21:24:34.109611+0000","last_unstale":"2024-05-22T21:24:36.450467+0000","last_undegraded":"2024-05-22T21:24:36.450467+0000","last_fullsized":"2024-05-22T21:24:36.450467+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:24:39.530150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452685+0000","last_change":"2024-05-22T21:24:33.735666+0000","last_active":"2024-05-22T21:24:36.452685+0000","last_peered":"2024-05-22T21:24:36.452685+0000","last_clean":"2024-05-22T21:24:36.452685+0000","last_became_active":"2024-05-22T21:24:33.735397+0000","last_became_peered":"2024-05-22T21:24:33.735397+0000","last_unstale":"2024-05-22T21:24:36.452685+0000","last_undegraded":"2024-05-22T21:24:36.452685+0000","last_fullsized":"2024-05-22T21:24:36.452685+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:34:12.778286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452802+0000","last_change":"2024-05-22T21:24:34.118551+0000","last_active":"2024-05-22T21:24:36.452802+0000","last_peered":"2024-05-22T21:24:36.452802+0000","last_clean":"2024-05-22T21:24:36.452802+0000","last_became_active":"2024-05-22T21:24:34.118291+0000","last_became_peered":"2024-05-22T21:24:34.118291+0000","last_unstale":"2024-05-22T21:24:36.452802+0000","last_undegraded":"2024-05-22T21:24:36.452802+0000","last_fullsized":"2024-05-22T21:24:36.452802+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:10:58.574775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109426+0000","last_change":"2024-05-22T21:24:33.733829+0000","last_active":"2024-05-22T21:24:35.109426+0000","last_peered":"2024-05-22T21:24:35.109426+0000","last_clean":"2024-05-22T21:24:35.109426+0000","last_became_active":"2024-05-22T21:24:33.730850+0000","last_became_peered":"2024-05-22T21:24:33.730850+0000","last_unstale":"2024-05-22T21:24:35.109426+0000","last_undegraded":"2024-05-22T21:24:35.109426+0000","last_fullsized":"2024-05-22T21:24:35.109426+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:25:39.220280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106182+0000","last_change":"2024-05-22T21:24:34.117690+0000","last_active":"2024-05-22T21:24:35.106182+0000","last_peered":"2024-05-22T21:24:35.106182+0000","last_clean":"2024-05-22T21:24:35.106182+0000","last_became_active":"2024-05-22T21:24:34.117478+0000","last_became_peered":"2024-05-22T21:24:34.117478+0000","last_unstale":"2024-05-22T21:24:35.106182+0000","last_undegraded":"2024-05-22T21:24:35.106182+0000","last_fullsized":"2024-05-22T21:24:35.106182+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:02:21.712263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109860+0000","last_change":"2024-05-22T21:24:33.729477+0000","last_active":"2024-05-22T21:24:35.109860+0000","last_peered":"2024-05-22T21:24:35.109860+0000","last_clean":"2024-05-22T21:24:35.109860+0000","last_became_active":"2024-05-22T21:24:33.728199+0000","last_became_peered":"2024-05-22T21:24:33.728199+0000","last_unstale":"2024-05-22T21:24:35.109860+0000","last_undegraded":"2024-05-22T21:24:35.109860+0000","last_fullsized":"2024-05-22T21:24:35.109860+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:34:13.252170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":338,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-05-22T21:24:34.104117+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:24:34.104117+0000","last_peered":"2024-05-22T21:24:34.104117+0000","last_clean":"2024-05-22T21:24:34.104117+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:24:34.104117+0000","last_undegraded":"2024-05-22T21:24:34.104117+0000","last_fullsized":"2024-05-22T21:24:34.104117+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1622,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193689,"statfs":{"total":200000000,"available":198338166,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.36699999999999999},"max":{"1min":0.95099999999999996,"5min":1.022,"15min":2.0369999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72099999999999997,"15min":0.70199999999999996},"min":{"1min":0.52500000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97899999999999998,"5min":1.1759999999999999,"15min":1.889},"last":0.78500000000000003}]},{"osd":1,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72099999999999997,"15min":0.74199999999999999},"min":{"1min":0.48099999999999998,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.93300000000000005,"5min":1.0329999999999999,"15min":3.0920000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.73399999999999999},"min":{"1min":0.60199999999999998,"5min":0.45200000000000001,"15min":0.39800000000000002},"max":{"1min":0.94699999999999995,"5min":1.1100000000000001,"15min":3.1499999999999999},"last":0.60799999999999998}]},{"osd":2,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.501,"5min":0.433,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":1.002,"15min":1.923},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.40999999999999998},"max":{"1min":0.96699999999999997,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72399999999999998,"15min":0.70999999999999996},"min":{"1min":0.54900000000000004,"5min":0.44700000000000001,"15min":0.40100000000000002},"max":{"1min":0.88100000000000001,"5min":1.026,"15min":1.026},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.76200000000000001,"15min":0.77200000000000002},"min":{"1min":0.64700000000000002,"5min":0.501,"15min":0.44500000000000001},"max":{"1min":0.99399999999999999,"5min":1.0740000000000001,"15min":2.1389999999999998},"last":0.76100000000000001}]},{"osd":4,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.29399999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.96599999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.628,"15min":0.61899999999999999},"min":{"1min":0.504,"5min":0.40500000000000003,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.95999999999999996,"15min":1.018},"last":0.57499999999999996}]},{"osd":5,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.51700000000000002,"5min":0.39300000000000002,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0289999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67300000000000004,"15min":0.65200000000000002},"min":{"1min":0.46300000000000002,"5min":0.42099999999999999,"15min":0.33400000000000002},"max":{"1min":0.91200000000000003,"5min":0.99299999999999999,"15min":0.996},"last":0.66500000000000004}]},{"osd":6,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.50600000000000001,"5min":0.5,"15min":0.36799999999999999},"max":{"1min":0.92500000000000004,"5min":1.052,"15min":1.052},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69999999999999996,"15min":0.67000000000000004},"min":{"1min":0.60999999999999999,"5min":0.48699999999999999,"15min":0.41199999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.97699999999999998},"last":0.745}]}]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1037,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198937118,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69499999999999995,"15min":0.69399999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0649999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.51900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95899999999999996,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.77600000000000002}]},{"osd":1,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69399999999999995,"15min":0.69099999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89500000000000002,"5min":0.94299999999999995,"15min":1.107},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.70599999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":0.93500000000000005,"5min":1.0009999999999999,"15min":1.081},"last":0.71799999999999997}]},{"osd":2,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.442},"max":{"1min":0.93000000000000005,"5min":1.026,"15min":1.087},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75700000000000001,"15min":0.746},"min":{"1min":0.56499999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0089999999999999,"5min":1.419,"15min":1.419},"last":0.83099999999999996}]},{"osd":3,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.749,"15min":0.746},"min":{"1min":0.46600000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98599999999999999,"5min":1.28,"15min":1.415},"last":0.874},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.55400000000000005,"5min":0.48999999999999999,"15min":0.45400000000000001},"max":{"1min":0.878,"5min":0.93300000000000005,"15min":1.788},"last":0.69999999999999996}]},{"osd":4,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.374},"max":{"1min":0.89300000000000002,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.67500000000000004},"min":{"1min":0.45600000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.90900000000000003,"5min":1.1850000000000001,"15min":2.718},"last":0.90600000000000003}]},{"osd":5,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61499999999999999,"15min":0.67800000000000005},"min":{"1min":0.42399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84799999999999998,"5min":0.97499999999999998,"15min":8.5749999999999993},"last":0.752},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67900000000000005,"15min":0.69999999999999996},"min":{"1min":0.49299999999999999,"5min":0.44800000000000001,"15min":0.42099999999999999},"max":{"1min":0.92000000000000004,"5min":0.95799999999999996,"15min":1.1140000000000001},"last":0.85199999999999998}]},{"osd":7,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92800000000000005,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69699999999999995,"15min":0.72699999999999998},"min":{"1min":0.49099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85499999999999998,"5min":0.89700000000000002,"15min":1.8009999999999999},"last":0.79700000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194273,"statfs":{"total":200000000,"available":198936536,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:23:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67400000000000004,"15min":0.69199999999999995},"min":{"1min":0.504,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.0409999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69599999999999995,"15min":0.68500000000000005},"min":{"1min":0.54600000000000004,"5min":0.38400000000000001,"15min":0.378},"max":{"1min":0.85999999999999999,"5min":0.93700000000000006,"15min":1.0089999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Wed May 22 21:23:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.497,"5min":0.372,"15min":0.372},"max":{"1min":0.90900000000000003,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.57099999999999995,"5min":0.45500000000000002,"15min":0.44},"max":{"1min":0.91100000000000003,"5min":1.321,"15min":1.321},"last":0.748}]},{"osd":2,"last update":"Wed May 22 21:23:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.70799999999999996},"min":{"1min":0.51500000000000001,"5min":0.46300000000000002,"15min":0.33100000000000002},"max":{"1min":0.996,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73499999999999999,"15min":0.69899999999999995},"min":{"1min":0.58099999999999996,"5min":0.432,"15min":0.42499999999999999},"max":{"1min":0.93200000000000005,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed May 22 21:23:36 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72699999999999998,"15min":0.71599999999999997},"min":{"1min":0.58699999999999997,"5min":0.48799999999999999,"15min":0.47999999999999998},"max":{"1min":0.92700000000000005,"5min":1.004,"15min":1.1879999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.627,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.89100000000000001,"5min":1.0469999999999999,"15min":1.111},"last":0.72599999999999998}]},{"osd":4,"last update":"Wed May 22 21:23:36 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.39600000000000002,"15min":0.35099999999999998},"max":{"1min":0.86899999999999999,"5min":0.92400000000000004,"15min":1.272},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.49299999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86399999999999999,"5min":0.95499999999999996,"15min":1.038},"last":0.84999999999999998}]},{"osd":6,"last update":"Wed May 22 21:23:36 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.51300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.98899999999999999},"last":0.872},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65800000000000003,"15min":0.66100000000000003},"min":{"1min":0.45600000000000002,"5min":0.443,"15min":0.41099999999999998},"max":{"1min":0.84099999999999997,"5min":1.0069999999999999,"15min":1.2290000000000001},"last":0.94799999999999995}]},{"osd":7,"last update":"Wed May 22 21:23:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.59199999999999997,"5min":0.48999999999999999,"15min":0.47399999999999998},"max":{"1min":0.89300000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.755,"15min":0.77300000000000002},"min":{"1min":0.58099999999999996,"5min":0.47399999999999998,"15min":0.46200000000000002},"max":{"1min":1.0149999999999999,"5min":1.018,"15min":1.2909999999999999},"last":0.96799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1045,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194266,"statfs":{"total":200000000,"available":198929206,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71499999999999997,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30399999999999999},"max":{"1min":0.97299999999999998,"5min":0.98899999999999999,"15min":6.6539999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.52100000000000002,"5min":0.46000000000000002,"15min":0.436},"max":{"1min":0.872,"5min":0.96299999999999997,"15min":5.4370000000000003},"last":0.76300000000000001}]},{"osd":1,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.39200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91400000000000003,"5min":0.95799999999999996,"15min":1.7749999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69699999999999995,"15min":0.70899999999999996},"min":{"1min":0.48299999999999998,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":2.2210000000000001},"last":0.67600000000000005}]},{"osd":2,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72299999999999998,"15min":0.71199999999999997},"min":{"1min":0.50900000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":0.97299999999999998,"15min":2.4660000000000002},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.54400000000000004,"5min":0.45600000000000002,"15min":0.378},"max":{"1min":0.92900000000000005,"5min":1.01,"15min":2.2530000000000001},"last":0.72999999999999998}]},{"osd":3,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71299999999999997,"15min":0.70199999999999996},"min":{"1min":0.54200000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.998,"5min":1.042,"15min":1.857},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77100000000000002,"15min":0.78200000000000003},"min":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.47499999999999998},"max":{"1min":0.96599999999999997,"5min":1.012,"15min":1.897},"last":0.83299999999999996}]},{"osd":5,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.629},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.379},"max":{"1min":0.95199999999999996,"5min":1.0169999999999999,"15min":1.1850000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.35599999999999998},"max":{"1min":0.872,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.51200000000000001}]},{"osd":6,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85599999999999998,"5min":1.044,"15min":1.0609999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.44,"5min":0.44,"15min":0.434},"max":{"1min":0.996,"5min":1.095,"15min":1.1419999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.68500000000000005},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.40799999999999997},"max":{"1min":0.84799999999999998,"5min":1.1579999999999999,"15min":1.7390000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70999999999999996,"15min":0.746},"min":{"1min":0.505,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86499999999999999,"5min":1.069,"15min":1.4510000000000001},"last":0.626}]}]},{"osd":3,"up_from":13,"seq":55834574987,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1044,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194268,"statfs":{"total":200000000,"available":198930806,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42799999999999999,"15min":0.41999999999999998},"min":{"1min":0.312,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59399999999999997,"5min":0.73099999999999998,"15min":0.86799999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47199999999999998,"15min":0.438},"min":{"1min":0.33600000000000002,"5min":0.30399999999999999,"15min":0.28399999999999997},"max":{"1min":0.65300000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.434}]},{"osd":1,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48199999999999998,"15min":0.46400000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.26500000000000001},"max":{"1min":0.69799999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.52900000000000003},"min":{"1min":0.40699999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78700000000000003,"5min":0.80600000000000005,"15min":0.90000000000000002},"last":0.50800000000000001}]},{"osd":2,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55100000000000005,"15min":0.51600000000000001},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89500000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.50800000000000001},"min":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.33600000000000002},"max":{"1min":0.69599999999999995,"5min":0.82799999999999996,"15min":0.93400000000000005},"last":0.55400000000000005}]},{"osd":4,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":0.71999999999999997},"min":{"1min":0.55900000000000005,"5min":0.48299999999999998,"15min":0.39400000000000002},"max":{"1min":1.0389999999999999,"5min":1.1839999999999999,"15min":1.2050000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.75700000000000001,"15min":0.73599999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.435},"max":{"1min":1,"5min":1.093,"15min":1.5760000000000001},"last":0.79900000000000004}]},{"osd":5,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.45100000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.234},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.73899999999999999,"15min":0.72399999999999998},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.3049999999999999},"last":0.77900000000000003}]},{"osd":6,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73199999999999998,"15min":0.72699999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.44900000000000001},"max":{"1min":1.0149999999999999,"5min":1.0409999999999999,"15min":1.276},"last":1.016},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76900000000000002,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.53400000000000003,"15min":0.50600000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.2869999999999999},"last":0.88200000000000001}]},{"osd":7,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79000000000000004,"15min":0.77400000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.495},"max":{"1min":1.071,"5min":1.0880000000000001,"15min":1.2709999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.76200000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0169999999999999,"5min":1.133,"15min":1.2969999999999999},"last":1.093}]}]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198937050,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.434,"15min":0.42299999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.23599999999999999},"max":{"1min":0.59899999999999998,"5min":0.66500000000000004,"15min":2.3090000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.435,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.28199999999999997,"5min":0.26900000000000002,"15min":0.23400000000000001},"max":{"1min":0.64100000000000001,"5min":0.80400000000000005,"15min":2.532},"last":0.376}]},{"osd":1,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45600000000000002},"min":{"1min":0.30599999999999999,"5min":0.29899999999999999,"15min":0.29399999999999998},"max":{"1min":0.56899999999999995,"5min":0.68899999999999995,"15min":2.948},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.64800000000000002,"5min":0.71799999999999997,"15min":2.8159999999999998},"last":0.47099999999999997}]},{"osd":3,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.32700000000000001},"max":{"1min":0.62,"5min":0.83499999999999996,"15min":2.75},"last":0.495},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.5,"15min":0.48899999999999999},"min":{"1min":0.36099999999999999,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":3.0619999999999998},"last":0.436}]},{"osd":4,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78000000000000003,"15min":0.74199999999999999},"min":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.52700000000000002},"max":{"1min":1.022,"5min":1.103,"15min":2.7610000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.78100000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.375},"max":{"1min":0.97499999999999998,"5min":1.208,"15min":2.9279999999999999},"last":0.751}]},{"osd":5,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.753,"15min":0.73799999999999999},"min":{"1min":0.56799999999999995,"5min":0.51900000000000002,"15min":0.47299999999999998},"max":{"1min":0.89500000000000002,"5min":1.133,"15min":2.9790000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.45900000000000002},"max":{"1min":0.99099999999999999,"5min":1.0549999999999999,"15min":2.7069999999999999},"last":0.83699999999999997}]},{"osd":6,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.73699999999999999},"min":{"1min":0.54600000000000004,"5min":0.51300000000000001,"15min":0.495},"max":{"1min":0.90000000000000002,"5min":1.0649999999999999,"15min":2.8580000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.748,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.496,"15min":0.46500000000000002},"max":{"1min":1.048,"5min":1.147,"15min":2.6880000000000002},"last":0.67100000000000004}]},{"osd":7,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.79700000000000004,"15min":0.77000000000000002},"min":{"1min":0.67000000000000004,"5min":0.50700000000000001,"15min":0.438},"max":{"1min":1.004,"5min":1.2210000000000001,"15min":3.0099999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81799999999999995,"15min":0.80100000000000005},"min":{"1min":0.67000000000000004,"5min":0.57299999999999995,"15min":0.499},"max":{"1min":1.006,"5min":1.1339999999999999,"15min":2.827},"last":0.79100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1037,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198937106,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:23:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44800000000000001,"15min":0.434},"min":{"1min":0.254,"5min":0.216,"15min":0.216},"max":{"1min":0.59299999999999997,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46600000000000003,"15min":0.45100000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.81299999999999994,"5min":0.879,"15min":0.879},"last":0.53900000000000003}]},{"osd":2,"last update":"Wed May 22 21:23:37 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50900000000000001,"15min":0.497},"min":{"1min":0.32000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.73599999999999999,"5min":1.76,"15min":1.76},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48399999999999999,"15min":0.50900000000000001},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68000000000000005,"5min":0.83599999999999997,"15min":1.923},"last":0.51100000000000001}]},{"osd":3,"last update":"Wed May 22 21:23:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55000000000000004,"15min":0.55600000000000005},"min":{"1min":0.38500000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.755,"5min":0.82999999999999996,"15min":2.1030000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52600000000000002,"15min":0.502},"min":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65800000000000003,"5min":0.745,"15min":0.745},"last":0.68600000000000005}]},{"osd":4,"last update":"Wed May 22 21:23:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73499999999999999,"15min":0.748},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.45100000000000001},"max":{"1min":0.97699999999999998,"5min":1.0780000000000001,"15min":2.1360000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72799999999999998,"15min":0.70699999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.004,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Wed May 22 21:23:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.75700000000000001,"15min":0.748},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.45300000000000001},"max":{"1min":1.0549999999999999,"5min":1.819,"15min":1.819},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.55600000000000005,"5min":0.46300000000000002,"15min":0.40100000000000002},"max":{"1min":1.0469999999999999,"5min":1.1060000000000001,"15min":1.97},"last":0.86099999999999999}]},{"osd":6,"last update":"Wed May 22 21:23:37 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.47299999999999998},"max":{"1min":0.98699999999999999,"5min":1.1359999999999999,"15min":1.272},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.80100000000000005,"15min":0.77300000000000002},"min":{"1min":0.54500000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.097,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.78000000000000003}]},{"osd":7,"last update":"Wed May 22 21:23:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76500000000000001,"15min":0.77000000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.42099999999999999},"max":{"1min":0.995,"5min":1.1379999999999999,"15min":2.0310000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.56699999999999995,"5min":0.54900000000000004,"15min":0.52000000000000002},"max":{"1min":1.018,"5min":1.0660000000000001,"15min":2.0659999999999998},"last":0.84699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1613,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193698,"statfs":{"total":200000000,"available":198347391,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.43099999999999999},"min":{"1min":0.27000000000000002,"5min":0.24199999999999999,"15min":0.22800000000000001},"max":{"1min":0.63400000000000001,"5min":0.69999999999999996,"15min":0.88300000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.50800000000000001},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.33100000000000002},"max":{"1min":0.754,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51400000000000001}]},{"osd":2,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.48399999999999999},"min":{"1min":0.33100000000000002,"5min":0.32100000000000001,"15min":0.29399999999999998},"max":{"1min":0.70299999999999996,"5min":0.85899999999999999,"15min":0.93799999999999994},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.46999999999999997,"15min":0.44900000000000001},"min":{"1min":0.39400000000000002,"5min":0.28899999999999998,"15min":0.27900000000000003},"max":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.92300000000000004},"last":0.44700000000000001}]},{"osd":3,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47599999999999998,"15min":0.45400000000000001},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":1.0760000000000001},"last":0.38},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55700000000000005,"15min":0.53200000000000003},"min":{"1min":0.41699999999999998,"5min":0.38900000000000001,"15min":0.36899999999999999},"max":{"1min":0.79800000000000004,"5min":0.89700000000000002,"15min":0.98299999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.72799999999999998},"min":{"1min":0.53200000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0189999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.748},"min":{"1min":0.50600000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.008,"5min":1.2330000000000001,"15min":1.2709999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.72999999999999998},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":1.135,"15min":1.135},"last":0.505},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.54200000000000004,"5min":0.499,"15min":0.47299999999999998},"max":{"1min":0.95099999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.61599999999999999}]},{"osd":6,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81799999999999995,"15min":0.79300000000000004},"min":{"1min":0.61399999999999999,"5min":0.54400000000000004,"15min":0.51500000000000001},"max":{"1min":0.97299999999999998,"5min":1.175,"15min":1.2090000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80100000000000005,"15min":0.78300000000000003},"min":{"1min":0.60199999999999998,"5min":0.50800000000000001,"15min":0.502},"max":{"1min":0.99099999999999999,"5min":1.3149999999999999,"15min":1.5},"last":0.63300000000000001}]},{"osd":7,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78200000000000003,"15min":0.753},"min":{"1min":0.53200000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.034,"5min":1.214,"15min":1.214},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.64100000000000001,"5min":0.52800000000000002,"15min":0.497},"max":{"1min":0.98599999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.65300000000000002}]}]}],"pool_statfs":[]}} 2024-05-22T21:24:38.176 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:24:38.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.347+0000 7fece2800640 1 Processor -- start 2024-05-22T21:24:38.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.347+0000 7fece2800640 1 -- start start 2024-05-22T21:24:38.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.347+0000 7fece2800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecdc1597f0 0x7fecdc0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:38.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.347+0000 7fece2800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fecdc0b3070 0x7fecdc0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:38.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.347+0000 7fece2800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecdc071040 con 0x7fecdc070be0 2024-05-22T21:24:38.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.347+0000 7fece2800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecdc0b3500 con 0x7fecdc0b3070 2024-05-22T21:24:38.349 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.347+0000 7fece2800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc159bd0 con 0x7fecdc1597f0 2024-05-22T21:24:38.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.351+0000 7fecdb400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecdc1597f0 0x7fecdc0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:38.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.351+0000 7fecdbe00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fecdc0b3070 0x7fecdc0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:38.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.351+0000 7fecdb400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecdc1597f0 0x7fecdc0706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:57978/0 (socket says 172.21.15.161:57978) 2024-05-22T21:24:38.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.351+0000 7fecdb400640 1 -- 172.21.15.161:0/4006581787 learned_addr learned my addr 172.21.15.161:0/4006581787 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:38.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.351+0000 7fecdaa00640 1 -- 172.21.15.161:0/4006581787 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722975545 0 0) 0x7fecdc071040 con 0x7fecdc070be0 2024-05-22T21:24:38.350 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.351+0000 7fecdaa00640 1 -- 172.21.15.161:0/4006581787 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecc0005740 con 0x7fecdc070be0 2024-05-22T21:24:38.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.351+0000 7fecdbe00640 1 -- 172.21.15.161:0/4006581787 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecdc1597f0 msgr2=0x7fecdc0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:38.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.351+0000 7fecdbe00640 1 --2- 172.21.15.161:0/4006581787 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecdc1597f0 0x7fecdc0706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.351+0000 7fecdbe00640 1 -- 172.21.15.161:0/4006581787 >> v1:172.21.15.179:6789/0 conn(0x7fecdc070be0 legacy=0x7fecdc0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:38.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.351+0000 7fecdbe00640 1 -- 172.21.15.161:0/4006581787 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc0cbdd0 con 0x7fecdc0b3070 2024-05-22T21:24:38.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.351+0000 7fecdbe00640 1 --2- 172.21.15.161:0/4006581787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecdc0b3070 0x7fecdc0cc550 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009d90 tx=0x7fecd401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c37dd083e419c8e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:38.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.351+0000 7fecdaa00640 1 -- 172.21.15.161:0/4006581787 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd40a59b0 con 0x7fecdc0b3070 2024-05-22T21:24:38.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.351+0000 7fecdaa00640 1 -- 172.21.15.161:0/4006581787 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd40a5b10 con 0x7fecdc0b3070 2024-05-22T21:24:38.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.351+0000 7fecdaa00640 1 -- 172.21.15.161:0/4006581787 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd40a5ea0 con 0x7fecdc0b3070 2024-05-22T21:24:38.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fece2800640 1 -- 172.21.15.161:0/4006581787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecdc0b3070 msgr2=0x7fecdc0cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:38.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fece2800640 1 --2- 172.21.15.161:0/4006581787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecdc0b3070 0x7fecdc0cc550 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009d90 tx=0x7fecd401c970 comp rx=0 tx=0).stop 2024-05-22T21:24:38.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fece2800640 1 -- 172.21.15.161:0/4006581787 shutdown_connections 2024-05-22T21:24:38.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fece2800640 1 --2- 172.21.15.161:0/4006581787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecdc0b3070 0x7fecdc0cc550 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fece2800640 1 --2- 172.21.15.161:0/4006581787 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecdc1597f0 0x7fecdc0706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fece2800640 1 -- 172.21.15.161:0/4006581787 >> 172.21.15.161:0/4006581787 conn(0x7fecdc0b0bc0 msgr2=0x7fecdc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:38.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fece2800640 1 -- 172.21.15.161:0/4006581787 shutdown_connections 2024-05-22T21:24:38.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fece2800640 1 -- 172.21.15.161:0/4006581787 wait complete. 2024-05-22T21:24:38.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fece2800640 1 Processor -- start 2024-05-22T21:24:38.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fece2800640 1 -- start start 2024-05-22T21:24:38.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fece2800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecdc070be0 0x7fecdc146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:38.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fece2800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecdc0b3070 0x7fecdc155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:38.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fece2800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecdc1597f0 0x7fecdc0cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:38.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fece2800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc146cb0 con 0x7fecdc1597f0 2024-05-22T21:24:38.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fece2800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc15aa10 con 0x7fecdc0b3070 2024-05-22T21:24:38.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fece2800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc1549b0 con 0x7fecdc070be0 2024-05-22T21:24:38.357 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fecdbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecdc1597f0 0x7fecdc0cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:38.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fecdbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecdc1597f0 0x7fecdc0cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59708/0 (socket says 172.21.15.161:59708) 2024-05-22T21:24:38.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fecdbe00640 1 -- 172.21.15.161:0/1718139441 learned_addr learned my addr 172.21.15.161:0/1718139441 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:38.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fecdb400640 1 --2- 172.21.15.161:0/1718139441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecdc070be0 0x7fecdc146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:38.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fecd2a00640 1 --2- 172.21.15.161:0/1718139441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecdc0b3070 0x7fecdc155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:38.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fecdbe00640 1 -- 172.21.15.161:0/1718139441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecdc070be0 msgr2=0x7fecdc146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:38.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fecdbe00640 1 --2- 172.21.15.161:0/1718139441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecdc070be0 0x7fecdc146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fecdbe00640 1 -- 172.21.15.161:0/1718139441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecdc0b3070 msgr2=0x7fecdc155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:38.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.355+0000 7fecdbe00640 1 --2- 172.21.15.161:0/1718139441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecdc0b3070 0x7fecdc155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.359+0000 7fecdbe00640 1 -- 172.21.15.161:0/1718139441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd4009a70 con 0x7fecdc1597f0 2024-05-22T21:24:38.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.359+0000 7fecd2a00640 1 --2- 172.21.15.161:0/1718139441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecdc0b3070 0x7fecdc155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:38.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.359+0000 7fecdb400640 1 --2- 172.21.15.161:0/1718139441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecdc070be0 0x7fecdc146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:38.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.359+0000 7fecdbe00640 1 --2- 172.21.15.161:0/1718139441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecdc1597f0 0x7fecdc0cd040 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fecd40b0040 tx=0x7fecd4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:38.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.359+0000 7fecd3e00640 1 -- 172.21.15.161:0/1718139441 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd40a59b0 con 0x7fecdc1597f0 2024-05-22T21:24:38.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.359+0000 7fecd3e00640 1 -- 172.21.15.161:0/1718139441 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd4009450 con 0x7fecdc1597f0 2024-05-22T21:24:38.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.359+0000 7fecd3e00640 1 -- 172.21.15.161:0/1718139441 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd40b9690 con 0x7fecdc1597f0 2024-05-22T21:24:38.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.359+0000 7fece2800640 1 -- 172.21.15.161:0/1718139441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecdc154c90 con 0x7fecdc1597f0 2024-05-22T21:24:38.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.359+0000 7fece2800640 1 -- 172.21.15.161:0/1718139441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecdc15acf0 con 0x7fecdc1597f0 2024-05-22T21:24:38.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.359+0000 7fece2800640 1 -- 172.21.15.161:0/1718139441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecdc067220 con 0x7fecdc1597f0 2024-05-22T21:24:38.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.359+0000 7fecd3e00640 1 -- 172.21.15.161:0/1718139441 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fecd40b97f0 con 0x7fecdc1597f0 2024-05-22T21:24:38.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.359+0000 7fecd3e00640 1 --2- 172.21.15.161:0/1718139441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fecb8066690 0x7fecb8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:38.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.359+0000 7fecd3e00640 1 -- 172.21.15.161:0/1718139441 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fecd411fa00 con 0x7fecdc1597f0 2024-05-22T21:24:38.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.359+0000 7fecdb400640 1 --2- 172.21.15.161:0/1718139441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fecb8066690 0x7fecb8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:38.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.363+0000 7fecdb400640 1 --2- 172.21.15.161:0/1718139441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fecb8066690 0x7fecb8068b50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7feccc009b00 tx=0x7feccc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:38.364 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.363+0000 7fecd3e00640 1 -- 172.21.15.161:0/1718139441 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fecd40ecd60 con 0x7fecdc1597f0 2024-05-22T21:24:38.514 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.515+0000 7fece2800640 1 -- 172.21.15.161:0/1718139441 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fecdc0b3ce0 con 0x7fecb8066690 2024-05-22T21:24:38.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.523+0000 7fecd3e00640 1 -- 172.21.15.161:0/1718139441 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285127 (secure 0 0 0) 0x7feccc011a00 con 0x7fecb8066690 2024-05-22T21:24:38.525 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:38.526 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:24:38.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.527+0000 7fece2800640 1 -- 172.21.15.161:0/1718139441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fecb8066690 msgr2=0x7fecb8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:38.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.527+0000 7fece2800640 1 --2- 172.21.15.161:0/1718139441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fecb8066690 0x7fecb8068b50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7feccc009b00 tx=0x7feccc008040 comp rx=0 tx=0).stop 2024-05-22T21:24:38.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.527+0000 7fece2800640 1 -- 172.21.15.161:0/1718139441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecdc1597f0 msgr2=0x7fecdc0cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:38.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.527+0000 7fece2800640 1 --2- 172.21.15.161:0/1718139441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecdc1597f0 0x7fecdc0cd040 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fecd40b0040 tx=0x7fecd4019710 comp rx=0 tx=0).stop 2024-05-22T21:24:38.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.527+0000 7fece2800640 1 -- 172.21.15.161:0/1718139441 shutdown_connections 2024-05-22T21:24:38.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.527+0000 7fece2800640 1 --2- 172.21.15.161:0/1718139441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fecb8066690 0x7fecb8068b50 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.527+0000 7fece2800640 1 --2- 172.21.15.161:0/1718139441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecdc1597f0 0x7fecdc0cd040 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.527+0000 7fece2800640 1 --2- 172.21.15.161:0/1718139441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecdc0b3070 0x7fecdc155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.527+0000 7fece2800640 1 --2- 172.21.15.161:0/1718139441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecdc070be0 0x7fecdc146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.527+0000 7fece2800640 1 -- 172.21.15.161:0/1718139441 >> 172.21.15.161:0/1718139441 conn(0x7fecdc0b0bc0 msgr2=0x7fecdc0b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:38.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.527+0000 7fece2800640 1 -- 172.21.15.161:0/1718139441 shutdown_connections 2024-05-22T21:24:38.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.527+0000 7fece2800640 1 -- 172.21.15.161:0/1718139441 wait complete. 2024-05-22T21:24:38.545 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":524,"stamp":"2024-05-22T21:24:36.550572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9479,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553020,"statfs":{"total":1600000000,"available":1590293379,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000743"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.108999+0000","last_change":"2024-05-22T21:24:34.113887+0000","last_active":"2024-05-22T21:24:35.108999+0000","last_peered":"2024-05-22T21:24:35.108999+0000","last_clean":"2024-05-22T21:24:35.108999+0000","last_became_active":"2024-05-22T21:24:34.113298+0000","last_became_peered":"2024-05-22T21:24:34.113298+0000","last_unstale":"2024-05-22T21:24:35.108999+0000","last_undegraded":"2024-05-22T21:24:35.108999+0000","last_fullsized":"2024-05-22T21:24:35.108999+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:10:34.453663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452422+0000","last_change":"2024-05-22T21:24:34.116680+0000","last_active":"2024-05-22T21:24:36.452422+0000","last_peered":"2024-05-22T21:24:36.452422+0000","last_clean":"2024-05-22T21:24:36.452422+0000","last_became_active":"2024-05-22T21:24:34.115180+0000","last_became_peered":"2024-05-22T21:24:34.115180+0000","last_unstale":"2024-05-22T21:24:36.452422+0000","last_undegraded":"2024-05-22T21:24:36.452422+0000","last_fullsized":"2024-05-22T21:24:36.452422+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:22:41.425404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.453002+0000","last_change":"2024-05-22T21:24:34.113906+0000","last_active":"2024-05-22T21:24:36.453002+0000","last_peered":"2024-05-22T21:24:36.453002+0000","last_clean":"2024-05-22T21:24:36.453002+0000","last_became_active":"2024-05-22T21:24:34.113191+0000","last_became_peered":"2024-05-22T21:24:34.113191+0000","last_unstale":"2024-05-22T21:24:36.453002+0000","last_undegraded":"2024-05-22T21:24:36.453002+0000","last_fullsized":"2024-05-22T21:24:36.453002+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:00:28.671610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451007+0000","last_change":"2024-05-22T21:24:34.112336+0000","last_active":"2024-05-22T21:24:36.451007+0000","last_peered":"2024-05-22T21:24:36.451007+0000","last_clean":"2024-05-22T21:24:36.451007+0000","last_became_active":"2024-05-22T21:24:34.112003+0000","last_became_peered":"2024-05-22T21:24:34.112003+0000","last_unstale":"2024-05-22T21:24:36.451007+0000","last_undegraded":"2024-05-22T21:24:36.451007+0000","last_fullsized":"2024-05-22T21:24:36.451007+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:30:31.031688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452953+0000","last_change":"2024-05-22T21:24:34.114924+0000","last_active":"2024-05-22T21:24:36.452953+0000","last_peered":"2024-05-22T21:24:36.452953+0000","last_clean":"2024-05-22T21:24:36.452953+0000","last_became_active":"2024-05-22T21:24:34.114674+0000","last_became_peered":"2024-05-22T21:24:34.114674+0000","last_unstale":"2024-05-22T21:24:36.452953+0000","last_undegraded":"2024-05-22T21:24:36.452953+0000","last_fullsized":"2024-05-22T21:24:36.452953+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:36.398308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450677+0000","last_change":"2024-05-22T21:24:34.110047+0000","last_active":"2024-05-22T21:24:36.450677+0000","last_peered":"2024-05-22T21:24:36.450677+0000","last_clean":"2024-05-22T21:24:36.450677+0000","last_became_active":"2024-05-22T21:24:34.109839+0000","last_became_peered":"2024-05-22T21:24:34.109839+0000","last_unstale":"2024-05-22T21:24:36.450677+0000","last_undegraded":"2024-05-22T21:24:36.450677+0000","last_fullsized":"2024-05-22T21:24:36.450677+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:09:44.245622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451104+0000","last_change":"2024-05-22T21:24:34.110619+0000","last_active":"2024-05-22T21:24:36.451104+0000","last_peered":"2024-05-22T21:24:36.451104+0000","last_clean":"2024-05-22T21:24:36.451104+0000","last_became_active":"2024-05-22T21:24:34.110319+0000","last_became_peered":"2024-05-22T21:24:34.110319+0000","last_unstale":"2024-05-22T21:24:36.451104+0000","last_undegraded":"2024-05-22T21:24:36.451104+0000","last_fullsized":"2024-05-22T21:24:36.451104+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:56:27.788790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452313+0000","last_change":"2024-05-22T21:24:34.114810+0000","last_active":"2024-05-22T21:24:36.452313+0000","last_peered":"2024-05-22T21:24:36.452313+0000","last_clean":"2024-05-22T21:24:36.452313+0000","last_became_active":"2024-05-22T21:24:34.114554+0000","last_became_peered":"2024-05-22T21:24:34.114554+0000","last_unstale":"2024-05-22T21:24:36.452313+0000","last_undegraded":"2024-05-22T21:24:36.452313+0000","last_fullsized":"2024-05-22T21:24:36.452313+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:24:39.530150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452597+0000","last_change":"2024-05-22T21:24:34.111703+0000","last_active":"2024-05-22T21:24:36.452597+0000","last_peered":"2024-05-22T21:24:36.452597+0000","last_clean":"2024-05-22T21:24:36.452597+0000","last_became_active":"2024-05-22T21:24:34.111192+0000","last_became_peered":"2024-05-22T21:24:34.111192+0000","last_unstale":"2024-05-22T21:24:36.452597+0000","last_undegraded":"2024-05-22T21:24:36.452597+0000","last_fullsized":"2024-05-22T21:24:36.452597+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:19:40.350127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.453038+0000","last_change":"2024-05-22T21:24:34.117375+0000","last_active":"2024-05-22T21:24:36.453038+0000","last_peered":"2024-05-22T21:24:36.453038+0000","last_clean":"2024-05-22T21:24:36.453038+0000","last_became_active":"2024-05-22T21:24:34.117119+0000","last_became_peered":"2024-05-22T21:24:34.117119+0000","last_unstale":"2024-05-22T21:24:36.453038+0000","last_undegraded":"2024-05-22T21:24:36.453038+0000","last_fullsized":"2024-05-22T21:24:36.453038+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:28:06.948625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.453357+0000","last_change":"2024-05-22T21:24:34.116325+0000","last_active":"2024-05-22T21:24:36.453357+0000","last_peered":"2024-05-22T21:24:36.453357+0000","last_clean":"2024-05-22T21:24:36.453357+0000","last_became_active":"2024-05-22T21:24:34.116107+0000","last_became_peered":"2024-05-22T21:24:34.116107+0000","last_unstale":"2024-05-22T21:24:36.453357+0000","last_undegraded":"2024-05-22T21:24:36.453357+0000","last_fullsized":"2024-05-22T21:24:36.453357+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:07:48.906348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452534+0000","last_change":"2024-05-22T21:24:34.115392+0000","last_active":"2024-05-22T21:24:36.452534+0000","last_peered":"2024-05-22T21:24:36.452534+0000","last_clean":"2024-05-22T21:24:36.452534+0000","last_became_active":"2024-05-22T21:24:34.114854+0000","last_became_peered":"2024-05-22T21:24:34.114854+0000","last_unstale":"2024-05-22T21:24:36.452534+0000","last_undegraded":"2024-05-22T21:24:36.452534+0000","last_fullsized":"2024-05-22T21:24:36.452534+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:02:21.712263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452687+0000","last_change":"2024-05-22T21:24:34.114415+0000","last_active":"2024-05-22T21:24:36.452687+0000","last_peered":"2024-05-22T21:24:36.452687+0000","last_clean":"2024-05-22T21:24:36.452687+0000","last_became_active":"2024-05-22T21:24:34.114183+0000","last_became_peered":"2024-05-22T21:24:34.114183+0000","last_unstale":"2024-05-22T21:24:36.452687+0000","last_undegraded":"2024-05-22T21:24:36.452687+0000","last_fullsized":"2024-05-22T21:24:36.452687+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:35:38.621202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452392+0000","last_change":"2024-05-22T21:24:34.117506+0000","last_active":"2024-05-22T21:24:36.452392+0000","last_peered":"2024-05-22T21:24:36.452392+0000","last_clean":"2024-05-22T21:24:36.452392+0000","last_became_active":"2024-05-22T21:24:34.117053+0000","last_became_peered":"2024-05-22T21:24:34.117053+0000","last_unstale":"2024-05-22T21:24:36.452392+0000","last_undegraded":"2024-05-22T21:24:36.452392+0000","last_fullsized":"2024-05-22T21:24:36.452392+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:55:04.400396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.108700+0000","last_change":"2024-05-22T21:24:34.111338+0000","last_active":"2024-05-22T21:24:35.108700+0000","last_peered":"2024-05-22T21:24:35.108700+0000","last_clean":"2024-05-22T21:24:35.108700+0000","last_became_active":"2024-05-22T21:24:34.111094+0000","last_became_peered":"2024-05-22T21:24:34.111094+0000","last_unstale":"2024-05-22T21:24:35.108700+0000","last_undegraded":"2024-05-22T21:24:35.108700+0000","last_fullsized":"2024-05-22T21:24:35.108700+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:33:42.410676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451196+0000","last_change":"2024-05-22T21:24:34.119754+0000","last_active":"2024-05-22T21:24:36.451196+0000","last_peered":"2024-05-22T21:24:36.451196+0000","last_clean":"2024-05-22T21:24:36.451196+0000","last_became_active":"2024-05-22T21:24:34.116424+0000","last_became_peered":"2024-05-22T21:24:34.116424+0000","last_unstale":"2024-05-22T21:24:36.451196+0000","last_undegraded":"2024-05-22T21:24:36.451196+0000","last_fullsized":"2024-05-22T21:24:36.451196+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:47:51.640158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451090+0000","last_change":"2024-05-22T21:24:34.118252+0000","last_active":"2024-05-22T21:24:36.451090+0000","last_peered":"2024-05-22T21:24:36.451090+0000","last_clean":"2024-05-22T21:24:36.451090+0000","last_became_active":"2024-05-22T21:24:34.117847+0000","last_became_peered":"2024-05-22T21:24:34.117847+0000","last_unstale":"2024-05-22T21:24:36.451090+0000","last_undegraded":"2024-05-22T21:24:36.451090+0000","last_fullsized":"2024-05-22T21:24:36.451090+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:05:35.517631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106191+0000","last_change":"2024-05-22T21:24:34.116233+0000","last_active":"2024-05-22T21:24:35.106191+0000","last_peered":"2024-05-22T21:24:35.106191+0000","last_clean":"2024-05-22T21:24:35.106191+0000","last_became_active":"2024-05-22T21:24:34.115667+0000","last_became_peered":"2024-05-22T21:24:34.115667+0000","last_unstale":"2024-05-22T21:24:35.106191+0000","last_undegraded":"2024-05-22T21:24:35.106191+0000","last_fullsized":"2024-05-22T21:24:35.106191+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:39:13.059738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451583+0000","last_change":"2024-05-22T21:24:34.115165+0000","last_active":"2024-05-22T21:24:36.451583+0000","last_peered":"2024-05-22T21:24:36.451583+0000","last_clean":"2024-05-22T21:24:36.451583+0000","last_became_active":"2024-05-22T21:24:34.114479+0000","last_became_peered":"2024-05-22T21:24:34.114479+0000","last_unstale":"2024-05-22T21:24:36.451583+0000","last_undegraded":"2024-05-22T21:24:36.451583+0000","last_fullsized":"2024-05-22T21:24:36.451583+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:59:49.394372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452601+0000","last_change":"2024-05-22T21:24:34.116803+0000","last_active":"2024-05-22T21:24:36.452601+0000","last_peered":"2024-05-22T21:24:36.452601+0000","last_clean":"2024-05-22T21:24:36.452601+0000","last_became_active":"2024-05-22T21:24:34.116219+0000","last_became_peered":"2024-05-22T21:24:34.116219+0000","last_unstale":"2024-05-22T21:24:36.452601+0000","last_undegraded":"2024-05-22T21:24:36.452601+0000","last_fullsized":"2024-05-22T21:24:36.452601+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:09:44.245622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450760+0000","last_change":"2024-05-22T21:24:34.119155+0000","last_active":"2024-05-22T21:24:36.450760+0000","last_peered":"2024-05-22T21:24:36.450760+0000","last_clean":"2024-05-22T21:24:36.450760+0000","last_became_active":"2024-05-22T21:24:34.117404+0000","last_became_peered":"2024-05-22T21:24:34.117404+0000","last_unstale":"2024-05-22T21:24:36.450760+0000","last_undegraded":"2024-05-22T21:24:36.450760+0000","last_fullsized":"2024-05-22T21:24:36.450760+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:10:58.574775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452674+0000","last_change":"2024-05-22T21:24:34.111859+0000","last_active":"2024-05-22T21:24:36.452674+0000","last_peered":"2024-05-22T21:24:36.452674+0000","last_clean":"2024-05-22T21:24:36.452674+0000","last_became_active":"2024-05-22T21:24:34.111567+0000","last_became_peered":"2024-05-22T21:24:34.111567+0000","last_unstale":"2024-05-22T21:24:36.452674+0000","last_undegraded":"2024-05-22T21:24:36.452674+0000","last_fullsized":"2024-05-22T21:24:36.452674+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:35:47.933629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450782+0000","last_change":"2024-05-22T21:24:34.114137+0000","last_active":"2024-05-22T21:24:36.450782+0000","last_peered":"2024-05-22T21:24:36.450782+0000","last_clean":"2024-05-22T21:24:36.450782+0000","last_became_active":"2024-05-22T21:24:34.113933+0000","last_became_peered":"2024-05-22T21:24:34.113933+0000","last_unstale":"2024-05-22T21:24:36.450782+0000","last_undegraded":"2024-05-22T21:24:36.450782+0000","last_fullsized":"2024-05-22T21:24:36.450782+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:58:32.687466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452315+0000","last_change":"2024-05-22T21:24:34.112238+0000","last_active":"2024-05-22T21:24:36.452315+0000","last_peered":"2024-05-22T21:24:36.452315+0000","last_clean":"2024-05-22T21:24:36.452315+0000","last_became_active":"2024-05-22T21:24:34.111480+0000","last_became_peered":"2024-05-22T21:24:34.111480+0000","last_unstale":"2024-05-22T21:24:36.452315+0000","last_undegraded":"2024-05-22T21:24:36.452315+0000","last_fullsized":"2024-05-22T21:24:36.452315+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:04:25.576271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452723+0000","last_change":"2024-05-22T21:24:34.114269+0000","last_active":"2024-05-22T21:24:36.452723+0000","last_peered":"2024-05-22T21:24:36.452723+0000","last_clean":"2024-05-22T21:24:36.452723+0000","last_became_active":"2024-05-22T21:24:34.114027+0000","last_became_peered":"2024-05-22T21:24:34.114027+0000","last_unstale":"2024-05-22T21:24:36.452723+0000","last_undegraded":"2024-05-22T21:24:36.452723+0000","last_fullsized":"2024-05-22T21:24:36.452723+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:19:52.242393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451106+0000","last_change":"2024-05-22T21:24:34.109702+0000","last_active":"2024-05-22T21:24:36.451106+0000","last_peered":"2024-05-22T21:24:36.451106+0000","last_clean":"2024-05-22T21:24:36.451106+0000","last_became_active":"2024-05-22T21:24:34.109443+0000","last_became_peered":"2024-05-22T21:24:34.109443+0000","last_unstale":"2024-05-22T21:24:36.451106+0000","last_undegraded":"2024-05-22T21:24:36.451106+0000","last_fullsized":"2024-05-22T21:24:36.451106+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:14:06.124437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452838+0000","last_change":"2024-05-22T21:24:34.111487+0000","last_active":"2024-05-22T21:24:36.452838+0000","last_peered":"2024-05-22T21:24:36.452838+0000","last_clean":"2024-05-22T21:24:36.452838+0000","last_became_active":"2024-05-22T21:24:34.111244+0000","last_became_peered":"2024-05-22T21:24:34.111244+0000","last_unstale":"2024-05-22T21:24:36.452838+0000","last_undegraded":"2024-05-22T21:24:36.452838+0000","last_fullsized":"2024-05-22T21:24:36.452838+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:14:40.086666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451142+0000","last_change":"2024-05-22T21:24:34.105705+0000","last_active":"2024-05-22T21:24:36.451142+0000","last_peered":"2024-05-22T21:24:36.451142+0000","last_clean":"2024-05-22T21:24:36.451142+0000","last_became_active":"2024-05-22T21:24:34.105479+0000","last_became_peered":"2024-05-22T21:24:34.105479+0000","last_unstale":"2024-05-22T21:24:36.451142+0000","last_undegraded":"2024-05-22T21:24:36.451142+0000","last_fullsized":"2024-05-22T21:24:36.451142+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:25:40.227199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452500+0000","last_change":"2024-05-22T21:24:34.117364+0000","last_active":"2024-05-22T21:24:36.452500+0000","last_peered":"2024-05-22T21:24:36.452500+0000","last_clean":"2024-05-22T21:24:36.452500+0000","last_became_active":"2024-05-22T21:24:34.116751+0000","last_became_peered":"2024-05-22T21:24:34.116751+0000","last_unstale":"2024-05-22T21:24:36.452500+0000","last_undegraded":"2024-05-22T21:24:36.452500+0000","last_fullsized":"2024-05-22T21:24:36.452500+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:12:42.267144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109616+0000","last_change":"2024-05-22T21:24:34.106292+0000","last_active":"2024-05-22T21:24:35.109616+0000","last_peered":"2024-05-22T21:24:35.109616+0000","last_clean":"2024-05-22T21:24:35.109616+0000","last_became_active":"2024-05-22T21:24:34.106043+0000","last_became_peered":"2024-05-22T21:24:34.106043+0000","last_unstale":"2024-05-22T21:24:35.109616+0000","last_undegraded":"2024-05-22T21:24:35.109616+0000","last_fullsized":"2024-05-22T21:24:35.109616+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:11:39.188888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109668+0000","last_change":"2024-05-22T21:24:34.111431+0000","last_active":"2024-05-22T21:24:35.109668+0000","last_peered":"2024-05-22T21:24:35.109668+0000","last_clean":"2024-05-22T21:24:35.109668+0000","last_became_active":"2024-05-22T21:24:34.111089+0000","last_became_peered":"2024-05-22T21:24:34.111089+0000","last_unstale":"2024-05-22T21:24:35.109668+0000","last_undegraded":"2024-05-22T21:24:35.109668+0000","last_fullsized":"2024-05-22T21:24:35.109668+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:23:28.758003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106081+0000","last_change":"2024-05-22T21:24:34.115265+0000","last_active":"2024-05-22T21:24:35.106081+0000","last_peered":"2024-05-22T21:24:35.106081+0000","last_clean":"2024-05-22T21:24:35.106081+0000","last_became_active":"2024-05-22T21:24:34.114756+0000","last_became_peered":"2024-05-22T21:24:34.114756+0000","last_unstale":"2024-05-22T21:24:35.106081+0000","last_undegraded":"2024-05-22T21:24:35.106081+0000","last_fullsized":"2024-05-22T21:24:35.106081+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:23:47.941680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450561+0000","last_change":"2024-05-22T21:24:34.108101+0000","last_active":"2024-05-22T21:24:36.450561+0000","last_peered":"2024-05-22T21:24:36.450561+0000","last_clean":"2024-05-22T21:24:36.450561+0000","last_became_active":"2024-05-22T21:24:34.107898+0000","last_became_peered":"2024-05-22T21:24:34.107898+0000","last_unstale":"2024-05-22T21:24:36.450561+0000","last_undegraded":"2024-05-22T21:24:36.450561+0000","last_fullsized":"2024-05-22T21:24:36.450561+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:31:45.904007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"189'4","reported_seq":15,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.108595+0000","last_change":"2024-05-22T21:24:33.737621+0000","last_active":"2024-05-22T21:24:35.108595+0000","last_peered":"2024-05-22T21:24:35.108595+0000","last_clean":"2024-05-22T21:24:35.108595+0000","last_became_active":"2024-05-22T21:24:33.737371+0000","last_became_peered":"2024-05-22T21:24:33.737371+0000","last_unstale":"2024-05-22T21:24:35.108595+0000","last_undegraded":"2024-05-22T21:24:35.108595+0000","last_fullsized":"2024-05-22T21:24:35.108595+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:54:11.566309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109561+0000","last_change":"2024-05-22T21:24:34.115450+0000","last_active":"2024-05-22T21:24:35.109561+0000","last_peered":"2024-05-22T21:24:35.109561+0000","last_clean":"2024-05-22T21:24:35.109561+0000","last_became_active":"2024-05-22T21:24:34.115228+0000","last_became_peered":"2024-05-22T21:24:34.115228+0000","last_unstale":"2024-05-22T21:24:35.109561+0000","last_undegraded":"2024-05-22T21:24:35.109561+0000","last_fullsized":"2024-05-22T21:24:35.109561+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:34:13.785204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"189'1","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109306+0000","last_change":"2024-05-22T21:24:33.733654+0000","last_active":"2024-05-22T21:24:35.109306+0000","last_peered":"2024-05-22T21:24:35.109306+0000","last_clean":"2024-05-22T21:24:35.109306+0000","last_became_active":"2024-05-22T21:24:33.730353+0000","last_became_peered":"2024-05-22T21:24:33.730353+0000","last_unstale":"2024-05-22T21:24:35.109306+0000","last_undegraded":"2024-05-22T21:24:35.109306+0000","last_fullsized":"2024-05-22T21:24:35.109306+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:27:57.830133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450840+0000","last_change":"2024-05-22T21:24:34.114629+0000","last_active":"2024-05-22T21:24:36.450840+0000","last_peered":"2024-05-22T21:24:36.450840+0000","last_clean":"2024-05-22T21:24:36.450840+0000","last_became_active":"2024-05-22T21:24:34.114368+0000","last_became_peered":"2024-05-22T21:24:34.114368+0000","last_unstale":"2024-05-22T21:24:36.450840+0000","last_undegraded":"2024-05-22T21:24:36.450840+0000","last_fullsized":"2024-05-22T21:24:36.450840+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:35:38.621202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"189'2","reported_seq":13,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451640+0000","last_change":"2024-05-22T21:24:33.731105+0000","last_active":"2024-05-22T21:24:36.451640+0000","last_peered":"2024-05-22T21:24:36.451640+0000","last_clean":"2024-05-22T21:24:36.451640+0000","last_became_active":"2024-05-22T21:24:33.730882+0000","last_became_peered":"2024-05-22T21:24:33.730882+0000","last_unstale":"2024-05-22T21:24:36.451640+0000","last_undegraded":"2024-05-22T21:24:36.451640+0000","last_fullsized":"2024-05-22T21:24:36.451640+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:00:27.664691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452828+0000","last_change":"2024-05-22T21:24:34.116987+0000","last_active":"2024-05-22T21:24:36.452828+0000","last_peered":"2024-05-22T21:24:36.452828+0000","last_clean":"2024-05-22T21:24:36.452828+0000","last_became_active":"2024-05-22T21:24:34.116508+0000","last_became_peered":"2024-05-22T21:24:34.116508+0000","last_unstale":"2024-05-22T21:24:36.452828+0000","last_undegraded":"2024-05-22T21:24:36.452828+0000","last_fullsized":"2024-05-22T21:24:36.452828+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:28:27.579964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"189'2","reported_seq":13,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451486+0000","last_change":"2024-05-22T21:24:33.732542+0000","last_active":"2024-05-22T21:24:36.451486+0000","last_peered":"2024-05-22T21:24:36.451486+0000","last_clean":"2024-05-22T21:24:36.451486+0000","last_became_active":"2024-05-22T21:24:33.732322+0000","last_became_peered":"2024-05-22T21:24:33.732322+0000","last_unstale":"2024-05-22T21:24:36.451486+0000","last_undegraded":"2024-05-22T21:24:36.451486+0000","last_fullsized":"2024-05-22T21:24:36.451486+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:18:21.802589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450704+0000","last_change":"2024-05-22T21:24:34.113280+0000","last_active":"2024-05-22T21:24:36.450704+0000","last_peered":"2024-05-22T21:24:36.450704+0000","last_clean":"2024-05-22T21:24:36.450704+0000","last_became_active":"2024-05-22T21:24:34.113007+0000","last_became_peered":"2024-05-22T21:24:34.113007+0000","last_unstale":"2024-05-22T21:24:36.450704+0000","last_undegraded":"2024-05-22T21:24:36.450704+0000","last_fullsized":"2024-05-22T21:24:36.450704+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:04:25.576271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"189'1","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452999+0000","last_change":"2024-05-22T21:24:33.737047+0000","last_active":"2024-05-22T21:24:36.452999+0000","last_peered":"2024-05-22T21:24:36.452999+0000","last_clean":"2024-05-22T21:24:36.452999+0000","last_became_active":"2024-05-22T21:24:33.736636+0000","last_became_peered":"2024-05-22T21:24:33.736636+0000","last_unstale":"2024-05-22T21:24:36.452999+0000","last_undegraded":"2024-05-22T21:24:36.452999+0000","last_fullsized":"2024-05-22T21:24:36.452999+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:20:16.845129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451980+0000","last_change":"2024-05-22T21:24:34.108250+0000","last_active":"2024-05-22T21:24:36.451980+0000","last_peered":"2024-05-22T21:24:36.451980+0000","last_clean":"2024-05-22T21:24:36.451980+0000","last_became_active":"2024-05-22T21:24:34.107431+0000","last_became_peered":"2024-05-22T21:24:34.107431+0000","last_unstale":"2024-05-22T21:24:36.451980+0000","last_undegraded":"2024-05-22T21:24:36.451980+0000","last_fullsized":"2024-05-22T21:24:36.451980+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:27:58.837052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"189'1","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451227+0000","last_change":"2024-05-22T21:24:33.732662+0000","last_active":"2024-05-22T21:24:36.451227+0000","last_peered":"2024-05-22T21:24:36.451227+0000","last_clean":"2024-05-22T21:24:36.451227+0000","last_became_active":"2024-05-22T21:24:33.732302+0000","last_became_peered":"2024-05-22T21:24:33.732302+0000","last_unstale":"2024-05-22T21:24:36.451227+0000","last_undegraded":"2024-05-22T21:24:36.451227+0000","last_fullsized":"2024-05-22T21:24:36.451227+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:05:34.510712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451412+0000","last_change":"2024-05-22T21:24:34.105228+0000","last_active":"2024-05-22T21:24:36.451412+0000","last_peered":"2024-05-22T21:24:36.451412+0000","last_clean":"2024-05-22T21:24:36.451412+0000","last_became_active":"2024-05-22T21:24:34.105011+0000","last_became_peered":"2024-05-22T21:24:34.105011+0000","last_unstale":"2024-05-22T21:24:36.451412+0000","last_undegraded":"2024-05-22T21:24:36.451412+0000","last_fullsized":"2024-05-22T21:24:36.451412+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:34:14.259089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452490+0000","last_change":"2024-05-22T21:24:33.739724+0000","last_active":"2024-05-22T21:24:36.452490+0000","last_peered":"2024-05-22T21:24:36.452490+0000","last_clean":"2024-05-22T21:24:36.452490+0000","last_became_active":"2024-05-22T21:24:33.739165+0000","last_became_peered":"2024-05-22T21:24:33.739165+0000","last_unstale":"2024-05-22T21:24:36.452490+0000","last_undegraded":"2024-05-22T21:24:36.452490+0000","last_fullsized":"2024-05-22T21:24:36.452490+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:11:38.181969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451657+0000","last_change":"2024-05-22T21:24:34.118026+0000","last_active":"2024-05-22T21:24:36.451657+0000","last_peered":"2024-05-22T21:24:36.451657+0000","last_clean":"2024-05-22T21:24:36.451657+0000","last_became_active":"2024-05-22T21:24:34.117822+0000","last_became_peered":"2024-05-22T21:24:34.117822+0000","last_unstale":"2024-05-22T21:24:36.451657+0000","last_undegraded":"2024-05-22T21:24:36.451657+0000","last_fullsized":"2024-05-22T21:24:36.451657+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:31:37.180335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451473+0000","last_change":"2024-05-22T21:24:33.728849+0000","last_active":"2024-05-22T21:24:36.451473+0000","last_peered":"2024-05-22T21:24:36.451473+0000","last_clean":"2024-05-22T21:24:36.451473+0000","last_became_active":"2024-05-22T21:24:33.728637+0000","last_became_peered":"2024-05-22T21:24:33.728637+0000","last_unstale":"2024-05-22T21:24:36.451473+0000","last_undegraded":"2024-05-22T21:24:36.451473+0000","last_fullsized":"2024-05-22T21:24:36.451473+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:29:46.561661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106052+0000","last_change":"2024-05-22T21:24:34.116873+0000","last_active":"2024-05-22T21:24:35.106052+0000","last_peered":"2024-05-22T21:24:35.106052+0000","last_clean":"2024-05-22T21:24:35.106052+0000","last_became_active":"2024-05-22T21:24:34.116343+0000","last_became_peered":"2024-05-22T21:24:34.116343+0000","last_unstale":"2024-05-22T21:24:35.106052+0000","last_undegraded":"2024-05-22T21:24:35.106052+0000","last_fullsized":"2024-05-22T21:24:35.106052+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:28:06.948625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451161+0000","last_change":"2024-05-22T21:24:33.736492+0000","last_active":"2024-05-22T21:24:36.451161+0000","last_peered":"2024-05-22T21:24:36.451161+0000","last_clean":"2024-05-22T21:24:36.451161+0000","last_became_active":"2024-05-22T21:24:33.736255+0000","last_became_peered":"2024-05-22T21:24:33.736255+0000","last_unstale":"2024-05-22T21:24:36.451161+0000","last_undegraded":"2024-05-22T21:24:36.451161+0000","last_fullsized":"2024-05-22T21:24:36.451161+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:55:41.991906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106095+0000","last_change":"2024-05-22T21:24:34.111367+0000","last_active":"2024-05-22T21:24:35.106095+0000","last_peered":"2024-05-22T21:24:35.106095+0000","last_clean":"2024-05-22T21:24:35.106095+0000","last_became_active":"2024-05-22T21:24:34.111161+0000","last_became_peered":"2024-05-22T21:24:34.111161+0000","last_unstale":"2024-05-22T21:24:35.106095+0000","last_undegraded":"2024-05-22T21:24:35.106095+0000","last_fullsized":"2024-05-22T21:24:35.106095+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:14:06.124437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"189'3","reported_seq":14,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450870+0000","last_change":"2024-05-22T21:24:33.734810+0000","last_active":"2024-05-22T21:24:36.450870+0000","last_peered":"2024-05-22T21:24:36.450870+0000","last_clean":"2024-05-22T21:24:36.450870+0000","last_became_active":"2024-05-22T21:24:33.734579+0000","last_became_peered":"2024-05-22T21:24:33.734579+0000","last_unstale":"2024-05-22T21:24:36.450870+0000","last_undegraded":"2024-05-22T21:24:36.450870+0000","last_fullsized":"2024-05-22T21:24:36.450870+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:13:43.138691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109675+0000","last_change":"2024-05-22T21:24:34.109809+0000","last_active":"2024-05-22T21:24:35.109675+0000","last_peered":"2024-05-22T21:24:35.109675+0000","last_clean":"2024-05-22T21:24:35.109675+0000","last_became_active":"2024-05-22T21:24:34.109538+0000","last_became_peered":"2024-05-22T21:24:34.109538+0000","last_unstale":"2024-05-22T21:24:35.109675+0000","last_undegraded":"2024-05-22T21:24:35.109675+0000","last_fullsized":"2024-05-22T21:24:35.109675+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:18:42.658301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"189'6","reported_seq":17,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109541+0000","last_change":"2024-05-22T21:24:33.730577+0000","last_active":"2024-05-22T21:24:35.109541+0000","last_peered":"2024-05-22T21:24:35.109541+0000","last_clean":"2024-05-22T21:24:35.109541+0000","last_became_active":"2024-05-22T21:24:33.730000+0000","last_became_peered":"2024-05-22T21:24:33.730000+0000","last_unstale":"2024-05-22T21:24:35.109541+0000","last_undegraded":"2024-05-22T21:24:35.109541+0000","last_fullsized":"2024-05-22T21:24:35.109541+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:26:36.568013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106147+0000","last_change":"2024-05-22T21:24:34.118827+0000","last_active":"2024-05-22T21:24:35.106147+0000","last_peered":"2024-05-22T21:24:35.106147+0000","last_clean":"2024-05-22T21:24:35.106147+0000","last_became_active":"2024-05-22T21:24:34.116654+0000","last_became_peered":"2024-05-22T21:24:34.116654+0000","last_unstale":"2024-05-22T21:24:35.106147+0000","last_undegraded":"2024-05-22T21:24:35.106147+0000","last_fullsized":"2024-05-22T21:24:35.106147+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:30:37.702584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"189'2","reported_seq":13,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451723+0000","last_change":"2024-05-22T21:24:33.727854+0000","last_active":"2024-05-22T21:24:36.451723+0000","last_peered":"2024-05-22T21:24:36.451723+0000","last_clean":"2024-05-22T21:24:36.451723+0000","last_became_active":"2024-05-22T21:24:33.727595+0000","last_became_peered":"2024-05-22T21:24:33.727595+0000","last_unstale":"2024-05-22T21:24:36.451723+0000","last_undegraded":"2024-05-22T21:24:36.451723+0000","last_fullsized":"2024-05-22T21:24:36.451723+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:09:43.238703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451396+0000","last_change":"2024-05-22T21:24:34.110202+0000","last_active":"2024-05-22T21:24:36.451396+0000","last_peered":"2024-05-22T21:24:36.451396+0000","last_clean":"2024-05-22T21:24:36.451396+0000","last_became_active":"2024-05-22T21:24:34.109974+0000","last_became_peered":"2024-05-22T21:24:34.109974+0000","last_unstale":"2024-05-22T21:24:36.451396+0000","last_undegraded":"2024-05-22T21:24:36.451396+0000","last_fullsized":"2024-05-22T21:24:36.451396+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:20:03.445202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106349+0000","last_change":"2024-05-22T21:24:33.731871+0000","last_active":"2024-05-22T21:24:35.106349+0000","last_peered":"2024-05-22T21:24:35.106349+0000","last_clean":"2024-05-22T21:24:35.106349+0000","last_became_active":"2024-05-22T21:24:33.731479+0000","last_became_peered":"2024-05-22T21:24:33.731479+0000","last_unstale":"2024-05-22T21:24:35.106349+0000","last_undegraded":"2024-05-22T21:24:35.106349+0000","last_fullsized":"2024-05-22T21:24:35.106349+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:42:25.944396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451058+0000","last_change":"2024-05-22T21:24:34.112801+0000","last_active":"2024-05-22T21:24:36.451058+0000","last_peered":"2024-05-22T21:24:36.451058+0000","last_clean":"2024-05-22T21:24:36.451058+0000","last_became_active":"2024-05-22T21:24:34.112525+0000","last_became_peered":"2024-05-22T21:24:34.112525+0000","last_unstale":"2024-05-22T21:24:36.451058+0000","last_undegraded":"2024-05-22T21:24:36.451058+0000","last_fullsized":"2024-05-22T21:24:36.451058+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:31:23.471827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452207+0000","last_change":"2024-05-22T21:24:33.730489+0000","last_active":"2024-05-22T21:24:36.452207+0000","last_peered":"2024-05-22T21:24:36.452207+0000","last_clean":"2024-05-22T21:24:36.452207+0000","last_became_active":"2024-05-22T21:24:33.730272+0000","last_became_peered":"2024-05-22T21:24:33.730272+0000","last_unstale":"2024-05-22T21:24:36.452207+0000","last_undegraded":"2024-05-22T21:24:36.452207+0000","last_fullsized":"2024-05-22T21:24:36.452207+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:31:36.173417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452905+0000","last_change":"2024-05-22T21:24:34.115931+0000","last_active":"2024-05-22T21:24:36.452905+0000","last_peered":"2024-05-22T21:24:36.452905+0000","last_clean":"2024-05-22T21:24:36.452905+0000","last_became_active":"2024-05-22T21:24:34.115647+0000","last_became_peered":"2024-05-22T21:24:34.115647+0000","last_unstale":"2024-05-22T21:24:36.452905+0000","last_undegraded":"2024-05-22T21:24:36.452905+0000","last_fullsized":"2024-05-22T21:24:36.452905+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:18:22.809508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"189'8","reported_seq":19,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109758+0000","last_change":"2024-05-22T21:24:33.728520+0000","last_active":"2024-05-22T21:24:35.109758+0000","last_peered":"2024-05-22T21:24:35.109758+0000","last_clean":"2024-05-22T21:24:35.109758+0000","last_became_active":"2024-05-22T21:24:33.728263+0000","last_became_peered":"2024-05-22T21:24:33.728263+0000","last_unstale":"2024-05-22T21:24:35.109758+0000","last_undegraded":"2024-05-22T21:24:35.109758+0000","last_fullsized":"2024-05-22T21:24:35.109758+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:32:13.689008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451558+0000","last_change":"2024-05-22T21:24:34.114448+0000","last_active":"2024-05-22T21:24:36.451558+0000","last_peered":"2024-05-22T21:24:36.451558+0000","last_clean":"2024-05-22T21:24:36.451558+0000","last_became_active":"2024-05-22T21:24:34.114182+0000","last_became_peered":"2024-05-22T21:24:34.114182+0000","last_unstale":"2024-05-22T21:24:36.451558+0000","last_undegraded":"2024-05-22T21:24:36.451558+0000","last_fullsized":"2024-05-22T21:24:36.451558+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:33:12.291343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"189'1","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452721+0000","last_change":"2024-05-22T21:24:33.736025+0000","last_active":"2024-05-22T21:24:36.452721+0000","last_peered":"2024-05-22T21:24:36.452721+0000","last_clean":"2024-05-22T21:24:36.452721+0000","last_became_active":"2024-05-22T21:24:33.735545+0000","last_became_peered":"2024-05-22T21:24:33.735545+0000","last_unstale":"2024-05-22T21:24:36.452721+0000","last_undegraded":"2024-05-22T21:24:36.452721+0000","last_fullsized":"2024-05-22T21:24:36.452721+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:32:09.388182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106200+0000","last_change":"2024-05-22T21:24:34.118506+0000","last_active":"2024-05-22T21:24:35.106200+0000","last_peered":"2024-05-22T21:24:35.106200+0000","last_clean":"2024-05-22T21:24:35.106200+0000","last_became_active":"2024-05-22T21:24:34.113673+0000","last_became_peered":"2024-05-22T21:24:34.113673+0000","last_unstale":"2024-05-22T21:24:35.106200+0000","last_undegraded":"2024-05-22T21:24:35.106200+0000","last_fullsized":"2024-05-22T21:24:35.106200+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:04:25.576271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"189'2","reported_seq":13,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.453199+0000","last_change":"2024-05-22T21:24:33.739366+0000","last_active":"2024-05-22T21:24:36.453199+0000","last_peered":"2024-05-22T21:24:36.453199+0000","last_clean":"2024-05-22T21:24:36.453199+0000","last_became_active":"2024-05-22T21:24:33.738926+0000","last_became_peered":"2024-05-22T21:24:33.738926+0000","last_unstale":"2024-05-22T21:24:36.453199+0000","last_undegraded":"2024-05-22T21:24:36.453199+0000","last_fullsized":"2024-05-22T21:24:36.453199+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:27:37.325046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452125+0000","last_change":"2024-05-22T21:24:34.107683+0000","last_active":"2024-05-22T21:24:36.452125+0000","last_peered":"2024-05-22T21:24:36.452125+0000","last_clean":"2024-05-22T21:24:36.452125+0000","last_became_active":"2024-05-22T21:24:34.107145+0000","last_became_peered":"2024-05-22T21:24:34.107145+0000","last_unstale":"2024-05-22T21:24:36.452125+0000","last_undegraded":"2024-05-22T21:24:36.452125+0000","last_fullsized":"2024-05-22T21:24:36.452125+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:26:37.574931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"189'1","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109089+0000","last_change":"2024-05-22T21:24:33.732972+0000","last_active":"2024-05-22T21:24:35.109089+0000","last_peered":"2024-05-22T21:24:35.109089+0000","last_clean":"2024-05-22T21:24:35.109089+0000","last_became_active":"2024-05-22T21:24:33.732310+0000","last_became_peered":"2024-05-22T21:24:33.732310+0000","last_unstale":"2024-05-22T21:24:35.109089+0000","last_undegraded":"2024-05-22T21:24:35.109089+0000","last_fullsized":"2024-05-22T21:24:35.109089+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T07:06:06.856153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451280+0000","last_change":"2024-05-22T21:24:34.116623+0000","last_active":"2024-05-22T21:24:36.451280+0000","last_peered":"2024-05-22T21:24:36.451280+0000","last_clean":"2024-05-22T21:24:36.451280+0000","last_became_active":"2024-05-22T21:24:34.116129+0000","last_became_peered":"2024-05-22T21:24:34.116129+0000","last_unstale":"2024-05-22T21:24:36.451280+0000","last_undegraded":"2024-05-22T21:24:36.451280+0000","last_fullsized":"2024-05-22T21:24:36.451280+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:39:13.059738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.108808+0000","last_change":"2024-05-22T21:24:33.737995+0000","last_active":"2024-05-22T21:24:35.108808+0000","last_peered":"2024-05-22T21:24:35.108808+0000","last_clean":"2024-05-22T21:24:35.108808+0000","last_became_active":"2024-05-22T21:24:33.737732+0000","last_became_peered":"2024-05-22T21:24:33.737732+0000","last_unstale":"2024-05-22T21:24:35.108808+0000","last_undegraded":"2024-05-22T21:24:35.108808+0000","last_fullsized":"2024-05-22T21:24:35.108808+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:49:38.137481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452612+0000","last_change":"2024-05-22T21:24:34.115193+0000","last_active":"2024-05-22T21:24:36.452612+0000","last_peered":"2024-05-22T21:24:36.452612+0000","last_clean":"2024-05-22T21:24:36.452612+0000","last_became_active":"2024-05-22T21:24:34.113165+0000","last_became_peered":"2024-05-22T21:24:34.113165+0000","last_unstale":"2024-05-22T21:24:36.452612+0000","last_undegraded":"2024-05-22T21:24:36.452612+0000","last_fullsized":"2024-05-22T21:24:36.452612+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:51:26.318295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"189'2","reported_seq":13,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451317+0000","last_change":"2024-05-22T21:24:33.732530+0000","last_active":"2024-05-22T21:24:36.451317+0000","last_peered":"2024-05-22T21:24:36.451317+0000","last_clean":"2024-05-22T21:24:36.451317+0000","last_became_active":"2024-05-22T21:24:33.731856+0000","last_became_peered":"2024-05-22T21:24:33.731856+0000","last_unstale":"2024-05-22T21:24:36.451317+0000","last_undegraded":"2024-05-22T21:24:36.451317+0000","last_fullsized":"2024-05-22T21:24:36.451317+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:10:57.567857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450839+0000","last_change":"2024-05-22T21:24:34.116014+0000","last_active":"2024-05-22T21:24:36.450839+0000","last_peered":"2024-05-22T21:24:36.450839+0000","last_clean":"2024-05-22T21:24:36.450839+0000","last_became_active":"2024-05-22T21:24:34.115805+0000","last_became_peered":"2024-05-22T21:24:34.115805+0000","last_unstale":"2024-05-22T21:24:36.450839+0000","last_undegraded":"2024-05-22T21:24:36.450839+0000","last_fullsized":"2024-05-22T21:24:36.450839+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:28:06.948625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.108791+0000","last_change":"2024-05-22T21:24:33.733103+0000","last_active":"2024-05-22T21:24:35.108791+0000","last_peered":"2024-05-22T21:24:35.108791+0000","last_clean":"2024-05-22T21:24:35.108791+0000","last_became_active":"2024-05-22T21:24:33.732341+0000","last_became_peered":"2024-05-22T21:24:33.732341+0000","last_unstale":"2024-05-22T21:24:35.108791+0000","last_undegraded":"2024-05-22T21:24:35.108791+0000","last_fullsized":"2024-05-22T21:24:35.108791+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T01:22:45.506309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.108902+0000","last_change":"2024-05-22T21:24:34.114041+0000","last_active":"2024-05-22T21:24:35.108902+0000","last_peered":"2024-05-22T21:24:35.108902+0000","last_clean":"2024-05-22T21:24:35.108902+0000","last_became_active":"2024-05-22T21:24:34.113589+0000","last_became_peered":"2024-05-22T21:24:34.113589+0000","last_unstale":"2024-05-22T21:24:35.108902+0000","last_undegraded":"2024-05-22T21:24:35.108902+0000","last_fullsized":"2024-05-22T21:24:35.108902+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:22:09.318242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451797+0000","last_change":"2024-05-22T21:24:33.734864+0000","last_active":"2024-05-22T21:24:36.451797+0000","last_peered":"2024-05-22T21:24:36.451797+0000","last_clean":"2024-05-22T21:24:36.451797+0000","last_became_active":"2024-05-22T21:24:33.734649+0000","last_became_peered":"2024-05-22T21:24:33.734649+0000","last_unstale":"2024-05-22T21:24:36.451797+0000","last_undegraded":"2024-05-22T21:24:36.451797+0000","last_fullsized":"2024-05-22T21:24:36.451797+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:35:37.614284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451641+0000","last_change":"2024-05-22T21:24:34.115089+0000","last_active":"2024-05-22T21:24:36.451641+0000","last_peered":"2024-05-22T21:24:36.451641+0000","last_clean":"2024-05-22T21:24:36.451641+0000","last_became_active":"2024-05-22T21:24:34.114873+0000","last_became_peered":"2024-05-22T21:24:34.114873+0000","last_unstale":"2024-05-22T21:24:36.451641+0000","last_undegraded":"2024-05-22T21:24:36.451641+0000","last_fullsized":"2024-05-22T21:24:36.451641+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:02:58.346797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109178+0000","last_change":"2024-05-22T21:24:33.733126+0000","last_active":"2024-05-22T21:24:35.109178+0000","last_peered":"2024-05-22T21:24:35.109178+0000","last_clean":"2024-05-22T21:24:35.109178+0000","last_became_active":"2024-05-22T21:24:33.732702+0000","last_became_peered":"2024-05-22T21:24:33.732702+0000","last_unstale":"2024-05-22T21:24:35.109178+0000","last_undegraded":"2024-05-22T21:24:35.109178+0000","last_fullsized":"2024-05-22T21:24:35.109178+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:35:40.028344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106282+0000","last_change":"2024-05-22T21:24:34.114522+0000","last_active":"2024-05-22T21:24:35.106282+0000","last_peered":"2024-05-22T21:24:35.106282+0000","last_clean":"2024-05-22T21:24:35.106282+0000","last_became_active":"2024-05-22T21:24:34.114297+0000","last_became_peered":"2024-05-22T21:24:34.114297+0000","last_unstale":"2024-05-22T21:24:35.106282+0000","last_undegraded":"2024-05-22T21:24:35.106282+0000","last_fullsized":"2024-05-22T21:24:35.106282+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:24:39.530150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"189'2","reported_seq":13,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106381+0000","last_change":"2024-05-22T21:24:33.733577+0000","last_active":"2024-05-22T21:24:35.106381+0000","last_peered":"2024-05-22T21:24:35.106381+0000","last_clean":"2024-05-22T21:24:35.106381+0000","last_became_active":"2024-05-22T21:24:33.732656+0000","last_became_peered":"2024-05-22T21:24:33.732656+0000","last_unstale":"2024-05-22T21:24:35.106381+0000","last_undegraded":"2024-05-22T21:24:35.106381+0000","last_fullsized":"2024-05-22T21:24:35.106381+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:12:37.414160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451191+0000","last_change":"2024-05-22T21:24:34.113899+0000","last_active":"2024-05-22T21:24:36.451191+0000","last_peered":"2024-05-22T21:24:36.451191+0000","last_clean":"2024-05-22T21:24:36.451191+0000","last_became_active":"2024-05-22T21:24:34.113610+0000","last_became_peered":"2024-05-22T21:24:34.113610+0000","last_unstale":"2024-05-22T21:24:36.451191+0000","last_undegraded":"2024-05-22T21:24:36.451191+0000","last_fullsized":"2024-05-22T21:24:36.451191+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:30:37.702584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.453088+0000","last_change":"2024-05-22T21:24:33.737243+0000","last_active":"2024-05-22T21:24:36.453088+0000","last_peered":"2024-05-22T21:24:36.453088+0000","last_clean":"2024-05-22T21:24:36.453088+0000","last_became_active":"2024-05-22T21:24:33.736981+0000","last_became_peered":"2024-05-22T21:24:33.736981+0000","last_unstale":"2024-05-22T21:24:36.453088+0000","last_undegraded":"2024-05-22T21:24:36.453088+0000","last_fullsized":"2024-05-22T21:24:36.453088+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T22:32:03.762097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.451725+0000","last_change":"2024-05-22T21:24:34.115655+0000","last_active":"2024-05-22T21:24:36.451725+0000","last_peered":"2024-05-22T21:24:36.451725+0000","last_clean":"2024-05-22T21:24:36.451725+0000","last_became_active":"2024-05-22T21:24:34.115406+0000","last_became_peered":"2024-05-22T21:24:34.115406+0000","last_unstale":"2024-05-22T21:24:36.451725+0000","last_undegraded":"2024-05-22T21:24:36.451725+0000","last_fullsized":"2024-05-22T21:24:36.451725+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T06:56:01.598828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452702+0000","last_change":"2024-05-22T21:24:33.739499+0000","last_active":"2024-05-22T21:24:36.452702+0000","last_peered":"2024-05-22T21:24:36.452702+0000","last_clean":"2024-05-22T21:24:36.452702+0000","last_became_active":"2024-05-22T21:24:33.738730+0000","last_became_peered":"2024-05-22T21:24:33.738730+0000","last_unstale":"2024-05-22T21:24:36.452702+0000","last_undegraded":"2024-05-22T21:24:36.452702+0000","last_fullsized":"2024-05-22T21:24:36.452702+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:42:25.944396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106289+0000","last_change":"2024-05-22T21:24:34.108832+0000","last_active":"2024-05-22T21:24:35.106289+0000","last_peered":"2024-05-22T21:24:35.106289+0000","last_clean":"2024-05-22T21:24:35.106289+0000","last_became_active":"2024-05-22T21:24:34.108629+0000","last_became_peered":"2024-05-22T21:24:34.108629+0000","last_unstale":"2024-05-22T21:24:35.106289+0000","last_undegraded":"2024-05-22T21:24:35.106289+0000","last_fullsized":"2024-05-22T21:24:35.106289+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T08:22:41.425404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106544+0000","last_change":"2024-05-22T21:24:33.735095+0000","last_active":"2024-05-22T21:24:35.106544+0000","last_peered":"2024-05-22T21:24:35.106544+0000","last_clean":"2024-05-22T21:24:35.106544+0000","last_became_active":"2024-05-22T21:24:33.734799+0000","last_became_peered":"2024-05-22T21:24:33.734799+0000","last_unstale":"2024-05-22T21:24:35.106544+0000","last_undegraded":"2024-05-22T21:24:35.106544+0000","last_fullsized":"2024-05-22T21:24:35.106544+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:11:38.181969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106288+0000","last_change":"2024-05-22T21:24:34.115785+0000","last_active":"2024-05-22T21:24:35.106288+0000","last_peered":"2024-05-22T21:24:35.106288+0000","last_clean":"2024-05-22T21:24:35.106288+0000","last_became_active":"2024-05-22T21:24:34.115049+0000","last_became_peered":"2024-05-22T21:24:34.115049+0000","last_unstale":"2024-05-22T21:24:35.106288+0000","last_undegraded":"2024-05-22T21:24:35.106288+0000","last_fullsized":"2024-05-22T21:24:35.106288+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:35:47.933629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109971+0000","last_change":"2024-05-22T21:24:33.729321+0000","last_active":"2024-05-22T21:24:35.109971+0000","last_peered":"2024-05-22T21:24:35.109971+0000","last_clean":"2024-05-22T21:24:35.109971+0000","last_became_active":"2024-05-22T21:24:33.727866+0000","last_became_peered":"2024-05-22T21:24:33.727866+0000","last_unstale":"2024-05-22T21:24:35.109971+0000","last_undegraded":"2024-05-22T21:24:35.109971+0000","last_fullsized":"2024-05-22T21:24:35.109971+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:22:34.599259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109757+0000","last_change":"2024-05-22T21:24:34.111301+0000","last_active":"2024-05-22T21:24:35.109757+0000","last_peered":"2024-05-22T21:24:35.109757+0000","last_clean":"2024-05-22T21:24:35.109757+0000","last_became_active":"2024-05-22T21:24:34.110763+0000","last_became_peered":"2024-05-22T21:24:34.110763+0000","last_unstale":"2024-05-22T21:24:35.109757+0000","last_undegraded":"2024-05-22T21:24:35.109757+0000","last_fullsized":"2024-05-22T21:24:35.109757+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:12:38.421079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452284+0000","last_change":"2024-05-22T21:24:33.728901+0000","last_active":"2024-05-22T21:24:36.452284+0000","last_peered":"2024-05-22T21:24:36.452284+0000","last_clean":"2024-05-22T21:24:36.452284+0000","last_became_active":"2024-05-22T21:24:33.728050+0000","last_became_peered":"2024-05-22T21:24:33.728050+0000","last_unstale":"2024-05-22T21:24:36.452284+0000","last_undegraded":"2024-05-22T21:24:36.452284+0000","last_fullsized":"2024-05-22T21:24:36.452284+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T00:55:03.393477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.450467+0000","last_change":"2024-05-22T21:24:34.109817+0000","last_active":"2024-05-22T21:24:36.450467+0000","last_peered":"2024-05-22T21:24:36.450467+0000","last_clean":"2024-05-22T21:24:36.450467+0000","last_became_active":"2024-05-22T21:24:34.109611+0000","last_became_peered":"2024-05-22T21:24:34.109611+0000","last_unstale":"2024-05-22T21:24:36.450467+0000","last_undegraded":"2024-05-22T21:24:36.450467+0000","last_fullsized":"2024-05-22T21:24:36.450467+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:24:39.530150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452685+0000","last_change":"2024-05-22T21:24:33.735666+0000","last_active":"2024-05-22T21:24:36.452685+0000","last_peered":"2024-05-22T21:24:36.452685+0000","last_clean":"2024-05-22T21:24:36.452685+0000","last_became_active":"2024-05-22T21:24:33.735397+0000","last_became_peered":"2024-05-22T21:24:33.735397+0000","last_unstale":"2024-05-22T21:24:36.452685+0000","last_undegraded":"2024-05-22T21:24:36.452685+0000","last_fullsized":"2024-05-22T21:24:36.452685+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T04:34:12.778286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:36.452802+0000","last_change":"2024-05-22T21:24:34.118551+0000","last_active":"2024-05-22T21:24:36.452802+0000","last_peered":"2024-05-22T21:24:36.452802+0000","last_clean":"2024-05-22T21:24:36.452802+0000","last_became_active":"2024-05-22T21:24:34.118291+0000","last_became_peered":"2024-05-22T21:24:34.118291+0000","last_unstale":"2024-05-22T21:24:36.452802+0000","last_undegraded":"2024-05-22T21:24:36.452802+0000","last_fullsized":"2024-05-22T21:24:36.452802+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T05:10:58.574775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109426+0000","last_change":"2024-05-22T21:24:33.733829+0000","last_active":"2024-05-22T21:24:35.109426+0000","last_peered":"2024-05-22T21:24:35.109426+0000","last_clean":"2024-05-22T21:24:35.109426+0000","last_became_active":"2024-05-22T21:24:33.730850+0000","last_became_peered":"2024-05-22T21:24:33.730850+0000","last_unstale":"2024-05-22T21:24:35.109426+0000","last_undegraded":"2024-05-22T21:24:35.109426+0000","last_fullsized":"2024-05-22T21:24:35.109426+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T21:25:39.220280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.106182+0000","last_change":"2024-05-22T21:24:34.117690+0000","last_active":"2024-05-22T21:24:35.106182+0000","last_peered":"2024-05-22T21:24:35.106182+0000","last_clean":"2024-05-22T21:24:35.106182+0000","last_became_active":"2024-05-22T21:24:34.117478+0000","last_became_peered":"2024-05-22T21:24:34.117478+0000","last_unstale":"2024-05-22T21:24:35.106182+0000","last_undegraded":"2024-05-22T21:24:35.106182+0000","last_fullsized":"2024-05-22T21:24:35.106182+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:33.710923+0000","last_clean_scrub_stamp":"2024-05-22T21:24:33.710923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T03:02:21.712263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-05-22T21:24:35.109860+0000","last_change":"2024-05-22T21:24:33.729477+0000","last_active":"2024-05-22T21:24:35.109860+0000","last_peered":"2024-05-22T21:24:35.109860+0000","last_clean":"2024-05-22T21:24:35.109860+0000","last_became_active":"2024-05-22T21:24:33.728199+0000","last_became_peered":"2024-05-22T21:24:33.728199+0000","last_unstale":"2024-05-22T21:24:35.109860+0000","last_undegraded":"2024-05-22T21:24:35.109860+0000","last_fullsized":"2024-05-22T21:24:35.109860+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:24:32.704004+0000","last_clean_scrub_stamp":"2024-05-22T21:24:32.704004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T23:34:13.252170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":338,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-05-22T21:24:34.104117+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:24:34.104117+0000","last_peered":"2024-05-22T21:24:34.104117+0000","last_clean":"2024-05-22T21:24:34.104117+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:24:34.104117+0000","last_undegraded":"2024-05-22T21:24:34.104117+0000","last_fullsized":"2024-05-22T21:24:34.104117+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1622,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193689,"statfs":{"total":200000000,"available":198338166,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.36699999999999999},"max":{"1min":0.95099999999999996,"5min":1.022,"15min":2.0369999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72099999999999997,"15min":0.70199999999999996},"min":{"1min":0.52500000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97899999999999998,"5min":1.1759999999999999,"15min":1.889},"last":0.78500000000000003}]},{"osd":1,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72099999999999997,"15min":0.74199999999999999},"min":{"1min":0.48099999999999998,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.93300000000000005,"5min":1.0329999999999999,"15min":3.0920000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.73399999999999999},"min":{"1min":0.60199999999999998,"5min":0.45200000000000001,"15min":0.39800000000000002},"max":{"1min":0.94699999999999995,"5min":1.1100000000000001,"15min":3.1499999999999999},"last":0.60799999999999998}]},{"osd":2,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.501,"5min":0.433,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":1.002,"15min":1.923},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.40999999999999998},"max":{"1min":0.96699999999999997,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72399999999999998,"15min":0.70999999999999996},"min":{"1min":0.54900000000000004,"5min":0.44700000000000001,"15min":0.40100000000000002},"max":{"1min":0.88100000000000001,"5min":1.026,"15min":1.026},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.76200000000000001,"15min":0.77200000000000002},"min":{"1min":0.64700000000000002,"5min":0.501,"15min":0.44500000000000001},"max":{"1min":0.99399999999999999,"5min":1.0740000000000001,"15min":2.1389999999999998},"last":0.76100000000000001}]},{"osd":4,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.29399999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.96599999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.628,"15min":0.61899999999999999},"min":{"1min":0.504,"5min":0.40500000000000003,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.95999999999999996,"15min":1.018},"last":0.57499999999999996}]},{"osd":5,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.51700000000000002,"5min":0.39300000000000002,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0289999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67300000000000004,"15min":0.65200000000000002},"min":{"1min":0.46300000000000002,"5min":0.42099999999999999,"15min":0.33400000000000002},"max":{"1min":0.91200000000000003,"5min":0.99299999999999999,"15min":0.996},"last":0.66500000000000004}]},{"osd":6,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.50600000000000001,"5min":0.5,"15min":0.36799999999999999},"max":{"1min":0.92500000000000004,"5min":1.052,"15min":1.052},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69999999999999996,"15min":0.67000000000000004},"min":{"1min":0.60999999999999999,"5min":0.48699999999999999,"15min":0.41199999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.97699999999999998},"last":0.745}]}]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1037,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198937118,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69499999999999995,"15min":0.69399999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0649999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.51900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95899999999999996,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.77600000000000002}]},{"osd":1,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69399999999999995,"15min":0.69099999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89500000000000002,"5min":0.94299999999999995,"15min":1.107},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.70599999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":0.93500000000000005,"5min":1.0009999999999999,"15min":1.081},"last":0.71799999999999997}]},{"osd":2,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.442},"max":{"1min":0.93000000000000005,"5min":1.026,"15min":1.087},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75700000000000001,"15min":0.746},"min":{"1min":0.56499999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0089999999999999,"5min":1.419,"15min":1.419},"last":0.83099999999999996}]},{"osd":3,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.749,"15min":0.746},"min":{"1min":0.46600000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98599999999999999,"5min":1.28,"15min":1.415},"last":0.874},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.55400000000000005,"5min":0.48999999999999999,"15min":0.45400000000000001},"max":{"1min":0.878,"5min":0.93300000000000005,"15min":1.788},"last":0.69999999999999996}]},{"osd":4,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.374},"max":{"1min":0.89300000000000002,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.67500000000000004},"min":{"1min":0.45600000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.90900000000000003,"5min":1.1850000000000001,"15min":2.718},"last":0.90600000000000003}]},{"osd":5,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61499999999999999,"15min":0.67800000000000005},"min":{"1min":0.42399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84799999999999998,"5min":0.97499999999999998,"15min":8.5749999999999993},"last":0.752},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67900000000000005,"15min":0.69999999999999996},"min":{"1min":0.49299999999999999,"5min":0.44800000000000001,"15min":0.42099999999999999},"max":{"1min":0.92000000000000004,"5min":0.95799999999999996,"15min":1.1140000000000001},"last":0.85199999999999998}]},{"osd":7,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92800000000000005,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69699999999999995,"15min":0.72699999999999998},"min":{"1min":0.49099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85499999999999998,"5min":0.89700000000000002,"15min":1.8009999999999999},"last":0.79700000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194273,"statfs":{"total":200000000,"available":198936536,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:23:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67400000000000004,"15min":0.69199999999999995},"min":{"1min":0.504,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.0409999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69599999999999995,"15min":0.68500000000000005},"min":{"1min":0.54600000000000004,"5min":0.38400000000000001,"15min":0.378},"max":{"1min":0.85999999999999999,"5min":0.93700000000000006,"15min":1.0089999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Wed May 22 21:23:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.497,"5min":0.372,"15min":0.372},"max":{"1min":0.90900000000000003,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.57099999999999995,"5min":0.45500000000000002,"15min":0.44},"max":{"1min":0.91100000000000003,"5min":1.321,"15min":1.321},"last":0.748}]},{"osd":2,"last update":"Wed May 22 21:23:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71499999999999997,"15min":0.70799999999999996},"min":{"1min":0.51500000000000001,"5min":0.46300000000000002,"15min":0.33100000000000002},"max":{"1min":0.996,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73499999999999999,"15min":0.69899999999999995},"min":{"1min":0.58099999999999996,"5min":0.432,"15min":0.42499999999999999},"max":{"1min":0.93200000000000005,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed May 22 21:23:36 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72699999999999998,"15min":0.71599999999999997},"min":{"1min":0.58699999999999997,"5min":0.48799999999999999,"15min":0.47999999999999998},"max":{"1min":0.92700000000000005,"5min":1.004,"15min":1.1879999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.627,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.89100000000000001,"5min":1.0469999999999999,"15min":1.111},"last":0.72599999999999998}]},{"osd":4,"last update":"Wed May 22 21:23:36 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.39600000000000002,"15min":0.35099999999999998},"max":{"1min":0.86899999999999999,"5min":0.92400000000000004,"15min":1.272},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.49299999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86399999999999999,"5min":0.95499999999999996,"15min":1.038},"last":0.84999999999999998}]},{"osd":6,"last update":"Wed May 22 21:23:36 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.51300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.98899999999999999},"last":0.872},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65800000000000003,"15min":0.66100000000000003},"min":{"1min":0.45600000000000002,"5min":0.443,"15min":0.41099999999999998},"max":{"1min":0.84099999999999997,"5min":1.0069999999999999,"15min":1.2290000000000001},"last":0.94799999999999995}]},{"osd":7,"last update":"Wed May 22 21:23:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.59199999999999997,"5min":0.48999999999999999,"15min":0.47399999999999998},"max":{"1min":0.89300000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.755,"15min":0.77300000000000002},"min":{"1min":0.58099999999999996,"5min":0.47399999999999998,"15min":0.46200000000000002},"max":{"1min":1.0149999999999999,"5min":1.018,"15min":1.2909999999999999},"last":0.96799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1045,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194266,"statfs":{"total":200000000,"available":198929206,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71499999999999997,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30399999999999999},"max":{"1min":0.97299999999999998,"5min":0.98899999999999999,"15min":6.6539999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.52100000000000002,"5min":0.46000000000000002,"15min":0.436},"max":{"1min":0.872,"5min":0.96299999999999997,"15min":5.4370000000000003},"last":0.76300000000000001}]},{"osd":1,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.39200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91400000000000003,"5min":0.95799999999999996,"15min":1.7749999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69699999999999995,"15min":0.70899999999999996},"min":{"1min":0.48299999999999998,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":2.2210000000000001},"last":0.67600000000000005}]},{"osd":2,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72299999999999998,"15min":0.71199999999999997},"min":{"1min":0.50900000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":0.97299999999999998,"15min":2.4660000000000002},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.54400000000000004,"5min":0.45600000000000002,"15min":0.378},"max":{"1min":0.92900000000000005,"5min":1.01,"15min":2.2530000000000001},"last":0.72999999999999998}]},{"osd":3,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71299999999999997,"15min":0.70199999999999996},"min":{"1min":0.54200000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.998,"5min":1.042,"15min":1.857},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77100000000000002,"15min":0.78200000000000003},"min":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.47499999999999998},"max":{"1min":0.96599999999999997,"5min":1.012,"15min":1.897},"last":0.83299999999999996}]},{"osd":5,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.629},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.379},"max":{"1min":0.95199999999999996,"5min":1.0169999999999999,"15min":1.1850000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.35599999999999998},"max":{"1min":0.872,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.51200000000000001}]},{"osd":6,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85599999999999998,"5min":1.044,"15min":1.0609999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.44,"5min":0.44,"15min":0.434},"max":{"1min":0.996,"5min":1.095,"15min":1.1419999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.68500000000000005},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.40799999999999997},"max":{"1min":0.84799999999999998,"5min":1.1579999999999999,"15min":1.7390000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70999999999999996,"15min":0.746},"min":{"1min":0.505,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86499999999999999,"5min":1.069,"15min":1.4510000000000001},"last":0.626}]}]},{"osd":3,"up_from":13,"seq":55834574987,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1044,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194268,"statfs":{"total":200000000,"available":198930806,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42799999999999999,"15min":0.41999999999999998},"min":{"1min":0.312,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59399999999999997,"5min":0.73099999999999998,"15min":0.86799999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47199999999999998,"15min":0.438},"min":{"1min":0.33600000000000002,"5min":0.30399999999999999,"15min":0.28399999999999997},"max":{"1min":0.65300000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.434}]},{"osd":1,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48199999999999998,"15min":0.46400000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.26500000000000001},"max":{"1min":0.69799999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.52900000000000003},"min":{"1min":0.40699999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78700000000000003,"5min":0.80600000000000005,"15min":0.90000000000000002},"last":0.50800000000000001}]},{"osd":2,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55100000000000005,"15min":0.51600000000000001},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89500000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.50800000000000001},"min":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.33600000000000002},"max":{"1min":0.69599999999999995,"5min":0.82799999999999996,"15min":0.93400000000000005},"last":0.55400000000000005}]},{"osd":4,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":0.71999999999999997},"min":{"1min":0.55900000000000005,"5min":0.48299999999999998,"15min":0.39400000000000002},"max":{"1min":1.0389999999999999,"5min":1.1839999999999999,"15min":1.2050000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.75700000000000001,"15min":0.73599999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.435},"max":{"1min":1,"5min":1.093,"15min":1.5760000000000001},"last":0.79900000000000004}]},{"osd":5,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.45100000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.234},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.73899999999999999,"15min":0.72399999999999998},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.3049999999999999},"last":0.77900000000000003}]},{"osd":6,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73199999999999998,"15min":0.72699999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.44900000000000001},"max":{"1min":1.0149999999999999,"5min":1.0409999999999999,"15min":1.276},"last":1.016},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76900000000000002,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.53400000000000003,"15min":0.50600000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.2869999999999999},"last":0.88200000000000001}]},{"osd":7,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79000000000000004,"15min":0.77400000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.495},"max":{"1min":1.071,"5min":1.0880000000000001,"15min":1.2709999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.76200000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0169999999999999,"5min":1.133,"15min":1.2969999999999999},"last":1.093}]}]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198937050,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.434,"15min":0.42299999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.23599999999999999},"max":{"1min":0.59899999999999998,"5min":0.66500000000000004,"15min":2.3090000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.435,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.28199999999999997,"5min":0.26900000000000002,"15min":0.23400000000000001},"max":{"1min":0.64100000000000001,"5min":0.80400000000000005,"15min":2.532},"last":0.376}]},{"osd":1,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45600000000000002},"min":{"1min":0.30599999999999999,"5min":0.29899999999999999,"15min":0.29399999999999998},"max":{"1min":0.56899999999999995,"5min":0.68899999999999995,"15min":2.948},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.64800000000000002,"5min":0.71799999999999997,"15min":2.8159999999999998},"last":0.47099999999999997}]},{"osd":3,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.32700000000000001},"max":{"1min":0.62,"5min":0.83499999999999996,"15min":2.75},"last":0.495},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.5,"15min":0.48899999999999999},"min":{"1min":0.36099999999999999,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":3.0619999999999998},"last":0.436}]},{"osd":4,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78000000000000003,"15min":0.74199999999999999},"min":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.52700000000000002},"max":{"1min":1.022,"5min":1.103,"15min":2.7610000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.78100000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.375},"max":{"1min":0.97499999999999998,"5min":1.208,"15min":2.9279999999999999},"last":0.751}]},{"osd":5,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.753,"15min":0.73799999999999999},"min":{"1min":0.56799999999999995,"5min":0.51900000000000002,"15min":0.47299999999999998},"max":{"1min":0.89500000000000002,"5min":1.133,"15min":2.9790000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.45900000000000002},"max":{"1min":0.99099999999999999,"5min":1.0549999999999999,"15min":2.7069999999999999},"last":0.83699999999999997}]},{"osd":6,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.73699999999999999},"min":{"1min":0.54600000000000004,"5min":0.51300000000000001,"15min":0.495},"max":{"1min":0.90000000000000002,"5min":1.0649999999999999,"15min":2.8580000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.748,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.496,"15min":0.46500000000000002},"max":{"1min":1.048,"5min":1.147,"15min":2.6880000000000002},"last":0.67100000000000004}]},{"osd":7,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.79700000000000004,"15min":0.77000000000000002},"min":{"1min":0.67000000000000004,"5min":0.50700000000000001,"15min":0.438},"max":{"1min":1.004,"5min":1.2210000000000001,"15min":3.0099999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81799999999999995,"15min":0.80100000000000005},"min":{"1min":0.67000000000000004,"5min":0.57299999999999995,"15min":0.499},"max":{"1min":1.006,"5min":1.1339999999999999,"15min":2.827},"last":0.79100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1037,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198937106,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:23:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44800000000000001,"15min":0.434},"min":{"1min":0.254,"5min":0.216,"15min":0.216},"max":{"1min":0.59299999999999997,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46600000000000003,"15min":0.45100000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.81299999999999994,"5min":0.879,"15min":0.879},"last":0.53900000000000003}]},{"osd":2,"last update":"Wed May 22 21:23:37 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50900000000000001,"15min":0.497},"min":{"1min":0.32000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.73599999999999999,"5min":1.76,"15min":1.76},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48399999999999999,"15min":0.50900000000000001},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68000000000000005,"5min":0.83599999999999997,"15min":1.923},"last":0.51100000000000001}]},{"osd":3,"last update":"Wed May 22 21:23:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55000000000000004,"15min":0.55600000000000005},"min":{"1min":0.38500000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.755,"5min":0.82999999999999996,"15min":2.1030000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52600000000000002,"15min":0.502},"min":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65800000000000003,"5min":0.745,"15min":0.745},"last":0.68600000000000005}]},{"osd":4,"last update":"Wed May 22 21:23:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73499999999999999,"15min":0.748},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.45100000000000001},"max":{"1min":0.97699999999999998,"5min":1.0780000000000001,"15min":2.1360000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72799999999999998,"15min":0.70699999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.004,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Wed May 22 21:23:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.75700000000000001,"15min":0.748},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.45300000000000001},"max":{"1min":1.0549999999999999,"5min":1.819,"15min":1.819},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.55600000000000005,"5min":0.46300000000000002,"15min":0.40100000000000002},"max":{"1min":1.0469999999999999,"5min":1.1060000000000001,"15min":1.97},"last":0.86099999999999999}]},{"osd":6,"last update":"Wed May 22 21:23:37 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.47299999999999998},"max":{"1min":0.98699999999999999,"5min":1.1359999999999999,"15min":1.272},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.80100000000000005,"15min":0.77300000000000002},"min":{"1min":0.54500000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.097,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.78000000000000003}]},{"osd":7,"last update":"Wed May 22 21:23:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76500000000000001,"15min":0.77000000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.42099999999999999},"max":{"1min":0.995,"5min":1.1379999999999999,"15min":2.0310000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.56699999999999995,"5min":0.54900000000000004,"15min":0.52000000000000002},"max":{"1min":1.018,"5min":1.0660000000000001,"15min":2.0659999999999998},"last":0.84699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1613,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193698,"statfs":{"total":200000000,"available":198347391,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.43099999999999999},"min":{"1min":0.27000000000000002,"5min":0.24199999999999999,"15min":0.22800000000000001},"max":{"1min":0.63400000000000001,"5min":0.69999999999999996,"15min":0.88300000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.50800000000000001},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.33100000000000002},"max":{"1min":0.754,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51400000000000001}]},{"osd":2,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.48399999999999999},"min":{"1min":0.33100000000000002,"5min":0.32100000000000001,"15min":0.29399999999999998},"max":{"1min":0.70299999999999996,"5min":0.85899999999999999,"15min":0.93799999999999994},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.46999999999999997,"15min":0.44900000000000001},"min":{"1min":0.39400000000000002,"5min":0.28899999999999998,"15min":0.27900000000000003},"max":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.92300000000000004},"last":0.44700000000000001}]},{"osd":3,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47599999999999998,"15min":0.45400000000000001},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":1.0760000000000001},"last":0.38},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55700000000000005,"15min":0.53200000000000003},"min":{"1min":0.41699999999999998,"5min":0.38900000000000001,"15min":0.36899999999999999},"max":{"1min":0.79800000000000004,"5min":0.89700000000000002,"15min":0.98299999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.72799999999999998},"min":{"1min":0.53200000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0189999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.748},"min":{"1min":0.50600000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.008,"5min":1.2330000000000001,"15min":1.2709999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.72999999999999998},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":1.135,"15min":1.135},"last":0.505},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.54200000000000004,"5min":0.499,"15min":0.47299999999999998},"max":{"1min":0.95099999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.61599999999999999}]},{"osd":6,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81799999999999995,"15min":0.79300000000000004},"min":{"1min":0.61399999999999999,"5min":0.54400000000000004,"15min":0.51500000000000001},"max":{"1min":0.97299999999999998,"5min":1.175,"15min":1.2090000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80100000000000005,"15min":0.78300000000000003},"min":{"1min":0.60199999999999998,"5min":0.50800000000000001,"15min":0.502},"max":{"1min":0.99099999999999999,"5min":1.3149999999999999,"15min":1.5},"last":0.63300000000000001}]},{"osd":7,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78200000000000003,"15min":0.753},"min":{"1min":0.53200000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.034,"5min":1.214,"15min":1.214},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.64100000000000001,"5min":0.52800000000000002,"15min":0.497},"max":{"1min":0.98599999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.65300000000000002}]}]}],"pool_statfs":[]}} 2024-05-22T21:24:38.550 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-22T21:24:38.550 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-22T21:24:38.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.735+0000 7f0990800640 1 Processor -- start 2024-05-22T21:24:38.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.735+0000 7f0990800640 1 -- start start 2024-05-22T21:24:38.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.735+0000 7f0990800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f098815cfa0 0x7f098815e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:38.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.735+0000 7f0990800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f098815d380 0x7f0988069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:38.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.735+0000 7f0990800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09880b47a0 con 0x7f09880aefc0 2024-05-22T21:24:38.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.735+0000 7f0990800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098815d8b0 con 0x7f098815d380 2024-05-22T21:24:38.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.735+0000 7f0990800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09880b4a00 con 0x7f098815cfa0 2024-05-22T21:24:38.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.735+0000 7f098e800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f098815d380 0x7f0988069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:38.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.735+0000 7f098e800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f098815d380 0x7f0988069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59714/0 (socket says 172.21.15.161:59714) 2024-05-22T21:24:38.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.735+0000 7f098e800640 1 -- 172.21.15.161:0/1309634727 learned_addr learned my addr 172.21.15.161:0/1309634727 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:38.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.735+0000 7f098de00640 1 --2- 172.21.15.161:0/1309634727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f098815cfa0 0x7f098815e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:38.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.735+0000 7f098e800640 1 -- 172.21.15.161:0/1309634727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f098815cfa0 msgr2=0x7f098815e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:38.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.735+0000 7f098e800640 1 --2- 172.21.15.161:0/1309634727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f098815cfa0 0x7f098815e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.735+0000 7f098e800640 1 -- 172.21.15.161:0/1309634727 >> v1:172.21.15.179:6789/0 conn(0x7f09880aefc0 legacy=0x7f09880b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:38.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.735+0000 7f098e800640 1 -- 172.21.15.161:0/1309634727 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098815db50 con 0x7f098815d380 2024-05-22T21:24:38.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.739+0000 7f098e800640 1 --2- 172.21.15.161:0/1309634727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f098815d380 0x7f0988069ed0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f0984009e50 tx=0x7f098401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7bb63241fe971c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:38.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.739+0000 7f098ca00640 1 -- 172.21.15.161:0/1309634727 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09840a57e0 con 0x7f098815d380 2024-05-22T21:24:38.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.739+0000 7f098ca00640 1 -- 172.21.15.161:0/1309634727 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09840a5940 con 0x7f098815d380 2024-05-22T21:24:38.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.739+0000 7f098ca00640 1 -- 172.21.15.161:0/1309634727 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09840a5cd0 con 0x7f098815d380 2024-05-22T21:24:38.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.739+0000 7f0990800640 1 -- 172.21.15.161:0/1309634727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f098815d380 msgr2=0x7f0988069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:38.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.739+0000 7f0990800640 1 --2- 172.21.15.161:0/1309634727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f098815d380 0x7f0988069ed0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f0984009e50 tx=0x7f098401ca00 comp rx=0 tx=0).stop 2024-05-22T21:24:38.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.739+0000 7f0990800640 1 -- 172.21.15.161:0/1309634727 shutdown_connections 2024-05-22T21:24:38.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.739+0000 7f0990800640 1 --2- 172.21.15.161:0/1309634727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f098815d380 0x7f0988069ed0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.739+0000 7f0990800640 1 --2- 172.21.15.161:0/1309634727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f098815cfa0 0x7f098815e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.739+0000 7f0990800640 1 -- 172.21.15.161:0/1309634727 >> 172.21.15.161:0/1309634727 conn(0x7f09880b1bd0 msgr2=0x7f09880b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:38.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.739+0000 7f0990800640 1 -- 172.21.15.161:0/1309634727 shutdown_connections 2024-05-22T21:24:38.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.739+0000 7f0990800640 1 -- 172.21.15.161:0/1309634727 wait complete. 2024-05-22T21:24:38.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f0990800640 1 Processor -- start 2024-05-22T21:24:38.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f0990800640 1 -- start start 2024-05-22T21:24:38.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f0990800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f09880aefc0 0x7f098814da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:38.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f0990800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f098815cfa0 0x7f0988154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:38.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f0990800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f098815d380 0x7f0988154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:38.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f0990800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09881582c0 con 0x7f098815d380 2024-05-22T21:24:38.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f0990800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0988158fa0 con 0x7f098815cfa0 2024-05-22T21:24:38.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f0990800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0988159240 con 0x7f09880aefc0 2024-05-22T21:24:38.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f098e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f098815d380 0x7f0988154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:38.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f098e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f098815d380 0x7f0988154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59722/0 (socket says 172.21.15.161:59722) 2024-05-22T21:24:38.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f098e800640 1 -- 172.21.15.161:0/4132893940 learned_addr learned my addr 172.21.15.161:0/4132893940 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:38.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f098d400640 1 --2- 172.21.15.161:0/4132893940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f098815cfa0 0x7f0988154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:38.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f098de00640 1 --2- 172.21.15.161:0/4132893940 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f09880aefc0 0x7f098814da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:38.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f098e800640 1 -- 172.21.15.161:0/4132893940 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f09880aefc0 msgr2=0x7f098814da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:38.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f098e800640 1 --2- 172.21.15.161:0/4132893940 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f09880aefc0 0x7f098814da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f098e800640 1 -- 172.21.15.161:0/4132893940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f098815cfa0 msgr2=0x7f0988154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:38.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f098e800640 1 --2- 172.21.15.161:0/4132893940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f098815cfa0 0x7f0988154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f098e800640 1 -- 172.21.15.161:0/4132893940 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0984009b30 con 0x7f098815d380 2024-05-22T21:24:38.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f098e800640 1 --2- 172.21.15.161:0/4132893940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f098815d380 0x7f0988154c70 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f098401a630 tx=0x7f09840197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:38.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f0982000640 1 -- 172.21.15.161:0/4132893940 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09840b8670 con 0x7f098815d380 2024-05-22T21:24:38.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f0982000640 1 -- 172.21.15.161:0/4132893940 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09840b87d0 con 0x7f098815d380 2024-05-22T21:24:38.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f0982000640 1 -- 172.21.15.161:0/4132893940 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09840b8b40 con 0x7f098815d380 2024-05-22T21:24:38.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f0990800640 1 -- 172.21.15.161:0/4132893940 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0988159520 con 0x7f098815d380 2024-05-22T21:24:38.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.743+0000 7f0990800640 1 -- 172.21.15.161:0/4132893940 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0988159ae0 con 0x7f098815d380 2024-05-22T21:24:38.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.747+0000 7f0990800640 1 -- 172.21.15.161:0/4132893940 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0988067220 con 0x7f098815d380 2024-05-22T21:24:38.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.747+0000 7f0982000640 1 -- 172.21.15.161:0/4132893940 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f09840b8ca0 con 0x7f098815d380 2024-05-22T21:24:38.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.747+0000 7f0982000640 1 --2- 172.21.15.161:0/4132893940 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f096c066690 0x7f096c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:38.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.747+0000 7f0982000640 1 -- 172.21.15.161:0/4132893940 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f098411f740 con 0x7f098815d380 2024-05-22T21:24:38.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.747+0000 7f098de00640 1 --2- 172.21.15.161:0/4132893940 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f096c066690 0x7f096c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:38.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.747+0000 7f098de00640 1 --2- 172.21.15.161:0/4132893940 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f096c066690 0x7f096c068b50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f097c00b9d0 tx=0x7f097c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:38.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.751+0000 7f0982000640 1 -- 172.21.15.161:0/4132893940 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f09840ecbc0 con 0x7f098815d380 2024-05-22T21:24:38.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.959+0000 7f0990800640 1 -- 172.21.15.161:0/4132893940 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f09880af920 con 0x7f098815d380 2024-05-22T21:24:38.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.963+0000 7f0982000640 1 -- 172.21.15.161:0/4132893940 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+0 (secure 0 0 0) 0x7f09840f01e0 con 0x7f098815d380 2024-05-22T21:24:38.962 INFO:teuthology.orchestra.run.smithi161.stderr:updated caps for client.0 2024-05-22T21:24:38.964 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.963+0000 7f0990800640 1 -- 172.21.15.161:0/4132893940 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f096c066690 msgr2=0x7f096c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:38.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.963+0000 7f0990800640 1 --2- 172.21.15.161:0/4132893940 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f096c066690 0x7f096c068b50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f097c00b9d0 tx=0x7f097c008040 comp rx=0 tx=0).stop 2024-05-22T21:24:38.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.963+0000 7f0990800640 1 -- 172.21.15.161:0/4132893940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f098815d380 msgr2=0x7f0988154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:38.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.963+0000 7f0990800640 1 --2- 172.21.15.161:0/4132893940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f098815d380 0x7f0988154c70 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f098401a630 tx=0x7f09840197a0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.963+0000 7f0990800640 1 -- 172.21.15.161:0/4132893940 shutdown_connections 2024-05-22T21:24:38.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.963+0000 7f0990800640 1 --2- 172.21.15.161:0/4132893940 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f096c066690 0x7f096c068b50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.963+0000 7f0990800640 1 --2- 172.21.15.161:0/4132893940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f098815d380 0x7f0988154c70 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.963+0000 7f0990800640 1 --2- 172.21.15.161:0/4132893940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f098815cfa0 0x7f0988154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.963+0000 7f0990800640 1 --2- 172.21.15.161:0/4132893940 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f09880aefc0 0x7f098814da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:38.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.963+0000 7f0990800640 1 -- 172.21.15.161:0/4132893940 >> 172.21.15.161:0/4132893940 conn(0x7f09880b1bd0 msgr2=0x7f0988070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:38.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.963+0000 7f0990800640 1 -- 172.21.15.161:0/4132893940 shutdown_connections 2024-05-22T21:24:38.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:38.963+0000 7f0990800640 1 -- 172.21.15.161:0/4132893940 wait complete. 2024-05-22T21:24:38.983 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:24:39.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.119+0000 7f7c20c00640 1 Processor -- start 2024-05-22T21:24:39.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.119+0000 7f7c20c00640 1 -- start start 2024-05-22T21:24:39.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.119+0000 7f7c20c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7c1c0b4420 0x7f7c1c06aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.119+0000 7f7c20c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7c1c06b420 0x7f7c1c06c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.119+0000 7f7c20c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c1c15db70 con 0x7f7c1c06c8f0 2024-05-22T21:24:39.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.119+0000 7f7c20c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c1c06b8b0 con 0x7f7c1c0b4420 2024-05-22T21:24:39.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.119+0000 7f7c20c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c1c0ad610 con 0x7f7c1c06b420 2024-05-22T21:24:39.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.119+0000 7f7c19e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7c1c0b4420 0x7f7c1c06aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.119+0000 7f7c19e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7c1c0b4420 0x7f7c1c06aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59728/0 (socket says 172.21.15.161:59728) 2024-05-22T21:24:39.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.119+0000 7f7c19e00640 1 -- 172.21.15.161:0/4281523497 learned_addr learned my addr 172.21.15.161:0/4281523497 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:39.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.119+0000 7f7c11400640 1 --2- 172.21.15.161:0/4281523497 >> v2:172.21.15.179:3300/0 conn(0x7f7c1c06b420 0x7f7c1c06c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.119+0000 7f7c19e00640 1 -- 172.21.15.161:0/4281523497 >> v2:172.21.15.179:3300/0 conn(0x7f7c1c06b420 msgr2=0x7f7c1c06c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:39.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.119+0000 7f7c19e00640 1 --2- 172.21.15.161:0/4281523497 >> v2:172.21.15.179:3300/0 conn(0x7f7c1c06b420 0x7f7c1c06c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.119+0000 7f7c19e00640 1 -- 172.21.15.161:0/4281523497 >> v1:172.21.15.161:6789/0 conn(0x7f7c1c06c8f0 legacy=0x7f7c1c160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:39.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.119+0000 7f7c19e00640 1 -- 172.21.15.161:0/4281523497 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c0b26f0 con 0x7f7c1c0b4420 2024-05-22T21:24:39.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.123+0000 7f7c19e00640 1 --2- 172.21.15.161:0/4281523497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c1c0b4420 0x7f7c1c06aee0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0661f0 tx=0x7f7c0c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3e4a3f4003663c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:39.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.123+0000 7f7c19400640 1 -- 172.21.15.161:0/4281523497 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c0a57f0 con 0x7f7c1c0b4420 2024-05-22T21:24:39.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.123+0000 7f7c19400640 1 -- 172.21.15.161:0/4281523497 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0c0a5950 con 0x7f7c1c0b4420 2024-05-22T21:24:39.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.123+0000 7f7c19400640 1 -- 172.21.15.161:0/4281523497 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c0a5ce0 con 0x7f7c1c0b4420 2024-05-22T21:24:39.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.123+0000 7f7c20c00640 1 -- 172.21.15.161:0/4281523497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c1c0b4420 msgr2=0x7f7c1c06aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:39.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.123+0000 7f7c20c00640 1 --2- 172.21.15.161:0/4281523497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c1c0b4420 0x7f7c1c06aee0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0661f0 tx=0x7f7c0c01c9f0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.123+0000 7f7c20c00640 1 -- 172.21.15.161:0/4281523497 shutdown_connections 2024-05-22T21:24:39.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.123+0000 7f7c20c00640 1 --2- 172.21.15.161:0/4281523497 >> v2:172.21.15.179:3300/0 conn(0x7f7c1c06b420 0x7f7c1c06c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.123+0000 7f7c20c00640 1 --2- 172.21.15.161:0/4281523497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c1c0b4420 0x7f7c1c06aee0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.123+0000 7f7c20c00640 1 -- 172.21.15.161:0/4281523497 >> 172.21.15.161:0/4281523497 conn(0x7f7c1c070630 msgr2=0x7f7c1c070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:39.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.123+0000 7f7c20c00640 1 -- 172.21.15.161:0/4281523497 shutdown_connections 2024-05-22T21:24:39.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.123+0000 7f7c20c00640 1 -- 172.21.15.161:0/4281523497 wait complete. 2024-05-22T21:24:39.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c20c00640 1 Processor -- start 2024-05-22T21:24:39.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c20c00640 1 -- start start 2024-05-22T21:24:39.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c20c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7c1c06b420 0x7f7c1c14be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c20c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c1c06c8f0 0x7f7c1c154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c20c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c1c0b4420 0x7f7c1c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c20c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c158d70 con 0x7f7c1c06c8f0 2024-05-22T21:24:39.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c20c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c154660 con 0x7f7c1c0b4420 2024-05-22T21:24:39.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c20c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c0cc910 con 0x7f7c1c06b420 2024-05-22T21:24:39.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c11400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c1c06c8f0 0x7f7c1c154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c11400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c1c06c8f0 0x7f7c1c154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59740/0 (socket says 172.21.15.161:59740) 2024-05-22T21:24:39.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c11400640 1 -- 172.21.15.161:0/2891812703 learned_addr learned my addr 172.21.15.161:0/2891812703 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:39.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c1a800640 1 --2- 172.21.15.161:0/2891812703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c1c0b4420 0x7f7c1c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c19e00640 1 --2- 172.21.15.161:0/2891812703 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7c1c06b420 0x7f7c1c14be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c11400640 1 -- 172.21.15.161:0/2891812703 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7c1c06b420 msgr2=0x7f7c1c14be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:39.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c11400640 1 --2- 172.21.15.161:0/2891812703 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7c1c06b420 0x7f7c1c14be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c11400640 1 -- 172.21.15.161:0/2891812703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c1c0b4420 msgr2=0x7f7c1c0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:39.129 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c11400640 1 --2- 172.21.15.161:0/2891812703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c1c0b4420 0x7f7c1c0ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c11400640 1 -- 172.21.15.161:0/2891812703 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0c009b30 con 0x7f7c1c06c8f0 2024-05-22T21:24:39.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c19e00640 1 --2- 172.21.15.161:0/2891812703 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7c1c06b420 0x7f7c1c14be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:39.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c11400640 1 --2- 172.21.15.161:0/2891812703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c1c06c8f0 0x7f7c1c154de0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400d5f0 tx=0x7f7c0400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:39.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c12a00640 1 -- 172.21.15.161:0/2891812703 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c040125f0 con 0x7f7c1c06c8f0 2024-05-22T21:24:39.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c12a00640 1 -- 172.21.15.161:0/2891812703 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c04012750 con 0x7f7c1c06c8f0 2024-05-22T21:24:39.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c12a00640 1 -- 172.21.15.161:0/2891812703 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c04012ae0 con 0x7f7c1c06c8f0 2024-05-22T21:24:39.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c20c00640 1 -- 172.21.15.161:0/2891812703 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1c0ccbf0 con 0x7f7c1c06c8f0 2024-05-22T21:24:39.130 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.127+0000 7f7c20c00640 1 -- 172.21.15.161:0/2891812703 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1c154900 con 0x7f7c1c06c8f0 2024-05-22T21:24:39.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.131+0000 7f7c20c00640 1 -- 172.21.15.161:0/2891812703 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c1c067220 con 0x7f7c1c06c8f0 2024-05-22T21:24:39.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.131+0000 7f7c12a00640 1 -- 172.21.15.161:0/2891812703 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7c04012c40 con 0x7f7c1c06c8f0 2024-05-22T21:24:39.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.131+0000 7f7c12a00640 1 --2- 172.21.15.161:0/2891812703 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7bf8066690 0x7f7bf8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.131+0000 7f7c12a00640 1 -- 172.21.15.161:0/2891812703 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c040845b0 con 0x7f7c1c06c8f0 2024-05-22T21:24:39.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.131+0000 7f7c19e00640 1 --2- 172.21.15.161:0/2891812703 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7bf8066690 0x7f7bf8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.131+0000 7f7c19e00640 1 --2- 172.21.15.161:0/2891812703 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7bf8066690 0x7f7bf8068b50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0ae000 tx=0x7f7c0c0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.135+0000 7f7c12a00640 1 -- 172.21.15.161:0/2891812703 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7c04051910 con 0x7f7c1c06c8f0 2024-05-22T21:24:39.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.319+0000 7f7c20c00640 1 -- 172.21.15.161:0/2891812703 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c1c06d670 con 0x7f7c1c06c8f0 2024-05-22T21:24:39.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.319+0000 7f7c12a00640 1 -- 172.21.15.161:0/2891812703 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4744 (secure 0 0 0) 0x7f7c04082e60 con 0x7f7c1c06c8f0 2024-05-22T21:24:39.321 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:39.322 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":86,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3304182060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3304182060},{"type":"v1","addr":"172.21.15.179:6835","nonce":3304182060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196309272},{"type":"v1","addr":"172.21.15.161:6836","nonce":196309272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/396408969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":396408969},{"type":"v1","addr":"172.21.15.179:6834","nonce":396408969}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:24:34.096673+0000","modified":"2024-05-22T21:24:37.112845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8784},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8784":{"gid":8784,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2172511349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2172511349},{"type":"v1","addr":"172.21.15.161:6837","nonce":2172511349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-05-22T21:24:39.322 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 86 2024-05-22T21:24:39.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.323+0000 7f7c20c00640 1 -- 172.21.15.161:0/2891812703 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7bf8066690 msgr2=0x7f7bf8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:39.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.323+0000 7f7c20c00640 1 --2- 172.21.15.161:0/2891812703 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7bf8066690 0x7f7bf8068b50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0ae000 tx=0x7f7c0c0a7070 comp rx=0 tx=0).stop 2024-05-22T21:24:39.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.323+0000 7f7c20c00640 1 -- 172.21.15.161:0/2891812703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c1c06c8f0 msgr2=0x7f7c1c154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:39.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.323+0000 7f7c20c00640 1 --2- 172.21.15.161:0/2891812703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c1c06c8f0 0x7f7c1c154de0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400d5f0 tx=0x7f7c0400dab0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.323+0000 7f7c20c00640 1 -- 172.21.15.161:0/2891812703 shutdown_connections 2024-05-22T21:24:39.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.323+0000 7f7c20c00640 1 --2- 172.21.15.161:0/2891812703 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f7bf8066690 0x7f7bf8068b50 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.323+0000 7f7c20c00640 1 --2- 172.21.15.161:0/2891812703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c1c0b4420 0x7f7c1c0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.323+0000 7f7c20c00640 1 --2- 172.21.15.161:0/2891812703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c1c06c8f0 0x7f7c1c154de0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.323+0000 7f7c20c00640 1 --2- 172.21.15.161:0/2891812703 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7c1c06b420 0x7f7c1c14be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.323+0000 7f7c20c00640 1 -- 172.21.15.161:0/2891812703 >> 172.21.15.161:0/2891812703 conn(0x7f7c1c070630 msgr2=0x7f7c1c06ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:39.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.323+0000 7f7c20c00640 1 -- 172.21.15.161:0/2891812703 shutdown_connections 2024-05-22T21:24:39.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.323+0000 7f7c20c00640 1 -- 172.21.15.161:0/2891812703 wait complete. 2024-05-22T21:24:39.704 INFO:tasks.ceph.mds.c.smithi161.stderr:2024-05-22T21:24:39.707+0000 7f9728400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T21:24:39.746 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-22T21:24:34.096673+0000', 'modified': '2024-05-22T21:24:37.112845+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8784}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8784': {'gid': 8784, 'name': 'c', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6837/2172511349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6835', 'nonce': 2172511349}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 2172511349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-22T21:24:39.747 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-22T21:24:39.747 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-22T21:24:39.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.883+0000 7f8438a00640 1 Processor -- start 2024-05-22T21:24:39.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.883+0000 7f8438a00640 1 -- start start 2024-05-22T21:24:39.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.887+0000 7f8438a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8434143c40 0x7f843415b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.887+0000 7f8438a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f843406c9b0 0x7f84340b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.887+0000 7f8438a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843415efb0 con 0x7f84341444a0 2024-05-22T21:24:39.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.887+0000 7f8438a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843415f290 con 0x7f843406c9b0 2024-05-22T21:24:39.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.887+0000 7f8438a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843415f570 con 0x7f8434143c40 2024-05-22T21:24:39.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.887+0000 7f8432800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f843406c9b0 0x7f84340b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.887+0000 7f8432800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f843406c9b0 0x7f84340b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59748/0 (socket says 172.21.15.161:59748) 2024-05-22T21:24:39.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.887+0000 7f8432800640 1 -- 172.21.15.161:0/658733546 learned_addr learned my addr 172.21.15.161:0/658733546 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:39.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.887+0000 7f8431e00640 1 --2- 172.21.15.161:0/658733546 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8434143c40 0x7f843415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.887+0000 7f8432800640 1 -- 172.21.15.161:0/658733546 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8434143c40 msgr2=0x7f843415b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:39.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.887+0000 7f8432800640 1 --2- 172.21.15.161:0/658733546 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8434143c40 0x7f843415b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.887+0000 7f8432800640 1 -- 172.21.15.161:0/658733546 >> v1:172.21.15.179:6789/0 conn(0x7f84341444a0 legacy=0x7f843406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:39.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.887+0000 7f8432800640 1 -- 172.21.15.161:0/658733546 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843415f850 con 0x7f843406c9b0 2024-05-22T21:24:39.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.887+0000 7f8431e00640 1 --2- 172.21.15.161:0/658733546 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8434143c40 0x7f843415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:39.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8432800640 1 --2- 172.21.15.161:0/658733546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843406c9b0 0x7f84340b2160 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f842c009870 tx=0x7f842c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aeedb06432169b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:39.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8430a00640 1 -- 172.21.15.161:0/658733546 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c0ab640 con 0x7f843406c9b0 2024-05-22T21:24:39.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8430a00640 1 -- 172.21.15.161:0/658733546 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c0a7070 con 0x7f843406c9b0 2024-05-22T21:24:39.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8430a00640 1 -- 172.21.15.161:0/658733546 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c0ab9c0 con 0x7f843406c9b0 2024-05-22T21:24:39.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8438a00640 1 -- 172.21.15.161:0/658733546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843406c9b0 msgr2=0x7f84340b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:39.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8438a00640 1 --2- 172.21.15.161:0/658733546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843406c9b0 0x7f84340b2160 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f842c009870 tx=0x7f842c01c520 comp rx=0 tx=0).stop 2024-05-22T21:24:39.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8438a00640 1 -- 172.21.15.161:0/658733546 shutdown_connections 2024-05-22T21:24:39.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8438a00640 1 --2- 172.21.15.161:0/658733546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843406c9b0 0x7f84340b2160 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8438a00640 1 --2- 172.21.15.161:0/658733546 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8434143c40 0x7f843415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8438a00640 1 -- 172.21.15.161:0/658733546 >> 172.21.15.161:0/658733546 conn(0x7f843406a2d0 msgr2=0x7f843406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:39.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8438a00640 1 -- 172.21.15.161:0/658733546 shutdown_connections 2024-05-22T21:24:39.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8438a00640 1 -- 172.21.15.161:0/658733546 wait complete. 2024-05-22T21:24:39.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8438a00640 1 Processor -- start 2024-05-22T21:24:39.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8438a00640 1 -- start start 2024-05-22T21:24:39.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8438a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843406c9b0 0x7f843413c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8438a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8434143c40 0x7f84341371c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8438a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f84341444a0 0x7f843415daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8438a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8434137700 con 0x7f843406c9b0 2024-05-22T21:24:39.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8438a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8434137910 con 0x7f8434143c40 2024-05-22T21:24:39.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.891+0000 7f8438a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8434137bf0 con 0x7f84341444a0 2024-05-22T21:24:39.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f8431e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843406c9b0 0x7f843413c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f8431e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843406c9b0 0x7f843413c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59762/0 (socket says 172.21.15.161:59762) 2024-05-22T21:24:39.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f8431e00640 1 -- 172.21.15.161:0/2881250392 learned_addr learned my addr 172.21.15.161:0/2881250392 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:39.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f8432800640 1 --2- 172.21.15.161:0/2881250392 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f84341444a0 0x7f843415daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f8431400640 1 --2- 172.21.15.161:0/2881250392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8434143c40 0x7f84341371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f8431e00640 1 -- 172.21.15.161:0/2881250392 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f84341444a0 msgr2=0x7f843415daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:39.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f8431e00640 1 --2- 172.21.15.161:0/2881250392 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f84341444a0 0x7f843415daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f8431e00640 1 -- 172.21.15.161:0/2881250392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8434143c40 msgr2=0x7f84341371c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:39.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f8431e00640 1 --2- 172.21.15.161:0/2881250392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8434143c40 0x7f84341371c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f8431e00640 1 -- 172.21.15.161:0/2881250392 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c0094d0 con 0x7f843406c9b0 2024-05-22T21:24:39.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f8431400640 1 --2- 172.21.15.161:0/2881250392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8434143c40 0x7f84341371c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:39.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f8431e00640 1 --2- 172.21.15.161:0/2881250392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843406c9b0 0x7f843413c4e0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f842400cf60 tx=0x7f842400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:39.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f842a000640 1 -- 172.21.15.161:0/2881250392 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842400d840 con 0x7f843406c9b0 2024-05-22T21:24:39.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f842a000640 1 -- 172.21.15.161:0/2881250392 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842400d9a0 con 0x7f843406c9b0 2024-05-22T21:24:39.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f8438a00640 1 -- 172.21.15.161:0/2881250392 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8434137ed0 con 0x7f843406c9b0 2024-05-22T21:24:39.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f842a000640 1 -- 172.21.15.161:0/2881250392 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842400dd30 con 0x7f843406c9b0 2024-05-22T21:24:39.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f8438a00640 1 -- 172.21.15.161:0/2881250392 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8434138490 con 0x7f843406c9b0 2024-05-22T21:24:39.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f842a000640 1 -- 172.21.15.161:0/2881250392 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8424015440 con 0x7f843406c9b0 2024-05-22T21:24:39.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f842a000640 1 --2- 172.21.15.161:0/2881250392 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84100666c0 0x7f8410068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.895+0000 7f842a000640 1 -- 172.21.15.161:0/2881250392 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8424084ba0 con 0x7f843406c9b0 2024-05-22T21:24:39.897 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f8431400640 1 --2- 172.21.15.161:0/2881250392 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84100666c0 0x7f8410068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 Processor -- start 2024-05-22T21:24:39.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 -- start start 2024-05-22T21:24:39.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f8431400640 1 --2- 172.21.15.161:0/2881250392 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84100666c0 0x7f8410068b80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f843413a3f0 tx=0x7f841c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:39.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da84349c0 0x559da8434da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da84352e0 0x559da83ea980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da83eaec0 0x559da83eb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da82b2e80 con 0x559da83eaec0 2024-05-22T21:24:39.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da8240af0 con 0x559da84352e0 2024-05-22T21:24:39.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x559da823e0c0 con 0x559da84349c0 2024-05-22T21:24:39.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f8432800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da83eaec0 0x559da83eb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f8432800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da83eaec0 0x559da83eb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59764/0 (socket says 172.21.15.161:59764) 2024-05-22T21:24:39.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f8432800640 1 -- 172.21.15.161:0/4036354504 learned_addr learned my addr 172.21.15.161:0/4036354504 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:39.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f8431e00640 1 --2- 172.21.15.161:0/4036354504 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da84349c0 0x559da8434da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f8431400640 1 --2- 172.21.15.161:0/4036354504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da84352e0 0x559da83ea980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f8432800640 1 -- 172.21.15.161:0/4036354504 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da84349c0 msgr2=0x559da8434da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:39.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f8432800640 1 --2- 172.21.15.161:0/4036354504 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da84349c0 0x559da8434da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f8432800640 1 -- 172.21.15.161:0/4036354504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da84352e0 msgr2=0x559da83ea980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:39.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f8432800640 1 --2- 172.21.15.161:0/4036354504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da84352e0 0x559da83ea980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f8432800640 1 -- 172.21.15.161:0/4036354504 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559da7f35890 con 0x559da83eaec0 2024-05-22T21:24:39.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f8432800640 1 --2- 172.21.15.161:0/4036354504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da83eaec0 0x559da83eb2a0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f842c0ad040 tx=0x7f842c0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f388170389213d73 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:39.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f8422a00640 1 -- 172.21.15.161:0/4036354504 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c0ab930 con 0x559da83eaec0 2024-05-22T21:24:39.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f8422a00640 1 -- 172.21.15.161:0/4036354504 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c0aba90 con 0x559da83eaec0 2024-05-22T21:24:39.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f8422a00640 1 -- 172.21.15.161:0/4036354504 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c0abe20 con 0x559da83eaec0 2024-05-22T21:24:39.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 -- 172.21.15.161:0/4036354504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da83eaec0 msgr2=0x559da83eb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:39.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 --2- 172.21.15.161:0/4036354504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da83eaec0 0x559da83eb2a0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f842c0ad040 tx=0x7f842c0a7000 comp rx=0 tx=0).stop 2024-05-22T21:24:39.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 -- 172.21.15.161:0/4036354504 shutdown_connections 2024-05-22T21:24:39.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 --2- 172.21.15.161:0/4036354504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da83eaec0 0x559da83eb2a0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 --2- 172.21.15.161:0/4036354504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da84352e0 0x559da83ea980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 --2- 172.21.15.161:0/4036354504 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da84349c0 0x559da8434da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 -- 172.21.15.161:0/4036354504 >> 172.21.15.161:0/4036354504 conn(0x559da83f1440 msgr2=0x559da83e9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:39.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 -- 172.21.15.161:0/4036354504 shutdown_connections 2024-05-22T21:24:39.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 -- 172.21.15.161:0/4036354504 wait complete. 2024-05-22T21:24:39.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.899+0000 7f843be5c000 1 -- start start 2024-05-22T21:24:39.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 1 Processor -- start 2024-05-22T21:24:39.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da83eaec0 0x559da83efd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da84349c0 0x559da8433e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da84352e0 0x559da8434370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da82399b0 con 0x559da83eaec0 2024-05-22T21:24:39.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da81d8c00 con 0x559da84349c0 2024-05-22T21:24:39.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x559da81f7070 con 0x559da84352e0 2024-05-22T21:24:39.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 10 client.0 _ll_register_callbacks cb 0x559da83d6ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:24:39.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8431e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da83eaec0 0x559da83efd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8431e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da83eaec0 0x559da83efd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59770/0 (socket says 172.21.15.161:59770) 2024-05-22T21:24:39.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8431e00640 1 -- 172.21.15.161:0/2623103114 learned_addr learned my addr 172.21.15.161:0/2623103114 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:39.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8431400640 1 --2- 172.21.15.161:0/2623103114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da84349c0 0x559da8433e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8432800640 1 --2- 172.21.15.161:0/2623103114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da84352e0 0x559da8434370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8431e00640 1 -- 172.21.15.161:0/2623103114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da84352e0 msgr2=0x559da8434370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:39.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8431e00640 1 --2- 172.21.15.161:0/2623103114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da84352e0 0x559da8434370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8431e00640 1 -- 172.21.15.161:0/2623103114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da84349c0 msgr2=0x559da8433e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:39.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8431e00640 1 --2- 172.21.15.161:0/2623103114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da84349c0 0x559da8433e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8431e00640 1 -- 172.21.15.161:0/2623103114 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843415f850 con 0x559da83eaec0 2024-05-22T21:24:39.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8431e00640 1 --2- 172.21.15.161:0/2623103114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da83eaec0 0x559da83efd60 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f842400ac60 tx=0x7f8424055ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d73d3d668e2b16d server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:39.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8417400640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:24:39.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8417400640 1 -- 172.21.15.161:0/2623103114 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84240552d0 con 0x559da83eaec0 2024-05-22T21:24:39.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8417400640 1 -- 172.21.15.161:0/2623103114 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f842405df20 con 0x559da83eaec0 2024-05-22T21:24:39.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 1 -- 172.21.15.161:0/2623103114 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559da81f65f0 con 0x559da83eaec0 2024-05-22T21:24:39.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8417400640 1 -- 172.21.15.161:0/2623103114 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f842405da90 con 0x559da83eaec0 2024-05-22T21:24:39.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 10 client.8907 fetch_fsmap learned FSMap version 86 2024-05-22T21:24:39.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 1 -- 172.21.15.161:0/2623103114 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x559da81feab0 con 0x559da83eaec0 2024-05-22T21:24:39.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8417400640 1 -- 172.21.15.161:0/2623103114 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8424071080 con 0x559da83eaec0 2024-05-22T21:24:39.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8417400640 1 -- 172.21.15.161:0/2623103114 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 86) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8424005d20 con 0x559da83eaec0 2024-05-22T21:24:39.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 10 client.8907 fetch_fsmap finished waiting for FSMap version 86 2024-05-22T21:24:39.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 10 client.8907 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-22T21:24:39.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 20 client.8907 populate_metadata read hostname 'smithi161' 2024-05-22T21:24:39.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 1 --2- 172.21.15.161:0/2623103114 >> [v2:172.21.15.161:6835/2172511349,v1:172.21.15.161:6837/2172511349] conn(0x559da8436100 0x559da8446530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 4 client.8907 mds_command: new command op to 8784 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-22T21:24:39.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f843be5c000 1 -- 172.21.15.161:0/2623103114 --> [v2:172.21.15.161:6835/2172511349,v1:172.21.15.161:6837/2172511349] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559da8236790 con 0x559da8436100 2024-05-22T21:24:39.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8431400640 1 --2- 172.21.15.161:0/2623103114 >> [v2:172.21.15.161:6835/2172511349,v1:172.21.15.161:6837/2172511349] conn(0x559da8436100 0x559da8446530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8431400640 1 --2- 172.21.15.161:0/2623103114 >> [v2:172.21.15.161:6835/2172511349,v1:172.21.15.161:6837/2172511349] conn(0x559da8436100 0x559da8446530 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eb61a479016d1f8 server_cookie=5c9ecd2d88e6fd41 in_seq=0 out_seq=0 2024-05-22T21:24:39.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.903+0000 7f8417400640 10 client.8907 ms_handle_connect on v2:172.21.15.161:6835/2172511349 2024-05-22T21:24:39.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.907+0000 7f8417400640 1 -- 172.21.15.161:0/2623103114 <== mds.0 v2:172.21.15.161:6835/2172511349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559da8236790 con 0x559da8436100 2024-05-22T21:24:39.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.907+0000 7f8417400640 10 client.8907 handle_command_reply: tid=0 2024-05-22T21:24:39.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.907+0000 7f843be5c000 1 client.8907 shutdown 2024-05-22T21:24:39.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.907+0000 7f843be5c000 1 -- 172.21.15.161:0/2623103114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da83eaec0 msgr2=0x559da83efd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:39.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.907+0000 7f843be5c000 1 --2- 172.21.15.161:0/2623103114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da83eaec0 0x559da83efd60 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f842400ac60 tx=0x7f8424055ca0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.907+0000 7f843be5c000 1 -- 172.21.15.161:0/2623103114 shutdown_connections 2024-05-22T21:24:39.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.907+0000 7f843be5c000 1 --2- 172.21.15.161:0/2623103114 >> [v2:172.21.15.161:6835/2172511349,v1:172.21.15.161:6837/2172511349] conn(0x559da8436100 0x559da8446530 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.907+0000 7f843be5c000 1 --2- 172.21.15.161:0/2623103114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da84352e0 0x559da8434370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.907+0000 7f843be5c000 1 --2- 172.21.15.161:0/2623103114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da84349c0 0x559da8433e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.907+0000 7f843be5c000 1 --2- 172.21.15.161:0/2623103114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da83eaec0 0x559da83efd60 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.907+0000 7f8417400640 0 client.8907 ms_handle_reset on v2:172.21.15.161:6835/2172511349 2024-05-22T21:24:39.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.907+0000 7f843be5c000 1 -- 172.21.15.161:0/2623103114 >> 172.21.15.161:0/2623103114 conn(0x559da83f1440 msgr2=0x559da83e9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:39.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.907+0000 7f843be5c000 1 -- 172.21.15.161:0/2623103114 shutdown_connections 2024-05-22T21:24:39.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.907+0000 7f843be5c000 1 -- 172.21.15.161:0/2623103114 wait complete. 2024-05-22T21:24:39.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.907+0000 7f843be5c000 20 client.8907 trim_cache size 0 max 16384 2024-05-22T21:24:39.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 Processor -- start 2024-05-22T21:24:39.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 -- start start 2024-05-22T21:24:39.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da83f1440 0x559da8445530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da83eaec0 0x559da8445a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da84349c0 0x559da8445fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da7ef6940 con 0x559da84349c0 2024-05-22T21:24:39.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da81f7ce0 con 0x559da83eaec0 2024-05-22T21:24:39.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x559da83f0890 con 0x559da83f1440 2024-05-22T21:24:39.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f8432800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da84349c0 0x559da8445fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f8432800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da84349c0 0x559da8445fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59776/0 (socket says 172.21.15.161:59776) 2024-05-22T21:24:39.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f8432800640 1 -- 172.21.15.161:0/581291928 learned_addr learned my addr 172.21.15.161:0/581291928 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:39.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f8431400640 1 --2- 172.21.15.161:0/581291928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da83eaec0 0x559da8445a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f8431e00640 1 --2- 172.21.15.161:0/581291928 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da83f1440 0x559da8445530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f8432800640 1 -- 172.21.15.161:0/581291928 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da83f1440 msgr2=0x559da8445530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:39.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f8432800640 1 --2- 172.21.15.161:0/581291928 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da83f1440 0x559da8445530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f8432800640 1 -- 172.21.15.161:0/581291928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da83eaec0 msgr2=0x559da8445a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:39.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f8432800640 1 --2- 172.21.15.161:0/581291928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da83eaec0 0x559da8445a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f8432800640 1 -- 172.21.15.161:0/581291928 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559da81e4c00 con 0x559da84349c0 2024-05-22T21:24:39.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f8432800640 1 --2- 172.21.15.161:0/581291928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da84349c0 0x559da8445fb0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f842c01c560 tx=0x7f842c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d1cc1762181af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:39.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f8423400640 1 -- 172.21.15.161:0/581291928 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c006b50 con 0x559da84349c0 2024-05-22T21:24:39.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f8423400640 1 -- 172.21.15.161:0/581291928 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c004040 con 0x559da84349c0 2024-05-22T21:24:39.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f8423400640 1 -- 172.21.15.161:0/581291928 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c0043d0 con 0x559da84349c0 2024-05-22T21:24:39.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 -- 172.21.15.161:0/581291928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da84349c0 msgr2=0x559da8445fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:39.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 --2- 172.21.15.161:0/581291928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da84349c0 0x559da8445fb0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f842c01c560 tx=0x7f842c004740 comp rx=0 tx=0).stop 2024-05-22T21:24:39.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 -- 172.21.15.161:0/581291928 shutdown_connections 2024-05-22T21:24:39.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 --2- 172.21.15.161:0/581291928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da84349c0 0x559da8445fb0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 --2- 172.21.15.161:0/581291928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da83eaec0 0x559da8445a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 --2- 172.21.15.161:0/581291928 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da83f1440 0x559da8445530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 -- 172.21.15.161:0/581291928 >> 172.21.15.161:0/581291928 conn(0x559da83eef50 msgr2=0x559da84373f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:39.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 -- 172.21.15.161:0/581291928 shutdown_connections 2024-05-22T21:24:39.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.923+0000 7f843be5c000 1 -- 172.21.15.161:0/581291928 wait complete. 2024-05-22T21:24:39.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 1 -- start start 2024-05-22T21:24:39.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 1 Processor -- start 2024-05-22T21:24:39.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da83eaec0 0x559da849b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da83f1440 0x559da849b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da84349c0 0x559da8455410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da8434600 con 0x559da84349c0 2024-05-22T21:24:39.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da823c510 con 0x559da83f1440 2024-05-22T21:24:39.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x559da822e4d0 con 0x559da83eaec0 2024-05-22T21:24:39.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 10 client.0 _ll_register_callbacks cb 0x7f842c0ab480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-22T21:24:39.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8432800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da84349c0 0x559da8455410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8432800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da84349c0 0x559da8455410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59792/0 (socket says 172.21.15.161:59792) 2024-05-22T21:24:39.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8432800640 1 -- 172.21.15.161:0/4199768189 learned_addr learned my addr 172.21.15.161:0/4199768189 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:39.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8431400640 1 --2- 172.21.15.161:0/4199768189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da83f1440 0x559da849b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8431e00640 1 --2- 172.21.15.161:0/4199768189 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da83eaec0 0x559da849b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8432800640 1 -- 172.21.15.161:0/4199768189 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da83eaec0 msgr2=0x559da849b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:39.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8432800640 1 --2- 172.21.15.161:0/4199768189 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da83eaec0 0x559da849b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8432800640 1 -- 172.21.15.161:0/4199768189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da83f1440 msgr2=0x559da849b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:39.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8432800640 1 --2- 172.21.15.161:0/4199768189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da83f1440 0x559da849b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8432800640 1 -- 172.21.15.161:0/4199768189 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842c0c3040 con 0x559da84349c0 2024-05-22T21:24:39.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8432800640 1 --2- 172.21.15.161:0/4199768189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da84349c0 0x559da8455410 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f842c0c2040 tx=0x7f842c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef5456115686b19 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:39.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8417400640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-22T21:24:39.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8417400640 1 -- 172.21.15.161:0/4199768189 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c006b50 con 0x559da84349c0 2024-05-22T21:24:39.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8417400640 1 -- 172.21.15.161:0/4199768189 <== mon.0 v2:172.21.15.161:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f842c0041b0 con 0x559da84349c0 2024-05-22T21:24:39.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 1 -- 172.21.15.161:0/4199768189 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559da8209460 con 0x559da84349c0 2024-05-22T21:24:39.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8417400640 1 -- 172.21.15.161:0/4199768189 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f842c004370 con 0x559da84349c0 2024-05-22T21:24:39.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 10 client.8913 fetch_fsmap learned FSMap version 86 2024-05-22T21:24:39.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 1 -- 172.21.15.161:0/4199768189 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x559da7f35cc0 con 0x559da84349c0 2024-05-22T21:24:39.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8417400640 1 -- 172.21.15.161:0/4199768189 <== mon.0 v2:172.21.15.161:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559da8209460 con 0x559da84349c0 2024-05-22T21:24:39.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8417400640 1 -- 172.21.15.161:0/4199768189 <== mon.0 v2:172.21.15.161:3300/0 5 ==== fsmap(e 86) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f842c0ab870 con 0x559da84349c0 2024-05-22T21:24:39.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 10 client.8913 fetch_fsmap finished waiting for FSMap version 86 2024-05-22T21:24:39.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 10 client.8913 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-22T21:24:39.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 20 client.8913 populate_metadata read hostname 'smithi161' 2024-05-22T21:24:39.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 1 --2- 172.21.15.161:0/4199768189 >> [v2:172.21.15.161:6835/2172511349,v1:172.21.15.161:6837/2172511349] conn(0x559da84352e0 0x559da8455950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:39.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 4 client.8913 mds_command: new command op to 8784 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-22T21:24:39.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f843be5c000 1 -- 172.21.15.161:0/4199768189 --> [v2:172.21.15.161:6835/2172511349,v1:172.21.15.161:6837/2172511349] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559da82029c0 con 0x559da84352e0 2024-05-22T21:24:39.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8431e00640 1 --2- 172.21.15.161:0/4199768189 >> [v2:172.21.15.161:6835/2172511349,v1:172.21.15.161:6837/2172511349] conn(0x559da84352e0 0x559da8455950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:39.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8417400640 10 client.8913 ms_handle_connect on v2:172.21.15.161:6835/2172511349 2024-05-22T21:24:39.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.927+0000 7f8431e00640 1 --2- 172.21.15.161:0/4199768189 >> [v2:172.21.15.161:6835/2172511349,v1:172.21.15.161:6837/2172511349] conn(0x559da84352e0 0x559da8455950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2a3aee185d149d3 server_cookie=a2c0fd2f4d5af0b8 in_seq=0 out_seq=0 2024-05-22T21:24:39.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f8417400640 1 -- 172.21.15.161:0/4199768189 <== mds.0 v2:172.21.15.161:6835/2172511349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x559da82029c0 con 0x559da84352e0 2024-05-22T21:24:39.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f8417400640 10 client.8913 handle_command_reply: tid=0 2024-05-22T21:24:39.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f843be5c000 1 client.8913 shutdown 2024-05-22T21:24:39.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f843be5c000 1 -- 172.21.15.161:0/4199768189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da84349c0 msgr2=0x559da8455410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:39.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f843be5c000 1 --2- 172.21.15.161:0/4199768189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da84349c0 0x559da8455410 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f842c0c2040 tx=0x7f842c01ad60 comp rx=0 tx=0).stop 2024-05-22T21:24:39.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f843be5c000 1 -- 172.21.15.161:0/4199768189 shutdown_connections 2024-05-22T21:24:39.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f843be5c000 1 --2- 172.21.15.161:0/4199768189 >> [v2:172.21.15.161:6835/2172511349,v1:172.21.15.161:6837/2172511349] conn(0x559da84352e0 0x559da8455950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f843be5c000 1 --2- 172.21.15.161:0/4199768189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559da84349c0 0x559da8455410 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f843be5c000 1 --2- 172.21.15.161:0/4199768189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559da83f1440 0x559da849b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f843be5c000 1 --2- 172.21.15.161:0/4199768189 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x559da83eaec0 0x559da849b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f8417400640 0 client.8913 ms_handle_reset on v2:172.21.15.161:6835/2172511349 2024-05-22T21:24:39.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f843be5c000 1 -- 172.21.15.161:0/4199768189 >> 172.21.15.161:0/4199768189 conn(0x559da83eef50 msgr2=0x559da8432190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:39.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f843be5c000 1 -- 172.21.15.161:0/4199768189 shutdown_connections 2024-05-22T21:24:39.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f843be5c000 1 -- 172.21.15.161:0/4199768189 wait complete. 2024-05-22T21:24:39.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f843be5c000 20 client.8913 trim_cache size 0 max 16384 2024-05-22T21:24:39.935 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:24:39.935 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:24:39.935 INFO:teuthology.orchestra.run.smithi161.stdout: "whoami": 0, 2024-05-22T21:24:39.935 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 8784, 2024-05-22T21:24:39.935 INFO:teuthology.orchestra.run.smithi161.stdout: "want_state": "up:active", 2024-05-22T21:24:39.936 INFO:teuthology.orchestra.run.smithi161.stdout: "state": "up:active", 2024-05-22T21:24:39.936 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs", 2024-05-22T21:24:39.936 INFO:teuthology.orchestra.run.smithi161.stdout: "rank_uptime": 5.8339479799999996, 2024-05-22T21:24:39.936 INFO:teuthology.orchestra.run.smithi161.stdout: "mdsmap_epoch": 86, 2024-05-22T21:24:39.936 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch": 190, 2024-05-22T21:24:39.936 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_epoch_barrier": 190, 2024-05-22T21:24:39.936 INFO:teuthology.orchestra.run.smithi161.stdout: "uptime": 11.083953602999999 2024-05-22T21:24:39.936 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:24:39.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f8438a00640 1 -- 172.21.15.161:0/2881250392 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84100666c0 msgr2=0x7f8410068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:39.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f8438a00640 1 --2- 172.21.15.161:0/2881250392 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84100666c0 0x7f8410068b80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f843413a3f0 tx=0x7f841c008040 comp rx=0 tx=0).stop 2024-05-22T21:24:39.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f8438a00640 1 -- 172.21.15.161:0/2881250392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843406c9b0 msgr2=0x7f843413c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:39.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f8438a00640 1 --2- 172.21.15.161:0/2881250392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843406c9b0 0x7f843413c4e0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f842400cf60 tx=0x7f842400cf90 comp rx=0 tx=0).stop 2024-05-22T21:24:39.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f8438a00640 1 -- 172.21.15.161:0/2881250392 shutdown_connections 2024-05-22T21:24:39.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f8438a00640 1 --2- 172.21.15.161:0/2881250392 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84100666c0 0x7f8410068b80 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f8438a00640 1 --2- 172.21.15.161:0/2881250392 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f84341444a0 0x7f843415daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f8438a00640 1 --2- 172.21.15.161:0/2881250392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8434143c40 0x7f84341371c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f8438a00640 1 --2- 172.21.15.161:0/2881250392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843406c9b0 0x7f843413c4e0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:39.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f8438a00640 1 -- 172.21.15.161:0/2881250392 >> 172.21.15.161:0/2881250392 conn(0x7f843406a2d0 msgr2=0x7f843406c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:39.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f8438a00640 1 -- 172.21.15.161:0/2881250392 shutdown_connections 2024-05-22T21:24:39.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:39.931+0000 7f8438a00640 1 -- 172.21.15.161:0/2881250392 wait complete. 2024-05-22T21:24:39.949 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:24:40.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.087+0000 7fdeff800640 1 Processor -- start 2024-05-22T21:24:40.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.087+0000 7fdeff800640 1 -- start start 2024-05-22T21:24:40.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.087+0000 7fdeff800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdef815e130 0x7fdef815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:40.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.087+0000 7fdeff800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdef815d410 0x7fdef815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:40.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.087+0000 7fdeff800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef8071670 con 0x7fdef815dd50 2024-05-22T21:24:40.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.087+0000 7fdeff800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef815d870 con 0x7fdef815d410 2024-05-22T21:24:40.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.087+0000 7fdeff800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef8070a60 con 0x7fdef815e130 2024-05-22T21:24:40.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.087+0000 7fdef7e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdef815e130 0x7fdef815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:40.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.087+0000 7fdefd800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdef815d410 0x7fdef815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:40.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.087+0000 7fdef7e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdef815e130 0x7fdef815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:58078/0 (socket says 172.21.15.161:58078) 2024-05-22T21:24:40.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.087+0000 7fdef7e00640 1 -- 172.21.15.161:0/2055956025 learned_addr learned my addr 172.21.15.161:0/2055956025 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:40.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.087+0000 7fdefce00640 1 --1- 172.21.15.161:0/2055956025 >> v1:172.21.15.179:6789/0 conn(0x7fdef815dd50 0x7fdef806d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:49602/0 (socket says 172.21.15.161:49602) 2024-05-22T21:24:40.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.087+0000 7fdef7e00640 1 -- 172.21.15.161:0/2055956025 >> v2:172.21.15.179:3300/0 conn(0x7fdef815d410 msgr2=0x7fdef815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:40.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.087+0000 7fdef7e00640 1 --2- 172.21.15.161:0/2055956025 >> v2:172.21.15.179:3300/0 conn(0x7fdef815d410 0x7fdef815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:40.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.091+0000 7fdef7e00640 1 -- 172.21.15.161:0/2055956025 >> v1:172.21.15.179:6789/0 conn(0x7fdef815dd50 legacy=0x7fdef806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:40.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.091+0000 7fdef7e00640 1 -- 172.21.15.161:0/2055956025 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef8070d40 con 0x7fdef815e130 2024-05-22T21:24:40.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.091+0000 7fdefd800640 1 --2- 172.21.15.161:0/2055956025 >> v2:172.21.15.179:3300/0 conn(0x7fdef815d410 0x7fdef815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:24:40.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.091+0000 7fdef7e00640 1 --2- 172.21.15.161:0/2055956025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdef815e130 0x7fdef815ced0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009870 tx=0x7fdee401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=434083f4742d8dc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:40.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.091+0000 7fdef7400640 1 -- 172.21.15.161:0/2055956025 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee40a6040 con 0x7fdef815e130 2024-05-22T21:24:40.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.091+0000 7fdef7400640 1 -- 172.21.15.161:0/2055956025 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee40aa600 con 0x7fdef815e130 2024-05-22T21:24:40.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.091+0000 7fdef7400640 1 -- 172.21.15.161:0/2055956025 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee40aa990 con 0x7fdef815e130 2024-05-22T21:24:40.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdeff800640 1 -- 172.21.15.161:0/2055956025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdef815e130 msgr2=0x7fdef815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:40.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdeff800640 1 --2- 172.21.15.161:0/2055956025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdef815e130 0x7fdef815ced0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009870 tx=0x7fdee401c4a0 comp rx=0 tx=0).stop 2024-05-22T21:24:40.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdeff800640 1 -- 172.21.15.161:0/2055956025 shutdown_connections 2024-05-22T21:24:40.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdeff800640 1 --2- 172.21.15.161:0/2055956025 >> v2:172.21.15.179:3300/0 conn(0x7fdef815d410 0x7fdef815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:40.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdeff800640 1 --2- 172.21.15.161:0/2055956025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdef815e130 0x7fdef815ced0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:40.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdeff800640 1 -- 172.21.15.161:0/2055956025 >> 172.21.15.161:0/2055956025 conn(0x7fdef80b0bc0 msgr2=0x7fdef80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:40.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdeff800640 1 -- 172.21.15.161:0/2055956025 shutdown_connections 2024-05-22T21:24:40.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdeff800640 1 -- 172.21.15.161:0/2055956025 wait complete. 2024-05-22T21:24:40.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdeff800640 1 Processor -- start 2024-05-22T21:24:40.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdeff800640 1 -- start start 2024-05-22T21:24:40.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdeff800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdef815d410 0x7fdef81504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:40.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdeff800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdef815dd50 0x7fdef8153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:40.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdeff800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdef815e130 0x7fdef8153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:40.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdeff800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81522f0 con 0x7fdef815e130 2024-05-22T21:24:40.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdeff800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef8152560 con 0x7fdef815d410 2024-05-22T21:24:40.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdeff800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef8152840 con 0x7fdef815dd50 2024-05-22T21:24:40.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdefd800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdef815e130 0x7fdef8153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:40.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdefd800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdef815e130 0x7fdef8153580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59800/0 (socket says 172.21.15.161:59800) 2024-05-22T21:24:40.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdefd800640 1 -- 172.21.15.161:0/2529164829 learned_addr learned my addr 172.21.15.161:0/2529164829 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:40.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdef7e00640 1 --2- 172.21.15.161:0/2529164829 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdef815dd50 0x7fdef8153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:40.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdefce00640 1 --2- 172.21.15.161:0/2529164829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdef815d410 0x7fdef81504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:40.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdefd800640 1 -- 172.21.15.161:0/2529164829 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdef815dd50 msgr2=0x7fdef8153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:40.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdefd800640 1 --2- 172.21.15.161:0/2529164829 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdef815dd50 0x7fdef8153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:40.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdefd800640 1 -- 172.21.15.161:0/2529164829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdef815d410 msgr2=0x7fdef81504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:40.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdefd800640 1 --2- 172.21.15.161:0/2529164829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdef815d410 0x7fdef81504b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:40.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdefd800640 1 -- 172.21.15.161:0/2529164829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef000b6c0 con 0x7fdef815e130 2024-05-22T21:24:40.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdef7e00640 1 --2- 172.21.15.161:0/2529164829 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdef815dd50 0x7fdef8153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:40.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdefce00640 1 --2- 172.21.15.161:0/2529164829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdef815d410 0x7fdef81504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:40.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdefd800640 1 --2- 172.21.15.161:0/2529164829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdef815e130 0x7fdef8153580 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fdef000ecc0 tx=0x7fdef000d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:40.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.095+0000 7fdef4c00640 1 -- 172.21.15.161:0/2529164829 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0004230 con 0x7fdef815e130 2024-05-22T21:24:40.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.099+0000 7fdef4c00640 1 -- 172.21.15.161:0/2529164829 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef0004390 con 0x7fdef815e130 2024-05-22T21:24:40.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.099+0000 7fdef4c00640 1 -- 172.21.15.161:0/2529164829 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0003840 con 0x7fdef815e130 2024-05-22T21:24:40.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.099+0000 7fdeff800640 1 -- 172.21.15.161:0/2529164829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee40094d0 con 0x7fdef815e130 2024-05-22T21:24:40.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.099+0000 7fdeff800640 1 -- 172.21.15.161:0/2529164829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef81509f0 con 0x7fdef815e130 2024-05-22T21:24:40.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.099+0000 7fdeff800640 1 -- 172.21.15.161:0/2529164829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef8067220 con 0x7fdef815e130 2024-05-22T21:24:40.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.099+0000 7fdef4c00640 1 -- 172.21.15.161:0/2529164829 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdef00039a0 con 0x7fdef815e130 2024-05-22T21:24:40.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.099+0000 7fdef4c00640 1 --2- 172.21.15.161:0/2529164829 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fded8066690 0x7fded8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:40.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.099+0000 7fdef4c00640 1 -- 172.21.15.161:0/2529164829 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdef00848e0 con 0x7fdef815e130 2024-05-22T21:24:40.100 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.099+0000 7fdefce00640 1 --2- 172.21.15.161:0/2529164829 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fded8066690 0x7fded8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:40.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.099+0000 7fdefce00640 1 --2- 172.21.15.161:0/2529164829 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fded8066690 0x7fded8068b50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fdee80012f0 tx=0x7fdee8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:40.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.103+0000 7fdef4c00640 1 -- 172.21.15.161:0/2529164829 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fdef0051c50 con 0x7fdef815e130 2024-05-22T21:24:40.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.291+0000 7fdeff800640 1 -- 172.21.15.161:0/2529164829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdef8152cf0 con 0x7fdef815e130 2024-05-22T21:24:40.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.291+0000 7fdef4c00640 1 -- 172.21.15.161:0/2529164829 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdef001d070 con 0x7fdef815e130 2024-05-22T21:24:40.292 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:24:40.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.295+0000 7fdeff800640 1 -- 172.21.15.161:0/2529164829 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fded8066690 msgr2=0x7fded8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:40.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.295+0000 7fdeff800640 1 --2- 172.21.15.161:0/2529164829 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fded8066690 0x7fded8068b50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fdee80012f0 tx=0x7fdee8009040 comp rx=0 tx=0).stop 2024-05-22T21:24:40.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.295+0000 7fdeff800640 1 -- 172.21.15.161:0/2529164829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdef815e130 msgr2=0x7fdef8153580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:40.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.295+0000 7fdeff800640 1 --2- 172.21.15.161:0/2529164829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdef815e130 0x7fdef8153580 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fdef000ecc0 tx=0x7fdef000d720 comp rx=0 tx=0).stop 2024-05-22T21:24:40.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.295+0000 7fdeff800640 1 -- 172.21.15.161:0/2529164829 shutdown_connections 2024-05-22T21:24:40.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.295+0000 7fdeff800640 1 --2- 172.21.15.161:0/2529164829 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fded8066690 0x7fded8068b50 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:40.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.295+0000 7fdeff800640 1 --2- 172.21.15.161:0/2529164829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdef815e130 0x7fdef8153580 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:40.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.295+0000 7fdeff800640 1 --2- 172.21.15.161:0/2529164829 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdef815dd50 0x7fdef8153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:40.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.295+0000 7fdeff800640 1 --2- 172.21.15.161:0/2529164829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdef815d410 0x7fdef81504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:40.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.295+0000 7fdeff800640 1 -- 172.21.15.161:0/2529164829 >> 172.21.15.161:0/2529164829 conn(0x7fdef80b0bc0 msgr2=0x7fdef814f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:40.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.295+0000 7fdeff800640 1 -- 172.21.15.161:0/2529164829 shutdown_connections 2024-05-22T21:24:40.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:40.295+0000 7fdeff800640 1 -- 172.21.15.161:0/2529164829 wait complete. 2024-05-22T21:24:40.313 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:24:40.314 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:24:40.314 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-05-22T21:24:40.314 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-22T21:24:40.314 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-22T21:24:40.314 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:24:40.314 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:24:40.314 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:24:40.314 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:24:40.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:40.314 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-05-22T21:24:40.323 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:24:40.323 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:24:40.323 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:24:40.324 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:24:40.324 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-05-22T21:24:40.324 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:24:40.324 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:24:40.324 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-05-22T21:24:40.324 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2024-05-22T21:24:40.324 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:24:40.324 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:24:40.324 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2024-05-22T21:24:40.324 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:24:40.324 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:24:40.324 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:24:40.325 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:24:40.325 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:24:40.325 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 41812sec preferred_lft 41812sec 2024-05-22T21:24:40.325 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-05-22T21:24:40.325 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:24:40.325 INFO:teuthology.orchestra.run.smithi161.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:24:40.325 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 02:42:b9:0f:57:df brd ff:ff:ff:ff:ff:ff 2024-05-22T21:24:40.325 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:24:40.325 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:24:40.325 INFO:teuthology.orchestra.run.smithi161.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:24:40.325 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:24:40.325 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:24:40.325 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:24:40.326 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:24:40.326 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:24:40.326 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:24:40.326 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-22T21:24:40.326 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:24:40.326 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:24:40.326 INFO:teuthology.orchestra.run.smithi161.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:24:40.326 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-22T21:24:40.326 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:24:40.326 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:24:40.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:40.327 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:24:40.386 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-05-22T21:24:40.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:40.389 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-05-22T21:24:40.443 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-05-22T21:24:40.443 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:24:40.444 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:24:40.444 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:24:40.444 INFO:teuthology.orchestra.run.smithi161.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:24:40.444 INFO:teuthology.orchestra.run.smithi161.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:24:40.444 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:24:40.444 INFO:teuthology.orchestra.run.smithi161.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:24:40.444 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:24:40.444 INFO:teuthology.orchestra.run.smithi161.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:24:40.444 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:24:40.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:40.444 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-05-22T21:24:40.445 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-05-22T21:24:40.445 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:24:40.445 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:24:40.445 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:24:40.445 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-05-22T21:24:40.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:40.538 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-05-22T21:24:40.590 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-22T21:24:40.590 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-22T21:24:40.590 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:24:40.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:40.591 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2024-05-22T21:24:40.653 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:24:40.653 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-22T21:24:40.705 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-22T21:24:40.705 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:24:40.705 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:24:40.756 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:24:40.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:40.756 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:24:40.845 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:24:40.845 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:24:40.845 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:24:40.860 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:24:40.860 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:24:40.861 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:24:40.861 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:24:40.861 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:24:40.869 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-22T21:24:40.869 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-22T21:24:40.869 INFO:teuthology.orchestra.run.smithi161.stdout:invalid new device string format 2024-05-22T21:24:40.869 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:24:40.869 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:24:40.869 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:24:40.869 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:24:40.869 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:24:40.873 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:24:40.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:40.873 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-22T21:24:40.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:40.892 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:24:40.953 INFO:teuthology.orchestra.run.smithi161.stdout:client8919 2024-05-22T21:24:40.954 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:24:40.954 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:24:41.009 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-22T21:24:41.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.183+0000 7fa7fe000640 1 Processor -- start 2024-05-22T21:24:41.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.183+0000 7fa7fe000640 1 -- start start 2024-05-22T21:24:41.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7fe000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7f8143ec0 0x7fa7f815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:41.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7fe000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f80715a0 0x7fa7f806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:41.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7fe000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa7f815ec90 0x7fa7f815f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:41.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7fe000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f815f660 con 0x7fa7f8143ec0 2024-05-22T21:24:41.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7fe000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f81442a0 con 0x7fa7f815ec90 2024-05-22T21:24:41.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7fe000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f8070260 con 0x7fa7f80715a0 2024-05-22T21:24:41.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7f7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7f8143ec0 0x7fa7f815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7f7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7f8143ec0 0x7fa7f815f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59816/0 (socket says 172.21.15.161:59816) 2024-05-22T21:24:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7f7400640 1 -- 172.21.15.161:0/3907787102 learned_addr learned my addr 172.21.15.161:0/3907787102 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7f7e00640 1 --2- 172.21.15.161:0/3907787102 >> v2:172.21.15.179:3300/0 conn(0x7fa7f815ec90 0x7fa7f815f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7eea00640 1 --2- 172.21.15.161:0/3907787102 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f80715a0 0x7fa7f806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7f7400640 1 -- 172.21.15.161:0/3907787102 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f80715a0 msgr2=0x7fa7f806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7f7400640 1 --2- 172.21.15.161:0/3907787102 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f80715a0 0x7fa7f806fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7f7400640 1 -- 172.21.15.161:0/3907787102 >> v2:172.21.15.179:3300/0 conn(0x7fa7f815ec90 msgr2=0x7fa7f815f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7f7400640 1 --2- 172.21.15.161:0/3907787102 >> v2:172.21.15.179:3300/0 conn(0x7fa7f815ec90 0x7fa7f815f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:41.188 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7f7400640 1 -- 172.21.15.161:0/3907787102 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f8070500 con 0x7fa7f8143ec0 2024-05-22T21:24:41.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7f7e00640 1 --2- 172.21.15.161:0/3907787102 >> v2:172.21.15.179:3300/0 conn(0x7fa7f815ec90 0x7fa7f815f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:41.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.187+0000 7fa7eea00640 1 --2- 172.21.15.161:0/3907787102 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f80715a0 0x7fa7f806fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:41.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.191+0000 7fa7f7400640 1 --2- 172.21.15.161:0/3907787102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f8143ec0 0x7fa7f815f830 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80b1be0 tx=0x7fa7e801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824be79f8179e3cd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:41.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.191+0000 7fa7f6a00640 1 -- 172.21.15.161:0/3907787102 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e80aa610 con 0x7fa7f8143ec0 2024-05-22T21:24:41.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.191+0000 7fa7f6a00640 1 -- 172.21.15.161:0/3907787102 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e80a6040 con 0x7fa7f8143ec0 2024-05-22T21:24:41.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.191+0000 7fa7f6a00640 1 -- 172.21.15.161:0/3907787102 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e80aa990 con 0x7fa7f8143ec0 2024-05-22T21:24:41.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.191+0000 7fa7fe000640 1 -- 172.21.15.161:0/3907787102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f8143ec0 msgr2=0x7fa7f815f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:41.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.191+0000 7fa7fe000640 1 --2- 172.21.15.161:0/3907787102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f8143ec0 0x7fa7f815f830 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80b1be0 tx=0x7fa7e801c4f0 comp rx=0 tx=0).stop 2024-05-22T21:24:41.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.191+0000 7fa7fe000640 1 -- 172.21.15.161:0/3907787102 shutdown_connections 2024-05-22T21:24:41.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.191+0000 7fa7fe000640 1 --2- 172.21.15.161:0/3907787102 >> v2:172.21.15.179:3300/0 conn(0x7fa7f815ec90 0x7fa7f815f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:41.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.191+0000 7fa7fe000640 1 --2- 172.21.15.161:0/3907787102 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f80715a0 0x7fa7f806fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:41.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.191+0000 7fa7fe000640 1 --2- 172.21.15.161:0/3907787102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f8143ec0 0x7fa7f815f830 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:41.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.191+0000 7fa7fe000640 1 -- 172.21.15.161:0/3907787102 >> 172.21.15.161:0/3907787102 conn(0x7fa7f80b0bc0 msgr2=0x7fa7f80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:41.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.191+0000 7fa7fe000640 1 -- 172.21.15.161:0/3907787102 shutdown_connections 2024-05-22T21:24:41.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.191+0000 7fa7fe000640 1 -- 172.21.15.161:0/3907787102 wait complete. 2024-05-22T21:24:41.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.191+0000 7fa7fe000640 1 Processor -- start 2024-05-22T21:24:41.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.191+0000 7fa7fe000640 1 -- start start 2024-05-22T21:24:41.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.191+0000 7fa7fe000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f8143ec0 0x7fa7f81573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:41.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7fe000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f815ec90 0x7fa7f80aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:41.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7fe000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7f8159d30 0x7fa7f815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:41.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7fe000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f8157920 con 0x7fa7f815ec90 2024-05-22T21:24:41.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7fe000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f80b21b0 con 0x7fa7f8159d30 2024-05-22T21:24:41.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7fe000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f8157b70 con 0x7fa7f8143ec0 2024-05-22T21:24:41.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7eea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f815ec90 0x7fa7f80aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:41.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7eea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f815ec90 0x7fa7f80aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59826/0 (socket says 172.21.15.161:59826) 2024-05-22T21:24:41.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7eea00640 1 -- 172.21.15.161:0/2167861579 learned_addr learned my addr 172.21.15.161:0/2167861579 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:41.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7f7e00640 1 --2- 172.21.15.161:0/2167861579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7f8159d30 0x7fa7f815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:41.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7f7400640 1 --2- 172.21.15.161:0/2167861579 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f8143ec0 0x7fa7f81573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:41.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7eea00640 1 -- 172.21.15.161:0/2167861579 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f8143ec0 msgr2=0x7fa7f81573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:41.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7eea00640 1 --2- 172.21.15.161:0/2167861579 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f8143ec0 0x7fa7f81573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:41.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7eea00640 1 -- 172.21.15.161:0/2167861579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7f8159d30 msgr2=0x7fa7f815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:41.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7eea00640 1 --2- 172.21.15.161:0/2167861579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7f8159d30 0x7fa7f815a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:41.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7eea00640 1 -- 172.21.15.161:0/2167861579 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e80094d0 con 0x7fa7f815ec90 2024-05-22T21:24:41.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7f7e00640 1 --2- 172.21.15.161:0/2167861579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7f8159d30 0x7fa7f815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:41.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7eea00640 1 --2- 172.21.15.161:0/2167861579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f815ec90 0x7fa7f80aed90 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000c7c0 tx=0x7fa7e000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:41.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7efe00640 1 -- 172.21.15.161:0/2167861579 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0007a30 con 0x7fa7f815ec90 2024-05-22T21:24:41.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7efe00640 1 -- 172.21.15.161:0/2167861579 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e0007b90 con 0x7fa7f815ec90 2024-05-22T21:24:41.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7efe00640 1 -- 172.21.15.161:0/2167861579 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e000ce70 con 0x7fa7f815ec90 2024-05-22T21:24:41.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7fe000640 1 -- 172.21.15.161:0/2167861579 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f8157e50 con 0x7fa7f815ec90 2024-05-22T21:24:41.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7fe000640 1 -- 172.21.15.161:0/2167861579 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f8158410 con 0x7fa7f815ec90 2024-05-22T21:24:41.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7fe000640 1 -- 172.21.15.161:0/2167861579 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f8158930 con 0x7fa7f815ec90 2024-05-22T21:24:41.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.195+0000 7fa7efe00640 1 -- 172.21.15.161:0/2167861579 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa7e00113f0 con 0x7fa7f815ec90 2024-05-22T21:24:41.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.199+0000 7fa7efe00640 1 --2- 172.21.15.161:0/2167861579 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7d4066690 0x7fa7d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:41.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.199+0000 7fa7efe00640 1 -- 172.21.15.161:0/2167861579 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7e0084730 con 0x7fa7f815ec90 2024-05-22T21:24:41.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.199+0000 7fa7f7400640 1 --2- 172.21.15.161:0/2167861579 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7d4066690 0x7fa7d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:41.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.199+0000 7fa7f7400640 1 --2- 172.21.15.161:0/2167861579 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7d4066690 0x7fa7d4068b50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80ae000 tx=0x7fa7e80a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:41.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.203+0000 7fa7efe00640 1 -- 172.21.15.161:0/2167861579 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa7f8158930 con 0x7fa7f815ec90 2024-05-22T21:24:41.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.387+0000 7fa7fe000640 1 -- 172.21.15.161:0/2167861579 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa7f8158930 con 0x7fa7f815ec90 2024-05-22T21:24:41.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.391+0000 7fa7efe00640 1 -- 172.21.15.161:0/2167861579 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa7f8158930 con 0x7fa7f815ec90 2024-05-22T21:24:41.390 INFO:teuthology.orchestra.run.smithi161.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-22T21:24:41.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.391+0000 7fa7fe000640 1 -- 172.21.15.161:0/2167861579 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7d4066690 msgr2=0x7fa7d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:41.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.391+0000 7fa7fe000640 1 --2- 172.21.15.161:0/2167861579 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7d4066690 0x7fa7d4068b50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80ae000 tx=0x7fa7e80a7000 comp rx=0 tx=0).stop 2024-05-22T21:24:41.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.391+0000 7fa7fe000640 1 -- 172.21.15.161:0/2167861579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f815ec90 msgr2=0x7fa7f80aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:41.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.391+0000 7fa7fe000640 1 --2- 172.21.15.161:0/2167861579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f815ec90 0x7fa7f80aed90 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000c7c0 tx=0x7fa7e000cc80 comp rx=0 tx=0).stop 2024-05-22T21:24:41.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.391+0000 7fa7fe000640 1 -- 172.21.15.161:0/2167861579 shutdown_connections 2024-05-22T21:24:41.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.391+0000 7fa7fe000640 1 --2- 172.21.15.161:0/2167861579 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa7d4066690 0x7fa7d4068b50 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:41.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.391+0000 7fa7fe000640 1 --2- 172.21.15.161:0/2167861579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7f8159d30 0x7fa7f815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:41.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.391+0000 7fa7fe000640 1 --2- 172.21.15.161:0/2167861579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7f815ec90 0x7fa7f80aed90 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:41.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.391+0000 7fa7fe000640 1 --2- 172.21.15.161:0/2167861579 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7f8143ec0 0x7fa7f81573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:41.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.391+0000 7fa7fe000640 1 -- 172.21.15.161:0/2167861579 >> 172.21.15.161:0/2167861579 conn(0x7fa7f80b0bc0 msgr2=0x7fa7f815d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:41.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.391+0000 7fa7fe000640 1 -- 172.21.15.161:0/2167861579 shutdown_connections 2024-05-22T21:24:41.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:41.391+0000 7fa7fe000640 1 -- 172.21.15.161:0/2167861579 wait complete. 2024-05-22T21:24:41.411 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-22T21:24:41.411 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-22T21:24:41.411 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi179.front.sepia.ceph.com 2024-05-22T21:24:41.411 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-22T21:24:41.411 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-22T21:24:41.411 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-22T21:24:41.411 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-22T21:24:41.411 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-22T21:24:41.411 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:24:41.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:41.412 DEBUG:teuthology.orchestra.run.smithi179:> ip addr 2024-05-22T21:24:41.422 INFO:teuthology.orchestra.run.smithi179.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-22T21:24:41.422 INFO:teuthology.orchestra.run.smithi179.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-22T21:24:41.422 INFO:teuthology.orchestra.run.smithi179.stdout: inet 127.0.0.1/8 scope host lo 2024-05-22T21:24:41.422 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:24:41.422 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 ::1/128 scope host 2024-05-22T21:24:41.422 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:24:41.422 INFO:teuthology.orchestra.run.smithi179.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:24:41.422 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:34 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:24:41.422 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f0 2024-05-22T21:24:41.423 INFO:teuthology.orchestra.run.smithi179.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:24:41.423 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:d9:3b:35 brd ff:ff:ff:ff:ff:ff 2024-05-22T21:24:41.423 INFO:teuthology.orchestra.run.smithi179.stdout: altname enp7s0f1 2024-05-22T21:24:41.423 INFO:teuthology.orchestra.run.smithi179.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-22T21:24:41.423 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5a brd ff:ff:ff:ff:ff:ff 2024-05-22T21:24:41.423 INFO:teuthology.orchestra.run.smithi179.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-22T21:24:41.423 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0c:c4:7a:88:72:5b brd ff:ff:ff:ff:ff:ff 2024-05-22T21:24:41.423 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.21.15.179/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-22T21:24:41.423 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft 41803sec preferred_lft 41803sec 2024-05-22T21:24:41.423 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::ec4:7aff:fe88:725b/64 scope link 2024-05-22T21:24:41.423 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:24:41.423 INFO:teuthology.orchestra.run.smithi179.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-22T21:24:41.424 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 02:42:bd:4e:25:3d brd ff:ff:ff:ff:ff:ff 2024-05-22T21:24:41.424 INFO:teuthology.orchestra.run.smithi179.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-22T21:24:41.424 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:24:41.424 INFO:teuthology.orchestra.run.smithi179.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-22T21:24:41.424 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-22T21:24:41.424 INFO:teuthology.orchestra.run.smithi179.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-22T21:24:41.424 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:24:41.424 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-22T21:24:41.424 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:24:41.424 INFO:teuthology.orchestra.run.smithi179.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:24:41.424 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-22T21:24:41.424 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-22T21:24:41.424 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:24:41.425 INFO:teuthology.orchestra.run.smithi179.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-22T21:24:41.425 INFO:teuthology.orchestra.run.smithi179.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-22T21:24:41.425 INFO:teuthology.orchestra.run.smithi179.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-22T21:24:41.425 INFO:teuthology.orchestra.run.smithi179.stdout: valid_lft forever preferred_lft forever 2024-05-22T21:24:41.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:41.426 DEBUG:teuthology.orchestra.run.smithi179:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-22T21:24:41.489 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-05-22T21:24:41.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:41.491 DEBUG:teuthology.orchestra.run.smithi179:> route 2024-05-22T21:24:41.545 INFO:teuthology.orchestra.run.smithi179.stdout:Kernel IP routing table 2024-05-22T21:24:41.545 INFO:teuthology.orchestra.run.smithi179.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-22T21:24:41.545 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-22T21:24:41.545 INFO:teuthology.orchestra.run.smithi179.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-22T21:24:41.545 INFO:teuthology.orchestra.run.smithi179.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-22T21:24:41.546 INFO:teuthology.orchestra.run.smithi179.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-22T21:24:41.546 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-22T21:24:41.546 INFO:teuthology.orchestra.run.smithi179.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:24:41.546 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:24:41.546 INFO:teuthology.orchestra.run.smithi179.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-22T21:24:41.546 INFO:teuthology.orchestra.run.smithi179.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-22T21:24:41.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:41.546 DEBUG:teuthology.orchestra.run.smithi179:> (cd / && exec bash -c ' 2024-05-22T21:24:41.546 DEBUG:teuthology.orchestra.run.smithi179:> set -e 2024-05-22T21:24:41.546 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-22T21:24:41.547 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-22T21:24:41.547 DEBUG:teuthology.orchestra.run.smithi179:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-22T21:24:41.547 DEBUG:teuthology.orchestra.run.smithi179:> ') 2024-05-22T21:24:41.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:41.646 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-22T21:24:41.655 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-22T21:24:41.655 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-22T21:24:41.656 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:24:41.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:41.656 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link set brx.0 up 2024-05-22T21:24:41.718 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:24:41.718 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-22T21:24:41.769 INFO:teuthology.orchestra.run.smithi179.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-22T21:24:41.770 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-22T21:24:41.770 DEBUG:teuthology.orchestra.run.smithi179:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:24:41.821 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-22T21:24:41.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:41.822 DEBUG:teuthology.orchestra.run.smithi179:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:24:41.917 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:24:41.917 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:24:41.917 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:24:41.932 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:24:41.932 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:24:41.932 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:24:41.932 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:24:41.933 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:24:41.942 INFO:teuthology.orchestra.run.smithi179.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-22T21:24:41.942 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-22T21:24:41.942 INFO:teuthology.orchestra.run.smithi179.stdout:invalid new device string format 2024-05-22T21:24:41.942 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:3300,172.21.15.179:3300,172.21.15.179:3301" 2024-05-22T21:24:41.942 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-22T21:24:41.942 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: resolved to: "172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790" 2024-05-22T21:24:41.942 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.161:6789,172.21.15.179:6789,172.21.15.179:6790:/ 2024-05-22T21:24:41.943 INFO:teuthology.orchestra.run.smithi179.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=d6fa8528-b1bf-41b6-9f76-c0d715394087" will pass to kernel 2024-05-22T21:24:41.946 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-22T21:24:41.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:41.946 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-22T21:24:42.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:42.011 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-22T21:24:42.067 INFO:teuthology.orchestra.run.smithi179.stdout:client7310 2024-05-22T21:24:42.068 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:24:42.068 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:24:42.118 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:24:42.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f7074400640 1 Processor -- start 2024-05-22T21:24:42.255 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f7074400640 1 -- start start 2024-05-22T21:24:42.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f7074400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f706c15e130 0x7f706c15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:42.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f7074400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f706c15d410 0x7f706c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:42.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f7074400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706c071670 con 0x7f706c15dd50 2024-05-22T21:24:42.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f7074400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706c15d870 con 0x7f706c15d410 2024-05-22T21:24:42.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f7074400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706c070a60 con 0x7f706c15e130 2024-05-22T21:24:42.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f7072400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f706c15d410 0x7f706c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:42.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f7071a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f706c15dd50 0x7f706c06d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59140/0 (socket says 172.21.15.161:59140) 2024-05-22T21:24:42.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f7071a00640 1 -- 172.21.15.161:0/1625494880 learned_addr learned my addr 172.21.15.161:0/1625494880 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:42.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f7071000640 1 --2- 172.21.15.161:0/1625494880 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f706c15e130 0x7f706c15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:42.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f7072400640 1 -- 172.21.15.161:0/1625494880 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f706c15e130 msgr2=0x7f706c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:42.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f7072400640 1 --2- 172.21.15.161:0/1625494880 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f706c15e130 0x7f706c15ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:42.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f7072400640 1 -- 172.21.15.161:0/1625494880 >> v1:172.21.15.161:6789/0 conn(0x7f706c15dd50 legacy=0x7f706c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:42.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f706be00640 1 -- 172.21.15.161:0/1625494880 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279464506 0 0) 0x7f706c071670 con 0x7f706c15dd50 2024-05-22T21:24:42.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f7072400640 1 -- 172.21.15.161:0/1625494880 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c070d40 con 0x7f706c15d410 2024-05-22T21:24:42.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.255+0000 7f7071000640 1 --2- 172.21.15.161:0/1625494880 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f706c15e130 0x7f706c15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:24:42.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.259+0000 7f7072400640 1 --2- 172.21.15.161:0/1625494880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f706c15d410 0x7f706c15b6c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f7064009d90 tx=0x7f706401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae45cf061920a9af server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:42.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.259+0000 7f706be00640 1 -- 172.21.15.161:0/1625494880 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70640a5970 con 0x7f706c15d410 2024-05-22T21:24:42.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.259+0000 7f706be00640 1 -- 172.21.15.161:0/1625494880 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70640a5ad0 con 0x7f706c15d410 2024-05-22T21:24:42.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.259+0000 7f706be00640 1 -- 172.21.15.161:0/1625494880 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70640a5e60 con 0x7f706c15d410 2024-05-22T21:24:42.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.259+0000 7f7074400640 1 -- 172.21.15.161:0/1625494880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f706c15d410 msgr2=0x7f706c15b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:42.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.259+0000 7f7074400640 1 --2- 172.21.15.161:0/1625494880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f706c15d410 0x7f706c15b6c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f7064009d90 tx=0x7f706401c8b0 comp rx=0 tx=0).stop 2024-05-22T21:24:42.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.259+0000 7f7074400640 1 -- 172.21.15.161:0/1625494880 shutdown_connections 2024-05-22T21:24:42.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.259+0000 7f7074400640 1 --2- 172.21.15.161:0/1625494880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f706c15d410 0x7f706c15b6c0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:42.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.259+0000 7f7074400640 1 --2- 172.21.15.161:0/1625494880 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f706c15e130 0x7f706c15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:42.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.259+0000 7f7074400640 1 -- 172.21.15.161:0/1625494880 >> 172.21.15.161:0/1625494880 conn(0x7f706c0b0bc0 msgr2=0x7f706c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:42.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.259+0000 7f7074400640 1 -- 172.21.15.161:0/1625494880 shutdown_connections 2024-05-22T21:24:42.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.259+0000 7f7074400640 1 -- 172.21.15.161:0/1625494880 wait complete. 2024-05-22T21:24:42.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7074400640 1 Processor -- start 2024-05-22T21:24:42.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7074400640 1 -- start start 2024-05-22T21:24:42.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7074400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f706c15d410 0x7f706c0b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:42.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7074400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f706c15dd50 0x7f706c1546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:42.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7074400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f706c15e130 0x7f706c0aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:42.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7074400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c0b14e0 con 0x7f706c15d410 2024-05-22T21:24:42.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7074400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c1528f0 con 0x7f706c15e130 2024-05-22T21:24:42.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7074400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706c154c20 con 0x7f706c15dd50 2024-05-22T21:24:42.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7071a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f706c15d410 0x7f706c0b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:42.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7071a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f706c15d410 0x7f706c0b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59840/0 (socket says 172.21.15.161:59840) 2024-05-22T21:24:42.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7071a00640 1 -- 172.21.15.161:0/4236391791 learned_addr learned my addr 172.21.15.161:0/4236391791 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:42.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7072400640 1 --2- 172.21.15.161:0/4236391791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f706c15e130 0x7f706c0aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:42.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7071000640 1 --2- 172.21.15.161:0/4236391791 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f706c15dd50 0x7f706c1546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:42.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7071a00640 1 -- 172.21.15.161:0/4236391791 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f706c15dd50 msgr2=0x7f706c1546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:42.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7071a00640 1 --2- 172.21.15.161:0/4236391791 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f706c15dd50 0x7f706c1546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:42.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7071a00640 1 -- 172.21.15.161:0/4236391791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f706c15e130 msgr2=0x7f706c0aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:42.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7071a00640 1 --2- 172.21.15.161:0/4236391791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f706c15e130 0x7f706c0aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:42.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7071a00640 1 -- 172.21.15.161:0/4236391791 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7064009a70 con 0x7f706c15d410 2024-05-22T21:24:42.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7071000640 1 --2- 172.21.15.161:0/4236391791 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f706c15dd50 0x7f706c1546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:42.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7071a00640 1 --2- 172.21.15.161:0/4236391791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f706c15d410 0x7f706c0b0fa0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f706000c520 tx=0x7f706000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:42.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7069600640 1 -- 172.21.15.161:0/4236391791 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70600074e0 con 0x7f706c15d410 2024-05-22T21:24:42.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7069600640 1 -- 172.21.15.161:0/4236391791 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7060007640 con 0x7f706c15d410 2024-05-22T21:24:42.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7074400640 1 -- 172.21.15.161:0/4236391791 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706c0b1b70 con 0x7f706c15d410 2024-05-22T21:24:42.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7069600640 1 -- 172.21.15.161:0/4236391791 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70600049f0 con 0x7f706c15d410 2024-05-22T21:24:42.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.263+0000 7f7074400640 1 -- 172.21.15.161:0/4236391791 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706c0b2130 con 0x7f706c15d410 2024-05-22T21:24:42.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.267+0000 7f7074400640 1 -- 172.21.15.161:0/4236391791 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706c067220 con 0x7f706c15d410 2024-05-22T21:24:42.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.267+0000 7f7069600640 1 -- 172.21.15.161:0/4236391791 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7060004b50 con 0x7f706c15d410 2024-05-22T21:24:42.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.267+0000 7f7069600640 1 --2- 172.21.15.161:0/4236391791 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f704c066690 0x7f704c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:42.267 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.267+0000 7f7069600640 1 -- 172.21.15.161:0/4236391791 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70600837a0 con 0x7f706c15d410 2024-05-22T21:24:42.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.267+0000 7f7071000640 1 --2- 172.21.15.161:0/4236391791 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f704c066690 0x7f704c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:42.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.267+0000 7f7071000640 1 --2- 172.21.15.161:0/4236391791 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f704c066690 0x7f704c068b50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f706c0b1860 tx=0x7f7058008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:42.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.271+0000 7f7069600640 1 -- 172.21.15.161:0/4236391791 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f7060050bb0 con 0x7f706c15d410 2024-05-22T21:24:42.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.455+0000 7f7074400640 1 -- 172.21.15.161:0/4236391791 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f706c070730 con 0x7f706c15d410 2024-05-22T21:24:42.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.455+0000 7f7069600640 1 -- 172.21.15.161:0/4236391791 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4744 (secure 0 0 0) 0x7f7060018020 con 0x7f706c15d410 2024-05-22T21:24:42.455 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:42.455 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":86,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3304182060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3304182060},{"type":"v1","addr":"172.21.15.179:6835","nonce":3304182060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196309272},{"type":"v1","addr":"172.21.15.161:6836","nonce":196309272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/396408969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":396408969},{"type":"v1","addr":"172.21.15.179:6834","nonce":396408969}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:24:34.096673+0000","modified":"2024-05-22T21:24:37.112845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8784},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8784":{"gid":8784,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2172511349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2172511349},{"type":"v1","addr":"172.21.15.161:6837","nonce":2172511349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-05-22T21:24:42.456 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 86 2024-05-22T21:24:42.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.455+0000 7f7074400640 1 -- 172.21.15.161:0/4236391791 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f704c066690 msgr2=0x7f704c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:42.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.455+0000 7f7074400640 1 --2- 172.21.15.161:0/4236391791 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f704c066690 0x7f704c068b50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f706c0b1860 tx=0x7f7058008040 comp rx=0 tx=0).stop 2024-05-22T21:24:42.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.455+0000 7f7074400640 1 -- 172.21.15.161:0/4236391791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f706c15d410 msgr2=0x7f706c0b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:42.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.455+0000 7f7074400640 1 --2- 172.21.15.161:0/4236391791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f706c15d410 0x7f706c0b0fa0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f706000c520 tx=0x7f706000c9e0 comp rx=0 tx=0).stop 2024-05-22T21:24:42.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.455+0000 7f7074400640 1 -- 172.21.15.161:0/4236391791 shutdown_connections 2024-05-22T21:24:42.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.455+0000 7f7074400640 1 --2- 172.21.15.161:0/4236391791 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f704c066690 0x7f704c068b50 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:42.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.455+0000 7f7074400640 1 --2- 172.21.15.161:0/4236391791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f706c15e130 0x7f706c0aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:42.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.455+0000 7f7074400640 1 --2- 172.21.15.161:0/4236391791 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f706c15dd50 0x7f706c1546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:42.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.455+0000 7f7074400640 1 --2- 172.21.15.161:0/4236391791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f706c15d410 0x7f706c0b0fa0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:42.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.455+0000 7f7074400640 1 -- 172.21.15.161:0/4236391791 >> 172.21.15.161:0/4236391791 conn(0x7f706c0b0bc0 msgr2=0x7f706c1574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:42.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.459+0000 7f7074400640 1 -- 172.21.15.161:0/4236391791 shutdown_connections 2024-05-22T21:24:42.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.459+0000 7f7074400640 1 -- 172.21.15.161:0/4236391791 wait complete. 2024-05-22T21:24:42.476 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-05-22T21:24:42.633 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":8784,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.5203505209999992,"mdsmap_epoch":86,"osdmap_epoch":190,"osdmap_epoch_barrier":190,"uptime":13.770358014999999} 2024-05-22T21:24:42.634 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 8784, "mdsmap_epoch": 86, "osdmap_epoch": 190, "osdmap_epoch_barrier": 190, "rank_uptime": 8.520350521, "state": "up:active", "uptime": 13.770358015, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:24:42.634 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-05-22T21:24:42.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.771+0000 7faa3f600640 1 Processor -- start 2024-05-22T21:24:42.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.771+0000 7faa3f600640 1 -- start start 2024-05-22T21:24:42.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.775+0000 7faa3f600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7faa3813d560 0x7faa380cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:42.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.775+0000 7faa3f600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa3815e1d0 0x7faa380ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:42.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.775+0000 7faa3f600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa380ce580 con 0x7faa3813cfe0 2024-05-22T21:24:42.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.775+0000 7faa3f600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa38140560 con 0x7faa3813d560 2024-05-22T21:24:42.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.775+0000 7faa3f600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa380ce830 con 0x7faa3815e1d0 2024-05-22T21:24:42.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.775+0000 7faa3cc00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7faa3813cfe0 0x7faa3815dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59152/0 (socket says 172.21.15.161:59152) 2024-05-22T21:24:42.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.775+0000 7faa3cc00640 1 -- 172.21.15.161:0/3860290346 learned_addr learned my addr 172.21.15.161:0/3860290346 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:42.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.775+0000 7faa37e00640 1 --2- 172.21.15.161:0/3860290346 >> v2:172.21.15.179:3300/0 conn(0x7faa3813d560 0x7faa380cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:42.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.775+0000 7faa3d600640 1 --2- 172.21.15.161:0/3860290346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa3815e1d0 0x7faa380ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:42.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.775+0000 7faa37400640 1 -- 172.21.15.161:0/3860290346 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2275138313 0 0) 0x7faa380ce580 con 0x7faa3813cfe0 2024-05-22T21:24:42.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.775+0000 7faa37400640 1 -- 172.21.15.161:0/3860290346 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa1c003530 con 0x7faa3813cfe0 2024-05-22T21:24:42.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.775+0000 7faa37e00640 1 -- 172.21.15.161:0/3860290346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa3815e1d0 msgr2=0x7faa380ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:42.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.775+0000 7faa37e00640 1 --2- 172.21.15.161:0/3860290346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa3815e1d0 0x7faa380ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:42.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.775+0000 7faa37e00640 1 -- 172.21.15.161:0/3860290346 >> v1:172.21.15.161:6789/0 conn(0x7faa3813cfe0 legacy=0x7faa3815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:42.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.775+0000 7faa37400640 1 -- 172.21.15.161:0/3860290346 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1723451628 0 0) 0x7faa1c003530 con 0x7faa3813cfe0 2024-05-22T21:24:42.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.775+0000 7faa37e00640 1 -- 172.21.15.161:0/3860290346 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa380ceb10 con 0x7faa3813d560 2024-05-22T21:24:42.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.779+0000 7faa37e00640 1 --2- 172.21.15.161:0/3860290346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa3813d560 0x7faa380cdb00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7faa200097c0 tx=0x7faa2001c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a752252566ce29c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:42.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.779+0000 7faa37400640 1 -- 172.21.15.161:0/3860290346 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28001680 con 0x7faa3813d560 2024-05-22T21:24:42.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.779+0000 7faa37400640 1 -- 172.21.15.161:0/3860290346 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa200a7050 con 0x7faa3813d560 2024-05-22T21:24:42.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.779+0000 7faa37400640 1 -- 172.21.15.161:0/3860290346 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa200ab620 con 0x7faa3813d560 2024-05-22T21:24:42.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.779+0000 7faa3f600640 1 -- 172.21.15.161:0/3860290346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa3813d560 msgr2=0x7faa380cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:42.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.779+0000 7faa3f600640 1 --2- 172.21.15.161:0/3860290346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa3813d560 0x7faa380cdb00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7faa200097c0 tx=0x7faa2001c300 comp rx=0 tx=0).stop 2024-05-22T21:24:42.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.779+0000 7faa3f600640 1 -- 172.21.15.161:0/3860290346 shutdown_connections 2024-05-22T21:24:42.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.779+0000 7faa3f600640 1 --2- 172.21.15.161:0/3860290346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa3813d560 0x7faa380cdb00 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:42.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.779+0000 7faa3f600640 1 --2- 172.21.15.161:0/3860290346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa3815e1d0 0x7faa380ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:42.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.779+0000 7faa3f600640 1 -- 172.21.15.161:0/3860290346 >> 172.21.15.161:0/3860290346 conn(0x7faa3806a0a0 msgr2=0x7faa3806a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:42.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.779+0000 7faa3f600640 1 -- 172.21.15.161:0/3860290346 shutdown_connections 2024-05-22T21:24:42.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.779+0000 7faa3f600640 1 -- 172.21.15.161:0/3860290346 wait complete. 2024-05-22T21:24:42.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.779+0000 7faa3f600640 1 Processor -- start 2024-05-22T21:24:42.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.779+0000 7faa3f600640 1 -- start start 2024-05-22T21:24:42.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa3f600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa3813cfe0 0x7faa3813eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:42.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa3f600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa3813d560 0x7faa381529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:42.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa3f600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa3815e1d0 0x7faa38152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:42.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa3f600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa38141430 con 0x7faa3813d560 2024-05-22T21:24:42.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa3f600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa381407b0 con 0x7faa3813cfe0 2024-05-22T21:24:42.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa3f600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa38140a90 con 0x7faa3815e1d0 2024-05-22T21:24:42.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa37e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa3813d560 0x7faa381529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:42.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa37e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa3813d560 0x7faa381529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59854/0 (socket says 172.21.15.161:59854) 2024-05-22T21:24:42.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa37e00640 1 -- 172.21.15.161:0/2934469921 learned_addr learned my addr 172.21.15.161:0/2934469921 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:42.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa3cc00640 1 --2- 172.21.15.161:0/2934469921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa3813cfe0 0x7faa3813eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:42.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa3d600640 1 --2- 172.21.15.161:0/2934469921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa3815e1d0 0x7faa38152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:42.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa37e00640 1 -- 172.21.15.161:0/2934469921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa3815e1d0 msgr2=0x7faa38152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:42.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa37e00640 1 --2- 172.21.15.161:0/2934469921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa3815e1d0 0x7faa38152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:42.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa37e00640 1 -- 172.21.15.161:0/2934469921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa3813cfe0 msgr2=0x7faa3813eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:42.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa37e00640 1 --2- 172.21.15.161:0/2934469921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa3813cfe0 0x7faa3813eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:42.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa37e00640 1 -- 172.21.15.161:0/2934469921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa20009420 con 0x7faa3813d560 2024-05-22T21:24:42.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa37e00640 1 --2- 172.21.15.161:0/2934469921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa3813d560 0x7faa381529c0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7faa2001cba0 tx=0x7faa200190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:42.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa34c00640 1 -- 172.21.15.161:0/2934469921 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa200b4400 con 0x7faa3813d560 2024-05-22T21:24:42.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa34c00640 1 -- 172.21.15.161:0/2934469921 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa200b4560 con 0x7faa3813d560 2024-05-22T21:24:42.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa3f600640 1 -- 172.21.15.161:0/2934469921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa38140d70 con 0x7faa3813d560 2024-05-22T21:24:42.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa34c00640 1 -- 172.21.15.161:0/2934469921 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa200b4910 con 0x7faa3813d560 2024-05-22T21:24:42.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa3f600640 1 -- 172.21.15.161:0/2934469921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa38153440 con 0x7faa3813d560 2024-05-22T21:24:42.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa3f600640 1 -- 172.21.15.161:0/2934469921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa38141570 con 0x7faa3813d560 2024-05-22T21:24:42.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.783+0000 7faa34c00640 1 -- 172.21.15.161:0/2934469921 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faa200b2040 con 0x7faa3813d560 2024-05-22T21:24:42.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.787+0000 7faa34c00640 1 --2- 172.21.15.161:0/2934469921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faa14066690 0x7faa14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:42.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.787+0000 7faa34c00640 1 -- 172.21.15.161:0/2934469921 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa2011e160 con 0x7faa3813d560 2024-05-22T21:24:42.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.787+0000 7faa3cc00640 1 --2- 172.21.15.161:0/2934469921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faa14066690 0x7faa14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:42.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.787+0000 7faa3cc00640 1 --2- 172.21.15.161:0/2934469921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faa14066690 0x7faa14068b50 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7faa28006b60 tx=0x7faa28001dd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:42.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.791+0000 7faa34c00640 1 -- 172.21.15.161:0/2934469921 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7faa38141570 con 0x7faa3813d560 2024-05-22T21:24:42.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:42.943+0000 7faa3f600640 1 -- 172.21.15.161:0/2934469921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7faa38141570 con 0x7faa3813d560 2024-05-22T21:24:43.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.143+0000 7faa34c00640 1 -- 172.21.15.161:0/2934469921 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v191) v1 ==== 181+0+0 (secure 0 0 0) 0x7faa38141570 con 0x7faa3813d560 2024-05-22T21:24:43.142 INFO:teuthology.orchestra.run.smithi161.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-05-22T21:24:43.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.143+0000 7faa3f600640 1 -- 172.21.15.161:0/2934469921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faa14066690 msgr2=0x7faa14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:43.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.143+0000 7faa3f600640 1 --2- 172.21.15.161:0/2934469921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faa14066690 0x7faa14068b50 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7faa28006b60 tx=0x7faa28001dd0 comp rx=0 tx=0).stop 2024-05-22T21:24:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.143+0000 7faa3f600640 1 -- 172.21.15.161:0/2934469921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa3813d560 msgr2=0x7faa381529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.143+0000 7faa3f600640 1 --2- 172.21.15.161:0/2934469921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa3813d560 0x7faa381529c0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7faa2001cba0 tx=0x7faa200190a0 comp rx=0 tx=0).stop 2024-05-22T21:24:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.143+0000 7faa3f600640 1 -- 172.21.15.161:0/2934469921 shutdown_connections 2024-05-22T21:24:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.143+0000 7faa3f600640 1 --2- 172.21.15.161:0/2934469921 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7faa14066690 0x7faa14068b50 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.143+0000 7faa3f600640 1 --2- 172.21.15.161:0/2934469921 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faa3815e1d0 0x7faa38152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.143+0000 7faa3f600640 1 --2- 172.21.15.161:0/2934469921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa3813d560 0x7faa381529c0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.143+0000 7faa3f600640 1 --2- 172.21.15.161:0/2934469921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa3813cfe0 0x7faa3813eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.143+0000 7faa3f600640 1 -- 172.21.15.161:0/2934469921 >> 172.21.15.161:0/2934469921 conn(0x7faa3806a0a0 msgr2=0x7faa38164310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.143+0000 7faa3f600640 1 -- 172.21.15.161:0/2934469921 shutdown_connections 2024-05-22T21:24:43.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.143+0000 7faa3f600640 1 -- 172.21.15.161:0/2934469921 wait complete. 2024-05-22T21:24:43.164 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-05-22T21:24:43.317 INFO:teuthology.orchestra.run.smithi161.stdout:{"osd_mon_report_interval":"5"} 2024-05-22T21:24:43.317 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-05-22T21:24:43.317 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2024-05-22T21:24:43.317 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:24:43.317 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:24:43.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:43.326 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2024-05-22T21:24:43.548 INFO:teuthology.orchestra.run.smithi161.stderr:16+0 records in 2024-05-22T21:24:43.548 INFO:teuthology.orchestra.run.smithi161.stderr:16+0 records out 2024-05-22T21:24:43.549 INFO:teuthology.orchestra.run.smithi161.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.164008 s, 102 MB/s 2024-05-22T21:24:43.550 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:24:43.550 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:24:43.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:43.558 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2024-05-22T21:24:43.767 INFO:teuthology.orchestra.run.smithi161.stderr:17+0 records in 2024-05-22T21:24:43.767 INFO:teuthology.orchestra.run.smithi161.stderr:17+0 records out 2024-05-22T21:24:43.767 INFO:teuthology.orchestra.run.smithi161.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.156951 s, 114 MB/s 2024-05-22T21:24:43.768 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-05-22T21:24:43.768 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:24:43.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.903+0000 7f887ee00640 1 Processor -- start 2024-05-22T21:24:43.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.903+0000 7f887ee00640 1 -- start start 2024-05-22T21:24:43.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.903+0000 7f887ee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f887815d430 0x7f88780707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:43.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.903+0000 7f887ee00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f88780ac8b0 0x7f887806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:43.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.903+0000 7f887ee00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8878071fd0 con 0x7f88780accf0 2024-05-22T21:24:43.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.903+0000 7f887ee00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8878070e80 con 0x7f88780ac8b0 2024-05-22T21:24:43.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.903+0000 7f887ee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887806c510 con 0x7f887815d430 2024-05-22T21:24:43.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.907+0000 7f887ce00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f88780accf0 0x7f88780b1ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59166/0 (socket says 172.21.15.161:59166) 2024-05-22T21:24:43.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.907+0000 7f887ce00640 1 -- 172.21.15.161:0/4267474727 learned_addr learned my addr 172.21.15.161:0/4267474727 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:43.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.907+0000 7f8877e00640 1 --2- 172.21.15.161:0/4267474727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f887815d430 0x7f88780707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:43.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.907+0000 7f8877400640 1 --2- 172.21.15.161:0/4267474727 >> v2:172.21.15.179:3300/0 conn(0x7f88780ac8b0 0x7f887806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:43.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.907+0000 7f8876a00640 1 -- 172.21.15.161:0/4267474727 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431963213 0 0) 0x7f8878071fd0 con 0x7f88780accf0 2024-05-22T21:24:43.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.907+0000 7f8876a00640 1 -- 172.21.15.161:0/4267474727 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f885c003530 con 0x7f88780accf0 2024-05-22T21:24:43.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.907+0000 7f8877e00640 1 -- 172.21.15.161:0/4267474727 >> v2:172.21.15.179:3300/0 conn(0x7f88780ac8b0 msgr2=0x7f887806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:43.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.907+0000 7f8877e00640 1 --2- 172.21.15.161:0/4267474727 >> v2:172.21.15.179:3300/0 conn(0x7f88780ac8b0 0x7f887806cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:43.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.907+0000 7f8877e00640 1 -- 172.21.15.161:0/4267474727 >> v1:172.21.15.161:6789/0 conn(0x7f88780accf0 legacy=0x7f88780b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:43.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.907+0000 7f8877e00640 1 -- 172.21.15.161:0/4267474727 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887806c7f0 con 0x7f887815d430 2024-05-22T21:24:43.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.907+0000 7f8877400640 1 --2- 172.21.15.161:0/4267474727 >> v2:172.21.15.179:3300/0 conn(0x7f88780ac8b0 0x7f887806cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:24:43.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.907+0000 7f8877e00640 1 --2- 172.21.15.161:0/4267474727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f887815d430 0x7f88780707c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f88680097c0 tx=0x7f886801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=369b18b5267ac047 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:43.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f8876a00640 1 -- 172.21.15.161:0/4267474727 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88680ab600 con 0x7f887815d430 2024-05-22T21:24:43.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f8876a00640 1 -- 172.21.15.161:0/4267474727 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88680a7050 con 0x7f887815d430 2024-05-22T21:24:43.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f8876a00640 1 -- 172.21.15.161:0/4267474727 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88680ab980 con 0x7f887815d430 2024-05-22T21:24:43.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ee00640 1 -- 172.21.15.161:0/4267474727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f887815d430 msgr2=0x7f88780707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:43.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ee00640 1 --2- 172.21.15.161:0/4267474727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f887815d430 0x7f88780707c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f88680097c0 tx=0x7f886801c3e0 comp rx=0 tx=0).stop 2024-05-22T21:24:43.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ee00640 1 -- 172.21.15.161:0/4267474727 shutdown_connections 2024-05-22T21:24:43.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ee00640 1 --2- 172.21.15.161:0/4267474727 >> v2:172.21.15.179:3300/0 conn(0x7f88780ac8b0 0x7f887806cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:43.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ee00640 1 --2- 172.21.15.161:0/4267474727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f887815d430 0x7f88780707c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:43.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ee00640 1 -- 172.21.15.161:0/4267474727 >> 172.21.15.161:0/4267474727 conn(0x7f88780b0bc0 msgr2=0x7f88780b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:43.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ee00640 1 -- 172.21.15.161:0/4267474727 shutdown_connections 2024-05-22T21:24:43.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ee00640 1 -- 172.21.15.161:0/4267474727 wait complete. 2024-05-22T21:24:43.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ee00640 1 Processor -- start 2024-05-22T21:24:43.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ee00640 1 -- start start 2024-05-22T21:24:43.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88780ac8b0 0x7f8878155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:43.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f88780accf0 0x7f887814c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:43.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ee00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f887815d430 0x7f887815e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:43.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ee00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8878155970 con 0x7f887815d430 2024-05-22T21:24:43.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88781562d0 con 0x7f88780ac8b0 2024-05-22T21:24:43.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8878155340 con 0x7f88780accf0 2024-05-22T21:24:43.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f887815d430 0x7f887815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f887815d430 0x7f887815e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59862/0 (socket says 172.21.15.161:59862) 2024-05-22T21:24:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ce00640 1 -- 172.21.15.161:0/2230231502 learned_addr learned my addr 172.21.15.161:0/2230231502 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f8877e00640 1 --2- 172.21.15.161:0/2230231502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88780ac8b0 0x7f8878155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f8877400640 1 --2- 172.21.15.161:0/2230231502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f88780accf0 0x7f887814c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ce00640 1 -- 172.21.15.161:0/2230231502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f88780accf0 msgr2=0x7f887814c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ce00640 1 --2- 172.21.15.161:0/2230231502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f88780accf0 0x7f887814c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ce00640 1 -- 172.21.15.161:0/2230231502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88780ac8b0 msgr2=0x7f8878155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ce00640 1 --2- 172.21.15.161:0/2230231502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88780ac8b0 0x7f8878155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.911+0000 7f887ce00640 1 -- 172.21.15.161:0/2230231502 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8864009420 con 0x7f887815d430 2024-05-22T21:24:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.915+0000 7f887ce00640 1 --2- 172.21.15.161:0/2230231502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f887815d430 0x7f887815e430 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f887000caa0 tx=0x7f887000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.915+0000 7f886fe00640 1 -- 172.21.15.161:0/2230231502 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887000ab40 con 0x7f887815d430 2024-05-22T21:24:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.915+0000 7f886fe00640 1 -- 172.21.15.161:0/2230231502 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887000aca0 con 0x7f887815d430 2024-05-22T21:24:43.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.915+0000 7f887ee00640 1 -- 172.21.15.161:0/2230231502 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8868009420 con 0x7f887815d430 2024-05-22T21:24:43.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.915+0000 7f886fe00640 1 -- 172.21.15.161:0/2230231502 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8870012640 con 0x7f887815d430 2024-05-22T21:24:43.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.915+0000 7f887ee00640 1 -- 172.21.15.161:0/2230231502 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887814a430 con 0x7f887815d430 2024-05-22T21:24:43.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.915+0000 7f887ee00640 1 -- 172.21.15.161:0/2230231502 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8878155ab0 con 0x7f887815d430 2024-05-22T21:24:43.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.915+0000 7f886fe00640 1 -- 172.21.15.161:0/2230231502 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8870007070 con 0x7f887815d430 2024-05-22T21:24:43.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.915+0000 7f886fe00640 1 --2- 172.21.15.161:0/2230231502 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8858066690 0x7f8858068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:43.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.915+0000 7f886fe00640 1 -- 172.21.15.161:0/2230231502 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8870083ff0 con 0x7f887815d430 2024-05-22T21:24:43.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.915+0000 7f8877e00640 1 --2- 172.21.15.161:0/2230231502 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8858066690 0x7f8858068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:43.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.919+0000 7f8877e00640 1 --2- 172.21.15.161:0/2230231502 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8858066690 0x7f8858068b50 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f88780ceae0 tx=0x7f8868008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:43.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:43.919+0000 7f886fe00640 1 -- 172.21.15.161:0/2230231502 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f88700b5cf0 con 0x7f887815d430 2024-05-22T21:24:44.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:44.075+0000 7f887ee00640 1 -- 172.21.15.161:0/2230231502 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8878067220 con 0x7f887815d430 2024-05-22T21:24:44.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:44.075+0000 7f886fe00640 1 -- 172.21.15.161:0/2230231502 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v191) v1 ==== 81+0+33594 (secure 0 0 0) 0x7f8870017090 con 0x7f887815d430 2024-05-22T21:24:44.076 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:44.076 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:24:44.077 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 191, 2024-05-22T21:24:44.077 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:24:44.077 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:24:44.077 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:24:43.137977+0000", 2024-05-22T21:24:44.077 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:24:44.077 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:24:44.077 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:24:44.077 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:24:44.077 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:24:44.077 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:24:44.077 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:24:44.077 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:24:44.078 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:24:44.078 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:44.078 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:24:44.078 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:24:44.078 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:24:44.078 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:24:44.078 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:24:44.078 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 23, 2024-05-22T21:24:44.078 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:24:44.078 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:24:44.078 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:24:44.078 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:24:44.079 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:24:44.079 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:24:44.079 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.079 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:24:44.079 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:24:44.079 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:24:44.079 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:24:44.079 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:24:44.079 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:24:44.079 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:24:44.079 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:24:44.079 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:24:44.079 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:24:44.080 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:24:44.080 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:24:44.080 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:24:44.080 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:24:44.080 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:24:44.080 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:24:44.080 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:24:44.080 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:24:44.080 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:24:44.080 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:24:44.080 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:24:44.080 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:24:44.080 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:24:44.081 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:24:44.081 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:24:44.081 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:24:44.081 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:24:44.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.081 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:24:44.081 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:24:44.081 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:24:44.081 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:24:44.081 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:24:44.081 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:24:44.081 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:24:44.081 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:24:44.081 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:24:44.082 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:24:44.082 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:24:44.082 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:24:44.082 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:24:44.082 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:24:44.082 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:24:44.082 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:24:44.082 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:24:44.082 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:24:44.082 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:24:44.082 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:24:44.082 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:24:44.082 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:24:44.083 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:24:44.083 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:24:44.083 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:24:44.083 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:24:44.083 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:24:44.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.083 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:24:44.083 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:24:44.083 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:24:44.083 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:24:44.083 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:24:44.083 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:24:44.083 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:24:44.084 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:24:44.084 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:24:44.084 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:24:44.084 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:24:44.084 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:24:44.084 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:24:44.084 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:24:44.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.084 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:24:44.084 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:24:44.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.084 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:24:44.085 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:24:44.085 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:24:44.085 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:24:44.085 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:24:44.085 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:24:44.085 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:24:44.085 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:24:44.085 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:24:44.085 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.085 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.085 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 22, 2024-05-22T21:24:44.085 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:24:44.085 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:24:31.797440+0000", 2024-05-22T21:24:44.086 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:24:44.086 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:24:44.086 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:24:44.086 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:24:44.086 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:24:44.086 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:24:44.086 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:24:44.086 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:24:44.086 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:24:44.086 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:24:44.086 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:24:44.086 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:24:44.087 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:24:44.087 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:24:44.087 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:24:44.087 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:24:44.087 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:24:44.087 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:24:44.087 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:24:44.087 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:24:44.087 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:24:44.087 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:24:44.087 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:24:44.087 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:24:44.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.088 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "189", 2024-05-22T21:24:44.088 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:24:44.088 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:24:44.088 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:24:44.088 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:24:44.088 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:24:44.088 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:24:44.088 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:24:44.088 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:24:44.088 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:24:44.088 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:24:44.088 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:24:44.088 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:24:44.089 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:24:44.089 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:24:44.089 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:24:44.089 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:24:44.089 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:24:44.089 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:24:44.089 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:24:44.089 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:24:44.089 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:24:44.089 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:24:44.089 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:24:44.089 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:24:44.089 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:24:44.090 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:24:44.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.090 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:24:44.090 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:24:44.090 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:24:44.090 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:24:44.090 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:24:44.090 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:24:44.090 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:24:44.090 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:24:44.090 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:24:44.090 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:24:44.091 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:24:44.091 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:24:44.091 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:24:44.091 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:24:44.091 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:24:44.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.091 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:24:44.091 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:24:44.091 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:24:44.091 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.092 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:24:44.092 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 2, 2024-05-22T21:24:44.092 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 2, 2024-05-22T21:24:44.092 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:24:44.092 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:24:44.092 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:24:44.092 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:24:44.092 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:24:44.092 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:24:44.092 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.092 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.092 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 23, 2024-05-22T21:24:44.093 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:24:44.093 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:24:33.098240+0000", 2024-05-22T21:24:44.093 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:24:44.093 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:24:44.093 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:24:44.093 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:24:44.093 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:24:44.093 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:24:44.093 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:24:44.093 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:24:44.093 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:24:44.093 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:24:44.093 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:24:44.094 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:24:44.094 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:24:44.094 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:24:44.094 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:24:44.094 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:24:44.094 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:24:44.094 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:24:44.094 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:24:44.094 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:24:44.094 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:24:44.094 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:24:44.094 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:24:44.095 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:24:44.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.095 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "191", 2024-05-22T21:24:44.095 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:24:44.095 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:24:44.095 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:24:44.095 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:24:44.095 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:24:44.095 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:24:44.095 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:24:44.095 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:24:44.095 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:24:44.096 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 33554432, 2024-05-22T21:24:44.096 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:24:44.096 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:24:44.096 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:24:44.096 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:24:44.096 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:24:44.096 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:24:44.096 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:24:44.096 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:24:44.096 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:24:44.096 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:24:44.096 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:24:44.097 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:24:44.097 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:24:44.097 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:24:44.097 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:24:44.097 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:24:44.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.097 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:24:44.097 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:24:44.097 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:24:44.097 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:24:44.097 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:24:44.097 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:24:44.097 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:24:44.097 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:24:44.098 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:24:44.098 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:24:44.098 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:24:44.098 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:24:44.098 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:24:44.098 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:24:44.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.098 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:24:44.098 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:24:44.098 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:24:44.098 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.099 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:24:44.099 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.3799999952316284, 2024-05-22T21:24:44.099 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.3799999952316284, 2024-05-22T21:24:44.099 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:24:44.099 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.3799999952316284, 2024-05-22T21:24:44.099 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.3799999952316284, 2024-05-22T21:24:44.099 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:24:44.099 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:24:44.099 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:24:44.099 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.099 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.099 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:44.099 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:24:44.100 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:24:44.100 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:44.100 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:44.100 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:44.100 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:44.100 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:44.100 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:44.100 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:44.100 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:44.100 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:44.100 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:44.100 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:44.101 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:44.101 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.101 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.101 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:24:44.101 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:44.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.101 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.101 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.101 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:24:44.101 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:44.101 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.101 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.102 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:44.102 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.102 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.102 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:24:44.102 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:44.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.102 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.102 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.102 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:24:44.102 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:44.102 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.103 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.103 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:44.103 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.103 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.103 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:24:44.103 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:44.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.103 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.103 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.103 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:24:44.103 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:44.103 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.104 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.104 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:44.104 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.104 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.104 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:24:44.104 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:44.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.104 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.104 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.104 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:24:44.104 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:44.105 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.105 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.105 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:24:44.105 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:24:44.105 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:24:44.105 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:24:44.105 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:44.105 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:44.105 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:44.105 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.105 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.106 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:24:44.106 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:44.106 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:44.106 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:44.106 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:44.106 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:44.106 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:44.106 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:44.106 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:44.106 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:44.106 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:44.106 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:44.106 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:44.107 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.107 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.107 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:24:44.107 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:44.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.107 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.107 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.107 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:24:44.107 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:44.107 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.107 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.108 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:44.108 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.108 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.108 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:24:44.108 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:44.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.108 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.108 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.108 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:24:44.108 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:44.108 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.108 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.109 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:44.109 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.109 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.109 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:24:44.109 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:44.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.109 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.109 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.109 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:24:44.109 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:44.109 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.110 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.110 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:44.110 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.110 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.110 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:24:44.110 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:44.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.110 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.110 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.110 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:24:44.110 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:44.111 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.111 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.111 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:24:44.111 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:24:44.111 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:24:44.111 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:24:44.111 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:44.111 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:44.111 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:44.111 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.111 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.111 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:24:44.112 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:44.112 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:44.112 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:44.112 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:44.112 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:44.112 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:44.112 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:44.112 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:44.112 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:44.112 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:44.112 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:44.112 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:44.112 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.113 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.113 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:24:44.113 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:44.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.113 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.113 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.113 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:24:44.113 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:44.113 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.113 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.113 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:44.114 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.114 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.114 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:24:44.114 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:44.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.114 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.114 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.114 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:24:44.114 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:44.114 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.114 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.115 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:44.115 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.115 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.115 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:24:44.115 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:44.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.115 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.115 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.115 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:24:44.115 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:44.115 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.116 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.116 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:44.116 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.116 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.116 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:24:44.116 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:44.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.116 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.116 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.116 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:24:44.116 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:44.117 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.117 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.117 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:24:44.117 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:24:44.117 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:24:44.117 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:24:44.117 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:44.117 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:44.117 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:44.117 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.117 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.118 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:24:44.118 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:44.118 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:44.118 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:44.118 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:44.118 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:44.118 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:44.118 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:44.118 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:44.118 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:44.118 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:44.118 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:44.119 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:44.119 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.119 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.119 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:24:44.119 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:44.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.119 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.119 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.119 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:24:44.119 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:44.119 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.119 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.120 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:44.120 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.120 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.120 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:24:44.120 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:44.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.120 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.120 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.120 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:24:44.120 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:44.120 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.120 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.121 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:44.121 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.121 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.121 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:24:44.121 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:44.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.121 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.121 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.121 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:24:44.121 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:44.121 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.121 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.122 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:44.122 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.122 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.122 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:24:44.122 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:44.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.122 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.122 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.122 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:24:44.122 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:44.123 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.123 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.123 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:24:44.123 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:24:44.123 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:24:44.123 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:24:44.123 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:44.123 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:44.123 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:44.123 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.123 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.124 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:24:44.124 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:44.124 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:44.124 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:44.124 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:44.124 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:44.124 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:44.124 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:44.124 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:44.124 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:44.124 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:44.124 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:44.124 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:44.125 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.125 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.125 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:24:44.125 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:44.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.125 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.125 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.125 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:24:44.125 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:44.126 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.126 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.126 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:44.126 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.126 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.126 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:24:44.126 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:44.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.126 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.126 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.127 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:24:44.127 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:44.127 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.127 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.127 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:44.127 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.127 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.127 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:24:44.127 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:44.127 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.127 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.128 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.128 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:24:44.128 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:44.128 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.128 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.128 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:44.128 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.128 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.128 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:24:44.128 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:44.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.128 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.129 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.129 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:24:44.129 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:44.129 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.129 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.129 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:24:44.129 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:24:44.129 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:24:44.129 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:24:44.129 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:44.129 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:44.129 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:44.129 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.130 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.130 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:24:44.130 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:44.130 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:44.130 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:44.130 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:44.130 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:44.130 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:44.130 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:44.130 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:44.130 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:44.130 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:44.131 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:44.131 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:44.131 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.131 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.131 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:24:44.131 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:44.131 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.131 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.131 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.131 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:24:44.131 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:44.131 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.131 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.132 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:44.132 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.132 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.132 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:24:44.132 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:44.132 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.132 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.132 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.132 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:24:44.132 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:44.132 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.133 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.133 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:44.133 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.133 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.133 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:24:44.133 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:44.133 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.133 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.133 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.133 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:24:44.133 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:44.133 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.134 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.134 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:44.134 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.134 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.134 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:24:44.134 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:44.134 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.134 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.134 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.134 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:24:44.135 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:44.135 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.135 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.135 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:24:44.135 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:24:44.135 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:24:44.135 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:24:44.135 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:44.135 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:44.135 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:44.136 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.136 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.136 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:24:44.136 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:44.136 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:44.136 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:44.136 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:44.136 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:44.136 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:44.136 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:44.136 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:44.136 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:44.137 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:44.137 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:44.137 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:44.137 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.137 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.137 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:24:44.137 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:44.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.137 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.137 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.137 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:24:44.137 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:44.137 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.138 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.138 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:44.138 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.138 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.138 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:24:44.138 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:44.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.138 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.138 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.138 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:24:44.138 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:44.139 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.139 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.139 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:44.139 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.139 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.139 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:24:44.139 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:44.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.139 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.139 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.139 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:24:44.140 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:44.140 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.140 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.140 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:44.140 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.140 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.140 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:24:44.140 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:44.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.140 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.140 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.140 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:24:44.141 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:44.141 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.141 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.141 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:24:44.141 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:24:44.141 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:24:44.141 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:24:44.141 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:44.141 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:44.141 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:44.141 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.142 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.142 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:24:44.142 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:44.142 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:44.142 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:44.142 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:44.142 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:44.142 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:44.142 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:44.142 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:44.142 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:44.142 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:44.143 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:44.143 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:44.143 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.143 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.143 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:24:44.143 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:44.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.143 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.143 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.143 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:24:44.143 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:44.143 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.144 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.144 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:44.144 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.144 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.144 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:24:44.144 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:44.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.144 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.144 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.144 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:24:44.144 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:44.145 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.145 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.145 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:44.145 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.145 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.145 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:24:44.145 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:44.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.145 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.145 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.145 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:24:44.146 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:44.146 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.146 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.146 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:44.146 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:44.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.146 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:44.146 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:24:44.146 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:44.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.146 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.146 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:44.147 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:24:44.147 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:44.147 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.147 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.147 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:24:44.147 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:24:44.147 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:24:44.147 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:24:44.147 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:44.147 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:44.147 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:44.147 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:44.147 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.148 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:44.148 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:24:44.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.148 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:24:44.148 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:44.148 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:44.148 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:44.148 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:44.148 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:44.148 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:24:44.148 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:44.148 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.148 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.149 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:24:44.149 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:44.149 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:44.149 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:44.149 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:44.149 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:44.149 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:24:44.149 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:44.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.149 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.149 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:24:44.149 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:44.149 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:44.150 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:44.150 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:44.150 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:44.150 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:24:44.150 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:44.150 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.150 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.150 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:24:44.150 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:44.150 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:44.150 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:44.150 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:44.150 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:44.151 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:24:44.151 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:44.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.151 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.151 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:24:44.151 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:44.151 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:44.151 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:44.151 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:44.151 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:44.151 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:24:44.151 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:44.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.152 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:24:44.152 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:44.152 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:44.152 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:44.152 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:44.152 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:44.152 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:24:44.152 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:44.152 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.152 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.152 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:24:44.152 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:44.153 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:44.153 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:44.153 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:44.153 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:44.153 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:24:44.153 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:44.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.153 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:44.153 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:24:44.153 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:44.153 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:44.153 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:44.153 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:44.153 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:44.154 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:24:44.154 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:44.154 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.154 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:44.154 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:24:44.154 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:24:44.154 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:24:44.154 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:24:44.154 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:24:44.154 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:24:44.154 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:24:44.154 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:24:44.154 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:24:44.155 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:24:44.155 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:24:44.155 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:24:44.155 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:24:44.155 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:24:44.155 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.155 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:44.155 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:24:44.155 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:24:44.155 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:24:44.155 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:24:44.155 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:24:44.155 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:24:44.156 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:24:44.156 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:24:44.156 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:24:44.156 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:24:44.156 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:24:44.156 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:44.156 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:24:44.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:44.079+0000 7f887ee00640 1 -- 172.21.15.161:0/2230231502 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8858066690 msgr2=0x7f8858068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:44.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:44.079+0000 7f887ee00640 1 --2- 172.21.15.161:0/2230231502 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8858066690 0x7f8858068b50 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f88780ceae0 tx=0x7f8868008040 comp rx=0 tx=0).stop 2024-05-22T21:24:44.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:44.079+0000 7f887ee00640 1 -- 172.21.15.161:0/2230231502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f887815d430 msgr2=0x7f887815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:44.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:44.079+0000 7f887ee00640 1 --2- 172.21.15.161:0/2230231502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f887815d430 0x7f887815e430 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f887000caa0 tx=0x7f887000cf60 comp rx=0 tx=0).stop 2024-05-22T21:24:44.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:44.079+0000 7f887ee00640 1 -- 172.21.15.161:0/2230231502 shutdown_connections 2024-05-22T21:24:44.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:44.079+0000 7f887ee00640 1 --2- 172.21.15.161:0/2230231502 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8858066690 0x7f8858068b50 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:44.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:44.079+0000 7f887ee00640 1 --2- 172.21.15.161:0/2230231502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f887815d430 0x7f887815e430 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:44.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:44.079+0000 7f887ee00640 1 --2- 172.21.15.161:0/2230231502 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f88780accf0 0x7f887814c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:44.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:44.079+0000 7f887ee00640 1 --2- 172.21.15.161:0/2230231502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88780ac8b0 0x7f8878155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:44.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:44.079+0000 7f887ee00640 1 -- 172.21.15.161:0/2230231502 >> 172.21.15.161:0/2230231502 conn(0x7f88780b0bc0 msgr2=0x7f88780af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:44.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:44.079+0000 7f887ee00640 1 -- 172.21.15.161:0/2230231502 shutdown_connections 2024-05-22T21:24:44.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:44.079+0000 7f887ee00640 1 -- 172.21.15.161:0/2230231502 wait complete. 2024-05-22T21:24:44.159 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-05-22T21:24:49.160 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:24:49.306 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.306+0000 7f9630c00640 1 Processor -- start 2024-05-22T21:24:49.307 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.306+0000 7f9630c00640 1 -- start start 2024-05-22T21:24:49.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.306+0000 7f9630c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f962c071290 0x7f962c0cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:49.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.306+0000 7f9630c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f962c0ce1c0 0x7f962c159d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:49.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.306+0000 7f9630c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f962c0cb1e0 con 0x7f962c071670 2024-05-22T21:24:49.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.306+0000 7f9630c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f962c0cb4c0 con 0x7f962c0ce1c0 2024-05-22T21:24:49.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.306+0000 7f9630c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f962c0cb7a0 con 0x7f962c071290 2024-05-22T21:24:49.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.306+0000 7f9621400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f962c0ce1c0 0x7f962c159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:49.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.306+0000 7f962a800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f962c071670 0x7f962c15de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:45560/0 (socket says 172.21.15.161:45560) 2024-05-22T21:24:49.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.306+0000 7f962a800640 1 -- 172.21.15.161:0/1930371993 learned_addr learned my addr 172.21.15.161:0/1930371993 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:49.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.306+0000 7f9629e00640 1 --2- 172.21.15.161:0/1930371993 >> v2:172.21.15.179:3300/0 conn(0x7f962c071290 0x7f962c0cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:49.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.306+0000 7f9621400640 1 -- 172.21.15.161:0/1930371993 >> v2:172.21.15.179:3300/0 conn(0x7f962c071290 msgr2=0x7f962c0cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:49.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.306+0000 7f9621400640 1 --2- 172.21.15.161:0/1930371993 >> v2:172.21.15.179:3300/0 conn(0x7f962c071290 0x7f962c0cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:49.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.306+0000 7f9621400640 1 -- 172.21.15.161:0/1930371993 >> v1:172.21.15.161:6789/0 conn(0x7f962c071670 legacy=0x7f962c15de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:49.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.306+0000 7f9621400640 1 -- 172.21.15.161:0/1930371993 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c0cba80 con 0x7f962c0ce1c0 2024-05-22T21:24:49.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.310+0000 7f9621400640 1 --2- 172.21.15.161:0/1930371993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962c0ce1c0 0x7f962c159d00 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f962c04e7b0 tx=0x7f961401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992eedfacb5aa43a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:49.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.310+0000 7f9629400640 1 -- 172.21.15.161:0/1930371993 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96140ac640 con 0x7f962c0ce1c0 2024-05-22T21:24:49.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.310+0000 7f9629400640 1 -- 172.21.15.161:0/1930371993 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96140ac7a0 con 0x7f962c0ce1c0 2024-05-22T21:24:49.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.310+0000 7f9629400640 1 -- 172.21.15.161:0/1930371993 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96140acb30 con 0x7f962c0ce1c0 2024-05-22T21:24:49.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.310+0000 7f9630c00640 1 -- 172.21.15.161:0/1930371993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962c0ce1c0 msgr2=0x7f962c159d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:49.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.310+0000 7f9630c00640 1 --2- 172.21.15.161:0/1930371993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962c0ce1c0 0x7f962c159d00 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f962c04e7b0 tx=0x7f961401c480 comp rx=0 tx=0).stop 2024-05-22T21:24:49.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.310+0000 7f9630c00640 1 -- 172.21.15.161:0/1930371993 shutdown_connections 2024-05-22T21:24:49.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.310+0000 7f9630c00640 1 --2- 172.21.15.161:0/1930371993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962c0ce1c0 0x7f962c159d00 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:49.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.310+0000 7f9630c00640 1 --2- 172.21.15.161:0/1930371993 >> v2:172.21.15.179:3300/0 conn(0x7f962c071290 0x7f962c0cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:49.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.310+0000 7f9630c00640 1 -- 172.21.15.161:0/1930371993 >> 172.21.15.161:0/1930371993 conn(0x7f962c0d00b0 msgr2=0x7f962c0d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:49.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.310+0000 7f9630c00640 1 -- 172.21.15.161:0/1930371993 shutdown_connections 2024-05-22T21:24:49.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.310+0000 7f9630c00640 1 -- 172.21.15.161:0/1930371993 wait complete. 2024-05-22T21:24:49.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9630c00640 1 Processor -- start 2024-05-22T21:24:49.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9630c00640 1 -- start start 2024-05-22T21:24:49.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9630c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962c071290 0x7f962c158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:49.315 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9630c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f962c071670 0x7f962c0c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:49.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9630c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f962c0ce1c0 0x7f962c0ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:49.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9630c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0c6b70 con 0x7f962c071290 2024-05-22T21:24:49.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9630c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0c9460 con 0x7f962c0ce1c0 2024-05-22T21:24:49.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9630c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0adff0 con 0x7f962c071670 2024-05-22T21:24:49.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9629e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962c071290 0x7f962c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:49.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9629e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962c071290 0x7f962c158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48302/0 (socket says 172.21.15.161:48302) 2024-05-22T21:24:49.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9629e00640 1 -- 172.21.15.161:0/319027854 learned_addr learned my addr 172.21.15.161:0/319027854 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:49.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9629e00640 1 -- 172.21.15.161:0/319027854 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f962c071670 msgr2=0x7f962c0c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:49.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9629e00640 1 --2- 172.21.15.161:0/319027854 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f962c071670 0x7f962c0c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:49.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9629e00640 1 -- 172.21.15.161:0/319027854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f962c0ce1c0 msgr2=0x7f962c0ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:49.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9629e00640 1 --2- 172.21.15.161:0/319027854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f962c0ce1c0 0x7f962c0ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:49.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9629e00640 1 -- 172.21.15.161:0/319027854 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9614009420 con 0x7f962c071290 2024-05-22T21:24:49.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9629e00640 1 --2- 172.21.15.161:0/319027854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962c071290 0x7f962c158150 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f961c00e930 tx=0x7f961c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:49.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9622a00640 1 -- 172.21.15.161:0/319027854 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c00ca00 con 0x7f962c071290 2024-05-22T21:24:49.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9622a00640 1 -- 172.21.15.161:0/319027854 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961c00cb60 con 0x7f962c071290 2024-05-22T21:24:49.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9622a00640 1 -- 172.21.15.161:0/319027854 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c013470 con 0x7f962c071290 2024-05-22T21:24:49.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9630c00640 1 -- 172.21.15.161:0/319027854 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f962c0c9700 con 0x7f962c071290 2024-05-22T21:24:49.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9630c00640 1 -- 172.21.15.161:0/319027854 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962c0c5070 con 0x7f962c071290 2024-05-22T21:24:49.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.314+0000 7f9630c00640 1 -- 172.21.15.161:0/319027854 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f962c067220 con 0x7f962c071290 2024-05-22T21:24:49.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.318+0000 7f9622a00640 1 -- 172.21.15.161:0/319027854 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f961c0135d0 con 0x7f962c071290 2024-05-22T21:24:49.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.318+0000 7f9622a00640 1 --2- 172.21.15.161:0/319027854 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f96080666c0 0x7f9608068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:49.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.318+0000 7f9622a00640 1 -- 172.21.15.161:0/319027854 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f961c084470 con 0x7f962c071290 2024-05-22T21:24:49.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.318+0000 7f9621400640 1 --2- 172.21.15.161:0/319027854 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f96080666c0 0x7f9608068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:49.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.318+0000 7f9621400640 1 --2- 172.21.15.161:0/319027854 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f96080666c0 0x7f9608068b80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f96140ae000 tx=0x7f9614008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:49.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.322+0000 7f9622a00640 1 -- 172.21.15.161:0/319027854 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f961c051800 con 0x7f962c071290 2024-05-22T21:24:49.477 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.478+0000 7f9630c00640 1 -- 172.21.15.161:0/319027854 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f962c0af1e0 con 0x7f962c071290 2024-05-22T21:24:49.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.478+0000 7f9622a00640 1 -- 172.21.15.161:0/319027854 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v191) v1 ==== 81+0+33594 (secure 0 0 0) 0x7f961c054e20 con 0x7f962c071290 2024-05-22T21:24:49.479 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:49.479 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:24:49.479 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 191, 2024-05-22T21:24:49.480 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:24:49.480 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:24:49.480 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:24:43.137977+0000", 2024-05-22T21:24:49.480 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:24:49.480 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:24:49.480 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:24:49.480 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:24:49.480 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:24:49.480 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:24:49.480 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:24:49.480 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:24:49.481 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:24:49.481 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:49.481 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:24:49.481 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:24:49.481 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:24:49.481 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:24:49.481 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:24:49.481 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 23, 2024-05-22T21:24:49.481 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:24:49.481 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:24:49.481 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:24:49.481 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:24:49.481 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:24:49.482 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:24:49.482 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.482 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:24:49.482 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:24:49.482 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:24:49.482 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:24:49.482 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:24:49.482 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:24:49.482 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:24:49.482 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:24:49.482 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:24:49.482 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:24:49.483 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:24:49.483 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:24:49.483 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:24:49.483 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:24:49.483 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:24:49.483 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:24:49.483 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:24:49.483 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:24:49.483 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:24:49.483 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:24:49.483 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:24:49.484 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:24:49.484 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:24:49.484 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:24:49.484 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:24:49.484 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:24:49.484 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:24:49.484 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.484 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:24:49.484 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:24:49.485 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:24:49.485 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:24:49.485 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:24:49.485 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:24:49.485 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:24:49.485 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:24:49.485 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:24:49.485 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:24:49.485 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:24:49.485 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:24:49.485 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:24:49.485 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:24:49.485 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:24:49.486 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:24:49.486 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:24:49.486 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:24:49.486 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:24:49.486 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:24:49.486 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:24:49.486 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:24:49.486 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:24:49.486 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:24:49.487 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:24:49.487 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:24:49.487 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:24:49.487 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.487 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:24:49.487 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:24:49.487 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:24:49.487 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:24:49.487 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:24:49.487 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:24:49.487 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:24:49.487 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:24:49.487 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:24:49.488 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:24:49.488 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:24:49.488 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:24:49.488 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:24:49.488 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:24:49.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.488 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:24:49.488 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:24:49.488 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.488 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:24:49.488 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:24:49.488 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:24:49.488 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:24:49.489 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:24:49.489 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:24:49.489 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:24:49.489 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:24:49.489 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:24:49.489 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.489 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.489 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.489 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 22, 2024-05-22T21:24:49.489 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:24:49.489 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:24:31.797440+0000", 2024-05-22T21:24:49.489 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:24:49.489 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:24:49.490 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:24:49.490 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:24:49.490 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:24:49.490 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:24:49.490 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:24:49.490 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:24:49.490 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:24:49.490 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:24:49.490 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:24:49.490 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:24:49.490 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:24:49.490 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:24:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:24:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:24:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:24:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:24:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:24:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:24:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:24:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:24:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:24:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:24:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "189", 2024-05-22T21:24:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:24:49.492 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:24:49.492 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:24:49.492 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:24:49.492 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:24:49.492 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:24:49.492 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:24:49.492 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:24:49.492 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:24:49.492 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:24:49.492 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:24:49.492 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:24:49.492 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:24:49.492 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:24:49.493 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:24:49.493 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:24:49.493 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:24:49.493 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:24:49.493 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:24:49.493 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:24:49.493 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:24:49.493 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:24:49.493 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:24:49.493 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:24:49.493 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:24:49.493 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:24:49.493 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.493 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:24:49.494 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:24:49.494 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:24:49.494 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:24:49.494 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:24:49.494 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:24:49.494 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:24:49.494 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:24:49.494 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:24:49.494 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:24:49.494 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:24:49.494 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:24:49.494 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:24:49.495 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:24:49.495 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:24:49.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.495 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:24:49.495 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:24:49.495 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:24:49.495 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.495 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.495 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:24:49.495 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 2, 2024-05-22T21:24:49.495 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 2, 2024-05-22T21:24:49.495 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:24:49.496 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:24:49.496 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:24:49.496 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:24:49.496 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:24:49.496 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:24:49.496 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.496 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.496 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.496 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 23, 2024-05-22T21:24:49.496 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:24:49.496 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:24:33.098240+0000", 2024-05-22T21:24:49.496 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:24:49.496 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:24:49.497 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:24:49.497 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:24:49.497 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:24:49.497 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:24:49.497 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:24:49.497 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:24:49.497 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:24:49.497 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:24:49.497 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:24:49.497 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:24:49.497 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:24:49.497 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:24:49.497 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:24:49.497 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:24:49.498 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:24:49.498 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:24:49.498 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:24:49.498 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:24:49.498 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:24:49.498 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:24:49.498 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:24:49.498 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:24:49.498 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.498 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "191", 2024-05-22T21:24:49.498 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:24:49.498 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:24:49.498 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:24:49.499 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:24:49.499 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:24:49.499 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:24:49.499 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:24:49.499 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:24:49.499 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:24:49.499 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 33554432, 2024-05-22T21:24:49.499 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:24:49.499 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:24:49.499 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:24:49.499 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:24:49.499 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:24:49.500 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:24:49.500 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:24:49.500 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:24:49.500 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:24:49.500 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:24:49.500 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:24:49.500 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:24:49.500 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:24:49.500 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:24:49.500 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:24:49.500 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:24:49.500 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.500 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:24:49.501 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:24:49.501 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:24:49.501 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:24:49.501 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:24:49.501 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:24:49.501 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:24:49.501 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:24:49.501 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:24:49.501 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:24:49.501 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:24:49.501 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:24:49.501 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:24:49.501 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:24:49.501 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.501 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:24:49.502 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:24:49.502 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:24:49.502 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.502 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.502 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:24:49.502 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.3799999952316284, 2024-05-22T21:24:49.502 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.3799999952316284, 2024-05-22T21:24:49.502 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:24:49.502 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.3799999952316284, 2024-05-22T21:24:49.502 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.3799999952316284, 2024-05-22T21:24:49.502 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:24:49.502 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:24:49.502 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:24:49.503 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.503 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.503 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:49.503 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:24:49.503 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.503 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:24:49.503 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:49.503 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:49.503 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:49.503 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:49.503 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:49.503 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:49.503 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:49.503 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:49.504 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:49.504 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:49.504 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:49.504 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:49.504 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.504 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.504 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.504 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:24:49.504 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:49.504 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.504 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.504 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.504 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:24:49.504 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:49.505 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.505 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.505 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.505 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:49.505 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.505 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.505 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.505 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:24:49.505 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:49.505 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.505 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.505 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.505 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:24:49.505 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:49.506 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.506 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.506 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.506 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:49.506 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.506 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.506 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.506 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:24:49.506 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:49.506 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.506 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.506 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.507 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:24:49.507 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:49.507 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.507 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.507 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.507 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:49.507 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.507 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.507 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.507 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:24:49.507 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:49.507 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.507 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.508 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.508 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:24:49.508 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:49.508 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.508 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.508 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.508 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:24:49.508 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:24:49.508 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:24:49.508 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:24:49.509 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:49.509 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:49.509 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:49.509 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.509 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.509 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.509 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:24:49.509 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:49.509 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:49.509 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:49.509 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:49.509 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:49.510 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:49.510 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:49.510 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:49.510 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:49.510 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:49.510 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:49.510 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:49.510 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.510 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.510 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.510 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:24:49.510 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:49.511 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.511 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.511 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.511 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:24:49.511 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:49.511 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.511 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.511 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.511 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:49.511 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.511 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.511 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.511 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:24:49.512 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:49.512 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.512 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.512 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.512 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:24:49.512 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:49.512 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.512 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.512 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.512 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:49.512 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.512 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.512 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.513 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:24:49.513 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:49.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.513 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.513 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.513 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:24:49.513 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:49.513 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.513 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.513 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.513 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:49.513 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.513 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.514 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.514 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:24:49.514 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:49.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.514 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.514 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.514 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:24:49.514 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:49.514 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.514 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.514 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.514 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:24:49.514 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:24:49.515 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:24:49.515 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:24:49.515 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:49.515 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:49.515 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:49.515 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.515 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.515 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.515 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:24:49.515 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:49.515 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:49.515 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:49.515 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:49.516 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:49.516 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:49.516 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:49.516 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:49.516 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:49.516 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:49.516 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:49.516 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:49.516 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.516 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.516 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.516 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:24:49.517 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:49.517 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.517 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.517 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.517 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:24:49.517 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:49.517 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.517 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.517 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.517 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:49.517 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.517 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.517 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.517 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:24:49.518 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:49.518 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.518 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.518 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.518 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:24:49.518 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:49.518 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.518 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.518 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.518 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:49.518 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.518 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.519 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.519 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:24:49.519 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:49.519 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.519 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.519 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.519 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:24:49.519 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:49.519 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.519 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.519 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.519 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:49.520 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.520 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.520 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.520 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:24:49.520 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:49.520 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.520 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.520 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.520 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:24:49.520 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:49.520 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.520 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.521 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:24:49.521 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:24:49.521 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:24:49.521 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:24:49.521 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:49.521 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:49.521 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:49.521 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.521 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.521 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.521 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:24:49.522 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:49.522 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:49.522 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:49.522 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:49.522 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:49.522 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:49.522 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:49.522 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:49.522 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:49.522 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:49.522 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:49.522 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:49.522 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.523 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.523 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.523 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:24:49.523 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:49.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.523 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.523 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.523 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:24:49.523 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:49.523 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.523 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.523 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.523 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:49.523 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.524 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.524 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.524 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:24:49.524 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:49.524 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.524 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.524 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.524 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:24:49.524 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:49.524 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.524 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.524 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.524 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:49.525 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.525 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.525 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:24:49.525 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:49.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.525 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.525 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.526 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:24:49.526 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:49.526 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.526 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.526 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:49.526 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.526 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.527 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.527 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:24:49.527 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:49.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.527 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.527 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.527 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:24:49.527 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:49.527 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.527 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.527 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.527 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:24:49.528 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:24:49.528 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:24:49.528 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:24:49.528 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:49.528 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:49.528 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:49.528 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.528 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.528 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:24:49.528 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:49.528 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:49.528 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:49.529 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:49.529 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:49.529 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:49.529 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:49.529 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:49.529 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:49.529 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:49.529 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:49.529 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:49.529 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.529 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.529 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.529 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:24:49.530 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:49.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.530 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.530 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:24:49.530 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:49.530 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.530 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.530 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.530 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:49.530 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.530 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:24:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:24:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:49.531 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:24:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:24:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.532 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:49.533 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.533 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.533 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:24:49.533 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:49.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.533 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.533 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.533 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:24:49.533 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:49.533 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.533 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.534 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:24:49.534 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:24:49.534 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:24:49.534 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:24:49.534 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:49.534 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:49.534 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:49.534 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.534 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.534 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:24:49.534 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:49.535 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:49.535 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:49.535 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:49.535 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:49.535 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:49.535 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:49.535 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:49.535 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:49.535 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:49.535 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:49.535 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:49.535 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.535 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.536 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.536 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:24:49.536 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:49.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.536 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.536 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.536 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:24:49.536 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:49.536 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.536 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.536 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:49.536 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.537 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.537 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:24:49.537 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:49.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.537 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.537 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.537 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:24:49.537 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:49.537 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.537 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.537 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.537 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:49.538 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.538 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.538 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:24:49.538 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:49.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.538 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.538 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.538 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:24:49.538 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:49.538 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.538 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.539 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:49.539 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.539 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.539 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.539 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:24:49.539 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:49.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.539 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.539 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.539 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:24:49.539 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:49.539 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.539 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.540 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:24:49.540 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:24:49.540 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:24:49.540 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:24:49.540 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:49.540 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:49.540 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:49.540 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.540 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.540 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:24:49.540 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:49.540 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:49.541 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:49.541 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:49.541 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:49.541 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:49.541 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:49.541 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:49.541 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:49.541 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:49.541 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:49.541 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:49.541 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.541 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.541 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.542 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:24:49.542 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:49.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.542 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.542 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:24:49.542 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:49.542 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.542 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.542 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:49.542 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.542 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.543 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.543 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:24:49.543 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:49.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.543 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.543 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.543 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:24:49.543 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:49.543 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.543 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.543 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.543 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:49.543 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.544 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.544 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:24:49.544 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:49.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.544 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.544 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.544 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:24:49.544 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:49.544 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.544 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.544 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:49.545 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.545 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.545 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:24:49.545 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:49.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.545 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.545 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.545 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:24:49.545 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:49.545 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.545 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.545 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:24:49.546 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:24:49.546 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:24:49.546 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:24:49.546 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:49.546 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:49.546 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:49.546 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.546 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.546 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.546 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:24:49.546 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:49.547 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:49.547 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:49.547 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:49.547 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:49.547 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:49.547 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:49.547 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:49.547 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:49.547 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:49.547 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:49.547 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:49.547 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.547 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.548 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.548 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:24:49.548 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:49.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.548 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.548 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.548 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:24:49.548 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:49.548 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.548 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.548 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:49.548 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.549 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.549 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:24:49.549 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:49.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.549 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.549 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.549 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:24:49.549 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:49.549 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.549 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.549 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.550 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:49.550 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.550 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.550 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:24:49.550 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:49.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.550 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.550 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.550 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:24:49.550 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:49.550 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.551 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.551 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:49.551 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:49.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.551 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:49.551 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:24:49.551 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:49.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.551 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.551 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:49.551 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:24:49.551 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:49.552 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.552 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.552 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:24:49.552 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:24:49.552 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:24:49.552 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:24:49.552 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:49.552 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:49.552 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:49.552 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:49.552 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.552 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:49.553 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:24:49.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.553 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:24:49.553 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:49.553 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:49.553 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:49.553 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:49.553 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:49.553 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:24:49.553 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:49.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.553 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.553 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:24:49.554 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:49.554 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:49.554 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:49.554 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:49.554 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:49.554 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:24:49.554 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:49.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.554 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.554 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:24:49.554 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:49.554 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:49.555 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:49.555 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:49.555 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:49.555 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:24:49.555 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:49.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.555 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.555 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:24:49.555 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:49.555 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:49.555 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:49.555 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:49.556 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:49.556 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:24:49.556 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:49.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.556 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.556 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:24:49.556 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:49.556 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:49.556 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:49.556 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:49.556 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:49.556 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:24:49.556 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:49.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.557 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:24:49.557 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:49.557 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:49.557 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:49.557 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:49.557 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:49.557 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:24:49.557 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:49.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.557 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.558 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:24:49.558 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:49.558 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:49.558 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:49.558 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:49.558 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:49.558 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:24:49.558 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:49.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.558 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:49.558 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:24:49.558 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:49.559 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:49.559 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:49.559 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:49.559 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:49.559 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:24:49.559 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:49.559 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.559 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:49.559 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:24:49.559 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:24:49.559 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:24:49.559 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:24:49.559 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:24:49.560 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:24:49.560 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:24:49.560 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:24:49.560 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:24:49.560 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:24:49.560 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:24:49.560 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:24:49.560 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:24:49.560 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:24:49.560 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:49.560 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:24:49.560 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:24:49.561 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:24:49.561 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:24:49.561 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:24:49.561 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:24:49.561 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:24:49.561 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:24:49.561 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:24:49.561 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:24:49.561 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:24:49.561 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:49.561 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:24:49.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.478+0000 7f9630c00640 1 -- 172.21.15.161:0/319027854 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f96080666c0 msgr2=0x7f9608068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:49.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.478+0000 7f9630c00640 1 --2- 172.21.15.161:0/319027854 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f96080666c0 0x7f9608068b80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f96140ae000 tx=0x7f9614008040 comp rx=0 tx=0).stop 2024-05-22T21:24:49.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.478+0000 7f9630c00640 1 -- 172.21.15.161:0/319027854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962c071290 msgr2=0x7f962c158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:49.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.478+0000 7f9630c00640 1 --2- 172.21.15.161:0/319027854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962c071290 0x7f962c158150 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f961c00e930 tx=0x7f961c00edf0 comp rx=0 tx=0).stop 2024-05-22T21:24:49.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.478+0000 7f9630c00640 1 -- 172.21.15.161:0/319027854 shutdown_connections 2024-05-22T21:24:49.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.478+0000 7f9630c00640 1 --2- 172.21.15.161:0/319027854 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f96080666c0 0x7f9608068b80 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:49.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.478+0000 7f9630c00640 1 --2- 172.21.15.161:0/319027854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f962c0ce1c0 0x7f962c0ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:49.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.478+0000 7f9630c00640 1 --2- 172.21.15.161:0/319027854 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f962c071670 0x7f962c0c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:49.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.478+0000 7f9630c00640 1 --2- 172.21.15.161:0/319027854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f962c071290 0x7f962c158150 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:49.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.478+0000 7f9630c00640 1 -- 172.21.15.161:0/319027854 >> 172.21.15.161:0/319027854 conn(0x7f962c0d00b0 msgr2=0x7f962c15db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:49.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.478+0000 7f9630c00640 1 -- 172.21.15.161:0/319027854 shutdown_connections 2024-05-22T21:24:49.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:49.478+0000 7f9630c00640 1 -- 172.21.15.161:0/319027854 wait complete. 2024-05-22T21:24:49.565 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-05-22T21:24:54.566 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:24:54.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.714+0000 7f323ee00640 1 Processor -- start 2024-05-22T21:24:54.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.714+0000 7f323ee00640 1 -- start start 2024-05-22T21:24:54.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.714+0000 7f323ee00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f323815d430 0x7f32380707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:54.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.714+0000 7f323ee00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f32380accf0 0x7f3238160e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:54.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.714+0000 7f323ee00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3238071fd0 con 0x7f32380ac8b0 2024-05-22T21:24:54.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.714+0000 7f323ee00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3238070e80 con 0x7f323815d430 2024-05-22T21:24:54.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.714+0000 7f323ee00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323806c510 con 0x7f32380accf0 2024-05-22T21:24:54.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.714+0000 7f3237e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f323815d430 0x7f32380707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:54.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.714+0000 7f3237e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f323815d430 0x7f32380707c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48310/0 (socket says 172.21.15.161:48310) 2024-05-22T21:24:54.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.714+0000 7f3237e00640 1 -- 172.21.15.161:0/3625996684 learned_addr learned my addr 172.21.15.161:0/3625996684 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:54.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.714+0000 7f323ce00640 1 --2- 172.21.15.161:0/3625996684 >> v2:172.21.15.179:3300/0 conn(0x7f32380accf0 0x7f3238160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:54.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.714+0000 7f3237e00640 1 -- 172.21.15.161:0/3625996684 >> v2:172.21.15.179:3300/0 conn(0x7f32380accf0 msgr2=0x7f3238160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:54.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.714+0000 7f3237e00640 1 --2- 172.21.15.161:0/3625996684 >> v2:172.21.15.179:3300/0 conn(0x7f32380accf0 0x7f3238160e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:54.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.714+0000 7f3237e00640 1 -- 172.21.15.161:0/3625996684 >> v1:172.21.15.179:6789/0 conn(0x7f32380ac8b0 legacy=0x7f323806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:54.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.714+0000 7f3237e00640 1 -- 172.21.15.161:0/3625996684 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323806c7f0 con 0x7f323815d430 2024-05-22T21:24:54.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.718+0000 7f3237e00640 1 --2- 172.21.15.161:0/3625996684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f323815d430 0x7f32380707c0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f32380aeac0 tx=0x7f322801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbc4b03a6d33af6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:54.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.718+0000 7f3237400640 1 -- 172.21.15.161:0/3625996684 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32280ab600 con 0x7f323815d430 2024-05-22T21:24:54.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.718+0000 7f3237400640 1 -- 172.21.15.161:0/3625996684 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32280a7050 con 0x7f323815d430 2024-05-22T21:24:54.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.718+0000 7f3237400640 1 -- 172.21.15.161:0/3625996684 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32280ab980 con 0x7f323815d430 2024-05-22T21:24:54.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.718+0000 7f323ee00640 1 -- 172.21.15.161:0/3625996684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f323815d430 msgr2=0x7f32380707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:54.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.718+0000 7f323ee00640 1 --2- 172.21.15.161:0/3625996684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f323815d430 0x7f32380707c0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f32380aeac0 tx=0x7f322801c460 comp rx=0 tx=0).stop 2024-05-22T21:24:54.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.718+0000 7f323ee00640 1 -- 172.21.15.161:0/3625996684 shutdown_connections 2024-05-22T21:24:54.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.718+0000 7f323ee00640 1 --2- 172.21.15.161:0/3625996684 >> v2:172.21.15.179:3300/0 conn(0x7f32380accf0 0x7f3238160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:54.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.718+0000 7f323ee00640 1 --2- 172.21.15.161:0/3625996684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f323815d430 0x7f32380707c0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:54.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.718+0000 7f323ee00640 1 -- 172.21.15.161:0/3625996684 >> 172.21.15.161:0/3625996684 conn(0x7f32380b0bc0 msgr2=0x7f32380b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:54.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.718+0000 7f323ee00640 1 -- 172.21.15.161:0/3625996684 shutdown_connections 2024-05-22T21:24:54.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.718+0000 7f323ee00640 1 -- 172.21.15.161:0/3625996684 wait complete. 2024-05-22T21:24:54.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f323ee00640 1 Processor -- start 2024-05-22T21:24:54.722 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f323ee00640 1 -- start start 2024-05-22T21:24:54.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f323ee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32380ac8b0 0x7f3238146280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:54.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f323ee00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32380accf0 0x7f3238144b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:54.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f323ee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f323815d430 0x7f323815f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:54.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f323ee00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32381517b0 con 0x7f32380accf0 2024-05-22T21:24:54.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f323ee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3238145070 con 0x7f323815d430 2024-05-22T21:24:54.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f323ee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32381511f0 con 0x7f32380ac8b0 2024-05-22T21:24:54.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f322f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32380accf0 0x7f3238144b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:54.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f3237e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32380ac8b0 0x7f3238146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:54.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f322f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32380accf0 0x7f3238144b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48324/0 (socket says 172.21.15.161:48324) 2024-05-22T21:24:54.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f322f400640 1 -- 172.21.15.161:0/2964136331 learned_addr learned my addr 172.21.15.161:0/2964136331 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:54.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f323ce00640 1 --2- 172.21.15.161:0/2964136331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f323815d430 0x7f323815f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:54.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f322f400640 1 -- 172.21.15.161:0/2964136331 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32380ac8b0 msgr2=0x7f3238146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:54.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f322f400640 1 --2- 172.21.15.161:0/2964136331 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32380ac8b0 0x7f3238146280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:54.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f322f400640 1 -- 172.21.15.161:0/2964136331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f323815d430 msgr2=0x7f323815f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:54.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f322f400640 1 --2- 172.21.15.161:0/2964136331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f323815d430 0x7f323815f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:54.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f322f400640 1 -- 172.21.15.161:0/2964136331 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32280094d0 con 0x7f32380accf0 2024-05-22T21:24:54.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f3237e00640 1 --2- 172.21.15.161:0/2964136331 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32380ac8b0 0x7f3238146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:24:54.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f322f400640 1 --2- 172.21.15.161:0/2964136331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32380accf0 0x7f3238144b30 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f322000c3c0 tx=0x7f322000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:54.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f3234c00640 1 -- 172.21.15.161:0/2964136331 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32200052d0 con 0x7f32380accf0 2024-05-22T21:24:54.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f3234c00640 1 -- 172.21.15.161:0/2964136331 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3220005430 con 0x7f32380accf0 2024-05-22T21:24:54.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f3234c00640 1 -- 172.21.15.161:0/2964136331 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220003840 con 0x7f32380accf0 2024-05-22T21:24:54.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f323ee00640 1 -- 172.21.15.161:0/2964136331 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32381514d0 con 0x7f32380accf0 2024-05-22T21:24:54.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f323ee00640 1 -- 172.21.15.161:0/2964136331 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3238152030 con 0x7f32380accf0 2024-05-22T21:24:54.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.722+0000 7f323ee00640 1 -- 172.21.15.161:0/2964136331 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3238067220 con 0x7f32380accf0 2024-05-22T21:24:54.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.726+0000 7f3234c00640 1 -- 172.21.15.161:0/2964136331 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f32200039a0 con 0x7f32380accf0 2024-05-22T21:24:54.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.726+0000 7f3234c00640 1 --2- 172.21.15.161:0/2964136331 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f32140666c0 0x7f3214068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:54.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.726+0000 7f3234c00640 1 -- 172.21.15.161:0/2964136331 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3220083680 con 0x7f32380accf0 2024-05-22T21:24:54.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.726+0000 7f3237e00640 1 --2- 172.21.15.161:0/2964136331 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f32140666c0 0x7f3214068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:54.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.726+0000 7f3234c00640 1 -- 172.21.15.161:0/2964136331 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=193}) v3 -- 0x7f321406e1f0 con 0x7f32380accf0 2024-05-22T21:24:54.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.726+0000 7f3237e00640 1 --2- 172.21.15.161:0/2964136331 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f32140666c0 0x7f3214068b80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f32280ad000 tx=0x7f3228008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:54.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.730+0000 7f3234c00640 1 -- 172.21.15.161:0/2964136331 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3220050a00 con 0x7f32380accf0 2024-05-22T21:24:54.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.882+0000 7f323ee00640 1 -- 172.21.15.161:0/2964136331 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f32380ae7a0 con 0x7f32380accf0 2024-05-22T21:24:54.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.882+0000 7f3234c00640 1 -- 172.21.15.161:0/2964136331 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v192) v1 ==== 81+0+33613 (secure 0 0 0) 0x7f3220017020 con 0x7f32380accf0 2024-05-22T21:24:54.886 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:54.886 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:24:54.886 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 192, 2024-05-22T21:24:54.886 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:24:54.886 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:24:54.886 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:24:50.309408+0000", 2024-05-22T21:24:54.886 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:24:54.886 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:24:54.886 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:24:54.887 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:24:54.887 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:24:54.887 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:24:54.887 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:24:54.887 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:24:54.887 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:24:54.887 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:54.887 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:24:54.887 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:24:54.887 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:24:54.887 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:24:54.887 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:24:54.887 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 23, 2024-05-22T21:24:54.888 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:24:54.888 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:24:54.888 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:24:54.888 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:24:54.888 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:24:54.888 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:24:54.888 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.888 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:24:54.888 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:24:54.888 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:24:54.888 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:24:54.888 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:24:54.888 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:24:54.889 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:24:54.889 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:24:54.889 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:24:54.889 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:24:54.889 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:24:54.889 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:24:54.889 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:24:54.889 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:24:54.889 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:24:54.889 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:24:54.889 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:24:54.889 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:24:54.889 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:24:54.890 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:24:54.890 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:24:54.890 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:24:54.890 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:24:54.890 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:24:54.890 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:24:54.890 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:24:54.890 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:24:54.890 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.890 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:24:54.890 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:24:54.890 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:24:54.891 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:24:54.891 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:24:54.891 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:24:54.891 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:24:54.891 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:24:54.891 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:24:54.891 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:24:54.891 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:24:54.891 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:24:54.891 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:24:54.891 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:24:54.891 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:24:54.891 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:24:54.891 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:24:54.892 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:24:54.892 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:24:54.892 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:24:54.892 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:24:54.892 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:24:54.892 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:24:54.892 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:24:54.892 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:24:54.892 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:24:54.892 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:24:54.892 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.892 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:24:54.893 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:24:54.893 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:24:54.893 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:24:54.893 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:24:54.893 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:24:54.893 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:24:54.893 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:24:54.893 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:24:54.893 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:24:54.893 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:24:54.893 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:24:54.893 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:24:54.893 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:24:54.894 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.894 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:24:54.894 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:24:54.894 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.894 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:24:54.894 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:24:54.894 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:24:54.894 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:24:54.894 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:24:54.894 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:24:54.894 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:24:54.894 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:24:54.895 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:24:54.895 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.895 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.895 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.895 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 22, 2024-05-22T21:24:54.895 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:24:54.895 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:24:31.797440+0000", 2024-05-22T21:24:54.895 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:24:54.895 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:24:54.895 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:24:54.895 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:24:54.895 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:24:54.895 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:24:54.896 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:24:54.896 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:24:54.896 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:24:54.896 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:24:54.896 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:24:54.896 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:24:54.896 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:24:54.896 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:24:54.896 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:24:54.896 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:24:54.896 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:24:54.896 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:24:54.897 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:24:54.897 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:24:54.897 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:24:54.897 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:24:54.897 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:24:54.897 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:24:54.897 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.897 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "189", 2024-05-22T21:24:54.897 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:24:54.897 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:24:54.897 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:24:54.897 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:24:54.898 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:24:54.898 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:24:54.898 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:24:54.898 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:24:54.898 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:24:54.898 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:24:54.898 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:24:54.898 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:24:54.898 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:24:54.898 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:24:54.898 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:24:54.898 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:24:54.899 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:24:54.899 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:24:54.899 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:24:54.899 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:24:54.899 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:24:54.899 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:24:54.899 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:24:54.899 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:24:54.899 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:24:54.899 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:24:54.899 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.899 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:24:54.899 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:24:54.900 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:24:54.900 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:24:54.900 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:24:54.900 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:24:54.900 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:24:54.900 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:24:54.900 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:24:54.900 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:24:54.900 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:24:54.900 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:24:54.900 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:24:54.900 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:24:54.900 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:24:54.900 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.901 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:24:54.901 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:24:54.901 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:24:54.901 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.901 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.901 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:24:54.901 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 2, 2024-05-22T21:24:54.901 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 2, 2024-05-22T21:24:54.901 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:24:54.901 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:24:54.901 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:24:54.901 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:24:54.901 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:24:54.902 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:24:54.902 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.902 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.902 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.902 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 23, 2024-05-22T21:24:54.902 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:24:54.902 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:24:33.098240+0000", 2024-05-22T21:24:54.902 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1027, 2024-05-22T21:24:54.902 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full,full_quota", 2024-05-22T21:24:54.902 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:24:54.902 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:24:54.902 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:24:54.902 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:24:54.903 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:24:54.903 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:24:54.903 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:24:54.903 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:24:54.903 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:24:54.903 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:24:54.903 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:24:54.903 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:24:54.903 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:24:54.903 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:24:54.903 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:24:54.903 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:24:54.903 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:24:54.903 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:24:54.904 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:24:54.904 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:24:54.904 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:24:54.904 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:24:54.904 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.904 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "192", 2024-05-22T21:24:54.904 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:24:54.904 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:24:54.904 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:24:54.904 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:24:54.904 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:24:54.904 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:24:54.904 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:24:54.905 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:24:54.905 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:24:54.905 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 33554432, 2024-05-22T21:24:54.905 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:24:54.905 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:24:54.905 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:24:54.905 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:24:54.905 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:24:54.905 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:24:54.905 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:24:54.905 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:24:54.905 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:24:54.905 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:24:54.906 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:24:54.906 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:24:54.906 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:24:54.906 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:24:54.906 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:24:54.906 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:24:54.906 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.906 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:24:54.906 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:24:54.906 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:24:54.906 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:24:54.906 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:24:54.906 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:24:54.907 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:24:54.907 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:24:54.907 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:24:54.907 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:24:54.907 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:24:54.907 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:24:54.907 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:24:54.907 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:24:54.907 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.907 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:24:54.907 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:24:54.907 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:24:54.908 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.908 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.908 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:24:54.908 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.3799999952316284, 2024-05-22T21:24:54.908 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.3799999952316284, 2024-05-22T21:24:54.908 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:24:54.908 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.3799999952316284, 2024-05-22T21:24:54.908 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.3799999952316284, 2024-05-22T21:24:54.908 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:24:54.908 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:24:54.908 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:24:54.908 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.908 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.909 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:54.909 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:24:54.909 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.909 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:24:54.909 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:54.909 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:54.909 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:54.909 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:54.909 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:54.909 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:54.909 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:54.909 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:54.909 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:54.910 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:54.910 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:54.910 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:54.910 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.910 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.910 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.910 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:24:54.910 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:54.910 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.910 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.910 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.910 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:24:54.911 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:54.911 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.911 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.911 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.911 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:54.911 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.911 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.911 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.911 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:24:54.911 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:54.911 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.911 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.911 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.911 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:24:54.912 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:54.912 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.912 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.912 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.912 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:54.912 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.912 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.912 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.912 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:24:54.912 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:54.912 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.912 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.912 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.912 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:24:54.913 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:54.913 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.913 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.913 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.913 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:54.913 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.913 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.913 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.913 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:24:54.913 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:54.913 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.913 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.913 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.913 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:24:54.913 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:54.914 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.914 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.914 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.914 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:24:54.914 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:24:54.914 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:24:54.914 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:24:54.914 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:54.914 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:54.914 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:54.914 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.914 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.914 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.915 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:24:54.915 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:54.915 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:54.915 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:54.915 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:54.915 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:54.915 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:54.915 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:54.915 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:54.915 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:54.915 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:54.915 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:54.915 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:54.916 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.916 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.916 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.916 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:24:54.916 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:54.916 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.916 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.916 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.916 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:24:54.916 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:54.916 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.916 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.916 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.917 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:54.917 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.917 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.917 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.917 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:24:54.917 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:54.917 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.917 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.917 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.917 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:24:54.917 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:54.917 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.917 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.917 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.918 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:54.918 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.918 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.918 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.918 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:24:54.918 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:54.918 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.918 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.918 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.918 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:24:54.918 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:54.918 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.918 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.919 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.919 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:54.919 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.919 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.919 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.919 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:24:54.919 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:54.919 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.919 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.919 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.919 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:24:54.919 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:54.919 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.920 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.920 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.920 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:24:54.920 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:24:54.920 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:24:54.920 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:24:54.920 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:54.920 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:54.920 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:54.920 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.920 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.920 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.920 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:24:54.921 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:54.921 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:54.921 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:54.921 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:54.921 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:54.921 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:54.921 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:54.921 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:54.921 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:54.921 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:54.921 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:54.921 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:54.922 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.922 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.922 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.922 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:24:54.922 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:54.922 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.922 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.922 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.922 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:24:54.922 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:54.922 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.922 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.922 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.923 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:54.923 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.923 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.923 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.923 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:24:54.923 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:54.923 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.923 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.923 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.923 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:24:54.923 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:54.923 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.924 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.924 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.924 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:54.924 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.924 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.924 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.924 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:24:54.924 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:54.924 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.924 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.924 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.924 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:24:54.924 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:54.925 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.925 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.925 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.925 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:54.925 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.925 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.925 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.925 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:24:54.925 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:54.925 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.925 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.925 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.925 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:24:54.926 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:54.926 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.926 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.926 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.926 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:24:54.926 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:24:54.926 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:24:54.926 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:24:54.926 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:54.926 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:54.926 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:54.926 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.926 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.927 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.927 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:24:54.927 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:54.927 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:54.927 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:54.927 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:54.927 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:54.927 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:54.927 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:54.927 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:54.927 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:54.927 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:54.927 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:54.928 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:54.928 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.928 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.928 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.928 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:24:54.928 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:54.928 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.928 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.928 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.928 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:24:54.928 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:54.928 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.928 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.928 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.929 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:54.929 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.929 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.929 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.929 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:24:54.929 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:54.929 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.929 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.929 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.929 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:24:54.929 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:54.929 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.929 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.930 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.930 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:54.930 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.930 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.930 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.930 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:24:54.930 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:54.930 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.930 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.930 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.930 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:24:54.930 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:54.930 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.931 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.931 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.931 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:54.931 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.931 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.931 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.931 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:24:54.931 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:54.931 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.931 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.932 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.932 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:24:54.932 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:54.932 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.932 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.932 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.932 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:24:54.932 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:24:54.932 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:24:54.932 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:24:54.932 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:54.932 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:54.932 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:54.933 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.933 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.933 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.933 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:24:54.933 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:54.933 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:54.933 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:54.933 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:54.933 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:54.933 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:54.933 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:54.933 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:54.933 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:54.933 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:54.934 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:54.934 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:54.934 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.934 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.934 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.934 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:24:54.934 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:54.934 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.934 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.934 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.934 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:24:54.934 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:54.934 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.935 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.935 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.935 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:54.935 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.935 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.935 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.935 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:24:54.935 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:54.935 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.935 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.935 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.935 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:24:54.936 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:54.936 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.936 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.936 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.936 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:54.936 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.936 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.936 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.936 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:24:54.936 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:54.936 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.936 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.936 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.937 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:24:54.937 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:54.937 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.937 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.937 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.937 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:54.937 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.937 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.937 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.937 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:24:54.937 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:54.937 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.937 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.937 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.938 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:24:54.938 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:54.938 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.938 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.938 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.938 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:24:54.938 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:24:54.938 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:24:54.938 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:24:54.938 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:54.938 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:54.939 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:54.939 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.939 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.939 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.939 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:24:54.939 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:54.939 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:54.939 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:54.939 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:54.939 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:54.939 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:54.939 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:54.940 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:54.940 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:54.940 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:54.940 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:54.940 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:54.940 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.940 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.940 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.940 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:24:54.940 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:54.940 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.940 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.940 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.941 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:24:54.941 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:54.941 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.941 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.941 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.941 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:54.941 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.941 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.941 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.941 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:24:54.941 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:54.941 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.941 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.942 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.942 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:24:54.942 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:54.942 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.942 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.942 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.942 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:54.942 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.942 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.942 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.942 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:24:54.942 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:54.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.943 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.943 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.943 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:24:54.943 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:54.943 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.943 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.943 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.943 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:54.943 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.943 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.943 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.943 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:24:54.944 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:54.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.944 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.944 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.944 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:24:54.944 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:54.944 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.944 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.944 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.944 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:24:54.944 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:24:54.944 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:24:54.944 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:24:54.945 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:54.945 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:54.945 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:54.945 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.945 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.945 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.945 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:24:54.945 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:54.945 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:54.945 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:54.945 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:54.945 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:54.946 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:54.946 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:54.946 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:54.946 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:54.946 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:54.946 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:54.946 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:54.946 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.946 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.946 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.946 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:24:54.946 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:54.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.947 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.947 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.947 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:24:54.947 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:54.947 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.947 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.947 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.947 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:54.947 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.947 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.948 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.948 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:24:54.948 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:54.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.948 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.948 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.948 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:24:54.948 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:54.948 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.948 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.948 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.948 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:54.948 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.949 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.949 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.949 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:24:54.949 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:54.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.949 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.949 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.949 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:24:54.949 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:54.949 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.949 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.949 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.949 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:54.950 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.950 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.950 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.950 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:24:54.950 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:54.950 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.950 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.950 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.950 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:24:54.950 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:54.950 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.950 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.951 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:24:54.951 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:24:54.951 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:24:54.951 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:24:54.951 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:54.951 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:54.951 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:54.951 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.951 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.951 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:24:54.951 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:54.952 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:54.952 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:54.952 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:54.952 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:54.952 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:54.952 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:54.952 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:54.952 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:54.952 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:54.952 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:54.952 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:54.952 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.952 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.953 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.953 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:24:54.953 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:54.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.953 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.953 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.954 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:24:54.954 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:54.954 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.954 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.954 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:54.954 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.954 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.954 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.954 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:24:54.954 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:54.954 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.954 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.955 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.955 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:24:54.955 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:54.955 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.955 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.955 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:54.955 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.955 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.955 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.955 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:24:54.955 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:54.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.956 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.956 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.956 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:24:54.956 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:54.956 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.956 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.956 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:54.956 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:54.956 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.956 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:54.956 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:24:54.957 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:54.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.957 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.957 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:54.957 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:24:54.957 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:54.957 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.957 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.957 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:24:54.957 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:24:54.957 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:24:54.957 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:24:54.958 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:54.958 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:54.958 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:54.958 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:54.958 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.958 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:54.958 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:24:54.958 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.958 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:24:54.958 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:54.958 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:54.958 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:54.958 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:54.959 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:54.959 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:24:54.959 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:54.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.959 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.959 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:24:54.959 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:54.959 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:54.959 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:54.959 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:54.959 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:54.959 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:24:54.959 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:54.959 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.960 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.960 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:24:54.960 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:54.960 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:54.960 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:54.960 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:54.960 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:54.960 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:24:54.960 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:54.960 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.960 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.960 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:24:54.960 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:54.961 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:54.961 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:54.961 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:54.961 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:54.961 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:24:54.961 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:54.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.961 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.961 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:24:54.961 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:54.961 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:54.961 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:54.962 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:54.962 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:54.962 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:24:54.962 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:54.962 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.962 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.962 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:24:54.962 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:54.962 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:54.962 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:54.962 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:54.962 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:54.962 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:24:54.963 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:54.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.963 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.963 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:24:54.963 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:54.963 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:54.963 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:54.963 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:54.963 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:54.963 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:24:54.963 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:54.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.963 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:54.963 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:24:54.964 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:54.964 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:54.964 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:54.964 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:54.964 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:54.964 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:24:54.964 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:54.964 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.964 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:54.964 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:24:54.964 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:24:54.964 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:24:54.964 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:24:54.965 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:24:54.965 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:24:54.965 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:24:54.965 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:24:54.965 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:24:54.965 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:24:54.965 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:24:54.965 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:24:54.965 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:24:54.965 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:24:54.965 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:54.965 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:24:54.965 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:24:54.966 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:24:54.966 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:24:54.966 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:24:54.966 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:24:54.966 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:24:54.966 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:24:54.966 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:24:54.966 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:24:54.966 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:24:54.966 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:54.966 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:24:54.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.886+0000 7f323ee00640 1 -- 172.21.15.161:0/2964136331 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f32140666c0 msgr2=0x7f3214068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:54.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.886+0000 7f323ee00640 1 --2- 172.21.15.161:0/2964136331 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f32140666c0 0x7f3214068b80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f32280ad000 tx=0x7f3228008040 comp rx=0 tx=0).stop 2024-05-22T21:24:54.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.886+0000 7f323ee00640 1 -- 172.21.15.161:0/2964136331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32380accf0 msgr2=0x7f3238144b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:54.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.886+0000 7f323ee00640 1 --2- 172.21.15.161:0/2964136331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32380accf0 0x7f3238144b30 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f322000c3c0 tx=0x7f322000c880 comp rx=0 tx=0).stop 2024-05-22T21:24:54.968 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.886+0000 7f323ee00640 1 -- 172.21.15.161:0/2964136331 shutdown_connections 2024-05-22T21:24:54.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.886+0000 7f323ee00640 1 --2- 172.21.15.161:0/2964136331 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f32140666c0 0x7f3214068b80 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:54.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.886+0000 7f323ee00640 1 --2- 172.21.15.161:0/2964136331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f323815d430 0x7f323815f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:54.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.886+0000 7f323ee00640 1 --2- 172.21.15.161:0/2964136331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32380accf0 0x7f3238144b30 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:54.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.886+0000 7f323ee00640 1 --2- 172.21.15.161:0/2964136331 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32380ac8b0 0x7f3238146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:54.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.886+0000 7f323ee00640 1 -- 172.21.15.161:0/2964136331 >> 172.21.15.161:0/2964136331 conn(0x7f32380b0bc0 msgr2=0x7f32380afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:54.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.886+0000 7f323ee00640 1 -- 172.21.15.161:0/2964136331 shutdown_connections 2024-05-22T21:24:54.969 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:54.886+0000 7f323ee00640 1 -- 172.21.15.161:0/2964136331 wait complete. 2024-05-22T21:24:54.970 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-05-22T21:24:54.970 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:24:54.970 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:24:54.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:54.978 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2024-05-22T21:24:55.180 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:24:55.183 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:24:55.406 INFO:teuthology.orchestra.run.smithi161.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-05-22T21:24:55.406 INFO:teuthology.orchestra.run.smithi161.stderr:50+0 records in 2024-05-22T21:24:55.406 INFO:teuthology.orchestra.run.smithi161.stderr:50+0 records out 2024-05-22T21:24:55.406 INFO:teuthology.orchestra.run.smithi161.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.373614 s, 140 MB/s 2024-05-22T21:24:55.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:24:55.407 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:24:55.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.542+0000 7fc51d000640 1 Processor -- start 2024-05-22T21:24:55.543 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.542+0000 7fc51d000640 1 -- start start 2024-05-22T21:24:55.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.542+0000 7fc51d000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc51815dd50 0x7fc51806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:55.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.542+0000 7fc51d000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc51815e130 0x7fc51815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:55.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.542+0000 7fc51d000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc518071670 con 0x7fc51815d410 2024-05-22T21:24:55.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.542+0000 7fc51d000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc51815d8a0 con 0x7fc51815dd50 2024-05-22T21:24:55.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.542+0000 7fc51d000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51806de40 con 0x7fc51815e130 2024-05-22T21:24:55.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.542+0000 7fc515e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc51815dd50 0x7fc51806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:55.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.542+0000 7fc515e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc51815dd50 0x7fc51806d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48338/0 (socket says 172.21.15.161:48338) 2024-05-22T21:24:55.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.542+0000 7fc515e00640 1 -- 172.21.15.161:0/2990785098 learned_addr learned my addr 172.21.15.161:0/2990785098 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:55.545 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.542+0000 7fc50d400640 1 --2- 172.21.15.161:0/2990785098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc51815e130 0x7fc51815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:55.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.546+0000 7fc515e00640 1 -- 172.21.15.161:0/2990785098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc51815e130 msgr2=0x7fc51815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:55.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.546+0000 7fc515e00640 1 --2- 172.21.15.161:0/2990785098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc51815e130 0x7fc51815ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:55.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.546+0000 7fc515e00640 1 -- 172.21.15.161:0/2990785098 >> v1:172.21.15.179:6789/0 conn(0x7fc51815d410 legacy=0x7fc51815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:55.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.546+0000 7fc515e00640 1 -- 172.21.15.161:0/2990785098 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc518070a60 con 0x7fc51815dd50 2024-05-22T21:24:55.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.546+0000 7fc515400640 1 -- 172.21.15.161:0/2990785098 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839954462 0 0) 0x7fc518071670 con 0x7fc51815d410 2024-05-22T21:24:55.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.546+0000 7fc515e00640 1 --2- 172.21.15.161:0/2990785098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc51815dd50 0x7fc51806d900 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fc50800bb10 tx=0x7fc50801eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9667ab64881042 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:55.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.546+0000 7fc515400640 1 -- 172.21.15.161:0/2990785098 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5080a7c40 con 0x7fc51815dd50 2024-05-22T21:24:55.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.546+0000 7fc515400640 1 -- 172.21.15.161:0/2990785098 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5080a7da0 con 0x7fc51815dd50 2024-05-22T21:24:55.548 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.546+0000 7fc515400640 1 -- 172.21.15.161:0/2990785098 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5080af660 con 0x7fc51815dd50 2024-05-22T21:24:55.549 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.546+0000 7fc51d000640 1 -- 172.21.15.161:0/2990785098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc51815dd50 msgr2=0x7fc51806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:55.549 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.546+0000 7fc51d000640 1 --2- 172.21.15.161:0/2990785098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc51815dd50 0x7fc51806d900 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fc50800bb10 tx=0x7fc50801eb70 comp rx=0 tx=0).stop 2024-05-22T21:24:55.549 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.546+0000 7fc51d000640 1 -- 172.21.15.161:0/2990785098 shutdown_connections 2024-05-22T21:24:55.549 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.546+0000 7fc51d000640 1 --2- 172.21.15.161:0/2990785098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc51815e130 0x7fc51815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:55.550 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.546+0000 7fc51d000640 1 --2- 172.21.15.161:0/2990785098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc51815dd50 0x7fc51806d900 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:55.550 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.546+0000 7fc51d000640 1 -- 172.21.15.161:0/2990785098 >> 172.21.15.161:0/2990785098 conn(0x7fc5180b0bc0 msgr2=0x7fc5180b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:55.550 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc51d000640 1 -- 172.21.15.161:0/2990785098 shutdown_connections 2024-05-22T21:24:55.550 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc51d000640 1 -- 172.21.15.161:0/2990785098 wait complete. 2024-05-22T21:24:55.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc51d000640 1 Processor -- start 2024-05-22T21:24:55.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc51d000640 1 -- start start 2024-05-22T21:24:55.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc51d000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc51815d410 0x7fc5180cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:55.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc51d000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc51815dd50 0x7fc5180b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:55.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc51d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc51815e130 0x7fc51814bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:55.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc51d000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5180cd920 con 0x7fc51815e130 2024-05-22T21:24:55.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc51d000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5180cdc00 con 0x7fc51815dd50 2024-05-22T21:24:55.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc51d000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5180cdee0 con 0x7fc51815d410 2024-05-22T21:24:55.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc516800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc51815e130 0x7fc51814bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:55.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc515e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc51815d410 0x7fc5180cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:55.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc516800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc51815e130 0x7fc51814bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:48340/0 (socket says 172.21.15.161:48340) 2024-05-22T21:24:55.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc516800640 1 -- 172.21.15.161:0/1924580148 learned_addr learned my addr 172.21.15.161:0/1924580148 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:55.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc50d400640 1 --2- 172.21.15.161:0/1924580148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc51815dd50 0x7fc5180b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:55.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc516800640 1 -- 172.21.15.161:0/1924580148 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc51815d410 msgr2=0x7fc5180cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:55.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc516800640 1 --2- 172.21.15.161:0/1924580148 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc51815d410 0x7fc5180cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:55.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc516800640 1 -- 172.21.15.161:0/1924580148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc51815dd50 msgr2=0x7fc5180b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:55.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc516800640 1 --2- 172.21.15.161:0/1924580148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc51815dd50 0x7fc5180b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:55.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc516800640 1 -- 172.21.15.161:0/1924580148 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50800b770 con 0x7fc51815e130 2024-05-22T21:24:55.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc516800640 1 --2- 172.21.15.161:0/1924580148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc51815e130 0x7fc51814bab0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fc510002240 tx=0x7fc51000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:55.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc50ea00640 1 -- 172.21.15.161:0/1924580148 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510003db0 con 0x7fc51815e130 2024-05-22T21:24:55.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc50ea00640 1 -- 172.21.15.161:0/1924580148 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51000ce30 con 0x7fc51815e130 2024-05-22T21:24:55.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc50ea00640 1 -- 172.21.15.161:0/1924580148 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510005270 con 0x7fc51815e130 2024-05-22T21:24:55.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc51d000640 1 -- 172.21.15.161:0/1924580148 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51814d260 con 0x7fc51815e130 2024-05-22T21:24:55.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.550+0000 7fc51d000640 1 -- 172.21.15.161:0/1924580148 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5180cd630 con 0x7fc51815e130 2024-05-22T21:24:55.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.554+0000 7fc51d000640 1 -- 172.21.15.161:0/1924580148 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc518067220 con 0x7fc51815e130 2024-05-22T21:24:55.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.554+0000 7fc50ea00640 1 -- 172.21.15.161:0/1924580148 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc5100053d0 con 0x7fc51815e130 2024-05-22T21:24:55.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.554+0000 7fc50ea00640 1 --2- 172.21.15.161:0/1924580148 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc4f4066690 0x7fc4f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:55.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.554+0000 7fc50ea00640 1 -- 172.21.15.161:0/1924580148 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5100838b0 con 0x7fc51815e130 2024-05-22T21:24:55.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.554+0000 7fc50ea00640 1 -- 172.21.15.161:0/1924580148 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=193}) v3 -- 0x7fc4f406e1c0 con 0x7fc51815e130 2024-05-22T21:24:55.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.554+0000 7fc515e00640 1 --2- 172.21.15.161:0/1924580148 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc4f4066690 0x7fc4f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:55.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.554+0000 7fc515e00640 1 --2- 172.21.15.161:0/1924580148 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc4f4066690 0x7fc4f4068b50 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc5080b1000 tx=0x7fc5080a9070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:55.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.558+0000 7fc50ea00640 1 -- 172.21.15.161:0/1924580148 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fc510017070 con 0x7fc51815e130 2024-05-22T21:24:55.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.714+0000 7fc51d000640 1 -- 172.21.15.161:0/1924580148 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc51806c310 con 0x7fc51815e130 2024-05-22T21:24:55.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.714+0000 7fc50ea00640 1 -- 172.21.15.161:0/1924580148 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v192) v1 ==== 81+0+33613 (secure 0 0 0) 0x7fc510050cc0 con 0x7fc51815e130 2024-05-22T21:24:55.717 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:55.717 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:24:55.717 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 192, 2024-05-22T21:24:55.717 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:24:55.717 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:24:55.717 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:24:50.309408+0000", 2024-05-22T21:24:55.717 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:24:55.717 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:24:55.717 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:24:55.717 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:24:55.717 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:24:55.717 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:24:55.718 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:24:55.718 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:24:55.718 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:24:55.718 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:55.718 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:24:55.718 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:24:55.718 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:24:55.718 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:24:55.718 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:24:55.718 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 23, 2024-05-22T21:24:55.718 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:24:55.718 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:24:55.718 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:24:55.719 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:24:55.719 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:24:55.719 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:24:55.719 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.719 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:24:55.719 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:24:55.719 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:24:55.719 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:24:55.719 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:24:55.719 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:24:55.719 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:24:55.719 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:24:55.720 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:24:55.720 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:24:55.720 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:24:55.720 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:24:55.720 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:24:55.720 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:24:55.720 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:24:55.720 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:24:55.720 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:24:55.720 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:24:55.720 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:24:55.720 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:24:55.720 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:24:55.721 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:24:55.721 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:24:55.721 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:24:55.721 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:24:55.721 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:24:55.721 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:24:55.721 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.721 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:24:55.721 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:24:55.721 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:24:55.721 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:24:55.721 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:24:55.721 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:24:55.721 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:24:55.722 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:24:55.722 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:24:55.722 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:24:55.722 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:24:55.722 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:24:55.722 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:24:55.722 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:24:55.722 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:24:55.722 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:24:55.722 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:24:55.722 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:24:55.722 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:24:55.723 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:24:55.723 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:24:55.723 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:24:55.723 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:24:55.723 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:24:55.723 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:24:55.723 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:24:55.723 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:24:55.723 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.723 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:24:55.723 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:24:55.723 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:24:55.723 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:24:55.723 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:24:55.724 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:24:55.724 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:24:55.724 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:24:55.724 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:24:55.724 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:24:55.724 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:24:55.724 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:24:55.724 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:24:55.724 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:24:55.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.724 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:24:55.724 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:24:55.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.725 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:24:55.725 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:24:55.725 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:24:55.725 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:24:55.725 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:24:55.725 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:24:55.725 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:24:55.725 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:24:55.725 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:24:55.725 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.725 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.725 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.725 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 22, 2024-05-22T21:24:55.726 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:24:55.726 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:24:31.797440+0000", 2024-05-22T21:24:55.726 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:24:55.726 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:24:55.726 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:24:55.726 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:24:55.726 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:24:55.726 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:24:55.726 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:24:55.726 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:24:55.726 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:24:55.726 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:24:55.726 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:24:55.727 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:24:55.727 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:24:55.727 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:24:55.727 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:24:55.727 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:24:55.727 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:24:55.727 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:24:55.727 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:24:55.727 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:24:55.727 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:24:55.727 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:24:55.727 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:24:55.727 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:24:55.728 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.728 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "189", 2024-05-22T21:24:55.728 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:24:55.728 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:24:55.728 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:24:55.728 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:24:55.728 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:24:55.728 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:24:55.728 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:24:55.728 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:24:55.728 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:24:55.728 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:24:55.728 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:24:55.728 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:24:55.729 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:24:55.729 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:24:55.729 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:24:55.729 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:24:55.729 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:24:55.729 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:24:55.729 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:24:55.729 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:24:55.729 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:24:55.729 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:24:55.729 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:24:55.729 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:24:55.729 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:24:55.729 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:24:55.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.730 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:24:55.730 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:24:55.730 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:24:55.730 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:24:55.730 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:24:55.730 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:24:55.730 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:24:55.730 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:24:55.730 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:24:55.730 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:24:55.730 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:24:55.730 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:24:55.731 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:24:55.731 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:24:55.731 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:24:55.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.731 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:24:55.731 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:24:55.731 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:24:55.731 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.731 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.731 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:24:55.731 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 2, 2024-05-22T21:24:55.731 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 2, 2024-05-22T21:24:55.731 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:24:55.732 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:24:55.732 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:24:55.732 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:24:55.732 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:24:55.732 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:24:55.732 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.732 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.732 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 23, 2024-05-22T21:24:55.732 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:24:55.732 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:24:33.098240+0000", 2024-05-22T21:24:55.732 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1027, 2024-05-22T21:24:55.732 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full,full_quota", 2024-05-22T21:24:55.733 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:24:55.733 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:24:55.733 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:24:55.733 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:24:55.733 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:24:55.733 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:24:55.733 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:24:55.733 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:24:55.733 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:24:55.733 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:24:55.733 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:24:55.733 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:24:55.733 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:24:55.733 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:24:55.734 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:24:55.734 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:24:55.734 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:24:55.734 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:24:55.734 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:24:55.734 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:24:55.734 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:24:55.734 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:24:55.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.734 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "192", 2024-05-22T21:24:55.734 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:24:55.734 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:24:55.734 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:24:55.738 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:24:55.738 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:24:55.738 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:24:55.738 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:24:55.738 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:24:55.739 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:24:55.739 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 33554432, 2024-05-22T21:24:55.739 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:24:55.739 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:24:55.739 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:24:55.739 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:24:55.739 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:24:55.739 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:24:55.739 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:24:55.739 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:24:55.739 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:24:55.739 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:24:55.739 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:24:55.740 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:24:55.740 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:24:55.740 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:24:55.740 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:24:55.740 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:24:55.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.740 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:24:55.740 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:24:55.740 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:24:55.740 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:24:55.740 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:24:55.740 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:24:55.740 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:24:55.741 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:24:55.741 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:24:55.741 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:24:55.741 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:24:55.741 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:24:55.741 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:24:55.741 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:24:55.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.741 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:24:55.741 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:24:55.741 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:24:55.741 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.742 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:24:55.742 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.3799999952316284, 2024-05-22T21:24:55.742 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.3799999952316284, 2024-05-22T21:24:55.742 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:24:55.742 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.3799999952316284, 2024-05-22T21:24:55.742 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.3799999952316284, 2024-05-22T21:24:55.742 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:24:55.742 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:24:55.742 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:24:55.742 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.742 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.742 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:55.743 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:24:55.743 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.743 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:24:55.743 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:55.743 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:55.743 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:55.743 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:55.743 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:55.743 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:55.743 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:55.743 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:55.743 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:55.744 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:55.744 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:55.744 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:55.744 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.744 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.744 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:24:55.744 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:55.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.744 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.744 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.744 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:24:55.744 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:55.745 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.745 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.745 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:55.745 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.745 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.745 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.745 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:24:55.745 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:55.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.745 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.745 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.746 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:24:55.746 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:55.746 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.746 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.746 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:55.746 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.746 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.746 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.746 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:24:55.746 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:55.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.746 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.747 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.747 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:24:55.747 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:55.747 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.747 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.748 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:55.748 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.748 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.748 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:24:55.748 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:55.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.748 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.748 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.748 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:24:55.749 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:55.749 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.749 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.749 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:24:55.749 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:24:55.749 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:24:55.749 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:24:55.749 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:55.749 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:55.750 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:55.750 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.750 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.750 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:24:55.750 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:55.750 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:55.750 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:55.750 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:55.750 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:55.750 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:55.750 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:55.751 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:55.751 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:55.751 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:55.751 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:55.751 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:55.751 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.751 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.751 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.751 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:24:55.751 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:55.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.752 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.752 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.752 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:24:55.752 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:55.752 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.752 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.752 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:55.752 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.752 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.752 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.753 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:24:55.753 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:55.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.753 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.753 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.753 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:24:55.753 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:55.753 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.753 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.753 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:55.753 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.753 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.754 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.754 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:24:55.754 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:55.754 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.754 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.754 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.754 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:24:55.754 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:55.754 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.754 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.754 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.754 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:55.754 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.755 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.755 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.755 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:24:55.755 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:55.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.755 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.755 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.755 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:24:55.755 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:55.755 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.755 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.755 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:24:55.755 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:24:55.756 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:24:55.756 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:24:55.756 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:55.756 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:55.756 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:55.756 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.756 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.756 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.756 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:24:55.756 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:55.756 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:55.756 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:55.756 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:55.756 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:55.757 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:55.757 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:55.757 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:55.757 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:55.757 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:55.757 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:55.757 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:55.757 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.757 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.757 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.757 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:24:55.757 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:55.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.757 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.758 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.758 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:24:55.758 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:55.758 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.758 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.758 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.758 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:55.758 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.758 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.758 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.758 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:24:55.758 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:55.759 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.759 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.759 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.759 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:24:55.759 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:55.759 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.759 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.759 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.759 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:55.759 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.759 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.759 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.760 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:24:55.760 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:55.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.760 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.760 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.760 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:24:55.760 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:55.760 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.760 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.760 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:55.760 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.760 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.761 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.761 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:24:55.761 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:55.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.761 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.761 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.761 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:24:55.761 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:55.761 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.761 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.761 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.761 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:24:55.761 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:24:55.762 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:24:55.762 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:24:55.762 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:55.762 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:55.762 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:55.762 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.762 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.762 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.762 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:24:55.762 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:55.762 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:55.762 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:55.763 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:55.763 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:55.763 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:55.763 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:55.763 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:55.763 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:55.763 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:55.763 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:55.763 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:55.763 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.763 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.763 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.763 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:24:55.764 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:55.764 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.764 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.764 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.764 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:24:55.764 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:55.764 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.764 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.764 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.764 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:55.764 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.764 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.764 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.764 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:24:55.765 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:55.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.765 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.765 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.765 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:24:55.765 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:55.765 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.765 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.765 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:55.765 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.765 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.765 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.766 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:24:55.766 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:55.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.766 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.766 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.766 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:24:55.766 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:55.766 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.766 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.766 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:55.766 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.766 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.767 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.767 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:24:55.767 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:55.767 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.767 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.767 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.767 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:24:55.767 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:55.767 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.767 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.767 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.767 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:24:55.767 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:24:55.767 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:24:55.768 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:24:55.768 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:55.768 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:55.768 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:55.768 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.768 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.768 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.768 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:24:55.768 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:55.768 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:55.768 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:55.768 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:55.768 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:55.769 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:55.769 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:55.769 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:55.769 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:55.769 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:55.769 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:55.769 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:55.769 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.769 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.769 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.769 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:24:55.769 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:55.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.769 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.770 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.770 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:24:55.770 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:55.770 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.770 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.770 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:55.770 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.770 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.770 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.770 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:24:55.770 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:55.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.771 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.771 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.771 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:24:55.771 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:55.771 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.771 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.771 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.771 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:55.771 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.771 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.771 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.771 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:24:55.771 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:55.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.772 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.772 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.772 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:24:55.772 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:55.772 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.772 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.772 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.772 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:55.772 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.772 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.772 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.773 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:24:55.773 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:55.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.773 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.773 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.773 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:24:55.773 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:55.773 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.773 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.773 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:24:55.773 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:24:55.773 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:24:55.774 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:24:55.774 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:55.774 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:55.774 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:55.774 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.774 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:24:55.774 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:55.774 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:55.774 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:55.775 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:55.775 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:55.775 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:55.775 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:55.775 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:55.775 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:55.775 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:55.775 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:55.775 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:55.775 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.775 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.775 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.775 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:24:55.776 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:55.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.776 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.776 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.776 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:24:55.776 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:55.776 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.776 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.776 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:55.776 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.776 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.776 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.776 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:24:55.777 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:55.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.777 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.777 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:24:55.777 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:55.777 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.777 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.777 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:55.777 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.777 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.777 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.777 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:24:55.777 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:55.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.778 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.778 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.778 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:24:55.778 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:55.778 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.778 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.778 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:55.778 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.778 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.778 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.778 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:24:55.779 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:55.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.779 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.779 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.779 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:24:55.779 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:55.779 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.779 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.779 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:24:55.779 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:24:55.779 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:24:55.779 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:24:55.779 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:55.780 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:55.780 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:55.780 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.780 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.780 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:24:55.780 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:55.780 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:55.780 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:55.780 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:55.780 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:55.780 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:55.780 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:55.780 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:55.781 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:55.781 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:55.781 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:55.781 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:55.781 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.781 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.781 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.781 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:24:55.781 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:55.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.781 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.781 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.781 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:24:55.782 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:55.782 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.782 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.782 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:55.782 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.782 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.782 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.782 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:24:55.782 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:55.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.782 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.782 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.783 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:24:55.783 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:55.783 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.783 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.783 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:55.783 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.783 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.783 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:24:55.783 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:55.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.784 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.784 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:24:55.784 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:55.784 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.784 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.784 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:55.784 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.784 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.784 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:24:55.784 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:55.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.785 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.785 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:24:55.785 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:55.785 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.785 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.785 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:24:55.785 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:24:55.785 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:24:55.785 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:24:55.785 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:55.786 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:55.786 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:55.786 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.786 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:24:55.786 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:55.786 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:55.786 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:55.786 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:55.786 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:55.787 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:55.787 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:55.787 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:55.787 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:55.787 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:55.787 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:55.787 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:55.787 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.787 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.787 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.787 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:24:55.788 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:55.788 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.788 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.788 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:24:55.788 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:55.788 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.788 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.788 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.788 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:55.788 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.788 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.788 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:24:55.789 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:55.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.789 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.789 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.789 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:24:55.789 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:55.789 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.789 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.789 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:55.789 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.789 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.790 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.790 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:24:55.790 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:55.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.790 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.790 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.790 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:24:55.790 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:55.790 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.790 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.790 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:55.791 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:55.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.791 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:55.791 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:24:55.791 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:55.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.791 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:55.791 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:24:55.791 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:55.791 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.791 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.792 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:24:55.792 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:24:55.792 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:24:55.792 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:24:55.792 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:55.792 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:55.792 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:55.792 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:55.792 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.792 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:55.792 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:24:55.793 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.793 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:24:55.793 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:55.793 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:55.793 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:55.793 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:55.793 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:55.793 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:24:55.793 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:55.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.793 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.793 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:24:55.794 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:55.794 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:55.794 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:55.794 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:55.794 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:55.794 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:24:55.794 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:55.794 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.794 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.794 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:24:55.794 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:55.794 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:55.794 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:55.795 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:55.795 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:55.795 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:24:55.795 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:55.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.795 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.795 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:24:55.795 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:55.795 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:55.795 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:55.795 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:55.795 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:55.795 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:24:55.796 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:55.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.796 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.796 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:24:55.796 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:55.796 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:55.796 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:55.796 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:55.796 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:55.796 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:24:55.796 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:55.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.796 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.797 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:24:55.797 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:55.797 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:55.797 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:55.797 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:55.797 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:55.797 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:24:55.797 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:55.797 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.797 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.797 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:24:55.797 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:55.797 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:55.797 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:55.798 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:55.798 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:55.798 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:24:55.798 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:55.798 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.798 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:55.798 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:24:55.798 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:55.798 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:55.798 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:55.798 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:55.798 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:55.798 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:24:55.799 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:55.799 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.799 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:55.799 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:24:55.799 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:24:55.799 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:24:55.799 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:24:55.799 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:24:55.799 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:24:55.799 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:24:55.799 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:24:55.799 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:24:55.799 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:24:55.799 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:24:55.800 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:24:55.800 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:24:55.800 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:24:55.800 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.800 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:55.800 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:24:55.800 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:24:55.800 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:24:55.800 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:24:55.800 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:24:55.800 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:24:55.800 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:24:55.800 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:24:55.800 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:24:55.801 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:24:55.801 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:24:55.801 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:55.801 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:24:55.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.718+0000 7fc51d000640 1 -- 172.21.15.161:0/1924580148 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc4f4066690 msgr2=0x7fc4f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:55.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.718+0000 7fc51d000640 1 --2- 172.21.15.161:0/1924580148 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc4f4066690 0x7fc4f4068b50 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc5080b1000 tx=0x7fc5080a9070 comp rx=0 tx=0).stop 2024-05-22T21:24:55.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.718+0000 7fc51d000640 1 -- 172.21.15.161:0/1924580148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc51815e130 msgr2=0x7fc51814bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:55.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.718+0000 7fc51d000640 1 --2- 172.21.15.161:0/1924580148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc51815e130 0x7fc51814bab0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fc510002240 tx=0x7fc51000cbe0 comp rx=0 tx=0).stop 2024-05-22T21:24:55.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.718+0000 7fc51d000640 1 -- 172.21.15.161:0/1924580148 shutdown_connections 2024-05-22T21:24:55.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.718+0000 7fc51d000640 1 --2- 172.21.15.161:0/1924580148 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fc4f4066690 0x7fc4f4068b50 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:55.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.718+0000 7fc51d000640 1 --2- 172.21.15.161:0/1924580148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc51815e130 0x7fc51814bab0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:55.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.718+0000 7fc51d000640 1 --2- 172.21.15.161:0/1924580148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc51815dd50 0x7fc5180b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:55.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.718+0000 7fc51d000640 1 --2- 172.21.15.161:0/1924580148 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc51815d410 0x7fc5180cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:55.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.718+0000 7fc51d000640 1 -- 172.21.15.161:0/1924580148 >> 172.21.15.161:0/1924580148 conn(0x7fc5180b0bc0 msgr2=0x7fc518157d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:55.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.718+0000 7fc51d000640 1 -- 172.21.15.161:0/1924580148 shutdown_connections 2024-05-22T21:24:55.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.718+0000 7fc51d000640 1 -- 172.21.15.161:0/1924580148 wait complete. 2024-05-22T21:24:55.804 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:24:55.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.938+0000 7f584e200640 1 Processor -- start 2024-05-22T21:24:55.941 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.938+0000 7f584e200640 1 -- start start 2024-05-22T21:24:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f584e200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5848161ec0 0x7f584806c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f584e200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58481612b0 0x7f5848161740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f584e200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f584806b880 con 0x7f584815d8e0 2024-05-22T21:24:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f584e200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5848161c80 con 0x7f58481612b0 2024-05-22T21:24:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f584e200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584806bb60 con 0x7f5848161ec0 2024-05-22T21:24:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5847400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f584815d8e0 0x7f58480cf790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:48098/0 (socket says 172.21.15.161:48098) 2024-05-22T21:24:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5847400640 1 -- 172.21.15.161:0/1729298454 learned_addr learned my addr 172.21.15.161:0/1729298454 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:24:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f583fe00640 1 --2- 172.21.15.161:0/1729298454 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5848161ec0 0x7f584806c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5847e00640 1 --2- 172.21.15.161:0/1729298454 >> v2:172.21.15.179:3300/0 conn(0x7f58481612b0 0x7f5848161740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5846a00640 1 -- 172.21.15.161:0/1729298454 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818915385 0 0) 0x7f584806b880 con 0x7f584815d8e0 2024-05-22T21:24:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5846a00640 1 -- 172.21.15.161:0/1729298454 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f582c003530 con 0x7f584815d8e0 2024-05-22T21:24:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5846a00640 1 -- 172.21.15.161:0/1729298454 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3795487177 0 0) 0x7f582c003530 con 0x7f584815d8e0 2024-05-22T21:24:55.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5846a00640 1 -- 172.21.15.161:0/1729298454 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5848161ec0 msgr2=0x7f584806c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5846a00640 1 --2- 172.21.15.161:0/1729298454 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5848161ec0 0x7f584806c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5846a00640 1 -- 172.21.15.161:0/1729298454 >> v2:172.21.15.179:3300/0 conn(0x7f58481612b0 msgr2=0x7f5848161740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5846a00640 1 --2- 172.21.15.161:0/1729298454 >> v2:172.21.15.179:3300/0 conn(0x7f58481612b0 0x7f5848161740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5846a00640 1 -- 172.21.15.161:0/1729298454 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584806be40 con 0x7f584815d8e0 2024-05-22T21:24:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5846a00640 1 -- 172.21.15.161:0/1729298454 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f5838001680 con 0x7f584815d8e0 2024-05-22T21:24:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5846a00640 1 -- 172.21.15.161:0/1729298454 >> v1:172.21.15.161:6789/0 conn(0x7f584815d8e0 legacy=0x7f58480cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5846a00640 1 --2- 172.21.15.161:0/1729298454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c003980 0x7f582c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5846a00640 1 --2- 172.21.15.161:0/1729298454 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5848161ec0 0x7f582c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5846a00640 1 --2- 172.21.15.161:0/1729298454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f582c004d10 0x7f582c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5846a00640 1 -- 172.21.15.161:0/1729298454 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5848161c80 con 0x7f582c003980 2024-05-22T21:24:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5846a00640 1 -- 172.21.15.161:0/1729298454 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584806bb60 con 0x7f582c004d10 2024-05-22T21:24:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5846a00640 1 -- 172.21.15.161:0/1729298454 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c00b990 con 0x7f5848161ec0 2024-05-22T21:24:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5847400640 1 --2- 172.21.15.161:0/1729298454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c003980 0x7f582c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:55.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f583fe00640 1 --2- 172.21.15.161:0/1729298454 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5848161ec0 0x7f582c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:55.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5847e00640 1 --2- 172.21.15.161:0/1729298454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f582c004d10 0x7f582c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:55.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5847400640 1 -- 172.21.15.161:0/1729298454 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5848161ec0 msgr2=0x7f582c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:55.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5847400640 1 --2- 172.21.15.161:0/1729298454 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5848161ec0 0x7f582c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:55.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5847400640 1 -- 172.21.15.161:0/1729298454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f582c004d10 msgr2=0x7f582c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:55.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5847400640 1 --2- 172.21.15.161:0/1729298454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f582c004d10 0x7f582c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:55.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.942+0000 7f5847400640 1 -- 172.21.15.161:0/1729298454 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c003530 con 0x7f582c003980 2024-05-22T21:24:55.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.946+0000 7f5847400640 1 --2- 172.21.15.161:0/1729298454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c003980 0x7f582c003da0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f583800c290 tx=0x7f583801cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275177198d7f2580 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:55.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.946+0000 7f5846a00640 1 -- 172.21.15.161:0/1729298454 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58380058e0 con 0x7f582c003980 2024-05-22T21:24:55.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.946+0000 7f5846a00640 1 -- 172.21.15.161:0/1729298454 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5838005a40 con 0x7f582c003980 2024-05-22T21:24:55.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.946+0000 7f5846a00640 1 -- 172.21.15.161:0/1729298454 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838005df0 con 0x7f582c003980 2024-05-22T21:24:55.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.946+0000 7f584e200640 1 -- 172.21.15.161:0/1729298454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c003980 msgr2=0x7f582c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:55.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.946+0000 7f584e200640 1 --2- 172.21.15.161:0/1729298454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c003980 0x7f582c003da0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f583800c290 tx=0x7f583801cd00 comp rx=0 tx=0).stop 2024-05-22T21:24:55.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.946+0000 7f5847e00640 1 -- 172.21.15.161:0/1729298454 reap_dead start 2024-05-22T21:24:55.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.946+0000 7f584e200640 1 -- 172.21.15.161:0/1729298454 shutdown_connections 2024-05-22T21:24:55.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.946+0000 7f584e200640 1 -- 172.21.15.161:0/1729298454 >> 172.21.15.161:0/1729298454 conn(0x7f58480d0eb0 msgr2=0x7f58480d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:55.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.946+0000 7f584e200640 1 -- 172.21.15.161:0/1729298454 shutdown_connections 2024-05-22T21:24:55.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.946+0000 7f584e200640 1 -- 172.21.15.161:0/1729298454 wait complete. 2024-05-22T21:24:55.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.946+0000 7f584e200640 1 Processor -- start 2024-05-22T21:24:55.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.946+0000 7f584e200640 1 -- start start 2024-05-22T21:24:55.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f584e200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c003980 0x7f5848154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:55.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f584e200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f584815bb00 0x7f584815bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:55.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f584e200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f584809c4e0 0x7f584809c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:55.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f584e200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584809cea0 con 0x7f582c003980 2024-05-22T21:24:55.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f584e200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5848156310 con 0x7f584809c4e0 2024-05-22T21:24:55.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f584e200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584809ae30 con 0x7f584815bb00 2024-05-22T21:24:55.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f5847400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c003980 0x7f5848154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:55.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f5847400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c003980 0x7f5848154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35880/0 (socket says 172.21.15.161:35880) 2024-05-22T21:24:55.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f5847400640 1 -- 172.21.15.161:0/2613070120 learned_addr learned my addr 172.21.15.161:0/2613070120 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:55.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f5847e00640 1 --2- 172.21.15.161:0/2613070120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f584809c4e0 0x7f584809c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:55.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f583fe00640 1 --2- 172.21.15.161:0/2613070120 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f584815bb00 0x7f584815bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:55.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f5847400640 1 -- 172.21.15.161:0/2613070120 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f584815bb00 msgr2=0x7f584815bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:55.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f5847400640 1 --2- 172.21.15.161:0/2613070120 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f584815bb00 0x7f584815bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:55.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f5847400640 1 -- 172.21.15.161:0/2613070120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f584809c4e0 msgr2=0x7f584809c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:55.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f5847400640 1 --2- 172.21.15.161:0/2613070120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f584809c4e0 0x7f584809c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:55.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f5847400640 1 -- 172.21.15.161:0/2613070120 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58481565b0 con 0x7f582c003980 2024-05-22T21:24:55.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f583fe00640 1 --2- 172.21.15.161:0/2613070120 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f584815bb00 0x7f584815bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:55.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f5847e00640 1 --2- 172.21.15.161:0/2613070120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f584809c4e0 0x7f584809c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:24:55.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f5847400640 1 --2- 172.21.15.161:0/2613070120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c003980 0x7f5848154fa0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f58380ab040 tx=0x7f582c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:55.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f583f400640 1 -- 172.21.15.161:0/2613070120 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58380047f0 con 0x7f582c003980 2024-05-22T21:24:55.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f583f400640 1 -- 172.21.15.161:0/2613070120 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5838004950 con 0x7f582c003980 2024-05-22T21:24:55.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f584e200640 1 -- 172.21.15.161:0/2613070120 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5848156890 con 0x7f582c003980 2024-05-22T21:24:55.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f583f400640 1 -- 172.21.15.161:0/2613070120 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838004d00 con 0x7f582c003980 2024-05-22T21:24:55.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f584e200640 1 -- 172.21.15.161:0/2613070120 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584809eb80 con 0x7f582c003980 2024-05-22T21:24:55.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f584e200640 1 -- 172.21.15.161:0/2613070120 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5848067220 con 0x7f582c003980 2024-05-22T21:24:55.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.950+0000 7f583f400640 1 -- 172.21.15.161:0/2613070120 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5838007070 con 0x7f582c003980 2024-05-22T21:24:55.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.954+0000 7f583f400640 1 --2- 172.21.15.161:0/2613070120 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5824066690 0x7f5824068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:55.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.954+0000 7f583f400640 1 -- 172.21.15.161:0/2613070120 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f583811dad0 con 0x7f582c003980 2024-05-22T21:24:55.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.954+0000 7f583fe00640 1 --2- 172.21.15.161:0/2613070120 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5824066690 0x7f5824068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:55.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.954+0000 7f583f400640 1 -- 172.21.15.161:0/2613070120 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=193}) v3 -- 0x7f582406e1c0 con 0x7f582c003980 2024-05-22T21:24:55.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.954+0000 7f583fe00640 1 --2- 172.21.15.161:0/2613070120 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5824066690 0x7f5824068b50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f58480c7c10 tx=0x7f5830008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:55.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:55.958+0000 7f583f400640 1 -- 172.21.15.161:0/2613070120 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f58380eada0 con 0x7f582c003980 2024-05-22T21:24:56.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.142+0000 7f584e200640 1 -- 172.21.15.161:0/2613070120 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58480b6e30 con 0x7f582c003980 2024-05-22T21:24:56.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.142+0000 7f583f400640 1 -- 172.21.15.161:0/2613070120 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4744 (secure 0 0 0) 0x7f58380adea0 con 0x7f582c003980 2024-05-22T21:24:56.145 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:56.145 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":86,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3304182060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3304182060},{"type":"v1","addr":"172.21.15.179:6835","nonce":3304182060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196309272},{"type":"v1","addr":"172.21.15.161:6836","nonce":196309272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/396408969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":396408969},{"type":"v1","addr":"172.21.15.179:6834","nonce":396408969}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:24:34.096673+0000","modified":"2024-05-22T21:24:37.112845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8784},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8784":{"gid":8784,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2172511349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2172511349},{"type":"v1","addr":"172.21.15.161:6837","nonce":2172511349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-05-22T21:24:56.145 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 86 2024-05-22T21:24:56.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.146+0000 7f584e200640 1 -- 172.21.15.161:0/2613070120 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5824066690 msgr2=0x7f5824068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:56.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.146+0000 7f584e200640 1 --2- 172.21.15.161:0/2613070120 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5824066690 0x7f5824068b50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f58480c7c10 tx=0x7f5830008040 comp rx=0 tx=0).stop 2024-05-22T21:24:56.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.146+0000 7f584e200640 1 -- 172.21.15.161:0/2613070120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c003980 msgr2=0x7f5848154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:56.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.146+0000 7f584e200640 1 --2- 172.21.15.161:0/2613070120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c003980 0x7f5848154fa0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f58380ab040 tx=0x7f582c005800 comp rx=0 tx=0).stop 2024-05-22T21:24:56.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.146+0000 7f584e200640 1 -- 172.21.15.161:0/2613070120 shutdown_connections 2024-05-22T21:24:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.146+0000 7f584e200640 1 --2- 172.21.15.161:0/2613070120 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5824066690 0x7f5824068b50 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.146+0000 7f584e200640 1 --2- 172.21.15.161:0/2613070120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f584809c4e0 0x7f584809c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.146+0000 7f584e200640 1 --2- 172.21.15.161:0/2613070120 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f584815bb00 0x7f584815bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.146+0000 7f584e200640 1 --2- 172.21.15.161:0/2613070120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c003980 0x7f5848154fa0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.146+0000 7f584e200640 1 -- 172.21.15.161:0/2613070120 >> 172.21.15.161:0/2613070120 conn(0x7f58480d0eb0 msgr2=0x7f5848156e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.146+0000 7f584e200640 1 -- 172.21.15.161:0/2613070120 shutdown_connections 2024-05-22T21:24:56.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.146+0000 7f584e200640 1 -- 172.21.15.161:0/2613070120 wait complete. 2024-05-22T21:24:56.166 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-05-22T21:24:56.322 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":8784,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":22.210952757000001,"mdsmap_epoch":86,"osdmap_epoch":192,"osdmap_epoch_barrier":192,"uptime":27.460961865000002} 2024-05-22T21:24:56.323 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 8784, "mdsmap_epoch": 86, "osdmap_epoch": 192, "osdmap_epoch_barrier": 192, "rank_uptime": 22.210952757, "state": "up:active", "uptime": 27.460961865, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:24:56.323 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-22T21:24:56.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:56.323 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-05-22T21:24:56.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:24:56.333 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-05-22T21:24:56.401 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:24:56.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.582+0000 7f0911200640 1 Processor -- start 2024-05-22T21:24:56.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.582+0000 7f0911200640 1 -- start start 2024-05-22T21:24:56.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.586+0000 7f0911200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f090c15d430 0x7f090c0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:56.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.586+0000 7f0911200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f090c0accf0 0x7f090c160e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.586+0000 7f0911200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090c071fd0 con 0x7f090c0ac8b0 2024-05-22T21:24:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.586+0000 7f0911200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090c070e80 con 0x7f090c15d430 2024-05-22T21:24:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.586+0000 7f0911200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090c06c510 con 0x7f090c0accf0 2024-05-22T21:24:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.586+0000 7f0909e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f090c15d430 0x7f090c0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.586+0000 7f0909e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f090c15d430 0x7f090c0707c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35894/0 (socket says 172.21.15.161:35894) 2024-05-22T21:24:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.586+0000 7f0909e00640 1 -- 172.21.15.161:0/2454580940 learned_addr learned my addr 172.21.15.161:0/2454580940 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.586+0000 7f090a800640 1 --2- 172.21.15.161:0/2454580940 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f090c0accf0 0x7f090c160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.586+0000 7f0909e00640 1 -- 172.21.15.161:0/2454580940 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f090c0accf0 msgr2=0x7f090c160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.586+0000 7f0909e00640 1 --2- 172.21.15.161:0/2454580940 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f090c0accf0 0x7f090c160e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.586+0000 7f0909e00640 1 -- 172.21.15.161:0/2454580940 >> v1:172.21.15.179:6789/0 conn(0x7f090c0ac8b0 legacy=0x7f090c06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:24:56.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.586+0000 7f0909e00640 1 -- 172.21.15.161:0/2454580940 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c06c7f0 con 0x7f090c15d430 2024-05-22T21:24:56.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0909e00640 1 --2- 172.21.15.161:0/2454580940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c15d430 0x7f090c0707c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009e50 tx=0x7f08fc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd74b8af7e4200c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:56.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0909400640 1 -- 172.21.15.161:0/2454580940 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc0a56c0 con 0x7f090c15d430 2024-05-22T21:24:56.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0909400640 1 -- 172.21.15.161:0/2454580940 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc0a5820 con 0x7f090c15d430 2024-05-22T21:24:56.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0909400640 1 -- 172.21.15.161:0/2454580940 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc0a5bb0 con 0x7f090c15d430 2024-05-22T21:24:56.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0911200640 1 -- 172.21.15.161:0/2454580940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c15d430 msgr2=0x7f090c0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:56.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0911200640 1 --2- 172.21.15.161:0/2454580940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c15d430 0x7f090c0707c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009e50 tx=0x7f08fc01c930 comp rx=0 tx=0).stop 2024-05-22T21:24:56.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0911200640 1 -- 172.21.15.161:0/2454580940 shutdown_connections 2024-05-22T21:24:56.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0911200640 1 --2- 172.21.15.161:0/2454580940 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f090c0accf0 0x7f090c160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:56.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0911200640 1 --2- 172.21.15.161:0/2454580940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c15d430 0x7f090c0707c0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:56.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0911200640 1 -- 172.21.15.161:0/2454580940 >> 172.21.15.161:0/2454580940 conn(0x7f090c0b0bc0 msgr2=0x7f090c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:56.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0911200640 1 -- 172.21.15.161:0/2454580940 shutdown_connections 2024-05-22T21:24:56.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0911200640 1 -- 172.21.15.161:0/2454580940 wait complete. 2024-05-22T21:24:56.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0911200640 1 Processor -- start 2024-05-22T21:24:56.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0911200640 1 -- start start 2024-05-22T21:24:56.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0911200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f090c0ac8b0 0x7f090c15c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:56.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0911200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c0accf0 0x7f090c14d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:56.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.590+0000 7f0911200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f090c15d430 0x7f090c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:56.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0911200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c14dea0 con 0x7f090c0accf0 2024-05-22T21:24:56.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0911200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c14e410 con 0x7f090c15d430 2024-05-22T21:24:56.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0911200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090c1570a0 con 0x7f090c0ac8b0 2024-05-22T21:24:56.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0901400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c0accf0 0x7f090c14d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:56.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0901400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c0accf0 0x7f090c14d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35906/0 (socket says 172.21.15.161:35906) 2024-05-22T21:24:56.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0901400640 1 -- 172.21.15.161:0/1097958741 learned_addr learned my addr 172.21.15.161:0/1097958741 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:24:56.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0909e00640 1 --2- 172.21.15.161:0/1097958741 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f090c0ac8b0 0x7f090c15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:56.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f090a800640 1 --2- 172.21.15.161:0/1097958741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f090c15d430 0x7f090c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:56.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0909e00640 1 -- 172.21.15.161:0/1097958741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f090c15d430 msgr2=0x7f090c15ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:56.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0909e00640 1 --2- 172.21.15.161:0/1097958741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f090c15d430 0x7f090c15ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:56.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0909e00640 1 -- 172.21.15.161:0/1097958741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c0accf0 msgr2=0x7f090c14d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:56.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0909e00640 1 --2- 172.21.15.161:0/1097958741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c0accf0 0x7f090c14d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:56.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0909e00640 1 -- 172.21.15.161:0/1097958741 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc009b30 con 0x7f090c0ac8b0 2024-05-22T21:24:56.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0909e00640 1 --2- 172.21.15.161:0/1097958741 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f090c0ac8b0 0x7f090c15c810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0ab000 tx=0x7f08fc0196d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:56.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0902a00640 1 -- 172.21.15.161:0/1097958741 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc0b8670 con 0x7f090c0ac8b0 2024-05-22T21:24:56.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0902a00640 1 -- 172.21.15.161:0/1097958741 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc0b87f0 con 0x7f090c0ac8b0 2024-05-22T21:24:56.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0902a00640 1 -- 172.21.15.161:0/1097958741 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc0b8ba0 con 0x7f090c0ac8b0 2024-05-22T21:24:56.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0911200640 1 -- 172.21.15.161:0/1097958741 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090c157380 con 0x7f090c0ac8b0 2024-05-22T21:24:56.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0911200640 1 -- 172.21.15.161:0/1097958741 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c14fcd0 con 0x7f090c0ac8b0 2024-05-22T21:24:56.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0911200640 1 -- 172.21.15.161:0/1097958741 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090c067220 con 0x7f090c0ac8b0 2024-05-22T21:24:56.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.594+0000 7f0902a00640 1 -- 172.21.15.161:0/1097958741 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f08fc0b8d00 con 0x7f090c0ac8b0 2024-05-22T21:24:56.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.598+0000 7f0902a00640 1 --2- 172.21.15.161:0/1097958741 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f08e8066690 0x7f08e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:24:56.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.598+0000 7f0902a00640 1 -- 172.21.15.161:0/1097958741 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08fc11fc20 con 0x7f090c0ac8b0 2024-05-22T21:24:56.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.598+0000 7f0901400640 1 --2- 172.21.15.161:0/1097958741 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f08e8066690 0x7f08e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:24:56.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.598+0000 7f0902a00640 1 -- 172.21.15.161:0/1097958741 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=193}) v3 -- 0x7f08e806e1c0 con 0x7f090c0ac8b0 2024-05-22T21:24:56.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.598+0000 7f0901400640 1 --2- 172.21.15.161:0/1097958741 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f08e8066690 0x7f08e8068b50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f090c15e980 tx=0x7f08f4009110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:24:56.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.602+0000 7f0902a00640 1 -- 172.21.15.161:0/1097958741 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f08fc0ed150 con 0x7f090c0ac8b0 2024-05-22T21:24:56.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.754+0000 7f0911200640 1 -- 172.21.15.161:0/1097958741 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f090c0ae7a0 con 0x7f090c0ac8b0 2024-05-22T21:24:56.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.758+0000 7f0902a00640 1 -- 172.21.15.161:0/1097958741 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v192) v1 ==== 81+0+33613 (secure 0 0 0) 0x7f08fc0bc030 con 0x7f090c0ac8b0 2024-05-22T21:24:56.758 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:24:56.758 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:24:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 192, 2024-05-22T21:24:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:24:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:24:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:24:50.309408+0000", 2024-05-22T21:24:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:24:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:24:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:24:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:24:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:24:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:24:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:24:56.759 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:24:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:24:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:24:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:24:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:24:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:24:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:24:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 23, 2024-05-22T21:24:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:24:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:24:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:24:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:24:56.760 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:24:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:24:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:24:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:24:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:24:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:24:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:24:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:24:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:24:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:24:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:24:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:24:56.761 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:24:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:24:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:24:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:24:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:24:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:24:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:24:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:24:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:24:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:24:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:24:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:24:56.762 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:24:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:24:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:24:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:24:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:24:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:24:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:24:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:24:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:24:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:24:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:24:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:24:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:24:56.763 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:24:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:24:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:24:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:24:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:24:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:24:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:24:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:24:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:24:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:24:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:24:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:24:56.764 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:24:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:24:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:24:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:24:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:24:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:24:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:24:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:24:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:24:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:24:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:24:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:24:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:24:56.765 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:24:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:24:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:24:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:24:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:24:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:24:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:24:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:24:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:24:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:24:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:24:56.766 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:24:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:24:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:24:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:24:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:24:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:24:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:24:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:24:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 22, 2024-05-22T21:24:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:24:56.767 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:24:31.797440+0000", 2024-05-22T21:24:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:24:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:24:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:24:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:24:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:24:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:24:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:24:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:24:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:24:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:24:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:24:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:24:56.768 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:24:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:24:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:24:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:24:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:24:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:24:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:24:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:24:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:24:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:24:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:24:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:24:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "189", 2024-05-22T21:24:56.769 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:24:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:24:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:24:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:24:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:24:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:24:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:24:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:24:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:24:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:24:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:24:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:24:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:24:56.770 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:24:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:24:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:24:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:24:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:24:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:24:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:24:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:24:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:24:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:24:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:24:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:24:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:24:56.771 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:24:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:24:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:24:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:24:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:24:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:24:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:24:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:24:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:24:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:24:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:24:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:24:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:24:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:24:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:24:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:24:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:24:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:24:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:24:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 2, 2024-05-22T21:24:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 2, 2024-05-22T21:24:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:24:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:24:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:24:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:24:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:24:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:24:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 23, 2024-05-22T21:24:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:24:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:24:33.098240+0000", 2024-05-22T21:24:56.774 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1027, 2024-05-22T21:24:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full,full_quota", 2024-05-22T21:24:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:24:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:24:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:24:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:24:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:24:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:24:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:24:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:24:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:24:56.775 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:24:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:24:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:24:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:24:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:24:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:24:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:24:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:24:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:24:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:24:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:24:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:24:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:24:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.776 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "192", 2024-05-22T21:24:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:24:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:24:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:24:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:24:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:24:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:24:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:24:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:24:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:24:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 33554432, 2024-05-22T21:24:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:24:56.777 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:24:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:24:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:24:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:24:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:24:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:24:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:24:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:24:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:24:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:24:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:24:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:24:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:24:56.778 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:24:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:24:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:24:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:24:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:24:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:24:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:24:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:24:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:24:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:24:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:24:56.779 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:24:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:24:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:24:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:24:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:24:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:24:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:24:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:24:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:24:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.3799999952316284, 2024-05-22T21:24:56.780 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.3799999952316284, 2024-05-22T21:24:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:24:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.3799999952316284, 2024-05-22T21:24:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.3799999952316284, 2024-05-22T21:24:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:24:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:24:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:24:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:24:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:24:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:56.781 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.782 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:24:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:24:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:56.783 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:24:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:24:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:56.784 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:24:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:24:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.785 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:24:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:24:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:24:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:24:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:24:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:24:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:24:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:24:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:56.787 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.788 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:24:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:24:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:24:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:24:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:24:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:24:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:24:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:24:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:24:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.792 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:24:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:24:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:24:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:24:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.793 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:24:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:56.794 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:24:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:24:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.795 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:24:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:24:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:24:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:24:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.797 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:24:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:24:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:24:56.798 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:24:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:24:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:24:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:24:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.799 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:24:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:56.800 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:24:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:24:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.801 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:24:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:24:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:56.802 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:24:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.803 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:24:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:24:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:56.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:24:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:24:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:24:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:24:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:24:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:24:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:56.805 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:24:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:56.806 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:24:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:24:56.807 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:24:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.808 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:24:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:24:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.809 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:24:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:24:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.810 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:24:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:24:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:24:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:24:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:24:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:24:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:56.811 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:56.812 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.812 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.812 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.812 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:24:56.812 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:56.812 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:56.812 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:56.812 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:56.812 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:56.812 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:56.812 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:56.812 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:56.812 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:56.813 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:56.813 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:56.813 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:56.813 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.813 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.813 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.813 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:24:56.813 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:56.813 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.813 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.813 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.813 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:24:56.813 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:56.814 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.814 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.814 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.814 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:56.814 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.814 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.814 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.814 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:24:56.814 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:56.814 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.814 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.814 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.814 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:24:56.815 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:56.815 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.815 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.815 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.815 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:56.815 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.815 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.815 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.815 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:24:56.815 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:56.815 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.816 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.816 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.816 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:24:56.816 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:56.816 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.816 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.816 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.816 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:56.816 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.816 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.816 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.816 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:24:56.816 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:56.816 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.817 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.817 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.817 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:24:56.817 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:24:56.817 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.817 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.817 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.817 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:24:56.817 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:24:56.817 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:24:56.817 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:24:56.817 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:56.817 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:56.818 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:56.818 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.818 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.818 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.818 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:24:56.818 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:56.818 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:56.818 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:56.818 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:56.818 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:56.818 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:56.818 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:56.818 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:56.819 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:56.819 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:56.819 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:56.819 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:56.819 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.819 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.819 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.819 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:24:56.819 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:56.819 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.819 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.819 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.819 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:24:56.820 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:56.820 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.820 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.820 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:56.820 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.820 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.820 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.820 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:24:56.820 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:56.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.820 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.820 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.821 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:24:56.821 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:56.821 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.821 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.821 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:56.821 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.821 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.821 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.821 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:24:56.821 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:56.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.821 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.822 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.822 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:24:56.822 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:56.822 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.822 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.822 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:56.822 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.822 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.822 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.822 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:24:56.822 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:56.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.823 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.823 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.823 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:24:56.823 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:24:56.823 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.823 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.823 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.823 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:24:56.823 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:24:56.823 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:24:56.823 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:24:56.823 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:56.824 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:56.824 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:56.824 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.824 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.824 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:24:56.824 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:24:56.824 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:24:56.824 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:24:56.824 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:24:56.824 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:24:56.824 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:24:56.824 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:24:56.825 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:24:56.825 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:24:56.825 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:24:56.825 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:24:56.825 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:24:56.825 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.825 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.825 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.825 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:24:56.825 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:56.825 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.825 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.825 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.826 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:24:56.826 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:56.826 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.826 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.826 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:24:56.826 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.826 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.826 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.826 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:24:56.826 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:56.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.827 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.827 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.827 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:24:56.827 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:56.827 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.827 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.827 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.827 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:24:56.827 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.827 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.827 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.827 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:24:56.827 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:56.827 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.828 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.828 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.828 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:24:56.828 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:56.828 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.828 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.828 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.828 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:24:56.828 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:24:56.828 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.828 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:24:56.829 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:24:56.829 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:56.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.829 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.829 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:24:56.829 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:24:56.829 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:24:56.829 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.829 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.829 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:24:56.829 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:24:56.829 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:24:56.830 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:24:56.830 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:24:56.830 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:24:56.830 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:24:56.830 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:24:56.830 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.830 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:56.830 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:24:56.830 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.830 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:24:56.830 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:56.831 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:56.831 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:56.831 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:56.831 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:56.831 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:24:56.831 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:56.831 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.831 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.831 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:24:56.831 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:56.831 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:56.831 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:56.831 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:56.831 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:56.832 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:24:56.832 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:56.832 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.832 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.832 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:24:56.832 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:56.832 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:56.832 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:56.832 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:56.832 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:56.832 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:24:56.833 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:56.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.833 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.833 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:24:56.833 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:56.833 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:56.833 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:56.833 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:56.833 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:56.833 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:24:56.833 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:56.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.834 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.834 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:24:56.834 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:56.834 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:56.834 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:56.834 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:56.834 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:56.834 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:24:56.834 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:56.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.834 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.835 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:24:56.835 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:56.835 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:56.835 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:56.835 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:56.835 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:56.835 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:24:56.835 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:56.835 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.835 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.835 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:24:56.835 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:56.835 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:56.836 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:56.836 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:56.836 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:56.836 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:24:56.836 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:56.836 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.836 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:24:56.836 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:24:56.836 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:24:56.836 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:24:56.836 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:24:56.836 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:24:56.836 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:24:56.837 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:24:56.837 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:24:56.837 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.837 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:24:56.837 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:24:56.837 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:24:56.837 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:24:56.837 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:24:56.837 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:24:56.837 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:24:56.837 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:24:56.837 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:24:56.837 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:24:56.837 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:24:56.838 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:24:56.838 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:24:56.838 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:24:56.838 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:24:56.838 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.838 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:24:56.838 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:24:56.838 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:24:56.838 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:24:56.838 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:24:56.838 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:24:56.838 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:24:56.838 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:24:56.839 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:24:56.839 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:24:56.839 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:24:56.839 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:24:56.839 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:24:56.839 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:24:56.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.758+0000 7f0911200640 1 -- 172.21.15.161:0/1097958741 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f08e8066690 msgr2=0x7f08e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:56.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.758+0000 7f0911200640 1 --2- 172.21.15.161:0/1097958741 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f08e8066690 0x7f08e8068b50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f090c15e980 tx=0x7f08f4009110 comp rx=0 tx=0).stop 2024-05-22T21:24:56.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.758+0000 7f0911200640 1 -- 172.21.15.161:0/1097958741 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f090c0ac8b0 msgr2=0x7f090c15c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:24:56.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.758+0000 7f0911200640 1 --2- 172.21.15.161:0/1097958741 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f090c0ac8b0 0x7f090c15c810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0ab000 tx=0x7f08fc0196d0 comp rx=0 tx=0).stop 2024-05-22T21:24:56.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.758+0000 7f0911200640 1 -- 172.21.15.161:0/1097958741 shutdown_connections 2024-05-22T21:24:56.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.758+0000 7f0911200640 1 --2- 172.21.15.161:0/1097958741 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f08e8066690 0x7f08e8068b50 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:56.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.758+0000 7f0911200640 1 --2- 172.21.15.161:0/1097958741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f090c15d430 0x7f090c15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:56.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.758+0000 7f0911200640 1 --2- 172.21.15.161:0/1097958741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f090c0accf0 0x7f090c14d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:56.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.758+0000 7f0911200640 1 --2- 172.21.15.161:0/1097958741 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f090c0ac8b0 0x7f090c15c810 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:24:56.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.758+0000 7f0911200640 1 -- 172.21.15.161:0/1097958741 >> 172.21.15.161:0/1097958741 conn(0x7f090c0b0bc0 msgr2=0x7f090c0af100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:24:56.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.758+0000 7f0911200640 1 -- 172.21.15.161:0/1097958741 shutdown_connections 2024-05-22T21:24:56.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:24:56.758+0000 7f0911200640 1 -- 172.21.15.161:0/1097958741 wait complete. 2024-05-22T21:24:56.842 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-05-22T21:25:01.843 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:25:01.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe74de00640 1 Processor -- start 2024-05-22T21:25:01.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe74de00640 1 -- start start 2024-05-22T21:25:01.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe74de00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe74806ef70 0x7fe748099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:01.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe74de00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe74806e180 0x7fe748099f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:01.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe74de00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe74809a4c0 0x7fe7481611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:01.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe74de00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe748160fc0 con 0x7fe74809a4c0 2024-05-22T21:25:01.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe74de00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7480cf6e0 con 0x7fe74806ef70 2024-05-22T21:25:01.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe74de00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480cf9c0 con 0x7fe74806e180 2024-05-22T21:25:01.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe747400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe74809a4c0 0x7fe7481611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:01.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe747400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe74809a4c0 0x7fe7481611b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35910/0 (socket says 172.21.15.161:35910) 2024-05-22T21:25:01.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe747400640 1 -- 172.21.15.161:0/3753131880 learned_addr learned my addr 172.21.15.161:0/3753131880 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:01.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe746a00640 1 --2- 172.21.15.161:0/3753131880 >> v2:172.21.15.179:3300/0 conn(0x7fe74806ef70 0x7fe748099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:01.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe73e000640 1 --2- 172.21.15.161:0/3753131880 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe74806e180 0x7fe748099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:01.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe747400640 1 -- 172.21.15.161:0/3753131880 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe74806e180 msgr2=0x7fe748099f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:01.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe747400640 1 --2- 172.21.15.161:0/3753131880 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe74806e180 0x7fe748099f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:01.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe747400640 1 -- 172.21.15.161:0/3753131880 >> v2:172.21.15.179:3300/0 conn(0x7fe74806ef70 msgr2=0x7fe748099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:01.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe747400640 1 --2- 172.21.15.161:0/3753131880 >> v2:172.21.15.179:3300/0 conn(0x7fe74806ef70 0x7fe748099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:01.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.990+0000 7fe747400640 1 -- 172.21.15.161:0/3753131880 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7480cfca0 con 0x7fe74809a4c0 2024-05-22T21:25:01.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.994+0000 7fe747400640 1 --2- 172.21.15.161:0/3753131880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74809a4c0 0x7fe7481611b0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fe74806aa60 tx=0x7fe74001c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92374cec905ba363 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:01.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.994+0000 7fe746000640 1 -- 172.21.15.161:0/3753131880 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7400ab600 con 0x7fe74809a4c0 2024-05-22T21:25:01.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.994+0000 7fe746000640 1 -- 172.21.15.161:0/3753131880 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7400a7050 con 0x7fe74809a4c0 2024-05-22T21:25:01.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.994+0000 7fe746000640 1 -- 172.21.15.161:0/3753131880 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7400ab980 con 0x7fe74809a4c0 2024-05-22T21:25:01.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.994+0000 7fe74de00640 1 -- 172.21.15.161:0/3753131880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74809a4c0 msgr2=0x7fe7481611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:01.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.994+0000 7fe74de00640 1 --2- 172.21.15.161:0/3753131880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74809a4c0 0x7fe7481611b0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fe74806aa60 tx=0x7fe74001c230 comp rx=0 tx=0).stop 2024-05-22T21:25:01.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.994+0000 7fe74de00640 1 -- 172.21.15.161:0/3753131880 shutdown_connections 2024-05-22T21:25:01.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.994+0000 7fe74de00640 1 --2- 172.21.15.161:0/3753131880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74809a4c0 0x7fe7481611b0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:01.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.994+0000 7fe74de00640 1 --2- 172.21.15.161:0/3753131880 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe74806e180 0x7fe748099f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:01.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.994+0000 7fe74de00640 1 --2- 172.21.15.161:0/3753131880 >> v2:172.21.15.179:3300/0 conn(0x7fe74806ef70 0x7fe748099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:01.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.994+0000 7fe74de00640 1 -- 172.21.15.161:0/3753131880 >> 172.21.15.161:0/3753131880 conn(0x7fe7480b2a40 msgr2=0x7fe7480b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:01.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.994+0000 7fe74de00640 1 -- 172.21.15.161:0/3753131880 shutdown_connections 2024-05-22T21:25:01.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.994+0000 7fe74de00640 1 -- 172.21.15.161:0/3753131880 wait complete. 2024-05-22T21:25:01.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe74de00640 1 Processor -- start 2024-05-22T21:25:01.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe74de00640 1 -- start start 2024-05-22T21:25:02.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe74de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74806e180 0x7fe7480b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:02.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe74de00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe74806ef70 0x7fe74815f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:02.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe74de00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe74809a4c0 0x7fe748154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:02.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe74de00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480b5ba0 con 0x7fe74806e180 2024-05-22T21:25:02.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe74de00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480b5e80 con 0x7fe74809a4c0 2024-05-22T21:25:02.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe74de00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480b6160 con 0x7fe74806ef70 2024-05-22T21:25:02.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe746a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74806e180 0x7fe7480b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:02.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe746a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74806e180 0x7fe7480b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35926/0 (socket says 172.21.15.161:35926) 2024-05-22T21:25:02.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe746a00640 1 -- 172.21.15.161:0/2445153932 learned_addr learned my addr 172.21.15.161:0/2445153932 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:02.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe747400640 1 --2- 172.21.15.161:0/2445153932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe74809a4c0 0x7fe748154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:02.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe73e000640 1 --2- 172.21.15.161:0/2445153932 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe74806ef70 0x7fe74815f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:02.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe746a00640 1 -- 172.21.15.161:0/2445153932 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe74806ef70 msgr2=0x7fe74815f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:02.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe746a00640 1 --2- 172.21.15.161:0/2445153932 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe74806ef70 0x7fe74815f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:02.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe746a00640 1 -- 172.21.15.161:0/2445153932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe74809a4c0 msgr2=0x7fe748154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:02.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe746a00640 1 --2- 172.21.15.161:0/2445153932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe74809a4c0 0x7fe748154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:02.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe746a00640 1 -- 172.21.15.161:0/2445153932 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7400094d0 con 0x7fe74806e180 2024-05-22T21:25:02.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe746a00640 1 --2- 172.21.15.161:0/2445153932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74806e180 0x7fe7480b54c0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fe73800d800 tx=0x7fe73800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:02.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe73fe00640 1 -- 172.21.15.161:0/2445153932 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73800b6c0 con 0x7fe74806e180 2024-05-22T21:25:02.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe73fe00640 1 -- 172.21.15.161:0/2445153932 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73800b820 con 0x7fe74806e180 2024-05-22T21:25:02.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe74de00640 1 -- 172.21.15.161:0/2445153932 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7480b6440 con 0x7fe74806e180 2024-05-22T21:25:02.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe73fe00640 1 -- 172.21.15.161:0/2445153932 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73800bb90 con 0x7fe74806e180 2024-05-22T21:25:02.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:01.998+0000 7fe74de00640 1 -- 172.21.15.161:0/2445153932 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7481546b0 con 0x7fe74806e180 2024-05-22T21:25:02.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.002+0000 7fe74de00640 1 -- 172.21.15.161:0/2445153932 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe748067220 con 0x7fe74806e180 2024-05-22T21:25:02.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.002+0000 7fe73fe00640 1 -- 172.21.15.161:0/2445153932 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe73800bcf0 con 0x7fe74806e180 2024-05-22T21:25:02.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.002+0000 7fe73fe00640 1 --2- 172.21.15.161:0/2445153932 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe724066690 0x7fe724068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:02.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.002+0000 7fe73fe00640 1 -- 172.21.15.161:0/2445153932 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe738084720 con 0x7fe74806e180 2024-05-22T21:25:02.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.002+0000 7fe73fe00640 1 -- 172.21.15.161:0/2445153932 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=193}) v3 -- 0x7fe72406e1c0 con 0x7fe74806e180 2024-05-22T21:25:02.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.002+0000 7fe73e000640 1 --2- 172.21.15.161:0/2445153932 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe724066690 0x7fe724068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:02.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.002+0000 7fe73e000640 1 --2- 172.21.15.161:0/2445153932 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe724066690 0x7fe724068b50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe730003fd0 tx=0x7fe730003a00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:02.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.006+0000 7fe73fe00640 1 -- 172.21.15.161:0/2445153932 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe738051a90 con 0x7fe74806e180 2024-05-22T21:25:02.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.166+0000 7fe74de00640 1 -- 172.21.15.161:0/2445153932 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe7480b6ed0 con 0x7fe74806e180 2024-05-22T21:25:02.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.166+0000 7fe73fe00640 1 -- 172.21.15.161:0/2445153932 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v192) v1 ==== 81+0+33613 (secure 0 0 0) 0x7fe738011340 con 0x7fe74806e180 2024-05-22T21:25:02.167 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:02.167 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:25:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 192, 2024-05-22T21:25:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:25:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:25:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:24:50.309408+0000", 2024-05-22T21:25:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:25:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:25:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:25:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:25:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:25:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:25:02.168 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:25:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:25:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:25:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:25:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:25:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:25:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:25:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:25:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:25:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 23, 2024-05-22T21:25:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:25:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:25:02.169 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:25:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:25:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:25:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:25:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:25:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:25:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:25:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:25:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:25:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:25:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:25:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:25:02.170 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:25:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:25:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:25:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:25:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:25:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:25:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:25:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:25:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:25:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:25:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:25:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:25:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:25:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:25:02.171 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:25:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:25:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:25:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:25:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:25:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:25:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:25:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:25:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:25:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:25:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:25:02.172 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:25:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:25:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:25:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:25:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:25:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:25:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:25:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:25:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:25:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:25:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:25:02.173 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:25:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:25:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:25:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:25:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:25:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:25:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:25:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:25:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:25:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:25:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:25:02.174 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:25:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:25:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:25:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:25:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:25:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:25:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:25:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:25:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:25:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:25:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:25:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:25:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:25:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:25:02.175 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:25:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:25:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:25:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:25:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:25:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:25:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:25:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:25:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:25:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:25:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.176 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 22, 2024-05-22T21:25:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:25:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:24:31.797440+0000", 2024-05-22T21:25:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:25:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:25:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:25:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:25:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:25:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:25:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:25:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:25:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:25:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:25:02.177 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:25:02.178 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:25:02.178 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:25:02.178 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:25:02.178 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:25:02.178 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:25:02.178 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:25:02.178 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:25:02.178 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:25:02.178 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:25:02.178 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:25:02.178 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:25:02.178 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:25:02.178 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:25:02.179 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.179 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "189", 2024-05-22T21:25:02.179 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:25:02.179 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:25:02.179 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:25:02.179 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:25:02.179 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:25:02.179 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:25:02.179 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:25:02.179 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:25:02.179 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:25:02.179 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:25:02.179 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:25:02.180 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:25:02.180 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:25:02.180 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:25:02.180 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:25:02.180 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:25:02.180 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:25:02.180 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:25:02.180 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:25:02.180 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:25:02.180 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:25:02.180 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:25:02.180 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:25:02.180 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:25:02.180 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:25:02.180 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:25:02.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.181 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:25:02.181 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:25:02.181 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:25:02.181 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:25:02.181 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:25:02.181 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:25:02.181 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:25:02.181 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:25:02.181 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:25:02.181 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:25:02.181 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:25:02.181 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:25:02.181 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:25:02.182 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:25:02.182 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:25:02.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.182 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:25:02.182 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:25:02.182 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:25:02.182 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.182 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:25:02.182 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 2, 2024-05-22T21:25:02.182 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 2, 2024-05-22T21:25:02.182 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:25:02.183 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:25:02.183 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:25:02.183 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:25:02.183 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:25:02.183 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:25:02.183 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.183 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.183 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 23, 2024-05-22T21:25:02.183 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:25:02.183 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:24:33.098240+0000", 2024-05-22T21:25:02.183 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1027, 2024-05-22T21:25:02.183 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool,full,full_quota", 2024-05-22T21:25:02.184 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:25:02.184 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:25:02.184 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:25:02.184 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:25:02.184 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:25:02.184 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:25:02.184 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:25:02.184 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:25:02.184 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:25:02.184 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:25:02.184 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:25:02.184 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:25:02.184 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:25:02.185 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:25:02.185 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:25:02.185 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:25:02.185 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:25:02.185 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:25:02.185 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:25:02.185 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:25:02.185 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:25:02.185 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:25:02.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.185 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "192", 2024-05-22T21:25:02.185 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:25:02.185 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:25:02.185 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:25:02.185 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:25:02.186 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:25:02.186 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:25:02.186 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:25:02.186 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:25:02.186 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:25:02.186 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 33554432, 2024-05-22T21:25:02.186 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:25:02.186 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:25:02.186 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:25:02.186 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:25:02.186 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:25:02.186 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:25:02.186 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:25:02.187 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:25:02.187 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:25:02.187 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:25:02.187 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:25:02.187 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:25:02.187 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:25:02.187 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:25:02.187 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:25:02.187 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:25:02.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.187 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:25:02.187 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:25:02.187 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:25:02.188 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:25:02.188 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:25:02.188 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:25:02.188 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:25:02.188 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:25:02.188 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:25:02.188 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:25:02.188 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:25:02.188 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:25:02.188 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:25:02.188 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:25:02.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.189 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:25:02.189 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:25:02.189 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:25:02.189 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.189 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:25:02.189 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.3799999952316284, 2024-05-22T21:25:02.189 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.3799999952316284, 2024-05-22T21:25:02.189 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:25:02.189 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.3799999952316284, 2024-05-22T21:25:02.189 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.3799999952316284, 2024-05-22T21:25:02.189 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:25:02.189 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:25:02.189 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:25:02.190 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.190 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.190 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:25:02.190 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:25:02.190 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.190 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:25:02.190 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:02.190 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:02.190 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:02.190 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:02.190 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:02.190 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:02.191 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:02.191 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:02.191 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:02.191 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:02.191 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:02.191 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:02.191 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.191 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.191 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:25:02.191 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:02.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.191 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.191 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.192 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:25:02.192 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:02.192 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.192 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.192 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:02.192 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.192 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.192 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.192 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:25:02.192 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:02.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.193 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.193 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:25:02.193 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:02.193 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.193 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.193 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:02.193 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.193 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.193 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.193 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:25:02.193 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:02.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.194 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.194 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:25:02.194 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:02.194 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.194 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.194 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:02.194 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.194 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.194 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.195 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:25:02.195 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:02.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.195 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.195 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.195 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:25:02.195 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:02.195 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.195 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.195 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:25:02.195 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:25:02.195 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:25:02.195 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:25:02.196 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:02.196 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:02.196 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:02.196 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.196 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.196 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:25:02.196 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:02.196 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:02.196 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:02.196 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:02.196 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:02.196 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:02.196 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:02.197 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:02.197 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:02.197 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:02.197 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:02.197 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:02.197 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.197 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.197 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:25:02.197 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:02.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.197 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.197 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.198 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:25:02.198 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:02.198 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.198 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.198 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:02.198 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.198 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.198 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.198 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:25:02.198 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:02.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.199 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.199 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:25:02.199 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:02.199 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.199 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.199 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:02.199 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.199 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.199 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.199 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:25:02.199 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:02.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.200 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.200 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:25:02.200 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:02.200 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.200 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.200 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:02.200 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.200 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.200 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.200 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:25:02.201 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:02.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.201 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.201 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.201 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:25:02.201 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:02.201 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.201 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.201 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:25:02.201 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:25:02.201 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:25:02.201 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:25:02.202 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:02.202 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:02.202 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:02.202 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.202 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.202 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.202 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:25:02.202 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:02.202 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:02.202 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:02.202 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:02.202 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:02.203 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:02.203 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:02.203 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:02.203 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:02.203 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:02.203 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:02.203 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:02.203 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.203 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.203 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.203 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:25:02.203 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:02.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.204 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.204 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:25:02.204 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:02.204 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.204 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.204 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:02.204 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.204 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.204 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.204 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:25:02.204 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:02.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.205 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.205 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:25:02.205 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:02.205 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.205 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.205 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:02.205 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.205 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.206 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.206 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:25:02.206 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:02.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.206 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.206 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.206 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:25:02.206 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:02.206 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.206 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.206 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:02.207 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.207 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.207 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:25:02.207 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:02.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.207 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.207 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.207 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:25:02.207 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:02.207 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.208 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.208 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:25:02.208 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:25:02.208 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:25:02.208 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:25:02.208 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:02.208 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:02.208 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:02.208 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.208 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.209 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:25:02.209 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:02.209 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:02.209 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:02.209 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:02.209 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:02.209 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:02.209 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:02.209 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:02.209 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:02.209 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:02.210 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:02.210 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:02.210 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.210 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.210 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:25:02.210 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:02.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.210 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.210 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.210 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:25:02.211 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:02.211 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.211 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.211 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:02.211 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.211 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.211 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:25:02.211 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:02.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.211 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.212 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.212 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:25:02.212 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:02.212 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.212 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.212 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:02.212 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.212 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.212 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.212 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:25:02.212 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:02.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.213 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.213 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:25:02.213 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:02.213 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.213 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.213 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:02.213 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.213 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.213 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.214 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:25:02.214 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:02.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.214 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.214 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.214 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:25:02.214 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:02.214 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.214 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.214 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:25:02.214 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:25:02.215 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:25:02.215 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:25:02.215 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:02.215 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:02.215 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:02.215 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.215 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.215 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:25:02.215 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:02.215 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:02.215 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:02.215 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:02.216 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:02.216 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:02.216 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:02.216 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:02.216 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:02.216 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:02.216 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:02.216 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:02.216 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.216 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.216 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.216 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:25:02.216 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:02.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.217 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.217 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:25:02.217 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:02.217 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.217 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.217 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:02.217 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.217 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.217 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.217 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:25:02.217 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:02.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.218 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.218 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:25:02.218 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:02.218 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.218 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.218 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:02.218 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.218 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.218 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.218 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:25:02.218 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:02.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.219 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.219 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:25:02.219 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:02.219 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.219 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.219 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:02.219 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.219 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.219 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.219 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:25:02.220 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:02.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.220 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.220 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.220 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:25:02.220 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:02.220 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.220 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.220 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:25:02.220 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:25:02.220 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:25:02.220 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:25:02.220 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:02.221 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:02.221 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:02.221 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.221 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.221 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:25:02.221 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:02.221 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:02.221 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:02.221 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:02.221 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:02.221 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:02.221 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:02.221 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:02.221 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:02.222 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:02.222 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:02.222 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:02.222 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.222 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.222 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:25:02.222 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:02.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.222 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.222 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.222 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:25:02.222 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:02.223 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.223 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.223 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:02.223 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.223 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.223 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:25:02.223 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:02.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.223 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.223 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.223 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:25:02.224 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:02.224 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.224 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.224 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:02.224 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.224 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.224 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:25:02.224 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:02.224 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.224 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.224 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.224 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:25:02.225 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:02.225 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.225 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.225 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:02.225 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.225 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.225 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:25:02.225 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:02.225 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.225 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.225 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.226 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:25:02.226 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:02.226 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.226 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.226 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.226 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:25:02.226 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:25:02.226 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:25:02.226 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:25:02.226 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:02.226 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:02.226 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:02.226 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.227 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.227 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:25:02.227 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:02.227 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:02.227 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:02.227 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:02.227 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:02.227 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:02.227 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:02.227 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:02.227 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:02.227 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:02.227 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:02.228 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:02.228 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.228 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.228 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:25:02.228 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:02.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.228 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.228 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.228 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:25:02.228 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:02.228 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.228 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.228 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:02.229 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.229 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.229 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:25:02.229 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:02.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.229 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.229 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.229 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:25:02.229 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:02.229 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.229 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.229 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.229 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:02.229 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.230 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.230 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:25:02.230 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:02.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.230 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.230 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.230 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:25:02.230 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:02.230 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.230 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.230 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.230 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:02.230 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.231 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.231 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:25:02.231 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:02.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.231 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.231 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.231 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:25:02.231 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:02.231 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.231 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.231 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.231 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:25:02.231 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:25:02.231 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:25:02.232 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:25:02.232 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:02.232 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:02.232 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:02.232 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.232 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.232 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.232 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:25:02.232 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:02.232 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:02.232 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:02.232 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:02.232 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:02.232 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:02.233 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:02.233 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:02.233 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:02.233 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:02.233 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:02.233 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:02.233 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.233 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.233 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.233 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:25:02.233 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:02.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.233 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.233 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.233 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:25:02.234 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:02.234 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.234 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.234 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:02.234 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.234 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.234 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:25:02.234 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:02.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.234 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.234 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.235 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:25:02.235 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:02.235 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.235 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.235 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:02.235 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.235 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.235 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:25:02.235 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:02.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.235 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.235 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.236 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:25:02.236 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:02.236 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.236 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.236 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:02.236 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:02.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.236 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:02.236 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:25:02.236 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:02.236 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.236 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.236 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:02.236 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:25:02.237 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:02.237 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.237 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.237 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:25:02.237 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:25:02.237 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:25:02.237 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:25:02.237 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:02.237 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:02.237 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:02.237 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:02.237 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.237 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:25:02.237 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:25:02.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.238 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:25:02.238 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:02.238 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:02.238 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:02.238 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:02.238 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:02.238 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:25:02.238 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:02.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.238 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.238 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:25:02.238 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:02.239 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:02.239 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:02.239 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:02.239 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:02.239 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:25:02.239 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:02.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.239 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.239 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:25:02.239 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:02.239 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:02.239 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:02.239 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:02.239 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:02.239 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:25:02.240 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:02.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.240 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:25:02.240 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:02.240 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:02.240 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:02.240 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:02.240 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:02.240 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:25:02.240 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:02.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.240 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.240 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:25:02.241 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:02.241 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:02.241 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:02.241 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:02.241 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:02.241 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:25:02.241 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:02.241 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.241 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.241 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:25:02.241 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:02.241 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:02.241 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:02.242 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:02.242 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:02.242 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:25:02.242 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:02.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.242 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.242 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:25:02.242 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:02.242 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:02.242 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:02.242 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:02.242 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:02.242 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:25:02.243 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:02.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.243 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:02.243 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:25:02.243 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:02.243 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:02.243 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:02.243 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:02.243 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:02.243 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:25:02.243 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:02.243 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.243 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:25:02.243 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:25:02.244 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:25:02.244 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:25:02.244 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:25:02.244 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:25:02.244 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:25:02.244 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:25:02.244 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:25:02.244 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:25:02.244 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:25:02.244 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:25:02.244 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:25:02.244 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:25:02.244 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:25:02.244 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:02.245 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:25:02.245 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:25:02.245 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:25:02.245 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:25:02.245 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:25:02.245 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:25:02.245 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:25:02.245 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:25:02.245 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:25:02.245 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:25:02.245 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:25:02.245 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:02.246 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:25:02.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.166+0000 7fe74de00640 1 -- 172.21.15.161:0/2445153932 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe724066690 msgr2=0x7fe724068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:02.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.166+0000 7fe74de00640 1 --2- 172.21.15.161:0/2445153932 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe724066690 0x7fe724068b50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe730003fd0 tx=0x7fe730003a00 comp rx=0 tx=0).stop 2024-05-22T21:25:02.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.166+0000 7fe74de00640 1 -- 172.21.15.161:0/2445153932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74806e180 msgr2=0x7fe7480b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:02.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.166+0000 7fe74de00640 1 --2- 172.21.15.161:0/2445153932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74806e180 0x7fe7480b54c0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fe73800d800 tx=0x7fe73800dcc0 comp rx=0 tx=0).stop 2024-05-22T21:25:02.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.166+0000 7fe74de00640 1 -- 172.21.15.161:0/2445153932 shutdown_connections 2024-05-22T21:25:02.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.166+0000 7fe74de00640 1 --2- 172.21.15.161:0/2445153932 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe724066690 0x7fe724068b50 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:02.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.166+0000 7fe74de00640 1 --2- 172.21.15.161:0/2445153932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe74809a4c0 0x7fe748154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:02.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.166+0000 7fe74de00640 1 --2- 172.21.15.161:0/2445153932 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe74806ef70 0x7fe74815f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:02.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.166+0000 7fe74de00640 1 --2- 172.21.15.161:0/2445153932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74806e180 0x7fe7480b54c0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:02.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.166+0000 7fe74de00640 1 -- 172.21.15.161:0/2445153932 >> 172.21.15.161:0/2445153932 conn(0x7fe7480b2a40 msgr2=0x7fe74806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:02.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.166+0000 7fe74de00640 1 -- 172.21.15.161:0/2445153932 shutdown_connections 2024-05-22T21:25:02.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:02.166+0000 7fe74de00640 1 -- 172.21.15.161:0/2445153932 wait complete. 2024-05-22T21:25:02.249 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-05-22T21:25:07.250 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:25:07.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.398+0000 7efdd3400640 1 Processor -- start 2024-05-22T21:25:07.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.398+0000 7efdd3400640 1 -- start start 2024-05-22T21:25:07.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.398+0000 7efdd3400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efdcc143ec0 0x7efdcc15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:07.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.398+0000 7efdd3400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdcc1442a0 con 0x7efdcc0715a0 2024-05-22T21:25:07.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.398+0000 7efdd3400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdcc070500 con 0x7efdcc15ec90 2024-05-22T21:25:07.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.398+0000 7efdd3400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdcc0707e0 con 0x7efdcc143ec0 2024-05-22T21:25:07.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.398+0000 7efdd0a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efdcc143ec0 0x7efdcc15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:07.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.398+0000 7efdc3e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7efdcc0715a0 0x7efdcc06fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39450/0 (socket says 172.21.15.161:39450) 2024-05-22T21:25:07.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.398+0000 7efdc3e00640 1 -- 172.21.15.161:0/2884620533 learned_addr learned my addr 172.21.15.161:0/2884620533 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:07.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.398+0000 7efdcbe00640 1 -- 172.21.15.161:0/2884620533 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3947144178 0 0) 0x7efdcc1442a0 con 0x7efdcc0715a0 2024-05-22T21:25:07.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.398+0000 7efdcbe00640 1 -- 172.21.15.161:0/2884620533 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdb0003530 con 0x7efdcc0715a0 2024-05-22T21:25:07.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.398+0000 7efdd0a00640 1 -- 172.21.15.161:0/2884620533 >> v1:172.21.15.179:6789/0 conn(0x7efdcc15ec90 legacy=0x7efdcc0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:07.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.398+0000 7efdd0a00640 1 -- 172.21.15.161:0/2884620533 >> v1:172.21.15.161:6789/0 conn(0x7efdcc0715a0 legacy=0x7efdcc06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:07.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.398+0000 7efdd0a00640 1 -- 172.21.15.161:0/2884620533 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc070ac0 con 0x7efdcc143ec0 2024-05-22T21:25:07.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.402+0000 7efdd0a00640 1 --2- 172.21.15.161:0/2884620533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdcc143ec0 0x7efdcc15f830 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0097c0 tx=0x7efdbc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9491d55d927f60c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:07.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.402+0000 7efdcbe00640 1 -- 172.21.15.161:0/2884620533 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc0ab670 con 0x7efdcc143ec0 2024-05-22T21:25:07.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.402+0000 7efdcbe00640 1 -- 172.21.15.161:0/2884620533 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdbc0ab7d0 con 0x7efdcc143ec0 2024-05-22T21:25:07.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.402+0000 7efdcbe00640 1 -- 172.21.15.161:0/2884620533 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc0abb40 con 0x7efdcc143ec0 2024-05-22T21:25:07.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.402+0000 7efdd3400640 1 -- 172.21.15.161:0/2884620533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdcc143ec0 msgr2=0x7efdcc15f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:07.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.402+0000 7efdd3400640 1 --2- 172.21.15.161:0/2884620533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdcc143ec0 0x7efdcc15f830 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0097c0 tx=0x7efdbc01c470 comp rx=0 tx=0).stop 2024-05-22T21:25:07.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.402+0000 7efdd3400640 1 -- 172.21.15.161:0/2884620533 shutdown_connections 2024-05-22T21:25:07.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.402+0000 7efdd3400640 1 --2- 172.21.15.161:0/2884620533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdcc143ec0 0x7efdcc15f830 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:07.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.402+0000 7efdd3400640 1 -- 172.21.15.161:0/2884620533 >> 172.21.15.161:0/2884620533 conn(0x7efdcc0b0bc0 msgr2=0x7efdcc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:07.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.402+0000 7efdd3400640 1 -- 172.21.15.161:0/2884620533 shutdown_connections 2024-05-22T21:25:07.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.402+0000 7efdd3400640 1 -- 172.21.15.161:0/2884620533 wait complete. 2024-05-22T21:25:07.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdd3400640 1 Processor -- start 2024-05-22T21:25:07.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdd3400640 1 -- start start 2024-05-22T21:25:07.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdd3400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdcc143ec0 0x7efdcc13ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:07.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdd3400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdcc15ec90 0x7efdcc160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:07.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdd3400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efdcc15d1f0 0x7efdcc161500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:07.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdd3400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc13f110 con 0x7efdcc15ec90 2024-05-22T21:25:07.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdd3400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc160520 con 0x7efdcc143ec0 2024-05-22T21:25:07.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdd3400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc15d680 con 0x7efdcc15d1f0 2024-05-22T21:25:07.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdc3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdcc15ec90 0x7efdcc160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:07.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdc3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdcc15ec90 0x7efdcc160fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46698/0 (socket says 172.21.15.161:46698) 2024-05-22T21:25:07.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdc3e00640 1 -- 172.21.15.161:0/1581971395 learned_addr learned my addr 172.21.15.161:0/1581971395 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdd0a00640 1 --2- 172.21.15.161:0/1581971395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdcc143ec0 0x7efdcc13ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdd1400640 1 --2- 172.21.15.161:0/1581971395 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efdcc15d1f0 0x7efdcc161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdc3e00640 1 -- 172.21.15.161:0/1581971395 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efdcc15d1f0 msgr2=0x7efdcc161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdc3e00640 1 --2- 172.21.15.161:0/1581971395 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efdcc15d1f0 0x7efdcc161500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdc3e00640 1 -- 172.21.15.161:0/1581971395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdcc143ec0 msgr2=0x7efdcc13ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdc3e00640 1 --2- 172.21.15.161:0/1581971395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdcc143ec0 0x7efdcc13ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdc3e00640 1 -- 172.21.15.161:0/1581971395 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc009420 con 0x7efdcc15ec90 2024-05-22T21:25:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdc3e00640 1 --2- 172.21.15.161:0/1581971395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdcc15ec90 0x7efdcc160fc0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7efdb4001ec0 tx=0x7efdb400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdc9600640 1 -- 172.21.15.161:0/1581971395 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb4005d50 con 0x7efdcc15ec90 2024-05-22T21:25:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdc9600640 1 -- 172.21.15.161:0/1581971395 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb4007070 con 0x7efdcc15ec90 2024-05-22T21:25:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdd3400640 1 -- 172.21.15.161:0/1581971395 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdcc1607c0 con 0x7efdcc15ec90 2024-05-22T21:25:07.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdc9600640 1 -- 172.21.15.161:0/1581971395 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb40073e0 con 0x7efdcc15ec90 2024-05-22T21:25:07.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdd3400640 1 -- 172.21.15.161:0/1581971395 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdcc15d960 con 0x7efdcc15ec90 2024-05-22T21:25:07.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.406+0000 7efdd3400640 1 -- 172.21.15.161:0/1581971395 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdcc160ce0 con 0x7efdcc15ec90 2024-05-22T21:25:07.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.410+0000 7efdc9600640 1 -- 172.21.15.161:0/1581971395 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efdb4007540 con 0x7efdcc15ec90 2024-05-22T21:25:07.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.410+0000 7efdc9600640 1 --2- 172.21.15.161:0/1581971395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efda8066690 0x7efda8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:07.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.410+0000 7efdc9600640 1 -- 172.21.15.161:0/1581971395 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efdb4083720 con 0x7efdcc15ec90 2024-05-22T21:25:07.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.410+0000 7efdd0a00640 1 --2- 172.21.15.161:0/1581971395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efda8066690 0x7efda8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:07.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.410+0000 7efdd0a00640 1 --2- 172.21.15.161:0/1581971395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efda8066690 0x7efda8068b50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0af000 tx=0x7efdbc0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:07.415 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.414+0000 7efdc9600640 1 -- 172.21.15.161:0/1581971395 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efdcc160ce0 con 0x7efdcc15ec90 2024-05-22T21:25:07.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.570+0000 7efdd3400640 1 -- 172.21.15.161:0/1581971395 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7efdcc160ce0 con 0x7efdcc15ec90 2024-05-22T21:25:07.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.574+0000 7efdc9600640 1 -- 172.21.15.161:0/1581971395 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v193) v1 ==== 81+0+33594 (secure 0 0 0) 0x7efdcc160ce0 con 0x7efdcc15ec90 2024-05-22T21:25:07.575 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:07.575 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:25:07.575 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 193, 2024-05-22T21:25:07.575 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:25:07.575 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:25:07.575 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:25:05.312760+0000", 2024-05-22T21:25:07.575 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:25:07.576 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:25:07.576 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:25:07.576 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:25:07.576 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:25:07.576 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:25:07.576 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:25:07.576 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:25:07.576 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:25:07.576 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:25:07.576 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:25:07.576 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:25:07.576 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:25:07.576 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:25:07.576 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:25:07.577 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 23, 2024-05-22T21:25:07.577 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:25:07.577 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:25:07.577 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:25:07.577 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:25:07.577 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:25:07.577 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:25:07.577 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.577 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:25:07.577 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:25:07.577 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:25:07.577 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:25:07.578 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:25:07.578 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:25:07.578 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:25:07.578 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:25:07.578 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:25:07.578 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:25:07.578 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:25:07.578 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:25:07.578 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:25:07.578 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:25:07.578 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:25:07.578 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:25:07.578 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:25:07.579 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:25:07.579 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:25:07.579 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:25:07.579 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:25:07.579 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:25:07.579 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:25:07.579 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:25:07.579 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:25:07.579 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:25:07.579 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:25:07.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.579 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:25:07.579 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:25:07.580 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:25:07.580 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:25:07.580 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:25:07.580 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:25:07.580 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:25:07.580 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:25:07.580 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:25:07.580 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:25:07.580 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:25:07.580 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:25:07.580 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:25:07.580 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:25:07.581 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:25:07.581 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:25:07.581 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:25:07.581 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:25:07.581 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:25:07.581 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:25:07.581 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:25:07.581 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:25:07.581 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:25:07.581 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:25:07.581 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:25:07.581 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:25:07.581 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:25:07.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.582 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:25:07.582 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:25:07.582 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:25:07.582 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:25:07.582 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:25:07.582 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:25:07.582 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:25:07.582 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:25:07.582 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:25:07.582 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:25:07.582 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:25:07.582 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:25:07.583 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:25:07.583 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:25:07.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.583 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:25:07.583 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:25:07.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.583 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:25:07.583 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:25:07.583 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:25:07.583 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:25:07.583 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:25:07.583 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:25:07.583 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:25:07.584 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:25:07.584 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:25:07.584 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.584 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.584 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 22, 2024-05-22T21:25:07.584 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:25:07.584 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:24:31.797440+0000", 2024-05-22T21:25:07.584 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:25:07.584 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:25:07.584 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:25:07.584 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:25:07.585 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:25:07.585 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:25:07.585 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:25:07.585 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:25:07.585 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:25:07.585 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:25:07.585 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:25:07.585 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:25:07.585 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:25:07.585 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:25:07.585 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:25:07.585 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:25:07.586 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:25:07.586 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:25:07.586 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:25:07.586 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:25:07.586 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:25:07.586 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:25:07.586 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:25:07.586 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:25:07.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.586 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "189", 2024-05-22T21:25:07.586 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:25:07.586 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:25:07.587 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:25:07.587 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:25:07.587 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:25:07.587 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:25:07.587 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:25:07.587 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:25:07.587 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:25:07.587 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:25:07.587 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:25:07.587 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:25:07.587 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:25:07.587 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:25:07.587 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:25:07.588 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:25:07.588 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:25:07.588 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:25:07.588 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:25:07.588 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:25:07.588 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:25:07.588 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:25:07.588 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:25:07.588 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:25:07.588 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:25:07.588 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:25:07.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.588 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:25:07.588 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:25:07.589 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:25:07.589 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:25:07.589 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:25:07.589 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:25:07.589 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:25:07.589 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:25:07.589 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:25:07.589 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:25:07.589 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:25:07.589 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:25:07.589 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:25:07.589 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:25:07.589 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:25:07.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.590 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:25:07.590 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:25:07.590 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:25:07.590 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.590 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:25:07.590 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 2, 2024-05-22T21:25:07.590 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 2, 2024-05-22T21:25:07.590 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:25:07.590 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:25:07.590 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:25:07.590 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:25:07.591 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:25:07.591 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:25:07.591 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.591 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.591 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 23, 2024-05-22T21:25:07.591 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:25:07.591 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:24:33.098240+0000", 2024-05-22T21:25:07.591 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:25:07.591 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:25:07.591 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:25:07.591 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:25:07.592 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:25:07.592 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:25:07.592 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:25:07.592 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:25:07.592 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:25:07.592 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:25:07.592 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:25:07.592 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:25:07.592 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:25:07.592 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:25:07.592 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:25:07.592 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:25:07.592 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:25:07.592 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:25:07.592 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:25:07.593 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:25:07.593 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:25:07.593 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:25:07.593 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:25:07.593 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:25:07.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.593 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "193", 2024-05-22T21:25:07.593 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:25:07.593 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:25:07.593 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:25:07.593 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:25:07.593 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:25:07.593 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:25:07.594 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:25:07.594 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:25:07.594 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:25:07.594 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 33554432, 2024-05-22T21:25:07.594 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:25:07.594 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:25:07.594 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:25:07.594 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:25:07.594 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:25:07.594 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:25:07.594 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:25:07.594 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:25:07.595 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:25:07.595 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:25:07.595 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:25:07.595 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:25:07.595 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:25:07.595 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:25:07.595 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:25:07.595 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:25:07.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.595 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:25:07.595 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:25:07.595 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:25:07.595 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:25:07.596 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:25:07.596 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:25:07.596 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:25:07.596 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:25:07.596 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:25:07.596 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:25:07.596 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:25:07.596 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:25:07.596 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:25:07.596 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:25:07.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.596 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:25:07.597 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:25:07.597 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:25:07.597 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.597 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.597 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:25:07.597 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.3799999952316284, 2024-05-22T21:25:07.597 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.3799999952316284, 2024-05-22T21:25:07.597 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:25:07.597 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.3799999952316284, 2024-05-22T21:25:07.597 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.3799999952316284, 2024-05-22T21:25:07.597 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:25:07.597 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:25:07.597 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:25:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:25:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:25:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:25:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:25:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:25:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:25:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.601 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.601 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:25:07.601 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:07.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.601 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.601 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:07.601 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.601 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.601 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.601 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:25:07.601 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:07.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.602 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.602 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:25:07.602 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:07.602 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.602 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.602 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.602 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:07.602 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.602 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.602 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.602 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:25:07.602 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:07.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.603 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.603 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.603 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:25:07.603 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:07.603 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.603 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.603 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:25:07.603 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:25:07.603 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:25:07.603 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:25:07.603 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:07.604 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:07.604 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:07.604 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.604 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.604 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:25:07.604 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:07.604 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:07.604 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:07.604 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:07.604 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:07.604 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:07.604 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:07.604 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:07.605 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:07.605 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:07.605 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:07.605 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:07.605 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.605 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.605 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:25:07.605 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:07.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.605 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.605 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.605 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:25:07.605 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:07.606 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.606 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.606 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:07.606 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.606 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.606 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:25:07.606 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:07.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.606 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.606 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.607 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:25:07.607 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:07.607 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.607 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.607 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:07.607 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.607 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.607 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:25:07.607 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:07.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.607 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.607 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.608 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:25:07.608 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:07.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.608 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.608 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:07.608 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.608 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.608 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:25:07.608 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:07.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.608 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.608 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.609 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:25:07.609 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:07.609 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.609 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.609 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:25:07.609 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:25:07.609 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:25:07.609 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:25:07.609 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:07.609 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:07.609 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:07.609 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.610 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.610 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:25:07.610 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:07.610 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:07.610 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:07.610 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:07.610 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:07.610 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:07.610 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:07.610 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:07.611 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:07.611 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:07.611 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:07.611 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:07.611 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.611 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.611 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:25:07.611 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:07.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.611 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.611 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.612 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:25:07.612 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:07.612 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.612 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.612 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:07.612 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.612 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.612 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:25:07.612 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:07.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.612 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.613 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.613 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:25:07.613 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:07.613 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.613 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.613 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:07.613 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.613 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.613 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.613 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:25:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:25:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.614 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:25:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:25:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:25:07.615 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:25:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:25:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:25:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:25:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:07.616 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:25:07.617 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:25:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.618 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:25:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:25:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.619 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:25:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:25:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.620 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:25:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:25:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:25:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:25:07.621 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:25:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:25:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:25:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:07.622 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:25:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.623 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:25:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:25:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:07.624 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:25:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:25:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:07.625 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:25:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:25:07.626 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:25:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:25:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:25:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:25:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:25:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:07.627 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:25:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:07.628 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:25:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.629 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:25:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:25:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.630 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:25:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:25:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.631 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:25:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:25:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.632 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:25:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:25:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:25:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:25:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:25:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:07.633 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:25:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:07.634 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:25:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:25:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.635 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:25:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:25:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:25:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:25:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:25:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:25:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.638 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:25:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:25:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:25:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:25:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.639 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:25:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:07.640 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:25:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:25:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.641 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:25:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:25:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:25:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:25:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.643 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:25:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:25:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:07.644 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:25:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:25:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:25:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:25:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:25:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:25:07.645 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:25:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:25:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:25:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:07.646 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:25:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:25:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:07.647 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:25:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:25:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:25:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:07.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:25:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:25:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:25:07.649 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:25:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:25:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:07.650 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:25:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:25:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:25:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:07.651 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:25:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:25:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:25:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:25:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:25:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:25:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:25:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:25:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:25:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:25:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:25:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:25:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:25:07.652 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:25:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:25:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:25:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:25:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:25:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:25:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:25:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:25:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:25:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:25:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:25:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:25:07.653 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:25:07.654 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.654 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:25:07.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.574+0000 7efdd3400640 1 -- 172.21.15.161:0/1581971395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efda8066690 msgr2=0x7efda8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:07.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.574+0000 7efdd3400640 1 --2- 172.21.15.161:0/1581971395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efda8066690 0x7efda8068b50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0af000 tx=0x7efdbc0a7000 comp rx=0 tx=0).stop 2024-05-22T21:25:07.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.574+0000 7efdd3400640 1 -- 172.21.15.161:0/1581971395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdcc15ec90 msgr2=0x7efdcc160fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:07.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.574+0000 7efdd3400640 1 --2- 172.21.15.161:0/1581971395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdcc15ec90 0x7efdcc160fc0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7efdb4001ec0 tx=0x7efdb400c960 comp rx=0 tx=0).stop 2024-05-22T21:25:07.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.574+0000 7efdd3400640 1 -- 172.21.15.161:0/1581971395 shutdown_connections 2024-05-22T21:25:07.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.574+0000 7efdd3400640 1 --2- 172.21.15.161:0/1581971395 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efda8066690 0x7efda8068b50 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:07.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.574+0000 7efdd3400640 1 --2- 172.21.15.161:0/1581971395 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efdcc15d1f0 0x7efdcc161500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:07.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.574+0000 7efdd3400640 1 --2- 172.21.15.161:0/1581971395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdcc15ec90 0x7efdcc160fc0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:07.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.574+0000 7efdd3400640 1 --2- 172.21.15.161:0/1581971395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdcc143ec0 0x7efdcc13ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:07.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.574+0000 7efdd3400640 1 -- 172.21.15.161:0/1581971395 >> 172.21.15.161:0/1581971395 conn(0x7efdcc0b0bc0 msgr2=0x7efdcc0af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:07.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.574+0000 7efdd3400640 1 -- 172.21.15.161:0/1581971395 shutdown_connections 2024-05-22T21:25:07.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.574+0000 7efdd3400640 1 -- 172.21.15.161:0/1581971395 wait complete. 2024-05-22T21:25:07.657 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-05-22T21:25:07.657 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-05-22T21:25:07.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.790+0000 7f1570c00640 1 Processor -- start 2024-05-22T21:25:07.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.790+0000 7f1570c00640 1 -- start start 2024-05-22T21:25:07.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.790+0000 7f1570c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f156c15c6d0 0x7f156c164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:07.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.790+0000 7f1570c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c071670 con 0x7f156c15e130 2024-05-22T21:25:07.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.794+0000 7f1570c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c070a60 con 0x7f156c15dd50 2024-05-22T21:25:07.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.794+0000 7f1570c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156c070d40 con 0x7f156c15c6d0 2024-05-22T21:25:07.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.794+0000 7f1561400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f156c15e130 0x7f156c15ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39456/0 (socket says 172.21.15.161:39456) 2024-05-22T21:25:07.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.794+0000 7f1561400640 1 -- 172.21.15.161:0/3907941287 learned_addr learned my addr 172.21.15.161:0/3907941287 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:07.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.794+0000 7f156a800640 1 --2- 172.21.15.161:0/3907941287 >> v2:172.21.15.179:3300/0 conn(0x7f156c15c6d0 0x7f156c164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:07.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.794+0000 7f1569400640 1 -- 172.21.15.161:0/3907941287 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165240979 0 0) 0x7f156c071670 con 0x7f156c15e130 2024-05-22T21:25:07.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.794+0000 7f1569400640 1 -- 172.21.15.161:0/3907941287 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1550003530 con 0x7f156c15e130 2024-05-22T21:25:07.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.794+0000 7f156a800640 1 -- 172.21.15.161:0/3907941287 >> v1:172.21.15.179:6789/0 conn(0x7f156c15dd50 legacy=0x7f156c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:07.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.794+0000 7f1569400640 1 -- 172.21.15.161:0/3907941287 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 162807852 0 0) 0x7f1550003530 con 0x7f156c15e130 2024-05-22T21:25:07.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.794+0000 7f156a800640 1 -- 172.21.15.161:0/3907941287 >> v1:172.21.15.161:6789/0 conn(0x7f156c15e130 legacy=0x7f156c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:07.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.794+0000 7f156a800640 1 -- 172.21.15.161:0/3907941287 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c071020 con 0x7f156c15c6d0 2024-05-22T21:25:07.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.794+0000 7f156a800640 1 --2- 172.21.15.161:0/3907941287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f156c15c6d0 0x7f156c164e80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f15640097c0 tx=0x7f156401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9abac2734b7fba25 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:07.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1569400640 1 -- 172.21.15.161:0/3907941287 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1554001680 con 0x7f156c15c6d0 2024-05-22T21:25:07.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1569400640 1 -- 172.21.15.161:0/3907941287 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15640a7050 con 0x7f156c15c6d0 2024-05-22T21:25:07.797 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1569400640 1 -- 172.21.15.161:0/3907941287 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15640ab620 con 0x7f156c15c6d0 2024-05-22T21:25:07.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1570c00640 1 -- 172.21.15.161:0/3907941287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f156c15c6d0 msgr2=0x7f156c164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:07.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1570c00640 1 --2- 172.21.15.161:0/3907941287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f156c15c6d0 0x7f156c164e80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f15640097c0 tx=0x7f156401c390 comp rx=0 tx=0).stop 2024-05-22T21:25:07.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1570c00640 1 -- 172.21.15.161:0/3907941287 shutdown_connections 2024-05-22T21:25:07.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1570c00640 1 --2- 172.21.15.161:0/3907941287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f156c15c6d0 0x7f156c164e80 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:07.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1570c00640 1 -- 172.21.15.161:0/3907941287 >> 172.21.15.161:0/3907941287 conn(0x7f156c0b0bc0 msgr2=0x7f156c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:07.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1570c00640 1 -- 172.21.15.161:0/3907941287 shutdown_connections 2024-05-22T21:25:07.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1570c00640 1 -- 172.21.15.161:0/3907941287 wait complete. 2024-05-22T21:25:07.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1570c00640 1 Processor -- start 2024-05-22T21:25:07.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1570c00640 1 -- start start 2024-05-22T21:25:07.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1570c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f156c15c6d0 0x7f156c1436d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:07.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1570c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f156c15dd50 0x7f156c0a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:07.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1570c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f156c15e130 0x7f156c0a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:07.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1570c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0a98b0 con 0x7f156c15c6d0 2024-05-22T21:25:07.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1570c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0a9010 con 0x7f156c15dd50 2024-05-22T21:25:07.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1570c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0a92f0 con 0x7f156c15e130 2024-05-22T21:25:07.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1569e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f156c15c6d0 0x7f156c1436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:07.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1569e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f156c15c6d0 0x7f156c1436d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46700/0 (socket says 172.21.15.161:46700) 2024-05-22T21:25:07.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.798+0000 7f1569e00640 1 -- 172.21.15.161:0/2610575511 learned_addr learned my addr 172.21.15.161:0/2610575511 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:07.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.802+0000 7f1569e00640 1 -- 172.21.15.161:0/2610575511 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f156c15e130 msgr2=0x7f156c0a89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:07.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.802+0000 7f1569e00640 1 --2- 172.21.15.161:0/2610575511 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f156c15e130 0x7f156c0a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:07.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.802+0000 7f1569e00640 1 -- 172.21.15.161:0/2610575511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f156c15dd50 msgr2=0x7f156c0a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:07.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.802+0000 7f1569e00640 1 --2- 172.21.15.161:0/2610575511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f156c15dd50 0x7f156c0a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:07.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.802+0000 7f1569e00640 1 -- 172.21.15.161:0/2610575511 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1564009420 con 0x7f156c15c6d0 2024-05-22T21:25:07.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.802+0000 7f1569e00640 1 --2- 172.21.15.161:0/2610575511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f156c15c6d0 0x7f156c1436d0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f155c00c7a0 tx=0x7f155c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:07.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.802+0000 7f1562a00640 1 -- 172.21.15.161:0/2610575511 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c003b80 con 0x7f156c15c6d0 2024-05-22T21:25:07.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.802+0000 7f1562a00640 1 -- 172.21.15.161:0/2610575511 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c003ce0 con 0x7f156c15c6d0 2024-05-22T21:25:07.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.802+0000 7f1562a00640 1 -- 172.21.15.161:0/2610575511 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c00aa90 con 0x7f156c15c6d0 2024-05-22T21:25:07.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.802+0000 7f1570c00640 1 -- 172.21.15.161:0/2610575511 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c0a95d0 con 0x7f156c15c6d0 2024-05-22T21:25:07.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.802+0000 7f1570c00640 1 -- 172.21.15.161:0/2610575511 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c152c50 con 0x7f156c15c6d0 2024-05-22T21:25:07.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.802+0000 7f1570c00640 1 -- 172.21.15.161:0/2610575511 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c155740 con 0x7f156c15c6d0 2024-05-22T21:25:07.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.802+0000 7f1562a00640 1 -- 172.21.15.161:0/2610575511 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f155c00abf0 con 0x7f156c15c6d0 2024-05-22T21:25:07.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.802+0000 7f1562a00640 1 --2- 172.21.15.161:0/2610575511 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1548066690 0x7f1548068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:07.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.806+0000 7f1562a00640 1 -- 172.21.15.161:0/2610575511 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f155c083f60 con 0x7f156c15c6d0 2024-05-22T21:25:07.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.806+0000 7f1561400640 1 --2- 172.21.15.161:0/2610575511 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1548066690 0x7f1548068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:07.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.806+0000 7f1561400640 1 --2- 172.21.15.161:0/2610575511 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1548066690 0x7f1548068b50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1554005890 tx=0x7f1554001970 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:07.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.810+0000 7f1562a00640 1 -- 172.21.15.161:0/2610575511 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f156c155740 con 0x7f156c15c6d0 2024-05-22T21:25:07.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.962+0000 7f1570c00640 1 -- 172.21.15.161:0/2610575511 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f156c155740 con 0x7f156c15c6d0 2024-05-22T21:25:07.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.962+0000 7f1562a00640 1 -- 172.21.15.161:0/2610575511 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v193) v1 ==== 81+0+33594 (secure 0 0 0) 0x7f156c155740 con 0x7f156c15c6d0 2024-05-22T21:25:07.965 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:07.966 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:25:07.966 INFO:teuthology.orchestra.run.smithi161.stdout: "epoch": 193, 2024-05-22T21:25:07.966 INFO:teuthology.orchestra.run.smithi161.stdout: "fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", 2024-05-22T21:25:07.966 INFO:teuthology.orchestra.run.smithi161.stdout: "created": "2024-05-22T21:13:05.415577+0000", 2024-05-22T21:25:07.966 INFO:teuthology.orchestra.run.smithi161.stdout: "modified": "2024-05-22T21:25:05.312760+0000", 2024-05-22T21:25:07.966 INFO:teuthology.orchestra.run.smithi161.stdout: "last_up_change": "2024-05-22T21:13:10.955061+0000", 2024-05-22T21:25:07.966 INFO:teuthology.orchestra.run.smithi161.stdout: "last_in_change": "2024-05-22T21:13:07.945035+0000", 2024-05-22T21:25:07.966 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-22T21:25:07.966 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_num": 5799936, 2024-05-22T21:25:07.966 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_set": [ 2024-05-22T21:25:07.966 INFO:teuthology.orchestra.run.smithi161.stdout: "pglog_hardlimit", 2024-05-22T21:25:07.967 INFO:teuthology.orchestra.run.smithi161.stdout: "purged_snapdirs", 2024-05-22T21:25:07.967 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_deletes", 2024-05-22T21:25:07.967 INFO:teuthology.orchestra.run.smithi161.stdout: "sortbitwise" 2024-05-22T21:25:07.967 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:25:07.967 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_version": 4, 2024-05-22T21:25:07.967 INFO:teuthology.orchestra.run.smithi161.stdout: "full_ratio": 0.69999998807907104, 2024-05-22T21:25:07.967 INFO:teuthology.orchestra.run.smithi161.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-05-22T21:25:07.967 INFO:teuthology.orchestra.run.smithi161.stdout: "nearfull_ratio": 0.60000002384185791, 2024-05-22T21:25:07.967 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_snapshot": "", 2024-05-22T21:25:07.967 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_max": 23, 2024-05-22T21:25:07.967 INFO:teuthology.orchestra.run.smithi161.stdout: "max_osd": 8, 2024-05-22T21:25:07.967 INFO:teuthology.orchestra.run.smithi161.stdout: "require_min_compat_client": "luminous", 2024-05-22T21:25:07.968 INFO:teuthology.orchestra.run.smithi161.stdout: "min_compat_client": "jewel", 2024-05-22T21:25:07.968 INFO:teuthology.orchestra.run.smithi161.stdout: "require_osd_release": "reef", 2024-05-22T21:25:07.968 INFO:teuthology.orchestra.run.smithi161.stdout: "allow_crimson": false, 2024-05-22T21:25:07.968 INFO:teuthology.orchestra.run.smithi161.stdout: "pools": [ 2024-05-22T21:25:07.968 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.968 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 1, 2024-05-22T21:25:07.968 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": ".mgr", 2024-05-22T21:25:07.968 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:13:12.449493+0000", 2024-05-22T21:25:07.968 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:25:07.968 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:25:07.968 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:25:07.968 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:25:07.969 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:25:07.969 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:25:07.969 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:25:07.969 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:25:07.969 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:25:07.969 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:25:07.969 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:25:07.969 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:25:07.969 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 1, 2024-05-22T21:25:07.969 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 1, 2024-05-22T21:25:07.969 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 1, 2024-05-22T21:25:07.970 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 1, 2024-05-22T21:25:07.970 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 1, 2024-05-22T21:25:07.970 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:25:07.970 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:25:07.970 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:25:07.970 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:25:07.970 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:25:07.970 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:25:07.970 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:25:07.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.970 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "113", 2024-05-22T21:25:07.970 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:25:07.971 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:25:07.971 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:25:07.971 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:25:07.971 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:25:07.971 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:25:07.971 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:25:07.971 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:25:07.971 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:25:07.971 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:25:07.971 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:25:07.971 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:25:07.972 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:25:07.972 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:25:07.972 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:25:07.972 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:25:07.972 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:25:07.972 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:25:07.972 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:25:07.972 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:25:07.972 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:25:07.972 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:25:07.972 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:25:07.972 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:25:07.973 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:25:07.973 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:25:07.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.973 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:25:07.973 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:25:07.973 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:25:07.973 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:25:07.973 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:25:07.973 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:25:07.973 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:25:07.973 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:25:07.973 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:25:07.974 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:25:07.974 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:25:07.974 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:25:07.974 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_max": 32, 2024-05-22T21:25:07.974 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 1 2024-05-22T21:25:07.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.974 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:25:07.974 INFO:teuthology.orchestra.run.smithi161.stdout: "mgr": {} 2024-05-22T21:25:07.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.974 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:25:07.974 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 8, 2024-05-22T21:25:07.974 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 8, 2024-05-22T21:25:07.975 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 0.25, 2024-05-22T21:25:07.975 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:25:07.975 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:25:07.975 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:25:07.975 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:25:07.975 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:25:07.975 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.975 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.975 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 22, 2024-05-22T21:25:07.975 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_metadata", 2024-05-22T21:25:07.975 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:24:31.797440+0000", 2024-05-22T21:25:07.975 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:25:07.975 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:25:07.976 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:25:07.976 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:25:07.976 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:25:07.976 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:25:07.976 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:25:07.976 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:25:07.976 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:25:07.976 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:25:07.976 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:25:07.976 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:25:07.976 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 32, 2024-05-22T21:25:07.976 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 32, 2024-05-22T21:25:07.976 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 32, 2024-05-22T21:25:07.976 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 32, 2024-05-22T21:25:07.977 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 32, 2024-05-22T21:25:07.977 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:25:07.977 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:25:07.977 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:25:07.977 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:25:07.977 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:25:07.977 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:25:07.977 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:25:07.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.977 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "189", 2024-05-22T21:25:07.977 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:25:07.977 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:25:07.977 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:25:07.977 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:25:07.978 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:25:07.978 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:25:07.978 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:25:07.978 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:25:07.978 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:25:07.978 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 0, 2024-05-22T21:25:07.978 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:25:07.978 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:25:07.978 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:25:07.978 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:25:07.978 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:25:07.978 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:25:07.979 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:25:07.979 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:25:07.979 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:25:07.979 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:25:07.979 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:25:07.979 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:25:07.979 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:25:07.979 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:25:07.979 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:25:07.979 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:25:07.979 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.979 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:25:07.979 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:25:07.979 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:25:07.980 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:25:07.980 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:25:07.980 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:25:07.980 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:25:07.980 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:25:07.980 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:25:07.980 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:25:07.980 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:25:07.980 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:25:07.980 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_bias": 4, 2024-05-22T21:25:07.980 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 16, 2024-05-22T21:25:07.980 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_priority": 5 2024-05-22T21:25:07.980 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.981 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:25:07.981 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:25:07.981 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": "cephfs" 2024-05-22T21:25:07.981 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.981 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:25:07.981 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 2, 2024-05-22T21:25:07.981 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 2, 2024-05-22T21:25:07.981 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:25:07.981 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 2, 2024-05-22T21:25:07.981 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 2, 2024-05-22T21:25:07.981 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:25:07.981 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:25:07.982 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:25:07.982 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.982 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.982 INFO:teuthology.orchestra.run.smithi161.stdout: "pool": 23, 2024-05-22T21:25:07.982 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_name": "cephfs_data", 2024-05-22T21:25:07.982 INFO:teuthology.orchestra.run.smithi161.stdout: "create_time": "2024-05-22T21:24:33.098240+0000", 2024-05-22T21:25:07.982 INFO:teuthology.orchestra.run.smithi161.stdout: "flags": 1, 2024-05-22T21:25:07.982 INFO:teuthology.orchestra.run.smithi161.stdout: "flags_names": "hashpspool", 2024-05-22T21:25:07.982 INFO:teuthology.orchestra.run.smithi161.stdout: "type": 1, 2024-05-22T21:25:07.982 INFO:teuthology.orchestra.run.smithi161.stdout: "size": 2, 2024-05-22T21:25:07.982 INFO:teuthology.orchestra.run.smithi161.stdout: "min_size": 1, 2024-05-22T21:25:07.983 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_rule": 0, 2024-05-22T21:25:07.983 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_count": 0, 2024-05-22T21:25:07.983 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_target": 0, 2024-05-22T21:25:07.983 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_barrier": 0, 2024-05-22T21:25:07.983 INFO:teuthology.orchestra.run.smithi161.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-22T21:25:07.983 INFO:teuthology.orchestra.run.smithi161.stdout: "object_hash": 2, 2024-05-22T21:25:07.983 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_autoscale_mode": "off", 2024-05-22T21:25:07.983 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num": 64, 2024-05-22T21:25:07.983 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num": 64, 2024-05-22T21:25:07.983 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_placement_num_target": 64, 2024-05-22T21:25:07.984 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_target": 64, 2024-05-22T21:25:07.984 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_pending": 64, 2024-05-22T21:25:07.984 INFO:teuthology.orchestra.run.smithi161.stdout: "last_pg_merge_meta": { 2024-05-22T21:25:07.984 INFO:teuthology.orchestra.run.smithi161.stdout: "source_pgid": "0.0", 2024-05-22T21:25:07.984 INFO:teuthology.orchestra.run.smithi161.stdout: "ready_epoch": 0, 2024-05-22T21:25:07.984 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_started": 0, 2024-05-22T21:25:07.984 INFO:teuthology.orchestra.run.smithi161.stdout: "last_epoch_clean": 0, 2024-05-22T21:25:07.984 INFO:teuthology.orchestra.run.smithi161.stdout: "source_version": "0'0", 2024-05-22T21:25:07.984 INFO:teuthology.orchestra.run.smithi161.stdout: "target_version": "0'0" 2024-05-22T21:25:07.984 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.984 INFO:teuthology.orchestra.run.smithi161.stdout: "last_change": "193", 2024-05-22T21:25:07.984 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend": "0", 2024-05-22T21:25:07.985 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-22T21:25:07.985 INFO:teuthology.orchestra.run.smithi161.stdout: "last_force_op_resend_preluminous": "0", 2024-05-22T21:25:07.985 INFO:teuthology.orchestra.run.smithi161.stdout: "auid": 0, 2024-05-22T21:25:07.985 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_mode": "selfmanaged", 2024-05-22T21:25:07.985 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_seq": 0, 2024-05-22T21:25:07.985 INFO:teuthology.orchestra.run.smithi161.stdout: "snap_epoch": 0, 2024-05-22T21:25:07.985 INFO:teuthology.orchestra.run.smithi161.stdout: "pool_snaps": [], 2024-05-22T21:25:07.985 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps": "[]", 2024-05-22T21:25:07.985 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_bytes": 33554432, 2024-05-22T21:25:07.985 INFO:teuthology.orchestra.run.smithi161.stdout: "quota_max_objects": 0, 2024-05-22T21:25:07.985 INFO:teuthology.orchestra.run.smithi161.stdout: "tiers": [], 2024-05-22T21:25:07.985 INFO:teuthology.orchestra.run.smithi161.stdout: "tier_of": -1, 2024-05-22T21:25:07.986 INFO:teuthology.orchestra.run.smithi161.stdout: "read_tier": -1, 2024-05-22T21:25:07.986 INFO:teuthology.orchestra.run.smithi161.stdout: "write_tier": -1, 2024-05-22T21:25:07.986 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_mode": "none", 2024-05-22T21:25:07.986 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_bytes": 0, 2024-05-22T21:25:07.986 INFO:teuthology.orchestra.run.smithi161.stdout: "target_max_objects": 0, 2024-05-22T21:25:07.986 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-22T21:25:07.986 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-22T21:25:07.986 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-22T21:25:07.986 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_flush_age": 0, 2024-05-22T21:25:07.986 INFO:teuthology.orchestra.run.smithi161.stdout: "cache_min_evict_age": 0, 2024-05-22T21:25:07.986 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profile": "", 2024-05-22T21:25:07.986 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_params": { 2024-05-22T21:25:07.987 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "none" 2024-05-22T21:25:07.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.987 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_period": 0, 2024-05-22T21:25:07.987 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_count": 0, 2024-05-22T21:25:07.987 INFO:teuthology.orchestra.run.smithi161.stdout: "use_gmt_hitset": true, 2024-05-22T21:25:07.987 INFO:teuthology.orchestra.run.smithi161.stdout: "min_read_recency_for_promote": 0, 2024-05-22T21:25:07.987 INFO:teuthology.orchestra.run.smithi161.stdout: "min_write_recency_for_promote": 0, 2024-05-22T21:25:07.987 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_grade_decay_rate": 0, 2024-05-22T21:25:07.987 INFO:teuthology.orchestra.run.smithi161.stdout: "hit_set_search_last_n": 0, 2024-05-22T21:25:07.987 INFO:teuthology.orchestra.run.smithi161.stdout: "grade_table": [], 2024-05-22T21:25:07.987 INFO:teuthology.orchestra.run.smithi161.stdout: "stripe_width": 0, 2024-05-22T21:25:07.987 INFO:teuthology.orchestra.run.smithi161.stdout: "expected_num_objects": 0, 2024-05-22T21:25:07.987 INFO:teuthology.orchestra.run.smithi161.stdout: "fast_read": false, 2024-05-22T21:25:07.988 INFO:teuthology.orchestra.run.smithi161.stdout: "options": { 2024-05-22T21:25:07.988 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_num_min": 64, 2024-05-22T21:25:07.988 INFO:teuthology.orchestra.run.smithi161.stdout: "target_size_ratio": 0.89999997615814209 2024-05-22T21:25:07.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.988 INFO:teuthology.orchestra.run.smithi161.stdout: "application_metadata": { 2024-05-22T21:25:07.988 INFO:teuthology.orchestra.run.smithi161.stdout: "cephfs": { 2024-05-22T21:25:07.988 INFO:teuthology.orchestra.run.smithi161.stdout: "data": "cephfs" 2024-05-22T21:25:07.988 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.988 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.988 INFO:teuthology.orchestra.run.smithi161.stdout: "read_balance": { 2024-05-22T21:25:07.988 INFO:teuthology.orchestra.run.smithi161.stdout: "score_acting": 1.3799999952316284, 2024-05-22T21:25:07.989 INFO:teuthology.orchestra.run.smithi161.stdout: "score_stable": 1.3799999952316284, 2024-05-22T21:25:07.989 INFO:teuthology.orchestra.run.smithi161.stdout: "optimal_score": 1, 2024-05-22T21:25:07.989 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_acting": 1.3799999952316284, 2024-05-22T21:25:07.989 INFO:teuthology.orchestra.run.smithi161.stdout: "raw_score_stable": 1.3799999952316284, 2024-05-22T21:25:07.989 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity_weighted": 1, 2024-05-22T21:25:07.989 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity": 1, 2024-05-22T21:25:07.989 INFO:teuthology.orchestra.run.smithi161.stdout: "average_primary_affinity_weighted": 1 2024-05-22T21:25:07.989 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.989 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.989 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:25:07.989 INFO:teuthology.orchestra.run.smithi161.stdout: "osds": [ 2024-05-22T21:25:07.989 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.989 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:25:07.990 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:07.990 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:07.990 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:07.990 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:07.990 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:07.990 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:07.990 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:07.990 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:07.990 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:07.990 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:07.990 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:07.991 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:07.991 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.991 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.991 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6808", 2024-05-22T21:25:07.991 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:07.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.991 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.991 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.991 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6809", 2024-05-22T21:25:07.991 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:07.992 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.992 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.992 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:07.992 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.992 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.992 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6810", 2024-05-22T21:25:07.992 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:07.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.992 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.993 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.993 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6811", 2024-05-22T21:25:07.993 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:07.993 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.993 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.993 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:07.993 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.993 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.993 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.993 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6814", 2024-05-22T21:25:07.993 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:07.993 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.993 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.994 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.994 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6815", 2024-05-22T21:25:07.994 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:07.994 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.994 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.994 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:07.994 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.994 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.994 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6812", 2024-05-22T21:25:07.994 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:07.994 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.994 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.995 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.995 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6813", 2024-05-22T21:25:07.995 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23413 2024-05-22T21:25:07.995 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.995 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.995 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.995 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6809/23413", 2024-05-22T21:25:07.995 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6811/23413", 2024-05-22T21:25:07.995 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6815/23413", 2024-05-22T21:25:07.995 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6813/23413", 2024-05-22T21:25:07.995 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:07.996 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:07.996 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:07.996 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.996 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.996 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.996 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:25:07.996 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:07.996 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:07.996 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:07.996 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:07.996 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:07.996 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:07.996 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:07.996 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:07.997 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:07.997 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:07.997 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:07.997 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:07.997 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.997 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.997 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6800", 2024-05-22T21:25:07.997 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:07.997 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.997 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.997 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.997 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6801", 2024-05-22T21:25:07.997 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:07.997 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.998 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.998 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:07.998 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.998 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:07.998 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6802", 2024-05-22T21:25:07.998 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:07.998 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.998 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:07.998 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:07.998 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6803", 2024-05-22T21:25:07.999 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:07.999 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:07.999 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:07.999 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:07.999 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:07.999 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:07.999 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.000 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.000 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6806", 2024-05-22T21:25:08.000 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:08.000 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.000 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.001 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.001 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6807", 2024-05-22T21:25:08.001 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:08.001 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.001 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.001 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.001 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:08.001 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.002 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.002 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6804", 2024-05-22T21:25:08.002 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:08.002 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.002 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.002 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.002 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6805", 2024-05-22T21:25:08.002 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23412 2024-05-22T21:25:08.003 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.003 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.003 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.003 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6801/23412", 2024-05-22T21:25:08.003 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6803/23412", 2024-05-22T21:25:08.003 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6807/23412", 2024-05-22T21:25:08.003 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6805/23412", 2024-05-22T21:25:08.003 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:08.003 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:08.003 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:08.003 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.004 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.004 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:25:08.004 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:08.004 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:08.004 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:08.004 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:08.004 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:08.004 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:08.004 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:08.004 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:08.004 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:08.005 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:08.005 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:08.005 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:08.005 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.005 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.005 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6824", 2024-05-22T21:25:08.005 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:08.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.005 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.005 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.005 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6825", 2024-05-22T21:25:08.006 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:08.006 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.006 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.006 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:08.006 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.006 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.006 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6826", 2024-05-22T21:25:08.006 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:08.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.006 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.007 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.007 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6827", 2024-05-22T21:25:08.007 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:08.007 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.007 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.007 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:08.007 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.007 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.007 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.007 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6830", 2024-05-22T21:25:08.007 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:08.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.008 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.008 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6831", 2024-05-22T21:25:08.008 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:08.008 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.008 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.008 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:08.008 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.008 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.008 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.008 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6828", 2024-05-22T21:25:08.008 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:08.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.009 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.009 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.009 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6829", 2024-05-22T21:25:08.009 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23414 2024-05-22T21:25:08.009 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.009 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.009 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6825/23414", 2024-05-22T21:25:08.009 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6827/23414", 2024-05-22T21:25:08.009 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6831/23414", 2024-05-22T21:25:08.009 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6829/23414", 2024-05-22T21:25:08.009 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:08.010 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:08.010 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:08.010 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.010 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.010 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:25:08.010 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:08.010 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:08.010 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:08.010 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:08.010 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:08.010 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:08.011 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:08.011 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:08.011 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:08.011 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:08.011 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:08.011 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:08.011 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.011 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.011 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6816", 2024-05-22T21:25:08.011 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:08.011 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.011 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.012 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.012 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6817", 2024-05-22T21:25:08.012 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:08.012 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.012 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.012 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:08.012 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.012 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.012 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.012 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6818", 2024-05-22T21:25:08.012 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:08.012 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.013 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.013 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6819", 2024-05-22T21:25:08.013 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:08.013 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.013 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.013 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:08.013 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.013 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.013 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.013 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6822", 2024-05-22T21:25:08.013 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:08.013 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.014 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.014 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6823", 2024-05-22T21:25:08.014 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:08.014 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.014 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.014 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:08.014 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.014 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.014 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.014 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6820", 2024-05-22T21:25:08.014 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:08.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.015 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.015 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.015 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.161:6821", 2024-05-22T21:25:08.015 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23415 2024-05-22T21:25:08.015 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.015 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.015 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.161:6817/23415", 2024-05-22T21:25:08.015 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.161:6819/23415", 2024-05-22T21:25:08.015 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.161:6823/23415", 2024-05-22T21:25:08.015 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.161:6821/23415", 2024-05-22T21:25:08.015 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:08.016 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:08.016 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:08.016 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.016 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.016 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.016 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:25:08.016 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:08.016 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:08.016 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:08.016 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:08.016 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:08.017 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:08.017 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:08.017 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:08.017 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:08.017 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:08.017 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:08.017 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:08.017 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.017 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.017 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.017 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6800", 2024-05-22T21:25:08.017 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:08.017 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.018 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.018 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6801", 2024-05-22T21:25:08.018 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:08.018 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.018 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.018 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:08.018 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.018 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.018 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.018 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6802", 2024-05-22T21:25:08.019 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:08.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.019 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.019 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6803", 2024-05-22T21:25:08.019 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:08.019 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.019 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.019 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.019 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:08.019 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.019 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.021 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.021 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6806", 2024-05-22T21:25:08.021 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:08.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.021 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.021 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.021 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6807", 2024-05-22T21:25:08.021 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:08.022 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.022 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.022 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:08.022 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.022 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.022 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6804", 2024-05-22T21:25:08.022 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:08.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.022 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.022 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.023 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6805", 2024-05-22T21:25:08.023 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23606 2024-05-22T21:25:08.023 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.023 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.023 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.023 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6801/23606", 2024-05-22T21:25:08.023 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6803/23606", 2024-05-22T21:25:08.023 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6807/23606", 2024-05-22T21:25:08.023 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6805/23606", 2024-05-22T21:25:08.023 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:08.023 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:08.023 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:08.023 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.024 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.024 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:25:08.024 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:08.024 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:08.024 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:08.024 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:08.024 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:08.024 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:08.024 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:08.024 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:08.024 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:08.025 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:08.025 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:08.025 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:08.025 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.025 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.025 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6808", 2024-05-22T21:25:08.025 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:08.025 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.025 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.025 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.025 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6809", 2024-05-22T21:25:08.025 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:08.026 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.026 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.026 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:08.026 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.026 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.026 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6810", 2024-05-22T21:25:08.026 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:08.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.026 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.026 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.027 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6811", 2024-05-22T21:25:08.027 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:08.027 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.027 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.027 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:08.027 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.027 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.027 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6814", 2024-05-22T21:25:08.027 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:08.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.027 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.028 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.028 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6815", 2024-05-22T21:25:08.028 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:08.028 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.028 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.028 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:08.028 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.028 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.028 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.028 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6812", 2024-05-22T21:25:08.028 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:08.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.029 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.029 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.029 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6813", 2024-05-22T21:25:08.029 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23607 2024-05-22T21:25:08.029 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.029 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.029 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6809/23607", 2024-05-22T21:25:08.029 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6811/23607", 2024-05-22T21:25:08.029 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6815/23607", 2024-05-22T21:25:08.030 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6813/23607", 2024-05-22T21:25:08.030 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:08.030 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:08.030 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:08.030 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.030 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.030 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:25:08.030 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:08.030 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:08.030 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:08.030 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:08.031 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:08.031 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:08.031 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:08.031 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:08.031 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:08.031 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:08.031 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:08.031 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:08.031 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.031 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.031 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.031 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6820", 2024-05-22T21:25:08.032 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:08.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.032 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.032 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.032 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6822", 2024-05-22T21:25:08.032 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:08.032 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.032 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.032 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:08.032 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.032 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.033 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.033 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6825", 2024-05-22T21:25:08.033 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:08.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.033 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.033 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.033 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6827", 2024-05-22T21:25:08.033 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:08.033 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.033 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.033 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:08.033 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.033 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.034 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.034 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6830", 2024-05-22T21:25:08.034 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:08.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.034 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.034 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.034 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6831", 2024-05-22T21:25:08.034 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:08.034 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.034 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.034 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:08.035 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.035 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.035 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6828", 2024-05-22T21:25:08.035 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:08.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.035 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.035 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.035 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6829", 2024-05-22T21:25:08.035 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23608 2024-05-22T21:25:08.035 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.035 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.035 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6822/23608", 2024-05-22T21:25:08.036 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6827/23608", 2024-05-22T21:25:08.036 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6831/23608", 2024-05-22T21:25:08.036 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6829/23608", 2024-05-22T21:25:08.036 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:08.036 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:08.036 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:08.036 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.036 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.036 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:25:08.036 INFO:teuthology.orchestra.run.smithi161.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-05-22T21:25:08.036 INFO:teuthology.orchestra.run.smithi161.stdout: "up": 1, 2024-05-22T21:25:08.037 INFO:teuthology.orchestra.run.smithi161.stdout: "in": 1, 2024-05-22T21:25:08.037 INFO:teuthology.orchestra.run.smithi161.stdout: "weight": 1, 2024-05-22T21:25:08.037 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_affinity": 1, 2024-05-22T21:25:08.037 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_begin": 0, 2024-05-22T21:25:08.037 INFO:teuthology.orchestra.run.smithi161.stdout: "last_clean_end": 0, 2024-05-22T21:25:08.037 INFO:teuthology.orchestra.run.smithi161.stdout: "up_from": 13, 2024-05-22T21:25:08.037 INFO:teuthology.orchestra.run.smithi161.stdout: "up_thru": 188, 2024-05-22T21:25:08.037 INFO:teuthology.orchestra.run.smithi161.stdout: "down_at": 0, 2024-05-22T21:25:08.037 INFO:teuthology.orchestra.run.smithi161.stdout: "lost_at": 0, 2024-05-22T21:25:08.037 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addrs": { 2024-05-22T21:25:08.038 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.038 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.038 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.038 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6816", 2024-05-22T21:25:08.038 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:08.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.038 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.038 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.038 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6817", 2024-05-22T21:25:08.038 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:08.038 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.038 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.039 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addrs": { 2024-05-22T21:25:08.039 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.039 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.039 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6818", 2024-05-22T21:25:08.039 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:08.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.039 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.039 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.039 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6819", 2024-05-22T21:25:08.039 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:08.040 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.040 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.040 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addrs": { 2024-05-22T21:25:08.040 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.040 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.040 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.040 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6824", 2024-05-22T21:25:08.040 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:08.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.040 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.040 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.040 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6826", 2024-05-22T21:25:08.040 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:08.041 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.041 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.041 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addrs": { 2024-05-22T21:25:08.041 INFO:teuthology.orchestra.run.smithi161.stdout: "addrvec": [ 2024-05-22T21:25:08.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.041 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v2", 2024-05-22T21:25:08.041 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6821", 2024-05-22T21:25:08.041 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:08.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.041 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.041 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-05-22T21:25:08.041 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "172.21.15.179:6823", 2024-05-22T21:25:08.042 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 23609 2024-05-22T21:25:08.042 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.042 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.042 INFO:teuthology.orchestra.run.smithi161.stdout: "public_addr": "172.21.15.179:6817/23609", 2024-05-22T21:25:08.042 INFO:teuthology.orchestra.run.smithi161.stdout: "cluster_addr": "172.21.15.179:6819/23609", 2024-05-22T21:25:08.042 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_back_addr": "172.21.15.179:6826/23609", 2024-05-22T21:25:08.042 INFO:teuthology.orchestra.run.smithi161.stdout: "heartbeat_front_addr": "172.21.15.179:6823/23609", 2024-05-22T21:25:08.042 INFO:teuthology.orchestra.run.smithi161.stdout: "state": [ 2024-05-22T21:25:08.042 INFO:teuthology.orchestra.run.smithi161.stdout: "exists", 2024-05-22T21:25:08.042 INFO:teuthology.orchestra.run.smithi161.stdout: "up" 2024-05-22T21:25:08.042 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-22T21:25:08.042 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.043 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:25:08.043 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_xinfo": [ 2024-05-22T21:25:08.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.043 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 0, 2024-05-22T21:25:08.043 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:08.043 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:08.043 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:08.043 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:08.043 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:08.043 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.198975+0000", 2024-05-22T21:25:08.043 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:08.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.043 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.043 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 1, 2024-05-22T21:25:08.044 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:08.044 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:08.044 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:08.044 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:08.044 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:08.044 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.091069+0000", 2024-05-22T21:25:08.044 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:08.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.044 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.044 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 2, 2024-05-22T21:25:08.044 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:08.044 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:08.045 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:08.045 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:08.045 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:08.045 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.207346+0000", 2024-05-22T21:25:08.045 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:08.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.045 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.045 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 3, 2024-05-22T21:25:08.045 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:08.045 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:08.045 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:08.045 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:08.046 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:08.046 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.202216+0000", 2024-05-22T21:25:08.046 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:08.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.046 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.046 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 4, 2024-05-22T21:25:08.046 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:08.046 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:08.046 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:08.046 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:08.046 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:08.046 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.137328+0000", 2024-05-22T21:25:08.046 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:08.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.047 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 5, 2024-05-22T21:25:08.047 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:08.047 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:08.047 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:08.047 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:08.047 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:08.047 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.127225+0000", 2024-05-22T21:25:08.047 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:08.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.047 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.047 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 6, 2024-05-22T21:25:08.047 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:08.048 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:08.048 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:08.048 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:08.048 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:08.048 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.120135+0000", 2024-05-22T21:25:08.048 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:08.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.048 INFO:teuthology.orchestra.run.smithi161.stdout: { 2024-05-22T21:25:08.048 INFO:teuthology.orchestra.run.smithi161.stdout: "osd": 7, 2024-05-22T21:25:08.048 INFO:teuthology.orchestra.run.smithi161.stdout: "down_stamp": "0.000000", 2024-05-22T21:25:08.048 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_probability": 0, 2024-05-22T21:25:08.048 INFO:teuthology.orchestra.run.smithi161.stdout: "laggy_interval": 0, 2024-05-22T21:25:08.048 INFO:teuthology.orchestra.run.smithi161.stdout: "features": 4540138322906710015, 2024-05-22T21:25:08.049 INFO:teuthology.orchestra.run.smithi161.stdout: "old_weight": 0, 2024-05-22T21:25:08.049 INFO:teuthology.orchestra.run.smithi161.stdout: "last_purged_snaps_scrub": "2024-05-22T21:13:09.159112+0000", 2024-05-22T21:25:08.049 INFO:teuthology.orchestra.run.smithi161.stdout: "dead_epoch": 0 2024-05-22T21:25:08.049 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.049 INFO:teuthology.orchestra.run.smithi161.stdout: ], 2024-05-22T21:25:08.049 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap": [], 2024-05-22T21:25:08.049 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_items": [], 2024-05-22T21:25:08.049 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_upmap_primaries": [], 2024-05-22T21:25:08.049 INFO:teuthology.orchestra.run.smithi161.stdout: "pg_temp": [], 2024-05-22T21:25:08.049 INFO:teuthology.orchestra.run.smithi161.stdout: "primary_temp": [], 2024-05-22T21:25:08.049 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklist": {}, 2024-05-22T21:25:08.049 INFO:teuthology.orchestra.run.smithi161.stdout: "range_blocklist": {}, 2024-05-22T21:25:08.049 INFO:teuthology.orchestra.run.smithi161.stdout: "erasure_code_profiles": { 2024-05-22T21:25:08.050 INFO:teuthology.orchestra.run.smithi161.stdout: "default": { 2024-05-22T21:25:08.050 INFO:teuthology.orchestra.run.smithi161.stdout: "crush-failure-domain": "osd", 2024-05-22T21:25:08.050 INFO:teuthology.orchestra.run.smithi161.stdout: "k": "2", 2024-05-22T21:25:08.050 INFO:teuthology.orchestra.run.smithi161.stdout: "m": "1", 2024-05-22T21:25:08.050 INFO:teuthology.orchestra.run.smithi161.stdout: "plugin": "jerasure", 2024-05-22T21:25:08.050 INFO:teuthology.orchestra.run.smithi161.stdout: "technique": "reed_sol_van" 2024-05-22T21:25:08.050 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-22T21:25:08.050 INFO:teuthology.orchestra.run.smithi161.stdout: "removed_snaps_queue": [], 2024-05-22T21:25:08.050 INFO:teuthology.orchestra.run.smithi161.stdout: "new_removed_snaps": [], 2024-05-22T21:25:08.050 INFO:teuthology.orchestra.run.smithi161.stdout: "new_purged_snaps": [], 2024-05-22T21:25:08.050 INFO:teuthology.orchestra.run.smithi161.stdout: "crush_node_flags": {}, 2024-05-22T21:25:08.051 INFO:teuthology.orchestra.run.smithi161.stdout: "device_class_flags": {}, 2024-05-22T21:25:08.051 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode": { 2024-05-22T21:25:08.051 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_enabled": false, 2024-05-22T21:25:08.051 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_bucket_count": 0, 2024-05-22T21:25:08.051 INFO:teuthology.orchestra.run.smithi161.stdout: "degraded_stretch_mode": 0, 2024-05-22T21:25:08.051 INFO:teuthology.orchestra.run.smithi161.stdout: "recovering_stretch_mode": 0, 2024-05-22T21:25:08.051 INFO:teuthology.orchestra.run.smithi161.stdout: "stretch_mode_bucket": 0 2024-05-22T21:25:08.051 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-22T21:25:08.051 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:25:08.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.966+0000 7f1570c00640 1 -- 172.21.15.161:0/2610575511 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1548066690 msgr2=0x7f1548068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:08.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.966+0000 7f1570c00640 1 --2- 172.21.15.161:0/2610575511 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1548066690 0x7f1548068b50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1554005890 tx=0x7f1554001970 comp rx=0 tx=0).stop 2024-05-22T21:25:08.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.966+0000 7f1570c00640 1 -- 172.21.15.161:0/2610575511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f156c15c6d0 msgr2=0x7f156c1436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:08.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.966+0000 7f1570c00640 1 --2- 172.21.15.161:0/2610575511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f156c15c6d0 0x7f156c1436d0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f155c00c7a0 tx=0x7f155c00cc60 comp rx=0 tx=0).stop 2024-05-22T21:25:08.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.966+0000 7f1570c00640 1 -- 172.21.15.161:0/2610575511 shutdown_connections 2024-05-22T21:25:08.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.966+0000 7f1570c00640 1 --2- 172.21.15.161:0/2610575511 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1548066690 0x7f1548068b50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:08.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.966+0000 7f1570c00640 1 --2- 172.21.15.161:0/2610575511 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f156c15e130 0x7f156c0a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:08.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.966+0000 7f1570c00640 1 --2- 172.21.15.161:0/2610575511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f156c15dd50 0x7f156c0a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:08.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.966+0000 7f1570c00640 1 --2- 172.21.15.161:0/2610575511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f156c15c6d0 0x7f156c1436d0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:08.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.966+0000 7f1570c00640 1 -- 172.21.15.161:0/2610575511 >> 172.21.15.161:0/2610575511 conn(0x7f156c0b0bc0 msgr2=0x7f156c0afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:08.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.966+0000 7f1570c00640 1 -- 172.21.15.161:0/2610575511 shutdown_connections 2024-05-22T21:25:08.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:07.966+0000 7f1570c00640 1 -- 172.21.15.161:0/2610575511 wait complete. 2024-05-22T21:25:08.054 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:25:08.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.190+0000 7f2bd4000640 1 Processor -- start 2024-05-22T21:25:08.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.190+0000 7f2bd4000640 1 -- start start 2024-05-22T21:25:08.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.190+0000 7f2bd4000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2bcc1597f0 0x7f2bcc0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:08.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.190+0000 7f2bd4000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bcc071040 con 0x7f2bcc0b3070 2024-05-22T21:25:08.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.190+0000 7f2bd4000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bcc0b34e0 con 0x7f2bcc070be0 2024-05-22T21:25:08.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.190+0000 7f2bd4000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bcc159bd0 con 0x7f2bcc1597f0 2024-05-22T21:25:08.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.190+0000 7f2bd1600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2bcc1597f0 0x7f2bcc0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:08.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.190+0000 7f2bd2000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f2bcc0b3070 0x7f2bcc0cc550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39466/0 (socket says 172.21.15.161:39466) 2024-05-22T21:25:08.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.190+0000 7f2bd2000640 1 -- 172.21.15.161:0/3905339251 learned_addr learned my addr 172.21.15.161:0/3905339251 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:08.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.190+0000 7f2bd1600640 1 -- 172.21.15.161:0/3905339251 >> v1:172.21.15.179:6789/0 conn(0x7f2bcc070be0 legacy=0x7f2bcc0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:08.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.190+0000 7f2bd1600640 1 -- 172.21.15.161:0/3905339251 >> v1:172.21.15.161:6789/0 conn(0x7f2bcc0b3070 legacy=0x7f2bcc0cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:08.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.190+0000 7f2bd1600640 1 -- 172.21.15.161:0/3905339251 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc0cbdd0 con 0x7f2bcc1597f0 2024-05-22T21:25:08.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.190+0000 7f2bcbe00640 1 -- 172.21.15.161:0/3905339251 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583260133 0 0) 0x7f2bcc071040 con 0x7f2bcc0b3070 2024-05-22T21:25:08.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.194+0000 7f2bd1600640 1 --2- 172.21.15.161:0/3905339251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bcc1597f0 0x7f2bcc0706a0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000a960 tx=0x7f2bc001d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f75dc4588025b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:08.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.194+0000 7f2bcbe00640 1 -- 172.21.15.161:0/3905339251 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00a68a0 con 0x7f2bcc1597f0 2024-05-22T21:25:08.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.194+0000 7f2bcbe00640 1 -- 172.21.15.161:0/3905339251 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc00a6a00 con 0x7f2bcc1597f0 2024-05-22T21:25:08.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.194+0000 7f2bcbe00640 1 -- 172.21.15.161:0/3905339251 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00a6d90 con 0x7f2bcc1597f0 2024-05-22T21:25:08.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.194+0000 7f2bd4000640 1 -- 172.21.15.161:0/3905339251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bcc1597f0 msgr2=0x7f2bcc0706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:08.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.194+0000 7f2bd4000640 1 --2- 172.21.15.161:0/3905339251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bcc1597f0 0x7f2bcc0706a0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000a960 tx=0x7f2bc001d870 comp rx=0 tx=0).stop 2024-05-22T21:25:08.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.194+0000 7f2bd4000640 1 -- 172.21.15.161:0/3905339251 shutdown_connections 2024-05-22T21:25:08.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.194+0000 7f2bd4000640 1 --2- 172.21.15.161:0/3905339251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bcc1597f0 0x7f2bcc0706a0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:08.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.194+0000 7f2bd4000640 1 -- 172.21.15.161:0/3905339251 >> 172.21.15.161:0/3905339251 conn(0x7f2bcc0b0bc0 msgr2=0x7f2bcc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:08.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.194+0000 7f2bd4000640 1 -- 172.21.15.161:0/3905339251 shutdown_connections 2024-05-22T21:25:08.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.194+0000 7f2bd4000640 1 -- 172.21.15.161:0/3905339251 wait complete. 2024-05-22T21:25:08.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.194+0000 7f2bd4000640 1 Processor -- start 2024-05-22T21:25:08.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd4000640 1 -- start start 2024-05-22T21:25:08.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd4000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2bcc070be0 0x7f2bcc15c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:08.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd4000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bcc0b3070 0x7f2bcc1491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:08.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd4000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bcc1597f0 0x7f2bcc155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:08.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd4000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc1587e0 con 0x7f2bcc1597f0 2024-05-22T21:25:08.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd4000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc15cd30 con 0x7f2bcc0b3070 2024-05-22T21:25:08.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd4000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc159d90 con 0x7f2bcc070be0 2024-05-22T21:25:08.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bcc1597f0 0x7f2bcc155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:08.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bcc1597f0 0x7f2bcc155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46724/0 (socket says 172.21.15.161:46724) 2024-05-22T21:25:08.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd2000640 1 -- 172.21.15.161:0/2169696041 learned_addr learned my addr 172.21.15.161:0/2169696041 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:08.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd0c00640 1 --2- 172.21.15.161:0/2169696041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bcc0b3070 0x7f2bcc1491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:08.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd1600640 1 --2- 172.21.15.161:0/2169696041 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2bcc070be0 0x7f2bcc15c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:08.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd2000640 1 -- 172.21.15.161:0/2169696041 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2bcc070be0 msgr2=0x7f2bcc15c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:08.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd2000640 1 --2- 172.21.15.161:0/2169696041 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2bcc070be0 0x7f2bcc15c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:08.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd2000640 1 -- 172.21.15.161:0/2169696041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bcc0b3070 msgr2=0x7f2bcc1491b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:08.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd2000640 1 --2- 172.21.15.161:0/2169696041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bcc0b3070 0x7f2bcc1491b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:08.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd2000640 1 -- 172.21.15.161:0/2169696041 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc000a5c0 con 0x7f2bcc1597f0 2024-05-22T21:25:08.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd0c00640 1 --2- 172.21.15.161:0/2169696041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bcc0b3070 0x7f2bcc1491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:08.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd2000640 1 --2- 172.21.15.161:0/2169696041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bcc1597f0 0x7f2bcc155710 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4001570 tx=0x7f2bc400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:08.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bc9600640 1 -- 172.21.15.161:0/2169696041 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc400a640 con 0x7f2bcc1597f0 2024-05-22T21:25:08.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bc9600640 1 -- 172.21.15.161:0/2169696041 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc400a7c0 con 0x7f2bcc1597f0 2024-05-22T21:25:08.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bc9600640 1 -- 172.21.15.161:0/2169696041 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc400ab70 con 0x7f2bcc1597f0 2024-05-22T21:25:08.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd4000640 1 -- 172.21.15.161:0/2169696041 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bcc157c60 con 0x7f2bcc1597f0 2024-05-22T21:25:08.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd4000640 1 -- 172.21.15.161:0/2169696041 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bcc158220 con 0x7f2bcc1597f0 2024-05-22T21:25:08.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.198+0000 7f2bd4000640 1 -- 172.21.15.161:0/2169696041 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bcc067220 con 0x7f2bcc1597f0 2024-05-22T21:25:08.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.202+0000 7f2bc9600640 1 -- 172.21.15.161:0/2169696041 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2bc400acd0 con 0x7f2bcc1597f0 2024-05-22T21:25:08.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.202+0000 7f2bc9600640 1 --2- 172.21.15.161:0/2169696041 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2bac066690 0x7f2bac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:08.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.202+0000 7f2bc9600640 1 -- 172.21.15.161:0/2169696041 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bc4083800 con 0x7f2bcc1597f0 2024-05-22T21:25:08.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.202+0000 7f2bd1600640 1 --2- 172.21.15.161:0/2169696041 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2bac066690 0x7f2bac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:08.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.202+0000 7f2bd1600640 1 --2- 172.21.15.161:0/2169696041 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2bac066690 0x7f2bac068b50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00b0000 tx=0x7f2bc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:08.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.206+0000 7f2bc9600640 1 -- 172.21.15.161:0/2169696041 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f2bc4050ad0 con 0x7f2bcc1597f0 2024-05-22T21:25:08.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.390+0000 7f2bd4000640 1 -- 172.21.15.161:0/2169696041 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bcc1496f0 con 0x7f2bcc1597f0 2024-05-22T21:25:08.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.390+0000 7f2bc9600640 1 -- 172.21.15.161:0/2169696041 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4744 (secure 0 0 0) 0x7f2bc4018020 con 0x7f2bcc1597f0 2024-05-22T21:25:08.393 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:08.393 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":86,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3304182060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3304182060},{"type":"v1","addr":"172.21.15.179:6835","nonce":3304182060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196309272},{"type":"v1","addr":"172.21.15.161:6836","nonce":196309272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/396408969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":396408969},{"type":"v1","addr":"172.21.15.179:6834","nonce":396408969}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:24:34.096673+0000","modified":"2024-05-22T21:24:37.112845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8784},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8784":{"gid":8784,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2172511349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2172511349},{"type":"v1","addr":"172.21.15.161:6837","nonce":2172511349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-05-22T21:25:08.393 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 86 2024-05-22T21:25:08.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.394+0000 7f2bd4000640 1 -- 172.21.15.161:0/2169696041 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2bac066690 msgr2=0x7f2bac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:08.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.394+0000 7f2bd4000640 1 --2- 172.21.15.161:0/2169696041 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2bac066690 0x7f2bac068b50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00b0000 tx=0x7f2bc0008040 comp rx=0 tx=0).stop 2024-05-22T21:25:08.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.394+0000 7f2bd4000640 1 -- 172.21.15.161:0/2169696041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bcc1597f0 msgr2=0x7f2bcc155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:08.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.394+0000 7f2bd4000640 1 --2- 172.21.15.161:0/2169696041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bcc1597f0 0x7f2bcc155710 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4001570 tx=0x7f2bc400cca0 comp rx=0 tx=0).stop 2024-05-22T21:25:08.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.394+0000 7f2bd4000640 1 -- 172.21.15.161:0/2169696041 shutdown_connections 2024-05-22T21:25:08.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.394+0000 7f2bd4000640 1 --2- 172.21.15.161:0/2169696041 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f2bac066690 0x7f2bac068b50 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:08.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.394+0000 7f2bd4000640 1 --2- 172.21.15.161:0/2169696041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bcc1597f0 0x7f2bcc155710 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:08.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.394+0000 7f2bd4000640 1 --2- 172.21.15.161:0/2169696041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bcc0b3070 0x7f2bcc1491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:08.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.394+0000 7f2bd4000640 1 --2- 172.21.15.161:0/2169696041 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2bcc070be0 0x7f2bcc15c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:08.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.394+0000 7f2bd4000640 1 -- 172.21.15.161:0/2169696041 >> 172.21.15.161:0/2169696041 conn(0x7f2bcc0b0bc0 msgr2=0x7f2bcc0cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:08.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.394+0000 7f2bd4000640 1 -- 172.21.15.161:0/2169696041 shutdown_connections 2024-05-22T21:25:08.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:08.394+0000 7f2bd4000640 1 -- 172.21.15.161:0/2169696041 wait complete. 2024-05-22T21:25:08.414 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-05-22T21:25:08.568 INFO:teuthology.orchestra.run.smithi161.stdout:{"cluster_fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","whoami":0,"id":8784,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":34.457456555999997,"mdsmap_epoch":86,"osdmap_epoch":193,"osdmap_epoch_barrier":192,"uptime":39.707468908000003} 2024-05-22T21:25:08.568 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "d6fa8528-b1bf-41b6-9f76-c0d715394087", "fs_name": "cephfs", "id": 8784, "mdsmap_epoch": 86, "osdmap_epoch": 193, "osdmap_epoch_barrier": 192, "rank_uptime": 34.457456556, "state": "up:active", "uptime": 39.707468908, "want_state": "up:active", "whoami": 0 } 2024-05-22T21:25:08.569 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-22T21:25:08.569 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:25:08.569 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:25:08.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:08.578 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-05-22T21:25:09.084 INFO:teuthology.orchestra.run.smithi161.stderr:50+0 records in 2024-05-22T21:25:09.084 INFO:teuthology.orchestra.run.smithi161.stderr:50+0 records out 2024-05-22T21:25:09.085 INFO:teuthology.orchestra.run.smithi161.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.455031 s, 115 MB/s 2024-05-22T21:25:09.086 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-22T21:25:09.086 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:25:09.159 DEBUG:teuthology.orchestra.run.smithi179:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-22T21:25:09.204 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-22T21:25:09.205 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:25:09.205 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:25:09.214 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:25:09.214 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:25:09.265 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:25:09.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.442+0000 7f3865a00640 1 Processor -- start 2024-05-22T21:25:09.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.442+0000 7f3865a00640 1 -- start start 2024-05-22T21:25:09.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.442+0000 7f3865a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3860143ec0 0x7f386015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:09.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.442+0000 7f3865a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38600715a0 0x7f386006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:09.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.442+0000 7f3865a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38601442a0 con 0x7f386015ec90 2024-05-22T21:25:09.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.442+0000 7f3865a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3860070260 con 0x7f3860143ec0 2024-05-22T21:25:09.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.442+0000 7f3865a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3860070500 con 0x7f38600715a0 2024-05-22T21:25:09.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.442+0000 7f385f400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f386015ec90 0x7f386006bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39476/0 (socket says 172.21.15.161:39476) 2024-05-22T21:25:09.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.442+0000 7f385f400640 1 -- 172.21.15.161:0/2177543810 learned_addr learned my addr 172.21.15.161:0/2177543810 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:09.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.442+0000 7f385ea00640 1 --2- 172.21.15.161:0/2177543810 >> v2:172.21.15.161:3300/0 conn(0x7f3860143ec0 0x7f386015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:09.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.442+0000 7f3856000640 1 --2- 172.21.15.161:0/2177543810 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38600715a0 0x7f386006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:09.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.446+0000 7f385ea00640 1 -- 172.21.15.161:0/2177543810 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38600715a0 msgr2=0x7f386006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:09.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.446+0000 7f385ea00640 1 --2- 172.21.15.161:0/2177543810 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38600715a0 0x7f386006fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.446+0000 7f385ea00640 1 -- 172.21.15.161:0/2177543810 >> v1:172.21.15.161:6789/0 conn(0x7f386015ec90 legacy=0x7f386006bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:09.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.446+0000 7f385ea00640 1 -- 172.21.15.161:0/2177543810 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38600707e0 con 0x7f3860143ec0 2024-05-22T21:25:09.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.446+0000 7f385e000640 1 -- 172.21.15.161:0/2177543810 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562331951 0 0) 0x7f38601442a0 con 0x7f386015ec90 2024-05-22T21:25:09.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.446+0000 7f385ea00640 1 --2- 172.21.15.161:0/2177543810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3860143ec0 0x7f386015f830 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f38480097c0 tx=0x7f384801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caea6bfc9d08cd1e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:09.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.446+0000 7f385e000640 1 -- 172.21.15.161:0/2177543810 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38480ab670 con 0x7f3860143ec0 2024-05-22T21:25:09.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.446+0000 7f385e000640 1 -- 172.21.15.161:0/2177543810 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38480ab7d0 con 0x7f3860143ec0 2024-05-22T21:25:09.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.446+0000 7f385e000640 1 -- 172.21.15.161:0/2177543810 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38480abb60 con 0x7f3860143ec0 2024-05-22T21:25:09.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.446+0000 7f3865a00640 1 -- 172.21.15.161:0/2177543810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3860143ec0 msgr2=0x7f386015f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:09.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.446+0000 7f3865a00640 1 --2- 172.21.15.161:0/2177543810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3860143ec0 0x7f386015f830 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f38480097c0 tx=0x7f384801c470 comp rx=0 tx=0).stop 2024-05-22T21:25:09.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3865a00640 1 -- 172.21.15.161:0/2177543810 shutdown_connections 2024-05-22T21:25:09.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3865a00640 1 --2- 172.21.15.161:0/2177543810 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38600715a0 0x7f386006fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3865a00640 1 --2- 172.21.15.161:0/2177543810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3860143ec0 0x7f386015f830 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3865a00640 1 -- 172.21.15.161:0/2177543810 >> 172.21.15.161:0/2177543810 conn(0x7f38600b0bc0 msgr2=0x7f38600b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:09.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3865a00640 1 -- 172.21.15.161:0/2177543810 shutdown_connections 2024-05-22T21:25:09.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3865a00640 1 -- 172.21.15.161:0/2177543810 wait complete. 2024-05-22T21:25:09.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3865a00640 1 Processor -- start 2024-05-22T21:25:09.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3865a00640 1 -- start start 2024-05-22T21:25:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3865a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3860143ec0 0x7f3860165a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3865a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f386015ec90 0x7f38601415c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3865a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f386013f0a0 0x7f386013ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3865a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38601411a0 con 0x7f386013f0a0 2024-05-22T21:25:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3865a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38601428f0 con 0x7f386015ec90 2024-05-22T21:25:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3865a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3860142330 con 0x7f3860143ec0 2024-05-22T21:25:09.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f385f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f386013f0a0 0x7f386013ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:09.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f385f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f386013f0a0 0x7f386013ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46738/0 (socket says 172.21.15.161:46738) 2024-05-22T21:25:09.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f385f400640 1 -- 172.21.15.161:0/4041637785 learned_addr learned my addr 172.21.15.161:0/4041637785 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:09.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f385ea00640 1 --2- 172.21.15.161:0/4041637785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3860143ec0 0x7f3860165a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:09.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3856000640 1 --2- 172.21.15.161:0/4041637785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f386015ec90 0x7f38601415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:09.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f385f400640 1 -- 172.21.15.161:0/4041637785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3860143ec0 msgr2=0x7f3860165a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:09.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f385f400640 1 --2- 172.21.15.161:0/4041637785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3860143ec0 0x7f3860165a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f385f400640 1 -- 172.21.15.161:0/4041637785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f386015ec90 msgr2=0x7f38601415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:09.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f385f400640 1 --2- 172.21.15.161:0/4041637785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f386015ec90 0x7f38601415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f385f400640 1 -- 172.21.15.161:0/4041637785 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3848009420 con 0x7f386013f0a0 2024-05-22T21:25:09.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f385f400640 1 --2- 172.21.15.161:0/4041637785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f386013f0a0 0x7f386013ea20 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f385800c780 tx=0x7f385800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:09.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3857400640 1 -- 172.21.15.161:0/4041637785 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385800ce90 con 0x7f386013f0a0 2024-05-22T21:25:09.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3857400640 1 -- 172.21.15.161:0/4041637785 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385800a640 con 0x7f386013f0a0 2024-05-22T21:25:09.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3857400640 1 -- 172.21.15.161:0/4041637785 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385800a9d0 con 0x7f386013f0a0 2024-05-22T21:25:09.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3865a00640 1 -- 172.21.15.161:0/4041637785 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3860142610 con 0x7f386013f0a0 2024-05-22T21:25:09.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.450+0000 7f3865a00640 1 -- 172.21.15.161:0/4041637785 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38601f9560 con 0x7f386013f0a0 2024-05-22T21:25:09.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.454+0000 7f3865a00640 1 -- 172.21.15.161:0/4041637785 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3860142a80 con 0x7f386013f0a0 2024-05-22T21:25:09.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.454+0000 7f3857400640 1 -- 172.21.15.161:0/4041637785 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f385800ab30 con 0x7f386013f0a0 2024-05-22T21:25:09.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.454+0000 7f3857400640 1 --2- 172.21.15.161:0/4041637785 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f383c0666c0 0x7f383c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:09.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.454+0000 7f3857400640 1 -- 172.21.15.161:0/4041637785 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38580834a0 con 0x7f386013f0a0 2024-05-22T21:25:09.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.454+0000 7f385ea00640 1 --2- 172.21.15.161:0/4041637785 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f383c0666c0 0x7f383c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:09.456 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.454+0000 7f385ea00640 1 --2- 172.21.15.161:0/4041637785 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f383c0666c0 0x7f383c068b80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f38480af000 tx=0x7f38480a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:09.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.458+0000 7f3857400640 1 -- 172.21.15.161:0/4041637785 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3860142a80 con 0x7f386013f0a0 2024-05-22T21:25:09.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.614+0000 7f3865a00640 1 -- 172.21.15.161:0/4041637785 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38601412e0 con 0x7f386013f0a0 2024-05-22T21:25:09.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.614+0000 7f3857400640 1 -- 172.21.15.161:0/4041637785 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+16159 (secure 0 0 0) 0x7f38601412e0 con 0x7f386013f0a0 2024-05-22T21:25:09.616 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:09.616 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":193,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:25:05.312760+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:24:31.797440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-05-22T21:24:33.098240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:25:09.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.618+0000 7f3865a00640 1 -- 172.21.15.161:0/4041637785 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f383c0666c0 msgr2=0x7f383c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:09.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.618+0000 7f3865a00640 1 --2- 172.21.15.161:0/4041637785 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f383c0666c0 0x7f383c068b80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f38480af000 tx=0x7f38480a7000 comp rx=0 tx=0).stop 2024-05-22T21:25:09.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.618+0000 7f3865a00640 1 -- 172.21.15.161:0/4041637785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f386013f0a0 msgr2=0x7f386013ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:09.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.618+0000 7f3865a00640 1 --2- 172.21.15.161:0/4041637785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f386013f0a0 0x7f386013ea20 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f385800c780 tx=0x7f385800cc40 comp rx=0 tx=0).stop 2024-05-22T21:25:09.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.618+0000 7f3865a00640 1 -- 172.21.15.161:0/4041637785 shutdown_connections 2024-05-22T21:25:09.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.618+0000 7f3865a00640 1 --2- 172.21.15.161:0/4041637785 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f383c0666c0 0x7f383c068b80 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.618+0000 7f3865a00640 1 --2- 172.21.15.161:0/4041637785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f386013f0a0 0x7f386013ea20 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.618+0000 7f3865a00640 1 --2- 172.21.15.161:0/4041637785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f386015ec90 0x7f38601415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.618+0000 7f3865a00640 1 --2- 172.21.15.161:0/4041637785 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3860143ec0 0x7f3860165a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.618+0000 7f3865a00640 1 -- 172.21.15.161:0/4041637785 >> 172.21.15.161:0/4041637785 conn(0x7f38600b0bc0 msgr2=0x7f386006bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:09.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.618+0000 7f3865a00640 1 -- 172.21.15.161:0/4041637785 shutdown_connections 2024-05-22T21:25:09.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.618+0000 7f3865a00640 1 -- 172.21.15.161:0/4041637785 wait complete. 2024-05-22T21:25:09.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:24:31.797440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '189', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:24:33.098240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '193', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:25:09.638 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:25:09.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.774+0000 7f851ee00640 1 Processor -- start 2024-05-22T21:25:09.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.774+0000 7f851ee00640 1 -- start start 2024-05-22T21:25:09.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.778+0000 7f851ee00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f851806c9b0 0x7f85180b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:09.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.778+0000 7f851ee00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8518144bb0 con 0x7f8518143c40 2024-05-22T21:25:09.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.778+0000 7f851ee00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851815efb0 con 0x7f851815b600 2024-05-22T21:25:09.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.778+0000 7f851ee00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851815ba60 con 0x7f851806c9b0 2024-05-22T21:25:09.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.778+0000 7f8517e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8518143c40 0x7f85181444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39492/0 (socket says 172.21.15.161:39492) 2024-05-22T21:25:09.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.778+0000 7f8517e00640 1 -- 172.21.15.161:0/1319893981 learned_addr learned my addr 172.21.15.161:0/1319893981 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:09.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.778+0000 7f851ce00640 1 --2- 172.21.15.161:0/1319893981 >> v2:172.21.15.179:3300/0 conn(0x7f851806c9b0 0x7f85180b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:09.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.778+0000 7f8516a00640 1 -- 172.21.15.161:0/1319893981 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292915728 0 0) 0x7f8518144bb0 con 0x7f8518143c40 2024-05-22T21:25:09.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.778+0000 7f8516a00640 1 -- 172.21.15.161:0/1319893981 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84fc003530 con 0x7f8518143c40 2024-05-22T21:25:09.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.778+0000 7f851ce00640 1 -- 172.21.15.161:0/1319893981 >> v1:172.21.15.179:6789/0 conn(0x7f851815b600 legacy=0x7f851806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:09.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.778+0000 7f8516a00640 1 -- 172.21.15.161:0/1319893981 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039176452 0 0) 0x7f851815efb0 con 0x7f851815b600 2024-05-22T21:25:09.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.778+0000 7f851ce00640 1 -- 172.21.15.161:0/1319893981 >> v1:172.21.15.161:6789/0 conn(0x7f8518143c40 legacy=0x7f85181444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:09.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.778+0000 7f851ce00640 1 -- 172.21.15.161:0/1319893981 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851815f290 con 0x7f851806c9b0 2024-05-22T21:25:09.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.782+0000 7f851ce00640 1 --2- 172.21.15.161:0/1319893981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f851806c9b0 0x7f85180b2160 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f85100097c0 tx=0x7f851001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48c30fe4d985d6c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:09.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.782+0000 7f8516a00640 1 -- 172.21.15.161:0/1319893981 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85100a6040 con 0x7f851806c9b0 2024-05-22T21:25:09.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.782+0000 7f8516a00640 1 -- 172.21.15.161:0/1319893981 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85100aa600 con 0x7f851806c9b0 2024-05-22T21:25:09.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.782+0000 7f8516a00640 1 -- 172.21.15.161:0/1319893981 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85100aa970 con 0x7f851806c9b0 2024-05-22T21:25:09.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.782+0000 7f851ee00640 1 -- 172.21.15.161:0/1319893981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f851806c9b0 msgr2=0x7f85180b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:09.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.782+0000 7f851ee00640 1 --2- 172.21.15.161:0/1319893981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f851806c9b0 0x7f85180b2160 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f85100097c0 tx=0x7f851001c4a0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.782+0000 7f851ee00640 1 -- 172.21.15.161:0/1319893981 shutdown_connections 2024-05-22T21:25:09.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.782+0000 7f851ee00640 1 --2- 172.21.15.161:0/1319893981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f851806c9b0 0x7f85180b2160 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.782+0000 7f851ee00640 1 -- 172.21.15.161:0/1319893981 >> 172.21.15.161:0/1319893981 conn(0x7f851806a2d0 msgr2=0x7f851806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:09.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.782+0000 7f851ee00640 1 -- 172.21.15.161:0/1319893981 shutdown_connections 2024-05-22T21:25:09.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.782+0000 7f851ee00640 1 -- 172.21.15.161:0/1319893981 wait complete. 2024-05-22T21:25:09.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.782+0000 7f851ee00640 1 Processor -- start 2024-05-22T21:25:09.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.782+0000 7f851ee00640 1 -- start start 2024-05-22T21:25:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f851806c9b0 0x7f85180affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ee00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8518143c40 0x7f8518135d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ee00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f851815b600 0x7f851813ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ee00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85180afb40 con 0x7f851815b600 2024-05-22T21:25:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8518136270 con 0x7f851806c9b0 2024-05-22T21:25:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ee00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8518156620 con 0x7f8518143c40 2024-05-22T21:25:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f851815b600 0x7f851813ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f851815b600 0x7f851813ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46740/0 (socket says 172.21.15.161:46740) 2024-05-22T21:25:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ce00640 1 -- 172.21.15.161:0/721594158 learned_addr learned my addr 172.21.15.161:0/721594158 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f8517400640 1 --2- 172.21.15.161:0/721594158 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8518143c40 0x7f8518135d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f8517e00640 1 --2- 172.21.15.161:0/721594158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f851806c9b0 0x7f85180affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:09.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ce00640 1 -- 172.21.15.161:0/721594158 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8518143c40 msgr2=0x7f8518135d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:09.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ce00640 1 --2- 172.21.15.161:0/721594158 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8518143c40 0x7f8518135d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ce00640 1 -- 172.21.15.161:0/721594158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f851806c9b0 msgr2=0x7f85180affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:09.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ce00640 1 --2- 172.21.15.161:0/721594158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f851806c9b0 0x7f85180affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ce00640 1 -- 172.21.15.161:0/721594158 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8510009420 con 0x7f851815b600 2024-05-22T21:25:09.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ce00640 1 --2- 172.21.15.161:0/721594158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f851815b600 0x7f851813ed10 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f8510009710 tx=0x7f851001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:09.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f850fe00640 1 -- 172.21.15.161:0/721594158 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85100b3600 con 0x7f851815b600 2024-05-22T21:25:09.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f850fe00640 1 -- 172.21.15.161:0/721594158 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85100b3760 con 0x7f851815b600 2024-05-22T21:25:09.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f850fe00640 1 -- 172.21.15.161:0/721594158 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85100b3ad0 con 0x7f851815b600 2024-05-22T21:25:09.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ee00640 1 -- 172.21.15.161:0/721594158 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85180af250 con 0x7f851815b600 2024-05-22T21:25:09.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ee00640 1 -- 172.21.15.161:0/721594158 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85180af7d0 con 0x7f851815b600 2024-05-22T21:25:09.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f851ee00640 1 -- 172.21.15.161:0/721594158 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85180afcb0 con 0x7f851815b600 2024-05-22T21:25:09.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.786+0000 7f850fe00640 1 -- 172.21.15.161:0/721594158 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f85100b3c30 con 0x7f851815b600 2024-05-22T21:25:09.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.790+0000 7f850fe00640 1 --2- 172.21.15.161:0/721594158 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84f8066690 0x7f84f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:09.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.790+0000 7f850fe00640 1 -- 172.21.15.161:0/721594158 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f851011e520 con 0x7f851815b600 2024-05-22T21:25:09.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.790+0000 7f8517e00640 1 --2- 172.21.15.161:0/721594158 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84f8066690 0x7f84f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:09.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.790+0000 7f8517e00640 1 --2- 172.21.15.161:0/721594158 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84f8066690 0x7f84f8068b50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8508005c10 tx=0x7f8508009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:09.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.794+0000 7f850fe00640 1 -- 172.21.15.161:0/721594158 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f85100eb930 con 0x7f851815b600 2024-05-22T21:25:09.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.946+0000 7f851ee00640 1 -- 172.21.15.161:0/721594158 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8518067220 con 0x7f851815b600 2024-05-22T21:25:09.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.946+0000 7f850fe00640 1 -- 172.21.15.161:0/721594158 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v193) v1 ==== 93+0+10 (secure 0 0 0) 0x7f85100eef50 con 0x7f851815b600 2024-05-22T21:25:09.947 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:25:09.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.946+0000 7f851ee00640 1 -- 172.21.15.161:0/721594158 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84f8066690 msgr2=0x7f84f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:09.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.946+0000 7f851ee00640 1 --2- 172.21.15.161:0/721594158 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84f8066690 0x7f84f8068b50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8508005c10 tx=0x7f8508009040 comp rx=0 tx=0).stop 2024-05-22T21:25:09.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.946+0000 7f851ee00640 1 -- 172.21.15.161:0/721594158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f851815b600 msgr2=0x7f851813ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:09.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.946+0000 7f851ee00640 1 --2- 172.21.15.161:0/721594158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f851815b600 0x7f851813ed10 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f8510009710 tx=0x7f851001ace0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.946+0000 7f851ee00640 1 -- 172.21.15.161:0/721594158 shutdown_connections 2024-05-22T21:25:09.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.946+0000 7f851ee00640 1 --2- 172.21.15.161:0/721594158 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f84f8066690 0x7f84f8068b50 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.946+0000 7f851ee00640 1 --2- 172.21.15.161:0/721594158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f851815b600 0x7f851813ed10 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.946+0000 7f851ee00640 1 --2- 172.21.15.161:0/721594158 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8518143c40 0x7f8518135d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.946+0000 7f851ee00640 1 --2- 172.21.15.161:0/721594158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f851806c9b0 0x7f85180affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:09.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.946+0000 7f851ee00640 1 -- 172.21.15.161:0/721594158 >> 172.21.15.161:0/721594158 conn(0x7f851806a2d0 msgr2=0x7f8518156d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:09.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.946+0000 7f851ee00640 1 -- 172.21.15.161:0/721594158 shutdown_connections 2024-05-22T21:25:09.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:09.946+0000 7f851ee00640 1 -- 172.21.15.161:0/721594158 wait complete. 2024-05-22T21:25:09.968 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:25:10.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.102+0000 7f49b4000640 1 Processor -- start 2024-05-22T21:25:10.104 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.102+0000 7f49b4000640 1 -- start start 2024-05-22T21:25:10.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.106+0000 7f49b4000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49ac143c40 0x7f49ac15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.106+0000 7f49b4000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f49ac1444a0 0x7f49ac06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.106+0000 7f49b4000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f49ac144900 0x7f49ac06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.106+0000 7f49b4000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ac15efb0 con 0x7f49ac144900 2024-05-22T21:25:10.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.106+0000 7f49b4000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ac15f290 con 0x7f49ac1444a0 2024-05-22T21:25:10.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.106+0000 7f49b4000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac15f570 con 0x7f49ac143c40 2024-05-22T21:25:10.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.106+0000 7f49b2000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f49ac144900 0x7f49ac06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.106+0000 7f49b2000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f49ac144900 0x7f49ac06c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46744/0 (socket says 172.21.15.161:46744) 2024-05-22T21:25:10.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.106+0000 7f49b2000640 1 -- 172.21.15.161:0/4012678279 learned_addr learned my addr 172.21.15.161:0/4012678279 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:10.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.106+0000 7f49b0c00640 1 --2- 172.21.15.161:0/4012678279 >> v2:172.21.15.179:3300/0 conn(0x7f49ac1444a0 0x7f49ac06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.106+0000 7f49b1600640 1 --2- 172.21.15.161:0/4012678279 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49ac143c40 0x7f49ac15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.106+0000 7f49b2000640 1 -- 172.21.15.161:0/4012678279 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49ac143c40 msgr2=0x7f49ac15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:10.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.106+0000 7f49b2000640 1 --2- 172.21.15.161:0/4012678279 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49ac143c40 0x7f49ac15b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.106+0000 7f49b2000640 1 -- 172.21.15.161:0/4012678279 >> v2:172.21.15.179:3300/0 conn(0x7f49ac1444a0 msgr2=0x7f49ac06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:10.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.106+0000 7f49b2000640 1 --2- 172.21.15.161:0/4012678279 >> v2:172.21.15.179:3300/0 conn(0x7f49ac1444a0 0x7f49ac06c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.106+0000 7f49b2000640 1 -- 172.21.15.161:0/4012678279 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac15f850 con 0x7f49ac144900 2024-05-22T21:25:10.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b2000640 1 --2- 172.21.15.161:0/4012678279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49ac144900 0x7f49ac06c7e0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f49ac071fe0 tx=0x7f49a401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522061034f02305a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:10.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49abe00640 1 -- 172.21.15.161:0/4012678279 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a40a57d0 con 0x7f49ac144900 2024-05-22T21:25:10.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49abe00640 1 -- 172.21.15.161:0/4012678279 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a40a5930 con 0x7f49ac144900 2024-05-22T21:25:10.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49abe00640 1 -- 172.21.15.161:0/4012678279 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a40a5cc0 con 0x7f49ac144900 2024-05-22T21:25:10.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 -- 172.21.15.161:0/4012678279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49ac144900 msgr2=0x7f49ac06c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:10.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 --2- 172.21.15.161:0/4012678279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49ac144900 0x7f49ac06c7e0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f49ac071fe0 tx=0x7f49a401c9f0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 -- 172.21.15.161:0/4012678279 shutdown_connections 2024-05-22T21:25:10.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 --2- 172.21.15.161:0/4012678279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49ac144900 0x7f49ac06c7e0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 --2- 172.21.15.161:0/4012678279 >> v2:172.21.15.179:3300/0 conn(0x7f49ac1444a0 0x7f49ac06c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 --2- 172.21.15.161:0/4012678279 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49ac143c40 0x7f49ac15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 -- 172.21.15.161:0/4012678279 >> 172.21.15.161:0/4012678279 conn(0x7f49ac06a2d0 msgr2=0x7f49ac06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:10.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 -- 172.21.15.161:0/4012678279 shutdown_connections 2024-05-22T21:25:10.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 -- 172.21.15.161:0/4012678279 wait complete. 2024-05-22T21:25:10.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 Processor -- start 2024-05-22T21:25:10.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 -- start start 2024-05-22T21:25:10.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49ac143c40 0x7f49ac1576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49ac1444a0 0x7f49ac1371c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49ac144900 0x7f49ac1357e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac155070 con 0x7f49ac144900 2024-05-22T21:25:10.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac158150 con 0x7f49ac143c40 2024-05-22T21:25:10.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b4000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac158430 con 0x7f49ac1444a0 2024-05-22T21:25:10.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49ac144900 0x7f49ac1357e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b1600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49ac143c40 0x7f49ac1576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49ac144900 0x7f49ac1357e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46746/0 (socket says 172.21.15.161:46746) 2024-05-22T21:25:10.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b2000640 1 -- 172.21.15.161:0/3916932567 learned_addr learned my addr 172.21.15.161:0/3916932567 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:10.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.110+0000 7f49b0c00640 1 --2- 172.21.15.161:0/3916932567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49ac1444a0 0x7f49ac1371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.114+0000 7f49b1600640 1 -- 172.21.15.161:0/3916932567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49ac1444a0 msgr2=0x7f49ac1371c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:10.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.114+0000 7f49b1600640 1 --2- 172.21.15.161:0/3916932567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49ac1444a0 0x7f49ac1371c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.114+0000 7f49b1600640 1 -- 172.21.15.161:0/3916932567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49ac144900 msgr2=0x7f49ac1357e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:10.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.114+0000 7f49b1600640 1 --2- 172.21.15.161:0/3916932567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49ac144900 0x7f49ac1357e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.114+0000 7f49b1600640 1 -- 172.21.15.161:0/3916932567 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a4009b30 con 0x7f49ac143c40 2024-05-22T21:25:10.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.114+0000 7f49b1600640 1 --2- 172.21.15.161:0/3916932567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49ac143c40 0x7f49ac1576a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f49a000d5f0 tx=0x7f49a000dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:10.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.114+0000 7f49a9600640 1 -- 172.21.15.161:0/3916932567 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a0014620 con 0x7f49ac143c40 2024-05-22T21:25:10.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.114+0000 7f49a9600640 1 -- 172.21.15.161:0/3916932567 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a0014780 con 0x7f49ac143c40 2024-05-22T21:25:10.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.114+0000 7f49b4000640 1 -- 172.21.15.161:0/3916932567 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ac154d00 con 0x7f49ac143c40 2024-05-22T21:25:10.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.114+0000 7f49a9600640 1 -- 172.21.15.161:0/3916932567 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a0014b10 con 0x7f49ac143c40 2024-05-22T21:25:10.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.114+0000 7f49b4000640 1 -- 172.21.15.161:0/3916932567 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ac135d20 con 0x7f49ac143c40 2024-05-22T21:25:10.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.114+0000 7f49b4000640 1 -- 172.21.15.161:0/3916932567 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ac067220 con 0x7f49ac143c40 2024-05-22T21:25:10.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.114+0000 7f49a9600640 1 -- 172.21.15.161:0/3916932567 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f49a0014c70 con 0x7f49ac143c40 2024-05-22T21:25:10.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.114+0000 7f49a9600640 1 --2- 172.21.15.161:0/3916932567 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f498c066690 0x7f498c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.114+0000 7f49a9600640 1 -- 172.21.15.161:0/3916932567 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49a0084700 con 0x7f49ac143c40 2024-05-22T21:25:10.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.114+0000 7f49b0c00640 1 --2- 172.21.15.161:0/3916932567 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f498c066690 0x7f498c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.118+0000 7f49b0c00640 1 --2- 172.21.15.161:0/3916932567 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f498c066690 0x7f498c068b50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f49ac06adc0 tx=0x7f4998008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:10.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.122+0000 7f49a9600640 1 -- 172.21.15.161:0/3916932567 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f49a0051b20 con 0x7f49ac143c40 2024-05-22T21:25:10.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.274+0000 7f49b4000640 1 -- 172.21.15.161:0/3916932567 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49ac0cd260 con 0x7f49ac143c40 2024-05-22T21:25:10.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.274+0000 7f49a9600640 1 -- 172.21.15.161:0/3916932567 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v193) v1 ==== 104+0+11 (secure 0 0 0) 0x7f49a0055140 con 0x7f49ac143c40 2024-05-22T21:25:10.274 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:25:10.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.274+0000 7f49b4000640 1 -- 172.21.15.161:0/3916932567 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f498c066690 msgr2=0x7f498c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:10.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.274+0000 7f49b4000640 1 --2- 172.21.15.161:0/3916932567 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f498c066690 0x7f498c068b50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f49ac06adc0 tx=0x7f4998008040 comp rx=0 tx=0).stop 2024-05-22T21:25:10.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.274+0000 7f49b4000640 1 -- 172.21.15.161:0/3916932567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49ac143c40 msgr2=0x7f49ac1576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:10.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.274+0000 7f49b4000640 1 --2- 172.21.15.161:0/3916932567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49ac143c40 0x7f49ac1576a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f49a000d5f0 tx=0x7f49a000dab0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.274+0000 7f49b4000640 1 -- 172.21.15.161:0/3916932567 shutdown_connections 2024-05-22T21:25:10.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.274+0000 7f49b4000640 1 --2- 172.21.15.161:0/3916932567 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f498c066690 0x7f498c068b50 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.274+0000 7f49b4000640 1 --2- 172.21.15.161:0/3916932567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49ac144900 0x7f49ac1357e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.274+0000 7f49b4000640 1 --2- 172.21.15.161:0/3916932567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49ac1444a0 0x7f49ac1371c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.274+0000 7f49b4000640 1 --2- 172.21.15.161:0/3916932567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49ac143c40 0x7f49ac1576a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.274+0000 7f49b4000640 1 -- 172.21.15.161:0/3916932567 >> 172.21.15.161:0/3916932567 conn(0x7f49ac06a2d0 msgr2=0x7f49ac159a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:10.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.274+0000 7f49b4000640 1 -- 172.21.15.161:0/3916932567 shutdown_connections 2024-05-22T21:25:10.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.274+0000 7f49b4000640 1 -- 172.21.15.161:0/3916932567 wait complete. 2024-05-22T21:25:10.295 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:25:10.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.434+0000 7ff940800640 1 Processor -- start 2024-05-22T21:25:10.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.434+0000 7ff940800640 1 -- start start 2024-05-22T21:25:10.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.434+0000 7ff940800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff938143c40 0x7ff93815b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.434+0000 7ff940800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff9381444a0 0x7ff93806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.434+0000 7ff940800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93815efb0 con 0x7ff938144900 2024-05-22T21:25:10.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.434+0000 7ff940800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93815f290 con 0x7ff9381444a0 2024-05-22T21:25:10.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.434+0000 7ff940800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93815f570 con 0x7ff938143c40 2024-05-22T21:25:10.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.434+0000 7ff93d400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff9381444a0 0x7ff93806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.434+0000 7ff93d400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff9381444a0 0x7ff93806c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46758/0 (socket says 172.21.15.161:46758) 2024-05-22T21:25:10.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.434+0000 7ff93d400640 1 -- 172.21.15.161:0/3342704250 learned_addr learned my addr 172.21.15.161:0/3342704250 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:10.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.434+0000 7ff93de00640 1 --2- 172.21.15.161:0/3342704250 >> v2:172.21.15.179:3300/0 conn(0x7ff938143c40 0x7ff93815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.434+0000 7ff93d400640 1 -- 172.21.15.161:0/3342704250 >> v2:172.21.15.179:3300/0 conn(0x7ff938143c40 msgr2=0x7ff93815b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:10.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.434+0000 7ff93d400640 1 --2- 172.21.15.161:0/3342704250 >> v2:172.21.15.179:3300/0 conn(0x7ff938143c40 0x7ff93815b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.434+0000 7ff93d400640 1 -- 172.21.15.161:0/3342704250 >> v1:172.21.15.179:6789/0 conn(0x7ff938144900 legacy=0x7ff9380b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:10.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.438+0000 7ff93d400640 1 -- 172.21.15.161:0/3342704250 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93815f850 con 0x7ff9381444a0 2024-05-22T21:25:10.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.438+0000 7ff93d400640 1 --2- 172.21.15.161:0/3342704250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9381444a0 0x7ff93806c2a0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7ff9380724b0 tx=0x7ff92801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef059f88f87f28f server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:10.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.438+0000 7ff93ca00640 1 -- 172.21.15.161:0/3342704250 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9280a7070 con 0x7ff9381444a0 2024-05-22T21:25:10.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.438+0000 7ff93ca00640 1 -- 172.21.15.161:0/3342704250 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9280ab430 con 0x7ff9381444a0 2024-05-22T21:25:10.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.438+0000 7ff93ca00640 1 -- 172.21.15.161:0/3342704250 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9280ab7c0 con 0x7ff9381444a0 2024-05-22T21:25:10.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.438+0000 7ff940800640 1 -- 172.21.15.161:0/3342704250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9381444a0 msgr2=0x7ff93806c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:10.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.438+0000 7ff940800640 1 --2- 172.21.15.161:0/3342704250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9381444a0 0x7ff93806c2a0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7ff9380724b0 tx=0x7ff92801c520 comp rx=0 tx=0).stop 2024-05-22T21:25:10.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff940800640 1 -- 172.21.15.161:0/3342704250 shutdown_connections 2024-05-22T21:25:10.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff940800640 1 --2- 172.21.15.161:0/3342704250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9381444a0 0x7ff93806c2a0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff940800640 1 --2- 172.21.15.161:0/3342704250 >> v2:172.21.15.179:3300/0 conn(0x7ff938143c40 0x7ff93815b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff940800640 1 -- 172.21.15.161:0/3342704250 >> 172.21.15.161:0/3342704250 conn(0x7ff93806a2d0 msgr2=0x7ff93806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:10.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff940800640 1 -- 172.21.15.161:0/3342704250 shutdown_connections 2024-05-22T21:25:10.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff940800640 1 -- 172.21.15.161:0/3342704250 wait complete. 2024-05-22T21:25:10.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff940800640 1 Processor -- start 2024-05-22T21:25:10.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff940800640 1 -- start start 2024-05-22T21:25:10.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff940800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff938143c40 0x7ff938159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff940800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9381444a0 0x7ff9381367f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff940800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff938144900 0x7ff9380712d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff940800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff938142110 con 0x7ff938144900 2024-05-22T21:25:10.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff940800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff938159c70 con 0x7ff938143c40 2024-05-22T21:25:10.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff940800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff938141240 con 0x7ff9381444a0 2024-05-22T21:25:10.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff93e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff938144900 0x7ff9380712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff93e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff938144900 0x7ff9380712d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46760/0 (socket says 172.21.15.161:46760) 2024-05-22T21:25:10.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff93e800640 1 -- 172.21.15.161:0/1196451044 learned_addr learned my addr 172.21.15.161:0/1196451044 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:10.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff93de00640 1 --2- 172.21.15.161:0/1196451044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff938143c40 0x7ff938159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff93d400640 1 --2- 172.21.15.161:0/1196451044 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9381444a0 0x7ff9381367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff93e800640 1 -- 172.21.15.161:0/1196451044 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9381444a0 msgr2=0x7ff9381367f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:10.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff93e800640 1 --2- 172.21.15.161:0/1196451044 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9381444a0 0x7ff9381367f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff93e800640 1 -- 172.21.15.161:0/1196451044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff938143c40 msgr2=0x7ff938159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:10.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff93e800640 1 --2- 172.21.15.161:0/1196451044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff938143c40 0x7ff938159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff93e800640 1 -- 172.21.15.161:0/1196451044 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9280094d0 con 0x7ff938144900 2024-05-22T21:25:10.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.442+0000 7ff93e800640 1 --2- 172.21.15.161:0/1196451044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff938144900 0x7ff9380712d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7ff93400c730 tx=0x7ff93400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:10.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.446+0000 7ff932000640 1 -- 172.21.15.161:0/1196451044 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934003b80 con 0x7ff938144900 2024-05-22T21:25:10.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.446+0000 7ff932000640 1 -- 172.21.15.161:0/1196451044 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff934003ce0 con 0x7ff938144900 2024-05-22T21:25:10.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.446+0000 7ff940800640 1 -- 172.21.15.161:0/1196451044 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff938141490 con 0x7ff938144900 2024-05-22T21:25:10.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.446+0000 7ff932000640 1 -- 172.21.15.161:0/1196451044 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934005310 con 0x7ff938144900 2024-05-22T21:25:10.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.446+0000 7ff940800640 1 -- 172.21.15.161:0/1196451044 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff938141a50 con 0x7ff938144900 2024-05-22T21:25:10.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.446+0000 7ff940800640 1 -- 172.21.15.161:0/1196451044 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9381f6ba0 con 0x7ff938144900 2024-05-22T21:25:10.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.446+0000 7ff932000640 1 -- 172.21.15.161:0/1196451044 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff934005680 con 0x7ff938144900 2024-05-22T21:25:10.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.446+0000 7ff932000640 1 --2- 172.21.15.161:0/1196451044 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff91c066690 0x7ff91c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.446+0000 7ff932000640 1 -- 172.21.15.161:0/1196451044 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff934083ff0 con 0x7ff938144900 2024-05-22T21:25:10.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.446+0000 7ff93de00640 1 --2- 172.21.15.161:0/1196451044 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff91c066690 0x7ff91c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.446+0000 7ff93de00640 1 --2- 172.21.15.161:0/1196451044 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff91c066690 0x7ff91c068b50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009b60 tx=0x7ff92c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:10.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.450+0000 7ff932000640 1 -- 172.21.15.161:0/1196451044 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff934051400 con 0x7ff938144900 2024-05-22T21:25:10.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.606+0000 7ff940800640 1 -- 172.21.15.161:0/1196451044 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff938067220 con 0x7ff938144900 2024-05-22T21:25:10.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.606+0000 7ff932000640 1 -- 172.21.15.161:0/1196451044 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v193) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff934054a20 con 0x7ff938144900 2024-05-22T21:25:10.608 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:25:10.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.610+0000 7ff940800640 1 -- 172.21.15.161:0/1196451044 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff91c066690 msgr2=0x7ff91c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:10.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.610+0000 7ff940800640 1 --2- 172.21.15.161:0/1196451044 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff91c066690 0x7ff91c068b50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009b60 tx=0x7ff92c009190 comp rx=0 tx=0).stop 2024-05-22T21:25:10.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.610+0000 7ff940800640 1 -- 172.21.15.161:0/1196451044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff938144900 msgr2=0x7ff9380712d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:10.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.610+0000 7ff940800640 1 --2- 172.21.15.161:0/1196451044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff938144900 0x7ff9380712d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7ff93400c730 tx=0x7ff93400cbf0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.610+0000 7ff940800640 1 -- 172.21.15.161:0/1196451044 shutdown_connections 2024-05-22T21:25:10.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.610+0000 7ff940800640 1 --2- 172.21.15.161:0/1196451044 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff91c066690 0x7ff91c068b50 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.610+0000 7ff940800640 1 --2- 172.21.15.161:0/1196451044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff938144900 0x7ff9380712d0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.610+0000 7ff940800640 1 --2- 172.21.15.161:0/1196451044 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9381444a0 0x7ff9381367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.610+0000 7ff940800640 1 --2- 172.21.15.161:0/1196451044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff938143c40 0x7ff938159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.610+0000 7ff940800640 1 -- 172.21.15.161:0/1196451044 >> 172.21.15.161:0/1196451044 conn(0x7ff93806a2d0 msgr2=0x7ff9380adcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:10.611 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.610+0000 7ff940800640 1 -- 172.21.15.161:0/1196451044 shutdown_connections 2024-05-22T21:25:10.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.610+0000 7ff940800640 1 -- 172.21.15.161:0/1196451044 wait complete. 2024-05-22T21:25:10.629 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:25:10.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f826b200640 1 Processor -- start 2024-05-22T21:25:10.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f826b200640 1 -- start start 2024-05-22T21:25:10.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f826b200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f826415aeb0 0x7f82640d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f826b200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f826406cbe0 0x7f8264069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f826b200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f826406d040 0x7f826406a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f826b200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82640ce8f0 con 0x7f826415aeb0 2024-05-22T21:25:10.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f826b200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82640cc5d0 con 0x7f826406cbe0 2024-05-22T21:25:10.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f826b200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826415b290 con 0x7f826406d040 2024-05-22T21:25:10.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f8269e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f826415aeb0 0x7f82640d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f826a800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f826406d040 0x7f826406a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f8269e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f826415aeb0 0x7f82640d05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46768/0 (socket says 172.21.15.161:46768) 2024-05-22T21:25:10.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f8269e00640 1 -- 172.21.15.161:0/3473886317 learned_addr learned my addr 172.21.15.161:0/3473886317 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:10.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f8269400640 1 --2- 172.21.15.161:0/3473886317 >> v2:172.21.15.179:3300/0 conn(0x7f826406cbe0 0x7f8264069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f8269e00640 1 -- 172.21.15.161:0/3473886317 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f826406d040 msgr2=0x7f826406a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:10.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f8269e00640 1 --2- 172.21.15.161:0/3473886317 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f826406d040 0x7f826406a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f8269e00640 1 -- 172.21.15.161:0/3473886317 >> v2:172.21.15.179:3300/0 conn(0x7f826406cbe0 msgr2=0x7f8264069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:10.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f8269e00640 1 --2- 172.21.15.161:0/3473886317 >> v2:172.21.15.179:3300/0 conn(0x7f826406cbe0 0x7f8264069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.766+0000 7f8269e00640 1 -- 172.21.15.161:0/3473886317 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82640cc870 con 0x7f826415aeb0 2024-05-22T21:25:10.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.770+0000 7f8269e00640 1 --2- 172.21.15.161:0/3473886317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f826415aeb0 0x7f82640d05f0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f826406c090 tx=0x7f825801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539a7cbdb87ea547 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:10.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.770+0000 7f8268a00640 1 -- 172.21.15.161:0/3473886317 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82580ab620 con 0x7f826415aeb0 2024-05-22T21:25:10.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.770+0000 7f8268a00640 1 -- 172.21.15.161:0/3473886317 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82580ab780 con 0x7f826415aeb0 2024-05-22T21:25:10.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.770+0000 7f8268a00640 1 -- 172.21.15.161:0/3473886317 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82580abb10 con 0x7f826415aeb0 2024-05-22T21:25:10.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.770+0000 7f826b200640 1 -- 172.21.15.161:0/3473886317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f826415aeb0 msgr2=0x7f82640d05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:10.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.770+0000 7f826b200640 1 --2- 172.21.15.161:0/3473886317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f826415aeb0 0x7f82640d05f0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f826406c090 tx=0x7f825801c460 comp rx=0 tx=0).stop 2024-05-22T21:25:10.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.770+0000 7f826b200640 1 -- 172.21.15.161:0/3473886317 shutdown_connections 2024-05-22T21:25:10.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.770+0000 7f826b200640 1 --2- 172.21.15.161:0/3473886317 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f826406d040 0x7f826406a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.770+0000 7f826b200640 1 --2- 172.21.15.161:0/3473886317 >> v2:172.21.15.179:3300/0 conn(0x7f826406cbe0 0x7f8264069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.772 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.770+0000 7f826b200640 1 --2- 172.21.15.161:0/3473886317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f826415aeb0 0x7f82640d05f0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.770+0000 7f826b200640 1 -- 172.21.15.161:0/3473886317 >> 172.21.15.161:0/3473886317 conn(0x7f826406afe0 msgr2=0x7f826406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:10.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.770+0000 7f826b200640 1 -- 172.21.15.161:0/3473886317 shutdown_connections 2024-05-22T21:25:10.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.770+0000 7f826b200640 1 -- 172.21.15.161:0/3473886317 wait complete. 2024-05-22T21:25:10.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f826b200640 1 Processor -- start 2024-05-22T21:25:10.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f826b200640 1 -- start start 2024-05-22T21:25:10.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f826b200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f826406cbe0 0x7f826414cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f826b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f826406d040 0x7f8264158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f826b200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f826415aeb0 0x7f8264159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f826b200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826414d110 con 0x7f826406d040 2024-05-22T21:25:10.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f826b200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82640b3e40 con 0x7f826406cbe0 2024-05-22T21:25:10.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f826b200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82640b4120 con 0x7f826415aeb0 2024-05-22T21:25:10.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f8269400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f826406d040 0x7f8264158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f8269400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f826406d040 0x7f8264158f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46772/0 (socket says 172.21.15.161:46772) 2024-05-22T21:25:10.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f8269400640 1 -- 172.21.15.161:0/1856183441 learned_addr learned my addr 172.21.15.161:0/1856183441 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:10.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f8269e00640 1 --2- 172.21.15.161:0/1856183441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f826406cbe0 0x7f826414cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f826a800640 1 --2- 172.21.15.161:0/1856183441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f826415aeb0 0x7f8264159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f8269400640 1 -- 172.21.15.161:0/1856183441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f826415aeb0 msgr2=0x7f8264159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:10.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f8269400640 1 --2- 172.21.15.161:0/1856183441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f826415aeb0 0x7f8264159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f8269400640 1 -- 172.21.15.161:0/1856183441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f826406cbe0 msgr2=0x7f826414cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:10.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f8269400640 1 --2- 172.21.15.161:0/1856183441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f826406cbe0 0x7f826414cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f8269400640 1 -- 172.21.15.161:0/1856183441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82580094d0 con 0x7f826406d040 2024-05-22T21:25:10.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f8269400640 1 --2- 172.21.15.161:0/1856183441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f826406d040 0x7f8264158f00 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f825400e8b0 tx=0x7f825400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:10.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f825e000640 1 -- 172.21.15.161:0/1856183441 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825400c940 con 0x7f826406d040 2024-05-22T21:25:10.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f825e000640 1 -- 172.21.15.161:0/1856183441 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825400caa0 con 0x7f826406d040 2024-05-22T21:25:10.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f825e000640 1 -- 172.21.15.161:0/1856183441 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825400ce30 con 0x7f826406d040 2024-05-22T21:25:10.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f826b200640 1 -- 172.21.15.161:0/1856183441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82640b4400 con 0x7f826406d040 2024-05-22T21:25:10.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f826b200640 1 -- 172.21.15.161:0/1856183441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8264161250 con 0x7f826406d040 2024-05-22T21:25:10.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.774+0000 7f826b200640 1 -- 172.21.15.161:0/1856183441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8264067070 con 0x7f826406d040 2024-05-22T21:25:10.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.778+0000 7f825e000640 1 -- 172.21.15.161:0/1856183441 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8254013470 con 0x7f826406d040 2024-05-22T21:25:10.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.778+0000 7f825e000640 1 --2- 172.21.15.161:0/1856183441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f82480666c0 0x7f8248068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:10.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.778+0000 7f825e000640 1 -- 172.21.15.161:0/1856183441 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8254084350 con 0x7f826406d040 2024-05-22T21:25:10.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.778+0000 7f8269e00640 1 --2- 172.21.15.161:0/1856183441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f82480666c0 0x7f8248068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:10.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.778+0000 7f8269e00640 1 --2- 172.21.15.161:0/1856183441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f82480666c0 0x7f8248068b80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f82580af000 tx=0x7f82580b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:10.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.782+0000 7f825e000640 1 -- 172.21.15.161:0/1856183441 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f82540516d0 con 0x7f826406d040 2024-05-22T21:25:10.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.946+0000 7f826b200640 1 -- 172.21.15.161:0/1856183441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f82640af630 con 0x7f826406d040 2024-05-22T21:25:10.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.946+0000 7f825e000640 1 -- 172.21.15.161:0/1856183441 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v193) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8254054cf0 con 0x7f826406d040 2024-05-22T21:25:10.949 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:25:10.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.950+0000 7f826b200640 1 -- 172.21.15.161:0/1856183441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f82480666c0 msgr2=0x7f8248068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:10.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.950+0000 7f826b200640 1 --2- 172.21.15.161:0/1856183441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f82480666c0 0x7f8248068b80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f82580af000 tx=0x7f82580b6040 comp rx=0 tx=0).stop 2024-05-22T21:25:10.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.950+0000 7f826b200640 1 -- 172.21.15.161:0/1856183441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f826406d040 msgr2=0x7f8264158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:10.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.950+0000 7f826b200640 1 --2- 172.21.15.161:0/1856183441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f826406d040 0x7f8264158f00 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f825400e8b0 tx=0x7f825400ed70 comp rx=0 tx=0).stop 2024-05-22T21:25:10.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.950+0000 7f826b200640 1 -- 172.21.15.161:0/1856183441 shutdown_connections 2024-05-22T21:25:10.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.950+0000 7f826b200640 1 --2- 172.21.15.161:0/1856183441 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f82480666c0 0x7f8248068b80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.950+0000 7f826b200640 1 --2- 172.21.15.161:0/1856183441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f826415aeb0 0x7f8264159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.950+0000 7f826b200640 1 --2- 172.21.15.161:0/1856183441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f826406d040 0x7f8264158f00 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.950+0000 7f826b200640 1 --2- 172.21.15.161:0/1856183441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f826406cbe0 0x7f826414cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:10.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.950+0000 7f826b200640 1 -- 172.21.15.161:0/1856183441 >> 172.21.15.161:0/1856183441 conn(0x7f826406afe0 msgr2=0x7f826415daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:10.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.950+0000 7f826b200640 1 -- 172.21.15.161:0/1856183441 shutdown_connections 2024-05-22T21:25:10.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:10.950+0000 7f826b200640 1 -- 172.21.15.161:0/1856183441 wait complete. 2024-05-22T21:25:10.970 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-22T21:25:10.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:10.971 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-22T21:25:11.029 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:25:11.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:11.029 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-22T21:25:11.046 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:25:11.046 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:25:11.055 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:25:11.055 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:25:11.105 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:25:11.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.238+0000 7f016a000640 1 Processor -- start 2024-05-22T21:25:11.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.238+0000 7f016a000640 1 -- start start 2024-05-22T21:25:11.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.242+0000 7f016a000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f016415e130 0x7f016415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.242+0000 7f016a000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f016415d410 0x7f016415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.242+0000 7f016a000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0164071670 con 0x7f016415dd50 2024-05-22T21:25:11.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.242+0000 7f016a000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016415d870 con 0x7f016415e130 2024-05-22T21:25:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.242+0000 7f016a000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0164070a60 con 0x7f016415d410 2024-05-22T21:25:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.242+0000 7f0163400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f016415dd50 0x7f016406d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39498/0 (socket says 172.21.15.161:39498) 2024-05-22T21:25:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.242+0000 7f0163400640 1 -- 172.21.15.161:0/1954549255 learned_addr learned my addr 172.21.15.161:0/1954549255 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.242+0000 7f0162a00640 1 --2- 172.21.15.161:0/1954549255 >> v2:172.21.15.179:3300/0 conn(0x7f016415e130 0x7f016415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.242+0000 7f0163e00640 1 --2- 172.21.15.161:0/1954549255 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f016415d410 0x7f016415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.242+0000 7f0162000640 1 -- 172.21.15.161:0/1954549255 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2510755858 0 0) 0x7f0164071670 con 0x7f016415dd50 2024-05-22T21:25:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.242+0000 7f0162000640 1 -- 172.21.15.161:0/1954549255 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0148003530 con 0x7f016415dd50 2024-05-22T21:25:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.242+0000 7f0162a00640 1 -- 172.21.15.161:0/1954549255 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f016415d410 msgr2=0x7f016415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.242+0000 7f0162a00640 1 --2- 172.21.15.161:0/1954549255 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f016415d410 0x7f016415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.242+0000 7f0162a00640 1 -- 172.21.15.161:0/1954549255 >> v1:172.21.15.161:6789/0 conn(0x7f016415dd50 legacy=0x7f016406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.242+0000 7f0162a00640 1 -- 172.21.15.161:0/1954549255 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0164070d40 con 0x7f016415e130 2024-05-22T21:25:11.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.242+0000 7f0163e00640 1 --2- 172.21.15.161:0/1954549255 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f016415d410 0x7f016415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:25:11.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f0162a00640 1 --2- 172.21.15.161:0/1954549255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f016415e130 0x7f016415ced0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f014c0097c0 tx=0x7f014c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ab10b76d00b926e server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:11.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f0162000640 1 -- 172.21.15.161:0/1954549255 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c0ab670 con 0x7f016415e130 2024-05-22T21:25:11.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f0162000640 1 -- 172.21.15.161:0/1954549255 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014c0ab7d0 con 0x7f016415e130 2024-05-22T21:25:11.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f0162000640 1 -- 172.21.15.161:0/1954549255 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c0abb60 con 0x7f016415e130 2024-05-22T21:25:11.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f016a000640 1 -- 172.21.15.161:0/1954549255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f016415e130 msgr2=0x7f016415ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:11.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f016a000640 1 --2- 172.21.15.161:0/1954549255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f016415e130 0x7f016415ced0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f014c0097c0 tx=0x7f014c01c470 comp rx=0 tx=0).stop 2024-05-22T21:25:11.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f016a000640 1 -- 172.21.15.161:0/1954549255 shutdown_connections 2024-05-22T21:25:11.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f016a000640 1 --2- 172.21.15.161:0/1954549255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f016415e130 0x7f016415ced0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f016a000640 1 --2- 172.21.15.161:0/1954549255 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f016415d410 0x7f016415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f016a000640 1 -- 172.21.15.161:0/1954549255 >> 172.21.15.161:0/1954549255 conn(0x7f01640b0bc0 msgr2=0x7f01640b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:11.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f016a000640 1 -- 172.21.15.161:0/1954549255 shutdown_connections 2024-05-22T21:25:11.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f016a000640 1 -- 172.21.15.161:0/1954549255 wait complete. 2024-05-22T21:25:11.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f016a000640 1 Processor -- start 2024-05-22T21:25:11.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f016a000640 1 -- start start 2024-05-22T21:25:11.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f016a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f016415d410 0x7f01640af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f016a000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f016415dd50 0x7f016414a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f016a000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f016415e130 0x7f016414ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f016a000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01640cc710 con 0x7f016415d410 2024-05-22T21:25:11.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f016a000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01640cc9f0 con 0x7f016415dd50 2024-05-22T21:25:11.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.246+0000 7f016a000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01640cccd0 con 0x7f016415e130 2024-05-22T21:25:11.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f0163400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f016415d410 0x7f01640af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f0163400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f016415d410 0x7f01640af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46780/0 (socket says 172.21.15.161:46780) 2024-05-22T21:25:11.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f0163400640 1 -- 172.21.15.161:0/2268374092 learned_addr learned my addr 172.21.15.161:0/2268374092 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:11.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f0162a00640 1 --2- 172.21.15.161:0/2268374092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f016415dd50 0x7f016414a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f0163e00640 1 --2- 172.21.15.161:0/2268374092 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f016415e130 0x7f016414ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f0163400640 1 -- 172.21.15.161:0/2268374092 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f016415e130 msgr2=0x7f016414ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:11.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f0163400640 1 --2- 172.21.15.161:0/2268374092 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f016415e130 0x7f016414ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f0163400640 1 -- 172.21.15.161:0/2268374092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f016415dd50 msgr2=0x7f016414a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:11.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f0163400640 1 --2- 172.21.15.161:0/2268374092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f016415dd50 0x7f016414a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f0163400640 1 -- 172.21.15.161:0/2268374092 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c009a70 con 0x7f016415d410 2024-05-22T21:25:11.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f0163e00640 1 --2- 172.21.15.161:0/2268374092 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f016415e130 0x7f016414ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:11.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f0163400640 1 --2- 172.21.15.161:0/2268374092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f016415d410 0x7f01640af710 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f015400c4d0 tx=0x7f015400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:11.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f015b400640 1 -- 172.21.15.161:0/2268374092 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154005cf0 con 0x7f016415d410 2024-05-22T21:25:11.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f015b400640 1 -- 172.21.15.161:0/2268374092 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0154005e50 con 0x7f016415d410 2024-05-22T21:25:11.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f015b400640 1 -- 172.21.15.161:0/2268374092 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154007270 con 0x7f016415d410 2024-05-22T21:25:11.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f016a000640 1 -- 172.21.15.161:0/2268374092 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014c009420 con 0x7f016415d410 2024-05-22T21:25:11.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f016a000640 1 -- 172.21.15.161:0/2268374092 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0164153730 con 0x7f016415d410 2024-05-22T21:25:11.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f016a000640 1 -- 172.21.15.161:0/2268374092 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01640706b0 con 0x7f016415d410 2024-05-22T21:25:11.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.250+0000 7f015b400640 1 -- 172.21.15.161:0/2268374092 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f01540073d0 con 0x7f016415d410 2024-05-22T21:25:11.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.254+0000 7f015b400640 1 --2- 172.21.15.161:0/2268374092 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0140066690 0x7f0140068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.254+0000 7f015b400640 1 -- 172.21.15.161:0/2268374092 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0154083730 con 0x7f016415d410 2024-05-22T21:25:11.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.254+0000 7f0162a00640 1 --2- 172.21.15.161:0/2268374092 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0140066690 0x7f0140068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.254+0000 7f0162a00640 1 --2- 172.21.15.161:0/2268374092 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0140066690 0x7f0140068b50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f014c0af000 tx=0x7f014c0b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:11.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.258+0000 7f015b400640 1 -- 172.21.15.161:0/2268374092 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f0154050ab0 con 0x7f016415d410 2024-05-22T21:25:11.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.410+0000 7f016a000640 1 -- 172.21.15.161:0/2268374092 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01640cc850 con 0x7f016415d410 2024-05-22T21:25:11.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.414+0000 7f015b400640 1 -- 172.21.15.161:0/2268374092 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+16159 (secure 0 0 0) 0x7f01640cc850 con 0x7f016415d410 2024-05-22T21:25:11.414 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:11.414 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":193,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:25:05.312760+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:24:31.797440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-05-22T21:24:33.098240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:25:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.414+0000 7f016a000640 1 -- 172.21.15.161:0/2268374092 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0140066690 msgr2=0x7f0140068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.414+0000 7f016a000640 1 --2- 172.21.15.161:0/2268374092 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0140066690 0x7f0140068b50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f014c0af000 tx=0x7f014c0b6040 comp rx=0 tx=0).stop 2024-05-22T21:25:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.414+0000 7f016a000640 1 -- 172.21.15.161:0/2268374092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f016415d410 msgr2=0x7f01640af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.414+0000 7f016a000640 1 --2- 172.21.15.161:0/2268374092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f016415d410 0x7f01640af710 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f015400c4d0 tx=0x7f015400c990 comp rx=0 tx=0).stop 2024-05-22T21:25:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.414+0000 7f016a000640 1 -- 172.21.15.161:0/2268374092 shutdown_connections 2024-05-22T21:25:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.414+0000 7f016a000640 1 --2- 172.21.15.161:0/2268374092 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f0140066690 0x7f0140068b50 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.414+0000 7f016a000640 1 --2- 172.21.15.161:0/2268374092 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f016415e130 0x7f016414ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.414+0000 7f016a000640 1 --2- 172.21.15.161:0/2268374092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f016415dd50 0x7f016414a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.414+0000 7f016a000640 1 --2- 172.21.15.161:0/2268374092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f016415d410 0x7f01640af710 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.414+0000 7f016a000640 1 -- 172.21.15.161:0/2268374092 >> 172.21.15.161:0/2268374092 conn(0x7f01640b0bc0 msgr2=0x7f0164154490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.414+0000 7f016a000640 1 -- 172.21.15.161:0/2268374092 shutdown_connections 2024-05-22T21:25:11.416 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.414+0000 7f016a000640 1 -- 172.21.15.161:0/2268374092 wait complete. 2024-05-22T21:25:11.434 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:24:31.797440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '189', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:24:33.098240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '193', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:25:11.435 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:25:11.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.574+0000 7f69f1600640 1 Processor -- start 2024-05-22T21:25:11.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.574+0000 7f69f1600640 1 -- start start 2024-05-22T21:25:11.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.574+0000 7f69f1600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f69ec1436d0 0x7f69ec0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.574+0000 7f69f1600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f69ec06fa80 0x7f69ec0b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.574+0000 7f69f1600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69ec070b70 con 0x7f69ec0b1b10 2024-05-22T21:25:11.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.574+0000 7f69f1600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ec0cc720 con 0x7f69ec06fa80 2024-05-22T21:25:11.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.574+0000 7f69f1600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ec06ff60 con 0x7f69ec1436d0 2024-05-22T21:25:11.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.574+0000 7f69eb400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f69ec06fa80 0x7f69ec0b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.574+0000 7f69eb400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f69ec06fa80 0x7f69ec0b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46794/0 (socket says 172.21.15.161:46794) 2024-05-22T21:25:11.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.574+0000 7f69eb400640 1 -- 172.21.15.161:0/4096452000 learned_addr learned my addr 172.21.15.161:0/4096452000 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:11.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.574+0000 7f69ebe00640 1 --2- 172.21.15.161:0/4096452000 >> v2:172.21.15.179:3300/0 conn(0x7f69ec1436d0 0x7f69ec0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.574+0000 7f69eb400640 1 -- 172.21.15.161:0/4096452000 >> v2:172.21.15.179:3300/0 conn(0x7f69ec1436d0 msgr2=0x7f69ec0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:11.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.574+0000 7f69eb400640 1 --2- 172.21.15.161:0/4096452000 >> v2:172.21.15.179:3300/0 conn(0x7f69ec1436d0 0x7f69ec0cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.574+0000 7f69eb400640 1 -- 172.21.15.161:0/4096452000 >> v1:172.21.15.179:6789/0 conn(0x7f69ec0b1b10 legacy=0x7f69ec071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:11.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.574+0000 7f69eb400640 1 -- 172.21.15.161:0/4096452000 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ec070240 con 0x7f69ec06fa80 2024-05-22T21:25:11.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.578+0000 7f69eb400640 1 --2- 172.21.15.161:0/4096452000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69ec06fa80 0x7f69ec0b2cc0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0724a0 tx=0x7f69d801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec803f1dfd9c105 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:11.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.578+0000 7f69eaa00640 1 -- 172.21.15.161:0/4096452000 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d80ab600 con 0x7f69ec06fa80 2024-05-22T21:25:11.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.578+0000 7f69eaa00640 1 -- 172.21.15.161:0/4096452000 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d80a7050 con 0x7f69ec06fa80 2024-05-22T21:25:11.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.578+0000 7f69eaa00640 1 -- 172.21.15.161:0/4096452000 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d80ab980 con 0x7f69ec06fa80 2024-05-22T21:25:11.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.578+0000 7f69f1600640 1 -- 172.21.15.161:0/4096452000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69ec06fa80 msgr2=0x7f69ec0b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:11.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.578+0000 7f69f1600640 1 --2- 172.21.15.161:0/4096452000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69ec06fa80 0x7f69ec0b2cc0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0724a0 tx=0x7f69d801c460 comp rx=0 tx=0).stop 2024-05-22T21:25:11.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.578+0000 7f69f1600640 1 -- 172.21.15.161:0/4096452000 shutdown_connections 2024-05-22T21:25:11.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.578+0000 7f69f1600640 1 --2- 172.21.15.161:0/4096452000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69ec06fa80 0x7f69ec0b2cc0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.578+0000 7f69f1600640 1 --2- 172.21.15.161:0/4096452000 >> v2:172.21.15.179:3300/0 conn(0x7f69ec1436d0 0x7f69ec0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.578+0000 7f69f1600640 1 -- 172.21.15.161:0/4096452000 >> 172.21.15.161:0/4096452000 conn(0x7f69ec0cec00 msgr2=0x7f69ec0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:11.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.578+0000 7f69f1600640 1 -- 172.21.15.161:0/4096452000 shutdown_connections 2024-05-22T21:25:11.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.578+0000 7f69f1600640 1 -- 172.21.15.161:0/4096452000 wait complete. 2024-05-22T21:25:11.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.578+0000 7f69f1600640 1 Processor -- start 2024-05-22T21:25:11.581 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69f1600640 1 -- start start 2024-05-22T21:25:11.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69f1600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f69ec06fa80 0x7f69ec140660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69f1600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69ec0b1b10 0x7f69ec140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69f1600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69ec1436d0 0x7f69ec0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69f1600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec1402d0 con 0x7f69ec0b1b10 2024-05-22T21:25:11.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69f1600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec13fff0 con 0x7f69ec1436d0 2024-05-22T21:25:11.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69f1600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec15af70 con 0x7f69ec06fa80 2024-05-22T21:25:11.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69eb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69ec0b1b10 0x7f69ec140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69eb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69ec0b1b10 0x7f69ec140ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46808/0 (socket says 172.21.15.161:46808) 2024-05-22T21:25:11.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69eb400640 1 -- 172.21.15.161:0/3507385653 learned_addr learned my addr 172.21.15.161:0/3507385653 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:11.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69f0c00640 1 --2- 172.21.15.161:0/3507385653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69ec1436d0 0x7f69ec0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69ebe00640 1 --2- 172.21.15.161:0/3507385653 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f69ec06fa80 0x7f69ec140660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69eb400640 1 -- 172.21.15.161:0/3507385653 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f69ec06fa80 msgr2=0x7f69ec140660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:11.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69eb400640 1 --2- 172.21.15.161:0/3507385653 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f69ec06fa80 0x7f69ec140660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69eb400640 1 -- 172.21.15.161:0/3507385653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69ec1436d0 msgr2=0x7f69ec0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:11.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69eb400640 1 --2- 172.21.15.161:0/3507385653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69ec1436d0 0x7f69ec0ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69eb400640 1 -- 172.21.15.161:0/3507385653 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d80094d0 con 0x7f69ec0b1b10 2024-05-22T21:25:11.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69eb400640 1 --2- 172.21.15.161:0/3507385653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69ec0b1b10 0x7f69ec140ba0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f69d80ae040 tx=0x7f69d8019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:11.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69e3e00640 1 -- 172.21.15.161:0/3507385653 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d80b4440 con 0x7f69ec0b1b10 2024-05-22T21:25:11.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69e3e00640 1 -- 172.21.15.161:0/3507385653 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d80b45a0 con 0x7f69ec0b1b10 2024-05-22T21:25:11.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69e3e00640 1 -- 172.21.15.161:0/3507385653 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d80b4930 con 0x7f69ec0b1b10 2024-05-22T21:25:11.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69f1600640 1 -- 172.21.15.161:0/3507385653 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ec15b250 con 0x7f69ec0b1b10 2024-05-22T21:25:11.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69f1600640 1 -- 172.21.15.161:0/3507385653 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec13fb60 con 0x7f69ec0b1b10 2024-05-22T21:25:11.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.582+0000 7f69f1600640 1 -- 172.21.15.161:0/3507385653 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ec15b0b0 con 0x7f69ec0b1b10 2024-05-22T21:25:11.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.586+0000 7f69e3e00640 1 -- 172.21.15.161:0/3507385653 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f69d80b2070 con 0x7f69ec0b1b10 2024-05-22T21:25:11.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.586+0000 7f69e3e00640 1 --2- 172.21.15.161:0/3507385653 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69cc066690 0x7f69cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.586+0000 7f69e3e00640 1 -- 172.21.15.161:0/3507385653 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69d811dff0 con 0x7f69ec0b1b10 2024-05-22T21:25:11.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.586+0000 7f69ebe00640 1 --2- 172.21.15.161:0/3507385653 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69cc066690 0x7f69cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.587 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.586+0000 7f69ebe00640 1 --2- 172.21.15.161:0/3507385653 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69cc066690 0x7f69cc068b50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009ac0 tx=0x7f69dc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:11.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.590+0000 7f69e3e00640 1 -- 172.21.15.161:0/3507385653 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f69ec15b0b0 con 0x7f69ec0b1b10 2024-05-22T21:25:11.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.742+0000 7f69f1600640 1 -- 172.21.15.161:0/3507385653 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f69ec15b0b0 con 0x7f69ec0b1b10 2024-05-22T21:25:11.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.746+0000 7f69e3e00640 1 -- 172.21.15.161:0/3507385653 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v193) v1 ==== 93+0+10 (secure 0 0 0) 0x7f69ec15b0b0 con 0x7f69ec0b1b10 2024-05-22T21:25:11.745 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:25:11.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.746+0000 7f69f1600640 1 -- 172.21.15.161:0/3507385653 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69cc066690 msgr2=0x7f69cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:11.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.746+0000 7f69f1600640 1 --2- 172.21.15.161:0/3507385653 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69cc066690 0x7f69cc068b50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009ac0 tx=0x7f69dc009190 comp rx=0 tx=0).stop 2024-05-22T21:25:11.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.746+0000 7f69f1600640 1 -- 172.21.15.161:0/3507385653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69ec0b1b10 msgr2=0x7f69ec140ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:11.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.746+0000 7f69f1600640 1 --2- 172.21.15.161:0/3507385653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69ec0b1b10 0x7f69ec140ba0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f69d80ae040 tx=0x7f69d8019200 comp rx=0 tx=0).stop 2024-05-22T21:25:11.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.746+0000 7f69f1600640 1 -- 172.21.15.161:0/3507385653 shutdown_connections 2024-05-22T21:25:11.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.746+0000 7f69f1600640 1 --2- 172.21.15.161:0/3507385653 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f69cc066690 0x7f69cc068b50 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.746+0000 7f69f1600640 1 --2- 172.21.15.161:0/3507385653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69ec1436d0 0x7f69ec0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.746+0000 7f69f1600640 1 --2- 172.21.15.161:0/3507385653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69ec0b1b10 0x7f69ec140ba0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.746+0000 7f69f1600640 1 --2- 172.21.15.161:0/3507385653 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f69ec06fa80 0x7f69ec140660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.746+0000 7f69f1600640 1 -- 172.21.15.161:0/3507385653 >> 172.21.15.161:0/3507385653 conn(0x7f69ec0cec00 msgr2=0x7f69ec0cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:11.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.746+0000 7f69f1600640 1 -- 172.21.15.161:0/3507385653 shutdown_connections 2024-05-22T21:25:11.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.746+0000 7f69f1600640 1 -- 172.21.15.161:0/3507385653 wait complete. 2024-05-22T21:25:11.766 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:25:11.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.902+0000 7f03b6400640 1 Processor -- start 2024-05-22T21:25:11.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.902+0000 7f03b6400640 1 -- start start 2024-05-22T21:25:11.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.906+0000 7f03b6400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f03b013cfc0 0x7f03b01402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.906+0000 7f03b6400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03b013fe50 0x7f03b006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.906+0000 7f03b6400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b013d3a0 con 0x7f03b015af60 2024-05-22T21:25:11.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.906+0000 7f03b6400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b00cd920 con 0x7f03b013cfc0 2024-05-22T21:25:11.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.906+0000 7f03b6400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b00cdc00 con 0x7f03b013fe50 2024-05-22T21:25:11.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.906+0000 7f03af400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f03b013cfc0 0x7f03b01402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.906+0000 7f03af400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f03b013cfc0 0x7f03b01402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46818/0 (socket says 172.21.15.161:46818) 2024-05-22T21:25:11.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.906+0000 7f03af400640 1 -- 172.21.15.161:0/2665042178 learned_addr learned my addr 172.21.15.161:0/2665042178 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:11.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.906+0000 7f03afe00640 1 --2- 172.21.15.161:0/2665042178 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03b013fe50 0x7f03b006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.906+0000 7f03af400640 1 -- 172.21.15.161:0/2665042178 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03b013fe50 msgr2=0x7f03b006fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:11.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.906+0000 7f03af400640 1 --2- 172.21.15.161:0/2665042178 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03b013fe50 0x7f03b006fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.906+0000 7f03af400640 1 -- 172.21.15.161:0/2665042178 >> v1:172.21.15.179:6789/0 conn(0x7f03b015af60 legacy=0x7f03b013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:11.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.906+0000 7f03af400640 1 -- 172.21.15.161:0/2665042178 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b00cdee0 con 0x7f03b013cfc0 2024-05-22T21:25:11.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.910+0000 7f03af400640 1 --2- 172.21.15.161:0/2665042178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b013cfc0 0x7f03b01402e0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009870 tx=0x7f03a001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700064311363c44b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:11.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.910+0000 7f03ae000640 1 -- 172.21.15.161:0/2665042178 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a00ab600 con 0x7f03b013cfc0 2024-05-22T21:25:11.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.910+0000 7f03ae000640 1 -- 172.21.15.161:0/2665042178 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a00a7050 con 0x7f03b013cfc0 2024-05-22T21:25:11.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.910+0000 7f03ae000640 1 -- 172.21.15.161:0/2665042178 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a00ab980 con 0x7f03b013cfc0 2024-05-22T21:25:11.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.910+0000 7f03b6400640 1 -- 172.21.15.161:0/2665042178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b013cfc0 msgr2=0x7f03b01402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:11.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.910+0000 7f03b6400640 1 --2- 172.21.15.161:0/2665042178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b013cfc0 0x7f03b01402e0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009870 tx=0x7f03a001c4a0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.910+0000 7f03b6400640 1 -- 172.21.15.161:0/2665042178 shutdown_connections 2024-05-22T21:25:11.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.910+0000 7f03b6400640 1 --2- 172.21.15.161:0/2665042178 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03b013fe50 0x7f03b006fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.910+0000 7f03b6400640 1 --2- 172.21.15.161:0/2665042178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b013cfc0 0x7f03b01402e0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.910+0000 7f03b6400640 1 -- 172.21.15.161:0/2665042178 >> 172.21.15.161:0/2665042178 conn(0x7f03b006a2d0 msgr2=0x7f03b006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:11.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.910+0000 7f03b6400640 1 -- 172.21.15.161:0/2665042178 shutdown_connections 2024-05-22T21:25:11.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.910+0000 7f03b6400640 1 -- 172.21.15.161:0/2665042178 wait complete. 2024-05-22T21:25:11.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.910+0000 7f03b6400640 1 Processor -- start 2024-05-22T21:25:11.912 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.910+0000 7f03b6400640 1 -- start start 2024-05-22T21:25:11.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03b6400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03b013cfc0 0x7f03b0140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03b6400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b013fe50 0x7f03b01562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03b6400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03b015af60 0x7f03b013d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.913 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03b6400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b0140770 con 0x7f03b013fe50 2024-05-22T21:25:11.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03b6400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b01415b0 con 0x7f03b013cfc0 2024-05-22T21:25:11.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03b6400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b01409a0 con 0x7f03b015af60 2024-05-22T21:25:11.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03aea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b013fe50 0x7f03b01562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03af400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03b013cfc0 0x7f03b0140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03afe00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03b015af60 0x7f03b013d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03aea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b013fe50 0x7f03b01562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46820/0 (socket says 172.21.15.161:46820) 2024-05-22T21:25:11.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03af400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03b013cfc0 0x7f03b0140230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:44310/0 (socket says 172.21.15.161:44310) 2024-05-22T21:25:11.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03aea00640 1 -- 172.21.15.161:0/1178156366 learned_addr learned my addr 172.21.15.161:0/1178156366 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:11.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03afe00640 1 -- 172.21.15.161:0/1178156366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03b013cfc0 msgr2=0x7f03b0140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:11.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03afe00640 1 --2- 172.21.15.161:0/1178156366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03b013cfc0 0x7f03b0140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03afe00640 1 -- 172.21.15.161:0/1178156366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b013fe50 msgr2=0x7f03b01562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:11.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03afe00640 1 --2- 172.21.15.161:0/1178156366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b013fe50 0x7f03b01562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:11.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03afe00640 1 -- 172.21.15.161:0/1178156366 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a00094d0 con 0x7f03b015af60 2024-05-22T21:25:11.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03aea00640 1 --2- 172.21.15.161:0/1178156366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b013fe50 0x7f03b01562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:25:11.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03afe00640 1 --2- 172.21.15.161:0/1178156366 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03b015af60 0x7f03b013d590 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f03a8007d90 tx=0x7f03a800e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:11.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03a7400640 1 -- 172.21.15.161:0/1178156366 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8018070 con 0x7f03b015af60 2024-05-22T21:25:11.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03a7400640 1 -- 172.21.15.161:0/1178156366 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a800edd0 con 0x7f03b015af60 2024-05-22T21:25:11.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03b6400640 1 -- 172.21.15.161:0/1178156366 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b0140c80 con 0x7f03b015af60 2024-05-22T21:25:11.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03a7400640 1 -- 172.21.15.161:0/1178156366 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8003bc0 con 0x7f03b015af60 2024-05-22T21:25:11.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03b6400640 1 -- 172.21.15.161:0/1178156366 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b0141240 con 0x7f03b015af60 2024-05-22T21:25:11.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03b6400640 1 -- 172.21.15.161:0/1178156366 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b00af760 con 0x7f03b015af60 2024-05-22T21:25:11.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.914+0000 7f03a7400640 1 -- 172.21.15.161:0/1178156366 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f03a8014420 con 0x7f03b015af60 2024-05-22T21:25:11.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.918+0000 7f03a7400640 1 --2- 172.21.15.161:0/1178156366 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f038c066690 0x7f038c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:11.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.918+0000 7f03a7400640 1 -- 172.21.15.161:0/1178156366 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03a80842b0 con 0x7f03b015af60 2024-05-22T21:25:11.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.918+0000 7f03af400640 1 --2- 172.21.15.161:0/1178156366 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f038c066690 0x7f038c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:11.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.918+0000 7f03af400640 1 --2- 172.21.15.161:0/1178156366 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f038c066690 0x7f038c068b50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f03a00ad000 tx=0x7f03a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:11.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:11.922+0000 7f03a7400640 1 -- 172.21.15.161:0/1178156366 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f03b00af760 con 0x7f03b015af60 2024-05-22T21:25:12.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.074+0000 7f03b6400640 1 -- 172.21.15.161:0/1178156366 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03b00af760 con 0x7f03b015af60 2024-05-22T21:25:12.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.074+0000 7f03a7400640 1 -- 172.21.15.161:0/1178156366 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v193) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03b00af760 con 0x7f03b015af60 2024-05-22T21:25:12.076 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:25:12.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.078+0000 7f03b6400640 1 -- 172.21.15.161:0/1178156366 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f038c066690 msgr2=0x7f038c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:12.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.078+0000 7f03b6400640 1 --2- 172.21.15.161:0/1178156366 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f038c066690 0x7f038c068b50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f03a00ad000 tx=0x7f03a0008040 comp rx=0 tx=0).stop 2024-05-22T21:25:12.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.078+0000 7f03b6400640 1 -- 172.21.15.161:0/1178156366 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03b015af60 msgr2=0x7f03b013d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:12.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.078+0000 7f03b6400640 1 --2- 172.21.15.161:0/1178156366 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03b015af60 0x7f03b013d590 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f03a8007d90 tx=0x7f03a800e4b0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.078+0000 7f03b6400640 1 -- 172.21.15.161:0/1178156366 shutdown_connections 2024-05-22T21:25:12.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.078+0000 7f03b6400640 1 --2- 172.21.15.161:0/1178156366 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f038c066690 0x7f038c068b50 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.078+0000 7f03b6400640 1 --2- 172.21.15.161:0/1178156366 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03b015af60 0x7f03b013d590 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.078+0000 7f03b6400640 1 --2- 172.21.15.161:0/1178156366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03b013fe50 0x7f03b01562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.078+0000 7f03b6400640 1 --2- 172.21.15.161:0/1178156366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03b013cfc0 0x7f03b0140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.078+0000 7f03b6400640 1 -- 172.21.15.161:0/1178156366 >> 172.21.15.161:0/1178156366 conn(0x7f03b006a2d0 msgr2=0x7f03b006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:12.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.078+0000 7f03b6400640 1 -- 172.21.15.161:0/1178156366 shutdown_connections 2024-05-22T21:25:12.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.078+0000 7f03b6400640 1 -- 172.21.15.161:0/1178156366 wait complete. 2024-05-22T21:25:12.097 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:25:12.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.234+0000 7f7058e00640 1 Processor -- start 2024-05-22T21:25:12.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.234+0000 7f7058e00640 1 -- start start 2024-05-22T21:25:12.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.234+0000 7f7058e00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f70541402e0 0x7f705413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:12.236 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.234+0000 7f7058e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f705413fc80 0x7f705406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:12.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.234+0000 7f7058e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70541400b0 con 0x7f705413cfc0 2024-05-22T21:25:12.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.234+0000 7f7058e00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705413d3a0 con 0x7f70541402e0 2024-05-22T21:25:12.237 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.234+0000 7f7058e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70540cd920 con 0x7f705413fc80 2024-05-22T21:25:12.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.238+0000 7f7052800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f705413fc80 0x7f705406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:12.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.238+0000 7f7051400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f70541402e0 0x7f705413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:12.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.238+0000 7f7052800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f705413fc80 0x7f705406fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:47666/0 (socket says 172.21.15.161:47666) 2024-05-22T21:25:12.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.238+0000 7f7052800640 1 -- 172.21.15.161:0/1553520343 learned_addr learned my addr 172.21.15.161:0/1553520343 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:12.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.238+0000 7f7051e00640 1 --1- 172.21.15.161:0/1553520343 >> v1:172.21.15.179:6789/0 conn(0x7f705413cfc0 0x7f705415af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:50646/0 (socket says 172.21.15.161:50646) 2024-05-22T21:25:12.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.238+0000 7f7052800640 1 -- 172.21.15.161:0/1553520343 >> v2:172.21.15.179:3300/0 conn(0x7f70541402e0 msgr2=0x7f705413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:12.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.238+0000 7f7052800640 1 --2- 172.21.15.161:0/1553520343 >> v2:172.21.15.179:3300/0 conn(0x7f70541402e0 0x7f705413f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.238+0000 7f7052800640 1 -- 172.21.15.161:0/1553520343 >> v1:172.21.15.179:6789/0 conn(0x7f705413cfc0 legacy=0x7f705415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:12.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.238+0000 7f7050a00640 1 -- 172.21.15.161:0/1553520343 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207009842 0 0) 0x7f70541400b0 con 0x7f705413cfc0 2024-05-22T21:25:12.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.238+0000 7f7052800640 1 -- 172.21.15.161:0/1553520343 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70540cdc00 con 0x7f705413fc80 2024-05-22T21:25:12.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.238+0000 7f7052800640 1 --2- 172.21.15.161:0/1553520343 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f705413fc80 0x7f705406fce0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f704c00d150 tx=0x7f704c01eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63b8727435e4e51 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:12.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.238+0000 7f7050a00640 1 -- 172.21.15.161:0/1553520343 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c00b2d0 con 0x7f705413fc80 2024-05-22T21:25:12.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.238+0000 7f7050a00640 1 -- 172.21.15.161:0/1553520343 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c00b430 con 0x7f705413fc80 2024-05-22T21:25:12.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.238+0000 7f7050a00640 1 -- 172.21.15.161:0/1553520343 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c009820 con 0x7f705413fc80 2024-05-22T21:25:12.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7058e00640 1 -- 172.21.15.161:0/1553520343 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f705413fc80 msgr2=0x7f705406fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:12.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7058e00640 1 --2- 172.21.15.161:0/1553520343 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f705413fc80 0x7f705406fce0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f704c00d150 tx=0x7f704c01eb70 comp rx=0 tx=0).stop 2024-05-22T21:25:12.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7058e00640 1 -- 172.21.15.161:0/1553520343 shutdown_connections 2024-05-22T21:25:12.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7058e00640 1 --2- 172.21.15.161:0/1553520343 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f705413fc80 0x7f705406fce0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7058e00640 1 --2- 172.21.15.161:0/1553520343 >> v2:172.21.15.179:3300/0 conn(0x7f70541402e0 0x7f705413f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7058e00640 1 -- 172.21.15.161:0/1553520343 >> 172.21.15.161:0/1553520343 conn(0x7f705406a2d0 msgr2=0x7f705406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:12.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7058e00640 1 -- 172.21.15.161:0/1553520343 shutdown_connections 2024-05-22T21:25:12.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7058e00640 1 -- 172.21.15.161:0/1553520343 wait complete. 2024-05-22T21:25:12.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7058e00640 1 Processor -- start 2024-05-22T21:25:12.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7058e00640 1 -- start start 2024-05-22T21:25:12.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7058e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f705413cfc0 0x7f7054157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:12.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7058e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f705413fc80 0x7f7054156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:12.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7058e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70541402e0 0x7f7054156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:12.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7058e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7054142130 con 0x7f705413cfc0 2024-05-22T21:25:12.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7058e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7054152f10 con 0x7f705413fc80 2024-05-22T21:25:12.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7058e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7054139260 con 0x7f70541402e0 2024-05-22T21:25:12.245 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7051e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f705413cfc0 0x7f7054157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:12.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7051e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f705413cfc0 0x7f7054157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46826/0 (socket says 172.21.15.161:46826) 2024-05-22T21:25:12.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7051e00640 1 -- 172.21.15.161:0/1140371630 learned_addr learned my addr 172.21.15.161:0/1140371630 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:12.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7052800640 1 --2- 172.21.15.161:0/1140371630 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70541402e0 0x7f7054156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:12.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.242+0000 7f7051400640 1 --2- 172.21.15.161:0/1140371630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f705413fc80 0x7f7054156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:12.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.246+0000 7f7051e00640 1 -- 172.21.15.161:0/1140371630 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70541402e0 msgr2=0x7f7054156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:12.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.246+0000 7f7051e00640 1 --2- 172.21.15.161:0/1140371630 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70541402e0 0x7f7054156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.246+0000 7f7051e00640 1 -- 172.21.15.161:0/1140371630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f705413fc80 msgr2=0x7f7054156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:12.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.246+0000 7f7051e00640 1 --2- 172.21.15.161:0/1140371630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f705413fc80 0x7f7054156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.246+0000 7f7051e00640 1 -- 172.21.15.161:0/1140371630 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c00bd40 con 0x7f705413cfc0 2024-05-22T21:25:12.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.246+0000 7f7051e00640 1 --2- 172.21.15.161:0/1140371630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f705413cfc0 0x7f7054157610 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f703c0011f0 tx=0x7f703c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:12.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.246+0000 7f704a000640 1 -- 172.21.15.161:0/1140371630 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c0074e0 con 0x7f705413cfc0 2024-05-22T21:25:12.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.246+0000 7f704a000640 1 -- 172.21.15.161:0/1140371630 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c007640 con 0x7f705413cfc0 2024-05-22T21:25:12.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.246+0000 7f704a000640 1 -- 172.21.15.161:0/1140371630 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c004a10 con 0x7f705413cfc0 2024-05-22T21:25:12.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.246+0000 7f7058e00640 1 -- 172.21.15.161:0/1140371630 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7054139540 con 0x7f705413cfc0 2024-05-22T21:25:12.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.246+0000 7f7058e00640 1 -- 172.21.15.161:0/1140371630 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7054139b00 con 0x7f705413cfc0 2024-05-22T21:25:12.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.246+0000 7f7058e00640 1 -- 172.21.15.161:0/1140371630 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705406c610 con 0x7f705413cfc0 2024-05-22T21:25:12.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.246+0000 7f704a000640 1 -- 172.21.15.161:0/1140371630 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f703c004b70 con 0x7f705413cfc0 2024-05-22T21:25:12.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.246+0000 7f704a000640 1 --2- 172.21.15.161:0/1140371630 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f70300666c0 0x7f7030068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:12.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.246+0000 7f704a000640 1 -- 172.21.15.161:0/1140371630 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f703c0832d0 con 0x7f705413cfc0 2024-05-22T21:25:12.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.250+0000 7f7051400640 1 --2- 172.21.15.161:0/1140371630 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f70300666c0 0x7f7030068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:12.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.250+0000 7f7051400640 1 --2- 172.21.15.161:0/1140371630 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f70300666c0 0x7f7030068b80 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f705415d970 tx=0x7f7044009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:12.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.250+0000 7f704a000640 1 -- 172.21.15.161:0/1140371630 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f703c050650 con 0x7f705413cfc0 2024-05-22T21:25:12.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.406+0000 7f7058e00640 1 -- 172.21.15.161:0/1140371630 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f705406ca60 con 0x7f705413cfc0 2024-05-22T21:25:12.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.406+0000 7f704a000640 1 -- 172.21.15.161:0/1140371630 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v193) v1 ==== 100+0+11 (secure 0 0 0) 0x7f703c053c70 con 0x7f705413cfc0 2024-05-22T21:25:12.408 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:25:12.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.410+0000 7f7058e00640 1 -- 172.21.15.161:0/1140371630 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f70300666c0 msgr2=0x7f7030068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:12.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.410+0000 7f7058e00640 1 --2- 172.21.15.161:0/1140371630 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f70300666c0 0x7f7030068b80 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f705415d970 tx=0x7f7044009190 comp rx=0 tx=0).stop 2024-05-22T21:25:12.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.410+0000 7f7058e00640 1 -- 172.21.15.161:0/1140371630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f705413cfc0 msgr2=0x7f7054157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:12.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.410+0000 7f7058e00640 1 --2- 172.21.15.161:0/1140371630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f705413cfc0 0x7f7054157610 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f703c0011f0 tx=0x7f703c00c9b0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.410+0000 7f7058e00640 1 -- 172.21.15.161:0/1140371630 shutdown_connections 2024-05-22T21:25:12.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.410+0000 7f7058e00640 1 --2- 172.21.15.161:0/1140371630 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f70300666c0 0x7f7030068b80 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.410+0000 7f7058e00640 1 --2- 172.21.15.161:0/1140371630 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70541402e0 0x7f7054156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.410+0000 7f7058e00640 1 --2- 172.21.15.161:0/1140371630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f705413fc80 0x7f7054156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.410+0000 7f7058e00640 1 --2- 172.21.15.161:0/1140371630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f705413cfc0 0x7f7054157610 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.410+0000 7f7058e00640 1 -- 172.21.15.161:0/1140371630 >> 172.21.15.161:0/1140371630 conn(0x7f705406a2d0 msgr2=0x7f70540af960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:12.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.410+0000 7f7058e00640 1 -- 172.21.15.161:0/1140371630 shutdown_connections 2024-05-22T21:25:12.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.410+0000 7f7058e00640 1 -- 172.21.15.161:0/1140371630 wait complete. 2024-05-22T21:25:12.429 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-22T21:25:12.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.562+0000 7f28d5e00640 1 Processor -- start 2024-05-22T21:25:12.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.562+0000 7f28d5e00640 1 -- start start 2024-05-22T21:25:12.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.566+0000 7f28d5e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f28d015b600 0x7f28d006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:12.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.566+0000 7f28d5e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d006cc40 con 0x7f28d006c7e0 2024-05-22T21:25:12.566 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.566+0000 7f28d5e00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d0144bb0 con 0x7f28d0143c40 2024-05-22T21:25:12.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.566+0000 7f28d5e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d015ba60 con 0x7f28d015b600 2024-05-22T21:25:12.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.566+0000 7f28ce000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f28d015b600 0x7f28d006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:12.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.566+0000 7f28ce000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f28d015b600 0x7f28d006c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46832/0 (socket says 172.21.15.161:46832) 2024-05-22T21:25:12.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.566+0000 7f28ce000640 1 -- 172.21.15.161:0/448281082 learned_addr learned my addr 172.21.15.161:0/448281082 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:12.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.566+0000 7f28cf400640 1 --1- 172.21.15.161:0/448281082 >> v1:172.21.15.161:6789/0 conn(0x7f28d006c7e0 0x7f28d00b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39504/0 (socket says 172.21.15.161:39504) 2024-05-22T21:25:12.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.566+0000 7f28ce000640 1 -- 172.21.15.161:0/448281082 >> v1:172.21.15.179:6789/0 conn(0x7f28d0143c40 legacy=0x7f28d01444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:12.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.566+0000 7f28ce000640 1 -- 172.21.15.161:0/448281082 >> v1:172.21.15.161:6789/0 conn(0x7f28d006c7e0 legacy=0x7f28d00b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:12.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.566+0000 7f28cd600640 1 -- 172.21.15.161:0/448281082 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911691474 0 0) 0x7f28d006cc40 con 0x7f28d006c7e0 2024-05-22T21:25:12.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.566+0000 7f28ce000640 1 -- 172.21.15.161:0/448281082 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d015efb0 con 0x7f28d015b600 2024-05-22T21:25:12.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.570+0000 7f28ce000640 1 --2- 172.21.15.161:0/448281082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d015b600 0x7f28d006c2a0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009870 tx=0x7f28bc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f5523041faa6d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:12.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.570+0000 7f28cd600640 1 -- 172.21.15.161:0/448281082 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0a9620 con 0x7f28d015b600 2024-05-22T21:25:12.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.570+0000 7f28cd600640 1 -- 172.21.15.161:0/448281082 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc0a9780 con 0x7f28d015b600 2024-05-22T21:25:12.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.570+0000 7f28cd600640 1 -- 172.21.15.161:0/448281082 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0a9b10 con 0x7f28d015b600 2024-05-22T21:25:12.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.570+0000 7f28d5e00640 1 -- 172.21.15.161:0/448281082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d015b600 msgr2=0x7f28d006c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:12.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.570+0000 7f28d5e00640 1 --2- 172.21.15.161:0/448281082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d015b600 0x7f28d006c2a0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009870 tx=0x7f28bc01c560 comp rx=0 tx=0).stop 2024-05-22T21:25:12.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.570+0000 7f28d5e00640 1 -- 172.21.15.161:0/448281082 shutdown_connections 2024-05-22T21:25:12.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.570+0000 7f28d5e00640 1 --2- 172.21.15.161:0/448281082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d015b600 0x7f28d006c2a0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.570+0000 7f28d5e00640 1 -- 172.21.15.161:0/448281082 >> 172.21.15.161:0/448281082 conn(0x7f28d006a2d0 msgr2=0x7f28d006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:12.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.570+0000 7f28d5e00640 1 -- 172.21.15.161:0/448281082 shutdown_connections 2024-05-22T21:25:12.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.570+0000 7f28d5e00640 1 -- 172.21.15.161:0/448281082 wait complete. 2024-05-22T21:25:12.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.570+0000 7f28d5e00640 1 Processor -- start 2024-05-22T21:25:12.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.570+0000 7f28d5e00640 1 -- start start 2024-05-22T21:25:12.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28d5e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28d006c7e0 0x7f28d0142ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:12.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28d5e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f28d0143c40 0x7f28d0136a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:12.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28d5e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d015b600 0x7f28d00b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:12.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28d5e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d0142290 con 0x7f28d015b600 2024-05-22T21:25:12.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28d5e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d00b3e10 con 0x7f28d006c7e0 2024-05-22T21:25:12.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28d5e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d00b40f0 con 0x7f28d0143c40 2024-05-22T21:25:12.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28cf400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d015b600 0x7f28d00b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:12.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28cf400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d015b600 0x7f28d00b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46842/0 (socket says 172.21.15.161:46842) 2024-05-22T21:25:12.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28cf400640 1 -- 172.21.15.161:0/3383744720 learned_addr learned my addr 172.21.15.161:0/3383744720 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:12.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28ce000640 1 --2- 172.21.15.161:0/3383744720 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f28d0143c40 0x7f28d0136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:12.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28cea00640 1 --2- 172.21.15.161:0/3383744720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28d006c7e0 0x7f28d0142ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:12.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28cf400640 1 -- 172.21.15.161:0/3383744720 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f28d0143c40 msgr2=0x7f28d0136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:12.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28cf400640 1 --2- 172.21.15.161:0/3383744720 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f28d0143c40 0x7f28d0136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28cf400640 1 -- 172.21.15.161:0/3383744720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28d006c7e0 msgr2=0x7f28d0142ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:12.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28cf400640 1 --2- 172.21.15.161:0/3383744720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28d006c7e0 0x7f28d0142ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28cf400640 1 -- 172.21.15.161:0/3383744720 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc0094d0 con 0x7f28d015b600 2024-05-22T21:25:12.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28cea00640 1 --2- 172.21.15.161:0/3383744720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28d006c7e0 0x7f28d0142ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:25:12.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28ce000640 1 --2- 172.21.15.161:0/3383744720 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f28d0143c40 0x7f28d0136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:25:12.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28cf400640 1 --2- 172.21.15.161:0/3383744720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d015b600 0x7f28d00b34e0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f28c800c470 tx=0x7f28c800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:12.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28c7400640 1 -- 172.21.15.161:0/3383744720 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c80074e0 con 0x7f28d015b600 2024-05-22T21:25:12.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28c7400640 1 -- 172.21.15.161:0/3383744720 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c8007640 con 0x7f28d015b600 2024-05-22T21:25:12.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28c7400640 1 -- 172.21.15.161:0/3383744720 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c8004a30 con 0x7f28d015b600 2024-05-22T21:25:12.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28d5e00640 1 -- 172.21.15.161:0/3383744720 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d01424a0 con 0x7f28d015b600 2024-05-22T21:25:12.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28d5e00640 1 -- 172.21.15.161:0/3383744720 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d0136f50 con 0x7f28d015b600 2024-05-22T21:25:12.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28d5e00640 1 -- 172.21.15.161:0/3383744720 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d0067220 con 0x7f28d015b600 2024-05-22T21:25:12.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.574+0000 7f28c7400640 1 -- 172.21.15.161:0/3383744720 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f28c8004b90 con 0x7f28d015b600 2024-05-22T21:25:12.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.578+0000 7f28c7400640 1 --2- 172.21.15.161:0/3383744720 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f28b0066690 0x7f28b0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:12.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.578+0000 7f28c7400640 1 -- 172.21.15.161:0/3383744720 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28c8083530 con 0x7f28d015b600 2024-05-22T21:25:12.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.578+0000 7f28cea00640 1 --2- 172.21.15.161:0/3383744720 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f28b0066690 0x7f28b0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:12.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.578+0000 7f28cea00640 1 --2- 172.21.15.161:0/3383744720 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f28b0066690 0x7f28b0068b50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f28c0007fd0 tx=0x7f28c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:12.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.582+0000 7f28c7400640 1 -- 172.21.15.161:0/3383744720 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f28c8050770 con 0x7f28d015b600 2024-05-22T21:25:12.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.750+0000 7f28d5e00640 1 -- 172.21.15.161:0/3383744720 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f28d00af320 con 0x7f28d015b600 2024-05-22T21:25:12.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.750+0000 7f28c7400640 1 -- 172.21.15.161:0/3383744720 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v193) v1 ==== 80+0+0 (secure 0 0 0) 0x7f28c8018020 con 0x7f28d015b600 2024-05-22T21:25:12.750 INFO:teuthology.orchestra.run.smithi161.stderr:listed 0 entries 2024-05-22T21:25:12.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.750+0000 7f28d5e00640 1 -- 172.21.15.161:0/3383744720 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f28b0066690 msgr2=0x7f28b0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:12.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.750+0000 7f28d5e00640 1 --2- 172.21.15.161:0/3383744720 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f28b0066690 0x7f28b0068b50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f28c0007fd0 tx=0x7f28c0009040 comp rx=0 tx=0).stop 2024-05-22T21:25:12.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.750+0000 7f28d5e00640 1 -- 172.21.15.161:0/3383744720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d015b600 msgr2=0x7f28d00b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:12.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.750+0000 7f28d5e00640 1 --2- 172.21.15.161:0/3383744720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d015b600 0x7f28d00b34e0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f28c800c470 tx=0x7f28c800c930 comp rx=0 tx=0).stop 2024-05-22T21:25:12.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.750+0000 7f28d5e00640 1 -- 172.21.15.161:0/3383744720 shutdown_connections 2024-05-22T21:25:12.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.750+0000 7f28d5e00640 1 --2- 172.21.15.161:0/3383744720 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f28b0066690 0x7f28b0068b50 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.750+0000 7f28d5e00640 1 --2- 172.21.15.161:0/3383744720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d015b600 0x7f28d00b34e0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.750+0000 7f28d5e00640 1 --2- 172.21.15.161:0/3383744720 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f28d0143c40 0x7f28d0136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.750+0000 7f28d5e00640 1 --2- 172.21.15.161:0/3383744720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28d006c7e0 0x7f28d0142ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:12.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.750+0000 7f28d5e00640 1 -- 172.21.15.161:0/3383744720 >> 172.21.15.161:0/3383744720 conn(0x7f28d006a2d0 msgr2=0x7f28d00af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:12.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.750+0000 7f28d5e00640 1 -- 172.21.15.161:0/3383744720 shutdown_connections 2024-05-22T21:25:12.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:12.750+0000 7f28d5e00640 1 -- 172.21.15.161:0/3383744720 wait complete. 2024-05-22T21:25:12.772 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-22T21:25:12.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:12.772 DEBUG:teuthology.orchestra.run.smithi179:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-22T21:25:12.800 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:25:12.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:12.801 DEBUG:teuthology.orchestra.run.smithi179:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-22T21:25:12.851 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:25:12.851 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:25:12.860 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:25:12.860 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:25:12.860 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:25:12.902 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:25:12.902 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:25:13.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.038+0000 7f43b0a00640 1 Processor -- start 2024-05-22T21:25:13.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.038+0000 7f43b0a00640 1 -- start start 2024-05-22T21:25:13.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.038+0000 7f43b0a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43ac0715a0 0x7f43ac15f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.038+0000 7f43b0a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f43ac15ec90 0x7f43ac15f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.038+0000 7f43b0a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ac1442a0 con 0x7f43ac143ec0 2024-05-22T21:25:13.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.038+0000 7f43b0a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ac15f660 con 0x7f43ac15ec90 2024-05-22T21:25:13.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.038+0000 7f43b0a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac070500 con 0x7f43ac0715a0 2024-05-22T21:25:13.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.038+0000 7f43aa800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f43ac15ec90 0x7f43ac15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:13.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.038+0000 7f43aa800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f43ac15ec90 0x7f43ac15f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46846/0 (socket says 172.21.15.161:46846) 2024-05-22T21:25:13.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.038+0000 7f43aa800640 1 -- 172.21.15.161:0/1272692643 learned_addr learned my addr 172.21.15.161:0/1272692643 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:13.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.038+0000 7f43a9400640 1 --2- 172.21.15.161:0/1272692643 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43ac0715a0 0x7f43ac15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:13.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.038+0000 7f43aa800640 1 -- 172.21.15.161:0/1272692643 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43ac0715a0 msgr2=0x7f43ac15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:13.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.038+0000 7f43aa800640 1 --2- 172.21.15.161:0/1272692643 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43ac0715a0 0x7f43ac15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.038+0000 7f43aa800640 1 -- 172.21.15.161:0/1272692643 >> v1:172.21.15.179:6789/0 conn(0x7f43ac143ec0 legacy=0x7f43ac06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.038+0000 7f43aa800640 1 -- 172.21.15.161:0/1272692643 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac0707e0 con 0x7f43ac15ec90 2024-05-22T21:25:13.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.038+0000 7f43a9400640 1 --2- 172.21.15.161:0/1272692643 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43ac0715a0 0x7f43ac15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:13.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.042+0000 7f43aa800640 1 --2- 172.21.15.161:0/1272692643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac15ec90 0x7f43ac15f120 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009e50 tx=0x7f43a401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d68fa16f88f98a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:13.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.042+0000 7f43a8a00640 1 -- 172.21.15.161:0/1272692643 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a40a57d0 con 0x7f43ac15ec90 2024-05-22T21:25:13.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.042+0000 7f43a8a00640 1 -- 172.21.15.161:0/1272692643 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a40a5930 con 0x7f43ac15ec90 2024-05-22T21:25:13.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.042+0000 7f43a8a00640 1 -- 172.21.15.161:0/1272692643 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a40a5cc0 con 0x7f43ac15ec90 2024-05-22T21:25:13.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.042+0000 7f43b0a00640 1 -- 172.21.15.161:0/1272692643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac15ec90 msgr2=0x7f43ac15f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.042+0000 7f43b0a00640 1 --2- 172.21.15.161:0/1272692643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac15ec90 0x7f43ac15f120 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009e50 tx=0x7f43a401ca00 comp rx=0 tx=0).stop 2024-05-22T21:25:13.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.042+0000 7f43b0a00640 1 -- 172.21.15.161:0/1272692643 shutdown_connections 2024-05-22T21:25:13.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.042+0000 7f43b0a00640 1 --2- 172.21.15.161:0/1272692643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac15ec90 0x7f43ac15f120 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.042+0000 7f43b0a00640 1 --2- 172.21.15.161:0/1272692643 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43ac0715a0 0x7f43ac15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.042+0000 7f43b0a00640 1 -- 172.21.15.161:0/1272692643 >> 172.21.15.161:0/1272692643 conn(0x7f43ac0b0bc0 msgr2=0x7f43ac0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:13.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.042+0000 7f43b0a00640 1 -- 172.21.15.161:0/1272692643 shutdown_connections 2024-05-22T21:25:13.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.042+0000 7f43b0a00640 1 -- 172.21.15.161:0/1272692643 wait complete. 2024-05-22T21:25:13.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43b0a00640 1 Processor -- start 2024-05-22T21:25:13.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43b0a00640 1 -- start start 2024-05-22T21:25:13.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43b0a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43ac143ec0 0x7f43ac15d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43b0a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43ac15ec90 0x7f43ac13cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43b0a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac13f850 0x7f43ac13d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43b0a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac13eab0 con 0x7f43ac13f850 2024-05-22T21:25:13.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43b0a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac13ed70 con 0x7f43ac15ec90 2024-05-22T21:25:13.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43b0a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac13f050 con 0x7f43ac143ec0 2024-05-22T21:25:13.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43aa800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac13f850 0x7f43ac13d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:13.048 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43aa800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac13f850 0x7f43ac13d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46852/0 (socket says 172.21.15.161:46852) 2024-05-22T21:25:13.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43aa800640 1 -- 172.21.15.161:0/2652536513 learned_addr learned my addr 172.21.15.161:0/2652536513 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:13.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43aa800640 1 -- 172.21.15.161:0/2652536513 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43ac143ec0 msgr2=0x7f43ac15d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43aa800640 1 --2- 172.21.15.161:0/2652536513 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43ac143ec0 0x7f43ac15d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43aa800640 1 -- 172.21.15.161:0/2652536513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43ac15ec90 msgr2=0x7f43ac13cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43aa800640 1 --2- 172.21.15.161:0/2652536513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43ac15ec90 0x7f43ac13cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43aa800640 1 -- 172.21.15.161:0/2652536513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a4009b30 con 0x7f43ac13f850 2024-05-22T21:25:13.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43aa800640 1 --2- 172.21.15.161:0/2652536513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac13f850 0x7f43ac13d0c0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f43a40af040 tx=0x7f43a40197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:13.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43a2000640 1 -- 172.21.15.161:0/2652536513 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a40b8670 con 0x7f43ac13f850 2024-05-22T21:25:13.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43a2000640 1 -- 172.21.15.161:0/2652536513 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a40b87d0 con 0x7f43ac13f850 2024-05-22T21:25:13.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43a2000640 1 -- 172.21.15.161:0/2652536513 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a40b8b80 con 0x7f43ac13f850 2024-05-22T21:25:13.049 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43b0a00640 1 -- 172.21.15.161:0/2652536513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ac13f330 con 0x7f43ac13f850 2024-05-22T21:25:13.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.046+0000 7f43b0a00640 1 -- 172.21.15.161:0/2652536513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ac15e0c0 con 0x7f43ac13f850 2024-05-22T21:25:13.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.050+0000 7f43b0a00640 1 -- 172.21.15.161:0/2652536513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43ac067220 con 0x7f43ac13f850 2024-05-22T21:25:13.050 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.050+0000 7f43a2000640 1 -- 172.21.15.161:0/2652536513 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f43a40b8ce0 con 0x7f43ac13f850 2024-05-22T21:25:13.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.050+0000 7f43a2000640 1 --2- 172.21.15.161:0/2652536513 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f438c066690 0x7f438c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.050+0000 7f43a2000640 1 -- 172.21.15.161:0/2652536513 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43a411f810 con 0x7f43ac13f850 2024-05-22T21:25:13.051 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.050+0000 7f43a9e00640 1 --2- 172.21.15.161:0/2652536513 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f438c066690 0x7f438c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:13.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.050+0000 7f43a9e00640 1 --2- 172.21.15.161:0/2652536513 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f438c066690 0x7f438c068b50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f439c000f70 tx=0x7f439c0079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:13.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.054+0000 7f43a2000640 1 -- 172.21.15.161:0/2652536513 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f43a40ecc80 con 0x7f43ac13f850 2024-05-22T21:25:13.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.238+0000 7f43b0a00640 1 -- 172.21.15.161:0/2652536513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43ac06dcc0 con 0x7f43ac13f850 2024-05-22T21:25:13.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.242+0000 7f43a2000640 1 -- 172.21.15.161:0/2652536513 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4744 (secure 0 0 0) 0x7f43a40f02a0 con 0x7f43ac13f850 2024-05-22T21:25:13.241 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:13.241 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":86,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3304182060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3304182060},{"type":"v1","addr":"172.21.15.179:6835","nonce":3304182060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196309272},{"type":"v1","addr":"172.21.15.161:6836","nonce":196309272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/396408969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":396408969},{"type":"v1","addr":"172.21.15.179:6834","nonce":396408969}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:24:34.096673+0000","modified":"2024-05-22T21:24:37.112845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8784},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8784":{"gid":8784,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2172511349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2172511349},{"type":"v1","addr":"172.21.15.161:6837","nonce":2172511349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-05-22T21:25:13.241 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 86 2024-05-22T21:25:13.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.242+0000 7f43b0a00640 1 -- 172.21.15.161:0/2652536513 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f438c066690 msgr2=0x7f438c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.242+0000 7f43b0a00640 1 --2- 172.21.15.161:0/2652536513 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f438c066690 0x7f438c068b50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f439c000f70 tx=0x7f439c0079c0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.242+0000 7f43b0a00640 1 -- 172.21.15.161:0/2652536513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac13f850 msgr2=0x7f43ac13d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.242+0000 7f43b0a00640 1 --2- 172.21.15.161:0/2652536513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac13f850 0x7f43ac13d0c0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f43a40af040 tx=0x7f43a40197a0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.243 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.242+0000 7f43b0a00640 1 -- 172.21.15.161:0/2652536513 shutdown_connections 2024-05-22T21:25:13.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.242+0000 7f43b0a00640 1 --2- 172.21.15.161:0/2652536513 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f438c066690 0x7f438c068b50 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.242+0000 7f43b0a00640 1 --2- 172.21.15.161:0/2652536513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43ac13f850 0x7f43ac13d0c0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.242+0000 7f43b0a00640 1 --2- 172.21.15.161:0/2652536513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43ac15ec90 0x7f43ac13cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.242+0000 7f43b0a00640 1 --2- 172.21.15.161:0/2652536513 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43ac143ec0 0x7f43ac15d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.242+0000 7f43b0a00640 1 -- 172.21.15.161:0/2652536513 >> 172.21.15.161:0/2652536513 conn(0x7f43ac0b0bc0 msgr2=0x7f43ac06c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:13.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.242+0000 7f43b0a00640 1 -- 172.21.15.161:0/2652536513 shutdown_connections 2024-05-22T21:25:13.244 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.242+0000 7f43b0a00640 1 -- 172.21.15.161:0/2652536513 wait complete. 2024-05-22T21:25:13.263 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:25:13.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.398+0000 7fa0f1a00640 1 Processor -- start 2024-05-22T21:25:13.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.398+0000 7fa0f1a00640 1 -- start start 2024-05-22T21:25:13.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.398+0000 7fa0f1a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa0ec071890 0x7fa0ec15b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.398+0000 7fa0f1a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec15bce0 con 0x7fa0ec15ed60 2024-05-22T21:25:13.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.398+0000 7fa0f1a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec15f340 con 0x7fa0ec1449f0 2024-05-22T21:25:13.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.398+0000 7fa0f1a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0ec15f620 con 0x7fa0ec071890 2024-05-22T21:25:13.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.398+0000 7fa0eb400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa0ec071890 0x7fa0ec15b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:13.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.398+0000 7fa0eb400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa0ec071890 0x7fa0ec15b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46860/0 (socket says 172.21.15.161:46860) 2024-05-22T21:25:13.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.398+0000 7fa0eb400640 1 -- 172.21.15.161:0/2448553627 learned_addr learned my addr 172.21.15.161:0/2448553627 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:13.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.398+0000 7fa0e2000640 1 --1- 172.21.15.161:0/2448553627 >> v1:172.21.15.161:6789/0 conn(0x7fa0ec15ed60 0x7fa0ec071180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39512/0 (socket says 172.21.15.161:39512) 2024-05-22T21:25:13.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.398+0000 7fa0eb400640 1 -- 172.21.15.161:0/2448553627 >> v1:172.21.15.179:6789/0 conn(0x7fa0ec1449f0 legacy=0x7fa0ec06a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.398+0000 7fa0eb400640 1 -- 172.21.15.161:0/2448553627 >> v1:172.21.15.161:6789/0 conn(0x7fa0ec15ed60 legacy=0x7fa0ec071180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.398+0000 7fa0ea000640 1 -- 172.21.15.161:0/2448553627 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208913145 0 0) 0x7fa0d4001120 con 0x7fa0ec15ed60 2024-05-22T21:25:13.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.398+0000 7fa0eb400640 1 -- 172.21.15.161:0/2448553627 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec15f900 con 0x7fa0ec071890 2024-05-22T21:25:13.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.402+0000 7fa0eb400640 1 --2- 172.21.15.161:0/2448553627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ec071890 0x7fa0ec15b7a0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009870 tx=0x7fa0e401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95aacc51a9ef6ac5 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:13.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.402+0000 7fa0ea000640 1 -- 172.21.15.161:0/2448553627 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40ab610 con 0x7fa0ec071890 2024-05-22T21:25:13.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.402+0000 7fa0ea000640 1 -- 172.21.15.161:0/2448553627 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e40ab770 con 0x7fa0ec071890 2024-05-22T21:25:13.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.402+0000 7fa0ea000640 1 -- 172.21.15.161:0/2448553627 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40abb00 con 0x7fa0ec071890 2024-05-22T21:25:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.402+0000 7fa0f1a00640 1 -- 172.21.15.161:0/2448553627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ec071890 msgr2=0x7fa0ec15b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.402+0000 7fa0f1a00640 1 --2- 172.21.15.161:0/2448553627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ec071890 0x7fa0ec15b7a0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009870 tx=0x7fa0e401c410 comp rx=0 tx=0).stop 2024-05-22T21:25:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.402+0000 7fa0f1a00640 1 -- 172.21.15.161:0/2448553627 shutdown_connections 2024-05-22T21:25:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.402+0000 7fa0f1a00640 1 --2- 172.21.15.161:0/2448553627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ec071890 0x7fa0ec15b7a0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.402+0000 7fa0f1a00640 1 -- 172.21.15.161:0/2448553627 >> 172.21.15.161:0/2448553627 conn(0x7fa0ec0b1c40 msgr2=0x7fa0ec0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:13.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.402+0000 7fa0f1a00640 1 -- 172.21.15.161:0/2448553627 shutdown_connections 2024-05-22T21:25:13.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.402+0000 7fa0f1a00640 1 -- 172.21.15.161:0/2448553627 wait complete. 2024-05-22T21:25:13.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0f1a00640 1 Processor -- start 2024-05-22T21:25:13.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0f1a00640 1 -- start start 2024-05-22T21:25:13.407 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0f1a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0ec071890 0x7fa0ec158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0f1a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa0ec1449f0 0x7fa0ec159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0f1a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ec15ed60 0x7fa0ec12d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0f1a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec0cf2f0 con 0x7fa0ec15ed60 2024-05-22T21:25:13.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0f1a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec159cb0 con 0x7fa0ec071890 2024-05-22T21:25:13.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0f1a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec0cf4d0 con 0x7fa0ec1449f0 2024-05-22T21:25:13.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0eb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ec15ed60 0x7fa0ec12d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:13.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0eb400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ec15ed60 0x7fa0ec12d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46866/0 (socket says 172.21.15.161:46866) 2024-05-22T21:25:13.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0eb400640 1 -- 172.21.15.161:0/3618722535 learned_addr learned my addr 172.21.15.161:0/3618722535 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:13.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0e2000640 1 --2- 172.21.15.161:0/3618722535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa0ec1449f0 0x7fa0ec159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:13.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0eaa00640 1 --2- 172.21.15.161:0/3618722535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0ec071890 0x7fa0ec158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:13.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0eb400640 1 -- 172.21.15.161:0/3618722535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa0ec1449f0 msgr2=0x7fa0ec159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0eb400640 1 --2- 172.21.15.161:0/3618722535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa0ec1449f0 0x7fa0ec159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0eb400640 1 -- 172.21.15.161:0/3618722535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0ec071890 msgr2=0x7fa0ec158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0eb400640 1 --2- 172.21.15.161:0/3618722535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0ec071890 0x7fa0ec158dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0eb400640 1 -- 172.21.15.161:0/3618722535 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e40094d0 con 0x7fa0ec15ed60 2024-05-22T21:25:13.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0eb400640 1 --2- 172.21.15.161:0/3618722535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ec15ed60 0x7fa0ec12d5b0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40ad040 tx=0x7fa0e401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:13.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0e3400640 1 -- 172.21.15.161:0/3618722535 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40b4800 con 0x7fa0ec15ed60 2024-05-22T21:25:13.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0e3400640 1 -- 172.21.15.161:0/3618722535 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e40b4980 con 0x7fa0ec15ed60 2024-05-22T21:25:13.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0e3400640 1 -- 172.21.15.161:0/3618722535 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40b4d30 con 0x7fa0ec15ed60 2024-05-22T21:25:13.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0f1a00640 1 -- 172.21.15.161:0/3618722535 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ec0cf7b0 con 0x7fa0ec15ed60 2024-05-22T21:25:13.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0f1a00640 1 -- 172.21.15.161:0/3618722535 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ec0cfd70 con 0x7fa0ec15ed60 2024-05-22T21:25:13.410 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.406+0000 7fa0f1a00640 1 -- 172.21.15.161:0/3618722535 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ec0cff60 con 0x7fa0ec15ed60 2024-05-22T21:25:13.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.410+0000 7fa0e3400640 1 -- 172.21.15.161:0/3618722535 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa0e40b3380 con 0x7fa0ec15ed60 2024-05-22T21:25:13.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.410+0000 7fa0e3400640 1 --2- 172.21.15.161:0/3618722535 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0c8066690 0x7fa0c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.410+0000 7fa0e3400640 1 -- 172.21.15.161:0/3618722535 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0e411e720 con 0x7fa0ec15ed60 2024-05-22T21:25:13.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.410+0000 7fa0eaa00640 1 --2- 172.21.15.161:0/3618722535 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0c8066690 0x7fa0c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:13.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.410+0000 7fa0eaa00640 1 --2- 172.21.15.161:0/3618722535 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0c8066690 0x7fa0c8068b50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc0012f0 tx=0x7fa0dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:13.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.414+0000 7fa0e3400640 1 -- 172.21.15.161:0/3618722535 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa0e40eba10 con 0x7fa0ec15ed60 2024-05-22T21:25:13.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.566+0000 7fa0f1a00640 1 -- 172.21.15.161:0/3618722535 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0ec067220 con 0x7fa0ec15ed60 2024-05-22T21:25:13.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.570+0000 7fa0e3400640 1 -- 172.21.15.161:0/3618722535 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+16159 (secure 0 0 0) 0x7fa0e40ef030 con 0x7fa0ec15ed60 2024-05-22T21:25:13.569 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:13.569 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":193,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:25:05.312760+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:24:31.797440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-05-22T21:24:33.098240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:25:13.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.570+0000 7fa0f1a00640 1 -- 172.21.15.161:0/3618722535 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0c8066690 msgr2=0x7fa0c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.570+0000 7fa0f1a00640 1 --2- 172.21.15.161:0/3618722535 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0c8066690 0x7fa0c8068b50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc0012f0 tx=0x7fa0dc009040 comp rx=0 tx=0).stop 2024-05-22T21:25:13.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.570+0000 7fa0f1a00640 1 -- 172.21.15.161:0/3618722535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ec15ed60 msgr2=0x7fa0ec12d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.570+0000 7fa0f1a00640 1 --2- 172.21.15.161:0/3618722535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ec15ed60 0x7fa0ec12d5b0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40ad040 tx=0x7fa0e401ac50 comp rx=0 tx=0).stop 2024-05-22T21:25:13.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.570+0000 7fa0f1a00640 1 -- 172.21.15.161:0/3618722535 shutdown_connections 2024-05-22T21:25:13.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.570+0000 7fa0f1a00640 1 --2- 172.21.15.161:0/3618722535 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0c8066690 0x7fa0c8068b50 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.570+0000 7fa0f1a00640 1 --2- 172.21.15.161:0/3618722535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ec15ed60 0x7fa0ec12d5b0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.570+0000 7fa0f1a00640 1 --2- 172.21.15.161:0/3618722535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa0ec1449f0 0x7fa0ec159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.570+0000 7fa0f1a00640 1 --2- 172.21.15.161:0/3618722535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0ec071890 0x7fa0ec158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.570+0000 7fa0f1a00640 1 -- 172.21.15.161:0/3618722535 >> 172.21.15.161:0/3618722535 conn(0x7fa0ec0b1c40 msgr2=0x7fa0ec15d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:13.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.570+0000 7fa0f1a00640 1 -- 172.21.15.161:0/3618722535 shutdown_connections 2024-05-22T21:25:13.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.570+0000 7fa0f1a00640 1 -- 172.21.15.161:0/3618722535 wait complete. 2024-05-22T21:25:13.589 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-22T21:13:12.449493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-22T21:24:31.797440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '189', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-05-22T21:24:33.098240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '193', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-22T21:25:13.590 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-22T21:25:13.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.726+0000 7fd1b0c00640 1 Processor -- start 2024-05-22T21:25:13.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.730+0000 7fd1b0c00640 1 -- start start 2024-05-22T21:25:13.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.730+0000 7fd1b0c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd1ac06b9f0 0x7fd1ac06bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.730+0000 7fd1b0c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd1ac15cb70 0x7fd1ac15cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.730+0000 7fd1b0c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ac06e7e0 con 0x7fd1ac06b1f0 2024-05-22T21:25:13.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.730+0000 7fd1b0c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ac06c310 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.730+0000 7fd1b0c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ac0d04a0 con 0x7fd1ac15cb70 2024-05-22T21:25:13.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.730+0000 7fd1a9e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd1ac06b9f0 0x7fd1ac06bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:13.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.730+0000 7fd1a9e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd1ac06b9f0 0x7fd1ac06bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46876/0 (socket says 172.21.15.161:46876) 2024-05-22T21:25:13.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.730+0000 7fd1a9e00640 1 -- 172.21.15.161:0/3680178995 learned_addr learned my addr 172.21.15.161:0/3680178995 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:13.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.730+0000 7fd1aa800640 1 --2- 172.21.15.161:0/3680178995 >> v2:172.21.15.179:3300/0 conn(0x7fd1ac15cb70 0x7fd1ac15cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:13.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.730+0000 7fd1a9e00640 1 -- 172.21.15.161:0/3680178995 >> v2:172.21.15.179:3300/0 conn(0x7fd1ac15cb70 msgr2=0x7fd1ac15cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:13.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.730+0000 7fd1a9e00640 1 --2- 172.21.15.161:0/3680178995 >> v2:172.21.15.179:3300/0 conn(0x7fd1ac15cb70 0x7fd1ac15cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.730+0000 7fd1a9e00640 1 -- 172.21.15.161:0/3680178995 >> v1:172.21.15.179:6789/0 conn(0x7fd1ac06b1f0 legacy=0x7fd1ac0cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.730+0000 7fd1a9e00640 1 -- 172.21.15.161:0/3680178995 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac06deb0 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.734+0000 7fd1a9e00640 1 --2- 172.21.15.161:0/3680178995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ac06b9f0 0x7fd1ac06bdd0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0c7f60 tx=0x7fd19c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5b9482f97bbc79 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:13.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.734+0000 7fd1a8a00640 1 -- 172.21.15.161:0/3680178995 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c0ab600 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.734+0000 7fd1a8a00640 1 -- 172.21.15.161:0/3680178995 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19c0a7050 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.734+0000 7fd1a8a00640 1 -- 172.21.15.161:0/3680178995 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c0ab980 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.734+0000 7fd1b0c00640 1 -- 172.21.15.161:0/3680178995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ac06b9f0 msgr2=0x7fd1ac06bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.734+0000 7fd1b0c00640 1 --2- 172.21.15.161:0/3680178995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ac06b9f0 0x7fd1ac06bdd0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0c7f60 tx=0x7fd19c01c460 comp rx=0 tx=0).stop 2024-05-22T21:25:13.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.734+0000 7fd1b0c00640 1 -- 172.21.15.161:0/3680178995 shutdown_connections 2024-05-22T21:25:13.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.734+0000 7fd1b0c00640 1 --2- 172.21.15.161:0/3680178995 >> v2:172.21.15.179:3300/0 conn(0x7fd1ac15cb70 0x7fd1ac15cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.734+0000 7fd1b0c00640 1 --2- 172.21.15.161:0/3680178995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ac06b9f0 0x7fd1ac06bdd0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.734+0000 7fd1b0c00640 1 -- 172.21.15.161:0/3680178995 >> 172.21.15.161:0/3680178995 conn(0x7fd1ac0d0f00 msgr2=0x7fd1ac0d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:13.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.734+0000 7fd1b0c00640 1 -- 172.21.15.161:0/3680178995 shutdown_connections 2024-05-22T21:25:13.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.734+0000 7fd1b0c00640 1 -- 172.21.15.161:0/3680178995 wait complete. 2024-05-22T21:25:13.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.734+0000 7fd1b0c00640 1 Processor -- start 2024-05-22T21:25:13.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.734+0000 7fd1b0c00640 1 -- start start 2024-05-22T21:25:13.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1b0c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1ac06b1f0 0x7fd1ac0c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1b0c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ac06b9f0 0x7fd1ac1f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1b0c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ac15cb70 0x7fd1ac15bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1b0c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac154ef0 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1b0c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0c7680 con 0x7fd1ac15cb70 2024-05-22T21:25:13.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1b0c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0c7960 con 0x7fd1ac06b1f0 2024-05-22T21:25:13.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1a9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ac06b9f0 0x7fd1ac1f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:13.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1a9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ac06b9f0 0x7fd1ac1f8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46878/0 (socket says 172.21.15.161:46878) 2024-05-22T21:25:13.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1a9400640 1 -- 172.21.15.161:0/2593142346 learned_addr learned my addr 172.21.15.161:0/2593142346 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:13.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1a9e00640 1 --2- 172.21.15.161:0/2593142346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1ac06b1f0 0x7fd1ac0c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:13.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1aa800640 1 --2- 172.21.15.161:0/2593142346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ac15cb70 0x7fd1ac15bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:13.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1a9400640 1 -- 172.21.15.161:0/2593142346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1ac06b1f0 msgr2=0x7fd1ac0c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1a9400640 1 --2- 172.21.15.161:0/2593142346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1ac06b1f0 0x7fd1ac0c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1a9400640 1 -- 172.21.15.161:0/2593142346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ac15cb70 msgr2=0x7fd1ac15bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1a9400640 1 --2- 172.21.15.161:0/2593142346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ac15cb70 0x7fd1ac15bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1a9400640 1 -- 172.21.15.161:0/2593142346 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c0094d0 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1a9400640 1 --2- 172.21.15.161:0/2593142346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ac06b9f0 0x7fd1ac1f8e30 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fd19800c500 tx=0x7fd19800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:13.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1a2000640 1 -- 172.21.15.161:0/2593142346 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd198005220 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1a2000640 1 -- 172.21.15.161:0/2593142346 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd198005380 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1b0c00640 1 -- 172.21.15.161:0/2593142346 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ac0c6a00 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1a2000640 1 -- 172.21.15.161:0/2593142346 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd198007270 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1b0c00640 1 -- 172.21.15.161:0/2593142346 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ac0c6fc0 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1b0c00640 1 -- 172.21.15.161:0/2593142346 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ac0c77c0 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.738+0000 7fd1a2000640 1 -- 172.21.15.161:0/2593142346 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd1980073d0 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.742+0000 7fd1a2000640 1 --2- 172.21.15.161:0/2593142346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd18c066690 0x7fd18c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:13.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.742+0000 7fd1a2000640 1 -- 172.21.15.161:0/2593142346 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd198083a80 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.742+0000 7fd1a9e00640 1 --2- 172.21.15.161:0/2593142346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd18c066690 0x7fd18c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:13.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.742+0000 7fd1a9e00640 1 --2- 172.21.15.161:0/2593142346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd18c066690 0x7fd18c068b50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0ad000 tx=0x7fd19c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:13.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.746+0000 7fd1a2000640 1 -- 172.21.15.161:0/2593142346 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fd1ac0c77c0 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.899 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.898+0000 7fd1b0c00640 1 -- 172.21.15.161:0/2593142346 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd1ac0c77c0 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.898+0000 7fd1a2000640 1 -- 172.21.15.161:0/2593142346 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v193) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd1ac0c77c0 con 0x7fd1ac06b9f0 2024-05-22T21:25:13.900 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-05-22T21:25:13.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.902+0000 7fd1b0c00640 1 -- 172.21.15.161:0/2593142346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd18c066690 msgr2=0x7fd18c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.902+0000 7fd1b0c00640 1 --2- 172.21.15.161:0/2593142346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd18c066690 0x7fd18c068b50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0ad000 tx=0x7fd19c008040 comp rx=0 tx=0).stop 2024-05-22T21:25:13.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.902+0000 7fd1b0c00640 1 -- 172.21.15.161:0/2593142346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ac06b9f0 msgr2=0x7fd1ac1f8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:13.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.902+0000 7fd1b0c00640 1 --2- 172.21.15.161:0/2593142346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ac06b9f0 0x7fd1ac1f8e30 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fd19800c500 tx=0x7fd19800c9c0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.902+0000 7fd1b0c00640 1 -- 172.21.15.161:0/2593142346 shutdown_connections 2024-05-22T21:25:13.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.902+0000 7fd1b0c00640 1 --2- 172.21.15.161:0/2593142346 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd18c066690 0x7fd18c068b50 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.902+0000 7fd1b0c00640 1 --2- 172.21.15.161:0/2593142346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1ac15cb70 0x7fd1ac15bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.902+0000 7fd1b0c00640 1 --2- 172.21.15.161:0/2593142346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ac06b9f0 0x7fd1ac1f8e30 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.902+0000 7fd1b0c00640 1 --2- 172.21.15.161:0/2593142346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1ac06b1f0 0x7fd1ac0c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:13.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.902+0000 7fd1b0c00640 1 -- 172.21.15.161:0/2593142346 >> 172.21.15.161:0/2593142346 conn(0x7fd1ac0d0f00 msgr2=0x7fd1ac0ae540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:13.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.902+0000 7fd1b0c00640 1 -- 172.21.15.161:0/2593142346 shutdown_connections 2024-05-22T21:25:13.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:13.902+0000 7fd1b0c00640 1 -- 172.21.15.161:0/2593142346 wait complete. 2024-05-22T21:25:13.921 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-22T21:25:14.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.058+0000 7f20f7200640 1 Processor -- start 2024-05-22T21:25:14.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.058+0000 7f20f7200640 1 -- start start 2024-05-22T21:25:14.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.058+0000 7f20f7200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20f013cfc0 0x7f20f01402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.058+0000 7f20f7200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f20f013fe50 0x7f20f006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.058+0000 7f20f7200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f013d3a0 con 0x7f20f015af60 2024-05-22T21:25:14.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.058+0000 7f20f7200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f00cd920 con 0x7f20f013fe50 2024-05-22T21:25:14.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.058+0000 7f20f7200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00cdc00 con 0x7f20f013cfc0 2024-05-22T21:25:14.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.058+0000 7f20f5200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f20f013fe50 0x7f20f006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.058+0000 7f20f5200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f20f013fe50 0x7f20f006fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46888/0 (socket says 172.21.15.161:46888) 2024-05-22T21:25:14.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.058+0000 7f20f5200640 1 -- 172.21.15.161:0/2648716451 learned_addr learned my addr 172.21.15.161:0/2648716451 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:14.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.058+0000 7f20efe00640 1 --2- 172.21.15.161:0/2648716451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20f013cfc0 0x7f20f01402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.058+0000 7f20f5200640 1 -- 172.21.15.161:0/2648716451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20f013cfc0 msgr2=0x7f20f01402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:14.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.058+0000 7f20f5200640 1 --2- 172.21.15.161:0/2648716451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20f013cfc0 0x7f20f01402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.058+0000 7f20f5200640 1 -- 172.21.15.161:0/2648716451 >> v1:172.21.15.179:6789/0 conn(0x7f20f015af60 legacy=0x7f20f013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:14.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.058+0000 7f20f5200640 1 -- 172.21.15.161:0/2648716451 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f00cdee0 con 0x7f20f013fe50 2024-05-22T21:25:14.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.058+0000 7f20efe00640 1 --2- 172.21.15.161:0/2648716451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20f013cfc0 0x7f20f01402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:25:14.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.062+0000 7f20f5200640 1 --2- 172.21.15.161:0/2648716451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f013fe50 0x7f20f006fce0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009e50 tx=0x7f20e801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aebf09da454d55a server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:14.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.062+0000 7f20ef400640 1 -- 172.21.15.161:0/2648716451 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e80a8450 con 0x7f20f013fe50 2024-05-22T21:25:14.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.062+0000 7f20ef400640 1 -- 172.21.15.161:0/2648716451 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e80a85b0 con 0x7f20f013fe50 2024-05-22T21:25:14.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.062+0000 7f20ef400640 1 -- 172.21.15.161:0/2648716451 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e80a8940 con 0x7f20f013fe50 2024-05-22T21:25:14.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.062+0000 7f20f7200640 1 -- 172.21.15.161:0/2648716451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f013fe50 msgr2=0x7f20f006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:14.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.062+0000 7f20f7200640 1 --2- 172.21.15.161:0/2648716451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f013fe50 0x7f20f006fce0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009e50 tx=0x7f20e801c930 comp rx=0 tx=0).stop 2024-05-22T21:25:14.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.062+0000 7f20f7200640 1 -- 172.21.15.161:0/2648716451 shutdown_connections 2024-05-22T21:25:14.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.062+0000 7f20f7200640 1 --2- 172.21.15.161:0/2648716451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f013fe50 0x7f20f006fce0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.062+0000 7f20f7200640 1 --2- 172.21.15.161:0/2648716451 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20f013cfc0 0x7f20f01402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.062+0000 7f20f7200640 1 -- 172.21.15.161:0/2648716451 >> 172.21.15.161:0/2648716451 conn(0x7f20f006a2d0 msgr2=0x7f20f006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:14.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.062+0000 7f20f7200640 1 -- 172.21.15.161:0/2648716451 shutdown_connections 2024-05-22T21:25:14.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.062+0000 7f20f7200640 1 -- 172.21.15.161:0/2648716451 wait complete. 2024-05-22T21:25:14.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20f7200640 1 Processor -- start 2024-05-22T21:25:14.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20f7200640 1 -- start start 2024-05-22T21:25:14.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20f7200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f013cfc0 0x7f20f0155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20f7200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20f013fe50 0x7f20f0140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20f7200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20f015af60 0x7f20f00b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20f7200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f015d670 con 0x7f20f013cfc0 2024-05-22T21:25:14.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20f7200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f015d0b0 con 0x7f20f013fe50 2024-05-22T21:25:14.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20f7200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f015d390 con 0x7f20f015af60 2024-05-22T21:25:14.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20efe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f013cfc0 0x7f20f0155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20efe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f013cfc0 0x7f20f0155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46890/0 (socket says 172.21.15.161:46890) 2024-05-22T21:25:14.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20efe00640 1 -- 172.21.15.161:0/3828244860 learned_addr learned my addr 172.21.15.161:0/3828244860 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:14.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20f5200640 1 --2- 172.21.15.161:0/3828244860 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20f015af60 0x7f20f00b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20e7400640 1 --2- 172.21.15.161:0/3828244860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20f013fe50 0x7f20f0140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20efe00640 1 -- 172.21.15.161:0/3828244860 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20f015af60 msgr2=0x7f20f00b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:14.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20efe00640 1 --2- 172.21.15.161:0/3828244860 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20f015af60 0x7f20f00b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20efe00640 1 -- 172.21.15.161:0/3828244860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20f013fe50 msgr2=0x7f20f0140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:14.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20efe00640 1 --2- 172.21.15.161:0/3828244860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20f013fe50 0x7f20f0140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20efe00640 1 -- 172.21.15.161:0/3828244860 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e8009b30 con 0x7f20f013cfc0 2024-05-22T21:25:14.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20efe00640 1 --2- 172.21.15.161:0/3828244860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f013cfc0 0x7f20f0155d70 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f20e000c7c0 tx=0x7f20e000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:14.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20ecc00640 1 -- 172.21.15.161:0/3828244860 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0007a30 con 0x7f20f013cfc0 2024-05-22T21:25:14.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20ecc00640 1 -- 172.21.15.161:0/3828244860 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e0007b90 con 0x7f20f013cfc0 2024-05-22T21:25:14.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20ecc00640 1 -- 172.21.15.161:0/3828244860 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e000ce70 con 0x7f20f013cfc0 2024-05-22T21:25:14.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20f7200640 1 -- 172.21.15.161:0/3828244860 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f00b2d30 con 0x7f20f013cfc0 2024-05-22T21:25:14.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20f7200640 1 -- 172.21.15.161:0/3828244860 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f01563d0 con 0x7f20f013cfc0 2024-05-22T21:25:14.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.066+0000 7f20f7200640 1 -- 172.21.15.161:0/3828244860 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f00b2b90 con 0x7f20f013cfc0 2024-05-22T21:25:14.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.070+0000 7f20ecc00640 1 -- 172.21.15.161:0/3828244860 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f20e00113f0 con 0x7f20f013cfc0 2024-05-22T21:25:14.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.070+0000 7f20ecc00640 1 --2- 172.21.15.161:0/3828244860 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f20cc066690 0x7f20cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.070+0000 7f20ecc00640 1 -- 172.21.15.161:0/3828244860 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20e00846f0 con 0x7f20f013cfc0 2024-05-22T21:25:14.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.070+0000 7f20e7400640 1 --2- 172.21.15.161:0/3828244860 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f20cc066690 0x7f20cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.070+0000 7f20e7400640 1 --2- 172.21.15.161:0/3828244860 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f20cc066690 0x7f20cc068b50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f20f0140d30 tx=0x7f20d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:14.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.074+0000 7f20ecc00640 1 -- 172.21.15.161:0/3828244860 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f20f00b2b90 con 0x7f20f013cfc0 2024-05-22T21:25:14.231 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.230+0000 7f20f7200640 1 -- 172.21.15.161:0/3828244860 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20f00b2b90 con 0x7f20f013cfc0 2024-05-22T21:25:14.232 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.230+0000 7f20ecc00640 1 -- 172.21.15.161:0/3828244860 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v193) v1 ==== 104+0+11 (secure 0 0 0) 0x7f20f00b2b90 con 0x7f20f013cfc0 2024-05-22T21:25:14.232 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-05-22T21:25:14.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.234+0000 7f20f7200640 1 -- 172.21.15.161:0/3828244860 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f20cc066690 msgr2=0x7f20cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:14.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.234+0000 7f20f7200640 1 --2- 172.21.15.161:0/3828244860 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f20cc066690 0x7f20cc068b50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f20f0140d30 tx=0x7f20d8008040 comp rx=0 tx=0).stop 2024-05-22T21:25:14.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.234+0000 7f20f7200640 1 -- 172.21.15.161:0/3828244860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f013cfc0 msgr2=0x7f20f0155d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:14.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.234+0000 7f20f7200640 1 --2- 172.21.15.161:0/3828244860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f013cfc0 0x7f20f0155d70 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f20e000c7c0 tx=0x7f20e000cc80 comp rx=0 tx=0).stop 2024-05-22T21:25:14.234 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.234+0000 7f20f7200640 1 -- 172.21.15.161:0/3828244860 shutdown_connections 2024-05-22T21:25:14.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.234+0000 7f20f7200640 1 --2- 172.21.15.161:0/3828244860 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f20cc066690 0x7f20cc068b50 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.234+0000 7f20f7200640 1 --2- 172.21.15.161:0/3828244860 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20f015af60 0x7f20f00b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.234+0000 7f20f7200640 1 --2- 172.21.15.161:0/3828244860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20f013fe50 0x7f20f0140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.234+0000 7f20f7200640 1 --2- 172.21.15.161:0/3828244860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f013cfc0 0x7f20f0155d70 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.234+0000 7f20f7200640 1 -- 172.21.15.161:0/3828244860 >> 172.21.15.161:0/3828244860 conn(0x7f20f006a2d0 msgr2=0x7f20f00af540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:14.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.234+0000 7f20f7200640 1 -- 172.21.15.161:0/3828244860 shutdown_connections 2024-05-22T21:25:14.235 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.234+0000 7f20f7200640 1 -- 172.21.15.161:0/3828244860 wait complete. 2024-05-22T21:25:14.252 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-22T21:25:14.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.390+0000 7f44e9400640 1 Processor -- start 2024-05-22T21:25:14.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.390+0000 7f44e9400640 1 -- start start 2024-05-22T21:25:14.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.390+0000 7f44e9400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f44e41402e0 0x7f44e413f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.390+0000 7f44e9400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f44e413fc80 0x7f44e406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.390+0000 7f44e9400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e4140090 con 0x7f44e413cfc0 2024-05-22T21:25:14.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.390+0000 7f44e9400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e413d3a0 con 0x7f44e41402e0 2024-05-22T21:25:14.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.390+0000 7f44e9400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e40cd920 con 0x7f44e413fc80 2024-05-22T21:25:14.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.390+0000 7f44e1e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f44e41402e0 0x7f44e413f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.390+0000 7f44e2800640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f44e413cfc0 0x7f44e415af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39524/0 (socket says 172.21.15.161:39524) 2024-05-22T21:25:14.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.390+0000 7f44e2800640 1 -- 172.21.15.161:0/577302637 learned_addr learned my addr 172.21.15.161:0/577302637 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:14.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.390+0000 7f44e3200640 1 --2- 172.21.15.161:0/577302637 >> v2:172.21.15.179:3300/0 conn(0x7f44e413fc80 0x7f44e406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.390+0000 7f44e1e00640 1 -- 172.21.15.161:0/577302637 >> v2:172.21.15.179:3300/0 conn(0x7f44e413fc80 msgr2=0x7f44e406fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:14.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.390+0000 7f44e1e00640 1 --2- 172.21.15.161:0/577302637 >> v2:172.21.15.179:3300/0 conn(0x7f44e413fc80 0x7f44e406fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.390+0000 7f44e1e00640 1 -- 172.21.15.161:0/577302637 >> v1:172.21.15.161:6789/0 conn(0x7f44e413cfc0 legacy=0x7f44e415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:14.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.390+0000 7f44e1e00640 1 -- 172.21.15.161:0/577302637 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e40cdc00 con 0x7f44e41402e0 2024-05-22T21:25:14.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.390+0000 7f44e1400640 1 -- 172.21.15.161:0/577302637 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39240943 0 0) 0x7f44e4140090 con 0x7f44e413cfc0 2024-05-22T21:25:14.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.394+0000 7f44e1e00640 1 --2- 172.21.15.161:0/577302637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e41402e0 0x7f44e413f740 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f44e406c5e0 tx=0x7f44cc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a592963de1d562 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:14.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.394+0000 7f44e1400640 1 -- 172.21.15.161:0/577302637 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc0ab600 con 0x7f44e41402e0 2024-05-22T21:25:14.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.394+0000 7f44e1400640 1 -- 172.21.15.161:0/577302637 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc0a7050 con 0x7f44e41402e0 2024-05-22T21:25:14.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.394+0000 7f44e1400640 1 -- 172.21.15.161:0/577302637 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc0ab980 con 0x7f44e41402e0 2024-05-22T21:25:14.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.394+0000 7f44e9400640 1 -- 172.21.15.161:0/577302637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e41402e0 msgr2=0x7f44e413f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:14.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.394+0000 7f44e9400640 1 --2- 172.21.15.161:0/577302637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e41402e0 0x7f44e413f740 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f44e406c5e0 tx=0x7f44cc01c3c0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.394+0000 7f44e9400640 1 -- 172.21.15.161:0/577302637 shutdown_connections 2024-05-22T21:25:14.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.394+0000 7f44e9400640 1 --2- 172.21.15.161:0/577302637 >> v2:172.21.15.179:3300/0 conn(0x7f44e413fc80 0x7f44e406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.394+0000 7f44e9400640 1 --2- 172.21.15.161:0/577302637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e41402e0 0x7f44e413f740 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.394+0000 7f44e9400640 1 -- 172.21.15.161:0/577302637 >> 172.21.15.161:0/577302637 conn(0x7f44e406a2d0 msgr2=0x7f44e406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:14.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.394+0000 7f44e9400640 1 -- 172.21.15.161:0/577302637 shutdown_connections 2024-05-22T21:25:14.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.394+0000 7f44e9400640 1 -- 172.21.15.161:0/577302637 wait complete. 2024-05-22T21:25:14.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e9400640 1 Processor -- start 2024-05-22T21:25:14.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e9400640 1 -- start start 2024-05-22T21:25:14.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e9400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f44e413cfc0 0x7f44e41569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e413fc80 0x7f44e413ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e9400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44e41402e0 0x7f44e413b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e9400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e413b5f0 con 0x7f44e413fc80 2024-05-22T21:25:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e9400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e4156f00 con 0x7f44e41402e0 2024-05-22T21:25:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e9400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e415b110 con 0x7f44e413cfc0 2024-05-22T21:25:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e413fc80 0x7f44e413ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e413fc80 0x7f44e413ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46910/0 (socket says 172.21.15.161:46910) 2024-05-22T21:25:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e1e00640 1 -- 172.21.15.161:0/3222933538 learned_addr learned my addr 172.21.15.161:0/3222933538 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e2800640 1 --2- 172.21.15.161:0/3222933538 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f44e413cfc0 0x7f44e41569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e3200640 1 --2- 172.21.15.161:0/3222933538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44e41402e0 0x7f44e413b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e1e00640 1 -- 172.21.15.161:0/3222933538 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f44e413cfc0 msgr2=0x7f44e41569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e1e00640 1 --2- 172.21.15.161:0/3222933538 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f44e413cfc0 0x7f44e41569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e1e00640 1 -- 172.21.15.161:0/3222933538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44e41402e0 msgr2=0x7f44e413b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e1e00640 1 --2- 172.21.15.161:0/3222933538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44e41402e0 0x7f44e413b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e1e00640 1 -- 172.21.15.161:0/3222933538 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc009420 con 0x7f44e413fc80 2024-05-22T21:25:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e1e00640 1 --2- 172.21.15.161:0/3222933538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e413fc80 0x7f44e413ab70 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0ae040 tx=0x7f44cc019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44daa00640 1 -- 172.21.15.161:0/3222933538 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc0b4840 con 0x7f44e413fc80 2024-05-22T21:25:14.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44daa00640 1 -- 172.21.15.161:0/3222933538 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc0b49c0 con 0x7f44e413fc80 2024-05-22T21:25:14.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44daa00640 1 -- 172.21.15.161:0/3222933538 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc0b4d70 con 0x7f44e413fc80 2024-05-22T21:25:14.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e9400640 1 -- 172.21.15.161:0/3222933538 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e415b3f0 con 0x7f44e413fc80 2024-05-22T21:25:14.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e9400640 1 -- 172.21.15.161:0/3222933538 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e415b9b0 con 0x7f44e413fc80 2024-05-22T21:25:14.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.398+0000 7f44e9400640 1 -- 172.21.15.161:0/3222933538 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e4067220 con 0x7f44e413fc80 2024-05-22T21:25:14.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.402+0000 7f44daa00640 1 -- 172.21.15.161:0/3222933538 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f44cc0b2040 con 0x7f44e413fc80 2024-05-22T21:25:14.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.402+0000 7f44daa00640 1 --2- 172.21.15.161:0/3222933538 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f44c0066690 0x7f44c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.402+0000 7f44daa00640 1 -- 172.21.15.161:0/3222933538 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44cc11e5b0 con 0x7f44e413fc80 2024-05-22T21:25:14.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.402+0000 7f44e2800640 1 --2- 172.21.15.161:0/3222933538 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f44c0066690 0x7f44c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.402+0000 7f44e2800640 1 --2- 172.21.15.161:0/3222933538 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f44c0066690 0x7f44c0068b50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f44d4000f70 tx=0x7f44d4007b60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:14.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.406+0000 7f44daa00640 1 -- 172.21.15.161:0/3222933538 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f44cc0eb880 con 0x7f44e413fc80 2024-05-22T21:25:14.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.562+0000 7f44e9400640 1 -- 172.21.15.161:0/3222933538 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f44e4071d00 con 0x7f44e413fc80 2024-05-22T21:25:14.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.562+0000 7f44daa00640 1 -- 172.21.15.161:0/3222933538 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v193) v1 ==== 100+0+11 (secure 0 0 0) 0x7f44cc11ce60 con 0x7f44e413fc80 2024-05-22T21:25:14.562 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 64 2024-05-22T21:25:14.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.562+0000 7f44e9400640 1 -- 172.21.15.161:0/3222933538 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f44c0066690 msgr2=0x7f44c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:14.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.562+0000 7f44e9400640 1 --2- 172.21.15.161:0/3222933538 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f44c0066690 0x7f44c0068b50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f44d4000f70 tx=0x7f44d4007b60 comp rx=0 tx=0).stop 2024-05-22T21:25:14.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.562+0000 7f44e9400640 1 -- 172.21.15.161:0/3222933538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e413fc80 msgr2=0x7f44e413ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:14.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.562+0000 7f44e9400640 1 --2- 172.21.15.161:0/3222933538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e413fc80 0x7f44e413ab70 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0ae040 tx=0x7f44cc019160 comp rx=0 tx=0).stop 2024-05-22T21:25:14.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.562+0000 7f44e9400640 1 -- 172.21.15.161:0/3222933538 shutdown_connections 2024-05-22T21:25:14.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.562+0000 7f44e9400640 1 --2- 172.21.15.161:0/3222933538 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f44c0066690 0x7f44c0068b50 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.562+0000 7f44e9400640 1 --2- 172.21.15.161:0/3222933538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44e41402e0 0x7f44e413b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.562+0000 7f44e9400640 1 --2- 172.21.15.161:0/3222933538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e413fc80 0x7f44e413ab70 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.562+0000 7f44e9400640 1 --2- 172.21.15.161:0/3222933538 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f44e413cfc0 0x7f44e41569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.562+0000 7f44e9400640 1 -- 172.21.15.161:0/3222933538 >> 172.21.15.161:0/3222933538 conn(0x7f44e406a2d0 msgr2=0x7f44e406a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:14.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.562+0000 7f44e9400640 1 -- 172.21.15.161:0/3222933538 shutdown_connections 2024-05-22T21:25:14.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.562+0000 7f44e9400640 1 -- 172.21.15.161:0/3222933538 wait complete. 2024-05-22T21:25:14.583 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:25:14.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.722+0000 7f1712c00640 1 Processor -- start 2024-05-22T21:25:14.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.722+0000 7f1712c00640 1 -- start start 2024-05-22T21:25:14.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.722+0000 7f1712c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c15dd50 0x7f170c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.722+0000 7f1712c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f170c15e130 0x7f170c15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.722+0000 7f1712c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f170c15d410 0x7f170c15a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.722+0000 7f1712c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170c071670 con 0x7f170c15d410 2024-05-22T21:25:14.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.722+0000 7f1712c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170c15d8a0 con 0x7f170c15e130 2024-05-22T21:25:14.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.722+0000 7f1712c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c06de40 con 0x7f170c15dd50 2024-05-22T21:25:14.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.722+0000 7f1710c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f170c15d410 0x7f170c15a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.722+0000 7f1710c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f170c15d410 0x7f170c15a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46914/0 (socket says 172.21.15.161:46914) 2024-05-22T21:25:14.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.726+0000 7f1710c00640 1 -- 172.21.15.161:0/705879178 learned_addr learned my addr 172.21.15.161:0/705879178 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:14.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.726+0000 7f170be00640 1 --2- 172.21.15.161:0/705879178 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c15dd50 0x7f170c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.726+0000 7f170b400640 1 --2- 172.21.15.161:0/705879178 >> v2:172.21.15.179:3300/0 conn(0x7f170c15e130 0x7f170c15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.726+0000 7f1710c00640 1 -- 172.21.15.161:0/705879178 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c15dd50 msgr2=0x7f170c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:14.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.726+0000 7f1710c00640 1 --2- 172.21.15.161:0/705879178 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c15dd50 0x7f170c06d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.726+0000 7f1710c00640 1 -- 172.21.15.161:0/705879178 >> v2:172.21.15.179:3300/0 conn(0x7f170c15e130 msgr2=0x7f170c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:14.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.726+0000 7f1710c00640 1 --2- 172.21.15.161:0/705879178 >> v2:172.21.15.179:3300/0 conn(0x7f170c15e130 0x7f170c15ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.726+0000 7f1710c00640 1 -- 172.21.15.161:0/705879178 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c070a60 con 0x7f170c15d410 2024-05-22T21:25:14.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.726+0000 7f170be00640 1 --2- 172.21.15.161:0/705879178 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c15dd50 0x7f170c06d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:14.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.726+0000 7f1710c00640 1 --2- 172.21.15.161:0/705879178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c15d410 0x7f170c15a500 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f170c072510 tx=0x7f170401eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e1ba37c38d90e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:14.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.726+0000 7f170aa00640 1 -- 172.21.15.161:0/705879178 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170400b2d0 con 0x7f170c15d410 2024-05-22T21:25:14.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.726+0000 7f170aa00640 1 -- 172.21.15.161:0/705879178 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170400b430 con 0x7f170c15d410 2024-05-22T21:25:14.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.726+0000 7f170aa00640 1 -- 172.21.15.161:0/705879178 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17040a7be0 con 0x7f170c15d410 2024-05-22T21:25:14.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 -- 172.21.15.161:0/705879178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c15d410 msgr2=0x7f170c15a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:14.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 --2- 172.21.15.161:0/705879178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c15d410 0x7f170c15a500 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f170c072510 tx=0x7f170401eb30 comp rx=0 tx=0).stop 2024-05-22T21:25:14.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 -- 172.21.15.161:0/705879178 shutdown_connections 2024-05-22T21:25:14.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 --2- 172.21.15.161:0/705879178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c15d410 0x7f170c15a500 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 --2- 172.21.15.161:0/705879178 >> v2:172.21.15.179:3300/0 conn(0x7f170c15e130 0x7f170c15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 --2- 172.21.15.161:0/705879178 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c15dd50 0x7f170c06d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 -- 172.21.15.161:0/705879178 >> 172.21.15.161:0/705879178 conn(0x7f170c0b0bc0 msgr2=0x7f170c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:14.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 -- 172.21.15.161:0/705879178 shutdown_connections 2024-05-22T21:25:14.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 -- 172.21.15.161:0/705879178 wait complete. 2024-05-22T21:25:14.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 Processor -- start 2024-05-22T21:25:14.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 -- start start 2024-05-22T21:25:14.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c15d410 0x7f170c0cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c15dd50 0x7f170c140d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f170c15e130 0x7f170c1459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c1465b0 con 0x7f170c15d410 2024-05-22T21:25:14.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c145ff0 con 0x7f170c15e130 2024-05-22T21:25:14.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1712c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c1462d0 con 0x7f170c15dd50 2024-05-22T21:25:14.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f170be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c15d410 0x7f170c0cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f170be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c15d410 0x7f170c0cde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46928/0 (socket says 172.21.15.161:46928) 2024-05-22T21:25:14.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f170be00640 1 -- 172.21.15.161:0/1933993765 learned_addr learned my addr 172.21.15.161:0/1933993765 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:14.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f1710c00640 1 --2- 172.21.15.161:0/1933993765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f170c15e130 0x7f170c1459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f170b400640 1 --2- 172.21.15.161:0/1933993765 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c15dd50 0x7f170c140d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f170be00640 1 -- 172.21.15.161:0/1933993765 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c15dd50 msgr2=0x7f170c140d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:14.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f170be00640 1 --2- 172.21.15.161:0/1933993765 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c15dd50 0x7f170c140d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f170be00640 1 -- 172.21.15.161:0/1933993765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f170c15e130 msgr2=0x7f170c1459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:14.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f170be00640 1 --2- 172.21.15.161:0/1933993765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f170c15e130 0x7f170c1459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f170be00640 1 -- 172.21.15.161:0/1933993765 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170400bd40 con 0x7f170c15d410 2024-05-22T21:25:14.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.730+0000 7f170be00640 1 --2- 172.21.15.161:0/1933993765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c15d410 0x7f170c0cde90 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00e810 tx=0x7f16fc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:14.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.734+0000 7f1703e00640 1 -- 172.21.15.161:0/1933993765 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00c8d0 con 0x7f170c15d410 2024-05-22T21:25:14.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.734+0000 7f1703e00640 1 -- 172.21.15.161:0/1933993765 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc00ca30 con 0x7f170c15d410 2024-05-22T21:25:14.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.734+0000 7f1703e00640 1 -- 172.21.15.161:0/1933993765 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00cde0 con 0x7f170c15d410 2024-05-22T21:25:14.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.734+0000 7f1712c00640 1 -- 172.21.15.161:0/1933993765 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170c150b80 con 0x7f170c15d410 2024-05-22T21:25:14.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.734+0000 7f1712c00640 1 -- 172.21.15.161:0/1933993765 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c143b20 con 0x7f170c15d410 2024-05-22T21:25:14.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.734+0000 7f1712c00640 1 -- 172.21.15.161:0/1933993765 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170c067220 con 0x7f170c15d410 2024-05-22T21:25:14.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.734+0000 7f1703e00640 1 -- 172.21.15.161:0/1933993765 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f16fc013410 con 0x7f170c15d410 2024-05-22T21:25:14.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.734+0000 7f1703e00640 1 --2- 172.21.15.161:0/1933993765 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16e8066690 0x7f16e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:14.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.734+0000 7f1703e00640 1 -- 172.21.15.161:0/1933993765 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16fc083fb0 con 0x7f170c15d410 2024-05-22T21:25:14.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.734+0000 7f170b400640 1 --2- 172.21.15.161:0/1933993765 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16e8066690 0x7f16e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:14.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.738+0000 7f170b400640 1 --2- 172.21.15.161:0/1933993765 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16e8066690 0x7f16e8068b50 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f170c0719c0 tx=0x7f16f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:14.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.738+0000 7f1703e00640 1 -- 172.21.15.161:0/1933993765 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f16fc051330 con 0x7f170c15d410 2024-05-22T21:25:14.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.926+0000 7f1712c00640 1 -- 172.21.15.161:0/1933993765 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f170c0ce450 con 0x7f170c15d410 2024-05-22T21:25:14.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.926+0000 7f1703e00640 1 -- 172.21.15.161:0/1933993765 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4744 (secure 0 0 0) 0x7f16fc054950 con 0x7f170c15d410 2024-05-22T21:25:14.927 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:14.928 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":86,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3304182060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3304182060},{"type":"v1","addr":"172.21.15.179:6835","nonce":3304182060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196309272},{"type":"v1","addr":"172.21.15.161:6836","nonce":196309272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/396408969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":396408969},{"type":"v1","addr":"172.21.15.179:6834","nonce":396408969}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:24:34.096673+0000","modified":"2024-05-22T21:24:37.112845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8784},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8784":{"gid":8784,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2172511349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2172511349},{"type":"v1","addr":"172.21.15.161:6837","nonce":2172511349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-05-22T21:25:14.928 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 86 2024-05-22T21:25:14.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.930+0000 7f1712c00640 1 -- 172.21.15.161:0/1933993765 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16e8066690 msgr2=0x7f16e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:14.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.930+0000 7f1712c00640 1 --2- 172.21.15.161:0/1933993765 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16e8066690 0x7f16e8068b50 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f170c0719c0 tx=0x7f16f4008040 comp rx=0 tx=0).stop 2024-05-22T21:25:14.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.930+0000 7f1712c00640 1 -- 172.21.15.161:0/1933993765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c15d410 msgr2=0x7f170c0cde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:14.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.930+0000 7f1712c00640 1 --2- 172.21.15.161:0/1933993765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c15d410 0x7f170c0cde90 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00e810 tx=0x7f16fc00ecd0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.930+0000 7f1712c00640 1 -- 172.21.15.161:0/1933993765 shutdown_connections 2024-05-22T21:25:14.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.930+0000 7f1712c00640 1 --2- 172.21.15.161:0/1933993765 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f16e8066690 0x7f16e8068b50 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.930+0000 7f1712c00640 1 --2- 172.21.15.161:0/1933993765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f170c15e130 0x7f170c1459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.930+0000 7f1712c00640 1 --2- 172.21.15.161:0/1933993765 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f170c15dd50 0x7f170c140d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.930+0000 7f1712c00640 1 --2- 172.21.15.161:0/1933993765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f170c15d410 0x7f170c0cde90 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:14.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.930+0000 7f1712c00640 1 -- 172.21.15.161:0/1933993765 >> 172.21.15.161:0/1933993765 conn(0x7f170c0b0bc0 msgr2=0x7f170c14f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:14.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.930+0000 7f1712c00640 1 -- 172.21.15.161:0/1933993765 shutdown_connections 2024-05-22T21:25:14.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:14.930+0000 7f1712c00640 1 -- 172.21.15.161:0/1933993765 wait complete. 2024-05-22T21:25:14.950 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:25:15.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.086+0000 7f3b0de00640 1 Processor -- start 2024-05-22T21:25:15.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.086+0000 7f3b0de00640 1 -- start start 2024-05-22T21:25:15.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.086+0000 7f3b0de00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3b080cf290 0x7f3b0806e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.086+0000 7f3b0de00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3b0806b150 0x7f3b0806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.086+0000 7f3b0de00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b080cf8c0 con 0x7f3b0806e980 2024-05-22T21:25:15.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.086+0000 7f3b0de00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b080cfba0 con 0x7f3b0806b150 2024-05-22T21:25:15.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.086+0000 7f3b0de00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b080cfe80 con 0x7f3b080cf290 2024-05-22T21:25:15.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.086+0000 7f3b07400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3b0806b150 0x7f3b0806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.086+0000 7f3b07400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3b0806b150 0x7f3b0806b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46942/0 (socket says 172.21.15.161:46942) 2024-05-22T21:25:15.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.086+0000 7f3b07400640 1 -- 172.21.15.161:0/2102597961 learned_addr learned my addr 172.21.15.161:0/2102597961 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:15.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.086+0000 7f3affe00640 1 --2- 172.21.15.161:0/2102597961 >> v2:172.21.15.179:3300/0 conn(0x7f3b080cf290 0x7f3b0806e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.086+0000 7f3b07400640 1 -- 172.21.15.161:0/2102597961 >> v2:172.21.15.179:3300/0 conn(0x7f3b080cf290 msgr2=0x7f3b0806e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:15.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.086+0000 7f3b07400640 1 --2- 172.21.15.161:0/2102597961 >> v2:172.21.15.179:3300/0 conn(0x7f3b080cf290 0x7f3b0806e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.086+0000 7f3b07400640 1 -- 172.21.15.161:0/2102597961 >> v1:172.21.15.179:6789/0 conn(0x7f3b0806e980 legacy=0x7f3b0806ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.086+0000 7f3b07400640 1 -- 172.21.15.161:0/2102597961 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b080d0160 con 0x7f3b0806b150 2024-05-22T21:25:15.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.090+0000 7f3b07400640 1 --2- 172.21.15.161:0/2102597961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b0806b150 0x7f3b0806b5c0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f3b08071000 tx=0x7f3b0001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed3cde6cb9821a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:15.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.090+0000 7f3b06000640 1 -- 172.21.15.161:0/2102597961 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b000a56c0 con 0x7f3b0806b150 2024-05-22T21:25:15.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.090+0000 7f3b06000640 1 -- 172.21.15.161:0/2102597961 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b000a5820 con 0x7f3b0806b150 2024-05-22T21:25:15.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.090+0000 7f3b06000640 1 -- 172.21.15.161:0/2102597961 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b000a5bb0 con 0x7f3b0806b150 2024-05-22T21:25:15.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.090+0000 7f3b0de00640 1 -- 172.21.15.161:0/2102597961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b0806b150 msgr2=0x7f3b0806b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.090+0000 7f3b0de00640 1 --2- 172.21.15.161:0/2102597961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b0806b150 0x7f3b0806b5c0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f3b08071000 tx=0x7f3b0001c9f0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.090+0000 7f3b0de00640 1 -- 172.21.15.161:0/2102597961 shutdown_connections 2024-05-22T21:25:15.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.090+0000 7f3b0de00640 1 --2- 172.21.15.161:0/2102597961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b0806b150 0x7f3b0806b5c0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.090+0000 7f3b0de00640 1 --2- 172.21.15.161:0/2102597961 >> v2:172.21.15.179:3300/0 conn(0x7f3b080cf290 0x7f3b0806e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.090+0000 7f3b0de00640 1 -- 172.21.15.161:0/2102597961 >> 172.21.15.161:0/2102597961 conn(0x7f3b080b2a40 msgr2=0x7f3b080b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:15.092 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.090+0000 7f3b0de00640 1 -- 172.21.15.161:0/2102597961 shutdown_connections 2024-05-22T21:25:15.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.090+0000 7f3b0de00640 1 -- 172.21.15.161:0/2102597961 wait complete. 2024-05-22T21:25:15.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.090+0000 7f3b0de00640 1 Processor -- start 2024-05-22T21:25:15.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.090+0000 7f3b0de00640 1 -- start start 2024-05-22T21:25:15.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b0de00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b0806b150 0x7f3b08158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b0de00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3b0806e980 0x7f3b0809dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b0de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b080cf290 0x7f3b0815a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b0de00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b08159d40 con 0x7f3b080cf290 2024-05-22T21:25:15.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b0de00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0809e470 con 0x7f3b0806b150 2024-05-22T21:25:15.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b0de00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0809e750 con 0x7f3b0806e980 2024-05-22T21:25:15.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b07400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b080cf290 0x7f3b0815a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b07400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b080cf290 0x7f3b0815a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46950/0 (socket says 172.21.15.161:46950) 2024-05-22T21:25:15.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b07400640 1 -- 172.21.15.161:0/931889959 learned_addr learned my addr 172.21.15.161:0/931889959 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:15.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3affe00640 1 --2- 172.21.15.161:0/931889959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3b0806e980 0x7f3b0809dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b06a00640 1 --2- 172.21.15.161:0/931889959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b0806b150 0x7f3b08158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b07400640 1 -- 172.21.15.161:0/931889959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3b0806e980 msgr2=0x7f3b0809dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b07400640 1 --2- 172.21.15.161:0/931889959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3b0806e980 0x7f3b0809dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b07400640 1 -- 172.21.15.161:0/931889959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b0806b150 msgr2=0x7f3b08158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b07400640 1 --2- 172.21.15.161:0/931889959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b0806b150 0x7f3b08158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b07400640 1 -- 172.21.15.161:0/931889959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b00009b30 con 0x7f3b080cf290 2024-05-22T21:25:15.096 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b07400640 1 --2- 172.21.15.161:0/931889959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b080cf290 0x7f3b0815a4d0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f3b000af040 tx=0x7f3b00019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:15.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3aff400640 1 -- 172.21.15.161:0/931889959 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b000b8670 con 0x7f3b080cf290 2024-05-22T21:25:15.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3aff400640 1 -- 172.21.15.161:0/931889959 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b000b87d0 con 0x7f3b080cf290 2024-05-22T21:25:15.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3aff400640 1 -- 172.21.15.161:0/931889959 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b000b8b80 con 0x7f3b080cf290 2024-05-22T21:25:15.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b0de00640 1 -- 172.21.15.161:0/931889959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0809ea30 con 0x7f3b080cf290 2024-05-22T21:25:15.097 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b0de00640 1 -- 172.21.15.161:0/931889959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b081592e0 con 0x7f3b080cf290 2024-05-22T21:25:15.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3b0de00640 1 -- 172.21.15.161:0/931889959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0809eee0 con 0x7f3b080cf290 2024-05-22T21:25:15.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.094+0000 7f3aff400640 1 -- 172.21.15.161:0/931889959 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3b000b8ce0 con 0x7f3b080cf290 2024-05-22T21:25:15.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.098+0000 7f3aff400640 1 --2- 172.21.15.161:0/931889959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ae4066690 0x7f3ae4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.098+0000 7f3aff400640 1 -- 172.21.15.161:0/931889959 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b0011f4b0 con 0x7f3b080cf290 2024-05-22T21:25:15.098 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.098+0000 7f3b06a00640 1 --2- 172.21.15.161:0/931889959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ae4066690 0x7f3ae4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.099 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.098+0000 7f3b06a00640 1 --2- 172.21.15.161:0/931889959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ae4066690 0x7f3ae4068b50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3af8007fd0 tx=0x7f3af8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:15.101 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.102+0000 7f3aff400640 1 -- 172.21.15.161:0/931889959 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3b000ec830 con 0x7f3b080cf290 2024-05-22T21:25:15.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.258+0000 7f3b0de00640 1 -- 172.21.15.161:0/931889959 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b08067220 con 0x7f3b080cf290 2024-05-22T21:25:15.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.262+0000 7f3aff400640 1 -- 172.21.15.161:0/931889959 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+16159 (secure 0 0 0) 0x7f3b00009510 con 0x7f3b080cf290 2024-05-22T21:25:15.262 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:15.262 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":193,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:25:05.312760+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:24:31.797440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-05-22T21:24:33.098240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:25:15.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.262+0000 7f3b0de00640 1 -- 172.21.15.161:0/931889959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ae4066690 msgr2=0x7f3ae4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.262+0000 7f3b0de00640 1 --2- 172.21.15.161:0/931889959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ae4066690 0x7f3ae4068b50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3af8007fd0 tx=0x7f3af8009040 comp rx=0 tx=0).stop 2024-05-22T21:25:15.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.262+0000 7f3b0de00640 1 -- 172.21.15.161:0/931889959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b080cf290 msgr2=0x7f3b0815a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.262+0000 7f3b0de00640 1 --2- 172.21.15.161:0/931889959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b080cf290 0x7f3b0815a4d0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f3b000af040 tx=0x7f3b00019790 comp rx=0 tx=0).stop 2024-05-22T21:25:15.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.262+0000 7f3b0de00640 1 -- 172.21.15.161:0/931889959 shutdown_connections 2024-05-22T21:25:15.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.262+0000 7f3b0de00640 1 --2- 172.21.15.161:0/931889959 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ae4066690 0x7f3ae4068b50 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.262+0000 7f3b0de00640 1 --2- 172.21.15.161:0/931889959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b080cf290 0x7f3b0815a4d0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.262+0000 7f3b0de00640 1 --2- 172.21.15.161:0/931889959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3b0806e980 0x7f3b0809dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.262+0000 7f3b0de00640 1 --2- 172.21.15.161:0/931889959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b0806b150 0x7f3b08158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.262+0000 7f3b0de00640 1 -- 172.21.15.161:0/931889959 >> 172.21.15.161:0/931889959 conn(0x7f3b080b2a40 msgr2=0x7f3b080b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:15.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.262+0000 7f3b0de00640 1 -- 172.21.15.161:0/931889959 shutdown_connections 2024-05-22T21:25:15.265 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.262+0000 7f3b0de00640 1 -- 172.21.15.161:0/931889959 wait complete. 2024-05-22T21:25:15.283 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-22T21:25:15.284 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:25:15.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.422+0000 7f31a1600640 1 Processor -- start 2024-05-22T21:25:15.422 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.422+0000 7f31a1600640 1 -- start start 2024-05-22T21:25:15.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.422+0000 7f31a1600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f319c15dd50 0x7f319c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.422+0000 7f31a1600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f319c071670 con 0x7f319c15d5e0 2024-05-22T21:25:15.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.422+0000 7f31a1600640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f319c06de40 con 0x7f319c15e130 2024-05-22T21:25:15.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.422+0000 7f31a1600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f319c070a60 con 0x7f319c15dd50 2024-05-22T21:25:15.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.422+0000 7f319aa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f319c15dd50 0x7f319c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.422+0000 7f319aa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f319c15dd50 0x7f319c06d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46954/0 (socket says 172.21.15.161:46954) 2024-05-22T21:25:15.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.422+0000 7f319aa00640 1 -- 172.21.15.161:0/933375077 learned_addr learned my addr 172.21.15.161:0/933375077 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:15.424 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.422+0000 7f319b400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f319c15d5e0 0x7f319c15b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39526/0 (socket says 172.21.15.161:39526) 2024-05-22T21:25:15.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.422+0000 7f319aa00640 1 -- 172.21.15.161:0/933375077 >> v1:172.21.15.179:6789/0 conn(0x7f319c15e130 legacy=0x7f319c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.422+0000 7f319aa00640 1 -- 172.21.15.161:0/933375077 >> v1:172.21.15.161:6789/0 conn(0x7f319c15d5e0 legacy=0x7f319c15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.422+0000 7f319aa00640 1 -- 172.21.15.161:0/933375077 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c070d40 con 0x7f319c15dd50 2024-05-22T21:25:15.425 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.422+0000 7f3199600640 1 -- 172.21.15.161:0/933375077 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1963812030 0 0) 0x7f319c071670 con 0x7f319c15d5e0 2024-05-22T21:25:15.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.426+0000 7f319aa00640 1 --2- 172.21.15.161:0/933375077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c15dd50 0x7f319c06d900 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f319400d150 tx=0x7f319401ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4e2a06381c48c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:15.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.426+0000 7f3199600640 1 -- 172.21.15.161:0/933375077 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31940a7cd0 con 0x7f319c15dd50 2024-05-22T21:25:15.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.426+0000 7f3199600640 1 -- 172.21.15.161:0/933375077 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31940a7e30 con 0x7f319c15dd50 2024-05-22T21:25:15.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.426+0000 7f3199600640 1 -- 172.21.15.161:0/933375077 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319400b2f0 con 0x7f319c15dd50 2024-05-22T21:25:15.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.426+0000 7f31a1600640 1 -- 172.21.15.161:0/933375077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c15dd50 msgr2=0x7f319c06d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.426+0000 7f31a1600640 1 --2- 172.21.15.161:0/933375077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c15dd50 0x7f319c06d900 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f319400d150 tx=0x7f319401ec00 comp rx=0 tx=0).stop 2024-05-22T21:25:15.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.426+0000 7f31a1600640 1 -- 172.21.15.161:0/933375077 shutdown_connections 2024-05-22T21:25:15.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.426+0000 7f31a1600640 1 --2- 172.21.15.161:0/933375077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c15dd50 0x7f319c06d900 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.426+0000 7f31a1600640 1 -- 172.21.15.161:0/933375077 >> 172.21.15.161:0/933375077 conn(0x7f319c0b0bc0 msgr2=0x7f319c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:15.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.426+0000 7f31a1600640 1 -- 172.21.15.161:0/933375077 shutdown_connections 2024-05-22T21:25:15.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.426+0000 7f31a1600640 1 -- 172.21.15.161:0/933375077 wait complete. 2024-05-22T21:25:15.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f31a1600640 1 Processor -- start 2024-05-22T21:25:15.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f31a1600640 1 -- start start 2024-05-22T21:25:15.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f31a1600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c15d5e0 0x7f319c141b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f31a1600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319c15dd50 0x7f319c14d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f31a1600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f319c15e130 0x7f319c14d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f31a1600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c14db00 con 0x7f319c15d5e0 2024-05-22T21:25:15.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f31a1600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c0b3e40 con 0x7f319c15e130 2024-05-22T21:25:15.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f31a1600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319c1476c0 con 0x7f319c15dd50 2024-05-22T21:25:15.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f319aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c15d5e0 0x7f319c141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f319aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c15d5e0 0x7f319c141b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46964/0 (socket says 172.21.15.161:46964) 2024-05-22T21:25:15.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f319aa00640 1 -- 172.21.15.161:0/3261132129 learned_addr learned my addr 172.21.15.161:0/3261132129 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:15.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f319b400640 1 --2- 172.21.15.161:0/3261132129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f319c15e130 0x7f319c14d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f319a000640 1 --2- 172.21.15.161:0/3261132129 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319c15dd50 0x7f319c14d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f319aa00640 1 -- 172.21.15.161:0/3261132129 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319c15dd50 msgr2=0x7f319c14d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f319aa00640 1 --2- 172.21.15.161:0/3261132129 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319c15dd50 0x7f319c14d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f319aa00640 1 -- 172.21.15.161:0/3261132129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f319c15e130 msgr2=0x7f319c14d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f319aa00640 1 --2- 172.21.15.161:0/3261132129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f319c15e130 0x7f319c14d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f319aa00640 1 -- 172.21.15.161:0/3261132129 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319400bd40 con 0x7f319c15d5e0 2024-05-22T21:25:15.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f319a000640 1 --2- 172.21.15.161:0/3261132129 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319c15dd50 0x7f319c14d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:15.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f319aa00640 1 --2- 172.21.15.161:0/3261132129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c15d5e0 0x7f319c141b70 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f31940b1040 tx=0x7f319401b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:15.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f3192a00640 1 -- 172.21.15.161:0/3261132129 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31940a7cd0 con 0x7f319c15d5e0 2024-05-22T21:25:15.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f3192a00640 1 -- 172.21.15.161:0/3261132129 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3194009a10 con 0x7f319c15d5e0 2024-05-22T21:25:15.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f3192a00640 1 -- 172.21.15.161:0/3261132129 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194009dc0 con 0x7f319c15d5e0 2024-05-22T21:25:15.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f31a1600640 1 -- 172.21.15.161:0/3261132129 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319c1479a0 con 0x7f319c15d5e0 2024-05-22T21:25:15.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f31a1600640 1 -- 172.21.15.161:0/3261132129 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319c1420b0 con 0x7f319c15d5e0 2024-05-22T21:25:15.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.430+0000 7f31a1600640 1 -- 172.21.15.161:0/3261132129 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319c067220 con 0x7f319c15d5e0 2024-05-22T21:25:15.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.434+0000 7f3192a00640 1 -- 172.21.15.161:0/3261132129 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f31940b8450 con 0x7f319c15d5e0 2024-05-22T21:25:15.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.434+0000 7f3192a00640 1 --2- 172.21.15.161:0/3261132129 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f31780666c0 0x7f3178068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.434+0000 7f3192a00640 1 -- 172.21.15.161:0/3261132129 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3194120e00 con 0x7f319c15d5e0 2024-05-22T21:25:15.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.434+0000 7f3192a00640 1 -- 172.21.15.161:0/3261132129 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=195}) v3 -- 0x7f317806e1f0 con 0x7f319c15d5e0 2024-05-22T21:25:15.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.434+0000 7f319a000640 1 --2- 172.21.15.161:0/3261132129 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f31780666c0 0x7f3178068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.434+0000 7f319a000640 1 --2- 172.21.15.161:0/3261132129 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f31780666c0 0x7f3178068b80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f319c1464f0 tx=0x7f318c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:15.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.438+0000 7f3192a00640 1 -- 172.21.15.161:0/3261132129 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f31940ee0d0 con 0x7f319c15d5e0 2024-05-22T21:25:15.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.622+0000 7f31a1600640 1 -- 172.21.15.161:0/3261132129 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f319c147800 con 0x7f319c15d5e0 2024-05-22T21:25:15.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.626+0000 7f3192a00640 1 -- 172.21.15.161:0/3261132129 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4744 (secure 0 0 0) 0x7f319c147800 con 0x7f319c15d5e0 2024-05-22T21:25:15.625 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:15.625 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":86,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3304182060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3304182060},{"type":"v1","addr":"172.21.15.179:6835","nonce":3304182060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196309272},{"type":"v1","addr":"172.21.15.161:6836","nonce":196309272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/396408969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":396408969},{"type":"v1","addr":"172.21.15.179:6834","nonce":396408969}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:24:34.096673+0000","modified":"2024-05-22T21:24:37.112845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8784},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8784":{"gid":8784,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2172511349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2172511349},{"type":"v1","addr":"172.21.15.161:6837","nonce":2172511349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-05-22T21:25:15.626 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 86 2024-05-22T21:25:15.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.626+0000 7f31a1600640 1 -- 172.21.15.161:0/3261132129 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f31780666c0 msgr2=0x7f3178068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.626+0000 7f31a1600640 1 --2- 172.21.15.161:0/3261132129 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f31780666c0 0x7f3178068b80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f319c1464f0 tx=0x7f318c009040 comp rx=0 tx=0).stop 2024-05-22T21:25:15.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.626+0000 7f31a1600640 1 -- 172.21.15.161:0/3261132129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c15d5e0 msgr2=0x7f319c141b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.626+0000 7f31a1600640 1 --2- 172.21.15.161:0/3261132129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c15d5e0 0x7f319c141b70 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f31940b1040 tx=0x7f319401b9a0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.626+0000 7f31a1600640 1 -- 172.21.15.161:0/3261132129 shutdown_connections 2024-05-22T21:25:15.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.626+0000 7f31a1600640 1 --2- 172.21.15.161:0/3261132129 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f31780666c0 0x7f3178068b80 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.626+0000 7f31a1600640 1 --2- 172.21.15.161:0/3261132129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f319c15e130 0x7f319c14d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.626+0000 7f31a1600640 1 --2- 172.21.15.161:0/3261132129 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f319c15dd50 0x7f319c14d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.626+0000 7f31a1600640 1 --2- 172.21.15.161:0/3261132129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f319c15d5e0 0x7f319c141b70 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.626+0000 7f31a1600640 1 -- 172.21.15.161:0/3261132129 >> 172.21.15.161:0/3261132129 conn(0x7f319c0b0bc0 msgr2=0x7f319c0ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:15.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.626+0000 7f31a1600640 1 -- 172.21.15.161:0/3261132129 shutdown_connections 2024-05-22T21:25:15.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.626+0000 7f31a1600640 1 -- 172.21.15.161:0/3261132129 wait complete. 2024-05-22T21:25:15.646 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:25:15.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f9901400640 1 Processor -- start 2024-05-22T21:25:15.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f9901400640 1 -- start start 2024-05-22T21:25:15.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f9901400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98fc15d2f0 0x7f98fc06eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f9901400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98fc0d0740 0x7f98fc06efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f9901400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98fc0b2d90 con 0x7f98fc0af1b0 2024-05-22T21:25:15.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f9901400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98fc0cf330 con 0x7f98fc0d0740 2024-05-22T21:25:15.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f9901400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc0cf610 con 0x7f98fc15d2f0 2024-05-22T21:25:15.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f98f3400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f98fc0af1b0 0x7f98fc160cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39536/0 (socket says 172.21.15.161:39536) 2024-05-22T21:25:15.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f98f3400640 1 -- 172.21.15.161:0/717702567 learned_addr learned my addr 172.21.15.161:0/717702567 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:15.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f98fbe00640 1 --2- 172.21.15.161:0/717702567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98fc15d2f0 0x7f98fc06eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f9900a00640 1 --2- 172.21.15.161:0/717702567 >> v2:172.21.15.179:3300/0 conn(0x7f98fc0d0740 0x7f98fc06efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f98fb400640 1 -- 172.21.15.161:0/717702567 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 424234586 0 0) 0x7f98fc0b2d90 con 0x7f98fc0af1b0 2024-05-22T21:25:15.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f98fb400640 1 -- 172.21.15.161:0/717702567 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98e0003530 con 0x7f98fc0af1b0 2024-05-22T21:25:15.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f98fb400640 1 -- 172.21.15.161:0/717702567 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2791630557 0 0) 0x7f98e0003530 con 0x7f98fc0af1b0 2024-05-22T21:25:15.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f98fb400640 1 -- 172.21.15.161:0/717702567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98fc15d2f0 msgr2=0x7f98fc06eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:15.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f98fb400640 1 --2- 172.21.15.161:0/717702567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98fc15d2f0 0x7f98fc06eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f98fb400640 1 -- 172.21.15.161:0/717702567 >> v2:172.21.15.179:3300/0 conn(0x7f98fc0d0740 msgr2=0x7f98fc06efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:15.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f98fb400640 1 --2- 172.21.15.161:0/717702567 >> v2:172.21.15.179:3300/0 conn(0x7f98fc0d0740 0x7f98fc06efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f98fb400640 1 -- 172.21.15.161:0/717702567 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc0cf8f0 con 0x7f98fc0af1b0 2024-05-22T21:25:15.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f98fb400640 1 -- 172.21.15.161:0/717702567 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f98e4001680 con 0x7f98fc0af1b0 2024-05-22T21:25:15.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f98fb400640 1 -- 172.21.15.161:0/717702567 >> v1:172.21.15.161:6789/0 conn(0x7f98fc0af1b0 legacy=0x7f98fc160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f98fb400640 1 --2- 172.21.15.161:0/717702567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98fc15d2f0 0x7f98e0003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f98fb400640 1 --2- 172.21.15.161:0/717702567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98e0003f00 0x7f98e00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.782+0000 7f98fb400640 1 --2- 172.21.15.161:0/717702567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98e0004d10 0x7f98e0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.786+0000 7f98fb400640 1 -- 172.21.15.161:0/717702567 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc0cf330 con 0x7f98e0004d10 2024-05-22T21:25:15.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.786+0000 7f98fb400640 1 -- 172.21.15.161:0/717702567 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc0cf610 con 0x7f98e0003f00 2024-05-22T21:25:15.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.786+0000 7f98fb400640 1 -- 172.21.15.161:0/717702567 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e000b990 con 0x7f98fc15d2f0 2024-05-22T21:25:15.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.786+0000 7f9900a00640 1 --2- 172.21.15.161:0/717702567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98e0004d10 0x7f98e0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.786+0000 7f98f3400640 1 --2- 172.21.15.161:0/717702567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98e0003f00 0x7f98e00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.786+0000 7f98fbe00640 1 --2- 172.21.15.161:0/717702567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98fc15d2f0 0x7f98e0003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.786+0000 7f9900a00640 1 -- 172.21.15.161:0/717702567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98fc15d2f0 msgr2=0x7f98e0003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:15.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.786+0000 7f9900a00640 1 --2- 172.21.15.161:0/717702567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98fc15d2f0 0x7f98e0003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.786+0000 7f9900a00640 1 -- 172.21.15.161:0/717702567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98e0003f00 msgr2=0x7f98e00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:15.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.786+0000 7f9900a00640 1 --2- 172.21.15.161:0/717702567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98e0003f00 0x7f98e00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.786+0000 7f9900a00640 1 -- 172.21.15.161:0/717702567 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e0003530 con 0x7f98e0004d10 2024-05-22T21:25:15.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.786+0000 7f9900a00640 1 --2- 172.21.15.161:0/717702567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98e0004d10 0x7f98e0005190 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009fc0 tx=0x7f98f401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95191bdbf2860613 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:15.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.786+0000 7f98fb400640 1 -- 172.21.15.161:0/717702567 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f40a5cd0 con 0x7f98e0004d10 2024-05-22T21:25:15.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.786+0000 7f98fb400640 1 -- 172.21.15.161:0/717702567 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f400f040 con 0x7f98e0004d10 2024-05-22T21:25:15.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.786+0000 7f98fb400640 1 -- 172.21.15.161:0/717702567 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f40ad680 con 0x7f98e0004d10 2024-05-22T21:25:15.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9901400640 1 -- 172.21.15.161:0/717702567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98e0004d10 msgr2=0x7f98e0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9901400640 1 --2- 172.21.15.161:0/717702567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98e0004d10 0x7f98e0005190 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009fc0 tx=0x7f98f401cc60 comp rx=0 tx=0).stop 2024-05-22T21:25:15.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9900a00640 1 -- 172.21.15.161:0/717702567 reap_dead start 2024-05-22T21:25:15.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9901400640 1 -- 172.21.15.161:0/717702567 shutdown_connections 2024-05-22T21:25:15.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9901400640 1 -- 172.21.15.161:0/717702567 >> 172.21.15.161:0/717702567 conn(0x7f98fc0cfc20 msgr2=0x7f98fc0d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:15.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9901400640 1 -- 172.21.15.161:0/717702567 shutdown_connections 2024-05-22T21:25:15.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9901400640 1 -- 172.21.15.161:0/717702567 wait complete. 2024-05-22T21:25:15.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9901400640 1 Processor -- start 2024-05-22T21:25:15.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9901400640 1 -- start start 2024-05-22T21:25:15.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9901400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98e0004d10 0x7f98fc142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9901400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98fc1593a0 0x7f98fc159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9901400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98fc15b770 0x7f98fc15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9901400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc159d60 con 0x7f98e0004d10 2024-05-22T21:25:15.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9901400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc15eba0 con 0x7f98fc15b770 2024-05-22T21:25:15.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9901400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc15dfd0 con 0x7f98fc1593a0 2024-05-22T21:25:15.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f98fbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98e0004d10 0x7f98fc142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f98fbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98e0004d10 0x7f98fc142740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:46980/0 (socket says 172.21.15.161:46980) 2024-05-22T21:25:15.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f98fbe00640 1 -- 172.21.15.161:0/3948530544 learned_addr learned my addr 172.21.15.161:0/3948530544 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:15.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f98f3400640 1 --2- 172.21.15.161:0/3948530544 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98fc1593a0 0x7f98fc159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9900a00640 1 --2- 172.21.15.161:0/3948530544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98fc15b770 0x7f98fc15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f98fbe00640 1 -- 172.21.15.161:0/3948530544 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98fc1593a0 msgr2=0x7f98fc159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f98fbe00640 1 --2- 172.21.15.161:0/3948530544 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98fc1593a0 0x7f98fc159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f98fbe00640 1 -- 172.21.15.161:0/3948530544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98fc15b770 msgr2=0x7f98fc15bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f98fbe00640 1 --2- 172.21.15.161:0/3948530544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98fc15b770 0x7f98fc15bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f98fbe00640 1 -- 172.21.15.161:0/3948530544 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc15e270 con 0x7f98e0004d10 2024-05-22T21:25:15.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9900a00640 1 --2- 172.21.15.161:0/3948530544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98fc15b770 0x7f98fc15bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:25:15.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f98fbe00640 1 --2- 172.21.15.161:0/3948530544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98e0004d10 0x7f98fc142740 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0119d0 tx=0x7f98ec011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:15.794 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f98f8c00640 1 -- 172.21.15.161:0/3948530544 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec00fb50 con 0x7f98e0004d10 2024-05-22T21:25:15.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f98f8c00640 1 -- 172.21.15.161:0/3948530544 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec00fcb0 con 0x7f98e0004d10 2024-05-22T21:25:15.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.790+0000 7f9901400640 1 -- 172.21.15.161:0/3948530544 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98fc15e550 con 0x7f98e0004d10 2024-05-22T21:25:15.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.794+0000 7f98f8c00640 1 -- 172.21.15.161:0/3948530544 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec014440 con 0x7f98e0004d10 2024-05-22T21:25:15.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.794+0000 7f9901400640 1 -- 172.21.15.161:0/3948530544 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc142c80 con 0x7f98e0004d10 2024-05-22T21:25:15.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.794+0000 7f9901400640 1 -- 172.21.15.161:0/3948530544 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98fc067070 con 0x7f98e0004d10 2024-05-22T21:25:15.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.794+0000 7f98f8c00640 1 -- 172.21.15.161:0/3948530544 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f98ec0145a0 con 0x7f98e0004d10 2024-05-22T21:25:15.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.794+0000 7f98f8c00640 1 --2- 172.21.15.161:0/3948530544 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f98d80666c0 0x7f98d8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:15.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.794+0000 7f98f8c00640 1 -- 172.21.15.161:0/3948530544 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98ec084ac0 con 0x7f98e0004d10 2024-05-22T21:25:15.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.794+0000 7f98f8c00640 1 -- 172.21.15.161:0/3948530544 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=195}) v3 -- 0x7f98d806e1f0 con 0x7f98e0004d10 2024-05-22T21:25:15.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.794+0000 7f98f3400640 1 --2- 172.21.15.161:0/3948530544 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f98d80666c0 0x7f98d8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:15.796 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.794+0000 7f98f3400640 1 --2- 172.21.15.161:0/3948530544 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f98d80666c0 0x7f98d8068b80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0cd680 tx=0x7f98e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:15.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.798+0000 7f98f8c00640 1 -- 172.21.15.161:0/3948530544 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f98ec051e50 con 0x7f98e0004d10 2024-05-22T21:25:15.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.986+0000 7f9901400640 1 -- 172.21.15.161:0/3948530544 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98fc0cebe0 con 0x7f98e0004d10 2024-05-22T21:25:15.989 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.990+0000 7f98f8c00640 1 -- 172.21.15.161:0/3948530544 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4744 (secure 0 0 0) 0x7f98ec055470 con 0x7f98e0004d10 2024-05-22T21:25:15.989 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:15.989 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":86,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3304182060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3304182060},{"type":"v1","addr":"172.21.15.179:6835","nonce":3304182060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196309272},{"type":"v1","addr":"172.21.15.161:6836","nonce":196309272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/396408969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":396408969},{"type":"v1","addr":"172.21.15.179:6834","nonce":396408969}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-22T21:24:34.096673+0000","modified":"2024-05-22T21:24:37.112845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8784},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8784":{"gid":8784,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2172511349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":2172511349},{"type":"v1","addr":"172.21.15.161:6837","nonce":2172511349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-05-22T21:25:15.990 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 86 2024-05-22T21:25:15.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.990+0000 7f9901400640 1 -- 172.21.15.161:0/3948530544 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f98d80666c0 msgr2=0x7f98d8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.990+0000 7f9901400640 1 --2- 172.21.15.161:0/3948530544 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f98d80666c0 0x7f98d8068b80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0cd680 tx=0x7f98e4009040 comp rx=0 tx=0).stop 2024-05-22T21:25:15.991 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.990+0000 7f9901400640 1 -- 172.21.15.161:0/3948530544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98e0004d10 msgr2=0x7f98fc142740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:15.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.990+0000 7f9901400640 1 --2- 172.21.15.161:0/3948530544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98e0004d10 0x7f98fc142740 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0119d0 tx=0x7f98ec011e90 comp rx=0 tx=0).stop 2024-05-22T21:25:15.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.990+0000 7f9901400640 1 -- 172.21.15.161:0/3948530544 shutdown_connections 2024-05-22T21:25:15.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.990+0000 7f9901400640 1 --2- 172.21.15.161:0/3948530544 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f98d80666c0 0x7f98d8068b80 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.990+0000 7f9901400640 1 --2- 172.21.15.161:0/3948530544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98fc15b770 0x7f98fc15bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.990+0000 7f9901400640 1 --2- 172.21.15.161:0/3948530544 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98fc1593a0 0x7f98fc159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.990+0000 7f9901400640 1 --2- 172.21.15.161:0/3948530544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98e0004d10 0x7f98fc142740 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:15.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.990+0000 7f9901400640 1 -- 172.21.15.161:0/3948530544 >> 172.21.15.161:0/3948530544 conn(0x7f98fc0cfc20 msgr2=0x7f98fc0ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:15.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.990+0000 7f9901400640 1 -- 172.21.15.161:0/3948530544 shutdown_connections 2024-05-22T21:25:15.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:15.990+0000 7f9901400640 1 -- 172.21.15.161:0/3948530544 wait complete. 2024-05-22T21:25:16.010 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:25:16.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb26ec00640 1 Processor -- start 2024-05-22T21:25:16.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb26ec00640 1 -- start start 2024-05-22T21:25:16.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb26ec00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb26815f3a0 0x7fb26815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb26ec00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2681442a0 con 0x7fb2680715a0 2024-05-22T21:25:16.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb26ec00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb268070500 con 0x7fb268143ec0 2024-05-22T21:25:16.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb26ec00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2680707e0 con 0x7fb26815f3a0 2024-05-22T21:25:16.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb25f400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb2680715a0 0x7fb26815ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59462/0 (socket says 172.21.15.161:59462) 2024-05-22T21:25:16.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb25f400640 1 -- 172.21.15.161:0/567519750 learned_addr learned my addr 172.21.15.161:0/567519750 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:16.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb267400640 1 -- 172.21.15.161:0/567519750 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203997559 0 0) 0x7fb2681442a0 con 0x7fb2680715a0 2024-05-22T21:25:16.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb26cc00640 1 --2- 172.21.15.161:0/567519750 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb26815f3a0 0x7fb26815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb267400640 1 -- 172.21.15.161:0/567519750 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb24c003530 con 0x7fb2680715a0 2024-05-22T21:25:16.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb267400640 1 -- 172.21.15.161:0/567519750 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2908062631 0 0) 0x7fb24c003530 con 0x7fb2680715a0 2024-05-22T21:25:16.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb267400640 1 -- 172.21.15.161:0/567519750 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb26815f3a0 msgr2=0x7fb26815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:16.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb267400640 1 --2- 172.21.15.161:0/567519750 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb26815f3a0 0x7fb26815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb267400640 1 -- 172.21.15.161:0/567519750 >> v1:172.21.15.179:6789/0 conn(0x7fb268143ec0 legacy=0x7fb26806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb267400640 1 -- 172.21.15.161:0/567519750 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb268070ac0 con 0x7fb2680715a0 2024-05-22T21:25:16.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb267400640 1 -- 172.21.15.161:0/567519750 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7fb250001680 con 0x7fb2680715a0 2024-05-22T21:25:16.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb267400640 1 -- 172.21.15.161:0/567519750 >> v1:172.21.15.161:6789/0 conn(0x7fb2680715a0 legacy=0x7fb26815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb267400640 1 --2- 172.21.15.161:0/567519750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb24c0039b0 0x7fb24c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb267400640 1 --2- 172.21.15.161:0/567519750 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb26815f3a0 0x7fb24c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb267400640 1 --2- 172.21.15.161:0/567519750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb24c004d10 0x7fb24c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb267400640 1 -- 172.21.15.161:0/567519750 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2680707e0 con 0x7fb24c0039b0 2024-05-22T21:25:16.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb267400640 1 -- 172.21.15.161:0/567519750 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c00b9c0 con 0x7fb24c004d10 2024-05-22T21:25:16.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb267400640 1 -- 172.21.15.161:0/567519750 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c00bb80 con 0x7fb26815f3a0 2024-05-22T21:25:16.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb267e00640 1 --2- 172.21.15.161:0/567519750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb24c0039b0 0x7fb24c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb25f400640 1 --2- 172.21.15.161:0/567519750 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb26815f3a0 0x7fb24c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.146+0000 7fb26cc00640 1 --2- 172.21.15.161:0/567519750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb24c004d10 0x7fb24c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb267e00640 1 -- 172.21.15.161:0/567519750 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb26815f3a0 msgr2=0x7fb24c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:16.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb267e00640 1 --2- 172.21.15.161:0/567519750 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb26815f3a0 0x7fb24c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb267e00640 1 -- 172.21.15.161:0/567519750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb24c004d10 msgr2=0x7fb24c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:16.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb267e00640 1 --2- 172.21.15.161:0/567519750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb24c004d10 0x7fb24c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb267e00640 1 -- 172.21.15.161:0/567519750 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c003530 con 0x7fb24c0039b0 2024-05-22T21:25:16.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb26cc00640 1 --2- 172.21.15.161:0/567519750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb24c004d10 0x7fb24c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:16.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb267e00640 1 --2- 172.21.15.161:0/567519750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb24c0039b0 0x7fb24c003d90 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fb258001f60 tx=0x7fb25801c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7540bab5044257d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:16.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb267400640 1 -- 172.21.15.161:0/567519750 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2580b0600 con 0x7fb24c0039b0 2024-05-22T21:25:16.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb267400640 1 -- 172.21.15.161:0/567519750 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2580b0760 con 0x7fb24c0039b0 2024-05-22T21:25:16.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb267400640 1 -- 172.21.15.161:0/567519750 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2580b0b10 con 0x7fb24c0039b0 2024-05-22T21:25:16.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb26ec00640 1 -- 172.21.15.161:0/567519750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb24c0039b0 msgr2=0x7fb24c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb26ec00640 1 --2- 172.21.15.161:0/567519750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb24c0039b0 0x7fb24c003d90 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fb258001f60 tx=0x7fb25801c7b0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb26cc00640 1 -- 172.21.15.161:0/567519750 reap_dead start 2024-05-22T21:25:16.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb26ec00640 1 -- 172.21.15.161:0/567519750 shutdown_connections 2024-05-22T21:25:16.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb26ec00640 1 -- 172.21.15.161:0/567519750 >> 172.21.15.161:0/567519750 conn(0x7fb2680b0bc0 msgr2=0x7fb2680b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:16.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb26ec00640 1 -- 172.21.15.161:0/567519750 shutdown_connections 2024-05-22T21:25:16.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.150+0000 7fb26ec00640 1 -- 172.21.15.161:0/567519750 wait complete. 2024-05-22T21:25:16.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb26ec00640 1 Processor -- start 2024-05-22T21:25:16.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb26ec00640 1 -- start start 2024-05-22T21:25:16.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb26ec00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb24c0039b0 0x7fb268136070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb26ec00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb26813e9a0 0x7fb26815db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb26ec00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb26813a810 0x7fb26813ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb26ec00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb268139770 con 0x7fb26813e9a0 2024-05-22T21:25:16.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb26ec00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb268139a50 con 0x7fb26813a810 2024-05-22T21:25:16.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb26ec00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb268139d30 con 0x7fb24c0039b0 2024-05-22T21:25:16.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb25f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb26813e9a0 0x7fb26815db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb25f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb26813e9a0 0x7fb26815db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59132/0 (socket says 172.21.15.161:59132) 2024-05-22T21:25:16.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb25f400640 1 -- 172.21.15.161:0/4235525374 learned_addr learned my addr 172.21.15.161:0/4235525374 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:16.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb267e00640 1 --2- 172.21.15.161:0/4235525374 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb24c0039b0 0x7fb268136070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb26cc00640 1 --2- 172.21.15.161:0/4235525374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb26813a810 0x7fb26813ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb25f400640 1 -- 172.21.15.161:0/4235525374 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb24c0039b0 msgr2=0x7fb268136070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb25f400640 1 --2- 172.21.15.161:0/4235525374 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb24c0039b0 0x7fb268136070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb25f400640 1 -- 172.21.15.161:0/4235525374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb26813a810 msgr2=0x7fb26813ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb25f400640 1 --2- 172.21.15.161:0/4235525374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb26813a810 0x7fb26813ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb25f400640 1 -- 172.21.15.161:0/4235525374 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26813a010 con 0x7fb26813e9a0 2024-05-22T21:25:16.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb26cc00640 1 --2- 172.21.15.161:0/4235525374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb26813a810 0x7fb26813ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:25:16.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb25f400640 1 --2- 172.21.15.161:0/4235525374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb26813e9a0 0x7fb26815db20 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fb2500072c0 tx=0x7fb250007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:16.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb264c00640 1 -- 172.21.15.161:0/4235525374 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25000f9c0 con 0x7fb26813e9a0 2024-05-22T21:25:16.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb264c00640 1 -- 172.21.15.161:0/4235525374 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25000fb40 con 0x7fb26813e9a0 2024-05-22T21:25:16.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb264c00640 1 -- 172.21.15.161:0/4235525374 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2500133f0 con 0x7fb26813e9a0 2024-05-22T21:25:16.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb26ec00640 1 -- 172.21.15.161:0/4235525374 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26813a2f0 con 0x7fb26813e9a0 2024-05-22T21:25:16.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.154+0000 7fb26ec00640 1 -- 172.21.15.161:0/4235525374 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26813b1d0 con 0x7fb26813e9a0 2024-05-22T21:25:16.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.158+0000 7fb26ec00640 1 -- 172.21.15.161:0/4235525374 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb268067220 con 0x7fb26813e9a0 2024-05-22T21:25:16.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.158+0000 7fb264c00640 1 -- 172.21.15.161:0/4235525374 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb250013550 con 0x7fb26813e9a0 2024-05-22T21:25:16.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.158+0000 7fb264c00640 1 --2- 172.21.15.161:0/4235525374 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb244066690 0x7fb244068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.158+0000 7fb264c00640 1 -- 172.21.15.161:0/4235525374 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb250084480 con 0x7fb26813e9a0 2024-05-22T21:25:16.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.158+0000 7fb264c00640 1 -- 172.21.15.161:0/4235525374 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=195}) v3 -- 0x7fb24406e1c0 con 0x7fb26813e9a0 2024-05-22T21:25:16.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.158+0000 7fb267e00640 1 --2- 172.21.15.161:0/4235525374 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb244066690 0x7fb244068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.158+0000 7fb267e00640 1 --2- 172.21.15.161:0/4235525374 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb244066690 0x7fb244068b50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb2580af000 tx=0x7fb2580a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:16.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.162+0000 7fb264c00640 1 -- 172.21.15.161:0/4235525374 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb250051750 con 0x7fb26813e9a0 2024-05-22T21:25:16.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.318+0000 7fb26ec00640 1 -- 172.21.15.161:0/4235525374 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb268139b90 con 0x7fb26813e9a0 2024-05-22T21:25:16.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.318+0000 7fb264c00640 1 -- 172.21.15.161:0/4235525374 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) v1 ==== 74+0+16178 (secure 0 0 0) 0x7fb268139b90 con 0x7fb26813e9a0 2024-05-22T21:25:16.321 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:16.321 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":194,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:25:15.316320+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-05-22T21:24:31.797440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-05-22T21:24:33.098240+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:25:16.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.322+0000 7fb26ec00640 1 -- 172.21.15.161:0/4235525374 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb244066690 msgr2=0x7fb244068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.322+0000 7fb26ec00640 1 --2- 172.21.15.161:0/4235525374 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb244066690 0x7fb244068b50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb2580af000 tx=0x7fb2580a7000 comp rx=0 tx=0).stop 2024-05-22T21:25:16.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.322+0000 7fb26ec00640 1 -- 172.21.15.161:0/4235525374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb26813e9a0 msgr2=0x7fb26815db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.322+0000 7fb26ec00640 1 --2- 172.21.15.161:0/4235525374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb26813e9a0 0x7fb26815db20 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fb2500072c0 tx=0x7fb250007780 comp rx=0 tx=0).stop 2024-05-22T21:25:16.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.322+0000 7fb26ec00640 1 -- 172.21.15.161:0/4235525374 shutdown_connections 2024-05-22T21:25:16.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.322+0000 7fb26ec00640 1 --2- 172.21.15.161:0/4235525374 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb244066690 0x7fb244068b50 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.322+0000 7fb26ec00640 1 --2- 172.21.15.161:0/4235525374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb26813a810 0x7fb26813ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.322+0000 7fb26ec00640 1 --2- 172.21.15.161:0/4235525374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb26813e9a0 0x7fb26815db20 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.322+0000 7fb26ec00640 1 --2- 172.21.15.161:0/4235525374 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb24c0039b0 0x7fb268136070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.323 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.322+0000 7fb26ec00640 1 -- 172.21.15.161:0/4235525374 >> 172.21.15.161:0/4235525374 conn(0x7fb2680b0bc0 msgr2=0x7fb26806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:16.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.322+0000 7fb26ec00640 1 -- 172.21.15.161:0/4235525374 shutdown_connections 2024-05-22T21:25:16.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.322+0000 7fb26ec00640 1 -- 172.21.15.161:0/4235525374 wait complete. 2024-05-22T21:25:16.342 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-22T21:25:16.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.478+0000 7f8bb9c00640 1 Processor -- start 2024-05-22T21:25:16.479 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.478+0000 7f8bb9c00640 1 -- start start 2024-05-22T21:25:16.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.478+0000 7f8bb9c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8bb4143c40 0x7f8bb415b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.478+0000 7f8bb9c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb415efb0 con 0x7f8bb41444a0 2024-05-22T21:25:16.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.478+0000 7f8bb9c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb415f290 con 0x7f8bb406c9b0 2024-05-22T21:25:16.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.478+0000 7f8bb9c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb415f570 con 0x7f8bb4143c40 2024-05-22T21:25:16.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.478+0000 7f8bb2000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8bb41444a0 0x7f8bb406c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59466/0 (socket says 172.21.15.161:59466) 2024-05-22T21:25:16.481 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.478+0000 7f8bb2000640 1 -- 172.21.15.161:0/520921904 learned_addr learned my addr 172.21.15.161:0/520921904 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:16.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.478+0000 7f8bb1600640 1 -- 172.21.15.161:0/520921904 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2993100155 0 0) 0x7f8bb415efb0 con 0x7f8bb41444a0 2024-05-22T21:25:16.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.478+0000 7f8bb2a00640 1 --2- 172.21.15.161:0/520921904 >> v2:172.21.15.179:3300/0 conn(0x7f8bb4143c40 0x7f8bb415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.478+0000 7f8bb1600640 1 -- 172.21.15.161:0/520921904 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b98003530 con 0x7f8bb41444a0 2024-05-22T21:25:16.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb1600640 1 -- 172.21.15.161:0/520921904 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 140547887 0 0) 0x7f8b98003530 con 0x7f8bb41444a0 2024-05-22T21:25:16.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb1600640 1 -- 172.21.15.161:0/520921904 >> v2:172.21.15.179:3300/0 conn(0x7f8bb4143c40 msgr2=0x7f8bb415b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:16.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb1600640 1 --2- 172.21.15.161:0/520921904 >> v2:172.21.15.179:3300/0 conn(0x7f8bb4143c40 0x7f8bb415b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb1600640 1 -- 172.21.15.161:0/520921904 >> v1:172.21.15.179:6789/0 conn(0x7f8bb406c9b0 legacy=0x7f8bb40b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb1600640 1 -- 172.21.15.161:0/520921904 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb415f850 con 0x7f8bb41444a0 2024-05-22T21:25:16.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb1600640 1 -- 172.21.15.161:0/520921904 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f8ba0001680 con 0x7f8bb41444a0 2024-05-22T21:25:16.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb1600640 1 -- 172.21.15.161:0/520921904 >> v1:172.21.15.161:6789/0 conn(0x7f8bb41444a0 legacy=0x7f8bb406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb1600640 1 --2- 172.21.15.161:0/520921904 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b98003980 0x7f8b98003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.483 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb1600640 1 --2- 172.21.15.161:0/520921904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b980047d0 0x7f8b98004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb1600640 1 --2- 172.21.15.161:0/520921904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b98005190 0x7f8b9800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb1600640 1 -- 172.21.15.161:0/520921904 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb415f570 con 0x7f8b980047d0 2024-05-22T21:25:16.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb1600640 1 -- 172.21.15.161:0/520921904 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9800c030 con 0x7f8b98005190 2024-05-22T21:25:16.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb1600640 1 -- 172.21.15.161:0/520921904 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9800c1f0 con 0x7f8b98003980 2024-05-22T21:25:16.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb2000640 1 --2- 172.21.15.161:0/520921904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b980047d0 0x7f8b98004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb2a00640 1 --2- 172.21.15.161:0/520921904 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b98003980 0x7f8b98003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb3400640 1 --2- 172.21.15.161:0/520921904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b98005190 0x7f8b9800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb2000640 1 -- 172.21.15.161:0/520921904 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b98003980 msgr2=0x7f8b98003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:16.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb2000640 1 --2- 172.21.15.161:0/520921904 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b98003980 0x7f8b98003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb2000640 1 -- 172.21.15.161:0/520921904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b98005190 msgr2=0x7f8b9800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:16.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb2000640 1 --2- 172.21.15.161:0/520921904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b98005190 0x7f8b9800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb2000640 1 -- 172.21.15.161:0/520921904 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b98003530 con 0x7f8b980047d0 2024-05-22T21:25:16.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb3400640 1 --2- 172.21.15.161:0/520921904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b98005190 0x7f8b9800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:16.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb3400640 1 -- 172.21.15.161:0/520921904 reap_dead start 2024-05-22T21:25:16.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.482+0000 7f8bb2000640 1 --2- 172.21.15.161:0/520921904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b980047d0 0x7f8b98004c50 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000c240 tx=0x7f8ba001f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a461a3737ae00fb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:16.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb1600640 1 -- 172.21.15.161:0/520921904 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0005ae0 con 0x7f8b980047d0 2024-05-22T21:25:16.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb1600640 1 -- 172.21.15.161:0/520921904 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba00ab070 con 0x7f8b980047d0 2024-05-22T21:25:16.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb1600640 1 -- 172.21.15.161:0/520921904 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0007070 con 0x7f8b980047d0 2024-05-22T21:25:16.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb9c00640 1 -- 172.21.15.161:0/520921904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b980047d0 msgr2=0x7f8b98004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb9c00640 1 --2- 172.21.15.161:0/520921904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b980047d0 0x7f8b98004c50 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000c240 tx=0x7f8ba001f3d0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb9c00640 1 -- 172.21.15.161:0/520921904 shutdown_connections 2024-05-22T21:25:16.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb9c00640 1 --2- 172.21.15.161:0/520921904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b980047d0 0x7f8b98004c50 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb9c00640 1 -- 172.21.15.161:0/520921904 >> 172.21.15.161:0/520921904 conn(0x7f8bb406a2d0 msgr2=0x7f8bb406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:16.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb9c00640 1 -- 172.21.15.161:0/520921904 shutdown_connections 2024-05-22T21:25:16.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb9c00640 1 -- 172.21.15.161:0/520921904 wait complete. 2024-05-22T21:25:16.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb9c00640 1 Processor -- start 2024-05-22T21:25:16.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb9c00640 1 -- start start 2024-05-22T21:25:16.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb9c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b980047d0 0x7f8bb41430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb9c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb41399e0 0x7f8bb4138160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb9c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8bb4141310 0x7f8bb4137510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb9c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb4139550 con 0x7f8bb41399e0 2024-05-22T21:25:16.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb9c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb4139e60 con 0x7f8b980047d0 2024-05-22T21:25:16.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb9c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb41386a0 con 0x7f8bb4141310 2024-05-22T21:25:16.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb41399e0 0x7f8bb4138160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb2000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb41399e0 0x7f8bb4138160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59150/0 (socket says 172.21.15.161:59150) 2024-05-22T21:25:16.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb2000640 1 -- 172.21.15.161:0/1667070802 learned_addr learned my addr 172.21.15.161:0/1667070802 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:16.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb3400640 1 --2- 172.21.15.161:0/1667070802 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8bb4141310 0x7f8bb4137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb2a00640 1 --2- 172.21.15.161:0/1667070802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b980047d0 0x7f8bb41430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb2000640 1 -- 172.21.15.161:0/1667070802 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8bb4141310 msgr2=0x7f8bb4137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb2000640 1 --2- 172.21.15.161:0/1667070802 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8bb4141310 0x7f8bb4137510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb2000640 1 -- 172.21.15.161:0/1667070802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b980047d0 msgr2=0x7f8bb41430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb2000640 1 --2- 172.21.15.161:0/1667070802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b980047d0 0x7f8bb41430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.486+0000 7f8bb2000640 1 -- 172.21.15.161:0/1667070802 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb4138940 con 0x7f8bb41399e0 2024-05-22T21:25:16.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.490+0000 7f8bb2000640 1 --2- 172.21.15.161:0/1667070802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb41399e0 0x7f8bb4138160 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f8ba001f200 tx=0x7f8b98005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:16.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.490+0000 7f8baaa00640 1 -- 172.21.15.161:0/1667070802 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0004af0 con 0x7f8bb41399e0 2024-05-22T21:25:16.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.490+0000 7f8baaa00640 1 -- 172.21.15.161:0/1667070802 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba0004c50 con 0x7f8bb41399e0 2024-05-22T21:25:16.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.490+0000 7f8bb9c00640 1 -- 172.21.15.161:0/1667070802 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb4138c20 con 0x7f8bb41399e0 2024-05-22T21:25:16.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.490+0000 7f8baaa00640 1 -- 172.21.15.161:0/1667070802 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0007070 con 0x7f8bb41399e0 2024-05-22T21:25:16.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.490+0000 7f8bb9c00640 1 -- 172.21.15.161:0/1667070802 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb41391e0 con 0x7f8bb41399e0 2024-05-22T21:25:16.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.490+0000 7f8bb9c00640 1 -- 172.21.15.161:0/1667070802 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb415ebc0 con 0x7f8bb41399e0 2024-05-22T21:25:16.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.490+0000 7f8baaa00640 1 -- 172.21.15.161:0/1667070802 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8ba00b9030 con 0x7f8bb41399e0 2024-05-22T21:25:16.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.490+0000 7f8baaa00640 1 --2- 172.21.15.161:0/1667070802 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8b940666c0 0x7f8b94068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.492 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.490+0000 7f8baaa00640 1 -- 172.21.15.161:0/1667070802 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ba00b7070 con 0x7f8bb41399e0 2024-05-22T21:25:16.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.490+0000 7f8bb2a00640 1 --2- 172.21.15.161:0/1667070802 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8b940666c0 0x7f8b94068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.490+0000 7f8baaa00640 1 -- 172.21.15.161:0/1667070802 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=195}) v3 -- 0x7f8b9406e1f0 con 0x7f8bb41399e0 2024-05-22T21:25:16.493 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.494+0000 7f8bb2a00640 1 --2- 172.21.15.161:0/1667070802 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8b940666c0 0x7f8b94068b80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4063da0 tx=0x7f8ba400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:16.496 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.494+0000 7f8baaa00640 1 -- 172.21.15.161:0/1667070802 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f8ba00c1020 con 0x7f8bb41399e0 2024-05-22T21:25:16.685 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.682+0000 7f8bb9c00640 1 -- 172.21.15.161:0/1667070802 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8bb4067220 con 0x7f8bb41399e0 2024-05-22T21:25:16.685 INFO:tasks.ceph.mon.a.smithi161.stderr:2024-05-22T21:25:16.686+0000 7f7d32800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-22T21:25:16.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.686+0000 7f8baaa00640 1 -- 172.21.15.161:0/1667070802 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(195..195 src has 1..195) v4 ==== 374+0+0 (secure 0 0 0) 0x7f8ba00f23c0 con 0x7f8bb41399e0 2024-05-22T21:25:16.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.686+0000 7f8baaa00640 1 -- 172.21.15.161:0/1667070802 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=196}) v3 -- 0x7f8b9406ea30 con 0x7f8bb41399e0 2024-05-22T21:25:16.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.694+0000 7f8baaa00640 1 -- 172.21.15.161:0/1667070802 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v87) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8ba00f2190 con 0x7f8bb41399e0 2024-05-22T21:25:16.695 INFO:teuthology.orchestra.run.smithi161.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-22T21:25:16.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.694+0000 7f8bb9c00640 1 -- 172.21.15.161:0/1667070802 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8b940666c0 msgr2=0x7f8b94068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.694+0000 7f8bb9c00640 1 --2- 172.21.15.161:0/1667070802 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8b940666c0 0x7f8b94068b80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4063da0 tx=0x7f8ba400b040 comp rx=0 tx=0).stop 2024-05-22T21:25:16.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.694+0000 7f8bb9c00640 1 -- 172.21.15.161:0/1667070802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb41399e0 msgr2=0x7f8bb4138160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.694+0000 7f8bb9c00640 1 --2- 172.21.15.161:0/1667070802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb41399e0 0x7f8bb4138160 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f8ba001f200 tx=0x7f8b98005800 comp rx=0 tx=0).stop 2024-05-22T21:25:16.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.694+0000 7f8bb9c00640 1 -- 172.21.15.161:0/1667070802 shutdown_connections 2024-05-22T21:25:16.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.694+0000 7f8bb9c00640 1 --2- 172.21.15.161:0/1667070802 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f8b940666c0 0x7f8b94068b80 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.694+0000 7f8bb9c00640 1 --2- 172.21.15.161:0/1667070802 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8bb4141310 0x7f8bb4137510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.694+0000 7f8bb9c00640 1 --2- 172.21.15.161:0/1667070802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb41399e0 0x7f8bb4138160 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.694+0000 7f8bb9c00640 1 --2- 172.21.15.161:0/1667070802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b980047d0 0x7f8bb41430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.694+0000 7f8bb9c00640 1 -- 172.21.15.161:0/1667070802 >> 172.21.15.161:0/1667070802 conn(0x7f8bb406a2d0 msgr2=0x7f8bb40cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:16.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.698+0000 7f8bb9c00640 1 -- 172.21.15.161:0/1667070802 shutdown_connections 2024-05-22T21:25:16.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.698+0000 7f8bb9c00640 1 -- 172.21.15.161:0/1667070802 wait complete. 2024-05-22T21:25:16.706 INFO:tasks.ceph.mds.c.smithi161.stderr: -9091> 2024-05-22T21:24:39.707+0000 7f9728400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-22T21:25:16.718 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-22T21:25:16.759 INFO:tasks.ceph.mds.c.smithi161.stdout:starting mds.c at 2024-05-22T21:25:16.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.858+0000 7f3d9c200640 1 Processor -- start 2024-05-22T21:25:16.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.858+0000 7f3d9c200640 1 -- start start 2024-05-22T21:25:16.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.858+0000 7f3d9c200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9413cfc0 0x7f3d941402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.858+0000 7f3d9c200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3d9413fe50 0x7f3d9406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.858+0000 7f3d9c200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d9413d3a0 con 0x7f3d9415af60 2024-05-22T21:25:16.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.858+0000 7f3d9c200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d940cd920 con 0x7f3d9413fe50 2024-05-22T21:25:16.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.858+0000 7f3d9c200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d940cdc00 con 0x7f3d9413cfc0 2024-05-22T21:25:16.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.858+0000 7f3d9a200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3d9413fe50 0x7f3d9406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.858+0000 7f3d9a200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3d9413fe50 0x7f3d9406fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59178/0 (socket says 172.21.15.161:59178) 2024-05-22T21:25:16.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.858+0000 7f3d9a200640 1 -- 172.21.15.161:0/4266595790 learned_addr learned my addr 172.21.15.161:0/4266595790 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:16.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.858+0000 7f3d99800640 1 --2- 172.21.15.161:0/4266595790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9413cfc0 0x7f3d941402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.858+0000 7f3d9a200640 1 -- 172.21.15.161:0/4266595790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9413cfc0 msgr2=0x7f3d941402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:16.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.858+0000 7f3d9a200640 1 --2- 172.21.15.161:0/4266595790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9413cfc0 0x7f3d941402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.858+0000 7f3d9a200640 1 -- 172.21.15.161:0/4266595790 >> v1:172.21.15.179:6789/0 conn(0x7f3d9415af60 legacy=0x7f3d9413f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:16.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.862+0000 7f3d9a200640 1 -- 172.21.15.161:0/4266595790 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d940cdee0 con 0x7f3d9413fe50 2024-05-22T21:25:16.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.862+0000 7f3d9a200640 1 --2- 172.21.15.161:0/4266595790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d9413fe50 0x7f3d9406fce0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c009870 tx=0x7f3d8c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd17cab275a3bb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:16.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.862+0000 7f3d93e00640 1 -- 172.21.15.161:0/4266595790 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c0ab600 con 0x7f3d9413fe50 2024-05-22T21:25:16.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.862+0000 7f3d93e00640 1 -- 172.21.15.161:0/4266595790 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8c0a7050 con 0x7f3d9413fe50 2024-05-22T21:25:16.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.862+0000 7f3d93e00640 1 -- 172.21.15.161:0/4266595790 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c0ab980 con 0x7f3d9413fe50 2024-05-22T21:25:16.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.862+0000 7f3d9c200640 1 -- 172.21.15.161:0/4266595790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d9413fe50 msgr2=0x7f3d9406fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.862+0000 7f3d9c200640 1 --2- 172.21.15.161:0/4266595790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d9413fe50 0x7f3d9406fce0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c009870 tx=0x7f3d8c01c4a0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.862+0000 7f3d9c200640 1 -- 172.21.15.161:0/4266595790 shutdown_connections 2024-05-22T21:25:16.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.862+0000 7f3d9c200640 1 --2- 172.21.15.161:0/4266595790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d9413fe50 0x7f3d9406fce0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.862+0000 7f3d9c200640 1 --2- 172.21.15.161:0/4266595790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9413cfc0 0x7f3d941402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.862+0000 7f3d9c200640 1 -- 172.21.15.161:0/4266595790 >> 172.21.15.161:0/4266595790 conn(0x7f3d9406a2d0 msgr2=0x7f3d9406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:16.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.862+0000 7f3d9c200640 1 -- 172.21.15.161:0/4266595790 shutdown_connections 2024-05-22T21:25:16.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.862+0000 7f3d9c200640 1 -- 172.21.15.161:0/4266595790 wait complete. 2024-05-22T21:25:16.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d9c200640 1 Processor -- start 2024-05-22T21:25:16.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d9c200640 1 -- start start 2024-05-22T21:25:16.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d9c200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d9413cfc0 0x7f3d9413c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d9c200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d9413fe50 0x7f3d9413e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d9c200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9415af60 0x7f3d9413eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d9c200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9406d6a0 con 0x7f3d9413fe50 2024-05-22T21:25:16.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d9c200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d940a9aa0 con 0x7f3d9413cfc0 2024-05-22T21:25:16.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d9c200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d940a8eb0 con 0x7f3d9415af60 2024-05-22T21:25:16.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d98e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d9413fe50 0x7f3d9413e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d99800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d9413cfc0 0x7f3d9413c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d98e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d9413fe50 0x7f3d9413e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59182/0 (socket says 172.21.15.161:59182) 2024-05-22T21:25:16.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d98e00640 1 -- 172.21.15.161:0/1050628838 learned_addr learned my addr 172.21.15.161:0/1050628838 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:16.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d9a200640 1 --2- 172.21.15.161:0/1050628838 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9415af60 0x7f3d9413eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d98e00640 1 -- 172.21.15.161:0/1050628838 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9415af60 msgr2=0x7f3d9413eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d98e00640 1 --2- 172.21.15.161:0/1050628838 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9415af60 0x7f3d9413eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d98e00640 1 -- 172.21.15.161:0/1050628838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d9413cfc0 msgr2=0x7f3d9413c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:16.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d98e00640 1 --2- 172.21.15.161:0/1050628838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d9413cfc0 0x7f3d9413c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:16.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d98e00640 1 -- 172.21.15.161:0/1050628838 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c0094d0 con 0x7f3d9413fe50 2024-05-22T21:25:16.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d98e00640 1 --2- 172.21.15.161:0/1050628838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d9413fe50 0x7f3d9413e5e0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000c3c0 tx=0x7f3d8000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:16.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d91600640 1 -- 172.21.15.161:0/1050628838 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d800052d0 con 0x7f3d9413fe50 2024-05-22T21:25:16.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d91600640 1 -- 172.21.15.161:0/1050628838 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d80005430 con 0x7f3d9413fe50 2024-05-22T21:25:16.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d9c200640 1 -- 172.21.15.161:0/1050628838 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d940a9100 con 0x7f3d9413fe50 2024-05-22T21:25:16.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d91600640 1 -- 172.21.15.161:0/1050628838 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80003840 con 0x7f3d9413fe50 2024-05-22T21:25:16.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.866+0000 7f3d9c200640 1 -- 172.21.15.161:0/1050628838 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d940a96c0 con 0x7f3d9413fe50 2024-05-22T21:25:16.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.870+0000 7f3d9c200640 1 -- 172.21.15.161:0/1050628838 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d940a9be0 con 0x7f3d9413fe50 2024-05-22T21:25:16.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.870+0000 7f3d91600640 1 -- 172.21.15.161:0/1050628838 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3d800039a0 con 0x7f3d9413fe50 2024-05-22T21:25:16.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.870+0000 7f3d91600640 1 --2- 172.21.15.161:0/1050628838 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3d74066690 0x7f3d74068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:16.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.870+0000 7f3d91600640 1 -- 172.21.15.161:0/1050628838 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3d80083710 con 0x7f3d9413fe50 2024-05-22T21:25:16.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.870+0000 7f3d91600640 1 -- 172.21.15.161:0/1050628838 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=196}) v3 -- 0x7f3d7406e2d0 con 0x7f3d9413fe50 2024-05-22T21:25:16.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.870+0000 7f3d99800640 1 --2- 172.21.15.161:0/1050628838 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3d74066690 0x7f3d74068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:16.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.870+0000 7f3d99800640 1 --2- 172.21.15.161:0/1050628838 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3d74066690 0x7f3d74068b50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3d88003b90 tx=0x7f3d880098d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:16.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:16.874+0000 7f3d91600640 1 -- 172.21.15.161:0/1050628838 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3d940a9be0 con 0x7f3d9413fe50 2024-05-22T21:25:17.062 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.062+0000 7f3d9c200640 1 -- 172.21.15.161:0/1050628838 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3d940a9be0 con 0x7f3d9413fe50 2024-05-22T21:25:17.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.694+0000 7f3d91600640 1 -- 172.21.15.161:0/1050628838 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v88) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3d940a9be0 con 0x7f3d9413fe50 2024-05-22T21:25:17.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.698+0000 7f3d9c200640 1 -- 172.21.15.161:0/1050628838 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3d74066690 msgr2=0x7f3d74068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.698+0000 7f3d9c200640 1 --2- 172.21.15.161:0/1050628838 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3d74066690 0x7f3d74068b50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3d88003b90 tx=0x7f3d880098d0 comp rx=0 tx=0).stop 2024-05-22T21:25:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.698+0000 7f3d9c200640 1 -- 172.21.15.161:0/1050628838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d9413fe50 msgr2=0x7f3d9413e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.698+0000 7f3d9c200640 1 --2- 172.21.15.161:0/1050628838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d9413fe50 0x7f3d9413e5e0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000c3c0 tx=0x7f3d8000c880 comp rx=0 tx=0).stop 2024-05-22T21:25:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.698+0000 7f3d9c200640 1 -- 172.21.15.161:0/1050628838 shutdown_connections 2024-05-22T21:25:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.698+0000 7f3d9c200640 1 --2- 172.21.15.161:0/1050628838 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3d74066690 0x7f3d74068b50 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.698+0000 7f3d9c200640 1 --2- 172.21.15.161:0/1050628838 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3d9415af60 0x7f3d9413eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.698+0000 7f3d9c200640 1 --2- 172.21.15.161:0/1050628838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d9413fe50 0x7f3d9413e5e0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.698+0000 7f3d9c200640 1 --2- 172.21.15.161:0/1050628838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d9413cfc0 0x7f3d9413c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.698+0000 7f3d9c200640 1 -- 172.21.15.161:0/1050628838 >> 172.21.15.161:0/1050628838 conn(0x7f3d9406a2d0 msgr2=0x7f3d940b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.698+0000 7f3d9c200640 1 -- 172.21.15.161:0/1050628838 shutdown_connections 2024-05-22T21:25:17.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.698+0000 7f3d9c200640 1 -- 172.21.15.161:0/1050628838 wait complete. 2024-05-22T21:25:17.718 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-22T21:25:17.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.902+0000 7f7308000640 1 Processor -- start 2024-05-22T21:25:17.904 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.902+0000 7f7308000640 1 -- start start 2024-05-22T21:25:17.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.902+0000 7f7308000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f730013cfe0 0x7f730015dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:17.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.906+0000 7f7308000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f730013d560 0x7f730015e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:17.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.906+0000 7f7308000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73000ce830 con 0x7f73000cdb00 2024-05-22T21:25:17.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.906+0000 7f7308000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73000ce650 con 0x7f730013d560 2024-05-22T21:25:17.905 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.906+0000 7f7308000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7300140560 con 0x7f730013cfe0 2024-05-22T21:25:17.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.906+0000 7f7306000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f73000cdb00 0x7f73000cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59470/0 (socket says 172.21.15.161:59470) 2024-05-22T21:25:17.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.906+0000 7f7306000640 1 -- 172.21.15.161:0/984256300 learned_addr learned my addr 172.21.15.161:0/984256300 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:17.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.906+0000 7f7305600640 1 --2- 172.21.15.161:0/984256300 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f730013cfe0 0x7f730015dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:17.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.906+0000 7f7304c00640 1 --2- 172.21.15.161:0/984256300 >> v2:172.21.15.179:3300/0 conn(0x7f730013d560 0x7f730015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.906+0000 7f72ffe00640 1 -- 172.21.15.161:0/984256300 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776163559 0 0) 0x7f73000ce830 con 0x7f73000cdb00 2024-05-22T21:25:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.906+0000 7f72ffe00640 1 -- 172.21.15.161:0/984256300 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72e8003530 con 0x7f73000cdb00 2024-05-22T21:25:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.906+0000 7f7304c00640 1 -- 172.21.15.161:0/984256300 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f730013cfe0 msgr2=0x7f730015dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.906+0000 7f7304c00640 1 --2- 172.21.15.161:0/984256300 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f730013cfe0 0x7f730015dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.906+0000 7f7304c00640 1 -- 172.21.15.161:0/984256300 >> v1:172.21.15.161:6789/0 conn(0x7f73000cdb00 legacy=0x7f73000cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.906+0000 7f7304c00640 1 -- 172.21.15.161:0/984256300 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73000ceb10 con 0x7f730013d560 2024-05-22T21:25:17.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7304c00640 1 --2- 172.21.15.161:0/984256300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f730013d560 0x7f730015e000 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0097c0 tx=0x7f72ec01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f20cf5d5198ed8fd server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:17.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f72ffe00640 1 -- 172.21.15.161:0/984256300 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0ac640 con 0x7f730013d560 2024-05-22T21:25:17.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f72ffe00640 1 -- 172.21.15.161:0/984256300 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec0ac7a0 con 0x7f730013d560 2024-05-22T21:25:17.909 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f72ffe00640 1 -- 172.21.15.161:0/984256300 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0acb30 con 0x7f730013d560 2024-05-22T21:25:17.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7308000640 1 -- 172.21.15.161:0/984256300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f730013d560 msgr2=0x7f730015e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:17.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7308000640 1 --2- 172.21.15.161:0/984256300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f730013d560 0x7f730015e000 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0097c0 tx=0x7f72ec01c3e0 comp rx=0 tx=0).stop 2024-05-22T21:25:17.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7308000640 1 -- 172.21.15.161:0/984256300 shutdown_connections 2024-05-22T21:25:17.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7308000640 1 --2- 172.21.15.161:0/984256300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f730013d560 0x7f730015e000 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:17.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7308000640 1 --2- 172.21.15.161:0/984256300 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f730013cfe0 0x7f730015dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:17.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7308000640 1 -- 172.21.15.161:0/984256300 >> 172.21.15.161:0/984256300 conn(0x7f730006a0a0 msgr2=0x7f730006a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:17.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7308000640 1 -- 172.21.15.161:0/984256300 shutdown_connections 2024-05-22T21:25:17.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7308000640 1 -- 172.21.15.161:0/984256300 wait complete. 2024-05-22T21:25:17.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7308000640 1 Processor -- start 2024-05-22T21:25:17.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7308000640 1 -- start start 2024-05-22T21:25:17.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7308000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73000cdb00 0x7f7300137670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:17.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7308000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f730013cfe0 0x7f7300126c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:17.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7308000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f730013d560 0x7f7300162fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:17.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7308000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7300128360 con 0x7f730013cfe0 2024-05-22T21:25:17.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7308000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73001276e0 con 0x7f73000cdb00 2024-05-22T21:25:17.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7308000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73001279c0 con 0x7f730013d560 2024-05-22T21:25:17.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7304c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f730013cfe0 0x7f7300126c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:17.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7304c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f730013cfe0 0x7f7300126c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59196/0 (socket says 172.21.15.161:59196) 2024-05-22T21:25:17.914 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.910+0000 7f7304c00640 1 -- 172.21.15.161:0/2420541769 learned_addr learned my addr 172.21.15.161:0/2420541769 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:17.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f7306000640 1 --2- 172.21.15.161:0/2420541769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f730013d560 0x7f7300162fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:17.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f7305600640 1 --2- 172.21.15.161:0/2420541769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73000cdb00 0x7f7300137670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:17.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f7304c00640 1 -- 172.21.15.161:0/2420541769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f730013d560 msgr2=0x7f7300162fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:17.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f7304c00640 1 --2- 172.21.15.161:0/2420541769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f730013d560 0x7f7300162fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:17.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f7304c00640 1 -- 172.21.15.161:0/2420541769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73000cdb00 msgr2=0x7f7300137670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:17.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f7304c00640 1 --2- 172.21.15.161:0/2420541769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73000cdb00 0x7f7300137670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:17.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f7304c00640 1 -- 172.21.15.161:0/2420541769 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec009420 con 0x7f730013cfe0 2024-05-22T21:25:17.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f7304c00640 1 --2- 172.21.15.161:0/2420541769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f730013cfe0 0x7f7300126c20 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f72ec01a010 tx=0x7f72ec01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:17.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f72fd600640 1 -- 172.21.15.161:0/2420541769 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0b7650 con 0x7f730013cfe0 2024-05-22T21:25:17.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f72fd600640 1 -- 172.21.15.161:0/2420541769 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec0b77b0 con 0x7f730013cfe0 2024-05-22T21:25:17.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f72fd600640 1 -- 172.21.15.161:0/2420541769 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0b7b60 con 0x7f730013cfe0 2024-05-22T21:25:17.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f7308000640 1 -- 172.21.15.161:0/2420541769 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7300127ca0 con 0x7f730013cfe0 2024-05-22T21:25:17.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f7308000640 1 -- 172.21.15.161:0/2420541769 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7300137bb0 con 0x7f730013cfe0 2024-05-22T21:25:17.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f7308000640 1 -- 172.21.15.161:0/2420541769 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73001284a0 con 0x7f730013cfe0 2024-05-22T21:25:17.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f72fd600640 1 -- 172.21.15.161:0/2420541769 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f72ec0b3070 con 0x7f730013cfe0 2024-05-22T21:25:17.916 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f72fd600640 1 --2- 172.21.15.161:0/2420541769 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72e0066690 0x7f72e0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:17.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f72fd600640 1 -- 172.21.15.161:0/2420541769 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f72ec11eb70 con 0x7f730013cfe0 2024-05-22T21:25:17.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f72fd600640 1 -- 172.21.15.161:0/2420541769 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=196}) v3 -- 0x7f72e006e2d0 con 0x7f730013cfe0 2024-05-22T21:25:17.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.914+0000 7f7305600640 1 --2- 172.21.15.161:0/2420541769 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72e0066690 0x7f72e0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:17.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.918+0000 7f7305600640 1 --2- 172.21.15.161:0/2420541769 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72e0066690 0x7f72e0068b50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f72f40074a0 tx=0x7f72f400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:17.920 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:17.918+0000 7f72fd600640 1 -- 172.21.15.161:0/2420541769 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f72ec0bb060 con 0x7f730013cfe0 2024-05-22T21:25:18.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.074+0000 7f7308000640 1 -- 172.21.15.161:0/2420541769 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7300067220 con 0x7f730013cfe0 2024-05-22T21:25:18.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.698+0000 7f72fd600640 1 -- 172.21.15.161:0/2420541769 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(196..196 src has 1..196) v4 ==== 296+0+0 (secure 0 0 0) 0x7f72ec0b3320 con 0x7f730013cfe0 2024-05-22T21:25:18.700 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.698+0000 7f72fd600640 1 -- 172.21.15.161:0/2420541769 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=197}) v3 -- 0x7f72e006c700 con 0x7f730013cfe0 2024-05-22T21:25:18.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.702+0000 7f72fd600640 1 -- 172.21.15.161:0/2420541769 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v196) v1 ==== 181+0+0 (secure 0 0 0) 0x7f72ec0b1020 con 0x7f730013cfe0 2024-05-22T21:25:18.705 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_metadata' removed 2024-05-22T21:25:18.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.706+0000 7f7308000640 1 -- 172.21.15.161:0/2420541769 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72e0066690 msgr2=0x7f72e0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:18.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.706+0000 7f7308000640 1 --2- 172.21.15.161:0/2420541769 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72e0066690 0x7f72e0068b50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f72f40074a0 tx=0x7f72f400b040 comp rx=0 tx=0).stop 2024-05-22T21:25:18.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.706+0000 7f7308000640 1 -- 172.21.15.161:0/2420541769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f730013cfe0 msgr2=0x7f7300126c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:18.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.706+0000 7f7308000640 1 --2- 172.21.15.161:0/2420541769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f730013cfe0 0x7f7300126c20 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f72ec01a010 tx=0x7f72ec01ac20 comp rx=0 tx=0).stop 2024-05-22T21:25:18.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.706+0000 7f7308000640 1 -- 172.21.15.161:0/2420541769 shutdown_connections 2024-05-22T21:25:18.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.706+0000 7f7308000640 1 --2- 172.21.15.161:0/2420541769 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f72e0066690 0x7f72e0068b50 secure :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f72f40074a0 tx=0x7f72f400b040 comp rx=0 tx=0).stop 2024-05-22T21:25:18.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.706+0000 7f7308000640 1 --2- 172.21.15.161:0/2420541769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f730013d560 0x7f7300162fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:18.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.706+0000 7f7308000640 1 --2- 172.21.15.161:0/2420541769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f730013cfe0 0x7f7300126c20 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:18.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.706+0000 7f7308000640 1 --2- 172.21.15.161:0/2420541769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73000cdb00 0x7f7300137670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:18.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.706+0000 7f7308000640 1 -- 172.21.15.161:0/2420541769 >> 172.21.15.161:0/2420541769 conn(0x7f730006a0a0 msgr2=0x7f730015bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:18.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.706+0000 7f7308000640 1 -- 172.21.15.161:0/2420541769 shutdown_connections 2024-05-22T21:25:18.707 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.706+0000 7f7308000640 1 -- 172.21.15.161:0/2420541769 wait complete. 2024-05-22T21:25:18.727 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-22T21:25:18.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.866+0000 7fb817a00640 1 Processor -- start 2024-05-22T21:25:18.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.866+0000 7fb817a00640 1 -- start start 2024-05-22T21:25:18.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.866+0000 7fb817a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb810144980 0x7fb81015c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:18.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.866+0000 7fb817a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb81006d560 0x7fb8100cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:18.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.866+0000 7fb817a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81006d0d0 con 0x7fb8100cca40 2024-05-22T21:25:18.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.866+0000 7fb817a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8100cced0 con 0x7fb81006d560 2024-05-22T21:25:18.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.866+0000 7fb817a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81015c680 con 0x7fb810144980 2024-05-22T21:25:18.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.866+0000 7fb807e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb81006d560 0x7fb8100cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:18.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.866+0000 7fb807e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb81006d560 0x7fb8100cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59204/0 (socket says 172.21.15.161:59204) 2024-05-22T21:25:18.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.866+0000 7fb807e00640 1 -- 172.21.15.161:0/1430135673 learned_addr learned my addr 172.21.15.161:0/1430135673 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:18.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.866+0000 7fb815000640 1 --2- 172.21.15.161:0/1430135673 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb810144980 0x7fb81015c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:18.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.866+0000 7fb807e00640 1 -- 172.21.15.161:0/1430135673 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb810144980 msgr2=0x7fb81015c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:18.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.866+0000 7fb807e00640 1 --2- 172.21.15.161:0/1430135673 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb810144980 0x7fb81015c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:18.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.870+0000 7fb807e00640 1 -- 172.21.15.161:0/1430135673 >> v1:172.21.15.179:6789/0 conn(0x7fb8100cca40 legacy=0x7fb81015c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:18.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.870+0000 7fb807e00640 1 -- 172.21.15.161:0/1430135673 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81006c4c0 con 0x7fb81006d560 2024-05-22T21:25:18.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.870+0000 7fb807e00640 1 --2- 172.21.15.161:0/1430135673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb81006d560 0x7fb8100cc500 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8009870 tx=0x7fb7f801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ced1d8812a876d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:18.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.870+0000 7fb80fe00640 1 -- 172.21.15.161:0/1430135673 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f80ab650 con 0x7fb81006d560 2024-05-22T21:25:18.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.870+0000 7fb80fe00640 1 -- 172.21.15.161:0/1430135673 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f80ab7b0 con 0x7fb81006d560 2024-05-22T21:25:18.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.870+0000 7fb80fe00640 1 -- 172.21.15.161:0/1430135673 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f80abb40 con 0x7fb81006d560 2024-05-22T21:25:18.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.870+0000 7fb817a00640 1 -- 172.21.15.161:0/1430135673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb81006d560 msgr2=0x7fb8100cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:18.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.870+0000 7fb817a00640 1 --2- 172.21.15.161:0/1430135673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb81006d560 0x7fb8100cc500 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8009870 tx=0x7fb7f801c560 comp rx=0 tx=0).stop 2024-05-22T21:25:18.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.870+0000 7fb817a00640 1 -- 172.21.15.161:0/1430135673 shutdown_connections 2024-05-22T21:25:18.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.870+0000 7fb817a00640 1 --2- 172.21.15.161:0/1430135673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb81006d560 0x7fb8100cc500 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:18.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.870+0000 7fb817a00640 1 --2- 172.21.15.161:0/1430135673 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb810144980 0x7fb81015c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:18.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.870+0000 7fb817a00640 1 -- 172.21.15.161:0/1430135673 >> 172.21.15.161:0/1430135673 conn(0x7fb81006a0a0 msgr2=0x7fb81006a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:18.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.870+0000 7fb817a00640 1 -- 172.21.15.161:0/1430135673 shutdown_connections 2024-05-22T21:25:18.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.870+0000 7fb817a00640 1 -- 172.21.15.161:0/1430135673 wait complete. 2024-05-22T21:25:18.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb817a00640 1 Processor -- start 2024-05-22T21:25:18.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb817a00640 1 -- start start 2024-05-22T21:25:18.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb817a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb81006d560 0x7fb8100ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:18.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb817a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8100cca40 0x7fb810071770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:18.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb817a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb810144980 0x7fb81013e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:18.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb817a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81015a7a0 con 0x7fb810144980 2024-05-22T21:25:18.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb817a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb810071cb0 con 0x7fb81006d560 2024-05-22T21:25:18.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb817a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8100ace90 con 0x7fb8100cca40 2024-05-22T21:25:18.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb815a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb810144980 0x7fb81013e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:18.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb807e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8100cca40 0x7fb810071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:18.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb815000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb81006d560 0x7fb8100ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:18.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb815a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb810144980 0x7fb81013e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59212/0 (socket says 172.21.15.161:59212) 2024-05-22T21:25:18.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb815a00640 1 -- 172.21.15.161:0/1434770633 learned_addr learned my addr 172.21.15.161:0/1434770633 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:18.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb815a00640 1 -- 172.21.15.161:0/1434770633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8100cca40 msgr2=0x7fb810071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:18.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb815a00640 1 --2- 172.21.15.161:0/1434770633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8100cca40 0x7fb810071770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:18.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb815a00640 1 -- 172.21.15.161:0/1434770633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb81006d560 msgr2=0x7fb8100ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:18.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb815a00640 1 --2- 172.21.15.161:0/1434770633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb81006d560 0x7fb8100ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:18.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb815a00640 1 -- 172.21.15.161:0/1434770633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f80094d0 con 0x7fb810144980 2024-05-22T21:25:18.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb815a00640 1 --2- 172.21.15.161:0/1434770633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb810144980 0x7fb81013e750 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fb80800c770 tx=0x7fb80800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:18.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb80d600640 1 -- 172.21.15.161:0/1434770633 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808003b80 con 0x7fb810144980 2024-05-22T21:25:18.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb80d600640 1 -- 172.21.15.161:0/1434770633 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb808003ce0 con 0x7fb810144980 2024-05-22T21:25:18.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb80d600640 1 -- 172.21.15.161:0/1434770633 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80800aa90 con 0x7fb810144980 2024-05-22T21:25:18.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.874+0000 7fb817a00640 1 -- 172.21.15.161:0/1434770633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8100ad170 con 0x7fb810144980 2024-05-22T21:25:18.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.878+0000 7fb817a00640 1 -- 172.21.15.161:0/1434770633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8100ad730 con 0x7fb810144980 2024-05-22T21:25:18.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.878+0000 7fb817a00640 1 -- 172.21.15.161:0/1434770633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81015a970 con 0x7fb810144980 2024-05-22T21:25:18.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.878+0000 7fb80d600640 1 -- 172.21.15.161:0/1434770633 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb80800abf0 con 0x7fb810144980 2024-05-22T21:25:18.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.878+0000 7fb80d600640 1 --2- 172.21.15.161:0/1434770633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7ec066690 0x7fb7ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:18.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.878+0000 7fb80d600640 1 -- 172.21.15.161:0/1434770633 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fb808083d40 con 0x7fb810144980 2024-05-22T21:25:18.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.878+0000 7fb815000640 1 --2- 172.21.15.161:0/1434770633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7ec066690 0x7fb7ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:18.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.878+0000 7fb80d600640 1 -- 172.21.15.161:0/1434770633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=197}) v3 -- 0x7fb7ec06d460 con 0x7fb810144980 2024-05-22T21:25:18.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.878+0000 7fb815000640 1 --2- 172.21.15.161:0/1434770633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7ec066690 0x7fb7ec068b50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb810158530 tx=0x7fb800009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:18.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:18.882+0000 7fb80d600640 1 -- 172.21.15.161:0/1434770633 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb81015a970 con 0x7fb810144980 2024-05-22T21:25:19.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.038+0000 7fb817a00640 1 -- 172.21.15.161:0/1434770633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb810067220 con 0x7fb810144980 2024-05-22T21:25:19.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.706+0000 7fb80d600640 1 -- 172.21.15.161:0/1434770633 <== mon.0 v2:172.21.15.161:3300/0 7 ==== osd_map(197..197 src has 1..197) v4 ==== 296+0+0 (secure 0 0 0) 0x7fb808054f70 con 0x7fb810144980 2024-05-22T21:25:19.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.706+0000 7fb80d600640 1 -- 172.21.15.161:0/1434770633 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=198}) v3 -- 0x7fb7ec06c1f0 con 0x7fb810144980 2024-05-22T21:25:19.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.710+0000 7fb80d600640 1 -- 172.21.15.161:0/1434770633 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v197) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb8080512b0 con 0x7fb810144980 2024-05-22T21:25:19.716 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cephfs_data' removed 2024-05-22T21:25:19.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.718+0000 7fb817a00640 1 -- 172.21.15.161:0/1434770633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7ec066690 msgr2=0x7fb7ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:19.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.718+0000 7fb817a00640 1 --2- 172.21.15.161:0/1434770633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7ec066690 0x7fb7ec068b50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb810158530 tx=0x7fb800009190 comp rx=0 tx=0).stop 2024-05-22T21:25:19.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.718+0000 7fb817a00640 1 -- 172.21.15.161:0/1434770633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb810144980 msgr2=0x7fb81013e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:19.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.718+0000 7fb817a00640 1 --2- 172.21.15.161:0/1434770633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb810144980 0x7fb81013e750 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fb80800c770 tx=0x7fb80800cc30 comp rx=0 tx=0).stop 2024-05-22T21:25:19.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.718+0000 7fb817a00640 1 -- 172.21.15.161:0/1434770633 shutdown_connections 2024-05-22T21:25:19.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.718+0000 7fb817a00640 1 --2- 172.21.15.161:0/1434770633 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb7ec066690 0x7fb7ec068b50 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:19.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.718+0000 7fb817a00640 1 --2- 172.21.15.161:0/1434770633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb810144980 0x7fb81013e750 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:19.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.718+0000 7fb817a00640 1 --2- 172.21.15.161:0/1434770633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8100cca40 0x7fb810071770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:19.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.718+0000 7fb817a00640 1 --2- 172.21.15.161:0/1434770633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb81006d560 0x7fb8100ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:19.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.718+0000 7fb817a00640 1 -- 172.21.15.161:0/1434770633 >> 172.21.15.161:0/1434770633 conn(0x7fb81006a0a0 msgr2=0x7fb81006a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:19.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.718+0000 7fb817a00640 1 -- 172.21.15.161:0/1434770633 shutdown_connections 2024-05-22T21:25:19.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.718+0000 7fb817a00640 1 -- 172.21.15.161:0/1434770633 wait complete. 2024-05-22T21:25:19.741 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-22T21:25:19.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.878+0000 7f9580600640 1 Processor -- start 2024-05-22T21:25:19.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.878+0000 7f9580600640 1 -- start start 2024-05-22T21:25:19.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.882+0000 7f9580600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9578141730 0x7f9578141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:19.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.882+0000 7f9580600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f957806fcc0 0x7f957815ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:19.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.882+0000 7f9580600640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f95780cbca0 0x7f95780cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:19.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.882+0000 7f9580600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95780cc650 con 0x7f957806fcc0 2024-05-22T21:25:19.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.882+0000 7f9580600640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95780cd150 con 0x7f95780cbca0 2024-05-22T21:25:19.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.882+0000 7f9580600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957815f550 con 0x7f9578141730 2024-05-22T21:25:19.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.882+0000 7f957d200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f957806fcc0 0x7f957815ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:19.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.882+0000 7f957d200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f957806fcc0 0x7f957815ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59228/0 (socket says 172.21.15.161:59228) 2024-05-22T21:25:19.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.882+0000 7f957d200640 1 -- 172.21.15.161:0/1957098320 learned_addr learned my addr 172.21.15.161:0/1957098320 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:19.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.882+0000 7f957d200640 1 -- 172.21.15.161:0/1957098320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9578141730 msgr2=0x7f9578141ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:19.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.882+0000 7f957d200640 1 --2- 172.21.15.161:0/1957098320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9578141730 0x7f9578141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:19.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.882+0000 7f957d200640 1 -- 172.21.15.161:0/1957098320 >> v2:172.21.15.179:3300/0 conn(0x7f95780cbca0 msgr2=0x7f95780cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:19.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.882+0000 7f957d200640 1 --2- 172.21.15.161:0/1957098320 >> v2:172.21.15.179:3300/0 conn(0x7f95780cbca0 0x7f95780cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:19.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.882+0000 7f957d200640 1 -- 172.21.15.161:0/1957098320 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957815f830 con 0x7f957806fcc0 2024-05-22T21:25:19.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.882+0000 7f957d200640 1 --2- 172.21.15.161:0/1957098320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f957806fcc0 0x7f957815ed20 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f957806d400 tx=0x7f956401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34de17d7ee6725f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:19.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.882+0000 7f9577e00640 1 -- 172.21.15.161:0/1957098320 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95640aa610 con 0x7f957806fcc0 2024-05-22T21:25:19.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9577e00640 1 -- 172.21.15.161:0/1957098320 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95640a6040 con 0x7f957806fcc0 2024-05-22T21:25:19.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9577e00640 1 -- 172.21.15.161:0/1957098320 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95640aa990 con 0x7f957806fcc0 2024-05-22T21:25:19.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 -- 172.21.15.161:0/1957098320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f957806fcc0 msgr2=0x7f957815ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:19.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 --2- 172.21.15.161:0/1957098320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f957806fcc0 0x7f957815ed20 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f957806d400 tx=0x7f956401c4f0 comp rx=0 tx=0).stop 2024-05-22T21:25:19.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 -- 172.21.15.161:0/1957098320 shutdown_connections 2024-05-22T21:25:19.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 --2- 172.21.15.161:0/1957098320 >> v2:172.21.15.179:3300/0 conn(0x7f95780cbca0 0x7f95780cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:19.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 --2- 172.21.15.161:0/1957098320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f957806fcc0 0x7f957815ed20 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:19.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 --2- 172.21.15.161:0/1957098320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9578141730 0x7f9578141ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:19.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 -- 172.21.15.161:0/1957098320 >> 172.21.15.161:0/1957098320 conn(0x7f957806a2d0 msgr2=0x7f957806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:19.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 -- 172.21.15.161:0/1957098320 shutdown_connections 2024-05-22T21:25:19.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 -- 172.21.15.161:0/1957098320 wait complete. 2024-05-22T21:25:19.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 Processor -- start 2024-05-22T21:25:19.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 -- start start 2024-05-22T21:25:19.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f957806fcc0 0x7f95781599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:19.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95780cbca0 0x7f95780a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:19.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9578141730 0x7f9578158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:19.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957813b670 con 0x7f95780cbca0 2024-05-22T21:25:19.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957815d910 con 0x7f957806fcc0 2024-05-22T21:25:19.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f9580600640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957815dbf0 con 0x7f9578141730 2024-05-22T21:25:19.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f957d200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95780cbca0 0x7f95780a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:19.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f957d200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95780cbca0 0x7f95780a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59232/0 (socket says 172.21.15.161:59232) 2024-05-22T21:25:19.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f957d200640 1 -- 172.21.15.161:0/2972422914 learned_addr learned my addr 172.21.15.161:0/2972422914 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:19.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f957dc00640 1 --2- 172.21.15.161:0/2972422914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f957806fcc0 0x7f95781599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:19.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.886+0000 7f957e600640 1 --2- 172.21.15.161:0/2972422914 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9578141730 0x7f9578158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:19.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.890+0000 7f957d200640 1 -- 172.21.15.161:0/2972422914 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9578141730 msgr2=0x7f9578158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:19.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.890+0000 7f957d200640 1 --2- 172.21.15.161:0/2972422914 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9578141730 0x7f9578158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:19.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.890+0000 7f957d200640 1 -- 172.21.15.161:0/2972422914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f957806fcc0 msgr2=0x7f95781599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:19.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.890+0000 7f957d200640 1 --2- 172.21.15.161:0/2972422914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f957806fcc0 0x7f95781599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:19.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.890+0000 7f957d200640 1 -- 172.21.15.161:0/2972422914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95640094d0 con 0x7f95780cbca0 2024-05-22T21:25:19.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.890+0000 7f957d200640 1 --2- 172.21.15.161:0/2972422914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95780cbca0 0x7f95780a84a0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f956401aba0 tx=0x7f956401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:19.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.890+0000 7f9575600640 1 -- 172.21.15.161:0/2972422914 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95640b3800 con 0x7f95780cbca0 2024-05-22T21:25:19.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.890+0000 7f9575600640 1 -- 172.21.15.161:0/2972422914 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95640b3960 con 0x7f95780cbca0 2024-05-22T21:25:19.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.890+0000 7f9575600640 1 -- 172.21.15.161:0/2972422914 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95640b3d10 con 0x7f95780cbca0 2024-05-22T21:25:19.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.890+0000 7f9580600640 1 -- 172.21.15.161:0/2972422914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957815ded0 con 0x7f95780cbca0 2024-05-22T21:25:19.892 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.890+0000 7f9580600640 1 -- 172.21.15.161:0/2972422914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957815af40 con 0x7f95780cbca0 2024-05-22T21:25:19.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.890+0000 7f9580600640 1 -- 172.21.15.161:0/2972422914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9578067220 con 0x7f95780cbca0 2024-05-22T21:25:19.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.890+0000 7f9575600640 1 -- 172.21.15.161:0/2972422914 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f95640b2380 con 0x7f95780cbca0 2024-05-22T21:25:19.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.890+0000 7f9575600640 1 --2- 172.21.15.161:0/2972422914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9558066690 0x7f9558068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:19.893 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.890+0000 7f9575600640 1 -- 172.21.15.161:0/2972422914 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f956411e5f0 con 0x7f95780cbca0 2024-05-22T21:25:19.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.890+0000 7f957dc00640 1 --2- 172.21.15.161:0/2972422914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9558066690 0x7f9558068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:19.894 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.894+0000 7f957dc00640 1 --2- 172.21.15.161:0/2972422914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9558066690 0x7f9558068b50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f956c0098f0 tx=0x7f956c006f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:19.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:19.894+0000 7f9575600640 1 -- 172.21.15.161:0/2972422914 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f95640ebbe0 con 0x7f95780cbca0 2024-05-22T21:25:20.052 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.050+0000 7f9580600640 1 -- 172.21.15.161:0/2972422914 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f957813bae0 con 0x7f95780cbca0 2024-05-22T21:25:20.057 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.058+0000 7f9575600640 1 -- 172.21.15.161:0/2972422914 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f95640aae60 con 0x7f95780cbca0 2024-05-22T21:25:20.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.058+0000 7f9580600640 1 -- 172.21.15.161:0/2972422914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9558066690 msgr2=0x7f9558068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:20.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.058+0000 7f9580600640 1 --2- 172.21.15.161:0/2972422914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9558066690 0x7f9558068b50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f956c0098f0 tx=0x7f956c006f90 comp rx=0 tx=0).stop 2024-05-22T21:25:20.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.058+0000 7f9580600640 1 -- 172.21.15.161:0/2972422914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95780cbca0 msgr2=0x7f95780a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:20.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.058+0000 7f9580600640 1 --2- 172.21.15.161:0/2972422914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95780cbca0 0x7f95780a84a0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f956401aba0 tx=0x7f956401ad30 comp rx=0 tx=0).stop 2024-05-22T21:25:20.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.058+0000 7f9580600640 1 -- 172.21.15.161:0/2972422914 shutdown_connections 2024-05-22T21:25:20.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.058+0000 7f9580600640 1 --2- 172.21.15.161:0/2972422914 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f9558066690 0x7f9558068b50 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:20.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.058+0000 7f9580600640 1 --2- 172.21.15.161:0/2972422914 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9578141730 0x7f9578158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:20.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.058+0000 7f9580600640 1 --2- 172.21.15.161:0/2972422914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95780cbca0 0x7f95780a84a0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:20.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.058+0000 7f9580600640 1 --2- 172.21.15.161:0/2972422914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f957806fcc0 0x7f95781599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:20.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.058+0000 7f9580600640 1 -- 172.21.15.161:0/2972422914 >> 172.21.15.161:0/2972422914 conn(0x7f957806a2d0 msgr2=0x7f9578141d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:20.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.058+0000 7f9580600640 1 -- 172.21.15.161:0/2972422914 shutdown_connections 2024-05-22T21:25:20.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.058+0000 7f9580600640 1 -- 172.21.15.161:0/2972422914 wait complete. 2024-05-22T21:25:20.078 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2024-05-22T21:25:20.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb741e00640 1 Processor -- start 2024-05-22T21:25:20.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb741e00640 1 -- start start 2024-05-22T21:25:20.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb741e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb73c160df0 0x7fb73c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:20.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb741e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb73c06f120 0x7fb73c15f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:20.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb741e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73c0d0460 con 0x7fb73c15ec30 2024-05-22T21:25:20.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb741e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73c0d0180 con 0x7fb73c160df0 2024-05-22T21:25:20.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb741e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c06df30 con 0x7fb73c06f120 2024-05-22T21:25:20.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb740a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb73c160df0 0x7fb73c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:20.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb740a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb73c160df0 0x7fb73c0cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59248/0 (socket says 172.21.15.161:59248) 2024-05-22T21:25:20.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb73be00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb73c15ec30 0x7fb73c06ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59480/0 (socket says 172.21.15.161:59480) 2024-05-22T21:25:20.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb740a00640 1 -- 172.21.15.161:0/405684861 learned_addr learned my addr 172.21.15.161:0/405684861 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:20.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb741400640 1 --2- 172.21.15.161:0/405684861 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb73c06f120 0x7fb73c15f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:20.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb740a00640 1 -- 172.21.15.161:0/405684861 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb73c06f120 msgr2=0x7fb73c15f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:20.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb740a00640 1 --2- 172.21.15.161:0/405684861 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb73c06f120 0x7fb73c15f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:20.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb740a00640 1 -- 172.21.15.161:0/405684861 >> v1:172.21.15.161:6789/0 conn(0x7fb73c15ec30 legacy=0x7fb73c06ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:20.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb740a00640 1 -- 172.21.15.161:0/405684861 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73c06e210 con 0x7fb73c160df0 2024-05-22T21:25:20.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.214+0000 7fb73b400640 1 -- 172.21.15.161:0/405684861 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38198045 0 0) 0x7fb73c0d0460 con 0x7fb73c15ec30 2024-05-22T21:25:20.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.218+0000 7fb740a00640 1 --2- 172.21.15.161:0/405684861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c160df0 0x7fb73c0cfb70 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fb730009870 tx=0x7fb73001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7b073dfa484598 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:20.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.218+0000 7fb73b400640 1 -- 172.21.15.161:0/405684861 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7300ac630 con 0x7fb73c160df0 2024-05-22T21:25:20.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.218+0000 7fb73b400640 1 -- 172.21.15.161:0/405684861 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7300ac790 con 0x7fb73c160df0 2024-05-22T21:25:20.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.218+0000 7fb73b400640 1 -- 172.21.15.161:0/405684861 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7300acb20 con 0x7fb73c160df0 2024-05-22T21:25:20.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.218+0000 7fb741e00640 1 -- 172.21.15.161:0/405684861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c160df0 msgr2=0x7fb73c0cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:20.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.218+0000 7fb741e00640 1 --2- 172.21.15.161:0/405684861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c160df0 0x7fb73c0cfb70 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fb730009870 tx=0x7fb73001c410 comp rx=0 tx=0).stop 2024-05-22T21:25:20.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.218+0000 7fb741e00640 1 -- 172.21.15.161:0/405684861 shutdown_connections 2024-05-22T21:25:20.219 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.218+0000 7fb741e00640 1 --2- 172.21.15.161:0/405684861 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb73c06f120 0x7fb73c15f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.218+0000 7fb741e00640 1 --2- 172.21.15.161:0/405684861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c160df0 0x7fb73c0cfb70 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.218+0000 7fb741e00640 1 -- 172.21.15.161:0/405684861 >> 172.21.15.161:0/405684861 conn(0x7fb73c0b2930 msgr2=0x7fb73c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.218+0000 7fb741e00640 1 -- 172.21.15.161:0/405684861 shutdown_connections 2024-05-22T21:25:20.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.218+0000 7fb741e00640 1 -- 172.21.15.161:0/405684861 wait complete. 2024-05-22T21:25:20.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.218+0000 7fb741e00640 1 Processor -- start 2024-05-22T21:25:20.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.218+0000 7fb741e00640 1 -- start start 2024-05-22T21:25:20.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb741e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c06f120 0x7fb73c156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:20.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb741e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb73c15ec30 0x7fb73c15dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:20.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb741e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb73c160df0 0x7fb73c1536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:20.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb741e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c159080 con 0x7fb73c06f120 2024-05-22T21:25:20.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb741e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c0c4840 con 0x7fb73c160df0 2024-05-22T21:25:20.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb741e00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c0c4b20 con 0x7fb73c15ec30 2024-05-22T21:25:20.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb740a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c06f120 0x7fb73c156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:20.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb740a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c06f120 0x7fb73c156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59252/0 (socket says 172.21.15.161:59252) 2024-05-22T21:25:20.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb740a00640 1 -- 172.21.15.161:0/3423822580 learned_addr learned my addr 172.21.15.161:0/3423822580 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:20.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb73be00640 1 --2- 172.21.15.161:0/3423822580 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb73c15ec30 0x7fb73c15dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:20.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb741400640 1 --2- 172.21.15.161:0/3423822580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb73c160df0 0x7fb73c1536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:20.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb740a00640 1 -- 172.21.15.161:0/3423822580 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb73c15ec30 msgr2=0x7fb73c15dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:20.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb740a00640 1 --2- 172.21.15.161:0/3423822580 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb73c15ec30 0x7fb73c15dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:20.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb740a00640 1 -- 172.21.15.161:0/3423822580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb73c160df0 msgr2=0x7fb73c1536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:20.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb740a00640 1 --2- 172.21.15.161:0/3423822580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb73c160df0 0x7fb73c1536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:20.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb740a00640 1 -- 172.21.15.161:0/3423822580 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7300094d0 con 0x7fb73c06f120 2024-05-22T21:25:20.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb741400640 1 --2- 172.21.15.161:0/3423822580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb73c160df0 0x7fb73c1536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:25:20.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb740a00640 1 --2- 172.21.15.161:0/3423822580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c06f120 0x7fb73c156350 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fb7300af040 tx=0x7fb7300191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:20.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb738c00640 1 -- 172.21.15.161:0/3423822580 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7300b7670 con 0x7fb73c06f120 2024-05-22T21:25:20.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb738c00640 1 -- 172.21.15.161:0/3423822580 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7300b77d0 con 0x7fb73c06f120 2024-05-22T21:25:20.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb738c00640 1 -- 172.21.15.161:0/3423822580 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7300b7b60 con 0x7fb73c06f120 2024-05-22T21:25:20.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb741e00640 1 -- 172.21.15.161:0/3423822580 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73c1592f0 con 0x7fb73c06f120 2024-05-22T21:25:20.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb741e00640 1 -- 172.21.15.161:0/3423822580 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73c15e620 con 0x7fb73c06f120 2024-05-22T21:25:20.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.222+0000 7fb741e00640 1 -- 172.21.15.161:0/3423822580 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73c067070 con 0x7fb73c06f120 2024-05-22T21:25:20.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.226+0000 7fb738c00640 1 -- 172.21.15.161:0/3423822580 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb7300b3070 con 0x7fb73c06f120 2024-05-22T21:25:20.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.226+0000 7fb738c00640 1 --2- 172.21.15.161:0/3423822580 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb71c066690 0x7fb71c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:20.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.226+0000 7fb738c00640 1 -- 172.21.15.161:0/3423822580 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb73011e670 con 0x7fb73c06f120 2024-05-22T21:25:20.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.226+0000 7fb73be00640 1 --2- 172.21.15.161:0/3423822580 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb71c066690 0x7fb71c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:20.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.226+0000 7fb73be00640 1 --2- 172.21.15.161:0/3423822580 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb71c066690 0x7fb71c068b50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb73c15ac30 tx=0x7fb728007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:20.230 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.230+0000 7fb738c00640 1 -- 172.21.15.161:0/3423822580 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb7300ebc50 con 0x7fb73c06f120 2024-05-22T21:25:20.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.442+0000 7fb741e00640 1 -- 172.21.15.161:0/3423822580 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7fb73c06ecd0 con 0x7fb73c06f120 2024-05-22T21:25:20.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.706+0000 7fb738c00640 1 -- 172.21.15.161:0/3423822580 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v578) v1 ==== 135+0+0 (secure 0 0 0) 0x7fb7300b10c0 con 0x7fb73c06f120 2024-05-22T21:25:20.708 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.706+0000 7fb741e00640 1 -- 172.21.15.161:0/3423822580 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb71c066690 msgr2=0x7fb71c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:20.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.706+0000 7fb741e00640 1 --2- 172.21.15.161:0/3423822580 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb71c066690 0x7fb71c068b50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb73c15ac30 tx=0x7fb728007c10 comp rx=0 tx=0).stop 2024-05-22T21:25:20.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.706+0000 7fb741e00640 1 -- 172.21.15.161:0/3423822580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c06f120 msgr2=0x7fb73c156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:20.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.706+0000 7fb741e00640 1 --2- 172.21.15.161:0/3423822580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c06f120 0x7fb73c156350 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fb7300af040 tx=0x7fb7300191b0 comp rx=0 tx=0).stop 2024-05-22T21:25:20.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.706+0000 7fb741e00640 1 -- 172.21.15.161:0/3423822580 shutdown_connections 2024-05-22T21:25:20.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.706+0000 7fb741e00640 1 --2- 172.21.15.161:0/3423822580 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb71c066690 0x7fb71c068b50 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:20.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.706+0000 7fb741e00640 1 --2- 172.21.15.161:0/3423822580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb73c160df0 0x7fb73c1536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:20.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.706+0000 7fb741e00640 1 --2- 172.21.15.161:0/3423822580 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb73c15ec30 0x7fb73c15dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:20.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.706+0000 7fb741e00640 1 --2- 172.21.15.161:0/3423822580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c06f120 0x7fb73c156350 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:20.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.706+0000 7fb741e00640 1 -- 172.21.15.161:0/3423822580 >> 172.21.15.161:0/3423822580 conn(0x7fb73c0b2930 msgr2=0x7fb73c06b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:20.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.706+0000 7fb741e00640 1 -- 172.21.15.161:0/3423822580 shutdown_connections 2024-05-22T21:25:20.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:20.706+0000 7fb741e00640 1 -- 172.21.15.161:0/3423822580 wait complete. 2024-05-22T21:25:20.727 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-05-22T21:25:20.728 INFO:tasks.cephfs_test_runner: 2024-05-22T21:25:20.728 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-22T21:25:20.728 INFO:tasks.cephfs_test_runner:Ran 10 tests in 684.957s 2024-05-22T21:25:20.728 INFO:tasks.cephfs_test_runner: 2024-05-22T21:25:20.728 INFO:tasks.cephfs_test_runner:OK 2024-05-22T21:25:20.729 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-22T21:25:20.738 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-22T21:25:20.748 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-22T21:25:20.748 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:25:20.748 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:25:20.757 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:25:20.757 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-22T21:25:20.757 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:25:20.809 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350304k,nr_inodes=4087576,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3025 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-22T21:25:20.809 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:25:20.809 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:25:20.819 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:25:20.819 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-22T21:25:20.819 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-22T21:25:20.870 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350260k,nr_inodes=4087565,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2902 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-22T21:25:20.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:20.870 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-05-22T21:25:20.879 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-22T21:25:20.879 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-22T21:25:20.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:20.880 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-22T21:25:20.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:20.942 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-22T21:25:21.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:21.009 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2024-05-22T21:25:21.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:21.180 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-05-22T21:25:21.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:21.189 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2024-05-22T21:25:21.254 INFO:teuthology.orchestra.run.smithi161.stderr:Cannot find device "ceph-brx" 2024-05-22T21:25:21.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:25:21.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:21.256 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-22T21:25:21.266 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-22T21:25:21.266 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-22T21:25:21.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:21.267 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-22T21:25:21.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:21.333 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-22T21:25:21.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:21.395 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link delete ceph-brx 2024-05-22T21:25:21.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:21.548 DEBUG:teuthology.orchestra.run.smithi179:> ip netns list 2024-05-22T21:25:21.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-22T21:25:21.557 DEBUG:teuthology.orchestra.run.smithi179:> sudo ip link delete ceph-brx 2024-05-22T21:25:21.627 INFO:teuthology.orchestra.run.smithi179.stderr:Cannot find device "ceph-brx" 2024-05-22T21:25:21.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:25:21.629 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-22T21:25:21.639 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-22T21:25:21.639 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:25:21.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.774+0000 7f3e90800640 1 Processor -- start 2024-05-22T21:25:21.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.774+0000 7f3e90800640 1 -- start start 2024-05-22T21:25:21.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.778+0000 7f3e90800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3e880715a0 0x7f3e8815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:21.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.778+0000 7f3e90800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8815ec90 0x7f3e8815f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:21.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.778+0000 7f3e90800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e881442a0 con 0x7f3e88143ec0 2024-05-22T21:25:21.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.778+0000 7f3e90800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e8815f660 con 0x7f3e880715a0 2024-05-22T21:25:21.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.778+0000 7f3e90800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e88070500 con 0x7f3e8815ec90 2024-05-22T21:25:21.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.778+0000 7f3e8e800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8815ec90 0x7f3e8815f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:21.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.778+0000 7f3e8d400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3e880715a0 0x7f3e8815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:21.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.778+0000 7f3e8e800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8815ec90 0x7f3e8815f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.161:58622/0 (socket says 172.21.15.161:58622) 2024-05-22T21:25:21.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.778+0000 7f3e8d400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3e880715a0 0x7f3e8815f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:49754/0 (socket says 172.21.15.161:49754) 2024-05-22T21:25:21.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.778+0000 7f3e8e800640 1 -- 172.21.15.161:0/3072435114 learned_addr learned my addr 172.21.15.161:0/3072435114 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:21.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.778+0000 7f3e8de00640 1 --1- 172.21.15.161:0/3072435114 >> v1:172.21.15.179:6789/0 conn(0x7f3e88143ec0 0x7f3e8806fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:33348/0 (socket says 172.21.15.161:33348) 2024-05-22T21:25:21.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.778+0000 7f3e8e800640 1 -- 172.21.15.161:0/3072435114 >> v2:172.21.15.179:3300/0 conn(0x7f3e880715a0 msgr2=0x7f3e8815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:21.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.778+0000 7f3e8e800640 1 --2- 172.21.15.161:0/3072435114 >> v2:172.21.15.179:3300/0 conn(0x7f3e880715a0 0x7f3e8815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:21.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.778+0000 7f3e8e800640 1 -- 172.21.15.161:0/3072435114 >> v1:172.21.15.179:6789/0 conn(0x7f3e88143ec0 legacy=0x7f3e8806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:21.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.778+0000 7f3e8e800640 1 -- 172.21.15.161:0/3072435114 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e880707e0 con 0x7f3e8815ec90 2024-05-22T21:25:21.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.778+0000 7f3e8d400640 1 --2- 172.21.15.161:0/3072435114 >> v2:172.21.15.179:3300/0 conn(0x7f3e880715a0 0x7f3e8815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:25:21.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e8e800640 1 --2- 172.21.15.161:0/3072435114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8815ec90 0x7f3e8815f120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009e50 tx=0x7f3e8401ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5331a48673c2e0c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:21.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e8ca00640 1 -- 172.21.15.161:0/3072435114 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e840a57d0 con 0x7f3e8815ec90 2024-05-22T21:25:21.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e8ca00640 1 -- 172.21.15.161:0/3072435114 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e840a5930 con 0x7f3e8815ec90 2024-05-22T21:25:21.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e8ca00640 1 -- 172.21.15.161:0/3072435114 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e840a5cc0 con 0x7f3e8815ec90 2024-05-22T21:25:21.782 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e90800640 1 -- 172.21.15.161:0/3072435114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8815ec90 msgr2=0x7f3e8815f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:21.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e90800640 1 --2- 172.21.15.161:0/3072435114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8815ec90 0x7f3e8815f120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009e50 tx=0x7f3e8401ca00 comp rx=0 tx=0).stop 2024-05-22T21:25:21.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e90800640 1 -- 172.21.15.161:0/3072435114 shutdown_connections 2024-05-22T21:25:21.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e90800640 1 --2- 172.21.15.161:0/3072435114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8815ec90 0x7f3e8815f120 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:21.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e90800640 1 --2- 172.21.15.161:0/3072435114 >> v2:172.21.15.179:3300/0 conn(0x7f3e880715a0 0x7f3e8815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:21.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e90800640 1 -- 172.21.15.161:0/3072435114 >> 172.21.15.161:0/3072435114 conn(0x7f3e880b0bc0 msgr2=0x7f3e880b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:21.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e90800640 1 -- 172.21.15.161:0/3072435114 shutdown_connections 2024-05-22T21:25:21.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e90800640 1 -- 172.21.15.161:0/3072435114 wait complete. 2024-05-22T21:25:21.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e90800640 1 Processor -- start 2024-05-22T21:25:21.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e90800640 1 -- start start 2024-05-22T21:25:21.785 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e90800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e88143ec0 0x7f3e881594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:21.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e90800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8815ec90 0x7f3e88141f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:21.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e90800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e88159d10 0x7f3e881424c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:21.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e90800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e880b3680 con 0x7f3e88159d10 2024-05-22T21:25:21.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e90800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e880b2a70 con 0x7f3e88143ec0 2024-05-22T21:25:21.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.782+0000 7f3e90800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e880b2d50 con 0x7f3e8815ec90 2024-05-22T21:25:21.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e8de00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e88143ec0 0x7f3e881594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:21.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e8e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e88159d10 0x7f3e881424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:21.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e8d400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8815ec90 0x7f3e88141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:21.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e8de00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e88143ec0 0x7f3e881594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:49762/0 (socket says 172.21.15.161:49762) 2024-05-22T21:25:21.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e8de00640 1 -- 172.21.15.161:0/1987807728 learned_addr learned my addr 172.21.15.161:0/1987807728 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:21.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e8d400640 1 -- 172.21.15.161:0/1987807728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e88143ec0 msgr2=0x7f3e881594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:21.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e8d400640 1 --2- 172.21.15.161:0/1987807728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e88143ec0 0x7f3e881594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:21.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e8d400640 1 -- 172.21.15.161:0/1987807728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e88159d10 msgr2=0x7f3e881424c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:21.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e8d400640 1 --2- 172.21.15.161:0/1987807728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e88159d10 0x7f3e881424c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:21.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e8d400640 1 -- 172.21.15.161:0/1987807728 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e78009420 con 0x7f3e8815ec90 2024-05-22T21:25:21.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e8de00640 1 --2- 172.21.15.161:0/1987807728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e88143ec0 0x7f3e881594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-22T21:25:21.787 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e8d400640 1 --2- 172.21.15.161:0/1987807728 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8815ec90 0x7f3e88141f80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e9b0 tx=0x7f3e7800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:21.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e82000640 1 -- 172.21.15.161:0/1987807728 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7800c9a0 con 0x7f3e8815ec90 2024-05-22T21:25:21.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e82000640 1 -- 172.21.15.161:0/1987807728 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7800cb00 con 0x7f3e8815ec90 2024-05-22T21:25:21.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e90800640 1 -- 172.21.15.161:0/1987807728 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e84009b30 con 0x7f3e8815ec90 2024-05-22T21:25:21.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e82000640 1 -- 172.21.15.161:0/1987807728 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e78013470 con 0x7f3e8815ec90 2024-05-22T21:25:21.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e90800640 1 -- 172.21.15.161:0/1987807728 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e880b3310 con 0x7f3e8815ec90 2024-05-22T21:25:21.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e90800640 1 -- 172.21.15.161:0/1987807728 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e88067220 con 0x7f3e8815ec90 2024-05-22T21:25:21.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e82000640 1 -- 172.21.15.161:0/1987807728 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3e780135d0 con 0x7f3e8815ec90 2024-05-22T21:25:21.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.786+0000 7f3e82000640 1 --2- 172.21.15.161:0/1987807728 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e6c066690 0x7f3e6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:21.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.790+0000 7f3e82000640 1 -- 172.21.15.161:0/1987807728 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3e780844a0 con 0x7f3e8815ec90 2024-05-22T21:25:21.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.790+0000 7f3e8de00640 1 --2- 172.21.15.161:0/1987807728 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e6c066690 0x7f3e6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:21.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.790+0000 7f3e8de00640 1 --2- 172.21.15.161:0/1987807728 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e6c066690 0x7f3e6c068b50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c000f70 tx=0x7f3e7c007f10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:21.793 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.794+0000 7f3e82000640 1 -- 172.21.15.161:0/1987807728 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3e78051b20 con 0x7f3e8815ec90 2024-05-22T21:25:21.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.942+0000 7f3e90800640 1 -- 172.21.15.161:0/1987807728 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e88072050 con 0x7f3e6c066690 2024-05-22T21:25:21.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.946+0000 7f3e82000640 1 -- 172.21.15.161:0/1987807728 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49822 (secure 0 0 0) 0x7f3e88072050 con 0x7f3e6c066690 2024-05-22T21:25:21.948 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:21.948 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:25:21.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.950+0000 7f3e90800640 1 -- 172.21.15.161:0/1987807728 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e6c066690 msgr2=0x7f3e6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:21.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.950+0000 7f3e90800640 1 --2- 172.21.15.161:0/1987807728 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e6c066690 0x7f3e6c068b50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c000f70 tx=0x7f3e7c007f10 comp rx=0 tx=0).stop 2024-05-22T21:25:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.950+0000 7f3e90800640 1 -- 172.21.15.161:0/1987807728 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8815ec90 msgr2=0x7f3e88141f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.950+0000 7f3e90800640 1 --2- 172.21.15.161:0/1987807728 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8815ec90 0x7f3e88141f80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e9b0 tx=0x7f3e7800ee70 comp rx=0 tx=0).stop 2024-05-22T21:25:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.950+0000 7f3e90800640 1 -- 172.21.15.161:0/1987807728 shutdown_connections 2024-05-22T21:25:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.950+0000 7f3e90800640 1 --2- 172.21.15.161:0/1987807728 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3e6c066690 0x7f3e6c068b50 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.950+0000 7f3e90800640 1 --2- 172.21.15.161:0/1987807728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e88159d10 0x7f3e881424c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.950+0000 7f3e90800640 1 --2- 172.21.15.161:0/1987807728 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8815ec90 0x7f3e88141f80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.950+0000 7f3e90800640 1 --2- 172.21.15.161:0/1987807728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e88143ec0 0x7f3e881594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.950+0000 7f3e90800640 1 -- 172.21.15.161:0/1987807728 >> 172.21.15.161:0/1987807728 conn(0x7f3e880b0bc0 msgr2=0x7f3e8815e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.950+0000 7f3e90800640 1 -- 172.21.15.161:0/1987807728 shutdown_connections 2024-05-22T21:25:21.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:21.950+0000 7f3e90800640 1 -- 172.21.15.161:0/1987807728 wait complete. 2024-05-22T21:25:21.968 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":553,"stamp":"2024-05-22T21:25:20.560984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":112219,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450280,"statfs":{"total":1600000000,"available":1485086973,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52488874,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-122,"num_write_kb":-85083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.239706"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":343,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-05-22T21:25:15.325279+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:25:15.325279+0000","last_peered":"2024-05-22T21:25:15.325279+0000","last_clean":"2024-05-22T21:25:15.325279+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:25:15.325279+0000","last_undegraded":"2024-05-22T21:25:15.325279+0000","last_fullsized":"2024-05-22T21:25:15.325279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574995,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9895,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185416,"statfs":{"total":200000000,"available":189866707,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.36699999999999999},"max":{"1min":0.95099999999999996,"5min":1.022,"15min":2.0369999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72099999999999997,"15min":0.70199999999999996},"min":{"1min":0.52500000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97899999999999998,"5min":1.1759999999999999,"15min":1.889},"last":0.74299999999999999}]},{"osd":1,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72099999999999997,"15min":0.74199999999999999},"min":{"1min":0.48099999999999998,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.93300000000000005,"5min":1.0329999999999999,"15min":3.0920000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.73399999999999999},"min":{"1min":0.60199999999999998,"5min":0.45200000000000001,"15min":0.39800000000000002},"max":{"1min":0.94699999999999995,"5min":1.1100000000000001,"15min":3.1499999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.501,"5min":0.433,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":1.002,"15min":1.923},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.40999999999999998},"max":{"1min":0.96699999999999997,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.69399999999999995}]},{"osd":3,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72399999999999998,"15min":0.70999999999999996},"min":{"1min":0.54900000000000004,"5min":0.44700000000000001,"15min":0.40100000000000002},"max":{"1min":0.88100000000000001,"5min":1.026,"15min":1.026},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.76200000000000001,"15min":0.77200000000000002},"min":{"1min":0.64700000000000002,"5min":0.501,"15min":0.44500000000000001},"max":{"1min":0.99399999999999999,"5min":1.0740000000000001,"15min":2.1389999999999998},"last":0.75800000000000001}]},{"osd":4,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.29399999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.96599999999999997},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.628,"15min":0.61899999999999999},"min":{"1min":0.504,"5min":0.40500000000000003,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.95999999999999996,"15min":1.018},"last":0.71599999999999997}]},{"osd":5,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.51700000000000002,"5min":0.39300000000000002,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0289999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67300000000000004,"15min":0.65200000000000002},"min":{"1min":0.46300000000000002,"5min":0.42099999999999999,"15min":0.33400000000000002},"max":{"1min":0.91200000000000003,"5min":0.99299999999999999,"15min":0.996},"last":0.82299999999999995}]},{"osd":6,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.50600000000000001,"5min":0.5,"15min":0.36799999999999999},"max":{"1min":0.92500000000000004,"5min":1.052,"15min":1.052},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69999999999999996,"15min":0.67000000000000004},"min":{"1min":0.60999999999999999,"5min":0.48699999999999999,"15min":0.41199999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.97699999999999998},"last":0.83499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574995,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19500,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175811,"statfs":{"total":200000000,"available":180031048,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69499999999999995,"15min":0.69399999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0649999999999999},"last":0.625},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.51900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95899999999999996,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.86399999999999999}]},{"osd":1,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69399999999999995,"15min":0.69099999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89500000000000002,"5min":0.94299999999999995,"15min":1.107},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.70599999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":0.93500000000000005,"5min":1.0009999999999999,"15min":1.081},"last":0.67200000000000004}]},{"osd":2,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.442},"max":{"1min":0.93000000000000005,"5min":1.026,"15min":1.087},"last":0.749},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75700000000000001,"15min":0.746},"min":{"1min":0.56499999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0089999999999999,"5min":1.419,"15min":1.419},"last":0.81999999999999995}]},{"osd":3,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.749,"15min":0.746},"min":{"1min":0.46600000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98599999999999999,"5min":1.28,"15min":1.415},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.55400000000000005,"5min":0.48999999999999999,"15min":0.45400000000000001},"max":{"1min":0.878,"5min":0.93300000000000005,"15min":1.788},"last":0.73399999999999999}]},{"osd":4,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.374},"max":{"1min":0.89300000000000002,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.67500000000000004},"min":{"1min":0.45600000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.90900000000000003,"5min":1.1850000000000001,"15min":2.718},"last":0.77600000000000002}]},{"osd":5,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61499999999999999,"15min":0.67800000000000005},"min":{"1min":0.42399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84799999999999998,"5min":0.97499999999999998,"15min":8.5749999999999993},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67900000000000005,"15min":0.69999999999999996},"min":{"1min":0.49299999999999999,"5min":0.44800000000000001,"15min":0.42099999999999999},"max":{"1min":0.92000000000000004,"5min":0.95799999999999996,"15min":1.1140000000000001},"last":0.70899999999999996}]},{"osd":7,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92800000000000005,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69699999999999995,"15min":0.72699999999999998},"min":{"1min":0.49099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85499999999999998,"5min":0.89700000000000002,"15min":1.8009999999999999},"last":0.60299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574995,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21549,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173763,"statfs":{"total":200000000,"available":177933516,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.48099999999999998,"5min":0.38200000000000001,"15min":0.32400000000000001},"max":{"1min":0.85599999999999998,"5min":0.98199999999999998,"15min":1.0409999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.54700000000000004,"5min":0.48699999999999999,"15min":0.378},"max":{"1min":0.91800000000000004,"5min":0.93700000000000006,"15min":1.0089999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71199999999999997,"15min":0.70499999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":1.0489999999999999,"15min":1.4339999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70999999999999996,"15min":0.69899999999999995},"min":{"1min":0.50700000000000001,"5min":0.45500000000000002,"15min":0.44},"max":{"1min":0.85499999999999998,"5min":0.92600000000000005,"15min":1.321},"last":0.503}]},{"osd":2,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.70899999999999996,"15min":0.70999999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.33100000000000002},"max":{"1min":0.96199999999999997,"5min":1.03,"15min":1.2969999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.74399999999999999,"15min":0.70499999999999996},"min":{"1min":0.56799999999999995,"5min":0.50700000000000001,"15min":0.42499999999999999},"max":{"1min":0.95999999999999996,"5min":0.98399999999999999,"15min":1.4910000000000001},"last":0.60999999999999999}]},{"osd":3,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73999999999999999,"15min":0.71899999999999997},"min":{"1min":0.54600000000000004,"5min":0.504,"15min":0.47999999999999998},"max":{"1min":0.92100000000000004,"5min":1.004,"15min":1.1879999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.749,"15min":0.73799999999999999},"min":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.996,"5min":0.996,"15min":1.111},"last":0.51600000000000001}]},{"osd":4,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59499999999999997,"15min":0.60399999999999998},"min":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.35099999999999998},"max":{"1min":0.74199999999999999,"5min":0.92400000000000004,"15min":1.272},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.627,"15min":0.628},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84999999999999998,"5min":0.95499999999999996,"15min":1.038},"last":0.52300000000000002}]},{"osd":6,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.371},"max":{"1min":0.872,"5min":0.96099999999999997,"15min":0.98899999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.48199999999999998,"5min":0.443,"15min":0.41099999999999998},"max":{"1min":0.94799999999999995,"5min":1.0069999999999999,"15min":1.2290000000000001},"last":0.623}]},{"osd":7,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72599999999999998,"15min":0.71399999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.47399999999999998},"max":{"1min":0.98299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76400000000000001,"15min":0.76800000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.46200000000000002},"max":{"1min":0.96799999999999997,"5min":1.018,"15min":1.2909999999999999},"last":0.67700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574995,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":7270,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":188041,"statfs":{"total":200000000,"available":192554899,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71499999999999997,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30399999999999999},"max":{"1min":0.97299999999999998,"5min":0.98899999999999999,"15min":6.6539999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.52100000000000002,"5min":0.46000000000000002,"15min":0.436},"max":{"1min":0.872,"5min":0.96299999999999997,"15min":5.4370000000000003},"last":0.65500000000000003}]},{"osd":1,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.39200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91400000000000003,"5min":0.95799999999999996,"15min":1.7749999999999999},"last":0.622},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69699999999999995,"15min":0.70899999999999996},"min":{"1min":0.48299999999999998,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":2.2210000000000001},"last":0.53500000000000003}]},{"osd":2,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72299999999999998,"15min":0.71199999999999997},"min":{"1min":0.50900000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":0.97299999999999998,"15min":2.4660000000000002},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.54400000000000004,"5min":0.45600000000000002,"15min":0.378},"max":{"1min":0.92900000000000005,"5min":1.01,"15min":2.2530000000000001},"last":0.67500000000000004}]},{"osd":3,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71299999999999997,"15min":0.70199999999999996},"min":{"1min":0.54200000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.998,"5min":1.042,"15min":1.857},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77100000000000002,"15min":0.78200000000000003},"min":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.47499999999999998},"max":{"1min":0.96599999999999997,"5min":1.012,"15min":1.897},"last":0.747}]},{"osd":5,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.629},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.379},"max":{"1min":0.95199999999999996,"5min":1.0169999999999999,"15min":1.1850000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.35599999999999998},"max":{"1min":0.872,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.73599999999999999}]},{"osd":6,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85599999999999998,"5min":1.044,"15min":1.0609999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.44,"5min":0.44,"15min":0.434},"max":{"1min":0.996,"5min":1.095,"15min":1.1419999999999999},"last":0.79700000000000004}]},{"osd":7,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.68500000000000005},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.40799999999999997},"max":{"1min":0.84799999999999998,"5min":1.1579999999999999,"15min":1.7390000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70999999999999996,"15min":0.746},"min":{"1min":0.505,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86499999999999999,"5min":1.069,"15min":1.4510000000000001},"last":0.83899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574995,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9260,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186051,"statfs":{"total":200000000,"available":190516796,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42799999999999999,"15min":0.41999999999999998},"min":{"1min":0.312,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59399999999999997,"5min":0.73099999999999998,"15min":0.86799999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47199999999999998,"15min":0.438},"min":{"1min":0.33600000000000002,"5min":0.30399999999999999,"15min":0.28399999999999997},"max":{"1min":0.65300000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.34999999999999998}]},{"osd":1,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48199999999999998,"15min":0.46400000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.26500000000000001},"max":{"1min":0.69799999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.52900000000000003},"min":{"1min":0.40699999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78700000000000003,"5min":0.80600000000000005,"15min":0.90000000000000002},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55100000000000005,"15min":0.51600000000000001},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89500000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.50800000000000001},"min":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.33600000000000002},"max":{"1min":0.69599999999999995,"5min":0.82799999999999996,"15min":0.93400000000000005},"last":0.36499999999999999}]},{"osd":4,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":0.71999999999999997},"min":{"1min":0.55900000000000005,"5min":0.48299999999999998,"15min":0.39400000000000002},"max":{"1min":1.0389999999999999,"5min":1.1839999999999999,"15min":1.2050000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.75700000000000001,"15min":0.73599999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.435},"max":{"1min":1,"5min":1.093,"15min":1.5760000000000001},"last":0.751}]},{"osd":5,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.45100000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.234},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.73899999999999999,"15min":0.72399999999999998},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.3049999999999999},"last":0.53000000000000003}]},{"osd":6,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73199999999999998,"15min":0.72699999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.44900000000000001},"max":{"1min":1.0149999999999999,"5min":1.0409999999999999,"15min":1.276},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76900000000000002,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.53400000000000003,"15min":0.50600000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.2869999999999999},"last":0.71699999999999997}]},{"osd":7,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79000000000000004,"15min":0.77400000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.495},"max":{"1min":1.071,"5min":1.0880000000000001,"15min":1.2709999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.76200000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0169999999999999,"5min":1.133,"15min":1.2969999999999999},"last":0.64100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574995,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21548,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173763,"statfs":{"total":200000000,"available":177934030,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.434,"15min":0.42299999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.23599999999999999},"max":{"1min":0.59899999999999998,"5min":0.66500000000000004,"15min":2.3090000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.28199999999999997,"5min":0.26900000000000002,"15min":0.23400000000000001},"max":{"1min":0.64100000000000001,"5min":0.80400000000000005,"15min":2.532},"last":0.53400000000000003}]},{"osd":1,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45600000000000002},"min":{"1min":0.30599999999999999,"5min":0.29899999999999999,"15min":0.29399999999999998},"max":{"1min":0.56899999999999995,"5min":0.68899999999999995,"15min":2.948},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.64800000000000002,"5min":0.71799999999999997,"15min":2.8159999999999998},"last":0.51000000000000001}]},{"osd":3,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.32700000000000001},"max":{"1min":0.62,"5min":0.83499999999999996,"15min":2.75},"last":0.435},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.5,"15min":0.48899999999999999},"min":{"1min":0.36099999999999999,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":3.0619999999999998},"last":0.45700000000000002}]},{"osd":4,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78000000000000003,"15min":0.74199999999999999},"min":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.52700000000000002},"max":{"1min":1.022,"5min":1.103,"15min":2.7610000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.78100000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.375},"max":{"1min":0.97499999999999998,"5min":1.208,"15min":2.9279999999999999},"last":0.82199999999999995}]},{"osd":5,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.753,"15min":0.73799999999999999},"min":{"1min":0.56799999999999995,"5min":0.51900000000000002,"15min":0.47299999999999998},"max":{"1min":0.89500000000000002,"5min":1.133,"15min":2.9790000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.45900000000000002},"max":{"1min":0.99099999999999999,"5min":1.0549999999999999,"15min":2.7069999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.73699999999999999},"min":{"1min":0.54600000000000004,"5min":0.51300000000000001,"15min":0.495},"max":{"1min":0.90000000000000002,"5min":1.0649999999999999,"15min":2.8580000000000001},"last":0.745},{"interface":"front","average":{"1min":0.752,"5min":0.748,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.496,"15min":0.46500000000000002},"max":{"1min":1.048,"5min":1.147,"15min":2.6880000000000002},"last":0.83999999999999997}]},{"osd":7,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.79700000000000004,"15min":0.77000000000000002},"min":{"1min":0.67000000000000004,"5min":0.50700000000000001,"15min":0.438},"max":{"1min":1.004,"5min":1.2210000000000001,"15min":3.0099999999999998},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81799999999999995,"15min":0.80100000000000005},"min":{"1min":0.67000000000000004,"5min":0.57299999999999995,"15min":0.499},"max":{"1min":1.006,"5min":1.1339999999999999,"15min":2.827},"last":0.88500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574995,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13356,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":181955,"statfs":{"total":200000000,"available":186322694,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.438,"15min":0.433},"min":{"1min":0.25900000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.65200000000000002,"5min":0.73499999999999999,"15min":1.3879999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47299999999999998,"15min":0.45600000000000002},"min":{"1min":0.28199999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.626,"5min":0.879,"15min":0.879},"last":0.498}]},{"osd":2,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.502,"15min":0.5},"min":{"1min":0.38200000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":1.76},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.495,"5min":0.48699999999999999,"15min":0.50600000000000001},"min":{"1min":0.36799999999999999,"5min":0.29799999999999999,"15min":0.28999999999999998},"max":{"1min":0.68700000000000006,"5min":0.83599999999999997,"15min":1.923},"last":0.35399999999999998}]},{"osd":3,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.41199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82899999999999996,"5min":0.82999999999999996,"15min":2.1030000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52600000000000002,"15min":0.50600000000000001},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.44800000000000001}]},{"osd":4,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.72199999999999998,"15min":0.74399999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.45100000000000001},"max":{"1min":0.88800000000000001,"5min":1.0780000000000001,"15min":2.1360000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.50800000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.022,"5min":1.1519999999999999,"15min":1.7829999999999999},"last":0.76500000000000001}]},{"osd":5,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":1.0549999999999999,"15min":1.819},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.746,"15min":0.748},"min":{"1min":0.61799999999999999,"5min":0.48799999999999999,"15min":0.40100000000000002},"max":{"1min":0.97899999999999998,"5min":1.0469999999999999,"15min":1.97},"last":0.749}]},{"osd":6,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.53300000000000003,"5min":0.52000000000000002,"15min":0.47299999999999998},"max":{"1min":0.93100000000000005,"5min":1.1359999999999999,"15min":1.272},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79000000000000004,"15min":0.77700000000000002},"min":{"1min":0.61099999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95399999999999996,"5min":1.1259999999999999,"15min":1.8029999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.55400000000000005,"5min":0.47999999999999998,"15min":0.42099999999999999},"max":{"1min":0.94699999999999995,"5min":0.995,"15min":2.0310000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76000000000000001,"15min":0.76500000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.52000000000000002},"max":{"1min":1.004,"5min":1.0660000000000001,"15min":2.0659999999999998},"last":0.77900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574995,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9836,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185475,"statfs":{"total":200000000,"available":189927283,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.43099999999999999},"min":{"1min":0.27000000000000002,"5min":0.24199999999999999,"15min":0.22800000000000001},"max":{"1min":0.63400000000000001,"5min":0.69999999999999996,"15min":0.88300000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.50800000000000001},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.33100000000000002},"max":{"1min":0.754,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.48399999999999999},"min":{"1min":0.33100000000000002,"5min":0.32100000000000001,"15min":0.29399999999999998},"max":{"1min":0.70299999999999996,"5min":0.85899999999999999,"15min":0.93799999999999994},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.46999999999999997,"15min":0.44900000000000001},"min":{"1min":0.39400000000000002,"5min":0.28899999999999998,"15min":0.27900000000000003},"max":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.92300000000000004},"last":0.53200000000000003}]},{"osd":3,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47599999999999998,"15min":0.45400000000000001},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":1.0760000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55700000000000005,"15min":0.53200000000000003},"min":{"1min":0.41699999999999998,"5min":0.38900000000000001,"15min":0.36899999999999999},"max":{"1min":0.79800000000000004,"5min":0.89700000000000002,"15min":0.98299999999999998},"last":0.51000000000000001}]},{"osd":4,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.72799999999999998},"min":{"1min":0.53200000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0189999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.748},"min":{"1min":0.50600000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.008,"5min":1.2330000000000001,"15min":1.2709999999999999},"last":0.85499999999999998}]},{"osd":5,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.72999999999999998},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":1.135,"15min":1.135},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.54200000000000004,"5min":0.499,"15min":0.47299999999999998},"max":{"1min":0.95099999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.83299999999999996}]},{"osd":6,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81799999999999995,"15min":0.79300000000000004},"min":{"1min":0.61399999999999999,"5min":0.54400000000000004,"15min":0.51500000000000001},"max":{"1min":0.97299999999999998,"5min":1.175,"15min":1.2090000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80100000000000005,"15min":0.78300000000000003},"min":{"1min":0.60199999999999998,"5min":0.50800000000000001,"15min":0.502},"max":{"1min":0.99099999999999999,"5min":1.3149999999999999,"15min":1.5},"last":0.89100000000000001}]},{"osd":7,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78200000000000003,"15min":0.753},"min":{"1min":0.53200000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.034,"5min":1.214,"15min":1.214},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.64100000000000001,"5min":0.52800000000000002,"15min":0.497},"max":{"1min":0.98599999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.91900000000000004}]}]}],"pool_statfs":[]}} 2024-05-22T21:25:21.970 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:25:22.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.106+0000 7f499fa00640 1 Processor -- start 2024-05-22T21:25:22.106 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.106+0000 7f499fa00640 1 -- start start 2024-05-22T21:25:22.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.106+0000 7f499fa00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f499815db70 0x7f4998142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.106+0000 7f499fa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4998140ed0 0x7f499806a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.106+0000 7f499fa00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4998071110 con 0x7f4998140a70 2024-05-22T21:25:22.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.106+0000 7f499fa00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49981430e0 con 0x7f4998140ed0 2024-05-22T21:25:22.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.106+0000 7f499fa00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49981433c0 con 0x7f499815db70 2024-05-22T21:25:22.108 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.106+0000 7f499da00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4998140ed0 0x7f499806a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.106+0000 7f499da00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4998140ed0 0x7f499806a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59258/0 (socket says 172.21.15.161:59258) 2024-05-22T21:25:22.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.106+0000 7f499da00640 1 -- 172.21.15.161:0/3605386232 learned_addr learned my addr 172.21.15.161:0/3605386232 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:22.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.106+0000 7f499d000640 1 --2- 172.21.15.161:0/3605386232 >> v2:172.21.15.179:3300/0 conn(0x7f499815db70 0x7f4998142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.106+0000 7f499da00640 1 -- 172.21.15.161:0/3605386232 >> v2:172.21.15.179:3300/0 conn(0x7f499815db70 msgr2=0x7f4998142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:22.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.106+0000 7f499da00640 1 --2- 172.21.15.161:0/3605386232 >> v2:172.21.15.179:3300/0 conn(0x7f499815db70 0x7f4998142ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.106+0000 7f499da00640 1 -- 172.21.15.161:0/3605386232 >> v1:172.21.15.161:6789/0 conn(0x7f4998140a70 legacy=0x7f49980cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.106+0000 7f499da00640 1 -- 172.21.15.161:0/3605386232 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49981436a0 con 0x7f4998140ed0 2024-05-22T21:25:22.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.106+0000 7f499d000640 1 --2- 172.21.15.161:0/3605386232 >> v2:172.21.15.179:3300/0 conn(0x7f499815db70 0x7f4998142ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:22.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.110+0000 7f499da00640 1 --2- 172.21.15.161:0/3605386232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4998140ed0 0x7f499806a490 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f49980b21b0 tx=0x7f499001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67328d17b2d18164 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:22.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.110+0000 7f4997400640 1 -- 172.21.15.161:0/3605386232 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49900ab600 con 0x7f4998140ed0 2024-05-22T21:25:22.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.110+0000 7f4997400640 1 -- 172.21.15.161:0/3605386232 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49900ab760 con 0x7f4998140ed0 2024-05-22T21:25:22.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.110+0000 7f4997400640 1 -- 172.21.15.161:0/3605386232 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49900abaf0 con 0x7f4998140ed0 2024-05-22T21:25:22.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.110+0000 7f499fa00640 1 -- 172.21.15.161:0/3605386232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4998140ed0 msgr2=0x7f499806a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.110+0000 7f499fa00640 1 --2- 172.21.15.161:0/3605386232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4998140ed0 0x7f499806a490 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f49980b21b0 tx=0x7f499001c4f0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.110+0000 7f499fa00640 1 -- 172.21.15.161:0/3605386232 shutdown_connections 2024-05-22T21:25:22.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.110+0000 7f499fa00640 1 --2- 172.21.15.161:0/3605386232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4998140ed0 0x7f499806a490 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.110+0000 7f499fa00640 1 --2- 172.21.15.161:0/3605386232 >> v2:172.21.15.179:3300/0 conn(0x7f499815db70 0x7f4998142ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.110+0000 7f499fa00640 1 -- 172.21.15.161:0/3605386232 >> 172.21.15.161:0/3605386232 conn(0x7f49980b5540 msgr2=0x7f49980b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:22.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.110+0000 7f499fa00640 1 -- 172.21.15.161:0/3605386232 shutdown_connections 2024-05-22T21:25:22.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.110+0000 7f499fa00640 1 -- 172.21.15.161:0/3605386232 wait complete. 2024-05-22T21:25:22.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499fa00640 1 Processor -- start 2024-05-22T21:25:22.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499fa00640 1 -- start start 2024-05-22T21:25:22.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499fa00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4998140a70 0x7f4998153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499fa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4998140ed0 0x7f4998137df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499fa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499815db70 0x7f49981f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499fa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499813e3b0 con 0x7f499815db70 2024-05-22T21:25:22.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499fa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4998153dc0 con 0x7f4998140ed0 2024-05-22T21:25:22.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499fa00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49980b3920 con 0x7f4998140a70 2024-05-22T21:25:22.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499da00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499815db70 0x7f49981f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499da00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499815db70 0x7f49981f6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59272/0 (socket says 172.21.15.161:59272) 2024-05-22T21:25:22.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499da00640 1 -- 172.21.15.161:0/360280960 learned_addr learned my addr 172.21.15.161:0/360280960 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:22.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499d000640 1 --2- 172.21.15.161:0/360280960 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4998140a70 0x7f4998153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f4997e00640 1 --2- 172.21.15.161:0/360280960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4998140ed0 0x7f4998137df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499da00640 1 -- 172.21.15.161:0/360280960 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4998140a70 msgr2=0x7f4998153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499da00640 1 --2- 172.21.15.161:0/360280960 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4998140a70 0x7f4998153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499da00640 1 -- 172.21.15.161:0/360280960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4998140ed0 msgr2=0x7f4998137df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499da00640 1 --2- 172.21.15.161:0/360280960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4998140ed0 0x7f4998137df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499da00640 1 -- 172.21.15.161:0/360280960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49900094d0 con 0x7f499815db70 2024-05-22T21:25:22.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499da00640 1 --2- 172.21.15.161:0/360280960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499815db70 0x7f49981f6540 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f49900ad040 tx=0x7f499001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:22.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f4994c00640 1 -- 172.21.15.161:0/360280960 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49900b4800 con 0x7f499815db70 2024-05-22T21:25:22.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f4994c00640 1 -- 172.21.15.161:0/360280960 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49900b4960 con 0x7f499815db70 2024-05-22T21:25:22.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499fa00640 1 -- 172.21.15.161:0/360280960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49980b3c00 con 0x7f499815db70 2024-05-22T21:25:22.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f4994c00640 1 -- 172.21.15.161:0/360280960 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49900b4cd0 con 0x7f499815db70 2024-05-22T21:25:22.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.114+0000 7f499fa00640 1 -- 172.21.15.161:0/360280960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49980b41c0 con 0x7f499815db70 2024-05-22T21:25:22.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.118+0000 7f499fa00640 1 -- 172.21.15.161:0/360280960 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499813e4f0 con 0x7f499815db70 2024-05-22T21:25:22.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.118+0000 7f4994c00640 1 -- 172.21.15.161:0/360280960 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f49900b3380 con 0x7f499815db70 2024-05-22T21:25:22.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.118+0000 7f4994c00640 1 --2- 172.21.15.161:0/360280960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4978066690 0x7f4978068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.118+0000 7f4994c00640 1 -- 172.21.15.161:0/360280960 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f499011dcb0 con 0x7f499815db70 2024-05-22T21:25:22.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.118+0000 7f499d000640 1 --2- 172.21.15.161:0/360280960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4978066690 0x7f4978068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.118+0000 7f499d000640 1 --2- 172.21.15.161:0/360280960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4978066690 0x7f4978068b50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4988009fd0 tx=0x7f49880097f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:22.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.122+0000 7f4994c00640 1 -- 172.21.15.161:0/360280960 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f499813e4f0 con 0x7f499815db70 2024-05-22T21:25:22.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.274+0000 7f499fa00640 1 -- 172.21.15.161:0/360280960 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f49980715c0 con 0x7f4978066690 2024-05-22T21:25:22.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.274+0000 7f4994c00640 1 -- 172.21.15.161:0/360280960 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49822 (secure 0 0 0) 0x7f49980715c0 con 0x7f4978066690 2024-05-22T21:25:22.277 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:22.278 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:25:22.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.278+0000 7f499fa00640 1 -- 172.21.15.161:0/360280960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4978066690 msgr2=0x7f4978068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.278+0000 7f499fa00640 1 --2- 172.21.15.161:0/360280960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4978066690 0x7f4978068b50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4988009fd0 tx=0x7f49880097f0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.278+0000 7f499fa00640 1 -- 172.21.15.161:0/360280960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499815db70 msgr2=0x7f49981f6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.278+0000 7f499fa00640 1 --2- 172.21.15.161:0/360280960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499815db70 0x7f49981f6540 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f49900ad040 tx=0x7f499001ad30 comp rx=0 tx=0).stop 2024-05-22T21:25:22.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.278+0000 7f499fa00640 1 -- 172.21.15.161:0/360280960 shutdown_connections 2024-05-22T21:25:22.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.278+0000 7f499fa00640 1 --2- 172.21.15.161:0/360280960 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4978066690 0x7f4978068b50 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.278+0000 7f499fa00640 1 --2- 172.21.15.161:0/360280960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f499815db70 0x7f49981f6540 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.278+0000 7f499fa00640 1 --2- 172.21.15.161:0/360280960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4998140ed0 0x7f4998137df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.278+0000 7f499fa00640 1 --2- 172.21.15.161:0/360280960 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4998140a70 0x7f4998153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.278+0000 7f499fa00640 1 -- 172.21.15.161:0/360280960 >> 172.21.15.161:0/360280960 conn(0x7f49980b5540 msgr2=0x7f49980afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:22.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.278+0000 7f499fa00640 1 -- 172.21.15.161:0/360280960 shutdown_connections 2024-05-22T21:25:22.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.278+0000 7f499fa00640 1 -- 172.21.15.161:0/360280960 wait complete. 2024-05-22T21:25:22.297 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":553,"stamp":"2024-05-22T21:25:20.560984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":112219,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450280,"statfs":{"total":1600000000,"available":1485086973,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52488874,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-122,"num_write_kb":-85083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.239706"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":343,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-05-22T21:25:15.325279+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:25:15.325279+0000","last_peered":"2024-05-22T21:25:15.325279+0000","last_clean":"2024-05-22T21:25:15.325279+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:25:15.325279+0000","last_undegraded":"2024-05-22T21:25:15.325279+0000","last_fullsized":"2024-05-22T21:25:15.325279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574995,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9895,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185416,"statfs":{"total":200000000,"available":189866707,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.36699999999999999},"max":{"1min":0.95099999999999996,"5min":1.022,"15min":2.0369999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72099999999999997,"15min":0.70199999999999996},"min":{"1min":0.52500000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97899999999999998,"5min":1.1759999999999999,"15min":1.889},"last":0.74299999999999999}]},{"osd":1,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72099999999999997,"15min":0.74199999999999999},"min":{"1min":0.48099999999999998,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.93300000000000005,"5min":1.0329999999999999,"15min":3.0920000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.73399999999999999},"min":{"1min":0.60199999999999998,"5min":0.45200000000000001,"15min":0.39800000000000002},"max":{"1min":0.94699999999999995,"5min":1.1100000000000001,"15min":3.1499999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.501,"5min":0.433,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":1.002,"15min":1.923},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.40999999999999998},"max":{"1min":0.96699999999999997,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.69399999999999995}]},{"osd":3,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72399999999999998,"15min":0.70999999999999996},"min":{"1min":0.54900000000000004,"5min":0.44700000000000001,"15min":0.40100000000000002},"max":{"1min":0.88100000000000001,"5min":1.026,"15min":1.026},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.76200000000000001,"15min":0.77200000000000002},"min":{"1min":0.64700000000000002,"5min":0.501,"15min":0.44500000000000001},"max":{"1min":0.99399999999999999,"5min":1.0740000000000001,"15min":2.1389999999999998},"last":0.75800000000000001}]},{"osd":4,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.29399999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.96599999999999997},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.628,"15min":0.61899999999999999},"min":{"1min":0.504,"5min":0.40500000000000003,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.95999999999999996,"15min":1.018},"last":0.71599999999999997}]},{"osd":5,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.51700000000000002,"5min":0.39300000000000002,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0289999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67300000000000004,"15min":0.65200000000000002},"min":{"1min":0.46300000000000002,"5min":0.42099999999999999,"15min":0.33400000000000002},"max":{"1min":0.91200000000000003,"5min":0.99299999999999999,"15min":0.996},"last":0.82299999999999995}]},{"osd":6,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.50600000000000001,"5min":0.5,"15min":0.36799999999999999},"max":{"1min":0.92500000000000004,"5min":1.052,"15min":1.052},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69999999999999996,"15min":0.67000000000000004},"min":{"1min":0.60999999999999999,"5min":0.48699999999999999,"15min":0.41199999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.97699999999999998},"last":0.83499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574995,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19500,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175811,"statfs":{"total":200000000,"available":180031048,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69499999999999995,"15min":0.69399999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0649999999999999},"last":0.625},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.51900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95899999999999996,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.86399999999999999}]},{"osd":1,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69399999999999995,"15min":0.69099999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89500000000000002,"5min":0.94299999999999995,"15min":1.107},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.70599999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":0.93500000000000005,"5min":1.0009999999999999,"15min":1.081},"last":0.67200000000000004}]},{"osd":2,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.442},"max":{"1min":0.93000000000000005,"5min":1.026,"15min":1.087},"last":0.749},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75700000000000001,"15min":0.746},"min":{"1min":0.56499999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0089999999999999,"5min":1.419,"15min":1.419},"last":0.81999999999999995}]},{"osd":3,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.749,"15min":0.746},"min":{"1min":0.46600000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98599999999999999,"5min":1.28,"15min":1.415},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.55400000000000005,"5min":0.48999999999999999,"15min":0.45400000000000001},"max":{"1min":0.878,"5min":0.93300000000000005,"15min":1.788},"last":0.73399999999999999}]},{"osd":4,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.374},"max":{"1min":0.89300000000000002,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.67500000000000004},"min":{"1min":0.45600000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.90900000000000003,"5min":1.1850000000000001,"15min":2.718},"last":0.77600000000000002}]},{"osd":5,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61499999999999999,"15min":0.67800000000000005},"min":{"1min":0.42399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84799999999999998,"5min":0.97499999999999998,"15min":8.5749999999999993},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67900000000000005,"15min":0.69999999999999996},"min":{"1min":0.49299999999999999,"5min":0.44800000000000001,"15min":0.42099999999999999},"max":{"1min":0.92000000000000004,"5min":0.95799999999999996,"15min":1.1140000000000001},"last":0.70899999999999996}]},{"osd":7,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92800000000000005,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69699999999999995,"15min":0.72699999999999998},"min":{"1min":0.49099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85499999999999998,"5min":0.89700000000000002,"15min":1.8009999999999999},"last":0.60299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574995,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21549,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173763,"statfs":{"total":200000000,"available":177933516,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.48099999999999998,"5min":0.38200000000000001,"15min":0.32400000000000001},"max":{"1min":0.85599999999999998,"5min":0.98199999999999998,"15min":1.0409999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.54700000000000004,"5min":0.48699999999999999,"15min":0.378},"max":{"1min":0.91800000000000004,"5min":0.93700000000000006,"15min":1.0089999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71199999999999997,"15min":0.70499999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":1.0489999999999999,"15min":1.4339999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70999999999999996,"15min":0.69899999999999995},"min":{"1min":0.50700000000000001,"5min":0.45500000000000002,"15min":0.44},"max":{"1min":0.85499999999999998,"5min":0.92600000000000005,"15min":1.321},"last":0.503}]},{"osd":2,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.70899999999999996,"15min":0.70999999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.33100000000000002},"max":{"1min":0.96199999999999997,"5min":1.03,"15min":1.2969999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.74399999999999999,"15min":0.70499999999999996},"min":{"1min":0.56799999999999995,"5min":0.50700000000000001,"15min":0.42499999999999999},"max":{"1min":0.95999999999999996,"5min":0.98399999999999999,"15min":1.4910000000000001},"last":0.60999999999999999}]},{"osd":3,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73999999999999999,"15min":0.71899999999999997},"min":{"1min":0.54600000000000004,"5min":0.504,"15min":0.47999999999999998},"max":{"1min":0.92100000000000004,"5min":1.004,"15min":1.1879999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.749,"15min":0.73799999999999999},"min":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.996,"5min":0.996,"15min":1.111},"last":0.51600000000000001}]},{"osd":4,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59499999999999997,"15min":0.60399999999999998},"min":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.35099999999999998},"max":{"1min":0.74199999999999999,"5min":0.92400000000000004,"15min":1.272},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.627,"15min":0.628},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84999999999999998,"5min":0.95499999999999996,"15min":1.038},"last":0.52300000000000002}]},{"osd":6,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.371},"max":{"1min":0.872,"5min":0.96099999999999997,"15min":0.98899999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.48199999999999998,"5min":0.443,"15min":0.41099999999999998},"max":{"1min":0.94799999999999995,"5min":1.0069999999999999,"15min":1.2290000000000001},"last":0.623}]},{"osd":7,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72599999999999998,"15min":0.71399999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.47399999999999998},"max":{"1min":0.98299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76400000000000001,"15min":0.76800000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.46200000000000002},"max":{"1min":0.96799999999999997,"5min":1.018,"15min":1.2909999999999999},"last":0.67700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574995,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":7270,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":188041,"statfs":{"total":200000000,"available":192554899,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71499999999999997,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30399999999999999},"max":{"1min":0.97299999999999998,"5min":0.98899999999999999,"15min":6.6539999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.52100000000000002,"5min":0.46000000000000002,"15min":0.436},"max":{"1min":0.872,"5min":0.96299999999999997,"15min":5.4370000000000003},"last":0.65500000000000003}]},{"osd":1,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.39200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91400000000000003,"5min":0.95799999999999996,"15min":1.7749999999999999},"last":0.622},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69699999999999995,"15min":0.70899999999999996},"min":{"1min":0.48299999999999998,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":2.2210000000000001},"last":0.53500000000000003}]},{"osd":2,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72299999999999998,"15min":0.71199999999999997},"min":{"1min":0.50900000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":0.97299999999999998,"15min":2.4660000000000002},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.54400000000000004,"5min":0.45600000000000002,"15min":0.378},"max":{"1min":0.92900000000000005,"5min":1.01,"15min":2.2530000000000001},"last":0.67500000000000004}]},{"osd":3,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71299999999999997,"15min":0.70199999999999996},"min":{"1min":0.54200000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.998,"5min":1.042,"15min":1.857},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77100000000000002,"15min":0.78200000000000003},"min":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.47499999999999998},"max":{"1min":0.96599999999999997,"5min":1.012,"15min":1.897},"last":0.747}]},{"osd":5,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.629},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.379},"max":{"1min":0.95199999999999996,"5min":1.0169999999999999,"15min":1.1850000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.35599999999999998},"max":{"1min":0.872,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.73599999999999999}]},{"osd":6,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85599999999999998,"5min":1.044,"15min":1.0609999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.44,"5min":0.44,"15min":0.434},"max":{"1min":0.996,"5min":1.095,"15min":1.1419999999999999},"last":0.79700000000000004}]},{"osd":7,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.68500000000000005},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.40799999999999997},"max":{"1min":0.84799999999999998,"5min":1.1579999999999999,"15min":1.7390000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70999999999999996,"15min":0.746},"min":{"1min":0.505,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86499999999999999,"5min":1.069,"15min":1.4510000000000001},"last":0.83899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574995,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9260,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186051,"statfs":{"total":200000000,"available":190516796,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42799999999999999,"15min":0.41999999999999998},"min":{"1min":0.312,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59399999999999997,"5min":0.73099999999999998,"15min":0.86799999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47199999999999998,"15min":0.438},"min":{"1min":0.33600000000000002,"5min":0.30399999999999999,"15min":0.28399999999999997},"max":{"1min":0.65300000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.34999999999999998}]},{"osd":1,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48199999999999998,"15min":0.46400000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.26500000000000001},"max":{"1min":0.69799999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.52900000000000003},"min":{"1min":0.40699999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78700000000000003,"5min":0.80600000000000005,"15min":0.90000000000000002},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55100000000000005,"15min":0.51600000000000001},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89500000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.50800000000000001},"min":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.33600000000000002},"max":{"1min":0.69599999999999995,"5min":0.82799999999999996,"15min":0.93400000000000005},"last":0.36499999999999999}]},{"osd":4,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":0.71999999999999997},"min":{"1min":0.55900000000000005,"5min":0.48299999999999998,"15min":0.39400000000000002},"max":{"1min":1.0389999999999999,"5min":1.1839999999999999,"15min":1.2050000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.75700000000000001,"15min":0.73599999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.435},"max":{"1min":1,"5min":1.093,"15min":1.5760000000000001},"last":0.751}]},{"osd":5,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.45100000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.234},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.73899999999999999,"15min":0.72399999999999998},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.3049999999999999},"last":0.53000000000000003}]},{"osd":6,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73199999999999998,"15min":0.72699999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.44900000000000001},"max":{"1min":1.0149999999999999,"5min":1.0409999999999999,"15min":1.276},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76900000000000002,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.53400000000000003,"15min":0.50600000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.2869999999999999},"last":0.71699999999999997}]},{"osd":7,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79000000000000004,"15min":0.77400000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.495},"max":{"1min":1.071,"5min":1.0880000000000001,"15min":1.2709999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.76200000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0169999999999999,"5min":1.133,"15min":1.2969999999999999},"last":0.64100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574995,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21548,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173763,"statfs":{"total":200000000,"available":177934030,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.434,"15min":0.42299999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.23599999999999999},"max":{"1min":0.59899999999999998,"5min":0.66500000000000004,"15min":2.3090000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.28199999999999997,"5min":0.26900000000000002,"15min":0.23400000000000001},"max":{"1min":0.64100000000000001,"5min":0.80400000000000005,"15min":2.532},"last":0.53400000000000003}]},{"osd":1,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45600000000000002},"min":{"1min":0.30599999999999999,"5min":0.29899999999999999,"15min":0.29399999999999998},"max":{"1min":0.56899999999999995,"5min":0.68899999999999995,"15min":2.948},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.64800000000000002,"5min":0.71799999999999997,"15min":2.8159999999999998},"last":0.51000000000000001}]},{"osd":3,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.32700000000000001},"max":{"1min":0.62,"5min":0.83499999999999996,"15min":2.75},"last":0.435},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.5,"15min":0.48899999999999999},"min":{"1min":0.36099999999999999,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":3.0619999999999998},"last":0.45700000000000002}]},{"osd":4,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78000000000000003,"15min":0.74199999999999999},"min":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.52700000000000002},"max":{"1min":1.022,"5min":1.103,"15min":2.7610000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.78100000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.375},"max":{"1min":0.97499999999999998,"5min":1.208,"15min":2.9279999999999999},"last":0.82199999999999995}]},{"osd":5,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.753,"15min":0.73799999999999999},"min":{"1min":0.56799999999999995,"5min":0.51900000000000002,"15min":0.47299999999999998},"max":{"1min":0.89500000000000002,"5min":1.133,"15min":2.9790000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.45900000000000002},"max":{"1min":0.99099999999999999,"5min":1.0549999999999999,"15min":2.7069999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.73699999999999999},"min":{"1min":0.54600000000000004,"5min":0.51300000000000001,"15min":0.495},"max":{"1min":0.90000000000000002,"5min":1.0649999999999999,"15min":2.8580000000000001},"last":0.745},{"interface":"front","average":{"1min":0.752,"5min":0.748,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.496,"15min":0.46500000000000002},"max":{"1min":1.048,"5min":1.147,"15min":2.6880000000000002},"last":0.83999999999999997}]},{"osd":7,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.79700000000000004,"15min":0.77000000000000002},"min":{"1min":0.67000000000000004,"5min":0.50700000000000001,"15min":0.438},"max":{"1min":1.004,"5min":1.2210000000000001,"15min":3.0099999999999998},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81799999999999995,"15min":0.80100000000000005},"min":{"1min":0.67000000000000004,"5min":0.57299999999999995,"15min":0.499},"max":{"1min":1.006,"5min":1.1339999999999999,"15min":2.827},"last":0.88500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574995,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13356,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":181955,"statfs":{"total":200000000,"available":186322694,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.438,"15min":0.433},"min":{"1min":0.25900000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.65200000000000002,"5min":0.73499999999999999,"15min":1.3879999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47299999999999998,"15min":0.45600000000000002},"min":{"1min":0.28199999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.626,"5min":0.879,"15min":0.879},"last":0.498}]},{"osd":2,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.502,"15min":0.5},"min":{"1min":0.38200000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":1.76},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.495,"5min":0.48699999999999999,"15min":0.50600000000000001},"min":{"1min":0.36799999999999999,"5min":0.29799999999999999,"15min":0.28999999999999998},"max":{"1min":0.68700000000000006,"5min":0.83599999999999997,"15min":1.923},"last":0.35399999999999998}]},{"osd":3,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.41199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82899999999999996,"5min":0.82999999999999996,"15min":2.1030000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52600000000000002,"15min":0.50600000000000001},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.44800000000000001}]},{"osd":4,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.72199999999999998,"15min":0.74399999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.45100000000000001},"max":{"1min":0.88800000000000001,"5min":1.0780000000000001,"15min":2.1360000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.50800000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.022,"5min":1.1519999999999999,"15min":1.7829999999999999},"last":0.76500000000000001}]},{"osd":5,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":1.0549999999999999,"15min":1.819},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.746,"15min":0.748},"min":{"1min":0.61799999999999999,"5min":0.48799999999999999,"15min":0.40100000000000002},"max":{"1min":0.97899999999999998,"5min":1.0469999999999999,"15min":1.97},"last":0.749}]},{"osd":6,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.53300000000000003,"5min":0.52000000000000002,"15min":0.47299999999999998},"max":{"1min":0.93100000000000005,"5min":1.1359999999999999,"15min":1.272},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79000000000000004,"15min":0.77700000000000002},"min":{"1min":0.61099999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95399999999999996,"5min":1.1259999999999999,"15min":1.8029999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.55400000000000005,"5min":0.47999999999999998,"15min":0.42099999999999999},"max":{"1min":0.94699999999999995,"5min":0.995,"15min":2.0310000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76000000000000001,"15min":0.76500000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.52000000000000002},"max":{"1min":1.004,"5min":1.0660000000000001,"15min":2.0659999999999998},"last":0.77900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574995,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9836,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185475,"statfs":{"total":200000000,"available":189927283,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.43099999999999999},"min":{"1min":0.27000000000000002,"5min":0.24199999999999999,"15min":0.22800000000000001},"max":{"1min":0.63400000000000001,"5min":0.69999999999999996,"15min":0.88300000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.50800000000000001},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.33100000000000002},"max":{"1min":0.754,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55700000000000005}]},{"osd":2,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.48399999999999999},"min":{"1min":0.33100000000000002,"5min":0.32100000000000001,"15min":0.29399999999999998},"max":{"1min":0.70299999999999996,"5min":0.85899999999999999,"15min":0.93799999999999994},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.46999999999999997,"15min":0.44900000000000001},"min":{"1min":0.39400000000000002,"5min":0.28899999999999998,"15min":0.27900000000000003},"max":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.92300000000000004},"last":0.53200000000000003}]},{"osd":3,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47599999999999998,"15min":0.45400000000000001},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":1.0760000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55700000000000005,"15min":0.53200000000000003},"min":{"1min":0.41699999999999998,"5min":0.38900000000000001,"15min":0.36899999999999999},"max":{"1min":0.79800000000000004,"5min":0.89700000000000002,"15min":0.98299999999999998},"last":0.51000000000000001}]},{"osd":4,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.72799999999999998},"min":{"1min":0.53200000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0189999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.748},"min":{"1min":0.50600000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.008,"5min":1.2330000000000001,"15min":1.2709999999999999},"last":0.85499999999999998}]},{"osd":5,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.72999999999999998},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":1.135,"15min":1.135},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.54200000000000004,"5min":0.499,"15min":0.47299999999999998},"max":{"1min":0.95099999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.83299999999999996}]},{"osd":6,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81799999999999995,"15min":0.79300000000000004},"min":{"1min":0.61399999999999999,"5min":0.54400000000000004,"15min":0.51500000000000001},"max":{"1min":0.97299999999999998,"5min":1.175,"15min":1.2090000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80100000000000005,"15min":0.78300000000000003},"min":{"1min":0.60199999999999998,"5min":0.50800000000000001,"15min":0.502},"max":{"1min":0.99099999999999999,"5min":1.3149999999999999,"15min":1.5},"last":0.89100000000000001}]},{"osd":7,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78200000000000003,"15min":0.753},"min":{"1min":0.53200000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.034,"5min":1.214,"15min":1.214},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.64100000000000001,"5min":0.52800000000000002,"15min":0.497},"max":{"1min":0.98599999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.91900000000000004}]}]}],"pool_statfs":[]}} 2024-05-22T21:25:22.299 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-22T21:25:22.299 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:25:22.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.434+0000 7f1e21a00640 1 Processor -- start 2024-05-22T21:25:22.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.434+0000 7f1e21a00640 1 -- start start 2024-05-22T21:25:22.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.434+0000 7f1e21a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1e1c15b420 0x7f1e1c15b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.434+0000 7f1e21a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c06de20 con 0x7f1e1c144550 2024-05-22T21:25:22.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.434+0000 7f1e21a00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c0cbcb0 con 0x7f1e1c06be70 2024-05-22T21:25:22.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.434+0000 7f1e21a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1c06ba50 con 0x7f1e1c15b420 2024-05-22T21:25:22.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.434+0000 7f1e1b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1e1c15b420 0x7f1e1c15b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.434+0000 7f1e1be00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1e1c144550 0x7f1e1c0cb5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59490/0 (socket says 172.21.15.161:59490) 2024-05-22T21:25:22.436 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.434+0000 7f1e1b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1e1c15b420 0x7f1e1c15b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59274/0 (socket says 172.21.15.161:59274) 2024-05-22T21:25:22.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.434+0000 7f1e1be00640 1 -- 172.21.15.161:0/875834070 learned_addr learned my addr 172.21.15.161:0/875834070 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:22.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.434+0000 7f1e1b400640 1 -- 172.21.15.161:0/875834070 >> v1:172.21.15.179:6789/0 conn(0x7f1e1c06be70 legacy=0x7f1e1c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.434+0000 7f1e1b400640 1 -- 172.21.15.161:0/875834070 >> v1:172.21.15.161:6789/0 conn(0x7f1e1c144550 legacy=0x7f1e1c0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.434+0000 7f1e1aa00640 1 -- 172.21.15.161:0/875834070 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206004188 0 0) 0x7f1e1c06de20 con 0x7f1e1c144550 2024-05-22T21:25:22.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.434+0000 7f1e1b400640 1 -- 172.21.15.161:0/875834070 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c15be10 con 0x7f1e1c15b420 2024-05-22T21:25:22.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.438+0000 7f1e1b400640 1 --2- 172.21.15.161:0/875834070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e1c15b420 0x7f1e1c15b8d0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009740 tx=0x7f1e0801c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7c72c2b4b6690c server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:22.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.438+0000 7f1e1aa00640 1 -- 172.21.15.161:0/875834070 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e080a4ea0 con 0x7f1e1c15b420 2024-05-22T21:25:22.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.438+0000 7f1e1aa00640 1 -- 172.21.15.161:0/875834070 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e080a7050 con 0x7f1e1c15b420 2024-05-22T21:25:22.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.438+0000 7f1e1aa00640 1 -- 172.21.15.161:0/875834070 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e080ab820 con 0x7f1e1c15b420 2024-05-22T21:25:22.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.438+0000 7f1e21a00640 1 -- 172.21.15.161:0/875834070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e1c15b420 msgr2=0x7f1e1c15b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.438+0000 7f1e21a00640 1 --2- 172.21.15.161:0/875834070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e1c15b420 0x7f1e1c15b8d0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009740 tx=0x7f1e0801c210 comp rx=0 tx=0).stop 2024-05-22T21:25:22.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.438+0000 7f1e21a00640 1 -- 172.21.15.161:0/875834070 shutdown_connections 2024-05-22T21:25:22.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.438+0000 7f1e21a00640 1 --2- 172.21.15.161:0/875834070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e1c15b420 0x7f1e1c15b8d0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.438+0000 7f1e21a00640 1 -- 172.21.15.161:0/875834070 >> 172.21.15.161:0/875834070 conn(0x7f1e1c06a4d0 msgr2=0x7f1e1c0b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:22.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.438+0000 7f1e21a00640 1 -- 172.21.15.161:0/875834070 shutdown_connections 2024-05-22T21:25:22.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.438+0000 7f1e21a00640 1 -- 172.21.15.161:0/875834070 wait complete. 2024-05-22T21:25:22.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e21a00640 1 Processor -- start 2024-05-22T21:25:22.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e21a00640 1 -- start start 2024-05-22T21:25:22.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e21a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e1c06be70 0x7f1e1c0ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e21a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e1c144550 0x7f1e1c0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e21a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e1c15b420 0x7f1e1c1f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e21a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c1f8ad0 con 0x7f1e1c144550 2024-05-22T21:25:22.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e21a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c1f8d20 con 0x7f1e1c15b420 2024-05-22T21:25:22.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e21a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c1f9000 con 0x7f1e1c06be70 2024-05-22T21:25:22.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e1b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e1c144550 0x7f1e1c0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e1b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e1c144550 0x7f1e1c0b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59288/0 (socket says 172.21.15.161:59288) 2024-05-22T21:25:22.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e1be00640 1 --2- 172.21.15.161:0/1828130036 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e1c06be70 0x7f1e1c0ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e21000640 1 --2- 172.21.15.161:0/1828130036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e1c15b420 0x7f1e1c1f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e1b400640 1 -- 172.21.15.161:0/1828130036 learned_addr learned my addr 172.21.15.161:0/1828130036 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:22.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e1b400640 1 -- 172.21.15.161:0/1828130036 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e1c06be70 msgr2=0x7f1e1c0ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e1b400640 1 --2- 172.21.15.161:0/1828130036 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e1c06be70 0x7f1e1c0ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e1b400640 1 -- 172.21.15.161:0/1828130036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e1c15b420 msgr2=0x7f1e1c1f7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e1b400640 1 --2- 172.21.15.161:0/1828130036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e1c15b420 0x7f1e1c1f7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e1b400640 1 -- 172.21.15.161:0/1828130036 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e08009420 con 0x7f1e1c144550 2024-05-22T21:25:22.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e1b400640 1 --2- 172.21.15.161:0/1828130036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e1c144550 0x7f1e1c0b0750 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f1e08018e20 tx=0x7f1e08018fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:22.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e13e00640 1 -- 172.21.15.161:0/1828130036 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e080b4610 con 0x7f1e1c144550 2024-05-22T21:25:22.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e13e00640 1 -- 172.21.15.161:0/1828130036 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e080abe90 con 0x7f1e1c144550 2024-05-22T21:25:22.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e13e00640 1 -- 172.21.15.161:0/1828130036 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e080b49d0 con 0x7f1e1c144550 2024-05-22T21:25:22.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.442+0000 7f1e21a00640 1 -- 172.21.15.161:0/1828130036 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1c1f92e0 con 0x7f1e1c144550 2024-05-22T21:25:22.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.446+0000 7f1e21a00640 1 -- 172.21.15.161:0/1828130036 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c0aed30 con 0x7f1e1c144550 2024-05-22T21:25:22.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.446+0000 7f1e21a00640 1 -- 172.21.15.161:0/1828130036 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1c1f9800 con 0x7f1e1c144550 2024-05-22T21:25:22.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.446+0000 7f1e13e00640 1 -- 172.21.15.161:0/1828130036 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1e080b2040 con 0x7f1e1c144550 2024-05-22T21:25:22.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.446+0000 7f1e13e00640 1 --2- 172.21.15.161:0/1828130036 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1dfc066690 0x7f1dfc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.446+0000 7f1e13e00640 1 -- 172.21.15.161:0/1828130036 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1e0811dff0 con 0x7f1e1c144550 2024-05-22T21:25:22.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.446+0000 7f1e1be00640 1 --2- 172.21.15.161:0/1828130036 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1dfc066690 0x7f1dfc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.446+0000 7f1e1be00640 1 --2- 172.21.15.161:0/1828130036 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1dfc066690 0x7f1dfc068b50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c000f70 tx=0x7f1e0c007240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:22.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.450+0000 7f1e13e00640 1 -- 172.21.15.161:0/1828130036 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f1e0814fcf0 con 0x7f1e1c144550 2024-05-22T21:25:22.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.602+0000 7f1e21a00640 1 -- 172.21.15.161:0/1828130036 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e1c0ce8a0 con 0x7f1dfc066690 2024-05-22T21:25:22.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.606+0000 7f1e13e00640 1 -- 172.21.15.161:0/1828130036 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49807 (secure 0 0 0) 0x7f1e0c001510 con 0x7f1dfc066690 2024-05-22T21:25:22.607 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:22.607 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:25:22.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.606+0000 7f1e21a00640 1 -- 172.21.15.161:0/1828130036 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1dfc066690 msgr2=0x7f1dfc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.606+0000 7f1e21a00640 1 --2- 172.21.15.161:0/1828130036 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1dfc066690 0x7f1dfc068b50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c000f70 tx=0x7f1e0c007240 comp rx=0 tx=0).stop 2024-05-22T21:25:22.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.606+0000 7f1e21a00640 1 -- 172.21.15.161:0/1828130036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e1c144550 msgr2=0x7f1e1c0b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.606+0000 7f1e21a00640 1 --2- 172.21.15.161:0/1828130036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e1c144550 0x7f1e1c0b0750 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f1e08018e20 tx=0x7f1e08018fb0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.606+0000 7f1e21a00640 1 -- 172.21.15.161:0/1828130036 shutdown_connections 2024-05-22T21:25:22.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.606+0000 7f1e21a00640 1 --2- 172.21.15.161:0/1828130036 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1dfc066690 0x7f1dfc068b50 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.606+0000 7f1e21a00640 1 --2- 172.21.15.161:0/1828130036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e1c15b420 0x7f1e1c1f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.606+0000 7f1e21a00640 1 --2- 172.21.15.161:0/1828130036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e1c144550 0x7f1e1c0b0750 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.606+0000 7f1e21a00640 1 --2- 172.21.15.161:0/1828130036 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e1c06be70 0x7f1e1c0ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.606+0000 7f1e21a00640 1 -- 172.21.15.161:0/1828130036 >> 172.21.15.161:0/1828130036 conn(0x7f1e1c06a4d0 msgr2=0x7f1e1c0ad7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:22.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.606+0000 7f1e21a00640 1 -- 172.21.15.161:0/1828130036 shutdown_connections 2024-05-22T21:25:22.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.606+0000 7f1e21a00640 1 -- 172.21.15.161:0/1828130036 wait complete. 2024-05-22T21:25:22.627 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":554,"stamp":"2024-05-22T21:25:22.561255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9840,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552659,"statfs":{"total":1600000000,"available":1589923811,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52489015,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-125,"num_write_kb":-85084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001292"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":346,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-22T21:25:18.708120+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:25:18.708120+0000","last_peered":"2024-05-22T21:25:18.708120+0000","last_clean":"2024-05-22T21:25:18.708120+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:25:18.708120+0000","last_undegraded":"2024-05-22T21:25:18.708120+0000","last_fullsized":"2024-05-22T21:25:18.708120+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574996,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1662,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193650,"statfs":{"total":200000000,"available":198297623,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.36699999999999999},"max":{"1min":0.95099999999999996,"5min":1.022,"15min":2.0369999999999999},"last":0.75},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72099999999999997,"15min":0.70199999999999996},"min":{"1min":0.52500000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97899999999999998,"5min":1.1759999999999999,"15min":1.889},"last":0.87}]},{"osd":1,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72099999999999997,"15min":0.74199999999999999},"min":{"1min":0.48099999999999998,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.93300000000000005,"5min":1.0329999999999999,"15min":3.0920000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.73399999999999999},"min":{"1min":0.60199999999999998,"5min":0.45200000000000001,"15min":0.39800000000000002},"max":{"1min":0.94699999999999995,"5min":1.1100000000000001,"15min":3.1499999999999999},"last":0.70399999999999996}]},{"osd":2,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.501,"5min":0.433,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":1.002,"15min":1.923},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.40999999999999998},"max":{"1min":0.96699999999999997,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.78600000000000003}]},{"osd":3,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72399999999999998,"15min":0.70999999999999996},"min":{"1min":0.54900000000000004,"5min":0.44700000000000001,"15min":0.40100000000000002},"max":{"1min":0.88100000000000001,"5min":1.026,"15min":1.026},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.76200000000000001,"15min":0.77200000000000002},"min":{"1min":0.64700000000000002,"5min":0.501,"15min":0.44500000000000001},"max":{"1min":0.99399999999999999,"5min":1.0740000000000001,"15min":2.1389999999999998},"last":0.81699999999999995}]},{"osd":4,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.29399999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.96599999999999997},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.628,"15min":0.61899999999999999},"min":{"1min":0.504,"5min":0.40500000000000003,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.95999999999999996,"15min":1.018},"last":0.55500000000000005}]},{"osd":5,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.51700000000000002,"5min":0.39300000000000002,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0289999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67300000000000004,"15min":0.65200000000000002},"min":{"1min":0.46300000000000002,"5min":0.42099999999999999,"15min":0.33400000000000002},"max":{"1min":0.91200000000000003,"5min":0.99299999999999999,"15min":0.996},"last":0.76500000000000001}]},{"osd":6,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.50600000000000001,"5min":0.5,"15min":0.36799999999999999},"max":{"1min":0.92500000000000004,"5min":1.052,"15min":1.052},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69999999999999996,"15min":0.67000000000000004},"min":{"1min":0.60999999999999999,"5min":0.48699999999999999,"15min":0.41199999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.97699999999999998},"last":0.72999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69499999999999995,"15min":0.69399999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0649999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.51900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95899999999999996,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.443}]},{"osd":1,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69399999999999995,"15min":0.69099999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89500000000000002,"5min":0.94299999999999995,"15min":1.107},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.70599999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":0.93500000000000005,"5min":1.0009999999999999,"15min":1.081},"last":0.503}]},{"osd":2,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.442},"max":{"1min":0.93000000000000005,"5min":1.026,"15min":1.087},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75700000000000001,"15min":0.746},"min":{"1min":0.56499999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0089999999999999,"5min":1.419,"15min":1.419},"last":0.71499999999999997}]},{"osd":3,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.749,"15min":0.746},"min":{"1min":0.46600000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98599999999999999,"5min":1.28,"15min":1.415},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.55400000000000005,"5min":0.48999999999999999,"15min":0.45400000000000001},"max":{"1min":0.878,"5min":0.93300000000000005,"15min":1.788},"last":0.57899999999999996}]},{"osd":4,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.374},"max":{"1min":0.89300000000000002,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.62},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.67500000000000004},"min":{"1min":0.45600000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.90900000000000003,"5min":1.1850000000000001,"15min":2.718},"last":0.75800000000000001}]},{"osd":5,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61499999999999999,"15min":0.67800000000000005},"min":{"1min":0.42399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84799999999999998,"5min":0.97499999999999998,"15min":8.5749999999999993},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67900000000000005,"15min":0.69999999999999996},"min":{"1min":0.49299999999999999,"5min":0.44800000000000001,"15min":0.42099999999999999},"max":{"1min":0.92000000000000004,"5min":0.95799999999999996,"15min":1.1140000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92800000000000005,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69699999999999995,"15min":0.72699999999999998},"min":{"1min":0.49099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85499999999999998,"5min":0.89700000000000002,"15min":1.8009999999999999},"last":0.68799999999999994}]}]},{"osd":5,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.48099999999999998,"5min":0.38200000000000001,"15min":0.32400000000000001},"max":{"1min":0.85599999999999998,"5min":0.98199999999999998,"15min":1.0409999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.54700000000000004,"5min":0.48699999999999999,"15min":0.378},"max":{"1min":0.91800000000000004,"5min":0.93700000000000006,"15min":1.0089999999999999},"last":0.60399999999999998}]},{"osd":1,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71199999999999997,"15min":0.70499999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":1.0489999999999999,"15min":1.4339999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70999999999999996,"15min":0.69899999999999995},"min":{"1min":0.50700000000000001,"5min":0.45500000000000002,"15min":0.44},"max":{"1min":0.85499999999999998,"5min":0.92600000000000005,"15min":1.321},"last":0.66800000000000004}]},{"osd":2,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.70899999999999996,"15min":0.70999999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.33100000000000002},"max":{"1min":0.96199999999999997,"5min":1.03,"15min":1.2969999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.74399999999999999,"15min":0.70499999999999996},"min":{"1min":0.56799999999999995,"5min":0.50700000000000001,"15min":0.42499999999999999},"max":{"1min":0.95999999999999996,"5min":0.98399999999999999,"15min":1.4910000000000001},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73999999999999999,"15min":0.71899999999999997},"min":{"1min":0.54600000000000004,"5min":0.504,"15min":0.47999999999999998},"max":{"1min":0.92100000000000004,"5min":1.004,"15min":1.1879999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.749,"15min":0.73799999999999999},"min":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.996,"5min":0.996,"15min":1.111},"last":0.81999999999999995}]},{"osd":4,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59499999999999997,"15min":0.60399999999999998},"min":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.35099999999999998},"max":{"1min":0.74199999999999999,"5min":0.92400000000000004,"15min":1.272},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.627,"15min":0.628},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84999999999999998,"5min":0.95499999999999996,"15min":1.038},"last":0.71999999999999997}]},{"osd":6,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.371},"max":{"1min":0.872,"5min":0.96099999999999997,"15min":0.98899999999999999},"last":0.745},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.48199999999999998,"5min":0.443,"15min":0.41099999999999998},"max":{"1min":0.94799999999999995,"5min":1.0069999999999999,"15min":1.2290000000000001},"last":0.76100000000000001}]},{"osd":7,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72599999999999998,"15min":0.71399999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.47399999999999998},"max":{"1min":0.98299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76400000000000001,"15min":0.76800000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.46200000000000002},"max":{"1min":0.96799999999999997,"5min":1.018,"15min":1.2909999999999999},"last":0.93200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574996,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71499999999999997,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30399999999999999},"max":{"1min":0.97299999999999998,"5min":0.98899999999999999,"15min":6.6539999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.52100000000000002,"5min":0.46000000000000002,"15min":0.436},"max":{"1min":0.872,"5min":0.96299999999999997,"15min":5.4370000000000003},"last":0.74199999999999999}]},{"osd":1,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.39200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91400000000000003,"5min":0.95799999999999996,"15min":1.7749999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69699999999999995,"15min":0.70899999999999996},"min":{"1min":0.48299999999999998,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":2.2210000000000001},"last":0.71299999999999997}]},{"osd":2,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72299999999999998,"15min":0.71199999999999997},"min":{"1min":0.50900000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":0.97299999999999998,"15min":2.4660000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.54400000000000004,"5min":0.45600000000000002,"15min":0.378},"max":{"1min":0.92900000000000005,"5min":1.01,"15min":2.2530000000000001},"last":0.59799999999999998}]},{"osd":3,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71299999999999997,"15min":0.70199999999999996},"min":{"1min":0.54200000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.998,"5min":1.042,"15min":1.857},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77100000000000002,"15min":0.78200000000000003},"min":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.47499999999999998},"max":{"1min":0.96599999999999997,"5min":1.012,"15min":1.897},"last":0.76500000000000001}]},{"osd":5,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.629},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.379},"max":{"1min":0.95199999999999996,"5min":1.0169999999999999,"15min":1.1850000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.35599999999999998},"max":{"1min":0.872,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.65600000000000003}]},{"osd":6,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85599999999999998,"5min":1.044,"15min":1.0609999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.44,"5min":0.44,"15min":0.434},"max":{"1min":0.996,"5min":1.095,"15min":1.1419999999999999},"last":0.63500000000000001}]},{"osd":7,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.68500000000000005},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.40799999999999997},"max":{"1min":0.84799999999999998,"5min":1.1579999999999999,"15min":1.7390000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70999999999999996,"15min":0.746},"min":{"1min":0.505,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86499999999999999,"5min":1.069,"15min":1.4510000000000001},"last":0.67500000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42799999999999999,"15min":0.41999999999999998},"min":{"1min":0.312,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59399999999999997,"5min":0.73099999999999998,"15min":0.86799999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47199999999999998,"15min":0.438},"min":{"1min":0.33600000000000002,"5min":0.30399999999999999,"15min":0.28399999999999997},"max":{"1min":0.65300000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54100000000000004}]},{"osd":1,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48199999999999998,"15min":0.46400000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.26500000000000001},"max":{"1min":0.69799999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.443},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.52900000000000003},"min":{"1min":0.40699999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78700000000000003,"5min":0.80600000000000005,"15min":0.90000000000000002},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55100000000000005,"15min":0.51600000000000001},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89500000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.50800000000000001},"min":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.33600000000000002},"max":{"1min":0.69599999999999995,"5min":0.82799999999999996,"15min":0.93400000000000005},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":0.71999999999999997},"min":{"1min":0.55900000000000005,"5min":0.48299999999999998,"15min":0.39400000000000002},"max":{"1min":1.0389999999999999,"5min":1.1839999999999999,"15min":1.2050000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.75700000000000001,"15min":0.73599999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.435},"max":{"1min":1,"5min":1.093,"15min":1.5760000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.45100000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.234},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.73899999999999999,"15min":0.72399999999999998},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.3049999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73199999999999998,"15min":0.72699999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.44900000000000001},"max":{"1min":1.0149999999999999,"5min":1.0409999999999999,"15min":1.276},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76900000000000002,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.53400000000000003,"15min":0.50600000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.2869999999999999},"last":0.84099999999999997}]},{"osd":7,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79000000000000004,"15min":0.77400000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.495},"max":{"1min":1.071,"5min":1.0880000000000001,"15min":1.2709999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.76200000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0169999999999999,"5min":1.133,"15min":1.2969999999999999},"last":0.81599999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574996,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.434,"15min":0.42299999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.23599999999999999},"max":{"1min":0.59899999999999998,"5min":0.66500000000000004,"15min":2.3090000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.28199999999999997,"5min":0.26900000000000002,"15min":0.23400000000000001},"max":{"1min":0.64100000000000001,"5min":0.80400000000000005,"15min":2.532},"last":0.314}]},{"osd":1,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45600000000000002},"min":{"1min":0.30599999999999999,"5min":0.29899999999999999,"15min":0.29399999999999998},"max":{"1min":0.56899999999999995,"5min":0.68899999999999995,"15min":2.948},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.64800000000000002,"5min":0.71799999999999997,"15min":2.8159999999999998},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.32700000000000001},"max":{"1min":0.62,"5min":0.83499999999999996,"15min":2.75},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.5,"15min":0.48899999999999999},"min":{"1min":0.36099999999999999,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":3.0619999999999998},"last":0.45900000000000002}]},{"osd":4,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78000000000000003,"15min":0.74199999999999999},"min":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.52700000000000002},"max":{"1min":1.022,"5min":1.103,"15min":2.7610000000000001},"last":0.628},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.78100000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.375},"max":{"1min":0.97499999999999998,"5min":1.208,"15min":2.9279999999999999},"last":0.60299999999999998}]},{"osd":5,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.753,"15min":0.73799999999999999},"min":{"1min":0.56799999999999995,"5min":0.51900000000000002,"15min":0.47299999999999998},"max":{"1min":0.89500000000000002,"5min":1.133,"15min":2.9790000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.45900000000000002},"max":{"1min":0.99099999999999999,"5min":1.0549999999999999,"15min":2.7069999999999999},"last":0.68100000000000005}]},{"osd":6,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.73699999999999999},"min":{"1min":0.54600000000000004,"5min":0.51300000000000001,"15min":0.495},"max":{"1min":0.90000000000000002,"5min":1.0649999999999999,"15min":2.8580000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.748,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.496,"15min":0.46500000000000002},"max":{"1min":1.048,"5min":1.147,"15min":2.6880000000000002},"last":0.58599999999999997}]},{"osd":7,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.79700000000000004,"15min":0.77000000000000002},"min":{"1min":0.67000000000000004,"5min":0.50700000000000001,"15min":0.438},"max":{"1min":1.004,"5min":1.2210000000000001,"15min":3.0099999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81799999999999995,"15min":0.80100000000000005},"min":{"1min":0.67000000000000004,"5min":0.57299999999999995,"15min":0.499},"max":{"1min":1.006,"5min":1.1339999999999999,"15min":2.827},"last":0.61499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.438,"15min":0.433},"min":{"1min":0.25900000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.65200000000000002,"5min":0.73499999999999999,"15min":1.3879999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47299999999999998,"15min":0.45600000000000002},"min":{"1min":0.28199999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.626,"5min":0.879,"15min":0.879},"last":0.38400000000000001}]},{"osd":2,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.502,"15min":0.5},"min":{"1min":0.38200000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":1.76},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.495,"5min":0.48699999999999999,"15min":0.50600000000000001},"min":{"1min":0.36799999999999999,"5min":0.29799999999999999,"15min":0.28999999999999998},"max":{"1min":0.68700000000000006,"5min":0.83599999999999997,"15min":1.923},"last":0.32500000000000001}]},{"osd":3,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.41199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82899999999999996,"5min":0.82999999999999996,"15min":2.1030000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52600000000000002,"15min":0.50600000000000001},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.42199999999999999}]},{"osd":4,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.72199999999999998,"15min":0.74399999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.45100000000000001},"max":{"1min":0.88800000000000001,"5min":1.0780000000000001,"15min":2.1360000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.50800000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.022,"5min":1.1519999999999999,"15min":1.7829999999999999},"last":0.73899999999999999}]},{"osd":5,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":1.0549999999999999,"15min":1.819},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.746,"15min":0.748},"min":{"1min":0.61799999999999999,"5min":0.48799999999999999,"15min":0.40100000000000002},"max":{"1min":0.97899999999999998,"5min":1.0469999999999999,"15min":1.97},"last":0.84399999999999997}]},{"osd":6,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.53300000000000003,"5min":0.52000000000000002,"15min":0.47299999999999998},"max":{"1min":0.93100000000000005,"5min":1.1359999999999999,"15min":1.272},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79000000000000004,"15min":0.77700000000000002},"min":{"1min":0.61099999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95399999999999996,"5min":1.1259999999999999,"15min":1.8029999999999999},"last":0.76900000000000002}]},{"osd":7,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.55400000000000005,"5min":0.47999999999999998,"15min":0.42099999999999999},"max":{"1min":0.94699999999999995,"5min":0.995,"15min":2.0310000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76000000000000001,"15min":0.76500000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.52000000000000002},"max":{"1min":1.004,"5min":1.0660000000000001,"15min":2.0659999999999998},"last":0.82099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574996,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1661,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193650,"statfs":{"total":200000000,"available":198298242,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.43099999999999999},"min":{"1min":0.27000000000000002,"5min":0.24199999999999999,"15min":0.22800000000000001},"max":{"1min":0.63400000000000001,"5min":0.69999999999999996,"15min":0.88300000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.50800000000000001},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.33100000000000002},"max":{"1min":0.754,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51600000000000001}]},{"osd":2,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.48399999999999999},"min":{"1min":0.33100000000000002,"5min":0.32100000000000001,"15min":0.29399999999999998},"max":{"1min":0.70299999999999996,"5min":0.85899999999999999,"15min":0.93799999999999994},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.495,"5min":0.46999999999999997,"15min":0.44900000000000001},"min":{"1min":0.39400000000000002,"5min":0.28899999999999998,"15min":0.27900000000000003},"max":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.92300000000000004},"last":0.34399999999999997}]},{"osd":3,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47599999999999998,"15min":0.45400000000000001},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":1.0760000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55700000000000005,"15min":0.53200000000000003},"min":{"1min":0.41699999999999998,"5min":0.38900000000000001,"15min":0.36899999999999999},"max":{"1min":0.79800000000000004,"5min":0.89700000000000002,"15min":0.98299999999999998},"last":0.55100000000000005}]},{"osd":4,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.72799999999999998},"min":{"1min":0.53200000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0189999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.628},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.748},"min":{"1min":0.50600000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.008,"5min":1.2330000000000001,"15min":1.2709999999999999},"last":0.68799999999999994}]},{"osd":5,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.72999999999999998},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":1.135,"15min":1.135},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.54200000000000004,"5min":0.499,"15min":0.47299999999999998},"max":{"1min":0.95099999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.745}]},{"osd":6,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81799999999999995,"15min":0.79300000000000004},"min":{"1min":0.61399999999999999,"5min":0.54400000000000004,"15min":0.51500000000000001},"max":{"1min":0.97299999999999998,"5min":1.175,"15min":1.2090000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80100000000000005,"15min":0.78300000000000003},"min":{"1min":0.60199999999999998,"5min":0.50800000000000001,"15min":0.502},"max":{"1min":0.99099999999999999,"5min":1.3149999999999999,"15min":1.5},"last":0.64600000000000002}]},{"osd":7,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78200000000000003,"15min":0.753},"min":{"1min":0.53200000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.034,"5min":1.214,"15min":1.214},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.64100000000000001,"5min":0.52800000000000002,"15min":0.497},"max":{"1min":0.98599999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.78500000000000003}]}]}],"pool_statfs":[]}} 2024-05-22T21:25:22.628 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-22T21:25:22.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.762+0000 7fb6cd200640 1 Processor -- start 2024-05-22T21:25:22.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.762+0000 7fb6cd200640 1 -- start start 2024-05-22T21:25:22.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.762+0000 7fb6cd200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb6c815dd50 0x7fb6c806d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6cd200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb6c815d5e0 0x7fb6c815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6cd200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c8071670 con 0x7fb6c815e130 2024-05-22T21:25:22.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6cd200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c806de40 con 0x7fb6c815d5e0 2024-05-22T21:25:22.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6cd200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c8070a60 con 0x7fb6c815dd50 2024-05-22T21:25:22.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6c6800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb6c815d5e0 0x7fb6c815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6c6800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb6c815d5e0 0x7fb6c815b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59304/0 (socket says 172.21.15.161:59304) 2024-05-22T21:25:22.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6bd400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb6c815e130 0x7fb6c815ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59496/0 (socket says 172.21.15.161:59496) 2024-05-22T21:25:22.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6c6800640 1 -- 172.21.15.161:0/3394065876 learned_addr learned my addr 172.21.15.161:0/3394065876 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:22.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6c5e00640 1 --2- 172.21.15.161:0/3394065876 >> v2:172.21.15.179:3300/0 conn(0x7fb6c815dd50 0x7fb6c806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6c6800640 1 -- 172.21.15.161:0/3394065876 >> v2:172.21.15.179:3300/0 conn(0x7fb6c815dd50 msgr2=0x7fb6c806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:22.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6c6800640 1 --2- 172.21.15.161:0/3394065876 >> v2:172.21.15.179:3300/0 conn(0x7fb6c815dd50 0x7fb6c806d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6c6800640 1 -- 172.21.15.161:0/3394065876 >> v1:172.21.15.161:6789/0 conn(0x7fb6c815e130 legacy=0x7fb6c815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6c5400640 1 -- 172.21.15.161:0/3394065876 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4294815924 0 0) 0x7fb6c8071670 con 0x7fb6c815e130 2024-05-22T21:25:22.767 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6c6800640 1 -- 172.21.15.161:0/3394065876 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c8070d40 con 0x7fb6c815d5e0 2024-05-22T21:25:22.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6c6800640 1 --2- 172.21.15.161:0/3394065876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c815d5e0 0x7fb6c815b6c0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8072330 tx=0x7fb6c001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3eb707adcc6a13 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:22.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6c5400640 1 -- 172.21.15.161:0/3394065876 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c00a5880 con 0x7fb6c815d5e0 2024-05-22T21:25:22.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6c5400640 1 -- 172.21.15.161:0/3394065876 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c00a59e0 con 0x7fb6c815d5e0 2024-05-22T21:25:22.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.766+0000 7fb6c5400640 1 -- 172.21.15.161:0/3394065876 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c00a5d70 con 0x7fb6c815d5e0 2024-05-22T21:25:22.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6cd200640 1 -- 172.21.15.161:0/3394065876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c815d5e0 msgr2=0x7fb6c815b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6cd200640 1 --2- 172.21.15.161:0/3394065876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c815d5e0 0x7fb6c815b6c0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8072330 tx=0x7fb6c001c9c0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6cd200640 1 -- 172.21.15.161:0/3394065876 shutdown_connections 2024-05-22T21:25:22.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6cd200640 1 --2- 172.21.15.161:0/3394065876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c815d5e0 0x7fb6c815b6c0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6cd200640 1 --2- 172.21.15.161:0/3394065876 >> v2:172.21.15.179:3300/0 conn(0x7fb6c815dd50 0x7fb6c806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6cd200640 1 -- 172.21.15.161:0/3394065876 >> 172.21.15.161:0/3394065876 conn(0x7fb6c80b0bc0 msgr2=0x7fb6c80b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:22.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6cd200640 1 -- 172.21.15.161:0/3394065876 shutdown_connections 2024-05-22T21:25:22.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6cd200640 1 -- 172.21.15.161:0/3394065876 wait complete. 2024-05-22T21:25:22.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6cd200640 1 Processor -- start 2024-05-22T21:25:22.771 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6cd200640 1 -- start start 2024-05-22T21:25:22.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6cd200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6c815d5e0 0x7fb6c80b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6cd200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c815dd50 0x7fb6c8156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6cd200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c815e130 0x7fb6c8158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6cd200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c8158da0 con 0x7fb6c815e130 2024-05-22T21:25:22.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6cd200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c80b3b20 con 0x7fb6c815dd50 2024-05-22T21:25:22.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6cd200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c8158ff0 con 0x7fb6c815d5e0 2024-05-22T21:25:22.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6c6800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c815e130 0x7fb6c8158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6c5e00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6c815d5e0 0x7fb6c80b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6c6800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c815e130 0x7fb6c8158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59312/0 (socket says 172.21.15.161:59312) 2024-05-22T21:25:22.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6c6800640 1 -- 172.21.15.161:0/445974765 learned_addr learned my addr 172.21.15.161:0/445974765 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:22.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.770+0000 7fb6bd400640 1 --2- 172.21.15.161:0/445974765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c815dd50 0x7fb6c8156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.774+0000 7fb6c6800640 1 -- 172.21.15.161:0/445974765 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6c815d5e0 msgr2=0x7fb6c80b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.774+0000 7fb6c6800640 1 --2- 172.21.15.161:0/445974765 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6c815d5e0 0x7fb6c80b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.774+0000 7fb6c6800640 1 -- 172.21.15.161:0/445974765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c815dd50 msgr2=0x7fb6c8156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.774+0000 7fb6c6800640 1 --2- 172.21.15.161:0/445974765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c815dd50 0x7fb6c8156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.774+0000 7fb6c6800640 1 -- 172.21.15.161:0/445974765 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c0009b30 con 0x7fb6c815e130 2024-05-22T21:25:22.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.774+0000 7fb6c6800640 1 --2- 172.21.15.161:0/445974765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c815e130 0x7fb6c8158860 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fb6c001a5f0 tx=0x7fb6c0019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:22.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.774+0000 7fb6bea00640 1 -- 172.21.15.161:0/445974765 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c00b8670 con 0x7fb6c815e130 2024-05-22T21:25:22.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.774+0000 7fb6bea00640 1 -- 172.21.15.161:0/445974765 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c00b87d0 con 0x7fb6c815e130 2024-05-22T21:25:22.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.774+0000 7fb6bea00640 1 -- 172.21.15.161:0/445974765 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c00b8b80 con 0x7fb6c815e130 2024-05-22T21:25:22.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.774+0000 7fb6cd200640 1 -- 172.21.15.161:0/445974765 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c80b1040 con 0x7fb6c815e130 2024-05-22T21:25:22.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.774+0000 7fb6cd200640 1 -- 172.21.15.161:0/445974765 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c80b1600 con 0x7fb6c815e130 2024-05-22T21:25:22.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.774+0000 7fb6cd200640 1 -- 172.21.15.161:0/445974765 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c8073ab0 con 0x7fb6c815e130 2024-05-22T21:25:22.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.774+0000 7fb6bea00640 1 -- 172.21.15.161:0/445974765 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb6c00b8ce0 con 0x7fb6c815e130 2024-05-22T21:25:22.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.774+0000 7fb6bea00640 1 --2- 172.21.15.161:0/445974765 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb6a4066690 0x7fb6a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:22.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.774+0000 7fb6bea00640 1 -- 172.21.15.161:0/445974765 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb6c011f390 con 0x7fb6c815e130 2024-05-22T21:25:22.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.774+0000 7fb6c5e00640 1 --2- 172.21.15.161:0/445974765 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb6a4066690 0x7fb6a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:22.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.778+0000 7fb6c5e00640 1 --2- 172.21.15.161:0/445974765 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb6a4066690 0x7fb6a4068b50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80af600 tx=0x7fb6b8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:22.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.778+0000 7fb6bea00640 1 -- 172.21.15.161:0/445974765 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fb6c00eca10 con 0x7fb6c815e130 2024-05-22T21:25:22.934 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.934+0000 7fb6cd200640 1 -- 172.21.15.161:0/445974765 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6c8067220 con 0x7fb6c815e130 2024-05-22T21:25:22.935 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.934+0000 7fb6bea00640 1 -- 172.21.15.161:0/445974765 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+12579 (secure 0 0 0) 0x7fb6c00f0030 con 0x7fb6c815e130 2024-05-22T21:25:22.935 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:22.936 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":197,"fsid":"d6fa8528-b1bf-41b6-9f76-c0d715394087","created":"2024-05-22T21:13:05.415577+0000","modified":"2024-05-22T21:25:19.703882+0000","last_up_change":"2024-05-22T21:13:10.955061+0000","last_in_change":"2024-05-22T21:13:07.945035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-22T21:13:12.449493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":23413},{"type":"v1","addr":"172.21.15.161:6809","nonce":23413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":23413},{"type":"v1","addr":"172.21.15.161:6811","nonce":23413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":23413},{"type":"v1","addr":"172.21.15.161:6815","nonce":23413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":23413},{"type":"v1","addr":"172.21.15.161:6813","nonce":23413}]},"public_addr":"172.21.15.161:6809/23413","cluster_addr":"172.21.15.161:6811/23413","heartbeat_back_addr":"172.21.15.161:6815/23413","heartbeat_front_addr":"172.21.15.161:6813/23413","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":23412},{"type":"v1","addr":"172.21.15.161:6801","nonce":23412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":23412},{"type":"v1","addr":"172.21.15.161:6803","nonce":23412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":23412},{"type":"v1","addr":"172.21.15.161:6807","nonce":23412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":23412},{"type":"v1","addr":"172.21.15.161:6805","nonce":23412}]},"public_addr":"172.21.15.161:6801/23412","cluster_addr":"172.21.15.161:6803/23412","heartbeat_back_addr":"172.21.15.161:6807/23412","heartbeat_front_addr":"172.21.15.161:6805/23412","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":23414},{"type":"v1","addr":"172.21.15.161:6825","nonce":23414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":23414},{"type":"v1","addr":"172.21.15.161:6827","nonce":23414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":23414},{"type":"v1","addr":"172.21.15.161:6831","nonce":23414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":23414},{"type":"v1","addr":"172.21.15.161:6829","nonce":23414}]},"public_addr":"172.21.15.161:6825/23414","cluster_addr":"172.21.15.161:6827/23414","heartbeat_back_addr":"172.21.15.161:6831/23414","heartbeat_front_addr":"172.21.15.161:6829/23414","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":23415},{"type":"v1","addr":"172.21.15.161:6817","nonce":23415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":23415},{"type":"v1","addr":"172.21.15.161:6819","nonce":23415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":23415},{"type":"v1","addr":"172.21.15.161:6823","nonce":23415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":23415},{"type":"v1","addr":"172.21.15.161:6821","nonce":23415}]},"public_addr":"172.21.15.161:6817/23415","cluster_addr":"172.21.15.161:6819/23415","heartbeat_back_addr":"172.21.15.161:6823/23415","heartbeat_front_addr":"172.21.15.161:6821/23415","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":23606},{"type":"v1","addr":"172.21.15.179:6801","nonce":23606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":23606},{"type":"v1","addr":"172.21.15.179:6803","nonce":23606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":23606},{"type":"v1","addr":"172.21.15.179:6807","nonce":23606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":23606},{"type":"v1","addr":"172.21.15.179:6805","nonce":23606}]},"public_addr":"172.21.15.179:6801/23606","cluster_addr":"172.21.15.179:6803/23606","heartbeat_back_addr":"172.21.15.179:6807/23606","heartbeat_front_addr":"172.21.15.179:6805/23606","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":23607},{"type":"v1","addr":"172.21.15.179:6809","nonce":23607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":23607},{"type":"v1","addr":"172.21.15.179:6811","nonce":23607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":23607},{"type":"v1","addr":"172.21.15.179:6815","nonce":23607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":23607},{"type":"v1","addr":"172.21.15.179:6813","nonce":23607}]},"public_addr":"172.21.15.179:6809/23607","cluster_addr":"172.21.15.179:6811/23607","heartbeat_back_addr":"172.21.15.179:6815/23607","heartbeat_front_addr":"172.21.15.179:6813/23607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":23608},{"type":"v1","addr":"172.21.15.179:6822","nonce":23608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6825","nonce":23608},{"type":"v1","addr":"172.21.15.179:6827","nonce":23608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":23608},{"type":"v1","addr":"172.21.15.179:6831","nonce":23608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":23608},{"type":"v1","addr":"172.21.15.179:6829","nonce":23608}]},"public_addr":"172.21.15.179:6822/23608","cluster_addr":"172.21.15.179:6827/23608","heartbeat_back_addr":"172.21.15.179:6831/23608","heartbeat_front_addr":"172.21.15.179:6829/23608","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":23609},{"type":"v1","addr":"172.21.15.179:6817","nonce":23609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":23609},{"type":"v1","addr":"172.21.15.179:6819","nonce":23609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":23609},{"type":"v1","addr":"172.21.15.179:6826","nonce":23609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6821","nonce":23609},{"type":"v1","addr":"172.21.15.179:6823","nonce":23609}]},"public_addr":"172.21.15.179:6817/23609","cluster_addr":"172.21.15.179:6819/23609","heartbeat_back_addr":"172.21.15.179:6826/23609","heartbeat_front_addr":"172.21.15.179:6823/23609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.198975+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.091069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.207346+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.202216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.137328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.127225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.120135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-22T21:13:09.159112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:6837/2172511349":"2024-05-23T21:25:16.689719+0000","172.21.15.161:6835/2172511349":"2024-05-23T21:25:16.689719+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-22T21:25:22.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.938+0000 7fb6cd200640 1 -- 172.21.15.161:0/445974765 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb6a4066690 msgr2=0x7fb6a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.938+0000 7fb6cd200640 1 --2- 172.21.15.161:0/445974765 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb6a4066690 0x7fb6a4068b50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80af600 tx=0x7fb6b8009190 comp rx=0 tx=0).stop 2024-05-22T21:25:22.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.938+0000 7fb6cd200640 1 -- 172.21.15.161:0/445974765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c815e130 msgr2=0x7fb6c8158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:22.937 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.938+0000 7fb6cd200640 1 --2- 172.21.15.161:0/445974765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c815e130 0x7fb6c8158860 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fb6c001a5f0 tx=0x7fb6c0019760 comp rx=0 tx=0).stop 2024-05-22T21:25:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.938+0000 7fb6cd200640 1 -- 172.21.15.161:0/445974765 shutdown_connections 2024-05-22T21:25:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.938+0000 7fb6cd200640 1 --2- 172.21.15.161:0/445974765 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fb6a4066690 0x7fb6a4068b50 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.938+0000 7fb6cd200640 1 --2- 172.21.15.161:0/445974765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c815e130 0x7fb6c8158860 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.938+0000 7fb6cd200640 1 --2- 172.21.15.161:0/445974765 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c815dd50 0x7fb6c8156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.938+0000 7fb6cd200640 1 --2- 172.21.15.161:0/445974765 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6c815d5e0 0x7fb6c80b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.938+0000 7fb6cd200640 1 -- 172.21.15.161:0/445974765 >> 172.21.15.161:0/445974765 conn(0x7fb6c80b0bc0 msgr2=0x7fb6c8151860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.938+0000 7fb6cd200640 1 -- 172.21.15.161:0/445974765 shutdown_connections 2024-05-22T21:25:22.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:22.938+0000 7fb6cd200640 1 -- 172.21.15.161:0/445974765 wait complete. 2024-05-22T21:25:23.958 INFO:tasks.ceph:Scrubbing osd.0 2024-05-22T21:25:23.958 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-22T21:25:24.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.106+0000 7fd73ea00640 1 Processor -- start 2024-05-22T21:25:24.107 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.106+0000 7fd73ea00640 1 -- start start 2024-05-22T21:25:24.109 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.106+0000 7fd73ea00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd73813cfe0 0x7fd73815dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.106+0000 7fd73ea00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd7380ce210 0x7fd73815e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.106+0000 7fd73ea00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7380ce830 con 0x7fd73813d560 2024-05-22T21:25:24.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.106+0000 7fd73ea00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd738140560 con 0x7fd73813cfe0 2024-05-22T21:25:24.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.106+0000 7fd73ea00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7380ceb10 con 0x7fd7380ce210 2024-05-22T21:25:24.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.106+0000 7fd737e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd73813cfe0 0x7fd73815dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.106+0000 7fd737e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd73813cfe0 0x7fd73815dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59314/0 (socket says 172.21.15.161:59314) 2024-05-22T21:25:24.110 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.106+0000 7fd737e00640 1 -- 172.21.15.161:0/2794164031 learned_addr learned my addr 172.21.15.161:0/2794164031 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:24.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.106+0000 7fd73ca00640 1 --2- 172.21.15.161:0/2794164031 >> v2:172.21.15.179:3300/0 conn(0x7fd7380ce210 0x7fd73815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.106+0000 7fd737e00640 1 -- 172.21.15.161:0/2794164031 >> v2:172.21.15.179:3300/0 conn(0x7fd7380ce210 msgr2=0x7fd73815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:24.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.106+0000 7fd737e00640 1 --2- 172.21.15.161:0/2794164031 >> v2:172.21.15.179:3300/0 conn(0x7fd7380ce210 0x7fd73815e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.110+0000 7fd737e00640 1 -- 172.21.15.161:0/2794164031 >> v1:172.21.15.179:6789/0 conn(0x7fd73813d560 legacy=0x7fd7380cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.110+0000 7fd737e00640 1 -- 172.21.15.161:0/2794164031 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73806fd80 con 0x7fd73813cfe0 2024-05-22T21:25:24.111 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.110+0000 7fd737e00640 1 --2- 172.21.15.161:0/2794164031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd73813cfe0 0x7fd73815dac0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fd73806c9c0 tx=0x7fd72801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e97fd8bfee1473 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:24.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.110+0000 7fd736a00640 1 -- 172.21.15.161:0/2794164031 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7280aa5f0 con 0x7fd73813cfe0 2024-05-22T21:25:24.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.110+0000 7fd736a00640 1 -- 172.21.15.161:0/2794164031 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7280aa750 con 0x7fd73813cfe0 2024-05-22T21:25:24.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.110+0000 7fd736a00640 1 -- 172.21.15.161:0/2794164031 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7280aaae0 con 0x7fd73813cfe0 2024-05-22T21:25:24.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.110+0000 7fd73ea00640 1 -- 172.21.15.161:0/2794164031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd73813cfe0 msgr2=0x7fd73815dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.110+0000 7fd73ea00640 1 --2- 172.21.15.161:0/2794164031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd73813cfe0 0x7fd73815dac0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fd73806c9c0 tx=0x7fd72801c490 comp rx=0 tx=0).stop 2024-05-22T21:25:24.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.110+0000 7fd73ea00640 1 -- 172.21.15.161:0/2794164031 shutdown_connections 2024-05-22T21:25:24.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.110+0000 7fd73ea00640 1 --2- 172.21.15.161:0/2794164031 >> v2:172.21.15.179:3300/0 conn(0x7fd7380ce210 0x7fd73815e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.110+0000 7fd73ea00640 1 --2- 172.21.15.161:0/2794164031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd73813cfe0 0x7fd73815dac0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.110+0000 7fd73ea00640 1 -- 172.21.15.161:0/2794164031 >> 172.21.15.161:0/2794164031 conn(0x7fd73806a0a0 msgr2=0x7fd73806a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:24.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.110+0000 7fd73ea00640 1 -- 172.21.15.161:0/2794164031 shutdown_connections 2024-05-22T21:25:24.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.110+0000 7fd73ea00640 1 -- 172.21.15.161:0/2794164031 wait complete. 2024-05-22T21:25:24.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd73ea00640 1 Processor -- start 2024-05-22T21:25:24.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd73ea00640 1 -- start start 2024-05-22T21:25:24.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd73ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7380ce210 0x7fd7381410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd73ea00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd73813cfe0 0x7fd73813b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd73ea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd73813d560 0x7fd738137200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd73ea00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7381415f0 con 0x7fd7380ce210 2024-05-22T21:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd73ea00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd738138fa0 con 0x7fd73813d560 2024-05-22T21:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd73ea00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7381391f0 con 0x7fd73813cfe0 2024-05-22T21:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd737e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7380ce210 0x7fd7381410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd737e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7380ce210 0x7fd7381410b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59326/0 (socket says 172.21.15.161:59326) 2024-05-22T21:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd737e00640 1 -- 172.21.15.161:0/2606141996 learned_addr learned my addr 172.21.15.161:0/2606141996 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd737400640 1 --2- 172.21.15.161:0/2606141996 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd73813cfe0 0x7fd73813b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd73ca00640 1 --2- 172.21.15.161:0/2606141996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd73813d560 0x7fd738137200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd737e00640 1 -- 172.21.15.161:0/2606141996 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd73813cfe0 msgr2=0x7fd73813b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd737e00640 1 --2- 172.21.15.161:0/2606141996 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd73813cfe0 0x7fd73813b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd737e00640 1 -- 172.21.15.161:0/2606141996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd73813d560 msgr2=0x7fd738137200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd737e00640 1 --2- 172.21.15.161:0/2606141996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd73813d560 0x7fd738137200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd737e00640 1 -- 172.21.15.161:0/2606141996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7280094d0 con 0x7fd7380ce210 2024-05-22T21:25:24.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd737e00640 1 --2- 172.21.15.161:0/2606141996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7380ce210 0x7fd7381410b0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fd7280ac040 tx=0x7fd72801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:24.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd72fe00640 1 -- 172.21.15.161:0/2606141996 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7280b3800 con 0x7fd7380ce210 2024-05-22T21:25:24.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd72fe00640 1 -- 172.21.15.161:0/2606141996 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7280b3980 con 0x7fd7380ce210 2024-05-22T21:25:24.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd72fe00640 1 -- 172.21.15.161:0/2606141996 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7280b3d30 con 0x7fd7380ce210 2024-05-22T21:25:24.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd73ea00640 1 -- 172.21.15.161:0/2606141996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7381394d0 con 0x7fd7380ce210 2024-05-22T21:25:24.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.114+0000 7fd73ea00640 1 -- 172.21.15.161:0/2606141996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd738139a90 con 0x7fd7380ce210 2024-05-22T21:25:24.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.118+0000 7fd73ea00640 1 -- 172.21.15.161:0/2606141996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd73813fe00 con 0x7fd7380ce210 2024-05-22T21:25:24.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.118+0000 7fd72fe00640 1 -- 172.21.15.161:0/2606141996 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd7280b2380 con 0x7fd7380ce210 2024-05-22T21:25:24.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.118+0000 7fd72fe00640 1 --2- 172.21.15.161:0/2606141996 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd718066690 0x7fd718068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.118+0000 7fd72fe00640 1 -- 172.21.15.161:0/2606141996 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd7280a7070 con 0x7fd7380ce210 2024-05-22T21:25:24.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.118+0000 7fd72fe00640 1 --2- 172.21.15.161:0/2606141996 >> [v2:172.21.15.161:6808/23413,v1:172.21.15.161:6809/23413] conn(0x7fd71806c4f0 0x7fd71806e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.118+0000 7fd72fe00640 1 -- 172.21.15.161:0/2606141996 --> [v2:172.21.15.161:6808/23413,v1:172.21.15.161:6809/23413] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd71806efc0 con 0x7fd71806c4f0 2024-05-22T21:25:24.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.118+0000 7fd72fe00640 1 -- 172.21.15.161:0/2606141996 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=197) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd728120510 con 0x7fd7380ce210 2024-05-22T21:25:24.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.118+0000 7fd737400640 1 --2- 172.21.15.161:0/2606141996 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd718066690 0x7fd718068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.118+0000 7fd73ca00640 1 --2- 172.21.15.161:0/2606141996 >> [v2:172.21.15.161:6808/23413,v1:172.21.15.161:6809/23413] conn(0x7fd71806c4f0 0x7fd71806e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.118+0000 7fd737400640 1 --2- 172.21.15.161:0/2606141996 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd718066690 0x7fd718068b50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd7381f7600 tx=0x7fd724009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:24.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.118+0000 7fd73ca00640 1 --2- 172.21.15.161:0/2606141996 >> [v2:172.21.15.161:6808/23413,v1:172.21.15.161:6809/23413] conn(0x7fd71806c4f0 0x7fd71806e910 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:24.121 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.122+0000 7fd72fe00640 1 -- 172.21.15.161:0/2606141996 <== osd.0 v2:172.21.15.161:6808/23413 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24459 (crc 0 0 0) 0x7fd71806efc0 con 0x7fd71806c4f0 2024-05-22T21:25:24.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.130+0000 7fd73ea00640 1 -- 172.21.15.161:0/2606141996 --> [v2:172.21.15.161:6808/23413,v1:172.21.15.161:6809/23413] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd738067220 con 0x7fd71806c4f0 2024-05-22T21:25:24.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.130+0000 7fd72fe00640 1 -- 172.21.15.161:0/2606141996 <== osd.0 v2:172.21.15.161:6808/23413 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7fd738067220 con 0x7fd71806c4f0 2024-05-22T21:25:24.132 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:25:24.132 INFO:teuthology.orchestra.run.smithi161.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-05-22T21:25:24.132 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:25:24.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.130+0000 7fd73ea00640 1 -- 172.21.15.161:0/2606141996 >> [v2:172.21.15.161:6808/23413,v1:172.21.15.161:6809/23413] conn(0x7fd71806c4f0 msgr2=0x7fd71806e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.130+0000 7fd73ea00640 1 --2- 172.21.15.161:0/2606141996 >> [v2:172.21.15.161:6808/23413,v1:172.21.15.161:6809/23413] conn(0x7fd71806c4f0 0x7fd71806e910 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.130+0000 7fd73ea00640 1 -- 172.21.15.161:0/2606141996 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd718066690 msgr2=0x7fd718068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.130+0000 7fd73ea00640 1 --2- 172.21.15.161:0/2606141996 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd718066690 0x7fd718068b50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd7381f7600 tx=0x7fd724009040 comp rx=0 tx=0).stop 2024-05-22T21:25:24.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.130+0000 7fd73ea00640 1 -- 172.21.15.161:0/2606141996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7380ce210 msgr2=0x7fd7381410b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.130+0000 7fd73ea00640 1 --2- 172.21.15.161:0/2606141996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7380ce210 0x7fd7381410b0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fd7280ac040 tx=0x7fd72801acd0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.130+0000 7fd73ca00640 1 -- 172.21.15.161:0/2606141996 reap_dead start 2024-05-22T21:25:24.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.130+0000 7fd73ea00640 1 -- 172.21.15.161:0/2606141996 shutdown_connections 2024-05-22T21:25:24.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.130+0000 7fd73ea00640 1 -- 172.21.15.161:0/2606141996 >> 172.21.15.161:0/2606141996 conn(0x7fd73806a0a0 msgr2=0x7fd73806a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:24.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.134+0000 7fd73ea00640 1 -- 172.21.15.161:0/2606141996 shutdown_connections 2024-05-22T21:25:24.134 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.134+0000 7fd73ea00640 1 -- 172.21.15.161:0/2606141996 wait complete. 2024-05-22T21:25:24.150 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-22T21:25:24.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.286+0000 7f3ad7400640 1 Processor -- start 2024-05-22T21:25:24.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.286+0000 7f3ad7400640 1 -- start start 2024-05-22T21:25:24.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.290+0000 7f3ad7400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3ad01402e0 0x7f3ad013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.290+0000 7f3ad7400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ad013fc80 0x7f3ad006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.290+0000 7f3ad7400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad01400b0 con 0x7f3ad013cfc0 2024-05-22T21:25:24.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.290+0000 7f3ad7400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad013d3a0 con 0x7f3ad01402e0 2024-05-22T21:25:24.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.290+0000 7f3ad7400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad00cd920 con 0x7f3ad013fc80 2024-05-22T21:25:24.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.290+0000 7f3acfe00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3ad01402e0 0x7f3ad013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.290+0000 7f3ad5400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ad013fc80 0x7f3ad006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.290+0000 7f3ad4a00640 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f3ad013cfc0 0x7f3ad015af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.161:33366/0 (socket says 172.21.15.161:33366) 2024-05-22T21:25:24.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.290+0000 7f3ad4a00640 1 -- 172.21.15.161:0/1232924840 learned_addr learned my addr 172.21.15.161:0/1232924840 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:24.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.290+0000 7f3acfe00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3ad01402e0 0x7f3ad013f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:49820/0 (socket says 172.21.15.161:49820) 2024-05-22T21:25:24.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.290+0000 7f3acf400640 1 -- 172.21.15.161:0/1232924840 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273062868 0 0) 0x7f3ad01400b0 con 0x7f3ad013cfc0 2024-05-22T21:25:24.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.290+0000 7f3acfe00640 1 -- 172.21.15.161:0/1232924840 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ad013fc80 msgr2=0x7f3ad006fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:24.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.290+0000 7f3acfe00640 1 --2- 172.21.15.161:0/1232924840 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ad013fc80 0x7f3ad006fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.290+0000 7f3acfe00640 1 -- 172.21.15.161:0/1232924840 >> v1:172.21.15.179:6789/0 conn(0x7f3ad013cfc0 legacy=0x7f3ad015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.290+0000 7f3acfe00640 1 -- 172.21.15.161:0/1232924840 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad00cdc00 con 0x7f3ad01402e0 2024-05-22T21:25:24.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.294+0000 7f3acfe00640 1 --2- 172.21.15.161:0/1232924840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad01402e0 0x7f3ad013f740 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00ba60 tx=0x7f3abc01eab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb2e2689b91395ad server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:24.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.294+0000 7f3acf400640 1 -- 172.21.15.161:0/1232924840 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0a77b0 con 0x7f3ad01402e0 2024-05-22T21:25:24.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.294+0000 7f3acf400640 1 -- 172.21.15.161:0/1232924840 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc0a7910 con 0x7f3ad01402e0 2024-05-22T21:25:24.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.294+0000 7f3acf400640 1 -- 172.21.15.161:0/1232924840 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0a7ca0 con 0x7f3ad01402e0 2024-05-22T21:25:24.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.294+0000 7f3ad7400640 1 -- 172.21.15.161:0/1232924840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad01402e0 msgr2=0x7f3ad013f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.294+0000 7f3ad7400640 1 --2- 172.21.15.161:0/1232924840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad01402e0 0x7f3ad013f740 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00ba60 tx=0x7f3abc01eab0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.294+0000 7f3ad7400640 1 -- 172.21.15.161:0/1232924840 shutdown_connections 2024-05-22T21:25:24.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.294+0000 7f3ad7400640 1 --2- 172.21.15.161:0/1232924840 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ad013fc80 0x7f3ad006fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.294+0000 7f3ad7400640 1 --2- 172.21.15.161:0/1232924840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad01402e0 0x7f3ad013f740 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.294+0000 7f3ad7400640 1 -- 172.21.15.161:0/1232924840 >> 172.21.15.161:0/1232924840 conn(0x7f3ad006a2d0 msgr2=0x7f3ad006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:24.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.294+0000 7f3ad7400640 1 -- 172.21.15.161:0/1232924840 shutdown_connections 2024-05-22T21:25:24.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.294+0000 7f3ad7400640 1 -- 172.21.15.161:0/1232924840 wait complete. 2024-05-22T21:25:24.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.294+0000 7f3ad7400640 1 Processor -- start 2024-05-22T21:25:24.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.294+0000 7f3ad7400640 1 -- start start 2024-05-22T21:25:24.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad7400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ad013cfc0 0x7f3ad01576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad7400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad013fc80 0x7f3ad013d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ad01402e0 0x7f3ad013df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad7400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad0157be0 con 0x7f3ad01402e0 2024-05-22T21:25:24.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad7400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad015c4b0 con 0x7f3ad013fc80 2024-05-22T21:25:24.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad7400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad015b8a0 con 0x7f3ad013cfc0 2024-05-22T21:25:24.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad5400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ad01402e0 0x7f3ad013df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad5400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ad01402e0 0x7f3ad013df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59328/0 (socket says 172.21.15.161:59328) 2024-05-22T21:25:24.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad5400640 1 -- 172.21.15.161:0/3188797248 learned_addr learned my addr 172.21.15.161:0/3188797248 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:24.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3acfe00640 1 --2- 172.21.15.161:0/3188797248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad013fc80 0x7f3ad013d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad4a00640 1 --2- 172.21.15.161:0/3188797248 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ad013cfc0 0x7f3ad01576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad5400640 1 -- 172.21.15.161:0/3188797248 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ad013cfc0 msgr2=0x7f3ad01576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad5400640 1 --2- 172.21.15.161:0/3188797248 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ad013cfc0 0x7f3ad01576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad5400640 1 -- 172.21.15.161:0/3188797248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad013fc80 msgr2=0x7f3ad013d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad5400640 1 --2- 172.21.15.161:0/3188797248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad013fc80 0x7f3ad013d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad5400640 1 -- 172.21.15.161:0/3188797248 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3abc00b6c0 con 0x7f3ad01402e0 2024-05-22T21:25:24.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3acfe00640 1 --2- 172.21.15.161:0/3188797248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad013fc80 0x7f3ad013d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:24.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad5400640 1 --2- 172.21.15.161:0/3188797248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ad01402e0 0x7f3ad013df10 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800e930 tx=0x7f3ac800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:24.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3accc00640 1 -- 172.21.15.161:0/3188797248 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac80096b0 con 0x7f3ad01402e0 2024-05-22T21:25:24.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3accc00640 1 -- 172.21.15.161:0/3188797248 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac800c9a0 con 0x7f3ad01402e0 2024-05-22T21:25:24.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad7400640 1 -- 172.21.15.161:0/3188797248 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad015bb80 con 0x7f3ad01402e0 2024-05-22T21:25:24.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3accc00640 1 -- 172.21.15.161:0/3188797248 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac800cd30 con 0x7f3ad01402e0 2024-05-22T21:25:24.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad7400640 1 -- 172.21.15.161:0/3188797248 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad015c140 con 0x7f3ad01402e0 2024-05-22T21:25:24.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3ad7400640 1 -- 172.21.15.161:0/3188797248 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad0067220 con 0x7f3ad01402e0 2024-05-22T21:25:24.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.298+0000 7f3accc00640 1 -- 172.21.15.161:0/3188797248 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3ac8013470 con 0x7f3ad01402e0 2024-05-22T21:25:24.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.302+0000 7f3accc00640 1 --2- 172.21.15.161:0/3188797248 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ab0066690 0x7f3ab0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.302+0000 7f3accc00640 1 -- 172.21.15.161:0/3188797248 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3ac80845a0 con 0x7f3ad01402e0 2024-05-22T21:25:24.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.302+0000 7f3ad4a00640 1 --2- 172.21.15.161:0/3188797248 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ab0066690 0x7f3ab0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.302+0000 7f3ad4a00640 1 --2- 172.21.15.161:0/3188797248 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ab0066690 0x7f3ab0068b50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0007fd0 tx=0x7f3ac0007ee0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:24.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.306+0000 7f3accc00640 1 -- 172.21.15.161:0/3188797248 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f3ac8051ba0 con 0x7f3ad01402e0 2024-05-22T21:25:24.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.458+0000 7f3ad7400640 1 -- 172.21.15.161:0/3188797248 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ad006cae0 con 0x7f3ab0066690 2024-05-22T21:25:24.457 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.458+0000 7f3accc00640 1 -- 172.21.15.161:0/3188797248 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3ac0001c90 con 0x7f3ab0066690 2024-05-22T21:25:24.458 INFO:teuthology.orchestra.run.smithi161.stderr:instructed osd(s) 0 to deep-scrub 2024-05-22T21:25:24.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.458+0000 7f3ad7400640 1 -- 172.21.15.161:0/3188797248 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ab0066690 msgr2=0x7f3ab0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.458+0000 7f3ad7400640 1 --2- 172.21.15.161:0/3188797248 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ab0066690 0x7f3ab0068b50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0007fd0 tx=0x7f3ac0007ee0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.458+0000 7f3ad7400640 1 -- 172.21.15.161:0/3188797248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ad01402e0 msgr2=0x7f3ad013df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.458+0000 7f3ad7400640 1 --2- 172.21.15.161:0/3188797248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ad01402e0 0x7f3ad013df10 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800e930 tx=0x7f3ac800edf0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.458+0000 7f3ad7400640 1 -- 172.21.15.161:0/3188797248 shutdown_connections 2024-05-22T21:25:24.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.458+0000 7f3ad7400640 1 --2- 172.21.15.161:0/3188797248 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f3ab0066690 0x7f3ab0068b50 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.458+0000 7f3ad7400640 1 --2- 172.21.15.161:0/3188797248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ad01402e0 0x7f3ad013df10 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.458+0000 7f3ad7400640 1 --2- 172.21.15.161:0/3188797248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad013fc80 0x7f3ad013d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.458+0000 7f3ad7400640 1 --2- 172.21.15.161:0/3188797248 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3ad013cfc0 0x7f3ad01576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.458+0000 7f3ad7400640 1 -- 172.21.15.161:0/3188797248 >> 172.21.15.161:0/3188797248 conn(0x7f3ad006a2d0 msgr2=0x7f3ad0158200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:24.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.458+0000 7f3ad7400640 1 -- 172.21.15.161:0/3188797248 shutdown_connections 2024-05-22T21:25:24.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.458+0000 7f3ad7400640 1 -- 172.21.15.161:0/3188797248 wait complete. 2024-05-22T21:25:24.479 INFO:tasks.ceph:Scrubbing osd.1 2024-05-22T21:25:24.479 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-22T21:25:24.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f7846200640 1 Processor -- start 2024-05-22T21:25:24.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f7846200640 1 -- start start 2024-05-22T21:25:24.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f7846200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f784006d330 0x7f78400b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f7846200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784006d8b0 con 0x7f7840159630 2024-05-22T21:25:24.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f7846200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f784006db90 con 0x7f78400707c0 2024-05-22T21:25:24.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f7846200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78400b21b0 con 0x7f784006d330 2024-05-22T21:25:24.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f7844e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f7840159630 0x7f78400700b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59510/0 (socket says 172.21.15.161:59510) 2024-05-22T21:25:24.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f7844e00640 1 -- 172.21.15.161:0/3661432348 learned_addr learned my addr 172.21.15.161:0/3661432348 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:24.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f7845800640 1 --2- 172.21.15.161:0/3661432348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f784006d330 0x7f78400b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783f400640 1 -- 172.21.15.161:0/3661432348 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357372985 0 0) 0x7f784006d8b0 con 0x7f7840159630 2024-05-22T21:25:24.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783f400640 1 -- 172.21.15.161:0/3661432348 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7828003530 con 0x7f7840159630 2024-05-22T21:25:24.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783f400640 1 -- 172.21.15.161:0/3661432348 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3995386442 0 0) 0x7f7828003530 con 0x7f7840159630 2024-05-22T21:25:24.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783f400640 1 -- 172.21.15.161:0/3661432348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f784006d330 msgr2=0x7f78400b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:24.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783f400640 1 --2- 172.21.15.161:0/3661432348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f784006d330 0x7f78400b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783f400640 1 -- 172.21.15.161:0/3661432348 >> v1:172.21.15.179:6789/0 conn(0x7f78400707c0 legacy=0x7f78400b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783f400640 1 -- 172.21.15.161:0/3661432348 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78400b2490 con 0x7f7840159630 2024-05-22T21:25:24.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783f400640 1 -- 172.21.15.161:0/3661432348 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f7834001680 con 0x7f7840159630 2024-05-22T21:25:24.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783f400640 1 -- 172.21.15.161:0/3661432348 >> v1:172.21.15.161:6789/0 conn(0x7f7840159630 legacy=0x7f78400700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783f400640 1 --2- 172.21.15.161:0/3661432348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78280039b0 0x7f7828003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783f400640 1 --2- 172.21.15.161:0/3661432348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78280047d0 0x7f7828004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783f400640 1 --2- 172.21.15.161:0/3661432348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f784006d330 0x7f78280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783f400640 1 -- 172.21.15.161:0/3661432348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78400b21b0 con 0x7f78280039b0 2024-05-22T21:25:24.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783f400640 1 -- 172.21.15.161:0/3661432348 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782800dcd0 con 0x7f78280047d0 2024-05-22T21:25:24.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783f400640 1 -- 172.21.15.161:0/3661432348 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782800de90 con 0x7f784006d330 2024-05-22T21:25:24.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f7844e00640 1 --2- 172.21.15.161:0/3661432348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78280039b0 0x7f7828003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783fe00640 1 --2- 172.21.15.161:0/3661432348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78280047d0 0x7f7828004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f7845800640 1 --2- 172.21.15.161:0/3661432348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f784006d330 0x7f78280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f7844e00640 1 -- 172.21.15.161:0/3661432348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f784006d330 msgr2=0x7f78280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:24.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f7844e00640 1 --2- 172.21.15.161:0/3661432348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f784006d330 0x7f78280051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f7844e00640 1 -- 172.21.15.161:0/3661432348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78280047d0 msgr2=0x7f7828004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:24.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f7844e00640 1 --2- 172.21.15.161:0/3661432348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78280047d0 0x7f7828004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f7844e00640 1 -- 172.21.15.161:0/3661432348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7828003530 con 0x7f78280039b0 2024-05-22T21:25:24.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.618+0000 7f783fe00640 1 --2- 172.21.15.161:0/3661432348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78280047d0 0x7f7828004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:24.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.622+0000 7f7844e00640 1 --2- 172.21.15.161:0/3661432348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78280039b0 0x7f7828003d90 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f783400c210 tx=0x7f783401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3677dcf514e51b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:24.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.622+0000 7f783f400640 1 -- 172.21.15.161:0/3661432348 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7834005ae0 con 0x7f78280039b0 2024-05-22T21:25:24.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.622+0000 7f783f400640 1 -- 172.21.15.161:0/3661432348 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7834005c60 con 0x7f78280039b0 2024-05-22T21:25:24.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.622+0000 7f783f400640 1 -- 172.21.15.161:0/3661432348 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78340a5870 con 0x7f78280039b0 2024-05-22T21:25:24.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.622+0000 7f7846200640 1 -- 172.21.15.161:0/3661432348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78280039b0 msgr2=0x7f7828003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.622+0000 7f7846200640 1 --2- 172.21.15.161:0/3661432348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78280039b0 0x7f7828003d90 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f783400c210 tx=0x7f783401cc80 comp rx=0 tx=0).stop 2024-05-22T21:25:24.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.622+0000 7f7845800640 1 -- 172.21.15.161:0/3661432348 reap_dead start 2024-05-22T21:25:24.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.622+0000 7f7846200640 1 -- 172.21.15.161:0/3661432348 shutdown_connections 2024-05-22T21:25:24.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.622+0000 7f7846200640 1 -- 172.21.15.161:0/3661432348 >> 172.21.15.161:0/3661432348 conn(0x7f78400cebc0 msgr2=0x7f78400cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:24.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.622+0000 7f7846200640 1 -- 172.21.15.161:0/3661432348 shutdown_connections 2024-05-22T21:25:24.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.622+0000 7f7846200640 1 -- 172.21.15.161:0/3661432348 wait complete. 2024-05-22T21:25:24.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f7846200640 1 Processor -- start 2024-05-22T21:25:24.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f7846200640 1 -- start start 2024-05-22T21:25:24.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f7846200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f78280039b0 0x7f784013ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f7846200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7840158180 0x7f78401569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f7846200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78400ab760 0x7f784013b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f7846200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784013f180 con 0x7f7840158180 2024-05-22T21:25:24.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f7846200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7840156ee0 con 0x7f78400ab760 2024-05-22T21:25:24.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f7846200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7840157cf0 con 0x7f78280039b0 2024-05-22T21:25:24.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f783fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7840158180 0x7f78401569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f783fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7840158180 0x7f78401569a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59348/0 (socket says 172.21.15.161:59348) 2024-05-22T21:25:24.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f783fe00640 1 -- 172.21.15.161:0/1538929247 learned_addr learned my addr 172.21.15.161:0/1538929247 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:24.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f783fe00640 1 -- 172.21.15.161:0/1538929247 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f78280039b0 msgr2=0x7f784013ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f783fe00640 1 --2- 172.21.15.161:0/1538929247 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f78280039b0 0x7f784013ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f783fe00640 1 -- 172.21.15.161:0/1538929247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78400ab760 msgr2=0x7f784013b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f783fe00640 1 --2- 172.21.15.161:0/1538929247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78400ab760 0x7f784013b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f783fe00640 1 -- 172.21.15.161:0/1538929247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78401570e0 con 0x7f7840158180 2024-05-22T21:25:24.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f783fe00640 1 --2- 172.21.15.161:0/1538929247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7840158180 0x7f78401569a0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f782c00e560 tx=0x7f782c00ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:24.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f783cc00640 1 -- 172.21.15.161:0/1538929247 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c0052e0 con 0x7f7840158180 2024-05-22T21:25:24.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f783cc00640 1 -- 172.21.15.161:0/1538929247 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c005440 con 0x7f7840158180 2024-05-22T21:25:24.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f7846200640 1 -- 172.21.15.161:0/1538929247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78401573c0 con 0x7f7840158180 2024-05-22T21:25:24.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f783cc00640 1 -- 172.21.15.161:0/1538929247 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c003840 con 0x7f7840158180 2024-05-22T21:25:24.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f7846200640 1 -- 172.21.15.161:0/1538929247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7840157980 con 0x7f7840158180 2024-05-22T21:25:24.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.626+0000 7f7846200640 1 -- 172.21.15.161:0/1538929247 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f78400b30c0 con 0x7f7840158180 2024-05-22T21:25:24.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.630+0000 7f783cc00640 1 -- 172.21.15.161:0/1538929247 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f782c0039a0 con 0x7f7840158180 2024-05-22T21:25:24.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.630+0000 7f783cc00640 1 --2- 172.21.15.161:0/1538929247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f78200666c0 0x7f7820068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.630+0000 7f783cc00640 1 -- 172.21.15.161:0/1538929247 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f782c083d10 con 0x7f7840158180 2024-05-22T21:25:24.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.630+0000 7f7844e00640 1 --2- 172.21.15.161:0/1538929247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f78200666c0 0x7f7820068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.630+0000 7f783cc00640 1 --2- 172.21.15.161:0/1538929247 >> [v2:172.21.15.161:6800/23412,v1:172.21.15.161:6801/23412] conn(0x7f782006c520 0x7f782006e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.630+0000 7f783cc00640 1 -- 172.21.15.161:0/1538929247 --> [v2:172.21.15.161:6800/23412,v1:172.21.15.161:6801/23412] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f782006eff0 con 0x7f782006c520 2024-05-22T21:25:24.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.630+0000 7f783cc00640 1 -- 172.21.15.161:0/1538929247 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=197) v2 ==== 24+0+0 (secure 0 0 0) 0x7f782c0862f0 con 0x7f7840158180 2024-05-22T21:25:24.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.630+0000 7f7845800640 1 --2- 172.21.15.161:0/1538929247 >> [v2:172.21.15.161:6800/23412,v1:172.21.15.161:6801/23412] conn(0x7f782006c520 0x7f782006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.630+0000 7f7844e00640 1 --2- 172.21.15.161:0/1538929247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f78200666c0 0x7f7820068b80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f78340ab000 tx=0x7f7834001a10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:24.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.630+0000 7f7845800640 1 --2- 172.21.15.161:0/1538929247 >> [v2:172.21.15.161:6800/23412,v1:172.21.15.161:6801/23412] conn(0x7f782006c520 0x7f782006e940 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:24.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.630+0000 7f783cc00640 1 -- 172.21.15.161:0/1538929247 <== osd.1 v2:172.21.15.161:6800/23412 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24459 (crc 0 0 0) 0x7f782006eff0 con 0x7f782006c520 2024-05-22T21:25:24.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.642+0000 7f7846200640 1 -- 172.21.15.161:0/1538929247 --> [v2:172.21.15.161:6800/23412,v1:172.21.15.161:6801/23412] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7840157e60 con 0x7f782006c520 2024-05-22T21:25:24.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.642+0000 7f783cc00640 1 -- 172.21.15.161:0/1538929247 <== osd.1 v2:172.21.15.161:6800/23412 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f7840157e60 con 0x7f782006c520 2024-05-22T21:25:24.644 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:25:24.644 INFO:teuthology.orchestra.run.smithi161.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-05-22T21:25:24.644 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:25:24.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.642+0000 7f7846200640 1 -- 172.21.15.161:0/1538929247 >> [v2:172.21.15.161:6800/23412,v1:172.21.15.161:6801/23412] conn(0x7f782006c520 msgr2=0x7f782006e940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.642+0000 7f7846200640 1 --2- 172.21.15.161:0/1538929247 >> [v2:172.21.15.161:6800/23412,v1:172.21.15.161:6801/23412] conn(0x7f782006c520 0x7f782006e940 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.642+0000 7f7846200640 1 -- 172.21.15.161:0/1538929247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f78200666c0 msgr2=0x7f7820068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.642+0000 7f7846200640 1 --2- 172.21.15.161:0/1538929247 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f78200666c0 0x7f7820068b80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f78340ab000 tx=0x7f7834001a10 comp rx=0 tx=0).stop 2024-05-22T21:25:24.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.642+0000 7f7846200640 1 -- 172.21.15.161:0/1538929247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7840158180 msgr2=0x7f78401569a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.642+0000 7f7846200640 1 --2- 172.21.15.161:0/1538929247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7840158180 0x7f78401569a0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f782c00e560 tx=0x7f782c00ea20 comp rx=0 tx=0).stop 2024-05-22T21:25:24.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.642+0000 7f7845800640 1 -- 172.21.15.161:0/1538929247 reap_dead start 2024-05-22T21:25:24.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.642+0000 7f7846200640 1 -- 172.21.15.161:0/1538929247 shutdown_connections 2024-05-22T21:25:24.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.642+0000 7f7846200640 1 -- 172.21.15.161:0/1538929247 >> 172.21.15.161:0/1538929247 conn(0x7f78400cebc0 msgr2=0x7f7840162cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:24.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.642+0000 7f7846200640 1 -- 172.21.15.161:0/1538929247 shutdown_connections 2024-05-22T21:25:24.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.642+0000 7f7846200640 1 -- 172.21.15.161:0/1538929247 wait complete. 2024-05-22T21:25:24.661 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-22T21:25:24.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.798+0000 7fe959c00640 1 Processor -- start 2024-05-22T21:25:24.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.798+0000 7fe959c00640 1 -- start start 2024-05-22T21:25:24.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.798+0000 7fe959c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe954143c40 0x7fe95415b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.798+0000 7fe959c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe9541444a0 0x7fe95406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.798+0000 7fe959c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe954144900 0x7fe95406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.798+0000 7fe959c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95415efb0 con 0x7fe9541444a0 2024-05-22T21:25:24.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.798+0000 7fe959c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95415f290 con 0x7fe954143c40 2024-05-22T21:25:24.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.798+0000 7fe959c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95415f570 con 0x7fe954144900 2024-05-22T21:25:24.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.802+0000 7fe94a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe9541444a0 0x7fe95406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.802+0000 7fe94a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe9541444a0 0x7fe95406c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59350/0 (socket says 172.21.15.161:59350) 2024-05-22T21:25:24.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.802+0000 7fe94a000640 1 -- 172.21.15.161:0/345457339 learned_addr learned my addr 172.21.15.161:0/345457339 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:24.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.802+0000 7fe952a00640 1 --2- 172.21.15.161:0/345457339 >> v2:172.21.15.179:3300/0 conn(0x7fe954143c40 0x7fe95415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.802+0000 7fe953400640 1 --2- 172.21.15.161:0/345457339 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe954144900 0x7fe95406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.802+0000 7fe94a000640 1 -- 172.21.15.161:0/345457339 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe954144900 msgr2=0x7fe95406c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:24.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.802+0000 7fe94a000640 1 --2- 172.21.15.161:0/345457339 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe954144900 0x7fe95406c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.802+0000 7fe94a000640 1 -- 172.21.15.161:0/345457339 >> v2:172.21.15.179:3300/0 conn(0x7fe954143c40 msgr2=0x7fe95415b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:24.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.802+0000 7fe94a000640 1 --2- 172.21.15.161:0/345457339 >> v2:172.21.15.179:3300/0 conn(0x7fe954143c40 0x7fe95415b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.802+0000 7fe94a000640 1 -- 172.21.15.161:0/345457339 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95415f850 con 0x7fe9541444a0 2024-05-22T21:25:24.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.802+0000 7fe952a00640 1 --2- 172.21.15.161:0/345457339 >> v2:172.21.15.179:3300/0 conn(0x7fe954143c40 0x7fe95415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:24.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.802+0000 7fe94a000640 1 --2- 172.21.15.161:0/345457339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9541444a0 0x7fe95406c2a0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fe9540724b0 tx=0x7fe93c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d449dbb128df0f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:24.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.802+0000 7fe952000640 1 -- 172.21.15.161:0/345457339 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c0ab640 con 0x7fe9541444a0 2024-05-22T21:25:24.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.802+0000 7fe952000640 1 -- 172.21.15.161:0/345457339 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93c0a7070 con 0x7fe9541444a0 2024-05-22T21:25:24.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.802+0000 7fe952000640 1 -- 172.21.15.161:0/345457339 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c0ab9c0 con 0x7fe9541444a0 2024-05-22T21:25:24.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 -- 172.21.15.161:0/345457339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9541444a0 msgr2=0x7fe95406c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 --2- 172.21.15.161:0/345457339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9541444a0 0x7fe95406c2a0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fe9540724b0 tx=0x7fe93c01c520 comp rx=0 tx=0).stop 2024-05-22T21:25:24.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 -- 172.21.15.161:0/345457339 shutdown_connections 2024-05-22T21:25:24.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 --2- 172.21.15.161:0/345457339 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe954144900 0x7fe95406c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 --2- 172.21.15.161:0/345457339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9541444a0 0x7fe95406c2a0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 --2- 172.21.15.161:0/345457339 >> v2:172.21.15.179:3300/0 conn(0x7fe954143c40 0x7fe95415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 -- 172.21.15.161:0/345457339 >> 172.21.15.161:0/345457339 conn(0x7fe95406a2d0 msgr2=0x7fe95406a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:24.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 -- 172.21.15.161:0/345457339 shutdown_connections 2024-05-22T21:25:24.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 -- 172.21.15.161:0/345457339 wait complete. 2024-05-22T21:25:24.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 Processor -- start 2024-05-22T21:25:24.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 -- start start 2024-05-22T21:25:24.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe954143c40 0x7fe95413e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9541444a0 0x7fe95415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe954144900 0x7fe95415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe954154120 con 0x7fe9541444a0 2024-05-22T21:25:24.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9541552b0 con 0x7fe954143c40 2024-05-22T21:25:24.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe959c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95413ec10 con 0x7fe954144900 2024-05-22T21:25:24.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe94a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9541444a0 0x7fe95415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe94a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9541444a0 0x7fe95415a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59364/0 (socket says 172.21.15.161:59364) 2024-05-22T21:25:24.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe94a000640 1 -- 172.21.15.161:0/2971431467 learned_addr learned my addr 172.21.15.161:0/2971431467 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:24.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe953400640 1 --2- 172.21.15.161:0/2971431467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe954144900 0x7fe95415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe952a00640 1 --2- 172.21.15.161:0/2971431467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe954143c40 0x7fe95413e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe94a000640 1 -- 172.21.15.161:0/2971431467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe954144900 msgr2=0x7fe95415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe94a000640 1 --2- 172.21.15.161:0/2971431467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe954144900 0x7fe95415a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe94a000640 1 -- 172.21.15.161:0/2971431467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe954143c40 msgr2=0x7fe95413e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe94a000640 1 --2- 172.21.15.161:0/2971431467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe954143c40 0x7fe95413e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.806+0000 7fe94a000640 1 -- 172.21.15.161:0/2971431467 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93c0094d0 con 0x7fe9541444a0 2024-05-22T21:25:24.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.810+0000 7fe94a000640 1 --2- 172.21.15.161:0/2971431467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9541444a0 0x7fe95415a1e0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fe93c01ca00 tx=0x7fe93c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:24.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.810+0000 7fe94b400640 1 -- 172.21.15.161:0/2971431467 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c0be3f0 con 0x7fe9541444a0 2024-05-22T21:25:24.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.810+0000 7fe94b400640 1 -- 172.21.15.161:0/2971431467 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93c0be550 con 0x7fe9541444a0 2024-05-22T21:25:24.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.810+0000 7fe94b400640 1 -- 172.21.15.161:0/2971431467 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c0be900 con 0x7fe9541444a0 2024-05-22T21:25:24.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.810+0000 7fe959c00640 1 -- 172.21.15.161:0/2971431467 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9541537f0 con 0x7fe9541444a0 2024-05-22T21:25:24.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.810+0000 7fe959c00640 1 -- 172.21.15.161:0/2971431467 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe954153db0 con 0x7fe9541444a0 2024-05-22T21:25:24.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.810+0000 7fe959c00640 1 -- 172.21.15.161:0/2971431467 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe954067220 con 0x7fe9541444a0 2024-05-22T21:25:24.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.810+0000 7fe94b400640 1 -- 172.21.15.161:0/2971431467 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe93c0b43f0 con 0x7fe9541444a0 2024-05-22T21:25:24.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.810+0000 7fe94b400640 1 --2- 172.21.15.161:0/2971431467 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe930066690 0x7fe930068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:24.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.810+0000 7fe94b400640 1 -- 172.21.15.161:0/2971431467 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe93c11e470 con 0x7fe9541444a0 2024-05-22T21:25:24.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.810+0000 7fe952a00640 1 --2- 172.21.15.161:0/2971431467 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe930066690 0x7fe930068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:24.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.814+0000 7fe952a00640 1 --2- 172.21.15.161:0/2971431467 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe930066690 0x7fe930068b50 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe954153530 tx=0x7fe944009810 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:24.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.814+0000 7fe94b400640 1 -- 172.21.15.161:0/2971431467 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fe93c0ebc10 con 0x7fe9541444a0 2024-05-22T21:25:24.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.970+0000 7fe959c00640 1 -- 172.21.15.161:0/2971431467 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe954155480 con 0x7fe930066690 2024-05-22T21:25:24.971 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.970+0000 7fe94b400640 1 -- 172.21.15.161:0/2971431467 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe954155480 con 0x7fe930066690 2024-05-22T21:25:24.971 INFO:teuthology.orchestra.run.smithi161.stderr:instructed osd(s) 1 to deep-scrub 2024-05-22T21:25:24.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.970+0000 7fe959c00640 1 -- 172.21.15.161:0/2971431467 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe930066690 msgr2=0x7fe930068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.972 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.970+0000 7fe959c00640 1 --2- 172.21.15.161:0/2971431467 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe930066690 0x7fe930068b50 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe954153530 tx=0x7fe944009810 comp rx=0 tx=0).stop 2024-05-22T21:25:24.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.970+0000 7fe959c00640 1 -- 172.21.15.161:0/2971431467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9541444a0 msgr2=0x7fe95415a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:24.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.970+0000 7fe959c00640 1 --2- 172.21.15.161:0/2971431467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9541444a0 0x7fe95415a1e0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fe93c01ca00 tx=0x7fe93c01ad60 comp rx=0 tx=0).stop 2024-05-22T21:25:24.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.970+0000 7fe959c00640 1 -- 172.21.15.161:0/2971431467 shutdown_connections 2024-05-22T21:25:24.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.970+0000 7fe959c00640 1 --2- 172.21.15.161:0/2971431467 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe930066690 0x7fe930068b50 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.970+0000 7fe959c00640 1 --2- 172.21.15.161:0/2971431467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe954144900 0x7fe95415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.970+0000 7fe959c00640 1 --2- 172.21.15.161:0/2971431467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9541444a0 0x7fe95415a1e0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.970+0000 7fe959c00640 1 --2- 172.21.15.161:0/2971431467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe954143c40 0x7fe95413e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:24.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.970+0000 7fe959c00640 1 -- 172.21.15.161:0/2971431467 >> 172.21.15.161:0/2971431467 conn(0x7fe95406a2d0 msgr2=0x7fe95406c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:24.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.970+0000 7fe959c00640 1 -- 172.21.15.161:0/2971431467 shutdown_connections 2024-05-22T21:25:24.973 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:24.970+0000 7fe959c00640 1 -- 172.21.15.161:0/2971431467 wait complete. 2024-05-22T21:25:24.992 INFO:tasks.ceph:Scrubbing osd.2 2024-05-22T21:25:24.992 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-22T21:25:25.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.130+0000 7fe6ae200640 1 Processor -- start 2024-05-22T21:25:25.131 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.130+0000 7fe6ae200640 1 -- start start 2024-05-22T21:25:25.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.130+0000 7fe6ae200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe6a80ccc10 0x7fe6a815c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.130+0000 7fe6ae200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a806d0d0 con 0x7fe6a8144980 2024-05-22T21:25:25.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.130+0000 7fe6ae200640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a806c4c0 con 0x7fe6a806d560 2024-05-22T21:25:25.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.130+0000 7fe6ae200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a806c7a0 con 0x7fe6a80ccc10 2024-05-22T21:25:25.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.130+0000 7fe6a7e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe6a80ccc10 0x7fe6a815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.130+0000 7fe6a7400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe6a8144980 0x7fe6a815c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59514/0 (socket says 172.21.15.161:59514) 2024-05-22T21:25:25.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.130+0000 7fe6a7400640 1 -- 172.21.15.161:0/1767050677 learned_addr learned my addr 172.21.15.161:0/1767050677 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:25.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.134+0000 7fe6a7e00640 1 -- 172.21.15.161:0/1767050677 >> v1:172.21.15.179:6789/0 conn(0x7fe6a806d560 legacy=0x7fe6a80cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.134+0000 7fe6a7e00640 1 -- 172.21.15.161:0/1767050677 >> v1:172.21.15.161:6789/0 conn(0x7fe6a8144980 legacy=0x7fe6a815c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.134+0000 7fe6a7e00640 1 -- 172.21.15.161:0/1767050677 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a806ca80 con 0x7fe6a80ccc10 2024-05-22T21:25:25.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.134+0000 7fe6a6a00640 1 -- 172.21.15.161:0/1767050677 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132177225 0 0) 0x7fe6a806d0d0 con 0x7fe6a8144980 2024-05-22T21:25:25.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.134+0000 7fe6a7e00640 1 --2- 172.21.15.161:0/1767050677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6a80ccc10 0x7fe6a815c880 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00097c0 tx=0x7fe6a001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c6286f67fc2979 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:25.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.134+0000 7fe6a6a00640 1 -- 172.21.15.161:0/1767050677 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00ab600 con 0x7fe6a80ccc10 2024-05-22T21:25:25.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.134+0000 7fe6a6a00640 1 -- 172.21.15.161:0/1767050677 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a00a7050 con 0x7fe6a80ccc10 2024-05-22T21:25:25.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.134+0000 7fe6a6a00640 1 -- 172.21.15.161:0/1767050677 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00ab980 con 0x7fe6a80ccc10 2024-05-22T21:25:25.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.134+0000 7fe6ae200640 1 -- 172.21.15.161:0/1767050677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6a80ccc10 msgr2=0x7fe6a815c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.134+0000 7fe6ae200640 1 --2- 172.21.15.161:0/1767050677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6a80ccc10 0x7fe6a815c880 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00097c0 tx=0x7fe6a001c4a0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.134+0000 7fe6ae200640 1 -- 172.21.15.161:0/1767050677 shutdown_connections 2024-05-22T21:25:25.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.134+0000 7fe6ae200640 1 --2- 172.21.15.161:0/1767050677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6a80ccc10 0x7fe6a815c880 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.134+0000 7fe6ae200640 1 -- 172.21.15.161:0/1767050677 >> 172.21.15.161:0/1767050677 conn(0x7fe6a806a0a0 msgr2=0x7fe6a806a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:25.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.134+0000 7fe6ae200640 1 -- 172.21.15.161:0/1767050677 shutdown_connections 2024-05-22T21:25:25.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6ae200640 1 -- 172.21.15.161:0/1767050677 wait complete. 2024-05-22T21:25:25.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6ae200640 1 Processor -- start 2024-05-22T21:25:25.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6ae200640 1 -- start start 2024-05-22T21:25:25.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6ae200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6a806d560 0x7fe6a80aab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6ae200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6a80ccc10 0x7fe6a813d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6ae200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6a8144980 0x7fe6a81f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6ae200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a81420a0 con 0x7fe6a806d560 2024-05-22T21:25:25.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6ae200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a8141a90 con 0x7fe6a80ccc10 2024-05-22T21:25:25.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6ae200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a80a9270 con 0x7fe6a8144980 2024-05-22T21:25:25.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6a7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6a806d560 0x7fe6a80aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6a7400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6a806d560 0x7fe6a80aab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59388/0 (socket says 172.21.15.161:59388) 2024-05-22T21:25:25.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6a7400640 1 -- 172.21.15.161:0/1279695512 learned_addr learned my addr 172.21.15.161:0/1279695512 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:25.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe69ea00640 1 --2- 172.21.15.161:0/1279695512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6a80ccc10 0x7fe6a813d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6a7e00640 1 --2- 172.21.15.161:0/1279695512 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6a8144980 0x7fe6a81f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6a7400640 1 -- 172.21.15.161:0/1279695512 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6a8144980 msgr2=0x7fe6a81f7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6a7400640 1 --2- 172.21.15.161:0/1279695512 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6a8144980 0x7fe6a81f7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6a7400640 1 -- 172.21.15.161:0/1279695512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6a80ccc10 msgr2=0x7fe6a813d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6a7400640 1 --2- 172.21.15.161:0/1279695512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6a80ccc10 0x7fe6a813d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6a7400640 1 -- 172.21.15.161:0/1279695512 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a0009420 con 0x7fe6a806d560 2024-05-22T21:25:25.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6a7400640 1 --2- 172.21.15.161:0/1279695512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6a806d560 0x7fe6a80aab90 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fe69800c780 tx=0x7fe69800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:25.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe69fe00640 1 -- 172.21.15.161:0/1279695512 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69800ce90 con 0x7fe6a806d560 2024-05-22T21:25:25.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe69fe00640 1 -- 172.21.15.161:0/1279695512 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69800a640 con 0x7fe6a806d560 2024-05-22T21:25:25.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe69fe00640 1 -- 172.21.15.161:0/1279695512 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69800aa70 con 0x7fe6a806d560 2024-05-22T21:25:25.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6ae200640 1 -- 172.21.15.161:0/1279695512 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a80a9550 con 0x7fe6a806d560 2024-05-22T21:25:25.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.138+0000 7fe6ae200640 1 -- 172.21.15.161:0/1279695512 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a80ab0d0 con 0x7fe6a806d560 2024-05-22T21:25:25.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.142+0000 7fe6ae200640 1 -- 172.21.15.161:0/1279695512 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe6a8168d10 con 0x7fe6a806d560 2024-05-22T21:25:25.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.142+0000 7fe69fe00640 1 -- 172.21.15.161:0/1279695512 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe69800abd0 con 0x7fe6a806d560 2024-05-22T21:25:25.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.142+0000 7fe69fe00640 1 --2- 172.21.15.161:0/1279695512 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe684066690 0x7fe684068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.142+0000 7fe69fe00640 1 -- 172.21.15.161:0/1279695512 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe698083430 con 0x7fe6a806d560 2024-05-22T21:25:25.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.142+0000 7fe69ea00640 1 --2- 172.21.15.161:0/1279695512 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe684066690 0x7fe684068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.142+0000 7fe69fe00640 1 --2- 172.21.15.161:0/1279695512 >> [v2:172.21.15.161:6824/23414,v1:172.21.15.161:6825/23414] conn(0x7fe68406c380 0x7fe68406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.142+0000 7fe69fe00640 1 -- 172.21.15.161:0/1279695512 --> [v2:172.21.15.161:6824/23414,v1:172.21.15.161:6825/23414] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe68406ee50 con 0x7fe68406c380 2024-05-22T21:25:25.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.142+0000 7fe69fe00640 1 -- 172.21.15.161:0/1279695512 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=197) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe698085a10 con 0x7fe6a806d560 2024-05-22T21:25:25.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.142+0000 7fe6a7e00640 1 --2- 172.21.15.161:0/1279695512 >> [v2:172.21.15.161:6824/23414,v1:172.21.15.161:6825/23414] conn(0x7fe68406c380 0x7fe68406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.144 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.142+0000 7fe69ea00640 1 --2- 172.21.15.161:0/1279695512 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe684066690 0x7fe684068b50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8141010 tx=0x7fe690009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:25.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.142+0000 7fe6a7e00640 1 --2- 172.21.15.161:0/1279695512 >> [v2:172.21.15.161:6824/23414,v1:172.21.15.161:6825/23414] conn(0x7fe68406c380 0x7fe68406e7a0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:25.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.146+0000 7fe69fe00640 1 -- 172.21.15.161:0/1279695512 <== osd.2 v2:172.21.15.161:6824/23414 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24459 (crc 0 0 0) 0x7fe68406ee50 con 0x7fe68406c380 2024-05-22T21:25:25.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.154+0000 7fe6ae200640 1 -- 172.21.15.161:0/1279695512 --> [v2:172.21.15.161:6824/23414,v1:172.21.15.161:6825/23414] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe6a81421e0 con 0x7fe68406c380 2024-05-22T21:25:25.156 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.154+0000 7fe69fe00640 1 -- 172.21.15.161:0/1279695512 <== osd.2 v2:172.21.15.161:6824/23414 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7fe6a00ab700 con 0x7fe68406c380 2024-05-22T21:25:25.156 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:25:25.157 INFO:teuthology.orchestra.run.smithi161.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-05-22T21:25:25.157 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:25:25.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.154+0000 7fe6ae200640 1 -- 172.21.15.161:0/1279695512 >> [v2:172.21.15.161:6824/23414,v1:172.21.15.161:6825/23414] conn(0x7fe68406c380 msgr2=0x7fe68406e7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.154+0000 7fe6ae200640 1 --2- 172.21.15.161:0/1279695512 >> [v2:172.21.15.161:6824/23414,v1:172.21.15.161:6825/23414] conn(0x7fe68406c380 0x7fe68406e7a0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.154+0000 7fe6ae200640 1 -- 172.21.15.161:0/1279695512 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe684066690 msgr2=0x7fe684068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.154+0000 7fe6ae200640 1 --2- 172.21.15.161:0/1279695512 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fe684066690 0x7fe684068b50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8141010 tx=0x7fe690009040 comp rx=0 tx=0).stop 2024-05-22T21:25:25.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.158+0000 7fe6ae200640 1 -- 172.21.15.161:0/1279695512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6a806d560 msgr2=0x7fe6a80aab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.158+0000 7fe6ae200640 1 --2- 172.21.15.161:0/1279695512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6a806d560 0x7fe6a80aab90 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fe69800c780 tx=0x7fe69800cc40 comp rx=0 tx=0).stop 2024-05-22T21:25:25.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.158+0000 7fe6a7e00640 1 -- 172.21.15.161:0/1279695512 reap_dead start 2024-05-22T21:25:25.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.158+0000 7fe6ae200640 1 -- 172.21.15.161:0/1279695512 shutdown_connections 2024-05-22T21:25:25.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.158+0000 7fe6ae200640 1 -- 172.21.15.161:0/1279695512 >> 172.21.15.161:0/1279695512 conn(0x7fe6a806a0a0 msgr2=0x7fe6a815b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:25.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.158+0000 7fe6ae200640 1 -- 172.21.15.161:0/1279695512 shutdown_connections 2024-05-22T21:25:25.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.158+0000 7fe6ae200640 1 -- 172.21.15.161:0/1279695512 wait complete. 2024-05-22T21:25:25.176 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-22T21:25:25.236 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:25:25.240 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:25:25.313 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.314+0000 7f03d8800640 1 Processor -- start 2024-05-22T21:25:25.314 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.314+0000 7f03d8800640 1 -- start start 2024-05-22T21:25:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.314+0000 7f03d8800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f03d01402e0 0x7f03d013f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.314+0000 7f03d8800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f03d013fc80 0x7f03d006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.314+0000 7f03d8800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d0140090 con 0x7f03d013cfc0 2024-05-22T21:25:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.314+0000 7f03d8800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d013d3a0 con 0x7f03d01402e0 2024-05-22T21:25:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.314+0000 7f03d8800640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d00cd920 con 0x7f03d013fc80 2024-05-22T21:25:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.314+0000 7f03d5400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f03d01402e0 0x7f03d013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.314+0000 7f03d6800640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f03d013fc80 0x7f03d006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.314+0000 7f03d5400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f03d01402e0 0x7f03d013f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59396/0 (socket says 172.21.15.161:59396) 2024-05-22T21:25:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.314+0000 7f03d5400640 1 -- 172.21.15.161:0/3762176710 learned_addr learned my addr 172.21.15.161:0/3762176710 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.314+0000 7f03d5400640 1 -- 172.21.15.161:0/3762176710 >> v2:172.21.15.179:3300/0 conn(0x7f03d013fc80 msgr2=0x7f03d006fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.314+0000 7f03d5400640 1 --2- 172.21.15.161:0/3762176710 >> v2:172.21.15.179:3300/0 conn(0x7f03d013fc80 0x7f03d006fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.314+0000 7f03d5400640 1 -- 172.21.15.161:0/3762176710 >> v1:172.21.15.179:6789/0 conn(0x7f03d013cfc0 legacy=0x7f03d015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.317 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.314+0000 7f03d5400640 1 -- 172.21.15.161:0/3762176710 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d00cdc00 con 0x7f03d01402e0 2024-05-22T21:25:25.318 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.318+0000 7f03d5400640 1 --2- 172.21.15.161:0/3762176710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03d01402e0 0x7f03d013f740 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f03d006c5e0 tx=0x7f03c001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3eb86c9682e040 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:25.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.318+0000 7f03d4a00640 1 -- 172.21.15.161:0/3762176710 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c00ab600 con 0x7f03d01402e0 2024-05-22T21:25:25.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.318+0000 7f03d4a00640 1 -- 172.21.15.161:0/3762176710 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c00a7050 con 0x7f03d01402e0 2024-05-22T21:25:25.319 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.318+0000 7f03d4a00640 1 -- 172.21.15.161:0/3762176710 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c00ab980 con 0x7f03d01402e0 2024-05-22T21:25:25.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.318+0000 7f03d8800640 1 -- 172.21.15.161:0/3762176710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03d01402e0 msgr2=0x7f03d013f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.318+0000 7f03d8800640 1 --2- 172.21.15.161:0/3762176710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03d01402e0 0x7f03d013f740 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f03d006c5e0 tx=0x7f03c001c460 comp rx=0 tx=0).stop 2024-05-22T21:25:25.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.318+0000 7f03d8800640 1 -- 172.21.15.161:0/3762176710 shutdown_connections 2024-05-22T21:25:25.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.318+0000 7f03d8800640 1 --2- 172.21.15.161:0/3762176710 >> v2:172.21.15.179:3300/0 conn(0x7f03d013fc80 0x7f03d006fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.318+0000 7f03d8800640 1 --2- 172.21.15.161:0/3762176710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03d01402e0 0x7f03d013f740 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.318+0000 7f03d8800640 1 -- 172.21.15.161:0/3762176710 >> 172.21.15.161:0/3762176710 conn(0x7f03d006a2d0 msgr2=0x7f03d006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:25.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.318+0000 7f03d8800640 1 -- 172.21.15.161:0/3762176710 shutdown_connections 2024-05-22T21:25:25.320 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.318+0000 7f03d8800640 1 -- 172.21.15.161:0/3762176710 wait complete. 2024-05-22T21:25:25.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d8800640 1 Processor -- start 2024-05-22T21:25:25.321 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d8800640 1 -- start start 2024-05-22T21:25:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d8800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03d013cfc0 0x7f03d0159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d8800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03d013fc80 0x7f03d013ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d8800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03d01402e0 0x7f03d015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d8800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d013df30 con 0x7f03d01402e0 2024-05-22T21:25:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d8800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d0159df0 con 0x7f03d013fc80 2024-05-22T21:25:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d8800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d013e1d0 con 0x7f03d013cfc0 2024-05-22T21:25:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d6800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03d01402e0 0x7f03d015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d6800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03d01402e0 0x7f03d015bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59410/0 (socket says 172.21.15.161:59410) 2024-05-22T21:25:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d6800640 1 -- 172.21.15.161:0/1246760614 learned_addr learned my addr 172.21.15.161:0/1246760614 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d5400640 1 --2- 172.21.15.161:0/1246760614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03d013fc80 0x7f03d013ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d5e00640 1 --2- 172.21.15.161:0/1246760614 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03d013cfc0 0x7f03d0159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d6800640 1 -- 172.21.15.161:0/1246760614 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03d013cfc0 msgr2=0x7f03d0159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d6800640 1 --2- 172.21.15.161:0/1246760614 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03d013cfc0 0x7f03d0159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d6800640 1 -- 172.21.15.161:0/1246760614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03d013fc80 msgr2=0x7f03d013ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d6800640 1 --2- 172.21.15.161:0/1246760614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03d013fc80 0x7f03d013ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d6800640 1 -- 172.21.15.161:0/1246760614 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c00094d0 con 0x7f03d01402e0 2024-05-22T21:25:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d5400640 1 --2- 172.21.15.161:0/1246760614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03d013fc80 0x7f03d013ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d6800640 1 --2- 172.21.15.161:0/1246760614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03d01402e0 0x7f03d015bdb0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00efc0 tx=0x7f03cc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03ca000640 1 -- 172.21.15.161:0/1246760614 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc014480 con 0x7f03d01402e0 2024-05-22T21:25:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03ca000640 1 -- 172.21.15.161:0/1246760614 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03cc00f040 con 0x7f03d01402e0 2024-05-22T21:25:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03ca000640 1 -- 172.21.15.161:0/1246760614 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc014820 con 0x7f03d01402e0 2024-05-22T21:25:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d8800640 1 -- 172.21.15.161:0/1246760614 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d013e4b0 con 0x7f03d01402e0 2024-05-22T21:25:25.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.322+0000 7f03d8800640 1 -- 172.21.15.161:0/1246760614 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d015c2f0 con 0x7f03d01402e0 2024-05-22T21:25:25.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.326+0000 7f03d8800640 1 -- 172.21.15.161:0/1246760614 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d0067220 con 0x7f03d01402e0 2024-05-22T21:25:25.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.326+0000 7f03ca000640 1 -- 172.21.15.161:0/1246760614 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f03cc014980 con 0x7f03d01402e0 2024-05-22T21:25:25.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.326+0000 7f03ca000640 1 --2- 172.21.15.161:0/1246760614 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f03b4066690 0x7f03b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.326+0000 7f03ca000640 1 -- 172.21.15.161:0/1246760614 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f03cc084a70 con 0x7f03d01402e0 2024-05-22T21:25:25.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.326+0000 7f03d5e00640 1 --2- 172.21.15.161:0/1246760614 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f03b4066690 0x7f03b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.328 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.326+0000 7f03d5e00640 1 --2- 172.21.15.161:0/1246760614 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f03b4066690 0x7f03b4068b50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000f70 tx=0x7f03c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:25.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.330+0000 7f03ca000640 1 -- 172.21.15.161:0/1246760614 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f03cc052040 con 0x7f03d01402e0 2024-05-22T21:25:25.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.486+0000 7f03d8800640 1 -- 172.21.15.161:0/1246760614 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f03d0166e70 con 0x7f03b4066690 2024-05-22T21:25:25.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.486+0000 7f03ca000640 1 -- 172.21.15.161:0/1246760614 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f03c4001370 con 0x7f03b4066690 2024-05-22T21:25:25.486 INFO:teuthology.orchestra.run.smithi161.stderr:instructed osd(s) 2 to deep-scrub 2024-05-22T21:25:25.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.486+0000 7f03d8800640 1 -- 172.21.15.161:0/1246760614 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f03b4066690 msgr2=0x7f03b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.486+0000 7f03d8800640 1 --2- 172.21.15.161:0/1246760614 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f03b4066690 0x7f03b4068b50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000f70 tx=0x7f03c4009040 comp rx=0 tx=0).stop 2024-05-22T21:25:25.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.486+0000 7f03d8800640 1 -- 172.21.15.161:0/1246760614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03d01402e0 msgr2=0x7f03d015bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.486+0000 7f03d8800640 1 --2- 172.21.15.161:0/1246760614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03d01402e0 0x7f03d015bdb0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00efc0 tx=0x7f03cc00c490 comp rx=0 tx=0).stop 2024-05-22T21:25:25.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.486+0000 7f03d8800640 1 -- 172.21.15.161:0/1246760614 shutdown_connections 2024-05-22T21:25:25.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.486+0000 7f03d8800640 1 --2- 172.21.15.161:0/1246760614 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f03b4066690 0x7f03b4068b50 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.486+0000 7f03d8800640 1 --2- 172.21.15.161:0/1246760614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03d01402e0 0x7f03d015bdb0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.486+0000 7f03d8800640 1 --2- 172.21.15.161:0/1246760614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03d013fc80 0x7f03d013ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.486+0000 7f03d8800640 1 --2- 172.21.15.161:0/1246760614 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03d013cfc0 0x7f03d0159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.486+0000 7f03d8800640 1 -- 172.21.15.161:0/1246760614 >> 172.21.15.161:0/1246760614 conn(0x7f03d006a2d0 msgr2=0x7f03d00b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:25.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.486+0000 7f03d8800640 1 -- 172.21.15.161:0/1246760614 shutdown_connections 2024-05-22T21:25:25.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.486+0000 7f03d8800640 1 -- 172.21.15.161:0/1246760614 wait complete. 2024-05-22T21:25:25.507 INFO:tasks.ceph:Scrubbing osd.3 2024-05-22T21:25:25.507 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-22T21:25:25.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.690+0000 7f58e4200640 1 Processor -- start 2024-05-22T21:25:25.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.690+0000 7f58e4200640 1 -- start start 2024-05-22T21:25:25.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.690+0000 7f58e4200640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f58dc144980 0x7f58dc15c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.690+0000 7f58e4200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f58dc0ccc10 0x7f58dc15c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.690+0000 7f58e4200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58dc06d0d0 con 0x7f58dc06d560 2024-05-22T21:25:25.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.690+0000 7f58e4200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58dc15c680 con 0x7f58dc0ccc10 2024-05-22T21:25:25.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.690+0000 7f58e4200640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58dc06c4c0 con 0x7f58dc144980 2024-05-22T21:25:25.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.690+0000 7f58e2200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f58dc0ccc10 0x7f58dc15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.690+0000 7f58e0e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f58dc06d560 0x7f58dc0cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59524/0 (socket says 172.21.15.161:59524) 2024-05-22T21:25:25.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.690+0000 7f58e0e00640 1 -- 172.21.15.161:0/1646937115 learned_addr learned my addr 172.21.15.161:0/1646937115 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:25.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.690+0000 7f58e2200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f58dc0ccc10 0x7f58dc15c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59424/0 (socket says 172.21.15.161:59424) 2024-05-22T21:25:25.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.690+0000 7f58e1800640 1 --2- 172.21.15.161:0/1646937115 >> v2:172.21.15.179:3300/0 conn(0x7f58dc144980 0x7f58dc15c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.690+0000 7f58e2200640 1 -- 172.21.15.161:0/1646937115 >> v2:172.21.15.179:3300/0 conn(0x7f58dc144980 msgr2=0x7f58dc15c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:25.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.690+0000 7f58e2200640 1 --2- 172.21.15.161:0/1646937115 >> v2:172.21.15.179:3300/0 conn(0x7f58dc144980 0x7f58dc15c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.690+0000 7f58e2200640 1 -- 172.21.15.161:0/1646937115 >> v1:172.21.15.161:6789/0 conn(0x7f58dc06d560 legacy=0x7f58dc0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.690+0000 7f58e2200640 1 -- 172.21.15.161:0/1646937115 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc06c7a0 con 0x7f58dc0ccc10 2024-05-22T21:25:25.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.694+0000 7f58e2200640 1 --2- 172.21.15.161:0/1646937115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58dc0ccc10 0x7f58dc15c880 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f58dc071fd0 tx=0x7f58d401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0f19408a4c0de0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:25.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.694+0000 7f58dbe00640 1 -- 172.21.15.161:0/1646937115 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d40a58e0 con 0x7f58dc0ccc10 2024-05-22T21:25:25.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.694+0000 7f58dbe00640 1 -- 172.21.15.161:0/1646937115 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d40a5a40 con 0x7f58dc0ccc10 2024-05-22T21:25:25.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.694+0000 7f58dbe00640 1 -- 172.21.15.161:0/1646937115 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d40a5dd0 con 0x7f58dc0ccc10 2024-05-22T21:25:25.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.694+0000 7f58e4200640 1 -- 172.21.15.161:0/1646937115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58dc0ccc10 msgr2=0x7f58dc15c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.694+0000 7f58e4200640 1 --2- 172.21.15.161:0/1646937115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58dc0ccc10 0x7f58dc15c880 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f58dc071fd0 tx=0x7f58d401c930 comp rx=0 tx=0).stop 2024-05-22T21:25:25.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.694+0000 7f58e4200640 1 -- 172.21.15.161:0/1646937115 shutdown_connections 2024-05-22T21:25:25.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.694+0000 7f58e4200640 1 --2- 172.21.15.161:0/1646937115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58dc0ccc10 0x7f58dc15c880 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.694+0000 7f58e4200640 1 --2- 172.21.15.161:0/1646937115 >> v2:172.21.15.179:3300/0 conn(0x7f58dc144980 0x7f58dc15c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.694+0000 7f58e4200640 1 -- 172.21.15.161:0/1646937115 >> 172.21.15.161:0/1646937115 conn(0x7f58dc06a0a0 msgr2=0x7f58dc06a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:25.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.694+0000 7f58e4200640 1 -- 172.21.15.161:0/1646937115 shutdown_connections 2024-05-22T21:25:25.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.694+0000 7f58e4200640 1 -- 172.21.15.161:0/1646937115 wait complete. 2024-05-22T21:25:25.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e4200640 1 Processor -- start 2024-05-22T21:25:25.698 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e4200640 1 -- start start 2024-05-22T21:25:25.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e4200640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58dc06d560 0x7f58dc13dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e4200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58dc0ccc10 0x7f58dc1422a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e4200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58dc144980 0x7f58dc0a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e4200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc141a70 con 0x7f58dc0ccc10 2024-05-22T21:25:25.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e4200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc140df0 con 0x7f58dc144980 2024-05-22T21:25:25.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e4200640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc1410d0 con 0x7f58dc06d560 2024-05-22T21:25:25.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e0e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58dc0ccc10 0x7f58dc1422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e0e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58dc0ccc10 0x7f58dc1422a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59426/0 (socket says 172.21.15.161:59426) 2024-05-22T21:25:25.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e0e00640 1 -- 172.21.15.161:0/2416555278 learned_addr learned my addr 172.21.15.161:0/2416555278 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:25.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e1800640 1 --2- 172.21.15.161:0/2416555278 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58dc06d560 0x7f58dc13dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.699 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e2200640 1 --2- 172.21.15.161:0/2416555278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58dc144980 0x7f58dc0a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e0e00640 1 -- 172.21.15.161:0/2416555278 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58dc06d560 msgr2=0x7f58dc13dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e0e00640 1 --2- 172.21.15.161:0/2416555278 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58dc06d560 0x7f58dc13dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e0e00640 1 -- 172.21.15.161:0/2416555278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58dc144980 msgr2=0x7f58dc0a9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e0e00640 1 --2- 172.21.15.161:0/2416555278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58dc144980 0x7f58dc0a9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e0e00640 1 -- 172.21.15.161:0/2416555278 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d4009a70 con 0x7f58dc0ccc10 2024-05-22T21:25:25.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e0e00640 1 --2- 172.21.15.161:0/2416555278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58dc0ccc10 0x7f58dc1422a0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f58c800c420 tx=0x7f58c800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:25.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58d9600640 1 -- 172.21.15.161:0/2416555278 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8005cf0 con 0x7f58dc0ccc10 2024-05-22T21:25:25.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58d9600640 1 -- 172.21.15.161:0/2416555278 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c8005e50 con 0x7f58dc0ccc10 2024-05-22T21:25:25.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e4200640 1 -- 172.21.15.161:0/2416555278 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc1413b0 con 0x7f58dc0ccc10 2024-05-22T21:25:25.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58d9600640 1 -- 172.21.15.161:0/2416555278 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c80072f0 con 0x7f58dc0ccc10 2024-05-22T21:25:25.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e4200640 1 -- 172.21.15.161:0/2416555278 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc13e1f0 con 0x7f58dc0ccc10 2024-05-22T21:25:25.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.698+0000 7f58e4200640 1 -- 172.21.15.161:0/2416555278 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f58dc1589d0 con 0x7f58dc0ccc10 2024-05-22T21:25:25.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.702+0000 7f58d9600640 1 -- 172.21.15.161:0/2416555278 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f58c8007450 con 0x7f58dc0ccc10 2024-05-22T21:25:25.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.702+0000 7f58d9600640 1 --2- 172.21.15.161:0/2416555278 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f58bc066690 0x7f58bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.702+0000 7f58d9600640 1 -- 172.21.15.161:0/2416555278 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f58c8083490 con 0x7f58dc0ccc10 2024-05-22T21:25:25.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.702+0000 7f58e1800640 1 --2- 172.21.15.161:0/2416555278 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f58bc066690 0x7f58bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.702+0000 7f58d9600640 1 --2- 172.21.15.161:0/2416555278 >> [v2:172.21.15.161:6816/23415,v1:172.21.15.161:6817/23415] conn(0x7f58bc06c4f0 0x7f58bc06e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.702+0000 7f58d9600640 1 -- 172.21.15.161:0/2416555278 --> [v2:172.21.15.161:6816/23415,v1:172.21.15.161:6817/23415] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f58bc06efc0 con 0x7f58bc06c4f0 2024-05-22T21:25:25.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.702+0000 7f58d9600640 1 -- 172.21.15.161:0/2416555278 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=197) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58c8085a70 con 0x7f58dc0ccc10 2024-05-22T21:25:25.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.702+0000 7f58e2200640 1 --2- 172.21.15.161:0/2416555278 >> [v2:172.21.15.161:6816/23415,v1:172.21.15.161:6817/23415] conn(0x7f58bc06c4f0 0x7f58bc06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.702+0000 7f58e1800640 1 --2- 172.21.15.161:0/2416555278 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f58bc066690 0x7f58bc068b50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f58d000b930 tx=0x7f58d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:25.704 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.702+0000 7f58e2200640 1 --2- 172.21.15.161:0/2416555278 >> [v2:172.21.15.161:6816/23415,v1:172.21.15.161:6817/23415] conn(0x7f58bc06c4f0 0x7f58bc06e910 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:25.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.702+0000 7f58d9600640 1 -- 172.21.15.161:0/2416555278 <== osd.3 v2:172.21.15.161:6816/23415 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24459 (crc 0 0 0) 0x7f58bc06efc0 con 0x7f58bc06c4f0 2024-05-22T21:25:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.714+0000 7f58e4200640 1 -- 172.21.15.161:0/2416555278 --> [v2:172.21.15.161:6816/23415,v1:172.21.15.161:6817/23415] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f58dc06d990 con 0x7f58bc06c4f0 2024-05-22T21:25:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.714+0000 7f58d9600640 1 -- 172.21.15.161:0/2416555278 <== osd.3 v2:172.21.15.161:6816/23415 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f58d4009450 con 0x7f58bc06c4f0 2024-05-22T21:25:25.716 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:25:25.716 INFO:teuthology.orchestra.run.smithi161.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-05-22T21:25:25.716 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:25:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.714+0000 7f58e4200640 1 -- 172.21.15.161:0/2416555278 >> [v2:172.21.15.161:6816/23415,v1:172.21.15.161:6817/23415] conn(0x7f58bc06c4f0 msgr2=0x7f58bc06e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.714+0000 7f58e4200640 1 --2- 172.21.15.161:0/2416555278 >> [v2:172.21.15.161:6816/23415,v1:172.21.15.161:6817/23415] conn(0x7f58bc06c4f0 0x7f58bc06e910 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.714+0000 7f58e4200640 1 -- 172.21.15.161:0/2416555278 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f58bc066690 msgr2=0x7f58bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.714+0000 7f58e4200640 1 --2- 172.21.15.161:0/2416555278 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f58bc066690 0x7f58bc068b50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f58d000b930 tx=0x7f58d0008040 comp rx=0 tx=0).stop 2024-05-22T21:25:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.714+0000 7f58e4200640 1 -- 172.21.15.161:0/2416555278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58dc0ccc10 msgr2=0x7f58dc1422a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.714+0000 7f58e4200640 1 --2- 172.21.15.161:0/2416555278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58dc0ccc10 0x7f58dc1422a0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f58c800c420 tx=0x7f58c800c8e0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.714+0000 7f58e2200640 1 -- 172.21.15.161:0/2416555278 reap_dead start 2024-05-22T21:25:25.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.714+0000 7f58e4200640 1 -- 172.21.15.161:0/2416555278 shutdown_connections 2024-05-22T21:25:25.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.714+0000 7f58e4200640 1 -- 172.21.15.161:0/2416555278 >> 172.21.15.161:0/2416555278 conn(0x7f58dc06a0a0 msgr2=0x7f58dc1438b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:25.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.714+0000 7f58e4200640 1 -- 172.21.15.161:0/2416555278 shutdown_connections 2024-05-22T21:25:25.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.714+0000 7f58e4200640 1 -- 172.21.15.161:0/2416555278 wait complete. 2024-05-22T21:25:25.735 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-22T21:25:25.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.870+0000 7f2453400640 1 Processor -- start 2024-05-22T21:25:25.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.870+0000 7f2453400640 1 -- start start 2024-05-22T21:25:25.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.870+0000 7f2453400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f244c06c9b0 0x7f244c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.870+0000 7f2453400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244c144bb0 con 0x7f244c143c40 2024-05-22T21:25:25.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.870+0000 7f2453400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244c15efb0 con 0x7f244c15b600 2024-05-22T21:25:25.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.870+0000 7f2453400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244c15ba60 con 0x7f244c06c9b0 2024-05-22T21:25:25.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.870+0000 7f2450a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f244c143c40 0x7f244c1444a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60552/0 (socket says 172.21.15.161:60552) 2024-05-22T21:25:25.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.870+0000 7f2450a00640 1 -- 172.21.15.161:0/4142204712 learned_addr learned my addr 172.21.15.161:0/4142204712 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:25.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.870+0000 7f2451400640 1 --2- 172.21.15.161:0/4142204712 >> v2:172.21.15.161:3300/0 conn(0x7f244c06c9b0 0x7f244c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.870+0000 7f2451400640 1 -- 172.21.15.161:0/4142204712 >> v1:172.21.15.179:6789/0 conn(0x7f244c15b600 legacy=0x7f244c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:25.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.870+0000 7f2451400640 1 -- 172.21.15.161:0/4142204712 >> v1:172.21.15.161:6789/0 conn(0x7f244c143c40 legacy=0x7f244c1444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.870+0000 7f2451400640 1 -- 172.21.15.161:0/4142204712 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c15f290 con 0x7f244c06c9b0 2024-05-22T21:25:25.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.870+0000 7f244b400640 1 -- 172.21.15.161:0/4142204712 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242611973 0 0) 0x7f244c144bb0 con 0x7f244c143c40 2024-05-22T21:25:25.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.874+0000 7f2451400640 1 --2- 172.21.15.161:0/4142204712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f244c06c9b0 0x7f244c0b2160 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f2444009d90 tx=0x7f244401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97a14dfd5fc1851 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:25.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.874+0000 7f244b400640 1 -- 172.21.15.161:0/4142204712 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440a59f0 con 0x7f244c06c9b0 2024-05-22T21:25:25.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.874+0000 7f244b400640 1 -- 172.21.15.161:0/4142204712 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24440a5b50 con 0x7f244c06c9b0 2024-05-22T21:25:25.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.874+0000 7f244b400640 1 -- 172.21.15.161:0/4142204712 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440ad440 con 0x7f244c06c9b0 2024-05-22T21:25:25.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.874+0000 7f2453400640 1 -- 172.21.15.161:0/4142204712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f244c06c9b0 msgr2=0x7f244c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.874+0000 7f2453400640 1 --2- 172.21.15.161:0/4142204712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f244c06c9b0 0x7f244c0b2160 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f2444009d90 tx=0x7f244401c930 comp rx=0 tx=0).stop 2024-05-22T21:25:25.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.874+0000 7f2453400640 1 -- 172.21.15.161:0/4142204712 shutdown_connections 2024-05-22T21:25:25.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.874+0000 7f2453400640 1 --2- 172.21.15.161:0/4142204712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f244c06c9b0 0x7f244c0b2160 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.874+0000 7f2453400640 1 -- 172.21.15.161:0/4142204712 >> 172.21.15.161:0/4142204712 conn(0x7f244c06a2d0 msgr2=0x7f244c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:25.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.874+0000 7f2453400640 1 -- 172.21.15.161:0/4142204712 shutdown_connections 2024-05-22T21:25:25.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.874+0000 7f2453400640 1 -- 172.21.15.161:0/4142204712 wait complete. 2024-05-22T21:25:25.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2453400640 1 Processor -- start 2024-05-22T21:25:25.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2453400640 1 -- start start 2024-05-22T21:25:25.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2453400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f244c06c9b0 0x7f244c1550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2453400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f244c143c40 0x7f244c151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2453400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f244c15b600 0x7f244c135d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2453400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c154c90 con 0x7f244c15b600 2024-05-22T21:25:25.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2453400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c152320 con 0x7f244c06c9b0 2024-05-22T21:25:25.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2453400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c1525a0 con 0x7f244c143c40 2024-05-22T21:25:25.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2451400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f244c15b600 0x7f244c135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2451400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f244c15b600 0x7f244c135d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42068/0 (socket says 172.21.15.161:42068) 2024-05-22T21:25:25.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2451400640 1 -- 172.21.15.161:0/3215500655 learned_addr learned my addr 172.21.15.161:0/3215500655 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:25.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2450a00640 1 --2- 172.21.15.161:0/3215500655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f244c06c9b0 0x7f244c1550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f244be00640 1 --2- 172.21.15.161:0/3215500655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f244c143c40 0x7f244c151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2451400640 1 -- 172.21.15.161:0/3215500655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f244c143c40 msgr2=0x7f244c151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2451400640 1 --2- 172.21.15.161:0/3215500655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f244c143c40 0x7f244c151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2451400640 1 -- 172.21.15.161:0/3215500655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f244c06c9b0 msgr2=0x7f244c1550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:25.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2451400640 1 --2- 172.21.15.161:0/3215500655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f244c06c9b0 0x7f244c1550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:25.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2451400640 1 -- 172.21.15.161:0/3215500655 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2444009a70 con 0x7f244c15b600 2024-05-22T21:25:25.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2451400640 1 --2- 172.21.15.161:0/3215500655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f244c15b600 0x7f244c135d80 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f24440b0040 tx=0x7f24440196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:25.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2448c00640 1 -- 172.21.15.161:0/3215500655 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440ade20 con 0x7f244c15b600 2024-05-22T21:25:25.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2448c00640 1 -- 172.21.15.161:0/3215500655 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24440b7410 con 0x7f244c15b600 2024-05-22T21:25:25.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2453400640 1 -- 172.21.15.161:0/3215500655 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244c1542f0 con 0x7f244c15b600 2024-05-22T21:25:25.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2448c00640 1 -- 172.21.15.161:0/3215500655 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440b7800 con 0x7f244c15b600 2024-05-22T21:25:25.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.878+0000 7f2453400640 1 -- 172.21.15.161:0/3215500655 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244c1548b0 con 0x7f244c15b600 2024-05-22T21:25:25.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.882+0000 7f2453400640 1 -- 172.21.15.161:0/3215500655 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244c154dd0 con 0x7f244c15b600 2024-05-22T21:25:25.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.882+0000 7f2448c00640 1 -- 172.21.15.161:0/3215500655 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f24440b4050 con 0x7f244c15b600 2024-05-22T21:25:25.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.882+0000 7f2448c00640 1 --2- 172.21.15.161:0/3215500655 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f242c066690 0x7f242c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:25.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.882+0000 7f2448c00640 1 -- 172.21.15.161:0/3215500655 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f244411f160 con 0x7f244c15b600 2024-05-22T21:25:25.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.882+0000 7f2450a00640 1 --2- 172.21.15.161:0/3215500655 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f242c066690 0x7f242c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:25.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.882+0000 7f2450a00640 1 --2- 172.21.15.161:0/3215500655 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f242c066690 0x7f242c068b50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f243c007fd0 tx=0x7f243c007d60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:25.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:25.886+0000 7f2448c00640 1 -- 172.21.15.161:0/3215500655 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f244c154dd0 con 0x7f244c15b600 2024-05-22T21:25:26.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.038+0000 7f2453400640 1 -- 172.21.15.161:0/3215500655 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f244c15ec40 con 0x7f242c066690 2024-05-22T21:25:26.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.038+0000 7f2448c00640 1 -- 172.21.15.161:0/3215500655 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f243c004c80 con 0x7f242c066690 2024-05-22T21:25:26.041 INFO:teuthology.orchestra.run.smithi161.stderr:instructed osd(s) 3 to deep-scrub 2024-05-22T21:25:26.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.042+0000 7f2453400640 1 -- 172.21.15.161:0/3215500655 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f242c066690 msgr2=0x7f242c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.042+0000 7f2453400640 1 --2- 172.21.15.161:0/3215500655 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f242c066690 0x7f242c068b50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f243c007fd0 tx=0x7f243c007d60 comp rx=0 tx=0).stop 2024-05-22T21:25:26.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.042+0000 7f2453400640 1 -- 172.21.15.161:0/3215500655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f244c15b600 msgr2=0x7f244c135d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.042+0000 7f2453400640 1 --2- 172.21.15.161:0/3215500655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f244c15b600 0x7f244c135d80 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f24440b0040 tx=0x7f24440196d0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.042+0000 7f2453400640 1 -- 172.21.15.161:0/3215500655 shutdown_connections 2024-05-22T21:25:26.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.042+0000 7f2453400640 1 --2- 172.21.15.161:0/3215500655 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f242c066690 0x7f242c068b50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.043 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.042+0000 7f2453400640 1 --2- 172.21.15.161:0/3215500655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f244c15b600 0x7f244c135d80 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.042+0000 7f2453400640 1 --2- 172.21.15.161:0/3215500655 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f244c143c40 0x7f244c151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.042+0000 7f2453400640 1 --2- 172.21.15.161:0/3215500655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f244c06c9b0 0x7f244c1550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.042+0000 7f2453400640 1 -- 172.21.15.161:0/3215500655 >> 172.21.15.161:0/3215500655 conn(0x7f244c06a2d0 msgr2=0x7f244c142990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:26.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.042+0000 7f2453400640 1 -- 172.21.15.161:0/3215500655 shutdown_connections 2024-05-22T21:25:26.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.042+0000 7f2453400640 1 -- 172.21.15.161:0/3215500655 wait complete. 2024-05-22T21:25:26.061 INFO:tasks.ceph:Scrubbing osd.4 2024-05-22T21:25:26.061 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-22T21:25:26.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.198+0000 7f4321000640 1 Processor -- start 2024-05-22T21:25:26.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.198+0000 7f4321000640 1 -- start start 2024-05-22T21:25:26.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.202+0000 7f4321000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f431c06d560 0x7f431c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.202+0000 7f4321000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f431c0cca40 0x7f431c15c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.202+0000 7f4321000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431c0cce90 con 0x7f431c144980 2024-05-22T21:25:26.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.202+0000 7f4321000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f431c06d0d0 con 0x7f431c06d560 2024-05-22T21:25:26.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.202+0000 7f4321000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f431c06c4c0 con 0x7f431c0cca40 2024-05-22T21:25:26.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.202+0000 7f4319e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f431c144980 0x7f431c15c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60562/0 (socket says 172.21.15.161:60562) 2024-05-22T21:25:26.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.202+0000 7f4319e00640 1 -- 172.21.15.161:0/947852631 learned_addr learned my addr 172.21.15.161:0/947852631 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:26.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.202+0000 7f4319400640 1 --2- 172.21.15.161:0/947852631 >> v2:172.21.15.161:3300/0 conn(0x7f431c06d560 0x7f431c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.202+0000 7f431a800640 1 --2- 172.21.15.161:0/947852631 >> v2:172.21.15.179:3300/0 conn(0x7f431c0cca40 0x7f431c15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.202+0000 7f4319400640 1 -- 172.21.15.161:0/947852631 >> v2:172.21.15.179:3300/0 conn(0x7f431c0cca40 msgr2=0x7f431c15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:26.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.202+0000 7f4319400640 1 --2- 172.21.15.161:0/947852631 >> v2:172.21.15.179:3300/0 conn(0x7f431c0cca40 0x7f431c15c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.202+0000 7f4319400640 1 -- 172.21.15.161:0/947852631 >> v1:172.21.15.161:6789/0 conn(0x7f431c144980 legacy=0x7f431c15c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.202+0000 7f4319400640 1 -- 172.21.15.161:0/947852631 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c06c7a0 con 0x7f431c06d560 2024-05-22T21:25:26.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.202+0000 7f4318a00640 1 -- 172.21.15.161:0/947852631 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413067939 0 0) 0x7f430c0014a0 con 0x7f431c144980 2024-05-22T21:25:26.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.202+0000 7f431a800640 1 --2- 172.21.15.161:0/947852631 >> v2:172.21.15.179:3300/0 conn(0x7f431c0cca40 0x7f431c15c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:26.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.202+0000 7f4319400640 1 --2- 172.21.15.161:0/947852631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431c06d560 0x7f431c0cc500 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f431c0ae9f0 tx=0x7f430801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ab498094225653 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:26.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4318a00640 1 -- 172.21.15.161:0/947852631 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43080ab600 con 0x7f431c06d560 2024-05-22T21:25:26.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4318a00640 1 -- 172.21.15.161:0/947852631 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43080a7050 con 0x7f431c06d560 2024-05-22T21:25:26.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4318a00640 1 -- 172.21.15.161:0/947852631 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43080ab980 con 0x7f431c06d560 2024-05-22T21:25:26.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4321000640 1 -- 172.21.15.161:0/947852631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431c06d560 msgr2=0x7f431c0cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4321000640 1 --2- 172.21.15.161:0/947852631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431c06d560 0x7f431c0cc500 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f431c0ae9f0 tx=0x7f430801c480 comp rx=0 tx=0).stop 2024-05-22T21:25:26.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4321000640 1 -- 172.21.15.161:0/947852631 shutdown_connections 2024-05-22T21:25:26.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4321000640 1 --2- 172.21.15.161:0/947852631 >> v2:172.21.15.179:3300/0 conn(0x7f431c0cca40 0x7f431c15c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4321000640 1 --2- 172.21.15.161:0/947852631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431c06d560 0x7f431c0cc500 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4321000640 1 -- 172.21.15.161:0/947852631 >> 172.21.15.161:0/947852631 conn(0x7f431c06a0a0 msgr2=0x7f431c06a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:26.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4321000640 1 -- 172.21.15.161:0/947852631 shutdown_connections 2024-05-22T21:25:26.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4321000640 1 -- 172.21.15.161:0/947852631 wait complete. 2024-05-22T21:25:26.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4321000640 1 Processor -- start 2024-05-22T21:25:26.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4321000640 1 -- start start 2024-05-22T21:25:26.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4321000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431c06d560 0x7f431c154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4321000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f431c0cca40 0x7f431c153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4321000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f431c144980 0x7f431c15b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4321000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c154950 con 0x7f431c06d560 2024-05-22T21:25:26.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4321000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c153f90 con 0x7f431c144980 2024-05-22T21:25:26.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4321000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c15be30 con 0x7f431c0cca40 2024-05-22T21:25:26.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4319e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431c06d560 0x7f431c154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4319e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431c06d560 0x7f431c154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42082/0 (socket says 172.21.15.161:42082) 2024-05-22T21:25:26.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4319e00640 1 -- 172.21.15.161:0/2328112193 learned_addr learned my addr 172.21.15.161:0/2328112193 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:26.209 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4319400640 1 --2- 172.21.15.161:0/2328112193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f431c0cca40 0x7f431c153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f431a800640 1 --2- 172.21.15.161:0/2328112193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f431c144980 0x7f431c15b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4319e00640 1 -- 172.21.15.161:0/2328112193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f431c0cca40 msgr2=0x7f431c153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4319e00640 1 --2- 172.21.15.161:0/2328112193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f431c0cca40 0x7f431c153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4319e00640 1 -- 172.21.15.161:0/2328112193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f431c144980 msgr2=0x7f431c15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.206+0000 7f4319e00640 1 --2- 172.21.15.161:0/2328112193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f431c144980 0x7f431c15b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f4319e00640 1 -- 172.21.15.161:0/2328112193 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4308009420 con 0x7f431c06d560 2024-05-22T21:25:26.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f431a800640 1 --2- 172.21.15.161:0/2328112193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f431c144980 0x7f431c15b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:26.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f4319e00640 1 --2- 172.21.15.161:0/2328112193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431c06d560 0x7f431c154d70 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f430c00c7c0 tx=0x7f430c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:26.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f4312000640 1 -- 172.21.15.161:0/2328112193 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c00a640 con 0x7f431c06d560 2024-05-22T21:25:26.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f4312000640 1 -- 172.21.15.161:0/2328112193 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c00a7c0 con 0x7f431c06d560 2024-05-22T21:25:26.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f4321000640 1 -- 172.21.15.161:0/2328112193 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431c15c110 con 0x7f431c06d560 2024-05-22T21:25:26.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f4312000640 1 -- 172.21.15.161:0/2328112193 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c00ab70 con 0x7f431c06d560 2024-05-22T21:25:26.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f4321000640 1 -- 172.21.15.161:0/2328112193 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c154570 con 0x7f431c06d560 2024-05-22T21:25:26.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f4321000640 1 -- 172.21.15.161:0/2328112193 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f431c06da10 con 0x7f431c06d560 2024-05-22T21:25:26.212 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f4312000640 1 -- 172.21.15.161:0/2328112193 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f430c00acd0 con 0x7f431c06d560 2024-05-22T21:25:26.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f4312000640 1 --2- 172.21.15.161:0/2328112193 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f42fc066690 0x7f42fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f4312000640 1 -- 172.21.15.161:0/2328112193 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f430c0838f0 con 0x7f431c06d560 2024-05-22T21:25:26.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f4319400640 1 --2- 172.21.15.161:0/2328112193 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f42fc066690 0x7f42fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f4312000640 1 --2- 172.21.15.161:0/2328112193 >> [v2:172.21.15.179:6800/23606,v1:172.21.15.179:6801/23606] conn(0x7f42fc06c4f0 0x7f42fc06e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f4312000640 1 -- 172.21.15.161:0/2328112193 --> [v2:172.21.15.179:6800/23606,v1:172.21.15.179:6801/23606] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42fc06efc0 con 0x7f42fc06c4f0 2024-05-22T21:25:26.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f4312000640 1 -- 172.21.15.161:0/2328112193 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=197) v2 ==== 24+0+0 (secure 0 0 0) 0x7f430c083c60 con 0x7f431c06d560 2024-05-22T21:25:26.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.210+0000 7f431a800640 1 --2- 172.21.15.161:0/2328112193 >> [v2:172.21.15.179:6800/23606,v1:172.21.15.179:6801/23606] conn(0x7f42fc06c4f0 0x7f42fc06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.214+0000 7f4319400640 1 --2- 172.21.15.161:0/2328112193 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f42fc066690 0x7f42fc068b50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f43080ad000 tx=0x7f4308008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:26.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.214+0000 7f431a800640 1 --2- 172.21.15.161:0/2328112193 >> [v2:172.21.15.179:6800/23606,v1:172.21.15.179:6801/23606] conn(0x7f42fc06c4f0 0x7f42fc06e910 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:26.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.214+0000 7f4312000640 1 -- 172.21.15.161:0/2328112193 <== osd.4 v2:172.21.15.179:6800/23606 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24459 (crc 0 0 0) 0x7f42fc06efc0 con 0x7f42fc06c4f0 2024-05-22T21:25:26.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.226+0000 7f4321000640 1 -- 172.21.15.161:0/2328112193 --> [v2:172.21.15.179:6800/23606,v1:172.21.15.179:6801/23606] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f431c154a90 con 0x7f42fc06c4f0 2024-05-22T21:25:26.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.226+0000 7f4312000640 1 -- 172.21.15.161:0/2328112193 <== osd.4 v2:172.21.15.179:6800/23606 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f431c154a90 con 0x7f42fc06c4f0 2024-05-22T21:25:26.226 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:25:26.227 INFO:teuthology.orchestra.run.smithi161.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-05-22T21:25:26.227 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:25:26.227 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.226+0000 7f4321000640 1 -- 172.21.15.161:0/2328112193 >> [v2:172.21.15.179:6800/23606,v1:172.21.15.179:6801/23606] conn(0x7f42fc06c4f0 msgr2=0x7f42fc06e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.226+0000 7f4321000640 1 --2- 172.21.15.161:0/2328112193 >> [v2:172.21.15.179:6800/23606,v1:172.21.15.179:6801/23606] conn(0x7f42fc06c4f0 0x7f42fc06e910 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.226+0000 7f4321000640 1 -- 172.21.15.161:0/2328112193 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f42fc066690 msgr2=0x7f42fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.226+0000 7f4321000640 1 --2- 172.21.15.161:0/2328112193 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f42fc066690 0x7f42fc068b50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f43080ad000 tx=0x7f4308008040 comp rx=0 tx=0).stop 2024-05-22T21:25:26.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.226+0000 7f4321000640 1 -- 172.21.15.161:0/2328112193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431c06d560 msgr2=0x7f431c154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.226+0000 7f4321000640 1 --2- 172.21.15.161:0/2328112193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431c06d560 0x7f431c154d70 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f430c00c7c0 tx=0x7f430c00cc80 comp rx=0 tx=0).stop 2024-05-22T21:25:26.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.226+0000 7f431a800640 1 -- 172.21.15.161:0/2328112193 reap_dead start 2024-05-22T21:25:26.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.226+0000 7f4321000640 1 -- 172.21.15.161:0/2328112193 shutdown_connections 2024-05-22T21:25:26.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.226+0000 7f4321000640 1 -- 172.21.15.161:0/2328112193 >> 172.21.15.161:0/2328112193 conn(0x7f431c06a0a0 msgr2=0x7f431c143940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:26.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.226+0000 7f4321000640 1 -- 172.21.15.161:0/2328112193 shutdown_connections 2024-05-22T21:25:26.228 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.226+0000 7f4321000640 1 -- 172.21.15.161:0/2328112193 wait complete. 2024-05-22T21:25:26.245 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-22T21:25:26.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.382+0000 7fa120400640 1 Processor -- start 2024-05-22T21:25:26.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.382+0000 7fa120400640 1 -- start start 2024-05-22T21:25:26.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.382+0000 7fa120400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa11813fe50 0x7fa118164e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.382+0000 7fa120400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11813d3a0 con 0x7fa11813cfc0 2024-05-22T21:25:26.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.382+0000 7fa120400640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1180cd920 con 0x7fa1181402e0 2024-05-22T21:25:26.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.382+0000 7fa120400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1180cdc00 con 0x7fa11813fe50 2024-05-22T21:25:26.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.382+0000 7fa11da00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa11813cfc0 0x7fa11815af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60572/0 (socket says 172.21.15.161:60572) 2024-05-22T21:25:26.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.382+0000 7fa11da00640 1 -- 172.21.15.161:0/3769815078 learned_addr learned my addr 172.21.15.161:0/3769815078 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:26.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.382+0000 7fa11e400640 1 --2- 172.21.15.161:0/3769815078 >> v2:172.21.15.161:3300/0 conn(0x7fa11813fe50 0x7fa118164e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.382+0000 7fa11e400640 1 -- 172.21.15.161:0/3769815078 >> v1:172.21.15.179:6789/0 conn(0x7fa1181402e0 legacy=0x7fa11813f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:26.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.382+0000 7fa11e400640 1 -- 172.21.15.161:0/3769815078 >> v1:172.21.15.161:6789/0 conn(0x7fa11813cfc0 legacy=0x7fa11815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.382+0000 7fa11e400640 1 -- 172.21.15.161:0/3769815078 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1180cdee0 con 0x7fa11813fe50 2024-05-22T21:25:26.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.386+0000 7fa11e400640 1 --2- 172.21.15.161:0/3769815078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa11813fe50 0x7fa118164e60 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fa110009d90 tx=0x7fa11001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d3ba5db194c1e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:26.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.386+0000 7fa117e00640 1 -- 172.21.15.161:0/3769815078 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1100a5850 con 0x7fa11813fe50 2024-05-22T21:25:26.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.386+0000 7fa117e00640 1 -- 172.21.15.161:0/3769815078 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1100a59b0 con 0x7fa11813fe50 2024-05-22T21:25:26.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.386+0000 7fa117e00640 1 -- 172.21.15.161:0/3769815078 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1100a5d40 con 0x7fa11813fe50 2024-05-22T21:25:26.388 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.386+0000 7fa120400640 1 -- 172.21.15.161:0/3769815078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa11813fe50 msgr2=0x7fa118164e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.386+0000 7fa120400640 1 --2- 172.21.15.161:0/3769815078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa11813fe50 0x7fa118164e60 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fa110009d90 tx=0x7fa11001c8a0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.386+0000 7fa120400640 1 -- 172.21.15.161:0/3769815078 shutdown_connections 2024-05-22T21:25:26.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.386+0000 7fa120400640 1 --2- 172.21.15.161:0/3769815078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa11813fe50 0x7fa118164e60 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.386+0000 7fa120400640 1 -- 172.21.15.161:0/3769815078 >> 172.21.15.161:0/3769815078 conn(0x7fa11806a2d0 msgr2=0x7fa11806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:26.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.386+0000 7fa120400640 1 -- 172.21.15.161:0/3769815078 shutdown_connections 2024-05-22T21:25:26.389 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.386+0000 7fa120400640 1 -- 172.21.15.161:0/3769815078 wait complete. 2024-05-22T21:25:26.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa120400640 1 Processor -- start 2024-05-22T21:25:26.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa120400640 1 -- start start 2024-05-22T21:25:26.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa120400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa11813cfc0 0x7fa11813c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa120400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa11813fe50 0x7fa11813ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa120400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1181402e0 0x7fa11813ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa120400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11813dfc0 con 0x7fa11813cfc0 2024-05-22T21:25:26.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa120400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11813e580 con 0x7fa11813fe50 2024-05-22T21:25:26.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa120400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11813c8c0 con 0x7fa1181402e0 2024-05-22T21:25:26.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa11da00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa11813cfc0 0x7fa11813c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa11da00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa11813cfc0 0x7fa11813c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42102/0 (socket says 172.21.15.161:42102) 2024-05-22T21:25:26.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa11da00640 1 -- 172.21.15.161:0/3447705606 learned_addr learned my addr 172.21.15.161:0/3447705606 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:26.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa11e400640 1 --2- 172.21.15.161:0/3447705606 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1181402e0 0x7fa11813ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa11d000640 1 --2- 172.21.15.161:0/3447705606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa11813fe50 0x7fa11813ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa11da00640 1 -- 172.21.15.161:0/3447705606 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1181402e0 msgr2=0x7fa11813ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa11da00640 1 --2- 172.21.15.161:0/3447705606 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1181402e0 0x7fa11813ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa11da00640 1 -- 172.21.15.161:0/3447705606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa11813fe50 msgr2=0x7fa11813ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa11da00640 1 --2- 172.21.15.161:0/3447705606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa11813fe50 0x7fa11813ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa11da00640 1 -- 172.21.15.161:0/3447705606 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa110009a70 con 0x7fa11813cfc0 2024-05-22T21:25:26.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa11e400640 1 --2- 172.21.15.161:0/3447705606 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1181402e0 0x7fa11813ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:25:26.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa11d000640 1 --2- 172.21.15.161:0/3447705606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa11813fe50 0x7fa11813ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:25:26.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa11da00640 1 --2- 172.21.15.161:0/3447705606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa11813cfc0 0x7fa11813c050 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00c400 tx=0x7fa10c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:26.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa115600640 1 -- 172.21.15.161:0/3447705606 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c005cf0 con 0x7fa11813cfc0 2024-05-22T21:25:26.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa115600640 1 -- 172.21.15.161:0/3447705606 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c005e50 con 0x7fa11813cfc0 2024-05-22T21:25:26.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa115600640 1 -- 172.21.15.161:0/3447705606 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0072d0 con 0x7fa11813cfc0 2024-05-22T21:25:26.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa120400640 1 -- 172.21.15.161:0/3447705606 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11813f610 con 0x7fa11813cfc0 2024-05-22T21:25:26.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.390+0000 7fa120400640 1 -- 172.21.15.161:0/3447705606 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11813e210 con 0x7fa11813cfc0 2024-05-22T21:25:26.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.394+0000 7fa120400640 1 -- 172.21.15.161:0/3447705606 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11813fac0 con 0x7fa11813cfc0 2024-05-22T21:25:26.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.394+0000 7fa115600640 1 -- 172.21.15.161:0/3447705606 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa10c007430 con 0x7fa11813cfc0 2024-05-22T21:25:26.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.394+0000 7fa115600640 1 --2- 172.21.15.161:0/3447705606 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0f8066690 0x7fa0f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.394+0000 7fa115600640 1 -- 172.21.15.161:0/3447705606 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa10c083340 con 0x7fa11813cfc0 2024-05-22T21:25:26.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.394+0000 7fa11d000640 1 --2- 172.21.15.161:0/3447705606 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0f8066690 0x7fa0f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.394+0000 7fa11d000640 1 --2- 172.21.15.161:0/3447705606 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0f8066690 0x7fa0f8068b50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa1180cc770 tx=0x7fa104009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:26.398 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.398+0000 7fa115600640 1 -- 172.21.15.161:0/3447705606 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fa10c050a50 con 0x7fa11813cfc0 2024-05-22T21:25:26.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.566+0000 7fa120400640 1 -- 172.21.15.161:0/3447705606 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa118159c20 con 0x7fa0f8066690 2024-05-22T21:25:26.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.566+0000 7fa115600640 1 -- 172.21.15.161:0/3447705606 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa118159c20 con 0x7fa0f8066690 2024-05-22T21:25:26.568 INFO:teuthology.orchestra.run.smithi161.stderr:instructed osd(s) 4 to deep-scrub 2024-05-22T21:25:26.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.570+0000 7fa120400640 1 -- 172.21.15.161:0/3447705606 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0f8066690 msgr2=0x7fa0f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.570+0000 7fa120400640 1 --2- 172.21.15.161:0/3447705606 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0f8066690 0x7fa0f8068b50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa1180cc770 tx=0x7fa104009040 comp rx=0 tx=0).stop 2024-05-22T21:25:26.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.570+0000 7fa120400640 1 -- 172.21.15.161:0/3447705606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa11813cfc0 msgr2=0x7fa11813c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.570+0000 7fa120400640 1 --2- 172.21.15.161:0/3447705606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa11813cfc0 0x7fa11813c050 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00c400 tx=0x7fa10c00c8c0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.570+0000 7fa120400640 1 -- 172.21.15.161:0/3447705606 shutdown_connections 2024-05-22T21:25:26.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.570+0000 7fa120400640 1 --2- 172.21.15.161:0/3447705606 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa0f8066690 0x7fa0f8068b50 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.570+0000 7fa120400640 1 --2- 172.21.15.161:0/3447705606 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1181402e0 0x7fa11813ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.570+0000 7fa120400640 1 --2- 172.21.15.161:0/3447705606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa11813fe50 0x7fa11813ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.570+0000 7fa120400640 1 --2- 172.21.15.161:0/3447705606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa11813cfc0 0x7fa11813c050 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.570+0000 7fa120400640 1 -- 172.21.15.161:0/3447705606 >> 172.21.15.161:0/3447705606 conn(0x7fa11806a2d0 msgr2=0x7fa11806a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:26.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.570+0000 7fa120400640 1 -- 172.21.15.161:0/3447705606 shutdown_connections 2024-05-22T21:25:26.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.570+0000 7fa120400640 1 -- 172.21.15.161:0/3447705606 wait complete. 2024-05-22T21:25:26.589 INFO:tasks.ceph:Scrubbing osd.5 2024-05-22T21:25:26.589 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-22T21:25:26.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.726+0000 7f1e90c00640 1 Processor -- start 2024-05-22T21:25:26.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.726+0000 7f1e90c00640 1 -- start start 2024-05-22T21:25:26.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e90c00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1e8c0cda00 0x7f1e8c0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e90c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8c0d1bd0 con 0x7f1e8c0b0420 2024-05-22T21:25:26.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e90c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8c06dff0 con 0x7f1e8c0cdde0 2024-05-22T21:25:26.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e90c00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e8c161e40 con 0x7f1e8c0cda00 2024-05-22T21:25:26.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e89400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1e8c0b0420 0x7f1e8c06edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60576/0 (socket says 172.21.15.161:60576) 2024-05-22T21:25:26.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e89400640 1 -- 172.21.15.161:0/3913420937 learned_addr learned my addr 172.21.15.161:0/3913420937 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:26.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e89e00640 1 --2- 172.21.15.161:0/3913420937 >> v2:172.21.15.179:3300/0 conn(0x7f1e8c0cda00 0x7f1e8c0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e88a00640 1 -- 172.21.15.161:0/3913420937 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351718133 0 0) 0x7f1e8c0d1bd0 con 0x7f1e8c0b0420 2024-05-22T21:25:26.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e88a00640 1 -- 172.21.15.161:0/3913420937 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e70003530 con 0x7f1e8c0b0420 2024-05-22T21:25:26.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e88a00640 1 -- 172.21.15.161:0/3913420937 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 508983525 0 0) 0x7f1e70003530 con 0x7f1e8c0b0420 2024-05-22T21:25:26.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e88a00640 1 -- 172.21.15.161:0/3913420937 >> v2:172.21.15.179:3300/0 conn(0x7f1e8c0cda00 msgr2=0x7f1e8c0cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:26.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e88a00640 1 --2- 172.21.15.161:0/3913420937 >> v2:172.21.15.179:3300/0 conn(0x7f1e8c0cda00 0x7f1e8c0cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e88a00640 1 -- 172.21.15.161:0/3913420937 >> v1:172.21.15.179:6789/0 conn(0x7f1e8c0cdde0 legacy=0x7f1e8c161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e88a00640 1 -- 172.21.15.161:0/3913420937 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8c06e2d0 con 0x7f1e8c0b0420 2024-05-22T21:25:26.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e88a00640 1 -- 172.21.15.161:0/3913420937 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1647110332 0 0) 0x7f1e780016a0 con 0x7f1e8c0b0420 2024-05-22T21:25:26.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e88a00640 1 -- 172.21.15.161:0/3913420937 >> v1:172.21.15.161:6789/0 conn(0x7f1e8c0b0420 legacy=0x7f1e8c06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e88a00640 1 --2- 172.21.15.161:0/3913420937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e70003980 0x7f1e70003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e88a00640 1 --2- 172.21.15.161:0/3913420937 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e700047d0 0x7f1e70004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e88a00640 1 --2- 172.21.15.161:0/3913420937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e70005190 0x7f1e7000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e88a00640 1 -- 172.21.15.161:0/3913420937 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c161e40 con 0x7f1e70003980 2024-05-22T21:25:26.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e88a00640 1 -- 172.21.15.161:0/3913420937 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7000c130 con 0x7f1e70005190 2024-05-22T21:25:26.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e88a00640 1 -- 172.21.15.161:0/3913420937 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7000c2f0 con 0x7f1e700047d0 2024-05-22T21:25:26.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e89e00640 1 --2- 172.21.15.161:0/3913420937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e70003980 0x7f1e70003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e8a800640 1 --2- 172.21.15.161:0/3913420937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e70005190 0x7f1e7000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e89400640 1 --2- 172.21.15.161:0/3913420937 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e700047d0 0x7f1e70004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e89e00640 1 -- 172.21.15.161:0/3913420937 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e700047d0 msgr2=0x7f1e70004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:26.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e89e00640 1 --2- 172.21.15.161:0/3913420937 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e700047d0 0x7f1e70004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e89e00640 1 -- 172.21.15.161:0/3913420937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e70005190 msgr2=0x7f1e7000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:26.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e89e00640 1 --2- 172.21.15.161:0/3913420937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e70005190 0x7f1e7000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e89e00640 1 -- 172.21.15.161:0/3913420937 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e70003530 con 0x7f1e70003980 2024-05-22T21:25:26.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e8a800640 1 --2- 172.21.15.161:0/3913420937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e70005190 0x7f1e7000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:26.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.730+0000 7f1e8a800640 1 -- 172.21.15.161:0/3913420937 reap_dead start 2024-05-22T21:25:26.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.734+0000 7f1e89e00640 1 --2- 172.21.15.161:0/3913420937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e70003980 0x7f1e70003da0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009ae0 tx=0x7f1e7c01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe560fa30ee506c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:26.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e88a00640 1 -- 172.21.15.161:0/3913420937 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0a5520 con 0x7f1e70003980 2024-05-22T21:25:26.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e88a00640 1 -- 172.21.15.161:0/3913420937 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7c0a5680 con 0x7f1e70003980 2024-05-22T21:25:26.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e88a00640 1 -- 172.21.15.161:0/3913420937 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0a5a30 con 0x7f1e70003980 2024-05-22T21:25:26.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e90c00640 1 -- 172.21.15.161:0/3913420937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e70003980 msgr2=0x7f1e70003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e90c00640 1 --2- 172.21.15.161:0/3913420937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e70003980 0x7f1e70003da0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009ae0 tx=0x7f1e7c01ca40 comp rx=0 tx=0).stop 2024-05-22T21:25:26.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e90c00640 1 -- 172.21.15.161:0/3913420937 shutdown_connections 2024-05-22T21:25:26.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e90c00640 1 --2- 172.21.15.161:0/3913420937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e70003980 0x7f1e70003da0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e90c00640 1 -- 172.21.15.161:0/3913420937 >> 172.21.15.161:0/3913420937 conn(0x7f1e8c0d0f20 msgr2=0x7f1e8c0d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:26.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e90c00640 1 -- 172.21.15.161:0/3913420937 shutdown_connections 2024-05-22T21:25:26.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e90c00640 1 -- 172.21.15.161:0/3913420937 wait complete. 2024-05-22T21:25:26.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e90c00640 1 Processor -- start 2024-05-22T21:25:26.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e90c00640 1 -- start start 2024-05-22T21:25:26.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e90c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e70003980 0x7f1e8c15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e90c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e8c15ae50 0x7f1e8c0b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e90c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e8c15a890 0x7f1e8c0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e90c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c0b0ed0 con 0x7f1e8c15a890 2024-05-22T21:25:26.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e90c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c0b42b0 con 0x7f1e8c15ae50 2024-05-22T21:25:26.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e90c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c0c75c0 con 0x7f1e70003980 2024-05-22T21:25:26.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e8a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e8c15a890 0x7f1e8c0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.738+0000 7f1e8a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e8c15a890 0x7f1e8c0b61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42108/0 (socket says 172.21.15.161:42108) 2024-05-22T21:25:26.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.742+0000 7f1e8a800640 1 -- 172.21.15.161:0/1401434940 learned_addr learned my addr 172.21.15.161:0/1401434940 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:26.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.742+0000 7f1e89e00640 1 --2- 172.21.15.161:0/1401434940 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e70003980 0x7f1e8c15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.742+0000 7f1e89400640 1 --2- 172.21.15.161:0/1401434940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e8c15ae50 0x7f1e8c0b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.742+0000 7f1e8a800640 1 -- 172.21.15.161:0/1401434940 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e70003980 msgr2=0x7f1e8c15a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.742+0000 7f1e8a800640 1 --2- 172.21.15.161:0/1401434940 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1e70003980 0x7f1e8c15a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.742+0000 7f1e8a800640 1 -- 172.21.15.161:0/1401434940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e8c15ae50 msgr2=0x7f1e8c0b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.742+0000 7f1e8a800640 1 --2- 172.21.15.161:0/1401434940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e8c15ae50 0x7f1e8c0b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.742+0000 7f1e8a800640 1 -- 172.21.15.161:0/1401434940 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8c0c78a0 con 0x7f1e8c15a890 2024-05-22T21:25:26.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.742+0000 7f1e8a800640 1 --2- 172.21.15.161:0/1401434940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e8c15a890 0x7f1e8c0b61f0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0ce1c0 tx=0x7f1e8400e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:26.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.742+0000 7f1e82000640 1 -- 172.21.15.161:0/1401434940 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84003c50 con 0x7f1e8c15a890 2024-05-22T21:25:26.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.742+0000 7f1e82000640 1 -- 172.21.15.161:0/1401434940 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e84003dd0 con 0x7f1e8c15a890 2024-05-22T21:25:26.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.742+0000 7f1e82000640 1 -- 172.21.15.161:0/1401434940 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84005310 con 0x7f1e8c15a890 2024-05-22T21:25:26.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.742+0000 7f1e90c00640 1 -- 172.21.15.161:0/1401434940 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8c0b0880 con 0x7f1e8c15a890 2024-05-22T21:25:26.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.742+0000 7f1e90c00640 1 -- 172.21.15.161:0/1401434940 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8c0c71e0 con 0x7f1e8c15a890 2024-05-22T21:25:26.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.742+0000 7f1e90c00640 1 -- 172.21.15.161:0/1401434940 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e8c0d0460 con 0x7f1e8c15a890 2024-05-22T21:25:26.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.742+0000 7f1e82000640 1 -- 172.21.15.161:0/1401434940 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1e84005470 con 0x7f1e8c15a890 2024-05-22T21:25:26.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.746+0000 7f1e82000640 1 --2- 172.21.15.161:0/1401434940 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e6c066690 0x7f1e6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.746+0000 7f1e82000640 1 -- 172.21.15.161:0/1401434940 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1e84083e50 con 0x7f1e8c15a890 2024-05-22T21:25:26.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.746+0000 7f1e89e00640 1 --2- 172.21.15.161:0/1401434940 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e6c066690 0x7f1e6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.746+0000 7f1e82000640 1 --2- 172.21.15.161:0/1401434940 >> [v2:172.21.15.179:6808/23607,v1:172.21.15.179:6809/23607] conn(0x7f1e6c06c4f0 0x7f1e6c06e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.746+0000 7f1e82000640 1 -- 172.21.15.161:0/1401434940 --> [v2:172.21.15.179:6808/23607,v1:172.21.15.179:6809/23607] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e6c06efc0 con 0x7f1e6c06c4f0 2024-05-22T21:25:26.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.746+0000 7f1e82000640 1 -- 172.21.15.161:0/1401434940 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=197) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e840841c0 con 0x7f1e8c15a890 2024-05-22T21:25:26.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.746+0000 7f1e89400640 1 --2- 172.21.15.161:0/1401434940 >> [v2:172.21.15.179:6808/23607,v1:172.21.15.179:6809/23607] conn(0x7f1e6c06c4f0 0x7f1e6c06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.746+0000 7f1e89e00640 1 --2- 172.21.15.161:0/1401434940 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e6c066690 0x7f1e6c068b50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0af000 tx=0x7f1e7c0046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:26.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.746+0000 7f1e89400640 1 --2- 172.21.15.161:0/1401434940 >> [v2:172.21.15.179:6808/23607,v1:172.21.15.179:6809/23607] conn(0x7f1e6c06c4f0 0x7f1e6c06e910 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:26.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.746+0000 7f1e82000640 1 -- 172.21.15.161:0/1401434940 <== osd.5 v2:172.21.15.179:6808/23607 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24459 (crc 0 0 0) 0x7f1e6c06efc0 con 0x7f1e6c06c4f0 2024-05-22T21:25:26.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.758+0000 7f1e90c00640 1 -- 172.21.15.161:0/1401434940 --> [v2:172.21.15.179:6808/23607,v1:172.21.15.179:6809/23607] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1e8c0b0d30 con 0x7f1e6c06c4f0 2024-05-22T21:25:26.759 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.758+0000 7f1e82000640 1 -- 172.21.15.161:0/1401434940 <== osd.5 v2:172.21.15.179:6808/23607 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f1e78013330 con 0x7f1e6c06c4f0 2024-05-22T21:25:26.759 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:25:26.760 INFO:teuthology.orchestra.run.smithi161.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-05-22T21:25:26.760 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:25:26.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.758+0000 7f1e90c00640 1 -- 172.21.15.161:0/1401434940 >> [v2:172.21.15.179:6808/23607,v1:172.21.15.179:6809/23607] conn(0x7f1e6c06c4f0 msgr2=0x7f1e6c06e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.758+0000 7f1e90c00640 1 --2- 172.21.15.161:0/1401434940 >> [v2:172.21.15.179:6808/23607,v1:172.21.15.179:6809/23607] conn(0x7f1e6c06c4f0 0x7f1e6c06e910 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.758+0000 7f1e90c00640 1 -- 172.21.15.161:0/1401434940 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e6c066690 msgr2=0x7f1e6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.758+0000 7f1e90c00640 1 --2- 172.21.15.161:0/1401434940 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f1e6c066690 0x7f1e6c068b50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0af000 tx=0x7f1e7c0046d0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.758+0000 7f1e90c00640 1 -- 172.21.15.161:0/1401434940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e8c15a890 msgr2=0x7f1e8c0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.758+0000 7f1e90c00640 1 --2- 172.21.15.161:0/1401434940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e8c15a890 0x7f1e8c0b61f0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0ce1c0 tx=0x7f1e8400e980 comp rx=0 tx=0).stop 2024-05-22T21:25:26.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.758+0000 7f1e8a800640 1 -- 172.21.15.161:0/1401434940 reap_dead start 2024-05-22T21:25:26.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.758+0000 7f1e90c00640 1 -- 172.21.15.161:0/1401434940 shutdown_connections 2024-05-22T21:25:26.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.758+0000 7f1e90c00640 1 -- 172.21.15.161:0/1401434940 >> 172.21.15.161:0/1401434940 conn(0x7f1e8c0d0f20 msgr2=0x7f1e8c1616b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:26.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.758+0000 7f1e90c00640 1 -- 172.21.15.161:0/1401434940 shutdown_connections 2024-05-22T21:25:26.761 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.758+0000 7f1e90c00640 1 -- 172.21.15.161:0/1401434940 wait complete. 2024-05-22T21:25:26.778 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-22T21:25:26.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.914+0000 7ff9d2800640 1 Processor -- start 2024-05-22T21:25:26.915 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.914+0000 7ff9d2800640 1 -- start start 2024-05-22T21:25:26.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.914+0000 7ff9d2800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9cc06c9b0 0x7ff9cc0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.914+0000 7ff9d2800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9cc144bb0 con 0x7ff9cc15b600 2024-05-22T21:25:26.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.914+0000 7ff9d2800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9cc15efb0 con 0x7ff9cc143c40 2024-05-22T21:25:26.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.914+0000 7ff9d2800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc15ba60 con 0x7ff9cc06c9b0 2024-05-22T21:25:26.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.918+0000 7ff9caa00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff9cc15b600 0x7ff9cc06c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60588/0 (socket says 172.21.15.161:60588) 2024-05-22T21:25:26.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.918+0000 7ff9caa00640 1 -- 172.21.15.161:0/690127984 learned_addr learned my addr 172.21.15.161:0/690127984 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:26.917 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.918+0000 7ff9cbe00640 1 --2- 172.21.15.161:0/690127984 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9cc06c9b0 0x7ff9cc0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.918+0000 7ff9ca000640 1 -- 172.21.15.161:0/690127984 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229425567 0 0) 0x7ff9cc144bb0 con 0x7ff9cc15b600 2024-05-22T21:25:26.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.918+0000 7ff9ca000640 1 -- 172.21.15.161:0/690127984 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9b0003530 con 0x7ff9cc15b600 2024-05-22T21:25:26.918 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.918+0000 7ff9ca000640 1 -- 172.21.15.161:0/690127984 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3046248761 0 0) 0x7ff9cc15efb0 con 0x7ff9cc143c40 2024-05-22T21:25:26.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.918+0000 7ff9ca000640 1 -- 172.21.15.161:0/690127984 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9cc144bb0 con 0x7ff9cc143c40 2024-05-22T21:25:26.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.918+0000 7ff9cbe00640 1 -- 172.21.15.161:0/690127984 >> v1:172.21.15.179:6789/0 conn(0x7ff9cc143c40 legacy=0x7ff9cc1444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.918+0000 7ff9cbe00640 1 -- 172.21.15.161:0/690127984 >> v1:172.21.15.161:6789/0 conn(0x7ff9cc15b600 legacy=0x7ff9cc06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.919 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.918+0000 7ff9cbe00640 1 -- 172.21.15.161:0/690127984 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc15f290 con 0x7ff9cc06c9b0 2024-05-22T21:25:26.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.918+0000 7ff9cbe00640 1 --2- 172.21.15.161:0/690127984 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9cc06c9b0 0x7ff9cc0b2160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009dc0 tx=0x7ff9c401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff654a9da87a43c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:26.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.918+0000 7ff9ca000640 1 -- 172.21.15.161:0/690127984 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40a5a50 con 0x7ff9cc06c9b0 2024-05-22T21:25:26.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9ca000640 1 -- 172.21.15.161:0/690127984 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c40a5bb0 con 0x7ff9cc06c9b0 2024-05-22T21:25:26.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9ca000640 1 -- 172.21.15.161:0/690127984 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40ad440 con 0x7ff9cc06c9b0 2024-05-22T21:25:26.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9d2800640 1 -- 172.21.15.161:0/690127984 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9cc06c9b0 msgr2=0x7ff9cc0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9d2800640 1 --2- 172.21.15.161:0/690127984 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9cc06c9b0 0x7ff9cc0b2160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009dc0 tx=0x7ff9c401c990 comp rx=0 tx=0).stop 2024-05-22T21:25:26.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9d2800640 1 -- 172.21.15.161:0/690127984 shutdown_connections 2024-05-22T21:25:26.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9d2800640 1 --2- 172.21.15.161:0/690127984 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9cc06c9b0 0x7ff9cc0b2160 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9d2800640 1 -- 172.21.15.161:0/690127984 >> 172.21.15.161:0/690127984 conn(0x7ff9cc06a2d0 msgr2=0x7ff9cc06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:26.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9d2800640 1 -- 172.21.15.161:0/690127984 shutdown_connections 2024-05-22T21:25:26.922 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9d2800640 1 -- 172.21.15.161:0/690127984 wait complete. 2024-05-22T21:25:26.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9d2800640 1 Processor -- start 2024-05-22T21:25:26.923 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9d2800640 1 -- start start 2024-05-22T21:25:26.925 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9d2800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc06c9b0 0x7ff9cc139530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9d2800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9cc143c40 0x7ff9cc137bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9d2800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9cc15b600 0x7ff9cc15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9d2800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc143410 con 0x7ff9cc15b600 2024-05-22T21:25:26.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9d2800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc06fce0 con 0x7ff9cc06c9b0 2024-05-22T21:25:26.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9d2800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc138110 con 0x7ff9cc143c40 2024-05-22T21:25:26.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9cbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9cc15b600 0x7ff9cc15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9cbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9cc15b600 0x7ff9cc15aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42110/0 (socket says 172.21.15.161:42110) 2024-05-22T21:25:26.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9cbe00640 1 -- 172.21.15.161:0/2566067076 learned_addr learned my addr 172.21.15.161:0/2566067076 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:26.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9caa00640 1 --2- 172.21.15.161:0/2566067076 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9cc143c40 0x7ff9cc137bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9cb400640 1 --2- 172.21.15.161:0/2566067076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc06c9b0 0x7ff9cc139530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.926 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9cbe00640 1 -- 172.21.15.161:0/2566067076 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9cc143c40 msgr2=0x7ff9cc137bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9cbe00640 1 --2- 172.21.15.161:0/2566067076 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9cc143c40 0x7ff9cc137bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9cbe00640 1 -- 172.21.15.161:0/2566067076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc06c9b0 msgr2=0x7ff9cc139530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:26.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9cbe00640 1 --2- 172.21.15.161:0/2566067076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc06c9b0 0x7ff9cc139530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:26.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9cbe00640 1 -- 172.21.15.161:0/2566067076 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c4009a70 con 0x7ff9cc15b600 2024-05-22T21:25:26.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.922+0000 7ff9cb400640 1 --2- 172.21.15.161:0/2566067076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc06c9b0 0x7ff9cc139530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:26.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.926+0000 7ff9cbe00640 1 --2- 172.21.15.161:0/2566067076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9cc15b600 0x7ff9cc15aac0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff9c401a5c0 tx=0x7ff9c4019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:26.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.926+0000 7ff9c3400640 1 -- 172.21.15.161:0/2566067076 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40adc80 con 0x7ff9cc15b600 2024-05-22T21:25:26.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.926+0000 7ff9c3400640 1 -- 172.21.15.161:0/2566067076 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c40adde0 con 0x7ff9cc15b600 2024-05-22T21:25:26.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.926+0000 7ff9d2800640 1 -- 172.21.15.161:0/2566067076 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9cc070280 con 0x7ff9cc15b600 2024-05-22T21:25:26.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.926+0000 7ff9c3400640 1 -- 172.21.15.161:0/2566067076 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40b9690 con 0x7ff9cc15b600 2024-05-22T21:25:26.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.926+0000 7ff9d2800640 1 -- 172.21.15.161:0/2566067076 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc139ed0 con 0x7ff9cc15b600 2024-05-22T21:25:26.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.926+0000 7ff9d2800640 1 -- 172.21.15.161:0/2566067076 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9cc070410 con 0x7ff9cc15b600 2024-05-22T21:25:26.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.926+0000 7ff9c3400640 1 -- 172.21.15.161:0/2566067076 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff9c40b97f0 con 0x7ff9cc15b600 2024-05-22T21:25:26.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.926+0000 7ff9c3400640 1 --2- 172.21.15.161:0/2566067076 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff9ac0666c0 0x7ff9ac068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:26.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.926+0000 7ff9c3400640 1 -- 172.21.15.161:0/2566067076 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff9c411f4c0 con 0x7ff9cc15b600 2024-05-22T21:25:26.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.926+0000 7ff9cb400640 1 --2- 172.21.15.161:0/2566067076 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff9ac0666c0 0x7ff9ac068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:26.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.930+0000 7ff9cb400640 1 --2- 172.21.15.161:0/2566067076 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff9ac0666c0 0x7ff9ac068b80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0011f0 tx=0x7ff9bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:26.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:26.930+0000 7ff9c3400640 1 -- 172.21.15.161:0/2566067076 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7ff9c40eca00 con 0x7ff9cc15b600 2024-05-22T21:25:27.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.086+0000 7ff9d2800640 1 -- 172.21.15.161:0/2566067076 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9cc15ec40 con 0x7ff9ac0666c0 2024-05-22T21:25:27.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.086+0000 7ff9c3400640 1 -- 172.21.15.161:0/2566067076 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff9bc0050d0 con 0x7ff9ac0666c0 2024-05-22T21:25:27.086 INFO:teuthology.orchestra.run.smithi161.stderr:instructed osd(s) 5 to deep-scrub 2024-05-22T21:25:27.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.086+0000 7ff9d2800640 1 -- 172.21.15.161:0/2566067076 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff9ac0666c0 msgr2=0x7ff9ac068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.086+0000 7ff9d2800640 1 --2- 172.21.15.161:0/2566067076 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff9ac0666c0 0x7ff9ac068b80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0011f0 tx=0x7ff9bc009040 comp rx=0 tx=0).stop 2024-05-22T21:25:27.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.086+0000 7ff9d2800640 1 -- 172.21.15.161:0/2566067076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9cc15b600 msgr2=0x7ff9cc15aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.086+0000 7ff9d2800640 1 --2- 172.21.15.161:0/2566067076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9cc15b600 0x7ff9cc15aac0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff9c401a5c0 tx=0x7ff9c4019730 comp rx=0 tx=0).stop 2024-05-22T21:25:27.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.086+0000 7ff9d2800640 1 -- 172.21.15.161:0/2566067076 shutdown_connections 2024-05-22T21:25:27.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.086+0000 7ff9d2800640 1 --2- 172.21.15.161:0/2566067076 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7ff9ac0666c0 0x7ff9ac068b80 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.086+0000 7ff9d2800640 1 --2- 172.21.15.161:0/2566067076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9cc15b600 0x7ff9cc15aac0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.086+0000 7ff9d2800640 1 --2- 172.21.15.161:0/2566067076 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9cc143c40 0x7ff9cc137bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.086+0000 7ff9d2800640 1 --2- 172.21.15.161:0/2566067076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9cc06c9b0 0x7ff9cc139530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.086+0000 7ff9d2800640 1 -- 172.21.15.161:0/2566067076 >> 172.21.15.161:0/2566067076 conn(0x7ff9cc06a2d0 msgr2=0x7ff9cc1570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:27.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.086+0000 7ff9d2800640 1 -- 172.21.15.161:0/2566067076 shutdown_connections 2024-05-22T21:25:27.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.086+0000 7ff9d2800640 1 -- 172.21.15.161:0/2566067076 wait complete. 2024-05-22T21:25:27.108 INFO:tasks.ceph:Scrubbing osd.6 2024-05-22T21:25:27.108 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-22T21:25:27.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.246+0000 7fa62b000640 1 Processor -- start 2024-05-22T21:25:27.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.246+0000 7fa62b000640 1 -- start start 2024-05-22T21:25:27.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.246+0000 7fa62b000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa62413cfe0 0x7fa62415dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.246+0000 7fa62b000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa62413d560 0x7fa62415e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.246+0000 7fa62b000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6240ce830 con 0x7fa6240cdb00 2024-05-22T21:25:27.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.246+0000 7fa62b000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6240ce650 con 0x7fa62413d560 2024-05-22T21:25:27.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.246+0000 7fa62b000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa624140560 con 0x7fa62413cfe0 2024-05-22T21:25:27.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.246+0000 7fa623400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa62413d560 0x7fa62415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.246+0000 7fa623400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa62413d560 0x7fa62415e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42120/0 (socket says 172.21.15.161:42120) 2024-05-22T21:25:27.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.246+0000 7fa629000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa6240cdb00 0x7fa6240cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60600/0 (socket says 172.21.15.161:60600) 2024-05-22T21:25:27.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.246+0000 7fa623400640 1 -- 172.21.15.161:0/3670106420 learned_addr learned my addr 172.21.15.161:0/3670106420 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:27.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.246+0000 7fa623e00640 1 --2- 172.21.15.161:0/3670106420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa62413cfe0 0x7fa62415dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa623400640 1 -- 172.21.15.161:0/3670106420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa62413cfe0 msgr2=0x7fa62415dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:27.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa623400640 1 --2- 172.21.15.161:0/3670106420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa62413cfe0 0x7fa62415dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa623400640 1 -- 172.21.15.161:0/3670106420 >> v1:172.21.15.161:6789/0 conn(0x7fa6240cdb00 legacy=0x7fa6240cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa623400640 1 -- 172.21.15.161:0/3670106420 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6240ceb10 con 0x7fa62413d560 2024-05-22T21:25:27.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa622a00640 1 -- 172.21.15.161:0/3670106420 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051553889 0 0) 0x7fa6240ce830 con 0x7fa6240cdb00 2024-05-22T21:25:27.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa623400640 1 --2- 172.21.15.161:0/3670106420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa62413d560 0x7fa62415e000 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009870 tx=0x7fa60c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca6ce99c3f409c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:27.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa622a00640 1 -- 172.21.15.161:0/3670106420 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0a6040 con 0x7fa62413d560 2024-05-22T21:25:27.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa622a00640 1 -- 172.21.15.161:0/3670106420 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60c0aa600 con 0x7fa62413d560 2024-05-22T21:25:27.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa622a00640 1 -- 172.21.15.161:0/3670106420 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0aa990 con 0x7fa62413d560 2024-05-22T21:25:27.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa62b000640 1 -- 172.21.15.161:0/3670106420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa62413d560 msgr2=0x7fa62415e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa62b000640 1 --2- 172.21.15.161:0/3670106420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa62413d560 0x7fa62415e000 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009870 tx=0x7fa60c01c4a0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa62b000640 1 -- 172.21.15.161:0/3670106420 shutdown_connections 2024-05-22T21:25:27.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa62b000640 1 --2- 172.21.15.161:0/3670106420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa62413d560 0x7fa62415e000 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa62b000640 1 --2- 172.21.15.161:0/3670106420 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa62413cfe0 0x7fa62415dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa62b000640 1 -- 172.21.15.161:0/3670106420 >> 172.21.15.161:0/3670106420 conn(0x7fa62406a0a0 msgr2=0x7fa62406a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:27.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa62b000640 1 -- 172.21.15.161:0/3670106420 shutdown_connections 2024-05-22T21:25:27.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.250+0000 7fa62b000640 1 -- 172.21.15.161:0/3670106420 wait complete. 2024-05-22T21:25:27.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa62b000640 1 Processor -- start 2024-05-22T21:25:27.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa62b000640 1 -- start start 2024-05-22T21:25:27.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa62b000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6240cdb00 0x7fa624137a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa62b000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa62413cfe0 0x7fa6241517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa62b000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa62413d560 0x7fa6240b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa62b000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa624151e10 con 0x7fa6240cdb00 2024-05-22T21:25:27.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa62b000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6240b3b00 con 0x7fa62413d560 2024-05-22T21:25:27.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa62b000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa624138180 con 0x7fa62413cfe0 2024-05-22T21:25:27.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa623e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6240cdb00 0x7fa624137a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa623e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6240cdb00 0x7fa624137a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42122/0 (socket says 172.21.15.161:42122) 2024-05-22T21:25:27.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa623e00640 1 -- 172.21.15.161:0/4095028149 learned_addr learned my addr 172.21.15.161:0/4095028149 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:27.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa629000640 1 --2- 172.21.15.161:0/4095028149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa62413d560 0x7fa6240b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa623400640 1 --2- 172.21.15.161:0/4095028149 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa62413cfe0 0x7fa6241517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa623e00640 1 -- 172.21.15.161:0/4095028149 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa62413cfe0 msgr2=0x7fa6241517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa623e00640 1 --2- 172.21.15.161:0/4095028149 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa62413cfe0 0x7fa6241517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa623e00640 1 -- 172.21.15.161:0/4095028149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa62413d560 msgr2=0x7fa6240b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa623e00640 1 --2- 172.21.15.161:0/4095028149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa62413d560 0x7fa6240b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa623e00640 1 -- 172.21.15.161:0/4095028149 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60c0094d0 con 0x7fa6240cdb00 2024-05-22T21:25:27.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa623e00640 1 --2- 172.21.15.161:0/4095028149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6240cdb00 0x7fa624137a20 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fa61400d830 tx=0x7fa61400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:27.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa61be00640 1 -- 172.21.15.161:0/4095028149 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61400b6c0 con 0x7fa6240cdb00 2024-05-22T21:25:27.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa61be00640 1 -- 172.21.15.161:0/4095028149 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61400b820 con 0x7fa6240cdb00 2024-05-22T21:25:27.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa61be00640 1 -- 172.21.15.161:0/4095028149 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61400bb90 con 0x7fa6240cdb00 2024-05-22T21:25:27.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa62b000640 1 -- 172.21.15.161:0/4095028149 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa624138420 con 0x7fa6240cdb00 2024-05-22T21:25:27.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.254+0000 7fa62b000640 1 -- 172.21.15.161:0/4095028149 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6241389e0 con 0x7fa6240cdb00 2024-05-22T21:25:27.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.258+0000 7fa62b000640 1 -- 172.21.15.161:0/4095028149 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa624067220 con 0x7fa6240cdb00 2024-05-22T21:25:27.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.258+0000 7fa61be00640 1 -- 172.21.15.161:0/4095028149 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa61401e460 con 0x7fa6240cdb00 2024-05-22T21:25:27.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.258+0000 7fa61be00640 1 --2- 172.21.15.161:0/4095028149 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6000666c0 0x7fa600068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.258+0000 7fa61be00640 1 -- 172.21.15.161:0/4095028149 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa614084620 con 0x7fa6240cdb00 2024-05-22T21:25:27.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.258+0000 7fa61be00640 1 --2- 172.21.15.161:0/4095028149 >> [v2:172.21.15.179:6820/23608,v1:172.21.15.179:6822/23608] conn(0x7fa60006c520 0x7fa60006e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.258+0000 7fa623400640 1 --2- 172.21.15.161:0/4095028149 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6000666c0 0x7fa600068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.258+0000 7fa61be00640 1 -- 172.21.15.161:0/4095028149 --> [v2:172.21.15.179:6820/23608,v1:172.21.15.179:6822/23608] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa60006eff0 con 0x7fa60006c520 2024-05-22T21:25:27.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.258+0000 7fa61be00640 1 -- 172.21.15.161:0/4095028149 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=197) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa614086c00 con 0x7fa6240cdb00 2024-05-22T21:25:27.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.258+0000 7fa629000640 1 --2- 172.21.15.161:0/4095028149 >> [v2:172.21.15.179:6820/23608,v1:172.21.15.179:6822/23608] conn(0x7fa60006c520 0x7fa60006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.258+0000 7fa623400640 1 --2- 172.21.15.161:0/4095028149 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6000666c0 0x7fa600068b80 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0ae000 tx=0x7fa60c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:27.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.258+0000 7fa629000640 1 --2- 172.21.15.161:0/4095028149 >> [v2:172.21.15.179:6820/23608,v1:172.21.15.179:6822/23608] conn(0x7fa60006c520 0x7fa60006e940 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:27.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.262+0000 7fa61be00640 1 -- 172.21.15.161:0/4095028149 <== osd.6 v2:172.21.15.179:6820/23608 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24459 (crc 0 0 0) 0x7fa60006eff0 con 0x7fa60006c520 2024-05-22T21:25:27.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.270+0000 7fa62b000640 1 -- 172.21.15.161:0/4095028149 --> [v2:172.21.15.179:6820/23608,v1:172.21.15.179:6822/23608] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa624138b70 con 0x7fa60006c520 2024-05-22T21:25:27.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.270+0000 7fa61be00640 1 -- 172.21.15.161:0/4095028149 <== osd.6 v2:172.21.15.179:6820/23608 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7fa624138b70 con 0x7fa60006c520 2024-05-22T21:25:27.273 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:25:27.273 INFO:teuthology.orchestra.run.smithi161.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-05-22T21:25:27.273 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:25:27.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.274+0000 7fa62b000640 1 -- 172.21.15.161:0/4095028149 >> [v2:172.21.15.179:6820/23608,v1:172.21.15.179:6822/23608] conn(0x7fa60006c520 msgr2=0x7fa60006e940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.274+0000 7fa62b000640 1 --2- 172.21.15.161:0/4095028149 >> [v2:172.21.15.179:6820/23608,v1:172.21.15.179:6822/23608] conn(0x7fa60006c520 0x7fa60006e940 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.274+0000 7fa62b000640 1 -- 172.21.15.161:0/4095028149 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6000666c0 msgr2=0x7fa600068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.274+0000 7fa62b000640 1 --2- 172.21.15.161:0/4095028149 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fa6000666c0 0x7fa600068b80 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0ae000 tx=0x7fa60c0a7000 comp rx=0 tx=0).stop 2024-05-22T21:25:27.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.274+0000 7fa62b000640 1 -- 172.21.15.161:0/4095028149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6240cdb00 msgr2=0x7fa624137a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.274+0000 7fa62b000640 1 --2- 172.21.15.161:0/4095028149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6240cdb00 0x7fa624137a20 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fa61400d830 tx=0x7fa61400dcf0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.274+0000 7fa629000640 1 -- 172.21.15.161:0/4095028149 reap_dead start 2024-05-22T21:25:27.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.274+0000 7fa62b000640 1 -- 172.21.15.161:0/4095028149 shutdown_connections 2024-05-22T21:25:27.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.274+0000 7fa62b000640 1 -- 172.21.15.161:0/4095028149 >> 172.21.15.161:0/4095028149 conn(0x7fa62406a0a0 msgr2=0x7fa62406d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:27.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.274+0000 7fa62b000640 1 -- 172.21.15.161:0/4095028149 shutdown_connections 2024-05-22T21:25:27.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.274+0000 7fa62b000640 1 -- 172.21.15.161:0/4095028149 wait complete. 2024-05-22T21:25:27.291 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-22T21:25:27.427 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.426+0000 7fea8dc00640 1 Processor -- start 2024-05-22T21:25:27.428 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.426+0000 7fea8dc00640 1 -- start start 2024-05-22T21:25:27.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.426+0000 7fea8dc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea88143c40 0x7fea8815b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.426+0000 7fea8dc00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fea8806c9b0 0x7fea880b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.426+0000 7fea8dc00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea8815efb0 con 0x7fea881444a0 2024-05-22T21:25:27.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.430+0000 7fea8dc00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea8815f290 con 0x7fea8806c9b0 2024-05-22T21:25:27.429 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.430+0000 7fea8dc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8815f570 con 0x7fea88143c40 2024-05-22T21:25:27.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.430+0000 7fea87400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fea8806c9b0 0x7fea880b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.430+0000 7fea86a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea88143c40 0x7fea8815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.430+0000 7fea87400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fea8806c9b0 0x7fea880b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.161:37954/0 (socket says 172.21.15.161:37954) 2024-05-22T21:25:27.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.430+0000 7fea87400640 1 -- 172.21.15.161:0/1074188623 learned_addr learned my addr 172.21.15.161:0/1074188623 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:27.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.430+0000 7fea87400640 1 -- 172.21.15.161:0/1074188623 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea88143c40 msgr2=0x7fea8815b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:27.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.430+0000 7fea87400640 1 --2- 172.21.15.161:0/1074188623 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea88143c40 0x7fea8815b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.430 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.430+0000 7fea87400640 1 -- 172.21.15.161:0/1074188623 >> v1:172.21.15.179:6789/0 conn(0x7fea881444a0 legacy=0x7fea8806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.430+0000 7fea87400640 1 -- 172.21.15.161:0/1074188623 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8815f850 con 0x7fea8806c9b0 2024-05-22T21:25:27.431 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.430+0000 7fea85600640 1 -- 172.21.15.161:0/1074188623 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281214421 0 0) 0x7fea8815efb0 con 0x7fea881444a0 2024-05-22T21:25:27.432 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.430+0000 7fea87400640 1 --2- 172.21.15.161:0/1074188623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea8806c9b0 0x7fea880b2160 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fea80009870 tx=0x7fea8001c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77f9e0b68c3a1b90 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:27.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea85600640 1 -- 172.21.15.161:0/1074188623 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea800aa620 con 0x7fea8806c9b0 2024-05-22T21:25:27.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea85600640 1 -- 172.21.15.161:0/1074188623 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea800aa780 con 0x7fea8806c9b0 2024-05-22T21:25:27.433 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea85600640 1 -- 172.21.15.161:0/1074188623 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea800aab10 con 0x7fea8806c9b0 2024-05-22T21:25:27.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea8dc00640 1 -- 172.21.15.161:0/1074188623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea8806c9b0 msgr2=0x7fea880b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea8dc00640 1 --2- 172.21.15.161:0/1074188623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea8806c9b0 0x7fea880b2160 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fea80009870 tx=0x7fea8001c560 comp rx=0 tx=0).stop 2024-05-22T21:25:27.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea8dc00640 1 -- 172.21.15.161:0/1074188623 shutdown_connections 2024-05-22T21:25:27.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea8dc00640 1 --2- 172.21.15.161:0/1074188623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea8806c9b0 0x7fea880b2160 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea8dc00640 1 --2- 172.21.15.161:0/1074188623 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea88143c40 0x7fea8815b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea8dc00640 1 -- 172.21.15.161:0/1074188623 >> 172.21.15.161:0/1074188623 conn(0x7fea8806a2d0 msgr2=0x7fea8806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:27.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea8dc00640 1 -- 172.21.15.161:0/1074188623 shutdown_connections 2024-05-22T21:25:27.434 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea8dc00640 1 -- 172.21.15.161:0/1074188623 wait complete. 2024-05-22T21:25:27.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea8dc00640 1 Processor -- start 2024-05-22T21:25:27.435 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea8dc00640 1 -- start start 2024-05-22T21:25:27.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea8dc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea8806c9b0 0x7fea8814a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea8dc00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea88143c40 0x7fea881587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea8dc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea881444a0 0x7fea88158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea8dc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8813abd0 con 0x7fea8806c9b0 2024-05-22T21:25:27.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea8dc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8815bb20 con 0x7fea881444a0 2024-05-22T21:25:27.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea8dc00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8813ae70 con 0x7fea88143c40 2024-05-22T21:25:27.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea86a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea8806c9b0 0x7fea8814a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea86a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea8806c9b0 0x7fea8814a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42138/0 (socket says 172.21.15.161:42138) 2024-05-22T21:25:27.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea86a00640 1 -- 172.21.15.161:0/672888475 learned_addr learned my addr 172.21.15.161:0/672888475 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:27.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea87400640 1 --2- 172.21.15.161:0/672888475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea881444a0 0x7fea88158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.434+0000 7fea86000640 1 --2- 172.21.15.161:0/672888475 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea88143c40 0x7fea881587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.438+0000 7fea86a00640 1 -- 172.21.15.161:0/672888475 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea88143c40 msgr2=0x7fea881587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.438+0000 7fea86a00640 1 --2- 172.21.15.161:0/672888475 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea88143c40 0x7fea881587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.438+0000 7fea86a00640 1 -- 172.21.15.161:0/672888475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea881444a0 msgr2=0x7fea88158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.438+0000 7fea86a00640 1 --2- 172.21.15.161:0/672888475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea881444a0 0x7fea88158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.438+0000 7fea86a00640 1 -- 172.21.15.161:0/672888475 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea800094d0 con 0x7fea8806c9b0 2024-05-22T21:25:27.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.438+0000 7fea86a00640 1 --2- 172.21.15.161:0/672888475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea8806c9b0 0x7fea8814a6c0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fea7800efc0 tx=0x7fea7800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:27.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.438+0000 7fea7ea00640 1 -- 172.21.15.161:0/672888475 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78014480 con 0x7fea8806c9b0 2024-05-22T21:25:27.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.438+0000 7fea7ea00640 1 -- 172.21.15.161:0/672888475 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7800f040 con 0x7fea8806c9b0 2024-05-22T21:25:27.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.438+0000 7fea8dc00640 1 -- 172.21.15.161:0/672888475 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea8813b150 con 0x7fea8806c9b0 2024-05-22T21:25:27.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.438+0000 7fea7ea00640 1 -- 172.21.15.161:0/672888475 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78014800 con 0x7fea8806c9b0 2024-05-22T21:25:27.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.438+0000 7fea8dc00640 1 -- 172.21.15.161:0/672888475 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea88159860 con 0x7fea8806c9b0 2024-05-22T21:25:27.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.438+0000 7fea8dc00640 1 -- 172.21.15.161:0/672888475 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea880b1a60 con 0x7fea8806c9b0 2024-05-22T21:25:27.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.438+0000 7fea7ea00640 1 -- 172.21.15.161:0/672888475 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fea78014960 con 0x7fea8806c9b0 2024-05-22T21:25:27.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.438+0000 7fea7ea00640 1 --2- 172.21.15.161:0/672888475 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea680666c0 0x7fea68068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.438+0000 7fea7ea00640 1 -- 172.21.15.161:0/672888475 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fea78084930 con 0x7fea8806c9b0 2024-05-22T21:25:27.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.438+0000 7fea86000640 1 --2- 172.21.15.161:0/672888475 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea680666c0 0x7fea68068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.442+0000 7fea86000640 1 --2- 172.21.15.161:0/672888475 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea680666c0 0x7fea68068b80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fea880ae4d0 tx=0x7fea74009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:27.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.442+0000 7fea7ea00640 1 -- 172.21.15.161:0/672888475 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fea78051fb0 con 0x7fea8806c9b0 2024-05-22T21:25:27.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.598+0000 7fea8dc00640 1 -- 172.21.15.161:0/672888475 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fea880ae980 con 0x7fea680666c0 2024-05-22T21:25:27.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.598+0000 7fea7ea00640 1 -- 172.21.15.161:0/672888475 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fea880ae980 con 0x7fea680666c0 2024-05-22T21:25:27.601 INFO:teuthology.orchestra.run.smithi161.stderr:instructed osd(s) 6 to deep-scrub 2024-05-22T21:25:27.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.602+0000 7fea8dc00640 1 -- 172.21.15.161:0/672888475 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea680666c0 msgr2=0x7fea68068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.602+0000 7fea8dc00640 1 --2- 172.21.15.161:0/672888475 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea680666c0 0x7fea68068b80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fea880ae4d0 tx=0x7fea74009040 comp rx=0 tx=0).stop 2024-05-22T21:25:27.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.602+0000 7fea8dc00640 1 -- 172.21.15.161:0/672888475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea8806c9b0 msgr2=0x7fea8814a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.602+0000 7fea8dc00640 1 --2- 172.21.15.161:0/672888475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea8806c9b0 0x7fea8814a6c0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fea7800efc0 tx=0x7fea7800c490 comp rx=0 tx=0).stop 2024-05-22T21:25:27.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.602+0000 7fea8dc00640 1 -- 172.21.15.161:0/672888475 shutdown_connections 2024-05-22T21:25:27.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.602+0000 7fea8dc00640 1 --2- 172.21.15.161:0/672888475 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fea680666c0 0x7fea68068b80 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.602+0000 7fea8dc00640 1 --2- 172.21.15.161:0/672888475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea881444a0 0x7fea88158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.602+0000 7fea8dc00640 1 --2- 172.21.15.161:0/672888475 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea88143c40 0x7fea881587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.602+0000 7fea8dc00640 1 --2- 172.21.15.161:0/672888475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea8806c9b0 0x7fea8814a6c0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.602+0000 7fea8dc00640 1 -- 172.21.15.161:0/672888475 >> 172.21.15.161:0/672888475 conn(0x7fea8806a2d0 msgr2=0x7fea8806c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:27.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.602+0000 7fea8dc00640 1 -- 172.21.15.161:0/672888475 shutdown_connections 2024-05-22T21:25:27.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.602+0000 7fea8dc00640 1 -- 172.21.15.161:0/672888475 wait complete. 2024-05-22T21:25:27.622 INFO:tasks.ceph:Scrubbing osd.7 2024-05-22T21:25:27.623 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-22T21:25:27.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.762+0000 7fd031800640 1 Processor -- start 2024-05-22T21:25:27.763 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.762+0000 7fd031800640 1 -- start start 2024-05-22T21:25:27.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.762+0000 7fd031800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd02c143a10 0x7fd02c06d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.762+0000 7fd031800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02c06d290 con 0x7fd02c161510 2024-05-22T21:25:27.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.762+0000 7fd031800640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02c0adf60 con 0x7fd02c143630 2024-05-22T21:25:27.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.762+0000 7fd031800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd02c06cc40 con 0x7fd02c143a10 2024-05-22T21:25:27.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.762+0000 7fd02a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd02c143a10 0x7fd02c06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.762+0000 7fd02b400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd02c161510 0x7fd02c06fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60610/0 (socket says 172.21.15.161:60610) 2024-05-22T21:25:27.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.762+0000 7fd02b400640 1 -- 172.21.15.161:0/197170422 learned_addr learned my addr 172.21.15.161:0/197170422 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:27.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.766+0000 7fd02a000640 1 -- 172.21.15.161:0/197170422 >> v1:172.21.15.179:6789/0 conn(0x7fd02c143630 legacy=0x7fd02c160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:27.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.766+0000 7fd02a000640 1 -- 172.21.15.161:0/197170422 >> v1:172.21.15.161:6789/0 conn(0x7fd02c161510 legacy=0x7fd02c06fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.766+0000 7fd029600640 1 -- 172.21.15.161:0/197170422 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587141142 0 0) 0x7fd02c06d290 con 0x7fd02c161510 2024-05-22T21:25:27.766 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.766+0000 7fd02a000640 1 -- 172.21.15.161:0/197170422 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c06cf20 con 0x7fd02c143a10 2024-05-22T21:25:27.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.766+0000 7fd02a000640 1 --2- 172.21.15.161:0/197170422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd02c143a10 0x7fd02c06d720 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fd0180097c0 tx=0x7fd01801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcef6c0b618290f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:27.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.766+0000 7fd029600640 1 -- 172.21.15.161:0/197170422 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0180ab600 con 0x7fd02c143a10 2024-05-22T21:25:27.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.766+0000 7fd029600640 1 -- 172.21.15.161:0/197170422 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0180a7050 con 0x7fd02c143a10 2024-05-22T21:25:27.768 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.766+0000 7fd029600640 1 -- 172.21.15.161:0/197170422 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0180ab980 con 0x7fd02c143a10 2024-05-22T21:25:27.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd031800640 1 -- 172.21.15.161:0/197170422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd02c143a10 msgr2=0x7fd02c06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd031800640 1 --2- 172.21.15.161:0/197170422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd02c143a10 0x7fd02c06d720 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fd0180097c0 tx=0x7fd01801c3e0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd031800640 1 -- 172.21.15.161:0/197170422 shutdown_connections 2024-05-22T21:25:27.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd031800640 1 --2- 172.21.15.161:0/197170422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd02c143a10 0x7fd02c06d720 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd031800640 1 -- 172.21.15.161:0/197170422 >> 172.21.15.161:0/197170422 conn(0x7fd02c06a0a0 msgr2=0x7fd02c06a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:27.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd031800640 1 -- 172.21.15.161:0/197170422 shutdown_connections 2024-05-22T21:25:27.769 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd031800640 1 -- 172.21.15.161:0/197170422 wait complete. 2024-05-22T21:25:27.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd031800640 1 Processor -- start 2024-05-22T21:25:27.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd031800640 1 -- start start 2024-05-22T21:25:27.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd031800640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd02c143630 0x7fd02c15d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd031800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd02c143a10 0x7fd02c159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd031800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd02c161510 0x7fd02c145f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd031800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c15ddc0 con 0x7fd02c143a10 2024-05-22T21:25:27.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd031800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c144bf0 con 0x7fd02c161510 2024-05-22T21:25:27.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd031800640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c144ed0 con 0x7fd02c143630 2024-05-22T21:25:27.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd02a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd02c143a10 0x7fd02c159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd02a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd02c143a10 0x7fd02c159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42148/0 (socket says 172.21.15.161:42148) 2024-05-22T21:25:27.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd02a000640 1 -- 172.21.15.161:0/1951119537 learned_addr learned my addr 172.21.15.161:0/1951119537 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:27.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd02aa00640 1 --2- 172.21.15.161:0/1951119537 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd02c143630 0x7fd02c15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd02b400640 1 --2- 172.21.15.161:0/1951119537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd02c161510 0x7fd02c145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd02a000640 1 -- 172.21.15.161:0/1951119537 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd02c143630 msgr2=0x7fd02c15d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd02a000640 1 --2- 172.21.15.161:0/1951119537 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd02c143630 0x7fd02c15d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd02a000640 1 -- 172.21.15.161:0/1951119537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd02c161510 msgr2=0x7fd02c145f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd02a000640 1 --2- 172.21.15.161:0/1951119537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd02c161510 0x7fd02c145f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd02a000640 1 -- 172.21.15.161:0/1951119537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd018009420 con 0x7fd02c143a10 2024-05-22T21:25:27.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd02a000640 1 --2- 172.21.15.161:0/1951119537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd02c143a10 0x7fd02c159540 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fd018018ff0 tx=0x7fd018019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:27.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd022a00640 1 -- 172.21.15.161:0/1951119537 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0180b4610 con 0x7fd02c143a10 2024-05-22T21:25:27.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.770+0000 7fd022a00640 1 -- 172.21.15.161:0/1951119537 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0180b4790 con 0x7fd02c143a10 2024-05-22T21:25:27.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.774+0000 7fd031800640 1 -- 172.21.15.161:0/1951119537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02c1451b0 con 0x7fd02c143a10 2024-05-22T21:25:27.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.774+0000 7fd022a00640 1 -- 172.21.15.161:0/1951119537 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0180b4b40 con 0x7fd02c143a10 2024-05-22T21:25:27.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.774+0000 7fd031800640 1 -- 172.21.15.161:0/1951119537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02c145770 con 0x7fd02c143a10 2024-05-22T21:25:27.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.774+0000 7fd031800640 1 -- 172.21.15.161:0/1951119537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd02c067220 con 0x7fd02c143a10 2024-05-22T21:25:27.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.774+0000 7fd022a00640 1 -- 172.21.15.161:0/1951119537 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd0180b2040 con 0x7fd02c143a10 2024-05-22T21:25:27.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.774+0000 7fd022a00640 1 --2- 172.21.15.161:0/1951119537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd00c066690 0x7fd00c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.774+0000 7fd022a00640 1 -- 172.21.15.161:0/1951119537 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd01811e350 con 0x7fd02c143a10 2024-05-22T21:25:27.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.774+0000 7fd02aa00640 1 --2- 172.21.15.161:0/1951119537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd00c066690 0x7fd00c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.774+0000 7fd022a00640 1 --2- 172.21.15.161:0/1951119537 >> [v2:172.21.15.179:6816/23609,v1:172.21.15.179:6817/23609] conn(0x7fd00c06c4f0 0x7fd00c06e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.774+0000 7fd022a00640 1 -- 172.21.15.161:0/1951119537 --> [v2:172.21.15.179:6816/23609,v1:172.21.15.179:6817/23609] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd00c06efc0 con 0x7fd00c06c4f0 2024-05-22T21:25:27.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.774+0000 7fd022a00640 1 -- 172.21.15.161:0/1951119537 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=197) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd01811e6c0 con 0x7fd02c143a10 2024-05-22T21:25:27.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.774+0000 7fd02aa00640 1 --2- 172.21.15.161:0/1951119537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd00c066690 0x7fd00c068b50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0012f0 tx=0x7fd01c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:27.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.774+0000 7fd02b400640 1 --2- 172.21.15.161:0/1951119537 >> [v2:172.21.15.179:6816/23609,v1:172.21.15.179:6817/23609] conn(0x7fd00c06c4f0 0x7fd00c06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.778+0000 7fd02b400640 1 --2- 172.21.15.161:0/1951119537 >> [v2:172.21.15.179:6816/23609,v1:172.21.15.179:6817/23609] conn(0x7fd00c06c4f0 0x7fd00c06e910 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:27.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.778+0000 7fd022a00640 1 -- 172.21.15.161:0/1951119537 <== osd.7 v2:172.21.15.179:6816/23609 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24459 (crc 0 0 0) 0x7fd00c06efc0 con 0x7fd00c06c4f0 2024-05-22T21:25:27.788 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.786+0000 7fd031800640 1 -- 172.21.15.161:0/1951119537 --> [v2:172.21.15.179:6816/23609,v1:172.21.15.179:6817/23609] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd02c160b70 con 0x7fd00c06c4f0 2024-05-22T21:25:27.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.790+0000 7fd022a00640 1 -- 172.21.15.161:0/1951119537 <== osd.7 v2:172.21.15.179:6816/23609 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7fd02c160b70 con 0x7fd00c06c4f0 2024-05-22T21:25:27.790 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-22T21:25:27.790 INFO:teuthology.orchestra.run.smithi161.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-05-22T21:25:27.790 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-22T21:25:27.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.790+0000 7fd031800640 1 -- 172.21.15.161:0/1951119537 >> [v2:172.21.15.179:6816/23609,v1:172.21.15.179:6817/23609] conn(0x7fd00c06c4f0 msgr2=0x7fd00c06e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.790+0000 7fd031800640 1 --2- 172.21.15.161:0/1951119537 >> [v2:172.21.15.179:6816/23609,v1:172.21.15.179:6817/23609] conn(0x7fd00c06c4f0 0x7fd00c06e910 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.790+0000 7fd031800640 1 -- 172.21.15.161:0/1951119537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd00c066690 msgr2=0x7fd00c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.790+0000 7fd031800640 1 --2- 172.21.15.161:0/1951119537 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fd00c066690 0x7fd00c068b50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0012f0 tx=0x7fd01c009040 comp rx=0 tx=0).stop 2024-05-22T21:25:27.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.790+0000 7fd031800640 1 -- 172.21.15.161:0/1951119537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd02c143a10 msgr2=0x7fd02c159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.790+0000 7fd031800640 1 --2- 172.21.15.161:0/1951119537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd02c143a10 0x7fd02c159540 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fd018018ff0 tx=0x7fd018019180 comp rx=0 tx=0).stop 2024-05-22T21:25:27.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.790+0000 7fd02b400640 1 -- 172.21.15.161:0/1951119537 reap_dead start 2024-05-22T21:25:27.791 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.790+0000 7fd031800640 1 -- 172.21.15.161:0/1951119537 shutdown_connections 2024-05-22T21:25:27.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.790+0000 7fd031800640 1 -- 172.21.15.161:0/1951119537 >> 172.21.15.161:0/1951119537 conn(0x7fd02c06a0a0 msgr2=0x7fd02c070b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:27.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.790+0000 7fd031800640 1 -- 172.21.15.161:0/1951119537 shutdown_connections 2024-05-22T21:25:27.792 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.790+0000 7fd031800640 1 -- 172.21.15.161:0/1951119537 wait complete. 2024-05-22T21:25:27.809 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-22T21:25:27.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc92a00640 1 Processor -- start 2024-05-22T21:25:27.945 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc92a00640 1 -- start start 2024-05-22T21:25:27.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc92a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efc8c06b260 0x7efc8c0d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc92a00640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efc8c0d12a0 0x7efc8c0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.946 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc92a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc8c0d16d0 con 0x7efc8c160720 2024-05-22T21:25:27.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc92a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc8c160b00 con 0x7efc8c06b260 2024-05-22T21:25:27.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc92a00640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc8c0b12b0 con 0x7efc8c0d12a0 2024-05-22T21:25:27.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc90c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efc8c06b260 0x7efc8c0d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc91600640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7efc8c160720 0x7efc8c0b0a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:60626/0 (socket says 172.21.15.161:60626) 2024-05-22T21:25:27.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc91600640 1 -- 172.21.15.161:0/1480423343 learned_addr learned my addr 172.21.15.161:0/1480423343 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-05-22T21:25:27.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc92000640 1 --2- 172.21.15.161:0/1480423343 >> v2:172.21.15.179:3300/0 conn(0x7efc8c0d12a0 0x7efc8c0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc90c00640 1 -- 172.21.15.161:0/1480423343 >> v2:172.21.15.179:3300/0 conn(0x7efc8c0d12a0 msgr2=0x7efc8c0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:27.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc90c00640 1 --2- 172.21.15.161:0/1480423343 >> v2:172.21.15.179:3300/0 conn(0x7efc8c0d12a0 0x7efc8c0cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc90c00640 1 -- 172.21.15.161:0/1480423343 >> v1:172.21.15.161:6789/0 conn(0x7efc8c160720 legacy=0x7efc8c0b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc90c00640 1 -- 172.21.15.161:0/1480423343 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c0b1590 con 0x7efc8c06b260 2024-05-22T21:25:27.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc92000640 1 --2- 172.21.15.161:0/1480423343 >> v2:172.21.15.179:3300/0 conn(0x7efc8c0d12a0 0x7efc8c0cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:25:27.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.946+0000 7efc8be00640 1 -- 172.21.15.161:0/1480423343 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846453170 0 0) 0x7efc8c0d16d0 con 0x7efc8c160720 2024-05-22T21:25:27.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.950+0000 7efc90c00640 1 --2- 172.21.15.161:0/1480423343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc8c06b260 0x7efc8c0d0d60 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0c48d0 tx=0x7efc7801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c420fc9792e11fa server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:27.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.950+0000 7efc8be00640 1 -- 172.21.15.161:0/1480423343 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780ab650 con 0x7efc8c06b260 2024-05-22T21:25:27.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.950+0000 7efc8be00640 1 -- 172.21.15.161:0/1480423343 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc780ab7b0 con 0x7efc8c06b260 2024-05-22T21:25:27.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.950+0000 7efc8be00640 1 -- 172.21.15.161:0/1480423343 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780abb40 con 0x7efc8c06b260 2024-05-22T21:25:27.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.950+0000 7efc92a00640 1 -- 172.21.15.161:0/1480423343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc8c06b260 msgr2=0x7efc8c0d0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.950+0000 7efc92a00640 1 --2- 172.21.15.161:0/1480423343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc8c06b260 0x7efc8c0d0d60 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0c48d0 tx=0x7efc7801c450 comp rx=0 tx=0).stop 2024-05-22T21:25:27.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.950+0000 7efc92a00640 1 -- 172.21.15.161:0/1480423343 shutdown_connections 2024-05-22T21:25:27.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.950+0000 7efc92a00640 1 --2- 172.21.15.161:0/1480423343 >> v2:172.21.15.179:3300/0 conn(0x7efc8c0d12a0 0x7efc8c0cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.950+0000 7efc92a00640 1 --2- 172.21.15.161:0/1480423343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc8c06b260 0x7efc8c0d0d60 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.950+0000 7efc92a00640 1 -- 172.21.15.161:0/1480423343 >> 172.21.15.161:0/1480423343 conn(0x7efc8c099a40 msgr2=0x7efc8c099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:27.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.950+0000 7efc92a00640 1 -- 172.21.15.161:0/1480423343 shutdown_connections 2024-05-22T21:25:27.951 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.950+0000 7efc92a00640 1 -- 172.21.15.161:0/1480423343 wait complete. 2024-05-22T21:25:27.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.950+0000 7efc92a00640 1 Processor -- start 2024-05-22T21:25:27.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.950+0000 7efc92a00640 1 -- start start 2024-05-22T21:25:27.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc8c06b260 0x7efc8c0c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92a00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8c0d12a0 0x7efc8c14fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc8c160720 0x7efc8c0aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c0ae6e0 con 0x7efc8c160720 2024-05-22T21:25:27.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c0adad0 con 0x7efc8c06b260 2024-05-22T21:25:27.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92a00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c0addb0 con 0x7efc8c0d12a0 2024-05-22T21:25:27.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc8c160720 0x7efc8c0aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc8c160720 0x7efc8c0aeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42160/0 (socket says 172.21.15.161:42160) 2024-05-22T21:25:27.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92000640 1 -- 172.21.15.161:0/536213787 learned_addr learned my addr 172.21.15.161:0/536213787 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:27.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc91600640 1 --2- 172.21.15.161:0/536213787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc8c06b260 0x7efc8c0c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc90c00640 1 --2- 172.21.15.161:0/536213787 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8c0d12a0 0x7efc8c14fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92000640 1 -- 172.21.15.161:0/536213787 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8c0d12a0 msgr2=0x7efc8c14fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92000640 1 --2- 172.21.15.161:0/536213787 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8c0d12a0 0x7efc8c14fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92000640 1 -- 172.21.15.161:0/536213787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc8c06b260 msgr2=0x7efc8c0c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:27.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92000640 1 --2- 172.21.15.161:0/536213787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc8c06b260 0x7efc8c0c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:27.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92000640 1 -- 172.21.15.161:0/536213787 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc78009420 con 0x7efc8c160720 2024-05-22T21:25:27.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92000640 1 --2- 172.21.15.161:0/536213787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc8c160720 0x7efc8c0aeff0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7efc8400c7f0 tx=0x7efc8400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:27.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc89600640 1 -- 172.21.15.161:0/536213787 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84007820 con 0x7efc8c160720 2024-05-22T21:25:27.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc89600640 1 -- 172.21.15.161:0/536213787 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc840079c0 con 0x7efc8c160720 2024-05-22T21:25:27.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc89600640 1 -- 172.21.15.161:0/536213787 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84007d70 con 0x7efc8c160720 2024-05-22T21:25:27.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92a00640 1 -- 172.21.15.161:0/536213787 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8c0ae090 con 0x7efc8c160720 2024-05-22T21:25:27.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92a00640 1 -- 172.21.15.161:0/536213787 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc8c0aeb70 con 0x7efc8c160720 2024-05-22T21:25:27.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc92a00640 1 -- 172.21.15.161:0/536213787 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc8c0ae850 con 0x7efc8c160720 2024-05-22T21:25:27.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.954+0000 7efc89600640 1 -- 172.21.15.161:0/536213787 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efc840113f0 con 0x7efc8c160720 2024-05-22T21:25:27.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.958+0000 7efc89600640 1 --2- 172.21.15.161:0/536213787 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc6c066690 0x7efc6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:27.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.958+0000 7efc89600640 1 -- 172.21.15.161:0/536213787 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efc84084340 con 0x7efc8c160720 2024-05-22T21:25:27.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.958+0000 7efc91600640 1 --2- 172.21.15.161:0/536213787 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc6c066690 0x7efc6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:27.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.958+0000 7efc91600640 1 --2- 172.21.15.161:0/536213787 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc6c066690 0x7efc6c068b50 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7efc800012f0 tx=0x7efc80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:27.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:27.962+0000 7efc89600640 1 -- 172.21.15.161:0/536213787 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7efc84051910 con 0x7efc8c160720 2024-05-22T21:25:28.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.114+0000 7efc92a00640 1 -- 172.21.15.161:0/536213787 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7efc8c0ae540 con 0x7efc6c066690 2024-05-22T21:25:28.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.118+0000 7efc89600640 1 -- 172.21.15.161:0/536213787 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efc80001e30 con 0x7efc6c066690 2024-05-22T21:25:28.117 INFO:teuthology.orchestra.run.smithi161.stderr:instructed osd(s) 7 to deep-scrub 2024-05-22T21:25:28.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.118+0000 7efc92a00640 1 -- 172.21.15.161:0/536213787 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc6c066690 msgr2=0x7efc6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:28.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.118+0000 7efc92a00640 1 --2- 172.21.15.161:0/536213787 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc6c066690 0x7efc6c068b50 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7efc800012f0 tx=0x7efc80009040 comp rx=0 tx=0).stop 2024-05-22T21:25:28.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.118+0000 7efc92a00640 1 -- 172.21.15.161:0/536213787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc8c160720 msgr2=0x7efc8c0aeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:28.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.118+0000 7efc92a00640 1 --2- 172.21.15.161:0/536213787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc8c160720 0x7efc8c0aeff0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7efc8400c7f0 tx=0x7efc8400ccb0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.118+0000 7efc92a00640 1 -- 172.21.15.161:0/536213787 shutdown_connections 2024-05-22T21:25:28.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.118+0000 7efc92a00640 1 --2- 172.21.15.161:0/536213787 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7efc6c066690 0x7efc6c068b50 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.118+0000 7efc92a00640 1 --2- 172.21.15.161:0/536213787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc8c160720 0x7efc8c0aeff0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.118+0000 7efc92a00640 1 --2- 172.21.15.161:0/536213787 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8c0d12a0 0x7efc8c14fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.118+0000 7efc92a00640 1 --2- 172.21.15.161:0/536213787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc8c06b260 0x7efc8c0c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.118+0000 7efc92a00640 1 -- 172.21.15.161:0/536213787 >> 172.21.15.161:0/536213787 conn(0x7efc8c099a40 msgr2=0x7efc8c15e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:28.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.118+0000 7efc92a00640 1 -- 172.21.15.161:0/536213787 shutdown_connections 2024-05-22T21:25:28.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.118+0000 7efc92a00640 1 -- 172.21.15.161:0/536213787 wait complete. 2024-05-22T21:25:28.138 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:25:28.273 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae18000640 1 Processor -- start 2024-05-22T21:25:28.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae18000640 1 -- start start 2024-05-22T21:25:28.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae18000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fae1015db70 0x7fae10142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:28.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae18000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fae10140a70 0x7fae100cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:28.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae18000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae10140ed0 0x7fae1006a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:28.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae18000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae10071110 con 0x7fae10140a70 2024-05-22T21:25:28.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae18000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae101430e0 con 0x7fae1015db70 2024-05-22T21:25:28.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae18000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae101433c0 con 0x7fae10140ed0 2024-05-22T21:25:28.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae14c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fae10140a70 0x7fae100cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:28.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae14c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fae10140a70 0x7fae100cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42172/0 (socket says 172.21.15.161:42172) 2024-05-22T21:25:28.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae14c00640 1 -- 172.21.15.161:0/1567759514 learned_addr learned my addr 172.21.15.161:0/1567759514 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:28.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae16000640 1 --2- 172.21.15.161:0/1567759514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae10140ed0 0x7fae1006a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:28.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae15600640 1 --2- 172.21.15.161:0/1567759514 >> v2:172.21.15.179:3300/0 conn(0x7fae1015db70 0x7fae10142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:28.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae14c00640 1 -- 172.21.15.161:0/1567759514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae10140ed0 msgr2=0x7fae1006a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:28.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae14c00640 1 --2- 172.21.15.161:0/1567759514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae10140ed0 0x7fae1006a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae14c00640 1 -- 172.21.15.161:0/1567759514 >> v2:172.21.15.179:3300/0 conn(0x7fae1015db70 msgr2=0x7fae10142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:28.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae14c00640 1 --2- 172.21.15.161:0/1567759514 >> v2:172.21.15.179:3300/0 conn(0x7fae1015db70 0x7fae10142ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae14c00640 1 -- 172.21.15.161:0/1567759514 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae101436a0 con 0x7fae10140a70 2024-05-22T21:25:28.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae16000640 1 --2- 172.21.15.161:0/1567759514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae10140ed0 0x7fae1006a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:25:28.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.274+0000 7fae15600640 1 --2- 172.21.15.161:0/1567759514 >> v2:172.21.15.179:3300/0 conn(0x7fae1015db70 0x7fae10142ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:25:28.278 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.278+0000 7fae14c00640 1 --2- 172.21.15.161:0/1567759514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae10140a70 0x7fae100cd010 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fae100b21b0 tx=0x7fadfc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8563e88faee4fc59 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:28.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.278+0000 7fae0fe00640 1 -- 172.21.15.161:0/1567759514 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc0aa430 con 0x7fae10140a70 2024-05-22T21:25:28.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.278+0000 7fae0fe00640 1 -- 172.21.15.161:0/1567759514 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadfc0aa590 con 0x7fae10140a70 2024-05-22T21:25:28.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.278+0000 7fae0fe00640 1 -- 172.21.15.161:0/1567759514 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc0aa920 con 0x7fae10140a70 2024-05-22T21:25:28.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.278+0000 7fae18000640 1 -- 172.21.15.161:0/1567759514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae10140a70 msgr2=0x7fae100cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:28.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.278+0000 7fae18000640 1 --2- 172.21.15.161:0/1567759514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae10140a70 0x7fae100cd010 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fae100b21b0 tx=0x7fadfc01c4f0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.278+0000 7fae18000640 1 -- 172.21.15.161:0/1567759514 shutdown_connections 2024-05-22T21:25:28.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.278+0000 7fae18000640 1 --2- 172.21.15.161:0/1567759514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae10140ed0 0x7fae1006a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.278+0000 7fae18000640 1 --2- 172.21.15.161:0/1567759514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae10140a70 0x7fae100cd010 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.278+0000 7fae18000640 1 --2- 172.21.15.161:0/1567759514 >> v2:172.21.15.179:3300/0 conn(0x7fae1015db70 0x7fae10142ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.278+0000 7fae18000640 1 -- 172.21.15.161:0/1567759514 >> 172.21.15.161:0/1567759514 conn(0x7fae100b5540 msgr2=0x7fae100b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:28.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.278+0000 7fae18000640 1 -- 172.21.15.161:0/1567759514 shutdown_connections 2024-05-22T21:25:28.280 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.278+0000 7fae18000640 1 -- 172.21.15.161:0/1567759514 wait complete. 2024-05-22T21:25:28.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.278+0000 7fae18000640 1 Processor -- start 2024-05-22T21:25:28.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae18000640 1 -- start start 2024-05-22T21:25:28.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae18000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae10140a70 0x7fae10136a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:28.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae18000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae10140ed0 0x7fae100b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:28.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae18000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae1015db70 0x7fae1013f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:28.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae18000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1013d730 con 0x7fae10140ed0 2024-05-22T21:25:28.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae18000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae100b4610 con 0x7fae10140a70 2024-05-22T21:25:28.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae18000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae10153b50 con 0x7fae1015db70 2024-05-22T21:25:28.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae14c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae10140ed0 0x7fae100b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:28.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae14c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae10140ed0 0x7fae100b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42182/0 (socket says 172.21.15.161:42182) 2024-05-22T21:25:28.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae14c00640 1 -- 172.21.15.161:0/1932414904 learned_addr learned my addr 172.21.15.161:0/1932414904 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:28.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae15600640 1 --2- 172.21.15.161:0/1932414904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae10140a70 0x7fae10136a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:28.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae16000640 1 --2- 172.21.15.161:0/1932414904 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae1015db70 0x7fae1013f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:28.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae14c00640 1 -- 172.21.15.161:0/1932414904 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae1015db70 msgr2=0x7fae1013f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:28.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae14c00640 1 --2- 172.21.15.161:0/1932414904 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae1015db70 0x7fae1013f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae14c00640 1 -- 172.21.15.161:0/1932414904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae10140a70 msgr2=0x7fae10136a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:28.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae14c00640 1 --2- 172.21.15.161:0/1932414904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae10140a70 0x7fae10136a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae14c00640 1 -- 172.21.15.161:0/1932414904 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc0094d0 con 0x7fae10140ed0 2024-05-22T21:25:28.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae16000640 1 --2- 172.21.15.161:0/1932414904 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae1015db70 0x7fae1013f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-22T21:25:28.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae14c00640 1 --2- 172.21.15.161:0/1932414904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae10140ed0 0x7fae100b3a60 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fadfc01aba0 tx=0x7fadfc01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:28.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae0d600640 1 -- 172.21.15.161:0/1932414904 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc0b3800 con 0x7fae10140ed0 2024-05-22T21:25:28.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae0d600640 1 -- 172.21.15.161:0/1932414904 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadfc0b3980 con 0x7fae10140ed0 2024-05-22T21:25:28.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae0d600640 1 -- 172.21.15.161:0/1932414904 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc0b3d30 con 0x7fae10140ed0 2024-05-22T21:25:28.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae18000640 1 -- 172.21.15.161:0/1932414904 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae10152fd0 con 0x7fae10140ed0 2024-05-22T21:25:28.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae18000640 1 -- 172.21.15.161:0/1932414904 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae10153590 con 0x7fae10140ed0 2024-05-22T21:25:28.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.282+0000 7fae18000640 1 -- 172.21.15.161:0/1932414904 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1013d8a0 con 0x7fae10140ed0 2024-05-22T21:25:28.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.286+0000 7fae0d600640 1 -- 172.21.15.161:0/1932414904 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fadfc0bd3f0 con 0x7fae10140ed0 2024-05-22T21:25:28.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.286+0000 7fae0d600640 1 --2- 172.21.15.161:0/1932414904 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fadf0066690 0x7fadf0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:28.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.286+0000 7fae0d600640 1 -- 172.21.15.161:0/1932414904 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fadfc11e320 con 0x7fae10140ed0 2024-05-22T21:25:28.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.286+0000 7fae15600640 1 --2- 172.21.15.161:0/1932414904 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fadf0066690 0x7fadf0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:28.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.286+0000 7fae15600640 1 --2- 172.21.15.161:0/1932414904 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fadf0066690 0x7fadf0068b50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fae0400a9a0 tx=0x7fae04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:28.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.290+0000 7fae0d600640 1 -- 172.21.15.161:0/1932414904 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7fadfc0ebac0 con 0x7fae10140ed0 2024-05-22T21:25:28.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.438+0000 7fae18000640 1 -- 172.21.15.161:0/1932414904 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fae1006ade0 con 0x7fadf0066690 2024-05-22T21:25:28.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.442+0000 7fae0d600640 1 -- 172.21.15.161:0/1932414904 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49820 (secure 0 0 0) 0x7fae0400a720 con 0x7fadf0066690 2024-05-22T21:25:28.442 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:28.443 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:25:28.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.442+0000 7fae18000640 1 -- 172.21.15.161:0/1932414904 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fadf0066690 msgr2=0x7fadf0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:28.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.442+0000 7fae18000640 1 --2- 172.21.15.161:0/1932414904 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fadf0066690 0x7fadf0068b50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fae0400a9a0 tx=0x7fae04008040 comp rx=0 tx=0).stop 2024-05-22T21:25:28.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.442+0000 7fae18000640 1 -- 172.21.15.161:0/1932414904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae10140ed0 msgr2=0x7fae100b3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:28.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.442+0000 7fae18000640 1 --2- 172.21.15.161:0/1932414904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae10140ed0 0x7fae100b3a60 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fadfc01aba0 tx=0x7fadfc01ad30 comp rx=0 tx=0).stop 2024-05-22T21:25:28.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.442+0000 7fae18000640 1 -- 172.21.15.161:0/1932414904 shutdown_connections 2024-05-22T21:25:28.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.442+0000 7fae18000640 1 --2- 172.21.15.161:0/1932414904 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7fadf0066690 0x7fadf0068b50 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.442+0000 7fae18000640 1 --2- 172.21.15.161:0/1932414904 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fae1015db70 0x7fae1013f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.442+0000 7fae18000640 1 --2- 172.21.15.161:0/1932414904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae10140ed0 0x7fae100b3a60 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.442+0000 7fae18000640 1 --2- 172.21.15.161:0/1932414904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae10140a70 0x7fae10136a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:28.444 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.442+0000 7fae18000640 1 -- 172.21.15.161:0/1932414904 >> 172.21.15.161:0/1932414904 conn(0x7fae100b5540 msgr2=0x7fae100d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:28.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.442+0000 7fae18000640 1 -- 172.21.15.161:0/1932414904 shutdown_connections 2024-05-22T21:25:28.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:28.442+0000 7fae18000640 1 -- 172.21.15.161:0/1932414904 wait complete. 2024-05-22T21:25:28.462 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":556,"stamp":"2024-05-22T21:25:26.561955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9840,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552659,"statfs":{"total":1600000000,"available":1589923811,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52489015,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-125,"num_write_kb":-85084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001442"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":346,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-22T21:25:18.708120+0000","last_change":"2024-05-22T21:13:14.029841+0000","last_active":"2024-05-22T21:25:18.708120+0000","last_peered":"2024-05-22T21:25:18.708120+0000","last_clean":"2024-05-22T21:25:18.708120+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:25:18.708120+0000","last_undegraded":"2024-05-22T21:25:18.708120+0000","last_fullsized":"2024-05-22T21:25:18.708120+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-22T21:13:12.978455+0000","last_clean_scrub_stamp":"2024-05-22T21:13:12.978455+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T09:09:31.750587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1662,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193650,"statfs":{"total":200000000,"available":198297623,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.70799999999999996,"15min":0.72399999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.36699999999999999},"max":{"1min":0.95099999999999996,"5min":1.022,"15min":2.0369999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72099999999999997,"15min":0.70199999999999996},"min":{"1min":0.52500000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97899999999999998,"5min":1.1759999999999999,"15min":1.889},"last":0.70799999999999996}]},{"osd":1,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72099999999999997,"15min":0.74199999999999999},"min":{"1min":0.48099999999999998,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.93300000000000005,"5min":1.0329999999999999,"15min":3.0920000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.73399999999999999},"min":{"1min":0.60199999999999998,"5min":0.45200000000000001,"15min":0.39800000000000002},"max":{"1min":0.94699999999999995,"5min":1.1100000000000001,"15min":3.1499999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.501,"5min":0.433,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":1.002,"15min":1.923},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.40999999999999998},"max":{"1min":0.96699999999999997,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.69499999999999995}]},{"osd":3,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72399999999999998,"15min":0.70999999999999996},"min":{"1min":0.54900000000000004,"5min":0.44700000000000001,"15min":0.40100000000000002},"max":{"1min":0.88100000000000001,"5min":1.026,"15min":1.026},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.76200000000000001,"15min":0.77200000000000002},"min":{"1min":0.64700000000000002,"5min":0.501,"15min":0.44500000000000001},"max":{"1min":0.99399999999999999,"5min":1.0740000000000001,"15min":2.1389999999999998},"last":0.498}]},{"osd":4,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.29399999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.96599999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.628,"15min":0.61899999999999999},"min":{"1min":0.504,"5min":0.40500000000000003,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.95999999999999996,"15min":1.018},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.51700000000000002,"5min":0.39300000000000002,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0289999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67300000000000004,"15min":0.65200000000000002},"min":{"1min":0.46300000000000002,"5min":0.42099999999999999,"15min":0.33400000000000002},"max":{"1min":0.91200000000000003,"5min":0.99299999999999999,"15min":0.996},"last":0.66700000000000004}]},{"osd":6,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.50600000000000001,"5min":0.5,"15min":0.36799999999999999},"max":{"1min":0.92500000000000004,"5min":1.052,"15min":1.052},"last":0.745},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69999999999999996,"15min":0.67000000000000004},"min":{"1min":0.60999999999999999,"5min":0.48699999999999999,"15min":0.41199999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.97699999999999998},"last":0.68100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69499999999999995,"15min":0.69399999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":1.0649999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.51900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95899999999999996,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.625}]},{"osd":1,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69399999999999995,"15min":0.69099999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89500000000000002,"5min":0.94299999999999995,"15min":1.107},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.70599999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":0.93500000000000005,"5min":1.0009999999999999,"15min":1.081},"last":0.56699999999999995}]},{"osd":2,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76000000000000001,"15min":0.76400000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.442},"max":{"1min":0.93000000000000005,"5min":1.026,"15min":1.087},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75700000000000001,"15min":0.746},"min":{"1min":0.56499999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0089999999999999,"5min":1.419,"15min":1.419},"last":0.55700000000000005}]},{"osd":3,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.749,"15min":0.746},"min":{"1min":0.46600000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98599999999999999,"5min":1.28,"15min":1.415},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.55400000000000005,"5min":0.48999999999999999,"15min":0.45400000000000001},"max":{"1min":0.878,"5min":0.93300000000000005,"15min":1.788},"last":0.52900000000000003}]},{"osd":4,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.374},"max":{"1min":0.89300000000000002,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.67500000000000004},"min":{"1min":0.45600000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.90900000000000003,"5min":1.1850000000000001,"15min":2.718},"last":0.45200000000000001}]},{"osd":5,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61499999999999999,"15min":0.67800000000000005},"min":{"1min":0.42399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84799999999999998,"5min":0.97499999999999998,"15min":8.5749999999999993},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67900000000000005,"15min":0.69999999999999996},"min":{"1min":0.49299999999999999,"5min":0.44800000000000001,"15min":0.42099999999999999},"max":{"1min":0.92000000000000004,"5min":0.95799999999999996,"15min":1.1140000000000001},"last":0.54200000000000004}]},{"osd":7,"last update":"Wed May 22 21:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92800000000000005,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69699999999999995,"15min":0.72699999999999998},"min":{"1min":0.49099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85499999999999998,"5min":0.89700000000000002,"15min":1.8009999999999999},"last":0.61099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.48099999999999998,"5min":0.38200000000000001,"15min":0.32400000000000001},"max":{"1min":0.85599999999999998,"5min":0.98199999999999998,"15min":1.0409999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.54700000000000004,"5min":0.48699999999999999,"15min":0.378},"max":{"1min":0.91800000000000004,"5min":0.93700000000000006,"15min":1.0089999999999999},"last":0.60399999999999998}]},{"osd":1,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71199999999999997,"15min":0.70499999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":1.0489999999999999,"15min":1.4339999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70999999999999996,"15min":0.69899999999999995},"min":{"1min":0.50700000000000001,"5min":0.45500000000000002,"15min":0.44},"max":{"1min":0.85499999999999998,"5min":0.92600000000000005,"15min":1.321},"last":0.66800000000000004}]},{"osd":2,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.70899999999999996,"15min":0.70999999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.33100000000000002},"max":{"1min":0.96199999999999997,"5min":1.03,"15min":1.2969999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.74399999999999999,"15min":0.70499999999999996},"min":{"1min":0.56799999999999995,"5min":0.50700000000000001,"15min":0.42499999999999999},"max":{"1min":0.95999999999999996,"5min":0.98399999999999999,"15min":1.4910000000000001},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73999999999999999,"15min":0.71899999999999997},"min":{"1min":0.54600000000000004,"5min":0.504,"15min":0.47999999999999998},"max":{"1min":0.92100000000000004,"5min":1.004,"15min":1.1879999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.749,"15min":0.73799999999999999},"min":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.996,"5min":0.996,"15min":1.111},"last":0.81999999999999995}]},{"osd":4,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59499999999999997,"15min":0.60399999999999998},"min":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.35099999999999998},"max":{"1min":0.74199999999999999,"5min":0.92400000000000004,"15min":1.272},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.627,"15min":0.628},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84999999999999998,"5min":0.95499999999999996,"15min":1.038},"last":0.71999999999999997}]},{"osd":6,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.371},"max":{"1min":0.872,"5min":0.96099999999999997,"15min":0.98899999999999999},"last":0.745},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.48199999999999998,"5min":0.443,"15min":0.41099999999999998},"max":{"1min":0.94799999999999995,"5min":1.0069999999999999,"15min":1.2290000000000001},"last":0.76100000000000001}]},{"osd":7,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72599999999999998,"15min":0.71399999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.47399999999999998},"max":{"1min":0.98299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76400000000000001,"15min":0.76800000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.46200000000000002},"max":{"1min":0.96799999999999997,"5min":1.018,"15min":1.2909999999999999},"last":0.93200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71499999999999997,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30399999999999999},"max":{"1min":0.97299999999999998,"5min":0.98899999999999999,"15min":6.6539999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.52100000000000002,"5min":0.46000000000000002,"15min":0.436},"max":{"1min":0.872,"5min":0.96299999999999997,"15min":5.4370000000000003},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.39200000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91400000000000003,"5min":0.95799999999999996,"15min":1.7749999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69699999999999995,"15min":0.70899999999999996},"min":{"1min":0.48299999999999998,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":2.2210000000000001},"last":0.71699999999999997}]},{"osd":2,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72299999999999998,"15min":0.71199999999999997},"min":{"1min":0.50900000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94599999999999995,"5min":0.97299999999999998,"15min":2.4660000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.54400000000000004,"5min":0.45600000000000002,"15min":0.378},"max":{"1min":0.92900000000000005,"5min":1.01,"15min":2.2530000000000001},"last":0.68200000000000005}]},{"osd":3,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71299999999999997,"15min":0.70199999999999996},"min":{"1min":0.54200000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.998,"5min":1.042,"15min":1.857},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77100000000000002,"15min":0.78200000000000003},"min":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.47499999999999998},"max":{"1min":0.96599999999999997,"5min":1.012,"15min":1.897},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.629},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.379},"max":{"1min":0.95199999999999996,"5min":1.0169999999999999,"15min":1.1850000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.35599999999999998},"max":{"1min":0.872,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.55000000000000004}]},{"osd":6,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85599999999999998,"5min":1.044,"15min":1.0609999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.44,"5min":0.44,"15min":0.434},"max":{"1min":0.996,"5min":1.095,"15min":1.1419999999999999},"last":0.74099999999999999}]},{"osd":7,"last update":"Wed May 22 21:24:33 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.68500000000000005},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.40799999999999997},"max":{"1min":0.84799999999999998,"5min":1.1579999999999999,"15min":1.7390000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70999999999999996,"15min":0.746},"min":{"1min":0.505,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86499999999999999,"5min":1.069,"15min":1.4510000000000001},"last":0.80900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.42799999999999999,"15min":0.41999999999999998},"min":{"1min":0.312,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.59399999999999997,"5min":0.73099999999999998,"15min":0.86799999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.47199999999999998,"15min":0.438},"min":{"1min":0.33600000000000002,"5min":0.30399999999999999,"15min":0.28399999999999997},"max":{"1min":0.65300000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54100000000000004}]},{"osd":1,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48199999999999998,"15min":0.46400000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.26500000000000001},"max":{"1min":0.69799999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.443},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.52900000000000003},"min":{"1min":0.40699999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78700000000000003,"5min":0.80600000000000005,"15min":0.90000000000000002},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55100000000000005,"15min":0.51600000000000001},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89500000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.50800000000000001},"min":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.33600000000000002},"max":{"1min":0.69599999999999995,"5min":0.82799999999999996,"15min":0.93400000000000005},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":0.71999999999999997},"min":{"1min":0.55900000000000005,"5min":0.48299999999999998,"15min":0.39400000000000002},"max":{"1min":1.0389999999999999,"5min":1.1839999999999999,"15min":1.2050000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.75700000000000001,"15min":0.73599999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.435},"max":{"1min":1,"5min":1.093,"15min":1.5760000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.45100000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.234},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.73899999999999999,"15min":0.72399999999999998},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.3049999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73199999999999998,"15min":0.72699999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.44900000000000001},"max":{"1min":1.0149999999999999,"5min":1.0409999999999999,"15min":1.276},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76900000000000002,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.53400000000000003,"15min":0.50600000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.2869999999999999},"last":0.84099999999999997}]},{"osd":7,"last update":"Wed May 22 21:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79000000000000004,"15min":0.77400000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.495},"max":{"1min":1.071,"5min":1.0880000000000001,"15min":1.2709999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.76200000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0169999999999999,"5min":1.133,"15min":1.2969999999999999},"last":0.81599999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.434,"15min":0.42299999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.23599999999999999},"max":{"1min":0.59899999999999998,"5min":0.66500000000000004,"15min":2.3090000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.28199999999999997,"5min":0.26900000000000002,"15min":0.23400000000000001},"max":{"1min":0.64100000000000001,"5min":0.80400000000000005,"15min":2.532},"last":0.44900000000000001}]},{"osd":1,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.44700000000000001,"15min":0.45600000000000002},"min":{"1min":0.30599999999999999,"5min":0.29899999999999999,"15min":0.29399999999999998},"max":{"1min":0.56899999999999995,"5min":0.68899999999999995,"15min":2.948},"last":0.498},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.64800000000000002,"5min":0.71799999999999997,"15min":2.8159999999999998},"last":0.53700000000000003}]},{"osd":3,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.32700000000000001},"max":{"1min":0.62,"5min":0.83499999999999996,"15min":2.75},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.5,"15min":0.48899999999999999},"min":{"1min":0.36099999999999999,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":3.0619999999999998},"last":0.51900000000000002}]},{"osd":4,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78000000000000003,"15min":0.74199999999999999},"min":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.52700000000000002},"max":{"1min":1.022,"5min":1.103,"15min":2.7610000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.78100000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.375},"max":{"1min":0.97499999999999998,"5min":1.208,"15min":2.9279999999999999},"last":0.76400000000000001}]},{"osd":5,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.753,"15min":0.73799999999999999},"min":{"1min":0.56799999999999995,"5min":0.51900000000000002,"15min":0.47299999999999998},"max":{"1min":0.89500000000000002,"5min":1.133,"15min":2.9790000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.45900000000000002},"max":{"1min":0.99099999999999999,"5min":1.0549999999999999,"15min":2.7069999999999999},"last":0.71299999999999997}]},{"osd":6,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.73699999999999999},"min":{"1min":0.54600000000000004,"5min":0.51300000000000001,"15min":0.495},"max":{"1min":0.90000000000000002,"5min":1.0649999999999999,"15min":2.8580000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.752,"5min":0.748,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.496,"15min":0.46500000000000002},"max":{"1min":1.048,"5min":1.147,"15min":2.6880000000000002},"last":0.69499999999999995}]},{"osd":7,"last update":"Wed May 22 21:24:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.79700000000000004,"15min":0.77000000000000002},"min":{"1min":0.67000000000000004,"5min":0.50700000000000001,"15min":0.438},"max":{"1min":1.004,"5min":1.2210000000000001,"15min":3.0099999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81799999999999995,"15min":0.80100000000000005},"min":{"1min":0.67000000000000004,"5min":0.57299999999999995,"15min":0.499},"max":{"1min":1.006,"5min":1.1339999999999999,"15min":2.827},"last":0.77300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.438,"15min":0.433},"min":{"1min":0.25900000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.65200000000000002,"5min":0.73499999999999999,"15min":1.3879999999999999},"last":0.436},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.47299999999999998,"15min":0.45600000000000002},"min":{"1min":0.28199999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.626,"5min":0.879,"15min":0.879},"last":0.60499999999999998}]},{"osd":2,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.502,"15min":0.5},"min":{"1min":0.38200000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":1.76},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.495,"5min":0.48699999999999999,"15min":0.50600000000000001},"min":{"1min":0.36799999999999999,"5min":0.29799999999999999,"15min":0.28999999999999998},"max":{"1min":0.68700000000000006,"5min":0.83599999999999997,"15min":1.923},"last":0.56699999999999995}]},{"osd":3,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.41199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82899999999999996,"5min":0.82999999999999996,"15min":2.1030000000000002},"last":0.622},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52600000000000002,"15min":0.50600000000000001},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.48999999999999999}]},{"osd":4,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.72199999999999998,"15min":0.74399999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.45100000000000001},"max":{"1min":0.88800000000000001,"5min":1.0780000000000001,"15min":2.1360000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.50800000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.022,"5min":1.1519999999999999,"15min":1.7829999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":1.0549999999999999,"15min":1.819},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.746,"15min":0.748},"min":{"1min":0.61799999999999999,"5min":0.48799999999999999,"15min":0.40100000000000002},"max":{"1min":0.97899999999999998,"5min":1.0469999999999999,"15min":1.97},"last":0.70499999999999996}]},{"osd":6,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.53300000000000003,"5min":0.52000000000000002,"15min":0.47299999999999998},"max":{"1min":0.93100000000000005,"5min":1.1359999999999999,"15min":1.272},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79000000000000004,"15min":0.77700000000000002},"min":{"1min":0.61099999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95399999999999996,"5min":1.1259999999999999,"15min":1.8029999999999999},"last":0.67100000000000004}]},{"osd":7,"last update":"Wed May 22 21:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.55400000000000005,"5min":0.47999999999999998,"15min":0.42099999999999999},"max":{"1min":0.94699999999999995,"5min":0.995,"15min":2.0310000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76000000000000001,"15min":0.76500000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.52000000000000002},"max":{"1min":1.004,"5min":1.0660000000000001,"15min":2.0659999999999998},"last":0.64600000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1661,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193650,"statfs":{"total":200000000,"available":198298242,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.43099999999999999},"min":{"1min":0.27000000000000002,"5min":0.24199999999999999,"15min":0.22800000000000001},"max":{"1min":0.63400000000000001,"5min":0.69999999999999996,"15min":0.88300000000000001},"last":0.432},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.50800000000000001},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.33100000000000002},"max":{"1min":0.754,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.50700000000000001}]},{"osd":2,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.48399999999999999},"min":{"1min":0.33100000000000002,"5min":0.32100000000000001,"15min":0.29399999999999998},"max":{"1min":0.70299999999999996,"5min":0.85899999999999999,"15min":0.93799999999999994},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.495,"5min":0.46999999999999997,"15min":0.44900000000000001},"min":{"1min":0.39400000000000002,"5min":0.28899999999999998,"15min":0.27900000000000003},"max":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.92300000000000004},"last":0.40799999999999997}]},{"osd":3,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47599999999999998,"15min":0.45400000000000001},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":1.0760000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55700000000000005,"15min":0.53200000000000003},"min":{"1min":0.41699999999999998,"5min":0.38900000000000001,"15min":0.36899999999999999},"max":{"1min":0.79800000000000004,"5min":0.89700000000000002,"15min":0.98299999999999998},"last":0.47099999999999997}]},{"osd":4,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.72799999999999998},"min":{"1min":0.53200000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0189999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.748},"min":{"1min":0.50600000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.008,"5min":1.2330000000000001,"15min":1.2709999999999999},"last":0.84499999999999997}]},{"osd":5,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.72999999999999998},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":1.135,"15min":1.135},"last":0.873},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.54200000000000004,"5min":0.499,"15min":0.47299999999999998},"max":{"1min":0.95099999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.86199999999999999}]},{"osd":6,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81799999999999995,"15min":0.79300000000000004},"min":{"1min":0.61399999999999999,"5min":0.54400000000000004,"15min":0.51500000000000001},"max":{"1min":0.97299999999999998,"5min":1.175,"15min":1.2090000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80100000000000005,"15min":0.78300000000000003},"min":{"1min":0.60199999999999998,"5min":0.50800000000000001,"15min":0.502},"max":{"1min":0.99099999999999999,"5min":1.3149999999999999,"15min":1.5},"last":0.78600000000000003}]},{"osd":7,"last update":"Wed May 22 21:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.78200000000000003,"15min":0.753},"min":{"1min":0.53200000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.034,"5min":1.214,"15min":1.214},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.64100000000000001,"5min":0.52800000000000002,"15min":0.497},"max":{"1min":0.98599999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.82799999999999996}]}]}],"pool_statfs":[]}} 2024-05-22T21:25:28.467 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-22T21:13:12.978455+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=22, tm_hour=21, tm_min=13, tm_sec=12, tm_wday=2, tm_yday=143, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=22, tm_hour=21, tm_min=25, tm_sec=22, tm_wday=2, tm_yday=143, tm_isdst=0) 2024-05-22T21:25:28.467 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-22T21:25:48.468 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-22T21:25:48.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.614+0000 7f469a400640 1 Processor -- start 2024-05-22T21:25:48.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.614+0000 7f469a400640 1 -- start start 2024-05-22T21:25:48.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.618+0000 7f469a400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f469415dd50 0x7f469406d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:48.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.618+0000 7f469a400640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f469415e130 0x7f469415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:48.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.618+0000 7f469a400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4694071670 con 0x7f469415d410 2024-05-22T21:25:48.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.618+0000 7f469a400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469415d8a0 con 0x7f469415dd50 2024-05-22T21:25:48.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.618+0000 7f469a400640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469406de40 con 0x7f469415e130 2024-05-22T21:25:48.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.618+0000 7f4693400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f469415dd50 0x7f469406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:48.618 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.618+0000 7f4693400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f469415dd50 0x7f469406d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55408/0 (socket says 172.21.15.161:55408) 2024-05-22T21:25:48.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.618+0000 7f4693e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f469415d410 0x7f469415a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59008/0 (socket says 172.21.15.161:59008) 2024-05-22T21:25:48.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.618+0000 7f4693400640 1 -- 172.21.15.161:0/860851018 learned_addr learned my addr 172.21.15.161:0/860851018 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:48.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.618+0000 7f468be00640 1 --2- 172.21.15.161:0/860851018 >> v2:172.21.15.179:3300/0 conn(0x7f469415e130 0x7f469415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:48.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.618+0000 7f4693400640 1 -- 172.21.15.161:0/860851018 >> v2:172.21.15.179:3300/0 conn(0x7f469415e130 msgr2=0x7f469415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:48.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.618+0000 7f4693400640 1 --2- 172.21.15.161:0/860851018 >> v2:172.21.15.179:3300/0 conn(0x7f469415e130 0x7f469415ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:48.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.618+0000 7f4693400640 1 -- 172.21.15.161:0/860851018 >> v1:172.21.15.161:6789/0 conn(0x7f469415d410 legacy=0x7f469415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:48.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.618+0000 7f4693400640 1 -- 172.21.15.161:0/860851018 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4694070a60 con 0x7f469415dd50 2024-05-22T21:25:48.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.622+0000 7f4693400640 1 --2- 172.21.15.161:0/860851018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f469415dd50 0x7f469406d900 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f469406c7c0 tx=0x7f468401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72628c1d7e68bab server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:48.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.622+0000 7f4692a00640 1 -- 172.21.15.161:0/860851018 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840aa650 con 0x7f469415dd50 2024-05-22T21:25:48.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.622+0000 7f4692a00640 1 -- 172.21.15.161:0/860851018 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840aa7b0 con 0x7f469415dd50 2024-05-22T21:25:48.621 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.622+0000 7f4692a00640 1 -- 172.21.15.161:0/860851018 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840aab40 con 0x7f469415dd50 2024-05-22T21:25:48.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.622+0000 7f469a400640 1 -- 172.21.15.161:0/860851018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f469415dd50 msgr2=0x7f469406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:48.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.622+0000 7f469a400640 1 --2- 172.21.15.161:0/860851018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f469415dd50 0x7f469406d900 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f469406c7c0 tx=0x7f468401c4f0 comp rx=0 tx=0).stop 2024-05-22T21:25:48.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.622+0000 7f469a400640 1 -- 172.21.15.161:0/860851018 shutdown_connections 2024-05-22T21:25:48.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.622+0000 7f469a400640 1 --2- 172.21.15.161:0/860851018 >> v2:172.21.15.179:3300/0 conn(0x7f469415e130 0x7f469415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:48.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.622+0000 7f469a400640 1 --2- 172.21.15.161:0/860851018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f469415dd50 0x7f469406d900 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:48.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.622+0000 7f469a400640 1 -- 172.21.15.161:0/860851018 >> 172.21.15.161:0/860851018 conn(0x7f46940b0bc0 msgr2=0x7f46940b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:48.622 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.622+0000 7f469a400640 1 -- 172.21.15.161:0/860851018 shutdown_connections 2024-05-22T21:25:48.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.622+0000 7f469a400640 1 -- 172.21.15.161:0/860851018 wait complete. 2024-05-22T21:25:48.623 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.622+0000 7f469a400640 1 Processor -- start 2024-05-22T21:25:48.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.622+0000 7f469a400640 1 -- start start 2024-05-22T21:25:48.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f469a400640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f469415d410 0x7f46940cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:48.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f469a400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f469415dd50 0x7f46941f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:48.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f469a400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469415e130 0x7f46940b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:48.625 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f469a400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46940b3000 con 0x7f469415dd50 2024-05-22T21:25:48.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f469a400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46940b23f0 con 0x7f469415e130 2024-05-22T21:25:48.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f469a400640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46940b26d0 con 0x7f469415d410 2024-05-22T21:25:48.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f468be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f469415dd50 0x7f46941f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:48.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f468be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f469415dd50 0x7f46941f5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55418/0 (socket says 172.21.15.161:55418) 2024-05-22T21:25:48.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f468be00640 1 -- 172.21.15.161:0/2805905799 learned_addr learned my addr 172.21.15.161:0/2805905799 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:48.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f4693e00640 1 --2- 172.21.15.161:0/2805905799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469415e130 0x7f46940b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:48.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f4693400640 1 --2- 172.21.15.161:0/2805905799 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f469415d410 0x7f46940cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:48.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f468be00640 1 -- 172.21.15.161:0/2805905799 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f469415d410 msgr2=0x7f46940cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:48.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f468be00640 1 --2- 172.21.15.161:0/2805905799 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f469415d410 0x7f46940cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:48.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f468be00640 1 -- 172.21.15.161:0/2805905799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469415e130 msgr2=0x7f46940b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:48.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f468be00640 1 --2- 172.21.15.161:0/2805905799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469415e130 0x7f46940b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:48.626 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f468be00640 1 -- 172.21.15.161:0/2805905799 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46840094d0 con 0x7f469415dd50 2024-05-22T21:25:48.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f4693e00640 1 --2- 172.21.15.161:0/2805905799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469415e130 0x7f46940b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-22T21:25:48.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f468be00640 1 --2- 172.21.15.161:0/2805905799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f469415dd50 0x7f46941f5900 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f467c00d990 tx=0x7f467c00de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:48.627 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f468b400640 1 -- 172.21.15.161:0/2805905799 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c007a90 con 0x7f469415dd50 2024-05-22T21:25:48.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f468b400640 1 -- 172.21.15.161:0/2805905799 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c007bf0 con 0x7f469415dd50 2024-05-22T21:25:48.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f469a400640 1 -- 172.21.15.161:0/2805905799 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46940b29b0 con 0x7f469415dd50 2024-05-22T21:25:48.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f468b400640 1 -- 172.21.15.161:0/2805905799 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c010470 con 0x7f469415dd50 2024-05-22T21:25:48.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f469a400640 1 -- 172.21.15.161:0/2805905799 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46940cdba0 con 0x7f469415dd50 2024-05-22T21:25:48.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f469a400640 1 -- 172.21.15.161:0/2805905799 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4694067220 con 0x7f469415dd50 2024-05-22T21:25:48.628 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.626+0000 7f468b400640 1 -- 172.21.15.161:0/2805905799 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f467c0105d0 con 0x7f469415dd50 2024-05-22T21:25:48.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.630+0000 7f468b400640 1 --2- 172.21.15.161:0/2805905799 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4670066690 0x7f4670068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:48.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.630+0000 7f468b400640 1 -- 172.21.15.161:0/2805905799 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f467c084220 con 0x7f469415dd50 2024-05-22T21:25:48.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.630+0000 7f4693400640 1 --2- 172.21.15.161:0/2805905799 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4670066690 0x7f4670068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:48.629 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.630+0000 7f4693400640 1 --2- 172.21.15.161:0/2805905799 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4670066690 0x7f4670068b50 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f46840ae000 tx=0x7f46840a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:48.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.634+0000 7f468b400640 1 -- 172.21.15.161:0/2805905799 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f467c051880 con 0x7f469415dd50 2024-05-22T21:25:48.784 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.782+0000 7f469a400640 1 -- 172.21.15.161:0/2805905799 --> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4694070730 con 0x7f4670066690 2024-05-22T21:25:48.786 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.786+0000 7f468b400640 1 -- 172.21.15.161:0/2805905799 <== mgr.4107 v2:172.21.15.161:6832/23334 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49763 (secure 0 0 0) 0x7f46840aa920 con 0x7f4670066690 2024-05-22T21:25:48.787 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:48.788 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-05-22T21:25:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.786+0000 7f469a400640 1 -- 172.21.15.161:0/2805905799 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4670066690 msgr2=0x7f4670068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.786+0000 7f469a400640 1 --2- 172.21.15.161:0/2805905799 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4670066690 0x7f4670068b50 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f46840ae000 tx=0x7f46840a7000 comp rx=0 tx=0).stop 2024-05-22T21:25:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.786+0000 7f469a400640 1 -- 172.21.15.161:0/2805905799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f469415dd50 msgr2=0x7f46941f5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.786+0000 7f469a400640 1 --2- 172.21.15.161:0/2805905799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f469415dd50 0x7f46941f5900 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f467c00d990 tx=0x7f467c00de50 comp rx=0 tx=0).stop 2024-05-22T21:25:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.786+0000 7f469a400640 1 -- 172.21.15.161:0/2805905799 shutdown_connections 2024-05-22T21:25:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.786+0000 7f469a400640 1 --2- 172.21.15.161:0/2805905799 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f4670066690 0x7f4670068b50 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.786+0000 7f469a400640 1 --2- 172.21.15.161:0/2805905799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f469415e130 0x7f46940b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.786+0000 7f469a400640 1 --2- 172.21.15.161:0/2805905799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f469415dd50 0x7f46941f5900 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.786+0000 7f469a400640 1 --2- 172.21.15.161:0/2805905799 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f469415d410 0x7f46940cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:48.789 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.790+0000 7f469a400640 1 -- 172.21.15.161:0/2805905799 >> 172.21.15.161:0/2805905799 conn(0x7f46940b0bc0 msgr2=0x7f4694153650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:48.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.790+0000 7f469a400640 1 -- 172.21.15.161:0/2805905799 shutdown_connections 2024-05-22T21:25:48.790 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.790+0000 7f469a400640 1 -- 172.21.15.161:0/2805905799 wait complete. 2024-05-22T21:25:48.808 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":567,"stamp":"2024-05-22T21:25:48.566718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9840,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552659,"statfs":{"total":1600000000,"available":1589923811,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002418"},"pg_stats":[{"pgid":"1.0","version":"197'119","reported_seq":360,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-05-22T21:25:28.269766+0000","last_change":"2024-05-22T21:25:28.269857+0000","last_active":"2024-05-22T21:25:28.269766+0000","last_peered":"2024-05-22T21:25:28.269766+0000","last_clean":"2024-05-22T21:25:28.269766+0000","last_became_active":"2024-05-22T21:13:14.029694+0000","last_became_peered":"2024-05-22T21:13:14.029694+0000","last_unstale":"2024-05-22T21:25:28.269766+0000","last_undegraded":"2024-05-22T21:25:28.269766+0000","last_fullsized":"2024-05-22T21:25:28.269766+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"197'119","last_scrub_stamp":"2024-05-22T21:25:28.269726+0000","last_deep_scrub":"197'119","last_deep_scrub_stamp":"2024-05-22T21:25:28.269726+0000","last_clean_scrub_stamp":"2024-05-22T21:25:28.269726+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T02:00:45.693822+0000","scrub_duration":0.0028609109999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1662,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193650,"statfs":{"total":200000000,"available":198297623,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:25:35 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.70299999999999996,"15min":0.71499999999999997},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.36699999999999999},"max":{"1min":0.91200000000000003,"5min":1.022,"15min":2.0369999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.71099999999999997,"15min":0.70099999999999996},"min":{"1min":0.441,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.87,"5min":1.1759999999999999,"15min":1.889},"last":0.78800000000000003}]},{"osd":1,"last update":"Wed May 22 21:25:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71099999999999997,"15min":0.73399999999999999},"min":{"1min":0.505,"5min":0.434,"15min":0.41699999999999998},"max":{"1min":0.86699999999999999,"5min":1.008,"15min":3.0920000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.71099999999999997,"15min":0.72699999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.39800000000000002},"max":{"1min":0.81499999999999995,"5min":1.1100000000000001,"15min":3.1499999999999999},"last":0.75600000000000001}]},{"osd":2,"last update":"Wed May 22 21:25:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70699999999999996,"15min":0.69999999999999996},"min":{"1min":0.504,"5min":0.433,"15min":0.38800000000000001},"max":{"1min":0.84699999999999998,"5min":1.002,"15min":1.923},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.54600000000000004,"5min":0.46800000000000003,"15min":0.40999999999999998},"max":{"1min":0.83699999999999997,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.68600000000000005}]},{"osd":3,"last update":"Wed May 22 21:25:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71899999999999997,"15min":0.71099999999999997},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.40100000000000002},"max":{"1min":0.90000000000000002,"5min":1.026,"15min":1.026},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.749,"15min":0.76300000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.44500000000000001},"max":{"1min":0.84199999999999997,"5min":1.0740000000000001,"15min":2.1389999999999998},"last":0.79500000000000004}]},{"osd":4,"last update":"Wed May 22 21:25:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.39300000000000002,"5min":0.36199999999999999,"15min":0.29399999999999998},"max":{"1min":0.89000000000000001,"5min":0.91600000000000004,"15min":0.96599999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.627,"15min":0.621},"min":{"1min":0.501,"5min":0.40600000000000003,"15min":0.36099999999999999},"max":{"1min":0.90100000000000002,"5min":0.95999999999999996,"15min":1.018},"last":0.81200000000000006}]},{"osd":5,"last update":"Wed May 22 21:25:35 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.47799999999999998,"5min":0.39300000000000002,"15min":0.371},"max":{"1min":0.95199999999999996,"5min":0.97899999999999998,"15min":1.0289999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.441,"5min":0.42099999999999999,"15min":0.33400000000000002},"max":{"1min":0.93600000000000005,"5min":0.99299999999999999,"15min":0.996},"last":0.72299999999999998}]},{"osd":6,"last update":"Wed May 22 21:25:35 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.57399999999999995,"5min":0.5,"15min":0.36799999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":1.052},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70299999999999996,"15min":0.67500000000000004},"min":{"1min":0.53600000000000003,"5min":0.48699999999999999,"15min":0.41199999999999998},"max":{"1min":0.89800000000000002,"5min":0.95899999999999996,"15min":0.97699999999999998},"last":0.84199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.67200000000000004,"15min":0.68899999999999995},"min":{"1min":0.35299999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82899999999999996,"5min":0.97399999999999998,"15min":1.0649999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.69499999999999995,"15min":0.69599999999999995},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86399999999999999,"5min":0.95899999999999996,"15min":1.0349999999999999},"last":0.83399999999999996}]},{"osd":1,"last update":"Wed May 22 21:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.67500000000000004,"15min":0.68700000000000006},"min":{"1min":0.42599999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82399999999999995,"5min":0.94299999999999995,"15min":1.107},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.67200000000000004,"15min":0.69899999999999995},"min":{"1min":0.39200000000000002,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":0.82799999999999996,"5min":1.0009999999999999,"15min":1.081},"last":0.80800000000000005}]},{"osd":2,"last update":"Wed May 22 21:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.73799999999999999,"15min":0.75900000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.442},"max":{"1min":0.876,"5min":1.026,"15min":1.087},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.45800000000000002},"max":{"1min":0.81999999999999995,"5min":1.0089999999999999,"15min":1.419},"last":0.79400000000000004}]},{"osd":3,"last update":"Wed May 22 21:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72999999999999998,"15min":0.74199999999999999},"min":{"1min":0.52400000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.84599999999999997,"5min":0.98599999999999999,"15min":1.415},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.70299999999999996,"15min":0.71599999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.45400000000000001},"max":{"1min":0.84299999999999997,"5min":0.90900000000000003,"15min":1.788},"last":0.92900000000000005}]},{"osd":4,"last update":"Wed May 22 21:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.622,"15min":0.63900000000000001},"min":{"1min":0.41399999999999998,"5min":0.40500000000000003,"15min":0.374},"max":{"1min":0.85599999999999998,"5min":0.93700000000000006,"15min":1.5720000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.65500000000000003,"15min":0.67100000000000004},"min":{"1min":0.434,"5min":0.375,"15min":0.375},"max":{"1min":0.88700000000000001,"5min":1.014,"15min":2.718},"last":0.59199999999999997}]},{"osd":5,"last update":"Wed May 22 21:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61499999999999999,"15min":0.67300000000000004},"min":{"1min":0.47999999999999998,"5min":0.41399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77500000000000002,"5min":0.97499999999999998,"15min":8.5749999999999993},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.69699999999999995},"min":{"1min":0.496,"5min":0.46500000000000002,"15min":0.42099999999999999},"max":{"1min":0.85799999999999998,"5min":0.95799999999999996,"15min":1.1140000000000001},"last":0.78100000000000003}]},{"osd":7,"last update":"Wed May 22 21:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70299999999999996,"15min":0.71999999999999997},"min":{"1min":0.47399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.87,"5min":1.0329999999999999,"15min":1.5529999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69399999999999995,"15min":0.71999999999999997},"min":{"1min":0.53400000000000003,"5min":0.49099999999999999,"15min":0.39600000000000002},"max":{"1min":0.88200000000000001,"5min":0.89700000000000002,"15min":1.8009999999999999},"last":0.71199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:25:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.32400000000000001},"max":{"1min":0.85699999999999998,"5min":0.98199999999999998,"15min":1.0409999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.69999999999999996,"15min":0.68500000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84799999999999998,"5min":0.93700000000000006,"15min":1.0089999999999999},"last":0.83999999999999997}]},{"osd":1,"last update":"Wed May 22 21:25:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.372},"max":{"1min":0.96099999999999997,"5min":1.0489999999999999,"15min":1.4339999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.71699999999999997,"15min":0.69799999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.321},"last":0.77100000000000002}]},{"osd":2,"last update":"Wed May 22 21:25:41 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.72399999999999998,"15min":0.71199999999999997},"min":{"1min":0.57099999999999995,"5min":0.437,"15min":0.33100000000000002},"max":{"1min":0.92900000000000005,"5min":0.996,"15min":1.2969999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.746,"15min":0.70699999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.42499999999999999},"max":{"1min":0.90900000000000003,"5min":0.98399999999999999,"15min":1.4910000000000001},"last":0.73899999999999999}]},{"osd":3,"last update":"Wed May 22 21:25:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.747,"15min":0.71999999999999997},"min":{"1min":0.56100000000000005,"5min":0.504,"15min":0.47999999999999998},"max":{"1min":0.89300000000000002,"5min":1.004,"15min":1.1879999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.747,"15min":0.73599999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.48299999999999998},"max":{"1min":0.996,"5min":0.996,"15min":1.111},"last":0.68700000000000006}]},{"osd":4,"last update":"Wed May 22 21:25:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.35099999999999998},"max":{"1min":0.84499999999999997,"5min":0.86899999999999999,"15min":1.272},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60699999999999998,"15min":0.623},"min":{"1min":0.44700000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79600000000000004,"5min":0.94299999999999995,"15min":1.038},"last":0.90200000000000002}]},{"osd":6,"last update":"Wed May 22 21:25:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.61199999999999999,"15min":0.627},"min":{"1min":0.45000000000000001,"5min":0.42899999999999999,"15min":0.371},"max":{"1min":0.745,"5min":0.96099999999999997,"15min":0.98899999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.46800000000000003,"5min":0.443,"15min":0.41099999999999998},"max":{"1min":0.76800000000000002,"5min":0.94899999999999995,"15min":1.2290000000000001},"last":0.82599999999999996}]},{"osd":7,"last update":"Wed May 22 21:25:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70799999999999996,"15min":0.71199999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.47399999999999998},"max":{"1min":0.91700000000000004,"5min":0.98299999999999998,"15min":1.0800000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.74099999999999999,"15min":0.76100000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.46200000000000002},"max":{"1min":0.93200000000000005,"5min":1.018,"15min":1.2909999999999999},"last":0.875}]}]},{"osd":4,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:25:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71599999999999997,"15min":0.74299999999999999},"min":{"1min":0.51000000000000001,"5min":0.30499999999999999,"15min":0.30399999999999999},"max":{"1min":0.92800000000000005,"5min":0.97299999999999998,"15min":6.6539999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.55800000000000005,"5min":0.46000000000000002,"15min":0.436},"max":{"1min":0.88300000000000001,"5min":0.96299999999999997,"15min":5.4370000000000003},"last":0.88500000000000001}]},{"osd":1,"last update":"Wed May 22 21:25:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.56799999999999995,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.7749999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.53500000000000003,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":2.2210000000000001},"last":0.77100000000000002}]},{"osd":2,"last update":"Wed May 22 21:25:36 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73399999999999999,"15min":0.71699999999999997},"min":{"1min":0.61299999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90600000000000003,"5min":0.97299999999999998,"15min":2.4660000000000002},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72099999999999997,"15min":0.71399999999999997},"min":{"1min":0.59799999999999998,"5min":0.45600000000000002,"15min":0.378},"max":{"1min":0.92900000000000005,"5min":1.01,"15min":2.2530000000000001},"last":0.73399999999999999}]},{"osd":3,"last update":"Wed May 22 21:25:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.55400000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86699999999999999,"5min":0.998,"15min":1.857},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.78000000000000003,"15min":0.78200000000000003},"min":{"1min":0.60299999999999998,"5min":0.47699999999999998,"15min":0.47499999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.897},"last":0.92400000000000004}]},{"osd":5,"last update":"Wed May 22 21:25:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.62,"15min":0.629},"min":{"1min":0.51400000000000001,"5min":0.39400000000000002,"15min":0.379},"max":{"1min":0.877,"5min":0.95199999999999996,"15min":1.1850000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.48099999999999998,"5min":0.38,"15min":0.35599999999999998},"max":{"1min":0.82499999999999996,"5min":0.90600000000000003,"15min":1.1180000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Wed May 22 21:25:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90300000000000002,"5min":0.94599999999999995,"15min":1.0609999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.67500000000000004,"15min":0.68899999999999995},"min":{"1min":0.55700000000000005,"5min":0.44,"15min":0.434},"max":{"1min":0.80900000000000005,"5min":0.996,"15min":1.1419999999999999},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed May 22 21:25:36 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66900000000000004,"15min":0.68600000000000005},"min":{"1min":0.58599999999999997,"5min":0.42699999999999999,"15min":0.40799999999999997},"max":{"1min":0.90600000000000003,"5min":1.02,"15min":1.7390000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.71299999999999997,"15min":0.74299999999999999},"min":{"1min":0.65800000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94799999999999995,"5min":0.97399999999999998,"15min":1.4510000000000001},"last":0.72299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42699999999999999,"15min":0.41999999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.23200000000000001},"max":{"1min":0.55900000000000005,"5min":0.73099999999999998,"15min":0.86799999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.47899999999999998,"15min":0.442},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.61599999999999999,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54300000000000004}]},{"osd":1,"last update":"Wed May 22 21:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.48099999999999998,"15min":0.46400000000000002},"min":{"1min":0.32000000000000001,"5min":0.31900000000000001,"15min":0.26500000000000001},"max":{"1min":0.69299999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55800000000000005,"15min":0.53100000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76000000000000001,"5min":0.80600000000000005,"15min":0.90000000000000002},"last":0.52600000000000002}]},{"osd":2,"last update":"Wed May 22 21:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.54900000000000004,"15min":0.51800000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.30599999999999999},"max":{"1min":0.69499999999999995,"5min":0.89500000000000002,"15min":0.92300000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.53000000000000003,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.33600000000000002},"max":{"1min":0.66900000000000004,"5min":0.82799999999999996,"15min":0.93400000000000005},"last":0.44900000000000001}]},{"osd":4,"last update":"Wed May 22 21:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.72199999999999998,"15min":0.71799999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0580000000000001,"5min":1.1839999999999999,"15min":1.2050000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.747,"15min":0.73699999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.435},"max":{"1min":0.94599999999999995,"5min":1.093,"15min":1.5760000000000001},"last":0.61199999999999999}]},{"osd":5,"last update":"Wed May 22 21:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76000000000000001,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.46800000000000003,"15min":0.45100000000000001},"max":{"1min":1.0389999999999999,"5min":1.044,"15min":1.234},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.72699999999999998,"15min":0.72399999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.50800000000000001},"max":{"1min":0.92700000000000005,"5min":1.036,"15min":1.3049999999999999},"last":0.69499999999999995}]},{"osd":6,"last update":"Wed May 22 21:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.48799999999999999,"5min":0.46000000000000002,"15min":0.44900000000000001},"max":{"1min":1.016,"5min":1.0409999999999999,"15min":1.276},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77200000000000002,"15min":0.74399999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.50600000000000001},"max":{"1min":1.014,"5min":1.0860000000000001,"15min":1.2869999999999999},"last":0.66800000000000004}]},{"osd":7,"last update":"Wed May 22 21:25:28 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78500000000000003,"15min":0.77500000000000002},"min":{"1min":0.60099999999999998,"5min":0.52200000000000002,"15min":0.495},"max":{"1min":0.99199999999999999,"5min":1.0880000000000001,"15min":1.2709999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.76800000000000002,"15min":0.76200000000000001},"min":{"1min":0.52700000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.093,"5min":1.133,"15min":1.2969999999999999},"last":0.63100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:25:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.434,"15min":0.42699999999999999},"min":{"1min":0.316,"5min":0.26500000000000001,"15min":0.23599999999999999},"max":{"1min":0.58799999999999997,"5min":0.64800000000000002,"15min":2.3090000000000002},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.443,"5min":0.46500000000000002,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.28199999999999997,"15min":0.23400000000000001},"max":{"1min":0.71399999999999997,"5min":0.80400000000000005,"15min":2.532},"last":0.51000000000000001}]},{"osd":1,"last update":"Wed May 22 21:25:38 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.44700000000000001,"15min":0.45700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.749,"5min":0.749,"15min":2.948},"last":0.44},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55200000000000005,"15min":0.54500000000000004},"min":{"1min":0.432,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":2.8159999999999998},"last":0.60199999999999998}]},{"osd":3,"last update":"Wed May 22 21:25:38 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.36499999999999999,"5min":0.34899999999999998,"15min":0.32700000000000001},"max":{"1min":0.69899999999999995,"5min":0.83499999999999996,"15min":2.75},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50600000000000001,"15min":0.49299999999999999},"min":{"1min":0.35099999999999998,"5min":0.33400000000000002,"15min":0.315},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":3.0619999999999998},"last":0.45300000000000001}]},{"osd":4,"last update":"Wed May 22 21:25:38 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79600000000000004,"15min":0.75},"min":{"1min":0.627,"5min":0.53800000000000003,"15min":0.52700000000000002},"max":{"1min":1.413,"5min":1.413,"15min":2.7610000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79900000000000004,"15min":0.78200000000000003},"min":{"1min":0.57899999999999996,"5min":0.44600000000000001,"15min":0.375},"max":{"1min":1.1739999999999999,"5min":1.208,"15min":2.9279999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Wed May 22 21:25:38 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76100000000000001,"15min":0.74299999999999999},"min":{"1min":0.57099999999999995,"5min":0.51900000000000002,"15min":0.47299999999999998},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":2.9790000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.749,"15min":0.73099999999999998},"min":{"1min":0.58499999999999996,"5min":0.46999999999999997,"15min":0.45900000000000002},"max":{"1min":1.198,"5min":1.198,"15min":2.7069999999999999},"last":0.83999999999999997}]},{"osd":6,"last update":"Wed May 22 21:25:38 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.54600000000000004,"5min":0.51300000000000001,"15min":0.495},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":2.8580000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75700000000000001,"15min":0.72199999999999998},"min":{"1min":0.57199999999999995,"5min":0.50900000000000001,"15min":0.46500000000000002},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":2.6880000000000002},"last":0.78800000000000003}]},{"osd":7,"last update":"Wed May 22 21:25:38 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.81599999999999995,"15min":0.77900000000000003},"min":{"1min":0.626,"5min":0.54000000000000004,"15min":0.438},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":3.0099999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81699999999999995,"15min":0.80200000000000005},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.499},"max":{"1min":1.284,"5min":1.284,"15min":2.827},"last":0.81100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198887991,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 22 21:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.44,"15min":0.432},"min":{"1min":0.29599999999999999,"5min":0.254,"15min":0.216},"max":{"1min":0.69799999999999995,"5min":0.73499999999999999,"15min":1.3879999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48799999999999999,"15min":0.46100000000000002},"min":{"1min":0.377,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68000000000000005,"5min":0.879,"15min":0.879},"last":0.48699999999999999}]},{"osd":2,"last update":"Wed May 22 21:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.34100000000000003,"5min":0.32000000000000001,"15min":0.252},"max":{"1min":0.66100000000000003,"5min":0.80300000000000005,"15min":1.76},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.49299999999999999,"15min":0.499},"min":{"1min":0.313,"5min":0.30599999999999999,"15min":0.28999999999999998},"max":{"1min":0.56699999999999995,"5min":0.83599999999999997,"15min":1.923},"last":0.60599999999999998}]},{"osd":3,"last update":"Wed May 22 21:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56599999999999995,"15min":0.55100000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.32600000000000001},"max":{"1min":0.76200000000000001,"5min":0.82999999999999996,"15min":2.1030000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53700000000000003,"15min":0.50800000000000001},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74399999999999999,"5min":0.75,"15min":0.75},"last":0.63800000000000001}]},{"osd":4,"last update":"Wed May 22 21:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.52600000000000002,"5min":0.48099999999999998,"15min":0.45100000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":2.1360000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72399999999999998,"15min":0.71699999999999997},"min":{"1min":0.53700000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.087,"5min":1.1519999999999999,"15min":1.7829999999999999},"last":0.80300000000000005}]},{"osd":5,"last update":"Wed May 22 21:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.73999999999999999,"15min":0.751},"min":{"1min":0.55300000000000005,"5min":0.53100000000000003,"15min":0.45300000000000001},"max":{"1min":1.0189999999999999,"5min":1.0549999999999999,"15min":1.819},"last":0.751},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.97},"last":0.81899999999999995}]},{"osd":6,"last update":"Wed May 22 21:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.78400000000000003,"15min":0.76500000000000001},"min":{"1min":0.61199999999999999,"5min":0.52000000000000002,"15min":0.47299999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.272},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.78100000000000003},"min":{"1min":0.56999999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.103,"5min":1.1259999999999999,"15min":1.8029999999999999},"last":0.88100000000000001}]},{"osd":7,"last update":"Wed May 22 21:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77200000000000002,"15min":0.76800000000000002},"min":{"1min":0.60599999999999998,"5min":0.47999999999999998,"15min":0.42099999999999999},"max":{"1min":1.046,"5min":1.046,"15min":2.0310000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77800000000000002,"15min":0.76500000000000001},"min":{"1min":0.55200000000000005,"5min":0.54900000000000004,"15min":0.52000000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":2.0659999999999998},"last":0.77500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1661,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193650,"statfs":{"total":200000000,"available":198298242,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 22 21:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.46500000000000002,"15min":0.433},"min":{"1min":0.28299999999999997,"5min":0.24299999999999999,"15min":0.22800000000000001},"max":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.88300000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54600000000000004,"15min":0.51200000000000001},"min":{"1min":0.41499999999999998,"5min":0.34799999999999998,"15min":0.33100000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.60999999999999999}]},{"osd":2,"last update":"Wed May 22 21:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.52500000000000002,"15min":0.48699999999999999},"min":{"1min":0.378,"5min":0.32100000000000001,"15min":0.29399999999999998},"max":{"1min":0.74099999999999999,"5min":0.85899999999999999,"15min":0.93799999999999994},"last":0.499},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.46400000000000002,"15min":0.45000000000000001},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.27900000000000003},"max":{"1min":0.56000000000000005,"5min":0.66200000000000003,"15min":0.92300000000000004},"last":0.54200000000000004}]},{"osd":3,"last update":"Wed May 22 21:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47599999999999998,"15min":0.45600000000000002},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":0.622,"5min":0.71899999999999997,"15min":1.0760000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.432,"5min":0.39600000000000002,"15min":0.36899999999999999},"max":{"1min":0.69199999999999995,"5min":0.89700000000000002,"15min":0.98299999999999998},"last":0.56899999999999995}]},{"osd":4,"last update":"Wed May 22 21:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.74099999999999999,"15min":0.72799999999999998},"min":{"1min":0.43099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93100000000000005,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76500000000000001,"15min":0.748},"min":{"1min":0.52900000000000003,"5min":0.50600000000000001,"15min":0.48599999999999999},"max":{"1min":0.91900000000000004,"5min":1.2330000000000001,"15min":1.2709999999999999},"last":0.70999999999999996}]},{"osd":5,"last update":"Wed May 22 21:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.749,"15min":0.72699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.873,"5min":1.135,"15min":1.135},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.75700000000000001,"15min":0.752},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.871,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.81499999999999995}]},{"osd":6,"last update":"Wed May 22 21:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.81299999999999994,"15min":0.79200000000000004},"min":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.51500000000000001},"max":{"1min":0.92600000000000005,"5min":1.175,"15min":1.2090000000000001},"last":0.751},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.79600000000000004,"15min":0.77900000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":0.90000000000000002,"5min":1.3149999999999999,"15min":1.5},"last":0.66100000000000003}]},{"osd":7,"last update":"Wed May 22 21:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.77500000000000002,"15min":0.753},"min":{"1min":0.499,"5min":0.499,"15min":0.49099999999999999},"max":{"1min":0.93700000000000006,"5min":1.214,"15min":1.214},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77900000000000003,"15min":0.76500000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.497},"max":{"1min":0.91900000000000004,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.68200000000000005}]}]}],"pool_statfs":[]}} 2024-05-22T21:25:48.810 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-22T21:25:48.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.946+0000 7f6c21c00640 1 Processor -- start 2024-05-22T21:25:48.947 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.946+0000 7f6c21c00640 1 -- start start 2024-05-22T21:25:48.948 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.946+0000 7f6c21c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6c1c143c40 0x7f6c1c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:48.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.946+0000 7f6c21c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1c15efb0 con 0x7f6c1c06c9b0 2024-05-22T21:25:48.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.946+0000 7f6c21c00640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1c15f290 con 0x7f6c1c1444a0 2024-05-22T21:25:48.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.946+0000 7f6c21c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c1c15f570 con 0x7f6c1c143c40 2024-05-22T21:25:48.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.946+0000 7f6c1aa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6c1c143c40 0x7f6c1c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:48.949 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.946+0000 7f6c1aa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6c1c143c40 0x7f6c1c15b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55432/0 (socket says 172.21.15.161:55432) 2024-05-22T21:25:48.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.946+0000 7f6c1aa00640 1 -- 172.21.15.161:0/2845028075 learned_addr learned my addr 172.21.15.161:0/2845028075 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:48.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.946+0000 7f6c1aa00640 1 -- 172.21.15.161:0/2845028075 >> v1:172.21.15.179:6789/0 conn(0x7f6c1c1444a0 legacy=0x7f6c1c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:48.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.946+0000 7f6c1aa00640 1 -- 172.21.15.161:0/2845028075 >> v1:172.21.15.161:6789/0 conn(0x7f6c1c06c9b0 legacy=0x7f6c1c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:48.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.946+0000 7f6c1aa00640 1 -- 172.21.15.161:0/2845028075 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c15f850 con 0x7f6c1c143c40 2024-05-22T21:25:48.950 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.946+0000 7f6c19600640 1 -- 172.21.15.161:0/2845028075 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912273300 0 0) 0x7f6c1c15efb0 con 0x7f6c1c06c9b0 2024-05-22T21:25:48.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.950+0000 7f6c1aa00640 1 --2- 172.21.15.161:0/2845028075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c143c40 0x7f6c1c15b600 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009870 tx=0x7f6c0c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251c5bd33d302d5b server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:48.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.950+0000 7f6c19600640 1 -- 172.21.15.161:0/2845028075 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c0aa620 con 0x7f6c1c143c40 2024-05-22T21:25:48.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.950+0000 7f6c19600640 1 -- 172.21.15.161:0/2845028075 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0c0aa780 con 0x7f6c1c143c40 2024-05-22T21:25:48.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.950+0000 7f6c19600640 1 -- 172.21.15.161:0/2845028075 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c0aab10 con 0x7f6c1c143c40 2024-05-22T21:25:48.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.950+0000 7f6c21c00640 1 -- 172.21.15.161:0/2845028075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c143c40 msgr2=0x7f6c1c15b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:48.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.950+0000 7f6c21c00640 1 --2- 172.21.15.161:0/2845028075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c143c40 0x7f6c1c15b600 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009870 tx=0x7f6c0c01c560 comp rx=0 tx=0).stop 2024-05-22T21:25:48.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.950+0000 7f6c21c00640 1 -- 172.21.15.161:0/2845028075 shutdown_connections 2024-05-22T21:25:48.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.950+0000 7f6c21c00640 1 --2- 172.21.15.161:0/2845028075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c143c40 0x7f6c1c15b600 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:48.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.950+0000 7f6c21c00640 1 -- 172.21.15.161:0/2845028075 >> 172.21.15.161:0/2845028075 conn(0x7f6c1c06a2d0 msgr2=0x7f6c1c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:48.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.950+0000 7f6c21c00640 1 -- 172.21.15.161:0/2845028075 shutdown_connections 2024-05-22T21:25:48.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.950+0000 7f6c21c00640 1 -- 172.21.15.161:0/2845028075 wait complete. 2024-05-22T21:25:48.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c21c00640 1 Processor -- start 2024-05-22T21:25:48.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c21c00640 1 -- start start 2024-05-22T21:25:48.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c21c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c1c06c9b0 0x7f6c1c139090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:48.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c21c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c143c40 0x7f6c1c142af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:48.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c21c00640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6c1c1444a0 0x7f6c1c13eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:48.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c21c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c1570e0 con 0x7f6c1c143c40 2024-05-22T21:25:48.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c21c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c1395d0 con 0x7f6c1c06c9b0 2024-05-22T21:25:48.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c21c00640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c143030 con 0x7f6c1c1444a0 2024-05-22T21:25:48.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c1a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c143c40 0x7f6c1c142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:48.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c1a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c143c40 0x7f6c1c142af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55438/0 (socket says 172.21.15.161:55438) 2024-05-22T21:25:48.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c1a000640 1 -- 172.21.15.161:0/37306447 learned_addr learned my addr 172.21.15.161:0/37306447 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:48.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c1aa00640 1 --2- 172.21.15.161:0/37306447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c1c06c9b0 0x7f6c1c139090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:48.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c1b400640 1 --2- 172.21.15.161:0/37306447 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6c1c1444a0 0x7f6c1c13eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:48.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c1a000640 1 -- 172.21.15.161:0/37306447 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6c1c1444a0 msgr2=0x7f6c1c13eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:48.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c1a000640 1 --2- 172.21.15.161:0/37306447 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6c1c1444a0 0x7f6c1c13eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:48.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c1a000640 1 -- 172.21.15.161:0/37306447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c1c06c9b0 msgr2=0x7f6c1c139090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:48.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c1a000640 1 --2- 172.21.15.161:0/37306447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c1c06c9b0 0x7f6c1c139090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:48.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c1a000640 1 -- 172.21.15.161:0/37306447 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c0094d0 con 0x7f6c1c143c40 2024-05-22T21:25:48.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c1a000640 1 --2- 172.21.15.161:0/37306447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c143c40 0x7f6c1c142af0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800c3c0 tx=0x7f6c0800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:48.959 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c12a00640 1 -- 172.21.15.161:0/37306447 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c080052d0 con 0x7f6c1c143c40 2024-05-22T21:25:48.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c12a00640 1 -- 172.21.15.161:0/37306447 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c08005430 con 0x7f6c1c143c40 2024-05-22T21:25:48.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c21c00640 1 -- 172.21.15.161:0/37306447 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1c1432d0 con 0x7f6c1c143c40 2024-05-22T21:25:48.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c12a00640 1 -- 172.21.15.161:0/37306447 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c08003840 con 0x7f6c1c143c40 2024-05-22T21:25:48.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.954+0000 7f6c21c00640 1 -- 172.21.15.161:0/37306447 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1c157610 con 0x7f6c1c143c40 2024-05-22T21:25:48.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.958+0000 7f6c21c00640 1 -- 172.21.15.161:0/37306447 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1c067220 con 0x7f6c1c143c40 2024-05-22T21:25:48.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.958+0000 7f6c12a00640 1 -- 172.21.15.161:0/37306447 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6c080039a0 con 0x7f6c1c143c40 2024-05-22T21:25:48.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.958+0000 7f6c12a00640 1 --2- 172.21.15.161:0/37306447 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6bfc0666c0 0x7f6bfc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:48.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.958+0000 7f6c12a00640 1 -- 172.21.15.161:0/37306447 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6c08083230 con 0x7f6c1c143c40 2024-05-22T21:25:48.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.958+0000 7f6c1aa00640 1 --2- 172.21.15.161:0/37306447 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6bfc0666c0 0x7f6bfc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:48.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.958+0000 7f6c1aa00640 1 --2- 172.21.15.161:0/37306447 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6bfc0666c0 0x7f6bfc068b80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0ae000 tx=0x7f6c0c0b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:48.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:48.962+0000 7f6c12a00640 1 -- 172.21.15.161:0/37306447 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f6c080509e0 con 0x7f6c1c143c40 2024-05-22T21:25:49.118 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.118+0000 7f6c21c00640 1 -- 172.21.15.161:0/37306447 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f6c1c0ceaa0 con 0x7f6c1c143c40 2024-05-22T21:25:49.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.122+0000 7f6c12a00640 1 -- 172.21.15.161:0/37306447 <== mon.0 v2:172.21.15.161:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6c08001130 con 0x7f6c1c143c40 2024-05-22T21:25:49.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.122+0000 7f6c12a00640 1 -- 172.21.15.161:0/37306447 <== mon.0 v2:172.21.15.161:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) v1 ==== 123+0+0 (secure 0 0 0) 0x7f6c08054000 con 0x7f6c1c143c40 2024-05-22T21:25:49.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.126+0000 7f6c21c00640 1 -- 172.21.15.161:0/37306447 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6bfc0666c0 msgr2=0x7f6bfc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:49.126 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.126+0000 7f6c21c00640 1 --2- 172.21.15.161:0/37306447 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6bfc0666c0 0x7f6bfc068b80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0ae000 tx=0x7f6c0c0b5040 comp rx=0 tx=0).stop 2024-05-22T21:25:49.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.126+0000 7f6c21c00640 1 -- 172.21.15.161:0/37306447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c143c40 msgr2=0x7f6c1c142af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:49.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.126+0000 7f6c21c00640 1 --2- 172.21.15.161:0/37306447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c143c40 0x7f6c1c142af0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800c3c0 tx=0x7f6c0800c880 comp rx=0 tx=0).stop 2024-05-22T21:25:49.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.126+0000 7f6c21c00640 1 -- 172.21.15.161:0/37306447 shutdown_connections 2024-05-22T21:25:49.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.126+0000 7f6c21c00640 1 --2- 172.21.15.161:0/37306447 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f6bfc0666c0 0x7f6bfc068b80 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:49.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.126+0000 7f6c21c00640 1 --2- 172.21.15.161:0/37306447 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6c1c1444a0 0x7f6c1c13eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:49.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.126+0000 7f6c21c00640 1 --2- 172.21.15.161:0/37306447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c1c143c40 0x7f6c1c142af0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:49.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.126+0000 7f6c21c00640 1 --2- 172.21.15.161:0/37306447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c1c06c9b0 0x7f6c1c139090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:49.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.126+0000 7f6c21c00640 1 -- 172.21.15.161:0/37306447 >> 172.21.15.161:0/37306447 conn(0x7f6c1c06a2d0 msgr2=0x7f6c1c15bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:49.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.126+0000 7f6c21c00640 1 -- 172.21.15.161:0/37306447 shutdown_connections 2024-05-22T21:25:49.127 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.126+0000 7f6c21c00640 1 -- 172.21.15.161:0/37306447 wait complete. 2024-05-22T21:25:49.147 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-22T21:25:49.147 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-22T21:25:49.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.282+0000 7f569d000640 1 Processor -- start 2024-05-22T21:25:49.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.282+0000 7f569d000640 1 -- start start 2024-05-22T21:25:49.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.286+0000 7f569d000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5698143ec0 0x7f569815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:49.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.286+0000 7f569d000640 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f569815ec90 0x7f569815f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:49.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.286+0000 7f569d000640 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56981442a0 con 0x7f56980715a0 2024-05-22T21:25:49.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.286+0000 7f569d000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569815f660 con 0x7f5698143ec0 2024-05-22T21:25:49.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.286+0000 7f569d000640 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5698070500 con 0x7f569815ec90 2024-05-22T21:25:49.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.286+0000 7f5695e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5698143ec0 0x7f569815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:49.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.286+0000 7f5695e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5698143ec0 0x7f569815f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55452/0 (socket says 172.21.15.161:55452) 2024-05-22T21:25:49.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.286+0000 7f5695e00640 1 -- 172.21.15.161:0/220055441 learned_addr learned my addr 172.21.15.161:0/220055441 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:49.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.286+0000 7f5696800640 1 --2- 172.21.15.161:0/220055441 >> v2:172.21.15.179:3300/0 conn(0x7f569815ec90 0x7f569815f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:49.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.286+0000 7f5695e00640 1 -- 172.21.15.161:0/220055441 >> v2:172.21.15.179:3300/0 conn(0x7f569815ec90 msgr2=0x7f569815f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:49.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.286+0000 7f5695e00640 1 --2- 172.21.15.161:0/220055441 >> v2:172.21.15.179:3300/0 conn(0x7f569815ec90 0x7f569815f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:49.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.286+0000 7f5695e00640 1 -- 172.21.15.161:0/220055441 >> v1:172.21.15.179:6789/0 conn(0x7f56980715a0 legacy=0x7f569806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-22T21:25:49.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.286+0000 7f5695e00640 1 -- 172.21.15.161:0/220055441 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56980707e0 con 0x7f5698143ec0 2024-05-22T21:25:49.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.290+0000 7f5695e00640 1 --2- 172.21.15.161:0/220055441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5698143ec0 0x7f569815f830 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f56980cedc0 tx=0x7f568801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f55cfb87819d6a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:49.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.290+0000 7f5694a00640 1 -- 172.21.15.161:0/220055441 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56880a5780 con 0x7f5698143ec0 2024-05-22T21:25:49.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.290+0000 7f5694a00640 1 -- 172.21.15.161:0/220055441 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f56880a58e0 con 0x7f5698143ec0 2024-05-22T21:25:49.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.290+0000 7f5694a00640 1 -- 172.21.15.161:0/220055441 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56880a5c90 con 0x7f5698143ec0 2024-05-22T21:25:49.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.290+0000 7f569d000640 1 -- 172.21.15.161:0/220055441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5698143ec0 msgr2=0x7f569815f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:49.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.290+0000 7f569d000640 1 --2- 172.21.15.161:0/220055441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5698143ec0 0x7f569815f830 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f56980cedc0 tx=0x7f568801c9c0 comp rx=0 tx=0).stop 2024-05-22T21:25:49.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.290+0000 7f569d000640 1 -- 172.21.15.161:0/220055441 shutdown_connections 2024-05-22T21:25:49.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.290+0000 7f569d000640 1 --2- 172.21.15.161:0/220055441 >> v2:172.21.15.179:3300/0 conn(0x7f569815ec90 0x7f569815f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:49.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.290+0000 7f569d000640 1 --2- 172.21.15.161:0/220055441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5698143ec0 0x7f569815f830 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:49.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.290+0000 7f569d000640 1 -- 172.21.15.161:0/220055441 >> 172.21.15.161:0/220055441 conn(0x7f56980b0bc0 msgr2=0x7f56980b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:49.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.290+0000 7f569d000640 1 -- 172.21.15.161:0/220055441 shutdown_connections 2024-05-22T21:25:49.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.290+0000 7f569d000640 1 -- 172.21.15.161:0/220055441 wait complete. 2024-05-22T21:25:49.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.290+0000 7f569d000640 1 Processor -- start 2024-05-22T21:25:49.292 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.290+0000 7f569d000640 1 -- start start 2024-05-22T21:25:49.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f569d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5698143ec0 0x7f56980b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:49.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f569d000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f569815ec90 0x7f569813d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:49.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f569d000640 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5698159cb0 0x7f569813d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:49.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f569d000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569815cff0 con 0x7f5698143ec0 2024-05-22T21:25:49.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f569d000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569815c120 con 0x7f569815ec90 2024-05-22T21:25:49.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f569d000640 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569815d410 con 0x7f5698159cb0 2024-05-22T21:25:49.294 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f5695e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5698143ec0 0x7f56980b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:49.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f5695e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5698143ec0 0x7f56980b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55456/0 (socket says 172.21.15.161:55456) 2024-05-22T21:25:49.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f5695e00640 1 -- 172.21.15.161:0/3477953965 learned_addr learned my addr 172.21.15.161:0/3477953965 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-22T21:25:49.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f5695400640 1 --2- 172.21.15.161:0/3477953965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f569815ec90 0x7f569813d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:49.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f5696800640 1 --2- 172.21.15.161:0/3477953965 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5698159cb0 0x7f569813d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:49.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f5695e00640 1 -- 172.21.15.161:0/3477953965 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5698159cb0 msgr2=0x7f569813d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:49.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f5695e00640 1 --2- 172.21.15.161:0/3477953965 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5698159cb0 0x7f569813d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:49.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f5695e00640 1 -- 172.21.15.161:0/3477953965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f569815ec90 msgr2=0x7f569813d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:49.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f5695e00640 1 --2- 172.21.15.161:0/3477953965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f569815ec90 0x7f569813d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:49.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f5695e00640 1 -- 172.21.15.161:0/3477953965 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5688009b30 con 0x7f5698143ec0 2024-05-22T21:25:49.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f5695e00640 1 --2- 172.21.15.161:0/3477953965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5698143ec0 0x7f56980b3490 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f56880af040 tx=0x7f5688019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:49.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f568e000640 1 -- 172.21.15.161:0/3477953965 <== mon.0 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56880b8670 con 0x7f5698143ec0 2024-05-22T21:25:49.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f568e000640 1 -- 172.21.15.161:0/3477953965 <== mon.0 v2:172.21.15.161:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f56880b87f0 con 0x7f5698143ec0 2024-05-22T21:25:49.295 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f568e000640 1 -- 172.21.15.161:0/3477953965 <== mon.0 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56880b8bc0 con 0x7f5698143ec0 2024-05-22T21:25:49.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f569d000640 1 -- 172.21.15.161:0/3477953965 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569815c370 con 0x7f5698143ec0 2024-05-22T21:25:49.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f569d000640 1 -- 172.21.15.161:0/3477953965 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569815c930 con 0x7f5698143ec0 2024-05-22T21:25:49.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f569d000640 1 -- 172.21.15.161:0/3477953965 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5698067220 con 0x7f5698143ec0 2024-05-22T21:25:49.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.294+0000 7f568e000640 1 -- 172.21.15.161:0/3477953965 <== mon.0 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f56880b8d20 con 0x7f5698143ec0 2024-05-22T21:25:49.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.298+0000 7f568e000640 1 --2- 172.21.15.161:0/3477953965 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5678066690 0x7f5678068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-22T21:25:49.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.298+0000 7f568e000640 1 -- 172.21.15.161:0/3477953965 <== mon.0 v2:172.21.15.161:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f568811f5b0 con 0x7f5698143ec0 2024-05-22T21:25:49.297 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.298+0000 7f5695400640 1 --2- 172.21.15.161:0/3477953965 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5678066690 0x7f5678068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-22T21:25:49.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.298+0000 7f5695400640 1 --2- 172.21.15.161:0/3477953965 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5678066690 0x7f5678068b50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f56980ccab0 tx=0x7f5684009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-22T21:25:49.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.302+0000 7f568e000640 1 -- 172.21.15.161:0/3477953965 <== mon.0 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175322 (secure 0 0 0) 0x7f56880ecca0 con 0x7f5698143ec0 2024-05-22T21:25:49.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.486+0000 7f569d000640 1 -- 172.21.15.161:0/3477953965 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f569806dcc0 con 0x7f5698143ec0 2024-05-22T21:25:49.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.486+0000 7f568e000640 1 -- 172.21.15.161:0/3477953965 <== mon.0 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f56880f02c0 con 0x7f5698143ec0 2024-05-22T21:25:49.488 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-22T21:25:49.488 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":88,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7199,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3304182060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6833","nonce":3304182060},{"type":"v1","addr":"172.21.15.179:6835","nonce":3304182060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6836/196309272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":196309272},{"type":"v1","addr":"172.21.15.161:6836","nonce":196309272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6834/396408969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":396408969},{"type":"v1","addr":"172.21.15.179:6834","nonce":396408969}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":9150,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3922635813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6835","nonce":3922635813},{"type":"v1","addr":"172.21.15.161:6837","nonce":3922635813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[]} 2024-05-22T21:25:49.488 INFO:teuthology.orchestra.run.smithi161.stderr:dumped fsmap epoch 88 2024-05-22T21:25:49.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.490+0000 7f569d000640 1 -- 172.21.15.161:0/3477953965 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5678066690 msgr2=0x7f5678068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:49.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.490+0000 7f569d000640 1 --2- 172.21.15.161:0/3477953965 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5678066690 0x7f5678068b50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f56980ccab0 tx=0x7f5684009040 comp rx=0 tx=0).stop 2024-05-22T21:25:49.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.490+0000 7f569d000640 1 -- 172.21.15.161:0/3477953965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5698143ec0 msgr2=0x7f56980b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-22T21:25:49.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.490+0000 7f569d000640 1 --2- 172.21.15.161:0/3477953965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5698143ec0 0x7f56980b3490 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f56880af040 tx=0x7f5688019760 comp rx=0 tx=0).stop 2024-05-22T21:25:49.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.490+0000 7f569d000640 1 -- 172.21.15.161:0/3477953965 shutdown_connections 2024-05-22T21:25:49.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.490+0000 7f569d000640 1 --2- 172.21.15.161:0/3477953965 >> [v2:172.21.15.161:6832/23334,v1:172.21.15.161:6833/23334] conn(0x7f5678066690 0x7f5678068b50 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:49.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.490+0000 7f569d000640 1 --2- 172.21.15.161:0/3477953965 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5698159cb0 0x7f569813d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:49.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.490+0000 7f569d000640 1 --2- 172.21.15.161:0/3477953965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f569815ec90 0x7f569813d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:49.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.490+0000 7f569d000640 1 --2- 172.21.15.161:0/3477953965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5698143ec0 0x7f56980b3490 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-22T21:25:49.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.490+0000 7f569d000640 1 -- 172.21.15.161:0/3477953965 >> 172.21.15.161:0/3477953965 conn(0x7f56980b0bc0 msgr2=0x7f5698141030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-22T21:25:49.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.490+0000 7f569d000640 1 -- 172.21.15.161:0/3477953965 shutdown_connections 2024-05-22T21:25:49.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-22T21:25:49.490+0000 7f569d000640 1 -- 172.21.15.161:0/3477953965 wait complete. 2024-05-22T21:25:49.509 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-22T21:25:49.509 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-22T21:25:49.510 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-22T21:25:49.510 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:49.522 INFO:tasks.ceph.mds.a:Stopped 2024-05-22T21:25:49.523 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-22T21:25:49.523 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:49.579 INFO:tasks.ceph.mds.c:Stopped 2024-05-22T21:25:49.579 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-22T21:25:49.579 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:49.594 INFO:tasks.ceph.mds.b:Stopped 2024-05-22T21:25:49.594 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-22T21:25:49.594 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:49.650 INFO:tasks.ceph.mds.d:Stopped 2024-05-22T21:25:49.651 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-22T21:25:49.651 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-22T21:25:49.651 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:49.698 INFO:tasks.ceph.osd.0:Stopped 2024-05-22T21:25:49.699 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-22T21:25:49.699 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:49.757 INFO:tasks.ceph.osd.1:Stopped 2024-05-22T21:25:49.757 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-22T21:25:49.757 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:49.825 INFO:tasks.ceph.osd.2:Stopped 2024-05-22T21:25:49.825 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-22T21:25:49.825 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:49.870 INFO:tasks.ceph.osd.3:Stopped 2024-05-22T21:25:49.870 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-22T21:25:49.870 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:49.921 INFO:tasks.ceph.osd.4:Stopped 2024-05-22T21:25:49.921 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-22T21:25:49.921 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:49.959 INFO:tasks.ceph.osd.5:Stopped 2024-05-22T21:25:49.959 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-22T21:25:49.959 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:50.052 INFO:tasks.ceph.osd.6:Stopped 2024-05-22T21:25:50.052 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-22T21:25:50.052 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:50.098 INFO:tasks.ceph.osd.7:Stopped 2024-05-22T21:25:50.098 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-22T21:25:50.098 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-22T21:25:50.098 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:50.160 INFO:tasks.ceph.mgr.y:Stopped 2024-05-22T21:25:50.160 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-22T21:25:50.161 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:50.211 INFO:tasks.ceph.mgr.x:Stopped 2024-05-22T21:25:50.212 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-22T21:25:50.212 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-22T21:25:50.212 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:50.231 INFO:tasks.ceph.mon.a:Stopped 2024-05-22T21:25:50.231 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-22T21:25:50.231 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:50.247 INFO:tasks.ceph.mon.b:Stopped 2024-05-22T21:25:50.247 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-22T21:25:50.247 INFO:teuthology.orchestra.run:waiting for 300 2024-05-22T21:25:50.309 INFO:tasks.ceph.mon.c:Stopped 2024-05-22T21:25:50.309 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-22T21:25:50.309 DEBUG:teuthology.orchestra.run.smithi161:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-05-22T21:25:50.341 INFO:teuthology.orchestra.run.smithi161.stdout:2024-05-22T21:13:09.980360+0000 osd.2 (osd.2) 3 : cluster [WRN] OSD bench result of 135369.788297 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.2. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2024-05-22T21:25:50.342 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-05-22T21:25:50.342 DEBUG:teuthology.orchestra.run.smithi161:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-05-22T21:25:50.422 DEBUG:teuthology.orchestra.run.smithi161:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-05-22T21:25:50.501 DEBUG:teuthology.orchestra.run.smithi161:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-05-22T21:25:50.575 INFO:teuthology.orchestra.run.smithi161.stdout:2024-05-22T21:13:09.980360+0000 osd.2 (osd.2) 3 : cluster [WRN] OSD bench result of 135369.788297 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.2. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2024-05-22T21:25:50.576 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:25:50.576 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-22T21:25:50.985 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:25:50.986 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-22T21:25:51.183 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:25:51.183 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-22T21:25:51.407 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:25:51.407 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-22T21:25:51.591 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:25:51.591 DEBUG:teuthology.orchestra.run.smithi179:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-22T21:25:51.832 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:25:51.832 DEBUG:teuthology.orchestra.run.smithi179:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-22T21:25:52.025 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:25:52.025 DEBUG:teuthology.orchestra.run.smithi179:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-22T21:25:52.183 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:25:52.184 DEBUG:teuthology.orchestra.run.smithi179:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-22T21:25:52.342 INFO:tasks.ceph:Archiving mon data... 2024-05-22T21:25:52.344 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718018/data/mon.a.tgz 2024-05-22T21:25:52.344 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2024-05-22T21:25:52.353 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.LZmKa8gCUo 2024-05-22T21:25:52.353 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.LZmKa8gCUo 2024-05-22T21:25:52.493 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.LZmKa8gCUo 2024-05-22T21:25:52.568 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.LZmKa8gCUo is 398KB 2024-05-22T21:25:52.635 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.LZmKa8gCUo 2024-05-22T21:25:52.643 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718018/data/mon.b.tgz 2024-05-22T21:25:52.643 DEBUG:teuthology.orchestra.run.smithi179:> mktemp 2024-05-22T21:25:52.652 INFO:teuthology.orchestra.run.smithi179.stdout:/tmp/tmp.H0gSdZWd7K 2024-05-22T21:25:52.652 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.H0gSdZWd7K 2024-05-22T21:25:52.792 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0666 /tmp/tmp.H0gSdZWd7K 2024-05-22T21:25:52.874 DEBUG:teuthology.orchestra.remote:smithi179:/tmp/tmp.H0gSdZWd7K is 399KB 2024-05-22T21:25:52.940 DEBUG:teuthology.orchestra.run.smithi179:> rm -fr /tmp/tmp.H0gSdZWd7K 2024-05-22T21:25:52.948 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718018/data/mon.c.tgz 2024-05-22T21:25:52.948 DEBUG:teuthology.orchestra.run.smithi179:> mktemp 2024-05-22T21:25:52.996 INFO:teuthology.orchestra.run.smithi179.stdout:/tmp/tmp.QgOyJz6bfd 2024-05-22T21:25:52.997 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.QgOyJz6bfd 2024-05-22T21:25:53.134 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0666 /tmp/tmp.QgOyJz6bfd 2024-05-22T21:25:53.209 DEBUG:teuthology.orchestra.remote:smithi179:/tmp/tmp.QgOyJz6bfd is 399KB 2024-05-22T21:25:53.275 DEBUG:teuthology.orchestra.run.smithi179:> rm -fr /tmp/tmp.QgOyJz6bfd 2024-05-22T21:25:53.283 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-22T21:25:53.284 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-22T21:25:53.287 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-22T21:25:53.348 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-05-22T21:25:53.348 DEBUG:teuthology.orchestra.run.smithi161:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-05-22T21:25:53.351 DEBUG:teuthology.orchestra.run.smithi179:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-05-22T21:25:53.380 INFO:teuthology.orchestra.run.smithi161.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-05-22T21:25:53.434 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-05-22T21:25:53.438 INFO:tasks.ceph:Archiving crash dumps... 2024-05-22T21:25:53.440 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718018/remote/smithi161/crash 2024-05-22T21:25:53.441 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-22T21:25:53.461 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718018/remote/smithi179/crash 2024-05-22T21:25:53.462 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-22T21:25:53.499 INFO:tasks.ceph:Shutting down logrotate 2024-05-22T21:25:53.499 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:25:53.511 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-22T21:25:53.592 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-22T21:25:53.611 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-22T21:25:53.630 INFO:tasks.ceph:Compressing logs... 2024-05-22T21:25:53.631 DEBUG:teuthology.orchestra.run.smithi161:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-22T21:25:53.655 DEBUG:teuthology.orchestra.run.smithi179:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-22T21:25:58.679 INFO:tasks.ceph:Archiving logs... 2024-05-22T21:25:58.679 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718018/remote/smithi161/log 2024-05-22T21:25:58.680 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-22T21:25:59.745 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718018/remote/smithi179/log 2024-05-22T21:25:59.746 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-22T21:26:00.202 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-22T21:26:00.211 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-22T21:26:00.212 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-22T21:26:00.215 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-22T21:26:00.298 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-22T21:26:00.307 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-22T21:26:00.308 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-22T21:26:00.311 DEBUG:teuthology.orchestra.run.smithi179:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-22T21:26:00.324 INFO:teuthology.orchestra.run.smithi161.stdout: remote refid st t when poll reach delay offset jitter 2024-05-22T21:26:00.324 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================== 2024-05-22T21:26:00.324 INFO:teuthology.orchestra.run.smithi161.stdout:*hv01.front.sepi 67.205.162.81 3 u 49 64 377 0.101 -0.849 1.905 2024-05-22T21:26:00.324 INFO:teuthology.orchestra.run.smithi161.stdout:+hv02.front.sepi 63.231.80.2 3 u 51 64 377 0.084 -2.397 2.327 2024-05-22T21:26:00.324 INFO:teuthology.orchestra.run.smithi161.stdout:+hv03.front.sepi 74.6.168.72 3 u 47 64 377 0.087 -6.901 1.324 2024-05-22T21:26:00.324 INFO:teuthology.orchestra.run.smithi161.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-05-22T21:26:00.326 INFO:teuthology.orchestra.run.smithi179.stdout: remote refid st t when poll reach delay offset jitter 2024-05-22T21:26:00.326 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================== 2024-05-22T21:26:00.326 INFO:teuthology.orchestra.run.smithi179.stdout:*hv01.front.sepi 67.205.162.81 3 u 46 64 377 0.093 -4.813 2.336 2024-05-22T21:26:00.326 INFO:teuthology.orchestra.run.smithi179.stdout:+hv02.front.sepi 63.231.80.2 3 u 49 64 377 0.086 -3.179 2.302 2024-05-22T21:26:00.326 INFO:teuthology.orchestra.run.smithi179.stdout:+hv03.front.sepi 74.6.168.72 3 u 43 128 377 0.085 -5.680 3.116 2024-05-22T21:26:00.327 INFO:teuthology.orchestra.run.smithi179.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-05-22T21:26:00.327 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-22T21:26:00.335 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-22T21:26:00.335 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-22T21:26:00.353 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-22T21:26:00.362 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-22T21:26:00.370 INFO:teuthology.task.internal:Duration was 1430.107235 seconds 2024-05-22T21:26:00.371 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-22T21:26:00.379 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-22T21:26:00.380 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-22T21:26:00.383 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-22T21:26:00.486 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-22T21:26:00.486 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-05-22T21:26:00.486 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-22T21:26:00.506 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi179.front.sepia.ceph.com 2024-05-22T21:26:00.506 DEBUG:teuthology.orchestra.run.smithi179:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-22T21:26:00.526 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-22T21:26:00.526 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-22T21:26:00.551 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-22T21:26:00.610 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-22T21:26:00.610 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-22T21:26:00.635 DEBUG:teuthology.orchestra.run.smithi179:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-22T21:26:00.723 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-22T21:26:00.732 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-22T21:26:00.732 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-22T21:26:00.752 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-22T21:26:00.772 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-22T21:26:00.781 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-22T21:26:00.803 DEBUG:teuthology.orchestra.run.smithi179:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-22T21:26:00.817 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-05-22T21:26:00.834 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = core 2024-05-22T21:26:00.852 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-22T21:26:00.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:26:00.887 DEBUG:teuthology.orchestra.run.smithi179:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-22T21:26:00.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-22T21:26:00.903 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-22T21:26:00.913 INFO:teuthology.task.internal:Transferring archived files... 2024-05-22T21:26:00.914 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718018/remote/smithi161 2024-05-22T21:26:00.914 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-22T21:26:00.955 DEBUG:teuthology.misc:Transferring archived files from smithi179:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-21_10:46:42-fs-reef-distro-default-smithi/7718018/remote/smithi179 2024-05-22T21:26:00.955 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-22T21:26:00.980 INFO:teuthology.task.internal:Removing archive directory... 2024-05-22T21:26:00.980 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-22T21:26:00.994 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-22T21:26:01.025 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-22T21:26:01.036 INFO:teuthology.task.internal:Not uploading archives. 2024-05-22T21:26:01.037 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-22T21:26:01.047 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-22T21:26:01.047 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-22T21:26:01.050 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-22T21:26:01.055 INFO:teuthology.orchestra.run.smithi161.stdout: 658122 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 22 21:26 /home/ubuntu/cephtest 2024-05-22T21:26:01.077 INFO:teuthology.orchestra.run.smithi179.stdout: 655428 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 22 21:26 /home/ubuntu/cephtest 2024-05-22T21:26:01.078 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-22T21:26:01.088 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-22T21:26:01.113 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 4fed890e8ce3966dbe380e9eb4b0a796eddb841c client.1-kernel-sha1: 4fed890e8ce3966dbe380e9eb4b0a796eddb841c description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} duration: 1430.1072347164154 failure_reason: '"2024-05-22T21:13:09.980360+0000 osd.2 (osd.2) 3 : cluster [WRN] OSD bench result of 135369.788297 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.2. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd''s IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log' flavor: default owner: scheduled_leonidus@teuthology success: false 2024-05-22T21:26:01.113 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-22T21:26:01.196 INFO:teuthology.run:FAIL